the australasian journal of information systems (ajis) began in 1993 and was based on the simple realisation that there was a need for an academic ‘outlet’ for is research in australia australasian journal of information systems volume 14 number 1 nov 2006 ajis featured theme: the information systems discipline in australian universities editor: professor guy gable it professional services research program faculty of information technology queensland university of technology brisbane, queensland, australia email: g.gable@qut.edu.au this ajis featured theme comprises 12 research papers that present a portrait of the information systems academic discipline in australian universities. this australian study is part of a broader study 'the state of the information systems academic discipline in pacific asia’, the results of which are forthcoming in a similarly titled special issue of communications of the ais. drawing on whitley’s theory of scientific change, the papers analyse the degree of ‘professionalisation’ of the information systems discipline, the overarching research question being “to what extent is information systems a distinct and mature discipline in australia?” all papers in the special edition have been subject to a three-tier peer review process: each paper has been initially reviewed by the ajis editors, feedback has been provided to the authors and revisions made. then, each paper has been sent to a local (state-based) is expert for review; the results of this review being conveyed anonymously to the author of the paper, and consequential revisions made by the authors. finally, all papers considered for the special edition were sent to two further is experts (international) for global review, with feedback from the reviewers again being forwarded to authors for revision of their papers. professor gable chairs the it professional services research program, faculty of information technology, queensland university of technology http://sky.fit.qut.edu.au/~gable/. his phd is from university of bradford and mba from ivey business school. he is senior editor journal of strategic information systems, associate editor misq, and on the editorial boards of journal of the ais, information systems frontiers, and australasian journal of information systems. key research interests include it professional services, it research methods, enterprise systems, and it evaluation. career emphasis is on research collaboration with industry and the professions chief investigator ($3.0m+) on grants with accenture, sap, cusc, institute of management consultants, and queensland treasury. he has published >100 refereed journal articles, conference papers and books. 101 http://sky.fit.qut.edu.au/%7egable/ australasian journal of information systems volume 14 number 1 nov 2006 102 ajis featured theme: the information systems discipline in australian universities ajis vol 9 no. 1 september 2001 editorial this is the first issue of volume 9 of the australian journal of information systems. as editor i am pleased to report the continuing interest in the journal both in australia as well as overseas. our first paper scalable fault-tolerant location management scheme for mobile ep is presented by jinho ann, sung-gi min and chongsun hwang from the department of computer science and engineering, korea university. this paper presents a new fault tolerant scheme using checkpointing and message logging which achieves low failure free latency even when the number of mobility agents increases. the next paper is presented by michael busing, department of information systems and operations management, james madison university and is entitled inter firm information sharing in enterprise resource planning systems: a call for timely but limited access to customer information. this paper presents a previously tested priority-sequencing rule that explicitly considers downstream shop conditions in determining which job to run next. julie fisher, school of information, management and systems, monash university presents or next paper entitled user satisfaction and system success: considering the development team. this paper explores the skills, other than technical skills, that are needed on development teams to produce systems that are successful from a user's perspective. our next paper is entitled ontological analysis of integrated process models: testing hypotheses is presented by peter green and michael rosemann, department of commerce, university of queensland. this paper demonstrates the usefulness of the bunge-wand-weber (bww) representation model for evaluating modelling grammars such as those constituting aris. jocelyn handy, school of psychology, massey university, richard whiddett and inga hunter, department of information systems, massey university present our next paper a technology acceptance model for interorganisational electronic medical records systems. this paper reports the findings of the first stage of an ongoing, longitudinal study into the implementation of an inter-organisational electronic medical records (emr) system. khaled khan, school of computing and information technology, university of western sydney, bruce lo, school of information technology and multi-media, southern cross university and torbjorn skramstad, department of informatics and computing science, norwegian university of sciences and technology present our next paper entitled tasks and methods of software maintenance: a process oriented framework. this paper proposes a framework of the tasks involved in the software maintenance process. our next paper a balanced approach to capturing user requirements in business-to-consumer web information systems is presented by m lane and akoronios, department of information systems, university of southern queensland. this paper presents a balanced approach to user requirements capture which will result in more complete and user centred requirements. nena lim, department of commerce, university of queensland presents our next paper entitled customers' beliefs behind business-to consumer electronic commerce. this paper extends the technology acceptance model (tam) to test the influence of experience, self-efficacy, perceived risk, and social influence on customers' acceptance of the business-to-consumer (b2c) e-commerce. peter o'donnell, decision support laboratory, monash university presents our next paper the use of influence diagrams in the design of spreadsheet models: an experimental study. this paper presents results that suggest that influence diagrams do help users to better understand problems, and as a result reduce the likelihood of omitting important variables from spreadsheet models. our next paper is entitled designing for web site usability and is presented by chris proctor and judith symonds school of information science and humanities, the open polytechnic of new zealand. this paper explores the customer perspective of world wide web (www) site design in the light of current www usability research. our final paper is presented by mark stansfield, department of computing and information systems, university of paisley and is entitled problems and issues in using computer-based support tools to enhance 'soft' systems methodologies. this paper explores the issue of whether computer-based support tools can enhance the use of'soft' systems methodologies as applied to real-world problem situations. once again i would like to pass on my thanks to the authors and consulting editors for the high quality of material in this issue. i would like to thank lily soh and diniz da rocha for their continued assistance with the journal and the maintenance of the home page. rob macgregor editor http://www.uow.edu.au/ajis/ajis.html microsoft word ajis vol13-2 o.doc 201 australasian journal of information systems volume 13 number 2 may 2006 striking a balance between ethics and ict governance graeme pye & matthew j. warren school of information systems, faculty of business and law, deakin university, geelong, victoria, australia, 3217 email: mwarren@deakin.edu.au abstract ethics and information communication technology (ict) governance both have their place in today’s business organisations, but can their practical applications present an ethical ambiguity for the it professional employed within the business organisation? the guidelines contained within various codes of ethics recommend principles regarding the ethical behaviour of individual it professionals. in contrast, it governance as outlined in the new australian standard for corporate governance of information and communication technology (ict) provides ict governance advice for business. this paper explores the difference between these viewpoints. keywords: ethics, it governance, ict and acs. introduction depending on your personal perspective, ethics can have a number of relevant meanings. in general terms ethics is regarded as the moral rationales that influence a person’s behaviour or the carrying out of an activity or alternatively, ethics can also refer to the area of knowledge that deals with moral principles (pearsall, 1998). however, from an information technology (it) business domain perspective, clarke’s (1999) view was that the term ethics is intended to refer to the guiding principles of doing what is right or wrong from a moral perspective, in reference to ethical behaviour of both the individual it professional and the governance of an it department within a business organisation. as an overview, this research seeks to investigate and philosophically appreciate the ethical perceptions, interpretations, principles and professed tenets of the australian computer society’s (acs) code of ethics (2003), while also investigating the genesis and potential influence of it governance in light of the recent publication of the australian standard for the recommended guiding principles of corporate governance of information and communication technology as 8015-2005 (2005). as a result of this circumstance, the it professional is now potentially faced with two behavioural conventions in the workplace that can both exert their own normative influences on the employees’ ethical behaviour. so initially this paper will explain and philosophically define ethics and morality, before proceeding to examine and compare the particular focus of the key principles underpinning the asc code of ethics (2005) and the australian standard for ict governance (2005). australasian journal of information systems volume 13 number 2 may 2006 202 philosophy of ethics and morialty according to the oxford dictionary of philosophy, ethics is ‘the study of the concepts involved in practical reasoning: good, right, duty, obligation, virtue, freedom, rationality, choice’ (blackburn p.126 1994) while, applied ethics is ‘the subject that applies ethics to actual practical problems …’ (blackburn p.126 1994). furthermore, the ethics and morality of an individual or people can be regarded as the same thing according to blackburn (1994), however a usage of morality by german philosopher kant (1724 – 1804) restricts the usage of morality to ideas of duty, obligation and principles of conduct, while reserving ethics for the aristotle (384 – 322 bc) approach of practical reasoning pertaining to the ideas of virtue and generally avoiding the separation of moral considerations from other practical considerations. these philosophical rationales will form the basis of a philosophical analysis from an ethical and moral perspective of the information content of the acs code of ethics (2003) and the ict governance standard (2005) respectively, to gain a deeper understanding of their philosophical basis, considered roles and application within the it domain of any business organisation. code of ethics many computing codes of ethics abound and are generally based on a number of perceived ethical or moral principles that provide guidance and engender a commitment toward ethical behaviour that is appropriate for and expected of it professionals. for example, the australian computer society code of ethics (acs 2003) is a code of conduct that applies only to their members. generally, such codes are used as guiding principles for professional computing industry associations as a resource for their members and as a means of outlining the ethical expectations of both the association’s membership and individual it professionals in general. acs code of ethics the acs code of ethics (2003) document delivers ethically based behavioural recommendations that are strongly focused on delivering sound ethical and moral advice to individual it professionals that are members of the acs, while also providing a reference resource that addresses the following ten principles: 1. honour and dignity; 2. personal commitment; 3. values and ideals; 4. standards of conduct; 5. priorities; 6. competence; 7. honesty; 8. social implications; 9. professional development; 10. it professional behaviour. the acs code of ethics (2003) seeks to deliver advice that is ethically right in relation to the appropriate ethical behaviour expected of an acs member or for any it professional that would be reasonably expected to deliver in their professional work. although the code addresses a wide area of principles, the advice only serves as guidance to the it professional from a personal behaviour aspect and does not seek to deliver a methodology for resolving ethical dilemmas between 203 australasian journal of information systems volume 13 number 2 may 2006 individuals or an individual and the business practice or goal. this is not to say that the individual could not practically reason for themselves what is ethically right or the morally appropriate behaviour to follow with due consideration of the situation, it is just that no specific guidance is given to the reader of how such an ethical issue could be resolved. from a philosophical standpoint the acs code of ethics (2003) uses an applied ethics approach to addressing the ten principles listed, and what the code is trying to achieve is that in regard to the code’s stated principles: these are the suggested ethical behaviours and obligations that can be practically applied by an individual, in order to be regarded or deemed as acting ethically in respect to the acs code of ethics (2003). it is therefore the view of the authors that from an ethical and moral perspective of the difference between right and wrong that the acs codes of ethics (2003) can guide it professional to ‘do the right thing’ and this also suggests that such adherence to the moral and ethical principles as laid out in the code would serve the it professional well from human-centric, behavioural and interactive perspectives. ict governance according to the oecd (2004), the principles of corporate governance should reflect the set of relationships between an organisation’s management, board, shareholders and other stakeholders to constitute the rights, roles and equitable treatment of shareholders; disclosure and transparency; and the responsibilities of the board. additionally, corporate governance should be based on sound strategic guidance of the business, effective monitoring of management by the board and accountability of the board to stakeholders. the australian standard for good governance principles as 8000-2003 (2003) reflects the oecd intent and builds further upon other principles covering authority, accountability, stewardship, leadership, direction and how control is exercised within an organisation. in taking governance one step further, the new australian standard for corporate governance of ict as 8015-2005 (2005) is a supporting standard that applies to the governance of it resources and associated technologies used to provide information and communication services within a business organisation. the ict governance standard provides guidance that allows organisational directors and managers to dictate how ict assets will be managed to ensure that they support the business goals set by the business owners, management or board of directors that are applicable to the principles of good ict governance. advantages of ict governance as it technology has become an ingrained and essential part of the enterprise management of transactions, information storage and knowledge management of enterprises, their subsequent dependence on technology has become fundamental to supporting, sustaining and growth of business enterprises. however, the risks associated with ict become more apparent when considered in the context of doing business on a global scale, twenty-four hours a day, seven days a week, with the reliance placed in ict resources to provide a competitive edge this can determine the very survivability of the business and ongoing prosperity (itgi 2003). the research of weill and ross (2004) has defined ten key principles essential to effective it governance that are as follows: australasian journal of information systems volume 13 number 2 may 2006 204 1. actively design governance and continue to provide adequate resources, support and attention; 2. know when to redesign and adjust the governance systems; 3. involve senior managers in committees, decisions and performance reviews; 4. make choices that are business strategic and manageable; 5. clarify the exception handling process to deal with the unexpected; 6. provide the right incentives that rewards alignment to the strategic business goals; 7. assign ownership and accountability for it governance to ‘champion’ the process; 8. design governance at multiple organisational levels; 9. provide transparency and eduction; 10. implement common mechanisms across the six key assets: • customer relationships; • product assets; • human assets; • it assets; • physical assets; • financial assets; these ten principles represent the key outcomes of weill and ross’s (2004) research and strongly supports the case for it governance as this research also affirms that businesses with effective it governance programs in place, have attained twenty percent higher profit margins than those businesses with poor quality governance programs that have similar strategic goals. therefore in the australian context, the australian standard for corporate governance of ict as 8015-2005 (2005) is focused on promoting effective, efficient and the acceptable use of ict by being ‘the system by which the current and future use of ict is directed and controlled. it involves evaluating and directing plans for the use of ict to support the organisation and monitoring this use to achieve plans. it includes the strategy and policies for using ict within an organisation.’ (standards australia p.6 2005). ethical focus of ict governance standard the ict governance standard (2005) document outlines six principles that establish well defined responsibilities for effective ict governance of australian businesses (standards australia 2005): 1. establish clearly understood responsibilities for ict; 2. plan ict to best support the organisation; 3. acquire ict validly; 4. ensure the ict performs well whenever required; 5. ensure ict conforms with formal rules; 6. ensure ict use respects human factors. the ict governance standard (2005) does deliver a framework for business that suggests how to managerially control it from a top-down perspective by engendering a view of compliance for good governance of ict. however, the standard does not deliver any framework or methodology to resolve ict governance issues within the business and therefore such decisions are deferred to the ethics of the employee to resolve within the business context, as principle five directs that “directors should direct that all actions relating to ict be ethical” (australian standard p.12 2005). from a philosophical prospective the ict governance standard (2005) is based in the traditional ethics approach by focusing on the ideas behind what is deemed good governance from the practical 205 australasian journal of information systems volume 13 number 2 may 2006 reasoning aspect of what is right, dutiful and obligatory across the governing tasks of evaluating, directing, and monitoring as recommended in the standard. ict governance and the acs code of ethics the australian standard for the governance of ict is ethically vague at the personal level and has been developed with a corporate aspect that prescribes what is deemed as necessary corporate behaviour to achieve good governance of ict. according to the standard, this can be achieved within a business organisation and with top-down management principles that originate at the board of directors level and therefore the ict governance standard (2005) applies a high-level view of compliance through the establishment of checks and balances in evaluating, directing and monitoring of the ict assets within the business, while providing no moral or ethical guidance to the individual it professional within the business. we can conclude that governance of ict as applied can deliver good governance for the management of ict, but this does not necessarily reflect the ethical standards required by individuals within the business or at the director level. therefore we cannot assume that good governance of ict will necessarily imply the presence of ethically virtuous employees doing what is right within the business. the acs code of ethics (2003) takes a human-centred focus to providing ethical guidance to it professionals that is didactic in nature and is achieved by explicitly directing the reader on what is deemed as acceptable ethical behaviour, however it is also pays little regard to the management issues that are the focus of the ict governance standard (2005). the documental ethical voicings melsar and byrne-armstrong’s (2000) research identified corporate voices and personal voices as being the two distinct discourses that were competing for dominance of the internet, with each identifying different ethical issues as being the most important to the internet. melsar and byrnearmstrong go on to establish that the two discourses exhibit differing values and visions of the internet, such as: the corporate vision as being an extension of property and income generation; while the personal voice is regarded as an extension of creativity and connection between humans. we can identify a similar use of corporate and personal voicing being employed in the acs code of ethics (2003) and the ict governance standard (2005) documents. this is evident in the acs code of ethics (2003) where the document voicing using the first person presents the reader with the more humanistic tone of personal voicing that is focused at addressing the individual. while conversely, the ict governance standard (2005) uses an instructive and impersonal tone in the third person that presents the language of corporate voicing, which also presents a formal and measured tone in regard to document content for guiding professional corporate behaviour. the particular voicing used in these documents is intended to address a particular target audience and therefore the primary focus of the ict governance standard (2005) is that as an officially sanctioned australian standard, it is focused on delivering sound practical advice that can assist business in managing in a top-down process, the control and management of it assets and personal within their respective business organisation. conversely, the acs code of ethics (2003) delivers ethical suggestions that are directly focused on empowering the individual with ethical advice to enable an it professional to decide upon the appropriate professional behaviour in the particular australasian journal of information systems volume 13 number 2 may 2006 206 ethical situation. the acs code of ethics (2003), from this prospective, promotes a bottom-up style of management that encourages the it professional to drive ethical behaviour for themselves, their colleagues and perhaps ultimately the ethical behaviour of the business organisation. this difference of documental approach to managing individual ethics from the bottom-up and the ict governance of business from the top-down, may potentially create an ethical dilemma in itself between the adopted code of ethics of the individual it professional and the ict governance requirements of the business. therefore further investigation is required into the resultant consequences of ethical decisions taken and if contradiction exists, to bring together individual ethical beliefs based on the acs code of ethics (2003) and also the obligations of the ict governance standard (2005), to develop a resolution framework to determine a conciliatory and ethical solution that represents the best intentions of both the individual and the business. conclusion we have determined that there is a marked philosophically ethical difference between the acs code of ethics (2003) and the ict governance standard (2005) that is due in part to the localised focus taken by each of the respective documents. they exist with little or no regard given to the potential influence of each, their effect upon the business or individual in regard to their application, the potential risks for creating conflictive ethical dilemmas or any in-depth consideration given to determining ethically balanced decisions in regard to both their respective applications. while the acs code of ethics (2003) takes a very humanistic and personal view in delivering ethical advice that is based upon the philosophy of applied ethics by employing ethical solutions practically. conversely it was determined that the ict governance standard (2005) takes an impersonal view that is focused on the recommended business management aspects of evaluating, directing, and monitoring business processes with a more traditional ethical view of complying to what may be deemed as good governance. the nature of the philosophically ethical difference between the code of ethics and ict governance and the real potential for ethical dilemmas to arise, further research is needed to recognise, address and determine such ethical resolutions by better understanding the consequential effects of such decisions made. to do this from an ethical and moral perspective we need to understand the moral theory behind the consequences of ethical decisions and solutions, before developing a framework to guide ethical resolution of issues between the acs code of ethics (2003) and the ict governance standard (2005). once this ethical framework is established, the next step in this research is to test the framework against real-world or hypothetical case studies to determine the effectiveness of the framework in addressing and resolving ethical issues between individuals and businesses through the application of applying the philosophical principles of consequentialism. 207 australasian journal of information systems volume 13 number 2 may 2006 references acs 2003, australian computer society code of ethics, (online), australian computer society, available from: <http://www.acs.org.au/about_acs/acs131.htm> (may 2005). blackburn s. 1994, the oxford dictionary of philosophy, oxford university press, oxford u.k. clarke r. 1999, ethics and the internet: cyberspace behaviour of people, communities and organisations., (online), anu, available from: <http://www.anu.edu.au/people/roger.clarke/ii/iethics99.html> (june 2005). itgi 2003, board briefing on it governance, 2nd ed., it governance institute. melser p. byrne-armstrong h. 2000, 'corporate voices, personal voices: the ethics of the internet.' in 2nd australian institute of computer ethics conference (aice2000), australian computer society, canberra, act. oecd 2004, oecd principles of corporate governance, oecd, paris, france. pearsall j. (ed) 1998, the new oxford dictionary of english, clarendon press, oxford. standards australia 2003, good governance principles. as 8000-2003, standards australia, sydney, nsw. standards australia 2005, corporate governance of information & communication technology. as 8015-2005, standards australia, sydney, nsw. weill r., ross j. w. 2004, it governance, harvard business school press, boston, massachusetts. ajis vol. 8 no. 1 september 2000 is a picture worth a thousand words? the use of non-text items en information systems research. alison parkes department of information systems massey university private bag 11 222 palmerston north new zealand a.j.parkes@massey.ac.nz abstract the use of non-text items (graphs, graphics and tables) is firmly established in the publishing tradition of information systems research. non-text images are used to convey complex information in a pictorial or diagrammatic sense, adding additional meaning to the discussion. non-text images also highlight results, and meaningfully represent both quantitative and qualitative data. this paper examines the use of non-text items in information systems journals. three measures of use are employed: rate of use of non-text items; academic and practitioner journal usage of non-text items; and citation rates of articles with varying levels of non-text items. the sample research articles represent a 30year timeframe and include articles from six prominent information systems journals. the research results show an increasing use of non-text items in information systems research. in addition, articles that utilise a higher rate of non-text items were more likely to be cited in other works. practitioner journals were found to utilise more non-text items than academic journals, and to utilise a different mixture of non-text items. it would appear that articles that use higher levels of non-text items are more widely cited, so if citedness is the definition of 'success', then it may be practical to incorporate a range of non-text items. if consideration is given to the motive behind selection of articles for citation when preparing research, it seems possible that increased use of non-text items improves the clarity of research findings, leading to their increased usage, uptake and citation levels. introduction information accumulated during research endeavours can be presented in a variety of ways. the most common methods are via text, (or narrative), or by non-text items depicting the information. there is minimal published research available in this area however a summary of some findings to date follows. an examination of how user perceptions and decision making varied under three different methods of information presentation (tabular, graphical and combined tabular graphical) found that when time limits were low, tabular reports produced better decisions, whereas graphic reports produced quicker decisions. combined tabular and graphic representations produced both faster and better decision making, and was found to be the preferred report format for most decision makers. benbasat & dexter. (1986). readers tend to assign higher levels of trust in data presented in a scientific manner. it was found that using numerical tables tended to make readers assume the existence of scientific rigor. davis (1981). information presented in a combination of text and non-text styles is more likely to be comprehended by the recipient of the information presentation. hamilton & ives (1983). the increasing use of graphics in technical writing may be related to the emergence of technology that facilitates usage of graphics in both oral and written information presentations. owen & van halm (1989). advances in publishing technology are likely to have facilitated increased usage of non-text items, however slow uptake of new technology may act as an obstacle to innovation in the publishing industry. mcgill (1990). research on the impact of various combinations of format and summarisation levels on different personality types defined by the myers-briggs typing instrument found that where the user personality type is unknown, raw data is the best offering, whereas if the personality type is known, there is a preferred option for each cognitive style. it is interesting to note that perceptions of value of information will vary between different personality types. moran & journet (1985). research question the purpose of this research is to determine whether the use of non-text items is increasing in published research, whether the use of non-text items affects the level of citedness of articles, and if there are 64 vo18 n»-1 september 2000 differing levels of usage of non-text items between practitioner and academic journals three hypothesis have been formulated. hi that the rate of occurrence of non-textual information in published information systems research is increasing over time. because of increasing awareness of the value of non-text items, and the improved ability of publishers to present non-text items easily, it was theorised that research would increasingly use non-text representations of data h2 that practitioner journals would exhibit higher frequency of non-text occurrences than academic journals. there are two main types of journals where information systems research is published, categorised as academic and practitioner journals. practitioner journals tend to be 'glossy', with a magazine style format. it was thought that due to the form of the journal, and the target audience, practitioner journals would use more non-text representations than academic journals. h3 that those articles that contain a higher number of occurrences of non-textual data are more often cited. the existence of higher numbers of non-text items in an article increases readability and comprehension of the content. it is thought that successful information systems research would use higher levels of nontext information. those articles that contain a high number of non-text representations may be more widely used, and, if so, would be cited more frequently. to test these hypotheses, a representative sample of journals was selected for examination. as an aid to determining a representative sample, a typology of journals was used to guide the selection. vogel & wetherbe (1984). a representative sample of articles was examined and occurrences of non-textual data recorded. definition of non-text items for the purposes of this research the following definitions apply: table: information displayed in matrix format, which is labelled or identified as a table within the article e.g. ihrr 1-plidp.lloa. cq.hjci. ud coodkt koolallco table i nin* sialiain t* sthtly 1'iirwnri roporaibuiiy llanfeflfi artivilicl uot-4s rebuoninip influence — hem 1 influence — item 2 influence — item 3 conflict— item 1 cnnoki— item 2 connie!— iiem j sililfacliwy conflict resolution — item 1 siumunry fi«l«l knolulinn— hem 2 man 0.29 042 oji 4.11 497 4.49 1.5! 3.46 367 4.11 4.70 std. dev. 0)1 0.32 0.30 2.20 1.94 2.06 1.76 2.11 19) 112 1.79 figure j. example of table 65 ajis vol. 8 no. 1 september 2000 1st 2nd 3rd 4th qtr qtr qtr qtr figure 2. example of graph graph: a pictorial representation of quantitative data made in a non-numeric illustration, for example a bar chart or pie graph, e.g. graphica picture or model illustration e.g. 1. model of otpntnlioul poulka perceptual. figure 3. example of graphic methods selection of journals journals were stratified into four levels of importance, and those levels divided into practitioner and academic orientations. two journals were selected from each orientation for level 1, and one journal was selected from each orientation for level 2 of the taxonomy; a total of six journals. levels 3 and 4 of the taxonomy were not considered as the information provided was too generalised to aid in journal selection. in order to select the most appropriate journals to utilise within each classification, the citation frequency was utilised. the citation frequency was combined with the corresponding taxonomy of journal importance. journals that were identified as attaining a higher number of citations within each level, for each orientation, were selected for this research. the rationale behind the utilisation of the citation frequency as the selector is that testing h3 will involve the use of a citation index. sample articles are more likely to appear in this index if high citation frequency has been advised for their publishing journal. the most important and frequently cited journals from levels 1 and 2, for both academic and practitioner orientation were as follows: 66 ajis vol. 8 no. 1 september 2000 level 1 2 orientation academic practitioner academic practitioner selected journal management science mis quarterly datamation harvard business review communications of the acm communications of the acm table 1. selected journals as the same journal was selected for both academic and practitioner orientation for level 2, it was decided to examine the second most frequently cited journal for those levels, the journal of systems management. to determine which journal was most representative of academic orientation, an analysis of submission preferences was conducted. this showed a higher academic preference for submission to communications of the acm. the journals selected for the final sample were: level 1 2 orientation academic practitioner academic practitioner selected journal management science mis quarterly datamation harvard business review communications of the acm journal of systems management table 2. final journal selection selection of articles in order to test h1: that the rate of occurrence of non-textual information in published information systems research is increasing over time, a longitudinal sample of journal articles was required. it was decided to sample from the years 1991, 1993, 1995 & 1997. the first issue for each year of the selected journals was selected. every article in the selected issues was examined. this resulted in a sample of 201 articles; containing 916 identified non-textual information representations. additional articles from 1971 and 1981, selected in the same manner, were added during the research process, to broaden the longitudinal scope of the research. this resulted in a total sample size of 271 articles containing 1148 non-text images. selection of citation index there are two possible information sources relating to citation frequency, the social sciences citation index and the science citation index. to determine the most appropriate index to use, an examination was conducted of the source listing for each index, with the results shown in table 3: 67 ajis vol. 8 no. 1 september 2000 v • journal management science datamation harvard business review mis quarterly communications of the acm journal of systems management social science citation index source »/ x •/ </ «/ x science citation index source x x x x • x table 3. citation index sources it was established that the social science citation index was the most appropriate source to use for checking citation frequency of the selected journal articles. selection of articles for citation checking. articles for citation checking were selected from journals listed in the sources of the social science citation index; viz, management science, harvard business review, mis quarterly and communications of the acm. the articles in these magazines were grouped into low, medium and high usage level groups. low usage was defined as 0 3 occurrences, medium as 4 8 occurrences and high as 9 or more occurrences of non-text items in an individual article. sample articles were randomly selected from journals in the year 1995. four articles were selected for each usage level, a total of 12 articles. results analysis of data relating to hi: that the rate of occurrence of non-textual information in published information systems research is increasing over time the initial sample, when analysed by calculating the mean of occurrences, resulted in no clearly identifiable trend relating to the increasing use of non-textual items as shown in figure 4. year 1991 b1993 d1995 ["11997] figure 4. initial sample mean of non-text item occurrence 68 ajis vol. 8 no. 1 september 2000 to determine if longitudinal magnitude was an issue, a further set of observations was made. utilising the previously identified journals, a sample was taken of articles from the first edition for each of the years 1971, and 1981. these results were added to material collected for 1991. following the amendment of the sample to constitute 1971, 1981 and 1991, the number of articles examined was as shown in table 4: orientation academic practitioner selected journal management science mis quarterly communications of the acm datamation harvard business review journal of systems management total number of articles 27 9 18 17 35 12 118 table 4. summary of articles selected the mean for the first year of each decade was graphed, see figure 5 for the results: year 1971 11981 d1991 figure 5. final sample mean of non-text occurrences by decade the results showed an increasing rate of usage of non-text items. therefore hi: that the rate of occurrence of non-textual information in published information systems research is increasing over time; was supported. analysis of data relating to h2: that practitioner articles would exhibit higher frequency of nontext occurrences than academic journals. in order to test h2: that practitioner journals would exhibit higher frequency of non-text occurrences than academic journals; a comparison was made between the mean of occurrences of non-text items in academic journals and those in practitioner journals. given a length disparity between academic and practitioner articles (practitioner article average length three pages; academic article average length ten pages) the frequency of usage of non-text items relative to the length of the published article was assessed with results shown in figure 6. 69 ajis vol. 8 no. 1 september 2000 journal type i practitioner • academic figure 6. mean of occurrences of non-text information, practitioner and academic journals. these results support the suggestion that practitioner-oriented journals exhibit higher frequency of nontext occurrences than academic journals. the outcome of the analysis was that practitioner journals in fact use nearly twice as many non-text items as academic journals. an examination of the usage levels of different types of non-text items used in the publications is depicted in figure 7: 100% 50% 0% practitioner academic d tables • graphs d graphics figure 7. comparison by percentage of types of non-text items used these results suggest that articles published in practitioner journals relied heavily on the use of graphics to support text, whereas articles published in academic journals used tabular presentation of material more frequently. analysis of data relating to h3: that those items that contain a higher number of occurrences on non-text data are more often cited. the articles selected for citation checking were verified against the social science citation index for the years 1995, 1996, & 1997, and for the first quarter of 1998. the results for the identified low, medium and high usage of non-text item usage were as follows in figure 8: 70 ajis vol. 8 no. 1 september 2000 o 0 usage level i low •medium dhigh figure 8. mean of citation frequency these results support h3: that those articles that contain a higher number of occurrences of non-textual data are more often cited. conclusion the results of the research show that information systems research is using increasing numbers of nontext items in published research. the results also identified that practitioner journals utilise more nontext items, relative to length, than academic journals; and that articles with a higher usage rate of nontext items are more likely to be cited. these results were accomplished by selecting a sample of articles and analysing the frequency of nontext item occurrences. all analysis was based on comparison of the means of occurrence of non-text items for the selected articles. a sub-sample of articles was checked for citation frequency to determine if a relationship existed between use of non-text items and levels of citation. discussion the research conducted used a large sample size, over a wide time span, and therefore the sample is likely to be representative of the population. more sophisticated statistical techniques would have been able to establish relevance and generalisability of the sample, however the resources to conduct this testing were not available. the sample selected for citation checking could have been broadened, with a larger sample over a wider time period possibly adding weight to the conclusion reached. the implications of this research revolve around the norms relating to the use of non-text items. it would appear that articles that use higher levels of non-text items are more widely cited, so if citedness is the definition of 'success', then it may be prudent to utilise a wider range of non-text items. the distribution of types of non-text items varied considerably between types of journals. over 50% of the items observed in practitioner journals were graphics, whilst the major component of non-text items in academic journals was tables. the data derived during this research also contains information relating to individual journals preference for usage level and type of non-text items used. a taxonomy of journal preferences could be developed during future research; to increase the probability of publication in an author's preferred journal. similar research by vogel and wetherbe (1984) examined journals to determine a taxonomy of preferred research methodologies, and universities to determine preferred journal for publication. as an example of the differences observed between journals figure 9 shows results obtained when comparisons were drawn between types of non-text items used. 71 ajis vol. 8 no. 1 september 2000 100% 50% 0% mis mgmnt comms datamation her jnl of sys quarterly science acm mgment 03 tables • graphs d graphics figure 9:usage levels for different types of non-text items an increasing use of colour was observed over time for the journals involved. 1970's non-text items tended to be black and white, and of relatively poor quality, whereas the 1980's and onwards showed a distinct improvement in the quality of non-text items used. printing and editing technology has improved over the time of this analysis; increased complexity and colour was observed in articles that are more recent. the type of graphics used has also varied considerably. 1970's magazines often used text items that were obviously 'cut and pasted' into the articles, apparent by the difference in font and general appearance to the flow of narrative. later articles tended to use non-text items that blend with the narrative text, using similar font and colours to create a cohesive effect. articles in practitioner journals tended to be much shorter in length, on average 3 pages as opposed to an average 10 pages in academic journals. additionally, practitioner journals contained articles that were more easily comprehended; however, conclusions were not necessarily supported by evidence. additionally the referencing in practitioner articles tended to be sparse compared to the voluminous references included in academic articles. if consideration is given to the motive behind selection of articles for citation when preparing research, there is a possibility that increased use of non-text items improves the clarity of research findings, leading to their increased usage, uptake and citation levels. to summarise, consideration of a journal's preferred usage of non-text items might assist when submitting articles for publication. use of a higher number of non-text items appears to lead to higher frequency of citation, and therefore increases the potential for recognition of the results of research endeavours. references benbasat i & dexter, a.s. (1986) "an investigation of the effectiveness of colour and graphical information presentation under varying time constraints," mis quarterly, vol. 10, pp. 59-83. davis, d. (1981) "an experimental investigation of the form of information presentation, psychological type of the user, and performance within the context." in college of business. florida: university of florida. hamilton s & ives, b. (1983) "the journal communication system for mis research," data base, vol. winter, pp. 3 14. owen, j mackenzie & van halm, j. (1989) innovation in the information chain, 1st ed. london: routledge. mcgill, l.t. (1990) "doing science by the numbers," in the rhetoric of social research: understood and believed, a. hunter, ed. new brunswick: rutgers u.p. moran, m. & journet, d. (1985) "research in technical communication," 1st ed. connecticut: greenwood press. vogel, d.r. & wetherbe, j.c. (1984) "mis research: a profile of leading journals and universities," data base, vol. fall, pp. 3 -14. 72 microsoft word special final.doc ajis special issue december 2004 116 3g multimedia content production as social communication christine satchell associate professor supriya singh rmit university and smart internet technology crc christine.satchell@rmit.edu.au supriya.singh@rmit.edu.au associate professor john zic university of new south wales and smart internet technology crc johnz@cse.unsw.edu.au abstract young people are taking advantage of the ever-increasing accessibility and technical capabilities of 3g phones and the internet to represent their experiences through multimedia content. however, this practice is inhibited by design shortcomings which don’t adequately protect the privacy of content, problemitise content management and which limit distribution. this paper explores how possible design solutions were envisioned thorough the use of a scenario, called the trophy room. keywords multimedia content production, privacy, archiving, distribution, 3g phones. introduction a central theme in recent studies by taylor and harper (2002) and howard et al (2002) is that the importance young users place on expressing their social identity in the real world extends into the digital world. this was supported in the two studies at the centre of this paper which were carried out as part of the smart internet technology crc research projects: the young people study and the secure identity management 6 project. the studies found that young people are taking advantage of 3g telephone systems and the internet to produce and merge bits of data to create their ‘ideal digital self’ through which they communicate socially. this means that instead of voice or text, users are communicating through a hybrid of still and moving images, sound-bytes, symbols and logos. this is in keeping with counts and fellheimer (2004) who found that “lightweight photo sharing, particularly via mobile devices, is fast becoming a common communication medium used for maintaining a presence in the lives of friends and family”. methodology the two studies looked at 35 technically savvy users aged 18-30, living in melbourne, australia. central to the research is the attempt to understand how young people express their social identity using new digital technologies. the search for emerging themes, user needs and design implications is at the center of the research. during the initial interviews the ‘open-ended interview method’ (minichello: 1995) was used. the data was analyzed using nud*ist 4, a computer program for the analysis of qualitative data. the following themes and user needs emerged from the analysis. each user needs maps onto a theme. 6 the authors would like to thank telstra for supporting this project ajis special issue december 2004 117 emerging themes & user needs overall theme – the importance of digital identity: users in the studies indicated that interpreting or reading other peoples’ virtual identities provided a means of identifying what the other person was like. as one user stated, “their synthesized persona is quite revealing.” furthermore, users indicated that they derive pleasure from creating a digital identity that reveals the persona they wish to convey. general user need – identity as disclosure: this is significant from a user needs perspective because it indicates that creating a digital identity is not only about how much information can be restricted but rather what is revealed. this illustrates that identity should not just be thought of in terms of how anonymity can be enforced. emerging theme 1 – ‘cut and paste’ multi media content as social communication. in order to create more dynamic digital identities users reported that they are using a ‘cut and paste’ approach to communicate through content. “i take digital photos and videos with my phone. then i email them to my computer, download songs, attach it all to an email and send it to my friends.” while another user stated “i can listen to my voice mail messages on the internet. then i can save them and use them as samples. then i can email them back to friends and include pictures of them, or bits of their favourite song or something as birthday present.” this is reminiscent of the behaviour of text messaging that taylor and harper (2002) reported, where a text message was not so much about what it said, but as an expression of friendship. user need 1 – maintaining control of the privacy of content: while users enjoyed using multimedia content as a form of social communication, 17 users in the studies explicitly stated that it is content that they have produced themselves that is the most important to protect in terms of privacy. one user in the studies said she is more concerned with protecting the privacy of the digital pictures on her phone than she is with protecting her internet banking details. other users indicated that a major concern with sharing content is making sure that only the intended people see it. these privacy concerns also explain why users said that while they wanted to communicate through home produced content, they were often reluctant to just send a picture or video out because once you do that you loose control over its distribution. emerging theme 2 – wrapping the message in content (digital tokens): other examples of the ways in which users in the studies were using content as a form of social communication were embedding a picture in the bottom of every email instead of a signature. furthermore, users in the studies indicate a desire for this paradigm to be turned around – so that the dominant part of the message is the picture itself embedded with the social message. “what i like about my new (3g) phone is that i can send a picture that i have taken and that’s the message. i can add text too if i want.” essentially, the content becomes the ‘wrapper’ for the message. it can be seen how for the users in the studies, digital communication can now be more dynamic, so instead of traditional voice or text a message can be a combination of multimedia data embed in a single message or ‘token’. however, with this increase in the use of multimedia content comes the difficulty in managing all the bits of data. ajis special issue december 2004 118 user need 2 – improved content management: users reported that they are mixing more and more bits of multimedia content together in a single message, reminiscent of the way a dj mixes music samples together into a single track. however, there is a distinct problem arising in that users are finding it increasingly difficult to manage the bits of data that constitute their digital content collection. “the problem is i can’t find things when i need them. where did i put that photo, or keep that song, or the last thing that i mixed together.” this indicates that users’ need a more centralised place to keep their multimedia content from which they can gather together the desired elements into a single message or ‘token’. furthermore, digital content can be quite sentimental and users in the studies report that content stored on 3g phones is particularly vulnerable to loss. three users reported that their phones crashed resulting in content being lost, while the handsets themselves are vulnerable to loss or theft. this indicates a need for a remote backup archive, and although users can currently back up content from 3g phones on their computer, this content it is not easily retrievable remotely via 3g phones. emerging theme 3 – content production across channels: the studies found that as 3g phones and internet technologies converge, users are merging the two communication channels. for example, taking photos with their phone, downloading them to their computers, editing them on their desktop and then distributing them for free via group email. this is significant because as counts and fellheimer (2004) point out, both channels have advantages and disadvantages. the user studies proved to be helpful in indicating how users are taking advantage of the best of both channels to combine mobile and internet technologies to create, circulate, distribute and archive content. user need 3 – peer to peer sharing: users in the studies reported that their ability to share experiences that they have produced themselves via text, images and sound bytes is limited by technical problems such as the incompatibility of handsets. for example, a homemade multimedia message produced on a nec phone cannot be transmitted to a nokia handset. users in the studies report that these fears and limitations force them to resort to channels where they can download, for a fee, commercially produced content. this is significant because for the users in the studies, most of the fun of producing their own content was sharing it instantaneously with friends. if users have to wait until they get to their computer to share their content the moment may be lost. this does not mean that users don’t still consume content rather they resort to downloading commercially produced content instead. translating user problems into a scenario content, such as digital pictures, play a big part in constructing digital identity, and this means that young people want to be able to express themselves via a set of negotiated rules as to what parts of their constructed identity and information others may be able to see. through the open-ended interview process, possible solutions emerged. five uses expressed a desire for ‘art gallery’ features for content, so visitors could come and browse selected images/sounds/text without being able to copy and re-distribute them. in order to communicate these findings to the technologists in the secure identity management project, a scenario was developed. it was in keeping with cooper’s (1999) approach in that it employed user archetypes to convey user needs and in keeping with carroll’s (2003) method in that it depicted a scenario of future interaction. the scenario, called the ‘trophy room’ essentially embodies the user needs from the studies by providing a digital art gallery where users can archive and display their multimedia content. it resides on the user’s mobile phone, although the trophy room can be accessed via the internet. selected visitors can then come and browse images/sounds/texts without being able to copy and re-distribution them. this solves the first user ajis special issue december 2004 119 need that while users are happy to share their content, it is more desirable if they can maintain it in their own space where visitors can come and listen or look. the trophy room provides a place to archive and display images which makes it a logical site for content distribution. this addresses the second user need for better management of multimedia content. user can then ‘cut and paste’ the desired bits of content into a central token which they can then be sms/mms or emailed to selected friends. furthermore, users would be able to have a choice over what the tokens looked like. for example, the token could be a text message embedded with pictures, or a picture embedded with sound bytes. finally, the user studies indicated that the third user need for multimedia content sharing as social communication to be successful is that the content has to be seamlessly distributed across platforms so that experiences can be shared instantaneously. when the scenario was presented to designers during a workshop for the secure identity management project, ways in which content could be carried across platforms included the use of mpeg21 and the use of instant messaging and peer-topeer systems such as jabber and jxta. in order to ensure rigour, the trophy room scenario was returned to seven users in the studies. a written description of the trophy room scenario was left with the users and they were asked to write any comments they had on the back of paper. the scenarios were picked up a few days later and when possible, a brief discussion in relation to the users’ comments took place. this was to be the most interesting part of the research because the scenario produced a high level of user feedback, providing not only a means for the users to reflect further upon their own use, but for them to suggest potential additions to the scenario. this indicates the effectiveness of scenarios, not only for communicating with computer scientists, but essentially as a form of ‘cultural probe’ (gaver, et al: 1999). in this case, the use of a scenario proved helpful in filling in the gaps in the user requirements left by the open-ended interviews. the use of both methods combined to provide an in-depth understanding of the participants’ use of content production as social communication. this was supported by wixon who stated that “different qualitative methods tend to complement each other, and other approaches. when the results gathered from different techniques point to similar conclusions, our confidence in the conclusions is increased” (wixon: 1995). it is important to note that the seven users who were selected to review the scenarios were chosen because of their ability to provide critical feedback. crabtree et al point out that “cultural probes are not analytic devices but ‘reflect’ the local culture of participants and are drawn upon to inspire design”(2003). this held true when using the trophy room scenario as a form of cultural probe in that giving the scenario back to users helped make the leap from user need to design solution. the users provided valuable insights into how the trophy room design could be improved to help the user meet their social and cultural goals. it emerged that users would not just want visitors to the trophy room to access all the content that was there, for example, there would be different groups of people with whom they would want to share different photos. six out of seven users stated that the idea of just providing free access for all would be highly undesirable because photos were often specific to events and only people that were at the events should have access to the photos. therefore, an important part of the trophy room scenario became that ‘selected’ others would have access to it or parts of it and amongst these selected others different visitors would be granted different levels of access. for example, only a best friend would be allowed to see the picture of the blind date the night before, but a video of a swimming race win could be accessible to any visitor. so in a sense the trophy room has a different look depending on who has access to it. for this to be achieved the mobile device must allow peer-to-peer authentication that can be easily controlled. ajis special issue december 2004 120 potential access levels when the trophy room scenario was returned again to the users, all seven users agreed that this level of control was an important addition and that part of the pleasure of having content displayed in the trophy room was deciding who got to see what. a table was developed to map out how visitors would be provided with different levels of access. this could be done by setting up profiles so that when they try and enter the trophy room the profile group that the owner of the phone set up for the visitor automatically grants them a level of access. the table was then returned to the seven users and their feedback was refined (table 1). the idea of a ‘day pass’ was included so someone could see a particular picture from an event once, but then not return unless invited. furthermore, three users indicated that a desirable feature would be if selected visitors could also contribute their own content, so that if for example, a group of friends went to an event, visitors at that event could add their own photos as well. table 1: potential access levels conclusion the studies described here show that 3g phones offer young people the opportunity to communicate through content that they have produced themselves; however, this practice is restricted by fears that the privacy of home produced content can not be protected, archiving difficulties and finally limitations such as the incompatibility of handsets. users reported that these fears and limitations force them to resort to channels where they can download, for a fee, commercially produced content. this indicates that the challenge for designers is to facilitate users in the creation, protection and management of their own multimedia content that they can seamlessly distribute across platforms. the trophy room scenario encapsulates a response to these user problems in a manner that can be used by designers. it suggests a series of design implications and demonstrates how dynamic access profile areas accessed potential visitor content allowed duration gold pass access all areas best friend, partner visitors can view all content and also contribute their own images, text and sounds etc. unlimited vip high level access friends, close relatives, trusted work friends. same as a gold pass but a vip visitor would not have access to every single part of the trophy room. unlimited general admission access to the ‘sanitized’ part of the trophy room. parents, colleagues at work the general admission lets visitors into a controlled part of the trophy room. general admission visitors can’t post their own content. unlimited day pass same as general admission. visitors that may want to see specific content. same as general admission. expires after 24 hours. ajis special issue december 2004 121 artefacts created and managed on 3g phones will afford users with the ability to broadcast their own personal experiences beyond current technology. this means that users can move past the limits of current technologies that have the potential to render them as passive consumers of commercialised ring tones and celebrity pictures. references carroll, john. 2003. hci models, theories, and frameworks. calif.: morgan kaufmann. cooper, allen. 1999. the inmates are running the asylum. indianapolis: sams. counts, scott, and fellheimer, eric. 2004. supporting social presence through lightweight photo sharing on and off the desktop. in proceedings of the 2004 conference on human factors in computing systems: acm press. crabtree, a, hemmings, t, rodden, t, cheverst, k, clarke, k, dewsbury, g, hughes, j and rouncefield, m. 2003. designing with care: adapting cultural probes to inform design in sensitive settings. proceedings of ozchi 2003: brisbane, australia. gaver, bill, dunne, tony and pacenti, elena. 1999. design: cultural probes. interactions 6 (1) : 21-29. howard, steve, carroll, jennie. murphy, john and peck, jane. 2002. using 'endowed props' in scenario-based design. in proceedings of the second nordic conference on humancomputer interaction: acm press. minichello, v., aroni r., timewell, e., alexander, l. 1995. in-depth interviewing: researching people, 2nd ed. melbourne: longman. taylor, a.s., and r. harper. 2002. the gift of the gab: a design oriented sociology of young people’s use of mobiles. paper read at proceedings of conference on human factors and computing systems, chi. wixon, dennis, ed. 1995. qualitative research methods in design and development: acm press. satchell, c, singh, s and zic, j © 2004. the authors assign to ozchi and educational and nonprofit institutions a non-exclusive licence to use this document for personal use and in courses of instruction provided that the article is used in full and this copyright statement is reproduced. the authors also grant a non-exclusive licence to ozchi to publish this document in full in the conference papers and proceedings. those documents may be published on the world wide web, cd-rom, in printed form, and on mirror sites on the world wide web. any other usage is prohibited without the express permission of the authors. ajis vol 9 no. 1 september 2001 inter-firm information sharing in enterprise resource planning systems: a call for timely but limited access to customer information michael e. busing computer information systems and operations management program james madison university msc 0202, harrisonburg, va 22807 usa abstract current trends in enterprise resource planning (erp) suggest that supply chain management and tight control over scheduling jobs within the supply chain are key tactical planning issues. modern erp software packages, in conjunction with the world wide web, allow for automated exchange of information within a company and also between two or more companies (i.e., conveyance of customer information to suppliers) of parts and components) for the purposes of effective planning and control. while ease of information exchange between a customer and supplier is increasingly critical to the success of modern-day planning and control efforts, the issue of information security is also a very real concern. suppliers can benefit from gaining access to a customer's dispatch list and material requirements plan (mrp) in order to determine real-time priority of jobs in queue at various work centers within their own organization. other customer information, however, should remain secure and unavailable to supplier firms for competitive reasons such as threat of forward integration. this paper presents a previously tested priority-sequencing rule that explicitly considers downstream shop conditions in determining which job to run next the rule proves to perform well on mean flow time and lateness as well as on variability of these measures. the rule is extended here to incorporate the case where a downstream work center is outside official corporate boundaries. with the call for free exchange of information comes the threat of other, perhaps proprietary, information being accessed by vendors or others outside the official corporate boundaries. the paper will propose information that should be freely exchanged between customers and suppliers and information that should remain secure. finally, practical measures to manage access to web-enabled erp information will be proposed. introduction review of the dispatching research for job shops indicates that this is a problem that has been studied for over four decades. dispatching is generally defined as a procedure that uses logical and simple decision rules to determine the priority of jobs waiting at a work center queue. the problem has been described as potentially difficult since for a given set of jobs, m, waiting in queue, there are m-factorial ways to sequence those jobs for processing at that work center. recent developments in dispatching practice allow web-enabled erp systems to engage, effortlessly, in information exchange between the company's planning and control system and: the company's own shop floor, the company's suppliers) of raw materials, parts, and components; the company's customers both of final product and of parts and components that will be further used in their production of product throughout the supply chain (i.e., b2b). the dissemination of detailed information about a particular shop order from the erp database in real time allows shop personnel to concentrate on serving customers while also keeping internal operations efficient (latamore, 2000). this will almost certainly allow more sophisticated dispatching rules to be employed in a practical settings. the ultimate goal of dispatching is to develop a schedule for the queue of waiting jobs that results in good shop performance with respect to both efficiency (e.g., flow time statistics) and effectiveness (e.g., consistent due date performance). with the advent of the world wide web, information can now be easily exchanged between suppliers and customers and efficiency and effectiveness can be extended throughout the supply chain. it needs to be reiterated, however, that the system for information exchange must insure that the information shared is directly applicable to the dispatching problem and that proprietary information is safeguarded. this is a key relationship between modern-day dispatching and the need for information security making the right information easily available to others while preventing other information from being exchanged. the following section will further discuss shop floor dispatching in light of recent advancements in erp systems and the world wide web. following that discussion, key security issues will be discussed. recent developments in dispatching research ovacik and uzsoy (1994) point out that advancement in shop floor information systems has outpaced dispatching rule research and development. current shop floor control modules not only have the ability to track the current location of a job and where it needs to go next, but can also determine, with significant accuracy, when the job is expected to arrive at a particular work center, given current shop conditions. current dispatching practice and research, however, does not take advantage of this readily available and potentially valuable information particularly when the information is contained in a customer's database. perhaps one underlying 15 ajis vol 9 no. 1 september 2001 reason is the issue of information security. with respect to job shop dispatching, there is a definite need to further develop and study rules that take advantage of global information readily available via modern erp systems. in the event that a particular job in queue at a work center is an input to a customer's manufacturing process (i.e., downstream in the supply chain), the global information should be accessed to determine a realistic assessment of priority. customers should be willing to share the information because of the cost savings resulting from increased efficiency that can potentially be passed along to them as long as proprietary information is not compromised and is unconditionally safeguarded. development of a shop floor dispatching rule as was discussed earlier, shop floor dispatching generally relies on due date information for the control of work flows. in theory, this should provide for good customer service with respect to meeting job due dates. a problem with such an approach, however, may result from congestion at overburdened work centers. if severe, this congestion can ultimately stifle job throughput and increase mean job flow time. at the other extreme, shop floor dispatching can rely solely on downstream work center conditions when determining which job to run next. for example, if two or more jobs are waiting in queue at a work station, then the job that has the least amount of expected waiting time at its immediate successor operation, based on current downstream queues, is chosen for processing next. intuitively, this approach should display a low mean job flow time (relative to a rule that relies exclusively on due date information) due to minimum likelihood of congestion and waiting at the downstream operation. the problem with this approach, however, is that without regard for due dates, jobs may either be delivered from the system early or late neither of which is desirable under a forbidden early shipment constraint. a shop floor dispatching rule that is classified somewhere in between on the continuum should be sensitive to due dates and to downstream work center conditions (i.e., amount of work currently in queue at the downstream operation). therefore, a rule must utilize job/operation due date information as well as information about actual shop floor conditions to be considered hybrid. both pieces of information may in fact come from customer erp system databases. shop floor dispatching rule the following dispatching rule allows a user to vary the degree to which a decision rule relies on either due date information or shop floor conditions (busing, 1996). this is important from the standpoint that the decision maker can fine-tune the model to meet his or her own customer service / operating cost objectives. in order for the model to be successful, however, customer databases must be accessed by supplier firms whenever both of the following conditions are met: 1) a job is at the last operation work center in the supplier company and 2) a job is used as an input to the manufacturing process of the customer's operation (i.e., further value is added before it reaches the customer's immediate successor in the supply chain). the rule requires information to routinely be accessed by supplying firms from a customer database, but should not be allowed if information security for proprietary information cannot be ensured. the model requires that a shop floor dispatching constant, a, be selected (0< a <1). for each job waiting in queue at a particular work center, the following hybrid index is computed: obtained from customer database when job is at last operation in supplying firm's shop dispatching index = v(£(ttd)x/' + (1-qi) \qtc } (1) where qi = shop floor dispatching constant (0 < a < 1), ttd = current operation's due time in hours current calendar time in hours, and qtc = present amount of work in hours waiting at the job's next operation. the job with the smallest dispatching index is selected for processing next. a small index is of high priority since it can become small due to a small amount of time remaining until due or due to a small amount of work (i.e., congestion) at the next operation (regardless of whether or not the next operation is internal or that of a customer). in the event of a tie between two or more jobs, shortest operation next (son) prevails as the decision rule due to its superior queue clearing properties and resulting guarantee of minimum mean flow time for a set of 16 ajis vol 9 no. 1 september 2001 jobs at a particular queue. when a job is at its last operation, the values for qtc and ttd are determined by accessing the customer's database. specifically, the job is matched with the customer's production schedule to determine which work center or facility will perform further processing in the supplier organization. in the event that the job has no further value added at the customer's organization or is simply a stock-item (to be used later), the dispatching index is exclusively tdd. this priority dispatching rule can be constructed such that it places more emphasis on operation due dates (dd 1), or so that it weights actual shop floor condition information more ( q u o ) . in addition, when d=0, the dispatching rule relies exclusively on downstream shop floor conditions and when 0=1, the rule relies exclusively on operation due dates. thus the model is extremely flexible. performance measurement performance of the shop floor dispatching rules is evaluated with respect to both internal and external criteria. internal system performance measurement is in terms of the average amount of time a job spends in the system, while external system performance is measured by customer service. blackstone, phillips, and hogg (1982) cite flow time, lateness, and tardiness as measures most often used in evaluating dispatching rule effectiveness in a job shop environment and the proposed rule was evaluated on all three. flow time flow time is the amount of time that a job spends in the system. in this study, flow time for an individual job is obtained as follows: 1. record current calendar time when new job is created, 2. record current calendar time when that job finishes processing at its last operation, and 3. (completion time (2) creation time (1)) is recorded as an observation for each completed job. the average and standard deviation of flow time was then calculated across all completed jobs for a given simulation run. flow time is really an indicator of both shop congestion and work-in-process inventory. for example, if two jobs with identical routings and processing times (actual) are run through two separate systems, then the job with the largest flow time clearly experienced more waiting time at its operations due to longer queue delays (congestion) and hence contributed to an increased work-in-process inventory cost for that particular system. lateness lateness is defined as an order's completion calendar time minus its published job due time. since lateness can take on positive or negative values, it is desirable to observe an average lateness statistic close to zero with a minimal standard deviation. that is, on average, orders are neither early nor late. since lateness is a function of how job due dates are established, it is necessary that it be considered only as a relative measure of performance for this study. in order to be consistent with other dispatching research (conway, johnson, and maxwell, 1960), a particular shop floor dispatching rule is judged not on an absolute sense or with respect to its ability to achieve a mean lateness statistic close to zero, but rather on its ability to maintain low mean and standard deviation of lateness relative to other shop floor dispatching rules. tardiness tardiness is defined as the maximum of either zero or an order's completion calendar time minus its published job due time. therefore, mean tardiness provides a measure of how late the late jobs are, on average. this prior study considered mean tardiness in evaluating push, pull, and hybrid shop floor dispatching performance (i.e., the mean lateness of the late jobs only). as is the case with lateness, tardiness is considered only a relative measure of performance among various shop floor dispatching rules. performance there is some preconceived expectation of system performance. first and foremost, it is expected that krajewski et al. (1987) are correct in stating that environment is important. in good environments (e.g., relatively low shop utilization and low standard processing time variability) any dispatching rule may produce acceptable results. however, as the shop environment becomes more difficult (e.g., extremely high shop 17 ajis vol 9 no. 1 september 2001 utilization and high standard processing time variability), it becomes unclear which, if any, of the rules yields acceptable performance. the type of shop (i.e., job shop versus flow dominant job shop) may prove to significantly affect relative performance of shop floor dispatching rules. for example, a flow dominant job shop is likely to generate temporary congestion at certain key work centers (gateway and finishing) which is not necessarily found in a comparable job shop. due date sensitive shop floor dispatching rules may be less capable of managing this congestion than are other shop floor dispatching rules. therefore, due date sensitive shop floor dispatching rules employed in a flow dominant shop are expected to display higher, more variant flow time and lateness statistics and higher tardiness than would the same due date sensitive shop floor dispatching rules employed in a job shop. it remains a bit unclear, however, whether or not all other shop floor dispatching rules (e.g., those that incorporate at least some information about jobs' downstream work center conditions) are affected by a flow dominant shop's congestion in a similar manner. with respect to specific performance measurement, it is expected that a due-date sensitive shop floor dispatching rule will generally display lower mean lateness and tardiness than shop floor condition sensitive rules, but may yield a higher mean flow time due to congestion at some work centers. a rule that incorporates both due date information and downstream work center information is expected to produce a significantly lower flow time statistic (mean and standard deviation) than a rule that exclusively utilizes due date information but is expected to yield significantly higher lateness (mean and standard deviation) and tardiness (mean) statistics than rules that exclusively utilize due dates when the shop floor dispatching constant, d, approaches zero. conversely, the dispatching rule proposed in equation 1 is expected to produce significantly higher flow time statistics than a rule that relies exclusively on shop floor condition information but is expected to yield significantly lower lateness (mean and standard deviation) and tardiness (mean) statistics than a rule that is predominantly due date sensitive. there is also expected to be an optimal value of d. such that the dispatching rule proposed in equation 1 can simultaneously achieve lateness and tardiness statistics that are not significantly different from those of due date sensitive rules and flow time statistics that are not significantly different from those of shop floor condition rules. information sharing this section will propose key customer information that should be easily accessed by suppliers when employing the dispatching rule detailed in equation 1. following that discussion, a proposal to safeguard other, perhaps proprietary, information contained in erf databases will be made. freely exchanged information for the model shown in equation 1 to be effectively employed by a supplier firm, specific job related information should be accessible by the supplier's erp system from the customer's manufacturing database. specifically, when a job is at the last operation at the supplying firm, the supplier should access the customer's database to determine: 1. whether or not the job requires further value (i.e., processing at the customer site) and if it is required to enter the production process immediately upon arrival at the customer location. if yes to both questions, continue gathering information below. if no to either question, schedule job according to tdd (time until due date); 2. the job's due date at the customer operation; 3. the job 's first operation area (via the job's routing file); 4. the amount of work (standard hours) in queue at the job's first operation area. the above represents the only information that is needed for employment of the model proposed in equation 1. furthermore, the above represents the only information that the customer should make available to its supplier. secured information in order to protect itself from loss of proprietary information and from threat of forward integration on the part of supplying firms, a customer should ensure that the following information is not made available to a supplier via the world wide web. 1. customer's customer contact information and other marketing data; 2. job pricing and costing information; 3. engineering drawings; 18 ajis vol 9 no. 1 september 2001 4. bill of materials files; 5. financial data (profit/loss statements, etc.); 6. payroll; 7. employee database and other human resources data; 8. work center or process proprietary information; 9. accounts receivable; 10. accounts payable; 11. inventory. the above list is by no means exhaustive, but does represent information that absolutely must be protected. protecting proprietary information since outsiders (i.e., entities other than suppliers of parts and components or those employed by the organization) represent only about 20 percent of the total attack threat (nichols, ryan, and ryan, 2000), this paper will focus on the larger, insider, threat (specifically, suppliers of parts and components). as illustrated above, it is clearly advantageous for a customer to share selected information with suppliers if the result is better shop performance and resulting lower costs that are passed on to the customer in terms of lower prices. however, a customer should also ensure that the proprietary information identified above is not shared with suppliers such that information security is compromised. nichols, ryan and ryan (2000) identify access controls as a means for limiting and managing access to data within an enterprise. two types of access controls may be employed in order to share appropriate information with supplying firms while prohibiting access to other, proprietary, information. they are discussed below as they relate to protecting customers from a supplying firm's misuse of information. role based access control in role based access control (rbac), information within the organization is classified according to subject matter. those personnel, both inside and outside the organization, are given access to specific information based on the role they play in the organization when rbac's are employed. for example, with the shop floor dispatching rule mentioned above, a supplier may be given access to shop floor information because of their role as a supplier. however, it should also be noted that the same supplier could conceivably access shop floor information that is not needed to perform the dispatching rule calculation based on their role as a supplier. this information could include work center or process proprietary information if role based access control is the only system in place to manage access to information. mandatory access control mandatory access controls (mac's) permit or prohibit viewing and transmission of data based on the information's sensitivity within the organization. when rbac is combined with mac, a much higher level of security is realized since mac generally takes priority over rbac. for example, when the above scenario is again considered with the addition of mac, a supplier has access to shop floor information because of his/her role in the organization as a supplier and because the firm has a rbac in place. however, the same supplier may not view the work center or process proprietary information because it is highly sensitive in nature and because a mac is in place. conclusion this paper has illustrated a practical scenario in enterprise resource planning where two or more firms can benefit from the snaring of information. specifically, a supplier of parts and components can very likely achieve cost savings by obtaining up to the minute information about a buyer's shop floor conditions when job priority and dispatching decisions need to be made. some portion of these cost savings can be passed on to the buyer firm. hence, willingness of both parties to share information should exist. however, with the sharing of selected information comes the need to protect more proprietary information. this paper shows how role based access control, combined with mandatory access control can facilitate protection of proprietary information. 19 ajis vol 9 no. 1 september 2001 references blackstone, j.h., phillips, d.t. & hogg, gl. (1982) "a state-of-the-art survey of dispatching rules for manufacturing job shop operations", international journal of production research, vol. 20, no. 1, pp 27-45. busing, m.e. (1996) "a comparison of push, pull, and hybrid approaches to shop floor dispatching", unpublished ph.d. dissertation, clemson university. carroll, d.c. (1965) "heuristic sequencing of single and multiple component jobs", unpublished ph.d. dissertation, sloan school of management, m.i.t. conway, r.w. (1965) "priority dispatching and work-in-process inventory in a job shop", journal of industrial engineering, vol. 16, no. 2, pp 123-130. conway, r.w., johnson, b.m. & maxwell, w.l. (1960) "an experimental investigation of priority dispatching", the journal of industrial engineering, vol. 11, no. 3, pp 221-229 day, j.e. & hottenstein, m.p. (1970) "review of sequencing research", naval research logistics quarterly, vol. 17, ppl 1-39. fry, d., philipoom, p.r. & blackstone, j.h. (1988) "a simulation study of processing time dispatching rules", journal of operations management, vol. 7, no. 4, pp 77-92. hershauer, j.c. & ebert, j. (1974) "search and simulation selection of a job shop scheduling rule", management science, vol. 21, no. 7, pp 833-843. krajewski, l.j., king, b.e., ritzman, l.p. & wong, d.s. (1987) "kanban, mrp, and shaping the manufacturing environment", management science, vol. 33, no. 1, pp 39-57. latamore, gb. (2000) "using erp data to get [close] to customers", apics: the performance advantage, vol. 10, no. 9, pp 26-30. nichols, r.k., ryan, d.j. & ryan, j.j.c.h. (2000) defending your digital assets, new york: mcgraw-hill publishing co. ovacik, i.m. & uzsoy, r. (1994) "exploiting shop floor status information to schedule complex job shops", journal of manufacturing systems, vol. 13, no. 2, pp 73-84. 20 the australasian journal of information systems (ajis) began in 1993 and was based on the simple realisation that there was a need for an academic ‘outlet’ for is research in australia australasian journal of information systems volume 14 number 1 nov 2006 information privacy: culture, legislation and user attitudes sophie cockcroft business school university of queensland brisbane, australia email: s.cockcroft@business.uq.edu.au abstract information privacy has received much public and research interest in recent years. globally this has arisen from public anxiety following the september 11 attacks and within australia a progressive tightening of privacy legislation in particular the privacy amendment (private sector) act of 2000 which became operative in 2001. this paper presents the results of a study into attitudes towards information privacy. based on an instrument developed and validated by smith et al (1996a) this study sets out to measure individual concerns regarding organisational use of information along four dimensions: collection, errors, unauthorised secondary use, and improper access. the survey was completed by 67 undergraduate and postgraduate students enrolled in an e-commerce security subject at the university of queensland. comparisons are drawn between the results of this study and an identical one carried out at the university of north alabama. whilst it is too early to draw conclusions about the impact of these attitudes on the success of e-commerce in general, the results should be of interest to those within universities seeking to expand the use of networking technologies for handling sensitive information such as enrolment and fee processing (vanscoy & oakleaf 2003) keywords: is security, information policy, managing is introduction electronic commerce has become a reality the expansion of telecommunications capabilities has meant that the information age is upon us. this has made it easy for companies to gather and store vast amounts of consumer data. whilst this can lead to competitive advantage for the firms concerned it heightens concerns over information privacy for the individual. this study aims to explore the level of concern regarding information privacy issues and to identify factors associated with possible differences in those attitudes. we compare results with a university based study at the university of north alabama (una) (alexander 1998). this study was chosen because it uses the raw, unaltered version of an instrument originally developed and validated by smith, milberg and burke (1996a). it was also used in an earlier study by the same authors (milberg et al. 1995). many authors have since used the 15 question instrument as starting point for further research, but in these studies the instrument was adapted (bellman et al. 2004a; liu et al. 2004; malhotra, kim & 55 australasian journal of information systems volume 14 number 1 nov 2006 agarwal 2004a; smith, milberg & burke 1996b). it measures participants' attitudes along four dimensions; collection, errors, improper access, and secondary use. the paper is organised as follows. in the following section the theoretical framework of the study is laid down. next, the research process is described. in the penultimate section the key findings are presented and compared with findings in the two earlier studies that used the same instrument. finally the contributions of the study and suggestions for further work are explored. information privacy legal background in the past ten years there has been increasing global interest in privacy. media attention has focussed people's attention on information privacy and their right to it (henderson 1999). there are two broad means by which personal information privacy can be achieved; company self regulation, and government legislation. an appropriate level of privacy protection is essential for successful ecommerce (forder & quirk 2001). the european union and the united states lie at two ends of the spectrum regarding their approaches. in the us the clinton administration saw privacy concerns as a threat to the development of e-commerce. it considered that private industry working in cooperation with consumer groups was preferable to government regulation (culnan 2000). europe has taken a much more hands on approach with the passing of the eu directive on privacy that came into effect in 1998 (eu 1998). this directive established minimum standards for processing and use of personal data. in australia the privacy act of 1988 applied only to federal government agencies. there was a long period in which state and federal governments have been uncertain whether to introduce legislation that applies to the public and private sectors. this uncertainty was resolved in 1999 when the privacy amendment act (private sector) was proposed. this came into force in december 2001 (australian privacy commissioner 2000). given that e-commerce has to occur in a global environment, these differing approaches can cause some conflict, and have the potential to hinder the smooth running of commercial operations. it is imperative that there is cooperation between citizens, merchants and governments of different nations. a full examination of the legal implications of assuring privacy is beyond the scope of this paper. it is the role of citizens in this three-way cooperation that is of interest here. the main aim is to take a closer look at individuals’ privacy concerns, and to contribute to the body of research that informs regulation, both self and governmental. a key feature of the study is the use of a validated instrument, which makes the results readily comparable to past and future studies. like the previous study mentioned, (alexander 1998), this study uses students as its subject. whilst the results of this study may not necessarily be generalised to the population as a whole, they are likely, at the least, to be of interest to system developers and policy makers within the university setting. records maintained by universities are, by their nature and by law, private. however universities are increasingly expected to provide access to information through networking technologies these include systems for managing enrolment, grades and payment of fees. for example at the university of queensland a system called si-net that supports these activities is undergoing an extensive upgrade (student systems group 2006). initiatives such as these heighten the need for vigilance regarding the security of these records. 56 australasian journal of information systems volume 14 number 1 nov 2006 the role of social and demographic factors researchers in this area have identified a number of factors influencing individual’s attitudes to information privacy. in particular frequency of use has been associated with lower levels of concern about privacy issues, and less experience with higher levels of anxiety (reported in (alexander 1998)). researchers have also explored the effects of gender, age and personality type on attitudes to information privacy (vance 2000). of interest in australia was the roy morgan poll which found that younger australians (14-24) were least fearful, and women were more inclined to be worried about privacy issues (roy morgan 1999). culnan (1993) made a particular study of secondary information use which she defined as "the use of personal information for other purposes subsequent to the original transaction between an individual and an organization when the information was collected". the key finding of this study was that concern over secondary use was correlated with the level of control the individual has over the secondary use. this type of control is exemplified by american express, which agreed to inform its customers that it tracked their buying habits and used the data to compile mailing lists, which it sold to other companies. american express further agreed to notify card holders of their ability to "opt out" of such lists (crenshaw 1992) in (culnan 1993) . milberg et al (1995) investigated cultural influences on attitudes to privacy amongst members of the information systems audit and control associations in a number of countries. one of their key findings was that lower levels of privacy concern were associated with countries with no privacy regulation and in countries which have the highest level of government involvement in corporate privacy management. this is a complex relationship indicating that where there is little awareness there is little concern and increasing levels of concern result in increasing legislation to the point where concern again drops off. in order to classify the level of government involvement in corporate privacy management they developed the scale illustrated in figure 1. australia was placed in the middle of the scale under data commissioner, and the us was placed lower down the scale under voluntary control. figure 1 level of government involvement in corporate privacy management (milberg et al. 1995) p66 the studies mentioned in the first two paragraphs are hard to compare because they all used different instruments and reviewed different dimensions. the following section explains how some of the factors identified in earlier studies were tested, and how comparisons were made with studies using the same instrument. 57 australasian journal of information systems volume 14 number 1 nov 2006 research method the survey is based on a parsimonious 15 question instrument developed by smith, milberg and burke (1996a): figure 2 survey questions (smith, milberg & burke 1996a, p170) the survey was made available to the 100 students of a postgraduate subject "securing the commercial internet" and the 70 students in the undergraduate subject of the same name. there 58 australasian journal of information systems volume 14 number 1 nov 2006 were 52 valid responses in the postgraduate subject giving a response rate of 52% and 15 in the undergraduate subject giving a response rate of 21% and an overall response rate of 39%. there were 30 women in the survey and 36 men and one person did not state their gender in the sample there were 34 students who identified themselves as being of asian origin, 19 who identified themselves as european/australian and 14 who did not state their race/ethnicity. the survey was delivered via webct, which has built in survey software that ensures the responses are anonymous. ethical approval was granted and students were given and overview of the project and assured that their participation in no way related to the marks for the course, and that they could withdraw at any time. following this a consent form was signed. in addition to the questions shown in figure 2, further questions were asked about demographics and other background factors described below that are often associated with attitudes to privacy. the supplementary questions selected here, followed those of alexander (1998) who carried out a comparable university based study and related to skill level, and work environment. the questionnaire measures participants' attitudes along four dimensions, as illustrated in figure 3: collection these questions deal with concerns about the volume of data being collected and the reasons for its collection. the “collection” subscale is made up of questions a,e,j and o shown in figure 2. it is reflected in principle 1 of the 10 privacy principles outlined in the australian privacy amendment act (office of the privacy commissioner 2000) “an organisation must not collect personal information unless the information is necessary for one or more of its functions or activities. an organisation must collect personal information only by lawful and fair means and not in an unreasonably intrusive way”. errors questions b,f,h and l of figure 2 make up the “errors” subscale. these questions examine participants’ attitudes towards errors, both accidental and maliciously planted, and the steps companies should take to avoid them. when an organization holds information about its customers it should allow them to find out what that information is, and contest its accuracy and completeness if they believe it is wrong. this is the underlying premise of the access/correction principle, another of the 10 privacy principles outlined in the australian privacy amendment act (office of the privacy commissioner 2000). improper access the questions on improper access probe participants’ attitudes to who has access to their information within the organisation that collects and stores it. the improper access subscale measured by questions d, i and n reflects the principle of data security from the australian privacy amendment act which states that an organisation must take reasonable steps to protect the personal information it holds from misuse and loss and from unauthorised access, modification or disclosure. secondary use these questions relate to the common practice of using data for purposes other than those for which it was collected. e.g collecting it for research and using it for marketing, or collecting data in the course of normal credit card transactions and using it for marketing. in an australian context this is 59 australasian journal of information systems volume 14 number 1 nov 2006 directly related to principle 2 of the privacy amendment act, use and disclosure, and measured by items c,g, k and m in the instrument. research propositions the analysis was conducted in two parts. first, comparisons were made between the results of this study, and two previous studies that used the same instrument. second with reference to the literature review above, six factors were identified which have been shown to have an effect on attitudes to privacy. these are illustrated in figure 3 and the relevant research propositions are identified below. these factors were tested using one-way anova. years in job figure 3: factors proposed to be associated with privacy concerns research propositions a number of authors have proposed a negative relationship between years of work, and use of a computer at work, and levels of privacy concern (alexander 1998; vance 2000). another common finding is that more experienced internet users will have lower levels of privacy concern (bellman et al. 2004b; rose 2006). rose (2006), for example, found that those who had used the internet for more than 4 years had significantly lower levels of concern for internet privacy. this leads to the first three proposals; p1 more years at work is negatively associated with high levels of privacy concern, p2 self-reported skill level will be negatively associated high levels of privacy concern (alexander 1998), p3 use of a computer at work will be negatively associated with high levels of privacy concern (alexander 1998; vance 2000). race or ethnic age gender use of computer for job p3 skill level privacy concern s p6 p2 p5 p1 p4 60 australasian journal of information systems volume 14 number 1 nov 2006 studies are emerging that explore the influence of national culture on attitudes to information privacy (dinev & hart 2004; heales & cockcroft 2005; rose 2006). in the context of government surveillance dinev et al. (2006) found that italians had less concern for information privacy than consumers in the us . in the current study, data collection did not take a fine grained approach to culture, participants were simply asked their race or ethnic origin amongst the demographic questions similar to earlier studies (milberg et al. 1995; whitman, townsend & hendrickson 1999). the concept of culture, and its relationship to attitude to information privacy, is an area that requires further research, however for this analysis it is proposed that: p4 race or ethnic origin significantly affects levels of privacy concern. national polls conducted in australia, both before and after the introduction of the privacy amendment act, suggested that gender was significant in users’ concern for information privacy (morgan 1999; office of the privacy commissioner 2004). more recently some marketing studies have found that women have perceptions of greater risk in online transactions than do men (garbarino & strahilevitz 2004) and more concern about information privacy (phillip & suri 2004). thus it is proposed: p5 gender significantly affects levels of privacy concern. several authors have found a negative correlation between age and privacy concern noting that older internet users show a higher level of concern (rose 2006) (heales & cockcroft 2005). this led to the proposition that: p6 age is negatively associated with high levels of privacy concern. (morgan 1999) findings the findings of this study are addressed under three headings, first in comparison to the alabama study, second with the 1995 milberg study and finally in terms of testing the research propositions identified. comparison with alabama study table 1 shows the attitudes of students in this survey as compared to business students at the university of alabama (una). university of queensland (uq) students illustrate higher levels of concern in all dimensions. 61 australasian journal of information systems volume 14 number 1 nov 2006 undergraduate uq n=15 postgraduate uq n=52 overall uq n=67 overall una n=124 mean std. dev. mean std. dev. mean std. dev. mean std. dev. errors 6.00 0.74 5.35 1.03 5.50 1.00 5.51 1.15 secondary use 6.63 0.52 6.28 0.84 6.36 0.79 6.1 1.24 improper access 6.36 0.84 6.04 0.96 6.11 0.94 6.03 1.25 collection 5.50 1.18 5.41 0.84 5.43 0.92 4.59 1.3 overall 6.10 0.55 5.79 0.66 5.86 0.64 5.56 0.99 table 1 comparison with alabama study it will be noted that american students are generally less concerned about privacy which fits with the milberg study described earlier (milberg et al. 1995). in addition postgraduate students appear to be less concerned about privacy than undergraduate students on all dimensions except collection. this is illustrated in figure 2: 4.5 5 5.5 6 6.5 7 colle cti on sec ond ary u se erro rs im prop er acc ess undergraduate uq mean postgraduate uq mean overall uq mean overall una mean figure 2 privacy concerns among business student at una and e-commerce students at uq the original table in milberg et al (1995), on which table 2 is based, served to support the hypothesis in that levels of concern are lower at each end of the governmental regulatory scale illustrated in figure 1 and higher in the middle. in this adapted table, countries are listed top-down in the order of government involvement. australia, canada and new zealand are in the middle ground. 62 australasian journal of information systems volume 14 number 1 nov 2006 collection secondary use errors improper access thailand 4.7(4) 5.8(2) 5.2(3) 6(1) france 5(4) 6.6(1) 5.5(3) 6.2(2) japan 4.9(4) 6.3(2) 5.6(3) 6.5(1) united states 5.4(4) 6.1(1) 5.5(3) 5.9(2) united states (una) 4.59(4) 6.10(1) 5.51(3) 6.03(2) australia 5.6(3) 6.6(1) 5.3(4) 5.8(2) australia (uq) 5.43 (4) 6.36(1) 5.50 (3) 6.11(2) canada 5.7(4) 6.4(1) 5.6(3) 6.1(2) new zealand 5.3(3) 6.4(1) 5.2(4) 5.9(2) denmark 4.8(4) 6.2(1) 5.5(3) 6.1(2) united kingdom 5.2(4) 6(1) 5.6(3) 5.8(2) table 2 level of information privacy concern by dimension in countries (adapted from (milberg et al. 1995)) first number represents the mean on a scale of 1-7 the number in parentheses represents the dimensions rank in a given country. 4.5 5 5.5 6 6.5 7 colle cti on sec ond ar y u se erro rs im prop er acc ess united states united states (una) australia australia (uq) figure 3. australian and us attitudes to privacy. mainstream data taken from (milberg et al. 1995). student data taken from (alexander 1998) at the university of north alabama and the current study. the university of queensland students go against their national trend in terms of how the concerns are ranked. american students, as expected, appear to be less concerned about privacy than australian students. it is hard to judge the significance of this without further study of the make up of these student groups, as we will see the uq group was far from homogenous. the highlighted part of the table is illustrated graphically in figure 3. 63 australasian journal of information systems volume 14 number 1 nov 2006 testing of research propositions the six propositions identified were tested using analysis of variance (see appendix). p1 was partially accepted. years at work positively correlated with privacy concerns over improper access. p2 was rejected. those reporting a higher level of skill were not significantly less concerned about privacy. p3 was rejected because the significance was in the opposite direction than expected, i.e. use of a computer at work was positively correlated with privacy concerns in one dimension. those that use a computer at work are more likely to be concerned about improper access. p4 was rejected; there was no relationship between race and privacy concerns. p5 was rejected; gender had no effect on privacy concerns. p6 was accepted in one dimension age is a significant factor in concerns about improper access. of the hypotheses that were accepted, there were no significant two way effects between age, years at work and use of a computer at work. discussion the u.s. and europe exhibit very different approaches to information privacy from both regulatory and managerial perspectives. it seems that these differences are grounded in different cultural values and assumptions about the meaning of privacy (a "human rights" issue in europe versus a contractual issue in the u.s.). australia falls between the two. this study supports earlier work in that the students in this study exhibited levels of privacy concern appropriate to the cultural setting within which they find themselves, i.e. australia. a closer examination of the make up of the class brings into focus one of the limitations of the study. two thirds of the sample is of asian decent, and it is impossible to tell whether this will affect their attitudes since those who were brought up in asia cannot be distinguished from those who are born or naturalized australians. the crude division of asian vs. non-asian is not reliable since the data was gathered with a free-text field and respondents entered a variety of answers. if we stick with this division a further findings emerges. students of asian origin are less concerned about privacy than those of non-asian origin. the two other key findings were that those who used a computer at work were more likely to be concerned about collection this probably reflects a high level of concern relating to a broader class of data than just their own personal data. it likely represents concerns relating to the management of data in a work setting in general and possibly the data they personally collect and how it is stored. a further interesting and possibly confounding factor is that these students were at the start of a required course in security in e-commerce. this could have resulted in heightened awareness of privacy and security in general. one aim of this study was to provide the students with an opportunity to examine their own values and concerns in relation to privacy. one limitation of the work is that whilst a number of studies have used the 15 question instrument developed by smith milberg and burke or adaptations thereof(bellman et al. 2004b; liu et al. 2004; malhotra, kim & agarwal 2004b; smith, milberg & burke 1996a). recently it has been suggested that cfip may be more parsimoniously represented as a higher-order factor than a correlated set of first-order factors(malhotra, kim & agarwal 2004b; stewart & segars 2002). further work in this area should proceed in cognisance of this new understanding. this study contributes to the body of knowledge in the area of consumer privacy; it identifies information practices that have the potential cause public concern along predefined dimensions and 64 australasian journal of information systems volume 14 number 1 nov 2006 particularly in an australian context. firms can use this information to self-regulate and develop and promulgate proactive privacy policies as opposed to fielding complaints in reactive mode. appendix improper access coefficients: estimate std.error t value pr(>|t|) (intercept) 3.22065 1.17987 2.730 0.00878 age 0.02232 0.02582 0.864 0.39166 compjob 0.24866 0.10595 2.347 0.02301 * skilllev 0.12498 0.14069 0.888 0.37870 gender 0.13328 0.25892 0.515 0.60903 newrace 0.03028 0.04430 0.684 0.49740 yearsjob -0.05681 0.03442 -1.651 0.10523 -- * p<.05 residual standard error: 0.8621 on 49 degrees of freedom multiple r-squared: 0.1959, adjusted r-squared: 0.09743 f-statistic: 1.99 on 6 and 49 df, p-value: 0.08516 coefficients: estimate std.error t value pr(>|t|) (intercept) 4.46836 0.64955 6.879 7.06e-09 compjob 0.28076 0.09852 2.850 0.00622 ** yearsjob -0.03682 0.02401 -1.533 0.13111 -- ** p<.01 residual standard error: 0.8463 on 53 degrees of freedom multiple r-squared: 0.1619, adjusted r-squared: 0.1302 f-statistic: 5.118 on 2 and 53 df, p-value: 0.009283 secondary use coefficients: estimate std.error t value pr(>|t|) (intercept) 4.10130 0.99648 4.116 0.000147 age 0.04479 0.02181 2.054 0.045332 * compjob 0.08946 0.08948 1.000 0.322332 skilllev 0.04340 0.11882 0.365 0.716480 gender 0.30027 0.21868 1.373 0.175969 newrace 0.00186 0.03741 0.050 0.960548 yearsjob -0.04145 0.02907 -1.426 0.160215 -- * p<.05 residual standard error: 0.7281 on 49 degrees of freedom multiple r-squared: 0.1375, adjusted r-squared: 0.03183 f-statistic: 1.301 on 6 and 49 df, p-value: 0.2743 65 australasian journal of information systems volume 14 number 1 nov 2006 references alexander, p 1998, 'attitudes toward information privacy: differences among and between faculty and students', paper presented to the ais americas conference, baltimore, maryland, august 14-16, 1998. australian privacy commissioner 2000, privacy and the private sector, australian government, viewed 21/05/2001 2001, <http://www.privacy.gov.au/>. bellman, s, johnson, ej, kobrin, sj & lohse, gl 2004a, 'international differences in information privacy concerns: a global survey of consumers', information society, vol. 20, no. 5, pp. 313-24. ---2004b, 'international differences in information privacy concerns: a global survey of consumers', the information society, vol. 20, pp. 313-24. culnan, mj 1993, ' `how did they get my name?': an exploratory investigation of consumer attitudes toward secondary information use', mis quarterly, vol. 17, no. 3, pp. 341-62. ---2000, 'protecting privacy online: is self regulation working?' journal of public policy and marketing, vol. 19, no. 1, pp. 20-6. dinev, t, bellotto, m, hart, p, russo, v, serra, i & colautti, c 2006, 'internet users' privacy concerns and beliefs about government surveillance: an exploratory study of differences between italy and the united states', journal of global information management, vol. 14, no. 4, pp. 57-93. dinev, t & hart, p 2004, 'internet privacy concerns and their antecedents measurement validity and a regression model', behaviour & information technology, vol. 23, no. 6, pp. 413-22. eu 1998, eu directive on personal data protection enters into effect, viewed 21/5/04 2004, <http://www.eurunion.org/news/press/1998-4/pr89-98.htm>. forder, j & quirk, p 2001, electronic commerce and the law, 1 edn, wiley, brisbane. garbarino, e & strahilevitz, m 2004, 'gender differences in the perceived risk of buying online and the effects of receiving a site recommendation', journal of business research, vol. 57, no. 7, pp. 768-75. heales, j & cockcroft, s 2005, 'national culture, trust and internet privacy concerns', paper presented to 16th australasian conference on information systems, sydney, 29 november 2 december. henderson 1999, 'personal information privacy: implications for mis managers', information and management, vol. 36, pp. 213-20. liu, c, marchewka, jt, lu, j & yu, cs 2004, 'beyond concern: a privacy-trust-behavioral intention model of electronic commerce', information & management, vol. 42, no. 1, pp. 127-42. malhotra, nk, kim, ss & agarwal, j 2004a, 'internet users' information privacy concerns (iuipc): the construct, the scale, and a causal model', information systems research, vol. 15, no. 4, pp. 336-55. ---2004b, 'internet users' information privacy concerns (iuipc): tthe construct, the scale, and a causal model', information systems research, vol. 15, no. 4, pp. 336-55. 66 http://www.privacy.gov.au/%3e http://www.eurunion.org/news/press/1998-4/pr89-98.htm%3e australasian journal of information systems volume 14 number 1 nov 2006 milberg, s, burke, s, smith, j & kallman, e 1995, 'values, personal information, privacy and regulatory approaches', communications of the acm, vol. 38, no. 12, pp. 65-74. morgan, r 1999, big brother bothers most australians, roy morgan, viewed 15 october 2006 2006, <http://www.roymorgan.com/news/polls/1999/3221/>. office of the privacy commissioner 2000, national privacy principles (extracted from the privacy amendment (private sector) act 2000), viewed october 13 2006, <http://www.privacy.gov.au/publications/npps01.html>. ---2004, community attitudes toward privacy 2004, roy morgan research, sydney. phillip, mv & suri, r 2004, 'impact of gender differences on the evaluation of promotional emails', journal of advertising research, vol. 44, no. 4, pp. 360-8. rose, ea 2006, 'an examination of the concern for information privacy in the new zealand regulatory context', information & management, vol. 43, no. 3, pp. 322-35. smith, j, milberg, s & burke, s 1996a, 'information privacy: measuring individuals' concerns about organisational practices', mis quarterly, vol. june, no. 2, pp. 167-96. ---1996b, 'information privacy: measuring individuals' concerns about organisational practices', mis quarterly, vol. june, pp. 167-96. stewart, ka & segars, ah 2002, 'an empirical examination of the concern for information privacy instrument', information systems research, vol. 13, no. 1, pp. 36-49. student systems group 2006, welcome to mysi-net, university of queensland, viewed 21/10/06 2006, <http://www.uq.edu.au/sinet-support/>. vance, da 2000, 'on the effects of exogenous and endogenous variables on information privacy concerns: a preliminary comparative study', paper presented to challenges of information technology management in the 21st century. 2000 information resources management association international conference., hershy. vanscoy, a & oakleaf, m 2003, 'online privacy vs. enhanced virtual reference: what do patrons really want?' paper presented to acrl eleventh national conference, charlotte, north carolina, april 10.13, 2003. whitman, me, townsend, am & hendrickson, ar 1999, 'cross-national differences in computeruse ethics: a ninecountry study', journal of international business studies, vol. 30, no. 4, pp. 673-87. 67 http://www.roymorgan.com/news/polls/1999/3221/%3e http://www.privacy.gov.au/publications/npps01.html%3e http://www.uq.edu.au/sinet-support/%3e australasian journal of information systems volume 14 number 1 nov 2006 the author sophie cockcroft is a member of the information systems group at the university of queensland business school. her primary research interests relate to data quality, integrity control, and data modelling specifically in the application areas of geographic and medical information systems. more recently she has been looking at the effects of national culture on attitudes to information privacy and software development. her publications have appeared in the information and software technology, journal of global information technology management, and geoinformatica. she teaches courses relating to information systems development, including analysis and design of electronic commerce and www applications and security. sophie has previously been an academic at city university, hong kong and otago university, new zealand. prior to becoming an academic, sophie worked in accountancy and as a computing geologist at bp exploration in london. 68 abstract legal background the role of social and demographic factors collection errors improper access secondary use research propositions research propositions comparison with alabama study testing of research propositions appendix the author microsoft word 13 1 marina.doc ajis vol 13, no. 1 september 2005 45 a research strategy for investigating business process management approaches james gibson decision systems laboratory university of wollongong nsw 2522 australia jgibson@uow.edu.au abstract we are witnessing a revolution in industry which, if successful, will change forever how business systems are developed and the type of staff required. this paradigm shift has only recently become possible as business process conceptual understanding evolved, technologies have matured and higher abstraction levels have become possible. industry leads business processing systems research as it has the strategic imperative and resources to be effective. academic research is faced with three challenges: firstly, how to do effective research in an area of such broad scope, secondly, how to make research relevant to practice, thirdly how to spend limited resources effectively. this paper defines the research framework for effective academic research at the university of wollongong by the software effective process group. effective research is enabled by co-ordinating research based on the primacy of the business model and its resultant effective representation in executable systems. the framework aims to build a core research team, promote strong synergy with existing research areas, and create academic and industry relevant research.. we report on the results to date of our pilot program and seek feedback and advice to help us refine our approach. a major australian project is utilising a new software development lifecycle for ‘system of systems’ development which has arisen out of this research strategy. later papers will report on both the theoretical basis and practical impacts of this work and other research by the group. introduction corporations have acted on the realisation that business process innovation and management provides uniqueness and the competitive edge in the marketplace. implementing this is difficult, traditional systems development approaches carry high risk, are not scaleable, and are prone to failure. new ways of developing systems are arising with models and artefacts based upon business process as opposed to technical semantics, and founded on defined systems architectures. some lifecycles target ‘systems of systems’ design with a pragmatic focus on issues of skills, staffing and scalability for large projects. other lifecycles pursue the logical extension by creating business execution engines to directly execute the business semantics. the i.t. industry has responded by extending lifecycles such as the rational unified process, developing business process management systems (bpms), notations such as bpmn, and related technologies (eg web services). conceptually, the emphasis is now on process and process processing rather than data and data processing. major processes are built up out of smaller atomic processes. both major and atomic processes can be modified to keep track of changing business imperatives. the aim is to have the business processes endure for the life of the enterprise. it is the corporation and evolves as the corporation changes. it is difficult to effectively evaluate both the new lifecycles, and their potentially executable forms. not only must the ability to express process at the requisite detail be examined, but there are many other issues. some of these issues are useability focussed, some test the ajis vol 13, no. 1 september 2005 46 suitability for use as mission critical systems, others involve legacy system integration, skill sets, scalability and maintainability. multiple competing standards exist in various stages of completeness. for those lifecycles requiring executable engines, there is a plethora of vendor solutions and lack of underlying formal descriptions for most of these systems. industry must consider their own unique circumstances, and choose between a traditional, or, an untested and potentially high risk executable approach. still it remains that these approaches must be evaluated, fundamentals understood, and possible guidelines for future evolution developed. industry and academia both have roles to play in the definition, implementation and evaluation of business process systems (bps) and related development lifecycles. as academic researchers, we have limited resources and staff but still want to make a worthwhile contribution. focus and synergy is essential. we must apply triage to select tasks which have the best outcomes not only for the individual but also for the group. in this paper, we present a research framework. this framework is intended to give us guidelines to choose between possible courses of action, and to best direct our efforts. the framework touches only lightly on the actual research areas, rather it is a set of rules for how to choose research and manage research, and build synergies both within academia and to industry. such a framework is important as it provides an antidote for the short term, operational approach currently encouraged by australian universities key performance indicators. it is hoped that this research framework will give structure to long term, strategic research with the aim of promoting productivity, communication and interaction amongst researchers in this area with practice 4 relevant outcomes. an earlier version of this paper was presented at the bpmds workshop at caise’05, portugal. background for the first fifty years, it has been about the processing, storing and transporting of information in digital form.(carr 2003) technology drove the marketplace and solutions were localised and resistant to change. businesses had the choice of either building custom applications or purchasing packaged products embodying fixed business processes such as erp solutions. it was thought of in terms of applications, hardware and network. importantly, none of these approaches enabled long term strategic outcomes and change was damagingly expensive. now, a clear distinction is emerging between core and support processes. support processes are those which every company requires and are not used for competition. for example, most companies do not compete on the effectiveness of their payroll system. core processes implement the business strategies and provide the differentiators on which the corporations survival depends. support processes, along with networking and hardware, are following the process of commoditisation. this distinction is reflected in ongoing discussion as to the future of it. some papers, notably nicholas carr in an article in the harvard business review entitled “it doesn’t matter”, have generalised commoditisation occurring in support processes and drawn 4 crucially for us, how can we as is researchers make a relevant practice based contribution? zmud, r. (1998). "editors comments." mis quarterly 28 no. 3: pp. xxxix-xxxii. ajis vol 13, no. 1 september 2005 47 analogies to railroad and electricity utilities, claiming that it is now not possible to gain competitive advantage from it.(carr 2003). other papers argue strongly that “the core functions of it is process processing. this stance encourages business leaders to finally execute strategy innovations using agile, unique, proprietary (to them) and technologyenabled business processes that harness commoditised infrastructural transport and processing technologies.”(smith h. 2003) stark decisions must be made which can directly lead to the success or failure of corporations. the gartner group in the book “the new cio leader”(broadbent m. 2003) argues that chief information officers will follow one of two paths based on these perspectives. one path leads to the static role of chief technology mechanic” the other to the role of new cio leader. a chief technology mechanic can cut costs and ‘hold the fort’, a new cio leader can drive business effectiveness and efficiency measures, and in breaking-away enterprises identify higher risk, long-term projects that offer possibilities for significant future competitive advantage. what is business process management? howard smith and peter fingar in their book, business process managementthe third wave, provide a succinct definition of the scope: “not only does business process management encompass the discovery, design, deployment, execution, interaction, control, optimisation, and analysis of processes, but also the executive, administrative and supervisory control over them to ensure they remain compliant with business objectives for the delight of the customers. processes are the main intellectual property and competitive differentiator manifest in all business activity.”(smith h 2003) bpm 5 (management) morphed out of bpm (modelling). initially, corporations modelled their business processes as part of business process re-engineering.(hammer and champy 1993). this initial use of modelling failed for reasons identified by the gartner group as including: the effort required, poor prior experience of computer aided software engineering (case) tools, lack of time, difficulty mapping models to software artefacts, and the rate of business change causing models to fall out of synch.(gartner 1997). two approaches grew out of this initial failure. the ‘lifecycle’ approach re-visited modelling with the aim of defining a more effective software development lifecycle and artefacts which would be effective in large systems building. the ‘executable’ approach saw modelling morphed to management as the realisation came that the models could be executable. models became executable and user interfaces, rather than being separate, became automatically generated from and by the process. executable business processes and process data become the key artefact and are defined, managed, maintained and executed either automatically or manually. both approaches have only recently become possible as the weaknesses in existing lifecycles became apparent, business process conceptual understanding evolved, technologies have matured and higher abstraction levels have become possible. theoretical underpinnings including pi calculus have arisen out of over twenty years of computer science research(milner), standards are being proposed by the business process 5 an unfortunate choice of acronyms! ajis vol 13, no. 1 september 2005 48 management initiative organisation (bpmiorg) and object modelling group (omg), and applied r & d is resulting in commercial products. the framework the framework guides us in the difficult task of picking winners for our bps research. it covers three areas: • choosing research. • attracting industry. • managing our research effort. the framework consists of a set of statements. it is not a prescriptive document nor is it complete. rather, it is a focus for ongoing discussion and iterative development. statements are not set in concrete, if there is a competing consistent assertion with sufficient countering evidence then this will be adopted. the framework is not complete, it is a philosophy on how to conduct research in this area, it can be added to and changed as we make mistakes and learn, not a detailed definition of how this research will be conducted. choosing research areas. concentrate on business domain solutions business strategy is about creating, re-creating and sustaining organizational forms that will enable a process of strategic response appropriate to life on the edge of chaos. what is important is the ability to create and effect new structures and processes quickly and at low cost.(sauer 2003) traditional software engineering processes promote either agile or ceremonial development lifecycles in the technical domain. almost all such processes encourage further requirements elicitation during technical implementation. the transformations between the business and technical domain cause unavoidable inefficiency and risk; and no business benefit. these processes are a historical legacy of a technology push approach to systems implementation. they are fundamentally flawed because they work in the wrong domain. a business process paradigm demands that the solution be expressed in the business domain, not the technical domain. lifecycle approaches can be thought of as choosing a magic happens reverse engineer technical business problem try to fix it here!!! ajis vol 13, no. 1 september 2005 49 business process oriented approach, as opposed to object oriented or procedural. this creates different development artefacts, modelled at a higher level of abstraction and still requiring final technical implementation. such approaches potentially provide immediate tangible benefits due to the new artefacts, plus an incremental pathway to later adoption of full executable systems if so desired. executable systems themselves promise an order of magnitude increase in productivity by largely avoiding the technical domain. enterprise is wary of executable systems as they have no proven track record and it is hard to distinguish capability from marketing at this early stage. improved business process understanding, maturity in systems frameworks and industry resolve promise success. statement: we will research the evolving business domain approaches as they promise the greatest benefit. traditional software engineering approaches will only be used for comparison. concentrate on executable process oriented solutions. there are two business domain approaches. development can be seen as the logical extension of existing coding languages, or as a totally new ‘process oriented’ language with its own conceptually different syntax and semantics running on a process execution engine. figure 1. comparing process engine to coding approaches a process engine executing a process oriented language allows separation of domain versus system implementation. technology no longer drives development. the process engine approach renders technology subservient and relegates it to being the glue binding autonomous and legacy systems. in contrast, a coding approach expresses business requirements in the technical domain and drives technical complexity throughout the solution. process oriented approaches provide a compelling argument that they will be more productive, maintainable and agile than ‘coding’ solutions. statement: our research will focus on process--oriented business domain systems. other approaches will form the basis for comparative studies. apply theory to understand complexity process oriented approaches have a strong unifying paradigm. the paradigm underpins the development of execution engines, process query languages, process activity monitors, and supporting case tools. this paradigm is based upon pi calculus and petri nets. it finds expression in the underlying software engineering meta-models and system executable process engine approach coding approach ajis vol 13, no. 1 september 2005 50 architectures. understanding this paradigm provides the analytic lever to express and understand not only process oriented systems but also emerging web services and other standards. statement: we will manage complexity by evolving a consistent paradigm; which will guide our analysis. attracting industry participation “in theory, there is no difference between theory and practice, in practice, there is” 6 target national strategic objectives the australian government has identified areas of national strategic importance which includes supply chains and health. strategic alliances in these application areas are required as we cannot address them by ourselves. the alliance must demonstrate a strong understanding of application area theory and issues. our role in the alliance is to provide the theoretical horsepower and pragmatic capability to reason about and realise these applications. statement: targeting national strategic objectives will stimulate research and make the decision making process by grant authorities as simple and attractive as possible. identify suitable industries there are three types of industry involvement. each can be addressed by different partners in the alliance: • application industries. industry partners from customer systems. examples being transport companies, health care, or major manufacturers • solution seeking industries. more localised industry partners with a specific, usually decision based, problem. for example, trying to solve transport fleet allocation and res-scheduling. • process and lifecycle. software engineering and implementation companies. product vendors. statement: we will work with our alliance to put together a complementary set of industry partners. provide tangible outcomes industry partners want results, not just blue sky theories. we will provide the pragmatic software engineering capability to implement, deploy and test such systems. statement: we will provide tangible deliverables based on sound theory 6 jan l.a. van de snepscheut. dutch-american computer scientist and educator (1953 1994) ajis vol 13, no. 1 september 2005 51 structuring our research effort. this research is centred on the primacy of the business process model. we assert that the model has primacy. the model is defined as: “a description of the system which captures domain specific requirements expressed within an executable meta-model” the model is built by taking a consistent view of many issues including enterprise strategic direction, stakeholder intent, risk management and maximisation of efficiency. simulations can be applied to scope the process and resources required. the system is implemented from the model. the figure below shows the relationship between the model and the research areas. emphasis on the model is intended to decouple research into ‘bite sized’ chunks and avoid fragmented many-to-many research relationships. • the ‘as is’ model represents the business processes currently performed by the enterprise. in our case, it must be explicit and captured using some formal representation. often it is implicit in the organisation and not well understood by management. it must be useable to be effective, implying it must present different views to technologists, business analysts, and other roles. these views must all be consistent and add value to the underlying model. • the ‘as desired’ model represents the target set of business processes which the company intends to migrate to. • “aspect oriented synthesis’ refers to the varying means employed to capture strategic intent and simulate systems in order to specify the ‘as desired’ model. considerations include enterprise strategic direction, stakeholder intent, risk management and maximisation of efficiency. all these considerations and potentially conflicting viewpoints must be synthesised in order to specify a single consistent ‘as desired’ model. • ‘enabling technology’ is the underlying technical infrastructure which specification aspect oriented synthesis enabling technology ‘as is’ model ‘as desired’ model implementatio migration useability monitoring simulation and intent ajis vol 13, no. 1 september 2005 52 implements the model. this infrastructure must be capable of executing the model based on the syntax and semantics of the meta-model. the enabling technology may consist of both manual and automated steps. a key characteristic is that only non-functional requirement information can be added at this point. statement: research must relate to the model, its implementation and its metrics. take a team approach there is a prevailing research culture in many australian universities. the culture insists on isolated, individual work; particularly when working towards a phd. this individuality arises from the mathematical history of algorithmic computer science research. unlike many academic areas, research in business process management systems is a software engineering problem. this means • a breadth of understanding across many academic research areas is required • theory must be applied to be understood. applied theory then acts as a concrete foundation for further research. • human factors are paramount. • look for leverage, do not re-invent the wheel. this paper rejects this individualist culture. bps is a software engineering problem, requiring: both breadth and depth of talent. this breadth combined with limited resources means we must choose research topics wisely so it benefits both the individual and the group. we will take a systematic approach to defining research areas and for each area we will have a thought leader. areas interact so synergy is achieved, and are constrained to critical issues of general impact, thus guaranteeing active and committed input from the team. an area leader is responsible for academic leadership and exploring inter-area synergy. publications are expected to be co-authored. from a phd standpoint, this raises the issue of how to assess academic contribution in an environment where the thought leaders ideas in an area may undergo significant revision or indeed lead to novel contributions from within the team. some universities, such as the australian national university, have alternate means of defining phd’s. initial indications are that satisfactory metrics could be determined based on • acknowledgment and attribution of thought leadership by the team and the resulting publications. • contributing effort to other papers in a secondary role. • industry involvement and case studies. statement: we will work in a collegial environment on strategic areas. . an area leader is responsible for academic leadership but is expected to welcome contributions and insights from other members. there is synergy between areas, and we expect robust contribution both from within and across areas. this will lead to many co-authored papers. new avenues to define a phd will be explored based on existing overseas and australian approaches. ajis vol 13, no. 1 september 2005 53 accept we are naive initially, researchers in this area will be naïve, traversing a steep learning curve, and exploring many wrong paths. it is important to have the freedom to move quickly and adopt and discard approaches rapidly. learning progresses best by taking standpoints and testing ideas. decisions are forced by milestones on pragmatic projects. this ensures theory and understanding become a concrete foundation for later research. in the initial phase, papers should be written only for compliance purposes and as an exercise. it is likely many papers will later be contradicted as mistakes are discovered. later research will be built on the foundation of strong applied knowledge of theory and pitfalls. this research gets us to the leading edge with the tools and experience required to pragmatically structure approaches, assess significance, and understand external contributions. at this point, papers should be significant, publishable in leading journals, and of interest to practitioners. statement: we are beginners before we are experts. we create integrated standpoints to be tested by pragmatic application and found wanting. we make progress via our mistakes. papers are the result of our work, not the objective. form research alliances we require an alliance between process research, decision systems and the application domain. • process systems tell us when to take an action or decision, what information exists at this point, and the flow. it acts as the systems integration. it does not tell us how to make a decision. • decision support systems tell us how to make choices, apply rules, and negotiate optimal outcomes amongst independent agents within the context of process. • application domain. to evaluate and exercise process and decision systems we need an application domains. the application domain is describable by concepts in the other domains.. statement: we will form alliances with complementary research groups and will lead the process and systems integration effort. call in expertise. process provides the context for specialist areas such as security and legacy system integration. when required, we will delegate to the specialist areas rather than become specialists ourselves. statement: there exist subsets in the task-oriented area but outside our area of expertise. call in this expertise and work in tight collaboration. leverage off standards without being paralysed there are many competing and evolving standards in this area. there is a high risk of suffering ‘analysis paralysis’ if we try to correlate standards and understand competing but similar approaches. we will make progress by setting our own objectives and building our own view of how such standards should operate. we can use this knowledge to examine ajis vol 13, no. 1 september 2005 54 various standards and determine both their applicability and their ability to form consistent and complete sets. we will apply them pragmatically to real domain problems. this will give us experience to get a handle on which aspects of standards are important plus their practical human factors and technical issues. statement: the diverse and conflicting range of standards will be managed by a requirements driven approach. we will set our own requirements based on our objectives and match to standards as opposed to an exhaustive understanding of all possible current standards. contribute to an open source project. targeting an open source project provides business, technical, and mentoring benefits. it allows us to make pragmatic progress whilst still referencing emerging standards where possible. • business benefits. there will be a commercial battle in the bps space. open source has a different group of supporters and has the potential to form an enduring sizeable niche for business development without getting crushed. • technical benefits. participating in development gives insights into how these systems really work and a framework from which we can develop extensions. concrete experience will help us define meta-models helping us to understand and evolve standards. • peer review. it will provide active external review and mentoring by people to whom pragmatic outcomes matter. statement: our strategic aim is to be a leading research and development group within the open source community. we will acknowledge and understand other approaches, but will not allow ourselves to be distracted. experiments are required experiments must be conducted for three reasons. • no system or process is useable unless it supports human factors issues and can cope with the fact that enterprise system development is a ‘wicked problem’ which cannot be resolved with traditional analytical approaches (rittel and webber 1973). • experiments force researchers to make pragmatic decisions. these decisions mitigate a tendency to ‘analysis paralysis’ due to complexity and divergence in this area. these decisions then provide concrete foundations for further research. • pragmatic outcomes demonstrate competency and provide a direct link to industry. wicked problems cannot be resolved with traditional analytical approaches (rittel and webber 1973). they consist of a set of interlocking issues and constraints that change over time, embedded in a dynamic social context. (poppendieck) attempting to baseline requirements and then use an analytical approach to reach a solution is a recipe for disaster with wicked problems. these problems are resolved through discussion, consensus, iterations, and accepting change as a normal part of the process. to evaluate the new ajis vol 13, no. 1 september 2005 55 approaches we need to be able to test the whole development process from stakeholder negotiation, through project planning and contracts to final implemented system. pragmatic decisions. there is significant risk of getting into ‘analysis paralysis’ in this area due to complexity and lack of consistency. by forcing test implementations, researchers will be compelled to make decisions and then experience the results. this will stimulate researchers to propose consistent and viable solutions and also provide a concrete foundation for further research. this foundation will then act as a base to compare competing approaches and a lever to aid understanding. statement: we will work in iterations of theory and practice. theory and research determines our next step, practice and experience clarify and make concrete our concepts a research laboratory is required. this document assumes the existence of a software engineering laboratory. this laboratory must act as a ‘process container’ to allow different approaches to be efficiently tested and studied. all elements common across projects must be extracted and standardised to ensure we do not waste effort on unimportant issues. the laboratory must also provide support for ongoing generation and retention of intellectual property and provide tools to assist teamwork. statement: progress requires practice based insights and demonstrated competency to test theory. to achieve this we require a properly designed research laboratory.7 results to date in the twelve months since this strategy was proposed the following has occurred: • the software effective process group has been formed and research begun. the group consists of a systems architect plus specialists in middleware, query languages, testing, and decision systems. supporting this team are four honours students plus one third year student project group. this gives a total of fourteen people working on interlinked research. • an alliance is established with the decision systems laboratory www.dsl.uow.edu • an alliance is established with the supply chain research centre at the university of wollongong. • a research laboratory has been established. this laboratory acts as a ‘software process’ container and allows target processes to be researched and tested. tools are also available for knowledge management and teamwork. • research laboratory commissioned and is now investigating jbpm. more information is available at www.dlab.uow.edu.au 7 see the working paper on dynamic lab. the pilot of dlab exists and is currently evaluating processes. ajis vol 13, no. 1 september 2005 56 • industry partners have been identified and linkage grants are currently under negotiation. • a series of papers outlining the strategic direction, research areas, research synergy and supporting dynamic laboratory infrastructure have been written. these are available for comment on www.dlab.uow.edu.au • a major australian project is utilising a new software development lifecycle for ‘system of systems’ development which has arisen out of this research strategy. later papers will report on both the theoretical basis and practical impacts of this work and other research by the group. there are a number of ongoing challenges. firstly, a funding key performance indicator catch-22 exists where, in order to be funded, a group must have a track record. given that the software effective process group is a start up, this situation is not satisfied. there is long term strategic funding available, but pursuing this is a high risk strategy in itself. secondly involves changing the academic culture. this culture supports individual academic research and impedes team based pragmatic collaboration. this paper proposed an alternate viewpoint to the benefit of both computer scientists and software engineers. the third challenge was somewhat unexpected. members of the software effective process group have become in such demand in industry that financial imperatives have led to some members, including this author, joining industry. every challenge is an opportunity. provided academic support can be maintained, having some of our team actually in industry presents tremendous leverage for action research and case studies. conclusion this paper presented a framework to conduct software engineering research into process oriented systems within an australian university. the framework provides a philosophy on how one should conduct such research; recognises the importance of theory and practice; and highlights alliances between faculties, across universities and with industry. such a framework is important as it provides an antidote for the short term, operational approach currently encouraged by australian universities key performance indicators. the direction provided by the framework is proving effective and results are already evident. significant results will take time, agility is required and much will be learnt through experience. discussion is welcomed on any aspect of this paper. references broadbent m., m. k., mcdonald, m. (2003). "it success requires discipline and innovation." gartner ft-20-0693. carr, n. g. (2003). "it doesn't matter." harvard business review(may). gartner (1997). nine reasons why is organisations do not do bpm. hammer, m. and j. champy (1993). reengineering the corporation, harper business, new york, ny. milner, r. theoretical foundations of mobile distributed processes and the pi calculus. poppendieck, m. "wicked problems." software development magazine(may 2002). rittel and webber (1973). "dilemmas in a general theory of planning." policy sciences vol. 4: pp 155-169. ajis vol 13, no. 1 september 2005 57 sauer, w. (2003). "establishing the business of the future: the role of organizational architecture and information technologies." european management journal 21(4): 497-508. smith h, f. p. (2003). business process management the third wave. smith h., f. p. (2003). it doesn't matter business processes do, meghan-kiffer press. smith, p. f. a. h. (2005). "the next generation." bptrends. w.m.p van der aalst, a. h. m. t. h., b. kiepuszewski, and a.p. barros "workflow patterns." zmud, r. (1998). "editors comments." mis quarterly 28 no. 3: pp. xxxix-xxxii. microsoft word ajisfinal 12 2 marina.doc ajis volume 12 no. 2 may 2005 134 a tale of two projects: why it projects fail (and why they sometimes actually succeed) david wilton institute of information and mathematical sciences massey university albany, new zealand email: d.r.wilton@massey.ac.nz abstract this case describes two similar outsourced is projects that took place in wellington, new zealand (nz) during the period 1992-99, with emphasis on the reasons why one project failed (the prime contractor repudiated the contract and the project was abandoned) and why the other succeeded. the same prime contractor was used for both projects and there were a number of other similarities, yet the outcomes were totally different. the case provides lessons in some of the fundamental aspects of is/it project management, including scope, time, risk, human resources, procurement and quality management. keywords information systems, project management introduction this case describes two similar outsourced is projects that took place in wellington, nz during the period 1992-99, with emphasis on the reasons why one project failed while the other succeeded. when attempting to compare two projects, a difficulty frequently encountered is the potentially large number of variables. these include: • location (including physical/geographic differences, national and local political and legal environments). • size ($ value) and scope of project. • type of organization (including mission and culture). • timeframe (in particular, technology available at that time). • acquisition strategy (outsourced or in-house; bespoke development or off-theshelf). • prime contractor (if outsourced). in the case, the organisations were very similar (both uniformed military or quasi-military federal government organisations), and the location, time period and prime contractor were identical. the budgets of the projects were of the same order of magnitude, and both were outsourced. this allows meaningful comparison of the projects, in terms of project management and acquisition strategies, particularly as one failed and the other was successful. the two projects that are the subject of this case are the nz defence force (nzdf) project fusion1 and the nz police project incis2. schwalbe (2000) lists the following key project management functions: scope, time, quality, human resources (hr), cost, risk, communication, and procurement management. the two projects will be described using these factors as a framework. 1 not an acronym. the name was chosen to symbolise the integration of key management information systems. 2 integrated national crime information system ajis volume 12 no. 2 may 2005 135 learning objectives the case provides lessons in some of the fundamental aspects of is/it project management; including scope, time, risk, human resources, procurement and quality management. it also provides an opportunity for students to apply is theory to the analysis and interpretation of a “real world” it situation. although the projects described in the case were set in a national government is context in a small southwest pacific nation, the lessons are believed to be applicable to a global is management context. target audience this case has been used successfully with second year undergraduate students undertaking both business and is majors, and with postgraduate is majors, to illustrate good and bad points of is/it project management. the similarities between the two projects, yet the totally different outcomes, provide telling lessons. project fusion organisational background nzdf nzdf comprises some 9,000 regular force and 2,500 territorial force (part-time) service men and women and 1,900 civilian staff across the armed services of navy, army and air force. staff are located in many different parts of nz with significant contingents serving in many different locations world-wide. nzdf has an operational history extending from service in the south african war over 100 years ago to the present. peace support operations have been a major element of operations undertaken in recent years. nzdf personnel are currently serving in 18 missions around the world, including afghanistan and the gulf of oman in support of operation enduring freedom, the global war against terrorism. the corporate headquarters is located in the national capital, wellington. most major it facilities (including servers for corporate applications) are also located in wellington. setting the stage nzdf have historically used telecommunications services to maintain voice and messaging services to, and within, mobile elements, including ships at sea, aircraft in flight, and ground forces deployed into field locations. since around 1980, mainframe computers have been used to store key corporate administrative data. the advent of the pc and affordable data communications networks in the 1980s facilitated the introduction of a distributed computing environment, allowing corporate data access from desktops in normal peacetime locations in nz and by force elements deployed into the field or overseas. prior to 1988, the various it elements of nzdf (telecommunications, administrative and operational computing) were regarded and managed as separate entities. at this stage, also, the emphasis was on in-house development of corporate is applications. following the merging of key it functions and responsibilities in the late 1980s, various actions were undertaken to formulate enterprise-wide policies and standards. the next significant step in rationalising it on an enterprise basis was an is strategy study, conducted in 1991. two of the key findings of this study were: • preference should be given to acquiring major is applications “off-the-shelf” ajis volume 12 no. 2 may 2005 136 rather than by in-house development. (in-house development could be used, however, on a case-by-case basis, if proved to be beneficial by a sound business case.) • the financial accounting system was at that stage outsourced to a bureau (in modern parlance, an application service provider – asp). costs were calculated on a per-transaction basis and had been seriously underestimated. the excessively high cost of this arrangement meant that the system needed to be brought back inhouse as soon as possible3. also, the linkages with the inventory management system needed to be improved. at that stage the in-house developed inventory management system had no financial visibility of assets once they were purchased and placed in inventory. this was contrary to the mode b accrual accounting policy that the government had introduced as part of the financial management reform program in the late 1980s. in early 1992, a small project team was formed and a project to rationalise the problems with the financial accounting and inventory management systems commenced. discussion point 1: nzdf is strategy expressed a preference for acquiring major applications “off-the-shelf” rather than by in-house development. what are the potential advantages and disadvantages of these two approaches? brief project history as outlined above, during 1991, nzdf completed an enterprise-wide is strategy study. one of the key recommendations was that the (then) outsourced financial accounting system needed to be brought back in-house and that the linkages with the inventory management system needed to be improved. project fusion commenced in 1992; key objectives were: • replacing the existing: o outsourced financial accounting system (due to excessive operating costs) o inventory management system (due to no financial visibility of assets once purchased) • with an integrated system. • the new system was to have potential for integration of other corporate functions later (including hr, fm, engineering). a brief chronology of the project is as follows: • registration of interest (roi) and request for proposals (rfp) were issued in 1992. • ibm (nz) was appointed prime contractor in late 1993. the system chosen was the enterprise resource planning (erp) integrated application sap/r3. • the finance system went live mid-1995. • the inventory management system went live in 1997. discussion point 2: the terms roi, rfi, rfp and rft tend to be used interchangeably with respect to procurement. what do they all mean and what is the difference between them? the project required extensive upgrade of existing data communications network and deployment of new facilities to defence establishments nz-wide and many overseas locations. the project was on budget, slightly over time of the order of 12 months. this was mainly due to the fact that the 3 the possibility of another, cheaper, asp arrangement was investigated; however, nothing was available at that time that provided the required functionality at a reasonable cost ajis volume 12 no. 2 may 2005 137 sap/r3 inventory management system did not have a close fit with business requirements and significant business process engineering was required. project scope project scope relates to defining and managing all the work involved in creating the deliverables required of the project and the processes used to create them. project stakeholders need to come to an agreement early on as to the scope of the project; that is, what the project will actually encompass and the resultant deliverables. the most common encapsulation of the agreed scope is in the project charter, or plan, preparation and agreement of which should be an early step in the project. in project fusion, while it was recognised in the strategic plan that the organisation should head towards an integrated erp solution for its administrative is, a decision was made to limit the deliverables from this project to financial accounting and inventory management modules. the main reason for this was that these two modules were the main business imperative, but there were also project implementation and management factors behind the decision. procurement management there are two possible approaches to acquiring or implementing a new information system: in-house development or by outsourcing (that is, contracting an external party to either develop or supply the required functionality “off-the-shelf”). mixtures of these options (in-house/outsourced and developed/off-the-shelf) are also possible. if the outsourcing approach is chosen, in whole or in part, the project will then involve the significant processes of contractor selection, contract negotiation and ongoing contract management. in both the projects under review, a decision was made to outsource. in project fusion, the desired procurement strategy was to obtain a system that was off-the-shelf as much as possible (i.e. minimal tailoring). an rfp was issued, evaluated and a preferred prime contractor selected. a contract was negotiated and signed, and successfully managed over the duration of the project. it is not possible to describe these processes in detail; however, a couple of noteworthy features were the decision to appoint a prime contractor, in a systems integration role, and the use of a skilled contract negotiator during the negotiations phase. while there were some differences between the parties during the project, all issues were subjected to discussion and the disputes procedure specified in the contract and all were resolved in an amicable fashion. time scope, time and cost are frequently cited as the key variables for judging the success (or otherwise) of a project. it is fairly obvious that stakeholders will require a project to be completed in a reasonable time frame, even if there are no critical deadlines resulting from legal, financial or external requirements. modern computer-based project management tools usually provide an automated time scheduling capability, often coupled with resource management functionality. in both projects, the original scheduled timeframes were exceeded. in the case of fusion, that was mainly because of the need to undergo business process re-engineering due to the fact that the offthe-shelf functionality of the inventory management system didn’t completely fit the business requirements. this caused a delay of approximately 12 months, but that didn’t create any major business problems. the use of automated scheduling software allowed close tracking of tasks, and the time slippage was predictable in advance and managed appropriately. ajis volume 12 no. 2 may 2005 138 quality project fusion established a quality assurance (qa) plan and a separate qa review team to oversee process quality, and to ensure that the prime contractor satisfactorily addressed product quality (as the system as off-the-shelf, this needed to be the prime contractor’s responsibility). the qa review team reported to the business sponsors, which ensured its impartiality and independence from the project manager. hr the importance of recruiting and retaining suitable staff is well known. phrases such as “people are our most important asset” have almost become clichés. within a project, there are many different skill sets, individuals and groups required; from within the organisation, contractors and third parties such as legal and technical advisers. within project fusion, the recruitment and management of project staff is rated as adequate. as the system was largely off-the-shelf, development staff didn’t pose significant issues. hr problem areas included: • initial difficulties in selecting a long-term nzdf project director, and the turnover (three individuals) during the duration of the project, • difficulties getting sufficient staff with appropriate business domain knowledge to undertake evaluation and testing, • turnover of prime contractor staff, including the key position of contractor pm. organisation culture aspects and attitudes to the project will be considered separately under communication. one aspect of project hr that perhaps deserves to be separated into a category of its own is that of project governance. this involves the organisational structure and processes established to undertake decision-making throughout the project. a large number of decisions will be required in any project: ranging from minor (normally required on a day-to-day basis) to major (usually on an infrequent basis). in nzdf, the implementation of the overall is strategy was the responsibility of a high-level is steering group, which was charged with making recommendations on key issues (such as recommended prime contractor and major financial approvals) to the chief executive for approval. within fusion, there was a project executive board, consisting of key stakeholders, responsible for oversight and other major decisions, and a project management committee, chaired by the nzdf project director, responsible for routine, day-to-day decisions. (the qa review team, as noted previously, was separate from any of these groups.) there was a matrix of key decisions and documents for review, detailing who was required to comment and what the final approval level was. cost project cost is fairly obviously a major factor in any project, which implies the criticality of management of all resources, including financial (almost all other resources can be viewed as having a value expressible in monetary terms, and therefore should be regarded and managed with the same degree of importance as money). although financial aspects of a project are important, they are not considered to be of critical issue in these cases and will not be described in detail. the total cost of project fusion was of the order of nz$15m. this was within, or very close to, budget. ajis volume 12 no. 2 may 2005 139 risk projects, by their very nature, invariably take place under conditions of uncertainty. this leads to a situation where identifying and countering risk is important. within project fusion, a risk assessment was carried out early in the project, and possible risks were identified. these were rated as relatively minor (and could therefore be managed) or could be mitigated with fairly modest counter-measures. the assessment was revalidated periodically throughout the project. communication communication is often considered to be the most critical factor impacting on the success, or otherwise, of a project, as it involves winning approval from all stakeholders, whether they be internal or external to the organisation, senior management or low-level system users. the broader process of gaining stakeholder approval and actually implementing the new system is usually termed change management. in these two cases, the terms will be used synonymously, although change management is usually considered to encompass a wider scope. within project fusion, the need for a sound communications plan was identified early on in project planning. the main targets for communication were identified as senior management, who would make decisions, and the “users” of the new system, who would include not only data entrytype users, but also managers at most levels of the organisation who would use reports and ad hoc queries from the system. the communications plan was to inform senior management through the project governance arrangements (project executive board, decision matrix, document review matrix) and to inform users by means of a regular project news bulletin, which became known as fusenews. this was produced periodically (at least monthly, or more often if there was significant development to report), initially in hard copy form but later distributed optionally by email. by the time the finance and inventory modules went live, all stakeholders were well informed, no significant cultural impediments were encountered and the implementations went relatively smoothly. current problems/challenges facing the organisation the nzdf has gained widespread internal acceptance of the deliverables from project fusion. at least two more modules of the erp package (facilities management and plant maintenance) have been implemented and consideration is being given to others, such as hr and payroll. ongoing challenges include the implementation of higher-level information systems, such as decision support and executive support systems to assist decision-making by higher levels of management, and to integrate information provided from disparate (external and internal) sources and systems. project incis organisational background nz police nz police is the single nation-wide law enforcement agency responsible for enforcing criminal law in nz. there is currently 8,800 staff (uniformed and civilian), operating from more than 400 community-based police stations throughout nz. police provide services 24 hours a day, 365 days a year. emergency response is a major component ajis volume 12 no. 2 may 2005 140 of police operations, and police respond to more than 1 million “111” calls each year. police are also involved in crime prevention as well as responding to crime. the corporate headquarters is located in the national capital, wellington. most major it facilities (including servers for corporate applications) are also located in wellington. setting the stage like the nzdf, nz police have used it extensively throughout their history; initially using telecommunications services to maintain voice communications with mobile elements, and more recently using corporate and desktop computing services to store and access administrative data. operational data is now also stored on corporate servers and desktop computers. the advent of the pc and affordable data communications networks in the 1980s facilitated the introduction of a distributed computing environment, allowing access to corporate data from desktops and by mobile force elements. the origins of the incis project go back to 1985. considerable work was done to develop the incis vision and concept. the incis concept and vision was consistent and integrated with the police five-year strategic plan published in 1993 which was developed around community oriented policing (cop). another factor in the gestation of the project was that, by the early 1990s, the thencurrent major database of criminal enforcement and court records, located in wanganui (approximately 150 km from wellington), was technologically obsolescent and in need of replacement. in 1993, a business case was prepared that outlined the objectives of the project, and attempted to justify the cost on the basis that the new system would reduce paperwork, thereby freeing up more effort for front-line police work. it was estimated that the new system could save up to 540 staff, as the result reduced administrative tasks. this would ultimately result in savings for the taxpayer. brief project history in 1990 and 1992 nz police issued requests for information (rfi) that described the purpose of incis as: … to support operational policing by providing improved information, investigation and analysis capabilities. this will contribute to the police mission statement of minimising the incidence and effects of crime on the community through the detection and apprehension of offenders and by crime prevention strategies. the proposed business functions to be performed by the system were: management of criminal information, case and investigation management, crime trend analysis, intelligence analysis, performance measurement, and special operations. a brief chronology of the project is as follows: • a request for tender (rft) was issued in december 1992. by april 1993, tenders had been evaluated and a prime contractor recommended. • the business case was presented to cabinet4 in august 1993. discussion point 3: what is the purpose of a business case and why would nz police have one at this stage? • significant revision of the business case in 1994. • ibm (nz) was appointed prime contactor in september 1994. the contract called 4 government executive body responsible for major funding decisions ajis volume 12 no. 2 may 2005 141 for bespoke development of applications to provide the required business functionality. • ibm repudiated the contract in august 1999. • in late 1999, the project was cancelled. a state services commission inquiry was held into failure. the subsequent report was widely publicized (small, 2000). project scope the initial scope of incis was deliberately very broad, encompassing all aspects of operational policing. small (2000 23) discusses the strategic objective as follows: “the overall strategic goal of ‘reducing the incidence and effects of crime’ transcends traditional crime prevention and seeks to stem and reverse crime trends”, and goes on to note: “this is an ambitious goal and will not be achieved without significant changes in policies, processes and attitudes within the new zealand police.” small (2000 27) reports that the scope changed significantly during the life of the project: “during its lifetime the project underwent significant changes in emphasis. its focus changed from police strategy to financial objectives and then to a technology project”. discussion point 4: what is project scope creep and why is it regarded as a problem? procurement management within project incis, it was acknowledged that the scope of the project was so broad and the requirement so novel that an off-the-shelf system didn’t exist at that time. a bespoke development contract was therefore sought. an rfp was issued and evaluated and a contract negotiated and signed (coincidentally, with the same prime contractor as for project fusion). small (2000 190) notes many difficulties with the execution and management of this contract: “in the initial stages of the contract there was a partnering arrangement with police. the inquiry believes, whilst a partnering arrangement is acceptable in some circumstances, in this case the partnering arrangement was unlikely to work because the parties had different objectives, they were not ready to enter into contract, there were difficulties in relation to the fixed price nature of the contract, and other factors.” after numerous disputes and much acrimony, the prime contractor eventually repudiated the contract, as previously noted. time the project was approximately two years past deadline when it was cancelled. quality there was no adequate qa regime within project incis. small (2000 161) states: “the incis project should have had comprehensive iqa [independent qa] as part of a risk management process. in fact, the project had no iqa. the failure to have comprehensive iqa was a material factor in significant risks not being identified and acted upon in the manner they should have been.” ajis volume 12 no. 2 may 2005 142 hr there were significant difficulties with hr in project incis. according to small (2000): “neither the police nor ibm, at that time, had the resources, skills and experience needed to successfully deliver the incis application.” (p.163) … “the turnover of personnel within ibm was excessive and the location of work outside of new zealand had significant risks.” (p. 190) organisation culture aspects and attitudes to the project will be considered separately under communication. in project incis, the governance structure was initially fairly standard (see small (2000) chapter 6 diagram 7). small (2000) states: “the structure for governance and management of the incis project shown in the business case was in accord with generally accepted practice.” (p. 84). however, staffing turbulence and personal conflicts eroded the quality of project governance over time: “the structure adopted by police in 1995 departed from generally accepted practice in that … the project director was reporting to the director of i&t [information and technical] and not to the project sponsor.” (p. 85) … “the above … factors led to a relationship of serious conflict between the director of i&t and the project director. … the relationship issues were well known to police governance. the commissioner and the sponsor spoke to the director of i&t and the project director about the relationship on a number of occasions but without any satisfactory result. the relationship issues had a negative effect on the incis project team. the police governance and management of the project became to some degree dysfunctional.” (pp. 99-100) discussion point 5: if you were the project sponsor and you became aware of staffing conflicts as described above, what could (or should) you do about them? what are the implications for the project and the organisation as a whole? discuss. cost for project incis, of the order of nz$130m was spent up until the time of cancellation. a significant portion of this was overspend, relative to an original budget of around nz$90m. risk within project incis, the situation regarding risk is described by small (2000): “the incis project was unique at the time. … the inquiry believes the fact that there were no existing packages that would meet the incis business specification should have been an early warning of the risks involved. the project was made more complex and with a higher risk by the decision to develop the incis application. …the inquiry believes that, although the project was of high risk, in 1993-94 the concept or vision of incis was sound …” (p. 30) “the business case for incis asked what are the key risks to manage? it then addressed risk management by stating that police culture and work processes must continue to evolve; that police were better prepared than ever before to undertake such a challenge; that the most critical element to success of the project was to ensure that police realised the benefits; and that a key part of realising benefits would be assumed by bpr. the right question was asked but the answer given was inadequate, vague and misdirected.” (p. 119) ajis volume 12 no. 2 may 2005 143 communication within project incis, the business case stated a benefit of reducing the amount of time spent on administration by front-line police officers, thereby making more time available for operational duties and having the net effect of saving around 540 staff (6% of the workforce at that time). this claim became a political issue and was often hotly debated in the national parliament in relation to annual appropriations and expenditure by nz police, especially when the project began slipping. the project, particularly in relation to its time and budget overruns, was widely criticised in the news media and attracted national headlines such as over-budget, overdue and now simply over (broomhall, 1999) and insights into incis debacle (gifford, 2002). internal incis communication was described by small (2000 106) as follows: “police had a … command-line culture which, whilst suitable for line management of the police, was not suitable for project management. … incis was a business or commercial project rather than a policing operation. in the view of the inquiry, a normal business and commercial culture, involving commitment, team-work, and open and frank reporting, would have been more successful than the application of line culture…” as there were few actual deliverables from incis, it is difficult to assess how smoothly implementation may have been, and what stakeholder attitudes would have been at the time of implementation. current problems/challenges facing the organisation in spite of the problems with the project, incis did deliver a number of benefits to nz police. the project delivered a technical infrastructure, including approximately 3,000 desktops, but this technical infrastructure was significantly modified from that specified in the original contract. the project also delivered certain aspects of the incis application (increment one) which was essentially a national intelligence system (nis) replacement. however, the project did not achieve its intended business case financial or business benefits (small, 2000 57). a separate project had to be undertaken to carry out the necessary updates to the wanganui computer and its applications. there has been no similar (to incis) project attempted, and nz police still face the challenge of automating many operational policing functions. in time, at least some applications may possibly become available off-the-shelf. case questions 1. the uk office of government commerce document why it projects fail (uk office of government commerce, 2002) lists the following factors that may lead to project failure: design and definition failures, decision making failures, project discipline failures, supplier management failures, people failure. the same document lists the following key questions that should be addressed relating to the project: • does the department and other key stakeholders understand how this project will affect the business? • is the project properly staffed to enable effective leadership, decision-making and risk management to begin from day one and continue consistently to the end? • how will the ways we work with our supplier help this project to succeed? analyse and compare these two projects using the above framework. list and explain the key reasons why one succeeded and the other failed. what could project incis management staff have done better, to ensure a higher probability of success? ajis volume 12 no. 2 may 2005 144 2. within project incis, the business case claimed a benefit of reducing the amount of time spent on administration by front-line police officers, thereby making more time available for operational duties and having the net effect of saving around 540 staff (some 6% of the thenworkforce). how realistic was this? do is projects normally allow large reduction of staff numbers (or save other resources such as finance)? 3. an aside: “core” technology architecture. the incis project technical substitution project plan of 17 march 1995 defined the core architecture as: ‘the components of the "core architecture" are deemed to be technologies which provide the basis for the design of the system technical architecture. that is to say that replacing any of the core architecture components will require the entire system design to be re-addressed. this will have an impact on cost and delivery schedules of incis. the core technologies are not subject to technology substitution. these core architecture components are: • host operating system mvs, • station server operating system os/2 & lan server; • desktop operating system os/2 & lan requestor; • host database db2, • station cabling token ring; and • network protocol – sna.’ comment on the above statement, particularly regarding the appropriateness of choosing these standards. 4. describe what is meant by the term risk in relation to projects. how is risk quantified and how can it be mitigated? describe and comment on the inherent levels of risk of the two projects and how well these were managed. 5. list and briefly discuss the reasons for, and possible advantages and disadvantages of outsourcing. what are the key considerations and requirements for managing a successful outsourcing relationship? how well were the relationships managed in the two projects outlined in this case? how could things have been done better? 6. communication is often considered to be the most critical factor impacting on the success, or otherwise, of a project, and involves winning approval from stakeholders. in general terms, what stakeholders may need to be targeted? what stakeholders were relevant in projects incis and fusion, and why the differences, if any? how successful was communication within the two projects? how well did the projects “fit” the cultures of the respective organisations? 7. describe what is meant by the term quality in relation to projects. describe and comment on the quality assurance regimes of the two projects and how well these were managed. who is (or should be) responsible for quality assurance when a project is outsourced? references broomhall, k. (1999) over-budget, overdue and now simply over, christchurch press, 11 august, christchurch, nz. gifford, a. (2002) insights into the incis debacle, nz herald, 11 june, auckland, nz. schwalbe, k. (2000) information technology project management, thomson learning, cambridge ma. small, f. (2000) ministerial inquiry into incis, url http://www.justice.govt.nz/pubs/reports/2000/incis_rpt/index.html, accessed 20 february 2001. uk office of government commerce (2002) why it projects fail, url http://www.ogc.gov.uk, ajis volume 12 no. 2 may 2005 145 accessed 15 january 2004. appendix 1: extract from project incis inquiry report the key recommendations of the inquiry were as follows (small, 2000 section 1.3): 1. the business case should reflect overall business strategy and should address technology resources and risks as well as financial issues. 2. projects should normally use proven technology. where it is necessary to use unproven technology, this should be reflected in an increased risk management process. 3. it is essential that a project be properly resourced in terms of skilled and experienced governance and management. 4. strategic consideration should be given to the form and nature of the contract. there should usually be separate contracts for infrastructure and applications that should be in modules to give delivery of specific business benefits. 5. the appropriate form of contract should be signed only when the government agency is satisfied that it has addressed and resolved all relevant issues including technology, resources and risks. 6. there must be comprehensive quality and risk management processes, including iqa5, in place. 7. the appointment of key personnel is critical and care needs to be taken in the appointment process to minimise the risk of conflict and, if possible, a top down appointment process is desirable. 8. any conflict or dispute that adversely affects the project must be addressed promptly and in an effective way. 9. to maximise the prospects of success, a project needs to have tight and effective management. 10. the government needs to define structure and role for approval and monitoring of large it projects. 11. the procedure for making application for approval of the cabinet or minister should be strengthened and made more rigorous to ensure that cabinet and ministers receive sound information and appreciate the risks of the project. 12. the approval by cabinet or a minister of a project should contain directions that ensure that the monitoring by the monitoring agencies is as effective as practicable. 13. the approval and monitoring structures need to be resourced to enable them to effectively meet the expectations of government. (note: 10-13 related to government it as a whole rather than specifically to nz police) a statement that alluded to the key difficulties of the project is as follows: “the implementation of the incis project after the signing of the contract was fraught with difficulties that arose from: the fact that the parties had entered the contract before they were ready. the technology and architecture contracted for could not be delivered. there were not sufficiently skilled and experienced personnel. a dispute as to whether the contract was sufficient for design of applications. business process re-engineering (bpr) not being commenced until after contract6. 5 independent quality assurance 6 this, presumably, would not have been a problem unless it was on the critical path, as it was also for nzdf project fusion ajis volume 12 no. 2 may 2005 146 requirement for extensive changes. other factors.” (small, 2000 section 6.9.1) microsoft word ajis10-2finala.doc ajis vol. 10 no. 2 may 2003 134 considering the problem of insider it misuse steven furnell and aung htike phyo network research group, university of plymouth, plymouth, united kingdom email: sfurnell@network-research-group.com abstract in recent years the internet connection has become a frequent point of attack for most organisations. however, the loss due to insider misuse is far greater than the loss due to external abuse. this paper focuses on the problem of insider misuse, the scale of it, and how it has effected the organisations. the paper also discusses why access controls alone cannot be used to address the problem, and proceeds to consider how techniques currently associated with intrusion detection systems can potentially be applied for insider misuse detection. general guidelines for countermeasures against insider misuse are also provided to protect data and systems. keywords: insider misuse; misuse detection; misuse countermeasures. introduction if one was to play a game of word association and use the terms ‘security breach’ or ‘cybercrime’ as the starting point, it is very likely that words like ‘hacker’ or ‘virus’ would be amongst the first responses. it is somewhat less likely that terms like ‘employees’ or ‘insiders’ would emerge as many peoples’ first choices. in reality, however, insiders are very often the cause of the most significant and costly security incidents, and a significant proportion of what is commonly classed as cybercrime can be attributed to them. indeed, the fact that insiders are already within the organisation often puts them in an ideal position to misuse a system if they are inclined to do so. although the great majority of the people are familiar with the generic meaning of the word 'misuse', when we try to map it to an it context, there is a need to clarify certain issues. insider it misuse can be a very subjective term, and one of the most challenging tasks is to draw a clear line that separates an it misuser from a person who is using a system in an acceptable way and for an approved purpose. the word ‘misuse’ implies the presence of rules that specify the conditions of allowable usage for the resources concerned. these rules are often embodied within an it usage policy. however, such a policy, and hence the definition of misuse, can differ from one organisation to the other. for example, where some would give priority to detecting datatheft and unsanctioned modification of data, others might want to detect denial of services and internet access abuse. thus no single definition of misuse is appropriate for all organisations. the aims of this paper are to present evidence of the insider misuse problem, and suggest possible means by which it could be addressed. the discussion begins by examining the scale of the problem, based upon evidence from computer abuse surveys from recent years. this is followed by a more specific consideration of what can actually be considered to constitute it misuse in an organisational context, which then leads into a discussion of methods that could potentially be employed to combat the problem. the scale of the insider misuse problem if one takes a look back to computer crime literature and surveys dating up to the mid-90s, the evidence presented would certainly suggest that the main threat was to be found from one’s own staff (with as much of 80% of computer crime believed to be the result of insider activity). in more recent years, however, many sources have indicated a significant rise in externally sourced incidents (principally in terms of internet-based attacks such as website defacement and denial of service), with the consequence that although insider misuse is still significant, it now accounts for a far lesser proportion of raw incidents. for example, in the uk, results from the department of trade & industry’s information security breaches survey 2002 revealed that only 34% of businesses considered their worst security incident to have been caused by an insider (dti 2002). this possibly accounts for why 60% of respondents in the same survey were either not very concerned or not at all concerned about threats originating from their own employees. however, when considering the large businesses (with over 250 employees) only, it should be noted that the split between those experiencing their worst incident as a result of internal staff versus external parties was almost equal. \another source that has monitored the changing trend regarding internal and external attack is the annual csi/fbi computer crime and security survey. looking back to 1995, a key observation from the csi was that “the greatest threat comes from inside your own organisation” (power 1995). in more recent years, however, the survey results have painted a rather different picture, and by 2002 it was reported that, for the fifth year running, more respondents had cited their internet connection as a frequent point of attack (74%), ajis vol. 10 no. 2 may 2003 135 than had cited internal systems (33%) (power 2002.). this may well be the case, but presenting the information in this manner tends to create something of a false impression, because the raw number of incidents is not necessarily the factor that we should be most concerned about. of more interest to most ceos, for example, will be the effect that the incidents had on their bottom line. many of the categories used in the csi/fbi results encompass incidents that could potentially have been both internally and externally sourced (e.g. theft of proprietary information, sabotage of data networks, and virus). however, three of the categories very clearly indicate the source, and it is interesting to see the level of the annual losses that were associated in each case. the relevant information is presented in table 1 (power 2002). system penetration by outsider inside abuse of net access unauthorized insider access 1998 $1,637,000 $3,720,000 $50,565,000 1999 $2,885,000 $7,576,000 $3,567,000 2000 $7,104,000 $27,984,740 $22,554,500 2001 $19,066,600 $35,001,650 $6,064,000 2002 $13,055,000 $50,099,000 $4,503,000 total $43,747,600 $124,381,390 $87,253,500 table 1 : annual losses for selected incidents from csi/fbi surveys it is quite evident from the results that, although they relate to a five-year period over which the proportion of externally sourced incidents had exceeded internal ones, the quantifiable losses in the latter case dwarf those attributable to outside hackers. it is therefore clear that, in real terms, the level of the insider threat is still much greater than that exhibited by external hackers. the csi figures relating to insider abuse of network access clearly suggest that, as well as bringing considerable advantages in terms of web and email communication, internet access has also ushered in a whole range of new problems. this can be further evidenced by a survey of 544 human resources managers, conducted in 2002 and targeting large uk companies (with ‘large’ in this case being defined as those employing an average of 2,500 people). the results revealed that almost a quarter of them (23%) had felt obliged to dismiss employees in relation to internet misconduct (with the vast majority of these cases – 69% being linked to the downloading of pornographic materials) (leyden, 2002). many other cases resulted in less severe courses of action, such as verbal warnings or a discreet word in the ear of the person concerned, and in total the results indicated that 72% of respondents had encountered internet misuse in some form. the nature of insider it misuse one of the csi/fbi categories from table 1 was that of ‘unauthorised insider access’. however, one of the complicating aspects with insiders, and the aspect that differentiates this from the other insider category listed in the table, is that incidents will not always relate to something that is unauthorised. indeed, the basic problem with insider misuse is that the person concerned has legitimate access to it resources of the target organisation. this means that he/she does not need to bypass the authentication mechanisms of the it infrastructure (no stealing or illegal reproduction of passwords and other forms of authentication tokens). thus, in an it context, insider misuse is the act of abusing granted privileges to cause harm. in this context, it can also be observed that users that know more about a system are more likely to abuse their privileges than users who are less knowledgeable. although this is not difficult to grasp, vagueness is introduced by the term misuse and what it means to different people or organisations. what is considered illegitimate use in one particular organisation can be perfectly acceptable for another. for example, browsing the web for personal use is outlawed entirely in some companies, whereas others are somewhat more relaxed about it and impose varying limits upon what is acceptable (e.g. some may permit up to 20 minutes per day, whereas others may allow twice this). in addition, there are myriad other activities that would likely be regarded as misuse in any organization, for example: − personal entertainment (e.g. playing games, writing personal email etc.) − downloading mp3s, pirated software, pornographic images, or other unsuitable material − fraud and theft (e.g. modifying payroll database to increase one’s wages) − sending out inappropriate material using company computers − installing and using pirated software. − reading or modifying another user’s files. ajis vol. 10 no. 2 may 2003 136 although the computer security research community has created a plethora of taxonomies that describe computer intrusions in general (see furnell et al. 2001 for an overview), little effort has been placed on the construction of a taxonomy that specialises in insider incidents. the earliest attempt to classify internal misuse of computer systems is presented by anderson (1980) and discusses borders of distinction between masqueraders, clandestine users, and misfeasors. masqueraders are insiders that exploit weaknesses of the authentication system, thus gaining the identity of other legitimate users. a clandestine user is related to authorised users and their capabilities to bypass audit, control and access resource mechanisms in a particular computer system. finally, misfeasors are insiders who do not need to masquerade, but abuse the power of their privileges to misuse the system. however, as the small selection of examples above shows, the single category of ‘misfeasor’ can encompass a whole range of different incidents. as a result, other works have focused more specifically upon the issue of insider misuse, and indicative examples are given below: − tuglular (2000). this is the first comprehensive taxonomy of misfeasor incidents. the taxonomy classifies computer misuse incident in three dimensions: incidents, response and consequences. the entire taxonomy is orientated towards data collection for insider incident response. − magklaras and furnell (2002). this taxonomy is human centric. mgklaras and furnell perceived that all actions that constitute it misuse lead back to human factors. the fundamental aspect for their taxonomy is classifying people in three basic dimensions: system role, reason of misuse and system consequences. this scheme is the most appropriate for threat prediction, but not suitable for detection. intentional misfeasor cases are performed for a variety of reasons. the best way to sub-divide them is to consider the motives in a way that could detect the ultimate goal of the abuser. it might be inferred, for example, that a legitimate user is trying to access sensitive data (data theft), take revenge against a particular person or an entire organisation (personal differences), cover indications of unprofessional behaviour, or deliberately ignore a particular regulation of the information security policy. unfortunately, despite evidence of the insider threat, there is no substantial effort devoted to addressing the problem of internal it misuse. in fact, the great majority of misuse countermeasures address forms of abuse originating from external factors (i.e. the perceived threat from hackers). a significant reason for this is the difficulty in actually monitoring and detecting the problem in order to enable a response to be mounted. in the cases above, for example, it is clear that the misuse would have been very difficult to control or prevent, as the perpetrators concerned were not violating any system-side access rules. combating insider misuse the problem with insider abuse is that, once a user is authenticated to use a system, what he does with the system or the objects he has access rights to is neither monitored nor logged most of the time. considering the list of potential misuses in the previous section, it is possible that appropriate access controls could be used to prevent some of them, but even these will not be sufficient for all contexts (consider, for instance, the case in which the misfeasor has legitimately been granted administrator level privileges). this epitomizes the difficulty in implementing access controls that resembles organisational hierarchy onto the it systems. it must also be remembered that one user/process/account having all the privileges can lead to serious misuse by exploiting the situation. neumann’s suggestion of multilevel systems and compartmentalization (neumann 1999) should be given a serious consideration before we proceed with the insider misuse detection. today’s commercial operating systems are based on the old systems developed years ago. at the time when the core components of these systems were developed, the users were expected to behave themselves. the problem of insider misuse was not an issue. however the research in the it security over the years has proved that people do misbehave and that insider misuse is a serious problem. since these systems were not developed with insider misuse in mind, the preventive mechanism and the logging present in today’s commercial systems are not optimized for misuse detection. existing access controls are not good enough to prevent insider misuse, making it more difficult to enforce insider misuse policies. for example, a user with administrator level privileges may not have the moral right to access confidential data on the system, but access controls present in today’s systems cannot prevent such actions. as such, it is considered that some form of supervision system is required to monitor for misuse activity. such technologies are already available to some extent in the form of intrusion detection systems (ids) (amoroso 1999), but as with many other mainstream security technologies, these are geared towards detecting attacks on the system rather than misuse of it by legitimate users. nonetheless, some of the principles are transferable. for example, current ids employ two main strategies to identify attacks namely misuse-based ajis vol. 10 no. 2 may 2003 137 detection and anomaly-based detection, and it is possible to see how each of these could be applied to the insider problem. − misuse-based detection in a traditional ids, this approach relies upon knowing or predicting the intrusion scenario that the system is to detect. intrusions are specified as attack signatures, which can then be matched to current activity using a rule-based approach. a similar approach could potentially be incorporated for misfeasor incidents, based upon those methods that employees have been known to exploit in the past, or those that they can be anticipated to attempt based upon the privileges and resources available to them. for example, at a conceptual level, one such misuse signature might relate to a user who is identified as attempting to modify a record about him/herself in a database (e.g. the payroll example indicated earlier). the principle here would be that, although their database privileges may allow them to do so, users should probably not be modifying details relating to themselves without someone else’s authority. another example could be to watch for any sequence of events where a user accesses confidential information and then attaches it in an email destined for a recipient outside the organization. neither of these rules would necessarily cause the user in question to be locked out of the system (because in some contexts the actions could still be quite legitimate), but they could be used to flag the activity for closer scrutiny. − anomaly-based detection rather than being based upon known or predicted patterns of misuse, this approach relies upon watching out for things that do not look normal when compared to typical user activity within the system. in a standard ids, the principle is that any event that appears abnormal might be indicative of a security breach having occurred or being in progress. the assessment of abnormality is based upon a comparison of current activity against a historical profile of user (or system) behaviour that has been established over time. for example, past behaviour might suggest that a particular user typically downloads an average of 5mb of material from the web per week, and the nature of the attachments they assign to emails are normally documents. therefore, if activity supervision detects a surge of download activity to 10mb in a single day, or a large number of email messages suddenly being sent with image attachments, then there would be reasonable grounds to investigate whether unsuitable activities might be in progress. although the above descriptions make the concepts sound relatively straightforward, it must be appreciated that neither technique can be considered 100% reliable, even in the context of traditional ids. the consequence is that they can lead to false positives (where legitimate activity is believed to be intrusive) and false negatives (where genuine intrusive activities are misjudged as acceptable). the concept of applying the techniques for the detection of misfeasor activity / insider misuse makes the task more difficult, because we are dealing with legitimate users who are not violating access controls. from a misuse-based detection perspective, it is more difficult to identify the ways in which an insider might misuse the resources to which they have legitimate access, while from an anomaly detection perspective the level of behaviour profiling would need to be much more detailed and precise. when basing the assessment upon a comparison against their behaviour profile, a legitimate user misbehaving will almost certainly be more difficult to identify than a total impostor who is masquerading under the legitimate user’s identity. in addition, in an adaptive system, the process of profile refinement might be exploited by wily misfeasors who gradually train the system to accept misuse behavior as normal. as such, this aspect is still an area of active research, as the technical approaches are not mature. when considering how to protect systems now, it is worth noting that preventative measures need not be technical. insider misuse is a management problem as much as it is a technical issue. as such, formal internal controls are as important as technical controls. security guidelines, such as the recommendations provided by the iso 17799 standard (bsi 2001), typically suggest a number of personnel-related measures, which if employed correctly could dramatically reduce the likelihood of insider misuse being successful: − check references of prospective new employees before hiring them; − ensure that employment contracts include a clause relating to the acceptable use of it resources; − ensure that adequate reminders about the ‘acceptable use’ policy are encountered by staff during their day to day use of systems; − ensure adequate supervision of staff by line management; − provide a means by which staff can confidentially report misuse of it systems, without fear of recrimination from colleagues. ajis vol. 10 no. 2 may 2003 138 − ensure proper division of duties (i.e. such that collusion between staff members would be necessary before significant opportunities for frauds could be identified). − concerning the access of data, make sure that access control policies resemble organisation’s management hierarchy or rules. − security and access control policies need to be maintained to keep up with the change in organisation’s management hierarchy. in the absence of an automated supervision approach, it would still fall to line managers and the like to enforce and monitor these aspects. conclusion insider misuse poses a great threat to organizations. even though the internet connection is the most frequent point of attack, the loss due to insider misuse is far greater than the loss due to external attacks. at the present time, the system level countermeasures that can be implemented are limited. current access control systems, although well-suited to guarding against unauthorized activities, cannot prevent insider misuse effectively if the subject is doing something within their legitimately assigned privileges. more advanced mechanisms, in terms of activity monitoring and supervision systems may offer a potential solution in the future. the authors’ ongoing research will design and evaluate approaches for realizing the latter approaches, and results will be detailed in future publications. references amoroso, e. (1999), ‘intrusion detection: an introduction to internet, surveillance,correlation, traceback, traps and response’, first edition, intrusion.net books, nj, isbn: 0966670078 anderson, j.p. (1980), ‘computer security threat monitoring and surveillance’, 1980. bsi. 2001. ‘information technology. code of practice for information security management’. bs iso/iec 17799:2000. british standards institution, 15 february 2001. isbn 0580369587 dti. 2002. ‘information security breaches survey 2002’. department of trade & industry, april 2002. urn 02/318. furnell, s.m., magklaras, g.b., papadaki, m. and dowland, p.s. (2001), ‘a generic taxonomy for intrusion specification and response’, proceedings of euromedia 2001, valencia, spain, 18-20 april 2001: 125131. leyden, j. 2002. ‘p45s for porn surfers’, the register, 9 july 2002. http://www.theregister.co.uk/content/6/26098.html magklaras, g.b, furnell, s.m, (2002). ‘insider threat prediction tool: evaluating the probability of it misuse’, computers & security, vol. 21, no 1, pp62-73. neumann, p.g. 1999. ‘the challenges of insider misuse’, sri computer science laboratory, paper prepared for the workshop on preventing, detecting, and responding to malicious insider misuse, 16-18 august 1999, at rand, santa monica, ca. power, r. 1995. ‘current and future danger: a csi primer on computer crime and information warfare’, san francisco, ca: computer security institute. power, r. 2001. ‘2001 csi/fbi computer crime and security survey’, computer security issues & trends, vol. vii, no. 1. computer security institute. spring 2001. power, r. 2002. ‘2002 csi/fbi computer crime and security survey’, computer security issues & trends, vol. viii, no. 1. computer security institute. spring 2002. tuglular, t. 2000. ‘a preliminary structural approach to insider computer misuse incidents’, eicar 2000 best paper proceedings: pp105-125. the australasian journal of information systems (ajis) began in 1993 and was based on the simple realisation that there was a need for an academic ‘outlet’ for is research in australia australasian journal of information systems volume 14 number 1 nov 2006 using contradictions to appreciate the history of i.s. education in south australia mike metcalfe school of management, university of south australia (city west) adelaide, sa, australia 5001 mike.metcalfe@unisa.edu.au abstract this paper presents one person's interpretation of the history of information systems (is) education in south australia (sa). the stance used to think about the history was that of seeking the contradictions, underlying tensions, which worked over time to create the present. the paper will argue this stance, suggested to the author, that is education in sa was influenced significantly by maintaining a "how to" view of teaching is which failed to distinguish itself from the engineering worldview. it is suggested that a 'trade school' mentality never turned into a critical academic perspective relevant to modern business schools. after explaining the underlying tension stance this paper will use extracts from a long semi-structured interview with two seminal is educators to support this argument. the problem statement something made is in sa what it is today. if we look at nations with differing constitutions, cultures, and worldviews then we have to quickly accept that particular past events, people and resources had a significant effect on what is seen today. while it is important to be looking forward to how things ought to be in the future, it also seems important to occasionally reflect on the forces that shaped the present. this paper will provide an outsider’s inquiry into the history of is education in south australia. history is written through the eyes of the winners. it is an interpretation; who, when and where are interpreted into why. recounting the history of is education in sa will also be such an interpretation (mason, 1997). it would therefore seem appropriate to identify the stance used to interpret the undisputed physical facts (checkland, 2000). the stance selected for this paper is one with a strong association to systems thinking. it is that of trying to appreciate events as the result of underlying tensions and contradictions. in this case, the tensions that influenced the evolution of is education in south australia, some were global, some national and some local. 261 australasian journal of information systems volume 14 number 1 nov 2006 more specifically, this paper will argue that looking at the history of is by looking at the perceived underlying tension reveals that is education in sa was influenced significantly by the act of trying to move is from a computer science influence without presenting it in a manner compatible with a modern business school perspective. is was offered within a trade school worldview, not one of academic critical inquiry. these tensions need to be explained further, but first the practice of looking at history by looking at underlying tensions needs a little justification. making the stance explicit the history of is could be studied from a variety of stances, a mix of a few, or by contrasting different stances (gardiner, 1959) (bannister, 2002). examples include taking an archaeological stance. this is perhaps the default or imagined neutral stance for many is academics looking at the history of their discipline, one aligned with the physical science. this stance encourages looking at the history of is in terms of its past physical attributes such as student numbers, staff numbers, publications, grant dollars, number of universities, budgets etc. it would still be necessary to explain why (theorise) is had these physical attributes. an alternative stance might have been knowledge creation. is could have been understood in terms of what is known, its body of knowledge, such as how to design, develop and implement a complex socio-technical system. many other possible stances could be taken. the stance chosen was a classic in the business domain, and one with interesting associations with systems thinking. it is that of seeking contradictions or underlying tensions, which has deep roots in european marxism and in some streams of american pragmatism (1997). moreover it does not offend positivist historicism (gardiner, 1959). identifying inherent contradictions, paradoxes, underlying tensions or the forces behind any social event is like looking ‘behind’ or ‘underneath’ the phenomenon or event. karl marx used the now famous example of observing the event of a caterpillar eating. no matter how carefully you measure and count details about the caterpillar, you do not really understand it unless you know about its life cycle (including its metamorphosis into a butterfly); in other words, unless you understand the underlying tensions of survival and evolution. sowell (1985) argues this was marx’s own research methodology where he used it to understand inherited privilege and the control of what we today might call ‘knowledge’ (the means of production) which had created the dreadfully oppressive class systems in europe in the mid 19th century. engles (1964), who worked closely with marx, saw analogies between the tension in society and the tensions in nature. the underlying tension in nature is a physical competition for survival. however, these dialectic forces can be creative. with the contradictions stance of history, what is seen as an artefact or a phenomenon is understood to be the result of hidden forces in tension. for example, the design of a building, a vehicle, or any system is the result of various technical, organisational and personal tensions. the same is true of christmas celebrations which can be seen to be the outcome resulting from the underlying tensions of pagan ritual, the desire for mid winter celebrations and christian church concerns over maintaining power and economics. contradictions between those with wealth and those without, causes the creation of a police force; contradictions between ideologies cause wars. another example is the mobile phone which can be seen in terms of contradictions such as both giving and taking away your personal freedom, and extending while thinning-down social relationships. phones are at least two ‘faced’ (janus faced), and perhaps even three or more (arnold, 2003). nielsen (1996) and mason (1969; 1996) identified various types of contradictions that might be useful to understand social systems. nielsen defines contradiction in terms of dialectic, “change that 262 australasian journal of information systems volume 14 number 1 nov 2006 emerges from the interplay of conflict and among differences and affirmation of areas of agreement”. he argues, in a dialectic with the reader, that these change processes differ in terms of emotion; they do not have to be quarrelsome, aggressive or conflictive but they can be. identifying these types of dialectic contradictions can form a further stance of critiquing articles. his first ‘ideas’ contraction involves asking yourself who is arguing with whom over what idea. who see the same thing differently? is the argument likely to result in one side winning or will it encourage the development of an improved idea? for example with the commercialisation of christmas there are concerns that product developers had the idea that traditional artefacts were merely commercial products, thus frequently rather impoverished versions are produced. nielsen’s second contradiction he calls ‘action’, drawing on the ‘act, reflect, act’ learning loops of schon and argyris (1996). the contradiction is between what is said and what is done. the classic example here is an organisation plan (words) that says something is important but the budget (the action) allocates very little to that activity. in the passage above, a contradiction or a tension of this type, could be identified between the christian meaning of christmas as a birth and the use of christmas trees and mid winter images. nielsen’s third contradiction is that of ‘voice’ where one human group develops their understandings and actions without wanting to be informed by others. in the passage above the product developers want to make cheap commercial ‘look-alikes’ of traditional artefacts for profit, without asking the traditional owners/users of that artefact what they think or feel. one reason may be that the developers do not want to have to deal with the ‘complaints’ of the traditional users who may be a whole community. second, there can be a feeling that the traditional community will not understand or have different priorities. as a result of ‘fear of confrontation’, knowledge is lost. the last of nielsen’s contradictions that will be identified here is caused by a vying for ‘resources’. human action can easily be seen as an endless process of vying for resources with other people and species. this can be used to critique the santa passage by commenting upon the pressure on people to make a living. in this struggle to make resources and profits flow in their direction, rather than to others, people invent popular products. the early church could be seen as vying for the resource of people’s souls, or more cynically for people’s expenditure, when offering forgiveness for sins. method(ology) the contradictions stance just discussed was use to design and interpret a semi structured 2 hour interview with two retired lecturers [mb and bw] who were employed at the only south australian university that offers an is degree. these two interviewees are the only people who were available that had personal experience of the emergence of is from a computing science program. i was the interviewer [mm]. mb had a politics undergraduate degree and an mba, bw had an economics and accounting undergraduate degree. both later took postgraduate courses more directly related to is when they became available, and both worked in industry in positions which involved the installation of it and/or it applications before taking up lecturing positions in south australia. semi structure was imposed on the interview (gubrium, 2000) by using neilsen’s four tensions (ideas, action, voice and resources) to think about the questions asked. the interviewees agreed to this approach. their responses are therefore being used not only to present a history of is from a specific stance but also to evaluate that stance. 263 australasian journal of information systems volume 14 number 1 nov 2006 however, given the interview was a three way semi formal discussion it was hard to be too dogmatic about sticking to discussion couched in neilsen’s contradictions terms. this was made more problematic due to it not being 100% clear which contradiction covered which issue, such as the impact of microprocessors on what was taught. while acknowledging that the discussion was initially designed around the contradictions, a semi-methodical data analysis, was also used to emerge issues from the full transcript the transcripts were analysed using the alexander method (1964) whereby the responses to questions are turned into a series of numbered statements. these varied from one to ten sentences in length. next the author identified which of the statements seemed connected. for example statement 4 and 30 were deemed to have the common element of the struggle to distinguish is from computer science. statement 4: bw: … but the driving force behind it, and mac agrees, who was probably the real father of is education in south australia was a fellow by the name of allan spears and who retired the same year that mac started. statement 30: mb: … the power politics were that, all through the eighties there’s no doubt the computing science was always more powerful than we were, because they had their own programs, their own school and so forth. we didn’t even have a minor. frankly we were just teaching courses within accountancy, as terry and bernell were, and we were doing it within management. we had a head out there, bob northcott and even andy duncan, who at worst were ambivalent and at best were probably mildly supportive of us. when we became a school and a program that’s when we became a threat and when the old guard was moving on and that’s when i think the tensions came in…also there was some teaching information technology in the school of communications, which was the old school of library…. the nursing school was teaching a computing subject and computer science wanted to take over that. a matrix of all the numbered statements and connections was then drafted, which was provided as input to a socio-network software package called ucinet 6 . this presents the matrix and a graph (see figure 1: transcript issues network). note that the nodes are the statement numbers and the connecting lines deemed by the author. this network suggests that there are three main issue clusters that emerged from the interview, some more tightly clustered than others. the amount of separation in a cluster is relevant, for example the top right cluster out from statement 66 suggest this issue was discussed fairly independently of other issues. in contrast the issue cluster around statement 6 is hard to extract from the issue cluster around statement 1. this suggests the discussion merged the two issues. 264 australasian journal of information systems volume 14 number 1 nov 2006 figure 1: transcript issues network cluster 1 the top-left issue cluster is generally the interdepartmental struggle between is and cs (eg 6,), trying to emphasise the vision of what is did that was different. this would seem to be a ‘resources contradiction’. woven in with this issue was that of personalities (eg. 4, 5, 34, 65, 58, 54, 26) which is assumed to be a ‘voice contradiction’. the personality of one head of school mentioned in one statement (54) encouraged the connecting of that statement to working conditions statements (eg. 60-68). this latter might be called an ‘action contradiction’ as there seemed to be some significant differences in staff workloads, qualifications and involvement in research outputs. 6: mb: bill is saying that allan effectively service taught systems analysis in computing science from the school of business. then cs did the flip, took it over and it ended up being business was being service taught by computing science instead of the other way around. we had that battle for a period didn’t we bill? 5: bw: [alan] was the driving force, initially behind teaching of systems analysis in south australia, which was novel in the school of computer studies at the levels at that time, out at mawson lakes. they always used to talk programming and computer hardware stuff and it was very hardware oriented, techo type course. industry was saying, we want systems analysts. allan, who was a very progressive sort of a guy, was also a qualified accountant. he said we really need to teach systems analysis and he got off his backside and actually devised one, he went to a couple of courses interstate himself, like a privately run course by private organisations. we’re talking ’74 now. 54: mb: can i tell you a story of tr? ok, tr, i don’t know why tr left the university of …, whatever the reason was he was actually hired by [our] school of computing science. 265 australasian journal of information systems volume 14 number 1 nov 2006 bill, correct me if i’m wrong, but i’m pretty sure this is right, to service teach the accountancy school because lb had either moved on or didn’t want to teach their information systems course any more. so tr was service teaching accountancy. at that stage, the stirrings were coming, with bw, myself and tb, who was basically the technology teacher at the low end of accountancy, to set up an information systems discipline. we could see in the longer term that there was strength in the accountancy and management schools getting together. we used to sort of spit at each other, but we began to realise there’s probably a bigger threat, and that was computing science. anyway, once you get old, you begin to forgive each other, their sins. we actually started talking to computer science instead of just, ignoring them. tr was the odd one, he would have to come over from computing science. i went up to head of accountancy, which in those days of course was what’s his name, kb, and said look, is bloody stupid to have tr sitting out at computing science, we’re building our own is discipline here with management, and at that stage, kb was getting into bed with rm, so there was support on their part to set up an is discipline. low and behold, 3 months later, tr was brought on to accountancy. … tr had sat on the board of the school of accountancy anyway. 66: bw: i think to be honest, it’s true to say that without being rude or anything, but under tr, some people who didn’t want to work didn’t have to work. tr was a very easy going forgiving person and there were some people who didn’t do too much at all… 63: mb: it was a different environment. it really was a different environment. that’s why i moved on in 1995, because it was inevitable in a town of this size, as it would be probably anywhere, when (a) the level 2 tertiary education institution becomes a level 1, (b) you had the cae’s and the universities, the institute of technology once was a cae, but you turn a cae into a university, and immediately all your academics are going to want to become a mini university of adelaide. i still remember the best thing that could happen to you as an academic at a cae was to get the calling to go the university of adelaide. now this is probably an outsiders viewpoint, a rather jaundiced one, but there was always that need to bid up, so it was inevitable that the institute of technology, and the cae, which was a teaching institution, and very practically focused on the community it was serving, in our case, businesses, witnessed by the number of courses that bill and i used to do, bill especially. there’s no longer any focus at all at this place, non whatsoever, except a bit of switch to research. we inherited our first 2 phd’s when we married up with this cae and that was mr and what’s his name, the big guy, big russian fella, ds. they were 2 phd’s. i mean, they were anomalies. cluster 2 the bottom-left issue cluster (eg. 8, 39, 52) is generally about ‘normalising’ and accrediting is in south australia with is interstate (37, 71) and overseas (eg. 59) programs, a very south australian preoccupation. this again may be an ‘ideas contradiction’ where differences need to be justified. statement 59 can be seen to connect seeking the input of an overseas (8) personality (4). 8: mb: i came out of an mba in canada, supposedly the top one out of western at the time. we had a business it subject, i’ve forgotten exactly what they called it, but what we learnt at one of canada’s top mba schools, in the second year of the mba was nothing like what i found happening here. i came here in ’76, joined in ’78. what bill was doing and what had been done under allan spears was certainly unique in the context as i knew it in the canadian academic environment. they were doing things here that as 266 australasian journal of information systems volume 14 number 1 nov 2006 a practicing information systems professional i could recognise as my professional area, as being is and not business computing, which was the environment i left behind where we were learning programming and that kind of thing. there was very little systems analysis in an mba course. it was an absolute eye opener to me in terms of what was happening in south australia. now whether that makes south australia unique or australia unique or canada out on the edge i don’t know, but my suspicion is that it was a personality buried in adelaide that was doing things that very few people were doing anywhere else in the world. 59: bw: but one other, i would think, just thinking outside the square a little bit, but something i think influenced, mb and i considerably, was in 1982, while i was there, i called into a seminar run by the university of california in los angeles. i was extremely impressed by a fellow by the name of edgar plane, who was the bloke co-ordinating the seminar. i talked to him and he sort of said he’s interested in coming to australia. when i came back i talked to gm and the guy who ran techsearch at the time about bringing him in running a series of, a 5-day management seminar in australia. when he came, and i had quite a lot to do with him, and i think he was also instrumental in my thinking. 71: bw: what we were trying to do, you had to sort of battle for respectability i suppose in a way. when i went to these information systems conferences, there was a quest for establishing what the body of knowledge that represents the discipline of information systems is. there were a lot of, especially in those early conferences, a lot of people giving papers about what their perspective and what they taught and what they thought the body of knowledge that was information systems as distinct from computer science. mb and i, i think, were trying to get plugged in to the debate that was going on around the world in terms of what is the body of knowledge that represents information systems and we undoubtedly gave it some distinctive sort of bits and pieces of our own. we wanted our graduates to be able to go off to sydney or new york or london or whatever, and be able to be relevant. cluster 3 the right hand issue cluster is generally about the ideological struggle between providing what professionals wanted to know compared to what computer scientists wanted to program (eg 1, 11, 12, 24, …) which is mixed up with the impact of microprocessors, spreadsheets and work processing (15, 16, 17, 18, 20..). this was an ideas contradiction, with the needs to advise accountants on how to decide their billing etc requirement so as not to be over or undersold computer power by an over enthusiastic salesperson or colleague. however, servicing accountants’ needs did provide a distinct income stream, so it is also a resources contradiction. 1: bw: i graduated with an economics degree in about 1963. i was working in a job shortly thereafter in which the company was talking about computerising as everything was done manually at the time with mechanical accounting machines. 11:bw: by that stage in the piece, but nobody is telling small businesses and medium businesses how to buy a computer, what to look for in buying a new computer. or if you’ve got a computer and putting a new system in, how to, how as a business person do you determine what your requirements are and those sorts of things. , if you’re the managing director of some company and you’ve got some smart techo guy comes and starts talking about bits and bytes and codec and so on. 267 australasian journal of information systems volume 14 number 1 nov 2006 12: bw: and how do you communicate [to computer engineers], how do you [professional groups] specify what your requirements are? and they [is academics] had identified that there was a huge need for this 24: bw: just as a matter of interest, digressing there. i remember at one stage i think mb was talking to a group about word processing and one of the blokes was from the computer society. i think he might have been from the school out at the levels. he started thinking; oh i could write one of those. he was starting to talk about how he could write the code for the people in his company or wherever he worked, to do his own word processing programs and so on. that was their orientation. 15: bw: well the original microprocessor/personal computer originally wasn’t used to, like they are now, to do corporate processing. it was used mainly for spreadsheets and word processing and those sorts of things. we started teaching spreadsheets and so on but of course that’s grown in power. 16: mb: mainly with small business people, accountants, medicine, doctors. 17: mb: i know bw disagreed with me on this but in terms of introducing spreadsheets to accountants there was only one person in adelaide who did it and that was bw. and i still have this image of, you’d sit an accountant down and you take him through the basics of spreadsheets and somewhere around about midday they’d leap out of their chair and jump up and down and do a little jig. 18: mb: well for a lot of small business people i certainly think it did, it put it on their desk and gave them the power to work with it. 20: bw: basically i still spend at least half of every day of four days working spread sheets. now, i’m writing programs associated with them. missing tensions? the analysis just summarised was an attempt to identify the underlying tensions that sculptured is education in sa from the early 1970’s to 2004. the ideas tension of lecturers being researchers now present in the discipline is seen as a late 1990’s phenomenon in sa, one still resented. indeed, the seminal characters interviewed seemed unwilling to accept that research for publication in academic journals was an effective use of their time. it almost represented a career change away from being ‘doers’ to one of wasting taxpayer money. mb commented that it was one reason he retired early. this reflected their understanding of their duties as educators as mainly to demonstrate to future practitioners how to manage it; requirements calculations, systems analysis, systems design and control. they taught ‘how’ not ‘why’. married to this was some acknowledgement that as teachers they should make students enthusiastic about the possibilities of it. my suggestions that an is discipline teach critical thinking skills in students, or to help them critique the consequences of technology beyond immediate efficiency issues, was not fully supported (mingers, 2000). indeed, my suggestions that is include decision making issues or forecasting were also not taken up apparently because they were primarily concerned with the effective design and control of routine corporate operations report generation systems, such as billing reports. given the interviewee keenness to communicate in the liberating effects of microprocessor and subsequent applications software, the discipline in sa did not develop down the 268 australasian journal of information systems volume 14 number 1 nov 2006 route of designing inquiry systems. both saw a significant moment in the development of software like word processing and spreadsheets, although they did not report that affected their mainstream teaching effort, systems analysis. their enthusiasm for this technology change seems mainly due to how it symbolised the shift in technical power (markus, 1987) from the engineer to the manager. it provided evidence of the distinction between is and it. however, issues of power were rarely discussed with students. the educational worldview in sa seems to have been very much on supplying useful employees for industry; rather than critical thinking or knowledge creation. this was done with the well-meaning intent of wanting to assist students to be competent for international employment. there was almost no tension seen in the alternative of their role being to develop the minds of the student beyond grasping a technical and managerial skill, albeit one that would survive many of the changes in hardware and software. there was no tension in seeing technology and industry were good; their job was to ready students as suitable input into those processes. moreover, there was no tension between their role as producing trained middle managers and their educating international thinkers or community leaders. subjects like strategic thinking and globalisation were not seen as core. there was also no tension expressed between their being very open to new ideas and experiences against the validity of these. there was almost no questioning of how to distinguish truth from lies, or how to inquire whether processes were unsustainable. text book materials were taught largely as unquestioned authority, if sometimes in error. control, measurement and error minimisation were not seen as being in tension with creativity, ethics or invention. none of which is unexpected given the ‘trade school’ culture of the educational organisations forced together as a university in the early 1990’s. however, a lack of tension between the validity of what was taught and how that validity was determined must be a concern for any higher education. in summation the ‘ideas’ tensions identified from the interview included the well documented ones of is concerning itself more with managing the technology into a commercial operation compared to designing computers and programming them. this seems to have been a discipline problem rather than only being specific to sa. the alternative tension between is and the sociology of technology seems to have not been explicit. the ‘voice’ tension was that computer science which had a much stronger presence in terms of academic credibility and numbers of professors. computer science’s calls for rationalisation were louder than is’s attempts to articulate even to itself its ideological differences. the ‘resources’ tensions were that the 1980’s boom in industry and in the demand for business studies qualifications over scientific one provided students to is over it. however, the research earning capacity of it schools exceeds that of is schools. the ‘actions’ tensions may be seen is teachers hiding under the blanket of the rhetoric of higher education as being the creator of knowledge, and being primarily concerned with developing the critical skill of individual students rather than merely being a provider of input to industry; a confusion of what industry wants being what the student needs. what was taught in the classroom as is in south australia would seem to have focused on system analysis. the analyst was seen to be the corporation’s representative in dealing with purchases from the large and powerful computer companies. bw in the interviews above said that he was under the impression that this was a fairly common vision for is education in australia. those responsible for the design of local programs did so after making themselves aware of what was being taught elsewhere through conferences and email. in 1999 the head of school at unisa was still trying, 269 australasian journal of information systems volume 14 number 1 nov 2006 somewhat unsuccessfully, to organise course sharing between australian universities. avgerou et al. (1999) reports that in europe during this period there was a very diverse range of themes in is from developing technologies per se, to the social impact of the new information and communications technologies. the author’s own experience in the uk was of teaching information theory, applications appreciation and expert systems architecture in courses called computerised information systems to accountants, who were originally thought of as the expert on data processing in organisations. this was in an environment of is in the uk seeming to be obsessed with information systems development methodologies in the late 1980s and 1990s. the emphasis on ‘developing technology per se’ and in applications in is in the us during this period, rather than systems requirements, is emphasised by the assumptions in watson et al.’s (1999) report on is in the us. they say ‘is, perhaps more than any other field, is impacted by changes in [the technology]”. this is in complete contrast to the interviewees’ response. when directly asked what difference the technology made to the design of is courses, they emphasised that is was about assessing the information requirements of a company. what technology delivered that information was only important in the sense that it might make sought after information viable in terms of cost or speed of delivery. in broader educational terms, this is tertiary education appears to have been delivered under a culture of a ‘trade school education’, failing to clearly distinguish itself from computer science to the universities’ senior management group. educating an is analyst was seen as explaining how to do it, rather than developing the graduate’s critical or theory based view of technology (marks, 2001). in this culture, the management of technology meant being enthusiastic about technology which needed to be developed as rapidly as possible. relevant research was seen as developing new, hopefully profitable, applications. it was not an education in thinking critically about technology in the business world. is was seen, by the hard working and enthusiastic lecturers who delivered it, as existing to provide basic is training in a profession, much like becoming an electrician or plumber. they saw the role of their is groups as government-provided basic training to supply local industry. this training was not explicitly informed by a strategic, educational or critical worldview beyond that it was good. 270 australasian journal of information systems volume 14 number 1 nov 2006 references alexander, c. (1964) notes on the synthesis of form, harvard university press, mass. argyris, c. & schon, d.a. (1996) organisational learning ii, addison wesley, mass. arnold, m. (2003) “on the phenomenology of technology: the 'janus-faces' of mobile phones”, information and organization, vol 13, pp 231-256. avgerou, c., siemer, j. et al. (1999) “the academic field of is in europe”, european journal of information systems, vol 8, pp 136-153. bannister, f. (2002). the dimension of time: historiography in information systems research. electronic journal of business research methods 1(1): 1-10. checkland, p. (2000) “soft systems methodology: a thirty year retrospective”, systems research and behavioural science, vol 17 no 1, s11-s58. engles, f. (1964) dialectics of nature, progress publishers, moscow. gardiner, p. (ed.) (1959) theories of history, the free press of glencoe, oxford. gubrium, j. f. & holstein, j. a. (eds.) (2000) handbook of interview research: context & method, sage, calif. marks, a. (2001). “a 'polytechnism' for the british universities: embracing a revised 'soviet' model for academia”, teaching in higher education, vol 6 no 2, pp 275-280. markus, m. l. & bjorn-andersen, n. (1987) “power over users: its exercise by professionals”, communications of the acm, vol 30 no 6, pp 498-504. mason, r. o. (1969) “a dialectical approach to strategic planning”, management science, vol 15, b-403-b-414. mason, r. o. (1996) “commentary of varieties of dialectic change processes”, journal of management inquiry, vol 5 no 3, pp 293-300. mason, r. o., mckenney, j., et al. (1997) “developing an historical tradition in mis research”, mis quarterly, (september), pp 257-278. mingers, j. (2000) “what is it to be critical?” management learning, vol 31 no 2, pp 219-237. nielsen, r. p. (1996) “varieties of dialectic change processes”, journal of management inquiry, vol 5 no 3, pp 276-293. sowell, t. (1985) marxism, unwin, london. watson, h. j. (1999) “leaders assess the current state of the is academic discipline”, communications of the association for information systems, vol 2 no 2, pp 2-27. 271 australasian journal of information systems volume 14 number 1 nov 2006 the author mike metcalfe presently works at the university of south australia, where his main duties are phd thesis adviser now in the school of management. his own phd was from adelaide university, on group problem solving. he has published 6 books and over 60 lead-author refereed academic articles on problem solving using pragmatic systems thinking and argumentative inquiry, in journals that include systems research and behavioral science, it & people, informal logic and the european journal of information systems. he grew up in england, egypt, germany, wales, aden, and singapore, moving to new zealand in 1985. mike has worked in the merchant navy, the british army parachute regiment reserves, the construction and food industry as a system designer, six universities as a lecturer, and as senior policy adviser to the deputy premier and treasurer of south australia. 272 missing tensions? the author ajis vol 9 no. 1 september 2001 problems and issues in using computer-based support tools to enhance 'soft' systems methodologies marie stansfield division of computing and information systems university of paisley high street paisley pa1 2be, scotland mark.stansfield@paislev.ac.uk abstract this paper explores the issue of whether computer-based support tools can enhance the use of 'soft' systems methodologies as applied to real-world problem situations. although work has been carried out by a number of researchers in applying computer-based technology to concepts and methodologies relating to 'soft' systems thinking such as soft systems methodology (ssm), such attempts appear to be still in their infancy and have not been applied widely to real-world problem situations. this paper will highlight some of the problems that may be encountered in attempting to develop computer-based support tools for 'soft' systems methodologies. particular attention will be paid to an attempt by the author to develop a computer-based support tool for a particular 'soft' systems method of inquiry known as the appreciative inquiry method that is based upon vickers' notion of 'appreciation' (vickers, 196s) and checkland's ssm (checkland, 1981). the final part of the paper will explore some of the lessons learnt from developing and applying the computer-based support tool to a real world problem situation, as well as considering the feasibility of developing computer-based support tools for 'soft' systems methodologies. this paper will put forward the point that a mixture of manual and computer-based tools should be employed to allow a methodology to be used in an unconstrained manner, but the benefits provided by computer-based technology should be utilised in supporting and enhancing the more mundane and structured tasks. introduction as a result of the widespread application of case tools within structured information systems development there has been a growing interest over the last decade in developing computer-based support tools for 'soft' systems methodologies. whilst computer-based support tools might provide obvious benefits for structured information systems development in terms of speed, efficiency and consistency, there are a number of questions and issues relating to the extent to which computer-based support tools might enhance the application of 'soft' systems methodologies. one of the problems is that 'soft' systems thinking and structured/'hard' systems thinking are rooted in very different theoretical foundations which has a profound affect on how the concepts and ideas are applied through various methodologies and approaches. whilst structured/'hard' systems methodologies are underpinned by precise rules and conventions and can be validated for consistency and correctness, 'soft' systems methodologies often lack precise rules and conventions, and thus, their application is guided more by the experiences of the user and the needs of the unique problem situation. this provides researchers with a number of obstacles and problems in attempting to develop computer-based support tools that remain faithful to the 'soft' concepts and ideas. in addition, once such tools have been developed, there is the issue of applying them within the context of real-world problem situations so that they can be assessed in determining the extent to which they actually enhance the 'soft' systems methodology that they are supposed to serve. computer-based support tools for 'soft' systems methodologies the application of computer technology in supporting the use of, or explaining the concepts relating to 'soft' systems thinking has been attempted by a number of authors and researchers over the last decade. some examples of such work have included the development of software tools that aid the user in carrying out ssm, such as a computer-based support tool named 'get rich quick!' (avison et al, 1992) that provides graphical support to the user in the production and development of rich pictures. ssamt (davenport and ayers-hunt, 1995) is a windows based modelling tool that provides support mainly with aspects relating to selecting relevant systems, formulating root definitions and developing conceptual models within ssm. in addition, facilities for problem diagnostics as well as help screens for each of the stages of ssm are provided. zhang et al., (1997) developed a computer-based support tool known as the 'softcase toolkit' that provides a rich picture builder to the user through the use of drawing tools and pre-stored icons. relevant systems can be named and associated with the rich picture as well as textual notes that can be included. in addition, the toolkit also provides the user support in the formulating root definitions and developing conceptual models as well as entering associations between the different stages of ssm. the full implementation of the toolkit also automatically generates a worksheet in comparing a conceptual model with the real-world, as well as automatically generating a maltese cross. dunning-lewis (1997) highlights a number of attempts at providing a computer-based support tool for the ssm user. these have included employing a dbms package with ole 102 ajis vol 9 no. 1 september 2001 links to create a support tool for hold working documents of an ssm study, as well as a shell application that provides an interface to the user, that also makes use of a range of commercial software packages that may be utilised during a study. work has been carried out in developing computer-based support tools for teaching 'soft' systems concepts such as ssm using expert system technology (stansfield, 1990; stowell and stansfield, 1991; stowell et ai, 1991). the computer-based support tool provided students with an overview of each of the seven stages comprising ssm, guidelines in the use of ssm as well as tutorial exercises, after which, the student can compare their answers with that of an expert's. this work was developed further using multimedia technology (stowell et al., 1993) in which a more enhanced graphical user interface was utilised as well as incorporating audio expert advice into the computer-based support tool. to date, most of these software tools that have been developed to support the learning and application of ssm are still very much in the prototyping stage. in many instances the researchers have experienced difficulties in developing the computer-based support tools whilst remaining faithful to the theoretical and practical elements that underpin many 'soft' systems methodologies. mentor (daellenbach and petty, 2000) is a multimedia, interactive educational package that was developed by operational research teachers as an alternative means for students to learn about systems thinking and operational research methodology to be used conjunction with other learning methods such as lectures, tutorials and a text book. use of the mentor multimedia computer aided learning package is considered to have helped staff cope with increased student numbers through decreasing tutorial preparation time and in providing continuity between topics (simpson and edwards, 2000). technological influences on 'soft' systems methodologies the problems experienced by researchers in applying computer technology to 'soft' systems thinking appear to stem from a situation similar to that described by winograd and floras (1986. p96). in describing a number of tasks that a computer programmer carries out in developing a computer program, they highlight the situation in which "the programmer must have a clear analysis of just what constitutes the task and potential set of actions for the program". this highlights the point that in developing computer software, the programmer must be able to predict in advance as many of the possible scenarios and actions that will be performed by the user of the software. the use of technology may lead to a narrowing of the learning with regard to a problem situation in that it may be viewed in ways which best fit the representation provided by the technology, as opposed to providing new ways of addressing it. this point was discussed by vickers (1983. p8) who warned about the dubious impact of technological systems by saying: "// shapes its users' minds and habits; it limits as well as enlarges. 'to a man who has only a hammer, 'it has been said, 'everything tends to look like a nail. 'a comprehensive survey of the computerized models now in use would surely show how closely concentrated they are on situations which lend themselves to such modelling ". the computer was regarded by vickers (1987) as a tool of thought and as such shapes organisations and lives in the way that it is used, since it is not neutral. thus, vickers was warning against viewing and creating a world that merely mirrors and represents the present understanding of computer-based technology (vickers, 1978). in using computer technology as a vehicle for applying methodologies based on 'soft' systems thinking, the practitioner may be in a situation in which he/she is declaring how the methodology will be used before any learning and appreciation has taken place among the participants. this anticipation of outcomes appears to run contrary to the theoretical underpinning of 'soft' systems thinking in which the way a particular method is applied within a problem situation is determined by the learning cycle. 'appreciative' experiences that are gained by the participants should not be dictated by a computer programmer or the limits of a particular technology. thus, a danger with computer-based technology is that it might constrain the user in terms of how they might apply it to a problem situation (dunning-lewis, 1997). in addition, the increasing ease of use of computer technology with the use of interactive screens offering many different options may either make a user feel overpowered or overwhelmed in that the technology might distract them from the exploration of the problem situation (dunning-lewis, 1997). equally as problematical might be the case in which the user is given the impression that through the technology they can easily grasp and understand the intricacies of a complex and challenging process of inquiry that underpins methodologies such as ssm (kreher, 1993). this may be applicable especially in situations when a computer-based support tools provides advice about how a problem situation might be addressed since the thought processes and the creativity of the user might be constrained (zhang et al., 1997). 103 ajis vol 9 no. 1 september 2001 the development of a computer-based support tool to enhance a 'soft' systems method of inquiry the 'soft' systems method of inquiry for which the computer-based support tool described in this section was developed to enhance was the appreciative inquiry method. the appreciative inquiry method (west, 1992; 1995) draws upon the work of vickers (1965) and checkland (1981) in providing the is analyst and other participants within the problem situation with a means of exploring less structured, difficult-to-describe elements associated with knowledge elicitation and information systems provision. attempts by other authors and researchers at providing computer-based support tools for 'soft' systems methodologies such as ssm, led to research into investigating the extent to which the appreciative inquiry method, that draws upon a number of concepts and ideas from ssm, might be enhanced by computer-based technology. the two software platforms that were considered by the author for the development of the computerbased support tool were multimedia authoring software and hypermedia authoring software. such software platforms appeared to be more suited as an output medium in which information is represented through various media in the form of an interactive 'book' through which the end-user can browse information deemed relevant. a study into small business marketing was used as a means of learning about the appreciative inquiry method and the computer-based support tool. due to pressures which arise from a lack of resources such as a time, manpower and money, many small business managers and business practitioners do not adequately reflect and explore the notion of marketing, how it is carried out and its affect upon the business as a whole. furthermore, there does not appear to be any clear method or process of inquiry mat practitioners can adopt easily to allow them to carry out this process of exploration and reflection. whilst the domain of small business marketing contains certain structured elements, the domain also relies on many experiential, subjective elements that are reliant upon the judgement of the practitioner. the study supported practitioners in their exploration and reflection of small business marketing as well as providing a real-world context within which to assess the computer-based support tool. the manual-based version of the appreciative inquiry method was used initially with the participants to allow lessons to be learnt that could be fed into the development of the computer-based support tool. once developed, the computer-based support tool was then used with the participants so that the tool could be assessed as well as comparisons made with the manual-based method of inquiry. in using the computer-based support tool a portable notebook computer was used by the is analyst which meant that the approach could be transported easily to the participants' place of work, thus causing the mitiinuim of disruption to their busy schedules. the participants in the study included small business owner-managers, small business advisors and consultants and business and marketing academics. the appreciative inquiry method can be broken down into three iterative phases: phase i systems maps systems maps are used as a means of representing those important elements that a participant might consider relevant to a problem situation. a systems map is a pictorial representation that draws upon the conventions of a venn diagram (t301, 1984). the participants are presented with a area of interest which is placed in the central element of the systems map. the area of interest is decided upon prior to the first session through discussions with the participants. this area of interest may be altered as a result of learning about the problem situation during the session. the participants are encouraged to explore a range of issues relating to this central element by adding a secondary layer of elements, and a tertiary layer linked to the secondary layer, if appropriate, and thus, building up the map. the elements comprising the map may be either separated or they may overlap each other to highlight some link or relationship depending on how the participant wishes to represent their view of the situation. the systems map also serves as a platform for discussion between the is analyst and the participants, and as a means of clarifying points, as well as learning and reflecting about the situation. with regard to the computer-based support tool, the systems map was created using a small control panel that enabled the participants to add map elements and move them around the screen as they wished. the users were able to edit and delete any of the elements on the map using the mouse or trackerball device. in comparison to the manual-based method of inquiry where large sheets of paper could be used, the size of the computer screen meant that the participants had little space to construct a systems map. this limitation was recognised and, consequently, two facilities were incorporated into the computer-based support tool in an attempt to minimise any possible adverse effects. the first of these was a scroll bar and zoom-in feature to allow the use of as much of the screen as possible, and the second, a small on-screen notepad facility allowed the participants to type in ideas whilst constructing the map. since in using the map manually the participants would often annotate their maps and make notes beside their maps, the notepad facility in the computer-based support tool provided the participants with the opportunity of continue making notes if they so wished. an example of a systems map produced using the computer-based support tools is shown in figure 1. 104 ajis vol 9 no. 1 september 2001 editor reset deuelop expert knowledge saue expert knowledge instructions maine: yeraloa: [ i | phase ot aim phase i: systems mtf notepad vhrthcrerrarteteojtjforunrprahrb. on atred or questionnaire, qtentblratrictod in tte three tediriquatnj given eoi stanza b tte type of qusttore thst hs stodd to «k1 ng taught vifl ebo irriufc enmrraawation jwib b enable uieclltrt tocnrryout the rearerch. secondary research tkhntqua art ato tougtt to enable thsdltenttoaarminfcrrnjijonins less palrtfi^l' v«y .la. figure 1: an example of a systems map produced using the computer-based support tool phase n catwoe and root definition systems map elements are translated into purposeful activities using the catwoe elements of ssm. the elements are customers, actors, transformation process, weltanschauung, owners and environmental constraints. the elements are used to provide a careful description of the 'system' through the development of a root definition. the importance of using the catwoe questions and formulating root definitions is that they provide a means of exploring the participants' weltanschauung concerning the problem situation. this is made explicit in investigating the participants' understanding of the named activities as a whole, as opposed to it being fragmented and 'piecemeal'. an activity model, that is similar in format to a conceptual model, is developed from the root definition as a means of representing diagrammatically the activities and relationships that need to be in place for the overall purposeful activity as identified by the participants to take place. in relation to the computer-based support tool, in providing the framework for asking questions relating to catwoe, a conscious effort was made not to use systems terms that may have confused the participants. this was an important lesson learnt from using the manual-based method of inquiry, in that the questions posed to the participants used terminology that could be understood easily. by choosing a catwoe element (e.g. 'who has control over the named activity', the owner element of catwoe), the participants were presented with a box in which to type in their response. the framework for asking questions relating to catwoe is shown in figure 2. 105 ajis vol 9 no. 1 september 2001 editor reset deuelop expert knowledge saue expert knowledge instructions a bid •specify cflttiloe elements (sub-map 2) [update cftttuoel sub-map 2 sub-map name who hu control ov»r the narord artviiy? which ptrson/jroup vill the «c*riiy have an imp set on? which penon/eroup cony out the named activity? development & implementation of marlttttnj process ram vhftt vtevpoint cfltwoe elements tion mvujfnnnt toon £ 5 •̂ wfeo is in control of thto ovenl! *ctftrlgr v&icli you haw named? figure 2: the framework for asking questions relating to catwoe using the computer-based support tool the answers to all of the catwoe elements as provided by a participant where input using pre-defined nodes and links from the previous screen where the participant typed in their response to the catwoe questions. the catwoe answers, which were colour coded to highlight and represent the different catwoe elements, could then be input into a pre-defined root definition. this allowed a workable root definition to be formulated quickly that could then be edited accordingly. this was considered by the is analyst as possibly contributing to helping with the somewhat time consuming task of formulating root definitions. any of the text in the root definition could be edited and the elements moved around the screen to provide a format to suit the is analyst's needs. the colour coding of the different catwoe elements also meant that the elements could be identified easily in the root definition. furthermore, the catwoe elements could be updated using pre-defined nodes and links to take into account any changes to the root definition. phase hi modelling and agenda the purpose of the activity model is to provide a means of representing the activities and relationships that need to be in place for the overall purposeful activity, as identified by the participants, to take place. the activity model is used as a means of furthering the learning process with the participants in that the model may be developed further and used as an agenda for further exploring the problem situation. in relation to the computer-based support tool, during the process of developing an activity model, both the root definition and the transformation process element of catwoe could be viewed easily on the screen that could be referred to in assisting the development of an activity model. the activity model could be drawn using a control panel that allowed the is analyst and the participants to create and name activities as well as connect them using arrows. furthermore, the participants could be prompted to define criteria relating to the 3 e's, namely effectiveness, efficacy and efficiency as a check as to whether the transformation process had been identified successfully (checkland and scholes, 1990). the activities as shown in an activity model were then transferred to an agenda using pre-defined nodes and links. the agenda was used with the participants as a way of comparing the situation in the real-world, with model(s) developed during the sessions. this comparison formed the basis for further investigation and appreciation of the situation. an example of an agenda developed from an activity model is shown in figure 3. 106 ajis vol 9 no. 1 september 2001 ^ reset oeuelop expert knowledge snue enpert hnomledge instructions flgonda (sub-map. sub-nap 2 ajrttitt dmcriinlnn preaepliq acwk futtiiw* from »m«u button <g»naj 1. 2 l5 objmtfvfi 2. o 0^ v 2 ^ o 0 m«»f i»4 whm ptrf «mv4n£> of rat «f <ht ompmv. ttw omnp4»y «wir«r «h» tlmt «nd t«y» te bt «m* «rv badly (raft «<m>«t»j hiv> «»1 v«rv ihtl* markf t inftrnuumi t« wort on m «• ftftitetfetmmi km 1. ix j04w « xkm|| cxitpnv b*l t»l (inw •ed wj» rtnirmt. orotuny vouuimt rm bvov»d in flrti rtrtrfetdl figure 3: an example of an agenda developed from an activity model a feature of the computer-based support tool was the provision of a computer-assisted learning facility with regard to the various techniques the participants may encounter. the use of the computer-assisted learning facility was used to support the is analyst in introducing and explaining the techniques to the participants through the use of examples, graphical representations and simulations. another feature provided by the computer-based support tool was a 'screen snap' facility that allowed 'snap shots' of the computer screen to be taken at any point in the discussion that could be stored for future reference with regard to the investigation. this also enabled the contents of the screen to be printed out so that the participants could receive a paper copy of the techniques that allowed them to reflect on any issues after the session. lessons from applying the computer-based support tool to a real-world problem situation there were a number of important lessons that were learnt by the author in relation to the role of the computerbased support tool as applied to the small business marketing study. some of the lessons are outlined in this section, however, a more detailed discussion and analysis of the lessons can be found in stansfield (1997). the computer-based support tool did provide some improvements, the most notable of which was: editing the majority of the participants considered the ability to easily edit their systems maps, root definitions and activity models encouraged them to refine their initial responses and thoughts regarding the problem situation, since they did not have to redraw a whole model or map. the ability to edit a model or map on the computer screen enabled them to interact with the techniques and the is analyst much more freely since they could try out ideas and explore thought processes. this contrasted with using the manual method of inquiry in which the maps and models would often become messy and difficult to read and follow as a result of numerous edits during a session. furthermore, it was time consuming and mundane having to manually redraw maps and models that often interrupted the flow of the session. in addition, the speed and ease of access to previous material that the computer-based support tool provided meant that material from previous sessions could be easily displayed on-screen. this was particularly useful when a number of weeks had elapsed between sessions and the participants wanted to refresh their memories as 107 ajis vol 9 no. 1 september 2001 to what had taken place in previous sessions. this contrasted with the manual-based method of inquiry in which the is analyst would often have to carry numerous documents and folders containing paper versions of maps, root definitions and models as well as notes from previous sessions which was cumbersome and took up a lot of space. however, there were a number of substantive problems with the computer-based support tool that included: flexibility the issue of flexibility was compounded by the need to predict in advance what a participant may require from the software in exploring their area of interest. because of the flexibility provided by the manual-based method of inquiry, the computer-based support tool proved inflexible due to the difficulty of anticipating all of the possible scenarios in which a participant may wish to use the approach. in more traditional, 'conventional' is analysis the use of more structured methods and approaches is usually clearly defined by specific syntax and notations, and hence, is more open to such predictions in developing computer software to operationalise them. apart from a limited number of simple guidelines, there are few formalised rules associated with 'soft' systems methodologies such as ssm and the appreciative inquiry method. the idea is that participant should be allowed to explore and express their views and ideas with the minimum of pre-defined constraints. when using the computer-based support tool there is a danger that if a participant wishes to perform a task that has not been predicted, then they have to alter their thinking to fit the particular format provided by the computer software. such a situation could be argued to interfere, and consequently limit a participant's learning experience and thus, undermine the underlying intention of 'soft' systems thinking. there is a danger that a participant's views and thinking may be misrepresented because of the problems created by the computer-based support tool. during the study, on several occasions, participants when using the computer-based support tool were unable to adapt a technique or use it in a particular way that suited their way of thinking because it had not been accounted for in the development of the computer software. when using the manual-based method of inquiry the participants could easily adapt techniques or bring in new ones that they considered appropriate to the situation. ease of use notwithstanding advances in the ease of use of computers, the computer-based support tool was not as easy to use as the manual-based method of inquiry. the manual approach could be learnt after a very short introduction and its flexibility allowed the participants to adapt it to suit their particular way of thinking. that was not the case when they used the computer-based support tool. for example, the participants had to use a keyboard, a mouse and some edit commands before they could begin to think about the problem situation being explored. this highlights the problem of making the user interface as natural and transparent as possible so that carrying out tasks using the computer-based support tool is at least no more complicated than carrying out the task manually. when using the manual-based method of inquiry, the participants did not require instructions about the mechanics of drawing and writing. although familiarisation with the computer or more advanced technology may reduce these problems, the very nature of the tasks seemed to act as a distraction to the main activity that did not exist in the manual-based method of inquiry. some of the participants had reservations about using the computer technology because of a claimed lack of computer literacy, physical disabilities and a reluctance to spend the time to learn to use it. participants have to be convinced that using a computer-based support tool will save them time and effort and are not just using computer-based technology for its own sake. issues relating to the extent to which computer-based support tools can enhance 'soft' systems methodologies there were a number of generic issues about the use of computer-based support tools for 'soft' systems methodologies that appear to transcend the choice of specific software and hardware. whilst the utilisation of more sophisticated software may have overcome some of the problems encountered by the author, there are a number of issues that could be attributed to computer technology generally, which need to be addressed in the future. to become familiar with computer technology the user has to invest a certain amount of time and effort in learning to use it. however, when this issue is related to the real-world problem situations, problems may arise in that certain participants may not be able to spare the extra time, effort and money required to familiarise themselves with the technology before the session can proceed. in addition, the time, effort and costs involved in becoming familiar with technology have to be assessed against the possible benefits that such technology can bring to the user. if the advantages of using computer-based technology are only marginal, then it might not be feasible to use a computer-based support tool as opposed to the manual-based method of inquiry. a point related 108 ajis vol 9 no. 1 september 2001 to the issue of familiarity with computer technology is the regularity with which the user may wish to perform a task. for example, if a user participates very occasionally in a 'soft' systems study or as a 'one-off* event, then the time spent in familiarising themselves with computer software may not be justified. they could use a manual-based method of inquiry without the need for spending extra time, effort and expense on training in certain computer-based technology. when a user is trained to carry out tasks using computer-based technology, there may be a danger that the user is led to believe that the task can only be performed in a certain way or that they have to follow particular conventions provided by the computer software. this may have an effect of constraining a user's thinking or misleading them about the task being performed. thus, computer-based technology may impose or reinforce rational and logical thinking about a problem or task when it may not necessarily be the case. thus, it is important that computer-based technology does not constrain the users' creativity in carrying out a task or thinking about a problem. therefore, any computer technology should be flexible enough to allow the user to be creative with regard to thinking about the problem situation. the user of the computer support tool should not be overburdened or overwhelmed by the extra responsibilities they may face in learning to use and apply the computer-based technology. some people may dislike using computer-based technology and it may cause them undue distress and pressure during the session, which may distract or inhibit them in exploring their problem situation. therefore, it is important that a computer-based support tool does not detract from any participative debate that may be required when thinking about a problem situation as a result of users becoming too focused on the intricacies of using the software. conclusion it is unlikely that computer-based support tools could fully automate the process of inquiry that underpins 'soft' systems thinking. many 'soft* systems methodologies such as ssm and the appreciative inquiry method provide few formal rules that dictate their use. participants can adapt and customise the approaches to suit the needs of a problem situation in a manner that would be largely unacceptable when using more 'hard'/structured systems methodologies and approaches. a more likely and feasible scenario is that a mixture of manual and computerbased tools could be employed during the course of a study. a manual-based method of inquiry could be used for the more creative aspects of the study in actually investigating the problem situation. this would allow a methodology to be used in an unconstrained manner so that tools and techniques can be adapted to suit the thinking of the participants, as well as being able to apply the methodology in a non-sequential manner. the main benefit that a computer-based support tool may provide in enhancing a 'soft' systems methodology is more at an administrative level in terms of allowing material to be stored and accessed more easily as well as in allowing material to be edited more freely during a session. thus, computer-based technology could support and enhance the more mundane and structured tasks associated with a 'soft* systems methodology. it is the author's view that the thinking should be left to the people involved in the problem situation who should be allowed to explore a problem situation unhindered by any constraints imposed by computer-based technology. the work presented in this paper presents some initial lessons from developing and applying a computer-based support tool to a real-world problem situation. much more work needs to be carried out in applying such computer-based support tools within the context of real-world problem situations before they can be assessed in terms of the extent to which they enhance the ideas and concepts that underpin them. references avison, d.e., golder, p.a. & shah, h.u. (1992) "towards an ssm toolkit: rich picture diagramming", european journal of information systems, vol 1. no 6, pp397-407. checkland, p.b. (1981) systems thinking, systems practice, wiley, cnichester. checkland, p.b. & scholes, j. (1990) soft systems methodology in action, wiley, chichester. daellenbach, h. & petty, n.w. (2000) "using mentor to teach systems thinking and or methodology to first-year students in new zealand", journal of the operational research society, vol 51, pp 1359-1366. davenport, m.s. & ayers-hunt, j. (1995) "soft systems analysis and the modelling tool (ssamt): computerbased support for conducting soft systems thinking", critical issues in systems theory and practice (eds.: k. ellis, a. gregory, b. mears-young, g ragsdell), plenum press, new york, pp291-295. dunning-lewis, p.j. (1997) "concerning computer-based support tools for the user of soft systems methodology", systems for sustainability, (eds.: f.a. stowell, r.l. ison, r. armson, j. holloway, s. jackson, s. mcrobb), plenum press, new york, pp459-463 kreher, h. (1993) "critique of two contributions to soft systems methodology", european journal of information systems, vol 2, no 4, pp304-308. stansfield, m.h. (1990) the development of an expert system to aid the user of soft systems methodology. m.sc. project, school of information science, portsmouth polytechnic (unpublished). 109 ajis vol 9 no. 1 september 2001 stansfield, m.h. (1997) the effect of computer-based technology in attempting to enhance a subjective method of knowledge elicitation, ph.d. dissertation, university of paisley (unpublished). simpson, g & edwards, j.s. (2000) "was it good for them? reflections on the use of mentor and changes in course design using general performance measures", journal of the operational research society, vol51,ppl352-1358. stowell, f.a. & stansfield, m.h. (1991) "a first step towards the automation of ssm?", systems thinking in europe, (eds.: m.c. jackson, g.j. mansell, r.l. flood, r.b. blackham and s.v.e. probert), plenum press, new york, pp319-325. stowell, f.a., west, d. & stansfield, m.h. (1991) "the application of an expert system shell to an unstructured domain of expertise: using expert system technology to teach ssm", european journal of information systems, vol 1, no 4, pp281-290. stowell, f.a., west, d. & javelaud, v. (1993) "developments in the automation of ssm: tutorials using multimedia technology", systems science: addressing global issues (eds.: f.a stowell, d. west, j.g howell), plenum press, new york, pp269-274. t301 (1984) complexity, management and change: applying a systems approach. block 1 introduction, 1984, the open university, milton keynes. vickers, g. (1965) the art of judgement: a study of policy making, chapman and hall, london vickers, g (1978) presidential address, 22nd north american annual meeting of the international society for general systems research, washington d.c. vickers, g (1983) human systems are different, harper and row, london. vickers, g (1987) policymaking, communication and social learning. (eds.: gb. adams, j. forester and b.l. cantron), transaction, inc, new jersey. west, d. (1992) "knowledge elicitation as an inquiring system: towards a 'subjective' knowledge elicitation methodology", journal of information systems, vol 2, pp31-44. west, d. (1995) "the appreciative inquiry method: a systemic approach to information systems requirements analysis", information systems provision: the contribution of soft systems methodology, (ed.: f.a stowell), mcgraw-hill book company, london, pp!40-158. winograd, t. & flores, f. (1986) understanding computers and cognition: a new foundation for design, ablex publishing corporation, norwood, new jersey zhang, j., smith, r. & watson, r.b. (1997) 'towards computer support of the soft systems methodology: an evaluation of the functionality and usability of an ssm toolkit", european journal of information systems, vol 6, pp!29-139. 110 microsoft word ajis vol13-2 marina.doc 5 australasian journal of information systems volume 13 number 2 may 2006 ajis editorial ajis publishes high quality contributions to theory and practice in the world-wide information systems (is) discipline, with an emphasis on the australasian context of the research. the journal welcomes submissions on research and conceptual development based in a very wide range of inquiry methods, ways of thinking and modes of expression. is is a complex and dynamic field of study and activity. its scope as a field of research and practice is contentious. an agreed hard and fast disciplinary boundary does not exist, and it would probably be counter-productive if it did. i expect, and hope, that the debate about what topics, issues, theories or methods are core to the discipline will be never-ending. however, for an editor these matters are not 'merely academic'. i, with the advice of editorial board members and colleagues, need to make judgements as to the relevance of submissions to ajis. in the development of this edition judged three submissions to be outside the scope of ajis and suggested alternative journals to the authors. in these cases the submissions did not go to review. with some trepidation i list below broad topic areas that are of interest to ajis and hasten to add that they are, of course, not exclusive. i have declined four papers on recommendation of reviewers and more papers are in their second or third revision. this edition features three 'themes'. the idea of a featured theme is to allow parts of the discipline to present some of the best of their current work. theme editors selected and evaluated papers then wrote an introduction showing why the theme is important in is and how the papers contribute to it. my compliments and thanks to the theme editors. i am keen to continue this idea as it is an opportunity to look at a facet of is more coherently and in more depth than is possible in a single paper. themes that have been suggested for the future include is and governance, is philosophy and virtual communities. please contact me with suggestions and offers to edit a theme in your area of interest. my thanks to the paper authors and to jennifer bradley for her work on copy editing this edition. thanks also to the school of information sciences and engineering, university of canberra for its financial support. craig mcdonald associate professor of information systems university of canberra, act, australia email: craig.mcdonald@canberra.edu.au broad information systems topics the nature of data, information and knowledge and their role in understanding, representing and acting in our personal, social and physical worlds. the systematic interaction of people, organisations and information technologies; architectures and the impact of unsystematic technology adoption. the analysis, design, deployment and management of information systems and the impacts of information systems on individuals, organisations and society. content domains (e-business, health informatics, government informatics, e-learning, etc) and cultural contexts for is. multi-disciplinary integration to address information and systems in complex situations. ajis vol. 8 no. 2 may 2001 the new 'knowledge speak': the implications of contested definitions of knowledge and information rob shields department of sociology and anthropology carleton university, canada edwina taborsky sociology and anthropology bishops university/university of toronto, canada abstract this paper presents a critique of notions of information and knowledge found in the management literature and amongst proponents of 'knowledge management' (davenport and prusak 2000) interviewed in the course of a study of changing operations of the federal government of canada vis a vis the 'knowledge-based economy' as it is found in canada (shields, taborsky, jones, and o'hara 2000). interviews with a range of federal government departments and agencies on a number of topics including knowledge management were conducted throughout the summer of 2000. during these interviews a number of competing definitions and numerous misunderstandings of the relationship between knowledge and information emerged.42 we will distinguish and contrast these with definitions derived from semiotic and information science frameworks. we argue for the importance of the collective and processual nature of knowledge. our conclusions allow us to specify the shortcomings of existing knowledge management approaches and to identify a necessary and specific focus for future knowledge initiatives in organizations. a public sector response to the knowledge based economy following international trends in government and private industry, the canadian government is moving to define canada as a 'knowledge based society', characterised by what is loosely labelled as the 'knowledge based economy' (kbe). in its 1999 throne speech, the canadian government challenged citizens and the public service to embrace necessary changes in ways of working together and in understandings of the importance of information as an economic asset: 'knowledge and creativity are now the driving force in a new economy our human talent, values and our commitment to working together will secure canada's leadership in the knowledge-based economy.' (throne speech: 1999). over the last several years the canadian public service has responded to this emerging orientation through a number of proposals and strategic initiatives. adopting the language of a select group of management consultants (wenger and snyder 2000; wah 1999) and following similar steps taken by private sector firms to integrate it into their business processes, the public service has focussed a set of initiatives around what is referred to as 'knowledge management' (km). these initiatives have included the facilitation of information exchange and problem solving by adapting email software, providing information to clients via world wide web directories and information pages, and heightening awareness of the need to share information within units by establishing chief knowledge officer positions. at the heart of these initiatives are notions of knowledge and information. these definitions are derived from technological sources (such as it), from lay interpretations of definitions presented in philosophical classics (aristotle), from ad hoc ideas generated in discussion and brainstorming groups and from the management literature, which continues to inflect and imbued the terms 'information' and 'knowledge' with new connotations in the search for concepts that harness the evocative power of the symbolic. although we recognise the challenges of operationalising a concept like 'knowledge' within the time horizons of public-, not to mention private-sector organizations, the definitions of these key terms are buffeted about in language such as 'infodynamics', 'knowledge work', 'knowledge workers' and 'the knowledge based economy'. what is the difference between knowledge and information? is it valid to say that our understanding of the nature of knowledge itself is changing and could the de-anchoring of the classical notion of knowledge be the reason that this term is suddenly 'sexy' and available for redeployment? 42 these interviews were conducted with research assistants kevin jones and charles o'hara as part of a strategic grant to rob shields, suzan ilcan, dan o'connor and edwina taborsky on 'challenges and opportunities of the knowledge based economy' funded by the social sciences and humanities research council of canada. we wish to thank our interviewees and informants, colleagues suzan ilcan and dan o'connor, and the organizers and participants in a sshrc strategic theme focus conference for their comments and suggestions. 142 ajis vol. 8 no. 2 may 2001 'knowledge-speak' some popular definitions knowledge management is a term that has originated in think tanks on private sector productivity and in schools of management in the united states. km proponents always cite the same consultants and writers. these common reference points reflect the important role of a small number of km conferences and texts in disseminating the approach. works or presentations by prusak, davenport and denning (2000) so on are the sources of a number of key terms, but in practice there is a diversity of approaches. for example, in one canadian government document, hunter provides this gloss: data are facts, observations, or measures that have been recorded but not put into meaningful context. a single musical note is data. information is data that has been arranged in a systematic way to yield order and meaning. a series of notes arranged into a tune is information. knowledge is information in the mind, in a context which allows it to be transformed into action. a musician is able to play a tune because of his knowledge (hunter 1999:np). this stimulus-response model takes the arrangement or processing of data to yield information, and the arrangement or processing of knowledge to yield action. a mechanical input-output process is presumed in which each state of data, information or knowledge, is presented as an object which is itself unchanged by the knowledge process, but which, in a serial manner, leads to the next stage. that is, they are all operating within a mode of discrete detachment. these definitions are representative of the language we have found in our survey. in both theory and practice, these models create problems because they do not clarify the nature of and the relationship of information to knowledge and the dynamic nature of knowledge as a process. it is often unclear to individual officers how information and knowledge should be applied in practice and how strategic choices could be made between different knowledge and information management systems. essentially, this current model operates as a traditional classical or newtonian mechanics, which establishes our reality as made up of discrete 'things' and examines how these discrete entities interact with each other either singly or collectively. if we translate this process into semiotic terms, we find a dyadic architecture where a discrete signifier or object will be bonded to its equally discrete signified or meaning. such a mechanical semiotics sets up a scenario where all interactions are, whether random or 'brute force', direct and unmediated. we see this architecture represented in numerous definitions of information and knowledge within the current management literature. in other discussions, definitions are more thoughtful but left implicit, buried in details or their implications are not clarified. the social dimension is implied but these discourses quickly turn to an individualistic perspective. 'knowledge has a whole host of attributes. it's alive not dead, which means you can query it. it has subtext and interaction. some kinds of knowledge just aren't easily captured in documents... try explaining to someone how to ride a bicycle... the more multidimensional knowledge is, the more difficult it is to stuff it into a flat report. it flourishes in, and is nourished by, conversation' (prusak 2000). like euclidean geometry and newtonian mechanics, simplified definitions have worked for over sixty years for those involved in manipulating information. however, knowledge management claims to go beyond mere information gathering, storage and retrieval. as a form of management post-modernism it is presented as something above and beyond, a sort of 'quantum theory' which abstracts, yet unifies the entire field of organizational phenomena. it aspires to produce not only timely decision-making and understanding but wisdom: 'ghandi and martin luther king created a vision of their world in their time by declaring "i have a dream"... the true promise or alchemy of the knowledge economy will be to make individual weaknesses irrelevant as we maximize individual and community strengths' (nicholson-o'brien 1999:32). following from these definitions we wish to provide some critical reflection. we focus on the interaction between knowledge and information because their relation is not a one-to-one process which sees both as 'things-in-themselves', with information possibly imagined as a 'billiard ball' and knowledge imagined as a 'collection of billiard balls'. this mechanical outline of information and knowledge considers that change can result only from an external force with the power to disperse or collect and re-categorize those billiard balls. but this architecture is unable to describe a generative process of information and knowledge development. important: such a dyadic semiotics ignores the necessity for a third system, a memory structure that is not merely 143 ajis vol. 8 no. 2 may 2001 a collection of units but is a generative mediate process of establishing relations between those units. what new understandings would be required to break with the established focus on manipulating information a model which is a staple of textbooks and the basis of multi-trillion dollar global it and management consulting industries? critical definitions we must therefore begin with definitions of information and knowledge. much of the language currently used to distinguish between them is at such a high level of abstraction that it is not only difficult for bureaucrats to understand, but it risks becoming meaningless; it is reduced to the level of rhetoric i.e. intentionally persuasive and overly extravagant. therefore, we find it helpfiil to provide our own definitions and to highlight some of the operational considerations that can be used to better focus these definitions. information definition: information is data a precise descriptive measurement or representation of matter. information is both locally contextual and relational and we can assign a definite quantitative and qualitative description to this data. semiotically, it can be defined as 'secondness', 'a mutual action between two things regardless of any sort of third or medium' (peirce. 1.322), and it operates in that interaction as 'an act of arbitrary will or of blind force' (peirce 1.328). that is, the relation is not interpretive but mechanical. operational considerations: when information, in this state of secondness, is transmitted from one context or place to another, it must be placed in a contextual bond to information in the new locality to provide that basic dyad of a 'mutual action'. if we were to transmit this type of information as isolate 'bits' without that contextual lock, it would quickly dissipate to a semiotic state of vague firstness, a 'state of unawareness of anything else' (peirce 1.306). it would be meaningless. how does one transmit a discrete unit as well as its precise relations? what is required is a condensation of this linked sign-interaction into a more general sign, which can then be moved to a different locality, where its generality enables it to 'lock into' and express itself within an entirely new local context. when information about one context is integrated with information about other contexts, this expansion-within-generalization becomes knowledge to the actors working in those different contexts. the next step of transforming this general knowledge into localized and pragmatically operational information allows it to then become manageable as a new knowledge that reflects the new context. therefore, one needs to ensure that there are people and processes that function as 'translators' or 'systems of mediation' who can broker the transformation of information out of one context, into a form of generalized knowledge, and then into a new form of information in a different context. our interviews indicate that these skills are held by traditional groups of 'knowledge professionals' and that it may be difficult for knowledge to move outside of these groups. this insight was also supported by comments made by interviews with senior managers regarding the problem the quality of knowledge and information in the public service. knowledge: definition: knowledge is an analytic operation on information. knowledge is a mediate process of generalization and semiotically can be understood with the peircean process of 'thirdness,' which is 'the medium or connecting bond' which acts as a 'synthetic consciousness' (peirce. 1.377). this operation must be understood as an interpretive process, for 'brute action is secondness, any mentality involves thirdness' (peirce 8.331). knowledge, as that mediate process of thirdness is a generative and future-oriented and evolutionary process of dynamic interpretation. bakhtin refers to this mediation process as 'dialogism'; it requires a transactional process between elements (i.e. information; bakhtin 1981). it develops the normative habits, the 'quiescent memory' that underlies all specific interactions of a community and as such a normative process, it is 'a matter of law, and law is a matter of thought and meaning' (peirce 1.345). neither knowledge nor information are static, but part of a 'knowledge process' and 'knowledge networks'. when it does take a monological form it ceases to be knowledge, rather it is information. thus a knowledge process involves a phase change mechanism between a reified state of secondness (information) and an energized, fluid, dialogical state of thirdness (knowledge). for example, one worker records the dimensions of a restaurant, walking physically back and forth with a tape measure. only a small amount of what this person learns and knows of the building is recorded, set apart and highlighted as information and importantly, may not become part of the knowledge-base of that particular worker or even of that worker's knowledge-community. long after this information has been gathered and archived a building regulation worker patiently translates fire regulations, understood as 'law' or the mediate 144 ajis vol. 8 no. 2 may 2001 generalization of thirdness, into the specifics of the limited number of patrons that the building, within its particular dimensions, may safely contain. without that mediative triadic process, the insertion of the constraints of regulatory habits, the possibly dangerous future relationship between the basic dyad -the dimensional space and the number of occupants -could never be established. operational considerations: this can be summed up by stating that knowledge is a social or collective operation of generalization and the development of normative habits, while information is the property of an isolated context. knowledge is processual. it only emerges via relating and exchanging different pieces of information, which undergo changes in that process. of central importance, is the suggestion that knowledge is embodied within collectives and communities and is an interpretive rather than reactive or reflective process. to return to the billiard ball analogy, knowledge is not the collection of balls but the rules of the game. and, although one can 'shoot pool' by oneself, to really 'play billiards' requires an opponent and thus a dialogical social context of several players (or at least one must play at being one's own opponent, but thereby imagine a social setting). knowledge networks: definition: within the understanding that knowledge is embodied within a collective, then, we must understand both knowledge and information as part of a dynamic and evolving network of relations within communities. information is the specific and local articulation of knowledge about the world. because knowledge exists at a higher order and scale, and is never completely formally articulated, it will continually evolve as information is passed between and articulated within the different contexts of the community or even, communities, of users. even archived data must be reassessed and rearticulated to be meaningful or informative. operational considerations: therefore, one cannot assume that there is a static stock of knowledge, much like that collection of billiard balls, which an organization possesses and which can be accessed by all workers at any given time. knowledge is dynamic and abstract rather than discrete. as people meet new situations, knowledge will, in its nature as an evolving process, engage in establishing innovative links, which will be articulated in novel units of information, which will themselves be used to revise that knowledge-base. in each of our definitions, knowledge is continuous, relational and abstract. it can be flexibly applied across contexts and holds the potential for innovation by being adapted to different contexts in which it must be rearticulated. information is the specific, local articulation of knowledge. our definitions see no distinction between data and information both are local, codified and organized and we resist ascribing essential normative qualities to communities (e.g. people with passion) because knowledge is an emergent quality continuously reworked by those communities of information users. because knowledge cannot be reduced to a specific single-voiced articulation (i.e. information), knowledge itself cannot be directly managed. in short, while one can manage how information moves in an organization, one cannot manage knowledge. these distinctions are not recognized within much of the language of knowledge management. this is why km proponents often end up proposing information technology solutions to the social barriers to knowledge work within organizations: what you get by adding multimedia is closer to the real face-to-face than a document... the firm can make knowledge visible so people will know where the buyers and sellers are and what is being sold... companies are tackling this with things like electronic "yellow pages," directories or maps of knowledge-holders across the company... you could annotate complex documents with contact names... (prusak 2000) knowledge networks with these definitions of knowledge and information in mind, we move to consider who / what is involved in promoting knowledge management initiatives and who / what has raised concerns about them. we will describe a networknot as a map of a discernible reality but as the means of conceptualizing the ways in which various actors (human and non-human, including databases, technologies, sets of rules and so on) come together and interact to construct complimentary or competing definitions of what knowledge management should, or does, entail. information management (im) is claimed to be distinguished from knowledge management (km): the former has to do with organizing information so that it is easily accessible to all employees (e.g. creating interactive and searchable databases). knowledge management concerns the organizational and 'cultural' changes needed to encourage people to share information and to use it tools. knowledge management initiatives often situate it, im and km within varying relations to one another, at times stressing one element over the others. often km is seen as integrating the other elements by positing it at a higher or more authoritative level of operation. the 145 ajis vol. 8 no. 2 may 2001 attempt to distinguish these points to a struggle between individuals for recognition and between organizational units for resources and control of how to best achieve departmental mandates. the inconsistent, uneven and sporadic ways in which knowledge-based initiatives are being pursued across and within departments in our case study makes it difficult to come up with a clear and stable set of 'canadian government' knowledge management definitions. competing definitions of knowledge and information have emerged. these are anchored more in the politics of knowledge than in the ontology of knowledge. for example, what might be labeled knowledge management in one department, or departmental branch, might be labeled information management or electronic document management for strategic and political purposes in another. similar projects are positioned and championed within differing locations of the organizational structure; sometimes overtly visible, while other times virtually invisible. the unevenness of knowledge and information definitions and procedures reflects the fact that km in its present form does not involve radically new concepts. after all, institutions have always engaged in some form of information sharing initiatives but these have generally operated within the 'billiard ball' model of newtonian mechanics, a process operating within a dyadic semiotics, which operates by transferring rather than transforming information. it follows that the agendas of political control will be reflected in the differing priorities, operational contexts and histories within this mechanical relocation of information. to illustrate these problems of both definition and operation, three heterogeneous and overlapping groups were found within the core workforce of the public service. all are important to consider in relation to all knowledge and information initiatives. it is important to remember that even within and amongst these groups the idea of knowledge management is usually limited to a localized collection of individuals and not spread evenly across the entire organization: km proponents within management there are a group of km promoters who share a similar vocabulary and vision as to what km should be. generally this coincides with the definitions and goals of km in the management literature. however, these managers tend to be isolated within special km / fm offices and projects the km language is by no means universal to managers in the public service. it appears that this group is still largely removed from dealing with the practical operational details of km proposals, operating instead as spokespersons, champions and km / im 'boosters'. it specialists as many of the public service's knowledge management initiatives were either originally developed, or continue to be developed, within information and informatics branches, this group of km promoters overlaps considerably with a second group of it specialists and informatics managers. this group is central to any km initiative, but presently they appear to be operating separately and at a more pragmatic level. in some cases it specialists have adopted the km language with a stronger technological focus and less focus on the implications for the social organization of day-to-day work and the cultures of their organizations. operational workers this then leads to a third groupworkers involved in the day-to-day operations of government and the provision of services. this may include groups as diverse as service delivery personnel and research scientists. what is common here is that the km language is not in use in the day-to-day activities of these groups. many had not even heard of the idea of km prior to our interviews. however, the operational workforce may be involved in a km initiative without giving it this label. in addition, when speaking with people from this group one gets the sense that efforts to produce and disseminate knowledge, to share information whether through the use of it or better organizational practices have always been part of their jobs. because they do not share the language, and in some cases are alienated by the rhetoric, knowledge management initiatives are often distrusted. not only do various uses of the terms information and knowledge jostle with each other but different interest groups such as the three above emerge out of their different positions in a knowledge network and different relationships to information and to the knowledge process. beyond a network of bits of information, some of which is held by individual bodies and not part of any data infrastructure that an organization may have established, there is thus a cleavage of individuals and offices into distinct knowledge classes or groups. our definitions of information and knowledge stress the importance of recognising that information and knowledge are distinct and yet operate in their uniqueness within a dynamic interactional communal process. we found that information is reflected upon and abstracted into knowledge collectively by a community of information users. in sociological terms, these communities are formed around an abstract process of knowledge generation, understood as a 'communal memory', which is brought together and actually engendered by the 146 ajis vol. 8 no. 2 may 2001 patterning of information flows related to the specific projects and services of that community. what designates a collection of individuals as a community of information users, and delineates one community from another, is not an overriding cultural norm or unifying means of identity, but the communal ability to take-up, communicate and adapt to specific, contextual and rapidly changing information processes in a different manner than another community. on the basis of our research we can, for example, specify that it is knowledge which is the key element in the everyday 'habitus' of social groups and classes (i.e. 'knowledge networks'), thereby extending the work of sociologists such as pierre bourdieu (1977). in the very concrete approach we have taken, knowledge, not the performance of normative codes (see bell et al 1999 re. 'performativity'), is the content and operative matter of such theories of social reproduction and cohesion. as a collective goal, tradition, or the components of a world view, knowledge is the 'glue' that gathers and maintains communities (see gadamer 1977). we of course do not mean, with this reference to 'knowledge as glue', only cognitive ideas or brain states, but that different human bodies with varying capabilities and the entire physical, digital and cultural infrastructure and procedures of information and of an organization, participate in its knowledge networks (cf. latour 1993). the distinction between knowledge networks and knowledge itself lies in the former's materiality and ability to materialize knowledge, that is to express it as information that can be applied pragmatically. knowledge also has its own historicity. knowledge networks in organizations such as the federal public service are advanced and effective at concretizing knowledge into information compared to, for example, a feudal court. but where other societies valued knowledge in the form of tradition, for example, modernity emphasized the rule of monological information, that is, knowledge as statistics to great technological and economic advantage. modern bureaucracies (whether public or private) are a social form of cybernetic control. they operate by transforming knowledge into information and thereby allowing it to be moved in a fixed, mechanical or monological state between offices in the form of memos, directives, rules and so on (as one of sociology's founders, max weber decried when he wrote of the 'iron cage of rationality'). hence the humanistic rhetoric in the call for balance, by km proponents. a change in our understanding of knowledge would entail recognizing its difference from information, something we 'moderns' have all trained ourselves out of. we hypothesize that recognition of the shortcomings of cybernetic command and control is happening across many societies in the wake of climatic change, health man-made disasters and systems failures. in part these are due to unexpected consequences, externalities and bifurcation points. but this signals a situation in which the status and relevance of information to any given decision is not clear. how wide a scope of variables, of information sources or points of view should be included in an organization's knowledge network? this is not the place for a discussion of the 'risk society' hypothesis (bech 1992) but it would appear that the degree of legitimation and trust in bureaucratic knowledge networks which operate monologically, at the level of information, is being challenged by demands that these organizations develop networks that operate at the dialogical, heterodox level of knowledge in order to include more forms and sources of information and to better anticipate sudden shifts in the organization's environment. as knowledge processes and information have become global and operative within diverse communities, so the 'politics of knowledge-communities' has become more critical. in an acutely global economy, membership in a community of information users is not restricted, but is fluid and evolving. outside the core workforce of the public service, networks of other groups in privatized agencies, consulting firms, recruitment specialists and broader networks such as professional and scientific communities extend one or another of the internal groups. individuals participate in a variety of communities and to various degrees and levels. for, example a scientist can simultaneously be a member of the international community of science, a practitioner of interdisciplinary theories, a member of the canadian public service, a member of a departmental research branch, and the member of a variety of work groups and informal networks. although communities of information users are neither fixed nor homogenous, their members often share similar backgrounds and means of communication, such as language or documentation, over a long period of time. this can lead to the development of a knowledge-base, that 'quiescent memory', that operates as a constraint rather than a dynamic process. therefore communicating across different knowledge cultures can be a basic obstruction for those wishing to better communicate information across, or between organizations. whether a policy worker, a scientist or an aviation inspector, it can be difficult to successfully communicate information to people who do not have the same skills or experience, the same 'knowledge-base'. however, the barriers that exist between different knowledge cultures between communities of information users are not limited to political agendas, nor to language or even the mechanical means of communication. instead, differences in the actual procedures by which knowledge cultures generate, store and communicate their information may establish these cultures as isolate and closed. for example a scientific terminology is not simply an alternative means to relay information, but is grounded within a specific methodology (i.e. experimentation) formalized in scientific institutions and reinforced by the credentials it grants its members. scientists have been very good at sharing information within their disciplinary community through journal publications, conferences and informal scientific networks both across and beyond the federal public service. 147 ajis vol. 8 no. 2 may 2001 where scientists fail to communicate information is with other communities of information users within an organization and more broadly with the canadian public at large. these failures are particularly evident when scientists are asked to consider alternative and sometimes contradictory knowledge approaches to a specific issue or problem. for example, at agriculture and agri-foods canada, interviewees suggested that knowledge management has been met with considerable hostility by the research branch. the innovation and preservation of knowledge is perceived as the domain of the scientist alone; their fears about the degeneration of the soundness of this knowledge when it is translated into other knowledge networks, for non-scientific users including policy analysts and the lawyers who draft legislation, have meant that initiatives which acknowledge the value of other knowledge communities have been discredited and discarded. the problem of the reliability of information and its potential degeneration when its context changes cannot be ignored. how is this to be dealt with? the community of information users abstract notions of km / im which treat knowledge and information as simple, static objects, that are moved in a shannon mechanical process from one site to another site obscure the durable cultures, structures, relations of power, and physical infrastructure and representations of information (e.g. limited access labs, it databases or paper documentation and records) which actually shape the way in which information flows and how it is interpreted in an organization. this dyadic semiotic frame ignores the vital process of a level of mediative interpretation that enables the development, not only of knowledge, but also of the transformation of information between disconnected knowledge communities and cultures. information is contextual i.e., it is the specific 'know-how' needed to perform a particular task. knowledge is decontextual, it is interpretive and general. however, knowledge never takes the form of an object. it comes to rest, so to speak, only when transformed, within a local context, into information. knowledge is abstracted from information by a community of users, and in turn, its mediative general representation allows a person to adapt and apply information to a different situation. knowledge, therefore, is an emergent property, it cannot be managed as a discrete entity and must be allowed to continually evolve dialogically. therefore, the language of knowledge management with its emphasis on sharing information obscures the fact that in practice one needs to focus, not on simply moving information (which can be a process based only around its nature as a discrete secondness) but on transforming and interpreting that information to a new context. therefore, the current misinterpretation of the different natures of information and knowledge have led to a dysfunctional km process that operates as a mechanical model of packaging and moving informational 'bits'. rather than attempting to manage knowledge by reducing it to these collections of data-bits, which are then 'shipped' from culture to culture or office to office, and end up as that meaningless, nebulous firstness, the focus should be on developing communities of information-users and understanding disabling elements of the existing knowledge network. thus, the development and operation of a dynamic and plastic dialogical community of information-users should be the focus of km. the globalization of corporations, growth of international institutions, increasing flows of international communications and the use of information and computing technologies (icts) in the workplace changes the processes of both the production and dissemination of information. communicating information across barriers is not simply an act of translation, but of transformation. information, to be pragmatically operative in a different locale, must be decontextualized from its former relations, transformed into the general qualities of knowledge, and then, transformed yet again, by its new community of users, into information that is pragmatically operational in the new situation. as peirce frequently pointed out, semiosis is an infinite process of interpretation. a diverse and ever-expanding community of users must develop processes that can perform this interpretive dynamics. it must be understood from a truly organizational perspective, not 'in the sphere of the "i" but in the sphere of the "we" (gadamer 1977:65). how can it do this? first, the information in all these cultures or 'realms of discourse' must be accessible. the political and hierarchical barriers must be eliminated. second, the users must accept the necessity for on-going interpretation. information must be understood as open and, in this sense, future-oriented. it cannot be assumed that it will remain in a monological form of discrete secondness. it must be interpreted and transformed within each local context or office. these first two processes require a constant 'hum' of discourse, based on an understanding of the risks and consequences of this triadic ongoing semiosis. there is yet a further requirement. information/ knowledge, in order to be useful, that is, to 'do work', must be pragmatic. information/knowledge must be accountable to that which it interprets. the community of users will acknowledge that information and knowledge will of necessity have an intention to, at some time, reflect or properly interpret our external reality. we are not advocating, in this outline of a flexible and interpretive semiosis, that these communities of users will produce a fictional discourse. nor are we advocating that they arrive at a final discourse. instead, a basic requirement of semiotics is that the signs that it generates must be pragmatically operational as reliable representations of that which they represent. knowledge must 'be really 148 ajis vol. 8 no. 2 may 2001 operative in nature' (peirce 5.99), for genuine knowledge is viewed as a process that, itself, considers the effects it has on its surroundings. it is this form of pragmatic knowledge management which focuses on communities of information and knowledge-users, we conclude, that must be addressed by km / im initiatives underway in the canadian federal government. references bakhtin, m. 1981. the dialogical imagination m. holquist ed. austin: university of texas press. bech, u. 1992. risk society london: sage. bell, v. 1999. 'introduction', theory culture and society special issue on performativity. bourdieu, p. 1977. outline of a theory of practice cambridge uk: cambridge university press. cida. 2000. knowledge management forum, may 25, 2000, report. ottawa: canadian international development agency. davenport, t. and prusak, l. 2000. working knowledge: how organizations manage what they know, cambridge m.a.: harvard business school press. denning, s. 2000 'what's your strategy for managing knowledge?' presentation, cida knowledge management forum, ottawa (may 25). drunker, peter r, 1999. 'knowledge-worker productivity: the biggest challenge' ieee engineering management review. 27:4 (winter), p.41. gadamer, h-g 1977. philosophical hermeneutics d. lings ed. berkeley: u of california press. hunter, g 1999. some "do's" of knowledge management, report. ottawa: health canada. latour, bruno 1993. we have never been modern hemel hempstead herts.: harvester-wheatsheaf. mcdonald, john 2000. 'information management in the government of canada a situation analysis for the chief iformation officer, treasury board secretariat and the national archivist' report. national archives of canada. nicholson-o'brien, d. 2000. 'government in the knowledge age: how governments can lead with knowledge,' in knowledge management review 3:1, pp. 30-33 . peirce, ch. s. 1931-35. collected papers. edited by: hartshorne, c., weiss, p. and a. burks. cambridge, ma.: harvard university press. citations are by volume and paragraph number. prusak, l. interview. online:http://www.ibm.com.thinkmag articles/pplpower/pplpower2.html shields, r., taborsky, £., jones, k., o'hara, c., 2000. a critical analysis of knowledge management initiatives in the canadian federal public service. report. ottawa: innovation management research, carleton university. wah, louisa, 1999. ;making knowledge stick' management review 2:2 (may), pp. 24-29. wenger, e.g. and snyder, w.m. 2000. 'communities of practice: the organizational frontier,' in harvard business review, (jan.-feb) 149 ajis vol. 8 no. 1 september 2000 framework architecture enabling an agent-based dmter-company integration with xml kjemem j. fellner and klaus turowski otto-von-guericke-university magdeburg, institute for technical and business information systems, business information systems, p.o. box 41 20, 39016 magdeburg, germany, phone: +49 (391) 67 1 83 86, fax: 12 16, e-mail: {fellner|turowski}@iti.cs.uni-magdeburg.de, url: http://www-wi.cs.uni-magdeburg.de abstract. more and more cooperating companies utilize the world wide web (www) to federate and further integrate their heterogeneous business application systems. at the same time, innovative business strategies, like virtual organizations, supply chain management or one-to-one marketing as well as trendsetting competitive strategies, like mass customisation are realisable. both, the necessary integration and the innovative concepts are demanding software supporting automation of communication as well as coordination across system boundaries. in this paper, we describe a framework architecture for intercompany integration of business processes based on commonly accepted and (partially) standardized concepts and techniques. further on, it is shown how the framework architecture helps to automate procurement processes and how a cost-saving black-box re-use is achieved following a componentoriented implementation paradigm. keywords: software agents; xml/edi; un/edifact; component-orientation innovative business strategies motivating inter-company integration the internet enjoys great popularity as a universally available communication channel. the world wide web (www) as its most popular part is changing from a media for information gathering to a platform for distributed (business) application systems. the number of companies offering goods and services via the www rises dramatically. furthermore, companies use the www to federate their mostly heterogeneous business application systems (gaedke & turowski, 1999). under these conditions innovative concepts, like virtual enterprises (arnold, faisst, harding, & sieber, 1995), supply chain management (houlihan, 1992) or one-to-one-marketing (filler & schoder, 1999) as well as trend-setting business strategies, like mass customisation (pine ii, 1993) are realizable. mass customisation, as the combination of the other mentioned concepts, aims at the efficient massproduction of customer-individual products and services (turowski, 1999a). competitive strategies, like mass customisation, imply the existence of state-of-the-art information structures enabling and improving inter-company process integration. this holds especially for small and medium enterprises (sme) participating in a cooperation (or virtual enterprise) following a mass customisation strategy. guaranteeing economic and effective communication between enterprises therefore constitutes an essential success factor when following one of the mentioned strategies. one way to improve communication between cooperating enterprises is the utilization of techniques for electronic data interchange (edi). in this paper, we propose an approach how this can be done using standardized and well-known protocols and techniques. generally, the advantage of edi lies in the organizational information-based surplus values (kuhlen, 1996, esp. p. 90), represented by, i.e., improved organization structure, improved processes, or time and cost savings. there is a rising demand on software that (partially) automates the business processes and that supports exchange of information between potentially incompatible business application systems across the borders of different enterprises. to allow a broad range of application scenarios, such software must feature ease of installation and use, easy integration in an existing application environment, as well as a reasonable price. in this paper we present a framework architecture enabling inter-company integration based on commonly accepted (and partially standardized) techniques and concepts including preparatory work (fellner & turowski, 1999), (rautenstrauch & turowski, 1999), (turowski, 1999a), (fellner & hunger, 2000). 41 ajis vol. 8 no. 1 september 2000 xml sgml mark-up languages: ontologies xml/serviceap procurement protocols components run-time environment (www) figure 1: framework architecture for inter-company integration the use of a platform independent mark-up language in combination with commonly accepted communication standards breaks down the borders caused by heterogeneous business application systems. (we propose to use a combination of the extensible mark-up language (xml) and un/edifact (electronic data interchange for administration, commerce and transport).) in this context, we introduce a (domain-specific) communication protocol for procurement processes, as well as a domain-independent protocol for negotiating communication channels and security demand, e.g. to ensure secure transactions. software agents use these protocols to fulfil negotiation tasks as well as the conversion of incompatible outputs of business application systems. the software agents are composed from software components. this allows for cost-saving re-use and easy adaptation to company-specific demands. a java-based component application framework (turowski, 1999b, pp. 5-7) for interorganisational communication and coordination works as agent integration platform and creates agents to support queued (business) tasks. for each procurement task, e.g., the component application framework generates a negotiation agent and a conversion agent. both agents are removed after the respective procurement task is completed. ontologies and integration with mark-up languages with un/edifact (un, 1995) the united nations established the best-known standard for interorganisational electronic data exchange. due to fundamental drawbacks, the standard did not get the expected recognition and implementation extent, (zbornik, 1996, pp. 92-93), (goldfarb & prescod, 1998, pp. 106-110). some major drawbacks of un/edifact are: • absence of semantic rules, e.g. for quantity or packaging units. • implicit assumption that every organization uses the same business processes and scenarios. • economic shortcomings, e.g. high implementation costs, especially for sme. • organizational shortcomings, e.g. slow adoption to changing business processes, complicate adjustment of established business processes and rules. the projects open-edl'object oriented-edi (tmwg, 1998), universal data element framework (udef) (harvey et al., 1998), basic semantic repository (bsr) and its successor beacon (steel, 1997) address the first-mentioned problems. consequently, the focus of these projects lies in the definition of common business scenarios and semantic rules. in contrast, the xml/edi group (peat & webber, 1997) emphasizes on the possible surplus value induced by the use of xml (bray, paoli, & sperberg-mcqueen, 1997). the xml/edi group, as well as related efforts (for an overview cf. (steffen, 2000)), aim at reducing implementation costs and improving flexibility. 42 ajis vol. 8 no. 1 september 2000 the mark-up language xml, a subset of the standardized generalized mark-up language (sgml) (cover, 2000), offers ways to define individual mark-up elements (tags) as well as document structures that are kept available in so-called document type definitions (dtd). a dtd allows the specification of grammar for the syntax on which a data exchange is based. if the identifiers of such a dtd are taken from a standardized format, or a term set upon which cooperating partners agreed on, messages based on this dtd are meaningful for the receiver, because every mark-up element (xml tag) refers directly to underlying semantics of the data. figure 2 shows an example message encoded in xml for an enquiry for a bicycle-saddle. the used xml tags, i.e. message, product, partnr, as well as the allowed syntax could be defined in an attached dtd. the enquiry itself contains information regarding the message itself (type, enquirydate) and the description of the demanded product (part-nr, description, material). <message> <type>enquiry</type> <enquiry-date>20.02.2000</enquiry-date> <product> <part-nr>230-239844-bez-531</part-nr> <description>bicycle saddle</description <material>leather</material> </product> </message> figure 2: enquiry encoded in xml the basic idea behind xml-based edi is to tag the data with (standardized) xml tags. this is utilized in this paper in combination with a multi-agent system improving the inter-organisational communication and subsequently the coordination of inter-organisational business processes. we use segment names from the un/edifact-standard as field descriptors to ensure communication between individual agents of the multi-agent system. the xml tags in figure 2, which were chosen to be readable by humans, will therefore be replaced by un/edifact segment names. figure 4 shows how to replace the enquiry-date from figure 2 to be compliant to the proposed combination of xml and un/edifact. the now standardized message means "this enquiry is from february 20, 2000.". one may say that this can also be seen in the message from figure 2. this holds, if both partners agree on the term enquiry-date, but normally one has to communicate with systems or partners not known at the time of definition. the names used for the xml tags correspond to the un/edifact definitions. figure 3 shows an excerpt from the corresponding document. the segment names are depicted in bold letters. high-level segment names are directly used as xml tag-names (dtm). dependent segment names (2005, 2380 and 2379) are preceded by the corresponding high-level segment name (dtm2005, dtm2380 and dtm2379). necessary attributes (2005) are presented as parameters of higher-level xml tags. optional attributes (2380 und 2379) are inserted as stand-alone, subordinate xml tags. dtm date/time/period to specify date, time, or period. 2005 date/time/period qualifier, m, an..3 137 document/message date/time, date/time when a document/message is issued. 2380 date/time/period, c, an..35 2379 date/time/period format qualifier, c, an..3 102 ccyymmdd, calendar date: c = century; y = year; m = month; d = day. figure 3: un/edifact segment names as various projects are on their way to establish repositories for domain-specific xml tags based on un/edifact, backward compatibility for the chosen naming conventions is guaranteed. the relation 43 ajis vol. 8 no. 1 september 2000 between new terms and the un/edifact categories can be preserved (harvey et al., 1998, esp. pp. 10-12). beyond this, the vocabulary of the agents may be adapted using mapping repositories. for example, an enterprise in the u.s. that uses the ansi x12 standard may use a edifact/x12-mapper for its agents to be able to understand un/edifact based xml messages. <dtm dtm2005="137"> <dtm2380>20000220</dtm2380> <dtm2379>102</dtm2379> </dtm> figure 4: xml-message based on un/edifact segment names software agents supporting inter-company integration based on the data flow in figure 5 we explain the role of software agents in the framework architecture presented in section 1. the procurement process of a manufacturer will serve as explanatory example. subsequently we describe the implementation. to keep the presentation as readable as possible, we restrict the example to one supplier. figure 5 shows the software agents and application systems on manufacturer's and supplier's side as well as message exchange between them. in common, a (software) agent is defined as an autonomous problem-solving unit that collaborates with other agents to achieve optimised results. for an in-depth discussion of software agents we suggest (bradshaw, 1997, pp. 4-12). it is assumed that both actors, the manufacturer as well as the supplier, utilize an application system for production planning and control (ppc), which, at least, provides an proprietary interface for exporting, respectively importing data in a non-standardized format. in a mass customisation setting, e.g., the manufacturer is responsible to procure the (customer-individual) parts necessary for producing the product configured by the customer. corresponding enquiries are generated by the ppc system. the generated enquiry must be transferred automatically to possible suppliers using edi. therefore, the output of the ppc system (the enquiry) is transferred to the conversion agent. the conversion agent translates the (proprietary) output of the ppc system into an xml format agreed by all actors. based on the information regarding the receiver of the message (the supplier), the conversion agent also filters the needed information to meet requirements of the addressee. this part is important to assure that critical information, like participating suppliers, are kept secret. the conversion agent then forwards the translated and filtered information to the addressee the supplier's software agent. the transfer is carried out using standardized internet protocols (cf. section 4) enabling the use of existing internet connections, thereby reducing transfer costs. on the supplier side, a listener (a system part of the framework component (cf. figure 7) that also generates agents) waits for incoming requests, and transfers the request (the enquiry) to the corresponding conversion agent. the conversion agent translates the enquiry to the format that is necessary for the local ppc system. here, only information relevant for the ppc system to generate an offer is translated. this is possible, since each transmitted data item is marked with a standardized or formerly agreed and therefore understandable xml tag categorizing the content (cf. figure 2 and figure 4). the supplier's ppc system (automatically) generates an offer based on the terms of delivery given hi the enquiry. then, it transfers the offer to the manufacturer using the same procedure and agents as stated above. this constitutes the general communication between the manufacturer and the supplier in a simple one-to-one-case where all terms of delivery are fixed in advance. in real world processes, terms of delivery, like date of shipment, quantity, deadline, or price, are most likely subject of negotiation. to include and automate these negotiation processes we propose to use negotiation agents. negotiation agents are especially useful, when more than one supplier is able to deliver the required parts (cf. figure 5). objects of negotiation are parts that have to be produced in order to assemble a (customer-individual) product. negotiation agents do not need to know much about these objects. identification and certain constraints, i.e. due dates or quantities, are sufficient. after negotiation took place, a negotiation agent returns the terms of delivery on which one ore more suppliers agree to deliver. the negotiation process may be carried out in several iterations with alternating offers and counteroffers. a negotiation agent at manufacturer-side, as well as at supplierside may decide to generate a counteroffer, accept an offer, or abort the negotiation process. in case of a successful negotiation, the results are passed over to the corresponding conversion agents, and subsequently, to the respective ppc system. last, the ppc system of the manufacturer generates an order for the winning supplier. 44 ajis vol. 8 no. 1 september 2000 the whole negotiation process is carried out utilizing existing conversion agents, thereby allowing extensive re-use of definitions (i.e. xml to ppc mappings) and implementations. java runtime environment www parts, dates supplier figure 5: message exchange for procurement communication and security until now we supposed that cooperating enterprises and their software agents are at least technically able to exchange the defined messages. so, those cooperating systems must communicate in a certain way with each other. when looking at real world application systems, this may not hold. especially business application systems provide mainly proprietary interfaces that must be addressed explicitly. utilizing our approach of a multi-agent system with an explicit conversion agent may also help to solve this problem. a publicly known or mutually agreed communication channel must be used by the agent system to establish the first-time communication. no critical business data is transmitted within this communication session. it is used only to negotiate on a secure communication channel for further transactions. we use the xml-based service advertising protocol (xml/serviceap) (fellner & hunger, 2000) for negotiation. the xml tags are based on the un/edefact quote and requote (request for quote) messages. for instance, the enquiry for communication channels is transmitted as xml/serviceap-requote and the corresponding answer is realized as xml/serviceap-quote. to determine all possible communication channels between two co-operating ppc systems the initiating ppc system sends an empty xml/serviceap request (i.e. <xmlserviceap client=' by'></xmlserviceap>). the receiver of this message subsequently delivers the information on all supported channels (see figure 6 for an example of this message). in this scenario the software agents are considered as part of the process initiators, the ppc systems. figure 6 shows an example of a ppc system that supports communication via encrypted http (hypertext transfer protocol) connections (i.e. the secure socket layer protocol (ssl) (nusser, 1998, pp. 124-127), the java cryptography extension (jce) (sun microsystems, 2000), and the (not encrypted) remote method invocation (rmi) (sun microsystems, 1998)). at this stage, the xml messages contain information about possible connections only, e.g. the public key for rsa (popular public key technique developed by rivest, shamir, and adelman) encryption (smith, 1997, pp. 203218). the connection itself is afterwards established using a suitable communication component for the chosen protocol, which will be loaded at run-time (cf. section 5). the described procedure helps to protect critical data from improper use. to add additional security to the submission, the channel may be randomly selected for each new connection. in conjunction with a secure first-time connection this helps to prevent potential internet protocol attacks. the secure first-time connection is important in this context, because all relevant information, like the port, or the used encryption is exchanged. as we now must know something about 45 ajis vol. 8 no. 1 september 2000 the communicating actor's communication protocols, this restricts the utilization of our approach within which we claim the possible communication with unknown suppliers (partners). possible attacks on the internal information systems may be prevented by allowing only checked (i.e. through certificates) or known partners to connect directly. for further information on general security issues when implementing edi with internet protocols we refer to (stein, 1998) or (stallings, 1995). <xmlserviceap client='by'> <protocol caption='jce'> jce:/7192.168.0.2:7002 ?publickey=b0011cb08d8689aao... +algo=rsa +amp=rsa/ecb/pkc s1padding </protocol> <protocol caption='tcp'> tcp://192.168.0.2:7001 </protocol> <protocol caption='rmi'> rmi://marcelpc:1100/rmiservero </protocol> </xmlserviceap> figure 6: example of a ppc system offering different communication channels communicating ppc systems have to support communication via at least one open interface (i.e. the common object request broker architecture (corba) (omg, 1998)). to enable a specific connection, all participating systems must know the specific information for the chosen protocol. table 1 shows the protocols actually supported by the prototype with the mandatory information. protocol / technology mandatory information tcp/ip tcp/ip with ce (encryption) remote method invocation (rmi) object request broker (orb) without name-service orb without name-service • host-name resp. ip-address • port, at which the server process runs • hostname resp. ip-address • port, at which the server process runs • public key, encryption algorithm, -methodsund -padding • url hostname resp. ip-address • port and service provided • host-name resp. ip-address and • service provided • reference number for the provided service (object) table 1: possible protocols with necessary connection information the software agents for inter-organisational coordination as well as the agents for negotiating the communication protocols use the same information infrastructure. they are implemented as multi-agent system. the multi-agent system is based on the contract net paradigm (smith, 1980, p. 1104) and is realized as a manager/contractor contract-net, cf. e.g. (zelewski, 1993, p. 20). an agent represents each actor involved in a contract net. the manager-agent addresses the contractor-agent directly. the contractor-agent in turn answers with an offer that is collected by the manager-agent. the manageragent subsequently chooses the best among all offers handed in and informs each contractor-agent upon the decision. in our application the negotiation agent at the manufacturer-side acts as a manager-agent, the negotiation agents at supplier-side act as contractor-agents. as the conversion agents execute the initial distribution and receipt of enquiries and the final order placement, they are part of the manager/contractor-net in a narrower sense. together with the negotiation agents they fulfil the task of so-called contract processors (kim, 1996, p. 23). 46 ajis vol. 8 no. 1 september 2000 with the use of xml the contract-net protocol (albayrak & bussmann, 1993, p. 61) is easier to set up compared to the standard manager/contract net. it is still necessary that the sequence of messages must be defined, but their content and its arrangement may be a superset of information mandatory for negotiation. this holds, since the negotiation agent only extracts relevant information. additionally, the use of xml allows extensive, cross-organizational re-use of negotiation agents, as the negotiation protocol may be exchanged or adapted easily. in particular, different negotiation protocols may be served by a superset of transmitted information. on the negative side, this information overload means higher network traffic, because each negotiating partner gets all the information independent from the need of the implemented negotiation algorithm. if the orders are placed and the business partners agreed on a integration of their business processes the same multi-agent system may be used for the cross-organizational coordination of the production process, i.e. in case of machine breakdown or failure. if a failure occurs in the production process of one participating actor, a negotiation agent may be instantiated with the actual, changed, delivery terms. the negotiation process stays the same as stated above, apart from the fact, that only the affected supplier is contacted. other suppliers are only contacted, if the negotiation with the actual supplier fails. to coordinate these inter-organizational production processes the well-known procedures for in-house coordination (corsten & gossinger, 1998) are of restricted use. especially, because prerequisites of these procedures, like the information allocation generally do not hold. suppliers may, i.e., only provide detailed information on their production plans (i.e. shift plans), when they are very closely connected to the supplier. furthermore, it may not hold that a solution convenient for the manufacturer causes a positive contribution to the target function of the supplier. in a narrower sense, the multi-agent system therefore does not carry out any planning procedures. rather, it offers additional coordination means for production plans of ppc systems of cooperating parties. another possibility to extend the described approach may be the utilization of alternative variants of the contract-net approach. following (zelewski, 1997), the combination of networks for coordination purposes and market mechanisms appears to be promising. implementation of the framework component all different agents are based on the same generalized kind of software agent. with this, the same information infrastructure can be re-used for different tasks. the underlying system architecture at the manufacturer-side is depicted in figure 7. the implemented prototype is written entirely in java and comprises the individual software agents as well as a rudimentary ppc system for demonstration purpose. the software agents are instantiated and coordinated through an additional software component the framework component. the initialisation of the software agents is done as required by the business process. for instance, the framework component is responsible for instantiating a negotiation agent for each incoming enquiry. the different software agents are executed within a java application and need a java runtime environment (jre). as a java virtual machine is available for all popular operating systems used in business environments, this prerequisite does not restrict the potential implementation environments of the proposed approach. another advantage of java is that the necessary functionality for inter-agentcommunication is already integrated in the java application programming interface (api) (sun microsystems, 1998). in the following we describe some implementation aspects for each software agent of the presented approach. the agents that convert the output of ppc systems are specific to a respective ppc system. supporting an additional ppc system implies the adaptation of existing conversion agent, or the implementation of a new conversion agent. besides this, the only prerequisite for ppc systems to take part in the given scenario is an accessible interface supporting a known output and input format for exchanging data. in a simple case, a conversion agent has to read in a text file created by a ppc system, to extract necessary data, and to create a corresponding xml messages. on the other hand, it is possible that the conversion agent gathers necessary information directly from the ppc system using remote function calls. for instance, one conversion agent in the prototype uses the business application programming interface (bapi) to gather data directly out of the business application system sap r/3 (sap, 1997). generally, the needed communication protocols are implemented as stand-alone components and will be loaded at run-time using the java class-loader facility. this allows the re-use of existing communication components in all agents as well as an easy utilization of the xml/serviceap described in section 4. like the agents converting the ppc output into xml (ppc-to-xml), the agents responsible for converting xml messages in a ppc system specific format (xml-to-ppc) are specific too. but the 47 ajis vol. 8 no. 1 september 2000 effort for implementing this agents is relatively modest. this is, because the parsing of xml messages can be implemented using freely available components, e.g. the xml-parsers from ibm (xml4java) or sun (xmlparser), and the mapping of the output can be done based on the implementation of the ppc-to-xml conversion agent. agent factory framework component agent figure 7: the manufacturer-side system architecture besides the conversion agents, negotiation agents are needed in our approach. the implemented negotiation agents are made up of a knowledge base, a problem solver and a communication component. this architectural pattern is used for all software agents in our approach. the conversion agents described above, for example, consist of a knowledge base containing a simple one-to-one mapping of xml tags to ppc system specific entries (i.e. positions in a comma-separated file), a problem solver implementing the mapping algorithm, and the communication component containing a xml parser and the protocols for the communication with the other agents as well as the ppc system (i.e. the bapi calls of sap r/3). the main difference between the same components of different agents (i.e. the problem solver) lies in the complexity of the supported task (i.e. mapping data vs. complex preference functions). the knowledge base of a negotiation agent contains all incoming offers for one enquiry, the enquiry itself and results of a preceding negotiation within a multi-step negotiation process. the data itself is encoded and stored in xml and can be extracted using the xml query language (xql) (robie, lapp, & schach, 1998). the problem solver (on manufacturer-side) is responsible for finding the best offer among all offers that are handed in. when a new offer arrives, relevant information is passed over to the problem solver utilizing the communication component. subsequently, the problem solver determines the best offer using a preference function. this process stops, when for each enquiry a recline or an offer has been received. additionally, criteria can be formulated on which the process stops immediately (i.e. time consumption). afterwards, the implemented preference function is used to choose the best available offer for a given enquiry. the selected offer may also serve as the starting point for a new round of negotiation. for instance, a new enquiry (based on the former on) is send to the supplier with the best offer containing a new (maybe shorter) delivery time. it is important to mention that this procedure can lead to a situation where a formerly positive offer may be withdrawn resulting in the need for a complete new procurement process. after an offer is accepted for an enquiry, the negotiation process stops and the other participants (negotiation agents, suppliers) are informed by the negotiation agent to cancel their offers. through strict separation of the different components a negotiation agent may be re-used in similar scenarios. for example, a company with different preference functions for products or group of products will re-use the same negotiation agent by adapting or exchanging the problem solver. as the negotiation agent is initiated with enterprise-specific information the same kind of negotiation agent also may be used at the manufacturer-side as well as the supplier-side. outlook and conclusions utilising an efficient and effective information infrastructure for inter-organizational integration is a critical success factor for enterprises following state-of-the-art business strategies. the presented framework architecture helps to improve necessary inter-organizational communication and • • cc • • is a * 48 ajis vol. 8 nov 1 september 2000 coordination. the framework architecture is based on commonly accepted, and, where available, standardized techniques and concepts. the combination of xml with established communication standards helps to bridge the gap between heterogeneous business application systems. the use of open standards, the re-usability of essential software components, and their platform independence allows for an easy adaptation of the framework architecture to enterprise-specific circumstances. especially small and medium enterprises may profit through lower initial costs. above the support of inter-organizational communication, the proposed approach allows a generic reuse of multi-agent systems enabling extensive automation of procurement, coordination of production across organizations, and negotiation on applicable communication protocols and security standards using an identical information infrastructure. the exploitation of the mentioned techniques may be carried out step by step. for example, after an initial use of software agents to improve communication, a multi-agent system can be set up to automate procurement. last, a multi-agent system to coordinate the manufacturing may be set on top re-using the existing components. references albayrak, s., & bussmann, s. (1993). kommunikation und verhandlungen in mehragenten-systemen. in h. j. muller (ed.), verteilte kiinstliche intelligenz: methoden und anwendungen (pp. 5581). mannheim. arnold, o., faisst, w., harding, m., & sieber, p. (1995). virtuelle unternehmen als unternehmenstyp derzukunft? hmd, 32(185), 8-23. bradshaw, j. m. (1997). an introduction to software agents. in j. m. bradshaw (ed.), software agents (pp. 3-46). menlo park: aaai press. bray, t., paoli, j., & sperberg-mcqueen, c. m. (1997). extensible markup language (xml). available: http://www.w3.org/tr/pr-xml.html (1998, 06-12]. corsten, h., & gossinger, r. (1998). produktionsplanung und -steuerung auf grundlage von multiagentensystemen. in h. corsten & r. gossinger (eds.), dezentrale produktionsplanungsund -steuerungs-systeme: eine einfuhrung in zehn lektionen (pp. 174-207). stuttgart: kohlhammer. cover, r. (2000). sgml: general introductions and overviews. oasis-group. available: http://www.oasis-open.org/cover/general.html [2000, 02-30]. fellner, k., & turowski, k. (1999). component framework supporting inter-company cooperation. paper presented at the proceedings 1999 third international enterprise distributed object computing conference (edoc'99), mannheim. fellner, k. j., & bunger, m. (2000). basistechnologie zum betriebsubergreifenden austausch von umweltinformationen fachliche und technische abstimmung. paper presented at the buis 2000 betriebliche umweltinformationssysteme, olten (ch). gaedke, m., & turowski, k. (1999). generic web-based federation of business application systems for e-commerce applications. paper presented at the second international workshop on engineering federated information systems (efis'99), kuhlungsborn. goldfarb, c. f., & prescod, p. (1998). the xml handbook. upper saddle river: prentice-hall. harvey, b., hill, d., schuldt, r., bryan, m., thayer, w., raman, d., & webber, d. (1998). position statement on global repositories for xml. available: ftp://www.eccnet.com/pub/xmledi/repos710.zip [1998, 12-01]. houlihan, j. b. (1992). international supply chain management. in m. christopher (ed.), logistics the strategic issues (pp. 140-159). london. kirn, s. (1996). kooperativ intelligente software. information management, / /(i) , 18-28. kuhlen, r. (1996). informationsmarkt: chancen und risiken der kommerzialisierung von wissen. (2 ed.). konstanz: universitatsverlag konstanz. nusser, s. (1998). sicherheitskonzepte im www. berlin: springer. omg (ed.). (1998). the common object request broker: architecture and specification (revision 2.2): omg. peat, b., & webber, d. (1997). introducing xml/edi: "the e-business framework". available: http://www.geocities.com/wallstreet/floor/5815/start.htm [1998, 12-01]. filler, f., & schoder, d. (1999). mass customization und electronic commerce: eine empirische einschatzung zur umsetzung in deutschen unternehmen. zfb, 69(\0), 1111-1136. pine ii, j. b. (1993). mass customization: the new frontier in business competition. boston: harvard business school press. 49 ajis vol. 8 no. 1 september 2000 rautenstrauch, c., & turowski, k. (1999). a virtual enterprise model for mass customizatioa paper presented at the second world manufacturing congress (wmc'99), international symposium on manufacturing systems (isms'99), durham. robie, j., lapp, j., & schach, d. (1998). xml query language (xql). available: http://www.w3.org/tands/q17ql98/pp/xql.html [1999, 01-01]. sap (ed.). (1997). bapis einfuhrung und uberblick. walldorf: sap. smith, r. e. (1997). internet cryptography. reading, massachusetts: addison wesley longman. smith, r. g. (1980). the contract net protocol: high level communication and controll in a distributed problem solver. ieee transactions an computers, 29, 1104-1113. stallings, w. (1995). network and internetwork security. upper saddle river, new jersey: prentice hall. steel, k. (1997). the beacon user's guide: open standards for business systems. available: http://www.cs.mu.oz.au/research/icaris/beaugl.doc [1998, 12-01]. steffen, t. (2000). internet-quellen zu xml/edi. wirtschaftsinformatik, 42(1), 78-86. stein, l. d. (1998). web security. reading, massachusetts: addison wesley longman. sun microsystems (ed.). (1998). jdk 1.1.6 documentation java development kit. mountain view: sun microsystems. sun microsystems (ed.). (2000). java cryptography extension. mountain view: sun microsystems. tmwg. (1998). reference guide: "the next generation of un/edifact": an openedi approach using uml models & oot (revision 12). available: http://www.harbinger.com/resource/klaus/tmwg/tm010rl.pdf [1998, 12-01]. turowski, k. (1999a). agenten-gestiitzte informationslogistik fur mass customization. in h. kopfer & c. bierwirth (eds.), logistik management intelligente i+k technologien (pp. 199-209). berlin: springer. turowski, k. (1999b). ordnungsrahmen fur komponentenbasierte betriebliche anwendungssysteme. paper presented at the tagungsband des 1. workshops komponentenorientierte betriebliche anwendungssysteme (wkba 1), magdeburg. un. (1995). united nations directiories for electronic data interchange for administration, commerce and transport. available: http://www.unece.org/trade/untdid/welcome.html [1998, 12-01]. zbornik, s. (1996). elektronische markte, elektronische hierarchien und elektronische netzwerke: koordination des wirtschaftlichen leistungsaustausches durch mehrwertdienste auf der basis von edi und offenen kommunikationssystemen, diskutiert am beispiel der elektronikindustrie. konstanz: universitatsverlag konstanz. zelewski, s. (1993). multi-agenten-systeme fur prozefikoordinierung in komplexen produktionssystemen. ein verteiltes problemlosungskonzept auf der basis von kontraktnetzen (arbeitsberichte des seminars fur allgemeine betriebswirtschaftslehre, industriebetriebslehre und produktionswirtschafts, arbeitsbericht 46). koln: universitat zu koln. zelewski, s. (1997). elektronische markte zur prozefikoordinierung in produktionsnetzwerken. wirtschaftsinformatik, 39(3), 231-243. 50 ajis vol. 8 no. 2 may 2001 editorial this is the second issue of volume 8 of the australian journal of information systems. this issue consists of three parts: th first section are general is papers, the second section is dedicated to ethics in information systems, the final section is dedicated to semiotics in information systems. i would like to thank mat warren for his assistance in providing the ethics section and rod clarke for his assistance in providing the section on semiotics. our first paper is entitled a method for reverse engineering of use case realizations in uml and is provided by dragan bojic and dusan velasevic from the university of belgrade. this paper provides a method for recovering elements of the um model including relationships between use cases and class roles. timothy shin, ming-chi lee, the sheng huang and lawrence deng, from tamking university taiwan provide the next paper entitled assessing software quality through visualised cohesion metrics. this paper presents cohesion metrics which aid in cohesion strength and which can be used in case tools to aid software engineers. our next paper is entitled synthesising structuration and institutional theory: three cases is presented by jason thatcher, florida state university, mark strite, university of wisconsin, manolis diakourakis and david kuhlmeier, florida state university. this paper develops a conceptual framework that links institutions and organizations social structures to the use and understanding of information technology in the public sector. mile and lily enders, charles sturt university present our next paper entitled cops, computers, and the right to privacy in the information age: unauthorised access and inappropriate disclosure of information complaints in new south wales. this paper examines the use of information by police and the consequences of unlawful disclosure of such information. chris simpson, swinburne university, presents our next paper entitled it takes more than ethics. this paper looks at changes to ethical guidelines and the reaction of various sections of society to those guidelines. our next paper entitled information warfare and ethics is presented by m. warren, deakin university and w. hutchison, edith cowan university. this paper looks at the ethics os the practice of information warfare both at a national as well as an international level. john weckert, charles sturt university presents our next paper it research and development: should there be control?. this paper looks at the underlying issues surrounding the responsibilities of professional researchers, in particular, those who work in artificial intelligence. mark williams, edith cowan university presents our next paper ethics in research. this paper examines the question of ethics in research through the reporting of a a series of self dialogues. our next section concerns semiotics. an overview entitled studies in organisational semiotics: an introduction is presented by rod clarke, university of wollongong. peter andersen, university of aalborg present the next paper entitled maritime work and communication. thi paper presents findings from a study examining the development instrument interfaces for use in shipping. our next paper is organisational semiosis: integration and separation between system features and work practices, rod clarke, university of wollongong. this paper looks at the application of systemic semiotics to the relationship between system features and work practices. clarissa de souza, cecilia da cuhna, raquel prates and simone barbosa, catholic university of rio de janeiro present our next paper echoes of semiotically-based design in the development and testing of a workflow system. this paper looks at the design and implementation of a web based workflow system, qualitas. our next paper is aspects of text mining from computational semiotics to systemic functional hypertexts by alexander mehler, university of trier. this paper looks at the introduction of text linkage in natural language text mining. our final paper is presented by rob shields, carleton university and is entitled the new knowledge speak: the implications of contested definitions of knowledge and information. this paper presents a critique of the notions of information and knowledge found amongst proponents of knowledge management. i would like to thank lily sob and diniz da rocha for heir continued assistance with the journal. rob macgregor http://www.uow.edu.au/ajis/ajis.html editor microsoft word 13 1 marina.doc ajis vol 13, no. 1 september 2005 269 a capabilities analysis of e-fulfilment businesses: transformation in the logistics industry janice burn & paul alexander school of management information systems edith cowan university 100 jooondalup drive, joondalup, perth wa, australia tel: 618 6304 5244 fax: 618 6304 5988 p.alexander@ecu.edu.au j.burn@ecu.edu.au abstract: this paper reports on the evaluation of practices of 48 leading e-fulfillment suppliers in the uk. initial findings suggest that while a comprehensive model of e-fulfilment is validated these organisations are also providing services not previously recognised as e-fulfilment capabilities by the literature. this leads to the development of a staged model of transformation, which implies that the industry will embrace radical change in some sectors and lead to a new definition of e-fulfillment businesses. more in depth investigation leads to a proposed definition, and a methodology to measure the degree of transformation. keywords: e-fulfilment, logistics, virtual organisations, online retail, industry transformation, knowledge introduction the online retail sector is already an important part of the global economy (table i) and growing rapidly. this in turn has spawned another industry: e-fulfilment, dealing with all associated logistics, warehousing and delivery services. these services are estimated to be worth us$1.006 trillion in the us alone, or 10.1% of their gdp in 2000 (rogers and tibben-lembke 2001). furthermore, 21% of all logistics transactions are expected to be online in 2005, with the long-term possibility that traditional freight companies will ultimately cease to exist (homs, meringer et al. 2001). clearly, not only is online retail a significant industry sector, but this growing area of e-fulfilment is set to lead a transformation in the whole area of storage and delivery services. online retail sales trends (fourth quarter comparisons) – us$b 2000 2001 2002 2003 8.1 11.8 12.6 20.4 table i: growth of us online retail, source (greenspan 2003) this paper explores the concepts which are encompassed in the term e-fulfilment, and presents a model of e-fulfilment activities based on the existing literature. the research study was conducted in three stages. firstly, the authors completed an analysis of efulfilment capabilities of 48 uk-based third party e-fulfilment companies in 2003. the findings from this analysis lead to the development of a model of transformation for the industry as a whole. a more in-depth review of the 48 organisations focused on these extended capabilities and provided insight into future developments within the industry. ajis vol 13, no. 1 september 2005 270 finally, four of these companies agreed to participate in a detailed review of ongoing and future services. the results promise to change the definition of e-fulfilment service providers. they also highlight the impact of e-business on industry strategies and suggest the need to explore multiple planning approaches within an overall vision of transformational alignment (walters, 2004; ash and burn, 2003). the scope of e-fulfilment. the needs of online retailers fulfilment for online retailers is recognised as incorporating physical, information and financial aspects of the value chain from supply through to ultimate delivery, and even product returns (bussiek 1999). in the new context of globalisation and with increasing emphasis on reduction of time frames, the logistics which support the fulfilment process necessitate advanced information management and communication systems (sauvage, 2003). increasingly, the term “e-fulfilment” has been coined to represent this whole process (pyke et al, 2000; punakivi, 2002). prior research identifies the following characteristics within the scope of e-fulfilment (barsh, crawford et al. 2000; pyke, johnson et al. 2000; colin 2001; raman 2001): • picking – selecting product (to be delivered) from warehouse shelves. related issues are location design and picking systems; • packing – specific packaging for delivery of products. this can include breaking original packages and repackaging. often must be customised for each order; • customer service – managing customer queries and complaints; • financial transactions – calculating and including fulfilment costs, and electronically settling these with appropriate organisations; • warehouse costs – associated with product storage; • delivery – systems and delivery alliances; • transport mechanisms and flows – using multiple delivery mechanisms to ensure deliveries arrive on time and undamaged • procurement management – purchasing arrangements automatically (electronically) integrated with fulfilment suppliers, triggering delivery transactions; • management information systems – concerned with integrating and managing all aspects of the process; • front end (ordering) services – which electronically trigger the fulfilment process automatically from a web-purchaser’s mouse-click; • after-sales service – to ensure fulfilment problems are resolved; • returns – manage reverse logistics related to incorrect, damaged or fit-foruse product issues. this must not only ensure convenient and quick return of goods, but often must initiate re-delivery (of the correct goods); • real-time tracking – for management of all pools of product, and also commitment to “promises” made by front-end ordering systems. these are recognised as integrated components that create virtual proximity between etrader and customer and cover everything a company does to satisfy customer demand within an e-commerce market (menezes 1999). ajis vol 13, no. 1 september 2005 271 a model based on capabilities although components may be aligned with existing activities, e-fulfilment requires its own unique supply chains and distribution networks (dimaria 2001). this is integral to the whole concept since the degree of integration between legacy logistics/delivery and supply chain systems must ensure end-to-end coordination and information sharing (strader, lin et al.). this need for partnerships sees the development of dynamic virtual collaboration networks and is spawning active transport-related research into “collaborative transportation management” (ctm) and “collaborative planning, forecasting and replenishment” (cpfr) (esper and williams 2003). in addition, the nature of the product range, customer delivery expectations, and delivery zones have all changed in the e-tailing world and impacted the supply chains which support them (rabinovich and evers 2003). brand new issues such as product returns, last-mile delivery and remote payments must now be addressed by these systems (punakivi, 2002; schulz, 2003; chopra, 2003. figure 1 collapses these concepts into a single diagram, and illustrates the scope of what, in this study, is explicitly termed e-fulfilment, and represents a model for what is currently understood to be an e-fulfilment provider. supply warehouse pick pack delivery returns transport logisticssupply chain end-to-end connected information systems manufacturers customers tightly linked processes figure 1: e-fulfilment capability model this paper reports on the first stage of a longitudinal study into e-fulfilment capabilities and organisational transformation in the logistics industry. the study. a wave of businesses providing third party (specialised) e-fulfilment services is apparent throughout europe and has already gained a significant presence in the uk. this is verified by the annual report produced in the uk by e.logistics magazine and known as the efulfilment index. [e-fulfilment index, 2003]. this provided the basis to examine 48 third party e-fulfilment service providers based in the uk. the group of organisations examined are based in a range of locations throughout england and scotland; mostly in large cities. their business is largely local in scope, though some organisations offer, or even concentrate on, international deliveries. these were analysed with respect to relevant features and activities not only as a means of examining a rapidly developing industry, but as a pointer to the future. this study includes an analysis of capabilities based on the operators’ reported activities and online web ajis vol 13, no. 1 september 2005 272 assessment. it examines these companies’ statements of “unique selling proposition” and desired market as an indicator to their tactical development plans. it also examines capabilities in terms of their degree of transformation. finally, selected companies are examined to determine their current roles and future strategies. study results results from analysis of the 48 uk third party e-fulfilment organisations is presented below. current capabilities the range of capabilities offered by the e-fulfilment providers examined is shown in figure 2. these 13 capabilities are in the following categories, which align with aspects of the model outlined in fig 1: • capabilities for carrying out physical parts of the fulfilment process. these cover delivery aspects (to homes and in an import/export capacity), “kitting” (delivery with associated installation for items such as phones or computers), picking and packing, rework and relabelling (ie. changing the format of the product), and warehousing. thus, companies such as new wave logistics provide tailored solutions for customers’ fulfilment needs, including sophisticated pick & pack systems. • capabilities that link processes. these are enabled by using call centres, and online track/trace capabilities. though not noted as specific capabilities, extensive information system integration is implied by these facilities. jem fulfilment services is typical in offering full call-centre facilities computer telephony integration (cti) and online track and trace. • capabilities that extend fulfilment into the suppliers’ and related providers’ value chains. thus our sample group have wide-spread capabilities to process financial transactions in conjunction with (or even on behalf of) other linked organisations (suppliers and/or customers), direct mail-out functions, product sourcing, “e-tailing” web site hosting, and even technical aspects of web development. intermail, for example, offers sophisticated web-based “storefront” software it calls “shop-in-abox”, which is designed to make it easier for clients to offer products online. this group of capabilities becomes the focus later in this paper, of measures of transformation. ajis vol 13, no. 1 september 2005 273 figure 2: e-fulfilment capabilities figure 3 shows the frequency distribution of the number of separate capabilities offered by each of the sample. it is clear from this that there are a minority of specialists, a large group of providers offering a half or more of the possible capabilities, and 30% of the sample offering all capabilities analysed. we see in fact that many third party providers are seeking to fill all and any aspects of e-fulfilment, which aligns strongly with the capabilities model we present. those not following this model appear to be specialised in alignment with their parent company’s characteristics. 0 2 4 6 8 10 12 14 16 18 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 more number of capabilities offered frequency figure 3. specialists or generalists ajis vol 13, no. 1 september 2005 274 products delivered e-fulfilment providers show preferences in the products they are prepared to handle. this is shown in figure 4. most companies are prepared to fulfil cds, books, clothes, gifts and electronic/electrical. a few specialists exist to service particular product ranges – often tied in with parent-company affiliations. drinks and health products are surprisingly popular. furniture and white goods are also popular with some businesses, perhaps those with a prior history of fulfilment in these areas. from these figures, it appears that e-fulfilment providers like small, uniform sized and neatly packaged products. this makes sense as the logistics of storing and delivering them is not only cheaper (eg. cd’s take little space), but also more predictable (same size products are easier to cost/process model). they are also able to be delivered to through the standard postal system. having said this, involvement with heavy-goods is not avoided by e-fulfilment operators, as furniture and whitegoods fulfilment is a major area of interest. while most of the sampled organisations favour the products already mentioned, few wish to carry food, especially hot food. this is likely due to the need for specialised storage because it is highly perishable, and because it is subject to strict health regulation (barnett and alexander 2003). a few specialist carriers will likely service this market using tight operating parameters and specialised equipment. figure 4. sectors serviced. last mile capabilities and intentions last mile is an area of intense interest with e-fulfilment providers. the capabilities considered (figure 5) all open customer delivery windows, or take advantages of goods reception strategies. slightly more than 50% of the sample have already adopted most of ajis vol 13, no. 1 september 2005 275 these capabilities, or are willing to. but tellingly, almost 50% (almost all the remainder) are considering or are willing to offer these capabilities. clearly, as these organisations feel the economic and operating pressures of last mile delivery, their activities in setting up new last-mile-specific solutions is frenetic. the data strongly indicates that the future direction for these businesses will include the provision of last mile capabilities. warehouse space figure 6 shows that most operations have less than 50,000 sqm of warehouse space, with few operators having storage space up to 1,000,000 sqm. as well, many operators have flexible short term arrangements for expanding warehousing facilities to satisfy immediate needs. because of the short-term turnaround of the deliveries typically undertaken by third party fulfilment operations, warehouses may be effectively replaced by “cross docks”, where goods received from suppliers are re-sorted, often simply in a sheltered dock, and immediately placed into vehicles for scheduled delivery. organisations with large amounts of warehouse space often act as outsourced warehouses for their customers, providing associated benefits of local stocks for customer order fulfilment, product mixing in the distribution system when production is divided among a number of locations, and provision of break-bulk facilities (cooper and davis). again this is noted as a growth area. warehousexchange and whichwarehouse are web-based businesses that provide an unbiased marketplace to find warehouse space. such marketplaces may also result in warehouse specialists acting as fourth-party outsourcers for e-fulfilment businesses. indeed, there is a hint of this already in the data. figure 5. last mile capabilities. ajis vol 13, no. 1 september 2005 276 0 2 4 6 8 10 12 14 0 1 0 0 0 0 0 2 0 0 0 0 0 3 0 0 0 0 0 4 0 0 0 0 0 5 0 0 0 0 0 6 0 0 0 0 0 7 0 0 0 0 0 8 0 0 0 0 0 9 0 0 0 0 0 1 0 0 0 0 0 0 m o re warehouse vol (sqm) frequency figure 6. warehousing capabilities. size and stability of the businesses most businesses in the sample have turnovers within the a$10m to $40m range, with a few much larger (figure 7). these can be considered small to medium sized businesses. some have the backing of, or are subsidiaries of larger parent businesses. e-fulfilment companies also appear to have stable market positions, perhaps by virtue of their links to parent companies, but also by providing needed services. the average age of companies studied is 8 yrs, with some being much older (figure 8). this indicates that quite a few must have evolved from traditional fulfilment/transport/warehouse operations to e-fulfilment operations. ajis vol 13, no. 1 september 2005 277 0 2 4 6 8 10 12 0 10 20 30 40 50 60 70 80 90 100 more turnover a$mill frequency figure 7. analysis of turnover 0 2 4 6 8 10 12 14 0 2 4 6 8 1 0 1 2 1 4 1 6 1 8 2 0 2 2 2 4 2 6 2 8 3 0 3 2 3 4 m o re years of operation frequency figure 8. analysis of age parent organisations while 40% of third party e-fulfilment providers have been set up independently, 60% are subsidiaries of other companies, some very large. the nature of the parent companies’ activities indicates the businesses have been spun-off to gain revenue in the emerging online market, or perhaps to capitalise on skills and assets of the parent organisation. some relationships could also indicate a requirement to act as an internal service arm for other ajis vol 13, no. 1 september 2005 278 parts of the business (though it must be emphasised that the entire sample is conducting open business in a free-market environment). in a few cases, they also indicate an attempt to service part of a vertical market in a specialised area. this relationship may result in inherited strengths, product interests, and assets and is an avenue for further research. web development though almost all e-fulfilment providers are derived from, or are in their own right, established logistics/warehouse/transport operators, 68% are able and willing to provide new services such as web development and hosting facilities. for established logistics providers, this is an interesting development. not only have these companies implemented the online and web-based changes required to be a credible e-fulfilment provider, but they appear to be keen to offer these services to customers, a phenomenon also observed in the e-grocery environment (koster 2002). this data clearly indicates new business opportunities in the industry as a whole; an idea that will be developed later in this paper. transformation of e-fulfilment businesses: a model over the last decade organisations have been forced to re-examine the role of ict as a support tool and accept that it has become a major driver for business change (ash and burn 2003). e-business is no longer optional and has become the standard mode of operating not only in financial services, publishing and retail where we have already seen rapid and profitable advancement, but everywhere business is conducted. indeed new business opportunities have arisen solely based on e-business; and e-fulfilment is one such example. this has been accelerated with the expansion of logistics operations from primarily a national base to international and global services (lemoine and dagnaes, 2003). based on our research discussed above we can see evidence of a staged transformation taking place within the industry which can be related to a model of e-business change identified by (deise, nowikow et al. 2000) – figure 9. this begins with the use of ict within the parent company (typically transportation or warehousing) to enhance distribution channels through some form of e-commerce. it’s followed by the application of ict within and across value chains, and extending into e-fulfilment. this inevitably leads to industry transformation as networks of organisations are formed through extended e-business operations; for example, the move we observe within efulfilment companies towards website design activities. finally, there will be a convergence where many e-fulfilment companies and their offshoots come together and work within the same e-space in virtual environments (burn and barnett, 1999; vlachopolou, 2003; walters, 2004). one example of this type of transformation can be seen in zendor, one of the uk’s best known names in e-fulfilment, who advertise themselves as ‘the total distance selling solution’ offering logistics, marketing services, ebusiness/interactive services, consultancy, customer services and merchandising. as one of their customers now states ‘zendor’s competences are n. brown’s competences. that is what we liked: plus their willingness to think and act as a long term partner and respect that this would be a partnership of equals’. n. brown is a home shopping giant and the parent corporation. zendor now counts in its client base disparate companies such as toys r us, stanley casinos, lloyds tsb ajis vol 13, no. 1 september 2005 279 screentrade and sony. convergence of efulfilment alliances itc as enabler and driver efulfilmen t industry transfor mation value-chain integration into efulfilment distribution channel enhancement strategic organisational change business value physcos knowcos leverage of e-fulfilment activities figure 9: the e-fulfilment transformation model (adapted from deise, nowikow et al. 2000). as e-fulfilment organisations move from left to right across the panorama painted by the transformation model (figure 9), they are likely to gain added value but also to encounter much greater risk. as they move through these stages, they will be looking to exploit means of revenue enhancement, cost reduction and relationship management. this transformation can be viewed as the transformation of a company reliant on physical assets to one, which is solely dependent on knowledge assets. as companies collaborate along their value chains the nature of the industry begins to change as organisations decide to outsource some of their traditional functions and focus only on their core competencies. the term "going to market" will no longer be defined as the way a company enters the marketplace but rather it will characterise the way an integrated group of companies creates a set of cascading values to transform the marketplace into a network of value providers. at this stage companies will make a conscious effort to orient their strategies toward becoming knowledge-based “knowco” or physical goods-based “physco” companies (deise, nowikow et al. 2000). this is not normally a complete transformation but rather an orientation towards one or the other. knowcos will focus on building brands, capturing ownership of the customer-end market relationship, and investing in knowledge-based core competencies such as emarketing and web services development. they may well expand into providing customer knowledge management services to other companies in their marketplace. physcos will become hubs of processing expertise. their success will be based on speed, quality and delivery. degree of transformation as the discussion on last-mile capabilities indicates, new offerings are being considered by ajis vol 13, no. 1 september 2005 280 e-fulfilment organisations all the time. they appear to be in a highly competitive market place with unmet customer demands offering significant commercial opportunities to be meeting them. the rapid uptake of technology, particularly related to mobile and online, is fuelling rapid change; change that is transformational, not just incremental (trappey, et al, 2004). deise’s definitions essentially treat an organisation as providing capabilities, all with the same level of transformation from physco to knowco. we see that a transformation model innately requires a heterogenous set of capabilities. it is also clear that each set of these capabilities contributes differently to the degree of transformation; from those that support purely physco characteristics, such as delivery, to those that are completely concerned with knowco capabilities, such as retail-web-site consulting services. we wanted to measure the degree of transformation of e-fulfilment organisations. thus, we have extended that model to meet these needs (figure 10). in this model we assess the degree of transformation of each of the e-fulfilment organisations’ capabilities. our ongoing research shows that e-fulfilment organisations make conscious decisions about what capabilities they will advance, grow, adopt, shrink and discard. evidence to be presented in this paper shows that these decisions are made as part of the organisation’s strategic planning processes in line with what is anticipated in the future, and tempered by the organisation’s culture and perceived overall capabilities. we propose therefore that degree of transformation is actually made of a portfolio of capabilities, each with a degree of contribution to overall transformation, which consequently determine an organisation’s overall degree of transformation. we focus on capabilities that are externally-facing relative to the e-fulfilment organisation (i.e. not internal) and make the assessments in two dimensions; the transformation associated with the online retailer (the e-fulfilment organisation’s customer), and that associated with the customer’s customers (also referred to as the end-customer). we do this because third party e-fulfilment organisations are acting as intermediaries in the supply chain between the end-customers and the online retailers and it is at the two ends of the supply chain that interaction occurs. by assigning a numerical weighting to the assessment of each capability, we can use the sum of these to quantitatively define (two) measures of transformation for an organisation, even though it has various heterogenous capabilities (figure 10). ajis vol 13, no. 1 september 2005 281 fig 10: two-dimensional model for transformation the following weightings were assigned to each capability for each of the surveyed organisations. (table ii). weighting assessment guideline 0 falls into the traditional fulfilment supply chain 1 fulfilment-related extension of the chain into the business processes of either end-customer or online retailer 2 major extension into online-retail or end-customer business, including outsourcing a complete non-fulfilment function 3 completely non-fulfilment business functions performed for endcustomers or online-retailers table ii: method of deriving weightings aggregating these weightings with separate measures for “customer-facing” and “endcustomer-facing” capabilities generates a plot point for any e-fulfilment organisation. the original deise model refers to “stages” of transformation. while we recognise and agree with this assessment, when aggregating capabilities it is quite possible that while the total capability portfolio will place a company clearly into one of the 4 stages of transformation, contributing capabilities may actually relate to other stages. rather than complicate the model we opt instead to represent the degree of transformation as an index of a baseline value, which is the current maximum for all organisations currently, surveyed. we give this maximum the value of 100 in each dimension. importantly, this approach allows us not only to measure companies against each other, but also to examine them increasing transformation impact of the sum of the capabilities geared towards online retailer’s requirements capabilities geared towards end-customer’s requirements physc knowc 0 100 100 ajis vol 13, no. 1 september 2005 282 individually or as groups longitudinally over time. knowledge capability ratings 2003 0 10 20 30 40 50 60 70 80 90 100 0 10 20 30 40 50 60 70 80 90 100 e-tailer facing e n d -c u s to m e r f a c in g customer outsource specialists generalists e-tailer outsource specialists physcos knowcos figure 11: knowco ratings for capabilities from this, it appears organisations have followed 3 distinct paths of degree of transformation: • they confine their transformation to e-tailor-facing capabilities; • they confine their transformation to end-customer-facing capabilities; • they develop transformation in both dimensions. adapting to changes in environment we surveyed a subset of the target companies in depth. we were particularly interested in the companies’ perceptions of near-future e-fulfilment challenges and opportunities, whether they were developments of physco or knocow capabilities, and how they were preparing for these. the results (table iii) suggest two relatively discrete approaches to the future environment, depending on whether the companies are general outsourcers (go) (representing 30% of e-fulfillers), those providing a full range of capabilities as defined by our e-fulfilment definition, or niche outsourcers (no), targeting a specific market. whichever category they might fall in, they appear to be comfortable in that approach, with no intentions to alter their customer or geographic focus. ajis vol 13, no. 1 september 2005 283 item general outsourcers niche outsourcers 1 changes in business focus none 2 geographical ranges serviced (next 2 years) incremental increases 3 perception of industry expansion (next 2 years) most respondents anticipate growth, with about half suggesting significant growth 4 development of capabilities hi tech, flexibility, expertise across a range of capabilities, experience, expanding range of capabilities sector knowledge, value-add capabilities 5 new capabilities (next 2 years) customer-facing activities internal improvements (transport, etc) 6 online services being supplied increased significantly in 2 years not increased much in the next 2 years 7 electronic integration with customers and customers’ customers high and increasing over next 2 years moderate or not very important 8 web site significance seen as highly significant be almost all respondents 9 web development outsourcing most outsource 10 intention to increase web expertise (next 2 years) moderate to major increases minor levels only 11 biggest concerns (next 2 years) competition from substituted services (i.e. bricks & mortar), postal infrastructure issues, security & fraud postal infrastructure issues, security & fraud 12 main opportunities (next 2 years) improved customer interfaces, improvements to delivery infrastructure, online processing table iii: future intentions the data indicate strong pressures to move to knowco capabilities in the gos, with some, but much less perceived imperative, in the nos, who are more focused on relationships with customers in their sector. this behaviour in turn appears to drive the nature of development of the capabilities. the gos are developing new customer-facing capabilities while the nos are more intent on improving internal processes and infrastructure of existing capabilities. regardless of their degree of specialisation, they are using, and recognise, the importance of web facilities for their organisations, though it is the gos who see online services as more significant. they are also more intent on increasing their expertise and developing these capabilities. interestingly, despite the importance of these services, most organisations are comfortable in outsourcing these. while this is understandable in ajis vol 13, no. 1 september 2005 284 nos, who do not appear to regard them as so central, it is unexpected, given the gos’ intended direction. in terms of future environments, e-fulfilment organisations are anticipating new opportunities generated by new software applications and by delivery infrastructure enhancements (often relating to the postal service). when it comes to threats, while all respondents are concerned with security aspects (more particularly, online security issues), gos are also concerned with substitutable products, which aligns with earlier noted intentions to enhance customer-facing capabilities. discussion these organisations all targeted online retailers, either primarily, or as a recognised sector of their market. (duffy and dale 2002) recognise 10 critical support processes required for success by such retailers, and e-commerce operators in general (table iv). of these 10, eight align with capabilities exhibited in our sample (the remaining two cannot be evaluated at this stage). they all have tight end-to-end electronic systems to control their processes, and many provide facilities call centres, payment systems and online components to integrate their services with those of the online retailers. e-commerce critical success capability, from (duffy and dale 2002) e-fulfilment operator capabilities in the sample order fulfilment yes revenue generation no revenue collection last mile capability financial control no it/web changes yes business processes significant integration e-integration yes order generation integrated call centre integration yes 24/7 operation enabler consumer behaviour influences customer experience table iv: critical e-commerce success and e-fulfilment capabilities this confirms the critical role of ict and supporting enterprise systems infrastructure (sauvage, 2003; trappey et al, 2004). in most cases, this integration appears to be to the point of providing a single outsourced solution for online retailers’ e-fulfilment needs, which in turn appears to result in offerings of diverse sets of services, rather than specialists in particular services. however, there is significant evidence of the e-fulfilment organisations themselves outsourcing certain of their functions to specialists; particularly noticeable with web site aspects of the businesses. in these cases, partner organisations provide those services – likely unnoticed by the ultimate customers. there is strong evidence to suggest that the successful e-fulfilment organisation of the future will be one where online retailer, the e-fulfilment partner, and the specialist e-fulfilment service provider are all cooperating as a virtual organisation (walters, 2004). ajis vol 13, no. 1 september 2005 285 our research has identified trends and activities within the uk logistics industry, and connected them to a capabilities model, which attempts to resolve what “e-fulfilment” actually is. while our results show clear alignment with that model, they also generates a series of questions related to parent-company influence, specialisation incentives, details of online offerings, and other aspects as highlighted in the paper. in assessing degrees of transformation, we have expanded the deise model to provide a two-dimensional measurement of companies movement towards either a knowledge based capability or physical processing model. the next stage of the study involves a comparison over three years of transformation followed by a series of case studies, which will be developed through in-depth interviews with 8-10 of these operators. references ash, c. g. and j. m. burn (2003). assessing the benefits from e-business transformation through effective enterprise management. european journal of information systems vol.12 (4): 297-308. barnett, m. and p. alexander (2003). can e-grocers survive the last mile? we-b conference, perth, western australia. barsh, j., b. crawford, b. and brosso, c. (2000). how e-tailing can rise from the ashes. mckinsey quarterly 2000(3): 98-109. burn, j. m. and barnett, m. (1999). communicating for advantage in the virtual organisation. ieee transactions on professional communications, 42 (4), 1-8. bussiek, t. (1999). the internet-based supply chain new forms of procurement utilising standard business software. electronic markets 9(3): 147-152. chopra, s. (2003). designing the distribution network in a supply chain. transportation research 39e(2): 123-130. colin, j. (2001). the impact of e-commerce on transport. joint oecd/ecmt seminar, paris. cooper, j. c. and m. davis (1984). why have a warehouse? retail & distribution management 12(5): 66-76. deise, m. v., c. nowikow, king, p. and wright, a. (2000). executive's guide to ebusiness, pricewaterhousecoopers. dimaria, f. (2001). faqs: is international e-commerce really for you? world trade 14(9): 36-38. duffy, g. and b. g. dale (2002). e-commerce processes: a study of criticality. industrial management & data systems 102(8/9): 432440. e-fulfilment guide (2003).spice court publications. esper, t. l. and l. r. williams (2003). the value of collaborative transportation management (ctm): its relationship to cpfr and information technology. transportation journal 42(4): 55. greenspan, r. (2003). more spending more. clickz stats retailing, clickz network. homs, c., meringer, j. and rehkopf, f. (2001). europe's online logistics push, forrester. august. klaus, p. (1998). supply chain management. weisbaden, gabler. koster, r. b. m. d. (2002). distribution structures for food home shopping. international journal of physical distribution & logistics management 32(5): 362. lemoine, w. and dagnaes, l. (2003). globalisation strategies and business organisation of ajis vol 13, no. 1 september 2005 286 a network of logistics service providers. international journal of physical distribution & logistics management 33(3): 209-228. menezes, j. (1999). baan reinvents self; erp to e-fulfilment. computing canada 25(44): 1-2. punakivi, m. (2001). solving the last mile issue: reception box or delivery box? international journal of physical distribution and logistics management 31(6): 427. punakivi, m. (2002). increasing the cost efficiency of e-fulfilment using shared reception boxes. international journal of retail & distribution management 30(10): 498-507. pyke, d., johnson, m. e. and desmond, p. (2001). e-fulfilment it's harder than it looks! supply chain management review, 5 (1), 26-33. rabinovich, e. and p. t. evers (2003). product fulfilment in supply chains supporting internet-retailing operations. journal of business logistics 24(2): 205. raman, p. (2001). meeting the demand for e-fulfilment service. computimes malaysia. rogers, d. s. (2002). reverse logistics: trends & practices. sao paulo, centre for logistics management. sauvage, t. (2003). the relationship between technology and third-party providers. international journal of physical distribution & logistics management 33(3): 236253. schulz, d. (2003). going the "last mile". traffic world, newark. shaw, m., strader, t. j. and lin, f. r. (1999). the impact of information sharing on order fulfillment in divergent differentiation supply chains. journal of global information management 7(1): 16-25. trappey, a. j. c., trappey, c. v., hou, j. l. and chen, b. j. g. (2004). mobile agent technology and application for online global logistic services. industrial management & data systems. 104 (2), 169-183. vlachopoulou, m. and manthou, v. (2003). partnership alliances in virtual markets. international journal of physical distribution & logistics management 33(3): 254267. walters, d. (2004). new economy new business models new approaches. international journal of physical distribution & logistics management 34 (3/4): 219-229. williams, l. r., esper, t. l. and ozment, j. (2002). the electronic supply chain, international journal of physical distribution & logistics management 32(85): 703-719. microsoft word ajis 11 1 final.doc ajis vol.11 no. 1 september 2003 19 a critical analysis of the knowledge creation process stephen k. probert computing and information systems management group cranfield university rmcs shrivenham swindon sn6 8la u.k. stephenkprobert@aol.com keywords knowledge creation, knowledge management, performativity, subjectification abstract the topic of knowledge creation (nonaka and takeuchi, 1995) will be analysed from a philosophical perspective. to begin with, the key motivations behind the discourse of knowledge creation and management are briefly discussed. the contemporary distinction between tacit and explicit knowledge is then explored. following this, the work of lyotard on knowledge in a (post)modern economy will be summarised. the paper then explores how some of the work by foucault and adorno may provide a new direction for developing research approaches in this area. introduction: why “knowledge management”? in the last ten years or so, an explosion of literature on knowledge management has occurred. this literature is nearly all managerialist in tone, and it is underpinned by a belief in the competitive advantage that can be obtained from the exploitation of knowledge – both for companies and countries – in the developed world; a typical argument being: “the long-predicted ‘information society’ and ‘knowledge economy’ are now emerging as tangible realities. leading management theoreticians argue that it is much more profitable for a company to invest a given sum in its knowledge assets than to spend the same amount on material assets.” (probst et al., 2000, p. 3) the challenge is, then, to both to create new knowledge and exploit existing knowledge (within a firm) more aggressively than hitherto. nonaka and takeuchi’s (1995) work contains one of the seminal accounts of these processes, and provides prescriptions for contemporary managers of competitive firms – concerning how to both create and exploit knowledge. they argue that, in order to persist, companies must perpetually offer competitive new products and services; moreover, “years of research on japanese [and other western] firms … convinces us that knowledge creation has been the most important source of their international competitiveness.” (nonaka and takeuchi, p. viii). as the marketplace is conceived of as being dynamic it follows that new knowledge is constantly needed for the existence of a company to be sustained, “by organisational knowledge creation we mean the capability of a company as a whole to create new knowledge, disseminate it throughout the organisation, and embody it in products, services and systems… the goal of this study is to formalise a general model of organisational knowledge creation.” (ibid., pp. iiv-ix). this approach and that of many others in the knowledge management field – hypostatises (and, no doubt, encourages) an unprecedented desire for knowledge within the firm. as fuller (2002) points out, this has little to do with a (quaint?) curiosity-based desire for knowledge. the knowledge management literature is focussed on the needs of competitive firms (or perhaps their shareholders), “the realisation that knowledge is the new competitive resource has hit the west like lightning. but all this talk about the importance of knowledge – for companies and countries – does little to help us understand how knowledge gets created.” (nonaka and takeuchi, p. 7). philosophically, this is significant. epistemology-to-date has been a largely regulative activity; most philosophers have concerned themselves with the question of how to evaluate a knowledge-claim (e.g. astronomy v astrology), and have largely left the generative aspects alone. the knowledge management literature stresses the generative aspects and leaves the regulative aspects largely untouched – indeed, the implication is that these problems have been solved. at times, one gets the distinct impression that ajis vol.11 no. 1 september 2003 20 as long as useful stuff is produced, then debates concerning the truth of this useful stuff are relegated to being (economically costly) scholasticism. there are profound implications of such a view, but further discussions of these lie outside the scope of this paper. to some extent, it can be argued that – from a critical perspective lyotard pre-empted these arguments in the seventies. lyotard’s contribution will be summarised shortly. one of nonaka and takeuchi’s key arguments is that knowledge falls into several categories; the primary distinction they draw is between tacit and explicit knowledge. these distinctions will now be discussed: tacit and explicit knowledge here is an “official” statement of this distinction: “tacit knowledge is personal, context-specific and therefore hard to formalise and communicate. explicit or ‘codified’ knowledge, on the other hand, refers to knowledge that is transmittable in formal, systematic language… therefore scientific objectivity is not a sole source of knowledge. much of our knowledge is the fruit of our own purposeful endeavours in dealing with the world…” (nonaka and takeuchi, 1995, pp. 59-60) explicit knowledge need not be subjective – and may reside in databases, written reports, etc. tacit knowledge is further sub-divided into two – not entirely discrete – categories: “tacit knowledge includes cognitive and technical elements… mental models [the cognitive elements] such as schemata, paradigms, perspectives, beliefs, and viewpoints, help individuals to perceive and define their world. on the other hand, the technical element of knowledge includes concrete know-how, crafts and skills. it is important to note here that the cognitive elements of tacit knowledge refer to an individual’s images of reality and visions for the future, that is, ‘what is’ and ‘what ought to be’.” (ibid., p. 60) it should be noted that technical skills are primarily bodily skills. lyotard and the performativity principle lyotard was one of the first people to link knowledge-production to economic well-being in a systemic way, and – in some ways – the knowledge management literature is a sort-of “joyful” extension of this thesis: “there is no denying the dominant existence today of techno-science, that is the massive subordination of cognitive statements to the finality of the best possible performance, which is the technological criterion. but the mechanical and the industrial, especially when they enter fields traditionally reserved for artists, are carrying with them much more than power effects. the objects and the thoughts that originate in scientific knowledge and the capitalist economy convey with them one of the rules which supports their possibility: the rule that there is no reality unless testified by a consensus between partners over a certain knowledge and certain commitments. this rule is of no little consequence. it is the imprint left on the politics of the scientist and the trustee of capital by a kind of flight of reality out of the metaphysical, religious and political certainties that the mind believed it held. this withdrawal is absolutely necessary to the emergence of science and capitalism.” (lyotard, 1984, pp. 76-77). however, as already stated, nonaka and takeuchi make a distinction between tacit and explicit knowledge; of these, “…[the] more important kind of knowledge is tacit knowledge.” (ibid., p. viii). interestingly, if nonaka and takeuchi are taken seriously (and the management literature appears to do so!) then for critical purposes – tacit knowledge is where the attention should be focused. however, much of lyotard’s argument relates more-or-less entirely to explicit knowledge; tacit knowledge is not considered to be predominantly textual. consequently – in this context the ajis vol.11 no. 1 september 2003 21 critical solution to the problems of the performativity-principle, advocated by lyotard, would no longer be tenable: “we are finally in a position to understand how the computerisation of society affects this problematic. it could become the ‘dream’ instrument for controlling and regulating the market system, extended to include knowledge itself and governed exclusively by the performativity principle… but it could also aid groups … by supplying them with the information they usually lack for making knowledgeable decisions. the line to follow for computerisation to take the second of these paths is, in principle, quite simple: give the public free access to the memory and data banks. language games would then be games of perfect information at any given moment.” (lyotard, 1984, p. 67). a critical theory of tacit knowledge dissemination will need to take a very different approach. furthermore, lyotard has little to say concerning the active management of the knowledge-creation processes in a site such as competitive firm. active management of the dynamic knowledge creation process what specific interventions are involved in the creation of knowledge in the knowledge management literature? a supercilious answer might be that there as many answers to this (sort of) question as there are books on knowledge management! nonaka and takeuchi go to some lengths to explicate their theoretical and pragmatic assumptions, and these help to throw light on the issues which must be grappled with. to begin with we might ask, ‘how can knowledge be created at all?’. of course, it is commonsensical enough to state that knowledge-generation doesn’t “just happen”, but it also seems commonsensical enough to assume that knowledge is generally discovered rather than created. the key here is to note that the use of the term ‘created’ implies an active process (in fact, various dynamic processes are advocated): “in our theory of organisational knowledge creation, we adopt the traditional definition of knowledge as ‘justified true belief.’ it should be noted, however, that while traditional western epistemology has focussed on ‘truthfulness’ as the essential attribute of knowledge, we highlight the nature of knowledge as ‘justified belief’… while traditional epistemology emphasises the absolute, static, and nonhuman nature of knowledge, typically expressed in propositions and formal logic, we consider knowledge as a dynamic human process of justifying personal belief toward the ‘truth’.” (nonaka and takeuchi, 1995, p. 58) truth is – most likely in scare-quotes because the justification for the “stuff” generated will have to be grounded in the future; in consumer acceptance of the products (services, etc.), produced, as a result of the “knowledge” generated, in the competitive marketplace – rather than being grounded on any direct evidence of truthfulness. in a sense, this is perhaps the ultimate conclusion of the justification-by-performativity argument put forward by lyotard (1984). what is different is that the exploitation (or liberation) of explicit knowledge is no longer deemed sufficient (or even central). furthermore, the processes involved in active management of the dynamic knowledge creation process (i.e., knowledge management) have largely escaped critical attention hitherto. to critically examine the tacit dimension, other approaches may yield important insights; one approach being that foucault’s analysis of pastoral power – insofar as it provides the beginnings of a genealogical approach to the study of the power / knowledge relations intrinsic in the typical contemporary descriptions and prescriptions of (tacit) knowledge management in competitive firms. foucault’s conception of pastoral power will now be explored. ajis vol.11 no. 1 september 2003 22 foucault’s conception of pastoral power and knowledge management foucault characterised this form of power thus: “this form of power applies itself to immediate everyday life which categorises the individual, marks him by his own individuality, attaches him to his own identity, imposes a law of truth on him which he must recognise and which others have to recognise in him. it is a form of power which makes individuals subjects... the modern western state has integrated into a new political shape, an old power technique which originated in christian institutions. we can call this power technique the pastoral power... this form of power cannot be exercised without knowing the inside of people’s minds, without exploring their souls, without making them reveal their innermost secrets. it implies a knowledge of conscience and an ability to detect it.” (foucault, 1982, pp. 212-214) one of the main techniques of (old) pastoral power was the religious confession, vital for obtaining a deep knowledge of the subjects (their intentions, aspirations, secrets, etc.). the original aim of pastoral power (and its associated confessional technology) was religious salvation. of course, in western (mainly) secular societies, religious salvation may have lost its traditional significance, however foucault argues that pastoral power, as a form of power, is still prevalent today but in other guises: “we may observe a change in its objective. it was no longer a question of leading people to their salvation in the next world, but rather ensuring it in this world. and in this context, the word salvation takes on different meanings: health, well-being, (that is sufficient wealth, standard of living), security, protection against accidents. a series of “worldly” aims took the place of the religious aims of the traditional pastorate...” (foucault, 1982, p. 215) it is in the appeal to secular salvation that the legitimisation of the active management of knowledge creation processes is grounded. furthermore, these are processes which transcend the normal boundaries of management understood as (an aspect of) traditional labour-capital relations this is discussed further below. foucault traces the genealogy of such a conception back to ancient hebrew, greek, and roman civilisations, a key notion in the themes traced is that of the shepherd: “i just want to show a few themes typical of pastoral power... the shepherd gathers together, guides, and leads his flock… what the shepherd gathers together is dispersed individuals. they gather together on hearing his voice: “i’ll whistle and will gather them together.”... in other words, the shepherd’s immediate presence and direct action cause the flock to exist... the theme of keeping watch is important. it brings out two aspects of the shepherd’s devotedness. first, he acts, he works, he puts himself out, for those he nourishes and who are asleep. second, he watches over them. he pays attention to them all and scans each one of them. he’s got to know his flock as a whole, and in detail. not only must he know where good pastures are, the season’s laws and the order of things; he must also know each one’s particular needs... the shepherd’s power implies attention paid to each member of the flock.” (foucault, 1988a, pp. 61-63) the senior managers of knowledge-creating companies (in theory) display many of these features, as their role is to leverage both the tacit and explicit knowledge, generated at lower levels of the organisation, for competitive advantage: “the basic role of knowledge officers, who are the senior managers of a company, is the management of the total organisational knowledge-creation process at corporate level… knowledge officers should be aware that their aspirations and ideals determine the quality of knowledge the company creates. while the ideals of top management are important, on their own they are not enough; they need to foster a high degree of personal commitment by other members of the knowledge creating crew. to do so, an open-ended and equivocal ajis vol.11 no. 1 september 2003 23 vision, which is susceptible to a variety of interpretations, is preferable. a more equivocal vision, which is susceptible to a wide variety of interpretations, is preferable. a more equivocal vision gives members of the self-organising team the freedom and autonomy to set their own goals, making them more committed to figuring out what the ideals of the top really mean.” (nonaka and takeuchi, pp. 156-157.) the management of knowledge-creation cannot be achieved using old-fashioned disciplinaryhierarchical management techniques. managing the “flock” of potential knowledge creators involves a good number of classic foucauldian power/knowledge themes particularly “subjectification” – ways in which people turn themselves into subjects, which has been explained by paul rabinov thus: “foucault’s third mode of objectification represents his most original contribution. let’s call it ‘subjectification.’ the process differs in significant ways from the other two modes… the dividing practices, broadly speaking, are techniques of domination … the interplay between these modes of domination and various social scientific form of classification, although given new clarity and power by foucault’s analysis and historical studies, has been recognised by other thinkers… in contrast, with the third mode, ‘subjectification’ – foucault looks at those processes of self-formation in which the person is active.” (rabinov, 1984, pp. 10-11 [emphases added]) the tacit knowledge creation process requires that the actors are dynamically self-forming to produce their own tacit knowledge – which can then be passed on (by various methods) to other members of the organisation for commercial exploitation, “let us start with the ontological dimension. in a strict sense, knowledge is created only by individuals. an organisation cannot create knowledge without individuals. the organisation supports creative individuals or provides contexts for them to create knowledge.” (nonaka and takeuchi, 1995, p. 59). however, to exploit such knowledge it will be necessary for it to be made available to others in the organisation, “the explanation of how japanese companies create new knowledge boils down to the conversion of tacit knowledge into explicit knowledge. having an insight or a hunch that is highly personal is of little value to the company unless the individual can convert it into explicit knowledge.” (nonaka and takeuchi, p. 11) . the self-forming activities the employee must undergo take place in several modes, both bodily and cognitive, as tacit knowledge can be created in both ways (as outlined earlier). this is essentially the process of subjectification identified by foucault, “this selfformation … takes place through a variety of ‘operations on [people’s] own bodies, on their own souls, on their thoughts, on their own conduct’.” (rabinov, 1984, p. 11). part of the requisite conduct-management will be the “pastoral” function of converting tacit (individual) knowledge to explicit (socialised) knowledge. a critical analysis of the knowledge-creation process nonaka and takeuchi (1995) provided four models of knowledge “conversion”. these will now be summarised. tacit to tacit the process described for this conversion is socialisation. tacit knowledge is gained from experience and this can be passed on to others. bodily training is as important as anything cognitive in this process. this bodily training is largely to be self-initiated, and requires adequate pastoral arrangements / incentives, etc. it should be noted that (in theory) disciplinary procedures play no (or very little) part in this process. ajis vol.11 no. 1 september 2003 24 tacit to explicit the process described for this conversion is called externalisation. this process is essentially one of making the (largely) bodily knowledge textual, although it will not always be possible to directly express this knowledge in prose or diagrams, “ when we cannot find an expression for an image through analytical methods of deduction or induction, we have to use a nonanalytical method. externalisation is, therefore, often driven by metaphor and/or analogy.” (nonaka and takeuchi, 1995, p. 65). however, to be ultimately useful, such knowledge will need to be codified in fairly precise language. there may be considerable scope for further critical research concerning this mode of conversion explicit to explicit this process is described (rather unfortunately, perhaps) as combination. any cognitive learning from primarily textual sources (e.g. databases) falls under the rubric of ‘combination’. interestingly, lyotard’s (political) plea to “give the public free access to the memory and data banks” seems to have been taken on board within the boundaries of the knowledge creating company. discussing the kao corporation (in japan), nonaka and takeuchi note the following: “to assure ‘free access to information,’ computer systems have been introduced throughout the kao organisation., with all information being filed in a database. through this system, anyone at kao can tap into databases included in the sales system, the marketing information system (mis), the production information system, the distribution information system, and the total information network covering all of its offices in japan. the unique feature of this system is that any member, no matter what his or her position or to what section she or he belongs, within the business system, has full access to the database (except for a limited amount of personal information). in other words, anyone can get access to the rich base of explicit knowledge that exists within the business system through this ‘free access to the information system’.” (nonaka and takeuchi, 1995, p. 172). in a way, this is striking no (or little) risk is perceived as a result of this policy; indicating a considerable discontinuity between this approach and the older “disciplinary” approaches to management (where security and “need to know” issues are paramount). explicit to tacit the process described for this conversion is internalisation. this is hard to describe – nonaka and takeuchi suggest “learning by doing” (p. 69-70). once again, the body is centrally involved, as it learns to behave in ways formally written down. explicit operations on the body are central here, as this example shows: “an example of internalisation through ‘learning by doing’ can be seen at matsushita when it launched a companywide policy in 1993 to reduce working time to 1800 hours… the policy’s objective was not to reduce costs but to innovate the mindset and management by reducing working hours and increasing individual creativity. many departments were puzzled about how to implement the policy, which was clearly communicated as explicit knowledge…[they] advised each department to experiment with the policy for one month by working 150 hours. through such a bodily experience, employees got to know what working 1800 hours a year would be like. an explicit concept, reducing working time to 1800 hours, was internalised through the one-month experience.” (nonaka and takeuchi, 1995, p. 70 [emphases added]) in this mode, written prescriptions will be internalised as bodily activities, processes, etc. 6.5 the four processes ajis vol.11 no. 1 september 2003 25 these four modes of knowledge conversion very explicitly link operations on the body and the mind. the settings in which these are to take place hardly resemble the “disciplinary” organisation (of the past), however there are strong indications that subjectification processes should be occurring. discussion and conclusion it is obvious that the knowledge creating company is (at least in theory) a very different beast from the disciplinary organisation. many of foucault’s later themes on power and subjectivity certainly seem relevant to analysing the power effects of knowledge creating companies on those who are employed within them, but it must be doubted as to whether these analyses could go beyond thematic conclusions. the most important examples given by nonaka and takeuchi (1995) are japanese companies – genealogical studies have been mostly european. whereas companies based in the usa follow many social patterns observed in european companies and there are considerable similarities in the histories of north america and western europe at the ideological / genealogical level very little (genealogical) material from japan is available. nor – until very recently – has there been a great deal of cross-fertilisation (of ideologies, social structures, etc.) between europe / north america and japan – although a review of the american influence (on japan), immediately post 1945, would seem worthwhile. nevertheless, genealogy concerns itself with what it is that makes management styles (etc., etc.) readily acceptable to people. in this respect many of foucault themes will be worth revisiting (particularly those on subjectification) if the styles of management advocated by nonaka and takeuchi prove to be more than fleeting. but will they? already the knowledge management displays many aspects of the bandwagon/fad effect of much prescriptive modern mismanagement literature. this study has focussed on nonaka and takeuchi’s (1995) work precisely because if it were to be widened to take into account the multifarious approaches to the topic of knowledge management, available in the literature, it might well prove impossible to do any serious analysis at all. moreover, some knowledge management literature is now almost totally focussed on it. at this point, it is worth mentioning that nonaka and takeuchi only include one extended discussion on is (there is nothing on it) in the whole of the 1995 book, and this discussion has been (largely) included in section 6.3 above. assuming that nonaka and takeuchi’s work is durable (for the moment), a focus on technology seems to miss all the homilies about the importance of tacit knowledge – especially its bodily character. furthermore, in this respected, it can be argued that the changes in management, seemingly advocated by nonaka and takeuchi, can appear to be (proposals for) changes in relations of production in european / north american firms, rather than changes in the forces of production (adorno, 1968). moreover, it is the illusion that these proposed changes in relations of production – are (critically) substantive that appears to give rise to the some of the enthusiastic academic receptions of these ideas in europe / north america, whereas – on the contrary – it is precisely because they are merely extensions / formalisations / etc. of techniques of management that have been gathering momentum since at least the 1960s that probably accounts for the enthusiasm shown by managers for these ideas. prima facie, that the body must be re-invigorated as a productive force – even in the most sedentary of occupational settings – would be a management prescription that would hardly have surprised michel foucault. further research is underway by the author to locate nonaka and takeuchi’s work in a wider discussion of the shift of emphasis taking place at the present – from constant capital enhancement (e.g. new it) to variable capital utilisation-enhancement (e.g. knowledge creation and management). references adorno, t. w. “is marx obsolete?,” diogenes (64), 1968, pp. 1-16. foucault, m. “the subject and power,” in michel foucault: beyond structuralism and hermeneutics (1st ed.), h. l. dreyfus and p. rabinov, brighton: harvester, 1982, pp 208-226. lyotard, j. (tr. bennington, g. & massumi, b.) the postmodern condition, manchester: manchester university press, 1984. nonaka, i. and takeuchi, h. the knowledge creating company, new york: oxford university press, 1995. ajis vol.11 no. 1 september 2003 26 probst, g., raub, s. and romhardt, k. managing knowledge, chichester: wiley, 2000. rabinov, p. “introduction,” in the foucault reader, p. rabinov (ed), harmondsworth: penguin, 1984. ajis vol 9 no. 1 september 2001 a technology acceptance model for inter-organisational electronic medical records systems jocelyn handy1, richard whiddett2 and inga hunter2 'school of psychology and 'department of information systems massey university, palmerston north, new zealand email: r.j.whiddett@massey.ac.nz abstract this article reports the findings of the first stage of an ongoing, longitudinal study into the implementation of an interorganisational electronic medical records (emr) system. the study adapted and expanded davis' (1993) technology acceptance model (tam) to investigate the attitudes of primary care practitioners towards a proposed system for maternity patients. all doctors and midwives holding maternity care contracts with a large urban hospital in new zealand were sent a questionnaire soliciting their views on a planned emr system linking the hospital and the primary care sectors. the results showed that whilst davis' two key factors of perceived ease of use and perceived usefulness were important to medical professionals, another key factor, perceived system acceptability, which concerns control and management of information is vitally important to the acceptance of the system. the study also showed that the two groups of professionals had differing requirements due to different levels of experience and practice computerisation. finally, the research highlights a number of wider organisational issues particularly relevant to the use of inter-organisational systems in general and healthcare systems in particular. introduction for some time now the health sector in developed countries has been seeking to exploit the use of information technology to improve both the quality of care it provides for patients and its organisational efficiency. within australia and new zealand two key strategies have been the introduction of electronic medical records (emr) systems and improved electronic communications links between the primary care sector and secondary care institutions (nhimac, 1999; ministry of health, 1996,2000). an emr is an integrated database system that consolidates all information relating to the healthcare of an individual patient. its aim is to improve the quality of patient care by ensuring that complete and up to date information is readily available to all caregivers (dick and streen, 1997). if the system also consolidates financial and costing information relating to the treatment of the patients, then better financial control may also be achieved. to date, most emrs have been located in either the secondary, or hospital-based, sector or within gp practices within the primary care sector. communication between the two sectors has generally been limited to the traditional mechanisms of letters or telephone calls between doctors. this approach has often been criticised for being slow and error-prone (dansky et al., 1999; sobel, alverson and lei, 1999). recently, there have been moves to link emrs across the two sectors in order to ensure a smoother and more efficient transmission of patient related information between sectors (ministry of health, 1996, 2000). whilst studies have shown that emrs do increase the accuracy and accessibility of patients' records, large scale health information systems have often been fraught with problems (hannah, 1998: sicotte et al., 1998). a classic example of this occurred within the british national health service when doctors boycotted an inter-organisational network designed to improve the exchange of information on the grounds that it threatened patients' privacy (davies, 1996; willcox, 1995). clearly, user acceptance of planned networks is crucially important in order to prevent the collapse of highly expensive information systems investments. the literature on health professionals' attitudes towards computerisation suggests that whilst there is an increasing trend towards the computerisation of administrative tasks within both hospital and general practices, many doctors and nurses remain apprehensive about the use of emrs. they are often unconvinced of the advantages, have severe reservations concerning the privacy, confidentiality and security of data, and often believe that the costs (both financial and in terms of the time taken to learn how to use the system) outweigh the benefits (thakurdas et al., 1996: bolton et al., 1999; sleutel and guinn, 1999; tai, donegan and nazareth, 2000). clearly, a hospital based emr linking the primary and secondary health sectors will not be readily embraced by primary healthcare professionals unless these concerns are addressed. it is therefore important to obtain practitioners' views prior to implementation in order to install a system that meets the requirements of the users. this article describes the first phase of an ongoing, commissioned, research project investigating the implementation of an emr system linking the primary and secondary healthcare sectors within the public health system of new zealand. the planned system will be based in a large, urban hospital and will link the hospital's information system to various primary healthcare providers. organisationally, the planned system is akin to a large central hub (the hospital) interacting with a substantial number of small, autonomous spokes (the primary care units). such systems are prone to various technical and organisational problems such as different and 39 ajis vol 9 no. 1 september 2001 incompatible technologies, inadequate training and support for the small units, different working practices and different attitudes towards computerisation (munkvold, 1999). in order to try to circumvent later problems, the hospital that is implementing the new system commissioned the researchers to study a pilot project which will link the hospital and primary care providers' emrs for maternity patients. the maternity emr will be one of the first systems to be made available to the primary care sector as it is seen to have the potential to offer some important benefits for patient care. the progress of a routine maternity case typically requires the patient to interact with several professionals in both the primary care setting and the hospital. this 'shared care' structure, in which patients are seen by a range of professionals within both sectors has often led to problems as patient records are often incomplete and information may take some time to travel from one sector the other (kivekas, hyvarinen and kinnunen, 1996). most women then end up arriving at the hospital at some unarranged time when they are already in labour. easy and prompt access to the patient's case history would therefore improve both the patient's comfort and the attending professional's ability to treat the patient efficiently. the first phase of this research investigated the opinions and attitudes of primary care professionals towards the proposed system. the aim of the study was to identify those factors that are likely to encourage or inhibit the adoption and use of the system. the findings of this phase will influence some of the details of the implementation process. the second phase of the study will be to monitor the implementation process, while the third phase will be to conduct post-implementation surveys of the users once the system has been running for some time. theoretical model of user acceptance the main purpose of this research was to investigate the attitudes of potential users towards a proposed emr system. rather than developing a new model of the factors that influence user-acceptance of information systems, this study adapted davis' widely used technology acceptance model (tam) (davis et al., 1989; davis, 1993; venkatesh and davis, 1996; lederer et al., 2000) to the healthcare environment. the aim was to use the adapted model as a heuristic to aid the analysis of a particular context the 1993 version of davis' model, illustrated in figure 1 below, proposes that systems design features will influence perceived usefulness and perceived ease of use of the system, and these factors are the fundamental determinants of user acceptance as they influence users' attitudes towards the system, which in turn influences actual system use. system design features i7 perceived usefulness perceived ease of use sk. attitude towards using ^actual system use figure 1: technology acceptance model (after davis 1993). although davis' model provides a useful and well researched heuristic for analysing the factors which influence users' perceptions of, and attitudes towards, information systems, the model in not entirely appropriate for the current study and it was therefore adapted in three ways. firstly, the model has generally been used to investigate the uptake of relatively innocuous systems such as electronic mail and text editors. for such systems, perceived ease of use and perceived usefulness may well be the key criteria which determine system usage. in contrast, the literature on the medical use of information technology suggests that health professionals may refuse to use systems which they perceive as both easy to use and useful, if the system does not deal adequately with critical issues such as the security of information and the potential uses of the information (thadkurdas et al., 1996). the model was therefore extended to include the concept of "perceived system acceptability" as a separate factor which will influence users' attitude towards the system and their actual use of the system. examination of the literature on acceptance of emrs indicated that the key criteria relate to information management issues, such as the classes of people who have access to the system, the type of information entered, the security of information and the uses made of the stored information. a second weakness of davis' (1993) model is that it simply investigates users' perceptions of system characteristics and does not consider contextual or organisational factors that may influence uptake. research has shown that whilst factors such as system costs and training and support costs have relatively little influence on end-users in large organisations, where the employer bears the costs, these factors become highly salient in small organisations, such as primary care practices, where users may encounter significant and ongoing personal 40 ajis vol 9 no. 1 september 2001 costs and frequently lack adequate system support (zinatelli et al., 1996; igbaria et al., 1997). many other factors have also been shown to impact on user-acceptance such as management support (e.g. igbaria, 1994) and consultation of users in the development process (e.g. ives and olson 1984; mckeen, guimaraes and wetherbe 1994). the model was therefore extended to allow organisational characteristics (such as sources of training and support, management support, consultation and level of practice computerisation), as well as system characteristics, to have a direct impact on user attitude towards the system. thirdly, individual characteristics of the users, such as age, gender and prior computer experience were also hypothesised as having a direct influence on users' attitude towards the planned system. the revised theoretical model is summarised in figure 2. the model is intended as a heuristic devise to guide the investigations, it is not intended to be an exhaustive representation of all of the mechanisms that link individual, system and organisational characteristics to users' attitudes. individual characteristics organisational characteristics perceived usefulness system / / k. s. \ \ perceived perceived system acceptability x k. .s* ^/ attitude towards using ^ actual system use figure 2: revised model of user acceptance. in addition to the modification of the basic theoretical model, it was also necessary to use a different research methodology to test the model. most of the reported work which has tested the tam has used has used questionnaires designed to measure users' perceptions of various aspects of a system and the extent of their use of the system after they have some experience of using it. researchers have then used regression analysis to determine the extent to which the various factors influence the acceptance and use of the system. however, since this research attempts to identify users' attitudes towards the system prior to its final design and installation, the format of the questions had to be modified so that respondents were asked to evaluate the desirability of features of a planned system, rather than to appraise features of a system of which they had actual knowledge. the fact that the system is yet to be implemented, also means that it was not possible to investigate the link between users' attitudes towards the system and actual system usage. methodology respondents all 167 doctors and midwives holding obstetric contracts enabling them to use the hospital were sent a 13 page postal questionnaire concerning the planned emr system. a total of 51 of the 80 doctors and 51 of the 87 midwives returned usable questionnaires, giving response rates of 64% for doctors and 59% for midwives. the questionnaire the questionnaire was divided into two parts. the first part asked questions relating to the demographic details, computer experience and working environment of the respondents. the second section investigated respondents' opinions concerning key characteristics of the planned system. this section contained five subdivisions measuring respondents' general attitudes towards the planned emr, the perceived desirability of the three key systems characteristics of perceived ease of use, perceived usefulness and perceived system 41 ajis vol 9 no. 1 september 2001 acceptability and finally key contextual factors such as preferred sources of training and support. each subsection contained several items and most questions used response categories anchored to a five point likert scale, ranging from "strongly disagree" to "strongly agree". appendix 1 contains the details of the questions that were used, which have been sorted to correspond with the order in which they are discussed in the results section of this paper. a number of open ended questions were included to gain additional information concerning users'views of the system. results analysis preliminary statistical analysis showed that, with the exception of a few items, there were no significant differences in response patterns associated with the gender, profession or age of respondents. the results presented here therefore apply to the whole sample unless otherwise stated. the detailed findings will now be discussed beginning with the overall demographics and organisational setting. the influence of individual, system and organisational characteristics will then be examined. demographic characteristics and organisational setting the gender distribution of the respondents was uneven with 70 females and only 32 males replying. this reflects the fact that gender was linked to profession, with 61% of the doctors being male and 39% female, whilst 98% of the midwives were female with just one male respondent. approximately 75% of both male and female respondents were in the 36 to 50 age range. there was a significant difference in the practice size of the two professional groups (p < .001). midwives tend to work in smaller practices, with a median size of two midwives, whilst gps generally worked in larger group practices with a median size of five doctors. there was also a statistically significant difference (p < .001) between the number of computerised doctor practices (92%) and computerised midwife practices (33%). however, since practice computerisation, practice size and occupation were all significantly correlated with each other (p < .01), it is not clear whether the difference in the extent of computerisation is due to the difference in the size of the practice or the type of practice. this finding indicates that there may be considerable differences in the costs involved in adopting the system for the different practices. computers were generally used for administrative rather than clinical purposes with only 41% of doctors and 14% of midwives using emrs within their practices. the actual experience of both groups with emrs is therefore limited. individual characteristics and general attitudes towards the emr system respondents were asked two sets of questions regarding their general attitude towards the proposed emr system plus an open-ended question inviting additional comments, the results are summarised in table 1, below. eighty five respondents made additional comments which were often negative and sometimes quite lengthy. perceived competence keeness apprehensiveness use depends on cost benefits hospital benefits self benefits patients refuse without benefits choice important total no. respondents 101 95 96 101 98 98 98 99 101 %age agreeing (score 4-5) 22 51 45 91 30 85 92 30 79 mean score 2.81 3.44 3.17 4.48 3.00 4.11 4.33 3.01 4.26 standard deviation 1.04 1.13 1.29 .88 .99 .8 .62 1.14 1.03 table 1: results of general attitudes towards the emr system. 42 ajis vol 9 no. 1 september 2001 the results indicate a generally low level of confidence concerning emr use (mean score 2.81) with 78% of respondents indicating that they only felt partially competent to handle the tasks and only 22% feeling competent. despite the lack of skills, about half of the respondents (51%) were keen to use the system (mean score 3.44) although many (45%) were slightly apprehensive (mean score 3.17). there was a significant negative correlation between perceived competence and apprehension (-.419, p < .01) and a negative correlation between keenness and apprehension (-.435, p < .01), but no relationship between competence and keenness. this pattern of results suggests that initial training to increase levels of competence may result in less anxiety concerning the proposed system but will not necessarily increase keeness to use the system. a second set of questions investigated the conditions under which the respondents would be willing to use the system. respondents were very clear that their agreement to use the system was dependent on the personal costs to them in terms of time and money (mean score 4.48, 91% in agreement), with 85% of respondents requiring demonstrable benefits for themselves or for their patients before they would agree to use the system. only 30% of respondents were willing to use a system whose primary beneficiary was the hospital. it is also clear that it would be difficult to impose the use of the system. many respondents (79%) indicated that it was important to them to have a choice about using the system and 30% indicated that they would refuse to use a system without demonstrable benefits even if its use was part of a contract. system characteristics influencing attitude towards using the emr. this section examines respondents' attitudes towards the system design features of perceived ease of use, perceived usefulness and perceived system acceptability, all of which are hypothesised as being important to user acceptance. perceived ease of use. the results strongly support davis' contention that perceived ease of use influences user acceptance with over 90% of respondents agreeing that they would be more likely to use an emr that was simple to use. the results, summarised in table 2 below, also indicate that respondents considered all of the facets of ease of use scale to be important, the system should therefore be clear, easy to remember, always available, user-friendly, and easy to learn. use if easy to use facets clear easy to remember always available user-friendly easy to learn total no. respondents 100 99 98 98 99 98 %age agreeing (score 4-5) 93 99 99 96 93 96 mean score 4.59 4.89 4.81 4.78 4.64 4.62 standard deviation .85 .35 .42 .63 .61 .68 table 2: results for perceived ease of use. perceived usefulness according to davis (93) perceived usefulness measures workers' perceptions that the system improves their performance or offers other work advantages. three different facets of system usefulness were investigated: firstly, the ability of the system to improve patient care, secondly, the ability of the system to simplify or diminish respondents' work and thirdly, the ability of the system to provide documentation that is acceptable within the legal arena. eighty nine percent of respondents agreed that they would be more likely to use a system that they perceived as being useful. the results, summarised in table 3 below, show that while the first two facets are likely to have a strong influence on system uptake, the administrative and legal advantages of the system were less important 43 ajis vol 9 no. 1 september 2001 use if improve performance work quicker improve quality improve communications improve seamlessness use if offer advantages decrease work work easier meets goals protect against litigation hold people accountable useable in court total no. respondents 99 99 99 99 99 100 98 99 98 99 99 100 %age agreeing (score 4-5) 89 97 95 93 91 93 97 97 96 94 77 68 mean score 4.51 4.70 4.66 4.59 4.56 4.57 4.71 4.68 4.68 4.64 4.2 3.81 standard deviation .84 .56 .61 .62 .66 .79 .61 .62 .55 .60 .88 1.05 table 3: results for perceived usefulness. perceived system acceptability certain key characteristics of the system are postulated to have a significant impact on users' perceptions of the acceptability of the system, these perceptions will in turn impact on their attitudes towards using the system. four key dimensions of perceived system acceptability were examined, namely, the classes of people who have access to the system, the type of information entered, the security of information and the uses of the stored information. items in this section were designed to identify acceptable boundaries for these dimensions. the respondents were also asked to indicate their probable course of action if they perceived the system to be unacceptable in some way. the results of both the quantitative measures, summarised in table 4 below, and the many comments written in the qualitative section of the questionnaire indicate that this is a highly salient issue for respondents. the classes of personnel who have access to the system was an important issue with nearly 90% of respondents favouring differential access to both the system and data contained within it. essentially, respondents felt that access to health information should be restricted to medical professionals directly treating the patient. whilst nearly all respondents felt that relevant obstetric and general medical information should be entered into the system and therefore accessible by other users. there was less consensus regarding areas such as mental health, sexual health, termination of pregnancy or social history, with only 60% to 75% of respondents supporting the inclusion of these factors. over 90% of the respondents agreed that they would need to feel confident that stored data was both secure and accurate before using the system. issues of patients' consent to having their records placed in the emr and concerns over who would have final ownership of the information were less salient but still important to 76% and 68% of respondents respectively. the various potential uses of stored information also concerned respondents. unsurprisingly there was strong support for using the information for patient care, with nearly all respondents indicating that this was an acceptable use. there was much less certainty about using information for audit, research or teaching purposes, with only 66% to 73% of respondents finding these uses to be acceptable. the use of the information for legal, administrative or financial purposes was acceptable to only 50% or less of the respondents. 44 ajis vol 9 no. 1 september 2001 access differential access hidden data free access information types all obstetric medical mental health social history sexual health terminations of pregnancy information management secure accurate patient consent ownership uses patient care audit research teaching legal administrative financial total no. respondents 98 100 99 99 98 99 99 99 99 101 101 100 98 98 99 99 99 98 98 98 %age agreeing (score 4-5) 89 87 24 95 93 69 73 63 62 91 91 76 68 90 71 73 66 56 52 49 mean score 4.47 4.52 2.45 4.76 4.55 3.95 3.86 3.76 3.73 4.5 4.5 4.15 3.98 4.57 3.94 3.90 3.78 3.65 3.47 3.46 standard deviation .81 .99 1.29 .57 .79 1.14 1.20 1.24 1.20 .83 .81 1.02 1.05 .67 1.09 1.04 1.07 1.13 1.10 1.07 table 4: results for perceived system acceptability. respondents were then asked to indicate their probable courses of action if they perceived the system to be unacceptable. eighty two percent of respondents indicated that in some circumstances they would only enter limited and incomplete information into the system and they would then pass on sensitive information using the telephone or face to face communication. these findings indicate that information management issues are crucial to the success of the system since many respondents indicated that they would boycott all or part of a system which they found unacceptable. implementation and support the final section of the questionnaire examined issues related to the planning, implementation, maintenance and support of the system. the results of this section are consistent with previous research which suggests that involvement of end-users in the planning stage and the provision of good maintenance and user support systems greatly increases user acceptance of a new system (bowns et al., 1999). most respondents (87%, mean score 4.38) agreed that end-user participation in the initial system design would incline them to use the system. however, both doctors and midwives were strongly in favour of having a representative from their own professional group participate, suggesting that where end-users belong to distinct professional groups it may be important to involve representatives of all the groups during the planning (see table 5 below). 45 ajis vol 9 no. 1 september 2001 . participation end-user participation participation by doctor participation by midwife total no. respondents 99 97 38 %age agreeing (score 4-5) 87 52 53 mean score 4.38 3.45 3.32 standard deviation 0.89 1.18 1.14 table 5: results for consultation variables. nearly all respondents felt that the costs of providing training and support should be borne by the hospital rather than by them personally. there was considerable variation in respondents' preferred supplier of training and support. the 85% of doctors and midwives who were members of the independent practitioner association (an umbrella association of practitioners within a geographical area which co-ordinates their work, provides centralised services and negotiates with the government for payments) preferred training and support to be provided by that organisation, whereas non-members preferred the services to be provided by the hospital (see table 6 below). these responses highlight the importance of considering not only the technical characteristics of the system, but also organisational politics and professional loyalties when implementing information systems. membership of primary care org. members non-members support training support training preferred supplier hospital 12 7 11 12 primary care org. 53 57 0 0 independent 10 15 4 4 total 75 79 15 16 table 6: results of preferred sources of training and support. discussion for many years now it has been recognised that information systems which are technically adequate often fail because of a lack of user acceptance. proponents of soft system methodologies (e.g. checkland and scholes, 1990) have long argued that the technical aspects of an information system cannot be considered in isolation from the social and organisational context hi which the system is embedded since the social and the technical systems influence each other in a reciprocal manner. failure is particularly common with complex interorganisational systems or distributed systems, such as the proposed system, which link either different organisations or different branches of the same organisation. such systems are prone to various technical and organisational problems such as different and incompatible technologies, inadequate training and support for the small units, different working practices and different user attitudes towards computerisation. (bensaou and venkatraman, 1996; meier, 1995). when planning the implementation of emr systems it is important to consider the unique characteristics that arise from the healthcare context and the inter-organisational nature of the project. the results of the study indicate that each of the features that were identified in the theoretical model, outlined in figure 2, are likely to have a strong influence on user acceptance of the proposed system. each of these issues will therefore need to be addressed in the design of the system and in the planning of the implementation process. at the individual level, the age profile of the respondents means that, in contrast to younger doctors or nurses currently in training, they will not have received much formal computer training during their professional education (kidd and mcphee, 1999). this is reflected in their relatively high levels of apprehension regarding the proposed system and means that the provision of high levels of support during the initial introduction of the system is likely to be crucial to the system's acceptance. in terms of systems design it is clear that davis' two factors of perceived ease of use and perceived usefulness are crucial to user acceptance. interestingly, many of the most negative comments in the qualitative section of the questionnaire came from gp.s who had prior experience of emrs within their own practices and who felt that such systems were more time-consuming and rigid than paper based records. sobel, alverson and lei (1999) explained a similar pattern of responses by suggesting that old style, chronological, paper records provided practitioners with a rich and flexible source of tacit knowledge which an experienced professional can sift through and interpret very rapidly. such information can be difficult to enter into some of the more rigidly 46 ajis vol 9 no. 1 september 2001 codified emr systems with the result that practitioners experience them as less satisfactory. this implies that there is still much work to be done to improve the interfaces to emrs. in terms of system design, it is also clear that davis' model needs adapting to take into account perceived system acceptability characteristics, such as security and use of the data and clear identification of appropriate usergroups. the results clearly show that the respondents had varied opinions about the types of information that they were prepared to share via an emr and the appropriate uses of information. knowledge of these opinions will enable the systems implementers to enhance the perceived acceptability of the system by introducing appropriate limits on the uses of information. however, even where these constraints are in place, the data within the emr is likely to remain incomplete in some respects, as many respondents will still rely on verbal communication to relay sensitive social history data. these findings suggest that considerable attention will need to be paid to ensuring that health professionals find the emr system easy, effective and acceptable to use from the start of the project. since i.t. staff and medical professionals often have differing perspectives on systems design it may be useful to involve potential users in the planning and implementation phases of the project in order to ensure that the system meets end user requirements. at the organisational level, the different levels of computerisation of the two professional groups and the multiple, incompatible technologies that are in use within different practices will have an impact on the costs of adopting the system and the level of support required for different types of practice. resolving the issue of who will pay for new technology and user support is crucial to the success of the system. in conclusion, this study has highlighted a number of specific organisational and system design issues that the hospital will need to resolve if the proposed emr system is to be successfully implemented and adopted by the target groups within the primary care sector. the study provides strong support for the modification of davis' technology acceptance model to include the concept of perceived system acceptability. it is clear from these findings that information management issues such as confidentiality, security and data quality are crucial factors in the acceptance of inter-organisational systems within the healthcare setting. although! the study has focused on the healthcare sector, it is likely that similar issues will arise in a wide variety of other inter-organisational systems. references bensaou, m. and venkatraman, n. (1996) "inter-organisational relationships and information technology: a conceptual synthesis and a research framework." european journal of information systems, vol 5, pp 8491. bolton, p., usher, h.e., mira, m., herding, l.m., prior, g, (1999). "information technology and general practice: a survey of general practitioner attitudes towards computerisation." aust. fam. physician, vol 28, pps19-s21. bowns, i., rotherham, g and paisley, s. (1999) 'tactors associated with success in the implementation of information management and technology in the nhs." health informatics journal, vol 5, pp 136-145. checkland, p. & scholes, j. (1991). soft systems methodology in action. chichester wiley. dansky, k., gamm, l., vasey, j. and barsukiewicz, c. (1999). "electronic medical records: are physsicians ready?" journal healthcare management, vol 44.6, pp 440-455. davis f.d., bagozzi r.p., warshaw p.r. (1989). "system acceptability of computer technology: a comparison of two theoretical models" management science, vol 35.8, pp 983-1003. davis f.d. (1993). "user acceptance of information technology: system characteristics, user perceptions and behavioural impacts" int. j. man-machine studies, vol 38, pp 475-487. davies, s., (1996) "dystopia on the health superhighway" information society. vol 12.1, pp 89-93. dick, r. and streen, e. (1997). the computer based patient record: an essential technology for healthcare washington: national academy press. hannah, t.j. (1998). "transporting an electronic patient record system across international bounderiesthe lessons learnt" medinfo, vol 9:1, pp 18-20. igbaria, m. (1994) "an examination of the factors contributing to technology acceptance" accounting, management and information technologies, vol 4.4, pp 205-224. igbaria, m., zinatelli, n., cragg, p., and cavaye, a, (1997) 'tersonal computing acceptance factors hi small firms: a structural equation model". mis quarterly. vol 21.3, pp 279-305. ives b., olsen m. (1984). "user involvement and mis success: a review of research" management science. vol 30.5, pp 586-603. kidd, m. and mcphee, w. (1999) "the lost generation it education for general practitioners" medical journal of australia, vol 171, pp 510-512. kivekas, e., hyvarinen, m. and kinnunen, j. (1996) "database management systems for maternity care" hoitotiede, vol 8.2, pp 59-69. 47 ajis vol 9 no. 1 september 2001 lederer, a., maupin, d., sena, m. and zhuang, y. (2000) "the technology acceptance model and the world wide web" decision support systems, vol 29.3, pp 269-282. mckeen j.d., guimaraes t, wetherbe j.c. (1994). "the relationship between user participation and user satisfaction: an investigation of four contingency factors" mis quarterly, vol 18, pp 427-451. meier, j. (1995) "the importance of relationship management in establishing successful interorganisational systems" j. strategic information systems, vol 4, pp 135-148. ministry of health (1996). health information strategy for the year 2000. wellington: ministry of health. ministry of health (2000). the new zealand health strategy. wellington: ministry of health. munkvold, b. (1999). "challenges of it implementation for supporting collaboration in distributed organisations" european journal of information systems, vol 8, pp 260-272. nhmac national health information management advisory council (1999) health online: a health information action plan for australia canberra: commonwealth of australia sicotte, c., denis, j.l., lehoux, p. champagne, f. (1998) "the computer based patient record challenges: towards timeless and spaceless medical practice" j. med. syst, vol 22, pp 237-256. sleutel, m. and guinn, m. (1999) "as good as it gets? going online with a clinical information system" computers in nursing, vol 17.4, pp 181-185. sobel, m. alverson, m. and lei, d. (1999) "barriers to the adoption of computerised technology in healthcare systems" topics in health information management, vol 19.4, pp 1-19. tai, s., donegan, c. and nazareth, i. (2000) "computers in general practice and the consultation: the health professionals' view" health informatics journal, vol 6.1, pp 27-31. thakurdas p., coster g, gurr e., arroll b. (1996). "new zealand general practice computerisation; attitudes and reported behaviour" n.z. med. j., vol 109, pp 419-422. venkatesh, v. and davis, f.d. (1996) "a model of perceived ease of use: development and test" decision sciences, vol 27.3, pp 451-481. willcox, d. (1995) "health scare" computing, 19 october, 28-29. zinatelli n., cragg, p. and cavaye, a. (1996). "end user computing sophistication and success in small firms" european journal of information systems, vol 5, pp 172-181. appendix 1. questions used in the survey questions used response categories anchored to a five point likert scale, ranging from "strongly disagree" to "strongly agree" unless otherwise noted. individual attitudes table 1. how would you rate your current level of computer competence (scale: not at all competent to very competent) i am keen to use an emr i am apprehensive about using an emr my willingness to use an emr will depend on the perceived personal cost (time or money) i would use an emr with demonstrable benefits only for the organisation i would use an emr with demonstrable benefits for myself i would use an emr with demonstrable benefits for my patients i would refuse to use an emr if there were no demonstrable benefits, even if it were part of a contract it is important to me that i have a choice about using the emr perceived ease of use table 2. i would be more likely to use an emr if it were easy to use an emr that is clear and understandable would be easier to use an emr that was easy to remember how to use would be easier to use an emr that could be access whenever one wants, not just at set times, would be easier to use an emr that has a user-friendly interface (windows-type) would be easier to use an emr that is easy to learn would be easier to use perceived usefulness table 3. i would be more likely to use an emr that i thought would improve my work performance an emr that enabled me to work more quickly would be useful an emr would assist in improving quality of patient care would be useful an emr that assisted in improving communication and information sharing (between primary and secondary care) would be useful 48 ajis vol 9 no. 1 september 2001 an emr would assist in improving seamlessness of care (between primary and secondary care) would be useful i would be more likely to use an emr that offered advantages for my work an emr that decreases the workload and/or pressure on myself and my practice would be useful an emr that makes my job easier would be useful an emr that meets the goals/needs of myself, my patients, practice, population would be useful i would be more likely to use an emr that helps to protect against medical litigation an emr that can hold anyone who makes any changes accountable for those changes would be useful an emr that was acceptable in court in the place of paper records would be useful perceived system acceptability table 4. access there should be different levels of access for different groups of people, ie.: health professionals, administrative staff, etc certain information should be 'hidden' from general access, ie.: sensitive issues like hiv results, mental health records, etc anyone with a legitimate reason to access the emr should be able to access any information they feel relevant for their reason information types what information you feel should be transferred to/from an organisation-wide emr: clinically relevant obstetric information past medical and surgical history mental health consultations personal and social history, such as marital status, smoking, alcohol, drug usage sexual health consultations termination of pregnancy details information management that the data is secure and confidential will make me more likely to agree to use an emr that the data is accurate and factual will make me more likely to agree to use an emr that there is patient consent (to put their information on the emr) will make me more likely to agree to use an emr that there is obvious ownership of the data will make me more likely to agree to use an emr uses the information within an emr should be used for clinical patient care the information within an emr should be used for audits the information within an emr should be used for research the information within an emr should be used for clinical teaching the information within an emr should be used for medico-legal purposes the information within an emr should be used for administration, such as bed occupancy figures, staffing levels. the information within an emr should be used for financial considerations, such as budgeting, contracting, managed care consultation variables table 5 i would be more likely to use an emr that i, or a member of my profession, had been consulted about having a gp representative participate in the development of the emr will make me more likely to agree to use an emr having a midwife representative participate in the development of the emr will make me more likely to agree to use an emr sources of training and support table 6 please rank, in order of preference, your preferred choices for the provision of emr support in the left-hand column and for emr training in the right-hand column. 1= most preferred 2= less preferred 3= least preferred 49 ajis vol 9 no. 1 september 2001 emr support secondary health care institution primary health care organisation external agencies/vendors/etc emr training 50 microsoft word special final.doc ajis special issue december 2004 52 usable security and e-banking: ease of use vis-à-vis security morten hertzum, niels jørgensen, mie nørgaard. roskilde university, denmark email: {mhz,nielsj,mnl}@ruc.dk abstract electronic banking must be secure and easy to use. an evaluation of six danish web-based electronic banking systems indicates that the systems have serious weaknesses with respect to ease of use. our analysis of the weaknesses suggests that security requirements are among their causes and that the weaknesses may in turn cause decreased security. we view the conflict between ease of use and security in the context of usable security, a concept that is intended to match security principles and demands against user knowledge and motivation. automation, instruction, and understanding can be identified as different approaches to usable security. instruction is the main approach of the systems evaluated; automation relieves the user from involvement in security, as far as possible; and understanding goes beyond step-by-step instructions, to enable users to act competently and safely in situations that transcend preconceived instructions. we discuss the pros and cons of automation and understanding as alternative approaches to the design of web-based e-banking systems. keywords usable security, ease of use, security, electronic banking, public key infrastructure, strong passwords. introduction e-banking, which we define as web-based electronic banking, is a high-risk area with a potential for substantial economic loss. the high risk makes security a prime concern. in 2001, an estimated total of $17 billion was spent on information-security products and services in the united states alone (idc, 2003). the diversity of e-banking users and the absence of any special training prior to becoming one make ease of use a prime concern as well. the current web-based variant of electronic banking is the latest of several generations of systems: automated teller machines (atms) were the first well-known machines to provide electronic access to customers of retail banks. next came phone banking where users call their bank’s computer system on their ordinary phone and use the phone keypad to perform banking transactions. pc banking superseded phone banking and allowed users to interact with their bank by means of a computer with a dial-up modem connection to the phone network. phone and pc banking entailed maintenance costs associated with keeping up to date with diverse modems and with avoiding prohibitively complex installation procedures. e-banking uses the web browser for the user interface and the internet for data transfer and download of software, and so has a potential for reducing maintenance costs. for users, e-banking provides current information, 24-hours-a-day access to banking services – in addition to the familiar browser interface. the primary services provided by ebanks are transferring money among one’s own accounts, paying bills, and checking account balances. loans, brokering, share trading, service bundling, and a host of other financial services are being added to these primary services (e.g., dewan & seidmann, 2001). since the late 1990s ebanking has developed from virtual insignificance to tens of millions of users worldwide, and due to their high penetration, e-banking systems must accommodate a wide range of users. e-banking is widely used in, among other places, the nordic countries. in 2001, e-banking was used by more than 25% of the population in norway, sweden, and finland, and by 15% of the population in denmark (oecd, 2001). in 2004, e-banking usage in denmark had grown to 45% (statistics denmark, 2004). the purpose of this paper is twofold. at the concrete level, we assess the security and ease of use of contemporary danish e-banking systems. at the conceptual level, the study is about usable security – ajis special issue december 2004 53 that is, matching security principles and demands against user knowledge and motivation. to address these issues, emphasis is on use-related aspects of security, such as risk awareness, installation procedures, and whether systems are oriented toward instruction or understanding rather than on purely technical issues, such as encryption algorithms. while we discuss usable security in the context of e-banking, it should be noted that similar security issues confront users and developers of e-commerce and e-government systems. within egovernment examples include the current endeavours toward further digitalization of public administration by providing citizens with access to their medical or tax data via the internet. in the remainder of this paper, we relate usable security to usability in general, describe our evaluation method, and present and discuss the results. usable security passwords provide an illustrative example of the conflict between security and ease of use and, thereby, of the need for a concept of usable security (e.g., adams et al., 1997; klein, 1990; morris & thompson, 1979; pinkas & sander, 2002; schultz et al., 2001). while passwords such as person names or other real words are relatively easy for a user to remember they are weak from a security point of view because they are vulnerable to dictionary attacks. strong passwords (e.g., x7h!t%c9) are less vulnerable to attack but at the same time more difficult to remember. the term ease of use is used in this paper in the sense of learnability and understandability of user interfaces, and both ease of use and security are considered to be contained within the concept of usability. within human-computer interaction, the iso 9241-11 (1998) definition of usability has gained widespread acceptance. the standard defines usability as the “extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use.” this broad definition equates usability with the quality of a system in use and, thereby, addresses the entire issue of whether the system meets the needs of actual users. specifically, effectiveness includes whether a system enables users to achieve their goals at an appropriate level of security, and likewise, ease of use is also contained within the iso 9241-11 definition of usability. in contrast, several other studies allude to a conflict between usability and security (e.g., adams et al., 1997; dourish & redmiles, 2002; schultz et al., 2001; smetters & grinter, 2002); these studies assume a narrower definition of usability, as conventional in software engineering (see iso/iec 9126-1, 2001). although security has become an aspect of virtually any system, and users habitually interact with access-control mechanisms for, among other things, personal computers, email accounts, and ecommerce or e-government sites, the security facilities of such systems are often less than fully usable. contributions to a better understanding of usable security include the set of characteristics proposed by whitten and tygar (1999) of the usability problem for security, including the weakest link property, the unmotivated user property, and the barn door property. the latter is the property that once a secret such as a private key has been compromised, then closing the barn door, e.g. setting up a firewall, does not restore security. several studies including schultz et al. (2001) suggest that security measures that are inconvenient for users may weaken security, for example because of lack of user acceptance or outright resistance. dourish and redmiles (2002) propose a distinction between theoretical and effective security. theoretical security concerns the level of security that is technically possible; for example, digital signatures provide strong authentication under the assumption that various computational problems related to prime numbers will not be solved within some time frame. effective security concerns the level of security achieved in practice, and is typically lower than theoretical security, due to weaknesses with respect to, among other things, algorithm implementations, protocol design – and ease of use. further, a concept of perceived security may be useful to capture the user’s subjective perception of the security risks involved in ebanking. several studies including jih et al. (2005) indicate that user adoption of e-banking is ajis special issue december 2004 54 affected by perceived security. this supports a view of security as crucial to the overall usability of e-banking systems. whitten and tygar (1999) suggest the following definition, against which we measure the e-banking systems evaluated in this study: security-related software is usable if the people who are expected to use it 1. are reliably made aware of the security tasks they need to perform; 2. are able to figure out how to successfully perform those tasks; 3. don’t make dangerous errors; and 4. are sufficiently comfortable with the interface to continue using it. this definition addresses the effectiveness (issues 1 and 3), efficiency (issues 1 through 3), and satisfaction (issue 4) of security-related software, and so accords with iso 9241-11 (1998). there are two ways in which it seems appropriate to re-interpret whitten and tygar's notion of a security task for use with our evaluation. first, many security tasks in e-banking are tightly integrated into proper e-banking tasks, e.g. typing a password to confirm a transaction, rather than independent tasks such as in whitten and tygar's evaluation of pgp, a tool for secure email. second, we stress that some security tasks in e-banking may be difficult for the system designers to predict in advance, which seems to contradict an underlying assumption of issue 1. thus, security tasks may be part of other tasks, and may not be predicable in advance. what we evaluated a single e-banking service, transfer of money to an account held by someone else, was selected to represent ordinary retail banking, that is, non-commercial account holders' ordinary interactions with their e-bank. we believe this service is the most important for most ordinary users, and certainly every e-bank should provide it in a secure and easy-to-use way. a sample of six danish e-banks was selected for the evaluation. the sample includes the three ebanks offered by the three largest danish banks; these develop their e-banking software in-house. each of the three other e-banks is based on software from one of three independent danish manufacturers of generic software for e-banking; all other danish e-banks are based on software from these three manufacturers, but customization and extension with new features may result in user interfaces that are different from those in the sample. the task of conducting a money transfer in the six e-banks requires that two or three other tasks are carried out, namely installation (five of the e-banks) and logon and logoff (all six e-banks). we include these prerequisite tasks in the evaluation. the prerequisite tasks serve to attain various security requirements: all six e-banks have pledged to comply with the security requirements laid down in the industry code defined by the danish bankers association. the code contains the following requirements, rephrased here to comply with standard security terminology (see itu 1991): user authentication to verify the identity of the user prior to completing transactions and prior to getting access to account information. nonrepudiation to enable banks to prove that a given transaction has been conducted by a given user. data confidentiality to protect transaction or account data communicated between the user and the bank from disclosure. there are various more specific requirements, most notably that user authentication is based on two independent secrets: 1. a secret the user knows (e.g., a memorized password) 2. a secret the user possesses (e.g., a private key stored on the user’s computer). the installation task is a prerequisite in the five banks that implement the industry code's two-secret requirement by means of a public key infrastructure (pki) solution. in contrast, no installation task is required to conduct money transfers or other services in the e-bank that makes use of a physical code card. table 1 summarizes how the two solutions implement the industry code's two-secret requirements. ajis special issue december 2004 55 the logon and logoff tasks establish and terminate a secure connection to the e-bank using the ssl/tls (secure socket) protocol, for data confidentiality. logon in all six e-banks requires both the possessed and known secrets. the full evaluation comprises the following four tasks having various specified security subtasks: installation, with two security subtasks: password definition and signature-file management. logon, with one security subtask: user authentication. money transfer, with one security subtask: user authentication. logoff, which is in itself a security task. the goal of the evaluation of these tasks is to answer the following questions, which are based on whitten and tygar’s (1999) definition of usable security: is the user made reliably aware of, and able to successfully perform the security subtasks? is the user prevented from making dangerous errors? what is the cost of the security subtasks in terms of added user-interface complexity is there a risk that the user does not feel sufficiently comfortable to continue using it? how we evaluated the e-banking systems were evaluated by a walkthrough of the four tasks (i.e., by analytic inspection). accounts were opened in the six banks so that we could conduct the four tasks as ordinary customers. the walkthroughs consisted of constructing sequence models of the actions involved in performing the four tasks. the sequence models, inspired by beyer and holtzblatt (1998), document the information provided by the systems and delineate the correct sequence of user actions required to perform the tasks. in a separate test we also attempted to define weak passwords, deliberately disregarding any advice given by the systems. to attain an approximate, quantitative measure of the complexity of the user interfaces, we counted the number of steps, codes, and concepts involved in each task, as recorded in the sequence models. a step is an action users must perform to provide input to the system, either by filling in a field or clicking a link. a code is a character string users must key in to identify themselves, including account numbers and passwords. concepts are security concepts presented to users as part of onscreen explanatory text. the evaluation was conducted on a pc with microsoft’s windows 2000 operating system and internet explorer 6.0 browser. the security settings of the browser were set to “custom” and the privacy level to “medium”. results in all six e-banks, the user authentication subtasks of logon and money transfer are tightly integrated. the user merely has to type the relevant passwords etc. when prompted. serious errors are probably prevented during these tasks, because the tasks cannot be completed until the user provides the input. ajis special issue december 2004 56 this integration facilitates daily use of the e-banks. still, the e-banks suffer from three main weaknesses: 1. the installation task associated with the five pki-based e-banks is highly demanding, and more complex than any other task. in contrast, the code-card solution relieves the user from installation, but as a consequence each logon is more cumbersome (see table 2). 2. security alerts and other messages from the browser are likely to confuse the user during installation (if any) and also during logon and logoff. 3. the user may not be made sufficiently aware of the security subtasks of installation; the user is not likely to be able to figure out how to perform them; and the creation of a weak password is not consistently prevented. in the remainder of this section we detail observations 1-3. the full evaluation results, including the sequence models, can be seen in hertzum et al. (2004). ebank1 (code card) ebank2 (pki) ebank3 (pki) ebank4 (pki) ebank5 (pki) ebank6 (pki) installatio n no installation 13 14 19 15 23 logon 9 5 5 6 6 7 transfer 3 + data 5 + data 5 + data 6 + data 6 + data 6 + data logoff 1 2 2 1 2 2 table 2. approximation of the complexity of the four steps in the evaluation by the number of steps they require. the transfer task additionally requires typing in data to specify the transfer, such as the amount to be transferred. the complexity of the installation task in the pki-based e-banks the complexity of installation in the six e-banking systems is described in more detail in table 3 based on quantitative data about the number of steps, codes, and concepts involved in the installation process (see the previous section for a definition of these terms). public key infrastructure (pki) code card possessed secret the private key. the key is stored in a signature file, typically on the user's computer, and is stored in encrypted form, to protect it from disclosure in case of theft of the signature file. the physical code card. the card is the size of a credit card and contains 80 pairs of keys and codes. it has no electronic or magnetic components; all the information it contains is conveyed in print. when prompted the user must type the code that corresponds to a given key. known secret a password used to decrypt the private key. a password which is unrelated to the code card. installation required ? yes. generation by software on the user's computer of a pair of private and public keys, and storage of them in the signature file. no. when the user receives a code card it is ready for use. each key-code pair is used only once, in a random order chosen by the e-bank system. eventually a new code card is sent to the user. table 1. user authentication by pki vs. code card, and how they implement the two-secret requirement of the danish bankers association's industry code. ajis special issue december 2004 57 a significant part of the installation process is the typing of various passwords and other codes, for identification and authentication. the user must type a temporary identification and a temporary password, for initial identification and authentication, and a permanent identification and a permanent password for subsequent use. all five pki-based e-banks require users to define their own permanent passwords. some of the banks merge the two identification codes, reducing the number of codes from four to three, as can be seen in the table. in addition to the codes users are also asked to provide other security-related input, such as where to store the signature file. the security concepts presented to the user during installation include concepts such as authenticity, verification, signing, certificate, secure connection, private key, and signature file. these concepts are foreign to most users, intricately related, and not easily understood. users are presented with 814 such concepts during installation of their e-banking system (table 3). this includes concepts appearing in browser messages (see below). of course, the number of steps increases beyond those in the table if the user chooses to inspect some of the links to explanatory information or has to backtrack to make changes or correct mistakes. the complexity of the installation task is aggravated by scant information about how far the user has progressed toward completion of the installation. the code card-based e-bank avoids installation altogether. assessment of the merits of the code-card technology against a public key infrastructure must consider the trade-off between avoiding installation and introducing some inconvenience in daily operation. installation may be a barrier that precludes further use of the e-banks, but is performed only once. the three other tasks will be executed repeatedly during everyday use of the e-banks. the code card is an external device and everyday use is somewhat less convenient because the card must be available. also, logon to the code card-based bank involves more steps (nine) than in the other e-banks, and the user is presented with 11 security concepts during logon. for logon and money transfer the user must type four codes, including the code-card number and a key, both of which must be read from the card. ebank1 ebank2 ebank3 ebank4 ebank5 ebank6 steps no installation 13 14 19 15 23 codes no installation 3 4 4 3 4 concepts no installation 8 8 14 12 8 table 3. the complexity of the installation process. ajis special issue december 2004 58 figure 1. a browser message that pops up during installation of two the pki-based e-banks. the message is triggered by download of code from the e-bank. figure 2. a browser message that pops up during log on to all e-banks. the message is triggered by the initiation of a secure socket connection. ajis special issue december 2004 59 the table gives the number of steps users must go through, the number of different security-related codes users must key in, and the number of security concepts users are presented with during installation (includes concepts that appear in browser messages). the browser messages in all six e-banking systems, information from the banks is interspersed with security messages from the browser during installation (if any) and during logon and logoff. figure 1 shows a browser message of a type that appears during installation of two e-banks; one of these provides written material to instruct users what to do, while the other e-bank does not prepare the user. figure 2 shows a type of browser message that appears at log on to all the e-banks, though only one of them attempts to prepare the user. thus in many cases users are unprepared for the browser messages, and may experience them as an indication that there is something wrong. in fact the messages are part of normal operation, where sound browser design prescribes that the user is asked about code download. most users, even if they recognize that the browser messages are part of normal procedures, will be unable to understand and assess the security implications of the messages. the security warning in figure 1 asks for the user’s permission to install and run a piece of software on the user’s computer. there are, however, multiple problems, including: (1) users cannot be sure that the piece of software is in fact distributed by the entity that claims to be the distributor. (2) hardly any users will know what an authenticity verification by “verisign class 3 code signing 2001-4 ca” is, or whether it is trustworthy. (3) the security warning violates established usability principles such as speak the user’s language (nielsen & molich, 1990) and support internal locus of control (shneiderman, 1998). the reason that the e-banks, in most cases, do not attempt to explain the browser messages may be that users have different browsers, versions, and configurations, so that the banks are unable to predict what messages are presented to the individual user by the browser. the e-bank that attempts to prepare the user for the message shown in figure 2 simply writes “if there is a ‘security message’, click ok”. indeed, the user has no alternative but to comply at each and every step, unless the user is prepared to discard the e-banking systems altogether. this may overshadow the user's legitimate security concerns. the user's awareness of and ability to successfully perform the security subtasks during installation the security subtasks of installation are password definition and signature-file management. the user is prompted to define a password and a path (possibly a suggested default) for the signature file during installation. when prompted for password definition, the user is about halfway through installation and has been presented with one or more browser messages. unfortunately many users at this stage may have already refrained from understanding the relevant security issues, started to disregard his or her security concerns, and may proceed simply by following instructions and accepting defaults. keeping in mind that the user-defined password serves to protect the private key from disclosure, the dialogues for creating and changing passwords display severe shortcomings: not enforcing strong passwords. four of the five pki-based banks do not enforce creation of strong passwords. in fact, three banks accept passwords that consist of multiple instances of the same character (e.g., ‘aaaaaaaa’) and one bank accepts passwords consisting of a digit followed by a sequence of instances of the same letter. ebank3 enforces several strongpassword rules. the only rule consistently enforced by the banks is that passwords must be at least eight characters long. ajis special issue december 2004 60 insufficient information about strong passwords. three of the banks provide no information about what constitutes a strong password. these banks leave it entirely to users to be aware of possible threats, to realize the consequences of these threats in relation to password creation, and to think of ways of creating strong but memorizable passwords. the two remaining pkibased banks warn against using real words and personal information (e.g., phone number). one of these banks also suggests how to create strong passwords, including several examples. the examples make the suggestions easier to understand but are at the same time a security weakness because experience shows that a disproportionate number of users will choose the examples, which are visible to attackers, as their passwords. preventing strong passwords. three of the banks restrict passwords to letters and digits, and ebank3 uses case-insensitive passwords. this significantly reduces the number of possible passwords, and increase their vulnerability to dictionary attacks. (obviously, simplifications such as case-insensitivity make correct typing of passwords easier.) management of the signature file may involve defining a non-standard file path, or storing the file on an external medium instead of on the computer. such measures may reduce the risk of theft of the signature file, but are only partly supported. an obvious and easily predictable task is a change of password in the situation where the user has made a backup copy of the signature file. only two of the five pki-based e-banks tell the user, during the dialogue for changing passwords, that a new backup copy should be made (the reason is that the old backup copy of the signature file still needs the old password for decryption of the private key). there are many other further issues in signaturefile management, such as transferring it to a new computer, that are not explained to the user at all. three approaches to usable security several of the weaknesses of the e-banking systems discussed above could, in our judgement, be overcome without fundamental changes in the design of the systems. suggested changes include explaining and enforcing strong passwords, using the same code for temporary and permanent identification, and using a consistent terminology throughout the user interface. however the confusing browser messages (found in all six e-banks) and the complexity of the installation task (the five pki-based e-banks) would remain, suggesting that more fundamental changes should at least be considered. three approaches to the design of usable secure systems are illustrated in table 4. automation merely requires that users know of security in the sense that they are aware of the existence and importance of security. however, automation alone is not enough because security is critically dependent on steps and precautions taken by users. instruction focuses on providing users with the how-to knowledge they need to complete security tasks in a step-by-step manner, without necessarily knowing why steps are taken or how individual steps are related to each other. understanding attempts to provide users with knowledge of security principles and relies on users to understand these principles sufficiently well to be able to take competent action in an unspecified range of situations. it may be useful to discuss usable security from the point of view of how various designs combine these three approaches. the six e-banking systems are biased toward instruction, and one may consider if the usability of their security facilities could be improved via greater emphasis on automation and/or understanding. instruction the advantage of instruction, the main approach of all six e-banks, is that the complexity inflicted upon users is reduced by instructing them what to do at each step of the process. this can be viewed as a standard approach, and as already noted, whitten and tygar’s definition of usable security ajis special issue december 2004 61 appears to assume that there is a given set of security tasks which the user can be made aware of and instructed about how to perform. however, instructions can only cover tasks predicted by system designers; they are useless when users are faced with novel tasks not covered by the instructions. this is a critical limitation because unpredicted tasks tend to be quite frequent in complex domains (rasmussen et al., 1994). the most glaring example of the limitations of instruction in relation to the e-banking systems is that part of the functionality and user interaction is outside the control of the e-banking systems because it is performed by the browser. users may have different browsers, different versions of the same browser, or their browsers may be set up differently. this defeats thorough instruction because neither the sequence of steps a user will go through nor their exact contents is known to the ebanking system. relying on instruction in such a situation involves a risk of confusing and frustrating users due to mismatches between the instructions and the actual system behaviour. since instruction lends itself to a dialogue where the system is in command and the user is assigned a residual role of filling in missing pieces of information, instruction also entails a risk of compromising security by confronting users with issues not covered by the instructions but nevertheless issues users need to make informed decisions about. automation automation may be considered a possible way of simplifying the user interfaces of the e-banking systems, for example in order to minimize the number of steps users must go through (see tables 2 and 3). one should note that the banks have already automated complex parts of the processes involved in secure use of the e-banks. for example, the pki-based e-banks partially automate key generation and digital signing of a payment transaction. of the seven steps listed by schultz et al. (2001) as typical for digital signing, four are automated. this is due to the integration of the digital signature into the e-banking system. a candidate for further automation would be a ‘remember password’ function, for the password that protects the digital signature. along with schultz’ et al. we find that this would introduce an unacceptable risk, because the legitimate user may leave the computer unattended. a better way to reduce the number of times the password must be typed is to allow for packaging of transactions, similarly to collecting multiple items in a shopping basket prior to purchase on an e-commerce site. another candidate for automation is the browser-based dialogs, because the browser messages are confusing and the user always must type the same answer, eg., “yes” in the dialog shown in figure 1. it might be possible to automate this step if the banks provide the user with a specialized user interface, for example a preconfigured browser dedicated to e-banking. however, this approach appears as a step backwards to pc-banking, the previous generation of electronic banking, in that it eliminates the advantage for users (in terms of convenience) and banks (in terms of maintenance costs) of reusing existing and well-known web-components. a third candidate for automation is the selection of a pathname for the signature file. however, removing the option of selecting a non-default path weakens security, because it makes it easier for hostile programs, transferred by a remote attacker to the user’s computer, to obtain a copy of the file. all in all, further automation does not seem to be a viable road given the web-based architecture and the requirement that authentication is based, in part, on a secret the user knows, as required – rightly, we think in the danish industry code for e-banking. understanding a major obstacle to providing the user with an understanding of security aspects of e-banking is the unforgiving nature of security breaches, i.e. the barn-door property mentioned in the usable security section. widely used usability heuristics such as ‘permit easy reversal of actions’ (shneiderman, ajis special issue december 2004 62 1998) carry with them the underlying assumption that designs should encourage exploration or, at least, allow for a trial-and-error approach to learning how to use systems. however, for e-banking and other security-sensitive systems, a trial-and-error approach is generally not acceptable because a security breach caused by an error may be exploited by an attacker before the error is revoked by the user. once a secret has been left accidentally unprotected, even for a short time, there is no way to be sure that it has not already been read by an attacker. this adds to the demands involved in learning to use these systems and sets this learning process apart from how most users learn to use text processors, spreadsheets, and similar systems. several questionnaire studies have attempted to identify the factors that influence the adoption of ebanking systems (e.g., liao & cheung, 2002; sohail & shanmugham, 2003; tan & teo, 2000). these studies unanimously find that security is important to users. a significant portion of the security understanding users must have to perform their e-banking transactions in a safe manner is not specific to e-banking but, rather, part of a general requirement for behaving safely and competently in a digitally networked world (e.g., claessens et al., 2002; schneier, 2000; whitman, 2003). however, most users may not be motivated to learn about security and may instead tend to experience it as an obstacle that complicates and slows down their business with their e-bank. users may be tempted to put off learning about security, since they generally do not sit down at their computers wanting to scrutinize the consequences of suggested security defaults. the security skills and active user involvement required to understand security issues may be more than most users are prepared to deliver. approach to security automation instruction understanding user's frame of mind trust procedural compliance active involvement task coverage formalized tasks only predicted tasks only predicted and unpredicted tasks type of user knowledge awareness knowledge how-to-knowledge principles knowledge table 4. three approaches to usable security. from a pragmatic point of view, instruction appears the main approach, but to handle complex security issues aspects of automation and understanding must be incorporated. conclusion attaining both security and ease of use in e-banking systems is difficult because security is not a system feature that can be provided automatically while users focus on their primary goal of accomplishing their business with their bank. in the six surveyed e-banking systems, security and ease of use are traded against each other at three levels: the individual-features level. passwords provide a prominent example of a feature that involves a trade-off between security and ease of use. four of the e-banking systems do not enforce strong passwords. further, three of the systems have drastically restricted the set of characters that can appear in passwords and thereby prevented strong passwords. some of these restrictions – for example case insensitivity – make passwords easier to use by avoiding frequent usability problems. the system-architecture level. five of the e-banking systems are public key infrastructure solutions and the last e-banking system makes use of one-time codes drawn from a physical card. both types of security solutions entail considerable complexity within three areas: the number of steps users must go through to complete transactions, the number of security concepts with which users are presented, and the number of security-related codes users must key in during system installation. by relying on the user’s browser as their basic platform, the e-banking systems get access to a standard set of well-tested communication and security ajis special issue december 2004 63 facilities. however, in making use of these facilities the e-banking systems give up some control over how security issues are negotiated with users, and unless the user is prepared to discard the e-banking system there is no alternative to granting the browser the privileges it requests and, hence, no reason to scrutinize even legitimate security concerns. the basic-assumptions level. the six e-banking systems share the assumption that information provided to users should be step-by-step instructions stipulating how to perform specified tasks. the instructions enable users to perform the specified tasks without concerning themselves with the underlying security issues but at the same time leave users without the understanding required to act competently in situations not covered by the instructions. this is a critical trade-off because, on the one hand, e-banking users cannot generally be expected to be motivated to learn about security but, on the other hand, they need to understand it well enough to avoid unsafe actions. the bias toward instruction gives priority to ease of use in predicted situations at the expense of making the systems more difficult to use safely in an unknown range of unpredicted situations. we conclude that most users are only able to complete installation of the e-banking systems by painstakingly following instructions, accepting defaults, and refraining from any real understanding of the involved security issues. however, the high penetration of e-banking systems in denmark suggests that many people are prepared to trust e-banking systems in the absence of compelling, publicly known evidence about security breaches leading to financial loss by e-banking customers. references adams, a., sasse, m.a., and lunt, p. (1997) “making passwords secure and usable” in people and computers xii: proceedings of hci’97, springer, berlin, pp. 1-19. beyer, h., and holtzblatt, k. (1998) contextual design: defining customer-centered systems, morgan kaufmann, san francisco, ca. claessens, j., dem, v., cock, d.d., preneel, b., and vandewalle, j. (2002) on the security of today’s electronic banking systems, computers & security, 21(3), 257-269. dewan, r., and seidmann, a. (eds.) (2001) current issues in e-banking (special section), communications of the acm, 44(6), 31-57. dourish, p., and redmiles, d. (2002) “an approach to usable security based on event monitoring and visualization” in proceedings of the 2002 workshop on new security paradigms, acm press, new york, pp. 75-81. hertzum, m., juul, n.c., jørgensen, n., and nørgaard, m. (2004) usable security and e-banking: ease of use vis-à-vis security. technical report. url: http://www.ruc.dk/~nielsj/research/papers/ebanking-tr.pdf. idc (2003) total it security market – including software, hardware, and services – to reach $45 billion by 2006, according to idc (press release), idc, framingham, ma. available at: www.idc.com/getdoc.jhtml?containerid=pr2003_01_28_085549. consulted: september 19, 2003. iso 9241-11 (1998) ergonomic requirements for office work with visual display terminals (vdts) part 11: guidance on usability, international organization for standardization, geneva. iso/iec 9126-1 (2001) software engineering – product quality – part 1: quality model, international organization for standardization, geneva. itu (1991) security architecture for open systems interconnection for ccitt applications. recommendation x.800, international telecommunication union, geneva. jih, w.-j., wong, s.-y., and chang, t.-b. (2005) effects of perceived risks on adoption of internet banking services: an empirical investigation in taiwan, international journal of e-business research, 1(1), 70-88. ajis special issue december 2004 64 klein, d.v. (1990) “’foiling the cracker’: a survey of, and improvements to, password security” in proceedings of the second usenix security workshop, usenix, berkeley, ca, pp. 514. liao, z., and cheung, t. (2002) internet-based e-banking and consumer attitudes: an empirical study, information & management, 39(4), 283-295. morris, r., and thompson, k. (1979) password security: a case history, communications of the acm, 22(11), 594-597. nielsen, j., and molich; r. (1990) “heuristic evaluation of user interfaces” in proceedings of the acm chi ‘90 conference on human factors in computing systems, acm press, new york, pp. 249-256. oecd (2001) electronic finance: economics and institutional factors, occasional papers no. 2, financial affairs division, organisation for economic co-operation and development, paris, france. pinkas, b., and sander, t. (2002) “securing passwords against dictionary attacks” in proceedings of the 9th acm conference on computer and communications security, acm press, new york, pp. 161-170. rasmussen, j., pejtersen, a.m., and goodstein, l.p. (1994) cognitive systems engineering, wiley, new york. schneier, b. (2000) secrets and lies: digital security in a networked world, wiley, new york. schultz, e.e., proctor, r.w., lien, m.-c., and salvendy, g. (2001) usability and security: an appraisal of usability issues in information security methods, computers & security, 20(7), 620-634. shneiderman, b. (1998) designing the user interface: strategies for effective humancomputer interaction, third edition, addison wesley, reading, ma. smetters, d.k., and grinter, r.e. (2002) “moving from the design of usable security technologies to the design of useful secure applications” in proceedings of the 2002 workshop on new security paradigms, acm press, new york, pp. 82-89. sohail, m.s., and shanmugham, b. (2003) e-banking and customer preferences in malaysia: an empirical investigation, information sciences, 150(3&4), 207-217. statistics denmark (2004). information society denmark. ict status 2004. url: http://www.dst.dk/infosamfundet2004.aspx. tan, m., and teo, t.s.h. (2000) factors influencing the adoption of internet banking, journal of the association for information systems, 1, article 5. available at: http://jais.isworld.org/articles/ default.asp?vol=1&art=5. consulted: december 5, 2003. whitman, m.e. (2003) enemy at the gate: threats to information security, communications of the acm, 46(8), 91-95. whitten, a., and tygar, j.d. (1999) “why johnny can’t encrypt: a usability evaluation of pgp 5.0” in proceedings of the 8th usenix security symposium, usenix, berkeley, ca. acknowledgements thanks to niels christian juul for contributions to this study. the study was supported, in part, by the development centre for electronic business, copenhagen business school, and the it university of copenhagen. copyright morten hertzum, niels christian juul, niels jørgensen, mie nørgaard © 2004. the authors assign to ozchi and educational and non-profit institutions a non-exclusive licence to use this document for personal use and in courses of instruction provided that the article is used in full and this copyright ajis special issue december 2004 65 statement is reproduced. the authors also grant a non-exclusive licence to ozchi to publish this document in full in the conference papers and proceedings. those documents may be published on the world wide web, cd-rom, in printed form, and on mirror sites on the world wide web. any other usage is prohibited without the express permission of the authors. diffusion of electronic commerce in small and medium enterprises robyn lawson school of computing and information technology university of western sydney campbelltown, australia email: r.lawson@.uws.edu.au dr carole alcock school of information technology and computer science university of wollongong wollongong, australia email: carole alcock@.uow.edu.au professor joan cooper dean, faculty of informatics university of wollongong wollongong, australia email: joan cooperis.uow.edu.au abstract using the internet for electronic business has become an area of action for the australian government. this paper presents research results from two regional areas which indicate that while most manufacturing smes use email, very few are involved in elecronic commerce activities. major barriers are concern about security and privacy of transactions, cost of consultants, and lack of it expertise of staff. employing people with appropriate knowledge has been added to current training methods such as on-the-job training. results from the two regional areas are found to be consistent. comparisons between small and medium organisations highlight some differences. keywords electronic commerce, electronic business, manufacturing, smes. introduction the evolution of the internet as a commercial tool for conducting business has led to an increase in the use of online services. e-commerce has emerged as a whole of business strategy offering a range of services and opportunities for electronic trading in the global marketplace. the use of information and communication technologies enables organisations to improve business processes and communication within the organisation and with trading partners. the way business is conducted is experiencing change worldwide, and is clearly evident in australia. globally, e-business revenue is estimated to reach us$6.8 trillion by 2004 representing 8.6% of the world-wide sale of goods and services (hobley 2001). the use of it, the internet and e-business activities have generated considerable interest. consequently, the australian government asserts that e-commerce will provide many new opportunities to expand exports and create thousands of new jobs, and so enhance economic growth (dfat 1997). certainly, the media have portrayed the image that business will predominantly be conducted on-line in the future, and organisations that do not embrace the new technologies will be left behind. as pointed out by roseberg (1976 cited in forester 1985:605) diffusion of new technology can take decades, and involves more than simply reproducing and distributing the technology. indeed, making full use of the new technologies will rely on the it skills of staff within organisations. in january 1999 the australian government released its vision for australia in the information age, and believes that industry will be uncompetitive without e-commerce (alston 1999). a priority is to target barriers and to encourage the use of business-to-business (b2b) e-commerce. technical barriers of bandwidth capacity and incompatible it systems are currently under investigation. a majority of small and medium sized enterprises (smes) have internet access in europe and united kingdom (70%), and in the united states (80%) (hobley 2001). internet access does not, however, equate with e-business practices, but is the first step to being involved in e-business. being involved in e-business activities raises many issues particularly for smes. this new way of doing business requires staff with expertise in the new technologies, and smes will not necessarily have dedicated it staff to enable them to gain the necessary competitive advantage in the global marketplace. while the advantages of e-commerce to large organisations can be justified, particularly if the organisation has specialised it staff, smes may have difficulties firstly in meeting the initial costs and then the on-going expenses of maintaining the operation (farmer 1996). government programs focus on raising the awareness of smes to the trading opportunities offered by e-commerce, however smes are not conducting business on-line as quickly as larger companies (dfat 1997). smes constitute 94-96% of commercial organisations in australia (abs 1998) and therefore play a major role in the australian economy, as well as contributing to private sector employment. abs defines manufacturing smes as organisations with less than 200 employees, and annual turnover less than $20 million. small 123 organisations will not take up new technologies unless they perceive an advantage to do so, or an immediate disadvantage in not taking them up (clarke 1997). barriers to doing business on-line can be categorised as having a technical or a social perspective. technical barriers include inadequacy of telecommunications infrastructure and security of transactions (wai-pun et al. 1997). social barriers range from generally not trusting information technology (crawford 1998); lack of knowledge about conducting business on-line and lack of it skill of staff (clarke 1996; marzbani 1998; crawford 1998); through to lack of awareness about possible uses of the internet (wai-pun et al. 1997). telstra (1998) conducted a survey of australian smes and found that the major barrier was lack of personal contact between the organisation and the customer. the viewpoint that customers were not ready to do business on-line was also rated highly as a barrier. interestingly only 5% of respondents felt security was a barrier to implementation. further, the australian industry group's (aig) report highlighted reasons for not obtaining an internet connection as cost of access, and lack of it skill (aig 1999). barriers to european smes not becoming involved in using the internet have been identified as lack of time (46%), lack of information (31%), and lack of training (30%) (hobley 2001). the australian industry group reported that 78% of its members had an internet connection (aig 1999). a study conducted in tasmania revealed that 77% of respondents used computers, with 37% having an internet connection, and 23% a web page (chau and lawrence 1998). further, 23% of respondents to the tasmanian study indicated plans for a web page within the next year. in queensland, aig found that 60.5% of members had an internet connection (aig 1998). us smes with internet access is 80% (hobley 2001), however, only 32% of american manufacturers had a web page (caswell 2000). considering that e-commerce is in its beginning phase, australia is viewed as a world leader in e-commerce initiatives by the oecd, which rates australia ahead of most of its 29 member countries. australia is ranked third in the world (behind finland and the us) in internet use. current literature in the area has produced success stories for smes, but not usually within the manufacturing industry. an internet presence for manufacturing organisations would see the geographical distances that historically have isolated australia from global trade broken down. this paper reports on a comparative research project that forms part of an integrated study of e-business in australia. the project will focus on similarities and difference between small and medium organisations in two manufacturing regions, ie south west (sw) sydney and south east (se) melbourne. previous studies in these regions will be used for comparative purposes (lawson et al. 1999; lawson et al. 2001). the regions were selected by industry associations as having significant manufacturing bases, and as being equivalent regions in manufacturing terms. specific aims of this project are to identify similarities and differences between small and medium organisations and between regions in terms of: • business use of it; • specific and planned use of the internet for electronic business; • barriers to using the internet for business; • organisation's satisfaction with level of it skills of staff; and • current and planned staff training methods. models of the stages that an organisation moves through when embracing e-commerce have been developed (cooper and burgess 1998; ho 1997; apt 1999). the internet e-commerce staged model (cooper and burgess 1998) will be used to evaluate the level of maturity of e-commerce for organisations in the study. this model was developed from existing models to reflect the modular adoption process for the manufacturing industry. the model was tested internationally to evaluate level of maturity of manufacturing organisations with a web presence. in 1998 most organisations began with an uncomplicated web page and added modules for functionality and complexity. this study will examine if manufacturing has progressed in their approach to web presence since 1998. the study manufacturing organisations located in sw sydney and se melbourne regions that have an entry in the australian industry group yearbook (1999) comprised the population-sampling frame. one thousand one hundred and fifty one organisations (363 in sw sydney and 788 in se melbourne) were identified as meeting the abs (1998) definition for a manufacturing sme. eight organisations were rejected from the sw sydney study as their principal officer had participated in validating the survey. from the remaining group of 355 in sw sydney, 178 organisations were selected. from the group of 788 in se melbourne, 394 organisations were selected. a total of 575 surveys were mailed to the principal officer (managing director, director, etc.) as set out in the aig yearbook. 124 overall 89 surveys were returned as left address or no longer in manufacturing (17 in sw sydney and 72 in se melbourne). in sw sydney 44 organisations completed the survey, representing 27.33% of the sw sydney sample, and in se melbourne 126 organisations completed the survey, representing 39.13% of the se melbourne sample. the overall response rate was 34.97%. the final page of the survey was a separate detachable page for the respondent to indicate their interest in the project by participating in an on-site interview. eight case studies were undertaken in sw sydney and six case studies in se melbourne. the survey consists of four parts: (1) the organisation; (2) use of computers and the internet; (3) planned use of the internet; and, (4) barriers to doing business on-line. demographic data was collected from the respondent (age, gender, and position). a five point likert scale was used for most questions, ranging from very important to definitely not important, or strongly agree to strongly disagree. space was provided for additional comments. from responses to the survey a profile of specific and planned use of the internet for business was established, and related to the internet e-commerce staged model (1998). case study interviews examined experiences of the individual organisation in their specific and planned use of the internet for business, and looked for explanations of the results of the survey, particularly barriers to involvement and staff expertise. demographics table 1 shows the manufacturing industry segments for each region as defined by the abs (1998). table 2 shows the samples broken down by small and medium sized enterprises. figure 1 illustrates the regional samples by industry segment and size of organisation. figure 2 shows the internet staged model used to ascertain the organisations' level of maturity in using the internet for business. segment' sw sydney se melbourne metal product manufacturing machinery & equipment manufacturing textiles, clothing, footwear & leather wood & paper product manufacturing food, beverage, tobacco petroleum, coal, chemical other manufacturing ""segments as defined by abs 20 (45.5%) 9 (20.5%) 3 (6.8%) 1 (2.3%) 0 0 11(25.0%) 44 (100.0%) 25 (19.8%) 24(19.0%) 12 (9.5%) 12 (9.5%) 8 (6.3%) 8 (6.3%) 37 (29.4%) 126 (100.0%) table 1: manufacturing industry segments by region small/medium * sw sydney se melbourne small: less than 20 employees medium: 20-200 employees * size as defined by abs 22 (50.0%) 22 (50.0%) 44 (100.0%) 54 (42.9%) 72 (57.1%) 126 (100.0%) table 2: size of organisation by region metal prod mach& textiles, wood& food, etc petrol, other equip etc paper coal, etc figure 1: regional industry segments by small and medium organisations 125 stage 3: processing stage 2: provision stage 1: promotion figure 2: internet staged model (cooper and burgess 1998) results demographic data for the survey revealed the mean age of the sample was 44.8 years, with a total of 146 males and 22 females. position of the respondent was divided into executive (managing directors, ceos, etc) and nonexecutive (general managers). the executive group consisted of 74 males and 3 female (mean age 45.29 years), and non-executive group consisted of 72 males and 19 females (mean age 44.32 years). based on t-test analysis, there were no significant differences in survey responses based on age, gender or position held. twelve males and two females participated in the case study interviews, with the mean age group of 40-49 years. respondents generally agreed that the important issues facing the organisation were trying to make a profit (89.9%) and staying ahead of the competition (87.1%). results of the study based on the aims of the project are set out below. business use of information technology respondents indicated that computers were essential for processing information for business (94.95%). table 3 shows the use of it for business, overall and broken down by region. five respondents indicated that the organisation did not use computers. results are based on the remaining 165 respondents. all organisations used desktop applications to conduct their business, with all organisations using spreadsheets and 97.25% using word processing. one hundred and fifty one organisations (88.8%) indicated that they had an internet connection, with 81.8% utilising email communications, 41.2% had a web page, and 14.7% made some use of transaction processing via the internet. 126 overall sw sydney se melbourne desktop applications n 165 163 136 % 97.1 95.8 80.0 n 43 41 32 /o 98.8 95.3 74.4 n 122 122 104 /o 96.8 96.8 82.5 accounting sw databases internet connection 151 88.8 39 90.7 112 88.9 email web page transactions processing cad/cam computers to assist manufacturing n=number of organisations 139 70 25 79 104 81.8 41.2 14.7 46.5 61.2 37 13 7 24 29 86.0 30.2 15.9 55.8 69.0 102 57 18 55 75 81.0 45.2 14.3 43.7 59.5 table 3: type of computer use overall and by region based on t-test analysis no significant differences were found between regions on business use of it. however, some significant differences were found between small and medium organisations. differences indicate that medium organisations use some technology more than small organisations, ie accounting software (t=3.045, p<.01), databases (t=2.987, p<.01), and computers to assist manufacturing (t=3.998, p<.01). no significant differences were found with internet connection, email, web page and transaction processing between small and medium organisations. the extent that manufacturing smes are using it for business in the both regions demonstrate an acceptance of computers, with some organisations moving to doing business on-line. from the organisations that took part in this study, a majority have indicated their plans to move to transaction processing via the internet within five years. at present, the main use of the internet for those with an internet connection is email as a communication tool. 127 actual and planned use of the internet table 4a shows the stage that the organisations with a web page (13 in sw sydney, 57 in sw melbourne) have reached in their use of the internet for business, and table 4b shows the stage that the all responding organisations planned to be within the next five years. sw sydney se melbourne % stage 1: promotion 15.4 2.3 stage 2: provision stage 3: processing * 13orgswithwebpageinsw sydney ** 57 with web page in se melbourne 4 7 13* 30.8 53.8 100. 36 18 57** 28.6 14.3 100.0 table 4a: actual internet use related to the internet staged model by regions sw sydney se melbourne m,mm, n % stage 1: promotion 11.3 7.1 stage 2: provision stage 3: processing * 44 total orgs in sw sydney ** 126 total orgs in sw melbourne 8 31 44* 18.2 70.5 100.0 43 74 126 ** 34.2 58.7 100.0 table 4b: planned internet use related to the internet staged model by regions based on t-test analysis no significant differences were found between regions or between small and medium organisations in relation to their actual or planned use of the internet for business. overall 41.0% of the organisations indicated that they had a web page, with 14.7% of respondents at stage 3: processing. however, a majority of the surveyed organisations (90.8%) planned to be at stage 2 or stage 3 within five years, with some positive attitudes towards moving in this direction: "when dealing with larger companies, it will become essential. the majority of people will eventually shop/enquire on line (maybe 5-10 years), so business will have to move with technology. long term, we expect that at least 50% of our business will have some exposure to an internet site that we are on, either our own or one of our customers' sites." (managing director, male, 45 years, small organisation in metal product manufacturing, se melbourne) the assertion, that smes will not take up new technologies unless they perceive an advantage to do so, or an immediate disadvantage in not taking them up (clarke 1997), is supported by the following comments: "the whole 'net' thing is too abstract for most 'conventional' businesses to absorb. when it is finally put into a 'what it will do for you' format without the you beaut extras, it will change." (managing director, male, 45 years, small organisation in metal product manufacturing, se melbourne) case study organisations used their internet connection to send email (both internal and external to the organisation), to do their banking, and to download files (eg drawings). all case study organisations were in 128 various stages of planning and upgrading their web sites. organisational web sites included stage 1 with some elements of stage 2, or stage 2 with some elements of stage 3 of the internet staged model. active planning was well under way for the next one to five years, with on-line ordering, and order status enquiry being viewed as important features for the evolution of the web site. the trend in the survey towards stage 3 is also reflected in discussions with the case study organisations. in sw sydney, case study organisations raised the issue of needing to conduct parallel business processes, for their online customers, and for their customers who do not use technology. they also highlighted that the manufacturing industry has organisations (particularly small organisations) that do not use technology. in addition, it was indicated that b2b transactions in manufacturing were generally carried out in person or by phone. moving to on-line transactions will involve a change in this traditional approach. as a group, the case study organisations identified benefits (wider marketing, better communication, improved business practices and increased business) and risks (cost of implementing and maintaining the web site, loss of knowledge of the transaction and possibly the product, and the web site not necessarily creating a positive image) associated with conducting business online (lawson et al. 2001). barriers to internet use based on the 170 organisations that responded to the survey in sw sydney and se melbourne, table 5 shows the top four (of 12 listed) barriers to the organisation becoming involved in e-commerce. respondents were asked to indicate their agreement with statements by circling a number on the likert scale (l=strongly disagree to 5=strongly agree). the four barriers shown returned a mean higher than 3. sw sydney had an additional barrier with a mean higher than 3 (ie, not sure how many people are using the internet, mean=3.14). concern about security and privacy of transactions cost of consultants lack of government incentives lack of it expertise of staff n=number of organisations can 44 29 28 10 m 3. 3. 3. 3. std dev 14 19 11 05 1. 8 1. 0 1. 6 1. 3 n 8 8 6 6 1.8 7.1 8.8 7.1 % 5 4 3 3 table 5: barriers to internet use for both regions t-test analysis revealed no significant differences between regions or between small and medium organisations on the top four barriers to becoming involved in doing business on-line. however, two barriers listed at equal fifth position overall warrant further examination: (a) lack of cost effective telecommunications infrastructure (overall mean 2.95) while no significant difference exists between regions, the difference between small and medium organisations was found to be significant: small organisations mean 2.78 medium organisations mean 3.13 t-test analysis (t=2.272, p<.05) indicates that medium organisations rate "lack of cost effective telecommunications infrastructure" as important to plans to becoming involved in doing business on-line. (b) not sure how many people are using the internet (overall mean 2.95) while no significant difference exists between regions, or between small and medium organisations based on t-test analysis, the mean for small organisations (3.06) indicates the these organisations believe that "not knowing how many people are using the internet" is important to plans to becoming involved in doing business on-line. barriers that have slowed down the adoption of internet as a viable way of doing business have similarities with other studies. in this study concern about security and privacy of transactions (51.8%) is one of the main obstacles. other studies have rated this barrier lower, and barriers such as inadequate speed of transmission, lack of knowledge about conducting business on-line, and lack of it skill of staff, higher. the high level of concern about security may be a characteristic of the manufacturing industry, or the regions. cost of consultants was the second highest barrier (51.8%). this finding is consistent with an overseas study (hoffman et al. 1998), and was a feature of the discussions with case study organisations both in regions. lack 129 of government incentives was the third highest barrier, and while governments are moving on this issue, perhaps not in practical ways which are relevant to the manufacturing industry. case study organisations, while acknowledging that security and privacy of transactions was a barrier, believed that cost of consultants, integrating current systems, and lack of it expertise within the organisation were barriers to be overcome in the evolution of conducting business on-line. getting the right people to set up the web site which looks good, provides functionality and easy navigation, as well as ongoing support was viewed as critical to maintaining a quality assured standard, particularly when considering functions that would be provided at stage 3. experiences of two organisations revealed that cost of consultants vary dramatically, with quotes appearing to offer the same service varying from $1,000 to $10,000. interestingly, a consultant provided a quote for a web site with particular features at $40,000, and when approached with the same specification the following year, quoted $6,000. these inconsistencies have created an atmosphere of suspicion, with a number of comments, ie: "not easy to get good it people who know what they are doing consultants charge about $120 per hour and are very expensive quality of their work depends on how well you know computers. " (manager, male, 40-49 years, medium organisation in other manufacturing, se melbourne). "the area is so new, consultants are learning at their customers expense at very high fees smoke and mirrors effect it is a very large field, hard to know it all. " (director, male, 30-39 years, small organisation in other manufacturing, se melbourne). all of the respondents to the survey in sw sydney, and 70% of respondents in se melbourne believed that the manufacturing industry has not been made aware of what is available for the move to doing business on-line. no significant difference was found with this belief between regions or between small and medium sized organisations. this lack of awareness is definitely a barrier to an organisations becoming involved in ecommerce. in sw sydney, not understanding the process was also emphasised as a barrier that overlaps with the need to deal with consultants. the need to go out and research was identified as a problem, not knowing who to trust, and the lack of a central place for information related to doing business on-line for smes. the need to work differently was highlighted and to get training for staff was also viewed as a barrier. currently, within some organisations there was none or very little expertise to maintain the web site, and very little time to allocate to training. in contrast, organisations in se melbourne already were in various stages of developing and enhancing their web sites, either by internal people (not necessarily designated it staff), or were outsourcing the work. ideas of what is available and possible were evident, with acknowledgement that the cost of consultants was a barrier. as a consequence, more confidence in dealing with consultants was apparent. lack of staff it expertise had led to existing information technology not being used effectively, and a belief by some of the case study organisations that the situation will be exacerbated with the introduction of new web technologies. in sw sydney, the issue of workload was raised by a number of the case study organisations, which were concerned about the need for parallel systems, and the need to deal with a higher volume of enquiries. parallel systems would be necessary to enable the organisation to deal with additional or existing customers with on-line capabilities, and to provide for existing customers without technology. strategies to deal with more enquiries were viewed as important in maintaining a quality assured standard. case study organisations were generally satisfied with the current level of staff it expertise for management and office staff categories, and were looking at recruitment practices to ensure new employees were computer literate in all categories. level of staff it skills and training one of the aims of the project is to determine level of satisfaction with it skills of staff. the lack of it expertise of staff was identified as one of the barriers to becoming involved in e-commerce. for this series of tables, respondents were asked to tick as many of the items as applicable to their organisation. table 6a shows the organisation's satisfaction with the current level of it skills by staff categories. table 6b shows the top 5 (of 8 listed) it training methods currently used for the various staff categories. table 6c shows the top 5 (of 9 listed) it training methods planned to meet the organisation's business strategies. _ _ management (salaried employees) office employees factory (award employees) n=number of organisations 111 123 51 65.3 72.4 30.0 table 6a: organisation's satisfaction with level of staff it skills by staff category 130 t-test analysis revealed no significant difference between regions or between small and medium organisations based on satisfaction with the current levels of it skills of staff at management, office and factory categories. management office factory n % n % n % in-house training with internal people 64.1 65.9 41.2 109 112 70 external training courses trained to current level of recruitment in-house training with external people computer based tutorials with product n=number of organisations 100 93 88 85 58.8 54.7 51.8 50.0 104 92 95 85 61.2 54.1 55.9 50.0 55 62 53 42 32.4 36.5 31.2 24.7 table 6b: organisation's it training methods used for staff category t-test analysis revealed no significant difference between regions on current training methods. however, some differences were revealed between small and medium sized organisations: (a) small organisations employ factory staff who are trained to their current level at recruitment more than medium organisations (t=3.746, p<.01). (b) medium organisations send office staff to external courses more than small organisations (t=2.239, p<.05). (c) medium organisations use in-house training with internal people more than small organisations for their management and office staff categories (t=2.750,p<.05). (d) medium organisations use in-house training with external people more than small organisations for their office and factory staff categories (t=2.530,p<.05). (e) medium organisations employ permanent it staff more than small organisations (t=2.400,p<.05). (0 n % in-house training with internal people employ people with appropriate knowledge self-taught staff within organisation knowledgeable employees consultants n=number of organisations 113 111 103 99 95 66.5 65.3 60.6 58.2 55.9 table 6c: organisation's it training methods planned t-test analysis revealed no significant difference between regions on planned training methods. however, small organisations used their knowledgeable employees more than medium organisations (t=2.548,p<.05). the level of it skills of staff will affect decisions about moving to doing business on-line, particularly if the level of skill is low, or problems have been experienced in getting staff trained. in these circumstances, lack of it expertise of staff is viewed as a barrier. organisations rely on in-house training with internal people, and plan to continue this trend to meet the organisation's strategic plan for the next five years. satisfaction with the level of it skills of management (65.3%) and office employees (72.4%) is much higher than satisfaction with factory employees (30.0%). organisations are now looking at employing staff with general computer literacy (not package specific) at the factory level: "i expect computer literacy on recruitment, even from factory staff. there is not a guy here who doesn't touch a keyboard." (managing director, male, 40-49 years, small organisation hi textiles, etc., se melbourne) consistent with the survey, case study organisations facilitated in-house training by sending one member of staff to an external course who then passes on the knowledge to staff: "owner/manager learnt system on external training course and then trained office staff' (owner, female, 37 years, small organisation in metal product manufacturing, sw sydney) 131 this approach was extended further by some case study organisations where introductory training (email, internet) was handled by internal staff, with external courses for more advanced training (office, myob, photoshop). however, the acknowledgment that new it skills are necessary if the organisation is planning to move to doing business on-line is demonstrated by the ranking of "employing people with appropriate knowledge" close to "inhouse training with internal people". case study organisations in se melbourne raised some interesting issues related to levels of it expertise of existing staff that needed to be addressed, ie top management under-utilising the technology, inevitable staff turnover, reluctance of factory and older employees. the need to recruit staff with general, rather than package specific skills was also highlighted, as was the need to address the issue of file management (lawson et al. 2001). discussion the results of the study found that while almost all organisations use computers, a majority have an internet connection that they exclusively use for email, and only a small number of organisations are involved in transaction processing. barriers are mostly non-technical, and planned it training methods for staff includes employing people with appropriate knowledge to supplement current training methods. medium organisations appear to be further along the diffusion process, with organisations planning the evolution of their web site and exploring issues related to content and design. while small organisations are planning to move to doing business on-line, their knowledge of the potential benefits offered by involvement in e-commerce is only superficial. one way to overcome this lack of knowledge would be for industry associations to get closer to the problem, by disseminating information and conducting seminars in a format acceptable to smes, with special focus on small organisations. this may need a change of focus, as mentioned in this comment: "industry groups are focussed on government regulations rather than industry tactics or direction of o/s competitors." (manager, male, 39, medium organisation in machinery and equipment manufacturing, sw sydney) the survey in both studies highlighted the need for manufacturing to be made aware of the benefits and risks of smes becoming involved in e-commerce. the case study interviews from both studies provided insight into the role that industry associations could provide, namely: • a central information centre of electronic commerce resources specifically for smes: newsletters, booklets, where to find programs and seminars; • liaison with regional associations to provide: seminars, briefings at regional level relevant to manufacturing smes; • globally raising the profile of manufacturing in australia. raising awareness that can lead to adoption of the new technologies is a priority of the australian government, however actually getting education to smes needs to be handled at a regional level to overcome barriers associated with doing business on-line. regional associations could provide a one-stop point for information for e-business. the process of diffusion identifies three groups that would need to be addressed: early adopters, those who are aware, and those who are not aware. some subsidy from the industry association was suggested as a way of involving smes who are currently not aware of the new technologies. certainly, without post-adoption support the early adopters group can plateau or move away from the new technologies. industry associations have a role in the diffusion process for manufacturing, and more specifically to smes without it personnel and expertise. the studies revealed a need for a global business focus in manufacturing, and a need for regional networking and education on how to achieve on-line business. the following comment raises the need to support the early adopters, as well as those who have not yet conducted some form of business on-line: "the early adopters have broken the ice, and others will come on board, so the next wave will be quite a big one." (director, male, 30-39 years, small organisation in other manufacturing, se melbourne) industry associations at national level could focus on providing a central point for resources for smes wishing to become involved in e-commerce, and liaise with regional associations to provide the face-to-face interaction at local level. almost all case study organisations mentioned that they did receive a lot of paper from their national industry association, which was mostly not read. this distribution by mail is not meeting the needs of smes, and a more timely way of conveying information could be explored. certainly, targeting early adopters, those who are aware, and those who are not aware, and providing these groups with relevant information would be a step in the right direction. modification to the internet stage model (cooper and burgess 1998) is recommended for manufacturing smes in australia based on survey results and case study interviews. as organisations have progressed in thenapproach since the introduction of the model and are planning for interactivity on their web site as an initial step, 132 the barrier between stage 1 and 2 should appear as a soft barrier indicated by a broken circle. the barrier between stage 2 and 3 should be strengthened to a hard barrier, as organisations believed that then" business practices and other issues needed to be addressed before crossing this barrier. in contrast, having a web presence that encompassed stage 1 and elements of stage 2 could be accomplished alongside existing business practices. the choice of sw sydney and se melbourne by industry associations based on the regions being equivalent in manufacturing terms, is supported by the findings that no significant differences were found between the regions on their business use of it, actual and planned use of the internet for business, barriers to adoption, and staff it expertise and training issues. significant differences were found between small and medium organisations on business use of some software applications, and current it training methods. medium organisations use some current it training methods and employ permanent it staff more than small organisations. small organisations could be targetted by government, industry associations and the education sector. the barrier of "lack of cost effective telecommunications infrastructure" had a mean of 3.13 for medium organisations. this is a current target for the australian government. the barrier of "not sure how many people are using the internet" had a mean of 3.06 for small organisations. industry associations could acknowledge that small organisations need a particular focus. conclusion while e-business means conducting business on-line using paperless methods, it needs a multi-disciplinary approach to the diffusion process. the technology is available, and adoption will rely on how well the diffusion process is handled by government and industry associations. this research has highlighted four integrated areas in the diffusion process: • what government and industry association can do: provide information to raise awareness adapted to manufacturing smes • what education sector can do: provide knowledge and skills in electronic commerce for graduates (not just it-related courses) • what smes can do: participate in the diffusion process by liaison with industry associations develop strategies for dealing with consultants identify and develop strategies about workload issues • what consultants can do: interact with smes to provide quality service as the diffusion process evolves in the future, the main issue in electronic business will move away from the technology, to a functional and service level. for smes, business will be carried out by staff with support from consultants. issues will continue to be it expertise of staff and quality of consultants. the next stage of this research involves exploring what industry associations are offering, and planning to offer smes; and what incentives are being offered by the australian government to encourage participation of smes in moving towards conducting business on-line. the use of the internet by manufacturing smes in regional australia (41.2% with a web page) is more than comparable with the national average of 23%, and with us manufacturers at 32%. manufacturing industry associations will have a role to play in supporting smes in developing strategies for doing business on-line. the evolution of the diffusion process will rely on awareness and education. this will be the key to moving towards achieving the australian government's goal of making industry competitive in the global marketplace references alston, r. (1999) australia's e-commerce report card [on-line]. department of communications, information technology and the arts. available from:http://www.dcita.gov.au/cgi-bin/graphics.pl7patlf3789 [accessed 3 july 2000]. apt strategies. (1999) strategic internet business study, [on-line]. available from:http://www.aptstrategies.com.au/review/researchmethod.htni [accessed 3 july 2000]. australian bureau of statistics. (1998) characteristics of small business,[on-line]. available from:http://www.abs.gov.au/ausstats/ [accessed 3 july 2000]. australian industry group. (1999) yearbook sydney: australian industry group. australian industry group. (1999) australian survey by regions. sydney : australian industry group. australian industry group. (1998) queensland members' survey: information technology/e-commerce survey. brisbane : australian industry group. 133 caswell, s. (2000) us manufacturers not b2b e-commerce ready. e-commerce times, [on-line]. available from:http://www.ecommercetimes.com/news/articles2000/q00223-6.shtml [accessed 3 july 2000]. chau, s., and lawrence, k. (1998) electronic commerce knowledge, capabilities and utilisation: the tasmanian perspective. in: collecter'98 conference proceedings, september 1998, [on-line]. available from: http://www.collecter.org/coll98.proc.html [accessed 3 july 2000]. clarke, r. (1997) what's holding up ec in australia?, [on-line]. available from:http://www.anu.edu.au/people/roeer.clarke/ec/impeds97.html [accessed 3 july 2000]. clarke, r. (1996) towards a theoretical framework for collaborative electronic commerce projects involving small and medium-sized enterprises. in: conference proceedings for 9th international conference, ediios, bled, slovenia, 10-12 june 1996, [on-line]. available from:http://www.anu.edu.au/people/roger.clarke/ec/camcla960612.html [accessed 3 july 2000]. cooper, j., and burgess, l. (1998) the status of internet commerce in the manufacturing industry in australia: a survey of metal fabrication industries. in: collecter '98 conference proceedings, september 1998, sydney, australia [on-line]. available from:http://www.collecter.org/coll98/ [accessed 3 july 2000]. crawford, j. (1998) networked enterprise web strategy: a project to get smaller enterprises on-line. department of industry, science and tourism, [on-line]. available from:http://www.dist.gov.au/infoind/busonline/gbo.pdffaccessed 3 july 2000]. department of foreign affairs and trade. (1997) putting australia on the new silk road. canberra : national capital printing. farmer, c. (1996) nothing but net. success, 43, 3, 58. forester, t. (1985) the information technology revolution. oxford : blackwell. ho, j. (1997) evaluating the www: a global study of commercial sites. journal of computer-mediated communication, 3, 1, (june), [on-line]. available from:http://www.ascusc.qrg/icmc/vol3/issuel/ho.html [accessed 3 july 2000]. hobley, c. (2001) just numbers: numbers on internet use, electronic commerce, it and related figures for the european community, january 2001. european commission's electronic commerce team (information society directorate general [on-line]. available from http://www.drecomnierce.com/iustnumbers/ [accessed 28 april 2001] hoffman, d., novak, t. and p chatterjee (1998). commercial scenarios for the web: opportunities and challenges. journal of computer mediated communication 1, 3, [on-line]. available from:http://www.ascusc.org/icmc/voll/issue3/hoffhian.html [3 july 2000]. lawson, r., cooper, j., and c. alcock. (1999) business on-line: manufacturing small and medium enterprises in south west sydney. in: collecter '99 conference proceedings, december, 1999, wellington, new zealand [on-line]. available from:http://www.collecter.org/coll99/ [accessed 3 july 2000]. lawson, r., alcock, c. and j. cooper (2001) e-business adoption by manufacturing smes in regional australia. in: conference proceedings for 14th bled electronic commerce conference, june, 2001, bled, slovenia. marzbani, r., wong, c., holmes, m., chick, h., ghassemi, k. and p. moore. (1998) stats.electronic commerce in australia. department of industry, science and tourism, [on-line], april 1998. available from:http://www.isr.gov.au/infoind/stats/ecomstat.pdf. [accessed 3 july 2000]. telstra corporation ltd. (1998) survey of electronic commerce in australian small and medium business, [on-line]. available from: http://www.pacificaccess.com.au/sbi/. [accessed 3 july 2000]. wai-pun, m., farhoomand, a., and v. tuunainen. (1997) a preliminary investigation of business opportunities on the internet in asia pacific. in: proceedings of the l$t pacific-asia workshop on electronic commerce, 5 april 1997, brisbane : queensland university of technology. 134 microsoft word ajis vol13-2 marina.doc 3 australasian journal of information systems volume 13 number 2 may 2006 editorial ajis founding editor, rob macgregor the australasian journal of information systems (ajis) began in 1993 and was based on the simple realisation that there was a need for an academic ‘outlet’ for is research in australia. we were fortunate enough in having a large consulting editor base from the recently run 3 rd acis conference in wollongong, some of whom have remained with the journal ever since. the development of the journal was not without controversy – some sections of the is academic community questioning ‘whether to support the project or white-ant it.’ our reaction to this was simple – a formal request to become a member of the consulting editor board was sent out and the problem magically disappeared. ajis’s first success was in 1994, being recognised as a ‘strong’, non-us journal and we were invited to present the journal’s goals and missions at the vancouver icis conference. in the early 2000’s ajis was ‘adopted’ by the newly formed aais and in 2004 reached the position of no. 25 in the world ranking of is journals. as editor from 1993 – 2005 i was fortunate to work with many gifted and insightful authors and a large, hard-working and dedicated group of consulting editors. this facet of the editorship i will miss. apparently my colleagues have also indicated that they miss my storming up and down corridors questioning the parentage of microsoft word as it invokes some hitherto unknown rule at publishing time. i would like to extend my thanks to everyone who worked with the journal – the authors, the consulting editors and particularly the staff of the department of is, university of wollongong, without whom the journal would not have been printed or distributed. ajis belongs to the australasian is community. with your ongoing support, its future is assured. i would like to wish the new editors every success and i hope they will gain as much from the role as i have. rob macgregor associate professor of information systems university of wollongong may 2006 ajis vol 9 no. 1 september 2001 the use of influence diagrams in the design of spreadsheet models: an experimental study peter a. o'donnell decision support systems laboratory monash university melbourne, australia peter.odonnell@sims.monash.edu.au abstract the spreadsheet is one of the most commonly used personal computer applications. many studies have shown that a very high percentage of spreadsheet models contain serious errors. the use of influence diagrams, which are a visual representation of the relationships among the variables in a problem, to plan the development of a spreadsheet model may reduce the likelihood of the spreadsheet model containing an error. this paper describes an experimental study that tests whether the use of influence diagrams improves spreadsheet modelling practice. the results of the study suggests that influence diagrams do help users to better understand problems, and as a result reduce the likelihood of omitting important variables from spreadsheet models. introduction the spreadsheet is one of the most commonly used personal computer applications (caine & robson, 1993a, 1993b; panko, 1988; sperling, 1997). spreadsheet models are now an integral part of the information analysis and decision making activity of many managers. tens of millions of spreadsheet models are created and used each year. however, both survey and experimental investigations have found that a significant proportion of spreadsheet models contain serious errors (brown & gould, 1987; edge & wilson, 1990; hall, 1996; hayen & peters, 1989; panko & halverson, 1994; panko & sprague, 1998; simkin, 1987; teo & tan, 1999). cragg & king (1993) state that empirical data suggests that 25% of all spreadsheets used in business contain some kind of flaw or bug. one metric used to measure spreadsheet errors is the cell error rate. this statistic is usually expressed as the number of errors per 100 cells. cell error rates of between 1 and 5 errors per 100 cells are commonly reported (panko, 1996). at first glance these cell error rates may seem low. however, it is typical for one cell in a spreadsheet to depend on the value in another which depends upon the value of another and so on. if one of the base level cells contains an error, that error is propagated through all the cells that depend on it these cell cascades, as panko (1996) calls them, mean that a low cell error rate will lead to a high probability of the spreadsheet model calculating incorrectly. influence diagramming is a modelling technique intended to aid the process of conceptual modelling. an influence diagram is a visual representation of the variables in a problem and of the relationships between these variables. if the developer of a spreadsheet model develops an influence diagram and then uses that influence diagram as the basis of their spreadsheet design, the resulting spreadsheet model may be of a higher quality than would otherwise be the case. this paper describes an experimental study intended to test this theory. spreadsheets spreadsheet software represents an easy to use and deceptively powerful class of modelling tool. whilst, spreadsheets are best suited to financial models, they are also used in a wide variety of model types and application domains. spreadsheets are commonly used to create simulation, optimisation, forecasting, statistical, network analysis, transport and inventory models (flaherty, morgan, lombardo, de silva, & wilson, 1999; ragsdale, 1998). perhaps the most important function provided by a spreadsheet is the ability to perform "whatif' analysis. as a user changes the value in one cell, all the cells whose formulas make use of that cell are updated. all the cells that refer to those cells are then also updated and so on as the change propagates down the cell cascade. the effect of a single change to a cell might cause the instantaneous recalculation of an entire model. ralph kimball refers to this functionality as the 'twinkling' lights of a spreadsheet (nardi & miller, 1991). the visual and interactive nature of the recalculation of a spreadsheet encourages users to change inputs and investigate different problem scenarios. spreadsheet models are very easy to develop. many users, with little or no formal training, can learn to quickly build complex models. the development environment, being visual and interactive, encourages users to make "on-the-fly" changes to models. typically users will create formulas by entering them directly into the spreadsheet as they think about the problem they are working on. the cells of the spreadsheet, arranged in rows and columns, become the conceptual model (landry, malouin, & oral, 1983) used by the developer to organise their thoughts about the problem. spreadsheets are a unique class of software in that they are the only software application in which end-users actually construct their own programs. formulas can be quickly and easily created by moving the cursor around the screen and by using menus to access built-in functions. unfortunately spreadsheets don't stop end-users from 79 ajis vol 9 no. 1 september 2001 making errors or help them find them. the spreadsheet doesn't know when a number has been input incorrectly or when a formula has variable missing. whilst it is possible to create some input error trapping devices within a spreadsheet and some applications include rudimentary debugging facilities, these seem to be little used and ineffective. spreadsheet models containing errors have been used as the basis of significant decisions with disastrous and embarrassing consequences. for example, government officials in australia are often forced to admit mistakes in spreadsheet models and the information released that is based upon them. when a mistake is discovered, a correction can be published however much political embarrassment is caused (for example, kemp, 1996-97). in one case a senate committee was told that a particular court case involving three judges had cost $818,00. later it was discovered that a triplication of the costs associated with the judges had occurred, the actual costs were only $493,106 (soden, 1996). commercial organisations are reluctant to allow knowledge of spreadsheet errors and their effects to become widely known. panko has published a bibliography of the few publicly reported instances of spreadsheet errors in commercial organisations (panko, 2000). in one startling and widely reported case a data entry error caused a general ledger to be in error by $2.4 billion dollars. the company, an investment funds manager, announced that the fund involved had made a profit of $2.3 billion when it had actually lost $0.1 billon. (godfrey, 1995; savitz, 1994). the mistake was not noticed for several months. a well structured and easy to understand system is just as easy to construct as a system that is poorly structured, difficult to understand and unreliable. however, most end-user developers don't understand the value of systems development techniques that create reliable maintainable systems. as a result, error-ridden spreadsheet models are common. the paradox of the use of spreadsheets is that the ease of use and power of the interactive modelling environment is also the reason why so many spreadsheets are developed with errors. ronen, pally, & lucas (1989) and also hayen & peters (1989) argue that the frequency and severity of spreadsheet errors can be reduced by the use of a structured approach to spreadsheet design. the proposals of ronen et al., (1989) include a development life cycle for spreadsheet models and a spreadsheet flow diagramming technique similar to data flow diagrams that are used in the development of traditional information systems. in a field experiment janvrin & morrison (2000), have shown that a structured approach to spreadsheet design can help lower the number of mistakes made developing formulas that link different spreadsheets. others, for example edge & wilson (1990) and simian (1987), have developed rules and procedures that should be followed to ensure the accuracy, consistency and completeness of spreadsheet models. despite experimental studies that have shown that spreadsheet errors are difficult for people to detect and correct (galletta, hartzel, johnson, joseph, & rustagi, 1996; panko & sprague, 1998) it is generally recognised that spreadsheets should be audited and tested using a formal process (berglas & hoare, 1999; whittaker, 1999). an audit of a complex set of spreadsheet-based models developed by the australian department of immigration to examine the effect of immigration on the government's budget revealed over 250 problems and issues which were corrected before the model was used (—, 1999). however, hall's (1996) review of australian spreadsheet development practice suggests that, in practice, few models are ever formally audited or tested. few tools or techniques are available to help model a problem prior to the development of a spreadsheet. those tools that exist aren't used much due to the ad hoc nature of most spreadsheet development and the time-pressure that most spreadsheet developers are under. a design technique is required is that is easy for spreadsheet users to learn, quick to use, that helps to conceptualise a problem, and is relevant to spreadsheet design. the influence diagram seems to have these attributes. spreadsheet errors one measure of the quality of a spreadsheet is to determine the number and type of errors present (brown & gould, 1987; galletta et al., 1996; panko & halverson, 1994; panko & sprague, 1998; teo & tan, 1999). cragg, grant, & king (1991) developed a simple classification of error types found in spreadsheet models. their classification was used to investigate the nature of errors found in operational spreadsheets. this error classification has been used in this study. the five major types in their classification are: erroneous formulae, incorrect ranges, omitted factors, data input errors and incorrect use of functions. erroneous formula: this means that the error exists within a formula. a typical example of this error is when the wrong cell co-ordinates are entered when composing the formula. for example, =a5+a6 may have been entered when the formula =a5+a7 was intended. errors of this kind are very common (brown & gould, 1987; chussil, 1988; ronen etal., 1989). incorrect ranges: this error involves the incorrect specification of a range of cells in a formula. it is a very easy error to make and can be very difficult to detect. an example of this error would be a user entering =sum(b10:b16) when the formula =sum(b10:b15) was intended. as in this example, this error is associated with the use of functions like sum and average that use ranges of cells as an input (ronen et al., 1989). 80 ajis vol 9 no. 1 september 2001 omitted factors: this is an error in the logic of the spreadsheet (chussil, 1988). it is committed whenever a variable is left out of a model or formula. data input errors: this category of errors is used to describe situations where invalid or incorrect data has been entered. for example a user may have entered 5,000 into a cell when 50,000 was intended. incorrect use of functions: the incorrect use of built-in functions can also introduce errors into spreadsheet models. for example, the incorrect use of the rounding function to format output to a certain number of decimal places can introduce errors when the numbers are used in other calculations (brown & gould, 1987). influence diagrams influence diagrams are directed graphs whose nodes represent the variables of interest in some problem and whose directed arcs represent the influence of the variables on each other (holtzman, 1989). figure 1 illustrates a simple influence diagram which consists of two nodes x and y and one arc directed from node x to node y. this diagram should be interpreted as indicating that the value of x has some influence on the value of y (owen, 1978), that is y = f(x). some prefer the term 'relevance' to 'influence' (howard, 1990) arguing the arc directed from node x to node y should be read as 'the value of the variable x is relevant when determining the value of the variable y. the visual depiction of the influence diagram does not allow the specification of the nature of the relationship between nodes. however, it does allow the specification of the variables of interest when determining the value of a particular node. y »kx) figure 1. a simple influence diagram (owen, 1978) the value of an influence diagram, as emphasised by bodily (1988), is the visual representation of the relationships among the variables of interest in a problem. this visual representation aids communication about, and discussion and understanding of, the structure of a model. spreadsheets and influence diagrams any spreadsheet model can be expressed as an influence diagram. alternatively, it is possible to represent a problem that is to be modelled using a spreadsheet as an influence diagram. that influence diagram can then be used to as an aid in the design of a spreadsheet model. the diagram specifies what variables are to be represented in the model. each variable shown on the influence diagram model must be represented by a cell in the spreadsheet. the directed arcs of the influence diagram specify which variables are involved in which spreadsheet formula. figure 2 shows the influence diagram that represents the cell c2 on a spreadsheet if it contained the formula =a2/b2. in figure 2, the cells a2 and b2 contain data input by the user and are represented on the associated influence diagram as border nodes. the formula for the cell c2, which contains references to the cells a2 and b2 indicates that arcs from the nodes representing those cells should be directed to the node representing the cell c2. figure 3 shows a typical simple spreadsheet model (for break-even/profit analysis) and its representation as an influence diagram. 81 ajis vol 9 no. 1 september 2001 figure 2. a simple spreadsheet formula and its influence diagram representation. there are several potential advantages to developing an influence diagram prior to beginning to develop a spreadsheet model. the influence diagram is easily understood and as a result, errors or misconceptions can be identified more easily than they could be if they were incorporated into spreadsheet code. the influence diagram is easier to manipulate and change than a spreadsheet model. it can be argued that the spreadsheet model is less likely to contain a serious error if it has been based upon a carefully developed influence diagram due to the conceptualisation of the problem afforded by the influence diagram. method this experiment involved the development of a spreadsheet model of in a laboratory setting. a schematic representation of the experimental method is shown in figure 4. one group of subjects was asked to develop a spreadsheet model to help solve a problem. this group was the control group. another group of subjects was asked to first develop an influence diagram based model of the same problem, and then develop a spreadsheetbased model. the models developed by the two groups were compared and analysed for errors. experimental procedure the subjects in the experiment were third year students studying a decision support systems elective within an it degree at an australian university. these subjects should have had similar knowledge of the development of spreadsheet based models and of influence diagramming. a study of spreadsheet modelling and influence diagramming is included in the decision support systems elective in which they were enrolled. some data about the background of the subjects was collected and used in the analysis. at the end of the experimental procedure the subjects were also asked to complete a questionnaire designed to gauge their reaction to the experimental procedure. briefly the procedure was as follows: after volunteering to participate in the experiment, the subjects were randomly assigned, by drawing lots, into two groups. each group of subjects, accompanied by an "experimenter", were taken to a personal computer laboratory. each laboratory was similarly configured, providing the subjects with access, via a network server, to the spreadsheet application with which they were familiar (lotus 1-2-3). each subject was given a kit containing their instructions, the problem they were to work on, a numbered work pad for drawing influence diagrams (not included in the kit for the control group), a short numbered questionnaire, and a numbered disk. after completing the exercise, subjects saved their spreadsheet model on the disk provided and returned thenkit to the experimenter. the experimenter then gave each subject a post-procedure questionnaire. after completing their post-procedure questionnaire the subjects were thanked by the experimenter and left the laboratory. a total of 49 students participated in the experiment. the random allocation of each student into either the control or experimental group unfortunately didn't produce two even sized groups. a total of 29 subjects were allocated to the control group and 20 were allocated to the experimental group. this number of subjects is not uncommon in spreadsheet research. sample sizes in previous studies have ranged from 9 (brown & gould, 1987) to 113 (galletta et al., 1996). 82 ajis vol 9 no. 1 september 2001 staple profit analysis selling price cost per units22j5dqjdqqt_l.itiir. ftrssi cosl per unit fixed costs total costs * iou,uuu| hdionirin! $6.400.000 selling [ £4 price prom figure 3. a typical (simple) spreadsheet model and its influence diagram representation. task the target problem used in this work is a standard problem that has been used in the experimental testing of the role of different factors on the quality of spreadsheets (panko & halverson, 1994). a description of the problem given to the subjects is provided in die appendix. panko (1996) reflecting on his earlier study thought it highly likely that many of the spreadsheet errors that were made by subjects working on this problem would have been caused by their lack of domain knowledge. the task assumes that the subjects have basic spreadsheet knowledge and know enough about accounting to construct a pro-forma income statement in conducting a follow up experiment he ensured that the task used was simple in that no specialist domain knowledge was required. in this second experiment a high rate of errors was again found. the subjects in this experiment had previously completed a spreadsheet assignment that required the development of a similar pro-forma income statement. in addition to this all had studied at least one introductory accounting subject as part of there it degree. 83 ajis vol 9 no. 1 september 2001 spreadsheet developers th rd year students study gtiecn oh uppotit random as grsnerit into two groups control group n = 29 i r r e pen errtal group n=20 pre-procedure questionnaire spreadsheet dev lopnent task ti eiteken rec rded , , spreadsheet dev lopmenttas ater influen e diagramm ng i post-procedure questionna re i e taken rec rded error analy s i s 1 dependant *" variab les 1 dependant vaiiables ^ h pojhes testes 1, h?, hsj h4, h5, h6) figure 4. schematic representation of the experimental method research hypotheses hypotheses 1 to 5 relate to one of the error classifications discussed earlier. hypothesis 6 will test whether there is any difference in the time taken to complete the task between the two groups of subjects. hypothesis 1: the use of influence diagrams as the basis of spreadsheet design will lead to fewer errors of the type classified as "errors in formulae". hypothesis 2: the use of influence diagrams as the basis of spreadsheet design will lead to fewer errors of the type classified as "incorrect ranges". hypothesis 3: the use of influence diagrams as the basis of spreadsheet design will lead to fewer errors of the type classified as "omitted factors". hypothesis 4: the use of influence diagrams as the basis of spreadsheet design will lead to fewer errors of the 84 ajis vol 9 no. 1 september 2001 type classified as "data input errors". hypothesis 5: the use of influence diagrams as the basis of spreadsheet design will lead to fewer errors of the type classified as "incorrect use of functions". hypothesis 6: the use of influence diagrams as the basis of spreadsheet design will not add to the overall time taken to develop a spreadsheet. measurement to measure the number of occurrences of each type of error, each model was compared to a "standard" answer for the problem. no attempt was made to make a judgement about the seriousness of the errors found; the errors were simply counted. considerable flexibility was used when comparing the models to the standard answer, as the subjects were given no specific instructions about the format or structure of the model that they were to develop. a research assistant performed the error analysis. this person was provided with the disks that the subjects had used to save their model. these disks were randomly numbered. no indication was given to the research assistant conducting the error analysis that could enable them to identify whether a particular model belonged to the experimental or control group. the time taken to develop the models was recorded in minutes by the experimenter. when the experimental procedure began the experimenter noted the time. as each subject completed the task and submitted their model the time was noted enabling the time taken by each subject to complete the task to be easily calculated. results demographics there was very little difference between the average ages of the experimental and control groups (23 and 24 years respectively). the oldest subject was 44 years old, the youngest was 19 years old. there were 20 male and 9 female subjects in the control group. the experimental group consisted of 11 male and 9 female subjects. the subjects were asked if they had experience developing spreadsheet models prior to the decision support systems course. overall, 59% of the subjects (29 out of 49) claimed prior experience with spreadsheet modelling. the subjects were asked about their major sources of spreadsheet skills. table la shows the sources of spreadsheet training of the subjects. the subjects ranked their spreadsheet development as either 'excellent', 'very good', 'adequate', 'fair' or 'poor'. table ib shows how the two groups ranked their spreadsheet skills. this data confirms the prior expectation that the subjects were not spreadsheet experts and that formal training within their degree course was their major source of spreadsheet knowledge. the experimental subjects were asked to indicate their major sources of influence diagramming skills. most had only learned of influence diagramming during the study of the decision support systems elective (13 out of 19). the others had been taught influence diagramming in other units of study in their degree program (6 out of 19). this data confirms the expectation that the subjects in the experimental group had no significant exposure to influence diagramming other than that obtained from the decision support systems elective. table 1. sources of spreadsheet skills table 2. subjects rating of their spreadsheet skills 'dss1 unit other subjects from a text book or manual at work on a training course control group3 17 4 13 2 7 experimental group4 7 4 9 2 3 excellent very good adequate fair poor control group 2 5 14 6 4 experimental group 2 1 12 5 0 3 not all subjects in this group answered. 4 subjects were allowed to select more than one item. 85 ajis vol 9 no. 1 september 2001 upon completion of the experimental procedure, subjects were asked to rate the difficulty of the task they performed on a 5-point likert scale. a rating of 1 indicated that they found the task very easy, a rating of 5 indicated that they found the task very difficult. most rated the task as moderately difficult. the mean rating of the subjects in the control group was 3.16 and the mean rating of the subjects in the experimental group was 2.94 (the standard deviations were 1.06 and 0.85 respectively). there was no significant difference between the ratings given by the two groups (t = 0.77). a surprisingly high number of subjects after completing the exercise, given that they thought that it was only of moderate difficulty, thought that their spreadsheet contained errors. (no prior indication had been given to the subjects that the spreadsheets they developed would be analysed for errors.) in the experimental group 55% (11 of 20) of the subjects believed their spreadsheet contained an error. in the control group 62% (18 of 29) of the subjects believed that their model contained an error. summary of the errors found as anticipated a large number of errors were found in the spreadsheet models developed by the experimental subjects. a summary of the number and types of errors found is shown in table 3. all of the models developed by subjects in the control group contained at least one error and the majority of the models developed by subjects in the experimental group contained at least one error. spreadsheets developed by subjects in the control group contained an average of 4.21 errors. those developed by subjects in the experimental group contained 3.5 errors. the most common type of errors were the errors classified as 'erroneous formula' and 'omitted factors'. errors of the types 'incorrect range' and 'incorrect use' of functions were rare. the error rate is high compared to other studies using the same experimental task. for example, individuals developing spreadsheets in panko & halverson's (1994) study made 2.36 errors on average (compared to 3.92 for all subjects in this study). in that study 79% of the spreadsheets developed by individuals contained at least one error (compared to 94% for all subjects in this study). there is no clear reason for the higher error rate found in this study. however, in panko & halverson's (1994) study a different technique was used to count errors and some errors were not included in the counts. table 3. summary of the number and types of errors found. number of subjects erroneous formula incorrect range omitted factors data input errors incorrect use of functions total number of all errors average number of errors per spreadsheet % of spreadsheets with errors control group count 29 53 4 48 12 5 122 4.21 100% mean 1.83 0.14 1.66 0.41 0.17 std dev. 1.71 0.52 2.24 0.78 0.66 experimental group count 20 44 0 12 12 2 70 3.5 85% mean 2.20 0.00 0.60 0.60 0.10 std dev. 1.64 0.88 1.23 0.45 total count 49 97 4 60 24 7 192 3.92 94% hypothesis testing upon observation of the data it appeared that the error-count data could not be assumed to be parametric. the lilliefors test for normality was conducted on each of the data sets. the results of the liffiefors test confirmed that a non-parametric test would be required to perform the hypothesis testing. the test used to perform the most of the hypothesis testing was the wilcoxon rank sum test (wilcoxon, 1945). this test uses a z-score to determine if the difference between two data sets can be attributed to chance. for each hypothesis, the intention was to discover whether the error rate for the experimental group was lower than that of the control group, hence a one-tailed test can be used. the critical value of z for a one-tailed hypothesis test at a 95% level of confidence is 1.65. the calculations associated with the wilcoxon rank sum test for each hypothesis are given in tables 4a, 4b, 4c, 4d and 4e. these calculations are summarised in table 5. these results show that the null hypothesis is accepted for all error types (h10, h20, h40 and h50) except errors associated with omitted factors. for errors associated with omitted factors the null hypothesis can be rejected and the alternative hypothesis (h3/0 can be accepted. that is, there is sufficient evidence to indicate that errors of omitted factors of the experimental group are less in general than that of the control group. 86 ajis vol 9 no. 1 september 2001 table 4a. calculations for wilcoxon rank sum test erroneous formula. error count 0 1 2 3 4 5 8 frequency (experimental group) 5 1 6 2 5 1 0 nconm>, = 20 frequency (control group) 5 9 10 1 2 1 1 •<* experimental ~ *•* ranks 40-49 30-39 14-29 11-13 4-10 2-3 1-1 mean rank 44.5 34.5 21.5 12 7 2.5 1 frequency by average rank (experimental group) 222.5 34.5 129 24 35 2.5 0 •" experimenter 447.5 frequency by average rank (control group) 222.5 310.5 215 12 14 12.5 1 tcontrol = 777.5 table 4b. calculations for wilcoxon rank sum test incorrect formula. error count 0 2 frequency (experimental group) 20 0 ncontml = 20 frequency (control group) 27 2 " experimental ~ *•* ranks 3-49 21-24 mean rank 26 1.5 frequency by average rank (experimental group) 520 0 texperimentat 520 frequency by average rank (control group) 702 3 t — 7fi^* control ' "•* table 4c. calculations for wilcoxon rank sum test omitted factors. error count 0 1 2 3 4 7 10 frequency (experimental group) 13 2 5 0 0 0 0 nconm, = 2q frequency (control group) 12 1 11 1 1 1 1 nexperimenuil = 29 ranks 25-49 21-24 5-20 4-4 3-3 2-2 1-1 mean rank 37 22.5 12.5 4 3 2 1 frequency by average rank (experimental group) 481 45 63 0 0 0 0 texperimenuf 589 frequency by average rank (control group) 444 45 138 4 3 2 1 tconml = 637 87 ajis vol 9 no. 1 september 2001 table 4d. calculations for wilcoxon rank sum test data input errors. error count 0 1 2 3 4 frequency (experimental group) 15 2 0 2 1 ^control = 20 frequency (control group) 21 5 2 1 0 ^experimental ^" ranks 14-49 7-13 5-6 2-4 1-1 mean rank 31.5 10 5.5 3 1 frequency by average rank (experimental group) 473 20 0 6 1 t[jcperimentai= 500 frequency by average rank (control group) 662 50 11 3 0 tcontrol ~ 726 table 4e. calculations for wilcoxon rank sum test incorrect use of functions. error count 0 2 3 frequency (experimental group) 19 1 0 tfwwrf-20 frequency (control group) 27 1 1 n experimental 19 ranks 4-49 2-3 1-1 mean rank 26.5 2.5 1 frequency by average rank (experimental group) 504 3 0 1jexperimentai= 506 frequency by average rank (control group) 716 3 1 t control =719 table 5. summary of the hypothesis tests using wilcoxon rank sum test. hi h2 h3 h4 h5 error type errors in formula incorrect ranges omitted factors data input incorrect use of functions t*• experiment at 447.5 520 589 500 506 1 control 777.5 705 637 726 719 e(t) 500 500 500 500 500 st 49.16 49.16 49.16 49.16 49.16 z -1.07 0.41 1.81 0.00 0.12 result accept ho, reject ha accept ho, reject ha reject ho, accept ha accept ho, reject ha accept ho, reject ha e(t) = nexperimental ( nexperimental + ncontrol + 1) 11 nexperimental = 20, ncontrol = 29, 2 = texperimcntol e(t) / st, rejection region (95% confidence) is z > 1.65 hypothesis 6 was tested using a t-test as the data used (the time taken in minutes) was parametric. the results show that there is no significant difference between the mean time taken by subjects in the control (mean 49.3, std dev 13.0) and experimental (mean 50.6, std dev 16.7) groups to complete the exercise (t = -0.283). discussion findings the primary question addressed in this investigation is whether the use of influence diagrams improves the quality of spreadsheet models. the hypotheses testing shows that they do. the subjects in the experiment who drew an influence diagram prior to developing their spreadsheet model did a better job of identifying the 88 ajis vol 9 no. 1 september 2001 variables relevant to the problem than the other subjects. as a result they made less errors of the type omitted factors. this finding can be explained by the fact that drawing an influence diagram represents an explicit stage in the design of the model where thought is given to the nature and type of the variables involved. the influence diagram becomes part of the developers conceptual model. without giving explicit thought to that process, the subjects who didn't draw a diagram were likely to leave a variable out of their model. the rows and columns of the spreadsheet that they are manipulating as they think about the problem becomes the major framework they use for thinking about the problem. in most problem situations this conceptualisation is not likely to be effective. one argument against drawing an influence diagram is that it is an extra task that must be performed that of constructing the diagram and that this will slow down the development process. in this study the subjects in the experimental group didn't take any extra time overall to develop their model. it is reasonable to conclude that the time spent drawing the diagram enabled them to develop the system in less time resulting in the same overall time. the argument that drawing an influence diagram prior to building a spreadsheet model will slow down overall development appears to be a weak one. it must also be noted that the experimental group made just as many of the other types of errors as the control group. it was thought possible that the drawing of an influence diagram would lead to a more considered and careful approach to system construction which would help lessen errors related to typing mistakes and errors in formulas that might have resulted from less careful attention to the task. this seems not have been the case. study limitations and further research it must be noted that the study is subject to several limitations that may limit the generalisability of the findings. first, undergraduate students with little 'real' experience of spreadsheet development were used. however, in a previous study by panko & sprague's (1998) there was no difference between the performance of undergraduate students and more experienced mba students in terms of the number and type of errors made. another limitation relates to the used of a standard 'canned' problem. the problem whilst realistic was not real. the subjects had no personal interest in the problem. they were not rewarded in any way for participation in the experiment nor was their performance to be rewarded or punished in any way. the use of participants working on a model that they have a personal interest in might lead to better performance and fewer errors. (it might also lead to a better influence diagram as well so the benefits of drawing a diagram might also be magnified). the laboratory environment in which the experiment was conducted is also a little removed from the real environment in which people build spreadsheets that are actually used for decision analysis. in their own office, a spreadsheet developer has other people to talk to and ask for help, manuals to refer to and perhaps access to old spreadsheets parts of which they might be able to reuse. this might significantly change the nature of the development task, having an effect of the efficacy of influence diagrams. a field experiment using experienced spreadsheet developers, following a similar experimental process to the one used in this study, working in their normal work environment is currently being conducted to investigate this. however, it should be noted that in previous studies there has been little difference between the error rates found in spreadsheets developed in laboratory experiments and those found in field experiments and field audits (panko, 1998). concluding comments spreadsheets are commonly used as a decision making aid. many decisions are made using spreadsheets that contain errors. a lot of research has been conducted to investigate the nature and frequency of these errors. unfortunately little work to date has been performed to develop techniques and methods to help improve spreadsheet modelling practice. the techniques and methods that do exist are rarely applied in practice. this study has investigated whether drawing an influence diagram before developing a spreadsheet model improves the quality of the resulting model. a laboratory experiment, using senior undergraduate it students as subjects, was conducted. a high number of errors was found in the spreadsheet models developed. the subjects in this experiment who drew an influence diagram before developing their models made fewer errors of the type 'omitted factors'. that is their models were less likely to have a relevant variable omitted in error. drawing the influence diagram didn't add to the overall time taken to build their model. in short, the results of the study show that if people draw an influence diagram before beginning to construct their spreadsheet then they will build a better model due to the improved conceptualisation of the problem afforded by the construction of the influence diagram. influence diagrams are simple to draw and understand, and simple to learn to draw. a brief tutorial on influence diagrams could easily and should be introduced to any course or training material that teaches spreadsheet modelling. 89 ajis vol 9 no. 1 september 2001 acknowledgements thanks to josephine chueng for her work as research assistant. the comments made by the anonymous reviewers were also greatly appreciated. references —. (1999, september 1999). impact of migrants on the commonwealth budget. australian immigration research, australian department of immigration and multicultural affairs. available: http://www.immi.gov.au/research/acill.htm [2000,28 april]. berglas, a, & hoare, p. (1999). spreadsheet errors, risks and techniques. management accounting, 77(7), 4647. bodily, s. e. (1988). modern decision making : a guide to modelling with decision support systems ( 2nd edition ed.). new york, ny: mcgraw-hill. brown, p. s., & gould, j. d. (1987). an empirical study of people creating spreadsheets. acm transactions on office information systems, 5(3), 258-272. caine, d. j., & robson, a. j. (1993a). models for decision making. management services(january), 28-30. caine, d. j., & robson, a. j. (1993b). spreadsheet modelling: guidelines for model development management decision, 31(1), 38-44. chussil, m. j. (1988). computer-based planning: a pc survival guide. journal of business strategy, 9(1), 3842. cragg, p. b., grant, b., & king, m. (1991). an exploratory study of spreadsheet modelling practice. paper presented at the 2nd australian conference on information systems, wollongong. cragg, p. b., & king, m. (1993). spreadsheet modelling abuse: an opportunity for or? journal of the operational research society, 44(8), 743-752. edge, w. r., & wilson, e. j. g. (1990). avoiding the hazards of microcomputer spreadsheets. internal auditor, 47(2), 35-39. flaherty, j., morgan, p., lombardo, r., de silva, b., & wilson, d. (1999). a spreadsheet approach to business quantitative methods. melbourne, aust: logistics group, rmit university. galletta, d. r, hartzel, k. s., johnson, s. e., joseph, j. l., & rustagi, s. (1996). spreadsheet presentation and error detection: an experimental study. journal of management information systems, 13(3), 45-63. godfrey, k. (1995). computing error at fidelity's magellan fund, [discussion archive]. the risks digest. available: http://cadess.ncl.ac.uk/risks/16.72.htniwteubjl.1 [2000, 28 april]. hall, m. j. j. (1996). a risk and control-oriented study of the practices spreadsheets application developers. paper presented at the 29th annual hawaii international conference system sciences. hayen, r. l., & peters, r. m. (1989). how to ensure spreadsheet integrity. management accounting(april), 30-33. holtzman, s. (1989). intelligent decision systems. reading, mass: addison wesley. howard, r. a. (1990). from influence to relevance to knowledge. in r. m. oliver & j. q. smith (eds.), influence diagrams, belief nets and decision analysis. chichester, uk: john wiley and sons. janvrin, d., & morrison, j. (2000). using a structured design approach to reduce risks in end user spreadsheet development. information and management 37(1), 1-12. kemp, r. (1996-97). question without notice (speech): midyear review 1996-97 (senate hansard ). canberra, australia: parliament of australia. landry, m., malouin, j.-l., & oral, m. (1983). model validation in operations research. european journal of operations research, 14,207-220. nardi, b. a, & miller, j. r. (1991). twinkling lights and nested loops" distributed problem solving and spreadsheet development. international journal of man-machine studies, 34,161-184. owen, d. l. (1978). the use of influence diagrams in structuring complex decision problems. paper presented at the 2nd lawrence symposium on systems and decision sciences, berkeley, ca. panko, r. r. (1988). end user computing: management, applications and technology. new york, ny: wiley. panko, r. r. (1996). hitting the wall: errors in developing and debugging a 'simple' spreadsheet model. paper presented at the 29th annual hawaii conference on system sciences. panko, r. r. (1998). what we know about spreadsheet errors. journal of end user computing, 70(2), 15-21. panko, r. r. (2000). reports of spreadsheet errors in practice, [html page]. available: http://panko.cba.hawaii.edu/ssr/cases.htm [2000,28 april]. panko, r. r., & halverson, r. p. j. (1994). individual and group spreadsheet design: patterns of errors. paper presented at the twenty-seventh hawaii international conference on systems sciences, kihei, hawaii. panko, r. r., & sprague, r. h. j. (1998). hitting the wall: errors in developing and code inspecting a 'simple1 spreadsheet model. decision support systems, 22(4), 337-353. 90 ajis vol 9 no. 1 september 2001 ragsdale, c. (1998). spreadsheet modelling and decision analysis: a practical introduction to management science.: south-western. ronen, b., pally, m. a., & lucas, h. c. j. (1989). spreadsheet analysis and design. communications of the acm, 32(1), 84-93. savitz, e. j. (1994, december 12). magellan loses its compass. barron's, 84. simkin, m. q (1987). how to validate spreadsheets. journal of accountancy, november, 130-138. soden, w. (1996). transcript of a meeting of the legal and constitutional legislation committee (senate,). canberra, australia: parliament of australia. sperling, e. (1997). spreadsheets. computer reseller news, 735(74), 1. teo, t. s. h., & tan, m. (1999). spreadsheet development and 'what if analysis: quantitative versus qualitative errors. accounting, management and information technologies, p(jul-september), 141-160. whittaker, d. (1999). spreadsheet errors-and techniques for finding them. management accounting, 77(9), 5051. wilcoxon, f. (1945). individual comparisons by ranking methods. biometrics, /, 80-83. appendix: the problem the problem the participants were asked to solve follows. it is known as the galumpkes word problem and it was taken from panko & halverson (1994) who used it in a similar laboratory-based experiment. your task is to produce a two-year pro-forma income statement using a spreadsheet — a projection of a company's financial performance. the company sells galumpkes, which are small food warmers used in restaurants. the owner will draw a salary of $80,000 per year. there is also a manager of operations, who will draw a salary of $60,000 per year. the tax rate is expected to be 25% in each of the two years. each galumpke will require $40 in materials cost and $25 hi labour cost the first year. these numbers are expected to change to $35 and $29 in the second year. there will be a capital purchase of $500,000 in the first year. for depreciation, assume 10% straight-line depreciation with no scrap value. unit sales price is expected to be $200 the first year and $180 in the second year. there will be three sales people. their average salary is expected to be $30,000 in the first year and $31,000 in the second. the rent will be $3000 per month. the company expects to sell 3000 galumpkes in the first year. in the second, it expects to sell 3200. 91 microsoft word vol 15 2 final.doc australasian journal of information systems volume 15 number 2, 2009 133 temporal convergence for knowledge management christopher phillip martin land operations division, defence science and technology organisation, department of defence, australian government. wayne philp land operations division, defence science and technology organisation, department of defence, australian government. william p hall national fellow, australian centre for science, innovation and society, university of melbourne. abstract time and knowledge have tended to be conceptualised in conventional knowledge management systems as either ‘timeless’ recordings of procedures, or time-stamped records of past events and states. the concept of temporal convergence was previously developed to help apply knowledge-management theory to complex military processes such as commander’s intent, shared situation awareness, and selfsynchronisation. this paper clarifies the concept and introduces several others in forming a framework to assist discussion and exploration of the types of knowledge required for complex endeavours, such as warfighting, characterised by opposition and uncertainty. the approach is grounded in a pragmatist philosophy and constructivist epistemology. argument proceeds along mathematical lines from a basis that the types of knowledge most valuable to goal-directed agents in uncertain environments can be modelled as directed graph topologies. the framework is shown to be useful in describing and reasoning about the knowledge requirements and prerequisites for distributed decision-making through the sharing of situational knowledge and common intentions, with practical application to the planning and execution of operations. to the designers of knowledge management systems seeking to address this space, it presents a challenge that cannot be addressed merely by construction, storage, search and retrieval of documents and records pertaining to the past. australasian journal of information systems volume 15 number 2, 2009 134 introduction business process improvement focuses on the recording and analysis of processes in hopes of making them more efficient. this paper extends into the realm of business processes characterised by uncertainty and opposition to address the question, “what is the nature of knowledge specific to such processes?” in so doing, we propose an epistemological framework which clarifies temporal convergence and the time-value of knowledge (dalmaris, hall and philp, 2006) and introduces several related concepts. we also show how this framework illuminates the knowledge requirements and prerequisites for distributed decision-making through the sharing of situational knowledge and common intentions. the paper begins by establishing the psychological and mathematical foundations for the theory to follow. the theory proper commences by presenting two contrasting perspectives an agent may hold regarding how to relate to its own possible futures, and concludes by describing processes internal to the agent that show complex interrelations between these two perspectives. an application of the theory follows, giving insight to some concepts of knowledge management in military affairs. further implications of the theory are drawn for processes of learning and decision-making. the paper concludes with some general suggestions for knowledge management practitioners. mathematical foundations we take it as given that a business faced with a problem will engage in some process in an attempt to solve the problem. a well designed knowledge management system should play an active role in the problem-solving process through assisting the assimilation, filtering, alerting, accessing and building of knowledge. for many business problems, solution processes are already well-known to someone and so the primary purpose of a knowledge management system in such cases is to record that process knowledge and enable others to reproduce it, correctly and efficiently. in this paper, we consider processes which can be described in terms of a system undergoing transitions from one discrete internal state to another. some state transitions occur due to external forces impinging upon the system, perturbing it. other state transitions occur as the result of the activity of components within the system. components may in turn be seen as systems in their own right with their own internal states. not considered in this paper are other classes of process which can be better described by other formalisms such as linear or dynamic equations. some problems can be addressed through optimisation techniques in which a fully known, closed set of parameters represent the dimensions of a coordinate space and the objective of the problem solving process is to search the coordinate space as efficiently as possible to find a set of coordinates—a combination of parameter values— that best satisfies some criteria; that is, an optimal solution. it is important to recognise that these problems have a ‘timeless’ quality; unless time is a parameter, solutions are otherwise independent of the passage of time. (this is not to say that we do not want them solved as quickly as possible.) if an optimal solution's coordinates were known in advance, then that optimal solution could be arrived at immediately. that is, there is no order to the points in the coordinate space and so any point in that space can be immediately accessed, independently of any other point in that space, simply by setting the parameters to appropriate values. furthermore, the coordinate space is ‘timeless’ in that the parameters themselves never change; their values do, but the space itself will not suddenly acquire a new parameter, an extra dimension. australasian journal of information systems volume 15 number 2, 2009 135 this paper addresses a class of problems that require transitioning through a sequence of intermediate states in search of one that satisfies some solution criteria. that is, even if all the relevant parameters and the solution state were known in advance, one could not necessarily attain that state immediately by ‘leaping’ directly from the initial state to the desired goal state. these problems have an inherent notion of time as a product of two factors: state transitions not being instantaneous; and changes in the set of relevant parameters concomitant with changes in state. in the worst cases, future states are indeterminate and hence fundamentally unknowable. in this paper, we address this type of problem space which can be represented as a directed graph topology. in the following discussion, in the interest of clarity some examples will be based on a problem space in which the goal state is defined as a specific point within a directed graph topology. however, we realise that this cannot adequately exemplify all problem spaces. rather, goals are more generally definable in functional terms. a problem space may contain many points which satisfy a given function. a functionally defined goal state may include a location parameter, as will some examples in the following discussion. alternatively, the function that defines a goal state may include a time parameter such as a deadline. it is a functionally defined goal that allows one to say, “i don't know what it looks like, but i'll know it when i see it.” the parameters of the function may be interrelated in complex ways; the more complex the interrelation, the more difficult it becomes to predict the end-state of a sequence of parameter adjustments. nevertheless, we presume that the use of simpler examples will not detract from the basic argument. psychological foundations we follow the position of jean piaget (1977) in the view that knowledge held by an individual is subjective and constructed from personal experiences and interpretations, resulting in personal (and unique) schemata. furthermore, in the style of thomas kuhn (1970), both individuals and groups can simultaneously maintain and selectively employ seemingly disparate—even conflicting— schemata in parallel, such as classical and quantum theories in the mind of a physicist. the following discussion also presumes the existence of goal-directed intentional agents. that is, an agent’s behaviour need not always be goal-directed but the agent has at least a presupposed desire to continue existing and a presupposed belief in its own ability, given opportunities, to take action in reshaping itself and its environment to satisfy its desires. such an agent may be attributed with an internal locus of control orientation with a high sense of self-efficacy (rotter, 1966; bandura, 2001). an agent’s situation knowledge web (bickhard, 1993) consists of information about the ongoing interactive properties of the environment, which interactions have been engaged in, which sorts of interactions are now available, and which might become available if certain other interactions are engaged in first. “in more complex organisms, such indications may branch and iterate into vast webs of organizations of conditional indications of interactive potentiality. this web is the basic resource for complex representation. such a web constitutes the organism's situation knowledge: knowledge of the potentialities constituting the current situation. a web of any complexity will of necessity be ongoingly changing, due to processes in the environment, new interactions and differentiations, and just from the passage of time: the environment is not, in general, static. the processes of creating, maintaining, and updating situation knowledge are those of apperception.” (bickhard, 2007, p. 4) australasian journal of information systems volume 15 number 2, 2009 136 perspectives on the future we contrast two perspectives on the future which could be adopted for decision-making: a view of the future as stochastic, and a view of the future as intentional. the stochastic future is anticipated by a divergent projection of the immediate past and present onto possible future states. the near future may be predicted, based on known causes and effects, with the assumption that environmental conditions will not substantially change. of course, the stochastic future tends to be quite near-term because environmental conditions do change continually over a continuum of time scales. some changes are highly predictable; some are imperceptible (or at least nearly so) but highly significant, such as an increase in ionising radiation; and some are unexpected and catastrophic. an agent’s relation to change must be somewhat opportunistic. it is important for the following discussion to note that the term stochastic future relates to an agent’s situation knowledge web. that is, it is about the ability of an agent to anticipate potentialities in the stochastic unfolding of events in its world (including itself). in a less formal manner of speaking, an agent’s stochastic future is its knowledge of what lies ahead. this knowledge will usually be incomplete and error-prone, requiring continual reconstruction. in contrast, an intentional future is anticipated by a belief in one’s ability and opportunity to take action in shaping oneself and the environment to realise a predetermined goal-state. the timing of the goal could be in the next second, or the next decade, or unspecified; what matters is that at the present moment, the goal has yet to be realised but at least some preconditions can be defined. taken together, these complementary perspectives may support many real-life decision-making processes categorised by choosing actions in the present to both: (a) manage the unfolding momentby-moment situation; and (b) effect change in oneself and one’s environment to shape future possibilities to converge on a goal-state. the following section presents a detailed example that, though simple and idealised, illustrates how relations between the stochastic and intentional perspectives on the future can be subtle and complicated. stochastic and intentional futures consider a very simple state-space of a type commonly found in games such as checkers. the simplest such space is described by the binomial tree, also known as pascal’s triangle. in such a state-space, states are represented by nodes in a directed graph. from any node, one may transition to only one of two other nodes; one chooses heads-or-tails, left-or-right. figure 1 shows such a state-space with a circular token on a node at the bottom of the figure representing the start state and a node at the top representing the goal state. the objective is to weave the token at start upwards from node to node, choosing either the left or right branch at each step, until either the finish node is reached or it becomes impossible to reach without backtracking (at which time one has failed). if one considers all possible six-step journeys from start, it can be calculated that there are exactly sixty-four (64) distinct paths and all of them will take the token to a node on the finish row. each node in the diagram is labelled by the number of paths that reach it from the start node. however, only twenty (20) of those paths will arrive at the finish node; the relevant branches are highlighted in the figure. if the choice to go left or right is based on the toss of an unbiased coin, then one can calculate the probability of reaching finish from start as: 3125.0 64 20)|( ==startfinishp australasian journal of information systems volume 15 number 2, 2009 137 start finish 1 1 1 1 1 1 1 1 1 1 1 1 1 2 3 3 64 4 10105 5 61520156 figure1: the number of paths to each reachable node in a binomial state-space. by inverting the triangle as shown in figure 2, it can be seen that of all nodes on the start row, the start node itself offers the most paths — twenty (20) — to the finish node. (readers may have noticed symmetry here in that a randomly chosen path from the start node is most likely to reach the finish node, and the finish node is most likely to be reached from the start node.) all possibilities within the stochastic future, from the start node as ‘now’, are revealed in. all possibilities of realising an intentional future at finish are revealed in figure 2. the highlighted nodes and branches in and figure 2 represent the intersection between stochastic and intentional futures respectively. this is a key observation and will return to focus later in the discussion. start finish 1 1 1 1 1 1 1 1 1 1 1 1 1 2 3 3 64 4 10105 5 61520156 figure 2: the number of paths to a given node from all possible starting nodes. australasian journal of information systems volume 15 number 2, 2009 138 it may be intuitive to think that as one approaches a goal state, one’s degrees of freedom will inevitably decrease and thus increasing vigilance will be required to ensure that the goal state is reached. consider figure 3, which shows the token part-way through a journey. it can be seen that there is only one (1) path that will lead to the finish node; the other seven (7) paths will inevitably lead away from the finish node. in this case, the probability of reaching the finish node has indeed decreased to ( )125.08 1 = . it is also worth noting that there are now two most equally likely endstates, neither of which is the finish node. that is, outcome probabilities are changing as the token moves through the state-space. start finish 1 1 1 1 1 1 1 2 3 3 figure 3: only one path out of eight will reach the desired goal. however, even though degrees of freedom may decrease with increasing temporal or spatial proximity to the goal, it does not follow that the risk of missing the goal will necessarily increase. rather, a decrease in degrees of freedom may counter-intuitively afford a decreased vigilance. consequently, it follows that there can be cases in which the goal state can be considered attained long before it is reached—a foregone conclusion—if, in fact, there are no degrees of freedom between the current and goal states. consider figure 4 in which the token has been fortunate to advance to within one step of the finish node. in this case, it can be seen that there are only two (2) possible actions, step-left or step-right, one (1) of which leads to the finish node. that is, as the token has approached the goal, the probability of reaching it has increased to ( )5.02 1 = . figure 4: only one degree of freedom but a greater likelihood of reaching the goal. start finish 1 1 1 australasian journal of information systems volume 15 number 2, 2009 139 this simple thought experiment provides a counter-example to the intuitive notion that as one approaches a goal state, decreasing degrees of freedom will necessarily demand increasing vigilance. that is, the mere decrease in degrees of freedom associated with an impending goal does not per se cause a decrease in the probability of reaching that goal. it was said above that a goal state can be considered attained when there are no degrees of freedom between the current and goal states. however, this may not be true when there is opposition to attaining the goal. consider a hypothetical game in which an opponent is seeking to prevent the token reaching the finish node by blocking the token’s movement options. if the opponent blocks the token’s movement down the right branch off the start node, then it can be calculated that the probability of the token reaching the finish node will remain at ( )3125.032 10 = ; that is, counterintuitively, it will have no effect on the probable outcome. however, the opponent could reduce the probability to zero with a single action simply by waiting for a situation as in figure 3 or figure 4 to arise. although in figure 4 the token has a 50% a priori probability of reaching the finish node, blocking the right branch leaves the token with no way to do so under the rules of the game. under such conditions, it matters not how favourably the token’s weaving carries it toward the finish node; it is simply impossible for the token to reach it from the very outset. probabilistic decisionmaking is futile when the probabilities can be forced to zero or one. to conclude: on approaching a goal state with decreasing degrees of freedom, the probability of reaching it increases under no opposition; but should opposition be encountered, the effectiveness of that opposition also increases. the intersection of future perspectives figure 5 depicts the common experience of not being able to predict sufficiently the stochastic and intentional futures. in such cases, the stochastic and intentional futures are being continually (re)constructed. the agent has advanced its token through a few state transitions. at each state in the sequence, it has been able to see only a step or two ahead. furthermore, it can only foresee the last step or two from which it might reach the goal state. however, at the moment depicted in the figure, the stochastic and intentional futures have not intersected. we can now more precisely define what was called the ‘point’ of temporal convergence (dalmaris, hall and philp, 2006) as the moment when an agent identifies an intersection region of the stochastic and intentional futures. in contrast, temporal divergence applies to the moment when it becomes apparent that the stochastic and intentional futures cannot intersect. (however, all is not lost if the agent can survive the failure to reach the desired goal. instead, surviving a case of temporal divergence provides the agent with an opportunity to learn. this will be addressed in a later section.) the phrase, “it becomes apparent” was used above; but apparent to whom? this is especially problematic for the definition of temporal divergence. it may be apparent to an omniscient observer, but not an involved agent, when the moment of temporal divergence has passed and failure is inevitable. in relation to figure 5, if the token was to take two more left branches, we may imagine filling in the graph to show that temporal divergence will have occurred at the moment of taking the second left. an agent may continue to engage in activity for quite some time before they realise that temporal divergence has occurred and failure is thus inevitable. in many real-world cases, the question of whether stochastic and intentional futures temporally diverge may be undecidable. recognising temporal convergence requires, at a minimum, finding a single point of intersection between the two futures. however, recognising temporal divergence requires either an exhaustive search of both futures, or a mathematical proof, to ensure that there can be no point of intersection. how can one be sure that an intersection is not “just around the next corner?” (in such cases, time limits become important parameters of functionally defined goal states.) in case of confusion, one australasian journal of information systems volume 15 number 2, 2009 140 should be prepared to specify whether one is speaking of a moment of temporal convergence or divergence in `absolute’ terms from the perspective of some detached, omniscient observer; or relative to the perspective of the agent(s) involved. start finish 1 1 figure 5: stochastic and intentional futures which have not yet intersected. finally, note that it is possible for temporal divergence to occur after temporal convergence. the “second left” mentioned above could occur as in figure 4 where clearly the two futures have intersected but, by chance, a single action leads to instant temporal divergence. temporal convergence is about finding the intersection between stochastic and intentional futures. substantiating the intentional future is another matter because the intersection changes as momentby-moment the anticipated potentialities of the stochastic future collapse into the experienced present. temporal divergence can occur at any time. processes of temporal convergence and divergence as well as moments of temporal convergence and divergence, one may also speak of processes of temporal convergence and divergence. a process of temporal convergence is one which seeks to construct a non-empty intersection region between stochastic and intentional futures; that is, it seeks a moment of temporal convergence. a process of temporal divergence is one which seeks to ensure that a non-empty intersection region between stochastic and intentional futures cannot be constructed; that is, it seeks to preclude moments of temporal divergence. a single agent may have the computational resources required to be simultaneously constructing both stochastic and intentional futures. however, in some contexts there may be two agents involved where each agent's computational resources are sufficiently consumed by the process of constructing just one of the futures. in such cases, temporal convergence will occur through both agents engaging in a dialogue as they search for a point or region of intersection between their two respective futures, or until it becomes clear that temporal divergence has occurred and the two futures cannot intersect. australasian journal of information systems volume 15 number 2, 2009 141 temporal divergence applies to both failure to attain one's desired goal, and prevention of an opposing force's attainment of its goals. by opposing forces, we mean that each force has its own intentional future which, at least in part, cannot intersect with the other’s. on the other hand, there is a ‘commonality’ between the two forces’ stochastic futures in that each one’s stochastic future includes a representation of the other force and the other’s stochastic future. both forces are embroiled in the conflict and each is somewhat aware of the other. note that the two stochastic futures may not intersect at all if either force is completely mistaken about the other’s situation. nonetheless, in times of conflict each force will seek to construct a functional stochastic future that includes the other force, and an intentional future purporting to be of the other force. furthermore, each force from its own perspective is trying to temporally converge what it believes to be a common stochastic future with its own intentional future while simultaneously trying to temporally diverge the common stochastic future with the other's intentional future. in this there is much potential for error because each force is trying to construct five things: a common stochastic future (sf); their own intentional future (ifself); the other's intentional future (ifother); the temporal convergence of sf and ifself; and the temporal divergence of sf and ifother. any of these constructions may be insufficiently complete or in error. application to military knowledge management the conceptual framework described above has general application to any strategic business operation, but perhaps the best source of apposite examples is the military domain. military operations are characterised by danger, high-tempo and uncertainty, and military practitioners have responded with command hierarchies, formal documentation and intensive training in standardised procedures. publicly available military policy documents contain terms whose meanings and implications can be clarified by recasting into our proposed framework. this in turn may illustrate the applicability of such concepts to other strategic business operations. three public-domain military concepts will be considered here: • commander’s intent; • shared situation awareness; • self-synchronisation. commander’s intent commander’s intent, in our terms, is the declaration of an intentional future by the commander. it is to be expected that the intentional future of a military campaign will be more difficult to define than that of our simple pascal’s triangle example above. however, the most difficult aspect of this is revealed when one considers that it is not possible for a commander to directly imprint his intentional future in the minds of his subordinates. instead, each subordinate must engage in a private process of situation knowledge web construction that will, hopefully, result in a situation convention (bickhard, 1980, 1992, 2003) that includes the same intentional future as the commander’s. military tradition and training practices have long addressed this problem through uniform training, and by raising up commanders through the ranks, in the hope that uniformity of behaviour and experience will produce uniformity in situation conventions and processes of situation knowledge web construction (in our terms)… but this is by no means guaranteed. consider a situation in which a commander gives a verbal briefing to his command staff who then prepare a series of written briefs for field commanders. the commander’s verbal briefing is the result of a highly complex constructive process drawing upon his private situation knowledge web and his personal perspective on the interaction with his command staff. consequently, an intentional australasian journal of information systems volume 15 number 2, 2009 142 future in each staffer’s private situation knowledge web is constructed by a similar but uniquely personal process. thus with a commander and n staff, we have n+1 processes all engaged in the construction of private situation knowledge webs and public artefacts (verbal utterances, handwritten notes, sketches, etc.) through which the processes interact. another set of n processes is then engaged to construct the written briefs for the field commanders. strictly speaking, a written brief is not actually the commander’s intent per se. rather, it is an artefact constructed by a command staffer in the hope that it can be used by a field commander as a resource in constructing an intentional future that intersects the intentional future of the commander. we should be amazed that so many distinct processes can produce intentional futures that sufficiently intersect. this is far more than a question of mere data-replication accuracy. analogies are often drawn with computers exchanging data over networks but such analogies are misleading because computers are far more uniform than organic entities such as people. furthermore, the capabilities of computers to sense and appreciate their environment are far more limited. that is, the universe of a computer is very simple. even so, data that was perfectly replicated can still be inappropriately processed. for example, a classic class of errors arises from confusion over what is called endian-ness. if a sending computer uses a numeric format that issues the most significant bit first, but the receiving computer uses a numeric format that expects the least significant bit first, then it doesn’t matter that the bits are perfectly replicated over the transmission medium; the overall result of the entire process is confusion. thus, a focusing of effort on network construction is misguided; the focus of effort must be on the constructive processes within each node. shared situation awareness in the definition of situation awareness as “the perception of the elements in the environment within a volume of time and space, the comprehension of their meaning, and the projection of their status in the near future” (endsley, 1988) we see a stochastic future. shared situational awareness then exists only in as much as two or more agents can engage in similarly constructive processes with similar resources such that their stochastic futures are likely to have a high degree of intersection. as already described with regard to intentional futures, this is no mean feat. within a command headquarters, shared situational awareness is the intended product of the continual back-briefing that occurs after the commander’s staff take his statement of intent and generate plans by which it may be achieved. as already mentioned, the chief of staff acts as the nexus through which the stochastic future is gradually constructed by the command staff shops. when the staff believe that they have identified a temporal convergence between the commander’s intentional future and their shared image of the stochastic future, this is presented to the commander as a plan to achieve his intent. in as much as the command staff have training and experience in common, and they share an immediate environment and resources, and there is an individual—the chief of staff—as a nexus, it is reasonable to expect that that there may be shared situational awareness. however, all this means is that the individuals have engaged in similarly constructive processes with similar resources such that their stochastic futures have a high degree of intersection. it does not mean that they are ‘complete’ or ‘correct’ in their understanding of the situation from an omniscient observer’s point of view. nor does it mean that they have not fallen victim to phenomena such as groupthink or polarisation; in fact, it is likely that they have (whyte, 1989). finally, a field commander or a soldier who does not have the same constructive resources on which to draw in constructing their own stochastic future is not likely to share the situation awareness of the command staff. australasian journal of information systems volume 15 number 2, 2009 143 self-synchronisation self-synchronisation “involves subordinate commanders re-tasking their units so that the commander’s intent is maintained… [and] helps warfighters adapt to changing circumstances and allows them to apply a warfighting advantage more effectively” (director general capability development and plans, 2006, pp. 10–11). in this definition we see references to both stochastic and intentional futures; stochastic with regard to the warfighter’s changing circumstances, and intentional with regard to the commander’s intent. the effective application of warfighting advantage is about identifying and substantiating temporal convergence between the two futures. the term self-synchronisation is perhaps a misnomer. the concept is more about subordinate units interacting directly to achieve their commander’s intent without needing their commander’s control or mediation. it is claimed that “shared situational awareness will enable self-synchronisation” (director general capability development and plans, 2006, p. 11). however, it is clear that the conditions necessary for shared situational awareness are a proper subset of the conditions necessary for selfsynchronisation. therefore, shared situational awareness is necessary for self-synchronisation but far from sufficient. self-synchronisation between subordinates a and b, with commander c, requires that: • a and b each hold an intentional future that intersects with c’s; • the stochastic futures held by a and b substantially intersect; • a and b have each identified a temporal convergence between their respective stochastic and intentional futures; • the temporal convergences of a and b substantially intersect; • in the temporal convergence held by a, there is a representation of b. ideally, b would also hold in his temporal convergence a representation of a. all the above conditions are required in order for a to reasonably select an action to perform in anticipation of (i) b’s actions or needs, and (ii) satisfaction of c’s desired end-state. it should not be surprising if self-synchronisation is difficult to achieve given the highly demanding conditions required for it to occur. temporal divergence and learning it was said above that surviving temporal divergence provides an agent with an opportunity to learn. in fact, a learning process could be engaged at any time that temporal divergence is even suspected although doing so may be costly in terms of processing resources. the following discussion outlines the nature of knowledge required for agents to learn from temporal divergence. knowledge for goal-driven agents we can formalise our approach thus far by considering an agent (or decision maker) to be goaldriven when, in the simplest case, the agent performs some action to achieve a desired outcome. (note, outcomes are always specified in terms that are internal to the agent. we take as trivial the case where the agent has already realised all desired outcomes, with no further action, or the agent is passive and merely waits for the desired outcome to occur through environmental flux.) if some actions lead to desirable outcomes, then by implication others do not. for some actions, there may be multiple possible outcomes of which only some are desirable. whichever outcome is realised australasian journal of information systems volume 15 number 2, 2009 144 may depend upon factors in the current situation defined as the totality of state with respect to both agent and environment. for our discussion, it does not matter where the influencing factors lie, in the agent or the environment, or both: what matters is that a desirable outcome will be realised from only some, not all, situations. for an action-outcome pair, any given situation may be classified in simplest terms as leading to either success or failure. in order to perform this classification, the agent performs the action and notes whether or not the anticipated outcome is realised. in other words, action-outcome pairs serve to differentiate situational classes. this paper introduces the concept of bifurcation of the situation knowledge web as the outcome of a process by which an agent uses an action-outcome pair to differentiate a situational class. bifurcations of the situation knowledge web are behind sentences that begin, “if i had known then what i know now…” consider the simplest possible case, as illustrated in figure 6. at time (t=0) representing the agent's present, the agent projects the future performance of a particular action at time (t=1). proceeding from that action are two parallel arcs to two different possible outcomes at time (t=2). however, which outcome will be realised? the agent's present situation at time (t=0) is undifferentiated with respect to the two outcomes in that the agent cannot predict which outcome will be realised at time (t=2). at time (t=1) the agent will perform the action. by time (t=2) there will no longer be any question. one of the outcomes, either a or ~a, will have been realised and the agent will be able to bifurcate its situation knowledge web to determine which class of situation held at time (t=0). that is, at time (t=2) the agent will have the hindsight to know which situation it was in at time (t=0): either a situation leading to particular outcome a, given a particular action; or a situation leading to some outcome other than a (i.e. ~a) given that same action. figure 6 shows the stages of the bifurcation process from the outcome realisation back to the identification of the actual situation. if there is no possibility of any undesirable outcome, then the agent has nothing to lose by trying the action. however, it does the agent no good to determine that it was unsafe to perform an action by performing that action! a risky situation for an agent can be characterised by three properties: (i) an action has either a distinctly desirable or distinctly undesirable outcome; (ii) the agent intends to attain the desirable outcome; and (iii) the desirable outcome cannot (to the agent's knowledge at that moment) be attained through alternative, safer actions. it becomes vital for an agent to somehow determine that the current situation is a member of the desirable-outcome class, as opposed to the undesirable-outcome alternative, before performing the action. (in practice, there could be a number of actions that the agent may perform. however, whether there is one action or many, the principle remains that a wise agent would determine the class of situation before acting.) suppose that outcome a in figure 6 is a desirable, safe outcome while outcome ~a is an undesirable, unsafe alternative. the agent needs to determine at time (t=0) if it is safe to perform the action at time (t=1). this may be achieved if the agent can identify some other action—with multiple possible outcomes, all of which are safe—to perform between times (t=0) and (t=1) such that the outcome realised will bifurcate the situation knowledge web. a stage in this process of ‘active sensing’ is shown in figure 7. the agent is choosing to perform action x, not because either outcome y or ~y is a desirable end in itself, but because of what the realised outcome will indicate for future action-outcomes. this is different to either passively waiting for whatever sensory input arrives, or applying some filter to all sensory input received. rather, the agent engages in activity that is intended to elicit or invite a particular class of sensory input/information from the environment, where the absence of that resultant stimulation may be just as telling as its presence. australasian journal of information systems volume 15 number 2, 2009 145 note that what we have been calling the “current situation” is actually the ultimate outcome of the penultimate action in a (potentially very long) sequence of overlapping outcome-action-outcome triads. bifurcation could be continued into the agent’s history of actions and outcomes, but this is not necessary. for the agent to predict the realisation of either outcome a or ~a, it is sufficient merely to stop bifurcation at action x. note that at time (t=0), action x is yet to be performed. that is, the agent is able to make decisions for action entirely on the basis of future events without any reference to, or even awareness of, past or current situations. readers who are familiar with traditional rule-based expert systems should note that to encode a situation knowledge web would require more complex rule expressions than is traditional. the difference is illustrated in table 1. in short, each traditional condition-action rule is bracketed by a preceding action and an expected post-action condition. agent's time (t) relative to the present t=0 t=1 t=2 stage 1 stage 2 stage 3 action action action action action outcome a outcome a outcome a outcome ~a outcome ~a outcome ~a situation ? situation ? situation affording outcome a situation affording outcome ~a figure 6: the process of situation knowledge web bifurcation. agent's time (t) relative to the present t=0 t=1 t=2 action x action action outcome y outcome a outcome ~y outcome ~a situation ? figure 7: active sensing. australasian journal of information systems volume 15 number 2, 2009 146 traditional situation knowledge web if condition c holds, if action x was performed resulting in condition c holding, then perform action a. then perform action a expecting condition y to hold. table 1: example rule expressions. the anticipation of outcomes from actions allows the agent to detect when its representations are in error. if a situation knowledge web composed of linked action-outcomes is a representation of the world, in that it can differentiate classes of situation, then it is possible for an agent to detect for itself when its representation is in error. the agent does not need a homunculus, nor an external advisor, to know when an anticipated outcome was not realised. (note, with one condition: the outcome must be specified only in terms from the domain of the agent's internal organisation.) the agent may not know what to do about the fact that an action produced an unexpected outcome, but it must at least recognise that there is an error to address, since the ability to detect that the world is not as expected is a fundamental requirement for any adaptive system. implications for knowledge management systems in any large organisation, we may find some parts in which there is rarely any doubt about the current situation, how it transpired and what to do next. we may also find other parts that frequently deal with novel or fluid situations. in the former case, situations are static or change in small degrees at slow time scales relative to the rate at which organisation members can adjust their situation knowledge webs. the organisation has the opportunity to create and disseminate procedural documentation via content management and portal systems. long-serving members seem to know without thinking how to react to particular situations because their stochastic and intentional futures have been well-formed—they have ‘seen it all before’—possibly to the point of being rather fixed. but in the latter case, the threat of temporal divergence is common and members’ situation knowledge webs undergo frequent (though costly) bifurcation and expansion. longserving staff are valued not just for the complexity of their situation knowledge webs built up in the course of much experience, but also in the skilled and relatively efficient processes they have developed for situation knowledge web construction and bifurcation, the ability to create a situation knowledge web being more valuable than the image itself. people who develop the latter capability have been ‘learning how to learn’ (bateson, 1973) and their fluency in constructing new knowledge is at least as valuable as the knowledge they have already constructed. success is highly dependent on the speed and efficiency with which temporal divergence can be detected and addressed through situation knowledge web bifurcation processes. if knowledge management systems are to be employed effectively beyond well proscribed domains, then beyond focusing on the storage, search and retrieval of documents and records with a focus on the past, they must begin to expressly support stochastic and intentional future concepts. ideally, they will even come to support meta-processes by which the processes of situation knowledge web construction and bifurcation are enhanced. we recognise that it may be difficult for some readers to accept ‘knowledge’ as having a future aspect. some philosophers hold that the future is epistemologically unknowable because: it cannot be experienced (e.g. empiricists); and induction from past events (or beliefs about past events) is prone to error, thus uncertain, thus unworthy of the verb to know (e.g. falsificationists). however, the kinds of knowledge in question here relate to one’s own and others’ future goals, and the likelihood of future events and how preceding actions and events may shift that likelihood. that is, we are not talking about knowledge of the future but knowledge of possible futures and of agents’ intentions for the future. because this ‘future’ australasian journal of information systems volume 15 number 2, 2009 147 knowledge is never certain, knowledge management systems must accept continual reconstruction. the time-stamped record of a past event or state may later turn out to be fictional, but that the record was made and was believed to be factual at the time of articulation is another kind of valuable knowledge that has a lifetime distinct from the record itself as long as it features in the situation knowledge web of some agent. the promulgation of false truths and intentions is a commonly used method for inducing temporal divergence in an opponent’s situation knowledge web, and that such falsehoods are available and amenable to that purpose is also a kind of knowledge. data aggregation systems and alerting systems may allow the anticipation of events. good search and retrieval on content management systems may allow those faced with unfamiliar situations to filter a large volume of prior experience and records for ideas and possibly matching situations. better still for organisational knowledge management systems to support the continual construction, extension and sharing of members’ situation knowledge webs (cf. bosse et al., 2005). conclusion dalmaris, hall and philp (2006) attempted to extend the knowledge-intensive business processes (kbpi) to incorporate the idea of the time-value of knowledge, defined as a qualification helpful in assessing the applicability of knowledge to solving problems with a variety of time horizons. this examination extended from the simple observation that knowledge requirements—knowledge objects, relationships and supporting infrastructure that make up business processes—for the typical knowledge worker such as a help desk customer support agent, were very different to those of a military commander or ceo, who selects actions in the present on the basis of future effects. this paper has presented a framework of terms and concepts to assist exploration and discussion of the types of knowledge required for complex endeavours, such as warfighting, characterised by opposition and uncertainty. we are not alone in arguing for the central importance of planning and human cognition as goal-directed constructive processes (bryant, 2003). knowledge management systems may play a very important role in that a commander needs to know, above all else, when a temporal convergence has been found between his intentional future and the stochastic future, or when temporal divergence has occurred and it is time for a strategic withdrawal and change of intent. in between is the grey area in which a commander does not know whether success is either possible or impossible, and he will draw on information from all sources and ask his knowledge management systems to help him substantiate temporal convergence or divergence. finally, he will want to use knowledge management systems in the production and dissemination of artefacts that his subordinates may use to (re)construct their own situation knowledge webs which primarily represent beliefs about relationships between possible actions and events that may or may not happen. references bandura, a. (2001), “social cognitive theory: an agentic perspective”, annual review of psychology, vol. 52, pp. 1–26. bateson, g. (1973), steps to an ecology of mind, paladin, st. albans, uk. bickhard, m. h. (1980), cognition, convention and communication, praeger, new york. bickhard, m. h. (1992), “how does the environment affect the person?”, in winegar, l. t. and valsiner, j. (eds.), children's development within social contexts: metatheory and theory, lawrence erlbaum associates, mahwah, nj, pp. 63–92. australasian journal of information systems volume 15 number 2, 2009 148 bickhard, m. h. (1993), “representational content in humans and machines”, journal of experimental & theoretical artificial intelligence, vol. 5 no. 4, pp. 285–333. bickhard, m. h. (2003), “the social ontology of persons”, in carpendale, j. i. m. and muller, u. (eds.), social interaction and the development of knowledge, lawrence erlbaum associates, mahwah, nj, pp. 111–132. online as <http://www.lehigh.edu/~mhb0/socontpersons.pdf>, last modified 06 february 2003 17:08:00 gmt, last accessed 13 march 2008 23:02:25 gmt. bickhard, m. h. (2007), “interactivism”, online as <http://www.lehigh.edu/~mhb0/interactivismrout2jan07.pdf>, last modified 23 january 2007 20:03:50 gmt, last accessed 13 march 2008 22:29:50 gmt. bosse, t., jonker, c. m., schut, m. c. and treur, j. (2005), “simulation and analysis of a shared extended mind”, simulation, vol. 81 no. 10, pp. 719–732. bryant, d. j. (2003), critique, explore, compare, and adapt (ceca): a new model for command decision making, defence r&d canada, toronto. dalmaris, p., hall, w. p. and philp, w. r. (2006), “the time-value of knowledge: a temporal qualification of knowledge, its issues, and role in the improvement of knowledge intense business processes”, in proceedings of the 3rd asia-pacific international conference on knowledge management (kmap06), hong kong, 11–13 dec 2006. director general capability development and plans (2006), explaining ncw, defence publishing service, department of defence, commonwealth of australia, canberra, act 2600. endsley, m. r. (1988), “design and evaluation for situation awareness enhancement”, in proceedings of the human factors society 32nd annual meeting, human factors society, anaheim, ca, pp. 97–101. kuhn, t. (1970), the structure of scientific revolutions, 2nd edition, university of chicago press, chicago. piaget, j. (1977), the development of thought: equilibrium of cognitive structures, viking, ny. rotter, j. b. (1966), “generalized expectancies for internal versus external control of reinforcement”, psychological monographs, vol. 80. whyte, g. (1989), “groupthink reconsidered”, the academy of management review, vol. 14 no. 1, pp. 40–56. microsoft word 13 1 marina.doc ajis vol 13, no. 1 september 2005 192 requirements engineering: a close look at industry needs and model curricula oliver minor university of koblenz-landau oliver.minor@web.de jocelyn armarego murdoch university, perth jocelyn@eng.murdoch.edu.au abstract research endeavours in software development have found that failures and deficiencies of software systems are often rooted in the requirements activities undertaken. one possible cause for poor requirements activities is the appropriateness of the education of those engaged with the requirements component of software development. this education is largely based on model curricula used as guidelines. this paper examines the requirements component of model curricula in the disciplines of computer science, information systems and software engineering. these are compared to the opinions of a small but representative group of practitioners, assembled through personal interviews. the results reveal that the model curricula address to a high degree the expectations on the formal education preparing for requirements activities practitioners have mentioned. however, the results also show that practitioners see shortcomings in formal education, particularly with respect to more generic skills, such as communication and team skills. introduction the information and communication technology (ict) sector is considered to have major relevance for the whole economy of australia. a recent study states that if a better and increased education in this field were conducted it would have substantial influence on the productivity and the overall performance of the economy (cie 2001). re, as a foundational element of the development of computer-based system, is central for ict. re as a fundamental discipline in the development of systems and software has been widely recognised as crucial within the last several years (alexander and stevens 2002; ferdinandi 2002; hull, jackson et al. 2002; young 2002; bray 2003). as early as 1976 bell and thayer observed that inadequate, inconsistent, incomplete, or ambiguous requirements have a critical impact on the quality of the resulting software (cited in van lamsweerde (2000)). surveys and studies underline the pivotal character of requirements engineering (standish 1994; esi 1996; al-karaghouli, alshawi et al. 1999; lee, dutta et al. 1999; van lamsweerde 2000). other studies reveal problems in communication (al-rawas and easterbrook 1996), monolithic and overloaded requirements in commercial-off-the-shelf-software projects (karlsson, dahlstedt et al. 2002) or in cultural differences in multi-site software development organisations (zwoghi, damian et al. 2001). yet another research project reveals that contingencies of the project, characteristics of the project managers and the composition should be considered (carroll and swatman 1999). this shows a variety of challenges have to be met, and also reveals opportunities to improve the re process. to tackle these challenges and make use of the opportunities, novice requirements engineers should be equipped with appropriate skills and knowledge. yet conn (2002) reports that it is a surprise to graduates that requirements is a major cause for software deficiencies. ajis vol 13, no. 1 september 2005 193 despite the number of books, articles and research findings published, the transfer and adaptation from these sources into practice has been seen as difficult (nikula, sjaniemi et al. 2000). morris et al. (1998) examined through workshops how companies absorb knowledge/knowledge diffuses from the academic world into practice. the workshop participants identified training as a key problem (amongst other problems). a main way of technology transfer into practice is training. training and education is often based on literature, though nguyen et al (2002) recognised that the actual practice of requirements engineering does not conform to its presentation in the literature. not only publications transfer knowledge into practice but education also has an influence (due to its roots in literature). lethbridge (2000) surveyed software practitioners in order to find out the relevance of their software engineering and computer science education. 60% of the respondents considered that requirements gathering and analysis is under-taught in education (lethbridge 1998; 2000). as a result it can be assumed that teaching does not reflect the needs of the practice. considering weaknesses in the requirements analysis or requirements engineering, and hints found in surveys about the education in computer science and software engineering as well as information systems, this paper examines the relationship between the opinion of practitioners and current model curricula in the respective disciplines. model curricula built the fundament and guidelines for tertiary education. before describing the research design the model curricula considered in this research are summarised. based on the research design the results and the implication for education are then presented. background reviewing literature on requirements engineering and systems analysis (davis 1993; macaulay 1996; robertson and robertson 1996; kotonya and sommerville 1997; leach 1999; nickerson 2001; kendall 2002), several skills and knowledge areas arise. table 1: knowledge and skills topics requirements engineering activities feasibility study elicitation determination analysis documentation verification requirements management success factors generic skills management of self management of information group/team skills management of tasks problem/opportunity identification general problem-solving strategies communications skills cultural insight / professionalism ajis vol 13, no. 1 september 2005 194 meta-cognitive strategies analytical skills these areas derive from process activities at the beginning of a software development/enhancement project. table 1 gives an overview of relevant topics. this table is neither perfect nor comprehensive nor complete. however, it does give an underlying framework for • the comparison of the model curricula • the questionnaire used in the interview study and • analysing the answers of the interview study. the table enables a transparent and reproducible process for analysing the curricula. a curriculum should reflect up-to-date research as well as the practice (avison, fitzgerald et al. 2001) in a discipline. work-groups of acm and ieee-cs (among others) have integrated requirements engineering in the model curricula and boks of computing curriculum – computer science (cccs) (engel and roberts 2001), the model curriculum and guidelines for undergraduate degree programs in information systems (is2002) (gorgone, davis et al. 2002; gorgone, davis et al. 2002a) and the computing curriculum – software engineering (ccse) (engel and roberts 2001; díaz-herrera and hilburn 2003). they are part of the so called computing curricula (engel and roberts 2001) effort. what the curricula have in common is that they present knowledge areas that each graduate of the respective discipline should know to a certain degree. several research endeavours have examined the industry expectations of graduates. doke and williams (1999) give in their article an overview of published research in the field of information systems. lee et al. (1995) examined the importance of different topics of information systems with the help of focus group interviews, forums and a survey among practitioners. in another study noll and wilikens (2002) examined what information system workers perceive as important skills and knowledge for future employees in information systems. turner and lowry (1999) asked students and company representatives, mainly human resource employees, about their perception of what is considered to be required on the job. lethbridge (1998; 2000) examined the relevance of computer science, computer engineering and software engineering education. requirements gathering and analysis was ranked among the top five regarding overall importance, although the amount learned by the respondents during their formal education was evaluated rather low. therefore a significant difference exists between the amount learned in formal education and the current knowledge level: this may indicate that requirements gathering & analysis is not considered in formal education to the extent it should. macaulay and mylopoulos (1995) compared courses from ten international (mainly from the uk, but also us, spain, canada, israel) universities and an industrial perspective of requirements engineering. all the above described studies cover in some way skills and knowledge needs for systems analysis and requirements engineering. however, they mostly only examine the general importance perceived by different stakeholders, such as practitioners, human resource staff or students. none of them examines in detail the activities that are necessary to perform systems analysis/requirements engineering and whether skills and knowledge needed for these activities are reflected in the respective model curricula. as the requirements process is a human endeavour (kotonya and sommerville 1997, formatted: bullets and numbering ajis vol 13, no. 1 september 2005 195 p.141) it is highly complex to find out what makes it a successful endeavour. therefore, it does not seem enough to find the underlying success factors of requirements engineering in asking for the importance of this topic. it seems rather interesting to search for topics that are crucial for successful requirements engineering. this search has been pursued already; the result of that can be seen in literature, which was presented very selectively in this chapter. the influence of topics that have been identified as crucial for successful requirements engineering on the practical undertaking of requirements engineering and their reflection in model curricula has not been subject of research yet (at least it did not occur to the authors during their background research). therefore, this paper tries to answer the following questions: • which knowledge and which skills are necessary to successfully conduct requirements activities? • which personal characteristics are needed? in order to find this out it seems necessary to ask how requirements engineering is conducted. in a second step the answers can be used to examine existing model curricula with respect to their accordance to the given findings. • how do model curricula guide towards required skills and knowledge? • which areas are neglected and which are covered by existing model curricula? research methods requirements engineering is a real world discipline applied in many areas of contemporary industry and since it involves heavy human interactivity it is suitable to use qualitative research methods (loucopoulos and karakostas 1995; leedy and ormrod 2001). teaching is also considered as a process highly involving human behaviour. the analysis of model curricula as the basis of teaching is also of qualitative nature because textual content (data) will be interpreted by means of human thinking and structuring tools. this research is therefore based on a qualitative approach with a small portion of quantitative analysis when analysing learning objectives. the questions asked as research questions above are aimed at evaluating model curricula. a three-step process was applied: • data gathering • analysis • presentation of analysis results the data gathering process was aimed at establishing a comprehensive view on (a) what is understood by requirements engineering and (b) practitioners’ perception of required skills, knowledge, and personal characteristics. in order to achieve a basic understanding of requirements engineering the results of a literature review were used. the structure and the content of the questionnaire as well as the analysing frameworks were based on these results. however, it is acknowledged that the view presented in the literature review is limited and biased by the selection and interpretation of authors and their texts. since the model curricula were also used for designing the interview guidelines certain limitations must be considered. the practitioners’ perceptions were captured with semi-structured personal or telephone ajis vol 13, no. 1 september 2005 196 interviews. patton (1990) calls it general interview guide approach. through the interviews opinions and experiences have been explored. personal interviews have the advantage that complex issues can be examined and discussed. furthermore, personal interviews raise a more conversation-like interview (patton 1990). an initial pilot interview and several informal reviews were made in order to improve the effectiveness of the questions. the pilot interview was used to improve the questionnaire with respect to wording and question sequence. thirteen interviews (excluding the pilot interview) had been conducted in six organisations of which two are considered heavy on computer science, two on information systems and two on software engineering. the selection of the organisations was a purposeful sampling (patton 1990). the organisations appointed staff members as interview partners under the conditions that the interviewees work in the field of requirements engineering and are not graduates of school of engineering science at murdoch university. the interviews were conducted with the ethics approval of murdoch university human research ethics committee with the permit number 2003/220. interviewee and company names are recoded for privacy reasons. after having set up appointments for the interviews, the interviewees were sent a letter of consent and the skills matrix to give them a first impression of the research. a three-page questionnaire was handed over to the interviewee at the beginning of each the interview. the interviews were recorded on tape and transcribed afterwards for examination. two interviewees denied their approval to the tape recording. some of the interviewees were later contacted for clarification, verification and probing questions via telephone. a second means of capturing information from the interview participants was a web-based questionnaire (armarego 2003). this questionnaire was mainly used to verify findings of the interviews. ten out of 14 (including the pilot interviewee) interviewees filled out the questionnaire. besides the interviews and web questionnaire, an internet search was conducted to uncover general company information of the interviewed organisations. during the data gathering phase a first, mainly implicit, analysis of data was done. the main analysis however, was performed in the aftermath of the data gathering. the interviews and the subsequent telephone follow-up were transcribed. the analysis of the interview transcriptions was done through a framework analysis, also called template analysis (richie and spencer 1994; king 1998; lacey and luff 2001). identifying the thematic framework (or template (king 1998)) was based on the topics that occurred in the literature and curriculum review and were therefore also represented in the interview questionnaire. reading, re-reading and listening to the interviews was the core activity for finding statements on the categories identified in the thematic framework. the thematic framework was modified during the course of analysis. derived from the structure of the questionnaire three main frameworks were used for coding the data: • company settings, • interviewee’s education and career path, and • the interviewee’s perception of requirements engineering, the needed skills and knowledge. after having categorised the data, they were analysed systematically for commonalities, differences and interrelationships. framework (3) is also used for analysing the model curricula. that enables the core ajis vol 13, no. 1 september 2005 197 examination on how practitioners view requirements engineering in comparison to model curricula. to overcome credibility issue (patton 1990; miles and huberman 1994) in this study data from personal interviews and from an accompanying web survey were used for triangulation. the content of the answers must be viewed critically. as argyris and schön ((1974) referred to in (anderson 1997)) describe in their work about espoused theory and theoryin-use, the interviewees’ answer may not completely reflect their actions. that means that the interviewee might say something about their requirements activities (espoused theory) but act differently (theory-in-use). results and discussion all companies had an international focus for their software development. besides two global players with more than 10,000 employees, the companies were in the range between 2,000 and 3,000 (is1, se1) or less than 50 (cs1, cs2). the number of people involved in software development at the premises in perth ranges from 12 up to 200 people. the companies are involved in such industry areas such as geographical information systems, image-processing software, financial sector, business information systems, defence industry, and telecommunication. all interviewees had a senior role in their company. they can be classed in the middle management and upper management. the interviewees’ involvement with requirements activities can be categorised in receivingdevelopers, activelyinvolved, or supervision. receivingdevelopers primarily receive requirements that they have to turn into design or code. one interviewee oversees these activities, so he can be classed in supervision. the other interviewees are activelyinvolved in requirements activities. these activities can either involve direct contact with the customers and users of the future system or be through sales and support people. ten interviewees have a bachelor of science degree or equivalent, one holds a diploma in education, one an associate diploma in computing and another did not attend any tertiary institution. the interviewees studied subjects such as computer science, information science, software engineering, mathematics, physics, biology, or chemistry. seven interviewees received their degrees (including the associate diploma) in australia, five from universities overseas. their practical experience in software related jobs measured in years is between seven and 25 years. analysis presentation of interview statements by category this section presents the analysis results of the statements given by above described interviewees. requirements engineering process the interviewees stated that between 5% and 25%, mostly around 10%, of their working time is spent on requirement related topics. some interviewees see the requirements phase as a distinct process (thomas, se1). the reason for that might lay in the strict compliance to a standard process. compliance to process standard is also mentioned: the one line codeajis vol 13, no. 1 september 2005 198 change is very controlled (john, se2). cs2 does not seem to have a formalised way for requirements, i didn’t really write any of this down. it just goes without saying i guess at some level. (simone, cs2). this shows that there are fundamentally different ways of approaching requirements activities. it varies from much formalised processes with well-defined sign-off points to requirements activities that are more implicit. two main demands on curricula could be drawn out of these statements: • students should know that the requirements process can vary tremendously. cccs (engel & roberts, 2001) demands that students understand the importance of the requirements process. software engineering students should comprehend the process and apply current theories, models, and techniques that provide a basis for problem identification and analysis (díaz-herrera & hilburn, 2003, p.10). is2002 (gorgone et al., 2002) postulates a knowledge about the life cycle model in general. the model curricula demonstrate certainly theoretically sound processes. they do not mention that real life processes might work differently. it might be a political matter, whether to poison students with imperfect processes or to teach them clinical processes. • students should understand the sense of process standards. the curricula cccs (engel and roberts 2001) and ccse (díaz-herrera and hilburn 2003) demand students to comprehend process standards. comprehension means the ability to grasp the meaning (díaz-herrera and hilburn 2003). in contrast, is2002 (gorgone, davis et al. 2002) does not mention process standards. feasibility study through the feasibility study the economical, political or technical feasibility for a project and parts of it is tested. most companies have no formalised feasibility study before entering a project. one interviewee describes the feasibility study as looking at the requirements or we are analysing whether it [requirement] makes sense (john, se2). it indicates that feasibility is tested with the experience and knowledge in the area where the requirement occurs. estimation techniques play a role in the feasibility study. depending on the initially estimated size of the proposed project either an informal estimation or a formal estimation is performed. the informal way of doing can be boiled down to gut feeling (john, se2). in literature it is called expert judgement or educated guess and relies on experience (pfleeger 2001). formally, estimation techniques such as the lines of code method are applied. in other companies they discuss a proposed list of requirements and prioritise them in a common effort of senior software developers and sales and support personnel. ajis vol 13, no. 1 september 2005 199 this leads to the conclusion that students should have at least a basic understanding of the feasibility study and its purpose. the model curricula of computer science and software engineering do not or only mention the feasibility study indirectly. is2002 (gorgone, davis et al. 2002) expects the students to know the basics. elicitation the interviewees reported several communication ways over which information about requirements is elicited. these are informal telephone conversations, formal telephone conversations such as customer hotlines or teleconferences, emails, web feedback forms, documents such as existing code, work-groups, jad-sessions, prototypes, or surveys. immanuel (se2) summarises that research skills are necessary for eliciting requirements. all model curricula include elicitation. cccs (engel and roberts 2001) and is2002 (gorgone, davis et al. 2002) expect student to be able to apply instruments for elicitation, whereas software engineering students should comprehend them. the lower level makes sense when arguing that in companies such as cs1, cs2, is1 or se2 the tendency exists that domain experts do some of the elicitation tasks. analysis and determination of requirements domain experts also mostly do the market-oriented evaluation. the interviewees were more involved in the technical evaluation and analysis of the requirements. according to the interviewees, the analysis can be seen as an interactive process in which employees with market competence and those with a technical understanding negotiate requirements. difficulties are mentioned, that sort of communication [with the sales and support people] is very, very difficult (karl, cs1). it is even stated to determine the requirements correctly is a matter of luck (karl, cs1). in other environments, technical people [are] talking to technical people (charlotte, is2). that means that they have to have knowledge about the technology, e.g. architectural issues, they apply or they have to use because of system constraints. two main points can be identified: • domain knowledge is important. • ability to analyse relevant information and communicate with people with a different (not computing) background and with a technical background. domain knowledge cannot be considered to be a demanded part of a computing curriculum. determining requirements is based on the analysis and communication between the involved stakeholders. analysis techniques such as modelling are considered by all curricula to be learned up to the application level. communications is stressed as an essential issue by all curricula. cccs (engel and roberts 2001) even proposes projects with other disciplines. documentation the documentation of the requirements gathering and analysis results differs in the degree ajis vol 13, no. 1 september 2005 200 of formalisation. at the informal end of documenting no formal document deliverables are required. at the formal end templates for the documents are given and the documents are formal sign-off points. an informal documentation is described, it gets up drawn on a white board and people take notes (simone, cs2) during meetings. on the other end of the scale, se1 or is1 have well defined documents (e.g., anne, is1). obviously, like in the case of is1 the degree of formality is higher because the activities of requirements and design/implementation are assigned to different teams or even different departments. the representation of documentation is again manifold. for all companies it depends on the audience they are primarily trying to reach with the documentation. sometimes they split it into a part for non-technical people and one for people with a computing background (cs1). in all kinds of documentation natural english plays a major role, we will go down to a literal description (charlotte, is2). this leads to the demand that students should have the ability to produce documents with a wide variety of representations, such as modelling or natural english. all three curricula expect students to achieve a knowledge level of application, which matches the requirements of the practice. verification documented requirements get tested against the real requirements. is1, is2, cs2 and se2 apply formalised reviews, walk-through or prototypes. besides the knowledge of the techniques for verification a demand for the ability to accept criticism can be derived. cccs (engel and roberts 2001) and ccse (díaz-herrera and hilburn 2003) included verification in their curricula matching the described practice. is2002 mentions verification only in the context of programming (gorgone, davis et al. 2002). only the curriculum ccse includes individual cognition (díaz-herrera and hilburn 2003). learning about individual cognition helps to recognise personal limits, such as limits of knowledge and skills. knowing personal limits is a prerequisite to accepting criticism and the development of personal skills and knowledge. verification activities can involve criticism and suggestion for the improvement of requirements. requirements management requirements may change during the course of a project, for example due to changes in organisations or due to legal changes, like one interviewee reports (sophie, is1). john underlines the difficulty, we keep getting requirements almost everyday… the requirement is the one that changes a lot (john, se2). these changes must be managed systematically throughout the development process (kotonya and sommerville 1997). this shows that the ability to handle not only large amounts of information but also changing information is needed. despite this need, the cccs (engel and roberts 2001) does not cover it. ccse (díaz-herrera and hilburn 2003) and is2002 (gorgone, davis et al. 2002) expect students to be only knowledgeable about requirements management. here, a mismatch between the practice and the curricula can be seen. ajis vol 13, no. 1 september 2005 201 generic skills table 2 includes some generic skills. the interviews revealed that two issues stand out in that list: communication skills and team skills. all interviewees regarded these two as highly important. other generic skills were mentioned relatively seldom in the interviews: if these topics were touched it was with minor importance. therefore, this section concentrates on communication and team skills. the requirements determination can be described that it’s then a back and forth sort of process (karl, cs1). this process involves the negotiation of requirements between, in that instance sales and support people and the software development team. that example can be seen as explanatory. it underlines the demand for negotiation skills, as part of communication skills. lethbridge (2000) already found that there is a big knowledge gap compared to the perceived importance. the need for communication skills in requirements activities is expressed as follows. the communications means to be able to talk to people extract stuff out, document it and understand it and agree to it. (anne, is1). all curricula refer to the need for effective communication skills more than once and emphasise it similarly as the interviewees perceive it. team skills are also mentioned by the interviewees and regarded also as generally important. people don’t get pigeonholed (charlotte, is2) in a strong team-oriented environment (arthur, cs1). again, the model curricula include team issues in their guidelines. students should learn the dynamics of working in teams (díaz-herrera and hilburn 2003, p.25) be able to work in teams through team projects ((engel and roberts 2001, p.236), (gorgone, davis et al. 2002, lu 80)). all curricula recommend that undergraduate students should participate at least in one team project. summary the above examined topics can be presented in a table. to simplify the representation a scale is applied to evaluate whether the findings match (+), partly match (o) or do not match (-) with the guidelines given in the curricula. ajis vol 13, no. 1 september 2005 202 table 2: topics match: how do curricula match the needs perceived by the interviewees? the model curricula have some insufficiencies as compared to the interviewees’ opinions. all three curricula do not fully match the topics of re process, feasibility study and requirements management. the is2002 furthermore lacks a sufficient coverage of requirements verification. the other topics can be regarded as sufficiently covered by the curricula. interviewees’ expectations the following section discusses what interviewees explicitly expect from graduates for requirements activities. in all companies it is, as one interviewee put it, very rare (albert, se1) that newly hired graduates are involved in requirements activities. the interviewees mention that almost exclusively more senior people (eva, is2) do requirements activities. some interviewees argue that experience is necessary. that confirms findings made by macaulay and mylopoulos (1995). one interviewee expects credibility and presence (thomas, se1) from somebody doing requirements activities. these characteristics are considered to be reserved to people more mature than most graduates are. although requirements engineering is no typical task for graduates the interviews revealed some issues that are of relevance for performing requirements tasks. these issues can be classed into four groups: • personality • interpersonal skills • technical skills • personal work organisation interviewees talked about certain personality characteristics that influence the performance of requirements activities positively. general personality qualities are that graduates should topics cccs is2002 ccse requirements engineering process o o feasibility study o elicitation + + + analysis + + + documentation + + + verification + + requirements management o o generic skills communication skills + + + team skills + + + ajis vol 13, no. 1 september 2005 203 be confident and faithful (simone, cs2). confidence can be underpinned by the knowledge and skills that are required in certain circumstances. to be faithful can be considered as rooted in upbringing. the model curricula include that in ethical concerns (engel and roberts 2001, p.64; gorgone, davis et al. 2002, lu12; díaz-herrera and hilburn 2003, p.16). the software engineering curriculum notes that confidence and a strong work ethic, also demanded by an interviewee (sophie, is1), can only be influenced subtly by university education. another general characteristic that is expected is to be proactive (john, se2) or self-started (simone, cs2). for requirements activities graduates should have an inquisitive nature (anne, is1). they should have the ability to ask people questions (simone, cs2) and accept to appear stupid (immanuel, se2). although the interviewee talks about nature, she thinks that techniques for questioning can be learnt (anne, is1). perseverance is also described to be of advantage (sophie, is1). these techniques are considered in general in the curricula, as noted above. finally, people should be teachable and willing to learn (sophie, is1). the curricula emphasise that need also with respect to the rapid changes in software development (engel and roberts 2001; díaz-herrera and hilburn 2003). interviewees as well as the literature, in particular literature about systems analysis (lee, trauth et al. 1995; marakas 2001; hoffer, george et al. 2002), regarded interpersonal skills as important. interpersonal skills are also considered to be only teachable to a certain degree. communication skills are considered as not teachable, by one interviewee, you’ve got ‘em or you haven’t (marie, cs2). other interviewees see the possibility to improve it (sophie, is1; rené, se2). to have the ability is considered to be up to the individual (sophie, is1). the interviewee also says that issues such as communication and team skills can be influenced best when people are young and amenable (sophie, is1). that leads to the conclusion that curricula have to consider these issues. technical skills that are of particular relevance for requirements are mentioned. architecture ‘cause quite often that comes into play in requirements (charlotte, is2). it must be remarked that architectural issues are a favourite of the interviewee, so a bias might be possible. furthermore, techniques such as facilitation of groups, estimation techniques and interviewing were mentioned. depending on the division of labour, background knowledge about the problem domain is needed to perform requirements tasks. (john, se2; sophie, is1). in the case of is1 they have a separation between business and technically oriented activities. in se2 the interviewee meant the knowledge about the technical background in which the piece of software that is to be developed will be integrated. in other cases such as se1 people, with a computing background tend to perform these tasks. they adopt the domain knowledge. personal work organisation is a more general issue that is not exclusively necessary for requirements activities but because of the usually high amount of information that must be handled it is considered here. the general expectations for graduates are in line with a variety of other job profiles. interviewees expect the combination of good communication skills paired with good team skills and a sound technical understanding. as special for the requirements activities it can be mentioned that the ability to handle large amounts of information is expected. ajis vol 13, no. 1 september 2005 204 implications for education as general learning and teaching advice interviewees point out their preference for more exposure to real life, exercises, team assignments (immanuel, se2) or industry projects. nguyen and swatman (2000) found that the requirements process as it is described by the literature and therefore taught at universities does not match reality. that can be confirmed by the presentation above. in a subsequent research nguyen et. al. (2002) postulate that curricula have to take an insight and creativity driven approach towards requirements into account. they demand an educational framework for requirements engineering based on the constructivist learning theory (hobmair 1994). that includes gaining experience in an authentic context (nguyen, armarego et al. 2002). that is confirmed by the demands stated by the interviewees. although the model curricula do not mention learning theories explicitly, they recommend unsupervised practice (gorgone, davis et al. 2002), a significant team project (engel and roberts 2001) and projects with a significant real world basis (díaz-herrera and hilburn 2003, p.42). as already mentioned briefly under the item of requirements process the demand for realworld basis does not reflect the knowledge given in textbooks. textbooks are usually the basis for tertiary education. furthermore, teaching with real-world basis does have implications on the education and background of teaching staff and the equipment of the faculties. a shift towards a real-world basis would also mean that other, maybe more underlying and scientific topics have to be shortened. with the current education, graduates are employed in the companies in a similar way. the interviewees state that graduates usually get tasks such as programming, code inspections or minutes of meetings (thomas, se1; immanuel, se2). conclusion and further research this paper has examined the relationship between the opinion of a small but representative group of practitioners and current model curricula on the topic of requirements engineering/systems analysis in the disciplines of computer science, information systems and software engineering. these three disciplines are considered to be the most visible of computing disciplines (glass, 1992). in order to describe that relationship, practitioners were interviewed personally and model curricula were examined. the results of these two activities (interviews and examination of curricula) have then been subject of a qualitative comparison. the comparison revealed a high degree of conformity between the recommendations of the selected model curricula and the expectations of interviewed practitioners. the conformity relates to the question of which topics students should learn and which level of knowledge they should achieve. these topics have been classed into a group of directly related topics to requirements activities and topics that cover generic skills such as communication or team skills. the interviewees consider requirements activities such as requirements elicitation, analysis and documentation as regarded appropriately by the curricula. only the topics of the requirements process, the feasibility study and requirements management can be seen as areas of neglect in the model curricula. a difference between the perceived importance by the practitioners as well as given importance by the curricula and the awareness of graduates seem to exist. for the cccs the explanation can be that only 2% of the recommended lessons are dedicated to requirements ajis vol 13, no. 1 september 2005 205 activities. however, the interviewees did not see a difference between graduates of different disciplines. the interviewees mentioned weaknesses in the areas of written as well as oral communication and team skills. the discussion of these skills revealed the question whether these skills are learnable and teachable. interviewees’ opinions on that question ranged from learnable and teachable to not teachable and not learnable. also, differences seem to exist between the objectives of the curricula and the final employment of graduates. the curricula suggest graduates to be equipped for performing requirements activities. however, new employees usually do not get assigned tasks related to requirements. the reason for that is mainly rooted in a mixture of experience and personality usually only more senior people have. although conformity between expectations and the recommended contents exist, graduates appear not to be equipped in an optimal manner to perform requirements activities. in order to find out how to improve the formal education of future employees several recommendations can be made. in general, the question has to be asked whether formal education is able to produce graduates that are prepared for requirements activities immediately after graduation or experience and on-the-job training is not substitutable. particular research endeavours can be suggested • recommendation 1 the above described results should be tested with a larger sample and a broader regional horizon. such research could also include the question of whether and to what extent differences exist between graduates of different disciplines • recommendation 2 how the model curricula are applied in actual curricula can be examined. these applied curricula can then be tested for their relevance and effectiveness in practice • recommendation 3 a third strand of research could examine the teaching and learning methods of the relevant topics. this could include whether certain topics are learnable and teachable in a formal setting. in order to provide an understanding of under which circumstances the research described was conducted and the results achieved weaknesses and problems must be mentioned. as the research was based on a qualitative approach credibility and interpretation of results can be seen as a problem. technical weaknesses such as the denied permission to tape the interviews or the selection of interviewees not directly involved in requirements activities did occur. furthermore, a strict questionnaire was not used and therefore absolutely comparable interviews were not conducted (which can be seen as part of the nature of the qualitative approach). despite these weaknesses the results can be used for further developments and improvements of model curricula in the examined disciplines as well as related areas where requirements activities are also of importance (nguyen and swatman 2000). improvements for the model curricula can be recommended for the topics of the requirements process, the feasibility study and requirements management. these topics are neglected and should be expanded in further curricula revisions. in particular, the cccs needs to put more emphasis on the importance of requirements activities in general. although all curricula regard communication and team skills as highly important it does not ajis vol 13, no. 1 september 2005 206 seem to be sufficient according to the interviewees’ statements. an improved education in the field of requirements activities could lead to improved development processes and hence, to software systems that do what their users want them to do. references alexander, i. and r. stevens (2002). writing better requirements. harlow, london, new york, addison-wesley. al-karaghouli, w., s. alshawi, et al. (1999). enhancing the understanding of customer requirements in business software engineering, university of westminster, brunel university. al-rawas, a. and s. easterbrook (1996). communication problems in requirements engineering: a field study. first westminster conference on professional awareness in software engineering, london, royal society. anderson, l. (1997). argyris and schon's theory on congruence and learning. 2004. argyris, c. and d. schön (1974). theory in practice: increasing professional effectiveness. san francisco, jossey bass. armarego, j. (2003). requirements engineering education relevance survey (2003). 2003. avison, d. e., g. fitzgerald, et al. (2001). "reflections on information systems paractice, edcation and research: 10 years of the information systems journal." information systems journal(11): 3-22. bray, i. k. (2003). an introduction to requirements engineering. harlow, london, new york, addison-wesley. carneson, j., g. delpierre, et al. (2000). designing and managing multiple choice questions. cape town south africa, university of cape town. carroll, j. m. and p. a. swatman (1999). managing the re process: lessons from commercial practice. cie (2001). breaking the skills barrier: demonstrating the benefits of investment in ict higher education in australia. canberra, centre for international economics. conn, r. (2002). "developing software engineers at the c-130j software factory." ieee software(september/october 02). davis, a. m. (1993). software requirements: objects, functions and states. new jersey, prentice hall. díaz-herrera, j. l. and t. b. hilburn, eds. (2003). computing curriculum software engineering public draft 1, the joint task force on computing curricula; ieee computer society, association for computing machinery. engel, g. and e. roberts, eds. (2001). computing curricula 2001 computer science, acm, ieee-cs. esi (1996). espiti project: european user survey analysis,, european software institute. ferdinandi, p. l. (2002). a requirements pattern: succeeding in the internet economy. harlow, london, new york, addison-wesley. gorgone, j. t., g. b. davis, et al. (2002a). appendix 5 body of information system knowledge. is 2002: model curriculum for undergraduate degree programs in information systems. j. t. gorgone, g. b. davis, j. s. valacichet al. park ridge (il), acm. gorgone, j. t., g. b. davis, et al. (2002). model curriculum and guidelines for ajis vol 13, no. 1 september 2005 207 undergraduate degree programs in information systems, association for computing machinery (acm), association for information systems (ais), association of information technology professionals (aitp). hobmair, h. (1994). pädagogik. köln, bildungsverlag eins. hoffer, j. a., j. f. george, et al. (2002). modern systems analysis and design. new jersey, prentice hall. hull, m. e. c., k. jackson, et al. (2002). requirements engineering. london, berlin, heidelberg, springer. karlsson, l., a. g. dahlstedt, et al. (2002). challenges in market-driven requirements engineering an industrial interview study. eighth international workshop on requirements engineering, essen, foundation for software quality. kearsley, g. (2000). explorations in learning & instruction: the theory in practice database. washington (dc), george washington university. kendall, k. e. k., julie e. (2002). systems analysis and design. new jersey, prentice hall. king, n. (1998). template analysis. qualitative methods and analysis in organizational research. c. cassell. london, thounds oaks, new delhi, sage publications: 118-134. kotonya, g. and i. sommerville (1997). requirements engineering: processes and techniques. weinheim, brisbane et. al., john wiley & sons. lacey, a. and d. luff (2001). qualitative data analysis, trent focus group. 2003. leach, r. j. (1999). introduction to software engineering, crc press llc. lee, d. m. s., e. m. trauth, et al. (1995). "critical skills and knowledge requirements of is professionals: a joint academic / industry investigation." mis quarterly 19(3): 313-340. lee, m., s. dutta, et al. (1999). an empirical analysis of software production problems in european software units. fontainebleau, france, insead. leedy, p. d. and j. e. ormrod (2001). practical research: planning and design. new jersey, prentice hall. lethbridge, t. c. (1998). a survey of the relevance of computer science and software engineering education. csee&t'98, atlanta, ieee. lethbridge, t. c. (1998). what knowledge is important to a software professional? a summary of the most important results. 2003: 44-50. lethbridge, t. c. (2000). "what knowledge is important to a software professional?" computer 33(5): 44-50. loucopoulos, p. and v. karakostas (1995). system requirements engineering. london, mcgraw-hill. macaulay, l. and j. mylopolous (1995). requirements engineering: an educational dilemma. 2003. macaulay, l. a. (1996). requirements engineering. berlin, heidelberg, new york, springer-verlag. marakas, g. m. (2001). systems analysis and design. new jersey, prentice hall. miles, m. b. and a. m. huberman (1994). an expanded sourcebook qualitative data analysis. thousand oaks, london, new delhi, sage publications. morris, p., m. masera, et al. (1998). requirements engineering and industrial uptake. third ieee international conference on requirements engineering, colorado springs, colorado, ieee computer society. nguyen, l., j. armarego, et al. (2002). understanding requirements engineering: a ajis vol 13, no. 1 september 2005 208 challange for practice and education. school of management information systems. melbourne, deakin university. nguyen, l. and p. swatman (2000). managing the requirements engineering process. 2003. nickerson, r. c. (2001). business and information systems. new jersey, prentice hall. nikula, u., j. sjaniemi, et al. (2000). a state-of-the-practice survey on requirements engineering in smalland medium-sized enterprises. lappeenranta, telecom business research center lappeenranta. noll, c. l. and m. wilikens (2002). "critical skills of is professionals: a model for curriculum development." journal of information technology education 1(3): 143154. patton, m. q. (1990). qualitative evaluation and research methods. newbury park, london, new delhi, sage publications. pfleeger, s. l. (2001). software engineering: theory and practice. new jersey, prentice hall. richie, j. and l. spencer (1994). qualitative data analysis for applied policy research. analysing qualitative data. burgess. london, routledge: 173-194. robertson, j. and s. robertson (1996). vollständige systemanalyse. münchen, wien, carl hanser verlag. sly, l. and l. j. rennie (1999). computer managed learning: its use in formative as well as summative assessment. third annual computer assisted assessment conference, loughborough, loughborough university. standish (1994). the chaos report. west yarmouth (mass), standish group international, inc. turner, r. and g. lowry (1999). educating information systems professionals: towards a rapprochement between new graduates and employers. 10th australasian conference on information systems, acis. van lamsweerde, a. (2000). "requirements engineering in the year 00: a research perspective." acm. young, r. r. (2002). effective requirements practices. boston et al., addison-wesley. zwoghi, d., d. damian, et al. (2001). field studies of requirements engineering in a multi-site software development organization: research in progress. 6th australian workshop on requirements engineering (awre'01), sydney. ajis vol. 9, no. 2 may 2002 mobile data technology for small business: needs, uses and adoption dr debra harker & jeanette van akkeren university of the sunshine coast, faculty of business, locked bag 4, maroochydore south, queensland 4558, australia abstract the technological environment in which contemporary smalland medium-sized enterprises (smes) operate can only be described as dynamic. the exponential rate of technological change, characterised by perceived increases in the benefits associated with various technologies, shortening product life cycles and changing standards, provides for the sme a complex and challenging operational context. the primary aim of this research was to identify the needs of smes in regional areas for mobile data technologies (mdt). in this study a distinction was drawn between those respondents who were full-adopters of technology, those who were partial-adopters, and those who were non-adopters and these three segments articulated different needs and requirements for mdt. overall, the needs of regional smes for mdt can be conceptualised into three areas where the technology will assist business practices; communication, ecommerce and security. keywords: mobile data technologies, ecommerce, sme, adoption, internet introduction the technological environment in which present australian smes operate can best be described as dynamic and vital, however businesses have made an important psychological jump in viewing the internet as a marketing tool rather than a technological issue (dearne 2001). the explosive rate of technological change, characterised by perceived increases in the benefits associated with various technologies, shortening product life cycles and changing standards, provides for the sme a complex and challenging operational context. the development of infrastructures capable of supporting the wireless application protocol (wap) and associated technologies represents the latest generation of technological innovation with potential appeal to smes and end-users alike. for all stakeholders, the primary appeal of these mobile data technologies, apart from mobility, is that associated 'services' are delivered on existing devices with which users are familiar mobile phones, palm-tops, or other personal digital assistants (pdas). however, whilst much attention has focused on high volume end-user 'lifestyle' applications for mobile data services (mobile data conference, 1999), less emphasis has been placed on applications and services that would meet the needs of smes. there are two objectives for this article. first, to discuss the literature in the area of adoption of mobile data innovation and, second, to report on the findings of an empirical study with smes in a regional setting. the primary aim of this research is to identify the needs of smes in regional areas for mobile data technologies mdt. the research question addressed being, 'what are the needs of regional smes in relation to mdt, and do these needs differ depending on the level of it adoption already in place?' literature review adoption of information technology early studies addressing the adoption of information technology (it) provide insight into reasons why decisionmakers adopt or do not adopt innovations. the diffusion of innovation theory (rogers 1995) suggests that characteristics of innovations help to persuade potential adopters to embrace or reject an innovation. other researchers have since completed work resulting in modifications to rogers' original theory that provide numerous models addressing it adoption and usage within an organisation. the technology acceptance model suggested by davis (1989) addresses it adoption, implementation and diffusion in terms of perceived ease of use and perceived usefulness based on behavioural intentions. belief about the system, perceived usefulness and perceived ease of use are seen as directly affecting attitude to use (gefen and straub 1997; agarwal and prasad 1997). further studies suggest that behaviour is a direct function of behavioural intention and perceived behavioural control that will impact on decision-makers choosing whether to adopt an innovation (ajzen 1991, taylor and todd 1995). therefore, in the adoption of mobile data technologies, will the same behavioural/psychological factors impact on the decision by owner/managers to adopt mdt? constructs used in the above models are generally based around perceptions, beliefs, attitudes and intentions of the decision-maker. more recent studies have identified other factors in addition to these that will impact on an owner/manager's decision to adopt new innovations such as mobile phones and internet technologies. these 17 ajis vol. 9, no. 2 may 2002 include economic factors such as return on investment and characteristics of the firm such as the size, sector and status, and the structural sophistication of the firm. adoption of nvecommerce by smes empirical studies have identified a variety of factors thought to affect e-commerce/internet technology adoption in small business (brooksbank, kirby, kane 1992, kirby and turner 1993, mien and raymond 1994, lacovou, benbasat, dexter 1995, thong and yap 1995, harrison, mykytyn, and rienenschneider 1997). from the adoption factors identified in earlier studies, van akkeren and cavaye (1999) developed a model (figure 1) based on a study on the adoption of e-commerce technologies thought to facilitate or inhibit technology adoption by sme owner/managers. figure 1: framework of sme adoption of innovations f . . ••.perceiyed..benefits-; .7 •': ..•'-•..; >j • ' ' ''' r-»-return,on investme• • • • • " i it adoption in small | f 7 " business ' • i (source: van akkeren and cavaye 1999) the first component of this model can be described as owner/manager characteristics and these are based on behavioural and attitudinal factors, being: • perceived benefits affect technology adoption in terms of the perceived ease of use and/or usefulness of the technology. if the owner/manager does not perceive the technology in a positive way, they will be reluctant to adopt (lacovau et al. 1995, kirby and turner 1993, thong and yap 1995). • the computer literacy of the business owner can also influence technology adoption. if the owners are unaware or do not understand the technologies available, they are unlikely to adopt them into their own business (kirby and turner 1993, thong and yap 1995). • the level ofassertiveness, rationality and interaction of business decision processes can also impact on it adoption. if owners of the firm are assertive in business decision processes, understand the benefits and uses of the technology to their organisation, and are able to rationalise that information, they will be more likely to adopt it (julien and raymond 1994, harrison et al. 1997). • perceived control relates to the amount of requisite opportunities and resources (time, money, skills, co-operation of others) someone possesses to be able to carry out the course of action (technology adoption). for example, a small business owner may decide that connection to the internet is an important competitive use of it. yet if there is a possible budget shortfall, or lack of time or understanding of the technologies, their decision to adopt will be influenced (harrison et al. 1997). 18 ajis vol. 9, no. 2 may 2002 • subjective norm affects technology adoption in terms of the strength of the person's normative beliefs that 'groups' think the behaviour of interest (technology adoption) should or should not be performed, multiplied by a person's motivation to comply with the group (harrison et al. 1997). • mistrust of the it industry relates to the attitudes of owner/managers that it professionals are untrustworthy, difficult to understand and overcharge (van akkeren and cavaye 1999). • lack of time refers to the owner/managers' attitude that they do not have sufficient time to invest in gaining knowledge and purchasing and implementing new technologies (van akkeren and cavaye 1999). the second factor in this model is based on economic considerations, or return on investment (roi). the need by small business owners for an immediate return on investment is due to the necessity of being concerned with medium-term survival rather than the long-term attainment of market share. when making a substantial outlay of capital resources the sme owner needs to see an immediate short-term roi (fichman and kemerer 1993). the final category of factors is based on firm characteristics: • organisational readiness refers to the level of technology currently incorporated into business processes. if there is little technology incorporated, or outdated/inefficient technology being utilised, a firm is less prepared to incorporate the technologies and therefore less likely to adopt (lacovou et al. 1995). • a small business owner/manager will be reluctant to adopt innovative it unless there is a specific request for it by their trading partners and/or customers. if this external pressure to adopt it is not present in the industry sector, then the business owner may perceive the technology as a waste of resources (thong and yap 1995, lacovau et al 1995). • the dependency of the small business customer on the supplier is linked to the previous factor. not only would the supplier need to have adopted the technology to make it viable, the small business owner would need to recognise and understand the benefits to his or her firm in adopting the technology. in addition, an organisation may perceive that their clientele was of a certain socio-economic level that would not readily benefit from the introduction of new technologies (kirby and turner 1993). • the structural sophistication of the firm in terms of centralisation and complexity will also influence technology adoption in its ability to incorporate new technologies into its work practices. a particularly complex structure could either inhibit or facilitate technology adoption and would be dependent on whether the owner believed that it could be easily incorporated and enhance operations, or excessively disrupt operations (julien and raymond 1994, harrison et al. 1997). • the size, sector and status of the organisation have been shown to influence technology adoption, particularly in relation to the sector and status. if competitors and trading partners within the sector have adopted it, an owner may be more inclined to adopt as well. the size of the business can also influence technology adoption, as a very small business with only two or three employees may not have the time or expertise to devote to implementing and using new technologies (thong and yap 1995, julien and raymond 1994, harrison etal. 1997). • finally, the level of information intensity within the organisation may influence the owner to adopt or not adopt a technology. for example, if large amounts of data and information are part of the business processes, an owner may be more likely to adopt technologies that could streamline operations and lead to improved information process within the organisation (thong and yap 1995). recent studies about why sme owner/managers adopt or do not adopt it and e-commerce technologies have highlighted both inhibitors and facilitators to adoption and are similar in content to the factors described above. thong (1999) discusses sme adoption as being determined by decision-maker characteristics, information system (is) characteristics, organisational characteristics and environmental characteristics. further, the need for is to offer better alternatives to existing practices are critical to adoption by smes (thong 1999). therefore, could the use of mobile data technologies provide the 'better service' that sme owner/managers seek? owner/manager characteristics, planning orientation, and the existence of alliances/networks influence technology adoption by smes (mcgregor and gomes 1999). specifically in terms of alliances/networks, previous literature also suggests that the decision to adopt or not adopt it will be influenced by the network within which the small business operates, particularly if time, cost and operational efficiencies can be realised (piovesana and raush 1998, lawrence 1998, sillence, macdonald, lefang, frost 1998). in contrast, bridge and peel (1999) in their study of smes in the uk, suggested that adoption of it would be improved through education of the benefits of the technologies and by fostering a planning orientation to is design and implementation. in an australian study on it adoption by smes, fink (1998) identified adoption decisions as having three distinct phases: assessing it benefits, organisational culture and firm-suited it; assessing internal resources and procedures; and evaluating external environment, support and services. in terms of mobile data technologies, 19 ajis vol. 9, no. 2 may 2002 one could argue that owner/mangers would need to assess what, if any, benefits can be derived by adopting the technologies and whether their firm was suited to adoption. part of that evaluation would necessarily need to include both the internal and external environment in which the firm operates. previous studies on sme adoption of it/e-commerce highlight a range of factors thought to influence an owner/managers' decision to adopt new technologies. the marketing of mobile data technologies to sme owner/managers will be competitive and intense. the marketing of innovations the marketing of innovations to smes has undergone considerable analysis and debate, particularly in relation to new technologies. mahajan and muller (1998) propose that with 'high-tech' products, the targeting of innovations does not necessarily lead to market success. further, they suggest that traditional 'laggards' are dropped from marketing strategies and 'innovators' and 'early adopters' be grouped together and targeted. in this way, the early and late majority can be grouped as the mainstream market and strategies developed separately for this group. major innovations may have to 'prove themselves' in new markets before they can displace other technologies (friar and balachandra 1999). it is the early adopters or innovators who will initially experiment with these technologies and hence marketing of new technologies to this group is a useful first step. in addition, the usefulness and ease-of-use will impact on owner/manager acceptance of the technologies (agarwal and prasad 1997). thus, focusing on these attributes may improve overall acceptance by sme owner/managers, a view supported by nambisan and wang (1999) who posit that the acceptance of web-based technologies is influenced by ease of use and perceived usefulness in terms of current is sophistication, complexity of the new technologies, and perceived costs and benefits. they add that strategies needed to market web-based technologies should address the needs of the user and identify for them the context to which the technology can be used for business purposes. previous studies suggest that the marketing of technological innovations to sme owner/managers is a minefield of emotion, attitudes, behavioural intention and perceptions, coupled with other factors considered important to owner/managers such as cost and technical complexity. further, issues such as computer literacy of the owner/manager, and the size, sector and status of the firm are also considered important. it would appear, therefore, that marketers of mdt have many barriers to overcome. mobile data technologies and smes in australia, the adoption of internet/e-commerce technologies varies in different states, and further, between regional and city-based firms. however, the adoption of mobile phones is consistently high across states and regions within australia. mdt, which marries mobile phones and e-commerce technologies, is seen as eliminating time and distance as barriers for regional businesses in their adoption of these technologies. a search of the literature has provided limited results on mdt adoption and/or marketing strategies, however, this is not surprising given the 'newness' of the technologies. the mass market for mobile data technologies is so far largely untapped (axby 1998, harrison 1999). estimates put usage of mobile devices as one billion worldwide by 2003 (greengard 2000), or us$1.3 trillion in relation to annual turnovers by 2003 in the us alone (thurston 2000). with such potential markets available to the vendors of mobile data services and devices, it is useful to understand reasons why the majority of potential end-users are so far resisting these new technologies. lack of speed is a barrier to adoption as mdt is slow and hence inefficient (taylor 1999, saunders, heywood, dornon, bruno, and allen 1999). another barrier is the lack of standardised it environment for developing mobile data applications and this impedes the growth of the mobile data market (harrison 1999, axby 1998). limited bandwidth, higher usage costs, increased latency, and a susceptibility to transmission noise and call dropouts are also possible barriers to adoption (duffy 1999). in addition, problems with mobile data devices such as limited memory and cpu size; small, monochrome screens; low bandwidth; and erratic connections are further adoption barriers (johnson 1999). it is possible therefore that adopters are 'sitting back' and waiting for at least some of these problems to be corrected before entering the mobile data market. another area of concern for end-users is that the wireless application protocol (wap), the emerging technology used to send data to and from handheld devices, has no in-built security mechanisms and this is of concern not only to the business user, but to the customers of the business as well (riggs and bachelor 1999, chan 2000). the marketing of mobile devices to potential customers will provide many challenges to the various players in the mobile data industry. luring potential users away from their desktop pcs will require innovative marketing strategies and the literature provides many different approaches to capture potential customers. wexler (1999) suggests three factors for the successful proliferation of mobile data services: coverage, ease of use and cost. in terms of segmentation, axby (1999) suggests that there are two distinct market segments for these products: 20 ajis vol. 9, no. 2 may 2002 white collar groups and blue collar groups, distinguishing these groups via their application needs. for example, access to e-mail, database and file transfer applications will vary between the two segments, therefore packaging the appropriate applications to the particular segment would be a useful strategy. it may also be argued that products would need to be developed for both business and the mass consumer markets. capturing users requires 'transparency', that is, users want information or communication access whenever and wherever they need it, using whatever device is most convenient at that moment (osowski 1999). consumers do not buy technology; they buy benefits (duffy 1999) and mdt benefits include easy communication through e-mail, ready access to information, entertainment, and improved lifestyle through ecommerce and home banking. a 'customer-centric' focus that personalises the device, providing simplicity, intimacy, transparency and immediacy, provides a complete e-business solution (horn 2000). clearly, the literature on marketing mobile data devices to date underlines the importance of highlighting the benefits of using the technologies, and the ease of use to potential customers. most literature on the adoption and marketing of mdt is not empirically based and is limited to discussing the technologies in terms of their application to business, rather than adoption barriers or marketing strategies. findings from this research provide a deeper understanding of the facilitators and barriers to the adoption of mdt by sme owner/managers and focuses on the needs of the businesses. research design data collection potential respondents were drawn from the yellow pages online with different business types grouped into their industry sector according to the australian and new zealand standard industry classification (anzsic) system. the database comprised approximately 5500 potential respondents in total and a sample of 500 respondents were interviewed. interviewers worked through their respective database in a random manner to contact potential respondents and a policy of three callbacks before disregarding the potential respondent was employed. response rates following completion of the 500 telephone interviews, and in conjunction with the data entry process, questionnaires were screened to gauge their usability. of the 500 survey response sheets submitted by interviewers as 'completed', 18 were deemed 'not usable' due to substantial insufficient collection of data on certain variables2, or due to the respondent falling outside certain sampling parameters3. the final sample size was thus 482, derived from 1251 telephone calls, indicating an overall response rate of 39% which is above the accepted norm. data was coded and entered into an spss data file. data analysis setting the p level at 0.05, as was done in this study, succeeded in filtering out weak correlations, thus we can be 95% confident that the results are actually true. unless otherwise stated the p-level here is significant at 0.05 or less, indicating that there is a 5% probability that the relation between the variables found in the results is a chance occurrence. this research was concerned with understanding the needs of smes for mdt. however, the literature suggests that it is foolish to treat all smes the same in respect of it and, thus, a measure of it adoption was used to ascertain whether a full-adopter of technology, for example, had the same needs for mdt as did a partialor non-adopter. thus, the rate of it adoption is crucial to business longevity in the 21st century; indeed, 62% of smes are in the process of becoming online businesses, with a further 29% recognising the need to do so (dearne 2001). further, more than nine in ten (95%) medium sized businesses are now connected to the internet (dearne 2001). with this in mind, analysis of variance (anova) was conducted to identify differences 2 insufficient collection of data on certain variables resulted from impromptu termination of the interview by the respondent. 'substantial' insufficient data was defined as a situation whereby the respondent completed less than 50% of the questionnaire, not including demographic data (if the demographic data was in fact completed). 3 'falling outside sampling parameters' refers to responses that were obtained (unknowingly on the interviewer's behalf) from outside the geographical sampling area, or from businesses with more than 200 employees (i.e. not an sme). 21 ajis vol.9, no. 2 may 2002 between more than two groups (for example, adoption levels). the anova tests were run with a tukey's post hoc evaluation'4 to determine significant differences between groups at the 5% significance level. adoption status table 1 displays adoption status characteristics of the sample. in brief, non-adopters were defined as those respondents who did not use computers for business purposes (i.e. in the day-to-day running of a commercial enterprise). partial-adopters were those respondents who used computers for business purposes, however did not use an internet connection for business purposes. full-adopters were defined as those respondents who used a computer and an internet connection for business purposes. table 1: adoption status sample characteristics ;'•?.'adoption status , non-adopters partial-adopters full-adopters total . n o . . 70 137 275 482 % of total sample <: 15% 28% 57% 100% findings 1. technology adoption by adopter level few studies concerned with the adoption of technology by smes have focused on past or current behaviour as an indicator of future intent. this study specifically categorised sme owner/managers based on current usage of it and then focused on the needs of these three categories in relation to the newest technology, mdt. given the obvious link between the internet and mobile phones, in terms of mdt, these two areas were specifically addressed in the survey. the internet more than half of the sample (57%) had an internet connection used for business purposes. these 'fulladopters' were asked their main reason for being connected to the internet as well as their secondary or other reasons (if any) (see table 2). the question used to elicit this information for all respondents was open-ended and responses were analysed and subsequently coded. table 2: internet usage by full-adopters reasons sme connected to net reason connected/use research or information search communicate/email advertise products and services download info, or software order products & services take orders (sales) network with other businesses monitor competition use directories like yellow pages banking/pay bills pay for products & services other don't know/can't say total no^ times •named as . 'main :.' -;-v 'reason'. ••; 90 83 31 18 16 11 8 6 5 4 1 1 1 275 .% of 'main reasons' "• . . 1 . • • . 33% 30% 11% 7% 6% 4% 3% 2% 2% 2% 100% no. times named as 'secondary •,,. reason' 47 52 15 25 12 8 6 10 4 7 1 1 188* %of 'secondary reasons' 25% 28% 8% 13% 6% 4% 3% 5% 2% 4% 100% "not all 'full-adopters' named a secondary reason for connecting to the net. 4 these tests and post hoc evaluation methods were administered as recommended by coakes & steed (1999) and sekaran (1992). 22 ajis vol. 9, no. 2 may 2002 the most significant reasons for smes being connected to the internet were to research, or information searching and communicating/email. these reasons rated highest as both the main and secondary reasons for connecting. far fewer smes reported conducting transactions such as 'ordering products and services', 'paying for products and services' and 'banking/paying bills'. while this does not mean that such activities were not undertaken, the data merely suggests that these are not (currently) the main reasons smes adopt internet technologies. table 3 outlines the reasons given to the same question by partialand non-adopters of technology. it should be noted that no distinction was made between the reasons for not using a computer (nons) and the reasons for not connecting to the net (partials). rather, the reasons listed refer to general reasons for 'not connecting'. table 3: reasons not connected (nons and partials) 11 •.*"' r ^ vk * * « * * * f t ""f <. ,\ reason not connected '• ' * ' * . ' " t s no benefit to business will connect/looking into it not interested don't know enough can't afford it wrong equipment no time don't know/can't say security worries total . no. times . named as 'main * reason? 67 52 28 24 12 10 7 4 3 207 %of*main reasons' 32% 25% 14% 12% 6% 5% 3% 2% 1% 100% no. times t , named as 'other reason' . 16 4 8 15 7 1 4 1 56* ." . %of'otheir reasons' ' 29% 7% 14% 27% 13% 2% 7% 2% 100% *not all nonor partial-adopters named an 'other' reason for not connecting. of the 207 respondents classed as either nonor partial-adopters, approximately one third (32%) saw no reason to connect, either to a computer or the internet, for business purposes, as they could see no benefit to their business. however, 25% of nonor partial-adopters conceded that they will connect in future or are 'looking into it'. 'not knowing enough' also ranked as an important primary and secondary reason for not connecting. mobile phones the majority of respondents (82%) used a mobile phone for business purposes. however, of those who did not use a mobile phone for business purposes, only a small proportion were 'non-adopters'. in other words, some smes used computers and the internet but did not use mobile phones. only 17 respondents, or 4% of the total sample, used neither a mobile phone nor a computer for business purposes. nearly half (47%) of the 86 respondents who did not use a mobile phone for business purposes were full-adopters and thus had an internet connection. when asked why a mobile phone was not used, the majority (81%) indicated they did not see a need. in terms of mobile phone use, most smes indicated that the majority of calls made or received on mobile phones were local. one hundred respondents (25% of those that use a mobile phone) indicated that 90-100% of the calls that they made or received were local. most respondents also indicated that the phone was used less than five times per day although over 50 smes (nearly 14% of those that use mobile phones) indicated usage levels exceeding 20 calls per day. 2. reaction to mdt by it adopter level the key issue in this research was to test how the different adopter levels compared to each other in their attitudes, needs and approach to technology as it is suggested that the more it-literate people are, the more aware they will be of their current and future new technology needs. table 4 displays the results of the anova analysis, showing the key differences between the three adopter groups in terms of mdt needs. 23 ajis vol.9, no. 2 may 2002 table 4 anova differences between adopter levels ' ' ! • ' • > • • . • ' yy* :-a;:y , ''"••••-• y ' : ' ' . ' , ' ' • • ••'^•.-•'-'^ / •• ' • : . :k statement ' • " • " . • • " " ' : • . " . ' ; . •. wait before investing (n=474) being able to email (n=460) owner/manager feels excited (n=471) trade with customers (n=456) being able to navigate (n=437) being able to monitor or operate equipment (n=447) live 2-way video (n=445) staff would feel threatened (n=452) overall interest in acquiring (n=474) fulladopter 2.09 2.3 2.54 2.61 2.73 2.83 2.92 3.79 2.49 partialadopter 1.74 2.66 2.87 3.02 3.11 * 3.23 3.54 * nonadopter * 2.9 3.04 * 3.13 3.42 * 3.05 2.93 ' only statistically significant results are shown. the anova results showed that there were clear differences in the attitudes and needs of respondents, depending on their different level of adoption of technology. full-adopters were more excited about the prospect of this technology compared to the partials and nons. this state of mind is carried through the data as, in terms of waiting before investing in the technology, the full-adopters indicated that they would not wait as long as the partial-adopters. similarly, in the business environment, there are key differences between the three groups. the non-adopters, for example, are significantly different to the other two groups in relation to staff reaction. it would appear that the less familiar the owner/manager is with current technology, the higher their perception is that their staff would likewise be uncomfortable with new technology. clearly, the acceptance of technology will influence attitudes and approaches to it; it will also influence perceived needs of different market segments. a key bank of questions in the questionnaire focused respondents on the use of mdt in a business setting and asked them to state to what extent they agreed or disagreed5 with a set of statements. to complement this bank of questions, an important, overarching question was also posed but placed later in the survey to alleviate any bias. this later question gauged the level of interest6 in acquiring the mdt, if it was available and affordable tomorrow. thus, by focusing on those respondents who answered either 'strongly agree' or 'agree' with each statement in the 'key benefits' bank, it can be shown which features of mdt are important to each adopter level. full-adopters full-adopters were most comfortable with technology compared to the rest of the sample. table 5 displays the rankings of key benefits for the full-adopter group, together with the 'overall interest in acquiring' response. table 5 full-adopters who 'strongly agree' or 'agree' with statements rank 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 " •. '••%:: '<': ^ ' ' . ' " statement ' •' " ""'• ' ' • • . ";" wait before investing being able to email owner/manager feels excited trade with customers banking and other admin. trade with suppliers monitoring business premises being able to navigate being able to access the net being able to monitor or operate equipment competitive pressure to adopt live 2-way video no need for this technology being able to shop staff would feel threatened overall interest in acquiring full-adopter % (n=275) 86 79 64 62 59 59 59 56 53 53 52 49 35 31 15 60 5 using a likert-type scale where l=strongly agree to 5=strongly disagree. 6 using a likert-type scale where l=very interested to 5=very uninterested. 24 ajis vol. 9, no. 2 may 2002 whilst full-adopters agree that they will wait a while before investing in the mobile data technology (86%), the owner/manager was excited by the prospect of it (64%). in terms of addressing the needs of the full-adopter sme owner/manager, the most important benefits were found to be using mdt for mobile email (79%), ecommerce via trading with customers (62%) and suppliers (59%), and to bank at their own convenience (59%) or monitor their premises (59%). six in ten (60%) full-adopters were very interested or interested in acquiring this technology. partial-adopters table 6 displays the rankings of key benefits for the partial-adopter group. table 6 partial-adopters who 'strongly agree' or 'agree' with statements rank 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 1 "~'f". »** ' statement' wait before investing being able to email banking and other admin. l monitoring business premises trade with suppliers owner/manager feels excited trade with customers competitive pressure to adopt no need for this technology being able to monitor or operate equipment being able to navigate being able to shop being able to access the net live 2-way video staff would feel threatened overall interest in acquiring partial-adopter % <n=*137) 93 64 60 56 53 48 46 46 46 44 42 42 40 38 26 59 nine out of ten (93%) partial-adopters agreed, or strongly agreed, that they would wait a while before investing in this technology. however, when focused on how the technology could help them in their business, partialadopters mainly felt that being able to deal with email (64%), banking (60%), and monitoring their business premises (56%), all in a remote fashion, would be advantageous. six in ten (59%) partial-adopters were interested or very interested in acquiring this technology. non-adopters non-adopters have little knowledge of it hi general, and use it least compared to the other two groups. table 7 displays the rankings of key benefits for the non-adopter group. 25 ajis vol. 9, no. 2 may 2002 table 7 non-adopters who 'strongly agree' or 'agree' with statements rank 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 . . " statement ; :. : wait before investing trade with suppliers banking and other admin. trade with customers being able to email being able to access the net live 2-way video being able to navigate no need for this technology monitoring business premises staff would feel threatened competitive pressure to adopt owner/manager feels excited being able to monitor or operate equipment being able to shop overall interest in acquiring non-adopter %(n=70) 87 62 52 51 51 47 45 45 44 44 43 43 42 30 29 44 whilst non-adopters indicated that they, like the other two groups, would wait a while before investing in mdt (87%), they did not feel as strongly on this as the partial-adopters of technology (93%). the features of mdt that the non-adopters valued most were e-commerce via trading with suppliers (62%) and customers (51%), remote banking (52%), and email (51%). in a marked difference to both the full-(60%) and partial-adopters (59%) of technology, less than half (44%) of the non-adopters were 'very interested' or 'interested' in acquiring this technology. 3. industry differences by adopter levels this research project was concerned with understanding the differences between different adopter status groups in relation to mdt needs; including the industry sector variable into this discussion at this point strengthens the analysis. by displaying a cross-tabulation of these two aspects of the data, greater insight is provided in terms of the profile of industry sector by adopter status (table 8). table 8 industry sector by adopter status (%) •' • -•••. . • ' ;• ' , ' -.-• .' \. • „. -"• • • •••: '•.'•• . '• '. • ' : •'. '•' ' -v '•.'•'•' ,";'' :• '"•'• '•. ' ' <; > ; ' industry sector "'*, ' ' property and business services (n=71) health and community services (n=45) agriculture, forestry and fishing (n=15) manufacturing (n=35) transport and storage (n=18) wholesale trade (n=28) retail trade (n=106) construction (n=79) accommodation, cafes and restaurants (n=35) personal and other services (n=38) communication services (n=4) total (n=482) fullad. 85 73 60 60 56 55 52 47 45 32 25 57 ; 'partialad. 15 25 13 17 33 28 27 36 33 55 50 28 nonad. 0 2 27 23 11 17 21 17 22 13 25 15 total industry ' 100 100 100 100 100 100 100 100 100 100 100 100 the property and business services (85%) sector has the highest proportion of full-adopters of technology, followed by the health and community services (73%) sector. industries with the lowest proportions of fulladopters include communication services (25%), personal and other services (32%) and accommodation, cafes and restaurants (45%). 26 ajis vol. 9, no. 2 may 2002 summary this article set two objectives. first, to discuss the literature in the area of adoption of mobile data innovation and, second, to report on the findings of an empirical study with smes in a regional setting. both of these objectives have been achieved and the empirical findings have assisted the primary aim of the research, which was to identify the needs of smes in regional areas for mdt. many factors impact on the adoption of mdt by companies, and many different applications will specifically encourage smes in regional areas to adopt. however, much of the literature to date has been of a speculative nature, with little empirical research to identify the variables that play a role in the adoption and marketing of these technologies, or in identifying the most appropriate applications. in this study, two key variables related to the firms were considered for analysis; the current it adoption level and industry sector. the needs of regional smes for mdt applications can, however, be classified into three areas; communication, e-commerce, and security. overall, smes on the sunshine coast would wait before investing in this technology. however this is not surprising, given that australian smes have generally been slower to adopt e-commerce technologies, compared with other developed countries such as japan, the us and singapore (forrester research 1997, lawrence et al 1998, van akkeren and cavaye 1999). this study has shown that smes have an overriding need for communication, closely followed by a need for e-commerce capabilities and security; thus, specific applications that address these needs include: • access to email, • trading with customers, • trading with suppliers, • banking and other administrative tasks, and • monitoring business premises. the anova analysis showed that there are in fact key differences between the adopter level groups and, in marketing and adoption terms, this should be taken into account. there is a distinct difference, for example, between fulland partial-adopters when considering how long to wait before investing in mdt; the fulls would adopt quicker than the partials. equally, the more comfortable the owner/manager is with current technology, the more excited they feel about the mdt, and this is further supported when dealing with staff, as the more itliterate the owner/manager, the less threatened they say thenstaff would feel. table 9 summarises the top ranking features of mdt needed by adopter group. table 9: top ranking features of mobile data technology by adopter groups statement being able to email trade with customers banking and other admin. monitoring business premises trade with suppliers full-adopter {n=275) 1st 2nd 3rd 3rd 3rd partial-adopter (n— 137) 1st 2nd 3rd .non-adopter (n=70) 3rd 3rd 2nd 1st broadly, small businesses have needs for communication (email), e-commerce (trade with customers and suppliers) and security (monitoring business premises) when considering mdt. the industry sector to which respondents belonged also had some influence on attitudes towards mdt and this supports findings in previous studies that industry sector will influence adoption (thong 1999, yellow pages 1999). interest in acquiring the technologies was strongest in the transport and storage, communication, and personnel and other services sectors. overall the reaction to the new technology was very positive as, in every industry except one, more than half of the respondents were either 'very interested' or 'interested' in acquiring mobile data technologies. (the industry least interested in acquiring the technology was construction at 46%). within these industry sectors, the top ranking applications that support the underlying principle of mobile data technologies (business anywhere, anytime) were consistently noted as email (communication), trading with customers, trading with suppliers, banking and other administrative tasks (e-commerce), and monitoring business premises (security). 27 ajis vol.9, no. 2 may 2002 conclusion findings from this study have allowed refinement and development of the model presented earlier (figure 1). the refined model, figure 2, incorporates a range of factors impacting on regional sme adoption of mdt specifically and it is hoped that this model can provide direction for future research on the adoption and diffusion ofmdtforsmes. figure 2 factors impacting sme adoption of mobile data technologies j . mobile i)eyicc applications r>;ten«ur':'..'.';'.:v ' • ^"trading withsuppk'ers adoption of mobile ; data technologies { smes, by definition, do not have the luxury of time and money that bigger firms do. if they are going to invest money in new technology, whether it be a mobile phone, computer, pda or mdt, they need to be able to reap immediate rewards. the most important aspects of sme owner/manager's daily business lives are concerned with communication and being in constant contact with the business in order to pursue contacts, orders, invoices and so on. however, they are also aware of the way the world is changing and, thus, identified a need of being e-commerce capable. these managers recognise the inherent cost savings of conducting electronic business and banking and this is important to them. finally, security is an vital issue for most smes; their business premises are their livelihood and any untoward actions (such as staff pilfering, burglaries) have an immediate and dramatic effect on their bottom-line. findings from this research project make both theoretical and practical contributions. theoretically we now know more about the adoption of the newest innovation, mdt, in the context of regional smes. this work has contributed to the disciplines of information systems, marketing and small business. in practical terms, both marketers and developers of mdt can learn from the findings presented here and strive to continue to be customer-focused. 28 ajis vol. 9, no. 2 may 2002 limitations and further research despite attempts to instil the utmost rigour into research designs, all studies will inevitably have limitations, and this one is no different. the two major limitations of this research are concerned with the data collection method employed and the lack of knowledge by respondents about this very new technology. the data collection method employed in this study was selected as being the most practical, given the nature of the task. the cost and time involved in conducting face-to-face interviews with 500 small and medium sized enterprises would have been prohibitive, and the poor response rates of mail surveys are well documented. thus, telephone interviewing was selected and the needs of the respondents were given great consideration. the interviews were no longer than ten minutes and the questionnaire utilised both open-ended and scaling questions to assist in time management. however, the drawbacks of this type of data collection technique with this population are that owner/managers of smes have little time to spend discussing new technologies over the phone, as they are very busy people. the second limitation of this study relates to the implications associated with conducting research into very new technology. whilst most people interviewed were aware of current technologies such as the internet, mobile phones and computers, knowledge of the new mobile data technology was obviously limited. this situation inevitably had an effect on soliciting views about the technology. whilst these two limitations are significant, every effort was made to make the data collected both relevant and rigorous. this project has far-reaching implications in terms of further research and a number of directions are proposed below for other researchers to pursue: 1. replicate this study in a metropolitan environment. 2. replicate this study in other regional areas. 3. replicate this study in other countries. 4. tracking studies, say, in 6 months, then one year: a. to test changes in adoption status. b. to test changes in needs and wants in terms of applications. c. to test changes in attitudes and needs of different industry sectors. 5. industry-specific needs study. 7. focus on adopter groups. all of these areas of further research will elicit information that will be useful to the development and adoption of mdt in australia. each area of further research is a significant project in its own right and different research methodologies should be employed to uncover the truth in each case. references agarwal, r. and prasad, j. (1997), 'the role of innovation characteristics and perceived voluntariness in the acceptance of information technologies'. decision sciences, v28, n3, pp.557-582. ajzen, i. (1991), "the theory of planned behaviour', organisational behavior and human decision processes, pp. 179-211 in taylor, s. and todd, p.a. 1995, 'understanding information technology usage: a test of competing models', information systems research, june, v6, n2, pp.144 -176 axby, e. (1998), 'creating a market for mobile data', telecommunications, september, v32, n9, pp. 37-39. bridge j. and peel m. j. (1999), 'a study of computer usage and strategic planning in the sme sector', international small business journal, london, v!7, n4, pp.82. brooksbank, r., kirby, d., and kane, s. (1992), 'it adoption and the independent retail business: the retail newsagency', international small business journal, apr-jun, vlo, n3, pp. 53-61 chan, t. (2000), 'cracks in the wap', america's network, v!04, n3, pp.26-27. coakes, s.j. and steed, l.g. (1999) spss: analysis without anguish, john wiley and sons, milton, australia. davis, f. (1989), 'perceived usefulness, perceived ease of use and user acceptance of information technology', mis quarterly, september, v!3, n3, pp. 319-340. dearne, karen (2001), 'smes learning the e-ropes', the australian, august 14th, p. 41. duffy, r. (1999), 'wireless set to take the lead', telecommunications, v33, nl 1, pp. 24-26. fichman, r. g. and kremerer, c.f. (1993), 'adoption of software engineering process innovations: the case of object-orientation', sloan management review, winter, pp. 7-22. fink, d. (1998), 'guidelines for the successful adoption of it in small and medium enterprises', international journal of information management, august, v!8, n4. forrester research incorporated (1997), forrester ranks world economics for ecommerce, 30 april, cambridge, ma, [online] <http://www.forrester.com.>. accessed august 1998. friar, j. h. and balachandra, r. (1999), 'strategies for marketing new technologies', research technology management, v42, n4, pp. 37-43. 29 ajis vol. 9, no. 2 may 2002 gefen, d., and straub, d.w. (1997), 'gender differences in the perception and use of e-mail: an extension to the technology acceptance model', mis quarterly, december, v21, n4, pp. 389-400. greengard, s. (2000) 'going mobile', industry week, v249, nl, pp. 18-22. harrison, d.a., mykytyn, p.p. jr, and rienenschneider, c.k. (1997), 'executive decisions about it adoption in small business: theory and empirical tests', information systems research, a journal of the institute of management sciences, june, v8, n2, pp. 171-195. harrison, h. (1999), 'wap: the key to mobile data', telecommunications, september, v33, n9, pp. 96-98. horn, d. (2000), 'e-business portals ease end-user service creation', telecommunications, v34, nl, pp.43-45. lacovou, c.l., benbasat, i., and dexter, a.a. (1995), 'electronic data interchange and small organisations: adoption and impact of technology', mis quarterly, december, v!9, n4, pp. 465-485 johnson, a.h. (1999), 'wap', computerworld, v33, n44, p. 69. julien, p.a., and raymond, l. (1994), 'factors of new technology adoption in the retail sector', entrepreseurship: theory and practice, summer, volls, n5, pp79-90. kirby, d., and turner, m. (1993), 'it and the small retail business', international journal of retail and distribution management, v21, nl, pp20-27. lawrence, k.l. (1998), 'factors inhibiting the utilisation of electronic commerce facilities in tasmanian smallto medium-sized enterprises', 8th australasian conference on information systems, october. mahajan, v. and muller, e (1998), 'when is it worthwhile targeting the majority instead of the innovators in a new product launch', journal of market research, v35, n4, pp.488-495. mcgregor, j. and gomes, c. (1999), technology uptake in smes: some evidence from new zealand', journal of small business management, v37, n3, pp94-102. mobile data conference (1999), sydney, nsw. nambisan s. and wang y. (1999) 'roadblocks to web-based technology adoption', association for computing machinery, communicationsoftheacm, new york, january, v42, nl. pp98-101. osowski, k. (1999), 'unifying communications', communications news, v36, n9, pp. 18-20 piovesana, r. and raush, b. (1998), "the web help small players connect', bobbin, may, v39, n9, pp. 50-51. riggs b. and bachelor b. (1999), 'vendors address issue of security for wireless devices', informationweek, december, issue 765, pp. 25-26. rogers, e.m. (1995), diffusion of innovations, 4th edn, the free press, new york. saunders, s., heywood, p., dornon, a., bruno, l. and allen, l. (1999), 'wireless ip: ready or not, here it comes', data communications, september, v28, n!2, pp. 42-68. sekaran, u. (1992), research methods for business, a skill building approach, 2nd edn, john wiley and sons, usa. sillence, j.a.a., macdonald, s., lefang, b. and frost, b. (1998), 'email adoption, diffusion, use and impact within small firms: a survey of uk companies', international journal of information management, v!8, n4, pp. 231-242. taylor, m. (1999), 'the need for speed', communications international, v26, n2, pp. 41-42. taylor, s. and todd, p.a. (1995), 'understanding information technology usage: a test of competing models', information systems research, june, v6, n2, pp.144 -176. thong, j.y.l. (1999), 'an integrated model of i/s adoption in small businesses', journal of management information systems, spring, v!5, n4, pp, 187-214. thong, j. and yap, c.s. (1995), 'ceo characteristics, organisational characteristics, and information technology adoption in small business', omega, august, v23, n4, pp. 429-442 thurston, c. (2000), 'economic waves wash onto all shores', global finance, v!4, n.l, pp. 121-123. van akkeren j k and cavaye a l m (1999), 'factors affecting entry-level internet adoption by smes: an empirical study', proceedings from the australasian conference in information systems, v2, pp. 17161728. wexler, j (1999), 'cut the cord', upside, vl 1, nl 1, pp. 201. yellow pages australia (1999), small business index: survey of computer technology and ecommerce in australian small and medium business, may. 30 microsoft word ajis vol13-2 o.doc 135 australasian journal of information systems volume 13 number 2 may 2006 accessing data: methods for understanding mobile technology use *penny hagen, *toni robertson, #melanie kan, *kirsten sadler *faculty of information technology, #faculty of engineering, university of technology, sydney ultimo, new south wales, australia email: toni@it.uts.edu.au abstract mobile devices, applications and services have become integrated into people’s daily lives on a personal and professional level. although traditional research methods are being used to understand the use of mobile devices and applications, methodological challenges still exist. researchers have responded to these challenges in a variety of ways, with an emphasis on developing methods that enable new ways of accessing, making available and collecting data about mobile technology use. this paper identifies, defines, describes and presents, a preliminary framework for understanding the methodological responses emerging in current mobile human computer interaction (mobile hci) research. introduction traditional methods, such as direct observation, used in human computer interaction (hci) and computer supported collaborative work (cscw) research are commonly applied and adapted to mobile environments. however, variations and new methods are emerging in response to the particular challenges of mobile technology use. these result from both the complexity that physical movement and changing variables present for data collection and research design (kjeldskov and stage, 2004) and the small scale and ubiquitous nature of mobile devices. for example, observations of mobile technology use can require researchers to follow people to various and multiple geographic locations, such as into confidential meetings (väänänen-vainio-mattila and ruuska, 1998), trips to the movies or the park, business trips (perry, o’hara, sellen, brown and harper, 2001), on shopping excursions (newcomb, pashley and stasko, 2003) and into the bedrooms of teenagers (grinter and eldridge, 2001) or long distance lovers (kjeldskov, gibbs, vetere, howard, pedell, mecoles and bunyan, 2004). the very nature of mobile devices means that we carry them with us and we use them in a number of ways and situations both professionally and personally. this paper contributes a preliminary framework for understanding the emerging methodological responses to the challenges of studying mobile technology use. this framework sets out the array of traditional, modified, novel and combined approaches currently being used by researchers in the exploration, evaluation and design of mobile devices. by gathering, grouping and describing the methodological options currently being exploited, our intention is to encourage and support further research and to offer a tool for interested parties to exploit as they design and develop their own projects. in particular, we have identified the issue of gaining and negotiating access to data as particularly significant for studies of mobile technology use. this is due to the variety of times, ways and places in which use occurs and the personal scale of mobile devices. as a result, many of the ways in which established research methods are being extended account for the difficulties of accessing and collecting valid data in mobile situations. furthermore, we found that traditional australasian journal of information systems volume 13 number 2 may 2006 136 approaches, such as ethnography and field studies, are being rethought—not in terms of their approach, motivation or theoretical commitment—but in terms of the methods used to achieve them. in their review of mobile hci research methods, kjeldskov and graham (2003) found very few examples of studies of actual use in real settings or of conceptual and theory building research. our work builds on and extends the contribution of kjeldskov and graham. however, this paper differs in that it explores new methodological approaches that are emerging in response to the complexities of mobile technology research, rather than categorising approaches according to existing methods. also, kjeldskov and graham differentiate between methods based on their research purpose, for example, to evaluate, to understand or to describe. although we are aware that these are different processes and certain methods lend themselves to different stages of the design cycle, accessing and collecting data is a challenge that faces researchers irrespective of their research purpose. these overall issues of making data available, whether for analysis, representation or to inform design, are the foci of this paper. literature search this paper is based on an extensive literature review of papers that reported on methods for studying and evaluating mobile technologies from the following conferences between the years 2000 and 2004 (hagen et al., 2005): • conference on human factors in computing, chi, acm • conference on computer supported cooperative work, cscw, acm • symposium on designing interactive systems dis, acm • international workshop on wireless and mobile technologies in education, wmte, ieee • symposium on human-computer interaction with mobile devices, mobile hci, springer • participatory design conference, acm • conference on ubiquitous computing, acm • conference on mobile and ubiquitous multimedia, mum, acm • conference of the computer-human interaction special interest group of the human factors and ergonomics society of australia, ozchi one hundred and forty nine papers, reporting aspects of studying use, were selected as relevant to our review. papers were included in the review of they had substantial focus on aspects of the use of mobile technologies. hagen et al., 2005 provides a full list of papers reviewed in the initial research including an annotated bibliography. in addition, we also draw here on more recent publications from these conferences, as well as other major reviews, critiques and comparisons of mobile methods in published literature and theses. there is a significant existing body of work available on lab-based methods for mobile research (e.g. for an extensive survey of lab based studies of mobile technology use see beck, et al. 2003; also kjeldskov and stage, 2004; po et al. 2004; pirhonen et al. 2002). rather than replicate this work, our original literature search focused on those research methods used to study mobile technology use in real settings. we should note here that any discussion on research methods is constrained both by a lack of consistency with which methods are 137 australasian journal of information systems volume 13 number 2 may 2006 named and considered within the human-centred design literature, as well as by the frequent lack of availability of detailed accounts of the method used. we have compensated by the breadth of our survey and by cross-referencing similar work by the same authors and related research by others. methodological challenges many of the complexities related to gathering data about mobile technology use stem from the fact that we are no longer attempting to gather data in a fixed office environment. as researchers we need to account for the potential for physical movement and changing geographical location of users. but we also need to negotiate access to private and public spaces that are not defined by the rules of the workplace. for example, researching people’s shopping habits in order to design improved mobile shopping assistants (newcomb et al., 2003), understanding how groups of friends might use an sms chat application to rendezvous at a particular location (axup, viller and bidwell, 2005), or gaining insight into why and how teenagers might use a mobile device to interact with an existing virtual community (still, isomursu, isomursu and koskinen, 2002). direct observation, which can include note taking, photography and video recording, has historically been favoured by user centred design researchers because it is a method that situates the researcher in the context in which technology use occurs. however, environments of mobile use are not always conducive to these methods. for example, researchers following people around shopping malls with conventional video cameras can be disruptive as members of the public may stop and stare (isomursu, kuutti and väinämö, 2004) and shadowing participants at the same time as trying to take notes can be rather impractical (palen and salzman, 2002). in addition, mobile devices are designed on a personal scale for relatively discrete use within our personal body space; a mobile phone or pda is intended to be used by one person at a time. therefore, capturing or analysing the interface actions of the user, or observing a devices screen, can be physically very difficult (kjeldskov et al., 2005; newcomb, et al., 2003; mark, christensen and shafae, 2001). furthermore, mobile devices, particularly personal communication devices, are having a significant impact on the ways in which we coordinate, communicate and conduct our daily lives (rheingold, 2002; plant, 2001; harper, 2003; ito, 2005; ling, 2004). such devices contribute to the constant reshaping of our body image and the constitution of new and evolving socio-technical relationships and identities (ito, 2005). as hosts to mobile technologies, our bodies become integrated into evolving information and communication systems (viseu, 2003). research into mobile technology use crosses boundaries of the personal, at a physical and conceptual level. these types of research environments require balancing the privacy concerns of participants with the need for researchers to gain access to data. while in this paper we focus on methods for access and collecting data about mobile technology use, we advocate and support continuing work into a broader methodological questioning and investigation that considers what it actually means to be conducting research in such environments and the types of methods that are appropriate. methodological responses researchers are responding to the methodological challenges of access by extending and combining existing methods and developing novel techniques. we have grouped the emerging responses under the following three categories: australasian journal of information systems volume 13 number 2 may 2006 138 1. mediated data collection. access to data and data collection about use in natural settings are mediated by participants and/or mobile technologies. this happens in three basic ways: • do it: participants actively collect the data. • use it: data collection occurs automatically as a side effect of using the technology. • wear it: participants carry or wear the data collection device. 2. simulations and enactments. data about use is made available through forms of ‘pretending’ that encompass aspects of real use situations. 3. combinations. existing methods and/or mediated data collection and/or simulations and enactments are combined to allow access to complementary data. table 1 provides a summary of the approaches, identifies established methods from which these approaches are derived and supplies examples of their use. for reasons of space we have allocated a number for examples; the number corresponds to the numbered reference list at the end of this paper. approach: description: derived from: examples mediated data collection data about use in naturalistic settings is collected via participant and/or technology. do it participants collect data using mobile devices. self-reporting, diaries, probes [26, 27, 31, 53] use it data about use is collected automatically as the device is used. use/data logs [15, 20] wear it data is collected automatically via devices worn on or by the participant. video observation, use/data logs [30, 38, 46] simulations and enactments data about existing or potential use is accessed through pretending and/or imagination. simulations physical, ergonomic or environmental props are used to simulate physical aspects of use. lab tests, heuristics, prototypes, nasa tlx, emulators, simulators, scenarios [7, 8, 19, 38] enactments role-playing, imagination and storytelling are used to explore potential use, often in situ. scenarios, roleplaying, prototyping, work shopping, story boarding [29, 46, 54] combinations various established and/or new methods are combined to enable access to complementary data. [2, 12, 13, 14, 15, 22, 60] table 1. emerging research methods in mobile hci mediated data collection: do it mobile hci researchers are employing self-reporting methods such as diaries and cultural probes (gaver, dunne and pacenti, 1999) to access data about users’ experiences and activities. in selfreporting, participants take on the role and responsibilities of data collection. for example, grinter 139 australasian journal of information systems volume 13 number 2 may 2006 and eldridge (2001) conducted a study into how teenagers used text messaging using a diary method. participants were not comfortable with being observed and instead were asked to log their sms activities in a hand-written diary. this self-reporting method gave the researchers access to data where conventional direct observation was not appropriate. cultural probes and diaries have also been augmented through the use of mobile technologies themselves as reporting tools. mobile probes (hulkko, mattelmäki, virtanen and keinonen, 2004), sms probes (jönsson, svensk, cuartielles, malmborg and schlaucher, 2002) and experience clips (isomursu, kuutti and väinämö, 2004) are all self-reporting methods that employ mobile technologies as data collection tools. for example, hulkko et al. (2004) developed mobile probes as a way for participants to document their shopping experiences using the in-built camera and sms functions of their mobile phones. this information was automatically uploaded to a server that displayed the users’ reports online in a predefined format. hulkko et al. outline the advantages of such methods as: “…the possibilities to remotely and simultaneously observe several users, to automatise the sorting of the data and to create digital user databases for the stakeholders. furthermore, users can become more active contributors instead of being only passive sources of data” (p 2). in addition, these types of self-reporting, mediated data collection methods have also been used to evaluate aspects of user experience and usability for new prototypes in the field, e.g. (isomursu et al., 2004). self-reporting methods simultaneously build a relationship between researchers and the community that they are studying and/or designing for, and enable remote data collection. the limitations and constraints of such methods include: the possibility of users having to learn new interfaces or technologies to operate self-reporting devices (jönsson et al., 2002; palen and salzman, 2002); the implications of participant-recording in inappropriate places such as shops where photos/video are often prohibited (hulkko et al., 2004; newcomb et al., 2003); and the impact on indirect and/or involuntary participants when their activities are recorded without permission. further research is required to understand the research implications, as well as the potential, of such self-reporting methods for use in private or sensitive spaces. in the discussion section below we briefly expand upon some of the ethical questions posed by the use of remote digital technologies as data collection tools. mediated data collection: use it the use it mode of mediated data collection relies on the automatic generation of use or data logs. researchers and participants are relieved of the burden of data collection as data and metadata about use is automatically generated as a side effect of using the device. for example, curtis et al. (2002) developed a tool to log and time-stamp the actions of students using a handheld device as part of their learning program. curtis et al. (ibid.) attributed the success of their research to the in-depth analysis that could be supported by a complementary combination of log files, student artefacts and observation. use logs have always been available to digital technologies, but technologies such as bluetooth add new dimensions to what, and how, information can be logged (eagle and pentland, 2005). for example, there are platforms such as aware (aware, 2005), which filter sms messages based on location and proximity information. cell tower ids can give a user’s location and bluetooth can identify the other active devices that were nearby at the time the sms was sent. while these methods are subject to similar ethical concerns as self-reporting methods that employ digital technologies, this approach also poses its own issues for consideration. automated data collection methods differ from methods in which users volunteer information. although users initially give permission to have australasian journal of information systems volume 13 number 2 may 2006 140 their actions recorded, they may decide later that they want to exempt certain actions. understanding and determining best practice for these emerging tools is ongoing. such concerns also apply to data collection tools that are worn by the participant. mediated data collection: wear it data collection can also be mediated by mobile technologies that participants wear, or carry. mini cameras are one such tool. as with use it mode, data collection occurs automatically as people go about their daily activities. mark et al. (2001) designed a mini camera that attached to a pair of glasses worn (along with a battery pack) by the participant. this method enables a form of remote video analysis and overcomes the impracticalities of following workers to various different locations. however, limitations include the length of time people can comfortably wear such devices, and whether the content that is captured provides enough peripheral information. mini cameras have also been used as a way of capturing screen display and interface actions that are hard to observe due to the small personal scale of mobile devices. in a field study of nurses using handheld devices, kjeldskov, skov, als and hoegh (2004) combined conventional and mediated data collection methods. a mini camera was attached to the handheld device and a microphone recorded the nurses’ comments. the small camera was able to capture high-resolution video of the screen display and interface data, which was sent over a wireless network to a researcher in range. the mini mobile recording devices enabled the capture of high-resolution video of the device itself, while also facilitating the collection of data at a level appropriate to the environment. the videoing of patients in a hospital setting is almost always ethically inappropriate; therefore a conventional video camera could not have been used. the mini camera method enabled the capture of very specific video data relating to the use of the mobile device while preserving the privacy of the patients. remote sensor driven technologies are also being developed to overcome issues of access that exist when studying mobility, and mobile device use. for example, intille, tapia, rondoni, beaudin, kukla, agarwal, bao and larson (2003) are seeking to improve observation techniques through various types of room-mounted, object-mounted and body-mounted sensors. context aware sensors (cas), for example, are triggered depending on contextual cues, such as when a person gets close to their local store, which can be determined by using a global positioning system (gps) (ibid.). remotely triggered methods that react to other environmental conditions remove the burden of selfreporting from the participant, and offer the opportunity to undertake longitudinal studies of use in natural settings. intille et al. (2003) acknowledge the data collected from such automated tools is only significant when contextual information about user activities is also available, and like all mediated data collection methods, benefits come from use in combination with other methods. simulations and enactments rather than focusing on methods for collecting data, simulations and enactments rely on participants acting out scenarios, using role-play or imagination, in order to make available information about device use that may not be otherwise accessible or even exist. these approaches make available information about potential or future use situations and allow prototypes to be tested and reflected upon through experiential means. for example, simulating physical movement while testing prototypes in a lab (beck et al., 2003), expanding traditional lab usability testing methods into the wild (goodman, brewster and gray, 2004), conducting mobile heuristic walkthroughs (vetere, howard, pedell and balbo, 2003), performing scenarios with a mobile prototype in the context of use (iacucci, kuutti and ranta, 2000), or storytelling to explore product concepts (still, 141 australasian journal of information systems volume 13 number 2 may 2006 isomursu, isomursu and koskinen 2002). enactments are particularly helpful for researching future technologies where an actual use context does not yet exist, while simulations provide data about use where natural settings cannot be accessed or controlled sufficiently for the purposes of observation and evaluation. both simulations and enactments rely on a form of pretending, role-playing, body storming or imagination, as researchers and participants simulate or act out a set of tasks, or a particular use scenario. simulations simulations rely on predefined tasks and generally use quantitative analytic methods including measuring aspects such as timing, error rates and workload. technology use is contingent on the variables of any given situation and traditional lab based user testing is criticised for isolating technology from the context in which it will be used. this means that data from lab testing does not necessarily translate well outside the lab context (beck et al., 2003; intille et al., 2003; kjeldskov et al., 2005). but because lab testing enables the controlled collection of comparable data, researchers have expended considerable effort to develop ways of making lab testing more relevant and viable for the evaluation of mobile technologies (pirhonen et al., 2002; kjeldskov and stage, 2004; kjeldskov, skov et al., 2004; po et al., 2004). simulations are also being employed by researchers to avoid the prohibitive costs, both financial and of time, of conducting field studies (kjeldskov, skov et al., 2004; po et al., 2004). the common factor in simulations is the use of techniques to “reflect or recreate a mobile use situation” (beck et al. 2003, p 107). simulations range in their level of sophistication, which is measured by the extent to which the environmental and ergonomic factors of the actual use context are recreated. for example, in conventional laboratory based testing users are seated at a desktop computer. in baber and westmancott’s (2004) trial of a multiplayer game, participants were asked to get up and move around while they used the device. props, such as treadmills, simulate aspects of mobile use related to physical movement and ergonomics (pirhonen et al., 2002; beck et al., 2003; kjeldskov and stage, 2004). goodman et al. (2004) extended typical lab usability measures into the street to measure how walking increased participant workload when using mobile devices. taking the life size experiments outside the lab did result in a reduction in control to some degree, however the advantages were found in the relative realism of the data (brewster, 2002; östergren, 2003). researchers can measure ways in which the mobility of participants and the environmental variables that result from that mobility affect device use, for example, navigating a device whilst walking (brewster, 2002; lumsden and brewster, 2003; mackay et al., 2005). simulations enable the controlled capture of comparable and measurable data that is sensitive in varying degrees to aspects of use in a natural setting. access to data is more feasible, established mechanisms such as observation and video recording can be used, and quantifiable data can be produced and compared. similarly simulation techniques and experiments may elucidate factors of ergonomics and device input while mobile. an example of a simulation on a much larger scale is the sophisticated e-home simulation investigated by koskela, väänänen-vainio-mattila and lehti (2004) with characteristics of both field and lab experiments. two participants lived in a real/simulated home for six months in order to test the usability and usefulness of a mobile phone as a remote control to their smart home. however the key limitation of simulations is that “social context” is missing from most simulation experiments (beck et al., 2003). while the study by koskela et al. (2004) provides rich data on the usability and the potential usefulness of such a device, most simulations are still unable to account for many aspects of actual use situations beyond superficial ergonomics. australasian journal of information systems volume 13 number 2 may 2006 142 in a comparison between a life size lab simulation and field trials, kjeldskov, skov et al. (2004) identified more usability problems in the lab simulation. however, they also describe how participants behaved differently performing the simulation, compared to actually using the device in the field. for example, in the field trial the participants (nurses) were concerned about the reliability of the system but did not express similar concerns in the lab simulation. similarly, while participants experimented with using a pen input device in the lab, participants did not use the pen at all in the field. kjeldskov, skov et al. concluded that there is little value in performing field trials. in contrast, we suggest that these two examples indicate that people act and feel differently toward a system in a simulation than they do in real use situations. in particular, simulations can be limited by their emphasis on participant/device interaction, at the expense of broader contextual considerations. this means that safe and reliable transferability of findings from simulations to use in natural settings cannot necessarily be relied upon. enactments simulations are experiments designed to isolate, compare and capture data about variables of use under certain circumstances. enactments on the other hand tend to be more qualitative in their approach. researchers facilitate, rather than drive, the process. enactments use tools such as roleplay, imagination and enacted scenarios to make available information about existing user experiences of mobile technologies and the ways in which mobile devices and applications might fit into future use practices. enactments often make such information available by playing out scenarios about potential devices in the context in which they might be used (iacucci et al., 2000; messeter et al., 2004). iacucci et al. (2000) developed situated and participative enactment of scenarios (spes) for this purpose. in spes researchers observe participants in their daily activities as they improvise possible scenarios with a mock-up of a potential mobile device. these scenarios, which are recorded and fed into the design process, are valuable because they have been created in collaboration with the user and are subject to the variables of the context in which they will be used (ibid.). we identify storytelling as another form of enactments, in that it relies upon the imagination of participants and is used as a way of making available data about use when there is no existing use context and conventional methods are not appropriate. still et al. (2002) were working with teenagers to identify possible devices to support an existing online community. they found that teenagers did not respond well in interview situations and little useful data about lifestyles and potential use was revealed. instead they asked their participants to create stories about a potential device and how they might use it in their daily activities. this approach provided successful methods for discovering potential use contexts and functionalities for a user group that is both difficult to access while, at the same time, a major user of mobile technologies. simulations and enactments involve participants acting out tasks, or imagining how devices might be part of their daily activities. they reveal quite different data to that of other conventional methods such as observation, interviews and focus groups. the understanding about use is experiential and based on participation, or a “way of playing with data in embodied ways” (oulasvirta, 2004, p. 251). in other words, enactments create a shared embodied experience between the researcher and the participant and enhance the researchers knowledge of the environment for which they are designing. however because enactments are not constrained to controlled experimental environments like simulations, they still encounter the limitations typically experienced with direct observation methods. the necessary presence of the researcher in both simulations and enactments restricts their length and viability for use in diverse or multiple locations. the result can be small data samples or short one off testing (barkhuus and dey, 2003; davies et al., 2005; cheverst et al., 2000). 143 australasian journal of information systems volume 13 number 2 may 2006 other challenges included how to record or capture the scenarios that are being enacted in the field without the researcher impacting excessively on the context of use (iacucci and kuutti, 2002). barkhuus and dey (2003), for example, addressed this by asking participants to use self-reporting diaries to capture their imaginary use of a mobile device. participants, involved in a study investigating preferences for context aware interactivity, kept a mobile device with them for five days, imagining that certain applications were available on their mobile phones and recording when they might have used it and why. using a range of methods to capture different kinds of mobile data can overcome a number of access challenges and enable a comprehensive approach to capturing data about mobile technology use. combinations combinations of conventional and established methods such as interviews, focus groups and questionnaires have been employed by researchers to collect data on existing user practices, opinions, needs, experiences and requirements, to elaborate on findings collected from direct and mediated observation and self-reporting, and to triangulate data. although combining methods is not new or particular to mobile research, combinations of the above approaches, as well as combinations including established techniques are particularly suited to mobile studies because of the complexities of accessing and collecting data about use in this domain. different methods allow researchers to access different data, some rapidly, some in depth and some in real-time. for example, in a study of teenage mobile phone use, carroll, howard, vetere, peck and murphy (2002) used a range of existing methods including questionnaires, diaries, focus groups and direct observation. each of these methods gives access to different types of information about users’ experience. focus groups, bringing together small groups of people to talk to each other, draw out different types of information than when participants speak directly to researchers, such as in interviews. in contrast, direct observation involves researchers being present during and studying, people’s everyday practices and activities. carroll et al. (2002) consider the success of their research to stem from this combination of existing methods; focus groups that provide concentrated interaction with the subjects of the research and direct observation that provides naturalness of setting and reliability of findings. in order to measure usefulness, however, or understand how people will appropriate technologies however, studies must take place over time. modes of mediated data collection for example, offer the potential for longitudinal studies and for this reason combinations of methods are advocated. adaptations on traditional ethnographic methods are also being explored for the purpose of negotiating access to personal and private information. for example, the ‘bag studies’ by robertson et al. (2005) and taking photos of backpacker notice boards by axup and viller (2005). these methods expand and extend traditional ethnographic methods to make information about mobile technology use known by analysing traces of mobility and the artefacts that make such actions possible. we anticipate future research that explores how various combinations of emerging and conventional methods can specifically enable new knowledge about the use of mobile technologies. discussion in this paper we have documented how emerging mobile hci methods are making available data about user experience and enabling this data to be collected in different ways. mediated data collection includes a range of approaches for remotely collecting various types of data by relying on the participant or the mobile technologies themselves. simulations enable the capture of data about device input, ergonomics and the impact of using a device while mobile. enactments allow australasian journal of information systems volume 13 number 2 may 2006 144 researchers to explore why we carry these mobile devices with us and what these devices give us the potential to do. combinations of these and conventional methods allow in-depth and complementary data to be collected. the use of mobile devices is personal, opportunistic, and dependent on contingencies and occurs over a range of times, ways and places. knowing when and how to capture actual ‘use’ instances can be difficult and so is accessing information about how the potential for use affects peoples’ behaviour and choices. axup and viller (2005) state, “choosing methods from a wide variety of options and then adapting them to fit the situation should be a standard activity” (p 4). indeed, we found an experimental approach to choosing methods was common across much of the research we reviewed, particularly in relation to field methods. furthermore, researchers wanting to capture data about and within realistic use situations have also adopted a similar experimental approach to recruiting participants. recruitment recruitment has been found to be problematic particularly when this might be occurring on location (axup et al., 2005; benford et al., 2004; bornträger et al., 2003; cheverst et al., 2000; isomursu et al., 2004) or when researchers are negotiating access to sensitive or personal realms or content (demumieux and losquin, 2005; grinter and eldridge, 2001; lamming and bohm, 2003; still et al., 2002). for example, cheverst et al. (2000) were concerned about disrupting the normal activities of the tourists they recruited to field test a mobile guide. isomursu et al. (2004) noted that recruiting participants to their experience clip method was more successful than conventional techniques because participants found the method fun and appealing; the autonomy of participants was retained and the method fitted in with their intended shopping activity. while technical solutions such as remote devices can overcome some of the physical aspects and challenges of gaining access to data, negotiating access to people's personal spaces is more complex. sensitive research domains require empathetic methodological approaches and while researchers may prefer conventional methods such as observation and interview, they may not be appropriate. the success of recruiting participants to certain methods may depend on how effectively this is considered. for example, grinter and eldridge (2001) found that teenagers were uncomfortable being observed, while still et al. (2002) found interviewing teenagers to be unsuccessful as an indepth research tool. both researchers sought alternative methods to access this important user group. similarly axup and viller (2005) describe in detail their attempts to firstly apply conventional, and then non-conventional, methods to mobile use situations, coming up against challenges posed by the very mobility of their backpacker subjects and their boundaries of personal privacy. in addition, the increasing use of non-conventional methods that explore the boundaries of the personal and the private is resulting in the need to reconsider the associated ethical issues. ethical implications are being raised both by the nature of working in such a domain and by the new tools we are developing to do so. ethics and privacy physical and conceptual boundaries are being crossed and renegotiated in the effort to study, observe and understand relationships and actions that occur close to our bodies and within sensitive contexts. explorations of mobile technology use by researchers are revealing the need to rethink the classic questions of the role of the researcher and the agency of the participant. these are important questions that influence the actions and perspectives that different methodological approaches can, or cannot support, and in turn the types of data that they make available. in both enactments, and the 145 australasian journal of information systems volume 13 number 2 may 2006 do it approaches of mediated data collection for example, the agency of the participant is a valuable component of the research. a relationship between participant and researcher is intentionally fostered and the data collected reflects the perspective of those for whom the mobile technologies are designed. modes of digital data collection, successful because of their adaptation of discrete mobile digital tools to support remote reporting, also have implications that require further consideration. for example, the social, ethical and possibly legal implications of asking participants to go out and use such devices to record data about their lives. hulkko et al. (2004) discovered that despite the simplicity of participants using phone cameras to document their actions, asking them to do so while shopping was quite impractical; most shops were unwilling to allow people to take such visual documentation. similarly, those using such methods must negotiate the responsibilities of participants capturing data, particularly those that might include photos or video, of in-direct or involuntary participants who are not informed or aware of the study. these questions only increase in significance as the debate in mainstream media over inappropriate use of mobile phone cameras in public places grows (e.g. lawlink, 2006; o2, 2006; bbc, 2006). our responsibility as researchers to indirect participants is little discussed in current literature and remains unclear. inadvertent data collection is further problematised by the digital nature of these mobile devices. meta data, as well as being a tool that assists auto sorting and filtering of data, can automatically include information and details about which the participants and those in their immediate surroundings might be unaware. while collecting information, such as cell location and the presence of other bluetooth devices, might be the purpose of using such a data collection model, full disclosure about exactly what type of data is being collected is required. a similar responsibility exists for negotiating ways in which participants have control over, or access to editing data that is automatically collected. as mentioned in the mediated data collection use it section of this paper, while participants initially give permission for data to be recorded they may later choose to exempt certain data from the study. clarifying any such concerns about the collection or storage of any such automatically collected data is an issue for both the researcher and participant (demumieux and losquin, 2005; lamming and bohm, 2003). the confidentiality of data collected and displayed online is another issue confronting those using such applications and the responsibilities for data storage, access, future use and ownership take researchers through an often unanticipated maze of competing interests from private and public domains that existing research ethics committees are ill-equipped to manage. as the presence of mobile devices continues to grow and new digital methods expand they will require more of our attention as researchers. conclusion the mobile hci community is steadily building research methods for understanding mobile technology use that rely on a range of different methods combined with valuable new opportunities created by the mobile technologies themselves. in addition, a significant research direction is emerging in the area of the automatic extraction of complex data about technology use through mediated data collection methods. although researchers (e.g. beck et al., 2003) report finding very little variety in the use of new techniques for mobile hci, we have found this not to be the case. the possibilities for researchers to understand mobile technology use in real settings are richer than might first appear from kjeldskov australasian journal of information systems volume 13 number 2 may 2006 146 and graham’s (2003) broad review of mobile research methods. as self-reporting techniques become more sophisticated, people using mobile technologies are increasingly being asked to also act as the observers of their own use. interestingly, neither of the reviews by kjeldskov and graham (2003) or kjeldskov and stage (2004) refer to forms of self reporting, which our survey has found to be a significant tool for studies of mobile use. it may be that this significance was not yet clear when these earlier studies were written. in order to understand mobile technology use, we must also ask how mobile devices shape and reshape the body and its relationship to the environment (viseu, 2003). in future research we hope to further examine the use of these different approaches and the physical, personal and ethical issues of accessing and collecting data for mobile technology use. research that moves beyond measuring user device actions or input while moving to a deeper understanding of the personal capacity for use afforded by mobility leads to a complex, largely intangible and difficult research domain. it is possible that simplistic conceptualisations of mobility may have contributed to an emphasis on mobile hci research methods that conceive of mobility as an ergonomic or physical characteristic, rather than, as in merleau-ponty’s (1962) philosophy for example, a fundamental constituent of human behaviour. there is little research to date that reflects upon the existing assumptions about mobility that form the basis from which much of the literature and resulting studies depart. conceptualisations of mobility that consider the implications of mobile devices as they integrate into our lives and enable different connections, communication and behaviours will enable us to increase our understandings of this particularly challenging area. at the same time they will assist in the continued development of appropriate and flexible research methods and data collection tools. acknowledgements this research is funded through the australia research council discovery grants program and the alcatel research partnership program. references editorial note: the numbering of references is used only for convenience in presenting table 2 above. 1. aware (2005) retrieved december 2004, from http://aware.uiah.fi/. 2. axup, j. and viller, s. (2005) “formative research methods for the extremely mobile: supporting community interaction amongst backpackers”, ozchi 2005, acm, canberra. 3. axup, j., viller, s. and bidwell, n.j. (2005) usability of a mobile, group communication prototype while rendezvousing. in cts 2005. saint louis, missouri, usa. 4. baber, c. and westmancott, o. (2004) social networks and mobile games: the use of bluetooth for a multiplayer card game. in mobile hci, glasgow, uk, pp 98-107. 5. barkhuus, l. and dey, a. (2003) “is context-aware computing taking control away from the user? three levels of interactivity examined”, ubicomp 2003, springer-verlag, seattle, wa, usa, pp 149-156. 6. bbc (2006), “mobile picture power in your pocket”, retrieved january, 2006, from http://news.bbc.co.uk/1/hi/technology/3991775.stm) 147 australasian journal of information systems volume 13 number 2 may 2006 7. beale, r. and lonsdale, p. (2004) mobile context aware systems: the intelligence to support tasks and effectively utilise resources. in mobile hci, glasgow, uk, 240-251. 8. beck, e.t., christiansen, m.k., kjeldskov, j., kolbe, n. and sage, j. (2003) “experimental evaluation of techniques for usability testing of mobile systems in a laboratory setting”, ozchi 2003, brisbane, pp 106-15. 9. benford, s., seager,w., flintham, m., anastasi, r., rowland, d., humble, j., stanton, d., bowers, j., tandavanitj, n., adams, m., row farr, j., oldroyd, a. and sutton, j. (2004) “the error of our ways: the experience of self-reported position in a location-based game”, ubicomp, springer, nottingham, uk. 10. bornträger, c., chervest, k., davies, n., dix, a., friday, a. and seitz, j. (2003) “experiments with multimodal interfaces in a context-aware city guide”, mobile hci, udine, italy, pp 116-130. 11. brewster, s. (2002), “overcoming the lack of screen space on mobile computers”, personal and ubiquitous computing, vol. 6, no. 3, pp 188-205. 12. brodie, j. (2003) designing to support communication on the move. in chi03, ft. lauderdale, florida, usa: acm press, pp 908-909. 13. carroll, j., howard, s., vetere, f., peck, j. and murphy, j. (2002) just what do the youth of today want? technology appropriation by young people. in 35th conference on system science. hawaii: ieee. 14. cheverst, k., davies, n., mitchell, k., friday, a. and efstratiou, c. (2000) developing a context-aware electronic tourist guide: some issues and experiences. in chi 2000, the hague, the netherlands: acm press, pp 17-24. 15. curtis, m., luchini, k., bobrowsky, w., quintana, c. and soloway, e. (2002) handheld use in k-12: a descriptive account. in international workshop on wireless and mobile technologies in education. ieee. växjö, sweden. 16. davies, n., cheverst, k., dix, a. and hesse, a. (2005) “understanding the role of image recognition in mobile tour guides”, mobile hci, salzburg, pp 191-198. 17. demumieux, r. and losquin, p. (2005) “gather customer's real usage on mobile phones”, mobile hci, acm, salzburg, pp 267-70. 18. eagle, n. and pentland, a. (2005) “social serendipity: mobilizing social software”, ieee pervasive computing, vol. 4, no. 2, pp 28-34. 19. esbjornsson, m., juhlin, o. and ostergren, m. (2003) motorcyclists using hocman field trails on mobile interaction. in mobile hci 2003. udine, italy 20. fitton, d., cheverst, k., rouncefield, m., dix, a. and crabtree, a. (2004) probing technology with technology probes. in equator workshop on record and replay technologies. 21. gaver, b., dunne, t. and pacenti, e. (1999) “design: cultural probes”, interactions, pp 21-29. 22. goodman, j., brewster, s.a. and gray, p.d. (2004) using field experiments to evaluate mobile guides. in mobile hci 2004, glasgow, uk: springer, pp 38-48. 23. grinter, r. and eldridge, m. (2001) y do tngrs luv 2 txt msg. in ecscw. amsterdam. 24. harper, r. (2003) people versus information: the evolution of mobile technology. in mobile hci, springer-verlag, pp 1-14. 25. hagen p., robertson, t. and kan m. (2005) methods for understanding use of mobile technologies, idwop technical report, interaction design and work practice laboratory, fit, uts. http://research.it.uts.edu.au/idwop/publications.html australasian journal of information systems volume 13 number 2 may 2006 148 26. hulkko, s., mattelmäki, t., virtanen, k. and keinonen, t. (2004) mobile probes. in nordichi 04. tampere, finland. 27. hutchinson, h., mackay, w., westerlund, b., bederson, b.b., druin, a., plaisant, c., beaudouin-lafon, m., conversy, s., evans, h., hansen, h., roussel, n., eiderbäck, b., lindquist, s. and sundblad, y. (2003) technology probes: inspiring design for and with families. in chi 2003, acm press, pp 17-24. 28. iacucci, g., kuutti, k. and ranta, m. (2000) on the move with a magic thing: role playing in concept design of mobile services and devices. in dis2000 (193-202). ny, usa: acm. 29. iacucci, g. and kuutti, k. (2002) everyday life as a stage in creating and performing for wireless devices. personal and ubiquitous computing, 6, pp 299-309. 30. intille, s., tapia, e.m., rondoni, j., beaudin, j., kukla, c., agarwal, s., bao, l. and larson, k. (2003) tools for studying behavior and technology in natural settings. in ubicomp 2003, seattle, wa, usa: springer-verlag heidelberg, pp 157-174. 31. isomursu, m., kuutti, k. and väinämö, s., (2004) experience clip: method for user participation and evaluation of mobile concepts. in participatory design conference, toronto, canada, pp 83-92. 32. ito, m., okabe, d. and matsuda, m. (eds) (2005) personal, portable, pedestrian: mobile phones in japanese life, mit press. 33. jönsson, b., svensk, a., cuartielles, d., malmborg, l. and schlaucher, p. (2002) mobility and learning environments engaging people in design of their everyday environments. retrieved november 2004, from http://www.certec.lth.se/doc/mobility1/. 34. kjeldskov, j. and graham, c. (2003) a review of mobile hci research methods. in mobile hci 2003. udine, italy. 35. kjeldskov, j., graham, c., pedell, s., vetere, f., howard, s., balbo, s. and davies, j. (2005) usability of a mobile guide: the influence of location, participants and resource. behaviour and information technology, 24, pp 51-65. 36. kjeldskov, j., gibbs, m., vetere, f., howard, s., pedell, s., mecoles, k. and bunyan, m. (2004). using cultural probes to explore mediated intimacy. in ozchi. wollongong, australia. 37. kjeldskov, j. and skov, m.b. (2003) “creating realistic laboratory settings: comparative studies of three think-aloud usability evaluations of a mobile system”, interact, pp 663670. 38. kjeldskov, j., skov, m.b., als, b.s. and hoegh, r.t. (2004) is it worth the hassle? exploring the added value of evaluating the usability of context-aware mobile systems in the field. in mobile hci, glasgow, uk, pp 61-73. 39. kjeldskov, j. and stage, j. (2004) new techniques for usability evaluation of mobile systems. international journal of human-computer studies (60), pp 599-620. 40. koskela, t., väänänen-vainio-mattila, k. and lehti, l. (2004) home is where your phone is: usability evaluation of mobile phone ui for a smart home. in mobilehci. glasgow, uk: springer. 41. lamming, m. and bohm, d. (2003) “specs: another approach to human context and activity sensing research, using tiny peer-to-peer wireless computers”, ubicomp 2003, springer, seattle, wa, usa, pp 192-8. 42. lawlink (2006), “what privacy issues do camera phones raise?”, retrieved january 2006, from http://www.lawlink.nsw.gov.au/lawlink/privacynsw/ll_pnsw.nsf/pages/pnsw_04_camerap hones. 149 australasian journal of information systems volume 13 number 2 may 2006 43. ling, r. (2004) the mobile connection: the cell phone's impact on society, morgan kaufmann, san francisco. 44. lumsden, j. and brewster, s. (2003) “a paradigm shift: alternative interaction techniques for use with mobile and wearable devices”, cascon ibm, pp 197-210. 45. mackay, b., dearman, d., inkpen, k. and watters, c. (2005) “walk .n scroll: a comparison of software-based navigation techniques for different levels of mobility”, mobile hci, acm press, salzburg, pp 183-190. 46. mark, g., christensen, u. and shafae, m. (2001) a methodology using a microcamera for studying mobile it usage and person mobility. in chi 2001. seattle, washington, usa. 47. merleau-ponty, m. (1962) phenomenology of perception, routledge and kegan paul, london. 48. messeter, j., brandt, e., halse, j. and johansson, m. (2004) contextualizing mobile it. in designing interactive systems, cambridge, ma, usa, pp 27-36. 49. newcomb, e., pashley, t. and stasko, j. (2003) mobile computing in the retail arena. in chi 2003. ft. lauderdale, florida, usa: acm press. 50. o2 (2006), “misuse of camera phones”, retrieved january, 2006, from http://www.o2.co.uk/misuse_camera_phones/0,,510,00.html 51. östergren, m. (2003), “sound pryer field trials: learning about adding value to driving”, mobile and ubiquitous multimedia 2003. 52. oulasvirta, a. (2004) finding meaningful uses for context-aware technologies: the humanistic research strategy. in chi. vienna, austria. 53. palen, l. and salzman, m. (2002) voice-mail diary studies for naturalistic data capture under mobile conditions. in acm conference on cscw. new orleans, louisiana, usa. 54. pedell, s. (2004) picture scenarios: an extended scenario-based method for mobile appliance design. in ozchi2004. wollongong, australia. 55. perry, m., o’hara, k., sellen, a., brown, b. and harper, r. (2001) dealing with mobility: understanding access anytime, anywhere. in tochi 2001. 56. pirhonen, a., brewster, s. and holguin, c. (2002) gestural and audio metaphors as a means of control for mobile devices. in chi 2002, minneapolis, minnesota, usa: acm press, pp 291-298. 57. plant, s. (2001) on the mobile, motorola, uk. 58. po, s., howard, s., vetere, f. and skov, m. b. (2004) heuristic evaluation and mobile usability: bridging the realism gap. in mobile hci, glasgow uk. 59. rheingold, h. (2002) smart mobs: the next social revolution, perseus books group, usa. 60. robertson, t., kan, m., sadler, k. and hagen, p. (2005), “uncovering traces of mobile practices: the bag study”, ozchi 2005, acm, canberra. 61. still, k., isomursu, m., isomursu, p. and koskinen, k. (2002) the potential for mobile media in a virtual community. in mum. oulu, finland. 62. väänänen-vainio-mattila, k. and ruuska, s. (1998) user needs for mobile communication devices: requirements gathering and analysis through contextual inquiry. in first workshop on hci with mobile devices. glasgow. uk 63. vetere, f., howard, s., pedell, s. and balbo, s. (2003) walking through mobile use: novel heuristics and their application. in ozchi 2003, brisbane. 64. viseu, a. (2003) “simulation and augmentation: issues of wearable computers”, ethics and information technology, vol. 5, no. 1, pp 17-26. australasian journal of information systems volume 13 number 2 may 2006 150 the australasian journal of information systems (ajis) began in 1993 and was based on the simple realisation that there was a need for an academic ‘outlet’ for is research in australia australasian journal of information systems volume 14 number 1 nov 2006 culture as an explanation of technology acceptance differences: an empirical investigation of chinese and us users mark srite associate professor management information systems sheldon b. lubar school of business university of wisconsin – milwaukee milwaukee, wi 53201 phone: (414) 229-5502 fax: (414) 229-5999 e-mail: msrite@uwm.edu abstract this article examines the issue of the acceptance of technology across two cultures. to do this an extended technology acceptance model was tested in china and the us. over one hundred participants, across both cultures, were surveyed as to their perceptions regarding technology acceptance. cultural values were also measured for each group. structural equation modeling was used to assess the research model. in general, the model explained a more than adequate amount of variance and achieved acceptable levels of significance. differences across the two cultures were explained utilizing the cultural values of the participants. implications for both research and practice were provided. introduction in the increasingly global business environment, there is a growing need to utilize information technology (it) to achieve efficiencies, coordination, and communication (laudon and laudon, 2006; porter and miller, 1985). business has become more international in today's society. clearly cultural differences between countries may have an impact on the effectiveness and efficiency of it deployment across national boundaries. research can provide valuable insight into the nature of processes, obstacles, and opportunities present in cross-cultural it environments and can enhance our understanding of the cultural factors relevant to the adoption and use of information technology. technology acceptance and usage across cultures is a crucial factor for deriving it benefits in multinational and transnational organizations. 5 mailto:msrite@uwm.edu australasian journal of information systems volume 14 number 1 nov 2006 research objectives the intention of this article is to examine differences in the acceptance of technology across two cultures, that of china and the us. cultural values will be used to provide an explanation these differences. the general research question is: how does national culture influence the adoption and use of an information technology? specifically, this paper reviews the existing technology acceptance literature to extend the technology acceptance model (tam) (davis, 1989) by adding subjective norms. tam was chosen as the basic model since it is a widely accepted yet parsimonious and robust model of technology acceptance. this model is then tested in the two cultures of interest. next, based on cross-cultural research literature, hofstede's (1984) widely cited dimensions of national culture are utilized to explain the results of the model testing. implications for research and practice are then put forward as well as limitations and directions for future research. literature review although the acceptance of technology has been a key component of information technology research and culture has been extensively studied in the anthropology and management disciplines little has been done to integrate these two areas. additionally, culture’s influence in the acceptance and use of technology, in the context of asia, has not been comprehensively examined. this review will explain the current models of technology acceptance, provide an overview of how national culture is conceptualized, and look at the extant studies of technology acceptance that focused on asia. technology acceptance models a number of behavioural models have been utilized to study the acceptance of technology. (e.g. the theory of reasoned action (tra) (ajzen and fishbein, 1980), the theory of planned behavior (tpb) (ajzen, 1991), the technology acceptance model (tam) (davis, 1986), and the unified theory of acceptance and use of technology (venkatesh et al, 2003)). these theories contain a number of similar constructs which are shown in table 1a. an explanation of the relationships between the constructs of each model is shown in table 1b. tra (ajzen and fishbein, 1980) tpb (ajzen, 1985) tam (davis et al, 1989) utaut (venkatesh et al, 2003) perceived usefulness “the prospective user’s subjective probability that using a specific application will increase his or her job performance within an organizational context” (p. 985). performance expectancy “the degree to which an individual believes that using the system will help him or her to attain gains in job performance” [p. 447]. this construct incorporates perceived usefulness, “extrinsic motivation” from motivational theory, “relative advantage” from innovation diffusion theory, and “outcome expectations” from social cognitive theory. perceived ease of effort expectancy “the degree of 6 australasian journal of information systems volume 14 number 1 nov 2006 use “the degree to which the prospective user expects the target system to be free of effort” (davis et. al., 1989, p. 985). ease associated with the use of the system” [p. 450]. this construct incorporates perceived ease of use, “complexity” from the model of personal computing utilization and innovation diffusion theory. attitude “an individual’s positive or negative feelings of performing the target behavior” (p. 216) attitude – same as tra attitude – same as tra and tpb in the original tam but later dropped. subjective norm “the person’s perception that most people who are important to him think he or she should or should not perform the behavior in question” (p. 216) subjective norm – same as tra social influence, “the degree to which an individual perceives that important others believe he or she should use the system” [p. 451]. this construct incorporates subjective norms, “social factors” from the model of personal computing utilization, and “observability” from innovation diffusion theory. perceived behavioral control “reflects beliefs regarding access to the resources and opportunities needed to perform a behavior, or alternatively, to the intention and external factors that may impede performance of the behavior” ( p.34). facilitating conditions “the degree to which an individual believes that an organizational and technical infrastructure exists to support the use of the system” [p. 453]. this construct incorporates perceived behavioral control, “compatibility” from innovation diffusion theory, and “facilitating conditions” from the model of personal computing utilization. behavioral intention – the strength of an individual’s intention to perform a specified behavior. behavioral intention – same as tra behavioral intention to use – same as behavioral intention in tra and tpb table 1a – technology acceptance constructs 7 australasian journal of information systems volume 14 number 1 nov 2006 model relationships: attitude and subjective norms influence behavioral intention model relationships: attitude, subjective norms, and perceived behavioral control influence behavioral intention model relationships perceived usefulness, and perceived ease of use influence behavioral intention to use. perceived ease of use also influences perceived usefulness. model relationships: effort expectancy, performance expectancy, social influence, and facilitating conditions influence table 1b – technology acceptance relationships of these models, davis's tam is arguably the most widely accepted yet parsimonious model of the phenomenon. it has been shown that it has equal predictive power to tra and tpb, yet it is the most economical of the three (taylor and todd, 1995, mathieson, 1991). venkatesh et al’s. (2003) utaut incorporates the constructs from tam and has received widespread support but was unfortunately not published at the time of the data collection. the model tested in this paper examines four constructs that have been well accepted in previous technology acceptance studies (perceived usefulness, perceived ease of use, subjective norms, and behavioral intention to use). all of these constructs, however, are incorporated into the constructs of utaut. although attitude was initially included in tam (davis, 1986, davis, et. al., 1989) recent conceptualizations have tended to omit this construct. in keeping with past research we posit that perceived usefulness, perceived ease of use, and subjective norms influence behavioral intention to use and that perceived ease of use influences perceived usefulness. the hypotheses relating to perceived ease of use and perceived usefulness have been studied by a number of tam researchers (please see venkatesh, et. al., 2003 for a review). however, tam in its initial form did not include subjective norms which are a major variable in the theory of reasoned action (tra) (ajzen and fishbein, 1980), the theory of planned behavior (tpb) (ajzen, 1991), tam2 (venkatesh and davis, 2000), and utaut (venkatesh et al., 2003). subjective norms have been examined in a number of studies focusing on the acceptance of technology (karahanna and straub, 1999; taylor and todd, 1995; thompson et al., 1991; venkatesh and davis, 2000; venkatesh and morris, 2000; venkatesh et al., 2003). hypotheses are as follows and the research model is shown in figure 1. • hypothesis 1: perceived usefulness positively influences an individual’s behavioral intention to use a technology. • hypothesis 2: perceived ease of use positively influences an individual’s behavioral intention to use a technology. • hypothesis 3: subjective norms positively influence an individual’s behavioral intention to use a technology. • hypothesis 4: perceived ease of use positively influences an individual’s perceptions of usefulness of a technology. 8 australasian journal of information systems volume 14 number 1 nov 2006 perceived usefulness perceived ease of use subjective norms behavioural intention to use figure 1 – research model in addition to a basic technology acceptance model this article intends to examine its impact across two cultures, china and the us. hence a brief review of culture and how it is measured along with some specifics as to how these issues apply in an asian context follow. national culture researchers have defined culture in different ways. most definitions boil down to culture being a common set of characteristics shared by a group of people. in anthropology, for instance, “culture consists of patterned ways of thinking, feeling and reacting, acquired and transmitted mainly by symbols, constituting the distinctive achievement of human groups, including their embodiments in artifacts; the essential core of culture consists of traditional (i.e., historically derived and selected) ideas and especially their attached values” (kroeber and kluckhohn, 1952, p. 86). hofstede defined culture as “the collective programming of the mind which distinguishes the members of one human group from another” (1980, p. 260). for a more through review of cultural definitions please see straub et al. (2002). in this study we employ hofstede's (1980) definition since it has been widely cited and used in many cross-cultural studies. in addition to an overall definition of culture research in this area is often done though the use of a set of bounded cultural constructs, such as dimensions. the use of dimensions allows researchers to better measure and examine culture. hofstede (1980) proposed a widely cited set of national cultural dimensions. he defined the following four dimensions: individualism/collectivism, power distance, uncertainty avoidance and masculinity/femininity. these four dimensions are shown in table 2 below. individualism/collectivism “societies in which the interests of the individual prevail over the interests of the group” versus “societies in which the interests of the group prevail over the interest of the individual” (hofstede, 1991, p. 50). power distance “the extent to which the less powerful members of institutions and organizations within a country expect and accept that power is distributed unequally (hofstede, 2001, p. 98). uncertainty avoidance “the extent to which the members of a culture feel threatened by 9 australasian journal of information systems volume 14 number 1 nov 2006 uncertain or unknown situations” (hofstede 1991, p. 113). masculinity/femininity “masculinity stands for a society in which social gender roles are clearly distinct… femininity stands for a society in which social gender roles overlap (hofstede, 2001, p. 297). table 2 – hofstede’s dimensions of culture this study examined the above cultural dimensions on the research model across two cultures, that of china and the us. in particular, the model was tested separately in both cultures. participant’s scores on each of the cultural dimensions were then utilized to explain differences across the results. unlike many previous studies that simply reused hofstede’s country scores that were initially published in the 1980s, and were based on research done considerably earlier, the participants of this study were surveyed as to their perceptions of national culture. technology acceptance and the asian culture the previous section discussed some of the dominant definitions and measures of culture that are used in cross-cultural research. this section will review the asian cross-cultural research that has been done relating to the area of technology acceptance. unfortunately there has been little direct research linking the field of technology acceptance and national culture. what little research there is will be discussed followed by a review of asian cross-cultural research in the in the broader area of information technology. the sole major study of technology acceptance in asia was by rose and straub (1998) who examined technology acceptance in five arab cultures, three of which were technically in asia (jordan, saudi arabia, and lebanon), the other two (egypt and the sudan) in africa. the authors examined the following tam relationships: perceptions of ease of use and perceptions of usefulness were posited to influence actual usage and perceptions of usefulness to mediate the effect of perceptions of ease of use on actual usage. the authors found support for all three hypotheses and suggested that tam is supported in the arab world. their findings were consistent with the majority of tam findings in the us. no cultural dimensions were measured and no direct comparisons were made between different cultures. in addition to the above study a number of studies that were tangentially related to the area of technology acceptance are shown in appendix i. the appendix shows the authors of the study, the year, and the countries/regions studied. method data were collected using student subjects at a large university in the southeast of the us. foreign students from china were contacted at an international student orientation during the first week of their first semester in attendance. a random sample of us nationals was also contacted. this method ensured sufficient variance in the dimensions of culture and that the perceptions of the foreign nationals were collected soon after they left their home country and before having spent any considerable amount of time in the us. data were collected using surveys. validated scales from davis (1989) were used to measure perceived ease of use, perceived usefulness, and behavioral intention to use. individualism/collectivism, uncertainty avoidance, power distance, and masculinity/femininity were 10 australasian journal of information systems volume 14 number 1 nov 2006 assessed using scales derived from hofstede (1980) and dorfman and howell (1988). subjective norms were measured using tra guidelines (fishbein and ajzen, 1975) measuring both normative beliefs, i.e., an individual’s beliefs of what relevant others expect the him or her to do with respect to using pcs, as well as his or her motivation to comply with these. the referent groups used to measure subjective norms were family, friends, and classmates. as per tra, normative beliefs for each referent were multiplied with the respondent’s motivation to comply with those referents to derive a single score for each referent group. behavioral intention to use focused on the participant’s intention to use personal computers (a technology that was common to all participants). the survey instrument went through two rounds of pilot testing. the technology acceptance and subjective norms constructs were found to have acceptable psychometric properties. the culture constructs were initially found to be somewhat problematic in terms of cross-loadings and low reliabilities. these items were refined for the final instrument. appendix ii shows the final items used in the study. a total of 101 individuals participated in the study (29 chinese and 82 americans). due to the nature of the data collection a response rate was unable to be precisely calculated. however, this study was part of a larger research program with a response rate of 24%. it should be noted that data was collected on the four constructs of the research model (perceptions of usefulness, perceptions of ease of use, subjective norms, and behavioral intention to use) as well as on the four dimensions of culture (individualism/collectivism, power distance, uncertainty avoidance, and masculinity/femininity). the cultural data will be used to provide an explanation for differences observed across cultures but will not be directly incorporated into the research model due to sample size considerations. data analysis validity and reliability assessment the total sample was split into two groups (chinese and us) and analysed separately. pls was used to assess the discriminant validity and internal consistency (reliability) of the constructs in the research model. although only the technology acceptance constructs of perceived usefulness, perceived ease of use, subjective norms, and behavioral intention to use were directly incorporated into the research model reliability and validity were also established for the cultural dimensions as they were utilized to provide an explanation of the results. confirmatory factor analysis results for the final items are presented in tables 3a (chinese) and 3b (us). item loadings greater than .70 are considered acceptable (fornell and larker, 1981). as can be seen the scales mostly meet this guideline with the exception of pu1 (chinese sample) and sn1 (us sample). since these items did not cross-load, were above the cut off in one of the two studies, and were fairly close to the .70 cut-off it was decided to retain them in the analysis. ic pd ua mf pu peou sn biu ic1 .807 -.376 .485 .048 .091 .387 .191 .125 ic2 .895 -.414 .220 .187 .189 .271 .452 .097 pd1 -.420 .820 -.350 -.026 -.157 -.481 -.129 -.084 pd2 -.334 .807 -.283 -.146 -.099 -.298 -.402 -.232 ua1 .390 -.390 1.000 .269 .315 .406 .358 .219 11 australasian journal of information systems volume 14 number 1 nov 2006 mf1 .109 -.119 .237 .976 .184 -.016 .274 .171 mf2 .220 -.033 .284 .781 .039 -.020 .110 .004 pu1 .187 .075 .231 .158 .678 .341 .284 .350 pu2 .299 -.086 .331 .176 .737 .465 .276 .268 pu3 .047 -.102 .176 .116 .838 .516 .477 .441 pu4 .031 -.309 .229 .050 .744 .515 .456 .568 peou1 .205 -.225 .294 -.133 .619 .756 .394 .588 peou2 .256 -.369 .392 .016 .453 .823 .452 .536 peou3 .333 -.592 .267 .127 .415 .744 .412 .356 peou4 .416 -.402 .351 -.040 .489 .895 .442 .565 sn1 .382 -.320 .366 .104 .407 .454 .890 .540 sn2 .294 -.071 .227 .230 .583 .405 .836 .611 sn3 .333 -.411 .315 .292 .324 .477 .822 .628 biu1 .157 -.283 .154 .083 .371 .602 .596 .890 biu2 .076 -.072 .236 .162 .618 .557 .663 .908 table 3a pls confirmatory factor analysis – chinese sample ic pd ua mf pu peou sn biu ic1 .954 .271 .065 -.169 .013 -.165 -.134 -.093 ic2 .951 .267 .035 -.249 -.089 -.116 -.145 -.072 pd1 .162 .960 .019 -.259 -.356 -.045 .119 -.311 pd2 .465 .779 .043 -.191 -.151 -.056 .061 -.239 ua1 .053 .029 1.000 -.332 -.101 -.141 .111 -.147 mf1 .085 .170 .326 -.760 -.138 .052 .057 -.012 mf2 .248 .263 .272 -.938 -.263 -.050 .043 -.177 pu1 -.146 -.305 -.138 .239 .774 .168 -.353 .445 pu2 .049 -.147 -.132 .230 .788 .342 -.156 .594 pu3 -.119 -.435 -.048 .286 .794 .107 -.180 .318 pu4 .061 -.185 .012 .041 .803 .279 -.262 .490 peou1 -.181 -.043 -.105 -.049 .162 .821 .171 .182 peou2 -.122 .025 -.134 -.019 .317 .927 -.025 .365 peou3 .000 -.019 -.175 .032 .240 .883 .042 .319 peou4 -.217 -.148 -.084 .076 .287 .872 -.008 .342 sn1 .019 -.100 -.060 .089 .067 -.082 -.695 .050 sn2 .176 -.077 .021 -.048 .243 .074 -.764 .151 sn3 .109 -.093 -.169 .088 .286 -.077 -.849 .276 12 australasian journal of information systems volume 14 number 1 nov 2006 biu1 -.091 -.301 -.169 .069 .559 .390 -.243 .917 biu2 -.064 -.273 -.091 .186 .521 .242 -.204 .886 table 3b – pls confirmatory factor analysis – us sample to assess discriminant validity two additional criteria need to be met (chin, 1988). first, indicators should load more strongly on their corresponding construct than on other constructs in the model. looking at table 3a (chinese sample) and table 3b (us sample) we can see that the loadings of items on their respective constructs are higher than the cross-loadings of the items on other constructs. second, the square root of the average variance extracted (ave) (leading diagonal in tables 4a and 4b) should be larger than the inter-construct correlations (implying that all constructs share more variance with their indicators than with other constructs). since both criteria were met we concluded that the constructs exhibited adequate discriminant validity across both samples. finally, to assess the reliability of the scales internal composite reliabilities (icrs) were calculated. as shown in tables 4a and 4b, these range from .797 to .930 and are above the .70 recommended level (fornell and larcker, 1981). taken together these results suggest that the scales used in the study exhibited adequate psychometric properties. it should be noted that due to issues of crossloading uncertainty avoidance had to be measured with a single item scale and hence no measure of discriminant validity or reliability can be calculated. constructs composite reliability ic pd ua mf pu peou sn biu ic .841 .852 pd .797 -.464 .814 ua na .390 -.390 1.00 mf .876 .149 -.104 .269 .884 pu .838 .171 -.158 .315 .158 .752 peou .881 .374 -.481 .406 -.018 .618 .807 sn .886 .396 -.324 .358 .250 .510 .527 .850 biu .894 .128 -.193 .219 .138 .556 .643 .701 .900 diagonal elements in the “correlation of constructs” matrix are the square root of the average variance extracted. for adequate discriminant validity, diagonal elements should be greater than corresponding off-diagonal elements. composite reliability = ρc = (σλi)2/[(σλi)2+σivar(εi)], where λi is the component loading to an indicator and var(εi) = 1λi 2 table 4a inter-construct correlations – chinese sample constructs composite reliability ic pd ua mf pu peou sn bi u ic .952 .953 pd .865 .282 .874 ua na .053 .029 1.00 mf .841 -.218 -.263 -.332 .853 pu .869 -.039 -.324 -.101 .251 .790 peou .930 -.148 -.053 -.141 .016 .297 .876 sn .814 -.147 .111 .111 -.054 -.296 .036 .772 biu .897 -.087 -.319 -.147 .136 .600 .356 -.249 .902 13 australasian journal of information systems volume 14 number 1 nov 2006 diagonal elements in the “correlation of constructs” matrix are the square root of the average variance extracted. for adequate discriminant validity, diagonal elements should be greater than corresponding off-diagonal elements. composite reliability = ρc = (σλi)2/[(σλi)2+σivar(εi)], where λi is the component loading to an indicator and var(εi) = 1λi 2 table 4b – inter-construct correlations – us sample table 5 presents the descriptive statistics of both the chinese and us samples. the demographics are, in general, consistent with those of the university. the average age of university students is fairly similar to that of the sample since it included students in the bachelor's, master's, and doctoral programs. the number of years of computer use did vary when comparing the chinese sample to that of the us. however, it was not surprising that students from the us had considerably more computer experience than their chinese counterparts. the gender split was approximately that of the university as a whole (44.4% male, 55.6% female). it should also be noted that the chinese had a considerably wider variability in years of computer use. finally, it should be observed that both samples came from a well-distributed cross-section of majors and colleges. china us age mean = 27.03 years standard deviation = 4.63 years mean = 25.49 years standard deviation = 7.65 years computer experience mean = 7.10 years standard deviation = 3.46 years mean = 10.44 years standard deviation = 5.32 years gender 48% male, 51% female 42% male, 58% female major 15 majors across the university 22 majors across the university table 5 descriptive statistics model testing pls was also used to test the research model. pls is a latent structural equation modeling technique that uses a component-based approach to estimation. because of this approach, it places minimal demands on sample size and residual distributions (lohmoller, 1989; fornell and bookstein, 1982; chin, 1988). loadings of measures of each construct can be interpreted as loadings in a principal component factor analysis. path coefficients are interpreted as standardized beta coefficients in a regression analysis. the path coefficients and explained variances for the model are shown in figures 2a and 2b. the significance of the path coefficients was determined using the t-statistic calculated with the bootstrapping technique. all constructs were modeled as reflective. the final model is shown in figures 2a and 2b. 14 australasian journal of information systems volume 14 number 1 nov 2006 .630** .105 .471** .337 behavioural intention to use (.610) perceived usefulness (.397) perceived ease of use subjective norms ** = significant at .005 .314* .517** .110 .201* behavioural intention to use (.419) perceived usefulness (.099) perceived ease of use subjective norms ** = significant at .005 * = significant at .05 figure 2a – pls results – chinese sample figure 2b – results – us sample since a holistic approach to scale validation was taken, the loadings of the items are those presented in tables 4a and 4b. because pls breaks models down into segments, the sample size needs to meet the demands of the most complex regression in the model. accordingly, the sample size needs to be either seven-to-ten times the greatest number of indicators in any construct, or seven-to-ten times the greatest number of antecedents to a construct (chin, 1998). with 29 chinese and 82 us respondents and a maximum of 4 indicators per construct and 3 antecedents per construct the chinese and us samples meets both the guidelines for using pls (4*7=28). results as can be seen from figures 2a and 2b a number of relationships were supported. although generally significant, the tam model showed some interesting differences when compared across the two samples. in both china and the us perceptions of ease of use significantly affected perceptions of usefulness and behavioral intention to use. however, in the chinese sample the relationship between perceptions of usefulness and behavioral intention to use was not significant while the relationship between subjective norms and behavioral intention to use was. the opposite held true with respect to the us sample. a more than adequate amount of variance in the dependent variable (behavioral intention to use) was explained by the determinants (r2 = .610 for the chinese sample and r2 = .419 for the us sample). perceived ease of use also explained a reasonable amount of variance of perceived usefulness. as noted earlier this article will utilize the data collected on the participant’s cultural values to provide an explanation for the differences observed across to two samples. in addition to establishing validity and reliability for the cultural dimensions scores were also calculated (by summing and averaging) for each dimension in both samples. these scores are shown below in table 6. additionally t-tests were conducted to determine if there was a significant difference between the two samples for each of the dimensions. as can also be seen in table 6 there were no significant differences between the scores of the chinese and us respondents in terms of 15 australasian journal of information systems volume 14 number 1 nov 2006 uncertainty avoidance and power distance. however, the chinese were significantly more collectivistic and masculine than their us counterparts. china us p-value significance ic (1=ind, 7=coll) 2.74.00 5 0.0001 significant pd (1=low, 7=high) ant 2.68 2.63 0.8180 non-signific ua (1=low, 7=high) 3.36 3.44 0.8452 non-significant mf (1=fem, 7=mas) 3.91 2.74 0.001 significant table 6 – culture scores & t-test results hese results would, to some extent help explain the differences across the two models. one of the he second major difference in the two models deals with the influence of perceived ease of use on final difference between the two models concerns the relationship between perceived usefulness t major differences in the models was in the relationship between subjective norms and behavioral intention to use. in the chinese sample this relationship was significant, while in the us sample it was not. since the chinese are more collectivistic it is not surprising that the influence of salient others such as family friends and classmates would be stronger. t behavioral intention to use. this relationship was significant only in the us sample. it can be argued that cultures that are more feminine will be more concerned with the ease of use of a technology. as defined by davis, perceived ease of use is "the degree to which the prospective user expects the target system to be free of effort" (davis et. al., 1989, p. 985). the idea of effort-free use or effortlessness in a technology is less concerned with instrumental goals. if a technology requires less effort to use it might be more widely used because it is pleasant to work with and results in less frustration. masculinity/femininity values concern the extent of emphasis on work goals and assertiveness, as opposed to personal goals and nurturing. hofstede (1984) sees personal goals as having a greater emphasis on cooperation, employment security, a friendly atmosphere, an environment where work is less central, and where achievement is defined in terms of human contacts. hofstede (1984) defines work goals to include an emphasis on earnings, recognition, advancement, challenge, greater work centrality, and achievement defined in terms of wealth. cultures that are less masculine might be more concerned with perceived ease of use which places less emphasis on instrumental goals and more on quality of life. a and behavioral intention to use. in the chinese sample this relationship was non-significant while in the us sample it was significant. given the argument in the previous paragraph the opposite would hold true. in a more masculine culture, such as that of china, it would be expected that perceptions of a technology’s usefulness would be more significant than in a less masculine culture, such as that of the us. unfortunately the exact opposite occurred in this study. these contrary results, while interesting, are not readily explained by differences in cultural values. it is possible that some other, unmeasured, construct is impacting this relationship such that it overwhelms the effect of masculinity/femininity. potential candidates for this inhibitor could be computer self-efficacy, quality of life, or perceived behavioral control factors such as accessibility or cost. one other possible explanation is that individual beliefs such as perceived usefulness and perceived ease of use are less important in a collectivistic culture. additional research is needed to explore this issue. 16 australasian journal of information systems volume 14 number 1 nov 2006 limitations and future directions although the findings of this study, and our conclusion that some of the differences in the acceptance of technology across cultures can be attributed to differences in individualism/collectivism and masculinity/femininity in the samples, represent an increase in our knowledge the study does not directly measure the influence of the various dimensions of culture on perceptions of usefulness, perceptions of ease of use, and subjective norms. given the size of our sample and the number of paths necessary results from such a model could be potentially spurious as this would be beyond the heuristic of the analysis method. however, in the interest of exploratory research results from this model are presented as appendix iii, but care should be taken when examining the paths and attributing significance to the sample. future studies should be conducted with a larger sample to allow direct measurement of the impact of cultural values on technology acceptance. sample size can also be seen as an indicator of another limitation. with only twenty-nine chinese participants it is possible that this small sample could be responsible for some of the non-significant relationships observed. however, this does not detract form the significant relationships that were found. the use of student subjects can sometimes be problematic in terms of students not being representative of their populations due to higher levels of education, and in the case of foreign students studying in the us, having a more international outlook. it should be noted, however, that the subjects of this study came from a wide variety of programs (undergraduate, graduate, and doctoral) and academic backgrounds (15+ major programs of study). this increase in variability combined with research that has shown that students and workers have essentially the same values and beliefs (voich, 1995) should alleviate some of the concerns regarding student subjects. implications for research and practice as noted previously there have been very few studies that examined both technology acceptance and national culture and hence very little work has been done to integrate these two topics. this study examined these two areas and focused on the particular culture of china. this integration is particularly relevant given the international proliferation of the internet and other information technologies as well as the rise in the globalization of business in general and the increase of multinational teams operating at a single site as well as dispersed across several time zones, countries, or continents. tam has generally been seen as a parsimonious and well-accepted model that predicts acceptance of an information technology. tam has mostly been tested (as well as developed) within the us. this study added subjective norms to tam which enhances its use in settings involving cultures other than that of the us, where the influence of salient others might be more pronounced, and allows for the assessment of the technology acceptance model across multiple cultures. results from this study also have some direct managerial implications. in general this study has shown that national culture affects the acceptance and use of information technologies. although managers with experience in multiple national cultures may understand this implicitly, all managers should recognize the cultural aspects of technology acceptance. this awareness may affect how a 17 australasian journal of information systems volume 14 number 1 nov 2006 manager chooses to handle the planning, design, introduction, and implementation of new technologies. the support of peers with different national backgrounds and the reactions from subordinates from other cultures to new technologies can vary. cultural awareness should be part of the training process for it managers and planners. reactions to it implementation can have cultural variations. resistance to a planned technology implementation may signal some cultural dimension that needs to be addressed. strategies, that take culture into account, can be developed to overcome resistance and to learn from the different reactions. it may also be beneficial to consider different implementation strategies in different cultures. for instance, group-based training in the technology and round-table discussions might be more appropriate in a collectivistic culture while individual on-line training could work better in individualistic cultures. references ajzen i. & fishbein, m. (1980) understanding attitudes and predicting social behavior, prentice hall, englewood cliffs, new jersey. ajzen, i., (1985) “from intention to actions: a theory of planned behavior”, in action control: from cognition to behavior, j.kuhl and j. beckman (eds.), springer-verlag, new york, new york, pp. 11-39. ajzen, i. (1991) “the theory of planned behavior”, organizational behavior and human decision processes, vol 50, pp. 179-211. barrett, m. & walsham, g. (1995) “managing it for business innovation: issues of culture, learning, and leadership in a jamaica insurance company.” journal of global information management, vol 3, pp. 25-33. burn, j. (1995) “the new cultural revolution: the impact of edi on asia”, journal of global information management, vol 3, pp. 16-23. burn, j., tye, e., & ma, l. (1995) “paradigm shift – cultural implications for development of is professionals”, journal of global information management vol 3 no 2, pp. 18-29. burn, j., saxena, k., ma, l., & cheung, h. (1993) “critical issues of is management in hong kong: a cultural comparison”, journal of global information management, vol 1 no 4, pp. 28-37. chau, p. y. k., cole, m., massey, a.p., montoya-weiss, m., & o’keefe, r. (2002) “cultural differences in the online behavior of consumers.” communications of the acm, vol 45 no 10, pp. 138-143. chin, w. (1988) “the partial least squares approach to structural equation modeling”, in modern methods for business research, g.a. marcoulides (ed.), lawrence erlbaum associates, mahwah, new jersey, 1988. chin, w. (1998) "issues and opinion on structural equation modeling", mis quarterly, vol 22 no 1, pp vii-xvi. chung, k. & adams, c. (1997) “a study on the characteristics of group decision making behavior: cultural difference perspective of korea vs. u.s.”, journal of global information management, vol 5 no 3, pp. 18-29. 18 australasian journal of information systems volume 14 number 1 nov 2006 davis, f. (1989) “perceived usefulness, perceived ease of use, and user acceptance of information technology”, mis quarterly, vol 13 no 3, pp. 319-339. davis, f. (1986) a technology acceptance model for empirically testing new end-user information systems: theory and results, doctoral dissertation, massachusetts institute of technology, cambridge, massachusetts. davis, f., bagozzi, r. & warshaw, p. (1989) “user acceptance of computer technology: a comparison of two theoretical models”, management science, vol 35 no 8, pp. 982-1003. dorfman, p. w. and j. howell (1988) “dimensions of national culture and effective leadership patterns: hofstede revisited”, advances in international comparative management. e. g. mcgoun (ed.), jai press, greenwich, connecticut, pp. 127-149. fishbein, m. & ajzen, i. (1975) belief, attitude, intention, and behavior: an introduction, addison-wesley, reading, massachusetts. foley-curley, k., meyer, m.h., & sorenson, e.v. (1996) “a comparison of us, japanese, and european software development practices and processes”, journal of global information management, vol 4 no 3, pp. 18-26. fornell, c. & bookstein, f. (1982) “two structural equation models: lisrel and pls applied to consumer exit-voice theory”, journal of marketing research, vol19, pp. 440-452. fornell, c. & larcker, d.f. (1981) “evaluating structural equations models with unobservable variables and measurement error”, journal of marketing research, vol 18 no 1, pp. 39-50. gallivan, m & srite, m (2005) “information technology and culture: identifying fragmentary and holistic perspectives of culture”, information & organization, vol 15 no 4, pp. 295-338. harrison, w. l. & farn, c.k. (1990) “a comparison of information management issues in the united states of america and the republic of china”, information & management, vol 18, pp. 177-188. hill, c., loch, k., straub, d., & el-sheshai, k. (1998) “a qualitative assessment of arab culture and information technology transfer”, journal of global information management, vol 6 no 3, pp. 29-38. hofstede, g. (1991) cultures and organizations: software of the mind. mcgraw-hill, london, united kingdom. hofstede, g. (2001) culture's consequences: comparing values, behaviors, institutions, and organizations across nation. sage publications, thousand oaks, california. hofstede, g. (1984) cultures consequences, sage publications, newbury park, california, 1984. hofstede, g. (1980) culture’s consequences: international differences in work-related values, sage publications, beverly hills, california. karahanna, e., evaristo, r. & srite, m. (2005) "levels of culture and individual behavior: an integrated perspective", journal of global information management, vol 13 no 2, pp. 1-20. karahanna, e. & d. straub (1999) "the psychological origins of perceived usefulness and ease of use", information & management, vol 35 no 4, pp. 237-250. kroeber, a. & kluckhohn, f.(1952) “culture: a critical review of concepts and definitions”, peabody papers, vol 27 no 1. 19 australasian journal of information systems volume 14 number 1 nov 2006 lally, l. (1994) “the impact of environment on information infrastructure enhancement: a comparative study of singapore, france, and the united states”, journal of global information management, vol 2, pp. 5-12. laudon, k.c. & laudon, j.p. (2006) management information systems: managing the digital firm, pearson/prentice hall, upper saddle river, new jersey. loch, k., straub, d., & kamel, s. (2003) “diffusing the internet in the arab world: the role of social norms and technological culturation”, ieee transactions on engineering management, vol 50 no 1, pp. 45-65. lohmoller, j. (1989) latent variable path modeling with partial least squares, physica-verlag, heidelberg, west germany, 1989. mathieson, k. (1991) “predicting user intentions: comparing the technology acceptance model with the theory of planned behavior”, information systems research, vol 2 no 3, pp. 173191. mcleod, r., kim, c., sounder, c., jones, j., schreel, c & estrada, m. (1997). “information management as perceived by cios in three pacific rim countries”, journal of global information management, vol 5 no 3, pp. 5-16. milberg, s., burke, s., smith, j., & kallman, e. (1995) “values, personal information privacy, and regulatory approaches”, communications of the acm, vol 38 no 12, pp. 65-74. moores, t. t. (1996) “key issues in the management of information systems: a hong kong perspective”, information & management, vol 30 no 6, pp. 301-307. palvia, s. & hunter, m.g. (1996) “information systems development: a conceptual model and a comparison of methods used in singapore, usa, and europe”, journal of global information management, vol 4 no 3, pp. 5-17. porter, m. and miller, v. (1985) how information gives you a competitive advantage”, harvard business review, july-august, pp. 149-160. rose, g. and straub, d. (1998) “predicting general it use: applying tam to the arab world”, journal of global information management, vol 6 no 3, pp. 39-46. slaughter, s. & ang, s. (1995) “information systems employment structures in the usa and singapore: a cross-cultural comparison”, information technology, and people, vol 8, pp. 17-36. straub, d. (1994) “the effect of culture on it diffusion: e-mail and fax in japan and the us”, information systems research, vol 5 no 1, pp. 23-47. straub, d., m. keil, & brenner, w. (1997) “testing the technology acceptance model across cultures: a three country study”, information & management, vol 33 no 1, pp. 1-11. straub, d., loch, k., & hill, c. (2001) “transfer of information technology to the arab world: a test of cultural influence modeling” journal of global information management, vol 9 no 4, pp. 6-28. straub, d., loch, k., karahanna, e., evaristo, j.r., & srite, m. (2002) “toward a theory based measurement of culture”, the journal of global information management, vol 10 no 1, pp. 13-23. 20 australasian journal of information systems volume 14 number 1 nov 2006 tai, l. a. & phelps, r. (2000) “ceo and cio perceptions of information systems strategy: evidence from hong kong”, european journal of information systems, vol 9 no 3, pp. 163172. tan, b. c. y., smith, h.j., keil, m., & montealegre, r. (2003) “reporting bad news about software projects: impact of organizational climate and information asymmetry in an individualistic and a collectivistic culture”, ieee transactions on engineering management, vol 50 no 1, pp. 64-77. tan, b., wei, k., watson, r.t., clapper, d.l., mclean, e.r. (1998) “computer mediated communication and majority influence: assessing the impact in an individualistic and a collectivistic culture”, management science, vol 44 no 9, pp. 1263-1278. tan, b., wei, k., watson, r., & walczuch, r. (1998) “reducing status effects with computer mediated communication: evidence from two distinct national cultures”, the journal of management information systems, vol 15 no 1, pp. 119-141. taylor, s. & todd, p. (1995) “assessing it usage: the role of prior experience”, mis quarterly, vol 19 no 4, pp. 561-571. thompson, r. l., higgins, c.a., & howell, j.m. (1991) "personal computing: toward a conceptual model of utilization", mis quarterly, vol 15 no 1, pp. 124-143. thanasankit, t. (2002) “requirements engineering: exploring the influence of power and thai values”, european journal of information systems, vol 11, pp. 128-141. venkatesh, v. & davis, f. (2000) "a theoretical extension of the technology acceptance model: four longitudinal field studies", management science, vol 46 no 2, pp. 186-204. venkatesh, v. & morris, m.g. (2000) "why don't men ever stop and ask for directions? gender, social influence, and their role in technology acceptance and usage behavior", mis quarterly, vol 24 no 1, pp. 115-140. venkatesh, v., morris, m. g., davis g. b., & davis, f. d. (2003) “user acceptance of information technology: toward a unified view”, mis quarterly, vol 27 no 3, pp. 425-478. voich, d. (1995) comparative empirical analysis of cultural values and perceptions of economy issues, prager, westport, connecticut. walsham, g. (2002) “cross-cultural software production and use: a structurational analysis”, mis quarterly, vol 26 no 4, pp. 359-380. wan, h. a. & m. t. lu (1997) “an analysis of chinese laws against computer crimes.” journal of global information management, vol 5 no 2, pp. 16-21. wang, p. & m. b. khan (1994) “mis skills of is graduates in the republic of china”, journal of global information management, vol 2 no 3, pp. 29-40. watson, r. t. & j. c. brancheau (1991) “key issues in information systems management: an international perspective”, information & management, vol 20, pp. 213-223. watson, r. t., ho, t.h., & raman, k.s. (1994) “culture: a fourth dimension of group support systems research”, communications of the acm, vol 37 no 10, pp. 44-55. watson, r. t., kelley, g.g., galliers, r.d., & branchaeu, j.c. (1997) “key issues in information systems management: an international perspective”, journal of management information systems, vol 13 no 4, pp. 91-115. 21 australasian journal of information systems volume 14 number 1 nov 2006 appendix i – cross-cultural it studies in asia adapted from gallivan and srite (2005) and karahanna, evaristo, and srite (2005)) authors year cultures barrett and walsham 1995 jamaica and india burn 1995 hong kong, singapore, and china burn, saxena, ma, and cheung 1993 hong kong, australia, europe, india, singapore, and us burn, tye, and ma 1995 hong kong and us chau, cole, massey, montoyaweiss, and o’keefe 2002 hong kong and us chung and adams 1997 korea and us foley-curley, meyer, and sorenson 1996 japan, europe, and us harrison and farn 1990 taiwan hill, loch, straub, and el-sheshai 1997 jordan, egypt, saudi arabia, lebanon, and the sudan lally 1994 singapore, france, and us loch, straub, and kamel 2003 arab business professionals mcleod, kim, sounder, jones, schreel, and estrada 1997 korea, mexico, and us milberg, burke, smith and kallman 1995 uk, canada, australia, france, japan, india, new zealand, thailand, and us moores 1996 hong kong, taiwan, and us palvia and hunter 1996 singapore, europe, and us rose and straub 1998 5 arab countries slaughter and ang 1995 singapore and us straub 1994 japan and us straub 1994 japan and us straub, keil, and brenner 1997 japan , switzerland, and us straub, loch, and hill 2001 5 arab countries tai and phelps 2000 hong kong and us tan, smith, keil, and montealegre 2003 singapore and us tan, wei, watson and walczuch 1998 singapore and us tan, wei, watson, clapper and mclean 1998 singapore and us thanasankit 2002 thailand walsham 2002 jamaica and india wan and lu 1997 china wang and khan 1994 china watson and brancheau 1991 australia, europe, singapore, and us watson, ho, and raman 1994 singapore and us watson, kelly, galliers, and brancheau 1997 australia, estonia, europe, hong kong, india, slovenia, taiwan, uk, and us 22 australasian journal of information systems volume 14 number 1 nov 2006 appendix ii – final items used perceived usefulness pu1 using computers enhances my productivity in college pu2 i find computers useful in my college activities pu3 using computers enhances my effectiveness in college pu4 using computers improves my performance in college perceived ease of use peou1 it is easy for me to become skillful in using computers peou2 i find computers easy to use peou3 i find it easy to get a computer to do what i want it to do peou4 learning to operate a computer is easy for me behavioral intention to use biu1 i intend to use a pc during my studies biu2 i intend to use a pc frequently during my studies subjective norms sn1 i believe that my relatives think i should use a computer * the opinions of my relatives are important to me sn2 i believe that my friends think i should use a computer * the opinions of my friends are important to me sn3 i believe that my classmates think i should use a computer * the opinions of my classmates are important to me masculinity/femininity mf1 there are some jobs in which a man can always do better than a woman mf2 it is more important for men to have a professional career than it is for women to have a professional career individualism/collectivism ic1 being accepted as a member of a group is more important than having autonomy and independence ic2 being accepted as a member of a group is more important than being independent power distance pd1 employees should not question their manager’s decisions uncertainty avoidance ua1 rules and regulations are important because they inform workers what the organization expects of them 23 australasian journal of information systems volume 14 number 1 nov 2006 appendix iii – cultural dimensions as direct antecedents to technology acceptance chinese sample .154 -.144 .180 .170 .262* .081 -.126 -.328~ .207 .249 .142 -.057 .709** .115 .473** .323~ individualism/ collectivism power distance uncertainty avoidance masculinity/ femininity behavioural intention to use (.603) perceived usefulness (.450) perceived ease of use (.319) subjective norms (.239) ** = significant at .005 * = significant at .05 ~ = significant at .10 us sample -.020 -.075 .200~ .111 -.157 .009 .160 -.026 -.292** -.202~ -.149~ .131 .229** .506** -.106 .210* individualism/ collectivism power distance uncertainty avoidance masculinity/ femininity behavioural intention to use (.405) perceived usefulness (.228) perceived ease of use (.044) subjective norms (.061) ** = significant at .005 * = significant at .05 ~ = significant at .10 24 australasian journal of information systems volume 14 number 1 nov 2006 the author mark srite is an associate professor in the sheldon b. lubar school of business at the university of wisconsin – milwaukee. he received his ph.d. from florida state university. his current research interests include cultural implications of information technology adoption, enterprise system implementation issues, and technologies fostering group decision making. his work has appeared, or is scheduled for publication, in mis quarterly, the journal of mis, decision support systems, information and organization, the journal of global information management, and elsewhere. 25 australasian journal of information systems volume 14 number 1 nov 2006 26 technology acceptance models national culture technology acceptance and the asian culture validity and reliability assessment results appendix i – cross-cultural it studies in asia appendix ii – final items used there are some jobs in which a man can always do better than a woman power distance chinese sample us sample the author microsoft word ajisfinal2003.doc australasian journal of information systems special issue 2003/2004 2 information systems development for the knowledge economy guest editorial this special issue covers a range of topics pertinent to information systems development in the context of rapidly changing social, economic, technical and industrial landscape. the development of information systems has paralleled technological developments and the deployment of that technology in all areas of society, including government, industry, community and in the home. more recently the convergence of information and communications technologies (ict) has presented a challenge for the isd profession in terms of accommodating mobility, access, interoperability, distribution, connectivity, media and the diversity and volume of information. is development, both as a professional and academic discipline, has responded to this challenge through methodologies, tools and theory development. as a practice-based discipline, isd has always promoted a close interaction between theory and practice and in particular has strived for reflective practice. this reflective practice has been influential in setting the isd agenda that has largely focused on the integration of people, processes and technology as well as their context. academically, the challenge has been to accommodate the broad range of concerns within a single disciplinary area. current debates within the is community are a testament that is as a discipline remains a work in progress. the context in which the academic and professional debates occur has been the changing social and economic environment characterised by new and emerging economic theories as well as social and industrial restructuring. some of the factors driving the shift away from goods and services to a knowledge-based economy are: • the re-conceptualisation of geography (globalisation); • an alternate temporal paradigm (24/7); • the dynamics of business relations (networks of alliances and partnerships); • the ubiquity of convergent technologies the organisational response to these factors has been a move towards global enterprises with very flat structures that, in principle, enable enterprises to react rapidly to changes in their operating environments. organisations operating in the knowledge economy require the capability to access and utilise large volumes of information without the constraint of media, geography or time. a critical factor of this capability is the speed with which organisations productively process such information. to date, technology has been effectively deployed to manage information that is encoded in procedures and information stores. however, in the changed environment there is an implied needs for organizations to innovate and learn, and to use ict to support these activities. the papers in this special issue represent a small sample of the broad isd agenda, ranging from theoretical explorations, to methodologies to case studies, to address organisational requirements in this changing environment. the papers, individually and collectively, attempt to address elements of the dilemma of isd in the knowledge economy. origins of this special issue this special issue originated from the 12 th international isd conference held in melbourne in 2003. twelve papers presented at this conference were identified by reviewers and associate editors as potential contributions for this special issue. the authors were invited to re-submit a revised journal paper that extended their original paper with more recent research to ensure at least 40% new content. all submitted papers were reviewed by at least two independent reviewers who had not previously reviewed the authors’ conference papers. based on these reviews, we selected 6 high quality papers. the accepted papers were revised in line with the reviewer comments and these changes were endorsed by the editors. australasian journal of information systems special issue 2003/2004 3 the papers we would like the reader to appreciate the range of topics covered in this special issue and the insight these papers provide into the complexities of isd. moreover these papers present a cross section of australian and european viewpoints that also includes a practitioner view. the first paper by helen hasan, “information systems development as a research method”, draws on design science to argue that isd can be a research method in certain situations. the paper extends action research to the design and use of information systems with the aim of understanding the development process as well as the organisational issues that the information system is designed to address. this research method is illustrated through a case study. this paper makes a important contribution to current debates about the nature of is as a discipline. paul taylor’s paper, “vernacularism in software design practice: does craftsmanship have a place in software engineering?” , highlights the tension between the professionalism of software engineering and the need for a crafts approach that relies on individual knowledge, skills and experience. this tensions is presented as an explanation for the failure of engineering based process models of isd. the paper goes on to argue that isd methods needs to incorporate the informality and creativity of the crafts in order to successfully design and development complex information systems. the next paper by lejla vrazalic, “evaluating distributed usability: the role of user interfaces in an activity system” presents a critique of existing approaches to usability and proposes an alternative approach. this approach does not focus on the artefact, the qualitative and quantitative attributes of a computer system, rather it addresses the activity system in which the artefact is embedded. this approach is presented as an alternate hci evaluation method. the fourth paper by jesper holck and niels jørgensen, “continuous integration and quality assurance: a case study of two open source projects”, raises important issues for isd in decentralised environment that has implications for globally distributed projects. significantly they use quality assurance as an analytical lens to study this process and highlight the potential contradictions between commercial imperatives for centralising control and the organisational imperatives to allow developers to work independently. jacek unold’s paper, “modeling the dynamics of an information system”, presents a mathematically defined model of the dynamics of an information system in order to understand the behaviour of its social context. the approach is illustrated through a model of the stock market. the final paper by witold abramowicz, tomasz kaczmarek and marek kowalkiewicz, “supporting topic map creation using data mining techniques”, highlights the utility of computer science methods for processing information within an information systems. the paper addresses the creation of sematic structures from unstructured information sources and the automation of this process. this is an issue of particular concern for new media , text repositories and of course the internet. acknowledgement we would like to thank all authors for an interesting and broad range of submissions. this special issue would not see the light without major effort from a dedicated team of reviewers. we would like to thank the group of 17 reviewers for their thorough and constructive comments that were the basis for improving the quality and readability of the original submissions. both the editors and authors appreciate their hard work. henry linger and julie fisher ajis special issue guest editors microsoft word ajis 11 1 final.doc ajis vol.11 no. 1 september 2003 55 investigating the partial relationship between testability and the dynamic range-to-domain ratio zuhoor a. al-khanjari1 martin r. woodward department of computer science, department of computer science, college of science, university of liverpool, sultan qaboos university, chadwick building, peach street, po box 36, al-khodh 123, liverpool l69 7zf, u.k. muscat, sultanate of oman tel: +44-151-794-3676 tel/fax: +968-515407 fax: +44-151-794-3715 email: zuhoor@squ.edu.om email: mrw@csc.liv.ac.uk abstract: the word ‘testability’ has been used variously in the software community to represent a number of different concepts such as how easy it is to test a program or how easy it is to achieve execution coverage of certain program components. voas and colleagues have used the word to capture a slightly different notion, namely the ease with which faults, if present in a program, can be revealed by the testing process. the significance of this concept is twofold. first, if it is possible to measure or estimate testability, it can guide the tester in deciding where to focus the testing effort. secondly, knowledge about what makes some programs more testable than others can guide the developer so that design-fortest features are built in to the software. the propagation, infection and execution (pie) analysis technique has been proposed as a way of estimating the voas notion of testability. unfortunately, estimating testability via the pie technique is a difficult and costly process. however, voas has suggested a link with the metric, domain-to-range ratio (drr). this paper reviews the various testability concepts and summarises the pie technique. a prototype tool developed by the authors to automate part of the pie analysis is described and a method is proposed for dynamically determining the inverse of the domain-to-range ratio. this inverse ratio can be considered more natural in some sense and the idea of calculating its value from program execution leads to the possibility of automating its determination. some experiments have been performed to investigate empirically whether there is a partial link between testability and this dynamic range-to-domain ratio (drdr). statistical tests have shown that for some programs and computational functions there is a strong relationship, but for others the relationship is weak. keywords: test and evaluation, testability, pie technique, msg-infection tool, domain-to-range ratio, empirical study. introduction software testing, which represents the last stage in the software development process, is currently a very active research field. the intent of testing is ideally to uncover faults and increase confidence in the correctness of the tested code. software testing is a very important approach in building reliable software systems. clearly therefore, it is desirable that testing should be both as easy and as effective as possible. the word ‘testability’ has been used in various ways to reflect this desire. one particular interpretation, due to voas et al. (1991), is to define testability as the ease with which faults, if present in a program, can be revealed by testing. under this definition, programs with high testability reveal their faults easily, while those with low testability may contain faults that are very difficult to expose. to measure testability, in the sense just described, the propagation, infection and execution (pie) analysis technique, has been proposed (voas et al., 1991). intriguing though this technique is, it is still a difficult and computationally expensive process. the details of which is given in section 3. to obtain an indication of the testability of a program early in the software development process (i.e. from a specification or a design) and without actually performing the pie analyses, voas and miller (1991) suggested use of a semantic metric, the domain-to-range ratio (drr): the ratio of the cardinality of the possible inputs to the cardinality of the possible outputs. the aim of this paper is to provide further investigation of the partial relationship between testability and drr. others have previously explored the possible link between testability and static program measures. the structure of the remainder of this paper is as follows. section 2 surveys various uses of the testability concept in the literature of the software community. section 3 provides a brief idea about 1 author to whom all correspondence should be addressed. ajis vol.11 no. 1 september 2003 56 mutation technique. it helps in understanding the concept of the pie technique. section 4 gives a brief summary of the pie analysis technique (voas, 1992b). section 5 considers the automation of testability measurement by introducing a prototype tool, msg-infection, which has been developed by the current authors for obtaining execution and infection estimates of c programs. a commercial tool called the pisces software testability analysis toolkit tm (friedman and voas, 1995), which has been developed by the reliable software technologies corporation is also mentioned. section 6 considers the drr metric in more detail and suggests that its inverse, the range-to-domain ratio (rdr), is more ‘natural’ in this context. in addition, a scheme for calculating rdr dynamically from a program execution is proposed, leading therefore to the dynamic range-to-domain ratio (drdr). section 7 describes a series of experiments, which have been conducted to investigate the possible partial relationship between testability and the proposed metric drdr. section 8 draws some conclusions from the experiments and summarises the contribution of this paper. the testability concept the word ‘testability’ has been used with a variety of meanings. one of the earliest and probably most influential studies concerning quality attributes of software was performed by boehm et al. (1976). in that study, testability was defined as the extent to which software “facilitates the establishment of verification criteria and supports evaluation of its performance”. boehm et al. linked testability with a number of lower level characteristics, namely: accountability, accessibility, communicativeness, self-descriptiveness and structuredness. among the specific features suggested as promoting testability were: “support of intermediate output and echo-checking of inputs”. in a later paper boehm (1984) discussed testability in terms of eliminating vagueness in the statement of software requirements or in the specification of software. the ieee definition, as given in the glossary of software engineering terminology, reflects both the aspects of testability discussed by boehm. the ieee defines testability as: (1) “the degree to which a system or component facilitates the establishment of test criteria and the performance of tests to determine whether those criteria have been met”; and (2) “the degree to which a requirement is stated in terms that permit establishment of test criteria and performance of tests to determine whether those criteria have been met” (ieee, 1990). interestingly in 1982 weyuker defined the negative of testability (weyuker, 1982). she focused on the absence of, or difficulties with, an output oracle. software was deemed to be non-testable if either of the following two conditions occurred: (1) no output oracle exists; and (2) it is theoretically possible, but too difficult in practice, to determine output correctness. in 1985 howden provided a more formal approach to the definition of testability than any previous approaches (howden, 1985). by further development of a test completeness criterion that he had identified in earlier work, howden defined testability as the property that “a finite set of tests can be specified that will determine if the program that implements a function contains one of a specified set of faults”. the quotation just given, summarising howden’s definition, is from an excellent survey on testing and verification by white (1987). in 1991 freedman discussed what he termed domain testability of software components (freedman, 1991). he introduced two properties of software called observability and controllability. observability is “the ease of determining if specified inputs affect the outputs”; controllability is “the ease of producing a specified output from a specified input” (freedman, 1991). by measuring the semantic ‘size’ of extensions, or program modifications, that enable a component to achieve domain testability, freedman was able to quantify both observability and controllability. wang et al. (1996) have also defined software testability metrics for observability and controllability, but by utilising the syntactic control structures in modules. others have also focused on the internal control structure of software when discussing testability. for example, bache and müllerburg (1990) define a number of testability metrics for control-flow based testing strategies. in essence the metrics they proposed determine the minimum number of paths, and hence the minimum number of test cases, needed in order to satisfy particular coverage criteria. bainbridge (1994) went on to show how the bache and müllerburg metrics could be calculated axiomatically. voas and colleagues have taken a radically different view of testability from most previous researchers (voas et al., 1991). they defined testability as the ease with which faults will manifest ajis vol.11 no. 1 september 2003 57 themselves as failures when the software undergoes the testing process. software with high testability, according to this definition, exposes its faults easily, whereas software with low testability may contain faults which ‘hide’, i.e. are difficult to expose. voas et al. went further by introducing an approach for measuring testability in terms of estimates from propagation, infection and execution (pie) analyses of software. this pie technique will be summarised in the next section. in the remainder of this paper, the word ‘testability’ is used entirely in the sense of voas and colleagues. bertolino and strigini (1996a) have rigorously investigated voas’s notion of testability and its use in dependability assessment. they modified voas’s definition and have given a more precise formulation of it. they then investigated the mathematics of using testability to estimate the probability of program correctness and the probability of failures. they have argued that high testability is not always a desirable program property and anyway, that program testability “can be more simply described in terms of assumptions on the probability distribution of the failure intensity of the program” (bertolino and strigini, 1996b). despite this rather critical sentence, testability in all its guises, remains a notion that continues to intrigue many researchers and the intention of the current authors is to add to the debate. mutation technique mutation testing is a powerful program testing technique used to generate good test data to uncover specific errors or classes of errors. the idea of mutation can be represented by simple changes made to a program, one at a time, e.g. changing the ‘-’ operator to the ‘+’ operator. in other words, the mutations represent different versions of the program, each of which differs from the original by a simple change. test data should then be constructed in a way that distinguishes each mutant from the original program. this means that if the output of the mutant differs from the output of the original program then the mutant is termed dead. however if the outputs are the same, then the mutant is called live and needs further investigation. if it is the case that no test data could ever ‘kill’ a live mutant, then that mutant is called equivalent to the original program [demillo, lipton and sayward, 1978; untch, offutt and harrold, 1993; woodward, 1993]. mutation testing, in the sense just described, which involves running the entire program till completion to get the result, is termed strong mutation testing. this is considered to be expensive and time consuming. as a result of this, weak mutation testing has been introduced. “weak mutation requires that a mutant program component yields a different ‘component outcome’ on at least one execution” [howden, 1982; woodward, 1991]. the pie model this section summarises the propagation, infection and execution (pie) model for measuring the sensitivity (or testability) of program locations. pie is a white-box analysis technique based on the syntax and semantics of the code (voas, 1992a; hamlet and voas, 1993). it makes predictions concerning future program behaviour by estimating the effect that input distribution, syntactic mutants and changed data values in data states have on current program behaviour (voas and miller, (1992a,1992b)). the pie assessment model implements the definition of testability promoted by voas and colleagues (voas and miller, 1995; voas, 1997; voas and mcgraw, 1998) by performing three independent dynamic analyses: execution, infection and propagation, which produce a set of estimates for each location of the given program. execution analysis is the process for predicting the probability that a location is executed when inputs are selected according to a particular input distribution, d say. infection analysis is the process for determining the probability that the succeeding data state of location l is different from the succeeding data state that a specific mutant creates, given that the original location and the mutant execute on a data state that would normally precede l. a data state is a collection of all variables and their associated values at some point during program execution. once infection estimates are calculated for all mutants of the specified location, the minimum probability among all will be considered as the infection estimate of that location. infection analysis is similar to weak mutation. propagation analysis is the process concerned with determination of the probability that a forced change in an internal computational state causes a change in the program’s ajis vol.11 no. 1 september 2003 58 output. once again propagation estimates are calculated for all forced changes, the minimum probability among all will be considered as the propagation estimate of that location. propagation analysis is similar to strong mutation. the three probability estimates can then be integrated to derive the sensitivity of each location and the overall testability of the program. the current authors have previously investigated the stability of the results from the pie technique itself when some of the parameters involved in the measurement process are altered (al-khanjari and woodward, 1998; al-khanjari et al., 2002). current testability tools automating the measurement of testability involves automating the three individual processes of sensitivity analysis: execution, infection and propagation analyses. this section introduces msginfection, a prototype tool that has been developed by the current authors for automating infection and execution analyses of c programs. first, however, a commercial tool called pisces is briefly described. the pisces tool a commercial tool called the pisces software testability analysis toolkit tm , that implements the pie algorithms, has been developed by the reliable software technologies corporation of sterling, virginia (friedman and voas, 1995). it would appear that pisces is the only commercial software for testability determination; it evolved from various proof-of-concept prototypes (voas et al., 1993a). pisces is written in c++ for analysing c programs. it generates testability estimates by developing an instrumented copy of the original program which should then be compiled and executed. pisces performs its infection analysis by creating appropriate c programming language mutants. the pisces toolkit tm is a combination of several individual tools or packages. one of the tools is the safteynet tool which incorporates propagation analysis to get an estimate for the fault tolerance of a program or indeed, for individual modules, functions, or even lines. the msg-infection tool a prototype tool called the msg-infection system, for determining both execution and infection analysis estimates, has been developed by the current authors. it uses a novel approach which is a modification of the mutant schemata generation (msg) technique for mutation testing of programs. the mutant schemata generation approach is used to improve the performance of mutation analysis systems by generating one metaprogram, which contains all encoded mutants for a given program (untch et al., 1993; untch, 1995). this approach based on program schema, which is a template. a partially interpreted program schema is defined by baruch and katz [baruch and katz, 1988]. the msg-infection system retains the spirit of the msg approach by encoding a number of mutants of each location in one single modified version of the original program. it uses an existing msg system for c programs (flanagan, 1997) which has been adapted in order to develop the msg-infection system. figure 1. the msg-infection system and its components msg-infection execution & infection testability management ajis vol.11 no. 1 september 2003 59 the msg-infection system splits into two major subsystems: an execution & infection system and a testability management system (see figure 1). the execution & infection system is, as has already been mentioned, a modification of an existing msg system which develops a parse tree corresponding to the given c program and modifies it to encode a variety of mutants at each location. the parse tree, which is modified to construct the metamutant version of the given c program, has been further altered in the msg-infection system so that each location l of the original program now becomes, in essence, the following: store pre-l; execute l; store post-l; for each mutant lm of l loop restore pre-l; execute lm; compare post-lm with post-l; end loop; restore post-l; in the above, ‘pre-l’ (‘post-l’) corresponds to the data state immediately before (after) location l. the result of each comparison of the post-lm state with the post-l state is saved and used to determine the infection estimate of location l. the storing of the data state prior to a location is achieved by instrumenting with assignments to a special array which remains undisturbed by execution of the location or any of its mutants. the restore operation after the location can then use the array to recover the values that the variables had before the location. note that, at present, only assignment statements are considered as locations by the msg-infection system. the testability management system manages the process of running the test cases against the original locations of the program and the corresponding mutants. it generates a file which contains all the required infection and execution estimates for each location of the given program. the msg-infection system currently incorporates four mutant classes: arithmetic operator replacement, variable replacement, constant replacement and statement deletion. several important macros have been included in the msg-infection system to facilitate the compilation process and further macros can easily be added to the system. the system has been built in a highly modular fashion with loose coupling between the modules to make future development as easy as possible. the system is user-friendly, interacting with the user as necessary to build the modified version of the original program and complete the compilation and execution processes of this metamutant program with the given test cases. the domain-to-range ratio and its inverse this section considers the domain-to-range ratio in more detail, suggests that its inverse the rangeto-domain ratio is more natural and proposes a method for its dynamic determination of the latter. domain-to-range ratio (drr) the domain-to-range ratio (drr) has been proposed by voas and miller (1991) as an approximate measure of implicit information loss. implicit information loss occurs when two or more different values of a parameter are used as input to a built-in function or a user-defined function and generate the same output. consider, for example, the function f(a) = a mod 2. this function generates one of only two possible outputs for every input value of a, namely 0 when a is even and 1 when a is odd. drr is a specification or functional description metric that can be defined as the ratio d / r, where d is the cardinality of the domain of the specification and r is the cardinality of the range (voas and miller, 1991). it is a fundamental characteristic of both mathematical and computational functions and, despite the fact that it may not always be apparent from a software specification, it can often be derived from semantic information found in such a specification (voas et al., 1992; voas et al., ajis vol.11 no. 1 september 2003 60 1993b). although the drr of a specification is fixed and can not be modified without changing the specification itself, the specification can be further decomposed into functions or modules, which can then be dealt with individually (voas and miller, 1991). it is possible to determine the drr for unary operators, binary operators and more complex expressions. in this way it may be possible to determine the drr at higher levels of granularity such as sub-specifications (i.e. specifications of modules), or even entire specifications (voas, 1991). the ratio is static and is determined independently of the actual distribution of input values. voas and miller (1993b) note that drr is related to freedman’s notion of controllability (freedman, 1991). indeed a controllable function can be considered as just the special case of a function f say, having drr = 1, i.e. if the output values of f are different, then so are the corresponding inputs that generate those outputs. because the domainto-range ratio is available early in the software life cycle it could help in exploring for hidden faults using empirical methods when code is eventually produced (voas and miller, 1993a). examples of drrs for various functions, as originally given by voas (though with corrections to the ranges of the circular functions 12, 13 and 14), are listed in table 1 (voas, 1991; voas and miller, 1993b). continuing voas’s practice, ∞i is used to represent the cardinality of the integers and ∞r the cardinality of the reals. table 1. drrs and testabilities of various functions as given by voas function testability drr comment 1. f a a a ( ) = <   0 0if otherwise low ∞i : ∞i / 2 a is integer 2. f(a) = a + 1 high ∞i : ∞i a is integer 3. f(a) = a mod b low ∞i : b testability decreases as b decreases 4. f(a) = a div b low ∞i : ∞i / b testability decreases as b increases, b q 0 5. f(a) = trunc(a) low ∞r : ∞i a is real 6. f(a) = round(a) low ∞r : ∞i a is real 7. f(a) = sqr(a) high ∞r : ∞r a is real, a ≥ 0 8. f(a) = sqrt(a) high ∞r : ∞r a is real, a ≥ 0 9. f(a) = a / b high ∞r : ∞r a is real, b q 0 10. f(a) = a – 1 high ∞i : ∞i a is integer 11. f(a) = even(a) low ∞i : 2 a is integer 12. f(a) = sin(a) low ∞i : ∞r a is integer (degrees), 13. f(a) = tan(a) low ∞i : ∞r a is integer (degrees), 14. f(a) = cos(a) low ∞i : ∞r a is integer (degrees), 15. f(a) = odd(a) low ∞i : 2 a is integer 16. f(a) = not(a) high 1 : 1 a is integer (0 or 1) although the drr for a specification has its own intrinsic value, its significance is greatly enhanced by the prospect of a connection with the ease of exposing potential faults in a corresponding implementation. voas et al. (1992) indicated that: “empirical observations suggest that a specification’s drr is related to its implementation’s ability to hide faults”. that is, high drr implies that faults are more likely to remain hidden in an implementation. however, low drr suggests that almost all faults in an implementation might be exposed during testing (voas et al., 1992). this means when drdr is low the program becomes more testable and the errors become more easy to find and correct. it has been argued by voas et al. (1992) that this potential link between drr and ease of fault exposure arises from data state error cancellation or simply from internal data state collapse. data state error cancellation occurs if an infected variable has its erroneous value corrected or perhaps never used in subsequent computations. internal data state collapse occurs when two different data states are input to some sub-component in a program and yet that sub-component produces the same ajis vol.11 no. 1 september 2003 61 output state. as voas and miller (1993b) remark: “when internal state collapse occurs, the lost information may have included evidence that internal states were incorrect. since such evidence is not visible in the output, the probability of observing a failure during testing is reduced.” this connection between drr and state collapse or likelihood of fault exposure is of course implying, in essence, a link between drr and testability, as defined in the sense of voas and colleagues. indeed voas et al. (1992) conjectured that “the testability of a program is correlated with the domain/range ratio”. more explicitly they stated that: “as the drr of the intended function increases, the testability of an implementation of that function decreases”. in other words, high drr is thought to lead to low testability and vice versa. for each of the functions listed in table 1, the testability as suggested by voas is given in the second column of the table (voas, 1991; voas and miller, 1993b). dynamic range-to-domain ratio (drdr) in order to assist with the investigation of the relationship between testability and domain-to-range ratio, which is the main focus of this paper, the current authors have adapted the ratio in two minor ways. the first (trivial) adaptation is simply to invert the ratio so that it becomes the range-to-domain ratio (rdr), i.e. the range-to-domain ratio is r / d where r is the cardinality of the range and d is the cardinality of the domain. this has the advantage that if there is a link between the ratio and testability, it is likely to be of the form: high values in one lead to high values in the other and similarly low values in one lead to low values in the other. this is clearly more natural than dealing with a mathematical relationship based on reciprocals. the second adaptation is to propose calculation of the range-to-domain ratio dynamically for individual program locations. in other words, the dynamic range-to-domain ratio (drdr) is defined as the ratio of the cardinality of the range to the cardinality of the domain for a given program location when that program is executed with test data from a given input distribution. the principle advantage of calculating the ratio dynamically (i.e. from program execution) is that it opens up the possibility of its automatic determination during white-box testing of the code. the remainder of this section illustrates the calculation of drdr with an assignment statement as an example. two possible methods are investigated: calculation using states and calculation using input/output tuples. calculation using states recall that it has been argued that there is a link between the domain-to-range ratio and ‘internal state collapse’ (voas and miller, 1993b). hence it seems worth investigating whether counting unique data states both before and after some program location on all executions of that location during a run of the program might provide a measure of ‘state collapse’. consider for example the assignment statement: z = x * x + y * y (1) with an input distribution such that: x, y = -1, 0, +1. also assume that z is initialised to zero prior to this location. then pre-states, the set of states involving the data values for variables x, y, z (given in angle brackets in that order) is as follows: pre-states = { < -1, -1, 0 >, < -1, 0, 0 >, < -1, 1, 0 >, < 0, -1, 0 >, < 0, 0, 0 >, < 0, 1, 0 >, < 1, -1, 0 >, < 1, 0, 0 >, < 1, 1, 0 > } the set of corresponding states after execution of the location, post-states, is: post-states = { < -1, -1, 2 >, < -1, 0, 1 >, < -1, 1, 2 >, ajis vol.11 no. 1 september 2003 62 < 0, -1, 1 >, < 0, 0, 0 >, < 0, 1, 1 >, < 1, -1, 2 >, < 1, 0, 1 >, < 1, 1, 2 > } thus: drdr = statespre tatesspost − − = 9 9 since the number of unique pre-states is the same as the number of unique post states, this would seem to imply that there is no reduction in the number of unique states for this location under these circumstances. clearly this contravenes expectations and indicates that, in general, if the inputs to a location are not altered, then there is no state collapse. calculation using input/output tuples since counting unique states before and after a program location does not lead to an appropriate way of determining drdr, an alternative approach is required. the approach adopted in this paper is based on counting unique tuples of input operands for the domain of a location and unique tuples of output operands for the range. at present, the only locations considered are assignment statements, as has already been mentioned, although in principle other types of location could be handled in a similar way. the range of an assignment location is the set of values assigned to the variable in question. the number of values that are unique then represents the cardinality of the range. determining the domain and its cardinality is a little harder. the current authors propose ascertaining all the data objects that may be considered input operands of the location. each execution of the location makes use of a tuple of values for these input operands. the number of unique input tuples will then be considered as the cardinality of the domain. the choice of the word ‘operand’ in the explanation just given is deliberate. the intention is to include constant data values that are inputs to a location as well as any variable data values. the reason for this is so that the assignment of a constant value to a variable may be considered to have input domain of cardinality one. the net effect of this entire approach for calculating the cardinality of the domain of an assignment location may be summarised as follows: (1) if a location assigns a constant to a variable, then the domain of this location has cardinality 1; (2) if the location assigns one variable to another, then the cardinality of the domain will be the number of unique values of the input variable; (3) if the location assigns to a variable using a combination of variables and constants, then the cardinality of the domain will be the number of unique combinations of the input variables and constants; (4) if the assignment involves a built-in function, then the cardinality of the domain will be the number of unique tuples which are used as parameters to the built-in function. to demonstrate how the calculations are performed, consider once again the assignment (1) with the same input distribution as given previously. the variable z constitutes the range and it achieves three unique values. the variables < x, y > constitute the tuple of input operands and there are nine unique combinations of values. therefore: |r| = card{ 0, 1, 2 } = 3 |d| = card{ < -1, -1 >, < -1, 0 >, < -1, 1 >, < 0, -1 >, < 0, 0 >, < 0, 1 >, < 1, -1 >, < 1, 0 >, < 1, 1 > } = 9 hence: drdr = r d = 3 9 = 0.33 which is clearly a more sensible interpretation of events than that given in the previous section. as another example, consider: z = 100 (2) ajis vol.11 no. 1 september 2003 63 to demonstrate the calculation when assigning a constant to a variable. for all possible executions of this location: |r| = card{100} = 1 |d| = card{100} = 1 hence: drdr = r d = 1 1 = 1 as far as implementation of an automated system to capture the drdr for program locations is considered, all that is required is the insertion of statements before and after each location to record the appropriate values of data objects in some file or files. the files can then be processed on termination of program execution to determine uniqueness of the domains and ranges of each location and report the corresponding values of drdr. this scheme could be generalised to sections of code, modules or even individual program paths once inputs and outputs are known. it is worth noting that the required ‘print’ statements are not unlike simple assertions which have long been advocated (stucki and foshee, 1975) and can be incorporated into an instrumented version of the original program with little difficulty. hence the drdr metric is inexpensive and relatively easy to calculate. if it can generate estimates that relate to testabilities of the chosen locations without conducting the full pie analysis, it will represent a significant saving of effort. empirical studies this section discusses the investigations, which have been undertaken to explore the link between infection and the metric, drdr. first the goal, the hypothesis and the significance of the experiment are given. then details are provided concerning the programs and test input distributions that have been used in the experiment. this is followed by a description of the experimental method. finally, the results of the experiment are reported and analysed. goal of the experiment the goal of this experiment is simply to investigate whether there is any relationship between infection and the drdr metric. if there is a relationship, then, in addition, it would be useful to know whether the two quantities follow the same or opposite trend. such a connection, if confirmed, could help in estimating the infection of programs without performing the actual calculations of the infection analysis which is part of the pie technique. hypothesis of the experiment the hypothesis of the experiment is that the drdr score of program locations is correlated with the infection estimates of those locations. in other words it is conjectured that the higher the drdr score, the higher the infection and vice versa. significance of the experiment if the drdr metric can give some indication concerning the infection of a program location, which is part of the sensitivity, this would have significant potential for savings on the effort, time and cost involved in conducting the analyses of the full pie technique. assuming that the drdr metric is a reflection of infection in some fashion, then the drdr metric could perhaps be used as a much simpler substitute for it and yet be useful in the same way that infection with conjunction with ajis vol.11 no. 1 september 2003 64 execution and propagation analyses are useful, namely for indicating those program locations where faults may ‘hide’ and further testing is desirable. the programs used in the experiment eleven programs written in c were selected for the experiment. the programs may be thought of as composed of three pairs: (1) three non-artificial programs; (2) three programs that implement the functions of table 1 in an unconnected fashion and the fourth program in this set represents the example program, which is concerned with checking reachability constraints, provided by friedman and voas [friedman and voas, 1995]; and (3) four programs that implement connected combinations of the same functions. for each of the eleven programs, table 2 lists the name, purpose, number of locations and, in the final column, the number of test cases used and their input domains. the first program (average.c) determines the average of a set of numbers supplied as input data. the second program (quadratic.c) determines whether a quadratic equation, whose coefficients are supplied as input data, has an integer solution. a deliberately erroneous version of this program has been used extensively in previous studies that showed it has very low overall testability (voas et al., 1991; al-khanjari and woodward, 1998). the third program (cubic.c) determines whether a cubic equation, whose coefficients are supplied as input data, has an integer solution. the fourth program (functions1.c) implements all the simple mathematical functions considered by voas (1991) and also voas and miller (1993b) and repeated in table 1 of this paper. in essence, each function is an assignment location and there is no connection between the assignments. the program requires a and b as test input – note b is not constant. the fifth program (functions2.c) implements just the functions numbered 3, 4 and 9, i.e. a mod b and a div b where a, b are integer, and a / b where a is real. these three functions depend on two input parameters, but in this particular program b is kept fixed. separate evaluations were performed with a range of values for b. the sixth program (functions3.c) implements similar set of the simple mathematical functions used in (functions1.c) program. the seventh program (functions4.c) implements the example program provided by friedman and voas to check the reachability constraints. the final four programs (artificial1.c, artificial2.c, artificial3.c and artificial4.c) are artificial in the sense that they were constructed specially for the experiment. they contain uses of several of the simple functions in connected combinations in the knowledge that these functions give a good spread for the scores of both infection and drdr table 2. the programs selected for the experiment name purpose locations test cases average.c calculates the average of a set s of values; next identifies each value in turn. 4 100 cases card s ∈ [2,16] next ∈ [0,6000] quadratic.c determines whether a quadratic equation has an integer root. 4 10000 cases a, c ∈ [0,10], b ∈ [1,1000] cubic.c determines whether a cubic equation has an integer root. 9 1000 cases a, b ∈ [-1000,1000] functions1.c implementation of all 16 of the simple mathematical functions given in table 1. 16 10000 cases a, b ∈ [-10000,10000] functions2.c implementation of functions: a mod b, a div b and a / b (b constant) from table 1. 3 10000 cases a ∈ [-10000,10000] functions3.c implementation of some of the simple mathematical functions given in table 1. 8 10000 cases a ∈ [-1000,1000] ajis vol.11 no. 1 september 2003 65 functions4.c implementation of the example program provided by friedman and voas to check the reachability constraints. 6 10000 cases a, b ∈ [-100,100] artificial1.c artificial program constructed from connected combinations of the functions in table 1. 7 1000 cases a, b ∈ [-10000,10000] artificial2.c artificial program constructed from connected combinations of the functions in table 1. 9 1000 cases a, b ∈ [-10000,10000] artificial3.c artificial program constructed from connected combinations of the functions in table 1. 7 1000 cases a, b ∈ [-1000,1000] artificial4.c artificial program constructed from connected combinations of the functions in table 1. 10 10000 cases a, c ∈ [0,10] b ∈ [1,1000] experimental method for the purposes of comparison between infection and drdr scores, the authors have observed that the infection estimate is the most crucial component of the pie testability for a given program location. also as mentioned previously the msg-infection tool, which uses the mutant schemata approach, has been generated mainly to improve the mutation analysis and accordingly to improve the performance of the infection analysis. as a reflection the experiments have concentrated on infection analysis only. on the other hand, as authors observed that for some programs the infection analysis will be zero or close to zero. this means conducting such experiments will be waste of time. because of that the purpose of the experiments was to find some concepts such as drdr, which might give an influence of the expected infection estimate without conducting the real analysis. for each assignment location in each of the chosen programs the drdr scores and the infection analysis estimates were obtained. the number of test cases used for each program is listed in table 2; each test case was constructed randomly using a uniform distribution within the input domains also listed in table 2. each c program together with its input data (test cases) was submitted to the msg-infection system and the infection estimate results were reported automatically. to determine the drdr scores however, a special instrumented version of each c program was created, which, when executed, generated files containing the domains and the ranges for each individual location of the c program under test. a simple post-processor tool was then used to determine the unique domain and range tuples and hence the drdr scores. experimental results this section reports the results of the experiment. for each of the eleven programs selected for the experiment, the infection analysis estimates and corresponding drdr scores for each program location are presented as pairs of graphs. figures 2 through 11 show infection estimates and corresponding drdr scores for all locations of the programs average.c, quadratic.c, cubic.c, functions1.c, functions3.c, functions4.c, artificial1.c, artificial2.c, artificial3.c and artificial4.c respectively. ajis vol.11 no. 1 september 2003 66 infection for 4 locations 0.86 0.88 0.9 0.92 0.94 0.96 0.98 locations in fe c ti o n drdr for 4 locations 0.5 0.6 0.7 0.8 0.9 1 0 1 2 3 locations d r d r figure 2(a). infection for average.c figure 2(b). drdr for average.c infection for 4 locations 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0 1 2 3 locations in fe c ti o n drdr for 4 locations 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0 1 2 3 locations d r d r figure 3(a). infection for quadratic.c figure 3(b). drdr for quadratic.c infection for 9 locations 0.5 0.6 0.7 0.8 0.9 1 0 1 2 3 4 5 6 7 8 locations in fe c ti o n drdr for 9 locations 0.75 0.8 0.85 0.9 0.95 1 0 1 2 3 4 5 6 7 8 locations d r d r figure 4(a). infection for cubic.c figure 4(b). drdr for cubic.c ajis vol.11 no. 1 september 2003 67 infection for 16 locations 0.4 0.5 0.6 0.7 0.8 0.9 1 0 1 2 3 4 5 6 7 8 9 1 0 1 1 1 2 1 3 1 4 1 5 locations in fe c ti o n drdr for 16 locations 0 0.2 0.4 0.6 0.8 1 0 1 2 3 4 5 6 7 8 9 1 0 1 1 1 2 1 3 1 4 1 5 locations d r d r figure 5(a). infection for functions1.c figure 5(b). drdr for functions1.c infection for 8 locations 0.75 0.8 0.85 0.9 0.95 1 0 1 2 3 4 5 6 7 locations in fe c ti o n drdr for 8 locations 0 0.2 0.4 0.6 0.8 1 0 1 2 3 4 5 6 7 locations d r d r figure 6(a). infection for functions3.c figure 6(b). drdr for functions3.c infection for 6 locations 0.6 0.65 0.7 0.75 0.8 0.85 0.9 0.95 1 0 1 2 3 4 5 locations in fe c ti o n drdr for 6 locations 0.92 0.929 0.938 0.947 0.956 0.965 0.974 0.983 0.992 0 1 2 3 4 5 locations d r d r figure 7(a). infection for functions4.c figure 7(b). drdr for functions4.c ajis vol.11 no. 1 september 2003 68 infection for 7 locations 0.4 0.5 0.6 0.7 0.8 0.9 1 0 1 2 3 4 5 6 locations in fe c ti o n drdr for 7 locations 0 0.2 0.4 0.6 0.8 1 0 1 2 3 4 5 6 locations d r d r figure 8(a). infection for artificial1.c figure 8(b). drdr for artificial1.c infection for 9 locations 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0 1 2 3 4 5 6 7 8 locations in fe c ti o n drdr for 9 locations 0 0.2 0.4 0.6 0.8 1 0 1 2 3 4 5 6 7 8 locations d r d r figure 9(a). infection for artificial2.c figure 9(b). drdr for artificial2.c infection for 7 locations 0.65 0.7 0.75 0.8 0.85 0.9 0.95 1 0 1 2 3 4 5 6 locations in fe c ti o n drdr for 7 locations 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0 1 2 3 4 5 6 locations d r d r figure 10(a). infection for artificial3.c figure 10(b). drdr for artificial3.c ajis vol.11 no. 1 september 2003 69 infection for 10 locations 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0 1 2 3 4 5 6 7 8 9 locations in fe c ti o n drdr for 10 locations 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 locations d r d r figure 11(a). infection for artificial4.c figure 11(b). drdr for artificial4.c finally infection estimates and corresponding drdr scores for the remaining program functions2.c, which is a special case, are presented with reference to its individual locations. this is because these locations represent different isolated functions involving a second parameter b, which has been considered as a constant when determining the infection and drdr scores. the most interesting functions are a mod b and a div b. results are reported for 40 different cases of the variable b, where the value of b varies from -10000 to +10000 in steps of 500. figures 12 and 13 show infection estimates and corresponding drdr scores for the a mod b and a div b functions of the functions2.c program respectively, for the 40 different cases of the variable b. -1 0 0 0 0 -8 5 0 0 -7 0 0 0 -5 5 0 0 -4 0 0 0 -2 5 0 0 -1 0 0 0 1 0 0 0 2 5 0 0 4 0 0 0 5 5 0 0 7 0 0 0 8 5 0 0 1 0 0 0 0 b 0.9985 0.999 0.9995 1 in fe c ti o n infection infection for 40 cases (z = a mod b) -1 0 0 0 0 -8 5 0 0 -7 0 0 0 -5 5 0 0 -4 0 0 0 -2 5 0 0 -1 0 0 0 1 0 0 0 2 5 0 0 4 0 0 0 5 5 0 0 7 0 0 0 8 5 0 0 1 0 0 0 0 b 0 0.2 0.4 0.6 0.8 1 d r d r drdr drdr for 40 cases (z = a mod b) figure 12(a). infection for functions2.c figure 12(b). drdr for functions2.c ajis vol.11 no. 1 september 2003 70 -1 0 0 0 0 -8 5 0 0 -7 0 0 0 -5 5 0 0 -4 0 0 0 -2 5 0 0 -1 0 0 0 1 0 0 0 2 5 0 0 4 0 0 0 5 5 0 0 7 0 0 0 8 5 0 0 1 0 0 0 0 b 0 0.2 0.4 0.6 0.8 1 in fe c ti o n infection infection for 40 cases (f = a div b) -1 0 0 0 0 -8 5 0 0 -7 0 0 0 -5 5 0 0 -4 0 0 0 -2 5 0 0 -1 0 0 0 1 0 0 0 2 5 0 0 4 0 0 0 5 5 0 0 7 0 0 0 8 5 0 0 1 0 0 0 0 b 0 0.001 0.002 0.003 0.004 0.005 0.006 d r d r drdr drdr for 40 cases (f = a div b) figure 13(a). infection for functions2.c figure 13(b). drdr for functions2.c analysis of results viewing the figures, one can see that there are often similarities between the infection and the drdr graphs for a given program. they frequently take the same shape even though the scales may be different. since the figures suggest that a correlation between infection and drdr is plausible, statistical analysis of the corresponding values has been conducted. the correlation coefficients for the selected programs are given in table 3. ajis vol.11 no. 1 september 2003 71 table 3. correlation (r 2 ) between infection and drdr for the selected programs program functions r 2 confidence relationship average.c all 0.2700 75% weak positive linear relationship quadratic.c all 0.9946 95% strong positive linear relationship cubic.c all 0.1742 50% weak positive linear relationship functions1.c all 0.3446 95% weak positive linear relationship functions2.c a mod b 0.6859 95% substantial positive linear relationship functions2.c a div b 0.4013 95% weak positive linear relationship functions3.c all 0.5764 95% substantial positive linear relationship functions4.c all 0.9982 99% strong positive linear relationship artificial1.c all 0.4852 90% weak positive linear relationship artificial2.c all 0.3980 90% weak positive linear relationship artificial3.c all 0.3051 80% weak positive linear relationship artificial4.c all 0.6561 95% substantial positive linear relationship from table 3, the percentages of the correlation that explain the variations in the infection using the drdr scores are about 27%, 99%, 17%, 34%, 69%, 40%, 58%, 100%, 49%, 40%, 31% and 66% of the selected programs respectively. as further evidence, the student’s t-distribution test, with the confidence level given in table3 for the selected programs, has been used to check the null hypothesis (that there is no relationship between the variables: infection and drdr), against the alternative hypothesis (that there is a relationship between the suggested variables). the results of the tests were to reject the null hypothesis and accept the alternative hypothesis. this means that there is a relationship between infection and drdr. an indication of the strengths of the relationship for each of the selected programs, or particular function within a program, is also given in table 3. summary and conclusions this paper has summarised and reviewed some of the many interpretations of the concept of ‘testability’ before focusing on the particular definition introduced by voas and colleagues. the propagation, infection and execution (pie) method of determining voas’s notion of testability of program locations has been briefly explained and a prototype system called msg-infection developed by the authors for automating infection and execution analyses has been introduced. msg-infection is novel in being an adaptation of an existing mutant schemata generation (msg) system. since it has been suggested that there is a possible link between testability and the so-called domainto-range ratio (drr) metric, an attempt has been made to investigate partial relationship between infection and drdr empirically. the authors have argued that the infection component of the pie testability estimate can be considered the most crucial component for this investigation. to assist this investigation the authors have proposed two minor changes: (1) inverting the metric, to make the potential partial relationship direct rather than reciprocal, and (2) measuring it dynamically, to help automate its determination. the resultant metric, the dynamic range-to-domain ratio (drdr), has been determined for the assignment locations in a number of programs. the infection estimates have also been determined for the same locations using the msg-infection system. the experiment utilised a mixture of artificial and non-artificial programs. the artificially constructed programs were designed to incorporate, either in isolation or in various combinations, a number of simple mathematical functions studied theoretically by voas. although in most cases the trends shown by both infection and drdr appear similar, when analysed statistically the correlation shows variation from a strong linear relationship to a weak linear relationship. the strongest correlation observed, somewhat surprisingly, was for the non-artificial program quadratic.c which determines whether a quadratic equation has integral solutions. the next strongest correlation observed, perhaps less surprisingly, was for the function a mod b. the graph of infection values as b varies provides pleasing confirmation of the expectation that testability decreases as b decreases and the corresponding graph of drdr provides dramatic evidence of the same trend, even though ajis vol.11 no. 1 september 2003 72 the numeric values are on different scales. however, the other programs demonstrate much weaker correlation between infection and drdr.the tentative conclusion is that, whilst it is intuitively tempting to suppose there might be a link, it would appear that any such link is far less direct than one might hope. hence, although this study has shed some light on the partial relationship between testability and the domain-to-range ratio, further research is certainly required. acknowledgements zuhoor al-khanjari gratefully acknowledges the support of sultan qaboos university and the government of the sultanate of oman. both authors thank steven flanagan for permission to use his msg system. references al-khanjari, z.a. and woodward, m.r. (1998) “investigations into the pie testability technique”, proceedings of the 4th international conference on achieving quality in software (aquis ’98), venice, italy, april 1998, iei-cnr, pp 25-34. al-khanjari, z.a., woodward, m.r. and ramadhan, h. (2002) “critical analysis of the pie testability technique”, software quality journal, december, vol. 10, no. 4, pp 97-108, kluwer academic publishers. bache, r. and müllerburg, m. (1990) “measures of testability as a basis for quality assurance”, software engineering journal, vol. 5, no. 2, pp 86-92. bainbridge, j. (1994) “defining testability metrics axiomatically”, software testing, verification and reliability, vol. 4, no. 2, pp 63-80. baruch, o. and katz, s. (1988) “partially interpreted schemas for cps programming”. science of computer programming, vol. 10, no. 1, pp 1-18. bertolino, a. and strigini, l. (1996a) “on the use of testability measures for dependability assessment”, ieee transactions on software engineering, vol. 22, no. 2, pp 97-108. bertolino, a. and strigini, l. (1996b) “acceptance criteria for critical software based on testability estimates and test results”, proceedings of the 15th international conference on computer safety, reliability and security (safecomp ’96), vienna, austria, 1996, springer, berlin, germany, pp. 83-94. boehm, b.w. (1984) “verifying and validating software requirements and design specifications”, ieee software, vol. 1, no. 1, pp 75-88. boehm, b.w., brown, j.r. and lipow, m. (1976) “quantitative evaluation of software quality”, proceedings of the second international conference on software engineering, san francisco, u.s.a., 1976, ieee computer society press, los alamitos, california, u.s.a., pp 592-605. demillo, r.a., lipton, r.j. and sayward, f.g. (1978) “hints on test data selection: help for the practicing programmer”, ieee computer, vol. 11, no. 4, pp 33-41. flanagan, s.j. (1997) mutation testing using mutant schemata, bsc dissertation, computer science department, university of liverpool, u.k. freedman, r.s. (1991) “testability of software components”, ieee transactions on software engineering, vol. 17, no. 6, pp 553-564. friedman, m.a. and voas, j.m. (1995) software assessment: reliability, safety, testability, wiley, new york, u.s.a. hamlet, d. and voas, j. (1993) “faults on its sleeve: amplifying software reliability testing”, proceedings of the 1993 international symposium on software testing and analysis (issta ’93), cambridge, massachusetts, u.s.a., june 1993, acm sigsoft software engineering notes, vol. 18, no. 3, pp 89-98. howden, w.e. (1982) “weak mutation testing and completeness of program test sets”, ieee transactions on software engineering, vol. 8, no. 4, pp 371-379. howden, w.e. (1985) “the theory and practice of functional testing”, ieee software, vol. 2, no. 5, pp 6-17. ieee (1990) glossary of software engineering terminology, ieee computer society press, los alamitos, california, u.s.a. ajis vol.11 no. 1 september 2003 73 stucki, l.g. and foshee, g.l. (1975) “new assertion concepts for self-metric software validation”, acm sigplan notices, vol. 10, no. 6, pp 59-71. untch, r.h. (1995) schema-based mutation analysis: a new test data adequacy assessment method, phd thesis, department of computer science, clemson university, south carolina, u.s.a. untch, r.h., offutt, a.j. and harrold, m.j. (1993) “mutation analysis using mutant schemata”, proceedings of the 1993 international symposium on software testing and analysis (issta ’93), cambridge, massachusetts, u.s.a., june 1993, acm sigsoft software engineering notes, vol. 18, no. 3, pp 139-148. voas, j.m. (1991) “factors that affect software testability”, proceedings of the 9th pacific northwest software quality conference, portland, oregon, u.s.a., pp 235-247. voas, j.m. (1992a) “dynamic testing complexity metric”, software quality journal, vol. 1, no. 2, pp 101-114. voas, j.m. (1992b) “pie: a dynamic failure-based technique”, ieee transactions on software engineering, vol. 18, no. 8, pp 717-727. voas, j.m. (1997) “software testability measurement for intelligent assertion placement”, software quality journal, vol. 6, no. 4, pp 327-335. voas, j.m. and mcgraw, g. (1998) software fault injection: inoculating programs against errors, wiley, new york, u.s.a. voas, j.m. and miller, k.w. (1991) “improving software reliability by estimating the fault hiding ability of a program before it is written”, proceedings of the 9th software reliability symposium, denver section of the ieee reliability society, colorado springs, u.s.a., 1991. voas, j.m. and miller, k.w. (1992a) “the revealing power of a test case”, software testing, verification and reliability, vol. 2, no. 1, pp 25-42. voas, j.m. and miller, k.w. (1992b) “improving the software development process using testability research”, proceedings of the 3rd international symposium on software reliability engineering, research triangle park, north carolina, u.s.a., 1992, pp 114-121. voas, j.m. and miller, k.w. (1993a) “applying a dynamic testability technique to debugging certain classes of software faults”, software quality journal, vol. 2, no. 1, pp 61-75. voas, j.m. and miller, k.w. (1993b) “semantic metrics for software testability”, the journal of systems and software, vol. 20, no. 3, pp 207-216. voas, j.m. and miller, k.w. (1995) “software testability: the new verification”, ieee software, vol. 12, no. 3, pp 17-28. voas, j.m., morell, l.j. and miller, k.w. (1991) “predicting where faults can hide from testing”, ieee software, vol. 8, no. 2, pp 41-48. voas, j.m., miller, k.w. and noonan, r. (1992) “designing programs that do not hide data state errors during random black-box testing”, proceedings of the 5th international conference on putting into practice methods and tools for information system design, nantes, france, 1992. voas, j.m., miller, k.w. and payne, j.e. (1993a) “software testing and its application to avionic software”, proceedings of computers in aerospace 9, ca. publisher: aiaa, san diego, u.s.a., 1993. voas, j.m., miller, k.w. and payne, j.e. (1993b) “an empirical comparison of a dynamic software testability metric to static cyclomatic complexity”, proceedings of the 18th annual software engineering workshop, 1993, nasa-goddard software engineering laboratory series report 93-003. wang, y., king, g., staples, g., ross, m. and court, i. (1996) “towards a metric of software testability”, proceedings of the 5th software quality conference, dudhope castle, university of abertay dundee business school, dundee, scotland, u.k., july 1996, pp 234-241. weyuker, e.j. (1982) “on testing non-testable programs”, the computer journal, vol. 25, no. 4, pp 465-470. white, l.j. (1987) “software testing and verification”, advances in computers, vol. 26, pp 335391. ajis vol.11 no. 1 september 2003 74 woodward, m.r. (1991) “concerning ordered mutation testing of relational operators”, journal of software testing, verification & reliability, vol. 1, no. 3, pp 35-40. woodward, m.r. (1993) “mutation testing-its origin and evaluation”, journal of information and software technology, vol. 35, no. 3, pp 163-169. understanding the diffusion of efficient consumer response: an australian survey study sherah kurnia7 john m. belts2 robert b.johnston3 '•' department of information systems the university of melbourne, australia 2 school of business systems monash university, australia abstract efficient consumer response (ecr) is designed to make the grocery industry more efficient. although it originated in the us, the concept has been adopted in many regions. to enrich the findings of the existing studies that indicate a slow diffusion rate of ecr, this study examines ecr adoption in australia by conducting a survey. the findings suggest that in australia, ecr diffusion has also been slow. differences in barriers, perceptions, and benefits experienced between manufacturers and retailers discovered in this study suggest that australian retailers are leading manufacturers in ecr implementation and that they experience more benefits than manufacturers. keywords: supply chain management, exploratory research, survey, innovation, australia. introduction efficient consumer response (ecr) originated in the united states in late 1980's and early 1990's (tripplet 1995) as a direct response to threats from alternative store formats which were taking market share away from the major supermarket chains. it can be defined as "a complex management theory that calls for changes in nearly all grocery work processes and practices to make the industry more efficient and responsive to consumers' needs" (tripplet 1995, p 3). ecr has the potential to remove significant costs from the grocery supply chain through the elimination of non-value-added activities, which will in turn, result in reductions in operating costs and inventory levels, and an increase in efficiencies at all levels within the supply chain (tripplet 1995). all this will ultimately allow the industry to offer greater value to the grocery consumers through the provision of better prices, better store assortment, better service, better convenience, and better quality products (kurt salmon associates 1993; wood 1996). the ability to provide greater value to consumer is crucial for industry survival, since competition has become more intense and consumers have gained more power. therefore, the concept of ecr has attracted many other regions, noticeably european countries and australia, with different motivations from the us (leggett 1996; wheatley 1997; kurnia et al. 1998). however, despite the benefits and potential savings obtainable from ecr, a number of studies indicate that the diffusion rate of the concept has been slow in the us and europe (kurt salmon associates 1995; leggett 1996; coopers and lybrand 1997; greenbaum 1997; ibm 1997; kurt salmon associates 1997). study of ecr diffusion is not well developed at present. differences in the retail industries and the environments of different countries should be exploited to increase understanding of the relative importance of various driving forces and the above three factors to ecr diffusion and adoption. australia has a very different market structure to other countries, in which the consumers are fewer in number and dispersed over a wide geographical area. in addition, the impetus for ecr adoption is different from that of the us and europe. competitive pressure from alternative store formats is not present in this country. the australian grocery industry is dominated by very few key players which constitute the "big four" australian retailers. the general motivation to embrace ecr, therefore, appears to be pressure from these large retailers. other companies within the industry were forced to get involved in ecr, regardless their perceptions of the concept. this study, therefore, aims to define the extent of ecr implementation in australia and to identify the barriers to ecr adoption, perception of ecr and the realisation of ecr benefits hi australia. for this purpose, a survey of the australian grocery industry was performed to explore the experience of the industry with ecr adoption. in addition to exploring the industry experience with ecr, we wanted to test the following two hypotheses: 1. in australia, retailers are leading manufacturers in the ecr implementation. 2. australian retailers have gained more benefits from ecr than manufacturers. the survey findings suggest that, as in the us and europe, the ecr diffusion rate is low in australia. furthermore, the results of the analysis demonstrate that both manufacturer and retailer groups differ in relation to barriers to ecr implementation, perception of ecr characteristics and the benefits gained, in such a way that they support the above two hypotheses. because of the uniqueness of the australian grocery industry, this study 7 address for correspondence: department of information systems, the university of melbourne, melbourne 3010, australia. phone: 61 3 8344 0870, fax: 61 3 9349 4596, e-mail: sherahk@unirnelb.edu.au 113 has the potential to enrich the findings of other studies in different countries, to better understand the ecr diffusion process in general. in the next section, we provide a brief description of ecr. we then describe the survey research method employed in this study and discuss the survey findings comprehensively. finally, we conclude the paper by outlining the limitations to this study and suggesting some related future research. ecr as a grocery supply chain management initiative ecr consists of strategic initiatives, operational programs and a number of enabling technologies, which are summarised in figure 1. figure 1. ecr components and their relationships (kurnia et al. 1998) the strategic initiatives promoted by ecr are based on four areas: store assortment, promotion, product introduction, and product replenishment (kurt salmon associates 1993): • efficient store assortment this initiative is aimed at optimising the productivity of inventory and shelf management at the store level. • efficient product introduction the objective of this initiative is to maximise the effectiveness of new product development and introduction activities, in order to reduce costs and failure rates in introducing new products. • efficient promotion this initiative aims at maximising the total system efficiency of trade and consumer promotions. this can be achieved by introducing better alternative promotions, such as pay for performance or every day low price program. • efficient product replenishment the objective of this initiative is to optimise time and cost in the replenishment system by the provision of the right product to the right place at the right time in the right quantity and in the most efficient manner possible. these four initiatives are supported by two operational programs, namely, category management (cm) and continuous replenishment program (crp). cm is an interactive business process in which retailers and manufacturers work together to manage categories as strategic business units within each store (information advantage 1996). it supports the first three initiatives of ecr discussed above. crp, on the other hand, is a practice of partnering among members of a distribution channel to allow for products to flow smoothly and continuously from manufacturers to consumers (martin 1994). it supports the efficient product replenishment initiative. 114 the operational programs are, in turn, supported by a number of enabling technologies: barcode/scanner, electronic data interchange (edi), computer aided ordering (cao), cross-docking, and activity based costing (abc): • barcodes / scanners the use of barcodes and scanners is a fundamental component for ecr implementation as it enables accurate and fast information capture as well as information sharing between trading partners (ean australia 1997). • electronic data interchange (edi) electronic data interchange (edi) is defined by emmelhainz (1990) as an inter-organisational exchange of business documents in a structured, machine-processable form. • computer-aided ordering (cao) computer aided ordering (cao) is an ordering system that automatically generates orders for replenishment when the inventory level drops below a pre-determined reorder level (ecr central 1997). • cross-docking cross-docking or "flow-through distribution " is a direct flow of products at the distribution centre from receiving to shipping, thus eliminating additional handling and storage steps in the distribution cycle (andel 1994). • activity-based costing activity-based costing is a new costing tool that works on the principle that activities (as opposed to product volumes or labour in traditional accounting) are what really affect costs. abc offers a better understanding of how profits are generated, as it increases the visibility of costs in a particular environment. it can be used to gain top management commitment and leadership to support the implementation of ecr and its key components (landry 1997). the survey research method we decided that a mail survey would be the most appropriate method of achieving the aims of the study, because this would allow us to reach a wide range of organisations within the australian grocery industry. managers, or any individual with specific knowledge on ecr-type implementations, were requested to answer the questionnaire. an initial inquiry letter describing the project and seeking participation was sent out to the senior executive of all organisations listed in the grocery industry marketing guide 1998 (retail world 1998) to identify companies interested in ecr studies. only those who registered some interest in ecr studies were included in the target population. those companies that are not interested in or aware of ecr studies were expected not to be able to complete the questionnaire. fifty-two companies indicated their interest in participating. two key retailers were interested in ecr studies but were not willing to participate in this study because they had just participated in another by coopers and lybrand (1998) which aimed to develop an ecr scorecard and the industry's level of maturity in implementing initiatives to improve supply chain performance. thus, while the target population for this study was from fiftyfour organisations, the actual questionnaire was only sent out to fifty-two organisations. after two follow-ups were made via mail and phone, the number of returned questionnaire was 42. despite the small sample size, the survey participants are representative of the population for each organisation type, due to their market share. the participants representing the 'retailer / wholesaler / distributor' group, which consists of eleven organisations, control more than 30 per cent of the total market share of the australian grocery industry. similarly, around 40 per cent of the participants representing the 'manufacturer / broker' group dominate more than 50 per cent of the market share of a number of product categories within the australian grocery industry (food and liquor week 1998; retail world 1999). in addition, the number of participants for the survey conducted by coopers and lybrand on behalf of the ecr australia was only thirtysix companies, indicating the small number of the australian companies involved in ecr at this stage (coopers and lybrand, 1998). given this fact, the small sample involved hi this study arguably constitutes a major part of the entire population of the australian companies involved in ecr. thus, the small sample of this study should by no means invalidate the findings of the survey, although it reduces the chances of obtaining results with high significance in the statistical tests of the survey responses. the questionnaire item generation was derived from a literature review on ecr, supply chain management, and diffusion of organisational innovation. some items were adapted from the survey questionnaire of kurt salmon associates for the us and european ecr progress study, since it is desirable to replicate existing well developed questionnaires in survey research (lucas 1991). before sending out the questionnaire to the sample population of 52 companies, pilot tests were conducted with a logistics researcher and a practitioner who resembled the actual respondents to whom the questionnaire would be sent. 115 in this study, survey responses were analysed as a function of company type. to improve the statistical significance of the results, manufacturers and brokers are classified as 'manufacturer' while retailers, distributors and wholesalers are classified as 'retailer'. this classification allows us to distinguish between the upstream and downstream players of supply chains. since two key retailers were included in this study, most of the participant manufacturers are the suppliers of the retailer group. frequency of responses for each group and rank order were used for nominal data. to find the relationships between nominal variables, the fisher's exact test was used because of the small sample size. for ordinal data, differences between the two groups were observed using means, which were found to be sensitive to small differences (argyrous 1996). the mann-whitney test for the difference of medians, based on the rank of responses, was used for significance testing of ordinal data (coakes and steed 1997). in this paper, the following notations are used to indicate the level of statistical significance discovered with each test: m difference between two groups is significant at 5 per cent level, using the mann-whitney test. ( difference between two groups is significant at 5 per cent level, using the fisher's exact test. the survey findings and discussion demographic information table i depicts the locations of the participants and the numbers of questionnaires sent to each location, as well as the numbers of questionnaires returned from each location. the majority of participants are located in the two most densely populated states: new south wales (nsw) and victoria (vic). the 42 respondents consist of 59% manufacturers, 14% brokers, 7% retailers, 10% wholesaler and 10% retailers, as shown in table ii. seventy percent of the respondents are smes, with an annual sale below $100 million based on the 1998 financial year, and 30% of the participants are large enterprises. distributed returned state nsw vic qld wa sa total frequency j% 19 18 8 6 1 52 |36 36 111 j2 j l o o frequency 14 16 7 4 1 42 1% :33 j38 j2 |100 company type manufacturer broker retailer wholesaler distributor total frequency 25 6 3 4 4 42 % 59 14 7 10 10 100 table i. survey distribution and responses by state ecr involvement levels table ii. survey respondents by company type to obtain the ecr involvement level of the participants, participants were asked to describe their level of ecr involvement from the following options: 'fully implemented', 'currently implementing', 'considering', 'not sure', and 'not involved'. only two respondents (5%) have fully implemented ecr, 17 respondents (40%) are currently implementing, four respondents (10%) are considering, two (5%) are not sure whether they are involved in ecr, and 17 respondents (40%) are not involved in ecr at all. this gives us a total of 25 participants who are most likely able to provide us with useful responses for detailed ecr questions in the questionnaire. some reasons for non-involvement have also been identified from the responses of those 17 participants. the most frequently cited reason (cited by 65% of the respondents) is 'do not understand ecr', suggesting that ecr is not widely understood in australia. other important reasons include 'other priority more important', 'lack of it infrastructure', and 'shortage of skilled personnel'. the driving forces from the other 25 participants, a number of driving forces to get involved in ecr have been identified, as shown in table iii. the analysis indicates that for manufacturers, the major driving force is exogenous, whereas for the retailer it is endogenous. this finding reinforces the idea that in most supply chains, retailers are the ones who initiate the ecr program in order to improve their internal operations. most manufacturers simply embraced the concept of ecr just to meet the requirement of their larger trading partners in order to stay in business. 116 catalysts pressure from trading partner unpredictable shipping performance under utilization of assets declining customer service unpredictable demand poor manufacturing efficiency inventory unbalances declining competitiveness increasing product costs improve competitiveness manufacturer (n=19) % 74> 11 26 16 42 26 16' 26 16 21 rank 1 3 2 3 3 retailer (n=6) % 33/ 17 33 33 17 17 68' 17 33 17 rank 2 2 2 1 2 table hi. ecr catalysts implementation level of ecr components to measure the ecr implementation level, participants were requested to describe their current implementation status with the ecr components from a range of responses: 'no plan to implement', 'keen to explore further', 'plan to begin in 12 months', 'in testing/pilot stage', and 'fully operational'. for each component, they were also asked to indicate if they were pursuing the component as part of ecr. figure 2 depicts the proportion of manufacturers and retailers who are actively pursuing the components, defined as either testing them or having fully implemented them. crp o manufacturer • retailer barcode edi cao ecr components x-dock abc nman=19; 1̂ = figure 2. actively pursuing the ecr components by company type the results demonstrate that a relatively large proportion of manufacturers and retailers are actively pursuing ecr components. retailers, however, are much more enthusiastic than manufacturers about the continuous replenishment program and computer aided ordering implementations. this suggests that retailers are more concerned with overall supply chain management than manufacturers. although the results suggest a high implementation level for most of the ecr components, most manufacturers indicated that they were pursuing each component not as part of the ecr program. half of the retailers indicated that they were involved in crp, edi, and cross docking as part of the ecr program. this finding again supports our hypothesis that retailers are leading the ecr implementation in australia, and manufacturers only attempt to conform to the retailers' requirement. further analysis indicates that there are more retailers than manufacturers who plan to begin pursuing the ecr . components within 12 months. the most prominent component to be pursued is edi, followed by crp, cao, and cross docking. manufacturers, by contrast, seem to be more interested in pursuing abc than retailers. this suggests that manufacturers are more sceptical about the ecr concept than retailers, and therefore, they plan to conduct activity based costing to investigate if ecr is indeed beneficial satisfaction level with the ecr initiatives participants were asked to describe thensatisfaction level with the four ecr initiatives based on the following five scale: -1 = very disappointed; -0.5 = disappointed; 0 = neutral; 0.5 = satisfied; 1 = very satisfied. the means of the responses were calculated and are presented in table iv. 117 the findings suggest that retailers are more satisfied with the ecr initiatives, particularly the efficient promotion and efficient product introduction initiatives. manufacturers, on the other hand, show a very low satisfaction level, suggesting that most of them were driven by retailers to get involved in ecr, without any particular interest. manufacturer retailer efficient store assortment (nman=18; nret=5) -0.11 0.20 efficient promotion (nman=19;nret=5) -0.05 0.40 efficient product introduction (nman=19;nret=5) 0.13 0.40 efficient product replenishment (nmm=17; n,«=5) 0.05 0.10 scale: -1 = very disappointed; -0.5 = disappointed; 0 = neutral; 0.5 = satisfied; 1 = very satisfied table iv. mean of the satisfaction level with the ecr initiative by company type implementation problems a number of implementation problems have also been identified from the survey. table v depicts the top six problems encountered by manufacturers and retailers. manufacturer (n=19)* barriers conflicting priorities for resources shortage of personnel with necessary skills lack of clear 'roadmap' inflexible information systems reluctance of trading partner to share information 'category managers' insufficiently trained mean 3.24m 3.05 2.94 2.94 2.83 2.67 retailer (n=6)* barriers shortage of personnel with necessary skills inflexible information systems functional territory issues resistance to change inaccurate/inappropriate performance measures conflicting priorities for resources mean 3.20 3.00 3.00 2.83 2.75 2.33 m scale: 1 = has not been a problem; 2 = minor problem; 3 = moderate problem; 4 = major problem * missing value exists table v. the top six barriers to ecr implementation ranked by the importance the similarities in the responses indicate that both groups experience 'conflicting priorities for resources', 'shortage of personnel with necessary skills', and 'inflexible information systems' as three of the major barriers to ecr implementation. the differences in the responses indicate that manufacturers and retailers have different implementation concerns. other major problems indicated by manufacturers are related to the pressure enforced by retailers. they include the problem of lack of a clear roadmap in their ecr implementation and lack of training for their category managers. the problem of information sharing experienced by manufacturers indicates that retailers still have low level of trust in manufacturers. this last problem is consistent with the problems of 'resistance to change' faced by retailers. for retailers, by contrast, the other barriers are all internal barriers, such as 'functional territory issues' and 'inaccurate / inappropriate performance measures'. attitude of management participants were asked to describe the overall attitude toward ecr, of various management functions, 12 months ago and currently, based on a five-point scale: from very negative (-1) to very committed (1). the analysis indicates that in general, over the last 12 months, there had been an improvement in the attitude of the management functions for both groups, indicating that they were becoming more convinced and enthusiastic about ecr. the results also suggest that, in general, the management functions of retailers were more committed than those of manufacturers. the difference between the two groups was significant at a 5% level for the it management, with p-value = 0.051. this finding reinforced the previous findings, which indicated that retailers were leading manufacturers in ecr implementation in australia. interestingly, the findings demonstrated that the marketing / merchandising function of retailers had very low commitment. the difference between the two groups was significant at a 5% level, with p-value = 0.045. the probable reason for this is that this management function is the one that interacts directly with manufacturers, who, in general, are not very enthusiastic about ecr. 118 the consequences of adoption respondents were asked to indicate how ecr affected each performance measure provided in the questionnaire (see table vi). the survey results showed some improvements in a number of performance measures. for manufacturers, the most noticeable improvement was identified in sales, customer satisfaction and marketing. the difference between manufacturers and retailers in cost reduction for marketing was significant at a 5% level (p-value = 0.041). small reductions in the finished goods inventory and administrative costs were also experienced by the manufacturers. * manufacturer (n=19) * retailer* ; (0=6)* increase is expected ' ... sales / turnover profits gross margin return on inventory investment warehouse fill rate labour productivity dollar sales per square foot profit margin customer satisfaction market share 0.22 0.27 0.13 0.08m 0.14 0.25 0.07 0.43 0.17 0.00 0.40 0.40 0.60 m 0.00 0.33 0.40 0.40 0.30 decrease is expected :" ' ; ' :v : ' . ' '""j '•';•• '?-l . . ' ; — ' • " " ' . :v. • . ' . ' ' . ' out-of-stocks finished goods inventory invoice costs raw material costs packaging costs manufacturing costs purchasing costs warehousing costs transport costs marketing (promotion) costs administrative costs -0.21 -.0.25 -0.10 -0.05 -0.05 -0.08 -0.05 -0.17 -0.21 -0.06 m -0.03 -0.30 -0.20 -0.10 -0.33 -0.17 0.00 -0.30 -0.20 -0.30 0.30 m 0.20 •increase :6r decrease,: depending on the business strategy variety of products number of skus category space allocation in store 0.00 0.00 0.21 -0.10 -0.30 0.12 scale: -l=reduced by over 20%; -0.5=reduced by up to 20%; 0=no change; 0.5=increased by up to 20%; l=increased by over 20%. * missing value exists for some performance measures. table vi. changes in performance measures by company type for retailers, the most salient improvements were identified in profit, warehouse fill rate, dollar sales per square foot, inventory level, customer satisfaction and market share. in addition, retailers experienced reductions in various costs such as for raw materials, purchasing, transport, warehouse, administrative, and marketing. thus, consistent with the idea that the retailers were leading ecr implementation in australia, these findings suggest that the retailers experienced more benefits than manufacturers from ecr. respondents were also asked if they had experienced any negative consequences of adopting ecr, as shown in table vii. for manufacturers, 'more costs involved' was the most frequently cited consequence, followed by 'lost of sales due to out-of-stock' and 'employee morale problem'. for retailers, diverse responses about the consequences were given. these findings suggest that ecr implementation incurs more additional costs at the manufacturer side than at the retailer side. the most frequently cited negative consequences by retailers are 'employee morale problem'. this is consistent with the previous finding that indicates 'resistance to change' is one of die major implementation problems faced by retailers. this suggests that retailers are implementing ecr enthusiastically that some employees experience difficulties in coping with the changes in work practices proposed by ecr. 119 negative consequences ; • . . ' ' • v losing key personnel more costs involved employee morale problem diminished customer service level losing valuable trading partners lost sales due to out-of-stock "manufacturer (n=l9)r: ' • > < " . percent 53 5 16 rank 1 3 2 retailer (n=6) 'percent 17 17 33 17 17 rank 2 2 1 2 2 table vii. negative consequences by company type the perceived characteristics of ecr respondents were asked to describe ecr in terms of its relative advantage, compatibility with their needs, trialability of ecr components, observability of ecr benefits, complexity, switching costs, and perceived risks. table viii summarises the mean of the perceived characteristics of ecr by manufacturers and retailers. relative advantage compatibility trialability observability complexity switching costs perceived risks manufacturer (n=19) 2.53 3.16 3.11 2.63 3.39 2.95 2.68 retailer (n=6) 3.00 2.67 2.50 3.33 3.33 3.33 2.67 scale: 0 = not relevant; 1 = very low; 2 = low; 3 = medium; 4 = high; 5 = very high table vm. mean of the perceived characteristics of ecr the findings demonstrate that both groups agree that ecr is a rather complex program, but the risks of getting involved in the program are low. the perceived relative advantage of ecr is higher for retailers than manufacturers. consistently, retailers can observe the benefits of ecr more clearly than manufacturers. these results reinforce that idea that retailers are leading manufacturers. interestingly, manufacturers' perceptions of the compatibility of ecr and the trialability of ecr components are higher than retailers and retailers perception of the switching costs for ecr is higher than manufacturers. conclusions this study has provided additional insights towards understanding of the slow diffusion rate of ecr, by surveying the australian grocery industry. while the small sample size meant that many differences observed in the data were not statistically significant, the accumulation of evidence favours the research hypotheses. as expected, the survey results demonstrate that the major driving force for manufacturers to get involved in ecr is pressure from their retailer trading partners. retailers, on the other hand, are interested in embracing ecr in order to improve their business performance. consequently, this affects the implementation level of ecr components among manufacturers and retailers. most manufacturers in our study have implemented some ecr components as a stand-alone system and they show little interest in improving supply chain management following the holistic approach of ecr. many manufacturers plan to adopt activity based costing in order to gain a better understanding of what they can achieve from ecr, before embracing the whole idea. by contrast, retailers are more enthusiastic in improving the efficiency of the supply chain as a whole using the concept of ecr. a later case study provided an explanation of this observation, which revealed that manufacturers gained the least of ecr benefits and, therefore, were particularly concerned with unequal distribution of ecr costs, benefits and risks within the supply chain (kuraia and johnston 2001). this study also suggests that the level of satisfaction with ecr initiatives is low among manufacturers. the later case study analysis explained that while most manufacturers do not perceive a need to embrace ecr, they must do so if they did not want to lose their major customers. by contrast, retailers have perceived real needs to change their way of business and, thus, have gained more benefits than manufacturers in implementing ecr. this, in turn, affects the satisfaction level between the two groups. in addition, the attitude of various management functions of retailers and manufacturers and their trading partners towards ecr indicated in this study reinforces the idea that retailers are driving the ecr implementation in australia. 120 in relation to the future of ecr adoption in australia, it should be recognised that ecr is an inter-organisational innovation. in order to gain the maximum potential benefits of ecr, all participants within the supply chain need to collaborate to achieve the goals of ecr. in australia, this kind of collaboration has not been achieved, as indicated by the survey results in general, and in particular by reluctance of retailers to share information with their trading partners, despite the fact that they are the ones who encourage the trading partners to get involved in ecr in most cases. in australia, the major reason for not being involved in ecr is found to be lack of understanding of the concept. this suggests that the grocery industry supply chain committee of australia, which was established hi 1996, needs to organise some conferences or seminars on ecr, or distribute publications on ecr to educate the industry. in addition, retailers need to educate their trading partners to ensure that the trading partners are well informed about the benefits of involvement in ecr. without good partnership, ecr will fail to achieve its ultimate objectives, and this, will in turn, result in slow diffusion rate of ecr. this study is part of an on-going research on diffusion of inter-organisational systems in general. many of the findings of the survey reported hi this paper have been confirmed and further elaborated through case study research (kurnia and johnston 2000; 2002). this project, however, will continue further by looking at supply chains as opposed to individual organisations as unit of analysis to complement the partial insights obtained from the survey into the nature of barriers to ecr diffusion, the perception of ecr and realisation of ecr benefits hi australia. references andel, t. (1994) "define cross-docking before you do it", transportation and distribution, 35(11), pp 9398. anonymous (1995) "account-specific marketing: working together (really)", promo/progressive grocer special report, february, p 5. coopers and lybrand (1997) "efficient consumer response europe: value chain analysis project overview", http://www.coopers.co.uk/managementconsulting/ecr/, last access: march 1998. coopers and lybrand (1998) the grocery industry supply chain committee: 1998 tracking study, grocery manufacturer of australia ltd. and australian supermarket institute, sydney, australia. coupe, k. (1995) "innovation is the key", progressive grocer, january, pp 29-33. ean australia (1997) ean numbering and barcoding of non-retail items: including an introduction to ean application identifiers and ean-128 barcodes, ean australia, melbourne, australia. ecr central (1997) "efficient consumer response", http://www.ecr-cenral.com/. last access: december 1997. emmelhainz m a (1990) electronic data interchange: a total management guide. van nostrand reinhold, new york. fowler, f. j. (1988) survey research methods, sage publications, inc. london. greenbaum, j. m. (1997) "efficient consumer response: how software is remaking the consumer packaged goods industry", software magazine, 17, pp 38-48. harris j. k., and swatman, p. m. c. (1997) "efficient consumer response (ecr): a survey of the australian grocery industry", proceedings of the 8th australasian conference on information systems, university of south australia, adelaide, pp 137-148 ibm (1997), 1997 european retail benchmark report ibm, london. information advantage (1996) "what is category management", http://www.categorvmanaeement.co/whatis.html. last access: april 1997. jackson, w. (1988) research method: rules for survey design and analysis, webcom ltd. canada. kurnia, s., swatman, p. m. c., schauder, d. (1998) "efficient consumer response: a preliminary comparison of u.s. and european experiences", proceedings of the eleventh international bled electronic commerce conference, bled, slovenia, pp 126-143. kurnia s and johnston, r. b. (2000), the need for a processual view of inter-organizational systems adoption", journal of strategic information systems, 9(4), pp. 295-319. kurnia, s. and johnston, r.b. (2001). "adoption of efficient consumer response: the issue of mutuality", supply chain management: an international journal, 6(5), pp 230-241. kurnia, s. and johnston, r. b. (2002), "adoption of efficient consumer response in australia: key issues and challenges", supply chain management: an international journal, (in press). kurt salmon associates (1995) ecr 1995 progress report, joint industry project on efficient consumer response, kurt slamon associates, new york. kurt salmon associates (1997) ecr europe annual tracking survey, kurt salmon associates, new york. landry, s. p., wood, l. m. and lindquist, t. m. (1997) "can abc bring mixed results?" management accounting, 78(9), pp 28-33. leggett, d. (1996) "i know what i want....and i want it now", marketing, pp 20-21. 121 lucas, h. c. (1991) "methodological issues in information systems survey research", in kraemer, k. l., the information systems research challenge: survey research methods. harvard business school, boston, pp 273-285. martin, a. (1994) infopartnering, oliver wight publication, inc. essex junction, vt. retail world (1998) grocery industry marketing guide 1998, retail world pty ltd, north paramatta, nsw. robin, g. (1994) "sailing into ecr's enchanted waters", stores, 76(10), pp 43-44. thayer, w. (1995) "how crp cut inventory, sped turn and grew sales", frozen-food-age, 43, pp 1,16,17. tripplet, t. (1995) "more u.s. grocers turning to ecr to cut waste", marketing news, 3, pp 12-13. wheatley, m. (1997) "efficient consumer response: minding the stores", cio, section 1, pp 70-78. wood, p. (1996) "preparing for ecr at the store level", chain store age executive, 5, may, p 230. acknowledgements the authors would like to thank professor paula swatman and professor don schauder for their contributions in the survey administration. 122 ajis vol. 8 no. 1 september 2000 incorporating database design in warmer method donald r. chand cis department bentley college waltham, ma 02452 dchand@bentley.edu abstract the warnier method, a highly prescriptive program design approach for file-oriented solutions, has been criticized for its lack of a database design component. this paper addresses this weakness by incorporating a logical database design step in warnier method. specifically, the paper presents rules for transforming the information in a warnier diagram into a set of relations. with this extension the wamier method complements the entity-relationship approach for data analysis and logical database design. introduction warnier [(1974), (1978), (1981)] design method, first introduced as logical construction of programs (lcp) and later elaborated as logical construction of system (lcs), is a highly prescriptive approach of program design for file-based, report-oriented solutions of data-processing problems. because it is highly prescriptive, different programmers produce almost identical programs for the same problem. wamier method is also self-documentary because the design is documented as the prescribed steps of warnier method are applied. these two attributes, self-documentation and highly prescriptive design steps, aid in synchronizing the mental and physical efforts of a programming team. despite these desirable attributes and outcomes, the warnier method was not widely used outside france. the major criticism of warnier method [mcclure & martin (1981)] is that it lacks a database design component, which is central in the design of business applications. this paper addresses this problem by incorporating in wamier method an additional step of transforming the data in a warnier diagram into a set of normalized relations. the transformation rules and the incorporation of logical database design considerations in warnier method are illustrated in section 3. section 2 reviews wamier method and section 4 discusses the broader implications this extension of warnier method. the warnier method j. d. warnier and his colleagues at honeywell-bull in paris, france developed a systematic approach for deriving structured programs from the logical data structures of the output reports. the premise of warnier approach is that it is necessary and feasible to derive the structure of the application program from the hierarchical structure of the data. it provides a graphical tool, called the wamier diagram, for modelling and documenting the data and program as a hierarchy. the design method consists of following five steps. stepl: model the layout of the output report into a hierarchical data structure and document this structure in a wamier diagram. stepl: using the structure of the output file developed in stepl, postulate a compatible input file. the compatibility of the input file ensures that sequential processing of the records in the logical input file yields the output records in the order prescribed on the report. model the data structure of the postulated compatible input file as a hierarchy in a wamier diagram. step3: using the hierarchical model of the compatible input file, conceptualised in step 2, derive a hierarchical program structure which is again documented in a wamier diagram. step4: using the output-report of stepl as a reference, discover the operations needed to produce the report. steps: allocate at the appropriate locations in the program structure of step3 the operations discovered in step4. the tool used for documenting the outcome of these steps is the warnier diagram. it is a hierarchy chart laid on its side, where each level of hierarchy is composed of sequence, selection and repetition structures, shown in figure (1). 17 ajis vol. 8 no. 1 september 2000 warnier diagram p (1) q (0,n) a (0,0 b (0,1) d 0,n) x (0,0 x (0,0 meaning s is composed of one occurrence of p and zero to many occurrences of q p is composed of either a or b but not a and b. a occurs zero or one time and b occurs zero or one time. q is composed of one occurrence of c and one to many occurrences of d. b is composed of zero or one occurrence ofx and zero or one occurrence of not x an illustration of warnier method figure 1 a simple single-output problem is used to illustrate the five steps of warnier method. the problem is to devise a program that processes the salary records of the full-time and part-time faculty at a small college to produce a department-wise summary report of faculty salaries. the layout of the prescribed report is shown in figure (2). department salary report dept. no. full-time total part-time total i i i i . ft-subtotal . total • pt-subtotal figure(2) stepl: model the output layout as a hierarchy and document the model in a warnier diagram. the logical data structure of the output report is shown in figure (3). 18 ajis vol. 8 no. 1 september 2000 report title (1) heading (1) (1) dept < (0,n) subtotals-line (1) total (1) dept no full-time-total (1) part-time-total (0 figure (3) step2: postulate a logical input-file whose data structure is compatible to the logical data structure of output report. a compatible input file and its data structure are shown in figures (4) and (5). compatible input file dept2 deptl / dept-no fac-name salary codefpt] j figure (4) 19 ajis vol. 8 no. 1 september 2000 input file structure input ) dept file (n) fac x (\ \ """̂ _ dept-no (1) fac-name (1) salary (1) code[p,(0,i)] (1) figure (5) program structure begin program< (1) program ^ process dept structure ^ (n) end program* begin dept (1) process fac end depfc (1) begin fac' (1) pt-fac (0,1) -c © pt-fac (0,1) end fac figure (6) steps: derive the program structure from the data structure of the logical input file. the program structure is shown in figure (6). 20 aj1s vol. 8 no. 1 september 2000 step 4: specify the operations needed to produce the prescribed output report. output operations print title print heading print dept-line print subtotal-line print total input operations read first record read next record processing update total update ft-fac-total update pt-fac-total update ft-subtotal update pt-subtotal initialisation initialise total initialise subtotals initialise ft-fac-total initialise pt-fac-total step 5: allocate the operations to the program structure in figure (6). program of figure (7). this yields the structured faculty salary summary program program begin program(1) print title; print heading initialize subtotals, total readfirst record process dept (n) begin depu (1) initialize ft-fac-total, _pt-fac-total process fac end program print dept-line update ft-subtotal update pt-subtotal begin fac (1) pt-fac \updatept(0,1) yac-total © update ftfac-totalpt-fac (0,1) end fac — (1) read next record enddept!— (1) print subtotal-line update total print total figure 7 the above illustration of warnier method reveals both its strength and weakness. the strength is that it systematizes the program composition process to a point where most designers using this method will produce almost identical solutions. the criticisms are: 1. it is not readily applicable to applications with multiple input and output files. 2. it does not address situations where the logical input file structure does not match the structure of the physical input file. 3. it provides no guidance for database design. the first criticism is addressed by partitioning the application program into a set of one-output programming problems. if n is the number of output reports of the application, the application design is split into n one-output programming problems. treating every output report as a programming problem and applying warnier method to it will yield n logical input files. these n logical input files form the 21 ajis vol. 8 no. 1 september 2000 database for the application. in summary, this design approach of partitioning of the original application produces a modular solution consisting of • n one-output programs, • a database of n logical input files, and • programs that map the given physical inputs of the application to the n logical input files of the database. the second criticism is that the method does not address the case when the logical input file is not compatible with the physical input file. this non-compatibility of the physical and logical files is essentially the structure-clash problem, recognized and addressed in jackson structured program (jsp) methodology (1975). jackson showed that when the logical file structure is not compatible with the physical file structure, then the problem should not be handled in a single program. he recommends that the problem should be partitioned into two sub-problems. one sub-problem is generating the specified output from the logical file structure, and the second sub-problem is converting the physical file structure to the logical file structure. the first sub-problem is directly addressed by the warmer approach illustrated above. this second subproblem can be addressed by breaking the physical data structure into its most elementary parts and then rebuilding the logical structure. jackson suggested that the same result can be achieved more efficiently by breaking the physical file structure into the largest common data structure that is common to both the physical and logical data structures. , the third criticism of lacking guidelines for database design is a short-coming of the original warmer method, and it is addressed next. it turns out that the approach of partitioning the application design into one-output programs provides a basis for addressing this third criticism. a common business application is used in section 3 to illustrate the integration of a database design component in warmer method. incorporation of a database design step in warnier method a database design step is added to the warmer method after the data structure of the compatible input file is modelled in a warnier diagram. in the database design step, the information in the warmer diagram is transformed into a set of normalized relations. thus, the initial two steps of warnier method are revised as follows. stepl: sketch the layout of each output report and analyse each report layout to design its compatible input file. model the data structure of each compatible input file in a warnier diagram. step2: apply the entity identification, key selection and key synchronization rules to transform the compatible input file information into a set of normalized relations. these rules are introduced and illustrated below. the inputs to an expense-accounting system [chand & yadav (1980)] are checks written to pay for expenses. each check contains following information: check number date of check payee name amount of check expense note (this occurs 1-20 times per check and contains expense description, expense amount, and accounting code consisting of product line number, cost center number, and expense account number) 22 vol. 8 no. 1 september 2000 expense account report ea# ea-desc prev-ytdea-exp i i i i i i curr-mthea-exp i i i i i i updtd-ytdea-exp l i i l l l , i i i 1 i i i prev-ytd-ea-total curr-mth-ea-total , updt-ytd-ea-total , figure (8) cost center expense account report cc# cc-desc ea# ea-desc prev-ytdcurr-mthupdtd-ytdcc-ea-exp cc-ea-exp cc-ea-exp j l i i j l j l j l prev-ytd-cc-total , curr-mth-cc-total , , updt-ytd-cc-total figure (9) 23 ajis vol. 8 no. 1 september 2000 product line expense account report pl# pl-desc cc# cc-desc ea# ea-desc prev-ytd-pl curr-mth-pl updtd-ytd-pl -cc-ea-exp -cc-ea-exp -cc-ea-exp i i j l l j l j i j l j i j l j i j l j i prev-ytd-pl-total curr-mth-pl-total j updt-ytd-pl-total figure (10) the data structure of the logical files compatible to the three reports in figures (8), (9), and (10) are respectively presented in figures (11), (12), and (13). ea-input file structure exp-acc input file ea# (1) ea-desc (1) prev-ytd-ea-exp (1) curr-mth-ea-exp (1) figure (11) 24 aj1s vol. 8 no. 1 september 2000 cc-input file structure cost-center input file cc# cc-desc (1) cc-exp-act/1 ea# (1) ea-desc (1) prev-ytd-cc-ea-exp (1) curr-mth-cc-ea-exp (1) figure (12) pl-input file structure pl# (i) pl-desc ( i ) cc# (1) cc-desc (1) prod-line^ input file cost-center < pl-cc-exp-act<: ea# (i) ea-desc (1) prev-ytd-pl-cc-ea-exp ( i ) curr-mth-pl-cc-ea-exp cd figure (13) database design step this section presents and illustrates rules for transforming the information in the compatible input-file structures into a set of normalized relations. the rules aid in identifying database entities, specifying primary keys, and synchronizing entity keys across the file structure hierarchy. entity identification rule: scan the warnier diagram sequentially from the right side. collect all the data items in the right most hierarchy into an entity. remove the rightmost hierarchy and the corresponding data item in its adjacent hierarchy from the warnier diagram. repeat this process until there are no more hierarchies in the warnier diagram. 25 ajis vol. 8 no. 1 september 2000 the application of the entity identification rule to the file structure in figure (11) yields one entity, which is shown below with its attributes. exp-acc (ea#, ea-desc, prev-ytd-ea-exp, curr-mth-ea-exp) the entity identification rule, applied twice to the file structure in figure (12), yields the following two entities. cc-exp-acc(ea#, ea-desc, prev-ytd-cc-ea-exp, curr-mth-cc-ea-exp) cost-center(cc#, cc-desc) the entity identification rule is applied three times to the file structure in figure (13) resulting in the following three entities. pl-cc-exp-acc(ea#, ea-desc, prev-ytd-pl-cc-ea-exp, curr-mth-pl-cc-ea-exp) cost-center(cc#, cc-desc) prod-line(pl#, pl-desc) key selection rule: identify the candidate keys in each entity definition and select as primary key among the candidate keys the one that is most meaningful to the user. the application of the key-selection rule updates the above entity definitions as follows. figure (11) entity: exp-acc (ea#. ea-desc, prev-ytd-ea-exp, curr-mth-ea-exp) figure (12) entities. cc-exp-acc(ea#. ea-desc, prev-ytd-cc-ea-exp, curr-mth-cc-ea-exp) cost-center(cc#. cc-desc) figure (13) entities. pl-cc-exp-acc(ea#. ea-desc, prev-ytd-pl-cc-ea-exp, curr-mth-pl-cc-ea-exp) cost-center(cc#. cc-desc) prod-lined^ pl-desc) key synchronization rule: scan the hierarchy of the warmer diagram sequentially from the left. define the key of the outer most hierarchy as the nest key. add the nest key to the entity corresponding to the next adjacent hierarchy. if there is a many-to-many relationship between these two entities, form a composite key by adjoining the nest key to the key of the adjacent hierarchy. otherwise, the nest key is added as a foreign key to the entity corresponding to the inner hierarchy. remove the outermost hierarchy, and repeat the process until there is no hierarchy [batra (1997)]. the key-synchronization rule does not apply to the hierarchy in figure (11). therefore, there is no change in the following relation. exp-acc (ea#. ea-desc, prev-ytd-ea-exp, curr-mth-ea-exp) the key-synchronization rule is applied once to the hierarchy in figure (12). since there is many-tomany relationship between cost-center and cc-exp-acc entities, the key-synchronization rule yields the following updated relations. cost-center(cc#. cc-desc) cc-exp-acc(cc#. ea#. ea-desc, prev-ytd-cc-ea-exp, curr-mth-cc-ea-exp) the key-synchronization rule is applied twice to the hierarchy in figure (13). since there is many-tomany relationship between prod-line and cost-center entities, the nest key, pl#, is concatenated with the cost-center key, cc#. the product line hierarchy is removed. the nest key is now the composite key, pl# cc#. once again there is a many-to-many relationship between cost-center and pl-cc-expacc entities, the nest key is concatenated to the pl-cc-exp-acc entity.key. the application of the synchronization rule yields the following updated relations. prod-line(pl#. pl-desc) cost-center(pl#. cc#. cc-desc) pl-cc-exp-acc(pl#. cc#. ea#. ea-desc, prev-ytd-pl-cc-ea-exp, curr-mth-pl-cc-eaexp) 26 ajis vol. 8 no. 1 september 2000 eliminating the traditional partial and transitive dependencies, and renaming one of the entities, results in the following set of third normal form relations. exp-acc (ea#. ea-desc, prev-ytd-ea-exp, curr-mth-ea-exp) cost-center(cc#. cc-desc) cc-exp-acc(cc#. ea#. prev-ytd-cc-ea-exp, curr-mth-cc-ea-exp) prod-line(pl#j pl-desc) cost-centertpltf, ccj) pl-cc-exp-acc(pl£ cc#. ea#. prev-ytd-pl-cc-ea-exp, curr-mth-pl-cc-ea-exp) this completes the illustration of a database design step in warnier method. implications the incorporation of a database design step frees warnier method from the stigma and perception that it is not suitable for designing solutions of large business applications and systems. as illustrated above, a large business application can be partitioned into a set of one-output programs and the schema for these one-output programs can be merged to define the database. the extended warnier approach presented here can serve as an effective bottom-up approach of developing a normalized database for an application. in addition, it can complement the traditional entity-relationship (e-r) approach. the entity-relationship approach is a popular and very successful approach for building data models during the analysis phase of the systems development life cycle. it has been found that the entity-relationship approach forces the analyst to acquire a deeper understanding of the business entities and business rules and, consequently, uncover requirement errors earlier in the system development life cycle. however, the conversion of the entity-relationship data model into a normalized relational schema does not take into account the processing requirements. since the warnier approach forces the designer to explicitly address each user output requirement in the development of a logical database, it can be used as a means of validating whether the entityrelationship model has missed any attributes or relationships needed by the user. there are other benefits of this approach. first, it provides a problem structuring approach where the problem is partitioned into program modules and each module produces a specific user output. second, since the logic and code of the program is derived from explicitly exploiting the data structures of the outputs using a standard self-documenting process, the approach synchronizes the mental and physical efforts of a team of software developers. in our judgement, this approach is an excellent approach for teaching and learning programming and bottom-up data base design. since the warnier method was invented in the 1970s, it is important to ask whether it has any role to play in the modern computing environment of electronic commerce, distributed computing and object orientation. we have reflected on this issue and we present our views as follows. first, it is important to keep in mind that there are three components of software design, namely process design, data design, and interface design. the advancements in the development and execution technologies and the changes in application domains often thrusts one or the other component of software design as being more prominent in terms of an area of study leading to the development of better theories and methods of practice. for example, today's focus on electronic commerce applications has made the interface design component more prominent. but this does not mean that process and data design have faded or become less important. in our opinion, process, data and interface designs are the three legs of the software stool and, all contributions to further the theory or practice in any one of these three areas are equally important. the extended warnier approach presented here furthers the understanding and practice of the data design component of software development. second, object-orientation is the dominant approach for developing systems software and commercial software products and it is also diffusing, although slowly, in the business applications arena. at present, in the information systems world the primary use of object technology is in the area of interface design, whereas the back-end technology is still process-based and is dominated by relational database technology. clearly, the extended warnier approach presented here is directly applicable to the current back-end technology. the question is what role warnier approach will play in the object world. at this time we can only say that just as this paper extends the original warnier approach to address the database design problem, we are hopeful that future enhancements of warnier approach with accommodate object solutions. 27 ajis vol. 8 no. 1 september 2000 references batra, d. (1997), "a method for easing normalization of user views," j. mgt. info. sys. vol.14, no. 1, pp. 215-233. chand, d. r. & yadav, s.b.(1980) , "logical construction of software," comm. of acm, vol. 23, no. 10, pp. 546-555. jackson, m. a.(1975), principles of program design, academic press, london, mcclure, carma & martin, james (1995), structured techniques, prentice hall warnier, j. d.(1974), logical construction of programs, van nostrand reinhold, n.y. warmer, j. d.(1978), program modification, martinus nijhoff warnier, j. d.(1981), logical construction of systems, van nostrand reinhold, n.y. 28 microsoft word 13 1 marina.doc ajis vol 13, no. 1 september 2005 150 securing xml query processing storage charles a. shoniregun school of computing and technology, university of east london longbridge road, dagenham, essex, rm8 2as, uk e-mail: c.shoniregun@uel.ac.uk oleksandr a. logvynovskiy business, computing, and information management (bcim), london south bank university borough road, se1 0aa, uk. kevin lu department of information systems and computing brunel university uxbridge ub8 3h, uk abstract the effective processing of semi-structured data queries is a preliminary part of data mining stage. xml queries employ regular path expressions to find structural patterns within xml documents. the operation of structural join is a crucial part of xml query processing. existing approaches reduce complex join expressions to several binary structural joins. in this paper, we are proposing a new structural join algorithm called sequence join algorithm, for sequential regular path expressions in securing xml query processing storage. it exploits information about position of the elements in a product to skip generation of the redundant intermediate lists. this paper further discusses the algorithm that performs the merge of several input lists of nodes in one pass. we carried out comparative experiments, and the results prove that the algorithm is better than multiple binary joins algorithm for queries of both small and large cardinality. keyword: semi-structured data mining, data model, indexing xml data, sequence algorithm, graph numbering, and query processing. introduction each industry requires certain safeguards to protect its data while in transit. bringing autonomic capabilities to storage systems would certainly be an improvement, but if computing systems that mine data in those storage repositories become next to impossible to manage, that partial automation will not yield much benefit. xml is now becoming a standard to represent and exchange semi-structured data over the web (xyleme 2001, bray et al 2000). the problem of storing xml data in one or several tables is challenging, since the xml tree describe some irregular structure while tables are by definition regular. in a situation where the xml document has no schema, or when the schema changes frequently, it has a more dramatic impact on performance. the key idea behind these structures is called partial schema. the partial schema helped to build a concise graph representing paths of the data (wang & liu 2001). it also served as a guideline for building indexes and views or a starting point for structure-based document clustering based on approach proposed by deutsch et al (1999) that uses the xml instance to infer a relational schema. the idea is to find regularities that may exist in a given xml data instance, and to organise the storage base on those regularities (deutsch et al 1999, cooper et al 2001). the challenge in any storage schema is that it has to be flexible enough to accommodate ajis vol 13, no. 1 september 2005 151 data, and efficient as regular data storage. the number of accessible xml documents tends to grow as more and more business are storing and interchanging data between applications using xml as a common format. semi-structured data representation we can represent semi-structured document as a graph or as a tree. the tree model reflects nesting of elements within xml file and treats reference elements (idref) as values. the graph model resolves semantics of reference elements and, thus, allows an element to have multiple parent elements. when semi-structured data is represent as a label, directed, and possibly cyclic graph, the vertices of such a graph correspond to objects that are either treated as containers for some other objects or associated with atomic values (such as text, multimedia content, etc.). edges of the graph stand for containment relationship between nodes and have object types as labels. an example of such a model is the object exchange model (oem) (papakonstantinou et al 1995), in this model, each xml element becomes an edge (labelled with the tag name) and directed towards an individual node. each node corresponds to an xml element and has element’s id as its label. each edge in the xml data graph has a label and a target, where the target is either a node representing a scalar data value (e.g. a string or integer) or a reference to an element node in the document via its id. the edge belongs to a class, which can be one of sub-element, attribute, or idref (id reference). both idref and sub-element edges always direct towards element nodes in the graph; attribute edges. every element node has precisely one incoming edge of sub-element type and any other incoming edges must be of idref type. if all idref edges of an xml data graph are converted into attribute edges (with destination value equal to the target node id), the data graph can be mapped into a tree. each element of the document forms a node in the tree labelled with the element type (tag name) and value. the edges of the tree stand for parent-child (containment) relationship between the elements. all sub-elements nested within an element appear in the tree as the child nodes directly connected with the edges to a parent node. the attribute in the elements represents in a nested sub-elements and form additional nodes in the tree, emanating from the associated parent nodes. pattern specification language a number of languages have been proposed for querying semi-structured and xml databases, which includes following xquery, lorel, xml-ql, and unql. a common characteristic of all existing language proposals is the existence of a pattern specification language e.g. xpath, which built around path and sub-tree expressions. these expressions replace the traditional sql form clause and enable selections based on value predicates as well as path navigation and branching through the xml data graph in order to reach relevant data elements. path queries are popularised in the context of object-oriented databases, while the pattern specification language proposed for xml data are substantially more complex. in particular, the xpath language, xquery, and xslt, is the dominant w3c language proposals for xml querying and transformation, which allows branching of other regular path expression to enable queries navigation along the paths of data that uses label names, wild cards, value predicates and branching predications on the existence of specific product paths. the key idea underlying the implementation of the existing join algorithms is the decomposition of the original query path expression into a set of simple (binary) path ajis vol 13, no. 1 september 2005 152 expressions. each binary expression produces an intermediate join result, which is used on the subsequent stage. the xiss system introduces three join algorithms: element-attribute (ea-join), element-element (ee-join), and kleene-closure (kc-join). the element-attribute algorithm joins two intermediate results from sub-expressions, which are a list of elements and a list of attributes. the element-element algorithm joins two lists of elements. the principal difference between these algorithms is that the latter one checks ancestordescendant relationship between each pair of the input lists while the former one tests parent-child relationship. the kleene-closure algorithms iteratively uses element-element algorithm to compute closure of the expression. it repeatedly applies ee-join to the result from the previous stage of iteration. both ea-join and ee-join algorithms have a loop over one input list nested into a loop over another list and, therefore, have time complexity o(|e1|·|e2|), which is quadratic in the size of the input lists. as kc-join depends upon eejoin, it has quadratic time complexity either. however, structural join algorithms proposed by al-khalifa et al (2002) exploit the advantage of element numbering to decrease the time of processing (al-khalifa et al 2002). the tree-merge join algorithm is an extension of relational equality merge join performed on sorted inputs. the time complexity of the tree-merge join is non-quadratic o(|e1|+|e2|), but may include multiple passes over the same input set of descendant nodes. to avoid this problem, the second of the proposed algorithms, stack-tree join algorithm, utilises stack of nodes and has time complexity o((|e1|+|e2|)/b), where b is the blocking factor. however, semi-structured data imposes new challenges for parallel algorithms and requires new methods. data model in representing semi-structured data, we use a graph data model labelled pseudo graph g = {v, a, l}, where v = {v1, … , vn} is a non-empty finite set of vertices, a = {(vi, vj)|vi, vj∈v} is a finite set of ordered pairs of vertices called arcs, and l = {l1, … , lk} is a set of labels ascribed to vertices and/or arcs. such definition of the database assumes that a graph can have loops and multiple arcs among its vertices. from the perspective of the database, a vertex of the graph is an object of the database and an arc is a relation. the data graph has an implicit order of its nodes obtained by the graph traversal. in order to make most of the graph numbering, we map original data graph into a directed cyclic graph. one of the important properties of the directed cyclic graph is that it has a cyclic ordering of nodes. we exploit this property to define a position of every node in the graph. each object and relation within the database forms a node of the numbering graph. the edges of the graph stand for relationships between both of these elements, objects and relations. relations that form cycles are reversed and relabelled. position is an important characteristic of graph nodes and intensely used for indexing and querying semistructured data. the position of the node ni is denoted as (di, si, ei, li), where di is the graph component identifier within the database; si, ei are distinct graph ordering numbers of the node ni (pre-order and post-order respectively), and li is the nesting depth of the node ni within the graph. the ancestor-descendant relationship gives a graph node of ni, and position (di, si, ei, li). the graph node nj and its position (dj, sj, ej, lj), the node ni is an ancestor of the node nj (and node nj is a descendant of the node ni) if di = dj (both nodes belong to the same component), si < sj and ei > ej (ancestor-descendant) or si > sj and ei < ej (descendant-ancestor). intermediate nodes nx and their positions (dx, sx, ex, lx) that constitute path between the two nodes are select to meet the criteria: ajis vol 13, no. 1 september 2005 153 dx = di = dj, si < sx < sj and ei > ex > ej (node ni is an ancestor of the node nj); or dx = di = dj, si > sx > sj and ei < ex < ej (node ni is a descendant of the node nj). the sibling relationship gives graph node ni and its position (di, si, ei, li). the graph node nj and its position (dj, sj, ej, lj), the node ni is a sibling of the node nj (and node nj is a sibling of the node ni) if di = dj (both nodes belong to the same component), si < sj and ei < ej (preceding) or si > sj and ei > ej (following). the path between these two nodes is selected with respect to one of the common ancestors of nodes ni and nj, nc and its position (dc, sc, ec, lc): dc = di = dj, sc < min(si, sj) and ec > max(ei, ej). then intermediate nodes nx and their positions (dx, sx, ex, lx) are select to meet the criteria: dx = dc, si < sx < sc and ei > ex > ec or sc < sx < sj and ec > ex > ej (ni precedes nj); or dx = dc, sj < sx < sc and ej > ex > ec or sc < sx < si and ec > ex > ei (ni follows nj). finding a relationship between a pair of nodes is the core operation of the semi-structured data query processing. graph numbering is the efficient way to determine it fast. nevertheless, the scale of the real-world data challenges its wide application. as one of the solutions to overcome scale problem, we use graph layering as shown in figure 1. figure 1. layered data view the original data graph is processed in order to determine its strong components. a strong component d of a graph g is a sub graph of every node in d and the removal of that node does not make d none connected. a node that ruins connectivity of the graph is a median. nodes within a strong component are always connected. nodes representing strong components of the graph along with medians form sets of trees. these trees are the next layer with its separate numbering. every node of the original graph ascribed to the pair of positions, within the component tree. the procedure of finding whether a couple nodes are connected or not takes two steps: a) determine if their component are connected and if they are then b) determine their relationship within the component. graph numbering and query processing we have proposed the graph numbering using indexing for querying connections among data that exploits the concept of node position to merge several input lists in one pass. as it data graph component tree ajis vol 13, no. 1 september 2005 154 processes several binary structural relationships that form a sequence, we call it sequence join algorithm. the basic idea of the algorithm is to synchronously read input lists to find first match of the node intervals and put it into the result list. if the intervals in two adjacent lists do not match, based on the result of their comparison, the record of one of the lists will be deleted. the propagation of changes goes from the last list back to first. the depth of the recursion is equal to the number of input lists, which relates to xml data tree input. for regular path expression a1/a2/…/an, both approaches require n selection operators resulting in lists of nodes a1, a2, …, an respectively. the execution plans for binary joins and sequence join are shown in figure 2 below. we consider the total time to perform join operation as the sum of time needed to read input lists (σ) and time necessary to create output list (τ): σ + τ. figure 2. execution plans of the binary and sequence joins the task of matching complex query reduces the performance of join operation for each binary structural relationship in query expression. for n input lists of nodes, it causes creation of intermediate lists of nodes. the next step is to perform binary join operation over the intermediate lists. the latter will be applied until the result is the only one, in result list. thus, the whole number of intermediate list (including the result list) is n – 1. in figure 2 a) these lists are denoted as an+1, …, a2n–1. the total time necessary to perform multiple pair wise join is: the τi and σi is the time required to create and read list i respectively. the sequence join reads input lists a1, …, an and project the result list a2n–1, as shown in figure 2 b). the total time necessary to perform sequence join is: the σi is time required to read input lists, and τ2n–1 is time needed to create the result table. … a1 a2 an+1 a2n–1 … an–1 an an+½n … … … a1 a2 a2n–1 an–1 an a) b) ajis vol 13, no. 1 september 2005 155 the parameters τi and σi depend on the capacity of the list i. if the number of nodes in each list is comparable then we can assume the times to create and read list are equal. the time differences between the two approaches in figure 2 above is illustrated in figure 3 below: figure 3. time differences however, the parameters of τi and σi depend on the capacity of the list i. if the number of nodes in each list is comparable then we can assume the times to create and read list are equal: ∀i, i = 1,…,n; σi = σ, τi = τ. time cost functions of the algorithms is: (σ + τ)(n – 2), (1a) where τ and σ is average time required to create an output and read an input list respectively, n is number of original input lists. the figure 4 shows the time cost graphs of sequence join algorithm performance. figure 4. time difference between the binary and sequence joins the key idea of our proposed framework is to consider the original database as a graph: • both objects and relations of the database are represented as graph nodes – this provides a unified way for their arrangement, indexing and storage. • the initial graph is direct and, generally, cyclic – we convert it into a directed cyclic graph by giving database relations status of nodes, then rearranging nodes so the graph is rooted. • every node of the graph ascribed with a couple of ordering numbers of the graph and post-order graph traversals analogous to a tree traversals. 1 2 3 4 5(σ+τ) (σ+τ) 2(σ+τ) 3(σ+τ) 4(σ+τ) (σn+τ) sequence join (2σ+τ)(n–1) binary join ti m e number of lists ajis vol 13, no. 1 september 2005 156 figure 5. framework of treating relational data as semi-structured figure 5 is performed has a pre-processing stage of storing new data or wrapping a legacy system. the query stage exploits graph numbering for fast selection of all intermediate nodes that constitute paths between nodes. prototype system we implemented a prototype system for storing, indexing, and querying connections among data. the system works as a wrapper for an existing relational database. the real world data set has been used as an input of the prototype. we used a subset of over 20,000 companies from a fame (financial analysis made easy) database. the fame database contains financial and statistical information about all companies in uk. we have limited our objects to company’s directors, shareholders, postcodes and ownership data. the prototype screenshot shows the sample query of the semi-structured data to find connection between a person and a company (see table 1 for sample queries). query xquery expression dataset rpe length q1 /fmdatabase/fccamradio/address/city ham-radio 3 q2 /play/act/speech/line shakespeare 4 q3 /country/province/city/name mondial 4 q4 xmark (100mb) 4 q5 /person/profile/interest/category xmark (1gb) 4 table 1. description and parameters of the test queries analyse db structure 1 create graph 2 apply numbering 3 select component path 1 select intermediate nodes 2 present as an expandable tree 3 storage stage query stage 1 5 2 4 3 ajis vol 13, no. 1 september 2005 157 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 q1 q2 q3 ti m e ( s ) sequence join multiple pair-wise join 20.0 40.0 60.0 80.0 100.0 120.0 q4 q8 ti m e ( s ) figure 6. query performance comparisons the cases with the query path length of more than 2, demonstrates improved performance of the sequence algorithm against the pair-wise algorithm (see figure 6). however, the selection of the result of all intermediate nodes that constitute paths between queries nodes are represented as a tree with expandable nodes. conclusion current approaches adopted the existing relational storages (and map semi-structured data into relational) or use native solutions. there exist hybrid solutions as well. since any input query may initiate both mining processes and storage/retrieval operations, it is necessary to define appropriate criteria and algorithms for splitting/joining results obtained on each level. we have developed the sequence join algorithm for regular path expressions. in contrast to pair wise approach, the algorithm takes several lists of elements as an input and exploits the position of the element within xml document to compute faster structural relationships between elements. the processing of xml documents may require a traversal of all document structure and, therefore, the cost could be very high. a strong demand for a means of efficient and effective xml processing has posed a new challenge for the q5 ajis vol 13, no. 1 september 2005 158 database world (shoniregun & logvynovskiy2004). therefore, the structural pattern are matched with available input lists at once but does not generate non-existent sub-results and hence eliminates creation of excessive intermediate data. references al-khalifa, s. jagadish, h.v. koudas, n. patel, j.m. srivastava, d. and wu, y.,(2002) “structural joins: a primitive for efficient xml query pattern matching”, in proceedings of the ieee international conference on database engineering (icde). bray, t., paoli, j. sperberg-mcqueen, c.m., maler, e., (2000) “extensible markup language (xml) 1.0 (second edition) ”, w3c recommendation. technical report recxml-20001006, available from http://www.w3.org/tr/rec-xml, (access date: 29 october 2004) cooper, b. sample, n. franklin, m. hjaltason, g. and shadmon, m. (2001) “a fast index for semistructured data”, in proceedings of vldb’01. deutsch, a. fernandez, m. and suciu, d. (1999) “storing semistructured data with stored”, in proceedings of sigmod conference papakonstantinou, y. garcia-molina, h. and widom, j. (1995) “object exchange across heterogeneous information sources”, in proceedings of the 11th international conference on data engineering. shoniregun, c. a. and logvynovskiy, o. (2004) “securing xml documents”, the australian journal of information systems (ajis), september, vol 11, pp 194-200. wang, k., liu, h.q. (2001) “mining is part of association patterns from semistructured data”, in proceedings of the 9th ifip 2.6 working conference on database semantics (ds-9), hong kong, april. xyleme, l. (2001) “a dynamic warehouse for xml data of the web”, in bulletin of the technical committee on data engineering, vol. 24, no. 2, june. . ajis vol. 8 no. 2 may 2001 information warfare and ethics m.j.warren w.hutchinson school of computing school of management information deakin university, edith cowan university, geelong, victoria, perth, western australia. australia. email contact: mwarren@deakin.edu.au abstract this paper examines the ethics of the practice of information warfare at both the national and corporate levels. initially examining the present and past actions of individual hackers, it moves to the more organised, future military and economic warfare scenarios. it examines the lack of legal or policy initiatives in this area. introduction during the last ten years there has been a growth of information systems and related internet technology within the developed world. in recent years the internet has grown from a solely military/academic network to one that can be used by business or individuals. in the years since the first www applications were developed, there has been an explosion in the global use of the internet. in that time we have seen the development and growth of electronic commerce and the first steps towards the development of a global information society (warren & hutchinson, 2000). we have seen a rise in computer misuse at a global level and also the development of new policies and strategies to describe organized computer security attacks against the information society these strategies are described as being 'information warfare'. this is very different to the traditional view of attack against computers by the individual, determined hacker, a cyber warrior with a code of conduct to follow. computer hacker the definition of the 'computer hacker' has been the subject of many debates in computing circles. caelli et al (1989) provide two definitions of the term: • those who take a delight in experimenting with system hardware, software and communication systems • unauthorised users who try to gain entry into a computer, or computer network, by defeating the computers access (and/or security) controls. in the contemporary world, the latter interpretation is by far the more common (although persons belonging to the former category of hacker would seek to more accurately define the latter group, particularly those with a malicious intent, as 'crackers'). hackers are by no means a new threat and have routinely featured in news stories during the last two decades. indeed, they have become the traditional 'target' of the media, with the standard approach being to present the image of either a 'teenage whiz kid' or an insidious threat. in reality, it can be argued that there are different degrees of the problem. some hackers are malicious, whilst others are merely naive and hence, do not appreciate that their activities may be doing any real harm. furthermore, when viewed as a general population, hackers may be seen to have numerous motivations for their actions (including financial gain, revenge, ideology or just plain mischief making). however, in many cases it can be argued that this is immaterial as, no matter what the reason, the end result is some form of adverse impact upon another party. issues of computer abuse became apparent in the sixties when the first computers were created and used. the levels of computer crime reported in the sixties and seventies, seem very small when compared to today as shown by table 1 (parker, 1976): 58 ajis vol. 8 no. 2 may 2001 year 1962 1963 1964 1965 1966 1967 1968 1969 1970 1971 1972 1973 1974 1975 number 2 1 11 8 2 4 12 15 33 54 67 69 59 43 tablet: number of security incidents in the usa bonn parker (parker, 1976) highlighted that the individuals involved in computer crime in the 1960's and 1970's were employed as key punch operators or clerks in edp organisation s and the crimes were crimes of opportunity. in the 1980's with the development of cheaper home micro and the introduction of modems a new generation of younger computer users were developed. one of the features of this younger group was a keen interest in the technologies that lead to the development of hackers. steven levy's book hackers: heroes of the computer revolution (levy, 1984) suggests that hackers operate by a code of ethics. this code defines main key areas: • hands on imperative: access to computers and hardware should be complete and total. it is asserted to be a categorical imperative to remove any barriers between people and the use and understanding of any technology, no matter how large, complex, dangerous, labyrinthine, proprietary, or powerful. • "information wants to be free" "information wants to be free" can be interpreted in a number of ways. free might mean without restrictions (freedom of movement = no censorship), without control (freedom of change/evolution = no ownership or authorship, no intellectual property), or without monetary value (no cost.) • mistrust authority. promote decentralisation. this element of the ethic shows its strong anarchistic, individualistic, and libertarian nature. hackers have shown distrust toward large institutions, including, but not limited to, the state, corporations, and computer administrative bureaucracies. • no bogus criteria: hackers should be judged by their hacking, not by 'bogus criteria' such as race, age, sex, or position. • "you can create truth and beauty on a computer." hacking is equated with artistry and creativity. furthermore, this element of the ethos raises it to the level of philosophy. • computers can change your life for the better. in some ways, this last statement really is simply a corollary of the previous one. since most of humanity desires things that are good, true, and/or beautiful. during the 80's and 90's this pure vision of what hackers are was changed by the development of new groups within various aims and values. mizrach (1997) states that the following individuals currently exist in cyber space: • hackers (crackers, system intruders) these are people who attempt to penetrate security systems on remote computers. this is the new sense of the term, whereas the old sense of the term simply referred to a person who was capable of creating hacks, or elegant, unusual, and unexpected uses of technology. • phreaks (phone phreakers, blue boxers) these are people who attempt to use technology to explore and/or control the telephone system. • virus writers (also, creators of trojans, worms, logic bombs) these are people who write code which attempts to a) reproduce itself on other systems without authorisation and b) often has a side effect, whether that be to display a message, play a prank, or destroy a hard drive. 59 ajis vol. 8 no. 2 may 2001 • pirates originally, this involved breaking copy protection on software. this activity was called 'cracking'. nowadays, few software vendors use copy protection, but there are still various minor measures used to prevent the unauthorised duplication of software. pirates devote themselves to thwarting these and sharing commercial software freely. • cypherpunks (cryptoanarchists) cypherpunks freely distribute the tools and methods for making use of strong encryption, which is basically unbreakable except by massive supercomputers. because american intelligence and law enforcement agencies, such as the nsa and fbi, cannot break strong encryption, programs that employ it are classified as munitions. thus, distribution of algorithms that make use of it is a felony • anarchists are committed to distributing illegal (or at least morally suspect) information, including, but not limited to, data on bomb making, lock picking, pornography, drug manufacturing, and radio, cable and satellite tv piracy. • cyberpunk usually some combination of the above, plus interest in technological self-modification, science fiction and interest in hardware hacking and 'street tech'. mizarch (1997) determined that new groupings with cyberspace had altered the initial code of ethics, and that the code of ethics in the 1990s was more concerned with: • "above au else, do no harm" do not damage computers or data if at all possible. • protect privacy people have a right to privacy, which means control over their own personal (or even familial) information. • "waste not, want not." computer resources should not lie idle and wasted. it's ethically wrong to keep people out of systems when they could be using them during idle time. • exceed limitations. hacking is about the continual transcendence of problem limitations. • the communication imperative. people have the right to communicate and associate with their peers freely. • leave no traces. do not leave a trail or trace of your presence; don't call attention to yourself or your exploits. • share! information increases in value by sharing it with the maximum number of people; don't hoard, don't hide. • self defence against a cyberpunk future. hacking and viruses are necessary to protect people from a possible orwellian '1984' future. • hacking helps security this could be called the 'tiger team ethic': it is useful and courteous to find security holes, and then tell people how to fix them. • trust, but test! you must constantly test the integrity of systems and find ways to improve them. this newer code of ethics is more based upon the view that hackers are helping in the development of the information society and adding to its distinct nature. the ethics of imposing these values on others who are unwilling 'victims' does not seem to be questioned. information warfare the advent of the contemporary concept of 'information warfare' (see schwartau, 2000, 1994; denning, 1999; waltz, 1998) has raised the tampering of computer systems to a new dimension. the individualistic, anarchistic, and rather naive actions of young hackers has been replaced by the determined, methodical, and organised workings of states, corporations, and criminal gangs. initially, the term 'information warfare' was concerned with damaging a country's national information infrastructure (nil) (schwartau, 1994). for the purposes of this paper, the nil is defined as the physical and virtual backbone of an information society and includes, at a minimum, all of the following (cobb, 1998): • government networks-executive and agencies; • banking and financial networks-stock exchanges, electronic money transfers; • public utility networks-telecommunication systems, energy and water supply (military and civil), hospitals, air traffic control and guidance systems, such as the global positioning satellite system and the instrument landing system both common to commercial aviation; • emergency services networks (including medical, police, fire, and rescue); • mass media dissemination systems-satellite, tv, radio, and internet; • private corporate and institutional networks, and 60 ajis vol. 8 no. 2 may 2001 • educational and research networks. information warfare is concerned with the full spectrum of offensive and defensive operations such as electronic warfare, cyber-terrorism, psychological operations and so on (main, 2000). mainly it has been associated with the so-called 'revolution in military affairs' (rma). in some respects iw is a sub-set of the rma, which is also concerned with the military application of new technologies to the 'battlespace' (cobb, 1998), such as stealth, precision guided munitions, and advanced surveillance capabilities. because of this military aspect of information warfare, its ethics have been associated with those of warfare. it is a development of the nature of warfare. the development of 'total war', arguably started in the actions of napoleon but certainly present in the second world war, has been extended by the advent of information warfare. the distinction between military and civilian targets has been blurred. the kosova conflict illustrated this. the bombing of a television station as a part of the serbian military machine displayed the importance of information in modern conflict (ignatieff, 2000). at the state level, there is a tendency to attempt to develop 'information superiority' over every competitor. international and national legal systems still have to catch up with this trend. there are ethical implications between developed and less developed countries. in terms of information warfare, each society has its advantages and vulnerability. for instance, the usa has an enormous advantage in digital facilities from fibre optic communications to satellites and sophisticated software production. however, this advantage also adds vulnerability, as digital systems are susceptible to attack. the infrastructure of a 'networked' society (eg power, water) is very exposed. developing countries at a lower level of development have an advantage of slower communication and processing systems, but this lack of sophistication lessens the vulnerability to an information attack. the advantages of developed societies in the information 'struggle' can be summarised as: • advanced infrastructure • have the intellectual property rights to most advanced developments • advanced technologies • have control of large corporations • advanced networked society possible, reliant on technology but infinitely flexible • capable of information dominance strategy • dominate perception management industries, eg media the advantages of developing nations can be summarised as: • lack of vulnerable electronic infrastructure • low entry costs to get into electronic systems development • web based systems know no geographic boundaries (in theory), hence neither does 'place' of company • networked society based on 'clans', difficult to penetrate • cheap labour, often with an educated elite, eg india the implications of interstate behaviours have yet to be fully worked out. of course, it is questionable whether 'ethics' per se are relevant in an environment where international law (which should reflect ethical issues) and national self interest dominate. information warfare has also spilled into the corporate arena. adams (1998) emphasises the movement of state aggression from military to economic. however, this conflict can be said to have also moved to corporate to corporate conflict, and even corporate to individual. the use of information warfare techniques can be seen as just another factor of business behaviour. grace and cohen (1998) illustrate the dilemma in business of behaving ethically. they argue that, all too often, arguments are polarised into two choices: behave unethically or fail. for many organisations, business is a form of competitive warfare. hence, techniques applicable to the military and intelligence services (information warfare) are viewed as feasible choices, although grace and cohen assert that it is wrong to assume that competitor's tactics are designed to destroy rivals. however, they do argue for an "international legal and normative infrastructure...the point is often lost in analogies with war, however, is that a great deal of this infrastructure already exists in private and public international law" (ibid, p.181). whilst this may be the case for general business practices, the advent of cyber-space has created problems of definition and jurisdiction. however, over the last few years, legislation in america, europe, and australia has allowed intelligence and law enforcement agencies to perform these operations. for instance in 1999, the australia security and intelligence organisation (asio) was allowed (with ministerial approval) to access data, and generally 'hack' into systems 61 ajis vol. 8 no. 2 may 2001 (lagan and power, 1999). therefore, legal hacking is the province of the authorities. hardly, an ethical, public stance to take. in fact, it just emphasises the efficacy of information warfare techniques to others. at the local australian and international level is the problem with the whole area of computer crime and the ambiguity of the law in this area. this is also confused by the potential international legal implications of foreign attacks. for instance, in the state of new south wales, the crimes act 1900 (nsw), section 310 states it is illegal to 'destroy, erase, insert, or alter data in a computer system.. .or interfere with, interrupt, or obstruct the lawful use of a computer' (internet law bulletin, 1998, p. 14). however, it is rare for these offences to be detected, prosecuted, or proven. morth (1998) further argues that although this form of information warfare may be illegal in international law, it is only states that are covered by this, not individuals or companies (for which there is no international law in this area). the perception of information warfare at the state or corporate level is to obtain as much benefit as possible and cause as much damage as possible to the 'enemy'. thus the intent is usually destructive; this is very different to the hacker code of ethics. the reason for this is the emergence of cyberspace attack as the critical ingredient in a new witch's brew of strategic conflict capability. information warfare tools and techniques allow the potential to destroy communications, information dissemination, and the functioning of critical equipment with no reference to geography or an ability to physically destroy anything in the conventional sense. such a capability poses a whole new kind of threat to international stability (molander and siang, 1998). this destabilising potential can equally have an organisation or individual as its targeted victim. conclusions information warfare has shifted the ethical issues from a naive group of technically, oriented, young individuals (hackers, etc.) to the organisational arena. the implications of this shift from the relatively benign impact of individual behaviour to that of organised groups has raised the magnitude of the ethical issues surrounding them. they need to be considered at both the legal and policy making levels of national, international, and corporate institutions before practices become entrenched or cause irreversible damage. but how feasible in reality would this approach be. the impact of hackers could be the destruction of a few number of information's system, the impact of information warfare could be the widespread destruction of a entire countries information systems network, the ethics behind these viewpoints and actions are completely different. references adams, j. (1998) the next world war, hutchinson, london caelli, w, longley, d., shain, m. (1989) information security for managers, stockton press, new york, usa. cobb.a. (1998) australia's vulnerability to information attack: towards a national information policy strategic and defence studies centre working paper #310, australia national university, australia. denning, d.e. (1999) information warfare and security. addison-wesley, new york. grace, d., cohen, s. (1998) business ethics second edition, oxford university press, melbourne. ignatieff, m. (2000) virtual war, chatto & windus, london. internet law bulletin. (1998). 'guilty plea on inserting data charge'. internet law bulletin, 1(1). lagan, b., power, b (1999) asio cleared to hack into computers, sydney morning herald, 26 march, 1999. levy, s (1994). hackers:heroes of the computer revolution. unknown, isbn 0385312105. main, b. (2000) information warfare: and its impact on the information technology industry in new zealand, proceedings of the naccq, wellington, new zealand. mizrach s (1997) is there a hacker ethic for 90s hackers? url: www.mfowar.com molander r, and siang s (1998). the legitimization of strategic information warfare:ethical consideration, professional ethics report, xi(4). morth, t.a. (1998) 'considering our position: viewing information warfare as a use of force prohibited by article 2(4) of the u.n.charter', case western reserve journal of international law, 30(2,3), pp. 567600. parker, d. (1976) crime by computer, charles scribner sons, usa. schwartau, w. (2000) cybershock. thunder's mouth press, new york, usa. schwartau, w. (1994). information warfare: chaos on the electronic superhighway. thunder's mouth press, new york, usa. waltz, e. (1998) information warfare principles and operations. artech house, norwood. warren, m., hutchinson, w. (2000). information warfare: fact or fiction. sec2000, ifip world congress, beijing, china. 62 microsoft word special final.doc ajis special issue december 2004 66 from documents to user interfaces universal design and the emergence of abstraction jason white abstract abstract representations of content which allow it to be automatically adapted to suit the delivery context, have emerged historically with the development of markup languages intended to facilitate the storage and processing of electronic documents. this technological tradition is reviewed in the first part of the paper, focusing predominantly on the nature and advantages of a ‘single authoring’ approach to the creation of content. some of the lessons to be derived from the evolution and deployment of markup systems are also discussed, then applied, in the second part of the paper, to the question of how such abstractions can be extended to the design of user interfaces. innovative work related to the generic specification of user interfaces is reviewed. it is argued that the advantages of an abstract approach depend for their realization on the development of more expressive style languages and more sophisticated adaptation mechanisms, as well as continued refinement of the semantics of markup languages themselves. introduction in a paper presented at the seventh international world wide web conference in 1997 [vanderheiden, 1997], gregg vanderheiden envisioned a technological future in which software applications and informational resources would be universally available, irrespective of the needs and circumstances of the user. whether behind a desk, in a vehicle, in a noisy environment or a remote location, to mention only some of the diverse possibilities, the same content and applications would be accessible, via whatever hardware and software the circumstances demanded. leaving aside the important but typically overlooked question of whether such ubiquitous network connectivity is socially desirable, this paper examines some of the basic technologies required to implement vanderheiden’s conception, reviewing the progress that has been made to date, and identifying problems that remain to be solved. not surprisingly, given the scope of this undertaking, the analysis can neither be detailed nor comprehensive. what is offered, rather, is a general overview of principles and technologies, suitably supported by examples drawn from languages and software systems with which the author is acquainted. the technical coverage offered is likewise diverse, touching on issues of general design as well as more practical questions of deployment and availability. markup languages and document processing the concept of markup has its origin in typesetting. by annotating or ‘marking up’ a manuscript, a designer would specify to a compositor precisely how the published document was to be formatted and presented. following the transition from manual to electronic printing, the typesetting activity itself, that is the construction of an image of the page which could be reproduced on paper, was now carried out not by human beings, but by a computer program. even so, the former division of labour between designer and compositor remained; the files supplied as input to the electronic typesetting system would include, in addition to the text of the document, control sequences to direct the typesetting algorithms. this electronic markup, clearly analogous to its manual predecessor, provides the historical and conceptual base from which modern markup languages have been formed. the subsequent development of markup systems has taken two distinct turns of central interest for the purposes of the present discussion: first, the move from concrete procedural semantics toward more abstract structural descriptions of the document, and secondly, the emergence of syntactically defined types of which document instances are specific manifestations. ajis special issue december 2004 67 by the first of these moves, instead of specifying the font changes, spacing and other characteristics of the manuscript throughout the file wherever they occurred, the structure of the document was represented at a higher level, leaving the details of the formatting to be supplied elsewhere, for example in a macro package. low-level typesetting operations were superseded, in the markup, by a more general description of the document, distinguishing headings, paragraphs, lists, footnotes, table structures and other components of the material. each instruction in the markup would be executed by a call to the corresponding macro containing the procedures to be performed in generating a properly typeset image of the page. of course, this description is something of an idealization; in practice, documents prepared in macro-based typesetting languages such as troff and tex [knuth, 1986] are usually an admixture of structural markup and lower level commands that directly control the presentation. nevertheless, having introduced a distinction between a declarative, structural description of the document, and the operations and parameters needed for its realization by a typesetting system, the conditions were laid for a more radical move, the conception of an electronic document as instantiating a formally defined grammar, a so-called ‘document type’, to use the terminology of the standard generalized markup language (sgml) [international organization for standardization, 1986], and its simplified but more popular successor, the extensible markup language (xml) [bray et al., 2004]. a document type definition (dtd), or more recently, an xml schema [fallside, 2001, thompson et al., 2001, biron and malhotra, 2001], specifies, most importantly, the names and possible orderings of the elements that can comprise documents of the defined type. textual content in the form of character strings, may also be permitted to occur in contexts prescribed by the document type definition; likewise, elements may carry attributes, the names and contents of which are again determined by the formal grammar. a principal function of an sgml or validating xml parser is to determine whether a supplied document instance is of the declared type, namely whether it fulfills the requirements set out in the dtd or schema. by providing a grammar with which to specify document types, sgml and xml facilitate the development of multiple, inter-operating application programs capable of processing document instances. no longer is the markup in the document tied directly to macro calls; instead it is read, parsed and processed by software, the details of which remain entirely unspecified by the markup language, and which may access the content for purposes other than typesetting. coordinate with these developments, standardized style languages have been created for associating elements of the document with rules, possibly in a quite contextually dependent fashion, that control the typesetting process [international organization for standardization , 1996, bos et al., 1998, adler et al., 2001]. these rules are generally stored in style sheets, files that are read by the formatting software along with the document instance and any other associated resources, such as vector or raster-based images, required to construct the final presentation. each style sheet defines a transformation which allows documents conforming to a particular dtd or schema to be presented as determined by the style sheet itself, as applied and executed by the typesetting application. any number of alternative style sheets, or indeed application programs, may be written that correspond to a particular document type, allowing document instances of that type to be processed in multifarious ways. it is this flexibility which enables a single document instance, in principle, to be formatted for a variety of visual, auditory and tactile displays, each with its own constraints and stylistic requirements, without ever modifying the markup of the document itself. in so far as document instances satisfy formally defined type definitions, the software or style sheet developer can proceed with the assurance that document instances will exhibit a tightly constrained and predictable structure, which, moreover, is specified in a formal language, thereby easing the construction not only of general tools for creating and reading marked up documents, but of highly customized applications and processes for treating documents of particular types. the advantages of the model that has thus been outlined, in which documents are written in formally defined markup languages, have already been indicated. nevertheless, in order that certain points can be elaborated further, there is value to be gained by identifying them more precisely. in contrast to ajis special issue december 2004 68 procedurally oriented markup systems, a greater separation is maintained between the electronic representation of the document, and the types of processing and transformation to which it may be subjected. this is achieved by abstraction: the markup typically identifies aspects of the document which the designer of the document type, having more or less regard to the various purposes for which it may be used, considers structurally and semantically significant. with application software of sufficient flexibility and sophistication, the document can be rendered to suit whatever output device, presentational needs and preferences the user may possess. the conversion of an abstractly marked up document instance into a concrete presentation can be carried out entirely by software operating on behalf of the author or the user; or the rendering procedure may alternatively be separated into distinct transformational stages, each carried out in succession at different points along the delivery path from the author’s input to the perceived presentation provided by the output device, for example a printer, a video monitor, a tactile display or a speech synthesizer. the characteristics of the hardware and software through which the user interacts with the document, along with those of the network by which it is delivered and the user’s needs and preferences, are collectively known as the ‘delivery context’ [gimson, 2003]. with this terminology, it can be said that an abstractly marked up document can automatically be adapted to, and presented in, a wide range of delivery contexts, subject to the availability of appropriate software and assuming that the markup specifies explicitly the necessary semantic distinctions to enable the required processing to be carried out. as a corollary, the presentation of the document can be systematically changed merely by altering the rules governing its presentation, for example a style sheet, without changing the document instance. this advantage, in its own right, has often been cited as a sufficient ground for deploying structured markup systems, even in contexts in which the adaptation of documents to a multiplicity of delivery contexts is not considered important. moreover, the declarative semantics of the markup enable document corpora to be searched more intelligently, with proximity queries limited to specified structural components of the documents, and techniques for ranking search results that rely on this inherent structure to gauge their relevance to the user’s interests. along with these considerable benefits come, not surprisingly, issues and problems that remain to be addressed. thus, style languages have not, in general, acquired the degree of expressive power to be found in macro-oriented typesetting languages such as tex and its associated tools. in practice on the world wide web, where markup languages have been most broadly and spectacularly deployed, the emergence of style languages has been gradual, and sophisticated style languages such as xslt [clark, 1999], capable of achieving relatively complex hierarchical transformations, remain a rarity, except to some extent in server applications. likewise, markup languages for representing fonts and vector graphics are far from common on the web, whereas languages such as postscript [adobe systems, 1999] lie at the core of modern versions of tex and other typesetting systems. similarly, to mention a further example, low-level markup languages designed to capture the prosodic parameters of a speech synthesizer by which document can be formatted and rendered in audio, have only been standardized quite recently [burnett et al., 2004], and their degree of practical implementation has so far been minimal. compounding these difficulties has been a tendency, which is again most prominent on the web, to implement document markup languages such as html [raggett et al., 1999] directly in the compiled code of the user agent software, rather than in a style language. consequently, support for a new markup language, or even, in many circumstances, an extension of or update to an already implemented markup language, can often only be achieved by providing compiled software that must be downloaded and installed by the user to be executed by the operating system. this compares unfavourably with a design in which core presentational functions are implemented in the user agent, with support for semantically rich markup languages being provided by interpreted or compiled code written in a high-level programming language, which can be downloaded dynamically, cached and executed in a secure environment established by the user agent. ajis special issue december 2004 69 the relative permissiveness of html user agents with regard to document instances that fail to conform to a document type definition, combined with the predominance of the graphical desktop workstation as a delivery context, has contributed to the common practice of abusing features of the markup language to achieve a desired layout or other presentational effect, instead of adhering to the semantics defined in the specification. the practical difficulty mentioned earlier of implementing and deploying an extension to a markup language widely supported in user agents, has only exacerbated these problems, leading ultimately to a vast legacy of syntactically invalid and presentationally-oriented content. both of these aspects are problematic: in so far as the documents are syntactically invalid and unpredictable, the difficulty of writing software that can successfully parse and process them is amplified; in as much as the documents fail to conform to the semantics prescribed for elements and attributes of the markup language, they are less amenable to automatic adaptation for presentation in diverse delivery contexts. two lines of advance toward solving these problems are already implicit in the preceding analysis. first, there is a need to develop more sophisticated document presentation systems, encompassing scalable rendering functionality suitable to a variety of devices and output modalities, and high-level style languages for generating optimal presentations of content in a range of delivery contexts. secondly, and of particular advantage to legacy user agents, there is the possibility of shifting more of the transformation and adaptation process from client to server, whether the latter be controlled by the original author of the document or a third party. in recent years, the latter approach has received considerable impetus from the emergence of portable computing devices and the concomitant need for web documents to be adapted to the comparatively small displays and limited functionality offered by this hardware. application of markup languages to user interfaces in the preceding section, the focus of the analysis was deliberately confined to the adaptation and presentation of electronic documents, setting aside altogether problems related to interactivity. thus the question arises whether the approach championed earlier in the paper, whereby content is written in abstractly specified and syntactically well defined markup languages, then transformed according to separately provided rules to suit the requirements of the delivery context, can be applied to the construction of user interfaces in general. promising work has recently been carried out in extending the basic approach to interfaces which, unlike static documents, accept input from the user and undergo modification, whether in response to input or other changes in the state of a program, as application software is executed. two related specifications encapsulate the results of this effort: the xml forms language (xforms) [dubinko et al., 2003], and the draft protocols for a universal remote console currently under development by the incits v2 working group [international committee for information technology standards , 2004a,b,c,d,e]. the overview which shall be offered here is necessarily partial; its purpose is to expound the common principles and abstractive techniques underlying the design of both specifications, while largely disregarding important differences between them. to that extent, vital technical details are omitted altogether, or mentioned only incidentally. for the sake of clarity, the terminology of the draft universal remote console specifications, which is more widely applicable to a variety of user interfaces, shall here be used, except where the xforms language is specifically under discussion; but this should not be taken to imply any claim that either design is technically superior to, or more general than, the other. with these reservations firmly established, the basic components of an abstractly represented user interface can now be identified. the purpose of a user interface is to enable the user of an application to enter input, call software functions which process it, and render the output. accordingly, the connection between the user interface and the data processing components of the application consists, first, of the constants and variables available to be presented to, and in the latter case set by, the user; and secondly, the application functions that the user can directly invoke. in the universal ajis special issue december 2004 70 remote console specifications, these components are collected in what is known as a user interface socket—an xml document in which variables, constants and application commands are declared. in xforms, the same purpose is served by the xforms model, combined with xml events [mccarron et al., 2003] to enable application functions to be activated. in both cases, the data type of each variable can be constrained by xml schema declarations [biron and malhotra, 2001], and xpath expressions [clark and derose, 1999] can be supplied to ensure that validity requirements encompassing one or more variables are met prior to the processing of the data. having thus described the data and functionality accessible via the user interface, it is necessary to characterize the actions that the user can perform to set or update the values of variables and to call application functions, together with the outputs that may be presented in response. this is achieved by defining elements referred to in the universal remote console specifications as abstract interactors, which in turn are based on the corresponding xforms form controls. as the name implies, the abstract interactors do not presuppose any particular style of presentation or input mechanism. they are equally suited, for example, to the construction of a speech-based interface oriented toward the processing of natural language, as to the creation of a graphical user interface. each abstract interactor is explicitly bound to an element of the user interface socket, subject to limitations imposed by its type. each type of interactor, represented as an xml element, supports a specific operation within the user interface: selection of a single item from a set, selection of multiple items from a set, selection of a value from a totally ordered set, text input, the presentation of output, and execution of application functions, among others [for an early development of these user interface abstractions see raman, 1997]. the data entered must of course satisfy the requisite type and validity constraints defined in the user interface socket. abstract interactors may be arranged in logical hierarchies reflecting the structure of the user interface, with related options and actions being grouped together. for example, in the user interface of a bibliographic database, the input fields into which the title, author and other details of the search query are entered, form a logically distinct group, separate from the options that place limits on the number of records to be returned or control the order in which matching records are to be sorted. even an abstractly defined user interface is not complete without labels and explanatory text, written in one or more natural languages, that identify and help the user to understand the options provided. in xforms, this essential information is included as part of each form control. in the universal remote console specifications as presently drafted, however, it is stored separately in resource sheets that may be retrieved and interpreted as part of the process of deriving a concrete user interface from the abstract description. this allows for a multiplicity of resource sheets corresponding to any given application, each customized to suit the natural language, culture and other characteristics of a given class of users. in addition to associating a label and help text with an abstract interactor, a resource sheet can specify key words that identify the interactor; these may be used, for instance, by speech recognition systems. correspondingly, a single character can be bound to the interactor to enhance operation via a keyboard or similar input device. reference to modalityspecific resources such as visual and auditory icons, or formal grammars for speech recognition systems, is also supported. it is the task of the software implementing the universal remote console to combine the abstract definition of the user interface with the data from an applicable resource sheet to create an interface suited to the requirements of the delivery context. just as generic markup has advantages over procedural markup in that it separates the details of layout and formatting from the structural description of a document, thus permitting a document instance to be processed in ways that are limited only by the ingenuity of the application or style sheet developer, it has the potential to yield corresponding benefits as applied to the abstract characterization of user interfaces. most user interfaces designed today are written in a conventional programming language, with a quite specific delivery context in mind, for example a graphical workstation, and are carefully designed for the intended environment. other delivery options are, at best, treated as secondary concerns: a user interface library may, for instance, support accessibility ajis special issue december 2004 71 via assistive technologies that provide alternative input and output mechanisms for people with disabilities, but the graphical interface remains the primary focus of the design. what distinguishes xforms and the universal remote console specifications is the achievement of a more adequate demarcation of the underlying structure from those aspects of the user interface which vary with the delivery context, while allowing additional resources to be specified that enable the interface to be automatically enhanced or customized for specific environments. furthermore, this generic approach challenges the traditional distinction between user interfaces as such, intended to be operated directly by human beings, and application interfaces exposed to external software. thus one can envisage complex and largely automated tools that locate and interact with devices and applications supporting such interfaces, with little, if any, direct human intervention. it is these possibilities that open the way to the prospect of sophisticated systems composed of collaborating, networked devices serving our needs through interactive and evolving interfaces that take advantage of whatever input or output mechanism the occasion demands [see the discussion in vanderheiden and zimmermann, 2004]. in short, it is partly through making the user interface amenable to automated as well as human interaction that the technological future which vanderheiden imagines can become actualized. although, as xforms and the universal remote console specifications illustrate, significant progress has been made in recent years toward the development of suitable abstractions with which to describe user interfaces generically, the problem of transforming these into concrete interfaces optimized for the delivery context, remains still to be solved. as has already been pointed out, in the application of generalized markup to electronic typesetting, the rules relating the structure of a document to its presentation in the final output, have come to be codified in style languages that control the rendering process. given a document instance conforming to a specified schema and an appropriate style sheet, a typesetting system can generate a quality presentation tailored to the delivery context for which the style rules are designed. the convenience of this solution lies as much in the availability of flexible style languages with which to specify presentation, as in the abstractions underlying the markup with which the structure of the document is captured. to gain the benefits of extending this basic model to the design of user interfaces, it is desirable to develop versatile and sophisticated style languages with which to express general rules whereby abstract descriptions can be converted into highly usable, effective and consistent interfaces, with each adapted to a particular delivery context. by thus partially automating the derivation of concrete interfaces from abstract specifications, it would become possible, as in the field of document processing, to apply a common set of style rules to a multiplicity of interfaces, making only such modifications as are needed to accommodate the unusual or exceptional features of a specific software application. the advantages noted earlier in connection with document processing, including consistency of presentation and the opportunity to make uniform changes simply by modifying applicable style rules, would carry over, mutatis mutandis, to the construction of user interfaces. thus, the predictability of interactional conventions within and among applications would be enhanced, without limiting the flexibility of the software developer, in writing specially crafted style rules, to customize the interface for chosen delivery contexts. adaptation of an existing abstract interface to a new delivery context could be accomplished by defining a new set of style rules, or, in more complex situations, implementing specialized software for interpreting the markup in which the interface is described. as is true of markup systems generally, languages for characterizing abstract user interfaces can be interpreted in a server or client environment according to the requirements of the implementation, including the extent to which compatibility with legacy software and standards is necessary[this point is further elaborated in maes and raman, 2001]. ajis special issue december 2004 72 conclusions: the challenge for user interface design structured markup languages have here been considered in connection with two domains of application: the representation of electronic documents, and of interactive user interfaces, respectively. these should not indeed be regarded as mutually exclusive categories; rather, the latter introduces a dynamic dimension that is absent from the former. a conventional document can thus be conceived of as a purely presentational, entirely non-interactive, user interface. it has been argued in this paper that the use of generic, syntactically defined, markup languages for the encoding of documents, is a technically advantageous practice in that it allows the abstract representation of the electronic content to be kept distinct from the various means by which it may be processed and displayed to a user. an important benefit of this design lies in the flexibility with which structurally marked up documents can be adapted, typically by applying style rules, to a wide range of delivery contexts. significant shortcomings in contemporary implementations of the preferred approach have also been noted, including a failure to build consistently upon well designed foundations provided for example by vector graphics and audio formatting languages [the concept of an audio formatting language is developed in raman, 1998], and the inadequacy of currently deployed style languages by contrast with typesetting systems such as tex. these limitations have also given rise to difficulties in deploying extensions to popular markup languages, in particular html. the demands placed on the design of style languages and other mechanisms that process, transform and render marked up content, become all the more complex if what is abstractly represented is not merely the structure of a static document, but instead that of a dynamic user interface. as recent work in the development of markup languages for describing user interfaces has shown, appropriate abstractions enable the essential aspects of an interface to be captured quite independently of any assumptions regarding the delivery contexts in which it may be presented. the technologies needed to adapt such abstract interfaces to various delivery contexts, and, where possible, to do so by applying rules that can be conveniently specified by the application designer or by a third party, are yet to be developed. to what extent the craft of user interface design, like that of typography, can be carried out by specifying rules for converting abstract structural descriptions into concrete presentations and interactions optimized for the delivery context, continues to be largely an unanswered question. in typesetting and document processing, there is already a strong argument for the use of generic markup systems[see also taylor, 1996]. many of the advantages of this technology which have been identified with respect to electronic documents, have counterparts, so it has been argued, in its application to user interfaces, assuming of course that the required processing and adaptation mechanisms, as already mentioned, can in fact be developed. in so far as these benefits are realized, the abstract solution may largely supersede the current practice of individually coding a user interface specifically for its intended delivery context. the technological possibilities which this would open up, and the benefits that would thereby accrue to users, have barely been touched on in this paper. however, the broad advantages have been clearly identified, leaving the detailed possibilities to be further explored as the technologies themselves are developed. references sharon adler, anders berglund, jeff caruso, stephen deach, tony graham, paul grosso, eduardo gutentag, alex milowski, scott parnell, jeremy richman, and steve zilles. extensible stylesheet language (xsl) version 1.0. recommendation, w3c, october 2001. url http://www.w3.org/tr/2001/rec-xsl-20011015/. adobe systems. postscript language reference. addison-wesley pub. co., third edition, 1999. paul v. biron and ashok malhotra. xml schema part 2: datatypes. recommendation, w3c, may 2001. url http://www.w3.org/tr/2001/rec-xmlschema-2-20010502/. ajis special issue december 2004 73 bert bos, håkon wium lie, chris lilley, and ian jacobs. cascading style sheets, level 2. recommendation, w3c, may 1998. url http://www.w3.org/tr/1998/rec-css2-19980512. tim bray, jean paoli, c. m. sperberg-mcqueen, eve maler, françois yergeau, and john cowan. extensible markup language (xml) 1.1. recommendation, w3c, february 2004. url http://www.w3.org/tr/xml11. daniel c. burnett, mark r. walker, and andrew hunt. speech synthesis markup language (ssml) version 1.0. recommendation, w3c, september 2004. url http://www.w3.org/tr/2004/recspeech-synthesis-20040907/. james clark. xsl transformations (xslt) version 1.0. recommendation, w3c, november 1999. url http://www.w3.org/tr/1999/rec-xslt-19991116. james clark and steve derose. xml path language (xpath) version 1.0. recommendation, w3c, november 1999. url http://www.w3.org/tr/1999/rec-xpath-19991116. micah dubinko, leigh l. klotz, jr., roland merrick, and t. v. raman. xforms 1.0. recommendation, w3c, october 2003. url http://www.w3.org/tr/2003/rec-xforms20031014/. david c. fallside. xml schema part 0: primer. recommendation, w3c, may 2001. url http://www.w3.org/tr/2001/rec-xmlschema-0-20010502/. roger gimson. device independence principles. note, w3c, september 2003. url http://www.w3.org/tr/di-princ/. international committee for information technology standards . ansi/incits 389, information technology—protocol to facilitate operation of information and electronic products through remote and alternative interfaces and intelligent agents: universal remote console, 2004a. international committee for information technology standards . ansi/incits 390, information technology—protocol to facilitate operation of information and electronic products through remote and alternative interfaces and intelligent agents: user interface socket description, 2004b. international committee for information technology standards . ansi/incits 391, information technology—protocol to facilitate operation of information and electronic products through remote and alternative interfaces and intelligent agents: presentation templates, 2004c. international committee for information technology standards . ansi/incits 392, information technology—protocol to facilitate operation of information and electronic products through remote and alternative interfaces and intelligent agents: target properties sheet, 2004d. international committee for information technology standards . ansi/incits 393, information technology—protocol to facilitate operation of information and electronic products through remote and alternative interfaces and intelligent agents: resource description, 2004e. international organization for standardization . iso 10179: document style semantics and specification language, 1996. international organization for standardization. iso 8879: standard generalized markup language, 1986. donald e. knuth. the texbook. addison-wesley pub. co., 1986. stéphane h. maes and t. v. raman. a “single authoring” programming model for the next web. in scientific emphasis proceedings, 2001. url http://www.stephanemaes.com/essem/scientificemphasis/proceedings/2001/06/singleauthoring .htm. shane mccarron, steven pemberton, and t. v. raman. xml events: an events syntax for xml. recommendation, w3c, october 2003. url http://www.w3.org/tr/2003/rec-xml-events20031014. dave raggett, arnaud le hors, and ian jacobs. html 4.01 specification. recommendation, w3c, december 1999. url http://www.w3.org/tr/html401. t. v. raman. auditory user interfaces: toward the speaking computer. kluwer academic publishers, 1997. ajis special issue december 2004 74 t. v. raman. audio system for technical readings, volume 0302-9743; 1410 of lecture notes in computer science. springer-verlag, 1998. conrad taylor. what has wysiwyg done to us? the seybold report on publishing systems, 26(2), september 1996. henry s. thompson, david beech, murray maloney, and noah mendelsohn. xml schema part 1: structures. recommendation, w3c, may 2001. url http://www.w3.org/tr/2001/recxmlschema-1-20010502/. gregg c. vanderheiden. anywhere, anytime (+anyone) access to the next-generation www. in proceedings of the sixth international world wide web conference, 1997. url http://decweb.ethz.ch/www6/technical/paper253/paper253.html. gregg c. vanderheiden and gottfried zimmermann. interface sockets, remote consoles, and natural language agents: a v2 urc standards whitepaper, august 2004. url http://www.incits.org/tc_home/v2htm/docs/v2/04-0065/v2whitepaper.htm. microsoft word ajis10-2finala.doc ajis vol. 10 no. 2 may 2003 127 doing ethics: a universal technique in an accessibility context christopher r. simpson1, liddy nevile2, oliver k. burmeister3 1 adjunct teaching fellow, school of information technology, swinburne university of technology, australia, e-mail: csimpson@swin.edu.au; 2 e-mail: liddy.nevile@motile.net; 3 swinburne computer-human interaction laboratory, school of information technology, swinburne university of technology, australia, e-mail: oburmeister@it.swin.edu.au; abstract thinking ethically is difficult without a background in moral philosophy. asking people to embrace ethics without offering practical explanation of the efficacy of so doing, is a thankless task. a technique that people can follow easily, that helps them value ethical behaviour is needed. illustrations of such a procedure show how professional scenarios are used in the domain of web accessibility. scenario design and extensions to the technique are considered. keywords ethics; accessibility; case-studies; dilemma resolution, it. introduction whether a student, a retiree or in professional practice, today one confronts many situations where it would be helpful to have a particular way of sifting through issues to determine appropriate courses of action. gordon preece (2002) writing on a recent topical issue put it this way: 'the womb is like an ethical war-zone. embryonic stem-cell research, deaf lesbians choosing deaf-babies, ivf embryos chosen and conceived to save existing children, single and lesbian women accessing ivf. hardly a day goes by without a new ethical dilemma. the pace of technological change and precedent makes it almost impossible to keep up. ' this paper gives the background that gave rise to this technique, the process itself, and then makes use of exemplary scenarios to illustrate how the procedure can be useful to practitioners. one scenario illustrates the approach, while a second illustrates the value in designing appropriate scenarios. the paper ends with suggestions of how the technique, first used in a student context, can be extended to suit professionals from different disciplines as multi-disciplinary teams engage in activities such as building websites. background at swinburne university of technology, 'professional issues in information technology’ (piit) is taught to all final year it students. it can be taken by students from other faculties as an elective, although students without industrial experience are strongly discouraged from taking the subject. over the past 12 years, as the subject has evolved, the focus has been to explore real cases that will confront young (and not so young) professionals in the workplace. this is achieved by having about half the lectures led by industry speakers, as well as through tutorial work, debate and discussion that explore cases collected over the years, and draw heavily on the industry based learning (ibl) experiences of the students. students have often expressed difficulty in thinking through ethical situations. piit is usually the only subject requiring this in their computing courses. despite their lack of familiarity with this type of work, almost all students, after finishing the subject, rate it as one of the most valuable to their professional development and one of the most enjoyable in their course. to overcome students' initial difficulties, a simple approach to scenario analysis has been developed. formative feedback during the semester from tutors has often involved answering student queries in terms of this process, getting them to think through pertinent issues. similarly, summative assessment has shown that critical thinking and meaningful consideration of multiple alternate solutions to derive a best fit, most frequently occur when this approach is adopted by students. although the technique was developed for student use, it is not limited to this context. in the view of the authors, it is widely applicable. to illustrate this point, the scenarios below offer a professional internet developer's viewpoint, with the technique applied as though carried out by professional peers. ajis vol. 10 no. 2 may 2003 128 the doing ethics technique the technique of analysis depends upon asking questions. it has been found that the order in which the questions are asked is also important. the following questions, in this order, have been found to work best (through trial and error over many semesters). q1 what is going on? – what are the facts? q2 what are the issues? q3 who is affected? q4 hence, what are the ethical issues and implications? q5 what can be done about it? what options are there? and q6 which option is best? – and why? benefits an injunction to 'think ethically' about a situation is not helpful. perhaps if one has a background in moral philosophy this would work, but usually both students and it professionals require some form of guidance as to how to achieve an appropriate outcome. the technique has proven itself as a means to achieving this, at least for piit students. this approach is not dependent on a particular standard, such as the code of ethics of a particular professional society. it is a technique that can be applied in a variety of circumstances, not limited by technological, cultural or religious background. the approach is not limited by one's moral philosophy. one can use this technique effectively and be an objectivist, holding that certain moral truths remain good independently of personal likes and dislikes, or a relativist, holding that truths are relative to the individual or one's culture. similarly, this approach can be used by consequentialists, holding that consequences determine if something is ethical, and by deontologists, holding that some things, regardless of the consequences, are right or wrong in themselves. the technique is a means of arriving at an ethical outcome appropriate within (or at least independent of) personal belief systems. limitations the best results obtained in the piit classes have been when the technique has been used in tutorial groups, rather than have students submit individual reflections on particular cases. individual reflection yields significant results and students have voiced their delight at being able to work through difficult situations using this technique, but even so, further insights and better solutions are derived when it is used in a group situation. solving scenarios collaboratively also adds credibility to the process, in that much of the professional development takes place in multidisciplinary teams. and that creates new a problem – which ethical standard should one adopt? in multidisciplinary teams, professionals often belong to different professional societies, each with similar, but not identical, codes of ethics. in a student context this leads to interesting discussions, in a professional setting it may lead to conflicting standards that are difficult to resolve. another limitation is objectivity. one advantage of thinking through cases, such as those below, before being confronted with real situations, is that it becomes easier to be objective. when confronted with an ethical dilemma in real life, subjectivity can prevent one from being able to make the most appropriate judgments. this is where reference to one's peers, co-workers or members of one's professional society can be helpful. such people can use this technique to work objectively through the issues and advise the person in the situation of the most appropriate course of action. illustrating use of the technique educational suppliers, world-wide, are depending upon their belief in the commodification and commercialisation of education to offset the increasing cost of education. typically, they look for or develop learning object repositories and skeletal structures or 'skins' within which to serve up compilations of bitesized pieces, custom-built for individual needs. this form of education could benefit those with disabilities at least as much, if not more, than others. such disabilities can take the form of physical or mental human disabilities, or resource or location based incapacities. this is the context of the first case. ajis vol. 10 no. 2 may 2003 129 case 1: universal access government funding has been given to a university consortium establishing a repository of resources for school teachers. they have engaged you to develop a search facility for teachers who will use it to discover small chunks of useful material, located anywhere. the chunks will be strung together to make customised lessons for students who cannot attend school, for one reason or another, or whose teachers do not include specialists in subjects the students want to study. the client group knows that it is more 'attractive' to their funding agency if there is a lot of multimedia product in the selection. you know that it is hard to produce accessible multimedia and there is hardly any 'out there'. you are asked to advise your client about accessibility. q1 what is going on? – what are the facts? i. i report to the university consortium. ii. i am to develop a repository search facility for school teachers to extract useful teaching materials to develop suitable lessons. iii. target students are supervised by non-specialists (possibly remotely, at home, or in care). iv. the government funding agency likes multimedia. v. accessible multimedia content is scarce and 'hard' to develop. vi. the consortium requires ‘accessibility’ advice. q2 what are the issues? vii. what are the contractual details/arrangements? viii. accessible multimedia may be costly to develop. there appears to be a funding bias towards multimedia content. how much multimedia is envisaged? would a simple multimedia put funding at risk? advice to the consortium might be rejected (along with this job) by recommending a simple, 'non-glossy' approach. ix. what are the exact requirements for the multimedia? what standards are to be applied? what is the budget for it and how much of that is for the extra accessibility work? x. sophisticated multimedia is unlikely to be easily accessed/afforded by all supervisors or students due to special needs and telecommunications inequities. xi. what range of resources do supervisors/teachers and students have ready access to? xii. what range of disabilities do the target students suffer, if any? how will these be identified and accounted for? xiii. what level of training and support is envisaged for teachers, non-specialist supervisors and students? xiv. it is not clear if 'normal' school teachers are to benefit from this system. q3 who is affected? adversely… xv. my team, if our carefully considered recommendations are 'too pedestrian' and so rejected. xvi. students with disabilities, if systems are badly designed or too costly. xvii. all students, if the system is too cumbersome (costly, slow or otherwise hard to use). xviii. all students, if the resulting lessons are inadequate, trivialised or unsupported. xix. teachers (particularly non-specialists), without sufficient opportunities (time, funds, help or equipment) to learn about and receive prompt, ongoing support with the system. xx. all those unjustly excluded from the system, through negligence or economics or whatever. xxi. the consortium (and every party concerned), if the system fails, for any reason. positively… xxii. everyone (including the community), if the job is done with transparency and integrity. q4 hence, what are the ethical issues and implications? xxiii. commitment, integrity and mutual trust between all stakeholders. xxiv. clarification of project intentions and removal of political bias by the funding agency and consortium, and transparency in discussions regarding advice supplied to them. ajis vol. 10 no. 2 may 2003 130 xxv. realistic definition of project scope, based upon: professional design, ethical prioritising of need, provision of sufficient ongoing support. xxvi. discovery of the real needs of targeted teachers and students. xxvii. strenuous avoidance raising false expectations. xxviii. maintenance of sufficient quality of teaching material. xxix. particular attention to support of non-specialist teachers. xxx. particular attention to support of students with disabilities. xxxi. particular attention to matching technical sophistication with affordability and accessibility. …without attention to the above points, the project cannot proceed successfully, many people and reputations will be hurt, and precious funds will be wasted. q5 what can be done about it? what options are there? xxxii. advise the client that you need to know more but suspect that this contract will need to specify high standards of accessibility 'in case' there are students or teachers with disabilities as it is almost impossible to make inaccessible content accessible retrospectively. xxxiii. advise them to: a. undertake extensive feasibility studies and exhaustive prioritisation before development. b. identify the primary beneficiaries of the system. c. build in mutual safeguards to help alleviate unforseen problems. d. ensure the ongoing viability of the resulting resource by having sufficient support services. e. avoid over-sophisticated technology unless it is clearly affordable by targeted users and disadvantages nobody. q6 which option is best? – and why? xxxiv. in this case, there are still too many unknowns for a decision to be made so the best advice will have to be about what needs to be discovered and how, so that such a decision can be based on informed criteria. scenario design case studies have frequently been employed to illustrate ethical principles and particularly conflicts and prioritisations amongst those principles. for instance, anderson et al. (1993) employ scenarios to explore the application of a new code of ethics in different professional settings. clement (1993), in writing up experiences in a computer-supported cooperative work workshop, found that discussing scenarios was a productive means to exploring privacy issues. similarly, burmeister (2000) used case studies to illustrate the application of the australian computer society (acs) code of ethics to professional practice. why design and explore scenarios? why not just wait until one is confronted with an ethical dilemma in the workplace and then apply the above technique? there are several reasons. in general terms there is the advantage of using scenarios to explore situations ahead of time, while one can engage objectively in the discussion, without a subjective involvement in a particular situation. more specifically, in the accessibility context, scenarios help to identify one's knowledge. considering accessibility, there are many factors that should be taken into account, as is seen in the discussion above regarding case 1 and in the discussion about possible extensions to the technique of ethical analysis that follows. similarly, in other contexts, scenarios can be employed to explore one's knowledge of a particular domain. by asking people what they would do in a particular situation, one can readily identify the limits of that person's knowledge. for the purposes of ethical scenario analysis, scenarios should be neither self-evident nor simple. instead, there should be ambiguities that require moral reflection, leading not always to a 'right' solution, but rather towards a solution that is satisfactory. as stated in the limitations above, further reflection by others should yield alternate and perhaps even better solutions than those suggested by the original participants in the discussion. similarly for case 1 above, this process gives rise to possible solutions, it does not necessarily lead to the best or even to a good solution. but it does guide people to consider broad, complex issues in a systematic fashion. it is a useful instrument or framework for ethical decision-making. another use of scenarios can be to raise awareness of issues. for instance, with little re-working, the two cases in this paper could be phrased so accessibility issues are inconspicuous. then, in an audience of web designers, the discussion can be used to raise awareness of accessibility considerations, through simple 'what if' additions … what if a user was visually impaired? what if a deaf person tried to access the video clip? ajis vol. 10 no. 2 may 2003 131 case 2: conflict of interest australian government sites were mandated to conform to at least single 'a' level of the world wide web consortium (w3c) web accessibility initiative (wai) standards, by the end of 2001, seen as best practice in australia (sampson-wild and burmeister, 2001). yet the recent call for tenders to audit compliance of government sites, which closed 20 th june this year (innes, 2002), is likely to find very few government sites complying with these standards. the following case assumes your organisation was the successful tenderer … your client is a local government agency. you have been asked to develop an accessibility assurance strategy as part of a preliminary contract. you are keen to pick up the contract to build whatever you convince the government to do about accessibility testing, so you are wondering what to recommend. you realise that the agency wants to be told that its materials are accessible, rather than the converse, so you are wondering how high to set the bar for accessibility. you know there are automatic checks available for all the 508 requirements. you could not afford to offer to do all the manual checks required by the wcag and anyway, suspect the agency's materials would not look good if tested by wcag. like the earlier case, this one has multiple facets with no easy resolution. recent scandals in the us have involved large accounting firm practices where auditors also have consulting roles, creating a conflict of interest that has caused various now well-publicised problems. such conflicts are not peculiar to accounting, as is seen in this case. as mentioned above, one use of scenario design is to explore the domain knowledge of participants. both cases above involve the government. this was a deliberate, because it raises questions about the applicability of legislation concerning accessibility. also, in the 2 nd case, are legal implications which can either be teased out by a facilitator or left to arise naturally in discussion by participants querying 'wcag' or '508'. thus, there is the possibility to discuss the boundaries between legal and ethical considerations. more is made of this point below. extending the technique: from student to practitioner the technique began in a student context, but to be more suitable for professionals, it should be extended. it work today brings together people from different disciplines. this is especially common in complex systems design, typical when building web sites. the multi-disciplinary requirements of professionals might better be met with some additional modifications. the extensions begin to apply at question 3… q3 who is affected? there are some situations in which, in addition to asking 'who' is affected, it may also be appropriate to ask how the 'environment' is affected. further, this need not be limited to the 'environment' outdoors, but could also refer to workplace culture, the connectedness of a community, or other situational circumstances. as stated above, it is significant the government is mentioned as the use of government funding or publication by the government may change the potential liability of the developers and publishers. another environmental consideration in case 1 involves small rural schools that cannot afford specialist teachers, given their small class sizes. in that case, other kinds of accessibility considerations come to the fore, especially in terms of telecommunications access. low bandwidth might have implications in case 1 on the type of multimedia design. for instance, high-quality images, ideal for high-speed transmission, might not be appropriate. if a rural student is also disabled, then that further complicates the issues. one frequently employed multimedia technique involves the use of 'chat' environments, yet rural students with disabilities would possibly be totally excluded from participation in such class activities. q5 what can be done about it? what options are there? this question could be extended to include the following three sub-questions: i. what would be 'the right thing' to do? ii. who/what suffers if you do not do the right thing? iii. what additional costs will be incurred if you do the right thing? doing 'the right thing' is a principle that is often cited in the us and is akin to 'setting a good example', or observing 'best practice'. 'the right thing' is not always the most economical, the fastest or the easiest thing to do, but it is valued for its ethical advantage. given the wide acceptance of the right thing in some ajis vol. 10 no. 2 may 2003 132 circumstances, it has also come to mean 'that which will have the most beneficial long-term as opposed to immediate effects'. q6 which option is best? – and why? this question could be extended to include the following sub-question: iv. who gains if you do the right thing? political advantage comes in many forms. it does not always coincide with economic or social advantage. understanding the potential for bias among the stakeholders can help evaluate potential decisions. v. and why? this inevitably casts one's view into the future, where the consequences of our decision will be thrust upon our successors and children, and their environment. will our decision be seen as wise and far-sighted? multi-disciplinary extensions the following two additional questions are inappropriate in the student context, because rarely do it students have sufficient experience during or prior to their courses to deal adequately with these issues. typically, the acs's code of ethics requires professionals to consider the legal ramifications of their work on their employer, their clients and the eventual users of the products. but when acs-accredited professionals work with professionals from other disciplines, they often find that ethical codes of behaviour differ in trivial, or significant, ways and have to make decisions about which 'ethics' to follow. in other contexts, there may be practical considerations to deal with. the technique could be further extended to include: vi. what does the law say? vii. what chance is there it will be enforced? particularly in web site development, where cross-border business is concerned, legal compliance becomes a complex issue. the cases above show that compliance in one jurisdiction does not imply one's product complies in another. for instance, there are subtle differences between complying in australia with the accessibility requirements for australian government sites, and compliance requirements for the same sites, in the us. in case 2, mention of particular standards (wcag and 508) are introduced to lead participants to focus on relevant legislation. conclusions though this technique evolved out of the need to help students think through ethical dilemmas in a systematic way, it is not limited to a student context. the process will work just as well in peer evaluative situations. for instance, one of the limitations referred to above is that when confronted with an ethical dilemma in the workplace, it is difficult to remain objective in one’s analysis. in such a situation, this technique will help a knowledgeable peer/co-worker to work objectively through the issues to develop appropriate advice on how best to proceed. similarly, members of the acs are encouraged, when confronted with ethical dilemma, to seek help from the society and/or from the australian institute of computer ethics (aice). people who are asked for advice need a way of sifting through the pertinent issues to devise an appropriate response. the acs code of ethics goes part of the way to do this. yet it really only sets the standard, as it does not provide a way of arriving at the best (or at least a satisfactory) solution. what has been proposed in this paper is a recipe for formulating such a solution. further work is needed to validate the technique. at this time, there is only anecdotal evidence from tutors involved in formative and summative assessment of student work, and from students themselves. there is also anecdotal work that supports the value of such scenarios when trying to alert practitioners to the problems of accessibility, and when working towards recommendations for action in response to those who request help from aice members. formal data needs to be gathered as to the efficacy of this technique for isolated practitioners, as well as to enhance it, if possible. ajis vol. 10 no. 2 may 2003 133 references anderson, r.e., johnson, d.g., gotterbarn, d. & perrolle, j. (1993) using the new acm code of ethics in decision making, communications of the acm, 36(1), pp.98-107. burmeister, o. k. (2000), applying the acs code of ethics, journal of research and practice in information technology, 32(2), may, pp.107-120. clement, a. (1993): privacy considerations in cscw: report of the cscw’92 workshop, sigchi, acm press, 25(4), pp.34-39. innes, g. (2002) request for proposals: web site access survey, hreoc disability rights, http://www.humanrights.gov.au/disability_rights/webaccess/survey02_rfp.htm, 18/7/2. preece, g. (2002) a war-zone in the womb, the melbourne anglican, may, no 393. sampson-wild, g. and burmeister, o. k. (2001) the continuing evolution of best practice principles in designing for web accessibility, proceedings ozchi2001 usability and usefulness for knowledge economics, nov, pp.122-127. microsoft word special final.doc ajis special issue december 2004 3 editorial this special issue of ajis features a selection of the papers from the ozchi2004 conference, which was held at the university of wollongong between 21 st and 24 th of november, 2004. ozchi is an annual conference held by computer human interaction special interest group (chisig) and is the premier australian conference on human computer interaction. ozchi2004 attracted academics and practitioners from around the world, with delegates from australia, canada, china, denmark, france, finland, germany, japan, korea, netherlands, new zealand, the uk, and the usa, among others. the conference consisted of a series of papers, posters and panels on the theme of: “supporting community interaction: possibilities and challenges” and other more general areas of interest to the hci community. the papers selected for this special issue give a real sense of the breadth of research carried out in the hci community. the papers explore different technologies, address different human issues, and use different theoretical and research approaches to do so. the papers describe novel uses of technology, such as the interactive library floor, intended to promote interaction between library users and to foster a sense of community (krogh et al). another paper questions the effectiveness of the current “desktop metaphor” and proposes a number of ways in which our computing space might be better organised (nielsen and bodker). in an attempt to provide systems for the domestic environment, kjeldskov et al. present the design of a memorabilia manager or a family digital assistant, while gronbaek and petersen address similar domestic issues in their design of a mediawall for use in domestic environments bennett and stevens explain the use of projection augmented models, physical models onto which graphical textures, colours and other effects can be projected. so the papers introduce us to a variety of exciting new technological developments. the papers also address a wide range of “human issues” – problems that arise because of the human dimension, rather than from the technology. the paper by weakley and edmonds looks at the needs of distributed groups of creative workers, specifically designers. kjeldskov et al. address the emotional aspects of hci, particularly the need to represent intimacy in a family setting, rather than in the business or organisational settings that have dominated much of hci research. hertzmun et al. explore the question of security and e-banking from a user’s point of view rather than from the technical aspects of encryption or user identification. other papers deal with the problems of hci designers themselves and the need for special design tools or approaches. subramanian and champoux explain that the design of tangible interfaces i.e. interfaces which we touch, are significantly different to other design tasks and they propose some design tools and approaches fro this special class of interface. not surprisingly, the variety of technologies and human issues covered in these papers requires a similarly broad set of research methods including case studies (nielsen and bodker ), the use of cultural probes to explore meaning in human interaction (kjeldskov et al), qualitative empirical studies (gronbaek and petersen) and prototyping ( krogh et al). whatever your background, there will be something new and exciting in these selected papers peter hyland ajis vol. 8 no. 2 may 2001 studies in organisational semiotics: an introduction rodney j. clarke department of information systems, university of wollongong, australia phone: +61 42 21 3752 fax: +61 42 21 4474 rodney_clarke@uow.edu.au orientation to semiotics the broad application of semiotic approaches to organisations has been considered by a number of information systems academics to be a necessary advance in information systems theory (see for examples land 1985, rzevski 1985, and tully 1985). along with psychology and sociology, semiotics is considered to be a foundation discipline for information systems within the ifip wg 8.1 frisco framework (falkenberg, et al eds/ 2000). semiotics examines the processes of production and consumption of meanings in organisations, institutions and society, and their underlying mechanisms by means of what pap (1991, 47) refers to as a "...systematic analysis of patterns of interpretive behaviour". although often unacknowledged, meaning is central to any definition of an information system. while the concept of meaning and meaning making is difficult to define, semiotic theory can assist by emphasising the distinctions between 'information', 'meaning', 'sense' and 'reference' for example (see noth 1990, 92-102). eco (1976, 8), provides a broad definition of semiotics as the study of "...all cultural processes as processes of communication". cultural processes are interpreted to include organisational contexts and processes thereby providing a link between systems and organisations. most applied semiotic studies start by identifying or defining one or more models of the sign as the basic unit of analysis. signs are usually glossed as 'something that stands for something else in some capacity or another'. depending on the model of the sign, mention may be made to an entity for whom the 'stands for' relationship applies. for a discipline often defined as the 'study of signs', there are a plethora of distinct sign models from which to choose. the reader is directed to winfried noth's handbook of semiotics for a detailed description of different sign typologies, sign models and disciplinary history (noth 1990, 79-91). the period of modern semiotics starts at the beginning of the 20th century with the emergence of two independent traditions. the work of ferdinand de saussure represents one of these traditions. considered to be the founder of modern linguistics, he is also a pivotal figure in semiotics by distinguishing the former from the later in cours de linguistique generate (1916)a volume assembled from lecture notes edited by colleagues and published three years after his death (currently available as bally, sechehaye and riedlinger eds/ 1993). the other tradition is represented by the work of charles sanders peirce, a founder of symbolic logic as well as an expert in philosophy, mathematics, and many other fields. peirce developed a pansemiotic view so all-inclusive and elaborate that the definitive, chronological collection of his work called the peirce edition project remains unfinished, only six of an estimated twenty volumes of have been published to date (see for example houser et al eds/ 1999). other semiotic traditions exist. there are semiotic forms of linguistics referred to here as semio-linguistics which adopt a semiotic theorisation of text rather than using an explicit sign model during analysis. examples include text semiotics, systemic functional linguistics (halliday 1978, 1985; martin 1992) and systemic semiotics (fawcett 1986). there is also at least one influential form of semio-linguistics that does uses an explicit sign model (hjelmslev 1943/1963), while social semiotics (kress 1985; hodge and kress 1988) develops concepts of discourse, text and social subjectivity based on the 'social sign' of bakhtin (see todorov 1984). despite the obvious applicability to information systems of a discipline that concerns itself with the study of 'patterns of interpretive behaviour', there have been impediments to more vigorous interchange between semiotics and informatics. there are several contributory reasons for this situation. a major contributory reason is that western semiotics likes to trace its lineage back to the stoics (approximately 300 bc to 200 bc) and the epicureans (300 bc to 0) through to the enlightenment and onto the 19th and 20th centuries. faced with this daunting history, and a diversity of researchers, theories, and terminology to rival the most entrenched of modem disciplines, it is not surprising that a casual 'dip' into the literature might prove unproductive. moreover, as semiotics is centrally involved with understanding communication, it becomes obvious that many commonsense notions of what constitutes communication would need to be reconsidered. as a consequence, semiotics unusually needs to employ meta-theory of one form or another, which leave it open to charges of obscurantism and elitism (sless 1986, 2). however, it is unreasonable to assume that the constructs used to explain and examine such complex and taken-for granted cultural phenomena must necessarily be simple. another major contributory reason is the difficulty of locating semiotics within any single disciplinesemiotics is inherently trans-disciplinary. broad application domains have included, but are not limited to, the cultural constitution of subjectivity, criticism and knowledge, communication and perception. a sample of subjects using semiotic theory of one form or another would include cultural and literary studies, film criticism, feminism, political science, legal studies, town planning and architecture, anthropology, biology and genetics. moreover, there is considerable debate about what constitutes the core criteria that defines semiotics (pap 1991). 75 ajis vol. 8 no. 2 may 2001 semiotics applied to informatic domains semiotics has always been implicit within informatic domains including computing science and information systems (marcus 1996, 1999). the realisation of the semiotic nature of information systems in organisational contexts has been apparent in the earliest work in semiotics applied to the informatic domains. this is particularly evident in europe and scandinavia, where recognition of the need to create organisationally appropriate information systems has helped to maintain an interest in applying semiotics to informatic domains in general. while specific schools may have flourished for a time, most schools have failed to reach critical mass and have been subsumed within traditional and accepted information systems theories and practices. some specific selections of semiotic theory have been utilised within informatic domains. the preeminent swedish methodologist borje langefors (1966) was an example of one of the earliest information systems academics, circa 1967, who developed an interest in semiotics (nilsson 1995). he proposed the research project that developed into the is ac methodology (dahlbom ed/ 1993, 1995) which had as its definition of elementary information the triadic sign developed by pierce (see noth 1990, 39-47 for a readable account of pierce's semiotics). fiol (1989) provides a relatively early and excellent example of a semiotic analysis of organisational boundaries. despite the apparent applicability of general semiotics within information systems shown by the examples of limited exchange between them there was a relatively sparse and episodic utilisation of general semiotic theory within the information systems and organisations domains. however, since the mid-1990s there has been a broad application of semiotic principles across many application areas. these include but are not limited to human computer interaction (merkle 1997), ergonomics (kostler 1999), text analysis and computational text semiotics (see for example, rieger 1981), hypersystems and hypermedia (andersen 1990, clarke 2001, fendt 1999), agent-based systems and associated methodologies (schmidt 1999, chong and liu 2001), organisational analysis (stamper 2001), knowledge management (van heusden and joma 1999), and systems evolution (clarke 1996, 2000). there has also been an explosion of activity both in the number of received terms in use to describe and label research in the area of semiotics and informatics, and the number of workshops and conferences that have put these ideas into circulation. for example, in 1999 there were four international workshops and conferences in the field of semio-informatics while prior to 1995 there were none, see table 1. received terms and emerging definitions in february 1995, approximately forty researchers, mostly from europe but also from scandinavia and australasia, attended the first international conference and workshop on organisational semiotics at twente university in the netherlands. the term organisational semiotics was coined by its chair ronald stamper, to refer to the use of any semiotic approach in the study of organisations. the measur research group (method for eliciting, analysing and specifying user requirements) that ronald stamper initiated at the london school of economics in the united kingdom, and continued at twente university in the netherlands, had since its formation consistently developed and advocated semiotic thinking in informatic domains (stamper 1973; kolkman 1993; liu 1993, 2000). a principle interest at the conference, also evident at a semiotics and informatics dagstuhl seminar (andersen, nadin and nake 1996) was the application of semiotics to the information systems discipline, referred to as semio-informatics, a term coined and used independently by the author (clarke 1995) and mihai nadin (1998). related to this is the intersection of linguistics and informatics domains, which has been referred to as work language studies (holmqvist and andersen 1987). although there are a number of methodological and substantive intersections between then, it is important to theoretically distinguish organisational semiotic work that utilises semiotic models of language peter b0gh andersen (1990) has coined the term computer semiotics to refer to the application of semiotic principles to computing in its most general sense. the kinds of activities included within the gamut of computer semiotics include amongst other things, program development, interface design, systems description, systems development, work analysis, organisational analysis, and technological assessment (andersen 1990, 18). however, within australia and north america there exists a strong institutional demarcation between computer science, information systems and communication technology, and these distinctions raise some problems in appropriately classifying studies. the use of the term 'computer' in computer semiotics is probably a consequence of the broader meaning of the term 'computer science' within scandinavia, which has propelled andersen's work and that of his colleagues involved in the system development environments and profession oriented languages (sydpol) group (andersen and bratteteig eds/ 1989). this group spanned many interests ranging from the consideration of computers as symbolic and semiotic machines, and the linguistic status of programming languages, to a consideration of work and professionally oriented language in organisations 76 ajis vol. 8 no. 2 may 2001 (andersen and holmqvist 1988). the work of nadin (1998) dating from 1977 and doben-henisch (1998) also use the concept of the computer as a semiotic machine, itself a term which has a history extending back to its coinage by pierce. the field of computer semiotics and semio-informatics are considered here to be synonyms for the intersection of semiotics and information technology is general, and therefore a part of the broader program of organisational semiotics. semiotic engineering has also been a term used by de souza (1998) since 1995 to designate a form of analysis and design which spans the interests of semio-informatics and computer semiotics. the university of technology, dresden recently hosted the 9th international congress of the german society for semiotics studies (dos), followed immediately by 7th international congress of the international association for semiotic studies (iass/ais), october 3-6, 1999. the former conference had as its general theme 'machines and history', where 'machine' was used in two senses referring to a general cultural concept, and also to the physical or conceptual devices describable within a broadly semiotic framework. the latter conference had as its general theme 'sign processes in complex systems' where 'complex system' was interpreted in a broad interdisciplinary sense. this interest from the mainstream semiotic community is due in part to the novel cases that are provided from information systems and computing science for testing and further elaborating traditional semiotic theory and developing new semiotic theories in an area now referred to as computational semiotics. a conference with this theme was organised in 1997 by claude vogel, suzanne bertrand-gastaldy, and jean-claude heudin at the pole universitaire leonard de vinci in paris, france (vogel et al 1997). the 1999 dos and iass/ais represented an opportunity to hold the first computational semiotics session in a mainstream semiotic conference. the session was entitled computational semiotic systems: theories and applications and was chaired by gerd doben-henisch, in response to discussions between prof. burghard rieger, alexander mehler, ricardo gudwin, gerd doben-henisch, lauwrence erasmus, and the author held at is as'98 with the aim of establishing a semiotic theory of computation. while one part of this community has aims similar to computer semiotics, other parts of this community aim to construct autonomous systems with semiotic processing that will provide them with abilities like intelligent behavior, perception, value judgement and behaviour generation (gudwin and gomide 1997; gudwin 1999). work in this issue the title of this theme has been chosen to introduce the new 'studies in organisational semiotics' books being published within the information and organisation design series by kluwer academic press. several of the theme's editors have contributed to this issue. the first volume is entitled "information, organisation, and technology" and is already available (liu et al 2001). a second volume provisionally entitled "coordination and communication using signs" will be in press by the time you read this. this issue is timely given the recent institutional debates concerning the centrality of semiotics within informatic domains. in 1995, ronald stamper was successful in arguing the case for semiotics to be considered, along with psychology and sociology, as a foundation discipline for information systems within the ifip wg 8.1 frisco framework. in particular, the efforts of ronald stamper and kecheng liu have led to the forthcoming ifip wg8.1 working conference entitled "organisational semiotics: evolving a science of information systems" which will soon be held between during 23rd to the 25th july, montreal, canada. in order to review the papers for this special issue we were ably assisted by a committee consisting of the following scholars including: maria cecilia calani baranauskas institute of computing, universidade estadual de campinas (unicamp), brazil; joseph davis basser department of computer science, the university of sydney, australia; ernest a. edmonds department of computer science, loughborough university, united kingdom; henk w. m. gazendam faculty of management and organization, university of groningen, the netherlands; aditya ghose department of information system, university of wollongong, australia; goran goldkuhl linkoping university (cmto), sweden; remigijus gustos department of information systems, karlstad university, sweden; martin kolhnan department of management, ball state university, usa and pricewaterhouse/coopers llp, usa; kecheng liu school of computing, staffordshire university, united kingdom; and laurent perrussel irit/ceriss, university of toulouse 1, france. the authors and i am indebted to the reviewers for their engagement with this work. the authors have been invited to participate in this issue based on their commitment to the field of organisational semiotics. they hail from australia, brazil, canada, denmark, and germanya demonstration of the extent to which this is an international community. specific topics have been chosen to exemplify the range and diversity of the organisational semiotics and to highlight its utility for information systems, organisational studies and related fields. the papers focus on diverse application domains including the analysis of maritime workpractices (peter b0gh andersen), scenario-based design of a workflow system (clarisse sieckenius de souza; cecilia kremer vieira da cunha, raquel oliveira prates and simone diniz junqueira barbosa), development of innovative approaches to text mining (alexander mehler), the analysis of a small operational 77 ajis vol. 8 no. 2 may 2001 level administrative system (rodney j. clarke), and a study into the discourses surrounding concerning knowledge management (rob shields and edwina taborsky). while exhibiting a variety of theoretical approaches and substantive interests, these papers share one feature. all recognise the centrality of human communication as a way of understanding meaning making in institutional and organisational contexts, whether this be signs in design scenarios, communication and work language, text mining and hypertext generation, systems features and workpractices, and the contestable definitions of information and knowledge. 78 ajis vol. 8 no. 2 may 2001 table 1: recent organisational semiotics and computational semiotics workshops and conferences. the shaded entries indicate mainstream conferences that have hosted relevant themes. entries above the thick line have been publicly announced and are forthcoming; those that are below the line have been held. 2001 2000 1999 1999 1999 1999. v 1998 1998 1997 1997 1996 1995 may28 june 2 july 4 oct 15 oct 12-14 oct 3-6 , • \ . ' . oct;3-6 = . > . ' . ' • . ' •.• oct 13-15 sept 14-17 •julyl3-18 may 26-27 feb 19-23 feb 10-15 ifip wg8.1 working conference on organisational semiotics host: montreal, canada chair. ronald stamper 3rd international workshop on organisational semiotics host: staffordshire university, uk committee: kecheng kiu, peter andersen, rodney j. clarke, ronald stamper organisational engineering workshopchallenging the oerparadigm host: delft university of technology, the netherlands chair. jan dietz 2nd international workshop on organisational semiotics host: twente university, the netherlands committee: kecheng liu, peter andersen, rodney j. clarke, jan dietz, marc halfkamp, ronald stamper, joaquim filipe, anastasia pagnoni holt, robert a. stegwee, yasser ades 7th international congressinternational association for semiotic studies (iass/ais) sign processes in complex systibmis : r ^ ;//o5/: techm'caj uniyersiry, dresden,"gennany •\';-''.^'~< /te/evan^^a/r walter schm^^ 9rt international congress of the german society for semiotic studies , (dcs)machtaesai&^ '•'•'^'^•vs/'.i' ':'^;"''' ' " host: technical urnv^ity^dresdei^germanyx . * . •-•.••relevant chair. walterschmitz .. >; , ; : '•:•• ' ' semiotics and the information sciences host: victoria college, university of toronto chairs: jean umiker-sebeok, marcel denesi intelligent systems and semiotics (isas) host: national institute of standards and technology, maryland usa relevant chair. alex m. meystel 6th international congressinternational association for semiotic studies (lass-alsysendoticsbrid^ host: mexican association for semiotic studies: .;;' , £: relevant chair. adrian ;s:,gimate-welsh ;l :^ ^ v~ '">' : . l" international workshop on computational semiotics (iwcs97) host: pole universitaire leonard de vinci, paris, france committee: claude vogel, suzanne bertrand gastaldy, jean-claude heudin, kathleen carley semiotics and informatics dagstuhl seminar host: schloss dagstuhlinrenationales begegnungs-und forschungs-zentrum fur informatik committee: peter b. andersen, mihai nadin, frieder nake l" international conference on organisational semiotics host: twente university, the netherlands committee: ronald stamper, duda nauta, rodney j. clarke 79 ajis vol. 8 no. 2 may 2001 references andersen, p. b. (1990) a theory of computer semiotics: semiotic approaches to construction and assessment of computer systems cambridge series on human-computer interaction cambraidge: cambridge university press andersen, p. b. and t. bratteteig eds/ (1989) computers and language at work: the relevance of language and language use in development and use of computer systems institute of informatics, university of oslo, norway: research reports no 126 andersen, p. b. and b. holmqvist (1988) "work language and information technology" in andersen, p. b. and t. bratteteig eds/ (1988) computers and language at work: the relevance of language and language use in development and use of computer systems sydpol: system development environments and professional oriented languages, institute of informatics research reports no. 126, institute of informatics, university of oslo, norway, 117-155 andersen, p. b.; nadin, m. and f. nake eds/ (1996) informatics and semiotics dagstuhl-seminar-report 135; schloss dagstuhlinternationales begegnungsund forschungszentrum fur informatik 9608, 19/02/9623/02/96 bally, c.; sechehaye, a. and a. riedlinger eds/ (1993) f. de saussure course in general linguistics (cours de linguistique generate) translated and annotated by roy harris uk: duckworth chong, s. and k. liu (2001) "a semiotic approach for distinguishing responsibilities in agent-based systems" in liu, k.; clarke, r. j.; andersen, p. b. and r. k. stamper eds/ (2001) information, organisation and technology: studies in organisational semiotics information and organizational design series, boston/dordrecht/london: kluwer academic publishers, chapter 6, pp. 173-186 clarke, r. j. (1995) "genre in systemic functional linguistics and the analysis of information systems in organisations" first international conference on organisational semiotics february 10th-15th, university of twente, the netherlands clarke, r. j. (1996) "the persistence of systems in organisations: genre analysis of systems decommissioning" in holmqvist, b.; andersen, p. b.; klein, h. and r. posner eds/ (1996) signs of work: semiosis and information processing in organisations berlin: walter de gruyter, pp. 59-106 clarke, r. j. (2000) "an information system in its organisational contexts: a systemic semiotic longitudinal case study" phd dissertation, department of information systems, university of wollongong clarke, r. j. (2001) "towards a systemic semiotic approach to multimedia interface design" in liu, k.; clarke, r. j.; andersen, p. b. and r. k. stamper eds/ (2001) information, organisation and technology: studies in organisational semiotics information and organizational design series, boston/dordrecht/london: kluwer academic publishers, chapter 9, pp. 247-270 doben-henisch, g (1998) 'the emergence of signs hi populations of artificial agents" isic/cira/isas'98: ieee international symposium on intelligent control (isic), held jointly with international symposium on computational intelligence in robotics and automation (cira) and intelligent systems and semiotics (isas), national institute of standards and technology, gaithersburg, maryland, usa, september 14-17, 1998, pp. dahlbom, b. ed/ (1993) essays on infology by borge langefors: summing up and planning for the future gothenburg studies in information systems august 1993, report 5, department of information systems, goteborg university, sweden dahlbom, b. ed/ (1995) the infological equation: essays in honor of borge langefors gothenburg studies in information systems march 1995, report 6, department of informatics, school of economics and commercial law, goteborg university, sweden de souza, c. s. (1998) personal communication eco, u. (1976) a theory of semiotics bloomington: indiana university press falkenberg, et al eds/ (2000) information system concepts: an integrated discipline emerging boston, dordrecht, london: kluwer academic publishers fawcett, r. (1986) "forward" in ventola, e. (1987) the structure of social interaction: a systemic approach to the semiotics of service encounters open linguistics series, london: frances pinter 80 ajis vol. 8 no. 2 may 2001 fendt, k. e. (1999) "non-linear coherence: authorship, readership, and the social construction of knowledge in hypermedia" in machines and history (machinen und geschichte) abstracts: 9lh international congress of the german society for semiotic studies (dcs), technical university, dresden, germany, october 3-6, 1999, p.86 fiol, c. m. (1989) "a semiotic analysis of corporate language: organisational boundaries and joint venturing" administrative science quarterly vol 34, pp. 277-303 gudwin, r. r. and f. a. c. gomide (1997) "an approach to computational semiotics" proceedings of the 1997 isas conference, gaithersburg, 1997,467-470. gudwin (1999) "from semiotics to computational semiotics" in machines and history (machinen und geschichte) abstracts: 9th international congress of the german society for semiotic studies (dcs), technical university, dresden, germany, october 3-6, 1999, p.77 halliday, m. a. k. (1978) language as social semiotic. the social interpretation of language and meaning maryland, usa: edward arnold halliday, m. a. k. (1985) an introduction to functional grammar melbourne: edward arnold van heusden, b. and r. joraa (1999) "knowledge management and innovation: on sign processes in organisations" in machines and history (machinen und geschichte) abstracts: 9 international congress of the german society for semiotic studies (dcs), technical university, dresden, germany, october 3-6, 1999,p.22 hjelmslev, l. (1943/1963) prolegomena to a theory of language menasha, wisconsin: the university of wisconsin press hodge, r. and g kress (1988) social semiotics cambridge, uk: polity press houser, n.; eller, j. r.; de tienne, a.; lewis, a. c.; de waal, c.; bront davis, d.; cummins, l. and c. l. clark eds/ (1999j writings of charles s.peirce, a chronological edition, volume 6 1886-1890 peirce edition project, indianapolis: indiana university press holmqvist, b. and p. b. andersen (1987) "work language and information technology" j. pragmatics 11, 327357 kolkman, m. (1993) problem articulation methodology phd thesis, twente university, enschede, the netherlands koster, l. (1999) "foundation of semiotic ergonomics" in machines and history (machinen und geschichte) abstracts: 9lb international congress of the german society for semiotic studies (dcs), technical university, dresden, germany, october 3-6, 1999, p.78 kress, g (1985) linguistic processes in sociocultural practice ecs806 sociocultural aspects of language and education, victoria: deakin university land, f. (1985) "is an information theory enough? the computer journal 28 (3) 211 -215 langefors, b. (1966) theoretical analysis of information systems lund: studentlitteratur liu, k. (1993) semiotics applied to information systems development university of twente, enschede, the netherlands liu, k. (2000) semiotics in information systems engineering cambridge, uk: cambridge university press liu, k.; clarke, r. j.; andersen, p. b. and r. k. stamper eds/ (2001) information, organisation and technology: studies in organisational semiotics information and organizational design series, boston/dordrecht/london: kluwer academic publishers marcus, s. (1996) "semiotically relevant aspects of informatics" in andersen, p. b.; nadin, m. and f. nake eds/ (1996) informatics and semiotics schloss dagstuhl, internatuionales begegnungsund forschungszentrum fur informatik, dagstuhl-seminar-report 135, 19th 23rd february 1996 marcus, s. (1999) "from abstract machines to semiotic machines and back: a semiotic itinerary" in machines and history (machinen und geschichte) abstracts: 9th international congress of the german society for semiotic studies (dcs;, technical university, dresden, germany, october 3-6, 1999, p.24 martin, j. r. (1992) english text: system and structure amsterdam: john benjamins merkle, l. e. (1997) "cognitive signs, human-computer interaction and semiotics: subliming multifold boundaries" semiotics bridging nature and culturevlth international congress, international association for semiotics studies (iass-ais), guadalajara, mexico, july 13-18, 1997 nadin, m. (1998) personal communication nilsson, a. g (1995) personal communication noth, w. (1990) handbook of semiotics bloomington and indianapolis: indiana university press pap, l. (1991) semiotics: an integrative survey toronto semiotic circle, victoria college in the university of toronto monograph series of the tsc, number 7 rieger, b. b. (1981) "feasible fuzzy semantics: on some problems of how to handle word meaning empirically" in eikmeyer, h. j. and h. rieser eds/ (1981) words, worlds, and contexts. new approaches 81 ajis vol. 8 no. 2 may 2001 in word semantics research in text theory 6, berlin and new york: walter de grayter, 193-209 rzevski, g (1985) "on criteria for accessing an information theory" the computer journal 28 (3) 200-202 schmidt, k-h. (1999) "a semiotic view on agents with computable perception" in machines and history (machinen und geschichte) abstracts: 9th international congress of the german society for semiotic studies (dcs), technical university, dresden, germany, october 3-6, 1999, p.78 sless, d. (1986) in search of semiotics london and sydney: croom helm stamper, r. (1973) information in business and administrative systems london: batsford stamper, r. (2001) "organisational semiotics: informatics without the computer" in liu, k..; clarke, r. j.; andersen, p. b. and r. k. stamper eds/ (2001) information, organisation and technology: studies in organisational semiotics information and organizational design series, boston/dordrecht/london: kluwer academic publishers, chapter 5, pp. 115-171 todorov, t. (1984) mikhail bakhtin: the dialogical principle translated by wlad godzich, theory and history of literature (thl) volume 13, minneapolis: university of minnesota press tully, c. j. (1985) "information, human activity and the nature of relevant theories" the computer journal 28 (3), 206-210 vogel, c.; bertrand-gastaldy, s. and j-c heudin (1997) statement for the 1st international workshop on computational semiotics. 26th-27thmay 1997 82 microsoft word ajis10-2finala.doc ajis vol. 10 no. 2 may 2003 91 student use of the internet: an extension of tam in technical & vocational institutions in brunei darussalam afzaal h. seyal+ mohd. noah abd. rahman +e-mail: ahseyal@itb.edu.bn fax: 673-2-249036 department of computing and information systems, institute of technology brunei, tungku link, bsb, be 1410 brunei darussalam. abstract in the educational setting, the internet is profiteering among academics and students. in fact, this has changed the functioning of the educational establishments. however, the success of use of the internet is largely dependent upon the user’s behaviour that in turn affects their attitudes. even when the formidable opportunities exist for the deployment of technology, adverse attitude can inhibit the use. keeping this in view, survey of 266 students of four technical and vocational colleges was conducted to study the attitudes of the students toward the use of the internet. the present study develop a normative model by using davis’s technology acceptance model (tam) and validates two specific attitudinal variables; perceived usefulness and perceived ease of use that are hypothesized to be fundamental determinants of the attitude that in turn predict the students’ use of the internet. by adding external variables such as; demographics, task characteristics, computer exposure, and institutional support further test the parsimony of the model. a structural equation modelling techniques is used to validate the model. the study has confirmed that majority of the students (85%) are using the internet. in the final model the external variables do contribute towards both of belief parts: perceived ease of use and perceived usefulness that further affects the attitude of the students in predicting the internet usage. keywords: internet usage, technology adoption model, technical & vocational students, structural equation model (sem). introduction the use of the internet for business purposes in developed countries has become widespread over the past twenty years (soh et al. 1997). it has become a vital tool for survival for various small businesses to compete with larger organizations (kaplan et al. 1997). this manifold increase has not only brought a revolution in the business world but has also made the internet as a favorite research topic among the researchers. with globalization the businesses are becoming competitive everyday and similarly the demand for resourceful, skillful and knowledge workers are increasing. in order to cope with the industry demand, educational institutions have to develop a new learning culture and to radically shift their existing resources of teaching to the new technology based curriculum. the use of the internet among educational setting is gaining threshold popularity with the large number of academics and students are getting the benefits of the internet technologies. given the opportunities and benefits that the internet can provide the usage is still limited to infant stage. besides countless promises of the technology, empirical research on the adoption and use of the internet is relatively sparse. most of the prior research in this discipline has predicted the user behavior based upon the data collected in the united states and europe. for example, in usa by 1995 more than 40 states provided public educators with some level of internet access (doty, 1995). from the period of 1994-1995, access to the internet from us public schools has increased from 35% to 50% (heaviside, et al. 1995). comparatively, less has been researched on the adoption, diffusion and impact of the internet among educational settings. no doubt, the internet is permeating professional life and is being promoted as the future way of information transfer. however, we believe that academic institutions are expected to be in the forefront of applications of new technologies. previously, the studies on the internet were undertaken among academics and students of the universities. little is however, researched about the use of the internet among the academics of the technical colleges. it is argued that task characteristics, level and skill of computer experience may differ from their counterpart in the universities. this difference may have an impact on the successful use of the internet among the technical institutions. this aspect was not addressed by the prior studies and hence there is a need to investigate this aspect into new geographical environments. as mentioned, a majority of studies were undertaken in the western countries. however, very little has been researched in the asia-pacific countries. in addition, all these studies have a number of weaknesses. they are based upon single institution survey and a complete picture of the potential users, therefore cannot be analysed. in the present study, an attempt is made to cover the students of technical colleges at large to bring the broader perspective of the internet use among the learners and to make the study different from the ajis vol. 10 no. 2 may 2003 92 previous studies. in addition, this study has used a structural equation modelling techniques to test the hypothesized model that was missing in the prior studied. thus the result of these prior studies might not be applicable to a small but technologically emerging country like brunei darussalam. the country is culturally different not only from those of the western world but also from several of the asia-pacific countries. brunei darussalam is a small sultanate located on the northwest coast of borneo island. strategically located between the two technological hubs i.e., singapore and malaysia. it has a total population of nearly 0.3 million (brunei statistical year book, 1999) with main economic activity dominated by the oil and gas sector. the country had gross domestic product per capita was b$ 23,865 (us$1= 1.70) in 1999. the government has conceived an it vision and has taken a range of measures to improve it infrastructure and the internet business environment in the country. realizing the limitation as to the size of the domestic market, brunei business environment is determined to utilize the internet as a major development tool. in the educational side, currently, two engineering colleges, one vocational college and one technical institute were established to produce technologically oriented professionals at various levels. no doubt, brunei is technologically oriented country with impressive it infrastructure, however, the utilization of the various technological elements are mainly dominated by cultural and belief systems of the users. in the absence of extensive research it is difficult to find out the root cause of this under utilization. so keeping in view the users’ aspect of utilization, the study was undertaken to collect empirical evidences of the internet usage among the students of technical colleges in brunei darussalam. objective of the study the central theme of this study is to examine the student use of the internet and this has been further classified into two sub-sections: (a) to assess the students’ current level of the internet use and to identify the factors contributing toward the use. (b) to test the hypothesized model by using structural equation models and to propose a model that have a greater parsim review of technology acceptance models the internet use among educational settings is the area least researched comparative to the other segments of the users. a variety of factors may have influenced the limited use. the success of the internet use however, is a multi-dimensional issue that requires the cooperation and commitment from all participants. the research on the internet in fact is the extension of previous researches on technology diffusion, it adoption and utilization. the studies were undertaken to develop various models. mis researchers have used different models over the last two decades as a base for studying the information systems acceptance, adoption and usage. in earlier work, triandis (1971) argued that behavior is determined by what people would like to do (attitudes) what they think they should do (social norms) what they have usually done (habits) and by the expected consequences of their behavior. he suggested that attitudes involve cognitive, affective, and behavioral components. the cognitive component of attitudes involves belief. thus attitudes involve what people believe (cognitive), feel (affective), and how they behave (behavioral) toward an object that can be a computer at early stage and later on can be substituted for information technology. the triandis’s theory has provided a base for conducting attitudinal studies. however, researchers in the later stages have proposed several other models based upon innovation diffusion theory (rogers, 1995), the theory of reasoned action (tra) purported by ajzen & fishbein (1980), and the technology acceptance model (tam) by davis (1989). of these models the tam has most widely used model to predict computer-usage behavior with generally providing determinants of technology acceptance that would help explaining user behavior towards the information technology usage. tam has also been shown to demonstrate good predictive validity (szanja, 1994). however, one of the constraints that was not undertaken by the tam was the ignorance of external factors whereas, later researches has provided us with several of external variables that have an influence on the usage behavior (pijpers et al. 2001). agarwal and prasad, (1997) have further suggested that external variables are the only channels for influencing behavior as the intermediate and dependant variables in tam are hypothesized to be an internal psychological process. tam asserts that actual use of system is determined by behavioral intentions to use and this in turn is related to attitude towards using the system. davis, (1989) while proposing his model further asserted that influence of external behavior is mediated through user belief and attitude. belief related to an action where as, attitude ajis vol. 10 no. 2 may 2003 93 is purely affective that relates to positive and negative feelings about performing the behavior. the belief segments that determine the attitude are perceived ease of use (peou) and perceived usefulness (pu). the tam has been widely studied in information systems research as an explanation of the use of is across information systems types and nationalities and found significant cross-cultural difference (gefen and straub, 1997). in a recent review, saga and zmud, (1993) identified twenty empirical studies that aimed at exploring the factors that determine it acceptance. among these twenty studies, tam originally proposed by davis is arguably one of the most cited. table 1 provides us with a detail of empirical studies based on tam. in fact, all these studies have focused primarily on the tam model in predicting the user intentions towards information technology. the literature tends to fall into a number of categories in the educational setting. most of the past studies were confined to the schoolteacher’s usage of the e-mail. it was undertaken at the level of adoption and utilization of the technological innovation. songan and noor (1999) have studied the students’ use of the internet. doyle and ponder (1977) and cuban (1986) found that positive relationship existed between acceptance of innovation when they seem to be compatible with the previously established value, norms, procedure and facility. brown and mcintyre (1982) found that perceived practicality and cost of implementing an innovation were crucial to utilization. the large part of these studies deals with its development and general definition and terms relating to the facilities that the internet provides (gould, 1995 and stix, 1994). palvi et al. (1995) studied both academic staff and undergraduate students population in one of the singaporean universities and indicated a higher level of awareness between both groups, however, the use was primarily limited to e-mail. survey has further indicated that majority of the respondents felt that undergraduate students must be internet literate. songan and noor (1999) studied internet utilization among students at an institution of higher learning in malaysia and found that gender and faculty’s type have no impact on the internet utilization. where as, factors such as relative advantage, compatibility, complexity, importance and interactivity affect the use of internet. table 1 empirical studies testing tam researchers year technology features studied davis 1989 e-mail and graphic software chart master davis et al. 1989 word processor & text-editor thompson et al. 1991 pc mathieson 1991 spreadsheet sproull 1991 e-mail rice and aydin 1991 e-mail adams et al. 1992 voice-mail and wp lu & gustafson 1994 computerized support system markus 1994 e-mail straub et al. 1995 voice-mail chin and gopal 1995 group support system igbaria et al. 1995 computer usage szanja 1996 data-base management system straub et al. 1997 a cross-cultural studies of e-mail use gefen & straub 1997 perception & use of e-mail lou et al. 2000 groupware acceptance yang & choi 2001 office-automation and internet use veiga et al. 2001 it acceptance pijpers et al. 2001 executive information systems seyal et al. 2002 academic use of internet in australia, spennermann (1997) surveyed student attitudes towards use of e-mail at one of the australian universities and significant differences in positive usage were noticed between external as well as internal students. the internal students are more likely to use e-mail as a method of communication with their academics. alscili (2000) has researched that despite the widespread use of the internet, there are no well-organized studies providing information about its use as a research tool and in classroom instruction at one school of information systems in usa. the studies cited above have also identified a variety of factors that affect the users’ attitudes of internet among academic work-settings. some of the frequently reported factors such as computer exposure include ownership of a pc, and pc experience, task characteristics. pc experience was given considerable attention from some authors in addition to two attitudinal factors pu and peou. ajis vol. 10 no. 2 may 2003 94 although the review of the literature has provided a base for the choice of the variables to be included in this study, however, final selection of the study variables are in fact based upon several others brunei-based studies in is/it (rahim et al. 2000; seyal et al. 2000, 2001). giving the ongoing importance to the use of the internet the selection of the variables are also in line with the objective to test a parsimonious predictive model that is multi-dimensional and posits multi-items variables: demographical variables, pc exposure, task characteristics, institutional support, belief and attitudinal aspect. by testing all the factors together in one model, we are able to investigate their relative contributions to the student use of the internet. this model can be generalized to other inter-organizational information technology usage. research model on the basis of existing literature and of several informal discussions with the members of academic staff of technical colleges during the early stage of research design, a normative model was developed. this model provides the basis of research. figure 1 theoretical research model (extension of tam) the normative model as depicted in figure 1 relates independent (externals) and dependent variables with the intervening or intermediate (perceived ease of use, perceived usefulness and attitude) variables. the relationship as shown in the model is associative rather than casual in nature. in this study, the student actual use of the internet is the dependent variable, while the normative model ties together four constructs having nine independent variables. justification for inclusion of each independent variable and to set up hypotheses in the model is presented in table 2. perceiv perceived usefulne ss atti tud actual use of demographic variables: age, gender, geographical location of the institutions pc exposure: pc know how frequency of pc use prior pc experience task related: task characteristics task variety institutional support ajis vol. 10 no. 2 may 2003 95 a. demographics age ha1 negative pijpers et al. 2001 gender ha2 men more positive songan & noor, 1999; gefen & straub, 1997; igbaria et al. 1995; pijpers et al. 2001 geographical location ha3 new variable studied positive in one studies & neutral with two others studies rahim et al, 2000; seyal at al. 2000, 2002 b. pc exposure pc ownership hb1 negative positive seyal et al. 2002 rahim et al. 2000; seyal et al. 2000 pc know how hb2 positive igbaria et al. 1995 frequency of use pc hb3 positive igbaria et al. 1995; igbaria & chakraborti 1990 pc experience hb4 positive igbaria & chakraborti, 1990; igbaria, 1992; aljabri & al-khaldi, 1997 c. task-related task characteristics hc1 positive negative kwon & zmud, 1987; yverbaum, 1988 igbaria et al. 1995 seyal et al. 2002 task variety hc2 positive ghani, 1992; goodhue & thompson, 1995 d. institutional support hd1 positive pijpers et al. 2001 e. intervening variables perceived ease of use he1 positive with pu goodwin, 1987; igbaria, 1992 perceived ease of use he2 positive with attitude to use davis et al. 1989; bajaj & nidumolu, 1998; pijpers et al. 2001 perceived usefulness he3 positive with attitude to use thompson et al. 1991; igbaria & tracker, 1994; pijpers et al. 2001 attitude towards use he4 positive with actual use bajaj & nidumolu, 1998 table 2 summary of hypotheses the prior literature is full of studies that has used tam and related it to numerous external variables. interestingly no consistency has been noticed in making choice of external variables. pijpers et al. (2001) have used twenty-seven variables and grouped them into demographical, organizational task related, characteristics of it resources and attitudinal variables that measure pu, peou and affective attitudes segment. as mentioned earlier, the selection of external variables for the present study is exclusively based upon the prior research in technical colleges of brunei darussalam. research methodology design of instrument the inclusion of the various multi-item, multi-dimensional constructs in all four parts of the instrument is based upon existing constructs taken from previous researches (davis, 1989; igbaria and chakraborti, 1990; igbaria and tan, 1997; turner and zheng, 1998; harris, 1999). construct measuring computer experience is adapted from igbaria (1992). table 3 provides the detail of items measured along with the mean and corrected-item total. the construct measuring attitudes part is adapted from bijaj and nidumolu, (1998). it is commonly believed that use of the internet is a multi-dimensional construct that measures the various types of the internet use. as such to determine the use of the internet is one of the objectives of this study, therefore table 4 provides the detail of various types of usage along with their mean and standard deviation. multipleact indicators measure the dependent variable; the use of the internet. likewise computer experience, the internet use was measured as overall indexes of total number of categories the respondents reported the use. construct measuring pu and peou intervening variables asks the respondents for twenty-six statements adapted from davis, (1989). constructs no prior relationship tested source of selection ajis vol. 10 no. 2 may 2003 96 table 3 computer experience items responses above average mean experience correct ed-item total using computer packages such as spreadsheet and word processing 56% 3.51 .34 using application languages such as, sql, oracle, dbase 1v, access 12.5% 1.95 .61 using decision support packages such as, financial, statistical or graphical 10% 1.96 .66 use of programming language such as, cobol, fortran, pascal, c, c++ 19% 2.19 .67 participation in the non-technical design of computer system 13% 1.76 .71 participation in the technical design of computer system 13% 1.79 .67 population and sample size there are 4 technical colleges in brunei darussalam that offer trade certificate, national diploma (nd) and higher national diploma (hnd) programs in various disciplines. the target population was the students of these technical colleges. the total numbers of students in all these four colleges were reported to be 1266 (anonymous, 1998). the questionnaire was therefore sent to one third of the students (422) based upon the stratified random sampling techniques and by fulfilling criteria (zikmund, 1984) out of these total 266 responses were received to further test the model. table 4 internet use items mean std.dev to gather student material 3.71 1.23 to gather teaching aids 3.61 1.13 to perform learner activities 2.52 1.21 to enhance general knowledge 3.46 1.13 to keep informed of interesting developments 3.20 1.36 to keep abreast of exhibitions/trade shows and conferences 2.30 1.34 to subscribe to scholarly users group 2.09 1.16 for recreational use 2.83 1.41 instrument validation and reliability there are several types of validity measure that include the face validity and construct validity. campbell and fiske (1959) proposed two types of construct validity: convergent and discriminant validity. in order to establish face validity, an initial version of the instrument was pre-tested using several students selected randomly from two colleges located in close proximity of the authors’ work place. the participants were asked to comment on the format and appropriateness of questions, and to suggest any items that they believed should be included in the instrument. in view of their suggestions, several amendments were incorporated into the instrument, with the inclusion of some new variables that has greatly improved clarity. the derived instrument was then tested for reliability. in line with our research model, as mentioned earlier, part of the instrument that measures the intervening variables pu and peou consists of 26 items is adapted after davis (1989) the original version was used to test e-mail and a graphic software chart-master, however, all the items were reworded for this study. these statements were passed through purification and elimination as proposed by churchill, (1979). 6-items were taken away because of their lowest corrected-item correlation (< .40). then the remaining 20-items were tested for exploratory factor analysis that has resulted for 2-factors solution. the remaining 20-items, provides with two factors solution (pu and peou) in table-5. ajis vol. 10 no. 2 may 2003 97 table-5 varimax rotated factor loading and eigenvalues with variance explained items no items description factor 1 factor 2 att-1 my job would be difficult to perform without the internet .80 att-2 use of internet gives me a great control over my job .82 att-3 use of internet improves my job performance .79 att-4 interacting with internet is confusing .82 att-5 i make frequent errors when using the internet .79 att-6 interacting with the internet is frustrating .78 att-7 the internet system addresses my job related needs .70 att-8 use of internet saves time .76 att-9 the internet enables me to accomplish task more quickly .82 att-10 i need help when i use the internet .65 att-12 overall i find internet useful in my job .69 att-13 i find it easy to get internet to do what i want to do .79 att-14 use of the internet allows more to accomplish more work than would otherwise be possible .83 att-15 use of the internet reduces the time i spend on unproductive activities .73 att-16 use of internet enhances my effectiveness on the job .77 att-19 my interaction with the internet is easy for me to understand .80 att-20 it is easy for me to remember how to perform tasks using the internet .85 att-22 use of the internet improves the quality of my work .85 att-23 use of the internet increases my productivity .84 att-24 use of the internet makes it easier to do my job .83 eigenvalue 15.51 1.68 % of variance 64% 17% note: factor 1 refers to perceived usefulness and factor 2 refers to perceived ease of use. both factors explain 81% of the variance. the kaiser meyer-olkin measure of sampling adequacy of .85 suggests that data is suitable for factor analysis. factor analysis has further revealed the construct validity as such all the items have highest factor loading (.50 or above) and load on two factors. the factorial validity further supports construct validity. the derived instrument was then tested for discriminant validity. there are several approaches to apply discriminant validity. the term refers to the extent to which a concept differs from others (campell and fiske, 1959). one way of determining this for a construct to see if it correlates with other constructs is less than its cronbach alpha coefficient (gaski and nevin, 1985). bajaj and nidumolu, (1998) has used the same technique in one of their attitudinal-based study. table 6 reflects the comparison of the cronbach alphas with the correlations and found to be true. ajis vol. 10 no. 2 may 2003 98 table 6 correlation between the various constructs constructs mean std dev interna l reliabili ty (.70 or above) 1 2 3 4 5 6 7 8 1.pc experience 2.15 .77 .83 1.00 2.task characteristics 3.15 .86 .89 .238* * 1.00 3.task variety 2.95 .84 .90 -.09 .517 1.00 4. internet use 2.95 .80 .92 .214* * .302* .325* * 1.00 5.institutional support 2.29 1.00 .92 -.507 -.062 .225* .220* * 1.00 6.perceived usefulness 3.63 .68 .97 -.012 .133 -.015 .288* * .280* * 1.00 7.perceived ease of use 2.87 .80 .83 .155* .234* * .285* * -.109 .199* -.056 1.00 8. attitude 3.45 .66 .87 -.036 -.049 .042 .185* -.110 .697* * .215* * 1.0 0 significance .** (p<.0.05) * (p<0.01) analysis & results 266 questionnaires received were analyzed to test the research model in fig 1 by using spss for descriptive analysis. sepath, sem program from statistica 6.0 statistical software was used to test the relationship and to further test the parsimony of the structural model. table 7 reflects the background profile of the students. structural equation model (sem) estimates a series of separate but interdependent multiple regression equation (stelzl, 1986). many researchers (joreskog and sorborn, 1983; kelloway, 1998) proposed a twostage process: (1) to estimate the measurement model and (2) to investigate the structural model. measurement model when studying the structural model, many researchers (kelloway, 1998; breckler, 1990) propose a two-stage model: (i) measurement model, (ii) investigating the structural model. theoretically, in a measurement model an investigation into the structure between items and constructs is carried out. normally a multi-trait, multidimensional approach is adopted. initial screening indicated that some of the external variables are not normally distributed and are skewed. by testing the measurement model we therefore test the reliability and validity of the instrument. structural model stelzl (1986) and lee and hersberger (1990) have proposed rules for generating equivalent model. in general, this involves altering the pattern of relationships or paths within the original model so as to produce an alternative model. as mentioned, single stage analysis was conducted for various sub-models (1-3). the result has indicated that the external variables had no significance at all. at the outset, demographic variables gender age and geographical location of the institution were trimmed off and than later on; pc ownership, pc use and pc knowledge were taken out. as such all these variables are either based upon dichotomous or categorical scale and the initial data analysis has revealed that these variables are very much skewed. the pc knowledge and pc use are self-reported items with possibility of response errors. so we have excluded them from our first measurement model. the first model was therefore tested for the remaining external variables with the intervening variables perceived usefulness and perceived ease of use. unfortunately this model has very little parsimony. ajis vol. 10 no. 2 may 2003 99 table 7 background profile of the students (%) geographical location of institution capital 1 capital 2 capital 3 remote 34 27 7 32 gender male female 55 45 age between 15-20 between 20-25 above 25 52 44 4 internet facility yes no 85 15 location of internet facility at home at institution both 67 5 28 pc use (frequency) less than a year between 1 – 5years between 5 – 10 years more than 10 years 15 59 23 3 pc ownership own one does not own 85 15 pc knowledge novice mature expert 25 66 9 table 8. comparison of goodness-of-fit measures goodness-of-fit measure recommended value * model 1 model 2 model3 chi-square/degree of freedom ≤ 3.0 10.7 8.7 3.2 goodness-of-fit index (gfi) ≥ .90 .71 .73 .78 adjusted goodness-of-fit-index (agfi) ≥ .90 .41 .49 .70 normed fit index (nfi) ≥ .90 .42 .25 .83 not-normed fit index (nnfi) ≥ .90 .02 .05 .85 comparative fit index (cfi) ≥ .90 .47 .24 .87 root mean square approx (rmsea) .005 .02 .17 .01 r2 .18 .20 .40 (*recommended values are adapted after kelloway, 1992) in model 2 all the external variables such as pc exposure, task related and institutional support were tested to determine the student’s use of the internet without the intervening variables. this model has very little support as evident from the table 8. it further explains the 20% of the total variations toward the use of the internet. while assessing the parsimony of the model falk and miller, (1992) criteria were considered that suggests that loadings on paths between latent variables and manifest variables should be 0.55. the pc ownership and ajis vol. 10 no. 2 may 2003 100 pc knowledge do not meet the criteria so they were dropped out from the model. after trimming the significant path analysis between exogenous constructs (pc experience, task characteristics, task variety and institutional support were studied. table 9 below shows that a number of external variables had a significant effect on pu and peou and attitude. as hypothesized, pu and peou determine attitude toward use jointly. although most of the prior research in tam exhibit roughly equivalent influence on the acceptance and use of an information technology, however, in our case the path coefficients between peou and attitude is not very attractive although the associated p-value is significant, yet it is not a significant predictor of attitude. in other words, the pu mainly determines attitude. ease of use is not perceived by the students to be related to the functionality of the internet--i.e. its usefulness, because it interferes with their daily routine work might be an aid to complete various assignments. finally, attitude had a positive relationship with the use of the internet and from table 9 seems to be very strong. this model had a better predictive power as compared to two others with about 40% of the total variance is explained. moreover, its chi-square/degree of freedom ratio is close to the recommended value as indicated in table 8. discussion one of the objectives of this study was to identify the current level of use of the internet among students. the study has produced interesting findings. 85% of the students have used the internet facility. out of them 67% are using the facility at home. the institutional resources need to be enhanced to support the increasing demand. the second objective of this study was to develop and test a model for better parsimony. this was achieved through structural equation modeling. to our best knowledge no previous study has ever targeted the students from technical & vocational colleges with the use of the internet and the application of robust theoretical model. in this study the research model was used to identify key factors and relationships likely to influence the acceptance and use of the internet. the result supports the theory that the link between the two beliefs and attitudes as well as between attitudes and actual use is strong. the study has also concluded that external variables influence usage behavior but the belief construct do not fully mediate this influence as the some of the external variables have strong paths coefficients for pu and some others have strong path coefficient for peou. in the study we have used at the outset a set of uncontrollable variables such as, gender, age, geographical location of the institution and the ownership of pc and were excluded in our structural model as being very much skewed. where as, other external variables such as; pc experience, task characteristics, task variety, institutional support are taken as controllable variables. the controllable as well as the intervening variables (pu and peou) are important because we believe that these have an impact on belief and behavior to further encourage the use. the result shows that pc experience is significantly linked to the pu and not significant with peou. this makes sense that as such pc experience measures the students’ competency and skill on pc so the ease of use does not matter at all. the findings are inline with igbaria & chakraborti, (1990); seyal et al. (2002). similarly, the task characteristics that involve the browsing the web for various types of jobs have a significant path coefficient for both pu and peou with minor difference in peou. on the other hand, task variety plays a significant role in influencing belief toward the use of the internet. task variety provides an additional dimension of task characteristics. it is further supported by strong path coefficient for pu. the findings support the preposition that an individual who performs a great variety of tasks by using the internet will become personally involved with technology that enhance the further use. the findings are in line with several prior studies (goodhue and thompson, 1995; harris, 1999). institutional support is in fact available to only 33% of the students. actually only two institutions at present provide this support. the aspect needs to be further examined by the relevant authorities. 67% of the students are using the internet facility at home and the existing setup of service provider is not very cost-effective so the students may foresee this with the ease of use. as more institutional support is expected it will in turn enhance the use at later stages. this is reflected in table 9 with strong path coefficient with the peou. our findings do not fully supports pijpers et al. 2001 who have found that organizational support enhances the actual use but not the belief part. ajis vol. 10 no. 2 may 2003 101 table 9. estimates of the parameters of structural equation model 3 path path coefficient from to perceived usefulness perceived ease of use a t t i t u d e use of the internet pc experience .391 not significant task characteristics .301 .311 task variety .763 .572 institutional support .401 .580 perceived usefulness .625 perceived ease of use .124 attitude .906 (all values are significant at p< .01 except the one) the findings further support the theory that the link between the two beliefs and attitudes as well as between attitude and actual use is strong but if we consider the path coefficient as a measuring parameter, the affect of perceived usefulness is more strongly believed to predict the attitude than perceived ease of use as reflected in table 9. several studies found that peou had no significant effects on intention to use the technology (davis, 1989; igbaria et al. 1995; chau, 1996). this is mainly because of the fact that users are more computer literate and technologies are more user friendly and even most of the college assignments regardless of the area of study involve more and more of web-based search, the peou has become less a factor in technology acceptance decision (chau, 1996). peou only encourage at the more novice to try the technology features and utilities that require wide participation more useful at the early stage. likewise, most of the previous studies, our study also found that perceived usefulness had the greater direct affects on intention to use. it further implies that students would only use the technology (internet) if they think that technology would help them to perform the task. this further leads to develop the favorable student’ attitude that eventually predict the student use of the internet in the technical colleges of brunei. as such, none of the re-specified models present an ideal situation of the acceptance criteria as laid down by kelloway,(1998). we should take precautions to accept the results. overall, model 3 has better parsimony compared to the others with relatively better r2 . as stated by chin & todd (1995), we should not purely rely on model fit criterion; instead, closer attention should be paid to the predictiveness of the model and on the path coefficients. standardized paths should be at least .20 and ideally above .30 in order to considered meaningful. fortunately, our model 3 meets this criterion and can therefore be used to deduct the meaningful deductions. conclusion the study has fulfilled both of the objectives of the research. we found that 85% of the students are using the internet facility. the study provide support for the core tam model as an adequate and parsimonious. as such tam assumes that the influence of external variables on use is channeled through two belief segments that have impact on attitude to predict the use of the technology. this study has confirmed the tam core model. under the three structural equation models the third one with better parsimony have further suggested that external variables; pc experience, task characteristics, task variety, institutional support channelize the two beliefs segment pu and peou that directly affect the attitude and the favorable attitude predicts the students’ use of the internet. these external variables are under the direct control of the administration of the technical colleges. so any attempt to either improve or enhance use of the internet such as more usage for academic purposes should optimize these key controllable variables. the executives and administrators of the colleges must consider these factors so that they can serve the needs of the students within their organization. this is further supported by the fact that overall total effect of perceived usefulness is greater when compared with the total effects of perceived ease of use. this further indicates that students with more practical ajis vol. 10 no. 2 may 2003 102 exercises and tasks are more driven to accept the technology primarily on the basis of usefulness compared to user friendliness. the study similar to several others studies is not free from weaknesses. several limitations of this study qualify the findings and suggest directions for future research. most of the items are self-reported that are liable to be further response-biasness. these response errors can be minimized by some initial workshops at the college level to educate and to provide some training to the students for the research design and research objectives. the study lacks direction in assessing the problems faced by the students at the institutional level and research design could be further improved by adding more items. based upon the results, we therefore recommend another study based upon more variables of the students population in the technical colleges as well as in other colleges and school by adding items measure social and cultural aspects and of perceived fun/enjoyment related. this will definitely provide a better insight of the students’ use of the internet. references adams, d.a, nelson, r.r., and todd, p.a. 1992, “perceived usefulness, ease of use and usage of information technology: a replication”, mis quarterly 16(2), pp. 227-247. agarwal, r and prasad, j. 1997, “the role of innovation characteristics and perceived voluntariness in the acceptance of information technology”, decision science, 28(3), pp. 557-582. ajzen, i and fishbein, m. 1980, understanding attitude and predicting social behavior, (prentice-hall, inc englewood, cliff, nj). al-jabri, m. i. and al-khaldi, a. m. 1997, “effects of user characteristics on computer attitudes among undergraduate business students”, journal of end user computing (spring), pp. 16-21. alsehli, f. 2000, “internet use within information science academic program”, ph.d candidate, initial findings, school of information studies, florida state university. anonymous, 1998, institut teknologi brunei, prospectus. bajaj, a., and nidumolu, s.r. 1998, “a feed back model to understand information system usage”, information and management, 33, pp.213-224. breckler, s.j. 1990, “application of covariance structural modeling in psychology: cause for concern?” psychological bulletin (107:2), pp.260-372. brown, s. and mcintyre, d. 1982, cost and reward of innovations: taking account of the teachers’ viewpoint in innovation in the science curriculum, (croom helm, london). brunei darussalam statistical year book., 1999 statistical department, brunei darussalam. campbell, d.t and fiske, d.w. 1959, “convergent and discriminant validation by the multi-trait, multimethod matrix”, psychological bulletin, 56, pp.81-105. chau, p.y.k. 1996, “an empirical assessment of a modified technology acceptance model”, journal of management information systems, 13:2, pp. 185-204. chin, w., and gopal, a. 1995, “adoption intention in gss: relative importance of beliefs”, data base (26:2/3), pp. 42-63. chin, w.w., and todd, p.a. 1995, “on the use, usefulness, and ease of use of structural equation modeling in mis research: a note of caution.” mis quarterly (19:2), pp.237-246. churchill, g. a. j. 1979, “a paradigm for developing better measures of marketing constructs” , journal of marketing research, 5, pp. 64-73. cronbach, l. j. 1951, “coefficient alpha and the internal structure of test,” psychometrika, 16, pp.297334. cuban, l. 1986, teachers and machine: the classroom use of technology since 1920, (teachers’ college press, new york). davis, f.d. 1989, “perceived usefulness, perceived ease of use and user acceptance of information technology”, mis quarterly 13:3, pp. 319-340. davis, f. d., bagozzi, r.p., and warshaw, p.r. 1989, “user acceptance of computer technology: a comparison of two theoretical models”, management science, 35:8, pp.982-1003. doty, r. 1995, “teacher’s aid”, internet world, 6:3, pp. 75-77. doyle, w., and ponder, g.a. 1977, “the practicality ethic in teacher decision-making”, interchange, 8:3, pp.1-12. falk, r.f., and miller, n.b. 1992, a premier for soft modeling , (university of akron press, akron, ohio). ghani, j. 1992, “task uncertainty and the use of computer technology”, information and management, 22, pp.69-76. gaski, j.f., and nevin, j.r. 1985, “the differential effects of exercised and unexercised power sources in marketing channel”, journal of marketing research, 22, pp. 130-142. ajis vol. 10 no. 2 may 2003 103 gefen, d., and straub, d.w. 1997, “gender difference in the perception and use of e-mail: an extension of technology acceptance model”, mis quarterly, pp.389-399. goodhue, d.l, and thompson, r.l. 1995, “task-technology fit and individual performance”, mis quarterly, 19, pp.213-236. goodwin, n.c. 1987, “functionality and usability”, communication of the acm, 30:3, pp.229-223. gould, s.b. 1995, “an intellectual utility for science & technology: the national research & education network”, government information quarterly 7:4, pp.415-425. harris, r. 1999, “attitudes towards end-user computing: a structural equation model”, behavior and information technology, 18:2, pp.109-125. hayduck, l.a. 1987, structural equation modeling with lisrel, (the john hopkins university press, baltimore, maryland). heaveside, s., farris, e., malitz, g.,& carpenter, j. 1995, “advanced telecommunication in us public school k-12(e.d. tabs, report no nces 95-731), (government printing office, washington dc., usa). igbaria, m., and chakrabarti, a. 1990, “computer anxiety and attitudes towards microcomputer use,” behavior and information technology, 9:3, pp.229-241. igbaria, m., guimaraes, t., and davis, g.b. 1995, “testing the determinants of micro computer usage via a structural equation model”, journal of management information systems, 11:4, pp.87-114. igbaria, m., and tan, m. 1997, “the consequences of information technology acceptance on subsequent individual performance”, information and management 32, pp. 113-121. igbaria, m., pavri, f., and huff, s. 1989, “ microcomputer applications: an empirical look at usage”, information and management, 16, pp.187-196. igbaria, m. 1992, “an examination of microcomputer usage in taiwan”, information & management, 22, pp.19-28. igbaria, m., and iivari, j. 1995, “the effects of self-efficacy on computer”, omega, 23, pp.587-605. igbaria, m., and toraskar, k. 1994, “impact of end-user computing on the individual: an integrated model”, information technology and people, 6, pp.271-292. joreskog, k.g.,and sorbom, d. 1983, lisrel8: structural equation modeling with the simplis command language, (hillsdale, nj: lawrence erlbaum associates) kaplan,t.e., johnson, r.w., pearce, c.g., and george, g. 1997, “the strategic role of communication technology in small business: where we are and where we should be going.” american business review, jan, vol. 15(1), pp. 86-91. karahanna, e. 1993, “evaluative criteria and user acceptance of end-user information technology: a study of end-user cognitive and affective processes”, unpublished doctoral dissertation, university of minnesota. kelloway, e.k. 1998 using lisrel for structural equation modeling, (london: sage publications) kwon, t.h., and zmud, r.w. 1987 “unifying the fragmented model of information systems implementation” in critical issues in information systems research, r.j. boland jr and r.a. hirschheim (eds), (john wiley & sons: ny). lee, s. and hershberger, s. 1990, “a simple rule for generating equivalent models in covariance structure modeling”, multivariate behavioral research, vol. 25, pp. 313-334. lou, h., lou, w., strong, d. 2000, “perceived critical mass effect on groupware acceptance”, european journal of information systems, vol. 9, pp. 91-103. lu, h.p., gustafson, d.h. 1994, “an empirical study of pu and peu on computerized support system use overtime”, international journal of information management, vol. 16(6), pp. 317-329. mathieson, k. 1991, “predicting user intentions: comparing the technology acceptance model with the theory of planned behavior”, information systems research, 2:3, pp.173-191. markus, m.l. 1994, “electronic mail as a medium of management choice”, organization science, 5:4, pp.502-527. palvi, s., tung, l.l., leng, p.t., chin, e. s., pei, l. w. 1995, “a study of internet awareness & usage of two singapore university” in the proceedings of pan pacific conference on information systems, singapore, june 29th to july 2nd, pp.118-120. pijpers, g. g. m., bemelmans, t. m.a., heemstra, f.j., montfort, k.a.g.m., 2001, “senior executives’ use of information technology”, information and software technology, 43, pp. 959-971. prescott, m., and conger, s. 1995, “diffusion of innovation theory: borrowing, extensions and modifications from it researchers”, data base for advances in information system 26:2&3, pp.20-41. rahim, m. md., rahman, m. n., and seyal, a.h. 2000, “software piracy among academics: an empirical study in brunei darussalam”, information management and computer security, 8:1. pp.13-26. ajis vol. 10 no. 2 may 2003 104 raymond, l. bergeron, f. 1992, “personal dss success in small enterprise”, information and management, 22, pp.301-328. rice, r.e., and aydin, c. 1991, “attitudes towards new organizational technology: network proximity as a mechanism for social information processing”, administrative science quarterly, 36, pp.219-249. roger, e.m. (4th ed), 1995, diffusion of innovation, (the freeman press, new york). saga, v.l., and zmud, r.w. 1993, “the nature and determinants of it acceptance, routinization, and infusion.” in, diffusion, transfer, and implementation of information technology: l, levine, ed,), software engineering institute, carnegie mellon university, pittsburgh, pa, pp. 67-86. seyal, a.h., rahim, m. md., and rahman, m.n. 2000, “computer attitudes of non-computing academics: a study of technical colleges in brunei darussalam”, information and management , 37, pp. 169180. seyal, a.h., rahman, m.n., yussof, h. 2001, “academic use of internet”, journal for vocational and technical education & training, vol.2. pp.35-42 seyal, a.h., rahim, m. md., and rahman, m.n. 2002, “ determinants of academic use of the internet: a structural equation model”, behavior and information technology, 21:1, pp. 71-86. soh, c., mah, q.y., gan, f.j., chew, d., reid, e. 1997, “ the use of the internet for business: the experience of early adopters in singapore”, internet research, 7(3), pp. 207-228. songan, p., and noor, f.m. 1999, “predictors of internet utilization among students in an institution of higher learning in malaysia”, in the proceedings of conference on information technology in asia (cita 99), kuching 16-17 th sept., pp. 38-49. spennemann, d.h. r. 1991, “do techno-skeptics use on-campus e-mail?” campus wide information systems, 14:2, pp.46-53. sproull, r.a. 1991, “lesson in electronic mail”, in sproull, l., and kiesler, s (eds.), connections. new ways of working in the net work, (mit press, cambridge, ma). stelzl, i. 1986,, “changing a causal hypothesis without changing the fit: some rules for generating equivalent path models, multivariate behavioral research 21, pp.309-331. stix, g. 1994, “the speed of write”, scientific american, (dec), pp.72-77. straub, d.w., limayem, m., karahanna-evaristo, e., 1995, “measuring information system usage: implications for theory testing”, management science, 41:8, pp.1328-1342. straub, d.w., keil, m., and brenner, w. 1997, “testing the technology acceptance model across cultures: a three country study”, information & management, 33. pp. 1-11. szanja, b. 1994, “software evaluation and choice: predictive validation of the technology acceptance instrument”, mis quarterly, 18(3). szanja, b., 1996, “empirical evaluation of the revised technology acceptance model.” management science, 42(1), pp.85-92. thompson, r.l, higgins, c.a., howell, j. m. 1991, “personal computing: toward a conceptual model of utilization”, mis quarterly, 15:1, pp. 125-143. triandis, h.c. 1971, attitude and attitude change, (ny: john wiley) turner, r., zheng, w. 1998, “staff use of the internet”, in the proceedings of 15th pan pacific conference in seoul. veiga, j.f., and floyd, s. 2001, “towards modeling the effects of national culture on it implementation and acceptance”, journal of information technology, vol. 16, pp. 145-158. yaverbaum, g.s. 1988, “critical factors in the user environment: an experimental study of users, organization and tasks”, mis quarterly, 12:1, pp.75-85. yang, h-d., choi, i-y. 2001 “revisiting technology acceptance model with social influence factors”, in the electronic proceedings of pacis 2001, seoul, korea., pp. 509-523. zikmund, w.g. 1984, business research methods, (usa: dryden press). microsoft word ajis112final.doc australasian journal of information systems vol 11 no 2 may 2004 3 this is the second issue of volume 11 of the australasian journal of information systems. recently, through the assistance of edith cowan university we conducted a ‘recruitment drive’ for new consulting editors. these new editors have been listed (i hope i didn’t miss anyone who volunteered). i would like to thank the respondent editors for their kind offers to work with the journal. our first paper is entitled ‘success factors and measures for public sector is/it co-sourcing contracts’ and is presented by erhan edguer and graham pervan from curtin university of technology. this paper explores the effectiveness of contract negotiations between buyers and suppliers in small government organizations, which collectively outsource their is/it activities to a single outsourcing vendor, usually referred to as ‘co-sourcing’. our next paper is presented by peter marshall from mt eliza business school and judy mckay fro monash university and is entitled ‘strategic it planning, evaluation and benefits management: the basis for effective it governance’. this paper reports on the results of an empirical study into the integration of strategic information systems planning and business-it alignment, it evaluation, and the proactive management of business benefits in large organisations, and to consider the linkages evident between these processes. alan peslak, from penn state university presents our next paper entitled ‘a firm level study of information technology productivity in europe using financial and market based measures’. this paper investigates information technology productivity with a new data set from a european published source, and measuring productivity using both market and financial based measures. our next paper is presented by aileen cater-steel and shelly grist from usq. the paper is entitled ‘steps to internet adoption success: a study of four small regional organisations’. this paper describes the steps taken by four small regional organisations to establish web sites and enter into ecommerce. this paper suggests that when adopting internet technologies, small organisations needs to focus on sound business objectives, plan for an evolutionary approach, and recognise the value of adapting a traditional systems development life cycle approach to web site development. jo rhodes presents our next paper entitled ‘south african rural ict implementation: a critical retrospective application of latour’s due process model’. this paper examines a specific technology implementation – a south african government sponsored telecentreusing latour’s due process model, an analytical tool grounded in actor-network theory, where technology implementation is viewed as a symmetrical treatment of technology and society within a single collective. limbie kelegai and michael middleton from queensland university of technology present the next paper entitled ‘factors influencing information systems success in papua new guinea organisations: a case analysis. this paper examines factors that have affected is implementation in two png organisations. our next paper is entitled ‘community sector organisations: a digital divide in communication and knowledge creation’ and is presented by tom denison fro monash university. this paper provides an in-depth analysis of the data obtained in that survey, setting the results in the context of research undertaken by zook (2001), castells (2001) and adamic (2001), who consider that not only is the internet a natural fit with “winner-take-all” markets but reinforces existing patterns of geographic centralisation in the information society and economy. our final paper is entitled ‘adoption of online purchasing in communities, and its socio-economic implications in regional central queensland’ and is presented by w. taylor, g zhu, j dekkers and s marshall from central queensland university. this paper analyses the general trends of online purchasing in central queensland (cq) communities during 1999-2002 and identifies the socioeconomic factors affecting online purchasing activities. microsoft word vol 15 2 final.doc australasian journal of information systems volume 15 number 2, 2009 91 application of multi-agent technology to information systems: an agent-based design architecture for decision support systems hao lan zhang school of computer and electrical engineering, rmit university, melbourne, australia gitesh k. raikundalia school of engineering and science victoria university yanchun zhang school of engineering and science victoria university xinghuo yu school of computer and electrical engineering, rmit university, melbourne, australia abstract one of the most difficult issues in building efficient information systems (is) is the integration of these systems with the organization’s other systems. this issue is particularly acute for decision support systems (dsss). to become more effective and efficient, a dss must have an open structure to adapt to the dynamic environment. however, current is, especially dsss, tend to rely excessively on traditional system development life cycle (sdlc) and this places limitations on current systems’ infrastructures. the emergence of multi-agent technology addresses this issue and its applications to is are becoming highly efficient. in this paper, we introduce a matrix-agent connection design, called agent based open connectivity for decision support systems (aocd), which balances the manageability and flexibility in a system and maximizes system performance. introduction during the past decade, the successful applications of it-based systems in business processes have declined because of obstacles in the development of effective is for business processes. a survey provided by the standish group (the standish group, 2001) shows that in u.s.a, 23% of is failed outright and 49% systems were late and or went over budget. apart from the u.s., similar cases australasian journal of information systems volume 15 number 2, 2009 92 were also found in the u.k and other countries. the integration difficulty is one of the most crucial reasons that causes the failure of is application (turban, et al, 2005). problems in existing dss among the various obstacles, steven alter (alter, 1999) summarizes five major obstacles when applying information technology in the real world, which include: (1) unrealistic expectations and techno-hype. (2) difficulty building and modifying it-based systems. (3) difficulty integrating it-based systems. (4) organisational inertia and problems of change. (5) genuine difficulty anticipating what will happen. most of these obstacles are caused by the obturation of current information system structures. figure 1 shows a schematic view of a traditional decision support system. from figure 1, we find the system structure of traditional dss is closed and that causes many problems, such as lack of compatibility, lack of adaptability, lack of expandability, and cost ineffectiveness in sdlc. to transform an it-based is into an open system, multi-agent technology can be deployed. multi-agent technology is a superior solution that can enhance a system’s connectivity, extensibility, reusability, and manageability. figure 1: schematic view of traditional dss. (source from turban, et al, 2005) intelligent agents are goal-oriented, collaborative, flexible, self-starting, intelligent, mobile and interactive (turban, et al, 2005). these characteristics of agents ensure the applications of multiaustralasian journal of information systems volume 15 number 2, 2009 93 agents in dss to be successful. rapid development of intelligent agent technology changes sdlc. traditional information systems involve seven development phases, which include: requirement phase, specification phase, design phase, implementation phase, integration phase, maintenance phase, and retirement. although an object-oriented sdlc emphasises individual object design, it still based on these seven phases with conceptual changes. multi-agent-based systems completely shift current system development methodologies. such systems offer “plug-and-play”, which allows external components (agents) to be integrated into an existing system without disturbing the system structure and working process. we believe multi-agent-based systems will eventually replace current information system model, including object-oriented system model. objects or components in current information systems will be replaced by intelligent agents, which are more powerful, more intelligent, more flexible, and proactive. based on previous studies (turban, et al, 2005; wagner, 2003; zhang et al, 2005), we draw a comparison between traditional is and aocd-based is (agent-based) as shown in table 1. table 1: comparison of traditional and aocd-based information systems multi-agents applications in dss back in 1975, hewitt et al. (1975) described an actor concept, which has many similarities to today’s multi-agent-based dss. current applications of multi-agent technology in dss have been mainly focusing on decentralized frameworks. for instance, ossowski et al. (2004) propose and implement the multi-agent decision support system in transportation management; vahidov (2006) suggests the use of intelligent agents in situated systems, and padgham and winikoff (2002) introduce the prometheus methodology for building belief-desire-intention (bdi) based distributed agent systems. however, these systems mainly rely on the efficient concurrent control and synchronous communication technologies. unfortunately, current technologies in these areas are not efficient enough, particularly, when they deal with large volumes of transactions. many decentralized multi-agent systems suffer from inefficient manageability (minar, 2002), particularly, with respect to the difficulties of handling concurrent control and synchronous communication problems. characteristics traditional information systems aocd based information systems compatibility poor good consistency good modest cost-effectiveness modest good expandability poor good flexibility poor good stability good uncertain reusability poor good australasian journal of information systems volume 15 number 2, 2009 94 here, we propose a novel multi-agent-based architecture for dss, called agent-based open connectivity for dss (aocd). aocd architecture adopts a matrix-agent connection design (see figure 2), which overcomes the limitations of decentralized multi-agent systems by using a hybrid framework. the matrix-agent connection design involves three major components: agents, a matrix center, and databases. the matrix-agent connection design adopts a two-phase communication mechanism, which involves communication between two agents and communication between the matrix and the agents. there are two stages in aocd framework design. the first stage involves the matrix-agent connection design, which forms the backbone of aocd architecture. the second stage involves the individual agent design, which focuses on solving specific problems. generally, once the matrix-agent connection design is made, it will not be changeable unless the overall system is replaced. on the contrary, the agents are dynamic components, which may be updated constantly. each agent has the ability to solve problems in its domain. an agent is able to seek cooperation from other agents when it has difficulties in solving a problem, and this makes the aocd architecture more powerful than other non-cooperative dss architectures. the aocd architecture is more powerful in that it mirrors the way human groups make decisions: by using a set of interacting agents, more creative and innovative decisions can be produced as a result of such cooperation and interaction. the aim of aocd architecture design is to provide an open, efficient, and flexible architecture for dss. it has been focusing on the architectural design of aocd system, which provides a foundation for future aocd research. the previous research involves: (1) introducing the concept of matrix-agent connection, which has been proven efficient and effective compared with other decentralized or centralized frameworks (wagner, 2003), (2) experimental design of agent network topologies in aocd-based systems (zhang, et al, 2005; 2006a). this paper is structured as follows; in the next section, we provide an example scenario to clarify the implementation of aocd. in the matrix-agent connection design section, we briefly introduce the design of the matrix-agent connection, which mainly focuses on (i) the matrix interface design, (ii) the matrix register design, and (iii) the agent architectural design. the final section concludes our research work and considers future issues that need to be addressed. a case study of aocd application in is most organisational managers often find it is very difficult to decide whether their organisational information systems need an upgrade or not (alter, 1999). there are many factors that plague decision makers about whether or not to upgrade their systems, which, of course, are traditional sdlc based systems. the first factor is the enormous time consumption for the development of a traditional information system. for traditional sdlc based systems, the upgrade period not only includes the time for designing specific components but also includes the time for re-evaluating user requirements, modifying existing system structure (possible re-design when upgrade components affect the overall system processes), integrating a new system to maintain system consistency and extra outsourcing manpower is often used for communication in every upgrade phase. meanwhile, the training processes for staff to understand new systems also take time. australasian journal of information systems volume 15 number 2, 2009 95 the second factor is the high possibility of upgrade failure. as we mentioned in the introduction section, the success rate of developing it-based systems for business process is disappointing. managers often face the risk of system failure when developing or upgrading a new system. figure 2: schematic view of aocd-based dss. (source from zhang, et al, 2005) thirdly, the dynamic environment might cause their upgrade useless. people are living in a socalled information explosion age, which makes our day-to-day environment very difficult to predict. new technologies, new business opportunities, and new inventions are emerging everyday. in this circumstance, any system upgrade cannot last for long. therefore, the constant upgrade processes confuse organisational mangers. certainly, there are many other reasons for the problems of upgrading or re-designing, such as the technology gap between existing and new system, system structural inefficiency that results in some unknown information loss between the new and existing system, human errors that might occur when constantly shifting their familiar interfaces to some unfamiliar interfaces, etc. the use of multi-agent technology is able to solve the above problems. aocd is a multi-agentbased architecture, which is open and flexible. to illustrate the effectiveness and efficiency of the architecture we use the following case as an example. nowadays, mergers of international corporations happen frequently as the globalisation phenomenon has extended to today’s economic section. hidden behind the encouraging benefits of merger, most merging corporations have encountered difficulties in integrating various information systems from different mergees. in many traditional information system development frameworks, the process of combining two different information systems, especially dsss, from two different organizations into one system is time-consuming, costly and inefficient. the traditional australasian journal of information systems volume 15 number 2, 2009 96 development process may involve re-analysing the user requirements, reconstructing the database, redeveloping the code, etc. for instance, during the merger process of the german automaker daimler benz with the u.s. automaker chrysler (alter, 1999; zhang, et al, 2008), difficulties were found when integrating daimler’s cad systems with chrysler’s cad systems because the two companies’ cad systems were designed based their own company’s business and manufacturing processes and their cad systems were incompatible with each other. many similar cases can be found when merging two corporations. using the aocd architecture can easily solve the integration obstacles that chrysler and daimler faced. to simplify the case, we use two fictitious companies, company a and company b, to explain the merging process in aocd-based systems. company a and company b are the manufacturer of vehicles. both companies have developed two different information systems based on aocd architecture. in company a, there are five major intelligent agents that include: sales agent, human resource agent, accounting agent, marketing agent, and manufactory agent (including cad systems). each agent is responsible for the information process in its business domain. the human resource agent manages current staff information and offers advice to the manager if it is necessary to reduce labour of the company. the sales agent provides information such as current sales performance, sales geographic distribution, etc., to decision makers. the marketing agent provides the information about the company’s current marketing status, rivals’ marketing status, and the market trends, and so on. the accounting agent mainly deals with staff salaries, reimbursement, supply, income payments, etc. the manufactory agent is responsible for controlling auto-manufacturing systems, cad systems, warehouse management, etc. company b’s information system is different from company a’s as its business strength is automotor design. there are four major intelligent agents in company b’s system including: human resource agent, financial agent (including accounting, sales and marketing functions), manufactory agent (including cad systems), and design agent. the human resource agent and manufactory agent of company b are similar to company a’s. the financial agent deals with all the financial issues, including current sales performance, sales geographic distribution, the company’s current marketing status, rivals’ marketing status, market trends, staff salary, reimbursement, supply, income payments, and so on. the design agent provides the latest auto-motor design information, existing auto-motor design pattern database, new auto-motor design software information, and so on. as the strategic alignment would benefit both companies, the executive boards of the two companies decide to merge two companies. there would be many problems to merge the companies’ information systems if they were using traditional information architectures because the headquarters of the two companies are located at two different cities and their cad systems embedded in manufactory agent are incompatible. now, most of the difficulties can be solved within aocd framework. figure 3 shows the two companies’ information systems before the merger. there are some similarities in the two companies’ information systems, the two human resource agents are very similar and company b’s financial agent can deal with all the financial issues in company a’s framework. australasian journal of information systems volume 15 number 2, 2009 97 figure 3: two companies’ aocd-based information systems (before merger) matrix in aocd is standardized; therefore, the integration process can use one matrix. because the human resource agents (hr agents) in two companies’ information systems are very similar, the integration process can use one hr agent instead of using both of them. company b’s financial agent can replace the sales agent, marketing agent and accounting agent of company a as the financial agent is able to deal with all the financial issues in company a. the integration process is simple and easy because aocd design supports plug-and-play. the integrated information system after merger (see figure 4) can support both companies’ business processes. the new company’s information system retains two manufactory agents because the former two companies’ cad systems were incompatible. the new information system also retains two knowledge bases, which are located at the two headquarters, respectively. matrix can be located at one of the headquarters and the agents are connected to the matrix through local area network, wide area network and the internet. as we can see from this case, the difficulties in the integration process, which plague traditional information systems, are solved in aocd architecture. in aocd architecture, there are no complicated re-design procedures involved in integration or upgrade process and the risk of upgrading failure is minimized or even completely eliminated. the advantages of using aocd architecture in information systems, especially dss, can be summarised as follows (bill, 1998): (i) australasian journal of information systems volume 15 number 2, 2009 98 speed-up and efficiency, (ii) robustness and reliability, (iii) enhancing scalability and flexibility, (iv) cost effectiveness and (v) reusability. figure 4: new company’s information system (after merger) matrix-agent connection design in aocd the matrix concept is one of the key ideas in the aocd architecture. the basic framework of aocd architecture is formed by matrix-agent connections, which are done through the connection interfaces provided by matrix. a connection interface will be allocated to an agent when the matrix receives a request of connection from the agent. an agent can connect to the matrix at any time without disturbing the current system. matrix offers a living platform for agents. the use of the matrix component distinguishes aocd architecture from other multi-agent based systems (weiss, 1999). the matrix consists of three major parts include (i) agent society that provides a virtual space to agents, (ii) information acquisition section that acquires knowledge from external environment, and (iii) matrix control panel is the core part that mainly handles agent matching, requests processing, and resource allocation. the matrix consists of four layers: interface layer, agent link layer, transmission layer, and data layer. the interface layer provides interfaces for establishing connection between agents and the matrix. the agent link layer provides two operation mechanisms. the first mechanism is a manual linking process, which enables a user to link different agents, in order to solve a problem through agent-cooperation. the other mechanism is an auto-match process, which allows the matrix to match the corresponding agents by using the information provided by the matrix register. the transmission layer provides users with agent language communication functions and message receiving and sending functions. the data communication layer enables matrix to establish a connection with the local database in the aocd framework. australasian journal of information systems volume 15 number 2, 2009 99 figure 5: operation of aocd system (source from zhang, et al 2006c) figure 5 shows the operation process of the aocd architecture. an aocd system starts with loading the matrix control panel, which contains the information of previously allocated connection interfaces, such as the number of connection interfaces allocated in the last system operation. when the connection interfaces have been established, the matrix control panel begins to receive requests from the agents. there are two types of requests: one is the request for connection and another is the request for corresponding agents. once a request for connection is received, the matrix control panel will change the matrix register and allocate a new connection interface for the new agent. in the meantime, a dynamic-detection process is frequently performed to ensure the system is receiving requests. if a request for a corresponding agent is received, the matrix will forward the request to the matrix register to search for a corresponding agent or agents. if a corresponding agent is found, the result will be delivered to the transmitter to encode/decode the request into a set of recognizable transmission protocol and operation commands. if the request agent requires information from the database, then the matrix needs to perform the database operation according to australasian journal of information systems volume 15 number 2, 2009 100 the decoded commands. the database sends the feedback to the transmitter. the final step is sending results to the corresponding agent via the connection interface. matrix interface and matrix register design matrix interfaces are located at the matrix interface layer. when a matrix receives the connection requests from the agents, the matrix allocates a number of interfaces (according to requests) and establishes connections between the matrix and the agents. the number of interface(s) provided by a matrix to connect with agents is unlimited. a dynamic-detection method is deployed to identify the number of interfaces generated for an agent. in dynamic-detection method, an interface tag is deployed to indicate whether a connection request has been received by the matrix. once a matrix detects the indication of a new connection, a new interface will be allocated to the connection. the matrix examines the interface tag and allocates interfaces to agents according to the value of the interface tag when the aocd system starts to load the matrix. a new connection request changes the value of the interface tag that triggers the re-allocation of the connection interface resources. the matrix register plays a primary role in the agent-link layer. the major functionalities of matrix register include: (i) searching for a corresponding agent based on the descriptions, and (ii) providing the corresponding agent address to transmitter. the matrix register makes use of a three-column list, which contains the agents’ details, functionality descriptions, and data structure information. the agent information column contains the information of an agent name and agent ip address. the functionality column contains the information of agent’s functionalities, which is used to provide agent information when the matrix searches for a corresponding agent. the detail description column consists of four major items: organization/department, data table name, data attribute, and description control (if applicable). a unified matrices architecture the matrix-agent framework combines the centralised topology and the peer-to-peer communication (see figure 2). the bottleneck problem rises when the framework expands to a large scale because of the limitation of the centralised topology. to tackle the bottleneck problem, a unified matrices framework has been introduced (zhang, et al, 2006b). in the unified matrices framework, a number of matrices are connected through the matrix interface layer. a matrix is allocated a number of agents according to the agents’ locations and domains. once the agents in a matrix cannot deal with a request, the matrix will forward the request to its nearest or most familiar matrix for further processing. this unified matrices framework minimizes the bottleneck problem in a large-scale agent network, such as aocd framework. the process of searching a service-providing matrix involves two major methods, including the most familiar partner method and supplemental partner method. we use a matrix capability description table to describe the functionalities and capabilities of a matrix. this table contains the information of all the connected agents’ functionalities and domain information, indicating the major area for which an agent is responsible. each agent’s functionality is described by several keywords and these keywords also represent the functionalities and capabilities of the matrices. generally, there are some keyword intersections among the matrices (figure 6). the number of keyword intersections of a requesting matrix with the other service-providing matrices are calculated to generate intersection scores. if a service-providing matrix has the greatest intersection score with a requesting matrix (compared to other matrices), then this matrix is regarded as the most familiar partner of the requesting matrix. whereas if a service-providing matrix has the smallest intersection score with a requesting matrix, then this matrix is regarded as the australasian journal of information systems volume 15 number 2, 2009 101 supplemental partner of the requesting matrix. both methods follow a four-step calculation procedure. the intersection score is calculated from the following equation. (1) where raj is the keyword intersection score between matrix a and matrix j. sa denotes a keyword set of matrix j that intersects with sa and sj. mi denotes the total number of the matrices (including the matrix a) that have the same intersections with matrix a. the detailed calculation procedure can be found in (zhang, et al, 2006b). figure 6. intersections of matrices’ capabilities (source from zhang, et al, 2006d) agent architectural design in aocd the agent architectural design in aocd architecture adopts bdi model (zhang, et al, 2006d), which was first described in irma architecture by bratman, israel, and pollack (zhang, et al, 2006c). the bdi model has been used widely in many industrial systems and it has been found efficient in some highly successful agent architectures to date (bratman, et al, 1988). bdi is not the only option for aocd agent design; layered architecture for agent design also has proven useful. layered architecture design decomposes an agent’s behaviours and creates separate subsystems to deal with these different types of behaviour. touring machines architecture (wooldridge, 1999) is a typical layered architecture design. the main problem with the layered architecture is it lacks the conceptual and semantic clarity of unlayered approaches. this disadvantage makes the transformation from logic-based semantics to a layered architecture a very difficult process, as ∑ = += sa i iajaj mrr # 1 )/1( australasian journal of information systems volume 15 number 2, 2009 102 many logic-based semantics cannot be separated completely. this weakness discourages the use of a layered architecture in aocd agent design because the aocd architecture is a dss-based architecture that involves many logic-based semantics. we deploy a knowledge base to represent the information that the agent has about its current environment. figure 7 shows the aocd agent architectural design. figure 7: aocd agent architecture design (source from zhang, et al, 2006c) an agent establishes a connection with the matrix or other agents through the matrix-agent connection interface. once a connection is established, the agent is able to send messages to or receive messages from the environment by using a message sender/receiver component. the message sender/receiver decomposes the message into several categories and forwards the content category to a message interpreter in the format of aocd acl. the message interpreter decomposes the received information and sends a set of commands to a functional processor. the functional processor provides the abilities of solving domain problems. a knowledge base provides the functional processor with a set of rules (beliefs) in the domain. the functional processor generates the results and delivers them to an option generator. the option generator maps a set of rules in the knowledge base (beliefs) and a set of intentions to a set of desires, which can be expressed logically as follows (bratman, 1988). we adopt the following notations, if a is an arbitrary set, then )( a℘ is the powerset of a, bel represents the rules in knowledge base, int australasian journal of information systems volume 15 number 2, 2009 103 represents the intention sets, and des represents the desires sets. the options can be represented as a mapping. options: )()()( desintbel ℘→℘×℘ the filter function performs a deliberation process, which can be expressed as follow. filter: )()()()( intintdesbel ℘→℘×℘×℘ after the filtering process, an agent is able to generate one or more intentions, which will be sent back to the matrix interpreter. matrix interpreter has the responsibility to determine whether to forward results to the matrix-agent (or agent-agent) connection or take actions immediately. the matrix interpreter is designed with this ability because some messages from the environment do not require feedback, e.g., a manager sends a message to the sales agent demanding a printout of today’s timetable. this message does not contain any complicated decision making process and is one-way. in this case, it is not necessary to send any feedback to the manager. thus the matrix interpreter would simply take the action, which is producing a printout of timetable, instead of performing any further activities. results of the aocd topological experiments aocd framework is a hybrid multi-agent network. we have conducted a set of experiments to evaluate the topological performance of aocd-based (hybrid) system (zhang et al, 2006a). figure 8 and figure 9 show two snapshots of the aocd topological experiments. figure 8: agent matching process of the experiment (source from zhang, et al, 2006d) australasian journal of information systems volume 15 number 2, 2009 104 figure 9: configuration interface of the experiment our study shows the hybrid topology presents a superior performance compared to the other two topologies, especially the centralised topology. in the centralised topological experiments, the total time consumption for processing 2917 requests is 28477 seconds. the average time consumption for one request in the centralised system is 9.76 seconds, whereas the average time is only 3.41 seconds in the hybrid topology (figure 10). in addition, the average success rates of the two topologies are 86.74% (centralised) and 95.45% (hybrid), respectively (figure 11). the experimental design and more detailed experimental analysis can be found in our previous work (zhang, et al , 2006a). figure 10: average processing time comparison figure 11: success rate comparison we noticed that for both centralised and hybrid topologies when the produce of the requestvolume/participated-agents increases, the success rate of processing the requests appears to decrease (figure 12 and 13). australasian journal of information systems volume 15 number 2, 2009 105 figure 12: decreasing trends of success rate in centralised topology (source zhang, et al, 2006d) figure 13: decreasing trends of success rate in hybrid topology (source zhang, et al, 2006d) these trends suggest that to increase the success rate of processing the requests in aocd-based systems, we need to balance the request volume and the number of participating agents. in other words, in order to process a number of requests, the number of participating agents should not be too small. the more agents that are involved in dealing with the requests, the probability of increasing the success rate is higher. conclusions and future work the significance of the aocd design is that it: (1) provides a flexible plug-and-play architecture, which eliminates the difficulties of integrating different information systems or upgrading information systems, australasian journal of information systems volume 15 number 2, 2009 106 (2) combines centralized communication and decentralized communication in the same framework, which allows the agents to communicate in a more effective way. by imitating the way human communities make decisions, more creative and innovative decision outcomes can be produced as a result. a concrete performance analysis also has been performed (zhang, et al, 2008), which indicates that such architecture is able to deliver good performance. future work on aocd will mainly focus on three aspects: the use of agent design patterns (ferguson, 1992; aridor, 1998) in aocd is considered to be helpful because of their flexibility, reusability, and understandability for agent application design. future work can conduct a survey on current methodologies in the agent design pattern area. developing an efficient agent-matching algorithm. agent matching in aocd is a crucial issue, which could enhance the efficiency of agent cooperation. analysing the possible negative impacts of using multi-agent technology in information systems. references alter, s. (1999) information systems – a management perspective (3rd edition), addison-wesley educational publisher inc., pp. 23-27. aridor, y., & lange, d. b. (1998) ‘agent design patterns: elements of agent application design’, proceedings of the autonomous agents 1998 conference, minneapolis, usa. bill, v. (1998) “the first global car colossus”, business week, pp. 40-43. bratman, m.e., israel, d.j., & pollack, m.e. (1988) ‘plan and resource-bounded practical reasoning’, computational intelligence, pp. 349-355. ferguson, i., a. (1992) ‘touring machines: an architecture for dynamic, rational, mobile agents’, phd thesis, clare hall, university of cambridge, uk. hewitt, c., bishop, p., & steiger, t. (1975) ‘a universal mosular actor formalism for artificial intelligence’, proceedings of the fourth international joint conference on artificial intelligence, pp. 235-245. minar, n. (2002) ‘distributed system topologies’. http://www.openp2p.com/lpt/a/1461 accessed 20 july 2005. ossowski, s., fernandez, a., serrano, j.m., perez-de-la-cruz, j.l., belmonte, m.v., hernandez, j.z., garcia-serrano, a.m., & maseda, j.m. (2004) ‘designing multiagent decision support system-the case of transportation management’, proceedings of 3rd aams, vol. 3, new york, u.s.a. padgham, l., & winikoff, m. (2002) ‘prometheus: a methodology for developing intelligent agents’, proceedings of aamas 2002. bologna, italy. the standish group international, inc. (2001) ‘extreme chaos’, publication of chaos report 2001 standish group international inc. turban, e., aronson, j. e., & liang, t. (2005) ‘decision support systems and intelligent systems’ (7th edition), prentice hall publication, pp. 223 and pp. 109, 707. australasian journal of information systems volume 15 number 2, 2009 107 vahidov, r. (2006) ‘design researcher’s is artifact: a representational framework’, proc. of first international conference on design science in information systems and technology, claremont, u.s.a. wagner, g. (2003) ‘the agent-object-relationship metamodel: towards a unified view of state and behavior’, information systems, 28:5. weiss, g. (ed.) (1999) ‘multiagent systems – a modern approach to distributed artificial intelligence’, the mit press, pp. 8-9. wooldridge, m. (1999) ‘intelligent agent’, in weiss, g. (ed.) multiagent systems-a modern approach to distributed artificial intelligence, the mit press, pp. 27-77. zhang, h. l., leung, c.h.c., and raikundalia, g. k. (2005), “aocd: a multi-agent based open architecture for decision support systems”, proceedings of international conference on computational intelligence for modelling, control & automation, vienna, austria, 2, pp. 295-300. zhang, h. l., leung, c.h.c., and raikundalia, g. k. (2006a), “performance analysis of network topology in agent-based open connectivity architecture for dss”, proceedings of 20th international conference on advanced information networking and applications, vienna, austria, vol. 2, pp. 257-261. zhang, h. l., leung, c.h.c., and raikundalia, g. k. (2006b), ‘towards the matrix concept: a virtual platform for intelligent agent application in decision support systems’, proceedings of the 7th international symposium on knowledge and systems science (kss2006), beijing, china, pp. 115 – 122, lecture notes in decision sciences 8, globallink publisher. zhang, h. l., leung, c.h.c., and raikundalia, g. k. (2006c), ‘a novel matrix-agent connection design in multi-agent based architecture for decision support systems’, in technical reports of the ifip tc8/wg 8.3 open conference, the london school of economics, london, uk, 28 june – 1 july 2006. zhang, h. l., leung, c.h.c., and raikundalia, g. k. (2006d), ‘matrix-agent framework: a virtual platform for multi-agents’, journal of system sciences and systems engineering, vol. 15, no. 4, pp. 436 – 456, springer-verlag press. zhang, h. l., leung, c.h.c., and raikundalia, g. k. (2008), ‘topological analysis of aocd-based agent networks and experimental results’, journal of computer and system sciences, vol. 74, no. 2, pp. 255 – 278, elsevier publication acknowledgements the experimental design of this research was supported by victoria university. thanks also go to dr frederic adam of university college cork for his comments on the previous version of the matrix design. microsoft word ajis12-1final repaired.doc ajis vol. 12, no. 1 september 2004 28 e-business developmental issues in the australian food industry mohini singh school of business information technology rmit university 239 bourke street melbourne victoria 3000, australia mohini.singh@rmit.edu.au abstract this paper discusses e-business developmental issues in the australian food industry that were identified from a research project funded by the australian research council. data was gathered from interviews with 11 food organisations in the year 2000, most of which can be classified as ‘bricks and clicks’. these 11 companies were traditional businesses that had adopted e-business as a new way of doing business. the findings of the paper highlight the fact that the b2b is the predominant e-business model in the australian food industry, edi is an important foundation technology platform for developing b2b e-business, e-procurement was an important reason for adopting b2b e-business and improved supply chain management was the most important achievement from e-business for these organizations. it also highlighted the fact e-business developed in-house with an it department managed e-business sites better than those that outsourced the development. key words e-business, e-commerce, edi systems, b2b e-business, b2c e-business, eprocurement introduction motivated by the potential for business to business and business to consumer commerce, australian industries have largely resorted to e-business as a new channel of business since 1998. research findings of the anderson consulting group (1999) indicated that in australia, the internet was used to satisfy communication, network and research needs, and increasingly to sell goods and services to consumers and other businesses. online revenue of australian companies during 2001 and 2002 reported in noie report (2002) contributed to $43 billion, the equivalent of 6.4 percent, of the gross domestic product. the report also indicated that the application of the internet by smes for eprocurement increased from 26 percent to 41 percent and the increase in e-payments increased from 23 percent to 40 percent. the australian food industry akin to other industries adopted e-business to acquire new customers, increase competitiveness and to achieve business efficiencies of automated processes and electronic communication and transmission of data. this paper discusses e-business developmental issues in the australian food industry that were identified from a research project funded by the australian research council. data was gathered from interviews with 11 food organisations in the year 2000, most of which can be classified as ‘bricks and clicks’. these 11 companies were traditional businesses that had adopted e-business as a new business channel. the findings of the research are presented in tables and discussed to highlight the fact that the b2b is the predominant e-business model in the australian food industry, edi was an important foundation technology platform for developing b2b e-business, e-procurement was an important reason for adopting e-business and improved supply chain management was the most important achievement from e-business in these organizations. it also highlighted the fact ebusiness developed in-house with an it department better managed e-business than those that outsourced the development. the paper includes a review of literature on e-business, describes the methodology applied for collecting data, research findings and a discussion of the findings addressing technological, business and b2b e-business applications. e-business and e-commerce are used interchangeably to mean the same thing in this paper. ajis vol. 12, no. 1 september 2004 29 literature review e-commerce generally means doing business on the internet. it is also referred to as internet commerce, digital business and online trade. turban et al. (2000) define it as the process of buying and selling or exchanging of products, services and information via computer networks including the internet. thuraisingham et al. (2002) refer to e-commerce as transactions on the web such as buying and selling products, and activities such as training, procurement, and supply chain management; and to e-business as any business on the web, and broader than e-commerce. increasing profitability and market share, improving customer service, and delivering products faster are some of the organisational performance gains allegedly possible with electronic commerce. activities such as the buying and selling of goods and services as well as transferring funds, utilising digital communications and all aspects of an organisation’s electronic interactions with its stakeholders, such as customers, suppliers, government regulators, financial institutions, managers, employees, and the public at large are included in e-commerce (singh, 2002). singh also explains that all intercompany and intra-company functions including marketing, finance, manufacturing, selling and negotiation that use electronic mail, edi, file transfer, fax, video conferencing, workflow, or interaction with a remote computer are components of e-commerce. the rapid adoption of the internet and the world wide web by businesses as a commercial medium in australia and other parts of the world is changing the face of business. the internet has created electronic marketplaces where buyers and suppliers meet to exchange information about prices and product and service offerings, and to negotiate and carry out business transactions (archer and gebauer, 2000). archer and gebauer are also of the opinion that although electronic marketplaces involve business to consumer (b2c) and business to business (b2b) systems, growth in b2b is estimated to be five times the value of b2c. e-commerce enables businesses to reduce costs, increase demand and create new business models. it has the potential to benefit all consumers through reduced prices and improved products and information flows (dunt and harper, 2002). small and large firms alike can access the internet and exploit near-zero marginal costs of distribution for their products (dunt and harper, 2002). although electronic commerce has been proved to be popular with large business enterprises, small and medium companies also create value by marketing and selling goods and services electronically (dublish, 2000). each company is constrained by the amount of graphics and design capability that the internet can deliver, so everyone starts from the same position with their web sites. with e-business development, technical, management and governance issues, in both large and small companies, have been evolving continuously. johnson and griffith (2002) suggest that for established firms, a well-designed web site helps reinforce the firm's brand equity, while a poorly designed web site can devalue its established brand equity. for new firms, whether dot.coms or local and regional traditional firms going online nationally or globally, a well-designed web site can help start the process of building brand equity, while a poorly designed web site may not engage the consumer, causing them to move to another web site without evaluating the site's content. ho and wu's (1999) research indicated that the most important factors in electronic commerce appreciated by customers are logistical support, technological characteristics, information characteristics, homepage presentation and product characteristics. logistical support requires quick response to customers' needs via e-mail or fax, quickly delivering goods to customers, and providing after sales service. technological factors include modern computer and network facilities and well-structured information systems. information factors encompass reliable output information and secure transaction. homepage presentation should provide for an easy to use interface and detailed information about goods. product characteristics include the details of a variety of goods and services made available at lower prices. relevant legal issues support and validate online contracts, ensure confidentiality and privacy, protect intellectual property, defamation, censorship, taxation, payment systems and the rights of consumers in the online world (stoney and stoney, 2003). the consumer needs assurance of privacy and the protection of consumer rights just as the trader needs jurisdiction to validate and enforce electronic contracts. this has led to the need for standards and regulations for online businesses both regional and global. socio-cultural barriers to e-buying identified by harrison-walker (2002) ajis vol. 12, no. 1 september 2004 30 include language preferences, income and literacy levels, attitudes toward distance buying, security and privacy of information, methods of payment, and the possibility of credit card fraud. australia is well placed on most measures of the adoption of new information and communications technology and the growth of e-commerce, especially b2b (dunt and harper, 2002). to support b2b e-business, australia has also seen the proliferation of intermediaries such as private exchanges to facilitate b2b exchanges. the increase in b2c e-business in australia has been for online services such as online banking; an increase from 2.71 million in 2001 to 5.23 million in 2002 (noie, 2003). e-business was particularly popular for early adopters in the australian food industry with the prospects of launching online grocery for b2c e-business and e-procurement for b2b e-business. in 1999, food organisations in australia that launched e-business were coles myer www.colesmyer.com.au for online grocery, aussie greengrocer www.aussiegreengrocer.com.au, getfresh www.getfresh.com.au, woolworth’s ‘fresh food people’ www.woolworths.com.au, virtual wine www.virtualwine.com.au, pizza now www.pizzanow.com.au, and others. e-business has had an impact on all industries in australia, however, the following discussion includes its application to the food industry addressing issues for the early adopters of e-business. research methodology this research project on e-business was carried out in the year 2000 to identify e-business issues in the australian food industry. the research was initiated as more large food organisations, both suppliers and buyers, supermarkets, grocery outlets and existing traditional retailers, adopted the internet and the world wide web to enhance business. it was funded by the australian research council to understand the developmental patterns in e-business in the food industry by identifying and establishing the objectives and opportunities, challenges, development issues, and improvements achieved by the early adopters of e-business. it was an exploratory study accomplished via interviews. the interview tool used was a semistructured questionnaire. yin (1994) describes exploratory studies to contain a number of 'what' and 'how many' types of inquiries, which were the type of questions included in the interview tool used to collect data. with the permission of the interviewees all interviews were recorded on tape and later transcribed for analysis. the companies investigated were selected on the basis that they had been identified as significant ecommerce initiators within the states of victoria and new south wales, and those that agreed to participate in the project. eleven organizations agreed to participate in this project. initial contact was made by telephone with the person who headed the electronic commerce project at these organisations. data collected were qualitatively analysed. findings of the study are presented in tables 1 to 4 and discussed in the following section of this paper. for reasons of confidentiality, names of companies discussed are not identified. in this paper they are referred to as companies a to k. findings a comparative study of the 11 organisations investigated reveal that some of the findings are common to all while some are quite different. although the reasons for the differences were not identified in this study, it is assumed that knowledge of electronic trade practices, existing technology, size of business, and in-house it support are some of the factors responsible for the variations. organisations investigated were mostly australian owned, well established and a combination of large and small businesses. although all of them were classified as organizations in the food industry, their business activities ranged from wholesalers, retailers, caterers, importers and distributors. all 11 organizations were traditional businesses with an online presence generally classified as ‘bricks and clicks’. electronic initiatives at these organizations were as old as ten years if the medium was an edi system, or 18 months to 2 years if they were based on the internet. data ajis vol. 12, no. 1 september 2004 31 presented in table 1 are findings representing the e-business model or models adopted, technologies that supported e-business, the project leader who managed the development. e-business models, infrastructure and champion the e-business model at these organizations was predominantly b2b (9 out of the 11 organisations), 2 had b2c, and 1 had both b2b and b2c. organizations with a b2b e-business model were suppliers and buyers, and both small and larger organizations. table 1: e-business models and infrastructure co e-bus model technology extra-net intra-net e-bus (fully electronic) e-bus champion a b2c 1800 number, internet & fax no no yes owner b b2c internet no no yes mis manager c b2b edi yes yes no ceo d b2c & b2b edi no yes no regional director e b2b edi no yes no mis manager f b2b internet no yes no accountant g b2b edi no yes yes financial controller h b2b edi yes yes yes ceo i b2b edi no no yes mis manager j b2b & b2c interactive website no no yes owner k b2b internet and extranet yes yes yes managing director technologies on which e-business was developed in these organisations were predominantly edi systems and the internet. b2c e-business models were adopted by those organizations that supplied consumer goods such as meat product, fresh fruits and vegetables to buyers with home deliveries. three of these companies had implemented extranets to support b2b exchanges with business partners and 7 had intranets to support business to employee (b2e) functions. when asked about the stages of e-business applications, 7 had full e-business capacity (i.e. orders, acknowledgement, marketing, payment, dispatch information, product tracking, etc.), whereas 4 had limited applications, generally advertising and online catalogues, indicating a phased development strategy. the two organisations that adopted b2c and b2b models clearly indicate that the two models adopted concurrently support customer reach as well as business exchanges with suppliers. the leader of e-business project in most organizations was someone at the senior management level. three of these were mis managers while others ranged from accountants and financial controllers to ceos and managing directors who were also business owners in a few cases. objectives of e-business development as seen from the data presented in table 2, the main reasons for developing b2c e-business were to have a 24 hour, 7 days a week shop front, for better customer service and for online marketing. b2b e-business development, however, was predominantly for supplier collaboration and to achieve e-procurement efficiencies. these were reduced costs, accurate data, transparent purchasing ajis vol. 12, no. 1 september 2004 32 processes, improved communication with business partners, ability to participate in reverse auctions, select new suppliers and for improved supply chain management. other reasons identified from this research were integrated business processes and backend systems to achieve business efficiencies and to manage larger volumes of data. company g was a supplier to large supermarkets which had changed all its operations to electronic means, compelling all its business partners to adopt ebusiness. enhanced business communication and improved internal communications were other reasons for developing e-business identified from this research. table 2: reasons for adopting e-business company objectives a reach out to more customers achieve delivery efficiencies quick dissemination of sales and marketing information b to be able to operate 24*7 better customer service, to stay in the marketplace, to stay customer focussed and to extend our distribution opportunities to expand business into other products c manage a larger volume of data via edi to integrate and automated the backend system, to be more efficient, to reduce the operational cost, to improve accuracy, to facilitate decision making. to have good communication with the companies, improve internal communications, increase market awareness and to possibly grow our market niche. d to streamline business, save costs, for accuracy of data. savings in regards to procurement and purchasing (reverse auctions), speed, streamlining reducing errors in the system, reduce working capital. e a follow on strategy from business partners. b2b hubs more suppliers, integration of technology with existing suppliers, better information sharing and management f to achieve business efficiencies and to use state of the art technology g directive from buyers (major supermarkets) improved business , convenience, easier, quicker and to make staff work smarter h to remove the non-value adding processes in the supply chain. to collaborate with suppliers for increased business to maximise sales i achieve business efficiencies, automate business processes, smart applications of technology reduce costs, smarter ways of operating, increase profits, increase marketshare and for 24*7 business hours j 24 * 7 business, to generate home shopping for our customers k to achieve business efficiencies, reduce costs, integrate and automate business processes, improve purchasing since most of the organisations engaged in b2b operations, it is apparent from this research that efficient and improved procurement processes, access to new suppliers and buyers, business efficiencies achieved from linking erp systems to front end operations and online real time processing of information for supply chain management were the most important reasons for developing e-business. making employees work smarter with technologies was also identified as a reason for e-business. ajis vol. 12, no. 1 september 2004 33 challenges and achievements of e-business for early adopters while the opportunities of electronic commerce are undeniable, responses listed in table 3 indicate that early adopters faced numerous challenges. table 3: challenges of e-business implementation co major challenges achievements a streamlining delivery system, distribution systems and ordering systems. improved supply chain, improved customer relationship management, automated business processes absolutely a new distribution channel yes disintermediation yes. b changing order fulfilment (filling orders) from traditional to automated system reduced advertising costs, increased customer base, improved customer relationship management, automated business processes c integrating and automating business processes internally and with business partners improved supply chain, improved customer relationship management, integrated backend system, automated business processes, reduced procurement costs, competitive advantage, disintermediation. d keeping up with evolving technologies, debugging technologies and to make them work, getting something that will work for everyone. reduced advertising costs, improved supplier relationship, reduced procurement costs, some disintermediation e a shortage of skills in-house and the huge costs of hiring it consultants improved supply chain management, improved customer relationship management, integrated backend system, automated business processes, knowledge management f getting business partners to use ebusiness, ‘every piece of paper has our web address on it,… but nobody uses it’ improved supply chain management with e-mail, knowledge management g training staff improved supply chain management, improved customer relationship management, automated business processes. h a large and complex project managing e-business development was like ‘trying to move a goliath’ improved supply chain management , improved supplier relationship management, integrated backend system, automated business processes, knowledge management, reduced procurement costs, competitive advantage, a new distribution channel i keeping up with technology costs improved supply chain management, improved customer relationship management, automated business processes. reduced procurement costs, ajis vol. 12, no. 1 september 2004 34 competitive advantage. j unable to realise benefits to achieve competitive advantage. k for our suppliers to accept technology and implement technology take hold of what we developed for business exchange and collaboration improved supply chain management, integrated back end system, automated business processes, knowledge management, reduced procurement slightly findings presented in table 3 indicate that with the adoption of e-business, early adopters faced the challenges of managing change from traditional business processes to automated processes, as well as getting their business partners to accept this new way of doing business. high cost of technologies is another important challenge identified from this research. unable to manage and organise a fast delivery system, changing the existing distribution system and integrating systems of business partners and internal business processes that were essential for e-business were also apparent. keeping up with evolving technologies and their applications, and getting a return on investment, were other issues identified as problems. a shortage of skilled staff to implement and manage ebusiness was highlighted as an important challenge faced by early adopters, as e-business was new and there were very few skilled personnel or consultants at the time. however, achievements from e-business, however small include supply chain improvements, reduced costs of operation, better customer and partner relationship management, better knowledge management, a competitive advantage and disintermediation of brokers and other sales agents. eight organisations indicated improved supply chain management to be the most important outcome. two indicated a reduced advertising cost; 6 indicated improved customer relationship; 3 successfully disintermediated middle people; and 1 increased its customer base. company a, which is a small organization, was happy to have acquired a new channel of business with a b2c ebusiness model. companies e, g, h and k indicated achievement of improved knowledge management within the organization with e-business. automated processes and integrated back end systems for efficiencies in data processing were achieved by 8 of the 11 companies. procurement efficiencies were achieved by five of the organizations that were all b2b and large organizations. other benefits of e-business achieved were improved marketing, business efficiencies from automated processes, reduced costs, a force with technology skills, speedy communication with customers and business partners, and better use of technology in the organization. other e-business development issues other e-business development issues identified from this research are presented in table 4. it is apparent from the findings listed in table 4 that most of the organisations depended on outside providers to develop and implement their e-business application. five respondents indicated that their e-business was developed in-house with the help of it consultants and web developers. only one organisation (company d) developed the entire e-business in-house. it is interesting to note that although most organisations extensively planned the project, companies e, f and j launched their ebusiness with no formal plans. for all those companies that outsourced e-business development, formal contracts were agreed upon, while those that developed in-house depended on a strategy developed in-house for project management. it is also evident that for the companies that developed e-business in-house either exclusively by their it department or in conjunction with it consultants managed regular updates of their web sites which is an essential feature of e-business. update of information on the web sites was identified to be periodic, ranging from daily, weekly to six monthly. those organizations that that outsourced e-business development updated their web sites weekly or less frequently. this clearly indicates the importance of having staff with it skills within the organisation. it is apparent that companies with it departments had an important role in the e-business project, while those that did not depended entirely on outside support. when asked about the volume of transactions, companies b, d, g, h and i indicated that it was high. it is also important to note that these companies also indicated that their e-business project was very ajis vol. 12, no. 1 september 2004 35 successful. companies a, c, d and k indicated that the volume of transactions with e-business was moderate indicating that they were somewhat unsuccessful. companies e and f indicated that their e-business was completely unsuccessful. companies e and f, were subsidiaries of larger organizations that lodged e-business without a formal plan and did not have skilled staff in the organisation. table 4: e-business development issues co e-bus development system dev method and planning web site updates role of the it dept e-bus transactions a it consultant with an in-house team extensive planning updated weekly none moderate – somewhat successful b bought a ec product from telstra, web designers, web hosting co formal planning mentally a lot of effort, financially not. weekly update. none high – very successful c web development co. and it department aligned to business plan marketing people do content management. moderate neither successful nor unsuccessful d predominantly inhouse. a formal strategy considerable amount of time. aligned to business strategy high slightly successful just starting e it consultant no formal plans not asked facilitating role. very low unsuccessful f web development co no formal plans e-business leader none. none very unsuccessful g financial controller and a web development company as directed by coles and woolworths half and hour each day. some role ec is only applicable to 3 departments. one successful, other two negligible h myself and my group formal plans no response installation support and backup by vendors high very successful i our current department it department developed plans not arduous but ongoing. transparent information high very successful j two private programmers and consultants no formal plans i have not been shown how to update. none very unsuccessful k self and contract programmers project management tasks and formal plans as required prices monthly, business information every six months their role was probably pivotal, without them we couldn't do it. high 100 orders per week slightly successful ajis vol. 12, no. 1 september 2004 36 discussion this research has highlighted some b2b developmental issues gleaned from early adopters of ebusiness in the australian food industry. discussion arising from this research mostly addresses b2b e-business issues, presented under the common themes of technology, business, and b2b ebusiness applications. technology from this research it is apparent that edi systems form the foundation technology supporting b2b e-business. being proven and trusted technologies, edi systems supported the adoption of other new technologies to further develop e-business. other technologies supporting early developers of ebusiness, besides the internet and the world wide web, were extranets, intranets, erp systems, and technologies to support online auctions, supply chain management, and e-procurement. although extranets may be regarded as an extension of the edi system, the increased adoption of intranets to support internal employee management clearly indicates the realisation of benefits of new technologies by these organisations. using known technologies minimises mistakes, costs and the need to customise software. some organizations developed e-business in-house and some outsourced its development. in-house development is dependent on the skills and expertise of the it department within the organization. e-business systems developed in-house also have the advantage of regular updates, an essential feature of successful e-business. developing e-business in-house also has the advantage of making the it manager or e-business project leader accountable for errors and warrants quick solutions. the indication that many of these organisations integrated back end systems to their front end e-business shows the value of integrated and automated business processes was well understood by organization in the food industry. rapid exchange of data and documents and electronic transactions are easily transferred to back office systems for record retention, information management and data mining, to predict demand and inventory management. outsourced e-business development lacks continuity and ongoing support. research also shows that e-business technology is fast evolving, therefore having technical expertise within the organization is an advantage enabling companies to capitalise on the benefits of new technologies. education and training bridge the gap between development and successful implementation of new technologies (singh, 2000). training provides new skills and methods required to maintain the new internet based electronic trade. for those organizations that do not have an it department or technical expertise, it is essential to provide ongoing training for staff to meet the needs of e-business. application of technology leads to quick retrieval of information, which also leads to improved supply chain management. business issues business opportunities of e-business and its adoption are positively related. efficiencies of reduced costs; managing partner relationships; using the internet as a marketing tool; market expansion; promotion and improved image; and a competitive advantage are returns that early adopters of ebusiness anticipate from e-business. promotion of goods to an increased number of buyers, and trading online with electronic ordering; online tracking of orders and sales; improved deliveries; online supply chain information; and electronic communication enhance trust amongst business partners and further expand business. disintermediation occurs when the services of middlemen become irrelevant. research findings indicate that many organisations aimed to transform the market chain; that is, reach out to their customers and business partners directly. not having to deal with middle people meant reduced costs for the sellers, providing faster service and sometimes more accurate information to buyers. organisations investigated achieved improved business efficiencies by automating business processes. automated processes result in efficiencies such as reduced paperwork, errors, time and ajis vol. 12, no. 1 september 2004 37 overhead costs. as suggested by cameron (1997), although the establishment and maintenance of a web site has costs of its own, the price of using the web compared to other sales channels is significantly less. as an advertising medium, promotion using a web site results in the sale of ten times the number of units with one-tenth of the advertising budget and one quarter of direct mail expenditures (cameron, 1997). there is no time lag between publishing information on a web site and it being accessed by buyers, manufacturers and other business partners. it is useful to note that even the early adopters of e-business had realised the opportunities of technology to evolve into a 'smart and innovative organisation', and used knowledge management to improve business. b2b e-business application e-procurement is the most important b2b e-business application in australia (noie, 2001). it includes catalogue management, requisition, control and approval, receiving and exception processing of orders, and financials and payment processing. singh and thomson (2002) advocate that e-procurement processes include sourcing of buyers and sellers, a digital catalogue of products, online bidding, ordering, payments, goods dispatch notices (fulfilment), logistics and supply chain management. research in the australian food industry indicates that in spite of the hype surrounding online groceries, which were to be achieved from b2c e-business, b2b exchanges and e-procurement were more important applications of e-business. edi systems have supported business purchases for the last ten years. the application of the internet, the world wide web, electronic market places, and other business applications further enhance b2b exchanges as apparent from this research. an improved supply chain management also leads to inventory management, which leads to reduced costs and additional business efficiencies. conclusion this research identified issues faced by early adopters of e-business. it highlights that proven technologies, such as edi systems are important foundation technologies supporting e-business development. e-business developed in-house warrants continuation and ongoing support, while those outsourced generally are given ad hoc attention. for e-business organisations an it department is an advantage. e-procurement has developed a lot more since 2000, with an increase in intermediary functions. more e-market makers resulted as business enterprises facilitating b2b exchanges. therefore instead of disintermediating middle people, there is now an increased reliance on them for e-procurement in the b2b e-business. application of e-business has also resulted in improved supply chain management for organizations, which was an unanticipated outcome at the time of implementation. the development of e-business for early adopters was ‘learn as you go’, evidenced from this research. however, the achievements show that the b2b e-business model will have an important role in the australian food industry. references anderson consulting, a review of e-commerce in australia (1999), http://www.ac.com/services/ecommerce/ecom_australia.html, april. archer, n., and gebauer, j., (2000), managing in the context of the new electronic marketplace, proceedings of the first world congress on the management of electronic commerce, mcmaster university, hamilton, canada (cd rom). cameron, d., (1997), electronic commerce: the new business platform for the internet, computer technology research corporation, south carolina, usa. dublish, s., (2000), retailing and the internet, proceedings of the first world congress on the management of electronic commerce, mcmaster university, hamilton, canada (cd rom). dunt, e. s. and harper, i. r. (2002), “e-commerce and the australian economy”, the economic record, vol. 78 no. 242, pp. 327-342. ajis vol. 12, no. 1 september 2004 38 harrison-walker, l. j. (2002), “if you build it, will they come? barriers to international emarketing”, journal of marketing theory and practice, vol. 10 no. 2, pp. 12-21. ho, c., and wu, w., (1999), antecedents of customer satisfaction on the internet: an empirical study of online shopping', proceedings of the 32nd hawaii international conference on systems sciences, pp 1 9. johnson, t. m. and griffith, d. a. (2002), “5 steps to online success”, marketing management, vol. 11 no. 5, pp. 34-39. kalakota r. and robinson m, 2001 'e-business: roadmap for success', addison –wesley, canada. noie report (2001), ‘b2b e-commerce: capturing value online’, commonwealth of australia. noie report (2003), the benefits of doing business electronically ebusiness. singh, m., (2000), electronic commerce in australia: opportunities and factors critical for success, proceedings of the first world congress on the management of electronic commerce, mcmaster university, hamilton, canada (cd rom). singh, m. and thomson, d., (2002), ‘an e-procurement model for b2b exchanges: an australian example’, 15th bled conference on ereality: constructing the eeconomy, june 17 – 19, slovenia, cd rom. singh, m.,( 2002), ‘electronic commerce opportunities, challenges and organisational issues for australian sme’s’, managing information technology in small businesses: challenges and solutions edited by burgess, s., idea group publishing, hershey, usa, pp 322 – 339. stoney, m. a. and stoney, s. (2003), “the problems of jurisdiction to e-commerce – some suggested strategies”, logistics information management, vol. 16 no. 1, pp. 74-80. thuraisingham, b., gupta, a., bertino, e. and ferrari, e. (2002), “collaborative commerce and knowledge management”, knowledge and process management, vol. 9 no. 1, pp. 43-53. turban, e., lee, j., king, d. and chung, h.,(2000), electronic commerce: a managerial perspective, prentice hall international, usa. yin, r., (1994), case study research design and methods, sage publications, new delhi, india. microsoft word ajis12-1final repaired.doc ajis vol. 12, no. 1 september 2004 3 editorial this is the first issue of volume 12 of the australasian journal of information systems. our first paper is presented by robyn davidson & susan lambert from flinders university and is entitled ‘applying the australian and new zealand risk management standard to information systems in smes’. unlike the identification of some business risks, identification of risks associated with information systems requires certain technical expertise. this paper advocates the use of the australia/new zealand risk management standard (sa/snz, 1999) in conjunction with of a modified version of birch and mcevoy’s (1992) structured risk analysis for information systems (sra-is) to identify information systems security risks in smes. charles shoniregun, university of east london and oleksandr logvynovskiy, london south bank university present our next paper, entitled ‘securing xml documents’. this paper discusses a fast and efficient indexing technique for xml documents, and introduces the xml graph numbering scheme. it can be used for indexing and securing graph structure of xml documents. mohini singh, rmit presents our next paper entitled ‘e-business developmental issue in the australian food industry’. this paper discusses e-business developmental issues in the australian food industry that were identified from a research project funded by the australian research council. bjorn lundell, university of skovde and brian lings, university of exeter, present our next paper entitled ‘on understanding evaluation of tool support for is development’. this paper concludes that that there is a need for enriched method support to deal with the complex socio-technical issues involved in the summative assessment of is tools. sadiq sait, king fahd university, khalid al-tawil, ministry of the interior, riyadh and syed hussain, king fahd university present our next paper entitled ‘e-commerce in saudi arabia: adoption and perspectives. this paper reports on the consequent findings, which identify factors that may significantly affect the adoption of ecommerce in the kingdom. uchenna eze and a.l. gilbert from nanyang technical university, singapore present our next paper entitled ‘the significance of electronic commerce to firms’ operations in relation to business location: an empirical investigation’. this research examines the implications of location for electronic commerce (ec) role in firm operations through the lens of managerial perceptions of ec systems, ec activities, agglomeration economies, firm-specific features and outputs relative to industry. our next paper is entitled ‘realist and postmodernist perspectives on information systems research: points of connection’ and is presented by philip dobson and peter love, edith cowan university. this paper contributes to that debate by contrasting the post-modernist approach with that of the critical realist. ronald becket, the reinvention network, sydney presents our next paper entitled ‘exploring virtual enterprises using activity theory’. this paper suggests that exploration of the nature of tools, communities etc is helpful in understanding ve operations, and applied at multiple levels, activity theory is also useful in understanding the evolution of those tools, communities etc. our final paper is presented by renzo gobbins, university of canberra and is entitled ‘the application of intentional subjective properties and mediated communication tools to software agents in online disputes resolution environments’. this paper examines the use of subjective properties in modeling an architecture for cooperative agents using agent communication language (acl) that is used as a mediating tool for cooperative communication activities between and within software agents. ajis special edition knowledge management december 2001 knowledge management technology for decision support: an empirical examination meliha handzic school of information systems, technology and management the university of new south wales sydney 2052, australia phone: (02) 9385 4935 fax: (02) 9662 4061 e-mail: m.handzic@.unsw.edu.au abstract this paper reports the results of an empirical examination of the effectiveness of one type of knowledge management technology, namely 'contextual knowledge repository', for supporting individual decision makers in a predictive judgement task context. 31 volunteer subjects participated in the study. the results indicate that a given technology was fairly useful, but insufficient to maximally enhance individual decision making. on one hand, subjects were found to extract more knowledge and make significantly smaller decision errors than their notional naive counterparts. on the other hand, subjects tended to extract less knowledge and make significantly larger decision errors compared to notional optimal counterparts. these findings suggest that individuals could potentially benefit from those knowledge management technologies that would provide additional explicit analytical and procedural knowledge, or those that would facilitate sharing of tacit knowledge through interaction with others. future research is necessary to address these issues. introduction human society is experiencing a major transformation from an industry-based society to a knowledge-based society. with this transition, there is a growing recognition among researchers and practitioners alike for the need to better understand the value of knowledge, what knowledge is, and how it should be managed. in general, knowledge management literature indicates a widespread recognition of the importance of knowledge with respect to the struggle for economic success (devlin, 1999; drucker, 1993; stewart, 1997), but little shared understanding of the construct itself (davenport and prusak, 1998; devlin, 1999; grayson and dell, 1998). there are also differences among researchers in what constitutes useful knowledge and the ways in which it is created. some theorists show more interest in codified repositories and information processing as enablers of 'explicit' objective and systematic knowledge (baxter and chua, 1999). others focus more on 'tacit' knowledge that people derive from their experiences and from social interaction with others (nonaka and takeuchi, 1995; nonaka, 1998). as organisations become more knowledge-based, their success increasingly depends on how successful knowledge workers are at creating and applying new ideas productively and efficiently. the central task of those concerned with knowledge management is to determine ways to better cultivate, nurture and exploit knowledge at individual, group and organisational levels. a generic model of knowledge management (aa, 1998) suggests four organisational initiatives leadership, culture, technology and measurement as major enablers that facilitate knowledge management processes and foster the development of new knowledge. however, given the current infancy of the knowledge management research, there is little empirical evidence regarding the actual impact of these initiatives on knowledge processes and returns from their application. the main purpose of this paper is to provide a deeper insight into the impact of one particular technology initiative codified knowledge repository on individual working knowledge and performance in the context of decision making. codified knowledge repositories fall into the category of knowledge management systems for managing knowledge artifacts that have an inherent structure (eg. sales figures in a data warehousing system) or those where the system imposes a structure on the contents (eg. keyword tagged report in a document repository). typically, these systems use database management systems and capture enterprise wide data from the organisational knowledge domain (hahn and subramani, 2000). it is implicitly assumed that the availability of such systems should lead to an increase in individual working knowledge and result in better job performance. this study intends to empirically test this assumption. various knowledge management systems implementations provide differing levels of support in locating and extracting knowledge and impose differing burdens to their users. the main objective of the current study is to determine the nature of assistance, the extent of assistance and the limitations of codified knowledge repositories in supporting knowledge work of managerial decision makers. in particular, this study examines whether and how explicit contextual knowledge that is captured in computerised stores may assist individual decision makers in enhancing 16 ajis special edition knowledge management december 2001 their working knowledge and improve the quality of their subsequent decisions in a specific judgemental decision making task. literature review and research questions contextual knowledge and decision making theoretically, the availability of environment and organisation specific knowledge, as well as a variety of causeeffect relationships, can play an important role in improving decision quality. the value of such contextual knowledge may be seen primarily in its ability to explain past and anticipate future changes in the behaviour of any particular variable of interest and enabling the decision maker to deal more competently with his or her decision task. in the case of production planning, for example, product and industry knowledge should help the decision maker to make more accurate estimates of future sales, translate them into more suitable production plans and, as a result, minimise losses due to overproduction or missed sales opportunities. this suggests that the provision of valid contextual knowledge in codified organisational databases should increase the decision maker's working knowledge base and consequently improve an individual's decision performance. past empirical studies indicate mixed and inconclusive findings. several studies using real world settings support the notion that contextual knowledge improves the quality of human judgement. edmundson et al. (1988) found that business practitioners who possessed considerable knowledge of the specific products being forecast made more accurate sales predictions than students with no such contextual knowledge. this finding was also supported by sanders and ritzman (1992). comparative analysis showed that the difference in the accuracy of forecasts generated with and without contextual knowledge was particularly significant under conditions of high variability of time series data. in addition, human judgement with contextual knowledge was found to lead to improved forecast accuracy over statistical models (edmundson et al., 1988; fildes, 1991). in the judgemental adjustment context, non-time series contextual knowledge was also found to be beneficial. mathews and diamantopoulos (1986, 1989) reported that managers who possessed considerable product knowledge were able to successfully adjust exponential smoothing forecasts. similarly, wolfe and flores (1990) showed that the accuracy of a statistical forecast could be improved when judgementally adjusted in the light of contextual knowledge when the statistical forecast was of low-quality and the time series involved contained a high level of variability. in contrast, a more recent field study of sales forecasting by lawrence et al. (1995) revealed that bias and inefficiency could mask any contribution of seemingly helpful contextual knowledge to the accuracy of judgemental forecasts. this study compared the accuracy of judgementally produced monthly product forecasts with naive forecasts of thirteen companies. contrary to expectations, the company forecasts were more accurate than the naive forecasts in only four of the companies, while the naive forecast was more accurate than the company forecasts in two companies. in the remaining seven companies, the difference was insignificant. empirical evidence from a number of laboratory studies also casts doubt on people's ability to effectively process explicit contextual/causal knowledge. in a study comparing the accuracy of judgemental causal forecasting with judgemental extrapolation, andreassen (1991) showed that the causal group, provided with current situational diagnostic variables, significantly underperformed the extrapolation group. similarly, harvey et al. (1994) found that cross-series forecasts of train passengers made from correlated train criminals series were less accurate than the corresponding within-series forecasts made from the past passengers series only. however, it is possible that these causal variables had poor predictive validity. finally, in a series of judgemental adjustment studies, lim and o'connor (1995, 1996a) found that people had difficulty in placing adequate weight on the reliable causal factors and did not learn to modify this behaviour over time. when the environment was multivariate, people were found to have problems in both evaluating and using the explicit contextual knowledge available (lim and o'connor, 1996b). task complexity may be one of the potential contributing factors to the reported inefficiency (beach and mitchell, 1978; payne, 1982; wood, 1986). research questions in view of inconsistent prior findings and concerns expressed, it is of particular interest to this study to examine the situation in which decision makers are provided with a small amount (to avoid complexity) of highly predictive contextual knowledge (to ensure validity) in regard to the decision response. it is argued that such a situation well represents a typical context of a real world organisation in which its critical knowledge is packaged for use in a 'knowledge mart'. the question is (i) whether and how well can individual decision makers make sense of what is available 17 ajis special edition knowledge management december 2001 in such knowledge repositories and enhance their working knowledge, and (ii) whether and what impact the application of that knowledge may have on their subsequent decision performance. research methodology experimental task the experimental task for the current study was a simple production planning activity in which subjects made decisions regarding daily production of fresh ice-cream. the participants assumed the role of production manager for an imaginary dairy firm that sold ice-cream from its outlet at bondi beach in sydney, australia. the company incurred equally costly losses if production was set too low (due to loss of market to the competition) or too high (by spoilage of unsold product). the participants' goal was to minimise the costs incurred by incorrect production decisions. during the experiment, participants were asked at the end of each day to set production quotas for ice-cream to be sold the following day. subjects were required to make thirty production decisions over a period of thirty consecutive simulated days. before commencing the task, participants had an opportunity to make five trial decisions (for practice purposes only). ice-cream was demanded by consumers whose decision to buy or not was based on a number of determinants. from pre-experiment discussions with actual store owners at bondi beach, three factors emerged as important in determining local demand for product: the ambient air temperature, the amount of sunshine and the number of visitors/tourists at the beach. this important contextual knowledge was provided to the participants in addition to past product demand to aid their decision making. subjects were free to access and use the available knowledge as they wished to. during the experiment, each subject was required to indicate his or her subjective assessment of the value of the available knowledge, and dominant strategy used in making production decisions. contextual time series were artificially generated with cue-criterion correlations set to r=0.65 to ensure high predictive power. optimal decision rule was derived by using a linear regression model with three contextual factors as independent and product demand as dependent variables in the equation. the task was challenging. the participants were provided with the meaningful task context, a history of task relevant variables to provide some clues to causal relationship, and forecast values of the three contextual variables to suggest future behaviour. however, they were not given any explicit analysis of the predictive quality of their factors, or optimal rule they could apply to make specific judgements. at the beginning of the experiment, task descriptions were provided to inform subjects about the task scenario and requirements. in addition, throughout the experiment, instructions and feedback were provided to each participant to analyse earlier performance and to adjust future strategies. experimental design and variables the experimental design for the study was a single factor design with experimental subject as the only independent variable. experimental subjects were divided into three groups. the first group consisted of actual student subjects. the other two groups consisted of their nominal naive and optimal counterparts. these were imaginary decision makers who produced their decisions by using naive (random walk) and optimal (linear) strategies respectively. subjects' performance was evaluated in terms of decision accuracy and subjective assessments of decision process. decision accuracy was operationalised by absolute error (makridakis, 1993) and calculated as the absolute difference between subject-estimated and actually-demanded units of product per day. in addition, the corresponding errors of nominal naive and nominal optimal decision makers were calculated. these scores were used to assess how much of the available contextual knowledge was used by the actual subjects in making their decisions. subjective assessments included perceived value of contextual knowledge and self-reported dominant decision strategy applied on the task. this information was collected to supplement the measure of decision accuracy, which was the primary indicator of the successful knowledge enhancement and application. subjects and procedure the real subjects were 31 graduate students enrolled in the master of commerce course at the university of new south wales, sydney. they participated in the experiment on a voluntary basis, had no prior knowledge of the task and received no monetary incentives for their performance. generally, graduate students are considered to be appropriate subjects for this type of research (ashton and kramer, 1980; remus, 1996; whitecotton, 1996). the experiment was conducted in a microcomputer laboratory. on arrival, subjects were given a diskette with an appropriate 18 ajis special edition knowledge management december 2001 version of the research instrument to be used. the instrument was specifically developed by the author in visual basic. subjects were briefed about the purpose of the study, read case descriptions and performed the task. the session lasted about one hour. results decision accuracy the collected data were analysed statistically using a series of t-tests to compare actual subjects' decision accuracy with that of their nominal naive and optimal counterparts. results are presented graphically in figure 1. the analysis performed found a significant difference in mean absolute error scores (mae) between actual subjects and nominal naive decision makers (429 vs 502, t=-4.45, df=929, p=0.000). as shown in figure 1, actual subjects tended to make smaller decision errors than their nominal naive counterparts. further analysis showed that they achieved 14.5% reduction in decision error due to the acquired contextual knowledge. the results of the analysis also revealed a significant difference in mean error scores (mae) between actual subjects and nominal optimal decision makers (429 vs 203, t=15.78, df=929, p=0.000). figure 1 shows that actual subjects a mae 500 -' 400 300 200 -' naive actual i optimal figure 1. decision performance (mae) by experimental subjects tended to make greater decision errors than their simulated counterparts applying an optimal linear strategy. further analysis found that they tended to use effectively only 24.4% of the available contextual knowledge from the given repository. subjective assessments the analysis of the participants' subjective responses to a questionnaire indicated that most of them perceived the available knowledge as valuable, but misinterpreted the relative importance of individual contextual factors. only 9.7% of the subjects accurately discovered that "all three factors are equally important in predicting sales". a majority of 64.5% assessed "all three contextual factors are important, but some more than others", while 16.1% found "two out of three contextual factors are relevant". the remaining 9.7% had various other interpretations. 19 ajis special edition knowledge management december 2001 with respect to self-reported dominant strategies, the analysis indicated that these were largely consistent with earlier reported subjective assessments. only 12.9% of the subjects applied normatively optimal "equal weighting" judgement strategies. a majority of 54.8% applied "differential weighting" linear models, while 16.1% "followed the direction of two factors in agreement". the remaining 16.2% applied various other mixed strategies. discussion main findings in summary, the main findings of this study indicate that valid contextual knowledge embedded in computerised repositories enhanced individual decision makers' performance, however performance gains were less than theoretically possible. this was demonstrated by significantly smaller decision errors found among actual subjects than their notional naive counterparts, but greater decision errors compared to notional optimal counterparts. the fact that participants performed better than naive decision makers indicates that they were able to discover and use some of the available contextual knowledge from their repositories. the analysis found that most subjects tended to identify as relevant and use at least two (out of three) contextual cues in making their decisions. as a result, they tended to achieve some improvement in their subsequent performance. in real terms, decision errors dropped by 14.5%. such findings seem to contradict the overly pessimistic picture of human ability to utilise contextual knowledge painted by earlier laboratory research in judgement and decision making (see eg. andreassen, 1991; harvey et al. 1994). one potential explanation for the difference may be in the characteristics of the decision tasks. participants in the current study were provided with a meaningful task context, a small number of highly predictive contextual variables, their historic time series to provide some clues to causal relationships, and forecast values to suggest future behaviour. it is also possible that a graphical form of presentation facilitated interpretation and enabled the subjects to better judge the right direction of future changes. this is consistent with lawrence et al. (1985) findings that graphical presentation enhances the accuracy of novice decision makers. despite this, the results indicate a lot of room for further improvement. actual subjects were found to make substantially greater decision errors than their nominal optimal counterparts. greater than optimal errors indicate that the subjects tended to uncover and use much less of the available knowledge than they possibly could. further analysis revealed that, on average, they tended to effectively internalise only 24.4% of the explicit knowledge provided to them. the failure to achieve optimal performance resulted mainly from the participants' misinterpretation of relative importance of individual contextual factors, and subsequent choice and application of inappropriate judgement strategy. a potential explanation for the observed suboptimality may be the lack of vital analytical and procedural knowledge in their repositories. subjects in the current research were not given any explicit analysis of the quality of their contextual factors, or rules they could apply to integrate the available factual knowledge into single decision response. instead, all subjects had an opportunity to learn from own experience through task repetition and from feedback. however, it seems that the period of thirty trials was too short to induce more effective learning. earlier studies on learning from feedback in multivariate tasks (for review see klayman, 1988) indicate that while people can reasonably well learn the existence and direction of cue-criterion relations over a number of trials, they generally have difficulty in learning their shape. as a result they tend to perform sub-optimally. in summary, the main findings of the present study indicate that the provision of a knowledge repository containing task relevant contextual knowledge was useful in improving decision making, but insufficient to enable individual decision makers to achieve optimal performance. limitations while the current study provides a number of interesting findings, some caution is necessary regarding their generalisability due to a number of limiting aspects. one of the limitations refers to the use of a laboratory experiment that may compromise external validity of research. another limitation relates to artificial generation of information that may not reflect the true nature of real business. the subjects chosen for the study were students and not real life decision makers. the fact that they were mature graduates may mitigate the potential differences. no incentives were offered to the subjects for their effort in the study. consequently, they may have found the study tiring and unimportant and not tried as hard as possible. most decisions in real business settings have significant consequences. further research is necessary that would extend the study to other subjects and environmental conditions in order to ensure the generalisability of the present findings. 20 aj1s special edition knowledge management december 2001 practical implications and directions for future research although limited, the findings of the current study may have some important implications for organisational knowledge management strategies. they suggest that decision makers could potentially benefit from additional knowledge management initiatives that would enhance their understanding of the value of explicit knowledge embedded in organisational repositories. one possible solution is to provide repositories with more meaningful analysis, procedural knowledge and learning histories that might potentially help such workers better understand what works when and why (kleiner and roth, 1998). this, in turn, may result in better performance. alternatively, organisations may employ specialists trained in analytical and statistical reasoning who would perform a knowledge filtering process for professional and managerial knowledge workers (godbout, 1999). initiatives aimed at creating working contexts that encourage communication and culture of knowledge sharing may also potentially have a beneficial effect on enhancing decision makers' working knowledge and performance. organisations have come to realise that a large proportion of the knowledge needed by the business is not captured on hard drives or contained in filing cabinets, but kept in the heads of people. sources report that between 40% (aaote, 1998) and 90% (hewson, 1999) of the needed knowledge is (in the lingo of the business) tacit. the spiral knowledge model postulates that the processes of sharing will result in the amplification and exponential growth of working knowledge (nonaka and takeuchi, 1995; nonaka, 1998). yet, little is known of the ways in which tacit knowledge is actually shared, conditions under which this sharing occurs, and the impact it has on performance. finally, by combining and integrating various knowledge management initiatives organisations may potentially create synergy effects that would lead to even higher levels of knowledge and performance. according to davenport and prusak (1997) only by taking a holistic approach to management, it may be possible to realise the full power of knowledge ecology. further research may look at some of these initiatives and approaches. conclusions the main objective of this study was to investigate the impact of contextual knowledge repositories on individual decision makers' performance in the specific predictive judgement task. the findings indicate that participants were able to uncover and use some, but not all, relevant knowledge embedded in their available repositories. accordingly, they achieved some performance improvement, but failed to fully accomplish what was possible. although limited to the specific task and context, these findings may have important implications for organisations, as they suggest the need for implementing additional knowledge management initiatives to further enhance individual knowledge and performance. therefore, more research is necessary to systematically address various initiatives in different tasks and contexts, and among different knowledge workers, if a better understanding of the area is to be achieved. references aa (1998) the knowledge management practices book, the global best practices research team, arthur andersen, january. aaote (1998) bc knowledge management, arthur andersen office of training and education, arthur andersen. andreassen, p. b. (1991) causal prediction versus extrapolation: effects on information source on judgemental forecasting accuracy, working paper, mit. ashton, r.h. and kramer, s.s. (1980) "students as surrogates in behavioural accounting research: some evidence", journal of accounting research, vol. 18(1), pp. 1-15. baxter, j. and chua, w.f. (1999) "now and the future", australian accounting review, vol. 9(3), pp. 3-14. beach l.r. and mitchell t.r. (1978) "a contingency model for the selection of decision strategies", academy of management review, july, pp. 439-449 davenport, t.h. and prusak, l. (1997) information ecology, oxford university press,oxford. davenport, t.h. and prusak, l. (1998) working knowledge, harvard business school press, boston. devlin, k. (1999) infosense:turning information into knowledge, w.h. freeman and company, new york drucker, p.f. (1993) post-capitalist society, harper business, new york. edmundson, r.h., lawrence, m.j. and o'connor, m.j. (1988) "the use of non-time series information in sales forecasting: a case study", journal of forecasting, vol. 7(3), pp. 201. 21 ajis special edition knowledge management december 2001 fildes, r. (1991) "efficient use of information in the formation of subjective industry forecasts", journal of forecasting, vol. 10, pp. 597-617. godbout, a.j. (1999) "filtering knowledge: changing information into knowledge assets", journal of systemic knowledge management, january, pp. 1-9. grayson, c.j. and dell, c.o. (1998) "mining your hidden resources", across the board, vol. 35(4), pp. 23-28. hahn, j. and subramani, m.r. (2000) "a frameworkof knowledge management systems: issues and challenges for theory and practice", proceedings of icis'2000, brisbane. harvey, n., bolger, f. and mcclelland, a. (1994) "on the nature of expectations", british journal of psychology, vol. 85, pp. 203-229. hewson, d. (1999), "start talking and get to work", business life, november, pp. 72-76. klayman, j. (1988) "learning from experience", in brehmer, b. and joyce, c.r.b. (eds) human judgement. the sjt view, north-holland, amsterdam. kleiner, a. and roth, g. (1998) "how to make experience your company's best teacher", harvard business review on knowledge management, harvard business school press. boston. lawrence, m., edmundson, b. and o'connor, m. (1985) an examination of accuracy of judgemental extrapolation of time series, international j. of forecasting, 1, 25-35. lawrence, m., o'connor, m. and edmundson, b. (1995) a field study of sales forecasting: its accuracy, bias and efficiency, working paper, school of information systems, the university of new south wales, july. lim, j.s. and o'connor, m. (1995) "judgemental adjustment of initial forecasts: its effectiveness and biases", journal of behavioural decision making, vol. 8, pp. 149-168. lim, j.s. and o'connor, m.j. (1996a) "judgemental forecasting with time series and causal information", international journal of forecasting, vol. 12, pp. 139-153. lim, j.s. and o'connor, m.j. (1996b) "judgemental forecasting with interactive forecasting support systems", decision support systems, vol. 16, pp. 339-357. makridakis, s. (1993) "accuracy measures: theoretical and practical concerns", international journal of forecasting, vol. 9, pp. 527-529. mathews, b.p. and diamantopoulos, a. (1986) "managerial intervention in forecasting: an empirical investigation of forecasting manipulation", international journal of research in marketing, vol. 3, pp. 3-10. mathews, b.p. and diamantopoulos, a. (1989) "judgemental revision of sales forecasts: a longitudinal extension", journal of forecasting, vol. 8, pp. 129-140. nonaka, i. (1998) "the knowledge-creating company", harvard business review on knowledge management, harvard business school press. boston. nonaka, i. and takeuchi, h. (1995) the knowledge creating company: how japanese companies create the dynamics of innovation. oxford university press, new york. payne, j.w. (1982) "contingent decision behaviour", psychological bulletin, vol. 92 (2), pp. 382-402 remus, w. (1996) "will behavioural research on managerial decision making generalise to managers?", managerial and decision economics, vol. 17, pp. 93-101. sanders, n.r. and ritzman, l.p. (1992) "the need for contextual and technical knowledge in judgemental forecasting", journal of behavioural decision making, vol. 5(1), pp. 39-52. stewart, t.a. (1997) intellectual capital: the new wealth of organisations, doubleday, new york. whitecotton, s.m. (1996) "the effects of experience and a decision aid on the slope, scatter, and bias of earnings forecasts", organisational behaviour and human decision processes, vol. 66(1), pp. 111-121. wolfe, c. and flores, b. (1990) "judgemental adjustment of earnings forecasts", journal of forecasting, vol. 9(4), pp. 389-405. wood, r.e. (1986) "task complexity: definition of the construct", organisational behaviour and human decision processes, vol. 37(1), pp. 60-82. 22 ajis vol 9 no. 1 september 2001 tasks and methods of software maintenance: a process oriented framework khaled md. khan bmcew. n. lo * torbjom skramstad ** school of computing and information technology university of western sydney locked bag 1797 penrith dc nsw 1797, australia email: k.khan@uws.edu.au * school of information technology and multi media southern cross university p.o.box 157lismore nsw 2480 australia email: blo@scu.edu.au ** department of informatics and computer science norwegian university of sciences and technology n7491 trondheim, norway. torbiom.skramstad@idi.ntnu.no abstract the paper proposes a framework of various tasks involved in the software maintenance process. the work reported in this paper disassembles the complex process of software maintenance into tasks as to aid in the allocation of resources, acquisition of appropriate tools, and distributing responsibilities of the software maintenance process. the associated toolsets, methods, input-output sources, and communication protocols between tasks are addressed in this paper. this work is intended to conveying a high-level understanding of the software maintenance process and its dimensions. software maintenance can be viewed in various ways depending on its purposes, nature and characteristics. we also show that software maintenance and development are two separate processes, but they are highly interrelated and interdependent. we attempt to find the intersection of activities between the software development and maintenance processes in the final part of this paper, and the software maintenance process is integrated with the development process into a high level software life cycle model. keywords software maintenance, software development, software life cycle. introduction information systems always tend to change and evolve, some are more frequent, some are less frequent. the evolution of an information system is unavoidable, and it tends to degrade desirable properties of the system over time (notkin 1993). the evolutionary nature of the information system is formally defined and characterised by lehman (1980). lehman has proposed the essential laws of the structure of evolving software. software evolutionary laws in fact leave us with two options: either to maintain the products with their continual changing nature, or to accept the degrading performance of the product. to maintain a software product properly over time, we need a well-defined software maintenance process model comprising the precise tasks and methods. some work have already been done towards software process modelling (humphrey 1989a), but comparatively a less significant progress has been made on the area of software maintenance process modelling. the research tracks on software maintenance reported particularly in the last ten years suggest that the software engineering community continues to work for an widely accepted solution for the complex task of software maintenance. it is apparent that a widely accepted software maintenance paradigm is difficult to materialise. this is partly because the software maintenance process involves multi-dimensional activities comprising varieties of application domains with diverse sets of design rationales, dynamic business rules, and highly subjective design and programming styles. a universally accepted process is difficult to be defined to solve a wide range of maintenance problems. the procedures followed in software maintenance by practitioners are normally not well defined. it is reported that very few organisations adopt a separate process for maintenance because they cannot make a distinction between software maintenance and software development. this leads to some fundamental questions like, 'is software maintenance a separate process?', 'can one process satisfy all types of software maintenance?', and "which factors or activities make software maintenance distinct from, or similar to the development process?'. no research work on software maintenance could avoid these three issues. 51 ajis vol 9 no. 1 september 2001 there are reasons to believe that a better formalised maintenance process would significantly improve the efficiency of software maintenance (haziza et al. 1992). a more defined formalism describing various tasks, tools and methods is required to enable a clear understanding of the process. in this direction, modelling the software maintenance process would be the initial requirement. we have defined a framework of various tasks involved in the software maintenance process in this paper to meet this initial requirement. the purpose of this work is to disassemble the complex process of software maintenance into related tasks to aid in the allocation of resources, acquisition of appropriate toolsets, and distributing responsibilities. the paper is intended to convey a high-level understanding of the software maintenance process. first we define software maintenance as a collection of well-defined tasks aimed at maintaining an existing application software product. the paper also highlights the relationship between the software maintenance and the software development processes in an integrated life cycle model. there are plenty of reasons to argue that software maintenance and development are essentially two different processes, but the commonality between activities of these two processes can easily be seen in a well defined software life cycle model. we attempt to define an intersection between the software development and maintenance processes later in this paper. finally, we briefly compare our work with other models. background of the work the motivation of the work reported in this paper was actually generated from a maintenance project of a pc based application software. our candidate system was a small inter bank reconciliation system (ibrs) used in a developing country in asia. the candidate system was relatively small in terms of lines of code (log), and was written in a fourth generation language (4gl). the system was organised hierarchically, and included more than 100 modules and almost 35 physical data files in various format. the system was capable of providing at least 40 various types of services to the user. the company developing the system, later wondered whether it was possible to transform the system into a more portable and efficient programming language platform, like c, keeping the entire functionality of the system intact. one of the authors of this paper was assigned the responsibility to lead the project. he realised that it involved considerable re-engineering task. the maintenance experience with the project and our previous work reported in khan et al. (1996) prompted us to propose a software maintenance process framework as reported in this paper. framework of software maintenance process after presenting our generic software maintenance model in (khan et al. 1996), we realised later that the model lacks some detail information associated with each task. from this observation, we refined our work further, coupling methods, tools, source and destination of input and output associated with each task. this work is an extension of our earlier work in (khan et al. 1996). 52 ajis vol 9 no. 1 september 2001 components of the software maintenance process people involved required knowledge supporting tasks • users of the system • maintenance staff • original development staff => operational environment => design and program structure • domain of the software => maintenance requirements analysis => determination => program comprehension ^localisation & impact analysis ^generating test data => implementation plan figure 1: components of software product maintenance process in software maintenance three ingredients are most important: • people who are involved, • supporting tasks which are well defined, and • knowledge (structural, semantics) about the software product these three are inter-related and interdependent to each other as shown in figure 1. we will mainly focus our attention on the supporting tasks involved in the software maintenance process in this paper. these tasks are believed to be fundamental for all types of maintenance work, but the degree of importance of each task varies from project to project. supporting tasks are augmented with the people involved, the readily available knowledge about the software product, and the nature of the operational environments. our presentation in this paper is not tied to any specific process modelling notations or process program language. the framework defines a collection of interrelated tasks describing the properties of the software product maintenance process. the framework of the maintenance process is based on a view of a sequence of tasks performed by agents. the agents can be toolsets, tasks, people, or, most often, a combination of these three. a task itself may also play the role of an agent. the tasks depict what is 'really going on' in software maintenance activity. each task has seven major components or properties as follows. objectives, • input information, and the source of input to feed the tasks, • output information that the task produces, • methods, the way a task performs its activities, • toolsets that support the objectives and methods of the task, • communication that establishes relationships among the tasks, and • status of the maintenance process. the high level software maintenance process shown in table 1 focuses on how various tasks in a chain are to be performed in the context of the maintenance. table 1 is a two dimensional matrix in which the first column shows all components required for a given task. each of the remaining columns is allocated for individual task 53 ajis vol 9 no. 1 september 2001 and their required components. we now briefly describe the components required for each of the tasks defined in our process. objectives each task has one or more predefined objective. a task performs certain function, and supports the activities of another task. this model consists of a sequence of defined tasks that must be performed in order to achieve the designated objectives. each task could be further enlarged into a complete model if the project is complex enough. input and output information each task is committed to understand the information it receives, and it performs its acts on this information. each task has two classes of information. one is the information required for the task to perform its service, called sources of input information. the other category is the information generated within the task, and used by the task itself called output information. one of the neglected elements in maintenance models is the source of input information. quite often it is not mentioned how the various tasks will get their input data. lack of input information to the tasks may lead to an unsuccessful project termination or premature termination. toolsets and methods each task is implemented by defined methods, in turn the methods are supported by automatic tools and human interactions with the process. tools must support the underlying methods of a task to accomplish the objective of the task. automated tools, in some cases may not support methods; these should be supported by human intervention. tools must be able to use the available data format in the task, and produce data which in turn will be used by the subsequent tasks. many available tools can be used for software maintenance, but they may not be aimed at the maintenance process specifically. automated toolsets, manual procedures, or a combination of both can support tasks. however, most tool environments in software engineering have deficiencies in terms of integration and coordination of the tools with projects (sharon et al 1997). the associated tools must be applicable to a wide range of projects. it is certainly desirable to extend the adaptability of the tools to keep the process operative for a longer time. toolset must fit into the cultural context of the maintainers. it must support methods and techniques used by the programmer. several tool classification schemes exist, but very few of them are intended to maintenance tools. more work on software maintenance tools can be found in (durant 1989, holbrook e< al. 1987, khan et al. 1997). 54 ajis vol 9 no. 1 september 2001 components of the tasks objectives sources of input output methods tools communicati on status software maintenance tasks maintenance requirements analysis trigger of the process enactment program execution at the operational site real users of the system refined maintenance requirement specifications interviews, prototyping not specific determination examines the technical and economical feasibility requirements specification, knowledge on software and its nature and characteristics, organisational policy, status of tools and staff availability requesting to filter req. spec, termination message, filtered requirements, primary knowledge about the software verify requirements cost estimation software . t \vj premature termination program comprehensio n understanding semantics and architecture of the software source code, information from original designer and programmers readable from program documents recovered system design artefacts, program domain program walk through, program slicing, execution of program reverse engineering, design recovery, debugging, static analyser * u localisation and impact analysis identifying program location and ripple effects requirements specification, source code, class hierarchy, function call sequences, data structures, data file format function names, variables declarations program walk through, program slicing, execution of program code analyser, design recovery, reverse engineering tools generating test cases tests cases defined for proposed changes req.spec. source code function names, variables used test data, program path spec. regression testing, quality control test tools <j> impl. plan development process table 1: tasks of software maintenance process infrastructure communications the communications between tasks are done through the feed-back and feed-front loops to express the cycle of the model. the link arrow from one task to another indicates that the first one must follow the other. the feedback loop is expressed by the ^ arrow, and the feed-front loop is shown by the ̂ arrow. a line without any arrow shows the supporting component of the tasks. 55 ajis vol 9 no. 1 september 2001 status status of the entire maintenance process can be a premature termination of the process, or a transition to a separate process. the process termination, or transition to the development process is expressed by the arrow. —•• supporting tasks we now briefly describe each of the tasks defined in our maintenance process as shown in table 1. modification requirements analysis this task is considered as the trigger of the process. the process enactment occurs by activating this task. in this task, requests from users or from the system itself are received. these may include adding new functions, improving the performance of existing functions, migrating the system to other operational platforms, modifying the existing function, or correcting faults in the system. these requests are analysed in detail so the entire modification requirements can be well understood both by the user and the staff involved in the maintenance project. maintenance requirements certainly are different from that of a new development, therefore, it is important to analyse the maintenance requirements from a different perspective. determination (management decision) whether the software can absorb the changes successfully or not must be tested before implementation of the changes. this is a constraint mechanism in the maintenance process. this task examines the technical and economical feasibility of the project based on the requirement specification and the characteristics of the candidate software product. in this task various management issues are analysed such as how much effort will be required to implement the maintenance requirements. the answers to this fundamental management issue will determine whether the maintenance project should go ahead or not. thus, cost to benefit ratio and the merits of technical aspect required for the project are considered as the determinant of the maintenance process. the user requirements are refined and filtered at this stage. it actually includes estimating cost, availability of resources and tools. a feedback loop exists between this task and the previous task for a finer granularity of the requirements. if the project is found feasible, the feed-front loop will be activated. if the project fails to meet certain criteria set by this task, the entire process will terminate by activating the exit loop. program comprehension if the design documents are missing or unreliable, or the original designers are not available for consultation, the entire program architecture must be understood by the maintainer programmers. automatic or semi-automatic program comprehension tools are required to aid the task program understanding is not only important for the task, but also subsequent tasks require the full understanding of the source code and the domain knowledge of the system. in most cases, there is no historical track of how the product was actually developed and why certain types of design were crafted (curtis 1992). comprehension of the source code requires code reading, program execution, and the use of existing design documents. in such a situation, the assistance of automatic or semiautomatic program understanding tools like reverse engineering or re-engineering can be sought. localisation and impact analysis if the maintenance project involves modernisation or corrections, then the exact location in the source code where the proposed modifications are to be made is identified. the most difficult task in introducing a change to software is to detect the ripple effects of the proposed change to other parts of the system. an unforeseen side effect on other parts of the code may occur when a change is introduced. this can be augmented either through variables or values, or through parameter passing. the project members must understand the impact of every change they are introducing. software maintenance in most cases suffers with ripple effects due to code modifications. it is often reported that corrections in the software may contribute to create more errors in the code (humphrey 1989b). this is an important and difficult phase in software maintenance. without proper attention to and mastery of the candidate system design, it would be hard to find out how the intended modification can cause side effects in the system, and where these would occur (bohner et al. 1996). 56 ajis vol 9 no. 1 september 2001 generating test cases the test cases are designed to test the proposed changes in the maintenance process. the test data should include a wide range of possible data. test paths and the regression testing procedures are to be defined. actual testing takes place after the implementation of the proposed changes to the system. implementation plan in this task, implementation of the proposed modification is planned based on the output produced by other tasks. it includes how to update the existing specification and design documents, and how to re-code and configure the new and modified components of the system. this is the final phase in the maintenance process. when this task is completed the usual development process is activated: the maintenance process triggers the requirement analysis phase of the development process. it is important to note that our maintenance process actually does not implement any features in the existing software product. ways in which the implementation details of the maintenance process can be carried out in conjunction with the development process in a complete life cycle setting is discussed in the next section. in fact, in the present context, maintenance is seen as a continuation of the development process that begins the moment a software product starts its operation. it has been claimed that a significant part of software maintenance is in itself the development of new functions (wild etal. 1991). all information produced by the tasks is stored in a database repository. a repository or a database plays a central role in storing the data and keeps all data for the projects. it is usually an active database that reacts to certain activities performed to its data (froehlich et al. 1995). it facilitates human understanding and communication regarding the project. hypertext technology can be used to present multiple views of the knowledge structure. the repository can also expose the expected behaviour of the task and the actual behaviour that it has performed. the discrepancies between the expected function and the actual performance of the tasks can be easily traced from the repository as well as. and the process behaviour could be tuned accordingly. all tasks have access to this repository. however, the tasks cited in table 1 show the process of software maintenance only. the maintenance process needs to be integrated into the entire software life cycle which will show how it can synchronise with the development process as well. in such a model, it is important to show that the development environment can support the underlying maintenance methods and activities within a software development life cycle framework. a complete and mature life cycle model must satisfy both the development and maintenance processes. maintenance process and software life cycle the concept of a software life cycle is a model used to describe and explain the software development and the maintenance processes. but within the software engineering context, it has not been yet well established how software maintenance would relate to the software development process, and how these two differ from each other. software life cycle models presented so far fail to focus on many fundamental aspects of the evolutionary nature of software products. researchers have proposed a number of software life cycle models partly or completely ignoring the phase of software maintenance. most of these models do not provide clear guidelines on how to integrate the maintenance into a development process. there are reasons to believe that software maintenance is a separate process, but it is essentially related to software development as some of the tasks are fundamental for both processes, and information from the development process is always needed in the maintenance process. the intersection of activities between the software development and maintenance processes is shown in figure 2. the top-down waterfall model (royce 1970, boehm 1976) has been widely accepted by the software community while the spiral model (boehm 1988) has received considerable attention. the latter emphasises the risk analysis aspect of a software project, while the former views maintenance as a single phase in the post development chain. software development and software maintenance, the two processes in software engineering, constitute a cycle for the entire life span of a software system. 57 ajis vol 9 no. 1 september 2001 design coding testing installation operation software maintenance process software development process figure 2: intersection between software maintenance and development processes new development requests maintenance requests operational sw needs maintenance maintenance implementation plan figure 3: a high level software life cycle model there is a need for a well defined software maintenance process for the practitioners supporting a more complete software life cycle (foster 1992, chapin 1988, rombach et al. 1988). it has been pointed out that a well-defined process should have the ability to blend maintenance and development homogenously into a single cycle (curtis 1992). after integrating our maintenance process with the development process, the resulted scenario of the complete life cycle model is illustrated in figure 3. this high level life cycle scenario focuses on phenomena that occur during software construction and reconstruction. thus, this can be used in both the software maintenance context as well as the software development context. 58 ajis vol 9 no. 1 september 2001 the life cycle shown in figure 3 can be interpreted as a generic software re-engineering model. it suggests that the re-engineering process begins with the existing system, and produces a new form of the old system. this approach may be seen as similar to the spiral model, where the development cycle is repeated in ever increasing spirals. comparison with other maintenance models the software life cycle model expressing the maintenance process integrated with software development proposed by skramstad et al. (1992) shows similar steps but in a reverse time sequence. the phases of two processes are identical to each other as presented in their model. theater software maintenance environment (tsme) model (cherinka et al. 1994) shows how the software maintenance process can be supported and automated through the use of an integrated software engineering environment. this high level process model does not address how this automated maintenance environment would be related to the development life cycle model. ripple effects analysis and localisation activities which are important for a maintenance process are not explicitly identified in the model. the maintenance model proposed by makoto ino (1992) is very similar to the development process model. the five phases of this maintenance model are: analyse user's maintenance requirement, design and approval, implementation, testing, and installation. this work does not actually focus on how a phase is actually constituted. a generic maintenance process model adopted within the esf/epsom project (harjani et al. 1992) does not include an important maintenance phase like 'program comprehension'. this model comprises eleven main activities. it is not clear how these activities are supported by tools, methods and input information. the cyclic nature of the entire life span of a software product including the development process is missing in this model. the maintenance assistance capability for software (macs) (desclaux 1991) basically concentrates its activities on program understanding process of the existing application software. it covers the phases of reverse engineering, modification management, and ripple effect analysis. the spiral model (boehm 1988) claims to cover software development and software maintenance coherently. however, the model does not explicitly integrate the two processes, and does not explain which information is shared between the development process and the maintenance process, nor does it show how the information is interchanged between the two processes. the ieee standard 1219-1992 (ieee 1992) describes a process for managing and executing software maintenance activities. the process model is described as a sequence of process task: problem identification, analysis, design, implementation, system test, acceptance test, and delivery. each task has specific control, input, process, output, and factors/metrics. each step is described in detail, and the input/output is clearly identified. however, in the ieee standard, a clear interaction between the development and maintenance processes is lacking with respect to information from the software development process to maintenance, and the activities related to program understanding is not explicitly covered. the strength of this standard is the inclusion of the supporting issues such as planning, verification and validation, risk assessment, quality assurance, configuration management, and software metrics. the standard also has specific references to toolsets for each of the activities. most of the maintenance process models presented in various forums do not address how each phase will be constituted, and what is their relationship with the development process in a fuller life cycle setting. the associated tools, methods, and input-output information are not often represented in the model. conclusion the software maintenance process and its relationships with the development process as presented in this paper provide software organisations with a task-oriented framework on how to control their process for maintaining software. the proposed framework includes various tasks, methods, input-output information sources, and communication protocols between tasks related to software maintenance process. finally, the paper has made an attempt to define the intersection of common tasks between the software development and maintenance processes. this model could be related to other existing development processes in the organisation. the tasks may be tuned to be compatible to the needs of the projects. it may require constant monitoring to repair the model faults during the real maintenance project management. there should be a provision to accommodate for the changes the way the process works. the software maintenance process can be dynamic depending on the types of maintenance. to carry out a maintenance process it is important that the process is capable of sustaining the changes it needs. if the nature of process activities changes, then associated tools may change, also in certain 59 ajis vol 9 no. 1 september 2001 extent the methods too. this framework is intended to be flexible to accommodate more tools and methods, and the communication protocols between tasks can be adjusted if required by the maintenance project. acknowledgement we are thankful to all anonymous referees for their valuable comments and suggestions to improve the quality of the paper. references boehm, b. w. (1976), 'software engineering', ieee trans, on computers, vol. c-25, december, 1976, 12261241. boehm, b. w.(1988). 'a spiral model of software development and enhancement', ieee computer, may 1988, 61-72. bohner, s. a., and arnold, r. s. (1996). software change impact analysis, ieee computer press, 1996, 1376. chapin, n. (1988), 'software maintenance life cycle', ieee proc. conf. on software maintenance, 1988, 613. cherinka, r., overstreet, c. m., cadwell, a., ricci, j. (1992). 'issues in software process automation -from a practical perspective', ieee proc. conf. on software maintenance, 1994,109-118 curtis, b. (1992). 'maintaining the software process', ieee proc. conf. on software maintenance 1992,28 desclaux, c., and ribault, m., (1991). 'macs: maintenance assistance capability for software a k.a.d.m.e.', ieee proc. conf. on software maintenance, 1991, 2-12. durant, j. (1989). 'classifying software tools', software maintenance news, vol. 7., no. 3, december 1989, 16. foster, j. (1992). 'survey report', european sig in software maintenance newsletter issue 3: june 1992. 57. froehlich, g and sorenson, p. (1995). 'providing support for process model enaction in the metaview matasystem', ieee proc. seventh international workshop on case, toronto, july, 1995,141-149. harjani, del-raj, queille, jean-pierre (1992). 'a process model for the maintenance of large space systems software', ieee proc. conf. on software maintenance 1992.127-136. haziza, m., voidrot, j. f., minor, e., pofelski, l., and blazy, s., (1992). 'software maintenance: an analysis of industrial needs and constraints', ieee proc. conf. on software maintenance, november 1992,18-26. holbrook, h. b. and thebaut, s. m. (1987). 'a survey of software maintenance tools that enhance program understanding', serc-tr-9-f, university of florida, september 1987. humphrey, w. (1989a). managing the software process, addisson-wesley, 1989.1-494 humphrey, w. (1989b). 'quality from both the developers and user viewpoint', ieee software , september 1989, 84-100. ieee (1992). ieee standard 1219-1992 , ieee standard for software maintenance. ino, m. (1992). 'current state of software maintenance in japan: in depth view', ieee proc. conf. on software maintenance 1992.2729. khan, m. k., ramakrishnan, m. k., and lo, w. n. bruce. (1997). 'assessment of software maintenance model: a conceptual framework', proc. pacific asia conf. on information systems (pacis'97), qut, brisbane, april 1997, 527-536. khan, m. k., rashid, m. a. and lo, w. n. b. (1996). 'a task-oriented software maintenance model', malaysian journal of computer science, vol. 2, december 1996,36-42. lehman, m. m. (1980). 'programs, life cycles, and the laws of program evolution', ieee proc. conf. software engineering, 1980, 1060-1076. lehman, m. m. (1987). 'process models, process programs, programming support', acm proc. 9th int'l conf. on software engineering, 1987,14-16. notion, d.(1993). 'software evolution', acm sigsoft software engineering notes, vol.18, no.l, january 1993, 47. rombach, h. d. and basili, v. (1988): 1a panel discussion, position statement, ieee proc. conf. on software maintenance, 1988. sharon, d. and tracey, a.(1997). 'a complete software engineering environment', ieee software, 1997, march/april 1997, 123-125. skramstad, t. and khan, m. k. (1992). 'a redefined software life cycle model for improved maintenance', ieee proc. conf. on software maintenance 1992.193-197 royce, w. (1970). 'managing the development of large software systems', proc. wescon, august 1970. wild, c., maly, k., and liu, l. (1991). 'decision-based software development' journal of software maintenance, research and practice, john wiley & sons, vol. 3, march 1991, 17-43 60 microsoft word ajis112final.doc australasian journal of information systems vol 11 no 2 may 2004 14 strategic it planning, evaluation and benefits management: the basis for effective it governance peter marshalla and judy mckayb auniversity of queensland/mt eliza business school 380 st kilda road, melbourne, victoria, 3000 pmarshal@mteliza.com.au bschool of information management & systems, monash university sir john monash drive, caulfield east, victoria, australia 3145 judy.mckay@sims.monash.edu.au abstract this paper reports on the results of an empirical study into the integration of strategic information systems planning and business-it alignment, it evaluation, and the proactive management of business benefits in large organisations, and to consider the linkages evident between these processes. an argument is developed which suggests that at the heart of good it governance practice is an integrated cycle of building a business case, alignment and prioritisation of it investments with business objectives and imperatives, evaluation, system acquisition, and post implementation proactive benefits realisation. keywords it governance, strategic information systems planing, it evaluation, it benefits management introduction in contemporary business, there is intense interest in deriving business value form information systems (is) and information technology (it) investments through effective is/it governance (agarwal & sambamurthy 2002, shane et al. 1999). in many organisations, investment in it represents a large proportion of capital outlay, and indeed, it expenditures often represent the fastest growing category of investment for the organisation (strassmann 1997). thus it seems reasonable to conclude that it assets (in terms of computer hardware, software, telecommunications facilities and human knowledge capital) are very significant, and therefore entitled to thoughtful management and careful attention as to their value and contribution, and return to the organisation (willcocks 1994). however, concerns are all too frequently voiced by senior management about the size of their firm’s investment in it, and more specifically, about whether the firm enjoys adequate returns on this investment (willcocks 1996). it could be postulated that developing a sound it governance framework may move an organisation towards achieving better outcomes with their it initiatives (brown 1997). currently, management faces some real dilemmas with respect to it. firstly, for competitive and political reasons, organisations can rarely exercise a choice not to invest substantially in it, even when economically they cannot find sufficient justification, and current evaluation practice cannot provide strong grounds for making the investment. secondly, as it infrastructure becomes an inextricable part of the organisation’s processes and structures, it becomes increasingly difficult to separate out the impact of it (both positive and negative) from that of other assets and activities. thirdly, it would appear that comparatively few senior executives feel that they understand it adequately, despite high levels of expenditure (willcocks and lester 1997). the conclusion must be drawn, therefore, that despite misgivings about return on investment, senior management continues to feel pressured into significant investment in it (mckague 1998). a number of reasons can be posited as to why there are concerns and perceptions of an inadequate rate of return on investment in it. firstly, it could be that there has been an inappropriate investment in and use of information, is and it in organisations, and hence concerns about the value of such investments. one often cited example of this stems from a failure to link is/it investments with business objectives and strategy initiatives (edwards et al. 1995, hochstrasser and griffiths 1991). alternatively, it could be symptomatic of a lack of, or ineffective, business and/or is/it planning. over time, a failure to achieve alignment of is/it strategies and business strategies could be argued to contribute to disappointing perceptions of it’s contribution to business performance. australasian journal of information systems vol 11 no 2 may 2004 15 secondly, it could be that current evaluation processes are either inadequate (or non-existent in some organisations), or that inappropriate evaluation techniques are being used (willcocks and lester 1997). perhaps a lack of confidence in the tools available leads to less than satisfactory practices. thirdly, it may also be that an inadequate rate of return on it investments arises because there are inadequate managerial procedures put in place to ensure the realisation of benefits from is/it (ward et al. 1996, remenyi et al. 1993). expected benefits are nearly always identified pre-investment for new systems and technology, but rarely are proactive behaviours adopted and changes made to support the post-implementation realisation and evaluation of these anticipated benefits (thorp 1998). fourthly, it may be symptomatic of inadequate it governance structures and processes that leads to disappointing outcomes, and perceptions of a ‘gap’ between it and the rest of the organisation (ward and peppard 1996, peppard and ward 1999). arguably, there are at least four key issues which will impact upon perceptions of the value of it investments: • that appropriate levels of business and is/it planning are undertaken, with the express aim of ensuring that proposals and priorities for it investment are aligned with, and directly support the achievement of corporate visions, strategies, and objectives; • that wide-ranging, qualitative and quantitative evaluation procedures and techniques to assess performance on a range of measures are adopted throughout the life cycle of is/it, and that the outcomes of this evaluation are actively fed into managerial decision making and action about on-going investment in that is/it; • that organisations implement explicit procedures to ensure that adequate pre-investment consideration of benefits anticipated from is/it is undertaken, and more importantly, that post-implementation of that is/it, procedures are put in place to deliberately ensure that anticipated benefits are actively realised and managed over time, • and that organisations implement an appropriate it governance framework and sound it governance practices to monitor and oversee the delivery of business value from it. this paper describes a study what examined the role and nature of the processes of is/it planning, evaluation and benefits management in contemporary organisations. the study also focused on the intersection between such processes. we would argue that this mix of planning, evaluation and benefits management is a vital part of it governance, as each of these components adopts a somewhat different (albeit important) focus on the other. changing notions of it and business value a review and analysis of the literature on the connection between investment in it and the delivery of business value from those it investments reveals some interesting and important evolutions in thinking about these important issues. while early thinking tended to reflect a view that investment in it would automatically deliver benefits to the organisation (this point is alluded to in the arguments of ward and peppard (2002), particularly in the early chapters of their book), more recent arguments seem to suggest a viewpoint that an integrated set of processes of planning, evaluation and benefits realisation are necessary to create the conditions for business benefits to be realised from it investments. arguably, this set of processes constitute the basis for a processual framework to support the governance of it. in his 1994 paper, earl outlined a progression of increasingly mature and sophisticated thinking with respect to it utilisation in organisations. the essence of earl’s argument is captured below in figure 1. australasian journal of information systems vol 11 no 2 may 2004 16 investing in it will bring business benefits. investing in it, and making necessary business changes, will bring business benefits. establishing clear business objectives & then investing in required it (and other resources) to achieve those objectives will bring business benefits. figure 1: increasing sophistication with respect to it earl (1994) seemed to be arguing for a move from the “it is good” mindset, to one that recognised (and practiced) the need for it investments to be derived from clearly articulated business need(s). indeed, it could be argued that this type of thinking underpinned much of the work with respect to information systems planning (isp) that occurred during the early 1990s (ward and peppard, 2002). improving isp was thus viewed as a serious concern for non-it and it managers in industry (galliers et al. 1994), and much of the focus of isp was in successfully achieving alignment between business imperatives and it investments. methods, tools and techniques were articulated to support this focus (see ward and griffiths 1996, tozer 1996, earl 1996, for example). while there was a deal of sophistication with respect to the argumentation and approaches articulated, there seemed to be an assumption implicit that desirable outcomes would be achieved if only alignment could be achieved. thus, in terms of figure 1, moving beyond stages 1 and 2, and embracing the thinking and actions implied by stage 3, seemed to be a way of overcoming disappointments with respect to it investments, as a failure to achieve satisfactory linkages between business and it initiatives has been cited as a contributing factor to a perceived lack of business benefits from it (edwards et al. 1995). an overlooked factor here may have been the evaluation of it investments. for the purposes of this discussion, a managerial perspective is adopted in defining it evaluation as “about establishing by quantitative or qualitative means the worth of it to the organization” (willcocks 1992). concerns have been voiced which suggest that the most frequently used approaches for it evaluation such as an accounting-based and narrowly focused cost-benefit analysis, may be unsuited to application to some it projects, and hence may fail to reveal benefits that have been derived from a particular investment (willcocks and lester 1997). in addition, some research indicates that formal it evaluation processes occur all too infrequently in many organisations (farbey et al. 1993), that formal evaluation is too often limited to project management-type measures of success (willcocks and lester 1997), and that inadequate or no evaluation is carried out in a number of cases (farbey et al. 1993). one difficulty with all evaluation is that while it may be helpful, indeed essential, to the identification of costs and expected or perceived benefits from a particular perspective, it does little to implement processes and procedures to ensure the management and realisation of those benefits over time. hence we see the emergence of benefits management approaches which typically institute procedures to ensure the realisation and management of expected benefits throughout the life cycle of an it investment (remenyi et al. 1993). benefits management approaches excel at identifying and managing the achievement of benefits but have few explicit means for linking these procedures to on-going decision making about further investments needed for modifications and enhancements, or actions to terminate, divest or outsource the investment, for example. thus, australasian journal of information systems vol 11 no 2 may 2004 17 consideration of figure 3 leads to a conclusion that there is a need to bring together evaluation and benefits management into an integrated, seamless approach to thinking and acting with respect to it in organisations. whereas it evaluation is concerned with methodologies and processes used to measure the costs and the potential and/or achieved benefits from is/it investments, benefits management is concerned with the management and delivery of actual is/it benefits to the organisation. however, there is a need to merge or meld these two approaches into a single and effective evaluation and benefits realisation approach, in order to reduce the inconvenience of going through the separate processes for evaluating, and managing and realizing the benefits from is/it investments (mckay and marshall 2000). invest ing in it will bring business benefit s. invest ing in it, and making necessary business changes, will bring business benefit s. est ablishing clear business object ives & t hen invest ing in required it (and ot her resources) t o achieve t hose object ives will bring business benefit s. planning it requirement s based on business imperat ives & t hen subject ing t hose invest ment s t o on going, mult i facet ed evaluat ions will enhance management of it, and hence bring business benefit s. planning it requirement s based on business imperat ives & t hen subject ing t hose invest ment s t o a rigorous process of analysing and managing benefit s and associat ed organisat ional changes will ensure t he realisat ion of benefit s for t he organisat ion. planning it requirement s based on business imperat ives & t hen subject ing t hose invest ment s t o on going, mult i facet ed evaluat ions linked t o proact ive benefit s realisat ion & organisat ion change processes will ensure appropriat e invest ment s and governance of it invest ment t o deliver business benefit s. investing in it will bring business benefits. investing in it, and making necessary business changes, will bring business benefits. establishing clear business objectives & then investing in required it (and other resources) to achieve those objectives will bring business benefits. planning it requirements based on business imperatives & then subjecting those investments to on going, multi faceted evaluations will enhance management of it, and hence bring business benefits. planning it requirements based on business imperatives & then subjecting those investments to a rigorous process of analysing and managing benefits and associated organisational changes will ensure the realisation of benefits for the organisation. planning it requirements based on business imperatives & then subjecting those investments to on going, multi faceted evaluations linked to proactive benefits realisation & organisation change processes will ensure appropriate investments and governance of it investment to deliver business benefits. figure 2: integrating planning, it evaluation and benefits management thus, the key to effective investment in is/it that is acknowledged to deliver value to an organisation over time is an integrated programme of is/it planning, evaluation and benefits management that is embedded in the day-to-day routines and rituals of the organisation (mckay and marshall, 2000). such an integrated cycle of activities should not only assure sensible and rational commitments to is/it initiatives, but also assure that such commitments remain viable, worthwhile and relevant. the authors would argue that for effective investment in is/it, a well-integrated and effective set of planning, evaluation and benefits realisation processes is necessary (see figure 3). indeed, such a set of processes is central to good is/it governance. australasian journal of information systems vol 11 no 2 may 2004 18 planning/alignment evaluation benefits * are we deriving the maximum benefit for the organisation? * are we addressing change management issues? processes training trading partners job redesign * where are we headed? * what are our objectives? * how can is/it support / enable achievement of objectives? * are we proposing coherent, strategic investments? * are we aware of the “value” of our it investments at certain critical points in the life cycle? * can we establish a realistic business case? * are we getting things well done? figure 3: broadening considerations of the value of it research method and design given the discussions and arguments in the previous sections, the overarching aim of this research was to establish the veracity or validity of the model presented in figure 3. thus, it was planned to ‘test’ the model to see whether empirical data would support such a framework. the research reported in this paper is part of a larger study into it governance processes and practices in australian organisations. six organisations form the basis for the discussion in this paper. the letters a through to f are used as pseudonyms for the company’s involved. they were selected for two main reasons. firstly, all six organisations, by their own admission, have gone through a ‘sea change’ with respect to their it management and governance practices in recent years. through their own admissions, the cios interviewed conceded that their organisation had undergone transformational change with respect to their arrangements for the exploitation of organisation-wide it capabilities, repositioning themselves from being essentially techno-centric (the it function is about technical things and does not know too much about the business) to business-centric (the it function is well integrated with the strategic business units across the organisation and is considered a key business partner on strategic initiatives, of which it will form an important part). these cios also believed that the delivery of business value from it was now more certain, and had improved dramatically in recent times. none felt that they were “perfect”, but these six cios expressed certainty that the business contribution of it had improved, and they now felt they had the support and confidence of other senior executives1. secondly, they were selected on the basis of availability and accessibility. the research was pursued via qualitative case studies (benbasat, et al. 1987, cavaye 1996). the chief information officer (cio) was interviewed in each of these six organisations, all large, established, successful companies within australia’s top 500 companies (baker et al. 2002)2. although this may seem a relatively small sample from which to generalise findings, this number does fit within the guidelines established by eisenhardt (1989), who recommends a sample of between 4 and 10 for in-depth qualitative case studies. a diverse range of industries from which the organisations were drawn was included to enhance the generalisability of the findings (eisenhardt 1989). data collection was primarily through semi-structured interviews (darke et al. 1998), each 1 subsequent informal conversations with selected executive level managers in these organisations supported these assertions by the cio. 2 in fact, the six organisations included in this paper were all within the top 35 companies in australia. australasian journal of information systems vol 11 no 2 may 2004 19 lasting between 60 and 120 minutes, and through analysis of internal corporate documents (strategic plans, it strategic plans, policy statements and so on). in some cases, particularly when executives were based outside of melbourne, telephone interviews were used in place of face-to-face interviews. all interviews were recorded, and the researchers took notes during the interviews. these recordings were then transcribed, and analysed for emergent issues, patterns of interest, and so on (darke et al. 1998), with particular attention paid to the cios thoughts and statements about it management and it governance processes and issues. our framework for ensuring the delivery of business value form it (see figure 3) was used as the basis for our interpretations of the governance activities in each organisation. interviews were conducted between august and october, 2002. there is a risk when using a relatively small number of cases that results reflect idiosyncrasies of the organisations involved. however, attempts were made to identify common patterns and trends across the cases and to use that as the basis of any generalisations (orlikowski 1993). the data was analysed in the first instance independently by both researchers on a case-by-case basis. categorisations were made of the transcribed data on the basis of the framework for effective it governance and the key it governance processes. the two views were then merged, with care taken to adequately deal with differences in opinion and perspective. in the final stage, we undertook a cross-case comparison, thus identifying common trends, patterns and issues of interest in terms of the it governance processes and behaviours of the six organisations under study. broad demographics of the companies involved are detailed in table 1 below. where no information was offered or made available, a blank has been left in the table. company a b c d e f industry retail pharmaceutical manufacturing financial services leisure/ entertainment food & beverages operating budget $100m $4m $35m $55m $50m $52 it as % of capital expenditure 12% 5% 7% 60% 30% capital expenditure $80100m $4m $7m $48m $15 $10m years as cio 3 12 6 4 5 3 table 1: company demographics findings and discussion analysis of the interview transcripts reveals broad support for most aspects of the proposed model in figure 3. while the specific activities, processes and structures to oversee and manage the acquisition of it varied from organisation to organisation, in all six organisations included in this research, there emerged a pattern of planning, evaluation and benefits management, which broadly can be categorized into five main categories or integrated phases. each of these is briefly discussed below. included in the discussion are characteristics of each phase that are regarded as important features of making the phase concerned an effective process or activity. phase 1: building the business case in all the organisations included, cios were very clear on the need for any proposed it investment to be demonstrably supporting of business goals objectives, and specific business initiatives. they were all very clear on the need for a compelling business case to be articulated before any it investment would even be contemplated. d: our investments today are targeted to support our strategic imperatives – if their strategic imperatives are right then they’re the things that count … we’ve got to check the allocation to australasian journal of information systems vol 11 no 2 may 2004 20 [it] investment is consistent with our strategy which is consistent with our strategic imperatives which is consistent where we want to play in the marketplace in terms of getting a larger share of the business that’s profitable f: well that depends on the strategy. we go to the business, and say ‘what is it that you’re trying to do with your business from a strategic point of view that you need our assistance with or that you say you need an it investment in relation to?’ and then each of those will then have to be presented with a business case in all organisations, the ideas for it investments in most circumstances originated either from the strategic business units (sbus), or from joint interaction between it and the sbus, but did not arise from the it department on its own. the only exception we discovered to this was in the case of it infrastructure investment proposals, where this was more likely to have originated from within the it department, a finding supported by weill and woodham (2002). in initiaves emanating from sbus, the it department is involved in the very early stages in the development of initial ideas. thus, the initiation of it investments is now regarded as a business responsibility, and the development of the early business case is seen also as a responsibility of the sbu, although the it department does provide support and assistance in this process, particularly in terms of what is feasible, practicable, and ‘do-able” in terms of the organisation’s current it infrastructure and it skill set. d: we have business project managers or sponsors who put the business cases up…they’re the ones who have said ‘if i get to that cost, that functionality, i’ll get that revenue or that cost cut’ e: there’s an informal process that happens …and that’s really just discussion between product managers deciding that they want innovation – there would be informal discussions with people in it as to what the technology for that will be – and then it builds into a planning paper – so it might be an initial feasibility paper – which then is teased out into typically more detailed business requirements paper – then around about that stage of the process a capital request would be developed…it’s discussed with the steering committee but it’s really done at a different level – the plan is developed within the planning process within the divisions the output of this first phase is an initial business case for a specific it investment proposal, which has been through some preliminary identification of anticipated costs and business benefits. sponsorship for the proposal from within the business unit must have been identified. phase 2: alignment and prioritisation once an sbu believes it has built a compelling business case and a sponsor for the project has been identified, typically organisations in our sample then require the proposal to be scrutinized by another entity or committee, usually associated with strategic planning. this might be some sort of steering committee, it might be conducted within the corporate strategy department, or it might be an independent business project office, but all reported some sort of higher level scrutiny of the sbu proposals. very large proposals could go all the way to the board for scrutiny. all the cios interviewed reported formal is/it planning processes and mechanisms in their organisations, with the typical planning cycle being 1 year (with a 3-5 year time frame). however, these plans and strategies were subject to more frequent reviews, often 3 or 6 monthly, where business goals, objectives, and priorities were re-evaluated, and then is/it initiatives re-evaluated in the light of possibly changing business initiatives. it was during this process of strategic planning and review that new proposals for is/it investments were considered, with very careful scrutiny of each proposal being conducted to ensure that any resultant it investment would be aligned and consistent with strategic business imperatives, and that responsibilities for costs and the delivery of business benefits was clearly identified. one cio interviewed described this scrutiny as “applying a blowtorch to investment proposals”. sensitivity to it projects failing to deliver on business benefits australasian journal of information systems vol 11 no 2 may 2004 21 and blowing out in terms of costs was evident, and attempts had been made in all organisations in our sample to effect governance mechanisms to try to prevent costly (and damaging public) failures with it projects. a: what we’ve got is, in each area of our business, we’ve got business steering groups that, within the budget of that business group, will approve the projects – now they typically approve up to a project of about $1 million. we’ve then got an overall it steering group which is chaired by our chief executive and that would approve the larger projects. when they are over a certain size they would end up going to the board as well projects that survived this process were those that could clearly be shown to be aligned with business goals and objectives. successful initiatives were then prioritised on the basis of the extent to which they were seen to be directly related to key business initiatives, and according to their perceived ability to deliver value to the business. for large initiatives (say those project proposals in excess of $500,000), the organisations in our sample were on average willing to spend between $20,000 and $30,000 to get to the point of proceeding with or killing off the initiative. c: we are trying to push away from these projects being seen as it projects… what we now say is, what’s the business’s new strategy – this is what we are trying to achieve – oh and by the way we might need some it to help us with that – rather than it coming in saying ‘if you get sap you’re going to be able to make all these business changes…’ so there is a very different approach to it…what happens is that even when something like that gets to board level – it’s not being pushed forward as an it project– so it becomes part of the business project – not an it project and that’s where the difference is i think …that’s not to say that it isn’t going to be out in the industry looking at potential solutions and opportunities and all of those sorts of things – but they need to always be done with a business opportunity in mind not a technology opportunity f: what is it that you’re trying to do with your business from a strategic point of view that you need our assistance with or that you say you need an it investment in relation to’ and then each of those will then have to be presented with a business case phase 3: evaluation at this stage, successful projects were typically exposed to more rigorous cost benefit analysis and the like. thus, although the projects surviving to phase 3 were seen as aligned with business strategy, yet more rigorous assessment of the costs and the likely benefits was considered necessary. typical of the sentiments expressed are the words of one manager who said, “we don’t like surprises”. the cios were very aware of the apparent ease with which it projects could get out of control, and hence were careful in conducting rigorous investigation and analysis to prevent this from occurring wherever possible. again, it was not uncommon for very large projects (say, more than $1,000,000) for perhaps $200,000 $300,000 to be spent in refining and firming up in terms of the project scope, its costs and its likely benefits. e: it investments…not all of them actually occur because that’s the stuff that goes in the planning the board signs off on – but then on a case by case basis you go through this sort of more detailed analysis process, and at that stage the business and the it guys have go to – you’ve got to commit in – you’ve got to say – ‘i will do this within this cost by this date and achieve this benefit’ – so if it doesn’t stand up to that more rigorous test – then it doesn’t go ahead at this phase, risk profiles were typically developed for all projects, as risk management was seen as one key aspect of successful project implementations. it was not generally the case that high risk projects were given lower priorities or abandoned, but rather that higher risk projects needed to deliver greater benefits to the business and hence additional efforts to ensure the realisation of those benefits was essential. australasian journal of information systems vol 11 no 2 may 2004 22 a: it’s basically what’s the return we are predicting versus the predicted cost with an allowance in there for the risk factor. we would expect a higher rate of return for a high risk project that we’re not as competent of delivering all the cios interviewed expressed concerns with their ability to deal with intangible business benefits. generally speaking, they did not try to allocate dollar values to intangibles, but this did mean that there was still considerable uncertainty as to the value of particular initiatives. concerns were also expressed about the time of realizing benefits: costs were generally easier to identify and were largely experienced early in the life of the system, whereas benefits often took some time to be realised. another factor in the delivery of benefits had to do with business change: some cios felt that the during the 6-12 months typically required to deliver a project, business change had occurred, reducing the possibility of benefit realisation as business assumptions had changed. the cios also commented that it was often difficult to attribute benefits solely to it: thus benefits were often derived from business change initiatives, of which it was just one component, and therefore being specific about the benefits realised from just the it components of that change were extremely difficult to articulate with any precision. the usual problems of enthusiastic project sponsors overstating benefits and understating costs were evident in most of the organisations interviewed. c: because you do get people that are champions about things or picture certain things, and they are obviously very enthusiastic about what their baby can deliver if you like. and i think that human nature will tend to overestimate the benefits…i can really only think of one major project – there’s lots of little things – but one major project where the outcome was a heck of a lot better than expected – anything else is either at or more typically below the expectations c: it is a very volatile kind of environment that you work in – and therefore to try and second guess where things are going to be in a year’s time is absolutely almost impossible – and that’s why i think they come in at below expectations – and you can’t stop the world while you’re delivering a project projects at this stage were still subject to being terminated. many of the cios stated that if there were still considerable doubts about the ability of the organisation to implement a viable system that delivered business value, then they would have no hesitation in cancelling the project at this stage. phase 4: system acquisition phase 4 saw the acquisition of the system, with cios facing the dilemmas of the build vs buy dichotomy. thus systems could be sourced through an in-house development project, but more often, it involved procuring packages and customizing them to suit in-house business processes. customisation was sometimes done in-house, but at other times was left to the software vendor, or recognised consultant, to do. all six cios reported they were under pressure not to build systems in-house from scratch. in five of the six organisations interviewed for this research, the ‘buy’ model was the preferred, but not sole, approach. the reasons stated for this was generally packaged software was seen as a cheaper option, it was seen as more predictable, and it generally took less time to implement a working system. in the sixth organisation where the norm was to develop systems in-house, and in the other five organisations in those cases where they decided to build in-house rather than buy software, there were a number of compelling reasons why this was done. firstly, cios reported that peculiarities of their business sometimes meant that purchasing systems was not appropriate. many could identify business circumstances and processes that meant that available software packages were a very poor fit. for example, one cio had implemented an erp package, but found that the production and manufacturing module simply did not suit their industry-specific production processes, meaning that their production control system and the like were developed in-house, or alternate modules were sourced from another software company with interfaces and integration australasian journal of information systems vol 11 no 2 may 2004 23 issues dealt with in-house. others, particularly those associated with the finance and insurance sectors argued that american packages offered an inadequate fit between the australian business environment, specifically australian tax laws and regulatory requirements, and the functionality embedded in the software. the third reason stated for in-house development being preferred occurred when there was heavy it involvement in the product/service mix of the company. for example, in the insurance and financial sectors, cios reported that it support and enablement could no longer be separated out from the product and service being offered. in such cases, in-house development was often the preferred approach. e: we have a portfolio of contract arrangements. we don’t have any full outsourcing, but we have people that provide various hardware services and hardware. we have organisations that from time to time we use to undertake developments, and we buy software products…generic ones, and also applications that might be quite specialised to an industry. we do a lot of development in-house and in part the reason for that is we operate in a specialised area of application where there really isn’t a mature market of packaged software…there’s not xxxs all around the world running the services we do and even though there is a fairly large population of them, there’s an enormous variety in product and also regulatory obligations…so it’s hard to get a generic package irrespective of the build or procure decision, on-going evaluation and scrutiny of the acquisition process was the norm during phase 4. all organisations reported rigorous governance mechanisms for their major it projects. the specifics varied, but essentially involved either regular reviews by a steering committee, or involvement of a business project office that oversaw the progress of each it project. while the evaluation conducted in phase 3 had typically sorted out suspect projects, it was not impossible for a project to be terminated at phase 4 if the entities charged with overseeing the projects felt that costs were escalating, or business benefits becoming less likely to be realised, or the business need was changing, and so on. d: we have a gating [evaluation] process where people get their money to get through the gates to do the project, so we have gate 0 to gate 1 which is really straight analysis…and there will be an it business solutions manager involved in that…gate 1 to gate 2 is really high level analysis – not so much high level but it’s the step 1 analysis and so on…gate 2 to gate 3 is detailed analysis and so on which just takes that a bit further. there’s gates to get through all along the way. f: we set milestones for each project then re-assess the investment on an ongoing basis…we’re very very heavy on strict project management many of the cios attributed these aspects of governance of reducing runaway projects. the cios acknowledged that not all projects were implemented on time and within budget, but did feel that careful management of the project from a business perspective has delivered much better results for their organisations. most cited now that between approximately 60% and 80% of project were on time, and between about 70% and 80% were within budget, both figures having shown substantial improvements in recent years. e: in general, we’ve been pretty good in terms of delivering things on budget compared with i think a lot of organisations, and i would say probably in the last 6 years we haven’t had disasters you know…we haven’t had the disastrous erp project like organisations where you’re running 30-50% over budget or 100%, or you failed to deliver at all. we haven’t had any of that, and i think it’s actually…in part it’s a reflection of how aggressively disciplined the whole investment process is australasian journal of information systems vol 11 no 2 may 2004 24 phase 5: implementation some cios described a rigorous process of post-implementation reviews, driven by the business sponsors of the project, with some inputs from it. part of this was a review of typical measure of project management success (on time, within budget, to specification). by contrast, some managers felt that extensive post-implementation reviews consumed resources which were needed elsewhere, and another admitted that while important, post-implementation reviews were not routinely done. however, many of the cios interviewed typically recognised the perhaps more important requirement to assess the delivery of business benefits (i.e. were the systems, once implemented, actually realizing the benefits identified pre-investment for the business?). some of our sample organisations had proactive benefits management processes in place, while others conceded that this was important and an area where they needed to improve. d: we do a post-implementation review and later on you do a benefits audit – that’s how i expect it however, some of the managers were concerned about the resources required to undertake such a benefits realisation process. take this quote, for example: a: we certainly do a post implementation review of all our projects but if the question is more around benefits, do we go back rigorously a year later or whatever and say ‘okay compared to what we predicted how did we do?’ i’ve got to say we haven’t been strong on that as i think we should’ve been but the other factor of course is that there’s always a lot changing in the business anyway, so it’s very hard to attribute an improvement to one specific project…it tends to be look we think we’ve got 80% of the benefits from what we’ve already done – rather than put a whole heap of extra effort into chasing proportionately less benefits, let’s stop, and accept that there is some weaknesses in what we’ve done but there are bigger opportunities elsewhere the issue being grappled with here is that as cios, they felt torn between the potential to derive greater benefits from the existing investments, as opposed to diverting resources to exploiting other it opportunities. interestingly, our cios tended to adopt the satisficing position (i.e. that delivering, say 80% of expected benefits was probably good enough and that the resources consumed in trying to achieve 100% or more would be better diverted elsewhere). conclusion the main premise on which this paper has been centred is that organisations may expect better results from their it investments and capabilities if there are good it governance processes and mechanisms in place. part of the governance processes is the development of procedures and structures which support an integrated cycle of strategic is/it planning and alignment, evaluation and benefits management. organisations in this study were well aware of the need for this cycle, but acknowledged that their attention to the realisation of benefits could be improved. the rigour of governance associated with planning and alignment, and evaluation was attributed with reducing disappointing outcomes with it acquisitions. while accepting that more could be done in terms of the delivery of benefits, the volatile business environment, changing business requirements, and complex, multiple organisational change initiatives of which it formed one part were all implicated in making it difficult to realise all anticipated business benefits from specific it investments. australasian journal of information systems vol 11 no 2 may 2004 25 references agarwal, r. and sambamurthy, v. (2002) principles and models for organizing the it function. mis quarterly executive, 1(1): 1-16. baker, j., baker, l., nadi, s., ng, p. and kaseman, p. (2002) the top 500. business review weekly, april 24-may 1, 2002:64-65. cavaye, a.l.m. (1996) case study research: a multi-faceted research approach for is. information systems journal, 6(1996): 22-242. darke, p., shanks, g. and broadbent, m. (1998) successfully completing case study research: combining rigour, relevance, and pragmatism. information systems journal, 8(1998): 273289. eisenhardt, k.m. (1989) building theories from case study research. academy of management review, 14(4): 532-550. earl, m.j. (1994) putting information technology in its place: a polemic for the nineties. in r.g. galliers and b.s.h. baker (eds.) strategic information management: challenges and strategies in managing information systems. butterworth heinemann, oxford, pp76-90. earl, m.j. (1996) an organisational approach to is strategy-making. in m.j. earl (ed.) information management: the organizational dimension. oxford university press, oxford. edwards, c., ward, j. and bytheway, a. (1995) the essence of information systems. 2nd ed. prentice hall, london. farbey, b., land, f. and targett, d. (1993) how to assess your it investment: a study of methods and practice. butterworth heinemann, oxford. galliers, r.d., merali, y. and spearing, l. (1994) coping with information technology? how british executives perceive the key information systems management issues in the mid 1990s. journal of information technology, 9(3). hochstrasser, b. and griffiths, c. (1991) controlling it investment: strategy and management. chapman & hall, london. mckague, a. (1998) time has come to deal with issue of it failures. computing canada, june 8. mckay, j. and marshall, p. (2000) the it evaluation and benefits management life cycle. in w. van grembergen (ed.) information technology evaluation methods and management. idea publishing group, hershey. peppard, j. and wrad, j. (1999) ‘mind the gap’: diagnosing the relationship between the it organization and the rest of the business. journal of strategic information systems, 8(1): 29-60. peters, g. (1994) evaluating your computer investment strategy. in l. willcocks (ed.) information management: the evaluation of information systems investments. chapman and hall, london. remenyi, d.s.j., money, a., and twite, a. (1993) a guide to measuring and managing it benefits. 2nd ed. ncc blackwell, oxford. shane, b., lafferty, p. and beasley, t. (1999) im/it governance framework. optimum, the journal of public sector management, 29(2/3): 30-35. strassmann, p.a. (1997) the squandered computer; evaluating the business alignment of information technologies. information economics press, new canaan, connecticut. thorp, j. (1998) the information paradox: realizing the benefits of information technology. mcgraw hill, toronto. tozer, e.e. (1996) strategic is/it planning. butterworth heinemann, boston. ward, j. and griffiths, p. (1996) strategic planning for information systems. 2nd edition wiley, chichester. ward, j. and peppard, j. (2002) strategic planning for information systems. 3rd edition, wiley, chichester. australasian journal of information systems vol 11 no 2 may 2004 26 ward, j. and peppard, j. (1996) reconciling the it/business relationship: a troubled marriage in need of guidance. journal of strategic information systems, 5(1): 37-65. ward, j., taylor, p. and bond, p. (1996) evaluation and realisation of is/it benefits: an empirical study of current practice. european journal of information systems, 4(1996): 214-225. weill, p. and woodham, r. (2002) don’t just lead, govern: implementing effective it governance. center for information systems research working paper no. 326, sloan school of management, ma. willcocks, l. (1992) evaluating information technology investments: research findings and reappraisal. journal of information systems (1992)2: 243-268. willcocks, l. (1994) introduction: of capital importance. in l.willcocks (ed.) information management. chapman and hall, london. willcocks, l. (1996) introduction: beyond the it productivity paradox. in l.willcocks (ed.) investing in information systems: evaluation and management. chapman and hall, london. willcocks, l. and lester, s. (1997) assessing it productivity: any way out of the labyrinth? in willcocks, l., feeny, d.f. & islei, g. (eds.) managing it as a strategic resource. mcgraw-hill, london. microsoft word ajisfinal 12 2 marina.doc ajis volume 12 no. 2 may 2005 121 grounded theorising applied to is research — developing a coding strategy bruce h. rowlands school of computing & information technology griffith university b.rowlands@griffith.edu.au abstract this paper provides an example of developing a coding strategy to build theory of the roles of methods in is development. the research seeks to identify and understand how system development methods are used in an is department within a large australian bank. the paper details a theoretical framework, particulars of data collection, and documents an early phase of analysis – data reduction and the generation of an initial coding scheme. guided by a framework to study the use of methods, the analysis demonstrates the framework’s plausibility in order to develop theoretical relationships with which to develop a grounded theory. key words— grounded theorising, coding schemes, systems development methods, interpretive case research. introduction this paper contributes to the discussion about developing a coding strategy for qualitative research in information systems. the paper provides an example of developing a coding strategy to build theory of information systems development. the approach to building theory makes use of a method-in-action framework by demonstrating its plausibility to illuminate an under-researched aspect of information systems practice – how systems development methodologies (sdms) are used in organisations. to contribute to the scientific documentation of method use, this paper provides aspects of a grounded study of the social and organisational issues surrounding the use of development methods in a large australian financial institution. this paper addresses methodical issues about the conduct of using grounded theory (gt) for inductively generating theory. while having its antecedents in sociology in the mid 60s, in the field of is there has been a growing application of gt, but limited methodological guidance for builders of theories, and few exemplars of conducted research. therefore it is contended that there remains a paucity of published accounts of the application of gt, and in particular – approaches to coding in the field of is. this paper makes a contribution to such a deficiency. the remainder of the paper is organised as follows. firstly, based on a review of the literature, the paper defines the features of a systems development methodology (sdm) and makes a case for considering the social and organisational context of method use. second, guided by the work of fitzgerald et al (2002:12), a framework is presented allowing focus on practitioner perceptions of the roles of development methods. next, the research design, data collection and data analysis is discussed. the research procedures section discusses alternate approaches to coding with gt and provides more details of the ‘how to’ of coding and grounded theorising. the paper ends with a positive conclusion vis a vis the plausibility of the framework to illuminate an isd practice, and also a call for further research in this area. a review of related research on systems development methods for the purpose of this paper, a systems development methodology (sdm), as defined by ajis volume 12 no. 2 may 2005 122 iivari et al (1998), is a codified set of goal-oriented procedures which are intended to guide the work and cooperation of the various parties (stakeholders) involved in the building of an information systems application. typically these procedures are supported by a set of preferred techniques and tools, and guiding principles. furthermore, the terms methodology and methods are used interchangeably and have become blurred by common usage, and now seem largely fused concepts in information systems parlance. sdms are promoted as a means of improving the end product of the software development process by specifying the activities and the documentation to be produced (avison & fitzgerald, 2003:20). the alleged benefits are standardisation of the process and consistency from one development project to the next. the assumption of desirability has been reflected in research that has emphasised methodology design, the comparison of technical features, and methodology selection. however, the research literature that emerged in the 1990s suggests that methodologies have not always been used, and if used, not always in the way they were intended (fitzgerald et al, 2002:100). there is a tension between the technical intent of methodologies (or methods) and how methods are actually used. recent empirical evidence suggests that methodologies are not used for technical reasons alone, but are in practice affected by organisational and behavioural factors (sauer & lau, 1997; fitzgerald 1998; truex et al, 2000;). in research addressing the issue of methodology use, fitzgerald, russo & stolterman (2002:12) proposed a framework to investigate the complex nature of systems development and the use of methods in practice. the framework has been constructed on the basis of detailed investigation of the systems development literature, and empirical research (fitzgerald, 1997, 1998; russo & stolterman, 2000; wynekoop & russo, 1993, 1995, 1997). despite the identified divergence between the intent of methodologies and concrete practice, researchers still have very little descriptive understanding of how practitioners use methodologies in their day-to-day work. the work that has been published is limited in its ability to consider the complex social and organisational context of methodology use. for instance, wynekoop & russo (1997) in a survey of the existing literature on sdms revealed that over half of the 123 research papers examined consisted of normative research in which concept development was not based on any empirical grounding, but merely on the authors’ speculations or opinions. most field research on methodology use (with the noted exception of madsen & kautz, 2002; fitzgerald et al, 2003; kiely & fitzgerald, 2003; hansen et al, 2003; and vidgen et al, 2004) has a method engineering and tool orientation that brings focus to structural aspects of the methodology, but without any consideration of organisational issues. few studies have been conducted in order to identify how sdms are selected or adapted, or how they are used. there also appears to be few practice demonstrations or case studies illustrating the method-in-action process with the noted exception of fitzgerald et al (2003) and vidgen et al (2004). clearly, the lack of process-oriented research in this area precludes a full understanding of how sdms are used. researchers have long called for research on methodologies in real life organisational situations (iivari & maansaari, 1998) and they continue to do so (beynondavies & williams, 2003). with the call in mind, and in the absence of a validated model of methodology use, the paper applies the fitzgerald et al (2002) method-in-action framework as a conceptual lens to understand methodology use within a single organisational setting. the framework is depicted in figure 1, and each element is explained briefly in the next section. a framework for conceptualising methodology use this section examines the usefulness of a framework for studying method use. most qualitative researchers attempt to avoid prior commitment to theoretical constructs before gathering any data. yet, as discussed by whetten (1989), two different approaches may be taken, or combined. in the ajis volume 12 no. 2 may 2005 123 first, the researcher works within an explicit theoretical framework. therefore, a theoretical framework becomes a researcher’s first cut at making some explicit theoretical statements (miles & huberman, 1994:18). this approach is known as deduction. in the second, the researcher tries not to be constrained by prior theory and instead sees the development of relevant theory, propositions, and concepts as a purpose of the project. this approach is generally known as induction. in this example, both approaches were combined since the main intent was to study a relatively unresearched topic (the enactment of sdms in development projects) within the bounds of an already well-established research program (process of software development). while it is possible to be inductive, i chose not to ignore previous work in the field, and instead adopted a loose conceptual model built on previous research that was a conceptual advance on the literature. the model of method use, as proposed by fitzgerald et al (2002), is comprised of six sets of issues. these issues were used to develop the initial coding scheme for the qualitative analysis of data (to be discussed in the research procedures section. the pivotal component of the model, the method-in-action process, is to be developed and presented in future research, and does not form part of this paper’s discussion. however, and this is a most important point, given that this study was aimed at theory building, not theory testing, the theoretical framework and model was used solely as a guide. it helped make sense of what occurred in the field, ensured that important issues were not overlooked, provided a set of tentative constructs to be investigated, and guided my interpretation and focus. the framework is briefly described in the following paragraphs. systems development methodologies can be commercial (sold or recognised outside a single organisation), home-grown (developed and used within a single organisation), or adapted (that is, tailored to suit the local contingencies of the task at hand). roles of method include two broad, but diametrically opposed categories of roles that methods can play in the development process. first, there are a set of rational or intellectual roles forming part of the conceptual basis and rationale behind the use of methods. these roles include, for example, the reduction of complexity of systems development, facilitation of project management & control, the division of labour, systematisation of development knowledge, and standardisation of the development process. these roles are countered (as alleged by fitzgerald et al, 2002:103) by a set of political roles that are more covert in nature. these roles include: professionalising is development work, help make is development more proactive in strategy formulation, a comfort/confidence factor, an audit trail, legitimacy factor, and a power base for a methodology champion. method-in-action is the actual use of the methodology and can be uniquely enacted by the developers. the unique enactment is reflected in the framework in fig 1 by the fuzzy, cloudy outlines. development context is represented by the cloud shape in fig 1, reflecting the fact that development and maintenance takes place in a unique real context. this context cannot be easily analysed to produce a neat and regular specification of user requirements. ajis volume 12 no. 2 may 2005 124 figure 1. a framework for isd method use (c.f. fitzgerald et al, 2002:90) developers is a term used in a broad sense to cover the multiplicity of stakeholders, systems users, analysts, designers, programmers, clients and problem owners. the developer is accorded a central role in the framework, thus reflecting the fact that it is people, not methods, who develop systems. according to the framework, the developer analyses the development context and uniquely enacts the method-in-action to develop an information processing system. information processing system: just as the development context is not always the same, the information processing systems being developed are not all alike. according to fitzgerald et al (2002:138), a number of families of systems can be identified, and depending on the different characteristics of each family, serves to affect the method-in-action that will be needed to develop them. in sum, the framework illustrates the nature of systems development, and some significant aspects and factors that need to be considered, including the concept of method-in-action. it needs to be restated that the framework is a conceptualisation of the systems development process, making it possible for researchers to reflect on isd as a complex process influenced by all the framework components and their interactions. research procedures this section justifies the use of grounded theory (glaser & strauss, 1967) when conducting process oriented research and describes a number of distinguishing features that characterise this approach – inductive, contextual, and processual – that fit with the primarily interpretive rather than positivist orientation of this research. development context developers method in action formalised method information processing system analyse develop enact may be basis of influence justify shapes • early analysis techniques • systems development life cycle • structured approach • data driven approaches: dssd, d2s2, information engineering • integrative approaches: ssadm, jsd • object orientation • ssm • historical perspective • early problems in development roles of method rationa l political reduction of complexity professionalise isd work facilitate project mngt help make isd dept more proactive division of labour comfort / confidence factor systematise devt knowledge audit trail standardisation of devt process legitimacy factor power base for method champion development context developers method in action formalised method information processing system analyse develop enact may be basis of influence justify shapes • early analysis techniques • systems development life cycle • structured approach • data driven approaches: dssd, d2s2, information engineering • integrative approaches: ssadm, jsd • object orientation • ssm • historical perspective • early problems in development roles of method rationa l political reduction of complexity professionalise isd work facilitate project mngt help make isd dept more proactive division of labour comfort / confidence factor systematise devt knowledge audit trail standardisation of devt process legitimacy factor power base for method champion ajis volume 12 no. 2 may 2005 125 reasons for using grounded theory techniques due to the lack of prior empirical research on the topic, a theory building research design was chosen, and sought to illustrate the framework’s capacity to illuminate a phenomenon in new or better ways. a grounded theory (gt) approach based on an in-depth case study of one organisation in the financial services industry was considered appropriate given the purpose was to develop an understanding of this under-researched phenomenon. i also chose grounded theory (gt) techniques to analyse my case study interview data because, according to strauss & corbin (1998), grounded theorising is well suited to capturing the interpretive experiences of it practitioners and developing theoretical propositions from them. in the same line of thought, an application of gt is appropriate when the research focus is explanatory, contextual, and process oriented (eisenhardt, 1989). similarly, gt has been effectively used in recent is research (galal, 2001; urquhart, 2001; golicic et al, 2002; douglas, 2003; hughes, 2004) to develop theory of is practice. the research design the first step in the research example involved stating the research questions and identifying a preliminary conceptual model that initially, told me where to look for relevant evidence. the role of loosely specifying a conceptual model prior to the conduct of any data collection is a major point of difference between the research adopted compared to a purely inductive approach. in fact, i chose this approach essentially because it provided a more “tighter” design, in keeping with the following contingencies. firstly, the literature provides a good prior acquaintance with the research problem; we know something conceptually about the phenomenon – in this case a bank of concepts providing at least a rudimentary conceptual model of the enactment process. there is a significant body of literature on factors influencing method use and theory on software process innovation, but not enough to house a theory of methodology enactment. i had some idea about how to gather the information and perhaps which questions to ask, and which incidents to attend to closely. as miles & huberman (1994:17) state, “not to ‘lead’ with your conceptual strength can simply be self-defeating”. yet, as these authors point out, no matter how “tight” the design is bounded, focused and organised, qualitative research designs are not copyable patterns or panaceas that eliminate the need for building, revising or ‘choreographing’ analytic work. according to miles & huberman (1994:17) tighter designs are a wise course providing clarity and focus for qualitative researchers worried about diffuseness and overload. the case the study focussed on an australian bank. the banking and financial services sector was chosen because of the extremely important role that it plays in the success of companies in this industry, and the bank selected has extensive experience and use of an in-house developed methodology. the selection of the case site was based on a combination of accessibility to the company’s it managers and project members, and interestingness – in the sense that the chosen bank is one of australia’s top 4 banks, and it’s it organisation is considered to be a leading player in providing state-of-the-art is solutions to customers. the paper uses the pseudonym the bank for the purpose of maintaining anonymity and confidentiality to the case study participants. the unit of analysis for the case study were it professionals (it managers and systems developers) within the systems support and systems development division. ajis volume 12 no. 2 may 2005 126 selecting the participants i made two major decisions in terms of selecting the participants. first, the population of interest were specified as it practitioners and project managers. the interviewees were purposely selected based on the researcher’s knowledge of the industry, and from discussions with industry figures. the second critical decision was to know when to stop adding participants to the study. one approach for deciding when to stop adding is to conclude the field research when theoretical saturation is reached (eisenhardt, 1989; strauss & corbin, 1998). this approach may be the ideal situation but is difficult for researchers (like grant recipients) who faced the real constraint of time schedules and funding. hence some researchers must develop alternate approaches to answer the question of when to stop adding interviews. an appropriate alternative stresses the importance of representing the variety found in the population rather than reproducing the proportions of characteristics found. this approach embodies the concepts of replication logic (yin, 2002). as suggested by smith (2000:78) in research at the organisational level, the researcher includes in the sample a variety of [responses] operating under different conditions to ensure the theory developed is robust and provides explanations of phenomena across a number of different settings. it is the variety of conditions found under replication logic that permit the generation of theory capable of explaining the diversity of situations typically found in organisations. following glaser and strauss' (1967) technique of theoretical sampling, three occupational functions within the bank were selected for their similarities as well as their differences. because the purpose of the research was to generate theory applicable to various occupational contexts, differences were sought in organisational type such as those working in development versus those working in a maintenance or support environment. these differences allow useful contrasts to be made during data analysis, which challenge and elaborate the emerging concepts. data collection instruments in my role as an outside observer, the semi-structured face-to-face interview was chosen as the primary data source, with observations and documents being minor sources of data. importantly, the reliance on multiple data collection methods increased the robustness of results through triangulation. the primary goal of the interviews was to elicit the respondent’s views and experiences in his or her own terms. as such, the research used the natural setting as the direct source of data and the researcher as the key data-gathering instrument. the data gathering task was to access other people’s interpretations, filter them through the researcher’s own conceptual apparatus, and then feed a version of the events back to others. next, the primary form of data collected – critical incidents, and each of the data sources used in this study are discussed. critical incidents are brief descriptions described by individuals about significant or key events that relate to a particular topic. this technique has been used widely in organisational research. miles & huberman (1994:115) cite the use of critical incidents and explained that ‘sometimes a researcher wants to limit an event listing to those events seen as critical, influential, or decisive in the course of some process’. several advantages accrued from using this method of collecting data. respondents were asked to talk about specific situations, events and people. the advantage was that the interviewee could focus and reflect on the described incident posed in the interview. a collection of these responses allowed the researcher to analyse meaningful data that was grounded in the actual experiences, needs and concerns of participants. interviews with it practitioners dealt with the following issues: reasons or motives for using the isd method; the conditions that shaped their use; and the enactment process itself. the average length of each interview was approximately 1 hour. interviews were taped (although one interviewee declined ajis volume 12 no. 2 may 2005 127 to be taped) and were later transcribed by a typist into a word-processed document. some respondents were interviewed a second time to follow-up on important issues that became evident in the data analysis phase. a total of 30 interviews were conducted with 25 informants from different projects and at varying levels in the organisation. the levels and project types of the respondents are shown in table 1. 16 interviews were carried out in the first round in december 2003 and a further 14 interviews in the second round in march 2004. only data from the first round of interviews are used in this paper. table 1. interviews by title within the bank, and project type job title project type or function project manager senior consultant consultant total support / maintenance 2 8 6 16 development 2 4 3 9 auxiliary management 3 2 0 5 7 14 9 30 development of the coding strategy this section describes how the data was coded and managed. two procedural issues were of great importance in starting the research: the initial definition of research questions; and the choice regarding a priori specification of constructs. each of these issues is examined in turn. first, the research questions provided the focus. the aim of the study was directed toward exploring and describing how and why the methods are used, and to develop process theory explaining the enactment process. in pursuit of this aim, three interrelated research questions were initially stated: q1. what issues and conditions influence the decision of it practitioners to use, modify or tailor isd methods? q2. what are the processes it practitioners go through when using isd methods at their work? q3. how can these processes of methodology enactment be depicted in a model? the research questions examined in the literature review provided a guiding focus to the research and permitted the specification of the kind of data to be gathered. this approach conforms with that of strauss & corbin (1998:47) who suggest that literature from the field be used not to develop hypotheses, but to stimulate theoretical sensitivity by providing concepts and relationships that are checked out against actual data. with respect to the second issue of using existing theoretical constructs to guide theory building, a loose conceptual model and its constructs were only used as a starting point. the conceptual model was intended to make sense of the interviews, ensure that possible issues were not overlooked, provide a set of constructs to be investigated, and guide the author’s interpretation and focus. however, as stressed by eisenhardt (1989), although early identification of possible constructs allows them to be explicitly studied in interviews, it is equally important to recognise that the constructs are merely tentative in the theory-building process. in this research, this was found to be true as new issues were identified during data collection that needed to be added to the analysis. ajis volume 12 no. 2 may 2005 128 analysis of data the approach to data analysis included two main steps: early steps in data analysis, and within case analysis. the early steps in analysis include use of the contact summary form for reviewing the interview, the development of a computerised database using the latest qsr qualitative research software (nvivo v2) specifically designed for storing, indexing, searching and theorising on qualitative data, the arranging and displaying of data in tables, and the development of a coding scheme to organise the data. within case steps involved detailed write-ups for each interview assisted by the identification of critical incidents. three analytical techniques adopted during each of these two teps are explained below. the first technique, combining data collection and analysis was the use of the contact summary form. this form was useful in revising the case interviews, and once transcribed, reflective remarks were recorded about the main issues identified, and initial thinking around the research questions. these remarks were ways of getting ideas down on paper and of using writing as a way to facilitate reflection and starting the analysis process. use of the contact summary form commenced analysis by presenting an overall evaluation of the case scenario. the initial analysis was performed manually, involving writing up the contact summary sheet with some further development of the coding scheme. the transcripts from each interview were then coded and analysed for emerging themes. a second set of techniques recommended by bazeley & richards (2000) highlighted data management issues, and the application of analytical software to the interpretive analysis of qualitative data. in this research, the process of analysis was assisted by and recorded in a developing database through procedures such as importing rich text files of transcribed interviews, setting up other links in which comments and reflections could be added, coding and sorting the interpretations; and text retrieval of selected instances into the body of the research report. the third technique was coding. while the data is being systematically collected (mainly through interviews), the researcher is confronted with volumes of rich text. a systematic way of handling the words was through coding. in this paper, a code is defined as an abbreviation or symbol applied to a segment of words (e.g. a sentence or paragraph) that captures the description of the phenomenon. coding, then, is labelling fragments of data by identifying key points, as opposed to coding by micro-analysis of the data, word-by-word, or line-by-line. the selection of key points accords with miles & huberman’s (1984:57) recommendation as a protection mechanism against data overload, and glasser’s (1992) concern of over-conceptualisation coding strategy coding is also a central activity in the process of generating grounded theory. since publishing their seminal article in 1967, glaser and strauss have parted in their views on the application of grounded theory. in terms of specifying the steps to be taken by a researcher in coding and analysing phenomena, strauss & corbin (1990) are more prescriptive, opting for a structured set of analytical steps and encourage the researcher to predetermine the general subject of inquiry before entering the site. glaser (1992), on the other hand, prefers an analytical method that is more general, and regards strauss & corbin’s (1998) analytical method of forcing rather than allowing emergence of theory. for reasons of a ‘tighter design’, the research followed strauss & corbin’s approach, where the author elected in advance to focus the observations and interview questions on a particular issue, such as methodology enactment. in terms of coding process, coding was then oriented around this issue, and a central concept is then sought to represent the nature and dimension of methodology enactment. ajis volume 12 no. 2 may 2005 129 as suggested by miles & huberman (1994:58) i created a provisional ‘start list’ of codes prior to fieldwork. most of the initial coding categories were drawn from the loose conceptual model (the fitzgerald et al framework), the list of questions, and key concepts the researcher brought to the study. to be consistent with the framework, the preliminary descriptive coding scheme developed in this study was divided into five broad categories: roles of method, formalised method, development context, developers & information processing system. the original list was then used to codify and extract the data from the transcript associated with the pilot interview. as a result of this process, i found the need to add new codes. this same format was carried through the entire data collection process (across the 1 st 16 interviews) and new open codes were developed for emerging themes. two coding techniques need to be discussed. in the first, coding takes place where transcripts are labelled by applying in vivo codes; meaning the literal word or phrase used by the respondent becomes the code. however, this research used an alternate approach where a set of pre-defined ‘start codes’ or researcher-constructed labels were applied to the text that best captured the description of the phenomenon. the development of the coding scheme was an on-going process throughout the transcription of each of the 16 interviews. 35 resulting codes within five major categories emerged from the analysis of the first round of interviews. a portion of the results of the revised coding scheme are presented and explained in appendix a, displaying the categories, and their coded components. discussion, conclusion & further research one case study of 30 interviews to establish the plausibility of the framework to establish start codes is enough. the findings are not suggesting that we can use the case as a definitive test of the framework. however, the case succeeds in establishing the plausibility of the fitzgerald et al (2002) theoretical framework by providing evidence of its capacity to provide start codes illuminating factors accounting for the use or otherwise of the systems development methodology. the derivation of start codes also provides initial evidence suggesting that the research proceed to the next phase of theory development where the method-in-action process and the derivation of axial and selective codes is specified further into a set of propositions and a graphical model. this is the first paper in a series of papers describing the process of researching methodology use in a large australian bank. for the author of this paper, the next phase will involve (i) analysing the remaining 14 interviews, and if the findings further support the plausibility of the coding framework; (ii) the research program will commence further steps of gt by generating selective codes, further categories and identifying a central theme; then (iii) the development of propositions and a substantive theory. in this theory-building phase, the researcher proposes to analyse the remaining set of interviews with the intention of generating a set of propositions explaining the method-inaction process. on the other hand, given the iterative nature of case research, additional analysis may produce conclusions that argue against the framework’s plausibility, thereby forcing a new round of theory specification research. ajis volume 12 no. 2 may 2005 130 appendix a the coding scheme rational roles rational factors code reduction of complexity isd methods imply a strategy that subdivides the development process (divide & conquer) + [r-comp] facilitate project mngt & control .. the influence of business users through their control of the sign-off process + [r-proj] division of labour different tasks require different skills, and economies in pay rates depending on skills required + [r-div-o--lab] systematise development knowledge capture knowledge from the developer to the methodology so that it can be stored, disseminated, exchanged + [r-know] standardisation of devt process .. standardisation of the development process across all projects within a large organisation + [r-stand] desirability for iso certification as part of the quality endorsement process + [r-cert] solution to software development … a widely help opinion that methods represent the best way to develop systems + [r-best-way] definitional anomalies problems as to what actually constitutes a method [r-def-anom] generalisation without validation an absence of peer review or evaluation of their applicability [r-validity] inadequacies of the rational scientific paradigm …. over-emphasis on technical rationality at expense of softer, social aspects [r-rational] goal displacement .. . slavish and blind adherence to the method while losing sight of the fact that development of the system is the real objective [r-goal-disp] methods are universally applicable .. failure to recognise the uniqueness of every development situation [r-unique] inadequate recognition of developer-embodied factors .. methods don’t cater for critical factors such as individual creativity, intuition, or learning over time [r-individ] political roles political factors code professionalise isd work … insulating developers from conceding to unreasonable deadlines and demands from business user departments. [p-prof] make isd more proactive .. promote the it department to a more proactive role in strategy formulation [p-proact] comfort / confidence .. method use provides for new it professionals some reassurance that proper practices are being followed [p-comfort] audit trail … providing an audit trail of the development process to afford protection if design decisions turn out wrong in the future [p-audit] construct explanation category or theme code ajis volume 12 no. 2 may 2005 131 legitimacy factor … use of the methodology enables the organisation to win government contracts, or achieve certification [p-legit] power base for method champion .. method champions may use it to raise their profile within the organisation [p-power] development context context factors code context & technology… new technology makes it possible to perform organisational tasks in different ways [c-technology] context & culture .. culture (collection of experiences expressed in a set of beliefs & values). .. a culture will influence what is and what is not possible to do within a specific context [c-culture] strategies for change … proactive (opportunity to be exploited) v reactive (responding or reactive to a change situation elsewhere) … [c-change] strategies for change … problem solving (getting away from an unwanted situation) v innovation (change is viewed creatively to form a new context) [c-change] strategies for change … incremental (on-going daily activity in small steps) v radical (change in large and more focussed steps) [c-change] strategies for change … high risk v low risk [c-risk] knowing the context … understanding of the context [c-knowing] developers developer factors code developer awareness of methods …. education and training in method use [d-awareness] experience … development experience and prior exposure to, and use of methods [d-exp] knowledge of application domain … experience of the banking business domain [d-domain] commitment & motivation … commitment & motivation to the project [d-commit] autonomy .. discretion in relation to the development process [d-autonomy] information processing system system factors code system purpose …. [s-purpose] system complexity .. size, technical dimension, user interaction, etc [s-complex] system solved or unsolved … common or routine situations v class of system not developed before [s-solved] uniqueness or standard … [s-standard] references avison, d., & fitzgerald, g. (2003) information systems development: methodologies, tools and techniques, mcgraw hill, berkshire. bazeley, p., & richards, l. (2000) the nvivo qualitative project book, sage publications. beynon-davies, p., & williams, m. (2003) the diffusion of information systems development methods, journal of strategic information systems, 12, 29-46. douglas, d. (2003) inductive theory generation: a grounded approach to theory generation, ajis volume 12 no. 2 may 2005 132 electronic journal of business research methods, vol 2(1). eisenhardt, k. (1989) building theories from case study research, academy of management review, vol 14, no 4, pp. 532-550. fitzgerald, b. (1997) the use of systems development methodologies in practice: a field study, information systems journal 7(3), 201-212. fitzgerald, b. (1998) an empirical investigation into the adoption of systems development methodologies, information & management 34, 317-328. fitzgerald, b., russo, n., & stolterman, e. (2002) information systems development: methods in action, mcgraw hill, berkshire. fitzgerald, b., russo, n., & stolterman, e. (2003) software development method tailoring in motorola, communications of the acm, 46(4), 64-70. galal, g. (2001) from contexts to constructs: the use of grounded theory in operationalising contingent process models, european journal of information systems 10(1), pp 2-14. glasser, b. (1992) basics of grounded theory analysis, california, sociology press. glaser, b., & strauss, a. (1967) the discovery of grounded theory – strategies for qualitative research, weiderfield & nicholson. london. golicic, s., davis ,d,. mccarthy, t., & mentzer, j. (2002) the impact of e-commerce on supply chain relationships, international journal of physical distribution & logistics management, vol 32(10), 851-871. hansen, b., kautz, k., & jacobsen, d. (2003) information systems development methodologies in practice, proceedings 12 th international conference on isd methods & tools theory & practice, melbourne, australia. hughes, j. (2004) reflections of the use of grounded theory in interpretive information systems research, electronic journal information systems evaluation, vol 6(1). iivari, j., hirscheim, r., & klein, h. (1998) a paradigmatic analysis contrasting information systems development approaches and methodologies, information systems research, 9 (2), 1-30. iivari, j., & maansaari, j. (1998) the usage of system development methods: are we stuck to old practices? information & software technology, 40 (9), 501-510. kiely, g., & fitzgerald, b. (2003) an investigation of the use of methods within information systems development projects, paper 34, ifip 8.2 conference, athens greece. madsen, s., & kautz, k. (2002) applying systems development methods in practice the rup example, proceedings of the 25th iris conference, august, bautahøj, denmark. miles, m., & huberman, a. (1994) qualitative data analysis: an expanded sourcebook, sage, thousand oaks. russo, n., & stolterman, e. (2000) exploring the assumptions underlying information systems methodologies, information technology & people, 13(4), 313-327. sauer, c., & lau, c. (1997) trying to adopt systems development methodologies – a case based exploration of business users’ interests, information systems journal, 7, 255-275. smith, a. (2000) casing the joint: case study methodology in vet research at the organisational level, anz journal of vocational education research, vol 8, no 1, pp. 7391. strauss, a., & corbin, j. (1998) basics of qualitative research, sage publications. truex, d., baskerville, r., & travis, j. (2000) amethodical systems development: the deferred meaning of systems development methods, accounting management and information technologies, 10, 53-79. urquhart, c. (2001) an encounter with grounded theory: tackling the practical and philosophical issues, in: eileen trauth (ed.), qualitative research in information systems: issues and ajis volume 12 no. 2 may 2005 133 trends, idea group publishing, 104-140. vidgen, r., madsen, s., & kautz, k. (2004) mapping the information systems development process, proceedings of the ifip wg8.6 working conference on it innovation, dublin, ireland. whetten, d. (1989) what contributes a theoretical contribution?, academy of management review, vol 14, no 4, pp 490-495. wynekoop, j., & russo, n. (1993) system development methodologies: unanswered questions and the research-practice gap, in j. degross, r.p. bostrom, & d. robey (eds.), proceedings of the fourteenth international conference on information systems, december 1993, pp. 181-190. wynekoop, j., & russo, n. (1995) systems development methodologies: unanswered questions, journal of information technology, 10, 65-73. wynekoop, j., & russo, n. (1997) studying systems development methodologies: an examination of research methods, information systems journal, 7, 47-65. yin, r. (2002) case study research: design and methods, 3rd ed, sage, beverly hills, california. microsoft word ajis10-2finala.doc ajis vol. 10 no. 2 may 2003 38 from adoption to use: the process of appropriating a mobile phone jennie carroll1 steve howard1 jane peck2 john murphy2 1 department of information systems the university of melbourne jcarroll@staff.dis.unimelb.edu.au 2 novell pty ltd 574 st kilda road, melbourne 3000 abstract users appropriate a technology through a process of evaluation that results either in rejection or adoption, adaptation and integration of the technology into their everyday activities. currently this process, and the influences on users’ decisions within that process, is poorly understood. this paper reports a research project that examines young people’s appropriation of mobile technologies. thirty young australians were supplied with a free wap phone for a month and were tracked from their expectations, the initial encounter and the first month of use of a wap phone. we found that the influences on their adoption of the phones were quite different to those on ongoing use and so the influences on adoption do not predict long-tern use. it is clear that studying adoption is only one step towards understanding technology acceptance and use and that rejection of a technology, even one that is supplied free of cost, is an ever-present option. keywords technology appropriation, technology adoption, diffusion of innovation, mobile technologies introduction technology innovation, adoption, acceptance and use have long been important topics for information systems (is) research (davis 1989). the life cycle of a technology—from designing and developing an innovation, communicating or diffusing information about it, deciding to adopt (selecting, purchasing or committing to use it) and then achieving persistent use—is poorly understood. few studies differentiate between preand post-adoption (karahanna, straub and chervany 1999); there has been little attention paid to the temporal dimensions of adoption and then only on a limited set of variables (for example, davis, bagozzi and warshaw 1989). our previous research (carroll et al. 2001; 2002a), like most is research, has examined perceptions of adoption behaviour after a technology has been adopted. this paper reports a research project that takes a processural approach to technology adoption and use and examines users’ expectations prior to encountering a technology, the initial encounter and then the first month of use. in this research project the focus is on young people. young australians are enthusiastic users of mobile technologies. about one half of young australians aged 14 to 24 own a mobile phone and the number is rapidly increasing (gooch 2002). our research illustrates that they are also discriminating users of mobile technologies. thirty young australians were supplied with a state-of-the-art wap phone. they had high expectations of wap mobile phones: the functions they would provide, the time saved and their usefulness in their everyday lives. however, even though the phones were provided free of charge (for the device and usage), many of the young people did not integrate the phones into their everyday practices. their verdict at the end of a month was ‘no thanks’. we note that young people persist with technologies only when they add significant value to their lives. further, the gap between their expectations prior to adoption and their evaluation after one month’s experience indicates that merely studying the decision to adopt a technology is insufficient, as it is followed by a whole process of deeper evaluation through use that may, or may not, result in persistent use. we have used research methods that are appropriate for the different experiences in this process to develop a rich view of young people’s adoption and use of mobile technologies. the findings extend our existing work on young people’s appropriation of mobile technologies and so contribute to is theory. the findings indicate that the influences on adoption are quite different to those on persistent use and so the influences on adoption do not predict persistent use. such a view contradicts the assumptions of much of the technology acceptance literature (see davis 1989). the implications of these findings are significant for is practice. developers, marketers, trainers and managers should note that it is not sufficient just to adopt a technology, rather it is important to be mindful of the influences that affect persistent use when designing and marketing technology innovations and training and managing staff throughout the appropriation process. after discussing the conceptual background of the research, the research approach is outlined, the research findings are presented and their implications are discussed. the paper concludes by presenting some of the shortcomings of the research that indicate areas for future work. ajis vol. 10 no. 2 may 2003 39 background this paper reports on the second stage of a collaborative research project with novell pty. ltd called ‘customers of the future’. it is a three stage project to investigate young people’s perceptions and use of mobile technologies as the basis for technological innovation. stage 1 (carroll et al 2001; 2002a) provided a benchmark of young people’s use of mobile technologies. it was also the basis for inducing a model of technology appropriation which we have defined as “the way that users evaluate and adopt, adapt and integrate a technology into their everyday practices” (carroll et al. 2002b). the model of technology appropriation was built through studying individual young people and their use of mobile phones – a technology that they had already appropriated (access to a mobile phone was a condition of participation in the study, see carroll et al. 2002a). in stage 2, discussed in this paper, we test the model by tracking young people from their initial encounter with an internet-enabled mobile phone through their first month of use. the model of technology appropriation the model of technology appropriation (carroll et al. 2002b), shown in figure 1, describes the transformation of a technology as it is envisaged by its designer (technology-as-designed) into technology as it is currently used (technology-in-use). the nature of this transformation is called the process of appropriation. technology -as-designed level 3 reinforcers technology -in-use process of appropriation nonappropriation disappropriation level 1 attractors filter appropriation level 2 appropriation criteria / disappropriation criteria figure 1: the model of technology appropriation (from carroll et al. 2002b) we suggest that users evaluate a technology at three levels, reflecting different degrees of familiarity with the technology at different times in the appropriation process (carroll et al 2002): • level 1 relates to users’ first encounter with a new technology, such as in a shop or a training session. at this level, initial judgements are made without any prolonged use of the technology. the outcomes of this filtering are either non-appropriation, where users are uninterested in the technology, or otherwise the decision to adopt (davis 1989; rogers 1995) initiates the process of appropriation. • level 2 reflects a deeper evaluation through use. if users are attracted by a technology, they will explore it in depth through the process of appropriation. there are two possible outcomes. appropriation occurs where the users take possession of its capabilities in order to satisfy their needs. disappropriation occurs when, at some stage during the appropriation process, users choose not to persist with the technology. • level 3 captures the longer-term use of a technology. the technology is appropriated and integrated into users’ everyday practices. this is not a one-off activity but rather is subject to ongoing reinforcement; changes in users’ evaluation of the technology may lead to disappropriation. this suggests that technology appropriation involves evaluation by users as they encounter (level 1), adopt and adapt (level 2) then integrate (level 3) a technology into their everyday practices. further, in stage 1 of the research we observed that different influences operate at the three levels. at level 1the initial filteringattractors encourage the users to further explore the technology and a lack of attractors may dissuade users from deeper evaluation, resulting in non-appropriation of the technology. at level 2during the process of appropriationcriteria encourage continuing evaluation or rejection of the technology. these ajis vol. 10 no. 2 may 2003 40 criteria influence the decision to either appropriate or disappropriate the technology. finally, a set of reinforcers act to maintain use. we have tailored the model to reflect our focus on young people and mobile technologies. during stage 1 we identified attractors, appropriation and disappropriation criteria, and reinforcers that influence young people’s appropriation of mobile technologies (carroll et al. 2002a). research approach the model of technology appropriation was derived from a study of young people’s use of mobile phones; it was based on a biased cohort as all participants owned or had access to a mobile phone and so by definition the filtering process at level 1 had been resolved in favour of deeper evaluation. in stage 2 we provide the participants with a new technology and so are able to study their initial encounters with a new technology (level 1) as well as medium term evaluation (level 2). due to the limited duration of stage 2, level 3 influences that reinforce use were not observed. in stage 2 we are seeking to confirm, disconfirm or extend the model of technology appropriation and the influences it describes on young people’s appropriation of mobile technologies. the characteristics of the new technology were selected to enable testing of the model. we studied the young people’s responses to: • the user interface. the internet-enabled phone was new in the australian mobile phone market and had an unfamiliar user interface. we were interested in examining the effect of a different interface on young people’s evaluation of the phones. • the network. the phones used a new network that was only partly functional at the time of the project, in that users were unable to send text messages outside the supplier’s network. we were interested in investigating the effects of this limitation on the young people’s sms (short messaging service) patterns, in particular whether – in the absence of cost concerns – voice calls and text messaging were interchangeable. • wap phones. the mobile phones used wireless application protocol (wap) to access the internet. wap is a standard for mobile internet access that enables wireless devices such as mobile phones and pdas to access special web pages. • internet access. the mobile phones were configured so that the internet was accessible via a security connection; this could be avoided if the young people reconfigured the hand set to overcome the inconvenience. we were interested in the way that the young people dealt with this limitation and whether they attempted to adapt the phones to make it easier to access the internet. the research was undertaken between may and july 2001 and involved 30 young people (21 had participated in stage 1) from melbourne and sydney. the research participants were recruited by professional recruiting companies on the basis of access to a mobile phone and the internet as well as possession of their own email address. all were aged between 16 and 22 and were either students (secondary or tertiary) or workers. we used a similar empirical process to that for stage 1 (carroll et al 2002a). questionnaires and focus groups collected data about young people’s expectations of internet-enabled phones. the participants’ initial encounter with the new technology was studied using a co-discovery process (o’malley, draper and riley 1984). this involved participants working in pairs exploring the internet-enabled phone and articulating their actions and perceptions to a researcher; the sessions were audio-recorded. these research methods were directed to examining the influences at level 1 of the model of technology appropriation: those that bear on the decision to adopt, also called ‘pre-adoption’ (karahanna, straub and chervany 1999). during the month of use, six participants in melbourne were observed in their everyday activities in leisure, social and educational contexts at different times during the week, the weekend and holidays. participants also completed an online diary for two days a week for the month, providing a record of their use of the phones. at the end of the month, a second questionnaire and focus group self-reported participants’ perceptions and use of the wap phones. finally, their itemised phone bills (separated into voice calls, sms messages, internet use, call diverts and voice mail) for the month were accessed and analysed. these research methods related to the deeper evaluation of a technology through use described by level 2 of the model of technology appropriation, also called ‘post-adoption’ (karahanna, straub and chervany 1999). as we were studying a process of appropriation, longitudinal data—rather than snapshots of use—were collected over one month. the data were analysed using coding (miles and huberman 1994); the concepts in the model of technology appropriation were used as the initial codes to guide the initial analysis, along with ‘any other’ codes to incorporate new themes that were noted in the data (carroll and swatman 2000). the resulting codes were refined through re-reading the qualitative data, comparing and contrasting this with the quantitative data and finally examining the temporal relationships between the resulting categories. together, these research methods provide a rich and detailed picture of the young people’s initial expectations of internet-enabled phones, their usage patterns over their first month of use and their evaluations of the phones at the end of a month of use. ajis vol. 10 no. 2 may 2003 41 findings expectations most (27) participants had heard of a phone that connects to internet and some (11) knew it as a wap phone but had not seen such a phone. only four of the participants recognised that wap sites would look different to regular web sites accessed from a personal computer (pc). thus there was little awareness that viewing the internet through a wap phone would provide access only to a very small number of web sites (a limited number of web sites are available in wap form) and only to a sub-set of the content of these web sites. many of the young people expected that the main uses of an internet-enabled phone would be reading email and accessing information sites. others had more limited expectations; one female participant said “it’s not like you can check your email or anything, as far as i know.” when asked about the likely effects on their lives, most participants were uncertain and wanted to see what an internet-enabled phone could do first. the model of technology appropriation: level 1 the influences noted on young people’s initial encounters with a technology in stage 1 were fashion/style, cost, convenience, usefulness, adaptability, familiarity and ‘our stuff’ (carroll et al 2002b). at the end of stage 1, one group of young people nominated fashion/style and cost as the two main drivers for adoption of a particular mobile phone. in stage 2: • fashion/style was evaluated on the size, weight and appearance of the handset: “i really thought that this would be heaps bigger and heavier… it’s actually really compact”, “looks nice”, “it’s really cool” and “it’s alright but nothing special.” one participant described how she evaluates a mobile phone by its looks rather than by the features it offers: “it’s more… how i like the phone to look, rather than it has to have the internet on it.” • the participants thought that the cost of both wap handsets and usage would be greater than normal mobile phones. even the idea of sending email was evaluated according to cost; asked whether she would send an email, one girl replied “it depends on… what the charge was, say per character or just the message.” • expected usefulness is the degree to which users expect that using a technology will provide them with benefits (adapted from davis 1989:320); a synonym for usefulness is convenience. the young people expected that the internet-enabled mobile phone would be used for email (18), gathering information such as movie times, news, weather and banking (12) and accessing the internet (11). they suggested that the ability to check email anytime anyplace, for example while waiting for a bus or an appointment, would be good. these expectations were supported by participants’ comments during the co-discovery: “this is fantastic, i’ve just got into the ski report bit… it tells you everything..um..lifts operating. this is fantastic. this tells you absolutely everything you need to know. ” being able to connect to the internet anywhere would save time and make contacting others easier: “you can find out information when you need it” and “it would make me a lot more mobile.” • adaptability: in stage 1 we noted the importance placed by young people on the ability to personalise a mobile phone, such as change ringtones or covers (change the colour or pattern of the face plate). most participants adapted the phone during co-discovery by resetting the ring tones, changing the volume or personalising the greeting banner. one of the first actions of many of the young people during the co-discovery was to customise the greeting banner: “i’ve just changed my banner. it says ‘sam rocks’. that’s cool.” a sixteen year old girl listened to the ring tone “i’m not happy with it – i think i’ll change it [different ring tones are heard].” • familiarity: the participants were familiar with certain interfaces and functions provided on mobile phones, especially nokia phones which are the most popular brand amongst this age group (owned by 70% of the participants). the different interface provided them with some problems; in the codiscovery a pair of young boys were trying to turn on the phone: “it’s no nokia. oh god, i hate doing this…it’s got all these tools but i don’t know how to turn the damn thing on.” others said: “it’s different. i’m so used to the nokias” and “it is a bit different to my phone… so just getting used to it.” some participants spent the early time in the co-discovery flipping through the manual while their partner explored the phone, but this was generally perfunctory: “the thing about using phones…it has all this stuff about ‘get things started’ and things like that but whenever we get a phone we skip all that sort of stuff and we go into the fun stuff like how to use the email account first”. lack of familiarity with the interface and internet access via a mobile phone led a number of participants to consider using the manual: “maybe we should actually read the book… we might actually be able to work it out.” ajis vol. 10 no. 2 may 2003 42 • ‘our stuff’: in stage 1, one participant described mobile phones as ‘our stuff’: young people’s technology with which they are comfortable. in stage 2, the participants readily identified the phone as suitable for their age group: “it’s really cool. it’s like a new toy” and “they’re ace.” features such as voice memo triggered enthusiastic responses: “oh this is so exciting… i’ve recorded my voice.” once they had discovered how to turn on the phone, most of the young people appeared very much at ease with basic features of the technology and quickly adapted it to their needs, for example by increasing the volume or selecting an appealing ring tone. however, one male participant struggling with interface said: “i feel like an old man with this, seriously.. you know how some old people have no idea…” it appears that ‘our stuff’ is a composite concept: our stuff is fashionable and fun, can be adapted to young people’s needs and desires, as well as familiar or easily mastered. two additional level 1 influences were noted in stage 2: • ease of use: in stage 1, usability was noted as an influence in level 2 (disappropriation criteria). however, in stage 2 we observed that ease of use was an important level 1 influence. a pair of school-aged boys was asked for their initial impression of the phone: “i like it… lots of nifty tools..” but when they were asked whether they knew enough to make a purchasing decision, the response was “not really, the functions are good but it also has to be user friendly. it can have all the functions in the world….[but] it took us ages to turn this thing on.” the handset had a central black button to control interaction; it toggled between five options: right, left, up, down and centre (enter). during the co-discovery, most participants found this hard to use: “i don’t like this button.”, “it’s so annoying. you push the button and you don’t know if it’s doing anything.”, “this black thing is so sensitive, hard to control” and “i so do not like this button.” problems with the black button led one participant to state: “this is not a useful tool.” the ease of use of the handset was enhanced by the large screen and the easy-to-read text: “huge screens – very nice” “the main thing is the bigger screen on it, it makes the phone bigger but it is more convenient.” however, one practical issue was noted: “no one would want to sit in front of a telephone reading pages of information.” • features: the features offered by a mobile technology are an important influence on whether it is adopted: “when you first get a phone you want to look at all the features, look at them, look at the games and stuff…” the phone offered a number of new features: “this is great new stuff, emails…” “it’s got a stop watch, calculator…not baaad…”, “i love the calendars, love that..” and the internet search: “wow! it’s unbelievable!” . other features including an alarm clock, tip calculator and voice memo; these were described as “cool” and “exciting”. participants liked the help on the phone (at bottom of screen). when asked how it differed from her normal phone, one female participant responded: “more options… it’s got a lot more things than i thought. it’s just a little smart cell.” the stage 2 analysis of the data on level 1 confirmed and extended our findings from stage 1 (see carroll et al. 2001; 2002a). an outcome of stage 2 was refined definitions of the influences at level 1 and the relationships between them. cost is composed of the ‘up front’ purchase cost and the expected usage costs. expected usefulness reflects the benefits that are expected from use of a technology and is an outcome of the application of the technology’s features in context (geographical, social, temporal). features are a combination of the physical functions that are offered by the phone and its associated services. ‘our stuff’ reflects the extent to which the young people associate the technology with their age group and is fashionable and fun, can be adapted to young people’s needs and desires, as well as familiar or easily mastered. fashion/style reflects the size, weight and appearance of the mobile phone. adaptability is the extent to which users can personalise a phone. familiarity is the degree to which users have knowledge of the technology, its interface and the services it offers. ease of use is the degree of effort required to use a technology; concepts such as ‘user friendly’ and ‘usable’ are reflections of ease of use. non-appropriation or adoption? the level 1 influences had mixed effects on the young people’s initial evaluation of the wap phone. expectations of usefulness and adaptability were positive, there were diverse responses to the fashion/style and the familiarity of the handset while it was generally accepted as ‘our stuff’ (it was a mobile phone, albeit with some additional and unfamiliar features). initial negative expectations of internet-enabled phones related to cost; by removing cost as a variable, we hoped to encourage the participants to explore the technology freely. there were differing views of the ease of use of the phones: negative views of the unfamiliar interface and the black button and positive views of the large screen and text. several participants also commented on possible negative social effects of the phones: “people are addicted to mobile phones and the internet as it is; to put the two together you would probably never stop playing with it.” and “it would make some people more anti-social, spending a lot of time walking around using the internet on their phone.” ajis vol. 10 no. 2 may 2003 43 overall, the participants’ comments at the end of the co-discovery were positive: “i’m going to have a lot of fun with this”, “i love it. i want to have one but it’s probably too expensive.” “i think it’s great – i am hooked” and “this is fantastic… i’ve got a very bad feeling i could spend hours on this.” one male studying computing at university asked: “at the end of the month do these go up for sale to us? cos i was seriously considering buying one… they’re pretty cool.” none of the participants discussed not using the phone or withdrawing from the research. these positive expectations at the time of adoption did not, however, predict ongoing use. in the next section we examine the young people’s experiences when using the wap phones over the next month. the model of technology appropriation: level 2 level 2 of the model of technology appropriation reflects users’ deeper evaluation of a technology through use. an unexpected finding was that the influence of a number of attractors noted at level 1 persisted beyond the initial encounter and into the process of appropriation. as the young people used the wap phones, the value given to some of the attractors changed. • the initial evaluation of their adaptability was positive but experience using the phones showed that only very limited personalisation was possible. at the second focus group there were negative comments about the inability to download ring tones from the internet and to buy covers for the phone to personalise its appearance (as is done with nokia phones). • many of the young people only discovered a sub-set of the features offered by the phones (this was exacerbated by its unfamiliar interface and their reluctance to use the manual). after a discussion in the second focus group of the different features, a female university student said (soto voce) “i didn’t realise that all of this was possible.” the value of offering a large number of features is questionable: young people want useful features. a young female said that she just wants a basic phone with different rings (different tones for different callers) and games “for when i get bored.” one male university student said that that it was “not the stuff to attract 18 to 19 year olds” while another stated that there were “plenty of options available but i don’t think they’ve set it out right”; it is the combination of features, their usefulness and their ease of use that is attractive. level 2 appropriation criteria the level 2 influences on young people’s appropriation of mobile phones noted in stage 1 in the research were social management, lifestyle organiser, leisure, safety and security, critical mass and contact (carroll et al 2002a). interesting outcomes related to social management/contact and leisure were noted in stage 2 as well as an extra criterion, usefulness. the addition of usefulness is supported by davis (1989) who argues that perceived usefulness and ease of use play a central role in predicting and explaining user behaviour. further, he suggests that ease of use may be an antecedent of usefulness because users adopt an application “primarily because of the functions it performs for them and secondarily for how easy or hard it is to get the system to perform those functions.” (davis 1989:333). • social management/ contact: the ability to use mobile technologies any time and any place, asynchronous and low-cost communication through sms and critical mass amongst a social group all help young people to manage their social lives and maintain contact with the diverse people in their lives. we were interested in whether voice calls and sms are interchangeable for young people. at the time of the trial, text messaging was limited to within the network (the network is now fully implemented and this limitation no longer operates). at the first focus group, participants suggested that they would ring friends using other networks to compensate for the limited sms service. however, during the month of use many participants paid to use their own mobile phones for sms while retaining the free wap phone for voice calls. young people are extremely sensitive to cost and their willingness to pay for the capability to sms shows its importance in their lives. we observed subtle nuances of friendship between friends that are smsed and those that are called. asked why her sms messages were not signed, a school-aged female responded that she uses sms between close friends so the identity of the sender is obvious. if she is smsing not-so-close friends, she starts off with “hey its sam” or, more usually, she will ring them. if it is someone she rarely speaks to, then she would email them. also, sms is more appropriate in some situations: “i couldn’t do messages when i am in a lecture, you can’t call… normally you can sit there and say ‘i’ll be out in half an hour’ but with this you would have to leave and make a call.” it was clear that voice calls and sms are not interchangeable and that sms is a particularly important technology to the young people. ajis vol. 10 no. 2 may 2003 44 • usefulness: the features of the wap phones, especially email and the internet, were not as useful as the young people had anticipated: “it didn’t help me with many things”, “it was not for my lifestyle... i don’t find it necessary” while a third participant described it as “just like a novelty.” email use was limited. many couldn’t collect email as hotmail (their favoured email provider) did not have a wap site. those who could access their email mainly read rather than sent email due to difficulties of entering email addresses and messages with phone keyboard. their experience of email as hard to use and unreliable to connect meant that “email is not worth it.” participants had high expectations of the internet but were frustrated when they realised the difference between usual internet sites and wap sites: “i thought ‘great’, an internet phone. i can search the web but i didn’t realise the web and wap are different and you need a web site to have a wap site on it to be able to go onto it.” in the end, “i just thought it was easier to use a computer” because a computer is stable, easier to use and a whole page can be seen at once. several participants described internet access from a phone as a novelty: “it’s a toy.” once the initial novelty wore off, they made little use of the internet as it did not provide value: “it promises a lot but when you get into the web, there is not much there.” there are no images so many sites were not worth visiting. the billing data shows that over the month, use of the internet dropped for almost all participants. the inability to sms to other networks was “a big annoyance.” this was one reason that participants disappropriated the technology: “i have found that i am using my phone less, most of all because of the factor that most of my friends are on different networks which makes sms use very hard.” we described usefulness above as an outcome of the application of the technology’s features in context. the wap phone was used in a wider range of contexts than expected: not just when out with friends or in transit but also in their bedrooms: “when i was in bed at night and i couldn’t be bothered going downstairs, if i wanted to check an email i would … or if someone else was using the computer”. even if a pc was available, some found that information such as movies and weather was easily found using the wap phone. one participant described the phone’ usefulness for multitasking: “my brother was sitting at the computer but used my phone to check the surf report… because he was playing a game…” level 2 disappropriation criteria criteria noted in stage 1 that influence young people’s decision to disappropriate a technology were hidden cost, health, reception, ease of learning and usability (carroll et al 2002b). two notable findings observed in stage 2 relate to ease of learning and ease of use (usability). while these were observed as two separate influences on the young people, previous research has indicated that ease of learning is a component of ease of use (davis 1989). • ease of learning: the young people played with the phones rather than read a manual: “i found things by accident… i wasn’t trying… you just play around and if you find something, you find something and if you don’t, you don’t.” as a result, different participants discovered different things about the phones. this confirms our assumption that technology in use is heterogeneous, as illustrated by the multiple technologies-in-use in the model of technology appropriation in figure 1. few participants admitted to using the manual except to try to resolve specific problems (for example, a female changed the language of her phone to spanish and was unable to read the menu in order to change it back to english). some participants found it hard to master the phone and many • leisure: several unexpected findings about the use of the phone for leisure activities were observed. the first relates to the design of the handset, where interaction occurs through a multi-functional black button. leisure use was inhibited in certain situations: “the black button is quite loud… when you are in a classroom the teacher can hear when you are playing [games] under the desk in class.” the second relates to the use of mobile phones for ‘pranking’. pranking often requires caller anonymity, which was not possible with this handset. this was described as “pretty annoying” and many of the school-aged participants complained about this: “i wanted to prank someone, but you can’t take your number off.” young people prank each other so that the recipient bears the cost of the phone call: “i’ve got a missed call, so i will ring back”. another aspect of pranking is to bully others – to send unpleasant and anonymous messages. a final aspect of pranking discovered by some participants involved use of the speaker phone feature. a participant rang an acquaintance and had an intimate conversation about relationships; the receiver was unaware that the conversation was being broadcast on a speaker phone to another three young people: “the speaker phone was good. it was good when you prank people with it.” this use of the speaker phone indicates how the young people adapted a feature of the phone to their leisure practices in a way that was unexpected and presumably outside the designers’ intentions. ajis vol. 10 no. 2 may 2003 45 noted that the period of exploration and play lasted only for a short period: they “didn’t have the time to fiddle after the first few days.” this was compounded by the lack of reward for effort: as the phone was “not serving my interests that much” there was little encouragement to continue to learn to use the phone. this limited opportunity for learning (see tyre and orlikowski 1994) and low use of manuals has implications for the introduction of radical new interfaces or features. if innovations are not quick and easy to learn, they may remain undiscovered by young people. in absence of effective alternatives to manuals, incremental additions to familiar interfaces may be preferred. • ease of use: the large screen and easy-to-read text was appreciated: “i actually didn’t think the screen was too big, it needs to be wide for the internet…. even with a big screen it is still very difficult to read emails and stuff on the screen.” there were several usability problems: the black button, lack of reverse scroll (so that users could not scroll up to the top and beyond to the bottom) and slow access to the internet. there were complaints about the number of steps needed to access contacts (quicker access is available but was not discovered by the participants) and aggregating all recent calls in the one list (they were differentiated by symbols but many of the users were not aware of this). data entry was a problem: “the phone does not have a keyboard which can make trying to gain access to websites and typing emails and other long information storing/data storing transfers slow and frustrating compared to using a pc or laptop.” in summary, one male university student said that his friends thought the phone looked “unreal”, but he thinks they would change their minds about the phone after using it; the difficulties of using it outweigh its appearance. a female participant described how “the idea of an internet phone has ‘wow’ value but when you realise how difficult it is to use, you say ‘*!@*^ that’.” appropriation or disappropriation? at the end of the month, many of the participants had appropriated aspects of the phones; for example, the internet “was pretty good, it did meet my expectations” and “voice activation … was good”. a few participants disappropriated the phone completely: “i just gave up” and “i just didn’t go near it i hated it so much”. overall, the high initial expectations were not matched by their experiences: “email is too hard” “apart from the free calls i’m pretty happy to go back to my own phone” and “i just thought it was easier to use the computer.” billing data supports these comments and show a decrease in internet access over the month. discussion n this research we have studied individual young people’s very early encounters with a new technology: their expectations (via a questionnaire and focus group) and then their actions as they unpacked, switched on and examined a wap phone (via a co-discovery session). at the end of the co-discovery—at the time of adopting the wap phones—the young people were enthusiastic with positive expectations of the phones. attractors such as expected usefulness, adaptability, fashion/style and familiarity were evident and the phones were generally accepted as ‘our stuff’. initial negative expectations of cost were neutralised by providing free phones and usage. some aspects of ease of use were viewed negatively as were possible social effects of the internet-enabled phones. we tracked the first month of use when deeper evaluation and adaptation of the new technology is undertaken. at the end of the month, many participants’ experiences of wap phones did not meet their expectations. the outcome of the deeper evaluation was that, for many of the participants, aspects of the technology were disappropriated and a few disappropriated the phone completely. these findings contribute to is theory in two ways: they extend our existing work on young people’s appropriation of mobile technologies and they indicate that focussing on the decision to adopt a technology is inadequate. they also contribute to is practice. the contributions are discussed below. technology appropriation we have defined appropriation as “the way that users evaluate and adopt, adapt and integrate a technology into their everyday practices” (carroll et al. 2002b). this research has refined and extended the model of young people’s appropriation of mobile technologies. we have revised the influences at level 1 and clarified their definitions and interrelationships. the model, shown in figure 1, presents three discrete levels. our findings from stage 2 of the research suggest that there is significant overlap between levels 1 and 2. while the attractors at level 1 relate to the users’ initial expectations (of cost, usefulness, fashion/style, adaptability, familiarity, ‘our stuff’, features and ease of use), some of the attractors continue to have influence during the ajis vol. 10 no. 2 may 2003 46 first few days or weeks of use. this suggests that appropriation involves a gradual drift through the levels rather than progression through clear-cut levels. in addition, our earlier work involved mobile technologies that had already been successfully appropriated. we have extended this by studying young people’s expectations and initial encounters with a new technology; the outcome is a understanding of the influences on their decision to adopt a new technology that is grounded in their actual practices rather than post hoc recollections. the research has also described the way that technology is adapted as part of the appropriation process. adaptation may take different forms: changing the physical configuration of the technology, changing the purposes for which existing features are used, and users changing their activities to accommodate the technology. limited adaptation was observed in this research; this reflects the physical capabilities of the handset, the ‘interpretive flexibility’ (bijker and law 1992) of the technology and the short duration of the trial period. the handset could be configured to enable easier access to the internet, to represent menus in text or icons and attributes such as ring tones and greeting banners could be altered within the alternatives provided. the technology was flexible in that some of its features could be shaped and re-interpreted to meet the young people’s needs: use of the speaker phone for pranking illustrates a popular adaptation. minimal adaptation of young people’s practices was observed, possibly due to the limited time of the trial. one such adaptation was the use of multiple phones in order to meet conflicting needs (saving money through use of the free phone for voice calls and social management through use of their normal phones for sms). adoption and appropriation we have argued (carroll et al 2002b) that different influences affect young people’s decisions to adopt and to persist in using (appropriate) mobile technologies. few is studies differentiate between these decisions, also called pre-adoption and post-adoption (karahanna, straub and chervanny 1999). the decision to adopt— often involving selection or purchase rather than implementation of a technology (fichman 1999)—has received a great deal of attention (for example, davis 1989; rogers 1995). this study demonstrates that a focus on adoption is not sufficient. in order to predict and explain technology use (davis 1989), we need to examine technology appropriation over time: users’ expectations, the decision to adopt and actual use. by triangulating data methods and collecting data over time, we have developed a rich view of technology adoption and use. we have also clarified the changing influences that affect users’ evaluations as they shift from expectations to experience. the research has also demonstrated the importance of studying new technologies in context – what users actually do – rather than asking users what they think they will do. initially the young people were not concerned about the limitations on sms and they suggested that they would use free voice calls instead. their intentions, however, were not carried out; sms is an important part of their social lives (and an important gauge of closeness of friendship in some social groups) and the participants were willing to pay the price, both financially and with the inconvenience of using two mobile phones, to use sms. also, intentions must be modified in response to contextual issues: it is only when users are in certain contexts that limitations of a technology become evident (for example, sms and voice calls are not interchangeable in a lecture theatre). further, no matter how innovative or lateral thinking designers may be, they cannot imagine every use and situation of use for their designs; this is illustrated by complaints about a noisy button when playing games in a classroom. is practice this research has implications for those involved with the development, adoption and use of technologies, such as designers, marketers, trainers and managers. the model of technology appropriation presents the influences on young people’s evaluation, adoption and use of mobile technologies. knowledge of the different influences that affect users’ practices at different times in the technology lifecycle enables more effective targeting of support, training and change management activities. it also implies that ceasing these activities at the time of adoption may jeopardise the long-term acceptance of new technologies; instead support, training and change management should continue throughout users’ deeper evaluation of the technology. another implication for is practice is the importance of being first-to-market with some technological innovations. nokia has captured the youth market for mobile phones and has set the standard for features and interfaces; the young people have mastered the nokia handset and this sets their expectations for other handsets. most of the young people were familiar with the nokia interface for mobile phones: their understanding of the essential functions of a mobile phone are set and they have an accepted standard for the interface. ajis vol. 10 no. 2 may 2003 47 conclusion stages 1 and 2 of the research project have produced a robust model of young people’s appropriation of mobile technologies. the model was developed in stage 1 through benchmarking young people’s current practices and examining their perceptions and use of mobile technologies (carroll et al 2001; 2002). in stage 2, reported in this paper, we tested and refined the model by studying young people’s responses to an unfamiliar technology. further work in stage 3 seeks to use this understanding of technology appropriation to envision young people’s future needs and situations of use in order to design innovative technologies (see howard et al. 2002). there are a number of shortcomings of the research reported in this paper. most important is the removal of cost as an influence on use. we were faced with a dilemma: young people are extremely cost-conscious but thorough exploration and use of a wap phone is expensive and so may prove too costly for many young people. we chose to remove cost as a variable but acknowledge that this affected the young people’s use patterns. as a consequence of providing a free handset and phone usage, there was a strong incentive at the initial encounter with the technology for the participants to move into the process of appropriation rather than into non-appropriation. in addition, having free use of a phone was acknowledged by most participants to have changed their behaviour. a particularly surprising outcome was that, despite having a free phone and free usage, many participants partially or wholly disappropriated the phones. secondly, the wap phones were trialled only for a short period (one month) and had to be returned at the end of the research. we were unsure of what length of time would constitute medium and long-term use for mobile technologies and estimated that one month was sufficient to see level 2 influences in operation. analysis of the data suggests that one month was somewhat too brief to study level 2 influences for a new technology and that the drift from level 1 to level 2 was only partially completed during this time. thirdly, knowing that the phones would be returned limited some participants’ activities: few used the schedule or entered confidential information; one participant stated that “i would have gone more in-depth into the phone if i was actually keeping it.” finally, a significant shortcoming of the research was the focus on individual not group appropriation. we have noted that social issues are critical to technology appropriation and that mobile technologies are social as well as personal technologies (carroll et al. 2001). this research examined individual young people and their appropriation of mobile technologies. in future research, therefore, we plan to investigate the appropriation of mobile technologies over longer periods of time and by groups of young people. acknowledgements this research was jointly funded by the university of melbourne through a department of information systems early career research grant and novell through their customers of the future program. references bijker, w.e. and law, j. (1992b). general introduction. in w.e. bijker and j. law (eds), shaping technology/building society: studies in sociotechnical change. the mit press: cambridge, ma. carroll, j., howard, s., vetere, f., peck, j. and murphy, j. (2001). identity, power and fragmentation in cyberspace: technology appropriation by young people. in g. finnie, d. cecez-kecmanovic and b. lo (eds), proceedings of the 12 th australasian conference on information systems (acis 2001). vol. 1, 95-102. carroll, j., howard, s., vetere, f., peck, j. and murphy, j. (2002a). just what do the youth of today want? technology appropriation by young people. in r.h. sprague (ed.) proceedings of the 35th hawaii international conference on system sciences (hicss-35), january 2002, maui, hawaii, institute of electrical and electronics engineers, inc. (ieee). cd-rom. 9 pages. carroll, j.m., howard, s., peck, j. and murphy, j. (2002b). young australians and mobile technologies. what do young people want? how do they use mobile technologies? journal of information technology theory and application (jitta) (forthcoming). carroll, j.m. and swatman, p. a. (2000). structured-case: a methodological framework for building theory in information systems research. european journal of information systems, 9:4, 235-242. davis, f.d. (1989). perceived usefulness, perceived ease of use, and user acceptance of information technology, mis quarterly, 13:3, 319-340. davis, f.d., bagozzi, r. and warshaw, p.r. (1989). user acceptance of computer technology, management science, 35:8, 982-1003. fichman, r.g. (1999). the illusory diffusion of innovation: an examination of assimilation gaps, information systems research, 10:3, 255-275. ajis vol. 10 no. 2 may 2003 48 gooch, l. (2002). upwardly mobile become soft targets, the age, p.2 (april 10). howard, s., carroll, j.m., murphy, j., and peck, j. (2002). provoking innovation: acting-out in contextual scenarios. the british computer society conference on hci, london, september 2002 (forthcoming). karahanna, e., straub, d.w. and chervany, n.l. (1999). information technology adoption across time: a cross-sectional comparison of pre-adoption and post-adoption beliefs, mis quarterly, 23:2, 183-213. o'malley, c.e, draper, s.w. and riley, m.s. (1984). constructive interaction: a method for studying human-computer-human interaction. proceedings of ifip interact '84: human-computer interaction, 269-274. rogers, e. (1995). diffusion of innovations, 4 th ed., new york: free press. tyre, m.j. and orlikowski, w.j. (1994). windows of opportunity: temporal patterns in technology adaptation in organizations, organization science, 5:1, 98-118. microsoft word ajis vol13-2 o.doc 89 australasian journal of information systems volume 13 number 2 may 2006 technology for the humdrum: trajectories, interactional needs and a care setting connor graham interaction design group, department of information systems, university of melbourne parkville, victoria 3010, australia email: cgraham@unimelb.edu.au keith cheverst and mark rouncefield department of computing, infolab21, lancaster university lancaster, england, united kingdom email: {kc, rouncefieldm}@comp.lancs.ac.uk abstract we report on a care setting where staff looking after ex-psychiatric hospital patients used mobile and stationary communications technology (e.g. mobile phones and a messaging system) and physical artefacts (e.g. whiteboards and post-it notes). building on previous ethnographic investigations, we show that the concept of trajectory (or an ongoing course of action) was important when generating a particular understanding of staff’s care work. we argue that sensitivity to this concept and related subconcepts was helpful in identifying the key transitions, cycles, plans and management issues in staff’s ongoing work. we present verified trajectory-informed scenarios and themes emerging from fieldwork and show that the snapshots of work described in the scenarios were useful for establishing current and future interactional needs among staff and residents. we also show how trajectory helped ground a design for a situated display. finally, we describe the strengths and benefits of trajectory as ‘a way of looking’ in fieldwork aimed at socio-technical system design in settings where supporting collective, ongoing, contingent care is important. introduction recent studies of new technology in situ (e.g. mobile phones, situated digital displays) recognise the problem of understanding such technology as frozen in time and place and offering an explicit, immediate and tangible set of functions to be discovered by the user and, in turn, interpreted by the investigator. the turn towards field and ethnographic studies to get at ‘what is really going on’ and to respond to the ever-changing role of technology has been a response to this problem in hci. this ‘turn’ could trace its origins to responses to problems encountered by cscw (computer-supported cooperative work) researchers, captured well by kuutti’s (1996, p21) comment on liam bannon’s critique of human factors: a third issue for bannon has been the growing recognition that actual use of systems is a long-term process that cannot be adequately understood by studying just the initial steps of usage. a large part of hci research has studied only inexperienced users and usually during a relatively short period. in real life, people develop their skills during australasian journal of information systems volume 13 number 2 may 2006 90 longer periods, and this skill-achieving dynamics and its factors have achieved too little emphasis in research. this well-worn quotation focuses on the shortcomings of brief studies of use to understand skill development using non-representative users. thus, this particular critique of hci research in 1996 was mainly concerned with methods. however, our main aim here is not to make claims concerning methods, despite sharing kuutti’s view that studying atypical people in artificial settings over a short period has its dangers. our main interest concerns how terms like ‘skill’ and ‘task’ seem less applicable to particular settings, namely those where ongoing care (e.g. crabtree et al., 2003) and communication not governed by capital production (crabtree & rodden, 2004) pervade. ‘skill’, and even ‘task’, seem to suffer from a paucity of expressive power in such settings. a goal has been defined as “a state of the system [or the world] that the human wishes to achieve” (preece, 1994, p411) and a task as “the activities required, used or believed to be necessary to achieve a goal using a particular device” (preece, 1994, p411). however, what if there is no system, device or easily articulatable wish for state change in the world? how can we meaningfully talk of skills to be acquired, work to be done, tasks to be modelled when the notions of achievement or aspiring to goals are somewhat hollow? what if a terminally ill patient needs ongoing, day-to-day, collaborative, negotiated and inevitably humdrum care, not short-term, goal-oriented treatment? how can hci respond to the design problems posed by these kinds of research settings? in this paper we attempt to tackle the problem of determining potential technology design and use in a setting where the main ‘task’ is the care of others. like pinelle et al., (2004, p3), we are interested in understanding the “work of working” and the activities of communication and coordination. we distinguish this setting from studies of hospital work or domestic environments. in doing so, we trial particular aspects of strauss’s (1993) notion of ongoing, contingent work for understanding this care setting. we argue that through drawing on this we can understand actors’ “interactional needs” (fitzpatrick, 2003) better in situations where action is purposeful, but less comprehensible in terms of fixed and stable human wishes for change. the role for technology support for medical work in hospitals, in particular on wards (e.g. bardram & bossen, 2005; reddy & dourish, 2002), in remote healthcare (e.g. fitzpatrick, 1997) and in specialist areas such as operating theatres (e.g. watson & sanderson, 2004), has already been addressed to some degree. these research settings are complex and intricate, but technology has had a role there for some time at least in larger hospitals in developed countries. our interest here is the potential role of new technologies in settings where ‘work’ is less about treatment and more about ‘mundane’ and ‘everyday’ actions and interactions. we are also concerned, like reddy and dourish (2002), with the management of important information. we report on research conducted at community facilities where ex-psychiatric patients, experiencing a variety of illnesses, lived and were cared for by staff. in doing so, like clarke (1990, p14), we explore how the concept of trajectory (strauss, 1993), and its accompanying subconcepts, is sensitising rather than definitive. in using it in this manner we attempt to “keep a balance between distortion and conceptualisation…” (strauss, 1993, p12). here we are not interested in developing “abstract, decontextualised, or general models” (crabtree et al., 2003), but in making sense of a small slice of the sometimes “bafflingly complex” (strauss, 1993, p12) world of social phenomena in order to orient the design technology supporting a variety of interactional needs. thus, in the name of sense-making, we explore the usefulness of trajectory, avoiding a debate concerning the notion’s ability to generate ‘richer’, ‘thicker’, more ‘reliable’, more ‘valid’ or more ‘generalisable’ descriptions than other sets of sensitivities. instead we, firstly, indirectly appeal to ‘higher’ criteria (often applied to theories), such as ‘fit’ and ‘understanding’ (strauss and corbin, 1998), to make a judgement concerning trajectory: do the findings map onto the reality being described? do they promote understanding or comprehensibility to respondents and practitioners? 91 australasian journal of information systems volume 13 number 2 may 2006 we, secondly, trace particular findings into technology designs, through the presentation of an, albeit completely non-functional, design (figure 1 below). finally, we reflect on and consider the work that trajectories did in generating the findings we got and present an account of how they helped and what they were good for. trajectories and the setting the appeal to trajectory for application to this setting was based on the knowledge that the care workers there, who were to be the focus of the study, were engaged in the primary activity of caring for ex-psychiatric patients, suffering from a variety of illnesses, in community housing facilities. previous applications of the notion of trajectory include chronic illness (strauss et al., 1984) and the articulation of project work (strauss, 1988). the fact that illness trajectory describes not only the physiological (e.g. the disease process itself) and the temporal (e.g. day-to-day activities) aspects of the disease, but also the sentimental (e.g. emotions concerning confrontation of the disease) and the social (e.g. the social consequences of dependency) promised to provide insights into the complexities of the setting. strauss’s notion of “articulation work” has already been applied to develop a notion of “mobility work” in hospital settings (bardram & bossen, 2005). mobility work attempts to deal with the “spatial aspect of a work trajectory” (ibid, p137) in settings where the configuration of the environment is crucial to ongoing work. for us, strauss’s notions promised to fit the phenomena observed promote understanding (strauss & corbin, 1998) of the care work itself: trajectory has already been used to describe chronic illnesses (strauss et al, 1984). we were less concerned with the ecology of space and the material environment, which was a key concern for bardram & bossen (2005). trajectory also promised to enable us to identify the key concerns at the setting and attend to how these concerns played out through the contingencies of everyday care work. strauss (strauss, 1993, p53) defines a trajectory as: “(1) the course of any experienced phenomenon as it evolves over time (an engineering project, a chronic illness, dying, a social revolution, or national problems attending mass or “uncontrollable” immigration) and (2) the actions and interactions contributing to this evolution.” strauss also noted how trajectory describes interactions among multiple actors that contribute to a course of action that is contingent and sometimes unmanageable. this could include the management of an individual’s psychiatric condition. thus, trajectories are stretched over time and space, can operate at different levels of abstraction, involve many people and may change course in unanticipated ways. these trajectories are engaged in by actors who belong to particular social worlds. a social world is defined as a “universe of regulated mutual response [whose boundaries are] set neither by territory nor formal membership but by the limits of effective communication” (shibutani, 1955, p524). strauss (1993, p159) stresses that shibutani was referring to “action” or the act of communicating when referring to “communication”. a social world could be a group of carers looking after a psychiatric patient. here, we are primarily concerned with supporting human action and interaction over time when we consider new technology design and thus we will discuss different kinds of interactional needs (fitzpatrick, 2003) emerging from notions of social world and trajectory. we are also concerned with how the ongoing membership of a social world is maintained through communication stretched over space and time. strauss (1978, p122) identified four main aspects of a social world: a primary activity, such as researching, with associated clusters of activity; australasian journal of information systems volume 13 number 2 may 2006 92 sites where these activities occur; technology used to carry out the activities; and evolving divisions of labour. from prior work at this setting (crabtree et al., 2003), it was clear that the care workers were very much part of “the…unfolding of a sick person’s disease”, “the total organisation of work done over that course” and “the impact on those involved with that work and its organisation” (strauss et al., 1984, p64), all important aspects of trajectory. thus, we approached this research with an acknowledgement that certain trajectory subconcepts may be more relevant than others to the care workers working at both sites. we hypothesised these would be trajectory phasing, trajectory scheme and trajectory management. trajectory phasing is “the researcher’s conceptualization of phases, in accordance with changes in the interaction occurring over time “around” the phenomenon as it evolves” (strauss, 1993, p54). strauss describes how phases “are properties of the sequence of interactions” (strauss 1993, p54). fitzpatrick (2003), in her locales framework, fleshed out strauss’s notion of phase to include temporal rhythms in her conceptualisation of phasing: “the temporal-related concepts of phase, rhythm, and schedule” (ibid, 2003, p127). her notion of phase is similar to strauss’s. trajectory rhythm describes the “periodicity of actions” (ibid). or “cycles of regularly occurring patterns of activity” (ibid, 2003, p128). trajectory scheduling describes “the way in which actions are related together in temporal work order” (ibid, 2003, p128). rhythms have gained considerable attention recently in studies of medical work (e.g. reddy & dourish, 2002) and domestic settings (e.g. crabtree and rodden, 2004). here we will use phase to capture important stages, changes and transitions in activity. we will use cycles to describe regularly occurring ‘rhythms’ of activity imposed by people acting and interacting at a setting as opposed to reoccurring events. the difference, although subtle, is an important one of perspective: cycle stresses people’s actions and interactions, whereas rhythm stresses history’s patterns and repetitions. trajectory scheme describes “the plan consciously designed to shape interaction as desired, given the content of a trajectory projection [or a vision of the expected course of interaction]” (strauss, 1993, p55). a scheme is “essentially envisioned as an overall strategy that when acted on becomes translated into actual actions” (strauss, 1993, p55) and could include the written plan for a psychiatric patient’s care. a scheme is “deliberate” and therefore available to the actor and other actors and may result in disagreement and debate. these schemes may be continually revised and reinterpreted and are contextualized by other schemes involving other actors. it is important to both acknowledge the multiple possible interpretations of a trajectory scheme and that these schemes are contextualized by “some organization of strategies” (strauss, 1993, p56). this notion resonates with the finding that an “abiding concern” (crabtree et al., 2003) for residents was ongoing planning and monitoring of their medication and that this responsibility was shared with staff at the setting and care professionals in the community. trajectory management is “the entire process by which the course of the phenomenon is shaped by actors, through all of its phases and perhaps subphases, by the carrying out of the trajectory scheme” (strauss, 1993, p56). strauss described how issues of influence, power, chosen modes of action, preferred interactional styles, spontaneous and deliberate action are all relevant here. management involves “a negotiated order (negotiation plus the remaining interactional processes)…whereby interactants must come to terms with the goals and actions of each other” (strauss, 1993, p57). the potential importance of this subconcept emerged from the finding that the environment at the care setting had to be carefully managed through the administering of medication by the residents themselves or by staff for example. this required “knowledge of the setting’s daily routines” 93 australasian journal of information systems volume 13 number 2 may 2006 (crabtree et al., 2003) by the resident and the care worker alike, communication and sometimes ongoing negotiation among a care team operating in the community. researching care work sites and participants we examined the work of health care workers operating across two sites. one site is staffed all the time, even at night, whereas the other is staffed at regular working hours. at the former site staff live and work alongside the residents, whereas at the latter site staff have a separate office and visit the residents in their semi-independent living flats, although residents regularly visit staff in the staff office. the permanently staffed site was a converted house where residents were free to roam around shared spaces. residents had their own rooms. the lounge room of this setting formed a communal area for staff and residents alike. adjoining the lounge there was a small office containing two computers, a fax machine and phone, bookshelves filled with paper files, a whiteboard containing information on residents and a safe to store residents’ money and medication. one participant pointed out this site was primarily the home of ten residents that also happened to be the care workers’ place of work. the same participant described how legal requirements at this setting demanded the existence of an office and the need to lock things away. the semi-independent living quarters consisted of terraced flats for residents and an office area for staff and residents alike. the office was similar to the space at the permanently staffed site: it also had two computers, a fax machine and phone, bookshelves, a whiteboard and a safe. staff moved around as part of their work at both sites. technology, at the time of this study, had already been designed and deployed at both sites based on prior ethnographic studies (crabtree et al., 2003). the spam (sms public asynchronous messaging) system was developed to support communication and coordination. specifically, the spam system was designed to run an sms messaging application, allowing staff at the two sites to communicate easily by composing messages using an on-screen keyboard displayed on a touch sensitive screen. one spam unit was deployed at each office in october 2002 and since then the units have been used regularly mostly by staff. the care workers themselves, and therefore the participants in this study, were aged between 25 and 58, with education varying from high school education (certificate of secondary education (cse) qualifications in england) to university degrees. staff tended to have work experience from unrelated fields and had a varying degree of experience with technology and computers. approximately half the staff were female. method of inquiry broadly, this research involved understanding the setting through field methods (schatzman & strauss, 1973) and collaboratively sketching the design of new technologies through participatory design (schuler & namioka, 1993) methods. the phases of the research are described in detail in graham et al., 2005b. we were influenced by participative inquiry approaches (reason, 1994) and co-realisation (hartswood et al, 2002): we asked participants to confirm our interpretations of their work; we involved participants actively in possible technology design; we adapted our methods of data collection to the sensitivities of the setting; and we coupled a study of participants’ work with a technology design process. in addition, this project built on previous ethnographic work, demonstrating a long-term engagement with the setting. the aim of the work was to provide details of the everyday practices through which work was accomplished, identifying the contingencies that could arise, how they were overcome and accommodated, how the interdependencies of a division of labour were actually achieved, how technology was incorporated into work activities, and so on. it was, at least in part, ‘being there’ which enabled us to identify the cooperative aspects of real time, australasian journal of information systems volume 13 number 2 may 2006 94 real-world work, such as the small-scale constellations of assistance and the awareness of others supporting the actual performance of work activities as actually done in real-time. thus, the sensitivity to trajectory operated in tandem with the fieldwork, enabling a particular account of the setting to emerge. the particular techniques we used for understanding participants’ work included analysis of server logs recording text interactions between the two sites using spam (cheverst et al., 2004), site visits, semi-structured interviews with researchers involved in the earlier ethnographic investigations, and semi-structured interviews with staff and management. a preliminary phase of the research involved examining the logs of interactions between the two sites using spam. this resulted in the selection of quotations from this data (e.g. “mu 2 bol 0 cheeky”, “im at the house talking to…”) to provoke discussion in the design phase of the research and to refine questions for semi-structured interviews, discussed below. the initial fieldwork reported on here involved site visits to familiarise one of the research team with the setting and a series of semi-structured interviews. the two semi-structured interviews with researchers and five semi-structured interviews with staff were informed by the concepts of social world and trajectory. thus, with regard to social world we asked open-ended questions like: how would you describe your key responsibilities? do you often work alone, or do you work with others? with regard to the concept of trajectory we asked open-ended questions such as: does your work have particular stages during the day? are you often aware of planning ahead…? the sketching of possible new technology designs involved the use of a participatory design workshop to discuss our observations and interpretations of participants’ work and to jointly envisage possible technology design through a focus group approach. this phase aimed to confirm that our observations made sense to the participants, making these interpretations public and providing the participants with vignettes around which they could build narratives, much in the way that gaver et al. (1999) conceive of cultural probes operating within a design process. we wanted to capture and represent some meaningful shards of staff’s work, informed by the three trajectory subconcepts discussed above (trajectory phasing, trajectory scheme and trajectory management) so that these ‘stories’ could be confirmed or disconfirmed, refined or elaborated on. this method involved generating vignettes that resembled ‘daily-use’ (cooper, 1999, p180) scenarios and presenting these ‘stories’ at the beginning of the design workshop. we then addressed each story’s typicality (how authentic the story was), frequency (how often the story occurred) and criticality (how potentially dangerous the consequences could be) in two multidisciplinary groups comprising one technologist, one hci researcher and two staff members. we also discussed current and potential use of communication technology using these narratives as a starting point. data collected & analysis performed in the preliminary phase of the work reported on here, a segment of the spam logs were perused and used to inform subsequent data collection. the first phase of the research, studying the participants, involved the collection of field and interview notes. this mode of data collection was largely driven by the sensitivity of the setting. we conducted a rapid analysis of the field notes and spam logs to generate six scenarios for use in the design workshop. this involved structuring ‘stories’ describing typical activity around the subconcepts of phasing, scheme and management. we also, through interrogation of the logs, field notes and the workshop video transcript (audio only), 95 australasian journal of information systems volume 13 number 2 may 2006 evolved a series of strauss-informed themes descriptive of staff’s work. we also photographed visuals describing possible new technology designs generated during the design workshop. results strass-informed findings social worlds and the care teams from the field data it emerged that staff participated in two main social worlds. the first social world, the everyday care social world was driven by the primary activity of the everyday care of residents within the context the two sites forming the organisation. this activity involved other clusters of activities (strauss, 1978, p122): visiting, spending time and interacting with residents; manual labour; and clerical duties. report writing, for instance, was essential to the care of the residents and involved recording residents’ mental health issues and documenting particular resident behaviour (e.g. a resident laughing). this ‘world’ exploited particular technology. this technology was essential for enabling actions and interactions which established and maintained membership and an ongoing division of labour. land and sometimes mobile phones were used for staff to solicit advice from management. documents and notices described legal requirements and organisational directives. staff used a medication book for recording the receipt and distribution of medication. a shared diary was used for recording important information, such as residents’ appointments. carers recorded important events during a work shift in a communication book. word processing applications were used for daily report writing. postit notes transferred important information from one care worker to another. cctv monitored events at the sites. spam, was used for social interaction (graham et al., 2005a), but also had an important role in maintaining particular divisions of labour and “communication zones” (nardi et al., 2000), particularly when a staff member was alone at one site. within this social world there was a deliberate attempt to share information about residents through ongoing talk and use of these technologies to sustain residents’ care. in the second social world, the community care social world, multiple individuals were involved in the management of each resident’s community care. individuals with overlapping roles were involved in a multidisciplinary team (mdt) responsible for the care of each resident: a key worker or staff member allocated to a resident (one staff member oversaw 2-3 residents); a consultant overseeing the resident’s care; a community psychiatric nurse (cpn); a social worker; and a general practitioner (gp). the staff at the setting described distinct roles and even a “lack of integration” within this social world. the membership and allocation of work seemed quite stable and these were less influenced by regular communication. there were also dependencies and particular divisions of labour within this social world: a cpn could not change a resident’s medication without contacting a gp for example. the care trajectory (described below) spanned both these social worlds, but was mainly contextualised by the everyday care social world because the focus of this work was the care workers at the setting. the care trajectory scheme there was a sense of a shared scheme among staff that aimed to promote activities among residents that would support an eventual move into the community. this was captured in a care plan for each resident. it was important for staff to be aware of these plans which involved the promotion of australasian journal of information systems volume 13 number 2 may 2006 96 routines such as healthy eating and consciously cultivating independent living skills such as being able to cook, adequately manage personal hygiene, manage money and self-medicate. medication and money management often required agreement: the former involved negotiations between the mdts and the residents and the latter discussions between staff and residents. on a more mundane level, staff were given a list of tasks to complete, written by the manager, supporting the everyday work of the care trajectory. management of residents’ care trajectories residents’ care trajectories were regarded as a shared responsibility among management and the staff: there was a genuine view of shared care. most of the practices at the setting were designed to shape a trajectory scheme, captured in the care plan for each resident. as already noted, this ‘trajectory scheme’ not only involved the everyday work of caring but also an attempt to progress residents’ psychological conditions positively. the everyday work of caring involved cycles of activity, which had key transition points, such as a change of shift (see below). care beyond everyday work was achieved through a shared awareness of residents’ current states and behaviour. this involved continuing vigilance and record-keeping. there was also an attempt, by many staff, to shape the care trajectory in a positive way. for example, staff tried to enable residents to become more independent through encouraging routines of personal hygiene. phasing of the care trajectory there were distinct phases in the trajectories of residents’ illnesses that involved collective management and planning. on a day-to-day basis, the care trajectory was supported by regular temporal rhythms (zerubavel, 1985) and cycles in staff’s work. staff had particular phases to their days, marked by the changeover, or handover between shifts. on a weekly basis, staff tended to work at different times. the three key phases during any shift were the handover from the last staff member, the activity and tasks during the shift itself (e.g. cleaning, giving out reminders to residents), and the handover to the next staff member. this process of handover was managed using talk and the exchange of postit notes and involved one staff member informing another of important events during their shift and referring to reports. the staff were also very much aware of residents’ phasing so they could react to any changes. a striking reality was that, in many cases, residents’ conditions were likely never to improve, but that, for many, they would cycle between being well and unwell. staff expressed the importance of their knowing about a resident’s current phase in her condition in order to manage their care effectively. strauss-informed scenarios in order to describe the routine and everyday work at the setting we wrote six scenarios. our intention was to capture the richness of the stories that emerged from the fieldwork so that these could be handed back to and confirmed or disconfirmed by participants. we were also concerned with embedding staff’s preoccupations with resident care in these narratives, their work’s collective and contingent nature and the negotiation of actions and interactions that could span over shifts. in the scenarios the staff’s social worlds (mainly the everyday care social world) operated as a backdrop to the particular actions and interactions captured by each trajectory subconcept. in writing the scenarios we consciously created an opportunity for reflection concerning possible new technology design. scenario 1: changing shift 97 australasian journal of information systems volume 13 number 2 may 2006 you are coming to the end of your shift and are talking through the day’s events with the next member of staff. a resident enters and wants to talk to you desperately. you ask politely for the resident to wait until you have finished talking to the staff member. the resident seems anxious. you stop the changeover and walk to the pool room to talk to the resident. this scenario captured transition in phasing and the need for management of the resident’s care trajectory despite the regular cycles of the day, in this case handover, being disturbed. this scenario was described to be “typical”, “irritating” and “vital”. scenario 2: forgetting something… you have come to the end of your shift and walk out of site 1. as you are on your way out you realise you forgot to write something in your daily report. it wasn’t urgent, but it was important for the resident’s long term care plan. you make a mental note to write it in a report tomorrow. this scenario captured transition in phasing and how care workers used an individual scheme (in this case making a mental note) to manage the residents’ trajectory of care. this scenario was confirmed as realistic, frequent and critical as the consequences of not recording or sharing important information about residents when staff did not sometimes come in for several days could be very serious. scenario 3: starting the day out… you start your shift, have your changeover, and then go straight to the table in the lounge room to see what your tasks for the day are. you turn to the other staff member on duty and say: “d’ya want to sort through what we are going to do?” your colleague nods and you work through the allocation of duties. we captured the subconcept of scheme here. we wanted to address how plans operate at the setting and who was responsible for them, and find out more about how staff managed the division of labour after the handover transition. the focus group addressing this scenario felt that this scenario was realistic and frequent and could be critical if it involved information pertinent to the provision of medication. scenario 4: next week’s fun… you are taking the residents to the cinema this afternoon and some of them are pretty excited about going. however, some residents don’t seem that enthused and have not indicated they are going. you have a chat to a few of them to encourage them to go. they seem a little interested, but seem to need more encouragement. here, we captured how staff could help manage resident’s care trajectory through a particular scheme that involved residents. the focus group discussing this scenario felt it was realistic, quite frequent and, although not critical in the short-term, very important for the resident’s longer-term care trajectory. scenario 5: keeping an eye out during changeover you are told that resident1 received some money the previous day and that he had popped out in the afternoon for several hours. after the changeover the resident approaches you australasian journal of information systems volume 13 number 2 may 2006 98 during medication time and asks for some additional medication and money. you are concerned and gently ask what the resident wants the money for. this scenario addressed trajectory management – the specifics of day-to-day management of resident spending in this case. this scenario was confirmed as “realistic” and quite frequent. the focus group discussing this scenario decided that it could be critical, but that it did not “necessarily have to be”: it being critical or not was determined by if this marked a change in the resident’s patterns of spending and/or medicating and the underlying reasons for this change. scenario 6: out of synch… you are trying to organise a day trip to blackpool for a few weeks’ time. during your morning routine when you check on residents you notice that one seems really excited about the trip as she has been to blackpool before and has some good memories of the place. this is the first time you’ve seen her as excited about something in a long time. in this scenario we wanted to explore the important phases and cycles of the activities supporting the care trajectory and how cycles of resident activity can sometimes change. the focus group that discussed this scenario agreed that it was infrequent but realistic and very important: they described how it was rare that residents actually had good feelings about themselves. the discussion in this focus group was less concerned with the everyday work of care and the most concerned with the personal nature of care. interactional needs and technology although detailed technology design is beyond the scope of this paper (a more detailed discussion is presented in graham et al, 2005b), we wish to explore how these trajectory-informed scenarios bridged into the design of new technology. the ‘stories’ presented above were successful in ‘provoking’ a response from workshop participants within a process of co-realisation. the notion of trajectory enabled us to capture some of the “temporal order” (barley, 1988) of the care setting in scenarios that acknowledged the particular nature of care work. we now wish to consider how these ‘stories’ helped describe issues important to technology design. analysis of the workshop transcript showed that participants discussed particular interactional needs through the scenarios: informational needs; broadcast needs; and therapeutic needs. informational needs, evident in scenario 2 for example, concerned the exchange of messages supporting the trajectory of residents’ care at the setting. these kinds of needs are similar to those reported by reddy & dourish’s (2002) on hospital wards and regarded the need for awareness of cycles of residents’ behaviour and patterns of residents’ spending and medication in particular for the purpose of care. information about medication was especially important: two residents at the setting had attempted to commit suicide. in both cases information concerning their trajectory of medication was essential. thus, these needs seemed to emerge from the description of trajectory phasing in the scenarios. broadcast needs, evident in scenario 2 and 4 for example, concerned the visibility of information supporting aspects of residents’ care trajectory. participants described how this information could include the staff on duty at a particular time, cinema information and advice on healthy eating. this information, as well as supporting awareness of ‘temporal rhythms’ (zerubavel, 1985) and cycles, 99 australasian journal of information systems volume 13 number 2 may 2006 could help manage residents’ conditions through allowing them access to information, such as film times, in a “non-confrontational” manner, while allowing filtering of information that “may not be good for their condition”, such as pornography. making information public concerning the phasing of people involved in residents’ care trajectory not only made work cycles visible but also served to support residents’ trajectory of care: one participant noted how residents “love to know who’s on duty” and want “to know who is going to be with us [them] for the next twenty-four hours”. these needs seemed to emerge from the description of trajectory phasing, scheme and management in the scenarios. therapeutic or more ‘phatic’ (vetere et al, 2005) needs, evident in scenario 6, concerned how social interaction, and, specifically, the act of sharing something, could support the trajectory of residents’ care. this could involve the sharing of photos by staff and residents alike. one participant described how it was essential to utilise a positive experience of a resident and how sharing a photograph meant sharing “a life story, of being someone, of being somewhere”. participants felt this could actually serve a therapeutic role: a positive memory could be used to make a resident feel better about herself. discussions of the scenarios revealed important aspects of message exchange that should be supported by any communication technology. these findings show there is a potential role for technology in exchanging and highlighting important information about residents over a period of time (such as information on their spending habits) and information concerning residents’ point in their care trajectory. these needs contributed to understanding trajectory beyond the singular notion of process towards a more complex, multi-dimensional flow of coordinated events and situations involving support for the psychiatric condition itself, the temporal aspect of the work supporting the condition, and the social consequences of these trajectories. participants described how interactional needs were being met currently by the technology in staff’s social world, such as printed reports on residents and terrestrial phones. the main focus of these ‘technologies’ was on meeting informational and broadcast needs, supporting residents’ care trajectories. one of the technology solutions that emerged from this discussion and a subsequent desktop design exercise was a situated display, meeting all three sets of needs. one of the focus groups described a large, flat touchscreen placed in a communal area. this display could support the receipt and display of filtered text and picture messages from staff and residents, as well as planned content. the ‘planned content’ could include information on healthy eating, pictures of staff and residents at different points in their lives, scheduled daytrips, and the staff rota. the potential design of this display is presented below (see figure 1) and described in more detail in graham et al., (2005b). australasian journal of information systems volume 13 number 2 may 2006 100 figure 1: sketch of possible situated display design the participants in the design workshop felt the messaged content would have to be governed by a policy or inappropriate images, such as residents’ bottoms, could be sent to the display. the positioning of the display also emerged as important: if the emphasis was on supporting broadcast and therapeutic needs, then the display should be placed in a communal area for residents; if the emphasis was on informational needs, the display should be placed in a communal area for staff. conclusion: the work that trajectories do “in comparison to the notion of the disease progress, trajectories provide a general analytical view of understanding the broad patterns of the way that people work together. the analytical power of trajectories lies in the ability that it provides us to examine the activities related to taking care of a patient, and the peoples and technologies with which a patient will come into contact during the course of treatment. as the illness unfolds, the work of managing that illness also unfolds. 101 australasian journal of information systems volume 13 number 2 may 2006 trajectories present us a way of examining this management from a number of different perspectives: the patient, nurse, physicians, family and other health-care providers. although activities and events re-occur during the course of an illness trajectory, the focus is on the work over time for a given patient rather than the re-occurrence per se.” (reddy and dourish, 2002, p351). the notion of a longitudinal, ongoing trajectory of care which is sustained by phased and cyclical actions and interactions, planned for, managed and somewhat contingent was useful for understanding care workers’ interactional needs in particular situations. the perspective of trajectories provided guidance during data collection (e.g. in interview question design), carved out a perspective on the phenomena during analysis (e.g. when generating scenarios) in this setting and provided a basis for enriching an understanding of staff’s interactional needs. the perspective also informed the description of particular features of new technology (e.g. information on staff work cycles) and issues pertinent to the implementation of this new technology (e.g. supporting sharing of information to engender trust). for example, a resident who was in the ‘high’ cycle of manic depression would require particular attention, monitoring, plans and actions depending on how her condition progressed. the care workers also would require particular alertness to the phase of this resident’s condition and the resident herself may feel she particularly needs to know which staff are on duty and how these staff could best be contacted – these describe particular broadcast needs which can be met using situated display technology. thus, we mostly agree with reddy and dourish (2002) concerning trajectory’s strength in describing the way people coordinate and work together, although it is easy to mistake the temporal aspect of trajectory for what reddy and dourish (2003) describe as a rhythm. we believe that the difference is one of perspective: trajectory offers a personalised and subjective view on the various aspects and consequences of, in our case, care work surrounding ex-psychiatric hospital patients living in a residential care facility, whereas rhythm draws attention to a more event-driven view on the repetitions and patterns in such care work. our key argument regarding the usefulness of trajectory is threefold. firstly, the concept of trajectory itself helped alert us to aspects of everyday care work that might otherwise be missed while doing fieldwork, such as care work being shared (in a multidisciplinary team for example) and being both routine and contingent or unpredictable: certain jobs had to be negotiated and performed during a shift and unexpected emergencies could also occur. secondly, particular trajectory concepts and subconcepts helped tune the fieldworker to important segments and cycles of ongoing work (phasing), plans, procedures and rules (scheme) and the detail of how these are carried out (management) when reflecting on the fieldwork. thirdly, the descriptions that emerged from this particular orientation are longitudinal, multi-dimensional, and have multifarious granularity. the care trajectory describes how care continued from one shift to the next, one day to the next, one week to the next and so on. it also describes care for the physiological, temporal and social aspects of care: residents had good and bad days and some residents’ conditions were more acute or degenerative than others; medication had to be distributed at particular times; and residents and staff alike needed to talk to someone at particular times. the trajectory-oriented descriptions can also be quite general, illustrated by the finding that there are three main phases in a day, or quite specific, illustrated by the finding that the main part of a shift, in turn, had its own set of phases such as an initial phase of task allocation. finally, and not unrelated to the last point, trajectory was important for evoking descriptions of needs beyond the ‘work’ aspect of care. scenario 4 and 6, for example, centred on supporting residents’ therapeutic needs and seemed the least ‘work’ oriented: there was little sense of goal, task or management of the physiological aspects of mental illness or negotiated, temporal order in either. instead, they reflect the sentimental and social aspects of carers’ day-to-day interactions dealing with an ongoing illness trajectory (strauss et al., 1984) and the need to cultivate australasian journal of information systems volume 13 number 2 may 2006 102 an organisational context to support the healthy progression of a psychiatric condition through its phases. the sensitivity to trajectory alone does not magically provide technology designs, but coupled with knowledge of the sites, clusters of activity and divisions of labour at the setting gained through fieldwork, the chance of technology design grounded in the key concerns of the setting is increased. this sensitivity and the fieldwork do not eliminate the need for the particular design approaches: for example, we used a design workshop to bridge into design. but part of the challenge here was making sense when the nature of the setting meant that limited proximity to participants was possible. thus, for instance, handover, and thus important conversational exchanges, could not be observed due to reasons of resident confidentiality, although the visits to the sites that necessarily surrounded the in-depth interviews informed the findings. this ‘distance’ was mitigated through the deployment of concepts and subconcepts used to describe not dissimilar contexts, i.e. trajectory, phasing, scheme and management, as well as particular research techniques such as repeated member checking (guba and lincoln, 1989) and discussions within the research team during data collection. randall et al., (2005) suggest that there are disciplinary “sensibilities” in cscw that describe “the kind of things that we often find” and provide us with “a way of looking”. these “tropes” are not exhaustive but instead offer a guide to doing design-oriented fieldwork while offering a compromise between thick ethnographic descriptions and the non-indexical categories and, at times, dehumanised descriptions emerging from approaches such as grounded analysis (glaser and strauss, 1967). like reddy and dourish (2002) we believe trajectory established an analytical focus for the work at the setting that evolved as it progressed: what were the important work groupings? what important patterns of work occurred and recurred in a work day? what concern did the carers have? what mechanisms did the carers have for continuing resident care? how did the carers carry out management plans and directives? what knowledge did the carers have that enabled them to do their jobs? what did the carers use particular technologies (e.g. the phone) for? these questions, like the ‘tropes’ described by randall et al. (2005), provided important guidance for presenting the nonlinear threads of the design-oriented fieldwork described here. these same questions also offer the possibility of being used in other settings where collective care is important and ‘work’ is multidimensional, ongoing and contingent in nature. references bardram, j., bossen, c. (2005). mobility work: the spatial dimension of collaboration at a hospital. computer supported cooperative work 14(2). 131-160. barley, s. r. (1988). on technology, time, and social order: technologically induced change in the temporal organization of radiological work. in dubinskas f. a. (ed). making time: ethnographies of high-technology organizations (pp.123–169). philadelphia: temple university press. cheverst, k., fitton, d., rouncefield, m. and graham, c. (2004). ‘smart mobs’ and technology probes: evaluting texting at work. in proceedings of the 11th european conference on information technology evaluation (ecite 2004) (pp. 73-80). amsterdam, holland: royal netherlands academy of arts and sciences. clarke, a. a social worlds research adventure: the case of reproductive science. (1990). in cozzens, s. and gieryn, t. (eds). theories of science in society (pp. 23-50). bloomington: indiana university press. cooper, a. (1999). the inmates are running the asylum. indianapolis, usa: macmillan. 103 australasian journal of information systems volume 13 number 2 may 2006 crabtree, a., hemmings, t., rodden, t., cheverst, k., clarke, k., dewsbury, g., hughes, j., rouncefield, m., (2003). designing with care: adapting cultural probes to inform design in sensitive settings. in proceedings of the 2004 australasian conference on computer-human interaction (ozchi2004), (pp. 4-13). brisbane, australia: ergonomics society of australia. crabtree, a. and rodden, t. (2004). domestic routines and design for the home. computer supported cooperative work, 13(2). 191-220. fitzpatrick, g. and kaplan, s. (1997). designing support for remote intensive-care telehealth using the locales framework. in proceedings of designing interactive systems (dis’97) (pp. 173-184). amsterdam: acm press. fitzpatrick, g. (2003). the locales framework: understanding and designing for wicked problems. london: kluwer academic publishers. gaver, w.h., dunne, a. and pacenti, e. (1999). design: cultural probes. interactions, 6(1). 21-29. glaser, b.g., & strauss, a.l. (1967). the discovery of grounded theory: strategies for qualitative research. chicago: aldine publishing company. graham, c, cheverst k, fitton d, & rouncefield m. (2005a). “how do you turn a duck into a soul singer? put it in the microwave until its bill withers”: some social features of a simple technology. in proceedings of the less is more conference 2005, microsoft, cambridge, april 2005. graham, c., cheverst, c., and rouncefield, m. (2005b). going more public: situated display design through co-realisation. in proceedings of dux 2005: conference on designing for user experience, san francisco, 3-5 november 2005. hartswood, m., procter, r., slack, r., voß, a., buscher, m., rouncefield, m., and rouchy, p. (2002). co-realisation: towards a principled synthesis of ethnomethodology and participatory design. scandinavian journal of information systems, 14(2). 9-30. kuutti, k. (1996). a framework for hci research. in nardi. b. (ed.) context and consciousness (pp. 17-45). cambridge, ma: mit press. nardi, b., whittaker, s. and bradner, e. (2000). interaction and outeraction: instant messaging in action. in proceedings of the 2000 acm conference on cscw (cscw’00) (pp. 79-88). philadelphia: acm press. pinelle, d., gutwin, c., and greenberg, s. (2003). task analysis for groupware usability evaluation: modeling shared-workspace tasks with the mechanics of collaboration. acm transactions of computer-human interaction, 10(4), 281-311. preece, j., rogers, y., sharp, h., benyon, d., holland, s., and carey, t. (1994). human-computer interaction. harlow, england: addison-wesley publishing company. randall, d., harper, r., and rouncefield, m. (2005). fieldwork and ethnography: a perspective from cscw. in proceedings of the first ethnographic praxis in industry conference (epic 2005) (pp. 81-99) redmond, usa, november 14-15 2005. reddy, m. and dourish, p. (2002). a finger on the pulse: temporal rhythms and information seeking in medical work. in proceedings of the acm conference on computer supported cooperative work (cscw’02) (pp. 344-353). new orleans, la.: acm press. reason, p. (ed.). (1994). participation in human inquiry. london: sage publications. schatzman, l. & strauss, a. (1973). field research: strategies for a natural sociology. new york: prentice hall. schuler d., and namioka a. (1993). participatory design, principles and practices. hillsdale, nj: lawrence erlbaum associates. shibutani, t. (1955). reference groups as perspectives. american journal of sociology, 60. 522-29. strauss, a. (1978). a social world perspective. studies in symbolic interaction, 1. 119-128. strauss, a., corbin, j., fagerhaugh, s., glaser, b., maines, d., suczek, b., and wiener, c. (eds.) (1984). chronic illness and the quality of life. st. louis: c.v. mosby. strauss, a. (1988). the articulation of project work: an organizational process. the sociological quarterly, 29(2). 163-178. australasian journal of information systems volume 13 number 2 may 2006 104 strauss, a. 1993. continual permutations of action. new york: aldine de gruyter. strauss, a. and corbin, j. (1998). basics of qualitative research, techniques and procedures for developing grounded theory. thousand oaks, ca: sage publications. vetere, f., howard, s. and gibbs, m.r. (2005). phatic technologies: sustaining sociability through ubiquitous computing. first international workshop on social implications of ubiquitous technology. acm conference on human factors in computing systems (chi 2005), oregon, 2-7 april 2005. watson, m. & sanderson, p. (2004). sonification helps eyes-free respiratory monitoring and task timesharing. human factors, 46(3). 497-517. zerubavel, e. (1985). hidden rhythms: schedules and calendars in social life. california: university of california press. acknowledgements this work was funded by the epsrc funded caside project (grant ref: ep/c005589) and the equator and dirc irc projects. the work also builds on work carried out under the epsrc funded casco project and was part funded by a melbourne university abroad travelling scholarship (mats). thanks to dan fitton, steve howard and frank vetere for support, help and advice throughout the research process and thanks to sonja pedell and martin gibbs for photos. special thanks to wally smith for a thoughtful critique of this version of the paper. microsoft word special final.doc ajis special issue december 2004 88 desktop computing distributed cognition in a tax office martin m. nielsen susanne bødker university of aarhus department of information and media studies university of aarhus aarhus, denmark email: mmn@imv.au.dk department of computer science university of aarhus aarhus, denmark email: bodker@daimi.au.dk abstract based on a detailed study of the use of representations in a tax assessment process, this paper presents an analysis of the use of the physical desktop and of paper documents, files and electronic information. this analysis challenges the ways in which the computer desktop is designed and used normally, and we present a number of challenges to user interface design. taking these seriously, means to revisit several taken-for-granted elements of the current wimp regime: the randomly overlapping windows on a non-structured background; the lack of traces of time and past location; and the individualised and non-activity-oriented set-up of the desktop. keywords distributed cognition, workplace studies, desktop metaphor, user interface, digitalization. introduction the starting point of this paper is the taken-for-grantedness of the desktop metaphor. the vast majority of computer applications today are based on a particular version of the (computer) desktop. this is the version where multiple applications run in multiple, randomly overlapping windows that are placed rather arbitrarily on the screen; where there is one cursor and only one active window at a time (van dam 1997). this approach to the desktop is predominant in windows applications, as well as on the macintosh. in short, we are concerned about how the current wimp regime rules the interaction with computer applications brought together on the computer desktop. the taken-forgrantedness of the current desktop style means that most software for e.g. administrative applications are designed with overlapping windows etc. without much deliberate concern: what if there are ways, e.g. of aligning paper documents, that get lost in the computerisation? historically, there was a time before this version of the desktop. experiments were made with tiling a variety of mechanisms to place documents next to each other on the screen, utilising the screen better, magnifying some documents while diminishing others (all of this is actually standard textbook stuff, see e.g. shneiderman & plaisant 2005). the smalltalk browser used connected windows, an element that seemed to have been forgotten, until it was taken up again in macintosh osx. and there are many more of such experimental implementations that have been more or less forgotten in commercial settings. in the following we will present a case study of a work process that is undergoing scrutiny by commercial software companies regarding the potential of “going electronic” while at the same time being a work process where e.g. juxtaposition of documents and use of physical desktop space matters, as we shall see in the following. the paper is based on a detailed study of the work process ajis special issue december 2004 89 based on the framework of distributed cognition. in this paper, our primary focus is on understanding how our findings challenge our understanding of desktop computing. in hci literature, many papers have been dedicated to accounts of studies of how people use their office/desk space, and to support this with technology. most prominent and often still referenced is tom malone’s paper from 1983 (malone 1983). however, that study and many more are primarily focusing on individuals’ personal information management (e.g. belotti and smith 2000, hayashi et al. 1998, kirsh 1995) – piling, filing and retrieving information that somehow “lives” in the office of the particular person studied. hertzum (1999), as well as e.g. malone (ibid.) points to the many roles of documents, and indeed we find similar uses of documents here. however, our study differs from the above mentioned in that we are not as interested in documents that live in the office, as we are in detailed studies of a particular activity that occupies most of the time of the case workers for several months every year; only to be continued in the next step in the annual cycle. furthermore we are not primarily concerned with the individual procedures as they are applied by individuals. our focus is more on the shared procedures as they have been crystallised and are relived year after year within the community of practice of the tax office. the case study the office under investigation in the present study is assessing taxes for self-employed. in most cases the taxpayer is a regular wage earner at a firm, but runs a sideline business. the group of selfemployed, however, includes traditional independent businesses such as carpentry, cleaning service, etc. as well. the self-employed are obligated to submit a balance sheet for their independent business. usually the balance sheet accounts for expenses and earnings, profits, turnover, etc., i.e. how the business reaches the bottom line as it appears from the tax form. the assessment of self-employed normally takes place by examining if the profits of the independent business are well accounted for; if the result is transferred to the tax form correctly; if the information from the tax form and the balance sheets are in concord with “control information” collected from various private and public sources. when the case officer at the tax office finds nothing suspicious the file is placed in the archive. when, on the other hand, something erratic is discovered, further information (vouchers and other documents) is called for. distributed cognition the description above (dressed in tax jargon) of the work process is a common way to think of the task at the tax office. in the following we will try to re-dress the process in the vocabulary of “distributed cognition” (hutchins 1995a, 1995b). we do this because we find that this vocabulary provides us with a privileged access to the workings of the tax office especially with regards to the informational make-up of the tax office. this approach is, as we will argue, valuable in design of informational systems. at first, however, we need to establish the theoretical framework of distributed cognition. distributed cognition originates within cognitive science and the orthodox “cognition as computation” framework (hutchins 1995a, 49). the basic presumption is that cognition and computation can be described as manipulation of representations. in the traditional interpretation of this view representations are mental constructs that somehow mirror salient structures in a task domain and thinking and problem solving take place through the manipulation of these mental constructs. the cognitive ethnographer edwin hutchins reformulates the traditional framework in his pathbreaking studies of ship navigation (1995a) and aviation (1995b). in accordance with traditional views he maintains that cognitive systems are computational and that the study of cognition is the study of representations and algorithms: “[o]ne may ask of this computation how its inputs and ajis special issue december 2004 90 outputs are represented and what algorithms are used to transform inputs to outputs” (hutchins 1995a, 130). the clash between traditional cognitive science and distributed cognition arises on the question as to what constitutes a computational/cognitive system. according to the traditional interpretation the cognitive system is the individual or more particular her mental apparatus. what hutchins discovers, however, is that a whole crew of navigators on a navy vessel satisfy the same functional constraints as has traditionally been attributed to the individual. furthermore, the navigational artefacts comprising compass, charts, alidades, hoey, etc. play an integral part in the problem solving at the navigation bridge. hutchins takes this to show that traditional cognitivism may have mis-attributed the functional makeup of whole socio-technical systems to individuals. instead cognitive science needs to realise that cognition is distributed across systems integrating external/socio-technical as well as internal/mental representations: “i propose a broader notion of cognition because i want to preserve a concept of cognition as computation, and i want the sort of computation that cognition is to be as applicable to events that involve the interaction of humans with artefacts and with other humans as it is to events that are entirely internal to individual persons. (...) [t]he actual implementation of many interesting computations are achieved by other than symbolic means. for our purposes, ‘computation’ will be taken, in a broad sense, to refer to the propagation of representational state across representational media” (1995a, 118). representational media are thus not exclusively mental constructs according to distributed cognition. representations may be materialised in artefacts and in work settings in general. in the navigation example, the chart (a representational media) defines a problem space where the navigators need to map their location (a representational state). as such, the computational processes of a socio-technical system can be examined by studying the “creation, manipulation, and propagation of representational states” (hutchins 1995a, 49) of the system. a great deal of these processes are observable “on the surface” of the systems and need not be extracted from the brains. choosing distributed cognition as the theoretical basis is in a way an experiment in itself (see nielsen 2004). with respect to the findings reported in this paper, we believe that it would have been possible to make rather similar analyses e.g. using the activity theoretical notion of mediation (bødker & bøgh andersen, submitted, bertelsen & bødker 2002, bødker 1991, engeström 1990). we do think that distributed cognition’s explicit commitment to understanding the propagations across informational/representational media constitute a minor analytical advantage in this particular study. however, our main focus in this paper is not on the theoretical underpinnings of our analysis as much as it is on the findings and their relationship with hci (which is discussed e.g. in hollan et al. 2000). studying tax assessment hutchins provides a “cognitive ethnography” (2003) according to which “the mind is not in the brain. the brain is in the mind.” the brain is part of a body participating in socially meaningful activity in a particular setting. in order to understand a cognitive system it is crucial to successively disentangle all these dimensions. in our case study we accede to this analytical framework, that help us elucidate the work practices. we opened the study in an ethnographical manner with shorthand observations of the process, formal conversations and interviews. this helped us identify crucial processes in the tax assessment. following distributed cognition we focused on the propagations of representational states across representational media, i.e. the tax forms, balance sheets, and control information. not surprisingly, the crucial propagations turned out to take place at the desktops in the case officers’ offices. as such the office in general and the desktop in particular constitutes our unit of analysis. in practice we examined how information is distributed across the various representational media at the desktops and how this information is integrated during the work processes. this distribution and integration of representational media across time and space constitutes the metric of the tax assessment. ajis special issue december 2004 91 obviously, nothing would get propagated without human intervention. equally obvious however, is the fact that no tax assessment would take place without the artefacts. we make this point in anticipation of critics of a dehumanising perspective inherent in distributed cognition (nardi 1996, 1998). this is not an anthropological study but a study of a certain activity distributed across humans and artefacts (see also berg 1997 for an elaboration of our position). inspired by jordan and henderson’s (1995) “interaction analysis” we recorded the work processes on video in order to track and capture the complex interactions between case officer and artefacts. moreover, the case officers were asked to “think-aloud” while performing tax assessment. we needed this double perspective – both the case officers’ verbalisations and the material manipulations – to account for the propagations across internal and external media. an important part of the analysis is to understand the representational structures of the central artefacts at the office. to do this we studied their informational content and how this defines the problem space in which the case officers search for solutions. by combining video recordings, verbal protocols, and tax documents collected from each case we have been able to produce detailed interaction protocols of the process. we collected material from 10 assessments corresponding to roughly 1 1/2 hours of video recordings and 40 pages of transcribed protocols. the metric of tax assessment we have already established the activity, i.e. tax assessment. figure 1 illustrates the overall layout of the office setting. as the figure illustrates the office is ordered according to some kind of inputoutput scheme with a mediating processing/work area. the pile of still unassessed files is spatially demarcated from the finished files (dismissed or assessed). as such the physical layout of the office (comprising documents, computers, furniture, calculators, etc.) underpins the informational process, thereby making the metric of the task more explicit. as noticed the central processing area – the desktop – is where the crucial computational processes take place. figure 2 illustrates the desktop in standard configuration observed at all the examined offices. it is important to note that as soon as the assessment begins the desktop is reconfigured according to the moment-to-moment needs of the computational processes. when comparing tax form and control information for example the case officer holds the relevant documents within proximity of the relevant information on the computer screen. however, the standard configuration is reconstructed frequently during assessments. the desktop is ordered in centre and periphery regions roughly defined by the senso-motoric reach (i.e. the bodily constraints) of the case officer. the centre region is occupied by the tax form and balance sheet under scrutiny. the computer screen exhibiting control information is also within the centre region. on the border of the region lies the balance sheet from the preceding year. this document displays background or reference information. further towards the periphery is the calculator, which is employed when needed. ajis special issue december 2004 92 figure 1: the overall layout of the office. figure 2: the desktop in standard configuration. having described the cognitive ecology of the tax assessment as it is constrained by activity, setting, and body, it is possible to turn to the actual computational processes facilitated by the case officer. before moving to the finer details of these, we provide a rough overview of the tax assessment process. to acquire this overview we asked the case officers to categorise a number of the artefacts encountered at the office at their own discretion. they decided to organise the artefacts as they “fit into the work process”. figure 3 illustrates the categorisation established by the case officers. we did make some omissions in the figure in order to obtain our analytical purposes just as we have tried to make the categorisation correspond to the 3 analytical concepts listed on the right. below we comment on how the 3 concepts relate to the tax assessment metric. ajis special issue december 2004 93 artefact topography analytical concepts income tax form balance sheet material control information work instruction body of tax laws assessment instruction pad pen calculator instrument visitation sheet production figures product figure 3: artefact topography of the tax assessment product the process that we are studying in detail here is part of a larger process; the annual cycle of the tax office. in this round, all tax forms are processed. the non-problematic ones are filed in the archive and not touched until next year when reaching a similar point in the cycle. the problematic or erratic forms are put aside for further processing, and classified. a tax officer later revisits them, though not necessarily the same one. when a case is found to be erratic the product of the assessment is a “visitation sheet” classifying the erratic tax form according to level of urgency, expected time consumption, etc. on the sheet a short identification of the encountered problem is also annotated. the classification serves several purposes (see e.g. bowker & star 1999) – it is used both as a way of making visible which types of tax revisions the office carry out and how many (as established by the “production figures” in which each case officer registers the outcome of the assessment), and it is intended as a help for further processing. hence, choosing the appropriate category is a negotiation of many concerns and it has consequences for the further treatment of the taxpayer, as well as of the actual tax forms and numbers. material a number of manipulations need to take place before reaching the end-point of the process, however. when opening a new case, the officer performs a short read-through of the metadata concerning the results of the assessment of the previous year. this “unpacking” (bannon & bødker 1997) of information helps the case officer establish the context of the tax payer and acquire a rough overview on what kind of business it is, special issues to be aware of, etc. upon opening the case the officer places last year’s documents in the periphery of the work field. subsequently, she takes the current year’s documents under consideration. the income tax form and the balance sheets are placed in the centre of the work field, and next to each other to allow easy propagation of figures. these documents constitute the material of the tax assessment. at first the ajis special issue december 2004 94 internal consistency of these documents is checked by comparing if the figures from the balance sheet equal the ones inserted on the tax form for instance. instrument after acquiring a rough overview of the case and having placed the material within reach, the actual control of the current year’s documents is initiated. usually this implies examining whether there are any inconsistencies between the case material and the control information. we categorise the control information as an instrument because it is used to test the validity of the material. as such it serves a manipulative function commonly attributed to tools and instruments. furthermore, the case officers themselves mention the control information (and their own knowledge and work instructions) among their primary tools/instruments. interestingly, they never mention neither case files nor visitation sheets as tools. figures 4 to 6 reconstructs a particular assessment and exemplify a typical control process. the circles on the illustrations indicate the primary focus of the officer and what representational structures are manipulated at each step. the figures are accompanied by a short description of each step including extracts of the verbalisation provided by the case officer. at this point we withhold the theoretical concepts of distributed cognition and try to keep the description as “natural” as possible in order to let the reader receive an impression of the work process in its own right. figure 4: establishing difference. the case officer checks the control information to see if the system has discovered any inconsistencies between the information reported by the taxpayer and those registered as control information. this is possible because key information from the tax forms was transferred to the system earlier in the process. at this point the system has localised a difference in the reported and the registered interest expenditure amounting to 37.165 dkk. figure 5: recalculating difference. next, the case officer examines how this difference was established. he consults the relevant pieces of interest information in the system and recalculates the difference between the registered and the reported figures: “then we take 95.870, there, and up here 21.259 and 45.563 were registered, in total 162.692 [total interest expenditure reported by tax payer]. and the system only knows about 32.655 and 92.872 [interest expenditure registered in the system]. this means that there is a difference of 37.165. that was also what we were told.” ajis special issue december 2004 95 figure 6: dissolving difference. the established difference serves as a definition of the problem the case officer needs to solve. he searches trough the balance sheets to see if they explain the difference. soon the case officer localises a “loss due to depreciation” amounting to 37.313: “here we got the explanation of the difference. it is very close to what we have got here [pointing at the display on the calculator].” this picture fits the frame of distributed cognition well. a representational state (the difference of 37.165) is propagated across a number of representational media (control information, calculator, and balance sheet). by aligning these media in a process of re-representations a representational inconsistency is dismantled. the inconsistency is resolved by localising “a loss due to depreciation” not registered in the system. here the officer’s knowledge of the body of tax laws enters the equation. in the wordings of one of the workers: “the system is stupid. (…) it just says that there is a difference from its perspective. we have to be able to tell if it is a genuine difference or not.” the computations performed at the tax offices are highly unstructured, opportunistic, situated, and data-driven. the desktop reveals itself as a trade-off space where multiple – often opposing – mental, social, material, technical, and bodily constraints seek reconciliation: “the system composed of a task performer, mediating structures, and the task world settles into a solution that satisfies as many constraints as is possible. (...) this constraint satisfaction is a computation” (hutchins 1995a, 300). we use the term “desktop computing” because the desktop serves as an interface between the individual case officer and the computational system at large. this is where the analytical framework of distributed cognition leaves us. by focusing on the construction, manipulation, and propagation of representational media through the system we acquire a detailed account of the workings of the tax office. we argue that this framework grasps the deep structures of the work processes. it omits large parts of the activities at the office only to focus on the minimal constraints, that are crucial to the tax assessment. we still need, however, to go beyond this descriptive stance to challenge the current exploitation of the desktop metaphor in computer applications. the analysis presented above points us towards structures and processes which are not well supported in the wimp regime persistent today. this perspective is crucial because the tax office is under pressure for “going electronic”, i.e. to digitalise all case material. characteristics of the tax desktop in the following we summarise and discuss the findings from the empirical analyses. some of these observations are evident from the example above. others are collected from the video recordings and discussed further in (nielsen 2004). we focus on the salient characteristics of the tax assessment as desktop computing, partly because the desktop has turned out to be a useful delimitation of the analysis of propagation and representations, and partly because we find it intriguing how different this tax desktop is from the standard wimp desktop. we will now turn to the latter of these points while sidetracking the analytical framework of distributed cognition. the theory served its purpose, i.e. understanding the desktop computing, but now we widen the perspective again to make room for observations not well accounted for in terms of manipulation of representations. ajis special issue december 2004 96 opening and closing of cases the case officers rely on highly institutionalised albeit largely unarticulated rules for handling the cases. all the case officers we visited open and close their case folders in a uniform fashion. the files from each year are ordered in separate folders. furthermore, the folders are rotated so that one year’s documents are up side down to the following year’s documents. it is important to the case officers to preserve the orderliness of the files, which is probably the reason for sustaining some documents in specific areas to avoid mixing documents from different years. of course date stamps help maintaining the divisions, but by delegating parts of the responsibility to well-established spatial boundaries the distinctions are no longer in need of conscious deliberation. as such, the physical orientation of the folders supports informational distinctions written on the folders. the browsing of the documents, where last years documents are studied first in their location at the back of the desk, follows the opening of the files. then the tax officer turns towards the actual tax statement of this year, and its accompanying documents. when bannon & bødker (1997) talk about the packaging and unpacking of information this is done rather metaphorically, whereas in this case we see that this process is actually thoroughly embedded in the physical/bodily process of laying out documents and orienting them in a certain order. the senso-motoric constitution of desktop the physical layout of the desktop enforces certain bodily exercises. as evident from the interaction protocol presented above (figures 4-6) the case officer transports figures between computer screen and calculator. having placed the calculator and the computer screen in opposite ends of the desktop, the case officer is gripping the table with one hand, moving closer to the monitor to read figures, drawing himself towards the other end of the table to punch in the figures on the calculator. in this case the case officer utilises his body to connect the two instruments. in other cases, however, the case officer reconfigures the desktop to minimise his bodily engagement. when comparing the turnover figures from the balance sheet to the control information for instance, the officer places the relevant figures from the documents within proximity of the relevant figures on the computer screen. this supports direct, visual propagation of representational states across external (non-mental) representational media. the case officers often mark relevant bits of information with fingers or pens while consulting figures in other parts of the documents. it seems to be crucial to the case officer to be able to keep track of a large number of entry points simultaneously. we have seen a tax officer using four fingers on the left hand to point at various documents, while pointing on the computer screen with his right hand. furthermore, when comparing the files of last year with those of the current year, the tax officer compares the material to appreciate if the “look and feel” is the same as last year. if something looks very different it raises his attention. evidently, it is important to understand the role the body plays in the constitution of the desktop. physical annotations/marks we have noticed elaborate institutionalised rules for making physical indications directly in the case material. some signs mean that the figures match, some that they do not. it is also common to write annotations in the case files. these notes provide the case officers with salient clues, which aid the analysis and interpretation of the documents (for themselves and for others) later in the process. with the visitation sheet presented earlier the case is categorised with respect to what steps to be taken next. this categorisation is accompanied by a short note indicating the problem. the ajis special issue december 2004 97 categorisation primarily serves to prioritise cases for the later processing whereas the notes are meant as a help to the later interpretation of the documents when they are returned to. the physical annotations serve the double purpose of helping the here and now individual information processing, as well as they serve as organisational annotations (hertzum 1999) as they are distributed over time and tax officers (bannon & bødker 1997). functional space the seemingly inconvenient constant manipulation and rearranging of the artefacts implies that the case officers reserve specific areas for specific functions. apparently, one region of the desktop is for calculations; another region is for reference information; a third region is for the momentarily handled documents, etc. rather than just segmenting the table according to reach and sight there seems to be salient functional demarcations as well. e.g. despite noticeable inconvenience, the form from last year is read at the back of the table and left there, not pulled toward the reader. the case officers are, in other words approaching the materials where they are, rather much in the fashion described by bertelsen & bødker (2002), which they call zooming with the feet. obviously in this case, there are no feet involved, yet the tax officer still moves around the desktop approaching the right information in its particular location – in some instances this entails substantial bodily engagement as described above. case logic oftentimes the case officers discover inconsistencies when reading through the balance sheets. whenever they come across figures that for one reason or another look suspicious they consult relevant control information. instead of having a predetermined list of controls to execute every time, they adapt convenient strategies that comply with the internal logic of the case. as evident from the interaction protocol above (figure5) the tax officer tries to reconstruct the calculation processes of the taxpayer in order to appreciate this internal logic of the case files. the calculations seem to be done to check how the difference was established, rather than to check the numbers as such. it seems to be important to the tax officer to be able to access, manipulate, and rearrange the representational structures according to the immediate needs for propagation. as such, the desktop undergoes constant reconfiguration to support the case logic. this makes it impossible to determine a fixed algorithm that is applicable in each case. the actual computational process at the desktop is a result of negotiating the constraints imposed by the case logic, the material layout of the desktop, the individual strategies of the case officers, and the institutionalised rules for carrying out tax assessment. as such, the algorithms for the propagation of representational states across representational media can only be spelled out post hoc. however, as discussed by bardram (1997), with inspiration from suchman (1987) rules and procedures are resources for the situated action, and need to be seen as such. they are not systematically imposed on cases, but activated whenever needed. challenging the design of the computer desktop in the following we want to utilise the insights gathered from the case study to explore alternative desktop setups in future digitally enhanced settings. we want to maintain crucial structures exhibited in the current work processes at the tax office while enhancing the computational processes with some of the benefits from digitally supported work settings. we have performed small-scale design experiments focusing on possibilities for and consequences of digitalization of the tax assessment process. most prominently these studies suggest that ajis special issue december 2004 98 digitalisation is unlikely to provide reasonable benefit to the case officer. paper seems to be our preferred media for reading and digesting large amounts of textual information, primarily because screen resolution limits the number of documents that can be showed side-by-side on a computer screen (even if this is projected onto a large screen display). in this article it is not our aim, however, to discuss design possibilities for an it-augmented taxation process per se. neither will we discuss solutions applying augmented reality technologies, such as tagging and tracking of paper documents as interface to a digital solution. instead we will take the desktop metaphor of computer applications literal and confront it with the tax desktop. in rather simple terms we see so many situations where new it solutions continue to be implemented without questioning the current wimp regime. in the light of this we will explore our findings. in addition, we hope that our analysis will contribute to the ongoing reformulation of alternative models of interaction, from post-wimp to virtual and augmented reality. we have chosen to focus on three elements: • the spatial arrangement of the desktop • traces of context and processes • tailoring for joint practices the spatial arrangement of the desktop most strikingly, our analysis shows that the physical desktop spaces are used in various ways other than just as random piles and documents on the desktop. throughout the taxation process, each document has a location on the desktop. other resources have locations in a similar manner, the calculator for instance. exactly the electronic information differs from this, because it is brought up on the computer screen in windows. the physical desktop offers functional spaces that are used to separate and structure the taxation process: last year’s documents at the back of the desk, etc. as we have illustrated, however, not all documents stay in one location. some are moved around as the taxation process takes place, and spatial mobility is in this way essential. what happens if we look at these findings as a “requirement specification” for the design of an electronic desktop? in most of what we recognise as a wimp interface, windows are totally random, disconnected and overlapping windows. over the years we have seen several examples of interconnected spaces and windows: the smalltalk browser allowed for interrelated inspection of program classes and methods; henderson & card (1986) created rooms to provide an activitycentred organization of documents, and e.g. topos (bücher et al. 2002, ørbæk & nielsen 2001) provides means of structuring collections of documents of various types according to activity. while topos does provide activity-centred collections of documents, these collections provide a bottomup structuring of particular documents. in our case on the other hand, we are looking for a structuring of the desktop that is shared across cases, but for a particular activity. i.e. we need each case and its documents to open on the desktop, structured to depend on the step in the taxation process that is currently taking place. in contrast to such solutions, bertelsen & bødker (2002) made a study of an object-oriented debugger where each break would launch a cascade of windows that were not visually or spatially connected. in the study, it was pointed out how the cascading caused a lot of confusion to the users, not least over time (as we shall return to below). we know of no applications where the screen/desktop space is pre-destined to support a particular physical arrangement, such as current case material to the right, last year’s material to the left. as a matter of fact this would probably be very difficult to build using wimp. this is both because the desktop interface is mainly seen as background, not as an active structuring mechanism, and because some sort of classification or typing of document contents is needed. however, we do see a possibility in revisiting some of the ajis special issue december 2004 99 various split screen and tiling mechanisms that were applied in the early years of the pc, since these are closer to the organization that we want. it is furthermore important for the tax officer to be able to see and point to several documents or particular pieces of information at the same time. this is a further challenge to the ruling windows regime, which basically supports only one cursor and one active window. traces of context and processes in the study we see that packing and unpacking are processes that are integral parts of the taxation process; they are physical and take time. the relationship between documents are created and maintained in this process: the history of the case as such is maintained because the separation and ordering of year is maintained when the material is laid out on the desk, and repackaged into the file. this ordering exists alongside the ordering that is created as part of the information processing of the task at hand. this is in contrast to the above mentioned debugger study where each new step in the debugging creates a new cascade of open documents, and where, after a short process, the connection between documents and the ordering of these in time are lost. the challenge is accordingly twofold: to provide mechanisms through which the history of placement and interconnectedness of documents can be maintained and retrieved. and to support the processes of a particular unpacking, which is the process through which the tax officer gets her initial understanding of the case. we project that this means that the user is supported in over-viewing documents as they have been left on the desktop, and opening them one at a time in the order preferred by the individual tax officer. the tailoring for joint practices wimp interfaces offer far-reaching flexibility for setting up the desktop for individual needs–as many documents and applications as one may desire on the desktop (or none), launching menus to the top, bottom or to the right, etc. while these elements may be brought together into a somewhat standardised set-up, what the tax setting demands is for the desktop to fit a particular element of the entire taxation process, in a manner that reflects the common practices of the office, shared by all and by all cases. at the same time the desktop setting needs to be changeable by the individual users (in general for all cases, and as part of the processing of a particular process). what we are after is an activity-oriented tailoring of the entire desktop. by tailoring we mean that the set-up can be created by somebody, shared by the whole group of tax officers, and adapted to their individual needs (by themselves or by somebody else) (gantt & nardi 1992, trigg & bødker 1994). this is in essence very different from the support for the individual to produce a personal desktop look and feel that lasts over time which wimp provides today. conclusion in this paper we have examined a tax assessment as it takes place at the (physical and virtual) desktops of the case officers. the processes observed are targets for commercial software companies regarding the potential for “going electronic” and digitalising case material. in order to recognise potential pitfalls of this process we performed a detailed field study of the tax office. we found the analytical framework of distributed cognition valuable in this study presented here, because it helps us understand the desktop computing at the tax office. furthermore, this concept allows us to rethink interfaces between the individual and the computational system at large. with the above discussion we hope to have illustrated how the study of a rather mundane work process such as this one challenges our conception of the desktop – the physical one, and not least ajis special issue december 2004 100 the computer desktop. even if we stay away from augmented reality solutions and stay with computer programs that run on the computer desktop, taking the challenges seriously means that we need to revisit several taken-for-granted elements of the current wimp regime: the randomly overlapping windows on a structureless background; the non-existing traces of time and past location; and the individualised and non-activity-oriented set-up of the desktop. references bannon, l. and bødker, s. (1997) “constructing common information space” in j. hughes, t. rodden, w. prinz and k. schmidt (eds.) ecscw’97: proceedings of the fifth european conference on cscw, kluver, dortrecht. bardram, j. e. (1997): “plans as situated action: an activity theory approach to workflow systems” in proceedings of the 5th european conference on computer supported cooperative work, lancaster, uk. kluwer academic publishers. bellotti, v. and smith i. (2000) informing the design of an information management system with iterative fieldwork, dis 2000, 227-238. berg, m. (1997) of forms, containers, and the electronic medical record: some tools for a sociology of the formal, science, technology, & human values, 22, 403-433. bertelsen, o. and bødker, s. (2002) “interaction through multi-artifacts” in s. bagnara, s. pozzi, a. rizzo and p. wright (eds.) ecce 11 cognition, culture and design eleventh european conference on cognitive ergonomics, rome: instituto di scienze e technologie della cognizione consiglio nazionale delle ricerche. bertelsen, o. and bødker, s. (1998) studying programming environments in use: between principles and praxis, nwper '98 the eighth nordic workshop on programming environment research, bergen/norway, sunday june 14-tuesday june 16 1998, url http://www.ifi.uib.no/konf/nwper98/proceedings.html. bowker, g and star, s.l. (1999) sorting things out, mit press, cambridge massachusetts. bødker, s. (1991) through the interface – a human activity approach to user interface design, hillsdale, nj: lawrence erlbaum associates. bødker, s. and bøgh andersen, p. (submitted). complex mediation. paper submitted for publication. büscher, m., christensen, m., grønbæk, k., krogh, p., mogensen, p., shapiro, d and ørbæk, p. (2000) collaborative augmented reality environments: integrating vr, working materials, and distributed work spaces, proc. conf. on collaborative virtual environments (cve 2000), san francisco, 9-12 september, acm press, new york, 47-56. engeström, y. (1990) learning by expansion, orienta konsultit, helsinki. gantt, m. and nardi, b. (1992) gardeners and gurus: patterns of cooperation among cad users, conference on human factors in computing systems, 107 – 117. hayashi, k., nomura, t., hazama, t., takeoka, m., hashimoto, s. and gudmundson, s. (1998) temporally threaded workspace : a model for providing activity-based perspectives on document spaces. conference on hypertext and hypermedia. proceedings of the ninth acm conference on hypertext and hypermedia, 87–96. henderson, d.a. and card, s. (1986) rooms: the use of multiple virtual workspaces to reduce space contention in a window-based graphical user interface, acm transactions on graphics, 5(3), 211-243. hertzum, m. (1999) “six roles of documents in professionals’ work” in s. bødker, m. kyng, and k. schmidt, proceedings of the sixth european conf. on computer supported cooperative work, kluwer, dordrecht. ajis special issue december 2004 101 hollan, j., hutchins, e. and kirsch, d. (2000) distributed cognition: toward a new foundation for human-computer interaction research, acm transactions on computer-human interaction, 7 (2), 174-196. hutchins, e. (1995a) cognition in the wild, mit press, cambridge massachusetts. hutchins, e. (1995b) how a cockpit remembers its speeds, cognitive science, 19, 265-288. hutchins, e. (2003) cognitive ethnography, presentation at cognitive science society conference, boston, 01.08.2003. jordan, b. and henderson, a. (1995) interaction analysis: foundations and practice, j. learn. sci., 4, 1, 39-103. kirsh, d. (1995) the intelligent use of space, artificial intelligence, 73, 31-68. malone, t. (1983) how do people organize their desks? implications for the design of office information systems, acm transactions of office information systems, 1(1), 99-112. nardi, b. (1996) ”studying context: a comparison of activity theory, situated action models, and distributed cognition” in b. nardi context and consciousness – activity theory and human-computer interaction, mit press. nardi, b. (1998) concepts of cognition and consciousness: four voices, journal of computer documentation, 22 (1), 31-48. nielsen, m. (2004) desktop computing, unpublished master thesis, university of aarhus. ørbæk p. and nielsen, m.b. (2001) finding hyper-structure in space: spatial parsing in 3d, the new review of hypermedia and multimedia, 7, 153-183. shneiderman, b. and plaisant, c. (2005) designing the user interface, addison wesley. suchman, l. (1987) plans and situated actions – the problem of human-machine communication, cambridge university press, cambridge. trigg, r, and bødker, s (1994) “from implementation to design: tailoring and the emergence of systematization in cscw” in r. futura and c. neuwirth proceedings of cscw 94, acm press. van dam, a. (1997) post-wimp user interfaces, communications of the acm, 40(2), 63 – 67. acknowledgements thanks to michael christensen for inviting us into the world of topos and for sharing of ideas, to the tax officers and management of the tax office in aarhus, and to anders thomsen and ann mølhave for useful comments. copyright susanne bødker/martin m. nielsen © 2004. the authors assign to ozchi and educational and nonprofit institutions a non-exclusive licence to use this document for personal use and in courses of instruction provided that the article is used in full and this copyright statement is reproduced. the authors also grant a non-exclusive licence to ozchi to publish this document in full in the conference papers and proceedings. those documents may be published on the world wide web, cd-rom, in printed form, and on mirror sites on the world wide web. any other usage is prohibited without the express permission of the authors. microsoft word 13 1 marina.doc ajis vol 13, no. 1 september 2005 209 consensus making in requirements negotiation: the communication perspective jenny price jpr@deakin.edu.au jacob l. cybulski jlcybuls@deakin.edu.au +61 3 9244 6847 school of information systems deakin university 221 burwood highway burwood, victoria 3125, australia abstract when developing an information system (is), organizational goals of various stakeholders are commonly in direct conflict. furthermore, individuals often rank their private objectives well over their management's directions. recognising and reconciling all these diverse goals, and reaching agreement among the stakeholders, are prerequisite to establishing project cooperation and collaboration. this paper focuses, in particular, on the negotiation and consensus making during requirements elicitation the earliest stages of the is development process. as requirements elicitation involves rich communication between project stakeholders, we therefore explore negotiation and consensus making from the communication perspective. the resulting model assists our understanding of the communication factors that influence the consensus process during requirements negotiation. introduction requirements engineering (re) is a cooperative learning process where stakeholders from different backgrounds with different experiences and objectives have to communicate to elicit and validate requirement (bustard 2002). not surprisingly this stakeholder interaction as a group provides an opportunity for the emergence of misunderstanding and conflict (eastbrook 1993, sommerville & kotonya 1996, weigers 2000). the catalyst being that even when common objectives are set, agreed to and acknowledged by others, the participants often misinterpret objectives due to the influence of their bias, personality and environment (eastbrook 1993, sommerville & kotonya 1996, weigers 2000. therefore alignment of viewpoints may be necessary to prevent conflict and its associated problems (easterbrook 1994). in the past (sommerville & kotonya 1996, finkelstein & sommerville 1994), it has been argued that viewpoints reconciliation could be achieved by means of their representation in a formal notation and the subsequent formalism manipulation. in our research, however, we take a different view of consensus building, essentially that it is a social process of communicative actions (cecez-kecmanovic & janson 1999, cecez-kecmanovic 2001). this paper further argues that consensus and the associated alignment of stakeholder viewpoints involves not only communication between individuals involved in the is project but also that it relies on the processes associated with the setting of management and organisational goals. thus the ensuing discussion will ultimately lead to the development of a model (see figure 1) representing the communication factors that influence stakeholder requirements negotiation and the consensus making process the ic model. ajis vol 13, no. 1 september 2005 210 intersubjectivity negotiation process conflict consensus making process compromise collaboration corporate culture cooperation power trust knowledge sporadic relationship dependent relationship communication process figure 1 the initial model of influences on consensus (ic) (as resulted from this study consensus the consensus making process has been defined as a group process where stakeholder input is carefully considered so an outcome that best meets the needs of the group can be crafted (deutsch 1973, dessein 2002, davis et al 2000, darke & shanks 1997). surprisingly, consensus does not require unanimity but in order for all participants to the process to accept the decision they must feel that their perspectives and ideas are acknowledged to allow the cooperative dynamic to be created (cecez-kecmanovic & janson 1999). however at this point it must be noted that consensus outcomes do not necessarily involve negotiation as a consensus outcome can be manipulated subsequently changing the state of the consensus from ‘natural’ where all parties have of their own accord, agreed to an acceptable outcome, to ‘forced’ where some of the stakeholders have been openly or surreptitiously coerced into agreement. following is a discussion of the communication factors that can influence the consensus process during requirements negotiation. the negotiation process in respect to the re field we understand that consensus outcomes not achieved through negotiation can lead to stakeholder dissatisfaction and non-acceptance of information systems (coombes 2001). consensus doesn’t occur without some active intervention and bargaining of the participating parties; instead it is commonly negotiated (weigers 2000). and so, the process of negotiation is deemed as having a direct affect on the consensus process, while it also has a natural affect on the emerging conflicts (see figure 1). negotiating in conflict situations is considered one of the main tools to facilitate ajis vol 13, no. 1 september 2005 211 satisfactory conclusion to the entire cycle of re (weigers 2000), and it may require some compromises on behalf of the participants (bustard 2002). it should be noted that consensus normally calls upon negotiation tactics, which when successful may avert conflict situations. lewicki et al (1999) examined various aspects of negotiation concluding that it consists of at least four typical elements. conflict is recognised as the first and the foremost element in negotiation. they argue that conflict situations may, in fact, influence stakeholder expectations of possible negotiation outcomes and fairness in settlement. settlements are commonly achieved through negotiating parties sacrificing their objectives and the expected outcomes. participants in negotiation are more willing to accept such sacrifice to the benefit of their counterpart if they have a positive relationship with that counterpart. in support, drolet et al (1998) argued when counterparts are in a negative relationship they tend to risk disadvantageous inequality in order to maximise their own expected outcomes and to increase the likelihood of obtaining advantageous inequality. the second negotiation element recognised by lewicki (1999) is a degree of interdependence between the counterparts. they believe that all parties involved in negotiation must be reliant on the other party to some extent or negotiation of an accepted outcome would not be necessary. the third element is an opportunistic interaction that provides the counterparts the access to influence each other (lewicki 1999). when negotiating this opportunistic interaction cannot occur without the counterparts communicating in some way. drolet et al. 1998 clearly acknowledge communication to facilitate cooperation in mixed motive conflicts. they have also demonstrated that the type of communication media used to facilitate negotiation influences the outcomes. in particular, the less a medium allows for synchronous, multiple channel expression of emotion, the less it will foster rapport. lewicki et al. (1999) discuss the fourth and final negotiation ingredient being the possibility of agreement. they argue that parties would not even sit at a negotiation table unless they could see the very possibility for reaching an acceptable outcome. the discussion so far demonstrates the importance of communication to negotiation and therefore the consensus making process. it should also be clear that the decision to adopt an appropriate negotiation strategy could determine the consensus outcome as well. conflict the initial stage of is system development, requirements elicitation, provides ample scope for early reconciliation of stakeholders’ opinions and thus also opportunities to avoid possible conflict (somerville & kotonya 1996, wood & webb 2002) (see figure 1). conflict has been defined as any interference in one party’s activities, needs or goals, caused by the activities of another party (easterbrook 1994). easterbrook (1993, 1994) and kirsch (2000) believe that conflict is inevitable in requirements elicitation; they feel that conflict is likely to occur because requirements are necessarily negotiated not captured, as it is commonly believed. they also believe that interdependence among parties and the divergence of interests, opinions or goals amongst the stakeholders contribute to conflict. several authors (hancock 2002, curtis et al 1988, rothstein & butler 1998) have indicated that to achieve consensus, the conflict resolution approach must emphasise communication between participants. this is because consensus is different from other kinds of decisionmaking as it stresses the cooperative development of decision making with group members ajis vol 13, no. 1 september 2005 212 working together rather than competing against each other. for this reason and even though conflict has been included as a factor in the model (see figure 1) we focus on those communication factors that directly contribute to the negotiation process and thus conflict resolution or consensus making. communication indisputably, stakeholder communication is a central element to reaching consensus (sandelin 1999) (see figure 1). bovee and thill (2000) examined the nature of business communication, defining it as a dynamic process involving distortion and interference, and recognising that it has the ability to facilitate and inhibit consensus negotiation. they clearly acknowledge that, in the most general sense, communication involves encoding and decoding of messages by communicative participants. this allows for perceptions, perspectives and interpretations to play part in an individual’s understanding or representation of communicated messages (bovee and thill 2000). if all parties to the message transmitted do not have exactly the same life experiences, then their individual prior experiences give them distinct cognitive viewpoints (bovee and thill 2000, kaplan & martin 1999). therefore with no regard to external interference, otherwise known as ‘noise’, it is almost impossible to expect two parties to a message to interpret it in the same way (bovee and thill 2000, kaplan & martin 1999). intersubjectivity interpretation has been labelled a barrier to communication and a contributor to misunderstanding (bovee and thill 2000). it has been reported that when communicating, people use selective perception, distortion or interference which can inhibit the communication (bovee and thill 2000). easterbrook (1994) further acknowledged that interpretation is based on intersubjectivity or ‘mutual understanding’ which changes with communication and a person’s position in a group at the time of communication (see figure 1). from a sociological point of view habermas acknowledged that language and socialisation greatly affect a person’s opinions and perspectives thus influencing the politics of communication. while applying habermas’ theories to the context of information systems, cecez-kecmanovic and janson (1999) categorise stakeholder communication actions as being: • 'oriented to understanding' when stakeholders aim to achieve their goals through 'communicative action'; or • 'oriented to success' when stakeholders aim to achieve their goal by intervening in the system or, by influencing others by acting strategically and disregarding others’ needs, interests or values. expanding further on the social issues, mills (2003) noted that individuals' interpretation, or ‘sense making’, of organisational communication is related to their emotional engagement and the prevailing issue climate. mills (2003) elaborated that individuals engaged emotionally with the discourse commonly use interpretations of shared communication distinct from indifferent individuals. however intersubjectivity can be achieved when the communicating stakeholders share the perception of the topic of conversation (easterbrook 1994, yin 1994). in fact, as participants increase their amount of shared experiences, or common ground, they concurrently increase their level of ajis vol 13, no. 1 september 2005 213 understanding of a shared issue (bovee and thill 2000). therefore intersubjectivity, as emerging through the continuum of stakeholders' communication, plays a vital role in the individuals' ability to reach an agreement in the consensus process. corporate culture corporate culture has been defined as a set of norms, beliefs and principles that shape the way an organisation’s staff perceive, interpret and communicate with each other resulting in each organisation’s distinctive character (bovee and thill 2000, yin 1993). in the workplace, intersubjectivity and the organisation’s culture influence each other qualifying this factor as having mutual influence on communication and the consensus process (bovee and thill 2000) (see figure 1). compromise, collaboration & cooperation requirements elicitation is considered a profoundly collaborative task that relies on effective communication and interaction between all participating stakeholders (darke & shanks 1997). therefore stakeholder misperceptions of communicative intentions could ultimately lead to misunderstandings and conflict kethers (2001). yet we should not consider good communication the only important influence on cooperative effort klischewski 2001, ransley 2000). in fact, projects benefit greatly from good requirements management principles, open and trusting environment, with business, stakeholders and developers working together in a collaborative and cooperative fashion, all to enable the sharing of viewpoints, clarification of conflict and consensus negotiation. not surprisingly, in any decision making group participants commonly form distinctive and dissenting factions, even in spite of their shared social status (kaplan & martin 1999). on the other hand, as collaboration requires shared perceptions of all parties involved, this is more likely to come from members of a shared social status (spekman 1996). though, it is not unusual for executives, who are usually more familiar with the company’s challenges and direction, to negotiate a consensus amongst themselves that could be quite opposite to that of subordinate opinions (berman & werther 1996). therefore we can confidently acknowledge that compromise, collaboration and cooperation have all been identified as having an influence on the consensus process directly (see figure 1). all three factors must be communicated to have influence on consensus therefore a dependent relationship between communication and the three factors will be represented in the ic model. power, knowledge and trust. deutsch (1973) recognises power as a direct influence on group decision-making. he further remarks that within the group dynamics, group power is facilitated through normative influence which is a social-relational form of influence in which pressure is applied to deviant members to conform to ‘the group norm’ (deutsch 1973). kaplan and martin (1999) raised the social status of individuals within the group as another aspect of minority power. eisenhardt and bourgeois (1986) and flood et al. (2000) supported this noting that chief executive officer (ceo) dominance reduces the degree of consensus that is achieved when group decision-making. deutsch (1973) also acknowledged informational influence as a source of individual or ajis vol 13, no. 1 september 2005 214 minority power saying that a group member might be influenced to accept third party information as being representative of a specific situation or fact even if that is not the case because of the informational power the influencing party has over them or the group. kaplan and martin (1999) supported this recognising expert power as an influence of minority power, defining it as a special knowledge or expertise about an issue that may influence a decision. recognising knowledge or expert power as a sub factor of power justifies its inclusion in the ic model. the power associated with knowledge is limited to situations where there is a disparity in the knowledge level of stakeholders. this will not always be the case therefore the relationship type between knowledge and power will be presented in the ic model as sporadic (see figure 1). bachmann (1999) raised a further influence on consensus trust. he stated that trust creates a reduction of possibilities in a decision making process and thus reduces complexity. this equates to trust having influence on the consensus process which has previously been defined as a decision making process. child and möllering (2001) believe that trust takes on three forms when it comes to people and technology, i.e. trust in technology, organisational trust, and trust in others in the organisational environment (social trust). bachmann (1999) further argued that power and trust are alternative mechanisms to coordinate social interactions efficiently and to allow for relatively stable relationships between co-operating social actors, such as decision making groups. he stated that trust works on the basis of positive assumptions whereas power is based on negative hypothetical possibilities, which is presented by the powerful actor and believed by the subordinate actor to not be in the interest of either side. even though bachmann (1999) classified power and trust as alternative mechanisms, he did not exclude them from occurring in combination. in fact he said that power often appears as a precondition rather than as an alternative to trust. farrell (2002) agreed stating that power and trust are not mutually exclusive. he says that disparities in power affect the way in which the proceeds of trust-based cooperation are distributed, but do not prevent trust from occurring. the acknowledgement of the association between trust, knowledge and power means that trust will also be included in the ic model of factors that influence consensus (see figure 1). farrell (2002) and giddens’ (1991) descriptions of trust established trust relationship to power, as being sporadic for each can exist without the presence of the other. however the relationship that exists between knowledge and trust must be dependent as the power associated with knowledge is diffused if there is no trust in the knowledge. the ic model at this point, the review of the extant literature had given us some preliminary insights as to the factors that could possibly influence the negotiation of a consensus and the relationships that may exist between these factors. a conceptual model of influences on consensus (ic) (see figure 1) was thus developed to initially capture the factors and then elaborate and evaluate their type, their existence and their relationships. relationships were classified as dependent or sporadic. a dependent relationship required that one factor’s influence be dependent on the existence of the other factor and was represented by a solid line connecting the two factors. a sporadic relationship required that dependency occurred in only certain circumstances and was represented by a dashed line connecting the two factors. as the objective of this research was to investigate the influence of stakeholder ajis vol 13, no. 1 september 2005 215 communication on the consensus process the element ‘consensus making process’ was represented as the pinnacle of the ic model. consensus requires communication to occur therefore a dependent relationship between these two factors was obvious. the type of consensus, i.e. natural or forced, is dependent on the presence or lack of presence of negotiation and negotiation requires communication therefore a dependent relationship was illustrated between these three factors. the sporadic relationship between consensus and conflict was also obvious as achieving consensus does not necessarily involve conflict. again to resolve conflict, negotiation is not necessarily required as a resolution can be dictated rather than achieved therefore the sporadic relationship between conflict and negotiation was apparent. there were further indications from the literature that the relationships between power and communication and again between power and trust were sporadic as none of these were dependent on each other for existence. however, there was also evidence that trust could not exist without knowledge and therefore they shared a dependent relationship. common sense indicated that cooperation, collaboration and compromise would be dependent on communication as all of these factors required communication to occur. it was further demonstrated that negotiation was dependent on cooperation, collaboration and compromise as it requires all three to be facilitated. also obvious were the relationships between intersubjectivity, corporate culture and communication. it is common knowledge that an organisations culture cannot exist without communication whether that be verbal or non verbal therefore this indicated a dependent relationship. it is also common knowledge that every communication that occurs, including communication of the corporate culture is subject to an individual’s intersubjectivity thus indicating dependent relationships. research approach as the proposed model of consensus was developed primarily through the hermeneutic interpretation of literature (giddons 1991), it was then necessary to formally evaluate the model, which called upon the second cycle of hermeneutic interpretation as based on the exploratory empirical work. this research involved the elicitation of people’s knowledge and experience, research of which (bovee & thill 2000, carlsson 2002) could effectively involve interpretivist epistemology which acknowledges that understanding of the social world can be gained from participant’s perspective (khazanchi & munkvold 2003). a single case study of one organisation was used to facilitate data collection to improve our understanding of the factors and their relationships. the method's adoption was motivated by the yin's argument (yin 1994, 1993) that the relative size of a sample does not improve the research findings if the parameters established for the research are met. it was decided that semi-structured, face to face, one on one interviews with the individual stakeholders 17 of the case study would be the most efficient and effective technique to provide rich data. representative sampling was used to find an organisation that had or was in the process of developing software requirements for an information system. the selected project had all stakeholders within one office and was nearly complete. the software requirements were being produced for the development of an interactive website for use by the organisation’s existing clientele. it was to facilitate client access to their account details and other general organisation information. five stakeholders from the project group were interviewed, who 17 developers of the specification will also be enveloped under the title stakeholder. ajis vol 13, no. 1 september 2005 216 were representative of the type of participants usually involved in the systems development process (yin 1993). the case study rich data was evaluated using the qualitative method as we had to facilitate collection and interpretation of data that would arise from stakeholder’s memory of events rather than observation and it was deemed highly unlikely that the participants would directly refer to all factors (yin 1993). additionally the level of influence of each factor was to be established based on stakeholder factual and emotional responses and these responses, which called upon qualitative analysis (coombes 2001), and so quantitative methods have not been considered for this research. an inductive approach was used to establish a model that might be useful as a tool for understanding consensus in general, the focus of the investigated social phenomena (hancock 2002). in keeping with the principles of hermeneutics, and to improve the richness of data, a third cycle was performed. following the establishment of the model of factors from the data collected through the case study two focus groups were convened. the first focus group, as consisting of three stakeholder types, were deemed representative of the stakeholders involved in the production of software requirements. this included a project manager, a developer and an end user. this was also the case with the second group of five people who were developers and project managers and all had experience as end users. to seek opinions from a diverse experience base the other criterion was: • that all participants to the focus group must have had some involvement in the production of more than one set of software requirements; • that none were to be associated with the original case study organisation; and • none should be working together. prior to the meetings each participant was provided with a definition of the factors in the ic model. as with the case study interviews the session was semi structured in nature. participants were informed at the beginning of the session that they were expected to voice their agreement or disagreement with others in the group on the position and or inclusion of factors in the ic model. they were also asked to justify any divergence with evidence based on their past experiences. the discussion of each factor in the ic model continued until all participants reached a consensus (sic!) on the position of a factor in the hierarchy of the ic model. from all the data collected conclusions were drawn and presented as findings. limitations due to the level of researcher interpretation of the data collected in this study and the use of a single case study with a small number of participants, the evidence for the ic model factors and the relationships in the data should be concluded as tentative at the moment. the ic model developed in this study can serve only as a guide for future researchers attempting examination of the various aspects of consensus as the opportunity for any one study to encounter naturally occurring situations in which formal, controlled tests of this model would be likely, appear virtually impossible. case study findings throughout the interview process the respondents discussed many issues, which support the ajis vol 13, no. 1 september 2005 217 notion that several factors influence the stakeholder consensus process. the factors highlighted by the interviews include: the communication and negotiation processes, conflict, power, knowledge, trust, corporate culture, intersubjectivity, cooperation, collaboration and compromise. this section reports on the shared views and concerns of the interview participants (here referred to by their code, i.e. p1, p2, ...) as evidence for the inclusion of each factor in the model, verification of each factor’s position, relationships and relationship type. finally, it culminates in the improved ic model, which was then used for discussion by the focus group to assist in further verification of the ic model’s completeness. in respect to the case study, nearly all respondents to the interviews, when asked about their involvement in conflict during the consensus process, denied the existence of any conflict. some participants (such as p5) claimed not to come into contact with work-related conflict at all. however there was also an indication (by p1 and p4) in the responses that conflict was occurring but may be suppressed, not recognised or purposely not communicated. some (p1) also noted that the project group tried to avoid conflict but when it had happened the conflict was resolved by the adjustment of one party’s opinion. furthermore, an acknowledgement was made (by p4) that a breakdown in communication between the factions of the project group actually occurred. although negotiation did not appear to be a major factor in our case study, the established models of group dynamics seem adamant that negotiation is always a major part of the consensus process and therefore should be considered an influence on it. while the literature acknowledged the communication process as another factor of influence on the consensus process, it also acknowledged that through communication, stakeholder actions could influence the consensus process. this was evidenced in the participants' responses. throughout the study it became apparent that the project team featured two distinct factions. there also appeared to be a disparity between the two factions perception as to the amount of communication occurring. most respondents reported that open communication occurred, however the majority indicated that this happened within the hierarchal levels of the project group and not between the levels. only one respondent (p1) felt that communication was open and flowing between the levels. power comes in many forms including ‘minority and group power’ or perceived and actual power. the responses in this study have shown that many forms of power had a significant influence on consensus. they also show that the power influence can have either detrimental or beneficial effect on the project. all respondents have been either a perpetrator or recipient of an exertion of power and in some cases both. all respondents acknowledged the influence of power on the consensus process in their answers. all admitted to either changing a decision or seeing others change their decision based on the influence of some form of power whether that be power of the personality, group power, power of someone’s social or employment position. the responses provide evidence that power is an intrinsic element of the ic model as a factor of influence on the consensus process. power does need to be communicated for it to have an influence on the consensus process. even perceived power is communicated in some way therefore a relationship will be shown between power and communication. as power’s influence on consensus is dependent on communication the relationship type will be dependent. a number of participants mentioned trust to be clearly associated with knowledge (p1, p3 and p5), both being of considerable importance to the consensus formation. in most ajis vol 13, no. 1 september 2005 218 observed cases, trust of an employee required some prior awareness of another person’s knowledge and their ability to make decisions, therefore the ic model amalgamates trust and knowledge as a joint factor, which could also be construed as sub factors of power. intersubjectivity encompasses people’s shared perspective, perception and interpretation, as these all contribute to shared understanding. the responses indicated that perspectives have a large influence on the consensus process. participants have indicated that their perspectives have changed the intersubjectivity of the group or that they have gained new intersubjectivity based on others perspective. it was acknowledged earlier that people distort perception to make things fit into their patterns of perception and that this can inhibit communication. therefore shared perception or intersubjectivity influences consensus by maintaining the flow of communication and will remain in the ic model. intersubjectivity can inhibit or improve communication but communication adds to the creation of intersubjectivity so a relationship exists between intersubjectivity and the communication process. intersubjectivity only has influence on consensus if it is shared. to be shared it must be expressed which cannot occur without communication therefore it has no direct relationship with consensus but does with communication. this relationship type was later shown of casual importance. interestingly, we have observed very little mention of the impact of corporate culture on consensus processes. there appears, however, to be two different opinions on the corporate culture in general. all respondents approached talking about this factor with extreme caution. the respondents did not raise this subject directly and when asked if they felt the corporate culture has any influence over their meetings the answers were often guarded or not forthcoming at all. a few respondents felt they could not comment. one observation from the data was that, in general, this topic made the people interviewed uncomfortable. in fact, only two respondents (p3 and p4) reported any direct influence of corporate culture on their work at all, not to mention the consensus. in fact, the majority of participants stated quite categorically that they felt corporate culture contributed more to a forced consensus rather than a natural one. one (p4) implied that corporate culture contributes in some ways to the increase in the level of communication and the possible success of the project. yet, another (p3) felt that corporate culture influences budget and it decisions rather than department segmentation. while it is not possible to fully assess this situation, it is likely that the corporate culture was contributing to the divide between the two factions. it was previously acknowledged that an organisation’s communication climate is a reflection of its corporate culture. an alternative view is that the perception of the corporate culture could be a result of the lack of communication, which may well be an oversight as opposed to a direct intention. it appears that either of these situations may be occurring in this organisation therefore corporate culture will remain in the ic model as a factor of influence on the consensus process. clearly a relationship exists between communication and corporate culture. due to the influence these factors have over each other the relationship type will be classified as dependent. however it must be pointed out that there is no direct relationship between corporate culture and consensus without communication. the interdependence between collaboration, cooperation and compromise (ccc) was not demonstrated in a direct way, however, most interviewed respondents felt that they compromised, collaborated or cooperated in some way throughout the consensus process. to our surprise, the case study data hinted that in the ic model, ccc should be a sub factor of power rather than the communication process as it was presented in the literature. some participants (p5) felt that they compromised their position by backing down on their ajis vol 13, no. 1 september 2005 219 opinion. others (p1 and p2) linked cooperation with their ability to change their mind to agree with others. these comments have demonstrated that compromise and cooperation involve a relinquishing of power. participants (p1 and p5) acknowledged collaboration to involve acquisition of certain results, which demonstrates that collaboration requires the establishment or strengthening of power. for this reason ccc will be represented as having a relationship to power in the ic model. the relationship type will be presented as dependent because ccc all required a shift in power. as there was very little literature on the influence of familiarity between people on consensus, it was found [22], however indirectly, that familiarity could alter the perceptions of power that is normally associated with social status or personality. this can in turn increase the alignment of intersubjectivity and alter the way people communicate by means of changing language style and tone, or by reducing the level of formality. familiarity can further assist in the avoidance of conflict but it could also create it. has it not been said that familiarity creates contempt; it can create a shared knowledge that results in the inducement or dissolution of trust. familiarity has also been known to aid or hinder in negotiation. it was quite surprising to us that familiarity was not mentioned by many of our participants as a factor of influence on consensus. seemingly, only one respondent (p2) raised the issue of familiarity as a factor of importance to consensus. however, after careful examination of transcripts it became evident that yet another participant had implicitly referred to the feeling of 'being at ease' with their peers during discussion and communication (p5), which can be construed as a facet of familiarity. it was finally surmised that familiarity can be considered to be part of corporate culture. yet we must acknowledge that in is projects frequently not all parties to the consensus process come from the same organisation, therefore it may take considerable time for these disparate stakeholders to perceive culture as having any influence on their work. however, any familiarity external parties have established with other participants, prior to the consensus process beginning, will influence them from the onset of the process. it was also surmised that intersubjectivity and familiarity could be the same factor as both are based on shared experiences and have a strong dependence on communication. at this point we decided to treat the two as separate factors and include familiarity in the ic model as a factor of influence on consensus so it could be further examined through the focus group. this prompted a need for further investigation so familiarity was presented in the ic model as having a relationship with consensus and communication based on its ability to influence all other factors through communication. the relationship type with communication would be dependent as it requires communication to have influence and its relationship type with consensus would be casual as consensus does not require familiarity to occur. focus groups findings during the focus groups a new ic model was produced. at the final stage of the meetings all respondents in the groups agreed that the initial ic model (see figure 1) represented their views on the factors based on their experiences. the researchers took notes on this process for later analysis. the following section discusses responses from the focus groups, which was eventually used to derive the final ic model (see figure 2). the focus groups agreed with the model positions of the factors; familiarity, conflict, the communication process, knowledge and trust, ccc and power. however, in general, the groups did not agree with the positions of intersubjectivity, corporate culture and the negotiation process. ajis vol 13, no. 1 september 2005 220 figure 2 the final ic model (as resulted from the first focus group.) (note that sporadic relationships have been removed from the model.) negotiation process conflict communication process consensus process knowledge, trust corporate culture cooperation, collaboration, compromise intersubjectivity power dependent relationship negotiation and power both focus groups agreed the negotiation process has an association with power. several members of group 2 raised the point that it is impossible to negotiate without encountering some form of power. both groups also agreed that negotiation should be presented at a lower level of the model than power because it has the ability to diffuse all types of power. corporate culture and power there was unanimous concurrence that corporate culture should be presented at a lower level in the hierarchy than power. in group 2 one participant felt that without a power base you would not have a corporate culture because it is established through a demonstration or lack of demonstration of power. in addition, in both groups, some members made statement to the fact that they had seen strong corporate culture suppressing the power of subordinate or weaker stakeholders. therefore, they thought the relationship between the two factors should be presented as dependent. intersubjectivity and consensus there was clear agreement between both groups that intersubjectivity should be a direct influence on the consensus process and therefore should have its status raised in the hierarchy. this belief was based on the opinion that every single stakeholder who enters the elicitation process comes with a preconceived opinion on every aspect of their own reality. the groups generally thought that this opinion may change with an increase in knowledge but intersubjectivity will still influence the consensus process from its commencement. in ajis vol 13, no. 1 september 2005 221 fact several participants from group 2 noted witnessing allegiances based on intersubjectivity that had enhanced the consensus process through resolution of conflict. however they also remarked on witnessing allegiances based on intersubjectivity that were used to hinder the consensus process by creating power bases. ic model revision literature and the case study data were examined to assess the focus groups’ decision on the positioning of ‘corporate culture’ in the ic model. as earlier it was clearly stated that an organisation’s communication climate is a reflection of its corporate culture and that this affects the quantity and quality of the information passing through the organisation. alternatively communication is required to convey the corporate culture. therefore a direct, dependent connection will be maintained between the factors corporate culture and communication process in the model. however, we were also made aware that the communication process and hence corporate culture is influenced by authoritarian management styles, which is a form of power. this supported the focus groups’ argument that corporate culture is determined by the levels of power demonstrated power. another interesting point is in the case study in support of this idea was raised by p5. when asked about the influence of corporate culture on the group, p5 made comment that they are expected to accept all decisions and hence feel party to a forced consensus. this also demonstrates an association between power and corporate culture. therefore corporate culture will be represented as having a direct dependent relationship with power in the ic model. the second factor in question was intersubjectivity. initially one member of the group 1 thought that intersubjectivity should only be represented as having a relationship with the communication process and hence should be lower than communication in the hierarchy. however, after much discussion both focus groups believed that every stakeholder enters the consensus process with preconceived perceptions on every issue to be raised, every person involved and the outcome they desire. they acknowledged that perception is a part of intersubjectivity and hence it would influence every other factor in some way. therefore intersubjectivity should hold the highest ranking in the hierarchy directly below consensus. it had already been established that intersubjectivity influences every aspect of our existence so there was no argument on that point. however, it was important to note that the influence comes from the communication of the intersubjectivity and not the intersubjectivity itself. this point was also acknowledged by the end user in the focus group 1. for this reason intersubjectivity is presented as a sub factor of communication with a dependent relationship (see figure 2). a further factor to discuss is the negotiation process. in general, the groups agreed that negotiation results in a loss or gain of power for all parties to the process. both focus groups also thought that the negotiation process requires ccc to work. not surprisingly, it had previously been noted that negotiation depends on compromise, which requires cooperation. therefore the association between negotiation and ccc will be presented in the ic model. as negotiation can’t occur without ccc the relationship will be presented as dependent. the argument that negotiation has an association with power, as it requires the increase or decrease of people’s power has also previously been discussed by other authors [41, 42]. these authors’ arguments substantiate the suggestion made by both focus groups, to associate negotiation with power. hence the relationship will be portrayed as dependent, based on the knowledge that any negotiation involves a loss or gain of power. the final factor to discuss is familiarity. the groups believed that familiarity and intersubjectivity have the same influence on the consensus process and therefore should be ajis vol 13, no. 1 september 2005 222 in the same level. what was interesting to note from this discussion was that both focus groups generally felt that intersubjectivity and familiarity had the same influence on the consensus process. supporting the thinking that familiarity is actually enveloped by intersubjectivity. for this reason familiarity was removed from the ic model under the assumption that it is a part of intersubjectivity. conclusions this study expanded our knowledge of consensus and provided the basis for further research into the role of consensus in requirements elicitation. the notion of a consensus process has also been defined and the characteristics of consensus identified. through literature review and the subsequent empirical study, it was determined how consensus impacts the requirements elicitation process in software development. the factors of influence on the consensus making process further highlighted various communication aspects. the resulting list of factors that influence consensus includes the communication and negotiation processes, conflict, intersubjectivity, power, knowledge, trust, corporate culture, collaboration, cooperation and compromise. the surprising discovery was that every one of these factors depends on communication for some level of influence. this result makes three important contributions. first this study emphasises the importance of controlled, yet open communication to the consensus process. second, this study highlights the factors that influence consensus so they may be addressed on an individual basis by each organisation so that unique prescriptions may be developed to improve the elicitation of requirements in a project. finally, this in turn contributes to the improvement of project management practices through a possibility of decreasing the project failure rates. references bachmann, r. trust, power and control in trans-organizational relations, in saseconference,1999, madison/wisconsin, u.s.a. berman, e.m. and w.b.j. werther, broad based consensus building, international journal of public sector management, 1996. 9(3): p. 61-72. bustard, d.w., an experience of principled negotiation in requirements engineering, in awre. 2002. carlsson, s. information systems development: participation and intersubjectivity is this just a matter of language and communication? in ecis conference. 2002. gdansk. cecez-kecmanovic, d. and m. janson. communicative action theory: an approach to understanding the application of information systems. in acis, australasian conference on information systems. 1999. wellington, new zealand. cecez-kecmanovic, d. critical information systems research: a habermasian approach, in bled 9th european conference on information systems. 2001. slovenia. child, j. and g. möllering, the development of organizational trust in the chinese business context. 2001. coombes, h., research using it. 2001, london: palgrave. curtis, b., h. krasner, and n. iscoe, a field study of the software design process for large systems, communications of the acm, 1988, 31(11). darke, p. and g. shanks, managing user viewpoints in requirements definition, in 8th annual conference on information systems. 1997: acis. ajis vol 13, no. 1 september 2005 223 davis, a.m., e. yourdon, and a.s. zweig, requirements management made easy. 2000. dessein, w., coordination through authority vs. consensus. 2002. deutsch, m., the resolution of conflict. 1973, new haven: yale university press. drolet, a., r. k. larrick, and m.w. morris. thinking of others: how perspective taking changes negotiators' aspirations and fairness perceptions as a function of negotiator relationships, in basic and applied social psychology. 1998. easterbrook, s., negotiation and the role of the requirements specification, in social dimensions of systems engineering: people, processes, policies and software development. 1993, ellis horwood: london. p. 144 164. easterbrook, s., resolving requirements conflicts with computer-supported negotiation. 1994. eisenhardt, k.m. and l.j. bourgeois, politics of strategic decision making in highvelocity environments: toward a midrange theory, academy of management journal, 1986. 31: p. 737-770. farrell, h., distrust. 2002, ontario: russell sage foundation. finkelstein, a. and i. sommerville, the viewpoints faq. 1994. fisher, r. and w. ury, getting to yes: negotiating agreement without giving in. 1983, new york: penguin books. flood, p.c., et al., chief executive leadership style, consensus decision making, and top management team effectiveness, european journal of work and organizational psychology, 2000. 9(3): p. 401-420. giddens, a., modernity and self-identity. 1991, california: stanford university press. hancock, b., an introduction to qualitative research, 2002, trent focus group. kaplan, m.f. and a.m. martin, effects of differential status of group members on process and outcome of deliberation. group processes & intergroup relations, 1999. 2(4): p. 347-364. kethers, s. capturing, formalising and analysing cooperation processes: a case study, in ecis. 2002. poland khazanchi, d. and b.e. munkvold. on the rhetoric and relevance of is research paradigms; a conceptual framework and some propositions, in 36th international conference on system sciences. 2003. hawaii. kirsch, l.j., software project management: an intergrated perspective for an emerging paradigm. 2000. klein, h.k. and m. meyers, a set of principles for conducting and evaluating interpretive field studies in information systems. mis quarterly, 1999. 23(1): p. 67-94. klischewski, r., commitments enabling co-operation in distributed information systems development. 2001. lewicki, r.l., et al., negotiation. 4th ed. 1999, burr ridge: mcgraw-hill/irwin. 552. mills, c.e. a unique angle on sensemaking about organisational communication during times of change, in anzca03 conference. 2003. brisbane. o'connell, f., how to run successful projects ii. 1996: prentice hall. ransley, p., white paper: a primer on requirements engineering & management version 0.2. 2000. rothstein, a. and c.t. butler, on conflict and consensus: a handbook on formal consensus decisionmaking. 1998, food not bombs publishing. sandelin, r., working with difficult behaviors in meetings, communities journal of cooperative living, 1999. 104(fall): p. 47-50. ajis vol 13, no. 1 september 2005 224 sommerville, i. and g. kotonya, requirements engineering with viewpoints, bcs/iee software engineering journal, 1996, 1(11): p. 2 26. spekman, r.e., consensus and collaboration: norm-regulated behaviour in industrial marketing relationships. journal of marketing, 1996, 31(11/12): p. 832-856. weigers, k.e., karl wiegers describes 10 requirements traps to avoid, software testing & quality engineering, 2000 january/february. wood, c. and t. webb, the human organizational impacts of technology uptake and use, in awre. 2002. yin, r.k., case study research: design and methods, ed. t. oaks. 1994: sage publications. yin, r.k., applications of case study research. 1993, london: sage publications. . microsoft word ajisvol10 1_final_edit.doc ajis vol. 10 no. 1 september 2002 3 the viewpoint paradigm: a semiotic based approach for the intelligibility of a cooperative designing process pierre-jean charrel université toulouse 1 & institut de recherche en informatique de toulouse, 21 allée de brienne, f-31042 toulouse cedex, france. tel: (33) 561 12 87 99 fax: (33) 561 12 88 80 email charrel@univ-tlse1.fr, charrel@irit.fr abstract the concept of viewpoint is studied in the field of the modelling and the knowledge management concerned in the upstream phases of a designing process. the concept is approached by semiotics, i.e. in dealing with the requirements so that an actor gives sense to an object. this gives means to transform the intuitive concepts of viewpoint and relation between viewpoints into the viewpoint paradigm: the sense of an object is the integration of the viewpoints which exert on it. the elements of this paradigm are integrated in a general model, which defines two concepts formally: viewpoint and correlation of viewpoints. the viewpoint paradigm is then implemented in operational concerns which are related with the intelligibility of the designing process. two models of viewpoint and correlation are proposed. they raise of viewpoints management such as one can identify them in the written documents of a project. study framework: the viewpoint concept viewpoints in everyday life the meaning most commonly given to the word viewpoint is that of a position providing a better view than other places. the summit of mount rhune, for instance, affords a magnificent view of the french basque coastline from hendaye all the way to bayonne. the existence of a viewpoint implies that there is something to view, in this case the basque coast, and someone to view it, i.e. one of the thousands of tourists who daily take the little rack-and-pinion train up to the viewpoint. furthermore, the conditions of observation make the viewpoint more or less attractive – clear weather and a good pair of binoculars are, for example, ideal viewing conditions. lastly, the viewpoint stops being a viewpoint when the object observed stops being observable, such as times when mist caps the rhune, the binoculars break or snow prevents the little train from reaching the summit. in the light of the above, the viewpoint is the meeting place of a given natural "object" (the basque coast), a context (the weather conditions and observation devices available) and an actor (the tourist). the observation allows the actor to access to a representation of the object a viewpoint has also the meaning of a specific opinion produced in someone by observing a specific object. in the world of film critics, for example, the book "hitchcock" (rohmer e., chabrol c. (1988)) contains a transcription of the viewpoints of eric rohmer and claude chabrol towards the film rear window by their fellow director alfred hitchcock. although the film still constitutes an object, it is, this time, an artificial one. the representation of the object, i.e. its screening at a matinee, is also given. moreover, the viewpoint is shared by two actors: their opinion is presented, and represented, in a book as a joint opinion. what we have here is a transcription of one of the two film directors' opinions, but one they did not want to personify. also, this opinion only exists because other opinions exist. finally, criticism can be seen as the representation of another kind of film on a different plane to that of the screen: a reader of the book can thus form his/her own opinion of the film by reading eric rohmer and claude chabrol's opinion, even if they have never seen the film. this two examples bind the notion of viewpoint exerted on an a given object, to the notions of actor, representation, and context. in the first case, the representation is also given, in the second case, the actor who exert his viewpoint produces a new representation of the object, accessible to other actors. let us now consider a situation where the object is not given, but produced. it is encountered in the theatre. by definition, a play is the meeting point of the two kinds of viewpoints discussed above, and two production processes. we are looking at three objects here: the written work, the play and the performance. the play as an object only exists if several viewpoints – the director's and the actors' – meet and correlate at each rehearsal throughout the production process. the play is the product of a process at the end of which these viewpoints finally converge. the end of the production process coincides with the beginning of the performance organisation process. each performance has its own context: place, date and time. other viewpoints now come into play: those of each spectator. the spectator, in a sense, is also an actor participating in the performance through his attention and reactions. moreover, each performance contributes to the play and affects the subsequent performance. an exemplary case is jérôme deschamps' play "lapin chasseur". the stage and theatre are divided into two parts, one representing a restaurant kitchen, the other the dining hall. from their seats, the spectators can only see one of the two areas although they can hear stage noise and laughter from the spectators on the other side. during the ajis vol. 10 no. 1 september 2002 4 interval, the spectators change sides and the actors perform the play again: from the viewpoint of the spectators on one side of the stage, the actors and spectators on the other side are part of the play. in this example, a viewpoint is attached to a given object – the script, the play, or a performance –, to a actor – a comedian, the director, or a spectator – and to a context – the rehearsal hall, the date, the place and time of a performance. and two additional elements now appear: • a produced object: the play, the performance • a collective production process of this object. the object produced changes depending on the viewpoint. . thus, from the director's viewpoint, the play is his representation of the script and is produced from the actual written work. the play is also the last rehearsal, at the end of the production process. from an actor's viewpoint, the play is also a representation of the written work at the end of the production process and becomes an object on the basis of which a "representation" is produced in the context of the organisation process. from a spectator's viewpoint, a performance "represents" the play, but it is also a given object. actually, the performance is a "representation" of itself, the polysemy of the term "representation" (i.e. as a synonym of "performance") standing as proof of this reflexive relationship. viewpoints for designing technological systems the theatre metaphor introduces the objects that will be discussed in this paper, i.e.: the technological systems, and more particularly space systems and software systems. let us consider the early designing phases, i.e. the phases during which requirements are analysed and specified. during these phases, the object being designed exists only in the form of a project. it is only a reality in the minds of the client, the contractor, the members of the project team and future users. in other words, the object only exists from the viewpoint of the actors involved. to design the object, these actors will be called upon to come up with a series of intermediate, transitory and accessory objects as well as specifications, prototypes, test sets, beta versions, etc., which can be seen as traces or representations of the different viewpoints exerting an influence on the future object. these representations are expressed during the validation phases of the project when different viewpoints converge, confront each other and correlate. the result of the final validation is the object itself. we can therefore say that the viewpoints on the object precede the object. this is reminiscent of the theatre play in its rehearsal phases, the play being the epitome of an artificial object if ever there was one, and thus a good metaphor of a technological system. an object as an integration of viewpoints? let us pursue the discussion on viewpoints in each of the previous situations. ultimately, every object is relative to a viewpoint: if an object can be observed or referred to using a word, then someone must exist to observe or refer to it. hence, at least one viewpoint on the object must exist. likewise, we can only describe the objects in the previous situations from the viewpoint corresponding to our position: as far as the basque coastline is concerned, the tourist's viewpoint is not the geographer's nor the guide book writer's. all these actors have their "own" basque coastlines. the viewpoints of both directors towards the film by their famous colleague are not identical and are not the same as those of a professional critic, the film's producer or a simple spectator, although in "rear window" hitchcock places the spectator in the director's shoes. and a play is not the same seen from the stage, backstage or the theatre above all if the theatre is split into two sections. the following question naturally arises as a consequence: can an object be the integration of all the viewpoints exerted on it? for the basque coastline, this would consists in defining it as the amalgam of all the elements used to write a guide book, all elements relating to its geography and all other considerations relative to the basque coast. this is tantamount to reducing the object to the aggregate of its viewpoints. however, no matter how numerous they may be, are the representations of the basque coast ever one with the actual coast? it is not, in fact, the entire basque coast that is invoked, but its visible portion only and the person(s) seeing it. "rear window" takes on a sense when we attend a screening of the film, read a critical article, or give our own opinion on it; the main concern of the director, actors and spectators of the play "lapin chasseur" is each individual performance; once launched into space, the spacecraft is only accessible via its representation, i.e. the signals it exchanges with the ground sector. lastly, the software system is an abstract object and therefore its own representation, given to its users. the artificial objects we are concerned with here, the play, the spacecraft and the software system cannot be reduced to the aggregate of their representations, but they could not be produced without the existence of at least one viewpoint. ajis vol. 10 no. 1 september 2002 5 it is the question of the object status to be produced with respect to its representations which is now posed. we do not intend to take up a stance among the formidable philosophical arguments raging on the conditions governing the existence or reality of an object or an idea. rather, our objective is to highlight the value of the viewpoint concept in attempting to explore this issue. the few examples above are intuitive illustrations of the viewpoint concept and its corollaries in everyday situations, art, science and technology where the concept becomes meaningful. they will serve as a reference to investigate the question posed. how this paper is organised this paper summarises studies and experiments on the viewpoint concept that we performed during the knowledge modelling and management phases early in the designing process of a new product, spacecraft or software system. the next section begins to reconcile the viewpoint concept with the precepts of semiotics on the conditions necessary for a actor to give an object sense. this reconciliation provides the wherewithal to transform our intuitive notions on viewpoints into a paradigm. the elements of this paradigm are then integrated into a general model which formally defines two concepts: viewpoint and correlation of viewpoints. lastly, we will present a translation of this model into an object formalism. then we present an implementation of the viewpoint paradigm. it gives rise to operational concerns relating to the intelligibility and control of the designing process, particularly controlling project convergence towards a final specification. this section presents two models of viewpoints and correlations: they issue from an experiment performed as part of a research effort (galarreta d., charrel p.j., rothenburger b & al. (1998)charrel p.j., rothenburger b., trousse b. & vogel c. (1998)) aimed at recognising and eliciting viewpoints and their correlation on the basis of written documents produced during a collective designing activity. the models are presented in an object-oriented formalism. at least we summarise related works investigating the viewpoint concept in the fields of requirement engineering, and concluding remarks about this initial anchor point for the viewpoint paradigm. viewpoints as a paradigm for designing in the introduction, we described situations suggesting that viewpoints could be the key of a paradigm in the sense given by thomas kuhn and whose characteristics are defined by edgar morin in (morin e. (1991)): as a social issue, a paradigm is a vision of the world that is neither verifiable nor disprovable, which is accepted as an axiom, excludes the issues it does not recognise, generates a feeling of reality and is recursively connected to the reasoning and systems it generates. the statement on which our paradigm is founded could be the following: the sense of an object is the integration of the viewpoints directed at it. two key points give rise to a definition of the elements in this paradigm: the viewpoint concept is central to two processes, the process whereby an object achieves sense and the process whereby actors in the project aimed at designing this object communicate amongst each other. the first key point is to take into consideration only objects that have a sense and actors that give these objects sense. in this way, the object to be designed and an actor participating in the project are not isolated entities: the object has sense when it is connected to how it is interpreted by an actor in a context through a representation that takes on the form of a statement using a symbology. any representation of an object is thus subjective and contextual. this identifies in a wider sense the object's sense and the result of the process used to design this object. the second point arises out of the following observation: the act of designing a space system, a satellite and its ground segment for instance, brings into play a great many technical, organisational and financial skills to find solutions to problems such as: electrical power supply, technical constraints, signal transmission, controlling cost prices, and managing and coordinating teams, over a period that may last for a very long time. it can be said that all actors contributing one of these skills to the project has his or her "own" satellite that must be integrated with that of his partners. the quality of communications between project participants is therefore a key factor of the project's success. indeed, in this concurrent engineering context, the actors are exchanging partial, incomplete and even contradictory representations. the basic idea of the research we undertook was to no longer only take into account the representations of the future object, but also the object itself and its designing process in all their manifold complexity, in the sense given by j.-l. lemoigne to this term as per his definition of a complex system (lemoigne j.l. (1990)): "… a system that cannot be reduced to a finite model, no matter how complicated, stochastic and sophisticated it may be, and irrespective of its size, number of components and the intensity of their interactions". ajis vol. 10 no. 1 september 2002 6 this position is conducive to a global – systemic – view of objects, actors, representations and the design process. semiotics is encountered at this point in its two flavors, english and french. we now demonstrate that the viewpoint concept is based on the concept of "sign". we shall then go on to define the elements of our viewpoint paradigm. lastly, we present a model of these elements in an object-oriented formalism. the semiotic foundations of the viewpoint concept semiotics postulates that the only sources of sense are differences and provides a set of tools to identify them. today, semiotics covers a number of "sign" theories: the sign is considered by some to be an element of the meaning process and as an element of the communication process by others. in each of these approaches, we will recognise several elements connected to the viewpoint concept discussed in the first section. viewpoints and meaning process peirce's english language semiotics defines a sign as "something that takes the place of something for someone with some respect for some reason " (peirce c.s. (1932)). according to umberto eco (eco u. (1980)) "with some respect" means "that the sign does not represent the entirety of the object but rather – through various abstractions – represents it from a certain viewpoint or with a view to a certain practical use". claude vogel adds in (galarreta d., charrel p.j., rothenburger b & al. (1998)) that "peirce's sign is conceptual and relative" and quotes peirce: "nothing is s sign if it is not interpreted as such". thus, according to peirce, a sign only acquires the status of a representation of an object in a relationship of three terms encompassing the object, the sign as a signifier or expression, and the signified as the content of the expression. the triad (object, signifier, signified) is often represented in the "semiotic triangle". figure 1 shows this triangle and gives some of the equivalent terms used by saussure, morris, and hjelmslev (eco u. (1979)). peirce adds two notions to his semiotic triangle: the first is that a content can itself be an expression (a sign), and the second is that the context decides which content to associate with the expression to give it sense. claude vogel quotes pierce (galarreta d., charrel p.j., rothenburger b & al. (1998)): "suppose we hear a word out of context, the word "grenade" for instance. (this) word has a sense. i know this for sure but i do not know which sense. i will only know if the sign "grenade" points to another sign acting as its interpretant, "city", "fruit", or "weapon". let admit… i come from spain. the word "city" is not… sufficient to me. but the interpretants of "grenade" do not stop there, in the syntactic order: spain, iberian peninsula, europe, mediterranean sea, arab occupation, islam, architecture, generalife, alhambra, garden, lion, an so on". object contentexpression signified concept mental image interpretant signifier symbol sign fig. 1 the semiotic triangle in the archetype of the meaning process represented by the semiotic triangle we find several elements connected with our viewpoint notion: the object and the conditions for an expression to be qualified for representing an object. viewpoints and communication process in our work on multiple viewpoint design, the links between the actors in the design project were also taken into consideration so that the representations they produce have a sense, i.e. so that they can be understood, evaluated, revised, modified and assembled with a view to final consistency. in the case of a space system, the first steps in the designing process generally involve producing documents in natural language. "formal" specification documents are only produced at the end of the process. all these documents are therefore expressions of viewpoints and their links. we now encounter the second semiotic foundation of the viewpoint in connection, this time, with the french language approach. according to this approach (deledalle g. (1990), greimas a.j. (1979)), semiotics provides a tool for visiting a document like a monument. it thus presents text as being inseparable from the author and the reader: the sense of a text is that given to it by its author, which is also the sense retrieved by the reader. thus we may observe that a ajis vol. 10 no. 1 september 2002 7 text is a communication medium if a reader is able to retrieve a sense from it, albeit a different one. a text thus only contains conditions placed by the author for retrieving the sense, these conditions being relative to the form – structure, presentation –, the literary genre, the language – english, terminology –, the style, etc. a coauthored text is already the product of a pooling of several sense retrieval conditions, i.e. those recognised by its authors. according to the philosopher bernard stiegler1 "all fabricated objects constitute memory because they are traces of an action". this states the fact that a fabricated object constitutes the trace of an action whose results are stored in the form of an object. in this respect, a text is an incomplete object: if, in the first degree, it is the memory of an act of writing, it requires another action for a sense to be reconstructed from it. a text is never completely finished: the reader always adds the follow-on. in other words, a text is a kind of specification that must be realised by the reader. it is a class whose instances are the multiple interpretations of its readers. the french semiotic method is conducive to highlighting the differences in a text. these differences are the sources of sense. they are elicited using three types of analysis. the first analysis consists in identifying "narrative programs" or contracts: these feature an "actant" – person or thing – that performs or has an action performed which is then evaluated. its opposite is an anti-narrative program which creates an opposition and is a source of sense. the second analysis looks at how the roles and interaction of the actants combine. roles are expressed on the basis of canonical types. the third analysis identifies the "isotopies" of the text, i.e. its themes and the related elements of the text. all these constructions are representations of the text. they throw direct light on invisible aspects of it. these are, in fact, relationships that can be presented in the form of graphs showing terms and concepts that although present in the text, cannot be isolated from it because of the latter's linear form. other elements linked to the viewpoint concept are thus found in these french semiotic analysis schemas: actors, objects, and relations between actors and objects. remark the absence of the interpret in peirce's semiotic triangle does not exclude the communicational aspect of the sign. peirce also formed the semiotic triangle for communication-related purposes: "the interpretant is the idea the sign gives birth to in the mind of the interpret – even if the presence of the effective interpret is not required" (eco u. (1979)). according to jürgen habermas’s commentary (habermas j. (1991)), "before accomplishing its function as a representation, the sign must also be interpretable". peirce excludes the actor and the context from the semiotic triangle in order not to attach the object's signifier and the signified to them, but he explains that "something cannot stand for something else without standing for someone". elements of the viewpoints paradigm reconciling the viewpoint notion with semiotics first leads to a formal definition of the basic concept of viewpoint and then to a set based representation which is used to define this of viewpoint correlation let us assume a given universe of discourse. definition 1 a viewpoint is the interpretation of an object by an actor: it is defined by the object on which the interpretation is performed, the actor performing it, the expression and content of the interpretation of the object by the actor, and the context in which this interpretation is performed. a viewpoint thus comprises five poles: • the actor holds at least one viewpoint, in the context of which he or she produces an interpretation of the object. • the object is the integration of the interpretations produced by the actors exerting a viewpoint on it. • the context is the condition governing the way the actor exerts his or her viewpoint. it can signify, according to the case, either the place from which the viewpoint is exerted, the moment in time it is exerted, the tool used by the actor to exert his or her viewpoint. 1 conference given by bernard stiegler on 3 february 1996 in blagnac during the "from the body to the technological corpus (du corps au corpus technologique)" talks. ajis vol. 10 no. 1 september 2002 8 • the expression is a statement, expressed in a symbol system, that is attached to the object by the actor within the context of the viewpoint to express his or her interpretation of the object. • the content is the meaning given within the context by the actor to the object by means of expression. the semiotic triangle does not mention the context in which the sign is produced nor the actor that produces it. a viewpoint can be considered to be a semiotic triangle situated for its actor in his interpretation context. actor context expression object content fig. 2 viewpoint and semiotic triangle figure 2 proposes a representation of the viewpoint as a quintuplet of poles: < actor, object, context, expression, content > in our paradigm, we do not attempt to define the object and the actor before the viewpoint. the viewpoint is pre-existent to the object and the actor. here, the concept of viewpoint is given a status of paradigm in the sense of emmanuel levinas who founds his paradigmatic method on the idea that "…it is the act that makes the form emerge in which it recognises its model, unseen until this instant " (levinas e. (1994)). the content, certainly the most "abstract" of the five poles, contributes information, or rather knowledge, given by the viewpoint on the object. in this connection, let us quote g.g. granger who wrote in (granger g.g. (1994) p. 33) "… to say that a notion has a content is presupposing that by attributing it to a subject or by entering into a relationship, it provides us with information. information is expressed in a statement virtually opposed to many other statements which, from a certain viewpoint, can be seen to have the same form". when the symbolic system used in expression is a formal one, i.e. when semantics is associated with each statement, the two poles expression and content merge and correspond to what we invoked at an earlier stage by the term representation. definition 2 the representation of a viewpoint is the pair < expression, content >. definition 3 the view of a viewpoint is the triplet < expression, content, context > definition 4 the universe p of viewpoints is the cartesian product: p = a x o x c x e x co in which a, o, c, e, co designate respectively the aggregate of actors, objects, contexts, expressions, and contents in the universe of discourse. . we use dotted notation to designate one of the components of a viewpoint. for example, p.a designates actor a of viewpoint p in universe p. for the following definitions, the universe is implicit. actually, it is the reference from which all the viewpoints can be defined. ajis vol. 10 no. 1 september 2002 9 definition 5 we will use <x>-correlation to designate any transitive relationship on x x x, where x designates one of the five sets of poles or the cartesian product of several of them. examples • an <a>-correlation is a transitive relation on a x a. • a <c, e>-correlation is a transitive relation on (c x e) x (c x e). this notion is extended to viewpoints in the following way: definition 6 two viewpoints p and p' are said to be <x>-correlated if an <x>-correlation exists between two of their poles. examples • in the universe of discourse of an organisation, the relationship "works in the same department" is an <a>-correlation. • in situations where the object is given, whether natural or otherwise, viewpoints are naturally correlated by the object, or <o>-correlated, and the relationship is the identity. this is the case, for instance, of the basque coast, where the <o>-correlation is the identity relationship. • for the example of the film ”rear window”, three <o>-correlated viewpoints exist: the viewpoints exerted by each of the two directors, and the viewpoint whose actor is the pair (chabrol, rohmer), whose expression is the critic on "rear window", and whose context is the book "hitchcock". definition 7 when two viewpoints are correlated by expression and content, i.e. <e, co>-correlated, they are said to be correlated by the representation, or <r>-correlated. definition 8 an <x>-correlator is a function: cr : x � x where x is one of the sets of poles or a cartesian product of several of them. notations when there is no ambiguity, cr is said to be a correlator. 2 viewpoints p and p' correlated by cr are crcorrelated. remark the transitive closure of a correlator defines a correlation. this notion is extended to viewpoints in the same way as <x>-correlation. examples • in the universe of an organisation, "is the hierarchical superior of " is an <a>-correlator over all viewpoints. • suppose that the expressions of two viewpoints p and p' are two knowledge bases made up of logical formulae, an <e>-correlator can be constructed to define the formulae of p' deducible from p. definition 9 a system of viewpoints s is defined by the n-uplet: < a, o, c, e, co, p, <x>-correlations, <x>-correlators > where: • a, o, c, e, co designate sets of actors, objects, contexts, expressions and contents, • p designates the universe of viewpoints, ajis vol. 10 no. 1 september 2002 10 • <x>-correlations and <x>-correlators designate the two sets of correlations and correlators defined on p definition 10 let s be a system of viewpoints. a set sp of viewpoints of p is said to be correlated if there exists a correlator cr of all x-correlators of s such that the viewpoints of sp are correlated 2 to 2 by cr. definition 11 let s be a system of viewpoints. a graph gp of viewpoints on s is defined as follows: • a node of gp is a viewpoint of p, • an edge between two nodes p and p' exists if there exists a correlator cr of <x>-correlators such that p and p' are cr-correlated. graph of viewpoints and intelligibility of a designing process according to the viewpoint paradigm, the process whereby a technical object is collectively designed gives rise to a particular viewpoint graph. here, the contexts of all viewpoints are the various milestones of the process. there exists an initial and a final node respectively related to two viewpoints whose actor is the project's customer, and intermediate nodes which are the various viewpoints exerted throughout the process. for the initial viewpoint, the object is the assignment on the customer's purchase order, the context is the instant the project is launched, and the representation is the entire set of documents produced for the customer and the design team. for the final viewpoint, the object is the designed product – and all the knowledge acquired on its manufacture, maintenance and operation –, the expression is the integration of all the expressions of the viewpoints in the final context, and the content then represents the formal acceptance by the customer of the designed object. the product only exists physically at the end of the designing process, and it is the object of all the viewpoints exerted in the final context of the process. the process is complete when the viewpoint of the customer actor is able to prevail. let us consider the graph of viewpoints for a designing process. we classify the correlators into two types: static correlators and dynamic correlators. dynamic correlators are relative to the design process and support the intelligibility of the process: they ensure its visibility and coherence as discussed in (charrel p.j., galarreta d., hanachi c., & rothenburger b. (1993),charrel p.j., rothenburger b., trousse b. & vogel c. (1997)). the context pole suits to represent the time scale of the process. so, dynamic correlators are <c>-correlators, where c context is time. static correlators are all <x>-correlators defined at the other poles or combinations of <c>-correlated viewpoint poles. they can take on the form of reasoning on viewpoints to organise, where necessary, their consistency. reasoning on viewpoints and the process facilitates the discovery and management of all significant differences. here, we once again encounter the semiotic rationale for viewpoints: when an actor produces a new representation during the design process, differences arise along the actor and object axes – actant2 and time in semiotic axiology. examples two remarkable situations can arise in a design process, which are interpreted using static and dynamic correlators: • “validate a representation” is an <r>-correlator which links the viewpoint p1 of the actor who produces the representation to be validated and the viewpoint p2 of the actor who validate it. the <e>component is the identity relationship, and the <co>-component is the boolean function: p1.co � {true, false}. • objects are created throughout the process. they are assembled and modified to constitute the final body of specifications. let us consider the history of the different versions of an object produced by the same actor during the process. each version is related to one viewpoint, and all these viewpoints are 2 unlike french semiology, a distinction is made between objects and actors ajis vol. 10 no. 1 september 2002 11 linked to each other by <a, o, c>-correlators where the <a> and <o> component are the identity relationship, and the <c> component is the relationship “next step”. representation model for viewpoints and correlations object-oriented formalisms are, actually, candidates for serving as a formal media for representing viewpoints and correlations between viewpoints. figure 3 presents a model of a graph of viewpoints, inspired by the metamodel of the uml language classes (rumbaugh j., jacobson i. & booch g. (1998)). the overall composition of a viewpoint is indicated in this model as a class related to five other classes that correspond respectively to the actors, objects, contexts, expressions and contents of the universe of viewpoint. the class poles aggregates the five associated classes with the viewpoint class. the cardinalities 0,1 express the primacy of the viewpoint with respect to its poles. the cardinality * indicates that the components may appear in several viewpoints. a representation necessarily comprises an expression, with or without a content, and a view is compulsorily composed of a representation, with or without a context. the correlations are instances of a class of associations that link viewpoints two by two, and the correlators are the methods of this class. the correlators are declared as methods of the correlation class. correlation of viewpoints pole correlators 0,1 view representation 1,1 viewpoints contentexpression context objectactor 0,1 0,1 0,1 1,1 0,1 * * * * * * * 0,10,1 0,1 0,10,1 1,1 1,1 1,1 1,1 1,1 fig. 3 object-oriented model of a graph of viewpoints implementing the paradigm of viewpoints for the management of a designing project the viewpoint paradigm generates specific representation models, two of them are presented in this section, for managing viewpoints generated by a designing project. in this section, we discuss propositions for viewpoint ajis vol. 10 no. 1 september 2002 12 models inspired by the studies carried out as part of the research effort on dynamic viewpoints quoted in the first section (charrel p.j., rothenburger b., trousse b. & vogel c. (1998)). the overall objective of the research was to demonstrate by experiment that the notions of viewpoint and correlation could be used and useful to structure, understand and process the results of a content analysis of a discourse. here, the discourse is composed of text that is rarely subjected to this type of analysis: a transcript of a conversation between three space technology engineers. the global results are presented in the form of : • a method and software tool for analysing texts used to identify viewpoints and their changes in the course of time – i.e. their <c>-correlation – throughout the project process, • tool specifications to manage a viewpoints base supporting project management. the framework of research was to study viewpoints and their correlation in the situations of a project for designing the architecture for a new little sized spacecraft. the objective was to provide tools to elicit the viewpoints brought into play during the design process and to highlight those which have effectively contributed to the success of the project. feedback from the experiment enabled to specify a viewpoint-oriented prototype supporting design project management. initially, we set forth this project in the viewpoint paradigm and then briefly describe the analyses performed. lastly, we specify two viewpoint management models. an object-based formalism is used for these models. the designing project in the viewpoint paradigm the context of the action is the following: a proposition for a specification was drawn up during a meeting between a project manager, named d1, and two engineers, named d2 and d3, in the space industry. a video recording and the written transcription of the discussions held during the meeting constitute the traces of the design process. the conversation transcript is the trace of the interventions of the actors d1, d2 and d3, but also of other actors mentioned by d1, d2 and d3 – individuals, companies –, of objects produced by the latter – “antenna”, “propulsion system” –, and of their interactions during the design process with respect to the components of the object being designed – layout of the satellite’s propellant tanks, size of the antennae, mass of the components, etc. this transcription, throughout the entire design process, of aspects relative to the structure, the functions, and the interactions between the different components of the spacecraft being designed can be interpreted as the modelling process of this object and its operating environment. the analyst of the situation is himself or herself an actor who exerts a viewpoint the, object of which is the designing process, i.e. the graph of converging viewpoints, the context is the period of time the meeting lasted between the initial instant at which the object was non-existent and the final instant at which the conversation was stopped by the project manager d1, and the representation is the transcription of the discussion – video representations are not dealt with. this representation is also that of the viewpoints exerted by the three actors in the meeting on the object to be designed. the representations of these viewpoints are the interventions – transcripts – by the actors during the meeting, and their contexts are the instants at which these interventions took place. the object is constructed as dialog progresses by adding and reviewing partial expressions in the viewpoints brought into play. analysis of the experiment let us sum up two analysis that bernard rothenburger and claude vogel respectively carried out (charrel p.j., rothenburger b., trousse b. & vogel c. (1997)) on the discussion transcript. first, the entire text was processed by a terminology classification software – alceste™. this software used statistics to identify the major thematic classes represented in a text. four classes were produced here. one of these classes includes quantitative notions: numerical values, units of measurement, comparisons between values, etc. it corresponds to the technical aspect of the discussion. the text body was then converted into a table of pronounced words classified by speaker and by minute, which enabled to calculate usage frequency of the various classes per minute of statement of each actor, to make statistical estimates on the results, and to produce curves showing trends in these statistics. the curve in figure 4 shows that the designers d2 and d3 display a difference in how they deal with technical terms. actually, d1 is the manager of the design group to which d2 and d3 belong. d1 starts off by explaining the issue by evoking its technical and quantitative components. at a later stage, d2 and d3, who have specific technical skills, appropriate these themes and look for solutions to the various problems posed by d1. the interpretation of the three curves are the following: between d2 and d3, the difference in how they deal with technical terms remains stationed in a fairly narrow interval while the difference they each display with respect to d1 increases regularly in time. ajis vol. 10 no. 1 september 2002 13 time d1-d3 d2-d3 d1-d2 fig. 4 <a, infometrical>-correlation between viewpoints on technical terms the second analysis used a statistical representation of the text as the source and not the text itself by means of semiolex™ tool. the text was divided into 6 periods of 30'. a glossary of keywords was selected from the general index of keywords constructed from the transcript. the keywords in the glossary occur at least twice in the index. on the basis of this glossary, the statistical analysis combines the two data representation planes built from the various samples: frequency distribution tables and co-occurrence tables. part of the results of statistical analysis is represented graphically by lexical networks attached to the cooccurrences. figure 5 illustrates an example of a network and the excerpts from the relevant text: the term "shielding" appears as a pivot for neighbour terms such as "impact" or "pressure" and so defines a kind of semiotic proximity between them. pumped propellants pressure square box shielding impact technology mass -excerpt 472 -d1: i would like to know if we shouldn't look deeper into the shielding thickness that would be required to significantly reduce the impact -excerpt 487 -d2: let me tell you. i don't understand why we don't use the--since we're using pumped propellants, there's no pressure so we can make a square box! and place the electronics in it!. let's put square boxes right there! -excerpt 870 -d3: for the missions we're talking about, we need a minimum strength at these rates because that puts a constraint on technology at some point! or maybe on the mass, since, if we have to increase the shielding, well, maybe not the shielding, you can see how quickly we reach the max! fig. 5 text excerpts with the lexical network which represent them correlation models for the management of a designing project the experiments involving the acquisition and recognition of viewpoints from text performed during the abovementioned research work leads to a proposition for a tool to facilitate the project steering by its terminological and infometrical components. the experiments have brought to the fore some remarkable forms or "regularities". for example, regularities occur during a particular period of the spacecraft design process. for example, the homomorphism of certain curves representative of the discussions between d2 and d3 involving design, or, on the contrary, the exactly opposite gradients of other curves as shown by other curves. ajis vol. 10 no. 1 september 2002 14 we can now deduce the features of a viewpoint management tool capable of assisting actors exerting viewpoints by facilitating or enriching the analyses that have been conducted. schematically, our proposition consists in using an object formalism to represent items of information – partial, not consistent from one to another at any instant of the process, and revisable – about the actors participating in design, and their productions at the various stages when they are made visible by these actors. object-based approaches are an answer to controlling the design process because they provide systematic means for translating the technical, functional and usage components of the spacecraft. for example, the spacecraft’s specification can be translated by a class diagram, its operating specification by a use-case model and statetransition diagram. we can reconcile viewpoints management operations and their corresponding actions expressed in their object representations. terminological correlation terminology is defined as a network of links between terms and concepts, where a term is composed of several words and designates a determined concept of a particular domain, and a concept is a piece of knowledge characterised by an informational definition and its position inside a network of concepts. in this network, two concepts are linked by relationships such as "is synonym of", "is a ", "is part of". the viewpoint notion leads to stress, in a given designing project, several terminologies: • viewpoint terminologies, each of them is local to a viewpoint, and is part of its context; • the project terminology, where synonyms, common terms, shared concepts, are defined. the project terminology is a <c>-correlation between all the viewpoints, and is named terminological correlation. it takes into account the insertion/deletion of terms or concepts. a query language and a visualisation tool can implement the terminological correlation. the query language can retrieve terms and concepts and their respective relationships, by means of operators like the followings: • for a term: number of links with concepts, • for a concept: integration/isolation – number of adjacent concepts – and density – number of linked concepts, • for a network of concepts: dimension – number of nodes and edges –, and density –maximal distance between two nodes, ratio number of concepts/number of terms, ratio number of concepts/number of links. the visualisation tool aims at presenting evolution curves of theses measures along the time. figure 6 presents a class schema which suits to the terminological correlation of the viewpoints. the corpus of written documents, from which terms are issued, are accessible via the "source documents" method of class term. the "classification" method represents the access to classification tools like used in the quoted research project. infometrical correlation the reasoning is now based on the particular representation of the text given by the lexical networks issued from the statistics processed by the statistical tool used in the quoted research action. the infometrical aspect of the viewpoints enriches the analysis, by offering selected views of the corpus owing to the networks of co-occurrences. among these views are: • the network local to a viewpoint, by selecting terms which co-occur with the actor of the viewpoint, • the network limited to a given instant of the designing project, • the networks the form of which is remarkable. like terminological networks, there are several possible measures, accessible by a query language or a visualisation tool: • the number of links between a term and other terms or other networks, • the integration or isolation of a network – number of adjacent network –, the density of a network – number of linked terms, number of links in the network, • the dimension of the network constituted by all the networks – number of nodes, number of links –, the network density – maximal distance between two nodes, ratio number of networks / number of terms, ratio number of networks / number of links. ajis vol. 10 no. 1 september 2002 15 concept name viewpoint classification regularities name network of concepts viewpoint terminology type : synonym, antonym, is-a, is-part-of context term terminological correlation expressionactor 0,1 0,1 0,1 0,10,1 * * * * * content ** * * object (documents) pole 1,1 1,1 1,1 1,1 1,1 ** 0,1 0,1 0,1 0,10,1 written documents time * * * * * fig. 6 object-oriented model of a terminological correlation for corpus of texts where chronology is a significant parameter – as the documents of a project – a computeraided management tool of the history of the representations of the viewpoints can provide qualitative measures like: • the trajectory of a term in the different viewpoints, i.e. its appearance and disappearance, increase of cooccurrences with other terms or progressive isolation, • scenarios issued from an infometrical analysis, i.e. recognition of regularities in the networks with reference to types of scenarios, acquisition of new scenarios. figure 7 presents a class schema which suits to the infometrical correlation of the viewpoints. these correlations can carry out two types of control actions. the first action relates to the management of the evolution of the lexical networks. each actor adds or modifies documents related to a particular feature of the project, or constitute the basis of a new viewpoint. then, these documents will add new terms and new links to the networks. the management of these extensions consists of measuring their influence on the quoted measure indexes. the second action aims at reducing the distance between two networks. synonymy and contradiction between two terms present in two different viewpoints can leads to likenesses in the networks. looking for these likenesses may, on the other hand, lead to collect synonyms and contradictions. the actors of the project are then able to simulate the actions to execute in order to delete these contradictions, or take these synonyms into account. ajis vol. 10 no. 1 september 2002 16 name viewpoint classification network of all lexical networks context term infometrical correlation representationactor 0,1 0,1 0,10,1 * * * * * * * * object pole 1,1 1,1 1,1 1,1 ** 0,1 0,1 0,10,1 id type internal structure … commentaries lexical network pivot type extract number source document time * * * * fig. 7 object-oriented model of a infometrical correlation related works and concluding remarks: viewpoints in requirement engineering the requirement engineering activity is known to collect knowledge, methods, techniques, tools able to analyse and specify all functions and properties that a system to be designed must hold to achieve its mission (nuseibeh b., kramer j. & finkelstein a. (1994)) .this activity is also described as the production process of functional and non functional requirements (leite j.c.s.p. & freeman p.a. (1991)). p. zave mentions in ((zave p. (1997)) about fifteen problems to be solved which relates to the requirement engineering activity. she identifies three classes of problems: (1) objectives, functions, and constraints investigation of the future system, (2) behaviour specification of the future system, (3) evolution management of the future system. requirement engineering is basically a multi-disciplinary operation, as states p. zave, which put together many actors, each of them holding his or her own perspective on the system (easterbrook s.m., finkelstein a., kramer j. & nuseibeh b. (1994)). so, the definition of the requirements of a complex system put into play multiple viewpoints using various skills, responsibilities, and knowledge. the concept of viewpoint has already been instantiated in requirement engineering. j.c.s.p. leite defines a viewpoint in (leite j.c.s.p. (1988) ) as an attitude or a mental position which a person adopts to observe the universe of the discourse. g. kotonya and i. sommerville refer in (kontonya g. & sommerville i. (1992)) to an external entity which interact with the system being analysed. with this respect, their voa project proposes to collect and solve requirements, classify and analyse interactions between viewpoints, and integrate functional and non functional aspects of the requirements. a. finkelstein and h. fuks speak in (finkelstein a. & fuks h. 1989)) of an agent, logical participant to the dialogue who is responsible of the maintenance of a particular perspective. they define a viewpoint as an "object, weakly coupled, locally managed, encapsulating knowledge about the representation modes, the design process and partial specifications of a system and its domain". formally, a viewpoint is defined as a tuple <style, plan, domain, specification, history> where style is the used representation formalism, plan describe the actions, process and development strategies, domain describe the scope of the viewpoint towards the global system, specification describe the viewpoint domain in its style, and history keeps the trace of the actions executed during the development process. ajis vol. 10 no. 1 september 2002 17 two facets of the notion of viewpoint have been studied: the expression of requirements and the means to deal with them, and the organisation of the requirement acquisition process. the description of a viewpoint poses two problems, due to the diversity: this of the formalism of the expression pole, and this of the content pole. the diversity of the expressions is inherent to the notion of viewpoint, since they must reflect the way the different actors express. nevertheless, using the same formalism facilitates the confrontation of the viewpoints during the validation phases of the project. so, until now, the heterogeneity of the formalism has never gone beyond the heterogeneity encountered in classical analysis methods: standard formalisms issued from software engineering – data flows, entity-relationships diagrams, class diagrams, etc. the diversity of the contents originates from the fact that, at the beginning of the designing, each actor holds his or her own concerns. we encounter once again the difficulty to put into relation formal specifications of distinct, or even independent, elements of the system. as the designing process aims at building a collection of formal, rigorous and consistent specifications of the object, some authors are looking for means to verify and/or enforce this consistency. this question is relevant to conflict resolution and refers to the cooperation and negotiation modes between agents. requirement engineering is here defined as the process which identifies the differences between viewpoints, classifies these differences, evaluates several solutions and at last integrate viewpoints in a unique representation. j.c.s.p. leite and p.a. freeman‘s early validation relies on vwpl language (kontonya g. &sommerville i. (1992)), which is the unique description language of the requirements. a comparison tool provides all the syntactic and semantic differences between the definitions. m. ainsworth, s. riddle, and p.wallis aim at checking formally that any given set of viewpoints is valid (ainsworth m., riddle s., wallis p. (1996)). they use z specification of viewpoints to build an amalgamation of individual viewpoints into a single coherent whole. the amalgamation process is defined on a particular relation between specifications that is a conditioned refinement relation. h. bowman present in (bowman h., derrick j., linington p., steen m. (1996)) a framework for checking the consistency of the five viewpoints defined by the open distributed processing standardisation initiative (itu recommendation x.901-904 – iso 10746 1-4 (1995)). they define consistency as a relation between two specifications: two specifications are consistent iff a third specification can be found, which is a development (refinement or conformance)) of both specifications and the third specification can be implemented. a. finkelstein and b. nuseibeh accept temporary inconsistencies. they claim temporary inconsistencies are not only possible, but useful. they implement inconsistency by means of the following particular mathematical relation r between viewpoints: for each p, there exist p’ such that prp’. the source viewpoint p is responsible to verify the relation r with the target viewpoint p’. about the relationships between viewpoints, b. nuseibeh, j. kramer and a. finkelstein stress three questions in (nuseibeh b., kramer j. & finkelstein a. (1994)): (1) how to ensure their definition, (2) who takes them into account and (3) how to apply them. actually, their proposition, based on core method (mullery g.p. (1979)), defines viewpoint templates and a plan which comprises four actions: an action to create viewpoints, an internal consistency control action inside each viewpoint, a consistency control between viewpoints, and action triggers. these actions are executed according to the rules of a development model. so, we share with almost all these authors, the opinion that the diversity of requirements is an unavoidable feature of a technological system, a software in particular. two statements distinguish the quoted works from our proposition. at first, requirement engineering is generally considered as the earliest stage of a design process. we try to act still earlier, i. e. on specifications expressed by natural language, before their formalisation. the viewpoint paradigm dissociates expression and content, and thus allows to consider expressions which aare not provided with formal semantics, as those of the natural language. the second statement is concerned with the relationships between viewpoints. in many works, the objective is to ensure above all the formal consistency between viewpoints. we consider consistency is one possible <e>-correlation, but is far to be the only relevant one. actually, the two correlation models we propose in this paper can take consistency checking as a part., because consistency is one of the parameters of the convergence state of a designing process. this paper put a first milestone of a theoretic and experimental foundation of the viewpoint paradigm. the presented models have not yet been validated by experiments in industrial large scale projects. this will be the next step of our work. the question of the computerised support of these object-based models must be studied in an existing environment, in order to integrate it as an easy desktop tool. these tools are also possible links with existing technical data management systems which are often based on object-oriented representation. figure 8 shows a draft view of the architecture of a viewpoint manager system. ajis vol. 10 no. 1 september 2002 18 object-based representations lexical networks corpus of written documents technical data management system translation translation observation viewpoint manager management control statistical classification tools fig. 8 architecture of a viewpoint manager software references ainsworth m., riddle s., wallis p. (1996) "formal validation of viewpoint specifications", software engineering journal, january 1996, pp. 58-66. bowman h., derrick j., linington p., steen m., (1996) "cross-viewpoint consistency in open distributed processing", software engineering journal, january 1996, pp. 58-66. charrel p.j., galarreta d., hanachi c., & rothenburger b. (1993) "multiple viewpoints for the development of complex software", proceedings of ieee conference on systems, man and cybernetics, le touquet, france, october 17-20, pp. 556-561. charrel p.j., rothenburger b., trousse b. & vogel c. (1996) "etat de l'art sur les points de vue en sémiotique, linguistique et informatique", research report d1.1 of the project etude des points de vue dynamiques, contract cnes/inria 871/94/cnes/1492. charrel p.j., rothenburger b., trousse b. & vogel c. (1997) "modèles de corrélation de points de vue", research report d3 of the project etude des points de vue dynamiques, contract cnes/inria 871/94/cnes/1492. charrel p.j., rothenburger b., trousse b. & vogel c. (1998) "spécification d'une maquette démonstrative et application à l'exploitation d'informations du secteur spatial", research report d4–d5 of the project etude des points de vue dynamiques, contract cnes/inria 871/94/cnes/1492. deledalle g. (1978) "commentaire", ecrits sur le signe, c. s. peirce, editions du seuil, paris.. pp. 203–251. deledalle g. (1990) lire peirce aujourd'hui, giovannangeli d., ed,. le point philosophique, de boeck. brussels. easterbrook s.m., finkelstein a., kramer j. & nuseibeh b. (1994) "co–ordinating distributed viewpoints: the anatomy of consistency check", proceedings of concurrent engineering research and applications, west bloofield. usa. august. eco u. (1980), segno, arnoldo mondadori editore. eco u. (1979) lector in fabula, bompiani ed. finkelstein a. & fuks h. 1989) "multi–party specification", proceedings of the 5th workshop on software specification and design, pittsburgh, ieee c. s. press. galarreta d., charrel p.j., rothenburger b & al. (1998) "study of dynamic viewpoints in satellite design", proceedings of the 9 th symposium ifac information control in manufacturing systems incom’98, nancy, june 24-26. granger g.g. (1994), formes, opérations, objets, vrin edirion. ajis vol. 10 no. 1 september 2002 19 greimas a.j. & courtes j. (1979) sémiotique. dictionnaire raisonné de la théorie du langage (tome 1), hachette. greimas a.j. (1966) sémantique structurale, larousse, paris. habermas j. (1991) texte und contexte, suhrkamp verlag. itu recommendation x.901-904 – iso 10746 1-4, "open distributed processing – reference model – parts 14", july 1995. kontonya g. &sommerville i. (1992) "viewpoints for requirements definition", ieee software engineering journal, vol 7, pp. 375-387. leite j.c.s.p. & freeman p.a. (1991) "requirements validation through viewpoint resolution", ieee transactions on software engineering, vol 17(12). leite j.c.s.p. (1988) viewpoint resolution in requirements elicitation, phd thesis, university of california, irvine. lemoigne j.l. (1990) la modelisation des systèmes complexes, dunod. levinas e. (1994) nine talmudic readings, indiana university press. morin e. (1991) la complexité humaine, champs l'essentiel. flammarion. mullery g.p. (1979) "core – a method for controlled requirement specifications", proceedings of the 4 th international conference on software engineering, münchen, germany; ieee cs press, pp. 126-135. nuseibeh b., kramer j. & finkelstein a. (1994) "expressing the relationships between multiple views in requirements specification", ieee transactions on software engineering, vol 20(10), pp 760-773. peirce c.s. (1932) collected papers, harvard u.p. rohmer e., chabrol c. (1988) hitchcock the first forty-four films, ungar film library. rumbaugh j., jacobson i. & booch g. (1998) the unified modelling language reference manual, addisonwesley. zave p. (1997) "classification of research efforts in requirement engineering". computing surveys, vol 29(4), pp.315-321. ajis special edition knowledge management december 2001 exploring knowledge management perceptions among information systems managers empirical sense-making through focus group research david yuh foong law department of decision sciences business school, national university of singapore c/o: cmit, fba 2, basement, business link, singapore 117591 fbalawvf@nus.edu.sg; davidlaw@bigfoot.com joo eng lee-partridge department of decision sciences business school, national university of singapore fba 1, 15 law link, singapore 117591 fbaleeie@.nus.edu.sg; jooeng@bigfoot.com keywords: knowledge management, empirical sense-making, frames of reference, focus groups research, information systems management. abstract despite the increasing knowledge management (km) awareness and interest among academia and industry, a very diverse range of views and perceptions still exists. there is a need to appreciate the issues and concerns surrounding km research and implementation among communities of researchers and practitioners. our research aims to provide a deeper empirical insight of practitioners in terms of the general level of awareness, the state of practice, and industrial perceptions on km issues in the context of a growing knowledge-based economy such as singapore. we chose to explore this topic from an information systems (is) management perspective, by exploring the levels of km understanding, the issues of concerns and requirements by chief information officers (cios) and senior is managers, and how they make sense of km. sense-making approach through focus group research is the primary methodology used. using social cognitive research techniques such as frames of reference, the participants' perceptions are summarised and presented broadly along the following themes: conceptual awareness of knowledge/km; facilitators and inhibitors of km; usefulness and value of km; km mechanisms and implementation approaches; factors influencing knowledge sharing, acquisition and capture; effectiveness of km technologies; and the prerequisites of a km practitioner. our findings are discussed in the light of their implications to is management practice in organisations. the findings also raises new research questions in is and km. introduction overview, motivation and objectives in the context of singapore, a small nation whose main resource is human skills and knowledge, it needs to transform into a knowledge-based economy in order to survive and compete economically. the ability and need to effectively exploit the intellectual resources within and around a business domain have thus become a major challenge for knowledge-intensive organisations. knowledge management (km) technologies and practices will play a major role in supporting knowledge work and related processes. km is an emerging area of focus where researchers and practitioners of varied background and disciplines approach it from diverse angles and perspectives. from the km literature, there is no lack of explicit guidelines and approaches developed for km implementation. liebowitz ed. (1999) compiled a comprehensive overview of these generic concepts and practical guidelines. km, given its multidisciplinary and cross-functional nature, as well as its ambiguous definitions and boundaries, its theoretical foundation has not been stabilised, and it has manifested its own cloud of confusion. as km is an emerging and evolving field, it remains a new an elusive concept to many organisations. practitioners have encountered difficulties and uncertainties in the adoption and implementation of km despite their attempts to follow some of the prescribed approaches and guidelines religiously. some of these difficulties are attributed to specific social, organisational and contextual factors, while others could be due to the misconceptualisation of the actual km problem which results in the deployment of an inappropriate km solution. therefore, despite the increasing volume of publications generated on the subject, the practical know-how commonly recommended by consulting firms and the increasing km awareness and interest in singapore, a certain degree of ambiguity still exists. it is still generally unclear how an organisation initiates and implements km projects and exactly 42 ajis special edition knowledge management december 2001 how km can be applied or contribute to business growth and developments. the current lack of both a well-defined view of the subject and empirical insights have motivated this study of km-related issues in singapore. the primary objective of our research is to carry out a preliminary qualitative assessment of the general level of awareness, the state of practice, and industrial perceptions on km issues in singapore. at this juncture, our research does not aim to discover new km theories or models and attempting to insert them into theory in literature. instead our contribution focuses on the construction of social cognitive profiles of practitioners as part of our attempt to understand related industry sentiments and practical issues, even as the practitioners are struggling to make sense of km concepts themselves. we have chosen to explore this topic from an information systems (is) perspective. as is is a key factor behind organisational management and business innovation in singapore, is management plays a crucial role in supporting and augmenting business managerial decision-making through the establishment of efficient is infrastructures. in the context of km adoption and implementation in organisations, is managers today face a new set of challenges. these include the need to understand the organisations' business and users' km needs; aligning the organisations' is plans to meet these needs; keeping abreast of new and emerging technologies and identifying suitable technologies for km; and the subsequent task to champion these technologies for their eventual adoption and acceptance in organisations. we surmise that is managers will play a very important role in supporting new km initiatives and overseeing is-based km implementations. such considerations have motivated our decision to commence our research on km issues and perceptions with is professionals. the study was carried out with a number of chief information officers (cios) and senior is executives from various institutions and organisations based in singapore. focus group research was used as an effective method for collecting rich and broad-based qualitative data, and the approach is flexible and appropriate for the study of emerging trends and issues. such a group setting was also ideal for peer interaction, sharing and exchange of opinions and perceptions. in particular from a sense-making perspective, focus groups are extremely useful mechanisms for exploring fuzzy empirical km issues and to assess the levels of km understanding among cios and senior is managers. a series of focus group sessions were conducted and the results were extracted and broadly categorised. this paper presents the collective views and perceptions on km from an is practitioner perspective. using social cognitive structures such as frames of references, the results are broadly framed in categories, such as: awareness and concept of knowledge/km; facilitators and inhibitors of km; usefulness and value of km; km mechanisms and implementation approaches; factors influencing knowledge sharing, acquisition and capture; effectiveness of km technologies; and the requirements and qualifications of a km practitioner. it includes an analysis and discussion of these empirical results, and their implications for km, with a view to facilitate the management, development and deployment of is-based information and decision support systems as strategic km enablers in organisations. knowledge management as the global economy becomes increasingly more knowledge-based in nature, organisations have realised the value of knowledge and the need to manage it effectively. within organisations, knowledge resources are fast becoming critical intellectual assets with strategic roles in organisational survival and competitiveness. the field of km is a fast emerging area of interdisciplinary research and practice. km is the formalisation of and access to experience, knowledge, and expertise that create new capabilities, enable superior performance, encourage innovation, and enhance customer value (beckman 1997). the key objectives of km can be summarised as: firstly to make the enterprise act as intelligently as possible to secure its viability and overall success; and secondly, to otherwise realise the best value of its knowledge assets (wiig 1997). in essence, km involves the creation of the environment and opportunities to enhance the potential for co-ordination and synergism between networks and pools of knowledge. beckman (1997) proposes an eight-stage process for km: identify, capture, select, store, share, apply, create and sell. alongside these km processes which guide and drive the construction of corporate memory repositories, a whole spectrum of advanced information technologies and techniques may be used for supporting the activities in a km cycle (wiig et al. 1997). km can be examined at different levels and explored along a wide range of perspectives. this reveals the diversity and complexity of km in research and practice, with converging contributions from various disciplines and perspectives such as organisation and management (hbr 1998), economy and policy-making (oecd 1996), and advanced information technologies (shariq 1998). the ambiguity and the lack of a commonly accepted definition for km are perhaps part of the reasons for the confusion surrounding this field, which accounts for the diverse research disciplines it attracts and its slow diffusion to and acceptance by industry. 43 ajis special edition knowledge management december 2001 research context and approach km adoption within organisations in singapore could be generally described as "a little quiet" despite growing awareness. it seems that a cloud of uncertainty relating to km persists among organisations. organisations in general have not embraced much of km practices and technologies into their own business and structures, which is in contrast to the enormous effort and attention given to other rapidly developing fields such as e-commerce. with km technologies and issues emerging rapidly, km concepts and the level of abstraction increases. as a result, most organisations struggle to comprehend fully the concepts surrounding knowledge, knowledge-intensive work and km, let alone km strategic planning and implementation. the meanings and perceptions of knowledge and km also vary between organisations and across industries. in this context, we aim to explore a diverse range of km-related issues, from the organisational, technological, managerial and conceptual perspectives. in the selection of study subjects or participants, we decided to focus on senior executives and practitioners with is-related background. the presumption was that this category of participants may tend to perceive a wider picture of km, especially technology implementation issues, as compared to non-is participants. through a series of focus group discussions, we hope to solicit viewpoints, opinions and perceptions from these is industry leaders and practitioners which could contribute to the empirical-grounding of fresh insights and structures on km in singapore. this approach will enable the framing of issues for analysis, highlighting industry concerns, and establishing missing links and new relationships among existing practices and research. we believe this research approach will narrow the gap between academia and practice in a contemporary environment of dynamic business needs and rapid technological development. research methodology as km is still an emerging field in its infancy in singapore, an exploratory qualitative research strategy is appropriate to solicit opinions and perceptions from various organisations in singapore. the challenges of environmental uncertainty and problem definition are occasions for sense-making (weick 1995), which may be situations that are novel, discrepant, or are based on deliberate initiative. with the existence of many ill-defined situations and concepts associated with km practice, we decided to adopt the sense-making approach to explore the situation in order to gain clearer insights of industry perceptions. according to weick (1995), sense-making, in its simplest form, literally means "the making of sense". it is a process which can be understood from various perspectives, such as structuring the unknown or explaining surprises. focus group research is a useful and effective approach in this context for the study of emerging trends and issues in km by providing a conducive platform to initiate the process of sensemaking. focus group is a qualitative research technique in which a group of eight to ten participants of similar demographics, attitudes, or behavioural patterns are led through a (usually) two-hour discussion of a particular topic by a moderator (greenbaum 1998). this technique has been popularly used in marketing research for gathering consumers reactions towards certain products and services, but has yet to be adopted or accepted in mainstream is research. in our context, focus groups are very useful mechanisms for making sense of fuzzy empirical km issues and to assess the levels of km understanding among cios and senior is managers. for our study of km issues and perceptions we have adapted the focus group technique as part of an overall grounded theory approach (strauss and corbin 1998) to obtain rich and qualitative insights from participants. large and rich amounts of data in the participants' own words can be readily obtained, thereby enabling them to obtain deeper levels of meaning, make important connections and identify subtle nuances in expression of meaning. using frames of reference, data from focus groups were first summarised (tables 1-4) and subsequently formalised into explicit structures (figures 1-7) which effectively map the cognitive frameworks of the focus groups participants. this activity represented the first key step in our grounded theory approach, from the perspective of sense-making, where new issues and concepts were induced from raw empirical data. these structures represent a set of cognitive representations where meanings and perceptions are shared, from which research interpretation may be derived or built upon. these frameworks will serve as seed structures as they evolve progressively, grounding on fresh data and insights from focus groups until a point of saturation where the frameworks attain a certain degree of stability. a series of three separate focus group sessions were conducted with an average of eight participants per group, each in their mid-forties with close to twenty years of industry experience. they were mainly cios, is directors and senior is executives from eighteen large singapore-based organisations, across various industries including government bodies, academic institutions, consulting firms and multinational corporations. each session was audio and video taped for subsequent coding and analysis. it may be useful to note that at this juncture of our research, comparison of our results with general concepts, views, opinions published in the literature is kept to a minimum because one of our objectives here is to solicit true empirical perceptions and issues largely from the participants' own organisational contexts and experiences, as we attempt to make sense of these findings from their unique perspectives. 44 ajis special edition knowledge management december 2001 perceptions among focus group participants the proceedings of all focus group sessions conducted were carefully video and audio-taped for analysis and transcription. the preliminary results documented were extracted and summarised, and broadly tabulated under general categories namely: (1) perceptions of knowledge concepts and km; (2) perceptions of the facilitators and inhibitors of km; (3) perceptions of km mechanisms and tools; and (4) perceptions of the roles and requirements in km. these findings reveal some of the issues, views and opinions discussed, reflecting the perceptions and concerns of the cios and senior is executives, and a glimpse of the current state of km adoption in their respective organisations. the issues discussed under each category could be related to any one or a combination of cultural, technological, managerial and other aspects of km. the data presented here are further analysed in the following section, where they serve as an empirical basis for deeper theoretical reflection, sense-making and new theory induction. table 1. perceptions of knowledge concepts and km. km issues perceptions, opinions and views from focus group participants knowledge vs information similarities: knowledge is a summary of information; a subset of information; related or specific or tailored to a domain, or a task at hand, or to pursue a business objective. differences: information is very general, covering wider scope; information can be gathered, analysed, but may not be internalised, while knowledge can be internalised (tacit), in-built within individuals. in this sense, knowledge can be regarded as internalised and actionable information. examples of knowledge and its applications and the benefits of km knowledge only becomes useful when applied. knowledge facilitates decision-making by allowing the person to access best available knowledge in order to make optimal decisions. organisational knowledge includes: processes, procedures, best practices (to increase productivity), and competitive advantage knowledge (to achieve better customer service, more profits, larger market share). km is the know-how to facilitate the sharing and utilisation of knowledge when it is needed. experts' knowledge will be increasingly focussed and specialised. meta-knowledge ("knowledge about knowledge") helps to track an organisation's intellectual resources and expertise for quick access and retrieval. doubts and scepticism km effort may not be useful or worthy; some knowledge may not be worth capturing due to: the nature of short project life cycles; and the quick obsolescence of knowledge and expertise (out-dated technologies, markets, products, etc.). 45 ajis special edition knowledge management december 2001 table 2. perceptions of the facilitators and inhibitors of km. km issues perceptions, opinions and views from focus group participants organisational culture cultivating the appropriate organisational cultural and changing mindsets are among some of the most important factors for the successful adoption of km concepts, practices and technologies. the ceo is the key figure in making or breaking such a culture. "push & pull" factor for km the adoption of km is largely influenced by the interplay between the "push" and "pull" factors, whereby the ceo should "push" it (km champion) while the technology providers will "pull" it (provide km solutions). organisational structure and existing resources and infrastructure an organisation with a relatively flat structure comprising more workgroups and teams are generally considered more favourable for km, compared to a more traditional, hierarchical, multi-layered organisational structure. a km practitioner should be given the authority to move around freely within an enterprise, gathering knowledge and accessing knowledge bases from various departments and sections. existing organisational structure and workflow should be modified to reflect such flexibility. from a technology and resources perspective, the existing infrastructure and management practices of large organisations in singapore are quite ready for the adoption and implementation of new km initiatives. generally this does not include intelligent systems. knowledge sharing at the organisational and individual levels asians tend to be more conserved and reticent, hence cultivating the appropriate knowledgesharing mindsets and culture could be relatively more difficult in asian-based organisations. organisational factors that affects knowledge sharing include: identifying sharing motivators and developing sharing passions; promoting mechanisms and practices to promote sharing (eg. conferences, seminars, discussion groups), and sharing using innovative techniques (e.g. story-telling); and adopting the appropriate technologies to support sharing processes and activities (e.g. groupwares, common knowledge bases, etc). at the individual level, the factors affecting personal sharing of tacit knowledge could be: his ability to express his knowledge clearly and adequately; his concerns of losing his expert status and value in the organisation if he shares; the correctness and completeness of expert knowledge; and knowledge documentation in an understandable form. confidentiality and security issues are also of concern, in developing policies and structures for organisational knowledge sharing. suggestions include making selective availability of knowledge and information to staff with different needs and security access levels, and possibly on a new-to-know basis. knowledge capturing and documentation in capturing and documenting knowledge, the symptoms and the prescribed solutions to a problem are usually recorded, but often the real causes of the problem and the process of formulating the solution are often not captured. part of km should focus on capturing such previously neglected knowledge. also, with the problem of information overload, a person could sometimes be more biased towards the type of information and knowledge which they wish to extract, with the possibility of missing out on other important aspects. other preoccupations and external factors: 1. economic crisis in asia; 2. y2k related problems 3. iso 9000 and other retrie economic crisis in asia over the last couple of years could be a major factor influencing the decision and rate of km adoption and diffusion. this may be viewed from two angles: on one hand, organisations could be more competitive through km which could enable new markets entry, offer new products and services. however on the other hand, there is a pressure to cut cost/budget through downsizing business operations, and retrenching highsalaried employees (unfortunately usually experienced and knowledgeable). it seems that the latter view has slowed down the adoption of km practices in organisations. at the turn of the century, most it departments have focused their attention and resources on solving the y2k millennium problem and developing measures to prevent its occurrence. most organisations in singapore are pursuing iso 9000 or related quality management certification programs, which involves a substantial amount of effort and resources being channelled to such activities. these programs are generally perceived to have more tangible implementation approaches and measurable outcomes. the above factors have resulted in organisations freezing or putting on hold km projects, or 46 ajts special edition knowledge management december 2001 lated quality management certification programs shifting these projects to a lower priority. this is especially so, when managers could not associate the outcome/effects of km with the direct benefits of applying km to their work and business domains. table 3. perceptions of km mechanisms and tools. km issues perceptions, opinions and views from focus group participants methods and approaches for knowledge capture in general, tacit knowledge in organisations (eg. a cultural practice, a particular way of doing a project, etc.) are not easily captured as they are not documented or formalised. some forms of explicit knowledge could be easily captured through practices such as iso9000 procedures/documentation. other methods of knowledge gathering include interviews and collecting frequently-asked-questions (faqs). knowledge acquisition should always have clear objectives and reasons. organisation could acquire knowledge within itself (eg. operations and processes) and externally (eg. competitors and customers knowledge). gathered knowledge should be carefully sorted and categorised for effective retrieval and use. suggested methods to motivate sharing include: staff suggestion schemes, reward scheme for sharing and good ideas, conferring recognition, incorporation into staff appraisal/promotion, encourage formation of project teams comprising members from diverse backgrounds, disciplines, expertise and experiences, etc. technological capabilities, related issues and general perceptions technological issues are of lower concerns in km. to support km adequately, a hybrid of technologies may be required to support the identification, acquisition, development, dissemination, use, and preservation of the enterprise's knowledge. some of these technologies are generally available in the market while new ones are actively in the research pipeline. popular tools adopted in organisations include groupwares (eg. lotus notes), internet/intranet/web servers, search engines, agents, information databases and data warehouses. intelligent and knowledge-based technologies (eg. artificial intelligence (ai), expert systems) are not commonly adopted in organisations, nor regarded as strategic enablers for km. data mining was only beginning to be adopted by organisations and were only loosely perceived as a km tool. modelling technologies were also not associated with km in any way. future trends and capabilities of technology should focus on preventing information overload; possess effective retrieval mechanisms, and intelligent means of personalising knowledge gathering, selection and delivery. with this wide spectrum of technologies to select from, the main challenge here would be to attempt to match the appropriate tool or sets of tools to a particular km need or activity. table 4. perception of the roles and requirements in km. km issues perceptions, opinions and views from focus group participants the need for a separate group of km practitioners in order to carry out quality km tasks, there may be a need for a separate dedicated team of km practitioners. such a team may comprise a chief knowledge officer (cko), knowledge managers and practitioners with cross-functional business knowledge, diverse training, background and specialised skills. desired attributes; and the perceived roles of a cko the cko could report directly to the ceo, and work very closely with the cio and other staff. alternatively the ceo could undertake this role in overseeing km initiatives. due to the emphasis and wide coverage of business and domain knowledge in km, generally it may not be appropriate for the cio or is practitioners to lead a km department or drive km projects. however is practitioners should continue to work very closely with the management and staff of various business domains, and to play an important role in supporting km efforts with the latest tools and technologies. 47 ajis special edition knowledge management december 2001 requirements of a km practitioner; and related issues the prospective candidate should possess the relevant domain knowledge; be able to understand the organisational business well; have good working knowledge and overview of information technologies (it); and good team-working and communications skills. in addition, the candidate should have some basic understanding of the psychological and educational aspects related to the thinking and learning processes in people. prior knowledge of the business domain could facilitate the comprehension, content organisation and indexing of the knowledge captured. however one concern here is that a km practitioner may be too close to the business domain itself, with thinking patterns very similar to that of the domain expert/practitioner. as a result this could possibly hinder the km practitioner from perceiving issues from a fresh perspective. also, by being too close to the domain may lead to assumptions of pieces of knowledge (which could be deemed common-sense or trivial by domain practitioners) which may result in the omission of such knowledge and information during the knowledge acquisition process. another view holds that km practitioners may not necessarily be required to possess domain knowledge, but instead should be equipped with specialised km expertise such as special knowledge acquisition, handling or engineering skills. findings and discussions the data summarised and presented in the above section serves as a basis for deeper reflection and sense-making from an interpretative and critical perspective, in the empirical context of the participants' backgrounds and work domains. in the following section, theoretical frameworks (figures 1-7) and their corresponding propositions are further induced which are grounded based on the empirical data collected from the focus groups. analysis of focus group results issues surfaced from the focus group discussions generally fall into two main streams. natural knowledge management (nkm) issues generally covers human-centred, non-technical, km issues and solutions at the individual or organisational level. artificial knowledge management (akm) issues are usually associated with technical or technology-based km issues and solutions. conceptualising knowledge and knowledge management from the data, discussion topics under this category mainly revolved around nkm issues. the literature has diverse views on the concept of knowledge and km as these concepts are explored from different angles. for instance, according to awad (1996), knowledge can be classified according to its nature and form, its source, the way it is used, and its purpose and relevance. foray and lundvall (1996) propose four different types of knowledge based on their contexts and usage: know-w/ia/; know-w/iy; know-/iow; and know-w/io. polanyi (1966) distinguishes between two aspects of knowledge: tacit and explicit knowledge. from a knowledge flow and conversion perspective, nonaka and takeuchi (1995) postulate four inter-connected, spiralling modes of knowledge conversion based on the assumption that knowledge is created through the interaction between tacit and explicit knowledge. these are socialisation (tacit knowledge sharing between individuals), externalisation (articulating tacit knowledge into explicit concepts), combination (integrating different explicit sources into new explicit knowledge base) and internalisation (embodying explicit knowledge into tacit forms through organisational practices). generally, focus group participants were unable to make a clear distinction between information and knowledge, in terms of form and structure. however, attempts have been made to differentiate these two concepts based on their levels of content summary, the manner in which it is internalised within a person, and the contexts of their applications. comprehension of the concept of tacit and explicit forms of knowledge, as proposed by polanyi (1966), was also demonstrated, although the process of the flow and inter-conversion of knowledge were not well understood. some attempts have been made to explore the different types of knowledge such as know-what, know-how, knowwhy and know-who, as mentioned by foray and lundvall (1996). the participants also appeared to have a better grasp of the concept and procedures of km than the more abstract concept of what is or is not "knowledge" and "information". this is not surprising since km may be regarded as a process with various types of activities associated 48 ajis special edition knowledge management december 2001 with it such as knowledge capture, sharing, storage, distribution, etc., which could be easily conceptualised and broken down into more tangible sub-processes and components. existing organisational structures and practices nkm issues were mainly discussed along this thread of topics. currently, according to the focus group participants, most organisations have yet to formulate or adopt a formal km agenda, despite the growing awareness and interest in km. having a team of km practitioners and the right organisational culture are considered the two most important factors that drive km. existing organisational practices such as staff suggestion schemes, discussion sessions, human resource functions (rewards, appraisals, defining new km roles and responsibilities) to facilitate and motivate knowledge sharing; managing business operations in line with quality management practice requirements; increased interest and support from the ceo and top management; all of which could be ideal foundations for creating a conducive km culture and for the development of a good organisational nkm strategy. some of these issues are commonly elaborated by authors in km practice-oriented literature (eg. liebowitz ed. 1999; cortada and woods eds 1999). technological issues akm issues were discussed along topics associated with this discussion thread. in general, technological issues for km appeared to be of lower concerns among the focus group participants, although the general consensus was that km may require a mix of technologies to support km activities such as the identification, acquisition, development, dissemination, use, and preservation of the enterprise's knowledge. technologies commonly implemented in organisations were perceived to be suitable for km. popular tools adopted in organisations include groupwares (eg. lotus notes), internet/ intranet/ web servers, search engines, agents, information databases and data warehouses. the focus group participants, all of which were is professionals, appeared confident of the current state of technological capabilities available in the market for the support of km tasks. however, noticeably, there were very little or no discussion about the significance of intelligent technologies and methodologies such as the use of artificial intelligence (ai) and expert systems for km, despite extensive support and claims from literature in this area advocating their links to km (eg. smith and farquhar 2000; liebowitz 2001). this category of systems have so far received marginal attention among the focus group participants as their technological capabilities were not discussed very much. apparently these technologies have not been widely adopted in organisations, nor were they regarded as strategic km enablers, even though they are expected to play an increasingly important role in km. other intelligent technologies, such as data mining, were only beginning to be explored by organisations in singapore and were generally not very much associated with the concept of km. enterprise modelling technologies were also not associated with km in any way at this stage of discussion. desired attributes of a km practitioner with regards to the qualifications and requirements of an ideal km manager or practitioner, there seemed to be a high calling for this new breed of professionals. the desired types of background training, skills and knowledge required or expected have been discussed at length. however the focus group participants did not express clearly what exactly should be the roles and responsibilities of these km professionals; how they can co-operate and work together with existing groups of professionals (eg. business and is professionals), and how and where they can fit into the organisation as a whole. discussion topics under this category mainly revolved around nkm issues, with occasional references to akm elements as the focus group participants attempted to propose necessary skills and background training in accordance to the needs of desired km technologies. presently there are very little or no discussion pertaining to issues associated with relevant skills and prerequisites of km practitioners in the is and km literature, although there are some literature which explore the roles and functions of km professionals. in summary, a growing number of these organisations appear to have existing infrastructures and management practices which we believe, would serve as a conducive foundation for implementing km. our study further reveals that the focus group participants are also grappling with a couple of open-ended issues which they have not quite come to terms with. these include: • how can the structure and context of knowledge be best defined? • how can we create a conducive, pro-km, organisational culture and environment? 49 ajis special edition knowledge management december 2001 • how can akm methods be used effectively for tapping tacit knowledge (facilitating tacit to explicit knowledge conversion) which is also a difficult problem commonly echoed in literature (eg. nonaka and takeuchi 1995, p.62; holtshouse 1998)? • what are the strategic roles of intelligent technologies in km in the respective domains of application? framing km issues and implications to is based on the results and km issues extracted from our focus group findings, tabulated in the previous section, we proceed to induce conceptual structures which frame up various aspects of km issues perceived by organisations. these frameworks, as illustrated in figures 1-7, put in perspective, the perceived influential dimensions, factors and contexts surrounding specific km issues raised by the focus group participants. the frameworks highlight conceptual, organisational and technological aspects of km along themes such as: characteristics of knowledge (as differentiated from information), usefulness of knowledge and km value; diffusion and adoption of km; knowledge sharing; knowledge acquisition and capture; effectiveness of km technologies; and requirements and qualifications of a km practitioner. direct inputs from the focus group discussions generated among the industry participants have facilitated the grounding of the constructs of each framework. these cognitive structures or frames, together with proposition statements, which collectively form a perception profile of cios and is managers on how or what they perceive about km in the is practice community within the singapore context. we wish to assert that these propositions induced from the data gathered at this stage reflect the mindsets, perceptions and attitudes of the is professionals present at the focus groups meetings, comprising a total sample size of slightly over 20. our research methodology adopted is not positivist in nature, but more of soliciting a diversity of empirical data for constructing frameworks, who will in turn be used to guide theory grounding and sense-making. proposition 1: is managers tend to perceive the concept of "knowledge" to be different from "information" based on attributes such as the degree of knowledge summarisation, relevance and internalisation. figure 1 suggests the perception that "knowledge" can be distinguished from "information" based on the degree of information summarisation, its relevance to a subject matter or task, and to the degree to which it can be internalised within a person. with respect to the outputs of is-based km systems, the content should be increasingly more distinct from "information" or "data" and becoming more knowledge-based, in the sense that it should be more readily comprehended and applied to various work contexts by users. in other words, the amount of physical and cognitive effort by users to pre-process the system content before it is ready to be absorbed and applied, should be reduced on the part of the human users. proposition 2: is managers tend to perceive the notion that km value depends on the perceived usefulness of knowledge and its associated attributes and factors. figure 2 suggests the perception that the ultimate value of a km effort in an organisation will depend on the perceived usefulness of the knowledge itself which can be measured according to its perceived applicability, its perceived ability to support a task at hand (eg. in decision-making or problem resolution), and its perceived obsolescence in the context of dynamic business, organisational and technological requirements, and certain nature of business and operations and the associated types of knowledge. in the contexts of developing is-based solutions to support km or implementing related km activities for the purpose of constructing effective and useful km systems, the effectiveness of the system in facilitating the application of knowledge to a domain problem or task at hand is an important consideration. summarisation of information relevance to subject matter intra-subjective internalisation n̂ / knowledge applicability of v r\ r*\\ t\ (*a n f* ability to support task at hand obsolescence of knowledge \ 4 t */ usefulness of knowledee ^ ^ value of km •w-i t*f . figure 1. characteristics of knowledge. figure 2. usefulness of knowledge and km value. 50 ajis special edition knowledge management december 2001 proposition 3: is managers tend to perceive the notion that the successful adoption and diffusion of km depends on both internal organisational factors (availability or appropriateness of km champion, solution provider, culture, mindset, structure, policy, resources and infrastructure) and external factors (pressures arising from economical, environmental, industry trends, etc). figure 3 suggests the perception that the diffusion and adoption of km is dependent on various factors including the state of infrastructure and resources; the intensity of "push" by the km champion and "pull" by solutions provider; organisational culture, structure, and mindset of workers; as well as other pre-occupations and external factors (eg. the impact and aftermath of the asian economic crisis, the y2k problems, iso 9000 and other quality management certification programs). these factors could potentially facilitate or inhibit km adoption by influencing managers' decisions whether to initiate or terminate potential km projects, depending on how organisations react towards these factors and how they prioritise km projects in the light these considerations. the perceived success of adopting tools and technologies associated with km and requirements of related is management efforts will indirectly depend on the above factors. proposition 4: is managers tend to perceive the notion that the success and effectiveness of knowledge sharing in an organisation depends on factors such as the willingness of people to share knowledge, the quality of knowledge content, the effectiveness of sharing mechanisms, and the ability to resolve confidentiality and security issues. figure 4 highlights perceived factors which may affect knowledge sharing at both individual and organisational levels. these factors include the quality of shared or documented knowledge (which depends on the degree of biases, clarity of expression, correctness and completeness); the willingness to share (which can be influenced by the sharing culture, the passion to share, and the presence or effectiveness of motivators); effectiveness of sharing mechanisms (including organisational and technological aspects), and ability to resolve information/knowledge confidentiality and security issues. in the context of a is-based km system, the content quality of the system should aim to achieve those of the knowledge quality proposed in figure 4. these tools should also incorporate suitable technologies to enable different modes of knowledge sharing among knowledge workers. km systems should also be designed in a manner which encourages users to share knowledge through the system, with the goal of meeting the users' learning and km needs as much as possible. km champion (push factor) resources and infrastructure solution provider (pull factor) organisational culture and mindset organisational structure/policy supporting practices and technologies sharing mechanisms confidentiality and security figure 3. diffusion and adoption of km. figure 4. knowledge sharing. proposition 5: is managers tend to perceive the notion that the effectiveness and efficiency of knowledge acquisition and capturing processes depends on factors such as motivation, rationale and applicability, the availability and suitability of existing procedures, knowledge gathering techniques required, and the scope of targeted knowledge. 51 ajis special edition knowledge management december 2001 figure 5 suggests the perception that the effectiveness and efficiency of the knowledge acquisition and capturing process is dependent on factors such as the objectives and motivations for knowledge capturing; the scope of the targeted knowledge; the availability and sophistication of the technique(s) required; and associated rationale and application context. from the systems perspective, conventional systems depend on data and information as raw inputs. as these systems become increasingly more knowledge-focussed, intelligent technologies will come into play. specialised knowledge engineering techniques, which are traditionally used in the development of expert and ai systems may become more significant in constructing km systems. existing procedures and practices scope of knowledge knowledge gathering techniques •̂ ^> < * knowledge acquisition and capture i i rationale and annlirahilitv ability to match/map tools to km activity ability to prevent information overload ability to personalise knowledge and related processes effectiveness of km technologies effectiveness of retrieval mechanisms figure 5. knowledge acquisition and capture. figure 6. effectiveness of km technologies. proposition 6: is managers tend to perceive the notion that the effectiveness of km technologies depends on factors such as retrieval mechanisms, prevention of information overload, ability to personalise knowledge and related processes, and the suitable matching of km tools with km activities and requirements. figure 6 suggests the perception that the perceived effectiveness of km technologies is dependent on factors such as the effectiveness of knowledge retrieval mechanisms; the ability to prevent information overload; ability to personalise knowledge and related processes; and also more importantly the organisation's ability to match/map suitable km tools to support appropriate km activities, needs and requirements. developers of km systems would be wise to consider and incorporate some of these requirements into their systems design and development. there are also strong indications that these suggested systems capabilities and features are not commonly found in traditional is analysis tools, and that these suggestions may be effectively realised and enabled, with the increasing use of more human-like systems approach such as specialised intelligent or al-based technologies. proposition 7: is managers tend to perceive the notion that an effective km practitioner should ideally possess the appropriate background knowledge, basic skill sets, familiarity with the domain, and some basic understanding of psychology and learning in people. figure 7 suggests the perception that in selecting an ideal km practitioner, some major considerations should be taken into account including his background knowledge (in areas of the business domains, it, and cross-functional training); his basic skill sets (such as general communications, team-working, and specialised km skills); his degree of familiarity and biases with respect to the business domain selected for km; and whether he has some basic understanding of the psychological and educational aspects relating to the cognitive processes in people such as learning and thinking. practitioners who are involved in the adoption, design and implementation of computer systems for the purpose of km should be aware of the diversity of skill sets and backgrounds desired as these know-how and expertise will largely influence the design and deployment of these systems in suitable work domains and contexts, and how effectively these is practitioners work with other non-technical km team members and users. for instance, a domain km problem may require a system capability to analyse loads of raw data and information. this may require the application of data mining technologies for km. in this case, systems developers involved should possess relevant background and training in data mining or related disciplines. at the same time, they should also have acquired sufficient domain knowledge in order to appreciate the nature and needs of the problem as they attempt to formulate and design an optimal km solution. 52 ajis special edition knowledge management december 2001 diverse or cross-functional \ general familiarity/biases related to a business/work domain general/basic it knowledge background knowledge specific or specialised knowledge related to a work task or dorequirements and prerequisites of a km practitionpr communication sulls teamworking skills basic understanding of psychological and educational aspects in people \ specialised km skills figure 7. requirements and prerequisites of a km practitioner. km perception gaps and challenges in our analysis of the focus groups results, some of the issues and factors highlighted in the previous sections have important implications to the research and practice of km and is. among these issues, two predominant km challenges have emerged. first of all, the need to establish the right organisational culture, mindset, structures (practices, policies and mechanisms) is one of the most important issues facing practitioners. managers and developers of is or km solutions need to bear this in mind as they strive to construct, use and manage such systems effectively, with the hope of generating positive side effects among users in supporting knowledge-intensive work. secondly, the desire to tap, retain and share tacit forms of knowledge for organisational benefits remains an elusive and difficult task, with no clear solutions. interestingly, from our observations during the focus group sessions, there were little discussions about the adoption and application of intelligent knowledge-based systems such as ai techniques, and expert systems. most technological related issues centred around more conventional technologies and platforms such as internet/intranet/web-based technologies, e-mails, group support systems, etc. on the other hand, researchers working on intelligent systems and knowledge engineering techniques which explicitly deal with knowledge, have hailed these technologies as future strategies for is and km (examples: aaai 1997; beckman 1997; borghoff and pareschi, 1998; dhar and stein 1997; neches et al 1991; wielinga et al 1997; wiig et al 1997; smith and farquhar 2000; tsui, et al. 2000; liebowitz 2001). this reveals a interesting gap between practitioners' and researchers' perceptions of the potential strategic roles of intelligent knowledge-based systems in facilitating or enabling km implementation. the following research questions are induced, as we try to probe further for reasons to help explain the differences in perceptions between the is practitioners (in singapore) and researchers, with the hope to expound on the burning issue of why is practitioners have not adopted intelligent systems as readily as they could have during the acquisition, development and management of is for km. • were is managers ignorant of the state-of-art research and technologies in intelligent systems, and their potential capabilities in augmenting km activities and decision support (eg. extracting, transforming and sharing of tacit knowledge)? similarly, were they contented with conventional information systems features to support km? • was there an acute shortage of specialised skills and expertise associated with the construction of intelligent systems among is practitioners? • was there resistance from is managers towards the adoption and use of intelligent systems, either due to past experiences of failures and undelivered promises, or due to some perceived negative impacts on the knowledge content of certain tasks and job quality (hendriks 1999)? • was it that the is managers did not regard the use of intelligent systems as a strategic enabler for km or they considered that there would be no significant benefits in using these systems? • a combination of the above reasons? 53 ajis special edition knowledge management december 2001 conclusion and future directions this study has illustrated the effectiveness of focus group research as a qualitative method for an exploratory, inductive study which allows a researcher to seek a clearer understanding of industry practitioners' perceptions, attitudes, behavioural and usage patterns with respect to a new emerging technology or paradigm such as km. focus group was used as a mechanism for sense-making and for grounding concepts from qualitative data. these concepts were represented using frames of reference. in our study, we have extracted and categorised km issues from the opinions and perceptions contributed by the focus group participants, from which a set of theoretical frameworks was grounded. the implications of the findings to is-based km have also been discussed. the results of this research effort reveal a general healthy state of km awareness and attitudes among cios and senior is professionals from several large organisations in singapore. the findings have overridden a previous presumption that km could be quite "unheard of among industries and organisations in singapore. at least this does not appear to be valid anymore within the is community. as the participants were mainly cios and senior is executives, the preliminary findings here may not necessarily apply to more junior is practitioners or other non-is executives since similar studies have yet to be conducted in singapore. presently some of the preliminary efforts for km in organisations are positive steps towards the institutionalisation of km. however these are merely exploratory and incremental in nature, often not co-ordinated by any km plans and strategies. our research also surfaces some difficult km problems and challenges (such as establishing the right organisational culture/mindset, and the task of tapping tacit knowledge), while at the same time reveals the existence of a perception gap between is practitioners and researchers (that is the perception of whether or not intelligent knowledge-based systems have potential strategic roles in is-based support for km). in general, based on the results gathered from the focus group participants, one or more of the following aspects of km are found to be lacking in some of these organisations. these include: • an in-depth insight into the parts of an organisation's business and process where knowledge bottlenecks could exist or which could be relatively more knowledge-intensive in nature than other parts. • an in-depth understanding of the magnitude and complexity of the km problem, involving a combination of both nkm and akm critical success factors. • a more focussed or integrated strategic km vision and plan; or incorporating km as part of their corporate strategic planning. • an appreciation of more sophisticated, intelligent or knowledge-based technologies and their relevance for is support in the context of km. this is understandable since km very often requires a multidisciplinary or cross-domain platform. • an understanding of the roles and responsibilities of km professionals and how they fit into the overall picture of the organisation. • an appropriate management approach to co-ordinate km processes and tasks; to merge or embed km activities into existing organisational practices and routines; as well as more specialised skill sets and tools necessary for engineering the knowledge content itself. we hope to extend our research exploration along some of the findings and research questions induced in the previous sections of this paper, with the hope that is researchers and practitioners could align is initiatives, planning, and management strategies, more accurately with those of km and business requirements. the outcome from such studies will enable organisations to reflect deeper on their existing practices in order to decide what type of km goals they should be setting for themselves realistically, and what level and complexity of km effort they would really need to invest in achieving these goals. such initiatives will facilitate their transformation into knowledge-efficient organisations as they compete in a dynamic knowledge-based economy. references aaai (1997) artificial intelligence in knowledge management, proceedings of 1997 aaai spring symposium, technical report: ss-97-01, aaai press, menlo park, ca. awad, e.m. (1996) building expert systems: principles, procedures and applications, west publishing company, st. paul, minn. beckman, t. (1997) "a methodology for knowledge management," in proceedings of the iasted international conference on artificial intelligence and soft computing, (asc '97), m.h. hamza (ed), banff, canada, iasted acta press, pp. 29-32. borghoff, u.m., and pareschi, r. (eds). (1998) information technology for knowledge management, springerverlag, new york. 54 aj1s special edition knowledge management december 2001 cortada, j.w., and woods, j.a. (eds). (1999) the knowledge management yearbook (1999-2000), butterworthheinemann, ma, boston, 1999. dhar, v., and stein, r. (1997) intelligent decision support methods the science of knowledge work, prentice-hall, hi. greenbaum, t.l. (1998) the handbook for focus group research, 2nd ed, sage publications, thousand oaks, ca, pp. 247. foray, d., and lundvall, b.a. (1996) "the knowledge-based economy: from the economics of knowledge to the learning economy," in oecd documents: employment and growth in the knowledge-based economy, oecd, paris, pp. 11-32. hbr (1998) harvard business review on knowledge management, harvard business school press, boston, ma. hendriks, p.h.j. (1999) "do smarter systems make for smarter organisations?" decision support systems, vol.27, pp. 197-211. holtshouse, d. (1998) "knowledge research issues," california management review, (40:3), pp. 277-280. liebowitz, j. (ed). (1999) knowledge management handbook, crc press llc, boca raton, fl. liebowitz, j. (2001) knowledge management: learning from knowledge engineering, crc press, boca raton, fl, pp. 49-56. neches, r., fikes, r., finin, t., gruber, t., patil, r., senator, t., and swartout, w.r. (1991) "enabling technology for knowledge sharing," ai magazine, (12:3), pp. 36-56. nonaka, i., and takeuchi, h. (1995) the knowledge creating company: how japanese companies create the dynamics of innovation, oxford university press, new york. oecd (1996) documents on employment and growth in the knowledge-based economy, oecd, paris. polanyi, m. (1966) the tacit dimension, routledge & kegan paul, london. shariq, s.z. (1998) "guest editorial: sense making and knowledge management: an assessment of technologies and applications from users' perspectives," journal of knowledge management, (2:2), pp. 7-9. strauss, a., and corbin, j. (1998) basics of qualitative research: techniques and procedures for developing grounded theory, 2nd ed, sage publications, thousand oaks, ca, pp. 3-14. smith, r.g., and farquhar, a. (2000) "the road ahead for knowledge management: an ai perspective," ai magazine, (21:4), winter 2000, pp. 17-40. tsui, e., garner, b., and staab, s. (eds). (2000) knowledge-based systems: special issue on knowledge management, (13:5). weick, k. e. (1995) sensemaking in organizations, sage publicaitons, thousand oaks, ca. wielinga, b., sandberg, j., and schreiber, g. (1997) "methods and techniques for knowledge management: what has knowledge engineering to offer?" expert systems with applications, (13:1), pp. 73-84. wiig, k.m. (1997) "knowledge management: an introduction and perspective," journal of knowledge management, (1:1), pp. 6-14. wiig, k.m., hoog, r., and spek, r. (1997) "supporting knowledge management: a selection of methods and techniques," expert systems with applications, (13:1), pp. 15-27. 55 ajis vol. 8 no. 1 september 2000 enhancing the linking of soft systems and information systems models through a uniform and formalised representation quan c. dang school of informatics and multimedia technology university of north london 2-16 eden grove. london n7 8ea, england abstract there are numerous works suggesting linking ssm models to information systems (is) models for information systems development. in these works links between ssm and is models are established at the conceptual level i.e. they provide ways to utilise ssm models to identify or to derive is models. however, most of them have not provided a method for representing the links explicitly in the combined model. consequently, the links become subtle and untraceable, looking at the combined model only at the representation level. this paper proposes an approach based on meta modelling to achieve a uniform and formalised representation of models that combine ssm and is models. the approach allows one to represent not only ssm and is modelling concepts but also relationships between the concepts in a meta model using the conceptual modelling language telos. telos supports meta models to be flexibly specified and extended to meet specific modelling requirements of a particular project. an exemplary application is presented to demonstrate ihe operationalisation of the proposed approach and to illustrate the uniform representation of combined models. the example also shows benefits of the formalised representation in terms of computer support for managing and retrieving a combined model's meta data. introduction it is well-recognised that the appreciation of the problem situation, in particular its relevant human activities, plays a crucial role in successful development of information systems, see e.g. checkland (1995), avison & wood-harper (1990) and stowell (1995). given this, soft systems methodology (ssm) (checkland, 1981) is advocated to provide the is analyst with a powerful methodology to obtain a rich understanding of the problem situation for which an information system is built (checkland, 1995; stowell, 1995). effectively, applications of ssm in information systems development (iso) range from the planning of is strategies (galliers, 1992), is requirement analysis (stowell & west, 1994), is analysis and design (avison & wood-harper, 1990), to evaluation of is methodologies (jayaratna, 1994). in this paper, we focus on the use of ssm in the context of information systems modelling (avison & fitzgerald, 1995; flynn & diaz, 1996; loucopoulos & zicari, 1992). specifically, we are concerned with the fashion in which soft systems models, and/or knowledge gained from soft systems analysis, are linked to information models, such as the data/structural model and process/behavioural model. for a review of these models, see e.g. avison & fitzgerald (1995). we begin by discussing issues inherent to the linkage of ssm to information systems modelling and drawbacks of the current approaches. subsequently, we propose an approach to overcome the drawbacks, based on meta modelling techniques; the language telos utilised to implement the approach and a method for constructing a meta model that unifies ssm and information models are outlined. this is followed by a section that presents an exemplary application of the method to build a meta model. benefits of using the meta model in modelling problem domains are highlighted. the final section gives an appraisal and discusses relationship of the proposed approach to other work. background: linking ssm and information systems models in soft systems practice, work that utilised ssm in helping information systems development was reported as early as in 1970 (checkland & griffin, 1970). following the publication of checkland (1981) a great many papers on using ssm in iso appeared, see e.g. stowell (1992), avison & fitzgerald (1993), stowell (1995). according to checkland (1995), these approaches can be classified into two main streams. the first stream involves linking ssm to well-established information systems design methods, e.g. ccta (1991), prior (1990), and savage & mingers (1996). the second stream explores ssm in creating an information strategy in an organisation, e.g. galliers (1992). the work in both streams faces the problems of making links between ssm's models and the data-oriented considerations or information systems models (hereafter called information models for short). the issues and problems related to linking ssm to iso are highlighted and discussed in numerous 29 ajis vol. 8 no. 1 september 2000 publications, from both methodological and technical viewpoints. detailed discussions can be found in, for instance, checkland (1992), miles (1988), mingers (1995) and checkland & howell (1998). this research has observed that none among the current approaches has provided a model combining ssm's and is models which has a uniform formalised representation. for a review of these approaches see e.g. mingers (1995). in fact, the issue of uniform and formalised representation of the combined model has not been addressed by most of the approaches. in gregory & merali (1993), a method for representing ssm conceptual models using prolog is described. however, this method does not address the general issue of the representation of the knowledge captured by soft system analysis. specifically, the knowledge captured by the rich picture is not included in the final gregory & merali's model. in other approaches, such as ccta (1991), savage & mingers (1996) and avison & wood-harper (1990), it is not unusual for combined models to be represented using several languages. these approaches just provide frameworks in which different (sub-)models are conceptually combined/linked, but the original languages of the constituting sub-models are still utilised as such. this results in the following drawbacks: • the links between the constituting models are not explicitly present in the final model, though they may be stored in the modeller's head or documented in an informal format; • the resultant models, or the knowledge captured by the modelling, are difficult to manage. because the models are represented in different formats, which may be both formal and informal, there is no formalised way to impose and check the model in terms of its consistency and (meta-)data redundancy over the entire model; • modelling activities are difficult to support by a software tool. these drawbacks may be overcome with a uniform and formalised representation of the combined model, provided that such a representation is capable of representing aspects captured by soft systems and information models as well as links between the aspects. in addition, the representation formalism used should not impose any extra structure on the soft systems modelling, in order not to constrict the exploratory nature of soft systems enquiry. the ssm incremental modelling process should also be supported when using the formalism. in the next section, an approach that implements these ideas using the conceptual modelling language telos (mylopoulos et al, 1990) is presented. a meta modelling approach meta modelling meta modelling is an activity of building a model of a model (kottemann & konsynski, 1984; mylopoulos et al, 1990; jeusfeld et al, 1997). it allows one to specify the semantics of, and structural relationships between, the modelling concepts of a model. the resultant meta model can then be exploited as a modelling language to represent the real world abstracted using the model. a meta model allows modelling languages to be specified in a meta modelling language, which facilitates the interoperability and communication among the constituting models (loucopoulos & zicari, 1992; jeusfeld et al, 1997; avison & fitzgerald, 1995). on the basis of a meta model, a case tool can be built to support modelling activities, maintaining the consistency and integrity constraints of the overall model. in order to achieve a uniform and formalised representation of the ssm models and information models we have utilised meta modelling techniques. our approach is to represent the set of modelling concepts of both ssm and information models along with their relationships using a meta modelling language. this language should meet the requirements indicated in the previous section. it is worth noting that the application of the meta modelling techniques is not straightforward because there is no universal meta modelling language. most of existing meta modelling languages have been developed to support a predetermined set of models, modelling formal and structured knowledge (jeusfeld et al, 1997; loucopoulos & zicari, 1992). the language to implement our approach needs to be capable of representing both formal and informal knowledge and supporting incremental modelling. the conceptual modelling language telos and the conceptbase system several languages have features that satisfy the indicated requirements, such as omega (attardi & simi, 1981), classic (brachman et al, 1992), and telos (mylopoulos et al, 1990). in comparison to omega and classic, telos is a conceptual modelling language that is specifically designed to 30 ajis vol. 8 no. 1 september 2000 represent the knowledge of categories that pertain to information systems development. for more details see mylopoulos et al (1990). telos is capable of representing informal knowledge that is represented in the natural language, as in soft systems models, because it implements the semantic network knowledge representation scheme, see e.g. quinllian (1968). besides, being an object-centred knowledge representation language, see e.g. nilsson (1982), telos supports various formal abstraction mechanisms such as classification, generalisation-specialisation and aggregation (brodie et al, 1984). a typed first order assertion sublanguage is offered in telos as a means of specifying integrity constraints and deductive rules. the other reason for choosing telos is that it is fully implemented in the conceptbase system (jarke, 1995 and jeusfeld et al, 1997). conceptbase is a deductive object base manager with a graphical user interface that allows telos objects to be created, stored, retrieved, visualised and queried using the system in both textual and graphical formats. specification of modelling languages as meta models in telos a meta model of modelling languages can be constructed using the primitive objects of the telos kernel object model (jeusfeld et al, 1997). this kernel model can be extended with objects that define new modelling constructs. structural and semantic constraints among the constructs are specified as telos objects, assertions and abstraction mechanisms. the resultant meta model's collection of objects can then be employed as a modelling language to represent application domain models. given the expressive power of telos, it can be used to build a meta model that is a modelling language that unifies ssm's models and information models. operationalisation of the approach: an exemplary application in this section, an exemplary application is presented in order to show the operationalisation of the proposed approach. the example demonstrates the use of telos to build a meta model of a model that combines ssm models and information models. the meta model is then used as a unifying modelling language for modelling a problem domain, resulting in a model with uniform and formalised representation. the score model (dang, 1997) is chosen for the purpose. however, it is worth noting that the presented method of building meta models is equally applicable to other models that combine ssm and information models. a bird's eye view of the score model the score model is designed with an aim to capture both human activity and information related aspects of an office environment in an integrated manner. in the model, an office environment is conceived as a problem situation with relevant human activities, and is modelled with soft systems. resultant soft systems models are then enriched with events, entities, rules and constraints that are involved in the relevant activities. the constructs of event, entity, rule and constraint are deliberately chosen to capture the dynamic, structural and declarative knowledge of a problem domain respectively. these are principal bodies of knowledge required to be captured by an information model (avison & wood-harper, 1990; loucopoulos & zicari, 1992; avison & fitzgerald, 1995 and flynn & diaz, 1996). thus, the score model represents the models that attempt to combine soft systems model and models of information modelling. a meta model of the score model a meta model of the score model has been built using telos, for details see dang (1997). the meta model defines all the modelling constructs of the score model (namely, rich picture, human activity system, conceptual model, entity, event, entity life cycle, rule and constraint) and the links between the constructs in the form of telos object specifications. the meta model's specifications capture explicitly the semantics of the score modelling constructs, which are of both soft systems and information modelling. this is illustrated by the object specification of the construct of conceptual model of a human activity system and the object specification of the construct of entity life cycle. the specification of the object conceptualmodels (figure 1) indicates that a conceptual model is built based on a root definition (see the attribute systemdefinition) and is composed of sub-activities linked 31 ajis vol. 8 no. 1 september 2000 by logical links (see the attributes subactivity and logicallink). moreover, the specification requires that at least two sub-activities must be specified in a conceptual model (see the constraint containstwodifferentsubactivities). individual conceptualmodels in scoreobjects,metaclass,class isa ssmconstructs with attribute,necessary,single systemdefinition : rootdefinitions attribute,necessary subactivity : humanactivitysystems; logicallink : links attribute monitorandcontrol : humanactivitysystems attribute,constraint containstwodifferentsubactivities : $ forall m/conceptualmodels al/humanactivitysystems (m subactivity al) ==> exists a2/humanactivitysystems ((m subactivity a2) and not (al==a2>) $ end figure 1: the specification of the conceptual model construct in comparison with the specification of the construct of conceptual model, the specification of the construct of entity life cycle is more complex (see figure 2). in the score model, the entity life cycle is represented using the program structure diagram (jackson, 1983). the specification is composed of a collection of related objects, namely, entitylifecycles, elc_elements, events, nullevents, sequences, iterations, selections and temporallinks. the specifications of these objects embrace not only three basic control flow structures (i.e. sequence, selection and iteration) and temporal order of events of an entity life cycle, but also commandments that must be conformed to when constructing an entity life cycle. for instance, the commandment that cross-links are not allowed in an entity life cycle's tree (jackson, 1983) is captured the constraint nocrosslink_in_a_lifecycle in the specification of entitylifecycles. in addition to the semantics of individual modelling constructs, links between the constructs are explicitly embedded in the meta model. these links are (meta) modelled in several ways: • by using attributes in specifications of modelling constructs, for example, the specification of the construct humanactivitysystems includes the attributes involvedentity, involvedevent and governedby, which indicate what entities, events, rules and constraints are involved in an activity; • by defining aggregate objects, for example entitylifecycles and richpictures; • by defining special objects to represent particular types of relationships between modelling constructs. for example, the constructs links, boundaries, binarylinks and directedlinks are useful for representing lines/arcs, circles, and arrows between elements of soft systems rich pictures and conceptual models, as well as for representing specific relationships between arbitrary objects of a problem domain. from the specification of the construct humanactivitysystems given in figure 3, it can also be seen that the meta model captures not only links that relate constructs belonging to an individual model (e.g. the relevance of a human activity to a problem situation and a conceptual model in a soft systems model), but also cross-model links that connect soft systems and information modelling constructs (e.g. the involvement of an entity in the execution of a human activity). 32 vol. 8 no. 1 september 2000 furthermore, because telos allows the meta-model to be adjusted and extended, new objects can be defined to meet specific modelling requirements, particularly to represent the knowledge that is not amenable to the pre-defined set of modelling constructs. this feature is useful, given the fact that the knowledge learned from a soft systems analysis is usually more than that is scripted using rich pictures, root definitions, conceptual models. in addition, using the conceptbase system, the model's consistency and integrity are easier to maintain. that is, in a conceptbase object base, deductive rules and integrity constraints defined in objects' specifications are automatically checked when a new object is added to the object base. thus individual entitylifecycles in scoreobjecte,metaclass,class with attribute.necessary consistof : elc_elements attribute,constraint nocros8link_in_a_lifecycle : $£orall e/elc_elements sql,sq2/sequences itl,it2/iterations sll,s!2/selections not(exists elcl/entitylifecycles (elcl consistof e) and ((eld consistof eql) and (elcl consistof itl) and (sql seq e) and (itl iter e)) or ((elcl consistof eql) and (elcl consistof sll) and (sql seq e) and (sll eel e)) or ((elcl consistof itl) and (elcl consistof sll) and (itl iter e) and (sll sel e)) or ((elcl consistof sql) and (elcl consistof sq2) and (sql seq e) and (sq2 seq e) and not(sq2==sql)) or ((elcl consistof itl) and (elcl consistof it2) and (itl iter e) and (it2 iter e) and not ( it 2 «i 11 ) ) or ((elcl consistof sll) and (elcl consistof s!2) and (sll sel e) and (e!2 sel e) and not(e12==sl1)))$ end individual elc_elements in scoreobjects,metaclass,class end individual events in scoreobjects,metaclass,class isa elc_elements with attribute underlyingactivity : humanactivitysystems; precondition : proposition; participatingentity : entities; effect : proposition end individual nullevents in scoreobjecta,metaclass,class isa events,elc_elements end individual sequences in scoreobjects,metaclass,class isa elc_elements with attribute,necessary seq : elc_elements attribute timeorder : temporallinks attribute,constraint containstwodifferentelements : $ forall s/sequences el/elc_elements (s seq el) ==> exists e2/elc_elements ((s seq e2) and not (el~°e2>) $; needstimeorder : $ forall s/sequences e/elc_elements (s seq e) = = > exists 1/temporallinks (s timeorder 1) $ end figure 2: telos objects necessary for representing the entity life cycle construct individual humanactivitysystems in scoreobjects,metaclass,class isa sshconstructs with attribute involvedentity : entities; involvedevent : events; governedby : rulesconstraints; relevantto : problems!tuations; hasrd : rootdefinitions; hasch : conceptualmodels end figure 3: the specification of the human activity system construct 33 ajis vol. 8 no. 1 september 2000 this supports soft systems incremental modelling. the example in figure 4 shows a snapshot of a conceptbase session in which an object that violates previously specified rules/constraints is rejected by the system when being added to the object base. once the meta model has been built, the modeller can employ the collection of the meta model's objects as a unified modelling language to model problem domains. server edit browsing options iielp tim* individual cm_admini3tering_admission in conceptualmodels, sbu_admissionobject3 with attribute, systeradefinition rootdef : rd_administeringadmi33ion attribute, subactivity al : administer_applicatioridocument3andprogress end protocol: tell: i nd i w i dua i crl_rdm i n i ster i ng_fldm i ss i on in conceptua i mode i s, sbu_fldm i ss i onob jects attr i bute,systemdef i n i t i on rootdef : rd_fidministeringfldmission attr i bute,subflct i u i ty a 1 : fldm i n i ster_rpp i i cat i ondocumentsflndprogress end error: pi connected rollback: how [71 of 71 figure 4: an object violating specified rules/constraint is rejected by conceptbase benefits of using the meta model the goal of building the meta model is to use it as a unifying modelling language to model a problem domain, which is abstracted with a combined set of ssm and information models' constructs. having said that, the proposed approach (viz. the unifying modelling language and the supporting software tool) is useful for documenting the post fact outcomes of the conceptualising activity, including the process of ssm. specifically, a model that has been represented using the language and stored in a conceptbase object base, documents only a particular objectively held view of a problem situation. continuing the score model example, the use and benefits of the proposed approach can be shown. in a case study, the score meta model is utilised to model the problem domain of the admission of students for university courses. for more details of this case study see dang (1997). the telos objects that represent the score meta model and the resultant model of the university admission can be seen as a hierarchy (see figure 5). 34 ajis vol. 8 no. 1 september 2000 metametaclass scoreobjects the unified modelling language metaclass entities v ° events/ 6 o / /\. humanactivitysys terns: ~ " ' simpleclass token applicants ..ii.courses "fin example problem domain model1 toni bsc.computing legends: ° telos-object the in relationship figure 5: the hierarchy of the objects representing the score model figure 5 illustrates the telos feature that there is no limitation of the classification/instantiation hierarchy, i.e. classes can belong to other (meta-) classes which can also belong to other (metame(a-) classes. it is this feature that imposes the problem domain model to inherit the semantic and structural properties of the modelling language defined by the meta model. to build a model of a problem domain, the objects of the meta model (e.g. humanactivitysystems, entities), which are metaclasses, are instantiated with objects to represent the problem domain's concepts (e.g. applicants and courses). the resultant objects are sunpleclasses that contain only tokens as their instances. tokens are objects that have no instances. tokens are employed to represent concrete objects of a problem domain, e.g. the applicant john and the course b.sc. computing, which are abstracted using the problem domain's concepts (viz. applicants and courses in this example, see figure 5). altogether, the score meta model (i.e. the unified modelling language), the problem domain model (i.e. the abstraction of problem domains in terms of concepts) and the concrete data of problem domains are uniformly represented as telos objects. the uniform and formalised representation of the meta model and the problem domain model as telos objects, which can be stored in the conceptbase system, enables the modeller to manage the knowledge captured by the model. a model's objects can be retrieved not only in the textual format of telos object specifications but also in graphical format. for example, the object conceptualmodels specified in figure 1 can be visualised as shown in figure 6. 35 ajis vol. 8 no. 1 september 2000 conceptbase graph browser conceptbase graph browser ness mĉ k... iff r: d subactivhy monitoranaconfcol containstwodiffeventsubactivities $ forall m/conceptualmodels al/hurnana "f_i * . erase any show attributes show instances show classes show subclasses show superclasses save layout load layout help quit figure 6: visualising an object with its classes and attributes using the described method, the linkage between ssm's and information models becomes seamless because their links have been incorporated in the meta model, whose objects are subsequently instantiated to model problem domains. the example in figure 7 shows that different aspects and relationships of the admission work are explicitly captured with telos object specifications. in the example, one can see not only individual modelled objects, such as the human activity handle_applicants_enquiries, the entities applicants, sbu_centraladmissionoffice and the event receiveenquiries, but also the links between them expressed by the attributes e.g. underlyingactivity, involvedentity and involvedevent. 36 ajis vol. 8 no. 1 september 2000 individual applicants in simpleclass,entities,sbu_admissionobjects,class attribute who : "people who apply for courses"; surname : string end individual receiveenquiries in simpleclass,events,sbu_admissionobjects,class with attribute,underlyingactivity al : handle_applicants_enquiries end individual handle_applicants_enquiries in humanactivitysystems,simpleclass,sbu_admissionobjects,class attribute,involvedentity ietyl : applicants; iety2 : sbu_centraladmissionoffice attribute,involvedevent ievtl : receiveenguiries; ievt2 : recordanenquiries end figure 7: unification of modelled aspects at the representation level the explicit capture of links in the model is seen by the fact that questions (about the knowledge captured by the model) like the ones listed below can be answered, retrieving data from the problem domain's model object base. • which entities share a given event? • which human activity systems share a given event? • which events happen in the performance of a given human activity system? • which entities are involved in the performance of a given human activity system? • in which human activity systems does a given entity participate? • which rules and constraints govern the performance of a given human activity system? answers to these questions can be automatically obtained using telos query classes and the display queries tool of conceptbase. for example, figure 8 illustrates the use of a query to retrieve the events, which are modelled and stored in the conceptbase knowledge base, involved in the human activity handle_applicants_enquiries. summary and discussions in this paper we have proposed an approach to achieve a uniform and formalised representation of a model that combines ssm's models and information systems models. the approach is motivated by the idea that the linking of ssm's and information models can be enhanced by the explicit capture of the links by a final model that has a uniform formalised representation. the idea is implemented using meta modelling techniques and the conceptual modelling language telos. in comparison to other approaches that attempt to link soft systems and information modelling, our approach has several advantageous features. first, it is capable of uniformly representing the knowledge gained from the soft systems analysis as well as the knowledge captured by information modelling concepts. second, the approach does not reduce ssm into a reductionist method, but it supports the soft systems model's extendibility and ssm incremental learning. finally, the representation of the unified model is formal in the form of telos objects, resulting in a model that can be maintained and exploited using a knowledge base management system (viz. conceptbase). having these features, the proposed approach has thus overcome the drawbacks highlighted in the second section of this paper. 37 ajis vol. 8 no. 1 september 2000 server edit browsing options receiveenquiries in eventsinvolvedinhandle_applicants_enquiries end recordftnenquiries in eventsinvolvedinhandle_applicants_enquiries avt ft ciiu. crossljfecycles eventslnvo)vedlnhanute_applicants_enquiries gq2_eventslnvolvedlnhumanactivity systems gq_crossljfecyctes gq_entitieslnvojvedln human activity reauit.:; anfyer receiueenquiries in euentsinyoluedinhandle_applicants_enquiries end recordfinenquiries in evervtslnvolvedlnhandle applicants enquiries end rollback: now figure 8: an example of a query class and its answered objects referring to the 'grafting' and 'embedding' taxonomy (miles, 1988), the meta modelling approach belongs to neither of the classes of approaches. this is because our approach does not address the issues of linking ssm models with an information modelling approach or design method at the conceptual level. the meta modelling approach deals only with the representation of models that combine ssm and information models. it takes a combined model as given and provides a method to produce a unifying modelling language derived from the given combined model. thus the method to generate a unifying modelling language of a combined model is applicable to models of both 'grafting' and 'embedding' approaches. in terms of the links between ssm and information models, the method outlined in the paper deals with the representation of only those links between ssm's and information models that have been purposively established by the modeller at the conceptual level. as such, the meta modelling method proposed by this approach may be utilised as a complementary component to achieve a uniform formalised representation of models combining ssm's and information models at the conceptual level proposed by other approaches such as multiview and savage & mingers' (1996). finally, because the use of the unifying language of a combined model requires the knowledge about telos and the supporting software tool, it might be difficult for the modeller to use the language to build initial models of problem situations without using the ssm conventional modelling language. however, the unifying language can be used to create a model's objects repository. the model's objects stored in the repository can subsequently be exploited using the conceptbase's tools for the analysis and reference purposes. 38 ajis vol. 8 no. 1 september 2000 acknowledgements the author wishes to gratefully acknowledge the helpful discussions and contributory suggestions of dilip patel, shushma patel and malcolm crowe to improve this paper. references attardi, g. and simi, m. (1981) consistency and completeness of omega, a logic for knowledge representation. proceedings of the 7th ijcai, vancouver, 504-510. avison, d.e. and wood-harper, a.t. (1990) multiview: an exploration in information systems development. blackwell, oxford. avison, d.e. and fitzgerald, g. (1995) information systems development: methodologies, techniques and tools. 2nd edition. mcgrawhill, london. brachman, r.j., borgida, a., mcguinness, d.l., patel-schneider, p.p. and resnick, l.a. (1992) the classic knowledge representation system, or, kl-one: the next generation. proceedings of the international conference on fifth generation systems, tokyo. brodie, m.l., mylopoulos, j. and schmidt, j.w. (eds) (1984) on conceptual modelling: perspectives from artificial intelligence, databases, and programming languages. springer-verlag, new york. ccta (central computer and telecommunication agency) (1991) ssadm version 4 reference manual: feasibility soft systems methodology. ccta, norwich. checkland, p.b. and griffin, r. (1970) management information systems: a systems view. journal of systems engineering, 1, 29-42. checkland, p.b. (1981) systems thinking, systems practice. john wiley & sons, chichester. checkland, p.b. (1992) ssm in information systems design: history and some current issues. systemist, 14, 90-92. checkland, p.b. (1995) soft systems methodology and its relevance to the development of information systems. in stowell (1995). checkland, p.b. and howell, s. (1998) information, systems and information systems: making sense of the field. wiley, chichester. dang, q.c. (1997) a soft-systems-conceived model with knowledge representation for information systems in the office environment. ph.d. thesis. south bank university, london. flynn, d.j. and diaz, o.f. (1996) information modelling, an international perspective. prenticehall, london. galliers, r. (1992) soft systems, scenarios, and the planning and development of information systems. systemist, 14, 146-159. gregory, f. and merali, y. (1993) inductions, modality and conceptual modelling. warwick business school research bureau research paper 79. warwick university. jackson, m.a. (1983) systems development. prentice-hall, hemel hempstead. jarke, m. (ed.) (1995) conceptbase v4.1 user manual. rwth aachen, aachen. jayaratna, n. (1994) understanding and evaluating methodologies, nimsad: a systemic framework. mcgraw-hill, maidenhead. jeusfeld, m.a., jarke, m., nissen, h.w. and staudt, m. (1997) conceptbase: managing conceptual models about information systems. in handbook on architectures of information systems, beraus, p., merlins, k. and schmidt, g. (eds). springer-verlag, berlin (in press). avison, d.e. and fitzgerald, g. (eds) (1993) journal of information systems, volume 3, issue 3. blackwell, oxford. kottemann, j.e. and konsynski, b.r. (1984) dynamic metasystems for information systems development. proceedings of the 5th international conference on information systems, arizona. loucopoulos, p. and zicari, r. (eds) (1992) conceptual modeling, databases, and case: an integrated view of information systems development. wiley, new york. miles, r.k. (1988) combining "soft" and "hard" systems practice: grafting or embedding?, journal of applied systems analysis, 15, 55-60. mingers, j. (1995) using soft systems methodology in the design of information systems. in stowell (1995), pp. 18-46. 39 ajis vol. 8 no. 1 september 2000 mylopoulos, j., borgida, a., jarke, m. and koubarakis, m. (1990) telos: a language for representing knowledge about information systems. acm transactions on information systems, 8, 325-362. nilsson, n.j. (1982) principles of artificial intelligence. springer-verlag, berlin. prior, r. (1990) deriving data flow diagrams from a "soft systems" conceptual model. systemist, 12, 65-75. quinllian, m.r. (1968) semantic memory. semantic information processing, minsky, m. (ed.). mit press, cambridge, mass.. savage, a. and mingers, j. (1996) a framework for linking soft systems methodology (ssm) and jackson system development (jsd). information systems journal, 6, 109-129. stowell, f. and west, d. (1994) client-led design, a systemic approach to information systems definition. mcgraw-hill, london. stowell, f. (ed) (1995) information systems provision the contribution of soft systems methodology. mcgraw-hill, london. stowell, f. (ed) (1992) systemist, special edition on information systems, volume 14, issue 3. 40 microsoft word 13 1 marina.doc ajis vol 13, no. 1 september 2005 78 executive information systems and the top-officers’ roles: an exploratory study of user-behaviour model and lessons learnt emmanuel matthew ikart school of management, marketing & industrial relations university of wollongong, nsw 2522 email: emi01@uow.edu.au abstract in recent years a number of organisations have implemented executive information systems (eis) in order to improve the performance of their executives’ jobs. although the use of eis is important in executives’ work, the majority of executives are unwilling to use eis applications because of their design flaws. by using social factors, habits and facilitation condition variables from triandis’ framework, this paper extends the technology acceptance model (tam) to derive useful variables to address the problem of the low usage of eis by executives. this paper reports on research in progress in australia on the adoption and usage of eis by executives. the preliminary results suggest that executives’ experiences in eis positively relates to their experiences in computer-based information systems. the results also suggest there is a high degree of perceived usefulness, perceived ease of use as well as positive attitudes towards using eis. further, the results suggest that executives consider social factors in using eis in their work. moreover, the results suggest that facilitating conditions such as eis development process, eis management process and organisational environment are strongly related to the adoption and usage of eis by executives. finally, the results suggest a higher degree of eis usage by middle managers than top-level managers, which an eis was meant to support. keywords: eis, cultural, social, organisational factors, executives, theoretical foundation, user acceptance, usage introduction executive information systems (eis) are computerised systems that provide executives with on-line easy access to internal and external information relevant to their business success factors (rainer and watson, 1995). the aim of eis is to bring information from the external environment and all parts of an organisation and present it in a way that is meaningful to executive users (mcbride, 1997; ikart, 2004a, 2004b). nonetheless, the actual engagement or use of these systems by executives is relatively low (young & watson, 1995; thodenius, 1995; fitzgerald, 1998). in recent times the use of eis in organisations has spread to managers at various levels (leidner & elam, 1994; nord & nord, 1995; fitzgerald, 1998; vlahos et al., 2000; salmeron, 2002; singh et al., 2002). although this spread has given eis new names such as enterprise information system, business intelligence (bi) software and balanced scorecard (liang & miranda, 2001, ikart 2004a, 2004b), the problem of underutilisation by the executives remains unresolved. in fitzgerald and murphy (1994) for instance, the usage gap between executives and middle managers was 36%. they found that only 32% of eis users were at executive level while the majority (68%) of users were at middle management level. this finding was confirmed by a further study by fitzgerald (1998). these findings suggest a higher degree of utilisation of eis at the middle management level, than at the strategic management level, which is the level an eis was meant to support. because the implementation and operations of eis are more or less championed by senior ajis vol 13, no. 1 september 2005 79 executives with broad and general management support (thodenius, 1995; mcbride, 1997; fitzgerald, 1998; poon & wagner, 2001) one would expect a significant level of use by the senior management. but the actual use by senior managers is very small (kraemer et al., 1993). a number of researchers (e.g., robey, 1979; szajna, 1993; davis, 1993; young & watson, 1995; mao, 2002) have investigated organisational and technological factors that determine user acceptance of is including eis. although these research efforts have provided some valuable results, they have been constrained by lack of appropriate reference theoretical foundations and variables for key determinants of user acceptance and use of information systems (ikart, 2004a). kling (1991) who studied the social impact of human computer argued that, “ in order to identify the social impact of computing one must have at least implicitly a theory of the casual power that computerised systems can exert upon individuals, groups, organisations, institutions, social networks, social world and other social entities” (p. 151). several studies (bergeron et al., 1995; kumar & palvia, 2001; singh et al., 2002) have reported the growing popularity of eis in organisations as new concepts such as enterprise resources planning (erp), data warehousing, data mining, web-base portal to “dashboard” and “scorecards” and the on-line analytical processing (olap) engine have paved the way for a new era of managing corporate data. despite these, the underutilisation of eis by senior managers remained an important challenge to user organisations. “top officers don’t use executive information systems” (wildt, 1991 p. 38). the characteristics of eis including the ability to move freely between a high-level view of data and a detailed view (‘drill-down’) a concentration on data relating to key performance indicators and critical success factors, the ability to highlight exceptions and variances automatically and to present information in graphical, tabular, textual and colours to the executives make eis a suitable tool for executives’ work (mcbride, 1979; ikart, 2004a, 2004b). the motivation for this paper is the marked limited research on the actual use of eis by executives and the lack of an appropriate reference theoretical foundation for individual, organisational, social and cultural variables in determining the factors for user acceptance and use of eis. the primary focus of the paper is to investigate organisational, social, cultural and individual factors that can explain executives’ behaviour towards the adoption and usage of eis in organisational settings. furthermore, the paper aims to highlight and explain the importance of these factors in determining the adoption and usage of eis by executives. the potential contribution of this paper is the research model based on organisational behaviour theories which provides a future direction in explaining executives’ behaviour towards eis adoption and usage. the preliminary results reported in this paper provide better understanding of the behaviour of executives in using eis. moreover, the results would assist eis developers to understand the core information processing requirements for executives’ tasks for which they are building eis in order to implement appropriate system functionalities to support those tasks. in an academic sense, the new framework and research model will assist researchers to further explain human behaviour towards is acceptance and usage. the remainder of the paper is organised as follows. first, we present different definitions of eis and choose the definition suitable for the present study. second, we examine the nature of executives’ work and how eis fits-in. third, we examine past research studies on eis usage. fourth, we address the research problem and research questions of the study. ajis vol 13, no. 1 september 2005 80 further, we look at the theoretical perspective in is research including tam and its implications in explaining user acceptance and use of is as well as triandis’ (1979) framework variables such as habits, social factors and facilitating conditions relevant for the study. next, we develop the research model for the study followed by the research hypotheses and implications from empirical studies. finally, we provide the research methodology, data collection methods, questionnaire design, the preliminary results, conclusions and future direction. definition of eis what is an eis? the definition of an eis varies according to the weight attached to the three elements in the name itself. executives’ roles differ in terms of personal management styles, company size and hierarchy (nord and nord, 1995). according to rockart and de long (1992), an eis is the routine use of a computer terminal for any business function. the users are either the ceo or a member of the senior management team reporting directly to him or her. executive support systems can be implemented at the corporate level and/or divisional level. bergeron et al (1995) also share the same view. thierauf, (1991) provides yet another classic definition of eis as, “computer-based systems with data sources and programs which locate the desired data, place it in a common format, massage it into useful form and present it as a useful information for the executive” (p. 9). supporting the contention, turban, (1993) defines an eis as “a computer based system that serves the information needs of the top executive. it provides rapid access to timely information and direct access to management reports. it is user friendly supported by graphics and provides exceptions reporting and “drill-down” capabilities. it is connectable to on-line information services and electronic mails” (p.394). for the purpose of the present study, eis is defined as a computerised system that provides executives with on-line easy access to internal and external information relevant to their critical business success factors (rainer and watson, 1995). figure 1 below depicts a general model of an organisational eis. it shows how management collects the internal transaction data. and how data from the external source can be retrieved and stored in an organisational database. data from these two sources can then be transformed and/or manipulated on eis screens into formats that facilitate the development of management reports and answers to queries. executives are particularly interested in summaries and compilations of information. this transformation provides meaningful formats in terms of colours, graphics and text to aid executive users (watson et al, 1991; nord and nord, 1995). ajis vol 13, no. 1 september 2005 81 figure 1: a model of an eis adapted from nord j.& nord g.1995, p.96 executives’ work and how eis fits-in the nature of executives’ work has been characterised as brevity, variety, interrupted and fragmented (e.g., mintzberg, 1973; kotter, 1982). as referenced in ditsa, (2002), “there is no position in the organisational hierarchy that is less understood than that of executives” (p.184). moreover, the functions, and how those functions are performed, vary between organisations and between executives within an organisation. indeed, one of the many reasons of eis failures in organisations has been the lack of understanding of executives’ work by the eis developers. executives’ work has traditionally been related to identifying problems and opportunities and making the decisions of what to do with those problems and opportunities. in addition to playing leadership roles expected of them much of the work of executives revolve around developing agendas, goals, strategies that may not be documented, establishing networks and developing corporate relationships with people inside and outside their organisations who may play a role in developing future agenda (ditsa, 2002). this section examines the nature of executives’ work in terms of 1), kotter’s “job demands” agenda setting and “network building” 2), anthony’s planning and control model and 3), mintzberg’s roles model theory. kotter’s “job demands” agenda setting and “network building” kotter (1982) studied fifteen executives through observation, questionnaire and interviews in nine usa corporations. he identified the job demands of executives and categorized them into agenda setting, network building and execution. according to him, in agendasetting, executives developed loosely connected goals and plans relating to short, medium and long terms responsibilities in the form of formal documentations and ideas in their heads. this is based on executives’ knowledge of their work and information they gathered through primary discussions with people. in network building, they develop networks of relationship with people such as subordinates, peers, bosses, outsiders and those they feel would make an important contribution to a successful implementation of the agendas. at internal transaction data management subset data transformation eis screen eis screen external data ajis vol 13, no. 1 september 2005 82 the execution level, executives mobilise support from the network for the implementation of the agendas. anthony’s planning and control model in his work, anthony (1992) suggests that executives’ roles fall into strategic planning, management control and task control. in strategic planning executives determine organisational goals and develop strategies for achieving the goals. in management control, they influence the participants to implement the established strategies. in task control, they ensure specific tasks are handled with effectiveness and efficiency. each of these activities requires different types of information. although at the strategic planning level executives rely on summary information from external sources, at the task control level, they rely on detailed information generated within their organisations. the information for management control falls within strategic planning and task control levels (ditsa, 2003). mintzberg’s roles model theory in his research work, mintzberg (1973) describes executives’ jobs in terms of ten roles. he categorises these roles into three major roles such as interpersonal, informational and decisional roles. he subdivides the interpersonal role into figurehead, leadership and liaison roles. furthermore, he subdivides the informational role into monitor, disseminator and spokesperson roles. the decisonal role, he subdivides into entrepreneur, disturbance handler, and resources allocator and negotiator roles. in our opinion, mintzberg’s roles model best characterises executives’ roles. moreover, these categorisations involve dealing with information. eis have been designed to support executives in these roles. eis clearly fit into mintzberg’s roles model theory of executives. perhaps, the advancement in is software and hardware such as web browser, cd-rom, internet, enterprise resources planning, data mining and data warehousing which are common features of eis contribute significantly to this fitness (thodenius, 1995; singh et al., 2002). although eis may not completely replace the ways that executives’ acquire and use their information such as intuition and face-to-face (mintzberg, 1973), they support executives’ work and relieve executives to focus on other unstructured problems. eis can capture both “hard” and “soft” forms of information from the internal and external business environments. the former refers to numbers and figures while the latter refers to explanations of the numbers presented e.g., hearsay, rumours and opinions. strategically, the soft part of information provides additional meanings and richness to the hard data. further, the capabilities of eis enable executives to quickly search and scan their organisational environment for threats and opportunities for immediate and appropriate decisions. eis are business tools that support and improve the decision-making process of the executive by providing the basic usable and relevant information from both the internal and external environments of organisation. moreover, because executives devote much of their time to acquire and analyse information through interactions with people and processing of documents, eis are designed to save considerable amount of executives’ time by facilitating the collection, retrieval, storage and analyse of information. the “what-if” analytical capability of eis combined with executives’ decision-making imagination and judgment support executives to arrive at a decision easily and with accuracy. ajis vol 13, no. 1 september 2005 83 past research studies on eis usage research studies on eis usage can be broken into five major areas as follows: (i) factors influencing and explaining use, (ii) overall benefits from eis, (iii) pattern of use and frequency of use, (iv) impact of eis on managerial activities, and (v) emergence of eis. in the classifications below (table 1), the majority of research on eis has been exploratory instead of theory testing. only a limited number of studies (e.g., bergeron et al., 1995; ditsa, 2003) in research area (i) employed appropriate reference theories to gain insight into factors influencing the actual use of the systems by senior managers. without appropriate reference theories, it will be difficult to realise the importance of the other four focussed areas. research areas factors influencing/explaining eis use ii overall benefits from eis iii patterns of use & frequency of use iv. impact of eis v. emergence of eis reference young & watson, 1995; rainer & watson, 1995; bergeron, et al., 1995; basu et al., 2000; poon & wagner, 2001; singh et al., 2002; ditsa, 2003 nord & nord, 1995; kelly, 1994; mcbride, 1997; nanhakumar & jones, 1997 thodenius, 1995; seeley & targett, 1999; rockart and delong, 1992; laidner & elam, 1994; handzic, 1997; wheeler, 1996; stein & nasib, 1997; liang & miranda, 2001; kumar & palvia, 2001; salmeron, 2002. houdeshel & watson, 1987; rockart & treacy, 1992; fitzgerald & murphy, 1994; fitzgerald, 1998, table 1: classification of eis usage research nandhakumar and jones (1997) witnessed an eis development project in their in-depth study of the development methods in organisations where potential executive users were not involved in the design phases. as a result, they suggest that there should be better theoretical conceptualisation of the dynamic relationship between the developers and executives to assist in understanding how the relationship shapes, and is shaped by various constraints. mcbride (1997) studied the progress of an eis project within a manufacturing organisation in the uk over a 9-year period. the study demonstrates the importance of the interaction between the business environment, the organisational environment and the perceptions and interpretations of events by stakeholders on the success or failure of eis. particularly, it illustrates the importance of the organisational context and the dynamic nature of the social, economic and technical factors critical in shaping acceptance and use of eis in organisations. ajis vol 13, no. 1 september 2005 84 the above two studies suggest that the roots of the success or failure of is including eis can be attributed to social, cultural and organisational factors and not technical factors alone. research problem and questions the research problem that this study seeks to address is the low usage of eis in organisations by executives. to provide a solution to the research problem, a research model from organisational behaviour as a theoretical foundation is employed. the main objective of this study is therefore to investigate and examine cultural, social, and organisational factors that explain senior managers’ behaviour in accepting and using eis. the following research questions are addressed in this study to provide answers to the research problem. the next subheading examines the theoretical foundation for the study. • what are the social, cultural and organisational factors that explain senior managers’ behaviour towards using eis? • what are the importances of these factors in determining eis usage by senior managers of organisation? theoretical perspective in is research a number of theories are used in is research areas to explain individuals’ behaviour towards computers. candidates among these theories include, task technology fit model (e.g., dishaw and strong, 1997); institutional theory, coordination theory and organisational complexity model (e.g., yager, 1997); contingency model (e.g., lauer and rajagopalan, 2002); variance theory and process theory (e.g., seeley and targett, 1999); general system theory (e.g., raisinghani and schkade, 1997); diffusion theory (e.g raisinghani and schkade, 1997; mao, 2002); activity theory (e.g., verenikina and gould, 1997) and the theory of plan behaviour (mathieson, 1991; taylor and todd, 1995). these theories have been acknowledged in is research because they assist researchers to gain a useful insight into the reaction of people towards computer technology and factors enabling the reactions. for instance, activity theory aims to explain the connection between human psychology and computer interface design in a social work environment. this establishes the relationship between human computer interactions and computer interface design by taking into consideration the context of the work environment (verenikina and gould, 1997). also, the task-technology fit model aims to match the capability of the technology to the demand of the technology in a work environment (dishaw and strong, 1997). unfortunately, none of the above theories examines explicitly organisational contextual factors such as cultural, social and organisational variables that can explain executives’ behaviour towards eis adoption and use because they have insufficient reference variables. the technology acceptance model (tam) (davis, 1989, 1993 davis et al., 1992), which is derived from the theory of reasoned action (tra) (fishbein, 1979), has been widely used, as a theoretical foundation to explain is acceptance and usage. moreover, triandis’ (1979) framework has been used as a theoretical foundation by previous studies (e.g., bergeron et al., 1995; ditsa, 2002, 2003) to address explicitly the social, culture and organisational factors that can explain executives’ behaviour towards the adoption and usage of eis for strategic activities. both tam and triandis’ framework have separately guided several researchers (davis 1989; davis, et al., 1992, davis, 1993; dishaw & strong, ajis vol 13, no. 1 september 2005 85 1997; srivihok, 1999; kwon & chidambaram, 2000; mao, 2002; thompson et al., 1991; bergeron et al., 1995; ditsa, 2002, 2003) to explain human behaviour towards the adoption and use of computers. this paper uses both tam and triandis’ framework as theoretical foundations. the paper extends tam with variables habits, facilitating conditions and social factors from triandis’ framework to derive a research model suitable for the adoption and use of eis by organisational executives. technology acceptance model (tam) davis (1986) developed tam to explain human computer-usage behaviour using fishbein and ajzen’s (1975) tra as the theoretical basis. the objective of tam is to provide an explanation of the determinants of computer acceptance that is capable of explaining the behaviour of users across a broad range of end-user computing and user populations while simultaneously being parsimonious and theoretically justified (davis, 1989). tam uses tra to specify causal linkages between two relevant sets of constructs perceived usefulness (pu) and perceived ease of use (peou) and user attitude (a), behavioural intention (bi) and actual computer usage behaviour. figure 2: technology acceptance model davis et al., (1989, p. 320) define pu as the user’s “subjective probability that using a specific application system will increase his/her job performance within an organisation context”. davis defines peou as “the degree to which an individual believes that using a particular system would be free of physical and mental effort” (davis, 1993 p. 447). while peou relates to the assessment of the intrinsic characteristics of it such as ease of use, ease of learning, flexibility and clarity of its interface, pu on the other hand is a response to user assessment of its extrinsic, i.e., task-oriented, outcomes. that is the manner in which it helps users achieve task-related objectives, such as task efficiency and effectiveness (gefen and straub, 2000). in tam both pu and peou influence an individual’s attitude towards using computers. pu and attitude influence the behaviour intention to use the system. actual system use is predicted by the behaviour intention. davis et al., (1989) note externa lvariabl es perceive d useful perceive d ease of use attitude towards using actual use behavio ur intention ajis vol 13, no. 1 september 2005 86 the arrows in tam in figure 2 above indicate the probable causality. a review of scholarly research on is acceptance and usage suggests that tam has emerged as the most influential model in this stream of research (robey, 1996; davis 1989; davis et al., 1989, ikart, 2004a, 2004b) including e-commerce and the adoption of internet technology (e.g., gefen and straub, 2000) as well as, knowledge management systems (e.g., money & turner, 2004). tam with its original emphasis on system design characteristics represents an essential theoretical contribution in understanding is usage and acceptance behaviours (davis et al., 1989). for instance, davis (1989) originally examined an email system and file-editor used at the time at ibm canada and found the peou and pu of tam to be significantly correlated with self-reported use of the system. on the contrary however, tam does not account for habits, facilitating conditions and social factors variables provided by triandis’ framework that influence behaviour. davis et al. (1989) realised that the omission of the subjective norm from tam represents an important area that requires further study. moreover, they observed that the theoretical basis of tra makes it difficult to distinguish if behaviour is caused by the influence of referent on one’s intent or by one’s own attitude. davis (1986) for instance noted that “the subject may want to do what referent x thinks he/she should do, not because of x’s influence, but because the act is consistent with the subject’s own [attitude]”. thus, davis and davis et al. underscored the importance of social norms that can explain behaviour. nonetheless, they highlighted the importance of the development knowledge from tam. in this paper we use tam as the basis of the conceptual model and incorporate social factors, habits and facilitating conditions from triandis’ framework as an extension. we examine the selected triandis’ (1979) framework variables that are relevant to the paper in the next section. triandis’ theoretical framework triandis (1979) presents a theoretical framework with a central theme which focuses on the relationships of values, attitude, and other acquired behavioural dispositions to action or behaviour. the framework defines the relationship involving these concepts. the variables to be used from triandis’ framework for the study are: social factor, habits and facilitating conditions. this study examines this subset of triandis’ framework. for a thorough discussion of the model, the reader should refer to triandis (1979). habits: triandis defines habits as “situation-behaviour sequences that are or have become automatic such that they occur without self-instruction” (p. 204). facilitating conditions: he defines facilitating conditions as “objective factors which are out there in the geographical environment such that several judges or observers can agree make an act easy to do” (p. 205). acts he says are socially defined patterns of muscle movements. social factors: personality, triandis states, internalises the cultural way of perceiving the social environment, called the subjective culture of the group. the subjective culture consists of: norms (self-instruction to do what is perceived to be appropriate by members of the culture in certain situations); values (the tendencies to prefer a state of affairs over another; roles (appropriate behaviour by a person holding an office in a group) and, social situation (a behaviour setting where more than one person is present). the internalisation of ajis vol 13, no. 1 september 2005 87 a culture, triandis argues, forms the social factors that influence the intention to behave. we develop the research model for the study in the next section. the research model the research model (figure 3 below) is based on tam plus an extension derived from selected variables habits, facilitating conditions and social factors – from triandis’ framework. as far as possible all constructs in the research model are operationalised on the basis of previous studies. habits consist of eis experience and ability to use eis. according to triandis (1979), habits are what people usually do and the individual’s is usually not conscious of the consequences, for example, driving a car. triandis links habits to an individual past experience and ability to perform a given act. he argues that the habitual nature of a behaviour will have an influence on the individual’s response to a given situation. he further argues that for many behaviours habit is more important than intentions (triandis, 1979). accordingly, habits are measured by assessing the number of years of executives’ experience in using eis and their ability to use eis (bergeron et al., 1995; ditsa, 2002, 2003). shneiderman (1998) classified computer users into novice users, knowledgeable intermittent (casual users) and frequent (expert users). the ability of executives to use eis will be measured by assessing executive user class. the facilitating conditions consist of eis development processes, eis management processes and organisational environment variables (ikart, 2004a, 2004b). the first variable is measured by assessing the degree to which the eis development process in an organization facilitates the use of eis by the executives using similar questions used by prior studies (e.g., nandhakumar & jones, 1997; ditsa, 2002, 2003) with five-point likert scales. the second variable is measured by assessing the degree to which the eis management process facilitates executives’ use of the systems using similar questions used by nandhakumar and jones, (1997) and (ditsa, 2002, 2003) using five point likert scales. the third variable is measured by assessing the degree to which an organizational environment facilitates the use of eis by the executives using five similar questions derived from nandhakumar & jones (1997) and, mcbride, (1997) and ditsa (2002, 2003), using five-point likert scales with 1 for strongly disagree and 5 for strongly agree. these scales are found to be reliable with cronbach’s alpha equal to 0.76, 0.74 & 0.70 respectively (ditsa, 2002, 2003). social factors consist of subjective norms, subjective roles, subjective values and subjective social situations (ikart, 2004a,b). in the eis research domain the social factors have been referred to the executives’ work group influence such as, peers; superiors; subordinates and is directors acting upon their eis use (bergeron et al., 1995; ditsa, 2002, 2003). the social factors variables are measured as follows: the subjective norms (self-instructions to do what is perceived to be correct and appropriate by the work group) are measured by obtaining users’ assessment of the influence of the work group upon their behaviour in general (four 5-point likert scales (-2: strongly disagree, +2: strongly agree) and multiplied by evaluating their probability that the work group wants them to use eis (bergeron et al., 1995; ditsa, 2003). subjective roles (an expected correct behaviour from executive users of eis) are measured by obtaining executives’ assessment of their roles and expected behaviours from group work in relation to eis usage using four 5-point likert scales. subjective values (the broad tendencies of the executives’ work group to prefer a certain state of affairs over others in relation to eis usage) are measured by obtaining executives’ ajis vol 13, no. 1 september 2005 88 assessment of the work group influence in relation to eis usage using 5-point likert scale. subjective social situations of the workplace setting are measured by obtaining executives’ assessment of their interpersonal relationships with their peers, superiors, subordinates, the is directors and the eis support group in relation to eis usage. five 5-point likert scales are used for the measure. the scales were obtained from past studies (e.g., bergeron et al., 1995; ditsa, 2003) as they appear to be reliable with a cronbach’s alpha of 0.81, 0.9 and 0.86. both the pu and peou constructs are operationalised by obtaining users’ assessment of their pu and peou of eis based on 12 similar items, six items for each developed, refined and streamlined by davis (1989) using 7-point likert scales. thus, pu and peou are adapted from several is studies (e.g., davis, 1989; davis et al., 1989; matheison, 1991; venkatesh & davis, 1996; moore & benbasat, 1991; venkatesh et al., 2003). prior studies (e.g., davis, 1989) demonstrate the validity and reliability of pu and peou with cronbach alpha of 0.98 and 0.94 respectively (davis, 1989). the scales have been adjusted to 5-point likert scale, with one being the negative end of the scale and five being the positive end of the scale. the adjustment was based on both the advice of the statistician consulted in the project design phase and the pre-test results. attitude towards usage is measured using five standard 5-point semantic differential scales for operational attitude toward behaviour. the question posed is “all things considered, my using eis in my job is good – bad; wise – foolish; favourable – unfavourable; beneficial – harmful; and positive – negative”. the scales are 5-point scale with midpoint labelled “neutral”. although, azjen & fishbein, (1980) suggested five standard 7-point semantic differential rating scales, the scales have been adjusted to 5 based on the advice of the statistician and the outcome of the pre-test. attitude measurement is therefore adapted from prior is studies (e.g., robey, 1979; davis, 1989; davis, 1993; malhotra & galletta, 1999; hubona & jones, 2002; mao, 2002). the actual system use construct is measured by obtaining users assessment in terms of frequency of use, (‘how often’). similar measures have been used in research on tam (e.g., davis, 1989; davis et al., 1989; malhotra & galletta, 1999; hubona & jones, 2002; ditsa, 2003). obtaining users’ assessment of the number of times they use eis in a week and/or their frequency of using eis is the first measure. the likert scales used for the measurement are adapted from davis (1989) and other prior is research studies (e.g., davis, 1989; mathieson, 1991; moore & benbasat, 1991; adam et al., 1992; davis, 1993; venkatesh & davis 1996; mao, 2002; hubona & jones, 2002; venkatesh et al., 2003). the scales have been adjusted from 7-point to 5-point scales with one being the negative end of the scale and five being the positive end of the scale based on the advice of a statistician and pre-test results. hypotheses and implications of the model triandis (1979) states that habits can be measured based on individual experience and ability in performing a given task. prior research has shown that habits are a strong predictor of behaviour. for instance (sugar, 1967) as referenced in thompson et al., (1991) measured the attitudes, norms and habits of college students regarding cigarette smoking. on separate occasions, the same students were offered a cigarette. the strongest single predictor of behaviour was habit, followed by norms and then attitudes. furthermore, in the eis domain, habits have been operationalised on the basis of eis experience and the ability to use eis (ditsa, 2002, 2003). in their study, dambrot et al., (1988), indicate that subjects ajis vol 13, no. 1 september 2005 89 who failed an assembly language programming course had significantly less computer experience than those who did not fail the course. in his study of text editing, rosson (1983) explains that experience was positively correlated with the number of lines edited per minute. elkerton and williges (1984) indicate in their work that experience explains more variance in information search times than do other individual variables. according to zmud (1979) people’s level of education influences their successful use of computer systems. more educated computer users significantly outperformed less educated people in training environments (davies & davies1990). it has been documented that a higher level of education negatively relates to computer anxiety whilst positively relates to favourable computer attitude (ikart, 2004b). according to lucas (1978), less educated individuals have more negative attitudes in using computer technology than individuals with better education. education is effective in overcoming negative attitudes towards computers (harrison and rainer, 1992). hubona and jones (2002) found in their study of user acceptance of email that length of time since first use and level of education directly influence email usage behaviour. education and length of time in using information technology parallel individual experience and ability to use computers. accordingly, we hypothesised that: h1a: eis experience will have a positive effect on perceived usefulness of eis. h1b: ability to use eis will have a positive effect on perceived usefulness of eis. h2a: eis experience will have a positive effect on perceived ease of use of eis. h2b: ability to use eis will have a positive effect on perceived ease of use of eis. ajis vol 13, no. 1 september 2005 90 figure 3: the research model triandis (1979) argues that behaviour will not occur if objective factors (facilitating conditions) of the geographical environment prevent it. in the information technology domain, facilitating conditions have been operationalised as eis development process, organisational environment and eis management process (ditsa, 2002; 2003). research efforts in eis development (e.g., watson, et al., 1991; rainer & watson, 1995; srivihok, 1999) have sought to understand the factors contributing to cost-effectiveness of eis projects in organisations. findings have linked this research to factors such as general top management support, committed executive sponsor, managing user resistance and expectations, managing system spread and evolution, delivering the first version of the system quickly, employing an evolutional development process, involving users in the development and linking the development to business objectives (watson, et al., 1991; nandhakumar & jones, 1997; mcbride, 1997; srivihok, 1999; poon & wager, 2001; ikart, 2004a, 2004b). in addition, it is linked to employing appropriate technology such as hardware and software, is staff and managing data problems (kelly, 1994; rainer & watson, 1995; ikart, 2004a, 2004b). accordingly, we hypothesised that: h3a: eis development processes will have a positive effect on the perceived h9 h8 h7 h3a h3b, h4a hb h4 h6a, h6b h6c, h5a, h5b h5c, h2a h2b habits: eis experience ability to use eis facilitating conditions: eis development process eis management process organisational environment social factors: subjective norms subjective roles subjective values social situations attitudes towards using actual system use perceived usefulness perceived ease of use h1a h1b ajis vol 13, no. 1 september 2005 91 usefulness of eis. h3b: eis management processes will have a positive effect on the perceived usefulness of eis. h3c: organisational environment will have a positive effect on the perceived usefulness of eis. h4a: eis development processes will have a positive effect on the perceived ease of use of eis. h4b: eis management processes will have a positive effect on the perceived ease of use of eis. h4c: organisational environment will have a positive effect on the perceived ease of use of eis the internalisation of the reference group’s subjective culture and specific interpersonal agreements that the individual has made with others, in specific social situations, triandis (1979) argues, constitutes the social factors that determine behavioural intention. he defines subjective culture as the, “human group characteristic way of viewing the humanmade part of the environment” (p. 208), consisting of ways of categorising experience such as values, norms and roles and social situations. in the information systems research domain, superior, peer and subordinate influences have been strong determinants of subjective norms (mathieson, 1991; taylor & todd, 1995; venkatesh & davies, 1996; elkordy, 2000; ditsa, 2002). mao (2002) investigated the it usage behaviour of 80 endusers in four organisations at two points in time based on the tam model with the subjective norm as a construct of the research model. it was implied that the subjective norms play a more important role in the long run within the group. for instance in the field studies (e.g., lucas and spitler, 1999; karahanna et al, 1999) as referenced in mao (2002) found subjective norm to be a significant determinant of is usage. also, kwon & chidambaram (1999) studied the patterns of cellular phone adoption and usage in an urban setting. they found social pressure to have a positive outcome among professionals as a motivation to adopt and use the systems. igbaria (1993) studied microcomputer usage in organisations and found social norms to have significant effects on system usage. in his studies, rogers (1986), indicates the importance of social norms on the rate of the diffusion of innovation. in the social psychology domain, lieberman (1956) cited in westen (1996) examined how workers’ attitudes change as a result of job promotions. he measured the attitudes of plant workers and then reassessed after some were promoted to foreman (a management position) and some as shop stewards. not surprising after the promotions, the foremen were more pro-company than they had been as workers, whereas the shop stewards had become more pro-union. more interestingly however, when the company later experienced financial problems and had to demote the foremen to their previous rank-and-file positions they returned to their original attitudes (westen, 1996; ikart, 2004a, 2004b). this shows that individual’s attitude has a significant influence on his/her behaviour. triandis (1979) argued that a behaviour setting in social situations has time-place coordinates, physical entities and processes and, it evokes particular behaviours that distinguish it from what is outside of it. it has structural features, and exists independently of any particular individual’s perception of it. triandis sets an example of this as a classroom with a particular location and time where members meet. the class has chairs, walls and black boards and in it members act in certain ways including talking, listening, writing, taking notes and others. according to triandis’ work “social situation in a ajis vol 13, no. 1 september 2005 92 behaviour setting is a bounded, self-regulated and orderly system composed of replaceable human and non-human components that interact in a synchronised fashion in carrying out an orderly sequence of events such as the setting program” (triandis 1979, p. 214). a similar idea is shared in ikart (2004a). in eis studies, (e.g., bergeron, 1995; ditsa, 2002, 2003) social factors are defined as the executive’s work group (peers, superior, subordinates, is directors). in several studies social factors have shown strong influence on behaviour and utilisation of is including eis (thompson et al., 1991; bergeron et al., 1995; ditsa, 2002, 2003; venkatesh et al., 2003). this suggests that the social factors of the conceptual model will relate strongly with behaviour. thus we hypothesised that: h5a: subjective norms will have a positive effect on perceived usefulness. h5b: subjective roles will have a positive effect on perceived usefulness. h5c: subjective values will have a positive effect on perceived usefulness. h5d: subjective social situation will have a positive effect on perceived usefulness. h6a: subjective norms will have a positive effect on perceived ease of use. h6b: subjective roles will have a positive effect on perceived ease of use. h6c: subjective values will have a positive effect on perceived ease of use. h6d: subjective social situation will have a positive effect on perceived ease of use. research indicates that voluntary computer usage is driven to a large extent by pu (davis, et al., 1992). robey (1979) studied industrial sales forces and observed that users’ expected performance impacts of a computerised sales record-keeping system were positively correlated with the measure of actual use of the system. other mis studies, (e.g., davis, 1989; mathieson, 1991; davis et al., 1992; adams et al., 1992; davis, 1993; taylor & todd, 1995; venkatesh & davis, 1996; dishaw, & strong, 1997; kwon & chidambaram, 2000; elkordy, 2000; mao, 2002; ditsa, 2002, 2003) have shown that pu and peou are strong determinants of user acceptance and adoption of computer technology. research on the adoption of innovation has suggested a relevant role for pu and peou. according to rogers (1986), relative advantage and compatibility are important attributes of innovations affecting adoption. rogers (1986) suggested a number of sub-dimensions of relative advantage including the degree of economic profitability, decrease in discomfort and saving in time. also, davis (1989) argued that compatibility, relative advantage and complexity have the most consistent significant relationships across a broad range of innovation types. pu parallels relative advantage (mao, 2002) and peou parallels compatibility and complexity of innovations (davis, 1989). this finding is consistent with the finding of tornatzky and klein (1982) in their meta-analysis of innovation diffusion literature. tornatzky and klein reviewed 75 articles and discovered more than 30 innovation characteristics. they investigated ten major innovation characteristics including: compatibility, relative advantage, complexity, communicability, cost, divisibility, profitability, social approval, observability and trial ability and found compatibility, relative advantage and complexity to strongly measure innovation attributes affecting innovation diffusion. thus, we hypothesised that: h7: perceived usefulness will have a positive effect on the attitudes towards using eis. h8: perceived ease of use will have a positive effect on attitudes towards using eis. ajis vol 13, no. 1 september 2005 93 fishbein (1979, p. 68) defined attitude as, “a function of beliefs”. fishbein and ajzen (1975) distinguished between beliefs and attitudes and specified how external stimuli like the objective feature of attitude object such as individuals, situation and social groups can be causally linked to beliefs, attitudes and behaviour. according to fishbein and ajzen (1975), individual beliefs about the behaviour, also known as the perceived consequence of the behaviour, refers to an individual’s subjective likelihood that performing the behaviour will lead to certain outcomes (p.233). on the other hand, attitudes toward the behaviours are an effective evaluation of the behaviour. attitude towards the behaviour is determined by an expectancy-value model of beliefs (fishbein, 1979, p.68) weighted by the evaluations of the consequences (fishbein and ajzen, 1975, p.233). in their study, they draw a distinction between two separate attitude constructs such as attitude towards the object and attitude towards the behaviour. the former refers to an individual’s effective evaluation of a specified attitude object while the latter refers to an individual’s evaluation of a specified behaviour that involves the object (fishbein & ajzen, 1975; davis, 1993). this paper employs attitude towards the behaviour because research studies (e.g., ajzen & fishbein, 1975; davis, 1993; robey, 1979; dishaw & strong, 1997; galletta & malhotra, 1999; srivihok, 1999; mao, 2002; hubona & jones, 2002; lim, 2002; venkatesh et al., 2003) have shown that attitudes toward the behaviour relate more strongly to a specified behaviour. we therefore hypothesised that: h9: executives’ attitudes towards using eis will have a positive effect on actual use of eis. research methodology data collection methods the data for the pilot study was collected by mail survey from two large organisations in the illawarra region of australia identified as using eis. the main concern of the pilot was to further pre-test the questionnaire as well as serve as an exploratory study for the main study to validate the research model. questionnaire design a six-page questionnaire was designed for the pilot. each question represented a component of the research model. the questions were selected for their theoretical importance and potential relevance to practice (ikart, 2004a). the statistical consultation service in the university of wollongong to verify the statical validity of the research model, hypotheses and questionnaire. the questionnaire was pre-tested on four academics and based on the feedback received from the representatives, some modifications were made to the individual questions and instructions. the revised questionnaire was then subjected to the next phase of pre-testing with academics and four doctoral students drawn from the faculty of commerce at the university of wollongong where the research is taking place. all comments and suggestions given by them regarding the clarity, validity and consistency of the questions were incorporated into the survey instrument. a cover letter explaining the purpose of the survey was designed. it was mailed with a prepaid envelope to the participants for the pilot study. the cover letter has a statement ajis vol 13, no. 1 september 2005 94 guaranteeing the confidentiality of the respondents and a statement of how the research has been reviewed by the human research ethics committee (hrec) as required in australia. although there is no widely agreed sample size for a pilot survey, between 12 and 30 subjects is generally recommended (e.g., hunt et al., 1982). the pilot survey was carried out on a representative sample of 30 executives who actually use eis. after a follow-up by telephone calls 19 returns were received giving a response rate of 63 percent. the returned questionnaires were carefully examined for signs that respondents had difficulty in understanding the questions. all 19 returned questionnaires were good for analysis. there were some suggestions and comments from respondents, which were noted to improve the questions for the main study data analysis and preliminary results due to the size of the data for pilot study, a qualitative analysis of the data was carried out. among the 19 participants for the pilot, in the gender category, the results show that 73% (14) were males while 26.3% (5) were females. in the age category, the modal group from the results was 46 – 55 with 47.36% (9) followed by 36 – 45 group with 36.84% (7) participation rate. although there was no participant from the over 55 years group, participation rates from 18 – 25 and 26 – 35 groups were the lowest participation rates, with 5.26% (1) and 10.53% (2) respectively. at the educational level, the results show that the majority of eis users are highly educated. as shown in table 2, while 52.26% (10) of the participants had a postgraduate degree, approximately 37% (7) of the participants held an undergraduate degree. but only 5.26% (1) of the participant held high school and tafe qualifications respectively. in terms of job position, the results show that while the majority, 47.36% (9) of the eis users are middle management position holders, 26.3% (5) are top management position holders. the finding that the majority of eis users at organisational levels are middle managers confirms the finding of the study by fitzgerald (1998) study in the united kingdom, which also concluded that the majority of eis users are middle managers rather than the top-level managers, which is the team an eis is meant to support. ajis vol 13, no. 1 september 2005 95 demographic factors the valid items frequency percent % gender female 5 26.3 male 14 73.68 age 18 – 25 1 5.26 26 – 35 2 10.53 36 – 45 7 36.84 46 – 55 9 47.36 over 55 0 0 level of education high school 1 5.26 tafe 1 5.26 undergraduate 7 36.84 postgraduate 10 52.26 job position top management 5 26.3 middle management 9 47.36 lower management 4 21.02 others 1 5.26 table 2 demographic factors of respondents in the pilot (n=19) the preliminary results suggest that first, executives’ experience in eis positively relates to their experience in computer-based information systems (cbis). second, the result suggests that although some managers have significant knowledge in eis due to length of use, executives who have been using eis applications for a greater length of time have greater knowledge of the systems. third, the results suggest a high degree of perceived usefulness, perceived ease of use as well as a positive attitude towards using eis by executives. this may be due to the value outcome, user-friendliness and clarity of the systems to users. furthermore, the results suggest that executives consider social factors in using eis in their work. finally, the results suggest facilitating conditions variables such as eis development process, eis management process and organisational environment are strongly related to the adoption and use of eis by executives. conclusions and future direction the preliminary results of this study suggest that cultural, social, individual and organisational variables are of vital importance in explaining executives’ behaviour towards the adoption and use of eis at the strategic management level. this preliminary result provides future direction in explaining executives’ behaviour towards eis adoption and usage. perhaps, future findings together with this result will assist system designers to understand the core information processing requirements for executives’ tasks for which ajis vol 13, no. 1 september 2005 96 they are building eis in order to implement appropriate system functionalities to support those tasks. in addition, the model is important in the is research domain with regards to user behaviour. limitations there are some limitations associated with the investigation techniques in the pilot. first, the sample size was relatively small, with just 19 participants. this substantially limits the analysis to qualitative analysis and presents some potential for unrepresentative results. moreover, although there was diversity presented by the respondents of organisations used for the pilot, the pilot study was only limited to two organisations. work in progress and future directions at the time of submitting this paper, 400 questionnaires have been prepared and distributed for the main survey. the questionnaires were mailed out to mainly executives such as cfos, ceos, cios and other top-level managers of 200 organisations using eis in australia. the questionnaires were mailed out in four batches within 2 weeks. the date of each mailed out batch has been recorded. further, all mailed out questionnaires have been pre-numbered, in order to facilitate the follow-up process of non-responses. in addition, a table to lock in all returns, comments and suggestions to be given by respondents has been developed. the follow-up of non-responses for each batch of questionnaires mailed out is planned to commence three weeks after the initial mail out. although the questionnaire in the planned follow-up will be exactly the same as the initial ones, a reminder letter will be attached. moreover, in order to encourage the sample non-respondents to complete and return the follow-up survey, the construction of the reminder letter will be business-like and will place much emphasis on the importance of the survey. references: adams, d., nelson, r. & todd, p. (1992) “perceived usefulness, ease of use, and usage of information technology: a replication” mis quarterly/june16, 227-247. ajzen, i. & fishbein, m. (1980) understanding attitudes and predicting social behaviour. englewood cliffs, nj: prentice-hall. bergeron, f., raymond, l., rivard, s. and gara m. (1995) “determinants of eis use: testing a behavioural model”, decision support systems, 14, 131-146. basu, c., poindexter, s. and addo, t. (2000) “diffusion of executive information systems in organisations and the shift to web technologies”, industrial management & data systems. mcb university press (100) 6, 271-276. anthony, r.n., dearden,j. and govindarajan, v. (1992) management control systems: 7 th ed. erwin. dambrot, f. h.,silling s. m. & zook s. (1988) “psychology of computer use: sex differences in prediction of course grades in a computer language course”, perceptual and motor skills, 66, 627 – 636. davies, l. d. & davies f.d. (1990). “the effects of training techniques 7 personal characteristics on training end users of information systems,”. journal of management information system, 7, 93-110. davis, f.d. (1989) “perceived usefulness, perceived ease of use, and user acceptance of ajis vol 13, no. 1 september 2005 97 information technology”, mis quarterly (13:3), 319-339. davis, f.d. (2001) “perceived usefulness, perceived ease of use, and user acceptance of information technology”, mis quarterly / september (319 –339). davis, f.d., (1986) “a technology acceptance model for empirically testing new enduser information systems: theory and results”, in mit sloan school of management. cambridge, ma: mit sloan school of management. davis, f. d. (1993) “user acceptance of information system technology: system characteristics, user perceptions and behavioural impacts”, international journal of man-machine studies, academic press limited, 38, 475 – 487. davis, f. d., bagazzi, r. p. & warshaw, p. r. (1992) “extrinsic and intrinsic motivation to use computers in the workplace”, journal of applied social psychology, (14) 22, 11111132. davis, f.d., bagozzi, r. p. and warsaw, p.r. (1989) “ user acceptance of computer technology: a comparison of two”, management science (35:8) 982-1003 delong, d.w & rockart, j.f. (1992) executive information systems: emergence. development. impact: john wiley & son, inc.,(257-277). dishaw m. & strong d.(1997). extending technology acceptance model. retrieved 27 th september, 2003 from://hsb.baylor.edu/ramsower/ais.ac.97/papers/dishaw.htm ditsa, g. (2002) executive information systems use in organisation contexts: an exploratory study in issues and trends of it management in contemporary organisations, idea group publishing london, 182-191. ditsa, g. (2003). executive information systems use in organisational contexts: an exploratory user behaviour testing. information management: support systems & multimedia technology irm press london (109-155). elkerton, j. and williges, r. c. (1984) “information retrieval strategies in a file-search environment”, human factors, 26, 171-184. elkordy, m.m. (2000) “an integrated model of eis use”, in proceedings of 2000 information resources management association, international conference anchorage, alaska, usa, 624-627. fishbein, m. (1979) “a theory of reason action: some applications and implications”, in nebraska symposium on motivation, h. howe and page (edn). university of nebraska press, lincoln, nb. 65-116. fishbein, m. and ajzen, i.(1975) belief, attitude, intention and behaviour: an introduction to theory and research: addison-wesley, reading, ma. fitzgerald, b. & murphy, c. (1994) “the eis paradox: putting the executive into executive information system”, journal of information technology vol. 9 no.4, 288-296. fitzgerald, b. (1998) “executive information systems without executives”, proceedings of the 3 rd conference of uk academy for information systems. lincoln university, uk mcgrawhill (288-310), in avison. d. & edgar nevill d. (eds.) matching technology with organisational needs. gefen, d. & straub, d. (2000) “the relative importance of perceived ease of use in is adoption: a study of e-commerce adoption”, journal of the association for the information system 1(8). hubona, g. s. & jones, a. b (2002) “modelling the user acceptance of e-mail”, proceedings of the 36 th hawaii international conference on system sciences (hiss’03) handzic, m. (1997) “the impact of information reliability on utilisation and effectiveness of executive information systems”, 8 th australasian conference on information systems harrison, a.w. & rainer jr. r.k. (1992) “the influence of individual differences on skills in ajis vol 13, no. 1 september 2005 98 end user computing”, journal of management information systems, 9, 93-111. houdeshel, g. & watson, h.j. (1992) executive information systems: emergence. development. impact: john wiley & son, inc.,(13-31). hunt, s. d., sparkman, r. d., and wilcox, j. b. (1982) “the pretest in survey research: issues and preliminary findings”, journal of marketing research, 19 (may), 269-273. igbaria, m. (1993) “user acceptance of microcomputer technology: an empirical test”, omega, (21), 1, 73-90. ikart, e.m. (2004) “factors influencing executive information systems adoption and usage by organisational executives”, conference on information sciences and technology management, july 8-9 2004, ancient library of alexandria, egypt. ikart, e.m. (2004) “a research model for the investigation of top management adoption and use of executive information systems”, proceedings of the 2004 international business information management conference july 4 – 6 amman, jordan, pp.169-181. kelly, f. (1994) “implementing an eis (executive information system)”, retrieved 23 rd september2003 from: www.itmweb.com/essay519.htm . kling, r. (1991) “excerpts from ‘social analysis of computing: theoretical perspectives in recent empirical research”, in dunlop, c. & kling eds/ (1991) computerization and controversy: value conflicts and social choices boston: academic press, part11, 150166. kotter, j.p.(1982). the general manager: the free press, new york. kraemer, k. l, danziger, j. n, dunkle, d. e & king j . k., (1993) “the usefulness of computer-based information systems to public managers”, mis quarterly june, 129-148. kwon, h. s. and chidambaram, l. (2000) “a test of technology acceptance model: the case of cellular telephone adoption”, proceedings of the 33 rd hawaii international conference on system sciences, ieee. leidner, d. & elam, j. (1994) “senior and middle management use of eis: a descriptive study”, proceeding of the twenty-seventh annual hawaii international conference on system sciences,135-144. liang, l. y. and miranda, r. (2001) dashboards: executive information systems for the public sector: government officers association usa. lim, k. s. (2002) “validation of the technology acceptance model with academic users”, decision sciences institute annual meeting proceedings, 14021407. lucas, h. c. (1978) “empirical evidence for a description model of implementation”, mis quarterly, 2, 27-52. malhotra, y., and galletta, d. (1999) “extending the technology acceptance model to account for social influence: theoretical bases and empirical validation”, proceeding of the 32 nd hawaii international conference on system sciences ieee. mao, e. (2002) “an investigation of it usage over time”, eight americas conference on information systems, pp.1307-1312 mathieson, k. (1991) “ predicting user intentions: comparing the technology acceptance model with the theory of planned behaviour”, information system research, 2 (3), 173191. mcbride, n. (1997) “the rise and fall of an executive information system: a case study”, information systems journal (1997) 7, 277287. mintzberg, h. (1973) the nature of managerial work: harper & row publishers london. money, w. & turner, a. (2004) “application of technology acceptance model to a knowledge management systems”, proceeding of the 37 th hawaii international conference on system science. ieee. ajis vol 13, no. 1 september 2005 99 moore, g. c & benbasat, i. (1991) “development of an instrument to measure the perceptions of adopting an information technology innovation”, information system research, 2, (192-222). nandhakumar, j. & jones m. (1997) “designing in the dark: the changing user-developer relationship in information systems development”, proceedings of the eighteenth international conference on information systems, atlanta georgia, 75-89. nord j. and nord g., (1995) “executive information system: a study and comparative analysis”, information & management 29, 95 – 106 elsevier science b.v. poon, p. & wagner, c. (2001) “critical success factors revisited: success and failure cases of information systems for senior executives”, decision support systems 30, 393-418 elsevier science b.v. rainer, jr. r. & watson, h.,(1995) “what does it takes for successful executive information systems?”, decision support systems (14), elsevier science b.v., 147-156. raisinghani, m. s. & schkade l. (1997) “strategic decision making: a framework for multicriteria decision analysis of technology investigation and a field survey”, retrieved 27 th october 2003 from://hsb.baylor.edu/ramsower/ais.ac.97/papers/raising2.thm robey, d. (1979) “user attitude and management information system use”, academy of management journal, 22(3), 527-538. rockart j.f. & treacy m.e. (1992) executive information systems: emergence. development. impact: john wiley & son, inc.,(3-12). rockart j. f. & delong d. w. (1992) executive information systems: emergence. development. impact: john wiley & son, inc., (315-325). rogers, e.m. (1986) communication technology – the new media in society. new york, ny: the free press rosson, m.b. (1983) “patterns of experience in text editing”, proceedings of the chi’83 human factors in computing systems,171-175. salmeron, j. l (2002) “eis evolution in large spanish businesses”, information & management, 40, 41-50, elsevier science b.v., seeley, m. & targett d., (1999) “patterns of senior executives’ personal use of computer”, information management (35), 315-330, elsevier science b.v. singh s. k, watson h. j. & watson r. t, ( 2002) “eis support for strategic management process”, decision support systems (33), 71-85. elsevier science b.v. stein a., & nasib z., (1997) “executive computing a benefit matrix”, retrieved 9 th september 2001 from ://hsb.baylor.edu.ramsower/ais.ac.97/papers/stein.htm. srivihok, a. (1999). “understanding executive information systems implementation: an empirical study of eis success factors”, proceedings of the 32th hawaii international conference on system sciences, ieee. szajna, b. (1993) “determining information system usage: some issues and examples”, information and management (25), 147-154. elsevier science b.v. taylor, s. and todd, p.a (1995) “understanding information technology usage: a test of competitive models”, information system research, 6(2)144-176 thierauf r., (1991). executive information systems: a guide for senior management and mis professionals: quorum book new york. thodenius b., (1995) “the use of executive information systems in sweden”, cems academic conferencerecent development in economics & business administration” at the wirtschaftsuniversitat wien, austria, april 20-22. thompson, r.l. higgins, c.a and howell, j.m, (1991) “personal computing towards a conceptual model of utilisation”, mis quarterly, 15(1) 125-143 ajis vol 13, no. 1 september 2005 100 triandis h.c. (1979). values, attitudes, and interpersonal behaviour. nebraska symposium on motivation: beliefs, attitude, and values: university of nabraska press, 195-259. triandis h.c., (1971) attitude and attitude change: john wiley & son, inc. new york usa. turban e. (1993). decision support expert systems: management support systems: macmillan publishing company, new york. venkatesh v., morris m., davis g.b. & davis f.d. (2003) “user acceptance of information technology: toward a unified view”, mis quarterly vol.27 no.3, pp. 425-478. venkatesh, v. and davies f.d. (1996) “a model of antecedents of perceived ease of use: development and test”, decision sciences, 27(3), 451-482. verenikina i. & gould e. (1997) “activity theory as a framework for interface design”,. ascilite december 7-10’97. vlahos g.e., ferratt t. w. & knoepfle g., (2000) “use and perceived value of computerbased information systems in supporting the decision making of german managers”, sigcpr evanston illinois usa, acm, (111-123). watson, h. j., rainer jr. r.k. & koh c. e., (1991) “ executive information systems: a framework for development and survey of current practices”, mis quarterly, march, 13-30. westen, d. (1996) psychology mind, brain, & culture. john weiley & son, inc. new york usa. wheeler, f. p. (1996). “the potential for executive information systems to support the management of business reconfiguration”, retrieved 20 th september 2002 from: //www.shu.ac.uk/schools/fsl/fisjnl/vol1996/pprs1996/p96003.htm wilt f., (1991) top officers don’t use executive info systems in national underwriter life & health/financial services. yager s. e. (1997) “the role of information technology support mechanisms in coordination management for virtual organisations”, september 28 th 2003 from://hsh.baylor.edu/ramsower/ais.ac.97/papers/yager.htm young d. & watson h. j., (1995) “determinates of eis acceptance”,. information & management 29, (153-164). elsevier science b.v. zmud, r. w.(1979) “individual differences & mis success: a review of the empirical literature,” management science, 25, 966-979. microsoft word ajis vol 15-1 m.doc australasian journal of information systems volume 15 number 1 december 2007 137 explaining intention to use an information technology innovation: an empirical comparison of the perceived characteristics of innovating and technology acceptance models sam jebeile macquarie university, nsw, australia email: sam.jebeile@mq.edu.au robert reeve macquarie university, nsw, australia email: r.reeve @mq.edu.au abstract this study examines the issue of technology acceptance in a multi-campus secondary college in sydney, australia. seventy-five teachers across two campuses were surveyed regarding technology acceptance. regression analysis was used to compare the explanatory power of the perceived characteristics of innovating model (pcim), and the technology acceptance model (tam). both models explained a substantial amount of variation in technology acceptance. however, our findings indicate that it is preferable to use the pcim, rather than the tam, to explain intention to use an information technology innovation. implications for both future research and practice are discussed. introduction over the last two decades considerable research has been conducted to provide insight into factors influencing adoption and use of new information technology (davis 1989, 1993; davis, bagozzi, & warshaw 1989; adams, nelson & todd 1992, taylor & todd 1995a, 1995b; warshaw & davis 1985; venkatesh 1999; venkatesh & davis 1996; venkatesh & morris 2000). interest in understanding the factors that explain why information technology innovations are adopted and diffuse in organizations remains high as reflected by recent publications on this topic in this journal (van akkeren & cavaye 1999; jones, hecker & holland 2002; carroll, howard, peck & murphy 2003; seyal & rahman 2003) and other major journals in the discipline (lee, kozar & larsen 2003; teo, wei & benbasat 2003; venkatesh, morris, davis & davis 2003). two models that look at adoption and have resulted in instruments to measure relevant factors at the individual level are the technology acceptance model (tam) developed by davis (1989) and the perceived australasian journal of information systems volume 15 number 1 december 2007 138 characteristics of innovating model (pcim) developed by moore & benbasat (1991). the tam contains only two explanatory factors while the pcim contains these two factors and five additional factors. it is a question of both academic and practical importance whether the more elaborate pcim performs better than the more parsimonious tam in explaining adoption of new information technology. the purpose of this paper is to present the results of an empirical comparison of these models ability to explain intention to use an information technology innovation. a brief review of the development of these two models and their instruments is also provided. the technology acceptance model (tam) the technology acceptance model (tam) was developed by davis (1989). he drew on research from a wide variety of fields including diffusion of innovations, marketing, human-computer interaction and self-efficacy theory to investigate the causes underlying user adoption of information technology. according to davis (1989, p.323) there existed a striking convergence among the wide range of theoretical perspectives supportive of a conceptual and empirical distinction between usefulness and ease of use. as a result, he focused on providing improved measures of perceived usefulness and perceived ease of use. davis (1989, p.320) defined usefulness as “the degree to which a person believes that using a particular system would enhance his or her job performance” and ease of use as “the degree to which a person believes that using a particular system would be free from effort” (p.320). davis developed, refined, and streamlined the new measures in a multi-step process. the first step involved developing explicit definitions of the constructs following a theoretical analysis from a variety of perspectives regarding why usefulness and ease of use are hypothesized determinants of system use. based on this analysis, 14 items were developed for each construct. to enhance content validity he had an interdisciplinary panel of experts categorize the items resulting in the retention of 10 items for each construct. these two 10-item measures were then tested for validity and reliability in a field study of 112 users and two systems (electronic mail and file editor). cronbach’s (1951) alpha measure of internal consistency reliability for the two systems were 0.97 for perceived usefulness and 0.91 for perceived ease of use. multitrait-multimethod analyses supported convergent and discriminant validity. factor analysis supported the notion that each scale was measuring a unidimensional construct. the results of the field study indicated the need for further streamlining of the scales resulting in the deletion of four items from each of the scales. the resulting 6-item scales were then subjected to further construct validation through a laboratory test involving 40 participants using two graphics applications (chartmaster and pendraw). half of the participants worked through a series of tasks in pendraw followed by another series in chartmaster. the remaining participants completed the same tasks in reverse order. following the tasks participants completed the 6-item scales and selfpredicted future use of the applications. cronbach’s alpha reliability for perceived usefulness was 0.98 and 0.94 for perceived ease of use. convergent, discriminant, and factorial validity of the two 6-item scales were again supported, demonstrating that the “new scales exhibited excellent psychometric characteristics” (davis 1989, p.333). details of the items in the two 6-item scales are provided in appendix a. according to davis (1989) one of the most significant findings in his study was the relative strength of the usefulness-usage relationship compared to the ease of use-usage relationship. “in both australasian journal of information systems volume 15 number 1 december 2007 139 studies, usefulness was significantly more strongly linked to usage than was ease of use. examining the joint direct effect of the two variables on use in regression analyses, this difference was even more pronounced: the usefulness-usage relationship remained large, while the ease of use-usage relationship was diminished substantially” (p.333). davis (1989 p.334) further stresses that this finding has important implications for designers who have tended to over emphasize ease of use and overlook usefulness. davis’ tam instrument was replicated and validated by adams, nelson & todd (1992) and has subsequently been used, in its original or modified forms, in numerous studies in a range of contexts. an extensive meta-analysis of tam studies was undertaken by lee et al. (2003). the perceived characteristics of innovating model (pcim) the perceived characteristics of innovating model (pcim) was developed by moore and benbasat (1991). it is based on the diffusion on innovation (doi) theory developed by rogers (1983, 1995) and incorporates the constructs from the tam (davis 1989). rogers (1983, 1995) defined five attributes or characteristics of innovations which influence an individual’s attitude towards an innovation during the adoption process. relative advantage is the degree to which an innovation is perceived as better than its precursor. compatibility is the degree to which an innovation is perceived as being consistent with the existing values, needs, and past experiences of potential adopters. complexity is the degree to which an innovation is perceived as being difficult to use. trialability is the degree to which an innovation may be experimented with before adoption. observability is the degree to which the results of an innovation are observable to others (moore & benbasat 1991, p.195). moore and benbasat (1991) developed an instrument to measure an individual’s perceptions concerning these attributes of an innovation. they renamed rogers’ complexity construct as ease of use, consistent with davis (1989). they also developed the image construct which was defined as “the degree to which use of an innovation is perceived to enhance one’s image or status in one’s social system” (moore & benbasat 1991, p.195). according to moore and benbasat (1991), rogers included the essence of the image construct in his definition of relative advantage. however, research indicating that it was separate from relative advantage was strong enough for moore and benbasat (1991) to decide to measure it as a separate construct. also, as a result of their instrument development process, moore and benbasat (1991) found that observability construct separated into two constructs: result demonstrability and visibility. result demonstrability “concentrated on the tangibility of using the innovation, including their observability and communicability” (moore & benbasat 1991, p.203). visibility, on the other hand, focused on the physical presence of the innovation in the organizational setting. thus, the constructs measured by the moore and benbasat (1991) instrument were relative advantage, compatibility, image, ease of use, result demonstrability, visibility, and trialability. it is important to note that moore and benbasat (1991)’s model, and related instrument, is based on user perceptions about using the innovation (i.e. innovating), which differs from rogers’ (1983, 1995) model, which focuses on the user perceptions of the innovation itself. according to moore and benbasat (1991): innovations diffuse because of the cumulative decisions of individuals to adopt them. thus, it is not the potential adopters’ perceptions of the innovation itself, but rather australasian journal of information systems volume 15 number 1 december 2007 140 their perceptions of using the innovation that are the key to whether the innovation diffuses (p.196). the instrument was developed in three stages. during stage one, the items were created following an extensive literature review examining previously developed scales including the 12 items used by davis (1989) to measure ease of use and usefulness. stage two focused on establishing construct validity and involved four rounds of sorting by four different groups of expert judges. once the construct validity had been established the pcim instrument was tested using two pilot tests and a final field study. the result was a 38-item instrument comprising eight scales, which showed very good psychometric characteristics. the one area of concern was that relative advantage and compatibility emerged as a single factor rather than as separate factors as had been expected (moore & benbasat 1991, p. 208). moore and benbasat (1991) also developed a short form of their instrument consisting of 25 items by deleting 13 items which, for their sample, did not significantly reduce the cronbach’s (1951) alpha for any scale. this short form of the instrument was used in this study. details of the items in the eight multi-item scales in the short form instrument are provided in appendix b. moore and benbasat (1991)’s pcim instrument has also been used in numerous studies demonstrating reliability and validity in a range of contexts (moore & benbasat 1996; miller, kelly, & harper 1997; karahanna, straub & chervany 1999). method: an empirical comparison of the tam and pcim the empirical component of this research involved gathering and analyzing data to compare the ability of the technology acceptance model (tam) and the perceived characteristics of innovating model (pcim) to explain intention to use an information technology innovation. in this study the innovation chosen is the use of the internet/web for teaching purposes. as the pcim incorporates the tam’s constructs the empirical comparison will mainly investigate whether the additional constructs included in the pcim significantly improve its explanatory power. the incorporation of the tam’s constructs within the pcim also means that the necessary data to make a meaningful comparison of the two models can be obtained by administering the short form of moore and benbasat’s (1991) pcim instrument. instrument administration a questionnaire completed in a controlled environment was used to administer the instrument. the sample surveyed consisted of teaching staff in a multi-campus nsw secondary college, who use the internet/web or who might intend to use the internet/web in future for teaching purposes. the unit of analysis in this study is the individual teacher. previous studies of this kind (for instance davis 1989, adams et al. 1992;) indicated that the lowest correlation scores obtained using similar instruments are of the order of 0.30. borg and gall (1983) estimated that, for a correlation of 0.30 to be statistically significant at the 0.01 level, at least 47 participants are required. therefore, after allowing for potential absentees and those unwilling to participate in this study, we required the participation of at least two campuses (40 teachers are employed at each campus). the total number of teaching staff in attendance at the participating campuses on the day that the survey was administered was 75. all questionnaires were completed and useable. this satisfied our sample size requirements, and eliminated concerns relating to non response bias. australasian journal of information systems volume 15 number 1 december 2007 141 all of the teachers that participated in the study indicated prior usage of the internet/web. further, all of the participating teachers had attended an in-service relating to the use of the internet/web in teaching at the beginning of the school year approximately six months prior to the date that the survey was administered. demographic data for each participant was requested in the questionnaire including data on age, gender, education level, and main teaching area. the sample was balanced in terms of gender (51% female and 49% male). in the sample 80% had completed four years of university education; 5% had completed three years education at either university or teachers college; while 15% had completed five years of university education. there was a fairly even spread of ages amongst the six age categories used; 17% of the sample were less than 29 years of age; 15% were between 30 and 34; 16% were between 35 and 39; 17% were between 40 and 44; 27% were between 45 and 49; and 8% were over the age of 50. all nine of the key learning areas in nsw secondary schooling with 7% from creative arts; 16% from mathematics; 11% from science; 16% from english; 5% from health studies; 17% from human society and its environment; 4% from languages other than english; 3% from vocational education; and 17% from technology and science. all teacher participants were volunteers and completed the questionnaire during staff meetings, with the cooperation of the campus principals. prior to conducting the research, research ethics approval was obtained from the nsw department of education strategic research directorate, and the macquarie university ethics review committee (human research). prior to the administration of the survey teacher participants were given several minutes to read the covering letter explaining the nature and purpose of the research. this was followed by a short introduction to the research by one of the researchers who was available throughout the process for any questions or problems regarding the survey. when all teachers had completed the survey they were asked to seal the survey inside the envelope provided. once this procedure was completed the researcher collected the envelopes, matching the number of envelopes collected with a head-count of teachers taken earlier. data analysis measurement of variables the independent variables were measured using the short form of the moore and benbasat (1991) instrument which was modified slightly for use in this study, based on discussions with the schools’ principals. the voluntariness construct was excluded because at this stage use of the web by teachers for teaching purposes was voluntary. the ease of use construct was reduced from 4 items to 2 items. the image construct was reduced from 3 items to 2 items and the results demonstrability construct was reduced from 4 items to 2 items. the reductions in the number of items were to reduce repetitiveness, or seeming lack of relevance to use of the internet/web for teaching purposes. the exclusion of these items resulted in the reduction of number of items from 25 to 18. the items used are shown in appendix c and cross-references to the tam and pcim are shown in appendices a and b. descriptive statistics for the variables from this sample are provided in table 1 below. note that because the pcim incorporates the tam the values for ease of use in the pcim are the same as the values for perceived ease of use in the tam (they are based on the same three questions) and the values for relative advantage in the pcim are very similar to the values for perceived usefulness in the tam (they share four questions). australasian journal of information systems volume 15 number 1 december 2007 142 the dependent variable, intention to use the internet/web for teaching, was measured using a single question which appears as item 19 in appendix c. using the internet/web for teaching may involve perceptions about several aspects of this activity including the use of the internet and the content available on the internet. in this study the specific features of using the internet for teaching are not examined individually. empirical research has shown that questions about behavioural expectations provide an effective means of estimating future behaviour (davis 1989; warshaw & davis, 1985). variable app c item nos n min max mean sd independent variables: pcim relative advantage 1,2,3,4,5 75 1 7 4.33 1.63 compatibility 6,7,8 75 1 7 4.12 1.65 image 9,10 75 1 7 3.14 1.76 visibility 11,12 75 1 7 3.68 1.65 ease of use 13,14 75 1 7 4.37 1.78 result demonstrability 15,16 75 1 7 4.74 1.74 trialability 17,18 75 1 7 5.00 1.61 independent variables: tam perceived usefulness 1,2,3,4 75 1 7 4.40 1.63 perceived ease of use 13,14 75 1 7 4.37 1.78 dependent variable: intention to use the internet/web for teaching 19 75 2 7 5.35 1.59 table 1: descriptive statistics analysis of validity and reliability factorial validity of the two tam scales was confirmed by factor analyzing (principal components analysis with varimax rotation) the six items making up these two scales. inspection of the results in table 2 below show that the six items loaded cleanly on to two factors as expected. perceived usefulness was by far the more important factor, with an eigenvalue of 4.8 and accounting for over 80% of the variation in the data set. the perceived ease of use factor accounted for less than 10% of the variation and had an eigenvalue of less than one. australasian journal of information systems volume 15 number 1 december 2007 143 scale item no f1 f2 perceived usefulness 1 .749 .536 2 .874 .364 3 .849 .385 4 .889 .363 perceived ease of use 13 .477 .831 14 .336 .916 % of variance 80.5 9.6 eigenvalue 4.8 .57 table 2. factor analysis of the tam scales (principal component analysis with varimax rotation) factorial validity of the seven pcim scales (excluding voluntariness) was confirmed by factor analyzing (principal components analysis with varimax rotation) the 18 items making up these seven scales. inspection of the results in table 3 below show that the 18 items loaded cleanly on to six factors. this was similar to the results in the original moore and benbasat (1991) study where relative advantage and compatibility also loaded on to a single factor. the combined relative advantage and compatibility factor was the most important, with an eigenvalue of 10.5 and accounting for 58.7% of the variation in the dataset. image, trialability, and ease of use were all relatively important factors, having eigenvalues greater than one, whereas visibility and result demonstrability had eigenvalues of less than one. scale item no f1 f2 f3 f4 f5 f6 f7 relative advantage 1 .766 .174 .246 .370 .065 .105 .189 2 .621 .244 .186 .305 .181 .144 .558 3 .827 .097 .168 .231 .192 .037 .260 4 .669 .156 .201 .269 .248 .175 .517 5 .787 .185 .202 .236 .068 .211 .281 compatibility 6 .814 -.002 .188 .267 .193 .092 -.038 7 .899 .168 .190 .150 .124 .192 -.051 8 .847 .207 .216 .134 .082 .224 .060 image 9 .144 .946 .080 .015 .171 .066 .076 10 .154 .966 .039 .005 .073 .026 .031 visibility 11 .282 .429 -.016 .020 .673 .316 .260 12 .152 .100 .116 .103 .953 .021 .009 ease of use 13 .531 .019 .178 .733 .146 .146 .181 14 .472 -.036 .253 .773 .044 .210 .065 result demonstrability 15 .432 .081 .258 .405 .190 .670 .082 16 .551 .178 .385 .232 .180 .535 .182 trialability 17 .248 .064 .886 .116 .077 .012 .180 18 .303 .054 .815 .201 .059 .242 -.070 % of variance 58.7 11.7 6.1 5.8 3.4 2.7 2.2 eigenvalue 10.5 2.1 1.1 1.0 .62 .48 .40 table 3. factor analysis of the pcim scales (principal component analysis with varimax rotation) australasian journal of information systems volume 15 number 1 december 2007 144 internal consistency reliability of the two tam scales and seven pcim scales was analyzed using cronbach’s (1951) alpha. inspection of the results in table 4 below show the alpha values for all scales to be well above nunnally’s (1978) benchmark of .70 and comparable with the values in the moore and benbasat (1991) and davis (1989) studies. m & b 1991 this study pcim scales items alpha items alpha relative advantage 5 .90 5 .96 compatibility 3 .86 3 .93 ease of use 4 .84 2 .92 result demonstrability 4 .79 2 .89 image 3 .79 2 .96 visibility 2 .83 2 .81 trialability 2 .71 2 .83 davis 1989 (study 2) this study tam scales items alpha items alpha perceived usefulness 6 .98 4 .95 perceived ease of use 6 .94 2 .92 table 4. internal consistency reliability of the multi-item scales results regression analyses the empirical comparison of the ability of the tam and pcim to explain intention to use an information technology innovation was undertaken using three stepwise multiple regression analyses, each with intention to use the internet/web for teaching as the dependent variable. the tam variables, perceived usefulness and perceived ease of use, were analyzed in the first regression model. the pcim variables, relative advantage, compatibility, image, visibility, ease of use, result demonstrability and trialability, were analyzed in the second regression model. all tam and pcim variables were analyzed together in the third regression model. no serious breaches of the assumptions of normality, linearity, homoscedasticity or independence of residuals were identified in any of the three models. some variables in the third regression model were very highly correlated but those variables were not selected by the stepwise procedure. the results of these regression are summarized in table 5 below. tam pcim tam & pcim independent variables beta t p-value beta t p-value beta t p-value perceived usefulness .475 3.79 .000 perceived ease of use .309 2.47 .016 result demonstrability .508 4.55 .000 .508 4.55 .000 australasian journal of information systems volume 15 number 1 december 2007 145 compatibility .326 2.92 .005 .326 2.92 .005 variance explained (r2) .547 .616 .616 adjusted r2 .535 .605 .605 significance of f p-value .000 .000 .000 table 5: results of stepwise regressions of tam and pcim independent variables on dependent variable intention to use the web for teaching inspection of the first regression model in table 5 shows that the model was significant, and that both the tam variables were selected by the stepwise procedure. they both had positively signed coefficients and together explained 54.7% of the variation in the dependent variable. perceived usefulness was selected to enter the model first and explained 50.9% of the variation and perceived ease of use explained an additional 3.8%. these two variables are correlated (r = .77) and when perceived ease of use was forced to enter first it explained 45.7% and perceived usefulness explained an additional 9.1%. inspection of the second regression model in table 5 shows that the model was significant, but only two of the pcim variables were selected by the stepwise procedure. they both had positively signed coefficients and together explained 61.6% of the variation in the dependent variable. result demonstrability was selected to enter the model first and explained 57.0% of the variation and compatibility explained an additional 4.6%. these two variables are correlated (r = .76) and when compatibility was forced to enter first it explained 50.5% and result demonstrability explained an additional 11.1%. the third model was identical to the second model. implications for research and practice the tam variables, perceived usefulness and perceived ease of use, performed well, together explaining 54.7% of the variation in intention to use the web for teaching in our sample of teachers. these two variables were correlated, with perceived usefulness showing slightly better explanatory power than perceived ease of use. of the seven pcim variables result demonstrability and compatibility had the best explanatory power, together explaining 61.6% of the variation in intention to use the web for teaching in our sample of teachers. this was substantially better than the two tam variables. these two variables were correlated, with result demonstrability showing slightly better explanatory power than compatibility. none of the other five pcim variables had significant explanatory power over and above that provided by result demonstrability and compatibility. this is interesting because two of these five variables which did not provide additional explanatory power are the tam variables which are incorporated in the pcim (ease of use is identical to perceived ease of use and relative advantage is almost identical to perceived usefulness [r = .995]). in other words, although perceived usefulness and perceived ease of use had significant explanatory power when used on their own, there were other variables (result demonstrability and compatibility) that provided better explanatory power in this situation. these results may be context specific to school teachers and their intended use of the internet/web for teaching, and may even be specific to our sample of teachers. nevertheless, the two pcim australasian journal of information systems volume 15 number 1 december 2007 146 variables explained over 10% more of the variation in intention to use the internet/web for teaching than did the two tam variables. our findings lead to the conclusion that it is preferable to use the pcim, rather than the tam, to investigate intention to use an information technology innovation. this is because the pcim incorporates the tam variables (ease of use is identical to perceived ease of use and relative advantage is almost identical to perceived usefulness) and also offers five additional variables (compatibility, image, visibility, result demonstrability, and trialability), based on diffusion of innovation theory (rogers 1983, 1995). these additional variables can perform better than the tam variables in some situations, of which this study of teachers’ intended use of the internet/web is one example. limitations and future directions the survey method was utilised in our investigation of the relationships between seven innovation adoption variables and intention to use the internet/web by teachers for teaching purposes. data was collected through the use of a self-administered questionnaire. this method of data collection provided several advantages including the utilisation of an existing valid and reliable survey instrument and the ability to collect data in an efficient manner. while the method did provide several advantages, the limitations of the method should not be overlooked. first, the method relies on participants providing as veridical a response as possible to the researchers’ questions. also, as was the case in this study, the research then calls for examining the relationships between two or more of these self-reported pieces of evidence. kline, sulsky, and rever-moriyama (2000) point out that this approach raises concern that in addition to the “true” relationship expressed by the calculated correlation coefficient, some of the correlation coefficient is actually measuring a “spurious” relationship. the utilisation of this type of research design and analysis introduces a potential source of bias referred to as common method variance, where responses to one question may affect the response to another question. another limitation of this investigation is that the survey was cross-sectional in nature, and from only two schools at a single point in time. as such, there is the potential for the confounding effects of any exogenous factors outside the variables under investigation to distort the results. future studies may utilize a longitudinal survey design, which was not possible to due the time constraints of this particular investigation. a further limitation of this study is the potential for systematic measurement error through either social desirability or demand characteristics (singleton & straits 1999). future studies should utilise a multitrait-multimethod approach (kline et al. 2000). the current study did not make use of the demographic data as part of the regression analysis. future studies could use various demographic variables such as access to technology, computer proficiency, level of it training, general internet/web use for non-school purposes and other factors to see if it is possible to explain more of the variance in the data concerning internet/web adoption in learning and teaching. future studies may also be designed to capture a very large state-wide sample to examine the effects of the demographic factors. teachers at schools in remote or difficult socio-economic areas may respond differently. a future study could incorporate longitudinal design, combined with some qualitative methodology. for example, the introduction of a new internet/web technology into a school district could be examined at various stages throughout the implementation process in order to ascertain the stability or otherwise of teacher perceptions regarding the innovation attributes examined in this research. australasian journal of information systems volume 15 number 1 december 2007 147 our findings have implications for educational administrators and researchers involved in designing, developing or implementing internet/web-based learning and teaching programs. this study shows that the tam and pcim provide meaningful explanations of teachers’ intentions of using the internet/web for teaching in schools. future studies could examine the applicability of the tam and pcim instruments in predicting the adoption of other technology innovations in learning and teaching. references adams, d.a., nelson, r.r, and todd, p.a. (1992) ‘pereceived usefulness, ease of use, and usage of information technology: a replication’. mis quarterly, 16(2), 227-247. borg, w.r. and gall, m.d. (1983) educational research: an introduction, 4th edition, new york, longmans. carroll, j., howard, s. peck, j. and murphy, j. (2003) ‘from adoption to use: the process of appropriating a mobile phone’. australian journal of information systems, 10(2), 38-48. cronbach, l.j. (1951) ‘coefficient alpha and the internal structure of tests’, psychometrika, 16, 297-334. davis, f.d. (1989) ‘perceived usefulness, perceived ease of use, and user acceptance of information technology’. mis quarterly, 13(3), 319-340. davis, f.d. (1993) ‘user acceptance of information technology: system characteristics, user perceptions and behavioral impacts’. international journal of man-machine studies, 38(3), 475-487. davis, f.d, bagozzi, r.p. and warshaw, p.r. (1989) ‘user acceptance of computer technology: a comparison of two theoretical models’. management science, 35(8), 982-1002. jones, c., hecker, r. and holland, p. (2002) ‘ small firm internet adoption: a market oriented approach’. australian journal of information systems, 10(1), 99-109. karahanna, e., straub, d.w. and chervany, n.l. (1999) ‘information technology adoption across time: a cross-sectional comparison of pre-adoption and post-adoption beliefs’. mis quarterly, 23(2), 183-213. kline, t.j.b., sulsky, l.m., and rever-moriyama, s.d. 2002. common method variance and specification errors: a practical approach to detection. journal of psychology, 134, 401-421. lee, y, kozar, k.a. and larsen, k.r. (2003) ‘the technology acceptance model: past, present, and future’. communications of the association for information systems, 12(50), 752-780. miller, m.d., kelly, r.k. and harper, j. (1997) ‘ the unidimensionality, validity, and reliability of moore and benbasat’s relative advantage and compatibility scales’, journal of computer information systems, fall, 38-46. moore, g.c. and benbasat, i. (1991) ‘development of an instrument to measure the perceptions of adopting and information technology innovation’. information systems research, 2(3), 192-222. australasian journal of information systems volume 15 number 1 december 2007 148 moore, g.c. and benbasat, i. (1996) ‘integrating diffusion of innovations and theory of reasoned action models to predict utilization of information technology by end-users’, in diffusion and adoption of information technology, kautz, k. & pries-hege, j. (eds.), london, chapman and hall. nunnally, j.c. (1978) psychometric theory, 2nd edition, new york, mcgraw-hill. rogers, e.m. (1983) diffusion of innovations, 3rd edition, new york, free press. rogers, e.m. (1995) diffusion of innovations, 4th edition, new york, free press. seyal, a. h. and rahman, m.n. (2003) ‘student use of the internet: an extension of tam in technical and vocational institutions in brunei darussalam’. australian journal of information systems, 10(2), 91-104. singleton, r. a. and straits, b. c. 1999. approaches to social research (3rd ed.). oxford university press, new york, usa. taylor, s. and todd, p.a. (1995a) ‘assessing it usage: the role of prior experience’. mis quarterly, 19(4), 561-570. taylor, s. and todd, p.a. (1995b) ‘understanding information technology usage: a test of competing models’. information systems research, 6(2), 144-176. teo, h.h., wei, k.k. and benbasat, i. (2003) ‘predicting intention to adopt interorganizational linkages: an institutional perspective’. mis quarterly, 27(1), 19-49. van akkeren, j.k. and cavaye, a.l. (1999) ‘confusion with diffusion? unravelling is diffusion and innovation literature with a focus on smes’. australian journal of information systems, 7(1), 60-67. venkatesh, v. (1999) ‘creation of favorable user perceptions: exploring the role of intrinsic motivation’. mis quarterly, 23(2), 239-260. venkatesh, v. and davis, f.d. (1996) ‘a model of the antecedents of perceived ease of use: development and test’, decision sciences, 27(3), 451-481. venkatesh, v. and morris, m.g. (2000) ‘why don’t men ever stop to ask for directions? gender, social influence, and their role in technology acceptance and behavior’, mis quarterly, 24(1), 115-139. venkatesh, v., morris, m.g., davis, g.b. and davis, f.d. (2003) ‘user acceptance of information technology: toward a unified view’, mis quarterly, 27(3), 425-478. warshaw, p.r. and davis, f.d. (1985) ‘disentangling behavioral intention and behavioral expectation’, journal of experimental social psychology, 21, 213-228. australasian journal of information systems volume 15 number 1 december 2007 149 appendix a technology acceptance model scale items (davis 1989) perceived usefulness 1. work more quickly (item 1 in appendix c) 2. improve job performance (item 2 in appendix c) 3. increase productivity (no item in appendix c) 4. enhance effectiveness (item 4 in appendix c) 5. makes job easier (item 3 in appendix c) 6. useful in my job (no item in appendix c) perceived ease of use 1. easy to learn (item 14 in appendix c) 2. controllable (no item in appendix c) 3. clear and understandable (no item in appendix c) 4. flexible (no item in appendix c) 5. easy to become skillful (no item in appendix c) 6. easy to use (item 13 in appendix c) australasian journal of information systems volume 15 number 1 december 2007 150 appendix b perceived characteristics of innovating scale items (modified short form) (moore & benbasat 1991) relative advantage 1. work more quickly (item 1 in appendix c) 2. improves quality of work (item 2 in appendix c) 3. makes job easier (item 3 in appendix c) 4. enhance effectiveness (item 4 in appendix c) 5. control over work (item 5 in appendix c) compatibility 1. compatible with my work (item 6 in appendix c) 2. fits well with way i like to work (item 8 in appendix c) 3. fits into my work style (item 7 in appendix c) image 1. greater prestige (item 9 in appendix c) 2. higher profile (no item in appendix c) 3. status symbol (item 10 in appendix c) visibility 1. commonly seen (item 12 in appendix c) 2. visible (item 11 in appendix c) ease of use 1. clear and understandable (no item in appendix c) 2. controllable (no item in appendix c) 3. easy to use (item 13 in appendix c) 4. easy to learn (item 14 in appendix c) results demonstrability 1. no difficulty telling others of results (item 15 in appendix c) 2. communicate consequences to others (no item in appendix c) 3. results are apparent to me (item 16 in appendix c) 4. no difficulty explaining if beneficial (no item in appendix c) trialability 1. able to properly try it out (item 17 in appendix c) 2. permitted to use it on trial basis (item 18 in appendix c) australasian journal of information systems volume 15 number 1 december 2007 151 appendix c questionnaire regarding using the internet/web in teaching please keep the following definition in mind when completing this section. the internet/web for teaching refers to your (teacher) use of the internet/web inside and outside the classroom to assist with your preparation and delivery of learning content. for example, use of the web for researching units of work, your electronic communication (eg. via email, cyber forums) with other teachers regarding teaching activities, or your use of unit web pages as a medium for delivering the content of your lessons. i expect that using the internet/web for teaching preparation and delivery activities will: 1 enable me to accomplish my tasks more quickly. 2 improve the quality of work i do. 3 make it easier to do my job. 4 enhance the effectiveness of my teaching. 5 give me greater control over my work. 6 be compatible with all aspects of my work. 7 fit into my work style. 8 fit well with the way i like to work. 9 provide me with greater prestige at school. 10 provide a status symbol at my school. 11 be very visible at my school. 12 be common in many classrooms in my school. 13 be easy to use. 14 be easy to learn. 4. i would have no difficulty telling others about the results of using the internet/web for teaching purposes. 5. the results of using the internet/web for teaching purposes are apparent to me. 6. before deciding whether to use the internet/web for teaching purposes i would have the opportunity to properly try it out. 7. i would be permitted to use the internet/web for teaching purposes on a trial basis long enough to see what it could do. 8. i plan to use the internet/web for teaching purposes in the future. (editors note: responses to the above styatements are measured on a 7 point scale, 1 strongly disagree to 7 strongly agree) australasian journal of information systems volume 15 number 1 december 2007 152 microsoft word vol 15 2 final.doc australasian journal of information systems volume 15 number 2, 2009 7 electronic commerce publications and the implications for research quality output in australia rens scheepers department of information systems university of melbourne, victoria, australia email: r.scheepers@unimelb.edu.au helana scheepers faculty of information systems technology monash university, monash, australia email: hscheepers@infotech.monash.edu.au julie fisher caulfield school of information technology, monash university, victoria, australia email: julie.fisher@infotech.monash.edu.au abstract australian universities and academics have recently had to consider the way research is reported and funded. it was expected that by 2008, the research quality framework (rqf) would be implemented. as a result we saw increased activity within universities focusing on the proposed criteria. there was, as part of this, significant activity around the area of quality outputs. whilst the proposed rqf will not happen in its current form, some changes are proposed by the new government. the proposed rqf sought to have research assessed according to quality and impact. part of both quality and impact relates to where research is published. for academics, although the previous rqf guidelines are not to be implemented, it continues to be increasingly important to target high quality journals if the research is to be rated as high quality. the question this raises for information systems academics is where do we publish for maximum impact? the information systems (is) field is diverse with researchers working in many areas and a publication outlet for one area may not be relevant for another. one area where many australian is researchers have focused their research interest is the field of electronic commerce (e-commerce). the research reported in this paper identified the publication outlets that would be regarded as amongst the highest quality for researchers wishing to publish ecommerce research. the authors analysed e-commerce research papers by australian researchers published in the period 2000 to 2005. the results describe where australian researchers are publishing in this field. the paper also provides australasian journal of information systems volume 15 number 2, 2009 8 guidance to those working in the e-commerce field on which journals and conferences to target to ensure their work rates highly. introduction in 2005 the then minister for education, science and training announced the rqf saying “once implemented, the rqf will provide the australian government with the basis for redistributing research funding to ensure that areas of the highest quality of research are rewarded” (nelson 2005). this was further described as focusing on: • “the quality of research including its intrinsic merit and academic impact academic impact relates to the recognition of the originality of research by peers and its impact on the development of the same or related discipline areas; and • “its broader impact or use, i.e. the extent to which research is successfully applied broader impact or usefulness relates to the recognition by qualified end-users that quality research has been successfully applied” (nelson, 2005). the then new minister for education, science and training, the honourable julie bishop, reinforced this message stating that the rqf “would measure quality through a combination of metrics and review by domestic and international peers" (bishop, 2006). the suggested quality ranking scale of 1 to 5 suggested a ranking of 5 would be research in the top 20% of research in its field internationally. based on this, universities began the planning process for the 2008 implementation (see for example butler, 2006; tang, 2006). in doing this we were forced to consider questions such as ‘how will quality be measured?’ and ‘how will the broader impact be measured?’ in relation to all types of publications. although the rqf, in its original form, will not be implemented, the incoming government has indicated that there will be changes to the way research quality is determined and that the work that has been done on publication metrics will be utilised (carr, 2007). the answer to these questions therefore continues to have a significant impact on the way australian scholars target research outlets. in this paper, we look at publications and publication outlets for e-commerce research from the perspective of quality. for this research, we use journal rankings and citation indexes as measures of quality. this is consistent with clarke’s (2006) position who suggests that a citation analysis can be used for “evaluating the quality and/or impact of works and their authors by means of the references made to them in refereed journal articles” (clarke, 2006). we examine the ranking process and identify conferences and journals regarded as high quality. in addition, the citation impact for the published papers is also provided. we also report on our analysis of these publications, which examined what has been published by australian information systems (is) researchers in the area of electronic business/commerce. the paper identifies appropriate outlets for e-commerce research, presents statistics on where australians are publishing, the choice of outlets and citations in the area of e-commerce1. specifically we sought to answer the following research questions: 1 it should be noted that although the rqf is not proceeding, is had been accepted as a distinct discipline for panel 4. australasian journal of information systems volume 15 number 2, 2009 9 1. where are australians focusing their e-commerce research in terms of publication outlets? 2. what is the quality of the publication outlets that are mainly chosen by australian researchers in e-commerce research? 3. what is the citation rate of publications published by australian e-commerce researchers? the paper is structured as follows. first we discuss the identification of quality publication outlets and the use of citation indexing. this is followed by a description of the research method used for the identification of the research output. the research method described here is adapted from a previous study by scornavacca et al., (2005). in the results section we start with an overview of all e-commerce research by the information systems community. this follows a description of the ecommerce journal rankings and citations for the research output of australian researchers in ecommerce. identifying high quality publication outlets until now publications have been categorized by dest in terms of the type of publication for example: book, book chapter, journal or conference. from a reporting perspective, little attention has been paid to the quality of the publication outlet. for example, all conference publications are treated as equal and attract the same funding ‘reward’ in terms of the dest audit. in institutions overseas, more attention has been paid to the quality of the journal or conference particularly when tenure is considered. one measure of quality of a research output is the quality of the journal or conference publishing the work. this is generally determined by either isi factor or a ranking of journals and conferences undertaken by respected academics in the field. ranking conferences and journals the discipline of information systems has been considering journal quality for the last 2 years. in 2006 the australian council of professors and heads of is (acphis, 2007) began the process of developing a ranked list of journals. in 2007 a review of the first ranking occurred at the annual acphis workshop. it is not an insignificant task to compile a ranking of conferences and journals; this problem is recognized by researchers in other fields (uncles, 2004). anyone attempting such a task must begin by asking themselves on what basis will the ranking be determined. the options include research method i.e. quantitative or qualitative, editorial board, publisher or rankings based on the judgment of those researching in the field. another option is to look at other published rankings. in the is field most of the previously published rankings adopt a north american perspective. mort et al., (2004) reflected on similar difficulties in the field of marketing. they concluded that: "the marketing discipline in australia and new zealand has continued to develop and mature, reflecting a dynamic fusion of the disciplines american, european and asia pacific characteristics” (mort et al., 2004). it could be argued that australian information systems research has developed along similar lines. using previously published rankings, which adopt a north american perspective, therefore may not be entirely appropriate. very recently, national ict australia (nicta), on behalf of all ict related disciplines, distributed for comment through the ishods list2 a ranked conference list (lloyd, 2006). currently there is no 2 ishods is an e-mail list established by the australian council for professors and heads of information systems and distributed to all heads and professors of information systems. australasian journal of information systems volume 15 number 2, 2009 10 is representative in nicta, however their list contains conferences nicta regards as is and these have been ranked (nicta, 2006). for many is researchers this list may not prove to be very helpful. for example, the european conference on information systems (ecis), hawaii international conference on system sciences (hicss) and international resource management association international conference (irma) are all ranked two by nicta. this specific ranking from an is perspective is highly debateable. it should be noted, however, that this is the first attempt to compile a comprehensive list for all those in the ict field. it illustrates the importance for the australian is community to debate and compile our own rankings if such rankings are to accurately reflect what our community believes is quality. identifying high quality publication outlets for specific research areas will provide guidance to researchers in the new research quality environment. that said however, the focus of this paper is not on the rankings per se, our focus is on reflecting over a period of time, where is researchers working in the field of e-commerce are publishing and what we can learn from this. our ranking of e-commerce conferences and journals in order to conduct an analysis of australian e-commerce publications it was necessary to identify high quality publication outlets in the field. to do this we used two rankings: • the first ranking was by bharati and tarasewich (2002) who reviewed perceptions of journal publications for e-commerce by e-commerce researchers. the perceptions were obtained through an e-mail survey. two different journal rankings were developed from the data, one based on appropriateness of a journal for publishing e-commerce research and the other based on the quality of journal for e-commerce publications. the ranking for appropriateness was also further subdivided into geographical areas (europe, australasia and north america). • the second ranking was based on the acphis (2007) broader ranking of information systems journals. this ranking was developed based on an analysis of several other published rankings, two north american rankings and two european rankings. these rankings were from the london school of economics, louisiana state university, isworld and vienna university. the australian is senior scholars also reviewed and revised the final list resulting in the current ranking. this ranking was distributed for further comment among senior is academics late 2007. acphis identified 182 journals regarded as is. the aim of the final list was to have 5% of journals ranked a+, 15% ranked a, 30% ranked b and the rest c. from this list, publication outlets relating to e-commerce were identified. a comparison of the bharati and tarasewich (2002) ranking and the acphis ranking of journals predominantly publishing e-commerce research were made to develop a final list. other leading journals in the field that also published electronic commerce research were also identified. the following journals were selected {acphis ranking for the journal is indicated in {brackets after the journal}: • the australian journal of information systems (ajis) {b} • communications of the acm {a} • electronic commerce research {a} • electronic markets (em) {a} • european journal of information systems (ejis) {a+} • information systems journal (isj) {a+} australasian journal of information systems volume 15 number 2, 2009 11 • information systems research (isr) {a+} • information technology and people {a} • international journal of electronic commerce {a} • journal of the association for information systems {a+} • journal of electronic commerce research (jecr) {c} • journal of management information systems (jmis) {a+ } • journal of organizational computing and electronic commerce {a} • mis quarterly (misq) { a+} all of the journals except for australian journal of information systems and information technology and people appear in both rankings. the australian journal of information systems was included because it is the only local journal of information systems. journals such as harvard business review was considered, but in the end excluded as its audience and contributors are primarily business not information systems professionals or researchers. the international journal of electronic business was also in bharati and tarasewich (2002) top 10 however was only launched in 2003 so could not be included. one journal, jecr, ranked c has been included as it was included in the bharati and tarasewich (2002) list and is specific to e-commerce. there is very limited information on conference rankings. acphis (2007) considered ranking some conferences and the following reflects those rankings • international conference on information systems (icis), the premier outlet for information systems research. {a+} • european conference on information systems (ecis), the premier outlet for information systems research in europe. {a} • australian conference on information systems (acis), the only local information systems conference. {b} • pacific asian conference on information systems (pacis), a regional conference to which many australian academics contribute. {a} • bled electronic commerce conference (bled), a key conference in the electronic and mobile business field. {not classified, however included because it is the longest running ecommerce conference} these were therefore the specific journals and conferences that were used for our analysis in the area of e-commerce, the results of that analysis provide important information for researchers publishing in this area. the authors acknowledge that there are likely to be other journals and conferences publishing e-commerce research however, we are focusing on those outlets considered specifically is. citation indexing for publications bibliometrics is the statistical analysis of scientific publications (glänzel, 2004) and has been studied since the early twentieth century. two measures of bibliometrics can be utilized in evaluating quality: journal impact factor and citation impact (tang, 2006). the journal impact factor is often used as a substitute for the prestige of a journal (tang, 2006). glänzel (2004), states that the number of citations to a publication is a measure of the impact of a publication in a field of study. australasian journal of information systems volume 15 number 2, 2009 12 the recording of bibliometric information can be found in isi (thompson/institute for scientific information) databases and non isi databases (tang, 2006). the isi publish the: australia university indicators (aui), web of science (wos), journal citation report (jcr), journal performance indicators (jpr), national citation report (ncr) and essential science indicators (esi). it should be noted however that thompson’s isi has serious limitations; few is journals are indexed and the reliance on citations as an indicator of quality is problematic (lamp et al., 2007). non isi databases are: google scholar, citeseer and scopus. various studies have compared the databases with one another and identified problems. a comparison between wos, scopus and google scholar by jacso (2005) found that substantial factual information is available about wos and scopus. it is therefore easy to ascertain what journals are indexed and how the citation indexing is performed. similar information for google is not available although frequently asked questions outline the features of the software. the process of citations collection is fraught with problems that each database approaches from different perspectives, each with its own advantages and disadvantages. specifically, it is mentioned that there are records in the databases that are not accurate and that will influence the search for specific authors or publications. in the evaluation by jacso (2005), google scholar was criticised for the limited and dysfunctional search options and lack of regular updating of the database. the databases also cover different subject areas. wos mainly covers social science, science, arts and humanities with publication records going back to 1945. scopus does not include references to arts and humanities and has a modest record of publications in the social sciences. scopus mainly focuses on health and life sciences and other scientific areas such as chemistry and physics. scopus commenced recording of publications at a later date and states that publications are included from 1995. in contrast, google scholar does not identify any information about the subjects covered, and how far back the records in the database go. the type of publication that is included in each of the databases also differs from one another. wos started to include conference proceeding recently. the search facilities on the conference proceedings are not yet available for citation searches. scopus does include conference proceedings as well as books. google scholar does not provide any information on what publications are included but appears not to cover books for example. the subject area coverage of the databases is problematic for a field such as information systems (clarke, 2006). in a comparison done by jacso (2005), showed that in general, wos has more citations for a subject followed by scopus and google scholar has the least citations for a subject. this stands in contrast to the study undertaken by clarke (2006) in which the citations to the publications of leading information systems researchers has been considered. in this study, the citations recorded in thompson/isi google were determined and compared, and it was found that google scholar delivered double the citations compared to thompson/isi. in an evaluation of the entries found in google scholar, jacso (2005) highlights that the results included absurd entries. clarke (2006) also criticises the results returned by google scholar such as broken links and information returned involves the fulfilment of specific requirements to validate the response. the results that are provided by wos are not however without problems either. clarke has identified a number of errors in the database of isi that has an influence on the citation index. despite all of these criticisms, the citation databases are a very helpful tool to identify high quality publications as well as to show the impact that specific papers have had on an academic field of study. in the process of identifying quality publication outlets the reports published by thompson/isi, which compare journal citation rankings and reports on average citations for specific australasian journal of information systems volume 15 number 2, 2009 13 fields are invaluable. in a study conducted by clarke (2006) he concluded that google scholar is an appropriate tool for identifying citations for the is field due to the broad nature of the field. research background this section will examine the use of literature reviews in evaluating the research conducted over a period. this is followed by the definition of e-commerce and a detailed description of how the literature was identified and classified in our study. research rationale undertaking an examination of the research literature to understand what is happening in a field is a well accepted method in many disciplines and this is also the case in the field of information systems (scornavacca et al., 2005). webster and watson (2002) note the paucity of published review articles in information systems, and argue the importance to the field of such research. they go on to note “as a result the progress of our field is impeded” (p. xiii). banker and kauffman (banker et al., 2004) examined 50 years of information systems research literature in 2004. they argued that the value to readers was in providing “…a retrospective view of the research contributions that form the foundational knowledge in the field of information systems” (banker et al., 2004, 294). more recently, zhang and li (2005) sought to answer a number of questions relating to the development of the hci discipline. they concluded: “the study is informative in providing the state-of-the-art of research issues and concerns, research emphases on gaps, potential research directions and publication and employment opportunities.” (p. 278). amongst the questions they sought to answer were questions such as “what topics are often studied together?” and “what methods are used to study what topics?” (p. 229). to answer those questions they examined hci articles from seven of the leading mis journals over a period of 13 years. through our examination of the research published by australian researchers in the field of e-commerce we can document the contributions made and the directions the research in this area has taken. this informs both the field of e-commerce generally as well as specifically how australian is researchers approach the publishing task. the research questions scornavacca et al., (2005) sought to answer included the focus of the research was, the methods used, how the data collection was carried out, and the main contribution of the research. defining electronic commerce electronic commerce for the purpose of this paper has been defined relatively broadly. the authors recognise that the field of e-commerce has attracted researchers from a number of areas including marketing, law, computer science and the arts to name just a few. in order therefore, to identify which papers would be considered relevant for this study we first identified those areas where ecommerce research took an information systems perspective. we believe the following three areas best reflect what is central to is researchers involved in e-commerce research: • commercial environments where the research involves the study of ecommerce within a commercial setting • research that investigates ecommerce processes or transactions • research investigating an it artefact such as a web site or e-commerce system. australasian journal of information systems volume 15 number 2, 2009 14 to avoid including papers that were for example particularly technical or marketing focused each paper had to involve research into two of the areas above. the area of e-government was excluded from the study because it did not meet the criteria as described above. selected conferences and journals webster and watson (2002) argue that, “a complete review covers relevant literature on the topic and is not confined to one research methodology, one set of journals, or one geographic region.” (p. xv -xvi) consistent with this it was deemed important to examine a breadth of publications when undertaking a review such as this. we identified a number of leading journals, both from north america and europe and considered carefully the conferences to be included as well as a ranking of bharati and tarasewich (2002) which reviewed perceptions of journal publications for electronic commerce by e-commerce researchers. analysis of publications all available conference proceedings and journal editions during the time period 2000-2005 were examined. all the journal and conference proceedings were examined by two of the authors to identify the candidate papers to be included in this research. the following describes the process undertaken. • identification that at least one of the authors’ affiliations was australian. • determination, based on the title of the paper whether the paper was very broadly in the area of e-commerce. if the paper was australian based research and deemed relevant the abstract was read and a decision made to include or excluded. published papers conducted by australian researchers based on research conducted outside of the australian context were not included. • compilation of a list of papers and authors • the third author resolved differences and determined the inclusion or otherwise of papers based on the criteria. • endnote was used to record all included papers. a database was developed to capture all classifications and to analyse the publication data. • the number of citations for each paper was then determined through google scholar in july 2006, based on the authors and title of each. the number of citations that was reported by google scholar was recorded in the database. in the event that no record was found for the paper a further search on the title of the paper or the authors’ names was performed. the authors evaluated all of the candidate papers. in approximately 95% of the cases the authors agreed on the inclusion of the papers. results the results section begins with an overview of all e-commerce research by the information systems community. this is followed by a description of the e-commerce journal rankings and citations for the research output of australian researchers in e-commerce. journal papers published table 1 details the total number of e-commerce papers published each year in each of the journals, and the number of papers by australian researchers published in the journal. it should be noted that australasian journal of information systems volume 15 number 2, 2009 15 some of the e-commerce journals are highly technical in nature for example the journal of electronic commerce research. apart from the ajis, electronic markets was the most popular outlet for australian researchers publishing in e-commerce over the period. journal 2000 2001 2002 2003 2004 2005 total (20002005) total australian australian journal of information systems 0 1 6 3 2 0 12 12 communications of the acm 23 13 15 29 13 14 107 1 electronic commerce research n/a* 25 20 18 22 19 104 1 electronic markets 35 32 30 26 32 34 189 7 european journal of information systems 1 1 4 8 3 2 19 4 information systems journal 1 0 1 1 4 1 8 0 information systems research 0 1 4 0 0 4 9 0 information technology & people 3 0 0 5 3 2 13 2 international journal of electronic commerce 9 24 30 25 24 27 139 2 journal of electronic commerce research n/a** 24 19 16 21 17 97 0 journal of management information systems 2 2 4 0 10 3 21 0 journal of organizational computing and electronic commerce 4 7 6 8 4 4 33 0 journal of the association for information systems 5 1 2 1 2 2 13 0 mis quarterly 1 0 2 1 1 0 5 0 total 84 131 143 141 141 129 769 29 table 1: electronic commerce papers published in target journals * journal commenced in 2001 **publications for this year were not available. conference papers published table 2 details the number of papers published by australian researchers in e-commerce each year at each of the conferences. figures in box brackets indicate total number of e-commerce papers (all researchers). for the australian conference on information systems only the number of papers relating to electronic commerce, by australians, is provided as the majority of presenters are australian. the results are presented in alphabetical order of conference. as presented in table 2, australians have targeted mostly regional conferences (acis, pacis, and ecis) and bled as publication outlets for their research work. given its population and geographical location, australian publications are very well represented at outlets such as ecis (20%) and bled (21%). this stands in stark contrast to icis, where no australian e-commerce researchers published during the period. comparison: conference vs. journal papers a total of 135 papers were published by australians in the target conferences and a total of 29 papers published in the target journals. figure 1 presents graphically the comparison between journals and conferences published by australians during the period. the figure also suggests that total australian publications (conferences and journals) in electronic commerce peaked in 2003 with a total of 35 conference and five journal papers. australasian journal of information systems volume 15 number 2, 2009 16 conference 2000 2001 2002 2003 2004 2005 total ec publications total australian ec publications n/a* 9 12 12 12 3 48 n/a* [9] [16] [12] [15] [3] [55] n/a* 12 7 10 3 6 38 n/a* [49] [46] [71] [53] [50] [269] 2 3 4 5 8 3 25 [20] [19] [20] [23] [24] [17] [123] 0 0 0 0 0 0 0 [11] [12] [14] [20] [15] [10] [82] 6 3 2 8 2 3 24 [14] [16] [14] [20] [13] [11] [88] total australian ec publications 8 27 25 35 25 15 135 total overall ec publications [45] [105] [110] [146] [120] [91] [617] australian conference on information systems bled electronic commerce conference european conference on information systems international conference on information systems pacific asian conference on information systems table 2: electronic commerce papers published in target conferences * publications for this year were not available. figures in box brackets indicate total number of e-commerce papers figure 1: journal and conference publications by australians during period 21 27 25 35 25 15 3 2 9 5 4 6 0 5 10 15 20 25 30 35 40 2000 2001 2002 2003 2004 2005 conferences journals australasian journal of information systems volume 15 number 2, 2009 17 journal and conference publication ranking using the journals and conference ranking developed by acphis as outlined in section 2.2, we have determined the number of publications in each of the categories. table 3 details the number of e-commerce journal publications for the period 2000 – 2005. journal category journals in which publications were found number of australian publications total per journal per category a+ european journal of information systems 4 4 a communications of the acm 1 electronic commerce research 1 electronic markets 7 information technology & people 2 international journal of electronic commerce 2 13 b australian journal of information systems 12 12 table 3: number of journal publications per category table 3 highlights firstly the challenge of having any work published in a+ journals but specifically the limited number of e-commerce papers that have been published in a+ journals. over the 6 year period investigated a total of 75 papers were published in e-commerce in the top journals. during this period, just over 700 papers were published in a+ journals. of these, 11% were publications relating to e-commerce. the majority of journal publications are in journals classified as a and b. few papers have been published by australians in a+ journals. less than 0.5% of all e-commerce publications in a+ journals were written by australians. table 4 details the number of e-commerce papers published at conferences. conference category number of australian publications a+ 0 a 49 b 48 not classified 38 table 4: number of conference publications per category the figures show that most of the publications are in the lower ranked conferences and no publications appeared at a+ conferences, despite the fact that over the period 2000 – 2005, 82 papers were presented at icis in the e-commerce field. australasian journal of information systems volume 15 number 2, 2009 18 journal and conference publication citations the total number of citations for each of the papers was determined through google scholar. table 5 presents the number of papers not indexed in google scholar, papers that were found in google scholar but did not have any citations and papers that had one or more citations in google scholar. figure 2 shows the categories of paper citations found in google scholar as percentages. citations number not indexed 66 one or more citations 63 zero citations 48 table 5: citations of publications (google scholar) not indexed 38% one or more citations 35% zero citations 27% figure 2: citations in google scholar the citations were totalled and an average citation was computed per outlet. the results of this calculation are presented in table 6. as was expected, the journals and conferences that were classified higher attracted more citations. an average over all paper citations of 4.67 was computed. to place this average into perspective, thompson isi published an average citation for computer science publications for the period 1994 – 2004 of 2.32 (butler 2006). the average citations computed for the papers under study thus compare favourably. it should however be taken into consideration that google scholar is a ‘brute force’ citation index and provide far more citations compared to thompson isi (clarke 2006). according to clarke (2006) who compared the citation count for authors on google scholar and thompson/isi, thompson/isi have half the citation count compared to google scholar. using this as a general guideline the number of computer science citations could thus be doubled to 4.64. the average citations computed for e-commerce publications are then of a similar order. australasian journal of information systems volume 15 number 2, 2009 19 outlet number of cited papers total citations average european journal of information systems 3 92 30.7 electronic commerce research 1 12 12.0 information technology and people 1 10 10.0 communications of the acm 1 9 9.0 electronic markets 3 13 4.3 pacis 6 22 3.7 bled ecommerce conference 28 90 3.2 ecis 5 12 2.4 acis 11 26 2.4 international journal of electronic commerce 2 4 2.0 australian journal of information systems 2 4 2.0 table 6: total and average citations for each outlet discussion the analysis of conference and journal papers published by australian researchers in the field of ecommerce has provided an interesting insight into where australians are publishing and the quality as possibly judged by rqf-type criteria. it needs to be noted however that the use of any ranking can be problematic. the acphis ranking which we used was based on a number of other published rankings. the ranking, however, indicated that the journals, which are highly regarded by e-commerce researchers (bharati et al., 2002), were ranked lower and were excluded in our study. on the other hand citation databases are criticised for a lack of completeness and unknown content. the study that was done by (clarke 2006) identified that for the field of is google scholar might be an appropriate database for finding citation figures. our discussion focuses on the implications of these results from the perspective of quality and the quality evaluation of research publications. the results depicted in this paper highlight that australian is researchers would need to consider carefully where they target their publications in the future. currently, the data suggests we are not publishing regularly, and in some cases not at all, in the a+ journals and conferences. for australian is researchers working in the e-commerce field, we would recommend that researchers be strategic in terms of which journals to target particularly in terms of quality. for example, journals such as ejis and jmis publish more e-commerce research than the other journals in the a+ category. for a journals we should be considering electronic markets, electronic commerce research and the international journal of electronic commerce. these journals are well regarded, highly ranked, well cited and focus on e-commerce. citation counts could become important under a quality evaluation of research publications. our research highlights the importance of being strategic in the journals and conferences we target for our papers. we must also be mindful of the fact that the older the paper the more citations it will attract. the statistics therefore that are presented here understate the impact of the papers. it is likely that the citations will significantly increase with time. journals that are not ranked within a discipline will present researchers with a dilemma. do we seek to publish in a journal that is highly reputable and could very well have a high citation count australasian journal of information systems volume 15 number 2, 2009 20 yet is not ranked? it will be difficult under an evaluation such as the rqf for researchers to argue a journal is high-quality if it is not ranked. our research highlights the impact australian researchers have in the field of e-commerce internationally. given the relatively small population of is researchers in australia the number of publications produced is high. however, it needs to be noted that a closer analysis suggests a few people account for many of the papers. there are a small number of papers produced by australian researchers that are highly cited. nevertheless the research does indicate that australians are publishing good quality research in the field of e-commerce. however, it also tells us that we are not strategically targeting where this research is published. for example, as illustrated in table 2 and table 6, the bled conference and acis are the two avenues where most work is published, yet both have a very low citation average, 3.2 and 2.4 respectively. whilst ecis is also low, authors can argue that this conference is ranked highly which is not the case for either bled or acis (making it more difficult to argue for the quality of a paper published at the latter two conferences). what this also tells us is that as is researchers, working in the field of e-commerce, we need to be targeting conferences, which give us visibility and citations and then build on these papers with a view to publishing in highly ranked journals. international collaborations will also become increasingly important. the research also raises the question whether e-commerce as a research field is still relevant. if we look at the trends over the six-year period it would seem that during 2002/3 e-commerce, as measured by publications, peaked. however a closer look, particularly at the journal publications, suggests that there is still interest. the second-highest number of papers published by australians in journals in the field was in 2005. a look at the trend of journal publications overall supports this with an increase in some journals in 2005. conclusion the timeframe of this study is for the period 2000 – 2005. in all likelihood the evaluation of research publications will take place in 2009 (carr 2007) and will include the publications in this timeframe. whilst our findings cover “the past” they do provide guidance for the future. irrespective of how the field of e-commerce develops, this research is important in providing a picture of publishing activity by australians. many of the findings will be relevant to other is research areas. an interesting extension of this project would be to examine all publications by australian is researchers for the same period. references acphis "2008 journal rankings", http://www.acphis.org.au/, date accessed: december 2007 banker, r., and kauffman, r. (2004) "the evolution of research on information systems: a fiftiethyear survey of the literature in management science," management science, vol. 50 no. 3, pp 281-298. bharati, p., and tarasewich, p. "global perceptions of journals publishing e-commerce research," communications of the acm (45:5) 2002, pp 21 -26. bishop, j. "keynote address knowledge transfer and engagement forum sydney june", http://www.dest.gov.au/ministers/media/bishop/2006/06/b001160606.asp, date accessed: july 2006 australasian journal of information systems volume 15 number 2, 2009 21 butler, l. "bibliometrics and the rqf. caul – dest performance indicators forum." http://www.caul.edu.au/meetings/rqf2006performanceindicators.pdf, date accessed: july 2006 carr, k. "cancellation of research quality framework implementation", http://minister.industry.gov.au/senatorthehonkimcarr/pages/cancellationofresea rchqualityframeworkimplementation.aspx, date accessed: january 2008 clarke, r. "a pilot citation analysis of australian information systems researchers", http:/www.anu.edu.au/people/roger.clarke/sos/aiscit0605.html, date accessed: july 2006 glänzel, w. "bibliometrics as a research field: a course on theory and application of bibliometric indicators." http://www.norslis.net/2004/bib_module_kul.pdf, date accessed: august 2006 jacso, p. "as we may search comparison of major features of the web of science, scopus, and google scholar citation based and citation enhanced databases," current science (89:9) 2005, pp 1537-1547. lamp, j., milton, s., dawson, l., and fisher, j. "rqf publication quality measures: methodological issues," 18 th australasian conference on information systems, the university of southern queensland, toowoomba, queensland, 2007, pp. 478-486. lloyd " ict rankings letter," correspondence to ishods, 2006. mort, g., mccoll-kennedy, j., kiel, g., and soutar, g. "perceptions of marketing journals by senior academics in australia and new zealand," australasian marketing journal (12:2) 2004, pp 51-61. nelson, b. "research quality framework: assessing the quality and impact of research in australia the preferred model", http://www.dest.gov.au/sectors/research_sector/policies_issues_reviews/key_issues/research _quality_framework/rqf_preferred_model.htm#the_rqf_preferred_model, date accessed: july 2006 nicta "conference_tiers_june_combined," distributed to ishods list, 2006. scornavacca, e., barnes, s., and huff, s. "mobile business research, 2000 -2004: emergence, current status, and future opportunities," 13th ecis, university of regensburg, regensburg, 2005. tang, r. "bibliometrics and citation analysis", http:/www.monash.edu.au/quality/bibliometricscitation-analysis.ppt, date accessed: july 2006 uncles, m. "journal rankings: how much credence should we give them?," australasian marketing journal (12:2) 2004, pp 67-72. webster, j., and watson, r. "analysing the past to prepare for the future: writing a literature review," misq (26:2) 2002, pp xiii-xxiii. zhang, p., and li, n. "the intellectual development of human -computer interaction research: a critical assessment of the mis literature (1990 2002)," journal of the association for information systems (6:11) 2005, pp 227-292. microsoft word ajis112final.doc australasian journal of information systems vol 11 no 2 may 2004 46 south african, rural ict implementation: a critical retrospective application of latour’s due process model jo rhodes 8/1 manns avenue neutral bay 2089 jorhodes44@yahoo.com abstract the potential developmental role of icts can pressurise governments to engage in ‘catch up’ and ‘leapfrog’. consequently, analysis of the accompanying sociopolitical dimensions and risks can be, disastrously, neglected. this paper examines a specific technology implementation – a south african government sponsored telecentreusing latour’s due process model, an analytical tool grounded in actor-network theory, where technology implementation is viewed as a symmetrical treatment of technology and society within a single collective. it is used here, retrospectively, to make sense of why the telecentre both failed to institutionalise within a successful actor-network, and, contributed to the destabilization and partial destruction of a successfully established women’s development organisation. introduction history indicates that technological innovation is essential for human development and recently the debate regarding the efficacy of ict, as a development tool, has been robust amongst academics, practitioners and policy makers. current initiatives seem to focus on infrastructure development and on expanding and delivering information from the ‘core’ to the ‘periphery’. a substantial body of literature outlines potential benefits and roles of the new technology in development (world bank 2001, 2000, undp 2001, maddon 2000, laudeman 2003, dalsgaard 2001, chapman and slaymaker 2002, heeks 1999a, 1999b, morales-gomez and melesse 1998). yet, the jury has not yet returned its verdict on the matter of icts and development in remote african rural locations. many reasons are cited for the unsuitability of icts to african rural development, such as the lack of connectivity, low tele-density, limited internet access, scant ict and computer education, limited literacy, and minimal human and technological networks. nevertheless, it is considered axiomatic that if developing countries turn their back on technological innovation, they may condemn themselves to continued marginalisation. nepad (the new partnership for african development) is targeting ict as a priority sector for the achievement of sustainable development in africa in the 21st century. its’ objectives are to diffuse icts and achieve ereadiness for all countries in africa. this intention is paralleled by the southern african development community ict policy and regulatory support program (siprs), which aim to advance policy harmonization by producing model policies in key areas, including, human resource development and information sharing. it is anticipated that the siprs project will lead to greater liberalization of the ict sector and foster a competitive market within the region. whilst popular development dialectical reasoning points to the promise of significant economic and social transformations, as a result of the new technology and ict applications, the paradox remains that minimal success is observed to support the premise. this however, seems not to slow down the growing trend to view icts as a panacea for underdevelopment, which stimulates many unrealistic and unrealizable hopes and expectations. this in turn puts mounting pressure on government and development agencies to introduce them before adequately understanding the social context and dynamics into which the ict is implanted. this urge to play ‘catch-up and leapfrog’ often results in inadequate and insufficient assessment of the risks icts pose to sustainable development (skuse 2001). few successes are identifiable and many projects end in failure as strategists, academics and policy makers’ struggle with the conundrum of ict bounded development issues. this paper further explores ict, as a development tool in a specific context, through examining the case of a telecentre implementation in a remote rural 3 south african community. using latour’s 3 in most countries, this term indicates a low-density population or a dependence on farming. however this is not applicable in this sense as in apartheid south africa many areas defined as rural were areas without basic human services such as electricity, water and sanitation with a high population density of displaced (mostly through forced removals) people with no economic base. australasian journal of information systems vol 11 no 2 may 2004 47 due process model the findings of this study highlight how complex adversarial development conditions can impact on technology implementation. the paper, including the introduction, is in five parts. part two discusses the rwa telecentre development context. within this section the rwa organisation and the prevailing, historical and present local development conditions framing the ict initiative are described. part three, is a retrospective analysis of the telecentre implementation process using latour’s due process model, a practical tool grounded in actornetwork theory. part four discusses the research conclusions and the paper finishes with a discussion of forthcoming research. the rwa telecentre development context this section describes the rwa organisation (the entity assigned the responsibility for operating the telecentre) and discusses the complex socio-political context within which local economic development in this area is forged. it concludes with a description of the rwa telecentre. the rwa sister lydia pardeller of the franciscan order of nuns founded the rwa, a roman catholic development initiative. in 1992 sister lydia was charged with locating the 'poorest of the poor' community in south africa to facilitate improvement in the lives of women and children. her search led her to the village of apel, in sekhukhuneland, 150 km southeast of polokwane, formerly pietersburg, the major town in the limpopo province (formerly northern province and formerly the northern transvaal). the rwa is regarded as a rare example of a successful women’s development organisation in a region beset with political complexity, instability and morbid conflict. it has grown, in a short space of time, into a modern, legally incorporated, highly resourced (money, people, skills, equipment, infrastructure) organisation with a successful track record of achievements (rhodes 2000). within 4 years of inception the rwa self-help projects eradicated decades of both child malnutrition-caused mortality and hunger, and generated a small income for some of its members. despite this very commendable achievement, substantial economic growth and prosperity eludes these rural communities. rural poverty remains a major problem and access to markets remains stagnant. the prevailing local development conditions the rwa emerged from a turbulent, conflict-ridden space where, historically, development has been, and still is, highly politicised. this poorly developed rural area has a long tradition of revolt and underdevelopment. the former transitional local government set up after the first democratic election in 1994 has, only recently, become a democratically instituted municipality. the area is designated a special presidential area, prioritized for integrated rural development with a keen focus on gender empowerment. the multiple layers of complexity, which underpin local development, are represented by figure 1. australasian journal of information systems vol 11 no 2 may 2004 48 figure 1. layers of complexity figure 1 reveals only a partial vista: added to this situation is a legacy of generations of poverty, deprivation and violent turbulence, compounded by years of debilitating drought. an age-old conflict between the surrounding villages endures, as over the last 100-year period there has been no dispute settling mechanism outside of a fractious paramouncy within a fractured community (delius 1996). the various chiefs have been legitimized at different times in history, some by apartheid bantu authority and some, in the past and most recently by the current african national congress government. whilst the south african government is attempting to craft a modern state modeled on liberal principles, in apel (the head office of the rwa), everyday life is lived with a deep-rooted fear of malicious supernatural forces. the new democratic and gendered government dispensation is superimposed upon historical layers of other forms of (conflicting) government and belief systems. at present two conflicting models of government (traditional and democratic), underpinned by different belief systems, co-exist in a state of heightened tension. the new south african constitution abolished bantustans, whilst at the same time protected the existing status of traditional /tribal authorities as well as enshrining a democratic bill of rights. equal rights for women conflicts with traditional authority but both are expected to function alongside each other. hunger, poverty and unemployment (all pre-existing conditions, which accompanied the transition from apartheid into democracy,) create jealousies that can quickly turn to anger and vengeance and this can translate into witchcraft accusations. kohnerts’ (2003) research found that witchcraft related violence increased dramatically during the transition from apartheid rule to democracy. in the 1990’s in limpopo province, where the rwa is sited, the death toll from witch killings was in the 1000’s. apart from the immeasurable harm that ‘witchcraft violence’ inflicts on individuals and their families it also destabilizes the social, economic and political setting of a whole region, and endangers the legitimacy of the new post-apartheid government. in 1996 the limpopo ministry of safety and security created a commission of inquiry into witchcraft, violence and ritual killing. the situation necessitated the provision of a witch sanctuary (in helena village also known as garamase) 35 km west of polokwane city, where hundreds deemed guilty and in danger of being killed by mob action took refuge. the commission on gender equity (cge 1999, ralushai et al. 1996) requested that witchcraft related violence be declared a national priority crime, resulting in the 1998 thoyandou declaration of “ending witchcraft violence” bill. in apel, the village in which the rwa operates, disturbances in the social system (such as murder and theft) are often attributed to evil people and evil power. the residents of this area exist in a space of spiritual insecurity where the sangoma (a spiritual doctor, a medium of sacred or transcendental power) plays a major part in harmonizing society and in so doing reinforces traditional authority. witchcraft beliefs have serious repercussions for conflict resolution, a fundamental base for sustainable democratization and development. secondly, witchcraft accusations are historically a form of gender violence and control, both of which have serious australasian journal of information systems vol 11 no 2 may 2004 49 implications for gender development. as women are more vulnerable to accusations than men, and as vulnerability increases with age (gorman and heslop 2002), gender development and ict development occurring within a gendered organisation is particularly at risk from this phenomenon. the rwa telecentre throughout africa, telecenters have been viewed as the primary vehicle to provide icts (benjamin 2000). telecentre, is a loose term used to describe a place where public access to computers, training and the internet is available along with a number of business services such as typing and photocopying. the telecentre programme is heralded as one of the most important initiatives of the 1990s in ensuring that developing communities can make use of modern icts, and to link micro and small enterprises with timely market information (ntca 2000). in south africa, telecentres were installed in remote rural locations by the usa (universal service agency), a statutory body established under the 1996 telecommunications act responsible for ensuring universal access to all telecommunications services for disadvantaged communities in south africa. this programme committed to provide sustainable telecentres and socio-economic information services to disadvantaged communities and in line with this objective awarded the rwa a telecentre. the rwa telecentre equipment, installed in 1998, included 4 computers (two with internet provision), 6 telephone lines and a photocopier. two rwa members, appointed as telecentre managers, received 6 weeks training before being charged with the responsibility of building a financially viable business. since then (in the rwa telecentre) there has been minimal support and guidance. unfortunately the telephone lines were to stay unconnected for 4 years due to a dispute between various government departments and telkom (the south african monopolist telephone provider). most technology transfer, at the local african rural level, is donor led. baark and heeks (1998) study of the process of donor funded technology transfer found that donors are good at shifting boxes from the west to developing countries, but making those boxes operational at the other end is problematic and often they fail. they attribute this to the scant attention paid to the fact that technology is more than just equipment and has a surrounding shell of infrastructural requirements and technical and managerial skills. often these necessary elements in the surrounding shell are missing. the usa telecentres are heavily dependent on donor support. although these projects stress community participation and sustainability, to date none have proven that they can be self sustaining post external funding. of the 64 usa telecentres only three are functional and only four have online internet (benjamin 2001). in april 2002, a series of traumatic events (the murder of the rwa telecentre security guard, accusations of theft and nepotism amongst the rwa members, the theft of valuable agricultural equipment and the reluctance of the police to act) precipitated witchcraft accusations from some of the rwa garden projects members. these events opened a window for traditional authority to gain a foothold in a gendered organisation through the members’ request for a sangoma to harmonize the situation. this realignment of interests in the rwa actor-network and the inclusion of other actornetworks, assisted the traditional authority, the chief, to disparage the legal rwa dispute settling mechanism and, giving credence to witchcraft accusations, he attempted to take control of the rwa. this resulted in the stalling of many of the rwa development projects and in the rwa management team being forced from their office through violence and intimidation. a consequence of these events was the closure of the rwa telecentre, as a day after the telecentre security guard was murdered the usa representative removed all equipment from the telecentre. at the time of writing (jan 2004) the telecentre was still inoperative. the next section retrospectively applies latours’ due process model to the rwa telecentre implementation to analyze why the telecentre failed to institutionalize (that is, to become an accepted and stable fact). the relationship between society and technology may be conceptualized in many ways along a continuum ranging between technological determinism and social constructivism, each with its own bias and conceptual difficulties. the due process model is considered particularly relevant here as it treats technology and society symmetrically and focuses on the set of elements that influence, shape or determine an action such as technology implementation. australasian journal of information systems vol 11 no 2 may 2004 50 the due process model and telecentre institutionalisation there are at least three ways to consider technology implementation. (1) technological determinism, where technological development is viewed as separate from and outside of society. (2) technological constructivism, where technology is shaped by social forces, implying that technology in itself, plays no role and (3) the sociotechnical view which links social and technical systems together, but still treats them independently of each other. mcmaster, vidgen and wastell (1998) suggest that none of these lenses offer adequate opportunities for a true sociotechnical understanding as they treat technology and society asymmetrically, as two single collectives, subject to different languages, meanings, beliefs, analysis and interpretive tools. this approach creates two unbridgeable collectives and the subsequent collapsing of these two single collectives into a sociotechnical whole is artificial. a more valuable way of viewing technology implementation is through a symmetrical treatment of technology and society, within a single collective using actor network theory (law and hassard 1999, callon 1999,callon 1991, latour 1987, callon and law 1986). actor-network theory (ant) regards technology and society as fundamentally equal entities subject to symmetrical treatment. ant is an antipositivist epistemology as it observes differences in a network and not the concepts of right and wrong. it proposes a shift away in emphasis from the primacy of the human actor, considering him/ her as just another actor in the network. an actor-network is a description of the way things are and is comprised of a series of heterogeneous animate and inanimate elements that have been linked to each other over time. according to latour (1997), an actor-network is a series of translations of interests by the actors. the actornetwork is not reducible to the actor (human or non-human) alone and such networks are unreliable as they can become unstable with the entry of new actors and the desertion of existing actors. a simple example of an actor-network is an organisation such as a university, where the actors will include (but not be limited to) entities such as chancellors, professors, lecturers, students, constitutions, policy documents, reporting proceedures, computers, enterprise systems, campus, the internet, physical infrastructure and government legislation. a change in this network (for example adding actors in, subtracting actors, or reconfiguring the hierarchy of actors) can reveal that power effects are relational and that ordering is contestable. an actor exists in context and juxtaposition to other actors where each modification affects both this network and the networks of the actors themselves. if for example, a new chancellor is installed, or a new policy implemented, this change in the actor-network will often precipitate many others. the existing actors in the network will affect the outcome of the change. sometimes, these changes are unintended, unanticipated consequences referred to as the ourobos effect and depicted as the serpent devouring its own tail, a metaphor for how some of the best attempts to solve problems come back to bite. ant began appearing, with increasing frequency, in the information systems literature during the last decade (macome 2002) to track the formation of facts, also referred to as black boxes, within heterogeneous networks of aligned interests. this is an active, critical process through which specific candidates for admission (technical and social) into the network are either accepted or rejected. ant perceives facts or black boxes as being the result of the strengthening of claims and alliances through the enrollment of additional actors into a network; this perspective presumes that facts are constantly transformed (translated) as the network lengthens across time and space. ‘a major focus of the theory when applied in particular contexts is to try to trace and explain the processes whereby relatively stable networks of aligned interests are created and maintained, or alternatively, to examine why such networks fail to establish themselves. successful networks of aligned interests are created through the enrollment of a sufficient body of allies, and the translation of their interests so that they are willing to participate in particular ways of thinking and acting that maintain the network.” (walsham 2001:46). the due process model, grounded in ant, is a practical, valuable analytic device used to overcome the mammoth complexity involved in investigating both human and non human-network processes. these actors affect both the creation of black-boxes (extant uncontestable facts) and the degree of stability and irreversibility in the actor-network. the application of the model is useful in forcing a rethink of issues before an ict implementation takes place (or, lamentably, retrospectively to make australasian journal of information systems vol 11 no 2 may 2004 51 sense of a technology implementation outcome). in proactively planning for the introduction of a new ict into a community or organisation the due process model is used to observe the existing actors in the network and to systematically explore the factors that may secure a successful outcome (referred to as a black-boxed and irreversible technology). the due process model applied here follows the mcmaster, vidgen and wastell’s (1998) and the roode (forthcoming) approach. it is used retrospectively to make sense of the process of installing a telecentre into a remote rural african community, using a woman’s development organisation as the embedding mechanism. figure 2. due process model -after latour 1998 (mcmaster, vidgen, wastell, 1998). the model has four parts as illustrated in figure 2 and discussed briefly below. whenever new candidates for existence (actors), which can be facts, claims, technologies, policies or people etc, are introduced into a network they increase the existing levels of uncertainty adding to the perplexity of the network. perplexity revolves around the issues of confusion, grasping meaning and intent and thinking clearly and logically. if perplexity is insufficiently addressed a false sense of understanding predicates the next stage of the model, making perplexity settlement crucial to the usefulness of all three following components of the model. anecdotal evidence is plentiful in the interchange between rural, illiterate people and their sometimes ‘magic bullet’ conception of ict capability. because these perplexities are often not logically addressed, unrealistic expectations can later create tensions that contribute to conflict. in some cases the self interests (for example the donors performance objectives set by government ministries) of the actor supplying the telecentre conflict with the timelines and debate necessary to achieve sufficient perplexity settlement within the recipient actor-network. consultation is a deliberation between parties where advice and opinion is sought and given and is a requirement to confer legitimacy on the new candidate for admission (actor). consultation involves assessing the suitability of the new actor’s entry into the network. different consulting styles and positions will impact significantly in this phase. the power relations between the telecentre supplier and the community recipient will often be asymmetrical, which biases the consultation, particularly australasian journal of information systems vol 11 no 2 may 2004 52 in a remote and impoverished locale where any change is almost universally perceived as an improvement to existing conditions. questions to be addressed in this phase include, who is represented on the negotiation team, is there sufficient and authentic negotiation happening, and are the concepts adequately understood by all appropriate and necessary parties to ensure authentic negotiations. next, debate is encouraged on the relative importance of the candidate in the hierarchy of the network. hierarchy involves organizing elements into a ranking, with each rank subordinate to the one above it. this is the process of understanding how the new actor will be positioned in the network rendering it acceptable to the current network configuration. an example of debatable issues include: where will the telecentre be positioned in the organisation, what resources will be allocated to it, will other projects loose resources in the reallocation, does this require new alliance building, will the new configuration be supported across the organisation. this process of perplexity, consultation and hierarchy does not happen sequentially and linearly and all concepts are used in all parts of the model at some stage. for example, new hierarchical decisions will require further perplexity settlement and negotiation. these three parts of the model translate (or not) the candidate of existence into the fact (black-box) and imbue it with the values required for it to be accepted into the network. this increases the likelihood of the fact proceeding into the fourth phase institutionalization (which means the innovation is accepted into the network as an unquestioned and supported fact). if it is not successful, the actor(the telecentre) is rejected from the network but this does not preclude it from returning at a later stage to attempt a readmission in the network. the due process model is applied below to the rwa telecentre, the new candidate for admission into the rwa actor-network. the telecentre, as a candidate for admission in the rwa actor-network, introduced many new perplexities including but not limited to the following. what is a telecentre? what will it do? how will it do it? how is it used? who will use it? how many will use it? who will pay for the services offered? how much will they pay? what will they pay for? what is affected by its use? what equipment will it have? what type of change will it drive in the community? what skills will be required? who will provide these skills? to who will the skills be provided? why will people use these services? what difference will it make to their lives? how will it reduce poverty? minimal debate around these perplexities occurred around the telecentre, as a new candidate for admission into the rwa actornetwork, as the telecentre supplier (the usa) had a preset plan dictating the telecentre function. other related rwa organisational issues left unanswered included the following: is the telecentre a development initiative? how will it enhance development? is it a business? is it both? how will we blend business and development into a meaningful hybrid? where are the limits to blending business and development? what importance is the telecentre to the rwa? does it fit into our strategic vision? where does it fit? how does it fit? how can it improve our work? how can it help us access more profitable markets? who will show us how to use it? little discussion occurred regarding the latter perplexities and limited consultation happened as to how the telecentre would operate and how it would fit into the wider community. the central role of the telecentre became an ms word software training unit. the usa, using a “one size fits all” approach unilaterally decided on equipment needs, on how the telecentre would be managed and who would own it. it was painted in the distinctive usa colors and called the usa telecentre. a contract was drawn up declaring the rwa the legal owner of the telecentre; this however proved to be a false claim as the usa unilaterally removed all of the telecentre equipment four years later, in complete contravention of the existing legal contract, effectively destroying the telecentre operation. the legitimacy of the rwa candidature as the telecentre owner was unexamined in relation to other possible contenders, nor were the potential vulnerabilities relating to gender owned ict initiatives in the local traditional authority hierarchy debated. this was particularly remiss in light of the current and new changes to the political landscape (the new south african post-apartheid constitution) that legislated for gender equality whilst expediently maintaining gender-opposed traditional authority in the locality. previous development failures in the area remained unexamined despite the turbulent history of former development initiatives in the area. the choice of the rwa as the telecentre beneficiary seemed, superficially, a simple and workable option for the usa, with its impressive track record of development successes and its eligible building to physically house the telecentre in. deleted: . australasian journal of information systems vol 11 no 2 may 2004 53 the usa, in its race to achieve the objectives of its own actor-network (the department of communications) to implement 64 telecentres by a particular date and report back to the minister of communications, neglected to adequately consider the particular rwa sociopolitical factors. the lack of debate and consultation around the latter issues resulted in situating the telecentre with little reference to the local habitat. additionally, no vision of the position of the telecentre in the rwa actor-network hierarchy was formulated. as it was a requirement from the usa that the telecentre operate as an autonomous self-sustaining business, under strong usa branding, it was, in reality, detached from the rwa organizational strengths and influences. little attempt to integrate and align the telecentre with the rwa development and business objectives was visible. consequently, insignificant guidance on such issues as pricing, the provision of services, customer care, opening hour’s etc was given. ominously, in its first rwa strategic planning workshop, in 2002, the telecentre was absent from the agenda. applying the due process model retrospectively, uncovers the fact-making trajectory and shows that effectively the rwa telecentre, as a candidate for admission into the rwa actor-network, was hurled from the moment of entrance practically into the fourth moment, institutionalization, leaping across the processes of perplexity, consultation and hierarchy. if the innovation is accepted into the institutionalized box without first passing through consultation, the innovation may be rejected or excluded from the network. the processes of perplexity, consultation and hierarchy are important stabilizers and valuable in order to saturate the candidate for inclusion with the values and norms required for the candidate to institutionalize within the actor-network. failure is defined here as an unstable network that is liable to destruction. this definition applied to the rwa telecentre candidature shows a major unintended consequence of this failure as the destabilization and perhaps destruction of the rwa actor-network. some conclusions preceding this section it is noted that there are no final narratives and no last instances. stability and social order, according to ant, are continually negotiated as a social process of aligning interests. this implies that the following conclusions can relate only to this specific analysis. subsequent events in the rwa actor-network will dictate further (altered) conclusions. successful development, implementation and management of technology requires attention to a number of multi faceted, multidisciplinary considerations including: planning philosophies, people, technology, organizational processes, products and services, financial constraints, environmental circumstances, structural change, behavioral change, and alignment of technological capabilities with the achievement of organizational goals and objectives. often a long and social process precedes the successful introduction of technology. the rwa telecentre was inserted into a complex socio-political context amongst a myriad of actors and actor-networks. when these actors clashed, the telecentre was an unwitting victim of the resulting conflict. the implementation of the telecentre barely paid lip service to the social components of the development process, such as gender, traditional authority and the implications of witchcraft accusations within a community that historically has been traumatized and destabilized by this phenomenon. the usa deterministically drove the rwa telecentre implementation, with little authentic consultation with the wider community. the usa operated in a business process mode to implement technology into a development context. business process mode tends to be linear, formal, with logical criterion to achieve the optimal outcome. whereas development is a complex, constrained, informal process with objectives that encompass social political and economic factors. this situation provides a vivid example of western rationalization and linear determinism meeting the chaos of development, where one actor’s (generally the more powerful) planning objectives are pushed through at the expense of the others. the usa belonged to another actor-network that was driving the process and as such was merely an implementer of government strategy. the strategy was formulated outside of the recipient actornetwork and aligned, primarily, with government strategy (the need to be seen to be alleviating poverty speedily and effectively), and not with consensual negotiated community based development objectives. a consequence of this was that the telecentre suffered from a trajectory failure. ‘…where the trajectory was visible after the fact, but subject to a considerable degree of australasian journal of information systems vol 11 no 2 may 2004 54 happenstance. our concern with projectory means we move beyond such passive chronicling to exert influence and steer the direction in which a project moves.’ (mcmaster, vidgen, wastell 1998: 353). the due process model can increase the probability of a successful projectory outcome, as it can integrate and align technology implementation with organisational strategy to exert influence and steer the direction in which the implementation trajects. in effect, the lack of attention to due process reduced the rwa role to that of telecentre custodian and lacking a purposefully directed trajectory, it became inoperable and unsupported. although the government invested substantial political capital into this remote rural telecentre program it did not manage to solve any development problems, conversely it created new problems and rather than being an answer it seemed to be a solution in search of a problem. this analysis provides further evidence that icts are technologies and they cannot solve political and social problems that are often the true roots of poverty. this analysis highlights the dangers related to the separation of strategy and implementation. roode (forthcoming) in his analysis of the south african ict sector development framework observes a notable flaw in the strategy development processes as the separation of strategy formulation and implementation. further he notes that ‘while the separation of strategy formulation and implementation does not have to lead to failure, it all too often does – even to default failure when implementation never starts, as if the implementers were almost scared by the sheer complexity of the blueprints provided by the strategists’. the rwa telecentre analysis exposes the vulnerability of failing to cohesively integrate technology implementation both with the social context and with the dynamics of the insitu actor-network. mcmaster, vidgen, wastell (1998:354) found that ‘…in is development, the tradition has been to separate social and technical issues and to apply different treatments. this means that the due process is not enacted and the likelihood of the network becoming aligned is reduced….’ whilst not a prerequisite for success, the due process model offers a framework that, when followed, may increase the likelihood of the local institutionalization of icts. attempts made by an experimental government body (the usa) to institutionalize the perplexing notion of icts before submitting them to the due process of perplexity, consultation and hierarchy proved a failure. the concomitant failed human expectations, loss of self-esteem and diminished social cohesiveness proved detrimental to the socio-political development process and contributed to the partial destruction of a hitherto successful women’s development organization. further research the rwa telecentre case reaffirms that the diffusion of icts and the increased emphasis on information activities, sometimes advocated in current development discourse, do not deterministically lead to socio-economic growth and organizational change. intentionally or not, the usa telecentre programme was deterministic and failed to take cognizance of the prevalent preexisting socio-political context. perhaps other routes to ict implementation outside of the community owned telecentre paradigm need to be explored. avgerou (1998) notes that at the level of the business firm, it is well accepted that the most significant benefits accruing from information technology (it) are often consequences of the technology supporting organizational change. avgerou’s research further suggests that it systems do not deliver sizeable benefits, unless part of an effort to introduce wider changes in the organization. at the community development level gurstein (1999, 2000) agrees with avegerou noting that whilst it is useful in overcoming distance sensitivity, the presence of electronic resources alone will not meet a community's needs unless projects promote the use of local information systems. the inference drawn from this preceding argument, and in considering the rwa telecentre experience, is that successful ict introduction is multidimensional. the socio-political context, changes in organisational structures, in work processes and in products, services and markets can all be determinants of ict acceptance, none of which were meaningfully explored during the telecentre implementation. whilst there is a fair level of consensus on the need to embed telecentres and icts into the economic well being of rural communities, there is little theory-based “how to’ information to guide this. australasian journal of information systems vol 11 no 2 may 2004 55 forthcoming research by the author aims to provide information to bridge this gap by taking a bottom up and theory-based approach to building a conceptual model to embed icts within a specific business process and organisation type. using ethnographic style action research, the project attempts to integrate a business process knowledge gap (marketing) with an information knowledge gap (market information) in a rural gender development context (the rwa), using e-commerce concepts. previous research (rhodes 2003) identified marketing as a deficient and required business process within the rwa. a conceptual framework (rhodes 2002) was used to guide the data collection during the action research project. secondly, the framework was used to expose the rwa operational and strategic decision-makers (the management team) to the concepts of marketing and e-commerce. thirdly, it acted, as a sense-making device to describe and capture the managers’ mental maps of the situational perceived interplay, alignment and relevance of marketing and e-commerce concepts for a rural african gender development organisation. the data was overlaid onto the conceptual framework to develop a bottom up, conceptual, community based business model. references avgerou, c. (1998). how can it enable economic growth in developing countries? information technology for development, 8 (1):15-29 baark, e., heeks r. (1998). development informatics evaluation of donor-funded information technology transfer projects in china: a life-cycle approach. development informatics working paper series. working paper no. 1. retrieved jan 8 2002 from http://www.man.ac.uk/idpm benjamin, p. (2000). african experiences with telecentres on the internet.retrieved june 10, 2003 from www.isoc.org/oti. benjamin (2001). telecentres and universal capability. a study of the telecentre programme of the universal service agency in south africa, 1996 – 2000. phd candidate aalborg university. sweden callon, m. (1999). the market test in actor network theory and after. in j. law and j. hassard (eds), actor network theory and after (pp.132-161) blackwell, oxford callon, m. (1991). techno-economic networks and irreversibility in: j. law (ed) a sociology of monsters? essays on power, technology and domination: 132-161. sociological review monograph 38. routledge : london. callon, m., law. j. (1986). mapping the dynamics of science and technology. macmillian press london. chapman, r., slaymaker, t. (2002). icts and rural development: review of the literature, current interventions and opportunities for action. working paper 192 results of odi research presented in preliminary form for discussion and critical comment overseas development institute. commission of gender equality (cge). 1999. the national conference on witchcraft violence. braamfontain, conference report cited in dirk kohnert j. of modern african studies, 41( 2): 1-29 dalsgaard, s. (2001). digital discourses: an essay on ict in development. retrieved march 4 2003 from http://www.comminit.com/st2003/sld-7425.html delius, p. (1996). a lion amongst the cattle – reconstruction and resistance in the northern transvaal. ravan press, johannesburg. gorman, m., heslop, a. (2002). poverty, policy, reciprocity and older people in the south. journal of international development, 14:1143–1151 gurstein, m. (1999). flexible networking, information and communications technology and local economic development. first monday. 4 (2). retrieved may 29 th 2001 from www.firstmonday.dk/issues/issue4_2/gurstein. gurstein, m, (2000). community informatics: enabling community uses of information and communications technology, boston, idea group publishing australasian journal of information systems vol 11 no 2 may 2004 56 heeks, r. (1999a). information and communication technologies, poverty and development, working paper no.5, institute for development policy and management. manchester university.uk heeks, r. (1999b). information icts and small enterprises. working paper no.7, institute for development policy and management. manchester university. uk kohnert, d. (2003). witchcraft and transnational social spaces: witchcraft violence, reconciliation and development in south africa’s transition process. journal of modern african studies, 41 (2): 1-29 laudeman,g .(n.d). digital development a framework for applying information technology to community socioeconomic development. retrieved february 26th 2003 at http://www.comminit.com/st2003/sld-7425.html latour, b. (1987). science in action. harvard universitypress. law, j., hassard, j. (1999). actor network theory and after. blackwell, oxford. macome, e, (2002). the dynamics of the adoption and use of ict-based initiatives for development: results of a field study in mozambique. ph.d. dissertation. faculty of engineering, built environment and information technology, university of pretoria, south africa. madon, s. (2000). the internet and socio-economic development: exploring the interaction, information technology & people, 13 (2): 85-101. mcmaster, t., vidgen, r.t. and wastell, d.g. (1998). networks of association and due process in is development, ifip wg 8.2 and 8.6 joint working conference on is: current and future changes, helsinki, finland, december 10-13/1998. morales-gomez, d.,melesse, m. (1998). utilising information and communication technologies for development: the social dimensions. information technology for development 8, (1):3-11 national telephone co-operative association (ntca) 2000, initial lessons learned about private sector participation in telecentre development, guide for policy makers in developing appropriate regulatory frameworks. ralushai, n.v. et al. (1996). report of the commission of inquiry into witchcraft violence and ritual murders in the northern province of the republic of south africa. unpublished cited in dirk kohnert j. of modern african studies, 41(2): 1-29 rhodes, j. (2003) can e-commerce enable marketing in an african rural women's community based development organization? informing science, special series on community informatics, 6 (1): 157-172. rhodes, j. (2002) can e-commerce build community assets to empower the trading activities of rural women? perspectives on global development and technology 1 ( 3-4 ): 269-293. rhodes, j. (2000) an internet based model for building a local commodity/trade exchange for a rural women's organisation. proceedings of the second international get smart conference using community informatics for regional transformation rockhampton, central queensland, australia. december 2000: 68-88. roode, d. ( forthcoming) a framework for achieving sustainable development through ict interventions. skuse, a. (2001). information communication technologies, poverty and empowerment. dissemination note no.3 . department for international development. retrieved march 15 th 2003 from www.dfid.gov.uk. walsham, g. (2001). making a world of difference: it in a global context, john willey and sons, west sussex, england. undp human development report (2001). chapter 2 today’s technological transformations creating the network age. oxford university press uk. the world bank development report on poverty (2000). retrieved january 20 th 2001, from http://www.worldbank.org/poverty/wdrpoverty/report/. the world bank human development report on poverty (2001). retrieved august 15th 2001, from http://www.worldbank.org/poverty. ajis vol. 8 no. 1 september 2000 editorial this is the first issue of volume 8 of the australian journal of information systems. as editor i am pleased to report the continuing interest in the journal both in australia as well as overseas. our first paper 'an empirical analysis of the effect of criticality, complexity and organisational influence on software reliability', is presented by paul bowen, jon heales & anne speed, department of commerce, university of queensland. this paper reports on a study examining whether reliability is affected by criticality or complexity and how the organisation deals with these issues. donald chand, bentley college, waltham, usa presents our next paper 'incorporating database design in warmer method'. this paper addresses the issue of incorporating into the warnier method, additional steps to transform data into normalised relations. quan dang, school of informatics, university of north london, presents our next paper, 'enhancing the link of soft systems and information systems models through a uniform and formalised representation'. this paper presents an approach based on meta modelling which not only allows ssm & is modelling, but also represents the relationships between the concepts in a meta-model. our next paper is presented by klement fellner & klaus turowski, business information systems, university of magdeburg, and is entitled ' framework architecture enabling an agent-based inter company integration with xml. the paper describes a framework for inter company integration which helps to automate procurement processes. guntram graef & martin gaedke, university of karlsruhe present our next paper 'evolution of web-based applications using domain-specific markup languages. this paper suggests the adoption of a paradigm based on component-technology and component-software architecture when developing web-based applications. alison parkes, department of information systems, massey university presents our next paper, 'is a picture worth a thousand words? the use of non-text items in information systems research'. this paper examines the use of non-text items in information systems journals and shows that articles, which use non-text items, are more often cited than those that don't. les worrall, university of wolverhampton, dan remenyi, brunei university & arthur money, henley management college present our next paper 'a methodology for evaluating user perceptions of the delivery of ict services: a comparative study of six uk local authorities'. the paper used both gap analysis and perceptual mapping techniques to develop an understanding of the aspects of ict service delivery. our next paper, entitled 'transitioning existing content: inferring organisation-specific documents is presented by arijit sengupta, georgia state university & sandeep purao, agder university, norway. this paper proposes a method of inferring organisation-specific document structures, which involves artificial intelligence and natural language processing techniques. our last three papers have been chosen as the best papers from the recent efis 2000 conference in dublin. mark roantree provides a brief overview of the conference and the papers. uwe hohenstein & andreas ebert, siemens ag, germany present the first of the three papers from efis 2000. it is entitled 'plugging files in database federations. the paper presents a specification-based approach to support the migrating of file data into relational databases. kai-uwe sattler, department of computer science, university of magdeburg, stefan conrad, department of computer science, university of munich & gunter saake, department of computer science, university of magdeburg present our next paper entitled 'adding conflict resolution features to a query language for database federations. the paper presents sql extensions for defining and querying database federations that address the resolution conflicts by providing mechanisms for mapping attributes. willem-jan van den heuvel, wilhelm hasselbring & mike papazoglou, department of information management, tilburg university, present our final paper. the paper is entitled 'top-down enterprise application integration with reference models'. the paper presents a methodology that uses reference models as a starting point for a topdown integration of enterprise applications. a book review of 'the new sociotech: graffiti on the long wall' by elayne coakes, dianne willis & raymond lloyd-jones is presented by lejla vrazalic, department of information systems, university of wollongong. once again i would like to pass my thanks on to the authors and consulting editors for the high quality of material in this issue. i would like to thank lily soh for her continued assistance with the journal and diniz da rocha for the maintenance of the home page. rob macgregor editor http://www.uow.edu.au/ajis/ajis.html microsoft word ajisfinal 12 2 marina.doc ajis volume 12 no. 2 may 2005 77 editorial for the following five articles the first international conference on qualitative research in it & it in qualitative research (qualt2004) was held at the tranquil nathan campus of griffith university in brisbane on 24-26 th november 2004. the conference attracted 85 local and international guests from a variety of disciplines providing a forum for the presentation of 42 research papers, 3 keynote addresses from world-renowned speakers and 4 topical panel sessions. keynote speakers professor michael myers, professor eileen trauth, and dr tom richards were a major contribution to the success of the event with interesting presentations on the current state of information systems research, the challenges in the use of software tools for qualitative inquiry, and understanding the impact that qualitative software tools can have on qualitative research. in regard to the papers published here, the program committee had a very difficult task to choose only five of the many interesting and well-presented research papers that were accepted for the conference. we decided to choose papers that reflected the great variety of it research projects and introduced innovative ways of applying qualitative methods. gillian whitehouse and chris diamond’s paper “hybrids and the gendering of computing jobs in australia” is a well-articulated report on research that involves a large survey and qualitative analysis of interviews. the paper demonstrates the richness of applying mixed methodologies. their paper addresses the problems of labour statistics and is a significant contribution to the research on it workforce management. moreover, this paper outlines the concept of is graduates being the go-between from management and it departments, giving the message of new and exciting work opportunities for is graduates. the paper “grounded theory and user requirements: a challenge for qualitative research” by supriya singh, kylie cassar bartolo, and christine satchell provides a very clear and detailed description of how grounded theory can be combined with a user-centred analysis to design a system based on smart internet technologies. the paper is based on the research group’s work within the smart internet technology cooperative research centre. the authors also provide helpful guidelines to other researchers looking for ways to employ qualitative analysis in design science projects. jennifer patashnick and michael rich’s paper “researching human experience: video intervention/prevention assessment” discusses how video recording serves as a method of data collection in exploring human experience with chronic illness and how this data can be analysed using grounded theory and qualitative analysis. the program committee believes that information systems researchers who are considering using video recording in their projects will benefit from the paper’s detailed discussion of the issues associated with analysing video data. the paper “painting a rich picture: the use of qualitative and quantitative methods to study social learning at the australian defence organisation” by leoni warne, celina pascoe, and irena ali is an interesting discussion of methodological choices in the australian defence organisation and how the choice of qualitative research methods can be justified. this paper also provides helpful advice to other researchers facing the problem of justifying qualitative analysis in an environment where the concept is not widely accepted. bruce rowlands’ paper “grounded theorising applied to is research – developing a coding strategy” explains the issues of coding associated with the analysis of data on the use of systems development methodologies. not only does this paper describe the theoretical justification of coding, but it also discusses its application to research data collected in a large australian bank. the feedback from participants reflected the success of the conference: “thank you so much for a wonderful, stimulating and enjoyable conference. i gained so much from the papers, the discussions and the networking”, “again, congrats, it was a very successful and thoroughly ajis volume 12 no. 2 may 2005 78 enjoyable conference” and “i am looking forward to the next one!” more information on qualit 2004 and the future qualit conferences can be found at http://www.griffith.edu.au/conference/qualit2004/. liisa von hellens and jenine beekhuyzen on behalf of qualit 2004 program committee microsoft word ajisvol10 1_final_edit.doc ajis vol. 10 no. 1 september 2002 73 information and communications technology in asia pacific countries: implications for small and medium enterprises in malaysia tim beal, ph.d school of marketing and international business victoria university of wellington, new zealand tel: +64 4 463 5080; fax: +64 4 463 5231 e-mail: tim.beal@vuw.ac.nz associate professor moha asri abdullah, ph.d division of geography universiti sains malaysia 11800 penang, malaysia tel: 604-6577888, fax: 604-6563707 e-mail: moha@usm.my this paper is prepared for the second international conference on smes in a global economy: sustaining sme innovation, competitiveness and development in the global economy, university of wollongong, wollongong, australia, 12-13 july 2002. abstract information communications and technology (ict) is transforming the environment in which smes operate and changing the paradigms of value creation, customer satisfaction and business effectiveness. this paper attempts to explore global and asia pacific ict usage and the major factors influencing smes. it focuses particularly on malaysia where the development of ict usage amongst smes is one of the main policy items in the agenda of the malaysian government. it describes a survey of smes in north and eastern peninsula malaysia 1999-2000. introduction many refer to information and communications technology (ict) as another revolution, which is a driving force transforming economic, business, commercial activities and socio-political changes in a borderless world. ict covers hardware, software, telecommunications, digital and database management and other information processing techniques/technologies used in computerand/or electronic-based information systems. ict is also described as the materials, digital resources, virtual and psychical data or technical knowledge that are processed in a meaningful way to a potential user and that posses values to affect a potential user’s future behavior or decision (davis and olson 1985). nowadays, the words, phrases and acronyms such as ict, internet, ecommerce, k-economy and knowledge-based information are in wide currency as the importance of information as a business resource is increasingly accepted. while the words are often used interchangeably, many consider that information technology and the internet are sub-sets of ict and that e-commerce is a business utilization of ict (see for instance, moha asri 2001 and beal 2000). however, the definition of e-commerce is vague; some confine it to cases where there is electronic payment (williams 1999:12) others use it more broadly (williams 1999:1). in this paper the broad definition is used: e-commerce is the utilization of ict for business purposes. there are dimensions of this phenomenon that can be further presented in view of accelerating process of trade, investment, finance and, more significantly, information technology across national boundaries. it is seen as reaching at a point where a small ticking drop anywhere in the globe, may and can trigger a stock crash, a recession, or a massive movement of capital and foreign exchange across the world. it is within this framework that globalisation and information is not simply confined to trade, investment and financial flows. it is now also extended to the flows of technology, businesses, services, ideas and persons. at the same time, it has been increasingly recognized that the development and prosperity of small and medium enterprises (smes) within this global information and communication technology environment is becoming ever more important and more vital to any economy, not least to the economies in the asia pacific countries. smes will increasingly have to confront a number of issues and challenges rising from global information and communication technology. it is reported that the growth of internet use by american small businesses has been inexorable and the small business administration has predicted that 85 percent of them will be conducting business ‘over the internet’ by 2002 (williams 2000:2 ). the trend seems to be even more fundamental than the many advantages and opportunities ict open ups for smes, such as expanding their geographical market (lituchy and rail 2000, mc donagh and prothero 2000). many have expressed confidence that smes would be dramatically transformed and would fast adapt to meet the new situation. they argue that smes and their activities will be more global, highly dependent on information and knowledge-based activities and the internet in the overall so-called “digital era of information and communication technology”. nonetheless, some pessimistic views are expressed since many smes are hampered by their low capital ratio and low productivity ajis vol. 10 no. 1 september 2002 74 which results in the slowness of traditional smes to utilize ict. global information and communication technology development provides great challenges for smes in a number of aspects concerning their survival, changes, prosperity as well as policies and strategies. this paper attempts to address the development of ict in asian pacific countries and its implications for smes with special reference to the case of malaysia. ict in asia pacific region the development of global ict, particularly the internet, has been extremely swift and e-commerce is rapidly emerging as a key component of business practice, certainly in the united states. in 1998, for instance, the retail income from internet in the united states was us$8 billion but increased to us$18.6 billion in 1999. it was expected to reach us$80 billion by the year 2003 (beal 2001). it was also recorded that while one quarter of the enterprises in europe were involved in e-commerce activities in 1999, more than two-thirds were by 2000. electricnews.net has estimated that european b2b e-commerce will grow from $500 billion in 2002 to $2,300 billion in 2005 while gartner forecasts that global e-commerce revenues will increase from $1,930 billion to $8,530 billion over the same period (nua2000b). moreover, t is widely acknowledged that e-commerce is driving massive restructuring of information-intense and service-oriented industries. these include entertainment, banking, travel, advertising, gaming, stock trading and utilities. one of the most significant applications of ict nowadays is the widespread use of internet for commerce and business activities even in traditional industries. this process is accelerated in industries where the product or service is largely digital, and new forms of products and services are being generated, and this provides entirely new ways of delivering value to the customer. often businesses or enterprises that are relatively consumer-focused will want to bypass intermediaries and sell directly to customers. e-commerce has the least impact when physical goods are delivered and the only major task is to gather or reconcile the flow of goods and information. moreover, internet and specifically intranets are becoming the core backbone for the emerging electronic workplace. in many countries in the asia pacific the internet has moved to the forefront of popular culture and seized public consciousness. south korea is an interesting example of pro-active government policy aimed at developing ecommerce and ict usage (mocie 2000). the government has put forward a “e-commerce roadmap” (kt 2001a and kim deok-hyon 2001a) in order “to lay foundation for e-korea, knowledge-based society (kh 2001a). in addition, president kim dae-jung pledged 10 trillion won (us$7.8 billion) in august 2001 to support r&d in key next generation sectors such as ict, biotechnology, nanotechnology, environment technology and cultural technology. the aim is to make korea a leader in ict in asia (kim mi-hui 2001). ajis vol. 10 no. 1 september 2002 75 figure 1: trend of internet worldwide (1996-2002) 0 100 200 300 400 500 600 jan-96 jan-97 jan-98 jan-99 jan-00 jan-01 jan-02 m il li o n s o n li n e january 1996 30 million february 2002 542 million internet market has grown 20 fold in six years soirce: nua surveys downloaded 18 march 2002 source: http://www.nua.ie/surveys/how_many_online/world.html downloaded 18 march 2002 meanwhile, based upon surveys compiled by the irish company nua (figure 1), the global internet market – the number of people connected to the internet – has grown 20-fold over the last six years, from 30 million at the beginning of 1996 to 542 million in february 2002. although there are methodological and definitional issues (beal 2001) it is clear that the growth has been explosive. table 1 distribution of internet users by region (2002) number (million) share (%) world total 544.20 100.0 africa 4.15 0.8 asia/pacific 157.49 28.9 europe 171.35 31.5 middle east 4.65 0.9 canada & usa 181.23 33.3 latin america 25.33 4.7 source: nua<http://www.nua.ie/surveys/how_many_online/>; downloaded 18 march 2002 table 1 shows that this internet market is currently divided almost equally between asia/pacific, europe and north america, with only a small part elsewhere; africa has just 0.8% of global internet users. in the early days the united states had the lion’s share of internet connections but that dominance is fading although it is still has by far the largest number (figure 2). within the asia/pacific itself (which by this definition includes australia and new zealand) there is considerable variation in the number of users, connectivity rate and growth. table 2 shows online numbers and connectivity rate (the percentage of the population connected to the internet) for the major asia/pacific economies in 1997 and 2002. date for the world and for the united states is included for comparison. according to this table, which utilizes the latest comprehensive data available, japan still leads the region in terms of numbers, though not in connectivity. however, a recent report claims that china has leapt to the lead with “56.6 million households with access to the internet”, some 5.5% of the population (nua 2002a). whatever the precise number of chinese connected to the internet may be it is clear that china, and more broadly the chinese-language area (i.e. including hong kong and taiwan), will be the major internet market in the asia pacific if not now, then fairly soon. moreover, just has china has overtaken the united states as a mobile phone market it a relatively short space of time it will also become the largest internet market in the world. the implications of that for business, including smes, is profound. ajis vol. 10 no. 1 september 2002 76 figure 2: asia pacific catches up with us in share of world internet population, 1997-2002 0 10 20 30 40 50 60 70 80 90 1997 2002 % o f w o rl d i n te rn e t p o p u la ti o n asia/pacific us sour ce: source: adopted from nua, downloaded on 19 march 2002 <(http://www.nua.com/surveys/how_online/index.html> in terms of connectivity rate the salient points from table 2 are • connectivity is highest in the more developed economies – australia, new zealand, the ‘city states’ of singapore and hong kong sar, taiwan and south korea. • the connectivity rates in these economies is at advanced world levels and matches that of the united states • japan’s rate is lower than might be expected from her general level of economic development and technology leadership • the larger, and less developed counties, such as china, indonesia, philippines and thailand have broadly similar national rates, about 2% to 5%. however, these national averages mask considerable variation and urban rates can be quite high. coastal urban china, in particular, could be considered as a separate, medium developed, internet market • malaysia occupies a medium position, but will probably achieve advanced internet connectivity rate, say 50%+, in a couple of years. ajis vol. 10 no. 1 september 2002 77 table 2: internet connectivity in asia pacific countries 1997-2002 area number (millions) connectivity rate (% of population) 1997 2002 1997 2002 world 70.0 544.2 1.7 9.0 australia 1.2 10.6 6.7 54.4 china 0.2 33.7 0.0 2.7 hong kong 0.5 4.3 7.9 59.0 indonesia 0.1 4.4 1.9 japan 8.0 49.7 6.3 39.2 malaysia 0.1 3.7 0.7 17.0 new zealand 0.3 2.0 9.1 49.9 philippines 0.1 2.0 2.5 singapore 0.5 2.3 14.7 50.8 south korea 0.7 22.2 1.5 46.4 taiwan 1.7 11.6 7.7 51.9 thailand 0.1 1.2 2.0 total a/p 13.5 147.7 na na us 56.0 164.1 21.0 58.5 share of world internet population asia/pacific 19.3 27.1 na na us 80.0 30.2 na na source: adopted from nua, downloaded on 19 march 2002 <(http://www.nua.com/surveys/how_online/index.html> notes: dates are approximate and refer to the surveys nearest to the beginning of 1997 and to february 2002. ‘asia/pacific refers to the sum of the specified countries and so differs slightly from the definition in table 1 the role of government ict era is marked with the use of the internet by government and its agencies to conduct its internal business, to communicate with citizens and the world at large. of particular relevance in this context is the government’s use of ict to interact with business as a purchaser and supplier of goods and services and as a source of regulation and facilitation. governments are also moving towards the electronic collection of customs data and duty, and taxes a survey conducted by world market research centre 2001 found that asia pacific countries such as australia, singapore, taiwan are among the top countries in ‘e-government’ (see table 3). malaysia, though ranked 16 overall, was fourth in the region. ajis vol. 10 no. 1 september 2002 78 table 3: asia pacific countries in global e-government rankings rank country % 1 us 57.2 2 taiwan 52.5 3 australia 50.7 8 singapore 43.4 16 malaysia 39.0 26 new zealand 36.8 38 japan 34.9 47 south korea 33.4 52 philippines 32.8 53 vietnam 32.8 56 brunei 32.7 71 thailand 30.8 83 china 30.2 87 lao pdr 30.0 88 indonesia 30.0 91 cambodia 29.6 119 myanmar 26.8 note: hong kong is not included in this survey source: world markets research centre (wmrc) 2001 http://www.worldmarketsanalysis.com/pdf/e-govreport.pdf infrastructure developments infrastructure market, especially as regards bandwidth, is still experiencing high growth and looks set to continue to grow. the networking equipment market consists of four main categories networking communications, voice communications, personal communications, and public network communications. a robust outlook for the telecommunications networking equipment market worldwide which is growing at an rate of 8.3 percent per annum is expected to reach us$400 billion by the year 2003. by 2002, the world personal communications market is the fastest growing item which is predicted to grow to approximately us$59 billion (gartner 1999). traditional public telephone infrastructure and new public data infrastructure is expected to increase by 8.8 percent per annum amounting to us$156 billion in the same year (see table 4). the report emphasizes that during the next five years, telecommunications monopolies will steadily move towards oligopolies, managed competition and open competition much of which will be driven by global and multinational enterprises. meanwhile, trends are moving towards more broadband networks which will dominate the communications scene. these include integrated broadband capabilities such as real-time video streaming and conferencing and real-time digital audio and voice. ajis vol. 10 no. 1 september 2002 79 table 4: distribution of global spending projection on equipment network type of equipment spending forecast (us$ billion) 1997 2002 public wireless network private data network private voice network public voice network public data network other network equipment 75 23 48 76 10 17 156 59 61 69 15 49 source: ministry of entrepreneurs development 1999, report on global it usage (unpublished): kuala lumpur the expansion of the global ict market is driven by both growth in ict usage, as described above, and the development of new ict services. these include ict service-providers which are now competitively developing innovative packages and many of them are beginning to penetrate the sme market. in a survey carried out by information week research, 200 ict managers of large and small companies in the usa spend an average of 20 percent of their annual ict budgets on ict services. in order to match customer demand for support services as well as to be innovative in creating a new mechanism for product sales, businesses, enterprises and vendors have been involved in increasing mergers and acquisitions. as the global trend of ict is rapidly growing, competition is reaching new heights. as a result many regard ict no longer merely as a tool of process efficiency and effectiveness, but as a determinant of survival and prosperity. in this connection, ict decisions will be driven by the ability of vendors and information system organizations to transform cost improvements in basic technologies to business value-added tools. likewise, the global trend in ict is driving the primary focus of ict investment towards value creation and business effectiveness, where a new form of dynamic enterprise will appear that uses its knowledge assets to innovate rapidly and seize market opportunities. increased consumer access to information has also driven enterprises to focus on core competencies by which they excel and offer a broader range of products and services. recent high turnover in the ict labour market provides opportunities for enterprises and businesses to employ more competent and qualified ict professionals than before. to compete effectively, enterprises have to invest time and efforts to attract these professionals into their firms. a report noted that the annual turnover of ict professionals in the usa and europe have traditionally been from 6 percent to 10 percent and from 2 percent to 5 percent respectively. however, in the past two years, turnover has risen to an alarming rate of 15 percent and this trend is not expected to change for the next four to five years (ministry of entrepreneurs development 2000). last but not least, the development of networking computing activities also plays its role in the use of ict that will steadily become the new platform for ict application. network computing is an evolved client/server application architecture with dynamic application deployment, execution and management. network computing can drive down total cost of ownership in some cases, but it will be the business benefits, not the cost implications, that will drive network computing investments. ultimately, network computing enterprises will spend more on ict. without network computing, however, enterprises will still spend heavily on ict but achieve less in the long run for every dollar. this means that network computing is a reduction in the incremental cost of expanding capabilities. nonetheless, network computing practices are still at an early stage, even though it is an emerging market. some businesses and enterprises around the globe have already investing quite considerably in network computing technology, even though this technology is still in an early stage. this investment is expected to grow in the next few years as enterprises and businesses seek to achieve a strategic business advantage out of the current trends that will directly benefit their future revenue statistics. indeed, the mentioned factors will have multiplier effects on businesses and enterprises and surely the ict applications will be growing up to a point when it will suggest a note for possible caution. after this point, real use of the ict in terms of effective e-commerce will not be automatically grow. the role of smes smes have a very important role in bringing about the benefits of ict. just as there is a ‘digital divide’ amongst the population at large, so in the business world there is a divide between those firms that are firmly based on ict utilization, and have high productivity, and those who don’t have ict, or whilst they may posses ict do not exploit it properly (the united states department of commerce, 2000a). there is a further dimension to the status of smes in the era of e-commerce. almost all e-commerce firms start off with few employees, even the most successful enterprises stay small in terms of employees. therefore, many believe that within the ict era, enterprises, including smes, tend to be small compared with the past, because there is a tendency downsize their operations and outsource many functions to other firms in a network of partnerships and alliances. in the usa, ajis vol. 10 no. 1 september 2002 80 the importance of smes as an incubator of ideas and innovation is widely recognised (hlava 1999). due to this recognition, big businesses which normally use more ict, frequently have an incentive to develop relationships with smes and in doing so upgrade their ict capacity. table 5: sme’s diffusion rate of ict tools in japan it tools rate of use (percent) personal computer e-mail website network among firms via internet cad/cam (in manufacturing) 60 60 40 20 10 source: meti 2001 ict usage amongst asian smes varies widely. the singapore government provides a number of incentives and infrastructure development to upgrade training and technical capabilities in upgrading the use of ict among smes. recent figures in japan shows that smes are far from utilizing ict fully – only 60% in a ministry of economy, trade and industry survey used pcs (table 5). on the other hand, although smes in japan generally recognized the need to participate in the ict revolution they encountered problems ranging from lack of ict skills among staff to a lack of knowledge by management of how to utilize ict (fig 3). if smes in japan find utilizing ict fraught with difficulties and uncertainties, what of smes in less developed parts of the region such as malaysia? fig 3: problems encountered by japanese smes introducing ict 0 5 10 15 20 25 30 35 40 45 improvement of employees it literacy secure of it specialities secure of cost for maintenance, etc. security no idea of good it investment no consultant suspicion for effect of it investment other no problems % of responses invested no investment source: meti 2001:fig 1 development of ict industry in malaysia the development of ict usage in smes is very much influenced by the overall direction of the ict industry in malaysia as well as the advancement of global ict trends. this is despite the fact that individual enterprises or entrepreneurs, skilled employees, size and resource of individual smes, vendors’ support services, type of ict and type of industry are equally noted to be important in influencing smes to utilize ict (see moha asri 2001). structurally, malaysia is intensifying her efforts to build a strong foundation for the ict industry to achieve vision 2020. although the recent economic crisis has slowed down the growth of ict industry, growth is still expected albeit at a slower pace since ict industry revenue are closely linked to malaysian gross domestic product (gdp) trends. ict industry revenue growth recorded a compounded annual growth rate of 22 percent from 1993 to 1997 and is projected to grow about 6.9 percent between 1998 and 2000. the ict industry’s importance can be seen by the increase in its contribution towards gdp. on average, ict sector has contributed about 2.8 percent to the gdp between 1994 and 1999. ict industry’s share of gdp has increased from 2 percent to 3.4 percent between 1994 and 1997, and 5 percent in 1999. recent trend indicates that ict industry is ajis vol. 10 no. 1 september 2002 81 expected to outpace the gdp growth in the next three years, especially in response to the recovery of the malaysian economy in 1999. in view of its strategic importance, the overall ict industry growth is projected to increase rapidly from 10.8 percent 1999 to 13.9 percent 2000 and 2001 (ministry of finance, 2000). in its efforts to build a strong foundation for the ict industry, the government of malaysia launched a mega project known as the “multimedia super corridor” (msc) in 1995. this project provides impetus to the growth of the ict industry and the country’s economy through a number of benefits, including; development of intelligent cities with ict and communications infrastructure such as cyberjaya and putrajaya areas; tax exemption for msc status companies for 10 years or a 100 percent investment tax allowance, and unrestricted employment of knowledge workers. in addition, the national information technology agenda was prepared as a framework in the development of three strategic elements of human resource, infrastructure and ict-based application in 1996. incentives such as removal of sales tax on computer and components as well as granting of accelerated capital allowance for expenses on computers and other ict equipments were given to encourage the usage of ict. in order to enhance the nation’s move towards ict strong foundation, some common laws have also been passed, in line with ict potential that will be explored in order to reach to the fullest possible levels. these acts, among others, include digital signature act 1997; copyright (amendment) act 1997; computer crime act 1997; telemedicine act 1997, and; communications and multimedia act 1998 (see ministry of international trade and industry 1996). although the ict industry is growing fast in line with the government efforts, the industry is also hindered by the shortage of professionals and skilled human resource. during a period of 1996 –2000, it was estimated that the country’s shortage of ict staffs reached 7,063, i.e. about 75 percent of the demand were met, as stated in the seventh malaysia plan 1996-2000 (malaysia 1996). this means that the surge in demand for ict skilled workers and professionals is exerting on the supply of ict personnel especially in the area of networking and internet. ideally, high demand for ict skills has prompted many companies to offer increasing pay to attract these personnel. as many other countries worldwide, competition for top ict talents and entrepreneurs becomes tough. elsewhere, the united states remains the top destination for her leading edge technology and favorable conditions. singapore, just across the border, has always been a competitor in attracting malaysia’s talents. besides the shortage of human resource, another key challenge faced by the ict industry is the lack of entrepreneurial talent in ict. based on a study recently, lack of entrepreneurial ict talents impedes the development of innovative world class technology and products (ministry of entrepreneurs development 1999). this is especially so for entrepreneurs in smes. to date, numerous efforts have been implemented to promote ict applications in smes. a rm20 million fund for smes to participate in electronic commerce known as the electronic commerce grant scheme was, for instance, launched in 1999, as well as ict entrepreneurial development training, infrastructure supports, technical and advisory services etc. the aim is to enable smes to integrate themselves into the mainstream of e-commerce, communications, and information technology as well as helping them to find a place to survive in the global marketplace (the star july 2000). this is on top of the malaysian government efforts to develop a “mega” project such as multimedia super corridor (msc), to act a proxy to expand participation of local smes into the information technology and advanced techniques in their operation, production, marketing and distribution system. the efforts are also initiated in order to establish wider business networks between government and smes, smes and mncs, smes and smes, locally and globally. implications of ict usage in malaysia’s smes the study of ict in malaysia is still relatively new and does not have a long research tradition. there is no agreement on appropriate research methods to be used. however, this research study uses two main method i.e. the secondary data and documents, and primary data. the first method i.e. the secondary data and documents is essential especially when it explores the literature to identify the important variables that are found by researchers to determine the use of ict in smes. it derives from a short review of literature, which indicates that there are a number of variables especially entrepreneurial characteristics, explaining the level of use of it in smes. the second method is primary data based on a survey. this survey study is derived from a case study on the use and adoption of it in smes in the eastern and northern regions of peninsular malaysia from june 1999 to april 2000. this means that it involves five states of kelantan, terengganu, perlis, kedah and penang. the criteria for defining smes in this study as an enterprises employing full-time workers of less than 200; and the amount of paid up capital less than rm2.5 million12. procedures used in data collection are divided into a few phases. at the outset, the names and addresses of smes were compiled from a number of governmental and non-governmental agencies in malaysia and from the five states in the two regions. those include small and medium industrial development corporation (smidec), federal malaysian manufactures (fmm), penang 12 this definition is widely used in malaysia by several agencies such as world bank 1986, unido (1982) and adb (1990), other than individual researchers such as saleh (1989), chee (1986), moha asri (1997, 1999 etc.) ajis vol. 10 no. 1 september 2002 82 development corporation (pdc), state economic development corporation of kedah, state economic development of perlis, state economic development of kelantan and state economic development of terengganu. a total of 1,512 smes were identified in those five states. compiling the list of smes in the study area was not a simple task since there was hardly any comprehensive source where the list of all smes could be obtained. from 1,512 smes in the compiled list, about 44 percent of them could not be traced when initial visits to their respective addresses were made (867 firms). two likely reasons for explaining these untraceable firms are; firstly, that they may have gone out of business (bankruptcies); and secondly they may have moved out from the study areas. moreover, 116 smes were well known as large companies and multi-national corporations. in addition, 262 smes in the study area refused to cooperate or to give interviews. therefore, only 665 were interviewed. the study was conducted in two phases i.e. a pilot study and a questionnaire survey. eventually, only 414 smes were available for a questionnaire survey, while in the pilot phase, 15 smes were randomly chosen to do the pre-test the questionnaire. smes in the survey produce a wide variety of products. however, the study categorised them into four main products i.e. electrical products, food, drink and medical products, and non-food oriented products (see moha asri at el. 2000). the size of smes in the samples is relatively small in terms of their number of employees, paid up capital as well as value of output. a total of 42.0 percent of smes are sole proprietor, while others are private limited firms and family ownership or/and friend-ventured firms. in terms of ict usage, out of 414 smes, only 7.2 percent have a high level of ict usage and 34.8 percent have used some ict as compared to 58.0 percent smes that have not used ict at all (see table 6). this data reflects that the level of ict usage among smes tend to be higher than entrepreneurs knowledge on ict. this may reflect that there are smes in the survey that use ict, even though their owner/entrepreneurs do not have ict knowledge. the research study also carried out a task to determine other factors contributed to ict usage in smes using the pearson correlation and multiple regression analysis. table 6: the overall knowledge of ict among smes’ entrepreneurs and the overall use of ict among smes. knowledge of it level of it usage knowledge level no. percent no. percent no knowledge some knowledge high knowledge 293 108 13 70.8 26.1 3.1 240 144 30 58.0 34.8 7.2 total 414 100.0 414 100.0 the correlation matrix is presented in tables 7 and 8. it is illustrated in two phases. firstly the correlation between all sub-variables of ict knowledge of entrepreneurs and the ict use as in table 7. secondly, the correlation between composited score of ict knowledge together with all other variables and the ict use as in table 8. table 8 shows that ict adoption is positively correlated with all sub variables in ict knowledge among entrepreneurs. these include knowledge of ict on operating system; microsoft office; lotus application; multimedia system; printing operation; computer hardware; web-page use; network configuration; and group application software with all at the significance level of 0.01. the findings suggest that entrepreneurs of smes are the key factor in determining the use of ict. only entrepreneurs who have more innovative would be realising and willing to improve their business efficiency and effectiveness, confirming earlier suggestion made by howell and higgins (1990). entrepreneurs are the person who have a direct concern over economic feasibility and are more likely to use ict if there is tangible economic justification in terms of costs and benefits derived. all the sub-variables in ict knowledge of entrepreneurs then composited into one-single score that was tested with the cronbach alpha of over 0.70 (i.e. 0.945) meaning that the validation is highly accepted. the variable is then analysed with the other independent variables to see their correlation as illustrated in table 8. it shows that ict usage is positively correlated with ict knowledge of entrepreneurs, education level of entrepreneurs, location of smes, budget allocated to ict and plan for ict expansion, number of ict workers and total value of output and paid up capital at the significance level of 0.01 except for the total paid up capital which has a significance level of 0.05. as ict knowledge of entrepreneurs is explained earlier, the level of education among the entrepreneurs which has positive correlation with ict usage means that the higher the level of education, the higher the level of ict use among smes. thus, it is plausible to assume that entrepreneurs with the higher educational level tend to realise more on the benefits and economic justification for the ict use in their firms. the finding shows that knowledge of ict among entrepreneurs is the most important factor influencing the use of ict, it is necessary to educate and expose entrepreneurs themselves with ict knowledge. location of smes is also positively correlated with smes. this correlation means that smes which situated in the industrial zone tend to use more ict than the ones which are situated in the illegal and backyard areas. there is a possibility that, smes which located their operation in the industrial site and light industrial zone relatively bigger in terms ajis vol. 10 no. 1 september 2002 83 of size and resource of operation. hence, they are more affordable and economically viable and justifiable to upgrade their business operations through the ict use as opposed to smes located in other sites 13. the analysis is further carried out using the multiple analysis which involves the use of the partial correlation coefficient (pr2). this is one of the major methods by which multiple analysis of relationship is calculated14. the partial coefficient of correlation (pr2) for the eleven independent variables that are use in the multiple regression analysis and their relative important for correlation with the dependent variable are shown in table 9 which illustrates that ict knowledge of entrepreneurs, their level of education and skilled professionals contributed much of it15. 13 however, age of entrepreneurs and age of smes is negatively correlated with ict use also at significance level of 0.01. this finding illustrates that smes with younger age appear to use ict more than the older age. this variable may be correlates to innovative entrepreneurs at early year of business involvement, who are more likely to try out new things and take risks (kirton 1984). in this relation, only entrepreneurs who are much younger and innovative would be willing to improve their business efficiency and effectiveness by doing thins differently. the younger the entrepreneur, the higher realisation towards using ict and more adoptive on ict use. more interestingly, ict use is highly correlated with ict knowledge of entrepreneurs with the biggest score of 0.587 as compared to other variables. 14 the test allows the research study to know the strength of the independent variable in relation to dependent variable when the linear effects of other independent variables have been removed or kept constant14. all eleven variables is determined by their partial coefficient of correlation (pr2). the dependent variable that has the highest absolute pr2 is considered to be the most important variable in explaining the ict use. a positive value for pr2 implies that the independent variable is positively related to the dependent variable and vice versa. 15 the table raises several issues. first, the average of pr2 variable is very convincing with significant level of 99 percent. this means that most of the variables analysed in the research study explain the most substantive ajis vol. 10 no. 1 september 2002 84 table 9: multiple regression analysis for the ict use in smes. dependent variable independent variable beta t significance level the use of it it knowledge of entrepreneur level of education number of it workers ownership status paid up capital age of entrepreneurs budget allocated for it value of output age of smes development plan for it .349 .226 .235 -.157 .071 .102 .078 .020 .047 .062 6.260 4.042 4.738 -3.467 1.672 1.915 1.843 0.450 -1.032 1.240 .000 .000 .000 .001 .096 .057 .066 .653 .303 .216 percent of variance explain (r2) 0.611 note: ***p< 0.001, **p< 0.01, *p< 0.05 part of the dependent variable. second, from eleven independent variables analysed, average score of ict knowledge of entrepreneurs is the most highly correlated to the ict use with the beta score of 0.349, followed by the number of ict workers (0.235), while level of education comes third (0.226). other than knowledge of entrepreneurs on ict, another variable seems to influence ict use is employees’ ict knowledge. smes that have workers who are more knowledgeable on ict are more likely to use more ict. the finding further validates the earlier statement by senn and gibson (1981) who admitted that lack of knowledge and insufficient awareness of the potential benefits of ict use among workers may also inhibit businesses from using ict. third, all these three variables have the significance level of p<0.001 or 99 percent. fourth, there are other variables which show the significance level of p<0.05 include, paid up capital, age of entrepreneurs, budget allocated for ict and plan for ict development. fifth, the total r2 variance is 0.611 which is extremely high meaning that variables surveyed in the research study explain a significant factor for the ict use among smes in malaysia. finally, some limitations of the research study are essential to note. as this research study was conducted in smes in the northern part of malaysia, the findings may not be appropriately generalizable to large enterprises or even smes in other countries with different stages of economic development and environment. the research study acknowledged that there may be other variables which, may be significant determinants of ict use by smes, but were not included in the selected variables. not withstanding these limitations, this research study has contributed a significant factor in understanding the use of ict in smes. table 7: pearson correlation matrix on sub-variables of ict knowledge of entrepreneurs in smes sub-variable of knowledge (1) (2) (3) (4) (5) (6) (7) (8) (9) (1) operating system 1.000 (2) microsoft office .8142** 1.000 (3) lotus application .6811** .6140** 1.000 (4) multimedia .6884** .7485** .6948** 1.000 (5) printing operation .6587** .7129** .6341** .8202 1.000 (6) computer hardware .5868** .6359** .6028** .6290 .6171** 1.000 (7) web-page use .6574** .6833** .6335** .6946 .6806** .7528** 1.000 (8) network configuration .5708** .5556** .5999** .6110 .5831** .7688** .7842** 1.000 (9) group application software .5454** .5699** .6761** .5733 .5316** .7439** .7368** .8082** 1.000 (10) group application software .4341** .4727* .5098** .5077 .4555** .5028** .5282** .4989** .5244** note: ***p< 0.001, **p< 0.01, *p< 0.05 table 8: pearson correlation matrix on all variables variable (1) (2) (3) (4) (5) (6) (7) (8) (9) (10) (11) (1) age of entrepreneur 1.000 (2) entrepreneur’s education -.5537** 1.000 (3) age of sme -.3770** .4506** 1.000 (4) value of output .4782 ** -.3273** -.2371** 1.000 (5) number of worker -.2917** .3865 ** .4165** -.0952 1.000 (6) value of paid up capital -.1735* .2097 ** .2238 ** -.0323 -.0046 1.000 (7) value of output -.2447** .3322** .3773** -.0759 .3810** .1449 ** 1.000 (8) overall it knowledge -.4562** .6089 ** .4716 ** -.2822** .3455 ** .2593 ** .3034 ** 1.000 (9) number of it workers -.0891 .0490 .1367 -.0485 .1557* -.0056 .1507* .1579* 1.000 (10) budget allocated to it -.1278 .1534 * .1056 -.1037 .1950 ** .1045 .1810* .3172 ** .0639 1.000 (11) plan for it expansion -.3312** .4134** .4079** -.2077** .4574** .1534* .1555* .5045** .1750* .2851** 1.000 (12) overall use of it -.2504** .4683** .4471** -.1943** .4022** .1864* .3234** .5868** .2291** .4200** .4024** note: ***p< 0.001, **p< 0.01, *p< 0.05 concluding remarks ict is rapidly emerging as a key component of business practice in asia pacific countries. while e-commerce is the driving force in restructuring of information-intense and service-oriented industries, the internet is moving to the forefront of popular culture and seized great public consciousness. many governments in the regions play a very pro-active role in developing e-commerce and ict usage. many of them, if not all are not only moving towards the electronic collection of customs data and duty and taxes, but more importantly ict is also being used as a source of regulation and facilitation as well as a purchaser and supplier of goods and services. despite the fact that malaysia occupies a medium position among the asia pacific countries with a greater prospects to achieve advanced internet connectivity rate in a couple of years, the government laid down a strong foundation for the ict industry to grow and expand with a launch of the “multimedia super corridor”. this project provides great emphasis on the development of ict and the country’s economy through a range of spin-off effects. after almost eight years in its implementation, the implications on the development of smes and specifically the effect on the usage of ict among malaysia’s smes, is hard to gauge. through a sample survey in the study, the findings indicate that out of 414 smes, only 7.2 percent have a high level of ict usage and 34.8 percent have used some ict as compared to 58.0 percent smes that have not used ict at all. the findings reflect that the actual level of ict usage among smes tends to be higher than entrepreneurs’ knowledge of ict. this may mean that there are smes in the survey that use ict, even though their owner/entrepreneurs do not have explicit knowledge of ict. in addition, smes that have entrepreneurs with ict knowledge, higher level of education, employees with ict knowledge, higher paid up capital, younger age of entrepreneurs, budget allocated for ict and plan for ict expansion are more likely to use ict. in view of the findings, an effective way to promote ict use is to cultivate greater ict knowledge and exposure to entrepreneurs especially concerning its benefits, quality, productivity and efficiency. it could be useful too to enhance ict knowledge to employees of smes to create a greater awareness among smes, in order to improve their overall attitude towards ict use. this indeed could be done in a number of way such as training programmes to entrepreneurs and employees; seminars and workshops; other than providing seek capital for smes to enable them to purchase and meet necessary requirement. moreover, information intensity with respect to various types of ict and their applications to business purposes are equally essential. therefore, providing sufficient information on the availability of ict products, suppliers or customers, match-making procedures between the business need and the softwares and hard wares, as well as operation system are also crucial. a specific agency dealing with these requirements and needs would be highly desirable. references beal, tim (2001), “patterns of internet development in the new economy: implications for asian smes”, in moha asri abdullah (ed.) small and medium enterprises in the information age: asian perspectives, wisdom house publications inc.; leeds, u.k. beal, tim, (2000), “smes and e-commerce in china: foreign paths, chinese steps", proceeding of international conference in the internet age, 21st-24th june 2000, beijing, china. daily yomiuri, may 3, (2001) (thursday) davis g. and oslon m. (1985), management information systems, macmillan: new york. fraza v. (1998) “making the investment”, industrial distribution 87(9 (59-62) gregory g. (2000), “globalisation of mall and medium enterprises (smes) in korea: a response to the asian financial crisis”, proceeding of smes in a global economy conference, 16th-17th june 2000, wollongong, australia. harrison d.d et. al. (1997), “executive decisions about adoption of information technology in small business: theory and empirical tests”, information system research, 8 (2) (171-195) hassan h. (2000), “cut-price it consultants for smes”, proceedings of the smes in a global conference, 16-17 june 2000, wollongong, australia hlava m.m.k. (1999), “expanding partnerships for smalland medium-sized enterprises”. bulletin of the american society for information science, 25(3) (7-9). ho c.f. (1996), “information technology implementations strategies for manufacturing organisations: a strategic alignment approach”, international journal of operations and production management, 16(7) (77-100) howard a., kochhar a. and diworth j. (1999), “functional requirement of manufacturing planning and control systems in medium-sized batch manufacturing companies”, integrated manufacturing systems, 10 (3) (113) ajis vol. 10 no. 1 september 2002 87 howell j.m. and higgins c.a. (1990), “champions of technological innovation”, administrative science quarterly, 35 (317-341) kim b.o. and lee s.m. (1996), “logistics information’s role within an it systems architecture in a world-class organization”, logistic information management, 9 (3) (19-26) kirton m.j. (1994), “adaptors and innovators-why new initiatives get blocked”, long range planning 17 (2) (137-143) malaysia (1996), seventh malaysia plan 1996-2000, government printer; kuala lumpur ministry of economy, trade and industry (meti 2001), small and medium enterprise agency of japan, white paper on small and medium enterprises in japan 2000, <http://www.chusho.meti.go.jp/english/wp2000/wp2000.html> ministry of entrepreneurs development (2000), study on bumiputera’s participation in the information technology industry, (unpublished and restricted report); kuala lumpur ministry of finance (2000), annual economic report 1999/2000, bank negara; kuala lumpur ministry of international trade and industry (1996), second industrial master plan 1996-2005: executive summary, kuala lumpur modie a.m (1999), “getting the right advice isn’t easy”, australia financial review, february 23, 1999 moha asri abdullah (1999), small and medium enterprises in malaysia: ashgate publishing ltd: london. moha asri abdullah (2000(a)), the use of information technology in small and medium enterprises, irpa short-term grant, universiti sains malaysia; penang. moha asri abdullah (2000(b)), "globalised economy and the strategic importance of smes in malaysia's industrial structure", international conference on smes in east asia in aftermath of the asian financial and economic crisis, 15th 17th june 2000, university of wollongong, australia moha asri abdullah (2001), asian small and medium enterprises: challenges in the 21st century, wisdom house inc; leeds: united kingdom. myers m. (1998), quantitative research in information system, isworld url; http://isworld.dis.unimelb.edu.au/isworld.html naumes w. (1978), the entrepreneurial manager in small business, addison-wesley publ. company: sydney newsfactor (2002) “report: at-home web population keeps growing”, newsfactor network, 9 may 2002, <http://www.newsfactor.com/perl/story/17684.html> nua (2002a) “reuters: china number two in home net usage”, nua, 22 april 2002, <http://www.nua.ie/surveys/index.cgi?f=vs&art_id=905357873&rel=true> nua (2002b), “electricnews: b2b ecommerce sales set to increase”, nua 9 may 2002, <http://www.nua.com/surveys/index.cgi?f=vs&art_id=905357936&rel=true> ohmae k. (1995), the end of the nation state: the rise of regional economics, the free press: new york. porter m. and miller v.e (1985), “how information gives you competitive advantage”, harvard business review 63 (4), 149-160 robert m. and wood m. (1997), “the use of computerised information systems to gain strategic competitive advantage: the case of start up manufacturing company”, logistics information management, 10 (4), (178-183) rothwell r. (1997), “the characteristics of successful innovators and technically progressive firms”, r&d management, 7 (3) (31-42) senn j.a. and gibson v.r. (1981), “risks of investment in microcomputer for small business management”, journal of small business management, july (24-32) the daily yomiuri, may 3, (2001) the star, july 5, (2000) williams, victoria (1999), e-commerce: small businesses venture online, us small business administration, <http://www.sba.gov/advo/stats/e_comm.pdf> williams, victoria (2000), small business expansions in electronic commerce, us small business administration, <http://www.sba.gov/advo/stats/e_comm2.pdf> yap c.s (1989), “computerization problems facing small and medium enterprises: the experience of singapore”, proceeding of the twentieth annual meeting of the midwest decision sciences institute, ohio; miami university (128-134) microsoft word ajisvol10 1_final_edit.doc ajis vol. 10 no. 1 september 2002 110 electronic commerce and market focus: some findings from a study of swedish small to medium enterprises lejla vrazalic department of is, university of wollongong, wollongong 2500, australia ph (61 2) 4221 3750 fax (61 2) 4221 4474 lejla@uow.edu.au deborah j bunker school of is, technology and management, university of new south wales, sydney 2052, australia ph (61 2) 9385 4255 fax (61 2) 9662 4061 d.bunker@unsw.edu.au robert c macgregor department of is, university of wollongong, wollongong 2500, australia ph (61 2) 4221 3758 fax (61 2) 4221 4474 robert_macgregor@uow.edu.au sten carlsson department of is, karlstad university, se-651 88 karlstad, sweden ph (46 54) 700 11 35 fax (46 54) 700 14 46 sten.carlsson@kau.se monika magnusson department of is, karlstad university, se-651 88 karlstad, sweden ph (46 54) 700 21 70 monika.magnusson@kau.se abstract in the past, organisations relied on traditional quantitative metrics, such as return on investment (roi) to make decisions when investing in technology. with the advent of electronic commerce (ec), these decisions are becoming less reliant on roi measures. instead different driving forces are taking precedence in the decision making process. this paper presents the findings of a study of 118 swedish small to medium enterprises (smes) that have adopted ec. the results of the study suggest that improvements to customer service, internal efficiency and organisational competitiveness have become equally important when making ec investment decisions. the study also examined whether major market focus (local, regional, national or international) had an influence on the organisation’s decision to adopt ec. findings indicate no associations between the market focus and the driving forces, however the existence of a fully developed plan for ec adoption and the size of the business were found to be highly associated with the development of new markets as an ec adoption criteria. introduction electronic commerce (ec) has led to changes in the day-to-day procedures of organisations (kuljis et al, 1998). technology is no longer viewed as a set of stand alone systems, but has become fully integrated with the overall organisational strategy (turban et al, 2000; raisch, 2001; porter, 2001; rajkumar, 2001). organisations that have adopted ec or are in the process of considering this type of information technology (it) investment, have found that traditional return on investment (roi) calculation methods are not sufficient for the purpose of accurately measuring it payback (moozakis & lewis, 2000; lewis, 2000a; violino, 2000b). this has led organisations to use alternative evaluation methods and different sets of organisational criteria against which to measure investments in it. this is particularly relevant to the small to medium enterprise (sme) sector where changes brought about by the advent of ec are having a significant impact. this paper begins by briefly examining some of the changes brought about by the advent of ec, in particular changes to the metrics used by organisations when considering investment in ec technology. this is followed by a discussion of the nature and internationalisation of smes, in particular their approach to the acquisition, investment and use of ec technology. the paper then presents a study of 118 swedish smes which investigated the major reasons behind their adoption of ec. the study also examines whether the respondent smes’ major market focus (local, regional, national or international) influenced their decision making in the ec adoption process. finally, the study endeavours to ascertain whether factors such as the size of the business or the existence of a fully developed plan for the acquisition of ec are associated with the importance of reaching new markets as a criterion for ec adoption. ajis vol. 10 no. 1 september 2002 111 ec technology decision making processes organisations have encountered different problems in attempting to formally measure the adoption and profitability of it investments in ec. a number of authors (kollman, 1998; dugan, 1999; hlava, 1999; wilder, 1999; violino, 2000a; cameron, 2000) have argued that with the advent of ec, traditional measurements have become obsolete or invalid. some of the problems associated with traditional metrics include the inability of accounting for hidden costs and intangible strategic benefits of ec and the necessity of constantly revising evaluation methods in line with the fast pace of it developments. conventional roi calculations fail to alleviate these problems. moozakis and lewis (2000) argue that this is due to the lack of product tangibility and an increase in virtual organisational structures. these developments are in direct contrast to the formal, material nature of roi measurements. there have been diverse attempts to apply different forms of evaluation methods and calculations in order to gauge the payback of it investments. indeed, wilder (1999:48) reports that many organisations are “tossing out conventional thinking about the need for a return on investment and focusing on how the initiatives advance their overall business strategy, whether it’s to improve customer satisfaction, increase brand awareness, or open new sales channels”. organisations have begun to rely on in-house, as well as off-the-shelf products for this purpose (moozakis & lewis, 2000). some of the evaluation criteria used by organisations include improvement in lead time (riemenschneider & mykytyn, 2000), development of new markets (wilder, 1999), improved relationships with business partners (korchak & rodman, 2001) and improved marketing (cameron, 2000). the methods employed range from: identifying sources of sales revenue generated through and attributable to ec and customer satisfaction surveys (violino, 2000a; 2000b); measuring the it performance of an ec website as a whole; operational cost savings, observing customers’ surfing and online purchasing habits to expand marketing strategies; determining the revenues generated during off-business hours; counting online transactions and the number of online visitors (lewis, 2000); as well as benchmarking against competitors. where these methods fail, they are replaced by the instincts and experience of managers and decision makers (violino, 2000a). not surprisingly, some organisations (specifically large businesses) have begun viewing ec outlays as absolutely necessary, regardless of cost, in order to remain competitive (quinn, 2000). cameron (2000) further confirms the abandonment of formal roi calculations by stating that it projects and investments can be justified for any number of reasons, without ever measuring any form of roi, thus highlighting the importance placed on ec in the modern organisation. evans (2001) argues, however, that before ec can achieve any impact it must first be fully aligned and integrated with the overall strategy of the business. indeed, brynjolfsson and hitt (1998) assert that ec cannot exist without investment in organisational change. the study presented in this paper has focused on smes, therefore it is now appropriate to examine the nature and internationalisation of smes, in particular their acquisition and use of it and ec technology. the nature of small business the special circumstances of small business have been the topic of many government committee findings as well as research initiatives. barnett and mackness (1983) suggest that small businesses should not be thought of as miniature versions of large businesses, but quite unique in their own right. small business tends to be more risky than its larger counterpart (brigham & smith 1967; walker 1975), it is subject to higher failure rates (klatt 1973; cochran 1981) and tends to keep less adequate records (markland 1974; rotch 1987). reynolds et al (1994) provide the following summary of features of small business generally not shared by larger organisations: a small management team; strong owner influence; multi-functional management; a close and loyal work team; centralised power and control; lack of specialist staff; informal and inadequate planning and control systems; lack of promotable staff; lack of control over the business environment; limited ability to obtain finance; labour intensive work; limited process and product technology; narrow product/service range, limited market share and heavy reliance on few customers; intuitive rather than rational decisions; reactive rather than innovative responses to change; personal rather than tasks oriented leadership; practical but narrow education, experience and skills; low employee turnover; product dedication rather than customer orientation; reluctance to take risks; management heavily swayed by the owner’s personal idiosyncrasies; strong desire for independence; and intrusion of family interests. in line with general views concerning small business, the acquisition and subsequent use of computer technology in the small business environment cannot simply be considered a miniaturised version of the larger organisation. clearly, many of the features gleaned from studies of larger businesses can be applied to small business. however, the nature of small business operations presents a variety of factors which must be considered when computer technology is acquired in the small business environment. increasingly, the acquisition of ec technology in particular has also meant increased global exposure for smes. this trend will be briefly discussed in the next section. ajis vol. 10 no. 1 september 2002 112 internationalisation of smes it is now widely acknowledged that the internet has changed the way smes do business by revolutionising the dynamics of international commerce and enabling cross-border information flows and transactions (quelch & klein, 1996). one of the main uses of the internet has been for globalisation, allowing smes to achieve rapid growth and internationalisation (ellsworth and ellsworth, 1997). the creation of global markets and the advent of ec have brought about a new set of opportunities and challenges to smes (raymond, 2000; nielson & morris, 2001; quelch & klein, 1996). quelch and klein (1996) argue that the internet will cause a reduction in global advertising costs and increase advertising efficiency, erode the competitive advantages of scale economies, decrease information dissemination and communication costs by abolishing geographical and temporal barriers, thus, allowing smes to reach a critical mass of customers. in order to achieve this low cost access to global markets and customers, smes have had to invest in and acquire ec. it is now appropriate to examine ec acquisition by smes, in particular the criteria used by smes for ec adoption, as well as the effect, if any, of potential globalisation. acquisition of it and ec by smes there have been many studies, over the years, investigating the acquisition and use of both it and ec by smes. these studies have often been directed towards trying to determine the it success in smes. in the past, many of the studies tended to be directed towards specific sectors of the sme: users, vendors and management (see schultz et al, 1984; mcdoniel et al, 1993). user-oriented issues included perceptions and expectations of it, user involvement, acceptance and training. investigators have studied the impact of users’ beliefs and attitudes on their usage behaviour (robey, 1987; ginzberg, 1982; ives et al, 1983), and how these internal beliefs and attitudes are, in turn, influenced by various external factors. these factors include the system’s technical design characteristics (gould et al, 1983 and dickson, desanctis & mcbride, 1986 cited in ditsa, 1994); user involvement in system development (baroudi et al 1986; tait & vessey, 1988; amoako-gyampah & white, 1993; burton et al, 1993); the type of system development methodology used (alavi, 1984); the nature of the implementation process (ginzberg, 1978); and cognitive style (szajna & scamell, 1993). on the other hand, ‘vendor’ studies have concluded that ongoing success with it was positively associated with vendor support, vendor after sales service, vendor training and vendor expertise (see yap et al, 1992; macgregor & cocks, 1995; macgregor et al, 1995). finally, management involvement in the acquisition of it by smes has been reported by bergeron et al (1990), jarvenpaa and ives (1991), and fink and tjarka (1994). more recently, however, studies surrounding the adoption of both it and ec (see macgregor & bunker, 1996a; ibid 1996b; thong, 1999, bunker & macgregor, 2000; mirchandani & motwani, 2001) have suggested the need to look at the driving forces or criteria behind ec adoption in relation to the overall strategy of the sme rather than simply relating the driving forces to specific sectors. in a study of 308 ceo’s, riemenshcneider and mykytyn (2000) nominated a variety of reasons, which they termed ‘most important things’ that prompted smes into the adoption of ec. these included reduction of costs, increased productivity, faster feedback of information and access to customers. a similar study of the asia pacific region by price waterhouse coopers (1999), as well as studies by fuller (2000) and kendall (2000) suggested factors such as improved customer service, customer information exchange, improved competitiveness, increased revenue, attracting new investment and access to international markets as reasons put forward by smes for their adoption of ec. access to new markets and global customers has, in fact, been singled out by various authors as one of the most important driving forces behind ec adoption and use by smes. a national small business united and arthur andersen survey of smes in 1997 indicated that 86.7% of the respondents that had a website used it to reach new customers, while 30.4% had a website to support global expansion. in 1996, the same survey reported that only 8% of the respondents had home pages on the web and more than 77% did not use the internet to conduct business. clearly, claims that ec is changing the way smes view globalisation and competition (poon 2000) are legitimate. in fact, the same survey in 1998 reported that smes viewed the internet as their most favoured growth strategy in the next 12 months. adopting ec and developing an online strategy has enabled smes to expand their customer base globally (tetteh & burn, 2001; maloff, 1995) and reach markets and customers that were once unreachable (shewmake & sapp, 2000; bennett, 1997) allowing smes to compete locally, nationally and internationally. quelch and kelin (1996) argue that ec will lead to rapid internationalisation of smes. it has also been argued that the internet might decrease export barriers faced by smes (bennett, 1997; hamill, 1997). bennett (1997:327) argues that “… the use of the internet for global marketing enables firms to leapfrog the conventional stages of internationalisation, as it removes all geographical constraints, permits the instant establishment of virtual branches throughout the world, and allows direct and immediate foreign market entry to the smallest of ajis vol. 10 no. 1 september 2002 113 businesses”. finally, a study by poon and swatman (1997) suggests that smes believe it is cheaper to compete globally through the internet. the implications of this are significant. a research study of swedish smes which aimed to investigate the relationship between organisational factors and driving forces or criteria for ec adoption by smes will now be described. the study also examined whether the major market focus of smes (local, regional, national or international) was associated with any of these driving forces. survey instrument a questionnaire was developed for sme managers. a series of factors termed ‘driving forces for the adoption of ec’ were developed. the inclusion and exclusion of factors was based on previous studies, in particular those carried out by price waterhouse coopers (1999) and riemenschneider and mykytyn (2000). only those factors which were reported as having a greater than 50% response as important were included in the survey instrument. the factors included in the questionnaire were: customer demand, pressure by competition, pressure by suppliers, reduced cost, improvements to customer services, improvements to lead time, increased sales, improvements to internal efficiency, improved relationships with business partners, development of new markets, improved organisational competitiveness, improved marketing and improved internal control of the business. respondents were asked to rate each of these factors as pertaining to their adoption of ec across a 5 point lickert scale where 1 indicates that the factor was unimportant to the decision to adopt ec, 2 indicates little importance to the decision to adopt ec, 3 indicates some importance to the decision to adopt ec, 4 indicates the factor was quite important to the decision to adopt ec and 5 indicates it was of greatest importance to the decision to adopt ec. respondents were also asked about their market focus (local, regional, national or international), whether a full enterprise wide plan had been developed and documented for the use of ec within the organisation and the size of the business (sole entrepreneur, 1-9, 10-19, 20-49, >50). questionnaires were distributed to smes around the four regional areas of sweden: karlstad, filipstad, saffle and arvika. analysis of responses i) overview of responses a total of 1170 questionnaires were distributed. responses were obtained from 260 sme organisations giving a response rate of 22.2%. this response rate was considered to be acceptable for the purposes of this study, however, further analysis indicated that of the 260 responses, only118 stated that they were using ec in their day-to-day business activities. these figures highlight that as little at 10% of the total potential respondents surveyed were using ec in their day-to-day business operations. this fact is of critical importance to researchers and practitioners alike when we think of the billions of dollars spent annually by organizations, vendors and governments on the development, implementation and use of ec technologies and approaches for business improvement and strategic advantage. of course, further studies would need to be done to qualify this as a possible issue, however, responses (or lack of them) may indicate the relative unimportance of ec to sme’s (in this case within the swedish context). the questionnaire instrument (relevance to each sme, interpretation of content, sme motivation to answer a questionnaire etc) may have also been a barrier to potential respondents, hence the low response rate. it must also be stressed that the data from this study represents only the most superficial indication of what these businesses deem to be important in ec uptake and use. it is envisaged that the research team will conduct indepth interviews with some of the respondent businesses in order to contextualise responses and obtain a greater insight into why these business feel that these factors are so important. the respondent group in this study covered retail, industrial, service and financial sectors. 96% of the respondents had been in business for more than 2 years. 92% of respondents indicated that they had been using some form of it for at least the last 2 years. table 1 provides the overall responses for the major driving forces behind the adoption of ec. some respondents did not complete the questionnaire fully. this was taken into account in the statistical analysis. ajis vol. 10 no. 1 september 2002 114 table 1 factors which were the major driving forces behind the adoption of ec no importance little importance some importance very important greatest importance customer demand 48 25 20 19 6 pressure by competition 36 20 32 19 9 pressure by suppliers 57 23 19 13 3 reduced cost 13 12 28 37 27 improved customer services 6 1 16 43 52 improvement to lead time 29 14 23 30 19 increased sales 17 13 35 25 26 improved internal efficiency 6 2 20 54 39 development of new markets 18 15 21 33 30 improved competitiveness 13 5 18 41 39 improved marketing 24 8 21 30 33 greater control of business 25 16 28 29 19 improved relationships with business partners 16 11 32 39 19 ii) analysis of responses a series of chi-square tests were carried out to determine whether the major market focus was significantly associated with the rating of any of the driving forces or criteria behind ec adoption. the results indicate that the major market focus was not associated with any of the driving forces. however, the existence of a fully developed plan for ec and the size of the business were associated with the development of new markets as a driving force behind ec adoption (see tables 3 & 4). there was also a clear association of size of smes of a larger size being more likely to have a plan (see table 2). previous studies (macgregor & bunker (1996a), bunker & macgregor (2000)) found that sme’s in general were more likely to have an operational focus and therefore little strategic motivation for formal planning. within the context of this study, as these smes grow larger they seem more likely to shift their focus to strategic management approaches like the production of formal ec plans. ajis vol. 10 no. 1 september 2002 115 table 2 rating of the number of employees as a criteria for the adoption of ec (developed plan versus no plan ***includes non ec and it adopting organizations) single <10 employee employees 10-19 employees 20-49 employees 50-200 employees >200 employees fully developed plan 10 53 24 22 18 5 no plan *** 15 52 8 4 1 0 (p < .001) table 3 rating of the importance of developing new markets as a criteria for the adoption of ec (developed plan versus no plan) no importance little importance some importance very important greatest importance fully developed plan 7 13 15 24 22 no plan 10 0 6 6 5 (p < .005) table 4 rating of the importance of developing new markets as a criteria for the adoption of ec (number of employees) no of employees no importance little importance some importance very important greatest importance 0 4 1 0 3 1 1 – 9 8 3 9 8 21 10 – 19 1 1 6 8 5 20 – 49 1 4 5 8 1 > 50 4 6 1 6 2 (p < .001) discussion before examining the data in detail, it is interesting to note that of the 260 respondents, only 118 (45.4%) indicated that they are using ec in their day-to-day business activities. this will be the subject of a study to be produced by the authors. a number of authors and studies (wilder, 1999; price waterhouse coopers, 1999; violino, 2000b; riemenschneider & mykytyn, 2000) have suggested that factors other than strict roi are being used by smes as criteria to be considered for the adoption of ec. a closer examination of table 1 would tend to support this, at least amongst the respondent group. while reduction of cost was seen as important as a criterion for the adoption of ec, by far the strongest reasons appear to be improvements to customer services, improvements to internal efficiency and improvements to organisational competitiveness. while these figures do not actually reject roi, they suggest that for smes, roi is only one of several factors that need to be taken into consideration when adopting ec. the informationweek research global it strategies (2001) study found that organisations which are classified as export leaders are more likely to invest in leading edge it (cited in thyfault, 2001). according to this study, most export leaders invest in e-business which results in a higher percentage of their revenue coming from global sales. the results of the survey presented in this paper indicate, however, tha major business markets were not significantly associated with any of the driving forces behind ec adoption. export smes in sweden were no more likely to adopt ec than smes operating locally and nationally. several reasons have been put forward by different authors for smes’ lack of interest in exporting and international markets. nielson and morris (2001) suggest that, despite the opportunities afforded by ec, smes still face traditional problems including poor transport and distribution networks, inefficient customs procedures and barriers to market access in key export markets, such as high tariffs or restrictive trade practices. shewmake and sapp (2000) also list obstacles that smes may encounter, including language barriers, tariffs, diverse currencies and customs regulations. they state that “as a result of these problems, many companies have yet to move their e-business beyond their native shores” (ibid, 2000:30). bennett’s study (1997) shows that one of the reasons some ajis vol. 10 no. 1 september 2002 116 organisations have not set up a web site is because they do not perceive the internet as making exporting any easier. hamill and gregory (1997) identified four different categories of export barriers including psychological, operational, organisational and product/market. it appears that these barriers remain obstacles to the internationalisation of smes despite the advent of ec. this can also be related to organisational size as a larger business is more likely than an sme to have available resources (specialist it staff, dedicated budgets, multinational experience) to facilitate a global ec strategy. despite these results, an examination of table 2 indicates that larger smes (>20 employees) are more likely to have a fully developed ec plan (almost 100% of these organisations). this leads us to ask the question: is planning a natural function of size of a business? once an sme increases in size and sees the need and necessity to plan, does the business have a different perspective of opportunity and the use of it and ec due to its resource base increase and market intensity. does this naturally give the larger sme access to more business opportunities (developing new markets) as a result? an examination of table 3 indicates that 57% of those who had developed a plan for ec adoption saw the development of new markets as being of extreme importance (categorised by respondents as being very important or of greatest importance) as a criterion for ec adoption, and only 8% saw no importance in the criterion. this reinforces the findings in table 2. by comparison, of those respondents who had not developed a plan, 41% saw the driving force development of new markets as being of extreme importance (categorised by respondents as being very important or of greatest importance) as a criterion for ec adoption, while 37% saw no importance. tetteh and burn (2001) support the view that smes need to adopt a strategic outlook in their planning and management in order to adopt an e-business infrastructure, while hamill (1997) highlights the need for export planning and adopting a strategic approach to new markets. it is interesting to note that the development of an ec plan gave those smes an insight into the importance of new markets, however the responses also indicate that these businesses did not link ec directly to the development of new markets. this may indicate the importance of planning rather than technology in creating new (perceived) business opportunities for these businesses. this is an especially critical finding given the more operational orientation of most sme’s supported by previous studies (macgregor & bunker (1996a), bunker & macgregor (2000). finally, table 4 indicates that smaller sized smes (2 to 20 employees) placed a higher level of importance on the use of ec to develop new markets than those organisations which consisted of the sole entrepreneur and those whose staffing level was greater than 20. in fact, 40% of these small sized smes indicated they saw the development of new markets as being of extreme importance (categorised by respondents as being very important or of greatest importance) as a criterion for ec adoption. as we can see by the previous tables, however, the likelihood of smes of this size having a formal ec plan (and thus an understanding of the market opportunities) is only at a level of 50%. this represents a potential barrier to the effective uptake and use of it and ec by these smaller smes. conclusion this research has sought to identify factors that influence the adoption of ec by smes. improvement in customer services, improvement to internal efficiency (small and larger smes) and improvement to organisational competitiveness (larger sme) were rated as the most important criteria for the adoption of ec in the organisation. clearly, while roi remains an important consideration, the ‘all pervasiveness’ of ec within the organisation has meant that roi is only one of many factors needing to be considered. the globalisation and internationalisation of markets presents a new challenge to smes (raymond, 2001). meeting this challenge has been made easier by the advent of ec which has opened up new opportunities and new markets for, not only large, but also small and medium organisations, including those from developing countries (nielson & morris, 2001). the research presented here has indicated that the adoption of ec by smes was not associated with their major market focus, but may be better correlated to the size of the business (in terms of employee numbers). it appears that export smes still face similar challenges to those that existed before the advent of ec. raymond (2001) and quelch and klein (1996) argue that smes need support from government organisations in developing marketing strategies that are appropriate in a global business environment. nielson and morris (2001) also suggest that in order to increase cross-border trade, much has to be done by way of international regulatory cooperation, while hamill (1997) states that internet training should be a top government priority. it would appear that many of these issues become a greater reality (and therefore more critical) as the size of the sme increases. as sme’s grow in size perhaps they will identify new markets and exports through more formal planning mechanisms, rather than governments and it vendors attempting to create these market and export opportunities in an artificial manner. recent studies have shown that for the adoption of ec to be successful, there is a need to fully plan its acquisition and use. this study supports this notion by showing that the existence of an organisational wide plan is associated directly with the importance rating of developing new markets as a criterion for ec adoption. this ajis vol. 10 no. 1 september 2002 117 is further correlated with the size of the sme (in terms of employee numbers) that appears to have some bearing on the importance placed on the development of new markets when decisions are being made. clearly, additional research is required to refine and understand the issues identified in this and other studies, however, it would appear that ec adoption is just as prevalent among local and national smes, as it is among smes that export. acknowledgements the authors would like to express their appreciation to s berggren and a stjärnkvist for their help in distributing and collating the questionnaires. references alavi, m. (1984) an assessment of the prototyping approach to information systems development, communications of the acm, 27(6), 556 – 563 amoako-gyampah, k. & k.b. white (1993) user involvement and satisfaction: an exploratory contingency model, information & management, 25(1), 1 – 10 barnett, r.r. and j.r. mackness (1983) an action research study of small firm management, journal of applied systems, 10, 63 – 83 baroudi, j.j., m.h. olson and b. ives (1986) an empirical study of the impact of user involvement on system usage and information satisfaction, communications of the acm, 29(3), 232 – 238 bennett, r. (1997) export marketing and the internet: experiences of web site use and perceptions of export barriers among uk businesses, international marketing review, 14(5), 324 – 344 bergeron, f., s. rivard and l. de serre (1990) investigating the support role of the information centre, mis quarterly, 14(3), 247 – 260 brigham, e.f. and k.v. smith (1967) the cost of capital to the small firm, the engineering economist, 13(1), 1 – 26 brynjolfsson, e. and l.m. hitt (1998) beyond the productivity paradox, communications of the acm, 41(8), 49 – 55 bunker, d.j. & r.c. macgregor (2000) successful generation of information technology (it) requirements for small/medium enterprises (sme’s) – cases from regional australia. in proceedings of smes in a global economy, 72 – 84, wollongong, australia burton, f., y. chen and v. grover (1993) an application of expectancy theory for assessing user motivation to utilise an expert system, journal of management information systems, 9(3), 183 – 198 cameron, p. (2000) measuring up, cma management, 74(2), 26 – 28 cochran, a.b. (1981) small business mortality rates: a review of the literature, journal of small business management, 19(4), 50 – 59 ditsa, g.e.m. (1994) measures of user acceptance of information systems, master of commerce dissertation, department of information systems, university of wollongong dugan, s. (1999) the revenue factors: strategies for maximizing e-commerce success, infoworld, 21(40), 70 – 72 ellsworth, j.h. and m.v. ellsworth (1997) marketing on the internet, john wiley & sons, new york evans, c. (2001) an e-strategy for online e-business, information systems management, 18, 8 – 21 fink, d. and f. tjarka (1994) information systems contribution to business performance: a study of information systems executives’ attitudes, australian journal of information systems, 2(1), 29 – 38 fuller, t. (2000) the small business guide to the internet: a practical approach to going online, international small business journal, 19(1), 105 – 107 ginzberg, m.j. (1978) steps towards more effective implementation of ms and mis, interfaces, 8(3), 57 – 63 ginzberg, m.j. (1982) key recurrent issues in mis implementation program, mis quarterly, 5(2), 47 – 59 hamill, j. (1997) the internet and international marketing, international marketing review, 14(5), 300 – 323 hamill, j. and k. gregory (1997) internet marketing in the internationalisation of uk smes, journal of marketing management, 13(1-3), 9 – 28 hlava, m.m.k. (1999) expanding partnerships for small-and medium-sized enterprises (smes) needed: the right kind of support, bulletin of the american society of information science, 25(5), 7 – 9 ives b., m.h. olsen and j.j. baroudi (1983) the measurement of user information satisfaction, communications of the acm, 26(10), 785 – 793 ajis vol. 10 no. 1 september 2002 118 jarvenpaa, s. & b. ives (1991) executive involvement and participation in the management of information technology, mis quarterly, 15(2), 205 – 225 kendall, k.e. (2000) e-commerce aesthetics: e-commerce for e-commerce’ sake, information resources management journal, 13(3), 3 – 4 klatt, l.a. (1973) small business management: essentials of entrepreneurship, wadsworth, ca kollman, t. (1998) marketing for electronic market places – the relevance of two ‘critical points of success, electronic markets, 8(3), 36 – 39 korchak, r. and r. rodman (2001) ebusiness adoption among us small manufacturers and the role of manufacturing extension, economic development review, 17(3), 20 – 25 kuljis, j., r. macredie and r.j. paul (1998) information gathering problems in multinational banking, journal of strategic information systems, 7(3), 233 – 245 lewis, d. (2000) some retailers de-emphasize web payback, internetweek, 834, 94 mcdoniel, p.l., j. palko and t.p. cronan (1993) information systems development: issues affecting success, journal of computer information systems, 34(1), 50 – 62 macgregor, r.c. & r.s. cocks (1995) computer usage and satisfaction in the australian veterinary industry, australian veterinary practitioner, 25(1), 32 – 39 macgregor, r.c., d.j. bunker, j.k. pierson and k.a. forcht (1995) levels of satisfaction with pre and post installation vendor contact: their effect on small business and college/university technology course requirements, education foundation of the data processing management association, 78 – 84 macgregor, r.c. & d. j. bunker (1996a) the effect of priorities introduced during computer acquisition on continuing success with it in small business environments. in proceedings of the information resource management association international conference, 271 – 277, washington d.c. macgregor, r.c. & d.j. bunker (1996b) the effect of setting performance criteria when acquiring computer technology on the perceptions of ongoing success with information technology in the small business environment. in proceedings of the 25th annual meeting of the western decision sciences institute, 277 – 279 maloff, j. (1995) the virtual corporation, internet world, 5, 46 – 50 markland, r.e. (1974) the role of the computer in small business management, journal of small business management, 12(1), 21 – 26 mirchandani, d.a and j. motwani (2001) understanding small business electronic commerce adoption: an empirical analysis, journal of computer information systems, 41(3), 70 – 73 moozakis, c. and d. lewis (2000) enterprises tailor roi to e-business, internetweek, 842, 90 national small business united and arthur andersen (1998) annual survey of small and mid-sized business, nsbu national small business united and arthur andersen (1997) annual survey of small and mid-sized business, nsbu national small business united and arthur andersen (1996) annual survey of small and mid-sized business, nsbu nielson, j. & r. morris (2001) e-commerce and trade: resolving dilemmas, organisation for economic cooperation and development. the oecd observer, no 224, 37 – 39 poon, s. (2000) business environment and internet commerce benefit – a small business perspective, european journal of information systems, 9, 72 – 81 poon, s. and p.m.c. swatman (1997) small business use of the internet: findings from australian case studies, international marketing review, 14(5), 385 – 402 porter, m. (2001) strategy and the internet, harvard business review, march, 63 – 78 price waterhouse coopers (1999) sme electronic commerce study final report, 37pp quelch, j.a. and l.r. klein (1996) the internet and international marketing, sloan management review, 37(3), 60 – 75 quinn, j. (2000) untenable roi spreads e-panic, computing canada, 26(6), 15 raisch, w.d. (2001) the emarketplace: strategies for success in b2b, mcgraw hill, new york rajkumar, t.m. (2001) e-procurement: business and technical issues, information systems management, 18, 52 – 60 raymond, l. (2001) determinants of web site implementation in small business, internet research: electronic networking applications and policy, 11(5), 411 – 422 riemenschneider, c.k. and p.p.jr mykytyn (2000) what small business executives have learned about managing information technology, information & management, 37, 257 – 267 reynolds, w., w. savage and a. williams (1994) your own business: a practical guide to success, itp ajis vol. 10 no. 1 september 2002 119 robey, d. (1987) implementation and the organisational impact of information systems, interface, 17(3), 72 – 84 rotch, w. (1987) management of small enterprises: cases and readings, university of virginia press schultz, r.l., m. j. ginzberg and h.c.jr lucas (1984) a structural model of implementation. in schultz, r.l. and m.j. ginzberg (eds) application of management science, jai press, greenwich, 55 – 87 shewmake, b. and g. sapp (2000) bringing down the international barriers, infoworld, 22(18), 30 szajna, b. and r. scamell (1993) the effects of information systems user expectations on their performance and perceptions, mis quarterly, 17(4), 493 – 614 tetteh, e. and j. burn (2001) global strategies for sme-business: applying the small framework, logistics information management, 14(1/2), 171 – 180 tait, p. and i. vessey (1988) the effect of user involvement on system success: a contingency approach, mis quarterly, 12(1), 91 – 108 thong, j.y.l (1999) an integrated model of information systems adoption in small businesses, journal of management information systems, 15(4), 187 – 214 thyfault, m.e. (2001) heavy exporters spend big on leading-edge it, informationweek, 834, 54 turban, e., j. lee, d. king and h. chung (2000) electronic commerce: a managerial perspective, prentice hall, new york violino, b. (2000a) it excellence 500, internetweek, 754, 48 – 62 violino, b. (2000b) payback time for e-business, internetweek, special issue 811, 1 walker, e.w. (1975) investment and capital structure decision making in small business. in walker, e.w. (ed) the dynamic small firm: selected readings, austin press, texas wilder, c. (1999) roi: e-business strategic investment, informationweek, 735, 48 – 56 yap, c.s., c.p.p. soh and k.s. raman (1992) information system success factors in small business, omega international journal of management science, 20(5-6), 597 – 609 microsoft word ajis vol13-2 o.doc 31   australasian journal of information systems     volume 13 number 2    may 2006  demystifying a hermeneutic approach to is research    phyl webb  school of information systems  university of tasmania  private bag 87, hobart 7001, australia  email: webbp@utas.edu.au    carol pollard  cis department, john a. walker  college of business  appalachian state university, raley hall  boone, nc 28607, usa  email: pollardce@appstate.edu      abstract    while hermeneutics is firmly embedded in social science research and has proven merit  as  a  qualitative  research  philosophy  and  data  analysis  technique,  the majority  of  is  researchers  continue  to be  hermeneutic neophytes.   this paper  seeks  to promote  and  demystify  this valuable but  challenging  tool,  exploring  the  evolution of hermeneutics  and  its  different  approaches,  presenting  a  brief  overview  of  previous  is  hermeneutic  research, as reported in the literature and describing the applicability of hermeneutics to  qualitative  is  research    in  illustration, and  in an attempt  to  smooth  the way  for other  novice  ì hermeneutsî ,  it  presents  the  choice  and  application  of  one  hermeneutic  approach based on  the work of ricoeur  and gadamer,  in  a  research  in progress. the  paper  concludes with  a  reflection on  the value of  the hermeneutic  approach  that was  adopted in the research process and its contribution to is research.      introduction    while  some  is  researchers  have  embraced  the  use  of  hermeneutics,  to  many  it  still  remains  a  complex,  mysterious  approach.    in  the  following  sections,  the  authors  attempt  to  describe  the  challenges  they  encountered  on  the way  to  gaining  enlightenment  as  ì hermeneutsî   by  describing  their application of one approach  to a  research  in progress.   the  results of using  the hermeneutic  approach are also discussed together with suggestions to guide those researchers who may choose to  use a similar strategy in their research.    hermeneutics  is by nature  interpretivist.  it deals with  the  interpretation and understanding of  text,  including text analogues (myers and avison, 2002).     many hermeneutic scholars have extended  their conception of  text  to  include not  just the documentary artefacts that human subjects create, but also their individual  actions,  group  behaviours,  and  even  social  institutions,  all  of  which,  as  text  analogues, have meanings that can be read and interpreted (lee 1994, p149)   this suggests that hermeneutics is an approach that lends itself to application within organisational  contexts where there is commonly, a preponderance of documentation describing and explaining the  organisationís  focus,  mission,  processes  and  procedures.  in  addition,  the  employees  themselves  represent  a  rich  and  valuable  source  of  text  or  text  analogue,  most  commonly  in  the  form  of  australasian journal of information systems     volume 13 number 2    may 2006     32  interview transcripts.   to some, hermeneutics is more than a data analysis technique or method.   it  can  instead be  treated as both an underlying philosophy and specific mode of analysis (myers and  avison 2002). common  to  all  forms of  hermeneutics  is  the  concept of  the hermeneutic  circle or  spiral.   this  concept  encapsulates  the  idea  that,  in  an  attempt  to  understand  the  true meaning  of  something,  one  seeks  a  pre-existing  understanding  of  the  article  under  study  from  oneís  own  experience.   this,  in  turn,  enhances  and  expands knowledge  and understanding of  the  article  and  leads  to  greater  understanding  (mallery,  hurwitz  and  duffy  1987).    to  gain  perspective  on  hermeneutic research, table 1 provides a brief chronology of its prominent scholars.    scholar  lived  friedrich daniel ernst schleiermacher  1768-1834  wilhelm dilthey  1833-1911  martin heidegger  1889-1976  hans-georg gadamer  1900-2002  jurgen habermas  1929-  paul ricoeur  1913-2005    table 1 - prominent hermeneutic scholars: a chronology    hermeneutics has been described in a number of ways.   mallery et al (1987) include the following  classification in their informative discussion of hermeneutics:  •   classical methodological hermeneutics  •   philosophical hermeneutics  •   critical hermeneutics  •   phenomenological hermeneutics    these categories are a useful starting point  for beginning  to understand hermeneutics.   this paper  contains  a  brief  discussion  of  the  proponents  of  hermeneutics most  commonly  encountered  in  is  research  framed within  the  categories  provided  above.   this  is  followed  by  a  brief  overview  of  published is research  that applies a hermeneutic approach and an explanation of  the application of  hermeneutics in an ongoing it governance research project.      classical methodological hermeneutics    hermeneutics originated  as  the  study  and  interpretation of  literary  and biblical  texts.   the greek  scholars  who  undertook  the  study  of  texts  analysed  and  interpreted  works  for  consistency  in  grammar and literary style. biblical scholars applied similar methods.  however they also made use  of allegory in their interpretation and application of hermeneutics (mallery et al 1987).    according  to  bleicher  (1980),  schleiermacher,  an  early  and  important  contributor  to  the  development of classical methodological hermeneutics, maintained that the purpose of hermeneutics  was  to  derive  understanding  from  the  text  under  study.    in  this  way,  schleiermacher  began  to  develop a philosophical focus for hermeneutics.  he did this through his consideration of the process  of understanding and the conditions that are necessary for a valid interpretation (bleicher 1980).  in  recognising  this  need  for  a  more  consistent  systematic  approach  to  hermeneutic  study  schleiermacher made  a  valuable  contribution  to  the  evolution of  hermeneutics.   he  also  initiated  increased  interest  in  examining  the  epistemological  issues  of  interpretation  and  understanding,  (mallery et al 1987).    33   australasian journal of information systems     volume 13 number 2    may 2006    bleicher (1980) writes on schleiermacher: ì given adequate historical and linguistic knowledge, the  interpreter  is  in  a position  to understand  the  author better  than he  understood himselfî   (p15)  and  credits him with developing a ì systematic hermeneuticsî  compromised of two interpretations:  1.  grammatical  2.  psychological    schleiermacher  provided  forty-four  canons  to  explain  grammatical  interpretation  (bleicher  1980).  the first two are considered most important.  in summary, these encapsulate a view that:  1.  in order to comprehend or understand the text one needs to understand the linguistic context  of the text.  2.  in order  to understand an  individual part of  the  text  the surrounding words, within which  the part under study is situated, also need to be considered.  schleiermacher  also  developed  canons  for  psychological  interpretation  with  the  intention  of  exploring the ëcreative inner processí (dilthey 1976).    in his writing about schleiermacher, dilthey (1976) describes the aim of the hermeneutic process as  one that allows an understanding of ì the author better than he has understood himself: a proposition  which  is  the necessary consequence of  the doctrine of unconscious creationî  (p116).   this view of  schleiermacher is also supported by gadamer (1976).    following  schleiermacher  came dilthey,  a  key  figure  in  the  development  of  hermeneutics  who  emphasised  the role of understanding (verstehn) who contrasted  it with  the pre-kantian, cartesian  practices  of  explanation  (erklarung)  (hamilton  1994,  p64).    dilthey  spent  much  time  studying  schleiermacherís work and was heavily influenced by him and is often grouped with schleiermacher  as a romanticist.    diltheyís  approach  to  hermeneutics  differs  from  schleiermacherís  in  this  movement  between  understanding and explanation as  two key  foci.    ì explainingî   refers  to  the provision of a general  description  of  the  type,  providing  its  key  characteristics.  ì understandingî   infers  a more  in-depth  knowledge of  the  individual  internal characteristics and  intentions of a member of  the  type (olson  and carlisle,  2001).    this movement  from  inner  to  outer was  summarised  epistemologically  by  bleicher  (1980)  who  proposed  that  ì generally  valid  knowledge  can  be  derived  both  from  appearances and inner experienceî  ( p22).    dilthey is credited with having developed a methodological hermeneutics but was also criticised for  his ì backslidingî  into objectivism.  he ascribed to a belief that the interpreter could understand the  authorís intentions through a process called re-living, which effectively results in the conclusion that  there  is  one  true  account,  or  objectification  (bleicher  1980).   diltheyís  heremeneutics  advocates  acceptance of the text at face value without any critical consideration or review. in this way it differs  from  some  of  the more  contemporary  hermeneuts  (myers,  1994).   modern  day  hermeneutics  is  indebted to both schleiermacher and dilthey for their important contributions to the advancement of  the concept of hermeneutics.      philosophical hermeneutics    heidegger has been described  as  the  ì main proponentî  of dialectical hermeneutics  (mallery  et  al  1987).   this  is  an  approach  that  heralded  a  new philosophy  for hermeneutics.   his work had  an  ontological leaning that sought to illuminate not just the method of understanding but also the nature  australasian journal of information systems     volume 13 number 2    may 2006     34  of being.   ì heidegger moves hermeneutics from a theory of interpretation to a theory of existential  understandingî  (mallery et al 1987, p5).    heidegger saw three fore-structures of understanding (bleicher 1980):  1.  fore-having, already owning knowledge of the system, context or situation within which  the phenomenon under study exists. this is a broad general view (demetrio 2001).  2.  fore-sight, this level is specific to the phenomenon being studied but is still at a general  level (demetrio 2001) and  3.  fore-grasping,  the  detailed  understanding  of  the  phenomenon  under  study  (demetrio  2001)    heideggerís intention in outlining fore-structures is made clearer by grondin (1994):  ö in  order  to  interpret  a  text,  for  example,  it  is  necessary  to  make  our  ownì iì   situation transparent so that we can appreciate precisely the otherness and alterity of  the text ñ that is, without allowing our unelucidated prejudices to dominate the text  unwittingly and so conceal what is proper to it (p97).  his view of the importance and necessity for fore-structures represents a radical change in thinking  regarding the hermeneutic circle.    gadamer,  a  student  of  heidegger,  built  on  heideggerís  thinking  in  developing  philosophical  hermeneutics (fleming et al 2003) and suggests that this approach represents movement away from  the  view  of  the  subjectivity  or  objectivity  of  the  circle.    in  fact,  gadamer  suggests,  the  circle,  ì describes  understanding  as  the  interplay  of  the movement  of  tradition  and  the movement  of  the  interpreterî  (gadamer 1976, p120). he contends that an individualís current experience and learning  contribute to their ability and means of understanding (mallery et al 1987).    similarly, gadamer rejected objectivism as a delusion of human thought. he argued that there is and  can  be  no  such  thing  as  objectivism  since  all  human  understanding  is  influenced  subjectively  (gadamer 1976, bleicher 1980).  in essence, gadamerís hermeneutics is not a method (prasad 2002,  fleming  et  al 2003), but  rather  a way of  life.   he  viewed hermeneutics  as  going beyond  a mere  methodological approach and rather occurring as a pervasive universality (grondin 1994).    gadamer on  the other hand, differed from heidegger  in his belief  that experience of other cultures  and manners of thinking can moderate the effects of oneís own prejudices by providing a contrasting  view.  he saw language as the key to unlocking the interpreter to other cultural view points and as a  consequence  language becomes  the key  to understanding  that  ì does not scientifically reconstruct a  speakerís  intention,  but  instead  mediates  between  the  interpreterís  immediate  horizon  and  his  emerging oneî  (mallery et al 1987, p6)      critical hermeneutics    habermas is a critical theorist.  his aim is to see beyond the interpretation of the text to the provision  of  a  critique  of  the  textís  ideological makeup  (prasad  2002).   this  view  has  resulted  in  critical  theorists  entering  into  debate  with  philosophical  hermeneuts,  such  as  gadamer,  with  a  view  to  increasing the critical focus of the practice of hermeneutics (prasad 2002). habermas (1968) and the  habermas/gadamer debate can be briefly summarised in the following three points:  1.  effects of  fore-structure & understanding (bleicher 1980).   habermas contests  these are  not  ëgivení  and  always  legitimate,  but  rather  can  be  built  through  self-reflection  (prasad 2002).  35   australasian journal of information systems     volume 13 number 2    may 2006  2.  the  status  of  language  (bleicher  1980).    habermas  (1968)  was  in  agreement  with  gadamer  regarding  the  relationship of  language  to understanding  and  interpretation,  and  the  fundamental  nature  of  language  as  a  foundation  for  social  institutions.  however,  he  does  not  accept  that  language  is  ontological  as gadamer  believes  but  rather that it is subject to alteration by social movements (prasad 2002).  3.  the justifiability of taking a critical position (bleicher 1980).  gadamer contended that it  is not possible  for a  truly critical position  to be  taken  from within  the culture under  study, and  that as previously stated,  the development of a narrow method should not  be the goal of hermeneutics but rather to enable understanding (prasad 2002).      phenomenological hermeneutics    ricouer  emphasises  the  complementary  nature  of  the  work  of  gadamer  (1976)  and  habermas.   rather than being opposed, ricouer contends that they are both necessary and can comfortably coexist (mallery et al 1986, prasad 2002).  this view is best summarised by ricouer himself:  the  consequence  for  hermeneutics  is  important:  we  can  no  longer  oppose  hermeneutics  and  the  critique  of  ideology.  the  critique  of  ideology  is  the  necessary detour which self-understanding must  take,  if  the  latter  is  to be formed  by  the matter of  the  text and not by  the prejudices of  the  reader  (ricouer 1981,  p144).    in  addition  to ricoeurís work  that  draws  together  the  arguments  of gadamer  and habermas,  his  theory  of  interpretation  attempted  to  reconcile  diltheyís  dichotomy  of  understanding  and  explanation  (mallery  et  al  1986,  scott-baumann  2003).   ricouer made  a  notable  contribution  to  hermeneutics through his attempts to acknowledge, understand and, where appropriate, reconcile the  apparently  diverse  positions  of  several  other  prominent  hermeneutic  scholars  (bleicher  1980).   ricoeurís  view  of  hermeneutics  incorporates  both  philosophy,  as  represented  by  ontology  and  epistemology  (bleicher 1980) while his  theory of  interpretation provides  a more general practical  approach  to  derive  meaning  from  text  although  it  cannot  be  described  as  a  method  (wiklund,  lindhulm  and lindstrom 2002).   ricoeurís notion of  the  text  is  any discourse  that  is  captured  in  writing.  he does not, as some authors claim, see everything as text. he does, however, argue there  are similarities between human action and text and for this reason they can be understood in similar  ways (langdridge 2004).    ricoeur  sees  the  text  as  the  objectification  of  the  participantís  subjectivities.    the  discourse  represents the participantís view of their world and once objectified in this way, can be  interpreted  by others in light of its own inherent meaning.  the intention of the originator of the discourse is of  less consequence  than  the meaning  that emerges directly  from  the  text  (leonardo 2003).   ricoeur  (1981) offers the following insights:  hermeneutics  can  be  defined  no  longer  as  an  inquiry  into  the  psychological  intentions which are hidden beneath  the  text, but  rather as  the explication of  the  being-in-the-world displayed by the text. what is to be interpreted in the text is the  proposed world which  i could  inhabit and  in which  i could project my ownmost  possibilities (p112).    this brief discussion of some prominent hermeneutic scholars and their work provides an overview  of  the  evolution  of  hermeneutics  over  time.    there  are  many  more  prominent  and  influential  hermeneutic  scholars  than have been  included here; however  those  listed provide  a  representative  cross-section  of  the  hermeneutic  tradition  and  its  historical  development.   more  importantly,  the  australasian journal of information systems     volume 13 number 2    may 2006     36  scholars described here are  the proponents whose approaches are most commonly discussed  in  is  research.  the use of hermeneutics in is research is discussed next.      hermeneutics in information systems research    hermeneutics  has  been  adopted  and  applied  in  many  disciplines  including  nursing,  education,  anthropology  and  information  systems  (myers  1995).    the  application  of  hermeneutics  in  information systems has an ever-expanding list of proponents. among the earliest of these is boland  (1985) who recognised the appropriateness of hermeneutics and the related area of phenomenology  for research in the is area:  information  systems  are  data  becoming  information  in  consciousness;  organizations are socially constructed  through  language, and our reasoning about  both these processes takes place in dialogue (boland 1985, p181).    hermeneutics  has  been  used  in  information  systems  for  a  variety  of  different  purposes,  table  2  provides an overview of hermeneutic is research,  its research focus and  the hermeneutic approach  adopted. it should be noted that the table is not complete but merely representative; authors such as  boland,  lee  and myers  have  published  more  than  has  been  cited  here.  in  addition  there  will,  undoubtedly, be other is researchers who have used hermeneutics and are not included in this list. it  can  be  seen  from  the  table  that  a  critical  approach  to  hermeneutics  has  been  favoured  by  is  researchers and, given the practical orientation of the discipline, this seems an appropriate choice.    it is interesting, however, to note that gadamer features so prominently in the is critical hermeneutic  discussion, given  that he saw his work conflicting with  the work of hermeneuts such as habermas  (1968), who are commonly categorised as critical.  gadamer did, however, acknowledge a need for  critical self-consciousness (mallery et al 1987).  ricouer (1981) identified that the views of gadamer  and habermas were actually mutually compatible and the inclusion of ricouerís work does help to  mediate  the  distance  between gadamer  and  the  practice  of  critical  hermeneutics.   the  following  section discusses  the choice  and application of a hermeneutic approach adopted  in an ongoing  it  governance project by way of illustration.      the research setting    one  australian  large  organisation  was  the  focus  of  this  case  study,  since  it  was  felt  that  large  organisations are more likely to have formal it governance processes.  the reason for limiting this  research  to within one geographical and national boundary was  to eliminate differences  in national  culture  in  it  (robbins  and  stylianou  2002,  kim  and  peterson  2003),  corporate  governance  mechanism  and  country-specific  regulatory  frameworks  (charkham  1995)  from  the  research  situation.    limiting the study to one organisation provided an opportunity for the researcher to become deeply  immersed  in  the  research  situation.    controlling  the  research  in  this  way  helped  to  reduce  any  potential resistance to the research from within the organisation that may have occurred where there  was a perception that issues of commercial sensitivity or security may be compromised by a detailed  cross-organisational study.      research topic citation hermeneutic approach butler, t. (1998), "towards a hermeneutic method for interpretive research in information systems", journal of information technology, vol. 13, pp. 285-300. phenomenological hermeneutics (derived from heidegger and gadamer) systems development butler, t. (2003), "an institutional perspective on developing and implementing intranet and internetbased information systems", information systems journal, vol. 13, pp. 209-231. phenomenological hermeneutics (derived from heidegger and gadamer) bussen, w. and myers, m. d. (1997), "executive information systems failure: a new zealand case study", journal of information technology, vol. 12, pp. 145-153. critical hermeneutics (derived from myers) myers, m. d. (1994), "a disaster for everyone to see: an interpretive analysis of a failed is project", accounting, management & information, vol. 4, no. 4, pp. 185201. critical or dialectical hermeneutics (derived from gadamer & ricoeur) implementation myers, m. e. (1995), "dialectical hermeneutics: a theoretical framework for the implementation of information systems", information systems journal, vol. 5, no. 1, pp. 51-70. critical or dialectical hermeneutics (derived from gadamer & ricoeur) communication richness lee, a. s. (1994). "electronic mail as a medium for rich communication: an empirical investigation using hermeneutic interpretation." mis quarterly vol. 18 no. 2, pp. 143157. hermeneutics (derived from ricouer) evaluation sheffield, jim (2005). “the evaluation of gss-enabled interventions: a habermasian perspective” in proceedings of the 38th hawaii international conference on system sciences (ed sprague r h), big island, hawaii. phenomenological (derived from habermas) boland jr, r. (1985). “phenomenology: a preferred approach to research on information systems”. research methods in information systems. e. mumford, r. hirscheim, g. fitzgerald and t. wood-harper. new york, north-holland: pp193-201. hermeneutics (derived from gadamer) boland jr, r. j. (1991). “information systems use as a hermeneutic process”. in information systems research: contemporary approaches & emergent traditions. h.-e. nissen, h. k. klein and r. hirscheim. amsterdam, north-holland: 439-464. hermeneutics (discussed with ricoeur, pepper & rorty) methodological lukaitis, s. and cybulski, j. (2004), "a hermeneutic analysis of the denver international airport baggage handling system", in proceedings of the information systems foundations workshop: constructing and criticising, (eds, hart, d. n. and gregor, s. d.) australian national university, canberra. critical hermeneutic (derived from gadamer & kidder) philosophical myers, m.d. (2004). “hermeneutics in information systems research”, in social theory and philosophy for information systems, mingers, j. & willcocks, l. chichester, west sussex. john wiley, pp 103-128 critical hermeneutic (derived from gadamer and ricoeur) table 2 hermeneutics in is research s420794 text box australasian journal of information systems volume 13 number 2 may 2006 s420794 text box 37 australasian journal of information systems     volume 13 number 2    may 2006     38  the organisation  that participated  in  this  research has  in excess of 100,000 employees, distributed  across approximately 1000 sites throughout australia. the organisation operates in the retail industry  sector.  the research reported focussed on the organisationís largest division consisting of more than  700 sites across the country.  the it unit supporting this division employs in excess of 600 staff.     data  were  gathered  from  semi-structured  interviews  and  organisational  artefacts  e.g.  company  documents.   the data were collected during  two stages, six months apart.   in addition  to using  the  rigorous approach of hermeneutics  to analyse  the data,  the collection of multiple data  types within  this research, collected at different times, from different participants enabled triangulation of the data  to  occur  (denzin  1970, macdonald  and tipton  1993),    thus  increasing  the  likelihood  of  a more  accurate  representation of  the phenomenon under  study.   twenty-five nominated  individuals  from  different levels, different units and sub-units and from both business and it within the organisation  were invited to participate in face-toñface interviews, to discuss their perceptions and views of how  it governance worked in theory and practice with the subject organisation.      each participant was  interviewed  in  the  first phase of data  collection.    in order  to  ensure  a more  complete  picture  of  the  organisational  reality,  the  interviewees  were  drawn  from  senior  it  management, it staff, senior business managers and business staff.  this cross section of roles was  selected in part, because of the recognised ëgap between it and the rest of the business that exists in  many organisations (peppard 2001), and in part because of the nature of it governance, which as a  sub-set of corporate governance  is not  just a matter  for  the  it area but  is very much an  issue  for  business units and the entire organisation.     the  second  stage  of  data  collection  was  conducted  in  order  to  explore,  test  and  add  detail  and  richness  to  the  insights  generated  from  the  first  round  of  data  collection.   the  three  participants  selected for this round represented areas within the organisation included in the study, i.e. one from  business, one from it and one who served in a checking or monitoring role.  recent and long term  employees were interviewed in round two.  the second round of data collection was completed by  telephone.  in all, 28 interviews were conducted over the two stages of the data collection.      choosing and applying a hermeneutic approach    the research  in progress  that  is  the focus of  this paper adopts an  interpretivist approach  to explore  the  prescribed  versus  practiced  it  governance  and  the  attendant  consequences  of  the  differences  between  theory  and  praxis  in  a  large  organisation  in australia.   data was  collected  through  two  rounds  of  semi-structured  interviews  conducted  with  employees  who  were  involved  in  it  governance  at  various  levels  and  in  various departments  throughout  the organisation  and  through  analysis of company documents. the research seeks to answer the following research questions:  1.  what is the nature of it governance?  a)  as expressed by management (prescribed) and  b)  as  it occurs  in  the  actual manifestation of  (practiced)  it governance within  the  organisation  2.  how  do  the  differences  between  theory  and  praxis manifest  themselves  within  the  organisation?  3.  what  are  the  consequences  of  the  differences  between  theory  and  praxis  to  the  organisation?    the  use  of  hermeneutics was  first  considered  since  it  has  been  suggested  that  hermeneutics  is  a  recognised  framework  for  the  analysis  of  organisations  (bryman  1989).    in  addition,  frost  et  al  39   australasian journal of information systems     volume 13 number 2    may 2006  (1985) elaborate that it is particularly useful when looking at organisational culture and others have  applied it to the analysis of socio-technical interactions (barley 1986, harvey and myers 2002).  the  hermeneutic  approach  was  adopted  in  this  research  as  being  most  appropriate  because  of  its  suitability to the conduct of research within socio-technical and organisational contexts.    moreover the epistemology applied in this research is interpretivist constructivism and draws heavily  from hermeneutics.  in addition, within the interpretivist epistemology, a hermeneutic approach was  considered  the most  appropriate  choice  for  this  research  supporting  as  it does  the  construction of  understanding from the analysis and interpretation of the data under study.  dilthey considered:  ö   the  central difficulty of  all  interpretative  skill. the whole of  a work must be  understood  from  the  individual  words  and  their  combinations,  and  yet  the  full  comprehension of the details presupposes the understanding of the whole (dilthey  1976, p115).  given that a number of hermeneutic approaches have been adopted in is research, the next challenge  was  to consider  the different points of view.   after much consideration, an approach  informed by  both ricoeur (1981) and gadamer (1976) was adopted for the following reasons.  1.  it was  felt  that  the  hermeneutics  advocated  by ricoeur  offered  richness  for  the  study and interpretation of complex social situations such as those investigated  in  this  study  in  that  it  draws  together  the  work  and  thoughts  of  prominent  hermeneutic  scholars  and  reconciles  the  two  foci  of  understanding  and  explanation, the specific and the general.  2.  ricoeurís focus on the text as the vehicle of meaning, whilst balancing this against  a  need  for  critical  analysis  and  judgement,  supported  the  selection  of  a  hermeneutic approach inspired by his work.  3.  gadamer contributed his  recognition of  the need  for awareness of prejudices, his  view of  the importance of questions, and his concept of horizon and fusion of  horizons.    a diagrammatic representation of the research philosophy and design appears in figure 1.  ricoeurís  concepts  of  distanciation,  appropriation,  understanding,  explanation  and  the  application  of  a  critically  self  reflexive  and  questioning  approach were  used  to  inform  this  research  in  progress.   these concepts are discussed next.    distanciation, also  referred  to as historicality by myers  (1994),  refers  to  the distance between  the  text and its origins.  distance from a text can exist as a consequence of temporal, cultural, or lingual  factors, due  to  the  intention of  the author  to deceive or mislead  the  interpreter or due  to changing  social  practices  (lukaitis  and  cybulski  2004).    ricoeur  sees  that  the  starting  point  of  any  interpretation  should  be  the  text.    ricoeurís  concept  of  distanciation  differs  from  the  ëtemporal  distanceí of gadamer through the movement to a text focus (langdridge 2004).  in ricouerís words:  for  the  distanciation  revealed  by  writing  is  already  present  in  discourse  itself,  which contains the seeds of the distanciation of the said from the saying, to follow  hegelís  famous  analysis  at  the  beginning  of  the  phenomenology  of mind:  the  saying vanishes, but the said persists (ricoeur 1981, p92)    ricouer also encourages  the  interpreter  to distance  themselves  from  their own prejudices and preconceived  understandings  and  also  from  their  reading  of  the  text.    he  advocates  a  ëcritique  of  prejudiceí in order that the interpreter may proceed through the analysis and interpretation in a self  critical and self-reflexive manner (langdridge 2004).  distanciation from oneself demands that the appropriation of the proposed worlds  offered by the text passes through the disappropriation of the self. the critique of  australasian journal of information systems     volume 13 number 2    may 2006     40  false consciousness can  thus become an  integral part of hermeneutics, conferring  upon  the  critique  of  ideology  that  meta-hermeneutical  dimension  habermas  assigns to it (ricouer 1981, p94-95).    the  process  of  interpretation  thus  becomes  a  dialectic  between  distanciation  and  appropriation  (geanellos 2000).    fusion of horizons horizon understanding explanation appropriation distanciation horizon prejudices c rit ic al  r ef le xi vi ty q ue st io ns     figure 1 - reflexive hermeneutic research approach  derived from ricouer (1981) and gadamer (1976)    in this study, it was envisioned that distanciation of the text would help ensure that during the data  analysis the meaning was derived from the text and that it emerged from the conversation between  the text and the researcher (prasad 2002).  this process was extended to the core of the data analysis  process through the construction of additional texts, representing contexts and concepts derived from  the data that informed and represented the construction of meaning (harvey and myer 2002).  this is  a  recursive process  that could continue  indefinitely  (harvey and myer 2002).   it  is  the  role of  the  interpreter  to determine  the appropriate moment  to finalise  the  interpretation.   this occurred when  there was minimal change in  interpretation and no new themes or concepts emerged from the data.   intercoder  reliability  was  used  within  this  study  to  provide  confidence  in  the  quality  of  the  interpretation.  in the words of harvey and myer (2002):  every meaning is constructed, even through the very constructive act of seeking to  deconstruct,  and  the  process whereby  that  textual  interpretation  occurs must  be  self-critically reflected on (ricouer 1974) (harvey and myer 2002, p176)    the inclusion of ricouerís critical self-reflexivity was an important element in this research study.  it  encouraged  the  researchers  to  continually  check  and  recheck  the  appropriateness of  their  analysis  against  the  gradually  evolving meanings  derived  from  the  data  under  analysis.    the  researchers  sought  at  all  times  to  remain  critical  and  self-reflexive.    this  self-reflexivity  informed  the  recruitment  of  research  participants  from  a  broad  range  of  roles  and  perspectives  within  the  organisation, it also enabled the researcher to draw concepts and questions from the it governance  41   australasian journal of information systems     volume 13 number 2    may 2006  literature  to  develop  the  interview  guide,  whilst  still  remaining  conscious  of  the  potential  that  exposure to the literature might impact upon their views of the interview texts.    ricouerís concept of appropriation refers to the process of making the text oneís own (boland 1991,  lee  1994,  geanellos  2000,  langdridge  2004).    appropriation  refers  to  the  researcher  deriving  meaning  from  the  text,  independently  of  the  historical,  social,  cultural  factors  that  influenced  its  creation, and independently of authorial intent.  ricouer saw this not as a reaching back to determine  the meaning of the text but rather as a moving forward and exploring the area in front of the text and  in  this  way  challenging,  shaping  and  altering  the  horizons  and  prejudices  of  the  researcher  (langdridge 2004).  thus enabling the hermeneutic circle to begin again, with the researcher gaining  a  new  understanding  of  herself,  with  which  to  recommence  the  process  of  understanding  the  meaning  of  the  text  (geanellos  2000).    in  the  course  of  this  research  appropriation  is  another  mechanism that was used to challenge and expand prejudices and preconceptions, resulting as it did  in a movement of the researchers own horizon.    ricoeurís theory of interpretation is underpinned by two concepts:  1.  explanation, or what the text says; and  2.  understanding, or what the text talks about (geanellos 2000, p114)    the explanation of the text refers to the detailed and careful analysis of the individual components  of the text, while understanding the text requires the researcher to lift their focus to take in the whole  meaning of the complete text (geanellos 2000).  for the hermeneutic researcher this process should  be a circular and  iterative one,  in  this way  the depth of  the  interpretation  is strengthened, moving  from  simple  initial  interpretations  to  deeper  and  more  complex  subsequent  meanings.    in  this  research this approach was adopted to analyse the data.    gadamer sees the use of questions as key to enabling a valid interpretation (butler 1998), according  to gadamer understanding  is not possible without  the activity of questioning  (fleming et al 2003,  p117).   gadamer  sees  questions  as  a means  to  open  possibilities  and  to  keep  them  open  (butler  1998).    they  can  also  be  used  to  offset  or  mediate  the  impact  of  a  researcherís  pre-existing  prejudices.  questions can do this by assisting the researcher to remain constantly in touch with what  those prejudices are and  to assist with not  taking anything  for granted.    in  this  research questions  were used, for example, to challenge the researchers existing prejudices, to aid the identification of  an appropriate participant profile and to inform and develop the research interview guide.    in addition, gadamerís belief  in  the plurality of meaning was acknowledged as adding strength  to  ricouerís approach that implies that one valid truth can be found (geanellos 2000).  in this research  the  researcher  made  a  conscious  effort  to  uncover,  communicate  and  encompass  the  broadest  possible plurality of meaning  into  the  research  study  in order  that both  the data  and  the  resulting  findings are of the highest quality.  this attempt to remain open to plurality of meaning throughout  the  research  process  manifested  in  the  number  and  selection  of  interviewees,  the  selection  of  organisational artefacts and has been extended  to  the selection of research  literature  the researcher  incorporated into the study.    gadamer was  strongly  aware  of  the  importance  of  acknowledging  prejudices  (butler  1998)  and  actions  and  activities  to  highlight  the  researcherís  prejudices  have  been  incorporated  into  the  research approach in order to assist the researcher to constantly challenge her initial and subsequent  interpretations.  this was done by:  1.  including intercoder checking into the data analysis,  2.  continually questioning conclusions and ideas that emerged from the data,  australasian journal of information systems     volume 13 number 2    may 2006     42  3.  seeking participants  from a broad  range of  roles within  the organisation, so  that multiple  perspectives and views were discovered and as a consequence questioned and  integrated,  and  4.  informing the researcherís review of the existing literature and how that impacted upon the  researcherís opinions and views.    finally gadamerís (1976) notion of horizon and  the eventual fusion of horizons  that occurs at  the  moment understanding is reached (butler 1998) was applied in this research.  gadamer saw horizons  as  everything  that  can  be  seen  from  one  particular  point  (butler  1998,  fleming  et  al  2003).   a  horizon is a constantly changing phenomenon.  it is suggested that a personís view is coloured and  varied by their cultural background, their historical understanding of the phenomenon being viewed,  and  by  social  and  traditional  factors.    it  is  through  challenging  and working  through  these  preexisting  impacts  and  prejudices  that  oneís  horizon  can move  over  time.    the  result  of working  toward common understandings in this way can be a fusion of horizons between individuals.  when  there is a fusion of horizons, there also is understanding (butler 1998).    having  the  researcher  complete  the  data  collection  and  transcription  of  all  interviews  and  other  documents  for  analysis within  this  study was  a  decision  also  based  on gadamerís  hermeneutics.   gadamer  believed  that  understanding  is  gained  through  dialogue,  and  that  activity,  such  as  conducting  and  transcribing  interviews,  should  not  be  delegated  as  they  contribute  to  a  fuller  immersion in the topic and a greater opportunity for shared understanding through dialogue between  researcher  and  interviewee  (fleming  et  al  2003).   this  in  turn  should  support movement  toward  fusion  of  horizons,  or  greater  understanding  between  researcher  and  participants.  it  is  should  be  noted though that in the words of myers (1994):  the hermeneutic-dialectic approach does not accept uncritically participantsí own  views on a particular topic; rather, it recognises that the researcher (or developer)  attempts  to  critically  evaluate  and  transform  social  reality,  a  reality  that  is  historically constituted (p191).    at some stage the researcher must call a halt to what gadamer believed should be an endless cycle  of interpretation, reach the final understandings and record the results of the research. in pursuit of  valid research outcomes and to help in establishing an endpoint to the interpretation of the data, the  authors  chose  to  employ  the  technique  of  intercoder  checking.  this  is  an  additional  step  that  introduces added rigour into the research method and relies on the use of questions, to help identify,  challenge and overcome  interpreter prejudices and  to confirm  the appropriate point  in  the research  process to discontinue the cycles of interpretation.  thus the hermeneutic circle is closed.      results of using the hermeneutic approach    it  is  important  to  recognise  that  the  interpretivist epistemology adopted  in  this  research  is not one  clear definable approach.  rather interpretivism is moulded and formed by the intent and application  of the researcher (schwandt 1994).  for example, it is commonly acknowledged that hermeneutics is  one  form of  interpretivism  (lee 1994).   other  forms of  interpretivism  include postmodernism and  deconstructionism  (kein  and  myers  1999)  klein  and  myers  (1999,  p70)  go  on  to  describe  interpretivism  as  ì a  family  of  paradigms  too  large  and  diverse  to  tackle  for  this  projectî .    the  purpose  of  drawing  the  readersí  attention  to  the  diversity  of  forms  is  to  illustrate what might  be  described  as  the  illusory  nature  of  interpretivism.    illusory  because  the  purpose  of  the  epistemological level is to consider issues of being and knowing rather than remain focussed on the  minutia of  the mechanism by which knowledge or understanding  is derived.   however, because of  43   australasian journal of information systems     volume 13 number 2    may 2006  the nature of interpretivism and the need for the researcherís epistemological intention and approach  to  be  clear,  it  is  important  for  the  researcher  to  carefully  document  and  record  the  ì process  of  meaning  constructionî   (schwandt  1994,  p118).    in  this  research,  the  use  of  a  clearly  defined  hermeneutic  approach  proved  beneficial  in  this  regard  enabling  the  researcher  to  maintain  a  consistent philosophical position,  thus  ensuring  the basic  integrity  and  internal  consistency of  the  research process.     another major advantage of the hermeneutic approach used in this research is in its recognition and  embracing of prejudice.  in this research, the primary researcher spent considerable time and effort in  becoming  aware  of  her  own  prejudices  and  recognising  these  as  the  point  where  understanding  begins  (myers 2004). this  researcher became a part of  the  research situation.    interview data was  socially constructed in the exchange between researcher and participant and again in the process of  transcription  and  analysis.    the  hermeneutic  approach  provided  recognition  of  this  reality  and  a  useful mechanism  for  raising  an  awareness of  these prejudices  such  that meaning  in  the  research  became apparent (myers 2004).      the  hermeneutic  approach  also  offered  benefits  in  increasing  the  validity  of  the  research.    as  indicated earlier, carefully and accurately documenting  the research process  is an  important step  in  interpretivist  research.    in  this  research,  the  development  and  application  of  a  clearly  articulated  hermeneutic  approach  supported  detailed  recording  of  the  researcherís  reflections  throughout  the  research  journey.   transferability  refers  to  relevance of  the  research  results  to other  settings.   to  facilitate  transferability  in  this  study,  the  research  process,  setting,  participants  (burke  johnson  1997),  and  the  findings  were  detailed  in  a  richer,  more  accurate  and  more  complete  manner  supported  by  the  hermeneutic  approach  (guba  and  lincoln  1989, trochim  2001).   this  detailed  account  should  enable  a  reader  of  the  research  results  to  make  their  own  decision  about  the  appropriateness of the research findings for their situation.    since dependability relies on the researcherís rich and detailed description of processes, methods and  approaches  used  (flick  2002)  the  writing  process  in  this  research  began  early  to  ensure  higher  degrees of accuracy and the writing continued thoughtfully throughout the research journey.     in  addition,  the  conscientious  application  of  intercoder  reliability  by  both  researchers  increases  dependability  in  this  research by providing a  structured opportunity  for  feedback and  to challenge  the primary researcherís evolving interpretations and analyses (flick 2002).     the critical reflexivity demanded by  the hermeneutic approach also enhanced  the dependability of  the  research  process  through  continually  challenging  the  researchers  to  reconsider  and  review  emerging interpretations (flick 2002).    the  rich,  detailed  documentation  and  interpretations  will  assist  the  reader  to  clearly  follow  the  research decisions made and will contribute to the auditability of the process.     applying  the hermeneutic  approach  also  supported  confirmability,  i.e.,  the  extent  to which others  can corroborate the results of the research (trochim 2001).  this occurred through the use of:  1.  intercoder reliability during the data analysis phase,   2.  careful and detailed documentation of the process,   3.  seeking  out  and  reviewing  occurrences  in  the  data  that  did  not  support  the  emerging  research findings, negative cases (burke johnson 1997, mays and pope 2000).     australasian journal of information systems     volume 13 number 2    may 2006     44  in  addition  the  critical  reflexivity  inherent  within  the  hermeneutic  approach  used  offered  an  opportunity  for  continuous  and  productive  challenging  of  the  research  process.  this  enabled  identification of paradoxes and contradictions in interpretation, and how it evolved over the period of  the research, and the subsequent documentation of the research (mays and pope 2000, lincoln and  guba 2003).    the overall benefit gained in this research from the adoption of a hermeneutic approach was a more  robust research framework that was synergistic with the philosophical underpinnings of the research.   this enabled the researchers to move with confidence through the research process knowing that at  every stage in the journey each step was designed to ensure greater immersion in the research and to  enhance  the  richness  of  the  data  collection  and  analysis.    perhaps  equally  as  important was  the  knowledge  that  any prejudices held or  assumptions made would be  subject  to  challenge,  through  questioning, critical self-reflexivity or the use of other techniques, such as intercoder checking.      concluding remarks    in  exploring  the most  appropriate  of  the  various  qualitative  research  techniques  for  an  ongoing  research project in it governance, the authors happened upon a most interesting and exciting option  and were thus introduced to the intricacies of the ancient art of hermeneutics.  given the anticipated  richness and rigour the application of hermeneutics promised to add to their qualitative process, the  authors  persevered  through  many  difficult  and  complex  descriptions  of  hermeneutics  until  they  gained  a  level  of  comfort  and  confidence  in  its  applicability  and  use.  the  comprehensive  chronological  discussion  of  its  evolution  and  comparative  discussion  of  the  various  hermeneutic  approaches provide a greater understanding of the breadth and depth of hermeneutics and the distinct  applicability of its various approaches to case studies in general and is research in particular.  this  report on the application of one approach to hermeneutics in a case study of it governance in a large  australian organisation  is an  attempt  to ease  the way of others who may be considering a  similar  choice.    in  addition,  it  illustrates  the  unique  benefits  to  be  gained  by  using  hermeneutics  in  is  research to strengthen the dependability and validity of qualitative inquiries and hopefully promotes  an increased understanding of the various approaches to hermeneutics.      is research is uniquely poised to benefit greatly from hermeneutics as a valuable qualitative research  tool.    the  is  research  community  is  continually  called  upon  to  conduct  inquiries  into  emerging  phenomena.   such  inquiries could benefit  from  robust  tools  that go beyond descriptive analysis  to  rigorously  explain  their  impact.    hermeneutics  is  one  such  tool  that  will  allow  qualitative  is  researchers  to  transcend  the  usual  everyday  descriptive  character  of  qualitative  understanding  of  these  emerging  phenomena  to  identify  paradoxes  and  contradictions,  challenge  assumptions  and  plumb  the depths of  inquiry  to develop strong, dependable explanatory foundations  for subsequent  quantitative inquiries.    in  sharing  the  authorsí  insights  into  the  challenges  of  understanding,  choosing  and  applying  hermeneutics  as  an  appropriate  research  philosophy  and  data  analysis  tool  this  paper  is  the  first  attempt to demystify the often misunderstood and overrated complexities of applying hermeneutics  to  is  research  and  is  designed  to  encourage  more  widespread  acceptance  and  application  of  hermeneutics within the broader is research community.      45   australasian journal of information systems     volume 13 number 2    may 2006  references    barley, s. r. (1986). ì technology as an occasion for restructuring: evidence from observations of  ct scanners and the social order of radiology departmentsî . administrative science  quarterly, vol 31, iss 1, pp 78-108.  bleicher, j. (1980). contemporary hermeneutics: hermeneutics as method, philosophy and critique.  routledge & kegan paul, london.  boland jr, r. (1985). ì phenomenology: a preferred approach to research on information  systemsî . research methods in information systems. e. mumford, r. hirscheim, g.  fitzgerald and t. wood-harper, (eds.)., north-holland, new york: pp193-201.  boland jr, r. j. (1991). ì information systems use as a hermeneutic processî . information systems  research: contemporary approaches & emergent traditions. in h-e. nissen, h. k. klein  and r. hirscheim, (eds.). north-holland, amsterdam: pp 439-464.  bryman, a. (1989). research methods and organization studies, unwin hyman, london.  burke johnson, r. (1997). ì examining the validity structure of qualitative researchî . education  118(2): 282-292.  bussen, w. and m. d. myers (1997). ì executive information systems failure: a new zealand case  studyî . journal of information technology vol 12, pp 145-153.  butler, t. (1998). ì towards a hermeneutic method for interpretive research in information systemsî .  journal of information technology vol 13,pp 285-300.  butler, t. (2003). ì an institutional perspective on developing and implementing intranet - and  internet- based information systemsî . information systems journal vol 13, pp 209-231.  charkham, j. p. (1995). keeping good company: a study of corporate governance in five  countries. oxford, oxford university press.  demeterio, f. p. a. i. (2001). ì dialectical hermeneuticsî . diwatao vol 1, iss 1:  http://www.geocities.com/philodept/diwatao/dialectical_hermeneutics.htm, accessed 25  august 2003.  denzin, n. k. (1970). the research act: a theoretical introduction to sociological methods.  chicago, aldine publishing company.  dilthey, w. (1976). ì the rise of hermeneuticsî . in critical sociology, selected readings. p.  connerton, (ed.). penguin books ltd, harmondsworth.  fleming, v., u. gaidys, and y robb. (2003). ì hermeneutic research in nursing: developing a  gadamerian - based research methodî . nursing inquiry vol 10, iss 2: pp 113-120.  flick, u. (2002). an introduction to qualitative research: second edition. london, sage  publications.  frost, p. j., l. f. moore, m.r. louis, c.c. lundberg, j martin, (eds.) (1985). organizational  culture. sage, beverly hills, ca.  gadamer, h.-g. (1976). ì the historicity of understandingî . in critical sociology: selected  readings. p. connerton, (ed.). penguin books ltd, harmondsworth.  geanellos, r. (2000). ì exploring ricoeur's hermeneutic theory of interpretation as a method of  analysing research textsî . nursing inquiry vol 7, pp 112-119.  grondin, j. (1994). introduction to philosophical hermeneutics.yale university press, new haven.  guba, e. g. and y. s. lincoln (1989). fourth generation evaluation. newbury park, california,  sage.  habermas, j. (1968).  knowledge & human interest.  polity press.  hamilton, d. (1994). ì traditions, preferences, and postures in applied qualitative researchî  in.  handbook of qualitative research. n. k. denzin and y. s. lincoln, (eds.), sage  publications, thousand oaks.  australasian journal of information systems     volume 13 number 2    may 2006     46  harvey, l. j. and m. d. myers, (eds.) (2002). ì scholarship and practice: the contribution of  ethnographic research methods to bridging the gapî  in. qualitative research in information  systems: a reader. sage publications london.  kim, c. s. and d. k. peterson (2003). ì a comparison of the perceived importance of information  systems development strategies by developers from the united states and koreaî .  information resources management journal 16(2): 1-18.  klein, h. k. and m. d. myers (1999). ì a set of principles for conducting and evaluating interpretive  field studies in information systemsî . mis quarterly 23(1): 67-93.  langdridge, d. (2004). ì the hermeneutic phenomenology of paul ricoeurî . existential analysis  vol 15, iss 2, pp 243-255.  lee, a. s. (1994). ì electronic mail as a medium for rich communication: an empirical  investigation using hermeneutic interpretationî . mis quarterly  vol 18, iss 2, pp 143-157.  leonardo, z. (2003). ì interpretation and the problem of domination: paul ricoeur's hermeneuticsî .  studies in philosophy and education, vol 22, pp 329-350.  lincoln, y. s. and e. g. guba (2003). ì paradigmatic controversies, contradictions, and emerging  confluencesî . in the landscape of qualitative research: theories and issues. n. k. denzin  and y. s. lincoln, (eds.). thousand oaks, sage publications: 253-291.  lukaitis, s. and j. cybulski (2004). ì a hermeneutic analysis of the denver international airport  baggage handling systemî . proceedings of the information systems foundations workshop:  constructing and criticising, australian national university, canberra.  macdonald, k. and c. tipton (1993). ì using documentsî . in researching social life. n. gilbert,  (ed.). london, sage publications: 186-200.  mallery, j. c., r. hurwitz, and g duffy. (1987). ì hermeneutics: from textual explication to  computer understanding?î  in the encyclopaedia of artificial intelligence. s. shapiro, (ed.).  john wiley and sons, new york.  mays, n. and c. pope (2000). ì qualitative research in health care: assessing quality in qualitative  researchî . british medical journal 320(7226): 50-52.  myers, m. d. (1994). a disaster for everyone to see: an interpretive analysis of a failed is  project. accounting, management & information vol 4, iss 4, pp 185-201.  myers, m. d. (1995). ì dialectical hermeneutics: a theoretical framework for the implementation  of information systemsî . information systems journal vol 5, iss 1, pp 51-70.  myers, m.d. (2004). ì hermeneutics in information systems researchî , in social theory and  philosophy for information systems, mingers, j. & willcocks, l. chichester, john wiley,  west sussex. pp 103-128.  myers, m. d. and d. avison, (eds.) (2002). ì introducing qualitative methodsî . qualitative  research in information systems: a reader. sage publications london.  olson, d. l. and j. carlisle (2001). ì hermeneutics in information systemsî . proceedings of the  americas conference on information systems, boston, massachusetts.  peppard, j. (2001). ì bridging the gap between the is organization and the rest of the business:  plotting a routeî . info systems journal 11: 249-270.  prasad, a. (2002). ì the contest over meaning: hermeneutics as an interpretive methodology for  understanding textsî . organizational research methods vol 5, iss 1, pp 12-33.  ricoeur, p. (1981). hermeneutics and the human sciences: essays on language, action and  interpretation. j. b. thompson, (ed.). cambridge university press, cambridge.  robbins, s. s. and stylianou, a. c. (2002), "a study of cultural differences in global corporate  web sites", the journal of computer information systems, vol. 42, no. 2, pp. 3-9.  scott-baumann, a. (2003). ì reconstructive hermeneutical philosophy: return ticket to the human  conditionî . philosophy social criticism vol 29, iss 6, pp 703-727.  47   australasian journal of information systems     volume 13 number 2    may 2006  schwandt, t. a. (1994). ì constructivist, interpretivist approaches to human inquiryî . in handbook  of qualitative research. n. k. denzin and y. s. lincoln, (eds.). thousand oaks, sage  publications.  trochim, w. m. k. (2001). ì qualitative validityî . research methods knowledge base  http://trochim.human.cornell.edu/kb/qualval.htm, accessed 2001(10 september 2001).  wiklund, l., l. lindholm, and u lindstrom. (2002). ì hermeneutics and narration: a way to deal  with qualitative dataî . nursing inquiry vol 9, iss 2, pp 114-125.    australasian journal of information systems     volume 13 number 2    may 2006     48  microsoft word 13 1 marina.doc ajis vol 13, no. 1 september 2005 58 a longitudinal study of isp reactions to australian internet content regulation laws sigi goode school of business and information management faculty of economics and commerce australian national university acton act 0200 sigi.goode@anu.edu.au abstract this paper discusses a longitudinal study which surveys a targeted selection of australian isps to determine both initial and subsequent effects of and attitudes towards the legislation. the paper observes that, initially, isps were generally opposed to the legislation, offering stiff opposition to its introduction. the initial results suggested dissatisfaction with the legislation on the part of isps, and foreshadowed adverse effects on the online industry. concerns were also raised that the legislation would not be effective. two years later, however, isps had generally observed little change in operations, arguing that the legislation had had little overall effect. the study also raises a number of interesting issues that are outside the scope of this paper. these issues merit further research. keywords internet regulation, electronic commerce, internet service provider introduction when the internet began receiving patronage as a commercial (vadapalli and ramamurthy 1997, hallahan 2004), educational (stair and reynolds 1998), informational (applegate et al. 1996, xiao et al. 2004), social (brien 1996, fan 2005), and academic (salam et al. 1997) tool, observers around the world saw the number of internet users increase at a dramatic rate (chiou 2004). coincidentally, a number of governments began making attempts to regulate the access to, and provision of, online content (applegate et al. 1996, strauss and rogerson 2002). for some time, the research literature saw substantial debate over the efficacy, value and effect of such regulation. teo et al. (1997) discussed the importance of governmental and legislative support in the adoption of technology in industry, and note the positive effects of government regulation and endorsement of online commerce in singapore. fan (2005) and soh et al. (1997) observed similar findings within asia in general. hulsink and van bolhuis (1998) argued that the effective implementation and policing of such legislation would enable e-commerce strategies to flourish by improving trust and customer confidence (consistent with gefen and straub 2004). conversely, the adverse effects of enforced regulation may be alleviated through voluntary co-operation of firms with regulators (xiao et al. 2004, williams 1994) or with each other through co-optation (barringer and harrison 2000). also, internet use and policy development may vary between countries, so legislation for one state may not work for another (baumer et al. 2004). at the same time, recent global events have changed the way personal and national security are viewed. this concern has extended to the world wide web, as countries attempt to legislate their national online communications infrastructures. isps are particularly important in this strategy for a number of reasons. first, despite recent growth in the uptake of broadband (stern et al. 2004) and wireless services, much online access still requires ajis vol 13, no. 1 september 2005 59 conventional public switched telephone networks (pstn) (naldi 2002). internet service providers (isps) offer users internet in return for payment (chiou 2004) and, accordingly, the isp is very much the “entry point to the e-market” (francalanci et al. 2001:145). many have argued that the isp market should be regulated in some way. australian users are able to complain to the telecommunications industry ombudsman (tio), however that body has been largely powerless to assist users with bankrupt isps or online content issues. to this end, the internet industry association of australia (the iia) was formed as a membership and lobby group for organisations in the online industry (including, for example, website developers, law firms and isps). the group has also acted in an advisory capacity to the australian broadcasting authority (aba) in the development of this online legislation. other than the guidelines provided by these bodies, very little legislation pertains directly to isps and their operation. australia is one country which has commenced a program of internet regulation, by prescribing a ratings scheme for online content at the isp source. this legislative bill was among the first in the world, and some longitudinal analysis of its efficacy would be benefit practitioners and policy makers. in addition, the popularity of the internet and the potential effects of this legislation suggest that this analysis would benefit to the information systems research body by providing new evidence as to the value of online content regulation. this paper documents the findings of an ongoing study into network regulation. the first stage of the study is discussed in goode (2002), and the reader is directed to that work for additional context. the rest of this paper is structured as follows. in order to “report sufficient contextual and environmental conditions of their research to enable other researchers to judge the limitations of the conclusions” (nunamaker et al. 1991), the next section provides a brief overview of internet service providers and the new aba legislation. the study’s research method is then discussed. the paper then discusses the results, which is followed by analysis and implications. finally, conclusions and areas for further research are explored. definitions it is first necessary to establish nomenclature and definitions for the terms used in the paper, especially with regard to legal terminology. hancher and moran (1989:271) define regulation as “involving...the design of general rules, the creation of institutions responsible for their implementation, the clarification of the exact meaning of a general rule in particular circumstances and the enforcement of the rule in those circumstances”. the term, “bill” describes a draft statute that has not yet been passed through parliament (cook et al. 2005). the term, “act” describes a statute or an element of legislation (cook et al. 2005). “legislation” describes both the acts enacted by parliament, described above, and the subordinate laws developed by authorised groups (such as a city council or the governor-general) (cook et al. 2005). the aba legislation the legislation was an amendment to the australian broadcast services act (1992) and was designed to work in concert with an isp code of practice. the legislation was designed to restrict online “content that is illegal or highly offensive, or maybe harmful to children” at the access provider. the legislation does not apply to private networks or distribution media (such as electronic mail or lans). the legislation allows members of the public to ajis vol 13, no. 1 september 2005 60 complain to the australian broadcasting authority (aba) about material on the internet that they find offensive. once a complaint has been made, the aba investigates the content and assigns it a viewing classification similar to those awarded to films (for instance, “x”, “r”, or “pg”). if the material receives an “x” rating, a notice may be issued, based on “technical and financial feasibility”, to either remove the material (if the host is local) or block the material (if the material is outside of australia). material that receives an “r” rating is deemed unsuitable for general access and must be restricted by way of an age verification system. once the aba has issued such an interim removal notice, the isp has a full business day to comply with the notice. if the isp does not comply, fines of aus$5,500 for an individual or aus$27,500 for a company (adams and burke 1999) may be imposed. the legislation specified the creation of an isp code of practice, which is similar in many respects to the aba television code of practice (originally designed to give television viewers an avenue for complaint). this code of practice, developed by the iia in conjunction with the aba, is not mandatory until the aba directs a given isp to the code, whereupon the code must be followed. the code of practice makes provision for dealing with customer complaints and makes special distinction between the isp’s responsibilities with respect to content available outside of australia and sites hosted within australia. the code also describes, in broad terms, methods for filtering or blocking material through client-side software such as net nanny and cyber patrol. content monitoring and sitespecific online age verification systems are also discussed. the bill, in its current form, has been in development for several years, and planning began in late 1995 (de souza 1999) with the first senate committee investigation into the regulation of online services. table 1 gives a chronological overview of the legislation’s development. development involved submissions from lobby groups, the commonwealth scientific and industrial research organisation (csiro) and the internet industry association. alternatives to the legislation, including client-side filtering software such as net nanny, were deemed too technical and too difficult to enforce for widespread use. table 1: timeline of significant events during the development of internet regulation legislation 21-jun-91 senate select community standards committee established. initially tasked with reporting on telephone sex services, the committee will eventually also comment on television material and computer game content. 24-jul-95 the minister for communications and the arts directs the aba to conduct an investigation into the content of online information and entertainment services, including internet services. 2-mar-96 1996 australian federal election. liberal/national party coalition wins power. policy statements do not mention the internet or its regulation. 1-jun-97 senate select community standards committee releases "report on the regulation of computer on-line services", which recommends support for criminal prosecution and police intervention of online systems 27-aug-97 select community standards committee is disbanded. the senate select committee on information technologies is formed in its place. ajis vol 13, no. 1 september 2005 61 16-dec-97 australian chief censor comments that it will be impossible to ban internet content, regardless of legislation 14-jun-98 senate select committee on information technologies scheduled to convened to examine internet self-regulation. imminent federal election means the meeting does not take place. 3-oct-98 1998 australian federal election. liberal/national party coalition wins power, featuring internet regulation policy as part of election platform 19-mar-99 minister for communications, it and the arts announces new content classification bill. days later, the iia meet with the senator and other government officials to clarify the bill 21-apr-99 first senate reading of bill 27-apr-99 senate select committee on information technologies meets for the first time to discuss the bill 11-may-99 senate select committee on information technologies tables findings from meeting of april 27, arguing that the bill should be approved in its current form 26-may-99 senate passes broadcasting services amendment bill 30-jun-99 house of representatives passes broadcasting services amendment bill 4-jul-99 first survey of this study takes place 16-dec-99 aba registers isp content codes, as required by the broadcasting services amendment 1-jan-00 broadcasting services amendment bill and isp content codes come into effect 8-nov-00 south australian attorney-general introduces a bill into south australian state parliament "aimed at making it illegal to make offensive material available on the internet" 8-mar-01 internet industry code of practice registered with aba 1-jun-01 isp content codes due for formal review 7-nov-01 bill is introduced in nsw state parliament to make it a criminal offence to make online content available to adults if that content is unsuitable for minors 4-nov-01 follow-up survey of this study takes place 4-dec-01 both houses of nsw parliament pass nsw bill 5-dec-01 nsw attorney-general refers the nsw content regulation bill to standing committee on social issues for inquiry and comment sources: gibson (2000), libertus.net (2005) research method theoretical framework the analysis of technology regulation and legislation has received some sporadic attention in the research literature, however very little attention has been given to the regulation of ajis vol 13, no. 1 september 2005 62 the internet in particular. this is unsurprising, given the global nature of the technology and the fact that conventional legislation for traditional environments may not be suited to the dynamic structure of the world wide web (post 1995). accordingly, few frameworks exist for researching the effects of internet regulation. given this dearth of literature and the exploratory nature of this research, this paper is positioned within a policy analysis framework as developed by easton (1957) and refined by sharkansky (1970), rose (1973) and hogwood and gunn (1991). the framework is shown in figure 1. is readers may observe the resemblance between the appearance of this model and that of the system development lifecycle (sdlc). the element of “feedback”, in particular, also suggests similarity to the general systems model: indeed, easton (1957), in passing, described this model as a system. it is also interesting to note that opponents of the policy analysis framework make similar arguments to those put forward by critics of the sdlc. specifically, these critics argue that the framework is inflexible, that it prohibits activity in two stages simultaneously, and that the stages in reality are not so clearly delineated (stephens 1972). this study is particularly interested in the “impact and evaluation” stage of the model, and aims to add context to the analysis by exploring this impact longitudinally. figure 1: policy analysis feedback framework approach a number of research methods are available to the is scholar. this research is concerned with the ex post facto analysis of a particular phenomenon. cook and campbell (1979) ajis vol 13, no. 1 september 2005 63 argue the importance of sound method with respect to such research: ex post facto analysis makes no use of pre-testing, hence threatening the inference of causality in the findings. galliers (1992) analyses approaches to social measurement within the information systems discipline including the laboratory experiment, archival research, the case study and survey methods. within the information systems discipline, the survey method has received the most use, and has been found to consistently provide usable results (straub 1989). consistent with the inference vulnerabilities of ex post facto research discussed by cook and campbell (1979), galliers argues that the survey method may give little insight into causal relationships, can be open to respondent and researcher bias, and only portrays the state of affairs at a single point in time. however, galliers also argues that a large number of variables can be examined under real world conditions in a short space of time. kerlinger (1986) argues that behavioural survey research can be divided into explanatory research and exploratory research. the former seeks causal and correlative relationships between variables within a given phenomenon. often, this method tests hypotheses that have been developed over time. the latter aims to better understand a phenomenon within the formative stages of a topic area’s development (malhotra and grover 1998). such research acknowledges that, for any phenomenon, there may be multiple objective points of view. such perspectives may exhibit differences, however each may be intrinsically valid (walsham 1993) and is hence particularly useful in the early stages of research (dubin 1978). grover and malhotra argue that while exploratory survey research may lack causal rigour, the method’s power can be augmented using longitudinal analysis, whereby a given phenomenon or unit of analysis can be revisited over time in order to better quantify any changes taking place. such an approach also assists in triangulating the research findings and serves to improve the value of the research study to the wider research community. this paper admits the shortcomings of exploratory research, however observes the immaturity of research in the area. this study hence takes a longitudinal approach to examining the phenomenon within an exploratory context. this analysis is also guided by the literature on survey method (notably dillman 1978 and, in an is context, straub 1989), and the burgeoning literature on network regulation. data were required from isps of varying sizes and geographic locations. the data had to be obtained quickly so as to enable the analysis of initial reactions to the enactment of the legislation. telephone (dillman 1978) and postal/email questionnaires (schuldt and totten 1994) were the two survey methods that afforded speed and geographic disparity. email surveys may be problematic, as email messages run the risk of being labeled as unsolicited junk mail (“spam”), or filtered out and not read. network difficulties may mean an email message does not get delivered for some days. there is evidence in the literature that respondents may be reluctant to complete conventional postal surveys, even when reminded that the results are urgently required (diamantopoulos and schlegelmilch 1995). telephone surveys were preferred to email surveys as they allow respondent’s answers to be probed for further information about the issues under investigation (hessler 1992). researchers in the literature have observed that comparison between data sets acquired using different media are not always valid (liefeld 1988); in the interests of consistency the telephone survey method was used for both the initial and subsequent stages of the study. survey design this phased of the study used a similar instrument as was used in the first stage of the ajis vol 13, no. 1 september 2005 64 study. a literature search was initially used to gather established survey questions from other studies. this literature search revealed few such questions. baldwin et al. (1995) develop a general framework of questions to ask when exploring regulatory intervention, and hosein and whitley (2002) apply it with success to the analysis of e-commerce regulatory intervention. ramsay (1985) offers a similar framework for examining regulation in consumer marketplaces, as follows: • why is regulation introduced for an area? • how is the form of the regulatory intervention determined? • how is the process of introducing and implementing the regulation managed? this paper contextualises the baldwin and ramsay frameworks by exploring internet regulation longitudinally. as this study is concerned with determining the effects of and attitudes towards the legislation, the survey focused on the third point of this framework. respondents were asked for demographic information and questions concerning: • the effect of the legislation on the isp’s operations • the nature of feedback received from any groups (e.g. users, clients) • potential effects on the online industry • general legislation implementation issues • anecdotal comments on the legislation itself to contextualise their earlier comments questions were phrased in as objective a manner as possible (consistent with anastasi 1988). a copy of the survey instrument is included in appendix a. administration when administering the surveys, recommendations from other researchers in the area were taken into account. before surveying began, a list of potential respondent queries and prepared answers was generated so that each respondent would be given the same information about the study (dillman 1978). this information sheet contained pre-written answers to potential questions concerning anonymity (stanton 1998), the origin of the survey (fraser et al. 1988, schneider and johnson 1995), the eventual destination of the results, and a brief overview of the legislation. the answer sheet also included a “don’t know” (dk) option, which graeff (2002) and hawkins and coney (1981) argue can reduce uninformed response bias. where possible, the isp manager was sought to answer questions (frohlich 2002), and all survey respondents were given the opportunity of receiving a copy of the final research report (katzer et al. 1982). respondents were reminded that this was an independent study being sponsored by an established academic institution (schneider and johnson 1995, bruvold and comer 1988), and care was taken to avoid leading respondents or otherwise biasing their replies (klassen and jacobs 2001). the survey instrument was pre-tested before it was put into operation (straub 1989). the pre-test revealed a reluctance on the part of many isps to participate in surveys. based on these findings, the survey was restructured so as to remove four questions concerning competitive strategy and operations, and care was taken to remind respondents that these questions were optional. the same basic instrument was reused in the second phase so as to make good use of the pilot testing undertaken in the first round of surveying. twenty-four months later, in 2001, a follow-up survey was conducted. the goal in this second stage of surveying was to follow-up on the original group of respondents. in this ajis vol 13, no. 1 september 2005 65 vein, bankruptcy and merger documents held by the australian securities and investment commission (asic) were examined in order to determine the current state of each of these original respondents. two isps had been purchased by other isps and were now whollyowned subsidiaries. both of these isps were still in existence, albeit under new names. these isps were included in the follow-up survey. respondent profiles surveying during both initial and follow-up implementations was spread over two days. in the first phase, each phone interview took approximately fifteen to twenty minutes. in the second phase, interviews took between twenty and thirty minutes on average. after the interview, additional time was used to recap the conversation and make sure all salient points were noted. in order to improve accuracy, demographic details were triangulated against data from the australian isp directory (http://www.australianispdirectory.com). interestingly, one large national isp answered the survey but declined to participate in the study. their responses have been discarded. table 2 details the demographic breakdown of the isp respondents from the first phase. phase 1 phase 2 network spread number of clients access points isp age network spread number of clients access points isp age isp 1 international n/a 7 8 months internation al n/a 7 2.5 years isp 2 international n/a 8 n/a isp 3 e national n/a 5 2.5 years isp 4 d national n/a 30 3 years national 70,000 60 18 months isp 5 a national 150 n/a 4 years isp 6 statewide 200 1 3 years statewide <500 1 5 years isp 7 statewide 1500 1 1.5 years statewide n/a 1 3 years isp 8 b statewide 1200 1 2.5 years national 5,500 4 3.5 years isp 9 c statewide 130 1 2 years isp 10 statewide 3,000 2 3 years national 7,000 6 6 years isp 11 b regional 4,000 17 5 years statewide 12,000 3 7 years isp 12 regional n/a 13 n/a statewide 7,000 25 5 years isp 13 regional 2,000 13 2 isp 14 regional n/a 7 n/a ajis vol 13, no. 1 september 2005 66 table 2: phase 1 and 2 respondent demographics notes: ‘n/a’ = respondent declined to provide information a. isp 5 had been bought out by a software development house and no longer provided public internet access. b. isp 8 and 11 had been bought out by other isps. these isps were included in the followup survey. c. isp 9 was no longer in existence. d. isp 4 merged with another isp, which was included in the follow-up survey. the age given is that of the merger. e. isp 3 had been bought out by isp 7 table 2 details the demographic breakdown of the isp respondent group in terms of network spread, the number of clients (customers who use the isp to gain network access), access points (points of presence or network dialup ‘on-ramps’) and isp age. the phase 2 demographics show that a number of isps that had participated in the earlier stage of the study no longer existed. this change in isp population was due to isps either going out of business (either voluntarily or involuntarily), or being bought out by another isp. takeover behaviour has been common in the isp market, where providers have been interested in increasing their market spread without undertaking time-consuming infrastructure layout or staff training. isp setups are instead purchased wholesale and rebranded to match the new owner. one important implication of this takeover pattern concerns “organisational memory”, which describes the mental models (levitt and march 1988:326) and shared knowledge of an organisation (fiol 1994). this organisational memory includes individuals, structure, culture, transformations and ecology (walsh and ungson 1991). if some of the original personnel are retained after the takeover, those organisations may maintain some of their previous organisational memory. this may cloud the longitudinal analysis of these firms, as the researcher would be unable to determine whether the opinions given relate to the new organisation or the old one. the effect of this implication is uncertain. results this section details the results of the study. reaction to the legislation during the first round of surveys was mixed. the international isp was more reserved about the legislation’s effects, while smaller inner city isps signalled that their course of action would depend on those of competitors. rural isps appeared to be more passionate about the possible effects of the legislation. apart from these three points, there appeared to be little variation across isp size or age. effect on isp operations in the first round of surveys, most respondents were unclear about the immediate effect on operations. most respondents predicted immediate and sizeable expenditures on mandatory filtering hardware and staffing. in general, respondent isps signalled that these costs would be borne by customers. two respondents also argued that the additional costs could provoke a “make or break” situation particularly among smaller isps, as the alternative to the installation of new hardware was significant fines: “two or three fines of aus$27,500 (approximately us$14,000) will kill our business”. ajis vol 13, no. 1 september 2005 67 the second round of surveys painted a different picture. only two respondents noted any change in operations. isp 4 now offers filtering software to new dialup clients and isp 11 now enforces user age verification for websites containing adult content, “just like overseas”. no other respondents had seen any cost increases, internal policy changes or other modifications to operations: “we haven’t had to hire any extra staff or anything”. feedback from clients in the first survey, respondents noted that client feedback had been mixed. some clients had simply been unaware of the legislation, its meaning or its effects. one isp respondent stated that they had received no feedback despite having informed clients of the legislation. other isps signalled that commercial clients had been changing their plans for future internet use: one client had abandoned their entire online e-commerce venture. some parents and schools had been vocal in their protests against the legislation: parents were “outraged at the government’s taking control out of their hands”. the follow-up survey showed similar findings: respondent feedback clearly fell into two points of view. the first point of view was that some isps observed little in terms of client feedback. one national isp said, “it doesn’t bother most people: it’s just business as usual”. isp 11 noted that, with respect to clients, and despite all the emails circulating at the time of the legislation’s introduction, “it had been a total non-event”. the other point of view expressed by respondents was that they had received negative feedback from clients. “australia is seen as draconian with its attitude to the ‘net”, argued isp 1. “people don’t want to be told what to look at”, said isp 7. these isps also noted that this negative client feedback has had little real effect on operations. effect on online industry in the first phase of this study, most respondents foreshadowed adverse effects of the legislation. one statewide isp argued that the legislation would damage smaller isps who did not have the technical or financial support to sufficiently address the complaints or legislative stipulations. one isp noted that the legislation would be a barrier to commercial website or e-commerce adoption because it increased the cost to the isp of providing online access. one outcome, signaled by almost every isp in the first phase of the study, was that content would move offshore where such restrictions were not present. one isp was constructing servers in france and the united states in order to deal with this issue. this threatened a reduction in australian internet providers and a possible fall in the number of commercial web users. some two years later, the isp landscape has changed dramatically, as foreshadowed by respondents in the first stage of the study. however, in contrast to these arguments, many smaller isps are still operational (isp 6, for instance, still had fewer than 500 dialup clients). additionally, the recent “dot-com” downturn may have affected global isp numbers. in the follow-up survey, only one respondent noticed any industry effects. isp 4 had blamed part of the online downturn on the legislation. all other respondents observed minimal or no change in online industry. isp 8 argued, “our discussion groups have shown no effects at all on the industry – hosting prices in australia are uncompetitive anyway so people look elsewhere for hosting porno sites”. another respondent echoed this: “sites can just go elsewhere and everybody knows it”. isp 12 discussed their initial concerns over the ajis vol 13, no. 1 september 2005 68 vast number of complaints they would receive: “that influx has never eventuated”. implementation issues most of the isps surveyed, irrespective of the degree of network spread, held that the legislation would be difficult to enforce. problems of policy implementation included the ease with which local content, once removed, or foreign content, once blocked, could easily be moved or republished elsewhere. isps were required to undertake the technical process of restricting content access, however users can still access the material itself. respondents in the first stage also argued that sites that keyword blocking was unlikely to work. one rural isp respondent noted that some of their clients maintained sites on animal breeding: respondents had been concerned that these sites might be automatically blocked by search engines or could be forced to adopt age verification measures. similar sentiments were raised concerning sites discussing safe sex and advocating std awareness. the second stage of the survey showed very few implementation issues among those isps left in the respondent group. isp 10 signalled no loss of business, however three isps argued that the online industry in australia has already suffered because of its small isp population and “rorting from the [telecommunications] monopolies”. one isp did argue that, “if the objective is to protect people from porn and such material…people can get this stuff elsewhere if they really want to. it’s more of a public relations scheme to give the impression that [the government is] trying to do something”. this suggests that isps do not perceive the legislation’s effect to have met with its original goals. comments on the legislation in the first stage of the study, respondents generally felt the legislation was vague (the section concerning provider indemnification, in particular). specifically, respondents were concerned that a single indemnification rule would apply regardless of the difficulty associated with blocking the site. most respondents argued for user-moderated censorship instead: an international isp argued for “education, empowerment and enforcement”, while a smaller local isp suggested that users should be informed of how to restrict their own net usage. one respondent argued that given the expense of implementing and maintaining the legislation it would have been cheaper for the government to purchase copies of filtering software to distribute to all internet users in australia. another respondent argued that if this software was too technical, a local software developer could be engaged to develop an acceptable piece of filtering software. two smaller isps argued that the legislation should be rescinded as the regulation would hurt business but would not prevent access to offensive material: “it’s like asking [a motor vehicle manufacturer] to build a car that doesn’t crash”. the second stage of the study saw similar comments, despite the apparent lack of effect in the industry. respondents were generally still opposed to the legislation: “prohibition does not work”, “if people want to have these things blocked, they should take care of it themselves and buy the relevant software” and “it’s down to the parents to take responsibility for what their kids are doing”. interestingly, respondents across the study signalled that they fully supported restricting access to “sensitive” material. ajis vol 13, no. 1 september 2005 69 table 3: summary of findings phase 1 phase 2 effect on operations increase in system acquisition increase in costs due to implementation, monitoring and fines increase in staff to undertake monitoring no significant change in operations feedback from clients some client feedback opposition from parents and schools in particular little feedback overall effect on online industry fall in commercial web use fall in isp numbers as costs rise and client numbers drop content moving offshore fewer isps in marketplace no other industry effects implementation issues difficulty in effectively removing/blocking content confusion regarding legislation some content intrinsically difficult to block isp industry may have already suffered no other implementation issues conclusions in contrast to the findings of hosein and whitley (2002) regarding network legislation in the uk, the legislation has had seemingly little effect on the online industry in australia. this finding also comes despite isp claims in the first stage of this study to the contrary. it should be noted that whereas the introduction of this legislation may have affected the isp population directly, its implications in this regard are difficult to gauge given the effect of the recent downturn in e-commerce business and “dot-coms”. the results generally suggest that, in the opinions of the isps surveyed, the legislation in its present form may not meet the original goals of effectively controlling or moderating online content, and is hence flawed. offensive content can be moved elsewhere if the original host is blocked, hence undermining the efficacy of the regulation. in addition, with isps being an integral part of the legislation’s function, their apparent dissatisfaction with the regulation may also undermine its effectiveness. the results echo the arguments of authors such as teo et al. (1997) and soh et al. (1997), who discuss the importance of governmental support for evolving online technology. while hulsink and van bolhuis (1998) advocate the introduction of effective bilateral online standards, the legislation in its present form may hinder the evolution of online commerce in australia by raising isp costs (which would be passed on to clients) or deterring online ventures. these implications would support the arguments of williams (1994) that online regulation and legislation may have negligible or even adverse effects on the technology and its users. several respondents argued that this legislation would have little effect outside australia; it is possible, however, that given the legislation’s potential effect on domestic electronic commerce, global electronic commerce could be adversely affected if similar articles of legislation were to be introduced in other countries. this evidence would seem to support the arguments of authors such as clarke (1995) and lambrick (1996), who note that government regulation, improperly implemented, can have stifling and detrimental ajis vol 13, no. 1 september 2005 70 effects on system use. the results from this research have three main implications. first, regulators should be prepared for significant resistance from network access providers over the introduction of regulatory legislation. this is consistent with the introduction of legislation in other environments (stigler 1971). the results seem to suggest however, that this resistance could be alleviated through discussion with industry stakeholders. resistance may also be alleviated through a more open or higher profile review process. as kawamoto (1997:103) argues, “[reform] can be significantly advanced when a number of sectors are dealt with simultaneously, because that makes it more difficult for special interests to resist reform given the higher political profile and more transparent policy agenda”. the findings highlight the difficulty of legislating for and regulating rapidly changing technology groups. this generally supports the arguments of authors such as post (1995) and chong (1995), who observe that conventional legislation may be ineffective in online environments. the results also suggest, however, that this new online legislation has been of questionable benefit to regulators, isps and policy makers. while new provisions have been written in order to adapt the conventional legislation for these new environments and hence improve the legislation’s effectiveness, the findings also suggest that the enforcement of this legislation is of significant importance. however, as stigler (1970:526) observes, “there is one decisive reason why the society must forego complete enforcement of the rule: enforcement is costly”. further, this difficulty is increased if the underlying technology is frequently changing. on a methodological note, the study highlights the difficulties associated with researching online environments. the rapidly changing technology coupled with the keen competition in the online marketplace mean that respondents are often reluctant to divulge information about their operations or strategy. this may explain the dearth of research material in the area: despite the importance of this topic to many groups, scholarly research may simply prove too difficult to undertake. the degree and effect of these implications is uncertain, and their exploration remains an avenue for further research. the study is subject to a number of important limitations. the study examines only a small sample of the isp population in australia, which recent figures place at approximately six hundred access providers. these numbers have been falling over the past twelve months, which would be consistent with the demographic findings presented here. this changing population poses significant problems for longitudinal research in the area. also, in the interests of expeditious analysis, a telephone survey was used in both phases of the research. while every effort was made to maintain high levels of reliability and comparability between interviews through the use of a structured question sheet, these issues could not be guaranteed. it could be argued that only those isps who were not affected by the legislation were left in the sample group by the time the second phase began. that is, isps for whom legislative compliance proved to be too problematic may have expired. the same could be said for those firms who have moved offshore (consistent with hosein and whitley 2002). similarly, if the legislation was as easy to circumvent as was suggested in the earlier stages of the study, subsequent surveys would reveal no significant implementation issues or effects. both of these issues may explain the lack of variation between responses in the second phase of the study. a range of possible research topics would be of benefit and interest to the information systems discipline as a result of this research. this paper is an investigation into reactions to legislation regarding internet content restrictions. if further bills are passed through ajis vol 13, no. 1 september 2005 71 parliament, studies should examine the different ways the isps in this study are dealing with the new requirements on their business operations. this research could be conducted with respect to their current operational position and stance on the regulation issue. particular attention should be paid to the movement of isps and their clients given the evidence presented here. additional research could also focus on other interest groups. for instance, if one of the legislation’s original goals was to moderate offensive online content, then some scholarly analysis of the amount of offensive material available would be of benefit. similarly, it would be interesting to see how parents have reacted to the legislation. an analysis of their attitude to the type and degree of content available online would make for a most fascinating study. additionally, while the legislation was designed to affect online content, it does not apply to email transaction. malicious, offensive or irrelevant email can be detrimental to productivity in the workplace (peat 2002). it would be beneficial to see if employers have noticed changes in the internet use habits of their staff over the period of the legislation’s introduction. this study adds evidence to the ongoing debate surrounding online regulation. while the immediate effects of this evidence are undetermined, the area makes a healthy environment for future research. references abell, w., lim, l., (1996), “business use of the internet in new zealand: an exploratory study”, ausweb 96 proceedings, 33 39. abs, (2000), household use of information technology, australian bureau of statistics, australia. altmann, j., chu, k., (2001), “how to charge for network services flat-rate or usagebased?”, computer networks, vol. 36, no. 5-6, 519-531. anastasi, a., (1988) psychological testing, sixth edition, new york: macmillan publishers. applegate, l. m., holsapple, c. w., kalakota, r., radermacher, f. j., whinston, a. b., (1996), “electronic commerce: building blocks of new business opportunity”, journal of organisational computing and electronic commerce, vol. 6, no. 1, 1-10. baldwin, r., abel-smith, b., cave, m., fenn, p., hodges, m., marsden, c., mossialos, e., scott, c., woolcock, s., (1995), regulation in question: the growing agenda, london: merck, sharp and dohme. barringer, b. r., harrison, j. s., (2000), “walking a tightrope: creating value through interorganizational relationships”, journal of management, vol. 26, no. 3, 367-403. bauer, j. m., berne, m., maitland, c. f., (2002),"internet access in the european union and in the united states", telematics and informatics, vol. 19, no. 2, 117-137 baumer, d. l., earp, j. b., poindexter, j. c., (2004),"internet privacy law: a comparison between the united states and the european union", computers & security, vol. 23, no. 5, 400-412 bhimani, a., (1996), “securing the commercial internet”, communications of the acm, vol. 39, no. 6, 29 – 35. brien, c., (1996), “internet: the new trade in goods, services and ideas”, journal of law and information science, vol. 7, no. 1. bruvold n. t., comer, j., (1988), “a model for estimating the response rate to a mailed survey”, journal of business research, vol. 16, no. 2, 101-116. carr, i., williams, k., (1994), “the singapore computer misuse act – better protection for the victims?”, journal of law and information science, vol. 5, no. 2, 210 – 226. ajis vol 13, no. 1 september 2005 72 cazavos, e. a., (1996), “the legal risks of setting up shop in cyberspace”, journal of organisational computing and electronic commerce, vol. 6, no. 1, 51-60. chiou, j., (2004), “the antecedents of consumers' loyalty toward internet service providers”, information & management, vol. 41, no. 6, 685-695 chong, r. b., (1995), “trends in communication and other musings on our future”, federal communications law journal, vol. 47, 213-219. clarke, r. a., (1995), “a normative regulatory framework for computer matching”, the john marshall journal of computer and information law, vol. 13, no. 4, 585 –633. clarke, r., (1992), “a contingency model of edi’s impact on industry sectors”, journal of strategic information systems, vol. 1, no. 3, 143 – 151. cook, t. d., campbell, d. t., (1979), quasi-experimentation: design and analytical issues for field settings, chicago, rand mcnally publishers. cook, c., creyke, r., geddes, r., holloway, i., (2005), laying down the law, 6th ed., lexis nexis butterworths, nsw, australia de souza, m., (1999), “should net content be regulated?”, internet.au magazine, june. diamantopoulos, a., schlegelmilch, b. b., (1996), “determinants of industrial mail survey response: a survey-on-surveys analysis of researchers' and managers' views”, journal of marketing management, vol. 12, 505-531. dillman, d. a., (1978), mail and telephone surveys: the total design method, new york: john wiley and sons. downes, t., greenstein, s., (2001), “universal access and local internet markets in the us”, research policy, vol. 31, no. 7, 1035 1052. dubin, r., (1978), theory building, new york, free press. easton, d., (1957), “an approach to the analysis of political systems”, world politics, vol. 9, no. 3, 383 – 400. fan, q., (2005), "regulatory factors influencing internet access in australia and china: a comparative analysis", telecommunications policy, vol. 29, no. 2-3, 191-203 feher, a., towell, e., (1997), “business use of the internet”, internet research: electronic networking and policy, vol. 7, no. 3, 195 – 200. fiol, c. m., (1994), “consensus, diversity, and learning in organizations”, organization science, vol. 5, no. 3, 403-420. forcht, k. a., fore, r. e., (1995), “security issues and concerns with the internet”, internet research: electronic networking applications and policy, vol. 5, no. 3, 23 – 31. foros, o., hansen, b., (2001), “competition and compatibility among internet service providers”, information economics and policy, vol. 13, no. 4, 411-425. francalanci, c., willcocks, l., kern, t., (2001), “internet and the horizontal integration of it businesses”, european management journal, vol. 19, no. 2, 145-156. fraser, c., hite, r. e., sauer, p. l., (1988), “increasing contributions in solicitation campaigns: the use of large and small anchor points”, journal of consumer research, vol. 15, 284-287. frohlich, m. t., (2002), “techniques for improving response rates in om survey research”, journal of operations management, vol. 20, no. 1, 53-62 . galliers, d., (1992) information systems research, blackwells, oxford, uk. gefen, d., straub, d. w., (2004), "consumer trust in b2c e-commerce and the importance of social presence: experiments in e-products and e-services", omega, vol. 32, no. 6, 407424 gibson, h., (2000), “shooting the messenger: a critique of australia’s internet content regulation regime”, issue analysis, vol. 10, 1-11. ajis vol 13, no. 1 september 2005 73 graeff, t. r., (2002), “uninformed response bias in telephone surveys”, journal of business research, vol. 55, no. 3, 251-259. hallahan, k., (2004),"protecting an organization's digital public relations assets", public relations review, vol. 30, no. 3, 255-268 hancher, l., moran, m., (1989), “organizing regulatory space”, in: hancher, l., moran, m., eds, capitalism, culture and regulation, claredon press, oxford hancock, b., (1999), “security views”, computers and security, vol. 18, no. 3. hawkins, d. i., coney, k. a., (1981), “uninformed response error in survey research”, journal of market research, vol. 18, 370-374. heilbronn, g., latimer, p., nielsen, j., (2005), introducing the law, 6th ed, cch, sydney, australia hessler, r. m., (1992), social research methods, st. paul: west publishing company. hogwood, b. w., gunn, l. a., (1991), policy analysis for the real world, oxford, uk: oxford university press. hosein, i., whitley, e. a., (2002), “the regulation of electronic commerce: learning from the uk's rip act”, the journal of strategic information systems, vol. 11, no. 1, 31-58. hulsink, w., van bolhuis, h., (1998), “electronic commerce and cyber entrepreneurship potential and problems of european startup firms in ict”, 11th conference on electronic commerce, bled. jarvenpaa, s. l., todd, p. a., (1996), “consumer reactions to electronic shopping on the world wide web”, international journal of electronic commerce, vol. 1, no. 2, 59 – 88. jia, j. h., (2003), "internet copyright regulation china: copyright infringement in the network environment china's perspective", computer law & security report, vol. 19, no. 2, 101-111 katzer, j., cook, k. h., crouch, w. w., (1982), evaluating information: a guide for users of social science research, boston: mcgraw-hill publishers. kawamoto, a., (1997), “regulatory reform on the international trade policy agenda”, the journal of world trade, vol. 31, no. 4, 81-116. kerlinger, f. n., (1986), foundations of behavioural research, new york: holt, rinehart and winston. klassen, r. d., jacobs, j., (2001), “experimental comparison of web, electronic and mail survey technologies in operations management”, journal of operations management, vol. 19, no. 6, 713-728. kleinrock, l., (2004),"the internet rules of engagement: then and now", technology in society, vol. 26, no. 2-3, 193-207 koomen, k., (1998), “illegal and harmful content on the internet: some issues and options”, computers and law, vol. 35, no. 1. lambrick, j., (1996), “censorship and the internet – not a good start”, computers and law, 3 – 4. levitt, b., march, j. g., (1988), “organizational learning”, annual review of sociology, vol. 14, no. 3, 319-340. libertus.net, “censorship in australia – history from 1920 to 1995”, http://libertus.net/censor/hist20on.html (2005), accessed february 5, 2005. liefeld, j. p., (1988), “response effects in computer-administered questioning”, journal of marketing research, vol. 25, no. 4, 405-409. lu, m., yeung, w., (1998), “a framework for effective commercial web application development”, internet research: electronic networking applications and policy, ajis vol 13, no. 1 september 2005 74 vol. 8, no. 2, 166-173. madden, g., savage, s. j., coble-neal, g., (1999), “subscriber churn in the australian isp market”, information economics and policy, vol. 11, no. 2, 195-207. malhotra, m. k., grover, v., (1998), “an assessment of survey research in pom: from constructs to theory”, journal of operations management, vol. 16, no. 4, 407-425. mclennan, s., (1999) yearbook australia, australian bureau of statistics, government publishing service. naldi, m., (2002), “internet access traffic sharing in a multi-operator environment”, computer networks, vol. 38, no. 6, 809-824. ng, h., pan, y. j., wilson, t. d., (1998), “business use of the world wide web: a report on further investigations”, information research, vol. 3, no. 4, 291 – 314. nicholls, w. l., baker, r. p., martin, j., (1997), “the effect of new data collection technologies on survey data quality”, in lyberg, l., biemer, p., collins, m., et al. (eds.), survey measurement and process quality, new york: wiley publishers, 221-248. nunamaker, j. f., chen, m., purdin, t. d. m., (1991), “systems development in information systems research”, journal of management information systems, vol. 7, no. 3, 89 – 106. parker, l., (1992), “collecting data the e-mail way”, training and development, 52-54. peat, g., (2002), “the threat of malicious outbound email”, computer fraud & security, 2002, 4, 10-11. peltzman, s. (1995), the economic theory of regulation after a decade of deregulation, the political economy of privatisation and deregulation, elgar reference collection, international library of critical writings in economics, 44, aldershot, uk, 168-208. post, d., (1995), “new rules for the net?”, the american lawyer, 112. ramsay, i., (1985), “framework for regulation of the consumer marketplace”, journal of consumer policy, vol. 8, 353-372. robey, d., boudreau, m., rose, g. m., (2000), “information technology and organizational learning: a review and assessment of research”, accounting, management and information technologies, vol. 10, no. 2, 125-155. rose, r., (1973), “comparing public policy: an overview”, european journal of political research, vol. 1, 67-96. salam, a. f., pegels, c. c., rao, h. r., (1997) “an exploratory investigation of the internet involvement: instrument development, measurement and implications for electronic commerce”, association of information systems conference proceedings. schneider, k. c., johnson, k. c., (1995), “stimulating response to market surveys of business professionals”, industrial marketing management, vol. 24, 265-276. schuldt, b. a. totten, j. w., (1994), “electronic mail vs. mail survey response rates”, marketing research: a magazine of management and applications, vol. 6, no. 1, 3639. sharkansky, i., (1970), “environment, policy, output and impact: problems of theory and method in the analysis of public policy”, in i. sharkansky (ed.), policy analysis and political science, chicago: markham publishers. soh, s., mah, q. y., gan, f. j., chew, d., reid, e., (1997), “the use of the internet for business: the experience of early adopters in singapore”, internet research: electronic networking applications and policy, vol. 7, no. 3, 217 – 228. stair, r. m., reynolds, g. w., (1998), principles of information systems: a managerial approach, 3rd edition, boston, course publishers. stanton, j. m., (1998), “an empirical assessment of data collection using the internet”, ajis vol 13, no. 1 september 2005 75 personal psychology, vol. 51, no. 3, 709-725. stephens, j., (1972), “an appraisal of some system approaches in the study of international systems”, international studies quarterly, vol. 16, no. 3, 321 – 349. stern, s. e., gregor, s., martin, m. a., goode, s., rolfe, j., (2004), "a classification tree analysis of broadband adoption in australian households", proceedings of the 6th international conference on electronic commerce, delft, the netherlands, 451 456 stigler, g. j., (1970), “the optimum enforcement of laws”, the journal of political economy, vol. 78, no. 3, 526-536. stigler, g. j., (1971), “the theory of economic regulation”, the bell journal of economics and management science, vol. 2, no. 1, 3-21. straub, d. w., (1989), “validating instruments in mis research”, management information systems quarterly, vol. 13, no. 2, 147 – 169. strauss j.,rogerson, k. s., (2002), "policies for online privacy in the united states and the european union", telematics and informatics, vol. 19, no. 2, 173-192 tan, m., teo, t. s. h., (1997), “factors influencing the adoption of the internet”, international journal of electronic commerce, vol. 2, no. 3, 5 – 18. teo, t. s. h., tan, m., buk, w., k., (1997), “a contingency model of internet adoption in singapore”, international journal of electronic commerce, vol. 2, no. 2, 95 – 118. tian, y., (2004), "business implications of anti-circumvention legislation and recommendations", computer law & security report, vol. 20, no. 6, novemberdecember, 445-452 trewin, d., (2002), internet activity, publication 8153.0, australian bureau of statistics. vadapalli, a., ramamurthy, k., (1997), “business use of the internet: an analytical framework and exploratory case study”, international journal of electronic commerce, vol. 2, no. 2, 71 – 94. walsh, j. p., ungson, g. r., (1991), “organizational memory”, academy of management review, vol. 16, no. 1, 57-91. walsham, g., (1993), interpreting information systems in organisations, chichester, uk: wiley publishing. weible, r., wallace, j., (1998), “cyber research: the impact of the internet on data collection”, marketing research, vol. 10, no. 3, 19-25. williams, t. a., (1994), “government regulation through voluntary cooperation: the strategic impact of information technology”, journal of strategic information systems, vol. 3, no. 2, 149-156. wright, h., (1996a), “the net: the beginning or the end for free expression?”, computers and law, vol. 29, 4 – 5. wright, h., (1996b), “convergence and communication values on the net”, journal of law and information science, vol. 7, no. 1. xiao, j., yang, h., chow, c. w., (2004), "the determinants and characteristics of voluntary internet-based disclosures by listed chinese companies", journal of accounting and public policy, vol. 23, no. 3, 191-225 yemini, y., dailianas, a., florissi, d., huberman, g., (2000), “marketnet: protecting access to information systems through financial market controls”, decision support systems, vol. 28, no. 1-2, 205-216. ajis vol 13, no. 1 september 2005 76 appendix a telephone questionnaire sheet isp name:__________________________________________________ how many clients do you have (approx.)? _______________________ how many access points do you have?___________________________ how old is the isp business?___________________________(specify years or months) does the respondent know about the legislation?______________(yes or no) people can complain to the australian broadcasting authority about offensive online material this material will receive a rating by the national classification board (e.g. x or r) the australian broadcasting authority can then issue notice to isp to either remove material if it is local or block it if it is foreign. the aba can also issue an interim removal notice pending a decision. will this legislation have an impact on your operations? (why or why not? please give details) _________________________________________________________________________ _________________________________________________________________________ _________________________________________________________________□ don’t know have you had any positive feedback (from any sector) yet? _________________________________________________________________________ _________________________________________________________________________ _________________________________________________________________□ don’t know have you had any negative feedback (from any sector) yet? _________________________________________________________________________ _________________________________________________________________________ _________________________________________________________________□ don’t know how does the respondent feel this will affect the online industry, if at all? _________________________________________________________________________ _________________________________________________________________________ _________________________________________________________________□ don’t know could the legislation be improved? if so, how? _________________________________________________________________________ ajis vol 13, no. 1 september 2005 77 _________________________________________________________________________ _________________________________________________________________□ don’t know does the respondent have any comments about policing within the isp? _________________________________________________________________________ _________________________________________________________________________ _________________________________________________________________□ don’t know does the respondent have personal feelings on the matter? _________________________________________________________________________ _________________________________________________________________________ _________________________________________________________________□ don’t know would the respondent like a copy of the results? if so, please give address: _________________________________________________________________________ _________________________________________________________________________ _________________________________________________________________□ don’t know microsoft word ajis12-1final repaired.doc ajis vol. 12, no. 1 september 2004 111 the application of intentional subjective properties and mediated communication tools to software agents in online disputes resolution environments. renzo gobbin school of information sciences and engineering university of canberra abstract this paper examines the use of subjective properties in modeling an architecture for cooperative agents using agent communication language (acl) that is used as a mediating tool for cooperative communication activities between and within software agents. the role that subjective and objective properties have in explaining and modeling agent internalization and externalization of acl messages is investigated and related to vygotsky’s developmental learning theories such as mediated activity theory. a novel agent architecture alma (agent language mediated activity) based on the integration of agents’ subjective and objective properties within an agent communication activity framework will be presented. the relevance of software agents subjective properties in modeling applications such as e-law online dispute resolution for e-business contractual arrangements using natural language subject/object relation in their communication patterns will be discussed. introduction the ability of corporations, businesses and web users to conduct commercial transactions on the internet is continually growing together with the high number of legal disputes originated by online activities. following this growth, business web users have found that they need better and newer ways to conduct the equivalent of their traditional business functions under current commercial law. for example if a law firm has had a traditional way of executing dispute resolution, by entering the e-business environment the same law firms will be wondering what the best ways are in utilizing the internet and associated technologies for business purposes. many e-business online contracts are today executed by computer systems employing software agents without any human intervention (bing & sartor 2003). in the area of law and online dispute resolution the use of intelligent agents requires a fine tuned speech-act or natural language communication and an investigation in the way intelligent agents can internalise and externalize their communications to achieve their purpose. in the legal context intelligent agents will need to use a range of communicative tools to transfer meaningful messages used in a range of cooperative activities. cooperation, defined as working together to achieve a common aim, i.e. multiple software agents working together, requires agents’ communicative capacity in order to exchange information about agents’ common goals, their own identity, and their current status (franklin & gaesser 1996, odell 2000, klush 2000, cohen & levesque 1995). while performing cooperative activities with other agents, an agent can send and receive acl speech-act messages (finin et al 1994, fipa 2004). although deprived of the richness of human verbal communication, a substantial form of communication is achieved. we have applied vygotsky’s developmental learning theories (vygotsky 1978, 1986) on the internalisation and externalisation processes of speech and thought and the use of communication as mediating tool and modeled vygotsky’s theories in a communicative agents’ architecture (gobbin 1998a, b, jentzsch & gobbin 2003). by using mediated activity theories a cooperative agent architecture could be modeled according to the following characteristics: • the ability to use natural language, acl or symbolic patterns as communicative tools mediating agents cooperative activities. • the ability to use subjective and objective properties required by intelligent agents to perform bi-directional multiple communication activities. • the ability to internalise representations of perceived communicative patterns from agents or humans. ajis vol. 12, no. 1 september 2004 112 • the ability to externalise internally stored representations of communication patterns to other agents or humans. the use of language as tool, as suggested in mediated activity theories proposed by vygotsky (1978, 1986) for the explanation of cognitive development, will provide a proper theoretical framework for investigating multiple cooperative agent communication activities. in particular the aspects of internalisation and externalisation of information, together with related intentional subjective qualities inherent to the communicative processes can provide a useful paradigm for novel software agents communication architectures. subjective / objective qualities the differentiation between subjective and objective qualities in entities has been one of the important philosophical investigations since aristotle and is still actively researched by modern scholars. it is to be noted here that subjective and objective concepts have changed in modern times from the ones used in seventeen century by descartes and spinoza. german philosophers kant and fitche were the originator of the changes so that these concepts are used in an opposite way than the seventeen century. while the subject is said to be the knowing entity, the object on the contrary is something which, while subjected by the thinking processes of the knowing entity still maintains its own objective nature in independence of the opinions of the knowing entity. in this paper i will apply post-kant philosophical concepts of subjectivity and objectivity into the area of intelligent software agent communication. intelligent systems, such as communicative software agents, should be able to represent themselves as subjects in an objective world. work on subjectivity done in modern times philosophy spacing from kant and hegel through peter strawson, gareth evans and recently paul ricoeur are important in agent self representation within this context (ricoeur 1992, kant 1929, strawson 1966, evans 1982, nagel 1974, lycan 1990) in this context an intelligent software agent able to communicative activity and of representing itself as a subject in an objective world whose existence is independent of the agent itself should have the following properties: • objectivity refers to the view that the truth of a thing is independent from the observing subject (in our case an intelligent agent or a person). this notion entail that there are objective entities existing independently from a software agent and that they are external to the agent itself. objective truths are independent from the software agent’s desires, beliefs and intentions. in our investigation the concept of objectivity is important for the internalization communicative activities and their necessity for software agent inferential internal states. • subjectivity on the other end denotes that the truth of some class of externalised statements depends on the internal states or reactions of the software agent making the statement. the notion of subjectivity is that knowledge is restricted to a software agent own perceptions and the objective qualities experienced by a software agent are subject to its internal interpretations, beliefs, desire and intentions. legal aspects of software agents externalised subjectivity. whenever software agents (sa) execute their agency behaving as intelligent entities we tend to attribute to these artificial entities a behavioural mode similar to humans. if, as an example, an sa enters in communication mode exchanging contracts online with the related legal issues involved then we tend to expect a conventional behaviours in this situated environment and also attribute to the sa some kind of mental states. this attribution seems to clash with the notion that mental states only apply to humans. in this paper we will only examine human and software agents cognitive mental states and behaviour relevant to the law. an issue emerges at this point: if we have to take into account human mental states and artificial artifacts mental states differently, then we will need to legislate and apply a different or duplicate law for intelligent software agents (bing & sartor 2003). an alternative to this course of action is the use of dennett’s intentional stance that interpreting mental concepts in a flexible way ajis vol. 12, no. 1 september 2004 113 can be applied in legal analysis interaction activities of both humans and intelligent software agents (bing & sartor 2003, dennett 1997) in choosing dennett’s intentional stance we assume that behaviour and related belief, desire and intention (bdi) forms of cognitive mental states are directing and motivating an intelligent software agents activity including their epistemic states (information on object in the world) and their conative states. (information on what objective goals to achieve) by applying an intentional stance to the analysis of software agents activities we should: 1 treat the software agent as a rational agent. 2 examine the bdi the agent should have. (given his objective place in the environment) 3 forecast rational normative activities the agent should perform in light of its bdis. 4 confirm that the agent is performing his activities in a consistent and rational way. whenever in the light of the considerations above, an intelligent software agent appears to possess the capacity to process and communicate knowledge as intentional subjective state then we are justified in attributing to such an agent a number of bdis and associated mental states (bing & sartor 2003). accordingly, the law should give legal relevance and recognise subjective intentionality in sofware agents acting in the online marketplace for their legal owners. recognition of intelligent software agents systems and mixed systems including humans and agents subjective intentionality would imply: interacting software agents above would be authorised to attribute subjective intentional states to hybrid systems they interact with. in this case software agents will appear to have subjective intentional states and/or to exhibit them by speechact communication. interacting software agents will not be able to avoid objective interpretation of the other system behaviour which corresponds to intentional states on the basis of ontological conventions applicable to the communicative interaction at hand. in an hybrid online ebusiness environment, characterised by human and software agents interactions, the alternative between objectifying human computer interaction (hci) communicative activities in behavioural terms and subjectifying the same activities by using a bdi intentional stance needs to be considered. adopting the intentional stance is not an arbitrary choice. such perspective represents most of the times the only possibility to explain and foresee objective behavioural activity of complex software agents (bing & sartor 2003). subjective intentional stance can be useful in putting humans and agents in a equal position in legal relationships and disputes. when collective organisational transactions involving customers, software agents and companies interact together in online communicative fashion then all the collective internal and external interactive members need to be considered as possessing subjective intentional states and bdi. the intentional stance can provide more information for legal enforcement and judgment in case of disputes than the behavioural examination of activities. online dispute resolution involving software agents and their legal owners (online companies) will therefore require a clear identification of who are the legal subjects in the dispute and the examination of relevant intentional motivations behind online behavioural activities. mediated activity theory and software agents an important aspect of mediated activity theory is the use of tools as mediators in performing an activity thence the term mediated (gobbin 1998 a,b, kaptelinin 1996). a theoretical approach for cooperative multi-agents can make use of communicative tools mediation and the cognitive view of tool mediation that applies the concept of tool mediation to the use of language or signs as mediating factors (kuuti 1996, lock & peters 1996, nardi 1996). the dialectic process generated by subjectobject relationship clearly influenced recent theories on thought and language and their developmental processes (parker 1993, kuuti 1991). ajis vol. 12, no. 1 september 2004 114 subject communication tool communication tool subject object object agent a agent b figure 1: tool mediated activity model cognitive scientist and philosophers are often implying, in a metaphorical way, that speech and language are in fact mediating tools used in a communication activity. for example, wittengstein (1958) in his “philosophical investigations” relates tools in a toolbox with word generation functionality while kempson (1977) contrasts austin’s (1965) concept of speech-act with grice’s (1989) co-operative principle of communication. vygotsky’s tool mediation theories have been recently proposed using the model described in figure 1 in research on human computer interaction software applications design (kaptelinin 1992). the use of language tool mediation in the area of cooperative multi-agents communication is novel but appropriate as an agent identity represents the dialectical difference between external and internal communication activities. subjective and objective properties acquired during communication can therefore be merged in a single agent entity as illustrated in figure 2. figure 2. multi-agent communicative tool mediated activity model goal/ object subject tool activity environment ajis vol. 12, no. 1 september 2004 115 from a computer science perspective agents are considered autonomous, asynchronous and using distributed processes with their distinct objective related traits. from the ai (artificial intelligence) perspective agents are considered communicative, intelligent and rational with the possibility of intentional communication so that they could qualify for subjective traits. these two perspectives require different architecture and modeling approaches. while the first perspective have objective characteristics of software tools the second implies intelligent communication and therefore requires a subjective paradigm. by using a mediated activity model with agents’ subjective as well as objective characteristics, an integration of computer science perspectives and ai perspectives can be achieved. the new model for software agents under investigation can describe subjective intentional agent mediated activities and at the same time takes into account the agent objectivity while communicating with other agents. the agent tool mediated activity model proposed analyses agent subjectivity, its internal and external software tool processes and its objectivity in asymmetric fashion. mediated activity models provide a framework for the analysis of agent activities in a historical time frame leading to an understanding of temporal changes in agent’s knowledge and behaviour. this type of analysis is broadened to cover pattern of activity rather than episodic staged situations. the asymmetry between communicative tools and agents provides a comprehensive model for multi-agents where activity frames that can be examined singularly or in an historical succession. the range of agent activities necessarily involves agents’ motives that are mainly of environmental origin. while agent actions are always goal driven, agent operations involve formal interaction of the agents’ subjectivity with agent objectivity with the mediation of communicative tools. an agent can be a substitute for a range of human activities in a situated commercial or industrial context and mediated activity is an ideal framework for modeling agent activities where agents’ individual actions and activities are analysed in a contextual environment. agents’ activities are also dynamic and under continuous development this is especially true in an historical related environment. the agent language mediated activity (alma) architecture currently being investigated differentiates processes at various categories and levels taking into consideration the objects to which these processes are oriented. activities are oriented to motives. each motive is a conceptual object that satisfies an agent need. actions are the processes functionally and hierarchically subordinated to activities and they are oriented at specific agent own goals. actions are realised through step-by-step operations that are also determined by the actual condition of activity and its environment. the alma model is also a modeling tool useful in the analysis of cooperative agents team activities supporting the bdi (belief, desire, intention) agent architectures (busetta et al 1999). metarepresentations and relevance theories can also be applied to alma architectures with agent internal activities designed for cooperative communication concepts (edmond & papazoglou 1998, wiederhold 1994, doran et al 1997, sperber 2000) externalisation and internalisation of software agent communication activities often software agents communicative activities are misunderstood as an encoding/decoding process related only to the interpretation of complex linguistic codes. the work of an encoding/decoding device is not certainly inferential or creative. it is not inferential because the symmetrical relation between a message and a signal is quite different from the asymmetric relation of premises to conclusion i.e. the meaning of a sentence doesn’t logically follows from its sound. also an encoding/decoding device is rarely creative. it would be dangerous if it was so. a creative encoding/decoding that will change the process symmetry quite often will generate communication errors. therefore encoding/decoding can only be just an ancillary process of any creative and inferential language mediated activity (sperber 1994). a creative and inferential communication activity will require the asymmetric model provided by the alma architecture in order to be able to perform internal communicative activities as well as the external ones. inferential activities are encapsulated inside an agent architecture involving the ajis vol. 12, no. 1 september 2004 116 processing of internalized conceptual representations stored in the agent internal knowledge bases. in the mediated activity model described above, communicative subjective expressions are externalised by an agent using mediating linguistic tools. subsequently the same agent can internalise communicative expressions in an objective fashion. we can see two important activities performed by a communicating agent: 1 externalisation of agents’ internal representations inferences encoded in acl by using a subjective activity. 2 internalisation of communicative external acl representations by using an objective activity. an agent should be able initially to externalise internally stored ontologies and knowledge bases data ontogenetically implanted by the system designer at agent creation time (gruber 1993, guarino et al 1994, guarino & welty 2000). an agent should be able also to dynamically internalise communicative activities into its internal knowledge base. the capability of externalise inferential knowledge previously internalized by agents during the process of communication mediated activities implies the performance of internal inferential activities in the agent architecture. the analysis of these activities are extremely important for modeling agents’ adaptation, learning and development processes and will be thoroughly investigated in our next stage of research. agent language mediated activity model the agent language mediated activity model (alma) agent architecture currently under research is based on the mediated activity framework described above and is able to describe a range of internal and external communication activities performed by multi-agents. the alma model is designed to perform activities in an environment described in in figure 3 where the communicative process between agents uses an agent communication language and is based on subjective and objective communication interfaces. these interfaces are specifically designed to provide the range of communicative activity capabilities necessary for an agent to perform the externalization of internally produced acl messages and the internalization of acl messages received from the environment. subjective and objective inference engines are inbuilt inside the agent model because the subjective and objective properties are created internally by the inferential and conceptualization process. alma will use also a number of specialized knowledge bases tailored for the each of the specific subject and object modules. a knowledge base for high level meta-representations will provide also an integrated kbs environment using specialized ontologies. a multiple ontology environment capable to store communicative representations for both internalization and externalization activities will be available together with an ontology for conceptual meta-representation. the alma model could also be associated with recent cognitive sciences theories such as language meta-representations and relevance together with theories of social cooperation. alma internal modules will be developed taking into account the current standardization efforts by a number of multi-agents research organisations. the alma architecture will evolve as the research continues in the area of meta-representations, conceptual representation and relevance modules (castelfranchi & werner 1994, keil 1979, sperber & wilson 1995, chomsky 1975, 1980). ajis vol. 12, no. 1 september 2004 117 subjective inference engine objective inference engine meta knowledge base system and conceptual meta-representations engine subjective module objective module figure 3. alma architecture conclusion the application of mediated activity theory in the area of human computer interaction (hci) analysis. can be extended to the analysis and construction of cooperative agents’ architectures performing communicative and learning activities. this extension is generating a number of interesting artificial intelligence (ai) considerations that are quite different from mediated activity applications in the hci field. internalisation and externalisation together with subjectivity and objectivity aspects of cooperative agents tool mediated activity requires a multi-disciplinary approach quite consistent with intelligent systems and multiagent technology. the dialectical relations between internalisation and externalisation together with agent subjectivity and objectivity can provide new paradigms for agent internal processes such as self-identification, kb storage of internalized concepts and externalization of inferred concepts from the internal knowledge base. we are currently investigating the implementation of the alma architecture under research in legal applications such as online dispute resolutions where the human/agent natural language interaction and semantic communication is used more frequently. the subjective intentional paradigm inbuilt in the alma model can integrate hybrid systems where human customers and software agents communicative activities are performed thus providing equal subjective and intentional position for legal assessment and resolution of online disputes. references austin, j.l. (1965) how to do things with words. oxford university press, new york. bing, j. and sartor, g. (2003). the law of electronic agents, 76-114. unipunskriftserier. oslo. busetta, p., hodgson, a. and ronnquist, r. (1999). specification of coordinated agent behaviour. ijcai ’99 workshop on team behavior and plan recognition. pp 75-81. castelfranchi, c. and e. werner, (1994). artificial social systems, lnai, springer. chomsky, n., (1975). reflections of language. random house, new york. chomsky, n., (1980). rules and representations. columbia university press, new york. ajis vol. 12, no. 1 september 2004 118 cohen, p.r. and h.j. levesque, (1995). communicative actions for artificial agents. proceedings of the first international conference on multi-agent systems, aaai press, san francisco, june 1995. damasio, a.r., (2003). looking for spinoza. william einemann, london. dennett, d. c., (1997). kinds of minds: towards an understanding of consciousness. basic books, new york, us doran j.e., s. franklin, n.r. jennings, and norman t.j., (1997). on cooperation in multi-agents systems. the knowledge engineering review, vol. 12(3). edmond, d. and papazoglou, m.p., (1998). reflection is the essence of cooperation. m.p. papazoglou and g. schlaegeter (eds.), cooperative information systems: trends and directions, academic press, pp.233-262. evans, g., (1982). varieties of reference. ed. j. mcdowell. oxford university press, uk. finin, t., d. mckay, r. fritzson, and r. mcentire, (1994). kcaml, information and knowledge exchange protocol. k. fuchi and t. yokoi (ed.), knowledge building and knowledge sharing, ohmsha and ios press, 1994. http://www.csee.umbc.edu/pub/arpa/kqrrd/papers/ fipa, agent communication language. http://www.fipa.org/spee/fipa97/fip.k97.html. last accessed 02 march 2004. franklin, s. and gaesser, a., (1996). “it is an agent or just a program? taxonomy for autonomous agents.” proc. 3rd international workshop on agent theories, architectures and languages (atal-96), springer, lnai. gobbin, r., (1998). adoption or rejection: information systems and their cultural fitness. information systems and activity theory: tools in context. h. hasan, e. gould and p. hyland (eds.) wollongong university press. gobbin, r., (1998). the role of cultural fitness in user resistance to information technology tools. interacting with computers, 9, 275-285. elsevier science uk. grice, p., (1989). studies in the way of words, cambridge, mass.: harvard university press. gruber, t.r., (1993). a translation approach to portable ontology specifications. knowledge acquisition, 5:pp199-220. guarino, n. and welty, c., (2000). ontological analysis of taxonomic relationships. laender,a and storey, v., eds., proocedings of er-2000: the international conference on conceptual modeling. october. springer-verlag lncs. guarino, n., carrara, m., giaretta, p., (1994). an ontology of meta-level categories. journal of knowledge representation and reasoning: proceedings of the fourth international conference (kr94), morgan kaufmann, san mateo, california. j. mariga ed., irm press, us. jentzsch, r. and gobbin, r., (2003). a cooperative communicative intelligent agent model for ecommerce. managing e-commerce and mobile computing technologies. kant, immanuel. critique of pure reason, trans. norman kemp smith., 1929 edition, st.martin's press, new york. kaptelinin, v., (1992). human computer interaction in context: the activity theory perspective. in proceedings, east-west human computer interaction conference, st petersburg. kaptelinin, v., (1996). computer mediated activity. context and consciousness, nardi, b. ed., mit press, cambridge. keil, f., (1979). semantic and conceptual development: an ontological perspective. cambridge, ma: harvard university press. kempson, r.m. (1977). semantic theory. cambridge university press, uk. klush, m., (2000). intelligent information agents: agent based information discovery and management on the internet, springer verlag. kuutti, k., (1991). activity theory and its applications to information systems. h. e., nissen ed., information systems research, amsterdam, elsevier science publisher. kuutti, k., (1996). activity theory as a potential framework for hci research. context and consciousness, nardi, b., ed., mit press, cambridge. lock, a.j. and peters, c.r., (1996). social relations, communication and cognition. handbook of human symbolic evolution. eds lock a.j. and peters c.r., clarendon press, oxford. ajis vol. 12, no. 1 september 2004 119 lycan, w., (1990). mind and cognition. blackwell, oxford. nagel, t., (1974)."what is it like to be a bat?" philosophical review n.83 (1974), pp.435-50 nardi, b.a., (1996). studying context. context and consciousness. nardi, b., ed.,mit press, cambridge. odell, j. ed., (2000). agent technology, omg document 00-09-01, omg agents interest group, september 2000. parker, s.t. (1993) higher intelligence, propositional language and culture as adaptation for planning. tools, language and cognition in human evolution. eds gibson,k. and ingold, t. cambridge university press, uk. ricoeur, p., (1992). yourself as another. university of chicago press, usa sperber, d. and wilson, d., (1995). relevance: communication and cognition, blackwell. oxford. sperber, d., (1994). understanding verbal understanding in what is intelligence? j. khalfa ed., cambridge university press. sperber, d., (2000). metarepresentations in an evolutionary perspective. in dan sperber representations: a multidisciplinary perspective. oxford university press. spinoza, b., ethics. trans. w.a. white& a.h. sterling, 2001 edition,wordsworth, uk, strawson, peter f., (1966). the bounds of sense: an essay on kant's critique of pure reason. methuen, london. vygotsky, l.s., (1978). minds in society. harvard university press, cambridge. vygotsky, l.s., (1986). .thought and language. mit press, cambridge. wiederhold, g., (1994). interoperation, mediation and ontologies. proceedings of international workshop on heterogeneous cooperative knowledge bases, tokyo. wittgenstein, l. (1958). the blue and brown book. harper and row. new york. microsoft word ajisfinal2003.doc australasian journal of information systems special issue 2003/2004 26 evaluating distributed usability: the role of user interfaces in an activity system lejla vrazalic information systems university of wollongong wollongong nsw 2522 lejla@uow.edu.au abstract traditional definitions of usability localise this fundamental human computer interaction (hci) concept in the user interface and reduce it to a variety of qualitative and quantitative attributes of the computer system. this simplistic view of usability has been used as the basis for developing design and evaluation methods in the discipline. this paper argues that, as a result, hci methods are ineffective and suffer from various shortcomings. it is proposed that the notion of usability must be extended to include contextual factors, and viewed as being distributed across an activity system. adopting this notion of distributed usability then requires a review of existing hci methods. usability testing, as a complete and self-contained hci method, was chosen for this purpose, and the result, a distributed usability evaluation method (duem), is presented in this paper. introduction in his article, trouble in paradise: problems facing the usability community, rhodes (2000) outlines a gloomy future for usability by stating “usability as we know it is dying”. he argues that usability is outdated, misunderstood and faces serious challenges in the face of emerging web technologies because new usability ideas, techniques and methods are not being developed. rhodes (2000) arguments are compelling, but his observations are not new. the human computer interaction (hci) and usability communities are being faced with mounting and pressing concerns for which an instantaneous remedy is not readily available. it is the premise of this paper that in order to begin resolving these concerns, it is necessary to reflect on the very fundamental concept that the discipline is based on – the concept of usability, and then examine the implications of this on hci methods, techniques and tools. this paper argues that there are intrinsic problems with our current definition of usability and that it is necessary to redefine usability to encompass more than just the computer system in isolation. spinuzzi’s (1999) notion of distributed usability and nardi and o’day’s (1999) information ecology are proposed as a means of doing this. the implications of extending the definition of usability are significant to the way in which existing hci methods are viewed and used. usability testing, as a complete and self-contained hci method, has been chosen to demonstrate these implications. a case will be made that the problems associated with current usability testing are symptomatic of the traditional view of usability and can be overcome by developing a testing method based on distributed usability. activity theory, and its associated principles, will be proposed as the underlying framework for developing this method. a brief description of the method will be provided with an example of its application to demonstrate its benefits. an operational model of the method is currently being finalised. for the purposes of this discussion, the terms “computer system”, “system” and “interface” will be used interchangeably. traditional usability “many developers dream of an algorithm giving an exact measure of the usability of a product. ideally, one could take the source code of a program, run it through an analysis program giving a single number in return: usability = 4.67” (sintef group, 2002) hci as a discipline bases itself on the need to design, evaluate and implement computer systems for human use (acm sigchi, 1992). the notion of usability is therefore fundamental to hci. usability australasian journal of information systems special issue 2003/2004 27 is an abstract concept intended to encase and encompass both the design and evaluation of computer systems. it is the glue that binds the entire systems design and development process together. to some, usability is a science. to others, it is an art form. as such, usability is a concept that does not lend itself to a precise and clear-cut definition. generally, usability refers to the ease of operating a system interface. the simplicity of this statement may appear to be misleading due to the plethora of other, seemingly more comprehensive, definitions of usability that exist. however, most of these definitions are based on this central notion of “ease of use” (miller, 1971 cited in shackel, 1986). since the notion of ease of use in itself is somewhat vague, some authors and researchers have defined usability in terms of multiple high-level criteria or attributes. shackel (1986) proposes that usability can be specified and measured numerically in terms of four operational criteria: effectiveness (a required level of performance by a percentage of specific users within a range of usage environments), learnability (a pre-defined time period from the start of user training and based on a specified amount of training), flexibility (the levels of adaptation and variability in possible tasks) and attitude (user satisfaction levels after continued use). nielsen (1993) views usability as a narrow concern when compared to the issue of system acceptability, and models usability as an attribute of system acceptability. however, similarly to shackel (1986), nielsen argues, usability itself can be further broken down into five attributes: learnability (the system should be easy to learn), efficiency (the system should be efficient so that high levels of productivity are possible), memorability (the system should be easy to remember and not require relearning), errors (the system should have low error rate and enable quick recovery after errors) and satisfaction (the system should be pleasant to use). in contrast to these high-level criteria and attributes, norman (1988) conceptualises usability in terms of design principles based on a combination of psychological theory and everyday user experiences. the aim of these principles is to help designers make improvements to the system and explain different aspects of their design to the various stakeholders (thimbleby, 1990). while numerous principles have been operationalised in hci, the most well-known have been proposed by norman (1988) and include: visibility (making the system functions visible so that each function corresponds with a control), mapping (a direct, natural relationship between the controls and their functions), affordance (perceived and actual properties of an object that determine how it can be used), constraints (limiting the behaviours and possible operations on an object) and feedback (sending back information to the user about what action has been done). what is clear from the above is that a single universal definition for usability does not exist, which makes it a confusing concept to explain and, more importantly, justify to the business community (rhodes, 2000). however, the general implication in most definitions of usability is that usability is located within the system itself. it can be thought of purely as an ‘attribute of the entire package that makes up a system’ (dumas and redish, 1993) and reduced to factors such as ease of use (of the interface), learnability (of the interface), memorability (of the interface), visibility (of the interface), mapping (of the interface) or any of the other attributes and principles discussed above. shackel (1986) was one of the first researchers to alert the hci community to need for an extended view of usability, embracing four principal components (the user, the task, the system and the environment). usability, shackel argued, was about achieving harmony in the interaction between these four components. since then, a number of other researchers have increasingly become critical of the “traditional” localised view of usability (thomas and macredie, 2002; spinuzzi, 1999; nardi and o’day, 1999; beyer & holtzblatt, 1998; nardi, 1996; engeström and middleton, 1996; hutchins, 1995; kling & iacono, 1989) because it does not take into consideration those attributes which extend beyond the computer system, including the social context, the work practices and the historical development of the activities that the computer system supports. thomas and macredie (2002) went so far in their criticism of traditional usability as to argue that the current conceptions of usability are ill-suited, unwieldy, meaningless and unable to handle the “digital consumer” (p 70). as a result, an alternative notion of distributed usability has emerged. australasian journal of information systems special issue 2003/2004 28 distributed usability spinuzzi (1999) argues that the traditional view of usability is inadequate because it disregards the influences and consequences of contextual factors such as the interaction between humans, the use of artifacts other than the system and the actual work practices of users. karat (1997) supports this view by describing usability as an attribute of the interaction with a system in a context of use. in spinuzzi’s (1999) opinion, usability is distributed across an activity network which is comprised of assorted genres, practices, uses and goals of a given activity. an activity network represents a unit of analysis that takes into account individual users working with others as part of a larger activity. thinking about usability in this way provides us with a more encompassing view of the system (and its interface), the users, as well as the users’ goals, and, spinuzzi argues, leads us to consider solutions that we may not have if we had studied individuals alone. for example, in his study, spinuzzi found that breakdowns in the users interaction with the system (as defined by bødker, 1991) were not caused simply by the size of the mouse pointer or even the levels of user training, but could be attributed to “deeper discoordinations” between the interface and other genres relating to the context of use. this interpretation of usability would not have been possible if only localized attributes of the interface, such as ease of use, learnability and memorability, were examined. spinuzzi advocates the study of on-screen and off-screen genres, or typified forms, and their mediatory relationships in the context of an ecology of interrelated tools and activities. nardi and o’day (1999) also view this arrangement of tools, which jointly mediate activities, as belonging to an information ecology. they define an ecology as a “system of people, practices, values, and technologies in a particular local environment” (p.49) which focuses on human activities served by technology, rather than technology itself. indeed, where traditional views of usability relate only to a system consisting of hardware and its associated software, distributed usability proposes to extend the system to include an ecological context consisting of people, their work practices and activities (supported by the hardware and software system as well as other tools) and the social context of these activities. this idea echoes shackel’s (1986) four principal components of usability (the user, the task, the system and the environment), however spinuzzi (1999) and nardi and o’day (1999) view these components as being interrelated. the components cannot be fully understood individually because together they make up an extended system that is more than just a sum of its parts. the distribution of usability across this extended system or ecology has significant implications for both the design and evaluation of systems. clearly, the design process could not proceed without taking into account and devoting substantial resources to understanding the larger context of user activities, and incorporating this context into the design process. the implications for system evaluation, however, are even more considerable in light of existing evaluation methods, which rely on the traditional view of usability and therefore primarily focus on the system as an artifact in isolation and on assessing the localised attributes of that isolated system. if the hci community is to adopt a distributed view of usability, it will be necessary to revisit our existing evaluation methods to determine their usefulness, reliability and validity, and develop new methods that are based on a fundamentally different notion of usability. to this end, a distributed usability evaluation method has been developed. usability testing usability testing is “the gathering of information about the use of prototypes of software products from users who are not involved in the design of the products themselves” (holleran, 1991). in the 1980s, laboratory based usability testing, as described by rubin (1994) and dumas and redish (1993) emerged as the golden standard for usability evaluation (lewis, 2001). since then, other usability evaluation methods have been developed, including heuristic evaluation (nielsen & molich, 1990) and cognitive walkthroughs (polson et al, 1992). unlike, usability testing, which involves observing users directly interact with a system, these methods are predictive and conducted by teams of experts. bailey (1993) and tullis (1993) have questioned experts’ ability to predict australasian journal of information systems special issue 2003/2004 29 usability problems following empirical studies, which indicated that experts experienced problems when trying to predict human performance. the results of these studies should come as no surprise to anyone who has been involved in usability testing with users. gould and lewis (1985) laid down the basic principles of user-centred design as being early focus on users and tasks, empirical measurement and iterative design. usability testing is fundamental to the achievement of all three principles because it involves users, it enables empirical measurement to be undertaken, and it supports the notion of iterative design following testing of early system prototypes. due to this central role of usability testing in user-centred design approaches, it was decided that usability testing, rather than predictive evaluation methods, would form the basis for developing a method to demonstrate the use of distributed usability. however, usability testing in its current form is plagued by practical and methodological problems. these will be discussed next. problems associated with usability testing despite its status as the “golden standard” in usability evaluation, a number of authors have identified pitfalls associated with the usability testing process. wixon and wilson (1997), for example, have drawn attention to the problems with setting the usability testing goals during the initial stages of usability testing. they claim that usability testing goals may be too ambitious or too many in number. this in turn creates the need for more elaborate and lengthy test procedures, as well as generating more data to analyse. this is highly undesirable, considering that human factors experts spend on average 33.2 hours per user when conducting usability testing (jeffries et al, 1991). increasing the complexity of the testing process also increases the risk of failure (wixon & wilson, 1997). conversely, arguments have been put forward that usability testing cannot be used to evaluate every aspect of the system (wilson & wixon, 1997), making the process deficient in collecting data about the whole system. therefore, there appears to be a trade-off between the complexity of the test objectives and the need to test the system as a whole. holleran (1991), on the other hand, identified a number of methodological pitfalls in the usability testing process associated with sampling. he bases his argument that generalisability confidence increases with the number of participants who are representative of the target system users. while the former issue has been addressed by the work of nielsen and landauer (1993) and virzi (1992), holleran’s (1991) concern lies in the degree to which the participants are representative of typical users. this issue is problematic for two reasons: participants may only be as representative as the evaluators’ ability to understand and categorise the target user population (rubin, 1994) and may be difficult to identify, access and recruit representative users, particularly where online systems are being tested. generating the scenarios to be used in the usability tests is another area of contention owing to the complexity of the task. scenarios are intended to reflect the activities that typical users perform and their associated goals. however, in practice, most scenarios simply reflect functions and aspects of the system that are representative of its capabilities. the implication is that, rather than evaluating whether the system supports the users’ tasks, usability testing is concerned with the evaluation of specific functions of the system that have been implemented, regardless of the usefulness or relevance of these functions. generally, scenarios for usability testing are generated by the developers (hartson et al, 2001) and/or evaluators which introduces an inherent bias into the evaluation. jacobsen et al (1998) and hertzum and jacobsen (2001) have termed this bias the evaluator effect. the presence of the evaluator effect means that different evaluators who are testing the same system will detect substantially different sets of usability problems because the use of their judgement is required. hertzum and jacobsen (2001) claim that the evaluator effect “persists across differences in the system domain, system complexity, prototype fidelity, evaluator experience, problem severity, and with respect to detection of usability problems as well as assessments of problem severity” (p 439). their findings show that on average the agreement between any two evaluators can range from 5% to 65% which is an astonishing discrepancy. the cue studies by molich et al (1998; 1999) have found even more divergent levels of agreement. australasian journal of information systems special issue 2003/2004 30 the involvement of evaluators and/or developers in the task scenario generation process also often results in the wrong terminology being used to describe users’ tasks. it is critically important that scenarios are written in the language of the users because the way in which the users interpret the scenarios will ultimately have an effect on the results of the usability testing. if, for example, a user is unable to understand the requirements of the task, this indicates that the task is not representative and that the user may not be able to complete the tasks because of interpretation rather than problems with the system. evolving from formal experiments, usability testing has traditionally been situated in a laboratory setting. this controlled environment and the use of testing equipment such as recorders and cameras is designed to enable evaluators to evaluate a system as objectively as possible and generate quantitative data. this artificially created space, however, carries with it a series of negative connotations, with users known as subjects, controlled by the evaluators measuring specific variables. despite the seeming objectivity enabled by the testing environment, there are also several subjective factors involved at crucial stages of the testing process, including during the setting of usability goals, the generation of task scenarios and the interpretation and analysis of results, both carried out by expert evaluators. while the quantitative performance data collected during a usability test may be useful in general terms, it may not be a reflection of actual performance because contextual influences that are inherent to real user activities have not been factored into the data. finally, the use of descriptive and inferential statistics does not necessarily provide insights into whether a system actually works. measures of statistical significance used in analysing quantitative data is simply a measure of probability that the results did not occur due to chance (rubin, 1994). this type of ‘micro-level’ analysis does not actually prove that the system is usable, or more importantly, useful. a major shortcoming of a laboratory based usability test is the unnaturalness and artificiality of the environment (hartson et al, 2001; wilson & wixon, 1997; rubin, 1994). testing carried out in a laboratory is radically different to the natural, everyday practices that humans engage in through interaction with other humans, systems and tools. this reduces the ecological validity of the evaluation process. according to thomas and kellog (1989), ecological validity refers to how close a testing situation is to the real world. they identify four ecological gaps in laboratory based usability testing. the mismatch between the users’ real context and a test context is referred to as a “work-context gap”. the work-context gap does not only include differences in the physical context, but the job context and social and cultural contexts of user activities. rubin (1994) also points out that any form of usability testing (in a laboratory or in the field) depicts only the situation of usage and not the situation itself. naturally, this will have an effect on the test findings because the situation in this instance represents the context of the usage and the two are inextricably linked. rubin (1994) has suggested creating laboratory spaces which resemble the users’ real context, however, even with these improvements, usability testing is not a perfect indicator of field performance (nielsen & phillips, 1993). one of the reasons for this lies in the participants’ motives. holleran (1991) argues that participants in a usability test will persevere in doing tasks which they are unlikely to do in their own context out of willingness to comply with the evaluators’ requirements. thomas and kellog (1989) have called this the “user gap”. it is also more widely known as the hawthorne effect and amplified by the presence and use of video cameras in a usability lab to record testing sessions. the artificiality of the laboratory is also conducive to the use of brief and clearly defined task scenarios which are usually completed within a specific time period. this is in stark contrast to the ill-defined and ongoing activities that users actually engage in. this mismatch is known as the “task gap”. finally, the “artifact gap” refers to the differences between short-term system usage during a test and long-term usage in the real world. some usability problems may only emerge after prolonged system usage which is clearly not possible in the context of a laboratory. holleran (1991) also questions the validity and reliability of the data collected during usability testing. validity refers to whether the evaluators are actually measuring what they intend to measure. considering the problems associated with deriving representative task scenarios, the validity of usability testing results remains a controversial issue. furthermore, it is not always possible to australasian journal of information systems special issue 2003/2004 31 collect quantitative data or data for which suitable statistical measurements are possible (holleran, 1991). reliability is the extent to which the data produced in one usability test will match the data produced in another if the testing is replicated under the same conditions. the cue studies (molich, 1998; 1999) have shown the reliability of usability tests to be quite low. the end result of a usability test is a series of, presumably, usability problems that need to be fixed. just as there is no clear definition of usability, the characterization of a usability problem remains elusive and can perhaps be seen as one of the “holy grails” of hci. the term usability problem is commonly used to refer to any difficulties or trouble a user may have while using the system, or any faults in the system which cause a break down in the interaction. however, there is no explicit criteria defining when such a difficulty or fault constitutes a usability problem (hertzum & jacobsen, 2001) and so any problem reported is deemed to be a usability problem. different evaluation methods identify different usability problems and no one single method can be relied on to uncover every usability problem with a system. in fact, it is quite possible for the same method to produce different outcomes as molich et al. (1998, 1999) demonstrated in the cue studies. usability testing is also plagued by logistical problems such as participants not showing up (wilson & wixon, 1997), scheduling convenient times for the testing, and problems with obtaining and maintaining specialised equipment. as such, usability testing is the most expensive and time consuming uem (hartson et al, 2001; jeffries et al, 1991). it also requires a working prototype and specialised expertise to conduct (jeffries et al, 1991) because, even though users take part in the process their role is restricted. usability testing is strongly controlled by the evaluators at every stage (mayhew, 1999) and driven by the system (sweeney et al, 1993). the users are reduced to being passive participants who are controlled, observed, recorded and surveyed in order to collect performance data. the evaluators decide what, how, when and where to evaluate and the users have no input into the design of the evaluation or the interpretation of the results. finally, one of the most critical disadvantages of usability testing is that, like most other uems, it has no theoretical basis. loosely based on the formal experiment, the method emerged from practice and has since been widely adopted and applied. different approaches to usability testing are used and then compared to find out which approach works better and why, without any theoretical framework to allow this type of analysis. holleran (1991) refers to this as “dustbin empiricism”. the problems with usability testing identified above are symptomatic of the narrow, traditional view of usability as a localised attribute or quality of a single system. sweeney et al (1993) support this view by arguing that usability testing in a laboratory tends to be driven by the system. the use of a laboratory for the testing process enables evaluators to focus only on the system and eliminate any external or contextual variables that would interfere with the assessment of the system performance, thus making it easier to isolate and identify problems. the system and its functions drive the selection of users for the test sample because representative users are usually chosen after the functions of the system are known. the tasks developed for the usability test are also based on the system functions, rather than user needs. instead of assessing whether the system does what the users want it to do, usability testing usually tests how well the system does what it can do. finally, the confusion surrounding usability is reflected in the difficulty of defining a usability problem. by adopting the notion of distributed usability as the starting point for the development of a usability testing method, it may be possible to eliminate the problems discussed above. however, a notion in itself is insufficient for this purpose. as mentioned above, it is necessary to support the notion with a theoretical framework on which to base the distributed usability testing method. cultural historical activity theory (chat) offers such a framework. cultural historical activity theory cultural historical activity theory, or simply activity theory (at) as it is widely known, provides a broad conceptual framework that can be applied to the human-computer interface in such a way as to empower the computer user with the necessary tools to work though the interface in order to achieve desired outcomes. historically, at draws on the vygotskian (1978) theory of tool mediation or the mediation of human activities by the use of tools. this approach deviates from the cognitive australasian journal of information systems special issue 2003/2004 32 approach in that the computer system is seen as distinctly different in both character and composition to its human user. from an at perspective, people are embedded in a socio-cultural context and their behaviour cannot be understood independently of it. furthermore they are not just surrounded by the context but actively interact with it and change it. humans are continually changing activities and creating new tools. this complex interaction of individuals with their surroundings has been called an activity and is regarded theoretically as the fundamental unit of analysis, a system that has structure, its own internal transitions and transformations, its own development (leont’ev, 1981). at is becoming more widely known by hci researchers in the west (kuttii, 1996; engeström, 1995; kaptelinin, 1994; bødker, 1991; nardi, 1996) since it was introduced in russia in the eighties and early nineties. it’s most current and widely-adopted form is engeström’s (1987) systemic model shown in figure 1. in this model, the subject refers to the individual or group engaged in the activity, while the object refers to that (either 'raw material' or 'problem space') at which the activity is aimed. the object defines the activity and is transformed into an outcome using physical and symbolic mediating tools. the community consists of individuals who share the same object of the activity, while the division of labour refers to both the horizontal division of tasks between the members of this community and to the vertical division of power and status. the rules refer to the explicit and implicit regulations, norms and conventions constraining the interactions within the activity system (engeström, 1987). kuutti (1996) describes the key principles of at as follows: activity as the basic unit of analysis instead of analysing only human actions, at proposes that a minimal meaningful context for these actions should be included in the analysis and this unit comprising actions in a context is an activity. figure 1. the human activity system (engeström, 1987) history and development activities are in a constant state of evolution and therefore, it is necessary to historically analyse an activity in order to gain an understanding of the current situation. a hierarchical system of contradictions is inherent to activity systems and it is these contradictions that cause an activity to develop over time. engeström (1987) states that contradictions emerge as a result of conflicts within and between activity systems. he categorizes four levels of contradictions: • primary contradictions within the elements of the central activity, usually between the value and exchange value of an element; • secondary contradictions arise between the elements of the central activity; • tertiary contradictions take place between the object of the activity and the object of a more culturally advanced activity, and • quaternary contradictions occur between the activity and its ‘neighbouring’ activities, such as the tool-producing activity, subject-producing activity, etc. division of labour subject object community rules 1.1.1.1.1.1.1.1 t outcome australasian journal of information systems special issue 2003/2004 33 artifacts and mediation activities are mediated by artifacts and artifacts themselves are created during the development of an activity. this dual relationship further implies the developmental nature of activities. structure of an activity an activity is directed towards an object and the object is what distinguishes one activity from another. the transformation of the object into the outcome motivates the existence of the activity. furthermore, the object and motive could undergo changes during the development of an activity. levels of an activity an activity, which is driven by motives, is realised through conscious actions which are directed towards specific goals. those actions, in turn, are implemented through operations dependent on the available conditions. the relationship between the elements of this hierarchy, depicted in figure 2, is dynamic. for example, when working with a computer for the first time, using a mouse is a conscious action requiring the deliberate attention of the user. through practice, this action will collapse to the level of operations where it becomes habitual and subconscious. however, if the conditions change such that the mouse stops working, the user will be forced to focus his/her actions towards the mouse once again, returning the operation to the level of a conscious action. this type of interruption in the internal structure of an activity is termed a breakdown (bødker, 1991). breakdowns occur for various reasons, but they are evident in system use when the system itself becomes the object of the user’s actions, rather than having a mediating role. figure 2. structure of an activity (leont’ev, 1981) the notion of distributed usability implies the distribution of usability across an activity network comprised of assorted genres, practices, uses and goals of a given activity. activity theory provides a unifying framework for these elements because the given activity is the basic unit of analysis consisting of the users (the subjects), their task (the object), the system and other objects utilised to complete the task (the mediating tools) and the social context (the community, the division of labour and the rules). furthermore, the structure of an activity is concerned with the users’ motives, their goals and the conditions in which these are made possible. a distributed usability testing method based on activity theory has been developed. the following section will briefly outline the operationalisation of this method. australasian journal of information systems special issue 2003/2004 34 figure 3. a model of the distributed usability evaluation method (duem) distributed usability evaluation method the distributed usability evaluation method (duem) places the users at the centre of the evaluation by involving them in the design of the evaluation process, and reduces the computer system to a support role as one of the many mediating tools in user activities. the focus, instead, is on identifying problems across the entire activity network. specific usability attributes, such as ease of use, learnability and memorability are not examined or measured. the main concern is to identify usability problems which are caused by breakdowns in the interaction and contradictions in the activity network. a working model diagram of the method is shown in figure 3. phase i: understanding users’ activities the initial phase involves gaining an in-depth understanding of real user activities by observing and interviewing users who actually use the system being tested in their everyday activities. however, the system itself is not the central focus of this phase. it is only important to the extent to which it supports the users’ activities. where appropriate, field interviews and observations can be carried out in order to understand users’ needs, desires and their approach to the work they do (beyer and holtzblatt, 1999). the interviews can be carried out on a one-to-one basis or in focus groups involving teams that carry out the same activity. this provides a forum for discussing and observing the social interactions between users, and for developing an understanding of the social context by gathering information, stories and anecdotes. due to the problematic nature of gathering this type of ad hoc information, the at principles described previously can be used to make sense of the information gathered and also provide evaluators with a common vocabulary (nardi, 1996) as at terminology is a close reflection of users’ activities and, as such, easily understood by users. questions based on at such as the following can be used as a guide to understand an activity: • “who are the users?” – to collect information about the users’ background in relation to three things: the general activities they perform which are supported by the system being evaluated; the system itself; and the use of the system specifically to carry out the activities. this provides evaluators with an insight into the users’ knowledge of the domain (activity), the system (mediating tool) and the system use (actions). • “what is the object of the users’ activity?” – to appreciate the essence of what the users are engaged in. the object is the “ultimate goal” of the user. an understanding of the object allows evaluators to situate the system and its role in the activity. it draws the attention away from the system itself, and positions it in relation to what the user is trying to achieve. • “what are the tools that the user makes use of?” – to determine the complete set of tools that a user makes use of in carrying out the activity. the system will constitute only one of design evaluation (with users) 1.1.1.1.1.2 o focus group discussion user activities observation and discussion (with users) analysis of results 1.1.1.1.1.3 p 1.1.1.1.1.4 p1.1.1.1.1.5 p results australasian journal of information systems special issue 2003/2004 35 many tools, both material and psychological in nature. since the use of the tools mediates the users’ activities and different actions can involve the use of a different tools, the evaluators will gain an insight into the alternative ways in which the activity can be undertaken. if the users’ indicate a preference for a specific set of actions over others to carry out an activity, this may have implications for the interface. • “who does the user interact with in carrying out the activity?” – to understand the users’ community, the rules of this community and the division of labour within the community. this represents the users’ social context and allows evaluators to identify a full set of system stakeholders. • “how was the activity carried our prior to the system being evaluated?” – to define the historical development of the activity and any remnants of the previous state of the activity which may have an impact on its current form. the information collected from the interviews or focus groups provides an integrated, holistic view of the main activity and other intersecting activities and a description of the various mediating tools used in performing the activity, as well as an explanation of how they are used. the key objective of this phase is to explore the users’ work practice (borgholm & madsen, 1999) and gain an understanding of real user activities. it is important to allow the evaluators to immerse themselves in the users’ practice and, by applying at principles, gain a shared understanding and interpretation of what transpires during a typical activity which is supported by the system being evaluated. once a common interpretation has been developed, the evaluators can proceed with phase two, which involves planning the evaluation process with the users. phase ii: evaluating with users unlike with traditional usability testing, the involvement of users does not end with phase i. instead, users are directly involved in planning the evaluation process. during phase ii, the evaluators and users work jointly and iteratively to design and develop a set of evaluation goals based on the user activities identified during phase i. the evaluation goals are not system specific. they are defined in relation to user activities. this is in contrast to traditional usability testing which uses the system as the starting point for developing evaluation goals. based on these goals, a means of conducting the evaluation is collaboratively negotiated between the users and the evaluators. this can involve the use of activity scenarios or simply a free-form evaluation whereby users and evaluators use and discuss the interface in the users’ natural environment. the latter form is preferable because it enables evaluators to study the impact of the social context. since the duem does not focus on attributes of the system and micro-level analysis (such as the number of clicks, or time taken to complete a task) per se, the use of cameras and recording is not required. it is important to note that during phase i of the evaluation process, the primary focus was on user activities, and not the system. the system is discussed only to the extent of the role it plays in the users’ activities in conjunction with other tools. in phase ii, this role is actually evaluated from the point of view of the activities. the use of prescriptive scenarios to test the functions of the system is not advocated. instead, descriptive scenarios may be drawn up by users and evaluators based on the user activities as a guide to direct the evaluation process. the outcome of phase ii is qualitative data about the users’ interaction with the system in the form of written notes based on observations and discussions which is then analysed using activity theory. phase iii: analysis with users during this phase, the rich data collected in phase ii is analysed in terms of the breakdowns and contradictions. the notes are used to identify breakdowns (as defined by bødker (1991)) in the activity system and co-relate these breakdowns to the contradictions (as defined by engeström (1987)). breakdowns do not necessarily occur only in the actual use of the system. they can result from a change of focus in any element of the activity system but may have in impact on the system use. these breakdowns can be mapped to contradictions that are identified within and between australasian journal of information systems special issue 2003/2004 36 activities. the mapping of breakdowns and contradictions defines the usability problems in the activity network. this process will be demonstrated with an example based on a system described by nielsen (1990). the lyre system, a french hypertext system used for teaching poetry, allows students to analyse poetry by adding new annotations to poems using hypertext anchors. the system was based on the french tradition of students working within a framework set up by the teacher, so it did not allow students to add new viewpoints, a facility reserved only for the teachers. in scandinavian countries, the focus of teaching is on increasing students’ potential to explore and learn independently. had the lyre system been implemented in scandinavia, its use would have resulted in a series of breakdowns caused by primary contradictions within the tool (lyre), secondary contradictions between the tool (lyre), the subjects (the students and their expectations), the object (to explore and learn independently) and the division of labour (between the teacher and the students). tertiary contradictions would also have occurred between the object of the french teaching activity (to analyse poetry) and the object of the scandinavian activity (independent student exploration and learning). by identifying usability problems with the system as breakdowns caused by these contradictions, it would have been possible to determine whether the system supports the students’ activities, instead of focusing on the ease of use, learnability or memorability of the system. it would have been possible to establish that the system is intrinsically flawed. the duem proposes that even the analysis of the results be carried out collaboratively with the users. their presence ensures that breakdowns in the activities can be clarified and considered without the need for re-interpretation by the evaluators. it also enables evaluators to assess which breakdowns are more serious in nature and discuss with users how these can be resolved. for example, during the evaluation of a web-based system of a large government organisation, a serious breakdown occurred which was mapped to a number of contradictions in the activity system. the system was developed to support investigative activities. in order to carry out those activities, users had to submit requests to supervisors using the system to access personal information about clients. users were required to complete an online form which was then submitted to a supervisor for approval in the form of an e-mail message. however, this was observed to be ineffective in those cases where urgent approval was required. if, for example, a supervisor was away, the approval would be delayed (with serious consequences). to overcome this problem, users would first contact each supervisor by phone to determine who was available to approve a request immediately. they would then submit the request to that supervisor using the system. the breakdown in this activity occurred because the users had to shift their focus from the investigative activity to finding an available supervisor by phone. although the users’ direct interaction with the system did not cause this breakdown, a contradiction between the system (the tool), the object of the activity and the roles of stakeholders involved did, because the system was not designed to support the activity’s division of labour effectively. when the analysis was carried out with the users, they provided two suggestions for fixing the problem: designing the system so that it alerts the users which supervisor is available (i.e. logged on to their e-mail account) and flagging the e-mail message so that supervisors were able to detect these types of requests easily and distinguish them from other e-mail messages. had the above system been evaluated using the traditional usability testing method which focuses primarily on the interface, the usability problem would have remained undetected because there is no provision for evaluating the usability across the entire activity system. since the problem was not directly related to the interface and a usability laboratory would not have afforded observing and understanding the way in which the actual activity was carried out, a serious flaw in the system may have been omitted. duem offers a number of advantages to both researchers and practitioners. primarily, the method overcomes most of the problems associated with traditional usability testing described previously, by involving users in the design of the evaluation process, and focusing user activities instead of system functions. duem does not evaluate the system in isolation from other elements in the activity network. this situation is only made possible by starting off with a view to distributed usability in the first place. australasian journal of information systems special issue 2003/2004 37 conclusion this paper has argued that fundamental problems with our current understanding of usability have emerged and that it is necessary to redefine usability to encompass more than just the computer system in isolation. the notion of distributed usability has been proposed as a means of achieving this. however, the implications of redefining this basic concept are significant to our existing usability evaluation methods, and usability testing in particular owing to its role in user-centred design. it has been argued that the problems associated with the current usability testing method are symptomatic of the traditional view of usability and can be overcome by developing an evaluation method based on distributed usability. activity theory, and its associated principles, has been proposed as the underlying framework for the distributed usability evaluation method (duem). a brief description of the method has been provided with two examples to demonstrate its use and benefits. references acm sigchi (1992) curricula for human-computer interaction, acm press. bailey, r.w. (1993) performance vs preference, in: proceedings of the human factors and ergonomics society 37th annual meeting, pp. 282-286. beyer, h. and holtzblatt, k. (1998) contextual design: defining customer-oriented systems, morgan kaufmann publishers. beyer, h. and holtzblatt, k. (1999) contextual design, interactions. 6, pp. 32-42. bødker, s., 1991, through the interface: a human activity approach to user interface design, lawrence erlbaum. borgholm, t. and madsen, k.h. (1999) cooperative usability practices, communications of the acm. 42, pp. 91-97. dumas, j. s. and redish, j. c. (1993) a practical guide to usability testing, ablex publishing. engeström, y. and middleton, d. (eds), 1996, cognition and communication at work, cambridge university press. engeström, y. (1987) learning by expanding: an activity-theoretical approach to developmental research, orienta-konsultit. engeström, y. (1995) polycontextuality and boundary crossing in expert cognition: learning and problem solving in complex work activities, learning and instruction. 5, pp. 319-336. gould, j. d. and lewis, c. (1985) designing for usability: key principles and what designers think, communications of the acm. 28:3, pp. 300-311. hartson, h. r., andre, t. s. and williges, r. c. (2001) criteria for evaluating usability evaluation methods, international journal of human-compute interaction, 13:4, pp. 373-410. hertzum, m. and jacobsen, n.e. (2001) the evaluator effect : a chilling fact about usability evaluation methods, international journal of human-computer interaction. 13:4, pp. 421-444. holleran, p.a. (1991) a methodological note on pitfalls in usability testing, behaviour & information technology. 10:5, pp. 345-357. hutchins, e. (1995) cognition in the wild, mit press. jacobsen, n. e., hertzum, m. and john, b.e. (1998) the evaluator effect in usability tests, chi 98 conference summary, acm press, pp. 255-256. jeffries, r., miller, j. r., wharton, c. and uyeda, k. m. (1991). user interface evaluation in the real world: a comparison of four techniques, proceedings of chi 91, acm press, pp. 119-124. kaptelinin, v. (1994) activity theory: implications for human computer interaction, in: humanmachine communication for educational systems design, m.d. brouwer-janse and t.l. harrington, eds., springer-verlag. karat, j. (1997) user-centred software evaluation methodologies, in: handbook of humancomputer interaction, helander, m. g., landauer, t. k. and prabhu, p. v., eds., elsevier science, pp. 689-704. australasian journal of information systems special issue 2003/2004 38 kling, r. and iacono, s. (1989) the institutional character of computerized information systems, office: technology & people. 5:1, pp. 7-28. kuutti, k. (1996) activity theory as a potential framework for human-computer interaction, in: context and consciousness: activity theory and human computer interaction, b. nardi, ed., mit press. leontiev, a. n. (1981) problems of the development of the mind, progress publishers. lewis, j. r. (2001) current issues in usability evaluation, international journal of humancomputer interaction. 13:4, pp. 343-350. mayhew, d. j. (1999). the usability engineering lifecycle: a practitioner's handbook for user interface design, morgan kaufmann publishers. molich, r., thomsen, a. d., karyukina, b., schmidt, l., ede, m., van oel, w. and arcuri, m. (1999) comparative evaluation of usability tests, in: proceedings of chi’99, acm press. molich, r., bevan, n., curson, i., butler, s., kindlund, e., miller, d. and kirakowski, j. (1998) comparative evaluation of usability tests, in: proceedings of the upa conference, pp. 189-200. nardi, b. (1996) activity theory and human-computer interaction, in: context and consciousness: activity theory and human computer interaction, b. nardi, ed., mit press. nardi, b. a. and o’day, v. l. (1999) information ecologies: using technology with heart, mit press: cambridge. nielsen, j. (2000) why you only need to test with 5 users, alertbox. march 19. nielsen, j. (1993) usability engineering, academic press. nielsen, j. (1990) designing for international use, in: human factors in computing systems, proceedings of chi'90, acm press, pp. 291-294. nielsen, j. and landauer, t. (1993). a mathematical model of the finding of usability problems. proceedings of interchi ’93, acm press. nielsen, j. and phillips, v. l. (1993). estimating the relative usability of two interfaces: heuristic, formal, and empirical methods compared, proceedings of interchi 93, acm press, pp. 214-221. nielsen, j. and molich, r. (1990) heuristic evaluation of user interfaces, in: proceedings of chi’90, acm press, pp. 249-256. norman, d. a. (1988) the psychology of everyday things, basic books. polson, p. g., lewis, c., rieman, j. and wharton, c. (1992) cognitive walkthroughs: a method for theory based evaluation of user interfaces, international journal of man-machine studies. 36, pp. 741-773. rubin, j. (1994) handbook of usability testing: how to plan, design, and conduct effective tests, wiley. rhodes, j. s. (2000) trouble in paradise: problems facing the usability community, (april 17, 2003); http://webword.com/moving/death/ html. shackel, b. (1986) ergonomics in design for usability, in: people and computers: designing for usability, proceedings of the 2 nd conference of the british computer society human computer interaction specialist group, harrison, m. d. and monk, a. f., eds., cambridge university press, pp. 45-64. sintef group (2002) the foundation for scientific and industrial research at the norwegian institute of technology, (march 15, 2003); http://www.oslo.sintef.no/avd/ 32/3270/brosjyrer/ engelsk/6.html. spinuzzi, c. (1999) grappling with distributed usability: a cultural-historical examination of documentation genres over four decades, in: proceedings of the 17th annual international conference on computer documentation, new orleans, pp. 16-21. spool, j. and schroeder, w. (2001) testing websites: five users is nowhere near enough, in: proceedings of chi’2001, acm press, pp. 285-286. sweeney, m., maguire, m. and shackel, b. (1993) evaluating user-computer interaction: a framework, international journal of man-machine studies. 38, pp. 689-711. australasian journal of information systems special issue 2003/2004 39 thimbleby, h. (1990) user interface design, addison wesley, harlow (uk). thomas, j.c. and kellogg, w.a. (1989) minimizing ecological gaps in interface design, ieee software, 6:1, pp. 78-86. thomas, p. and macredie, r. d. (2002) introduction to the new usability, acm transactions on computer-human interaction, 9:2, pp. 69-73. tullis, t. s. (1993) is user interface design just common sense?, in: proceedings of the 5th international conference on humancomputer interaction, pp. 9-14. virzi, r.a. (1992) refining the test phase of usability evaluation: how man subjects is enough?, human factors, 34:4, pp. 457-468. vygotsky, l. s. (1978) mind in society, harvard university press. whiteside, j., bennett, j. and holtzblatt, k. (1988) usability engineering: our experience and evolution, in handbook of human-computer interaction, m. helander, ed., northholland. wixon, d. and wilson, c. (1997) the usability engineering framework for product design and evaluation. in helander, m. g., landauer, t. k. and prabhu, p. v. (eds) handbook of human-computer interaction, 2 nd edition, elsevier science, pp 653-688. the australasian journal of information systems (ajis) began in 1993 and was based on the simple realisation that there was a need for an academic ‘outlet’ for is research in australia australasian journal of information systems volume 14 number 1 nov 2006 case studies of e-commerce adoption in indonesian smes: the evaluation of strategic use mira kartiwi information systems university of wollongong wollongong, nsw 2522 australia email: mk03@uow.edu.au abstract the use of e-commerce in small medium-sized enterprises (smes) has become an important topic in information systems research. at present, there is a large number of well-documented benefits derived from e-commerce for small-medium enterprises (smes) both in theoretical and practical literature. despite the enormous attention given to encourage smes to adopt e-commerce both by academics as well as governments, little research has been carried out in identifying strategies of ecommerce adoption for smes in developing countries, especially indonesia. it is, therefore, the aim of this paper to understand the factors and combinations of factors that smes need to be considered before embracing e-commerce into their business by providing a closer look at actual experience of indonesian smes. case study research was carried out to analyse and explain the underlying factors that are likely to determine the varying extent of e-commerce adoption in smes, especially the service sector. the findings of the case studies have further extended into the development of proposed practical framework to illustrate how e-commerce adoption should be carried out from a strategic perspective. introduction small and medium-sized enterprises (smes) have grown in importance in the global economy during the last couple of decades. both theoretical and practical economic and business development literature acknowledges the key contributions of smes to the development of both national and international growth of economy. this fact is not only measured by the number of smes which represents nearly 90% of the total establishments across the world, but also their significant role in creating employment opportunities (hall 2002). information communication and technology (ict), on the other hand, is found to play an important role for any organisation (kaiuwe brock 2000). the use of ict that range from mainframe to personal computers, from word processing to sophisticated application and systems have made considerable inroads into large, 69 australasian journal of information systems volume 14 number 1 nov 2006 medium and even small organisations (doukidis et al. 1996). recent research also found positive signs that smes can take advantage of electronic commerce (e-commerce), as a type of ict, in helping their business to expand (macgregor et al. 2002). a number of e-commerce adoption methodologies and types of e-commerce applications have been suggested through literature both the academic press and trade journals and are being undertaken by many organisations. most of this research, has however, focused on developed countries where the structure of the economy is common. there are only few studies that shed some light on prescribing strategies of e-commerce adoption for smes in developing countries, especially indonesia. therefore, this research was aimed at addressing this gap in our knowledge – to understand the factors and combinations of factors that smes need to be considered before embracing e-commerce into their business. case study research was carried out to analyse and explain the underlying factors that are likely to determine the varying extent of e-commerce adoption in smes, particularly the service sector. the findings of the case studies have further extended into the development of proposed practical framework to illustrate how e-commerce adoption should be carried out from a strategic perspective. literature background characteristics of ict in smes the adverse impact of the economic slowdown in south east asia region, especially indonesia, and the globalisation of the world economy underscore the need for smes as the backbone of the national economy to become more resilient and competitive. thorp (1998) believes that successful implementation of ict offers the prospect of substantial competitive advantage for large and smes alike. however, smes are not ‘little’ large business (culkin and smith 2000, dandridge 1979). in terms of e-commerce implementation, it requires one to give thorough attention and proper understanding towards smes characteristics before delineating further issues of technology adoption and strategies. the renowned characteristic of smes that are common in most countries is the lack of in-house ict expertise and financial resources. furthermore, in relation to ict usage, previous research conducted by heikkila et al. (1991) found that there are three major differences between smes and large organisations: • smes tend to use computers more as tools and less as a communication medium; • the small number of stakeholders involved in an sme means that there are likely to be fewer problems in terms of organisational politics; • smes have considerably fewer resources available to implement ict solutions. much of the research also shows that the decision for ict adoption in smes are more likely made by the owner (doukidis et al. 1996, matlay and addis 2003), which often have little concern towards the importance of ict strategy and planning within their business (beckett 2003). furthermore, yap et al. (1992) found in their research that the management involvement is crucial to it success within smes. 70 australasian journal of information systems volume 14 number 1 nov 2006 research on smes and e-commerce e-commerce can be defined as any economic or business activity through web storefronts to enable the buying and selling of products and services and to facilitate the transaction of business and activities between and amongst individuals and organisations (schneider 2002). in a wider definition, however, e-commerce is not limited to buying and selling products online. along with customers, an online business will also find its suppliers, accountants, payment services, government agencies, and competitors online. these online partners demand changes in the way they do business from production to consumption (alter 2002, bidgoli 2002). succinctly, ecommerce is a new way of doing business (seyal and rahman, 2003; schneider, 2002; rahman and raisinghani, 2000). numerous studies into the use of e-commerce in smes recently have been of an exploratory and qualitative nature (drew 2002). while there is rapid growth and development of dot-coms and the ‘new economy’, smes in developing countries have been slower to adopt e-commerce than their developed countries (intrapairot and srivihok 2003). moreover, there are several issues that have been raised in recent sme literatures, which include: • exploring the advantage and disadvantage of e-commerce in thai smes (intrapairot and srivihok, 2003) • the importance of e-commerce infrastructure in facilitating the e-commerce adoption initiatives for smes in developing countries (jennex et al. 2004, stylianou et al. 2003). • the use of e-commerce by manufacturing sector of italian smes (santarelli and d'altri, 2003). • adoption attributes that may affect e-commerce use in brunei smes (seyal and rahman 2003) • factors influencing e-commerce adoption decisions in small and medium enterprises (smes) in thailand (lertwongsatien and wongpinunwatana 2003) • the use of e-commerce in australian smes (macgregor et al. 1998). in addition, the business press and trade journals have also given coverage to many of the issues faced by smes in moving into e-commerce (matlay and addis 2003, phillips 1998, schlenker and crocker 2003). however, there are some issues that have received very little attention in research to date, such as: • framework that helps smes evaluate the strategic use of e-commerce as well as guiding the adoption process. • assessment of e-commerce application and post adoption experiences in different sectors of indonesian smes. impact and benefit of strategic use of e-commerce in sme one might question why the presence of e-commerce has expanded very rapidly in recent years. the answer is simply because of the opportunities and benefits that are evident from the current implementation by many organisations. the concept of strategic use e-commerce through the utilisation of internet software and services also endeavours to link internet use with opportunities it offers to the firm (sadowski et al. 2002). it further recognises that the strategic potential internet use 71 australasian journal of information systems volume 14 number 1 nov 2006 allows exercising the control over the bounds of relationships and interfaces used in relationships between customers and the business, which are also related to the firm’s strategies. e-commerce has offered a variety of potential benefits both to smes and large business. numerous studies claim their findings on the benefit and impact of e-commerce. those findings are summarised into the following points (chan 2001, schneider 2002): • e-commerce offers unmatched savings in terms of transaction costs • the reduction of cost in advertising and promotion • speed communication between buyer and seller • companies can shorten their traditional supply chains, minimise transport obstacles, and reduce delivery costs. • physical limitations of time and space are removed. despite the attractive benefits that smes may obtain from adopting e-commerce into their business, smes possess significant problems in identifying the appropriate application of e-commerce and its strategy due to the lack of knowledge and planning in ict. consequently, it is hardly surprising to witness that most of the smes’ owner/managers will finally develop their ict strategy through a ‘trial-and-error’ method. e-commerce and smes in indonesia during the last twenty years, there has been considerable growth in terms of the number of smes throughout asian economies, with indonesia in particular. indonesian smes have proven to be the most dynamic and vibrant sector, especially during the time of the financial crisis in 1997 (urata, 2000). for the purpose of this research, the official definitions from two indonesian government bodies the central bureau of statistics (cbs, 2003) and the ministry of cooperatives and small medium enterprises (depkop, 2000) will be adopted. the definition is described as follows (see table 1): table 1. definition of small and medium enterprises in indonesia the significance of smes to the indonesian economy is further highlighted by their contribution to national development and by the fact that, as a sector, they provide and create jobs as well as facilitate the elimination of poverty (depkop 2005). furthermore, indonesia is slowly joining the information superhighway. with the huge and widely spread population, the development of internet technology and infrastructure will not only become a 72 australasian journal of information systems volume 14 number 1 nov 2006 potential market of e-commerce business but also a means to bridge hundreds of islands spread around the indonesian archipelago. therefore, a national campaign has been carried out by the government through seminars and other media in order to convince the benefit of e-commerce and to increase the use of e-commerce by businesses in indonesia (depkop 2005). methodology as mentioned earlier, our research objective is to understand the factors and combinations of factors that smes need to be considered before embracing e-commerce into their business in indonesia. given the little empirical knowledge on the e-commerce management practices in indonesian organizations, and the exploratory nature of this study, the qualitative case-study approach was considered appropriate for this research. one of the primary limitations of case research is the limited generalisability of the findings (eisenhardt 1989). working within this constraint, in order to achieve a fairly generalisable set of results and useful information, this study is analysed from interviews with two smes owners/managers. the two smes were selected on the basis of their business size as well as their unique post ecommerce adoption experience. all of the participants were contacted and asked for their willingness to participate in the study. it is important to note that all participants are involved in this research on a voluntary basis. furthermore, this study did not include any multinational or transnational organisations, as our intent was to study ‘local’ business firms operating in typical developing country settings. due to the non-disclosure agreements with these organisations, the smes involved in this study will be referred to as company a/b. all interviews were recorded and the entire conversations are based on open and semi structured interviews (mahemba and de brujin 2003). data collection and analysis the two hours of semi-structured interviews were used to gather the details of experience from each participant. this method has the advantage of obtaining unambiguous responses to certain issues, although at the risk of influencing responses by attitudes and the structuring of questions. these weaknesses can be addressed by control and evaluation questions. for the analysis of interviews purposes, each interview was recorded and later transcribed verbatim. this ensured that the notes compiled during the interview were an accurate reflection of the information being provided by the interviewees. moreover, in order to add to the usefulness of information collected, multiple sources of information were used. this data triangulation method involved using multiple data sources in the interview process. apart from conducting interviews, the researcher cited relevant documents and was often invited to observe company premises. a multiple research approach is cited as one method of ensuring the reliability and validity of qualitative research (creswell 2003). case studies company a: educational service company a is located in west java, indonesia. the primary business of this firm is to provide students from any junior and senior high school in town with educational coaching services. 73 australasian journal of information systems volume 14 number 1 nov 2006 the e-commerce adoption was finally initiated by the manager so as to give the students a more convenient way to retrieve results, download any course material of their study or even post the questions online simply from the nearest internet café or even at their own convenience at home. a subsidiary goal is to find potential students that are interested in learning by distance so it will save some cost on employing more tutors. considering that the performance of the web may affect the convenience of the customer, the manager was decided to invest in technology by hiring professional web developer to design and develop their website. the problem has finally emerged that only a few students are utilising the website facility, while no students have enrolled in distance learning classes. after the survey evaluation and interview with a few students completed by the organisation, the owner realised that most of the students believed there is not much difference in terms of cost between logging into the website from the nearest internet café and obtaining the result personally from the office. indeed, by going to the office personally, the students would have some chance to ask and confirm their answer directly to the tutors. company b: transportation service established in 1979, company b is a privately owned inner city bus operator with bus rental, which is located in central java, indonesia. it was the owner’s initiation to embrace e-commerce into their business, which was actually recommended by one of his colleagues that had just finished attending an e-commerce seminar, organised by the government agencies. his colleague believes that e-commerce will help the business by cost reduction, increased sales and improved productivity. there is only one staff member with the appropriate skills in using a pc and email applications. however, there are no staff with skills in designing and developing the website, so the only available solution for the owner is to hire a programmer to build the website for the company. in 2002, the company has successfully set up the website that allows the potential customers to find out the available services, types of vehicle available for hire, pricing information, bus route and details of the office location. within just the first few months of e-commerce adoption, the company has started facing several challenges in managing their business online. the owner said that the company had received a number of complaints from customers who felt unsatisfied with the response time in replying to their message through email. in addition, the majority of customers appear to prefer traditional ways of gathering information, ie. through phone and fax machine. discussion there are several points which have emerged from the analysis of the two companies involved in the case studies above, which will now be discussed. 74 australasian journal of information systems volume 14 number 1 nov 2006 the critical value of business strategy numerous literature acknowledges the fact that the lack – or absence in some casesof business strategy in smes may hamper their ability to thrive (ballantine et al. 1998, morison et al. 2003, shaleh 1986). this claim had been supported by the two cases previously described above. it is obvious that both companies portrayed are sharing some common interests, ie. to look for ways of taking advantage and getting as much benefit as possible from their investment in e-commerce. however, what has been missing from the above adoption processes is the initial process of identifying their business strategy prior to the e-commerce adoption decision. thorough identification of the business strategy facilitates the company’s knowledge of the business objectives, strength of the business, available resources and business environment that in a later stage can also be used for defining what the key business processes are. furthermore, it is important to emphasise that e-commerce is not entirely about technology. instead, it is more appropriate to perceive e-commerce as a way for organisations and industry to gain value from and transform key business processes with the internet and related technologies. the failure in e-commerce adoption can also be triggered if the owner/manager believes that the existing business process issues can be resolved simply by converting them into electronic form. in fact, applying technology without careful planning and justification frequently results in poorer performance. the proposed practical framework in order to assist smes formulate an effective e-commerce adoption decision, this paper has extended the analysis of the qualitative data into the development of a practical framework as shown in figure 1. a full description of steps involved will be presented as follows. figure 1. proposed e-commerce adoption framework for smes 75 australasian journal of information systems volume 14 number 1 nov 2006 step 1 initial assessment of e-commerce adoption along with identifying key business process and business strategy prior to e-commerce adoption, it is also important for the business to gather as much as information on e-commerce readiness of the external entity – such as the customer – that will also have an impact and influence on the ecommerce success in the business. this is due to the fact that most smes in the service sector considered e-commerce not only improves internal productivity and cost saving, but also engages the customer by increasing the customer's comfort. in addition, mitra and chaya (1996) propose that there is a need to quantify the benefits from the investments in building quality e-commerce and also to require solid evidence of value-add to customers. this is due to the fact that the added value for customers will result in additional profit for the organisation, as they are able to maintain current customer relationships and develop new ones based on the attractive offerings a new e-commerce can afford. after the company had quantified the benefits of adopting e-commerce both from business and customers’ perspectives, the next factor to identify is the alignment of the business strategy to the technology. drew (2002) found in his study that the establishment of e-commerce technology as a competitive advantage will depend upon how well the sme can integrate the process into the total business strategy. step 2 – the website development loop if through the initial assessment of e-commerce adoption process it is found that e-commerce will benefit the business, the next step to be carried out is defining the website development plan. during this step, the business should identify the functionality of the website such as by defining to what extent the website will serve. is it only for marketing purposes or is it developed to replace the service that a bricks and mortar shop can provide? an example boundary of website development plan is illustrated in figure 2. the figure provides an example of how the development process can be divided into three stages. in this case, during initial implementation, the company implemented three additional static pages to the main page of the website. as the business expands, the business strategy evolved and further enhancement of the website may be required. this enhancement can be fulfilled by adding additional features such as online ordering forms, online payment facilities and so on. 76 australasian journal of information systems volume 14 number 1 nov 2006 figure 2. sample of website development plan the next step to implement in the website development loop is designing the content of the website. in this step the company may hire a graphics designer and/or web programmer, and work closely to determine what content may be appropriate for the business. if the website is only dedicated to customers within the country, then the use of local language would be appropriate to minimise the confusion. finally, the entire document collected from this step – such as the website layout and contents – can then be used to implement the website. in addition, it is also important for the company that utilises the website as a marketing tool to consider registering the website on several search engines or promote it through other media. although the steps in the framework depicted in figure 1 seem like a sequential process, it is possible to have iteration between steps. by the maturity of the business, the boundaries of the website can be adjusted to fulfil the business needs. the needs for external support another point to be emphasised is the importance of continuing support from the government or any non-government bodies that are concerned with the development of smes. this implies support bodies need to provide more comprehensive assistance, which is to guide smes from beginning to end in the adoption process. this assistance is critical due to the fact that most of smes lack inhouse expertise, which means some assistance is needed in identifying issues prior to adoption decisions – such as defining business strategies as stated above. as much as important of convincing smes the benefits of e-commerce, helping them in the adoption process may also become one of the solutions in eliminating chances of failure or inefficient use of e-commerce. 77 australasian journal of information systems volume 14 number 1 nov 2006 finally, in order to be effective in helping smes, it is also important for the government to ensure that the information on what kinds of services are provided widely to all smes. it is relevant to note that information provided by asean (association of south east asian nations) (asean (association of south east asian nations) 1997) shows that in indonesia less than 20 per cent of the smes entrepreneurs have received assistance from the governments. limitations and future research directions the research context of this study limits the conclusion that can be drawn. one of the key weaknesses in this study lies in the difficulty of generalising the proposed framework to the adoption of b2b types of e-commerce. this is due to the restricted nature of the study context – that is the adoption of b2c website by small service company – which limits the generalisability and applicability of the findings to other study contexts. further study is therefore needed to assess the applicability as well as the significance of the proposed determinants by incorporating the impact of culture, management styles and other variables that may also impact on adoption processes. it should also consider the views of successful adopters to help develop a more holistic understanding of factors in thriving e-commerce adoption. conclusions electronic commerce (e-commerce) adoption case studies in two indonesian smes have been presented in this paper. it shows an analysis of the importance of aligning business strategy with ecommerce adoption in smes. the study also underlines several factors and a step-by-step process of adopting e-commerce through a proposed framework. the results of this study are useful not only for managers of smes but also for government bodies, in developing countries such as indonesia, that have economic reasons to be concerned about the development of smes. the author believes that the results will also provide some insights to ict consultants – with sme customers – in order to improve their service and customer satisfaction. this paper advances the field of qualitative research in it by providing an actual experience of how the use of case study can be applied to identify important factors that smes need to consider before adopting e-commerce. the data gathered from the case study had also been used to build a framework for e-commerce adoption. it is hoped that the experiences we had in employing the methodology will also serve as basis information for any researcher interested in conducting case study in developing country, particularly indonesia. although the research context is quite specific, it is believed that the findings are of relevance to other industries and other countries. references 78 australasian journal of information systems volume 14 number 1 nov 2006 alter, s. (2002) information systems : foundation of e-business. prentice hall. upper saddle river, nj ; [great britain]. asean (association of south east asian nations) (1997) small and medium enterprises in indonesia. association of south east asian nations. available from: http://aeup.brel.com/sme/sme1.html. (accessed: 16 march 2000) ballantine, j., levy, m. and powell, p. (1998) 'evaluating information systems in small and medium-sized enterprises: issues and evidence'. european journal of information systems. vol. 7 no. 4. pp.241-251. beckett, h. (2003) 'half of smes have no it strategy'. computer weekly. vol. pp.34. bidgoli, h. (2002) electronic commerce : principles and practice. academic. san diego, calif. ; london. chan, h. (2001) e-commerce : fundamentals and applications. wiley. new york ; chichester. creswell, j. w. (2003) research design : qualitative, quantitative, and mixed methods approaches. sage publications. thousand oaks, calif. culkin, n. and smith, d. (2000) 'an emotional business: a guide to understanding the innovations of small business decision takers'. qualitative market research: an international journal. vol. 3 no. 3. pp.145-157. dandridge, t. c. (1979) 'children are not little grown ups: small business needs its own organizational theory'. journal of small business management. vol. pp.53-57. depkop (2005) renstra 2004-2009. ministry of cooperatives, small and medium enterprises republic of indonesia. available from: www.depkop.go.id. (accessed: november 2005) doukidis, g. i., lybereas, p. and galliers, r. d. (1996) 'information systems planning in small business: a stages of growth analysis'. the journal of systems and software. vol. 33 no. 2. pp.189. drew, s. a. w. (2002) 'e-business research practice: towards an agenda'. electronic journal on business research methods. vol. 1 no. 1. pp.18-25. eisenhardt, k. m. (1989) 'building theories from case study research'. academy of management. the academy of management review. vol. 14 no. 4. pp.532. hall, c. (2002) profile of smes and sme issues 1990-2000. asia-pacific economic cooperation. singapore. heikkila, j., saarinen, t. and saaksjarvi, m. (1991) 'success of software packages in small business: an exploratory study'. european journal of information systems. vol. 1 no. 3. pp.159-169. intrapairot, a. and srivihok, a. (2003) 'the e-commerce of smes in thailand'. in e-commerce and cultural values. (ed, thanasankit, t.). idea group publishing. singapore. pp.199. jennex, m. e., amoroso, d. and adelakun, o. (2004) 'e-commerce infrastructure success factors for small companies in developing economies'. electronic commerce research. vol. 4 no. 3. pp.263-286. kai-uwe brock, j. (2000) 'information and communication technologyin the small firm'. in enterprise and small business: principles, practices and policy. (eds, carter, s. and jonesevans, d.). prentice hall. singapore. pp.385-408. 79 http://aeup.brel.com/sme/sme1.html http://www.depkop.go.id/ australasian journal of information systems volume 14 number 1 nov 2006 lertwongsatien, c. and wongpinunwatana, n. (2003) 'e-commerce adoption in thailand: an empirical study of small and medium enterprises (smes)'. journal of global information technology management. vol. 6 no. 3. pp.67. macgregor, r. c., bunker, d. and waugh, p. (1998) electronic commerce and small/medium enterprises in australia: an edi pilot study, in proceedings of proceedings of 11th international bled e-commerce conference. 8th-10th june. bled slovania. pp.331-345. macgregor, r. c., vrazalic, l., carlsson, s., bunker, d. and magnusson, m. (2002) 'the impact of business size and business type on small business investment in electronic commerce: a study of swedish small businesses'. australian journal of information systems. vol. 9 no. 2. pp.31-39. mahemba, c. and de brujin, e. (2003) innovation activities by small and medium-sized manufacturing enterprises in tanzania. in creativity and innovation management. vol. 12. pp.162. matlay, h. and addis, m. (2003) 'adoption of ict and e-commerce in small businesses: an heibased consultancy perspective'. journal of small business and enterprise development. vol. 10 no. 3. pp.321-335. mitra, s. and chaya, a. (1996) analyzing cost-effectiveness of organizations: the impact of information technology spending. in journal of management information systems. vol. 13. pp.29-57. morison, a., breen, j. and ali, s. (2003) 'small business growth: intention, ability, and opportunity'. journal of small business management. vol. 41 no. 4. pp.417-425. phillips, m. (1998) successful e-commerce : 10 case studies to show small business how to profit form online commerce. bookman press. melbourne. sadowski, b. m., maitland, c. and van dongen, j. (2002) 'strategic use of the internet by small and medium-sized companies: an exploratory study'. information economics and policy. vol. 14 no. 1. pp.75-93. schlenker, l. and crocker, n. (2003) 'building an e-business scenario for small business: the ibm sme gateway project'. qualitative market research. vol. 6 no. 1. pp.7. schneider, g. p. (2002) electronic commerce. course technology. australia. seyal, a. h. and rahman, m. n. a. (2003) 'a preliminary investigation of e-commerce adoption in small & medium enterprises in brunei'. journal of global information technology management. vol. 6 no. 2. pp.6. shaleh, i. a. (1986) industri kecil: sebuah tinjauan dan perbandingan. lp3es. jakarta. stylianou, a. c., robbins, s. s. and jackson, p. (2003) 'perceptions and attitudes about ecommerce development in china: an exploratory study'. journal of global information management. vol. 11 no. 2. pp.31. thorp, j. (1998) the information paradox : realizing the business benefits of information technology. mcgraw-hill ryerson. toronto. yap, c. s., soh, c. p. p. and raman, k. s. (1992) 'international systems success factors in small business'. international journal of management science. vol. 5 no. 6. pp.597-609. 80 characteristics of ict in smes impact and benefit of strategic use of e-commerce in sme e-commerce and smes in indonesia data collection and analysis company a: educational service company b: transportation service the critical value of business strategy the proposed practical framework the needs for external support microsoft word ajis112final.doc australasian journal of information systems vol 11 no 2 may 2004 57 factors influencing information systems success in papua new guinea organisations: a case analysis. mr. limbie kelegai school of information systems faculty of information technology, queensland university of technology 2 george street, brisbane, queensland australia phone: +617 3864 1956 fax: +617 3964 1969 email: l.kelegai@student.qut.edu.au mr. michael middleton school of information systems faculty of information technology, queensland university of technology 2 george street, brisbane, queensland australia phone: +617 3864 2870 fax: +617 3964 1969 email: m.middleton@qut.edu.au abstract information systems (is) applications have proliferated in least developed countries (ldcs) as organisations in these countries begin to realise the benefits of utilising these technologies to improve business process and enhance productivity. however much is implementation in ldcs has been unsuccessful. successful approaches should take account of technological factors, and also heed social context including economic, political, cultural and behavioural influences. as a component of a wider study to identify factors that influence is success in developing countries, this paper examines factors that have affected is implementation in two papua new guinea (png) organisations. introduction we have been investigating the factors influencing information systems (is) success in organisations in png, including information technology (it) preparedness. as part of that process, we have conducted a number of case studies. here we discuss organisational, individual, and external (or contingency) issues that influence is success in two png organisations. initial indications are that these are representative of the wider range of organisations being studied. is implementation and success in organisations has received much attention over the years as it has been utilised to enhance productivity and meet organisational goals, particularly in developed countries (dcs). recently is implementation has increased in ldcs as organisations in these countries realise the benefits of utilising it as development tools (harris & davison, 1999). despite increased implementation, the level of is success may be far lower in ldcs. many studies on is implementation success have been undertaken, and a number of these are referenced in the following section on success factor identification. few have concentrated on ldcs along the lines of anakwe, anandarajan & igbaria (1999). there is growing concern that implementation and utilisation of is is being undertaken with insufficient consideration of economic, social and political difficulties. identifying the determinants for is success within these differing settings is important in ldcs. therefore this study takes this into account in investigation of is implementation. investigation is undertaken at individual and organisational levels. factors external to the enterprises are also considered. for example, the availability of skilled is and it professionals in the country and its effect on organisational is implementation and sustainability. australasian journal of information systems vol 11 no 2 may 2004 58 is success measurement identification of success factors various approaches have been taken over the last two decades to study and classify is research. for example, lyytinen (1987) classified the approaches as: (a) is architecture, (b) information need, (c) socio-technical, (d) evaluation, and (e) success factors. the success factors approach attempts to model user and organisational features to predict is implementation outcomes. it is argued that if practitioners are aware of the ‘factors and address them during implementation, the system is more likely to succeed’, (larsen & meyers 1999, p.397). furthermore, grover, jeong & segars (1996) suggest that research devoted to the measurement of is effectiveness may be categorised into four distinctive research streams. these streams are: (a) criteria demonstration: theoretical development and justification of criteria, (b) measurement: evaluation and statistical development of effectiveness measures, (c) criteria relationship: linking effectiveness criteria, and (d) antecedent of is effectiveness: theoretical and statistical determination of antecedents to is success. the study on antecedents of is effectiveness mainly focuses on determinants of is effectiveness rather than the criteria themselves. over the years much attention has been paid to success factor approaches (for example by delone & mclean 1992, jain 1997, myers, kappelman, & prybutok 1998). these approaches have been criticised for a number of reasons, such as viewing implementation as a static phenomenon and ignoring factors that can have varying levels of importance at various levels of implementation, inconsistency in defining success factors, and being based on an underlying mechanistic view of is. the study and measure of effectiveness is difficult because of the multidimensional constructs and being subject to varying perceptions of stakeholders and evaluators. furthermore these factors may be differentiated between dcs and ldcs as a consequence of their social/organisational and cultural settings. delone & mclean (1992) suggest 3 levels of analysis: process, organisational and individual. their investigation surveyed 180 articles and proposed that the existing measures be classified in to six dimensions: (a) system quality, (b) information quality, (c) information use, (d) user satisfaction, (e) individual impact and (f) organisational impact. however it seems that this study identified only system issues and not the human issues. other approaches to categorisation have included that of ein-dor, segev, & orgad, (1993) who in their review of international and cross-cultural studies classified contributing factors as cultural, environmental, structural and procedural; and odedrastraub (1993) who identified various factors contributing to failures (and success) by reviewing a number of case studies and classifying the levels as national and organisational; and markus & robey (1988), who use the levels: individual, organisational and society. the study by bailey & pearson (1986) has received much attention and been adapted by many academics and practitioners, because of the reliability, content, and construct validity. it identified 39 factors that influence user is satisfaction and proposed an instrument to measure them. miller & doyle (1987) were among those to apply the bailey & pearson’s work. they attempted to measure is effectiveness by surveying managers (i.e. at an individual level) of 21 financial institutions in south africa, by adapting 24 of these factors. although bailey & pearson provided a solid foundation for evaluating individual response, grover, jeong & segars (1996) argue that a greater scope of is effectiveness is necessary. is success studies ang, davis & finlay (2001) in a recent study investigated the impact of it on tqm in public organisations in malaysia by measuring the extent of it use at a macro level. the objective of the study was to determine if public sector organisations with differing levels of it usage have different factors that facilitate or inhibit it use. they categorised success factors into organisational, technological and external dimensions. the results indicated that organisational and technological factors had a strong influence on it utilisation. in a similar environment jain (1997) conducted a case study on is implementation in five south-east asian countries and attempted to identify organisational factors that contribute to is success in the context of ldcs. australasian journal of information systems vol 11 no 2 may 2004 59 hence it seems that is research has been non-cumulative and fragmented lacking an overarching framework regarding the context in which effectiveness criteria are applied. this study groups the influencing factors into three levels: individual, organisational and external and attempts to investigate these factors in organisations in png. methodology employed in study design this research adopted the study design as suggested by miles & huberman (1994), in which two data collection approaches are alternated, beginning with exploratory qualitative fieldwork, leading to the development of quantitative instruments such as questionnaires to deepen the qualitative data and strengthen theory. stage 2 of the study, which involves a questionnaire survey is currently in progress. a purposive framework of sampling was applied for this study (neuman 2000; yin 1994). sample requests in the form of letters were mailed to 50 organisations. they were selected from prior contacts and listings in the png telephone directory. of the 50 requests, 40% (20) replied and agreed to participate. no prior studies have been conducted in this domain, and with limited is literature, little is known about is implementation and success in organisations in png. ten organisations were selected following the purposive framework as suggested and similar considerations to jain (1997). factors such as limitation of funding, location and difficulty accessing the sites, maturity of the is and it infrastructure in the organisations, law and order problems, and organisation size were considered during the process of selecting the cases. method case study reliability was sought using several strategies including case study protocol (yin 1994, jain 1997). it was reviewed and verified for face and content validity before being implemented. the protocol was further reviewed after the pilot study. a brief form of the procedures and protocol were mailed to the participating organisations together with the letter of introduction before the resumption of the study. interviews were conducted on site. the participants were grouped into 3 categorises, managers (top and middle), users, and is staff. all interviews were transcribed verbatim and database created for all data collected during the field study. using nvivo, a qualitative analysis package, qualitative data was coded and recoded into elementary parts, and than regrouped into categories such as those appearing in table 2, 3, and 4. cultural interpretations used in interviews are described under ‘cultural features’ in the context section following. context png geography png has a population density of 8 persons per square kilometre, and annual population growth rate is estimated at 2.4%. png has more languages (over 800) than any other country in the world, with the possible exception of india. the official language is english with pidgin and hiri motu being used widely as lingua franca to facilitate communication among people of diverse linguistic backgrounds. its parliamentary democracy elects a single legislature, the national parliament. png has distinctive cultural norms. like other ldcs, png’s unstable political environment is one of the major hindrances to economic growth. of its 4.5 million people, 85% live in rural areas. for organisations to function effectively, an understanding, scanning and analysis of the environmental forces surrounding the firm is essential (kavanamur 1996). literature suggests that australasian journal of information systems vol 11 no 2 may 2004 60 the environment in ldcs is volatile, and particularly with it constantly changing. table 1 presents a summary of the png environment. it industry in png there appears to be no publicly documented evidence of the early implementation of is or it in png organisations. however organisations in the country were beginning to realise the benefits of using it as a tool and increasingly employing is at various levels. the it industry is small but slowly developing, therefore relies on technology imported from foreign countries such as australia. given the opportunity pngeans are able to adopt these technologies. the it industry is driven by the industry itself, because as is argued by some respondents, there was a lack of is or it planning and direction at the national level. other limitations include the lack of a national information infrastructure. cultural features this study pays particular attention to the cultural factors that may have an influence on is success. we have adapted hofstede’s (1983, p.23) approach with its definition of national culture as a ‘collective programming of the mind which distinguishes the members of one group from those of another’. in this context, culture deals with a collective way of thinking by a group of people. hall (1977) classified culture in two contexts, high context and low context. high context societies are characterised as collectivist, while low context societies are individualist. however a nation or country may not be classified has having a single culture (hofstede, 1997; myers & tan, 2002), thus acknowledging the presence of subcultures and individual differences within nations and regions. for example, in png there are more the 800 languages and different regions with varying cultural beliefs and values. hofstede and hall used classifications as follows: • individualism/collectivism: individualist societies have loosely knit social structure where individualist approaches are encouraged. collectivist societies prefer tightly knit social framework where in-group ideals take precedent. • large/small power distance(pd): this measures the extent to which the members of a society accept that power is distributed unequally. lpd has a hierarchical order in which everyone has a place with no justification. spd societies strive for power equalisation. • strong/weak uncertainty avoidance(ua): this measures the degree to which members of societies feel uncomfortable with uncertainty and ambiguity. strong ua societies are less tolerant to deviant ideas and maintain rigid codes. weak ua societies are more tolerant to deviance and are open to new ideas. • masculinity/femininity: masculine society prefers achievements, heroism assertiveness, and materialism and is less caring with an individualist approach. femininity emphasises relationship, modesty and quality of life. each society arrives at its answers as a consequence of these fundamental issues, which represent the common element of structure in the cultural system of the country. png scores closer to a collectivist culture with an egalitarian social structure. australasian journal of information systems vol 11 no 2 may 2004 61 png developed countries a. economic factors labour(is professionals) agricultural/industrial capital gnp inflation infrastructure national information infrastructure technology flow availability b. political factors instability institutions c. cultural factors social structure uncertainty avoidance individual/collective power distance masculinity-femininity abstractive-associative thinking d. work culture creative potential time perspective time units of action success orientation environment people orientation scarce agricultural <us$744 high weak recipient scarce high weak egalitarian high collectivism high femininity low abstractive; high associative limited past and presented short term moralism context dependent paternalistic abundant industrial >us$15,000 low strong donor abundant low strong less rigid low individualist low masculinity high abstractive; low associative unlimited future-oriented long term pragmatism context independent participative table 1. summary of environment factors in png (derived from interpretation by kavanamur (2002) of factors identified by hofstede (1997) case studies brief description of the organisations organisation a organisation a is a semi government organisation established by an act of parliament as a commercial statutory authority (csa) .the organisation is autonomous, and operates on a commercial basis as stipulated under the act. organisation b organisation b is a private organisation involved in the vehicle and spare parts industry. it is a division of a group of companies involved in a wide range of business activities throughout png with offices internationally. ict in the organisations organisation a has had wide experience in is development, and implementation. there is also a high level of it knowledge and literacy, particularly at the upper management level. in addition, top australasian journal of information systems vol 11 no 2 may 2004 62 management is supportive of is as a strategic tool to realise the organisation’s objectives and achieve operational goals. organisation a recently implemented a management information system. the system integrates finance, administrative and billing system. at the end user computing level, a range of desktop packages is utilised. these packages range from office automation systems such as ms office professional to engineering packages such as autocad. prior to the current management system, organisation a implemented a system administered by a is service organisation. the system was primarily used for invoicing and collection of rates in the operational centres. their mainframe was located in port moresby and operational centres connected via dedicated lines with dumb terminals on each site. organisation a migrated to a pc based environment with the assistance of expatriate consultant. a png national officer was recruited to understudy the consultant. the current system was developed overseas but customised to organisation a’s requirements. data from remote sites were transferred electronically via email and processed in port moresby. printed invoices were than posted to the respective districts. organisation b implemented two major computer information systems, a stock control system and the vehicle system supported, which were maintained by an organisation based outside the country. organisation b is a division of a private organisation, therefore is investments were assessed in terms of achieving efficiency in the organisation process with the primary objective to maximise its profit margin. table 2, 3, and 4 presents a comparison of is success factors as interpreted in the organisations. they are accompanied by brief summaries of some of the identified factors is the factor important for is success in the organisation? is the factor achieved in the organisation? organisat ion a organisation b organisation a organisation b identified individual factors mgr (3) mgr (3) mgr (3) mgr (3) management is knowledge yes yes yes no management support yes yes yes mix funding yes yes no mix is division service & support yes yes mix yes is staff skill and competency yes yes mix mix user training yes yes no yes information availability yes yes mix yes timeliness of report yes yes mix yes system availability yes yes yes yes system flexibility yes yes yes yes system integrity yes yes yes yes table 2: comparison of is success factors, individual level australasian journal of information systems vol 11 no 2 may 2004 63 management is knowledge and support. the level of knowledge and literacy, and management understanding of is contributes to the management’s perception of is, its responsibilities, usefulness and strategic value to the organisation. it may influence management’s willingness to support is. top management in organisation a perceive is as a useful and strategic tool, and an integral part of the organisation. respondents in organisation b felt that, although top management perceived is to be useful, they were apprehensive in pushing is in the organisation, and required convincing before any actions were taken. one of the managers at organisation b stated that they lacked ‘support and push from the top’. the lack of is knowledge and awareness may be a contributing factor to the lack of enthusiasm by top management. therefore is knowledge, literacy and understanding were important ingredients for top management to be better informed in order for them to support and source funds. is the factor important for is success in the organisation? is the factor implemented in the organisation? organisatio n a organisation b organisation a organisation b identified organisational factors mg r (3) ceo (1) mgr (3) ceo mgr (3) ceo (1) mgr (3) ceo management is knowledge yes yes yes yes yes no management support yes yes yes yes yes mix funding yes yes yes no no mix prioritising is yes yes yes yes yes yes strategic tool yes yes yes mix yes yes is policy organisational yes yes yes no no no is planning yes yes yes no mix no ceo/cio proximity no no no no no no table 3: comparison of is success factors, organisational level funding funding is considered an important is success factor. in organisation b, funding was available if management were in agreement that the is project would yield more benefits for the organisation. with the increase in is experience and newer technology, management were willing to invest in is provided it increased benefits. organisation a was able to internally fund some projects, unlike the other government organisations that totally rely on government funding. in addition, organisation a was a recipient of donor assisted funding for its major projects including is investment. however, there were concerns about the long-term sustainability and continuity of the donor funded projects. furthermore, projects funded by international donor agencies may be helpful for good governance, however these projects should be developed within a framework and systematic policy commitment by the government. australasian journal of information systems vol 11 no 2 may 2004 64 prioritising is organisations a and b regarded is as a core business activity and a critical success factor. for instance, top management in organisation a said that is was ‘critical and a top priority’. therefore there is a high degree of is support from the top management at organisation a. similarly is was considered a priority for organisation b. is policy & planning strategic issues such as is policy and planning are significant guidelines for the organisations’ is development. however there was no is policy and long term planning evident for either organisations. one manager in organisation b stated that is was ‘mainly driven by need to automate because i do not think we have an is policy, and, if there is a need and new things in the market we try to get them’. similarly, it was stated that there was ‘no it policy’ in place in organisation a. it was also argued that long term is planning was lacking in the organisations. for example, when the management system in organisation a was upgraded to a new platform there were concerns and disagreement regarding the manner in which the new system was ‘developed and implemented’. there had been no plans for the migration, therefore the migration process disrupted the organisations functions. is division service and support the inability to provide effective is service and support by is division in the organisation may inhibit is success. managers in organisation b were content with the services provided by the is division, particularly with the network support and training. the organisation implemented off-theself packages purchased from service providers in the country and from vendors abroad, therefore there was no software development in organisation b. there were only 3 it support staff in organisation b. the information system in organisation a was specifically designed, developed and implemented by consultants. is staff were primarily involved with administration and ongoing maintenance of the system. respondents in organisation a indicated that support and service from the is division was inadequate. only 2 is staff were employed in organisation a. australasian journal of information systems vol 11 no 2 may 2004 65 is the factor important for is success in the organisation? is factor implications organisatio n a organisation b organisation a organisation b identified external factors mgr (3) ceo (1) mg r (3) ceo national is policy yes yes yes no clear policy no clear policy availability of is professionals yes yes yes major concern major concern consultants yes yes yes difficulty with consultants difficulty with consultants government planning and direction yes yes yes lack planning and direction lack of planning and direction telecommunication infrastructure yes yes yes inadequate infrastructure inadequate infrastructure national cultural issues yes yes yes concern concern political & economic conditions yes yes yes non conducive non conducive law & order yes yes yes concern concern is industry support yes yes yes lacking lacking table 4: comparison of is success factors, external level availability of is professionals both organisations were in agreement that skilled is professionals were scarce in png. one senior manager in organisation b said that it was ‘difficult to locate people with skills of specialised software packages’. similarly in organisation a it was stated that the country did not ‘necessarily have the expertise’, therefore it was difficult employing is professionals from within the country. the availability of skilled is staff is critical to the successful application of is in organisations in the country. foreign is consultants as a result of the lack of skilled is professionals in the country, organisations ‘engaged people from outside to assist in the is development’. it was argued that generally, is staff lack the knowledge, experience and confidence to engage in the development of large systems. foreign is consultants will continue to play a significant role in is development and implementation in the two organisations. thus organisations may have developed a culture of dependency on external consultants for the development and implementation of their information systems. a number of issues are highlighted: • is development projects were often incomplete. because of the limitations in funding, consultants are engaged for a short period and may not complete the project within the scheduled period. • access and support: consultation with consultants who operated from overseas was through the phone or email, however ‘communication through email was difficult’. australasian journal of information systems vol 11 no 2 may 2004 66 there was no face-to-face contact. from a cultural perspective, generally collectivist societies prefer face-to-face contacts and tend to dislike other forms of communication. • training and knowledge transfer: it was argued that consultants failed to train is staff and users. consultants were engaged to ‘fix’ the immediate is problem but did not place any emphasis on training and knowledge transfer. • reliance and high cost for consultants: consultant fees were a major cost area for both organisations. it was ‘ very expensive to bring people up to fix’ the problems. however, with the lack of skilled is professionals in the country, organisations have no choice but to rely on external consultants for assistance and direction. is training another identified is success factor was training. generally, the demand for is training was identified at a number of levels; is staff training, user training in the operations for a particular system, and a general it literacy program on the evolving technologies. is training was essential for both organisations. in organisation b, funds were committed for user training highlighting the organisations emphasis on is training. furthermore, is experience in the organisation was relatively new, therefore is knowledge and literacy programs may be required at all levels on a continuous basis. similarly, organisation a ‘emphasised is training’. however managers felt that ‘ training was not very good’. therefore is literacy programs were essential. in general one respondent stated that ‘we have the technology here but we have a problem of literacy and skills’, and implied that there was a lack of training plan for is staff and the users. finally, various types of training are required at all levels in both organisations ’ for managers, users and is staff’. information satisfaction both organisations emphasised information integrity and availability with a focus on information delivery. the quality and integrity of information delivered depends on a number of things including system integrity, and the quality of data input. the need for data input quality was emphasised by a senior manager in organisation a that ‘the important thing is the input’. furthermore, system integrity was essential to manipulate and deliver the necessary information for business operations and decision-making, and importantly reliable information is presented to the customers on time. managers require accurate information delivered on a timely basis government stability and planning environmental conditions in which organisations operate have an impact on their functions. in png, governments can change every 18 months as allowed by the constitution, thus projecting an atmosphere of political uncertainty. uncertainty increases costs, interferes with planning and often leads to bureaucratic bottleneck. due to the instability there are social upheavals, no basic government services reach the rural areas and increasing law and order problems. furthermore, most state owned enterprises are technically insolvent and is largely attributed to political intervention and wantokism. the ‘wantokism’ is a cultural notion in a collectivist society, were the kin or clan takes precedent over oneself. in addition, the high turnover rate of managers in the public sector may contribute to the lack of continuity in public policy formulation and execution (kavanamur, 2002). for instance the treasury and finance department has had six different secretaries in the last 5 years. particularly for a national is policy, no comprehensive policy guideline was put in place to take on this responsibility although several attempts have been made in the last few years. there were concerns by both organisations that the government at the national level was not providing the direction necessary to drive the it industry in the country. one respondent in organisation b stated, that they ‘hardly get any interest with the government on it’. another respondent stated that, there was ‘no push’ or ‘direction nor encouragement by the government’. as a result of the political and economic instability, law and order problems have australasian journal of information systems vol 11 no 2 may 2004 67 escalated. law and order is a major factor that has an impact on many organisations in the country. for example, telecommunication service lines were often disrupted when criminals dismantled solar panels from communication sites. telecommunication infrastructure telecommunication services is another factor that influences is success in organisations in the country. organisations depend on the communication infrastructure to effectively conduct their business. however, telecommunication infrastructure and services in the country was a major concern. although png was encouraged to invest in it infrastructure in order to contribute to economic growth and participate in the global trade, the government lacked the financial means to improve telecommunication infrastructure. a senior manager stated that, ‘the biggest problem with png is communication and the communication network is a nightmare’. in addition telecommunication costs were high, as one respondent expressed, ‘telecommunication has unrealistic prices… it is too expensive’. the vandalism of transmitter stations and land compensation claims by landowners may contribute to the disruption of telecommunication services. the people customarily own land titles in png. land compensation is one of the biggest hindrances to economic growth. national cultural issues adler (1998) argues that the national culture has a greater impact on employees then does the organisational culture. national cultural issues and factors may have an underlying effect on is adoption, use and success particularly for the ldcs. there were mixed feelings about the effect of culture on is success in both organisations. is was ‘ a fairly new thing’ in many other organisations in png, therefore, employees were initially reluctant to use the system. a respondent at organisation a commented that, generally ‘most (employees) were happy and keen to use computer systems, however some users were shy with computers’. although is training and literacy programs may alleviate this fear and reluctance, there may be some cultural aspects such as uncertainty avoidance which may have its roots in the png cultural mind-set as a collectivist society. generally, employees tend to adapt the new technology, however one cultural concern was that pngeans were usually less tolerant about the unknown and would refrain to avoid humiliation. one manager stated that pngeans, ‘hate to be shamed and humiliated, therefore we had to be very careful’ during training sessions. it was argued that employees were willing to learn, however they tend to be passive and less tolerant to deviant ideas and maintain a rigid code of belief and ethics. png features close to a collectivist society, therefore, the cultural issue raised here may be exemplifying the belief that cultural factors influence on is success in organisations in png. summary a number of factors and issues were identified and categorised into individual, organisational and external levels. the results are similar to that of factors identified by jain (1997) and ang, davis & finlay (2001). is are implemented in organisations which interact with the environment consisting of social, political, economical and cultural settings. results in this study indicate that external settings influence is success in the organisation. therefore the government must play a leading role in setting the direction for it development in the country if it is to confidently participate in the global environment. strategic issues such as information policy guidelines, national infrastructure, and economic and political stability, must be addressed. in addition it should place an emphasis on human resources development and it literacy programs because the availability of skilled is professionals continues to be a major obstacle at the national level as reflected in the organisations. unless this is achieved organisations will continue to rely on foreign expertise accumulating high costs. cultural factors also impact the uptake of is as these technologies impact social and technical change, therefore cultural implications must also be addressed. the lack of is knowledge, literacy and awareness was evident and need for is training was emphasised by the organisations in this study. organisations need to facilitate for is literacy, training and awareness programs on a regular basis for all staff, particularly the top management. australasian journal of information systems vol 11 no 2 may 2004 68 management support and funding depends on how is is perceived by top management. is investment was profit oriented in both institutions. while funding was an obstacle to is development in the organisation a management felt that there is a lack of is planning and direction at the national level. similarly there was no coherence of the aligning of business and is strategic planning. this may be attributed to proximity of the chief executive officer (ceo) and the is manager, however both organisations did not consider this factor to be important. system availability, accuracy and integrity of data and output were considered important in the organisations. conclusion the results indicate that organisational and external factors have a strong influence on is success in organisations in png. political and economical instability, escalating law and order problems, provision of proper ict infrastructure and the availability of skilled is professionals have an impact on is success in the organisations. the lack of is literacy and training, and understanding the capabilities of is by management results in the lack of proper is planning and management. unless there is is literacy among managers, poor strategic decisions will continue to be made on isolated initiatives without preconceived strategies or plans. this also reflects the lack of direction and commitment at the national level, which can lead to uncoordinated efforts and waste of much needed resources from individual organisations in the country. as a result of the poor economic environment and budget constraints, is investments in png government organisations are minimal or in some instances nil. this has cultivated a dependency syndrome among organisations, where they depend on external funding from donor agencies or international organisations such as the united nations development program (undp) for is investment. a top-down approach to is development in png is necessary, where the national government advocates is as a driver and strategic tool for change, and provide an overarching framework and direction for is development in the country. organisations will then function within this framework to coordinate their efforts in is investment. references adler, n. j. (1986). international dimensions of organizational behaviour, kent publishing: boston, ma. ang, c.-l., davis, m. a., & finlay, p. n. (2001). an empirical model of it usage in the malaysian public sector. journal of strategic information systems, 10(2): 159-174 anakwe, u. p., anandarajan, m., & igbaria, m. (1999). information technology usage dynamics in nigeria: an empirical study. journal of global information management, 7(2): 13-21. bailey, g. e., & pearson, s. w. (1983). development of tool for measuring and analysing computer user satisfaction. journal of management science, 29(5):530-545. bajwa, d., rai, a., & brennan, i. (1998). key antecedents of executive information systems success: a path analytic approach. decision support systems, 22: 31-43. delone, w. h., & mclean, e. r. (1992). information systems success: the quest for dependent variable. information systems research, 3(1): 60-95. ein-dor, p., segev, e., & orgad, m. (1993). the effect of national culture on is: implication for international information systems. journal of global information management, 1(1): 33-44. grover, v., jeong, s. r., & segars, a. h. (1996). information systems effectiveness: the construct space and patterns of application. information & management, 31(4): 177-191. hall, e. t. (1977). beyond culture. anchor press: ny. harris, h., & davison, r. (1999). anxiety and involvement: cultural dimensions toward computers in developing societies. journal of global information management, 7(1): 26-38. hofstede, g. (1983). culture and management development. undp/ilo: geneva. hofstede, g. (1997). cultures and organizations: software of the mind. mcgraw-hill: ny. jain, r. (1997). key constructs in successful is implementation: south -east asian experience. omega international journal of management science, 25(1): 267-284. australasian journal of information systems vol 11 no 2 may 2004 69 kavanamur, d. (2002), the interplay between politics and business in papua new guinea, (state, society and governance in melanesia project working paper 01/6). retrieved may 24, 2003, from http://rspas.anu.edu.au/melanesia/pdf/workingpaperkavanamur01_6.pdf larsen, m. a., & myers, m. d. (1999). when success turns into failure: a package-driven business process re-engineering project in the financial services industry, journal of strategic information systems, 8(4): 395-417. lyytinen, k. (1987). different perspectives on is: problems and solutions. acm computing surveys, 19 (1): 1-46. markus, m. l., & robey, d. (1988). information technology and organizational change: casual structure in theory and research. management science, 14(5): 583-598. miles, b. m., & huberman, a. m. (1994). qualitative analysis (2nd ed.). sage publications: thousand oaks, ca. miller, j., & doyle, b. a. (1987). measuring the effectiveness of computer-based information systems in the financial services sector. mis quarterly, 11(1): 106-123. myers, b. l., kappelman, l. a., & prybutok, v. r. (1998). a comprehensive model for assessing the quality and productivity of information systems functions: toward a theory of information systems assessments. in e. garrity & g. l. sanders (eds.), information systems success measurements (pp. 94-121) idea group: hershey, pa. myers, b. l., & tan, f. (2002). beyond models of national culture in information systems research. journal of global information management, pp. 24-32. neuman, w. l. (2000). social research methods, qualitative and quantitative approaches (4th ed.). allyn and bacon: boston. odedra-straub, m. (1993). critical factors affecting success of cbis: cases from africa. journal of global information management, 1(3): 16-32. yin, r. k. (1994). case study research: design and method (2nd ed.). sage publications: thousand oaks, ca. ajis vol.9, no. 2 may 2002 the systems approach to innovation studies dr karen manley school of construction management and property, queensland university of technology abstract the academic literature, and business practice, reveals a shift in the way analysts understand innovation processes. this shift is not revealed as a cohesive trend, rather it comprises contributions from a wide range of academic disciplines and empirical evidence. building on edquist (1997) this paper ties together the diverse new ideas which stress a systems approach to successful innovation. the paper presents an up-to-date overview of this fast moving field, with a view to assisting public policy makers and business managers in designing more effective innovation processes. keywords: innovation systems, business networking, literature review introduction* *the author is grateful to the queensland department of main roads, the australian research council and queensland university of technology for supporting research associated with this paper. defined simply, innovation can be regarded as new creations of economic significance. innovation is widely regarded as critical to economic growth processes. the academic literature contains considerable evidence of the value of innovation in driving growth. in commenting on such evidence, edquist (1997: 1) notes that: '[i]t is almost universally accepted that technological change and other kinds of innovations are the most important sources of productivity growth and increased material welfare...' (eg. oecd 2000). yet despite the value of innovation, it is only over the past decade or so that considerable progress has been made in understanding the key features of successful innovation. such progress has hinged on taking a systems perspective. this new perspective has the potential to improve innovation outcomes for firms, industries, regions and nations, through better informed public-policy and business initiatives. this shift to systems thinking is not revealed as a cohesive trend in the academic literature, nor in business practice, rather it comprises contributions from a wide range of academic disciplines and empirical evidence. it is the aim of this paper to bring together the diverse ideas on innovation systems, building on the substantial contribution of edquist (1997). this paper presents an up-to-date overview of this fast moving field, with a view to it assisting public policy makers and business managers in designing more effective innovation systems. from linear innovation processes to innovation systems during the 1950s, 1960s and 1970s, most innovation analysts viewed innovation processes as predominantly linear. the first linear model developed to explain innovation was the science-push approach where innovation was seen to begin with 'scientific discovery, passing through invention, engineering and manufacturing activities and ending with the marketing of a new product or process' (dodgson 2000: 17). the second linear model developed, the demand-pull model, portrayed innovation as being stimulated by demand. the figure below summarises the two models. figure 1: linear models of the innovation process science-push model | basic science | — > r&i demand-pull model demand — ̂ r&i ) | — ̂ production/construction | — ) — ̂ | production/construction — >• marketing | > | sales | 94 ajis vol.9, no.2 may 2002 note that neither model incorporates any feed-back mechanisms, failing to account for multi-directional information flows such as input into r&d by manufacturers or marketers. during the 1980s, more interactive models were developed so that by the 1990s, systems approaches had appeared, representing more complex views of innovation processes. such 'interactive' models are strongly supported by empirical evidence and are also intuitively appealing. a general form of this model may be represented as follows. figure 2: interactive model of the innovation process basic science/ demand marketing/ sales manufacturing/ construction this model highlights the existence of feedback loops between all stages in the innovation process. innovation is now understood as the result of interaction between various economic and social processes. these processes are rapidly increasing in complexity, through forces such as the it revolution and globalisation. to manage this complexity in the pursuit of innovation and growth, individual organisations need the capabilities of others. hence, the dominant unit of analysis in innovation studies has shifted from individual organisations, to groups of organisations in innovation systems, their interaction and potential for learning. indeed, 'learning has become the central core of the new canonical thinking about the source of wealth of nations' (de la mothe and paquet 1998: 328). this new 'systems' way of thinking has been gaining momentum from the 1970s to the present day, during which time a number of highly influential studies have emerged, including dosi et al (1998), porter (1990), lundvall (1992), nelson (1993), carlsson (1995) and oecd (1999b; 1999c) these studies all point to innovation processes driven by knowledge, relationships and learning, in the context of collaborations with external parties. these collaborations may be informal or formal; they often involve clients, customers, research organisation, regulators, finance providers and related organisations. a wide range of system configurations are replacing solo innovation efforts as firms adjust to their more complex surroundings. interactive innovation processes lie at the heart of success in the new economic circumstances. as the bie (1991:7) notes: 'for some time, studies of the innovation process have stressed the importance of networks to successful innovation, over-turning the traditional model which characterises innovation as a linear sequence running from basic research, through product development, to production and marketing. innovation is now seen as an interactive process requiring intense traffic in facts, ideas and reputational information within and beyond the firm'. the interactive view of innovation shown in figure 2 is the basis for a number of useful conceptual elaborations of the innovation process, all of which emphasise the increasing complexity of successful innovation and the importance of external knowledge sources. these approaches include: networks (freeman 1991); value-chains (walters and lancaster 2000), clusters (porter 1990; oecd 1999c), development blocks (dahmen 1988), complexes (glatz and van tulder 1989; marceau 1995), innovation milieux (camagni 1991; ratti et al 1997), complex products and systems (research policy, 29 2000); competence blocs (eliasson 1997); and innovation systems (edquist 1997). many of the ideas contained in these approaches fall broadly into an innovation systems 95 ajis vol.9, no. 2 may 2002 perspective, which is the subject of this paper as it currently receives the most attention from policy-makers and business people. readers interested in a broader review of associated literature are referred to manley (2001). innovation systems although the innovation system approach emerged only a decade or so ago, edquist and mckelvey (2000: xi) note it has: ... diffused surprisingly fast in the academic world as well as in the realms of public innovation policy making and firm innovation strategy formulation. the oecd has been particularly influential in using and further developing empirical analyses and research using this approach. 'systems of innovation' is at the centre of modern thinking about innovation and the relations of innovation to economic growth, competitiveness and employment. the systems perspective recognises that innovation is a collective undertaking where innovating organisations interact with others in particular institutional settings. innovations are developed in systems where organisations and institutional rules are key elements. system approaches to innovation focus on the nature and impact of the collective character of innovation. system features will have a 'decisive impact on the extent to which firms can make innovation decisions, and on the modes of innovation which are undertaken' (edquist 1999: 7-8). the innovation system approach is 'holistic and interdisciplinary' and has the potential to encompass all the determinants of innovation. it differs from earlier analytical approaches in assuming that innovation relies primarily on interactions between institutions and people (landry and amara 1998: 261). all the features of innovation systems stem from the collective nature of successful innovation efforts. for any organisation, success relies on relationships with external parties. this view is supported by numerous empirical studies, an example of which is a rigorous examination of an innovation system by landry and amara (1998: 274). their case-study work indicates that 'innovative firms develop more interactions with outside sources of ideas, information and technology than non-innovative firms do'. in may respects, an innovation system is a social system in which innovations emerge partly as a result of social interaction between economic actors from different organisations (cooke 1998: 11). the oecd has played a major role in the mobilisation of this approach to understanding economic growth. according to the oecd, innovation is the result of a complex set of linkages between actors creating, applying and distributing various kinds of knowledge. innovation performance depends critically on 'the way these actors relate to each other as elements of a collective system of knowledge creation and use ... ' (1997a: 9). the following sections highlight four key drivers of innovation in contemporary innovation systems knowledge flows, institutions, economic competence and interactive learning. these are the main innovation inputs discussed in the literature. knowledge flows the current popularity of the innovation system approach reflects the increasing knowledge intensity of economic activity. a key variable of interest is the 'knowledge distribution power' of a particular system, which involves the system's ability to facilitate effective knowledge flows. analysis of a particular system will involve tracking the linkages between industry, government and academia in the development of technological and organisational innovations. a key policy aim is to identify the main channels of knowledge flow, to evaluate bottlenecks and to suggest approaches to improve the effectiveness of knowledge diffusion (oecd 1997a: 11). the oecd has developed a model to guide empirical studies of knowledge flows in innovation systems, as shown in table 1. 96 ajis vol. 9, no. 2 may 2002 table 1: assessing innovation systems four basic flows of knowledge: five key research methods: joint activities between firms 1. firm surveys 2. literature-based alliance counting technical collaboration 3. value of research contracts informal interactions (tacit knowledge transfer) 4. co-patents and co-publications 5. citation analysis 2. joint activities between firms and public-sector r&d providers technical collaboration informal interactions (tacit knowledge transfer) innovation diffusion adoption of new processes, equipment and machinery developed elsewhere personnel mobility flows of tacit knowledge source: oecd 1997a in this model, four key types of knowledge flows are highlighted joint firm collaboration, joint privatesector/public-sector collaboration, innovation diffusion and personnel mobility. joint activities between firms often take the form of collaboration between producers to pool technical resources, gain economies of scale and achieve synergies from complementary technical and human assets. user-producer interaction is also important in stimulating innovation. in both cases producer/producer and user/producer interaction, formal and informal relations are important. joint activity between firms and public-sector r&d providers typically takes the form of contracts for r&d undertaken by the public-sector. the value of these linkages is determined by the quality of the research infrastructure and its accessibility to firms. private/public-sector interaction is particularly valuable within innovation systems, as the public-sector is often a key performer of r&d and resultant knowledge needs to be transferred to users in order to promote innovation and growth. innovation diffusion the adoption of advanced methods and technologies by new users this is the most traditional form of knowledge flow in innovation systems (oecd 1997a). the innovation performance of a particular system depends critically on the rate of such adoption and investment in appropriate adaption to ensure successful implementation. the movement of personnel and their knowledge capital is considered another key flow in innovation systems (oecd 1997a). personnel mobility reflects the movement of tacit knowledge knowledge which cannot be written down, which is often critical in the successful integration of innovations produced by external parties. as noted in table 1, there are five basic methods available for gathering data on knowledge flows: 1. firm surveys which involve the use of questionnaires and personal interviews to elicit information on collaborative behaviour and the sources of knowledge for innovation, particularly with regard to tacit knowledge flows. 2. literature-based alliance counting which gives a rough idea of the nature and extent of the different types of alliances within which knowledge is exchanged. it involves reviews of newspapers, journal articles, specialised books and journals, together with corporate annual reports and industry directories. 3. analysis of research contracts indicates the extent to which the private-sector is linked to public-sector r&d capabilities. information about the value of such contracts can be obtained from government funding sources, universities, private-sector research bodies and/or private-sector users. 4. the number of co-patents and co-publications, indicating inter-firm collaboration and private-public sector collaboration, can be ascertained from patent records and publication indices. 97 ajis vol. 9, no. 2 may 2002 5. users of r&d cite patents and publications in their literature on new developments. these citations can be counted and analysed to reveal collaboration patterns, particularly between the private and public sectors. this oecd approach focuses on the role of knowledge in innovation systems. statistics canada (anderson and manseau 1999: 3) also focuses on the role of knowledge, and its movement from the site of generation to the site of use, which requires: • a means of transmitting knowledge between sites; • the capacity of knowledge generators to transmit knowledge; and • the capacity of knowledge users to search for and absorb knowledge. from an individual firm's perspective, knowledge sites include those internal to the firm and those found in the external environment, for example (landry and amara 1998: 265): market sources • clients • competitors • suppliers • consultants education and r&d providers • universities • public and private r&d organisations local, state and national framework providers • regulators • finance providers • trainers • unions • professional associations generally available information • fairs/exhibitions • conferences/workshops/seminars • journals • patent disclosure institutions play an important role in determining the efficacy of knowledge flows between sources such as those described above. institutions institutions are the 'rules of the game' which govern how knowledge moves between system participants and the way in which subsequent learning and innovation takes place. nooteboom (2000: 916) suggests that firm-level innovation outcomes are, to a large extent, dependent on the institutional context. the following key institutions affect the performance of innovation systems (based on amable and petit 1999: 12): • the finance system; • the taxation system; • the intellectual property rights system; • the training system; • the education system; • the industrial relations system; • labour markets; • the internal structure of corporate firms and government bodies; • conceptions of fairness and justice held by capital and labour; • the structure of the state and its policies; and • idiosyncratic customs, traditions, norms, moral principles, rules, laws, standards and routines. 98 ajis vol. 9, no. 2 may 2002 the innovation performance of organisations is strongly influenced by these institutions. for example, if the education and training systems are under-resourced by government, firms may find it difficult to access suitably skilled employees to support their innovation efforts. another example is the australian federal government cutting back of the 150% r&d tax concession in 1996 which had a significant impact on r&d expenditure in australia (see marceau and manley 1999). on a more positive note, australian culture tends to support early adoption of new innovations, which can help tie australia into global innovation systems. interactive learning another key feature of innovation systems is interactive learning. the term interactive learning implies a reliance on multiple sources of tacit knowledge in the learning process. in turn, learning has been shown to be a key input in the innovation process (eg. dodgson 1996). dosi (1988: 222-223) observes that there are five problematic features of innovation processes that require interactive learning in order to promote positive outcomes, these are: uncertainty: including the lack of full information about the occurrence of known events, the existence of techno-economic problems whose solution procedures are unknown, and the inability to predict precisely the consequences of one's actions. the involvement of multiple players in interactive learning increases the stock of knowledge and the breadth of experience that can be drawn on to reduce uncertainty in the innovation process. scientific knowledge: the increasing reliance of major new technological opportunities on advances in scientific knowledge highlights the importance of linkages between r&d users and major r&d organisations in interactive learning. complexity: the increasing complexity of r&d activity means that multiple players are needed in order to access multiple knowledge sources, rather than just relying on the skills of individual innovators. experimentation: the increasing role of experimentation in the form of learning-by-doing and learning-by-using requires access to appropriate partners to maximise the value of experimentation. cumulativeness: the cumulative character of innovative activity means that past decisions shape future opportunities. hence, it is important to keep options open by maintaining a broad array of innovation interests through multiple relationships, through activities such as: user-producer interaction, formal r&d agreements, professional association functions, consultations with regulators/training organisations/fiance providers, production agreements, licencing, joint ventures, sub-contracting and conference/workshop/forum attendance. in summarising the importance of interactive learning, lundvall (1999: 3) notes that 'the last decade has witnessed a change in the mode of competition that implies that interactive learning, and forgetting, has become the most important process for determining the position of individuals, firms, regions and countries in competition'. economic competence the final feature of innovation systems reviewed here is the economic competence of a system's participants. the outcomes of interactive innovation process are 'a function of the level and content of economic competence on the part of various agents within the system' (carlsson and stankiewicz 1991: 113). this quality, or economic competence, is the microfoundation upon which the success of the innovation system will rest. economic competence involves the ability to initiate and exploit new business opportunities. it is a 'scarce and unequally distributed resource ... not all economic agents ... are equally adept at generating new ideas or absorbing new ideas from outside' (carlsson and stankiewicz 1991: 94). carlsson et al. (1999: 5) identify four crucial types of firm-level competence in innovation systems: 1. selective/'strategic ability: the ability to: make innovative choices between markets, products, technologies and organisation structures; engage in entrepreneurial activity; select key personnel; and acquire other key resources, including new competencies. an important part of strategic capability is the notion of receiver competence or absorptive capacity, which involves the ability to scan and monitor relevant technological and economic information; identify technical and market opportunities; and acquire knowledge, information and skills needed to exploit opportunities. 2. organisational/integrative/coordinating ability: the ability to organise and coordinate resources and economic activities within the organisation so that overall objectives are met. this includes the ability to generate and improve technologies through new combinations of existing knowledge and skills. this should be the main function of middle management in an organisation. 3. technical/functional ability: the efficient execution of various functions within the firm to implement technologies and utilise them effectively in chosen markets. 99 ajis vol.9, no. 2 may 2002 4. learning/adaptive ability, the ability to learn from success as well as failure, to identify and correct mistakes, and to diffuse technology throughout the organisation. this ability is essential for long-term survival. a firm that is both effective and efficient at a point in time eventually becomes neither unless it can adapt to changing circumstances (especially changing technology). later research by nooteboom et al. (2000: 120) suggests that a fifth relational ability is required. 5. relational ability: the ability to focus on core competencies and utilise complementary resources from other firms. this ability has increased in importance in recent years due to the pressures of globalisation, the increasing complexity of input and output markets and the increasing speed of technical change. the presence of these five abilities in the organisations which make up an innovation system will greatly assist the system's performance, particularly by improving the value of interactive learning opportunities. conclusions this paper has touched on four key drivers of innovation in innovation systems knowledge flows, institutions, interactive learning and economic competence. these drivers are the main ones to emerge from analysis of a wide range of contributions to the academic literature. compared to earlier linear approaches to understanding innovation processes, systems thinking has contributed a great deal to our understanding of effective innovation processes. although the literature acknowledges that an ideal system is not possible to specify (edquist 1997: 20), the following guidelines concerning the nature of a well-performing innovation system can be put forward: organisations: an innovation system is more likely to be effective if it includes a diverse range of organisations with different types, and high levels of, economic competence. the system is likely to operate more effectively if it includes firm and non-firm organisations, demanding customers, and new industry players. institutions: innovation system performance is critically affected by institutional arrangements. critical institutions, such as those relating to training, education, finance and intellectual property need to incorporate arrangements that are supportive of innovation. linkages with external parties: maximum advantage will be gained if external linkages are dense, multistranded, long-term, knowledge-intensive, vertical and horizontal, market and non-market oriented, and inclusive of innovation, production and distribution relationships. such factors support the development of effective dynamics within innovation systems. outcomes: innovation system outcomes will be maximised if individual participants perceive benefits beyond what they could achieve in isolation. the system is then more likely to be stable and productive. in the rapidly changing environment of the 21st century, the ability of businesses and governments to generate growth and jobs will rest critically on their innovation performance. the systems framework represents a new and increasingly popular approach to understanding and improving that performance. references anderson, f. and manseau, a. (1999) a systemic approach to generation/transmission/use of innovation in construction activities. paper presented at the third international conference on technology policy and innovation: global knowledge partnership-creating value for the 21st century, austin, texas, 30 august 2 september 1999. amable, b. and petit, p. (1999) on institutions, innovation and growth. provisional draft of paper presented at the danish research unit for industrial dynamics summer conference on national innovation systems, industrial dynamics and innovation policy, rebild, 9-12 june 1999. archibugi, d., howells, j. and michie, j. (1999) 'innovation systems and policy in a global economy'. in d. archibugi, j. howells and j. michie (eds) innovation policy in a global economy, uk: university of cambridge, 1-16. braczyk, h-j, cooke, p. and heidenreich, m. (1998) regional innovation systems: the role of governances in a globalised world, london: ucl press. breschi, s. and malerba, f. (2000) 'sectoral innovation systems: technological regimes, schumpeterian dynamics, and spatial boundaries'. in c. edquist (ed.) systems of innovation: technologies, institutions and organisations, chapter 6, london and washington: pinter, 130-156. bie (bureau of industry economics) (1991) 'networks: a third form of organisation'. bulletin on industry economics, 10: 5-9. 100 ajis vol. 9, no. 2 may 2002 bie (bureau of industry economics) (1995) beyond the firm. an assessment of business linkages and networks in australia. canberra: agps. camagni, r. (ed.) (1991) innovation networks: spatial perspective's. belhaven: london. carlsson, b. and stankiewicz. (1991) 'on the nature, function and composition of technological systems'. journal of evolutionary economics, 1: 93-118. carlsson, b., jacobsson, s., jolmen, m. and rickne, a. (1999) innovation systems: analytical and methodological issues. danish research unit for industrial dynamics working paper. cooke, p., uranga, m.g., and extebarria, g. (1997) 'regional innovation systems: institutional and organisational dimensions'. research policy, 26:475-491. dahmen, e. (!988) 'development blocks in industrial economies'. scandinavian economic history review and economic history, xxxvi(l): 3-14. de la mothe, j. and paquet, g. (1998) 'local and regional systems of innovation as learning socioeconomics'. in j. de la mothe and g. paquet (eds) local and regional systems of innovation, boston, dordrecht and london: kluwer academic publishers, 1-16. de la mothe, j. and paquet, g. (1998b) 'some lessons and challenges for model builders, data gatherers and other tribes'. in j. de la mothe and g. paquet (eds) local and regional systems of innovation, boston, dordrecht and london: kluwer academic publishers, 327-334. debresson, c. and amesse, f. (1991) 'networks of innovators: a review and introduction to the issue'. research policy, 20: 363-379. dodgson, m. (1996) 'technology and innovation: strategy, learning and trust'. in sheehan, p., grewal, b. and kumnick, m. (eds) dialogues on australia's future. melbourne: centre for strategic economic studies, victoria university. dodgson, m. (2000) systemic integration of the innovation process within the firm. contributed paper #2 at the national innovation summit, melbourne, 9-11 february 2000. dosi, g., freeman, c., nelson, r., silverberg, g. and soete, l. (eds) (1988) technical change and economic theory. london: frances pinter. easton, g. (1992) 'industrial networks: a review'. in b. axelsson and g. easton (eds) industrial networks: a new view of reality. london and new york: routledge, 1-27. edquist, c. (1997) systems of innovation: technologies, institutions and organisations. london: pinter. edquist, c. (1999) innovation policy a systemic approach. incomplete draft of paper presented at the danish research unit for industrial dynamics summer conference on national innovation systems, industrial dynamics and innovation policy, rebild, 9-12 june 1999. edquist, c. and m. mckelvey. (2000) 'introduction'. systems of innovation: growth, competitiveness and employment, volume 1. cheltenham, uk: edward elgar publishing limited, xi-xxii eliasson, g. (1997) competence blocks and industrial policy in the knowledge based economy. stockholm: department of industrial economics and management, the royal institute of technology. fagerberg, j. (1995) 'user-producer interaction, learning and comparative advantage'. cambridge journal of economics, 19: 243-256. freeman, c. (1987) technology policy and economic performance: lessons from japan. london: pinter. freeman, c. (1991) 'networks of innovators: a synthesis of research issues'. research policy, 20: 499-514. glatz, h. and van tulder, r. (1989) ways out of the international restructuring race? project proposal, annex, b. university of amsterdam, amsterdam. howells, j. (1999) 'regional systems of innovation?'. in d. archibugi, j. howells and j. michie (eds) innovation policy in a global economy, uk: university of cambridge, 67-93. howells, j. and roberts, j. (2000) 'from innovation systems to knowledge systems'. prometheus, 18(1): 1731. landry, r. and amara, n. (1998) 'the chaudiere-appalaches system of industrial innovations'. in j. de la mothe and g. paquet (eds) local and regional systems of innovation, boston, dordrecht and london: kluwer academic publishers, 257-276. lundvall, ba (1988) 'innovation as an interactive process: from user-producer interaction to the national system of innovation. in dosi, g., freeman, c., nelson, r., silverberg, g. and soete, l. (eds) technical change and economic theory. london: pinter. lundvall, b-a. (1992) national systems of innovation. london: pinter. lundvall, b-a. (1999) the danish innovation system. discussion paper on the challenges and opportunities for innovation policy. the concluding disko report. copenhagen: erhvervsfremme styrelsen. malerba, f. (1999) sectoral systems of innovation and production. paper presented at the danish research unit for industrial dynamics summer conference on national innovation systems, industrial dynamics and innovation policy, rebild, 9-12 june 1999. 101 aj1s vol. 9, no. 2 may 2002 manley, k. (2001) 'innovation systems developing an integrated framework for investigating the queensland road sector'. queensland department of main roads, brisbane. marceau, j. (1995) 'a networked nation or a complexes issue? reshaping industry analysis'. journal of industry studies, 2(2): 19-33. marceau, j. and manley, k. (1999) 'innovation checkpoint 1999 innovation in australian businesses'. australian business foundation, sydney. nelson, r. (ed) (1993) national innovation systems. new york: oxford university press. nooteboom, b. (2000) 'institutions and forms of co-ordination in innovation systems'. organisation studies, 21(5): 915-939. nooteboom, b., dejong, g., vossen, r.w., helper, s. and sako, m. (2000) 'network interactions and mutual dependence: a test in the car industry'. industry and innovation, 7(1): 117-144. normann, r. and ramirez, r. (1993) 'from value chain to value constellation: designing interactive strategy'. harvard business review, 71 (july-august): 65-78. oecd (1996) the knowledge-based economy. paris: oecd oecd (1997) national innovation systems: paris: oecd. oecd (1999a) the knowledge-based economy: a set of facts and figures. paris: oecd oecd (1999b) managing national innovation systems. paris: oecd. oecd (1999c) boosting innovation: the cluster approach. paris: oecd. oecd (2000) a new economy? the changing role of innovation and information technology in growth. paris: oecd. patel, p. and pavitt, k. (1994) 'national innovation systems: why they are important, and how they might be measured and compared'. economics of innovation and new technology, 3: 77-95. porter, m. (1990) the competitive advantage of nations. new york: free press. porter, m. (1990) 'the competitive advantage of nations'. harvard business review, march-april: 73-93. porter, m. e. (1998) 'clusters and the new economics of competition'. harvard business review, novemberdecember: 77-90. ratti, r., bramanti, a. and gordon, r. (1997) the dynamics of innovative regions. england: ashgate. rosenberg, n. (1982) inside the black box. cambridge: cambridge university press. sornn-friese, h. (2000) 'frontiers of research in industrial dynamics and national systems of innovation'. industry and innovation, 7(1): 1-13. statistics canada (1999) innovation, advanced technologies and practices in the construction and related industries. survey for science and technology redesign project. statistics canada (1999) survey of innovation 1999. survey by science, innovation and electronic information division, statistics canada. von hippel, e. (1988) the sources of innovation. new york: oxford university press. walters, d. and lancaster, g. (2000) 'implementing value strategy through the value chain'. management decision, 38(3): 160-178. 102 microsoft word ajis vol 15-1 m.doc australasian journal of information systems volume 15 number 1 december 2007 77 a foundation assessment of the influence of it management practices on customer relationship management (crm) in a large australian federal government agency judy young university of tasmania, tasmania, australia email: judy.young@infosys.utas.edu.au carol pollard appalachian state university, north carolina ,usa email: pollardce@appstate.edu paul gregg capital3 abstract there is currently a thrust in australia to move to whole of e-government. at the centre of this initiative is the need to ensure high quality customer relationship management (crm). there is a paucity of research into crm in the public sector and this research breaks new ground by exploring the level of customer relationship management (crm) implementation in a large australian federal government agency. using a quantitative survey approach, data were collected through the distribution of an electronic questionnaire to it heads of 23 departments in one large australian federal government agency. the main finding of the research is that although a few of the departments have implemented crm at the highest level, the majority of the departments are still in the lower levels of crm implementation. differences in it management practices and their impact on crm status are also measured and their impact on crm implementation is discussed. finally, recommendations for moving to more advanced stages of crm implementations are made. australasian journal of information systems volume 15 number 1 december 2007 78 introduction in developed countries, governments are now faced with increasing political and community pressure to provide efficient and cost effective services (kolsky & keller, 2001). this has created a goal of working towards e-government whereby there will be seamless interactions and transactions between government departments, agencies and clients. as a result, customer relationship management (crm) is receiving considerable attention because it underpins the thrust of these institutions to move to whole of e-government (armstrong, 2002). more specifically ‘…(i)t can assist in, among other benefits, streamlining government processes, improving inter-agency data sharing and providing self-service options to the public’ (accenture 2002:18). according to batista and kawalek (2004:128), ‘crm can be understood as a broad business strategy that implies the redevelopment of organisational structures so that there are new service units and new product offerings arranged around a refreshed understanding of customer needs’. customer relationship management, (crm), albeit in different forms, has long been employed by for-profit organisations to build customer loyalty and support to assure repeat business. in the context of the modern business environment crm has been facilitated by information and communication technologies (icts). in more recent times crm has not been limited to for-profit organisations. it is becoming an increasingly imperative issue for government not only in australia, but also throughout the world (accenture 2004; chadwick & may, 2003; marche & mcniven, 2003). however, despite the importance of implementing crm, many of the current infrastructures and processes in some government departments are inhibiting crm from becoming a reality (accenture, 2002). this situation provides a basis for this research. it is designed to explore the current level of crm implementations and the influence of it management practices in the public sector and specifically within one large agency within the australian federal government. accordingly, two research questions are posed: 1. what is the current level of crm implementation in departments of a large australian federal government agency? 2. what it management practices impact on the level of crm implementation? this research extends earlier work into the topic of crm in the private financial sector in the usa conducted by karimi, somers and gupta (2001). the perceived benefits of using this reference research are that it provides an available validated survey instrument and a customer service topology of crm adoption in the private sector that can be tested in the public sector. the current research also took an innovative approach by cross validating the topology through a discriminant function analysis of the customer service topology. background historically crm has a foundation in the era prior to1960, when direct selling was used to enlist and support customer loyalty based on personal attention. during the 1980s, advances in technology helped companies narrow their efforts by using target marketing. however, further deregulation of the market and the lack of coordination of channels and multiple messages being received about australasian journal of information systems volume 15 number 1 december 2007 79 different products from the same companies only added to the complexity of decision-making and the confusion experienced by the customer (ling & yen, 2001). in addition, during this period the focus was on individual sales and not on the individual customer (dibb & meadows, 2001). the crm era that commenced in the 1990s has seen a technology-enabled return to the direct sales era and a focus on the customer while still using the mass production and distribution systems that evolved through the intervening eras (ling & yen, 2001 corner, hinton & kay, 2003). modern information communications technology (ict) now provides the capability to service customers with personalised goods and services. it provides the means of addressing the labour intensive aspects of storing, upgrading and retrieving the profile and personal preferences across a wide range of very different customers (goodhue, wixom & watson, 2002) and enables functions not previously available (cenfetelli, benbasat & al-natour, 2005). more so than ever before, crm now affords for-profit organisations the ability to ‘establish, nurture and sustain long-term customer relationships’ (winer, 2001:89). in this context the notion of lifetime customer value (lcv) has emerged as an important outcome from the implementation of crm (winer, 2001). lcv provides the ability to assess customers in terms of the current and future profitability they can bring to the organisation. it also enables the opportunity to identify where additional profits can be gained from customers (winer, 2001). from this background, it is evident that to date an overwhelming majority of the literature relates to crm in the commercial business environment. within this context, research has been conducted by consultants and has focussed on the bottom line and the relationship between crm and increased revenue and profits (goodhue, wixom & watson, 2002; rigby, reichhald & schefter, 2002; swift, 2002). the increasing importance of crm in public sector organisations crm is no longer exclusively the domain of for-profit business activities. technology facilitated crm is now also becoming an essential component in government (batista & kawalek, 2004; accenture, 2003; noie, 2003). the driving force is the need to streamline large bureaucratic institutions. in particular public institutions are now more accountable in the cost and quality of the services they provide. since citizens are stakeholders, expectations of fairness, integrity and responsiveness are high (price & brodie, 2001). in terms of equality this also implies servicing the needs of citizens regardless of whether they live in regional or rural areas. the use of ict provides a means to capture personal profiles of clients in relation to their specific needs, behaviours and preferences. this information can be used to assure clients that they are being provided with a personalised service, even though federal government departments/agencies are actually servicing a very large number of clients with a wide range of requirements. the technology underpins the current aim of government to have seamless interactions and transactions working towards whole of e-government. while crm is pivotal to enable the achievement of these initiatives, (kolsky & keller, 2001) it has been suggested that currently the move towards the desired outcome is being inhibited by the infrastructures and processes within some australian federal government departments (accenture, 2004; clarke, 2003). this implies that until enterprise transformation, including associated management practices, are implemented the goal of collaborative crm at the highest level cannot be achieved (radcliffe, kirby & thompson, 2001). due to the relatively recent adoption of crm within a government context very little research has been reported on this issue. australasian journal of information systems volume 15 number 1 december 2007 80 stages of crm development table 1 provides a framework of the three stages of crm (operational, analytical and collaborative) proposed by karimi, somers & gupta (2001). in addition, it summarises the scope of the issues involved in each stage. this framework was constructed with reference to crm literature and the work of malhotra (2000), goodhue, wixom & watson (2002), chadwick & may (2003) and croteau & li (2003). stage 1, operational crm, is the first step an organisation takes towards the implementation of crm. it involves individual applications aimed at the process level that are typically not coordinated within the organisation. this implies that the infrastructure is at a localised departmental level and there are a limited number of users (goodhue, wixom & watson, 2002). at this stage the focus is on low cost that can be linked to the bottom line via productivity improvements. stage 2, analytical crm, involves the implementation of a dedicated crm infrastructure to facilitate the scrutiny of customer and market databases. this provides the means to better allocate resources (rigby et al., 2002; xu & walton, 2005). while this stage involves the implementation of a dedicated infrastructure across the business (goodhue, wixom & watson, 2002) it is often provided to automate or consolidate existing business practices and processes. stage 3, collaborative crm, involves organisational transformation and is the most expensive and risky stage. transformation implies change in the organisation as new business practices and applications centred on the customer are put in place (goodhue, wixom & watson, 2002). within this products and services are personalised to meet the needs of individual customers. transformation also encompasses non technical issues such as organisational culture. table 1 provides a comparative summary from a number of areas relative to crm within each of the stages. areas stage 1 stage 2 stage 3 operations analytic collaborative infrastructure localised departments crm infrastructure organisation transformation sponsorship departmental level corporate it top management involving all levels of the organisation applications multiple individual many individual new to support changed business practices data limited in scope & volume large from a disparate source of systems changing data infrastructure users limited (small group) growing user base nearly all personnel involved benefits productivity improvements: greater local revenues and profits cost savings from infrastructure consolidation potential for great increases in revenues and profits cost low medium-high expensive risk low medium high table 1: scope of the 3 stages of crm (karimi, somers and gupta, 2001) australasian journal of information systems volume 15 number 1 december 2007 81 customer service topology karimi, somers and gupta (2001) also established a customer service topology wherein the progress of organisations towards collaborative crm could be classified according to one of four quadrants. the application of this topology enables organisations to be defined as it leaders, it enabled customer focus organisations, it enabled operations focus organisations and it laggards. as figure 1 shows, an “it leader” organisation needs to score above average on both customer and operations focus. “it-enabled customer focussed” organisations are established on the basis of having an above average result for customer focus and a below average result for operations focus. an “it-enabled operations” focused organisation is one that is above average for operations focus and below average focus for customer focus. finally, “it-laggard” organisations score below average on both customer and operations focus. integrating figure 1 with table 1 suggests that it-laggard organisations fall within stage 1 of crm implementation. organisations determined to be either it enabled customer or operations focus are at stage 2, while it leaders are at the final, stage 3, collaborative crm. figure 1: customer service ttopology (karimi, somers & gupta, 2001:130). karimi, somers and gupta (2001), also link their customer service topology with it management practices. in so doing, they go beyond the basic operations or customer focus to identify it management sophistication and the role of the it leader as important measures of it management practice. it management sophistication is broken down into four dimensions: it planning, it control, it organisation and it integration. within this context, it planning involves the long-term strategic application of it. this implies a shift from a focus on a computing plan based on technology to the alignment of it with the business objectives of the organisation. it control describes the formal procedures that need to be in place so it’s potential for impact on customer focus (customer focus (cf)) customer service topology it’s potential impact on operations operations focus (of) high high low 3 it enabled operations focus organisations 1 it-leader organisations 4 itlaggard organisations 2 it enabled customer focus organisations australasian journal of information systems volume 15 number 1 december 2007 82 the performance of it can be formally defined and monitored. this procedure needs to have established guidelines as to how this should be conducted. it organisation focuses on how it is handled within the organisation. rather than being a stand-alone component of the organisation it needs to be incorporated into the business units to support the business objectives. in more crm advanced organisations a high degree of it integration is an essential prerequisite to achieve holistic crm. the role of the it leader is seen as crucial for the successful implementation of collaborative crm. the person working in this capacity needs to be a corporate member in the organisation who has a comprehensive understanding of the business and works closely with top management. in addition, the it leader needs to be capable of delivering it solutions to support the organisation achieve its objectives. this implies that the role of the it leader is not simply that of a technical manager, but that of an innovative business leader. in their study of 213 it leaders in the u.s. financial services industry, karimi, somers and gupta (2001) found a strong direct relationship between it management practice and level of crm implementation. in particular, they found that method this research is based on a quantitative approach in which a web-based questionnaire was administered to the chief information officer (cio) or an officer of an equivalent ranking in all 23 departments of one large information-intensive australian government agency that uses it strategically to deliver a new level of services to its constituents. the agency, located in canberra, the national capitol of australia was identified with the assistance of a federal government officer with considerable knowledge of the overall structure of government in australia. survey instrument the research instrument was adapted from karimi, somers & gupta (2001) who drew on existing research in the development of this questionnaire. for example, the it maturity scale was based on previous work from benbasat, dexter & mantha (1980), earl (1989) and sabherwal & kirs (1994). the it leader role items came from the earlier work of karimi, gupta & somers (1996) as well as that of applegate, mcfarlan & mckenney (1996) earl (1989) feeny, edwards & simpson (1992) and sager (1998). applegate (1996) was the source of reference for the impact of it on operations.. in adopting this instrument it was necessary to ensure the questionnaire was appropriate in the context of the australian public sector. in particular, items that were only relevant to for-profit organisations were removed. a new demographic section was developed that contained 14 questions aimed to enable comparative analysis of the departments of the australian federal government agency being studied. . following karimi, somers & gupta (2001) the main body of the questionnaire was divided into three main aspects of crm: it focus on operations, it focus on customer (client) interactions and it management practices (it sophistication and it leader’s role). responses in this section were based on a five-point likert scale where 1 indicated strong disagreement and 5 indicated strong agreement with 40 statements related to these three aspects of crm. australasian journal of information systems volume 15 number 1 december 2007 83 pilot the questionnaire was subjected to two rounds of pilot review to confirm that the instrument was suitable in the context of the australian public sector culture. the first was conducted using a paper-based draft of the questionnaire and involved two perspectives of review. an academic perspective provided input from those with experience in conducting information systems research. the evaluation by public sector officers enabled feedback from those working within the proposed research environment. the feedback obtained from this process provided an opportunity to refine the questionnaire before its conversion into electronic format. the second round of pilot testing was conducted using the electronic version of the questionnaire. seven senior officers from the host australian federal government agency participated. the outcome from this pilot process resulted in the instrument being subjected to further minor adjustments. for example, some terminologies were changed to fit the australian context. the pilot reviews addressed both the face validity and content validity of the questionnaire. administration of the main questionnaire the questionnaire was distributed to the heads of 23 departments in the designated australian government agency an introductory e-mail from the commissioner of the host federal government agency was also distributed to each department head to encourage recipients to urge their chief information officer or similarly-ranked it leader to take part in the research. the e-mail provided a link to the web-based questionnaire that could be completed electronically, saved and then returned via e-mail to the researcher. at the web site, potential respondents could access an information sheet that described the objectives of the research as well as establishing its compliance with the requirements of the human ethics committee at the university of tasmania. responses were received over a one-month period. the benefit of using an electronic questionnaire was that responses were collected and collated without any need for translation or manual coding. as such coding errors and omissions were minimised. results of the 23 surveys administered, 17 were completed and returned for a 74% response rate. this response rate is well above the level generally regarded as acceptable for survey research (neuman 2000). the surveys were completed by seven cios, two functional managers, two it managers, three is or knowledge service managers and three it infrastructure managers, who had served in their present position for periods ranging from one to five years. a profile of participating departments table 2 provides the characteristics of each of the departments of the australian federal government agency, as a whole and by customer topology. all the departments were relatively large in terms of annual budgets and number of employees. thirteen (77%) departments had budgets greater than $100 million and only one reported an annual budget between $10-$50 million and no department reported budgets below $10 million. similarly, it budgets ranged from $11 million to in excess of $100 million, with 10 departments. in terms of number of employees, the majority of the departments were relatively large employing more than 2,000 employees. only australasian journal of information systems volume 15 number 1 december 2007 84 three departments employed less than 500 employees. funding allocations were proportionate to number of employees. in most of the departments, the it functional head sat one or two levels below the ceo and all departments reported they had engaged in some form of it planning. no. employees n annual budget n it annual budget n >2000 9 >$100 million 13 >$100 million 4 1001-2000 2 $51-100 million 3 $11-100 million 10 500-1000 3 $10-50 million 1 $1-10 million 3 <500 3 it leader below ceo n it planning n 1 level below 6 1-5 years 10 2 levels below 6 6 -10 years 4 3 levels below 5 >15 years 3 table 2: profile of departments (n=17) the survey data were checked within spss and found to be of normal distribution with respect to skewness and fisher kurtosis, thus allowing the use of robust parametric statistical analysis. there were no outliers. the data were also tested for reliability. this tested the internal consistency of a set of scale items (francis 2004), the results of which are shown in table 3. although the instrument used had been previously validated, it is interesting to note that in the current setting it was necessary to remove 11 items from the original 40 items to achieve a cronbach alpha reliability measure of .70 or above as prescribed by nunnally (1993). the resulting 29-item questionnaire achieved reliability scores for the it focus on operations, it focus on customer (client) interactions, it management sophistication and it leader’s role ranging from .71 to .85. a copy of the questionnaire is available on request from the first author. table 3 also provides the level of agreement (standard deviation) for each item. the highest level of agreement (sd .55) relates to responses to the it leader’s role section of the questionnaire. the level of agreement for it planning and it organisation was slightly less (.79 and .72 respectively). the lowest level of agreement was recorded for it integration (.86). australasian journal of information systems volume 15 number 1 december 2007 85 construct mean (sd) cronbach’s alpha no. of items it-enabled focus adjusted original operations 3.32 (.81) 0.73 4 4 customer 3.63 (.83) 0.80 4 6 it management practices it sophistication 3.89 (.72) 0.85 it planning 4.07 (.62) 0.79 4 6 it organisation 3.88 (.67) 0.72 4 4 it control 3.87 (.74) 0.85 6 6 it integration 3.67 (.86) 0.75 3 6 it leader’s role 4.38 (.55) 0.71 4 8 total 29 40 table 3: construct reliability (n=17) to determine whether items of a construct correlate more highly with items within the construct than with those that measure other constructs it is useful to use discriminant analysis. to test the discriminatory properties of the items and constructs, two tests were performed. first, a correlation matrix was constructed to determine the percentage of within-construct correlations that associated higher with an item from another construct than with items within its own construct. this test revealed that only 28% of the items associated more highly with an item from another construct, thus meeting the criteria of less than 50% set by campbell and fiske (1959). next, bi-variate correlations between each of the construct scales were analysed and the average variance explained was computed as shown in table 4. these results demonstrate that all constructs were more highly correlated within their dimensions than they were with other constructs, indicating that the constructs were distinct. further support for the distinctiveness of the constructs was provided in that the average variance explained for all constructs exceeded the .50 level recommended by fornell and larcker (1981). construct (no. items) ops cust it soph it plan it org it cont it integ it leader role it-enabled focus operations (4) .73 customer (4) .272 .80 it management practices it sophistication (17) -.196 .043 .85 it planning (4) -.066 .075 .783 .79 it organisation (4) -.332 -.054 .700 .421 .72 it control (6) -.387 -.053 .708 .466 .779 .85 it integration (3) .189 -.064 .361 .361 .088 -.171 .75 it leader's role (4) -.057 .255 .702 .444 .495 .386 .582 .71 table 4. inter-correlation matrix of constructs australasian journal of information systems volume 15 number 1 december 2007 86 next, each of the 17 participating australian federal government departments were classified into one of the topology quadrants suggested by karimi, somers and gupta (2001) based on each department’s individual mean response for customer focus and operations focus. those scoring higher than the cumulative mean on customer focus or operations focus were classified in the high category for that dimension, and vice versa. those departments who scored higher than the cumulative mean on both dimensions were classified as it-leaders. those scoring lower than the cumulative mean on both dimensions were classified as it-laggards. to cross-validate the classification of each department, a hierarchical discriminant function analysis was conducted to assess prediction of membership into each of the four quadrants based on the five it management practice variables. the predictor variables were it planning, it organization, it control, it integration, and it leader’s role. of the three functions calculated, function 1 accounted for 86.3% of the between-group variability and 16 of the 17 cases were predicted as classified by the cumulative mean method described above and used by karimi, somers and gupta (2001), thus providing good support for the cumulative mean method and demonstrating the variability between quadrants of it management practices. a loading matrix of correlations between predictor variables and discriminant functions (table 5) suggests that the primary variable in distinguishing between quadrants is it integration, followed closely by it leader’s role. variable function 1 2 3 it integration .676 .722(*) -.111 it organisation .022 .210 .952(*) it control -.053 -.031 .811(*) it leader's role .671 -.175 .716(*) it planning .152 .403 .483(*) table 5: pooled within-groups correlations between discriminating variables and standardized canonical discriminant functions (*largest absolute correlation between each variable and any discriminant function) the classifications shown in figure 2 suggest that the customer service topology within australian federal government departments is mixed. three departments were classified as it leader organisations.. three were classified as it-enabled customer focus departments because of their above average focus on customers and their below average focus on operations. seven departments fell within the it-enabled operations focus topology. this implies they have an above average focus on operations and a below average focus on customers. the remaining four departments fell into the it laggard category. australasian journal of information systems volume 15 number 1 december 2007 87 figure 2: australian federal government departments by customer service topology on average, table 3 suggests that it’s impact was directed more toward a customer focus (3.63, sd.83) than operations (3.32, sd .81). in addition, departments were more sophisticated in it planning (3.67, sd 1.16) and control (3.67, sd .87), closely followed by it organisation (3.50, sd .87) and less so for it integration (2.78, sd .19). analysing the departmental characteristics by customer topology reveals some differences (see table 5). for example, while all departments had substantial annual budgets, only one company (an it laggard) reported an annual budget of $50m or less. this same observation is true of the it annual budget. similarly, while the it leader is typically one or two levels below the ceo, exceptions were reported in the smaller departments classified within the it-enabled operation or it-laggard topologies. customer topology/characteristic it leader it customer focus it operations focus it laggard annual budget >$100 million 3 2 6 2 $51-100 million 1 1 1 $10-50 million 1 it annual budget >$100 million 2 1 1 $11-100 million 1 2 5 2 $1-10 million 1 2 it’s potential for impact on customer focus (customer focus (cf)) customer service topology it’s potential impact on operations operations focus (of) high high low 3 it enabled operations focus organisations cf = 2.89 (sd .32) of = 4.00 (.46) n = 7 (41%) 1 it-leader organisations cf = 3.83 (sd .58) of = 4.42 (sd .14) n = 3 (18%) 4 itlaggard organisations cf = 2.75 (sd.61) of = 2.69 sd .94) n = 4 (24%) 2 it enabled customer focus organisations cf = 4.58 (sd.14) of = 3.25 (sd .43) n = 3 (18%) australasian journal of information systems volume 15 number 1 december 2007 88 no. employees >2000 3 2 4 1001-2000 1 1 500-1000 1 1 1 <500 1 2 it leader level below ceo 1 level below 2 1 2 1 2 levels below 1 2 3 3 levels below 2 3 it planning 1-5 years 2 1 3 4 6 -10 years 1 1 2 >15 years 1 2 table 5: department characteristics by customer topology analysing it management practices within customer topology quadrants (see table 6), provided some interesting results. for example, on average, it-laggard organisations were uncharacteristically higher on it organisation (4.19, sd .72) and it control (4.32, sd .71) than the other three quadrants. it-enabled operations focus organisations were highest on overall sophistication (4.02, sd .43) and it planning (4.21, sd .59), whereas the it leader’s role (4.92, sd .14) was highest in it-enabled customer focus organisations. surprisingly, it-leader organisations were not highest on any of the dimensions, as would have been expected. customer topology quadrant it-leader (n=3) it-enabled customer focus (n=3) it-enabled operations focus (n=7) it laggard (n=4) variable mean sd mean sd mean sd mean sd sophistication 3.40 .77 3.99 .22 4.02 .43 3.92 .47 it planning 3.67 1.16 4.08 .38 4.21 .59 4.13 .32 it organisation 3.50 .87 3.92 .58 3.86 .68 4.19 .72 it control 3.67 .87 3.94 .68 3.73 .82 4.21 .71 it integration 2.78 .19 4.00 .88 4.29 .30 3.17 1.04 it leader's role 3.75 .43 4.92 .14 4.57 .54 4.13 .25 table 6: variability in it management practices across customer topology quadrants following from the results of the discriminant functional analysis that suggested that it integration and it leader’s role were the two variables that most highly distinguished between the topology quadrants, a multiple analysis of variance (manova) test was conducted to determine if any of the it management practices varied significantly across the four quadrants. the results of the manova are summarised in table 7. australasian journal of information systems volume 15 number 1 december 2007 89 variable type iii sum of squares adjusted r-squared df mean square f sig. eta squared observed power (a) it planning 0.646 0.098 3 0.215 0.526 0.672 0.108 0.129 it organisation 0.819 0.092 3 0.273 0.551 0.657 0.113 0.134 it control 0.722 0.129 3 0.241 0.393 0.760 0.083 0.108 it integration* 6.344 0.435 3 2.115 5.101 0.015 0.541 0.815 it leader's role* 2.571 0.416 3 0.857 4.806 0.018 0.526 0.789 table 7. results of manova of differences in it practices between customer topologies * p < .05 of the five variables tested, it integration (p < .05) and it leader’s role (p < .05) were statistically different between the quadrants. while manova identifies statistically significant differences, it does not identify the between-group significance. to do so, requires the application of post hoc testing. as figure 2 shows, the groupings within the customer service topologies are unequal in size. applying the levene statistic, two of the six items failed to meet the criteria of homogeneity of variance. it integration and it planning variables were determined to be significantly different at the probability level of p<.05. it integration at df 3, p= .049 and it planning at df 3, p=.036. for these variables, to analyse multiple pair-wise comparisons the dunnett t3 post hoc test was used. according to garson (2002), this approach is appropriate in a situation where the assumption of homogeneity of variance and unequal sample sizes have not been met. the dunnett t3 post hoc analysis, showed that the mean for it integration was significantly different in it-enabled operations focus organisations and it leader organisations (p=.000). for the it leader’s role variable a significant mean difference was found between it-enabled customer focus organisations and it-laggard organisations (p=.017). discussion the results show that at the time of data collection only limited progress had been made towards achieving collaborative crm in one large agency of the australian federal government departments. only three of the 17 participant departments were classified as it-leader organisations.. an examination of the demographic results presented in table 3, provides some indication of the profile that might set these departments apart from the remainder of participant departments. it would appear that size may be a factor in promoting the uptake of collaborative stage 3 crm. however, it would appear that some progress is being made by small departments toward crm implementation maturity since 10 departments have moved to stage 2 crm through either an it-enabled customer focus or it-enabled operations focus and only four appear to be itlaggards. a comparison of the percentage of departments in each of the four customer topologies showed mixed results compared to that reported by karimi, et al. (2001). a much smaller percentage of departments in the current research (18%) had progressed to the it-leader category contrasted with 38% of those in the karimi, et al. (2001) sample of u.s. financial firms, while a greater percentage (41% vs. 27%) were classified as it-enabled operations focus organisations. the percentages of it-enabled customer focus (18% vs. 15%) and it-laggard departments (23% vs. 20%) were comparable. these differences may be due to the more recent uptake of crm in public australasian journal of information systems volume 15 number 1 december 2007 90 sector organisations relative to the longer-term uptake in the private sector, but these differences in findings might warrant further qualitative investigation. the literature suggests that stage 3 departments have undertaken the required organisational transformation and acquired new applications to support the change in business approach. while the organisations have the resources (both human and financial) to progress to this stage, it would appear they have also been prepared to face the high risks and costs involved to gain the benefits of increased revenues and profits. at this stage of crm implementation organisations are positioned to engage in participatory interaction with their customers unlike those in stage 1 or stage 2 where interaction is still either on a managerial or a consultative basis. stage 3 departments can potentially act as role models to act as motivators in encouraging the departments not so well placed in relation to crm implementation, to make the necessary organisational transformation to move toward a higher level of crm implementation. the benefit of having established role models in relation to crm implementation implies best practices are in place that can be adopted by less crm advanced departments. the results of this research have established that the bulk of departments within one large australian federal government agency departments (implying it enabled customer focus and it enabled operations focus organisations) are typically focussed on technical accuracy with the aim of gaining benefits in cost savings from a consolidated infrastructure. because these departments tend to still have individual applications based on a large amount of data from disparate systems, they are not yet in a position to attempt the enterprise transformation to move to stage 3 collaborative crm. at stage 2 crm implementation the costs are generally medium to high, while the risks are regarded as medium. while these departments do have a uni-linear information flow, interaction with clients is more consultative that participatory. potentially where this situation exists it can imply a barrier to collaborative crm due to a silo effect where individual departments are unwilling to share information. often this situation is maintained as department managers actively work to keep their power and maintain their status quo. turning to the customer service topology, those departments defined as it laggards focus on operations involving localised departments. this implies that multiple individual applications are used with data limited in scope and volume. based on the research presented in this paper, the departments defined as being it-laggards tend to be smaller in size with relatively lower overall and it budgets. it may well be that limited resources (both human and capital) have precluded any move to a higher stage of crm implementation especially in view of the high risks and costs involved. it management sophistication was tested as a whole and from four perspectives: it planning, it organisation, it control and it integration. manova tests failed to detect a significant difference in it sophistication at both the overall level and the dimension level. the manova test found a statistically significant difference in it integration between two of the four topology groups. the post hoc pair wide comparison revealed this to be between it enabled operations focus and it leader organisations. it enabled operations focus departments are typically large organisations with a heavy focus on administration and the financial aspects of government. on this basis, their highest level of it integration supports the view that substantial operationally focussed practices are embedded in these departments. the manova test also found a statistically significant difference in it leader’s role between itenabled customer focus organisations and it laggards. from the literature, the role of the it leader was identified as crucial for the successful implementation of collaborative crm. accordingly this created an expectation that the results would support this in the classification of it leader within the topologies. australasian journal of information systems volume 15 number 1 december 2007 91 a potential explanation for the difference between it enabled customer focus organisations and it laggard organisations can be due to the size of these departments. it enabled customer focus organisations are mostly large organisations that have high personnel content whether it be dealing with people in society or with their own employees. in contrast, it laggard departments are smaller in size and operations and if it is used it is often only at a rudimentary level. as such, it is reasonable to accept that the role of the it leader would be considerably different between these two topologies. the results of this research showed that generally it laggard organisations reported it leaders 3 levels below the ceo. this supports that the role of the it leader in these organisations is more involved with the management of it rather than providing innovative business solutions based on it. conclusions this research is exploratory and has provided some useful results. an encouraging outcome was that despite the recent uptake of crm in the public sector, some of the australian federal government departments appeared to be it-leaders at the collaborative stage of crm implementation. a further positive finding was that the bulk of the remaining departments were itenabled customer or operations focussed organisations with only a minority being identified as itlaggards. while some statistically significant differences were determined on the basis of the overall data and the customer service topology data, these results also highlighted a major limitation in this research. that is, the problem associated with small group numbers and equality of variance. however, since this research was based on a limited population of a large government agency, at this level of government the situation was unavoidable. potentially, the research presented in this paper could form the basis for future work to determine the status of crm implementation across federal government agencies to increase the research sample. references accenture (2002) egovernment leadershiprealising the vision. available online: http://www.accenture.com. accessed january 26th, 2006. accenture (2003) crm in government: bridging the gaps. available online http://www.accenture.com. accessed january 26th, 2006. accenture (2004) egovernance leadership: high performance, maximum value. available online: http://www.accenture.com. accessed january 26th, 2008. applegate, l., mcfarlan, r., mckenney, j. (1996) corporate information systems management: the issues facing senior executives. chicago. irwin. batista, l. and kawalek, p. (2004) translating customer-focused strategic issues into operational process through crm – a public sector approach. in egov 2004 traunmuller, r. (ed.) springer-verlag, heidelberg. 128-133. australasian journal of information systems volume 15 number 1 december 2007 92 benbasat, r., dexter, a. and mantha, r. (1980) impact of organisational maturity on information skill needs. mis quarterly 4:1 21-34. campbell, d.t. and fiske, d.w., (1959) convergent and discriminant validation by the multitrait, multi-method matrix. psychology bulletin 56:1. 81-150. cenfetelli, r.t., benbasat, i., and al-natour, s. (2005) information technology mediated customer service: a functional perspective, proceedings of the twenty-sixth international conference on information systems, 11-23. chadwick, a. and may, c., (2003) ‘interaction between states and citizens in the age of the internet: “e-government” in the united states, britain and the european union. governance: an international journal of policy, administrations and institutions. 6: 2. 271-300. clarke, e. (2003) managing the transformation to e-government: an australian perspective. thunderbird international business review. 45. 4 377-397. corner, i., hinton, m. and kaye, r. (2003) the evaluation of it systems implementation: risks and relationships for externally developed crm systems. proceedings of the 10th european conference on informatic technology evaluation, madrid 25-26th september. berghout and remenyi (eds.): 153-161. croteau, a. and li, p. (2003) ‘critical factors of crm technological initiatives’. canadian journal of administrative sciences. 20:1, 21-34. dibb, s. and meadows, m. (2001). the application of a relationship marketing perspective in retail banking. the service industries journal. 21:1, 161-194. earl, m. (1989) management strategies for information technology. new jersey. prentice hall. feeny, d., edwards, b. and simpson, k. (1992) understanding the ceo/cio relationship. misq. 16:4. 435-447. fornell, c. and larker, d.f. (1981) evaluating structural equation models with unobservable variables and measurement error, journal of marketing research, 18 (february) 39-50. francis, g. (2004) introduction to spss for windows. new south wales.pearson prentice hall. goodhue, d., wixom, b. and watson, h. (2002). realizing business benefits through crm: hitting the right target in the right way. mis quarterly executive. 2:1. 79-94. karimi, j., gupta, y.p. and somers, t.m. (1996) ‘the congruence between an organisation’s competitive strategy and information technology leader’s rank and role’. journal of management information systems. 13:1. 63-88. karimi, j., somers, t.m. and gupta, y.p. (2001) ‘impact of information technology management practices on customer service’. journal of management information systems. 17:4. 125158. kolsky, e. and keller, b. (2001 e-government and crm:slowly moving forward. available online: http://www.gartmer/com. accessed april 14th, 2002. ling, r. and yen, d. (2001). customer relationship management: an analysis framework and implementation strategies. journal of computer information systems. 41:3. 82-97. australasian journal of information systems volume 15 number 1 december 2007 93 malhotra, y (2000). knowledge management for e-business performance: advancing information strategy to ‘internet time’. information strategy, the executive’s journal. 16:4, 5-16. marche, s. & mcniven, j.d. (2003) e-government and e-governance: the future isn’t what it used to be. e-government and e-governance. 20:1, 74-86. national office for the information economy (noie). (2003). transforming government achievements in e-government. canberra. neuman, w. (2000). social research methods: qualitative and quantitative approaches. needham heights: allyn & bacon. nunnally, j.c. (1993). psychometric theory (3rd edition), new york, ny: mcgraw-hill. price, r. and brodie, r. (2001) transforming a public sector organisation from inside out to outside in. journal of service research. 4.1. 50-59. rigby, d., reichheld, f. and schefter, p. (2002) avoid the four perils of crm. harvard business review. 80:2, 101. sabherwal, r. and kirs, p. (1994) the alignment between organisational critical success factors and information technology capability in academic institutions. decision sciences. 25:2. 301-330. sager, m.t. (1988) ‘competitive information systems in australian retail banking’. information and management. 15:1. 59-76. swift, s. (2002) executive response: crm is changing our eras, the information we require, and our processes… mis quarterly executive. 2:1. 95-96. winer, r. (2001) a framework for customer relationship management. california management review. vol 43. no 4. 89-105. xu, m. and walton, j. (2005) gaining customer knowledge through analytical crm. industrial management & data systems. vol 105. no 7. 955-971. australasian journal of information systems volume 15 number 1 december 2007 94 microsoft word ajisvol10 1_final_edit.doc ajis vol. 10 no. 1 september 2002 50 business on-line? an empirical study of factors leading to the adoption of internet technologies by australian smes. peter slade economics/faculty of business university of the sunshine coast maroochydore dc, qld 4558 email: pslade@usc.edu.au jeanette van akkeren information systems/faculty of business university of the sunshine coast maroochydore dc, qld 4558 email: jvanakke@usc.edu.au abstract e-commerce technologies such as a web site, email and the use of web browsers enable access to large amounts of information, facilitate communication and provide niche companies with an effective mechanism for competing with larger organisations world-wide. however, recent literature has shown australian smes have been slow in the uptake of these technologies. the aim of this research was to determine which factors were important in impacting on small firms’ decision making in respect of information technology and e-commerce adoption. findings indicate that generally, the more a firm was concerned about its competitive position, so such a firm was likely to develop a web site. moreover, the ‘industry and skill demands’ dimension suggested that as the formal education of the owner/manager increased, coupled with the likelihood that the firm was in the transport and storage or communication services industries, and realising the cost of it adoption was in effect an investment, then such a firm would be inclined to develop a web site. firms that were presented with relatively geographically dispersed markets, and realising it was necessary to go through the time consuming process of adopting various it to reach these markets, were more likely to adopt web sites. lastly, owners/managers who reported their knowledge of business uses of computers was poor and who were likely to ask for support in installing and utilizing it were also more likely to use web sites than those who did not share these characteristics. introduction communications technology through the internet is recognised as a recent and important tool in providing operational efficiencies for small to medium sized enterprises (smes). e-commerce technologies such as a web site, email and the use of web browsers enable access to large amounts of information, facilitate communication and provide niche companies with an effective mechanism for competing with larger organisations world-wide. however, recent literature has shown australian smes have been slow in the uptake of these technologies (yellow pages small business index 1999, van akkeren and cavaye 1999, poon and swatman 1999, lawrence 1997). research into the adoption of internet technologies, particularly basic ecommerce ones such as a web site and email, has been largely ignored. australia’s slow adoption rate of these technologies, particularly when compared with countries with much higher adoption rates, underlie the importance of this research. this paper is structured as follows: firstly, literature is outlined on general it adoption, and then on internet technology and specifically web site adoption by small business. the research questions are identified and the method used to investigate the problem is then discussed. detailed analysis of findings from surveyed firms is presented and conclusions outlined. the purpose of the paper is to build a comprehensive framework of the factors associated with the adoption of internet technologies and specifically web sites by smes. general it adoption researchers have identified a variety of factors that affect technology adoption in small business and use differing models in determining reasons why firms adopt (or do not adopt) it. much of the earlier literature is based on roger’s (1995) diffusion of innovation theory (doi), which highlights the characteristics pertinent to the decision to adopt and diffuse information technology (it). it is these characteristics that help persuade potential adopters to embrace or reject an innovation. the five characteristics of an innovation identified by rogers (1995 pp. 15-16) are: relative advantage the degree to which an innovation is perceived as better than the idea it supersedes; compatibility the degree to which an innovation is perceived as being consistent with the existing values, past experiences, and the needs of potential adopters; complexity the degree to which an innovation is perceived as difficult to understand and use; trialability the degree to which an innovation may be experimented with on a limited basis; observability the degree to which the results of an innovation are visible to others. ajis vol. 10 no. 1 september 2002 51 premkumar et al (1994) suggest that the chance of a firm adopting an innovation and the extent of that adoption is largely dependent on the characteristics of the innovation as perceived by the adopting firm. coombs et al (1987) support this by suggesting that the risk associated with the innovation decreases as more firms adopt and explains diffusion of innovation over time follows an s-curve shape. hence, adoption can be graphically represented in the form of a logistic diffusion curve as shown in figure 1 where the rate of diffusion first increases up to an inflection point, and then decreases (coombes et al 1987). figure 1: logistic diffusion curve it would appear the characteristics of information technologies will impact on the adoption and diffusion of it. this is highlighted by the fact that in the past, australian smes have been entrepreneurial in adopting technologies and could therefore be placed in the upper portion of the logistic diffusion curve (yellow pages australia, 1997). however, more recent technologies such as the internet, and e-commerce in particular, have not been adopted and diffused at the same rate, particularly when compared to countries such as japan, singapore, the united kingdom and united state of america (forrester research 1997, van akkeren and cavaye 1999). doi theory has been challenged in recent years, particularly in relation to the fact that is/it innovations may change during the adoption/diffusion process. more recent studies show that doi theory only partially applies to it and software adoption/diffusion and it is now recognised that other approaches require further consideration (larsen 1997). an example is organisational psychology, which includes factors such as adoption beliefs, cognitive thinking and organisational/individual behaviours and beliefs. the next section of literature addresses these alternative approaches to is/it innovation and diffusion. behavioural and attitudinal studies on it adoption is adoption researchers have focussed on many different factors thought to influence adoption. the main constructs in many of the theories stem from roger’s (1995) work and are based on individual behavioural characteristics such as perception and attitude. the theory of planned behaviour (tpb) model by ajzen (1991) discusses behaviour as a direct function of behavioural intention and perceived behavioural control. for small business owner/managers, who are responsible for major decisions on it adoption and capital expenditure, their perceptions and attitudes towards technology could certainly influence their decision. taylor and todd (1995) extend ajzen’s original theory to the decomposed tpb, particularly in relation to the perceived characteristics of an innovation, relative advantage, and compatibility of the innovation to the adopter’s existing values, previous experiences and current needs. the technology acceptance model suggested by davis (1989) discusses it adoption, implementation and diffusion in terms of perceived use and perceived ease of use. for a small business owner/manager, their earlier experiences with it and their level of computer expertise will impact on their perceptions of new innovations. gefen and straub (1997) support this with findings that earlier experiences with it influenced an sme owner/manager’s decision to adopt new innovations. ajis vol. 10 no. 1 september 2002 52 constructs used in the above models generally focussed on perceptions, attitudes, beliefs and intentions of the decision-maker, combining innovation literature with psychological considerations. however, factors both within and outside the organisation have been shown in later studies to impact on it adoption and diffusion. an example of this is a study by thong (1999) which found the innovativeness and level of is knowledge of the owner/manager, is characteristics (relative advantage, compatibility and complexity of is), and organisational characteristics (size, level of employee is knowledge) will impact on the adoption of is. in another study, fink (1997) depicts three main groupings that impact on the rate of it adoption: it benefits, organisational culture, it availability and in-house it expertise; internal resources, it selection and it implementation; and external selection, it implementation. another factor that has been shown to impact on the rate of small business technology adoption is the industry group of the organisation, whether the organisation is franchised or independent, and the technology inherent in distribution channels used by the organisation. treadgold (1990) found small retail businesses with high it adoption rates had been influenced by the distribution channel, trade associations, wholesalers, and voluntary groups and franchisers that encouraged it adoption. internet technologies and small business adoption large organisations generally have the funding, resources and expertise to pursue technological innovation and adoption. in contrast, small businesses have to rely on a smaller number of employees and are further constrained in terms of time and capital availability. more recent literature supports this with findings that the characteristics of the firm (size, sector, status), and economic factors (cost, pressure from suppliers/customers/competitors) influence e-commerce adoption by smes. piovesana and raush (1998) found the cost of implementation inhibits adoption of e-commerce technologies. fielding (1996) adds that small businesses find internet technologies too complex, difficult to implement and expensive. similarly, lawrence (1997), in a study of australian smes found limited resources, a low level of existing hardware, the need for immediate returns on investment, resistance to change and lack of available information had inhibited internet technology adoption. further research by mcgowan and madey (1998) and sillence et al (1998) suggests the size of the small business, resource availability, the level of technological knowledge and pressure from external bodies influence internet technology adoption. van akkeren and cavaye (1999), in a review of the literature focussing on the adoption and use of information technologies (it) by small and medium sized enterprises argued that small firms are somewhat constrained by their size when considering the adoption and use of various it based business solutions. certainly, the expenditure of monies on technologies may seem foreign and outside the experience of small business owners would seem to be doubtful. against this, given some knowledge of computing and business based it, many owners and managers would be in a position to understand the possible scale economies that might reside in it. moreover, through time, the costs of it have been falling, and there have been more and more potential and actual uses of the technology in business settings generally. seemingly, an important aspect of the adoption of it by small businesses must lie in the knowledge and understanding of it potentialities by decision makers in the sector. iacovou et al (1995) identified the following factors as having an impact on it adoption by businesses; organisational readiness, financial and technological resources of the firm, and the level of external pressure adopt, both from competitors and trading partners. kirby and turner (1993) argued computer literacy of small business owners, the extent of knowledge of it benefits, and the extent of dependency of customers and suppliers had significant impacts on it adoption. julien and raymond (1994) argued that several factors had an influence on technology adoption by small businesses. they were the structural sophistication of the firm in terms of centralisation and complexity, the level of assertiveness, rationality and interaction in business decision processes, as well as the size and status (whether independent or affiliated) of the firm, and the sector in which it operates. thong and yap (1995) posit two main factors as important in influencing it adoption by small and medium sized businesses (sme’s). they were the individual characteristics of the owner or manager in respect of innovativeness, attitude towards it adoption, as well as it knowledge, and second, organisational characteristics, being business size, the competitiveness of the environment and information intensity. harrison et al (1997), in attempting to predict a small businesses owner’s/manager’s decision to adopt it, argued the following were important owner/manager characteristics that had a bearing on the decision making process. first, the decision processes of the key decision maker, which were largely attitudinally based, then subjective norm, the level of perceived control in respect of it adoption, the firm’s size, the sector it operated in and its status, and last, a set individual characteristics. van akkeren and cavaye (1999) constructed a typology of factors thought to bear on it adoption rates, and divided them into owner/manager characteristics and firm characteristics. additionally, they incorporated return ajis vol. 10 no. 1 september 2002 53 on investment (roi) as a single and important factor in determining it adoption in sme’s. the model is presented below. figure 2: it adoption model owner/manager characteristics • perceived benefits • computer literacy • assertiveness • subjective norm return on investment it adoption in sme firm characteristics • organizational readiness • external pressure to adopt • customer supplier dependency • structural sophistication of the firm • sector • status they argue that their model could be a take off point for further and more empirically based research. the question arises as to whether or not the factors outlined would have a significant effect on it adoption by themselves, or whether they tend to act in conjunction with other factors, in groups. this implies the possibility of conjoint action, and that taken individually the variables might have little effect. however, when two or more of them are present, then action is likely to occur. in other words, a situation of necessary and sufficient conditions would have to exist before anything were to happen. certainly, many of the models presented in this paper suggest some of the factors might act conjointly. for example, iacovou (1995), in identifying organisational readiness to adopt and external pressure to adopt as factors impelling adoption of it, imply various factors tend to act conjointly. kirby and turner (1993) argued structural sophistication of the firm, the nature of business decision making, and the nature of the industry that the firm is operating in are important in leading firms to adopt it. all these are constituted as a number of factors, and not isolated single variables. thus there is a strong possibility that conjoint effects offer more explanation than isolated effects operating alone. this conjointness tends to suggest the drivers of it adoption are probably more complex and difficult to model than what at first meets the eye. the aim of this research was therefore to determine which factors were important in impacting on small firms’ decision making in respect of information technology, particularly internet technology adoption. further, it was decided to investigate the possibility as to whether or not the factors might act in a conjoint fashion. following this the relative magnitude of the factors in determining the rate of adoption of some forms of it was carried out. from the literature review, a number of propositions were devised in respect of the factors impacting on small firms’ decision making when considering it and internet adoption. the propositions below outline each of the issues addressed in this research and the literature from which they originate. ajis vol. 10 no. 1 september 2002 54 propositions table 1: propositions and their source proposition reference 1. small firms, in terms of their numbers of employees and the levels of assets employed in the business, are less likely to adopt it such as websites, as compared with larger firms. piovesana and raush (1998), harrison et al 1997, thong and yap 1995, iavovou et al 1995, fielding 1996, mcgowan and madey (1998), sillence et al (1998) 2. many of the elements that are important in improving the competitive advantage of a firm can be enhanced by adopting or improving it. mcgowan and madey (1998), sillence et al (1998), kirby and turner 1993, sillence et al 1998, gefen and straub 1997, thong 1999, poon and swatman 1999. 3. smaller firms, lacking human, financial and physical resources, are impeded from adopting it. iacovou et al (1995), julien and raymond (1994), harrison et al 1997, piovesana and raush 1998, fielding 1996. 4. it is likely that firms operating in more technologically demanding environments impose a need for a higher level of formal education on their owners/managers. thong and yap (1995), kirby and turner 1993, thong 1999. 5. those firms with geographically spread markets (both in terms of suppliers and customers) are more likely to adopt various it’s to cover this space/time dimension, than those firms that buy and sell locally. thong and tap 1995, treadgold 1990, kirby and turner 1993, harrison et al 1997. the first proposition is congruent with findings from previous literature that the cost of implementation and the financial resources of the firm, along with the size of the firm (number of employees) will impact on adoption (harrison et al 1997, thong and yap 1995, iacovou et al 1995, fielding 1996). proposition 2 posits that firms can be impelled to adopt it as a result of market imperatives. small firms are impelled to adopt it to enhance or maintain their competitive situation, even though the owner/manager may not be particularly computer literate. such owner/managers would be forced to rely on the capability of sellers and installers of it. competitors, in adopting it, can threaten market share, customers come to expect levels of technological sophistication, and suppliers often force a need for the adoption of it (kirby and turner 1993, thong 1999, gefen and straub 1997). there is a necessity to maintain and improve roi and the benefits of it adoption can include the gaining of competitive advantage. mcgowan and madey (1998), poon and swatman (1999) and sillence et al (1998) argue that small businesses can gain a competitive advantage by the adoption of it, and external bodies play a role in its introduction. thus, even owners and mangers with no knowledge of computing are forced to adopt the it solution. the third proposition relates to the size of the firm in terms of financial and physical assets (piovesana and raush 1998, fielding 1996, harrison et al 1997). small firms are generally simpler structures than larger ones, and sole traders are less likely to adopt it than larger public companies with complex structures and reporting arrangements. iacovou et al (1995) point to organisational readiness, being financial and technological resources of the firm, as being important in the adopt/non-adopt decision. julien and raymond (1994) suggested the structural sophistication of the firm, as well as its size and status and the sector in which it operates, as being important in determining the likelihood of it adoption. the forth proposition posits that the formal education and it literacy of the owner/manager may or may not impact on adoption. owner/manager characteristics in terms of innovativeness, it adoption and it knowledge are listed as important determinants of it adoption (kirby and turner 1993, thong 1999). moreover, the businesses’ size, environment and information intensity are important in the it adoption decision (thong and yap 1995). the final proposition suggests that those firms with geographically spread markets (both in terms of suppliers and customers) are more likely to adopt various it’s to cover this space/time dimension, than those firms that buy and sell locally (thong and yap 1995, harrison et al 1997, treadgold 1990). the reverse might hold as well. this is because some small firms, freed up from geographical constraints by it would be more likely to locate away from areas of high rents and congested infrastructure represented by the larger urban concentrations. ajis vol. 10 no. 1 september 2002 55 research design data collection it was decided that the use of a mail out questionnaire was appropriate to address the research aims and propositions. much of the previous work reviewed to derive the propositions tended to be of the exploratory case study type. (see for example, fink (1997), gefen and straub (1997), harrison, mykytn and rienenschneider (1997), iacovou, benbasat and dexter (1995), julien and raymond (1994), lawrence (1997), poon and swatman (1999), van akkeren and cavaye (1999)) one of the aims of the current work was to expand on the previous exploratory research, and to try to understand how previously derived variables might be at work in a broad or wider context, than in just a few case studies. a random sample of businesses on the sunshine coast (queensland) was drawn from the “yellow pages australia on disk” database of australian businesses. a mail questionnaire was sent to 1750 businesses requesting information on their levels of it adoption and investment, e-commerce use, industry type, and so on. thus in all cases the survey relied upon self reporting by the respondents. three hundred and thirty nine (339) completed questionnaires were returned and two hundred and fifteen were sent back as address unknown. thus, the useable response rate was 19.37%, which is consistent with studies involving small business (see for example perry, meredith and cunnington (1988); callahan and cassar (1995)). to address the possibility of non response bias, the ‘final wave’ of respondents (to surrogate late respondents for non respondents) was compared with the initial groups of respondents within each industry. chi square and t test statistics revealed no significant differences at the 5% level for any of the variables in the study. definition and measurement of variables. there is a sense in which this work is somewhat tentative or exploratory. the variables were deduced from the literature and it was recognised that some of them might act in conjunction with one another, or that their full effect could only be observed in the presence of others. thus, the variables defined and selected in this study were recognised as being equivocal and open to challenge. moreover, given the large number selected (as shown in table 2), there is the possibility of a violation of ockham’s razor and a lack of parsimony. however, and once again, given the exploratory nature of this work, variable selection proceeded as follows. the questionnaire contained questions that investigated a number of variables taken from the literature review. those focussed on for the research were: factor analysis (principal components analysis). consideration of the literature dealing with the factors that impinge upon firms’ decisions to adopt or not adopt various forms of itt has revealed that there is a tendency for more than one factor to be important in the decision making process. see for example iacovou et al (1995), kirby and turner (1993), julien and raymoond (1994), thong and yap (1995), harrison et al (1997). these factors might be operating simultaneously or in a conjoint fashion. it is also conceivable that many of the factors might not be directly observable, and appear to be contructs derived from other, more directly observable variables. this is the case reported by julien and raymond (1994), where they suggest that structural sophistication of the firm in terms of centralisation and complexity, the level of assertiveness, rationality and interaction in business decision processes, as well as size and status of the firm, and the sector in which it operates. all these are held to impact n the decision making of the firm in respect of it adoption. ajis vol. 10 no. 1 september 2002 56 table 2: definitions of variables and their source. variable name definition reference ed the formal educational attainment level of the owner/manager. sillence et al 1998, thong and yap 1995 ind the industry the firm operated in, using the standard anzsic classification. julien and raymond 1994, harrison et al 1997 mkt the geographical spread of the firm’s market, for example, in the local area, the sunshine coast, south east queensland, all queensland, australia wide, exporting, etc. thong and yap, 1995, treadgold 1990 emps the number of employees in the business. harrison et al 1997, fielding 1996, mcgowan and maddy 1998 org the form of business ownership, from sole trader, through to public companies. julien and raymond 1994, harrison et al 1997 assets approximate level of business assets used in the business in the current year. van akkeren and cavaye 1999. know owner/manager’s assessment of their knowledge of business uses of computers. kirby and turner 1993, thong 1999, van akkeren and cavaye 1999 use the use to which various forms of electronic information technology in the business were put. julien and raymond 1994, iavovou et al 1995 bin* derived from use; whether or not the business had a web site. cost self reported assessment as to whether or not implementing internet technologies are (or would be) cost prohibitive. piovesana and raush 1998, fielding 1996, lawrence 1997 tcons the extent to which installing and implementing internet technologies is (or would be) time consuming. fielding 1996 tbens whether or not the time spent implementing internet technologies would be (or has been) beneficial. harrison et al 1997, iavovou et al 1995 roi the extent to which internet technologies have (or would be) an impact on a firm’s return on investment. lawrence 1997 will the extent to which the firm’s employees are ready to switch to and operate internet technologies. mcgowan and maddy 1998, sillence et al 1998 stech the extent to which other firms in the same industry having internet technologies lead to this firm adopting internet technologies. harrison et al 1997, iacovou et al 1995 cust the extent to which the firm’s customers would expect adoption of internet technologies. sillence et al 1998, kirby and turner 1993 comp the extent to which internet technologies are perceived to improve a firm’s competitiveness. kirby and turner 1993, taylor and todd 1995 pimp if a non-adopter of internet technologies, an assessment of the extent to which they could improve competitiveness. thong and yap 1995, taylor and todd 1995, iacovou et al 1995 hsell the extent to which installers and sellers of internet technologies have aided businesses to adopt the technologies. bens types of benefits arising from adopting internet technologies. kirby and turner 1993, fink 1997 * for the purposes of stage two of this work, a logistic regression was run using bin as the dependent variable. hence, use was recoded to bin, whereby a firm was assigned 1 if it adopted a web site, and 0 if it didn’t. thus a problem arises as to how best to analyse and represent a relatively large number of variables or factors that might be acting in some conjoint fashion, or which might essentially be constructs which are not directly observable or are made up of other non directly observable variables. factor analysis can be used to identify a relatively small number of factors among sets of many interrelated variables. further, the technique helps identify underlying, not directly observable constructs which might be operating within a milieux of many, closely related variables. thus the basic assumption of factor analysis is that ajis vol. 10 no. 1 september 2002 57 underlying dimensions or factors can be used to explain complex phenomena. accordingly, eighteen of the variables in the database were subjected to the process of factor analysis, using the principal components method. principal components was used because linear combinations of the observed variables were desired. given that the mathematical model for factor analysis is somewhat similar to a multiple regression equation, each variable (or extracted component) can be expressed as a linear combination of the original variables. thus, the groups of variables that cluster together constitute the factors. under most circumstances the factors (or components) that are found useful in characterising a set of variables are not known in advance, but are determined by the factor analysis. a total of six components were found to have eigenvalues greater than one. these six components between them explained 59 percent of the variance of the original eighteen variables. consideration of the component to variable correlation matrix (table 3) shows the following make up of components. cronbach’s coefficient alphas for reliability were estimated for each of the components to ascertain the extent to which variables making up each component shared a common core and the extent to which items in the questionnaire were related to each other. cronbach’s alpha is based on the average inter item correlation. table 4 shows that somewhat low reliability held for items which composed component 3 (ed, ind cost). component 1 shows a marginal result for reliability, while the rest of the components were satisfactory or better. given the exploratory nature of this research and bearing in mind that the main aim of this factor analysis is to reduce the number of variables to gain parsimony, the low cronbach’s alphas for two of the components was not considered to be of overwhelming importance. further, reliability tests are of more importance in research that aims, for example, to construct robust questionnaires through frequent sampling and testing, and for instruments for further and wider application. examples might include various psychological tests, market and product satisfaction surveys and so on. ajis vol. 10 no. 1 september 2002 58 table 3: correlation matrix component variables references correlation coefficients 1 comp kirby and turner 1993, taylor and todd 1995 .727 roi lawrence 1997 .594 pimp thong and yap 1995, taylor and todd 1995, iacovou et al 1995 -.592 cust sillence et al 1998, kirby and turner 1993 .539 stech harrison et al 1997, iacovou et al 1995 .507 tbens harrison et al 1997, iavovou et al 1995 .503 bens kirby and turner 1993, fink 1997 .496 2 emps harrison et al 1997, fielding 1996, mcgowan and maddy 1998 .635 assets van akkeren and cavaye 1999. .626 org julien and raymond 1994, harrison et al 1997 .543 3 ed sillence et al 1998, thong and yap 1995 .754 ind julien and raymond 1994, harrison et al 1997 .582 cost piovesana and raush 1998, fielding 1996, lawrence 1997 -.502 4 tcons fielding 1996 .595 mkt thong and yap, 1995, treadgold 1990 .494 5 will mcgowan and maddy 1998, sillence et al 1998 .613 6 know kirby and turner 1993, thong 1999, van akkeren and cavaye 1999 -.516 hsell .415 ajis vol. 10 no. 1 september 2002 59 table 4: cronbach’s alpha reliability component variables cronbach’s alpha 1, competitive advantage comp roi pimp cust stech tbens bens .5269 2, organisation and structure emps assets org .7867 3, industry and skill demands ed ind cost .3207 4, geographical and temporal demands tcons mkt .5770 5, staff capacity will n/a 6, support know hsell .7734 the variable ind appeared in component three. it comprised the respondents’ answers to the industry they considered they were operating in (the industries conformed to the anzsic). it was necessary to isolate the industries in which firms were more likely to adopt web sites (as a proxy for a given level of it adoption), from those industries where adoption was not common. accordingly, a wilcoxon-mann-whitney test was used to ascertain whether or not firms that adopted technology tended to be in industries that were different from those which did not adopt it. the results are presented in table 5. table 5: wilcoxon-mann-whitney test variable; ind by bin mann-whitney u 9466 wilcoxon w 31832 z -2.496 significance .01 perusal of the results of adopting firms by industry as against non-adopting firms by industry showed that adopters were more likely to be in either the transport and storage or communication services industries than non-adopters. non-adopters tended to be in retail trade, construction and accommodation, cafes and restaurants. thus there was a significant difference between firms that adopted web sites (specifically) and those that did not, in terms of the industry they were likely to be in. consideration of the dimensionality of each of the components proceeded next. component 1, comprising comp, roi, pimp, cust, stech, tbens and bens would suggest a strong focus on a firm’s competitiveness. thus, it appeared that it improved competitiveness, had a positive impact on roi, and its adoption, while time consuming would be worth it. moreover, firms’ customers expect it adoption, other firms in the same industry were adopting, and there was a need to emulate them to maintain competitiveness. firms that had not moved into it solutions still said that adopting various technologies would improve their competitiveness. thus, component 1 was labeled ‘competitive advantage’. the variables that constitute this component and its naming as competitive advantage are to an extent, a reflection of the literature and previous research. kirby and turner (1993) pointed to the importance of the extent of dependence on customers and suppliers, coupled with the nature of the industry the firm operates in as being important in the it adoption decision. thus, in a highly competitive environment and industry, and given low levels of customer and supplier dependency, it adoption is made much more likely. it is in an attempt to maintain or gain a competitive advantage that a firm is much more likely to adopt some form of it. iacovou et al (1995) pointed to the role of the level of external pressure to adopt it, from both competition and trading partners. sillence et al (1998) have also indicated that the role of pressure from external bodies acts as a spur oto it adoption. lastly, taylor and todd (1995) have pointed to the importance of gaining and maintaining a relative advantage through it adoption. ajis vol. 10 no. 1 september 2002 60 component 2, comprising emps, assets and org, suggested that as the level of assets used in a firm increased, so too did the number of employees, and that structure of ownership became more complex, and the more likely a firm was to adopt various forms of it. this component was named ‘organisation and structure’. this second component is in some ways reflective of the argument made by harrison et al (1997), who suggested that the size of the firm plays a role in it adoption. component 3, which was made up of ed, ind and cost, suggested that as the level of education of the owner/manager increased, coupled with the likelihood of the firm being in either the transport and storage or communication services industries, the more likely they were to be clustered together. moreover, such firms were likely to report that purchasing it was not seen as a prohibitive cost. this component was named ‘industry and skill demands’. thong and yap (1995) have argued that the individual characteristics of the firm’s owner in respect of innovativeness, attitude towards it adoption and it knowledge are important in the it decision process. component 4, which comprised tcons and mkt suggested that as the geographical spread of a firm’s market widens, so too the necessity to adopt it increases. however, firms also recognize that implementing it can be a time consuming process, but are prepared to spend time acquiring the technologies, to cover the broader market reach. this component was named ‘geographical and temporal demands’. component 5 was made up of one variable only, will, which was an estimate of the capacity of employees to adjust to it. it was named ‘staff capacity’. thong and yap (1995) have argued for the importance of the geographical spread of the market as being important in the adoption decision. component 6, made up of know and hsell, indicated that as owners or managers rated their knowledge of business uses of computers as poor, the more likely they were to find the sellers and installers of it as helpful. this component was named ‘support’. logistic regression. having derived components from the variables to manageable proportions by decreasing their number through principal components analysis (and thereby gaining a greater degree of parsimony), the next step was to examine the nature of their impacts upon the decision to either adopt or not adopt a web site. the dependent variable was thus binary. firms either adopted or did not adopt a web site. the independent variables, in the form of the components, were continuous. under these circumstances, the most appropriate statistical technique to model the phenomena was logistic regression analysis. to employ logistic regression the variable use was reclassified to bin whereby any firm that had implemented a web site was assigned the value of 1, whilst the rest, being non-adopters, were assigned a value of 0. following this, the six components (treated as independent variables) were logistically regressed against bin, being the dependent binary variable. descriptive statistics for the predictor variables are presented in table 6. table 6: descriptive statistics for components. component mean, non adopters mean, adopters student’s t statistic significance competitive advantage .196 .205 -2.347 .02 organization and structure 9.29*10 -02 9.72*10 -02 -1.097 .275 industry and skill demands .179 .187 -2.138 .034 geographical and temporal demands .336 .351 -4.2 .000 staff capacity 7.42*10 -02 7.76*10 -02 -0.874 .383 support .276 .289 -3.379 .001 the means for ‘competitive advantage’, ‘industry and skill demands’, ‘geographical and temporal demands’, and ‘support’ offered strong evidence that their means for adoption as against non adoption, were different from one another. given that the components derived from the principal components analysis are orthogonal and a result of varimax rotation, there would not be any multicollinearity present. the results of the logistic regression are presented in table 7. ajis vol. 10 no. 1 september 2002 61 table 7: diagnostic output for logistic regression 1. variable coefficient (b) wald statistic significance r exp (b) competitive advantage .5461 6.1096 .0134 .1493 1.7265 organization and structure .2400 1.3943 .2377 .0000 1.2713 industry and skill demands .4680 4.9477 .0261 .1265 1.5968 geographical and temporal demands .9469 15.8853 .0001 .2745 2.5777 staff capacity .2565 1.1193 .2901 .0000 1.2924 support .7647 12.2058 .0005 .2352 2.1484 constant -.0449 .0467 .8289 the wald statistic and its significance level for the variables, ‘organization and structure’, and ‘staff capacity’, showed that these two had coefficients that were not significantly different from zero. this is to say that the null hypothesis, that the independent variables were not significantly related with the presence or absence of a web site, could not be rejected. thus it was decided to drop ‘organisation and structure’ and ‘staff capacity’ from the model. by rerunning the logistic regression model without these variables, greater overall significance could be obtained, along with a substantial gain in model parsimony. in other words, more could be said with less, and more significant results would be obtained. accordingly they were dropped from the model, and the logistic regression was rerun without them. table 8 shows the results of the second logistic regression. table 8: diagnostic output for logistic regression 2. variable coefficient (b) wald statistic significance r exp (b) competitive advantage .522 5.927 .0149 .146 1.6859 industry and skill demands .460 4.770 .0290 .1226 1.5841 geographical and temporal demands .923 15.851 .0001 .2741 2.5158 support .747 11.942 .0005 .2323 2.1106 constant -.0463 .051 .8212 classification table predicted group actual group adopters non-adopters percent correct non-adopters 49 19 72.06% adopters 17 48 73.85% overall classified correctly 72.93% as things transpired, there was no improvement in significance levels for the remaining variables, and they were very good in the first place. further, greater model parsimony was obtained, because fewer independent variables explained the action of the dependent variable. using the wald statistic, the variables ‘geographical and temporal demands’, and ‘support’ were shown to be statistically significant reliable predictors of web site technology adoption by firms. the variables ‘competitive advantage’ and ‘industry and skill demands’, whilst not as statistically significant as the former two, are still useful as predictors. a test of the full model with all predictors against a constant only model was statistically reliable, χ2 (4, n=133) = 41.268, p <.001. this indicates that the predictors, as a set, reliably distinguished between businesses that adopted web sites and those that did not. in other words, the model will predict adoption outcomes more accurately than random guessing 99.9 percent of the time. the model chi square tests the null hypothesis that the coefficients for all the variables in the model (except the constant) are zero. prediction success was such that 74 percent of firms that were predicted to have web sites in fact did so, whereas 72 percent of those firms predicted as not having web sites, did not have one. the overall prediction success was 73 percent. in logistic regression, the regression coefficients work in the following way. consider the coefficient for the variable ‘geographical and temporal demands’, which is .923. when the variable changes from 0 to 1, and the other independent variables remain the same, the log of the odds of the firm adopting a web site increase by .923. further interpretation can be gained by considering the column headed exp (b). in this case, when the variable ‘geographical and temporal demands’ changes from 0 to 1, the odds of the firm adopting a web site ajis vol. 10 no. 1 september 2002 62 are increased by 2.5158. in other words by increasing the value of the variable by one unit, the log odds are increased by .9226, and by increasing the value of the variable from 0 to 1, the odds of adoption of a web site changed by a level of 2.5158. discussion it is possible to argue that any number of characteristics of small firms could have an impact on whether or not such firms are likely (or not) to adopt it of various types, and to build web sites specifically. much of the literature to date has been of a speculative nature, with little empirical work being done to identify and model variables that do have a role to play in a small firm’s decision making. in this work, many variables were considered for identification and modeling, and few of them were found to be important in having an effect by themselves. moreover, it was found that when combined with other variables they offered some explanatory power in respect of it and web site adoption. generally speaking, the presence or absence of one variable by itself was relatively unimportant. rather, it was only in the presence of other variables that much effect was observed. it seemed as though a variety of necessary and sufficient conditions was operable. this is to say that any one given variable might seem necessary for some effects to be observed, but the presence of the variable by itself was not sufficient to observe any effects. several variables were necessary and sufficient to observe some effects. given the possibility of a conjoint relationship between the variables, many of the propositions were shown to be significant in the owner/managers decision to adopt/not adopt web sites. however, single variables such as the size of the firm (proposition 1), were not significant, and only became a useful predictor in the presence of other variables: level of assets used in the business, and the form of business ownership (component 2 – organisation and structure). proposition 2, although not as statistically significant as other variables, was shown as a useful predictor in the adoption of web sites, particularly when combining competitive advantage with roi, and external pressure to adopt from suppliers and customers (component 1 – competitive advantage). in general terms, the more a firm was concerned about its competitive position, so such a firm was likely to develop a web site which supports proposition 2 based on previous studies undertaken by mcgown and maddy 1998, poon and swatman 1999, sillence et al 1998 and thong 1999. the third and fifth propositions matched component 4 (geographical and temporal demands) and were shown to be statistically significant predictors of web site adoption. in particular, the geographical spread of the firm’s market will impact on an owner/manger’s decision to adopt/not adopt a web site. this study therefore supports earlier findings by thong and yap 1995, treadgold 1990, and harrison et al 1997 that adoption of internet technologies is facilitated by the geographic distance of markets in terms of suppliers (business-to-business) and customers (business-to-consumer). added to this variable however, is that firms also recognise that implementing it can be a time consuming process, and they may need to invest to cover the broader market reach, supporting earlier findings by iacovou et al 1995, fielding 1996, julien and raymond 1994 and piovesana and raush 1998. similarly, proposition 4 (component 6 – support) offers strong evidence that the educational level of the owner/manger, the environment the firm operates in, and the level of information intensity will play a significant role on the owner/managers decision to adopt/not adopt it and internet technologies, supporting previous studies undertaken by thong and yap 1995, thong 1999 and kirby and turner 1993. the findings from this research are not always inconsistent with the ideas put forward by such writers as iacovou et al (1995), kirby and turner (1993), julien and raymond (1994) thong and yap (1995) and iacovou et al (1995) who argued that external pressure to adopt was important in determining it adoption. this dimension is similar to ‘competitive advantage’ and ‘industry and skill demands’. kirby and turner (1993) and julien and raymond (199 4) argue that the structural sophistication of the firm in terms of centralization and complexity as well as the size, sector and status of the firm are important determinants. these dimensions are similar to the variables ‘industry and skill demands’ and ‘geographical and temporal demands’, which were formulated in the principal components analysis in this research. lastly, julien and raymond (1994) suggested that certain organizational characteristics such as business size, competitiveness of the environment and information intensity have a role to play in the adoption of it. these findings are not inconsistent with the four variables used in the second logistic regression estimated in this research. the following points are made in respect of the logistic regression itself. generally, the more a firm was concerned about its competitive position (both in respect of the demands from its customers and as competitors moved towards it solutions, as well as having a recognition that there was the prospect of a positive roi), so such a firm was likely to develop a web site. moreover, the ‘industry and skill demands’ dimension suggested that as the formal education of the owner/manager increased, coupled with the likelihood that the firm was in the transport and storage or communication services industries, and realising that the cost of it adoption was in effect an investment, then such a firm would be inclined to develop a web site. firms that were presented with relatively geographically dispersed markets, and realising that it was necessary to go through the time ajis vol. 10 no. 1 september 2002 63 consuming process of adopting various it to reach these markets, were more likely to adopt web sites than those that did not share these characteristics. lastly, owners/managers who reported that their knowledge of business uses of computers was poor and who were likely to ask for support in installing and utilizing it were also more likely to use web sites than those who did not share these characteristics. the logit model was robust and was a clear distinguisher between adopters and non-adopters of web sites using the variables derived from the principal component analysis. references ajzen, i. (1991) the theory of planned behaviour, organisational behavior and human decision processes, pp. 179-211 in taylor, s. and todd, p.a. (1995) understanding information technology usage: a test of competing models, information systems research, june, vol. 6, no. 2, pp.144 –176. callahan, t. j. and cassar, m. d. (1995), small business owners’ assessment of their abilities to perform and interpret formal market studies, journal of small business management, october, vol. 33, no. 4, pp 1-9. coombs, r., saviotti, p., and walsh, v. (1987) economics and technological change, macmillan education limited, london. davis, f. (1989) perceived usefulness, perceived ease of use and user acceptance of information technology, mis quarterly, september, vol. 13, no. 3, pp. 319-340. fielding, j. (1996) getting out front with edi, inform, october, vol. 10, no. 9, pp. 12-18. fink, d. (1997) improving the competitiveness of smes through successful information technology adoption, conference proceedings from seaanz annual conference, september 1997. forrester research incorporated, forrester ranks world economics for ecommerce, 30 april 1997, cambridge, ma, [online] <http://www.forrester.com.>, accessed august 1998. gefen, d., and straub, d.w. (1997) gender differences in the perception and use of e-mail: an extension to the technology acceptance model, mis quarterly, december, vol.21, no. 4, pp. 389-400. harrison, d.a., mykytyn, p.p. .jr and rienenschneider, c.k. (1997) executive decisions about it adoption in small business: theory and empirical tests, information systems research, a journal of the institute of management sciences, june, vol. 8, no. 2, pp. 171-195. iacovou, c.l., benbasat, i., and dexter, a.a. (1995) electronic data interchange and small organisations: adoption and impact of technology, mis quarterly, december, vol. 19, no. 4, pp. 465-485. julien, p.a., and raymond, l. (1994) factors of new technology adoption in the retail sector, entrepreneurship: theory and practice, summer, vol. 18, no. 5, pp. 79-90. kirby, d., and turner, m. (1993) it and the small retail business, international journal of retail and distribution management, vol. 21, no. 7, pp. 20-27. larsen, t.j. (1997) diffusion theory as the basis for understanding information technology related change: true or false?, position statement presented at wg8.6, ambleside, usa [online] <http://www.bi.no/dep2/infomgt/research/tor-innovation/pr-diffusion.htm >, accessed june 2002. lawrence, k.l. (1997) factors inhibiting the utilisation of electronic commerce facilities in tasmanian small to medium-sized enterprises, 8th australasian conference on information systems, october. mcgowan, m.k. and madey, g.r. (1998) the influence of organisation structure and organisational learning factors on the extent of edi implementation in us firms, information resources management journal, summer, vol. 11, no. 3, pp. 17-27. piovesana, r. and rausch, b. (1998) the web help small players connect, bobbin, may, vol. 39, no. 9, pp. 5051. perry, c., meredith, g.g., and cunnington, h.j. (1988) relationship between small business growth and personal characteristics of owner/managers in australia, journal of small business management. april, vol. 26, no. 2, pp. 76-79. poon, s. and swatman p.m.c. (1999) an exploratory study of small business internet commerce issues, information and management, january, vol. 35, no. 1, pp. 9-18. premkumar, g., ramamurthy, k., and nilakanta, s. (1994) implementation of electronic data interchange: an innovation diffusion perspective, journal of management information systems, fall, vol. 11, no. 2, pp. 157-186. rogers, e.m. (1995), diffusion of innovations, 4 th edn, the free press, new york. sillince, j.a.a., macdonald, s., lefang, b. and frost, b. (1998) email adoption, diffusion, use and impact within small firms: a survey of uk companies, international journal of information management, vol. 18, no. 4, pp. 231-242. taylor, s. and todd, p.a. (1995) understanding information technology usage: a test of competing models, information systems research, june, vol. 6, no. 2, pp.144 –176. thong, j.y.l. (1999) an integrated model of information systems adoption in small businesses, journal of management information systems, spring, vol.15, no.4, pp. 187-214. ajis vol. 10 no. 1 september 2002 64 thong, j.y.l. and yap, c.s. (1995) ceo characteristics, organisational characteristics, and information technology adoption in small business, omega, august, vol. 23, no. 4, pp. 429-442. treadgold, a. (1990) the role of distribution channel agents in promoting innovative activity in small retail businesses: a case study of it adoption, service industries journal, vol. 10, no. 4, pp.651-663. van akkeren, j. and cavaye, a.l.m. (1999) confusion with diffusion? unravelling is diffusion and innovation literature with a focus on smes, australian journal in information systems, september, vol.7, no.1, pp 60-69. yellow pages australia (1997) small business index: a special report on technology in the small business sector, august. yellow pages australia (1999) small business index: survey of computer technology and e-commerce in australian small and medium business. may. microsoft word ajisfinal 12 2 marina.doc ajis volume 12 no. 2 may 2005 192 business value creation from it investments: towards a process theory of it governance prof peter marshall assoc. prof judy mckay dr adi prananto university of tasmania swinburne university of technology monash university school of information systems university of tasmania hobart, tasmania email: peter.marshall@utas.edu.au school of information technology swinburne university of technology hawthorn, victoria email: jmckay@it.swin.edu.au school of information management and systems monash university caulfield, victoria email: adi.prananto@sims.monash.edu.au abstract the history of information systems (is) as a discipline has been accompanied by vigorous debate on how, and indeed whether, it investments contribute to improving organisational performance and hence in delivering value to the organisation. a number of models have been put forward to describe the relationship between it expenditure and business value. this paper considers one such model, the process model of soh and markus (1995), and although this model has many attractive features, it is perhaps deficient in the way in which it links it investment proposals back to business objectives, drivers and requirements. a modification of this model is thus proposed, and empirically assessed through qualitative interviews with cios in large organisations to understand the practises and processes by which they assert that business value is achieved through it investments. the modified model was largely supported by the empirical study, but an additional factor, that of the need for and importance of sound it governance processes and procedures to be in place if business value is to be derived form it expenditure. keywords process model, it investment, it governance, business value introduction despite the demise of the dot-coms and much of the over-heated enthusiasm of the “e-business revolution” and the consequent collapse in both spending on it and confidence in it (coltman et al 2001), it is still critically important in today’s business world in terms of organisational efficiency and effectiveness and business competitiveness. today’s businesses could not survive, let alone compete without appropriately well designed and well implemented it systems. indeed it now represents the fastest growing area of capital expenditure (strassman 1997, willcocks and lester 1999), and in some large, it-dependent organisations, capital expenditure on it can exceed 50% of total capital expenditures (van der zee 2002). however, the relationship between it investments and firm performance is neither simple nor well understood. indeed, as the experience with, for example, erp systems shows, there is a lot more involved than buying a commodity product and plugging it in to the organisation. bharadwaj (2000: 169), considering the link between it investment and firm performance, writes “despite the widely held belief that information technology ajis volume 12 no. 2 may 2005 193 (it) is fundamental to a firm’s survival and growth, scholars are still struggling to specify the underlying mechanisms linking it to financial performance”. since the early days of it, but certainly from the 1980s onward there was vigourous debate and research regarding how it investments led to business value in the form of organisational efficiency and effectiveness and in the form of business competitiveness. the guiding theory of the firm used in these studies was the industrial economics view of the firm or the strategic positioning view (mcfarlan 1984, ives and learmonth 1984, porter and miller 1985, johnstone and vitale, 1988, feeny and ives 1990). the upshot of a decade of work on this view was the conclusion that while you may achieve some cost efficiencies with it, it was extremely difficult to improve organisational effectiveness, strategic positioning and business competitiveness via it (clemons and row 1991, kettinger et al 1994, peppard and ward 2004). there is, however, an emerging literature that utilises the resource –based view of the firm as a lens to examine the relationship between it investment and corporate effectiveness and competitiveness (soh and markus 1995, powell and dent-micallef 1997, bharadwaj 2000, sambamurthy et al 2003, peppard and ward 2004). this literature links firm performance to organisational resources and capabilities, one of which is it capability (bharadwaj 2000, peppard & ward 2004). a critical part of it capability is having the right business and management processes to be able to supply, deliver and exploit it systems, or, perhaps better, solutions (peppard and ward 2004). thus, a good strategy process and project processes are needed together with appropriately skilled and knowledgeable human resources. it investments will be successful in creating business value in terms of improved organisational effectiveness and business competitiveness, if the right it investment is chosen in the first place, and then the right it investment is linked to the right combination of process redesign, people skills and commitments and the like, and then the mix is well managed so that an effective organisational system emerges. thus, even if a good it strategy process selects the right it investments, then, simply put, these it investments alone will not create business value. it investments are likely to create business value if, when linked to the right human and business resources, their realisation as a working system is managed well. thus the way in which business value is created is still a focal area for concern among it practitioners and practitioners alike. indeed, it could be said to be the area of concern in the field of information systems! given these concerns, the study reported in this paper explored how senior it executives in large private organisations went about creating the conditions under which business value was harvested from it investments. a framework of structures and processes emerged that is reported later in the paper. we would argue that it is this framework that, broadly speaking, sets the conditions for the realisation of value from it investments. deriving business value from it as discussed above, over the years there has been considerable debate over the extent of it’s contribution to business value, and indeed, whether or not organisations have derived adequate or reasonable value from their it expenditure (farbey et al. 1999, kohli and devaraj 2004). at the heart of much of this debate are the associated issues of effectively establishing quality measures of value that are reliable, and hence credible to senior executives (kohli and devaraj 2004), and that of understanding the processes by which business value is created through expenditure on it (see weill 1992, soh and markus 1995, for example). the research reported in this paper was oriented towards this second issue, that of understanding the processes by which executives in organisations believe they accrue value from their it investments. in a seminal paper, earl (1990) argued that expenditure on it on its own would not lead to improvements in business performance, asserting that there was little evidence of it expenditure on its own directly resulting in improved business performance or outcomes. furthermore, earl (1990) ajis volume 12 no. 2 may 2005 194 claimed that greater sophistication in understanding the relationship between business value and it expenditure was required, and that conceptualising it investment as being accompanied by other required business changes was not sufficient to account for the contribution of it to improved business performance. his suggestion was that it expenditures would only have the desired effects when they resulted from business-led analysis and identification of opportunities and requirements, followed by appropriate it investments and acquisition of required it capabilities. subsequent to earl’s (1990) paper, a number of theoretical models appeared focused around addressing such issues during the 1990s (see lucas 1993, grabowski and lee 1993, for example). soh and markus (1995) considered the features of each of these models and frameworks, from which they synthesized a process model to explain how it creates business value. their model (see figure 1 below) outlines the necessary chain of events if the desired outcome of delivering business value is to be accomplished. it expenditure organisational performance it impacts it assets competitive process it use process it conversion process * it management / conversion activities * appropriate / inappropriate use * competitive position * competitive dynamics figure 1: a process model for the creation of business value from it (soh and markus 1995) the gist of the argument underpinning this model can be succinctly articulated as follows. soh and markus (1995) see business value and an expected positive contribution to business performance derived through three linked processes: the it conversion process, the it use process, and the competitive process. it assets are created through converting expenditure on it into artefacts regarded as assets of the organisation. effectively converting money and other resources into an asset for the organisation is contingent upon a number of it management activities. it assets, however, will only have the desired impact in an organisation if they are used appropriately and benefits from these investments are proactively managed. ensuring appropriate it use in given contexts thus becomes the means by which it assets create the desired it impacts on the organisation. appropriate redesign of business processes together with the appropriate restructuring of roles and responsibilities to ensure good organisational fit for the information system are important factors in this regard. however, whether or not it impacts have the desired effect on organisational performance is contingent on a number of factors, many of which are outside the control of the organisation. for example, the nature of competition in the industry, the behaviour of competitors, the general state of the economy, and so on, will all impact on organisational performance. thus, improvements to organisational performance through the impacts of it may result only when general business conditions are favourable (soh and markus 1995). there are a number of attractive features about this model. firstly, conceptualising the creation of business value from it as a process helps to underscore the on-going and dynamic nature of this transformation, and indeed, indicates that several levels of uncertainty exist in realising value form it. for example, if it expenditure relies on sound management practices for it to be converted into an it asset, then in all organisations there will be variability and some uncertainty as to how ajis volume 12 no. 2 may 2005 195 effectively and consistently this can be accomplished. similar uncertainties and issues are evident at each stage of the soh and markus (1995) process. secondly, it clearly illustrates that responsibilities for deriving value from it investments lies well beyond the bounds of the it staff, thus helping to avoid the issue alluded to by peppard et al. (2000) who note that when it 'disappoints', and there is uncertainty amongst management as to its benefits and value to the organisation, the dominant perspective in the literature attributes these shortcomings to the it function. the soh and markus model encourages an organisationally-based view and understanding of responsibilities in this process. of concern, however, is the starting point of this model as “it expenditure” with the concomitant concern that this might encourage a regression in thinking back to the notion that it expenditure will give rise to business benefits. admittedly, soh and markus (1995) acknowledge that it expenditure embraces it managerial activities including it strategy formulation, designing it structures, selecting and managing it projects. however, somewhat absent from their discussion is the linkage back to business strategy and business requirement. in our view, the model is thus deficient in that it fails to take account of earl’s (1990) claim that a business-led approach to it expenditure is required, with business opportunity and need driving decisions on it investments. this position is effectively supported by the recent work of kohli and deveraj (2004). thus, we would argue for a modification to the model, to ensure that the business focus of it expenditure is both explicitly recognised and featured in the model, and accorded the focus and prominence it requires and deserves (see figure 2 below). it expenditure it expenditure organisational performance it impacts it impacts it assets it assets competitive process it use process it conversion process * it management / conversion activities * appropriate / inappropriate use * competitive position * competitive dynamics strategic imperative strategic imperative it alignment process • strategic context appreciation / needs & opportunities identified figure 2 a modified process model for the realisation of business value from it in addition, the authors argue that this modified process model offers another attractive feature: in essence, it ‘closes the loop’ between organisational performance and business strategic thinking, resulting in a need for it expenditure. it is our contention that the strategic imperative of the organisation, appreciating business opportunities and needs, and making sound it investment decisions in order to support and enable these business opportunities and needs all arise as a result of a perception that business performance can be, or needs to be, improved. it thus serves to cement the processes of expending money and driving value from it not just subsequently to the desire to improve organisational performance, but also to business-based drivers, opportunities and strategic initiatives. key activities in this additional first stage then would include understanding the strategic, or competitive environment of the organisation, and the threats and opportunities inherent in such environments. it would involve identifying both business and technology opportunities, and in articulating a coherent investment strategy for it which would result in it investments clearly and overtly aligned to the business strategy and direction of the organisation. thus, achieving a strategic ‘fit’ between an organisation, its objectives and strategies, and its it investment strategy and ajis volume 12 no. 2 may 2005 196 portfolio would be a key concern here. so too would ensuring that the development of required it capabilities over time was appropriate for the business. it initiatives may also come from perceptions of needs and wants from business executives, but another activity at this stage would be to match these perceptions of want to business initiatives once again to ensure alignment and to facilitate prioritisation, via the formulation of a compelling and increasingly detailed and persuasive business case. the case for the modification of the soh and markus model is one based on the clarity, accuracy and authenticity given to the description of the it business value creation process by splitting the stage of it expenditure to it assets into two stages, the first of which is strategic imperative to it expenditure, and the second of which is the first stage in the previous model, visually it expenditure to it assets. we felt that our description of the business it value creation process would be improved and clarified by having this extra stage, or at least, by making this first stage explicit and distinct. some indication of the need for this extra focus on the analysis of strategic imperatives and what they imply in it terms will become apparent as readers assess the description of our research findings. it will become clear that this first stage is highly significant and important to the creation of it business value. research method and design as indicated in the previous section, the research sought to explore the issue of the creation of business value from it, in particular, the key processes, activities and practices by which senior executives believed their organisation ensured the achievement of benefit and value from it expenditure. we were thus addressing the question: how did senior executives ensure that it investments resulted in subsequent increases in organizational efficiency and effectiveness, and/or business competitiveness. we believed that the best sources of knowledge on it business value and the clearest insights would come from the large private companies with large it spends, long histories of applying it to business and hence possessing it maturity and sophistication, and a reputation for being leaders in the effective and possibly innovative management of it. organisations with the above characteristics were approached and the chief information officer (cio) was asked to make available up to two hours for an in-depth interview on it strategy and management in their organisations, with particular emphasis on their perceptions of how it value could best be derived. the cios of a number of such organisations agreed to take part in the study. the organisations that they represented either were among the largest private businesses in australia, or were the asia-pacific headquarters of huge global businesses with a substantial presence in australia and the asia-pacific. the interviews took between one and two and a half hours. relevant company documents were also provided to and read by the researchers. the interviews were recorded, transcribed and then subjected to qualitative content analysis. the researchers searched for themes, issues and practices that shed light on the derivation of business value from it. a number of common themes, issues and practices began to emerge from the research data. the interviews were continued with new organisations until a stable pattern of findings began to emerge, and few new insights began to occur. by this time, eleven organisations had been considered. basic data on these eleven organisations is shown in table 1. ajis volume 12 no. 2 may 2005 197 company industry it operating budget it capital expenditure it as % of capital expenditure years as cio a retail $100m $80-100m 12% 3 b pharmaceuticals $4m $4m 5% 12 c manufacturing $35m $7m 7% 6 d financial services $55m $48 60% 4 e leisure/entertainment $50m $15m 30% 5 f food & beverage $52m $10m 3 g financial services $500m $400m 70% 3 h financial services $400m $250m 60% 5 i financial services $42m $35m 55% 3 j food & beverage $28m $74* 7-8% 1 k manufacturing $58m $35m 3% 1 (note: all figures cited are expressed in a$, and for global companies, are for it expenditures just in australia) * $66m of this was a one-of erp acquisition and implementation table 1: company demographics the broad research objectives which shaped the interviews were motivated by the modified soh and markus (1995) model (see figure 2), and can be articulated as follows: • to understand the notion of business value from an organisational perspective; • to ‘test’ the proposed modifications to the soh and markus (1995) model, and thus to see whether there is adequate empirical evidence to support the additional ‘stage’ of strategic imperative, as outlined in previous sections. research findings the notion of business value all cios interviewed were very clear when it came to business value: ultimately, they saw business value from it as contributing ultimately to increased profitability. most reported that this was an integral part of the transformation their organisation had been through with respect to it: that their organisations had, in the past few years, moved quite dramatically away from an obsession about costs (it was viewed principally as a way of driving down costs) to a focus on value. it was seen and managed as a way of adding value to the organisation, either through reducing costs, or by increasing efficiency, or by increasing effectiveness in the form of valued service from a range of perspectives (customers, suppliers, alliances, etc.), through it-enablement of core business processes, or through increasing flexibility and responsiveness, or by taking advantage of new opportunities. ajis volume 12 no. 2 may 2005 198 d: well, ultimately, it comes back to improving your profit getting your unit costs down or improving your customer and advisor satisfaction, and that often comes from better business processes. f: i have to be able to show that the benefits derived from an investment generates sufficient value over and above the original investment over a period of time…the board insists on that…we don't worry so much about costs now well, within reason support for the modifications to the soh and markus (1995) model tables 2, 3, and 4 below summarise the organisational activities and practices according to the modified soh and markus (1995) model. in the eleven organisations in our study, these were identified form the transcripts as the practices contributing to the delivery of business value. not every one of these was present in each organisation, but a substantial subset would have been, and some cios flagged particular practices as ones that they did not do, or did not do well, but which they regarded as important. from strategic imperative to it expenditure phase practices strategic imperative to it expenditure formulating it vision and strategy aligning and embedding it strategy in business strategy identifying it opportunities prioritizing and selecting it investment proposals building a broad but rigorous business case for it investment opportunities, including evaluating proposals and identifying business benefits, and estimating the time required to realise benefits identifying it investment project risks constantly monitoring and reassessing proposals for organisational and strategic fit table 2: from strategic imperative to it expenditure our interviews would tend to suggest that this was the most emphasised, the most mentioned, and perceived to be the most critical phase in the ultimate achievement of business value from it. all the cios in our study emphasised the requirement to carefully articulate a vision and high-level strategy for it, which then served as an overarching framework for all proposed it investments. the it vision and strategy was generally articulated by a combination of it and senior business executives, with the determination that it should be aligned with, and thus support the achievement of business visions, goals, and objectives. the need for alignment, for vigilance in maintaining alignment, and in embedding it thinking and strategising in business need and opportunity was a repeated theme in our conversations with executives. this was seen as an on-going, ‘incessant’ activity, and one viewed as being essential to the ultimate delivery of business value e: i have spent my first eight months on three things. alignment, both to the business strategy and the global is standards, engagement both with the business to understand the business strategy and the global is business if you like. governance which is about getting a process by which you can prioritise and manage and it was the third thing that i did, and then financial control. so those are the four things i’ve been doing and none of them have the word ‘it’ in them. but it’s managing information systems, not “ok, which technology do i buy this week?” that people don’t understand. actually the technology decision is really easy if you get the bit at the top right. ajis volume 12 no. 2 may 2005 199 k: the picture i’ve got is there’s an umbrella across the top that talks about the business strategy and the is strategy or the it strategic plan for the company. then underneath that there are is plans for each part of the business. so what you’ve got is kind of like an umbrella that says we have an overall strategy. each of the business units have key is plans which are integrated to any business process change they are trying to do or any change in their business and then underneath that i have to have a layer which integrates all of those plans to an architecture. b: our investments today are targeted to support our strategic imperatives so if the strategic imperatives are right then they're the things that count … we've got to check the allocation to [it] investments is consistent with our strategy which is consistent with our strategic imperatives k: …regarding the business case and where it project ideas come from we should back up a bit and have to maybe talk for a minute about the governance structure first. what we have is both a top down and bottom up governance structure. so we have a business strategic plan for [company name] and we have business unit plans that link to the strategy, you know, strategic plan of the business. and then we have you have an is strategy which links to and supports the business strategy and then you have an is plan in each business unit which supports the business plan. so what happens then is that you must link the cios all believed that an important component of delivering value was to identify it opportunities. opportunities for it investments emerged from business needs or desires, or to avoid strategic disadvantage (i.e. in response to innovation by their competitors), but whatever the driver, cios agreed on the need for a compelling business case to be articulated before any it investment would be contemplated. the business case included consideration of likely costs and possible benefits from proposed it investments, and was revisited on a number of occasions, each iteration resulting in greater certainty as to costs and feasibility, increased confidence in the organisation’s ability to achieve the benefits, greater surety in the time both to acquire and to reap benefits from the proposed investment, and certainty as to the fact that the investment proposal involved the building of required and useful it capabilities. evaluation at this stage also emphasised the need to identify it-related risk, and to put in place a plan to manage exposure from the proposed investment. activities at this stage were fluid, with constant worrying and checking that proposed it investments were still aligned with business directions, that they represented a good strategic fit, and that there was clear understandings between business units and it on the nature of the requirements, expectations, and the like. all the cios interviewed reported formal is/it planning processes and mechanisms, with plans and strategies subject to more frequent reviews (3-6 monthly), where business goals, objectives, and priorities were re-evaluated, and then is/it initiatives re-evaluated in the light of possibly changing business initiatives. it was during this process that new proposals for is/it investments were considered, with very careful scrutiny of each proposal being conducted to ensure that any resultant it investment would be aligned and consistent with strategic business imperatives. h: somebody will come up with an idea in a business unit which is the way it works. that needs to be the first thing that happens because the planning managers are linked to me, and a single strategic planning group, and idea comes inside my group and we talk about whether it’s really a business unit idea or it’s an enterprise idea. we would then create a program manager sponsored by myself because i take the cross business projects, because i’m on the board it’s quite easy to do. you just sit at the board table and agree that we’re going to do this enterprise wide and then that program manager will work with the planning managers to develop the business case for the enterprise. the business ajis volume 12 no. 2 may 2005 200 justification and the benefits and the project itself must go up through each business unit’s it steering committee because each business unit must commit to the program and must to commit to their part of the benefits e: so we've got a very strict business case preparation and authorization process that has as its basis the calculation of the return on the investment over an accepted period of time…i don't even accept productivity gains anymore. if someone tries to present a business case based on productivity gains then i want names of the people that aren't going to be working here anymore, what their salaries are, and what day they're going to be leaving… from it expenditure to it assets phase practices it expenditure to assets managing it project risk redesign of business processes, and assessing and planning required business change preparing a detailed business case including the evaluation of specific costs and benefits project management – effectively managing all aspects associated with acquiring and/or developing the desired information system carry out evaluations of the partly developed system or proposed system including checks on changing business requirements table 3: from it expenditure to it assets when a decision had been made to proceed with a particular investment proposal, then the focus of activities changed somewhat, and a greater concentration on the project itself became evident. cios reported an increased focus on project management issues, with great emphasis now placed on adequately identifying and managing it project risk. changes to the system were in some cases paid careful attention, with an emphasis by some cios in ensuring that project changes did not compromise the ability to achieve benefits form the it investment. in addition, most organisations seemed to be more keenly aware of the need to carefully look at the business processes supported by it or embedded in newly purchased software, and to consider carefully the issues of business process reengineering (bpr) to accommodate some of the business changes initiated by the new it investment. most cios seemed very sympathetic to the position that implementing new it systems without revisiting the business processes underpinning these systems would be unlikely to result in the best value being derived by the organisation. throughout the project, analysis typically continued into both the costs and benefits of the it project. business cases were still subject to revision and confirmation, with regular reviews conducted to oversee the management or containment of costs, and the certainty associated with the delivery of benefits. in addition, these reviews included a revisiting of business goals and objectives, to ensure that projects were still tightly aligned with business directions. it was reported in a n umber of organisations that projects could and would still be cancelled at this stage (part way through development/acquisition) if there was any suggestion that business priorities had changed rendering the objectives of the it investment less useful than anticipated. c: we have a gating process where people get their money to get through the gates to do the project … that is the project is rigorously evaluated at each gate. so we have gate 0 to gate 1 which is really straight analysis – and there will be an it business solutions manager involved in that – gate 1 to gate 2 is really high level analysis – not so much high ajis volume 12 no. 2 may 2005 201 level but it’s the step 1 analysis and so on – gate 2 to gate 3 is detailed analysis and so on which just takes that a bit further – and then once you get to gate 3 test, implement – at 4 – the project finishing is after such and such a period – that’s the project finished – so we are continually analyzing, evaluating and re-evaluating you know checking on progress, spend, relevance and so on, trying to intelligently react to changes h: but we have pulled some it projects here and there, but if it’s only going to be a bit late and it’s running over a bit, well with some care of course, we might cut the scope back … it assets to it impacts phase pppppppppp practices it assets to it impacts post implementation reviews implementation of business process change measurement of the achievement of goals and objectives for it investments possible reallocation and restructuring of roles and responsibilities to achieve organisational fit management of the realisation of benefits from it investments evaluation of stakeholder satisfaction with it assessment of perceived value of it investments management of business change required to accommodate it implementation training and reskilling table 4: from it assets to it impacts one of the most common practices reported in ensuring that it investments had the desired impacts was to conduct post implementation reviews (pirs) for all projects. this involved typical project measures of whether the system was delivered on time, to budget, with the specified and required level of quality. however, some of the cios were aware that pirs did not necessarily contribute much to the delivery of business value. these cios were all implementing proactive benefits realisation processes, whereby post project, serious efforts were made to assess and measure the effects and impacts of the system implementation, and to check whether the anticipated benefits were being realised. pirs were argued to contribute to value from the perspective of enabling learning to take place about the effective management of projects. the intention of the benefits realisation or management processes evident in a few organisations however, were much more oriented towards ensuring that goals and objectives established pre-project for the it expenditure had or were, in fact, being achieved. thus these benefits-oriented types of evaluation tended to assess the it investment more holistically in its business context, and thus also considered the effects and impacts of business process change and other associated business changes that had been necessitated by the implementation of the new system. not only were the impacts of these process and business changes assessed, but the interview data also suggests that there was recognition of the need to change jobs, roles and responsibilities in line with the it investment, with the possibility acknowledged of a need for restructuring of the workforce and conditions. thus, it was not uncommon for cios in this study to comment on the vital element of reallocating and restructuring roles and responsibilities, with the concomitant need for training and reskilling in light of the changes and impacts of the it investment. for it to have the desired impact, such changes were deemed absolutely essential. ajis volume 12 no. 2 may 2005 202 it was relatively common in organisations in this study for post-implementation reviews to be conducted of stakeholder satisfaction. the logic typically driving these initiatives was that stakeholders that are satisfied are more likely to make better use of it investments, and hence the organisation is more likely to derive the business value and benefits identified pre-investment. however, some of our cios did acknowledge that stakeholder satisfaction was not always nor necessarily the same as deriving business value, but nonetheless, such surveys were generally regarded as useful indicators in many circumstances. j: yeah, we generally do post-implementation evaluation…not on time and budget issues. oh no, it's more like "did we achieve the business benefits and things like that?" it's on the positive side…and the achievement of benefits, or otherwise. um, it'll be the it people who pull together the review, but clearly it will need to draw on information and input from people right throughout the business discussion the modified soh and markus model, we believe is a valuable organising and sense making framework for describing and explaining the activities and processes involved in the effective realisation of business value from it investments. the importance and emphasis placed on the first stage of it strategy, it opportunity identification, and it-business analysis by the cios in our sample, we feel, justifies the explicit inclusion of the stage of strategic imperative to it expenditure. of all the phases represented in the modified soh and markus (1995) model, the importance of closely linking it initiatives to business drivers and requirements was regularly emphasised, and arguably was considered perhaps the most important element in ultimately deriving business value. we thus assert that there is empirical support for the modifications made to the model, and that the modified model (see figure 2) better represents the concerns and practices of cios, at least in our sample of organisations. it is of interest that we had few difficulties in identifying comments form he cios which supported this first phase, but that expressed support for the latter phase of moving from it assets to it impacts was less emphasised. for example, all cios talked a lot about issues of alignment, understanding the business imperatives, ensuring a business orientation to it investments. the cios, when prompted, all talked about issues and practices associated with the move form it assets to it impacts, but some did need to be prompted, and there was less explicit recognition that appropriate system usage and ensuring the derivation of expected benefits were regarded as an integral part of their responsibilities in efforts to achieve business benefits from it spend. in fact, while many of the cios interviewed recognised the requirement to assess the delivery of business benefits, this was the area where there was most divergence in the organisations in our study. only two of the eleven organisations reported formal, rigorous benefits realisation programmes, where serious attempts were made to measure benefits achieved over time. these two organisations linked expected benefits to performance measures of project sponsors and business unit budgets. so, for example, if a proposed system was claimed to be able to reduce operating costs in a particular business unit by 5% per annum, the operating budgets for that unit would be cut by 5% the following year, and the manager of that business unit would have his/her performance measures and rewards linked directly to the achievement of that 5% reduction. h: so they say to me, "david, we've got this fantastic system for our business, it's going to reduce costs by 2% if you give me that system". i say "go ahead, but i'm taking 2% out of your budget for next year." and the same thing on the revenue side, that would add to your budget. they've got to get it…it makes the investment proposals much more realistic and cautious…serious. ajis volume 12 no. 2 may 2005 203 most other organisations recognised that this was an area where they needed to improve, and many stated that it was on their agendas as a "to do" item, conceding that this was an important area where they needed to improve. however, some of the managers were concerned about the resources required to undertake such a benefits realisation process. a: we certainly do a post implementation review of all our projects but if the question is more around benefits, do we go back rigorously a year later or whatever and say 'okay compared to what we predicted how did we do?' i've got to say we haven't been strong on that as i think we should've been but the other factor of course is that there's always a lot changing in the business anyway, so it's very hard to attribute an improvement to one specific project…it tends to be look we think we've got 80% of the benefits from what we've already done rather than put a whole heap of extra effort into chasing proportionately less benefits, let's stop, and accept that there is some weaknesses in what we've done but there are bigger opportunities elsewhere the issue being grappled with here is that as cios, they felt torn between the potential to derive greater benefits from the existing investments, as opposed to diverting resources to exploiting other it opportunities. interestingly, most of the cios still tended to adopt the satisficing position (i.e. that delivering, say 80% of expected benefits was probably good enough and that the resources consumed in trying to achieve 100% or more would be better diverted elsewhere). the cios in this study were aware of proactive benefits realisation, and that looking to enabling business changes and activities could help achieve business benefits. however many cios admitted that this aspect was not well established, and that they had not established an effective process of benefits realisation as part of the accepted routines of the organisation. it expenditure organisational performance it impacts it assets competitive process it use process it conversion process * it management / conversion activities * appropriate / inappropriate use * competitive position * competitive dynamics strategic imperative it alignment process •strategic context appreciation / needs & opportunities identified governance it expenditure it expenditure organisational performance it impacts it impacts it assets it assets competitive process it use process it conversion process * it management / conversion activities * appropriate / inappropriate use * competitive position * competitive dynamics strategic imperative strategic imperative it alignment process •strategic context appreciation / needs & opportunities identified governance figure 3: it governance and the creation of business value from it what did emerge clearly from the interviews however, was the cios emphasis on issues of governance (see figure 3 above). previous quotes cited in this paper are testimony to this. in the minds of our cios, governance seems to involve a range of interconnected and overlapping elements. in the organisations studied, it appeared that governance was a function of articulating a vision and principles about the role and potential contribution of it in a particular organisation. this was coupled with clear structures by which decision making accountabilities and responsibilities were formally identified. principles and decision making structures, however, needed to be implemented via clearly articulated processes over time. but in addition, there was a concern, almost an obsession with many of our cios with building organisation wide it capabilities ajis volume 12 no. 2 may 2005 204 and relationships, and adapting these over time, in order that appropriate systems and services could be delivered to the organisation, creating appropriate business value (see figure 4 below). is implemented through effective it governance principles structures processes with the aim of monitoring & controlling it investments & associated activities to ensure the delivery of business value from it building of organisation-wide it capabilities & relationships requires is implemented through effective it governance effective it governance principles structures processes with the aim of monitoring & controlling it investments & associated activities to ensure the delivery of business value from it building of organisation-wide it capabilities & relationships building of organisation-wide it capabilities & relationships requires figure 4: the elements of it governance thus, it is our contention that good it governance processes accomplish at least three broad objectives: they support an environment for the development, exercise and exploitation of it resources and capabilities; they provide a framework for the fruitful exploration and explication of relationships between the it function and the rest of the organisation; and they formally define, identify and underpin a series of organisational routines and procedures through which the business value of it is realised and it risk contained. the processes and activities to ensure it business value will occur effectively, we believe, if they are formally defined and established by the organisation's it governance framework. it governance, we would argue, is a management framework consisting of the necessary structures and processes for the effective stewardship and management of business initiatives involving it, from the identification of it systems as part of business initiatives, right through to the realisation of benefits from implemented systems working in harmony with both people and business processes. such a framework shapes and determines the nature of the it business value creating processes, and we would argue, is necessary for effective and reliable business it value creation conclusion we have modified the soh and markus model by adding a first stage which concerned the activities of strategy formulation, it opportunity identification and it business analysis. this stage, in our view, was of critical importance to realising business value from it investments. this stage was also viewed as critically important by the cios we interviewed, as evidenced by their focus on the role of these activities in realising it business value. thus we believe we have justified the inclusion of the stage as an explicit and separate part of the model. we hope that our extended model will prove useful in helping practitioners visualise and organise the process of obtaining business value from it investments. we have also taken the view that the establishment of an appropriate it governance framework is critical to it business value creation. we can view the knowledge of how to perform it strategy formulation and it project management and it risk management activities and processes effectively as an internal resource that is critical to the task of extracting business value from it investments. such knowledge is partly tacit, but the more a firm can develop, grow and maintain such knowledge, and further, the more it can ensure such knowledge is shared among its employees, within the it ajis volume 12 no. 2 may 2005 205 function and beyond, the more likely it is to profit from its it investments. references bharadwaj, a. s. (2000) a resource-based perspective on information technology capability and firm performance: an empirical investigation. mis quarterly, volume 24, no. 1, march, pages 169-196. coltman, t., devinney, t. m., latukefu, a, and midgley, d. f. (2001) e-business: revolution, evolution or hype? california management review, volume 44, no. 1, fall, pages 57-86. earl, m. j. (1990). putting it in its place: a polemic for the nineties (oxford institute of information management, research, and discussion papers, rpd90/2). oxford, england: templeton college, university of oxford. farbey, b., land, f. and targett, d. (1999) moving is evaluation forward: learning themes and research issues. journal of strategic information systems 8(1999): 189-207. feeny, d. f. and ives, b. (1990) in search of sustainability: reaping long-term advantage from investments in information technology. journal of management information systems, volume 7, no. 1, summer, pages 27-32. grabowski, m. and lee, s. (1993) linking information systems portfolios and organizational strategy. in r.d. banker, r.j. kauffman and m.a. mahmood (eds.) strategic information technology management: perspectives on organizational growth and competitive advantage, pp33-54. idea publishing group, harrisburg. ives, b. and learmonth, g. p. (1984) the information system as a competitive weapon. communications of the acm, volume 27, no. 12, december, pages 1193-1201. johnstone, h. r. and vitale, m. r. (1988) creating competitive advantage with interorganisational information systems. mis quarterly, volume 12, no.2, june, pages 153-165. kohli, r. and deveraj, s. (2004) realizing the business value of information technology investments: an organizational process. mis quarterly executive, 3(1): 53-68. lucas, h.c. (1993) the business value from information technology: a historical perspective and thoughts for future research. in r.d. banker, r.j. kauffman and m.a. mahmood (eds.) strategic information technology management: perspectives on organizational growth and competitive advantage, pp359-374. idea publishing group, harrisburg mcfarlan, f. w. (1984) information technology changes the way you compete. harvard business review, volume 62, no. 3, may/june, pages 98-103. peppard, j. and ward, j. (2004) beyond strategic information systems: towards an is capability. journal of strategic information systems, 13(2004): 167-194. peppard, j., lambert, r., and edwards, c. (2000) whose job is it anyway?: organizational information competencies for value creation. information systems journal, 10(4), 291322. porter, m. e. and miller, v. e. (1985) how information gives you competitive advantage. harvard business review, volume 63, no. 4, july/august, pages 149-160. powell, t. c. and dent-micallef, a. (1995) information technology as competitive advantage: the role of human, business, and technology resources. strategic management journal, volume 18, no. 5, pages 375-405. soh, c. and markus, m.l. (1995) how it creates business value: a process theory synthesis. in ariav, g. et al. (eds.) proceedings of the 16th international conference on information systems, december 10-13, amsterdam, netherlands. strassmann, p.a. (1997) the squandered computer; evaluating the business alignment of information technologies. information economics press, new canaan, connecticut. ajis volume 12 no. 2 may 2005 206 van der zee, h. (2002) measuring the value of information technology. idea publishing group, hershey weill, p. the relationship between investment in information technology and form performance: a study of the valve manufacturing sector. information systems research, 3(4): 307-333. willcocks, l.p. and lester, s. (1999) information technology: transformer or sink hole? in willcocks, l.p. and lester, s. (eds.) beyond the it productivity paradox. wiley, chichester. copyright peter marshall, judy mckay and adi prananto © 2004. the authors assign to acis and educational and non-profit institutions a non-exclusive licence to use this document for personal use and in courses of instruction provided that the article is used in full and this copyright statement is reproduced. the authors also grant a non-exclusive licence to acis to publish this document in full in the conference papers and proceedings. those documents may be published on the world wide web, cd-rom, in printed form, and on mirror sites on the world wide web. any other usage is prohibited without the express permission of the authors. aj1s special edition knowledge management december 2001 editorial it is a pleasure to provide you with this special issue of the australian journal of information systems dedicated to knowledge management. sceptics continue to argue that knowledge management does not contribute anything new in solving organizational problems and in essence is just a 're-dressing' of old ideas to justify introduction of expensive computer infrastructure. however, the camp of such sceptics is getting smaller and smaller with the increased acceptance of the importance and value of knowledge processes in the success of modern organizations. it is now commonly recognized that knowledge is a special kind of resource that needs to be managed. however, recent studies and field reports regarding knowledge management initiatives in leading organizations worldwide clearly show that traditional organisational technology and management techniques are seldom capable of capturing and enhancing organisational knowledge bases. there is a need to develop new advanced approaches, which are specifically devised to stimulate knowledge creation and sharing supported by modern it systems. it was a hard task to choose papers for this volume out of the twenty initial submissions. the field of knowledge management is grounded on many diverse reference disciplines such as decision science, management and organization science, economics, cognitive science, computer science, information science as well as sociology, anthropology, linguistics, politics, psychology. the eleven papers that were selected for this special issue bring concepts from many of these diverse fields in introducing their results and providing justification of the value of the studies. this creates a somewhat blurred picture of what is and what is not in the scope of the field of knowledge management. i have deliberately selected papers with diverse views and approaches to allow the reader to grasp the breadth as well as depth of the field. therefore the volume contains contributions from researchers and practitioners from australia, europe, asia and the usa and present multi-disciplinary views on knowledge management. there is no visible link between these papers, yet one can see the way they complement each other in demonstrating the ways knowledge is treated in different contexts and the important role of technology in creating knowledge from information contained in a variety of organisational documents and making knowledge processes resulting in effective organisational learning. the papers can be broadly divided into two categories. papers in the first category deal with the theoretical concepts such as knowledge creation and organisational learning which are intrinsic to building and exploiting intelligent assets. papers in the second category report on practice and implications of knowledge management initiatives. all papers contain illustrations of the issues raised from various organizational studies or reflections on personal experiences. papers by pigott, hobbs and gammack damsgaard and scheepers, morrissey, schmid, as well as marsh and burke, and linger and wame propose some new concepts, architectures and approaches, which allow dealing with challenges in handling less structured environments and entities of information in order to transform them into useful knowledge. these authors also address the innovative ways that technology can be used for supporting knowledge management needs. the second group of papers present some results of exploratory studies in the implementation of knowledge management strategies. handzic discuss some results from experimenting with decision support system prototype as a means of contextual knowledge exploration and capture. lauer and tanniru propose an approach of reviewing knowledge resources in the organization and matching them to major organisational goals (knowledge audit) and a case study illustrating the application of the proposed methodology. the authors come up with some recommendations and discuss future implications of such an approach. papers by timbrell, andrews and gable and law and leepartridge present results from the studies of organisational knowledge management in relation to transfer of best practice and exploring the levels of understanding of issues, concerns and requirements by chief information officers (cios) and senior is managers. these papers are also interesting from the point of view of the research methodology they use in analysing the collected data. a paper by warne, ali and agostino from dsto and pascoe from university of canberra draws on experience the authors gained from ethnographic study of social learning. i would like to especially thank a group of twenty reviewers, including international experts in knowledge management for their help in selecting the finest quality papers for this issue. i would like to thank all the authors for the their contributions and considerable effort in meeting tight deadlines and high standards that have been imposed by the reviewers. i would also like to thank the editor of the ajis, prof rob mcgregor for giving me the opportunity to undertake this challenging project on behalf of the knowledge management research community. frada burstein, guest editor, school of information management and systems, monash university microsoft word ajis112final.doc australasian journal of information systems vol 11 no 2 may 2004 4 success factors and measures for public sector is/it co-sourcing contracts erhan edguer and graham pervan school of information systems curtin university of technology gpo box u1987, perth wa 6845, australia pervang@cbs.curtin.edu.au abstract the main objective of this research was to explore the effectiveness of contract negotiations between buyers and suppliers in small government organizations, which collectively outsource their is/it activities to a single outsourcing vendor, usually referred to as ‘co-sourcing’. a major finding of this study was that organizations could have a successful co-sourcing arrangement by determining and putting into practice certain critical success factors. this research was the first study of government cosourcing arrangements in australia that aimed to identify the success of a contract and the critical factors that affected it. in this regard, it can contribute to the existing body of knowledge in co-sourcing activities that have been growing rapidly in government departments as well as in the private sector. introduction outsourcing information systems/information technology (is/it) responsibilities has become a viable business for many companies. to ensure success, a relationship between the outsourcing vendor and the buyer that clearly defines roles and responsibilities must be developed. proper performance measurements and well-defined service levels need to be established and monitored to achieve the desired results. however, for many organizations, measuring the performance of the outsourcer is a frustrating task. confusion exists over what should be measured and how. many research studies have been conducted to inform businesses about these issues. the variety of research conducted in the area of is/it outsourcing includes: outsourcing types (lacity et al. 1996), outsourcing decision-making (gupta & gupa 1992), benefits of outsourcing (galliers et al. 1999, khosrowpour 1995), real cost of outsourcing (ang & straub 1998), the role of trust in outsourcing (sabherwal 1999), and assessing the risk of outsourcing (aubert et al. 1998). there is a need for empirical research to investigate government agencies that collectively outsource their is/it activities. this research case study was an exploratory investigation of a government it outsourcing contract. the data collected was qualitative and the research paradigm was interpretive. background of outsourcing the outsourcing concept is not new. during the 50s, 60s and the 70s many small and medium size businesses subcontracted their information processing activities to data processing service bureaus (khosrowpour 1995, lacity & willcocks 1998), because of the high costs of purchasing and developing computer technologies. however, improvements in information technologies and wide use of personal computers made these services redundant. in the 90s, the emergence of computer technology with communication technologies provided many organizations with new tools for the management of information and communication (khosrowpour 1995). new global economic forces forced organizations to again find ways of reducing their costs while becoming more effective in this new competitive market. as a result, many organizations were looking for alternative ways of accessing these tools. outsourcing was, and continues to be, one of these alternatives. kodak was the first large company to successfully outsource its information systems in 1989 and many other organizations have taken up this option. not all outsourcing attempts have been successful and many have not reaped the benefits of outsourcing. australasian journal of information systems vol 11 no 2 may 2004 5 what is outsourcing in this research, the outsourcing activity is referred to as the dynamic transition from internal provision to external provision as well as the resulting state of external provision. therefore the definition of outsourcing used in this study is: “is outsourcing is the commissioning of part or all of the is activities an organization needs, and/or transferring the associated human and other is resources, to one or more external is suppliers” (de looff 1997, p.30). where the outsourcing is a collective arrangement between one vendor and multiple clients, the activity is referred to as “co-sourcing” (aubert et al, 1998). why organizations outsource outsourcing has become an important issue for managers of information systems and technology for different reasons. competition is becoming fiercer in national and international markets. market shares are decreasing while global pressures are increasing. product life cycles are getting shorter, forcing organizations to become more efficient. a shortage of skilled is professionals places pressure on is managers to deliver on their promise of faster, cheaper and better solutions. the reasons for organizations to outsource are many, but can be categorised under strategic focus issues, economies of scale issues, external market forces issues, and technical considerations (mcfarlan & nolan 1995, rothery & robertson 1995). when to outsource outsourcing can be justified when the is department faces one or more of the following situations (gupta & gupta 1992): • there is immediate need to reduce costs. • there is a critical shortage of is talent available to continue with in-house development. • the given is function has no strategic advantage and an outside vendor is better suited to perform that task. • the organization lacks suitable is resources to respond to any sudden change in the environment to gain competitive advantage or keep up with the competitors. • time and talent is spent on maintaining old systems, thus preventing new developments. • outsourcing support functions can sharpen focus on strategic business. • hiring-freezes may force the outsourcing of essential is functions. factors in outsourcing decisions information is defined as a strategic resource regardless of an organization's line of business (galliers et al. 1999, khosrowpour 1995), therefore outsourcing decisions are strategic not just operational (gupta & gupta 1992). although many organizations use outsourcing to gain a competitive advantage or decrease market pressure, outsourcing may not be right for every organization. organizations considering outsourcing have to assess the management issues and potential problems and should perform a cost/benefit analysis as well as analysing the risks involved with outsourcing. common factors in outsourcing that are identified in the literature include: • vendor reputation (brewer 1998, de looff 1997). • quality of service (tarsh 1999, williams 1998). • strategic applications (gupta & gupta 1992, khosrowpour 1995). • loss of is positions (de looff 1997, khosrowpour 1995). • demoralised employees (gupta & gupta 1992). • untested waters (gupta & gupta 1992, rothery & robertson 1995). • loss of control (mcfarlan & nolan 1995, rothery & robertson 1995). • cost benefit analysis (de looff 1997, khosrowpour 1995). australasian journal of information systems vol 11 no 2 may 2004 6 outsourcing in government the outsourcing of it is not limited to the private sector and most government organizations outsource some or all of their is/it. major factors contributing to the trend towards government outsourcing include: tighter budgets, lack of specialised skills among government employees, and politicians’ discontent over spending on botched it programs (anonymous 1996). a 1993 study of businesses and government agencies by penton research services showed that 44% of the executives surveyed said they are doing more outsourcing than they did 5 years before, and 47% said they expected to increase the amount of work they outsourced by the year 2000 (kelley 1995). as outsourcing has grown in the last few years, so has its scope. government organizations are going through the same processes as their private counterparts. the success of outsourcing in government departments also depends on how they negotiate their outsourcing contracts with vendors. however, there is very little research done in the area of outsourcing is/it in government organizations, especially those that collectively outsource their is/it to a large vendor. research question and approach the outsourcing literature has clearly identified a rapid increase in outsourcing arrangements across the business sector for many different reasons. with the increase in outsourcing arrangements a number of issues have been identified that affect these outsourcing arrangements. whether outsourcing is total or selective, these issues affect the business. many research studies have been conducted to inform organisations about these issues, but analysis of the literature suggests that there has been little research done in the area of (a) co-sourcing contracts, (b) government it outsourcing, and (c) co-sourcing contract negotiations. therefore this study is focused on these areas of outsourcing identified above and. investigates the following research questions: a) what are the objectives of government agencies that collectively outsource selected it functions? b) how successful are co-sourcing contracts? c) how do government agencies measure the success of the contract? d) what critical success factors determine the success of the contract? e) what other factors contribute to the success of the contract? this study aims to explore the co-sourcing arrangements within government agencies and illustrate the issues involved by answering the above questions. selection of the research method the selection of a research method depends on the purpose of the research and context of the research question. galliers (1991) proposed a taxonomy to assist is researchers to match research method to objectives. given the above questions, the research is focused on organizations (not individuals or groups), on methodology (not technology), and on theory building (not testing or extension), so clearly the case study approach can be considered suitable. research design given the exploratory nature of the case, data collected was mostly qualitative and the research paradigm interpretive. data was collected in the six government agencies (referred to as agency a … f) that were part of a consortium that signed a co-sourcing agreement with an outsourcing service provider, in the government agency responsible for contract negotiation and management, and from the provider. the core data was obtained by interview and document review. australasian journal of information systems vol 11 no 2 may 2004 7 interview semi-structured interviews were conducted with two people responsible for the contract management at each of the six agencies (one at the strategic level and the other at the contract management level). the twelve interviews were tape-recorded, transcribed, emailed to the interviewee for accuracy checking, and corrected. approximately 150 pages of transcripts were then analysed. document review the second source was a document review, which was used to investigate several organizational issues within the chosen organizations and to provide a means of triangulating the data (mcfarlane 1998). the documents collected and analysed included (a) the head contract and service agreement, (b) contract promotional material, and (c) web sites of the agencies. discussion of findings the findings presented here are based on a content analysis of all of the data described above. thirteen success measures were identified in the analysis and the managers agreed that, overall, the contract successfully delivered services according to the levels negotiated and defined in the service agreements. the analysis also identified thirteen critical success factors that the managers believed had positively contributed to the success of their co-sourcing contract. these success measures and factors are listed in table 1 below. brewer (1998) suggested that organizations had to identify critical success factors to succeed in their outsourcing arrangement. perigo (1998) identified that government departments in the uk used critical success factors such as ‘departments made the outsourcing decision to meet their business needs’ to ensure successful outsourcing contracts. further analysis of the data collected established a relationship between the success measures and the critical success factors identified by the agency representatives. these relationships were vital to the measure of success in the contract and are presented in table 2 below. s u c c e s s m e a s u r e s (s) c r i t i c a l s u c c e s s f a c t o r s (f) 1. decrease the staff numbers 2. establish continuity within the is 3. focus on the core business 4. reduce costs and manage assets efficiently 5. better compete in the market 6. increase productivity 7. clear operational processes 8. reduce or control operating costs 9. access world-class leading-edge technology 10. access to skilled it professionals 11. implementation of emerging technologies 12. looking for a strategic partner for it services 13. compliance with service level agreements 1. meet the business needs 2. involved staff in the process 3. clear understanding 4. tracked the performance 5. understood costs performance and quality 6. communicated regularly and openly 7. flexible contracts 8. transparent transition to new provider 9. addressed problems, constructive not defensive 10. understood responsibilities and obligations 11. service provider is proactive 12. problems resolved at an appropriate level 13. continually review arrangements table 1: list of success measures and critical success factors australasian journal of information systems vol 11 no 2 may 2004 8 f1 f2 f3 f4 f5 f6 f7 f8 f9 f10 f11 f12 f13 s1 x x s2 x x x s3 x x x x s4 x x x x x x s5 x x x x x x s6 x x x x s7 x x x x x s8 x x x x x x s9 x x x s10 x x x x s11 x x x x s12 x x x s13 x x x x x x x x x x x x x table 2: relationship between success measures and critical success factors the analysis established that success measures s1, s2, s9, and s12 were affected by less than four csfs, while the others were affected by more than four. the least number of csfs contributed to s1 (decrease staff numbers) and the most csfs contributed to s13 (compliant with service level agreement). thus, s13 may be considered a major success measure for the contract and s1 less important. the analysis also identified that while some csfs (f2, f5, f6, f8, f9, f10, and f12) contributed to less than four success measures, others (f1 (12), f7 (9), and f13 (10)) contributed to many more success measures. this implies that f1, f7, and f13 are the most important csfs, contributing to most success measures. the research study also identified that all the agencies judged their performance by how much they achieved their objectives, a finding supported elsewhere (brewer 1998, gupta & gupta 1992, mcfarlane 1998). the thirteen success measures will now be discussed in some detail. decrease staff numbers to reduce expenditure (s1) decreasing staff numbers to meet business needs was achieved by all the agencies through staff reductions in their is departments. all the agencies acknowledged the effect of the change of their is department’s role in their organization. most of the staff went to work for the outsourcing service provider. this created immediate savings to the agencies through the reduction in the salaries paid to staff. establish continuity within the is activities (s2) establishing continuity within the is activities was achieved by all the agencies through the contract they signed with the outsourcing vendor. the contract had certain clauses to protect the agencies from the vendor going into liquidation or not meeting the contract service levels. money was deposited in a trust account to compensate the agencies and assist them to switch to another vendor if necessary. also, agency a kept the ownership of its equipment and only outsourced its services. focus on the core business (s3) focusing on their core business was a very important success measure and a major objective of the agencies in outsourcing their is functions. the agencies achieved this at different levels. while agencies a, c, and e were experiencing difficulties with their service provider’s delivery, the others recognised that they were able to focus on their core business after the outsourcing arrangement. this could be related to the agencies’ views on the critical success factors that affected this success measure. agencies a, c, and e were all concerned that their service provider was not being proactive in advising them in their business decisions. others were quite happy with the service australasian journal of information systems vol 11 no 2 may 2004 9 provider’s actions. this finding is supported by gupta and gupta (1992) who suggest that outsourcing their it functions could enable the organizations to focus more on their core business and leave it delivery to the outsourcers. reduce costs and manage assets more efficiently (s4) reducing costs and managing assets more efficiently was identified as being achieved with some aspects of the arrangement. all agencies agreed that cost reduction in hardware and software licences was achieved in the first three years of the contract but in the long run there were no cost savings. lacity and willcocks (1996) confirm this and suggest that organizations could reduce their it operating costs by selectively outsourcing their it functions for a limited period. enable the organization to better compete in the market (s5) brewer (1998) suggests that market forces could force an organization to resort to outsourcing and, by the outsourcer running this support function more efficiently, create a competitive advantage. this may not be case in government organizations as they usually don’t compete with other organizations in delivering goods and services in the market place. only agency d, with a different corporate structure and culture than the other government agencies, acknowledged the importance of this measure. increase in productivity (s6) most of the agencies claimed they achieved increased productivity through outsourcing. this was said to be facilitated by the service provider’s buying power and ability to use more efficient managerial processes. however, a and e said that they did not achieve the expected increase in productivity, perhaps because their expectations were too high. the other four agencies had more realistic expectation on their productivity improvement. jones (1997) supports this and suggests that outsourcing activities that are not core competencies allows companies to focus attention on areas that add the most value. clear operational processes (s7) four agencies (b, c, e, f) indicated that they achieved clearer operational processes through their outsourcing arrangement. this could be explained by their ability to respond to the changes made to their business procedures. they all believed that outsourcing enabled the processes to be more simple and accountable. agency a and d both claimed that the processes did not achieve any major improvements perhaps because they had problems adjusting to the new system. reduce or control operating costs (s8) reducing or controlling operating costs was an important success measure. one of the research outcomes was that the agencies were able to control and reduce operating costs through the contract. the outsourcing vendor could create economies of scale and deliver the same services more efficiently and cheaply through the ability to access bigger markets using the same infrastructure in similar contracts. all the agencies acknowledged that there was a significant cost saving in services delivered by the outsourcing vendor. however, all the managers interviewed said that these savings were only short term. this could be because the vendor also has to pay for new staff and new infrastructure as well as trying to deliver the services. lacity et al. (1996) suggest that organizations could reduce their it operating costs by selectively outsourcing their it functions for a limited period. australasian journal of information systems vol 11 no 2 may 2004 10 access world-class capabilities and leading-edge technology (s9) accessing world-class capabilities and leading edge technology was another success measure the agencies wanted to achieve. all said that their decision to outsource included the areas of networking, which incorporated helpdesk as well as lan and wan support, and sap application development. agency e claimed that it was impossible for them to in-source their new application development and also support the resources needed to implement this new business application, so the management decided to outsource the whole applications and network administration to the outsourcing vendor. agency f held the view that the outsourcing service provider had been successful in delivering the outsourced applications overall, but agency e had the opposite view on networking and remote logins. the reason for this could be that the relationship between the agency f and the vendor was very good. the dissatisfaction also could be due reasons such as lack of understanding, the current infrastructure, early days of the contract, the network being effected by factors outside the agency, and lacking a good relationship between two parties. further analysis of the data revealed that the relationships were more harmonious where the agencies had been more selective in the it functions outsourced. gupta and gupta (1992) suggested that one of the reasons for many organizations to outsource was to gain access to the outsourcing vendor’s capability of providing access to the leading-edge technologies. access to skilled it professionals (s10) accessing skilled it professionals through the outsourcing decision was very important. the agencies all had the problem of not being able to recruit skilled it professionals due to their staffing and salary arrangements. agency f claimed that they previously had to employ the particular skilled professionals they needed from a contracting company, had to pay twice the amount of a reasonable it salary for their services, and only employed them for a short period on a needs basis. all the agencies agreed they gained access to a wide range of skilled professionals through their outsourcing service provider. they claimed that it was very hard to run certain applications without an outsourcing vendor because they would not have all the right skilled people available in-house. the outsourcing arrangement helped the agencies to access a pool of it staff that was provided by the outsourcing vendor. however, a and c suggested that it wasn't always easy to access this pool of staff but it was there if the agency urgently needed them. this could be explained as the source being scarce but available when needed, according to the priority of the work to be conducted. brewer (1998) argued that outsourcing would assist organizations with addressing their resource constraints and skill shortages. implementation of emerging technologies (s11) implementing emerging technologies through outsourcing was another success measure. prior to their outsourcing decision, all the agencies had concerns over their inability to use emerging technologies. all the agencies believed that outsourcing would give them the opportunity to access these new and flexible technologies. the analysis revealed that the agencies did gain access to emerging technologies through their outsourcing arrangement. they all claimed that their platform changes were assisted and implemented by their outsourcing vendor during major infrastructure changing processes, such as switching from a mainframe to a networking environment. this could be attributed to the ability of the outsourcing vendor to provide the right skilled staff and the infrastructure. gupta and gupta (1992) argued that technical considerations push companies towards outsourcing their information systems needs rather than meeting them in-house. looking for a strategic partner for it service (s12) all managers interviewed agreed that the open-book type strategic partnership agreement had been their objective. the managers claimed that the current contract was an open-book strategic partnership agreement and was quite successful overall. the agencies had regular meetings at two different levels with the outsourcer: australasian journal of information systems vol 11 no 2 may 2004 11 • contract management level where the contract managers had regular monthly meetings. they openly and collectively discussed unresolved issues with the account manager from the outsourcer. • strategic management level where the information managers and heads of is departments came together every second month and discussed the unresolved issues and contract outcomes with a senior executive from the outsourcing vendor. all of the agencies commented on the importance of these regular meetings, and suggested that they were positively contributing to the success of the contract. the managers interviewed all acknowledged that this mechanism was there to keep the negotiations in place and help both parties to solve problems that couldn't be solved by the staff at a technical level on a daily basis. it was suggested that this mechanism was in place as a result of the partnership approach towards their outsourcing arrangement adopted by both sides. marcolin and mclennan, (1998) suggest that relationships are very important to the success of any outsourcing decision and concluded that their research on effective it outsourcing arrangements demonstrated that the organizations with strategic partnerships were more successful in their arrangement than the buyer/seller type of outsourcing. this may be the case for government agencies as well as in the private sector. compliance with service level agreements (s13) complying with the service level agreements was a very important success measure for all of the agencies. the strategic-level management of the organizations addressed the measurement of the performance of their outsourcing vendor by evaluating how much they complied with their service level agreements. lacity and hirschheim (1995) suggested that in order to complete the contract, service levels, cost structures and penalties for non-performance must be specified and agreed to by both parties. service level agreements play a major role in outsourcing contracts. arnold and butler (1995) also argue that it is normal for a contract to include provision that the supplier will provide the services by meeting the service levels identified in the schedules. they also further suggest that since a commercial relationship exists between the parties, the schedules have to identify what performance levels and standards are required. lacity and hirschheim (1995) also pointed out that identified service levels have to be subject to continuous monitoring and formal periodical review by the parties. other issues the study also discovered that the agencies were affected by the outsourcing decision at different levels. for example while one of the agencies (e) suggested that the effect of the outsourcing decision on its is department was minimal, the other five agencies described the effects from being devastating to substantial. they all measured the effect by the reduction in is personnel in their agency. however, e already had a contractual relationship with a service provider for its payroll system and in-house is service delivery was already minimal. the other agencies didn’t have prior outsourcing arrangements and relied heavily on in-house is service delivery. the study also identified that different agencies within the consortium had different views over the service provider’s performance. while b, d, and f described the service provider's performance as very good, the other agencies claimed that they had several major problems with the service provider. this could be attributed to the different agencies attitudes towards the outsourcing service provider and also the degree of selectiveness of the outsourcing. another reason could be that the government had pressured some to outsource their it functions. the agencies that said that outsourcing was totally their decision, regardless of government pressure, and (despite the problems mentioned above) confirmed that they were very happy overall with the service provider's performance. another important issue pointed out by a manager from agency e was the reporting mechanism that the agency had to deal with. it was claimed that the service provider provided the agency with regular reports of what they have done. by the time the management read and act upon this report it was two months old. during the interview this manager said "we are coming to the end of september, we will almost be in november when we receive the september report, so the validity of australasian journal of information systems vol 11 no 2 may 2004 12 the information we are dealing with becomes a bit grey. we recall from our memory that the issues are valid but it looses a quite a bit of its meaning". clearly, internal procedures have to be reengineered to gain maximum leverage from these arrangements. summary and conclusion outsourcing has become a popular business strategy being implemented by private and government organizations. varied research has been conducted to evaluate the benefits and risks involved in outsourcing. however, most of the studies done in this area have been carried out in the private sector, and very little published work has been conducted in the government sector. this study aimed to explore the effects of contract negotiations between the buyers and suppliers towards their success in government organizations. while there were some time and resource limitations for this case study, it did reveal that cosourcing contracts could be successful. this success could be measured in a number of ways and some thirteen different measures of success were revealed for these co-sourcing arrangements and were generally supported by past research which had been applied to investigating other outsourcing arrangements. this success was dependent on particular critical factors being identified and implemented correctly. some thirteen of these critical success factors were identified from the analysis and these were associated in varying degrees with some thirteen success measures. while many of these factors and measures are supported by previous research, most of this has been conducted in the private sector and this study confirms what the relevance of different factors and measures in a public sector it outsourcing environment. in particular, this research fills a gap for the situation where the outsourcing is a multi-client, single-vendor (co-sourcing) arrangement. references anonymous. (1996) “outsourcers,” government executive, vol. 28, no. 11, pp. 42-44. ang, s. & straub, d. w. (1998) “production and transaction economies and is outsourcing: a study of the u.s. banking industry,” mis quarterly, vol. 22, no. 4, pp. 535-552. arnold, c. & butler, r. (1995) outsourcing contracts: a specially commissioned report, pearson professional ltd, london. aubert, b., patry, m. & rivard, s. (1998) “assessing the risk of it outsourcing,” proceedings of the 31 st annual hawaii international conference on system sciences, hawaii, january 5-8, pp. 685-692. brewer, r. (1998) “understanding what outsourcing is and is not and what this will mean to your business or organization,” proceedings of second wave outsourcing and service level agreements in it, business law education centre, sydney, pp. 1-44. de looff, l. (1997) information systems outsourcing decision making: a managerial approach, idea group publishing, london. galliers, r.d. (1991) “choosing appropriate information systems research methodologies,” in nissen, h.e., klein, h.k., & hirschheim, r. (eds.) information systems research: contemporary approaches and emergent traditions, new-holland, amsterdam, pp. 327346. galliers, r. d., leidner, d.e. & baker, b.s.h. (1999) strategic information management: challenges and strategies in managing information systems, 2 nd ed., butterworthheinemann, oxford. gupta, g. & gupta, a. (1992) “outsourcing the is function: is it necessary for your organization?” information systems management, vol. 9, no. 3, pp 44-50. jones, w. (1997) “successful outsourcing”, cio perspective conference: core competencies and competitive advantage, october 12-15, online available: http://www.cio.com/conferences/1097_concurrent_c.html . kelley, b. (1995) “outsourcing marches on,” journal of business strategy, vol. 16, no. 4, pp. 38-42. khosrowpour, m. (1995) managing information technology investments with outsourcing, idea group publishing, london. australasian journal of information systems vol 11 no 2 may 2004 13 lacity, m.c. & hirschheim, r. (1995) beyond the information systems outsourcing bandwagon: the insourcing response. john wiley & sons ltd, england. lacity, m. & willcocks, l. (1996) “interpreting information technology sourcing decisions from a transaction cost perspective: findings and critique,” accounting, management & information technology, vol. 5, no. 3/4, pp. 203-244. lacity, m. & willcocks, l. (1998) “an empirical investigation of information technology sourcing practices: lessons from experience,” mis quarterly, vol. 22, no. 3, pp. 363-408. lacity, m., willcocks, l. & feeny, d. (1996) “the value of selective it sourcing,” sloan management review, spring, pp. 13 25. marcolin, b.l. & mclellan, k.l. (1998) “effective it outsourcing arrangements,” proceedings of the 31 st hawaii international conference on the systems sciences, hawaii, january 5-8, pp. 654-665. mcfarlan, f. w. & nolan, r.l. (1995) “how to mange an it outsourcing alliance”, sloan management review, winter, pp. 9-23. mcfarlane, p. (1998) “managing the relationship with the service provider,” proceedings of second wave outsourcing and service level agreements in it, business law education centre, sydney, pp. i-vi. neuman, w.l. (2000) social research methods: qualitative and quantitative approaches, 4 th ed., pearson education, usa. perigo, j. (1998) “critical success factors in is/it market testing and outsourcing.,” available [online]: http://policyworks.gov/org/main/mg/intergov/letter/perigo.html. rothery, b. & robertson, i. (1995) the truth about outsourcing, gower publishing limited, england. sabherwal, r. (1999) “the role of trust in outsourced is development projects”, communications of the acm, vol. 42, no. 2, pp. 80-86. tarsh, s. (1999) “managing the outsourcing relationship: a shared vision produces greater rewards,” available [online] http://www.outsourcing-search.com/resources.html . williams, o. (1998) outsourcing. a cio's perspective. st. lucie press, ohio. microsoft word ajisfinal 12 2 marina.doc ajis volume 12 no. 2 may 2005 90 grounded theory and user requirements: a challenge for qualitative research supriya singh kylie cassar bartolo christine satchell rmit university/smart internet technology cooperative research centre research development unit, rmit business melbourne, vic 3000 email: supriya.singh@rmit.edu.au abstract we discuss the methodological implications of moving from grounded theory to user requirements for the design of information and communication technologies. this is a problem that is particularly acute for sociology, where theory is seen as a sufficient contribution to knowledge. cultural theorists have potentially less of a problem moving from the cultural meaning of artefacts to design. the epistemological and methodological shifts are also narrower for the applied sciences. we submit the frameworks and sequencing of the open-ended interview need to be re-structured to ground both theory and user requirements. this is a sounder basis for detailing current and future user requirements from a social perspective. introduction we place the paper within three contexts: our user-centred design work in the smart internet technology cooperative research centre in australia; the grounded theory approach; and the need to translate from theory to user requirements for design. this is done in the first section. in the second section we reflect on how we conducted a qualitative study about how people control their personal information and identities in relation to activities. in the third section, we move to the young people study, detailing the use of the scenario prototype. in the fourth section we outline our approach aimed at moving beyond grounded theory to grounded design. in the fifth section we explore the shift in epistemology and methodology for sociology, applied science and cultural theory. we conclude by drawing the threads together and showing how the suggested approach can bring rigour to design within a compressed time frame. sitcrc and discovery ucd smart internet technology cooperative research centre (sitcrc) in australia brings together 12 australian universities and three main corporate partners. the research program at present comprises four technical streams focusing on smart personal agents, natural adaptive user interfaces, intelligent environments and smart virtual networks. the user environment (ue) program overlays these technical streams to ensure the user is at the centre of the design from the very beginning. the usercentred design (ucd) team is part of the ue program. the aim is to develop useful and commercialisable technologies. hence in sitcrc, we work at the intersections of the social, technological and business perspectives in this paper we report on two projects where the ucd team is working with technologists and business partners. the secure identity management project aims to design a mobile device that incorporates users’ control of personal information and identity. the ucd team conducted 17 openended interviews in australia to see how people control personal information. the qualitative user study was supplemented by a quantitative study of 1010 broadband users. it was envisaged these ajis volume 12 no. 2 may 2005 91 studies would feed into the user requirements of the mobile device. the second is the smart internet technology architecture project. it has a swarm phone at the centre of the architecture. the swarm phone concept was based on a qualitative study of young people’s use of mobile phones (satchell 2003). in both projects we are still at the discovery stage of ucd (singh et al. 2003). a mock-up of a possible design is still to be constructed. we are at the point where discussions with technologists are continuing. though all design projects are distinctive, our projects differ from those usually described in the ucd literature in two ways. firstly, we are working outside a single organizational context, with our members drawn from academia and industry in three australian cities. it has made possible a more extensive user study than is usually found in strictly commercial projects. however, there has been a sharper division of labour than we would have liked, between the ucd and technical people of our projects. in this context, reflection about methodology aids the process of team formation. this is also important within the ucd team, which comprises a social anthropologist, a psychologist and a communications researcher. in the ucd team we have worked closely with one of our business partners who is also a psychologist. secondly, in both projects we are considering issues like “privacy”, “identity” and “social networks” which go beyond the organizational boundaries implicit in much of ucd and human computer interaction (hci) literature. the social and cultural context becomes an important framework for the analysis of attitudes and behaviour. so in working out a methodology that has relevance for design, we need to be participant observers of our own project (singh et al. 2004) grounded theory grounded theory emphasises the fit between data and the emerging theory, rather than moving deductively down from a prior hypothesis (glaser 1978; glaser and strauss 1967; strauss and corbin 1990). the emergence of theory from data is one of the defining elements of qualitative research as morse and richards (2002) say, “all qualitative research seeks understanding of data that are complex and can be approached only in context" (morse and richards 2002). the literature on grounded theory differs as to how the analysis helps themes to emerge from the data. we have detailed our approach as we discuss the analysis of data in our two projects. following charmaz (2000), we do not use grounded theory to discover an objective reality, but as "flexible, heuristic strategies" (charmaz 2000, p. 510)to understand and interpret empirical worlds in their multiplicity and relativity. the translation process the focus on translation is important, for the grounded study of use does not necessarily lead to viable design concepts. as beyer and holtzblatt (1998) say, “design is a cognitive activity. it is thought work. it begins with a creative leap from customer data to the implications for design and from implications to ideas for specific features. a clear understanding of the customer doesn't guarantee any kind of useful system gets designed and delivered. design depends on being able to see the implications of data" (beyer & holtzblatt, 1998, p. 16). this opaqueness between user data, design, and business value is because of three reasons. firstly, users do not always know what they want. karen holtzblatt tells how she wanted windows in her den. but what she really wanted was a lot of light to see the garden (beyer & holtzblatt, 1998). the second reason for the lack of seamlessness in the process is that there is often a division of labour – the user study being done by social scientists while the design concepts are in the hands of the technologists (button, 2003). the business dimension often comes later. the third difficulty of ajis volume 12 no. 2 may 2005 92 translating user data into design is the “socio-technical gap” (ackerman, 2002). ackerman says one of the challenges of human computer interaction (hci) is to build technical systems that support the social world in all its nuances and contexts. the user study: secure identity management project in this section we reflect on how we conducted a qualitative study about the way people control their personal information and manage their identities for different activities within a social and cultural context. our activity focus shared much with “activity theory” which has its origins in psychology. we have the same emphasis on a “web of activities” (bertelsen and bødker 2003), rather than assume a linear, sequential structure. we also drew on studies of the social shaping of technology to investigate how technologies shape and are shaped by social relations and cultural values. it was clear from the outset we were conducting the user study to feed into the design of mobile devices with privacy mechanisms. however, we followed standard grounded theory procedures both in the conduct of the interview and the analysis of data for emerging theory. this meant that understandings of emerging theory were confirmed with participants during the interview and analysis. the interviews were transcribed and coded, using n6, a computer program for the analysis of qualitative data. as we were working in a team and came from different perspectives, we did broad coding. we coded at the activity and technology levels. the data were further analysed through the use of matrices. it is at this level that detailed patterns in the data emerged and led to initial explanations of the control of personal information. the grounded theory approach to analysis emphasises transcription and coding. this means that an hour of interview takes roughly four hours of transcription and around ten hours of analysis. it is a time intensive way of doing user research, particularly if we add the required meetings within the ucd team, and meetings with our technological and business partners. we thought the resulting rigour of the theory was worth it. firstly our study went deeper into the users’ perspective of the control of personal information than policy directed surveys, or technological studies of privacy. policy directed surveys tend to emphasise the importance of the value of privacy, without necessarily illustrating how people deal with information about different activities in various contexts. technological studies of privacy emphasise the need for withholding disclosure with pseudonymity and anonymity being along the spectrum of privacy. in contrast to previous studies, our user study led us to three main theoretical conclusions that are important for the design of all information and communication technologies (singh and cassarbartolo 2004). the first was that people saw privacy was about sharing personal information with people of their choice using the channels they preferred for the particular activity and audience. secondly, people shared personal information relating to money and health in different ways within the same social and cultural context. these conclusions became apparent during the data collection and analysis stages. we had particularly asked about control of personal information for different activities. as money and health emerged as the two of the most private areas, the focus was on how they shared information with their partners, close friends and family and others. the third cluster of theory revolved around connecting yet disentangling the concepts of privacy, identity, security and trust. the link was that of “user control”. however the centrality of user control not only for privacy and identity, but also for security and trust, emerged only in the analysis stage. hence there was “missing data” about the precise nature of that control. missing data however is a common occurrence in qualitative research (singh and richards 2003) and indeed in all scientific research where one problem leads to the articulation of another (popper 1979). it was another kind of missing data with which we became more concerned. we had not tested the scenarios and user requirements – required for design workshops with the technologists in the ajis volume 12 no. 2 may 2005 93 project team – in the same way that we had sought to confirm our understandings of theory. we had sufficient data from the user study to flesh out personas with sufficient knowledge behind us. however, the process of moving from data to scenario revealed gaps. these gaps were filled by previous work, particularly in the field of money and health. we came to feel we could have elicited more grounded data on the future design of mobile devices in the interviews. recognising it is difficult to get people to speak of future design, we felt we could at least have asked them what they liked about current design, particularly as it related to user control. presenting scenarios to them, incorporating their own experience, may have been one way of getting more valuable data on user requirements. of course, more interviews or focus groups could have been conducted addressing these problems, but there were time and resource constraints. let us illustrate how we translated the user data to scenarios. this process meant going from the broad conclusions about personal information to how one person dealt with his or her personal information in a particular activity and social context. at the broadest level, our theory was that people want to control their personal information in terms of channel, audience, content, and context. personal communication, finance and health are particularly sensitive areas in anglo-celtic australia. yet people control the information differently according to activity. in terms of channel, the telephone is seen as the most private digital device with the web being the least private. these conclusions are too broad and general for design or business. they are valuable for without this kind of information, designers run the risk of designing uniformly for different activities. however, by itself alone, this kind of theory does not lead to a business or design outcome. we need to dig narrower to answer the unasked “so what?” question. our first instinct was to place the emerging findings in one column of the table and side by side present the design and business implications. but we found we needed to re-cast the scope of the findings to come to usable starting points. scenarios not only worked as a communication tool, but also as a mode of translation. this is because the scenario presents the persona, the context and the way an activity is performed. it then extracts the user problem, and suggests a possible design approach. in this user study, for the privacy of money, we focused on joe, 65, who pays his bills via the phone. we had to go through four steps. first, we narrowed the wider field of money to payments and money management. it is important to have this “web of activities” for the success of a payments technology often depends on its success for money management. second, the telephone was chosen as the channel from all the possible options, not only because the phone is seen as most private, but because the emphasis of the project was on mobile devices. paying by phone was also pertinent to our business partners. third, we articulated the user problem that paying bills by phone is repetitive and time consuming. fourth, we put forward the design suggestion that the customer number and credit card details relevant for specified merchants be stored, to automate part of the process. we noted that to ensure control, the user needs to confirm the transaction for it to be valid. we also suggested that an e-mail be sent so that a paper record is possible. the format of this record should be compatible with personal financial management programs. the scenario captures only one part of the user study and its conclusions. in capturing the specifics of one persona and activity; elucidating the user problem; and suggesting an acceptable design solution – the user study becomes relevant for designers and businesses. the two processes however differed in terms of rigour. it was as if we made an intuitive jump to go from theory to design. we could not point to the data that underlay the precise detail of the scenario. some of the detail came from everyday knowledge. some of it came from prior study of money and channels of communication. however, it would have been more in the character of a rigorous user study if the scenario and user requirements had been a parallel subject of conversation and examination, when the interviews were being conducted. ajis volume 12 no. 2 may 2005 94 prio r theory and literature open ended in terviews – grounded theory of use analysis stage – g rounded theory, prelim inary scenarios and social user requirements w orkshopped ideas w ith technologists and business partners prototype participatory design figure 1: open-ended interviews: process of moving from theory to design young people study: user study and the scenario prototype the young people study had a different sequence. it was conducted as part of a phd. like the user study for the secure identity management project, this was a grounded study that produced theory about young people's use of information and communication technologies. the methods for theory production were as described in the section above. discussions with technologists and business partners began after data collection and preliminary theory analysis. scenarios were presented to technologists at workshops and repeatedly modified by going back to a select group of users, and then back to the technologists. the swarm scenario was returned to selected users in the study three times over an 18 month period. this initially happened to ensure the rigour of the scenario. it was interesting to note that the scenario generated in-depth user responses, and produced a high level of user feedback. the scenario thus was effective not only for communicating with computer scientists but as means of gathering extra user requirements, thereby filling in the gaps left by the open-ended interviews. each time the swarm was modified to reflect their needs. for example, the users’ initial desire was to be able to control who called them and when, this lead to the development of avatars so that a digital representation of could be programmed to act for the user. yet when revisiting the scenario the users stated that while this level of control was a desirable feature it should be just that a feature. there were times when you wanted a lot of control over who called you but at other times you didn’t want to have to have that level of involvement with the phone. hence, the swarm is now being modified with the addition of a default mode that indicates the user as either at work/university/school or at home. the modified scenarios were then taken back to the technologists. the discussion then was to convert the social user requirements to technical user requirements and then a prototype. the process is depicted in figure 2. the process has worked well in that the scenarios generated by this user study for a swarm phone have become the key scenarios in at least two of the technological projects. but we are also conscious that at least a year's work had been completed in the user study ajis volume 12 no. 2 may 2005 95 and that there had been sufficient attention paid to theory before the work on scenarios began. this kind of extensive user study is possible within a three-year time span, but is unlikely to be the norm for most user-centred design. prior theory and literature open ended interviews – grounded theory of use analysis stage – grounded theory and preliminary scenarios and social user requirements workshopped ideas with technologists and business partners go back to users with scenario prototype workshop modified sp with technologists and business partners prototype participatory design figure 2: the open-ended interview and the scenario prototype grounded theory and user requirements: the new approach based on our experience we suggest two modifications to qualitative research which aims to lead to both grounded theory and grounded user requirements design. firstly, the preparation for the qualitative research should not only include prior theory, but also some preliminary scenarios. this will enable the interviewer to follow up theory understandings with a confirmation of the adequacy of the preliminary scenarios. it is difficult for a person to articulate what he or she would like in the future from a technology that is still to be designed. showing a person a mock-up of future technology, can lead the participant to couch all the answers in that framework. however when a story is presented to the interviewee, it is easier to say whether the story rings true or not. in the discussion, the story gets modified, so that it is true to the social context of the activity. the scenario is also a way of communicating user understandings to the technologists and business partners, so that they keep the users at the centre of design. the second difference is that within the same open-ended interview, the interviewer checks the participant's view of the user requirements for future design. if a research objective of ucd is evidence-based applications, then the exploration and validation of applications must be an integral part of data collection. this will mean checking with the participant, his or her requirements of desirable design. it would add another layer to the interview process to end up with social user ajis volume 12 no. 2 may 2005 96 requirements. it will cut down the need to go back repeatedly to users, before the design of the prototype. this will help with the timeliness of the user study. we outline the process in figure 3. in figure 4 we summarise the difference between the process followed in the two user studies described above and the process we are suggesting in this section. these changes of process will require a shift in methodology and epistemology. we see the main difficulty is to overcome social science perspectives to go beyond theory to application at the broadest level. p r i o r t h e o r y a n d l i t e r a t u r e p r e l im i n a r y s c e n a r i o s o f u s e o p e n e n d e d i n t e r v i e w s – g r o u n d e d t h e o r y o f u s e , g r o u n d e d s c e n a r i o s a n d s o c i a l u s e r r e q u i r e m e n t s w o r k s h o p p i n g s c e n a r i o p r o t o t y p e s w i t h t e c h n o l o g i s t s a n d b u s i n e s s p a r t n e r s p r o t o t y p e p a r t i c i p a t o r y d e s i g n figure 3: qualitative research: grounded theory and requirements qualitative process the secure identity project the young people study grounded theory and requirements approach preparing for interviews prior theory and literature prior theory and literature prior theory and literature preliminary scenarios of use data collection via open-ended interviews grounded theory of use grounded theory of use grounded theory of use, scenarios and preliminary social user requirements data analysis theory, preliminary scenarios and social user requirements theory, preliminary scenarios and social user requirements grounded theory, grounded scenarios and social user requirements workshops with technologists scenarios, connecting social and technical user requirements scenarios, connecting social and technical user requirements scenario prototypes, connecting social and technical user requirements going back to users not reached that stage yet present and modify scenario prototypes not required ajis volume 12 no. 2 may 2005 97 prototype and participatory design going back to users with technologists observing going back to users with technologists observing going back to users with technologists observing figure 4: grounded theory and user requirements shift in epistemology and methodology the new grounded theory and user requirements approach requires a continuum between theory and practice. both theory and practice are important. theory can lead to a better concept of practice and exploration of practical applications can lead to a better theory. however, this connection between theory and practice is not automatic for each has a different tradition of knowing. this gap is particularly evident in the disciplinary frameworks of sociology and anthropology. it is difficult to reconcile the social science focus on theory to the particularities of the current and future design of technologies. the gap between theory and practice is also present in the applied sciences like engineering and psychology. though there is an assumption within the applied sciences that sound theoretical conclusions will lead to good practice, bridging theory and practice is a discrete and conscious step in itself. cultural theory has seldom been used for design but with its emphasis on meanings of artefacts slides more easily between theory and design. sociology’s tussle with ucd the aim of sociological studies is to arrive at rigorous theory, which can explain social phenomenon. there has been important work which has argued that technologies shape and are shaped by social relations and cultural values (mackenzie and wajcman 1999). another stream of sociology has dealt with situated action and everyday life (goffman 1959 ,suchman 1987). these frameworks are valuable for the design of technologies, for they focus on the way technologies are used within social and cultural contexts. it is a needed corrective to much of the human computer interaction (hci) literature where the focus is often on an individual interacting with a machine. the empirical traditions of sociology and anthropology have been used with good effect in hci to present the context of use, as with contextual inquiry (beyer and holtzblatt 1998). but as button (2003) says, particularly of the ethnographic approach in studying work practices, “ethnography in hci has really stood as a proxy for 'fieldwork,' grounded, we could add, in ethnomethodology" (p. 364). fieldwork in hci and anthropology try and get an understanding of the worlds of the user. however, in hci, the emphasis is on getting a quick appreciation of the context, whereas anthropology demands a long immersion and at the same time, a distance from the people being studied. while some observation is better than none, as carroll (2003) says, there is currently a prevalence of "quick and dirty" (p. 1) ethnography, without enough theoretical reflection. though the methods of anthropological inquiry have been embraced by hci, with looser standards of rigour, the theoretical frameworks of anthropology do not translate so easily to design. in anthropology, there remains a colonial bias against telling people how they should live. when computer scientists sommerville, rodden and sawyer reflected on their collaboration with sociologists in a project on automating aspects of an airport system, they said sociologists had contributed to the technological approach to systems development by being ….the users' champion and the designer's conscience. as the users' champion, the sociologists tell developers about the actual work practice and the users' perception of this work practice. this input is provided formally by the development of field notes which document the observations of the sociologists and less formally by the use of debriefing meetings where system developers question the ethnographer. " (p. 346) ajis volume 12 no. 2 may 2005 98 they also identified three problems. the first was that sociologists were more used to asking questions and observing, than formulating solutions. as (sommerville et al. 1992)) say, sociologists are not usually concerned with discovering improved ways of carrying out a task, with devising techniques and methods for supporting a particular set of behaviours or with inventing new ways of tackling a particular problem. they work very hard to avoid being classed as social engineers who pose 'solutions' to observed social problems (p. 351). the second was the time-consuming aspect of interacting with sociologists through their field notes and workshops. the third was that sociologists concentrated on the whole, whereas computer scientists needed to make decisions about individual tasks that should or should not be supported in the automated system. they say the key to building better systems is not to avoid making judgements but to bring in appropriate expertise so that decisions are better informed. our sociologist collaborators have now recognised that we can only build imperfect systems and that they cannot opt out of making judgements about the process which they are studying (p. 347). sommerville et al’s collaboration with sociologists was fruitful overall. however, in the process, sociologists had to give up some of their long held ways of working, so as to contribute to the emerging design. their approach was to work in parallel, with both the design and the user study emerging at the same time. applied sciences: the case of psychology and engineering practice is the desired outcome of good theory in the applied sciences. of central concern to practicing psychologists is providing efficacious psychological services. hence the dominant rationale underlying psychological research is the scientist-practitioner model − evaluating theory and applications against client outcomes and good practice. in applied psychology there is an acknowledgement that practice needs to have an empirical basis. however the path between research conclusions and psychologists applying evidence-based practices remains problematic. recently there has been a questioning of the scientist-practitioner model – does it polarise psychological theory and practice? is there a need for another more process-focused model to clarify how psychologists can learn from research conclusions and apply evidence-based practices (see lunt 1994, 2000)? in order to explore and validate design applications there needs to be a shift in epistemology. a difficulty with this more refined epistemology of practice in engineering is the split between theorists and technological designers. schmid (2001) says, theorists and designers have two fundamentally different thinking styles. for a designer, every new project starts with chaos, he or she is used to gaining experience until he or she achieves sufficient…[techniques]…to solve the problem. for the designer, immediate appearance, sensory evidence, and experience are the most important sources of knowledge, and theories are just tools, like maps to a hill-walker, which give a rough impression of where to go but do not tell when and how one has to jump over a rock...theorists are different. for them, the theories are the most important elements of science, and their most valued sources of knowledge are tradition, analogy, theoretical beauty, and logical reasoning. they are like birds, for whom the map used by the hillwalker is more than detailed enough, and who only dive down to look at details in places that look promising on the map. they often do a systematic search for such places. if theorists discuss design, they start from theories and incorporate more and more nonidealities to approach reality (schmid 2001, p.109). ajis volume 12 no. 2 may 2005 99 cultural theory cultural theory is rarely used for design and is not known for its empirical traditions. however, when cultural theory frameworks are used for empirical work, our work shows that cultural theory provides a useful framework for conducting and analyzing user studies and then translating the findings into design. this is because cultural theory is not only concerned with how the users interact with technologies but of the meanings that these interactions brings. there is a shared agreement amongst cultural theorists that technology is one of the defining discourses of our time, regardless of whether they are utopian (haraway 2001) or distopian (baudrillard 1983, 1995). both critiques place much importance on the power of the technological artefact as a cultural signifier so the incredible range of meanings that interactions with technologies brings are explored. cultural theory also provides a framework for understanding how users can be liberated through their interactions with digital technology. this is because they examine how digital technologies collapse currently held notions of time and space (baudrillard 1995, jameson 1991, rheingold 1993) allowing the user to freely traverse digital landscapes (deleuze and guatarri 1987). furthermore, haraway provides an understanding of how new technologies provide a space where users can recreate their identity free from the constructs of western ideas of beauty, gender and sexuality. when it comes to translating user studies into design, cultural theory provides insights into not only what users are doing but why they are doing it. it helps construct scenarios which envision a technological artefact that allows the users achieve not only their technical but their cultural goals. in the case of the swarm scenario discussed above, it meant that the scenario was constructed to meet not only the users’ technological needs but their cultural ones as well. for example, the data from the young people study revealed that users texted their friends constantly and even when they had nothing specific to say, they just liked to broadcast that they were engaged in a specific activity. a cultural theory analysis helps to understand this in terms of the nature of virtual worlds, (rheingold, 1993) where the boundaries between real and virtual have been eroded and a virtual interaction can be just as rewarding as a face-to-face interaction. thinking of user behaviours in this way lead to the development of the swarm scenario which depicted a mobile phone that had a virtual lounge room in it where users maintained a constant virtual presence. as the virtual lounge room was populated with avatars that depicted them as being involved in a particular activity, users are always able to communicate to others what they doing. the decision to use avatars was arrived at through cultural theory. users in the study mentioned that one of the fun parts of gaming was choosing an avatar through which to represent yourself. at face value this might not have seemed to have design implications for the swarm. the decision however, was influenced by haraway’s argument that the benefit of new technology is that you can re-construct your identity as you choose. another example was that users in the study revealed they liked to share digital pictures. again a cultural theory framework helped understand this concept in terms of baudrillard’s (1983) hyper reality, in that the creation, distribution and consumption of an experience can be as valuable if not more valuable than the experience itself. therefore the swarm scenario depicts users being able to use the video features on the phone to capture images and display them on the walls of their virtual lounge room, so those who are not there can still share in the experience. conclusion our challenge in ucd is to have theory and design inform each other, to produce rigorous research and effective and timely design. this is our problem. addressing this problem is no easy feat for it needs shifts in epistemology and methodology. epistemologically, we combine the approaches ajis volume 12 no. 2 may 2005 100 from multiple areas of study so that theory slides into practice. • sociology provides a means to build detailed relevant theory, particularly understandings of social contexts. what is needed is an emphasis on design. • the applied sciences provide the beginnings of research method that empirically connects theory to practice. • studies of cultural theory possibly provide a theoretical basis that translates more easily to design, specifically understanding the meanings of cultural artefacts for people. • human computer interaction addresses design issues directly. methodologically, we have sought to combine the empirical techniques of sociology and anthropology with a more grounded approach, not only to theory but to design. we have done this by combining the body of knowledge about qualitative research with the value of scenarios and personas as a methodology for design. these different disciplinary frameworks lead us to a view of qualitative research which provides grounded theory, and a way of situating design via dialogue within the world of the user. our grounded theory and user requirements approach synthesises these frameworks and techniques in the following two components: • preparation for the qualitative research should not only include prior theory, but also some preliminary scenarios. the interviewer can confirm the adequacy of preliminary scenarios with the user. • within the one-off open-ended interview, the interviewer checks the participant's view of the user requirements for future design to explore and validate design applications during the stage of data collection. this will help with the timeliness of the user study. if the objective is rigorous design and research, it is not enough to be a bird or a hill-walker. one needs to be an aviation explorer, willing to capture the bird’s eye view and dive down to walk and get a closer experiential look. references ackerman, m. s. (2002). the intellectual challenge of cscw: the gap between social requirements and technical feasibility. in j. m. carroll (ed.), human-computer interaction in the new millennium (pp. 303-324). new york: acm press. baudrillard, j. (1983). the ecstasy of communication in h forster, ed, the anti aesthetic essays on post modern culture. seattle: bay press. baudrillard, j. (1985) selected writings. 1985. in jean baudrillard selected writings. ed. m. poster. cambridge: polity. baudrillard, j. (1995). the gulf war did not take place. trans. p. parron. new south wales: power publications. bertelsen, o. w. and bødker, s. (2003). activity theory. in j. m. carroll (ed.), hci models, theories, and frameworks: toward a multidisciplinary science (pp. 291-356). beyer, h., and holtzblatt, k. (1998). contextual design: defining customer-centered systems. san francisco: morgan kaufmann publishers, inc. button, g. (2003). studies of work in human-computer interaction. in j. m. carroll (ed.), hci models, theories, and frameworks: toward a multidisciplinary science (pp. 357-380). sydney: morgan kaufmann publishers. carroll, j. m. (2003). introduction: toward a multidisciplinary science of human-computer interaction. in j. m. carroll (ed.), hci models, theories and frameworks: toward a ajis volume 12 no. 2 may 2005 101 multidisciplinary science (pp. 1-10). sydney: morgan kaufmann publishers. charmaz, c. (2000). grounded theory: objectivist and constructivist methods. in n. k. denzin and y. s. lincoln (eds.), handbook of qualitative research (second ed., pp. 509-535). london: sage publications. deleuze, g. and guattari, f. (1987). a thousand plateaus; capitalism and schizophrenia. translated by b. massumi. minneapolis:university of minnesota press. glaser, b. g. (1978) theoretical sensitivity: advances in the methodology of grounded theory, sociology press, mill valley, ca. glaser, b. g. and strauss, a. l. (1967) the discovery of grounded theory: strategies for qualitative research, aldine, chicago. goffman, erving. (1959) the presentation of self in everyday life. garden city, new york: doubleday. haraway, donna (1991). simians, cyborgs and women; the reinvention of nature. london: free association books. jameson, f. 1991. post modernism, or, the cultural logic of late capitalism. verso. london. lewis, j. (2002) cultural studies. the basics, sage publications ltd, london. lunt, i. and majors. k. (2000), the professionalism of educational psychology, educational psychology in practice, 15(4), 237-35. lunt, i. (1994). science and practice in the education and training of psychologists. paper presented at the 23 rd international congress of applied psychology 1994, madrid, 17 th -22 nd of july. mackenzie, d., and wajcman, j. (eds.). (1999). the social shaping of technology (second ed.). buckingham: open university press. mao, j.y., vredenburg, k., smith, p. w., and carey, t. (2001). user-centered design methods in practice: a survey of the state of the art. paper presented at the center for advanced studies conference, http://www.cas.ibm.com/archives/2001/proceed/cascon01/pdf/mao.pdf on 24/2/2002. morse, j. m. and richards, l. (2002) readme first for a user's guide to qualitative methods, sage publications, thousand oaks, calif. popper, k. (1979). objective knowledge: an evolutionary approach. revised edition. oxford: oxford university press. rheingold, h. 1993. the virtual community: homesteading on the virtual frontier. addison wesley. reading, ma satchell, c. (2003) the swarm: facilitating fluidity and control in young people's use of mobile phones. in ozchi brisbane, australia. satchell, c. and singh, s. (2004) user problem design solution swarms for nomads. work in progress. schmid, h. (2001) theory and practice: thinking styles in engineering and science. australian journal of information systems, special issue on knowledge management, 106-115. singh, s., burke, j., et al. (2003). the discovery phase of user-centred design: putting users first in the design of smart internet technologies. the australasian conference on information systems, perth, school of management information systems, edith cowan university. singh, s. and cassar-bartolo, k. (2004) the privacy of money and health, paper to be presented at the ozchi, wollongong, november 22-24. singh, s. and richards, l. (2003) missing data: finding 'central' themes in qualitative research, qualitative research journal, 3,(1), 5-17. singh, s. and zic, j. and satchell, c. and bartolo, k. c. and snare, j. and fabre, j. (2004) a ajis volume 12 no. 2 may 2005 102 reflection on translation issues in user-centred design, paper presented at the 7th international conference on work with computing systems, wwcs 2004, kuala lumpur. sommerville, i. and rodden, t. and sawyer, p. (1992) sociologists can be surprisingly useful in interactive systems design, paper presented at the hci'92. strauss, a. and corbin, j. (1990) basics of qualitative research: grounded theory procedures and techniques, sage publications, newbury park, calif suchman, l. (1987). plans and situated actions. cambridge: cambridge university press. acknowledgements we would like to acknowledge the support of our colleagues at the smart internet technology cooperative research centre. in particular, we would like to acknowledge the help of dr john zic who lead the secure identity management project. we would also like to note the importance of the discussions with dr. john fabre from telstra and john snare from adacel who were also part of the secure identity management project. copyright supriya singh, kylie cassar bartolo and christine satchell © 2004. the author/s assign griffith university a non-exclusive license to use this document for personal use provided that the article is used in full and this copyright statement is reproduced. the authors also grant a non-exclusive license to griffith university to publish this document in full in the conference proceedings. such documents may be published on the world wide web, cd-rom, in printed form, and on mirror sites on the world wide web. any other usage is prohibited without the express permission of the authors. the australasian journal of information systems (ajis) began in 1993 and was based on the simple realisation that there was a need for an academic ‘outlet’ for is research in australia australasian journal of information systems volume 14 number 1 nov 2006 the 2005 survey of information systems research in australia graham pervan school of information systems, curtin university of technology, perth. pervang@cbs.curtin.edu.au graeme shanks faculty of information technology, monash university, victoria. graeme.shanks@infotech.monash.edu.au abstract as part of a study to investigate the state of information systems research in australia, a survey of the heads of all is discipline groups in australian universities was conducted in mid 2005. the study revealed a wide range of topics researched (with rapid growth in electronic commerce and knowledge management), a range of foci, a balance between positivist and interpretivist research, survey was the most frequently used research method, and most research was directed at informing is professionals. a swot analysis identified the growing importance of industry relevance and collaboration. introduction the first academic programs in information systems (is) appeared in australia in the late 1960s and have steadily grown to be available in almost all australian universities. while the teaching of is has grown, the growth of is research has been slower and few studies have examined its progress. ridley et al. (1998) studied publication performance over a seven-year period, but there has been no formal examination of the research profile of is in australian universities. this paper investigates the australian is research field along lines similar to part of the study conducted by avgerou et al. (1999) in europe, except that it focuses only on research, and is based on an earlier study by pervan and cecez-kecmanovic (2001). the study targeted the views of the heads of discipline from australian is groups and was conducted on behalf of the australian council of professors and heads of information systems (acphis). 273 australasian journal of information systems volume 14 number 1 nov 2006 research approach in order to investigate the state of is research in australia we conducted a survey in mid 2005. we focused on the “school” level (where “school” represents a group of people primarily focused on teaching and researching is). the group of target respondents expected to represent these schools was the head of discipline for each of the groups identified by the information systems heads of department (ishods) mailing list. based on the previous study by avgerou et al. (1999) and aspects of paradigm and method from neuman (2006), a number of dimensions of the schools’ research activities were identified and incorporated into the original questionnaire. the survey had been conducted twice previously (in 2000 and 2004) and the questions had been progressively refined to the current instrument. the survey contained questions on: • people – number, level and research activity of staff, number of enrolled and completing ph.d. students • structures – school structural titles, actual names, and super-organisations • foci – topics of research interest, unit of analysis, human-technology spectrum, beneficiaries of the research • paradigm – positivist, interpretivist or critical • methods – survey, case study, action research, laboratory experiment, etc. • performance – publication output, research funds obtained, collaboration in addition, a brief swot (strengths, weaknesses, opportunities, threats) analysis of australian is research was added to the questionnaire. as indicated, the target group was the groups on the ishods mailing list (one response required from the head of discipline of each group) and the survey was distributed and received via email. results the questionnaires were distributed electronically and, after some follow-up, 24 responses were received (estimated 60% response rate). there is a mix of titles (most commonly schools, but also departments and other titles), but hereafter the groupings will be referred to as schools. the discipline titles of these schools of the 24 respondents were 50% (12 respondents) information systems, 25% information technology, and 25% other titles. furthermore, 14 of these schools were in a business/commerce faculty, 7 in an information technology faculty, and 3 in a mix of others. their average school size (not including sessional staff) was 19.2 academic staff (median 17) of which 17.2 were continuing and 2.0 on contact. the academic levels represented were (on average) 1.4 professors, 1.7 associate professors, 5.3 senior lecturers, 7.9 lecturer bs, 2.1 lecturer as, and 0.7 research fellows. the overall view is that there is a mix of names and locations for is schools, but the majority are fairly large groups with is in the name and reside in a business/commerce faculty where they can maintain a close association with the areas of application of information systems. on average 15.3 (80%) staff were regarded by their heads as being “research active”, though only 6.3 (33%) were research active according to the strict dest definition (research active staff satisfy at least two of: (a) external research income at least $5,000, (b) refereed publications (weighted) at least 1.5, (c) supervised hdr to completion, over the previous 3 years). almost half (average 9.3 per group) have phds and over 30% (average 5.2) are doing phds. on average, 6.2 (33%) are supervising higher degree by research students though only about a half of these (3.3) have 274 australasian journal of information systems volume 14 number 1 nov 2006 supervised a phd to completion. there are, on average, 12.4 current fte phd students (8.2 full time, 8.4 part time), but there were only 1.6 graduations per school in the previous year. respondents were asked to indicate the topics of research interest in the past, present and future, and these are summarized in table 1 below, sorted by future topic of interest. the results demonstrate the substantial interest in research on is management and strategy and the organizational implications of is and it, is adoption/diffusion, electronic commerce and knowledge management with almost all groups indicating an interest in these areas. in addition, interest is strong in topics such as is development and business modeling, mobile commerce, and the theoretical underpinnings of is. on the other hand, specific topics and technical issues such as computer and network applications and cscw/groupware are relatively less popular. it should be pointed out, however, that the table reveals how many groups are interested in these topics and does not show how large these groups are. so, further research is needed at the individual researcher level. the respondents were also asked to indicate the usual unit of analysis of their research which was the organisation (22 responses), groups/teams (19), clusters of organisations (19), industry (16), processes/tasks (15), individuals (14), national economy/society (8), and world economy/society (3). clearly, researchers focused most on organisations and the people within them, and significantly less on studying is at the national or global levels. this may represent an opportunity to collaborate with other researchers (e.g., economists) to investigate the impact of is and information technologies on australia’s economy, links with the region, and globally. in terms of research paradigm, responses revealed dominance of a positivist paradigm (in 71% of schools), but the interpretivist paradigm was also often used (54%). the survey data confirmed a growing recognition that is researchers in australia do include research based on non-positivist research paradigms. few mentioned any significant emphasis on research using a critical paradigm, which is also the case at the international level (mingers, 2001). topic past present future topic past present future organizational implications of is&t 19 21 20 human-computer interaction 11 11 10 is management/ strategy 17 17 16 systems development 13 10 9 electronic commerce 19 18 16 knowledge-based/ expert systems 11 12 9 is adoption/diffusion 16 16 15 economic effects of is&t 5 8 8 knowledge management 13 17 14 databases 9 9 8 theoretical underpinnings of is 15 14 13 dss/eis/data warehousing 9 9 7 is development methods 12 12 13 is outsourcing/ offshoring 5 7 7 mobile commerce 7 13 13 legal/ethical aspects of is&t 8 8 7 business modeling 10 13 13 computer & network applications 3 3 3 societal effects of is&t 10 12 11 cscw/groupware 6 3 3 is security 8 9 10 table 1: is research topics 275 australasian journal of information systems volume 14 number 1 nov 2006 when asked to indicate the specific research methods used, the responses reveal that the full range of research methods are being used (see table 2 below sorted by most often used research method). the survey method is the most popular, but so are positivist and interpretivist case studies, and design science. again, this data is at a school level, so a study of individual researchers is needed to reveal the true extent of usage of the different methods. never sometimes often always survey 0 7 16 1 interpretivist case study 2 7 15 0 positivist case study 3 10 11 0 design science 0 13 10 1 literature meta analysis 5 13 2 4 business modelling/simulation 7 11 4 2 secondary data analysis 5 13 5 1 is development 6 12 6 0 action research 6 13 5 0 conceptual study 4 16 3 1 ethnography 10 11 3 0 longitudinal case study 5 17 2 0 laboratory experiment 11 11 2 0 table 2: research methods used respondents clearly indicated that the primary beneficiaries of their research were other is academics (20), managers (17) and is professionals (15 responses), followed distantly by end users/workers (7), policy makers (6), and people in general (0). this may again show that we (is researchers) are not taking up the opportunity to influence governments and society, and this may be a major reason for the apparent lack of recognition of is as a discipline by some government agencies. respondents indicated that, where it occurred, most research collaboration occurred with is colleagues within that particular academic group. clearly, there is a need to widen the collaboration net nationally and internationally (which could help to increase quality) and with practitioners (which increases relevance and provides opportunities for funding, e.g., the australian research council linkage grants). statistical analysis of school research output is shown in table 3 (for 2004, the most recent reporting year). the average school generated about 43 publications, 29 of which were conference papers, 10 journal papers, and a small number of other types. the average publication output per academic staff member was about 2.3 papers p.a. and a small number of staff chaired conferences and were journal editors. 276 australasian journal of information systems volume 14 number 1 nov 2006 2004 average 2004 range refereed journal papers 9.8 1-58 refereed conference papers 29.1 4-65 chapters in books 3.4 0-20 authored books 0.4 0-2 edited books/proceedings 0.5 0-4 journal editorships 1.9 0-12 conferences chaired 1.5 0-9 table 3: research output research funding varied considerably with a few groups doing very well in gaining funds from external sources, but most having to depend on their own resources (see table 4 below). this was confirmed by research income over averaging over $300,000 p.a. (about 50% coming from australian research council (arc) linkage and discovery), but with a very wide range. generally, these figures compare poorly with other disciplines, including computer science and computer engineering. 2004 average 2004 range arc linkage 90.2 0-325 arc discovery 64.8 0-412 internal university 47.6 0-240 crc 46.5 0-540 industry contract 46.3 0-377 consulting 9.6 0-108 international 4.8 0-100 other 3.3 0-30 table 4: research grants ($k) the final part of the survey allowed each respondent to suggest the three main strengths, weaknesses, opportunities and threats (swot) for the is discipline research and a summary of the most frequently cited issues is provided in table 5. in total over 150 ideas were generated in the swot and the “top 5” in each category are presented here. strengths weaknesses industry relevance & links (10) lack of industry relevance (8) diversity of method (6) lack of identity of is as a discipline (6) diversity of research undertaken (5) poor funding & recognition by funding bodies (6) feeling of community (acis, acphis) (4) poor/variable research training (5) critical mass of quality is researchers (3) conflicts of research focus (5) 277 australasian journal of information systems volume 14 number 1 nov 2006 opportunities threats industry collaboration/linkage grants (13) falling student numbers/staffing (9) raising profile in industry & government (6) other fields claiming is as their own (8) cooperative doctoral research training (4) lack of research funding (6) collaboration generally (3) nelson higher education policy/research quality framework (6) improved quality & success (2) lack of industry relevance/recognition (4) table 5: results from the swot analysis the respondents clearly believe there is strength in our diversity and relevance. diversity was indicated in types of research undertaken, the research approaches taken (and the underlying epistemology), and in the breadth of experience most is researchers bring with them from their background in is practice and their grounding in practitioner activity. these strengths in diversity and relevance need to be nurtured and exploited. key weaknesses are lack of relevance and identity, poor funding (poor relative to computer science/computer engineering), which is associated with other weaknesses such as a lack of research culture in australian business and lack of recognition from funding agencies such as the arc. these and other research focus and training issues need to be overcome. the respondents clearly recognized there are numerous opportunities of which we should attempt to take advantage. in this collaboration is the key (with industry, international colleagues, and other australian universities). in addition and as indicated earlier, the opportunity exists for is to increase its profile and recognition by conducting research on societal and economic issues which may influence government policy. while industry collaboration was seen as an opportunity it may also be a threat if proper linkages are not built. research impact will be critical in the research quality framework, and perhaps the greatest threats to is research in australian universities lie in lack of recognition of is as a discipline and its location in the academic structure, the falling numbers and excessive teaching loads in most schools and the career and financial opportunities outside academia. conclusions and future work in this paper we have presented preliminary results of a survey of australian is ‘heads of discipline’ which shows something of the current ‘state of the art’ for australian is research. it should be noted that, the data collected and presented do not necessarily represent the views of individual is researchers. future work is needed to obtain those views. however, the paper can and should be used to initiate discussion on where we are, where we want to be in the future, and how we aim to get there. 278 australasian journal of information systems volume 14 number 1 nov 2006 references avgerou, c., siemer, j. & bjorn-andersen, n. (1999) “the academic field of information systems in europe”, european journal of information systems, vol 8, pp 136-153. mingers, j. (2001) “combining is research methods: towards a pluralist methodology”, information systems research, vol 12 no 3, pp 240-259. neuman, w.l. (2006) social research methods: qualitative and quantitative approaches, 6th ed., allyn and bacon, needham heights, massachusetts, usa. pervan, g.p. & cecez-kecmanovic, d. (2001) “the status of information systems research in australia: stage 1 – the hods view”, proceedings of the 12th australasian conference on information systems, coffs harbour, nsw, australia, december 5-7, pp 503-506. ridley, g., goulding, p., lowry, g. & pervan, g.p. (1998) “the australian information systems research community: an analysis of mainstream publication outlets”, australian journal of information systems, vol 5 no 2, pp 69-80. the authors professor graham pervan of curtin university of technology in perth, western australia, has over 30 years experience in education, research, and practice in information systems and information technology (is/it). he has published on various issues related to is/it management and decision support systems in journals such as journal of information technology, decision support systems, information and management, journal of computer information systems, journal of group decision and negotiation, and journal of research and practice in it, and most major conferences. he is the asia-pacific regional editor for the journal of information technology, the it management editor for the australian journal of management, and president of the australian council of professors and heads of information systems. professor graeme shanks is an honorary professor in the faculty of information technology at monash university, australia. his teaching and research interests include information quality, conceptual modelling, identity management, implementation and impact of enterprise systems, and decision support systems. graeme has published the results of his research in over 100 papers in journals such as journal of information technology, information systems journal, journal of strategic information systems, requirements engineering and information and management, and most major conferences. he is a member of several journal editorial boards, a member of ifip wg8.3 decision support systems, a fellow of the australian computer society and was recently a member of the college of experts of the australian research council. 279 australasian journal of information systems volume 14 number 1 nov 2006 280 ajis vol. 9, no. 2 may 2002 models of governance a viable systems perspective john davies school of management, victoria university of wellington, po box 600, wellington, new zealand, tel: +64-4-463 5382, fax: +64-4-463 5253, email: john.davies@vuw.ac.nz abstract this paper suggests a framework for examining governance issues spanning the corporate, public and nonprofit sectors that draws on existing frames-in-use, and that reflects the systemic communality apparent within varied attempts to describe and effect change in governance. the paper surveys alternative conceptualisations of governance that have surfaced in the academic and practitioner literature and draws attention to the cybernetic and systemic features of diverse views of governance. beer's viable systems framework is used to reinterpret alternative views of governance and to suggest a means of diagnosing the completeness/coherence of governance systems. keywords: governance, governance systems, corporate governance, viability, viable systems introduction this paper begins by outlining a seeming diversity of views and conceptions of governance. then, following a brief resume of beer's work, these views will be examined and reinterpreted using beer's systems framework. the paper will conclude with some observations on the usefulness of beer's framework in understanding the systemic roles and functions required of governance in determining the viability of organisations. issues in governance jessop (1998, p 29) claims that the notion of 'governance' has only recently entered what he terms the 'anglophone social science lexicorn'. yet even so, he states that in its short life, its usage has been eclectic, diverse and 'pre-theoretical'. kay et al. (1995, p 84) comment similarly that whilst issues of governance may have existed for as long as there have been social institutions, the now common-or-garden term 'corporate governance' did not emerge until the seventies. common conceptions of governance connote not only government and governing, but also the activities of governing boards and bodies, the terms often being used interchangeably (stoker, 1998, p 17). the opinion of maw et al. (1994, p i ) , based on their practitioner experiences, that '(c)orporate governance is a topic recently conceived, as yet ill-defined, and consequently blurred at the edges', is somewhat in keeping with these and many other views, and with empirical findings that describe corporate boards as 'complex, dynamic human systems charged with an ill-structured set of responsibilities' (demb et al, 1992), findings which have been endorsed by cadbury (1999, p 15), and which have a counterpart in the nonprofit sector (middleton, 1987, p 141). the implicit criticisms of the lack of a common view, or the lack of a commonly accepted view, and perhaps the lack of rigour in defining governance, have led to different research agendas amongst academics and practitioners. since the seventies, interest in corporate governance has been catalysed by considerable media attention given to poor company performance, corporate failure, inappropriate accounting/audit practices, excessive remuneration packages for senior managers and executive directors, insider trading, pension fund mismanagement etc. but, as implied by kay et al. (1995: 84), such events are not just recent phenomena, and much of the increased scrutiny can be attributed to the advent of harsher economic conditions that have drawn to the surface these manifestations of underlying corporate frailty, weaknesses or excesses. growing interest in governance can also be attributed to enhanced awareness by those organisations that operate in an international context, of the different governance practices that exist in an increasingly global corporate sector operating in global markets (witherell, 1999, p 78; lannoo, 1999, p 270; cadbury, 1999, p 13). additionally, the extent to which governance issues pervade society is exemplified by the behaviour of organisations in the voluntary or non-profit sector, and by their perceptions of the role and importance of governance. many leading sports bodies have restructured their governance processes voluntarily in recognition of a need to bring about greater organisational effectiveness (davies, 1997), accepting the notion that performance is predicated on effective governance (schlefer, quoted in byrne, 1998, p 82-85). others have engaged in reforms of the governance and management structures, for example, the football association in england, to effect change to the balance of stakeholder representation and stakeholder interests, and to limit any potential abuse of executive power. at the european level, the erstwhile success of uefa in developing its marketing, commercial, and financial strengths, followed by the collapse of its marketing agency in 2001, has focused attention on its governance processes, particularly the relationship between governors, executive management and their agents. indeed, the changes that are taking place in the world of sport reflect a climate that mirrors the movement for reform of corporate governance described by tricker (1984), cadbury (1992). in an interesting comparison to governance issues arising in sport, hampel (1998, p 9) contrasts the perspective of his report to that of earlier work (cadbury, 1992; greenbury, 1995). he suggests that whilst their approach and 57 ajis vol.9, no. 2 may 2002 guidelines 'concentrated largely on the prevention of abuse1, responding to 'things which were perceived to have gone wrong', his work was equally concerned with the articulation of principles of corporate governance that would make a positive contribution to organisational life. the collective result now appears as the combined uk code on corporate governance (parkinson et al., 1999, p 101; lannoo, 1999, p 283). similar initiatives have taken place worldwide. governance as activities, functions, networks and systems maw et al. (1994, p 3) have conceptualised corporate governance as a complex of relationships involving duties and obligations between the board, its shareholders, financiers, customers, employees, auditors and regulators. based primarily on substantial practitioner experience, they too, have developed a prescriptive framework a suggested code of practice for boards, favouring compliance to guide boards in meeting their 'a priori' duties, and legal obligations. however, somewhat differently, demb et al. (1992) have sought empirically to discover how board roles relate dynamically to the internal and external forces impinging on the organisation. they take a systems view and see boards as part of a wider system of governance, beyond duties and obligations. demb et al. (1992) conceptualise a governance system wherein a board, as a sub-system, has an integral, interdependent role interacting with, and being influenced by three other sub-systems the wider regulatory system, the system of ownership, and the societal system. they argue that '(c)orporate governance is the process by which corporations are made sensitive to the rights and wishes of their stakeholders', a stance disputed by argenti (1997), who acknowledges shareholders, as owners, to be the only legitimate stakeholders. we note that parkinson et al. (1999, p 101) suggest that the three major reports on corporate governance in the uk (cadbury, greenbury and hampel) were predicated on the shared assumption that 'governance is understood to be about the relationship between shareholders (ie the owners) and managers'. such debates that have taken place in the literature often reflect different perspectives of the nature of governance and of what constitutes an appropriate balance of these interests and activities. others have offered similar notions to demb et al. (1992), outlining 'governance frameworks' or 'systems of governance' within which organisations operate. cadbury (1998, p 2) describes a framework structured by interacting forces: by the force of law impacting upon organisations; by the regulatory forces of, for example, the stock exchange; by shareholder meetings and by the force of public opinion. allison (1998, p 29) has similarly commented that 'the system of contemporary world governance in sport' is also one of complex interdependence between international and national governing bodies, international law and the courts, the media, commerce and business, the fans and the public etc. worthy et al. (1983) offer yet a similar view that governance is 'concerned largely, though ... not exclusively with relating the corporation to the institutional environment within which it functions.' issues of governance for them include 'the legitimacy of corporate power, corporate accountability, to whom and for what the corporation is responsible, and by what standards it shall be governed, and by whom'. the implication is that discussing governance and governing boards in terms of structural elements size, committees, roles, meeting frequency, board fees, chairman and ceo relations may be necessary, but not sufficient to build understanding of the governance processes that facilitate effective functioning (demb et al., 1992; cadbury, 1998; charkham, 1994; pettigrew et al, 1995). indeed, as suggested by charkham (1994, p 6), understanding a system of governance 'means studying both its structure and dynamics', that is, not just the legal framework but also personal relationships and patterns of behaviour. in addition, demb et al. (1992) contend that effective board performance is linked to an appropriate understanding of the tensions, or paradoxes, that would otherwise 'destabilize' boards. the desire to promote effective corporate governance and to effect appropriate board member behaviour has been manifest in different prescriptive approaches. whilst maw et al. 's code of practice addresses perceived structural requirements, carver's approach (1999, 1997) is intended to guide a board's debate so that it embraces values in the consideration of necessary governance functions. maw et al. 's code relates to the composition of the board, specifying a minimum number of non-executive directors, and appointment criteria. it addresses the functioning of the board, specifying responsibilities and matters 'reserved' for board consideration; accountability: directors' responsibilities, disclosure and compliance requirements, sanctions; and the use of independent non-executive board committees for audit, remuneration and director nomination purposes. carver goes beyond maw et al. 's embrace of duties and obligations, and seeks to engender debate on values about ends, results, impacts, goals etc, and to whom, as constituents, they relate. he also focuses attention on the values that can be attributed to means that they are employed in a prudent and ethical way; and on values implicit in the board-executive relationship affecting the delegation of authority and power, the ceo's role, performance assessment etc. whilst some of these prescriptions are likely to have relevance for organisations in both the corporate and the non-corporate sector (charkham et al.: 1999, p 242-243), there is minimal guidance for those who seek to determine the appropriateness of any prescription. it is this latter inadequacy of descriptive and prescriptive 58 ajis vol. 9, no. 2 may 2002 frameworks that needs to be addressed. in the final sections, we seek to employ a beer's viable systems framework to develop perspectives and insights about governance that more directly address the nature of effective governance functioning, and the adequacy of prescriptions. beer's viable systems framework beer's approach (1979, 1981, 1985) to issues of organisational design and effectiveness, just like cadbury's approach to governance (1992, 1998; also quoted in demb et al., 1992, p vii-viii) is not pre-occupied with structure. neither is the approach pre-occupied with the organisational typologies often used to reflect structure, or the configurations that are often embodied in organisational charts. instead, beer focuses on the systemic functions that enhance organisational viability, and which provide a basis for adaptive learning about what are effective organisational behaviours and goals in a climate of complexity and change (davies, 1999). beer's research (1979, 1981, 1985) has demonstrated that certain systemic features or functions are necessary to any system's viability or survival. to be viable in beer's terms, that is, to survive and be effective, an organisation must be able to manage uncertainty and complexity by achieving requisite variety of response. it can develop requisite variety either by creating increased variety in its own systemic behaviour and functioning, or by acting as if to reduce the environmental variety to which it would otherwise be exposed. knowledge of these systemic functions can therefore be used to analyse the systemic strengths and weaknesses in existing organisations, and/or to, guide the design of organisations to provide required systemic features. beer's cybernetic framework for analysing organisational systems is known as the viable systems model (vsm). the use of cybernetic science to underpin the design of self-regulating, adaptive technical systems that can maintain required outputs, and work within established norms, is well known and predates beer's attempts to extend its use to organisational systems (francois, 1999; shenhav, 1995). however, beer's conceptualisation of cybernetics as 'the science of effective organisation the science of communication and control, in the animal and the machine', extends the applicability of cybernetics beyond natural and technical systems. indeed, in doing so, he emphasises the importance of communication as part of organisational systems design building communication channels, generating information flows and installing information feedback mechanisms to enhance learning and adaptive response, that is, to develop requisite variety in action. the viable system model beer conceptualises all viable systems as a network of communication channels bonding five complementary sub-systems. the sub-systems, whose effective functioning and communication links are necessary to any system's viability, comprise an operational system, si, of autonomous operational units that act out the very identity and purpose of the overall system, and a meta-system comprising four other sub-systemic functions: s2 effecting overall coordination of the autonomous units; s3 operational planning, monitoring, control and audit functions relating to the autonomous units; s4 intelligence and strategy development serving the whole organisation's future; and s5 the creation and promulgation of identity, vision, direction, purpose and mission, throughout the organisation and its wider environment (brocklesby et al, 1995). all sub-systems are part of the larger system under investigation, which is defined as the system-in-focus (sif). in terms of systems logic, no one sub-system is considered to be more important than another in contributing to the viability of the sif. however, it will be si's activities that directly serve the organisation's purpose; and it will be the meta-system's function to provide the organisational climate, the direction, resources and support for si to best manage in a changing complex environment, and for the si units to become viable sub-systems themselves at a lower level of recursion or embeddedness. the sif may, itself, be part of, or embedded within a larger organisation or system (davies, 1999). in contemplating the use of beer's vsm to examine aspects and models of governance, we take the view of rosenau (1992, p 3) who states that 'to presume the presence of governance ... is to conceive of functions that have to be performed in any viable human system'. interestingly, and without reference to beer, those functions are listed as setting goals, developing strategies and policies, procuring resources necessary for 'preservation and well-being', preventing conflict among its members or factions from tearing it apart etc, and more generally, as a measure of effective functioning, coping with environmental uncertainty and external challenges. 'whether the systems are local or global in scope, those functional needs are ever present if a system is to persist intact through time.' other models of governance this section outlines various models of governance which have been identified as exhibiting cybernetic features and functions, and uses beer's vsm to examine the systemic qualities, completeness and coherence of the different models. first, we look at the models of young (1996), who focuses on information and feedback mechanisms, and jessop (1998) and rosenau (1992) who explicitly refer to systemic functions of governance. 59 ajis vol. 9, no. 2 may 2002 then we use the beer's framework as an interpretive filter to identify the systemic qualities of models of governance proposed by carver (1997), charkham (1997), kay (1995), senge (1999). young (1996, p 15-17), has proposed a feedback model of governance. he has made explicit reference to the cybernetic concepts of positive and negative feedback in describing the function of governance as being 'to keep an organisation on course by detecting where it is going and providing information that will allow necessary corrections to that course.' young asserts that governance is not just about boards, but can be regarded as 'a cluster of mechanisms that measure an organisation's performance and feed it back.' young suggests that his model thus accommodates and complements the notions of middleton (1987, p 143) relating to the "boundaryspanning and control' role of boards in the necessary and purposeful transfer/feedback of external information to managers. he conceives of board roles as reflecting and harnessing the nature of feedback. for example, with negative feedback mechanisms, one measures the difference between desired and actual performance, in order to move performance closer to the target level, a process of homeostasis, that is, self-adjusting stability. young thus interprets the work of a 'negative governor1 as being to ensure that the organisation stays on course and focussed on achieving its mission. then, since positive feedback mechanisms work to reinforce movement away from pre-determined levels, creating either spiralling success or ever-decreasing performance levels, he describes the work of a 'positive governor' as being to provide the condition and support for growth and innovation. young understands the importance of feedback and information flows, but his conceptualisation does not differentiate between external feedback along horizontal communication channels and internal feedback via vertical communication channels, in the way that beer's framework does. this point may be illustrated with an example of feedback mechanisms operating within a system comprising an international association of member groups. if the international association is performing inappropriately or poorly, then as is now the case with fifa, it is possible for the continental confederations, acting as de facto s1 operational units, to vocalise their concerns 'upwards' along vertical communication channels to the fifa meta-system, perhaps demanding changes. this can be described as an example of "bottom-up1 accountability. on the other hand, fifa in seeking to make its member organisations accountable for specific activities and targets, may require formal reporting an example, in young's terms, of top-to-bottom accountability. both sets of information can be conceptualised as flow along vertical communication channels within the organisational system, with the confederations being si viable systems 'nested' within the system that is fifa. however, boundary-scanning activities can be described as an example of horizontal information exchange, with horizontal communication channels providing a conduit between the system and its various environments. young's model implicitly accepts governance responsibilities as including systemic functions such as 'steering' and the setting of strategic direction (s5); the maintenance of an intelligence function, the development of strategy and a capacity for self-assessment (s4); and the resourcing of activities together with the development of accountability and control mechanisms to keep things on course (s3). his explicit precondition is, of course, the development of information networks to build a capacity for learning, adaptability and requisite response. so, young's model of governance, whilst nominally emphasising feedback, can be stated to exhibit many of the meta-systemic qualities required of viable organisations. jessop (1998, p 29) conceptualises governance as referring 'to any mode of coordination of interdependent activities', such as that created through the development of a planned formal organisational hierarchy, a selforganising 'heterarchy', or even the anarchy of exchange that constitutes free markets. jessop sees heterarchy as being as relevant at the level of self-organising inter-personal networks as it may be in the self-organising of agencies, institutions or systems 'which are operationally autonomous from one another yet structurally coupled due to their mutual interdependence.' jessop (1998, p 30) contrasts the self-organisation of inter-organisational relations with the notion of "de-centred, context-mediated inter-systemic steering' as described by glagow and willke (1987). in the latter scenario, jessop suggests that whilst the often complex operational logic of a differentiated sub-system may prevent effective control of the wider system's development from the outside, it does not prevent some 'context specific' steering. he offers a view that inter-systemic "noise reduction' can be achieved through appropriate direct communication involving negotiation, cooperation in shared projects, and what he terms 'negative coordination" activities to prevent mutually undesirable outcomes. the latter notion equates with beer's anti-oscillatory behaviour. whereas young seemingly emphasises feedback and control, jessop emphasises coordination. however, jessop's notion of 'coordination' can be interpreted as contributing to various systemic functions in diverse ways at different levels of recursion relating to the system-in-focus (sif). in particular, his notion of inter-systemic 'noise reduction' can be regarded as equivalent to s5 building of shared values and identity for the organisational system that comes together with a common purpose of conducting mutually beneficial business. on the other hand, employing a code of practice to prevent undesirable outcomes emanating from si units acting with designed levels of autonomy, would constitute the coordination as being an s2 function for the sif. 60 ajis vol. 9, no. 2 may 2002 although jessop does not deny the importance of information, it does not hold as central a position in his model as it does for young. nevertheless, jessop's use of the term 'context-mediated steering' does imply building an s5 strategic flexibility or responsiveness to s4 knowledge of environmental change. indeed, in acknowledging the inevitable incompleteness of attempts at coordination, jessop (1998, p 43) implicitly surfaces the cybernetic necessities required of any viable system, which are to engage in double-loop learning (beer, 1959) and to build through this self-reflexive capacity for learning, an adaptability and requisite variety to respond to internal and external change. in enunciating his 'principles' of good corporate governance, charkham has stated that management must have freedom and autonomy to drive the enterprise forward, of course 'exercised within a framework of effective accountability1 (charkham, 1994, p 4; governor, 1992, p 211). indeed, he has stated that 'the main point of a good (governance) system is to give power to those best able to use it' (charkham, 1997, p 3). his view is that a system of corporate governance should be 'concerned with checks and balances on the exercise of power, and with its peaceful transfer'. indeed, charkham has been forthright in asserting that the success of a corporate governance system is dependent upon 'its ability to reconcile entrepreneurial freedom with effective accountability', or as beer frames it, to effect an appropriate balance of s1 autonomy and meta-system control. although working predominantly in the corporate sector, charkham (1993, p 391), claims that these principles of governance have relevance in many other spheres of activity economic, political, sporting etc. senge (1999, p 10-11) offers a different notion of what constitutes power or control, commenting that 'the source of legitimate power' in an organization is its guiding ideas, values and ideals, its mission, purpose and reason for being. he asserts the need for an organisation or any human community to express its aspirations and to articulate its identity to provide a basis for evaluating whether strategic and operational decisions reflect vision and mission oriented behaviour. in doing so, he is expressing the need for s5 functionality. he implies consequent impact on authority/control mechanisms and on operational activities, suggesting for example, the need for less formal approaches to say, coordination and control, as identity and values pervade the organisational system and its nested sub-systems, guiding behaviour and choice. such views fit within beer's framework, and provide a complementary view to the notions of authority and power expressed by charkham. in contrast with the prior frameworks of governance that give explicit emphasis to a subset of systemic features, rosenau's model (1992: 3) can be considered more comprehensive. he conceives of governance as equating with the broad provision of necessary meta-systemic functioning relating to setting objectives (s5), developing policy options (s4) and policy guidelines (s3), resource acquisition (s3), anti-oscillatory coordination (s2) etc. other writers offer models or views of governance that bear surface resemblance to, but are not founded on or legitimised by viable systems thinking. for example, without being specific about meta-system functions, chelladurai (1987) has suggested taking a 'parsonian' perspective on organisational design, and on board functioning. in relation to the design of amateur sports organisations, he has advocated the separation of the board as an 'institutional' sub-system from the managerial and technical sub-systems. however, any comparison between the contribution of the board to beer's meta-system and the role of the board in chelladurai's 'institutional' sub-systems is limited by the ambiguity and lack of clarity about the conceptual and functional make-up of chelladurai's sub-systems. we may conclude that any resemblance of the 'institutional' sub-system to beer's meta-system is perhaps coincidental rather than based on cybernetic considerations. however, carver's values-led-policy-driven governance framework (1997) can be considered to be as comprehensive as rosenau's, and to have notable systemic coherence. carver's prescriptions relate predominantly to s5/s4 meta-systemic functions, and cohere with beer's notions of the systemic functions that engender viability and the long-term sustainability of organisations. in particular, carver promotes governing board s5 behaviour that builds identity by facilitating diversity; that is relationship oriented, identifying and describing constituencies and accountabilities; that is values driven, explicitly addressing fundamental values; that is visionary; that is proactive, leading and creating rather than approving, reviewing and monitoring etc. similarly, he would attempt to strengthen s4 functionality by endorsing forward thinking, taking the long term view; by developing an external focus, building responsiveness; by valuing intelligence; by being self-aware and how it engages in self-monitoring; by how it represents the 'ownership', and how it seeks an understanding of its systemic role on any matter. carver's awareness of the need to seek the right balance between autonomy and control reflects the systemic balancing act between s3 and s4, and then between s3/s4 and si; and his call for decisions to be outcome driven, weighing decisions against purpose, would require and lead to enhanced s2/s3 functionality (carver, 1997, p 32-33; 1999, p 1-10). in many ways, kay et al.'s trusteeship model of corporate governance (1995, p 90-91) shares similarities with carver in its value-laden expression of governance responsibilities. their conception of governance requirements is in line with the views of young (1996, p 15-17), jessop (1998, p 29), and demb et al. (1992), in seeking 'to balance the conflicting interests of current stakeholders ... to weigh the interests of present and future stakeholders'. in kay et al.'s view, trusteeship responsibilities require effective governance 'to sustain the corporation's assets', including such assets as its identity, reputation, the skills of its employees, and the favourable expectations of customers and suppliers etc. effective governance, in kay et al.'s view, can thus be 61 ajis vol. 9, no. 2 may 2002 interpreted as being primarily dependent upon s5 functioning involving the building and sustenance of appropriate identity and values related to reputation, the values placed upon relationships with employees, suppliers, customers etc. as such, they concur with demb et al. (1992) in a concern 'to balance the intimacy of operational involvement with the detachment presumed to be required for objective strategic review and analysis etc1. discussion and summary charkham (1994: 248) has remarked how difficult it is to be objective with the familiar, and that to shed light on governance may require the examination of alternative governance systems, and the development of alternative perspectives. this paper has outlined a variety of views of governance practices, processes and systems, and has demonstrated the value of a systemic perspective and of beer's framework in attempting an evaluation of alternative models of governance. as espejo et al. (1989, p 445) propose, if we accept the notion of a 'model-as-convention', rather than as a description of reality, then, at the very least, we can accept beer's model as providing a context which provides 'a way of talking about something in a manner that is understandable and useful, that is, as a communication vehicle'. indeed, beer's organising framework, the vsm, 'provides a language in which to identify key features of a complex reality1, and can help to pose significant questions about that reality (stoker, 1998, p 1). as such, beer's model can act in the form of a sensitising framework as a way of seeing and thinking which alert the analyst to alternative ways of understanding (thomas, 1993, p 210). just as carver (1997, p xvi) claims that his values-based policy model enables board members to see their role in a new light, so, too can beer's framework provide an alternative perspective on the design of governance relations. beer's conceptualisation of viability in organisations is based on cybernetic and systems concepts and a consideration of the systemic functions that contribute to systems viability. beer would contend that organs, instruments, activities and processes of governance need to be effective in the sense that they establish or contribute to the maintenance of systemic identity and purpose(s) which have coherence, and which are projected, shared and accepted within and without the organisation by its internal and external constituents. that identity is obviously linked to the organisation's purpose, its raison d'etre, and together, they can provide a guiding beacon and logic that cultivate the values and ethics that underpin ends-oriented and mission-oriented behaviour. effective organs of governance must balance the competing attentions of multiple constituents, and of long and short-term objectives. in keeping with fundamental purpose and values, the processes of governance must decide on strategic direction and goals for the organisation, and be capable of assessing the performance of senior management in operationalising those goals. those involved in governance processes, for example, a governing board, must also be capable of critical self-reflection, self-monitoring and self-assessment, that is, they must be capable of embracing in concept, a 'model' of the organisation its purpose, identity, structure, functioning etc. these are primarily s5 and s4 functions within beer's framework. beer's approach, however, requires recognition of the need for quick response to environmental change, the need for adaptability or to use demb et al.'s terms (1992, p 195), 'the imperative of adaptability'. it implies autonomy to act, within charkham's framework of controls and accountabilities, and within the 'tight-loose' framework of peters and waterman (1980), but which in beer's terms and in carver's terms means working within a framework of s2 policy guidelines and 'controls', whose development is a necessary s3 systemic function, reflecting s5 identity and values. in reflecting on beer's model of viable organisations, we note the role and functions of governance as being a sub-set of meta-system roles and functions: creating identity, building shared values and purpose, setting direction, steering; providing resources and delegating authority for managerial and operational staff to act with autonomy and appropriate responsiveness in changing environments. however, many of those involved in governance roles fulfil other systemic functions within organisations. for example, in many voluntary organisations, at local, regional or national governing body level, those elected or appointed to take on s5 stewardship of the organisation, to guide its future direction and activities, are often the volunteers who undertake si operational activities. they may also engage in s3 operational planning and budgeting activities, and in s4 strategy development activities. the potential for systemic role conflict to exist is apparent. unless those who are charged with s4 governance functions are able to conceptualise the organisation's being, and to view its activities in a holistic sense, that is, in systemic terms, then it is possible that in failing to recognise the various systemic functions and roles that exist, they will, as volunteers, engage in multiple systemic activities, creating role overload, role overlap and role conflict. indeed, one common systemic ailment arises when, with the best of intent, the s5 governors necessarily engage in si activities, but in doing so denude si staff of their autonomy, undermining initiative etc. 62 ajis vol. 9, no. 2 may 2002 additionally, when board members, governors or trustees are elected from (or possibly to serve) particular constituencies, difficulties may arise in building common identity, purpose and values for the organisation. the systemic dysfunction most likely to be caused by parochialism amongst say, governors can then be described as that of s5 dysfunction lack of identity, lack of common purpose, lack of shared values etc. we conclude that understanding governance requires a holistic approach. conceptualising governance and board activities in terms of beer's systemic roles and functions not only delivers insights about the design of governance, but also contributes understanding to common issues of role overload and role conflict that arise among governing board members, and which can impact adversely on board/staff relationships. moreover, the vsm provides clarity in disentangling the various views about what constitutes governance. indeed, using the vsm as an interpretive framework suggests that there is evidence that concepts and notions that are cybernetic in nature, and which underpin viability, are incorporated within extant models of governance, but mostly in piecemeal fashion. such models can be regarded as incomplete when set alongside beer's model of viability and systemic functioning. however, pursuit of the beerian ideal may not be straightforward. beer's emphasis on systemic functioning supports the views of boyd (1995, p 301) and demb et al. (1992, p 182) that mimetic behaviour, copying the structure of another board, or following blanket prescription, will unlikely work any better than for, say, a production line subject to different demand patterns. as cadbury (1999, p 18) has stated, whereas market forces can be expected 'to bring about a convergence of governance standards and processes', he would 'neither expect nor see the need for a convergence of governance structures'. effective governance is dependent on an understanding of required systemic functioning and other systemic requirements, rather than structural or procedural mimicry. references allison, l. (1998) "the government and governance of sport", politics review, september, pp 28-30. argenti, j. (1997) "stakeholders: the case against", long range planning, 30(3), pp. 442-445. ashburner, i. (1997) "corporate governance in the public sector: the case of the nhs", in keasey, k., thompson, s. & wright, m. (eds.) corporate governance: economic and financial issues, oxford: oup, ch 13, pp. 279-297. beer, s. (1985) diagnosing the system for organisation, chichester: wiley. beer, s. (1981) brain of the firm (2ndedn), chichester: wiley. beer, s. (1979) the heart of the enterprise, chichester: wiley. beer, s. (1959) cybernetics and management, london: english universities press. blair, m.m. (1995) ownership and control: rethinking corporate governance for the twenty-first century, washington dc: brookings institute. boyd, b. k. (1990) "corporate linkages and organizational environment: a test of the resource dependence model", strategic management journal, 11(6), pp 419-430. boyd, b. k. (1995) "ceo duality and firm performance: a contingency model" strategic management journal, 16, pp 301-312. brocklesby, j., cummings, s. & davies, j. (1995) "demystifying the viable system model as a tool for organisational analysis", asia pacific journal of operational research, 25(1), pp 65-86. byrne, ja. (1996a) "the national association of corporate directors' new guidelines won't tolerate inattentive, passive, uninformed board members", business week, no. 3503, november 25, pp. 100-102 byrne, ja. (1996b) "commentary: putting your money where your mouth is", business week, no. 3503, november 25, p 106. cadbury, sir a. (1999) "what are the trends in corporate governance? how will they affect your company?", long range planning, 32(1), pp 12-19. cadbury, sir a. (1998) "the future for corporate governance: the rules of the game", journal of general management, 24(1), autumn, pp 1-14. cadbury, sir a. (1992) report of the committee on the financial aspects of corporate governance (1993). london: gee publishing aka the cadbury report campbell, a. (1997) "stakeholders: the case in favour", long range planning, 30(3), pp 446-449. carver, j. (1997) "boards that make a difference: a new design for leadership in nonprofit and voluntary organizations", 2nd ed., san francisco: jossey-bass. carver, j. (1999) the unique double servant-leadership role of the board chairperson. the reprint file: http:///www.carvergovernance.com/tr.htm. charkham, j. (1993) "the bank and corporate governance: past, present and future", bank of england quarterly bulletin, august, pp 388-392. charkham, j. (1994) keeping good company: a study of corporate governance in five countries, oxford: oup. 63 aj1s vol. 9, no. 2 may 2002 charkham, j. & simpson, a. (1999) fair shares: the future of shareholder power and responsiluvi'q', oxford: oup chelladurai, c. (1987) "the design of sport governing bodies: a parsonian perspective", in slack, t. & minings, c. r. (eds) the organization and administration of sport, london ontario: sports dynamics. collins (1994) collins english dictionary. 3rd edition, glasgow: harper collins publisher. coraforth, c. & edwards, c. (1998) good governance: developing effective board-management relations in public and voluntary organisations. london: cima. davies, j. (1997) "designing sustainable sporting organisations:, in stowell, f.a., ison, r.l., armson, r., holloway, j., jackson, s., & mcrobb, s. (eds) systems for sustainability people, organisations and environments, london: plenum, pp 191-197. davies, j. (1999) "the effective organisation of national sports bodies in nz a systems perspective", in trenberth, l. & collins, c. (eds) sports management in new zealand, palmerston north: dunmore press, pp 109-131. demb, a & neubauer, f-f. (1992) the corporate board: confronting the paradoxes. oxford: oup. espejo, r. & harnden r. (1989) "the vsm: an ongoing convention", in espejo r. & harnden, r. (eds), the vsm: interpretations and applications of stafford beer's vsm, chichester: wiley, pp. 445. francois, co. (1999) "systems and cybernetics in a historical perspective", systems research and behavioural science, 16(3), pp 203-220. glagow, m. & willke, h. (1987) dezentrale gesellschaftssteuerung: probleme der integration polyzentristischer geseeschaft. pfaffenweiler: centaurus-verlagsgesellschaft, quoted in jessop, b. (1998) "the rise of governance and the risks of failure: the case of economic development", international social science journal, 50(155), pp 29-45. goold, m. (1996) "the (limited) role of the board", long range planning. 29(4), pp 572-575. governor. (1992) "man bites watchdog", bank of england quarterly bulletin, may, pp 388-392. greenbury, sir r. (1995) "directors' remuneration: report of the study group", london: gee publishing. hampel, r. (1998). the committee on corporate governance: final report. london: gee publishingaka the hampel report. jessop, b. (1998) "the rise of governance and the risks of failure: the case of economic development", international social science journal, 50(155), pp 29-45. kay, j. & silbertson, a. (1995) "corporate governance", national institute economic review august 95, pp 84-97. lannoo, k. (1999) "a european perspective on corporate governance", journal of common market studies, 37(2), pp 269-294. maw, n.g., lord lane of horsell, craig-cooper, sir m. (1994) maw on corporate governance, aldershot: dartmouth publishers. middleton, m. (1987) "nonprofit boards of directors: beyond the governance function", in powell, w.w. (eds) the nonprofit sector: a research handbook, new haven & london: yale university press, pp 141153. parkinson, j. & kelly, g. (1999) "the combined code on corporate governance", political quarterly, january 1999, 70(1), pp 101-107. peters, t.j. & waterman, r.h. (1982) in search of excellence, new york: harper and row. pettigrew, a. & mcnulty, t. (1995) "power and influence in and around the boardroom", human relations. 48(8), pp 845-873. pound, j. (1995) "the promise of the governed corporation", harvard business review, march-april 1995, pp 89-98. rosenau, j.n. (1992) "governance, order and change in world politics", in rosenau, j.n. & czempiel, e-o. (eds), governance without government: order and change in world politics. cambridge: cup, pp 1-30. schlefer, r.m., quoted in byrne, j. (1996c) "the best and worst boards: our new report card on corporate governance", business week, no. 3503, november 25, pp 82-85. shenhav, y. (1995) "from chaos to systems: the engineering foundations of organization theory 1879-1932", administrative science quarterly, 40, pp 557-585. senge, pm (1999) "the discipline of innovation", executive excellence, june 1999, pp 10-11. stoker, g. (1998) "governance as theory: five propositions", international social science journal, 50(155), pp 17-28. thoma, j.e. & chalip, l. (1996) sport governance in the global community. morgantown, wv: fitness inform tech inc. thomas, a.b. (1993) controversies in management, london: routledge. tricker, r.i. (1984) corporate governance. aldershot: gower. witherell (1999) "the oecd and corporate governance", accountancy. june 1999, pp 78-79. 64 ajis vol. 9, no. 2 may 2002 worthy, j.c. & neuschel, r.p. (1983) emerging issues in corporate governance, chicago: northwestern up. young, d.r., hollister, r.m., virginia a hodgkinson & asscociates (eds) (1993) governing, leading, and managing nonprofit organisations: new insights from research and practice, san francisco: josseybass. young, d.r. (1996) "what does feedback have to do with nonprofit governance?", nonprofit world, nov/dec 1996, pp. 15-17. 65 ajis vol. 9, no. 2 may 2002 figure 1 views of governance collins ed, 1994: 669 parkinson etal., 1999: 101 lannoo 1999:272 blair, 1999:3 blair, 1995:273 cadbury, 1992: 15, para 2.5; charkham, 1994: 1 dembetal. , 1992 worthy etal. ,1983 pound, 1995:90 blair, 1995: 273 rosenau, 1992:3 young, 1996: 15-17 jessop, 1998:29 carver, 1997:24-25 ashburner, 1997: 280 kay etal., 1995:84 charkham, 1994: 3 charkham, 1994:4; governor, 1992: 211 cornforth, 1998:2 senge, 1999: 10-11 allison, 1998: 29 thomae/a/., 1996: xiv views of governance 'governance is the action, manner or system of governing; governing is directing, controlling, ruling, regulating, influencing' from the latin, 'gubemare: to steer1 'governance is understood to be primarily about the relationship between shareholders (viewed unproblematically as the owners of a company) and managers'. '(c)orporate governance ... is defined as the organization of the relationship between the owners and the managers in the control of a corporation.' 'corporate governance ... refers to the whole set of legal, cultural, and institutional arrangements that determine what public corporations can do, who controls them, how that control is exercised, and how the risks and returns from those activities they undertake are allocated' 'governance systems, broadly defined, set the ground rules that determine who has the control rights under what circumstances, who receives what share of the wealth created, and who bears what associated risks' 'corporate governance is the system by which companies are directed and controlled' '(c)orporate governance is the process by which corporations are made sensitive to the rights and wishes of their stakeholders' 'governance is ... concerned largely, though ... not exclusively with relating the corporation to the institutional environment within which it functions.' 'corporate governance is not, at its core about power, it is about finding ways to ensure that decisions are made effectively.' 'governance systems, broadly defined, set the ground rules that determine who has what control rights under what circumstances, who receives what share of the wealth created, and who bears what associated risks'. 'to presume the presence of governance ... is to conceive of functions that have to be performed in any viable human system ... to cope with external challenges, to prevent conflict . . ., to procure resources . to frame goals and policies designed to achieve them' 'the function of governance is 'to keep an organisation on course by detecting where it is going and providing information that will allow necessary corrections to that course.' 'governance is about ... a cluster of mechanisms that measure an organisation's performance and feed it back.' 'governance can refer to any mode of coordination of interdependent activities' 'excellence in governance begins when boards recognise . . . setting goals, . . . developing plans . . . depend on values and perspectives . . . (these values) . . .can be unstated as well explicit' 'good corporate governance is ... reliant . . . upon the (existence) introduction and maintenance of mechanisms to promote behaviour and performance on the part of the manager, which is in the interests of the organization's stakeholders.' "key goals of corporate governance: freedom for managers to manage, combined with real accountability for their performance' 'the main point of a good system . . . corporate governance ... is to give power to those best able to use it', corporate governance is 'concerned with checks and balances on the exercise of power, and with its peaceful transfer1 'the success of a corporate governance system is ... dependent upon its ability to reconcile entrepreneurial freedom with effective accountability' 'management must have freedom and autonomy to drive the enterprise forward, of course exercised within a framework of effective accountability' 'governance embraces) all the {unctions performed in organisations by the members of their governing bodies' 'the cornerstone of a truly democratic system of governance is not voting' 'the system of contemporary world governance in sport' is also one of complex interdependence between international and national governing bodies, international law and the courts, the media, commerce and business, the fans and the public etc.' 'sport governance ... is particularly concerned with the ways in which sport is organised and managed 66 microsoft word ajis12-1final repaired.doc ajis vol. 12, no. 1 september 2004 39 on understanding evaluation of tool support for is development björn lundell school of humanities and informatics university of skövde p.o. box 408 se-541 28 skövde sweden bjorn@ida.his.se brian lings school of engineering, computer science and mathematics university of exeter exeter ex4 4pt england b.j.lings@dcs.ex.ac.uk abstract evaluation of is tools has received considerable attention in the literature, yet no consensus is apparent on characterising the important dimensions of an evaluation process. from an analysis of the literature, we identify a weakness in extant methods to support evaluation activities, and posit a framework for is tool evaluation. we conclude that there is a need for enriched method support to deal with the complex socio-technical issues involved in the summative assessment of is tools. keywords: tool evaluation, evaluation dimensions, socio-technical evaluation, is development tool introduction several overlapping interpretations of the concept of evaluation have been presented in the literature (serafeimidis and smithson, 1996, p. 205). with acknowledgement to remenyi et al. (1997), forss et al. (1994) and ramage (1997), in this paper we consider evaluation as involved learning through a series of activities related to understanding, feedback and assessment. symons and walsham (1988) consider evaluation not in terms of tools and techniques, but as a process to be understood, emphasising that such understanding “should not be confused with an understanding of the situation under study” (p. 123). from this interpretive perspective one can argue that such understanding can only come with an awareness of: why evaluation is undertaken; the nature of the process in which it is undertaken; and the impact of and impact on the context in which evaluation is undertaken. the aim of this paper is to posit a framework which encompasses the key elements of evaluation, developed from a review of the evaluation literature, and to use this framework to draw attention to perceived weaknesses in current method support for evaluation – and in particular setting up an evaluation activity. whilst initially based on the serafeimidis and smithson (1996) framework for considering the elements of evaluation, the analysis behind our framework draws broadly from reported research in a number of fields in order to explore the variety of issues that might affect an evaluation activity. three key dimensions are identified in the framework: the stakeholder dimension, the contextual dimension and the activity dimension. we first present the new framework, and then present the analysis behind it focussing on each dimension in turn. we do this not only to justify the elements within the framework, but to act as a rich description of the elements identified. finally, we consider implications for practical evaluations and for research into method support for such evaluation activities. ajis vol. 12, no. 1 september 2004 40 dimensions of evaluation serafeimidis and smithson (1996) introduce what they consider to be the primary elements of information systems evaluation (see figure 1), presenting a “multiple perspective approach” (p. 206). they identify context, process and content as central. measurement process content outcomes history external c o n t e x tin t e r n a l co nt ex t figure 1: elements of information systems evaluation (serafeimidis and smithson, 1996) whilst agreeing with their general formulation, our analysis and experience suggests the utility of making further elements of evaluation explicit (see figure 2). in particular, the centrality of the stakeholder dimension is highlighted within the context of an evaluation. figure 2: elements of is development tool evaluation there is a resulting dynamic between an evaluation activity and the context in which it takes place. in particular, stakeholder motivations and goals are explicitly represented as strongly affecting an evaluation activity. further, outcomes are not represented as a distinct element, but affect as well as effect is included. this cycle accounts for the dynamic of an evaluation activity in context. the final ajis vol. 12, no. 1 september 2004 41 addition is the element of an evaluation framework, which is seen by many as fundamental to any evaluation activity. for example, according to kitchenham (1996), writing in a series of papers on evaluating software engineering methods and tools, this is the position taken in desmet, a project concerned with evaluation “within a particular organisation” (p. 11). in the fifth paper in the series, kitchenham and jones (1997) claim that “a common framework is necessary to make any sort of comparative evaluation of methods and tools.” (p. 14) (for further examples see bailer et al., 1993; ieee, 1992; iso, 1999; powell et al., 1997; smyth, 1998; lundell and lings, 2003). analysis while a majority of sources for the analysis could be classed as “general” sources from the field of information systems (interpreted broadly), sources from other fields have also been considered. these include: is management, is assessment, evaluation research, educational evaluation and software engineering (among others). such a broad basis for the study seems appropriate, since the issue of evaluation is important in all these fields and each brings its own insights. the development of the new framework was our response to understanding a seeming lack of consensus in the literature. focus on the stakeholder dimension different stakeholders for an evaluation one view of the term stakeholder (for example adopted by rossi and freeman (1993)) is that it denotes a human actor who is affected by or can affect a system (or an activity), and it is this view that will be adopted here. farbey et al. (1999) claim, based on findings from field studies, that “[t]here are few attempts to analyse the stakeholder role” (p. 202). however, there is a growing interest in the stakeholder dimension. for example, jones and hughes (2001) stress the importance of stakeholders in arguing that the “views, beliefs and assumptions of stakeholders must be exposed and considered with the is evaluation process” (p. 200), and mcauley et al. (2002) “extends farbey et al.’s (1999) concern that stakeholder analysis should be included as an important evaluation theme” (p. 242) based on a study concerning an outsourcing decision. from evaluation studies in the field of educational programmes, brandon (1998) concludes that “stakeholder participation can help enhance the likelihood of producing valid evaluation findings.” (p. 333) there are many reports in the literature concerning why it is important to seriously consider the impact of stakeholders on evaluations. etzerodt and madsen (1988) highlight the need to view problems and issues from different perspectives, in order to achieve an in-depth understanding, a view supported by ramage (1997). stakeholders’ individual experiences, perceptions, values and interests can all have an impact on an evaluation process. this highlights the importance of carefully considering which stakeholders to directly include in the activity. when selecting stakeholders it is important to acknowledge the importance of a broad and representative input to the evaluation process. for example, fetterman (1997) observes that the “consumer’s voice” is often excluded from the activity. consequently, if important ‘input’ to the process is left out, there is a potential risk that an evaluation activity might consider aspects that are not considered relevant by the community for which it is undertaken. it is also important to consider the closeness of a role to the “usage context” for any tool being evaluated (lundell and lings, 2003). this is not only an issue of the richness of the inputs to an evaluation activity, but affects the politics of evaluation. it has been claimed that “[e]valuation is, to a large extent, a political activity depending on the particular interests of the stakeholders, their personal values, and judgements.” (avgerou, 1995, p. 428) for example, with respect to tool support for is development, hughes and wood-harper (2000) note in reporting from a case study that a “common theme in respect of tool and methodology evaluation is politics, many suggested ‘power dominates’ evaluation in practice” (p. 401). walsham (1993), influenced by the work of guba and lincoln (1989), similarly elaborates on the need to ajis vol. 12, no. 1 september 2004 42 consider multiple interests among different stakeholders in an is evaluation. transparency of process helps to maintain motivation and encourage a sense of ownership. an important part of transparency is direct feedback, and rehbinder et al. (2001) claim that an evolving evaluation framework can act as an effective feedback mechanism. brandon (1998) found, from studies in the field of educational programmes, that stakeholder participation improves acceptance among stakeholders that are not directly involved in the evaluation process because they know that their colleagues have been involved. there is an important distinguished role of the evaluator identified in the literature, which has been characterised as a “key role” in is evaluation (serafeimidis and smithson, 2003, p. 270). the evaluator may come from the defined organisational context or from outside. according to avgerou (1995): “the evaluator is an organizer and facilitator of the evaluation process, rather than an objective assessor, capable to calculate or to prove the merits and the pitfalls of an information system.” (p. 436) the ability of an evaluator to acknowledge different viewpoints and perspectives becomes more important when multiple stakeholders are involved (symons, 1990, p. 195). according to symons (1990), an “evaluation is never value-free” (p. 195), and consequently certain aspects can be seen “very differently” (p. 195) among different stakeholders. therefore, it is important that an evaluator has the ability to understand such alternative viewpoints, since a failure to understand can “lead to unanticipated resistance and hence operational problems” (symons, 1990, p. 195). however the role of evaluator is defined, scriven (1996) acknowledges that it will be very demanding. different motivations and goals for addressing an evaluation the dynamics of a usage context are strongly influenced by the motivations of stakeholders for addressing a particular evaluation. therefore, the importance of clarifying its purpose is considered, by walsham (1993) and others, to be a “key element of the evaluation” (p. 173) irrespective of whether it is undertaken at an early proposal stage or at a post-implementation stage. budgen and thomson (2003, p. 58) present a variety of different motivations, or “agendas”, for conducting tool evaluation, their specific concern being case-tools. in the field of evaluation research, chelimsky (1997, p. 10) distinguishes three different purposes for an evaluation, namely: accountability (e.g. to measure results or efficiency), development (e.g. for helping managers to improve “institutional performance”), and knowledge (e.g. to achieve an indepth understanding with respect to some specific task or area). while acknowledging that these three purposes are not mutually exclusive and exhaustive, it is argued that this distinction can be helpful when analysing suitable approaches for an evaluation activity with respect to (potentially) different expectations of outcome. in terms of accountability, as organisations spend large proportions of their budgets on information systems and technology they consider it important to assess and measure organisational impact. this is true even where organisational benefits have become more difficult to estimate in financial terms so “measurement involves subjective estimates” (avgerou, 1995, p. 428) provided by human assessments. with reference to development, rossi and freeman (1993), from the field of evaluation research, claim that: “evaluations are undertaken to influence the actions and activities of individuals and groups who have, or are presumed to have, an opportunity to tailor their actions on the basis of the results of the evaluation effort.” (p. 46) from this perspective, it can be argued that evaluations are important in the change process, during which an organisation considers is development tools and information systems adoption (symons, 1991, p. 211). ajis vol. 12, no. 1 september 2004 43 with reference to knowledge, almstrum et al. (1996), in an educational context, adopt a learning perspective on the purpose for an evaluation, asserting that “[e]valuation is an educational process, not an end in itself” (p. 201). related to this, patton (1996), in the field of evaluation research, uses the term developmental evaluation when characterising long-term, ongoing evaluation processes that are aimed at facilitating an in-depth understanding of need. in such “long-term improvementoriented evaluation” (p. 135), the “development is an end in itself” (p. 135), and therefore the process itself is the goal for the evaluation activity. the primary concern for such formative evaluation is, according to farbey et al. (1999), to “illuminate and learn”, contrasting with summative evaluation, which is for “accounting and control.” (p. 200) whatever the underlying motives, it is important to be explicit about them and reasons for an evaluation being undertaken (walsham, 1993, p. 182). in the context of software evaluation, brown and wallnau (1996) argue that it is important to have “well-defined goals before starting the evaluation” (p. 40). as the concept of evaluation itself is so broad, it is not surprising that individual stakeholders can have very different goals, some of which may well be implicit. brown and wallnau’s (1996) statement refers to agreed, explicit, and shared goals only, and these will collectively be seen as organisational goals. when considering sources that have an emphasis on goals related to learning and understanding in the light of is development tool evaluation, we observe that they typically concern evaluation in a usage situation. in other words, the phenomenon to be evaluated is dynamic since human actors are involved. in the field of information systems, tools are often evaluated with the goal of facilitating learning among stakeholders within organisations. for example, serafeimidis and psoinos (1995) consider evaluation to be an important feedback mechanism for organisations that facilitates learning, and characterise this activity as a complex process that is far from rational. instead, they view it as an inherently social activity, influenced by the human bias of the evaluator. ramage (1997) identifies two benefits from taking this view. firstly, it allows involved stakeholders to derive different benefits from the evaluation outcome. secondly, it acknowledges that learning is stimulated simply due to the presence of an evaluator. a fundamentally different perspective obtains when evaluations emphasise the assessment of functionality against stated requirements for a specific is development tool. in other words, the phenomenon to be evaluated is to be considered as a static, technical artefact. this perspective is adopted in the iso standard for software product evaluation (iso 14598). according to an overview (iso, 1999b), the goal is “to decide whether relevant quality characteristics meet the requirements of the system.” (p. iv) a principal objective of an evaluation process, according to part 5 of this iso standard (iso, 1998, p. 4), is to promote four process characteristics: repeatability (“repeated evaluation of the same product to the same evaluation specification by the same evaluator should produce results that can be accepted as being identical”), reproducibility (“evaluation of the same product to the same evaluation specification by a different evaluator should produce results that can be accepted as being identical”), impartiality (“the evaluation should not be biased toward any particular result”), and objectivity (“the evaluation results should be factual, i.e. not coloured by the feelings or the opinions of the evaluator”). we interpret this standard as an example of an approach to evaluation that corresponds to what jackson and sulaksono (1998), in the context of information systems evaluation, refer to as evaluations being undertaken in “the efficiency zone” (p. 42). they claim that the overall aim for such evaluation activities is “the achievement of more precise measurement” (p. 42) of the evaluated systems’ functions. a similar example of this view, commonly found in the field of software engineering, in which the assessment and judgement aspects of an evaluation are more central, is exemplified by carney and wallnau (1998) with respect to commercial off-the-shelf (cots) software product evaluation. they argue that, by definition: “to evaluate is to assign value to something, or to judge something. we can improve this definition by considering why we evaluate: essentially, we evaluate in order to decide, that is, to choose between different possibilities.” (p. 852) ajis vol. 12, no. 1 september 2004 44 in terms of the formative/summative debate (patton, 1996), we would categorise this view of evaluation as summative, since its role typically is to constitute a basis for a “go/no-go” (rossi and freeman, 1993, p. 47) decision. for the evaluation of is development tools in use, an otherwise static phenomenon (a tool) is to be evaluated in a dynamic (usage) context. the dynamic nature of this context (is development organisations) implies that assessments may be formative as well as summative, and that the iso objectives must be treated with caution. adoption of a product is typically expensive and will precipitate a change of context, so any evaluation must take a long-term perspective. focus on the contextual dimension the context of an evaluation activity refers to those aspects that might influence and affect the activity, and thereby the outcome of the activity. given that one is considering is development tools in use, the context of an evaluation activity is at least partly defined by its stakeholders. being human actors, this context will therefore change as the activity progresses. however, ramage (1997) observes that software evaluation “often tends to go on in isolation from the ‘real world’ of work, in labs and the like.” (p. 775) from our own review of iso 14102 (lundell and lings, 2002), it is apparent that no consideration is given to the stakeholder dimension during an evaluation. we characterise such evaluations as ‘context-free’, in that no recognition is given to the feedback into the context. however, this is perhaps not surprising as the purpose of evaluation implied in the standard is summative, namely a go/no-go decision. even when considering evaluation of is development tools for software selection in context, there is a lack of consensus on what constitutes an appropriate context for an evaluation activity. in fact, silverman (1998) claims that “‘context’ itself is by no means a straightforward concept” (p. 6) as there may be a mismatch between an evaluator’s view and that of the (other) stakeholders. here we concentrate on the context for an evaluation, but in so doing elaborate further on the role of an evaluator. symons (1991) argues against what she sees as a perception common to much of the evaluation literature, namely that evaluation is a largely deterministic process, since such systems are part of and hence “inseparable from the organizational context” (p. 206). this view is based on the perception of an information system not only as a technical system but also as a dynamic social system. more recently, jones and hughes (2001) have noted that evaluation methods still tend to be “based on mechanistic and functional principles that are flawed in practice” (p. 189). this is clearly evident in the iso standard, in which the selection process involves the application of a “selection algorithm” (iso, 1995, p. 24) in order to decide whether to “acquire a tool or set of tools.” based on their experience, jones and hughes stress the importance of “an explicit concern for the social context” (p. 200). one can distinguish between an internal and an external context for an evaluation (serafeimidis and smithson, 1996) also referred to as inner and outer context (e.g. symons, 1991). an internal context consists of factors and ‘circumstances’ within an organisation that have an impact on the evaluation activity. examples include: stakeholder perspectives; stakeholder experiences and organisational history; informal communications and information flows within an organisation; organisational structure; political and organisational culture within an organisation; and corporate policies. it can also be important to consider which parts of an internal context represent the usage context (lundell and lings, 2003). typically, stakeholders in the usage context perceive that they may be strongly affected by the outcomes of an evaluation, and their motivations and goals are strongly affected by this belief. by contrast, stakeholders in the internal context but outside the usage context have higher-level (for example policy level) goals, which act more as constraints on evaluation activities. an external context includes that part which is typically beyond the control of the organisation in which an evaluation is undertaken. serafeimidis and smithson (1996, p. 207) claim that “the organization and its members need to respond to and accommodate” external factors to an evaluation, such as “national and local government policy, level of government support, markets and ajis vol. 12, no. 1 september 2004 45 market demands, competition, supplier availability and expertise, and other environmental pressures.” for the purpose of is development tool evaluations, iso-standards (and other ‘de jure’ standards, being defined external to an organisation) represent further examples of external contextual factors for an evaluation, since simply their existence (whether being considered or not) might, to a greater or lesser extent, affect an is development organisation. additionally, as part of a (high level) corporate policy within an is development organisation, there can also exist various corporate standards that individual projects must adhere to, for example, an organisational ‘standard’ for a modelling method and/or notation. finally, it is clear that the evaluator role is closely related to contextualisation. fetterman (1997) observes from the field of social program evaluations, based on the work of greene (1997), that an evaluator is always on somebody’s side, and that the choice of side being made by an evaluator is strongly related to whose question is being addressed in the evaluation. consequently, the evaluator’s choice of relevant factors and criteria to be considered in an evaluation activity is always related to the context in which the question to be addressed in the evaluation is expressed (fetterman, 1997). the activity dimension there is much general discussion in the literature of activities for tool evaluation, (e.g. budgen and thomson, 2003; kitchenham et al., 1997). few sources give detailed descriptions about how to conduct a tool evaluation activity, but a significant exception is the iso 14102 standard for evaluation, which identifies a rich set of activities. figure 3 gives an overview of a full evaluation process, which is divided into four main phases. most papers cover only a sub-set of these phases, and do not attempt to cover the whole process. however, the iso standard can be criticised for largely ignoring the other dimension of evaluation (lundell and lings, 2002), and in particular is weak with respect to acknowledging the stakeholder dimension. output is also limited to a simple, quantitative decision. initiation structuring evaluation selection processlegend: organisational needs evaluation framework evaluation report selection report product figure 3: a full evaluation process for is development tools, based on the iso 14102 standard evaluation activity process: initiation and structuring the survey instrument has been extensively used in a variety of contexts for assessing different stakeholder views (e.g. developer, user, researcher) of tools, and limitations associated with their use for tool evaluation have been discussed (e.g. coupe, 1994; coupe and onodu, 1997). although exceptions from large international organisations have been reported (e.g. maccari and riva, 2000, ajis vol. 12, no. 1 september 2004 46 2002), such surveys are often undertaken in the broad context of a general user (or developer) community. in other words, the interviewees in such assessments represent a context external to that in which a specific tool is used (or intended for use). for example, a variety of surveys have been undertaken to assess stakeholder perceptions (and/or experiences) of tools (e.g. lending and chervany, 1998; maansaari and iivari, 1999; post et al., 1998) in order to establish a general summative outcome from an evaluation activity. the content for such evaluation is typically based on a pre-defined set of issues/questions to be addressed. these may have been designed with the specific outcome in mind of making a recommendation for action, which can range from important lessons learnt, through guidelines for adoption practice to a definitive recommendation for adoption. alternatively, they may target the outcome of making a general assessment of the qualities of competing tools (or specific features of competing tools). in contrast, when an evaluation aims to establish an in-depth understanding of a specific organisation or usage context, qualitative approaches are often preferred. according to irani and love (2001, p. 185) there “is no doubt that interpretative approaches to it/is evaluation are needed so that a manager can gain an insight into how it/is investments will influence employee behaviour and performance.” sometimes, qualitative methods, such as grounded theory, have been used in evaluation for establishing developers’ or users’ experiences within specific organisations after adoption of specific tools. for example, cronholm (1995) used grounded theory to evaluate specific organisations after the adoption and use of specific case-tool products, and given that such tools are among the most complex products marketed today (post et al., 1999, p. 215) one could argue that statements made about case-tools generalise to all is products. there is an (implicit or explicit) underlying assumption in evaluation methods of an evaluation framework. such a framework comprises a representation of potentially relevant tool features and how they interrelate, sometimes referred to as a checklist. some published frameworks are aimed at general tool classes (e.g. ovum, 1993); others are tailored for use in pre-usage evaluation of (perhaps limited aspects of) a specific type of tool. for example, there are those designed for the analysis of: hypertext functionality in case-tools (e.g. kaipala, 1997); data modelling tools (e.g. moriarty, 1998); collaborative work (e.g. vessey and sravanapudi, 1995); conformance to a specific modelling notation (e.g. uml juric and kuljis, 1999); conformance to method rules in structured analysis (e.g. jankowski, 1997); usability of case-tools (e.g. cronholm, 1998). a number of problems have been acknowledged with published frameworks. firstly, published frameworks can differ significantly. for example, both henderson and cooprider (1990) and ovum (1993) have developed rather extensive frameworks for case-tools, but they differ in scope, structure and detail. secondly, frameworks must be continually updated to reflect technology changes. this problem is recognised explicitly in some frameworks (e.g. iso, 1995, p. 25). thirdly, even the developers of frameworks would accept that they cannot be used without modification in a specific usage situation, not least because frameworks may be interpreted differently in different contexts. most sources emphasise that the frameworks presented are to be taken as comprising potentially relevant aspects, to be considered in an evaluation if found appropriate for the evaluation task at hand. the method by which aspects are to be classified as appropriate is usually not elaborated. more fundamentally, it has been shown to be problematic to arrive at a broadly shared, in-depth understanding of even a single criterion that is defined a priori. this is clearly shown in an empirical and exploratory study (van reeken and trienekens, 1992) concerning method and case-tool usage, undertaken in sixteen of the largest organisations in the netherlands. each organisation was found to use the same terms in fundamentally different ways. this gives a strong motivation for developing an evaluation framework within a particular organisational setting, and even for a specific evaluation activity. evaluation activity process: evaluation and selection once an evaluation framework has been established an evaluation activity can proceed with a detailed analysis of tool features. various strategies have been proposed in the literature for this ajis vol. 12, no. 1 september 2004 47 stage, and several authors stress the importance of enforcing a rigid process for the actual evaluation and selection phases. for example, powell et al. (1997) claim that “evaluators should provide evidence that all relevant issues from [the content of the evaluation framework] have been addressed.” (p. 173) it should be emphasised that a selection decision can result in a decision not to adopt a tool. most descriptions of tool evaluation approaches are rather vague and provide limited detail on how to undertake an evaluation and selection process, and in this sense powell et al. (1997) is not untypical. however, as noted earlier, iso 14102 (iso, 1995) provides more detailed guidance, most notably on how to undertake the later phases of an evaluation activity. during the evaluation process, the tool “is evaluated in comparison with” the content of the evaluation framework that has been established. it is apparent that the view of an evaluation is seen as a process “of measurement, rating and assessment” (p. 19). the measurement can be undertaken in a variety of different ways, including examination of vendor-supported documentation, examination of code and the tool, and interviews with actual users of the tool. it is apparent that rating and assessment is mainly quantitative, and that measurements are undertaken at several levels of granularity using defined rating scales. specifically, an evaluation framework consists of the following levels: a set of characteristics (e.g. “functionality characteristics related to life-cycle processes” (p. 25)); characteristics (e.g. “development process” (p. 26)); subcharacteristic (e.g. “modeling” (p. 27)); and atomic subcharacteristic (e.g. “diagram development” (e.g. p. 27)). actual measurements are then aggregated “up to the characteristics level” (p. 21), and used as the basis for an evaluation report. in such, all evaluation results are provided and for each subcharacteristic, “the metric value measured” (p. 21) is given in terms of a rating level for that metric. for the actual selection process, iso (1995) use a “selection algorithm” (p. 23) and several different types are suggested in an appendix to the standard (iso, 1995), including those suggested for an organisation which addresses “evaluation data which are fuzzy or sparse and is having difficulties in amalgamating the viewpoints of multiple evaluators and/or selectors” (p. 53). among those mentioned are (iso, 1995, p. 53): borda’s algorithm (which is a “sum of the ranks” algorithm), condorcet’s algorithm (a “pairwise comparison” algorithm), dodgson’s algorithm (a “preference measurement” algorithm), fishburn’s algorithm (a “preference ordering” algorithm), and analytic hierarchy process (a “structured” algorithm). after completing the evaluation, a final validation of the selection decision (chapter 8.4 in iso 14102 (iso, 1995)) must be made against the organisation’s goals. “a check should be made to ensure that if the recommendation is accepted, the high level goals (or a sufficient number of them) will be met.” (iso 1995, p. 24) clearly, this validation is far from formal. the standard gives no advice on how to perform this task, however an associated technical report (iso, 1999) on how to use iso 14102 suggests using pilot studies in the later stages of evaluation: “a pilot project process should be conducted to provide a realistic trial for the case tools in their intended environment. while the tool was exercised during evaluation and selection, that process does not require a realistic use of the tool. evaluation and selection identifies the tool among the candidates with the most potential for the organization. the pilot project aims to be sure it can really perform for the organization in a real application.” (iso, 1999, p. 9) powell et al. (1997) also suggest the use of pilot studies, but in their case the suggestion is for their ‘early’ use in the overall activity. they stress that: “significant tool selection decisions should not be made without recourse to some form of pilot study either shadowing part of a project or applying it to a low risk part of a live project.” (p. 171) it is our belief that both views imply the importance of qualitative methods if the desirable goals of evaluation are to be met, since an inherent property of this type of evaluation is the potential subjectivity that comes with it, as expressed within the standard (iso, 1995) itself at several places. ajis vol. 12, no. 1 september 2004 48 different outcomes of an evaluation activity with the complexity involved in an evaluation activity, it is not always the situation that an evaluation outcome is necessarily a satisfactory problem solution. instead, almstrum et al. (1996) argue that a possible outcome can be simply “a better question.” (p. 202) their view of an evaluation process, as a long-term iterative activity that takes place in the context in which the phenomenon being evaluated is to be used, ultimately aims to illuminate the implications of a complex technology for that usage situation (almstrum et al., 1996, p. 202). thereby, the process can be viewed as a long-term learning process. the learning aspect has been emphasised by other authors with respect to evaluation of information systems. for example, etzerodt and madsen (1988) argue that: “information system evaluation should be seen as a process where the actors involved learn from their own experiences and not just as a process of measuring a fixed set of system values.” (p. 334) a very different view is exemplified by williams (1992) who, in the context of software product evaluation, states that: “[i]n order to compare products it is beneficial to reduce the information collected to a single score.” (p. 121) such an outcome can, essentially, be used as ‘input’ to decision making. as mentioned earlier, the goal for such evaluations has been characterised as “go/no-go” decisions (rossi and freeman, 1993, p. 47) and is, obviously, fundamentally different from an outcome which emerges as a consequence of an evaluation being undertaken with a learning emphasis. for such evaluations, the outcome can, instead, be an in-depth understanding that has been gained during the process of undertaking the evaluation activity. in other words, the outcome for such an activity is process oriented (where understanding and insight are central), whereas a ‘single score’ outcome from an evaluation very much has a product focus (where assessment and judgement are central). in reviewing work on a so-called “crisis in utilization”, blackler and brown (1988) observe a “general tendency to treat evaluation in isolation” (p. 361) from corporate policy and decision making, and not to use evaluation results “to aid long term organizational objectives.” (p. 362) for example, whatever the purpose of an evaluation activity, significant effort will have been expended on developing an evaluation framework. this framework will reflect both long-term objectives and pragmatic issues concerning specific tools under evaluation. it has been our experience that these aspects are distinct but related, and that in practice two evaluation frameworks should be maintained. the first of these represents only long-term organisational need, the second is instantiated for use in the specific evaluation activity in progress. even in a ‘go/no-go’ evaluation activity, the long-term evaluation framework can be seen as a useful outcome and could form a basis for organisational learning. dissemination of evaluation findings is an important aspect for any evaluation effort. communicating evaluation frameworks to stakeholders is an important facet of dissemination. however, other in-depth insights, and experiences gained by stakeholders involved in an evaluation activity undertaken within an organisational context, do not speak for themselves (fetterman, 1997). therefore, it is important that each specific evaluation activity has an appropriate means for communicating findings and results to stakeholders that are (directly or indirectly) affected by the outcome of the activity. fetterman (1997) argues that the evaluator has an important role in this communication process. conclusions in this paper we have critically considered the literature related to the evaluation of is development tools, and in particular that related to evaluation for possible adoption. where relevant, we have critically considered the iso standard on evaluation, both in terms of its contribution to the debate and in terms of its implied shortcomings. from our analysis, three primary dimensions are evident in which any evaluation is planned and carried out: the stakeholder dimension, the contextual dimension, and the activity dimension. we ajis vol. 12, no. 1 september 2004 49 have described each dimension, and explained why each should be actively considered in any evaluation effort. we summarise these findings below. stakeholders for an evaluation to succeed, that is produce valid and accepted findings, stakeholders must feel able to actively participate, and their beliefs and assumptions must be considered. validity is enhanced by the multiple perspectives offered by the inclusion of representative stakeholders, but effective and ongoing feedback is essential to preserve a sense of ownership. however, evaluation is a political activity, potentially strongly affecting stakeholders, so selection of stakeholders and the choice of evaluator are critical factors for success. when selecting stakeholders, it is important to have a clear strategy. for example, there may be a need to cover all aspects of a process life-cycle, or to include all roles within the context potentially affected by the evaluation activity. it is important to establish clear goals, but also to acknowledge that each stakeholder will have individual motivations and personal goals. goals can be very diverse, from educational goals to financial assessment and control. it is also possible to target several goals simultaneously, for example targeting both organisational learning and tool selection within the same activity. finally, there is the special role of evaluator, which may be taken by stakeholders themselves or may involve an external agency. when selecting an evaluator, there is a clear choice between appointing a stakeholder and using an external agency. these selection decisions are clearly political decisions, and should ideally be resolved transparently, and to the satisfaction of all stakeholders, prior to the commencement of any evaluation activity. failure to consider the stakeholder dimension can result in a feeling of lack of ownership of the process by stakeholders, and undesirable outcomes. these can range from poor tool selection, with consequential financial penalties, to political fallout. context serafeimidis and smithson (1996) separate context into internal, external, and history. there is also an argument for discriminating further with respect to internal context, differentiating between usage context and organisational context according to closeness to the phenomenon being evaluated (lundell and lings, 2003). for example, a tool user will be considered part of the usage context whereas a manager involved in policy documents for tool adoption will be considered as part of the more general organisational context. further, in the framework presented, history is considered only as one facet of each context, rather than a separate element, as it is important to emphasise the continuing change of contexts. failure to consider the contextual dimension can result in outcomes of limited relevance to the organisational setting, and in the extreme an evaluation becomes a context free comparison of tool features. activity a number of different approaches, and associated sets of activities, have been proposed for evaluation. the most descriptive of these is the iso 14102 standard (iso, 1995; lundell and lings, 2002) for case-tool evaluation and selection. it is clear that an appropriate set of activities for a given evaluation situation is very dependent on underlying assumptions about the context of the activities and the stakeholders involved in those activities. failure to consider the activity dimension can result in poorly conducted evaluations and unreliable outcomes, leading to a lack of stakeholder confidence in the whole activity. ajis vol. 12, no. 1 september 2004 50 further implications in considering the literature in the light of this framework, several specific weaknesses in current approaches are apparent. in brief, these relate to: • (implicit) assumptions about the ability to separate consideration of organisational needs from those of current technology when composing an evaluation framework; • lack of method support for considering both of the above aspects within an evaluation, and for adapting pre-existing frameworks; • the sequential nature of the assumed overall process in iso 14102, with an implied mutual independence of the outputs from each component process. we have earlier claimed that evaluation methods require improved support for evaluation framework development (lundell and lings, 2003). in the light of our analysis we would go further, and suggest that all phases of an evaluation process require enhanced support. in particular, we see the following as open questions worthy of further research: • how can evaluation frameworks be enhanced to effectively capture information related to the impact on an organisation should an is development tool fail to offer support for an implied requirement? • what decision mechanisms are appropriate for supporting selection through a full evaluation given an evaluation framework so enhanced? we believe that these questions must be answered in the full light all of the elements of evaluation identified in this paper, and in particular the stakeholder and contextual dimensions. references almstrum, v.l., dale, n., berglund, a., granger, m., little, j.c., miller, d.m., petre, m, schragger, p. & springsteel f. (1996) “evaluation: turning technology from toy to tool: report of the working group on evaluation”, acm sigcse bulletin, vol. 28 (special issue), pp 201217. avgerou, c. (1995) “evaluating information systems by consultation and negotiation”, international journal of information management, vol. 15 (6), pp 427-436. bailer, b., bauknecht, k. & schatzmann, c. (1993) “ways to case: an analysis of selected case experiences”, in h.-y. lee, t.f. reid & s. jarzabek (eds.) proceedings of the sixth international workshop on computer-aided software engineering: case ’93, ieee computer society press, los alamitos, california, pp 36-43. blackler, f. & brown, c. (1988) “theory and practice in evaluation: the case of the new information technologies”, in n. bjørn-andersen, g.b. davis (eds.) information systems assessment: issues and challenges proceedings of the ifip wg 8.2 working conference on information systems assessment, north-holland, amsterdam, pp 351-367. brandon, p.r. (1998) “stakeholder participation for the purpose of helping ensure evaluation validity: bridging the gap between collaborative and non-collaborative evaluations”, american journal of evaluation, vol. 19 (3), pp 325-337. brown, a.w. & wallnau, k.c. (1996) “framework for evaluating software technology”, ieee software, vol. 13 (5), 39-49. budgen, d. & thomson, m. (2003) “case tool evaluation: experiences from an empirical study”, journal of systems and software, vol. 67 (2), pp 55-75. carney, d.j. & wallnau, k.c. (1998) “a basis for evaluation of commercial software”, information and software technology, vol. 40 (14), pp 851-860. chelimsky, e. (1997) “the coming transformations in evaluation”, in e. chelimsky & w.r. shadish (eds.) evaluation for the 21st century: a handbook, sage publications, thousand oaks, california, pp 1-26. coupe, r.t. (1994) “a critique of the methods for measuring the impact of case software”, european journal of information systems, vol. 3 (1), pp 28-36. ajis vol. 12, no. 1 september 2004 51 coupe, r.t. & onodu, n.m. (1997), “evaluating the impact of case: an empirical comparison of retrospective and cross-sectional survey approaches”, european journal of information systems, vol. 6 (1), pp 15-24. cronholm, s. (1995) “why case tools in information systems development? an empirical study concerning motives for investing in case tools”, in b. dahlbom, f. kämmerer, f. ljungberg, j. stage & c. sørensen (eds.) proceedings of the 18th information systems research seminar in scandinavia: iris 18 “design in context”, gothenburg studies in informatics, report 7, pp 132-144. cronholm, s. (1998) metodverktyg och användbarhet en studie av datorstödd metodbaserad systemutveckling, ph.d. thesis, department of computer science, linköping university linköping, sweden (in swedish). etzerodt, p. & madsen, k.h. (1988) “information systems assessment as a learning process”, in n. bjørn-andersen & g.b. davis (eds.) information systems assessment: issues and challenges proceedings of the ifip wg 8.2 working conference on information systems assessment, north-holland, amsterdam, pp 333-345. farbey, b., land, f. & targett, d. (1999) “moving is evaluation forward: learning themes and research issues”, journal of strategic information systems, vol. 8 (2), pp 189-207. fetterman, d. (1997) “empowerment evaluation: a response to patton and scriven”, evaluation practice, vol. 18 (3), pp 253-266. forss, k., cracknell, b. & samset, k. (1994) “can evaluation help an organization to learn?”, evaluation review, vol. 18 (5), pp 574-591. greene, j.c. (1997) “evaluation as advocacy”, evaluation practice, vol. 18 (1), pp. 25-35. guba, e.g. & lincoln, y.s. (1989) fourth generation evaluation, sage publications, newbury park. henderson, j.c. & cooprider, j.g. (1990) “dimensions of i/s planning and design aids: a functional model of case technology”, information systems research, vol. 1 (3), pp 227254. hughes, j. & wood-harper, t. (2000) “an empirical model of the information systems development process: a case study of an automotive manufacturer”, accounting forum, vol. 24 (4), pp 391406. ieee (1992) ieee recommended practice for the evaluation and selection of case tools, ieee std 1209-1992, december 13, ieee standards board, new york. irani, z. & love, p.e.d. (2001) “information systems evaluation: past, present and future”, european journal of information systems, vol. 10(4), pp 183-188. iso (1995) information technology guideline for the evaluation and selection of case tools, iso/iec jtc1/sc7/wg4, iso/iec 14102:1995(e). iso (1998) information technology software product evaluation part 5: process for evaluators, iso/iec 14598-1:1998(e), 1998-07-01. iso (1999) tr 14471 software engineering guidelines for the adoption of case tools, iso/iec jtc1/sc7 wg4, iso/iec, 1999-04-20. iso (1999b) information technology software product evaluation part 1: general overview, iso/iec 14598-1:1999, 1999-04-15. jackson, b. & sulaksono, s. (1998) “going soft on information systems evaluation”, the australian journal of information systems, vol. 5 (2), pp 41-50. jankowski, d. (1997) “computer-aided software systems”, empirical software engineering: an international journal, vol. 2, 11-38. jones, s. & hughes, j. (2001) “understanding is evaluation as a complex social process: a case study of a uk local authority”, european journal of information systems, vol. 10 (4), pp 189-203. juric, r. & kuljis, j. (1999) “building an evaluation instrument for oo case tool assessment of unified modelling language support”, in proceedings of the thirty-second annual hawaii international conference on system science [cd-rom], ieee computer society press. ajis vol. 12, no. 1 september 2004 52 kaipala, j. (1997) “augmenting case tools with hypertext: desired functionality and implementation issues”, in a. olivé & j.a. pastor (eds.) advanced information systems engineering: 9th international conference, caise‘97, springer, berlin, pp 217-230. kitchenham, b.a. (1996) “evaluating software engineering methods and tool part 1: the evaluation context and evaluation methods”, software engineering notes, vol. 21 (1), pp. 1115. kitchenham, b.a. & jones, l. (1997) “evaluating software engineering methods and tool part 5: the influence of human factors”, software engineering notes, vol. 22 (1), pp 13-15. kitchenham, b., linkman, s. & law, d. (1997) “desmet: a methodology for evaluating software engineering methods and tools”, computing & control engineering journal, vol. 8 (3), pp 120-126. lending, d. & chervany, n.l. (1998) “case tools: understanding the reasons for non-use”, acm sigcpr computer personnel, vol. 19 (2), pp 13-26. lundell, b. & lings, b. (2002) “comments on iso 14102: the standard for case-tool evaluation”, computer standards & interfaces, vol. 24 (5), pp 381-388. lundell, b. & lings, b. (2003) “the 2g method for doubly grounding evaluation frameworks”, information systems journal, vol. 13 (4), pp 375-398. maansaari, j. & iivari, j. (1999) “the evolution of case usage in finland between 1993 and 1996”, information & management, vol. 36 (1), pp 37-53. maccari, a. & riva, c. (2000) “empirical evaluation of case tools usage at nokia”, empirical software engineering, vol. 5 (3), pp 287-299. maccari, a. & riva, c. (2002) “on case tool usage at nokia”, in 17th ieee international conference on automated software engineering (ase’02), ieee computer society, 23 27 september 2002, edinburgh, pp 59-68. mcauley, l., doherty, n. & keval, n. (2002) “the stakeholder dimension in information systems evaluation”, journal of information technology, vol. 17 (4), pp 241-255. moriarty, t. (1998) “searching for the right data modeling tool”, dm review, vol. 8 (6), pp 7476. ovum (1993) ovum evaluated: case products, ovum, london, england, january. patton, m.q. (1996) “a world larger than formative and summative”, evaluation practice, vol. 17 (2), pp 131-144. post, g., kagan, a. & keim, r.t. (1998) “a comparative evaluation of case tools”, journal of systems and software, vol. 44 (2), pp 87-95. post, g.v., kagan, a. & keim, r.t. (1999) “a structural equation evaluation of case tools attributes”, journal of management information systems, vol. 15 (4), pp 215-234. powell, a., vickers, a, williams, e. & cooke, b. (1997) “a practical strategy for the evaluation of software tools”, in s. brinkkemper, k. lyytinen & r.j. welke (eds.) method engineering: principles of method construction and tool support proceedings of the ifip tc8 wg8.1/8.2 working conference on method engineering, chapman & hall, london, pp 165185. ramage, m. (1997) “developing a methodology for the evaluation of cooperative systems”, in k. braa, e. monteiro (eds.) proceedings of the 20th information systems research in scandinavia, conference proceeding nr. 1, department of informatics, university of oslo, norway, pp 769-789. van reeken, a.j. & trienekens, j.j.m. (1992) the practical importance of methods and case tools: results of empirical research, merit 92-015, maastricht economic research institute on innovation and technology, university of limburg, maastricht, netherlands. rehbinder, a., lings, b., lundell, b., burman, r. & nilsson, a. (2001) “observations from a field study on developing a framework for pre-usage evaluation of case-tools”, in n.l. russo, b. fitzgerald & j.i. degross (eds.) new directions in information systems development: ifip wg 8.2 2001 conference, kluwer, boston, pp 211-220. remenyi, d., sherwood-smith, m. & white, t. (1997) achieving maximum value from information systems: a process approach, john wiley & sons, chichester. ajis vol. 12, no. 1 september 2004 53 rossi, p.h. & freeman, h.e. (1993) evaluation: a systematic approach, sage publications, newbury park, london. scriven, m. (1996) “types of evaluation and types of evaluator”, evaluation practice, vol. 17 (2), pp 151-161. serafeimidis, v. & psoinos, a. (1995) “emerging issues and a holistic approach for information technology investments evaluation into the 1990’s”, in 5th hellenic conference on informatics, athens, 7-9 december 1995. serafeimidis, v. & smithson, s. (1996) “the management of change for information system evaluation practice: experiences from a case study”, international journal of information management, vol. 16 (3), pp 205-217. serafeimidis, v. & smithson, s. (2003) “information systems evaluation as an organizational institution – experience from a case study”, information systems journal, vol. 13 (3), pp 251274. silverman, d. (1998) “qualitative research: meanings or practices?”, information systems journal, vol. 8 (1), 3-20. smyth, r.w. (1998) “use of case tools by a software vendor: a study of success indicators”, in w.r.j. baets (ed.) proceedings of the 6th european conference on information systems, euro-arab management school, aix-enprovence, pp 571-583. symons, v. (1990) “evaluation of information systems: is development in the processing company”, journal of information technology, vol. 5 (4), pp 194-204. symons, v.j. (1991) “a review of information systems evaluation: content, context and process”, european journal of information systems, vol. 1 (3), pp 205-212. symons, v. & walsham, g. (1988) “the evaluation of information systems: a critique”, journal of applied systems analysis, vol. 15, pp 119-132. vessey, i. & sravanapudi, a.p. (1995) “case tools as collaborative support technologies”, communications of the acm, vol. 38 (1), pp 83-95. walsham, g. (1993) “evaluation”, in g. walsham (ed.) interpreting information systems in organizations, john wiley & sons, chichester, england, pp 165-185. williams, f. (1992) “appraisal and evaluation of software products”, journal of information science: principles & practice institute of information scientists, vol. 18 (2), pp 121-125. the australasian journal of information systems (ajis) began in 1993 and was based on the simple realisation that there was a need for an academic ‘outlet’ for is research in australia australasian journal of information systems volume 14 number 1 nov 2006 the information systems discipline in australian universities: a contextual framework professor guy gable it professional services research program faculty of information technology queensland university of technology brisbane, queensland, australia email: g.gable@qut.edu.au abstract this paper presents the contextual framework for a multi-method, multi-study of the state of the information systems academic discipline in australia, and relates the genesis of the australian study as preliminary to a larger pacific-asia study. analysis of prior literature on the state of is and on relevant theory, underpins a series of individual australian state case studies that are combined with several conceptual analyses and a research issues survey. the paper outlines the methodological approach employed, with emphasis on the case study method of the multiple state studies. the process of multiple peer review of the studies is described. importantly, this paper summarises and analyses each of the related sub-studies, also outlining ongoing work, beyond the overarching study reported herein. introduction this paper presents the contextual framework for a multi-method, multi-study titled ‘the information systems discipline in australian universities’ (the ‘is-in-oz’ study)1. drawing on whitley’s theory of scientific change, the study analyses the degree of ‘professionalisation’ of the information systems discipline, the overarching research question being “to what extent is information systems a distinct and mature discipline in australia?” this australian study was spawned by a broader study of ‘the state of the information systems academic discipline in pacific asia’ (is-in-pa), the results of which are forthcoming in a similarly titled special issue of communications of the ais (cais)2. a main aim of the is-in-oz study is to initiate an ongoing, longitudinal evaluation of the state of the is academic discipline in australia. the ‘parent’ is in pacific asia study (is-in-pa) design figure 1 depicts the main components of the ‘parent’ is-in-pa study. the pacific-asia study (and its component australian study) is motivated from a recognition that information systems (is) as an academic discipline has evolved differentially around the world. for example, there is regional variation in the strength of its presence as an academic discipline; it may take on identifiably different local forms e.g. from a soft systems emphasis to a more technical focus; there may be 103 australasian journal of information systems volume 14 number 1 nov 2006 regional differences in topics taught and researched (as was observed by avgerou et al. (1999) across europe). the genesis of the study was a panel of the 6th pacific asia conference on information systems (pacis’02), tokyo, japan 2-4 september 2002, ultimately resulting in formal project commencement end 2004 with ais endorsement and seed funding3. australiakorea new zealandtaiwanchina (mainland) (china) hong kong singapore pakistanthailand multistate case study is placement swot methodological ar fac directory archival analysis is research issues delphi study is-in-pa pacis archival analysis malaysia india japan australiakorea new zealandtaiwanchina (mainland) (china) hong kong singapore pakistanthailand multistate case study is placement swot methodological ar fac directory archival analysis is research issues delphi study is-in-pa pacis archival analysis malaysia india japan figure 1 – the is in pacific asia region study (is-in-pa) the is-in-pa study includes nine main study components (see figure 1), the principal of which is a multiple case study across six states of the pacific asia region4 (pa). it was decided early on to restrict the first iteration of the is-in-pa study to those areas in the region where is is relatively more visible internationally – australia, hong kong (china), korea, new zealand, singapore and taiwan – the intent being in future to incrementally extend the study to other parts of the region5. table 1 lists the main is-in-pa study leaders. state study leader australia professor guy g gable, queensland university of technology hong kong (china) professor patrick chau, the university of hong kong korea professor jae-nam lee, korea university new zealand professor sid huff, the university of wellington singapore professor bernard tan, the national university of singapore taiwan professor tp liang, national sun yat-sen university table 1 – the is in pacific asia (is-in-pa) study team both the is-in-pa and is-in-oz studies have from the outset been designed and executed with the expectation that they will be extended and repeated over time. shaded ovals in figure 1 represent those components that are being completed in the first execution, with the detailed australian results 104 australasian journal of information systems volume 14 number 1 nov 2006 reported in a special issue of australasian journal of information systems (ajis)6. unshaded ovals represent components that are in progress (e.g. mainland china case study, is research issues delphi study, methodological action research/learnings) and dashed ovals represent planned components soon to commence (note that further study components are expected to evolve). the australian study (is-in-oz) design a meeting of a sub-group of the is-in-pa study team in auckland january 2004 (gable, huff, tan7) agreed that australia, having been active in is academe since the 70’s, and having a correspondingly long and internationally visible history of is research, would provide a useful ‘pilot’ study in advance of extending the multiple-state case study to other parts of pacific asia. this resulted in a proposal to conduct a multiple-case study of the australian states – the is-in-oz study (figure 2). table 2 lists the main is-in-oz study team members. home state member queensland professor guy g gable, queensland university of technology dr. robert smyth, queensland university of technology act dr. roger clarke, principal xamax consultancy pty ltd; and visiting professor, anu professor shirley gregor, australian national university professor ed lewis, australian defence forces academy, university of new south wales associate professor craig mcdonald, university of canberra south australia professor andy koronios, university of south australia associate professor mike metcalfe, university of south australia professor paula swatman, university of south australia tasmania dr. gail ridley, university of tasmania new south wales professor ernest jordan, macquarie university dr. jim underwood, university of technology -sydney victoria dr. elsie chan, australian catholic university associate professor carol pollard, appalachian state university professor graeme shanks, monash university west australia professor janice burn, edith cowan university dr chad lin, edith cowan university professor graham pervan, curtin university of technology professor craig standing, edith cowan university table 2 – the is in australia (is-in-oz) study team 105 australasian journal of information systems volume 14 number 1 nov 2006 as in figure 1, in figure 2 shaded ovals represent those components that have been completed and are reported together in the special issue of ajis. unshaded ovals represent components that are in progress (e.g. acis archival analysis) and which will be reported in future. there are 12 shaded ovals in figure 2, corresponding with the 12 completed is-in-oz sub-studies. these include 3 conceptual studies (the contextual framework, the history, and the theory base) and 9 empirical papers (7 state case studies, a research survey and a 'contradictions' piece)8. multiple case study (4) qld state report (smyth, gable 2006) (6) nsw state report (underwood, jordan 2006) (10) wa state report (burn, standing, lin 2006) (8) tas state report (ridley 2006) (7) sa state report (koronios, swatman 2006) acis archival analysis (2) history (clarke 2006) (12) research status survey (pervan, shanks 2006) is research issues delphi study methodological ar australia (1) contextual framework (gable 2006) korea new zealandtaiwanchina (mainland) (china) hong kong singapore pakistanthailand multistate case study is placement swot methodological ar fac directory archival analysis is research issues delphi study is-in-pa metaanalysis pacis archival analysis malaysia india japan (11) sa contradictions piece (metcalfe 2006) (3) theory base (ridley 2006) (9) vic state report (pollard, chan 2006) (5) act state report (gregor, lewis, mcdonald 2006) multiple case study (4) qld state report (smyth, gable 2006) (6) nsw state report (underwood, jordan 2006) (10) wa state report (burn, standing, lin 2006) (8) tas state report (ridley 2006) (7) sa state report (koronios, swatman 2006) acis archival analysis (2) history (clarke 2006) (12) research status survey (pervan, shanks 2006) is research issues delphi study methodological ar australia (1) contextual framework (gable 2006) korea new zealandtaiwanchina (mainland) (china) hong kong singapore pakistanthailand multistate case study is placement swot methodological ar fac directory archival analysis is research issues delphi study is-in-pa metaanalysis pacis archival analysis malaysia india japan (11) sa contradictions piece (metcalfe 2006) (3) theory base (ridley 2006) (9) vic state report (pollard, chan 2006) (5) act state report (gregor, lewis, mcdonald 2006) figure 2 – the is in australia (is-in-oz) study9 the is-in-oz (and is-in-pa) study has from the outset sought to establish the ‘beginnings’ of a cumulative and ongoing effort to track and report on, and reflect upon the evolution, state and diffusion of the is academic discipline in australia (and pacific asia, and ultimately other world regions). note that the study was never intended to be an exploration of the ‘core of the is discipline’10, though it is anticipated that the is-in-oz results (and those from the is-in-pa study and any subsequent replication) will contribute to that discussion. this is-in-oz study seeks to draw upon and complement other recent or planned studies of the state of the is discipline, notably those of avgerou et al (1999) in europe, and pervan and shanks (2004) in australia. the is-in-oz study, being exploratory and descriptive, and to some extent a pilot, did 106 australasian journal of information systems volume 14 number 1 nov 2006 not commence from a highly specific theory base. the ridley (2006) theory-base11 evolved throughout the study, largely in parallel with the study, and somewhat influenced by the study. a key study aim was to evolve and apply (and ‘test’) a process of evidence collection and review, for future extension and possible replication within the region and across the other world regions. this was to some extent in attention to past concern expressed (e.g. by phillip ein-dor in (gable 2002)) with the lack of a methodology and indicators for tracking diffusion of the is discipline. it was posited that the establishment of measures and indicators of the state of is, and a baseline snapshot of its current state, will facilitate tracking of the state and monitoring the effect of initiatives to promote is as a discipline. while emphasis here is on australia, many of the ideas, mechanisms and aims are generalisable to all ais regions. thus, one overarching aim of the study is to contribute to a general methodology with which to describe and monitor the evolving state of the is discipline in any region or country. other more specific study aims included: • to begin documenting characteristics of is programs across universities in australia. • to begin documenting characteristics of is research across universities in australia. • to begin assessing the strength of the is presence in australian universities. • to evaluate the maturity of is as an academic discipline in australia (as per the theory-base). • to identify emerging trends in is in australian universities. • to identify main influences on is in australian universities. study questions study questions and the units-of-analysis (sometimes the university; sometimes the state) too evolved over time, along with the study design, with varying emphasis across the sub-studies. broad study questions implicit in the final case study protocol12 include13 : • what is the relative size of the is presence at the university? • what is the administrative placement of is (including changes over time)? • to what extent has is at the university been impacted by local contingencies? • to what extent is is identified as a separate field at the university? • what are the distinctive features of the is curriculum at the university? • what are the distinctive features of is research at the university? • who are the key people who have impacted is in universities in the state? questions posed to within-state reviewers of the state reports included: • what to you believe to be the main challenges facing the is academic discipline in your state today? • what do you believe to be the main opportunities facing the is academic discipline in your state today? similar questions were posed to the global reviewers of the full set of related sub-study reports (the complete draft special issue of ajis). in concluding this section, it is noted that though each of the papers in the special issue of ajis has been written to stand alone, the papers in combination form an interrelated set of rich descriptions (if partial), analyses and interpretations of the state of the is academic discipline in australia as of this writing. detailed and more interpretive background and history is presented in the historical analysis. the next section briefly reviews literature on past studies of the is discipline and relevant theory. the subsequent section summarises the overall study approach, with emphasis on 107 australasian journal of information systems volume 14 number 1 nov 2006 repeatability a key study aim. next, overall study outcomes are related. the final section describes study limitations and potential further research. the literature past studies of the is discipline14 articles discussing the state of information systems tend to revolve around several themes: (1) the identity crisis within is, (2) is as an academic field, (3) the state of is research, and (4) the evolution of the field of is. one dominant theme is the existence or not of an identity crisis within is, concern being that the discipline's central identity is ambiguous (benbasat & zmud, 2003). articles debating the core and scope of is are several. the debate in recent times culminated in a series of 11 articles published in the communications of the ais. this is an important debate as the degree of convergence of a discipline can have political implications. ‘convergent communities are favourably placed to advance their collective interests since they know what their collective interests are, and enjoy a clear sense of unity in promoting them’. (becher, 1989, p160) the academic field of information systems is another recurring theme. avgerou et al (1999) comprehensively discuss the academic field of information systems in europe while other authors concentrate on a single topic. topics of discussion include: the status of is as an academic discipline (introna, 2003; khazanchi & munkvold, 2000); is educational programs (ang & jiwahhasuchin, 1998; lo, 1989); and the location of is departments in universities (sherer, 2002). articles on the state of is research include: paradigmatic and methodological examinations of is research (chen & hirschheim, 2004; orlikowski & baroudi, 1991; vessey, ramesh, & glass, 2002); regional differences in is research (evaristo & karahanna, 1997); and themes of is research (bacon & fitzgerald, 2001; palvia, leary, mao, pinjani, & salam, 2004). there are two types of article on the evolution of information systems: those that assess the current status of the field by tracing its historical evolution and the driving forces that shape it (adam & fitzgerald, 2000), and those that gauge the status of information systems development and evolution, by examining changes over time in topics, themes and research strategies in the literature (alavi & carlson, 1992; claver, gonzalez, & llopis, 2000; farhoomand & drury, 1999). relevant theory the study reported in this volume (is-in-oz) aims to investigate the state of the information systems academic discipline in australia from historical and current perspectives, collecting evidence across a range of dimensions. to maximise the descriptive potential of the study, the results need to be capable of integration, so that the relationships within and across the dimensions and geographical units are understood. a meaningful theoretical framework helps relate the results of the different dimensions of the study to characterise the discipline in the region, and assist in empowering the australian is research community. the ridley (2006) theory paper (paper #3 in the ajis special issue) reviewed literature on the development of disciplines, before deriving a theoretical framework for the broader study reported elsewhere in this volume. the framework considered the current and past state of is in australian universities from the perspective of the development of a discipline. the components of the framework were derived and validated through 108 australasian journal of information systems volume 14 number 1 nov 2006 a thematic analysis of both the is and non-is literature. the framework developed in the ridley paper, which has been partly guided by whitley’s theory of scientific change (1984a, 1984b), was used to analyse data collated from the australian regions. the degree of variation in australian is as an indication of its “professionalisation” and its maturity as a profession were used to frame the analysis. study approach following, study logistics are described. the overall study design is reflected in figure 2 above. sub-study methods and methodology are addressed in the individual papers of the volume. some details of the case study method are included below in order to avoid repetition across the 7 state case reports. overview the study process was a combination of deductive, top-down broad definitions of aims, questions and procedures; and inductive, bottom-up consideration of sources of evidence, project resources and feasibility. it could be said that early results were mainly inductive, these being followed by a more deductive, top-down review. project management individual state teams managed their respective case studies, with the overall project managed by guy gable (project leader) and bob smyth (project manager). main mechanisms of project management were: (1) a project web site, (2) regular status meetings of the project leader and project manager and related project status reporting by email, and (3) face-to-face team meetings as feasible (on several occasions in conjunction with acis or acphis). (4) several teleconference and skype sessions were also held, as a cost-effective alternative to face-to-face. (5) the case study protocol, too, served as a valuable mechanism for coordinating the study teams. complexities and issues encountered a range of complexities was encountered throughout the study, some of which were fully or partially overcome; others of which were not. several pertained to the scope and object of the study: academe vs the profession vs both; teaching vs research; past vs present; … there was discussion around the utility of ‘state’ based case studies, it being mooted that other groupings of institutions might be more meaningful e.g. sandstones vs the others; large is vs small is (i.e. based on numbers of undergraduate is students). the state case studies the case studies were largely exploratory and descriptive, with relatively lesser emphasis on interpretation and generalisability. the case study approach is well suited for investigation where there is little established theory on the topic (yin, 2003). the case study method also has the advantage of allowing the researcher to develop a good feeling for the complexities of interacting forces and subtleties that are more difficult to detect with other methods. walsham (1995) refers to the “rich insight” possible from the case study approach. topics explored in the case studies the data collected in the case studies concentrated on the following topics: • relative size of the is presence at the university • the administrative placement of is (including changes over time) 109 australasian journal of information systems volume 14 number 1 nov 2006 • the extent to which is at the university is impacted by local contingencies • the extent to which is is identified as a separate field at the university • distinctive features of the is curriculum at the university • distinctive features of is research at the university • the key people who have impacted is in universities in that state team data gathering it was planned that team members in australian states and territories (henceforth referred to collectively as the study ‘states’) would gather qualitative data about each university (as well as relevant state-level data). the case study method was agreed as the research approach for the team study. the data gathered was intended to provide insights into the distinctive characteristics of is in each university in relation to: degree of administrative autonomy, size and influence, curriculum, research, local influences, and significant persons in shaping is at that university. anchoring the data gathering and analysis was gail ridley’s (2006) evolving draft framework derived from theory on the emergence of disciplines. team members in each state sought to analyse the data across the universities engaged in is in that state, and to present general observations on the features of is as an academic discipline in that state. the protocol yin argues for the use of a case study protocol to guide any study employing the case study method. to this end, a multi-state case study protocol was developed for use by study team members. in this instance, it was intended that use of the protocol would contribute to: • comparability across the states • consistency across the individual case studies, and • efficiency in the conduct of the case studies, with potential for data gathering and some analysis being delegated to research assistants or other junior researchers yin strongly favours building a protocol around relevant theory. in this study, the protocol relies on a framework, emergence of a discipline, developed by ridley and articulated fully elsewhere in the ajis special edition. in practice, the ridley framework was refined in parallel with the data gathering and analysis for the individual state case studies. thus, data gathering in most states was guided by a partial version of the final framework, incorporating two constructs: (1) degree of professionalisation as a discipline and (2) maturity as a scientific field. both derive from whitley’s theory of scientific change (1984a, 1984b). also, many of the tenets of general systems theory (ackoff, 1971) are implicit in the approach to data gathering and the themes and questions documented in the multi-state case study protocol. the approach to data gathering advocated, based on semi-structured interviews utilising broad themes to tap the perceptions of interviewees, is consistent with general systems theory. this approach permits the researcher to take a more holistic approach to the topic, and allows the interviewee to touch on the multiplicity of interacting factors that might contribute to the distinctive characteristics of the is presence in each university. the protocol directs the researcher to just some of the potential interacting factors that might determine the distinctive characteristics of is at a given university: that the interaction of geography, administrative structure, individuals from within and outside the university, over time may influence curriculum and research at that university. another assumption implicit in the protocol, supported by the work of management researchers and is people like mcfarlan, nolan and norton (1973), is that form influences function. the protocol suggests that to know the structure of the is academic group and its position in the university's administrative framework is a good starting point for looking at the nature of is curriculum and research at that university. 110 australasian journal of information systems volume 14 number 1 nov 2006 the case study protocol represents “the main research mechanism” for the seven case studies15. in its earliest draft, the case study protocol lacked an explicit theory framework. instead, it relied solely on the principles of general systems theory and of form influencing function to guide the data gathering and analysis. this limited theory base was unsettling to some members of the study team. consequently, at a study team meeting in hobart, in december 2004, attention was given to deciding on a suitable, stronger theory base to underpin the case study protocol. the meeting agreed that prior work by team member gail ridley offered promise of a suitable framework for this study. dr ridley undertook to prepare a draft framework paper. in response to feedback from team members, this draft framework underwent several iterations, in parallel with modifications to the protocol and actual data gathering in several states. the form of the ridley framework, as described elsewhere in this edition was finalised late in the data gathering process for the individual states. hence, not all states were able to avail themselves of the full framework during the data gathering process but most chose to consult this latest framework in considering their analysis of the data gathered (it is anticipated that the framework will be of greatest use to researchers wishing to extend the australian study in other regions). regardless, study team members from the different states chose to follow the protocol to varying extents. the study favoured an approach that maximised the capacity for the state team members to draw out the distinctive features of is in universities in each state; thus, conscious deviation from the protocol was not discouraged where such deviations achieved the objective of highlighting distinctive characteristics of any state. this acceptance of deviation from the protocol was reinforced by the dynamic nature of the protocol, as the guiding framework evolved. in future studies it could be effective for all team members to reach consensus on the case study protocol prior to commencement of each study, with more uniform adherence to the agreed protocol. the queensland pilot to guide the direction of the state case studies, a pilot case was conducted in queensland. the early version of the case study protocol was followed carefully in data gathering and analysis. in the initial queensland write-up, additional contextual information was provided (e.g. explanation of the motivation for the single queensland state study in relation to the larger australia-wide study and the foreshadowed pacific-asia study) to enable the report to stand alone for review and reference purposes. the write-up of the queensland pilot case study was then made available to queensland interviewees and to all team members, across the states, for feedback. to the same end, the report of the queensland study was made available on the team website. the pilot was subsequently discussed at a team meeting in canberra. the approach adopted in the queensland pilot was endorsed as appropriate for the other states. this pilot approach to the case studies proved an effective tactic. mechanisms to increase representativeness given the descriptive and exploratory character of the overall study, the team harboured no illusions regarding the ultimate completeness of issues to be identified, related evidence to be gathered, and analyses to be conducted. it was acknowledged that the study offers a mere starting point for ongoing monitoring of the state of is in australia. regardless, efforts were made to achieve some level of representativeness of the evidence and perspectives reported. key mechanisms were: (1) selection of the study team; (2) review of draft state reports by interviewees; (3) review of state reports by within-state experts; and (4) global review of the draft special issue by appropriate experts. selection of the study team in establishing the study team, australia-wide representation was sought. this suggested state-based case reports. senior and long-standing is academics were 111 australasian journal of information systems volume 14 number 1 nov 2006 approached, in most cases those first contacted welcoming involvement but with some changes to the composition of the team over the course of the study. review of draft state reports by interviewees all interviewees received an early draft of the state report in which their views were recounted. feedback was channelled through the project manager back to the state teams, and changes implemented by the state teams. within-state reviewers in addition to careful review by state team members, interviewees, the project leader, and the project manager, each state report was given further local exposure in draft form prior to wider circulation to cross-case reviewers. selected ‘local experts’ were sent a copy of the draft report for review, aims being to: • minimise potential adverse reaction from perceived misrepresentation • try to ensure the report is as representative of the state as possible • enrich the report with further insights • ensure that the process of peer-review results in papers of strong academic standard for publication in the special edition. global reviewers two senior, and internationally active16 and notable is academics, iris vessey and bernard glasson, were sent a draft of the complete special issue and were requested to review the material and respond to a brief set of questions pertaining to the historical evolution of is-in-oz, the current state of is-in-oz and the possible future of is-in-oz. aims of this survey included: (1) gather further insights, (2) account for diverse perspectives, (3) keep the study team accurate and seek balance (solicit third-party reactions to our areas of emphasis and our interpretations), and (4) maximise rigour. apologies for omissions or oversights though extensive measures have been pursued to ensure representative input to the special issue and a balanced report, resource and time limitations have constrained what is possible. while such a report will unavoidably reflect certain emphases and biases and choices made at all stages of its production, the team nonetheless apologises for any omissions or oversights. given the desire that this study be replicated in future, feedback on any such omissions or oversights is all the more welcome. methodological action research the overall study effort was compounded substantially by the intention to document the approach for repeatability. mats alvesson uses the term ‘reflexive methodology,’ referring to an evolutionary approach that aims to maximise the quality of study results. we prefer the term ‘methodological action research’ (mar) whereby, in addition to results in relation to research questions posed, the study seeks generalisable contributions to knowledge as regards the research process. thus, we sought to establish a highly systematised and proceduralised approach, readily extended and repeatable across other countries and regions (and publication collections e.g. conferences or journals – see acis archival analysis in figure 2). note that there is a dotted arrow pointing up from the mar oval in figure 2, to a similar study at the is-in-pa level of that figure. this reflects how mar learnings on the is-in-oz study have informed the parent study by evolving, recording and piloting various methods, mechanisms, concepts and interpretations. it is noted that several of the is-in-oz research instruments and approaches needed only marginal adaptation for the purposes of the is-in-pa study. 112 australasian journal of information systems volume 14 number 1 nov 2006 study outcomes this section summarises outcomes of the 12 sub-studies. the information systems discipline in australian universities: a contextual framework the contextual framework introduces the is-in-oz study and the special issue, including background on study rationale and motivation, a summary of the study aims and conceptual design, and main outcomes. contributions include a cogent synthesis of the literature on past studies of the information systems academic discipline. the approach described offers valuable methodological guidance for the replication and extension of the overall study approach within australia, the region and to other world regions, and represents one of few attempts to methodically study the evolution of a discipline. with 18 core teams members in 12 sub-study teams, and in excess of 30 interviewees and a similar number of reviewers across australia, the study too serves as a useful example of large-scale research project management. most importantly, the contextual framework offers a meta-view of the underpinning sub-studies. key aspects of the history of the information systems discipline in australia clarke (2006), referring to 3 main eras, offers cogent discussion on the evolution of the is discipline in australia, suggesting that it did not follow on from development overseas, but rather emerged in parallel. during the emergence era (up to 1965) a highly consequential federal government initiative was the programmer in training (pit) scheme (from 1963), the syllabus of which emphasized systems analysis and design, and produced 100’s of ultimately influential senior managers in both public and private sector. early in the establishment era (1965-1973), departments were created in ‘colleges of advanced education (caes) to assist practice with the application (particularly application development) of computers in business and government. in parallel, topics about ‘how to apply the technology’ began to emerge in university accounting departments, these being relatively more concerned with the ‘information’ than the system. the transfer of pit to caes toward the end of this era heralded formalized is education, which expanded to institutes of technology. much of this early effort was service teaching, with is in universities migrating forward from “undergraduate service topics and units to sub-majors, majors and only later postgraduate teaching and research” (clarke, 2006). at the outset of the consolidation era (1974-1990), “unsw [university of new south wales] appointed the first professor of is, cyril brookes, and formed the first university is department … in 1978 the first australian was awarded a phd in is (ron weber at minnesota under gordon davis’ supervision)” (clarke, 2006). ron weber was the 2nd professor of is (in 1981) with several further such appointments in the late 80s and about 20 in the 1990s resulting in over 30 is professors by 2005. early published curricula from the uk and us was too comprehensive and either too computer science or business oriented for the mostly is service subjects being taught; ‘local’ is curriculum thus sometimes became almost insular. long awaited recognition of is within the australian computer society accreditation guidelines (clarke and lo, 1989) proved watershed, their core body of knowledge in 1990 evidencing 50% is content. “through the 1970s and the 1980s the vigour of the discipline in the usa resulted in that country establishing leadership in, and for many years even downright dominance over, the is discipline” (clarke, 2006). philosophies and methods of research were debated through this period, with some acceptance of pluralism. clarke, in maynard (1992), depicts is as occupying “space between the 113 australasian journal of information systems volume 14 number 1 nov 2006 technical and business disciplines, encompassing a range of applied and instrumentalist topics, and interacting closely with many other disciplines and sub-disciplines” (clarke, 2006). clarke lists 7 main clusters of topics or themes in australian is since 1965 (with much crossfertilization among the themes): i) technology as enabler and driver, ii) applications of technology, iii) data management, iv) organisations, v) systems thinking, vi) business school thinking, and vii) information management. the apparent diversity of both curricula and research domains being perhaps more varied even, than that of europe or america. characterising information systems in australia: a theoretical framework ridley’s (2006) theoretical explorations are based in a review of the literature on the development of disciplines, ultimately deriving a theoretical framework for the broader study. the framework considers the current and past state of is in australasian universities from the perspective of the development of a discipline. the components of the framework were derived and validated through a thematic analysis of both the is and non-is literature. other aspects of this study were described earlier in discussion on ‘relevant theory’. the 7 “state” case studies the next 7 studies are “state” case studies, for: queensland, act, nsw, south australia, tasmania, victoria, and west australia. case study: the state of information systems in queensland universities data was gathered from all 9 universities in queensland (including australian catholic university, which has common approaches in all its australian universities), as all teach is on at least one campus. the queensland case study (smyth gable, 2006) was commenced and completed first, and is in closest accord with the protocol, it being intended as an exemplar for the other teams. while the study revealed little evidence of a distinctive queensland-flavour of information systems it did find that the state of information systems in queensland reflects the highly decentralised nature of the state. relative to its population, queensland has a large number of universities, each of which is engaged in information systems teaching and research. the study reveals little evidence of a distinctive queensland-flavour of information systems. rather, there is a diversity of curriculum approaches and an equally broad range of research topics and research methods. two of the state’s regional universities are notable for the relative strength of their is presence, in terms of number of is staff, number of is students and range of campuses across which is is taught. the breadth of topics and approaches to is in queensland is evidenced by the existence of separate, competing is groups in each of two of the largest universities; in each case, one of the is groups is highly technical in orientation while the other is business oriented. across the eight queensland universities there is wide variability in terms of the administrative location of the information systems academic staff in the university structure. the study assesses the state of is in queensland universities in relation to criteria indicative of the maturity of a discipline. measured against these criteria, information systems in queensland universities cannot be considered a mature, distinct academic discipline. case study: the state of information systems in act universities (gregor lewis mcdonald, 2006) the act case study was prepared by three leaders in is from the territory’s three universities. the report depicts a vibrant is group in each university but voices concerns about the disparate administrative locations of the is academics and the relative lack of strong identity for is in the territory’s universities. is is prominent at all three universities, each having a distinctive background that reflects its position in canberra, australia’s seat of federal government. australian 114 australasian journal of information systems volume 14 number 1 nov 2006 defence force academy (university of new south wales) is essentially a private university for the australian defence organization; australian national university was set up to be a national research institution; and the university of canberra group for many years focused on meeting the training needs for computing professionals for the federal government. despite these distinguishing characteristics, the subject matter taught and researched in the three groups has much in common and “each group regards itself as ‘vibrant’ and happy with what it does”. nonetheless, a low degree of professionalisation is reported relative to longer-standing disciplines, it being suggested that this is to some extent due to there existing “a disjunct between what is taught as core knowledge and what is taught as research methods, a lack of social prestige, and a lack of acceptance as a discipline with a unique symbol system” (gregor lewis mcdonald, 2006). the state of is in australian universities: new south wales report the new south wales report (underwood jordan, 2006) identifies 12 separate is academic groups across 9 universities in the state. unlike the other state reports, the authors choose to address only a subset of the nsw universities; those they consider more prominent in is17. they observe that students undertaking strongly-identified is undergraduate degrees, can be found at few universities, with most offering a variety of majors within other programs. the size of the is presence would thus appear to depend upon the university’s enrolment in the core programs that offer the majors and the extent of compulsory is subjects in there programs. large accounting programs mean that many students will need to do is, thereby requiring larger is staff numbers. growing enrolments in commerce over 2 decades has advantaged is units and staff in that faculty. nonetheless, is would appear not to have a common home, but rather resides in a variety of locations, especially science and computer science. most is groups remain as ad hoc or informal groups within larger departments. while the universities reported by underwood and jordan (2006) display some structural recognition of is as a separate field, it was widely held by their interviewees that the distinctiveness of is was not well known in the wider university communities. indeed the interviewees themselves appeared to hold diverse definitions of is. research activity tends to be fragmented and diverse. small research groups, especially of doctoral students, have tended to exist without undergraduate programs to build the staff numbers to critical mass. a case study: the state of information systems in australian universities south australia report the south australia report (koronios and swatman, 2006) emphasizes the lead role of the university of south australia. data gathered suggest that the state of south australia’s is offerings were heavily influenced during the 1990s by the soft systems and critical systems approaches to the discipline, a situation which began to change at the turn of the century; and that the curriculum depends more heavily on industrial than political factors. though they report several substantive research centres or labs, it is also noted that there is little local funding for research, and while the three universities work fairly well together and have created some highly successful technical joint ventures, with is playing such a minor role in the state, obtaining funding for any research activities is extremely difficult. a case study: the state of information systems in australian universities – tasmania report the tasmania case study (ridley 2006) is distinctive in several respects. firstly, it reports on just one university. also, the data gathering approach applied is somewhat different. where reports in the other states used interviews from one or two senior academics in each university, in tasmania it was possible to draw on data from a wide range of academics, both current and former staff members from university of tasmania. the author of the tasmania report is also the author of the framework developed for the overall case study protocol. the tasmania case study findings suggest that an inverse relationship exists between the impact of local factors and the degree of 115 australasian journal of information systems volume 14 number 1 nov 2006 professionalism in this is setting. a surprising finding was that the relationship found varied for research and teaching issues. a review of information systems programs in universities in victoria victoria, more than any other state, was heavily impacted by the dawkins reforms to australian tertiary education (pollard chan 2006). amalgamations, mergers and takeovers were widespread in victoria, sometimes bringing together strong is groups with different cultures and different aspirations. though most interviewees expressed the view that local industry had had negligible influence on curriculum, universities in victoria universally appear to be seeking increased collaboration with the local community and industry as part of their strategic direction. distinctive themes taught within the many programs identified, varied considerably. despite diverse topics of research being pursued, is research output in universities in victoria was perceived as lesser than in other departments. efforts are however underway to bolster research output. interestingly, the victoria report states “the mode of is research in universities in victoria is predominantly interpretive. only 1 university reported using ‘multi-method, with an emphasis on quantitative techniques’.” although research is considered a high priority at almost all universities in victoria, available funding appears to have a negative correlation with the avowed importance of research. perceptions of “very little funding”, “dwindling funding” and “having trouble attracting arc and other external funding “ were evident in the data. information systems in west australian universities the western australia report (burn standing, 2006) identifies a degree of isolation, attributable to the size of the state and relative remoteness from the universities elsewhere in australia. the report suggests how these factors have impinged on development of is in the state’s universities and how response to local contingencies inhibits the perception of is as a mature discipline. research focus within the four universities is very different and this may be one of the reasons that all interviewees identified the low level of collaboration between wa universities. interviewees stressed the real need for is leadership and active involvement in is research by the professoriate. is groups without a professor tended to have significantly lower profile in their home university. using contradictions to appreciate the history of i.s. education in south australia this paper presents metcalfe’s (2006) interpretation of the history of information systems (is) education in south australia (sa). the stance used to think about the history was that of seeking the contradictions, underlying tensions, which worked over time to create the present. the paper argues that is education in sa was influenced significantly by maintaining a "how to" view of teaching is which failed to distinguish itself from the engineering worldview. after explaining the underlying tension stance this paper uses extracts from a long semi-structured interview with two seminal is educators to support this argument. the 2005 survey of information systems research in australia as part of a study to investigate the state of information systems research in australia, a survey of the heads of all is discipline groups in australian universities was conducted in mid 2005 (pervan shanks, 2006). the study revealed a wide range of topics researched (with rapid growth in electronic commerce and knowledge management), a range of foci, a balance between positivist and interpretivist research; survey was the most frequently used research method, and most research was directed at informing is professionals. 116 australasian journal of information systems volume 14 number 1 nov 2006 conclusion in conclusion, it is reiterated that this study is in many ways a pilot; the first phase of an ongoing, longitudinal and evolving analysis. further, specific evidence collections in-progress are described below. further, more in-depth analysis of that additional evidence in combination with evidence already gathered in this study, is planned. as reflected in extant literature, due to its youth, is has understandably been soul-searching for the past two decades. information systems as a separate academic discipline is relatively young and yet maturing. the extensive variation observed across the state case studies in both curriculum and research foci as well as placement and level of is in universities in australia further attests to it’s formative stage of evolution. following are described: (1) the communication of study results, (2) further planned or in-progress research, and (3) study limitations. communicating study results the main vehicle for communicating study results is the special issue of the australasian journal of information systems within which this paper appears. methodological learnings from the is-in-oz study have been mostly held in abeyance until the special issue of cais wherein the combined learnings from the is-in-oz and is-in-pa studies are reported. a consolidation of the australian state case studies appears in the special issue of cais as the australian state case study in the pacific asia region multiple case study. further research as per figure 2, and represented therein by unshaded ovals, three sub-studies that complement the results reported in this special issue have been commenced but are not yet completed. they are: (1) methodological action research, (2) the is research issues survey, and (3) the acis archival analysis. methodological action research the ‘reflexive methodology’ employed on this study, and the concomitant emphasis on repeatability, was described earlier. this effort is ongoing, overlapping substantially with the parent is-in-pa study. detailed methodological findings, including template instruments, will be reported in is-in-pa special issue of cais. the is research issues survey in march of 2005 a global survey of issues facing is researchers yielded over 800 responses. primarily due to resource and time constraints, this data, though cleaned, has yet to be fully analysed and interpreted. this study is progressing and results are expected to be available in early 2007, though outside the timeframe of the ajis special issue. australasian conference on information systems (acis) archival analysis in december 2004, at a meeting of the is-in-oz study team in hobart, in conjunction with acis’04, enthusiastic discussion ensued on the possible value to the is-in-oz study from a careful archival analysis of acis proceedings across its existence. a key assumption here was that the research reported at acis is representative of is research in australia. a proposal was submitted to the australian computer society (acs), information systems board, requesting seed funding for this activity. in may 2005 the funding was approved. although the acis archival analysis is not complete as of this writing (and thus is depicted as an unshaded oval in figure 2), it is worthy of note here as a 117 australasian journal of information systems volume 14 number 1 nov 2006 promising future source of insight into the ongoing evolution of the is academic discipline in australia. also, preliminary work completed has yielded a useful profile of acis across its 16 year history. limitations as acknowledged at various points in this paper, the study was a learning experience, a major aim being to evolve an approach that could be repeated across time and across regions; as such, its limitations are many, several of which have been specified throughout this paper. references ackoff, r. (1971) “towards a system of systems concepts”, management science, vol 17 no 11, pp 661-671. adam, f., & fitzgerald, b. (2000) "the status of the information systems field: historical perspective and practical orientation." information research, vol 5 no 4, http://informationr.net/ir/5-4/paper81.html accessed 12 march 2006. alavi, m., & carlson, p. (1992). a review of mis research and disciplinary development. journal of management information systems, vol 8 no 4, pp 45-62. ang, a. y., & jiwahhasuchin, s. (1998) “information systems education in thailand: a comparison between the views of professionals and academics”, journal of global information management, vol 6 no 4, pp 34-42. avgerou, c., siemer, j. & bjørn-andersen, n. (1999) “the academic field of information systems in europe”, european journal of information systems, vol 8 no 2, pp 136-153. bacon, c.j., & fitzgerald, b. (2001) "research contributions: a systemic framework for the field of information systems", acm sigmis database, vol 32 no 2, pp 46-67. becher, t. (1989) academic tribes and territories, society for research into higher education & open university press, milton keynes, england. benbasat, i., & zmud, r. (2003) "the identity crisis within the is discipline: defining and communicating the discipline's core properties", mis quarterly, vol 27 no 2, pp 183-194. burn, j., standing, c., lin, c. (2006) “the australian information systems academic discipline: a case study of the state of west australia”, australasian journal of information systems, vol 13 no 3, pp.147-155. chen, w., & hirschheim, r. (2004) "a paradigmatic and methodological examination of information systems research from 1991 to 2001", information systems journal, vol 14 no 3, pp. 197-235. clarke, r. (2006) “key aspects of the history of the information systems discipline in australia”, australasian journal of information systems, vol 13 no 3, pp. 18-37. claver, e., gonzalez, r., & llopis, j. (2000) "an analysis of research in information systems (19811997)", information & management, vol 37 no 4, pp 181-195. evaristo, j.r., & karahanna, e. (1997) "is north american is research different from european is research?", the data base for advances in information systems, vol 28 no 3, pp 32 43. 118 http://informationr.net/ir/5-4/paper81.html australasian journal of information systems volume 14 number 1 nov 2006 farhoomand, a.f., & drury, d.h. (1999) "a historiographical examination of information systems", communications of the ais, vol 1 no 19, pp 1-27. gable, g. (2006) “the information systems discipline in australian universities: a contex-tual framework’, australasian journal of information systems, vol 13 no 3, pp. 1-18. gable, g.g. (2002). state of the information systems academic discipline in the pacific asia region: a survey and multiple case study. proposal to association for information systems (ais) submitted to ais council at icis 2002 in barcelona. [data file]. available from http://sky.fit.qut.edu.au/~gable/ gregor, s., lewis, e., mcdonald, c. (2006) “case study: the state of information systems in australian capital territory universities”, australasian journal of information systems, vol 13 no 3, pp. 75-90. introna, l. d. (2003) "disciplining information systems: truth and its regimes", european journal of information systems, vol 12 no 3, pp 235-240. khazanchi, d., & munkvold, b.e. (2000) "is information systems a science? an inquiry into the nature of the information systems discipline", database for advances in information systems, vol 31 no 3, 24-42. koronios, a., swatman, paula. (2006) “the australian information systems academic discipline: a case study of the state of south australia”, australasian journal of information systems, vol 13, no 3, 98-107. lo, b. w. n. (1989) a survey of information systems educational programmes in australian tertiary institutions, working paper series no. 1, department of information systems, the university of wollongong. metcalfe, m. (2006) “using contradictions to appreciate the history of is education in south australia”, australasian journal of information systems, vol 13 no 3, pp.156-167. orlikowski, w., & baroudi, j.j. (1991) "studying information technology in organizations: research approaches and assumptions", information systems research, vol 2 no 1, pp 128. palvia, p., leary, d., mao, e., pinjani, p., & salam, a.f. (2004) "research methodologies in mis: an update", communications of the ais, vol 14, pp 526-542. pervan g. & shanks g. (2004) "is research activity in australia: results of the 2004 acphis research survey" presentation to austral. conf. infor. syst., december 2004 pervan, g., shanks, g. (2006) “the 2005 survey of information systems research in australia”, australasian journal of information systems, vol 13 no 3, pp.168-174. pollard, c. & chan, e.s.k. (2006) “the australian information systems academic discipline: a case study of the state of victoria”, australasian journal of information systems, vol 13 no 3, pp. 128-146. ridley, g. (2006) “characterising information systems in australasia: a theoretical framework”, australasian journal of information systems, vol 13 no 3, pp. 38-60. ridley, g. (2006) “the australian information systems academic discipline: a case study of the state of tasmania”, australasian journal of information systems, vol 13 no 3, pp. 108-127. sherer, s. a. (2002) "academic departments of information systems faculty in the u.s.", journal of information systems education, vol 13 no 2, pp 105-116. 119 australasian journal of information systems volume 14 number 1 nov 2006 smyth, r., gable, g. (2006) “case study: the state of information systems in queensland universities”, australasian journal of information systems, vol 13 no 3, pp. 61-74. underwood, j., jordan, e. (2006) “the australian information systems academic discipline: a case study of the state of nsw”, australasian journal of information systems, vol 13 no 3, 9197. vessey, i., ramesh, v., & glass, r.l. (2002) "research in information systems: an empirical study of diversity in the discipline and its journals", journal of management information systems, vol 19 no 2, pp 129-174. walsham, g. (1995) “interpretive case studies in is research: nature and method”, european journal of information systems, vol 4, pp 74-81. whitley, r. (1984a) “the development of management studies as a fragmented adhocracy”, social science information, vol 23 nos 4/5, pp 775–818. whitley, r. (1984b) the intellectual and social organization of the sciences, clarendon press, oxford, uk. yin, robert k. (2003) case study research: design and methods, 3rd ed., sage publications. acknowledgements this study of the information systems academic discipline in australian universities has made significant demands on queensland university of technology (qut), in terms of the time and efforts of professor gable, dr. robert smyth and ms. karen stark; the provision of the study team website; facilities for conferencing; and numerous other resources to support the project. i wish to acknowledge here my appreciation to qut for this significant provision of resources to make the study achievable. a project of this magnitude involves a large number of people. many have contributed their valuable time to the study. in addition to the core study team members, we express our sincere thanks to: (1) state report reviewers, who reviewed a draft of one (or more) of the state case reports; (2) interviewees, who gave of their time for evidence collection; and (3) special issue reviewers. we offer our special thanks to the special issue reviewers, who reviewed the complete special issue, offering ideas and advice particularly valuable and influential in the finalising of the papers. thanks also to ais for endorsing the study and for seed funding. finally, my personal thanks to bob smyth for his untiring support as project manager throughout the study duration. we could not have made it without him; his input throughout has been substantial. my sincere thanks also to karen stark, who crafted sections of the overview; whose due diligence insured maximum accuracy and completeness; and whose meticulous eye introduced a level of presentation quality otherwise not possible within project scope. 120 australasian journal of information systems volume 14 number 1 nov 2006 endnotes 1 the author acknowledges the substantial contribution to this paper and the overall study from both dr. robert smyth (project manager) and ms. karen stark, both senior research associates with qut’s it professional services research program. 2 to appear 2007 3 the panel, titled ‘integrating the global is academic community: the asia-pacific connection,’ included phillip ein-dor (then president ais), kk wei (then president-elect ais), and ryutaro manabe (pacis 2002 conference chair) and others. during that discussion, gable (2002) recorded thoughts on a possible multiple case study of the ais region3 pacific asia region (par), with the suggestion that ais might be a sponsor. early ideas were encouraged by phillip ein-dor and kk wei, who suggested submission of a formal proposal for consideration by the ais council at icis 2002 in barcelona (gable 2002). the then proposed study of ‘the state of is as an academic discipline in pacific asia’ (is-in-pa) was endorsed by ais exec in barcelona and formally approved by ais council mid-2003 to proceed on a smaller scale with ‘seed’ funding from ais; that funding received end 2004. the study received subsequent strong endorsement from rick watson and michael myers. 4 note that association for information systems (ais) the main international association of information systems academics, organizes its activities around three world regions: (1) the americas, (2) europe, africa and the middle east, and (3) pacific asia. 5 the study is currently being extended to mainland china (in progress), japan, india, malaysia, pakistan, and thailand, for which tentative state study team leaders have been identified. 6 the several articles of the special issue of ajis have been consolidated in an australian case study to appear in the special issue of cais, along with case studies of the five other states in pacific asia, and related sub-studies as reflected in figure 1. 7 felix tan, auckland university of technology, who was then the elected ais region3 council representative. 8 each of these studies is briefly described later in this paper. 9 australia is made up of 6 states and 2 territories: nsw – new south wales, sa south australia, qld – queensland, tas – tasmania, vic – victoria, wa – west australia, act – australian capital territory, nt – northern territory. note that there little identifiable, substantive is presence at any nt university as of this writing. 10 as debated extensively in 2003 and subsequently. this was considered beyond the study scope. 11 this paper presents a possible theory-lens for viewing the evolution of the is discipline. it evolved in parallel with the study, and particularly influenced the multiple-case study protocol and cross-case and meta-analyses. 12 note that the state teams varied in their reliance upon the protocol. 13 the protocol includes a long list of more specific questions, based around each of these broad questions. 121 australasian journal of information systems volume 14 number 1 nov 2006 14 this section offers a very brief overview of past studies of the is discipline. individual articles in the special issue make more specific reference to relevant literature. 15 methodological learnings from the is-in-oz and ‘parent’ is-in-pa studies are reported in detail in a special issue of communications of the ais to appear in 2007 16 though both highly active internationally throughout their careers, bernard glasson is now retired and iris vessey semi-retired. 17 some might disagree with their assessment. 122 the information systems discipline in australian universities: a contextual framework abstract introduction the ‘parent’ is in pacific asia study (is-in-pa) design the australian study (is-in-oz) design study questions the literature past studies of the is discipline relevant theory study approach overview the state case studies mechanisms to increase representativeness methodological action research study outcomes key aspects of the history of the information systems discipline in australia the 7 “state” case studies conclusion communicating study results further research limitations references acknowledgements microsoft word ajis12-1final repaired.doc ajis vol. 12, no. 1 september 2004 94 realist and postmodernist perspectives on information systems research: points of connection philip j. dobson and peter e.d. love school of management information systems edith cowan university, joondalup, western australia, australia 6027 corresponding author: email: p.dobson@ecu.edu.au abstract there has been considerable debate in the information systems (is) literature as to which research methodology is most appropriate to address is related problems. this paper contributes to that debate by contrasting the post-modernist approach with that of the critical realist. a major strand within the postmodernist perspective is concerned with the various aspects of knowing – that is, it is primarily epistemological in focus. in contrast, traditional realist approaches are more ontological in focus in their acceptance and concern for a reality external to perception. realist and postmodernist approaches are generally presented as in opposition to each other, yet as this paper will argue, a suitably aware contemporary realist approach to research can accommodate many of the proposals of the postmodernist. critical realism is presented as such a contemporary realist philosophy in that it takes account of both ontological and epistemological viewpoints. it is argued that such a philosophy has the potential to provide a basis for developing a robust methodological approach to is research. keywords: research methodology, critical realism, postmodernism, information systems introduction for several years now, there has been ubiquitous debate amongst information systems (is) academics as to which research methodology is most appropriate to research is problems (e.g., galliers(1991); orlikowski and baroudi (1991); walsham (1995); iivari et al. (1998); mingers (2001); eldabi et al.(2002)). recent debate has suggested that the complex nature of is research suggests that a hybrid approach is most preferable (lee, 1991; mingers, 2001; eldabi et al., 2002). nissen et al. (1991) point out research arguments tend to concentrate on either the nature of what is investigated (ontology) or the nature of human knowledge and understanding that can possibly be acquired through different types of research and the appropriateness of the methods of investigation (epistemology). in a practical field such as is research has tended to examine epistemological issues, often ignoring the important nature of the objects of enquiry. for example, recent calls for pluralism in is research have tended to focus on methodological matters rather than the ontological issue as to the underlying nature of the objects of enquiry (e.g. mingers and gill, 1997). pluralist approaches can be seen to be in line with the ‘post-modern condition’, which is described by jackson (1991) as thriving on “instability, disruption, disorder, contingency, paradox, and indeterminacy”, and requiring recognition of multiple interpretations of the world. in a similar vein robson (1993) suggests that if a research approach relied (solely) on a singular methodological standpoint, some unknown part(s) or aspect(s) of the results obtained would be attributable to the restrictive aspect(s) of the method (not) used, in obtaining such result(s). robson (1993) states that “we can never obtain results for which some method has not been used to collect them, [so] the only feasible strategy [therefore] is to use a variety of methods” (p.230). yet methodology cannot be considered separately from underlying conceptions of what makes up the world. for example, archer (1995, p. 28) has suggested that “once social analysts have been assured that ontology and methodology are separate issues, why should they not conclude that they can merely select the methodology which pragmatically seems most useful to them (thus sliding rapidly into instrumentalism), because if ontology is a separate concern, then it need to be no concern of theirs”. it is impossible to formally describe aspects of the world without such being grounded on a particular implicit or explicit, conscious or subconscious philosophy. within such a practically focused area such as is design and implementation philosophical and associated ontological issues are sometimes relegated to relatively minor role in the research process. ormerod (1997, p. 421) presents such a view when he emphasizes the pre-eminent importance of practical issues: ajis vol. 12, no. 1 september 2004 95 any choice mechanism should, in my view, be rooted in practical requirements rather than in theoretical considerations with which very few practitioners could feel at home. in simple terms the approach (methods and their theories) chosen must support a process of intervention (practice) in a particular context to achieve the desired outcome …if a philosophical justification for action is needed it can be found in american pragmatism, bhaskar's transcendental realism, feyerabend’s iconoclastic critique of philosophical positions or postmodernism. such an argument presents a not uncommon ‘consultancy’ type approach to is research with practicality dominating the researcher perspective. ormerod (1997) further suggests that: “the choice of methods will depend on the organizational context, the degree of participation envisaged the consultant’s skill and on the nature of the outcome required” (p. 415). ignoring the fact that perceptions as to organizational context and practical outcomes necessarily reflect an implicit philosophical position, such a view also neglects the useful role that philosophy can explicitly play as underlabourer to research and practice, underlabourer being taken from locke (1894) as “clearing the ground a little...removing some of the rubbish that lies in the way of knowledge” (p.14). in contrast to post modernist approaches, which tend to neglect philosophy (see kilduff and mehra, 1997), the critical realist argues for an intimate and important role for philosophy within research. against this contextual backdrop, this paper suggests that philosophical considerations do have an important role to play in is research and practice. we will also contrast the differing perspectives offered by the post-modernist and the realist and suggest that a suitably aware realist approach can provide a necessary recognition of both epistemological and ontological issues. philosophical challenges to positivism bhaskar (1989) places critical realism within contemporary philosophy alongside post-modernism and critical theory as a particular response to the crisis of positivism. in a similar fashion orlikowski and baroudi (1991) classify is research traditions as basically following three major philosophical approaches – positivist, interpretive and critical, the interpretive and the critical responding to shortcomings in the positivist. their neglect of contemporary realist approaches perhaps reflects the fact that a “naïve” form of realism provided the foundational philosophy for positivism. as detailed below, critical realism rejects many of the earlier proposals underlying such a naïve or traditional realism. the use of the term “critical” within “critical realism” is perhaps unfortunate in that it can be seen to align the philosophy with habermas’ critical theory. the use of the word “critical” within critical realism seems to have grown out of bhasker’s (1989) argument for a critical or careful application of the scientific method to social situations. critical theory and critical realism have a similar emancipatory focus but bhaskar would suggest that critical theory along with interpretivism suffers from a neglect of ontological issues. he argues in “philosophy and the idea of freedom” (bhaskar, 1991) that critical and interpretive approaches are too concerned with epistemological issues. he suggests that whilst the hermeneutic focus on identification and uncovering of inequitable structures is an important first step in emancipatory action, unless something is actually done with this knowledge real effective emancipation is not possible. this ontological focus is a vital part of the critical realist approach. blaikie (1993) suggests that contemporary responses to dissatisfaction with positivism include: • critical theory; • realism; • contemporary hermeneutics; • structuration theory; and • feminism. mingers (2001) suggests that the interpretive/hermeneutic and the positivist perspective have tended to dominate the study of is in recent years. both approaches provide markedly different views of the world and the means to study it. guba and lincoln (1994) state that in terms of epistemology or the relationship between the knower, would-be-knower and what can be known, positivism is dualist and objectivist in that it assumes: ajis vol. 12, no. 1 september 2004 96 • objective reality can be captured and analyzed meaningfully; • the observer can be separated from the observed; • observations and generalizations are free from situational and temporal constraints; • causality is linear and there are no causes without effects, no effects without causes; and • inquiry is value free. the interpretivist and the positivist also have fundamentally different stances on the role of experiment in research. empiricism refers to a set of philosophical beliefs formed around the idea that experience, rather than reason, is the source of robust knowledge of the world (morick, 1980). the term has also come to mean the practice of investigating the nature of the world using practical or experiential methods, rather than by applying or developing theories, or assuming guiding principles. empiricism dictates that one settles questions about the nature of human thought and action by accepting only assertions and claims that can be probed by direct observation. the interpretivist is, however, more receptive to theory and principles, not necessarily insisting that what cannot be measured should be ignored in this context. positivism places emphasis on facts as distinct from values or meanings, and the use of scientific method in which theory is deduced as a result of formulating and testing hypotheses. this approach can identify cause-and-effect through ‘the constant conjunction’ of events, resulting in what has been called the ‘covering law’ orthodoxy (popper, 1959). this orthodoxy is essentially devoted to the pursuit of explanations, which take the form of general laws. the reasoning behind interpretivism (induction) on the other hand proceeds from specific observations (data but not necessarily in quantitative form) to general principles (laws). the observations may suggest generalizations, which if repeatedly tested and confirmed can lead to discovery of a lawful relationship (babbie, 1992). according to babbie (1992) both induction and deduction are essential to the process of hypothetico-deductive scientific method. induction always involves some selection of what data to attend, implying ‘theory’, no matter how implicit. deductive inference, however, involves some intuitive aspects in that its predictions are tested inductively. deductive inference is also heavily dependent on the initial step of generating hypotheses (or propositions) from general laws or theories. hence, if the general law is false, then deduction can be logically valid but false in relation to the real world. according to popper (1959) theories can never be conclusively proven from repeated observations that ‘confirm’ them. they can however be falsified by just one instance, that is, where their predictions are not confirmed. critical realism, on the other hand, argues that within the social sciences predictive use of theory and falsification makes no sense in the open contextual systems within which social science must operate. fundamentally, the epistemological position of the interpretive perspective rejects the natural sciences as an appropriate foundation for social science inquiry. the ‘anti-naturalist’ school of thought believes that the subject matters of the social sciences are so different to that of the natural sciences that an entirely different approach to empirical work is needed. in contrast to this approach bhaskar's critical realist approach argues for critical and careful application of the methods of the sciences to the study of society – a critical naturalism. bhaskar (1989) argues that the methods of the sciences can be applied to social situations although in significantly different ways grounded in the significantly different subject matters. he sees science as an ongoing developing process of explanation and enlightenment rather than the derivation (or popperian falsification) of immutable scientific laws; scientific and social research involving the steady unearthing of the increasingly deeper structures and mechanisms which make up the objects under study. walsham (1995) discusses three different ways of viewing reality: • external realism which considers reality as existing independently of our construction of it; • internal realism which views reality as an inter-subjective construction shared between individuals; and • subjective idealism which sees reality as a personal construction of each individual in particular, walsham (1995) has suggested that interpretivist research in the is area has tended to concentrate on internal and subjective realism. ajis vol. 12, no. 1 september 2004 97 along with many realist approaches bhaskar (1978, 1979, 1986) argues for an external realism in that he suggests that there exists a reality totally independent of our representations of it; the reality and the “representation of reality" operating in different domains, roughly a transitive epistemological dimension and an intransitive ontological dimension. as lawson (1997) suggests critical realism as proposed by bhaskar (1978) is developed around a scientific realist position which asserts that “the ultimate objects of scientific investigation exist for the most part quite independent of, or at least prior to, their investigation” (p. 15). this common external realist position is however extended under critical realism in that it also presents a philosophical argument for the nature, constitution and structure of the underlying objects of enquiry. such realism is heavily concerned with ontology or metaphysics, that is, the nature of being and existence. bhaskar (1978) develops his realist philosophy of science founded on the question “under what conditions is science possible?”. this philosophical questioning concludes a, so-called, depth realism that proposes that “the world is composed not only of events and our experience or impression of them, but also of (irreducible) structures and mechanisms, powers and tendencies, etc. that, although not directly observable, nevertheless underlie actual events that we experience and govern or produce them” (lawson, 1997, p. 8). this proposal for a depth realism is similarly extended to the social arena in bhaskar’s (1979) text entitled “the possibility of naturalism” where he addresses the question “under what conditions is social science possible?”. information system projects are essentially human enterprises, and cannot be understood solely in terms of technical relations among components and from a purely ‘scientific’ approach. mingers and stowell (1997) specifically note that is research must concern itself with the general evolution of human communication. information systems project management, for example, is a relatively recent research domain within the field of is. as a result, the theory within this domain is rather limited in its scope (callahan and moretton, 2001). thus, in order to extend the scope of theory within the domain of is project management (e.g., understanding the antecedents and consequences of errors in software development, managing complex systems in increasingly virtual environments and being able to better anticipate and satisfy stakeholder needs, and understand why projects fail), we proffer that is researchers should adopt approaches that ameliorate their understanding of both the ontological and epistemological aspects of is. a useful means of encouraging this focus is to adopt a philosophical position (such as critical realism) that incorporates both ontological and epistemological issues. critical realist and postmodernist challenges to empiricism postmodern thinking eschews reductionism and the fixing of meaning in favour of a style, which privileges action, movement, process and emergence (chia, 1995). in fact, post-modernists consider that there is no one theory or no one right theory and argue that all perspectives are equally acceptable. a central consequence of such post-modernist argument is varying degrees of relativism as is demonstrated by kuhn (1970) when he suggests that “though the world does not change with a change of paradigm, the scientist afterward works in a different world” (p. 121). for the critical realist, kuhn’s quotation is an example of what is termed the epistemic fallacy. the critical realist sees kuhn’s statement as incorrect in that it confuses two different worlds – an intransitive world that is natural and (relatively) unchanging and a transitive world that is social and historical. the critical realist would suggest the re-phrasing of kuhn’s statement in an unremarkable and nonparadoxical manner: “though the (natural (or object)) world does not change with a change of paradigm, the scientist afterward works in a different (social (or cognitive)) world” (bhaskar 1991, p. 10). such a re-phrasing acknowledges the presence of a deep or real element to the world along with an empirical, perception based world it avoids the so-called epistemic fallacy through its recognition of the difference between statements about the knowledge domain and statements about the ontological domain. bhaskar (1991) argues that this is a fundamental error of much of ajis vol. 12, no. 1 september 2004 98 postmodernist work which assumes that statements about being (ontological statements) can be analyzed in terms of statements about knowledge of that being (epistemological statements). realism is often taken as implying a privileged access to truth in that it is presented as proposing an objective reality that can be found if we dig deep enough. however given that perhaps the defining feature of realism is its acknowledgement of a world independent of our knowledge of it, this presumption must be inaccurate, since, as sayer (2000) has argued, the division of the world into an intransitive ontological domain and a transitive knowledge focused domain must accept the fallibility of knowledge as an enduring feature of the world as we know it. despite representations to the contrary (for example wilson, 1999) the contemporary realist acknowledges that the world can only be known under particular descriptions and within available discourses but this, however, is not taken to imply that we cannot judge between different descriptions or explanations. critical realism recognizes that knowledge gained through realist examination will typically be provisional, fallibist, incomplete and extendable. the post-modernist’s argument presents realism as believing an objective reality can somehow be understood and accounted for, which thereby enables us to control it (patel, 2001). for the realist, postmodernism is seen as epistemology suggesting that the world is constituted by our shared language and that we can only ‘know’ the world through the particular forms of discourse that our language(s) can create. clegg (1990) identified seven organisational dimensions that distinguish modernity and post-modernity (table 1). clegg’s (1990) development of these dimensions derives from recognition that by the 1980s, organizations were responding to the decline of fordism and moving toward internationalization, de-industrialization and changing centre-periphery relations. clegg (1990) also suggested that the contemporary japanese organization has become a ‘beacon’ of post-modernity, in contrast to the typification of highly differentiated bureaucratic organizations (characterized on the left hand side of table 1). similarly, alvesson and deetz (1996) identified seven themes within postmodernism but placed emphasis on epistemology. these themes were, the centrality of discourse; fragmented identities; the critique of the philosophy of presence; the loss of foundations and master narratives; the knowledge/power connection; hyper-reality; and research as resistance and indeterminacy. alvesson and deetz (1996) stated “the point of social science is not to get it right but to challenge guiding assumptions, fixed meanings and relations, and to re-open the formative capacity of human beings in relation to others and the world” (p.211). in line with our argument such post-modernist discussion tends to concentrate more on epistemological issues (how we come to know) and neglects ontological issues related to the ultimate constituents of the world. table 1. organisational dimensions of modernity/postmodernity (clegg, 1990) organizational dimension modernity postmodernity 1 mission goals, strategies and main functions specialization diffusion 2 functional alignments bureaucracy hierarchy democracy market 3 co-ordination and control dis-empowerment laissez-faire empowerment industry policy 4 accountability and role relationships extra-organizational inflexible intra-organizational skill formation, flexible 5 planning and communication short-term techniques long-term techniques 6 relation and performance reward individualized collective 7 leadership mistrust trust ajis vol. 12, no. 1 september 2004 99 postmodern approaches have begun to enter mainstream is research and have important consequences for the conduct, presentation and assessment of theory (ciborra, 1997; patel, 2001). hassard (1993) drew on the work of lyotard to suggest that the method of epistemological postmodernity is serious play rather than empiricism. hassard (1993) stated that “in doing science, we only enter into a number of games with our colleagues. we are in fact involved in a form of ‘serious play’, which sees us intervene in a variety of language-games, make moves in a number of debates or discussions, and seek to oppose the moves and positions of other players while advancing our positions” (p. 10). in line with the increased notions of play and interplay, alvesson and deetz (1996) argue that postmodern research “aims at resistance and indeterminacy where irony and play are preferred to rationality, predictability and order” (p. 205). kilduff and mehra (1997: p. 458) similarly suggest that postmodern research can be provocative research, and that researchers “can mix and match various styles in order to contrast with tradition” (p. 458). they point to the importance of irony, arguing for a post-modernity that is similarly informed by, and yet ambivalent toward classic statements and techniques of the field in question. for the critical realist such a pluralist approach is acceptable providing that the researcher appreciates they are operating within the transitive epistemological dimension. in contrast to positivism, post-modernism no longer argues for the separation of the researcher and subject. indeed post-modernity is directed against a picture theory of language in which physical properties of the world are considered fixed, while language can be adjusted to meet the needs of description (hassard, 1993). rather than attempting “to capture the impression of a pregiveness in the object of analysis” (chia, 1995: p. 589), post-modernity attempts resuscitation (clegg and hardy, 1996). furthermore, if the living subject is no longer an object, its representation, capture and transmission can become more difficult (stablein, 1996). guba and lincoln (1994) distinguish positivist, post-positivist and post-modernist inquiry, grouping post-modernity and poststructuralism within the domain of critical theory. the nature of reality, or ontology, assumed by positivism is seen to be what is now considered to be naïve realism, where reality is assumed to exist driven by immutable natural laws and mechanisms (alvesson and deetz, 1996). in post-modernity kilduff and mehra (1997) stated that “style matters”, as researchers eschew passive objectivity for an active authorial voice, and carefully craft an aesthetically pleasing narrative. according to alvesson and deetz (1996) however, style is not all that matters as indicated by postmodernism’s strong critique of empiricism and its emphasis on data as constructions open up to a multitude of interpretations. they suggest that this does not mean that reflective empirical work is not worth doing. similarly, chia (1995) stated that postmodernism thinking is not anti-empirical, but rather, it is ultra-empirical; dealing as it does with the concrete logic of order and organization of phenomena. such an approach therefore needs by default to embrace many, if not all of the principal research paradigms referred to earlier. points of connection? kilduff and mehra (1997) argue that there are two styles of postmodernism: the skeptical and the affirmative. “from the skeptical perspective all interpretations of phenomena are equally valid, and the world is so complicated that concepts such as prediction and causality are irrelevant. everything is related to everything else so the search for causes or origins must be discontinued.” (p. 455). in contrast the affirmative postmodernist “retains the possibility of making discriminations among competing interpretations.” the affirmative postmodernist “would underscore novelty and reflexivity as it looks at the richness of difference and concentrates on the unusual, the singular and the original” (rosenau (1992) cited in kilduff and mehra (1997), p. 456). stones (1996) from a realist perspective, argues against the defeatism of the skeptical postmodernist. he celebrates the post-modernist emphasis on plurality and diversity and recognizes the enormous difficulty in providing accurate accounts of the social world. he feels that "yes, the world is that complex; yes, it is often very hard to get at; yes, sociologists and other social scientists often claim an authority they have no right to; and, no, the defeatist postmodernists are not right to imply that the only alternative to a complete and total knowledge of a very complex world is a retreat into fiction”. the critical realist is optimistic concerning knowledge claims in that they suggest that ajis vol. 12, no. 1 september 2004 100 sensible claims can be made, even though historically dependent and socially situated. knowledge is cumulative and can progressively add to a body of knowledge. contemporary realism can be seen to be “ontologically bold and epistemologically cautious” in that the realist appreciation of the richness and complexity of social life has not been matched by an increased sophistication in the necessary means to acquire knowledge of that complexity. realist methodology has much to learn from post-modernist thought in that there needs to be a continual commitment to caution, skepticism and reflexivity. avgerou (2001) similarly argues for the crucial importance of is practitioners considering the social context in which is development takes place and also the broader organizational processes under which it operates. he suggests that such contextualized examination has been limited in the past and most of the knowledge developed and taught within the is field promotes normative professional practice which has been founded upon a limited understanding of what exactly occurs in the business domain. as a result, there is a need to develop appropriate analytical knowledge to equip is professionals with capabilities to pursue contextualist analyses. in addressing this concern avgerou (2001) has suggested three principles to be followed in order to address the contextual processes involved in is implementation. first, technological innovation should be considered in relation to socio-organizational change. second, analysis should consider the national and international context together with the local organizational and third it should also consider both the technical/rational decisions and actions involved in the innovation process along with the important cultural, social and cognitive forces of such a process. such requirements require recognition of what mingers (2001) has termed the social, personal and material world. postmodernist and realist methodologies that recognize each of these different dimensions of reality are necessary for is research. thus we suggest that post-modernist approaches should not be dismissed out-of-hand by realist researchers and similarly nor should contemporary realist approaches be ignored by the post-modernist. each have valuable insights to offer on the complex social reality in which is research must operate. references alvesson, m., and deetz, s. (1996), critical theory and postmodernism approaches to organizational studies. in clegg, s.r. hardy, c and nord, w.r. (eds.) handbook of organization studies, sage, london, pp.191-217. archer, m. (1995), realist social theory: the morphogenetic approach, cambridge university press, cambridge. avgerou, c. (2001), the significance of context in information systems and organizational change, information systems journal, vol: 11, no: 1, pp: 43-63. babbie, e. (1992), the practice of social research. wadsworth, belmont california. bhaskar, r. (1991), philosophy and the idea of freedom. oxford: blackwell. bhaskar, r. (1989), reclaiming reality: a critical introduction to contemporary philosophy. verso, london. bhaskar, r. (1986), scientific realism and human emancipation, verso, london. bhaskar, r. (1978), a realist theory of science, harvester press, sussex. bhaskar, r. (1979), the possibility of naturalism, harvester wheatsheaf, hemel hempstead. blaikie, n. (1993), approaches to social enquiry, polity press, cambridge, uk. callahan, j., and moretton, b. (2001), reducing software development time. international journal of project management, 19(1), pp.59-70 chia, r. (1995), from modern to postmodern organizational analysis: thinking beyond the unthinkable. in d. grant and c. oswick (eds.), metaphor and organizations, sage, london, pp.127-145. ciborra, c.u.a. (1997), theory of information systems based on improvisation. in currie, w.l., and galliers, b. (eds.) rethinking management information systems. oxford university press, pp.136-156. clegg, s.r. (1990), modern organizations: organization studies in a postmodern world. sage, london. ajis vol. 12, no. 1 september 2004 101 clegg, s.r. and hardy, c. (1996), organizations, organizations and organizing. in s.r. clegg, c. hardy and w.r. nord (eds.) handbook of organization studies, sage, london, pp.1-28 eldabi, t., irani, z, love, p.e.d.., and paul, r.j. (2002), quantitative and qualitative methods in simulation modeling. management decision 40(1), pp.64-73. galliers, r. (1991), choosing appropriate information systems research approaches. a revised taxonomy. in galliers, r. (ed.) information systems research: issues, methods, and practical guidelines, blackwell, oxford, pp.144-162. guba, e., and lincoln, y.s. (1994), competing paradigms in qualitative research. in denzin, n.k and lincoln, y.s (eds.). handbook of qualitative research, sage, thousand oaks, pp.105117. hassard, j. (1993), sociology and organization theory: positivism paradigms and postmodernity. cambridge university press, cambridge, uk iivari, j, hirschheim, r. klein, h.k. (1998), a paradigmatic analysis contrasting information systems development approaches and methodologies, information systems research, vol: 9, iss: 2, pp: 164-193. kilduff, m., and mehra, a. (1997), postmodernism and organizational research. academy of management review, vol: 22, pp.453-481. kuhn, t. (1970), the structure of scientific revolutions, second edition, the university of chicago press, chicago. jackson, m. c. (1991), systems methodology for the management sciences, plenum press, new york lawson, t. (1997), economics and reality, routledge, london. lee, a. s., (1991), integrating positivist and interpretive approaches to organizational research, organization science, 2: 342-365 locke, j. (1894), an essay concerning human understanding, a.c. fraser (ed.): vol. 1, clarendon, oxford. mingers, j. and gill, a. (1997) (eds), multi-methodology: the theory and practice of combining management science methodologies, john wiley & sons, chichester mingers, j. and stowell, f. (1997), information systems: an emerging discipline. mcgraw-hill, london, uk mingers, j. (2001), combining is research methods: towards a pluralist methodology. information systems research, 12(3), pp.240-259. morick, h. (ed.) (1980), challenges to empiricism. methuen, london nissen, h.-e., klein, h.k. and hirschheim, r. (1991) (eds), the information systems research arena of the 1990s: challenges, perceptions and alternative approaches, amsterdam: north holland. orlikowski, w., and baroudi, j. (1991), studying information technology in organizations. information systems research, 2(1), pp.143-169. ormerod, r.j. (1997), the design of organizational intervention: choosing the approach, omega, 25(4), pp.415-435. patel, n. (2001), evaluating evolutionary information systems. a post modernist perspective. in van grembergen, w. (ed.), information technology evaluation, methods and management, idea group publishing, london, uk popper, k. (1959), the logic of scientific discovery. hutchinson, london. popper, k. (1976), unended quest. fontana/collins, glasgow. robson, c. (1993), real world research: a resource for social scientists and practitionerresearchers. blackwell publishers, oxford. rosenau, p.m. (1992), postmodernism and the social sciences: insights, inroads and intrusions, princeton, nj: princeton university press sayer, a. (2000), realism and social science, sage. stablein, r. (1996), data in organisation studies. in s.r. clegg, c. hardy and w.r. nord (eds.) handbook of organization studies, sage, london, pp.509-525. stones, r. (1996), sociological reasoning: towards a past-modern sociology, macmillian. ajis vol. 12, no. 1 september 2004 102 walsham, g. (1995), the emergence of interpretivism in is research. information systems research, 6(4), pp.376-394. wilson, f. (1999), flogging a dead horse: the implications of epistemological relativism within information systems methodological practice, european journal of information systems, 8, 161-169. ajis vol 9 no. 1 september 2001 user satisfaction and system success: considering the development team julie fisher school of information, management and systems monash university melbourne australia julie.fishci@sime.monash.cdu.au abstract two key indicators of system success are that the software performs as intended and the users are satisfied with the system. frequently, however, the major and overwhelming focus of the development team is on building and refining the technical side of the system. the result is that many systems are delivered where the users are not satisfied. this paper explores the skills, other than technical skills, that are needed on development teams to produce systems that are successful from a user's perspective. keywords: design, human factors, information systems development, user expectations, system success introduction early computer systems were built mainly for scientific purposes and often built by those who would ultimately use them. there was little need to explain how the systems worked because those who used them had also programmed them (avison and fitzgerald 1995,17). grudin (1991) notes that as the costs of computers fell and user numbers increased, expectations of usability also grew. he argues that the users needs are not met because the information about the users' needs takes time to get through to the developers (grudin 1991, 61). with the growth of e-commerce systems comes a need for development teams to pay more attention to the usability aspects of these systems. the audience that may use an e-commerce system is frequently unknown. further, e-commerce systems must cater for people who often have little knowledge or interest in the technology. system development teams must deliver systems that are innately intuitive. this paper will argue that there are two sides to producing an effective information system, the technical /functional side and the human side. frequently systems development teams are lacking in people with skills that focus on the human factors aspects of systems and therefore the systems produced are not successful because the users are not satisfied with them. systems development methodologies avison and fitzgerald (1995) note that although there are many hundreds of systems development methodologies, many are very similar. only a small number of basic and well understood methodologies are widely used in the it development community. the most common methodology is the traditional approach (also known as the waterfall method or systems development life cycle (sdlc)). many of the newer development methods are a variation on the traditional approach (avison and fitzgerald 1995; rouse, watson et al. 1995). its acceptance as a widely employed methodology is reflected in the number of computing text books referring to it and the number of courses that teach the traditional methodology (nicholas, 1990; gibson and huches, 1994). figure 1 is a diagrammatic representation of the traditional approach to systems development and includes the skilled roles normally involved at the different stages. the sdlc, because it is still the development methodology most widely used, has been selected to illustrate how and where current practice in developing systems is not meeting the needs of the users. the most commonly recognised roles in the development process include the client or project sponsor, system analyst, managers, developers (or programmers), and users (nicholas 1990; avison and fitzgerald 1995). skills each member of a development team brings to the process a skill set relevant to the activity or role they will perform. table 1 describes the key skills or knowledge we would expect each of the team members normally associated with systems development, to possess. 21 ajis vol 9 no. 1 september 2001 involves • client • system analyst feasibility study client system analyst involves • system analyst • managers • users involves • system analyst • managers • users investigation determine system requirements interview users and managers establish expectations of new system system analysis understand old system and problems establish what new system will provide involves • system analyst • developers involves • • developers users involves • • • system analyst developers users system design determine development tool design the system implementation install and test train users write userjianuals review of system • fix errors • maintain program • add enhancements figure 1 traditional systems development life cycle team member client (system sponsor) project manager/ team leaders system analysts developers (programmers) skills /knowledge knowledge of the business and business process. understanding of what the new system should provide management (people and financial) skills verbal and interpersonal communication and listening skills technical competence communication skills analytical skills technical skills problem solving programming analytical skills technical skills system testing problem solving table 1 skills of the development team the basis of this information comes from a range of sources including (nichols 1994, 174-180; simon 1994; misic 1996; turner and lowry 1999). there are of course many other skills that are needed however this list is restricted to those skills specific to development of an information system as distinct from management or other 22 ajis vol 9 no. 1 september 2001 skills. it should be noted that the skills listed in table 1 are also the primary skills taught in australian university information systems courses and are recognised by the australian computer society (underwood and maynard 1996). system success there are numerous reasons proffered as to why systems are not successful, amongst these are: • inadequate management planning and control (mccomb and smith 1991; weinberg 1991) • poor management of time and budget (mccomb and smith 1991) • problems with implementation (myers 1993) • poor systems quality and performance (mccomb and smith 1991; ballantine, bonner et al. 1996) in a detailed assessment, delone and mclean identified six major categories for measuring system success, these are: system quality, information quality, use, user satisfaction, individual impact and organisational impact (delone and mclean 1992). this paper will focus on those less obvious factors that directly relate to the use of the system, that is: user satisfaction and individual impact of use. these factors are those that impact most on users and relate to the human factors aspects of systems. two key measures of success from the users' perspective are: • meeting user expectations. a system may be functionally correct but if it does not meet user requirements or expectations fails. grudin made an important point when he said: there is a strong consensus that intuition and indirect approaches to understanding users and their work are usually insufficient. the design principles formulated by gould and his colleagues at ibm. are: (a) focus early and continuously on users, (b) integrate consideration of all aspects of usability, (c) test versions -with users early and continuously, and (d) iterate the design. despite being widely cited, these principles are not often followed. (grudin 1991) • effective human factors elements. poor communication between users and developers often leads to deficient analysis and poor usability. the technologists are unable to communicate to the users what the system will look like and the users are unable to explain to the technologists what is they want (grudin 1991; bresko 1995). bresko (1995) cites communication problems as a major cause of system failure and argues that development methods and management techniques are the cause of many of the problems. failing to meet the needs of the users and poor communication between system developers and users typically result in the failure of the human element or human factors aspects of the system; that is, the user is unable to effectively use the system. the majority of development methodologies including the traditional method, describe in detail how and when the technological aspects of a system will be dealt with. they however, rarely mention how and when the human elements of a system are designed. avison and fitzgerald make the point that: although not simple, the technological aspects are less complex than the human aspects in an information system, because the former are predictable in nature. however, many information systems methodologies only stress the technological aspects. this may lead to a solution which is not ideal because the methodologies underestimate the importance and complexity of the human element. (avison and fitzgerald 1995, 41) this is a significant and expensive problem for the it industry. a contributing issue, is that those on the development team usually responsible for the usability aspects of systems and are not the people with the most appropriate skills. defining a successful system the users' perspective from a technologist's perspective the key elements that define a successful system would be: a system that is functionally correct, provides accurate information, operates at an optimum speed relative to the technology it is run on and meets the needs of the organisation. this is the technical and functional side. from a user's perspective however, unless the user can use the system and use it effectively and fully, the system cannot be deemed to be a success. this is the human side. if the user cannot use the system then it must be regarded as a failure. the question is then, what are the elements of an information system that make it successful from a user's perspective? the literature suggests a number of factors that contribute strongly to users rating a system as successful. the elements presented in table 2 are those that relate just to the development process prior to implementation. 23 ajis vol 9 no. 1 september 2001 factor authors / research user expectations of a system task-technology fit understanding the user's perspective. the system reflects the cognitive style and mental model of the users meeting user expectations or perceptions of the system. (hirschheim and newman 1988; lyytinen 1988; goodhue and thompson 1995) (geriach and kuo 1991; kubie, melkus et al. 1996; williams 1996) (lawrence and low 1993; szajna and scamell 1993; ballantine, bonner et al. 1996; jordan and burn 1997) system usability quality and effectiveness of the interface design quality and effectiveness of user documentation and information ease of use (doll and torkzadeh 1988; mantel and teorey 1989; kay 1990; norman 1990 b; geriach and kuo 1991; gould, ukelson et al. 1996) (doll and torkzadeh 1987; geriach and kuo 1991; torkzadeh and doll 1993; kekre, krishnan et al. 1995; coe 1996) (doll & torkzadeh 1988; kekre et al. 1995; kubie et al. 1996;) user acceptance and ownership of a system level of user involvement in the development process participation in the development leading to greater commitment the extent of user involvement and participation the quality of user-developer communication. the quality of the system and system reliability (lawrence and low 1993; szajna and scamell 1993; goodhue and thompson 1995; robert 1997) (hirschheim & newman 1988; lawrence & low 1993; robert 1997) (montazemi 1988; barki and hartwick 1989; lawrence and low 1993) (szajna and scamell 1993; mckeen, guimaraes et al. 1994; robert 1997) (doll & torkzadeh 1988; kekre et al. 1995) table 2 user satisfaction factors figure 2 summaries these elements in diagrammatic form. understanding the users' perspective task-technology fit meeting user expectations user expectations level of user representation participation of users system quality and reliability user-developer communication user acceptance and ownership user satisfaction system usability user information ease of use interface effectiveness design figure 2 user satisfaction factors drawing on the factors identified in the literature as important measures of user satisfaction, a number of key elements that are within the control of the development team have been identified. these factors are: • the design of the user interface 24 ajis vol 9 no. 1 september 2001 • the match between the users' work-flow and the design of the system. • the provision of effective user information both online and offline • the effectiveness and usability of the system and error messages • the quality of the communication between users and developers that is, how well the developers understood the needs of the users and how well the users were able to communicate their needs to the developers. • how well the users' perspective was understood. • how effectively the users were able to participate in the development process. human factors skills following on from this: if the factors described above are important in determining the success of a system from a users' perspective, what skills are needed to design a system that has regard for these factors? some of the key skills needed for developing the human elements of a system are: • written and oral communication skills • understanding of users and how they work • organisation of information • design and layout of information • graphic design and illustration skills • interface design skills • understanding of how people work • understanding of users' work-flow. (shand 1994 b; avison and fitzgerald 1995; bresko 1995) table 1, listed the skills generally recognised as those sought by a project manager forming a team. there is therefore a mismatch between the skills required to meet the needs of the users, as described above, and the skills required for developing the technical aspects of a system. the skills we teach to information systems students generally are the traditional, technical skills as described in table 1. researchers such as bresko (1995), shand (1994) and avison and fitzgerald (1994) acknowledge a high rate of systems failure resulting from the failure of the human aspects of a system . development teams however rarely include people with the skills necessary to effectively design the human factors elements of systems. others who can contribute to the development process to produce a system that is both technically correct and satisfies the needs of the users a wider range of experts need to be included on development teams and included early. table 3 lists some of other experts who can and should make a contribution to the development of an information system. profession graphic designers technical communicators (professional writers) cognitive psychologists / human factors experts usability experts skills illustration graphic design written and oral communication skills organisation of information understanding of user perspective illustration semiotics understanding of how users work user centred design skills contributes to interface design including icons user information including messages. determining user requirements development of menus and information structures interface design error other interface design work-flow overall usability user task analysis ergonomic design table 3 other professionals and their skills whilst the role of these other experts is acknowledge in the literature as being of value to the development process they are rarely included in it teams (avison and fitzgerald, grudin (1991 a)., shand, mccomb and smith (1991)) 25 ajis vol 9 no. 1 september 2001 graphic designers graphic designers bring to the development process an understanding of what works in terms of illustrations, icons, colours and fonts for example. they also understand the layout and design of graphics and aesthetic effectiveness. selecting the most appropriate media for the task, photographs or illustrations, sound or no sound. knowing how to place graphics on the screen to be most effective. graphic designers understand topography, how text should look to be most accessible to the user. technical communicators technical communicators are professional writers. they bring to a development team specialist writing skills. writing material for users is a specialist skill particularly when it comes to online help. online help and error messages have to be brief, written in an accessible language and be effective, the se are the skills of a specialist writer. technical communicators also understand about how to layout text so that it is easy to read and attracts the eye of the reader, they know about typography that is how the written text looks on the screen. previous research by the author statistically demonstrated that: • users were more satisfied with systems where a technical communicator was involved. • users rated systems more successful when a technical communicator was involved • users used online help more frequently and rated the quality higher when it was written by a technical communicator. (fisher 1999a; fisher 1999b) human factors experts computer human interaction (chi) the field of human factors is well established and professionals in the field work on a range of every day items whose design we would not think twice about. those working in the area of human factors are involved in designing for usability. norman a leading expert in the field in 1988 wrote 'the psychology of everyday things' this book continues to be one of the leading texts in the area. the field of computer human interaction is an extension of the work of human factors people but specialises in the design of computer systems. their interest and expertise is in usability, ensuring that users are able to interact effectively with a system. discussion traditional methods for developing systems typically do not consider the involvement of other, non-technical .professionals, in designing systems. current practice has changed little and yet the number of non-technical people exposed to computer systems, is growing exponentially as a result of the world wide web. there is a need to include on to the development team, people who have the expertise and knowledge of how users work and think. figure 3 defines an information system in terms of these two identified sides, the technical /functional side and the human factors side. 26 ajis vol 9 no. 1 september 2001 skill set • analytical • programming • technical • problem solving • communication • business knowledge skill set • graphic design • communication • organisation of information • illustration • interface design • tjsahilitv involves client project manager system analyst developer/s involves graphic designer technical communicator human factors expert 6 *^m^t<^i^*£fr!m$/ echnical /functional side figure 3 skills need to produce a successful system from the user's perspective conclusion what i have attempted to do in this paper is to argue that there is a gap between the skills of the people usually involved in the development of an information system and the skills that are needed to develop an effective usable system. if we want to deliver to users the most effective system then consideration has to be given to expanding development teams to include those with skills that fill the current gaps. what we have today on development teams are people who are technically competent but have limited knowledge, understanding or skills of how to develop a system for non-technical users. we should not expect that those with the technical skills also have skills in graphic design, technical communication or cognitive psychology. these are specialist fields in their own right. but if we want systems, and in particular e-commerce systems, that can be used by anyone then we need people with both the technical /functional skills as well as skills to ensure the system is effective and usable. 27 ajis vol 9 no. 1 september 2001 references avison, d. £. and g fitzgerald (1995). information systems development: methodologies, techniques and tools. london, the mcgraw-hill companies. ballantine, j., b. bonner, et al. (1996). "the 3-d model of information systems success: the search for the dependent variable continues." information resources management journal (fall): 5-14. barki, h. and j. hartwick (1989). "rethinking the concept of user involvement." mis quarterly 13(1) 59 82 bresko, l. l. (1995). the need for technical communicators on the software development team. perspectives on the profession of technical communication. m. flammia. florida, society for technical communication. 1: 213-217. coe, l. (1996). "five small secrets to systems success." information resources management journal (fall): 29-38. delone, w. and e. mclean (1992). "information systems success: the quest for the dependent variable." information systems research 3(1): 60-95. doll, w. and g torkzadeh (1987). "the quality of user documentation." information and management 12: 73-78. doll, w. and g torkzadeh (1988). "the measurement of end-user computing satisfaction." mis quarterly 12(2): 259-274. fisher, j. (1999a). "the value of the technical communicator's role in the development of information systems." ieee transactions on professional communication 42(3): 145-155. fisher, j. (1999b). "improving the usability of information systems: the role of the technical communicator." european journal of information systems 8(4): 294-303. geriach, j. and f.-y. kuo (1991). "understanding human-computer interaction for information systems design." mis quarterly 15(4): 527-549. gibson, m. l. and c. t. huches (1994). systems analysis and design: a comprehensive methodology with case. united states, boyd & fraser publishing company. goodhue, d. and r. thompson (1995). "task-technology fit and individual performance." mis quarterly 19(2): 213-236. gould, j., j. ukelson, et al. (1996). improving user interfaces and application productivity by using the its application development environment. human-computer interface design. m. rudisill, c. lewis, p. poison and t. mckay. san francisco, morgan kaufinann: 173-197. grudin, j. (1991a). "interactive systems: bridging the gaps between developers and users." ieee transactions on computers april: 59-69. grudin, j. (1991b). "systematic sources of suboptimal interface design in large product development organizations." human computer interaction 6(2): 147-196. hirschheim, r. and m. newman (1988). "information systems and user resistance: theory and practice." the computer journal 31(5): 398-408. jordan, e. and j. burn (1997). information systems acceptance: a consequence of cultures. pacis '97, brisbane, information systems management research concentration, queensland university of technology, australia 451-459 kay, a. (1990). user interface: a personal view. the art of human-computer interface design,. b. laurel. usa, addison wesley publishing comp: 191-207. kekre, s., m. krishnan, et al. (1995). "drivers of customer satisfaction for software products: implications for design and service support." management science 41(9): 1456-1470. kubie, j. j., l. a. melkus, et al. (1996). "user-centered design for productive systems." information systems management (2): 38-48. lawrence, m. and g low (1993). "exploring individual user satisfaction within user-led development." mis quarterly 17(2): 195-208. lyytinen, k. (1988). "expectation failure concept and systems analysts' view of information system failures: results of an exploratory study." information and management 14:45-56. mantel, m. and t. teorey (1989). "incorporating behavioral techniques into the system development life cycle." mis quarterly 13(3): 257-274. mccomb, d. and j. y. smith (1991). "system project failure: the heuristics of risk." journal of information systems management 8(1): 25-34. mckeen, j., t. guimaraes, et al. (1994). "the relationship between user participation and user satisfaction: an investigation of four contingency factors." mis quarterly 18(4): 427-451. misic, m. (1996). "the skills needed by today's systems analysts." journal of systems management 47(3): 34-37. montazemi, a. (1988). 'tactors affecting information satisfaction in the context of the small business environment." mis quarterly 12(2): 239-256. 28 ajis vol 9 no. 1 september 2001 myers, m. d. (1993). a disaster for everyone to see: an interpretive analysis of a failed is project. proceedings 4th australian conference on information systems. 4th australian conference on information systems, brisbane, department of commerce, the university of queensland. 595-614 nicholas, j. m. (1990). managing business & engineering projects. new jersey, prentice-hall inc. nichols, m. c. (1994). "using style guidelines to create consistent online information." journal of the society for technical communication 41(3): 432-438. norman, d. (1990 b). why interfaces don't work. the art of human-computer interface design,. b. laurel. usa, addison wesley publishing comp: 209-219. robert, d. (1997). "creating an environment for project success." information systems management 14(1): 73-77. rouse, a., d. watson, et al. (1995). system development practices: a comparison of the experience of australian and united states' organizations. 1995 pan pacific conference on information systems, singapore, national university of singapore. 46-57 shand, r. m. (1994 b). "user manuals as project management tools: part ii practical applications." ieee transactions on professional communication 37(3): 123-141. simon, h. (1994). the executive as decision-maker. readings in decision support systems. d. arnott and p. ot)onnell. melbourne, department of information systems, monash university: 59-64. szajna, b. and r. scamell (1993). "the effects of information system user expectations on their performance and perceptions." mis quarterly 17(4): 493-516. torkzadeh, g and w. doll (1993). "the place and value of documentation in end-user computing." information and management 24: 147-158. turner, r. and g lowry (1999). educating information systems professionals: towards a rapprochment between new graduates and employers. australasian conference on information systems, wellington, new zealand, school of communications and information management, victoria university of wellington, new zealand. 1049-1058 underwood, a. and g maynard (1996). acs guidelines for the accreditation of courses in universities at the professional level australasian conference on information systems 1996, hobart, tasmania, university of tasmania. 759-765 weinberg, r. s. (1991). "prototyping and the systems development life cycle." journal of information systems management 8(2): 35-43. williams, r. (1996). the impact of cognitive style on user satisfaction with an information systems: a preliminary study. australasian conference on information systems 1996, hobart, tasmania, university of tasmania. 789-797 29 ajis vol 9 no. 1 september 2001 ontological analysis of integrated process models: testing hypotheses peter green, michael rosemann dr peter green department of commerce university of queensland ipswich, australia p.green@mailbox.uq.edu.au +61(0)73381 1121 +61 (0)7 3381 1029 abstract integrated process modeling is achieving prominence in helping to document and manage business administration and it processes in organizations. the aris framework is a popular example for a framework of integrated process modeling not least because it underlies the 800 or more reference models embedded in the world's most popular erp package, sap r/3. this paper demonstrates the usefulness of the bunge-wand-weber (bww) representation model for evaluating modeling grammars such as those constituting aris. it reports some initial insights gained from pilot testing green and rosemann's (2000) evaluative propositions. even when considering all five views of aris, modelers have problems representing business rules, the scope and boundary of systems, and decomposing models. however, even though it is completely ontologically redundant, users still find the function view useful in modeling. introduction the aim of this paper is to report the results of some initial data gathering and testing of propositions that derived from an ontological evaluation of integrated process modeling (green & rosemann 2000). this evaluation used the aris framework popularized by scheer (1999) for integrated process modeling. over the last ten years, process management has received increased attention within the business administration and information systems communities. this increased attention derives from the fact that understanding and managing processes are tasks critical to achieving benefits from such new management philosophies as total quality management, activity-based costing, and business process re-engineering (hammer 1990; davenport 1993). accordingly, integrated process modeling as a means of documenting, analyzing, and evaluating processes has also risen to prominence in organizations over the last decade. scheer's aris framework was selected as the example of integrated process modeling for the ontological evaluation for a number of reasons. first, it integrates five views of the process (data, function, organization, output, and process) to provide the user with a comprehensive modeling framework that can be used to generate relatively easy-to-understand process models. second, its implementation through aris toolset claims in excess of 20,000 licenses worldwide. accordingly, there appears to be a large, mature user base against which analytical propositions can be tested. finally, aris underlies the reference models provided in one of the world's most popular enterprise resource planning (erp) packages worldwide at the present time, sap r/3. this work is motivated by the desire to develop and begin testing the propositions generated by green and rosemann's (2000) ontological evaluation of scheer's aris integrated process modeling framework in doing so, we can develop and initially test a data-gathering instrument. we can use the instrument to collect some preliminary data on the propositions. from the results, we can refine the data-gathering instrument but, more importantly, we can begin providing some insights into the usefulness of the ontological evaluation for process modelers using aris. accordingly, this paper reports the formulation of a testing instrument and essentially its initial pilot testing with a cohort of post-graduate information systems students studying process modeling. the paper unfolds in the following manner. first, some further background is provided on what is ontology and how the ontological evaluation was performed. next, green and rosemann's (2000) analytical results are summarized. then, the research methodology used in this work is explained. following that explanation, the results of this initial work are presented and discussed. finally, we explain briefly how this work will be progressed. background as grammars for information systems analysis and design have proliferated over the years, researchers and practitioners alike have attempted to determine objective bases on which to compare, evaluate, and determine when to use these grammars (e.g., batra, hoffer, & bostrom 1990; karam & casselman 1993). throughout the 80's and 90's, and now into the new millennium however, it has become increasingly apparent to many researchers that without a theoretical foundation on which to base information systems analysis and design (isad) grammar specification, incomplete evaluative frameworks of factors, features, and facets would continue 30 microsoft word ajis 11 1 final.doc ajis vol.11 no. 1 september 2003 27 patterns of programmers’ use of computer-mediated communication systems chatpong tangmanee faculty of commerce and accountancy chulalongkorn university bangkok 10330, thailand tel. 66-2-218-5870, fax 66-2-218-5868 fcomctm@phoenix.acc.chula.ac.th abstract communication behavior of programmers plays an essential role in success of software development. computer-mediated communication (cmc) system, such as e-mail, or the world wide web (www), have substantial implications for coordinating work of programmers. yet, no studies have dealt systematically with cmc behaviors of programmers. drawing upon theories in organizational studies, information science, computer-mediated communication and software engineering, this research examines what programmers accomplish through cmc systems. data were gathered from survey questionnaires mailed to 730 programmers, who are members of the association of computing machinery (acm) and are involved in a variety of programming work. based on factor analysis, the study found that programmers use cmc systems (1) to achieve progress in work-related tasks (i.e., task-related purposes), (2) to satisfy their social and emotional needs (i.e., socio-emotional purposes), and (3) to explore for information (i.e., exploring purposes). the findings of this research extend an insight into important patterns for which programmers use cmc systems. this insight has advanced theories of computer-mediated communication in the context of computer programmers. also, practitioners, especially in software development, may use the results as guidelines in fostering a firm’s feasible network policy that fits with what their programming staff accomplish through computer-mediated communication. introduction the goal of this research is to extend our understanding of computer-mediated communication (cmc) behavior of programmers. similar to other professionals, programmers must possess certain skills besides their programming abilities, one of which is communication expertise. empirical work has shown the importance of programmers’ communication behavior. at microsoft, for example, programmers often exchange feedback with colleagues regarding their work (cusumano & smith, 1997). formal communication, such as meetings with clients, improves the quality of software products (kraut & streeter, 1995). also, informal communication such as interaction among programmers at a vending machine appears to have a significant impact on programming quality (fowler, 1999; weinberg, 1998). in addition to the basic principle that programmers should have open communication with other people, software engineers have developed rigorous guidelines to improve programming performance. these guidelines offer instructions by which programmers learn how to integrate programming expertise with communication efforts in a way that maximizes their productivity (humphrey, 1997; yourdon, 1996, 1999). code reviews, for example, are software engineering techniques that encourage programmers to share and review code with peers and correct flaws which may be detected during the reviews (yourdon, 1996). despite the importance of computer-mediated communication behavior of programmers, no empirical studies have dealt exclusively with this topic. the current study explores one important, yet largely unknown aspect: the manner in which programmers are engaged in computer-mediated communication. computer-mediated communication systems computer-mediated communication (cmc) systems are defined in this study as programmers’ perceptions of a collection of tools that facilitate human communication via computers and electronic communication networks. because of the evolution of technology, a great number of cmc systems offer various features such as information searching or information filtering. although these features are not central to the communication, they extend the utilitarian functions of ajis vol.11 no. 1 september 2003 28 cmc systems. it must be clear that the focus of this research is on cmc systems as perceived by programmers to primarily facilitate human communication. the systems that do not support human communication are therefore excluded from the study. specifically, the systems that mainly (1) support the software development process (e.g., case tools), (2) promote the management of projects, or (3) provide access to database systems are not part of the family of cmc systems and therefore excluded from this study. in addition, neither voice mail nor facsimiles are included because a large number of users do not perceive the communication via these two channels as computer-mediated (rice & steinfield, 1994). thus, the instances of cmc systems investigated in this research include computer conferencing systems (e.g., e-mail or visual conferences), internetbased communication (e.g., internet relay chat (irc), internet phone, or the www), group support systems, groupware (e.g., lotus notes) and the systems that support communication via e-mail, newsgroups and/or listserves. cmc system usage and programmers the study’s main focus is to examine what programmers accomplish through the use of cmc systems. this focus is consistent with rice’s (1980, p. 238) critical remark that “… it would be fruitful to determine just what it is that people do when they interact via computer.” findings from previous studies note that people are engaged in computer-mediated communication for at least two key purposes: to attain progress in work-related tasks and to satisfy social and emotional needs. researchers and practitioners in organisational communication have acknowledged the use of cmc systems to achieve progress or completion in work-related tasks. sawyer and guinan (1994, 1998) examined how computer-mediated communication would affect intra-group conflict-solving process and control management among teams of software developers. “by focusing [on] the work product, and not [on] each other, the product becomes less attached to any one person: it is shared by the team” (sawyer & guinan, 1994, p.82). with help from cmc systems, eliciting program requirements from users appears to be more effective than doing the same with no help (liou & nunamaker, 1993). retrieval features in major cmc systems can also reduce clerical errors in code reviews (johnson, 1998). in addition to task-related benefits, the use of cmc systems can accommodate the social and emotional needs of programmers. this type of use has been ascertained in various investigations, although only a few have been conducted in the programmer context. for example, workers used email not only to handle work assignments, but also to stay in touch with friends or family, or to meet people with the same interests (rice & steinfield, 1994). with a speedy transmission of distributed computing technology and algorithms that facilitate network security, programmers can complete any financial transaction (i.e., purchase a plane ticket or buy stock) over the internet as well as locate almost any information of interest. they can also be entertained by a variety of information offered through cmc systems, or download numerous kinds of information for their own pleasure (mehta & plaza, 1997). besides the two major purposes (i.e., task-related and socio-emotional), evidence from the literature suggest that programmers may use cmc systems for other purposes. for instance, rice and steinfield (1994) identified the surveillance purposes for using e-mail in an organisational setting. other writers comment that organisational members may use communication to explore innovative ideas from the environment and, perhaps, share the knowledge with colleagues (heath, 1994; farace, et al., 1977). methodology the current research employs a self-administered mail survey to gather data from 780 actual programmers who are members of the association of computing machinery (acm). in the typical worst case of a possible 30% response rate commonly known in the survey literature, the sample size of 780 is likely to yield more than 230 survey respondents, the number of which is acceptable to provide a statistically significant finding (babbie, 1992). it is important to note that a set of fifty programmers from the acm list were drawn to participate in a pilot study, leaving the total of 730 programmers for the actual survey. ajis vol.11 no. 1 september 2003 29 questionnaire development the study’s questionnaire was developed based upon (1) an extensive review of literature on computer-mediated communication, software engineering, and organisational research and (2) the researcher’s interviews with programmers. from the literature review, questionnaire scales associated with various purposes for using cmc systems. the subsequent interviews with actual programmers allowed for appropriate adjustments to the scales so that the questionnaire could be clear and understandable to programmers. when the first draft was ready, the questionnaire was pre-tested with a group of programmers, survey scholars and information system researchers. based on feedback from these pretest participants, changes to the questionnaire items were made to reduce ambiguous wording, poor transition among sections in the questionnaire, or unclear explanation, thereby enhancing the clarity and comprehensibility of the instrument. after the pretest, the questionnaire was pilot-tested with fifty programmers drawn from the acm list. statistical techniques were applied to the pilot test so as to assess the questionnaire’s reliability and validity and to modify it based on the results. results from the pilot test confirmed the acceptable quality of the questionnaire, thereby yielding reliable and valid data necessary to satisfy the research’s objectives. survey administration one of the drawbacks in using a mail survey is a low response rate. sometimes, it is so low that the final conclusion is improbable. the researcher has made an effort to draw as many responses as possible by following recommendations from survey researchers (babbie, 1992; dillman, 2000). the effort included (1) notifying subjects about the survey before they receive the questionnaire, (2) preparing all survey documents (e.g., a cover letter or a questionnaire) in the ways that motivate subjects to promptly and accurately respond to the survey and (3) implementing a three-step followup plan. within a three month period of data collection, 438 programmers returned usable questionnaires. this amounts approximately to a 60% response rate. according to babbie (1992), it is a good response rate. about 40% of the subjects did not return the questionnaires. it may therefore pose bias between respondents and non-respondents. using the trend projection approach (hertman, et al., 1985; smith, 1997), however, no bias was detected. results respondent characteristics table 1 presents important characteristics of programmers who participated in this research. the highlights of these characteristics are as follows: • the majority (88%) of participants are men. about half (56%) of the respondents hold their highest degree in computer science while other individuals are from adjacent fields: mathematics (13%), engineering (11%), management and business administration (6%), information science (4%) and physics (3%). ajis vol.11 no. 1 september 2003 30 table 1: characteristics of participating programmers major characteristics respondents n (%) gender (n=431) male 381 88 female 50 12 age (n=434) 20-29 yrs. 18 4 30-39 134 31 40-49 166 38 50+ 116 27 highest education (n=434) college degrees or some college work 118 27 masters or some graduate work 273 63 doctoral or higher 43 10 major (n=429) computer science 239 56 mathematics 55 13 engineering 49 11 management or business administration 27 6 information science 16 4 physics 13 3 others (e.g., education, music, etc.) 30 7 number of hours per day doing programming work (n=429) 1-3 hrs 127 30 3-6 118 27 6+ 184 43 work responsibility (n=432) developing in-house systems 181 42 developing packaged software 149 35 installing packaged software in-house 17 4 combination of the above three 23 5 others (e.g., educational software) 62 14 • considering that the sample was selected from regular members of the acm who described themselves as programmers, it is not surprising that more than half of the respondents (63%) hold master degrees and about the same percentage (65%) are forty years old or higher. it seems that young programmers or those fresh from college are not acm members as they may not yet realize the benefits of the memberships. still, it is possible that selecting members of a professional association as research participants may bias the study. • the research collected data from actual programmers, instead of from computer-related students. for at least six hours per day, a fair number of participants (43%) are engaged in programming work such as designing, testing, writing or maintaining computer software. furthermore, responses to the questionnaire’s "work responsibility" item seem to confirm that the participants are in charge of various types of programming projects, ranging from developing in-house software systems (42%) and building packaged software products (35%), to installing packaged software (4%). also, they reported that they have been working as programmers for approximately eighteen years. these findings thus ensure that ajis vol.11 no. 1 september 2003 31 the survey participants encompass professional programmers holding various actual programming responsibilities, not student programmers working on class assignments. patterns of cmc system usage thirty-five items reflecting various activities in which one may be engaged through the use of cmc systems were included in the questionnaire (see these items in appendix a). to indicate how often subjects use cmc systems for these activities, they rated the items from never (0) to very often (4). an “other” item was added as subjects may use cmc systems for purposes not listed. however, only five programmers responded to the “other” item, two of whom used cmc systems for monitoring a stock market and the other three used them for miscellaneous activities. the responses to the “other” item were subsequently excluded from subsequent analysis. to uncover the key purposes underlying what programmers accomplish through the use of cmc systems, the thirty-five items were factor-analyzed. prior to the analysis, however, the items with marginal variance were excluded as they would not serve to differentiate among emerging factors (comrey & lee, 1992). an objective criterion of a standard deviation of less than one is used to determine which items should be dropped from the analysis. as a result, four of the 35 items were excluded. these four items are: enjoying provocative contents, acquiring non-computer knowledge, locating people with the same conditions and learning about social events. based on factor analysis with principle axis extraction and oblique rotation, three meaningful factors that underscore the major purposes for which programmers use cmc systems emerged. table 2 displays the three purpose factors and the items that reflect on each purpose. also included are weights of the items on the three factors. the three factors together explained about 42% of the variance among the purpose items. according to table 2, factor i accounted for 29.1% of the variance. highest weights of the nine items on the first factor seem to reflect the "task-related" use of cmc systems. factor ii explained 8.3% of the variance. four items loaded highest on this factor, indicating that programmers use cmc systems for "socio-emotional" benefits. the final factor, factor iii, accounted for 4.5% of the variance. highest weights of the other four purpose items tend to suggest the "exploring" purpose for using cmc systems. assessments of the factor structure (e.g., kaiser-meyerolkin (kmo) index, residual correlation analysis, and bartlett’s test of sphericity) suggest that the discovery of these three purposes behind programmers’ use of cmc systems is conceptually parsimonious and methodologically sound. conclusions programmers’ accomplishments through use of cmc systems upon the study’s results, programmers appear to use cmc systems for three major purposes. they are task-related, socio-emotional and exploring. the first two purposes are commonly recognized in the literature. however, the recognition of the third purposes is relatively limited. researchers and practitioners have known that communication serves in general as a means to (1) transfer information for individuals to achieve their work progress and (2) satisfy one’s social and emotional needs. the study’s findings of these two purposes may stress the essential combination of “work” and “play”, particularly in the workplace environment; and therefore indicate that the use of cmc systems for socio-emotional purposes is as useful as the use to gain progress in work-related tasks. the use of cmc systems for exploring purposes is noted by a few scholars (choo, 1998; farace, et al., 1977; metroyer-duran, 1993). these researchers have acknowledged exploring functions of computermediated communication as the transfer of knowledge or innovative information between an organisation and its environment. this piece of information may help an organisation to cope with changes, especially when the organisation’s environment is undergoing dramatic shift. given the dynamic and various changes in the software development environment, it is reasonable to argue that programmers conduct an exploration via cmc systems, perhaps, in search of software innovation and creative ideas (e.g., ready-to-use programming applets, or details of product upgrades) so as to survive the turbulent condition. choo (1998) comments that the internet--an instance of cmc systems—offers ajis vol.11 no. 1 september 2003 32 a unique channel through which technology-oriented industries such as software development firms could explore and benefit from various types of information. implications implications for existing theories in many fields may be derived from the study's findings. regarding communication functions, the current study has confirmed the need to incorporate all three major purposes (i.e., task-related, socio-emotional and exploring) into research on computer-mediated communication. much research has addressed the use of communication to achieve task-related benefits as well as to satisfy one's social and emotional needs. however, relatively little research has covered the purposes of exploring for information in one's environment. overlooking one purpose may result in an incomplete understanding of communication functions. table 2: factor analysis results: purposes for using cmc systems purpose items factors i ii iii factor i: "task-related" discuss work information with co-workers .75 .06 -.01 coordinate work with distant colleagues .52 -.01 -.04 schedule meetings .65 -.05 -.04 give or receive feedback on work assignments .74 .04 .03 send confirmation to colleagues/clients .64 .02 .01 discuss work with clients .54 .04 .08 resolve work conflicts or disagreements .60 .03 .07 transfer files .54 .05 .31 keep track of what’s happening in a company .53 .10 .06 factor ii: "socio-emotional" fill free time -.02 .77 -.02 greet people on social occasions (e.g., sending friends electronic cards) .16 .52 .05 be entertained (e.g., electronic humor) .02 .68 .02 take a break from work -.02 .78 -.06 factor iii: "exploring" check out new services/products .03 -.04 -.75 stay up-to-date on computer or product upgrades .02 .04 -.73 seek out alternatives to work problems .05 .03 -.62 download information .00 -.02 -.83 percent of variance explained 29.1% 8.3% 4.5% =41.9% implications implications for existing theories in many fields may be derived from the study's findings. regarding communication functions, the current study has confirmed the need to incorporate all three major purposes (i.e., task-related, socio-emotional and exploring) into research on computermediated communication. much research has addressed the use of communication to achieve taskrelated benefits as well as to satisfy one's social and emotional needs. however, relatively little ajis vol.11 no. 1 september 2003 33 research has covered the purposes of exploring for information in one's environment. overlooking one purpose may result in an incomplete understanding of communication functions. in addition to theoretical implications, the findings also have practical utility. not only did the study confirm that programmers use cmc systems to attain work progress and to satisfy their own social and emotional needs, it also empirically noted that the programmers are engaged in exploration via cmc systems. these exploratory activities include downloading information over the internet, staying upto-date on computer-related issues, checking out new services and seeking out alternatives to work problems. limitations inferences from this research are limited by two major factors. first, the demographics of the participants appear to temper the study’s generalizability of results to the programmer population in general. the programmers who participated in this study are dominantly male, between 30-49 years of age and with at least a college degree. this consequently limits the generalization of the study, despite the random sample of about 700 members, the high percentage of survey returns and the respondents' various programming responsibilities. the second limitation is more methodological. this is a cross-sectional study. further, the phenomenon under study (i.e., cmc system usage) is dynamically changing due to rapid development of computer technology. hence, the analyses and the conclusions in this manuscript present only a snapshot of how programmers use cmc systems. additionally, prior to this study, very little was known about programmers’ computer-mediated communication behavior. the study's execution was therefore made based upon an exploratory approach. this is the main reason that the investigated patterns were neither hypothesized nor tested. nevertheless, the study has ascertained the three purposes for which programmers use cmc systems. future studies may thus test any specific hypothesis on their use of computer-mediated communication. ajis vol.11 no. 1 september 2003 34 appendix a how often do you use cmc systems of all kinds to accomplish the following activities? think of the activities that you complete by using cmc systems. please circle the appropriate number. i use cmc systems generally in order to: never rarely some times often very often not applicable distribute or discuss work information with coworkers (e.g., design alternatives)...................... 0 1 2 3 4 8 check out the weather........................................... 0 1 2 3 4 8 place a purchase order............................................ 0 1 2 3 4 8 look up job or promotion openings...................... 0 1 2 3 4 8 coordinate programming work with distant colleagues............................................................ 0 1 2 3 4 8 check out new services/products (e.g., updated software)............................................................. 0 1 2 3 4 8 seek help for non-work problems (e.g., how to fix a floor)..........................................................….. 0 1 2 3 4 8 schedule meetings with colleagues/clients............ 0 1 2 3 4 8 update an address book.......................................... 0 1 2 3 4 8 give or receive feedback on work assignments..... 0 1 2 3 4 8 fill free time.......................................................... 0 1 2 3 4 8 complete a work assignment from outside the office………………………………………….. 0 1 2 3 4 8 greet people on social occasions (e.g., birthday).. 0 1 2 3 4 8 send confirmation to colleagues or clients............ 0 1 2 3 4 8 stay up-to-date on or learn about computer technology or product upgrades.......................... 0 1 2 3 4 8 discuss work information with clients.................. 0 1 2 3 4 8 be entertained (e.g., electronic humor or computer games)................................................................. 0 1 2 3 4 8 stay updated on an area of interest (e.g., a hobby)................................................................. 0 1 2 3 4 8 enjoy provocative contents.................................... 0 1 2 3 4 8 keep in touch with friends or family members...... 0 1 2 3 4 8 seek out alternatives to work problems (e.g., a better algorithm or reusable code)...................... 0 1 2 3 4 8 ajis vol.11 no. 1 september 2003 35 references babbie, e. r. (1992). the practices of social research (6th edition). ca: wadsworth publications. choo, c. w. (1998). the knowing organization: how organizations use information to construct meaning, create knowledge, and make decisions. ny: oxford university press. comrey, a. & lee, h. b. (1992). a first course in factor analysis. nj: lawrence erlbaum. cusumano, m. a., & smith, s. a. (1997). “beyond the waterfall: software development at microsoft”. in d. b. yoffie (ed.), competing in the age of digital convergence (pp. 371412). ma: harvard business school press. dillman, d. (2000). mail and internet surveys. ny: wiley. farace, r. v., monge, p. r., & russell, h. m. (1977). communicating and organizing. reading, ma.: addison-wesley. fowler, j. (1999). “understanding it: a perspective”. target marketing., 22(7), 68-71. hartman, b. w., fuquua, d. r., & jenkins, s. j. (1985). “the problems of and remedies for nonresponse bias in educational surveys”. journal of experimental education, 54(4), 85-91. heath, r. l. (1994). management of corporate communication. nj: lawrence erlbaum. humphrey, w. s. (1997). managing technical people. ma: addison-wesley. johnson, p. m. (1998). “reengineering inspection”. communications of the acm, 41(2), 49-52. kraut, r. e., & streeter, l. (1995). “coordination in software development”. communications of the acm, 38(3), 69-81. liou, y. i., & nunamaker, j. f. (1993). “an investigation into knowledge acquisition using a group decision support system”. information and management, 24(3), 121-132. i use cmc systems generally in order to: never rarely some times often very often not applicable learn about or participate in social events (e.g., protest a new law, time to enjoy fall foliage)...... 0 1 2 3 4 8 locate human experts for help with work (e.g., programming) problems...................................... 0 1 2 3 4 8 find people of the same conditions or same social interests (e.g., antique-car lovers)...........……… 0 1 2 3 4 8 resolve work conflicts or disagreements............... 0 1 2 3 4 8 check out sports events (e.g., golf tournament results or scores of last night’s games)............... 0 1 2 3 4 8 take a break from work......................................... 0 1 2 3 4 8 learn non-computer knowledge (e.g., new way of recycling or health-related information)............. 0 1 2 3 4 8 create and maintain bookmarks............................. 0 1 2 3 4 8 transfer files to colleagues.................................... 0 1 2 3 4 8 keep track of what’s happening in my company............................................................. 0 1 2 3 4 8 hold a work conference with colleagues (or clients)................................................................. 0 1 2 3 4 8 other (please specify............................................) 0 1 2 3 4 8 ajis vol.11 no. 1 september 2003 36 mehta, m. d., & plaza, d. e. (1997). “pornography in cyberspace: an exploration of what's in usenet”. in s. kiesler (ed.), culture of the internet (pp. 53-67). nj: lawrence erlbaum. metoyer-duran, c. (1993). “information gatekeepers”. in m. e. williams (ed.), annual review of information science and technology (pp. 111-150). nj: learned information. rice, r. e. (1980). “the impacts of computer-mediated organizational and interpersonal communication” in m. w. williams (ed.), annual review of information science and technology (pp. 221-249). ny: knowledge industry publications, inc. rice, r. e., & steinfield, c. w. (1994). “experiences with new forms of organizational communication via electronic mail and voice messaging”. in j. h. andriessen & r. a. roe (eds.), telematics and work (pp. 106-137). east sussex, uk: lawrence erlbuam. sawyer, s., & guinan, p. (1994). “team-based software development using an electronic meeting system: the quality pay-off”. in m. lee, b. barta & p. jaliff (eds.), software quality and productivity: theory, practice, education and training (pp. 78-83). new york, ny: chapman hall. sawyer, s., & guinan, p. (1998). “software development: processes and performance”. ibm systems journal, 37(4). smith, j. (1997). “nonresponse bias in organizational surveys: evidence from a survey of groups and organizations working for peace”. nonprofit and voluntary sector quarterly, 26(3), 359368. weinberg, g. m. (1998). the psychology of computer programming (the silver anniversary edition). ny: dorset house inc. yourdon, e. (1996). rise and resurrection of the american programmer. nj: yourdon press. yourdon, e. (1999). death march. ny: prentice hall, inc. ajis vol 9 no. 1 september 2001 ontological analysis of integrated process models: testing hypotheses peter green, michael rosemann dr peter green department of commerce university of queensland ipswich, australia p.green@mailbox.uq.edu.au +61(0)73381 1121 +61 (0)7 3381 1029 abstract integrated process modeling is achieving prominence in helping to document and manage business administration and it processes in organizations. the aris framework is a popular example for a framework of integrated process modeling not least because it underlies the 800 or more reference models embedded in the world's most popular erp package, sap r/3. this paper demonstrates the usefulness of the bunge-wand-weber (bww) representation model for evaluating modeling grammars such as those constituting aris. it reports some initial insights gained from pilot testing green and rosemann's (2000) evaluative propositions. even when considering all five views of aris, modelers have problems representing business rules, the scope and boundary of systems, and decomposing models. however, even though it is completely ontologically redundant, users still find the function view useful in modeling. introduction the aim of this paper is to report the results of some initial data gathering and testing of propositions that derived from an ontological evaluation of integrated process modeling (green & rosemann 2000). this evaluation used the aris framework popularized by scheer (1999) for integrated process modeling. over the last ten years, process management has received increased attention within the business administration and information systems communities. this increased attention derives from the fact that understanding and managing processes are tasks critical to achieving benefits from such new management philosophies as total quality management, activity-based costing, and business process re-engineering (hammer 1990; davenport 1993). accordingly, integrated process modeling as a means of documenting, analyzing, and evaluating processes has also risen to prominence in organizations over the last decade. scheer's aris framework was selected as the example of integrated process modeling for the ontological evaluation for a number of reasons. first, it integrates five views of the process (data, function, organization, output, and process) to provide the user with a comprehensive modeling framework that can be used to generate relatively easy-to-understand process models. second, its implementation through aris toolset claims in excess of 20,000 licenses worldwide. accordingly, there appears to be a large, mature user base against which analytical propositions can be tested. finally, aris underlies the reference models provided in one of the world's most popular enterprise resource planning (erp) packages worldwide at the present time, sap r/3. this work is motivated by the desire to develop and begin testing the propositions generated by green and rosemann's (2000) ontological evaluation of scheer's aris integrated process modeling framework in doing so, we can develop and initially test a data-gathering instrument. we can use the instrument to collect some preliminary data on the propositions. from the results, we can refine the data-gathering instrument but, more importantly, we can begin providing some insights into the usefulness of the ontological evaluation for process modelers using aris. accordingly, this paper reports the formulation of a testing instrument and essentially its initial pilot testing with a cohort of post-graduate information systems students studying process modeling. the paper unfolds in the following manner. first, some further background is provided on what is ontology and how the ontological evaluation was performed. next, green and rosemann's (2000) analytical results are summarized. then, the research methodology used in this work is explained. following that explanation, the results of this initial work are presented and discussed. finally, we explain briefly how this work will be progressed. background as grammars for information systems analysis and design have proliferated over the years, researchers and practitioners alike have attempted to determine objective bases on which to compare, evaluate, and determine when to use these grammars (e.g., batra, hoffer, & bostrom 1990; karam & casselman 1993). throughout the 80's and 90's, and now into the new millennium however, it has become increasingly apparent to many researchers that without a theoretical foundation on which to base information systems analysis and design (isad) grammar specification, incomplete evaluative frameworks of factors, features, and facets would continue 30 ajis vol 9 no. 1 september 2001 to proliferate. furthermore, without a theoretical foundation, one framework of factors, features, or facets is as justifiable as another for use (floyd 1986; bansler & bodker 1993). wand and weber (1989a, 1989b, 1990a, 1990b, 1991, 1993, 1995) have investigated the branch of philosophy known as ontology (or meta-physics) as a foundation for understanding the process in developing an information system. ontology is a well-established theoretical domain within philosophy dealing with models of reality. wand and weber (1989b, 1990a, 1990b, 1993, 1995) and weber (1997) have taken, and extended, an ontology presented by bunge (1977) and applied it to the modeling of information systems. their fundamental premise is that any isad modeling grammar must be able to represent all things in the real world that might be of interest to users of information systems; otherwise, the resultant model is incomplete. if the model is incomplete, the analyst/designer will somehow have to augment the model(s) to ensure that the final computerized information system adequately reflects that portion of the real world it is intended to simulate. the bunge-wand-weber (bww) (1989b, 1990a, 1990b, 1993, 1995) models consist of the representation model, the state-tracking model, and the good decomposition model. this work focuses on the representation model. the representation model defines a set of constructs that, at this time, are thought by the wand and weber to be necessary and sufficient to describe the structure and behavior of the real world.1 weber (1997) clarifies two major situations that may occur when an isad grammar is analyzed according to the representation model. after a particular isad grammar has been analyzed, predictions on the modeling strengths and weaknesses of the grammar can be made according to whether some or any of these situations arise out of the analysis. 1. ontological incompleteness (or construct deficit) exists unless there is at least one isad grammatical construct for each ontological construct. 2. ontological clarity is determined by the extent to which the grammar does not exhibit one or more of the following deficiencies: construct overload exists in an isad grammar if one isad grammatical construct represents more than one ontological construct. construct redundancy exists if more than one isad grammatical construct represents the same ontological construct. construct excess exists in an isad grammar when an isad grammatical construct is present that does not map into any ontological construct. accordingly, the representation model in particular provides a theoretical foundation on which isad grammars including those used in integrated process modeling frameworks like aris can be evaluated and developed. ontological analysis of aris and derived propositions table 1 shows the detailed results of the green and rosemann (2000) evaluation of aris. each of the five views of aris was analyzed using the ontological constructs of the bww representation model. green and rosemann (2000) explain that, examining only the process view, instances of ontological incompleteness lead to the following propositions: 1. because there are no direct representations for thing, class, and/or kind, users will lack conceptual clarity regarding the object(s) in the real world to which the epc relates. the epc can represent indirectly attributes of the thing (business object) as attributes of the function type but not the thing itself. as weber (1997) explains, the world is made up of things. things in the world are identified via their properties; not the other way round. accordingly, some other symbol/grammar/view will be needed in conjunction with the process view to overcome this ontological deficiency. modelers may well find the existence of the organizational and output views useful to overcome this deficiency because they provide symbols that give representation to individual things (instances) in the real world. for a detailed description of the all the constructs in the representation model, see green and rosemann (2000) or weber (1997). 31 ajis vol 9 no. 1 september 2001 ontolosical construct thing property: in particular in general intrinsic mutual emergent hereditary attributes class kind state conceivable state space state law lawful state space event process conceivable event space transformation lawful transformation lawful event space history acts on coupling: binding mutual property system system composition system environment system structure subsystem system decomposition level structure external event stable state unstable state internal event well-defined event poorly-defined event '*rocess*iewm".' &3j-: function type attribute type event type (only the state variables that trigger the function) function type -» connector -> event type event type -» function type — > event type process model function type function type event type -» connector -» function type series of function type or event type decomposition indicators start event type (no ingoing arrow) end event type (no outgoing arrow) event type — > function type -» event type event type -» function type — » event type jtota-vjew^k.^fsksr domain attribute type, attribute group entity type specialization /generalization (is-a) specialization/general iza tion descriptors; [mia, max.] cardinalities relationship type (no symbol for relationship in grammar) cluster cluster function vlewj:-* ? >5;i function type attribute type, goal, goal structure specialization /generalization (is-a) specialization/generaliza tion descriptors function type, business process process oriented function decomposition function type series of function type decomposition indicators organizational view organizational unit, position, user (instances) location, qualification, requirements, profile location, qualification, requirements, profile attribute type organizational type, position, user (class) specialization /generalization (is-a) role organizational hierarchy organizational object organizational modeling levels ?oatpot'vtew! ' ' -• ••}. ,-•' product catalogue, product model, bill of materials (instances) cost type, cost rate material output/input, services specialization /generalization (is-a) product model hierarchy product modeling object product modeling levels table 1. bww representation model analysis of integrated process modeling views in aris. 2. because the process view does not have representations for conceivable state space, lawful state space, conceivable event space, and lawful event space, a sufficient focus to identify all important state and transformation laws may not be present during modeling. these laws are the basis of what are known in systems analysis as business rules. accordingly, problems may be encountered in capturing all the potentially important business rules of the situation. moreover, the modeler may neglect to 32 ajis vol 9 no. 1 september 2001 incorporate important process alternatives and the resultant model becomes less relevant or incomplete. again, some other symbol/grammar may need to be employed in combination with the process view to overcome this deficiency. 3. because the process view does not have representations for system, system composition, system environment, system structure, system decomposition, and coupling, the process view's usefulness for defining the scope and boundaries of the system being analyzed is undermined. moreover, the usefulness of the process view for undertaking "good decompositions" (see, for example, wand & weber 1995) during analysis is degraded. this issue is important because, in a comprehensive process modeling project, hundreds of individual process models could emerge. for the modeler using the process view, serious questions arise as to how to structure and factor the complexity of large projects into constituent models. again, the use of some other symbol/grammar would appear necessary to overcome this deficiency. when the analysis of the five views in combination in table 1 is considered, some interesting propositions issue. 4. even across the five views, no representations exist for conceivable state space, lawful state space, conceivable event space, or lawful event space. again, a sufficient focus to identify all important state and transformation laws may not be present during modeling. accordingly, problems may be encountered in capturing all the potentially important business rules of the situation. for example, in aris, it is not clear how to specify which organizational units are responsible for a function under certain conditions without having to split the function type. some other symbol/grammar may need to be employed to overcome this deficiency. 5. across the five views, no representations exist for system, system environment, system structure, and system decomposition. again, its usefulness for defining the scope and boundaries of the system being analyzed is undermined. moreover, the usefulness of integrated process modeling for undertaking "good decompositions" during analysis would seem questionable. for example, there is no requirement in aris to perform corresponding decompositions between the interrelated process view and data view of a model. indeed, because of this deficit situation, integrated process modeling would appear more useful in modeling existing situations to be modified (i.e., where a bottom-up approach is appropriate) as opposed to novel situations (i.e., where a top-down approach might be more appropriate). a modeling grammar seems to be of high quality if it is ontologically complete and ontologically clear. in other words, every empty row in table 1 indicates that even the entire aris approach incorporating the five views is ontologically incomplete. that is, certain ontological constructs do not appear, e.g., conceivable state space, conceivable event space, system, and system structure. these non-appearances form the bases of propositions 4 and 5. however, the converse argument could apply. the bww representation model might be misspecified or wrong. • could the bww representation model be over-engineered? maybe it includes constructs that are not relevant. the ontological analyses of various modeling grammars to date (wand & weber 1993, 1995; weber & zhang 1996; green 1997; green & rosemann 2000) have consistently identified certain ontological constructs that do not have representations in the grammars examined, e.g., conceivable state space, conceivable event space, and lawful event space. the ontological analyses to date in themselves form an empirical study around this hypothesis of overengineering. one possible conclusion then could be the identification of the need for a reduction in the number of constructs thought to be sufficient and necessary in the bww representation model. • even if the bww model is not over-engineered, most modeling grammars usually focus on modeling particular aspects of the real-world, e.g., statics, dynamics, processes, data, actors, actions, goals, and the like. apparently, the objectives of the modeling grammar need to be taken into account during the ontological analysis. such work identifies a need for individualization of the bww model by means of designing subsets and specializations of the bww model. • finally, there may be a need for extending the bww model. weber (1997) has already extended the understanding of the ontological construct, property, by explaining the various types of property, e.g., property in general, property in particular. the growing importance of strategic enterprise modeling might lead to the explication of the bww model to explicitly incorporate constructs to represent such abstract items as business objectives, strategies, goals, or knowledge. whether the situation exemplified in propositions 4 and 5 is indeed the case or the converse situation explained above holds is a question that can only be answered by evidence gained through a number of empirical investigations. the work reported in this paper contributes to the required body of evidence. green (1997) operationalized the weber and zhang (1996) concept of a minimal ontological overlapping (moo) set of grammars. effectively, grammars are combined to form a set that is as ontologically complete as possible. if the process view is selected as a starting point, the data view, and either the organizational or the output view 33 ajis vol 9 no. 1 september 2001 would need to be employed to form the moo set. the function view however appears to be ontologically redundant when compared to the moo set of grammars. for example, construct redundancy and construct overload are displayed by the function type symbol in the function view. this situation suggested the following proposition: 6. because the function view is completely ontologically redundant when compared to the combination of process view, data view, and either organizational view or output view, its use with the other views will undermine clarity and may cause confusion to users. for example, it is not clear whether function types should be entered in the function view or in the process view first. if a function type is split in the process view, this change is not necessarily reflected in the function view. this situation can lead to consistency problems, especially, if a modeling tool with an integrated repository is not available. accordingly, limited usefulness and use of the function view is expected. research methodology we created a questionnaire to gather data on the propositions outlined above. we intended in the first instance to conduct a limited empirical study by administering this instrument to students from a postgraduate subject (rein 1994). in this way, we might to achieve two goals. first, we could pilot test the raw instrument with a group of respondents. second, we could gather some initial data on which we could gain some insight into our propositions. based on the results of this initial testing, we could make a reasoned decision as to the usefulness of persisting with further, more comprehensive testing. in the longer term, we intended to administer our instrument to a sample of process modeling practitioners and academics both in australia and internationally. design of the questionnaire the questionnaire consisted of 11 pages and was divided in three sections a, b, and c. the purpose of section a was to collect information about the background of the respondents. this information included such items as years of general work experience, years of process modeling experience, years of experience with the aristoolset, and the general purpose of the activities for which the integrated process modeling tool was used documentation of organizational or ft processes. section b was constructed to gather data about the specific propositions. moreover, it focused on how respondents working individually used process models in the aristoolset. section c by contrast asked for feedback regarding the use of integrated process modeling techniques when using aris-toolset as a member in project teams. because there was no well-accepted, validated instrument known to the researchers for gathering data on such propositions, a new, raw instrument had to be created. green (1996) developed and validated a similar instrument for gathering data on ontologically-based hypotheses involving the use of isad grammars in combination by analysts/designers when using structured upper case tools. this validated instrument was used as the initial basis for the formulation of the instrument for this study.2 the participants the participants of this pilot survey were identified because of their enrolment in an advanced post-graduate subject dealing with aris at an australian university. sixteen post-graduate students were identified in the subject called "projects in process engineering". as a prerequisite for this subject, every participant had to been enrolled in the subject process engineering. in the process engineering subject, the students were taught the basics of business process management. they were introduced to the purposes of process modeling and the architecture of integrated information systems (aris) framework was explained in detail. in one comprehensive assignment, the students had to design "as-is" and "to-be" process models for selected case studies. thus, they developed the theoretical knowledge and had hands-on experiences with designing process models using aris. in the subject "projects in process engineering", students had the opportunity to apply this knowledge in a real world context (rosemann, sedera & sedera 2000). organized in four teams of four students each, over a period of 12 weeks they analyzed the following processes at a queensland government department: procurement, corporate card, leave request, and recruitment the students used the aris-toolset 4.1 for the design of the "as-is" and "to-be" models. some of the students also performed an aris-based process simulation. all process models were presented to the contact people from the collaborating department. students (or "novices") were considered to be appropriate subjects for this pilot test. in the words of vessey and conger (1994), students might actually be more appropriate subjects when comparing/evaluating analysis techniques for two reasons: a copy of the complete instrument can be obtained from the authors on request. 34 ajis vol 9 no. 1 september 2001 "first,...it is easier to teach them (novices) to apply a specific methodology than it is to teach new methods to people who may already be experts in developing systems...second, examining expert problem solving can be quite difficult, since experts automate their processes to the point at which they are no longer able to articulate what they are doing." furthermore, we followed the main guidelines of rein (1994) regarding the use of students as substitutes for business teams. the selected task was a real project for the student and it accounted for a significant portion of each student's grade. results the questionnaire was distributed in the last seminar of the semester (semester 1, 2000) and the students were asked to either complete it there and then, or return it within seven days. seven of the 13 students present at the seminar finally returned the questionnaire, giving a response rate of 54 percent. table 2 profiles the seven respondents. item^!?%'£tfx$?&&t^'^m&--&& working in business: <2 years 2-5 years 5-15 years > 15 years involved in process modeling: < 1 year 1-2 years 2-5 years >5 years used aris-toolset: < 1 year 1-3 years >3 years most recent version of aris-toolset used: aris-toolset 4.1 primary purpose of process modeling: organizational processes it processes my organization: public sector private sector i-̂ -̂ -̂ '̂ -jv.̂ --!̂ -̂--̂ .̂-.̂ ::̂ ^^ •••k/xtvc •->' '' dp.ercentagejresponser^v -& 85 15 0 0 57 43 0 0 100 0 0 100 57 43 100 0 table 2. profile of respondents. all participating students were post-graduate students with a maximum of two years working experience; one student had between 2 and 5 years working experience. they were involved in process modeling for a maximum of two years. all students had less than one year's experience with the conceptual aris framework as well as with the corresponding aris-toolset. four students identified the redesign of organisational processes as their main modeling purpose. three students saw their objectives rather in the redesign of it processes. fifty-seven percent (four of the seven) of the participants answered that they only used one view the process view to design business processes with the aris-toolset. these respondents indicated that the process view included a sufficient set of symbols to represent all the concepts they needed. however, three of the four (75 percent) also indicated that they used one view only because that was the way they were trained originally to do process modeling. most interestingly also, three of the four (75 percent) indicated that they created new symbols within the one view to enhance their modeling capabilities. so, while all four respondents felt that there were sufficient symbols in the one view for their modeling activities, three of the four found a need to enhance the single view with new symbols to handle all their modeling requirements. those who used more than one view (43 percent) always started with the process view. the models were usually completed with models derived from the organizational and the function views. asked which models they are likely to use for their future modeling, respondents selected all four views besides the process view with the data and the function views being the most popular. of the three respondents who used more than one view for their modeling, only one student saw a need to model real-world objects ("things"). in the context of the process view only, using an existing symbol in the process 35 ajis vol 9 no. 1 september 2001 view and changing its meaning did this modeling of things. the data and the output view were regarded as relevant views for modeling real-world objects if there was such a need. sixty-six percent (two of the three) of the respondents saw limitations in incorporating all the different business rules of the situation being modeled. similar to the limitations involving the modeling of real-world objects, the respondents saw the solution in changing the meaning of existing symbols provided in the process view. the other views were regarded as not useful to depict business rules. one hundred percent of respondents who modeled using more than one view (three out of three) wanted to describe the scope and boundary of the overall system. however, the answers regarding how to model the system's scope and boundary varied significantly. their responses included that it is not possible to describe the scope and boundary in aris, that existing symbols in the process view can be used, and that a redefinition of existing symbols in the process view is sufficient. asked whether symbols from other views might be helpful to defining the scope and boundary of a system, only one response suggested the use of symbols from the data and output views. again, one hundred percent of respondents who modeled using more than one view (three out of three) agreed that a decomposition of models into more detailed models is required. while 66 percent of all answers indicated that existing symbols provided in the process view could be used for this purpose, 33 percent answered that a decomposition is not possible. approaches to decompose models using symbols from other views included symbols from the function, the organisational, the data, and the output views. all three multi-view respondents perceived the function view as useful. furthermore, leaving the companymandated standard set of process modeling views to one side, the preferred combination of grammars (two from three respondents) was the process, function and organisational views. all but one student (six out of seven) responded that they were working in a project team while doing process modeling. seventy-one percent of all respondents used only the process view. the organizational view was used in all cases that included more than one view (two out of seven). in one case also function and data models were designed. discussion of results in the light of the derived propositions, this study gives some initial valuable feedback. green and rosemann (2000) proposed that the ontological weakness of a missing symbol for things, class or kind leads to a lack of clarity, if real-world objects are supposed to be modeled. however, only one of three multi-view respondents saw a need at all to model real-world objects. thus, this situation could be a case in which a theoretically derived ontological weakness is of only minor consequence for practical use. it was predicted that, because of the fact that the process view does not have representations for conceivable state space, lawful state space, conceivable event space, and lawful event space, a sufficient capability to identify and model all relevant business rules might not be given in aris. this proposition was supported by 66 % of the multi-view respondents. more importantly, even when all five views of aris were taken into account, respondents still maintained they could not model all necessary business rules. hence, this situation could possibly be a weakness of aris. moreover, it was proposed that, because of the missing representations for system, system composition, system environment, system structure, system decomposition, and coupling, the process view's usefulness for defining the scope and boundaries of a system is undermined. moreover, the usefulness of the process view for undertaking "good decompositions" during analysis is degraded. the fact that all multi-view respondents stressed the importance of defining precisely the scope and boundaries of a system and the opportunity to further decompose a model indicates that aris does not support this capability sufficiently. the strategies suggested by the respondents to overcome this problem vary significantly and further investigations are necessary. finally, green and rosemann (2000) proposed that the function view is completely ontologically redundant when compared to the combination of process view, data view, and either organizational view or output view. surprisingly, all three multi-view respondents saw the function view as far from useless. as the functions of the function view are also described in the context of the business processes in the process view, the function view does not seem to add value. one possible explanation for the contrary response received here could be that the user of aris appreciates redundancy as a mechanism to handle complexity in modeling. table 3 summarizes the initial insights gained on the green and rosemann (2000) propositions. 36 ajis vol 9 no. 1 september 2001 *rop6sitionv i^wm $& *& process view onlv: pi another symbol/view for thing, class/kind p2 another symbol/view for business rules p3 another symbol/view for scope and boundary of system, and to assist in decompositions more than one view in combination: p4 another symbol/view for business rules ps another symbol/view for scope and boundary of system, and to assist in decompositions p6 function view is of limited use linigi^feii^>;^^:tf ••'•> %;.<.s^:^.>'.v •.little support supported supported supported supported unsupported perhaps manage complexity ontological redundancy helps table 3. summary of insights gained on propositions. limitations, summary, and future work this is a preliminary study. it gives some initial feedback regarding the green and rosemann (2000) propositions. furthermore, this work has assisted the development of a data-gathering instrument. from the responses received to this initial version of the instrument, it is clear, inter alia, that the order of questions in section b needs to be adjusted and that further explanatory opportunities need to be provided particularly for responses regarding the use of the function view. nevertheless, the number of the respondents (7) and thenmodest experience in aris (less than one year) make it obvious that any inferences from this initial study are necessarily limited and generalizations of the results are not possible. accordingly, further investigations with professional (practitioner and academic) aris users are indispensable. this work has developed and pilot tested a survey instrument for gathering data to test propositions derived by green and rosemann (2000) from the ontological evaluation of a popular instance of an integrated process modeling framework aris. the work has provided valuable feedback on the instrument and it has suggested a number of improvements to the instrument for future empirical data gathering. furthermore, the feedback from the limited responses indicates that, the need to model real-world objects things, classes/kinds was not generally perceived. where the need was perceived, it was accomplished by using an existing symbol and changing its meaning. it was generally perceived that within the process view, and even when other views were added to the process view, severe limitations existed in the modeling of all necessary business rules. there is a strong perceived need to represent the scope and boundary of a system, and to be able to decompose the system when using integrated process modeling. within the process view, and even across the five views in combination, the respondents generally either could not model and decompose the scope and boundary of a system, or they used existing symbols and changed their meaning to meet their purposes. the proposition that, because the function view is completely ontologically redundant when compared to the combination of process, data, and either organizational or output views, the function is of limited use is generally unsupported. in further work, we intend to refine the data-gathering instrument according to the feedback we have received in this limited study. then, we will administer the refined instrument to a sample of academic and practitioner aris process modelers located both in australia and overseas. this expanded, more experienced sample will provide sufficient data to dramatically improve the inferences that may be made about the propositions and the generalization of the results. references bansler, j. p. & bodker, k. (1993) "a reappraisal of structured analysis: design in an organizational context", acm transactions on information systems, 11(2), pp. 165-193. batra, d., hoffer, j. a. and bostrom, r. p. (1990) "comparing representations with relational and eer models", communications of the acm, 33(2), pp. 126-139. bunge, m. (1977) treatise on basic philosophy: volume 3: ontology i: the furniture of the world, reidel, boston. davenport, t. h. (1993) process innovation. reengineering work through information technology, boston. floyd, c. (1986) "a comparative evaluation of system development methods", in information systems design methodologies: improving the practice, north-holland, amsterdam, pp. 19-37. green, p. (1996) an ontological analysis of isad grammars in upper case tools, unpublished phd 37 ajis vol 9 no. 1 september 2001 thesis, university of queensland, brisbane, australia. green, p. (1997) "use of information systems analysis and design (is ad) grammars in combination in upper case tools an ontological evaluation", in proceedings of the 2nd caise/ifips.l international workshop on the evaluation of modeling methods in systems analysis and design (emmsad'97). barcelona, spain, pp. 1-12. green p. and rosemann m. (2000) "integrated process modeling: an ontological evaluation", information systems, 25(2), pp. 73-87. hammer, m. (1990) "reengineering work: don't automate, obliterate", harvard business review, 68(7-8), pp. 104-112. karam, g m. and casselman, r. s. (1993) "a cataloging framework for software development methods", ieee computer, feb., pp. 34-46. rein, g l. (1994) a prolegomena on the use of student subjects in group work research, working paper 775, state university of new york. rosemann, m., and green, p. (2000) "integrating multi-perspectives into ontologies", proceedings of the international conference on information systems icis 2000, eds.: w. j. orlikowski et al., brisbane, 10-13 december 2000. rosemann, m., sedera, w. and sedera, d. (2000) "industry-oriented education in enterprise systems", in proceedings of the 11th australasian conference on information systems (acis 2000), eds. g g gable and m. vitale. brisbane, 6-8 december. scheer, a.-w. (1999) arts business process frameworks, 2nd ed., springer, berlin. vessey, i. and conger, s. (1994) "requirements specification: learning object, process, and data methodologies", communications of the acm, 37(5), pp. 102-113. wand, y. and weber, r. (1989a) "a model of control and audit procedure change in evolving data processing systems", the accounting review, lxtv(l), pp. 87-107. wand, y. and weber, r. (1989b) "an ontological evaluation of systems analysis and design methods", in information system concepts: an in-depth analysis, eds. e. d. falkenberg, p. lindgreen, north-holland, pp. 79-107. wand, y. and weber, r. (1990a) "an ontological model of an information system", ieee transactions on software engineering, 16(11), pp. 1281-1291. wand, y. and weber, r. (1990b) "mario bunge's ontology as a formal foundation for information systems concepts", in studies on mario bunge's treatise, eds. p. weingartner, g j. w. dora, rodopi. atlanta, 123149. wand, y. and weber, r. (1991) "a unified model of software and data decomposition", in proceedings of the 12th international conference on information systems (icis '91), eds. j. degross, i. benbasat, g desanctis, c.m. beath, new york, pp. 101-110. wand, y. and weber, r. (1993) "on the ontological expressiveness of information systems analysis and design grammars", journal of information systems, 3(4), pp. 217-237. wand, y. and weber, r. (1995) "on the deep structure of information systems", information systems journal, 5, pp. 203-223. weber, r. (1997) ontological foundations of information systems, coopers & lybrand accounting research methodology. monograph no. 4. melbourne. weber, r. and zhang, y. (1996) "an analytical evaluation of niam's grammar for conceptual schema diagrams", information systems journal, 6(2), pp. 147-170. acknowledgements we would bice to thank the students of the subject itn246/291 project in process engineering at queensland university of technology, brisbane, for their participation in this study. 38 ajis vol. 8 no. 2 may 2001 it research and development: should there be control? john weckert jwecken@csu.edu.au centre for applied philosophy and public ethics, charles sturt university, and the australian institute of computer ethics abstract two views are prevalent with respect to suitable topics for scientific research. one, typically held by researchers, is that any topic of research is fair game for the researcher. the result of research is knowledge, and knowledge in itself is neither good nor bad, therefore there can be no moral reasons for restricting research in any area. the proviso of course is that the research is undertaken in an appropriate manner. the other view is that science, even pure science, should serve the needs of the broader society, and therefore the society should have a say in what scientific research is conducted. some should be avoided or even forbidden because it is harmful or will most likely be put to some harmful uses, because it is useless and therefore a waste of public assets, or because other research is more important. this debate rests on a tangled web of theories, reasoning and assumptions. this paper will explore some of these underlying issues and examine the responsibilities of professional researchers, with particular reference to research in artificial intelligence. introduction nearly twenty-five years ago joseph weizenbaum made the following strong statement that there are some kinds of research in which computer scientists should not engage: there are ... two kinds of computer applications that either ought not be undertaken at all, or, if they are contemplated, should be approached with utmost caution. the first kind i would call simply obscene. these are ones whose very contemplation ought to give rise to feelings of disgust in every civilized person. the proposal i have mentioned, that an animal's visual system and brain could be coupled to computers, is an example. ... i would put all projects that propose to substitute a computer system for a human function that involves interpersonal respect, understanding, and love in the same category. ... the second kind ... is that which can easily be seen to have irreversible and not entirely foreseeable side effects. (1984, pp. 268270) more recently, bill joy, co-founder and chief scientist of sun microsystems, said this: it is most of all the power of destructive self-replication in genetics, nanotechnology, and robotics (gnr) that should give us pause. ... in trust, we have had in hand for years clear warnings of the dangers inherent in widespread knowledge of gnr technologies of the possibility of knowledge alone enabling mass destruction. ...the only realistic alternative i see is relinquishment: to limit development of the technologies that are too dangerous, by limiting our pursuit of certain kinds of knowledge. (wired, 8.04, 2000) if weizenbaum and joy are right then there is some research that computer scientists ought not do, and if they do, they can be held morally responsible for the consequences of that research. or so it would seem. there are however a number of issues which need to be sorted out before we can be confident in affirming this. one concerns the difference between pure research on the one hand and technological development on the other. the first question here is whether there is any pure research that should not be undertaken. this issue is often discussed under the heading of "forbidden knowledge"; is there any knowledge that we should not attempt to discover? a related question is whether there is any technology that should not be developed. the final question is the moral responsibility of scientists and developers of technology, and the related question of their freedom. are scientists and technologists morally responsible for the consequences of their discoveries and developments? forbidden knowledge we need to draw a tripartite distinction. the knowledge itself must be distinguished from both the method of gaining that knowledge, and from the uses to which it is put. clearly certain methods for gaining knowledge are wrong, for example those which harm people. similarly certain uses of knowledge ought to be avoided if those uses cause harm. (in particular situations it might be that there is some greater good that makes some degree of harm permissible.) the question of whether the knowledge itself ought to be forbidden is, or seems to be, quite a different matter. the knowledge is neither morally good nor bad in the way that the methods or uses might be. it is difficult to make sense of the claim that there is some knowledge that is morally bad in itself regardless of any consequences. it is certainly difficult to find any examples. it is easy to find examples of knowledge that have harmful consequences. joy himself seems to acknowledge this when he said that knowledge of nanotechnology should be limited in order to limit the technologies that would be developed from that knowleldge. if knowledge has harmful consequences should it be forbidden? not always, because many types of knowledge can be put to both beneficial and harmful uses, and we do not want to automatically rule out the beneficial. to help to clarify matters, the consequences of knowledge can be divided into two groups, physical 63 ajis vol. 8 no. 2 may 2001 and mental. physical consequences are always uses to which the knowledge is put. the knowledge that is of most concern is that which will almost certainly be used in harmful ways. there may be no way to prevent the harm without preventing the knowledge in the first place. a case could be made that that knowledge is not the fit subject of research and ought to be forbidden. mental consequences do not necessarily involve uses. some knowledge is such that simply knowing it has negative consequences on life. nicholas rescher puts it this way: some information is simply not safe for us to have not because there is something wrong with its possession in the abstract, but because it is the sort of thing we humans are not well suited to cope with. there are various things we simply ought not to know. if we did not have to live our lives amidst a fog of uncertainty about a whole range of matters that are actually of fundamental interest and importance to us, it would no longer be a human mode of existence that we would live. instead, we would become a being of another sort, perhaps angelic, perhaps machine-like, but certainly not human. (1978, p. 9) suppose that as a result of research in it it became known how to build machines which in behaviour were indistinguishable from humans, and moreover, that it was obvious that these machines were purely deterministic and without freewill. if we knew this, we would obviously have to see ourselves in a new light. would we, in our present stage of evolution, be able to cope? if the gnr technology discussed by joy develops in the manner that he fears, would we be able to continue to live happy and satisfying lives? if some knowledge has profound effects on the way we see ourselves, should it be forbidden? it seems that here, just as in the case of knowledge that almost inevitably leads to harm, a plausible argument can be made for forbidding it. it has been argued that it can be justifiable to restrict or prohibit research, but who should do this restricting or prohibiting? according to johnson (1996, pp. 197-217), while it is justifiable to prohibit knowledge (or certain kinds of research), there is nobody to do the forbidding in the way that god was able to forbid adam and eve from eating the fruit in the garden of eden. however, we must seek the potential forbidder! four possibilities are considered by marcello pera; the state, committees of experts, the scientists themselves, and the people. he rejects state intervention for a number of reasons, the most plausible being that it is a threat to individual freedom. the committee of experts is rejected because there are no experts in ethics and because it would lower the sense of responsibility of scientists, and he also rejects the scientists themselves, because they would serve their own interests. he concludes that "the supervision of science should be granted to everyone" (1989, pp. 58-72). pera's conclusion is hardly satisfactory given that we are not told how "everyone" would make the decisions. the only possibilities seem to be numerous referenda, which would be impracticable, or through the state, which he rejects. the most plausible suggestions are the state or the scientists themselves. there are problems with state intervention, but with appropriate legislation these can be minimised. there are also problems with selfregulation by the scientists, but again despite pera, this could work. with respect to the state, singer writes in general, the social consequences of attempting to put some areas of knowledge beyond reach will be worse than the consequences of gaining the knowledge, and doing our best to see that it is used responsibly (1996, p. 229) agazzi also has worries about state intervention in science: if we were to accept social control of science, meaning total planning of scientific research to make it completely goal-oriented, we would also have to accept two highly undesirably consequences. first, we would have to limit the freedom of science ... second, ... we could not avoid the question: "who will determine the goals?" (1989, pp. 215-216) a strong statement for the freedom of science from political control is supplied by david baltimore: first, the criteria determining what areas to restrain inevitably express certain sociopolitical attitudes that reflect a dominant ideology. such criteria cannot be allowed to guide scientific choices. second, attempts to restrain directions of scientific inquiry are more likely to be generally disruptive of science than to provide the desired specific restraints. (1979, p. 41) a number of arguments are offered to support these claims. first, is what baltimore calls the "error of futurism", that is the supposition that we can predict the consequences of any research accurately enough to make any sensible decisions. the second argument is a version of one of john stuart mill's. freedom of speech and expression allows the development of new ideas, increases the choices in life and generally renews and vitalises life and makes it richer. a third argument is that repression in scientific research is likely to lead to repression in other areas and so will increase fear rather than strength in society. a fourth argument is based on the unpredictability of science. even if some research is not allowed, the knowledge to which it may have led might emerge from other research quite unexpectedly. this argument is aimed at pure or basic research, and baltimore admits that the further one moves toward applications of research, the weaker these argument become. if these arguments hold for pure research, then all of the responsibility for undertaking worthwhile research rests on the scientists themselves, which is where, according to baltimore and others, it ought to rest we will return to some of these others later). it must be noted here that it does not necessarily follow that the scientists' responsibilities extend to the uses to which the knowledge is put. they create or generate the knowledge from their research but others decide how it is to be 64 ajis vol. 8 no. 2 may 2001 used. whether this separation of responsibilities is ultimately sustainable is another matter, but for the moment we will accept it. we now return to baltimore's arguments that research should not be externally controlled. his first argument, the "error of futurism" is that prediction is too unreliable to provide the basis for any restrictions on research. consider for example weizenbaum's prediction that research into speech recognition could have no useful consequences (1984, p. 271). it now appears to be an important tool in hci for users with certain disabilities. prediction is certainly fraught with danger, however we often must base our actions on predictions, on what we believe may happen, and it is not clear why the case of research should be any different. the second argument is that freedom of speech and expression allows the development of new ideas, thereby increasing lifes' choices and generally making it richer. this is true, and this form of freedom is undoubtedly an important good, but it is not the only one, and can be in conflict with others. in general we are restricted in the performance of actions which will, or are likely to, harm others. again it is unclear why research should be treated differently. the third argument is that repression in scientific research is likely to lead to repression in other areas and so will increase fear rather than strength in society. while we can agree that repression is not good, many things are restricted, or repressed in a civilised society, for example, certain research using human subjects and driving under the influence of alcohol, but these restrictions would surely reduce rather than increase fear. if probable harm is as closely associated with knowledge as suggested earlier, there is no reason why pure research should be treated differently from other aspects of life. the final argument was that, because of the unpredictability of science, the undesired knowledge might emerge unexpectedly from research other than that which was disallowed. this is true but not to the point. while it may not be possible to ensure that some undesirable knowledge will be discovered, it is almost certainly possible to reduce that probability. it is then, permissible or even obligatory on occasions to restrict or forbid research on the ground that mental or physical harm is likely to result from it. however, this should not be done lightly, because freedom in this area is important, not only for the good of science but also for the good of society. there should be a presumption in favour of freedom of research. if research is to be restricted the burden of proof should be on those who want to restrict it. however, there seems to be a conflicting intuition here. if a prima facie case can be made that some particular research will most likely cause harm, either mentally or physically, then the burden of proof should be on those who want the research carried out to demonstrate that it is safe. the situation then appears to be this. there is a presumption in favour of freedom until such time as a prima facie case is made that the research is dangerous. the burden of proof then shifts from those apposing the research to those supporting it. at that stage the research should not begin or be continued until a good case has been made that it is safe. the conclusion to this point then is that the case against the state having a role in the control of scientific research has not been made, but that such control has dangers and it should not be embraced lightly. the argument so far has focussed primarily on pure research, because that is where it is most difficult to make a case for control. however, it is not obviously much different in the case of technological development, one of the fruits of pure research. just as a scientist can say that he or she is just adding to knowledge and therefore has no responsibility for the use to which that knowledge is put, so technologists can say that they are just developing tools, and it not up to them how those tools are used. responsibility we will leave this now for the moment and turn to the responsibilities of scientists. according to agazzi again: a responsible person may be defined as one who is aware of his duties and ready to fulfil them. scientists are already accustomed to respect some duties in their profession. these are mostly connected with what we call "intellectual integrity," but scientists have been less accustomed to considering other duties related to the 'context' of their research activity. the time has come for them to do so ...(1989, p. 217) this needs closer examination. scientists are obviously responsible for the knowledge that they discover. if the knowledge has profound effects on the manner in which we see ourselves, whether this be beneficial or harmful, then the scientists must be responsible for that change. they must also take responsibility for the harmful consequences of knowledge that almost inevitably leads to harm. however, some clarification of "responsibility" is in order. hart isolates four such senses 1970, (pp 211-12): role-responsibility, causal-responsibility, liability-responsibility, and capacity-responsibility. these can be demonstrated as follows: 1. role-responsibility. the network manager is responsible for managing the computer network.. 65 ajis vol. 8 no. 2 may 2001 2. causal-responsibility. dick is responsible for the virus on this computer because he knowing copied an infected file. 3. liability-responsibility. legal dora hacked into the university's student record system and changed some grades, so she is legally liable for this action. moral in the same circumstances, dorothy is liable in the sense of being morally blameworthy. 4. capacity-responsibility. dick and dora are responsible for their respective actions in the sense that they had the ability to choose to do or not to do them. while all of these senses are important, the central one here is 3, liability-responsibility, which is closely related to accountability, the next concept that we need to consider. accountability concerns who should be praised or rewarded if some good action is performed, or blamed or punished if a bad one occurs. it is not enough however, that one performed an action to be held accountable for it. when is a person p accountable for some event e? three conditions must be satisfied: (1) p must have caused e, or knowingly allowed e to happen when he could have prevented it; (2) p must have intended to cause e or intended to allow it, or allowed it through neglect or carelessness (which implies that p did not have any intention not to cause or allow it) (duty of care); (3) p must have been free to choose to cause e or free to allow it. duty of care is a bit like accountability, but it is concerned primarily with carelessness or negligence. it is essentially the duty, in certain circumstances, to take care. one account states that 'whenever a person is so placed in relation to other people that failure to exercise care may foreseeably cause them injury, a duty of care is owed' (creyke and weeks, 1985, p 1). perhaps the most quoted statement on duty of care is this, by lord atkin in 1932: the rule that you are to love your neighbour becomes in law, you must not injure your neighbour; and the lawyer's question, who is my neighbour? receives a restricted reply. you must take reasonable care to avoid acts or omissions which you can reasonably foresee would be likely to injure your neighbour. who, then, in law is my neighbour? the answer seems to be persons who are so closely and directly affected by my act that i ought reasonably to have them in contemplation as being so affected when i am directing my mind to the acts or omissions which are called in question (balkin and davis, 1996, p. 200). the emphasis here is not so much on refraining from actions which intentionally harm others, but rather on taking care to avoid actions which are likely to unintentionally harm others. so while accountability covers more than just duty of care, this duty highlights one aspect of accountability easily overlooked. those who cause, usually indirectly and unintentionally, injury to others through negligence or carelessness, are accountable because they satisfy the conditions for accountability mentioned before. in particular, they satisfy the third alternative in condition (2). in this section we have outlined the link between professionalism and duty of care. claiming professionalism is claiming some special status in society, particularly with respect to trust, and with this status come special responsibilities. because one is responsible in a certain way, one is accountable for one's actions and for the results of those actions if one does not take proper care. professionals do have a duty of care, so to the extent that people are professionals, or to the extent that they claim to be, they have this duty, and this applies to scientists as well. conclusion should there be control of it research and development, or put another way, should any knowledge or technological development be forbidden? the answer is complex, but we have seen that there is a case for saying that there should be some restrictions. it must be acknowledged that this answer is not without dangers. control of science by the state can easily be abused, and has been. for this reason the state should only have a role in extreme circumstances. a much more satisfactory situation is one in which the scientists and developers themselves make the decisions on what to research and develop. however, they should not have the view that they cannot be held responsible for the results of their discoveries and developments. they have as professionals, it was argued, a duty of care. the situation is summed up well by barry allen: rather than an extra-scientific prohibition we should think about how scientific training undermines any nascent moral sense students may have of their responsibility for the knowledge entrusted to them. no prohibition, no forbidding of knowledge can begin to address a problem that can only be solved through changes in practice, especially in education... (1996, p. 308) the concerns of weizenbaum and joy should be taken seriously. this paper has considered the issues generally, and not specifically in relation to it. this is, a necessary first step, but only a first step. the effects, or likely 66 ajis vol. 8 no. 2 may 2001 effects, of it research and development must be examined carefully. perhaps an analogy can be drawn with the field of genetics. research and development in genetic engineering continues, but at least there is open discussion of ethical questions, particularly regarding possible and likely consequences for the lives of humans, and life in general. so far this kind of discussion has been lacking in it, but it is surely time that it began. references agazzi, e. "responsibility: the genuine ground for the regulation of free science", in w.r. shea and b. sitter (eds) scientists and their responsibilities, canton, ma: watson publishing company international, 1989, pp. 203-19. allen, barry, "forbidding knowledge", monist, 79, 1996, pp. 294-310. balkin, r.p. and davis, j.l.r. law of torts, 2nd. ed. sydney: butterworths, 1996. baltimore, david, "limiting science: a biologist's perspective", in g holton and r.s. morrison (eds) limits of scientific inquiry, new york: w.w. norton, 1979, pp. 37-45. creyke, robyn and weeks, phillipa. duty of care: law and the direct care worker. canberra: shield press, 1985. hart, h.l.a. punishment and responsibility. oxford. clarendon press, 1970. johnson, deborah, "forbidden knowledge and science as professional activity", monist, 79, 1996, pp. 197-217. joy, bill, "why the future doesn't need us", wired 8.04, 2000. pera, marcello, "should science be supervised, and if so, by whom?", in w.r. shea and b. sitter (eds) scientists and their responsibilities, canton, ma: watson publishing company international, 1989, pp. 58-72. rescher, nicholas, forbidden knowledge and other essays on the philosophy of cognition, dordrecht, holland: d. reidel, 1987. singer, peter, "ethics and the limits of scientific freedom", monist, 79,1996, pp. 218-229. weizenbaum, joseph, computer power and human reason: from judgement to calculation, harmondsworth: penguin books, 1984 (originally published san francisco, w.h. freeman 1976). 67 customer relationship management systems (crm) and information ethics in call centres 'you are the weakest link. goodbye!' helen richardson' information systems research centre, university of salford. m54wt uk h.richardson@salford.ac.uk and kate richardson manchester metropolitan university • oxford road manchester abstract this paper catalogues the rise and rise of call centres in the north west of england, uk and their use of crm systems. crm systems often imply new technologies and new ways of working. however, in this account we explore the historical development of the telegraph and work in early telephone exchanges and find the same old story. our consideration of the ethics of crm system use and some inherent contradictions are in terms of privacy, communication richness, management methods and computer ethics in an organizational context. call centres today are viewed by some as offering satisfying employment of intrinsic value, for others, they are the 'new sweatshops of the 21" century' (belt et al 2000). our interpretative field study makes a contribution to this debate. keywords: crm, information ethics, call centres, scripts. introduction we are said to be living in a new economy, a virtual age where information is power and strategic use of information technology (it) paramount. electronic media can enable organizations to deliver products and services more efficiently and effectively allowing for competitive advantage through internal networks and external customer management (loebbecke 1998). in this context crm is the new buzz phrase. crm systems are about 'allowing organizations to do a better job in contacting customers, caring for them and providing quality, service and value' (yourdon 2000). crm is increasingly about use of call centres and datamonitor predicts that the global call centre software market will reach £5.3bn by 2003 (ft 2000). crm is not without it's critics. it is popular for those 'managing' customers but can stimulate different emotions if you are the 'managed'. 'where is the r in crm it's all about customer management' (professor stone in ft 2000). there are startling contrasts in the image of crm in call centres. for the consultants and employers, crm use in call centres means knowledge intensive, strategic use of technology, flexible working and utility of all those new ways of working like flattened organizations, teamwork and empowerment. for others, they are the 'sweatshops of the 21n century' (belt et al 2000). our study is about call centres in the nw of england. ironically these are often housed in the old mills and engineering shops once foremost in the industrial revolution but long since idle. that is until the 'virtual' age and we consider whether this is indeed the new economic era, or history repeating itself. call centre work is characterised by temporary, short-term contracts, agency working and casual employment. nevertheless, we have shadowed 4 call centre workers over a period of 2 years. we have documented their tales of stress, burn out, sacking, job changes and frustration but also the community, solidarity and collective action that suggest a more rosy future than the 'dark satanic call centre mills'10 seems to offer. the ethical issues we raise lie in the inherent contradiction of crm systems and their use in call centres. the paper shall proceed as follows. firstly, we look at the technologies and the new ways of working implied. then we consider the inherent contradictions in crm systems. next we introduce our interpretative field study based on workers in a number of different call centres in the north west of england, uk. we conclude by offering two future scenarios of crm system use in call centres, one supporting the 'dark satanic mills' image and one a more ethically robust picture. crm systems and call centres telephone call centres are specialist technology-intensive offices that deliver services to customers over the telephone, replacing or complementing face-to-face interaction with the public (richardson 1994). this front line work is vital and significant to an organization. it is often strategically important work. it is rarely 9 to whom correspondence should be addressed 10 part of a poem by william blake, now a famous hymn called 'jerusalem' 166 completely routinized with a need to tailor to customer requirements and be sensitive to business changes. workers are 'on-stage' undertaking tasks that involve emotional labour. so front line workers need to be emotionally resilient and flexible, there are no buffers to protect workers from the inevitable 'spikes' of dealing with customers (frenkel et al 1999). crm system use is aimed at streamlining and shortening the key business processes that define a global organization's relationship to the markets and customers. crm is not a technology but a strategic business issue that requires technology support (ciborra 2000). however a number of it tools can be utilised e.g. automated call distribution systems (acd) using computer telephony integration in order to retrieve customer records. they can use voice recognition and response software to get initial information from the customer, integrated volume response and automated response, use of the world wide web and speech recognition. these may all further decrease the need for high value agents (frenkel et al 1999). a most highly treasured aspect of crm system use though is the opportunities afforded for electronic performance monitoring. access is gained to employee's terminals and headsets and all aspects of work, worker or team performance can be monitored and workflow measured. from a customer-relations point of view however, gutek (1995) has questioned how deep the relationships are. they are more like 'customer encounters'. our research confirms this with call time strictly monitored and pressure for quick 'closure'. we consider the inherent contradictions between crm systems and the strategically important front line work in terms of privacy, communication richness, management methods and computer ethics in an organizational context. privacy even for managers, crm raises questions of privacy with it 'strategic guerrilla's' locking in systems and knowledge, of others in the value chain. for call centre workers, issues of privacy are tied up with work practices. headsets are worn with call after call streaming in always another one waiting. crm systems means codifying intellectual capital (light 2001) with software providing 'scripts' enabling monitoring and call analysis. this metaphorically, if not physically, welds the worker and machine into a streamlined and controlled knowledge system. no aspect of a call centre workers day is unaccounted for. they are profiled, listened in to, their opening remarks are analysed as are wrap-up time and phrases used. the sales content of thenconversations, achievement of call numbers and sales targets scrutinised. their off-line frequency, intimate toilet habits are considered and discussed at teamwork supervision sessions as are perceived attitudes displayed during conversations with customers. scripts are a technique to analyse the social construction of the user of information systems (is) and a means to look at the constraints of free action (akrich 1992). even commonsense knowledge, in these terms, is classified and assigned. in our everyday world, the common reasons that we might consult a call centre or respond as a worker in a call centre is organized by a 'predetermined, stereotyped sequence of actions that define a well-known situation or script' (schank and abelson 1977). architects of crm systems, for example, want to 'predict the future world of the user and the machine' but wilson and howcroft (2000) stress the important focus of contradiction and resistance, always apparent and possible with humancomputer system interaction in an organizational context. in these terms we note the imperative to formalize, standardize and rationalize with the use of scripts and crm systems in call centres. however we also identify resistance from non-compliant users actively or otherwise failing to act out the 'script'. intense monitoring in the process of the control of machine and user is not confined to the introduction of crm systems however. julia o'connor, the president of the telephone operators department of the international brotherhood of electrical workers noted in 1920 'headset tests on an operator's position have neither morale nor service justification.. .this person constantly on the job to discover your faults... is one of the meanest elements' (norwood 1990 p39). communication richness crm rhetoric emphasises quality customer care and how this is related to effective communication. this is particularly by the gatekeepers a role delivered by call centre workers. we contend that a consideration of communication richness in an organizational context is important, that goes beyond considering information communication technologies merely as information transferors and processors, but constitute new statuses and formation of new identities (flores 1998). it is difficult to associate genuine identities in the components of virtual communities in a global business world. flores (1998) notes that our identities are vexed because all the terms with which they are given us are in the control of the community through various agencies. whether cost efficiency or customer satisfaction is esteemed and so on, all these choices affect how our personal identities are evaluated. in terms of customer relations 'one moment partnerships with customers are championed...the next.. .we read about the sad shape of companies that went too far in their partnership arrangements and had thenprofit margins crushed' (flores 1998 p354). in terms of call centre work the front line worker is a 'sacrifice of oneself (mills 1951). moreover using the controlled phrases or scripts that crm systems demand can be likened to working in the new york telephone exchanges in the early 20* century. 'you had to ask permission to 167 use anything but the set phrases. as a result the operator seemed 'strangely impersonal...an automaton, wound up always to say the same things over and over again' (norwood 1990 p36). our research implies the same old story. others have suggested that 'good' management should adopt rich media (ngwenyama and lee 1997) and how limiting electronic media is in its richness. nevertheless our research reveals how call centre staff circumvent the scripts and controlled phrases and rich human communication and problem solving makes the job more satisfying. this is despite facing frequent and sustained verbal abuse from customers. management methods crm system use in call centres goes hand in hand with the adoption of teamwork and empowerment management methods. again contradictions are evident with crm systems designed to dis-empower sales staff encoding their customer knowledge and so on (light 2001). our research showed how sales roles, for example, were increasingly regimented, with initiative, discretion and enterprise removed as far as possible. to start the process of ethical decision making, some have suggested that 'when it comes to virtual organizations, we must first have a set of commonly accepted values and norms' (gotterbam 1999). we look at the realities of using teamwork and empowerment techniques in call centres and suggest that the gap is too wide between call centre workers and management for this to be achieved. the remoteness of management control through the it tools may seem like technology is controlling rather than it being a human activity. it has to be noted however that there are indeed choices in management methods. teamwork sessions in our study included dissemination of orders from the top down and attempts to exert team pressure on the 'weakest link'. often these measures backfired with general community solidarity of call centre workers overriding team identity. furthermore, this is not a new way of working but these practices are part of the process and imperative of capital accumulation. mumford (1999) in a critique of business process re-engineering talks about the 'pressures for rapid gains in productivity without concern for employee gains'. moreover, is departments need to 'justify their overall profitability in the face of threats to outsource and also doubts about the financial returns from investment in it' (robinson and richardson 1999). the development and use of crm systems in call centres needs to be seen in this context. computer ethics in an organizational context ethical issues of workplace privacy generally centre around concepts of personal autonomy and dignity infringed by these practices, yet some would suggest workplace privacy should extend further, worthy of becoming a matter of organizational justice (introna 1999). it is relevant here to look at the contradictions of call centres and crm systems. they are exploitative and work against communication richness. they increase bureaucracy yet supposedly embrace empowerment. they are about control of communication and yet to provide quality to customers, need to enhance the "company ambassadorial" role of call centre workers (frenkel et al 1999). computer ethics and its adherence to technological determinism and liberalism are problematic for understanding issues of equality and participation (adam 1999), and moreover power structures play a large part in the making and accepting of knowledge (adam and richardson 2001). in this context it is our contention that a debate on the ethical issues of crm systems, information management and call centres is long overdue. our field study our area of study has the largest number of call centre jobs in the ukover 72,000. pay is 59% of average uk earnings with the mid-point average being £12,000 a year (tuc 2001). there are many types of call centres. these include dotcoms and public service centres e.g. help desks. they can involve business to business intranets with internal distribution of jobs or outsourced services e.g. free phone lines offering multi organization services. they can be portal models that rely on attracting maximum users to a web site to drive advertising, negotiate bulk purchasing deals or charge 3rd parties to access customers. research methods our research rather than taking a company or case study model representing examples from each call centre type, instead aimed to follow the fortunes of 4 call centre workers over 2 years as an interpretative and 'mobile' field study. we held focus group sessions and conducted in-depth individual interviews. here we show life on the front line, present some stories and conclude on contrasting futures. 168 life on the 'front line' 'telephone operating was always low paying and became an increasingly regimented and alienating form of labor' (norwood 1990 p6). far from a contemporary observance, this statement refers to the early 20 century. our research reveals the same old story. blauner (1964) has identified several individual task related freedoms that together comprise control over the immediate work process, namely control over the pace of work, freedom from pressure, freedom of physical movement and ability to control the quantity and quality of production and techniques of work. when technology and work organization do not permit active intervention in these areas then the alienating tendencies of modem industry that makes the worker 'a responding object, an instrument of the production process, are carried to their furthest extremes' (norwood 1990 p63). such is the experience of alienation facing the call centre workers in our study. we have chosen names from a uk pop group s club 7 to conceal the identities of our interviewees. hannah's story the average length of time spent working in a call centre is 18 months (ft feb. 20th 2001). hannah is an exception to this. in her late 40's, hannah has been working in call centres for nearly 10 years. most of this has been for a telecommunications organization in her hometown. she became an example of a highly valued agent, with intensive knowledge of the area and customers (frenkel et al 1999) and could give a highly personalized and rich service. redundancy followed as the organization transferred to a large, impersonal and centralized service utilizing crm systems to the full. later hannah joined this call centre as an agency worker. she was alarmed at the change in working conditions finding teamwork and supervision intimidating and the work pressurized and impersonal. there is a myth that crm brings about greater speed and efficiency of work. hannah's case belies this. in her previous job with the firm, hannah could give out 5 numbers to customers and on average her calls lasted 15 seconds. in the new call centre, customers were only permitted 2 numbers per call and the average length of call is 30 seconds. hannah reported that calls were dissected into 4 parts. #1 a welcome and introduction; #2 offering products and services; #3 recap call; #4 positive close. imposing exact phraseology in terms of 'scripts' on the workers enables greater monitoring and analysis by crm systems. hannah faced difficulties with management for failing to use the exact words "and to recap". supervisors made hannah put labels on her monitor with the words on and she had to tick a 'star chart' when she used the words. this is a technique also used by psychologists for behaviour modification in young children. her calls were monitored closely and listened in to. saturdays were colloquially called the 'sacking day'. it was easier and less obvious for management to dismiss someone on a saturday. one saturday, hannah was sacked for again using words similar to, but not actually "and to recap". bradley to contrast hannah, an experienced call centre worker with a long history of this type of employment, bradley is a young, gay man who is a 'dj' at night and dreams of travel and going back to college. bradley works from call centre agencies but is intolerant of bullying and intimidation from management. if he doesn't like the work or atmosphere, he walks down the road to the next call centre. hence in 18 months of our 'shadowing', bradley's call centre experience was as follows: large telecommunications firm 1 year; financial sector 3 months; telecommunications services 2 months; retail catalogue 6 weeks; ticket sales 3 weeks; credit card company 1 day. he mainly disliked abusive customers and the constant remote monitoring and listening in. bradley's comments on privacy: 'you can't be early, can't be late, can't go for a sh*t'. john john is in his early 40's and was a supervisor. he was trying hard to build up experience to become a manager but despite this was supportive to staff, 'he drove me home when i freaked one day' said jo. he had respect from colleagues. one day his line manager told him 'you're alright for a queer'. john said 'call me a f***ing queer again and i'll punch your lights out.' management tried to discipline him, but support from the staff meant they failed. jo jo is in her 30's and during the months of this study, worked in two call centres dealing with the public and distributing work to engineers. jo also studied part-time for a postgraduate diploma in business and it and would get annoyed with the 'management clap-trap' on the course that was hot on it strategy for competitive advantage but cool on understanding the realities for low paid call centre workers. after 18 months, jo suffered 169 stress and burn-out driven by the acd systems, lack of control, remote listening, abusive customers and low pay. 'i hated being happy, happy, happy all the time the voice with a smile'. call centre workers today often face periods when they can't stop crying and exhibit other symptoms of stress. jo remembers with amusement all the statistics that were kept and all the codes they had to use, 'there was even a code for running to the restroom in tears because of stress'. jo loved the part of the job that involved dealing with people and sorting problems out. despite the scripts and regulated phraseology, jo would refer customers to appropriate agencies that could help with debt and so on. one distressed customer had recently fallen downstairs but was dealing with an upsetting letter relating to her telephone being cut off. as is necessary with customer service occupations, in contradiction to increasing regulation of all aspects of the work that crm systems demand, workers also have to have discretion and freedom to act at times. workers cannot be completely manipulated to adhere to configured scripts. jo stopped the process that should have resulted in cutting the phone line of the customer and instead rang a relative and ambulance. for this she gained a company 'heroism' award. community and solidarity a recent trades union congress (tuc) report catalogues problems in call centres (tuc 2001). these included workers victimised if they join a union, lack of toilet and other breaks, increased health problems of 'acoustic shock' that are freak sound bursts on telephone headsets, leaving victims in severe pain with loss of short term memory. our stories were not all grim however. a lot of solidarity and community feeling arose among the call centre workers in our study. someone was burgled and they had a collection to replace items. cakes were baked for birthdays. one bonus for surpassing sales levels was the issuing of supermarket vouchers. bradley won £400 worth of vouchers in a national sales competition, although he admitted privately that this had been achieved by adding items to customer bills and waiting for complaints later! if workers were hard up for cash, these vouchers were exchanged for a cash value and on this basis bradley managed to buy a plane ticket to jamaica. management methods often intimidated, yet solidarity helped in many cases. in one team, a number moved from agency work to permanent contracts but management wanted to impose poorer working conditions. the whole team took off their headsets and refused to work. in a short time, old conditions were restored. workers would 'look out' for each other. workers would use 'non-verbal communication' to indicate to others that they were dealing with an abusive customer or difficult call. friends would then also listen in to the call to give help and solidarity and lessen the stress. this showing of solidarity is again not confined to call centre work in the 21st century. tom standage has written about the development of the 'victorian internet' namely the telegraph system in the 19th century: 'during quiet periods on-line interaction really got going with stories, gossip, jokes etc circulating along the wires. on one occasion an out-of-hours meeting was held between boston and maine attended by hundreds of operators in 33 offices along 700 miles. each speaker tapped out their views in morse code and all the offices received the remarks at the same time 'annihilating space and time and bringing together the different parties....as though they were in the same room" (standage 1998 pi25). during the time of our study, there was a major official strike against bullying in one of the organizations. this precipitated the tuc and call centre organizations to consider good practice and a more ethical future (tuc 2001). call centres alternative futures frenkel et al (1999) identifies two contrasting images of call centres work. workers are regimented, routinized, and viewed as replaceable resources. constant surveillance, high performance standards, programmed learning and strict discipline is the norm. it is emotional labour with inadequate compensation. it is coercive and enervating work. alternatively, the work is intrinsically rewarding and human resources departments provides skill and knowledge enhancing competencies that help develop employee's careers. the work is enabling, competency enhancing and socially satisfying. tendencies in service work suggest that the work will become more complex, more product variety will lead to policy and procedural changes so there will be a need for better access to information and conversion to useable knowledge. greater emphasis on sales will require better social skills. 'smart technology ...could result in the realisation of the empowered front line workforce' (frenkel et al p272). the other response could be that service work is outsourced and 'dumbed down' with a low knowledge, low skill strategy pursued. reliance on technology in this scenario will lead to increased standardisation. this suggests increasing atomisation and high labour turnover (frenkel et al 1999). indeed labour turnover today is a problem. organizations publicly admit to 20/30% but privately admit it's double that figure (tuc 2001). the tuc suggest best and ethical practice is unionised workplaces, allowing for flexibility to fit in with family lives and zero tolerance of harassment and bullying. there should be training programmes and skill development indicating valuing of employees. which is the future scenario? let's leave the last words to hannah who adamantly exclaimed 'i'll never work in a call centre again'. 170 references adam, a. (1999) computer ethics in a different voice, in the proceedings of the 1st critical management conference. manchester july 1999 adam, a., richardson, h. (2001) "feminist philosophy and information systems." information systems frontiers, 3 (2) akrich m (1992)the de-scription of technical objects in: bijker w.e. and law j. (eds) shaping technology/building society mit press belt, v., richardson r., webster, j. (2000) "women's work in the information economy: the case of telephone call centres." information communication and society 3:3 2000 blauner, r. (1964) alienation and freedom, cited in norwood, s.h. (1990) labor's flaming youth telephone operators and worker militancy 1878-1923. university of illinois press. 1990 ciborra, c. u. and associates (eds) (2000) from control to drift. the dynamics of corporate information infrastructures. oxford university press. ft. (2000) understanding crm. ft financial times. spring 2000 www.ft.com/crm/ flores, f. (1998) "information technology and the institution of identity." information technology and people vol. 11 no 4 1998 frenkel, s., korczynski, m., shire, k., tarn, m. (1999) on the front line: organization of work in the information economy. (cornel university press) gutek, b.a. (1995) the dynamics of service cited in frenkel, s., korczynski, m., shire, k., tarn, m. (1999) on the front line: organization of work in the information economy. (cornel university press) gotterbam, d. (1999) an ethical decision support tool in the proceedings of the 4th ethicomp conference 1999 introna, l. (1999) privacy, autonomy and workplace surveillance in the proceedings of the 4tb ethicomp conference 1999 light, b. (2001) a review of the issues associated with customer relationship management systems in the proceedings of the 9th european conference on information systems, 27-29 june, bled, slovenia. 2001. loebbecke, c. (1998) economics of content provision on the internet in the proceedings of the 8tb annual bit conference. manchester november 1999 mills, c. w. (1951) white collar, cited in frenkel, s., korczynski, m., shire, k., tarn, m. (1999) on the front line: organization of work in the information economy. (cornel university press) mumford, e (1999) routinisation, re-engineering and socio-technical design: changing ideas on the organisation of work in currie, w.l and galliers b. (eds) rethinking management information systems oxford university press ngwenyama, o.k., and lee, a.s, (1997) "communication richness in e-mail" misq vol. 21 no 2 june 1997 norwood, s.h. (1990) labor's flaming youth telephone operators and worker militancy 1878-1923. university of illinois press. 1990 robinson b., richardson h. (1999) the crisis of the information system discipline: a vygotskyan analysis in the proceedings of the ist critical management conference. manchester july 1999 schank,r., abelson r. (1977) scripts, plans and knowledge in johnson-laird p., wason p. (eds) thinking: readings hi cognitive science cambridge university press cited in suchman l.a. (1987) plans and situated actions the problem of human machine communication cambridge university press. standage, t. (1998) the victorian internet weidenfield and nicholson tuc (2001) it's your call. tuc call centre workers campaign. trades union congress. february 2001 wilson m., howcroft d. (2000) the role of gender in user resistance and is failure in baskerville r., stage j., and degross j. (eds) organizational and social perspectives on information technology, kluwer academic publishers, boston, mass, 453-472 yourdon, e. (2000) "introduction to the special issue on crm." cutter journal vol. 13 no 10 october 2000 171 ajis vol. 8 no. 1 september 2000 adding conflict resolution features to a query language for database federations19 kai-uwe saltier1 stefan conrad2 guntersaake1 'department of computer science, university of magdeburg, p.o. box 4120, d-39016 magdeburg, germany department of computer science, university of munich, oettingenstr. 67, d-80538 miinchen, germany abstract a main problem of data integration is the treatment of conflicts caused by different modeling of realworld entities, different data models or simply by different representations of one and the same object. during the integration phase these conflicts have to be identified and resolved as part of the mapping between local and global schemata. therefore, conflict resolution affects the definition of the integrated view as well as query transformation and evaluation, hi this paper we present a sql extension for defining and querying database federations. this language addresses in particular the resolution of integration conflicts by providing mechanisms for mapping attributes, restructuring relations as well as extended integration operations. finally, the application of these resolution strategies is briefly explained by presenting a simple conflict resolution method. introduction nowadays integrating heterogeneous data sources is a significant challenge to the database community. the availability of numerous sources, ranging from legacy systems and enterprise databases to public internet sources, increases the demand for tools and techniques integrating, condensing and abstracting data. recently, several integration approaches were developed, particularly multidatabase systems (litwin & abdellatif (1986), bright et al. (1992)), mediators (wiederhold (1992)) and federated database systems (sheth & larson (1990)). more or less these approaches are based on the idea of providing an integrated view on the sources. defining this view is subject of schema integration. during the integration process the individual schemata are analyzed, the global schema is defined and finally the mapping between local and global schema is described. the mapping information provides the base for query processing. a global query is decomposed according to the mapping and translated into subqueries for the individual sources. after local evaluation of the sub-queries, the sub-results have to be combined to the global result. independent from the direction of the schema integration process, bottom-up integration of all the relevant data vs. top-down integrating data for a given goal (hasselbring (1999)), the designer has to resolve conflicts resulting from the heterogeneity of the participating data sources. examples of conflicts are among others different identifiers for the same fact (e.g. entities or attributes), using different modeling concepts for representing real-world entities or conflicts, arising from overlapping of data. resolving these conflicts is an important step in defining the mapping between local and global schema and therefore affects the query processing. in this paper we present the query language fraql, a lightweight sql extension for defining integrated object-relational schemata as well as formulating queries on them. this language is implemented as part of a query system for federated databases. the main contribution is the treatment of integration conflicts. the paper is organized as follows: after a brief survey on related work in the following section, we introduce the query language and the underlying software architecture. next, the resolution of conflicts with the help of the fraql language features is discussed. then, we sketch basic principles of a method which we are currently developing for resolving conflicts as part of data integation. finally, we conclude the paper and outline future work. related work the more general problem of schema integration is addressed by several approaches (batini et al. (1986), pitoura et al. (1995)). for describing conflicts arising in the integration phase various classifications were developed, e.g. in kim & seo (1991), saltor et al. (1993), spaccapietra et al. (1992). data models and query languages supporting the integration of heterogeneous sources are particularly multidatabase languages like msql (grant et al. (1993)), sql/m (kelley et al. (1995)) and 19 this work was supported in part by the german research council (dfg): for 345/1-1. 116 ajis vol. 8 no. 1 september 2000 schemasql (lakshmanan et al. (1996)). examples of system implementations are federated database systems like iro-db (gardarin et al. (1996)), pegasus (ahmed et al. (1991)) or ibm datajoiner (venkataraman & zhang (1998)) as well as mediator-based systems like tsimmis (garcia-molina et al. (1997)) or information manifold (levy et al. (1996)). msql provides basic features for accessing schema labels and converting them into data values. sql/m addresses mainly description conflicts by providing mechanisms for scaling and unit transformation. more advanced conflict resolution is addressed for example by the restructuring techniques proposed in schemasql, which support the specification of relations with data dependent output schemata. pegasus uses a functional object-oriented data manipulation language called hosql with nonprocedural features, datajoiner is based on db2 and therefore provides essentially standard sql features for conflict resolution. in mediator systems such as tsimmis the mediator is specified by a set of rules. each rule maps a set of source objects into a virtual mediator object. in this way, conflicts are resolved by defining appropriated rules. the special problem of combining objects from different sources (object fusion) in mediators is addressed in papakonstantinou et al. (1996). furthermore, structural conflicts and resolution strategies are discussed in detail in kim et al. (1995). techniques for managing schematic heterogeneity (meta conflicts) based on schemasql features are presented in miller (1998). resolving description conflicts by using a rule-based data conversion language is described in cluet et al. (1998), milo & zohar (1998) presents a schema-based data translation solution. in kent (1991) solving domain and schema mismatch problems with an objectoriented database language is discussed. in lim et al. (1999) an approach is proposed, where the origin of integrated data is included as an additional tuple attribute in order to improve the interpretation of global data. another approach, presented in sciore et al. (1994), introduces the notion of semantic values enabling the interoperability of heterogeneous sources by representing context information. fraql: an overview the objective of the fraql development is the investigation of techniques for query processing in loosely-coupled database federations. in this context several problems arise, which make query processing and optimization more difficult. these include: heterogeneity of data, missing or uncertain statistical information about data distribution and access paths, the limited query capabilities as well as non-predictable responses of the sources (ives et al. (1999)). in the following we will focus on the aspect of resolving conflicts caused by heterogeneity of data. fraql is a query language for object-relational database federations. it extends sql by features for defining federations, accessing meta-data in queries, restructuring query results, and resolving integration conflicts. this is comparable with other multidatabase languages like msql or schemasql, but in contrast to these fraql is extensible by user-defined data types and functions and it supports dynamic integration of new sources. with this features fraql could form the base for advanced data integration and fusion tasks (sattler & saake (1999)). in this context, fraql is not intended as an end user language, but an intermediate language for specifying the integrated views. therefore, users can query the global integrated relations by means of usual sql operations without knowledge of the fraql extensions. in fraql a federation is a set of databases consisting of relations. a database can be provided by a full-featured dbms or even by a web source encapsulated by a wrapper (roth & schwarz (1997), sattler & hoding (1999)). this wrapper has to implement the query mechanisms which are not supported directly by the source. fraql is based on an object-relational data model: it supports the definition of object types and object tables derived from types. using object-relational features simplifies the integration of post-relational data sources (e.g. odbms-based sources or xml datastores) and provides more advanced modeling concepts for schema definition. object types describe the structure of objects as sets of attributes and their domains. types can be organized in a specialization hierarchy. object tables represent global virtual relations of the federation, i.e. data from the sources are not materialized, except for caching purposes in order to speed up query evaluation. here we distinguish between import and integration relations. an import relation is a projection of a local relation of a data source. the import relation is defined by specifying the origin (the identifier of the source) and, if required, a mapping between local and global attributes. create type type_name [ under type_names ] ( attrib_definitions 117 ajis vol. 8 no. 1 september 2000 create table global_name of type_name as import from source.local_name [ mapping_definitions ]; a data source is specified by the required database adapter and additional connection information: register source source_name at >dsn=db;\jid=user,pwd=password' using 'adaptor_name'; an integration relation is a view on other global relations combined by using operators like union, 6join and outer join. in addition, the standard sql operations selection and projection are provided, too. an integration relation is defined as follows, where the term table_expression denotes a sql view definition with extensions explained later. create table global_name of type_name as table_expression; furthermore, fraql supports user-defined functions, which are stored in the database of the federation layer (i.e., in the query processing server) and are callable in queries. these functions are implemented in java and registered in the query system. in order to be able to rewrite queries during optimization, two functions can be specified as inverse to each other. similar to sql, the union and join operations can be refined by an on clause specifying the comparison attributes (for the union operator) resp. the comparison expression (for join operators), e.g.: table 1 union table2 on attrl, attr2 tablel join table2 on tablel.attrl = table2.attr2 both the union and join operators can be applied with an additional reconciled by clause which denotes a user-defined function for conflict resolution (see the following section): tablel join table2 on attrl = attr2 reconciled by func restructuring of relations is implemented in a way inspired by schemasql. variables of a query can not only be bound to relations as tuple variables, but also to meta-data, like the set of attributes of a relation or the set of relations of a schema. but in contrast to schemasql, where meta-data access in queries is implemented as a language extension, in our approach the schema catalog is used. so, the catalog relation catalog.columns contains information about attributes of all global relations, whereas the relation catalog.tables describes the global relations. naturally, any global user relation with information about other relations can be used as meta-data source, too. as an extension to standard sql, attributes of tuple variables in queries can be obtained during evaluation. this means, while in sql names of attributes and relations are constants, in fraql they can be constructed from current values of other tuple attributes. this variable substitution is written in the notation $var and can appear everywhere in a query, where names of attributes or relations are expected. for example, the expression tbll.$(tbl2.col) means the attribute of the current tuple of relation tbll, whose name is obtained from the current value of tb!2.col. in the same way, a relation in the from clause or a query could be dynamically determined. the following query selects isbn and title information from all relations implementing the object type book. so it is equivalent to a union of all these relations. select t2.isbn, t2.title from catalog.tables tl, $(tl.table_name) t2 where tl .type_name = 'book'; this technique enables a flexible transformation of schemata by view definition. but we have to take into consideration the effects on query optimization. therefore, static optimization techniques, which create a complete query plan before beginning the evaluation, are inappropriated for queries containing variable substitutions. better approaches should support runtime re-optimization at certain points of query processing (graefe & ward (1989), kabra & dewitt (1998)). fraql is implemented as part of a federated query system. this system consists of the following main components: the query parser, the decomposer and the global optimizer, the query evaluator, the java vm for evaluating user-defined functions, and the catalog. the adapter layer contains the management component as well as the individual adapters providing a uniform access interface to the data sources. the interface to the adapters and the query processor itself are implemented using corba. therefore, adapters can be plugged into the system at runtime. on top of the query interface we have developed a jdbc driver and an interactive query tool. conflict resolution in fraql in fraql an integrated schema is defined only by global relations which are views on local relations. therefore, data integration and conflict resolution are parts of query processing, particularly view 118 ajis vol. 8 no. 1 september 2000 decomposition, query transformation and result composition. so the main issues for conflict resolution in fraql are: • renaming of attributes as well as transformation of attribute values, • integration operations like union and joins, and • restructuring of relations by combining data and meta-data. in the following we discuss the application of these techniques for description conflicts, semantic conflicts, and structural conflicts. these are three of the four conflict classes introduced in spaccapietra et al. (1992). we close this section by briefly considering the remaining class of heterogeneity conflicts. description conflicts first, we consider description conflicts. this kind of conflicts occurs, when the same real-word entity is modeled with different properties. in fraql we eliminate these differences by defining an import relation. beginning with the object type describing the desired global properties, we specify how the local relation implements this type. here the following rules apply to this mapping: 1. each local attribute corresponding to an attribute defined by the global object type in terms of identifier and type becomes an attribute of the global relation. 2. the notation g_name is l_name means renaming the local attribute to g_name. this requires type compatibility. 3. the notation g_name is func(l_name) defines that the global attribute value is calculated by using the user-defined function func on the local attribute value. 4. the definition g_name is @tbl(l_name, src, dest, default) means that the database table tbl is used for mapping the values from the local attribute l_name. this value of the global attribute is obtained by looking for the value of attribute l_name in column src and retrieving the corresponding value of column dest. the field default denotes a default value, either as literal or as local attribute, which is assigned to the global attribute, if the value of l_name is not found in the table. in fact, this kind of attribute mapping is evaluated by a left outer join, whereas the null value is replaced by the default value default. 5. the remaining local attributes are suppressed. 6. to all attributes of the global relation without a mapping the null value is assigned. the next example demonstrates these concepts. first we define an object type book: create type book ( isbn varchar (20), title varchar( 100), price float); furthermore we assume a local relation buch from the source src, which differs from the object type by the attribute names and the currency of the price attribute (dm instead of dollar). therefore, a function dm2dollar for converting the value is required which could be implemented by accessing a database table. now we can define the global relation german_books as follows: create table german_books of book as import from src.buch ( title is titel, price is dm2dollar (preis)); based on these definitions the global query select title, isbn from german_books where price < 50; is transformable into a local query. beside renaming attributes this requires a transformation of the selection expression. because user-defined functions are available only at the federation layer and not in the source itself, the selection operation has to be performed at global level or for a constant expression the expression has to be pre-computed by applying the inverse function. for the given example this results in the following transformed query: select titel, isbn from buch where preis < 93.60; of course, this requires that a function, e.g. dollar2dm, is being registered in the global query system as the inverse of dm2dollar. semantic conflicts semantic conflicts arise, when the relations overlap, which have to be integrated, i.e. there are tuples from both relations representing the same real-world object. first of all, this kind of conflicts can be 119 ajis vol. 8 no. 1 september 2000 resolved in fraql by applying the standard sql union operation. however, the following problems remain: • we have to decide, when two tuples from different relations represent the same real-world object (tuple identity). • how to process tuples representing the same object but containing different values for the same attribute (data conflicts) ? isbn isbn author author title title 382578 382578 williams, t. tad williams otherland otherland 326523 276830 gibson, w. stanislaw lem idoru solaris (a) books 1 (b)books2 fig. 1. tuple identity conflicts the problem of tuple identity is solved by specifying the attributes relevant for deciding equivalence, as illustrated in fig. 1: create table books of book as books 1 union books2 on isbn; the attributes have to identify the tuples of each relation uniquely, e.g. by using the primary key, in order to avoid duplicates. data conflicts are resolved in fraql with the help of user-defined reconciliation functions. a reconciliation function is called for each pair of tuples fulfilling the comparison condition. the affected tuples are passed as arguments to the function, the resulting tuple is inserted into the global relation. isbn isbn isbn number number number 3324524 3324524 3324524 2 1 3 6710767 1267894 1267894 2 2 2 6710767 2 (a) bstorel (b) bstore2 (c) stores fig. 2. resolving data conflicts in the example from fig. 2 we want to integrate two overlapping relations bstorel and bstore2. both relations contain an attribute number. in the integrated relation stores this attribute should represent the sum of both values. therefore we define this relation as follows: create table stores of book as bstorel union bstore2 on isbn reconciled by book_resolve; the reconciliation function is implemented as a stored function in java and registered in the query system: ii java public static book resolver (book bl, book b2) { book b = new book (); 120 ajis vol. 8 no. 1 september 2000 b.isbn = bl.isbn; b.number = b 1 .number + b2.number; return b; // fraql create function book_resolve (book, book) returns book external 'books.resolver'; in this example the fraql type book is mapped to a java type. however, in the current implementation we use generic objects for representing tuples. structural conflicts representing a real-world aspect by different modeling concepts results in structural conflicts during integration. depending on the variety of the data models several kinds of conflicts can occur. in the following we focus only on two special kinds: partitioning and meta conflicts. the resolution of other structural conflicts, particularly for the relational data model, is discussed for example in kim et al. (1995). partitioning occurs, when the relations which have to be integrated represent different aspects of the global relation, but still contain semantically equivalent attributes. this kind of conflicts is usually resolved by applying a 6-join or outer join operation. like for the union operator a reconciliation function can be specified for resolving data conflicts. meta conflicts arise, when a concept is represented as data object in one schema, whereas it is modeled as schema object (attribute or relation) in another one. id id id title title title kind 382660 017062 382660 databases computer databases book 361556 000102 java cacm 017062 computer journal (a) dbl.publication (b) db2.book (c) db2.journal fig. 3. meta conflicts: relation vs. attribute in the example in fig. 3 the database dbl stores books and journals in a single relation publication, where each tuple contains a discriminating attribute kind with possible values "book" and "journal". in database db2 books and journals are stored in separate relations. integrating both databases requires the adaptation of the relations from db2 to the structure of publication. a straightforward solution would be a union of the relations book and journal with a constant value for the attribute kind: create table publication of publ_type as select id, title, "book1 as kind from db2.book union select id, tide, 'journal' as kind from db2.journal; in a more flexible approach the names of the relations in the from clause are determined from the schema catalog by using variable substitution: create table publication of publ_type as select t2.id, t2.title, tl.table_name as kind from catalog.tables tl, $(tl.table_name) t2 where tl .schema = 'db2'; in this way no modifications of the global relation are required, when new relations (e.g. representing another publication type like reports) have to be added. 121 ajis vol. 8 no. 1 september 2000 without the language extension for variable substitution the above query could be evaluated by a dynamic sql program as illustrated in the following pseudo code: exec sql create table publication (proj_list); exec sql select name from catalog.tables where schema = 'db2'; foreach tuple t qstr := 'insert into publication select id, title,' || tname ||' from' || tname; exec sql prepare query from :qstr; exec sql execute query; end in contrast, in fraql this query is processed directly by the query evaluator. isbn isbn price meyers supplier jones 38934237 38934237 59.0 59.0 meyers 48.0 38934237 22660513 48.0 27.0 jones 29.0 (a)dbl.book (b)db2.book fig. 4. meta conflicts: attributes vs. data values in the second example (fig. 4) the relation of database dbl contains books with their prices and suppliers. for each supplier a book is represented by a separate tuple. in contrast, the relation of database db2 contains the supplier prices as separate attributes. by using the schema catalog and variable substitution we are able to transform the relation of db2 according to the relation of dbl. select b.isbn, b.$(c.column_name), c.column_name from db2.book b, catalog.columns c where a.table = "book1 and c.column_name o 'isbn'; heterogeneity conflicts this kind of conflicts occurs if the data of the local data sources is represented in different data models. in fraql these conflicts are resolved mainly by the adapters which map the modeling concepts and hide system-dependent differences (e.g. sql dialects). a method for conflict resolution based on the features discussed above we can develop a simple method for conflict resolution as part of data integration. it is not intended as a replacement for schema integration, rather it complements existing integration methods. in the following we sketch the basic ideas of such method. we propose three steps reflecting the main concepts of fraql: object types, import relations and integration relations: 1. first, the global types have to be defined, either top-down or bottom-up by integrating the local types. for that 'classical' schema integration methods should be applied. 2. next, we have to define the import relations. here we resolve description conflicts by specifying the mappings of attributes. 3. in the last step we try to integrate import relations representing semantically quivalent realworld entities. in this context we have to consider two subtasks: • resolving structural conflicts using standard sql operations like projection, selection, renaming, the join operation with reconciliation functions as well as the restructuring mechanisms provided by fraql. 122 ajis vol. 8 no. 1 september 2000 • resolving semantic conflicts by applying the union operator in conjunction with reconciliation functions. a stepwise integration simplifies conflict resolution, because only one kind of conflict has to be considered at once. however, integration is often an iterative process, therefore conflict resolution strategies should be applied and refined in every iteration. in practice a lot of existing conflicts are not obvious from only considering the schemata of data sources to be integrated. therefore, classical schema integration techniques must be enhanced by additional means to detect such conflicts. due to the fact that data from the data sources is available at integration time we can directly use their data to validate decisions made in the integration process. in addition, conflicts can be detected which are only visible on the data layer. therefore, the stepwise integration of data sources by means of a language like fraql seems to be essential for obtaining an integration result of high quality. in particular, we expect the following kinds of contributions a stepwise integration using fraql can make to database integration: • following the (preliminary) results of schema integration fraql can be used to define corresponding queries (or views) describing the integrated relations. by applying such queries on real data from the data sources the fraql system can check whether there are additional conflicts on the data layer, e.g. not identical values for the same property of corresponding data objects in different data sources. by presenting all non-matching values (or, in case the number of conflicting values is too large, by presenting selected examples) the system can give important hints for resolving such conflicts of which we were not aware when integrating the schemata. • furthermore, we strive for enhancing the fraql system in such a way that the system can already propose possible ways for resolving such conflicts detected when computing a query (or view) describing an integrated relation. if the system proposes a conflict resolution it should directly generate a reconciliation function which can be used for that query (in the resolved by clause). of course, such an automated generating of reconciliation functions is only possible in certain, restricted cases, e.g. if the conflict is due to a different scaling of numerical values in the data sources. assuming that a large number of conflicts occuring in practice is of a kind for which a conflict detection and automated generation of conflict resolution functions is possible, this is an important and valueable help in carrying out an integration. by means of this functionality we aim at supporting an integration method based on an integration by example principle. in particular, the third issue, i.e. giving the system examples of conflicting values for automated conflict resolution, will be an essential part in this method. conclusion resolving conflicts is an important step in integrating heterogeneous data sources. here, differences at schema level as well as problems caused by different data representations have to be eliminated. in this paper we have discussed conflict resolution techniques as part of the query language fraql. integration conflicts are resolved at global level by defining views on the imported relations. in addition to the well-known sql operations our language provides more advanced mechanisms like renaming, value transformation as well as structural transformation. by using these mechanisms most of the integration conflicts are resolvable. regarding conflict resolution we see our approach as an extension to datajoiner and schemasql. datajoiner's query language sql permits accessing and querying foreign databases without further resolution mechanisms. schemasql contributes among other things features for restructuring relations and with it for resolving meta conflicts and fraql extends these by additional resolution techniques for description and structural conflicts. fraql is currently being implemented in c++ as part of a query processing system for loosely-coupled database federations. furthermore, we have implemented adaptors to the oracle and mysql dbms. apart from the jdbc driver and the query tool mentioned before, a graphical design und query workbench for interactive definition of integrated schemata is under development. moreover, in the future we will examine dynamic optimization techniques for interleaving query planning and execution. 123 aj1s vol. 8 no. 1 september 2000 references ahmed, r., de smedt, p., du, w., kent, w., ketabchi, m.a., litwin, w., rafii, a. & shan, m.-c. (1991) "the pegasus heterogeneous multidatabase system", ieee computer, vol. 24, no. 12, pp 19-27. bright, m.w., hurson, a.r. & pakzad, s.h. (1992) "a taxonomy and curent issues in multidatabase systems", ieee computer, vol. 25, no. 3, pp 50-60. batini, c., lenzerini, m. & navathe, s.b. (1986) "a comparative analysis of methodologies for database schema integration", acm computing surveys, vol. 18, no. 4, pp 323-364. cluet, s., delobel, c., simeon, j. & smaga, k (1998) "your mediators need data conversion!", in haas, l.m. & tiwary, a. (eds.), sigmod 1998, seattle.washington, pp 177-188, acm press. gardarin, g., gannouni, s., finance, b., fankhauser, p., klas, w., pastre, d., legoff, r. & ramfos, a. (1996) "iro-db a distributed system federating object and relational databases", in bukhres, o.a. & elmagarmid, a.k. (eds.) object-oriented multidatabase systems a solution for advanced applications, chapter 20, pp 684-712, prentice hall, eaglewoods cliffs, nj. grant, j., litwin, w., roussopoulos, n. & sellis, t. (1993) "query languages for relational multidatabases", vldb journal, vol. 2, no. 2, pp 153-171. garcia-molina, h., papakonstantinou, y., quass, d., rajaraman, a., sagiv, y., ullman, j.d, vassalos, v. & widom, j (1997) "the tsimmis approach to mediation: data models and languages", journal of intelligent information systems, vol. 8, no. 2, pp 117-132. graefe, g. & ward, k. (1989) "dynamic query evaluation plans", in clifford, j., lindsay, b.c. & maier, d. (eds.) sigmod 1989, portland, oregon, pp 358-366, acm press. hasselbring, w. (1999) "top-down vs. bottom-up engineering of federated information systems", in conrad, s., hasselbring, w. & saake, g. (eds.) proc. 2nd int. workshop on engineering federated information systems (efis'99), kuhlungsborn, germany, pp 131-138, infix-verlag, sankt augustin. ives, z.g., florescu, d., friedman, m.a., levy, a.y. & weld, d.s. (1999) "an adaptive query execution system for data integration", in delis, a., faloutsos, c. & ghandeharizadeh, s. (eds.), sigmod 1999, philadephia, pennsylvania, pp 299-310, acm press. kim, w., choi, i., gala, s. & scheevel, m. (1995) "on resolving schematic heterogeneity in multidatabase systems", in kim, w. (ed.) modern database systems, chapter 26, pp 521-550, acm press, new york, nj. kabra, n. & dewitt, d.j. (1998) "efficient mid-query re-optimization of sub-optimal query execution plans", in haas, l.m. & tiwary, a. (eds.) sigmod 1998, seattle,washington, pp 106117, acm press. kent, w. (1991) "a rigorous model of object reference, identity, and existence", journal of objectoriented programming, june, pp 28-36. kelley, w., gala, s., kim, w., reyes, t. & graham, b. (1995) "schema architecture of the unisql/m multidatabase system", in kim, w. (ed.) modern database systems, chapter 30, pp 621-648, acm press, new york, nj. kim, w. & seo, j. (1991) "classifying schematic and data heterogeneity in multidatabase systems", ieee computer,vol. 24, no. 12, pp 12-18. litwin, w. & abdellatif, a. (1986) "multidatabase interoperability", ieee computer, vol. 19, no. 12, pp 10-18. lim, e.-p., chiang, r.h.l. & cao, y. (1999) 'tuple source relational model: a source-aware data model for multidatabases", data & knowledge engineering, vol. 29, no. 1, pp 83-114. levy, a.y., rajaraman, a. & ordille, j.j. (1996) "querying heterogeneous information sources using source descriptions", in vijayaraman, t.m., buchmann, a.p., mohan, c. & sarda, n.l. (eds.) vldb'96, mumbai (bombay), india, pp 251-262, morgan kaufinann. lakshmanan, l.v.s., sadri, f. & subramanian, i.n. (1996) "schemasql a language for interoperability in relational multi-database systems", in vijayaraman, t.m., buchmann, a.p., mohan, c. & sarda, n.l. (eds.) vldb'96, mumbai (bombay), india, pp 239-250, morgan kaufinann. miller, r.j. (1998) "using schematically heterogeneous structures", in haas, l.m. & tiwary, a. (eds.), sigmod 1998, seattle.washington, pp 189-200, acm press. milo, t. & zohar, s. (1998) "using schema matching to simplify heterogeneous data translation", in gupta, a., shmueli, o. & widom, j. (eds.) vldb'98, new york city, new york, pp 122-133, morgan kaufinann. 124 ajis vol. 8 no. 1 september 2000 papakonstantinou, y., abiteboul, s. & garcia-molina, h. (1996) "object fusion in mediator systems", in vijayaraman, t.m., buchmann, a.p., mohan, c. & sarda, n.l. (eds.) vldb'96, mumbai (bombay), india, pp 413-424, morgan kaufmann. pitoura, e., bukhres, o. & elmagarmid, a.k. (1995) "object orientation in multidatabase systems", acm computing surveys, vol. 27, no. 2, pp 141-195. roth, m.t. & schwarz, p.m. (1997) "don't scrap it, wrap it! a wrapper architecture for legacy data sources", in jarke, m., carey, m.j., dittrich, k.r., lochovsky, f.h., loucopoulos, p. & jeusfeld, m.a. (eds.) vldb'97, athens, greece, pp 266-275, morgan kaufmann. saltor, f., castellanos, m. & garcia-solaco, m. (1993) "overcoming schematic discreprancies in interoperable databases", in hsiao, o.k., neuhold, e.j. & sacks-davis, r. (eds.) interoperable database systems, proc. of the ifip wg 2.6 database semantics conf, ds-5, lome, victoria, australia, pp 191-205. saltier, k. & hoding, m. (1999) "adapter generation for extraction and querying data from web sources", proc. of 2nd acm sigmod workshop webdb'99. saltier, k. & saake, g. (1999) "supporting information fusion with federated database technologies", in conrad, s., hasselbring, w. & saake, g. (eds.) proc. 2nd int. workshop on engineering federated information systems (efis'99), kuhlungsbom, germany, pp 179-184, infix-verlag, sankt augustin. sheth, a.p. & larson, j.a. (1990) "federated database systems for managing distributed, heterogeneous, and autonomous databases", acm computing surveys, vol. 22, no. 3, pp 183236.spaccapietra, s., parent, c. & dupont, y. (1992) "model independent assertions for integration of heterogeneous schemas", the vldb journal, vol. 1, no. 1, pp 81-126. sciore, e., siegel, m. & rosenthal, a. (1994) "using semantic values to facilitate interoperability among heterogeneous information systems", acm transactions on database systems, vol. 19, no. 2, pp 254-290. venkataraman, s. & zhang, t. (1998) "heterogeneous database query optimization in db2 universal datajoiner", in gupta, a., shmueli, o. & widom, j. (eds.) vldb'98, new york city, new york, pp 685-689, morgan kaufinann.wiederhold, g. (1992) "mediators in the architecture of future information systems", eeee computer, vol. 25, no. 3, pp 38-49. 125 ajis vol. 8 no. 2 may 2001 organisational semiosis: integration and separation between system features and workpractices rodney j. clarke department of business systems, university of wollongong northfields avenue, north wollongong, nsw australia, 2500 phone: +61 02 4221 3752 fax: +61 02 4221 4474 e-mail: rodney_clarke@uow.edu.au www: http://www.uow.edu.au/commerce/buss/sysco.htm abstract traditional information systems theory and practice assumes a tight coupling or integration between workpractices in organisations and the information systems which are notionally built to support them. the relationship between the integration and the separation of workpractices and system features has been theorised as dialectical. it has also been argued that the goal of system design would be to achieve a dynamic equilibrium within this dialectic. however, this paper argues that the above mentioned dialectic forged between integrationist and separations! views can be usefully critiqued by applying systemic semiotics. systemic semiotics refers to a combination of systemic functional linguistics (a semiotic model of language) and its extensions into a general semiotic framework called social semiotics. the latter draws heavily on the notion of dialogism which this paper proposes is useful in rethinking the relationship between workpractices and information systems. in addition, concepts of text and context are drawn from systemic functional linguistics in analysing the workpractices associated with the use of actual information systems features. two examples are used to explicate this dialogic relationship, including: (i) the dynamic renegotiation of a workpractice which is assumed to be closely integrated to a system feature (negotiated separation), and (ii) the extension of the system into other locations by means of communicatively organising materials and users in the workplace (indirect integration). introduction goldkuhl (1993) questions a prevailing assumption within the information systems discipline that designers should work to create a tight integration between workpractices or activities in organisations, and the information system features designed to support them. while current design practices assume integration, he notes that there have been periods throughout the history of is design when a separation between workpractices and systems features has been assumeda conclusion easily substantiated in the literature. goldkuhl proposes that is designers should consider system features simultaneously from a basis of integrationassuming system features are tightly coupled to a workpractice, and separationassuming system features are loosely coupled to a workpractice. in order for system designs to be evaluated simultaneously from these two standpoints, either distinct integration-oriented and separation-oriented design practices must be used, or alternative practices must be applied or developed which can facilitate this kind of design evaluation (see figure la). modelling the design of any given system feature twice using integration-oriented and separation-oriented design practices is generally impractical in terms of time and effort. however, goldkuhl (1993) employs methods, originally developed in the isac systems development methodology, to demonstrate how several system features may be considered from both integrationist and separationist standpoints. goldkuhl's (1993) work has important implications for the development of organisational semiotics1, since the use of an information system could also be considered from integrationist and separationist standpoints. a movement from the domain of system design to that of systems use requires: • substituting the planned functionality afforded by systems design with the actual functionality afforded by systems implementation of specific information systems features, while also • substituting the proposed organisational activity to be supported or created using the information system, with enacted workpractices in organisational contexts the only exception to a general agreement with goldkuhl's (1993) thesis, is that there is, or should be, a dialectical relationship between integration and separation in design and, by extension, systems use. williams (1988, 106-108) defines 'dialectic' in terms of the "continual unification of opposites, in the complex relation of parts to wholes [emphasis own]". as there appears to be no complex relation evident between integration and separation, there is cause to question the existence of a dialectical relationship between these concepts in systems design and by extension in system use. in the following section, integration and separation will be reconsidered as a dichotomy to be deconstructed using derrida's reading tactics (norris 1982). figure 1: (a) shows the dialectical relationship between integration and separation in systems design (after goldkuhl 1993), while (b) shows a dialogical relationship between integration and separation in systems use. 103 ajis vol. 8 no. 2 may 2001 (a) (b) design goldkuhl(1993) use clarke (1997) organisational activity workpractice dialogic centrifugal forces dialectical relationship o dialogical relationship centripetal forces monologic information system information system feature bntegration and separation as a dichotomy integration and separation appear to be end points or extrema along a simple cline or gradethe degree of binding between a workpractice and a system feature. consequently, in systems design and use, a 'large' degree of integration presupposes a 'small' degree of separation, and visa versa. demonstrating that integration and separation are in a kind of proportional rather than a dialectical relationship shows that the relationship between these entities is actually a dichotomy, see figure 2a. dichotomies in social theory often function to suppress one of their terms, that is, dichotomies are not equally balanced. derrida's deconstruction provides a set of reading tactics for interrogating dichotomies (norris 1982). the first deconstructive tactic used to interrogate the integration/separation dichotomy, is reversal, that is we reverse the relative positions of the two terms in privileging separation, refer to figure 2b. reversal enables us to demonstrate that integration and separation are not logically necessary or unalterable in their relationship to each other. 104 ajis figure 2: vol. 8 no. 2 may 2001 in (a), the terms integration and separation are shown as a dichotomy, see the upper panel, with the dominant term highlighted, in the lower panel. in (b), the dichotomy is interrogated using derrida's reading tactics (see text). dichotomy integration separation (a) dichotomy in current use integration separation reading tactics (derrida 1988) (b) 1: reversal separation 2: displacement integration, 3: hinge negotiation. as noted earlier, goldkuhl (1993) uses the literature to disrupt the dominance of integration, while at the same time valuing the repressed term separation. however, such an operation still preserves the dichotomous relationship between the two terms. derrida's second deconstructive tactic is referred to as displacement. the repressed term separation must be displaced, not out of the dichotomy altogether but by positioning it within the core of the dominant term as its logical condition. this makes explicit the unacknowledged debt that the dominant term of integration owes to the secondary term of separation. by demonstrating that the integration/separation dichotomy may not be logically necessary, the tactic of displacement foregrounds the fact that the dichotomy could be replaced by entirely different concepts. the third reading tactic for interrogating dichotomies is the creation of a relevant hinge term which is 'outside' the binary opposition between integration and separation but which participates in both terms. a hinge term may be derived from texts being examined or it might be a neologism devised to interrogate a specific dichotomy. the function of the hinge term is to provide a logical precondition from which the dichotomy is constructed. the 105 ajis vol. 8 no. 2 may 2001 hinge term is in effect that which is leftover, unrepresented and uncontained by the dichotomy. parenthetically, we have used the term 'reading tactics' as a way of avoiding using the term method since derrida's 'reading tactics' do not suggest, for example, any candidates for the hinge term. the hinge term employed here to interrogate the dichotomy is negotiation. negotiation is the logical precondition from which the integration/separation dichotomy is constructed in information systems theory. in using this term we foreground the fact that integration or separation between workpractices and information systems features is never historically fixed, but rather is always at risk. as a hinge term, negotiation is compatible with goldkuhl's (1993) aim of describing a 'dynamic view' of the relationship between information system features and organisational activities. the choice of the term negotiation is strategic because it enables the relationship between workpractices and information systems to be theorised as dialogic according to mikhail bakhtin's sense of the term (todorov 1992), and as applied in clarke (1992). integration and separation as dialogic bakhtin's research involved the development of translinguistics and was applied to the study of the novel, which at the time was a relatively recent literary genre. he focused on developing a dynamic theory of language and meaning emphasising the interrelationship between three categories: (i) the active and productive capacity of language; (ii) the evaluative nature of meaning; and (iii) social subjectivity. according to todorov's (1984) reading of bakhtin's work, the first category involves discourse which is theorised in terms of the production of actualised meaning in a communicative event, as a responsive interaction between social beings. the second category involves the fact that an apparently initial utterance is never actually the first utterance in any sense. in effect, every word or utterance looks back to the word or utterance it is answering, while simultaneously looking forward to the anticipated word or utterance it will partly determine in advance (this is the dialogic property of language attributed to workpractices. see below). the third category involves the social basis of meanings since bakhtin viewed language and thought as intertwined. for bakhtin, language cannot exist without thought, nor can thought exist without language, consequently both are social not individual. refer to dore (1995, 151-176) for a detailed explication of the social constitution of the individual. in applying bakhtin's concepts to information systems in organisations, workpractices may be defined as texts which conform to bakhtin's notion of the dialogic. the characteristic of workpractices as 'negotiated', proposed above, accords with bakhtin's theories of text (todorov 1984; see also the development of bakhtin's work by kristeva reviewed in noth 1990, 321-324). this is demonstrated in the ways that workpractices are actively renegotiated by relevant social subjects, so that workpractices may exhibit more than one preferred realisation. however, the imposition of information systems into workpractices often acts to create an opposite tendency toward what bakhtin refers to as the monological, or "the reduction of potentially multiple voices' (or characters) into a single authoritative voice ...[reducing the production of actualised meaning which] is sometimes inescapable." (fowler 1987, 58-60), refer to figure ib. translinguistics is especially useful as a means for theorising how systems position users and others in organisations, and for providing a dynamic view of systems use. however, it does not provide methods for the analysis of actual work texts (for example transcripts) in specific situational and cultural contexts. bakhtin was critical of the traditional linguistics of his day, developing substantial critiques of formalism and structuralism in linguistics. specifically, he viewed traditional linguistic theories as monologic in that they attempting to account for discourse as if it consisted of single meanings. therefore, translinguistics resists the kind of operationalisation necessary in a design discipline such as information systems. in order to study texts associated with workpractices, we need methods which can be applied to the task. a text-semiotic model of language called systemic-functional linguistics (sfl) developed by halliday (1985), and elaborated into the relatively new field of social semiotics (kress 1985; hodge and kress 1988; thibault 1991), is used to analyse transcripts associated with workpractices2. theoretical affinities3 can be identified between translinguistics and sfl which permit them to be used in conjunction with each other (see table 1). in addition, sfl genre theory developed by martin (1992, 493-590), is used to analyse transcripts in order to derive the typical arrangement of stages in the enactment of a specific workpractice and its associated information system (the generic analysis and the genre digraph notation used here are provided in clarke 1998). genre analysis demonstrates that workpractices are negotiated. related realisations of a workpractice are shown as alternative sequences in a genre digraph. when information systems are associated with workpractices, these technologies are also negotiated albeit from the point of view of the workpractice. from a social semiotic perspective, negotiating the typical arrangement of stages in a workpractice corresponds to an adoption of the dominant reading position of the system feature. the reading position is adopted by users who comply with the dominant discourses informing the workpractice, that is, the discourses which produce the 'coherence' of the workpractice. from this position, the system feature will appear to be the most obvious, natural, and uncontested negotiation of the workpractice. in adopting the dominant reading position, users comply with those discourses which produce centripetal forces tending to a monological, or integrationist instance of systems use. 106 ajis vol. 8 no. 2 may 2001 current genre theory emphasises a synoptic view of genre, and as a consequence it is not well suited to explaining divergences from the typical arrangement of stages in workpractice genres. translinguistics can be of use in theorising these atypical realisations of workpractice genres, as an adoption by users of a non-dominant, resisting subject position. at times users may: (i) re-negotiate one type of workpractice into a different form of workpractice, which still has the same overall purpose although realised by different means, or (ii) re-negotiate a workpractice into a completely different form. these new forms may not be organisationally sanctioned. as a consequence, resistant readings of a workpractice and its associated system features run the risk of failure in organisational contexts, in so far as the pragmatic goals of the workpractice may not be achieved. in some contexts, resistant readings may be viewed as an infringement of workplace regulations, best practice agreements or relevant acts of parliament. however, in some circumstances the adoption of a non-dominant (resisting) subject position may lead to a successful renegotiation of the workpractice. by adopting a non-dominant subject position, users mobilise discourses that produce centrifugal forces tending to a dialogic or separationist instance of systems use. here, an associated information system may be used in an unorthodox way, or effectively bypassed using a manual work-around. exemplification: integration and separation in systems use in order to exemplify the theoretical approach to integration and separation in systems use described above, two examples are drawn from a longitudinal study of an information systems in its organisational context (see clarke 1996, 1998). alabs (automated library and borrowing system) was a transaction processing system (tps) developed by staff at the microcomputer laboratories (mcl), university of wollongong, to implement the loan and return of a variety of items to its clients including software (on disks), manuals and hardware. it was a stand-alone system developed prior to wholesale networking of the mcl. both examples consist of: (i) a description of the case using sfl genre theory and (ii) a reappraisal using translinguistics. student loans many actual instances of texts associated with a specific kind of work are analysed in order to develop a workpractice genre. each text is recorded, transcribed, and analysed for its constituent genre elements. the results of this analysis are sequences of genre elements, one sequence for each text. these sequences are merged together to form a digraph representation of the genre, where triangular symbols represent the start and end of the genre, labelled circles represent the constituent elements of the genre, and where arcs indicate one or more possible alternative sequences between elements of the genre. 107 ajis vol. 8 no. 2 may 2001 table 1: theoretical affinities between relevant terms in translinguistics and systemic functional linguistics, presented across rows. translinguistics systemic functional linguistics social subjectivity • emphasises the way in and by which the social domain discursively constitutes subjects • subjects navigate a maze of organisational discourses in workplaces social roles (tenor) • theorised as a component of register, tenor involves established role relationships in texts environment • "...constitutes [a social subject's] situation in its entirety..." (roberts in morris ed/ 1994, 250) in terms of purview and also those elements which are provided by others context of culture (malinowski 1923) • theorised using the contextual strata of genre (martin 1992,493-588) • can be glossed as societal and organisational culture in the study of workpractices purview • "the limited extent of [a social subjects]... vision (literally and figuratively) as an individual consciousness (q.v.), or as the member of a social group" (roberts in morris ed/1994, 250) context of situation (malinowski 1923) • theorised using the contextual strata of register (martin 1992,493-588) • viewed as "everything significant that was going on concurrently with the speech activity." (hasan undated, 10) genre • "the particular way by which consciousness models experience... broadly correspond[ing] to... artistic genres (roberts in morris ed/ 1994, 248) • a specific type of genre, the speech genre is divided into primary speech genres (e.g. ordering lunch), and secondary speech genres (e.g. workpractice texts) genre • using martin's (1992) stratified model of sfl, genre becomes its own contextual strata, see context of culture • viewed as synoptic and dynamic, although all methods emphasise the former, while some research has raised doubts about the possibility of the latter (see hasan undated, 10) utterance • any stretch or unit of language as small as a word or as large as a text • not a purely linguistic concept but an encounter between a social subject's self consciousness, mind, and the world with all its socio-historical meaning (paraphrase of roberts in morris ed/1994,251) text the text is the unit associated with the discoursesemantic strata in martin's (1992) model: supported below by two 'lower level' language strata, and above by two 'higher level' contextual strata (register and genre) considered as both product (eg. transcript) and as a (social) process meaning • opposed to any fixed notion of meaning, rejecting views of meaning as totally textbound, or views of meaning as the product of exclusively extra-textual factors (paraphrase of roberts in morris ed/1994,249) meaning • sfl conforms to a contextual theory of meaning, that is, meaning is ultimately derivable from its observed context (paraphrase of noth 1990, 100) • meaning is not determined, rather it is construed as social semiotic (see kress 1995; halliday 1978) 108 ajis vol. 8 no. 2 may 2001 the student loan workpractice is represented by a genre digraph in figure 3. the collected texts form service encounters between a student (a client) and a labstaff member (a responsible agent) who operated the alabs system. the most frequently occurring sequence of elements are arranged along the horizontal line which joins the start and end symbols of the digraph. generally students and/or labstaff would start the workpractice by exchanging salutations (represented by a phatic greeting element) prior to establishing the required items (represented by the service request element). on occasion the phatic greeting element is not realised in student loan texts, indicated by the curved arc in figure 3, and both client and responsible agent move directly to resolving the sr element. at the identification sought element, students are required to provide their student identification (sid) card to the labstaff member for scanning into alabs. students must meet the following criteria in order for the workpractice to continue: (i) the student must be currently enrolled on alabs; (ii) they must have a valid sid card; and (iii) they must not have any outstanding loans. failure to meet any of these criteria will likely lead to the termination of the workpractice. the sid card is retained by the labstaff for the duration of the loan. once the is element has been completed, the requested items are provided to the student during the materials out element. a subsequence consisting of mo and sr may occur when the student requests additional items. mo is generally followed by the phatic 'leave taking' talk of the finis element. alternatively, the arc connecting mo to the end of digraph symbol, shows a less common path where f is not realised. in order to enrol students on alabs, a distinct regulations and enrolment subsequence was inserted between is and mo. during interviews conducted with participants engaged in student loans, an incident was recounted in which a student who had lost their sid card, provided an expensive watch as collateral against the software they were borrowing. in this case the is element was renegotiated into an element whose function more closely approximated the value of a retained item. the participants conducted the loan manually created a negotiated separation. between the workpractice and the system, shown by dashed lines in figure 3. the users jointly renegotiated one type of workpractice into a completely different form of workpractice, despite the fact that the new form was neither sanctioned within the organisation nor supported by alabs. this atypical realisation of the student loan workpractice is not explainable within traditional sfl genre theory, which has no provision for a change in the function of a 'mandatory' genre element without a corresponding change in genre. i regard the negotiated separation as an instance of a workpractice exhibiting dialogism. systems features have a tendency to be monological, often designed with a particular set of sectional interests firmly in mind (of the designer) and often employing a managerial or analytical perspective. most traditional design practices and methodologies lead to the production of systems that in use reduce the number of 'voices' in workplaces. furthermore, those systems which have embedded security and identification features (is) are often impossible for workers and clients to renegotiate and become 'inescapable' in bakhtin's terms. many transaction processing systems (tps) in administrative contexts are so 'brittle' in use, that workers and clients may renegotiate them 'out of the picture' entirely. the system will be bypassed and a manual work-around thus realising a new form of workpractice. tutor loans and returns tutors were responsible for teaching classes in the computer laboratories during supervised laboratory times. a responsible academic known to the labstaff introduced new tutors to the microcomputer laboratories. once the tutors' details were entered into alabs, a unique two-digit barcode was assigned to them in the barcode book that resided at the service desk. the two-digit barcode enabled the quick entry of the tutor details during tutor loan and return operations. prior to the start of their class, a tutor would pick up two or more boxes of disks (10 disks per box) and a class loan form. the disks had been sorted into boxes by the labstaff prior to the start of the class. because all the supervised laboratory times were pre-booked, labstaff would prepare boxes of the disks in advance of the arrival of the tutor at the service desk. no transcripts are available for these workpractices, however, the staging of these service encounters has been reconstructed from interviews with participants who were involved with them. the genre digraphs for these service encounters are available in clarke (1998), but are omitted here due to space restrictions. 109 ajis figure 3: vol. 8 no. 2 may 2001 the student loan genre represented using the digraph notation (after clarke 1998). the typical staging of this workpractice is shown using solid lines. the dashed lines show an instance where is is renegotiated into an entirely different element, the value of a retained item. this realisation of the student loan genre involves a negotiated separation between the system and the workpractice, which cannot be explained using sfl genre theory. tutor loans and returns were unique to alabs. while all other users were only ever involved in a single location at any one timethe service desk, tutors were required to function in two distinct locations: the service desk where the tutor borrowed or returned software items, and the teaching laboratories where these items were used. tutors involved in a tutor loan service encounter, received both the class sets of disks and a so-called class loan form. the class loan form, a written text, acted as a shared material that bridged two locations. the class loan form was the means by which tutors acted as surrogate labstaff members responsible for conducting manual student loans in teaching laboratories during supervised laboratory times. once in a laboratory, the tutor was required to record the student number and disks' details on the class loan form, so that an audit trail could be maintained for the use of each disk. the recording of details on the class loan form by tutors and the distribution of disks to students during a supervised laboratory time formed a service encounter, referred to as the class loan. at the end of a tutorial, the tutor was required to collect all disks from students. generally this involved 'ticking off' each returned disk on the class loan form. this workpractice also constituted a service encounter, which is referred to as a class return. upon completion of the class return, the tutor returned the class sets of disks and the completed class loan form to the labstaff member at the service desk. the resulting workpractice formed a service encounter called a tutor return. see figure 3, which shows the configuration of these workpractices. 110 ajis vol. 8 no. 2 may 2001 figure 3: configuration of tutor loan (a) and tutor returns (b) workpractices in alabs as nested service encounters. (a) service desk class set class loan of disks form (2) class loan tutor student i teaching laboratory i use (b) (4) tutor return tutor labstaff. service desk class loan class set form of disks (3) class return tutor student teaching laboratory the success of a tutor return depends on the successful completion of the class return, class loan and tutor loan that preceded it. tutors were dependent on the labstaff to provide them with the correct type of disks, the correct number of disks and at the appropriate level of quality (zero defects or no faulty disks). if one or more of the disks in a class set provided during the tutor loan was faulty, then delays would occur in the supervised laboratory class while either (i) students moved to other machines or exchanged disks between themselves, or (ii) if the tutor is forced to leave the class to replace faulty disks at the service desk. situation (i) increases the likelihood of erroneous attribution of disks to students, while situation (ii) increases both the likelihood of erroneous attribution of disks, as well as the possibility of mislaying additional disks brought into the laboratory by the tutor after the tutor loan had taken place. the labstaff members in situation (ii) required vigilance, so that these additional disks were also accounted for during the tutor return. furthermore, if the class loan service encounter was not correctly completed either the (i) student numbers, (ii) disk numbers, (iii) or both, might not be correctly recorded on the class loan form, and the effectiveness of the class return and tutor return would be compromised. in effect, tutor loans, class loans, class returns, and tutor returns constitute what the author calls a nested service encounter. the concept of nested service encounters and the interdependencies between several service encounters and written texts have never been described in the sfl literature. tutor loan and return features demonstrate the extension of the system into other locations through organising materials and discursively positioning users within the workplace. this case demonstrates an indirect integration where centripetal forces tend towards the monologic. additional system features were dedicated to ensuring compliance from tutors (for example, overdue loan notices printed daily). interestingly, according to a systems perspective only the tutor 111 ajis vol. 8 no. 2 may 2001 loan and tutor return operations would be considered part of alabs. in effect, an automation boundary would be drawn separating the location of the laboratory from the location of the service desk. the class loan form would not be seen as a shared material but rather as data which was only visible at the start and conclusion of the tutor loan is started, and at the time the tutor return is concluded. in contrast, a systemic semiotic perspective does not recognise the existence of an automation boundary as inevitable or obvious. rather it may be seen as a negotiable feature entirely dependent on the specific view of the relation between workpractices and is, if at all. conclusions in this paper, the author has attempted to re-theorise integrationist and separationist views of systems use by examining specific systems features notionally developed to support several workpractices. the dialectic between integrationist and separationist views presumed by goldkuhl (1993), was seen to be a dichotomy. derrida's reading tactics were applied to this dichotomy. the hinge term 'negotiation' was identified which could form a logical precondition for the dichotomy. concepts from bakhtin's social semiotics were used to theorise 'negotiation'. integration in use was theorised as monologism, the tendency towards associated with centripetal forces, while separation in use was theorised as dialogism, the tendency associated with centrifugal forces. despite providing a useful way of theorising a dialogical relationship for is, bakhtin's translinguistic project purposefully resists efforts at operationalisation. consequently, relevant concepts from systemic functional linguisticsa semiotic model of languagewere selected in order to be able to undertake applied studies in information system use. no simple mapping exists between concepts and units in translinguistics and systemic functional linguistics, however, theoretical affinities may be established. two examples were used to test the veracity of these theoretical affinities. spoken and written language texts associated with actual workpractices and supported by system features of an information system (alabs at the microcomputer laboratories), were collected and analysed using the genre theory of systemic functional linguistics, and reinterpreted using concepts from translinguistics. the student loan example illustrates the active renegotiation by participants of what would otherwise be considered mandatory components of a workpractice. augmented with a translinguistic account, the student loan is seen as an instance of a negotiated separation between a workpractice and its system feature, demonstrating centrifugal forces tending towards the dialogic. the tutor loan and return example, comprising two service encounter genres supported by the system and two which are not. the tutor loan and return demonstrates the extension of the system into other locations that the system cannot directly control. in a translinguistic account, such practices are made possible through the positioning of users within the workplace. this case demonstrates an indirect integration where centripetal forces tend towards the monologic. the concepts of negotiated separation and indirect integration, observed in these case studies have not previously been described in the systems design or systems use literatures. notes emerging applied semiotic fields, variously referred to as organisational semiotics (stamper 1995), computer semiotics (andersen 1992), semiotic engineering (de souza 1993), computational semiotics (vogel 1997) and semio-informatics (clarke 1995), can be usefully classified according to their respective points of departure. organisational semiotics and semio-informatics take the organisation as their point of departure while following a trajectory which leads to computer systems, information systems and technologies. however, computer semiotics, semiotic engineering and computational semiotics use computer systems, information systems and technologies as a point of departure while following a trajectory which leads to the organisation. while the combination of bakhtin's translinguistics and halliday's systemic functional linguistics is usually considered as part of social semiotics (see thibault 1991), the author uses the more specific term systemic semiotics, to characterise the theoretical approach adopted here. the term 'affinity' is used to designate a theoretical connection between a concept from one body of theory to one or more concepts in the other body of theory, and is synonymous with the qualitative analysis term theoretical triangulation (miles and huberman 1988). acknowledgments my thanks go to efi hatzimanolis of the communications and cultural studies program, faculty of arts, university of wollongong, australia for her detailed critical reading of this paper. this research was conducted 112 ajis vol. 8 no. 2 may 2001 under the auspices of the system in context (sysco) research sig of the information systems in organisations (isorg) research group, at the department of business systems, university of wollongong, australia. references andersen, p. b. (1991) a theory of computer semiotics: semiotic approaches to the construction and assessment of computer systems cambridge human computer interaction series, uk: cambridge university press belsey, c. (1980) critical practice new accents london: methuen clarke, r. j. (1992) "some applications of social semiotics in information systems discipline and practice" in macgregor, r.; clarke, r. j.; little, s.; gould, t. and a. ang eds/ (1992) information systems as organisational processesisop'92: proceedings of the third australian conference on information systems wollongong, nsw australia, 67-79 clarke, r. j. (1996) "the persistence of systems in organisations: genre analysis of systems decommissioning" in holmqvist, b.; andersen, p. b.; klein, h. and r. posner eds/ (1996) signs of work: semiosis and information processing in organisations berlin, new york: walter de gruyter, 59-106 clarke, r. j. (1997a) "organisational semiosis: rethinking integrationist and separation-ist views of information systems" abstract in sankey, m. r.; gimate-welsh, a. and p. pellegrino eds/ (1997) semiotics bridging nature and culture vlth international semiotics congress, guadalajara, mexico, july 14-18 1997, international association for semiotic studiesassociation internationale de semiotique (iass-ais) and asociacion internacional de estudios semioticos, 102 clarke, r. j. (1997b) "an information system in its organisational context: a systemic semiotic longitudinal case study" unpublished phd dissertation crowley, t. (1989) "bakhtin and the history of language" in hirschkop, k. and d. shepard eds/ (1989) bakhtin and cultural theory manchester and new york: manchester university press, 68-90 derrida, j. (1988) limited inc evanston, ii.: northwestern university press de souza, c. s. (1993) "the semiotic engineering of user interface languages" international journal of manmachine studies 39, 753-773 dore, j. (1995) "the emergence of language from dialogue" in mandelker, a. ed/ (1995) bakhtin in contexts: across the disciplines evanston press: northwestern university press, 151-176 fowler, r. ed/ (1987) a dictionary of modern critical terms london and new york: routledge goldkuhl, g (1993) "on the relations between information systems and organisational activities: integration and separation as thought models" research report vits june 1993 lith-ida-r-93-18 institutionen for datavetenskap: unversitetet och tekniska hogskolan, linkoping, sweden halliday, m. a. k. (1985) spoken and written language australia: deakin university press hasan, r. (undated) the conception of context in text mimeo, 188pp. hirschkop, k. (1989) "introduction: bakhtin and cultural theory" in hirschkop, k. and d. shepard eds/ (1989) bakhtin and cultural theory manchester and new york: manchester university press, 1-38 hodge, r. and g. kress (1988) social semiotics "chapter 2: the founding fathers revisited" united kingdom: polity press, pp. 13-36 holquist, m. ed/ (1981) the dialogic imagination: four essays by m. m. bakhtin translated by carl emerson and michael holquist, university of texas press slavic series, no. 1, austin: university of texas press kress, g r. (1985) linguistic processes in sociocultural practice australia: deakin university press malinowski, b. (1923) "the problem of meaning in primative languages" in ogden, c. k. and i. a. richards (1923) the meaning of meaning london: kegan paul, trench trubner mandelker, a. (1995) "logosphere and semiosphere: bakhtin, russian organicism, and the semiotics of culture" in mandelker, a. ed/ (1995) bakhtin in contexts: across the disciplines evanston press: northwestern university press, 177-190 miles, m. b. and a. m. huberman (1988) qualitative data analysis: a sourcebook of new methods newbury park: sage publications roberts, g (1994) "a glossary of key terms" in morris, p. ed/ (1994) the bakhtin reader: selected writings of bakhtin, medvedev, voloshinov london: edward arnold, 245-252 norris, c. (1982) deconstruction: theory and practice london and new york: methuen noth, w. (1990) handbook of semiotics bloomington and indianapolis: indiana university press pechey, g (1989) "on the borders of bakhtin: dialogisation, decolonisation" in hirschkop, k. and d. shepard eds/ (1989) bakhtin and cultural theory manchester and new york: manchester university press, 39-67 threadgold, t. (1986) "introduction: semiotics ideology language" in threadgold, t; grosz, e. a.; kress, g. and m. a. k. halliday eds/ (1986) semiotics ideology language sydney studies in society and culture no. 3, sydney, australia: the sydney association for studies in society and culture, 15-60 113 ajis vol. 8 no. 2 may 2001 thibault, p. j. (1991) social semiotics as praxis: text, social meaning making, and nabokov's ada theory and history of literature, volume 74 minneapolis and oxford: university of minnesota press todorov, t. (1984) mikhail bakhtin: the dialogical principle translated by wlad godzich theory and history of literature, volume 13 minneapolis: university of minnesota press williams, r. (1988) keywords: a vocabulary of culture and society london: fontana press 114 the australasian journal of information systems (ajis) began in 1993 and was based on the simple realisation that there was a need for an academic ‘outlet’ for is research in australia australasian journal of information systems volume 14 number 1 nov 2006 case study: the state of information systems in australian capital territory universities shirley gregor school of accounting & business information systems, the australian national university. shirley.gregor@anu.edu.au edward lewis school of information technology and electrical engineering, australian defence forces academy, university of new south wales. e.lewis@adfa.edu.au craig mcdonald school of information sciences and engineering, university of canberra. craig.mcdonald@canberra.edu.au abstract this paper describes the information systems groups at the australian defence force academy (adfa), the australian national university (anu), and the university of canberra (uc). each group has a distinctive background that reflects its position in canberra, australia’s seat of federal government. adfa is essentially a private university for the australian defence organization; anu was set up to be a national research institution; and the uc group for many years focused on meeting the training needs for computing professionals for the federal government. despite these distinguishing characteristics, the subject matter taught and researched in the three groups has a large degree of commonality and each group regards itself as ‘vibrant’ and happy with what it does. a low degree of professionalisation is perceived, however, relative to older disciplines, as there is a disjunct between what is taught as core knowledge and what is taught as research methods, a lack of social prestige, and a lack of acceptance as a discipline with a unique symbol system. 177 australasian journal of information systems volume 14 number 1 nov 2006 introduction the australian capital territory (act) differs from the australian states in that it is a “territory” rather than a state – a distinction shared also by the northern territory. the act was chosen as the seat of the australian government and the home for its capital city, canberra, in 1908. a neutral location for the capital and the new territory was selected between sydney and melbourne, because of the rivalries among the states that federated to form the nation of australia in 1901. the territory is less than 100 square kilometres and is tiny compared with the large state of new south wales that surrounds it. the population of the act is approximately 300,000 and almost all its inhabitants live in canberra, which is home to the three university institutions described in this report. the descriptions of the institutions should be read against this common background of being situated in a relatively new, planned, government city with a smaller population than all but one of the capital cities in the other states in australia. the three institutions represented are the australian national university (anu), the university of canberra (uc) and the university college of the university of new south wales (unsw) at the australian defence force academy (unsw@adfa). the act also has campuses of the australian catholic university and charles sturt university but as they do not have information systems (is) programs they are not included. each of the three authors of the report represent one of the institutions included and they were able to collaborate face-to-face in preparing the report as they are all located within such a small geographical area. the authors are used to working together and interacting around common interests on other occasions. a research assistant, ahmed imran, interviewed each of the authors separately as an initial source of information, with the data obtained being supplemented by archival research. the report was then extended and refined by additions from each of the authors and colleagues at the three institutions. the universities in this study table 1 presents some summary statistics for the universities in this study. university number of full-time equivalent students approximate number of full-time equivalent staff (academic and general) unsw@adfa 1711 (932 u/g) in 2004 300 the australian national university (anu) > 13,920 (8,731 u/g) > 3,600 university of canberra (uc) 6000 (4200 u/g) > 1000 table 1 demographics for the home universities for the is groups, as of 2005 the academic component of the australian defence force academy is the university college of the unsw, which has the status of a faculty. the university of new south wales has an agreement with the department of defence to provide tertiary education to officer cadets and midshipmen of the armed forces of australia, new zealand, singapore, and thailand. in effect, it is a private university for the australian defence organisation. however, it also offers postgraduate education to any qualified applicant, not necessarily with any defence affiliation. 178 australasian journal of information systems volume 14 number 1 nov 2006 anu differs from other contemporary australian universities in that it was established by an act of federal parliament. on its establishment in 1946 it was given no undergraduate teaching responsibilities. its mandate was to undertake "postgraduate research and study both generally and in relation to subjects of national importance" (anu, 2005a, 2005b). undergraduate teaching was gained only when the anu joined with the canberra university college in 1960. the canberra university college had been in existence for some time as a college of the university of melbourne. the anu continues to be one of australia’s most research-intensive universities, with a high ratio of academic staff to students. the anu was also given an unusual structure, with an institute of advanced studies, which is primarily engaged in research, and ‘the faculties’, which undertake undergraduate teaching as well as engaging in postgraduate training and research. information systems is situated in the faculties. the university structure has changed recently, with colleges formed in january 2006 that more closely integrate the component parts of the institute and the faculties. the university of canberra was formed by an act of the act government in 1990 from the canberra college of advanced education that had been established in 1969. as a college of advanced education (cae), its educational programs focused on professional and para-professional disciplines. as a university it has increased its focus on research. overview of the is presence in the act universities table 2 gives an overview of the information systems groups in the three universities. university administrative entity home faculty number of full-time is academics number of is students * unsw@adfa is group in school of it and electrical engineering university college 7.5 100 the australian national university (anu) is group in school of accounting and business information systems economics and commerce 6.5 100 university of canberra (uc) is group in school of information sciences and engineering business, law and information sciences 11 250 * approximate number of full-time equivalent students in is courses, including service courses, not is programs alone. table 2: demographics for the is groups (2005) australian defence force academy (adfa) computer science was taught at the royal military college (rmc) duntroon for some time. initially, it was presented by the department of mathematics but a separate department of computer science, under vance gledhill, was in place in the mid 1980s. whilst still at rmc, the department introduced a first year is subject in 1985. in 1986 the australian defence force academy opened and the university moved from rmc to adfa. unsw@adfa has provided education in information systems since its inception in 1986, within the school of computer science. since 1986, each year saw another year added to the is curriculum, with an honours year established in 1991. the first postgraduate research student also started in 1991. 179 australasian journal of information systems volume 14 number 1 nov 2006 in 1990 there were 120 students in is first year, with 50 in second year. at that time, the services were encouraging students to undertake is or computer science studies to at least second year. this ‘encouragement’ has dropped off since then and the numbers fell to about 30 in is first year in 2003. since then the numbers have risen in all years, despite the dip in enrolment in other institutions, to about 65 in first year and 30 in second year. in 2003, the schools at unsw@adfa were amalgamated and now is is taught within the school of information technology and electrical engineering. this year, the is staff members have started to provide vocational education and training (vet) level training in enterprise architecture under contract to the department of defence. they have established a specialist stream in the msc (it) for enterprise architecture, the first in australia. they also provide a service course to the second year students in the school of business (about 25 students). despite being part of the unsw, the is group at adfa has little contact with the school of information systems, technology and management in the faculty of commerce at the main campus at kensington, nsw. the australian national university (anu) at anu, information systems has been part of the faculty of economics and commerce since courses in is were first taught. an early influence was ron weber who had an appointment in the then department of department of accounting and public finance in 1977-1978. professor weber had just completed his phd at the university of minnesota and has been identified as the first australian to obtain a phd in is. he recalls that at that point there were few openings for an information systems academic and he had to rely on his background in accounting and auditing to get a job. information systems appears to have been first taught as a separate unit in 1982-1983, by michael mccrae and roger debreceny. in 1984 the faculty made the strategic appointment of a reader in information systems. roger clarke led the discipline until 1995. joan cooper was also a reader in the department, 1990-1992. the decade 1984-1995 was marked by growth. a major was jointly developed with the separate computer science department, and successful joint bachelor of information technology programs were introduced. these programs ranged from highly technical to highly business-oriented programs, and were linked to the bachelor of science and bachelor of engineering at one end, and the bachelor of commerce and bachelor of economics on the other. the foundations for a comprehensive postgraduate program were laid, four phd supervisions were completed, and external research grants were awarded by bodies including the arc. from 1995 a number of pressures meant that the discipline contracted. although the number of is staff at one point briefly reached five, a number left in the mid-1990s. the faculty made no senior appointment, and for the next five years the discipline languished. by 2000, the group had shrunk to just two staff. at that time the dean of the faculty, professor tim brailsford, showed confidence in the future of information systems and electronic commerce and appointed the first professor of information systems, with an endowment from the central administration. the group has now increased so that there are eight staff and a good number of phd, masters and honours students. research activity has increased considerably, with much output over the last few years. as at other institutions, research is driven by staff interest and grant funding. this has led to significant diversity in research output, with key papers in areas such as the philosophy of 180 australasian journal of information systems volume 14 number 1 nov 2006 technology, technology adoption and use, decision making, business ethics, open source software and market modeling. the anu also has a college of engineering and computer science which offers programs in information technology (it), computer science and software engineering. the is group works with the it staff in offering joint degrees and in course specification and design. university of canberra (uc) is is a discipline in the school of information sciences and engineering alongside network engineering, software engineering and mathematics and statistics. the school is in a division with the business and law schools. uc has two other divisions, communication and education and health, design and science. computing was a founding discipline at uc when the institution was created as a college of advanced education in 1971. the major in-house course for it in the commonwealth public service, the programmer-in-training program, became the core of the computing degree, initially as part of the school of administrative studies. the foundation head was digby pridmore and within three years computing became a separate school. information systems has always been organisationally co-located with the technologists who can reify designs rather than with the user areas. this has allowed a concrete, effective approach to is teaching and research rather than is being an abstract discipline that 'someone else can actually do'. staff initially did not belong to streams such as software engineering, is or computer hardware; rather they taught in one or more of many computing areas. separation has never been complete and is is core to the bachelor of software engineering and bit as well as its own program, the bachelor of business informatics. is continues to teach a significant major into the business program and the major is available to the rest of the university, particularly attracting education students who want to become it teachers. it also continues to teach its introductory unit into all disciplines. the post-graduate picture is similar with is teaching into the mit as well as establishing the mbi and teaching into the mba, m. knowledge management and so on. in its 30 year history is has remained with its sister it disciplines through a range of reorganisations. this history has allowed is at uc to integrate information, workflow and knowledge management to complement the traditional data management in organisations. there are some areas in uc that teach is-like units (the communications school teaches knowledge management and multi-media, the resources school teaches gis, marketing teaches some internet, education some e-learning and so on). mostly this teaching is coordinated with the is program. since becoming a university in 1990 uc has emphasised its research program and has reached supervisory saturation point. 181 australasian journal of information systems volume 14 number 1 nov 2006 distinctive features of is curriculum table 3 gives an overview of the programs offered at the three universities. university undergraduate courses/programs postgraduate courses/programs unsw@adfa bsc, ba, service to bbus msc (it), including streams in enterprise architecture and governance the australian national university (anu) bcomm (is major) bit/bcomm minfsysst minfsys university of canberra (uc) bse, bit, bbi*, is major, double degrees eg b.comm/bbi; bse/bbi, bbi/law mit, mbi * bi = business informatics table 3: courses from which is students are drawn as of 2005 australian defence force academy (adfa) unsw@adfa follows the acm/ ais curricula for its undergraduate and postgraduate courses. it reviews its curriculum every three years, with the last review in 2002. as the third year designed by that review has just finished, so another review is just starting. the is group within the school uses problem-based learning as much as possible within its undergraduate courses. there is an emphasis upon systems thinking, including alter’s ‘work systems’ concepts (as represented by the third year subject managing work systems). the components of it are not discussed until the operating information systems in second year. however, all first year students are expected to pass the international computer drivers licence made available through acs. there are no electives in the is minor course and only one elective in the is major. all students are required to undertake a project in the third year that acts as a capstone for the course. the postgraduate courses are taught using ‘flexible education’; that is, classes are available on campus but most of the students are distance students. webct is used to manage the courses, with student notes delivered as word documents linked to webct or to a student portal for some subjects. the australian national university (anu) the undergraduate is curriculum is based on the australian computer society (acs) accreditation guidelines. major curriculum reviews occur approximately every three years, with minor revisions occurring on a yearly basis. staff levels have historically dictated the number of course offerings available, though recent increases in staff levels have meant more specialised classes can be offered at both graduate and undergraduate levels. because of the group’s proximity to the business disciplines, course delivery is frequently undertaken from a business and commercial perspective. key undergraduate courses are offered in information systems foundations (encompassing knowledge management, decision support 182 australasian journal of information systems volume 14 number 1 nov 2006 systems, design, and ethics), electronic business, analysis and design, information systems management and project management. graduate level courses are offered in is research issues, ecommerce for managers, strategic information systems, communications technologies and issues in organisational information systems. staff are encouraged to take their own approach to delivering course content, guiding students and providing pastoral care. this approach includes sharing course tutelage between staff members in order to support different perspectives and teaching styles. the combined four-year bit/bis degree program that gives both a business-focused is major as well as the more technical knowledge in the it major offered by the department of computer science has been very popular with students until recently. all undergraduate courses maintain at least one course website. content portals vary between staff members, classes and year levels. some courses in the school use webct to manage and provide course content, while others maintain their own content web sites under a faculty web server model. some courses are also using weblogs to furnish class content. university of canberra (uc) is at uc takes the position that there is a core of is knowledge and that this facilitates responsible and effective technology-based systems creation and adoption by a wide range of professional disciplines. business is one application area, and uc runs courses at undergraduate and postgraduate levels in 'business informatics' that combine is and business. the is group also teaches into areas including health informatics and educational informatics, but to date with much lower numbers than for business. on the technology side, is has majors and electives in the bachelor of information technology and bachelor of software engineering and a combined bbi/bit course the is core comprises typical is units (information systems in organizations, database design, human computer interaction, systems analysis and modeling, document and workflow management, systems project and quality management) plus electives (in human computer interaction, usability evaluation, knowledge management systems, business intelligence systems, database systems, advanced information modelling, general systems theory, is management, business informatics case studies, information security, information law and it and business alignment). the development of students who have an understanding of the interpersonal and group processes required of is and it professionals is a feature of is teaching at uc. students engage in group projects, role-playing case studies, online virtual organisations and presentations and they develop visual and written communication skills, critical thinking and teamwork skills. all units are required to show how they embody a) current research literature, b) appropriate it, c) industry participation and d) pedagogical technique. all units have a website for teaching material and to facilitate communications. units are developed by teams and each academic is on several teams in order to keep the unit content coherent. academics are encouraged to develop new units in the areas of their research. industry interaction is important so uc has tutors and guest lecturers drawn from industry, panels to advise on curriculum content and the is group teach into specific industry courses (for example, with the australian bureau of statistics and kaz computing). the bit has an industry project and 183 australasian journal of information systems volume 14 number 1 nov 2006 the bachelor of business informatics has an industry-supported internship that is becoming a significant vehicle for industry interaction. distinctive features of is research table 4 shows the main areas of research interest in the information systems groups in each of the universities. university areas of is research focus is research groupings australiandefence forces academy governance, dss for crisis/ antiterrorist management, risk management, tools for human thinking virtual environment and simulation laboratory (part), decision support ict governance and enterprise architecture group (see adfa, 2005c) the australian national university strategic use of it, technology adoption, intelligent systems/decision support, humancomputer interaction, project management, theoretical foundations, open source software national centre for information systems research university of canberra hci, ethics, ontology & systems modelling, virtual behaviour, informatics (e-research, e-learning, health informatics, e-law) human-computer communication laboratory, informatics program (see uc, 2005b) table 4: information systems research in act universities australian defence force academy (adfa) the unsw@adfa is group prefers the ‘design theory’ approach to research, which focuses on knowledge and theory concerning effective methods and structures for the development of information systems. all of its members have consultancy and ict industry backgrounds, so they favour the development of tools and techniques that are useful and usable. these group members undertake several consultancies each year, which often act as feedstock for their research papers. generally, the is group members avoid applying for external grants. they prefer to obtain industry funding for their research, usually in the form of consultancy activity, which can form the basis for papers about how the work was undertaken. the research interests of the members of the is group include systems planning (business cases, tender evaluation), enterprise architecture, ict governance (strategic planning, performance measurement), aids to human decision-making, social networks, value systems, viable systems theory, household information systems, and e-government. the members of the group work with other staff of the school in gaming, complex systems, and optimisation. 184 australasian journal of information systems volume 14 number 1 nov 2006 the australian national university (anu) the anu group is very active in research. the primary research concerns through the 1980s were in the areas of accounting applications, software development and project management. during 1988-95, the department ran a major research program in supra-organisational systems. particular foci included strategic information systems, edi and other emergent forms of e-commerce, and the economics of aarnet and the then-new public internet. staff have recently been increasingly successful in obtaining grants, with a number of arc grants and applied research projects with government and industry. as an example, members of the group completed a large applied research project in 2005 commissioned by the department of communications, technology and the arts (dcita) concerning the realisation of value from information and communication technologies. the report from this project was launched by the minister responsible for dcita, helen coonan, in sydney in april, 2005 and copies of the report have been distributed nationally in conjunction with the australian computer society. the group aims to focus on research issues that are of fundamental importance to the is discipline and to this end hosts a biennial workshop on “information systems foundations”. the national centre for information systems research (ncisr), formerly known as the electronic commerce research group, is a vehicle for collaborative research within the is group. members include other faculty staff as well as external researchers and business figures. the centre has a regular seminar program that is designed to be of interest to a wide range of people. university of canberra (uc) in the 15 years since uc became a university is research has been largely individual and therefore eclectic. most of the is staff come originally from industry, but have been in the education field for some time. of the 11 staff, 5 have phd's and 2 are enrolled. the group is roughly half active researchers, a quarter occasional researchers and a quarter who are not research active. in arc terms the research is not strong, but there is a history of work in human-computer interaction, modeling, ethics, it education and e-government. much of the research work is done in collaboration with commonwealth government agencies and semi-government organisations such as greening australia and kaz computing. the informatics theme is an attempt to bring the group's research work into a coherent framework, in particular, to research informatics (or e-research, being the application of ict to the research process and the use of its products), e-learning, e-law and health informatics. the development of two research coursework units, research proposal and research methodology, and the weekly research seminar series have assisted in giving research students a better grounding and provided a forum for staff and visitor interaction with research students. 185 australasian journal of information systems volume 14 number 1 nov 2006 key figures who have influenced is in act universities it is not possible to include all the people who have contributed to is within the act and just a few key people are included. some background for each of the three authors is included as each is currently a leader of is at one of the three universities and so have some degree of influence on what is happening and has happened, for varying lengths of time. australian defence force academy (adfa) dr edward lewis has taught is at unsw@adfa since 1986. after 12 years in the australian army, working in is policy and personnel areas, he was posted to the academy as a visiting military fellow (effectively an academic in uniform). he converted to civilian life in 1988, after realising that being an academic “sure beats working for a living”. he taught every subject as the curriculum developed from first year through to the introduction of the honours year in 1991. he supervised the first three honours students in that year, as well as the first msc and phd students, also starting in that year. until this year, as the only is staff member with a phd, he has supervised all of the seven phd graduates in is and is currently supervising 12 postgraduate students. dr lewis is active in consulting projects, mostly in strategic planning, business cases, and thinking tools. he is currently the chair of the it-030 committee for standards australia that recently published as8015: 2005 corporate governance of information and communication technology and as8018: 2004 ict service management. the australian national university (anu) dr roger clarke led the establishment of is at the anu, where he was a reader in the department of commerce from 1984 to 1995. roger has also made his mark more widely. he founded the australasian is directory in 1988, was a foundation committee-member of the australasian conference in information systems (acis) 1991-94, was a foundation asia-pacific councillor of the ais in 1995-96, and established the isworld country section, including the page for australia, in 1996. he returned to consultancy in 1995, focussing on strategic and policy aspects of ebusiness, information infrastructure, and dataveillance and privacy and is now a visiting professor at anu’s department of computer science. dr shirley gregor came to the anu in 2001 as the foundation professor in information systems and was given the role of developing the strengths of the information systems group. shirley began her career as a programmer-in-training in melbourne and had a number of years in the computing industry in australia and the united kingdom before becoming an academic. professor gregor was inaugural president of the australasian association of information systems 2002-2003 and is vicepresident of the australian council of professors and heads of information systems. professor gregor was made an officer of the order of australia in the queen’s birthday honours list in june 2005 for services as an educator and researcher in the field of information systems and in the development of applications for electronic commerce in the agribusiness sector. at the anu she has focused on facilitating research through the formation of the national centre for information systems research. university of canberra (uc) dr. igor hawryszkiewycz was head of is during the late 1970's and early 1980's and is now a professor at the university of technology, sydney, specialising in collaborative technologies. 186 australasian journal of information systems volume 14 number 1 nov 2006 dr errol martin was an associate professor and head of the school in which is was located in the late 90's. he maintained strong links with industry and chaired the acs is board for many years. penny collings has been a long-time teacher and researcher at uc after specialising in computer supported collaborative learning and behavioural simulations. she became the head of the is discipline (1992-2001) and is responsible for the development of several industry-based panels that provide input into the course redesigns that occur every three years. dr craig mcdonald has been head of is since 2001. he spent 15 years in industry before moving to the university sector in 1985. he has conducted innovative research and practical application of knowledge management and expert systems to support the viticulture industry with the crc viticulture and currently publishes in ethics, e-research and concept modelling. dr mcdonald is currently editor of the australasian journal of information systems. dr john campbell joined uc as ise research director in 2005. prior to his current appointment, he spent 17 years at griffith university in brisbane where his responsibilities included head of school in information systems and management science, and also as deputy head of school in the school of management. his major research focus is how users interact through information systems in the social world and, in particular, the ways in which organisational decision-making and community interaction are enacted through collaborative technologies. since 2001, he has assumed an active member role in cpa australia’s information technology and management centre of excellence. the status of is as a distinct discipline in act universities the status of is in the act universities is analysed against the framework contributed by ridley (2006) as a common basis for comparison for the cases in this volume. this framework encompasses the following dimensions: the nature of the “body of knowledge” in the disciplinary units, where the knowledge could be of the types distinguished by fitzgerald (2003), that is, rules or laws (as in the natural sciences); evidenced guidelines; and normative guidelines. the degree of professionalisation of the discipline, which is expected to decrease as the impact of local contingencies increases. the degree of professionalisation will be evidenced by the extent of variation in the nature of is research. the acceptance as a distinct scientific field, which is conditioned on: scientific reputations becoming socially prestigious and critical rewards controlled; standards of research competence and skills being established a unique symbol system is developed that allows the exclusion of outsiders. we have used this framework in our analysis, although we do not necessarily endorse the use of the word “science” or “scientific” to describe the is discipline. this is not to say that is research should not be rigorous, but the word science has many connotations and at least some of the work done in is does not conform to common understandings of what makes a science, in terms of the natural and behavioural sciences (for example, see nagel, 1979). 187 australasian journal of information systems volume 14 number 1 nov 2006 nature of ‘body of knowledge’ the domain of knowledge is examined by analysing the commonalities in syllabi across all three institutions. table 5 below shows the areas of knowledge that are common across all of the three universities in the core knowledge in their information systems majors. two of the three universities follow australian computer society guidelines in planning their curriculum but these guidelines, as described in underwood (1997), leave a great deal of discretion to the institution applying them. acs core knowledge area nomenclature* adfa course (is major in ba, bsc) anu course (is major in bcomm) uc course (is major) introductory unit** introduction to information systems (problem solving approach) foundation of information systems and e-commerce information systems in organisations program design and implementation (or systems building/software construction) exercises in perl within is in organisation; web design within design of is introduction to programming use of sql in database design and system building in it project. some use of design tools in designing human computer interaction and systems analysis & modelling systems analysis and design design of is information systems analysis systems analysis & modelling, advanced information modelling, document and workflow management, knowledge management systems database management is in organizations and design of is relational databases, database systems database design, database systems project management and quality assurance selection of systems, management of work systems project management and information systems systems project and quality management managerial and organizational issues** management of work systems information systems management is management, it business alignment specialist areas** application of is (elective choice of topics, including ecommerce, data management) electronic business, accounting information systems, electronic commerce strategic issues, information technology in electronic commerce document and workflow management, knowledge management systems, business intelligence systems, general systems theory, informatics case studies ethics/social implications/professional practice is in organisations, management of work systems in many courses in many courses, especially it project 188 australasian journal of information systems volume 14 number 1 nov 2006 interpersonal communications introduction to is, management of work systems, is project all courses in many courses throughout. specialist unit (professional practice in it) for international students industry project third year team project, 50% load second semester, from defence project management and information systems it project; business informatics internship *the australian computer society’s core body of knowledge terminology unless noted. **not named as such in the acs core body of knowledge table 5 knowledge areas common to the core of information systems at all three universities. however, from table 5 it can be seen that there is some commonality across the institutions in the core of their courses, which is evidence of common understanding of the domain knowledge which is central to the discipline. the areas in common include systems analysis and design, database design and management, project management, managerial and organisational issues, ethics/social implications/professional practice and interpersonal communications. each university provides coverage of specialist knowledge and application areas that reflect the interests and capabilities of staff. there is variation in the degree to which programming and software construction are required. at anu an introductory programming course is mandatory. unsw@adfa provides little education in programming in its is courses, with its computer science programs covering programming in more depth. the main reason is that the careers of the undergraduates are prescribed and they involve no programming on the job, rather the supervision of civilian or service trade-trained personnel. uc requires no mandatory programming study, which again is covered in information technology programs, but students do get experience of systems building in their project courses. another lack of commonality concerns data communications and networks, which is a mandatory course only at anu. there is a range in the types of knowledge presented in the courses taught in common. in the terms of fitzgerald (2003): rules or laws (as in the natural sciences). examining the common syllabi shows that there is relatively little core knowledge of the natural science type – there are few, if any, law-like generalisations. rather the knowledge falls more into the second and third categories in fitzgeralds’s taxonomy below. the knowledge, however, does rely on underlying theories of the natural science type for justification and explanation of system-building knowledge. for example, guidelines for the construction of decision support systems rely on behavioural science knowledge of human decision making. evidenced guidelines. the common syllabi include knowledge that in the form of guidelines arising from practice. for example, the project management courses include knowledge of project management tools and practices such as software cost estimation that are based on empirical observation. normative guidelines. the courses in the common syllabi include what can be termed normative guidelines or ‘design theory’ (see gregor, 2002, 2006). for example, codd’s relational database theory is taught in all the database courses across the institutions. 189 australasian journal of information systems volume 14 number 1 nov 2006 our conclusion is that we have a recognised common domain that might be expressed in this way: the information systems discipline analyses human activity, determines the kinds of data, information and knowledge needed to enable people to act effectively and responsibly, designs technology-based systems to support them and evaluates the impact of those systems. our knowledge is based on theories of information, human activity and organisations, is studies analysis techniques and methodologies, human-computer interaction, data and knowledge representation and design, systems construction and validation, and the impacts of systems on people, organisations and society. in this unique role is bridges the essentially content-free information technologies with the content-rich but often unstructured domain knowledge. to perform this function, is embraces types of knowledge and epistemologies that differ from those traditional in the natural sciences and exhibits a more well-grounded, participative style – congruent with a discipline that is at the intersection of science, technology and human and organisational behaviour. degree of professionalisation our reflection leads us to conclude that we have a relatively low degree of professionalisation of the discipline, using the criteria of the “extent of variation in the nature of is research”. in terms of the content of our research endeavours there is not a great deal of variation, as all our researchers focus on areas which are recognisably within the disciplinary bounds, and we have common understanding of each other’s interests. a number of the people in our schools (and all the authors) do applied research, in that we build or help people to build artifacts. however, in terms of the methods used in our research there is very wide variation and the research methods taught to students do not necessarily suit the particular nature of is as, at least in part, a discipline concerned with the construction of artifacts (as evidenced in our common syllabi). the is students at anu are taught research methods in the same class as accounting, auditing, management and marketing students and do not currently have any exposure in these courses to ‘design theory” (a shortcoming to be addressed in the near future). the situation at adfa is similar, with only uc explicitly addressing design-type research. this disjoint between what is taught as core knowledge and what is taught as research methods leads us to conclude that there is a low degree of professionalisation. acceptance as a distinct field the consensus of opinion among the authors is that our field is not ‘socially prestigious’. all of us have had experience with appointment and promotion committees where it is clear that the nature of work in is is not well understood or regarded highly. many of our staff have had experience in industry before joining in academia so they have had relatively limited time to establish a scientific reputation in terms understood by committees with social and physical scientists who have worked almost their whole career in universities. neither are we accepted as a distinct field in terms of a unique symbol system by outsiders. we do have commonalities in our symbol system with parts of computer science and software engineering, which is understandable as we are allied fields with some degree of overlap. however, we have 190 australasian journal of information systems volume 14 number 1 nov 2006 overlap which is of more concern in that much of our symbol system is also used by people in application areas who use is and it as tools. we find it hard to ‘exclude these people as outsiders’ and many cannot see why they should not teach is topics within their own areas, as in health informatics. people outside is, even within universities, have very poor understanding of what the term “information system” means, either in relation to the artifacts constructed, or as name of the discipline. the continuing debate within is about how our discipline should be defined only adds to the confusion. to conclude, each of the universities has a vibrant is group which appears happy with the work they do themselves. there are areas of concern, however, reflected in the different organisational placements of each groups and the disciplines with which they most interact within their university. it would be hard to argue that the groups have an unambiguous identity visible to those outside the groups. references adfa (2005a) http://www.unsw.adfa.edu.au/ adfa (2005b) school of itee http://www.itee.adfa.edu.au/ adfa (2005c) itee research www.itee.adfa.edu.au/research/ adfa (2005d) portal www.itee.adfa.edu.au/~ejl/portal anu (the australian national university) (2005a). http://info.anu.edu.au/discover_anu/about_anu/profile/index.asp (accessed sept., 2005) anu (the australian national university) (2005b). http://info.anu.edu.au/discover_anu/about_anu/vicechancellor_xxs_welcome/index.asp (accessed sept., 2005) fitzgerald, g. (2003) information systems: a subject with a particular perspective, no more, no less, european journal of information systems, vol.12, no.3, pp. 225–228. gregor, s. (2006, forthcoming). the nature of theory in information systems. mis quarterly, 30, 3. gregor, s. (2002). design theory in information systems. australian journal of information systems, special issue, 14-22. nagel, e. (1979). the structure of science, hackett publishing co., indianapolis, in. ridley, g. (2006). characterising information systems in australia: a theoretical framework, (see this issue) uc (university of canberra) (2005a) http://wwww.canberra.edu.au/ uc (university of canberra) (2005b) ise research http://www.canberra.edu.au/schools/ise/research/ underwood, a. (1997). the core body of knowledge for information technology professionals, australian computer society. http://www.acs.org.au/national/pospaper/bokpt1.htm whitley, r. (1984). the intellectual and social organization of the sciences, clarendon press, oxford, uk. 191 http://www.unsw.adfa.edu.au/ http://www.itee.adfa.edu.au/ http://www.itee.adfa.edu.au/research http://www.itee.adfa.edu.au/%7eejl/portal http://info.anu.edu.au/discover_anu/about_anu/profile/index.asp http://info.anu.edu.au/discover_anu/about_anu/vice-chancellor_xxs_welcome/index.asp http://info.anu.edu.au/discover_anu/about_anu/vice-chancellor_xxs_welcome/index.asp http://wwww.canberra.edu.au/ http://www.canberra.edu.au/schools/ise/research/programs_and_projects australasian journal of information systems volume 14 number 1 nov 2006 the authors professor shirley gregor is head of the school of accounting and business information systems at anu. professor gregor spent a number of years in the computing industry in australia and the united kingdom before beginning an academic career. her current research interests include the adoption and strategic use of information and communications technologies, intelligent systems and human-computer interface issues, and the theoretical foundations of information systems. dr ed lewis is a senior lecturer at the australian defence forces academy (adfa). dr lewis was in the australian army for 12 years before being posted to adfa. his research interests include methods for enhancing human problem-solving, the development of planning tools, including risk-remedy, preparation of business cases, tools for tender evaluation and the design of information management strategy and policy. dr. craig mcdonald is associate professor of information systems at the university of canberra. for nearly four decades he has been developing, teaching, researching and consulting in systems that manage data, information and knowledge. these systems have been in the public service, ngos, private enterprise and in research organisations. his current work is in e-research, ontologies, informatics and applied ethics. 192 the universities in this study overview of the is presence in the act universities australian defence force academy (adfa) the australian national university (anu) university of canberra (uc) distinctive features of is curriculum australian defence force academy (adfa) the australian national university (anu) university of canberra (uc) distinctive features of is research australian defence force academy (adfa) the australian national university (anu) university of canberra (uc) australian defence force academy (adfa) the australian national university (anu) university of canberra (uc) the status of is as a distinct discipline in act universities nature of ‘body of knowledge’ degree of professionalisation acceptance as a distinct field the authors the australasian journal of information systems (ajis) began in 1993 and was based on the simple realisation that there was a need for an academic ‘outlet’ for is research in australia australasian journal of information systems volume 14 number 1 nov 2006 assessing joint service opportunities through a consideration of the motivating and constraining factors dr mark borman school of business university of sydney sydney nsw 2006 australia email: m.borman@econ.usyd.edu.au abstract in a wide range of industries services are increasingly being developed, or evolving, to support groups of organisations. not all such joint service initiatives though have been successful. the paper aims to highlight potential issues that need to be addressed when investigating the introduction of a joint service by identifying the motivators and constraints. the approach outlined draws upon network externality theory to provide the motivation for a joint service, and resource based and dependency theories to highlight the constraining factors. three instances of joint services – in the banking, telecommunications and travel sectors – are subsequently examined. it is concluded that as well as providing externality benefits joint service initiatives can also improve the terms of access to a service – in particular through realising economies of scale. furthermore it would appear that organisations will have to think carefully about the best way to create, structure and manage a joint service initiative – including who to partner with – given their own particular circumstances, as multiple alternative approaches, with potentially differing ramifications, are available. keywords: joint service, resource based theory, resource dependency theory, externalities, economies of scale. 27 mailto:m.borman@econ.usyd.edu.au australasian journal of information systems volume 14 number 1 nov 2006 introduction historically a number of information and communication technology (ict) enabled applications, or activities, have evolved over time to support an industry rather than an individual firm; for example automated teller machines in banking and sabre, a reservation system, in airlines. more recently organisations have begun to actively seek opportunities where existing services could be provided jointly. in the finance sector a number of banks and unisys have established a joint cheque processing venture (roberts, 2004) while in the global lodging sector a joint distribution consortia – travelweb – has been created (tedeschi, 2003). not all joint service initiatives though are likely to be successful. kambil and van heck (2002) supply a long list of failures across a range of sectors. there are also significant potential risks for participants when a third party serves as the joint service provider. the benefits may, for example, primarily accrue to that provider (clemons and kleidorfer, 1992). the economist (2005) acknowledges such a risk with regard to third party joint service providers in the travel industry. when investigating joint service opportunities it would therefore be useful for organisations to have an understanding of the factors that should be considered. building upon research examining business process outsourcing (bpo) the current paper seeks to present an approach to assist organisations examine and assess joint service opportunities. the results of a case study of three different joint service initiatives, examined under the aegis of that approach, are then presented. the focus of the research is at the strategic, rather than the implementation, level and ensuring that a joint service contributes to an organisation’s strategic objectives and competitive position. the paper contributes to the literature in two principal ways. firstly, by providing an approach by which organisations can evaluate joint service opportunities. secondly, by identifying that there are benefits beyond those associated with network externalities, such as improving the terms of access to a service, the paper suggests that the research agenda should be extended beyond the historical focus on market like initiatives. business process outsourcing and joint service provision joint services can in many ways be considered as a specific manifestation of outsourcing whereby a group of organisations come together to obtain a common service (gallivan and oh, 1999). as such in examining joint services this paper will draw from the outsourcing literature, and in particular that concerning bpo, to address the questions of why adopt a joint service approach and for what activities. bpo has been defined as “the delegation of one, or more, information and communication technology intensive business processes to an external service provider” (tornbohm and andrault, 2005, p3). while research examining the motivation for, and objective of, outsourcing draws from many theoretical perspectives, two are particularly relevant with regard to determining what to outsource from a strategic perspective: resource based theory and resource dependency theory (dibbern et al, 2004; lee and kim, 1999). the resource based and resource dependency theories both view a firm’s resources as being the foundation for its strategy and do not inherently conflict with each other (duncan, 2002) but rather can be seen as complementary (and have previously been combined, for example by grover et al, 1994). here they are synthesised so that the decision regarding what areas to enter into joint service arrangements for takes into account both the strategic 28 australasian journal of information systems volume 14 number 1 nov 2006 contribution of an activity to an organisation and the relationship with the ultimate supplier. transfer of the field of investigation from bpo to the development of joint services though also requires an additional consideration – that a joint approach offers advantages over an individual one. it is proposed that network externality theory can be used to identify such circumstances. network externality theory network externality theory suggests that the “value of a unit of a [network] good increases with the number of units sold” (economides, 1996, p678). the classic example is telecommunications where the value increases as the number of customers, and hence potential connections, grows. the work of katz and shapiro (1994) suggests that joint services may be beneficial for activities where there are network effects that are either direct – where the joint service facilitates connectivity – or indirect – where the value of a unit of the good increases with the number of units sold; for example as a consequence of the wider availability of complementary goods. the underlying proposition in this paper is that externality benefits provide the motivation for joint service initiatives – that the value of, for example, airline reservation systems is greatly enhanced if users are able to search and book flights from multiple airlines rather than just one. resource based theory resource based theory is founded in the work of penrose (1959) and has subsequently been developed by authors such as rumelt (1984), wernerfelt (1984) and barney (1986). it suggests that firms secure success by utilising their unique resources comprised of intangible and tangible assets that are tied semi-permanently to the firm (wernerfelt, 1984). however such resources provide a sustained competitive advantage only when competitors are unable to acquire and deploy similar resources (mata et al, 1995). furthermore, according to coyne (1986), to provide an advantage the resources must contribute to “a consistent difference in important attributes between the producer’s product and those of his competitors” (p51). from the resource based perspective, success is maximised where organisations focus their attention on those areas where their distinctive capabilities lie (hagel and seely brown, 2001) and rely on others for the provision of ancillary activities. joint service initiatives, as with other more traditional outsourcing ventures (dibbern et al, 2004), can free up organisations to concentrate on their core capabilities. there is of course, the added complexity of having to ensure that none of the likely varied, distinctive capabilities of the multiple organisations involved is compromised. resource dependency theory resource dependency theory states that organisations are dependent upon their environment and are faced with choices regarding how they manage that dependency (thompson, 1967). organisations seek to adopt strategies to manage their dependency on external parties (pfeffer and salancik, 1978) and ensure access to the resources those parties supply is stable and secure (kotter, 1979) with the objective of enhancing their competitive position vis-à-vis those parties. according to teng et al (1995) the extent of any dependency is determined by a combination of the importance of the resource, the number of potential suppliers available and the cost of switching suppliers. of particular interest here is the management of the relationship with the joint service provider to ensure that the benefits realised are not appropriated by that provider (katz, 1987). managing dependency may be of particular concern from a joint service perspective given the existence of network externalities which may ultimately lead to a single service provider rising to dominance. casciaro and piskorski (2005) have argued that one way in which organisations can limit dependencies is through actions that “bypass the source of constraint by .. cultivating alternative 29 australasian journal of information systems volume 14 number 1 nov 2006 sources of supply” (p167). fundamental questions are whether such an approach is viable when externalities are present, and are there other dependency limiting options available in a joint service context. taken together the externality, resource based and resource dependency theories provide a basis by which joint service opportunities can be evaluated. essentially they suggest that joint service ventures will be appropriate if they generate benefits that the organisations involved are able to appropriate, as opposed to those benefits accruing to the joint service provider, and do not weaken the distinctive resource based competitive advantage of any of those organisations (see figure 1). resource based resource dependency c on st ra in ts joint serviceexternality m ot iv at io n resource based resource dependency c on st ra in ts joint serviceexternality m ot iv at io n figure 1. motivators and constraints for joint services methodology having proposed an approach to assess joint service opportunities in theory, three cases were conducted and assessed using that approach to determine whether it would be useful in practice. the cases can be seen as principally explanatory in nature (yin, 1984) and draw upon the work of yin (1984) and dubé and paré (2003) with regard to the approach followed. internal and public domain documents (primarily websites, press reports and annual reports) were used to maximise understanding of each case study before interviews were conducted and to corroborate and augment items raised during those interviews. the interviews themselves followed a semi-structured approach to ensure consistency while enabling unique insights to be followed up on. the use of multiple sources of evidence was seen as maximising the likely accuracy of the understanding developed for each case study (yin, bateman and moore, 1983). following the approach of ciborra (2000) the intention has been to provide sufficient information to readers, alongside the author’s interpretation, to allow readers to draw their own conclusions alongside the ones presented here. the specific cases – emobile, ebreak and ebank – were selected on the basis of reflecting the variety of roles, functionality and services that can be provided1. emobile emobile commenced operations with an application for mobile phones that provides users in shopping centres with targeted information from retailers such as special offers or movie times. the 1 all cases spoke on the condition of anonymity 30 australasian journal of information systems volume 14 number 1 nov 2006 service is based upon a combination of bluetooth (to determine location) and gprs technologies (for the transfer of information). users have to register with emobile to use the service and pay an access fee – au$1.25 per day or au$5 per month. users also have to pay their mobile operator’s standard rates for gprs use. any additional, information related, charges depend upon the type of information accessed and are billed via emobile. emobile is carrier and network agnostic and is currently being trialled in a major shopping centre in sydney australia. emobile has also launched a second, adjacent service which allows users to access information regardless of location via gprs. again the service is network and carrier agnostic. emobile argue that being accessible to all mobile phone users regardless of carrier will be a key advantage over alternative carrier specific services which often create walled gardens of content for users2. with emobile users will be provided with the ability to access a wider array of content while information providers will gain access to a broader potential user base. ebreak ebreak acts as an independent online aggregator for “time sensitive” short break products providing an additional distribution channel and booking mechanism for accommodation providers. the company’s primary distribution channel is via the internet; though two call centres provide support, answering user questions and maintaining relations with accommodation providers. accommodation providers place details of their individual inventory offers and pricing on the ebreak website for a limited future time frame – typically seven days. potential customers can search all available inventory using criteria such as location and are provided with details of the qualifying accommodation including availability, prices and inclusions. bookings take place via the internet. details are received initially by ebreak and then passed on to the accommodation providers. payment, by credit card, is to ebreak who reimburse accommodation providers periodically via aggregated invoice. both users and accommodation providers pay a per transaction service fee to ebreak. ebreak had grown to become the third most popular travel website in australia by early 2005 and is currently used by over 6000 hotels, motels, serviced apartments, resorts, guesthouses and bed & breakfasts. over time ebreak has extended its product offering to include other types of bookings, gifts and toys. ebank ebank comprises two separate but related initiatives. both operate as member owned companies. the first – cua – was founded in 1996 by credit unions3 to licence and customise a core banking system (cbs) to support deposits, loans, cards management and collections. the second initiative – cub – was established to operate a computer bureau that would host cbs. over time the membership of both companies has varied as has their remit. cua now has 10 credit unions as shareholders and responsibility for the development or licencing of a range of technology applications beyond cbs. cub has 35 shareholders and hosts a variety of applications in addition to cbs. a credit union can choose to be a member of one initiative or both depending upon its particular needs. a number of the original founders of cua have subsequently withdrawn as they have discovered over time that cbs is more complex and expensive to operate than required by their operations. baseline stakes and fees payable with regard to cua and cub are determined by reference to a credit union’s customer and capital base. shareholders of cua and cub can 2 the leading telco in australia for example was providing access to a total of 210 websites in april 2005. 3 a credit union is a member-owned financial co-operative 31 australasian journal of information systems volume 14 number 1 nov 2006 however vary the level of functionality and service they receive and this is reflected in the actual fee payable. input into the decision making process – ie the votes available to be cast – is proportionate to a credit union’s stake. for each joint service interviews were conducted with the joint service provider as well as the organisations that use it. two interviews were conducted with emobile – with the general manager and the sales and marketing manager – together with three interviews with companies currently using the service (each with retail managers). three interviews were conducted with ebreak – with the chief executive officer (ceo), chief operating officer (coo) and the senior it manager – together with two at accommodation providers (with the person responsible for marketing). one interview was conducted with ebank – with a director and two with the ceos of credit union users4. while the underlying rationale of the interviews was purposeful, to collect data pertinent to the theoretical approach proposed it was deliberately non-directive so as not to preclude the emergence of concepts not previously considered (patton, 2002). as such it is in line with the methodology presented by eisenhardt (1989). with regard to analysis, data was first reviewed and coded in line with the principal dimensions of each theoretical lens5. descriptive codes were used and interview transcripts coded in sentence or multi-sentence chunks. it was possible for the same piece of text to be multi-coded if it related to more than one perspective and proposition. such an approach is in accord with the recommendations of miles and huberman (1994) who suggest that the level of coding detail should be aligned with the objectives of the research. as also suggested by miles and huberman (1994) the data was then collated into conceptually clustered data displays in order to make it readily accessible. where interview data did not code to the concepts identified a priori as of interest it was further assessed to determine if additional concepts could be formed. appendix 1 is a summary data display table. results: motivation network externality both emobile and ebreak exhibit actual, or potential, network externalities. emobile is positioned as a platform that enables communication regardless of mobile telephony provider, offering potential network benefits to information providers. without such a platform it would be necessary to develop separate interfaces to each mobile telephony provider further adapted for each model of mobile phone supported. by enabling across network compatibility, emobile maximises the reach of information providers to users and vice versa. “if you sign with [telco] you only get access to the content they have signed up. if you want to go to the movies with your friends and want to check out what’s on you can’t if you are all with different carriers” emobile – general manager “if we work with [telco] we cut out everybody else.. what works for them doesn’t work for us” emobile – retailer 1 4 the ebank and ebreak cases reported here are components of larger ongoing research studies 5 for example codes related to resource dependency theory included: rd-s(significance), rda(alternatives), rd-c (cost of change) 32 australasian journal of information systems volume 14 number 1 nov 2006 combined, the effects increase the attractiveness of delivering information to mobiles and the likelihood of attaining a sustainable critical mass. it should also be noted that the willingness of emobile to invest, develop and facilitate the service was seen as critical to the information providers, none of whom would have initiated such a service individually. turning to ebreak, the key proposition is that it extends the distribution reach of accommodation providers – that aggregation makes ebreak more attractive to users than any individual property could be. “i can not afford to pay to get that sort of coverage” ebreak – accommodation provider 2 one of ebreak’s providers however did suggest that the reality might differ somewhat from the promise with ebreak primarily serving as an access point for existing customers rather than attracting new ones. “bookings are from corporates who have an established relationship and corporate rate using them as a convenient booking mechanism. we are not getting new customers but now have to pay commission” ebreak – accommodation provider it is clear though that network externalities are not the only motivating force. this is especially apparent with ebank where the principal benefits of joint provision do not appear to be related to improvements to the service itself but rather come from improving the terms of access. in essence ebank aggregates the demand of individual credit unions. with cua this aggregation results in a body of demand that delivers volume discounts and ensures that the input and views of credit unions are taken into consideration as the application is updated by its provider over time. clearly the larger the demand, the greater the leverage. “if three or four of the large credit unions chose to go their own way there would be problems in sustaining this” ebank – director “transaction stuff is volume based and we’re simply not going to get the discounts that the bigger ones can if we were to go on our own” ebank – credit union 1 with regard to cub, a joint approach provides access to services that would not be viable for many of the credit unions if they acted individually. “[when credit unions] moved into mainline banking systems to handle the transaction volumes…evident needed to come together to handle 24/7 operations, backup and recovery.. none of us had the resources” ebank – credit union 2 it should be noted however that there are warning signs regarding the future sustainability of ebank. as competition in the overall financial services sector has increased there has been an increasing reluctance on the part of many of the larger credit unions to cross subsidise smaller ones. “while we accept the cooperative basis of credit unions, as competition with the banks and others has stepped up we need we need to ensure any cross subsidy is not too large” ebank – credit union 2 in addition the needs of the larger credit unions have started to be recognised as different to those of the rest of the sector. one result has been the withdrawal of a number of smaller and mid-sized credit unions from cua. 33 australasian journal of information systems volume 14 number 1 nov 2006 “32 signed up.. only 10 turned on.. large number put their money in .. realised that the system requires you to be larger to run it .. high maintenance system.. a lot that signed up for it realised that they can’t afford to go on” ebank – director results: constraints resource based from a resource based perspective, emobile, as an advertising medium, is not seen as impinging upon the ability of information providers to differentiate themselves6. emobile by managing the technical and many of the operational aspect of the service is viewed as freeing up users to focus on their core businesses as compared to if they were to operate the service themselves. “we simply fax our offers through to them and they load it up” emobile – retailer 2 accommodation providers viewed ebreak as one among many distribution channels. there was some disagreement though regarding whether it limited or liberated them. on the positive side was the view that the core capabilities of accommodation providers lie in their location, brand, facilities, service and star rating. on the negative was that providers often considered themselves indistinguishable on these bases and saw ebreak as focusing competition on price by facilitating comparison. “there are typically many, similar providers in close proximity…[and] comparison consequently causes downward pressure on rates” ebreak – accommodation provider 2 a variety of core capabilities were identified by the credit unions interviewed including customer service and the development of compelling financial products. ebank by supporting operational and transactional aspects of the business was universally seen as allowing them to better focus on these areas. in addition ebank was also seen as making available capabilities that some small and midsized credit unions might not otherwise be able to support individually. “advantage in terms of new releases .. we do not have to go first and wear the pain.. can also piggyback and access to features that others have funded at below cost price” ebank – credit union 1 one of the credit unions also suggested that it was not a particular concern even if differentiation between credit unions was reduced since their competitors were not other credit unions but the banks. “each credit union has its core constituency be it by geography or job .. [and] we do not actively target the markets of other[s]” ebank – credit union 2 resource dependency information providers have no concerns regarding dependency on emobile seeing it as a minor additional advertising channel. for their part, credit unions suggested that multiple alternatives were available to them for both aspects of the service provided by ebank if they wanted to use them. 6 in addition many of the retailers using the service operate in different sectors. 34 australasian journal of information systems volume 14 number 1 nov 2006 one did note however that there would likely be a significant cost incurred with any changeover – arising from both technical and contractual perspectives. “when we merge with other credit unions we look to move them over to our systems .. [in one case] it would have cost $8m to end the existing contract… [with data] is often too hard and risky to migrate so we simply set up all their customers as new customers” ebank – credit union 2 any concern, at least with regard to cub, was also mitigated through its ownership by a consortium of credit unions. however this was also seen as having drawbacks. “actually created the entities that we outsourced to.. unique thing about having our own cooperative is that we would never sue [if service level agreements were not met] .. the costs go back to us.. circular thing” ebank – credit union 1 for ebreak dependency was also seen as limited due to the number of alternatives that are both available and used7. however it was suggested – by ebreak as well as an accommodation provider – that as ebreak moved to offer a service that integrated with internal it systems “to realise backend efficiencies” consolidation was likely as providers would choose to work with fewer services. furthermore not only would choice be reduced but so would the ability to move services due to the cost and difficulty of porting integrations from one provider to another. “everybody has their own systems and works with multiple distributors.. integration would be expensive, complex and restraining.. the more extensive the functionality provided the more complex the integration.. [customers] would likely have to cut the number of distributors they use” ebreak – coo discussion when considering why organisations engage in joint services, network externalities do not appear to be the only motivation. the cases suggest an additional driving force related to improving the terms of access to a service8. such improvements largely appear to be derived from economies of scale; specifically the ability to defray fixed costs over a larger production base (romer, 1990). ebank and emobile for example enable costs to be spread over a number of parties to the extent of enabling access to capabilities that few of the organisations involved would be able to realise individually. clearly for most scenarios that can be envisaged the existence of externalities will be associated with terms of access benefits; after all as the group of users grows it is likely that the cost of servicing each individual will fall. consider, for example, airline reservation systems such as sabre which become more useful as more airlines and agents subscribe but also benefit from economies of scale as costs can be defrayed over a large user base. what is important though is that the example of ebank suggests that joint services can be warranted where only terms of access benefits exist. 7 both accommodation providers used other similar services to ebreak – one used five and the other seven 8 externalities improve the utility of that service 35 australasian journal of information systems volume 14 number 1 nov 2006 the cases also suggest that there are likely to be limits to the number of users of a joint service. one reason, as evidenced by ebank, is due to the difficulty of aligning common interests and maintaining that alignment over time as the organisations involved follow differing development paths. a second reason is that the benefits of adding extra users may not continue indefinitely. for example, with regard to externalities, authors such as liebowitz and margolis (1994) have argued that once the group of users reaches a critical mass there is little additional benefit from securing additional users. the existence of diseconomies of scale is also well documented (see for example coase, 1937; arrow, 1974; williamson, 1975). one might speculate that such a scenario has eventuated with ebreak resulting in a variety of competing services exist in tandem9. furthermore as the number of accommodation providers with similar offerings using any particular service has risen, price based competition has increased10. the optimum membership size is likely to vary across contexts and may also be complicated by the existence of multiple distinct users groups – with potentially differing requirements and equilibria. emobile for example requires, and facilitates, interaction between a minimum of three distinct groups – mobile service providers, information providers and end users. in terms of organisations that use joint services ensuring that they retain their core capabilities and manage their dependency on the provider, the results are broadly positive though there are some concerns. the ability of organisations using ebreak to differentiate themselves, for example, may be inhibited by the imposition of a standard template to describe accommodation. for emobile’s information providers there may be both positives and negatives related to bundling. while a bundle may be more attractive than the elements individually it also ties a provider in with others. with regard to dependency on the joint service provider the main strategy adopted appears to be to ensure there are alternatives available. for information providers using emobile it is simply a case that emobile is only one of many different advertising channels. for ebreak it is that there are many different similar services available and used1112. the cases also illustrate that there are different approaches possible with respect to the establishment, funding and ownership of a joint service. for emobile and ebreak an independent third party established and paid for the joint service with organisations joining individually. ebank, by contrast, is a cooperative created and owned jointly by its credit union members. in the latter case dependency is likely to be less of a concern13. one further factor influencing dependency may be the degree of systems integration. integration – as signalled by ebreak – may increase the likelihood of lock-in. interestingly the ebank case suggests that dependency concerns may extend beyond managing the relationship with the service provider, with dependencies also created with other users. in ebank’s case of prime concern is the impact if one or more credit unions choose to leave the service. indeed interviewees suggested that one of the most difficult tasks was managing the balance between the individual credit union and the group as a whole. 9 if externalities were infinite it would be expected that the market would be likely to ‘tip’ to a single service (shy,2002) 10 the externalities associated with ebreak potentially switching from being positive to negative. 11 such a multi-homing response – whereby consumers on one or more sides of a joint service use more than one provider (evans, 2003) – is clearly only an option where multiple services can co-exist profitably. 12 it is perhaps interesting that mobile service providers do not appear to have any ready opportunity to manage any dependency on emobile since the service simply overlays their activities with no need to involve or consult them. 13 at the same time however such cooperatives may not have divested themselves of non core activities as fully as where the joint service is owned and operated by a third party – while they no longer have responsibility for the activity at the individual level they still do at the group level. 36 australasian journal of information systems volume 14 number 1 nov 2006 conclusions and future research overall the cases suggest that the approach proposed may have some value for organisations in assessing joint service opportunities. clearly though it needs to be amended to incorporate terms of access benefits as a motivating factor. from a practitioner perspective the considerations proposed form a basis to systematically assess opportunities to ensure that benefits are available, core capabilities are not impacted and dependency on the provider can be managed. the research also specifically identifies potential sources of benefits – cost savings, voice etc – and means to manage dependency – primarily ensuring alternatives are available. in addition decision makers are alerted to the need to consider with care issues of compatibility, alignment, dependency and benefit distribution regarding the other organisations involved in the joint service initiative. such considerations may potentially reduce the attractiveness ceteris paribus of initiatives with open, as compared to selective, membership. further contributions relate to the extent and ownership of the joint service provider. the ebank case highlights that the scope of initiatives can vary14 and that organisations should seek to select, or tailor, the one that best fits their particular circumstances. regarding ownership, while a number of recent initiatives (roberts, 2004; tedeschi, 2003) have seen organisations act to retain ownership, ebank highlights that such an approach may also have disadvantages. the principal contribution of the research from an academic viewpoint is the beginnings of a theoretical approach that considers both the motivations and constraints to joint service provision. the identification of terms of access benefits in addition to network externalities also highlights that the potential for joint service initiatives is broader than is typically considered in the academic literature where the focus is overwhelmingly on emarkets and communications and exchange (bailey and bakos, 1997; kaplan and sawhney, 2000). a third contribution relates to the need to examine the joint service decision from a number of different, and potentially divergent, perspectives. going forward there are a number of potential opportunities for future research. a shortcoming of the current research is that it has not sought to examine in detail how the relationship between the joint service provider and the users of the service should be constructed and managed effectively. it would be desirable, for example, to look more closely at the alternative governance structures available (see rochet and tirole, 2003 as a potential starting point). there are also likely to be numerous organisational considerations – relating to factors such as strategy, structure, processes and culture (newell et al, 2001; burke and litwin, 1992 and scott-morton, 1991) – when ensuring that there is an appropriate “fit” between the provider and users, and the different users themselves. the example of ebank suggests that it might be difficult to identify a set of common interests that is stable over the long term. more extensive investigation is warranted to determine if this difficulty will be a common problem, the characteristics of joint service initiatives which are most likely to be stable and beneficial over the long term and whether there are sectoral difference that will influence the likelihood of organisations working together (as hagedoorn (1993) might suggest). from a sectoral perspective, for example, variations in the extent of environmental uncertainty (williamson, 1985) and the degree of competition (porter, 1985) may have an impact. it is indeed possible that joint service initiatives will have most potential in those parts of the public sector where there is a 14 and that even for any given initiative it may be possible to vary the range of involvement or services taken. 37 australasian journal of information systems volume 14 number 1 nov 2006 well established ethos of collaboration and competitive pressures are less (becker and patterson, 2005; gamm and benson, 1998). finally it would be valuable to examine the relationships between the motivating and constraining factors and in particular the potential for tradeoffs between them – for example what is the specific impact of maintaining alternative providers – in an attempt to reduce dependency – on externality benefits. references arrow, k.j. (1974) the limits of organization, norton, new york. bailey, j.p. and bakos, y. (1997) “an exploratory study of the emerging role of electronic intermediaries”, international journal of electronic commerce, 1, pp7-20. barney, j.b. (1986) “strategic factor markets: expectations, luck and business strategy”, management science, 32(10), pp1231-1241. becker, f. and patterson, v. (2005) “public-private partnerships: balancing financial returns, risks and roles of the partners”, public performance & management review, 29(2), pp125-144. burke w.w. and litwin g.h. (1992) “a causal model of organizational performance and change”, journal of management 18(3), pp532-545. casciaro, t. and piskorski, m.j. (2005) “power imbalance, mutual dependence, and constraint absorption: a closer look at resource dependence theory”, administrative science quarterly, 50, pp167-199. ciborra, c. (2000) from control to drift, oxford university press, oxford. clemons, e.k. and kleindorfer, p.r. (1992) “an economic analysis of interorganizational information technology”, decision support systems, 8(5), pp431-446. coase, r.h. (1937) “the nature of the firm”, economica, 4(16), pp386-405 coyne, k.p. (1986) “the anatomy of sustainable competitive advantage”, the mckinsey quarterly, spring, pp50-65. dibbern, j., goles, t., hirschheim, r. and jayatilaka, b. (2004) “information systems outsourcing: a survey and analysis of the literature”, database, (35) 4, pp6-102. dubé, l. and paré, g. (2003) “rigor in information systems positivist case research: current practices, trends, and recommendations”, mis quarterly, 27(4), pp597-635. duncan, n.b. (2002) is integration in the internet age. in information systems outsourcing, r. hirschheim, a heinzl and j. dibbern (eds.), springer, berlin, pp395-414. economides, n. (1996) “the economics of networks”, international journal of industrial organization, 14, pp673-699. eisenhardt, k.m. (1989) ”building theories form case study research”, academy of management review, 14(4), pp532-550. the economist (2005) flying from the computer, october 1, pp61-63. 38 australasian journal of information systems volume 14 number 1 nov 2006 evans, d.s. (2003) “the antitrust economics of multi-sided platform markets”, yale journal of regulation, 20(2), pp325-381. gallivan, m.j. and oh, w. (1999) “analysing it outsourcing relationships as alliances among multiple clients and vendors”, proceedings of the 32nd hawaii international conference on system science, january 5-8, maui, hawaii. gamm, l. and benson, k. (1998) “the influence of governmental policy on community health partnerships and community care networks: an analysis of three cases”, journal of health politics, policy, and law, 23(5), pp771-794. grover, v., cheon, m.j and teng, j.t.c. (1994) “an evaluation of the impact of corporate strategy and the role of information technology on is functional outsourcing”, european journal of information systems, 3(3), pp179-190. hagedoorn, j. (1993) “understanding the rationale of strategic partnering: interorganizational modes of cooperation and sectoral differences”, strategic management journal, 14, pp371385. hagel, j.iii. and seely brown, j. (2001) “your next it strategy”, harvard business review,79(9), pp105-113. kambil, a. and van heck, e. (2002), making markets, harvard business school press, boston kaplan, s. and sawhney, m. (2000) “e-hubs: the new b2b marketplaces”, harvard business review, 78(3), p97. katz, m.l. (1987) “the welfare effects of third-degree price discrimination in intermediate good markets”, american economic review, 77(1), pp154-167 katz, m.l. and shapiro, c. (1994) “systems competition and network effects”, journal of economic perspectives, spring. kotter, j.p. (1979) “managing external dependence”, academy of management review, 4(1), pp8792. lee, j. and kim, y. (1999) “effect of partnership quality on is outsourcing success: conceptual framework and empirical validation”, journal of management information systems, 15(4), pp29-61. liebowitz, s.j. and margolis, s.e. (1994) “network externality: an uncommon tragedy”, journal of economic perspectives, 8(2), pp133-150. mata, f.j., fuerst, w.l. and barney, j.b. (1995) “information technology and sustained competitive advantage”, mis quarterly, 19(4), pp487-505. miles, m.b. and huberman, a.m. (1994) qualitative data analysis: an expanded sourcebook, 2nd edition, sage, thousand oaks. newell, s., pan, s.l., galliers, r.d. and huang, j.l. (2001) “the myth of the boundaryless organization: technology does not make cultural and business boundaries disappear simply because it exists”, communications of the acm, 44(12), pp74-78. patton, m.q. (2002) qualitative research and evaluation methods, 3rd edition, sage, thousand oaks. penrose, e. (1959) the theory of the growth of the firm, basil blackwell, london. 39 australasian journal of information systems volume 14 number 1 nov 2006 pfeffer, j. and salancik, g.r. (1978) the external control of organisations: a resource dependency perspective, harper and row, new york. porter, m.e. (1985), competitive advantage, the free press, new york. roberts, c. (2004) “back-office central”, business review weekly, jan 22-28, p18. rochet, j.c. and tirole, j. (2003) “platform competition in two-sided markets”, journal of the european economic association, 1(4), pp990-1029. romer, p.m. (1990) “endogenous technological change”, journal of political economy, 98(5), pp71-102. rumelt, r.p. (1984) “towards a strategic theory of the firm”. in lamb, r.b. (dir), competitive strategic management, prentice-hall, englewood cliffs. scott-morton, m.s. (ed) (1991) the corporation of the 1990s, oxford university press, new york. shy, o. (2002) the economics of network industries, cambridge university press, cambridge. tedeschi, b. (2003) “hotels put together their own web site”, new york times, 9 march, 2003. teng, j.t.c., cheon, m.j. and grover, v. (1995) “decision to outsource information systems functions”, decision sciences, 26(1), pp75-103. thompson, r.l. (1967) organizations in action, mcgraw-hill, new york. tornbohm, c., and andrault, m. (2005) user survey: business process outsourcing, western europe, 2004, report id: g00126191, gartner group, stamford. wernerfelt, b. (1984) “a resource-based view of the firm”, strategic management journal, 5, pp171-180. williamson, o.e. (1975) markets and hierarchies, the free press, new york. williamson, o.e. (1985) the economic institutions of capitalism: firms, markets, relational contracting, the free press, new york. yin, r. (1984) case study research: design and methods, sage, beverly hills. yin r., bateman p.g. and moore, g.b. (1983) case studies and organizational innovation: strengthening the connection, cosmos corporation, washington. 40 australasian journal of information systems volume 14 number 1 nov 2006 appendix 1: summary data display motivation network externality other factors emobile • network agnostic not provider specific • need multiple users to be attractive • could provide consolidated and comparative demographic data • defray cots over multiple users • access to skills, sites and phone manufacturers retailer 1 • need multiple users to be attractive retailer 2 • need multiple users to be attractive • cost prohibitive to implement service individually ebreak • extend distribution reach • ease of combining travel packages (accommodation, entertainment etc) • greater the participation, the greater the attractiveness • consolidation reduces cost accommodation provider 1 • extend distribution reach – but often existing rather than new customers • destination for choice • savings from outsourced bookings • greater promotion resources accommodation provider 2 • destination for choice • impact of new entrants extending from other sectors • previously reach attainable limited by available resources ebank • can fund development jointly • ensure no big cross subsidy of the small by the large credit union 1 • cost savings • cost sharing • voice • access to skills and capabilities • not all credit unions have the same needs credit union 2 • access to services for small • ensure sure no big cross subsidy of the small by the large 41 australasian journal of information systems volume 14 number 1 nov 2006 constraints resource based theory resource dependency theory emobile • provide as a platform • users control usage • users from multiple sectors – non competing • subscribe to the service • no restrictions on content or format retailer 1 • stock and pricing key • extra channel • limited use retailer 2 • information service • promulgate offers immediately • extra channel • limited use ebreak • service differentiator • brand – but often significant variation within a chain • choice often price driven • multiple markets and sectors • easy to switch because little integration • manage pricing and availability • seeks to own the customer accommodation provider 1 • many comparable hotels • standard template • service/experience differentiator • star rating differentiator • customer bypass to save fees • use top 3 sites as alternatives • manage pricing and availability • multiple channels accommodation provider 2 • serviced apartments serves as differentiator • brand • cheaper & more flexible than previously dominant alternatives • impact of supply and demand • manage pricing and availability • use multiple services and maintain own site • limited to distressed inventory ebank • key differentiator is the market niche targeted • risk of large players pulling out • own the service provider • alternatives available credit union 1 • front end it enables differentiation but not back end • trusted advisor as core capability • commercial partners more responsive than credit union ones • problems when large credit unions move off a solution • solutions all comparable • need for, and problems of achieving, compromise • largest credit unions have the greatest say • supplier relationships long term & changed infrequently • aggregated purchasing power often sufficient credit union 2 • key differentiator is the market niche targeted • risk of large players pulling out • need for, and problems of achieving, compromise • largest credit unions have the greatest say • commercial partners more responsive than credit union owned ones • cost of breaking contracts • cost of change 42 network externality theory resource based theory resource dependency theory emobile ebreak ebank network externality results: constraints resource based resource dependency appendix 1: summary data display microsoft word ajis10-2finala.doc ajis vol. 10 no. 2 may 2003 3 employing interpretive research to build theory of information systems practice bruce rowlands school of computing & information technology griffith university, nathan, australia b.rowlands@griffith.edu.au abstract this paper provides guidance and an example for carrying out research using an interpretive framework. until quite recently, there has been little available in the is literature to guide the interpretive researcher to build theory of is practice. while structured as a typical research paper, this paper is different in that the focus is on conceptual issues and the research methods rather than the findings. unlike positivist research, there is no accepted general model for communicating interpretive research. similarly, few guidelines exist for conducting the inductive process central to interpretive research. throughout the paper, issues relating to the choice and application of the methods in terms of conducting inductive research are discussed. overall, the focus provides an in-depth discussion of the particular interpretive research that i undertook so that other researchers can read of an example that may be similar to their own and therefore guide their work. keywords: is research methodologies, interpretive perspective, case study, grounded theorising. introduction this paper provides guidance and an example for carrying out research using an interpretive framework to build theory of is practice. the paper provides an example of (a) developing a theoretical framework, (b) how to choose an appropriate research method, (c) particulars of data collection and analysis, and (d) appropriate evaluative criteria applicable to interpretive research. the research example is a large-scale doctoral study of decision-making by owner-managers of small firms in the it industry. the aim of the study (1) was directed toward exploring and describing the decision making process of owner/managers regarding their participation with on-the-job training schemes for the first time; and (2) to develop process theory explaining their participation. while structured as a typical research paper, this paper is different in that the focus is on describing the research process, conceptual issues and the research methods used rather than the findings. this format is important for two reasons: (1) unlike positivist research, there is no accepted general model for communicating interpretive research. (2) similarly, few guidelines exist for conducting the inductive process central to interpretive research. throughout the paper, issues relating to the choice and application of the methods in terms of conducting inductive research are discussed. given both the practical importance of, and scarcity of, interpretive research in information systems i feel that documenting the decisions about the research process may be particularly valuable to researchers in the information systems community. the nature of interpretive research it is important first of all to define what i mean by interpretive research and to draw a distinction between a related term — qualitative research. firstly, the term interpretive is not a synonym for qualitative. qualitative research can be interpretive or positive depending on the philosophical assumptions of the researcher (cavaye, 1996). according to van maanen (1979), qualitative research is an umbrella term covering an array of techniques which seek to describe, decode, translate, and somehow come to terms with the meaning, rather than the measurement or frequency of phenomena in the social world. in other-words, qualitative research tends to work with text rather than numbers. interpretive research, on-the-other-hand, is a more specific term and is defined in terms of epistemology. following klein & myers (1999), the foundation assumption for interpretive research is that knowledge is gained, or at least filtered, through social constructions such as language, consciousness, and shared meanings. in addition to the emphasis on the socially constructed nature of reality, interpretive research acknowledges the intimate relationship between the researcher and what is being explored, and the situational constraints shaping this process. in terms of methodology, interpretive research does not predefine dependent or independent variables, does not set out to test hypotheses, but aims to produce an understanding of the social context of the phenomenon and the process whereby the phenomenon influences and is influenced by the social context (walsham, 1995). in terms of phenomenological approach, in interpretive research theory is generated or induced from the data collected, thus it is grounded in the data. in comparison, theory is deduced by testing hypotheses in positivist research. ajis vol. 10 no. 2 may 2003 4 scope & organisation of this paper keeping the above definition of interpretive research in mind, the scope of this paper is to clarify (by way of example) the nature of interpretive research methods, and to advise others how to employ interpretive methods. in reference to klein & myers’ (2001) classification scheme for interpretive research in is, this paper can be identified and characterised as advancing interpretive research methods. before embarking on a research project a researcher needs to define how he/she is conducting the research, what theoretical lens is being applied, and what methods are most appropriate to collect and analyse the data. we look at these issues and then illustrate how they were applied in the particular research example. the paper is organised as follows. the first section lays the foundation by considering important factors that influence the choice of qualitative methods, and in particular, an interpretive approach for is research. the section following is applied and focuses on building a theoretical framework, and conducting interpretive process-oriented research, respectively. subsequent sections describe the particulars of data collection and analysis and associated evaluative criteria once the research method was chosen. the last section discusses implications of the methods employed and some issues faced when conducting this kind of research. factors influencing the choice of qualitative methods trauth (2001) lists five factors influencing the choice of qualitative methods in is research. the first factor is the nature of the research problem, second is the researcher’s theoretical lens, and the third is the degree of uncertainty surrounding the phenomenon. these three main factors are now illustrated by way of example in the ensuing sections. the research problem trauth argues that the nature of the research problem should be the most significant influence on the choice of a research methodology. "that is, what one wants to learn determines how one should go about learning it" (trauth, 2001: 4). i go further and state that what we want to learn will help shape the research questions posed, and the questions posed will depend on the stage of knowledge accrual about the phenomenon. these two factors may be distinct but in my view they are nevertheless interrelated. based on research documented in rowlands (2001), this section provides a narrative of how i came to commence the research, how i initially identified the research problem, and how the subsequent research questions were posed. the impetus began in 1995. for a number of years prior to the conduct of this study, i was a manager within a large training provider in new south wales, australia. toward the end of 1995, i was approached by an information technology industry training advisory board to manage a pilot program involving the introduction of an australian qualification framework level-4 traineeship in information technology. this program provided employees with a recognised qualification and competencies in networking, communications equipment, and pc hardware implementation. as part of this pilot program, i was responsible for securing the participation of industry, and managing the delivery of the off-the-job training. it was then, from a training provider’s perspective, that the vexing problem of a lack of employer participation became evident. one question that kept recurring in my mind was, given that there was (and still is) considerable demand by industry for the skills and competencies supplied by the program, and a large pool of suitable trainees to recruit from, why weren’t small and medium sized enterprises participating? eventually, this experience led to the commencement of this study, the framing of the problem, and to the later definition of the research questions. as a manager responsible for providing it training, i wanted to research this problem and hopefully contribute some of the findings back into practice. as a starting point i turned to the literature for possible answers. a review of the literature told me that there had been a number of studies undertaken and reports published providing solid data identifying key constructs and variables relating to training in small business in general. for instance, enough was known about the following kinds of questions: what is the general attitude of small business to current training reforms? what factors and contextual elements influence the decision of small firms to participate? what is the current level of knowledge in small business about formal workplace training procedures? these predominantly ‘what’ type questions indicated to me that only one half of the problem had been examined. for instance, we did not know enough about why some firms decided to participate, of those that did, how they arrived at their decision. in other-words, using trauth’s phrase ‘what i wanted to learn’ was expressed as: i wanted to know the processes small firms in the it industry went through in making their decision.. ajis vol. 10 no. 2 may 2003 5 i identified a need for a research method enabling me to explore and then provide an explanation to this problem – which involved the ‘how one should go about learning it’. from knowledge obtained in graduate courses on research methods, i chose the in-depth case study because my desire was to uncover the "story behind the factors" about the reluctance of small employers to participate with formal on-the-job training schemes. the case study has been an essential form of research in the social sciences, and has been used in research involving small business (chetty, 1996), and extensive research within organisations (cavaye, 1996). according to yin (1994), a major strength of the case study is that it allows the researcher to understand the problem, the nature and complexity of the process taking place; and valuable insights can be gained into new topics emerging in the rapidly changing field, such as training practices in the it industry. in addition, case research can contribute to knowledge by relating findings of the particular to generalisable theory. after this initial review of the literature, i was firming in my mind that the research approach most appropriate to the problem would be exploratory, most likely inductive, and as discussed later, would be a process study. at this early stage i hadn’t chosen a method of data analysis, but i knew of grounded theory and i do admit to having a personal preference for qualitative research over quantitative approaches. to recap at this stage, what one wants to learn — the subtleties of the decision-making process — determines how one should go about learning it — the case method. the next section discusses the second of trauth’s factors – theoretical lens – when choosing qualitative, interpretive methods. the researcher's theoretical lens trauth's (2001) second important influence on the choice of research method is the theoretical lens that is used to frame the investigation. by theoretical lens, trauth is referring to philosophical issues of epistemology and a choice among positive, interpretive and critical studies. for researchers, the starting point is to identify our philosophical and theoretical assumptions leading us to choose an appropriate methodology. the following paragraphs make explicit my fundamental assumptions about the nature of knowledge (epistemology), and the nature of ways of studying phenomena (methodology). like all fields of inquiry, organisational study is paradigmatically anchored. an interpretive paradigm is based on the view that people socially and symbolically construct their own organisational realities (berger & luckman, 1967). by adopting an interpretive approach, i also assume that the participation decision-making process and the perceived meaning of on-the-job training schemes are not objective phenomena with known properties or dimensions. the research approach, accordingly, is consistent and compatible with the epistemological and ontological assumptions that the world and reality are interpreted by people in the context of historical and social practices. that is, experience of the world is subjective and best understood in terms of individuals' subjective meanings rather than the researcher’s objective definitions. by choosing the assumption of subjectivity and interpretivist methods for research, this example claims that the aspects of the phenomena under investigation – the participation process – are too complex to define and measure with standard instruments. in order to gain greater knowledge about owner managers' willingness to participate with on-thejob training schemes, the example proposes a method capable of capturing social meanings of participation, as generated by owner/managers of small firms. the proposed method is the interpretive case study. researchers working with the case method can employ either a positivist or interpretivist approach, or a combination of both, irrespective of whether the intent is to test or develop theory (markus, 1994; cavaye, 1996). however, interpretivism and positivism rely on quite different assumptions about the nature of knowledge and demand different approaches to research. positivism for instance, assumes training initiatives to be an objective, external force that have relatively deterministic impacts on organisations. the researcher is seen to play a passive, neutral role in the investigation, and does not intervene in the phenomenon of interest. positivist studies are premised on the existence of a priori fixed relationships within phenomena, typically investigated with structured instrumentation. such studies serve primarily to test theory, in an attempt to increase predictive understanding of phenomena. positivist studies are characterised by the inclusion of formal propositions, quantifiable measures of variables, hypothesis testing, and the drawing of inferences about a phenomenon from the sample to a stated population. in contrast, an interpretive study – such as this research – focuses on the human action aspect of training initiatives, seeing participation as a product of interpretations, interventions and individual decisions. interpretive researchers thus attempt to understand phenomena through accessing the meanings that participants assign to them. in direct contrast to the positivist studies, interpretive researchers reject the possibility of an ‘objective’ or ‘factual’ account of events and situations, seeking instead a relativistic, albeit shared (between the researcher and the interviewee) understanding of phenomena. generalisations from the setting, usually from a small number of case studies, to a population is not sought; rather, the intent is to understand the deeper structure of a phenomenon, which it is believed can then be used to inform other settings. for a more detailed account of the positivist and interpretivist research philosophy, see orlikowski ajis vol. 10 no. 2 may 2003 6 and baroudi (1991), lee (1991), klein and myers (1999) or the mis quarterly website at http://www.qual.auckland.ac.nz/ and click on ‘philosophical issues’. because of the importance attached to understanding decision-making from the perspective of owner/managers and the absence of specific prior research describing the processes involved in training initiatives participation; the research approach used was both interpretive and primarily inductive. according to lincoln & guba (1985) the use of an inductive approach is appropriate when studying organisational phenomena from a primarily interpretive perspective. this perspective regards humans as active agents and their behaviour as generally indeterminate. unlike physical stimuli, the training initiative scheme may produce novel interpretations by owner/managers as they are applied in this particular social context. in summary, the emphasis in this research example was on interpreting how owner/managers understand their situation, their attitudes towards training initiatives and their relationships with training providers. the researcher’s theoretical lens – involving both perspective and method – is in the realm of interpretive and qualitative research. trauth’s third factor, the amount of uncertainty surrounding the phenomenon under study, is discussed next. degree of uncertainty surrounding the problem while i had made clear my own epistemological preference in the previous section, certain contingencies of the problem — such as degree of uncertainty surrounding the topic — also confirmed my qualitative, inductive, interpretive stance. for example there was little prior research investigating the conditions under which small businesses were prepared to engage in formal training schemes, or the issues involved in their decision-making process. of the research that had been undertaken, the dominant paradigm (or theoretical lens) had been positivist with an emphasis on factor analytic studies and surveys as the main methods of analysis and data collection. the preliminary review of the literature identified previous research to be overreliant upon mail surveys and telephone interviews with factor analysis as the main data analysis technique. the positivist lens told us that various factors were influential — "the what", but it could not tell us “why” managers participated as they did. it could not provide us with an in-depth look at the worldviews that sat behind the facts shared by the owner-managers. there was very little attention given to the intentions, actions, context or processes surrounding participation that explained how these issues interact and how and why participation outcomes were associated. in light of the paucity of previous research on the process of participation, the research example provided an alternative perspective to an emerging research topic. i argued that without more emphasis on the dynamic nature of the participation process, an incomplete understanding of the uptake problem would result. i argued further that more attention should be paid to the development of new theory more fully specified through grounded research that are better able to account for the phenomenon under investigation. in pursuit of these dual objectives, the outcome focus of this study was aimed at theory building, not theory testing, for the purpose of describing and explaining the participation process. in sum, the degree of uncertainty surrounding the problem — limitations in the literature and the nature of the problem — influenced me to choose an inductive approach and grounded theory techniques for data analysis. grounded theory will be explained in the research procedures section. to conclude this section, i have reviewed and applied three main factors as espoused by trauth (2001) influencing the choice of qualitative methods for is research. the three factors are: the nature of the research problem, the researcher’s theoretical lens, and thirdly, the degree of uncertainty surrounding the problem. i also introduced the interpretive case study as a method for seeking a shared (between the researcher and the interviewee) understanding of phenomena. before ending this section it needs to be acknowledged that trauth (2001) identified two additional factors that influence the choice of qualitative methods for is research. the 4 th factor is the researcher’s skills. i have already alluded to my preference based on prior research training to using qualitative methods. the 5 th factor – academic politics, was not an issue as the research centre i was working in expressed no preference involving the choice of positive, interpretive or critical methods. however, each researcher’s circumstances are different and these two additional factors may well prove to be relevant when choosing an ‘appropriate method’ for is research. further issues that accompany these influences are at the core of this paper. the following section focuses on three further practical issues that were involved in the conduct of the empirical research. these issues involved (1) how i conceptualised the problem, (2) an emphasis on process oriented research, and (3) building a theoretical framework, respectively. ajis vol. 10 no. 2 may 2003 7 practical issues related to developing a theoretical framework conceptualising the problem given my epistemological stance, i rationalised that an interpretive analysis of the texts were needed to get at the why of the participation decision-making behaviour and the mechanics of the how within the particular context. however, this was not sufficient in itself to commence the theory building process. as part of this process, i asked myself as a researcher, how could i improve our research models, and also our methodologies and perspectives so that the results of my work would be of greater value to policy-makers and practitioners. to provide an alternate perspective to this under-researched topic, i conceptualised the problem of training participation as a process of socio-technical innovation. in undertaking this research, i qualified this judgement by reviewing and discussing definitions of innovations, technology, social technologies and argued a case for understanding on-the-job training schemes as an innovation in the process of acquiring skills within the firm. for example, i borrowed from perrow (1967) who sees organisations as places where raw materials are transformed, thus defining what is done and how it is done − the process − as the technology of organisations. this perspective of viewing on-the-job training as a process of socio-technical innovation departs from the bulk of the literature (saunders, 2001) that shared a conventional economic focus on how firms make decisions about skilling. alternatively, by focusing on the problem as a process of socio-technical innovation, i began to develop a theoretical framework comprised of individual, organisational, social, governmental and economic forces that introduced some typically unexamined aspects of participation within small firms. in building this framework, the thesis discussed theory that addressed the concepts of innovation and social technologies. one theory – the social construction of technology (bijker et al, 1987) – describes a theoretical approach to studying the meanings of technology, and how those meanings affect the adoption of technology within an organisation. a second theory – innovation diffusion theory (rogers, 1995) provided a general explanation for the way new ideas and objects spread through a social system over time. these literatures and a framework of process-oriented research (to be discussed next) provided valuable tools for the examination and analysis of the participation decision-making process. an emphasis on process oriented research the second major conceptualisation that departed from the bulk of previous research involved a focus on process. i advocated a need for process-oriented research based on mohr’s (1982) classification. mohr (1982) suggests that two fundamentally different types of theoretical approaches can be used to investigate organisational phenomena: variance and process research. the majority of prior research (smith, 1995; hayton, 1996) was of the variance persuasion with a focus on correlations between groups of variables and a specific outcome, while my approach involving process research aimed to understand the sequence of events leading to some result over time. to understand more about process-oriented research wolfe (1994) differentiates two generations of process research. earlier work, called stage model research, conceptualised innovation as a series of stages that unfolded over time. the purpose of this early work was to determine whether the innovation process involved identifiable stages, and, if so, what they are and in what order. the second generation of process research involves in-depth, longitudinal, research conducted to fully describe the sequences of, and the conditions which determine, innovation processes. this type of research often involves theory building and qualitative data collection. examples of process research that i reviewed following this research stream included: eisenhardt (1989a), wolfe (1994), langley & truax (1994), markus (1994), and orlikowski (1996). these studies tend to be inductive, in-depth, examinations of how innovations develop over time. methods employed include historical analysis of archival data and published reports, interviews, questionnaires, and field observations. the form of process modelling adopted in my research example is that of second generation process theory, where the objective is to provide a better understanding of how and why the “pieces of the puzzle” interact and work together to produce a participation decision. building a theoretical framework the last practical issue relates to building a theoretical framework. a theoretical framework consists of a selection of concepts and relations among them, grouped so as to enable its users to easily see their structure (whetten, 1989). to borrow again from whetten, a theoretical framework can be understood by considering four building blocks, or four essential elements. each element is described briefly in the following paragraphs. ajis vol. 10 no. 2 may 2003 8 the first building block of a theoretical framework, “the what”, refers to the elements (variables, constructs, concepts) that should be considered as part of the explanation of the phenomenon. as reported from an initial review of the literature, some conceptual and empirical research had provided researchers with a preliminary list of factors believed to be critical to participation. however, the main contributions of this research were situated within the second building block of theory development, to determine conceptually “how” and “why” the elements relate to each other. having already argued that participation cannot be adequately explained by considering or manipulating one or two factors (the positive, dominant economic perspective), i mounted a case for a new perspective based on the concepts of socio-technical innovation, decision-making and process. the third building block refers to a theory’s assumptions – that is, the theoretical glue that welds the model together. answers to the “why” component push back the boundaries of our knowledge by providing compelling and logical justifications for altered views. “only when a researcher can specify his (sic) logic, then he can follow certain rules in determining the propositions he can make about his theory” (whetten, 1989). in a previous part of this section i discuss the use of 2 nd generation process theory (wolfe, 1994) as a meta-theoretical framework for studying the participation decision-making process. this meta-theory was used as part of my conceptual lens for describing and understanding the participation decision-making process. the last building block places limitations on the propositions generated from the theoretical model. specifically, the “who”, “where”, “when” elements set the boundaries of generalisability, and as such establish the range of theory (whetten, 1989). in this example, the research attempted to go beyond previous research by developing an initial set of theoretical propositions (sharpened by recourse to the full literature) regarding the dynamic nature of training initiatives participation in the it industry in se queensland, australia. inductive versus deductive research in applying these ‘four building blocks’, in the development of my theoretical framework, i had to make a major decision of approach. for example, most qualitative researchers attempt to avoid prior commitment to theoretical constructs before gathering any data (yin, 1994). yet, as discussed by whetten (1989), two different approaches may be taken, or combined. in the first, the researcher works within an explicit theoretical framework. therefore, a theoretical framework becomes a researcher’s first cut at making some explicit theoretical statements (miles & huberman, 1994). this approach is known as deduction. in the second, the researcher tries not to be constrained by prior theory and instead sees the development of relevant theory, propositions, and concepts as a purpose of the project. this approach is generally known as induction. in my research example, both approaches were combined since the main intent was to study a relatively unresearched topic (the participation decision-making process involving small businesses and training initiatives in the it industry) from a different perspective (training initiatives being defined as socio-technical innovations) within the bounds of an already well-established research program (technology adoption). yes, it is possible to be inductive, but i chose not to ignore previous work in the field. i developed a loose conceptual model built on over ten years of research that was a conceptual advance on the literature, especially australian work. the model of the participation context within which small it firms operated was, i proposed, comprised of at least six sets of issues: (1) the impact of the external environment such as the economy and government; (2) small business and adopter industry characteristics; (3) owner-managers attitudes (individual); (4) characteristics of the innovation (the training scheme) itself; and (5) supplier factors, such as the impact of training agencies/sponsors and training providers. these issues were also used to develop the initial coding scheme for the qualitative analysis of data (to be discussed in the research procedures section). (6) the sixth, and pivotal component of the model, the participation decision process, was developed and presented in the findings section of the thesis, but does not form part of this paper’s discussion. however, and this is a most important point, given that this study was aimed at theory building, not theory testing, the theoretical framework and model was used solely as a guide. it helped make sense of what occurred in the field, ensured that important issues were not overlooked, provided a set of tentative constructs to be investigated, and guided my interpretation and focus. to summarise at this point, this section introduced the notion of a theoretical framework and second generation process theory to frame the empirical work. second generation process theory provided some methodological guidelines and a meta theoretical framework for understanding process, participation and innovation, from the perspective of owner managers. it also offered a perspective from which i could look at multiple local meanings of participation and proceed to understand how these meanings become crystallised and subsequently influence individual and organisational-level action. this use of the theory accords with klein & myers’ (2001) recommendation that the empirical research needs to be guided by (or at least informed by) one or more social theories. ajis vol. 10 no. 2 may 2003 9 having developed a theoretical framework and a conceptualisation of the problem as a process of sociotechnical innovation, i was then in a position to pose the specific research questions and to specify the research procedures in more detail. in the next section, the paper states the research questions and discusses a number of methodological issues associated with conducting the inductive process so central to interpretive research. methodological issues this section firstly justifies the use of grounded theory (glaser & strauss, 1967) when conducting process oriented research and describes a number of distinguishing features that characterise this approach – inductive, contextual, and processual – that fit with the primarily interpretive rather than positivist orientation of this research. next, the overall research design involving phases adapted from key exponents of the case method are detailed. a model depicting three main phases of inductive theory building using multiple cases presented in figure 1. the section concludes with a discussion of credibility and reliability issues as part of the research design. the primary objective of this evaluative component was to manage potential threats to the validity of the results given the ‘subjective’ nature of data collection. reasons for using grounded theory techniques i chose grounded theory (gt) techniques to analyse my case study interview data because, according to strauss & corbin (1990), grounded theorising is well suited to capturing the interpretive experiences of owner/managers and developing theoretical propositions from them. in the same line of thought, an application of gt is appropriate when the research focus is explanatory, contextual, and process oriented (eisenhardt, 1989b). similarly, gt has been effectively used in recent is research (jones & hughes, 2001; galal 2001; urquhart, 2001) to develop theory of is practice. the research procedures section provides more details of the ‘how to’ of coding and grounded theorising, while readers are encouraged to consult urquhart (1997) and urquhart (2001) for specifics of some practical and philosophical issues associated with its application. in brief, the methodology of grounded theory is iterative, requiring a steady movement between concept and data, as well as comparative, requiring a constant comparison across types of evidence to control the conceptual level and scope of the emerging theory. to facilitate this iteration and comparison, eight field sites were studied, with the research design expanded upon in the next section. the research design the research design for theory building is illustrated in detail in figure 1. figure 1 is an adaptation from yin (1994) and follows eisenhardt’s (1989b) replication approach to multiple case studies. ajis vol. 10 no. 2 may 2003 10 figure 1 the specific research plan, adapted from yin (1994) & eisenhardt (1989b) the first step involved stating the research questions and defining a preliminary conceptual model that initially, told me where to look for relevant evidence. the role of loosely specifying a conceptual model prior to the conduct of any data collection is a major point of difference between the research adopted compared to a purely inductive approach. in fact, i chose this approach essentially because it provided a more “tighter” design, in keeping with the following contingencies. firstly, i had a good prior acquaintance with the research problem; i knew something conceptually about the phenomenon – in this case a bank of concepts providing at least a rudimentary conceptual model of the participation process. there was a significant body of literature on skill formation practice in small firms and theory on technology adoption, but not enough to house a theory of training participation. i had some idea about how to gather the information and perhaps which questions to ask, and which incidents to attend to closely. as miles & huberman (1994:17) state, “not to ‘lead’ with your conceptual strength can simply be self-defeating”. yet, as these authors point out, no matter how “tight” the design is bounded, focused and organised, qualitative research designs are not copyable patterns or panaceas that eliminate the need for building, revising or ‘choreographing’ analytic work. according to miles & huberman (1994) tighter designs are a wise course providing clarity and focus for beginning researchers worried about diffuseness and overload. to conclude this section on methodological issues, the final paragraphs describe the issues involved and procedures used to evaluate the credibility of the subsequent research findings. interpretive concepts of credibility the traditional criteria used for evaluation of research (internal validity, external validity, and reliability) are difficult to apply to interpretive research, and are particularly problematic in case research. consequently, different standards and criteria have been developed to apply to research outside the positivist tradition. alternative criteria appropriate to this approach are credibility, transferability, and dependability (hirschman, 1986). according to lincoln and guba (1985) these three concepts have an evaluative role in interpretive research analogous to that of the concepts of internal validity, external validity, and reliability, in positivist science. the requirement, then, was to demonstrate that the descriptions of the different social interpretations were derived in a credible manner. a number of strategies were employed to ensure credibility. the first tactic consisted of developing a logical chain of evidence. in this study, such a chain was established firstly by having sufficient citations in the full report relating to the relevant portions of the case study data base. secondly, the chain was established by developing a case study protocol in which all firms and all interviewees were subject to the same entry and exit procedures and interview questions, and by creating similarly organised case databases for each firm interviewed. a second tactic was to submit the interpretations to the scrutiny of the individuals upon whom they are based, and to seek their responses to its authenticity – known as member checking. a third tactic was the use of comparative or collective case analysis. multiple phase 1 define and design phase 2 data collection & within-case analysis phase 3 cross-case analysis step 1. select study area, describe questions and loose conceptual model. step 2. identify firms and select cases step 3. design data collection protocol & instruments step 4. enter field, and conduct case studies step 5. analyse data draw cross-case conclusions, and within-case analysis step 6. shape propositions confirm, extend & sharpen theory step 7. enfold the literature. build credibility & transferability step 8. reaching closure step 4 continued write individual case reports x 8. analyse data. . •identify firms in both categories •obtain access • interviews •transcribe cases • coding • develop new categories and properties of data related to concepts ajis vol. 10 no. 2 may 2003 11 case studies enabled a higher degree of corroboration of findings to take place. the last tactic involved the researcher clarifying his assumptions, worldview, and theoretical orientation at the outset of the project, as suggested by merriam (1988). the second criterion, transferability, deals with the problem of knowing whether a study’s findings are generalisable beyond the immediate set of cases. interpretive studies, however, do not seek to produce results that are universally applicable. this research only attempted to generalise a particular set of results to some broader theory or research proposition. secondly, transferability can be viewed as reader or user generalisability, where the extent to which findings can be applied to another situation is determined by the people in these situations (merriam & simpson, 1995). consequently, it is not up to the researcher to specify how findings can be applied; it is up to the consumer of the research. the third criterion, dependability, relates to repeating the operations of the study with similar expected results. in social science, the notion of reliability is problematic because human behaviour is never static, nor is what many experience necessarily more reliable than what another person experiences (merriam & simpson, 1995). in other words, there can be numerous interpretations of the same data. in interpretivist inquiry therefore, the issue is whether the researcher’s judgements are dependable or consistent with the available data and free from bias and errors. in this research example, i sought dependability (reliability) through the use of three tactics. first, a case study protocol, containing each of the interview guides was used. second, a case study database was maintained. each of the eight cases (plus the pilot case) contained the following elements, as recommended by miles & huberman (1994): (1) raw materials (including interview transcripts, researcher’s field notes, other documents collected from the field); (2) partially processed text (including edited transcriptions and “commented-on” versions); (3) coded text (write-ups with specific codes attached); (4) coding scheme; (5) memos and other analytical material (researcher’s reflections on the conceptual meaning of text); and (6) data displays (matrices used to display retrieved information). third, an audit trail showed how the data were collected, how categories were derived, and how decisions were made. in conclusion, this section has discussed methodological considerations in researching the process of participation. in particular, the nature, the utility, and the appropriateness of the case method, grounded theorising and the research design have been explained. given the broad and complex nature of the research questions to be asked (see next section research procedures), the case study was chosen as it is a particularly useful open methodology for exploring an area of practice not well researched or conceptualised. the specific procedures involved in data gathering and analysis, and the inductive process of grounded theorising for the purpose of theory building is described in detail in the following section. research procedures this section describes steps 1 to 8 and some of the procedures in detail for generating a set of propositions based on the research plan depicted previously in figure 1. the section describes how the participants were selected, how the data was collected, and how the data was managed, analysed, and displayed. getting started three procedural issues were of great importance in starting the research: (1) the initial definition of research questions; (2) the choice regarding a priori specification of constructs; and (3) the consideration of a priori theory. each of these issues is examined in turn. first, the research questions provided the focus. the aim of the study was directed toward exploring and describing the decision making process of owner/managers regarding their participation with on-the-job training schemes for the first time, and to develop process theory explaining their participation. in pursuit of this aim, three interrelated research questions were initially stated: • q1. what issues and conditions influence the decision of small it businesses to participate with onthe-job training? • q2. what are the processes small it firms go through when participating with on-the-job training schemes for the first time? • q3. how can these processes be depicted in a model? the research questions examined in the literature review provided a guiding focus to the research and permitted the specification of the kind of data to be gathered. this approach conforms with that of strauss & corbin (1990:50-51) who suggest that literature from the field be used not to develop hypotheses, but to stimulate theoretical sensitivity by providing concepts and relationships that are checked out against actual data. with respect to the second issue of using existing theoretical constructs to guide theory building, a loose conceptual model and its constructs were only used as a starting point. the conceptual model was intended to make sense of the cases, ensure that possible issues were not overlooked, provide a set of constructs to be ajis vol. 10 no. 2 may 2003 12 investigated, and guide the author’s interpretation and focus. however, as stressed by eisenhardt (1989b), although early identification of possible constructs allows them to be explicitly studied in interviews, it is equally important to recognise that the constructs are merely tentative in the theory-building process. in this research, this was found to be true as new issues were identified during data collection that needed to be added to the analysis. the third issue, the consideration of a priori theory is now discussed. an objective of this research was to develop a process theory of training participation. one approach to theory building research begins as close as possible to the ideal of no theory under consideration since preordained theoretical perspectives may bias and limit the findings. however, as stressed by many, it is quite impossible to achieve the ideal of a clean theoretical slate. hence, although the research did not identify specific relationships between the constructs identified in the conceptual model, i found it helpful to make use of a meta-theory called 2 nd generation process theory (wolfe, 1994). this theory reflected my basic assumptions about the nature of the phenomena being studied (innovation, social technology), assumptions that were later supported by strong evidence in the data. the application of 2 nd generation process theory was of great assistance in focussing the research efforts at the outset of the study, as it provided a frame from which the author could 'observe' the participation process and identify the key events of interest out of numerous ones that had or were occurring. selecting the participants i made two major decisions in terms of selecting the participants. first, the population of interest was specified. the population of interest were owner/managers of small firms in the it industry who had recently decided to participate with a formal on-the-job training scheme for the first time, or those that had been approached to participate but declined. the cases were purposely selected based on the researcher’s knowledge of the industry, and from discussions with industry figures. the second critical decision was to know when to stop adding cases to the study. one approach for deciding when to stop adding cases is to conclude the field research when theoretical saturation is reached (eisenhardt, 1989b; strauss & corbin, 1990). this approach may be the ideal situation but is difficult for researchers (like doctoral students) who faced the real constraint of time schedules and funding. hence some researchers must develop alternate approaches to answer the question of when to stop adding cases. an appropriate alternative stresses the importance of representing the variety found in the population rather than reproducing the proportions of characteristics found. this approach embodies the concepts of replication logic (yin, 1994). as suggested by smith (2000:78) in research at the organisational level, the researcher includes in the sample a variety of cases operating under different conditions to ensure the theory developed is robust and provides explanations of phenomena across a number of different settings. it is the variety of conditions found under replication logic that permit the generation of theory capable of explaining the diversity of situations typically found in organisations. following glaser and strauss' (1967) technique of theoretical sampling, eight organisations were selected for their similarities as well as their differences. because the purpose of the research was to generate theory applicable to various organisational contexts, differences were sought in organisational type such as the participatory mode – half were participants, and the other half declined participation. these differences allowed useful contrasts to be made during data analysis, which challenged and elaborated the emerging concepts. data collection instruments in my role as an outside observer, the semi-structured face-to-face interview was chosen as the primary data source, with observations and documents being minor sources of data. importantly, the reliance on multiple data collection methods increased the robustness of results through triangulation. as stressed earlier, the primary goal of the interviews was to elicit the respondent’s views and experiences in his or her own terms. as such, the research used the natural setting as the direct source of data and the researcher as the key data-gathering instrument. the data gathering task was to access other people’s interpretations, filter them through the researcher’s own conceptual apparatus, and then feed a version of the events back to others. next, the primary form of data collected – critical incidents, and each of the data sources used in this study are discussed. critical incidents are brief descriptions written by or described by individuals about significant or key events that relate to a particular topic. this technique has been used widely in organisational research. miles & huberman (1994:115) cite the use of critical incidents and explained that ‘sometimes a researcher wants to limit an event listing to those events seen as critical, influential, or decisive in the course of some process’. several advantages accrued from using this method of collecting data. respondents were asked to ajis vol. 10 no. 2 may 2003 13 talk about specific situations, events and people. the advantage was that the interviewee could focus and reflect on the described incident posed in the interview. a collection of these responses allowed the researcher to analyse meaningful data that was grounded in the actual experiences, needs and concerns of participants. interviews with owner/managers dealt with the following issues: reasons or motives for adopting or rejecting the training scheme; the conditions that shaped their participation; the participation process itself; and the relationship between the small business and the off-the-job training provider. the average length of each interview was approximately 1 ½ hours for participants, and about one hour for non-participants. interviews were not taped (a unanimous respondent request) but were transcribed by hand in all cases. some respondents were interviewed a second time to follow-up on important issues that became evident in the data transcription, data analysis phase. the hand-written transcripts and researcher comments were typed and sent back to the interviewees for accuracy and checking within three days. when accuracy of transcription was confirmed with the respondents, the case evidence was deemed suitable for analysis. the interviews produced a detailed case study report for each firm. analysis of data analysing data is the heart of building theory from case studies, but it is both the most difficult and least codified part of the process. qualitative studies tend to produce large amounts of data that are not readily amenable to mechanical manipulation, analysis, and data reduction. the problem then, was how to manage the data. the solution became the search for coherence, order and regularity. inspired by the work of miles and huberman (1994) for data presentation and strauss and corbin (1990) for an application of grounded theory, the approach to data analysis included three steps: early steps in data analysis, within case analysis and cross case analysis. the early steps in analysis include use of the contact summary form for reviewing the interview, the development of a computerised database for storage and easy retrieval of data, the arranging and displaying of data in tables, and the development of a coding scheme to organise the data. within case steps involved detailed write-ups for each case assisted by the identification of critical incidents, a time line displaying stages of the participation process, the development of a logical chain of evidence, and the writing of a narrative story. cross case analysis involved the search for cross-case patterns by combining information from several cases into a single table. from that, a new set of process oriented codes were developed, using a form of content analysis known as ‘open coding’ and ‘axial coding’ making connections between sub-categories of data into a more comprehensive set of concepts. the analytical techniques adopted during each of these three phases are explained below. early steps in analysis this section presents the analytical techniques adopted in the early stages of data analysis. several publications give detailed instructions for organising and analysing data (merriam, 1988; strauss & corbin, 1990; miles & huberman, 1994); and mcintyre (1998) for specifics on using computer databases for qualitative analysis. miles and huberman (1994) was drawn upon extensively for techniques to process the qualitative data. one of their techniques used in this study was to arrange empirical evidence in tables in the form of words rather than numbers. by looking at these tables, within-group similarity and across-group differences could be ascertained. a second set of techniques were those recommended by mcintyre (1998) who highlighted data management issues, and applied database software to the interpretive analysis of qualitative data. in this research, the process of analysis was assisted by and recorded in a developing database through procedures such as entering chunks of data in fields, setting up other fields in which comments could be added, coding and sorting the interpretations; and text retrieval of selected records into the body of the research report. a third technique, combining data collection and analysis was the use of the contact summary form. this form was useful in revising the case interview, and once transcribed, reflective remarks were recorded about the main issues identified, and initial thinking around the research questions. these remarks were ways of getting ideas down on paper and of using writing as a way to facilitate reflection and starting the analysis process. use of the contact summary form commenced analysis by presenting an overall evaluation of the case scenario. the initial analysis was performed manually, involving writing up the contact summary sheet with some further development of the coding scheme. the transcripts from each interview were then coded and analysed for emerging themes. the fourth technique was coding. a code is an abbreviation or symbol applied to a segment of words (e.g. sentence, paragraph). codes served as retrieval and organising devices which allowed the rapid retrieval and clustering of all the segments related to a particular question, concept or theme. according to miles & huberman (1994), coding is analysis; and coding in qualitative research involves segmenting the data into ajis vol. 10 no. 2 may 2003 14 units, and then rearranging them into categories that facilitate insight, comparison, and the development of theory. as suggested by miles & huberman (1994:58) i created a provisional ‘start list’ of codes prior to fieldwork. most of the initial coding categories were drawn from the loose conceptual model, the list of questions, and key concepts the researcher brought to the study. to be consistent with the conceptual model developed in phase 1 of the design, the preliminary descriptive coding scheme developed in this study was divided into five broad categories: environmental, organisational, individual, object and supplier. the original list was then used to codify and extract the data from the transcripts associated with the pilot case interview. as a result of this process, i found the need to add new codes. this same format was carried through the entire data collection process (across all 8 cases) and new codes were developed for emerging themes. immediately after the transcripts were verified by the respondents they were again read carefully and relevant portions highlighted. the highlighted portions were then keyed into the database into the field called ‘excerpt’ as chunks of rich text. all of the transcripts, starting with the first interview, were coded using the preliminary set of codes developed from the pilot case. records were labeled by case along with the identifying transcript page and question number. occasionally, a segment of the transcript resulted in the creation of a new code, or the refinement of an existing code or even the amalgamation of codes with similar meaning. the development of the coding scheme was an on-going process throughout the transcription of each of the eight cases. in fact, the formal cataloguing of “instances” into conceptual codes and categories was undertaken concurrently while the data were being collected and entered into the database. thirty-six resulting codes within six major categories emerged from the analysis of the entire eight cases. in addition to descriptive codes, the study identified and defined pattern or inferential codes during data analysis. pattern codes are those that identify an emergent theme, pattern or explanation that the site suggests to the researcher; and is for qualitative researchers, an analogue to the cluster-analytic and factor-analytic devices used in statistical analysis (miles & huberman, 1994). pattern coding served two main functions in this study. first, it reduced large amounts of data into a smaller number of analytic units and, second, it helped the researcher build a cognitive map, an evolving schema for understanding what was happening in each case. using the critical incident as a base, the participation issues and decisions were also analysed for emerging patterns and themes. some of the issues and decisions proved to be more critical than others and some were a springboard for other decisions made at both the stage to adopt, and during the implementation stage of the training program. these issues were marked as a critical event and coded for further analysis. the emergence of the critical events adoption process (ap) theme and the detection of patterns expanded the initial coding scheme from five major categories to six. field notes were another important means of managing complexity in this study. as described by van maanen (1988), field notes are an ongoing stream of conscious commentary about what is happening in the research. by reviewing my field notes regularly (as written on the contact summary form), important issues or conflicting responses provided by different individuals were identified immediately. second, after the interview was transcribed (and verified by the respondent as accurate), reflective remarks were directly entered into the database record within the field called ‘interpretation’. figure 2 below shows a sample data entry screen including excerpts, coding and reflective remarks. using these remarks was my way to facilitate reflection and analytic insight. as suggested by miles & huberman (1994), they are a way to convert the researcher’s perceptions and thoughts into a visible form that allows reflection. in short, the reflective remarks helped me to make deeper and more general sense of what was happening, and to explain things in a conceptually different way. ajis vol. 10 no. 2 may 2003 15 figure 2 sample data entry screen showing excerpts, codes & reflective remarks to summarise at this point, a distinguishing feature of research to build theory from case studies is the frequent overlap of data analysis with data collection. in the early phase, the afore-mentioned techniques were used separately and in combination to help the researcher identify themes, develop categories, and explore similarities and differences in the data, and relationships among them. within-case analysis the second phase, within-case-analysis, typically and necessarily involved detailed write-ups for each case. these writings were often simply pure descriptions, but were central to the generation of insight because they helped me cope early in the analysis process with the enormous volume of data. however, there is no standard format for such analysis. the procedures followed to analyse each case are summarised in table 1. table 1 within-case analysis procedures step 1: development of a file maker pro database 1.1 codify and extract data from the transcripts using the validated coding scheme 1.2 group extracted segments under categories (codes and pattern codes) 1.3 read and add reflective remarks and observational notes to database records step 2: displaying data: developing summary tables 2.1 factors cited for participation/non participation in on-the-job training were displayed in tables. 2.2 critical incidents and a time line describing stages of the participation process were displayed in a table. step 3: development of a logical chain of evidence 3.1 identify set of distinct evaluative elements 3.2 identify initiating opportunities from in-depth analysis of case accounts 3.3 establish the logical chain of evidence between the participation / nonparticipation process and the conditions above 3.4 provide a decision 'story' that explains the extent to which each challenge was overcome ajis vol. 10 no. 2 may 2003 16 as a first step the researcher developed a set of computer records in a database for each participation and rejection case. the database organised and documented the data collected for each case. the eight case files generated records containing the following elements: excerpt, codes, themes, interpretations as reflective remarks and other identifying fields such as case number, research question, industry etc. a sample data entry and retrieval screen from the database is shown in figure 2. the second step employed some of the displays proposed by miles & huberman (1994:91) as one approach to managing the problem of huge volumes of data. these authors define a display as a “visual format that presents information systematically, so the user can draw valid conclusions and take needed action”. the within-case displays adopted in this research include the critical incident chart, and the time ordered matrix. displays such as these made ideas visible and permanent. they also served two other key functions: data reduction, and presentation of analysis that allowed it to be grasped as a whole. in order to streamline the data analysis process and still obtain relevant information, a table highlighting critical incidents and a timeline were developed. the objective was to describe the factors and issues that influenced the decision process, and secondly to describe the critical events that took place during the decision process to preserve chronology and illuminate the processes occurring. critical incidents provided a focused way to examine the decision making process. the critical incident time line was developed for each participating case and chronologically arranged as a series of decisions made by owner/managers during the participation process. these critical incidents enabled the researcher to focus on the stages of the decision making process. the participants were asked to identify and describe the decisions and sequence of events that were made about the traineeship program during the time before participation. to help the participant reflect upon this stage, prompts of ‘early’, ‘middle’, ‘late’ and ‘now’ were added to the time line. the idea to add these prompts came after the pilot study was conducted to help the participants recall the information more clearly and provide as much detail as possible about the decisions made during this stage. it also helped myself to construct the chronological flow of information. miles & huberman (1994:115) advocate the use of a time line to supplement the critical incidents and provide a basis to “… examine and compare events that occurred during a given time period”. in order to complete the third step of within-case analysis, to understand the ‘how’ and the ‘why’ associated with each case and hence to provide answers to the research questions, a logical chain of evidence (yin, 1994) was established. this chain of evidence involving participation reasons and contextual conditions was built in sequence. step 3.1 involves identifying and then describing a set of distinct evaluative elements for each case. each element is described in turn. the first element – background required details of the it industry and general economic conditions. facilitating conditions required a description of favourable contextual condition(s) which enhanced or strengthened the participation/ rejection factors and hence increased the chances of the participation process. opportunity meant a favourable event, which, if seized, could help fulfil an envisioned goal. challenge implied an envisioned goal or unexpected endeavour whose fulfilment pushed the process closer to success. tactic implied an event representing an action or decision or a series of actions and decisions aimed at preparing the small business for the training scheme and introducing it in such a way as to ensure its success. unexpected event describes an event, which affected the anticipated progress of the training scheme. unfavourable condition included contextual condition(s) which impeded or hindered the participation process. compensatory mechanism refers to a contextual condition compensating for a strategy violating one of the accepted beliefs concerning ‘known factors’ impacting on technology adoption. outcome, the end result of the process. the second sub-step (3.2) identifies the initiating opportunities encountered during the participation process. these were identified through an in-depth analysis of the interviewee’s accounts, and the context or background surrounding each case. because this was the first time an on-the-job training scheme had been adopted in each firm, the challenge in each case was to describe the tactics adopted to cope with the encountered problems, anticipated or not, and the facilitating and unfavorable conditions. each of the elements in this chain were identified through accounts in the transcripts (having previously been coded and entered in the database). sub-step 3.3, involved documenting the chain of evidence by having sufficient citations in the report linked to the relevant portions of the case study database, and the initial research questions. in other words, clear crossreferencing to methodological procedures and to the resulting evidence. fourth, sub-step 3.4 entails combining the qualitative responses into narratives or decision 'stories'. these 'stories' overview and correlate the chain of evidence. throughout the story, various displays are used to summarise the participation issues cited by the owner/managers. decisions made during the participation process are explained and displayed in chronological order in a time line. in the time lines, the chart indicates the temporal order of events, choices and activities and has been decomposed into periods (‘early’, ‘middle’, ajis vol. 10 no. 2 may 2003 17 ‘late’, and ‘now’). however, these time-line periods should not be viewed to be synonymous with any of the phases found in the decision-making literature — they were merely a descriptive organising device for rendering a complex sequence of events more comprehensible. however, the periods do have conceptual significance in the resulting model, which is explained further in rowlands (2001), in that the activities involved in the early phase were concerned with forming a favourable commitment to on-the-job training. furthermore, the middle and later phases were more concerned with financial and operational detail respectively. cross-case analysis the last analytic strategy, cross-case analysis (involving phase 3 of the design) is now described. first, the theory building process commenced with the development and presentation of an initial conceptual model based on evidence from the literature, the coding scheme resulting from the pilot case, and the theoretical assumptions associated with 2 nd generation process theory. the conceptual model then became a vehicle for generalising to the other eight cases. second, the design of the research lent itself to cross-case analysis of data and the search for patterns. replication logic (which formed a basis for the selection of the cases) became the key to the rigorous analysis of the cross case data. rowlands (2001) documents cross-case analysis of the four participating firms and made comparisons with the four non-participating firms. a number of analytic techniques suggested by miles & huberman (1994) were used to cross-analyse the data. the most basic way of cross-analysing the data from several cases is with the unordered descriptive metamatrix. this device assembles data from several cases in an efficient, manageable format providing ‘inclusion’ of all the relevant information. secondly, the table tabulates the frequency of events and as such draws rapid attention to the dominant issues, keeping the researcher analytically honest while protecting against bias. table 2 below provides an example descriptive meta-matrix that was used extensively in the research. the use of the unordered descriptive meta-matrix establishes a grounding of the data. ajis vol. 10 no. 2 may 2003 18 table 2 example unordered descriptive meta-matrix: participants citing organisational factors as relevant to their participation decision. organisational factors (and code) 1 2 3 4 5 6 7 8 total phase of business in start-up mode [o-start] 0 the need for an additional worker [o-recruit] � � � � � 5 cost of involvement [o-cost] � 1 a way of acquiring extra skills for staff and the business [otraining] � � � 3 the owner had time & resources to train [o-resource] � � � � 4 accustomed to doing a lot of training on the job [o-fit] � � � � � � � 7 at the conclusion of working through each of the cases as described above in meta-matrix format, the analysis then focused on developing process codes wholly grounded in the research data involving cross-case comparisons. while the excerpts in the transcripts had been coded according to the codes described earlier, a new set of process-oriented codes were derived and entered in a file maker pro field called theme. the triggers which initialised the dynamic scanning of transcripts for significant data items were the critical incidents from the interview questions, the time-lines and logical chains of evidence developed for each case. the temporal sequence of classifying the data became a key organising structure in this process, and using the notions of “early”, “middle” and “late”, the analysis process started a formal full-scale cataloguing of dataitems under various higher-level conceptual categories. this intensive back and forth analysis of the entire set of database excerpts, along with a continually growing body of results, finally results in producing eight major process categories (c.f. table 3). table 3 descriptive titles for the process categories identified category descriptive title information information on on-the-job training schemes from various sources. psych-inhibitor internal or external events inhibiting interest in on-the-job training schemes. sensitisor a positive attitude towards young people in general and a willingness to train on the job. impetus internal or external events precipitating serious consideration of on-the-job training schemes. diagnostic-cost defining and confirming costs and benefits. feasibility consideration of feasibility and impact of new apprentices on the firm. facilitator positive influence to facilitate training provider and trainee choice. interrupter negative forces that slowed operational choices. a distinguishing characteristic to be observed through this cross-case analysis process was that, as more and more database transcripts became part of the analysis, fewer and fewer new categories emerged, and the existing ones become “saturated”. thus, instead of the collection, coding and counting of activities, which dominated activity in the early steps and within-case analysis, the analytical aspects of the research process became more intense. moreover, in this later integrative stage of analysis the focus of the constant comparison moved up, from the level of the data item, to the level of the conceptual categories themselves. patternmatching aspects of this category-level analysis proved to be instrumental in recognition of the broader “themes” in the results. the procedure used by myself was a form of content analysis where the data were read and categorised into concepts that were suggested by the data and guided by initial temporal constructs. this technique is known as ‘open coding’ (strauss & corbin, 1990) and it relies on an analytic technique of identifying possible categories and their properties and their dimensions. once all the data were collected and coded, the concepts were organised by recurring theme. these themes became prime candidates for a set of stable and common construct and code cases frequency of events. the higher the number, the dominant the issue ajis vol. 10 no. 2 may 2003 19 categories, which linked a number of associated concepts. this is known as ‘axial coding’ (strauss & corbin, 1990) and relies on a synthetic technique of making connections between sub-categories to construct a more comprehensive scheme. the eight cases were then re-examined and re-coded using this proposed scheme, the goal being to determine that set of categories and concepts that covered as much of the data as possible. this iterative examination yielded a set of broad categories and associated concepts that described the salient conditions, events, experiences, and consequences associated with the participation process. the iteration between data and concepts ended when enough categories and associated concepts had been defined to explain what had been told across all sites, and no additional data were being collected to add to the set of concepts or categories, a situation glaser & strauss (1967) refer to as ‘theoretical saturation’. the resultant framework is empirically valid as it can account of the unique data of each site, as well as generalise patterns across the sites. that is, the categories describe the data, and they also interpret the data. this interpretation lead into the next step of analysis (step 6), involving making inferences and formulating propositions. space limitations of the paper preclude a detailed description of step 7 (enfolding the literature) and step 8 (reaching closure). however, both steps involved discussing the propositions with the extant literature. they did so by returning to the literature to note consistencies with and departures from findings of earlier research. this involved asking what it is similar to, what does it contradict, and why? in pursuit of this objective, for each proposition, the research indicated the extent to which it was supported by previous research and the extent to which the research has added some new perspective or idea when thinking about the process of training participation. to re-cap on the data analysis procedures, i found presenting information systematically in a visual format to be very helpful in conducting the analysis. to illustrate this, the primary methods of data analysis and presentation were through the use of matrices, a logical chain of evidence involving a critical incident time line, a software database to systematically conduct textual analysis, and the presentation of a “story” piecing together interviewee accounts taken from the case study database. the data were then analysed using withincase and cross-case matrix construction. unstructured computer software was used to systematically conduct textual analyses producing accounts based on instantiation through quotations of text. the chain of evidence, the time-lines, and database structure constituted an intermediate level of theorising between the accounts the interviewees gave and a more abstract and general process model. the thematic analysis embedded in the database structure thus became both an expression of my emerging ideas about the participation process in general, and a useful tool in comparing the eight cases in a systematic fashion. for further procedural descriptions of how the grounded theory was developed and the propositions generated, the reader is referred to rowlands (2001) discussion, implications & conclusions given that this is a paper about conducting interpretive research, it is appropriate to focus the discussion on the particular research approach taken. the research process faced a number of conceptual and methodological problems when developing process knowledge in this field of study. the first problem involved perspective. the findings of the literature review suggested that differences in intentions, processes, and contexts around participation with training initiatives have been largely overlooked by research that tends to focus on inputs or outputs but not process. the approach taken in this research was different from existing frameworks on training participation research, which tend to share three characteristics. firstly, these models are deterministic and assume that objective factors outside human intention are responsible for adoption. secondly, they are variance models and hence do not adequately capture the contextual and processual issues that are fundamental to examining organisational change. thirdly, they focus primarily on factors of economic cost and benefit, and hence do not examine, over time, the dynamics and interplay of social influences, business context, and individual owner/manager action. in this particular example, it was made clear that a socio-technical and organisational innovative perspective approaching participation as a multi-dimensional construct locally defined by the owner/managers taking into account social issues represents a more realistic approach than more common attempts to find objective criteria such as training expenditure or economic benefits. in short, this research studied the process of participation from the viewpoint of the owner/manager and how they defined their situation locally. the second problem concerns the contribution of ‘process’ research as opposed to ‘variance’ models to the understanding of organisational phenomena. clearly, process studies address questions that are unanswerable by variance research: for example, what are the typical events followed as a small firm moves from limited knowledge of a training initiative to willingness to participate in a trial? as van de ven & huber (1990:213) state, ‘process studies are fundamental to gaining an appreciation of dynamic organisational life’. however, it can be argued that because the ‘process’ approach usually only allows the consideration of small samples of organisations, it is difficult to draw generalisable conclusions. ajis vol. 10 no. 2 may 2003 20 third, it is important to consider the overall demand of this methodological approach on the researcher. for instance, interpretive research usually results in the collection of large amounts of data almost surpassing human ability to compile without the aid of specialist software for textual storage and retrieval. b fourth, because of the demands and problems encountered during qualitative research, researchers must have a great interest in and dedication to the object of the research. while it is essential to gain the trust and confidence of owner/managers future researchers should not underestimate the time and effort required to conduct these kinds of studies, nor the difficulty in gaining access to busy and sometimes sceptical managers for lengthy and detailed interviews. another related problem to be overcome from this research approach concerns the logistics in carrying out indepth case research. it was learned that one must be willing to meet with the informants a number of times (always at a time that suited them) in order to establish rapport before informants were ready and willing to be interviewed. despite these constraints, in-depth case studies remain, this paper suggests, the best approach available to collecting rich data. the reward clearly appears to be a deeper and broader understanding of small business decision-making and the ability to contribute significantly to cumulative knowledge in the field. fifth, the discussion now turns to the design and procedures i used for conducting the inductive process so central to interpretive research. i have already drawn attention to the fact that phase 1 of the design (c.f. figure 1) was quasi deductive, so the question remains, does this compromise the claim that the research approach was genuinely interpretive? to address this question, i will re-state my reasons for the a priori use of literature, and then evaluate the research procedures based on a set of principles suggested by klein & myers’ (1999) for conducting interpretive research. in terms of the design, i chose to be initially guided by the literature, yet the bulk of the reflection with the literature came after the substantive propositions were generated in phase 3 of the design. nevertheless, the preliminary literature review provided a guiding focus to the research and stimulated theoretical sensitivity by providing concepts and relationships that were checked out against actual data. however, in my defence, the overall design and case method allowed the research to flow in the more common direction from data to theory – the process of inducting theory. secondly, by referring to the principles suggested by klein & myers (1999), i can defend the research as being interpretive by reference to our definition and four key principles. when applying klein & myers (1999) definition of interpretive research (as discussed in the paper’s introduction), the research is interpretive given that there was no use of formal propositions, quantifiable measures of variables, or drawing inferences from a representative sample to a population. nor were any dependent or independent variables defined. the research approach instead was intent on understanding the phenomena through the meanings that the owner/managers assigned to them. this was achieved by the use of unstructured interviews for data collection and grounded theory techniques for data analysis. the gt techniques of open and axial coding are well suited to the supporting klein & myers’ fundamental first principle for conducting interpretive field research, that of the hermeneutic circle. this principle suggests that all human understanding is achieved by iterating between considering the interdependent meaning of the parts (open codes) and the whole that they form (axial codes). klein & myers’ (1999) second principle of contextualisation requires critical reflection of the social and historical background. in this example, i documented the context of the study and demonstrated how important it was to understand how policy has evolved in terms of the emphasis successive governments have placed on promoting employment and training initiatives within small firms. the fourth principle — abstraction and generalisation, was attained through my continuation of the analytic generalisation process by combining the findings with the innovation and organisational decision-making literature. in step 6, the research developed a set of theoretical propositions that defined and explained the participation process. the research then discussed the propositions by returning to the literature in step 7 to note consistencies with and departures from findings of earlier research. the sixth principle — multiple interpretations, was achieved by a research design based on replication logic and the deliberate intention of comparing and contrasting differences in interpretations by the participants and non-participants as expressed in their narratives and critical incidents. to conclude, the research recognised the serious lack of established theory and prior empirical research on training within the it industry in general. therefore, at this stage of knowledge accrual about training participation, the need for greater precision in research was viewed in balance with the long-term benefits of first generating meaningful, and field-relevant theory. in this sense, the research circumstances of this investigation were clearly favourable for using an inductive, interpretive and grounded theory approach. ajis vol. 10 no. 2 may 2003 21 references berger & luckmann (1967), the social construction of reality: a treatise in the sociology of knowledge, doubleday, new york. bijker, w., hughes, p., & pinch, t., (1987), the social construction of technological systems: new directions in the sociology and history of technology, mit press, cambridge mass. burrell, g., & morgan, g., (1979), sociological paradigms and organisational analysis, london, heinmann. cavaye, a., (1996), case study research: a multi-faceted research approach for is, information systems journal, vol 6, pp 227-242. chetty, s., (1996), the case study method for research in small and medium sized firms, international small business journal, vol 15, no 1, pp. 73-85. eisenhardt, k., (1989a), making fast strategic decisions in high velocity environments, academy of management journal, vol 32, no 3, pp. 543-576. eisenhardt, k., (1989b), building theories from case study research, academy of management review, vol 14, no 4, pp. 532-550. galal g (2001) from context to constructs: the use of grounded theory in operationalising contingent process models. european journal of information systems 10, 2-14. glaser, b., & strauss, a., (1967), the discovery of grounded theory: strategies for qualitative research, aldine, chicago. hayton, g., mcintyre, j., mcdonald, r., sweet, r., noble, c., smith a., & roberts p., (1996), final report: enterprise training in australia, office of training and further education, melbourne. hirschman, e., (1986), humanistic inquiry in marketing research: philosophy, method and criteria, journal of marketing research, 23, pp. 237-249. jones s & hughes j (2001) understanding is evaluation as a complex social process: a case study of a uk local authority. european journal of information systems 10, 189-203. klein, h., & myers, m., (1999), a set of principals for conducting and evaluating interpretive field studies in information systems, mis quarterly, vol 23, no 1, pp 67-94. klein, h., & myers, m., (2001), a classification scheme for interpretive research in information systems, chapter 9 from trauth (2001), 218-239. langley, a., & truax, j., (1994), a process study of new technology adoption in smaller manufacturing firms, journal of management studies, vol 31, no 5, pp 593-651. lee, a., (1991), integrating positivist and interpretive approaches to organisational science, organisational science, vol 2, no 4, pp, 342-365. lincoln, y., & guba, e., (1985), naturalistic inquiry, sage, newbury park, california.. markus, m. l., (1994), electronic mail as the medium of managerial choice, organisation science, vol 5, no 4, pp 502-527. mcintyre, j., (1998), using databases for qualitative analysis, chapter 8 in higgs. j., writing qualitative research, cpea, hampden press, pp 81-92. merriam, s., (1988), case study research in education: a qualitative approach, jossey-bass, san francisco. merriam, s., & simpson, e., (1995), a guide to research for educators and trainers of adults, 2 nd ed, kreiger publishing, florida. miles, m., & huberman, a., (1994), qualitative data analysis: an expanded sourcebook, sage, thousand oaks.. mohr, l., (1982), explaining organisational behaviour, jossey-bass, san francisco. orlikowsi, w., & baroudi, j., (1991), studying information technology in organisations: research approaches and assumptions, information systems research, vol. 2, no. 1, pp. 1-28. rogers, e.m., (1995), diffusion of innovations (4th ed), free press, new york. rowlands, b., (2001), an interpretive study of training initiative participation among small firms in the it industry, australian conference on information systems, southern cross university, pp. 547-556. saunders, s., (2001), issues and directions from a review of the australian apprenticeship and traineeship literature, ncver, http://www.ncver.edu.au/research/proj/nr9012i.pdf [accessed: may 2001]. ajis vol. 10 no. 2 may 2003 22 smith, a., roberts, p., noble, c., hayton, g., & thorne, e., (1995), enterprise training: the factors that effect demand, vol 1, otfe, melbourne. smith, a., (2000), casing the joint: case study methodology in vet research at the organisational level, anz journal of vocational education research, vol 8, no 1, pp. 73-91. strauss, a., & corbin, j., (1990), basics of qualitative research: grounded theory procedures and techniques, sage, newbury park, california. trauth, e. m., (2001), qualitative research in is: issues and trends, idea publishing. urquhart, c., (1997), exploring analyst-client interaction communication: using grounded theory techniques to investigate interaction in informal requirements gathering, in information systems and qualitative research, lee, a., liebenau, j., & degross, j., (eds), chapman & hall, london. urquhart, c., (2001), an encounter with grounded theory: tackling the practical and philosophical issues, chapter 5 from trauth (2001), pp 105-139. van de ven, a., & huber, g., (1990), longitudinal field research methods for studying processes of organisational change, organisational science, vol 1, pp 213-219. van maanen, j., (1979), the fact of fiction in organisational ethnography, administrative science quarterly, vol 24, no 2, pp. 539-550. walsham, g., (1995), interpretive case studies in is research: nature and method, european journal of information systems, vol 4. no 2, pp.74-81. whetten, d., (1989), what contributes a theoretical contribution?, academy of management review, vol 14, no 4, pp 490-495. wolfe, r., (1994), organisational innovation: review, critique and suggested research directions, journal of management studies, vol 31, no 3, pp 405-431. yin, r., (1994) case study research: design and methods, 2 nd ed, sage, beverly hills, california. microsoft word 13 1 marina.doc ajis vol 13, no. 1 september 2005 254 the relevance of value net integrator and shared infrastructure ebusiness models in managing chronic conditions susan lambert, school of commerce, flinders university of sa abstract there is widespread support for chronic condition management (ccm) programs that require a multi-disciplinary, care-team approach. implementation of such programs represents a paradigm shift in primary care service delivery and has significant resource implications for the general practice. integral to the widespread uptake of care-team based ccm is information collection, storage and dissemination amongst the care-team members. this paper looks to ebusiness models for assistance in understanding the requirements of general practitioners (gps) in providing multi-disciplinary team care to patients with chronic conditions. the role required of gps in chronic condition management is compared to that of a value net integrator. the essential characteristics of value net integrators are identified and compared to those of gps providing multi-disciplinary team care to patients with chronic conditions. it is further suggested that a shared infrastructure is required. introduction growing evidence supports the notion that patients with chronic conditions cannot be effectively treated using the traditional, general practice, acute service model and that a new service model is required. there is some consensus in the literature that a multidisciplinary, care-team service model involving the general practitioner (gp) and other members of the primary health care value network (value net) is needed (wagner, 2000, veale, 2003, fox et al, 1998, berenson and horvarth, 2003). although there appears to be general recognition that the multi-disciplinary team based model serves patients with chronic conditions better than the acute service model, gps are struggling to implement it and most are not even attempting to implement these programs. the changes required to successfully implement multi-disciplinary care-team based primary health care are not incremental, they constitute fundamental changes to the general practice business model and require significant changes to the underlying resources of the general practice. in this paper it is suggested that to enable care-team based primary health care on a large scale the business model needs to incorporate the characteristics of what weill and vitale (2001) call the value net integrator (vni) and a shared infrastructure. firstly, the traditional general practice business model will be described using the business model framework of weill and vitale (2001) demonstrating a close resemblance to the direct-to-customer business model. the characteristics of the care-team based chronic care model will be described and the shortcomings of the traditional general practice business model to deliver such care will be identified. next, attributes of the vni business model will be described and it will be demonstrated that the vni business model, supported by a shared infrastructure, has the potential to provide a better vehicle for the delivery of team based chronic care than the direct-to-customer business model. questions will be raised as to whether the gp is in the best position to take on the role of vni or whether there is an entity or person better placed to take on this role. ajis vol 13, no. 1 september 2005 255 the traditional general practice business model a business model is a business concept that has been put into practice (hamel, 2000). many definitions and constructions of business models exist but essentially they describe the value offered by the entity, the value received in return and how the entity relates with other entities in the value network (lambert, 2003). table 1 lists the generic business model attributes (lambert 2003) and the values of these attributes for traditional general practices. it is from the business model that the essential resources, strategies and organisational structures of the entity can be derived. the traditional general practice business concept is that of gps providing an acute care service to patients in return for a government subsidised fee. the traditional general practice business model can be best described as a non-electronic, direct-to-customer (patient) business model. table 1: traditional general practice business model attributes generic business model attributes general practice business model attributes value offering and to whom primary health care services offered to patients using predominantly an acute service model. value received and from whom a government subsidised fee from the patient. the firm’s position in the value domain the general practice is a direct-to-patient service provider, utilising no intermediaries. the nature and channels of interaction of the firm with other entities (allies) within the value domain three categories of allies interact with the general practice through various channels. allied health service providers. divisions of general practice the federal government. gps make themselves available to patients in their consulting rooms and patients make appointments for consultations. patients present to the gp and describe their conditions, the gp treats the patients and or recommends further consultations with themselves or other allied health service providers. the gp does nothing more unless the patient presents again. this is known as an acute service model, patients presenting in an ad hoc fashion primarily for acute symptoms. the general practice business model is illustrated in figure 1. the dashed line represents flows of information between entities and the solid line represents a flow of services and products. the interactions with allies are significant characteristics of the general practice business model distinguishing it from business models of many other professional service firms. allies play a major role in the successful delivery of primary care to patients therefore it is crucial that the nature of these relationships is clearly understood. ajis vol 13, no. 1 september 2005 256 traditional relationships with allies as stated in table 1, gps interact with three categories of allies and each serve very different roles in the primary health care domain. allied health service providers this group consists of medical specialists, hospitals, pharmacists, health care providers such as dieticians, dentists, asthma educators, social workers and psychologists, and community service providers such as alcohol and drug support workers, disability service coordinators, carer respite and support workers and education providers. these entities interact directly with the patient, sometimes being referred to the patient by the gp and other times without the gps knowledge. although there is some exchange of information and ongoing communication between these allies and the gp it is mostly of a reporting nature with the patient often not privy to the communications. general practice patients divisions of general practice doha allied health service provider allied health service provider allies service or product flow information flow legend figure 1: traditional general practice business model ajis vol 13, no. 1 september 2005 257 divisions of general practice this body provides professional and business support to gps. divisions of general practice are “…local organisations, funded by the australian government department of health and aging (doha) to improve health outcomes for patients by encouraging gps to work together and link with other health professionals. …divisions have a role in helping general practices and individual gps to work more collaboratively with other health professionals. they advocate and negotiate on behalf of their members with governments, hospitals, other health care providers….” (primary health care research & information service 2004). some federal government funding is channelled through the divisions to gps to provide professional development and resources to the gp such as practice nurses to rural practices. the federal government the department of health and aging (doha) is the federal government body that has both a monitoring and a regulating role with respect to gps. general practice revenue models are determined by the doha which implements government health fiscal policies by controlling the gp fee structure and the conditions by which gps can claim additional funding. the doha channels funding through the divisions as well as directly to the gps. the patient the traditional relationship between gp and patient can best be described as didactic; the gp having a predominantly authoritarian role and the patient a passive one (holman & lorig, 2000). in the past most patients would present with acute symptoms, seeking treatment or instructions from the gp on what to do to treat the problem. the need for a multi-disciplinary care-team approach to chronic condition management the gp business model has evolved to assist and support gps in achieving their traditional objective, which is to provide medical care to patients presenting with predominantly acute ailments. in recent times however the majority of patients present not with acute ailments but with chronic conditions. chronic conditions represent approximately seventy percent of all general practice encounters (veale, 2003) and most of these involve comorbidities (i.e. multiple chronic illnesses) (grumbach, 2003). the problem is that the traditional, acute care model does not work well for patients with chronic illness, especially where patients present with comorbidities. one of the reasons is that the goals of chronic care are not the same as the goals of acute care. the main goal of acute care is to treat the ailment so that it goes away and the patient’s health is restored. the goals of chronic care are to “…enhance functional status, minimize distressing symptoms, cope with the psychosocial stresses of pain and disability, and prolong life through secondary prevention.” (grumbach, 2003). the complexity and long-term nature of chronic illness requires a range of skills all of which cannot be provided by the gp alone. the multi-disciplinary care-team approach requires ajis vol 13, no. 1 september 2005 258 the gp to work in conjunction with other healthcare groups to provide a more holistic service to patients. chronic care programs take a variety of forms but common to all is the development of long-term care plans that set goals for the patient, record treatments and results. essential to the success of care-team based chronic care is the ability of all care-team members to have access to patient data and be able to update the records in real time. the collaborative teamwork required for effective chronic care and the complementary information processing, storage and distribution activities require radical changes to general practice operations and have significant resource implications. resource implications of chronic care-team based medicine key competencies the training and skills of gps have been tailored to suit the acute service, didactic model, the emphasis being on what balogun and jenkins (2003) refer to as ‘explicit knowledge’. this equates to the gp being able to diagnose and apply the appropriate treatment to the illness. training of gps in australia has ‘remained rooted in urban medical schools and their affiliated teaching hospitals, where the focus is on acute disease’ (balakrishnan and finucane, 2003). these skills and knowledge are obviously still required in a team based approach however additional skills and knowledge are required to facilitate the functioning of the care-team. increasingly the gp must be able to utilise the services and treatments offered by allied health service providers for the benefit of their chronically ill patients and therefore need to acquire skills in quality assurance, supervision and review. they also need to be able to manage the patient data that is a necessary part of a long-term approach to chronic care along with the associated technologies. it infrastructure the information technology and communication (itc) requirements of gps have, to date, been relatively basic. they are aimed at assisting gps to access and update patient data, issue prescriptions and order diagnostic tests. many gps maintain predominantly manual systems for internal information management. effective chronic condition management requires greater utilisation of itc resources to monitor patient condition and communicate with other care-team members. (celler et al, 2003). internet based application programs that allow gps to remotely monitor patient conditions and programs that allow for the sharing of patient health records are gradually becoming available, although considerable research into the effectiveness of these applications is ongoing (celler et al, 2003). strategy and structure to a large extent the revenue model dictates practice strategy. the revenue model is a feefor-service model, determined by the doha. the gp is able to charge patients per visit the majority of the fee coming from government rebate and a small proportion (if any) coming from the patient. the ability to charge for practice nurse services is strictly limited and ajis vol 13, no. 1 september 2005 259 administrative services are not chargeable. it follows that the traditional strategy adopted by gps is to maximise the number of patient visits and minimise practice support staff which in turn maximises practice revenue and to utilise it systems and human resources to provide support services to the gp without substituting for gp-patient time. innovative use of technology that eliminates the need for clinical visits is not supported by the existing revenue model. care-team coordinator as value net integrator coordination of the multi-disciplinary care-team and management of patient data requires a team member to assume the role of vni. although the pure vni abandons roles connected with the physical world and the care-team coordinator does not, there are important similarities. “value net integrators add value by improving the effectiveness of the value chain by coordinating information.” (weill and vitale, 2001, p221) the vni is responsible for gathering information, processing it and distributing it to all other entities in the value net and for coordinating product flows from suppliers and allies to the customer. the goal of the vni is to know more about the customers’ needs and transactions than any other entity in the value net, to know the needs and capabilities of all other entities in the value net and be able to manage relationships between customers and other entities. to be successful the vni must be trusted by the customers and the other entities in the value net and be aware of the services and products supplied by all suppliers and allies in the value net. arguably the vni role can be carried out by any of the care-team members however the gp is well positioned in the primary health care industry to become the vni. the gp and the staff of the gp interact with the patients for multiple reasons including acute ailments and therefore has the potential to have more information about the patient and their needs than any other entity in the value net. in addition the gp is well informed as to the services and products offered by the allied health services and how they can assist the patient. the vni business model is very similar to the general practice, direct-to-customer business model. most importantly it recognises the significance of and the richness of communication between the general practice and the allies. it also acknowledges that the general practice owns the primary relationship with the patient. owning, or having the potential to own, the primary relationship with the patient means the vni is in a position to know more about the patient than any other entity in the value net. this is only possible if the allies keep the general practice informed of all dealings with the patient. figure 2 illustrates the application of the vni business model to the delivery of multidisciplinary team care to chronically ill patients. it is noteworthy that the general practice rather than the general practitioner is the entity of interest. it is possible and likely that support staff within the general practice will assume the role of team coordinator and that the gp will assume the role of supervisor and quality assurance reviewer. to operate successfully the vni must be able to gather, synthesise and distribute information to all entities within the value net. this implies a level of itc that is not present in most practices. it is not likely that individual general practices will have the skill and the financial resources to develop and manage their own patient databases in such a way as to allow input from allies dealing with the patient, processing of the data and to allow dissemination of patient data to the relevant allies and the patient. ajis vol 13, no. 1 september 2005 260 it seems appropriate for the itc resources and management to be provided under a shared infrastructure business model as described by weill and vitale (2001). in general, a shared infrastructure is formed and financed by competitors in the same industry with a view to providing a service that is not otherwise possible due either to cost factors or because the service can only be provided if competitors share some or all data and or assets. figure 3 depicts the general practice operating as a vni and being supported by a shared infrastructure. to be successful the partners of the shared infrastructure must perceive that they all have the same opportunity to gain from the arrangement and that no single partner dominates. there must be effective partner conflict management structures and processes and the infrastructure must be fully interoperable not only with respect to the partners but with respect to all users of the system. financially a shared infrastructure would benefit gps providing care-team based chronic care. economies of scale could be recognised and an overall more efficient and productive system could be developed and maintained. the other entities within the value net would also benefit from having to deal with a single service provider rather than a multitude of independently operated databases. general practice vni patients divisions of general practice doha allied health service provider allied health service provider figure 2: general practice vni business model allies primary relationship information flow ajis vol 13, no. 1 september 2005 261 a generic problem with shared infrastructures is the management of the project, coordinating partners to the project, managing the relationships between partners and ensuring equitable opportunities for partners to contribute to the development of the project. since the divisions of general practice already operate to encourage collaboration amongst gps and other health service providers and since the divisions have access to federal government funding, the management of the shared infrastructure might be a natural extension of division services. general practice vni patients divisions of general practice doha allied health service provider allied health service provider figure 3: general practice vni business model with shared allies primary relationship information flow shared infrastructure general practice vni general practice vni shared infrastructure ajis vol 13, no. 1 september 2005 262 limiting factors security and privacy of patient data is a major obstacle to the implementation of any database that contains data of a personal and sensitive nature. unauthorised access and data tampering is of paramount concern to patients and to medical team members. the propensity of gps to embrace change and to manage such radical change will be a determinant of the success of multidisciplinary, care-team, chronic condition management. one of the factors that will influence the motivation of gps to make the change is the ability of them to benefit monetarily. the gp revenue model will need to recognise the non-gp input to the system and allow gps to benefit from providing web based support, practice nurse resources and other administrative services to patients. although these issues are not the subject of this paper it is recognised that they need to be addressed before any patient data sharing programs can be implemented and before gps will contemplate implementing the itc systems that are being developed. references balakrishnan, r. and finucane p, (2003) “reforming medical education to enhance the management of chronic disease”, mja; 179 (5): 257-259. balogun, j. and jenkins, m., (2003) “re-conceiving change management: a knowledgebased perspective”, european management journal vol. 21, pp 247-257. berenson, r and horvath, j. (2003) “confronting the barriers to chronic care management in medicare”. health affairs. web exclusives, 22 january. www.healthaffairs.org/webexclusives/berenson_web_excl_012203.htm (accessed 7 january 2004). celler, b., lovell, n., and basilikis, j., (2003) “using information technology to improve the management of chronic disease”, mja; 179 (5): 242-246. fox, p.d., etheredge, l., jones, s. (1998) “addressing the needs of chronically ill persons under medicare”, health affairs. chevvy chase: mar/apr. vol. 17, iss 2; pg 144. grumbach, k., (2003) “chronic illness comorbidities and the need for medical generalism”, annals of family medicine,1:4-7. hamel, g.(2000) leading the revolution, harvard business school press, boston, massachusetts. holman, h. and lorig, k. editorial, (2000) bmj vol 320, february, www.bmj.com. lambert, s. (2003) “making sense of business models”, school of commerce research paper series 03-10, issn: 1441-3906. primary health care research & information service 2004, about divisions of general practice [online], available: http://phcris.org.au/resources/divisions/info_about_divisions_mainframe.html, [accessed 12 january 2004]. veale b., (2003) “meeting the challenge of chronic illness in general practice”, mja; 179 (5): 247-249. wagner, e., (2000) “the role of patient care teams in chronic disease management” bmj vol 320, pg 569. weill, p. and vitale, m., (2001) place to space, migrating to ebusiness models, harvard business school press, boston, massachusetts. microsoft word ajis10-2finala.doc ajis vol. 10 no. 2 may 2003 139 designing systems for e-commerce shona leitch and matthew warren school of information technology, deakin university, australia. e-mail: shona@deakin.edu.au abstract ethics is an important element in all aspects of computing, but proves to be a real problem in the development and delivery of e-commerce systems. there are many aspects of ethics that can affect e-commerce systems, but often research is focused on the ethics after the e-commerce system has been implemented, focusing on issues such as trust, privacy and disclosure. this paper will discuss how ethical matters can affect the design of e-commerce systems and how a framework designed for e-commerce can be used to create and deliver effective, ethical e-commerce systems. keywords: e-commerce, ethics and development. introduction there is a lot of talk especially in the popular press as to the virtual explosion of e-commerce over the past few years, however many reports indicate otherwise. or at least indicate that this ‘explosion’ is only taking place in certain sub-sections of the e-commerce industry. according the national office for the information economy (noie, 2001) over a half of australia’s population has a personal computer in their homes and 35% have internet access. along the same lines, most medium-sized businesses and over 80% of what is termed small business use personal computers. ‘over 35 per cent of all businesses have an online presence—a comparative business advantage that translates into australia being consistently rated in the top ten nations globally for its e-commerce environment.’ however this does not necessarily mean that australia is performing and using e-commerce well, a web presence does not necessarily mean ‘electronic commerce’ either. and further enlightenment from noie comes from the following statement: ‘five per cent of australian adults shopped via the internet in the 12 months to february 2000, and 74 per cent paid for their purchases online. take up rates of other kinds of e-commerce—such as telephone banking and electronic funds transfer—were even higher.’ (noie, 2001) the aim of the research was to produce a workable framework, which incorporates parts of previously validated systems design methodologies that encompasses all the needs and requirements of users in an ecommerce environment. by using a mixture of methodologies, we are able to also consider the fact that the system is designed ethically. one concern that we need to consider is that the ‘systems’ part of an ‘information system’ (including ecommerce systems) represents a way of seeing the set of interacting components, such as: • people (e.g. systems analysts, users) • objects (e.g. computer hardware devices) • procedures (e.g. those suggestions made in an is development methodology) (avison, 1999) often in systems design, the developers focus on the objects, i.e. making the most technologically advanced system. in the case of e-commerce b:c systems, the users are focussed upon in an attempt to sell more product and market more effectively, and often in a way that encourages unethical practices when designing systems rather than producing a system that is in the users best interests. project success and failure at the present time internet businesses have access to a wide audience with 11% of australian households with internet access and over 40% of u.s households. therefore, there must be other reasons why people do not utilise online web retail or undertake b:b transactions. ajis vol. 10 no. 2 may 2003 140 e-commerce poses unique design issues (leitch, 2001): • fast development time • lack of expertise in systems design means the need for an easy to use method • high failure rate, due in part to poor systems analysis and design • heavy user considerations, particularly in b:c one example of an e-commerce failure was the uk online retailer, boo.com. who even though had a potentially huge client base failed to capitalise on this due to poor systems design and web design and a lack of consideration of users: figure 1 – website of boo.com the reason for failure was that a large proportion of its potential market was unable to use boo's site because the website design was too advanced for most computers and access frustratingly slow (bbc, 2000). other reasons for boo.com failure included: • the site could not be seen by people who use macintosh computers. • it used lots of graphics, pop-up windows and 3-d images that only those with a 56k modem could see it without waiting minutes for it to load. • the site was very difficult to navigate around. shoppers could get lost and find no way back to their starting point. • boo.com was funded to the tune of some a$180m (£80m). boo's problems also centered on its inability to build up numbers of buying customers fast enough, to generate revenues to offset steep set-up costs. it is probable that a use of systems analysis techniques could have overcome these problems. “firms that cannot deliver either because their web site is poorly designed or because their distribution system is not up to scratch are bound to fail” (bbc, 2000) the concerns of system success and failure are very real. within any new computer system, a great deal of time, effort and monies are invested, with little guarantee of there being success at the end of the project. ajis vol. 10 no. 2 may 2003 141 companies that employ system designers, or model, analyse and plan their system would seem to have the best chance at achieving a truly helpful, workable system. the rise of the internet as a locus of business is changing marketing, for instance. it makes possible one-onone marketing by tracking the customer, recording his or her preferences and proclivities, and presenting the customer with products that he or she is likely to want to buy. high street stores, similarly to tv ads, have to generalise their marketing to an average consumer. the internet makes it possible for a company to design a system that lets each consumer have an individual fit. this can be done to aid the user, by remembering certain likes and dislikes, but more often that not, it is done unethically to entice people to buy things they do not need through a huge purchase driven experience. by incorporating a participational aspect to systems design, and including the user at this stage it is very likely that a system would be developed that is more ethically sound to those b:c e-commerce customers. but other studies have identified advantages in using a participational approach. adams (1984) found that: • staff/users have ideas which can be useful; • effective upwards communications are essential to effective decision making at the top; prior research had found that failure of e-commerce design in an australian context could have been resolved by a number of factors including design participation of users (leitch and warren, 2003). framework overview the authors have developed a framework which could be used for the design of e-commerce systems, focusing upon user requirements. the issues of ethical design can be resolved by ensuring that users are key stakeholders in the design of the systems that they will be using. the stages of the framework are: stage 1 – strategic and business evaluation 1. assess present level of e-commerce against criteria. criteria: level 1 – no web presence level 2 – have web presence (website with contact data etc) level 3 – have web presence and use b:b level 4 – as level 3 and use b:c level 5 – use both b:b and b:c 2. decide future level, of e-commerce using the above criteria. 3. create detailed plan for business/systems development set up project team identify key business activities (bsp) develop a formal plan (objectives, goals, resources) (lederer and mendelow, 1989) link e-commerce plan to business plan perform cost benefit analysis – identifying tangible and intangible benefits prepare for change 4. perform feasibility and swot analysis define information systems outline select feasibility options identify strengths, weakness, opportunities and threats create feasibility report stage 2 – systems analysis and design 1. create rich pictures from level 2 and 3 of stage 1 to create a conceptual model of system and user requirements. 2. apply a participational approach to level 1 use a cross census of users to discuss the rich pictures reflect on outcomes of discussions if possible, assess e-commerce user requirement, also using the participational approach (perhaps with a pilot group of internet users) reflect again on these outcomes 3. create final rich picture stage 3 – systems e-commerce design 1. undertake process modelling 2. assess technical considerations 3. assess security considerations ajis vol. 10 no. 2 may 2003 142 stage 4 – implementation 1. the physical implementation of the web-site. stage 5 – post implementation 1. using the participational approach, discuss and reflect on issues arising through implementation. stage 1 of the framework could be compared to traditional systems analysis and design methods that implement feasibility studies, requirements elicitation or other problem identifying skills. although part of the beginning of this framework aims to also provide some insight into the feasibility of an e-commerce project, it also provides a much deeper, strategic and business evaluation than using just one of the above. from current statistics of organisations use of feasibility studies, two thirds of organisations did not conduct / sometimes conducted a feasibility study (leitch, 2001). it is also believed that the inclusion of a feasibility aspect of the design, will encourage companies to assess their business function and customers wants, needs and expectations. the benefits of feasibility studies are widely reported, and therefore it was decided to include an ad-hoc one within this framework. initially it was decided that a developer would first have to decide at what ‘stage’ of e-commerce the organisation was at. this is deemed to be important, as many e-commerce businesses as they define themselves are little more than an online advertisement. therefore by categorising themselves, it is easier to proceed through other levels of e-commerce development. as at this initial level, it is suggested that the developer/analyst also decides to which level the organisation wishes to pertain through the development of the present project. a large amount of focus on the new framework for e-commerce was put into the strategic and planning stages, although this is not aimed at taking up a large amount of a potential analyst’s time, it is perceived to be straightforward enough to do in a timely manner whilst still providing a developer with enough information and safeguards to know that their project has merit. therefore, in the 3rd step of stage 1, it is suggested that the developers do a number of formal steps to aid in the proposed project. in stage 2 of the framework, a number of traditional steps were used, in order to provide the best possible analysis stage for the area of e-commerce. the needs of e-commerce in terms of systems analysis and design are not adequately covered in any of the major methodologies and frameworks that have been written so far. however parts of these methodologies and frameworks have aspects that are relevant and useful to ecommerce design. the use of a form of rich pictures (checkland, 1981) will both incorporate the users of a system, as well as providing an easy to understand and relate to method of eliciting and discussing information between the designers and other stakeholders. it is suggested that ethics (mumford, 1983) is used as the way to conduct these meetings. this information can then be taken by the designers and created into simple dfd style diagrams for the coding stage. again post-implementation will make use of a particpational approach to assess the new system. stage 3 of the framework is focused upon a more traditional system design viewpoint in terms of process definition, use of appropriate technologies and how to implement required security features. stage 4 is the actual implementation of the system. stage 5 focuses upon the post implementation aspect. the design groups reflects upon the design of the system and uses a participational approach to overcome any problems that may have arisen. it is not being suggested that designers follow methodologies closely, this framework is suggesting the best methods to use to create a new e-commerce system, and recommends the principles of some tried and tested methodologies are implemented, but as the need for a fast creation time is paramount, then the full and detailed layout of each type of methodology will not be followed. this framework will be validated by helping to designing a number of e-commerce systems. conclusion the results of this unique research, with the analysis of the most commonly utilised systems analysis methodologies as well as considering the wants and needs of organisations creating e-commerce solutions, will allow for the production of a framework capable of assisting in more effective, successful and ethical ecommerce systems. ajis vol. 10 no. 2 may 2003 143 references adams, w. (1984) “participation today”, the industrial participation association, uk. avison, p. & fitzgerald, g., (1999) ”information systems development: methodologies, techniques and tools”. mcgraw-hill, uk. bbc (2000) ‘from boo.com to boo.gone’ http://news.bbc.co.uk/1/hi/business/753782.stm [accessed 14/08/02]. checkland, p. (1981) “systems thinking, systems practice”. wiley, chichester. lederer, a. l. & mendlelow, a. l. (1989) “information systems planning: incentives for effective action” data base, fall. leitch, s. (2001) “the use of electronic commerce and systems design within australian organisations”, deakin university technical paper, australia. leitch, s. and warren, m.j. (2003) a quality indicator of australian e-business, the e-business review, vol iii, international academy of e-business, usa. mumford, e. (1983) “designing participatively”, manchester business school, uk, isbn 0-903808-29-3. noie (2001) “e-commerce across australia” noie (national office for the information economy), canberra, australia. the australasian journal of information systems (ajis) began in 1993 and was based on the simple realisation that there was a need for an academic ‘outlet’ for is research in australia australasian journal of information systems volume 14 number 1 nov 2006 the state of information systems in australian universities – south australia report andy koronios school of computer & information science university of south australia. andy.koronios@unisa.edu.au paula swatman school of computer & information science university of south australia. paula.swatman@unisa.edu.au abstract this paper examines the status of information systems (is) at the university of south australia (unisa). on a broad scale this study is part of a larger case study regarding the state of the is academic discipline in australasian universities but, in view of the face that information systems degrees are offered only at one university within the state, the south australia study, by definition, becomes a study of that university. the paper clarifies the role of is within unisa, with particular reference to the degree of professionalism and the impact of local contingencies on is teaching and research. data were gathered by means of qualitative surveys with present and former academics of the institution, as well as via statistical information. the results suggest that the state of south australia’s is offerings were heavily influenced during the 1990s by the soft systems and critical systems approaches to the discipline, a situation which began to change at the turn of the century; and that the curriculum is depends more heavily on industrial than political factors. introduction this paper reports on the status of information systems (is) discipline at the university of south australia (unisa). although there are a number of other tertiary institutions in adelaide, none of these offers degrees in information systems. carnegie mellon university has not yet begun to offer its very high-cost degrees and, while there are degree programs at the other two major south australian universities (flinders university and the university of adelaide) which could be seen as targeting a similar market, closer observation shows that none of these degrees is either an information systems degree, or focused on the same likely employment outcomes: 201 australasian journal of information systems volume 14 number 1 nov 2006 • flinders university offers a bachelor of information technology degree, which has definite similarities with unisa’s bachelor of information technology program – both these degrees focus primarily on it, rather than is, and offer a ‘half-way house’ between computer science and information systems; • flinders bachelor of arts (geographical information systems) program offers students taking an arts degree the opportunity to include a major sequence in the area of gis. this might be seen as a possible attraction for students considering an is degree, but is a rather more specialised offering; • the university of adelaide offers a bachelor of business information technology which has many similarities with unisa’s bachelor of business (information technology) hybrid degree – both these degrees offer commerce/business students an it/is major in a businessfocused degree. the major difference between them is that the university of adelaide’s offering has significantly more computer science and less information systems than its unisa equivalent. this case study, therefore, was undertaken by academics of unisa only. this study draws upon the theoretical framework outlined in paper 3 in this volume and is therefore not restated in the present study. this review of the south australian experience of information systems education is a one of a series of similar studies undertaken in each australian state, the australian capital territory and new zealand; each of which is reported separately in this overall review. unisa was founded on 1 january 1991 through the amalgamation of the south australian institute of technology and the magill, salisbury and underdale campuses of the south australian college of advanced education. since then, it has quickly earned a reputation as a national leader in collaborative research, has been recognised nationally for innovation in teaching and has south australia's largest intake of international students. number full-time equivalent students number full-time equivalent is students number of academics teaching is 21,304 of the total students in the school of cis (1677), approximately 450 are enrolled in is courses 10 + sessional staff table 1 – summary statistics of unisa research approach the unisa study is based on the positivist case study method (yin 2003), enhanced by statistical data drawn from the university’s internal and external data gathering. case studies of this type, as in the present case, frequently make use of a detailed case study protocol – which was developed by the australasian study team members to ensure consistency of data collection across the universities studied. major objectives of this protocol were to facilitate: • comparability across the states, • consistency across the individual state case studies (not relevant to the south australian situation); and • efficiency in the data gathering process. 202 australasian journal of information systems volume 14 number 1 nov 2006 while some staff members contributed their material during interviews, the majority of responses were gathered by means of email questionnaires, which were filled in by staff and returned to the authors for analysis and review. existing documentary and archival material was also gathered to supplement the interview data and to provide some triangulation of observations. data gathered from interviews and questionnaires were then analysed using thematic analysis, so that qualitative data collected from participants and quantitative data collated from other sources could be integrated more effectively (boyatzis 1998; constas, 1992; dixon-woods, agarwal, jones, young and sutton 2005). the themes which formed the basis for this analysis were primarily those of the overall study’s theoretical framework (reported in ridley (2006)) and are therefore not specifically related to the unisa environment, but rather designed to provide comparability across states within a national study. the discussion which follows was, as has already been noted by the tasmanian study “weighted towards themes that appear to have a high level of explanatory value”, rather than reflecting how frequently the themes were reported (dixon-woods et al. 2005, p. 47). this similarity with the tasmanian study is particularly apt, in that both unisa and utas are the only universities in their respective states teaching information systems – although the parallels are not complete, as unisa is one of three south australian universities which could potentially choose to teach this material, while utas is the only university in tasmania. a quite separate study of three retired staff members, which provides a unique view of unisa during the last two decades of the 20th century, is published as a separate paper within this series. the present paper focuses on unisa’s information systems activities in the 21st century and, in particular, the ways in which the university is evolving subsequent to the major re-organisation which saw the information systems discipline move from the division of business to become a part of a broader school of computer & information science within the division of information technology, engineering & the environment. overview of size and presence of is at unisa the school of computer and information science is located within the division of information technology, engineering and the environment and has existed for the life of the university. during 2004, a review of the division of business resulted in the restructuring of schools within that division and the is discipline, which was at the time located in the school of accounting & information systems, was transferred to the school of cis – a much larger school. this merger has resulted in a large and vibrant computing school with a wide variety of programs, covering the full range from computer science, through software engineering and information technology to information systems. the university was founded in 1991 and, at that time, some members of antecedent organisations decided to form a school of information systems within the new business division. some of these staff members came from the institute of technology, while others had previously been a part of the sa college of advanced education: we had all been in business schools teaching is topics. we believed it worthwhile to have an is concentration and develop the area academically as it was useful for the uni and the citizens of sa. at the time is had quite a high profile. i was a founding member along with approx 21-23 others. we were the first and only school of information 203 australasian journal of information systems volume 14 number 1 nov 2006 systems in sa. flinders university had nothing in the way of is and adelaide university had one subject in computing science. we developed a range of undergraduate programs which were very well received. in 1996 the university reduced funding to the school, which resulted in staff cuts and redundancies. in 1996 there were cuts across the business division of which is was a part and staff were advised that the annual budget had been reduced from $2.8 million to $1.8 million. the group had significant financial problems at the time, although it had developed a successful industry placement program with about 40 industry partners and earned a very positive reputation for its engagement with industry. at least one respondent (now retired) believes the group had a mixed relationship with the division of business from its inception, claiming that there had been several attempts to move them out of the business division. the group was for most of its time located in an accounting and is school which also had a group from the administrative management area. although some feel that the group is now smaller and less visible, it now has three full-time is professors, in addition to two european adjunct professors and an associate professor who are also is-focused; and launched a bachelor of information systems for the first time in its history following the merger (and a complete revamp of all degree programs offered by the school). placement of is the merger allowed unisa to rationalise the location of is staff. originally, the majority of the university’s is teaching staff (12 academics) were based in the school of accounting and information systems at city west campus, which was composed of three discipline areas: accounting, information systems and a group known as administrative management (am), which had originally taught secretarial studies and was slowly evolving towards a new focus on the management of corporate administrative functions. some of the is courses (those courses having a prefix of infs) were taught by am staff; and some of the am courses were taught by is staff. these blurred lines of control had simply evolved over time, but were rationalised at the time of the merger. during the merger discussions, 11 of the 12 is academic staff agreed to move to the new school and 1 of the former am staff also decided to move to the school of cis. prior to the merger, some 5 is academics had been located in the ‘old’ cis at the mawson lakes campus in the north of adelaide; and their courses became a part of the revised degrees offered by the enlarged, combined school. the majority of the is teaching takes place at the city west campus of unisa, where most of the is staff are still located, with some (predominantly graduate) is courses also being offered at mawson lakes. undergraduate courses offered postgraduate courses offered no. students enrolled in is subjects dbbi, dbeb, dbin, dbit, dbms dcmi, dgec, dgmi, dmec, dmis, dmmi, lmct 450 approx. table 2 – coursework offerings within the school 204 australasian journal of information systems volume 14 number 1 nov 2006 how much is is impacted by local contingencies although the curriculum is affected by industry requirements, the school also attempts to be an industry leader instead of only responding to industry demand. local demand does affect it and is teaching structure within the state, particularly what is taught, because students must have certain skills to be employed by local and inter-state businesses. it is probably no more affected by industry than other applied disciplines, such as tourism or oenology, however. unisa has made significant structural changes which affect the teaching of is – specifically, the move of the is group from the school of accounting & is within the division of business to the school of computer & information science within the division of it, engineering & the environment. this has enabled the development of a bachelor of is degree (is was formerly offered only as one alternative within the bachelor of business degree). as the only university in south australia offering is, there is little (if any) political pressure on the school to make any particular offerings, although the bbis/mbis program (a funded joint program with industry which takes high performers and fast-tracks them via a combined academic/industry placement scheme) has had an influence on academic outcomes, as most of the brightest students choose this alternative in place of doing an honours degree. like all other australian (and overseas) universities, unisa has suffered from the ict downturn in terms of local student numbers. employment in south australia in the ict industry is extremely difficult and the current state government’s decision to outsource all major ict developments to inter-state and/or international companies has made this situation still more difficult. many of our graduates are forced to seek work in other states, so that we are probably suffering more from the downturn than any other university in the sample. recent moves to source staff for major commonwealth government projects from south australia are likely to have a positive impact on student placements, however, and have been warmly welcomed by the university. recent events, such as the large defence contractor presence in south australia and the sa government’s mou to invite the defence studies arm of cranfield university and the public policy and ict programs of carnegie mellon university, have the potential to be advantageous to the is programs at unisa. it is not entirely clear whether the potential benefits of these changes will eventuate, however, as the south australian government does not have a single focus on the ict industry. south australia has six ministers who share parts of the ict portfolio – making it difficult to produce a considered and consistent government response to any new initiative. the school has an industry advisory group which has been very supportive of is programs and has an excellent reputation with local business and with the public sector. to what extent is is identified as a separate field is was acknowledged as a separate field when it was part of business but is less clearly separated now that it is part of a large, combined computing school. as part of the business division, the is group had perforce to teach all aspects of computing, so that there were staff who taught programming and other more technical subjects. these subjects have now been rationalised and are 205 australasian journal of information systems volume 14 number 1 nov 2006 generally taught by the more technical se and cs staff (although this is not entirely the case – some of the programming courses are still taught by is staff at city west). both in its former existence as part of the school of ain and in its current existence as part of the school of cis, is at unisa has been a component of a larger school. it has been many years since there was a school of is at unisa. to a large degree this can be traced to the comparative lack of local student interest in is – the fact that neither the university of adelaide nor flinders university even bothers to offer is at all provides further evidence for this atypical lack of interest, which predates the current is student crisis. the move from the business division to the division of itee has forced the unisa is group to think very carefully indeed about the composition of its degrees. the school of cis went through a complete degree restructuring in 2004/2005 and now has an entirely new suite of offerings right across the field of computing: cs/se/it/is. the school has also had to distinguish its offerings very carefully from those of administrative management, which teaches some material offered by is in other universities – so awareness of the school’s specific role in the computing spectrum is probably better and more finely focused than at other universities. research in is is led by the three is professors and their research groups, as the majority of the other is staff have still to complete their doctorates. the foci are: requirements engineering/cscw and electronic business / business information management, which are wide enough to include any members of staff interested in becoming involved in the information systems laboratory (insyl). in terms of winning australian-based competitive grants, it is obviously much more difficult for is academics than for cs academics in australia – especially since the creation of nicta, which absorbs the lion’s share of the australian it research funding. distinctive features of is in curriculum at unisa the school of cis endeavours to offer a suite of offerings which cover the entire computing spectrum, from cs right through to is. our degree programs, therefore (both at ug and pg level), offer a wide range of options. incentives for both teaching and research are excellent. indeed, the school of cis at unisa rates itself well above average in terms of providing incentives for research. the newly-developed bachelor of information systems is a broadly-based degree, with specialisations possible in a variety of more finely targeted areas. at the graduate level, the school offers both conversion masters degrees, which are extremely popular with international students, as well as an advanced masters degree in is. we are also developing a range of double degrees – both with the business division in areas such as accounting and marketing; and with the health studies and education, arts & social sciences divisions. identifying niche markets is a major focus of the is strategic planning and is likely to become increasingly important over time. a major new venture has been the industry-funded chair of business information management, which was established within the school at the start of 2006. funded by the state library of south australia, state records sa (a division of the department of administrative and information services) and fuji xerox, the chair is based on the awareness that the previously separate disciplines of library management, preservation, records and archival management and ict are 206 australasian journal of information systems volume 14 number 1 nov 2006 becoming increasingly inter-twined. the sponsors have funded the development of a suite of new degree programs which will provide a holistic approach to the impact of all these areas on the management of business information. distinctive features of is in research at unisa the school has a very large phd student population, of whom 17 are specifically is students. as is the case in so many australian is schools/departments, lack of sufficient numbers of phd supervisors is a limiting factor. qualified supervisors therefore tend to have the maximum number of phd students permitted by university regulations – although this lack is slowly becoming less critical, with a number of phd graduates obtaining teaching positions within the school and taking on supervisory roles themselves. this has been especially beneficial within individual research laboratories, as newly-qualified staff members expand both the supervisory capacity and the laboratories’ ability to develop grant applications. the school has one major research ‘centre’, which is funded by the university to support research activities: the advanced computing research centre (acrc) which, in turn, supports a number of research ‘laboratories’ (the term is taken from mit’s approach to grouping researchers). inevitably, there is some overlap between the activities of many of these laboratories, but two of them are primarily is-focused – the information systems laboratory (insyl) being a solely is-focused ‘broad church’ group; and the strategic information management laboratory (siml) being strongly is-focused. there are two further laboratories (health informatics and enterprise security) which have at least a partial is focus and some interaction with the is group. the table below summarises the research foci of the two major is research groups. school of cis information systems research laboratory foci insyl siml socio-organisational systems data acquisition business information management data quality education interoperability standards e-business/e-commerce knowledge management decision support and km data warehousing and data mining is methodology data security table 3 – research group foci within the school there is a lack of local funding for research in any area of computing within south australia, although the three universities work fairly well together and have created some highly successful technical joint ventures (the sacitt group, and the sabrenet initiative, for example). with is playing such a minor role in the state, obtaining funding for any research activities has proven extremely difficult (although, as noted a little later in this paper, individual academic staff members have been able to build up 1-on-1 relationships at various levels of state and local government; and thus obtain focused funding for specific projects). lack of a single ict minister at the state government level adds to this problem. as already noted, south australia has six ministers responsible for various aspects of ict, so that there is little consistent policy on any part of the ict research or practice spectrum. this makes is very difficult 207 australasian journal of information systems volume 14 number 1 nov 2006 to deal with the state government at a strategic level – relationships exist between individual government agencies and individual institutions (and even individual academic staff members). despite this impediment, however, the school has been highly successful in building 1-on-1 relationships which have led to significant funding opportunities. the ‘branch office’ status of adelaide means that developing strategic relationships with private sector companies is also very difficult. most of the school’s partner companies must refer all decisions to their hos, usually located in sydney or melbourne. research partnerships with industry therefore tend to be focused on innovative smes, rather than on larger companies – as is common for universities in other states. given the current lack of funding at the state and federal levels (with nicta absorbing almost all the federal ict funding, it is increasingly difficult to obtain government funding for is within australia at all), other avenues of research funding must be found. industry support is the logical place to look, as well as less obvious sources of government funding (targeted funding of one sort or another). this approach is beginning to bear fruit for the school. characteristics of is in the state south australia has a number of very innovative and successful smaller businesses working in the ict sector. this, combined with the lack of larger companies in the area has created an unusual profile in the state. even eds, which had a sizeable representation in adelaide under the former government, having won a contract to handle all government ict outsourcing, has now ‘rightshored’ its operations to a significant extent following the current state government’s change of its outsourcing policy (all government ict activities are now outsourced to inter-state and offshore companies). the major ict group within south australia is, of course, the defence industry. the decision to build the ‘smart ships’ in adelaide has bolstered the existing activities of many high-tech defence companies which had been working with asc (formerly the australian submarine corporation) on the development of the collins class submarines. key people who have impacted the is curriculum at unisa a number of industry players have had an influence on the is curriculum over past decades and, of course, past and present heads of school have also had an impact – terry robbins-jones was particularly influential during the 1990s. a short-lived doctoral school, developed by trevor woodharper and mike metcalfe in the mid-to-late 1990s, defined research directions for about five years. more recently, the curriculum has been completely redeveloped by andy koronios and paul swatman to reflect more contemporary approaches to is education. 208 australasian journal of information systems volume 14 number 1 nov 2006 references boyatzis, r.e. (1998) transforming qualitative information: thematic analysis and code development, sage publications, newbury park, ca. constas, m. a. (1992) qualitative analysis as a public event: the documentation of category development procedures, american educational research journal, vol. 29, no. 2, pp. 253266. dixon-woods, m., agarwal, s., jones, d., young, b. and sutton, a. (2005) “synthesising qualitative and quantitative evidence: a review of possible methods”, journal of health services research & policy, vol. 10, no. 1, pp 45–53. ridley, g. (2006) “characterising information systems in australia: a theoretical framework”, australasian journal of information systems, vol 13 no 3, pp. 38-60. yin, r. (2003) applications of case study research (2nd ed.), sage publications, newbury park, ca. 209 australasian journal of information systems volume 14 number 1 nov 2006 the authors professor andy koronios is the head of the school of computer & information science, in the division of information technology, engineering & the environment. andy has extensive experience in both commercial and academic environments and has interests in electronic commerce, information security, internet & web engineering, data quality, multimedia systems as well as online learning systems. professor andy koronios is the head of the school of computer & information science, in the division of information technology, engineering & the environment. andy has extensive experience in both commercial and academic environments and has interests in electronic commerce, information security, internet & web engineering, data quality, multimedia systems as well as online learning systems. paula swatman currently has a double role at unisa. she is the inaugural dais/fuji xerox/state library of s.a. professor of business information management within the school of computer & information science; in the division of information technology, engineering & the environment. in addition to her role in business information management, paula is also the continuing professor of information systems within the school of cis. she has significant academic & industry experience, as well as research & teaching interests, in the area of e-business / e-commerce and many of the newer extensions of this area (such as m-commerce and e-learning). her specialist interests are in the strategic use of e-commerce / ebusiness; the creation and diffusion of e-business models; the use and application of standards; the evolution of the digital content sector; emarkets & supply chain management; rural telecommunications and the impact of m-commerce on rural australia; the implications of the world wide web and the online economy; and e-learning and education in the new economy. 210 placement of is the authors microsoft word ajis112final.doc australasian journal of information systems vol 11 no 2 may 2004 27 a firm level study of information technology productivity in europe using financial and market based measures alan r. peslak, ph.d. assistant professor of information sciences and technology penn state university – worthington scranton 120 ridge view drive dunmore, pa usa 18512 570-963-2640 voice 570-963-2524 fax 570-947-8620 cell arp14@psu.ed abstract for many years, business has invested significant resources in information technology, hardware, software, and manpower. the productivity paradox is the seeming lack of productivity gains despite the increased investment in it. for many years the existence of a productivity paradox has been the subject of research interest. conflicting results have been obtained from a variety of data sets. until this time however there has been no study that has investigated european companies’ use of information technology and its impact on productivity. the objective of this study was to investigate information technology productivity with a new data set from a european published source, and measuring productivity using both market and financial based measures. results of the study indicated that information technology did have a consistent positive impact on firm level productivity in europe for the years 1996, 1997, and 1998. both market and financial based productivity measures provided consistent positive significant returns with regard to it productivity. the major contribution of the study is that it provides an analysis of the impact of european information technology on firm and economic productivity. since 1987, many researchers such as erik brynjolfsson, paul strassman, and loren hitt have studied the problem of whether the huge investment in information technology (it) has had a positive impact on overall productivity in the economy and specifically on the firm. a variety of data sources has been analyzed across different perspectives and researchers have come to different conclusions on this central question. the productivity paradox concept started in 1987 with robert solow, the nobel prize-winning economist, who said that computers can be seen everywhere but in the productivity statistics (solow, 1987). the paradox as presented by strassmann is that, despite large investments in information technology, productivity as measured by cost of goods sold has not increased (mccune, 1998). loveman, in 1988, studied information technology capital versus output over a five-year period, and found no correlation between information technology spending and output increase (brynjolfsson, 1993). the productivity paradox simply stated that empirical investigations in the late 1980s and early 1990s seemed to show that information technology investments, by a variety of measures, were not contributing to overall productivity gains. since the late 1980s, however, a series of studies have provided different, more positive results for information technology investments. the studies have included brynjolfsson and hitt (1996), bharadwaj, bharadwaj, and knosynski (1999) and dewan and kraemer (1998). relevance of the study the general question addressed in this research is similar to many previous studies, i.e., does investment in information technology have a significant positive effect on overall firm productivity and performance. this work, however, adds to the literature in several ways: 1. this empirical study analyzes both market based and financial productivity measures. 2. this research examines current information. 3. it includes european firms. australasian journal of information systems vol 11 no 2 may 2004 28 barriers and issues as with any empirical research, the biggest challenge was to find an appropriate data source to empirically investigate the proposed research problem. the first step in obtaining a data set was to review the data sets used by other researchers in the field. the data sources for the studies presented in the literature search vary from government sources to major publications’ survey data, to private empirical surveys. a data set heretofore unanalyzed was the top 100 it spenders in europe from the publication information strategy. hypotheses to be investigated this study empirically investigates the following research hypothesis: positive productivity gains are recognized for both european firms, in the recent time frame, from information technology investment. significance of the study this study represents a significant research issue due to the sheer size of information technology spending in the economy as a whole, and its expected positive impact on firm level productivity. the significance of the study is that 32.5% of all business capital investment is it related, not including software and systems development. (dos santos, peffers, and mauer, 1993) this is a very significant expenditure for business. it should be determined whether it increases productivity. the study of the productivity impact of information technology on organizations started slowly. though commercial applications for computer technology started in the late 1950s and accelerated through the 1960s and 1970s, there was little research on measuring the benefits gained from information technology spending. the implementation of management information systems and related technology were accepted in organizations through the perceived savings in manpower gained from automating clerical tasks such as payroll, accounts payable, and other financial applications. but beginning in the early 1980s, researchers tried to measure the impact that it was having on the individual firm, in specific applications, and on the economy as a whole. no significant studies focusing on it productivity were developed until the early 1980s. then the pace of studies significantly accelerated and reached its peak in the period 1987-1995. results from two decades of studies have resulted in little consensus on whether it spending is having a significant favorable impact on individual firms or the economy as a whole. data collection as noted, the data source for this study was culled from a published source. information strategy, a united kingdom publication, prepares an annual survey of the top 100 information technology spenders for european businesses. data were obtained from this organization. all it expenditures and personnel information were supplemented with published financial data from compustat and standard and poor’s financial publications and databases. least squares regression methods were used to determine the impact of information technology expenditures on firm productivity. both market based and financial productivity measures were reviewed and tested prior to use. specific financial and market based measures were determined through analysis of compustat and other data sources, and included generally accepted measures such as return on sales, return on equity, tobin's q, market value per share, and return on assets. approach the approach of this study was an empirical analysis at the firm level using financial and market based measures. firm level data were used since most researchers suggest it provides the most accurate indicator of information productivity in the economy as a whole. macroeconomic studies are affected by other significant economic factors and deal only with aggregate data. application australasian journal of information systems vol 11 no 2 may 2004 29 level studies cannot be extrapolated to the economy as a whole. firm level studies as performed by brynjolffson and hitt (1996, 1998a), lehr and lichtenberg (1999), and kwon and stoneman (1995) have proven the most reliable. this study uses the same methodology as a study by the author on us firms (peslak, 2003). the data were entered into separate excel spreadsheets. financial and market information was obtained from the compustat global database. the global database information was obtained through nova southeastern university and school of computer and information sciences. the information obtained from the compustat global database was used to prepare the financial and market based performance measures. the information is noted in table 1. . table 1. compustat data general company information financial data financial ratios market ratios company name current assets return on assets – roa price/earnings ratio – pe ratio primary sic code current liabilities return on equity – roe industry group total assets return on investment – roi employees total long-term debt 3 year return % sales current liabilities cash flow earnings before interest and taxes market data shares outstanding stock price from these data other measures were calculated including • market value = shares outstanding x market price • current ratio = current assets / current liabilities • non-current assets = total assets – current assets • capital intensity = sales / capital • it spending as a percent of sales • tobin’s q (defined in chapter 1) • market to book ratio (market value/book value) • market to cash flow ratio (market value/cash flow) • debt to asset ratio (debt/assets). the financial measures were • return on assets • return on equity • return on investment • ebit -earnings before interest and taxes • three-year return • cash flow. market based measures were • tobin's q. • market value • price/earnings ratio • market price to book ratio • price/cash flow per share. australasian journal of information systems vol 11 no 2 may 2004 30 these served as the dependent variables, or it productivity measures. the key independent variable was some measure of information technology expenditures. for the regression analyses performed, linear equations were used for all analyses as well as the cobb-douglas production function for non-ratio analyses. ordinary least squares regression analysis was performed on the data sets. a new analysis was performed on the information strategy european database and global compustat databases to determine whether similar productivity gains were recognized in the time period 1996 -1998 for european firms. the general form of the non-ratio equation was performance index (x)= a + b current assets + c total assets+ d total long-term debt+ e total sales+ f total it budget+ g total non-current assets+ h number of employees. the equation was x = a + b ca + c ta + d tltd+ e ts +f itbud + g tnca + h emp. the dependent performance measures were tested against these independent variables -cash flow, earnings before interest and taxes, and market value. the cobb-douglas production function was used for these dependent variables. the equations were changed as follows: log performance index (x)= a + b log current assets + c log total assets+ d log total long-term debt+ e log total sales+ f log total it budget+ g log total non-current assets+ h log number of employees. the equation was log x = a + b log ca+ c log ta + d log tltd+ e log ts +f log itbud+ g log tnca + h log emp. the dependent performance measures were tested against the noted independent variables – log cash flow, log earnings before interest and taxes, and log market value. ratios were also used on the information strategy database as performance measures. the form of the equation and the independent variables were as follows: performance index (x)= a + b it as percent of sales+ c capital intensity+ d debt to assets ratio+ e current ratio the equation was x = a + b it+ c ci + d dta+ e cr. the following dependent performance measures were tested against these independent variables – market to book ratio, market to cash flow, pe ratio, roa, roe, roi, tobin’s q, and 3 year return percent. for the 14 performance measures tested, the overall adjusted r 2 (coefficient of determination) was calculated as well as the it variable correlation coefficient and its significance. a p value of .05 was used as the threshold for significance of the independent variable on the dependent performance measure. the significance of other independent variables was also reviewed. australasian journal of information systems vol 11 no 2 may 2004 31 this analysis proceeded through each year of the information strategy data 1996, 1997, and 1998. the results were then reviewed to determine whether the results have been consistent for the period 1996-1998 for european firms. results the result tables use abbreviations for each dependent variable performance measure. the explanations of these abbreviations are shown in table 2. table 2. performance measures abbreviation measure type of measure 3yr 3 year return financial cf cash flow financial cfl log cash flow financial eb earnings before interest and taxes financial leb log earnings before interest and taxes financial lmv log market value market mv market value market mtb market to book ratio market mtc market to cash flow ratio market pe price/earnings ratio market roa return on assets financial roe return on equity financial roi return on investment financial tob tobin’s q ratio market the database from information strategy was assessed with the noted financial and market measures. the first year analyzed is shown in table 3. in eight of the 13 measures, which could be tested, information technology expenditures were significant and positive. (one measure, market value log, was unable to be calculated because of insufficient data points). total observations were fairly small however, at 13 to 16 observations. but for cash flow, market value, market to book value, return on assets, return on equity, return on investment, tobin’s q and 1-year return (3 year return was unavailable for the international database), significant and positive correlation coefficients were recorded. australasian journal of information systems vol 11 no 2 may 2004 32 table 3. 1996 information strategy european summary measure adjr 2 overall p it coefficient it p value cf 0.938 0.0005 2.613 0.041 eb 0.584 0.025 3.424 0.068 mv 0.794 0.01 30.964 0.048 cflog 0.987 0.081 -3.169 0.131 eblog 0.82 0.296 -4.106 0.401 mvlog na na na na mtb 0.916 0.0005 51.782 0.0005 mtc -0.35 0.878 74.838 0.672 pe -0.404 0.926 7.046 0.992 roa 0.54 0.008 116.499 0.001 roe 0.337 0.061 218.133 0.011 roi 0.513 0.011 187.234 0.002 tob 0.921 0.0005 28.609 0.0005 3yr 0.348 0.056 1.278 0.012 in the 1997 analysis of top european firms, the number of factors significant and positive was reduced, however. only five of 14 were significant and positive at the p < .05 level. they were market to book, market to cash flow, tobin’s q, 1 year return, and return on assets. at the p < .10 level two more factors would be included, earnings before interest and taxes, and market value. complete results are shown in table 4. the number of data points rose to 22 to 26 depending on the dependent performance measure. table 4. 1997 information strategy european summary measure adjr 2 overall p it coefficient it p value cf 0.16 0.21 -0.0966 0.974 eb 0.713 0.0005 1.577 0.067 mv 0.638 0.0005 15.358 0.075 cflog -0.599 0.772 0.0793 0.967 eblog -0.077 0.669 -0.0249 0.988 mvlog 0.584 0.288 0.08213 0.946 mtb 0.24 0.03 11.749 0.011 mtc 0.623 0.0005 38.691 0.002 pe 0.126 0.165 38.698 0.151 roa 0.379 0.003 27.911 0.0005 roe 0.103 0.154 7.381 0.797 roi 0.227 0.032 10.475 0.516 tob 0.439 0.001 8.806 0.0005 3yr 0.919 0.0005 2.106 0.0005 the final year of the information strategy database available and analyzed was 1998. this year generally had 40-60 observations. here, once again, there were eight measures that had significant p values (at p < .05) for the information technology spending coefficients. the coefficients for cash flow, earnings before interest and taxes, market value, cash flow log, market to book, return on assets, return on investment, and tobin’s q all were positive. this suggests a positive correlation between information technology expenditures and the dependent performance measure. another australasian journal of information systems vol 11 no 2 may 2004 33 factor, return on equity, was positive and significant at the p < .10 level. complete details are shown in table 5. table 5. 1998 information strategy european summary measure adjr 2 overall p it coefficient it p value cf 0.886 0.0005 5.401 0.0005 eb 0.754 0.0005 3.768 0.0005 mv 0.356 0.0005 79.726 0.001 cflog 0.795 0.0005 0.447 0.046 eblog 0.755 0.0005 0.236 0.109 mvlog 0.17 0.155 0.539 0.193 mtb 0.142 0.012 102.418 0.002 mtc -0.077 0.937 56.737 0.632 pe -0.065 0.865 -14.153 0.922 roa 0.183 0.003 137.869 0.0005 roe 0.12 0.022 319.641 0.071 roi 0.253 0.0005 390.652 0.0005 tob 0.106 0.034 44.695 0.002 3yr -0.01 0.484 5.932 0.25 examining the european information strategy information, overall, from a financial versus market measure perspective results in tables 6 and 7. table 6 shows the european results for the 3 years for all financial measures. twelve of the twenty-four analyses demonstrate strong positive relationships between information technology and the dependent performance measure significant at the p < .05 level. of these, return on assets was positive and significant at the p < .05 level for all three years. australasian journal of information systems vol 11 no 2 may 2004 34 table 4-36. is financial european summary year measure adjr 2 overall p it coefficient it p value 96 3yr 0.348 0.056 1.278 0.012 97 3yr 0.919 0.0005 2.106 0.0005 98 3yr -0.01 0.484 5.932 0.25 96 cf 0.938 0.0005 2.613 0.041 97 cf 0.16 0.21 -0.0966 0.974 98 cf 0.886 0.0005 5.401 0.0005 96 cflog 0.987 0.081 -3.169 0.131 97 cflog -0.599 0.772 0.0793 0.967 98 cflog 0.795 0.0005 0.447 0.046 96 eb 0.584 0.025 3.424 0.068 97 eb 0.713 0.0005 1.577 0.067 98 eb 0.754 0.0005 3.768 0.0005 96 eblog 0.82 0.296 -4.106 0.401 97 eblog -0.077 0.669 -0.0249 0.988 98 eblog 0.755 0.0005 0.236 0.109 96 roa 0.54 0.008 116.499 0.001 97 roa 0.379 0.003 27.911 0.0005 98 roa 0.183 0.003 137.869 0.0005 96 roe 0.337 0.061 218.133 0.011 97 roe 0.103 0.154 7.381 0.797 98 roe 0.12 0.022 319.641 0.071 96 roi 0.513 0.011 187.234 0.002 97 roi 0.227 0.032 10.475 0.516 98 roi 0.253 0.0005 390.652 0.0005 the market measures shown in table 7 show similar strength in relationships between information technology expenditures in european firms and productivity. nine of the seventeen analyses showed positive and significant (at p < .05) relationships. the tobin’s q measure was significant at p < .05 and its correlation coefficient was positive in all three years analyzed. the results of the information strategy database thus confirm the conclusion that there is no significant difference between financial and market based measures in determining information technology productivity. australasian journal of information systems vol 11 no 2 may 2004 35 table 7. is market european summary year measure adjr 2 overall p it coefficient it p value 96 mtb 0.916 0.0005 51.782 0.0005 97 mtb 0.24 0.03 11.749 0.011 98 mtb 0.142 0.012 102.418 0.002 96 mtc -0.35 0.878 74.838 0.672 97 mtc 0.623 0.0005 38.691 0.002 98 mtc -0.077 0.937 56.737 0.632 96 mv 0.794 0.01 30.964 0.048 97 mv 0.638 0.0005 15.358 0.075 98 mv 0.356 0.0005 79.726 0.001 96 mvlog na na na na 97 mvlog 0.584 0.288 0.08213 0.946 98 mvlog 0.17 0.155 0.539 0.193 96 pe -0.404 0.926 7.046 0.992 97 pe 0.126 0.165 38.698 0.151 98 pe -0.065 0.865 -14.153 0.922 96 tob 0.921 0.0005 28.609 0.0005 97 tob 0.439 0.001 8.806 0.0005 98 tob 0.106 0.034 44.695 0.002 overall it can be stated that a significant positive relationship was generally established between information technology expenditures and european firm level productivity. the hypothesis was supported. conclusions and implications the overall objective of the study was to determine whether a productivity paradox existed and currently exists at the firm level for european firms. in other words, the question was whether information technology had a positive impact on european firm level productivity. the results of the study generally found a positive relationship between it spending and firm level productivity for european firms. a productivity paradox at the firm level was not observed in this study. anticipated benefits the implications of the findings may influence corporate spending on information technology in europe, since information technology expenditures generally added to the productivity of the european firm. positive correlation between it spending and productivity for european firms was found to exist. however, it spending should be scrutinized closely. a relationship was found for european firms, but not all years and productivity measures were affected. care must be taken with european it expenditures. this study advances knowledge of the impact of information technology spending for current time frame, international analyses, using both financial and market based measures. references bharadwaj, a., bharadwaj, s., and knosynski, b. (1999). “information technology effects on firm performance as measured by tobin's q”, management science, vol. 45 no.6, pp. 10081024. australasian journal of information systems vol 11 no 2 may 2004 36 brynjolfsson, e. (1993). “the productivity paradox of information technology”, communications of the acm, vol. 36 no.12, pp. 67-77. brynjolfsson, e. and hitt, l. (1996). “paradox lost? firm-level evidence on the returns to information systems spending”, management science, vol. 42 no.4, pp. 541-558. brynjolfsson, e. and hitt, l. (1998). “beyond the productivity paradox”, communications of the acm, vol. 41 no.8, pp. 49-55. kwon, m. and stoneman, p. (1995). “the impact of technology adoption on firm productivity”, economic innovation and new technology, vol. 3, pp.219-253. lehr, b., and lichtenberg, f. (1999, april). “information technology and its impact on productivity: firm-level evidence from government and private data sources, 19771993”, the canadian journal of economics, vol. 32 no. 2, pp. 335-362. mccune, j. (1998, march). “the productivity paradox”, american management association international, pp. 38-40. peslak, a. (2003, summer). “it productivity in us using financial and market based measures”, journal of computer information systems, vol. 43 no.4, pp. 72-80. solow, r. (1987, july 12). “we'd better watch out”, the new york times, pp. 36. tam, k. (1998, march). “the impact of information technology investments on firm performance and evaluation: evidence from newly industrialized economies”, information systems research, vol. 9 no.1, 85-98. the australasian journal of information systems (ajis) began in 1993 and was based on the simple realisation that there was a need for an academic ‘outlet’ for is research in australia australasian journal of information systems volume 14 number 1 nov 2006 the state of is in australian universities – new south wales report jim underwood department of information systems, university of technology, sydney jim@it.uts.edu.au ernie jordan graduate school of management macquarie university, sydney ernest.jordan@mgsm.edu.au abstract this paper, examining information systems in new south wales universities, highlights the significance of new south wales as the most populous state in australia. rather than offering a comprehensive coverage of all information systems courses in the state, the paper gives a broad overview of information systems in the state’s universities while seeking to highlight the distinctive characteristics of some of the universities, deemed to have particular significance in the state. the view portrayed is of an environment threatening the continuing existence of information systems in some of the state’s universities. again, the state of information systems research in the state’s universities is characterised by diversity and limited collaboration. introduction the information and communications technology (ict) industry is a significant part of the nsw economy, ranking 17th in the world according to a nsw government report (nsw, 2001). this report also suggests that this significance is reflected in some 70% of the top 250 australian it head offices being located in nsw. there are long established university programs that support this economy. as the most populous state, it is natural that there are more universities and more programs to report on. this report will concentrate on those universities that have been most significant in the development of the is discipline in nsw and give outline information for the others. the basis for this selection will be explained in the following section of the report. the universities that are reported in detail are university of new south wales (unsw), the university of sydney, wollongong university, university of technology sydney (uts) and charles sturt university. 193 australasian journal of information systems volume 14 number 1 nov 2006 overview of the is presence in nsw universities as some universities have more than one administrative entity that is concerned with is, these are shown separately in table 1 below. the university of nsw has had an historical place in the development of is in australia going back to the leadership of cyril brookes. this place has wavered over time but the school remains a significant player in is research and education. uts also has a long history of is that has complemented and competed with unsw. both of these have organisational units with names including is and the only other such unit is the formal ‘discipline’ of is at wollongong. this alone merits their in-depth treatment. the only other university that is considered in detail is u sydney, for two reasons. first, the relatively recent establishment of a ‘discipline group’ of is in the faculty of commerce is seen to be a strategic move warranting consideration, and second, the significant status and historic role of the dept of computer science and its is members. charles sturt university has long had is related courses, and has been a major provider of distance learning. here we discuss mainly the riverina operation, which has been closely associated with library studies. dealing with the other units in turn, university of western sydney (uws) had a long term role going back to data processing days, but this is much reduced today. macquarie university has several is degrees, including majors in the bachelor of business administration (bba) but the few is faculty remain fragmented. southern cross university (scu) has significant distance education enrolments and is a relatively new entrant in the marketplace with some significant innovation. their influence on the discipline in nsw is as yet not large, but the research activity at scu in particular could establish a future place. both university of new england (une) and newcastle are under-developed in terms of is. we now present an outline of the five identified universities. university administrative entity home faculty no. of is academics unsw school of information systems, technology and management commerce and economics 20 unsw group agsm 1 u sydney group in school of it science ca.5 u sydney business is ‘discipline’ in school of business faculty of business and economics 12 uts dept of information systems information technology 20 u wollongong ‘discipline’ of is in school of economics and information systems commerce 15 u western sydney group in school of computing and mathematics health and science ca.10 macquarie u group in dept of computing information & communication sciences ca.5 macquarie u group in dept of accounting economic & ca.3 194 australasian journal of information systems volume 14 number 1 nov 2006 & finance financial studies macquarie u group mgsm 1 charles sturt u group in school of information studies science and agriculture 10+ southern cross u group in school of multimedia and it commerce and management 12 u new england nil? u newcastle group in school of design, communication and it science and it ca.3 table 1 is presence in nsw universities university of new south wales the school of information systems, technology and management (sistm) has a long and varied history – there was a department of information systems in the early 1980s. today it is placed in the commerce faculty but does not have a core unit in that faculty’s flagship bcomm program. it has its own bsc in is as well as a major in is available in many degrees. high entrance scores are required in all programs. the distinctive bit degree has industry training (which entails students spending a period of time working in industry during the degree studies), industry-funded student scholarships, and includes an honours year. a master’s degree in is is being started but there is support teaching for other master’s degrees. the school has long been active in research, with strong performance in quantitative aspects of is, especially in the 1980s and early 1990s. there is also a strong performance in major research grants and doctoral student completions. with the appointment of a new head of school (dubravka), qualitative research methods in is are likely to become more prominent. highly-rated is research in management areas is conducted at the australian graduate school of management (agsm) but there is no collaboration with sistm. sistm is seen to be distinct from computer science within the unsw. university of sydney as a traditional university is at university of sydney has until recently been primarily located in computer science, with a sybase chair in is being appointed there in 1998, although the position is now called ‘language technology’ reflecting the interests of the incumbent. other areas, such as librarianship and informatics (in the faculty of arts) have developed independently. the major in is offers the same units to various degrees. there are now some 30 students (out of 100) doing an is project in their bsc honours degree – the largest honours group in nsw. the computer science department is developing a masters in it that will have an is major. by contrast, the newly established business is ‘discipline’ (one of ten) in the faculty of commerce teaches the is core to some 1000 students in the bcomm program, aiming to recruit them to the is major. students in the accounting stream take additional is units, also in mbus, mcomm. a new mbusis degree starts in 2006. the establishment of this discipline was seen as a strategic move by the dean, to give visibility and to promote interdisciplinary research. five research areas have been selected to build strength in the group. 195 australasian journal of information systems volume 14 number 1 nov 2006 there is not much visibility of information systems (in either incarnation) in the wider university community. university of technology, sydney from the days of the nsw institute of technology, vocationally-oriented programs have been developed. at the undergraduate level, all courses were in either part-time or "sandwich" mode, requiring students to spend usually one year in suitable paid employment before graduation. in response to an increasing number of international students and the decline of employment opportunities, industrial experience became optional in 2002, and the number of students graduating with work experience has gradually decreased. to some extent this experience is now "simulated" in a large group systems development project. since 1968 computing has been taught in a variety of science-oriented faculties. a second business information systems group developed within the faculty of business. these two groups were brought together around 1980 in a faculty of mathematical and computer sciences; this faculty became the faculty of it in 2000. a number of conflicts between the "science" and "business" groups in the 1980s are still impacting on the faculty. in the faculty of it, information systems was established as one of three departments but with limited autonomy. as a matter of policy, all faculty courses are inter-departmental, as the "integration" of computer science and information systems is seen as a key strength of uts. in the late 1990s the faculty grew rapidly and was a strong contributor to uts income, but the faculty is now suffering from the reduced enrolments in it common across the sector. this has particularly impacted the information systems department as it was had a large number of students from the faculty of business enrolled in information systems majors. in terms of research profile and general visibility within the university, the is department remains relatively weak alongside the other departments. research in the is department took a lower profile historically and even today the proportion of staff without doctorates is comparatively high. university of wollongong the is discipline has some 400 eftsu at the moment, making it probably the largest outside sydney. there are both undergraduate and graduate programs. its history goes back to 1984 when an is group from the then wollongong institute joined the commerce faculty. in 1991 a faculty of informatics was created but is stayed with commerce. programs were supported by the local presence of bhp, one of australia’s largest companies, and some research success was achieved in the 1990s. currently the discipline is perceived as suffering somewhat from the increasing proportion of overseas students, difficulties in attracting research students and lack of distinctive recognition within the university. charles sturt, riverina charles sturt is a multi-campus regional university, so the administrative position and visibility of is depends to a large extent on cross-campus structure and politics. at present is lies within the faculty of science and agriculture, which controls staffing, major funding and course and subject design. at wagga wagga, on the other hand, arrangements have remained fairly stable, with is as a "discipline area" in the school of information studies. in the past this placement has encouraged an emphasis on the "information science" aspects of is (an aspect largely ignored elsewhere) but this specialisation is diminishing. 196 australasian journal of information systems volume 14 number 1 nov 2006 as a regional university, charles sturt has a majority of students studying in distance mode, and has good relationships with the community, tafe and local business (which is not strong). research projects are mainly individual, and a number of full-time academics are currently enrolled in phds. collaboration tends to be with universities in the nearby act rather than with the rest of nsw. researchers do not particularly identify (or feel the need to identify) with is as such. there is a feeling that regional universities are (for political reasons) less threatened than others by fluctuations in student demand and research fashion. results the following sections summarise and discuss the responses under the headings adopted by the survey instrument that is presented in ridley (2006) in the special edition. relative size of is presence students undertaking strongly identified is undergraduate degrees can only be found at unsw. uts has an it degree with is as an "equal" component, and a cross-faculty business/computing degree. these universities also offer a variety of majors in non-is programs. this format, majors in other programs, is the bread-and-butter of is in all the other universities. the size of the is presence therefore depends upon the university’s enrolment in the core programs that offer the majors and the extent of compulsory subjects in those programs. large accounting programs mean that many students will need to do is and larger staff numbers are entailed. with more than 1000 students in commerce at the university of sydney, a larger is unit is created. another characteristic is the fragmentation of is academics across departments or faculties. administrative placement of is with the growth of enrolments in commerce over the last 20 years, there has been some advantage to those is units that are placed in that faculty. however, a variety of locations, especially science and computer science, persist. most is groups remain as ad hoc or informal groups within larger departments. local contingencies it is noteworthy that when asking participants about ‘local contingencies’ responses reflected ‘very local’ issues, such as organisational politics, nearby employers and competitors. there were no “nsw” issues that came up. for example, wollongong’s support from bhp; alternative perspectives at uts and unsw; faculty rules at sydney. is as a separate field while the four universities highlighted have some structural recognition of is as a separate field, it was widely held that this did not extend to the wider university communities. indeed the interviewees would be challenged to agree on a definition of is that was mutually satisfactory. for example, icis, acis, etc were not universally agreed as central conferences. the fragmented state at macquarie university (ad hoc is group in computing, accounting is group in accounting, information management taught in statistics and is management in the management school) is symptomatic. 197 australasian journal of information systems volume 14 number 1 nov 2006 distinctive is curricula the language technology emphasis in one group at u sydney is the most distinctive. support classes within commerce programs are possibly the least distinctive across universities. uts has systematically innovated with specialist programs over the years, especially at the master’s level. “cooperative” programs, that is with an industrial training component, are only offered at uts and unsw. distinctive features of is research the is group at unsw has been productive over a long period particularly in quantitative aspects (some would describe as software engineering). this emphasis has declined and qualitative research is growing there. each group at u sydney is pursuing independent research strategies, with the scam detection project of jon patrick being very large, although not labelled as is. at uts igor hawryszkiewycz leads a large ongoing project on collaborative systems software, but few full-time is staff are involved in this project. ernest edmonds' new "creativity and cognition" studio is under the is umbrella, but works with computer scientists as much as with other is academics. large research projects in is are otherwise few in number. research activity tends to be fragmented and diverse. small research groups, especially of doctoral students, at agsm and macquarie graduate school of management (mgsm) have existed without undergraduate programs to build the staff numbers to critical mass. perceptions of other universities founding roles of unsw, uts and sydney were widely recognised although there was also some recognition of interstate universities and researchers. key people in the region “round up the usual suspects”. members of acphis and acis regulars were the most widely cited. ron weber has been influential outside australia – perhaps more than inside. dubravka cecez-kecmanovic and steve elliott are in significant strategic positions in nsw at the moment. conclusion the already fragmented condition of is in most universities means that with further reductions in undergraduate recruitment their positions will not improve and can deteriorate further. in the view of the authors, the strong candidates for survival and prosperity are sistm at unsw, and business is at u sydney. the chasing pack is led by the department of is at uts and the is discipline at wollongong, which face difficulties at the moment but do have the critical mass to move forward, if the way is clear. postgraduate specialist programs remain strong at uts. research is characterised by almost no cross institution cooperation and little within institutions. the ‘changing of the guard’ at unsw and u sydney may bring about revivals. the problem will be, however, to avoid the "lost opportunities of the 70s". nsw (and sydney in particular) had at least two institutions with plenty of highly employable undergraduate students, the respect of the it 198 australasian journal of information systems volume 14 number 1 nov 2006 industry and other institutions, and proximity to a large part of the australian it industry for support and research opportunities. yet is in the institutions failed to develop an identity or a presence. it can be argued that this was partly due to a premature desire to define a discipline. elias (quoted in ridley (2006)) suggests that modelling low-status on high-status disciplines is bound to fail. the moves suggested by ridley (2006) for establishing a discipline will, if pursued by individuals or specialised research groups, work against the establishment of a broadly supported discipline. individual prestigious reputations are established at the expense of wide collaboration, standard research methods and unique symbol systems are used to exclude rather than enrol. is (and perhaps all new disciplines) defines itself by what it is not, particularly "not computer science". it is easy for those outside a developing prestigious group to think "but that's not really is". the drive for academic recognition, particularly in the current university research climate, can also damage relations with industry, relations that are vital to the identity of is. the question is, how can we build a "discipline that isn't a discipline". references nsw government (2001) “skilling people for an information society”, office for ict, nsw government, http://www.oict.nsw.gov.au/pdf/3.4.1-ict-skills.pdf, accessed march 2006 ridley, g. (2006) “characterising information systems in australia: a theoretical framework”, australasian journal of information systems, vol 13 no 3, pp. 38-60. the authors jim underwood is a senior lecturer in the department of information systems at the university of technology, sydney. he previously worked at the canberra cae and, a long time ago, in several it consulting companies. he has also held visiting positions at uc berkeley, lse, birkbeck college, staffordshire university and university college cork. his interests include the politics of information systems development, the dynamics of learning, and the meaning of knowledge and self in a virtual society. these are approached through the philosophical lenses of foucault, latour and deleuze (amongst others). he has a phd in information systems, a bsc in pure mathematics and diplomas in administration and recreation planning. professor ernie jordan has worked as an academic and it practitioner in four continents and is now professor of management at the macquarie graduate school of management in sydney. he teaches in the areas of it management, statistics and operations research. he is currently the director of higher degrees research at mgsm, managing the phd and dba programs. his research has specialised in the area of strategic application of information technology with an emphasis on electronic business. more recently, he has researched areas of it risks, both strategic and operational, with significant activity in the areas of business continuity and it governance. this work led to a recent book “beating it risks” with wiley. 199 http://www.oict.nsw.gov.au/pdf/3.4.1-ict-skills.pdf australasian journal of information systems volume 14 number 1 nov 2006 200 overview of the is presence in nsw universities university of new south wales university of sydney university of technology, sydney university of wollongong charles sturt, riverina relative size of is presence administrative placement of is local contingencies is as a separate field distinctive is curricula distinctive features of is research perceptions of other universities key people in the region the authors microsoft word ajis12-1final repaired.doc ajis vol. 12, no. 1 september 2004 54 e-commerce in saudi arabia: adoption and perspectives sadiq m. sait 1 khalid m. al-tawil 2 syed ali hussain 1 1 computer engineering department, box 5065 college of computer science & engineering, king fahd university of petroleum & minerals dhahran, saudi arabia {sadiq,hussain}@ccse.kfupm.edu.sa 2 director, network information center ministry of interior, riyadh, saudi arabia altawil@ccse.kfupm.edu.sa abstract among one of the most far-reaching influences of the internet, especially from the perspective of business and market models, is the concept and application of ecommerce. today, this erstwhile ‘experimental’ commerce model has become a standard platform for streamlining business flows and broadening product outreach. this is validated by the significant emphasis by a number of countries in developing robust ecommerce architectures tailored to suit their economical-social systems. an interesting case study in this perspective is the kingdom of saudi arabia, especially given its emphasis on local, regional and religious traditions. presently, with a tremendous growth rate in internet connectivity, the kingdom is actively moving towards large-scale adoption of ecommerce systems. in 2001, a two year project was initiated to study the present influence of internet on social, educational and business systems within the kingdom. one of the major aspects was the analysis of user responses collected over the two-year period through a number of surveys, which were designed based on two prominent behavioural models: theory of planned behaviour and roger’s theory of diffusion of innovations. the consequent analysis of the results was supported and in some cases verified with linear regression models. this paper reports on the consequent findings, which identify factors that may significantly affect the adoption of ecommerce in the kingdom. introduction modern saudi arabia today represents a unique and convergent blend of social conservatism and technological prowess, a tremendous transformation from an isolated, desert land that it was over 50 years ago. achieving this however has often mandated novel approaches, which strive to merge technology into the existing societal structure and avoid possible confrontations of interest. the growth of the global internet and its absorption into the mainstream saudi society has arguably been the most significant of such endeavours. the effort is ongoing as the kingdom further opens internet access, exploring opportunities for the internet in education, government and commerce. the saudi government has shown increasing interest in pursuing ecommerce systems for promoting national and regional businesses, especially through the structuring of a legal framework for online transactions. given the kingdom’s regional influence, such a move would very likely impact business flow and commerce systems in the middle east. this study reports on the factors in terms of both infrastructural support for, and behavioural responses to ecommerce adoption within the kingdom. as such, it is highly significant as a tool for policy makers and business planners with commercial interests in the middle east. although saudi arabia has been linked to the internet for several years, wide-scale public access to the world-wide-web was initiated in january 1999. given the unmediated nature of the medium, the main focus areas emphasized achieving effective censorship of inappropriate content without creating any noticeable bottleneck in throughput and access speeds. the technological implementation involved routing all internet access through a central proxy server at the king abdul-aziz city for science and technology (kacst) in the capital city of riyadh. the history of the internet in the kingdom and its present architecture are well documented both officially and by independent reviewers (isu 1998; tawil 2001). over the past decade, the kingdom has taken measured but strong steps towards developing a diversified economy, with emphasis on developing its information technology sector. a core issue ajis vol. 12, no. 1 september 2004 55 has been the introduction of robust ecommerce initiatives through out the country spanning both vertical business-to-consumer (b2c) and lateral business-to-business (b2b) market systems. as of the time of this study, these initiatives are still in the pipeline and haven’t yet been fully achieved. however, to realize the afore-mentioned generic market systems, two different sets of variables have to be understood and measured. the first of these deals with the infrastructural prerequisites to adopting ecommerce systems such as ubiquitous internet access, broadband connectivity, transaction security etc. the second set of variables addresses the social and market response factors, such as acceptance of internet among general society, inclination towards online transaction systems etc. given the strongly conservative nature of saudi society and its gender-based segregation principles, these social variables form an exceedingly unique case study into ecommerce adoption. in 2001, a research team from the king fahd university of petroleum & minerals (kfupm), under a project grant from kacst initiated a comprehensive study of these two interrelated sets of variables. the study, which lasted for two years concluding in 2003, surveyed a wide range of internet users and businesses regarding their internet usage, inclination towards ecommerce and existing ebusiness systems if any. the questions and the survey methodology were based on concepts drawn from the theory of planned behaviour (tpb) and roger’s theory of diffusion of innovations (doi). the resulting model focuses on studying parameters influencing society’s response and behaviour. the results were analyzed to determine trends in usage as well as inclination and bias regarding online commerce. these results collectively portray a clear view of factors that will significantly impact ecommerce adoption as saudi arabia pursues its it vision. the sectional layout of the paper is as follows. the next section provides a review of recent literature on ecommerce adoption in developing countries. following this, a brief description of the saudi ecommerce scenario is discussed with reference to the country’s present telecommunications infrastructure, internet access facilities and socio-economic demographics. then the research methodology is briefly explained with reference to survey implementation, data collection and the respondent profile. the results of the survey are subsequently presented where the findings are supported through linear regression analysis. finally, conclusions, potential and the various implications of ecommerce deployment in saudi arabia are summarized. literature review the literature available on ecommerce adoption and diffusion in developed countries is quite extensive and diverse. numerous studies have been documented that focus on both b2b and b2c domains. the extensive analysis available is not only based on major generic themes like models, strategies, technology and social issues, but also covers specific issues like electronic branding, security, trust, e-payment, performance measures and privacy (e.g., cronin 2000; kumar et al. 2000; plant 2000; rajput 2000). however, interestingly, the literature documenting ecommerce adoption and growth in developing countries is scarce and often simply anecdotal (travica 2002; young 2003). in this section, we report on some of the recent studies in ecommerce that have focused on developing countries. in 1999 a study reported on e-commerce impediments characterizing argentina kenya, india, and armenia (petrazzini & kibati 1999). these included limited internet accessibility, a lack of competition in international telephone traffic that made access to the international network expensive, a lack of intra-regional infrastructure, and a disproportionate penetration of the telephone in the urban as opposed to rural, more populated areas. another study which focused on south korea identified a major obstacle for ecommerce adoption – lack of customer trust in online merchants (lee 1999). there is a fear that merchants might sell products with defects, and that online payments cannot be recovered even if the product is not delivered. in central and eastern europe, online stores may have some design and infrastructure characteristics that are less conducive to success in international e-commerce, such as longer server response time, less inspiring aesthetics, excessive graphics and animation, and lower transactional capability (travica & cronin 1996; travica & olson 1998). ajis vol. 12, no. 1 september 2004 56 china, with the highest population, still doesn’t embrace ownership innovations (clark 1999). national networks have been developed through competition between two ministries, which eventually merged into one. as of the writing of the study, b2b transactions had been recorded in mainland china, while b2c systems were still being initiated. the impediments include expensive access, lack of a tradition of remote shopping/selling, lack of trust in product quality, and no provision for customers' recourse. a similar study covering impediments to ecommerce adoption in costa rica, and quite representative of existing conditions in latin america was carried out in 2002 (travica 2002). some of the factors identified were direct shopping preference, lack of trust for product quality, oral culture, nascent internet and email culture, infrastructure (limited telecommunication transport and postal services), government monopoly and expensive internet access. another interesting and nonobvious obstacle in the delivery of ecommerce goods is the lack of building numbers in the country. though there are street names in costa rica, however, buildings are not referenced with a precisely enumerated location. in 2002 a study was carried out that reported on limitations for b2b e-commerce in three developing countries, namely bangladesh, kenya and south africa. it was concluded that internet-based b2b ecommerce is not effective in reducing transaction costs or in providing a gateway to global markets for regional enterprises. furthermore, this e-commerce model has only marginally altered the way in international businesses dealing in the garments and horticulture sectors, which are these country’s main trade areas, deal with one another. the reasons in this case are similar to those seen earlier. the conclusion drawn from these case studies showcases a generic set of factors, the lack of which impedes ecommerce growth and adoption building trust, ground rules, information infrastructure and maximizing benefits by seamless globalization (dekleva 2000). in the case of saudi arabia, there has been no study on the adoption and user perspectives about ecommerce. there have been a few efforts that address the issue of internet penetration in the middle east in general (ghashghai 2002). this paper documents the first study working in a broad framework of identifying uses and effects of internet in the country. saudi arabian e-commerce scenario this section briefly documents the present status of saudi arabia’s demographics and infrastructure and relates them to the perspective growth in ecommerce adoption in the kingdom. the strength of the saudi ecommerce market potential lies in the kingdom being the centre of the islamic world with the largest economy ($168 billion) among the arab countries with virtually no direct taxes. with a high gdp per capita income rated at 9000 purchasing power parity (compared to the regional and global averages of 6167 and 7416) and a high annual population growth rate of 3.4%, saudi society provides a potentially wide consumer base for commerce and technology driven initiatives. further, the significant youth population and the presence of large it companies in the kingdom give ecommerce a major advantage and a competitive edge regionally (source: icon group international, 2000). infrastructure-wise, a key indicator for ecommerce readiness is the extent of internet access in the kingdom. according to the most recent estimate by kacst (december 2003), there are 584,000 internet subscribers in the kingdom. with an average of 2.5 users per subscriber, the number of internet users in the kingdom is estimated at 1,462,000. this represents around 5% of the complete population. there are currently thirty licensed internet service providers (isps) in the country responsible for providing internet access to businesses and individuals. in the arab region, this is second only to morocco, which indicates a strong liberalization policy in this sector. the kingdom’s eighteen educational and related organizations avail direct connection from isu with an average bandwidth of 2mbps. dial-up connectivity remains the ubiquitous internet access medium for most individuals, with five million access lines estimated by 2005. figure 1, which shows the growing teledensity throughout the country, implies a strong telecommunication infrastructure capable of supporting large-scale dial-up connectivity. ajis vol. 12, no. 1 september 2004 57 figure 1: growth of teledensity in saudi arabia (source: stc, us census bureau) a reliable metric for growth in business-oriented internet applications is the number of leased lines in the country. these form the main building blocks used by businesses to construct networks capable of supporting high-speed, voluminous ecommerce traffic. currently, the number of leased lines in the kingdom is over 10,000 and estimated to reach 30,000 by the year 2005. [source: saudi telecommunications company (stc)]. regarding broadband connectivity, the government owned saudi telecom company (stc) as well as private isps provide dsl connections for individuals and businesses. the number of internet hosts is another commonly referred metric of internet growth. a host is a domain name that has an ip address ‘record’ associated with it. internet surveys of hosts and servers provide one indicator of comparative internet development between countries. the main limitations in these surveys is the inability to reach all hosts or servers, and the structure of the domain name system being such that there is no guarantee that all hosts under a particular domain are located in a certain geographic location. among the arab countries, the uae tops the list with 59621 hosts with saudi arabia recording 10024 hosts. statistics for other countries in the region are egypt: 18706, lebanon: 7629, and kuwait: 3188 (isc, 2001). however, more relevant to ecommerce growth is the level of security awareness and implementation among individuals and businesses especially in terms of strong encryption measures. currently there are only seven servers in saudi arabia employing strong encryption algorithms (40 bit or more) with four other servers using less robust encryption algorithms (less than 40 bit). this is primarily due to the lack of emphasis on security in the private sector and can be addressed through awareness driven initiatives. presently, kacst is drawing plans to institute a public key infrastructure (pki) system that will enable secure kingdom-wide ecommerce transactions. the saudi arabian monetary agency (sama) is piloting an online payment system for business-to-business (b2b) ecommerce based on the widely used ‘society for worldwide interbank financial telecommunication’ (swift) technology. it is also developing a new electronic securities trading system that will deliver straight and transparent processing of payment and share transfers to individual and business entities. in summary, the overall business environment of the kingdom gives ecommerce great potential for success, which would position the country as a regional leader. however, to achieve this, saudi arabia requires substantial improvements in telecommunication infrastructure with easy and affordable internet access, while shifting towards broadband connectivity. further, this infrastructure would have to be supported by trained and skilled local it professionals to meet the demand for ajis vol. 12, no. 1 september 2004 58 human resources in this technology-intensive area. concurrently, new avenues of imparting the requisite education have to be explored, given the present stress on existing educational institutions. in the next section, the research methodology and the application of theoretical models to survey questionnaire design for the saudi scenario are discussed. research methodology underlying model adoption models strive to achieve a robust predictability for social response, behaviour towards adoption of new trends and technologies. most such models pursue a linear approach, wherein progress is defined in stages such as knowledge acquisition, problem framing, decision, implementation and confirmation (mintzberg et al. 1976; simon 1977; rogers 1995). although some of these models focus only on adoption related activities while taking a black-box approach for the adopter, there has been an increasing trend of incorporating the concept of sense-making into the study. sense-making is defined as the cyclical process of taking action, extracting information from stimuli resulting from that action, and incorporating information and stimuli from that action into the mental frameworks that guide further action (seligman 2000). the models using this concept also account for the adopter’s mental perceptions and the effect of these perceptions towards his or her adoption of new behaviours. there are many models that pursue this adopter-centric approach such as diffusion of innovations theory (doi), theory of reasoned action (tra), technology acceptance model (tam) and theory of planned behaviour (tpb). diffusion of innovations theory (roger 1995) identifies the following five characteristics of an innovation that augment its chances of wide acceptance within the population: its relative advantage, compatibility, complexity, trialability and observability. according to this theory, relative advantage is the degree to which an innovation is perceived as better, while compatibility represents the degree to which the innovation is consistent with existing values and customs. complexity measures the degree to which innovation is considered to be difficult. the last two factors, trialability and observability, measure the degree to which the innovation can be experienced and observed. theory of reasoned action (tra) (fishbein et al. 1975) focuses on two independent factors for determining behavioural intention – the attitude towards behaviour and the attitude towards subjective norms. these determinants correspond to behavioural and normative beliefs where the former refers to the extent of the adopter’s favourable or otherwise reaction (evaluation, appraisal, etc.) toward a given behaviour while normative beliefs considers the probability that referent persons approve or disapprove of performing a given behaviour. tra with its robust nature lends itself well to generalization and has been applied to numerous areas including the modelling of technology acceptance (scannell 1999). further, it has been used to develop a common frame of reference for comparison of independent research studies (davis et al. 1989) acceptance of new trends and systems. with the growing global impact of technology on social systems, these models have been widely deployed to study the trends that govern a society’s responsiveness and behaviour to this technology’s diffusion. tam, which is a derivative of the tra, was designed for the specific purpose of explaining computer-usage behaviour (davis 1989). much less general than the tra, this model incorporates components from information system literature, and assumes two key determinants for describing technology adoption behaviour – the technology’s perceived usefulness and its perceived ease of use. subjective norms are not taken as significant factors in this model. tam has been widely used for studying adoption of various technologies from computer usage to ecommerce (devaraj 2002; moon 2001; jiang 2000; gefen 2000; lederer 2000). the tra model, though has been widely applied in numerous contexts has one chief flaw – it does not address the effect of volitional control on behaviour. the tpb model (ajzen 1991) compensates for this by assuming three independent factors that influence intention: • attitude towards behaviour: this factor, also included in the tra model represents individual perceptions ajis vol. 12, no. 1 september 2004 59 • subjective norms. this factor represents perceived social compulsion to perform or not to perform the specified behaviour. • perceived behavioural control. this factor, which distinguishes this model from tra, represents the perceived ease or difficulty of performing the behaviour and depends on past experience and future impediments. each of these determinants corresponds to three types of beliefs: behavioural, normative, and control. these collectively define the individual’s intention to perform the concerned behaviour or action. their extent of influence on intention varies with situation and behaviour; as such all three factors need not necessarily be considered in every context. the tpb model also has been used for studying technology adoption (scannell 1999). this framework is schematically shown in figure 1. in this study, the potential for ecommerce adoption in saudi society and its widespread acceptance is studied using the theory of planned behaviour, which in turn relies on some of the characteristics drawn from the diffusion of innovations theory. this contrasts with the majority of other studies which are based on tam modelling, which concentrates only on ‘usefulness and ease-of-use’. the tpb with doi accommodates both social and non-social factors. given saudi arabia’s singular social, traditional and religious values, this approach provides a more robust and valid syntax for modelling behaviour predictions. our approach information gathering in the form of responses to a web-based survey was adopted as the primary medium of data collection. this approach was the obvious choice as the objective was to achieve a representative but random sample of internet users in the kingdom. over 4000 responses were collected. the survey questions were meant to achieve feasible metrics that could properly gauge user responses to ecommerce adoption. the tpb model, which was briefly described in an earlier section, has been extensively used for predicting behaviour patterns in numerous contexts including technology adoption (tan 2000; pavlou 2001; pavlou 2002). adapting this model to the saudi context, a person’s intention to adopt ecommerce depends on his attitude towards ecommerce, subjective norms as defined by ‘perceived social pressure to adopt or not to adopt ecommerce’ and perceived behaviour control defined by ‘anticipated ease of using ecommerce’. in the early stages of innovation, subjective norms significantly affect behaviour, as individuals do not have direct experience with the technology. therefore, the views of friends, family members and other social groups around the individual influence his perception. however, when the individual directly interacts and experiments with the technology, his decisions are directed by his own experiences rather than by the views of others (tan 2000; hartwick 1994; taylor 1995). as the survey was web-based and was directed at internet users, who would likely have their own individual perceptions from their online experiences and attitude toward subjective norm perceived behavioral control intention to perform behavior performance of actual behavior figure 2: framework for theory of planned behavior ajis vol. 12, no. 1 september 2004 60 information, this study excludes the impact of subjective norms. only factors contributing to the other two determinants, namely ‘attitude’ and ‘perceived behavioural control’ are enumerated. the research principle was based on the simple but crucial assumption that if internet focused business initiatives such as ecommerce and online banking are perceived by individuals and businesses as facilitating economic progress, there would be a much more favourable inclination to adopt these new technologies. hence the overall objective is to decide which of the above determinants has the most influence on these perceptions. three characteristics from the doi theory are included in our model. these include relative advantage, complexity and compatibility. the factors trialability and observability cannot be used in this study as ecommerce has not taken off in the country as of yet, and correspondingly survey respondents would not be able to relate to any experiences. the rest of this section below identifies and elaborates on the influencing factors that have been considered in this study. attitude towards adoption of ecommerce attitude of the individual is based on his positive or negative views about the intended behaviour, i.e., adoption of ecommerce. the individual develops this perception on the basis of different factors, which are enumerated below (tan 2000; tornatzky 1982). 1. the relative advantage of ecommerce on the individual’s life, i.e., whether it makes life easier forms a core issue. a realistic assumption is that if an individual believes that ecommerce as a business medium will make life easier then it is quite likely that he will adopt ecommerce. the analysis of all subsequent factors is done against this variable. 2. practical compatibility of the user with new technology has a positive effect on its adoption (tornatzky 1982). therefore factors like long-term computer usage, internet experience and wider use of internet services would enhance the chances of adoption of ecommerce. 3. the lower the perceived risk associated with ecommerce services, the higher will be the chances of individuals accepting the technology. this assumption can be supported by considering the argument in a converse manner importance given to internet security and privacy by individuals while shopping online can be interpreted as scepticism of internet services. this would mean that an individual considers online shopping as a high-risk activity, thus resulting in a non-favourable view of ecommerce on the whole. other studies have also considered security and privacy over the internet as an impeding factor to the growth of ecommerce (bhimani 1996). perceived behavioural control perceived behavioural control refers to the anticipated ease or difficulty of performing the said behaviour. in the present context of ecommerce adoption, it refers to the perceived ease and efficiency that users can expect from this new technology. in earlier studies of a similar nature, researchers identified three factors that affect the perceived behavioural control of the individual – his confidence in performing the behaviour, facilitating conditions like availability of resources needed to accomplish the behaviour, and the ease of use of technological innovation (tan 2000; benham 1996). these three factors are translated into the present ecommerce context as follows: 4 an individual’s confidence in his internet skills adds to the chances of his inclination towards ecommerce. 5 the greater the extent of technological support for internet, the more likely that ecommerce and home buying will be adopted. 6 ease of use of home-based or online shopping can significantly enhance chances of ecommerce adoption. additional factors apart from the above two determinants that are part of the theory of planned behaviour, some additional factors have also been identified. although these do not form a part of the tpb they ajis vol. 12, no. 1 september 2004 61 nevertheless add more information to understand the possible effects on user’s online shopping and home-based retailing intentions. 7 gender of internet surfers may play a significant role in influencing ecommerce adoption in the kingdom. given the gender-based segregation policy at all levels of both education and employment, this factor becomes critically important 8 location of residence can also impact people’s perception of ecommerce. naturally, people living in cities are more likely to adopt home-based online shopping rather than those in rural areas. 9 tech-savvy individuals, i.e., people comfortable with using new technologies, are more likely to be inclined towards ecommerce. therefore, wide-scale and extensive use of technological gadgets in a household would be a factor influencing individual intentions regarding ecommerce. the relative advantage of ecommerce is determined by analyzing responses against the effect of ecommerce on an individuals’ life. except for this ‘relative advantage’ factor, all other independent factors are analyzed by studying the linear regression between them with ‘effect of ecommerce on individual’s life’ as the dependent variable. for factors related to practical compatibility, information such as duration of computer usage, internet experience, number and type of internet services used by the individual was collected. further in order to measure the user-perceived risk factor, individual responses on internet security and internet privacy were analyzed. if an individual gives less importance to these factors while agreeing to adoption of ecommerce then it shows his confidence in the internet. in turn, the more confidence one has in internet as a business and transaction medium, the lower is the risk associated with ecommerce. regarding availability of technical support for internet-related problems, users were queried regarding their satisfaction with their isp’s technical help systems. with regards to the ease of home-based online shopping, user opinion as regards to facilities like easy placing and cancelling of purchase orders, reliable delivery, handling of returns and refunds, easy payment procedures etc were collected. a number of the factors mentioned earlier were also addressed directly, such as user confidence in their internet skills, nature of job, education, gender and residence. the technology savvy-ness of users is measured with two metrics number of computers owned and the number of technology gadgets in his/her household. following the above elaboration of factors, the subsequent section reports and discusses the respondent profile and the relevant results seen in the study. results and discussion respondent profile this section presents a profile of the survey respondents in terms of their age, gender, education, and place of residence. this information presents a platform for extending the survey results to a much larger, macro-level statistical construction of ecommerce adoption trends within an entire society. an essential pre-requisite for such an extrapolation is a reasonably large collection of respondent inputs with an effectively random distribution in terms of their attributes. the survey questionnaire was purely web-based, hosted on a king fahd university of petroleum & minerals (kfupm) web server in the public domain. soliciting respondent input was achieved primarily through banner advertisements on broad-spectrum sites such as those belonging to internet service providers and general-purpose commercial portals. as such, with the absence of any targeted advertising, the set of respondent profiles collected can be considered random. with over 4,000 inputs the statistic results can be justifiably extended to represent expected e-commerce adoption trends within the entire populace. ajis vol. 12, no. 1 september 2004 62 age distribution among internet users 2.6 42 33.5 15.7 4.3 0.7 0 5 10 15 20 25 30 35 40 45 5-15 16-25 26-35 36-45 46-55 56-65 age group p e rc e n ta g e percentage of male/female respondents 0 20 40 60 80 100 male female gender p e rc e n ta g e figure 3: respondent profile based on age distribution and gender figure 3 presents the respondent profile based on age and gender distributions. it is seen that most of the respondents fall within the 16-35 age groups, thus showing internet-awareness to be most prominent among the younger and middle-aged generations. the latter graph gains special significance given saudi arabia’s principles of gender-segregation. as expected here, the number of male respondents considerably outweighs females, a fact that can be attributed to wider access opportunities for males such as internet cafés, offices etc. considering the employment status of the survey respondents shown in figure 4, it is seen that a significant percentage are students, which serves to support the age group trend seen in figure 3. figure 4: respondent profile based on employment ajis vol. 12, no. 1 september 2004 63 regarding the demographic distribution of respondents as seen in figure 5, most are from urban areas as compared to towns and rural locations. this is due to the more widely available online access opportunities in cities. distribution of internet users 0 20 40 60 80 100 major city small town rural area p e rc e n ta g e figure 5: respondent profile based on location in conclusion, it can be summarized that the respondent profile mainly represents a majority of young males, mainly students and living in urban areas. this adequately models the present population and social structure in the country, with a large segment of the population under thirties. the lack of female and rural respondents can be best gauged as representative of the internet’s nascent status and application in saudi society. however, these trends will most likely change with the increasing online connectivity in homes and the government’s investment in telecommunications infrastructure in non-urban areas. results majority of the respondents reported positive views about ecommerce and internet banking. as shown in figure 6, 65 % agreed that the use of internet for shopping and banking would make life easy while 10% felt otherwise, with the rest being uncertain. hence, the conclusion is that the relative advantage of ecommerce and internet banking as perceived by saudi internet users would favourably effect their inclination to adopt ecommerce. this finding is in line with earlier similar studies reported in literature (tan 2000; sia et al. 1998). ajis vol. 12, no. 1 september 2004 64 figure 6: relative advantage of ecommerce and internet banking on individual’s life for all the other factors, a linear regression analysis was carried out using the statistical software ‘statistica’. a confidence level of 95% (i.e. α = 0.05) was used. as reported earlier, the analysis of these independent factors is done to identify their effect on adoption of ecommerce based on an assumption that “if use of internet for commerce and banking will make life easier then it is more likely that ecommerce and related services will be adopted”. the results are compiled in table 1 with the significant β values highlighted in bold. considering the ‘attitude towards ecommerce’ determinant, the practical compatibility factor is further broken down into three elements, each of which has a strong significant effect on the adoption of ecommerce. this is because experienced internet and computer users, i.e., those who have used the internet and its wide array of services extensively, are convinced with the importance of ecommerce than those with limited experience. the results are in line with earlier findings where it was reported that increased compatibility with an innovation has a positive influence on individual behaviour (tan 2000; tornatzky 1982; harrington 1999). these three factors computer usage duration, internet experience and use of internet services, are discussed independently below. table 1: effect of different factors on adoption of ecommerce classification factor β intercept attitude practical compatibility a. computer usage duration b. internet experience c. use of number of internet services -0.14 -0.24 -0.25 2.958 2.943 2.669 risk a. importance of internet security b. importance of internet privacy 0.186 0.124 1.660 1.827 perceived behaviour control confidence on internet skills 0.205 1.702 extent of technological support -0.100 2.200 ease of ecommerce -0.35 2.477 ajis vol. 12, no. 1 september 2004 65 additional factors gender 0.121 1.748 city of residence 0.038 2.086 technological savvy-ness a. number of computers in house-hold b. number of tech gadgets in house-hold -0.11 -0.16 2.560 2.594 0.00 5.00 10.00 15.00 20.00 25.00 30.00 35.00 40.00 45.00 < a month 1-6 months 6 months to 1 yr 1-2 yrs > 2 yrs computer usage duration p e rc e n ta g e o f re s p o n d e n ts agree strongly agree a little neither agree nor disagree disagree a little disagree strongly figure 7: effect of computer usage duration on adoption of ecommerce individuals using computer for longer duration agree more strongly with the argument that ecommerce will make life easier. there is a significant relationship between these two factors with β value of -0.14. it is negative because the computer usage period increases in ascending order while agreement with the argument increases in descending order. this is due to the design of the questionnaire. these two factors are plotted in figure 7. it is clear from this figure that as the computer usage duration increases, higher percentages of respondents agree that ecommerce will make life easier. like other compatibility factors, online experience of an individual also significantly affects his/her inclination towards ecommerce. it means that the more exposure to and consequent experience with the internet, users would be more inclined towards ecommerce. this conclusion is drawn after analyzing the results of the regression analysis (β= -0.24) and the graph shown in figure 8. ajis vol. 12, no. 1 september 2004 66 figure 8: effect of internet experience on adoption of ecommerce the third compatibility factor is the number of internet services or operations utilized or performed by the individual. users who have availed a varying range of internet services are more likely to embrace ecommerce. the regression analysis shows that there is a significant relationship between these two factors (β= -0.25). the corresponding graph shown in figure 9 also supports this conclusion. figure 9: effect of use of number of internet services on adoption of ecommerce ajis vol. 12, no. 1 september 2004 67 user perception of risk associated with the internet is measured by the importance given by an individual to online security and privacy. low importance given to these two factors translates into lower perceived risk. its effect on the adoption of ecommerce is seen through the regression analysis. it reports a significant influence of both security and privacy factors on ecommerce adoption. it is seen that those who give less importance to online security and privacy are confident that the same will not be compromised, i.e., they are confident of the internet’s ability to maintain their security and privacy. but these are the people who are comparatively not eager in adoption of ecommerce. however, those who are serious about ecommerce and online businesses are sceptical of the internet’s ability to maintain their security and privacy and thus are wary of these two factors while shopping online. for successful adoption of ecommerce, user confidence in online security and privacy needs to be enhanced. hence it can be concluded that security and privacy concerns are obstacles for adoption of ecommerce. figure 10(a) and (b) further support this conclusion. both figures have an almost identical pattern where the bar chart shows that those who don’t give importance to privacy and security on the internet are not strongly in favour of adoption of ecommerce. however, users who are strongly inclined towards online commerce are sceptical of security and privacy. this result is in line with other studies where researchers have pointed the widely reported concerns of security and privacy as impeding factors in the growth and adoption of ecommerce and internet banking (bhimani 1996). moving further on to the analysis of factors under the “perceived behavioural control” determinant, the effect of factors such as ‘confidence in internet skills’, ‘extent of technological support’ and ‘ease of ecommerce’ are discussed. the confidence on internet skills holds a strong significance for the adoption of ecommerce (β=0.205). users who are comfortable with the internet are more likely to adopt ecommerce as shown in figure 11. other studies have also shown similar results where user confidence in a technology had a significant influence on adoption of new innovations (tan 2000). ajis vol. 12, no. 1 september 2004 68 figure 10(a): importance of internet privacy against ecommerce adoption figure 10(b): importance of internet security against ecommerce adoption the extent of technological support available to users does not significantly affect ecommerce adoption. there might be many explanations for this behaviour. one possible explanation is that local internet users in saudi arabia are not exactly aware of what technological components or ajis vol. 12, no. 1 september 2004 69 building blocks are needed for ecommerce and therefore are indifferent to this important factor. this result is in line with other similar studies in developed countries (tan 2000). another plausible reason could be that users view existing technological support as an inherent component of technology services, and hence don’t give any significance to this factor. figure 11: effect of confidence of internet skills and adoption of ecommerce the factors that facilitate application and implementation on ecommerce have significant effect on its adoption (β=-0.35, intercept = 2.477). individuals who associate significant importance to ease of ecommerce are more likely to be inclined towards it, as evident from the regression results. the findings are compatible with similar studies available in literature (benham 1996). after this analysis of ‘attitudinal’ and ‘perceived behaviour control’ factors, the ‘additional factors’ identified in the research methodology earlier are discussed. these are identified as gender of survey respondent, location of residence and technological savvy-ness. in case of gender, males are more likely to adopt ecommerce than females (β=0.121). this behaviour is in accordance with the particular social structure of saudi arabia where a majority of women stay at home while earning responsibility lies with the male members of the family. therefore men find little time for traditional shopping, which often is a significant recreational activity for the women. therefore, they are less likely to opt for ecommerce. figure 12 shows the graph generated by cross querying gender and inclination towards ecommerce. individuals living in major cities are more likely to adopt ecommerce than those who live in small cities/towns (β=0.038). possible analysis of this finding can be as follows: people living in cities lead busy lifestyles and hence ecommerce is an important facility for effective time-management; also there is a greater likelihood of ecommerce implementation within urban areas given the available infrastructure. the final factor is technological savvy-ness. this is measured on the basis of two metrics: number of computers and number of technology gadgets in home. as reported earlier, both of these factors have significant effect on intention to adopt ecommerce (table 1). as the number of computers in a ajis vol. 12, no. 1 september 2004 70 household increases, the likelihood of ecommerce adoption also increases. the negative sign in the corresponding β value is due to opposite order of these two factors in the survey questionnaire. similarly, more technological gadgets in household are indicative of a user confidence and comfort with technology; this translates favourably for ecommerce adoption. this result is further supported by the plot of corresponding cross queries as shown in figure 13. in this section, the significant factors that influence user intention and subsequent adoption of ecommerce were documented. the analysis was based on a well-known and effective theory of social sciences known as the ‘theory of planned behaviour’ that draws from diffusion of innovations theory (doi). the tpb model analyzed two determinants namely ‘attitude’ and ‘perceived behaviour control’. this research methodology was applied on the sample data collected using a web-based questionnaire. the conclusions were drawn on the basis of linear regression and graph plots of corresponding cross queries. figure 12: effect of gender on adoption of ecommerce (home buying) ajis vol. 12, no. 1 september 2004 71 figure 13(a): effect of technological savvy-ness on adoption of ecommerce versus number of computers application of results the essence of the study lies in its providing a framework for identifying key areas crucial to a successful adoption of ecommerce in the country. we define it as a framework, as these focus points may have to be altered and expanded after the ecommerce implementations to adapt to changes. moreover, given the present nascent status of internet driven commerce in the kingdom, especially with the government evaluating large-scale initiatives, this study provides guidelines to policy-makers as well as relevant businesses. the following are the key areas inferred from the above results and analysis: internet access: internet connectivity within the kingdom is primarily dialup especially for individuals at homes. there is a need to make broadband connectivity more affordable for the end users as compared to its present costs. at the same time, there is a need to focus on communication infrastructure in non-urban areas. similarly, the lack of internet usage among females has to be addressed, primarily through awareness campaigns and lowering of internet rates. ajis vol. 12, no. 1 september 2004 72 figure 13(b): effect of technological savvy-ness on adoption of ecommerce versus number of tech gadgets in household ecommerce facilities: these mainly refer to the need for instituting the required information infrastructure, security of transactions, and a government accountable certification authority and public key infrastructure (pki) encryption system. another important aspect is establishing an efficient and reliable postal address system for residences and businesses. ecommerce awareness and promotion: the lack of online exposure within the kingdom would require awareness and promotion campaigns to enlighten the public about the advantages and applications of ecommerce. also such a program would serve to dispel myths and concerns regarding security of transactions and privacy of data in turn providing a transparent view of ecommerce operations to the end users. limitations of the study any study done on a national scale does have to consider trade-offs between feasibility and accuracy. this may introduce limitations in the results and conclusions drawn as well as variations in the interpretation of data. one such curbing aspect of this study relates to the cross-section of the saudi population from which the data sample was obtained. since the survey model was predominantly web-based, the results presented and the inferences drawn pertain to internet users within the kingdom and may not apply to a large segment of the saudi population, which does not have internet connectivity. however, since this study was conducted in the context of internet ecommerce, targeting existing online users as the focus group of this study can be considered valid. since banner advertising was also used to solicit responses, potential biases such as self-selection bias may be inherent in the data. thus, the findings should be interpreted with caution. conclusions this paper focuses on saudi arabia’s ongoing transition towards the large-scale adoption of ecommerce in the kingdom. the factors that would most likely influence ecommerce especially within the context of its existing infrastructure, internet awareness and social and islamic traditions are elaborated. the work draws on the response of users collected through a web-based survey that lasted for two years. the survey was designed using the ‘theory of planned behaviour’ to develop metrics for behaviour and attitude towards ecommerce in the kingdom. based on the factors uncovered and analyzed, a brief set of recommendations can be put forth. it is clear that one of the main focus areas would be to promote wide-scale use of internet among the populace, both by reducing connectivity cost, enhancing infrastructure and moving towards ajis vol. 12, no. 1 september 2004 73 broadband access. ecommerce awareness campaigns should be launched targeting both consumers and businesses. in terms of promoting ebusiness reliability and security, a central certification authority should be set up with a public-key infrastructure (pki) encryption system. this would ensure confidentiality and provide access control, integrity, authentication, and non-repudiation services for businesses moving into ecommerce. a legislative framework would also have to be drawn for addressing the issue of online privacy and security. achieving these goals in addition to human skill and resources development would require continuous monitoring of progress and inherent flexibility for quick decisions and choices. hence, a steering committee could be set up drawing individuals both from the government and private business sector to oversee implementation and progress. acknowledgement the authors would like to thank king abdul-aziz city for science and technology under project grant ar-19-16 and king fahd university of petroleum & minerals for their support. also the assistance rendered by salman khan, mohammad wasiq, syed sanaullah and mohammad faheemuddin throughout the duration of the project especially in the collection, analysis, and documentation of results is highly appreciated. references ajzen, i. (1991), “the theory of planned behavior”, organizational behavior and human decision processes, vol 50, pp.179-211 benham h. c. & raymond b. c. (1996), “information technology adoption: evidence from a voice mail introduction”, acm sigcpr computer personnel volume 17, issue 1, january, pp. 3 – 25. bhimani a. (1996) “securing the commercial internet,” communications of the acm, vol. 39, issue 6, pp. 29-35. clark, ted (1999), “electronic commerce in china”, chapter in doing business on the internet, sudweeks and romm (eds.), springer, london. clark, ted (1999), “electronic commerce in china”, chapter in doing business on the internet, sudweeks and romm (eds.), springer, london. cronin, mary j. (2000), “unchained value: the new logic of digital business.” boston, ma. harvard business press.davis, 1989: davis f. d. (1989) “perceived usefulness, perceived ease of use, and user acceptance of information technology.” mis quarterly, vol 13, pp. 319-340. davis f. d., bagozzi r. p., & warshaw p. r. (1989) “user acceptance of computer technology”, management science, vol 35, pp. 982-1003. dekleva, s. (2000), “electronic commerce: a half-empty glass,” communications of ais, june, vol. 3, article 18. pp 74-80. devaraj s., fan m., & kohli r. (2002). “antecedents of b2c channel satisfaction and preference: validation e-commerce metrics”, information systems research, vol 13, no 3, pp. 316-333. fishbein, m., & ajzen, i. (1975) belief, attitude, intention and behavior: an introduction to theory and research, reading, ma: addison-wesley. gefen d. & straub d. w. (2000). “the relative importance of perceived ease-of-use in is adoption: a study of e-commerce adoption”, journal of the association for information systems. vol, 1no 8, pp.1-27. elham ghashghai, and rosalind lewis (2002), “issues affecting internet use in afghanistan and developing countries in the middle east,” rand corporation, ip-231-cmepp, http://www.rand.org/publications/ip/ip231/index.html hartwick j. & barki h. (1994), “explaining the role of user participation in information system use,” management science, vol 40, no 4, pp. 440-465. internet software consortium (2001), “internet domain survey”, http://www.isc.org/ds/. ajis vol. 12, no. 1 september 2004 74 isu (1998), internet services unit, king abdul-aziz city for science and technology, www.isu.net.sa jiang k., hsu m.k., klein g. & lin b. (2000). “e-commerce user behavior model: an empirical study”. human systems management, vol 19, no 4, pp. 265-277. harsha kumar, fingar, peter, & tarun sharma (2000), “enterprise e-commerce: the software breakthrough for business-to-business commerce.” tampa, fl: meghan-kiffer press. lederer a., maupin d., sena m. & zhaung y. (2000). “the technology acceptance model and the world wide web”. decision support systems, vol 29, no 3, pp. 269-282. lee, o.,(1999) ”an action research report: a south korean e-commerce firm’s case”, doing business on the internet: its pitfalls and opportunities, sudweeks & romm(eds), springerverlag, uk, mintzberg, h., raisinghani d., & theoret a. (1976) “the structure of unstructured decision processes”, administrative science quarterly, vol 21, no 2, 1976, pp. 246-275. moon, 2001: moon, j. & young-gul, k. (2001). “extending the tam for a world-wide-web context”, information and management, vol 38, pp. 217-230. pare, d. j. (2002), “b2b e-commerce services and developing countries: disentangling myth from reality,” international conference on internet research 3.0 (aoir), maastricht, the netherlands, october 13-16. pavlou, p. a. (2001) “consumer intentions to adopt electronic commerce – incorporating trust and risk in the technology acceptance model”, digit workshop, new orleans, louisiana, december 16. pavlou, p. a. (2002), “what drives electronic commerce? a theory of planned behavior perspective,” best paper, proceedings of the academy of management conference, denver, colorado, august 9-14. petrazzini, b., and m. kibati (1999), “the internet in developing countries,” communications of the acm, vol. 42, no. 6, pp 46-57 plant, robert (2000), ecommerce: formulation of strategy. upper saddle river, nj: prenticehall rajput, wasim e. (2000), e-commerce systems: architecture and applications. boston: artech house. rogers, e. (1995) diffusion of innovations (4th ed.), new york: the free press. scannell, t.v. (1999) behavioural models of advanced manufacturing technology selection, thesis, michigan state university, east lansing, mi. sia, c., teo, h., tan b.c. & wei k., (1998), “examining environmental influences on organizational perceptions and predisposition toward distributed work arrangements: a path model”, proceedings of the international conference on information systems (icis), helsinki, finland, pp. 88-102. simon, h. (1977) the new science of managerial decision making, englewood cliffs, nj: prentice hall. tan, m., & teo, t. s., (2000), “factors influencing the adoption of internet banking”, journal of the association for information systems, vol 1, article 5, pp. 1-42. tawil, k. m. (2001), “the internet in saudi arabia”, telecommunications policy, elsevier publications, vol 25, issue 8-9, pp. 625-632. taylor s. & todd p. a. (1995), “understanding information technology usage: a test of competing models”, information systems research, june, pp. 144-176. tornatzky l. g., & klein k. j. (1982), “innovation characteristics and innovation adoptionimplementation: a meta-analysis of findings”, ieee transactions on engineering management, vol 29, no 1, february, pp. 28-45. travica, b (2002), “diffusion of electronic commerce in developing countries: the case of costa rica,” journal of global information technology management, vol 5, no 1, pp: 4-24. young, j and ridley, g (2003): “e-commerce in developing countries,” ejisdc, 11, 1, pp: 1-6. travica, bob, & blaise cronin (1996), “the business web in russia: usability for the western user,” proceedings of the amcis 1996 conference, october 21-24, baltimore, md. travica, bob, & rebecca olson (1998), “usability of business web sites in east and central europe,” proceedings of the amcis 1998 conference, october 24-29, pittsburgh, pa. microsoft word ajis vol13-2 o.doc 119 australasian journal of information systems volume 13 number 2 may 2006 the f@ framework for designing awareness mechanisms in instant messaging 12 minh hong tran, yun yang fict, swinburne university of technology po box 218 hawthorn 3122 australia {mtran, yyang}@ict.swin.edu.au gitesh k. raikundalia itarl, victoria university po box 14428 melbourne city 8001 australia gitesh.raikundalia@vu.edu.au abstract this paper presents our research on awareness support in instant messaging (im). the paper starts with a brief overview of an empirical study of im, using an online survey and face-to-face interviews to identify user needs for awareness support. the study identified a need for supporting four aspects of awareness—awareness of multiple concurrent conversations, conversational awareness, presence awareness of a group conversation, and visibility of moment-to-moment listeners and viewers. based on the empirical study and existing research on awareness, we have developed the f@ (read as “fat”) framework of awareness. f@ comprises of an abstract level and a concrete level. the former level includes an in-depth description of various awareness aspects in im, whilst the latter level utilises temporal logic to formalise fundamental time-related awareness aspects. f@ helps developers gain a better understanding of awareness and thereby design usable mechanisms to support awareness. applying f@, we have designed several mechanisms to support various aspect of awareness in im. introduction instant messaging (im) is an application that supports nearly synchronous communication between people over networks. in recent years, im has increasingly become a popular communication tool, used by millions of worldwide users at home and at work. im was once mostly used by teenagers for chitchat over the internet, but the service has quickly moved to the mainstream as many home users and business users find im as an easy, fast and convenient way of communication with family members, friends and colleagues (isaacs et al. 2002b, nardi et al. 2000). the population of im home 12 this paper is an extension of papers that have been published in ozchi’2005, canberra, australia, november 2005; and in apweb’2006, harbin, china, january 2006. australasian journal of information systems volume 13 number 2 may 2006 120 users increased 28% from 42 million in september 2000 to 53.8 million in september 2001 (perera 2001), and enterprise im is growing at approximately 20% annually (shukla 2003). as reported in february 2003, msn messenger alone has more than 75 million registered users (yudkowsky 2003). research showed that 77% of ibm employees responded that im had enhanced their communication by reducing the time they often spent on e-mail, telephone and face-to-face communications. also, more than 75% of ibm’s clients, who have im, acknowledged that im enhanced their productivity (shukla 2003). whilst email is more like exchanges of digital letters, im closely resembles face-to-face spoken conversations in which exchanges are often short, quick and even incomplete sentences (dix et al. 2004, smith et al. 2000). due to the conversational style of im, there is a strong need for maintaining awareness between conversants. for example, awareness of the availability of other people assists users in deciding if they should move into conversations; and awareness of other people’s activities helps to coordinate im conversations naturally and effectively. research on awareness in im has gained much attention within the cscw and hci communities. many studies, such as (cech & condon 2004, isaacs et al. 2002a, segerstad & ljungstrand 2002, tang et al. 2001, viegas & donath 1999) have researched various techniques to support different aspects of awareness including users’ presence awareness, awareness visualisation, turn-taking convention, and so on. although different awareness features have been included in im systems, there are still many aspects of awareness need to be supported, as discussed in the next section. this paper summarises our empirical study of awareness in current im. the study helps us gain realworld users’ experience with current awareness support. the study has identified four issues of awareness support that are currently lacking in im—awareness of multiple conversation awareness, conversational awareness, presence awareness in a group chat, and in-progress viewing and listing awareness. based on the study and existing research on awareness, we have developed the f@ (read as “fat”) framework of awareness. f@ includes an abstract level and a concrete level that examine the concept of awareness in im and aim to extend designers' understanding of awareness for purposes of designing useful supporting mechanisms. whilst the abstract level presents essential elements of knowledge that contribute to maintaining awareness in im, the concrete level adopts temporal logic to present fundamental time-related aspects of awareness in formal way. applying f@, we have designed several mechanisms to enhance awareness support in im. the remainder of the paper is organised as follows. the next section reviews related research on awareness in im, followed by the summarised results of our empirical study. then, the paper presents the f@ framework and describes briefly our innovative awareness mechanisms by applying f@. finally, we conclude the paper by recapitulating the findings of our work and presenting a view of future research. importance of awareness in instant messaging this section reviews different techniques developed by previous research to maintain awareness in im. presence awareness providing presence awareness information about the availability of other users is one of the primary and most important features of im. presence awareness helps users decide if and when to move into conversations (nardi et al. 2000). at a rudimentary level, presence awareness informs users if their 121 australasian journal of information systems volume 13 number 2 may 2006 buddies are online or offline. this feature has been implemented in all popular im clients such as messengers of aol, icq, msn, yahoo and jabber (e.g., yahoo messenger shows yellow icons to indicate online users and grey for offline users). at a higher level, im systems incorporate many other features such as sound alerts and live video to inform when buddies come online and go offline. hubbub (isaacs et al. 2002a) uses auditory cues to support presence awareness—whenever users’ buddies go online, their ‘sound ids’ are played at the user’s site. furthermore, im users can even set presence messages (e.g., ‘on the phone’ and ‘stepped out’) so that other users trying to contact them know that they are on the phone or not in the office. some im systems not only show if users are available, but also provide the level of their activities. for example, activity meter (isaacs et al. 2002a) shows users’ level of activities within the last 15 seconds. chat circles (viegas & donath 1999) changes colours of users’ circles to indicate how active they are. when im becomes part of an integrated communication platform, more sophisticated support for presence awareness is required. for example, as the mobility factor is added to im, a new degree of presence awareness is introduced. hubbub shows if users are online, and also indicates whether they logged into im networks from their pcs or their pdas. webwho (ljungstrand & segerstad 2000) provides awareness of both virtual and physical presence. webwho displays a notification if users are present in an im network (i.e., virtual presence), and their geographical location (i.e., physical presence). a study of webwho shows that both virtual and physical presence awareness affect content of im messages. imvis (neustaedter et al. 2002) and chat circles explore alternative metaphors to represent presence awareness. imvis develops a 3d tunnel to show available buddies around the outside edge of the tunnel, and less available buddies closer to the vanishing point of the tunnel. chat circles represents users as coloured circles. the circles expand as a new message arrives, and become blurry after a period of idleness. turn-taking awareness turn-taking has been well-known as a fundamental process in human conversations (dix et al. 2004). in face-to-face communication, turn-taking is supported by a suite of fine-grained back channels such as body language, eye-contact, voice intonation, facial expression, and so on. but those fine-grained back channels are almost missing in im. moreover, im inherits many generic problems of text-based communication tools with respect to turn-taking such as visibility of listening-in-progress, control over turn positioning and visibility of turns-in-progress as discussed comprehensively in (herring 1999, smith et al. 2000). several solutions for maintaining turn-taking in im have been developed. yet, effective support for organising turn-taking rules and resolving floor control conflicts is still very limited (cech & condon 2004, voida et al. 2002, woodruff & aoki 2003). the simplest solution for turn-taking is that a conversant explicitly offers the floor to other conversants by asking direct questions such as “what do you think, bob?”. however, this solution is limited as it does not suit the conversational style of im in which exchanged messages are short and instant (dix et al. 2004). other alternative solutions for turn-taking have been studied. for example, im systems provide awareness cues such as the textual ‘who is typing’ indicator in msn messenger and yahoo messenger, the visual ‘focusing’ and ‘not-focusing’ cues used in hubbub, and the auditory typing cues used in babble. threaded chat (smith et al. 2000) adopts the threaded conversational style, which has been widely implemented in discussion boards to support turns and replies in im conversations. australasian journal of information systems volume 13 number 2 may 2006 122 when voice chat is added to im, supporting turn-taking in auditory conversations is even more challenging compared to text-based conversations. im systems have introduced a visual indicator that signals when a person is talking in im conversations. for example, woodruff and aoki (2003) examine an effect of the push-to-talk mechanism on turn-taking convention. contextual awareness contextual awareness refers to information that provides users with context of im conversations. commonly, im applications support contextual awareness by displaying a quasi-shared window—a window containing messages sent by all participants in a conversation. we define the term, ‘quasishared’, to refer to that window because currently local messages appear on that window instantly, but remote messages are displayed in the order of their arrivals at a central server. consequently, the order of messages shown on conversants’ screens can be different from one to another. despite being un-identical, the quasi-shared window still provides im users with some degree of a common understanding of the flow of messages. providing information about other conversants’ activities, such as if they are typing, talking, focusing or not focusing on a chat window (discussed in the previous section), also helps to maintain contextual awareness in im. chat circles uses the cadence of size of coloured circles on a user’s screen to show the flow of conversations. babble uses a graphical representation called ‘social proxies’ to show the activity that people carry out with the application. this also helps to provide users with an intuitive sense of context in conversations. some other im clients such as gaim (http://gaim.sourceforge.net) and trillian (http://ceruleanstudios.com) even notify users when their buddies close chat windows, and display a timeout flag if a conversation is inactive for too long. one aspect of maintaining contextual awareness in im is to support identity awareness. awareness of people’s identities is crucial for managing oneself in a public space. in the case of im, although each user is not publicly seen by everyone in the network, they are a part of a big group in which each individual presents their own sense of identity. providing identity awareness enhances context of conversations as it shows people with whom they are communicating. commonly, im systems support identity awareness by associating im users with different nicknames, colours, avatars, coloured circles (viegas & donath 1999) and even ‘sound ids’ (isaacs et al. 2002a). these attributes are indicators of identity in the im virtual community. our study reported in (tran et al. 2004) shows that im users even reach out for multiple identities that allow them to project themselves differently to different users. emotional awareness emotions are a social need and play an important role in human communication. both our own affective state and our perception of that of others influence the process and outcome of our conversations (damasio 1994). there has been a growing interest in providing expressive representation of emotions in im (garcia et al. 1999). at the most basic (but very popular and effective) level, im users convey their emotional state like happiness, anger or sadness by using punctuations and acronyms e.g., :-) stands for a smiling face, and ;-) is for a winking face (dix et al. 2004). advancing from that, im systems have integrated those punctuations with animated graphical emoticons (rivera et al. 1996). recently, yahoo messenger has developed animated utterance called ‘audibles’. audibles include animated images and auditory track, used to deliver messages (e.g., hello, goodbye and flirt) and also to reflect the affective state of a sender and the illocutionary force of the messages. 123 australasian journal of information systems volume 13 number 2 may 2006 in addition to emoticons, the avatar is another graphical representation that is also commonly used by users to portray their emotional state (e.g., “i use a funny avatar when i am happy”). furthermore, im users express their emotions through their online status 13 . as reported “i often use status to tell my friends if i am sad or happy and also edit my status to tell them what i am doing like studying, cooking, and stuff” (tran et al. 2004). conductive chat (dimicco et al. 2002) explores a new metaphor to convey im users’ emotion by incorporating users’ skin conductivity levels into im. the study and results this section reviews our empirical study of im that was used to gain users’ feedback on their realworld experience with awareness support in im, and summarises the results of the study. the study was composed of an online survey and face-to-face interviews. the reader is referred to (tran et al. 2004, 2005) for the detailed description of the empirical study. in brief, the online survey consisted of demographic multiple choice questions, 7-point likert scale questions and open-ended questions. 173 participants (56 females and 117 males) took part in the survey. the participants were students from several universities in australia. most of them were in their early twenties and had used im for more than 3 months. after the survey was completed, we conducted further informal face-to-face interviews with 6 participants (2 females and 4 males), who were selected from the 173 participants of the online survey. open-ended and follow-up questions were used in the interviews. the follow-up questions were used to probe participants so that they talked more about their use of im. listening to participants’ stories of how they had used im helped to gain valuable insights we would not have gained otherwise. the study overall yielded much data, and some results of the study have been published in (tran et al. 2004, 2005, 2006a). this section briefly summarises four findings related to awareness support in im. awareness of multiple concurrent conversations the online survey showed that it is common behaviour for im users to interact with many people simultaneously: 92% of the respondents had used im to chat with two people or more at the same time. five out of the six interviewed participants responded that at one time or another they had typed into a window that was not the one intended, especially when they had multiple conversations with many buddies at the same time. such a mistake may happen because support for managing multiple conversations in current im is weak. conversational awareness more than 60% of the survey participants responded that they often want to show other people which earlier messages of the same conversation that they want to refer to. but current im applications provide very limited support for this, thus users have to copy the messages to which they want to refer and paste them to a new message. in addition, more than half of the respondents said that they want to link emoticons with exact messages posted by other people. 13 an online status is a text-based description composed by a user and can be seen by the user’s buddies. australasian journal of information systems volume 13 number 2 may 2006 124 presence awareness in a group chat there is a lack of awareness support in a conversation that involves a group of more than two people. many awareness cues, which are provided in a one-to-one conversation, are either missing or become significantly less effective in a group conversation. for example, a visual ‘who is typing’ cue is missing in a group chat especially when more than one person is typing at the same time. this leads to many problems in maintaining turn-taking and resolving floor control conflicts in a group chat. furthermore, im applications do not provide any presence awareness of those who were in the group chat and had already left, or those who are going to join the group chat. visibility of moment-to-moment listeners and viewers the study showed that im systems should provide some level of visualisation to support awareness about the visibility of listeningand viewing-in-progress. current im does not provide moment-bymoment information about people who are listening to auditory messages and/or viewing webcam. in addition, our respondents suggested that im systems should provide information about if other users are able to participate in an audio or video chat. framework of awareness in instant messaging based on the results of several empirical studies of synchronous groupware, including the empirical study of im presented in previous section, we have developed the f@ framework of awareness. f@ was developed with a twofold objective: provide a better understanding of awareness and facilitate the design of awareness mechanisms. f@ addresses many aspects of awareness in synchronous distributed collaboration, as presented in (tran et al. 2006b). this paper merely focuses on aspects that are relevant to awareness in im. f@ consists of two parts: an abstract level and a concrete level. the abstract level presents in-depth descriptions of awareness. the concrete level exploits temporal logic to formalise some fundamental time-related aspects of awareness. it is not a goal of the concrete level to formalise all elements of group awareness which are addressed at the abstract level. instead, the formulas presented at the concrete level aim to demonstrate the feasibility of formalising the concept of awareness as an approach of defining precise requirements of designing supporting mechanisms. abstract level of f@ awareness in im involves users’ knowledge of a conversation. this type of knowledge includes answers to questions like, “who is talking?”, “who is listening?”, “can they hear me?”, “do they pay attention?” and “who is talking next?”. by answering such mechanical questions, awareness helps users maintain a sense of awareness of what is happening in a conversation (gutwin & greenberg 2002). when a group is no longer co-located in the same room, a rich set of verbal and non-verbal cues (e.g., intonation, eye contact, facial expressions, etc.) which is often naturally available in a face-toface conversation, becomes difficult to find over distance. to address this issue, there has been a large body of research committed to investigating the benefits of media spaces technologies for facilitating distributed conversations. for example, significant effort has been devoted to studying how synthetic audio and video links can help to support awareness (bly et al. 1993, dourish & bly 1992, finn et al. 1997, tang & minneman 1991). although computer-integrated audio-video media 125 australasian journal of information systems volume 13 number 2 may 2006 are useful in providing rich context of a conversation, these technologies are faced with many problems such as turn-taking control, eye contact, gesture, privacy, and so on (fussell et al. 2004, gaver 1992, kraut et al. 2003). in addition, a physical setting largely reliant on video is often not flexible enough to deliver appropriate images for the context of a conversation (hudson & smith 1996). in many cases, text-based links are preferred over rich medias like audio and video for several reasons, such as their low-bandwidth requirement and relaxed styles (erickson et al. 1999). textbased communication tools are able to support conversational awareness in their own way such as allowing copying-and-pasting contents of previous messages, colouring and highlighting emphasised words, providing textual cues of who is typing, and so on. commonly, text-based communication tools such as chat and im often organise messages in chronological sequence. this sequential method of presenting conversation has been found inefficient in supporting conversational awareness in group discussion for several reasons, such as lack of mapping between people and their messages, no listening-in-progress, poor turn-taking support, etc. (smith et al. 2000, vronay et al. 1999). in order to understand and support conversational awareness, it is useful to determine a set of components that are involved in the conduct of a conversation. according to coordination theory (malone & crowston 1994), awareness needs to be supported in the communication process, which involves senders, receivers, messages and languages. in this research, it is assumed that senders and receivers use the same language. thus awareness depends on the relationships between the three components of senders, receivers and messages 14 . our empirical study of im shows a need for improving four major aspects of awareness, including multiple concurrent conversations, conversational context, presence in a group chat and listening and viewing-in-progress. these four aspects of awareness can be interpreted from the perspective of the three group components—senders, receivers and messages. for example, awareness of presence involves a sender’s knowledge of the availability of a receiver, and vice-versa. awareness of listeningand viewing-in-progress involves a receiver’s perception of a sender who sent a message as well as a sender’s perception of a receiver who would receive a message. in addition to the three group components, it is valuable to consider a specific set of information that needs to be provided to support awareness in im. adopting the approach used in gutwin and greenberg’s framework of workspace awareness (2002), we present elements of knowledge that relate to awareness as a set of the 5w1h (i.e., “who”, “what”, “where”, “when”, “why”, and “how”) dimensions, as shown in table 1. these six dimensions articulate and categorise information that should be provided to help people stay aware of a conversation. 14 a message can be conveyed in various forms including, but not limited to, text, graphic, audio and video. australasian journal of information systems volume 13 number 2 may 2006 126 dimensions awareness elements specific questions who presence (current) presence (past) presence (future) identity (current) identity (past) who participates in a conversation? who was in a conversation? who is going to join a conversation? who is this person? who is talking? who is listening? who was this person? what context (current) context (future) what does this message respond to? what are responses to this message? what is a question of this response? what does a person refer to? who is talking next? where location (current) where is the latest message? when event (past) event (future) when did a person join a conversation? when was this message sent? when will a person leave a conversation? why motivation (current) motivation (past) behaviour (current) why is a person saying that? why was this message sent? why does a person stop talking? how reaction (current) reaction (past) reaction (future) how do other people react? how have others changed their behaviour? how will others react after i send this message? table 1: dimensions of awareness in im the specific elements and questions listed in table 1 are fundamental and common sense, and cover typical conversational interaction and behaviour. therefore, they are not an exhaustive list of possible questions. besides, in different situations, the specific information required to make up a person’s awareness varies depending on the nature of a conversation and the surrounding environment. concrete level of f@ drawing on the importance of the “when” dimension, the concrete level aims to model some essential time-related properties of awareness, using temporal logic as a vehicle. temporal logic (tl) is an extended version of first-order logic (davis 1990) by adding temporal aspects to it. let ℑ be a linear-tree of time and s be a node in ℑ. let ϕ be a proposition which can hold at some nodes in 127 australasian journal of information systems volume 13 number 2 may 2006 ℑ. a notion of proposition ϕ being satisfied (i.e., |=) at node s in ℑ is defined as: m (s, ℑ) |= ϕ. as mentioned earlier, it is not the intention of the concrete level to formalise every aspect of awareness that is addressed at the abstract level. the concrete level models two aspects presence awareness and awareness of turn-taking. this is to illustrate the viability of formalising awareness, whilst the formalisation of other aspects is left as future work. the abstract level shows that awareness of a conversation involves three components of senders, receivers, and messages. therefore, a conversation can be considered as a set: conversation = {pp, mg}, where pp denotes people and mg denotes messages. a person is said to be part of a conversation iff (if and only if) the person is in conversation at node s. this relation can be expressed as follows: m (s, ℑ) |= part_of(p, conversation) ⇔ m (s, ℑ) |= (∃x∈ pp) (p = x) the predicate part_of(p, conversation) is useful to indicate whether or not a person is in a conversation, but is unable to specify the temporal points at which a person joins or leaves a conversation. thus, two additional relations between a person and a conversation are defined: join(p, conversation) and leave(p, conversation). m (si, ℑ) |= join(p, conversation) ⇔ m (si-1, ℑ) |= ¬ part_of(p, conversation) ∧ m (si, ℑ) |= part_of(p, conversation) m (si, ℑ) |= leave(p, conversation) ⇔ m (si-1, ℑ) |= part_of(p, conversation) ∧ m (si, ℑ) |= ¬ part_of(p, conversation) presence awareness of past/ current/ future conversants in addition to showing “current conversants” who are currently participating in a conversation, it is useful to provide information about the presence of “past conversants” who were in the conversation and had already left, and “future conversants” who might join the conversation. im systems often use a history of events to track the presence of past conversants and current conversants. regarding future conversants, systems can use an invite-and-accept protocol to identify who are going to join a conversation. (a) past conversants: past conversants are those who were once in a conversation. operator once is denoted as ‘♦’. person p is considered once in conversation, iff at state sj in the past, p joined a conversation, and at another state sk after sj also in the past p left the conversation. m (si, ℑ) |= ♦ part_of(p, conversation) ⇔ m (sj, ℑ) |= join(p, conversation) ∧ m (sk, ℑ) |= leave(p, conversation), where j < k < i (b) current conversants : current conversants are those who joined conversation at a state in the past and have not left the conversation. m (si, ℑ) |= part_of(p, conversation) ⇔ m (sj, ℑ) |= join(p, conversation) ∧ m (sk, ℑ) |= (∀k: j≤ k ≤ i) ¬ leave(p, conversation) australasian journal of information systems volume 13 number 2 may 2006 128 (c) future conversants : future conversants are those who will eventually join a conversation and are not currently in the conversation. operator eventually is denoted as ‘◊’. person p is considered eventually in conversation, iff p is not currently in the conversation now (si), and will join the conversation later (sj: j > i). m (si, ℑ) |= ◊ part_of(p, conversation) ⇔ m (si, ℑ) |= ¬ part_of(p, conversation) ∧ m (sj, ℑ) |= join(p, conversation), where j > i based on the formulas of past/ current/ future conversants above, the phenomenon by which a user is aware of the presence of conversants is defined as: m (s, ℑ) |= aware(user, presence-conversants) ⇐ (m (s, ℑ) |= (∀pi: p ≠ user) ♦part_of(pi, conversation) ∧ know(user, pi)) ∧ (m (s, ℑ) |= (∀pj: p ≠ user) part_of(pj, conversation) ∧ know(user, pj)) ∧ (m (s, ℑ) |= (∀pk: p ≠ user) ◊ part_of(pk, conversation) ∧ know(user, pk)), where pi, pj, pk, user ∈ conversation. awareness of turn-taking awareness of turn-taking involves people’s fundamental knowledge of who is sending a message and who is receiving a message. at node sn during a conversation, an action of sending a message from a sender to a receiver can be modelled as: m (sn, ℑ) |= send(sender, receiver, message), similarly, at node sm during a conversation, an action of confirming that a receiver receives a message can be modelled as: m (sm, ℑ) |= receive(receiver, message) to support a person’s awareness, the following conditions need to hold when a sender sends a message to a receiver. first, a sender needs to be aware of the fact that the sender sends a message to a receiver: m (s, ℑ) |= aware(sender, send(sender, receiver, message) ⇔ m (s, ℑ) |= send(sender, receiver, message) second, a sender needs to know that the receiver actually receives a message. in the following formula, it is assumed that a transmission is real-time: m (sj, ℑ) |= aware(sender, receive(receiver, message)) ⇔ m (si, ℑ) |= send(sender, receiver, message) ∧ m (sj, ℑ) |= receive(receiver, message), where j > i 129 australasian journal of information systems volume 13 number 2 may 2006 third, a receiver needs to know that a sender sent the receiver a message: m (sj, ℑ) |= aware(receiver, send(sender, receiver, message)) ⇔ m (si, ℑ) |= send(sender, receiver, message) ∧ m (sj, ℑ) |= receive(receiver, message), where j > i designing awareness support for instant messaging this section summarises our four mechanisms 15 (tran et al. 2005) that have been designed to enhance awareness support in im. the design of these mechanisms is evolved in part from our empirical study of im, in part from the knowledge presented in f@, and in part from the design of existing awareness mechanisms. 1. conversation dock (condoc): condoc utilises a focus+context visualisation technique (greenberg et al. 1996) to help users manage multiple concurrent conversations. condoc shows all active conversations in a miniature window and magnifies a particular conversation as a user moves a mouse over the conversation. 2. relaxed instant messenger (rim): rim combines the threaded styles and linear styles to organise im messages, and thereby accommodates more flexible turn-taking, and provides richer contextual awareness. displaying messages in threads allows a structured and coherent conversation, whilst a linear format is useful in providing a point of focus in a conversation and keeping users updated with the latest messages. 3. group list: group list supports presence awareness of people who are no longer in a group chat and who are going to join the group chat. group list also conveys awareness of multiple users’ activities (e.g., who are typing and who are talking) by providing visual ‘someone is typing’, ‘someone is talking’ indicators. for example, whenever a user is typing, an animated keyboard icon appears next to the user’s name on the contact list. 4. track view: track view informs users of who is currently listening to their auditory conversation and who is currently viewing their webcam. track view also allows the local user to stop any current viewer from listening to their voice or seeing their video. out of these four mechanisms, we have implemented and evaluated condoc and rim. the evaluations show positive results and feedback on the design (tran et al. 2006a). reflections on the use of f@ up to this point, the paper has presented user needs and the design of four awareness mechanisms. this section steps back from the detailed descriptions, and presents higher-level reflection that can be drawn from the design and development of these mechanisms. in particular, the reflection focuses on a comparative analysis of relationship between the principles of f@ and the designs of mechanisms. reflections on the abstract level table 2 presents a list of awareness elements that are addressed in f@ and supported by our new awareness mechanisms. as seen in table 2, many, though not all, awareness elements raised by f@ are supported by our new awareness mechanisms. the evaluations of condoc and rim confirm that awareness elements provided by the mechanisms help to enhance awareness. this confirmation 15 due to space limits, screenshots of the four mechanisms are not included in this paper. the reader is referred to tran et al. (2005) for more details. australasian journal of information systems volume 13 number 2 may 2006 130 indicates that awareness elements presented at the abstract level are useful and can be applied to designing other mechanisms to support awareness of a conversation. dimensions, awareness elements, design of rim and condoc proposed mechanisms: the “what” dimension context (current) rim uses a tree to create a structural layout of messages rim allows posting questions or answers explicitly to a particular message rim uses the same colour for messages of the same topic track view supports listeningand viewing-in-progress condoc displays multiple conversations in a miniature view, and magnifies a particular conversation when the mouse moves over the conversation the “where” dimension location (current) rim shows the latest message at the bottom of the linear window, while the message can be allocated at any node in the threaded window condoc indicates when new messages arrived at a particular conversation the “when” dimension event (past/ current) rim shows elapsed time since users joined or left a conversation rim indicates if anyone is typing a message rim indicates if anyone joins or leaves a conversation the “why” dimension motivation (past/ current) users send messages to ask questions explicitly rim supports a logical and structural layout of message that allows users to indicate the purpose of sending a particular message (e.g., to answer or to question, etc.) behaviour (past/ current) users send messages to ask questions explicitly the “how” dimension reaction (past/ current/ future) expressed via emoticons users send messages to ask questions explicitly table 2: f@ and mechanisms for conversational awareness 131 australasian journal of information systems volume 13 number 2 may 2006 reflections on the concrete level the concrete level primarily formalises two aspects of awareness—presence awareness of conversants and awareness of turn-taking. this section considers how those temporal logic formulas are interpreted and applied to the design of the mechanisms introduced in this paper. presence awareness as presented at the concrete level, the presence aspect of awareness is formulated as follows: m (s, ℑ) |= aware(user, presence-conversants) ⇐ (m (s, ℑ) |= (∀pi: p ≠ user) ♦part_of(pi, conversation) ∧ know(user, pi)) ∧ (m (s, ℑ) |= (∀pj: p ≠ user) part_of(pj, conversation) ∧ know(user, pj)) ∧ (m (s, ℑ) |= (∀pk: p ≠ user) ◊ part_of(pk, conversation) ∧ know(user, pk)), where pi, pj, pk, user ∈ conversation. buddy list enhances support for presence awareness of conversants by showing both past and current conversants. awareness of turn-taking the concrete level presents three conditions that need to hold when a sender sends a message to a receiver in a conversation. m (s, ℑ) |= aware(sender, send(sender, receiver, message) ⇔ m (s, ℑ) |= send(sender, receiver, message) (ca1) m (sj, ℑ) |= aware(sender, receive(receiver, message)) ⇔ m (si, ℑ) |= send(sender, receiver, message) ∧ m (sj, ℑ) |= receive(receiver, message), where j > i (ca2) m (sj, ℑ) |= aware(receiver, send(sender, receiver, message)) ⇔ m (si, ℑ) |= send(sender, receiver, message) ∧ m (sj, ℑ) |= receive(receiver, message), where j > i (ca3) in face-to-face conversations, people are naturally aware of to whom they are talking and whether the listeners can hear them through verbal and non-verbal cues. however, these three conditions often do not hold in computer-mediated communication tools such as im applications. in general, im meets (ca1) and (ca3) by providing visual cues such as “who is typing” in the case of text-based conversation, and a coloured bar that raises in the case of an audio conversation to inform a sender and a receiver that the sender is sending a message. but, that is often only supported for the case of one-to-one conversations. when a conversation involves a group of more than two people, these two conditions often do not hold. for example, when two people are typing at the same time, im does not indicate who they are; or when two people are talking at the same time, there is no visual indicator showing who is talking. group list was designed to support ca1 and ca3 by showing people who are concurrently typing in a conversation. current im applications fail to meet (ca2). that is, a sender does not know whether a receiver actually receives the message. hence, in many cases, a sender needs to ask the receiver explicitly for confirmation (i.e., if the receiver actually receives the message). to address this issue, im applications can distinguish between the cases when a message is delivered to a receiver successfully australasian journal of information systems volume 13 number 2 may 2006 132 and when a message does not reach an intended receiver. im applications can provide awareness mechanisms such as a message pool that keeps all failed-to-deliver messages, and allows a sender to choose if the sender wants to re-send or simply ignore those messages. the issue of providing support for (ca2) is even more problematic in the case of audio and video communication. im users are provided with no awareness cues informing them if receivers attend to their broadcasted audio and video contents. im applications can include awareness mechanisms such as track view that are used to inform a local user of who else is currently listening to the user’s auditory track and who else is currently viewing the user’s webcam. conclusions and future work this paper presents our f@ (read as “fat”) framework for designing mechanisms to support awareness in im. the f@ framework is developed based on our empirical study of awareness support in im, and on previous related research on awareness. f@ includes an abstract level and a concrete level which examine different aspects of the concept of awareness. the abstract level presents an in-depth description of various aspects of awareness that im users need to be supported, whilst the concrete level presents the formalisation of awareness, particularly focusing on timerelated aspects. f@ is used to help designers' better understand the concept of awareness and thereby design useful awareness mechanisms. we have applied f@ to designing four mechanisms that can be used to enhance awareness support in im: conversation dock supports awareness of multiple concurrent conversations, relaxed instant messenger assists turn-taking convention and provides richer conversational awareness, group list enhances presence awareness in a group chat, and track view supports awareness of in-progress listeners and viewers. as future work, we will enhance the f@ framework based on the findings from the evaluation of our awareness mechanisms, and focus on the development and evaluation of other mechanisms to improve awareness support in im. acknowledgement the research reported in this paper was supported financially by smart internet technology cooperative research centre (sitcrc), australia. we wish to thank shane grund and john gheybi for their contributions to developing some awareness mechanisms, as well as participants involved in our empirical user study and evaluation. references bly, s. a., harrison, s. r. & irwin, s. (1993) media spaces: bringing people together in a video, audio, and computing environment. communications of acm, 36(1), pp. 28-47. cech, c. g. & condon, s. l. (2004) temporal properties of turn-taking and turn-packaging in synchronous computer-mediated communication. proceedings of 37th hicss’04, big island, ieee press, pp. 107-116. damasio, a. r. (1994) descarte's error: emotion, reason, and the human brain, gosset putnam press, new york. 133 australasian journal of information systems volume 13 number 2 may 2006 davis, e. (1990) representations of commonsense knowledge, morgan kaufmann, san mateo, ca. dimicco, j. m., lakshmipathy, v. & fiore, a. t. (2002) conductive chat: instant messaging with a skin conductivity channel. poster presentation, cscw'02, la, acm press. dix, a. j., finlay, j., abowd, g. d. & beale, r. (2004) human-computer interaction, 3rd ed., pearson, prentice hall, harlow, england. dourish, p. & bly, s. a. (1992) portholes: supporting awareness in distributed work group. proceedings of chi'92, monterey, acm press, pp. 541-547. erickson, t., smith, d. n., kellogg, w. a., laff, m., richards, j. t. & bradner, e. (1999) socially translucent systems: social proxies, persistent conversation, and the design of “babble”. proceedings of chi '99, pennsylvania, acm press, pp. 72-79. finn, k. e., sellen, a. j. & wilbur, s. b. (eds.) (1997) video-mediated communication, lawrence erlbaum associates, mahwah, nj. fussell, s. r., setlock, l. d., yang, j., ou, j., mauer, e. & kramer, a. d. i. (2004) gestures over video streams to support remote collaboration on physical tasks. human-computer interaction, 19(3), pp. 273-309. garcia, o., favela, j. & machorro, r. (1999) emotional awareness in collaborative systems. proceedings of criwg'99, mexico, ieee press, pp. 296-303. gaver, w. w. (1992) the affordances of media spaces for collaboration. proceedings of cscw'92, toronto, acm press, pp. 17-24. greenberg, s., gutwin, c. & cockburn, a. (1996) using distortion-oriented displays to support workspace awareness. people and computer xi (proceedings of the hci'96), london, springer-verlag, pp. 229-314. gutwin, c. & greenberg, s. (2002) a descriptive framework of workspace awareness for realtime groupware. computer supported cooperative work, the journal of collaborative computing, 11(3-4), pp. 411-446. herring, s. c. (1999) interactional coherence in cmc. proceedings of the 32nd hicss’99, maui, ieee press, p. 2022. hudson, s. e. & smith, i. (1996) techniques for addressing fundamental privacy and disruption tradeoffs in awareness support systems. proceedings of cscw’96, boston, acm press, pp. 248-257. isaacs, e., walendowski, a. & ranganthan, d. (2002a) hubbub: a sound-enhanced mobile instant messenger that supports awareness and opportunistic interactions. proceedings of chi'02, minnesota, acm press, pp. 179-186. isaacs, e., walendowski, a., whittaker, s., schiano, d. j. & kamm, c. (2002b) the character, functions, and styles of instant messaging in the workplace. proceedings of cscw'02, louisiana, acm press, new york, pp. 11-20. kraut, r. e., fussell, s. r. & siegel, j. (2003) visual information as a conversational resource in collaborative physical tasks. human-computer interaction, 18(1-2), pp. 13-49. ljungstrand, p. & segerstad, y. h. (2000) awareness of presence, instant messaging and webwho. acm siggroup bulletin, 21(3), pp. 21-27. malone, t. w. & crowston, k. (1994) the interdisciplinary study of coordination. acm computing surveys, 26(1), pp. 87-119. nardi, b. a., whittaker, s. & bradner, e. (2000) interaction and outeraction: instant messaging in action. proceedings of cscw'00, philadelphia, acm press, pp. 79-88. neustaedter, c., greenberg, s., & carpendale, s. (2002) imvis: instant messenger visualization. video proceedings of cscw'02, new york, acm press. perera, r. (2001) study: instant messaging at work up 110 percent. http://www.cnn.com/ 2001/tech/internet/11/16/workplace.im.idg/index.html, accessed on 19 december 2005. rivera, k., cooke, n. j. & bauhs, j. a. (1996) the effects of emotional icons on remote communication. proceedings of chi'96, vancouver, acm press, pp. 99-100. australasian journal of information systems volume 13 number 2 may 2006 134 segerstad, y. h. a. & ljungstrand, p. (2002) instant messaging with webwho. international journal of human-computer studies, 56(1), pp. 147-171. shukla, u. (2003) the future of enterprise instant messaging. http://www. expresscomputeronline.com/20030505/tech1.shtml, accessed on 19 december 2005. smith, m., cadiz, j. j. & burkhalter, b. (2000) conversation trees and threaded chats. proceedings of cscw'00, philadelphia, acm press, pp. 97-105. tang, j. c. & minneman, s. l. (1991) videodraw: a video interface for collaborative drawing. acm transactions on information systems, 9(2), pp. 170-184. tang, j. c., yankelovich, n., begole, j. b., kleek, m. v., li, f. & bhalodia, j. (2001) connexus to awarenex: extending awareness to mobile users. proceedings of chi'01, seattle, acm press, pp. 221-228. tran, m. h., yang, y. & raikundalia, g. k. (2004) consumption of multiple concurrent identities: the need from the instant messaging virtual community. the australasian journal of information systems, special issue, pp. 4-20. tran, m. h., yang, y. & raikundalia, g. k. (2005) supporting awareness in instant messaging: an empirical study and mechanism design. proceedings of ozchi’05, canberra, cd isbn 159593-222-4. tran, m. h., yang, y. & raikundalia, g. k. (2006a) awareness support for synchronous interpersonal electronic messaging: user needs, mechanisms and evaluations. technical report, faculty of information and communication technologies, swinburne university of technology, australia. http://www.it.swin.edu.au/personal/mtran/pub/tr06-im.pdf. tran, m. h., yang, y. & raikundalia, g. k. (2006b) f@: a framework of group awareness in real-time distributed groupware. proceedings of apweb'06, harbin, lecture notes in computer science, vol. 3841, springer-verlag, pp. 461-473. viegas, f. b. & donath, j. s. (1999) chat circles. proceedings of chi’99, pennsylvania, acm press, pp. 9-16. voida, a., newstetter, w. c. & mynatt, e. d. (2002) when conventions collide: the tensions of instant messaging attributed. proceedings of chi'02, minnesota, acm press, pp. 187-194. vronay, d., smith, m. & drucker, s. (1999) alternative interfaces for chat. proceedings of uist'99, north carolina, acm press, pp. 19-26. woodruff, a. & aoki, p. m. (2003) how push-to-talk makes talk less pushy. proceedings of group'03, florida, acm press, pp. 170-179. yudkowsky, c. (2003) im in a corporate environment: is instant messaging a nuisance or an untapped tool? http://accounting.smartpros.com/x37078.xml, accessed on 19 december 2005. microsoft word ajis vol13-2 o.doc 49 australasian journal of information systems volume 13 number 2 may 2006 demonstrations of the activity theory framework for research in information systems dr kate crawford & dr helen hasan school of economics and information systems, university of wollongong wollongong 2522, australia email: hasan@uow.edu.au abstract the holistic and insightful nature of the (cultural-historical) activity theory has led to its use by several researchers as a suitable vehicle for understanding and analysis in many areas of is research and practice. this paper demonstrates the application of activity theory to the study of socio-technical systems which mediate complex, collective activities in the modern workplace and in everyday life. vignettes from five ongoing research projects are reported in order to illustrate not only the explanatory power of the activity theory research framework but also its use in determining appropriate methods used to manage the data collection and analysis processes as well as its interpretation., the paper demonstrates the variety of is topics where an activity theory based approach is able to add richness and insight. introduction the field of information systems (is) is concerned with both technical and human aspects of organisational and social systems. is covers a broad range of possible topics of study and the application of a wide variety of research methods. in some areas of is research there is a need for a holistic theoretical framework to provide a basis for research into complex socio-technical phenomena. although the (cultural-historical) activity theory was proposed long before the advent of computers, its holistic and insightful nature has the potential to provide a suitable vehicle for understanding and analysis in many areas of is research and practice. a growing band of researchers recognise that the theory provides a rich holistic understanding of how people do things together with the assistance of sophisticated tools in complex dynamic environments (hasan 1999). this paper explores the explanatory, analytical and interpretive power of activity theory for the study of socio-technical systems which mediate complex, collective activities in the modern workplace. one advantage of the theory is that it supports analysis of the dialectic interactions between people and the ways technologies shape and are shaped by human activity. the paper describes the framework and discusses the circumstances under which an activity theory framework could be employed for research in is, as well as in the related fields of human-computer interaction and knowledge management. some of the research techniques and methods of data collection and analysis that are appropriate within an activity theory framework are also presented. vignettes from five ongoing research projects are reported to illustrate the variety of is topics where an activity theory framework is being used. in each case, the methods used to conduct the research and manage the data collection and analysis processes are presented through the activity theory australasian journal of information systems volume 13 number 2 may 2006 50 framework. the paper concludes with a discussion of the relevance and suitability of activity theory as a framework for current complex problems in is. an overview of activity theory activity theory is a social-psychological theory that has its roots in the work of the russian psychologist vygotsky during the first half of the 20 th century. vygotsky’s important insight into the dynamics of consciousness was that it is essentially subjective and shaped by the history of each individual’s social and cultural experience (vygotsky 1978). in addition, vygotsky saw human activity as quite distinct from that of non-human entities in that it is mediated by tools, the most significant of which is language. vygotsky’s work was continued by others, amongst them leontiev who developed a conceptual framework for a complete theory of human activity (leontiev 1981). according to leontiev (1981), activity is a system that has structure, its own internal transitions and transformations, and its own development. the notion of “activity” is interpreted from the theory of leontiev (1981) which is, in turn, based on the psychology of vygotsky (1978). the constructs of vygotskian psychology that are fundamental to activity theory concern those elements of activity and consciousness that distinguish humans from other beings. essentially, vygotsky (1978) defined human activity as a dialectic relationship between subject and object, i.e. a person working at something. in this dynamic, purposeful relationship the 'always active' subject learns and grows while the object is interpreted and reinterpreted by the subject in the ongoing conduct of the activity. vygotsky (1978) also proposed an 'instrumented' structure of activity within a 'system of interrelationships' between people (verenikina & gould 1998). in other words all human activity is purposeful, is carried out through the use of 'tools' and is essentially social. vygotsky also believed that tools play a mediating role in all human activities and mental processes which can only be understood in terms of the tools and signs that mediate them. tools expand our possibility to manipulate and transform objects but also restrict what can be done within the limitation of the tool, which, in turn, often stimulates improvements to the tool (verenikina & gould 1998). wertsch (1985) believed that this rich concept of mediation was vygotsky's most unique contribution. wartofsky (1979) proposed that the tools that mediate human activities could be primary (tangible, external or physical), secondary (internal, semiotic or mental), or tertiary (schematics where mind and culture act together such as environments or ecosystems). this describes the full spectrum of tools and is not a rigorous classification or separation of types. with the advent of the internet, computer-based technologies fall into all three types. leontiev (1981) proposed that “activity” should be placed at the top of the hierarchy shown in figure 1, associated sustained human endeavour with a long-term purpose and strong motives. this is a conceptual level above that at which most business analysis takes place, which is at the level of actions, undertaken towards specific, and often short-term, goals. under certain conditions, conscious actions can be driven to a lower level of automation, often in computer systems, as they become standardised as operations. an activity is comprised of sets of actions (towards specific goals) and operations (routine and well known habitual cognitive or behavioural processes, now commonly the domain of it systems). whereas an activity is defined by purpose and motive, and is typically a long-term affair, actions are more planned with specific goals and a more limited time span. actions are not meaningful in themselves unless they are part of an activity. for example, it makes no sense to drive to work (an action) unless there is a work activity to go to. 51 australasian journal of information systems volume 13 number 2 may 2006 figure 1. the definitive hierarchy of leontiev (1981) there may be legitimate alternative sets of actions that can enable the successful performance of an activity, for example: it is common practice in is development to assess the feasibility of different design solutions to an organisational problem and then choose one solution to implement based on a cost benefit analysis. however there may be instances where it is feasible to allow concurrent different solutions (i.e. different sets of actions) for an activity under different circumstances (e.g. in different countries where cultures vary or in different divisions of a company). it is important however to have a common understanding of the object (purpose) of the activity at the top of the hierarchy. kuutti and virkkunen’s research (1995) has used activity systems as a representation of the common object of organisational work which cannot be studied by reducing the scope to one or another element, but where a minimum meaningful system as a whole should be taken as the unit of analysis and intervention. engeström (1987) gave a more concrete expression to this structure in the triangular representation, shown in figure 2, which is commonly used to depict an activity. the core of an activity is a dialectic relationship between subject (human) and object (purpose) mediated by tools and community. this is a two-way concept of mediation where the capability and availability of tools mediates what is able to be done and tools, in turn, evolve to hold the historical knowledge of how the communities behaves and is organised. this is particularly powerful when the tools are computer-based (kaptelinen 1996). the formal, or informal, rules and division of labour of the community, in which the activity occurs, also dynamically mediate the subject-object relationship. to be able to analyse complex interactions and relationships, engeström (1987) proposes a new unit of analysis he calls a human activity system. according to engeström the system is object-oriented, tool mediated, and culturally mediated. engeström suggests that it is the internal tensions and contradictions of such an activity system, which includes both historical continuity and locally situated contingency that are the motive for change and development. dynamic cycles of expansive learning are of crucial importance to the historical understanding of activity systems. these cycles, shown in figure 2, combine the process of internalisation and externalisation where internalisation is the reproduction of culture by socialising and training individuals to be members of the activity system, and creative externalisation is the creation of new artefacts through innovations. australasian journal of information systems volume 13 number 2 may 2006 52 figure 2. the subject-object relationship, which defines an activity, is mediated by tools and community through rules and division of labour. the subject may be individual or collective and outcomes of the activity are distinct from its object/purpose. in addition to engeström’s structure of activity (figure 2) and leontiev’s hierarchy of activity, actions and operations (figure 1), there are several groups of researchers (kuutti & virkunnen 1995; hasan & gould 2001, engeström 1999) who use frameworks of interrelated activities to represent complex organisational situations as shown in figure 3. engeström (ibid) uses the cycle of expansive learning shown in figure 4 to demonstrate the development that takes place within an activity system due to contradictions within and between related activities. taken together the three aspects of human activity will be used to analysis and present the case described in the following section of the paper. figure 3. sets of interrelated activities in the research of engeström (1999), hasan & gould (2001) and kuutti & virkunnen (1995) 53 australasian journal of information systems volume 13 number 2 may 2006 figure 4. the cycle of expansive learning (engeström 1999) as already stated, an activity is defined by the tool-mediated relationship between subject and object that is, between the doer and their purpose. the mediation is a mutual development of both the activity and the tools which including primary (physical) tools, secondary tools (ideas, models etc) and tertiary tools, such as the community within which the activity takes place. the capability and availability of tools, mediates what can be done in the community at all three levels. why use activity theory? a major reason for the use of activity theory in is research is that it provides a well developed framework for analysing the complex dynamics of the settings that are investigated. these settings characteristically involve interacting human and technical elements – subjective, creative and emotionally experienced human behaviours together with automated, logical, analytic and routine modes of activity that can be embedded in modern technology. within this framework the core of an activity is the subject-object dialectic and it is the experience of the authors that this dialectic view of socio-technical systems is appropriate in complex settings. research in this field needs to take into consideration the dynamic interplay over time of information and communication technologies, emerging cultural trends, patterns of human experience, activity and use, and the related cultural and organizational processes that emerge, or are inhibited, as the dimensions and purpose of the whole system of work activity changes. activity theory with its focus on accumulating factors that affect the subjective interpretations, the purpose, and sense making of individual and group actions and operations, also provides a useful paradigm for the ways in which human experience, needs and creativity shape the design and effectiveness of emerging technologies where to use activity theory during the 1980s and 1990s a growing body of researchers began to recognise the relevance of activity theory to the study of information systems (is) and related disciplines. some representative publications include the work of kuutti (1991), gould (1998), korpela et al (2000) and star (1996) in information systems as well as bodker (1990), bannon (1990), kaptelinin (1992), draper (1993) in australasian journal of information systems volume 13 number 2 may 2006 54 the related field of human computer interaction. this work stems from an awareness by the researchers of the unique position of the computer as a primary, secondary and tertiary tool following the availability of the personal computer as a support tool for individuals with software such as wordprocessing, spread sheets and other decision support systems initially and then for groups through email and other web-based applications of the internet. information systems in the 1960s and 1970s had revolutionised the way organisations function by automating the day-to-day processing of billions of individual operational transactions, thereby collecting and storing huge quantities of information-producing data. there is however another class of computer-based information systems that support creative work practices rather than automate those that are routine. following the impact of the personal computer and the internet in the latter part of the twentieth century, the twenty-first century has seen computer-based connectivity, systems and devices become indispensable tools for almost everything we do. this has resulted in a complex and ever-changing work-life environment enabled by complex socio-technical tools involving ever more sophisticated and ubiquitous applications which are being developed to take advantage of the rapid advances in digital technologies. the complex phenomena associated with the socio-technical systems that have emerged in this environment are prime targets for research using activity theory as an underlying framework because activity theory provides a framework for analysing data about emerging patterns of human activity in terms of changing purposes, awareness, focus of attention, and tools. rather than the view that information systems are composed of discrete interacting elements, activity theory sees at their core the dialectic parings of subjective and objective, learning and doing, individual and collective, social and technical as well as tacit and explicit knowledge. while many developments in information systems are targeted at the global mass market or multinational corporate world, these are not the ones that most interest activity theory researchers. in highly commercial firms, government departments and educational institutions, desirable outcomes are commonly achieved, not at the organisational or individual level, but at the group level in work units, cross-functional teams or communities of people who have come together with a common interest. (linger & warne 2001). in her ground-breaking thesis, zuboff (1988) anticipated this trend noting the distinctive contextual characteristics of applications, such as decision support systems (dss) and personal productivity tools, and assigned them to the category of systems which 'informate' rather than 'automate' work practices. more recently with the convergence of communication and information technologies, such systems also shape patterns of interaction and communication. the system designs are also increasingly shaped by factors associated with communication needs and social relationships. how to use activity theory traditionally scholastic activity has been associated with ‘ivory tower’ institutions and concerns that are removed from the practical issues of work activity. in searching for research methods appropriate to the study of activities involving dynamic and complex socio-technical systems, the authors are in accord with the notion of a “new scholarship” (mcniff 2000) that is used to describe a new way of knowing that meets the everyday needs of people working in real-life situations. real-life practices are messy, uncontrolled and unpredictable and are seriously separated from the sanitised world of abstract theorising. mcniff (ibid) proposes that learning from experience, although not highly valued by the academy, can be reinforced through intellectual study. he contrasts this kind of intellectual work to traditional forms of scholarship, which values facts and information without reference to context or human purposes and experiences. 55 australasian journal of information systems volume 13 number 2 may 2006 the approach to this research is rooted in reflection-in-action, which implies that the research will be participatory, evolutionary, contextual, holistic and developmental. the developmental research method is a disciplined investigation conducted in the context of the creation and implementation of a product or program, in this case a socio-technical system and model, for the purpose of improving either the thing being developed or developer. it is holistic, contextual and evolutionary, where a prototype model is constructed and used with the target group. the dynamics of the system are analysed through participatory observation before the prototype (both the technology and the emerging new social system) is revised. the approach is strongly influenced by the expanding spiral of learning in the developmental work research (dwr) approach (engeström 1987), where communities of learning and practice are viewed as activity systems (virkkunen & kuutti 2000). dwr provides a dynamic framework that can accommodate a multifaceted analysis of the community members, their motives and purpose for belonging, their relationships within the community and the tools that mediate community activity. in this research the tools include technology together with social and learning processes. discipline is imposed on the investigation by the analysis of each case as an activity system, in the tradition of activity theory. in an activity system, the unit of analysis is the work activity itself seen as a dialectic of the subject (person or persons doing the work) and the object and purpose of the doing. work activity is culturally, technically and historically located and, where is is involved, there is usually a complex mix of interrelated activities in one activity system. the following is a activity theory framework for representing and analysing such situations an activity theory framework for is research this framework takes the concept of activity is used as the unit of analysis to explain what individuals or small groups of people do in a variety of contexts when supported by socio-technical systems. in all research there are conceptually two activity systems: that of the researchers and that which forms the object of their research. from a dwr perspective these activity systems are so closely related that, at the core of the research activity, is a dynamic subject-object dialectic where the researchers may actively participate or intervene in the activities which are the object of their research. in the is tradition of action research this lack of detachment of the researchers often poses a dilemma (mcniff 2000). however in this framework it is a basic assumption that provides a balance of depth and breadth to the scope of a study to include a richness of understanding (depth) and the inclusion of multiple interrelated and dynamic factors (breadth) that is not possible in other approaches. scaffolding onto this basis the framework then emphasises work/learning activity system being studied is generally comprised of the following components: 1. the purpose to which members of the work community direct their activity 2. individual workers/learners, their colleagues and co-workers/learners 3. the conceptual models, tools and equipment they use, and 4. the rules, culture and context that govern how they work, and learn through their work the activity theory framework also recognises the unique and pervasive nature of the computer as a primary, secondary and tertiary tool underpinning an enormous variety of organisational information systems. this is reflected in the ways in which researchers in is use computers. for example computers are used as primary tools to search for information, as secondary tools to create reports and analyse data, and increasingly web based virtual interactive spaces are used as tertiary tools to support and structure quality interaction between people in a working is system. australasian journal of information systems volume 13 number 2 may 2006 56 each vignette will now be presented within this framework under the following headings: 1. the object of the research activity – i.e. the work/learning activity system being studied in terms of the subject-object dialectics of the constituent activities and how these activities make up an activity system. 2. the subjects/people of the activities – their composition and growth as individuals and groups as the activities evolve and change 3. the purpose of those activities the various, sometimes conflicting, motives that may be involved 4. the primary, secondary and tertiary tools used for the work/learning activities being studied 5. the culture and context within which they occur 6. the tools of the research activity 7. research interventions and its evolution. illustrations of activity theory in is research the research described in the vignettes below illustrates several aspects of activity associated with research and development of information systems. in some the system is the tool that supports the research process in others the system is the object of research and also a tool for collecting data of various kinds. thus, in vignette 1 the research activity has a focus outside a technical system on structuring data collection using the q-sort method and analysing the resulting information through statistical analysis. in vignette 2 the focus is more the way new tools are used to assist in creating and analysing models of complex activities. vignettes 3 and 4 describe research that analysis the complex activity of developing information systems of various kinds and the final vignette (5) discusses the emergent research into patterns of human collaborative activity within one of the systems. vignette 1: a q-study to elucidate user requirements 1. the object of the research activity – i.e. the activity system being studied people have always sought information to assist with activities in which they are engaged. in some cases information seeking becomes an activity in its own right. nowadays the internet provides easy access to copious amounts of information through various general-purpose search engines. this research investigates the user requirements for search engines when they become tools for some wider activity, such as online shopping. the core activity of interest to this research is the process of information seeking on the internet. also of interest, are the activities for which the information sought will be used. this research uses results of a qanalysis 6 to define archetypical activities of information seekers in the context of the internet. (meloche and cheng 2005) 2. the subjects/people of the activities being studied almost everyone is a potential user of internet search-engines, however most of those in the study are younger people who have grown up in the information age. 3. the purpose of those activities and the motives of the subjects 6 http://www.qmethod.org 57 australasian journal of information systems volume 13 number 2 may 2006 this research identifies the difference in purpose of different sets of information seekers and hence sees these as different types of activities. 4, the primary, secondary and tertiary tools used the tools of interest here are internet search engines which are primary tools for the information seeker. the q-methodology uses a statistical analysis to interpret the resulting qsort data. this is a secondary research tool as is uses a model of gathering original statements and allowing participants to subjectively sort these before a factor analysis is undertaken of the sorts. 5. the culture and context within which they occur among the young people used in this study the culture of using the internet is second nature and the prime way they seek information. however, the study explores the differences between people of different groups in how they interpreted the object of the activity. 6. the tools of the research activity the research uses the q-methodology to elucidate an authentic set of requirements aligned to the user activities. the q-method involves, in the first instance, a collection of statements that represent a broad range of possible views on the topic, in this case attributes of search engines. a number of typical search engine users then individually sort these statements ranking them from those they agree with most to least. these sorts are processed to identify sets of users who have similar ranking patterns. the highly ranked statements of each user set are then recommended as design requirements for search engine incorporated into the relevant activity. 7. research interventions and its evolution the q-method elicits opinions as statements from participants in the research and then brings all statements back to allow participants to judge them in the sort. vignette 2: organisational knowledge management a multi-level systems modelling approach 1. the object of the research activity – ie the activity system being studied the current push for managers in leveraging their organisation knowledge for competitive advantage was stimulated by the work of nonaka (1994). his seci model, depicted in figure 5, describes knowledge creation in organisations spiralling from an individual to organisational level through four processes which relate to two types of knowledge, tacit and explicit. one legacy of the popularity of this model has been the stagnation of its concepts, ie that knowledge has two distinct forms, tacit knowledge held by individuals and an explicit form stored in technology as organisational memory. many managers see the ideal knowledge management system in their organisation as one that captures all individual tacit knowledge and makes it explicit and available to all. australasian journal of information systems volume 13 number 2 may 2006 58 figure 5 the seci model of knowledge creation (nonaka 1994) the research described in this vignette addresses issues of knowledge and knowledge work in organisations taking a middle-out perspective, which is neither organisational nor individual, with the objective of opening a new discourse on knowledge work in the context of modern complex organisations (linger et al 2005). within the activity theory framework, the tacitexplicit forms of knowledge can be seen as dialectic where knowledge work always involves a synthesis of the two forms. knowledge creation within an organisation is not seen as a linear process that starts with the individual and works towards the organisations but rather as dynamically emerging from the level of informal workgroups within an organisation’s knowledge eco-system (as5037-2005). the central activity of the activity system being studied is therefore the work unit where knowledge work is a dialectic of learning and doing. this relates on one hand to the activities of individual workers and on the other to the activities of managers who are concerned with strategic objectives, sustainability and competitive advantage. this research has identified a severe deficiency in the way knowledge management is understood in organisations, where the focus is either on individual knowledge or on formal processes at the organisational level. neither of these recognises that most knowledge is created through work in groups and networks that are not visible in the formal organisational structures and processes. the object of the research is to understand the complexity of how work is actually done, what is being done, who does what and how learning occurs in those activities. 2. the subjects/people of the activities being studied this research focuses on the work group as a collective subject of knowledge work, the core of the activity system being studied. the research is particularly interested in how self-directed groups can survive and grow in typical bureaucratic organisations and be the source of innovation enabling large organisations to be more agile and adaptable in rapidly changing environments. individual workers and managers are subjects of secondary activities that interact with the core activity. 3. the purpose of those activities and the motives of the subjects a thriving work group works to further the purpose of an organisation through innovation and initiative. its main motive is often job satisfaction and self-fulfilment in contrast to the motives of individual workers and managers which may tend to be more self-centred; workers to get promoted, managers to maintain control. 59 australasian journal of information systems volume 13 number 2 may 2006 4. the primary, secondary and tertiary tools used this research is particularly interested in the use of processes and technologies to further the purpose of particular activities. it contrasts the use of systems that are used to maintain the hierarchical command structures with those of self-directed work-groups which often make innovative use of socio-technical systems such as communities of practice, virtual work-spaces etc. 5. the culture and context within which they occur as mentioned previously the concept of a knowledge eco-system is used to describe the culture and context of knowledge management in organisations where a balance of elements and enables is desirable but often fraught with conflict and disharmony. this follows the lead of the australia knowledge management standard (as5037-2005). 6. the tools of the research activity the first phase of this research used the content analysis tool, leximancer 7 , to compare the use of activity theory with similar work done in a task-based approach to knowledge management (see linger et al 2005). the current phase is using the systems dynamics modelling tool, stella 8 , to simulate the interaction between individual, group, organisational and environmental knowledge activities as depicted in figure 6. both tools would be considered primary tools from an activity theory standpoint. ? individualk ? collectivek ? organisationalk ? societalk ? learning ? forgettingrusting numberofpeople ? collectivelearning ? linksbreaking? linkforming groupsize ? numberofgroups ? orglearning ? orgforgetting ? growing ? decaying ? intellectualcapital ? socialcapital ? orgculture figure 6. a stella model showing the structure and relationships of four levels of knowledge: individual, collective, organisational and societal. 7 http://www.leximancer.com 8 http:// www.iseesystems.com australasian journal of information systems volume 13 number 2 may 2006 60 7. research interventions and its evolution the researchers will use each version of the model to get feedback from workers and managers in organisations in an evolutionary process to develop the model grounded in practice. vignette3: systems development for knowledge creation _ the eviva system 9 1. the object of the research activity – ie the activity system being studied is in a relatively new field of study that has borrowed and adapted research methodologies from older disciplines. recent claims have been made that some original form of design science could be a suitable research methodology that is suited to is as its own. in this study the activity under investigation is the evolutionary development of innovative technological systems to support web-based community groups and enable research into the behaviour and effectiveness of such communities. this process may take place over an extended period of time as prototypes are created, then used and evaluated with a community over many months giving feedback for improvements to the next use of the prototype until a stable system results. the findings of this project (hasan 2003) have shown that evolutionary system design is a knowledge creation activity at various levels including the research process itself. activity theory has been used by others (e.g. bodker 1990) to study the interaction between the activities of it developers and the activities of the users of their products (activity outcomes). this research takes the activity theory analysis to another level where this interaction is seen as a tertiary tool in the knowledge creation processes of research. 2, the subjects/people of the activities being studied the team of developers of the systems studied have a mixture of skills from technical expertise to social scientists who collect and analyse the data. the composition of the team changes over time, according the cycle of research and development, but there are usually one or two who initiate and drive the project. 9 http://www.eviva.com.au 61 australasian journal of information systems volume 13 number 2 may 2006 figure 7. a screen from a networked community in eviva showing a discussion forum and community poll. 3. the purpose of those activities and the motives of the subjects these systems are being developed both to support quality interaction and team work in online communities (see figure 7) and to enable data collection for research in the area. 4. the primary, secondary and tertiary tools used the system development was sponsored by a software development company and trials of the early versions of the system were undertaken as a part of the infra-structure for an existing community network. as it is rare that such systems attract substantial funding, the development takes advantage of whatever technical and research resources are available. 5. the culture and context within which they occur the development culture is one of a small enthusiastic team with mixture of genders, ages and ethnicities. the communities using the systems are often composed of volunteers active in their community and most very grateful for the support. 6. the tools of the research activity the research methodology is a mix of design science and action research but takes cognisance of the unique properties of computers as a tool to support and inform emergent knowledge processes. this follows the steps of design science as described by march and smith. (1995). the development activity was iterative, user centred and responsive to user feed back in early trials. the unique environment could be considered a tertiary tool mediating the development activity. australasian journal of information systems volume 13 number 2 may 2006 62 7. research interventions and its evolution this project uses the development of a knowledge support system to be used in practice but also to research complex areas of online community sustainability and growth. this metaresearch process results from interpretation and analysis of the knowledge created through the evolutionary development of innovative computer-based tools. vignette 4: network-centric organisations – simulated with the go*team game 1. the object of the research activity – i.e. the activity system being studied many organisations are adopting network-centric configurations in which workers leverage information and make operational decisions through the collaborative efforts of small agile and self directed teams. go*team is a computerised client-server team version of the ancient chinese strategy game of go that has been designed to simulate contexts requiring complex collaborative strategic activity by teams. traditionally, go is played with black and white stones on a 9x9 to 19x19 grid where individual players take turns to place their stones,. unlike standard go, teams playing go*team no longer have to take turns; a team’s next turn can be taken after a “relaxation time”, specified via the server, regardless of whether or not the opposing team has done anything in the interim. there is also no preset command structure built into the go*team game. as far as the game software is concerned all team members are peers; with no predetermined roles and there is no “team leader” with more power or capabilities than other team members. the client screens for each player show only a partial view of the board so that there is a need for team members to communication their view of the board to others as well as to discuss strategies. players on the same team make use of modern communication tools such as email, voice over ip, chat rooms and the like, to effect the cooperation and coordination they need to successfully play the game. the specific set network-centric elements to be investigated with go*team include: emergent leadership, communication quality, cooperative behaviour, situation awareness, information sharing, group dynamics, trust and effective decision-making. possible independent variables are stress and tempo (timing variable in go*team), uncertainty of information, the degree of training, team structure and organisation and communication mode. the activities studied in this research are sessions where teams play go with various values of the independent variables. figure 8 a still of the server screen captured showing the whole board and the chat of the black team during a game with students. the audio communication of the white team is also on the video recording. 63 australasian journal of information systems volume 13 number 2 may 2006 2. the subjects/people of the activities being studied members of highly competitive organisations such as the military and big business are typical of the people who will be recorded playing go*team. of interest are the collective subjects (the teams) and the individuals within teams. 3. the purpose of those activities and the motives of the subjects there are three possible reasons to play go*team. on is for the purpose of research where sets of volunteers will be used. other objectives are to use go*team for training small group cooperative behaviours and another would be to profile members of an organisations for their cooperative nature. 4. the primary, secondary and tertiary tools used the go*team and associated protocols of use are the tools used in the activity. go*team embeds players in a unique environment which balances the need to cooperate within that of competition. it can therefore be considered a tertiary tool. 5. the culture and context within which they occur a mentioned most go*team will be undertaken in competitive cultures but a useful comparison will be among behaviours where team members come from diverse backgrounds. 6. the tools of the research activity the researchers are recording go*team sessions using camtasia, screen capture application. the zing groupware meeting system 10 will be used in post game debriefing sessions to collect qualitative data from participants. these are primary tools. 7. research interventions and its evolution as there are so many possible independent of variables that can be used, many of which are difficult to measure, experience of each go*team session is used to design the setting and parameters of subsequent sessions. vignette 5: content analysis of online group interaction 1. the object of the research activity – i.e. the activity system being studied this study investigates the activities of groups using an innovative web-based groupware package to support their ongoing collaborations and project development. the groups are in fact using a combination of face-to-face and online environments in carrying out these activities. however, the virtual arena for interaction extends the opportunities for shared experiences and provides an explicit and auditable record of shared interaction, patterns of emerging activity, and data about practical outcomes. 2. the subjects/people of the activities being studied users of the on-line community seeking to use the system as a context for quality interaction and intentional outcomes for projects within the group 3. the purpose of those activities and the motives of the subjects the purpose of the activity for the subjects was to work in teams on joint projects or to coordinate such activities. an emerging issue in research on network –centric systems of this 10 http://www.anyzing.com/ australasian journal of information systems volume 13 number 2 may 2006 64 kind is the need to explicitly negotiate and specify different ways of working together (rules, division of labour and community expectations) among subjects with a long cultural history of competitive and individualistic approaches to work in hierarchically organized systems. the system has been designed to provide feedback to the community as a means to support the new work practices. 4. the primary, secondary and tertiary tools used is this research the system (see figure 7) is used by both researchers and community participants as a primary tool for communication and exchange, a secondary tool as it is modified and used to provide feedback and ideas and the experience shared activity embedded within the system means it functions also as a tertiary tool. 5. the culture and context within which they occur the system is consistent with a new set of activities within the emerging digital culture. these reflect earlier experiences using older technologies but the convergence of information and communication technologies within the civil culture of developed countries has the potential to radically transform human activities and the resulting awareness and capabilities of people with a history of such experiences. 6. the tools of the research activity the researchers collect data from the online tool where transcripts of the group interaction can be captured over time. the collected text is subjected to sophisticated content analysis tool (leximancer) to dynamically map concepts contained in the texts as the group matures over time. the system also has the capability to map and report group activity, to poll user opinions and responses, and to facilitate user-based initiatives to structure their emerging activities and meet emerging needs. researching quality and purposeful interaction within the system opens up a whole new field of research in is where studies of the kind described in vignette 1 can now be carried out within the system itself. 7. research interventions and its evolution in this research the activities of the groups vary greatly and the tool used supports the variety of communicative and collaborative actions in a ubiquitous way that enables the subjects to retain focus on the object of their activities. in that way the data collection and analysis can be done in a non-intrusive manner so that the researchers obtain a more genuine interpretation of the object of their study. discussion from the work of the authors (hasan & crawford 2003a,b) and others (eg korpella et al 2000) there is copious evidence that activity theory is appropriate for is research. this paper illustrates situations where activity theory is eminently suitable as the underlying research framework. these situations invariably concern complex and dynamic knowledge-intensive work in groups or communities supported by socio-technical systems. in the paper, vignettes, based on real research projects, have been used to illustrate such situations as well as describing the research methods appropriately used for each study. in each of the vignettes described above, there is an interaction of several activities, which form innovative socio-technical systems, involving groups of workers and learners acting as a supportive and interactive community environment and enabled by new information and communications 65 australasian journal of information systems volume 13 number 2 may 2006 technologies. the key element of differentiation between each of the individual or group activities is their object or motive. the purpose of the worker is markedly different from that of the researcher. however, even within a particular group a range of multifaceted objects play out, often in unexpected ways. activity theory provides a rich, holistic understanding of how people do things together with the assistance of sophisticate tools in complex dynamic environments where socially-constructed, collective knowledge is the predominant source of learning, creativity and innovation. an activity theory analysis begins with a description of the elements of these activities and the relationships between them. the seven point framework used to analyse the vignettes highlights some of these key elements. in the technical area, the ict systems are tools that shape activities in different ways. this is partly because the objects (purposes) of users are diverse both within and between groups and within each work activity. in addition each person in a socio-technical system brings a unique cultural history, personality and knowledge base and thus interprets, makes sense of and responds to the system individually. the diversity and complexity of human responses increases both the complexity and potential of the system. within each activity, tensions arise between the sub elements of the system. for example, often the community expectations, rules and roles are not aligned with the potential permitted by new tools. as a result, they are used badly or not used at all in the activity system. in our view these tensions are the necessary precursors to evolution of work activity systems. key concepts of activity theory are consciousness shaped by experience and the subjectivity of human awareness. thus, tools are what people think they are and this can vary depending on prior experiences, culture and/or purpose. as well as the interpretations of the people changing, in the research described above, the technical systems also change their function according to the object of the activity – the same system may function as primary, secondary or tertiary tool according to the activity undertaken. in complex systems such as that described in the last vignette the technology often functions in all three ways. the expansive cycle, described by engeström incorporates the concepts of internalisation and externalisation in a collective sense and hence is related to the creation of collective knowledge. in activity theory, internalisation is manifest in the reproduction of culture by socialising and training individuals to be members of the activity system. creative externalisation is seen in innovations with the design of new artefacts and transformations of structure and process (engeström 1999). as already stated, an activity is defined by the tool-mediated, dialectic relationship between subject and object i.e. between the doer and their purpose. the mediation is a mutual development of both the activity and the tools which including primary (physical) tools, secondary tools (ideas, models etc) and tertiary tools, such as the community within which the activity takes place. the capability and availability of tools, mediates what can be done in the community at all three levels. conclusion activity systems generally now involve complex inter-related factors that shape and are shaped by both people and technology. these situations are central to current research in is. activity theory provides a comprehensive, holistic and dynamic analytical framework that, in our opinion, is ideally suited as a secondary tool for research. in this paper we have developed a seven point activity theory framework and used it to present and analyse the five research activities depicted as vignettes. this has demonstrated not only a method of australasian journal of information systems volume 13 number 2 may 2006 66 applying of activity theory to the study of socio-technical systems which mediate complex, collective activities but also the class of is topics where an activity theory based approach is able to add richness and insight. activity theory is relevant not only where situations have a significant historical and cultural context but also in dynamic situations where people , their purposes (objects) and their tools are in a process of rapid and constant change. such is the case of the current environment of is in research and practice. references as5037-2005 (2005) australian standard knowledge management, standards australia bannon l. 1990. a pilgrims progress: from cognitive science to cooperative design, ai and society 4 259-275. bødker s. 1990 through the interface: a human activity approach to user interface design, hillsdale, nj, erlbaum. draper s. 1993. activity theory; the new direction for hci, international journal of man-machine studies, 36/6, pp812-21. ellison m. and mcgrath g. m. (2001) business process modelling using activity theory: an approach to data capture and analysis, in h. hasan, e. gould, p. larkin and l. vrazalic eds, information systems and activity theory: volume 2, wollongong university press. 143172. engeström y. (1987). learning by expanding: an activity-theoretical approach to developmental research. helsinki: orienta-konsultit. engeström y. (1999) innovative learning in work teams: analysing cycles of knowledge creation in practice, in engeström y. miettinen r. punamaki r. eds perspectives on activity theory, cambridge university press, cambridge uk. hasan h. (1999) integrating is and hci using activity theory as a philosophical and theoretical basis, australian journal of information systems 6/2 44-55 hasan h. (2003) knowledge creation through systems development, proceedings of isd2003 melbourne. hasan h. and crawford k. (2003) a multifaceted approach to distributed communities of learning and practice. proceeding of the is perspectives and challenges in the context of globalization conference, athens, june 2003 for ifip wg 8.2 and wg 9.4. hasan h. and gould e. (2001) support for the sense making activity of managers, dss journal, special issue on knowledge management, 31/3 71-86 kaptelinin v. (1996) activity theory: implications for human-computer interaction in b.nardi ed context and consciousness: mit press, 103-116. korpela, m., soriyan, h. a. and olufokunbi, k. c. (2000): activity analysis as a method for information systems development: general introduction and experiments from nigeria and finland, scandinavian journal of information systems, vol. 12, 2000, pp. 191-210. kuutti, k. (1991): activity theory and its applications to information systems research and development', in h-e. nissen, h. k. klein and r. hirscheim (eds.): information systems research: contemporary approaches and emergent traditions, elsevier, amsterdam, 1991, pp. 529-549. kuutti k.and molin-juustila t.1998 integrating work activities into a network organisation by a document integration system in h. hasan, e. gould and p. hyland information systems and activity theory: tools in context, wollongong university press. kuutti k and vikkunen j (1995) organisational memory and learning network organisation: the case of finnish labour protection inspectors, proceedings of hicss28. leontiev a.n. (1981) problems of the development of mind moscow: progress 67 australasian journal of information systems volume 13 number 2 may 2006 linger, h, burstein f. and hasan h (2005) articulating knowledge work: the contributions of activity theory and task-based knowledge management, in g. whymark and h. hasan (eds,) activity as the focus of information systems research, knowledge creation press, ucq, 71-90 linger h. and warne l. (2001) making the invisible visible: modelling social learning in knowledge management context., australasian journal of information systems special edition, 56-66. march, s.t. and smith, g. (1995) design and natural science research on information technology, decision support systems 15/4,. 251-266. mcniff j (2000) action research in organisations, routledge, london meloche j and cheng li (2005) a contextual approach to design and development of interfaces for search engines, using activity theory, as the basis for informing design decisions in g. whymark and h. hasan (eds,) activity as the focus of information systems research, knowledge creation press, ucq, 93-116. nonaka i. (1994) a dynamic theory of organisational knowledge creation, organisation science 5/1, 14-37. star s.l. 1996. working together: symbolic interactionism, activity theory and information systems in y engeström and d. middleton (eds) cognition and communication at work. cambridge universty press, ny. p296-318. virkkunen j. and kuutti k. (2000) “understanding organisational learning by focusing on ‘activity systems’”, accounting, management and information technology 10 291-319. vygotsky, l.s. (1978) mind and society,: harvard university press, cambridge, ma verenikina i. and e. gould (1998) cultural-historical psychology and activity theory in hasan h, gould e. and hyland p. (eds) information systems and activity theory: tools in context wollongong university press. 1-18. wertsch, j.v. (1985) the concept of internalization in vygotsky's account of the genesis of higher mental functions in j.v. wertsch (ed) culture, communication and cognition: vygotskian perspectives, cambridge university press. warne l. ali i., hasan h. (2003) social learning through activity theory, in h. hasan i. verenikina and e gould. eds information systems and activity theory volume 3 expanding the horizon, uow press, pp 96-120. wartofsky, m.w. (1979): perception, representation, and the forms of action: towards an historical epistemology, (1973) models, representation and the scientific understanding, boston: d. reidel publishing co. zuboff s (1988) in the age of the smart machine, basic books. australasian journal of information systems volume 13 number 2 may 2006 68 ajis vol. 9, no. 2 may 2002 the evolution of confusion: soft systems methodology and social theory revisited l. houghton and p.w.j. ledington faculty of business the university of the sunshine coast piedingt@usec.edu.au abstract soft systems methodology (ssm) is a potentially powerful tool for improving the management of the complex social systems aspect of information systems. yet if it is to be employed effectively is managers need to understand the theory of social systems that makes ssm a meaningful practical approach. however finding out about that social theory is not straightforward. it is 20 years since the first discussions of the social reality implied by soft systems methodology (ssm) and the area has been given little attention since. yet ssm itself has progressed dramatically since those first critiques of its underpinning social theory were first developed. this paper revisits the area in order to provide a contemporary perspective and foundation for future development. it reveals apparent weaknesses in the research debate about ssm and social theory, and shows how the evolution of ssm has apparently been affected by that debate. ssm is introduced and examined according to the primary literature and re-evaluated using burrell and morgan's four-paradigm matrix of social theory paradigms in order to understand the social reality implied by ssm. the paper examines criticisms of ssm, the recent evolution of ssm, and suggests future directions for development. keywords: soft systems methodology, social reality, information systems management introduction soft systems methodology is a potentially powerful approach to facilitating the management of the social systems aspects of information systems. the major 'ssm-in-is' research streams have, however, concentrated upon the application of ssm to information systems development (see for example checkland & holwell 1998, lewis 1994). the application of ssm within the broader field of is management is relatively undeveloped by comparison. to develop this area further requires two distinct changes in research focus. first it requires research on treating the information system and its wider environment as a 'problem-situation-to-be-improved' and applying the principles of ssm towards that end. second, there needs to be an improved way of transferring the resulting set of ssm-based principles for is management into the practice of everyday is managers. if however information systems are viewed as 'problem situations' in this way then implicitly they are being treated as 'social systems' or at the least as socially constructed artifacts. however just what is meant by the term 'social system' or 'social construction' in this context and what implications does the definition of these terms have for information systems management.? to explore this issue further is to ask the question "what social theory provides the foundation for soft systems methodology ?". it is an important question that was addressed to some degree during the early development of ssm yet despite considerable evolution in the principles of ssm it seems not to have been given much recent attention. the aim of this paper is therefore to revisit this question and to provide a contemporary review of the issues involved. background the social reality implied by ssm has been important to the development of the applied systems thinking discipline. it forms the intellectual basis for the distinction between the so-called 'hard' and 'soft' approaches and has been seen as a limitation to ssm, which in turn provided intellectual legitimization for to the development of critical systems ideas. in this paper we re-visit this important area and by doing so aim to create a clearer understanding of the ways in which the social reality implied by ssm can be addressed. the examination will expose a change in ssm from that of a problem/action orientation to that of learning system orientation (checkland and holwell, 1998, p:12:160). ssm therefore has moved from being concerned with facilitating change in the real world to a focus on learning in the real world. the implications of this change in direction for ssm in terms of social reality need to be clearly understood. ssm began it's long process of evolution in the late 1960's when traditional methods for systems analysis were found wanting. it was observed that the traditional hard systems methods for organisational problem solving had difficulty in handling the complex nature of real world management problems. a research program into this type of real-world problem situation of a socially rich nature was launched at lancaster university eventually leading to the establishment of soft systems methodology as a distinctive management approach. there are many misconceptions about ssm, but the most glaring is the lack of a critical treatment of the radical humanist elements within ssm. in order to examine this area, a paradigm-based examination of ssm is required. whilst such a treatment was developed originally in systems thinking, systems practice (see checkland, 1982 pp.280-4) it seems as though this area has not been addressed with contemporary ssm and its 75 ajis vol.9, no. 2 may 2002 focus on the 'learning system' (checkland and holwell, 1998, p.l2:p!60) rather than action oriented problem solving (checkland, 1981, pl54:p!61). the situation may be a reaction to an argument made against ssm primarily by prevost (1976) and jackson (1982, 2001). the social reality implied by ssm is seen as interpretive by jackson and as functionalist according to prevost. however checkland argued that ssm is both interpretive and partially radically humanist (checkland, 1981, pp.280-4 and checkland, 1982, p.36-9), although the implications of this position have not been examined in the research literature. these issues are examined in more detail in the rest of the paper. the examination begins by establishing a basic understanding of ssm based upon the primary and secondary literature. a look at the paradigmatic framework derived from this literature will then be provided and then the evolution of ssm will be examined. these three points lay a foundation for examining the paradigmatic framework in the primary and secondary literature. the section that then follows takes an outward look at criticisms of ssm's paradigmatic framework. a graphical examination of the various points of view will be developed by using burrell and morgan's four paradigms as a contextual framework and will allow comparison differing views on ssm's paradigmatic positioning. the paper concludes with a summary of the major points and offers some ideas for future research. ssm in the primary literature the core texts forming the main body of research-directed work on ssm are: systems thinking, systems practice (stsp), soft systems methodology in action (ssma) and information, systems and information systems (isis) and soft systems methodology: a thirty year retrospective (ssm30). two main texts form the body of the core secondary literature. information in action (davies and ledington, 1991) and systems: concepts methodologies and applications (wilson, 1984:1990. ssm according to checkland checkland (1981) argues that the traditional scientific approach to management problems has three major faults. firstly, there are problems for the scientific method when it comes to complexity because: "cursory inspection of the world suggests that it is a giant complex with dense connections between its parts. we cannot cope with it in that form and are forced to reduce it to some separate areas which we can examine separately." (checkland, 1981,p.60). second, reducing the world into separate parts is not feasible in networks of social complexity. checkland argues that science, both natural and social sciences, ineffectively deal with the social world, especially the organisations in the social world. by this checkland means 'human activity systems' where humans participate in some purposeful/rational activity, not necessarily companies but groups of humans pursuing purposeful activity. this moves away from the idea that all systems have goals or objectives and towards the idea that these systems of human activity pursue purposeful activity in a social context. the third area that science fails in is 'management'. real world 'management' problems are everyday things that humans have to deal with that simple hypothesis testing to destruction cannot accommodate. more specifically checkland found that is it useful to divide the world into two types of problems -'hard' problems and 'soft' problems. hard problems are easy to define technical issues that often have specific properties or forms. (checkland, 1981, p.74) these properties are the same every time and the same solution could be given every time. this means that a hard scientific view of the world could be applied using 'systems ideas' and problem solving could take place. this form of problem-solving approach is termed 'hard systems thinking'. soft problems are those other forms of management' problem noted above. the way ssm deals with such social 'management' problems is well-documented. remembering that checkland wanted to use systems thinking, he took as his starting point the complexity, social science and management issues and moved towards a systems approach for real world problem solving. the next ten years were then taken up with the formulation and presentation of ssm. checkland and associates tested the systems ideas and concepts for problem solving and eventually came up with what is now known as 'prescriptive' ssm. (checkland, 1999, a35-a36). the seven stage model of ssm that emerged is an iterative process of flexible learning, not a rigid method. the process of ssm begins with the problem situation and ends with problem situation improved, ideally. this raises questions as to what the methodology hopes to achieve in the real world. ssm and it's paradigmatical context in 1979 burrell and morgan published their now classical four-paradigm model of approaches to social reality. most critics who have tackled ssm have critiqued it according to this model: (prevost, 1976, naughton, 1979, mingers, 1980:1984, checkland, 1981,p.280, jacksonl982:2001, flood, 1999). 76 ajis vol. 9, no. 2 may 2002 sociology of radical change radical humanist/neo humanis subjective reality interpretive/social relativist radical structuralism objective reality functionalist/positivist sociology of regulation (burrell and morgan, 1979, p22) the functionalist paradigm is the dominant framework for social research. it assumes pure functionality and objectivity in the social world. functionalist schools of thought are what checkland called 'scientific methods' or the method of science, and hard systems thinking belongs to this view. the key assumption here for systems theory is that the world is made up of systems that can be engineered. interpretive thought is a subjective approach to reality based firmly upon the assumption that social reality is created continually, by humans in the social world. trying to engineer the unknowable is likened to trying to mow die lawn with tweezers or trying to get a watermelon into a shot glass, it's impossible! the interpretive paradigm seems to be the paradigm that has little or nothing to do with action as such, just meaning, and it is important to note that interpretive paradigm sees: "the social world as an emergent social process that is created by the individuals concerned." and: "social reality, insofar as it is recognized to have any existence outside the consciousness of any single individual, is regarded as being little more than a network of assumptions and intersubjectively shared meanings." (ibid and p.31) the third, and arguably the most controversial, quadrant of the paradigm matrix is the radical humanist paradigm which embodies many differing doctrines including, anarchy, emancipation, consciousness, antiorganisation theory and existentialism. however: ...it's frame of reference is committed to a view of society which emphasises the importance of overthrowing existing social arrangements. one of the most basic notions of this paradigm is that the consciousness of man is dominated by these ideological superstructures with which he interacts, and dial these drive a cognitive wedge between himself and his true consciousness. this wedge is die wedge of 'alienation' or 'false consciousness', which inhibit or prevents true human fulfillment. (burrell and morgan, 1979, p.32) an important part of this view is critical theory. it uses existing disciplines, ideologies and dieories to provide a critique of the status quo in order to drive change. this change is supposed to 'emancipate' the human from ideological superstructures of the consciousness imposed on the individual by the social world in which he/she lives. therefore they: ...seek to change the social world through a change in cognition and consciousness. (burrell and morgan, 1979, p.33) ssm deals with perceptions and not cold hard facts about the real world. central to diis is the idea of the worldviews (or weltanschauung) in human activity systems. checkland argues: 77 ajis vol. 9, no. 2 may 2002 the concept of human activity system is crucially different from the concepts of natural and designed systems. these latter, once they are manifest, 'could not be other than they are', but human activity systems can be manifest only as perceptions by human actors who are free to attribute meaning to what they perceive. there will thus never be a single (testable) account of human activity system, only a set of possible accounts all valid according to particular weltanschauugen. (checkland, 1981, p. 14). the worldview is subjective. so the social reality of ssm is subjective. looking at the four paradigms, that places ssm in either the radical humanist or interpretive paradigm. the next test is the degree of change that the methodology will perform. according to the primary literature: "...given the analyst's complete freedom to select relevant systems which, when compared with the expression of the problem situation, embody either incremental or radical change, the area occupied must include some of the subjective/radical quadrant." (checkland, 1981, p.281). before examining criticisms of checkland's idea of the social nature of ssm, it is important to examine a trend in ssm. this is the move from action based to learning based orientation. ssm and it's journey from 1981 to 1998 ssm as it emerged was presented in the form of a seven-stage model. the seven stage-model was re-shaped into a process involving two streams of action (checkland and scholes, 1990, p.29) combined with three streams of analysis in 1988. one of the analyses -the social system analysis comes straight from the work of sir geoffrey vickers and the appreciative system model (checkland and scholes, 1990, p.48). checkland argues that three things interact with each other, these are: roles, norms and values. each continually defines, redefines and is itself defined by the other two. (checkland and scholes, 1990, p.49) it is a continuous process, out of which the analyst can successfully create a mental picture of norms, roles and values in the organisation. the political system analysis is a variation of the social systems analysis in as much as it is derived from vickers' model (ibid, p.50). checkland argues: "...analysis three in the stream of cultural analysis accepts that any situation will have a political dimension, and it needs to be explored." (ibid, p.50) and politics is taken to be: ".. .a process by which differing interests reach accommodation..." (ibid, p.50) checkland goes on to add that politics, which is endemic in human affairs, is about managing the relationships between those of differing interests. the next big advance by 1990 was the invention of mode 2 ssm which is the process that happens when ssm becomes internalised. however, several changes in ssm apparently take place when the shift from mode 1 to mode 2 occurs. firstly ssm mode 2 users will focus less on intervention and more on interacting in the problem situation, and intensive reflection, and 'using ssm to make sense' (checkland and scholes, 1990, p.284). mode 2 has permanently changed the focus of ssm, to that of a learning system, as shown below. model methodology driven intervention sometimes sequential ssm as an external recipe versus versus versus versus versus mode 2 situation-driven interaction always iterative ssm as a internalized model (adapted from checkland, 1999, a36) moreover the learning gained from the situation is the goal rather than change in the real world (see ssma p.283) 1998 by this stage of the development, ssm has only five stages. (see 75/5 p. 160) however, the approach now is taken to be five models of purposeful activity molded into one process as a learning system. 78 ajis vol. 9, no. 2 may 2002 contemporary ssm contains many of the principles found in previous incarnations but with specific re-wording and the complete demolition of the systems thinking line (see checkland and tsouvalis (1997) for a full explanation of this). in this incarnation the focus is on learning and a lot less on intervention and problem solving. this is a decided shift in parts of ssm from humanism to the interpretive paradigm. the original reason ssm was developed was to 'solve' hard to define problems in the real world. now the focus in ssm seems to be on the learning, and the improvement is not as important contemporary ssm the contemporary view of ssm is that of isis. the constitutive rules of are now described as: 1. you must accept and act according to the assumption that social reality is socially constructed continuously; 2. you must use explicit intellectual devices consciously to explore, understand and act in the situation in question; and 3. you must include in the intellectual devices 'holons' in the form of systems models of purposeful activity built of the basis of declared world views. (checkland, 1999, a35) changes to the existing processes have been suggested by ledington and ledington (1999) in the form of decision variable partitioning and extending the comparison phase of ssm (1999) but few other theorists have offered any worthwhile contributions to date. interpretations and criticisms of ssm's paradigmatic framework in the research literature. the most consistent critic of ssm is critical systems theorist mike jackson. in 1982 he suggested, after mingers (1980) that ssm had no provision for political structure and was limited by it's paradigmatic framework. for example. "using burrell and morgan's framework i argued soft systems thinking is situated within the interpretive paradigm in that it's guiding assumptions are subjective and regulative." (jackson, 2001, p.236) jackson also argues that because of ssm's conservative bias it would never be able to help the oppressed in anyway. jackson's critique continued by stating that ssm could not be radical and emancipatory like critical theory but would only help to confirm the status quo because of it's philosophical implications (jackson 1982). this philosophical examination of ssm by jackson was done according to the burrell and morgan paradigms. jackson in essence was arguing that ssm fits into the interpretive paradigm. checkland presented the methodology as fitting over radical humanism/interpretive paradigms as mentioned earlier. mingers (1984) opened the debate again. however, mingers who suggested that critical theory and ssm could share a mutual dialogue (mingers, 1980) has chosen to focus on the critical systems thinking developed by jackson. jackson says that soft systems methodology has an inbuilt inability to handle complex human and social aspects of social situations then offers no physical evidence to support this claim (ibid, p.235). ssm seems flexible enough to jump paradigmatical boundaries into humanism or even back to functionalism if the user so desires. it is a methodology not a method. at least checkland some has evidence to support his position (see ssma chapters 6,7,8,9 and isis chapter 7 especially). others have looked at ssm's paradigmatic framework, not to the extent that jackson did but they are worth noting. it is interesting to note that hirshheim et al, (1995) and ledington and ledington have both criticized the modeling phases of ssm as too simple and possibly functionalist. naughton (1979) argued against the case that ssm is functionalist, a charge made by prevost (1976) but stated that ssm had a whiff of functionalism (checkland, 1981, p.252). despite all these critiques of the social reality implied by ssm there is no evidence from either side to support their the cases. checkland in 1981 said: .. .the comparison will lead to a discussion of possible changes. these are of several kinds, and any combination maybe appropriate in a particular situation. (checkland, 1981, p. 180) is this interpretive practice? checkland argues that it isn't: ...given the analysts complete freedom to select relevant systems which, when compared with the expression of the problem situation, embody either incremental or radical change... (checkland, 1981, p.281) this will be limited by a desire for action in the real world (ibid) but the ideals are still humanist ideals. 79 ajis vol. 9, no. 2 may 2002 secondly, ssm contains underlying principles that can provide a critique of the status quo in any given organisation. stage 5 provides a tool for using the systems ideologies against the real world perceptions found in stage 2. it is hoped that be comparing stage 4 with 2 that a debate will be started about feasible changes. this suggests that ssm is built around rudimentary principles rooted in critical theory. mingers (1980) recognized this: "...three major points of agreement. firstly both take seriously the problem of human action at the same time purposive/rational ... and natural or unchangeable as a result of the characteristics of human animal. secondly, both conclude that hard systems analysis, cannot cope adequately with the multivaried complexities of the real world. finally both deny the inevitability of the divorce between rationality and the values which characterizes natural science." (checkland, 1981,p.283) in order to understand the various perceptions of ssm and its paradigmatic framework, this paper will now present tables that map ssm. the mapping of ssm the first map presented is that representing the view of checkland, 1981. radical structuralist subjective reality sociology of radical change ical humanist/neo hur la list interpretive pbenomen.soc. sociology of regulation objective reality functionalist/positivist (burrell and morgan, 1979, p22) as previously mentioned checkland argued that ssm had compatibilities with critical theory therefore should be mapped that way. next is jackson's case. 80 ajis vol. 9, no. 2 may 2002 sociology of radical change radical humanist/neo humanist subjective reality interpretive phenomena. soc. radical structuralism objective reality functionalist/positivist sociology of radical change now the critics of conceptual modeling which are hirschheim et al. and ledington and ledington sociology of regulation radical humanist/neo huma list radical structuralist subjective reality intepriretive phenomen.! objective reality functionalist/positivist sociology of regulation (burrell and morgan, 1979, p22) hirscheim et al. and ledington and ledington have argued that ssm's conceptual modeling is too simple. there is evidence (unlike most critics of ssm's paradigmatic framework) in ledington and ledington to suggest this (ledington and ledington, 1999) but it is not expressed as a direct argument against ssm's paradigmatic framework. the above is just an assumption derived from the literature and is not explicitly stated. the maps here and the previous section lead to the question. what then is the social reality implied by ssm? the answer to the question is that the literature is highly confused about ssm and its relation to social theory; which becomes evident when examining the material presented here. there is no agreement as to whether or not ssm is mappable over two paradigms or even one. jackson provided no evidence for his position and checkland provided no evidence for his either. ssm has been developed from practice through a process of action research and checkland continually argues the case for experience being the foundation for the principles of ssm. in this sense it could be argued it reflects the nature of the situations with which it deals. given that such situations are likely to be much more complex and subtle than the theory that purports to represent them then this apparent 81 ajis vol. 9, no. 2 may 2002 mis-match between ssm and social theory is understandable. therefore the conclusion that can be drawn is that the current literature cannot provide a definitive answer to the nature of the social reality implied by ssm and therefore that this area and its contribution to improving the understanding of ssm is ripe for further research efforts. implications and conclusions the reading of the ssm literature given in earlier sections shows clearly that it can no longer be seen as a singular coherent structure. there are at least three epochs of ssm. we might call them early, middle, and late checkland early checkland is focussed upon the transition from hard to soft systems thinking and the formulation of ssm as an iterative seven-stage process. middle checkland depicts ssm as a linear two-stream (cultural analysis and logic-based analysis) process whilst being concerned to make sense of practice in terms of the emergent ideas of formalized, external (mode 1) and informal, internalized (mode 2) types of usage. late checkland images ssm as a bare iconic structure of four activities forming a learning process, but with mode 1 and mode 2 embedded as a structure relevant to practice. ssm therefore emerges as phases of evolution each of which involves re-defining the core principles, or constitutive rules, of ssm. it is however far from clear how these three phases interrelate with each other and what implications these changes have for the discussion of underpinning social theory. the critiques of ssm and its implied social reality are stranded in early ssm and do not address the changes in the later research literature. further they tend to rely upon the notion of the incommensurability of paradigms as proposed by burrell & morgan. on this basis ssm can only fit into one or other of the four quadrants and cannot reflect aspects of more than one. the evidence examined in this paper suggests however that the relationship between management practice and social theory is perhaps more complex that has previously been assumed. again the consequences of multi-paradigm mapping and how this might help focus attention on areas of strength and weakness in ssm, and hence provide a basis for seeking to develop it further have not been examined. further the discussion of this area is sparse, limited, repetitive, and inconsistent. different authors form different conclusions about the issue, and there is not resolution. the area is clearly problemmatical and confused. if we accept that thirty years or so of careful development and application, supported by public scrutiny, provides some substance to the claims of ssm then some implications follow. the core that runs through all of the epochs of ssm is that it provides a set of tools for 'structuring', or facilitating (some aspects of) the processes of social construction of reality in a particular context. we must therefore assume that such processes are an inherent part of the context or situation, and that these processes can be more or less effective in generating effective collective purposeful action. in simpler language, ssm operates by improving complex information processing. it is therefore implicitly a fundamental framework for information management. in principle it would appear ssm ought to provide a foundation for creating more effective forms of information management. the challenge is to test this proposition hi practical situations. references burrell, g and morgan, g (1979) sociological paradigms and organisational analysis, ashgate press, uk. checkland, p (1981) systems thinking, systems practice, john wiley, uk. checkland, p (1982) soft systems methodology as process: a reply to m.c. jackson, journal of applied systems analysis, volume 9, department of systems, university of lancaster, uk. checkland, p (1990) soft systems methodology in action. john wiley, uk. checkland, p and tsouvalis, c (1997) reflecting on ssm: the link between root definitions and conceptual models, journal of systems research and behavioral science, vol 14, no.3, wiley, uk. checkland, p and holwell, s (1998) information, systems and information systems, john wiley, uk. checkland, p (1999) soft systems methodology: a thirty year perspective, john wiley, uk. churchman, c. w (1968) the systems approach, delta publishing, usa. davies, l. j. and ledington, p.w.j (1991) information in action, macmillian, uk. flood, r l. (1999) re-thinking the fifth discipline, routledge, new york. hirschheim, r., klien, h. k. & lyytinen, k. (1995) information systems development and data modeling: conceptual and philosophical foundations,_the university press, cambridge, uk. jackson, m c. (1982) the nature of soft systems thinking: the work of churchman, ackoff and checkland, journal of applied systems analysis, volume 9, department of systems, university of lancaster, uk. jackson, m c. (2001) critical systems thinking and practice, european journal of the operational society, volume 128, elsevier. 82 ajis vol. 9, no. 2 may 2002 ledington, j and ledington, p.w.j (1999) decision variable partitioning: an alternative modeling approach in soft systems methodology, european journal of information systems, volume 8, operational research society, uk. lewis, p (1994) information systems development,. pitman, london mingers, j (1980) towards an appropriate social theory for applied systems thinking: critical theory and soft systems methodology, journal of applied systems analysis, volume 7, department of systems, university of lancaster. mingers, j (1984) subjectvism and soft systems methodology a critique, journal of applied systems analysis, volume 11, department of systems, university of lancaster. mingers j (2000) variety is the spice of life: combining soft and hard or/ms methods, international transactions in operational research, volume 7, pergamon. naughton, j (1979) functionalism and systems research: a comment, journal of applied systems analysis, 6, department of systems, university of lancaster. prevost p( 1976) "soft" systems methodology, functionalism, and the social sciences,_journal of applied systems analysis, 5, department of systems, university of lancaster. wilson, b (1984) systems: concepts, methodologies and applications,. chichester, wiley. wilson, b(1990) systems: concepts, methodologies and applications: second edition^ chichester, wiley. 83 microsoft word special final.doc ajis special issue december 2004 122 shared visualizations in support of distributed creative communities alastair weakley, ernest edmonds creativity & cognition studios university of technology, sydney australia email: alastair@weakley.org.uk, ernest@ernestedmonds.com abstract the paper is concerned with support for distributed groups of creative knowledge workers: in this case designers. we consider requirements that designers have regarding internalisation and externalisation of ideas and concepts as well as requirements relating to collaboration. we review an online system whose facilities for the graphical representation of data were found to be popular. the evaluation was in the context of a group task and the results, including instances of tacit knowledge sharing, have led us to formulate a number of recommendations as to how such systems might be made still more effective for collaborative working. keywords design, creativity, collaboration, distributed groups, support systems introduction this paper is concerned with collaboration among distributed, loosely-formed groups whose members are engaged in creative knowledge work. specifically the interest is with how such groups can, or could, carry out the sorts of knowledge work that are traditionally associated with closelycoupled face-to-face working. in this case, creative knowledge work has been chosen as the domain of study because this type of activity involving, as it does, both problem-setting and problem-solving (schön, 1983) presents particularly difficult challenges to designers of support systems. in this paper, we consider some specific needs that creative knowledge workers have in relation to externalising, visualizing and communicating concepts and ideas as they develop. we consider how well these needs have been met by existing systems and how future support systems might be made still more effective. the area of creativity has been the subject of a great deal of research and discussion, and the term can be used to cover a wide range of activities. in this work the specific branch of creative endeavour that has been chosen as the area of study is design. the act of design in this case is defined, following papanek (papanek, 1972), as the planning and patterning of any act towards a desired, foreseeable end. we describe some common aspects of design work. additionally, the nature of groups and group working is considered. in this case, the primary concern is with loosely structured groups rather than well-defined teams. such groups usually share some common interest or association although this is not necessarily the primary interest of each individual member. within such a group, associations and links may form and change as work progresses. in particular, this section is concerned with the nature of the communications needed to sustain such relationships. drawing on the results of an evaluation of the wisa (web interactive scrapbook application) system, we arrive at some recommendations as to how such systems might be improved. ajis special issue december 2004 123 visualization in design: shneiderman (shneiderman, 2002) identifies eight important 'tasks for creativity'. these are: searching and browsing digital libraries, the web and other resources; visualizing data and processes to understand and discover relationships; consulting with peers and mentors for intellectual and emotional support; thinking by free associations to make new combinations of ideas; exploring solutionswhat-if tools and simulation models; composing artefacts and performances step-by-step; reviewing and replaying session histories to support reflection; disseminating results to gain recognition and add to the searchable resources. here we are specifically concerned with those relating to visualization. the organization and analysis of complex data is important at two distinct stages in the type of creative work under consideration here. one the one hand, designers, in their search for knowledge and information that will help them to first understand and then solve the problems with which they are faced, must sift through, and get to grips with large amounts of data. as shneiderman suggests, visualization of such data can help with organization and understanding: (shneiderman, 2002) "[d]rawing mental or concept maps of current knowledge helps users organize their knowledge, see relationships and possibly spot what is missing." it is, in the first case, during this investigative process that visualization of complex information can be useful. this process is directed towards understanding and coming to grips with new information and knowledge: internalising it in some way. there is another area where designers commonly use visualizations of one sort or another. this is in the course of an activity that can be described as 'externalisation'. here the designer may use models of one sort or another to externalise their ideas, as shneiderman describes: thinking by free associations and carrying out 'what-if' style investigations. this in turn they may do for two reasons: to allow for visual organization, manipulation or examination of another sort as an aid to understanding as part of the creative process, and also to aid in the communication of ideas to others. lawson, for example, reports that it is clear from his analysis that "the designer must inevitably expend considerable energy not just in solving but also in identifying the problems confronting him". a large part of the designer's work is in defining and understanding the problem that is to be solved. often the exact nature of this problem does not become clear until late in the process when a considerable amount of investigation and exploration has already been done. indeed, design is a process of "incremental formalization" (gross, 1996) and it is generally accepted that at the start of the process design problems are poorly-defined. schön (schön, 1983) describes how design, as well as necessitating finding the solution to a problem, also involves the definition of the problem itself. this problem setting he describes as the means "by which we define the decision to be made, the ends to be achieved, the means which may be chosen." there are therefore two distinct areas where visualization in some form is important: during the process of internalisation and during the process of externalisation. in the first case, the designers are trying to grasp and assimilate external knowledge and in the second case the knowledge is in some sense, at least in part, already internal. these areas, while distinct and each with their own specific requirements do have certain similarities and so they shall be considered in tandem here. a major concern in this discussion will be the extent to which, and in what ways, technological systems can affect or enhance these processes of internalisation and externalisation. examples of systems supporting internalisation one recently-developed system is market map (wattenberg, 1999a) described by (wattenberg, 1999b). here in an online system data is presented about the new york stock market. the companies themselves are represented as nodes in a treemap (shneiderman, 1991): each company is ajis special issue december 2004 124 represented by a rectangle on the screen whose size is proportional to the market capitalization of the company. in turn, the company rectangles are arranged to fill larger rectangles representing each stock market sector (technology, pharmaceuticals and so on). the colours of the rectangles represent the recent changes in the stock price of the company: shades of red represent a fall in price, shades of green a recent rise. as the mouse is moved over each company-rectangle, a box appears showing the company name, the current share price, the amount by which its price has changed in a particular timeframe. a single click on a company-rectangle brings up a menu offering links to greater detail on that company or the market sector to which it belongs: charts, news articles and financial details are all available with one more click. it is possible to move very quickly from a high-level overview down to precise detail on one particular company. this very effective system demonstrates exactly the properties that are desirable in the process of internalisation: a complex mass of data is made rapidly accessible. (stappers and pasman, 2000) describe an interactive system aimed at allowing searches with complex criteria to be carried out easily. in this case, the multidimensional scaling approach is used to display database records arranged, again graphically in this case, according to similarity. thus, were for example the contents of a database of wines to be displayed, those with similar flavours might be grouped together. to start with, only a few records are displayed. the user may see details of each one. when they click on a gap between records, a new query is generated and executed: a new sample is displayed that most closely matches the query. in this way, to take stappers and pasman’s example, it would be possible to find a wine 'somewhat sweeter than a burgundy' from a database of wines, whereas carrying out the same search using text criteria might be very difficult. externalisation now we shall consider visualization as applied to externalisation. it is common during, and often as an integral part of, the design process for designers to externalise their thoughts and ideas in the form of models of one sort or another. these models may take many different forms, for example they may be mathematical models, they may be vocal descriptions, they may be physical models or they may be diagrammatical models. commonly designers will make such re-representations either as an integral part of the design process or in order to share some thought, concept or idea with someone else. drawing and sketching, for example, is seen as a very important part of the design process. as lawson (lawson, 1990) explains "[t]he whole purpose of doodles, sketches or models is to act as a kind of additional memory to freeze and store spatial ideas which can then be evaluated and manipulated." schön expands on this describing how these externalisations themselves inform further work. here he discusses visualisation in the context of architectural design: (schön, 1992) "a designer sees, moves and sees again. working in some visual medium .... the designer sees what is 'there' in some representation of a site, draws in relation to it, and sees what he/she has drawn, thereby informing further drawing." schön describes this process as the designer having a 'reflective conversation' with the representation. it is the two-way nature of the conversation that differentiates this process from the exploration of data made possible by, for example, market map. the feedback loop which schön describes where the designer finds new meaning and understanding in the representation they have created is a key cause of the difference in the representations made and therefore in the requirements for systems that support them. an important point that schön makes is that it is through the "unintended effects of action" that this new understanding can arise. the act of re-representation in this context is a voyage of discovery and not just one of exploration. it has been the case that traditional computer systems tend to support the process of internalisation more fully than externalisation. however, a number of flexible systems have been developed. nakakoji and yamamoto for example present the art series of systems (nakakoji and yamamoto, 2001, ajis special issue december 2004 125 yamamoto et al., 2000, yamamoto et al., 2001, yamamoto et al., 2003, yamamoto et al., 2002) which support 2d spatial positioning during the process of creating for example text or multimedia files. as they explain (yamamoto et al., 2000) 2d spatial positioning can take the place of sketching in domains where sketching is not appropriate. art stands for 'amplifying representational talkback' and a concern of this work is with how the use of such systems can help to promote the two-way conversations between the user and the representation as described above. in a similar vein, the spatial hypertext editor tinderbox (bernstein, 2004) allows arrangements and hierarchies of objects to be viewed in many different ways, and for those objects to be easily and quickly rearranged. here again, the system offers considerable scope both in terms of what can be represented and how this can be done. communication in groups design problems have been described as 'wicked' and it is commonly the case that a single person does not have sufficient skills to solve the problem single-handed (fischer, 2000). in this paper, the concern is not with well-defined teams of people but with loosely-structured groups: that is groups whose members are for the most part only casually connected. within this loose structure, group members may seek out collaborators for specific tasks. closer bonds, then, may form temporarily where members work together on specific problems. this closer work might last for a few weeks or months. in such a group, there is a fluid network of ties of various degrees of strength between members. in addition, members of such groups may also need to interact with individuals outside the group to complete their assignments (ancona and caldwell, 1990). as ancona and caldwell point out, the more typical model of group work concentrates exclusively on interactions within the group rather than including those that extend beyond its boundary. yet, these external bonds are crucially important. in fact, it might be more correct to consider the boundary of the group as itself being fluid. as collaborations form new members, possibly whole subgroups, may be temporarily subsumed into or appended to the main group. (egido, 1990) notes that ‘[i]t is often over informal chats outside of official meeting rooms that important information is transmitted and real decisions are made.’ she notes that, for example, teleconferencing systems may be unpopular with politically-savvy employees because they do not commonly support such casual interaction. kraut, egido and galegher (kraut et al., 1990) found that physical collocation is an important factor in the formation of collaborative partnerships. they studied the frequency of collaborations among researchers whose offices were at a range of distances from one another. significantly, the distances in question were typically measured in metres rather than kilometres: there was no question of longdistance teleconferencing here. the indications of their research were again that it is often informal interaction, chance meetings and casual discussions that lead to the formation of new relationships. they suggest that the opportunity for casual discussion enables people to discover shared interests and establish common ground with potential collaborators. where people work at even a relatively short distance from one another, these casual meetings are less likely to happen and so new working relationships are less likely to form. as kraut et al explain: (kraut et al., 1990) ‘most often, naturally occurring, informal contact and communication provide the opportunity for potential collaborators to learn about each other, and also serve as the framework within which collaborative tasks are accomplished.’ rosenberg (rosenberg, 2001) points out the value of being able to overhear conversations in a shared workspace. as she says, '[i]t is often important for people sharing the same workspace to be aware that a meeting or a conversation is taking place, so that they can join in or leave at appropriate points.' (perry and rosenberg, 1998) also suggest that electronic communications media typically do not capture or transmit this valuable information. she says that such systems commonly reduce ajis special issue december 2004 126 opportunities for informal communication such as: chatting, overhearing conversations, seeing current work laid out on other people's desks, and so on. modern technologies have provided ways for us to communicate with others no matter where they are. the use of mobile communications devices, for example, allows us to connect directly to a person wherever they happen to be. in contrast, with the traditional telephone system, we connected to a place in the hope that the person we wanted to speak to was there (wellman, 2001). of course this means that we may not initially know the social context of the person we are calling: are they at home or at work for example? it is often the case that such contextual information is not shared when we use modern communications systems. additionally, (luff et al., 2003) suggest that it is becoming increasingly evident that interpersonal communication relies on shared access to artefacts within the environment. these might be documents, models, or other physical artefacts, which are either directly the objects of discussion or which are used to clarify points or which passively facilitate communication. (gaver, 1992), for example points out that face-to-face collaboration is "situated within a shared, encompassing space, one which is rich with perceptual information about objects and events that can be explored and manipulated." we are here concerned not only with talking about these objects, but with using them as shared communicative facilitators: talking through the objects and the surroundings. there are principally three ways in which the environment or environmental artefacts become part of a conversation: when they are, perhaps unconsciously, used to conceptualise a statement, when they are deliberately used to illustrate a point or are themselves the object of discussion, and as passively-informative objects in the environment. we will consider each in turn: our environment as a whole particularly contextualizes what we have to say, although, when we are face to face, this is often not explicitly stated nor even consciously registered. as, for example, sperber and wilson suggest "a single sentence, with a single semantic representation, can express an unbounded range of thoughts" (sperber and wilson, 1986). we can only pin down the meaning when we consider the setting (social as well as physical) in which it is made. once our communication is decontextualized by separating it from its environment, it can become ambiguous: the communication is compromised. churchill explains the role of artefacts that are deliberately used in conversation: (churchill and erickson, 2003) "people elaborate on their statements by picking things up or walking around them, by turning them around, by pointing to certain features, by inviting and indicating novel visual stances or perspectives to take, by glancing in the direction of objects as they become the focus of the conversation, and even by turning their backs on them." here we can see that objects are used to clarify, illustrate and explain; again where parties to the conversation do not all have access to these objects, communication becomes unclear. in a face-to-face setting, it is possible to see where someone is looking, and therefore to work out how an object looks to them. as kraut et al (kraut et al., 2003) describe: "[w]hen people can see where each person is looking, it is easier to establish common ground." a hotel key rack, as (perry and rosenberg, 1998) suggests is a good example of a passive environmental communicative shared artifact. by looking at the key rack we can gather a great deal of information. we can tell how many rooms there are in the hotel, and how many of them are occupied, whether a particular person is in or out and whether they have mail. our worlds are filled with such common artefacts that allow us to deduce useful information about our situations and our surroundings. a number of systems have been developed with the aim of addressing the issue of sharing such environmental information. rosenberg and perry (perry and rosenberg, 1998), for example, presents the people and information finder (pif). the pif is a web-based system aimed at supporting communication among distributed teams in the construction industry. as they explain, these teams are dynamic or 'agile', typically only being drawn together for a single project. the pif ajis special issue december 2004 127 presents information about team members to help in the establishment of common ground. the information is divided into sections: the first consists of data about locating the team member 'in their physical space' : telephone number, address and so on, also included here are photographs and video images showing their offices. additionally there are team spaces which provide similar information about other teams within the organization. also it is possible to see page owner's past activities. as perry and rosenberg explain: (perry and rosenberg, 1998) "the pif and its underlying informational structure provides a world reduced in detail, but augmented in contextual complexity". studying such a system allows one to place an individual and a team within the context of their office in the first instance. seeing images of how this is arranged provides contextual information about that person. equally the ability to study past work adds important information about their interests and areas of expertise. another system, this time slightly more dynamic in its contextualization but in this case providing less detail has been developed by zhao and stasko (zhao and stasko, 2000). described as an 'opportunistic peripheral interface' the 'what's happening?' application displays a small window on the desktop of all its users. in this window appears a scrolling list of announcements. as well as displaying general information: the community calendar, the weather forecast and so on, it is possible for users to add personal information 'it's my birthday' for example. by clicking next to a message users can add comments which everyone can see. as they note, this is a tool aimed at maintaining awareness of what is happening in the distributed community. it is unobtrusive but it provides shared context for future discussions. strengthening and maintaining common ground by making sure that everyone knows both about important activities, and about personal contextual information on individual group members. this is exactly the sort of background information that can support communication among people who do not share the same environment. in this case, though, the information is not so much related to shared physical artefacts and settings but to shared events. the wisa system wisa, (the web interactive scrapbook application, described more fully in (weakley and edmonds, 2004)) aims to support designers in gathering, organizing and sharing useful resources. the process of design often involves a significant amount of research into new technologies and practices and wisa provides an online facility in which to store and organize references to pictures, people, notes and so on: the results of such research. collections are organized in a series of 'projects' equivalent to pages in a paper scrapbook. like cuttings in a conventional scrapbook, the individual objects ('resources' in the figure 14: resource tile ajis special issue december 2004 128 figure 15: main project view ajis special issue december 2004 129 parlance of this system) can be arranged and grouped in two-dimensional space. where wisa extends the functionality of the traditional paper scrapbook is in its support for the sharing of resources. wisa can be accessed using a normal web browser: this makes it possible for several people to share a project and to work on it together from multiple locations. additionally, it is possible to search for useful material: the contents of all projects in the system are available for searching. in this way, wisa supports members of the group in discovering new, unexpected, information about one another. the system aims not just to support sharing of the information that its users have collected but also to support discovery of common interests and complementary skills. in the wisa system, individual resources are represented as tiles in the project space (see figure 1: resource tile). within this space, which represents a single page of the scrapbook, individual tiles can be grouped and rearranged as desired (see figure 2: ). tiles are arranged by dragging and dropping, they can also be stacked on top of one another. the position of each tile is recorded along with details about who put it there and when. it is therefore possible to re-enact the development of a project at any time and to see just how the current arrangement of tiles came about. when a project is replayed, the tiles appear in the browser window and move into the positions they occupied as the project developed. it is by means of this fluid rearrangement that the system aims to support externalisation in the research phases of the design process. the users may experiment with different arrangements and juxtapositions of resource tiles in an attempt to make new associations and connections between them. additionally, when a search is carried out, data from other group members' projects may be uncovered. it is possible to make such searches sensitive to the spatial arrangements and groupings that these other users of the system have made. when a location-sensitive search is carried out, resource tiles matching the search criteria are displayed together with those that have been placed near to the ones that match. in this way, the system aims to expose associations between objects made by other users. looking at these connections and groupings allows one to learn about the resource objects themselves as well as about other members of the user group. each resource tile, as well as containing data such as text or an image, may be annotated by the users of the system (see figure 3: annotation). many notes may be attached to each tile in a project. again this supports sharing of information both about the objects in the collection and about the users making the annotation. figure 16: annotation ajis special issue december 2004 130 evaluation of wisa in an evaluation (weakley and edmonds, 2004), two groups of volunteer users were asked to imagine that they were preparing to write a conference paper. they were invited to use the wisa system to start to build a collection of useful material for their paper. in this case, then the groups were defined at the start and the evaluation therefore concentrated on support for existing groups rather than formation of new relationships. the users were asked whether they would like to have a text-only display of resources (for example a list) instead of the graphical display. they were also asked if they would have liked a text-only display in addition to the graphical one. in both cases the answer was unanimously 'no' and this clearly indicates that the graphical interface of the wisa system has significant benefits over plain text. nevertheless, looking at the layout of another person’s project was found to be only moderately useful. this raises the interesting question of whether such spatial arrangements may be of greater use for people working on their own, or where their communication is more closely-coupled, such as during a face-to-face meeting, rather than when working in a distributed group where communication relies to a great extent on the visual layout as in this specific case. a number of comments were made about confusion caused when one user rearranged objects in a project and the others did not know the rationale behind the new arrangement. it seems clear that to support group working additional communication facilities must be added to wisa. there were also instances where users who had previously worked together for some time discovered new common interests, outside the scope of the evaluation, and were able to share information that had been collected and stored in wisa. these connections were made when one user came across an interesting-looking resource belonging to another. it is significant to note that, while these new connections might not have been made without wisa, they were not made by one person interpreting the spatial arrangement made by another. in this sense, the connection could be made independent of a knowledge of context. it was not necessary to know why the interesting resource was present or what purpose it served for its owner its mere presence was sufficient. wisa has proved to be useful in one sense, but like so many other collaboration support systems it transmits only part of the data. while resources are presented in the context of a project (which is useful to the project's creator), in order to be more useful to others the required context is about the user themselves: this might lead one to an understanding of the reason behind their action. it is the addition of support for this sort of information exchange that should form the basis of future versions of this system. as one of the system's evaluators put it: "there are many depths and features in the background engine which aren't adequately represented in the interface": there is clearly scope to expose more of the information that the system captures. we can therefore start to formulate recommendations for additional features that might be incorporated in future versions of wisa as follows: support direct communication among users support awareness of other users’ past actions more clearly than at present: what have they done support the recording and sharing of information that contextualises the actions of the users: when did they do it and what else were they doing at the same time, for example support ‘overhearing’ among users: provide unobtrusive alerting facilities where possible support users in announcing what they are doing or what they require. conclusion we have considered the processes of internalisation and externalisation in the design process and have identified approaches for supporting this. we have also briefly touched upon the needs of ajis special issue december 2004 131 members of loosely formed groups in particular relating to the sharing of contextual information of various sorts. in considering the wisa system, we found that it appears to support a personal, reflective approach to internalisation and externalisation and furthermore that it can support the communication and sharing of tacit knowledge among its users. moreover, evaluation of the system exposed some areas that could be improved. it is clear that the author of a spatial arrangement or grouping of objects in the wisa system may invest in it significances that are beyond those that become apparent to an observer studying the arrangement alone, useful as this may be. adding contextual information about what the user was doing and why would improve matters. additionally, returning to the work of rosenberg and zhao and stasko: adding alerting features that would allow one to ‘overhear’ what another user was engaged in would be useful. it is clear that whilst the visual layout and sharing of project information presented by a system such as wisa makes an important contribution, it should be seen as only one element in a multi-modal communication facility. references ancona, d. g. and caldwell, d. f. (1990). information technology and work groups: the case of new product teams. in intellectual teamwork: social and technological foundations of cooperative work(eds, galegher, j., kraut, r. e. and egido, c.) lawrence erlbaum associates, hillsdale new jersey, pp. 173-190. bernstein, m. (2004). tinderbox. eastgate systems inc., watertown, ma. churchill, e. f. and erickson, t. (2003). introduction to this special issue on talking about things in mediated conversations. in human computer interaction, 18, 1-11. egido, c. (1990). teleconferencing as a technology to support cooperative work: its possiblities and limitations. in intellectual teamwork: social and technological foundations of cooperative work(eds, galegher, j., kraut, r. e. and egido, c.) lawrence erlbaum associates, hillsdale new jersey, pp. 351-371. fischer, g. (2000). symmetry of ignorance, social creativity, and meta-design. in knowledgebased systems, 13, 527-537. gaver, w. w. (1992). the affordances of media spaces for collaboration. in cscw 92 proceedingsacm press, pp. 17-24. gross, m. d. (1996). the electronic cocktail napkina computational environment for working with design diagrams. in design studies, 17, 53-69. kraut, r. e., egido, c. and galegher, j. (1990). patterns of contact and communication in scientific research collaboration. in intellectual teamwork: social and technological foundations of cooperative work(eds, galegher, j., kraut, r. e. and egido, c.) lawrence erlbaum associates, hillsdale new jersey, pp. 149-171. kraut, r. e., fussell, s. r. and siegel, j. (2003). visual information as a conversational resource in collaborative physical tasks. in human computer interaction, 18, 13-49. lawson, b. (1990). how designers think: the design process demystified. butterworths, london. luff, p., heath, c., kuzuoka, h., hindmarsh, j., jamazaki, k. and oyama, s. (2003). fractured ecologies: creating environments for collaboration. in human computer interaction, 18, 51-84. nakakoji, k. and yamamoto, y. (2001). spatial positioning as a representation that helps us think. in position paper for hypertext01 workshop on spatial hypertext. papanek, v. (1972). design for the real world. thames and hudson, london. ajis special issue december 2004 132 perry, m. and rosenberg, d. (1998). configuring a co-operative information system theoretical and methodological issues. in proceedings of the iiis'98: workshop on innovative internet information systems(eds, divitini, m., brasethvik, t. and scwartz, d.) pisa, italy. rosenberg, d. (2001). communictive aspects of social intelligence. in jsai-synsophy international workshop on social intelligence designmatsue, japan. schön, d. (1992). designing as reflective conversation with the materials of a design situation. in knowledge-based systems, 5, 3-14. schön, d. a. (1983). the reflective practitioner: how professionals think in action. arena, ashgate publishing limited, aldershot, uk. shneiderman, b. (1991). tree visualization with treemaps: a 2-d space-filling approach. in acm transactions on graphics, 11, 92-99. shneiderman, b. (2002). creativity support tools. in communications of the acm, 45, 116-120. sperber, d. and wilson, d. (1986). relevance: communication and cognition. blackwell, oxford, uk. stappers, p. j. and pasman, g. (2000). exploring databases for taste of inspiration with interactive multi-dimensional scaling. in iea2000. wattenberg, m. (1999a). market map. vol. 2004 www.smartmoney.com. wattenberg, m. (1999b). visualising the stock market. in chi '99 extended abstracts on human factors in computing systemsacm, pittsburgh, pennsylvania, pp. 188-189. weakley, a. j. and edmonds, e. (2004). web-based support for creative collaboration. in proceedings of the iadis interantional conference web-based communities 2004(eds, kommers, p., isias, p. and nunes, m. b.) iadis press, lisbon, portugal, pp. 239-246. wellman, b. (2001). physical place and cyberplace: the rise of personalized networking. in international journal of urban and regional research, 25. yamamoto, y., aoki, a. and nakakoji, k. (2001). time-art: a tool for segmenting and annotating multimedia data in early stages of exploratory analysis. in chi2001, extended abstract. yamamoto, y., nakakoji, k. and aoki, a. (2002). visual interaction design for tools to think with: interactive systems for designing linear information. in proceedings of the working conference on advanced visual interfaces (avi2002)acm press, torento, italy, pp. 367372. yamamoto, y., nakakoji, k. and aoki, a. (2003). a spatial hypertext tool for sculptural authoring. in hypertext 2003nottingham. yamamoto, y., nakakoji, k. and takada, s. (2000). hands-on representations in a two-dimensional space for early stages of design. in knowledge-based systems, 13, 375-384. zhao, q. a. and stasko, j. t. (2000). what's happening? the community awareness application. in chi '00 extended abstracts on human factors in computing systemsacm press, the hague, the netherlands, pp. 253-254. copyright alastair weakley, ernest edmonds © 2004. the authors assign to ozchi and educational and nonprofit institutions a non-exclusive licence to use this document for personal use and in courses of instruction provided that the article is used in full and this copyright statement is reproduced. the authors also grant a non-exclusive licence to ozchi to publish this document in full in the conference papers and proceedings. those documents may be published on the world wide web, cd-rom, in printed form, and on mirror sites on the world wide web. any other usage is prohibited without the express permission of the authors. microsoft word ajis vol13-2 o.doc 7 australasian journal of information systems volume 13 number 2 may 2006 recognition as a distinguishing criterion of is journals john lamp school of information systems deakin university geelong, victoria, australia email: john.lamp@deakin.edu.au abstract the number of journals publishing information systems (is) research has grown dramatically over the past few decades. this has resulted in an environment where authors have a wider choice of journals in which to place articles. electronic journals are now as readily recognised by authorities as print journals. this paper provides firm evidence in support of the assertion that the number of journals publishing is research has increased. the paper also examines the australian context where the selection of a journal in which to place an article is influenced by recognition from the department of education science and training (dest). in australia, obtaining dest recognition as a recognised research journal is not an onerous task, and yet a significant number of is journals have not done this. publishing in a dest recognised journal is essential for australian researchers to contribute to their organisation’s research quantum and hence research funding. attention is drawn to an increasing number of is journals not recognised by dest, and consequent action is recommended. keywords: k.7.m [the computing profession]: miscellaneous; general terms: management, documentation; additional key words and phrases: journals, publish, recognition introduction this paper reports on research which was undertaken to explore two issues. firstly, it presents a definitive study of the year of first publication of journals publishing is research. this study was undertaken to provide clear evidence of the increasing number and rate of entry of is journals, which is often referred to anecdotally within the is community, but has not been published previously. secondly, the issue of recognition of is journals by authorities is examined within the australian context. both the requirements for and process of recognition are described and the extent of recognition of is journals are examined. from this analysis recommendations for authors of journal papers and members of journal editorial boards are made. this paper begins with a review of the use of journals and analysis of journals within the is discipline. it then discusses the issue of recognition of journals by authorities and details the australian context. following that, an analysis is undertaken of the recognition of is journals recorded in the index of information systems journals (lamp 2004). conclusions which may be drawn from this analysis are then presented. australasian journal of information systems volume 13 number 2 may 2006 8 journals and the is discipline readership surveys have shown that the amount of reading by academics of library holdings of journal publications is increasing. this increase is largely attributed to a decline in personal subscriptions and the increased accessibility of journal articles through online bibliographic searching and electronic journal collections. these factors are also believed to contribute to a broader range of journals being read (king et al 2003). also contributing to the broader range of journals being read is the proliferation of electronic journals, which are gaining greater acceptance as an appropriate publication outlet for significant research and are now accepted in indexes such as isi1 and ulrich’s2 as readily as an equivalent paper journal (galvin 2004). this trend has been evident in the is community and was commented on over a decade ago by walstrom et al (1995). the precise impact of these changes is still a matter of ongoing discussion within the is community, eg peffers et al (2003a). a number of studies have been undertaken into the number and quality of journals publishing is research. table 1 presents data from several of these studies on the number of journals publishing is research. study number of journals studied boyer & carlson (1989). 110 (34 refereed) gillenson & stutz (1991). 38 walstrom et al (1995). 26 (based on previous studies) hardgrave & walstrom (1997). 53 (based on 1991 study) walczak (1999). 23 (based on previous studies) whitman et al (1999). 80 (based on previous studies and peer discussions) mylonopoulos & theoharakis (2001). 87 (based on previous studies) walstrom & hardgrave (2001). 51 (based on 1997 study) bharati & tarasewich (2002). 62 (journals publishing ecommerce) katerattanakul et al (2003a). 27 (in citation indexes) katerattanakul et al (2003b). 38 (in citation indexes) peffers et al (2003b). 326 (included magazines as well as journals) lamp (2004) 405 (at time of survey, nov 2004) table 1: previous information systems journal studies the figures presented in table 1 are difficult to compare, as the basis for the various studies varies widely – some focus on particular publication areas (eg bharati & tarasewich, 2002 on ecommerce), others look at particular aspects of journal use (eg walczak, 1999 on relative use by specific research foci within information systems), some are population studies (eg boyer & carlson 1989, mylonopoulos & theoharakis 2001, peffers et al 2003b) where others have used subsets based on a specific form of recognition (eg katerattanakul et al, 2003a, katerattanakul et al, 2003b using journals tracked in citation indices). table 1 by no means lists all the research undertaken on the issue of the identification and relative ranking of information systems journals. a more comprehensive listing of ranking and analysis research can be found in the index of information systems journals (lamp 2004). to address the issue of the lack of any definitive analysis of publication history in the is area, the data in the index of information systems journals was used to compile such an analysis. the primary purpose of the index is to record journals which publish is research, as a guide to authors 1 http://www.isinet.com/journals/ 2 http://ulrichsweb.com/ulrichsweb/ 9 australasian journal of information systems volume 13 number 2 may 2006 seeking publication outlets and, therefore, the scope of the index includes all journals which publish is research. the index was used to discover, from the external links within it, the year of first publication of those journals. it was surprisingly difficult to establish the year of first publication of some journals. only elsevier and idea group explicitly recorded this, and even those publishers’ websites did not record this for all publications. for the remaining cases ulrich’s periodicals directory was consulted. the year of first publication was established for 387 journals. this approach was not able to establish the year of first publication for 14 journals, of which 6 were no longer in publication. it should also be noted that some broadly focused journals, such as the harvard business review, which commenced publication in 1922, were not necessarily publishing is research over their entire history. examining individual journal issues to determine the first publication of is research was beyond the scope of this study but may be undertaken in further research. there is therefore a small unavoidable over statement of available journals. three journals commenced publication between 1922 and 1949 and the balance are presented in figure 1, which highlights the growth in the number of is journals over recent decades. 0 5 10 15 20 25 1950 1955 1960 1965 1970 1975 1980 1985 1990 1995 2000 year n um be r figure 1: journals publishing is research by year of first publication 1950-2004 recognition of journals researchers are strongly encouraged by their affiliated institutions to publish research in highly regarded or prestigious journals. as well as the surveys mentioned above, individual institutions may publish their own lists of journals that they consider should be targeted by researchers (eg gsu 1996). in some institutions the success or failure of an individual academic may be measured by their publication record with these targeted journals. such a simplistic approach is not universally endorsed, and one major survey of quality indicators, the journal quality list, specifically states: the journal quality list is a collation of journal rankings from a variety of sources. it is published primarily to assist academics to target papers at journals of an appropriate australasian journal of information systems volume 13 number 2 may 2006 10 standard. we would be concerned if the list were used for staff evaluation purposes in a mechanistic way. (harzing, 2004) in addition to evaluations by academics as a focus of, or incidental to their research, some governments, including the australian government, specify which journals they consider to be significant outlets. in the case of the australian government, publication in such specified journals is used as one measure of research output. this measure, along with others, is then used to calculate a research quantum which is used to rank researchers and institutions and impacts on the allocation of research funds. future funding, therefore, as well as prestige and promotion, becomes a motivator in selecting a journal for publication. journal recognition in australia the australian government journal recognition arrangements are administered by the department of education science and training (dest). at this point it should be noted that it is possible for australian researchers to seek dest recognition of an individual article, by submission of evidence to establish that the article had undergone a process of peer review (see appendix 2) the focus of this paper is on journal, rather than article, recognition. the requirements for journal recognition are contained in the specifications for the collection of 2004 data (dest 2005). given the volatility of material on websites, the definition of journal articles is reproduced at appendix 1 to this article and the definition of peer review is at appendix 2. according to those sections of the specifications, for a journal article to meet the dest criteria it must: meet the definition of research, be published in a scholarly journal, have been peer-reviewed, have an international standard serial number (issn) dest uses the oecd definition of research, which it cites as: creative work undertaken on a systematic basis in order to increase the stock of knowledge, including knowledge of man, culture and society, and the use of this stock of knowledge to devise new applications. any activity classified as research is characterised by originality; it should have investigation as a primary objective and should have the potential to produce results that are sufficiently general for humanity's stock of knowledge (theoretical and/or practical) to be recognisably increased. most higher education research work would qualify as research. pure basic research, strategic basic research, applied research and experimental development. (dest 2005). the term “scholarly journal” is not defined in the specifications. however, in the section on “peer review” (see appendix 2), there is a set of criteria that define whether or not a journal can be considered to be peer reviewed. in addition to tests that may be applied to individual articles, the journals which fall into the following categories are deemed to be peer reviewed: the journal is listed in one of the institute for scientific information indexes (www.isinet.com/journals) 11 australasian journal of information systems volume 13 number 2 may 2006 the journal is classified as ‘refereed’ in ulrich’s international periodicals directory (volume 5 refereed serials) or via ulrich’s web site www. ulrichsweb.com the journal is included in the department’s register of refereed journals (dest 2005). from these various criteria, it can be seen that, regardless of the content or reviewing process of an individual article, to contribute to their research quantum australian researchers should seek journals which: have an issn and are listed in: one of the isi indexes, or ulrich’s as refereed, or the dest register of refereed journals. obtaining an issn is done by application to the issn national centre in the country in which the particular publication is published, or where no such centre exists, an issn can be obtained through the international centre in paris (issn 2005). in australia the national library of australia administers issns and no charge is made for issuing an issn (nla 2005). journal listing with ulrich’s and dest is by application to those organisations and in both cases no charge is made. listing with isi is subject to different criteria, as isi aims “to provide comprehensive coverage of the world’s most important and influential research” (thomson isi 2004). in order to undertake this they are guided by what they term bradford’s law which stated that the core literature for any given scientific discipline was composed of fewer than 1,000 journals. of this 1,000 journals, there are relatively few with a very strong relevance to the given topic, whereas there are many with a weaker relevance to it. those with a weak relevance to the given discipline or topic, however, typically have a strong relevance to another discipline. thus, the core scientific literature can form itself around various topics, with individual journals becoming more or less relevant depending on the topic. (thomson isi 2004) hence isi confine themselves to tracking approximately 8,700 journals from all fields which they believe accounts for 85% of published research and 95% of cited publications. they recognise that this core is not static and their editorial team focuses on evaluating journals on an ongoing basis to determine their eligibility for membership of this core. while they may evaluate around 2,000 new journals per year, only 10-12% become members of the core, and although it is not explicitly stated in their document, it can be assumed that there is an ongoing rate of exit from their core. publishers may submit journals for evaluation, but inclusion is subject to evaluation by the editorial team. australasian journal of information systems volume 13 number 2 may 2006 12 journals publishing is research at the time of this survey (november 2004) there were 405 journals listed in the index. in addition to determining the year of first publication, the journals were filtered through the requirements for recognition. the result of this analysis is summarised below: 405 journals on index 4 never intended to have refereed status 401 which could potentially be recognised 315 actually recognised by authorities 86 not recognised 8 with no issn 77 with issn 26 with no clear statement on refereeing of papers 51 with no obvious impediment to recognition it should be noted that four publication outlets recorded in the index namely, misq discovery, the computing research repository, interstat, and the isworld working papers series, while significant, are not and were never intended to be refereed publications. of the remaining 401 journals, 315 met the dest criteria (table 3), while 86, over 20%, did not. on dest register 36 in isi index 161 refereed according to ulrich’s 288 table 2: number of is journals recognised by authorities3 of the 86 journals which did not meet the dest criteria, the following eight journals had no issn: applied computing review foundations of information systems: toward a philosophy of information technology international journal of digital evidence journal of behavioural and applied management it & society: an online journal journal of online law idea: the journal of law and technology international journal of information technology education twenty-six of the unrecognised journals did not have clear statements on their websites indicating that double blind refereeing was undertaken by the journal. eleven of those twenty-six have print editions and information on refereeing may be contained in those editions. fifty-one journals would appear to meet the minimum requirements for recognition by dest. seven of those journals were also ranked highly in the peffers et al (2003b) study: communications of the association for information systems data base for advances in information systems electronic markets international journal of accounting information systems 3 an individual journal can be recognised by more than one of the three bodies. 13 australasian journal of information systems volume 13 number 2 may 2006 journal of association of information systems journal of information systems education scandinavian journal of information systems thirty-eight of the fifty-one unrecognised refereed journals appeared to be independent publications, however several major is publishers were represented, including taylor and francis, springer, acm, idea group and inderscience. further more, thirty-three of the fifty-one unrecognised publications were electronic only. a graph of the cumulative numbers of is journals by first year of publication appears in figure 2. recognised journals are distinguished from the total number of journals. there is a steadily widening gap developing between the total number of journals and the number of recognised journals. 0 50 100 150 200 250 300 350 400 450 1950 1955 1960 1965 1970 1975 1980 1985 1990 1995 2000 year n um be r all journals recognised journals figure 2: number of journals publishing is research by year of first publication 1950-2004 (cumulative) conclusions this research has empirically documented that the number of new journals publishing information systems research has grown from less than five titles per year twenty years ago to over twenty titles per year currently. also shown is the growing number of titles, fifty-one at the time of surveying, which are not recognised by ulrich’s, isi or the australian government schemes, despite the apparent lack of any real impediment to their doing so. extremely surprising was the fact that seven journals which are very highly regarded in the is community are not recognised by dest. unrecognised journals limit the number of publication outlets for australian authors and for authors in other countries which may have a similar journal evaluation or recognition scheme where funding or promotion is dependent upon publishing in recognised journals. in an environment of increasing australasian journal of information systems volume 13 number 2 may 2006 14 choice of publication outlets, recognition is a criterion which will be used to choose a journal in which to publish. it has been shown that the requirements for obtaining recognition are not onerous, either in the work required, or cost, and that 51 unrecognised journals could become recognised by the submission of a single form. i strongly suggest that this is an issue that australian information systems journal editorial board members should ensure is attended to during their tenure and as part of their activities as an editorial board member. there would be direct benefits to the journal by increasing their value to authors. there would also be direct benefits to australian authors by increasing the number of publications which contribute to their research quantum. failure to act on this issue will also be to the detriment of the is discipline generally. references bharati, p., & tarasewich, p. (2002). “global perceptions of journals publishing e-commerce research”. communications of the acm, 45(5), 21-26. boyer, g., & carlson, g. (1989). “characteristics of periodical literature for the potential reader or author in information management”. mis quarterly, 13(2), 221-229. dest (2005) specifications for the collection of 2004 data [online] available: http://www.dest.gov.au/highered/research/documents/specs2005.pdf accessed: 18/03/2005 galvin, j. (2004). “the next step in scholarly communication: is the traditional journal dead?” electronic journal of special and academic librarianship, 5(1), 1-9. gillenson, m. l., & stutz, j. d. (1991). “academic issues in mis: journals and books”. mis quarterly, 15(4), 447-452. gsu. (1996) gsu target journals. georgia state university. [online] available: http://www.cis.gsu.edu/~dstraub/research/targetjournallist/target%20journals.htm accessed 14/04/2005. hardgrave, b., & walstrom, k. (1997). “forums for mis scholars”. comm. acm, 40(11), 119-124. harzing a (2004) journal quality list [online] available: http://www.harzing.com/jql.htm accessed: 14/04/2005 issn (2005) getting or checking issn numbers [online] available: http://www.issn.org:8080/english/pub/getting-checking accessed 20/05/2005 katerattanakul, p., han, b., & hong, s. (2003a). “objective quality ranking of computing journals”. comm. acm, 46(10), 111-114. katerattanakul, p., razi, m., & kam, h. (2003b). “is journal rankings versus citation analysis: consistency and concerns”. paper presented at the ninth americas conference on information systems, tampa, florida. king, d. w., tenopir, c., montgomery, c. h., & aerni, s. e. (2003). “patterns of journal use by faculty at three diverse universities”. d-lib magazine, 9(10), 1-10. lamp, j. w. (2004). index of information systems journals. geelong, deakin university. [online] available: http://lamp.infosys.deakin.edu.au/journals/ accessed 12/04/2005 mylonopoulos, n. a., & theoharakis, v. (2001). “global perceptions of is journals: where is the best is research published?” comm. acm, 44(9), 29-33. nla (2005) the australian issn agency [online] available http://www.nla.gov.au/services/issn.html accessed 20/05/2005 15 australasian journal of information systems volume 13 number 2 may 2006 peffers, k., avison, d., ein-dor, p., & zmud, r. w. (2003a). “scholarly products in is: will advances in electronic media promote evolutionary or radical change”. comm of the ais, 11, 498-512. peffers, k., & ya, t. (2003b). “identifying and evaluating the universe of outlets for information systems research: ranking the journals”. j information technology theory and application, 5(1), 63-84. thomson isi (2004) the isi® database: the journal selection process [online] available http://www.isinet.com/essays/selectionofmaterialforcoverage/199701.html/ accessed 20/05/2005 walczak, s. (1999). “a re-evaluation of information systems publication forums”. j computer information systems, 40(1), 88-97. walstrom, k. a., hardgrave, b. c., & wilson, r. l. (1995). “forums for management information systems scholars”. comm. acm, 38(3), 93-107. walstrom, k. a., & hardgrave, b. c. (2001). “forums for information systems scholars: iii”. information & management, 39, 117-124. whitman, m. e., hendrickson, a. r., & townsend, a. m. (1999). “research commentary. academic rewards for teaching, research, and service: data and discourse”. information systems research, 10(2), 99-109. appendix one: australian government definition of journal articles 11.9.3. journal articles to be included in this category the journal article must meet the definition of research (see 2.4) as amplified in the key characteristics of research publications (see section 11) and: must be published in a scholarly journal must have been peer-reviewed as defined at 11.6 (note that the fact that an article has been peer reviewed does not automatically mean that it is eligible. the article must still meet the definition of research as well as all other criteria) must have an international standard serial number (issn). some journals may be regularly published as separate volumes with an isbn rather than an issn. provided that the publication is clearly identified as an edition of a journal, and not a book, articles in such publications may be eligible if they meet all other criteria. if an issn does not appear in the journal, external evidence such as an issn number being cited in an extract from ulrich’s or isi is sufficient to demonstrate the existence of the number the author must be affiliated with the claiming hep. the types of journal articles that may meet the criteria include: commentaries and communications of original research research notes letters to nature critical scholarly texts which appear in article form articles reviewing multiple works or an entire field of research invited papers in journals articles in journals which are targeted to both scholars and professionals articles in a stand alone series. the types of journal articles that are unlikely to meet the criteria include: australasian journal of information systems volume 13 number 2 may 2006 16 letters to the editor case studies articles designed to inform practitioners on existing knowledge in a professional field articles in newspapers and popular magazines editorials book reviews brief commentaries and communications of original research reviews of art exhibitions, concerts, theatre productions. appendix two: australian government definition of peer review 11.6. peer review for the purposes of the herdc, an acceptable peer review process is one that involves an independent, expert review. the peer review process must involve assessment of the publication: in its entirety – not merely an abstract or extract before publication by appropriately independent, qualified experts. independent in this context means independent of the author. for journal articles, any of the following are acceptable as evidence: the journal is listed in one of the institute for scientific information indexes (www.isinet.com/journals) the journal is classified as ‘refereed’ in ulrich’s international periodicals directory (volume 5 refereed serials) or via ulrich’s web site www.ulrichsweb.com the journal is included in the department’s register of refereed journals (see 11.10) there is a statement in the journal which shows that contributions are peer reviewed there is a statement or acknowledgement from the journal editor which shows that contributions are peer reviewed a copy of a reviewer’s assessment relating to the article. note: a statement from an author that a publication was peer reviewed will not be accepted. the existence of a national or international advisory board is not sufficient evidence that all relevant publications are assessed by members of it. microsoft word 13 1 marina.doc ajis vol 13, no. 1 september 2005 263 a pda based point of care e-health solution for ambulatory care daniel walsh, carole alcock, lois burgess university of wollongong joan cooper flinders university abstract the adoption of pdas and mobile communication is expected to provide a solution to the use of computer technology by healthcare workers at the point-of-care. the australian national health information strategy, health online, is providing national leadership for approaches to address the quality and availability of information to assist in the planning and delivery of care. one area for potential growth is the availability and capture of information at the point of care by healthcare providers. a key factor in the lack of adoption of systems, is that traditionally health care information systems have been designed for desktop computing whereas many healthcare workers are highly mobile. this paper discusses phase one of a larger, four-phase project which aims to develop information access applications at point-of-care for ambulatory care services. the initial phase of the research (phase one) involves workflow analysis, requirements specification and the development and testing of a system prototype to assess the feasibility of achieving increased efficiencies in workflow at the ambulatory care service. keywords: pdas, e-health, mobile communications, workflow analysis, requirements specifications, prototypes introduction effective healthcare delivery within community based health services depends on efficient information access (nsw health information management and it strategic plan, 2001). the aim of this is paper is to present work in progress comprising the initial phase of a larger, four phase project to develop and evaluate a point-of care application for ambulatory care services to facilitate more effective delivery of healthcare. the full project involves the design and development of a personal digital assistant (pda) based delivery system that utilises generic, reusable and scalable components. a pda based point-of-care system is significant in that it provides for collection, delivery and exchange of timely information (both text and images) at the point-of-care, leading to a more efficient health care system. a more efficient system will overcome identified issues with existing systems by reducing the number of errors in transferring data between healthcare professionals, reducing the time spent entering and recording data (american health consultant, 2002) and providing for greater reliability, robustness and ease of use. the objectives of the four phase project are: • production of detailed workflow diagrams through analysis of workflows in ambulatory care services (illawarra ambulatory care). • knowledge of, and solutions to, work practice issues associated with the implementation of pdas in mobile 22 healthcare situations. 22 mobile healthcare refers to any healthcare worker working within the community providing healthcare services outside the hospital or clinical setting, eg in the home. ajis vol 13, no. 1 september 2005 264 • development of a suite of middleware applications for interface software suitable for point-of-care access to databases, knowledge-repositories and directories. • development of generic, scalable and re-usable object oriented software components that are easily transferable to similar healthcare applications and compatible with chime 23 (community health information management enterprise) (britt & miller, 1997). • development of a security, authentication and privacy framework (including policies and procedures) for ambulatory care information systems (acis). • demonstrate that computer-based information (both text and images) capture and retrieval can be integrated into point-of-care practice. • integration of work practice issues, scalable software and security protocols into a successful prototype pda based healthcare delivery system. • provide accurate, relevant and timely access to patient management information by gps. current systems at ambulatory care are paper based and are limited to what the healthcare worker can effectively carry. the proposed system will provide for collection, delivery and exchange of timely information at the point of care. the key advantages of the system will be its high mobility and flexibility in matching complex healthcare workflow requirements as well as immediate updating of healthcare records. such a system has the potential to save people’s lives or at least significantly improve their health outcomes by responding more quickly and with more appropriate action. this will result in better outcomes for the patient and the healthcare provider in terms of improved healthcare, better workload management and greater efficiency and effectiveness. this paper presents the results of a prototype pilot that was developed from an initial workflow analysis and requirements specification. the pilot was developed to ascertain whether the efficiencies identified from the workflow analysis stage of the project and requirements specifications could be achieved through the use of mobile technologies deployed at the point of care. background information systems for healthcare have been available for some decades. however, the lack of the adoption of such systems by healthcare workers around the world has limited the realisation of benefits of health information and communication technologies (ict). it leaves healthcare exposed to problems associated with paper-based exchanges, handwriting, lack of integration of information and limited availability and capture of information at the point-of-care (soar and yu, 2002). a key factor in the lack of adoption of these systems is that they are normally designed for desk-based users, whereas, many healthcare workers are highly mobile (burgess and cooper, 2001). the new technologies of hand-held devices (note-books, pdas and tablets), bluetooth, 802.11 wireless and broadband cdma/3g are changing this. according to versel (2002), pdas appear to be 23 chime is the information management system being adopted by nsw health. ajis vol 13, no. 1 september 2005 265 the driving force behind the increased rate of adoption of ict by healthcare workers. there is however, a multiplicity of issues that need to be resolved before healthcare can take full advantage of these innovations. issues that need addressing include the development of models and principles aimed at advancing the knowledge base for mobile healthcare services generally, as well as specific software application solutions. research approach the research was conducted in four stages: workflow analysis, requirements specification, prototype development and testing and requirements specifications and workflow refinement. the objectives were: • to gain a complete understanding of the current workflows of the illawarra ambulatory care unit • to construct workflow models that represent the activities of the ambulatory care service • to analyse the current workflows of the service to identify the initial requirements of the ambulatory care pda system prototype • to identify problematic or inefficient areas in the workflow • to develop and test a prototype • to refine workflow and requirements specification workflow analysis if a transition to a mobile technology based system is to be successful, an analysis of existing workflows is an essential element. consequently, objectives in this phase of the research are workflow focused. workflow analysis provided for successful definition of user needs and requirements (kirwan & ainsworth (1992). task analysis was also used to model the general activity of the ambulatory care staff at work, the processes of their interaction with the system and the processes occurring in the whole socio-technical system (mancini et al, 2002). during workflow and task analysis, special attention was given to the wide range of worker perspectives and expectations, to issues such as patient registration, clinical records and notes, patient care plans, and access to reports, patient correspondence, appointment management and communications. analysis of these elements was essential to maximise the value of the prototype system design. the analysis used in this study was based on observation, in-depth interviews and examination of existing workflow documents. workflows are closely tied to requirements. kirwan & ainsworth (1992) argue that workflow analysis is a key method for the successful definition of user needs and requirements. the importance of workflow analysis is emphasised by lowry (1997), who asserts that workflows impact on the design, which, in turn, impacts the workflows, and requirements are incorporated within these workflows. consequently, the execution of the workflow analysis of the illawarra ambulatory care service played a pivotal role in the construction of a requirements specification for the prototype. the workflow analysis identified a number of areas of operational inefficiencies in the task set comprising the workflow model. major tasks identified include processes related to 1) patient admission, 2) pre-patient visit, 3) patient visit, 4) post patient visit and 5) patient separation from the service. from this set, tasks 1, 2, and 3 were mapped to the prototype. ajis vol 13, no. 1 september 2005 266 requirements specification a number of focused interviews with ambulatory care staff identified the initial requirements of the ambulatory care point of care (poc) system during the workflow analysis stage. this enabled identification of a potential solution for the operational inefficiencies in the existing workflows. these solutions form a vital part of the requirements specifications, defining important functional requirements of the prototype. in addition, staff at ambulatory care have identified a number of additional specific requirements they believe to be critical to the successful implementation of the poc system. an initial requirement list was developed from the workflow analysis and discussed with ambulatory care staff for verification. an analysis of the initial requirements list enabled the construction of the initial requirements specification document. this guided the development of the system prototype and provided a basis from which the final requirements specification was constructed. development and testing of prototype the prototype was developed and implemented to identify possible errors in the requirements specification and to stimulate further feedback from ambulatory care staff. it is to be noted that it was not intended that the final system would be modelled on the prototype. rather, the results from the prototype and the final requirements specification will provide a foundation from which the final system is developed. the prototype system did not fully replace existing systems, but ran in parallel; hence, there could be no accurate assessment on the systems effect on the efficiency of the entire workflow. however, by implementing the prototype system in specific areas of the workflow, system functionalities could be tested. this process allowed ambulatory care staff to accurately test and verify efficiency gains achieved (for individual workflow tasks: that is, tasks 1, 2, and 3), through the implementation of the prototype system. ambulatory care staff were observed operating the system prototype in the three specific areas of the workflow including: • patient admission data-entry process • patient visit (point of care) data-entry process • patient separation data-entry process in order to automate the three processes listed above, hospital and community health databases containing patient data required by ambulatory care were simulated on the prototype system. this process facilitated simulation of database communication during the prototype implementation, including: • the process of importing patient data from the hospital database during the admission process • the uploading of patient data to the community health database during the patient admission and discharge (separation) process • the process of recording patient notes and observations at the point of care during the patient visit • the process of retrieving historical patient episodes to acquire patient's medical background at the point of care during the patient visit ajis vol 13, no. 1 september 2005 267 observing specific workflow activities allowed mapping of prototype-integrated workflows to the paper-based data-entry workflow. efficiency gains achieved through the implementation of the system were more accurately measured by this process. the majority of workflow areas identified as inefficient were captured in the overall data-entry process. consequently, the majority of the requirements and functionalities identified during the workflow analysis and requirements specification stages focus on achieving efficiency gains in the data entry process. these functionalities were implemented in the prototype system to allow their impact on the ambulatory care service workflow to be assessed. results and discussion the initial phase of the research revealed that the implementation of a pda based emr system at the ambulatory care service will enable more medical and patient information to be mobilised to the point of care. these findings are consistent with existing research, in particular ammenwerth et al.(2000), versel (2002), and anderson (2000) who suggest that “computer based information systems can increase access to clinical information” (anderson, 2000). this research phase also concludes that current centralised emr systems available to the service are not used as the primary database because they do not support ambulatory care staff at the point of care. the prototype implementation further revealed that the implementation of an electronic system would facilitate better communications with external medical databases, a view also held by anderson (2000). the results of the prototype implementation also indicate that the implementation of the ambulatory care pda system will support critical workflow activities, address the operational inefficiencies of the ambulatory care service (by automating and streamlining workflow activities) and increase access to patient and medical information at the point of care. a number of other benefits accrued from the prototype implementation. firstly, achievable time saving gains through the simulation of data entry and retrievable processes. as well, additional requirements including: data entry audit controls, patient allergy reminders, data formatting standardisation, patient record search (database query), gui design, data sharing interface, patient notes, controls and search mechanisms, usability issues, and changing and unpredictable data set. the identification of these additional requirements led to refinement of the requirement specification document, providing a more accurate reflection of the requirements of the final pda system. conclusion the application of existing workflow analysis methodologies resulted in the ambulatory care service workflow being accurately captured, described and documented with the identification of workflow sub-tasks and the creation of workflow diagrams. the analysis of these workflow diagrams enabled the initial requirements of the ambulatory care prototype pda system to be identified. analysis of the workflow models also allowed inefficient or problematic areas of the ambulatory care service workflow to be identified. through the development and implementation of the prototype system, requirements of the final ambulatory care pda system, that will assist in resolving operational inefficiencies, have now been identified. results of this phase of the project also highlighted requirements that leverage new pda technologies to add value to current workflows, specific requirements aimed at overcoming the limitations of paper based medical record systems, and functional requirements of the ambulatory care service. future phases of this project ajis vol 13, no. 1 september 2005 268 will build on this foundation research to achieve a high quality point of care ehealth application for ambulatory care. references american health consultant ”pda’s help nurses improve care and save time: devices previously used only by physicians” healthcare benchmarks and quality improvement, october, 2002:v.9(10): 44(3) ammenworth, w. et al. mobile information and communication tools in the hospital. international journal of medical informatics. 57 (2000):21-40. anderson, j. computer-based ambulatory care information systems: recent development. journal of ambulatory care management. 23.2 (2000):53-60. britt, h and miller, g “ recent developments in information management for primary and community health services” health information management 1997 v26 (4): 193197 burgess, l. and cooper, j (2001) “ the potential of wap in mobile healthcare delivery in rural and remote communities in australia” e-business review vol. 1 journal of the international academy of e-business. kirwan, b and ainsworth, l (1992) “ a guide to task analysis” taylor and francis, london lowry, j (1997) “workflow analysis” available: http://edhs1.gsfc.nasa.gov/waisdata/workshops/clientwk106.pdf 18 december 2002. mancini, g, marti, p, palmonari, m and rizzo, a (2002) “task analysis and workflow methodology for cooperative work settings” (url: www-sv.cict.fr/cotcos/pjs/method…analyismethods/analysispapermancini.htm/) new wales health. information management division. “information management and technology strategic plan” december, 2001. soar, j and yu, p “the future of ehealth through next generation wireless and mobile connectivity” proceedings health informatics conference, hic2002, melbourne, august, 2002 versel, n. "wave of the (not-so-distant) future: annual healthcare it survey shows rise in technology adoption." modern physician 6.11 (2002) microsoft word ajis10-2finala.doc ajis vol. 10 no. 2 may 2003 49 e-commerce experiences in the real estate industry. a preliminary study in regional queensland. diana gray jeanette van akkeren faculty of business university of the sunshine coast sunshine coast, queensland, australia email: dgray@usc.edu.au abstract computer systems have become commonplace in most smes and technology is increasingly becoming a part of doing business. in recent years, the internet has become readily available to businesses; consequently there has been growing pressure on smes to take up e-commerce. however, e-commerce is perceived by many as being unproven in terms of business benefit. this research aims to determine what, if any, benefits are derived from assimilating e-commerce technologies into sme business processes. this paper presents three in-depth case studies from the real estate industry in a regional setting. overall, findings were positive and identified the following experiences: enhanced business efficiencies, cost benefits, improved customer interactions and increased business return on investment. keywords: electronic commerce, experiences, smes, critical success factors introduction computer systems have become commonplace in most smes and technology is increasingly becoming a part of doing business. in recent years, the internet has become readily available to businesses; consequently there has been growing pressure on smes to take up e-commerce. however, e-commerce is perceived by many as being unproven in terms of business benefit. those who have undertaken e-commerce projects have had mixed results, while many are still waiting for e-commerce to demonstrate its business worth. this study concentrates on real estate agencies as many are smes and are abundant on the sunshine coast. as the ultimate focus of this research is the experiences of smes in regard to e-commerce, it is logical to specifically select those real estate agencies who are using the internet for business purposes, rather than those who are not. while this may give a somewhat biased result, the purpose of the research, which is investigating the experiences of e-commerce, should be fulfilled. literature review there are many affirmations that businesses should be educated on the potential of e-commerce and that managers need to become more involved in formulating and implementing e-commerce strategies (lawrence et al 2000, kalakota and whinston 1997, timmers 1999). smes constitute over 97 percent of nongovernment businesses in australia (abs 2000), consequently they contribute significantly to the economy of australia and provide a large proportion of the public sector employment. to assist in the understanding of the workings of electronic commerce and its applicability to smes, models of the activity have been examined. the current e-commerce frameworks and/or models in the literature seem to come from differing perspectives presenting a range of varying elements. chan, e. and swatman (1999) have developed a dynamic model of e-commerce, representative of all e-commerce types, activities and capabilities that demonstrate its performance in product and service terms. the complexity of the framework is modelled into a fluid representation of the varying viewpoints of the stakeholders as displayed in figure 1. infrastructure e-commerce definition services legal source: chan, e. and swatman (1999) figure 1: the meta view of e-commerce the meta view level is made up an e-commerce definition surrounded by three components containing various objects: infrastructure (technical), comprising: databases, edi, isp, security, web site, smart card devices, ajis vol. 10 no. 2 may 2003 50 development multimedia, communications/telecommunications/network technologies, internet/intranet/extranet. legal, comprising: government policy, ethics/computer crime, privacy, copyright, legal issues. services, comprising: eftpos, eft, e-publishing, e-catalogues, information kiosks, on-line shopping and education, other internet commerce activities (chan, e. and swatman 1999). the three basic components: ‘infrastructure’, ‘services’ and ‘legal’ comprise the framework of e-commerce. the boundaries are fluid; subsequently the size of each area is determined by the specific views of the parties involved. any alteration of the borders does not affect the actual weight of the components within the model. the model also allows for changes over time as objects may be added to or removed from the basic components. while the authors state that this model needs further refining, it appears to be most effective at modelling the components, structure and dynamic nature of e-commerce. the following section details previous studies on e-commerce experiences. ecommerce experiential factors a search of the literature has revealed a number of empirical studies detailing the experiences of smes once they have implemented an e-commerce initiative. these results are from australian studies, with the exception of golden and griffin (ireland) and adam and deans (australia and new zealand). the studies examined have determined that there are a number of common factors contributing to the experiences of smes and their e-commerce projects. experiential factors – difficulties a number of difficulties have been experienced by smes after they have initiated their e-commerce projects. these difficulties begin with ineffective planning in the e-commerce adoption stages and progress into problems with website consultants, difficulties determining the extent of internet involvement in customer interactions, and a lack of immediate roi. the first experiential factor is that of ad hoc adoption. a majority of the studies claimed that the smes investigated instigated their e-commerce initiatives using a relatively ad hoc method (bode and burn 1999, boon and hewett 2000, castleman and cavill 2001, chappell and feindt 1999, golden and griffin 1998, marshall et al 2000, poon and swatman 1999, singh and fisher 1999). consequently, most of these businesses either had no business plan or did not include e-commerce strategies in any existing business plans, nor did they re-engineer business processes for e-commerce integration. another issue for smes was dissatisfaction with website consultants. there was general dissatisfaction with web site design consultants, with many smes feeling that they were pressured into a site that was not a planned business initiative (marshall et al 2000). lawrence (1997) noted that external consultants were perceived as a ‘necessary evil’ who gave inappropriate and ambiguous advice and did not provide satisfactory follow-up service. in the bode and burn (1999) investigation of sme experiences with website consultants, the organisations felt that their individual business needs were not met. this is supported in the study by golden and griffin (1998) where 30 percent of smes stated that they were not satisfied with their website designer. indetermination of internet involvement was another factor identified. determining if e-commerce has increased profits, reduced expenses or attracted new customers has proved problematic for many smes. generally, businesses had not ascertained if their customers had used the internet to gain information regarding the business or its products before they approached the company (marshall et al 2000, poon and strom 1997). poon and strom (1997) explained that businesses were not able to keep records on how much the internet increased sales or brought in business due to the difficulties of ascertaining causal relationships. finally, lack of return on investment (roi) was identified by a number of the previous studies as a problematic area. they report that smes who have undertaken e-commerce initiatives have not experienced any immediate return on their investments (chau and pederson 2000, golden and griffin 1998, marshall et al 2000, poon and strom 1997, poon and swatman 1999). poon and swatman (1999) report that internet sales and on-line transactions were largely disappointing for the organisations in their study. it has been recognised that many smes translated perceived benefits into competitive advantage (drennan and kennedy 2000, poon 2000, poon and strom 1997, yellow pages 2000). this could be explained by the fact that e-commerce, being a new mode of business, is not utilised by every organisation. therefore, when an sme experiences benefits, they feel they have an advantage over their non-e-commerce compliant competitors. the benefits experienced by smes also include enhanced communications and improved customer and business interactions. many of these factors were also identified as initial facilitators to ecommerce adoption. experiential factors benefits ajis vol. 10 no. 2 may 2003 51 many of the organisations felt that the internet had provided a strategic advantage by allowing them to be first movers or to keep ahead of customer and supplier demands (golden and griffin 1998, marshall et al 2000, poon and strom 1997). additionally, e-commerce implementation was seen as a necessary prerequisite by smes to avoid being left out if large corporations and administrations required electronic access (singh and fisher 1999). smes in general seem motivated to take up e-commerce to avoid being left out. perceived by smes as a facilitator to e-commerce adoption, access to information and communications were felt to be factors that had been enhanced through e-commerce. it was considered that the internet provided an efficient information resource to access competitors’ sites, obtain supplier information and research new markets (golden and griffin 1998, marshall 2000, poon 2000, poon and swatman 1999). enhanced communications through e-mail and edi resulted in cost savings and allowed time and geographic limitations to be overcome through the ability to conduct asynchronous communication (o’brien and van akkeren 2000, poon and strom 1997, poon and swatman 1999). additionally, organisations studied found that e-commerce benefited their customer interactions. they believed that the internet lowered costs and improved responsiveness as it increased organisational efficiency and produced faster reaction times to customer needs, particularly in processing orders and correspondence (chau and pederson 2000, yellow pages 2000). also identified was the benefit to business reported to be the ability to establish a network of similar businesses that can support each other (o’brien and van akkeren 2000, swatman 2000). while support between businesses can be beneficial, as revealed in the section dealing with inhibiting factors, many smes are reluctant to share knowledge or information as it is felt that this could threaten their competitive advantage (love et al 1999). finally, despite the difficulties experienced by many organisations, most seem firmly convinced that the internet has potential for their businesses in the future (chau and pederson 2000, golden and griffin 1998, marshall et al 2000, poon and strom 1997, singh and slegers 1999). while many smes had experienced marginal savings or earnings from the internet, they resolved to continue their internet use even though it was difficult to justify in terms of immediate roi. critical success factors in comparing the experiential factors with the critical success factors, one particular issue from both areas is mentioned by a majority of authors. most businesses seem to approach e-commerce projects in a somewhat ad hoc manner with no clear e-commerce strategies. many adopt e-commerce, but do not incorporate it into the processes of the business. as golden and griffin (1998) observed, in hindsight many smes regretted the omission of e-commerce from business processes. hence the most noted critical success factor recommends that if an organisation is committed to an e-commerce project, it a greater chance of success if e-commerce is integrated into business practice. other success factors included the encouragement of customers or business partners to use e-commerce, a unique or innovative product or service will be more successful, and that support can be obtained from other like-minded businesses. the research suggested that competent use of technology combined with an effective web site would ensure that the expenditure on these facilities was justified. if smes are aware of these factors, their e-commerce initiatives may be more likely to eventuate in financial benefit and enhanced customer interactions. the authors of the studies reviewed conclude that had the businesses they researched been aware of the various factors critical to success, the businesses would have had greater chance of success in their ecommerce ventures. one of the aims of the current study is to determine if this is in fact the case. research design this study involved the use of interpretive research methods to conduct hermeneutic field studies with the aim of discovering participant’s beliefs, actions and experiences with e-commerce in their place of work, that is, real estate agencies. benbasat, goldstein and mead (1987) advocate the use of a case research strategy for studying information systems in a natural setting to gain an understanding of the nature and complexity of the situation and to research an area with fewer previous studies. walsham (1995) argues that, particularly in interpretive studies, outside observers derive their information chiefly from interviews. he states that interviews are the richest form of case study data as they provide the most comprehensive coverage of the of participants’ interpretation of actions and events in the past or present and the views and aspirations of themselves and others. given the objective of this research, real estate agencies that utilise some form of e-commerce were selected to ensure that the owner/managers of these organisations could provide some comment on their experiences with conducting business over the internet. real estate agencies were selected as the specific field of research for this study as this industry has adopted e-commerce with relative enthusiasm. many utilise not only e-mail, ajis vol. 10 no. 2 may 2003 52 but also have a web presence. this is important to the research as the experiences of smes since their adoption of e-commerce is a prime focus of this investigation. rich data was collected from the organisations through conducting multiple case study research in the form of field studies. interviews were conducted with three real estate agencies to gain insight into their use of e-commerce. the interviews commenced with broad questions. as the interview progressed, participants were prompted to provide information according to a pre-defined topic list. as eisenhardt (1986) recommends, specifying constructs a priori assists in the more accurate measurement of concepts. the interviews aimed at surfacing issues relevant to the individual organisations, thereby supplying rich data that is provided freely and spontaneously by the participants (sekaran 1992, moutinho et al 1999). as advocated by walsham (1995) the interviews were tape recorded on a micro cassette recorder (with the permission of the participants), thus providing a full account of all that is said during the interview. the recordings were supplemented with notes, which included important points or additional observations or impressions taken by the researcher. once the data has been collected, it was examined for themes, patterns and meanings within and between the organisations. findings this section analyses the information collected from the three interviews. these interviews assisted in providing preliminary findings by raising pertinent issues. three real estate organisations with differing types of web presences were selected. the aim of this approach was to facilitate the development of a comprehensive course of action for the main phase of the research. to assist in conducting a clear discussion and to comply with ethical guidelines, the cases will be referred to as case a, case b and case c. these field studies are briefly described in table 1. field stu dy services years business in operation years of current owner/mana ger number of employ ees type of web presence casea property sales, rentals 52 5 9 own domain web site caseb property sales, rentals 29 2.5 44 franchised agency group sub-site casec property sales, rentals 30 6 5 real estate portal sub-site table 1: field study details these interviews occurred from september to october 2001 on the sunshine coast in queensland. the interviews were conducted with the owners/managers of the three real estate agencies. all the interviews were approximately one hour in duration and were taped on a micro cassette recorder. the cases are discussed in line with the areas of research, that is: e-commerce experiences and e-commerce success contributors. preceding these points for each case is a brief description of the organisation. case a – own domain web site case a embarked on its internet presence with a one-page web site with its own domain name, containing contact information and email links. one year ago, this was upgraded to quite a comprehensive web presence, where clients can access a wide range of information through numerous pages within the site. the property listings on the site are updated daily and take the form of sales and rentals (both residential and commercial). these are accessed through an internal link within the frame of the agency’s site to realestate.com, a portal with comprehensive real estate search facilities. the site adds value for the customer in the form of a property search facility, loan calculator and information about the sunshine coast. the site also contains email links and a form where clients can electronically list a property with the agency. negative experiences with e-commerce. the owner expressed a discontent with outside consultants. he felt that there are many consultants in the industry who cannot be trusted: ajis vol. 10 no. 2 may 2003 53 “there is an enormous amount of ‘shonks’ in this industry” he stated that often the business does not always get what it wants or needs from these consultants. the business, to date has not experienced a return on its initial investment. however, the owner stresses that the cost of maintaining the site each year was not a large financial burden as it is roughly the profit on only one sale (approximately $5000). moreover, the owner firmly believes in the future potential of his e-commerce project: “judging by the last six months, by the end of this year it will be returning dividends.” positive experiences with e-commerce. the owner maintained that e-commerce has greatly enhanced the efficiency of his business and had delivered cost-benefits and time-savings by providing instant access to on-line resources such as property information and tenant checking. he stated: “that’s one way our office has changed for the better, it’s more streamlined. the efficiencies are paying off. we can source the information we need, start a task, get the information, finish the tasks, put it to bed.” the agency has gained new customers, some international, through the greater market penetration facilitated by the web site. the owner believes that the web site has enhanced customer interactions through allowing customers to conduct their own research on available properties. customer relationships have also been enhanced by the agency replying to emails immediately, with customer feedback such as: “i got onto twenty agents, you’re the only guy who got back to me.” e-commerce success contributors. the e-commerce adoption facilitator of customer demand ensured that the agency had a critical mass of customers to provide their service over the internet. the owner’s determination to persevere with ecommerce despite having setbacks with it/web site contractors is undoubtedly one of the main contributors to the relative success of the e-commerce venture for this agency. an additional facilitator to the success of the e-commerce venture was the owner’s strong motivation and commitment to e-commerce: “if i’m going to do it, i’ll make sure that it works.” it appears that the owner’s high confidence level with technology and the internet also helped in this respect as he spent a great deal of time working out problems himself and then training the staff: “there is a wealth of information. i am going to control it, it is not going to control me.” the owner clearly recognises the value of information to the firm and how being in control of it can enhance his business. a further contributor to the success of the e-commerce initiative has been the incorporation of e-commerce into the processes of the business. this is evident through the use of the on-line property and tenant checking systems and the use of the internet for research. additionally the owner feels that producing a high standard of e-commerce presence has encouraged customers to continue to use this agency’s e-commerce and which thus ensures its continued success. interestingly, the owner stated that he had not had a formal plan or specific strategy for his e-commerce initiative. case b – sub-site of franchised agency group overview and status. case b, as part of a franchised real estate group, has a two-page web presence located as a sub-site of the parent organisation’s site. the site, established in 1994, contains a search facility that can link to properties specifically listed with this agency or to the wider franchise group properties. the site is accessed through the parent company’s web site and is not easy to locate. the agency does not have its own domain name. the sunshine coast offices have forty-four staff, most with computers and internet access. they utilise internet technologies such as email, internet banking, accessing legislation, the australian bureau ajis vol. 10 no. 2 may 2003 54 of statistics, the tenants registry (ticker) the parent company website and secure site for updating. the agency handles commercial and residential sales and rentals. negative experiences with e-commerce. the owner feels that with regard to e-commerce, that: “there’s a lot more down sides than positives.” when asked if he had experienced a return on his investment, the owner replied: “absolutely not. it’s only because its high maintenance, it’s so prolific the way it grows, the way it changes... every six months. for us its probably just been an extra $30,000 $40,000 cost.” the owner explained that due to e-commerce, there are problems with staff technical skills: “it’s because the computer programs in any sort of business have become so specialised, when you have a girl leave or go on holidays, it’s an extra cost to having someone come in who understands that particular program.” the agency does get email enquiries initiated from their internet presence. however, the owner states that regarding daily property enquiries, they may get twenty phone calls, but only five emails. the owner does not view this as creating more business for his agency, but states that this demonstrates that a web site is no more effective in generating interest than traditional methods. he felt that the web site played only a small role in the marketing of the agency: “having properties on the internet, to my mind, is just like having another sign in the marketplace, having another ad in the paper or having another window card.” real estate, according to the owner, is not really suited to sales over the internet: “we get a lot of enquiry off the internet but we don’t make a lot of sales from it.” the owner felt that it was hard to determine if a sale was due to the client initially accessing property information from the web site. he likened it to a customer finding a property advertised in the newspaper, visiting the open house and deciding to buy the property – the sale could not be attributed to any one factor. “initially, it makes it easier for them to access us but at the end of the day, when they’re going to buy property, they’re going to be here in person anyway.” like case a, this agency also has weekly approaches from web site development contractors about setting up a site. this agency had not had any feedback from customers, nor had the agency asked for any feedback. positive experiences with e-commerce. the owner nominated a number of benefits, in terms of customers, to having e-commerce: “on the sales side of things, we’re probably accessing more of a market.” the agency has also been able to enhance customer interactions by emailing details and photos of rental property inspections to owners who are not in the area. “i think at the end of the day people will find you but it has made it easier for us to retain people from overseas.” the owner was convinced that internal business operations was where e-commerce was most worthwhile, but continued to state that the benefits of e-commerce were outweighed by its expenses and disappointments: “definitely on the business side of business, is a lot easier to just have that one girl being able to handle the pays and that sort of thing. but it costs to have those programs as well. ” ajis vol. 10 no. 2 may 2003 55 the owner believes that the internet has made researching information more straightforward. however, the owner considers that the expense incurred has not justified the outlay: “it’s great for research, and it’s probably easier, but there is a price to be paid for being easy. it’s much the same as when mobile phones came in. all they did was double the phone bills of businesses.” e-commerce success contributors. the owner felt that to be successful in e-commerce, it was important to be cautious and not take on new technology until it is thoroughly tested. this belief is clearly at odds with the first mover philosophy of the company ceo. “be careful not to be the cutting edge. it’s good to be at the cutting edge. it’s good to have all the technology. but don’t be out there experimenting with things.” he felt that the success of his e-commerce project was obvious in the internal functions of the organisation and in enhancing b-2-b interactions, rather than in sales or marketing: “i think it’s been a success more from running the business as opposed to promoting the business and creating the business. inside the business it’s been great, there’s no doubt about that”. ultimately, the owner felt that the main benefit of e-commerce for his organisation was through facilitating internal business operations. the owner believed that e-commerce proved to be worthwhile and valuable in the area of increased business efficiency. however, he believed that these benefits were outweighed by the expense, extra work and inconvenience of e-commerce. case c – sub-site of real estate portal overview and status. case c is an agency with five staff and is run by the owner. the agency has been operating for thirty years; the current owner has owned the business for six years. as with the previous two cases, the owner perceives that all of his competitors use e-commerce. he upgraded the existing computer system, which seemed to consist of one centrally located computer, for the instigation of e-commerce. the agency uses e-commerce for email, tenancy checking through ticker, internet banking and accessing legislation. negative experiences with e-commerce. the owner felt that his business had changed since the advent of e-commerce and that sometimes internet technology provided too many options, leading to a feeling of being overwhelmed, thereby threatening his control of the situation. “it’s a lot faster now, it’s more complex. now, there’s a lot more options. i think people expect a lot more services and that’s why you’ve got to keep up with the times. things are changing so quickly now with the equipment – i don’t know where it’s going to end.” overall, the owner did not seem content with changes brought about by e-commerce. “the industry is changing rapidly, the workload is double what it used to be. now that it [e-commerce] is about, there are more services that we must provide to stay competitive. it’s very good for us to have, too, but we’re working a lot harder”. the owner has not received any feedback from his customers regarding his e-commerce initiatives. he did not experience any unforseen benefits from e-commerce as others in the real estate industry had told him what to expect. positive experiences with e-commerce. the owner believed that e-commerce had enhanced customer enquiry: and become more of an industry standard: ajis vol. 10 no. 2 may 2003 56 certainly our industry, over the last five years has definitely changed. it’s been good for us too, we’ve got a lot of enquiry from that site that we would never have had.” he does use the internet to conduct research such as checking on competition and states that: “its regularly updated and it’s a great source of information.” according to the owner, he has experienced a return on his investment in adopting e-commerce because he did not spend a large amount of money: “we haven’t gone overboard with it, just kept up with the times. business is building and this is definitely partly due to e-commerce.” as a form of marketing, the owner believes that e-commerce has worked quite well for his business. e-commerce success contributors. the owner considers that his e-commerce venture has been successful and ascribes this to the fact that he waited to see what his competitors did, demonstrating that first mover advantage was irrelevant to him. he feels he had the time to then conduct his e-commerce initiative effectively. he stated that: “we didn’t rush into it. we weren’t the first doing it, by far. but when we had it up and running, we were using it to its best advantage.” this restrained uptake could also be attributed to the owner’s lack of technical knowledge in not realising the possible benefits of e-commerce to his business. an additional success contributor was the education he received from the seminar conducted by the technology and website development organisation. the owner felt that the seminar had taught him valuable information about the internet, and allowed him make more informed decisions, facilitating a sense of control and commitment. he was also very happy with the contractor service that he received in setting up his web presence. summary the experiential issue data has from the three field studies has been analysed and is now summarised. the negative experiential factors of e-commerce varied, with a number of negative experiences not evident in the literature emerging from the interviews. firstly, staff with specialised skills were required; secondly, the workload of the business had increased; and finally, there was a lack of perceived benefits from e-commerce. there were several positive experiential factors associated with e-commerce which were common to cases a, b and c: the ability to access on-line resources for research; improved interactions with customers; better penetration of the local marketplace; and enhanced efficiency within the business. although not commonly noted in the literature, all of the organisations utilised b-2-b interactions such as internet banking, tenant checking and legislation access through the internet. this, along with the use of email was seen as improving the efficiency of their business. two positive experiences that did not appear in the literature surfaced from the interviews. case a identified that the efficiencies that e-commerce had created in the workplace had resulted in the staff being more contented and having lower stress levels. both cases a and c stated that they had experienced an increase in business since their uptake of e-commerce. interestingly, while cases a and c perceived future e-commerce benefits would be experienced, case b did not believe that e-commerce would produce any further benefits to his agency in the future. however, due to pressure from the parent company, he seemed to feel compelled to continue its use. a number of the factors shown in the literature review to be inhibitors to e-commerce adoption manifested as actual negative experiences by the real estate agencies. they are: poor experiences with it consultants; lack of roi; lack of perceived benefits; and increased staff workload. likewise, various e-commerce adoption facilitators identified in the literature emerged as positive experiential factors for the cases studied. they are: enhanced business efficiency; cost benefits; improved customer interactions; and increased business roi. this demonstrates that organisations are becoming more informed about the relevant issues likely to impact on their businesses before they adopt the relatively new business process of e-commerce. ajis vol. 10 no. 2 may 2003 57 the owners in each of the cases a, b and c perceived a number of contributors to e-commerce success. all of the agencies had a service that could be provided over the internet. each agency had been able to source a competent consultant and had incorporated e-commerce into their business processes. all of the cases had incorporated e-commerce into their businesses to some extent, most commonly through the use of email and b-2-b interactions. although b-2-b dealings such as internet banking, tenant checking and accessing legislation were cited as being a factor in the literature regarding e-commerce experiences, this was not revealed in the literature as a contributing success factor for e-commerce. in fact, case b nominated this factor as the most successful aspect of e-commerce, as his agency’s on-line presence had not met his expectations regarding increased sales. other noted contributors to the success of e-commerce that were not found in the literature were: effective web site promotion; taking a cautious, wait and see approach and not utilising unproven business applications; and continued customer use of e-commerce. one issue that was noticeable by its omission was that none of the organisations went into e-commerce with a formal plan or a defined business case with clear objectives. they all simply allowed the project take its own course. the owners all learnt along the way from their own or others’ experiences. this seems to be consistent with the manner in which many small businesses function. conclusion findings from this preliminary study have identified e-commerce experiences by smes in the real estate sector. it has resulted in preliminary findings that can be used for further in-depth research into e-commerce experiences and critical success factors (csfs). the findings have been informed from the literature on e-commerce experiences and csfs, as well as the three in-depth field studies undertaken. in many ways, this study raises many questions and has implications for further research in this important field. it would be useful to undertake further field/case studies in the real estate sector to improve the rigour of the findings. in addition, broadening the sme industry sectors would also help to improve the generalisability. a comparison between other regional area in australia, or replication in other metropolitan areas would also provide an important contribution to this area of research. however, findings from this research make both theoretical and practical contributions to is literature. in practical terms, is professionals and sme owner/managers can learn from the findings and improve strategic planning and assimilation techniques. theoretically, researchers in this field of study have a basis to undertake further research to more clearly identify lessons learnt, experiences and csfs in e-commerce adoption and assimilation. references abs (australian bureau of statistics), 2000, 8129.0 business use of information technology australia, [on-line], http://www.abs.gov.au/ausstats/abs%40.nsf/b06660592430724fca2568b5007b8619/9c7742890adec989c a2568a900139423!opendocument, accessed 15 may 2001. adam, s. and deans, k. r., 2000, “on-line business in australia and new zealand: crossing a chasm”, ausweb2k sixth australian world wide web conference, southern cross university, lismore, [on-line], http://www.ausweb.scu.edu.au/aw2k/papers/adam/paper.html, accessed 7 june 2001. benbasat, i., goldstein, d. k. and mead, m., 1987, “the case research strategy in studies of information systems”, mis quarterly, vol 11, issue 3, pp 369-387, september. bode, s., and burn, j., 1999, “smes and the role of consultants in establishing e-business: a case analysis”, proceedings of collecter ’99, victoria university, new zealand, [on-line], http://www.collecter.organisation/coll99/bode.pdf, accessed 17 march 2001. boon, o. and hewett, w. g., 2000, requirements gathering in implementing a web presence: a study of the warrnambool region initiative, school of management information systems, faculty of business and law, deakin university, [on-line], http://mis.deakin.edu.au/research/working_papers_2000/2000_11_hewett.pdf, accessed 19 march 2001. castleman, t. and cavill, m., 2001, “e-everything: e-commerce, e-government, e-household, edemocracy”, proceedings of the 14th bled electronic commerce conference, bled slovenia, june 2526, [on-line], http://mis.deakin.edu.au/research/working_papers-2001/2001_02_castleman.pdf, accessed 27 june 2001. critical e-commerce success factors ajis vol. 10 no. 2 may 2003 58 castleman, t., swatman, p., and swatman, p. m. c., 2000, the role of government in internet commerce: views from the fields, [on-line], http://mis.deakin.edu.au/research/working_papers-2000/2000_09_castleman.pdf, accessed 6 may 2001. chan, e. and swatman, p. m. c., 1999, “electronic commerce: a component model”, proceedings of collecter ’99, victoria university, new zealand, [on-line], http://www.collecter.organisation/coll99/elsiechan.pdf, accessed 17 march 2001. chappell, c. and feindt, s., 1999, “analysis of e-commerce practice in smes”, knowledge and information transfer on electronic commerce (kite), european union on-line, [on-line], http://europa.eu.int/ispo/e-commerce/sme/reports/kitebsetpractice.doc, accessed 9 april 2001. chau, s. and pederson, s., 2000, “the emergence of new micro businesses utilising electronic commerce”, proceedings of the 11 th australian conference of information systems (acis), cable, g. g. and vitale, m. r. (eds), brisbane, [cd-rom]. drennan, j. and kennedy, j. (2000) “internet acceptance and use by small firms: a study of the pharmacy sector”, conference proceedings of 45th icsb world conference, brisbane, 7-10 june [cd-rom]. eisenhardt, k. m., 1989, “building theories from case study research”, academy of management. the academy of management review, october, vol 14, issue 4, pp 532-551. golden, w. and griffin, m., 1998, a survey of internet commerce in small and medium enterprises, national university of ireland, galway, [on-line], http://www.nuigalway.ie/acc/staff/wgolden/aurvey/survey_report.htm, accessed 23 april 2001. kalakota, r. and whinston, a. b., 1997, electronic commerce: a manager’s guide, addison wesley longman, inc, usa. lawrence, e., corbitt, b., tidwell, a., fisher j., and lawrence j. r., 2000, internet commerce: digital models for business, john wiley & sons australia ltd, brisbane. lawrence, k. l., 1997, “factors inhibiting the utilisation of electronic commerce facilities in tasmanian smallto medium-sized enterprises”, 8th australasian conference on information systems, october. love, p. e. d., irani, z., warren, m., coldwell, j. and warren, s., (2000), “barriers to implementing ecommerce in construction smes”, anzam conference, sydney, australia, dec 3-6, [online], http://www.gsm.mq.edu.au/conferences/2000/anzam/archive_pap&abst/fullpaper_frameset.html, accessed 10 may 2001. marshall, p., sor r. and mckay, j., 2000, “electronic commerce and car retailing: an industry case study”, international journal of electronic commerce research, vol 1, no1. moutinho, l., goode, m. and davies, f., 1999, quantitative analysis in marketing management, john wiley, brisbane. o’brien, p. and van akkeren, j., 2000, “the effectiveness of project-assisted adoption of e-commerce by small businesses in the tourism industry”, conference proceedings of 45th icsb world conference, brisbane, 7-10 june [cd-rom]. poon, s., 2000, “business environment and internet commerce benefit – a small business perspective”, european journal of information systems, june, vol 9, issue 2, pp 72-81, [abi inform]. poon, s., and strom, j., 1997, “small businesses’ use of the internet: some realities”, proceedings of the seventh annual conference of the internet society (inet 97), the internet: the global frontier, kuala lumper, june 24-27, [on-line], http://www.isoc.org/inet97/proceedings/c2/c2_1.htm, accessed 4 june 2001. poon, s. and swatman, p. m. c., 1999, “an exploratory study of small business internet commerce issues”, information & management, jan, vol 35, issue 1, pp 9-18, [abi inform]. sekaran, u., 1992, research methods for business: a skill building approach, 2nd edition, john wiley & sons, inc, usa. singh, m. and fisher, j., 1999, “electronic commerce issues: a discussion of two exploratory studies”, proceedings of collecter ’99, victoria university, new zealand, [on-line], http://www.collecter.organisation/coll99/singh.pdf, accessed 17 march 2001. singh, s., and slegers, c., 1998, the story of small business and electronic commerce, centre for international research on communication and information technologies (circit), rmit, melbourne. swatman, p., 2000, “internet for smes: a new skill road?” international trade forum, geneva, issue 3, pp 22-24, [abi inform]. timmers, p, 1999, electronic commerce: strategies and models, for business-to-business trading, john wiley & sons ltd, england. walsham, g., 1995, ‘interpretive case studies in is research: nature and method’, european journal of information systems, vol 4, issue 2, pp74-81. ajis vol. 10 no. 2 may 2003 59 yellow pages australia, 2000, small business index: survey of computer technology and e-commerce in australian small and medium businesses, telstra corporation limited and the national office for the information economy, australia, june. copyright diana gray and jeanette van akkeren © 2002. the authors assign to acis and educational and non-profit institutions a non-exclusive licence to use this document for personal use and in courses of instruction provided that the article is used in full and this copyright statement is reproduced. the authors also grant a non-exclusive licence to acis to publish this document in full in the conference papers and proceedings. those documents may be published on the world wide web, cd-rom, in printed form, and on mirror sites on the world wide web. any other usage is prohibited without the express permission of the authors. microsoft word ajisvol10 1_final_edit.doc ajis vol. 10 no. 1 september 2002 65 introduction by the editors this special issue of ajis focuses on the importance of information systems and technology to smalland medium-sized enterprises (smes). the papers appearing in this issue were originally presented at the second international conference on smes in a global economy, which was held at the university of wollongong in july 2002. all papers were double-blind refereed. the theme of the conference, which was organised by the centre for sme research and development in the faculty of commerce, university of wollongong, was “sustaining sme innovation, competitiveness and development in the global economy”. the conference built on continuing and increasing interest, from business, government and academic quarters, on the role of smes in their economies. in the apec countries, smes typically comprise more than 90 percent of the business enterprises, and account for more than half of employment, and more than three-quarters of employment growth. they produce half of total output, and are responsible for about a third of exports. smes therefore comprise a vital part of their economies, and are the source of much innovation and entrepreneurial energy. while the papers presented at the conference reflect the wide range of academic research into smes, the five papers included in this issue indicate something of a concentration on the actual and potential benefits for smes of information and communications technology (see the paper by beal and moha asri) and electronic commerce (see the papers by poon; vrazalic et al.; and jones et al.). traditionally at a competitive disadvantage because of their lack of size and access to finance and markets, many smes, particularly those in the knowledge economy, are now finding that the competitive environment has tilted towards a more level playing field where they can compete on a more equal footing. in the emerging industries of the knowledge economy, entry costs are dramatically reduced and size matters less. the emerging research indicates that the factors that matter are a firm’s knowledge base (see the paper by al-hawari and hasan) and its ability to harness and build on it to innovate effectively. although the benefits of utilizing ict seem undeniable, research also shows that generally smes are having difficulty implementing ict solutions and translating these into successful business strategies. while the authors clearly flag the need for more research in the area, their work here represent useful inputs into the continuing inquiry. charles harvie and boon-chye lee editors quality of goals a key to the human-oriented technology barbara begier, dept. of control, robotics, and computer science, poznan university of technology, pi. m. sklodowskiej-curie 5 60-965 poznan, poland e-mail: barbara.begierts.put.poznan.pl abstract a kind of an essay on some social and ethical impact of new technologies has been given in the first five sections. the next two sections bring a study on quality referred to the software process. then some ethical issues are presented in the context of it development and computer applications. the need for a quality of goals has been emphasized. human objectives should gain the top position, especially in a coming post-computer era. some conceptual research in the topic of quality of goals has been suggested to safe humanistic values. instead of an introduction the exhibition "planet of vision" presented on expo 2000 in hanover (germany) pointed out some social utopias in 20* century. one of them showed a large city where millions people stand their good chance. nowadays many previous inhabitants of large cities prefer to escape from all town inconveniences and to move out to green suburbia. the other utopia promised jobs for everybody or at least for those who are skilled enough not true in practice, again. next, a hope that a technical progress would result in a decrease of work actually people are busier at their work than ever before. in general, it is been a great hope (expressed and represented mostly by business circles) that any technically possible product or solution will be socially accepted and even much desired. the last statement seems to be false but it still holds good. in my opinion, a new utopia was born. some of its essentials are as following: • technical progress brings only benefits for its consumers • proposals of information technology may satisfy everybody • informationalization means always social progress • it products may improve everything • multiculruralism is within one's grasp • face-to-face relations may be replaced by electronic media • sitting all day long at a computer screen causes no health problems. strong points of it solutions are well known. their enthusiasts cannot imagine to start talking loudly also about weak points. new technologies have not decreased the bureaucracy so far, rather just the opposite. a progress in the computing technology continues to accelerate but at the same time people realize social threats of it. computing technology changes more and more forms of work, teaching, and every-day-life. unfortunately, each technical achievement becomes obvious after some period of time. also it products may delight people no longer. computers may share ups and downs of prior technical inventions, which are just useful tools today, like vehicles, vacuum cleaners, or factory robots. so it is time to start talking about the post-computer era like we often use a term of the post-industrial era although the industry has not disappeared, of course. positive vs. negative social and ethical impact of it from a global point of view, computing techniques and presence of internet have initiated considerable changes on a scale of the whole world. from among many achievements and practical issues, which form a wide spectrum of positives of it results, one may chose the most essential ones and express some crucial statements accepted today: • knowledge has been highly valued and even admired. • high education and practical skills have received wide recognition. • results of technical progress may be fast expanded around the world. • computer networks provide quickest means of communication in a global scale. • fast communication makes business and life easier. • awareness of being a citizen of only one world has become more and more a remedy to make it smaller and less hostile. • representatives of various nations and organizations cooperate in a distributed mode in almost all areas of activities like university research staff, employees in global corporations, international teams in scientific and social ventures, and so on. 148 on the other hand, growing dependency on the computing technology may possibly cause even a disaster in a technical sense, the same as an electrical power blackout, for example. it may happen accidentally or may be intended by some anti-global individuals or organizations. taking into considerations, on a high level of abstraction, only social impacts one may form a long list of possible wrong effects: • next step towards so called mcdonaldization [ritzer, 1996] of society; the step caused by a constant focus on effectiveness, standardization, repeatability, and fastness all of them referred to any sort of activity. • progressive loneliness of society members, each of them well equipped with a computer. • lowering creativity of passive screen-watchers. • possible incompetence and thoughtlessness of some users of an informational supermarket. • replacing face-to-face relations by technical means in e-commerce and e-learning [begier, 2000], and losing a chance to follow good personal patterns. • farther development of bureaucracy supported by it. • lack of privacy parallel to the growing number of various databases. • replacing a joy of being a citizen of a global village by a fear that each step in the internet is possibly monitored and tracked. • deepening gap between the rich and the poor part of society. • ignoring ethical principles [maclntyre, 1966] in a context of growing competitiveness. it's not my intention to describe problems in black and white or to propagate just skeptic opinions. for example, the lack of good personal patterns is a problem noticed in many areas of living, not just among computer users. it seems to be worth considering all together, positive and negative impact of the mass use of it products. to recapitulate, the professional and ethical responsibility has to be strengthened [chroust, 2000] across all 'computer' community, including designers and users of computer applications. human aspects in the context of it products societies are under the pressure of marketing activities and mass media, which strongly depend on the number of commercials. also universities and scientific conferences are supported by companies of the it profile. so everything concerning computers must be ok and the reliable critics has rather small chance to appear. a modem patronage and some forms of political correctness may also be seen as new sources of indoctrination and may restrict unconstrained thinking. software authors are often not able to look at their products from distance. they do not seem to be interested in it. but a tendency to 'do not touch' any social problems related to applying it products is no longer valid. citizens are forced to behave in a way that the authors of information systems and computer applications have designed, often ignoring human predisposition and habits, his/her likes and dislikes, and so on. documents concerning tax return and national insurance are also good examples of it. a person is supposed not only to learn how to use computer systems but she/he should adapt himself to the mode provided by the new technology. the computer-centered technology in its core encourages users to act and work individually, to believe computer records, to trust a content of any document, etc. and as a consequence, it forces people to adapt to it products and rarely adjusts products to human beings. so the real question is are users supposed to act just like human robots as an addition to computers? software designers verbally declare their efforts to protect multiculturalism against a domination of one culture. but this care is usually reduced only to the language translation [chroust, 2000]. the first aim of any it products is to increase an effectiveness of work, usually ignoring any other values, which are important in some cultures. one good example of it is a focus on a superior value of a whole community in a contradiction to the role of a separate individual. feelings of safety and consciousness of a membership of a given community play a great role. products of it technology are not only modern tools — all together may be also seen as a mirror of western civilization. the huntington's clash of civilizations may become a fact sooner than we may expect. modern inventions may be useful to achieve quite different goals than the intended ones. and let us do not forget that all development of computing techniques has been driven by grants of the military sector of economy. a notion of comfort and self-service from the very beginning a notion of comfort in a civilized world has gone hand in hand with a social division of labour. to some degree it was not contradictory to the efficiency of work. it appeared more convenient to hire one more worker than to do many various activities by the same one. this approach has been also applied to the home duties. it also results in many direct contacts between people. but maybe it is not true any more. 149 self-service has already replaced salesclerks in supermarkets and removed human staff from most gas stations. using e-business people do by themselves all activities performed earlier by clerks just making hotel reservations, ordering any goods, using home banking applications, applying for many things, and so on. by the way people are supposed to learn how to use so many various systems when at the same time a professional clerk operates only one or two. at the beginning mentioned above solutions may even amuse some people, especially those who are proud to keep up modern technologies. when the time goes on the same users will realize that their distant activities bring profit to someone else. there are mainly economic reasons to pass clerks' duties on to clients. it is profitable but it works only on a local scale of one company. societies face all problems of unemployment and later, of social underclass. a notion of comfort includes also sense of having time to work and time to spare. high effectiveness often means flexible working hours. the last term is just the today's euphemism since it simply means to be ready to work all day long. also tele-working means to be always busy. probably many people are even not aware of the manipulation they are involved in. there is a focus on quantitative criteria although a lot of words have been told about quality. quality has been mostly identified only with a large number of anything (goods, income, points in tests, graduates, etc.). maybe a society is rather close to the mcdonaldization (where everything ought to be standardized and effective) than to the better quality of living. educational self-service at universities the problem, how to provide higher education for more and more students has been reported not only in poland. educational problems need new solutions. computing technique has been applied or at least is supposed to be widely introduced in education. there is also impossible to run a university without it systems. but does it mean that it's now a time for educational self-service? it is the question: can educational computer program, videotapes and teleconferences cause the same desired effects as direct meetings at seminars? the president of harvard university, lawrence summers doesn't worry that online education might displace traditional schools. in the interview for the newsweek magazine (march 26, 2001) he really doubts that there will ever be a substitute for personal interaction between a teacher and a learner. one should distinguish enthusiastic declarations from real possibilities. many students do not read any printed books in practice but at the same time they are optimistically supposed to read electronic publications from the internet. is the goal of education at the university level to produce just human 'robots' packed with information and knowledge of a theoretical background, and mastering some skills? university is supposed to educate an intellectual elite of a society. apart teaching about a given domain of study the university should produce so called public intellectuals, which are prepared to take over a leadership in a society. graduates are supposed to act for a community. the following features should be developed among others: • ability to provide quality in any area of activity • creativity • respect for many cultures • responsibility for himself and for others • openness to new ideas and other cultures • skills in a team work in contrast to an individual style of work • familiarity with technical culture. using it in an educational process may result in a extra cult of technique and technical devices. a young user may easily believe that technical achievements are superior to any other forms of human activities. so it is a short way to ignore those communities who are not world leaders in technology. disrespect of other societies is, at least, against a political correctness. an educational process is supposed to school oneself in humility and politeness, also towards unknown cultures. any modern process is supposed to be effective. the common measure of effectiveness is time. nevertheless, higher effectiveness may result in a simplified product. the criterion of fastness may fail in the educational process. dehumanization of education may lead to change, in general, life conditions to more dehumanized forms. from the social point of view, mass education at the university level may produce people jobless in the future and maybe also next generation of the leisure class [veblen, 1899] in its modern version. in general, any occupation, concerned with power, politics, religion, war, hunting, and entertainment, is representative for the leisure class, which does not process any material goods in its activities. at any times in the history, leisure class has carefully preserved its privileged position which has been superior to the rest of society. 150 quality in terms of tqm and iso 9001 edwards deming [deming, 1982], one of the best-known advocates of quality has suggested the quality-oriented way of production. continuously improved quality will result in a long-term prosperity of a company. going further, everything you may produce and sell is an appropriate thing. deming prescribes neither the social needs nor what needs to be done to match them. the only criterion of a quality is an increased market share of a given product and a profit. it does not make sense to revise principles of the market economy just goals and intentions of some projects need some concern with their social effects. especially projects, which deal with any social sphere, like education, health care, social securities, and other kind of insurance. total quality management (tqm this abbreviation became the well-known term in the mid-1980s) put an emphasis on the process to provide quality of products and places responsibility for the quality of a process primarily with the owners of the process. so tqm is a management approach to success on a market. customer satisfaction is its fundamental idea. all employees of a company are supposed to participate in quality activities like improving processes and the culture of work. the tqm approach specified and applied first in motorcar industry has been widely accepted in other domains and organizations, and adapted to the software production, too. the underlying essence of the iso 9000 series of standards [iso 9001:1994 and all family of quality standards and then'derivatives'] applied to software development is that the supplier must establish the quality system that should be described and completed, controlled, constantly improved, and effective. the iso certificate has become a common required document and it does not often work as a distinguishing mark of preference. quality management and the client-oriented policy are recommended in the new version of quality standards, namely iso 1901:2000, which underline the importance of client's requirements and expectations. the today's standards encourage to the process-oriented approach to develop, use, and improve the quality system. wellknown method of the pdca cycle (plan-do-check-act) is recommended with reference to processes. ongoing improvement of all processes should result in a better quality of products and services. software quality the international standard iso 9126:1991 defines software quality as the totality of features and characteristics that bear on its ability to satisfy stated or implied needs. the same standard says that software quality may be evaluated by characteristics of its functionality, reliability, usability, efficiency, maintainability, and portability. each of them is a set of attributes that may be assessed and measured. the question is how to build in the good quality characteristics into a software product. with respect to the quality of software several categories of recommended solutions are suggested [dunn, 1994]: • leadership, starting from the top • software measurements, analysis of measurements, and dissemination of the results of analyses • planning of continuous quality improvement • continuous upgrading of the skills of the programming staff, and involvement of the staff in quality improvement • continuous upgrading of software engineering practices and tools • assessing the results of software quality improvement processes (and comparing the results with those of others) • working closely with customers. the recipe is given how to build a quality system in the case of the software process [oskarson, 1996]. all employees must be involved in that system to meet quality requirements. all activities are to be up-to-date documented. the conformance to the 20 elements, specified by the iso 9001:1994 standard, has been required. they are worth recollecting and interpreting their meaning for the software process [schmauch, 1995]: (e) management responsibility the quality policy must define responsibilities, authority, and interrelations. (f) quality system it must be established, documented, implemented, and carefully maintained; the manual defines all quality procedures. (g) contract review there are documented procedures to ensure that requirements specified in the contract are understood and agreed by organizations of the supplier and of the customer. (h) design control design reviews and verification procedures are to assure a customer that results of the design phase meet the expectations for the product; it applies also to design changes. (i) document and data control defined procedures serve to control all documents and data available only to the right people. 151 (j) purchasing there are procedures to ensure that any parts obtained from outside the organization also meet the specified requirements, (k) control of customer-supplied product specified procedures are applied to the products supplied by the customer. (1) product identification and traceability there must be procedures for identifying and tracing the product during all stages of production, delivery, and installation, (m) process control the software process must be continuously monitored to ensure the specified input requirements are to be met. (n) inspection and testing ensures that all required testing is completed and documented, (o) control of inspection, measuring, and test equipment ensures that test tools serve thenintended purpose, (p) inspection and test status test status of the product must be identified since there are many test stages and iterations, (q) control of nonconforming product — there must be procedures that specify what to do when a product does not meet its requirements, including defects discovered after the product is delivered to a client, (r) corrective and preventive action there must be procedures to invest the causes of defects and to initiate corrective actions, (s) handling, storage, packaging, preservation, and delivery all parts of the product are stored in a safe, secure, and controlled place, (t) control of quality records quality records must be identified and carefully stored to demonstrate when necessary and to improve the effectiveness of the quality system, (u) internal quality audits qualified personnel is provided to carry out periodic internal quality audits, which determine that the quality system is being used and that is effective, (v) training training needs should be identified for each of the involved employee; required training activities are planned in advance, (w) servicing there are required documented procedures for correcting defects and any nonconformity found in the product after its delivery, (x) statistical techniques metrics and measurements used during development should be correct and accurate enough to determine the quality of the product and the appropriateness of the quality system. software must conform to the specifications. it must be suitable for the intended application and satisfy its users. many solutions and methods have been adapted from other engineering disciplines (decomposition of a process, prototyping, providing modularity, quality function deployment, verification and validation built into the software process, and others) and applied successfully in the software process [begier, 1999]. a general assumption, still underlying any computer application, is that users are working in a very disciplined manner and they are carefully acting according to procedures, designed by software developers. can we trust it, in general? involving ethical issues into objectives being challenged to recapitulate two above sections, standard elements of the iso 9000 refer to the requirements carefully specified at the very beginning of the software process. there is only a great hope that a purchaser, especially in the case of great public or other statewide information system, is able to express all requirements directly. there is still no answer to the basic questions what are the goals underlying the given order? is the required software product really good for the society, in general? what will be the price for possible social losses (in health or accepted habits, etc.), to the detriment of common goods? satisfying standards does not mean getting the better society, or at least to secure human objectives in a modern society. otherwise, it products may support wrong ideas and improper requirements. when henry ford invented the assembly line and applied it in his motorcar factories it was found as a great achievement and worldwide progress in technology. but later people stop marveling at it and didn't want to perform the same activities at their work, year by year. it is far not enough to specify only requirements for a given software product. maybe it would be better not to build this product and use it at all. the words 'effective' and 'fast' cannot always be synonyms of the good and valuable. it's time to formulate human objectives in general. quality criteria should refer to goals not only processes and their results. sociologists together with application designers might seek for an answer to the question what features of it products do their nonprofessional users complain of? an interdisciplinary research is needed to solve some noticed problems: poor ability to communicate with other people (those who order pizza via internet instead of 152 meeting and talking together), superficial interests, lack of responsibility, weak personality, tendency to imitate someone else's idea rather than to show an own creativity, etc. universities have had a very positive influence on their native and foreign students. so there is a new challenge for universities to predict some dangers for society and to initiate preventive activities on a large scale. social aspects of applying software refer to the language used by computer users. a language means a culture. different language means also a different culture not just words. so there is a need to organize multicultural versions of software to avoid a suspecting of discrimination. the language used by intemauts is very poor. often it is simply a jargon. computer users are younger and younger. computer software propagates one culture, with a focus on purchasing and admiration of technical products. a domination of one culture has always lead to conflicts. what is a paradise for technocrats (it products generate a lot of figures, reports, documents, etc.) may be not the same for the others. using internet has become an element of the e-business. commercials included on www pages may suggest that a human being is a new kind of homo oeconomicus, which is only intended to buy goods on a market. the simplified view of a society resolves itself into the statement the more buyers, the better business, which, in turn, means only happiness in a global scale. there is a danger that some unexpected event caused by followers of the opposite outlook upon life may break down this idyll. quality of goals a challenge for software engineering software engineering adapts many techniques and procedures applied in other technical disciplines. the software process usually is still far from satisfactory although a lot of words have been said about quality. the new proposal is not to limit quality features to those basic six, specified in the standard iso 9126, which refer mostly to technical aspects of a software product. we should specify quality criteria in much wider meaning than it's been practiced so far. first of all is the concern for quality of goals. a term of a risk means a possibility of unwanted consequences or losses, which may occur. these factors should not be limited to software supplier and its purchaser directly. users are more and more the common peoples who indirectly use software but directly may suffer from software defects or weak points! so the list of risk sources should contain not only technical risks but also risks that are the opposite of human and social goals, their reception and ethical attitudes. risk management is an important area of activities of software engineers. risk identification is a basis of the risk management and it refers to technical and business factors of the software process. there are recommended seven generic categories of risk [pressman, 1997]: product size, business impact (of an organization of the purchaser), customer characteristics, process definition, development environment, technology to be built, staff size and experience. software quality used to be referred to the quality policy. there are four levels of the quality policy [fleurquin, 1998]: testing, quality control, quality assurance, and quality management. all of them are focused on the software process again. there is a suggestion to add one more level: quality of goals. it is still a hope to respect an individualization of needs and even taste. let us start asking users, including indirect software users, for quality of it products. quality and iso certificates have been not intended to serve mainly as a marketing trick. although it would be hard to specify what constitute the quality of life, some efforts in this direction may help to avoid losses and troubles. a free development of an individual may be in conflict with a computer style of living spending a lot of time at the screen and acting to fulfill standardized templates and forms. human-oriented approach in a post-computer era it does not contradict the human-oriented approach, in general. on the large scale there is no monopoly in any area of human activity. next generations of computer systems are expected to adapt in high degree to individual requirements instead of adapting people to it requirements. the right of choice does not mean only a chance to choose one of the computer options. there is a new belief that people want to spend all their time at work and at home sitting at computers. interpersonal relations are still as important as earlier. the famous psychologist, professor philip zimbardo form stanford university says that the percentage of shy people in our population exceeds 50% just because of an isolation when using computers. a chance to develop an individual style of work, habits, and living should be the main goal. it is a right time to say good-bye to the mentioned at the beginning new utopia and to begin living in a postcomputer era. so it is worth reassuming what is valuable in today's information technology and what are its barriers. probably a common humility along with some criticism may help predicting new needs and realizing next inventions. 153 some conclusions and critical remarks have born in my mind after due consideration of software process and as results of several students' team projects. some research concerning quality policies and risk management has been conducted. i would like to share these experiences with people, who are able to look at things from some distance, remembering the well-known pavlov's effect. maybe the cooperation of software designers with sociologists, psychologists and even philosophers, should become an every day practice. references begier b. (1999), software engineering quality issues (in polish), wydawnictwo politechniki poznariskiej, poznari. begier b. (2000), quality of web-based applications for educational purposes, edict 2000, proceedings of the international conference on information and communication technologies in education (e. riedling, g. davis, eds.), vienna, 6-9* december 2000, 201-208. chroust gerhard (2000), internationalization is more than language translation, in: idimt-2000 proceedings, 8>h interdisciplinary information management talks, september 20-22, 2000 zadov, 431-440. deming w. e. (1982), quality productivity and competitive position, mit center for advanced engineering study, cambridge, massachusetts (usa). dunn r. h., ullman r. s. (2nd ed. 1994), tqm for computer software, mcgraw-hill, inc., usa. fleurquin r., motet g. (1998), introduction & la qualiti logicielle, polish translation: krakowskie centrum informatyki stosowanej, krakdw. iso 9001:1994 (2nd ed.), quality systems model for quality assurance in design, development, production, installation and servicing, geneva, switzerland. iso/iec 9126:1991 (e), information technology software product evaluation quality characteristics and guidelines for their use, geneva, switzerland. krawczyk h., sikorski m., szejko s., wiszniewski b. (2000), an assessment of the essentials of software quality characteristics (in polish), in the proceedings of the 2nd national conference on software engineering, zakopane, poland, 18-20 october 2000, 179-186. maclntyre a. (2000), a short history of ethics, 2nd edition of polish translation: pwn warszawa, (originally: the macmillan publishing company 1966). oskarson 6., glass r. l. (1996), an iso approach to building quality software, prentice hall ptr, upper saddle river, new jersey, usa. pressman r. s. (1997), software engineering. a practitioner's approach (4th edition), mcgraw-hill, new york. ritzer g. (1996), the mcdonaldization of society, pine forge press, a sage publications company. schmauch ch. h. (1995), iso 9000 for software developers, asoq quality press, milwaukee, wisconsin veblen t. (1899), the theory of the leisure class, last polish edition: muza s.a, waszawa, 1998. 154 microsoft word special final.doc ajis special issue december 2004 4 consumption of multiple concurrent identities: the need from the instant messaging virtual community1 minh hong tran, yun yang centre for internet computing and e-commerce (cicec) faculty of information and communication technologies swinburne university of technology p.o. box 218 hawthorn 3122 australia {mtran, yyang}@it.swin.edu.au tel: 03 9214 8934 fax: 03 9819 0823 gitesh k. raikundalia internet technologies & applications research lab (itarl) school of computer science and mathematics victoria university melbourne city mc 8001 australia gitesh.raikundalia@vu.edu.au tel: 03 9688 4691 fax: 03 9688 4050 abstract in recent years, instant messaging (im) has become one of the most growing online communities, reaching millions of users at home and at work worldwide. as the community expands, assuming more than one identity at the same time becomes a common behaviour of im users. this paper reports our ongoing research on support for multiple concurrent identities (mci’s). our study used an online survey and face-to-face interviews to identify user needs in supporting mci’s in im. the study has identified five themes including single nickname, multiple avatars, multiple statuses, boundary control and interoperability. reflecting on these themes, we propose a framework of mci support in im. keywords: instant messaging, multiple identities, virtual community, avatars. introduction the scale of computer networks has expanded rapidly over the past thirty years. hundreds of millions of pc users are now connected in varied networks including the internet. solidly, computer networks have become one of major channels facilitating human communication. they broaden the temporal and spatial dimensions of human connectedness to the level of anytime-at-anyplace connectivity, and increase the number of ways in which people can connect with one another. in addition to the support for fast, ongoing conversations between people, computer networks also foster new social spaces, called virtual communities or online communities (we use these two terms interchangeably henceforth) (harasim 1993, smith and kollock 1999). the concept of virtual communities has described variedly by different people in different context (preece 2000). one of the earliest and most well-known definition of virtual communities was coined by rheingold (1994), he described virtual communities as “cultural aggregations that emerge when enough people bump into each other often enough in cyberspace” (p57). in the past twenty years, there has been an increasing number of virtual communities, where different degrees of online interaction such as making friends, exchanging ideas, acquiring knowledge, entertaining, etc. take place (jones 1998, mccaughey and ayers 2003, preece and maloney-krichmar 2003, renninger 1 a shorter version of this paper was published in ozchi 2004, wollongong, australia, cd isbn 1 74128 079. ajis special issue december 2004 5 and shumar 2002, rheingold 1993, turkle 1996). online communities are formed by many means of computer-mediated communication such as electronic mail (e-mail), bulletin board systems (bbss), multi-user dungeons (muds), usenet newsgroups, internet relay chat (irc) and world wide web (www). every time people send e-mails to other people, post messages to bulletin boards or browse web pages, they are part of online communities. “it’s like having the corner bar complete with old buddies and delightful newcomers… except instead of putting on my coat, shutting down the computer, and walking down to the corner, i just invoke my telecom program and there they are. it’s a place.” rheingold (1993) describes his mud virtual community. virtual communities are relatively open playgrounds where members can assume many identities for different purposes such as adapting themselves to the community or accomplishing their playing roles (e.g., lurkers or posters) (nonnecke et al. 2004). people can be whoever they want to be such as of the opposite sex, different races, having higher status (e.g., through a work occupation), lacking or having a disability, etc. (donath 1999, o'brien 1999). some people build their identities based on their true personality, while others create online identities with the hope that their online characters will help developing their characters in real life. furthermore, similar to real-world, in an online community a person may have different virtual identities, and it is likely that other members in the community may not know all identities of that person. “i’m not one thing, i'm many things when i’m muding” (turkle 1996). “i have three or four personae myself in different virtual communities” (rheingold 1993). owing to the absence of physical appearance in online communities, changing and switching virtual identities are often easier than in the physical world. few keystrokes can easily lead to the termination or creation of virtual identities. thus it is a common practice that people assume more than one identity in virtual communities. for example, irc users register several nicknames to participate in different irc channels; bbs users use different nicknames to send postings to different boards. supporting multiple identities is, therefore, essential for conducting interactions in virtual communities. this paper reports our ongoing study of support for multiple concurrent identities (mci’s) in one particular virtual community, i.e. the instant messaging (im) community. when using the term mci’s, we refer to a user’s consumption of many identities at the same time. also, within the scope of this study, we are not concerned with the issue of identity deception, which has been examined comprehensively elsewhere by donath (1999). the remainder of the paper is organised as follows. the next section introduces the im virtual community and examines support for mci’s in current im. section 3 describes the study by detailing an online survey and face-to-face interviews. section 4 reports the results of our study regarding five emerged themes of single nickname, multiple avatars, multiple statuses, boundary control and interoperability. section 5 discusses possible solutions for the five issues, and presents our proposed framework of mci support. finally, we conclude the paper by recapitulating the findings of our study and presenting a view of future research and development. ajis special issue december 2004 6 multiple identities in instant messaging in this section, we first quickly introduce im and how the technology has been used in different locales (e.g., home and business). then, current support for mci’s in im is examined. im: a growing virtual community in the early days of im, it was largely used as a text-based messaging service for teenagers’ chitchat, increasingly many features such as voice chat, webcam, emoticons, avatars, etc. have been added to im. today, both text-based and multi-media messages (e.g., audio, video and graphic contents) are exchanged over im networks. im users are no longer limited to teenagers, millions of home users and business users are now members of the im community. the population of im home users increased 28% from 42 million in september 2000 to 53.8 million in september 2001 (perera 2001), and enterprise im is growing at approximately 20% annually (shukla 2003). as an example, research shows that 77% of ibm employees say that im has enhanced their way of communication by lessening the time they often spend on e-mail, telephone and face-to-face communications. more than 80% of employees at ibm say that the technology has made their job easier. in addition to that, more than 75% of ibm’s clients, who have im, acknowledge that im has enhanced their productivity (shukla 2003). of late, there is a large body of research on im (grinter and palen 2002, handel and herbsleb 2002, isaacs et al. 2002, nardi et al. 2000). those studies mainly investigate the adoption of im in a particular cultural group. for example, grinter and palen (2002) examined im’s support of interpersonal communication amongst teenagers. nardi et al., handel and herbsleb, and isaacs et al. explored the use of im in the workplace. those studies have shown varied evidence of how im has been used effectively to support people’s work. the ethnographic study conducted by nardi et al. identified different uses of im in the workplace such as broadcasting information, negotiating availability, switching media, etc. handel and herbsleb’s analysis of message content showed that there was a significant use of chat in facilitating work discussion and group coordination. unfortunately, to the best of our knowledge, no research to date examines support for multiple identities including mci’s in im. support for multiple concurrent identities in im as im becomes popular, it is possible that one user uses im in different domains (e.g., domestic and commercial) for many purposes under different identities. for examples, previous research (handel and herbsleb 2002) has shown that people in the workplace often participate in many different groups. hence, there is a need for supporting mci’s in im. the current framework of mci support in im is that one username carries one virtual identity. this framework is used by four most popular im networks including aim (american online instant messenger), icq or ‘i seek you’ (icq inc.), msn (microsoft service network messenger) and yahoo (yahoo! messenger). currently, im users assume mci’s by registering usernames with one or many im networks. when a user enters the im community, this user can take on many identities simultaneously by logging into different im networks with different usernames. for example, a user can have two yahoo usernames and three msn usernames; s/he can log into both yahoo and msn networks simultaneously using two usernames—one yahoo and one msn. in addition to that, current im does not support different settings for different people in a user’s messenger list or buddy list. that means the user’s setting appears the same to all people in the buddy list. for example, a yahoo user customises her/his status as ‘boo, i miss you’, then everyone in the user’s buddy list sees the same status. ajis special issue december 2004 7 as turkle argues, the self in cyberspace is fragmented and multiple, but “multiplicity is not viable if it means shifting among personalities that cannot communicate…how can we be multiple and coherent at the same time?” (turkle 1996, p258). putting this argument in the context of im, identity multiplicity is supported in current im. the users can create many identities by registering different usernames with many im networks. however, there is no coherence between many identities of a user. moreover, im does not provide a platform that supports flexible switching between a user’s identities. in summary, the current framework of mci support in im encourages multiplicity of identities, but highly limited in facilitating communication between a user’s identities. the study this section describes our study of im, which is composed of an online survey and face-to-face interviews. the survey covered many aspects of im such as awareness information (e.g., displaying users’ local time and users’ physical location), emotional support (e.g., customised emoticons), identities (e.g., avatars and statuses), and privacy (e.g., live video and live audio), etc. however, in this paper we only report the results in relation to the issues of mci’s. whilst the purpose of the survey was to gain a broad understanding of the use of im including how mci’s are currently assumed in the im community, the interviews were conducted to get a deeper understanding of mci assumption in im. online survey the online survey consisted of demographic multiple choice questions, 7-point likert scale questions and open-ended questions2. examples of questions in the survey included: demographic questions (e.g., “when do you often set your status online i.e., available or visible”); likert scale questions (e.g., “i want to set different statuses for different people in my buddy list” and “i don’t want to know geographical locations of other people”); and open-ended questions (e.g., “what other features do you want when using instant messengers?”). 173 participants3 took part in the survey. the participants comprised of 56 females (32%) and 117 males (68%). in this study, we did not aim to study the different uses of im between males and females, hence we set no gender balance target, and as the nature of online survey we accepted responses as they came. the participants were students from several universities in australia and most of them (75%) were in their early twenties. one requirement of our survey was that participants must have used im for at least 3 months. most of the participants used im regularly: 35.1% used im many times per day, 32.4% used few times per day and 23.1% used few times per week. face-to-face individual interviews after finishing the survey, we were interested in knowing more about the participants’ use of im, so we conducted further informal face-to-face interviews with 6 participants including 2 females and 4 males, who were selected from the 173 participants of the online survey. similar to the online survey, we did not set gender balance target for our interviews. the six participants were chosen based on a first-responded, first-selected fashion. open-ended questions were used in the interviews. the questions were chosen based on the results of the online survey. for example, the participants’ responses to the needs for different statuses and 2 the list of all questions used in the survey is included in appendix a of the paper. as mentioned earlier, not all questions are related to the issue of multiple identities. 3 some participants did not answer all questions, thus the number of valid respondents vary amongst questions. ajis special issue december 2004 8 avatars (as reported in the next section) came out strongly in the survey. hence, in the interviews we focused asking the participants questions about their current uses of statuses and avatars, and what support they wanted from im in relation to the two issues. examples of questions used in the interviews are “can you tell me about your use of avatars”, “why do you use avatars?” and “why do you want to use different avatars?”. the interviews were informally structured. in the interviews, apart from the pre-chosen open-ended questions, we also asked follow-up questions based on the participants responses. we found that technique really useful for the study. as an example, one of our finding (i.e., the issue of single nickname) though was not anticipated by us before the interviews, it emerged strongly from the interviews, as described in the next section. results the study overall yielded much data which we are still analysing. this section reports preliminary results that mainly look at trends of the respondents’ beliefs and usage. five main themes have emerged from the study: single nickname, multiple avatars, multiple statuses4, boundary control, and interoperability. single nickname members of the im community recognise one another from their nicknames5. a nickname can be edited by the owner or other members; thus members’ nicknames can be the same or different from their im usernames registered with im networks. in some im systems like msn and icq, the users are allowed to edit their nicknames to what they want to be seen by other members, and others cannot modify those nicknames. for example, if a user creates a nickname as ‘dark angel’ then appears as ‘dark angel’ in other people’s buddy lists, and they are unable to alter that nickname. however, other im systems like yahoo take an opposite approach by letting im users to edit how they want to call people in their buddy lists. this issue of single nickname was not covered in the online survey, but arose from the interviews. all six participants of the interviews found the method of managing nicknames used by msn and icq very inconvenient. as stated by one participant “msn list is very annoying as it makes so difficult to find people. many times, i want to look for a friend [in the buddy list] but i don’t know who is who because my friends often change their nicknames. i have to go through the entire list and check their email addresses to find the one i am looking for”. the study found that allowing im users to edit their nicknames freely causes inconsistency in other users’ buddy lists, hence providing a single nickname is a viable approach in supporting mci’s. multiple avatars our respondents found avatars interesting and useful. all respondents like avatars and use 4 for short, we use terms multiple avatars and multiple statuses to refer to multiple concurrent avatars and multiple concurrent statuses henceforth. 5 in this paper, we distinguish between a user’s username and the user’s nickname. the username is what the user selects when registering with an im network, whereas the nickname is what the user is seen in the im network. in some cases, the username and nickname can be the same but in many cases they are different. figure 1: responses to multiple avatar support. ajis special issue december 2004 9 them for varied purposes ranging from a simple purpose such as “i like avatar, it gives me something to look at when chatting” to a more meaningful purpose such as expressing their mood to online friends, “i use a funny avatar when i am happy”. around 80% of the respondents want to display different avatars when they chat with different people (mean = 5.8; std. dev = 1.6; n = 149), as seen in figure 1. one participant responded “i often use my real photo [as avatar] when chatting with mom, dad and my brother. but i’d use someone else’s photos or cartoons when chatting with friends”. on a practical level im supports communication, but for many respondents im is a place for social interactions, and sharing interests. we found that avatars are certainly effective in supporting im to serve those purposes. as reported by one respondent “if i have an interesting picture, i used to email to my friends. but now i often set the picture as my avatar so my friends can see it. but some pictures, i don’t want all people [in the buddy list] see it”. our respondents’ comments reflected the overwhelming interest in supporting multiple avatars in im. multiple statuses while an avatar is a graphical expression of users, online status (referred to as status for short) is a text-based method commonly used by im users to project themselves to other online friends. “i often use status to tell my friends if i am sad or happy and also edit my status to tell them what i am doing like studying, cooking, and stuff” said one participant. the text-based representation makes status very flexible in supporting the users to describe themselves to their online friends. “whenever my computer is switched on, i am in yahoo. i don’t turn off my yahoo even when i am busy doing assignments. i don’t want to use yahoo default status like ‘busy’ i often use something like ‘working on law assignment’ instead.” nearly 75% of the respondents want to use different statuses with different people in their buddy lists (mean = 5.4; std. dev = 1.61; n = 149) (figure 2). it shows a strong need of providing multiple statuses in im. it is a common behaviour for im users to interact with many people simultaneously. 92% of our respondents had used im to chat with two people or more at the same time: 16.2% chat with a maximum of 2 people; 23.7% chat with a maximum of 3 people; and 52% chat with more than 3 people (mean = 3.20, std. dev = 0.09, n = 173). often, topics of those concurrent conversations are different and people compose their status to reflect the topic of a conversation. one participant responded that after watching ‘shrek 2’, she often uses the ‘donkey + dragon = so cute’ status when chatting with friends about the movie. therefore, supporting multiple statuses allows im users to customise their conversations with different people at the same time. figure 2: responses to multiple status support. ajis special issue december 2004 10 boundary control we found that the way in which im allows the user to be contacted at any time is a concern. around 75% of the respondents want to control who can see them online (mean = 2.6; std. dev = 1.93; n = 149), as shown in figure 3. the results indicate that the majority of the respondents disagree with the idea of not being able to control who can see them online. currently, msn messenger provides a ‘block’ feature that allows users to stop others from knowing if the users are online. although this technique allows users to control who can see them online, it was not well received by our participants. one participant commented “i don’t like block [the block feature], the name itself sounds so negative to me”. more interestingly, on the one hand im users want to control their online visibility that is seen by others, on the other hand they do not like to be blocked by other users, as responded by one participant “i want to be able to find out who has blocked me on their buddy list, but i want to go online and appear offline”. some respondents choose to be invisible all the time because they do not want to be interrupted by other people. this was encapsulated by one respondent who stated, “i never set status to available. i often start a conversation if i need to”. it indicates that im users are very careful about policing their boundaries of the social networks created by the im community. interoperability “i am using msn since most of my mates use it” is how one respondent justified why msn is his favourite im. it reflects the fact that im users choose one im tool not only because of its functionality, but also because of the choices of other people in their virtual community. currently, there is a lack of interoperability across im networks. each im network only supports its own registered users, for instance, yahoo users cannot log into msn network and vice versa. one respondent said “i like to use trillian and gaim, because i can install them in [computer] labs and i can log in both msn and yahoo”. when the users assume mci’s across different im networks, it is crucial that they can easily switch between different identities. the majority of the participants (92%) had used im to chat with two or more people at the same time. and, almost haft of them (46%), who chat with more than one person simultaneously, had used different im tools when chatting with different people (figure 4). hence, there is a strong need for interoperability between different im networks. that is, once the users join the im community, they can assume any identity figure 3: responses to “i don’t want to control who can see me online”. figure 4: participants’ choice of im when chatting with different people. ajis special issue december 2004 11 they want without having to know the im service provider. the users’ shift from one identity to another should be seamless and transparent to im services. discussion in this section, we discuss possible approaches to tackle the five issues reported above. first, regarding interoperability, this can be seen as a technical problem, which can be resolved by appropriate resource-sharing mechanisms between im network providers. at an application level, developing systems like trillian (cerulean studios) or gaim (gaim at sourceforge), which allow users to log into many im networks simultaneously, can be one possible approach. second, boundary control is a techno-socio problem that involves balancing between users’ privacy and trust, and enforces high security. although many studies (e.g., kahin and keller 1995, lazar et al. 2001) have looked at this issue, much further work needs to be done to find viable working policies for controlling a personal virtual boundary in im. in order to be effective, policies must be strong enough to protect users’ privacy but still flexible enough to adapt to the rapid change of the community (e.g., change of users’ behaviour, change of users’ trust one to another, etc.). third, to support a single nickname, multiple avatars and multiple statuses, we propose a framework that can support the one-to-many relationship between a user’s username and the user’s many identities. as shown in the class diagram in figure 5, our framework extends the current framework of im by adding one-to-many and many-to-many associations between different im components. for examples, • associations between ‘username’ and ‘avatar’, and between ‘username’ and ‘status’ are one-to-many (1…*) relationships. that means each username can use many avatars and edit many online statuses; • associations between ‘group’ and ‘avatar’, and between ‘group’ and ‘status’ are many-tomany relationships. that means each group in a user’s buddy list can use many avatars and each avatar can be used by different groups in the buddy list; • associations between ‘avatar’ and ‘nickname’, and between ‘status’ and ‘nickname’ are one-to-many relationships. that means only one avatar and one status can be set for each nickname (i.e., a person appears in a buddy list) at a time. however, any avatar and status can be shared among many nicknames. ajis special issue december 2004 12 in the remainder of our discussion, we briefly present the interpretations of our framework in designing im tools. drawing on the framework, we worked together with the participants of our interviews to design a low-fidelity prototype of an im tool using a paper-prototyping technique. the purpose of our paper-based prototype is to sketch out ideas and possible user interface (ui) to support mci’s. after being happy with an overall layout in a paper prototype, we quickly transformed the layout into a higher fidelity prototype. figure 6 shows an example of high-fidelity ui after being converted from a low-fidelity paper-based prototype. in figure 6, a chat window contains a conversation between two users: local user and remote user. their details are illustrated in table 1. the prototype shown in figure 6 supports multiple avatars and multiple statuses by allowing a local user to set different avatar and different status for each conversation. for example, minh (i.e., a local user) has options to change another avatar and another status. when a change is made, new avatar and new status are merely applied to a current conversation (i.e., conversation between minh and darkangel) by default. however, if minh wants, he also can set a new avatar and new status as global; and they are automatically applied to all conversations between minh and people other than darkangel. in a situation when a global avatar and a global status are used, it is similar to what being implemented in current im tools. however, in more general situations, minh can display more than one avatar and edit more than one status. figure 5: class diagram shows the framework of mci support. ajis special issue december 2004 13 the mock-up screenshot in figure 6 is the result of our very early work in designing ui, thus it is not completed in any way. however, it does show essential features that are lacking in current im tools. and more importantly, it illustrates how our proposed framework can be interpreted in ui design. conclusion the paper has reported our study of multiple concurrent identity (mci) support in the instant messaging (im) community. the study has identified five themes of single nickname, multiple concurrent avatars, multiple concurrent statuses, boundary control and interoperability. a strong message growing out of our study is that in many cases im users want to represent themselves differently when interacting with different people simultaneously. the study shows strong needs for providing mci’s in im. mci’s can be supported in many ways such as allowing users to use different avatars and different statuses at the same time. based on the results of this study, we proposed a new framework of mci support in im. as future work, reflecting on these five themes, we will develop personas and scenarios that can be used to translate user needs into the design process. in addition to that, we are also interested in improving support for presence awareness (i.e., information about the availability of im users) and message awareness (i.e., information about received and sent messages) in im. acknowledgements we would like to acknowledge the support of smart internet technology crc (sitcrc) australia, and thank people from swinburne usability laboratory (sul) for their valuable comments on an early draft of this paper. local user remote user nickname “minh” “darkangel” location left hand side of the window right hand side of the window status “stop raining pls .. life is not fair” “coffee at darling harbour .. coolest” avatar table 1: details of local user and remote user, who are illustrated in figure 6. ajis special issue december 2004 14 ajis special issue december 2004 15 references american online instant messenger, http://www.aim.com/, accessed december 7 2005. cerulean studios, http://www.ceruleanstudios.com/, accessed december 7 2005. donath, j. s. (1999) identity and deception in the virtual community, in m.a., smith and p. kollock (eds.) communities in cyberspace, routledge, london, pp. 29-59. gaim at sourceforge, http://gaim.sourceforge.net/, accessed december 7 2005. grinter, r. e. and palen, l. (2002) instant messaging in teen life. proc. of the acm conference on computer supported cooperative work, new orleans, acm press, new york, pp. 21-30. handel, m. and herbsleb, j. d. (2002) what is chat doing in the workplace? proc. of the acm conference on computer supported cooperative work, new orleans, acm press, new york, pp. 1-10. harasim, l. m. (1993) global networks: computers and international communication, mit press, cambridge, england. icq inc., http://www.icq.com/, accessed december 7 2005. isaacs, e., walendowski, a., whittaker, s., schiano, d. j., and kamm, c. (2002) the character, functions, and styles of instant messaging in the workplace, proc. of the conference on computer supported cooperative work, new orleans, acm press, new york, pp. 11-20. jones, s. g. (1998) cybersociety 2.0: revisiting computer-mediated communication and community, sage, thousand oaks, california, usa. kahin, b., and keller, j. (1995) public access to the internet, mit press, cambridge: ma. lazar, j., hanst, e., buchwalter, j., and preece, j. (2001) collecting user requirements in a virtual population, webnet journal: internet technologies, applications, and issues, 2(4), 20-27. mccaughey, m. and ayers, m (2003) cyberactivism: online activism in theory and practice, routledge, usa. microsoft service network messenger, http://messenger.msn.com/, accessed december 7 2005. nardi, b. a., whittaker, s., and bradner, e. (2000) interaction and outeraction: instant messaging in action. proc. of the acm conference on computer supported cooperative work, acm press, new york, pp. 79-88. nonnecke, b., preece, j., and andrews, d. (2004) what lurkers and posters think of each other, proc. of the hicss-37, maui, hawaii. o'brien, j. (1999) writing in the body: gender (re)production in online interaction, in smith, m. a. and kollock, p. (eds.). communities in cyberspace, routledge, london, pp. 76-104. perera, r. (2001) study: instant messaging at work up 110 percent, http://www.cnn.com/2001/tech/internet/11/16/workplace.im.idg/index.html, accessed december 7 2005. preece, j. (2000) online communities: designing usability, supporting sociability, john wiley & sons, england. preece, j. and maloney-krichmar, d. (2003) online communities. in j. jacko and a. sears, a. (eds.) handbook of human-computer interaction, lawrence erlbaum associates inc. publishers. mahwah: nj, pp. 596-620. renninger, k. a. and shumar, w. (eds.) (2002) building virtual communities: learning and change in cyberspace, cambridge university press, england. rheingold, h. (1993) the virtual community, addison-wesley, new york, http://www.rheingold.com/vc/book/, accessed december 7 2005. rheingold, h. (1994) a slice of life in my virtual community, in l.m. harasim (ed) global networks: computers and international communication, mit press, cambridge, pp. 57-80. ajis special issue december 2004 16 shukla, u. (2003) the future of enterprise instant messaging, http://www.expresscomputeronline.com/20030505/tech1.shtml, accessed december 7 2005. smith, m. a. and kollock, p. (eds) (1999) communities in cyberspace, routledge, london. turkle, s. (1996) life on the screen: identity in the age of the internet, weidenfeld & nicolson, london. yahoo! messenger, http://messenger.yahoo.com/, accessed december 7 2005. ajis special issue december 2004 17 appendix a a.1. demographic multiple choice questions 1. gender 2. age 3. which instant messengers have you used and how long have you been using them? (if you started using an instant messenger, say 3 years ago and have used it for, say 6 months, your response to this question would be 6 months.) − yahoo messenger − icq − aol’s instant messenger (aim) − msn/windows messenger 4. what is your most favourite instant messenger? and, why is it your most favourite messenger? (if your favourite instant messenger is not one of the following four instant messengers, please select the "other" option and specify the name of your favourite instant messenger.) − yahoo messenger − icq − aol’s instant messenger (aim) − msn/windows messenger − other (specify name) why is it your most favourite instant messenger? 5. how often do you login to instant messengers? 6. when do you login instant messengers? (your status can be offline or online) 7. do you ever login instant messengers but you set your status as offline or invisible? 8. when do you often set your status online (i.e., available, visible)? 9. for how long per day are you actively using instant messengers to chat? 10. how often do you have webcam conversations? 11. how often do you have audio conversations? 12. what is a maximum number of people you ever chat with at the same time? (can be in a group chat or separate private chats) if a maximum number of people you ever chat with is more than 1, do you ever use different instant messengers at the same time to chat with different people? (no/yes) 13. do you ever find instant messengers limited to express your ideas? 14. do you ever chat with people located at different countries? a.2. seven-point likert scale questions questions strongly disagree moderately disagree slightly disagree undecided slightly agree moderately agree strongly agree i like to have webcam conversations. when my webcam is on, i want to know who are currently viewing my webcam. ajis special issue december 2004 18 for each person in my buddy list, i want to know whether the person is able to have video conversations (i.e., whether the person has a webcam installed). having video on all the time is useful. i like to have audio conversations (i.e., voice chat). when i speak in an audio conversation, i want to know who can hear me. for each person in my buddy list, i want to know whether the person is able to have audio conversations (i.e., whether the person has microphone and speakers installed). having audio on all the time is useful. in a group audio conversation, if a person is speaking, i want to know who that person is. i feel more comfortable with text-based conversations than audio or webcam conversations. i want to set different statuses for different people in my buddy list. i don’t want to control who can see me online. i want to set different avatars for different people in my buddy list. sometimes, i want to show other people, which earlier messages of the same conversation i want to refer to. displaying the time each message was sent is important. ajis special issue december 2004 19 after i send a message, i want to know if other people actually receive my message. i want to link emoticons with correct messages sent earlier by other people. i don’t want to know geographical locations of other people. i want to know other people's local time. i want to know how long other people have been idle. i want to know how long other people have been online. knowing if a person in my buddy list is busy chatting with someone else is necessary. when i am in a group chat, i want to know a total number of messages sent by each person. (for example, how many messages were sent by person a, how many messages were sent by person b, and so on) playing sound when a friend comes online is not necessary. when i join a group chat, i want to know who were there before and have already left. when i join a group chat, i want to know who are going to join. when a conversation is getting long, i want to know which part of the conversation other people are viewing. in a group chat, i want to see pictures of other people next to their nicknames. playing sound when a friend goes offline is not necessary. ajis special issue december 2004 20 playing sound when a new message arrives is annoying. being able to send offline message is important. in a group chat, i want to send private voice messages. i want to be able to edit previous messages of the same conversation. i want to customise my emoticons. a.3. open-ended questions 1. what are your uses of audio and video chats? 2. what are your uses of avatars, online statuses and sound alerts? 3. what other features do you want when using instant messengers? microsoft word ajis10-2finala.doc ajis vol. 10 no. 2 may 2003 70 the role of information systems and technology in case management: a case study in health and welfare insurance. helen richardson information systems and technology the open polytechnic of new zealand wellington, new zealand email: helen.richardson@openpolytechnic.ac.nz beverley hope school of information management victoria university of wellington wellington, new zealand email: beverley.hope@vuw.ac.nz abstract this study reports the role of information system and technology (ist) in supporting case management at the accident compensation corporation (acc) of new zealand. case management is a managerial approach that seeks to gain improved business performance by enhancing both employee and customer satisfaction. despite millions of dollars spent annually by health, social, and insurance agencies in automating case management, little research has been conducted into the role of ist in this practice. the findings of this study show that for acc, ist’s most valuable role is enhancing the relationship between client and case manager rather than replacing it for, even after the addition of ist, the most valuable knowledge continues to accrue from the face-to-face interaction of client and case manager. the findings also show two distinct phases to the development of case managers as knowledge workers. the first phase focuses on control of the processes and the second on the delivery and sharing of information resources. keywords: case management, knowledge management, process management, quality management, customer relationship management introduction case management is a human-centred managerial approach that seeks to improve business performance by enhancing employee and client satisfaction. in the last decade case management has made a resurgence in a new form (davenport & nohria 1994; gemignai 1999). the new case management incorporates process management practices and knowledge management (km) enabled by information systems and technology (ist). in this form, it is being used in both the private and public sectors to help organisations become more responsive to customers. ist is thought to provide case managers with insight into the support process. however, little empirical research has been conducted into the role of ist in case management (srm 1999). this study examines the research question: how does information systems and technology support effective case management? to address this question we undertook an organization-wide study at accident compensation corporation of new zealand (acc), a major public sector health insurer. the study sought to determine the strategic objectives of ist use and how that strategy filtered down through the organisation. the work is founded in the theory of quality management and knowledge work, and the results are considered to have relevance to both practitioners and academics the remainder of this paper is presented in five sections. first we review the relevant literature, and then we present the research design. the findings report on the transformation of public servants into knowledge workers, the role of ist in turning case managers into knowledge workers, the improvement of client relations through improved knowledge, and concludes with eight necessary ist practices. in the discussion section the findings are related to the literature and to practice. literature review case management case management, like so many terms, can mean different things to different people. for example, in the health and social sector the focus is on the client’s rehabilitation as in gill (2000). a process directed at coordinating resources and creating flexible, cost-effective health-care options in collaboration with the treatment team for individuals and their families, to facilitate optimum care. (gill 2000) ajis vol. 10 no. 2 may 2003 71 in the newer business applications, where powerful ist systems are used to provide case managers with the information they need, the emphasis is on process efficiency as davenport & nohria (1994) describe: the case manager role represents a break with the conventional approach to the division of work. individuals or small teams perform a series of tasks from beginning to end, often with the help of information systems that reach through the organisation. case managers provide a way to increase organisational efficiency, timeliness, and customer satisfaction. (davenport & nohria 1994, p.1). to understand these differences, it is instructive to briefly review the development of case management. the practice first appeared in the1920’s in the health and social sectors (biestek 1957; davenport & nohria, 1994). however it was not an influential feature of welfare programmes until the 1960’s when it became recognized and often directly funded (walker 1996; gill 2000). in the 1990's case management was taken up by some businesses that sought to compete, or merely to survive, following the 1987 stock market crash (abell 2000). new managerial approaches and restructuring sought to turn companies into lean, innovative and adaptive organisations that could fight to survive in the new environment. one of the most famous approaches was bpr. case management in business came to be seen as a derivative of bpr in as much as both are process orientated, and both sought to improve efficiency and client service (davenport & nohria 1994). during the 1990's recession, governments realised that they, too, must downsize and focus on core competencies; it was no longer acceptable to run inefficiently (blackwell 1993; johnston, romzek & barbara 1999). this was particularly evident in new zealand which led the world in economic reform (myers 1996; eggers 1997). furthermore, the rise in welfare benefit claims during the recession placed an increased strain on the government resources (halachmi 1995; walker 1996). this led government agencies into an era of efficiency and they began to follow the lead of their private sector counterparts with new management approaches. case management philosophy centres on building a lasting and effective relationship between case manager and client (biestek 1957). a client focus is kept as one worker or one team performs a service rather than allowing it to pass through numerous functional units (davenport & nohria 1994). service quality is further enhanced through improved satisfaction of employees (davenport & nohria 1994). this is consistent with heskett et al’s (1994) “putting the service profit chain to work.” in this organisational culture, the employee becomes an asset or ‘human capital’ to be supported, encouraged, and enhanced. part of this support will be provision of necessary information to complete the job. thus, case management combines the principles of quality management, process management, and km, and it provides the support. process management and case management the economic transformation of post-war japan was founded upon a focus on process control backed by a supportive management philosophy of customer focus, long-term focus, objectivity, and teamwork. in the 1980’s total quality management gurus such as deming (1986) and crosby (1979) began to popularise this approach in the united states. the new approach was shown to not only improve quality, but also to reduce costs. proponents of bpr subsequently picked up the process focus. bpr highlighted the process focus, paid some attention to customer focus, but tended in practice to overlook employee contributions and satisfaction. bpr was often used to justify mass redundancies or downsizing (klein 1994; uzzi 1995). in these cases ist was employed to eliminate, not enhance, the human component (manion 1995) and dumbsizing ensued as people walked out the door, taking their knowledge with them (eggers, 1997; rayburn & rayburn, 1999). despite many failures, a small number of companies succeeded by adopting human-centred approaches (senge 1995). case management was one such approach (blackwell 1993; halachmi 1995; zaidifard 1998). the human-centred focus of case management was enabled by new ist that could simplify procedures and centralise decision-making knowledge (carey 1993; hammer & champy 1993). empowered employees became involved in decision-making and created unique flexible solutions for clients (hammer & champy 1993). the strategy was aided by the rise of knowledge work which some dubbed ‘post-modern reengineering’ (manville & foote 1996). knowledge management and case management the concept of knowledge work was spawned in 1966 by peter drucker, who observed job changes in whitecollar work following the introduction of ist (drucker 1993). computers allowed for the refining and organising of data such that it added value by producing information that could be used in new ways. to ajis vol. 10 no. 2 may 2003 72 manage this new resource academics sought to define knowledge. davenport and prusak (1998, p.1) defined it as: information combined with experience, context, interpretation, and reflection. two types of knowledge are recognised: tacit and explicit. tacit knowledge is “know-how possessed by individuals. it is often intuitive and demonstrated more in how someone goes about work in a knowledgeable way.” explicit knowledge is: “systematically documented know-how that becomes available to everyone in an organisation” (cortada & woods 1999, p. xi). knowledge management is based around three key notions: electronic information storage, the knowledge worker, and knowledge management (lytras, athanasia, and poulymenakou 2002). ist is central to the concept of km as it increases the ability to store, process, retrieve, and distribute knowledge (thom 1996). after an era in which database systems churned out endless reports that nobody read, academics and practitioners realised that knowledge was more than summarized data; it included sharing and development of unique insights (thom 1996; davenport, de long & beers 1998; huber, davenport & king 1998). km technologies include data warehousing, data mining, groupware, intranets, extranets, web portals, document and intellectual capital applications, and scanning technologies (orna 1999). but km, too, has failed for many organisations and practitioners are beginning to realise that what is important is how the technologies are used (bhatt 2002). km is not the sinking of information into a “hum of servers, software and pipes” (hildebrand 1999), but is the use of ist to transform data and information into knowledge. the objective of km is to make the knowledge worker more productive (drucker 1999; gao, li & nakamori 2002). organisational success rests in the hands of competent, motivated knowledge workers who understand the importance of using information to further business goals. to achieve full productivity, knowledge workers need self-autonomy, the ability to innovate, the chance to continuously learn, and recognition that quality is at least as important as quantity (drucker 1999). job satisfaction is also important. ist-enhanced case management can turn mundane tasks into satisfying tasks by empowering workers to be proactive (maniccia 1999). in this environment, workers become an asset rather than a cost (drucker 1999; ulrich 1998). knowledge management is a pre-requisite to any activity involving knowledge sharing (rowley & farrow 2000). it involves not only the storage of information, but also the ability to retrieve it in a useful form (mccune 1999). but this idea goes deeper incorporating an ability to have the best flow of information (parsons 1996; fahey & prusak 1998; gao, li & nakamori 2002). changing the information flows can solve problems of organisational change by breaking down functional stores of knowledge. in summary, the new case management is a knowledge-based form of process management enabled by ist. nevertheless, the primary focus in case management is not on ist nor on processes, but rather on the relationship between the worker and the client (beistek 1957; gill 2000). many of the critics of bpr attributed its mass failure to implementations being technology-focused rather than human-focused. case management is an approach that faces this criticism head on. in contrast to bpr, km emphasises the freeing up of employees and creation of an environment for learning. case management embraces both these views and thus may offer a new methodology for the knowledge age. research design this research was bounded by a three-month period yet sought to gain a holistic insight that would end in a rich description to portray the reality of the organisation. the chosen method was interpretive case research which allows for the study of contemporary events of a single social unit in a natural setting within a bounded time and duration whilst allowing for an intensive, holistic description (yin 1979; merriam 1988). the principal researcher was the primary instrument for data collection and analysis. data were mediated through the researcher, who clarified and summarised data as the study continued. this was enhanced by the researcher reflecting on her own work experience and using it to negotiate a shared reality with participants. this also aided in collection of sensitive data due to the trust-building associated with this activity. data collection and analysis acc is one of several new zealand government agencies that have adopted case management. it was selected for this study because it had experienced success with case management through investment in ist. acc is a government-administered organisation, established in 1974 to provide a comprehensive range of benefits for accident injuries. compensation is payable regardless of fault, making it a form of compulsory social insurance. changes to government policy in the mid-90’s placed acc in an environment where they had to compete commercially. survival required ist be brought up to the industry standard with specialised ajis vol. 10 no. 2 may 2003 73 software, call centres, intranets, and extranets. around the same time there was an internal shift in focus from claims processing to aiding recovery. acc adopted the practice of it-enabled case management and by 2000 had achieved business success participants for the study were selected from different levels within acc to gain a holistic view. these included the ceo, three ist/business development managers, and four case managers. clients were not included due to privacy concerns. mangers in acc undertook the selection of case managers because of the highly sensitive nature of the work. a major criterion in selecting case managers was that they be competent with the computer system used. two of the four case managers were power uses of ist, while the other two were confident users. all had long-term experience as case managers within this or another organisation. participants include both males and females and a variety of cultural groups. information was gathered from interviews held in a meeting room and from limited observations of computer screens and workstations. the hour-long interviews were semi-structured with open-ended questions designed to encourage disclosure. a grand tour question was used, with a series of subquestions designed to prompt participants if issues of interest did not naturally arise. questions were developed using a combination of merriam (1988) and spradley's (1979) categories. data analysis occurred simultaneously with data collections, starting at first contact and ending on completion of writing of the rich description. data analysis involved looking for patterns, which were not known in advance but were left to emerge from the analysis. significant meaning was gained from single instances as well as from repeated comments (stake 1995). the researcher continually checked with participants to verify these patterns. this approach to analysis is taken from de laine (1997), and involved the selection and definition of the phenomena, and the incorporation of individual findings into a rich description. interviews were coded with single words or phrases that tied related concepts together (de laine 1997). internal validity for this interpretive study was viewed as the accuracy of information and whether it matched reality. triangulation using different sources was not possible due to the sensitive nature of work and prohibition on observation and document collection. instead the study employed: triangulation within and across groups: categories were continually checked between members of the same group (same employment category) and across different groups. this led to categories being confirmed or deleted. checking validity of interpretations: this was achieved in follow-up meetings to ensure that the categories reflected the participants' reality. acknowledgment of researcher bias. this was achieved by keeping a journal to acknowledge the researcher's emotional state at the time of interviews and later to debrief the researcher prior to transcribing and coding. checking for plausibility and credibility. the researcher drew on her own work experience in a similar job to test if findings were plausible. generalisability for this study comes from the ability of the rich description to create a portrait of reality for this organisation that allows the reader to reflect on their knowledge and experience and gain insight beyond this single case (merriam 1988). the second form of generalisability is seen in the ability to extract hypotheses from this research for future studies, both qualitative and quantitative (yin 1979). this research has provided multiple concepts in the areas of case management and knowledge work that can be later expanded with further research. the findings the transformation of public servants to knowledge workers from a prior position of deficit, acc has created budget surpluses in the past three years that are greater than telecom's profits. this transformation is due to the vision of the ceo who has taken acc on a journey that has transformed public servants into professional knowledge workers. one of the keys to success has been the supporting of case management with ist. turning case managers into knowledge workers with ist for case managers at acc the use of ist has been empowering, but not in the traditional sense of increased flexibility and innovative work practices. rather, ist has allowed case managers to present themselves as professionals to the clients they serve. as one case manager commented: ajis vol. 10 no. 2 may 2003 74 initially you get very intimidated by new systems, especially when you are used to working the old way handwriting things down. you think, “this is the paperless world.” to me [the new system] is very helpful. it is so much easier to deal with the client on a professional level. you have it there at your fingertips a full description of the [previous] conversation was with that client…. for instance, it brings up an automatic date, and you can say, “yes, well i spoke to you on the 18th”. it portrays an image that we are accurate with information gathering which is important…. that we have taken notice. in the past, when a client asked a question the case manager had to sort through paper files … and often returned with the answer "i don’t know." the client's response was "you don't know and don't care." this signalled the closing off of a cooperative relationship between case manager and client. (case manager 4). now case managers can type into their desktop computer and find out exactly what the situation is for their client or for a colleague's client as one case manager observed: i can access anyone’s clients, if i am dealing with that client. [the system] is user friendly, i know what to do…. it’s time saving. instead of having to go physically to find a file i can look up the activity log of the client it is there, the whole activity log …. when they phone, and we can answer them it’s much more helpful than “er, hold on.” (case manager 3) it is also useful for managing the managers, as the ceo observed: it's putting in intelligence, and checking standards against the practice of case managers. so it can be used as a tool to manage claimants but also as a tool to manage case managers. (ceo) so, the key to success has been the control of information. in the past information was lost. not only accidentally taken away by a cleaner or knocked down behind a cabinet but lost in files, lost between case managers, lost in another branch. ist is being used to help locate information, by providing standardisation, control, and access. asked about the role of it in supporting case managers, the ceo responded: the main role is to provide information for the case manager that is fresh and current. the case manager is dealing with the individual in real time and information needs to be available in real-time. we are developing a system called pathway where all interaction is available along the pathway nationwide to anyone. so if i go on holiday to invercargill and hurt myself, but live in whangarai, wherever i am my details move with me…. the real role of it is to have information available. so the case manager can pick it up and say, “what does this woman need? where's she at? where are they in the process?” (ceo) improved client relations knowledge as a relationship for some years, acc has had to function in the shadow of their past image, trying to prove to clients that there is a new way of doing business. the information that ist provides is helping clients see the new corporation. so, ist has been an important part in rebuilding acc’s image. electronic control of information has created stronger bonds with clients. as a case manager explains: [frequently] when i write a letter, i cut and paste it into the system as well. it’s really good because if the client phones and says, “you didn’t send me that letter,” i can go into the system and say “on the 17th of july i sent you this letter, and i've got copy here”…. or, if someone comes to see us they can see it on the system. the interview rooms down stairs all have computers, so we can show it to them, which is good. it makes them feel good because they think, "oh, you are telling me the truth, you’re not just saying to me that you did". (case manager 3) case managers now have more knowledge at their fingertips, and they are using this knowledge in intelligent ways. they do not use the system to make decisions; they use it to negotiate with the client. a client may want something that acc cannot deliver, but the case manager can access the client's file to identify other areas to work on. so, ist has aided the human face of case management. people come in battered, bruised and bewildered, case managers help them gain their legal entitlements, get help, and re-motivate them back to work. a difficult task when you are managing 100 clients, but a task made possible by ist. a case manager explains ajis vol. 10 no. 2 may 2003 75 in my team, the tail team, we are dealing with people who have been [clients] 10 to 15 years. it has become the longest job they have ever held, acc. you have to be patient and work with them and all the service providers as well. we work with vocational assessors. we try to maintain good relationships with clients’ gps…. like, where can we refer clients for specialist reports so we can get a full picture for the claimant and the gp…. it makes it easier for us. (case manager 1) so at the front line, case managers have found ist provides them with the ability to function professionally and develop closer relationships with clients. however, the system at acc was not developed by chance. behind this improvement has been a story of putting in place the right ist practices. the right ist practices sound ist management practices have helped acc achieve the transition from public servant to professional knowledge workers. these practices are: leadership and vision, linking ist to the vision, inculcating a human centred strategy, harvesting ist, employing a flexible architecture, standardising business processes, understanding the core business, and fostering the right culture. 1. leadership and vision change management and system development literatures call for a champion to lead the change. acc’s ceo is such a champion. he has played an important role in inspiring workers to move forward into a new era of professionalism. his vision for the company sees a ‘virtuous circle’ of staff satisfaction, customer satisfaction, and improved business performance, and ist has been used with this vision in mind. the ceo understood that ist was the only effective way to achieve standardisation and control. furthermore, he inspired case managers so that they recognized that the new controls were designed not to hinder them but rather to cause the corporation run smoothly. it is no good having case management if you do not know what you’re doing or you do not have good control systems…. as we’ve tightened our control systems, we've had remarkable gains. we have reduced costs by about 50% in 3 years … and that’s due to case management. but you need clarity concerning what you want to do. we spent a lot of time talking to the case managers so that they understand the vision of the company…. you cannot be beside them all the time when they are making those calls. and later … what we have found is that if you do it right the first time, if you do it quickly the first time, you are very likely to reduce cost and get greater customer satisfaction, a happy customer, a virtuous circle. the it part of this is to provide a tool to support case managers. (ceo) a case manner confirms this, with: it has made us more conscious of our statistics on the kpis by which they measure our performance [so] it is not only helping case management, it is helping at head office. (case manager 4) 2. linking ist to the vision from day one, ist has been linked with business strategy. initially the ist people worked in the same building as the business people each learning from the other. now separated, the bonds remain strong. each has a rich understanding of the other's expertise. on the business side, care was taken to be not only ‘top down,’ driven by the ceo and general managers, but also to go back to the case mangers and see how it is working. so, acc has achieved the normative model of ist strategy, top down bottom up. employing a former case manager as head of the business team has helped in understanding case managers’ needs. as a result of restructuring we ended up working here alongside the technology people…. that was a key thing that we had business people and technology people in the same place, working together. the technical were learning about the business, and the business were learning about the options. (manager 1) ajis vol. 10 no. 2 may 2003 76 asked about future ist plans for acc, the ceo responded: making more and more systems available on-time, on-line to case managers…. another plan is to use e-commerce more and more that’s the big strategy. most of our interfaces with doctors, such as ordering forms and receiving forms, are on-line. which is quite good. we get claims in real-time and notification of accidents in real-time. so, instead of waiting 7-8 days for the practice nurse to bundle them up and put them in the envelope, we are getting them the next day. 3. supporting a human-centred strategy ist has enhanced the way people work. it has helped managers to handle large numbers of clients whilst still focusing on each one as an individual. as previously mentioned, ist has been linked with business strategy, and the strategy at acc is focused on supporting employees in performing their job. the ceo commented: if you just let god do his work and let people motivate themselves … a large proportion will make it back [to work] themselves. we are looking at the marginal peoples who need a bit of a hand to get that way. the other issue is that people have an entitlement, in law, to acc care. but some people get confused and bewildered by the process. if you have a traumatic brain injury as a consequence of a car accident or severe concussion and you need physiotherapy or help to re-jig your life, you really do need assistance…the earlier we can get people in to be re-crafted, the better. 4. harvesting ist acc has adopted a lot of the latest technology including intranets, and extranets. their aim, however, is to keep current without using risky technology. we are leading edge, not bleeding edge. (manager) acc has plans for advancing the technology available to case managers and for supporting processes but it is not rushing into new initiatives. rather, acc is working to understand business needs and ensuring functionality prior to purchase. this is seen in their adoption of the internet. access has been tightly controlled so that the business is supported, not hindered, by access. 5. employing a flexible architecture businesses in rapidly changing environments need flexible it architectures. no new zealand government agency has had to change more than acc to meet the needs of new legislation. in many respects, their environment is more chaotic than that of businesses downtown. their flexible it architecture allows rapid adaptation. as one manager said: [our it system] has to be responsive to changing business requirements. particularly at acc, where changes of government result in complete changes in legislation, so we have to completely rip out what we put in a few years ago…. we have to be very adaptive. saying we can't change it till a year’s time is not good enough. fortunately we have a platform that is very flexible. it can talk to completely different databases all over the place. and that was not accidental. (manager 2) 6. standardising business processes in the past case managers steered away from adherence to the relentless stream of legislated requirements that is part of being a government department. however non-standardisation led to less efficient work practices that could delay recovery for a client or result in underpayment or overpayment of claims. in the new standardised system, the more routine aspects of claims such as required legislative payments are calculated automatically, making for improved accuracy and adherence to legislation. in addition, the system keeps track of case managers, enabling supervisors to identify and support those who are not working productively. the application of iso 9001 standards helps ensure that established good processes are maintained. as the ceo commented one of the keys is to get control over processes. you can have flexibility when you have the process in control. chaos is not flexibility; chaos is always chaos. there is a huge ajis vol. 10 no. 2 may 2003 77 misunderstanding, that flexibility is good. flexibility is good, but only when it should and ought to be applied…. when i took over this organisation it was in chaos, not in control. the guys were trying to run self-managing teams. no one was taking responsibility for anyone. so there was heaps of flexibility, but there was heaps of fear because people did not know the right way to do things. (ceo) 7. understanding the core business one of the ways acc has coped with change is by separating the core business from the change. staff felt proud that throughout recent changes they managed to keep going business as usual. this has been possible because of their ability to isolate the change in a project. for example, when the new labour government brought back lump sum payments, acc’s reaction was, “it does not effect day-to-day business. it is just another system.” as the ceo described it: the classic to date is the reintroduction of lump sum payments for accident victims.… that will require development of an entire new it system to support case management of lump sums, which is a fun exercise.… [but] that’s all right. the last change in legislation we ran something like 32 projects… governments come and governments go, the main thing is to make sure you are not running on the spot all the time. we had a good run last year. we managed to keep focused on the key dynamics, while changing at the top. we tried to protect the organisation from [the change] and to keep them focused on the core discipline. … it's easier if you are project orientated. we were able to set up project teams to isolate the change process from the operational process… the greater the extent you are able to keep them separated the greater the efficiency you get during the change. 8. fostering the right culture in moving from a public service culture to a business culture, acc has maintained the most important requirement of those serving the public's interest, interest in helping people. case managers have chosen this career not only to make a living, but also to make positive changes for the people they manage. in other privatised agencies, privatisation has often led to a bottom dollar focus. asked why they became case managers, one responded: i ask myself that question? (laughter). um, probably because i was a claimant myself. i know what it feels like to be on the other side of the door. i want to help people and i enjoy what i do. (case manager 2) and from another … many of the clients are agro (aggressive) and volatile. but there are a lot who aren't. and there are people who are very grateful. and that is wonderful a huge pat on the back. “hey, i have made a difference to that person. it has gone smoothly while he hasn't been able to work.” ... i just like to know i made a difference. end of story. i don't know how else to explain it. (case manager 4). the reason for success has been that acc understood that cost savings could be made by doing things more quickly that by gaining faster access to surgery, people could get back to work sooner. case management ensures that information does not get lost between departments and ist has taken this further by allowing claims to be managed, no matter where the client is located. a manager explains: i see ist as providing a system to enable case mangers to track and manage claims. to manage rehab to the optimum. the ultimate goal is to get people to work and get them well.… making sure that a standard rehab for an injury is happening…. so getting the right entitlement is a key objective…. it’s all driven by [the ceos] vision of reducing costs and through reduced durations of claims, achieving customer and staff satisfaction. (manager 2) in summary, case management has resulted in a more proactive stance at acc. clients are assessed both physically and vocationally. without ist, the successful coordination of treatments would be difficult. appointments might be missed, claimants might sit forgotten at the bottom of a pile of claims, and an environment of information chaos could exist. with ist, they have the information they need to do their jobs. ajis vol. 10 no. 2 may 2003 78 ist has enabled standardisation by supplying all workers with core knowledge. furthermore, converting information into an electronic format has speeded up information flow. but above all, ist at acc has enriched the manager-client relationship. case management, coupled with ist has converted public servants into knowledge workers who care. discussion we have seen little discussion of case management in the business literature, particularly in terms of how ist is applied to this practice. yet, every year, in hospitals and social welfare agencies, millions of dollars are spent on automating this practice. this study provides some insight into this phenomenon. theoretical discussion of bpr and km is prolific but it has been difficult to translate this into practice (davenport, eccles & prusak 1992; harkness, kettinger & segars 1996). case management brings these two streams together in a cohesive practice and addresses many of the shortfalls that practitioners have found with them individually. bpr and km can be seen to be at opposite ends of a spectrum, with bpr ‘dumbsizing’ organisations while km creates workers who are innovative and creative. bpr ignored the humans in the system and km did not control the information flow leading to potential information overload (knights and mccabe 1998; mccune 1999; orna 1999). together, as in case management, they provide balance, allowing managers to focus efforts on caring, going beyond client satisfaction to relationships formation and mentoring (biestek 1957). projects to implement ist on the scale undertaken at acc cannot be achieved overnight. not only must the workflow change, organisational culture must change. from our study of acc, we learned that km is a twostage process. first the organisation must get control of their information, and only then can it move to the next level, that of providing access that develops individual knowledge. the first phase is the long haul and includes getting workers to use the system and getting them to standardise practices. this research provides a picture of a company moving into the knowledge age. it demonstrates the two phases to knowledge work and suggests some best business and ist practices to be followed. by building pictures of such cases, a useful methodology can be articulated. in this way, we can provide a clearer picture of what needs to happen to achieve success. this study also provides insight into the concept of knowledge itself and how best to leverage it. the study supported the view of knowledge as a social phenomena (polanyi 1958; nonaka & konno 1998), that knowledge in its richest form is not stored in an individual’s head nor held electronically, but is created during face-to-face interaction. face-to-face interaction cannot be stored electronically but can be supported by ist through provision of accurate and timely information. in the past too much emphasis may have been placed on the electronic side of km, and too little on supporting the face-to-face relationship. the research has implications for acc and the industry as a whole. acc has managed to change public servants into knowledge workers, clerical workers into professionals. recognition of their professional status is needed. case management associations are calling for professional standards worldwide to help ensure that people are appropriately qualified and trained to undertake a job that carries with it a social responsibility (richardson 1997; gill 2000). the study showed that case managers are an important source of ideas. once focused on rehabilitation, they seek tools to help them do their job more effectively. in an age where people understand the potential of technology, case managers can provide suggestions not only of their needs but also of how technology can be applied to these needs. as their skills in handling client relationships develop, successful interactions may be shared on internal systems. this will enable valuable tacit knowledge to be shared, particularly with novice case managers. acc may wish to be more active in collecting this valuable information. in summary, ist has been used at acc as part of a case management approach that has enabled public servants to become knowledge workers. this has been achieved by gaining control of the underlying processes that support case managers, freeing case managers to concentrate on their clients. ist systems are built around this relationship to enable rather than eliminate it. this research was exploratory, and a more in-depth study of case management is needed, particularly to see how widespread are the practices found at acc. comparisons with other organisations who have used ist in case management would add further to our knowledge. it is recommended that data be collected with interviews. in the current study, interviews were found to be effective providing rich information that would be hard to elicit from surveys because knowledge is highly personal and is best negotiated face-to-face. ajis vol. 10 no. 2 may 2003 79 references abell, a. (2000). skills for knowledge environments. the information management journal, 34(3), 33-41. bhatt, g. d. (2002). management strategies for individual knowledge and organizational knowledge. journal of knowledge management, 6(1), 31-39. biestek, f.p. (1957). the casework relationship. chicago: loyola university press. blackwell, g. (1993). re-engineering the claims process. i.t. magazine, 25(1), 14-18. carey, d. (1993). nine i.t. executives tell you how to get reengineering success. i.t. magazine, 25(11), 12-20. collins, j.c. (1995). change is good but first, know what should never change. fortune, 131(10), 141. cortada, j.w. & woods, j.a. (1999). the knowledge management yearbook 1999 2000. boston: butterworth-heinemann. crosby, p. b. (1979). quality is free. new york, ny: mentor books. davenport, t.h. (1995). the fad that forgot people: why reengineering failed. fast company, 1(1), 70-74. davenport, t.h., de long, d.w & beers, m.c. (1998). successful knowledge management projects. sloan management review, 39(2), 43-57. davenport, t.h., eccles, r.g. & prusak, l. (1992). information politics. sloan management review, 34(1), 53-65. davenport, t.h. & nohria, n. (1994). case management and the integration of labor. sloan management review, 35(2), 11-23. davenport, t.h. & prusak, l. (1998). working knowledge. executive excellence, 15(9), 10. de laine, m. (1997). ethnography: theory and application in health research. sydney: maclennan+petty. deming, w. e. (1986). out of the crisis. cambridge, ma: massachusetts institute of technology. drucker, p.f. (1993). managing for the future: the 1990s and beyond. new york: truman talley books/plume. drucker, p.f. (1999). knowledge-worker productivity: the biggest challenge, california management review, 41(2), 79-94. eggers, w.d. (1997). the wonder down under. government executive, 29(3), 32-39. fahey, l. & prusak, l. (1998). the eleven deadliest sins of knowledge management. california management review, 40(3), 265-276. gao, f., li, m. & nakamori, y. (2002). systems thinking on knowledge and its management: systems methodology for knowledge management. journal of knowledge management, 6(6), 7-17. gemignai, j. (1999). the case management explosion. business & health, 17(10), 56. gill, a. (2000). case management. recovery, 5(1), 1-3. halachmi, a. (1995). re-engineering the disability determination process: a case study. work study, 44(8), 21-27. hammer, m. and champy, j. (1993). reengineering the corporation. new york: harper business. harkness, w.l., kettinger, w.j. & segars, a.h. (1996). sustaining process improvement and innovation in the information services function: lessons learned at the bose corporation. mis quarterly, 20(3), 349-368. heskett, j. l., jones, t. o., loveman, g. w., sasser, w. e, jr., & schlesinger, l. a. (1994). putting the service-profit chain to work. harvard business review, 72(2), 164-174. hildebrand, c. (1999). does it=km? a dangerous idea could retard your km efforts. cio enterprise, september 15th. huber, g.p., davenport, t.h. & king, d.r. (1998). perspectives on organisational memory. proceedings of the thirty-first annual hicss conference, hawaii, 1-19. johnston, j., romzek, m. & barbara, s. (1999). contracting and accountability in state medicaid reform: rhetoric, theories, and reality. public administration review, 59(5), 383-399. klein, m.m. (1994). reengineering methodologies and tools. information systems management, 11(2), 30-35. kleiner, a. (2000). revisiting reengineering. strategy + business, (20), 5-10. ajis vol. 10 no. 2 may 2003 80 knights, d. & mccabe, d. (1998). when “life is but a dream”: obliterating politics through business reengineering? human relations, 51(6), 761-798. lytras, m.d., athanasia, p. & poulymenakou, a. (2002). knowledge management convergence expanding learning frontiers. journal of knowledge management, 6(1), 40-51. mccune, j.c. (1999). thirst for knowledge. management review, 88(4), 10-12. mcdonald, t. (1998). holding your aces. successful meetings, 47(11), 31. maniccia, m.d. (1999). managed care and the two-headed monster. compensation & benefits management, 15(4), 32-35. manion, r.c. (1995). the new wave of business change. the canadian business review, 22(22), 39-40. manville, b. & foote, n. (1996). harvest your workers' knowledge. datamation, 42(13), 78-82. merriam, s.b. (1988). case study research in education: a qualitative approach. san francissco: jossey-bass publishers. myers, m.d. (1996). can kiwis fly?: computing in new zealand. communications of the acm, 39(4), 11-15. nonaka, i. & konno, n. (1998). the concept of "ba": building a foundation for knowledge creation. in j.w. cortada, & j.a. woods, (eds.) the knowledge management yearbook 1999 2000. ). boston: butterworth-heinemann. orna, e. (1999). practical information policies: what every organisation needs to know. hampshire: gower. papows, j. (1998). enterprise.com. reading: perscus books. parsons, j. (1996). information the fourth resource. in the fourth resource: information and its management. d.p. best, (ed.), hampshire: aslibgower. polanyi, m. (1958). personal knowledge : towards a post-critical philosophy. london: routledge & kega. rayburn, j.m. & rayburn, l.g. (1999). smart alternatives to downsizing. competitiveness review, 9(2), 49-57. richardson, d. (1997). case management in workers’ compensation: moving from the micro to the macro. the case manager, 8(2), 63-65. rowley, j & farrow, j. (2000). organizing knowledge: an introduction to managing access to information. hampshire: gower publishing limited. senge, p. (1995). making a better world. executive excellence, 12(8), 18-19. spradley, j.p. (1979). the ethnographic interview. new york: holt, rinehart and winston. srm (shared resource management inc). (1999). leveraging technology to implement knowledge management for the case worker. an srm white paper, april. stake, r.e. (1995). the art of case study research. thousand oaks: sage publications. thom, w. (1996). managing the fourth resource. in d.p. best, (ed.) the fourth resource: information and its management. hampshire: aslibgower. ulrich, d. (1998). intellectual capital equals competence x commitment. sloan management review, 39(2), 15-26. uzzi, j. (1995). reengineering doesn't have to be a dirty word. national underwriter (property & casualty/risk & benefits management), 99(15), 21. yin, r.k. (1979). case study research: design and methods. newbury park, ca: sage. walker, h.m. (1996). case management in acc : what influences its effectiveness? research paper, public policy. m.p.p. (victoria university of wellington). zaidifard, d.r. (1998). reframing the behavioural analysis of re-engineering: an exploratory case. journal of information technology, 13(2), 127-138. copyright helen richardson and beverley hope © 2002. the authors assign to acis and educational and non-profit institutions a non-exclusive licence to use this document for personal use and in courses of instruction provided that the article is used in full and this copyright statement is reproduced. the authors also grant a non-exclusive licence to acis to publish this document in full in the conference papers and proceedings. those documents may be published on the world wide web, cd-rom, in printed form, and on mirror sites on the world wide web. any other usage is prohibited without the express permission of the authors. microsoft word ajisfinal2003.doc australasian journal of information systems special issue 2003/2004 40 continuous integration and quality assurance: a case study of two open source projects jesper holck copenhagen business school department of informatics howitzvej 60 dk-2000 frederiksberg denmark jeh.inf@cbs.dk niels jørgensen roskilde university computer science universitetsvej 1 dk-4000 roskilde denmark nielsj@ruc.dk abstract a decentralized variant of continuous integration can be defined in terms of two fundamental rules: (1) developers’ access to add contributions to the development version at any time, and (2) developers’ obligation to integrate their own contributions properly. decentralized, continuous integration may adapt well to organizations where developers work relatively independently, as in many open source projects. the approach raises the issue of how these organizations can exercise central control, as attaining the benefits of continuous integration requires that contributions are useful and satisfy the project’s definition of successful integration. we have investigated the use of continuous integration in freebsd and mozilla. our account of quality assurance activities in the two open source projects distinguishes between mintzberg’s three complementary forms of central control: standardization and control of work output, work processes, and worker skills. our study indicates that two major challenges face projects using decentralized, continuous integration: (1) to balance the access to add contributions against the need to stabilize and mature the software prior to a release, and (2) to consider the developers’ limited time and resources when interpreting their obligation to integrate their changes properly. introduction in the term ‘continuous integration’, integration refers to assembly of software parts and continuous to the absence of time-constraints. several development methods label certain activities as continuous integration, including unified process (jacobson, booch, & rumbaugh, 1999), and extreme programming, where it is one of 12 recommended best practices (beck, 1999). continuous integration may supplement a phased approach, where ‘continuous’ refers to the way integration takes place during a phase of integration and tests; or it may be part of iterative methods with (as in xp) or without (as in freebsd and mozilla) processes prescribing that modules and their interfaces are designed and documented prior to implementation. by ‘integration’ we understand more specifically all activities required to assemble a complete system of software from its parts (herbsleb & grinter, 1999), and by ‘continuous’ we understand that developers integrate frequently and throughout all phases in the development life cycle. in this broader sense, continuous integration is used in open source as well as in commercial projects. the definition comprises both relatively controlled processes, as used in microsoft (cusumano & selby, 1997) with a daily build cycle and central management of the build process (most notably of correcting a broken build), and less structured processes as used in mozilla and freebsd. the latter, more decentralized approach has the following two characteristics: first, the developers’ access to add software contributions to the development branch at any time, and second, the developers’ obligation to integrate their own contributions properly. this approach is decentralized in the sense that both the decision of when to integrate and the responsibility for a successful integration are delegated to the individual. there is no sharp distinction between the centralized and decentralized australasian journal of information systems special issue 2003/2004 41 approaches, and one of the projects in our study, mozilla, can be characterized as borderline due to its semi-structured development process. according to studies of commercial projects using daily builds and thus some form of continuous integration, advantages of continuous integration include reduced integration risks (errors are found early) and motivation (you see a working system) (cusumano & selby, 1997; ebert, parro, suttels, & kolarczyk, 2001; olsson & karlsson, 1999). continuous integration is also an alternative to ‘big bang’ integration, where all modules are combined in one go, and which usually results in large numbers of errors, hard to isolate and correct owing to the vast expanse of the program (pressman, 1992). disadvantages of continuous integration may include degeneration of architecture due to lack of focus on overall design and time spent on too frequent releases of too poor quality (olsson & karlsson, 1999). continuous integration, especially in a decentralized variant, may be of particular interest in open source projects due to the difficulty associated with imposing structured approaches in such projects and their participants. some of the prevalent characteristics of open source projects are: few specifications: many of the documents traditionally regarded as essential for coordination seem to be missing in most open source projects, including plans and schedules (mockus, fielding, & herbsleb, 2002). geographical distribution: a study of linux contributors showed that they “come from a truly worldwide community spanning many organizations” (dempsey, weiss, jones, & greenberg, 2002), and a recent on-line survey of 2,784 open source participants (ghosh, glott, krieger, & robles, 2002) showed that “most of the developers feature networks that consist of rather few people.” studies have shown that integration is particularly difficult in geographically distributed projects (herbsleb & grinter, 1999). volunteers: most developers contribute to the projects in their free time – an estimated 60% are not paid for their work (hars & ou, 2001; jørgensen, 2001) – and consequently they are less likely to accept to be ordered around and perform tedious tasks. self-directed egoists: this characterization of open source developers (raymond, 2001) may be crude, but it is not too distant from the results of empirical studies. according to lakhani et al. (2002), the two most important motivations for participating in an open source project were “intellectually stimulating” and “improves skill”. for only 20% of the developers, “work with team” was a key motivator. however, to attain the benefits of continuous integration, it is crucial that the project imposes some form of control on the contributions flowing into the development version of the project’s software. for example, the benefit of easier defect diagnosis, because build errors must be the result of very recent contributions (mcconnell, 1996a), is unobtainable if the build is broken from the outset. we use mintzberg’s work on organizational archetypes and different mechanisms of control (mintzberg, 1979) to categorize the activities in freebsd and mozilla, which we have found are related to quality assurance and continuous integration. as indicated above, in some ways open source projects lack structure and therefore might resemble mintzberg’s adhocracy archetype: a flat organization of specialists, forming small project groups from task to task, and supposedly the most appropriate organization for modern organizations that depend on their employees’ creative work. however, we find it rather intriguing and illuminating to examine the structured aspects of the two open source projects. in support of this, we may add that there are strong elements of stability in the projects: throughout their lives they have used the same technological infrastructure, and the projects are primarily developing new versions of the same product (an operating system and a web browser suite, respectively). therefore, we have organized our account of quality assurance and continuous integration in the two projects to follow mintzberg’s distinction between establishing central control via standardization of worker skills, work processes, and work output. each of these forms of control is predominant in one of mintzberg’s archetypes: the professional bureaucracy, the machine bureaucracy, and the divisionalized organization. the rest of the paper is organized as follows: first we describe the two projects in our case study and how we have made the survey. subsequently, we examine the projects’ quality assurance activities australasian journal of information systems special issue 2003/2004 42 focusing on control of worker skills, work processes, and work output, respectively. finally, in the last section we conclude and discuss our findings. freebsd, mozilla, and our survey freebsd and mozilla organize their source code repository (from now on: the repository) around a main branch (the trunk) into which most changes are inserted and one (mozilla) or more (freebsd) additional branches hosting the projects’ production releases. see table 1 for basic facts about the two projects. table 1. basic facts on freebsd and mozilla name freebsd mozilla product operating system web browser suite major product qualities robustness, security independence, open interfaces, user interface major platforms intel x86, alpha, sparc, pc-98 windows, linux, macos approximate size of trunk 29,000 files, 11 million lines 40,000 files, 6 million lines activity on trunk in october 2002 118 persons committed 2,063 changes 107 persons committed 2,856 changes 50% of these commits made by 12 developers 7 developers project management 9 person core team 11 person mozilla.org staff the repositories are stored at central locations and can be reached via the internet (www.freebsd.org and www.mozilla.org). developers contribute to the projects by continually and in parallel updating (changing, adding, and deleting) repository files; a process controlled by means of cvs augmented with extra tools. each file change (commit) creates a new version; all old file versions remain accessible, and thus it is always possible to go back to an older version of a file if problems or errors occur. the software product that results from building with the newest versions of all files is called the development version; because files are updated continuously, the development version will also be constantly evolving. the aggregation of all files related to the development version is called the trunk. because both projects maintain production releases and therefore have to isolate new development from maintenance of previous releases, there is a need to create branches in the repository. at a certain time, a branch is created as a (logical) copy of the most recent file versions from the trunk; subsequently, changes to the trunk will not directly influence the branch and vice versa, and developers must specify which branch they want to use when downloading or committing source files. in addition to the various development and production versions, both projects’ software is also adapted to a wide range of different platforms (the table only shows the most important). as a consequence, developers not only need to differentiate between releases (residing on different branches), they must also be able to commit changes intended for specific platforms, which is accomplished by using conditional compilation rather than branching. both projects employ several staff and management functions, including top-level management boards (mozilla.org staff and freebsd core team). but most of the development work in both projects takes place in one-man projects, where developers contribute new or revised source code, largely working by themselves (jørgensen, 2001). australasian journal of information systems special issue 2003/2004 43 we have pulled statistical data from freebsd’s and mozilla’s repositories in october 2002, studied the projects’ public mailing lists, and drawn on a survey of freebsd committers performed in november 2000 and also the basis for (jørgensen, 2001). the survey obtained 72 replies, corresponding to just over 35% of the project’s committers at that time. the developers were asked 29 multiple-choice questions and also asked for further comments. finally we have conducted email-based interviews with selected developers from freebsd and mozilla. controlling quality through skills: promotion based on merits mozilla and freebsd carefully control which developers are allowed to commit changes to their software. to get commit privileges, in both projects you must first demonstrate your competence, typically by making high-quality contributions over a period of time. the accreditation procedures for new committers are related to what mintzberg designates ‘standardization of skills’, the key coordination mechanism in professional bureaucracies, often illustrated by examples like hospitals and universities (mintzberg, 1979). in comparison with these, the standardizations of skills in freebsd and mozilla are rather simple. essentially, they only consist of filtering the admitted developers into the single category of ‘professionals’: the committers. promotion of external contributors to committers mozilla has a formal bureaucratic procedure for accrediting committers involving a formal application, acceptance from a voucher (person who already has commit privileges), and a written acceptance from three of the so-called ‘super reviewers’ (getting cvs write access to mozilla, 2003). in freebsd, you acquire the right to commit to the source code repository by the core team based on a request from one or more committers, but the procedure does not seem to be as formal as in mozilla. by only granting commit-privileges to developers that have demonstrated programming as well as interpersonal skills and interest in the project, freebsd and mozilla reduce the risks of low-quality contributions. the trial period, which contributors must go through before they become committers, can be seen as an apprenticeship where the persons gradually learn and adapt to the projects’ procedures, rituals, and culture. the contributors will only be given commit privileges if the adaptation appears successful: this process serves multiple purposes; after all, the freebsd community is made up of people who do the work. for committers, the work consists of creating useful and correct patches. if you don’t consistently and regularly create good patches, there’s no point in giving you commit access, now is there? … by the time you’ve submitted several dozen prs [problem reports], you’ll either work well with the freebsd team or everyone will understand that you and the team just can’t get along (lucas, 2002). because developers without commit privileges have to find committers that will approve their contributions and perform the actual changes to the repository, the committers have a strong motivation for delegating commit privileges to qualified developers: to a committer, taking patches from prs is a trivial annoyance. contributions are certainly appreciated, but they must be read, evaluated, and tested. … if you submit enough useful and correct prs, eventually some committer will get sick of taking care of your work and will ask you if you want to be able to commit them yourself (lucas, 2002). this pattern of developers, gradually becoming more involved in freebsd or mozilla, bears resemblance to professional communities evolving by gradually letting peripheral participants become fully qualified members, as described by lave and wenger (1991) in their study of what they term legitimate peripheral participation; an analogy also noted by nakakoji et al. (2002). according to a freebsd developer, the accreditation procedures seem to work very well: australasian journal of information systems special issue 2003/2004 44 by and large, most of the committers are better programmers than the people i interview and hire in silicon valley (freebsd developer). controlling the committers a critical concern in the projects is to ensure that developers with commit privileges also commit good source code contributions. both projects reserve the right to delete a change from the trunk and to revoke a developer’s commit privileges completely. removal of a change is technically a simple task provided that the change is independent of all subsequently committed changes, which is usually the case. in freebsd, there is a well-defined process for deleting a change already committed: any disputed change must be backed out […] if requested by a maintainer. […] this may be hard to swallow in times of conflict […] if the change turns out to be the best after all, it can easily be brought back (the freebsd committers’ big list of rules). moreover, a consensus between committers working in a certain area can be overridden for security reasons: security related changes may override a maintainer’s wishes at the security officer’s discretion (the freebsd committers’ big list of rules). the ultimate sanction is to revoke the commit privileges, and freebsd’s internal rules specify a procedure for doing this temporarily or permanently (the freebsd committers’ big list of rules). under normal circumstances, this requires three core team members to act in unison, and subsequently the core team is required to participate in a public hearing about the matter if requested by the developer in question. it is our estimate that the ultimate sanction, analogous to firing a hired developer, is used less than once a year and that it serves as a means to resolve collaborative issues rather than to maintain a high level of coding skills. controlling quality through process: laissez faire prior to the point where their new source code contribution is integrated into the development version, committers in both freebsd and mozilla are free to choose their own approach to develop the change. on one hand, this freedom is well suited to the voluntary nature of most of the development work. on the other, the lack of systematic approaches, e.g. thorough analysis and design activities, may increase previously identified important risks associated with the daily build and continuous integration (olsson & karlsson, 1999): excessively proactive development (developers failing to think before they act), architectural degeneration, and ‘quick and dirty’ changes. in general, there is a relatively small amount of control via standardization of work processes in freebsd and mozilla. the mintzberg archetype for organizations relying on standardization of work as a key coordinating mechanism is the machine bureaucracy, and example organizations include mcdonald’s and automobile manufacturers (mintzberg, 1979). this section describes the process rules and guidelines that apply to the developer’s pre-integration activities, including the requirements for public announcement, discussion, and review. the section also discusses the preintegration activities for which there are no such rules or guidelines, in particular design. work assignment the life cycle of a change begins when a developer decides to start working on a task. freebsd and mozilla's rules for work assignment are extremely liberal: in general, committers are free to pick any task they wish. the projects encourage developers to pick tasks that appeal to them: look through the open prs, and see if anything there takes your interest (hubbard, 2003). australasian journal of information systems special issue 2003/2004 45 this is in line with raymond’s thesis about open source projects relying on developers inclination to “scratch their personal itch” (raymond, 2001). the use of continuous integration is an important condition that makes it relatively easy for developers to define and choose on which task they want to work. as the decision when to integrate is delegated to the individual committer, there can be no specific order in which the various contributions are supposed to be integrated in the development version. thus, developers are largely free to choose, implement and commit any contribution to the repository with only a limited effort to coordinate with other developers. this independence is an obvious advantage in an open source project: from the point of view of the project as a whole, it reduces the need for a plan in which tasks are defined, allocated, and scheduled; a plan that would require a considerable effort to produce and maintain. from the developer’s point of view, the freedom to choose a task and integrate a change quickly may be highly motivating: ... there is a tremendous sense of satisfaction to the ‘see bug, fix bug, see bug fix get incorporated so that the fix helps others’ cycle (freebsd developer). this may also apply to developers who are paid by a company: i use freebsd at work. it is annoying to take a freebsd release and then apply local changes every time. when […] my changes […] are in the main release […] i can install a standard freebsd release […] at work and use it right away (freebsd developer). the developers’ free choice of task assignments are, however, balanced by various recommendations and rules. to see who is working on what, mozilla recommends that if the task, on which a developer chooses to work, has not already been reported as a bug, the developer should do this first: enter the task you’re planning to work on as enhancement requests and bugzilla will help you track them and allow others to see what you plan to work on (bugs, 2003). freebsd has no similar formal requirement that developers should announce their current task. there are at least two exceptions to developers’ free choice of task assignments. the first exception is soft in the sense that it is a recommendation rather than a rule: the encouragement to work on important bugs. the general call in mozilla is to “stay focused on the most important problems [i.e. bugs]” (the seamonkey engineering bible, 2003), especially stressing the issue before production releases. if in doubt, the developer is recommended to choose to work on one of the bugs reported in the bug-tracking system, and inform possible stakeholders of his or her plan: start with the prs that have not been assigned to anyone else. if a pr is assigned to someone else, but it looks like something you can handle, email the person it is assigned to and ask if you can work on it – they might already have a patch ready to be tested, or further ideas that you can discuss with them (hubbard, 2003). the second exception is an obligation to consult the person responsible for a given code area. both projects have a notion of code ownership in the sense that most files have a maintainer (freebsd) or module owner (mozilla), often responsible for entire directories or applications: the maintainer owns and is responsible for that code. this means that he is responsible for fixing bugs and answering problems reports […] (kamp, 1996). code ownership is a mechanism for coordination via a consensus process: [a commit should happen] only once something resembling consensus has been reached (the freebsd committers’ big list of rules). the requirements to announce and discuss one’s (intended) choice of work tasks publicly help to mitigate obvious risks: • the risk of doing duplicate (and hence wasted) work because different developers unknowingly might be working in parallel on the same problem. even though developer resources are gratis in open source projects, they are not unlimited and should be utilized efficiently. australasian journal of information systems special issue 2003/2004 46 • the risk of spending time on changes that will not be considered improvements by the community. this is a waste of the developer’s resources and may result in extra work for others if the changes need to be backed out of the repository. • the risk of integration problems because two or more developers want to make incompatible changes to the same module. work breakdown work is broken down into small tasks that are not too difficult to integrate. neither project has any rule pertaining to the granularity of work breakdown; rather working with small changes is a consequence of the obligation to integrate one’s own contributions. an interesting consequence of this approach is the difficulties associated with developing large, new features that are not easily broken down into independent pieces. an example of a very large task is the recent work in freebsd on symmetric multi-processing (smp) enabling the operating system to utilize multiple processors. the smp effort was organized as a subproject with its own project manager. the subproject considered encapsulating its work on a separate branch, but rejected this in fear of ‘big bang’ integration problems as experienced in another open source operating system project, bsd/os: … they [bsd/os] went the route of doing the smp development on a branch, and the divergence between the trunk and the branch quickly became unmanageable. […] we are completely standing the kernel on its head, and the amount of code changes is the largest of any freebsd kernel project taken on thus far. to have done this much development on a branch would have been infeasible. (freebsd smp project manager, 2000). so the smp subproject chose to add their radical kernel changes incrementally, but the problem of preserving the development version in a working state was a huge challenge and seen as a heavy burden – the task can be compared to transforming a van into a sports car while driving. changes were added incrementally over several years, but other developers’ work was seriously affected, especially in the autumn of 2000 when the smp work severely ‘destabilized’ the trunk causing build failures and other errors due to dependencies with other, concurrent work on the trunk. the obligation to preserve the development in a working state could be seen as implying an implicit rule saying: avoid the introduction of large and complex new features. however, it should be noted that concurrent development on an operating system kernel is inherently difficult, so this implicit rule may apply to other approaches to integration as well. design neither freebsd nor mozilla requires design to precede coding in the sense of writing, discussing, or approving design documents prior to coding. indeed, in practice there is typically no design document for the individual change: 31 of the 72 committers surveyed in freebsd responded that they had never distributed a design document (defined as a separate document, distinct from a source file). some documentation of the design of the systems’ basic architecture is accessible, though. however, the lack of an established practice to use design documents as a coordination mechanism should be viewed in the context of the projects as largely maintenance-oriented. freebsd has inherited a largely unaltered, basic architectural design from its predecessors, the first versions of which were developed in the late 1970s (about freebsd's technological advances). mozilla, being a much younger project with roots that only go back to the mid-90s, is more in need of providing its own design documentation. for example, the project has developed its own component model (xpcom) and uses a software layer originally developed by netscape, which provides a platformindependent interface to multithreading (nspr). these and other complex, project-specific parts of mozilla are described at an introductory level (hacking mozilla) and in more detail (core architecture) in a series of publicly available documents. australasian journal of information systems special issue 2003/2004 47 review there seems to be no requirement or tradition for design reviews, but both projects require review of source code changes prior to commit. if the developer is a committer, the review may be the only occasion where others are involved in approving the developer’s work prior to commit. freebsd’s committer’s guide rule no. 2 states “discuss any significant change before committing” (the freebsd committers’ big list of rules), and 86% of the committers surveyed said that they actually received feedback on their latest change when submitting it to review. mozilla has detailed rules requiring all changes to be reviewed by another committer, and in most cases to be ‘superreviewed’ as well. the ‘super-review’ is done by one or more of a designated group of strong hackers and examines the quality of the code itself, its potential effects on other areas of the tree, its use of interfaces, and its adherence to mozilla coding guidelines (eich & baker, 2003). to enforce reviews, mozilla requires that a committer always should state the names of the contribution’s reviewers when adding source code to the repository. controlling quality through work output: don’t break the build the major event in the life cycle of a change in freebsd and mozilla is the commit of the change to the central repository. prior to the commit, preliminary versions of the change have resided in the developer’s private repository, most likely on his or her own computer. the commit is the delivery of the developer’s work output to the project as a whole, and defines the point in time where it must meet the project’s standards. mintzberg’s archetype for organizations relying on control of work output as a key coordinating mechanism is the divisionalized organization exemplified e.g. by large multinational companies with relatively autonomous divisions, responsible for their own products (mintzberg, 1979). control of work output in freebsd and mozilla of software changes produced by their ‘divisions’, the individual developers, is merely qualitative rather than quantitative as in companies that also attempt to control the productivity of divisions. first, we discuss the standard defined by the projects and how compliance may be verified, and then we discuss how the projects modify the two basic rules of continuous integration: the developer’s build-obligation and the developer’s commit access. the standard that contributions must satisfy both in freebsd and mozilla it is emphasized that changes committed must keep the build working and comply with the projects’ coding guidelines. mozilla requires that developers run a number of simple tests before committing (duddi, 1999), and is very clear about the requirement not to break the build: breaking (run time, compile time, or link time) the tree is not ok. it costs lots of money (more than you can justify wasting) to have hundreds of engineers sitting idle waiting for a good tree to pull (working with the seamonkey tree, 2002). in freebsd, the requirements are given as part of the explanation to rule no. 10, “test your changes before committing them”: if your changes are to the kernel, make sure you can still compile [the kernel]. if your changes are anywhere else, make sure you can still [compile everything but the kernel] (the freebsd committers’ big list of rules). extensive coding guidelines exist in both projects. examples from mozilla include a code style guide (mozilla coding style guide, 2003) and a portability guide (williams, collins, & blizzard, 2003). freebsd provides a “kernel source file style guide”, code guidelines to facilitate software internationalization and a security guide with preventive rules such as: never trust any source of input [...] never use gets() or sprintf(), period (freebsd security guide, 1997). australasian journal of information systems special issue 2003/2004 48 as far as we understand, the projects do not have other requirements that pertain to code contributions. for example, there is no requirement such as extreme programming’s demand for pair programmers to write test programs before the actual coding is performed (beck, 1999). (as an aside, we will mention that both projects include subprojects, for example freebsd’s documentation project, producing other kinds of deliverables. however, the activities of these subprojects are outside the scope of this paper). verification the build process is fully automated, and therefore the verification that contributions meet the buildrequirement is straightforward. in freebsd, there is an automated routine for building the trunk twice a day on the major processor architectures, the so-called tinderbox-builds, the result of which are shown on a webpage (http://www.rtp.freebsd.org/~des). in mozilla, there is a well-defined process for a daily verification procedure using a cluster of build machines (representing all targeted platforms). at 8 am (pst) each working day, the build machines download the newest source code, build it, and execute a small number of regression tests. however, in both projects most build errors will be detected by currently active developers, reporting the problem to one or more mailing lists, even before the tinderbox builds (in freebsd) or the daily build verification (in mozilla). broken builds have immediate consequences for the active developers, because neither project operates with a so-called ‘holding area’. in order to preserve the development version in a sound state allowing developers to rely on it for testing their own code, mcconnell (1996b) recommends that projects using daily builds create a copy of the development version through which all changes must pass on their way to the (proper) development version to filter away changes not properly tested. neither project has any such filtering of the stream of changes flowing into the trunk. verification that code contributions comply with coding guidelines is facilitated by the visibility of contributions: • the repositories are browsable, providing easy public access to all sources files. • in both projects, an automatic mail message is sent to other developers immediately upon commit of a change. this visibility encourages developers to strive to produce code that will be perceived to have high quality. in responding to the statement “knowing that my contributions may be read by highly competent developers has encouraged me to improve my coding skill”, 57% of the 72 freebsd committers surveyed said “yes, significantly”, and 29 % “yes, somewhat”. one committer added: “embarrassment is a powerful thing.” the visibility of the code is in part due to the project being open source, but also to the approach of continuous integration: a large number of developers are working with the most recent version of the trunk, and monitor the changes made because their work depends on them. given, on the one hand, the projects’ reliance on the quality of the committed changes, and on the other hand, the frequent occurrences of broken builds, it is remarkable that a committed change rarely is removed from the repository. when it happens, it is normally not due to a broken build, but to disagreement about whether the change, correctly implemented or not, is in fact an improvement of the software. balancing the don’t break the build rule it appears to us that the projects’ reason essentially is as follows when faced with one of the frequent build-breaking changes: as long as the change, when corrected, will improve the trunk, we prefer to keep it in the trunk and correct is as fast as possible rather than exercise the right to delete the change and throw it back to the developer, possibly to an uncertain faith. i can remember one instance where i broke the build every 2-3 days for a period of time; that was necessary [due to the nature of the work]. that was tolerated – i didn’t get a single complaint (freebsd committer, 2000). australasian journal of information systems special issue 2003/2004 49 in mozilla, the daily build verification (see the previous section) is highly organized. almost as crucial as preventing broken builds is the requirement to be available after a commit that (sic) does break the build. developers that have committed changes since the previous day’s verification are said to be ‘on the hook’: if you are on the hook, your top priority is to be available to the build team to fix bustages. […] you are findable. you are either at your desk, or pageable, checking e-mail constantly, or on irc so that you can be found immediately and can respond to any problems in your code (hacking mozilla with bonsai). there are several good reasons for balancing the ‘don’t break the build’ rule with other considerations. correcting a broken build can be highly challenging, since the failure may be due to dependencies on files or modules outside the area of the developer’s primary expertise. moreover, some changes may be difficult to test in the first place. interpretation of the ‘don’t break the build’ rule is particularly important with respect to the effort that a developer should invest to prevent broken builds on any platform. as mentioned, both freebsd and mozilla are developed for many platforms, of which 4 and 3 are particularly prioritized. due to platform differences, a build may succeed on one and fail on another, which we refer to as a partially broken build. however, most developers only have access to a single platform, and therefore it may be impracticable to perform trial builds on each prioritized platform before check-in. in practice, the mozilla project in general accepts a large part of the responsibility of correcting partially broken builds. one of the reasons is the unattractive alternative of accepting a source code change on some platforms, but not on others (those that build with the change vs. those that don’t). as a general rule, the repository in mozilla is closed during the daily verification build until it has been terminated successfully. this means that no commits (except as part of the corrective effort) are allowed until all three prioritized platforms pass the test. this may last from two to several hours. the reason for ‘closing the tree’ for all platforms is described as follows: during the development of netscape navigator and netscape communicator it was argued many times that […] we should care less about a particular set of platforms and fix regressions on these “second-class” platforms later. we tried this once. the reason why we don’t have netscape communicator on win16 was the result of putting off the recovery of that platform until later. after a couple of weeks recovery became impossible. […] the problems will stack up […] as the codebase moves forward and it never catches up (yeh, 1999). in part, the problem of broken builds on other platforms is solved by making a set of central build machines available, to which sources can be uploaded and subjected to a trial build prior to commit. however, this is tedious and is not enforced as a general rule in either project. as a middle road, mozilla provides the previously mentioned portability guides with rules and recommendations for producing cross-platform software (e.g., williams et al., 2003). in principle, freebsd and mozilla delegate to the developers the responsibility to integrate their own contributions, but in practice it is a major challenge to strike a reasonable balance and to some degree accept that developers from time to time break the build and thus disrupt other developers’ work. indeed, making an absolute requirement that committed changes should be error-free would be absurd and defy the purpose of using the trunk for community testing, as discussed below. balancing the access to contribute. in both projects, a stabilization period is explicitly declared for several weeks prior to major production releases. this is an important exception from developers’ access to commit changes and may create tension in the projects. we will discuss the process leading to major production releases (e.g., freebsd’s 5.0 of january 2003, and mozilla’s 1.0 of june 2002). in addition, the projects also create minor production releases (freebsd 4.6, 4.7, etc.; mozilla 1.1, 1.2, etc.). the purpose of a stabilization period is to limit the changes allowed to be committed. during stabilization, only changes seen as necessary or useful for the purpose of stabilization are allowed, australasian journal of information systems special issue 2003/2004 50 most notably bug-fixes. in freebsd, the stabilization period for 5.0 lasted for two months. the first month was less strict allowing new features on a case-by-case basis at the release engineering team’s discretion. the second month was more strict and only allowed commits if they were bug-fixes. when the software is considered to be sufficiently stable, it is declared a production release. this is possible because prior to stabilization the software is already in a working state, which makes the use of special integration and testing teams unnecessary. it is a major advantage of continuous integration if a brief period of stabilization is indeed sufficient for changing ‘work in progress’ to ‘production release’. however, the required stabilization effort may be huge and seen as diverging resources from more important or interesting new development. in mozilla, the stabilization period prior to the 1.0 release can be seen as lasting more than 8 months, beginning with the 0.9.6 release (in november 2001) upon which the “the trunk is closed to all but a relative few bug fixes, and everyone is focused on testing” (eich, 2002). there is indication of pressure from mozilla developers to relax commit restrictions: […] we’re not looking for new features; we want stability, performance […], tolerably few bugs […]. features cost us time […] those implementing the features […] could instead help fix 1.0 bugs […]. if you think you must have a feature by 1.0, please be prepared to say why to drivers, and be prepared to hear “we can’t support work on that feature until after 1.0 has branched” in reply (eich, 2002). to allow for the resumption of new development on the trunk, the final two months of stabilization for mozilla’s 1.0 release took place on a separate branch (created april 2002). the isolation of bugfixing from new development is in some sense a departure from the strict adherence to continuous integration, and indicates the following dilemma associated with stabilization: • creating a separate stabilization branch allows for resumption of new development, which will otherwise be halted when destabilizing changes are prohibited on the trunk. • however, a separate stabilization branch requires that bug-fixes to the trunk are also made to the branch (and vice versa); it doubles the tasks related to managing a branch (assigned to branch drivers in mozilla), and divides the pool of user/developers between the branches: this branch [mozilla 1.0] obviously entails overhead in driving, merging, reviewing, and testing (eich, 2002). the dilemma makes it important and difficult to decide if and when to branch stabilization away from new development. discussion and conclusion viewed from the perspective of quality assurance our case study of freebsd and mozilla indicates that at a basic level the projects’ approach to software integration actually works: in spite of their difficult-to-control and geographically distributed developers that pick tasks at their own convenience and do not write design documents, freebsd and mozilla produce widely used software. it appears that the process of continuous integration, as used in the projects in a decentralized variant, to some degree replaces traditional software engineering coordination mechanisms like plans and design documents. incessant access to commit the first key principle of the projects’ decentralized approach to continuous integration is the committers’ access to add contributions to the development version at any time. this access appears to be highly motivating: developers are free – only limited by the need to reach consensus with module owners – to choose which tasks they want to work on, and they can commit changes without awaiting approval. this feels unbureaucratic and the developers can see that the result of their work quickly becomes part of the project’s software. using mintzberg’s terminology, the freedom to commit is absence of work process standardization. commit access is granted to developers according to their previous merits, which can be seen as australasian journal of information systems special issue 2003/2004 51 standardization of worker skills. once you are in, you are free to choose your own work process. however, this overall liberal work process regime is supplemented with various work process recommendations such as to work on prioritized bugs, and announce and discuss work. a major challenge for the projects, related to the incessant access to commit, is the conflict between stabilization and new development: a prolonged period of stabilization on the trunk entails the cost of holding back new development, but if the stabilization period is too short, the release may be of poor quality. alternatively, stabilization can be encapsulated on a separate branch, but this entails the cost of dividing developer resources for testing and managing between community testing and stabilization. prior to their most recent major releases, both projects did in fact decide to branch away stabilization from new development. these decisions were made reluctantly, because of the concern that branches may diverge to a point where useful changes cannot easily be merged from one branch to another, e.g. when a bug-fix on the stabilization branch is difficult or impossible to perform on the development branch. this concern underlies the decision in freebsd to avoid developing the new smp feature on a separate branch, and the principle in mozilla to never ‘leave behind’ any prioritized platform. the fact that the projects chose to branch prior to production release, despite these concerns, indicate that restricting commit access for a prolonged phase was seen as entailing a too high cost in terms of holding back new development. obligation to integrate own contributions the second key principle of the projects’ decentralized process is the developers’ obligation to integrate their own contributions and most notably to avoid breaking the build of the development version. from a long-term perspective, maintaining the development version in a healthy condition is crucial because it allows for the project to produce a software version that is mature enough for production release, merely through ‘stabilizing’ the development version. this may be a relatively painless process if the software is already in a working condition, implying that unexpected delays due to integration problems are avoided. from a short-term perspective, a working development version is crucial for the ongoing development and debugging effort: new development is halted if local source code changes cannot be tested against the newest source code in the trunk; and debugging is halted if developers and users can not run the software. a disadvantage is that with the source code in the trunk changing constantly, implementing a change is like hitting a moving target, and requires that developers’ local source code is synchronized frequently. placing the responsibility for integration on the developer or team developing a change may have a conservative effect, because large and complex new features (e.g. changes of the basic architecture) are difficult to divide into smaller, independent tasks that can be easily integrated. in mintzberg’s terminology, the projects’ ‘don’t break the build’ rule is standardization of work products. failure to comply with the rule is visible immediately. in both projects the ability to commit changes which are perceived to be of good quality constitute the merits on the basis of which commit access is granted, thus linking standardization of work products and worker skills in a manner akin to apprenticeships. a major challenge for the projects, relating to ‘don’t break the build’, is to interpret this rule in a suitable, pragmatic manner. there may be good reasons why developers frequently do not comply fully with their integration obligation, and rather than simply deleting error-prone changes, the projects organize a project-level, common effort to correct broken builds. the most structured effort takes place in mozilla, where a daily cycle blocks new commits until the trunk has been built successfully, and where committers of recent changes are compelled to participate in resolving any broken build situations. major reasons why developers inadvertently violate the ‘don’t break the build’ rule include the inherent intricacy of debugging complex systems and not having access to the range of different platforms on which they should (ideally) test their changes. australasian journal of information systems special issue 2003/2004 52 future research in future research, studies of commercial software projects may be of interest to clarify whether such projects can attain the advantages of decentralized continuous integration, and provide insight into different ways of attaining balanced interpretations of the access to contribute and the obligation to integrate. in commercial software development, both the motivational advantage of seeing changes integrate quickly and the reduced risk of release delays may be as important as in open source projects. while it is important to establish balanced interpretations of the access to contribute and the obligation to integrate, perhaps the major challenge, in open source as well as commercial projects, is to establish a project culture with strong encouragement to produce high quality code, but also with tolerance and mutual support. indeed, the approach of continuous integration may provide a basis for developing such a culture, since the quality of the project’s most recent source code becomes a common point of continuous, project-wide focus. references about freebsd’s technological advances. retrieved dec. 1, 2002, from: http://www.freebsd.org/features.html beck, k. (1999). extreme programming explained: embrace change. boston, usa: addison wesley. bugs (2003). retrieved april 8, 2003, from: http://www.mozilla.org/bugs/ core architecture. retrieved dec. 1, 2002, from: http://www.mozilla.org/catalog/architecture/ cusumano, m. a., & selby, r. w. (1997). how microsoft builds software. cacm, 40(6), 53-61. dempsey, b., weiss, d., jones, p., & greenberg, j. (2002). who is an open source developer? a qualitative profile of a community of open source linux developers. cacm, 45(2), 6772. duddi, s. (1999). pre-checkin tests. retrieved may 1, 2003, from: http://www.mozilla.org/quality/precheckin-tests.html ebert, c., parro, c. h., suttels, r., & kolarczyk, h. (2001). improving validation activities in a global software development. paper presented at the 23 rd international conference on software engineering (icse '01), toronto, canada. eich, b. (2002). mozilla 1.0 manifesto. retrieved nov. 15, 2002, from: http://www.mozilla.org/roadmap/mozilla-1.0.html eich, b., & baker, m. (2003). mozilla ‘super-review’. retrieved april 21, 2003, from: http://www.mozilla.org/hacking/reviewers.html the freebsd committers’ big list of rules. retrieved dec. 1, 2002, from: http://www.freebsd.org/doc/en/articles/committers-guide/rules.html freebsd security guide (1997). retrieved oct. 31, 2003, from: http://www.pl.freebsd.org/security.html getting cvs write access to mozilla (2003). retrieved april 22, 2003, from: http://www.mozilla.org/hacking/getting-cvs-write-access.html ghosh, r. a., glott, r., krieger, b., & robles, g. (2002). part 4: survey of developers, floss final report. hacking mozilla. retrieved dec. 1, 2002, from: http://www.mozilla.org/hacking/ hacking mozilla with bonsai. retrieved dec. 1, 2002, from: http://www.mozilla.org/hacking/bonsai.html hars, a., & ou, s. (2001). working for free? motivations of participating in open source projects. paper presented at the 34 th hawaii international conference on system sciences, australasian journal of information systems special issue 2003/2004 53 hawaii, usa. herbsleb, j. d., & grinter, r. e. (1999). splitting the organization and integrating the code: conway’s law revisited. paper presented at the international conference on software engineering (icse ‘99). hubbard, j. (2003). contributing to freebsd. retrieved oct. 31, 2003, from: http://www.freebsd.org/doc/en/articles/contributing/ jacobson, i., booch, g., & rumbaugh, j. (1999). the unified software development process. indianapolis, usa: addison-wesley. jørgensen, n. (2001). putting it all in the trunk: incremental software development in the freebsd open source project. information systems journal, 11(4), 321-336. kamp, p.-h. (1996). source tree guidelines and policies. retrieved dec. 1, 2002, from: http://www.freebsd.org/doc/en_us.iso8859-1/books/developers-handbook/policies.html lakhani, k. r., wolf, b., bates, j., & dibona, c. (2002). the boston consulting group hacker survey. retrieved may 8, 2003, from: http://www.osdn.com/bcg/bcg-0.73/ lave, j., & wenger, e. (1991). situated learning: legitimate peripheral participation (learning in doing). cambridge: cambridge university press. lucas, m. (2002). how to become a freebsd committer. retrieved april 22, 2003, from: http://www.onlamp.com/lpt/a/1492 mcconnell, s. (1996a). daily build and smoke test. ieee software, 13(4). mcconnell, s. (1996b). rapid development: taming wild software schedules. microsoft press international. mintzberg, h. (1979). the structuring of organizations. englewood cliffs, new jersey: prentice hall. mockus, a., fielding, r. t., & herbsleb, j. d. (2002). two case studies of open source software development: apache and mozilla. acm transactions on software engineering and methodology, 11(3), 309-346. mozilla coding style guide (2003). retrieved oct. 31, 2003, from: http://www.mozilla.org/hacking/mozilla-style-guide.html nakakoji, k., yamamoto, y., nishinaka, y., kishida, k., & ye, y. (2002). evolution patterns of open-source software systems and communities. paper presented at the international workshop on principles of software evolution (iwpse), orlando, florida. olsson, k., & karlsson, e.-a. (1999). daily build the best of both worlds: rapid development and control. lund, sweden: swedish engineering industries. pressman, r. s. (1992). software engineering a practitioner's approach (3 rd , international ed.). new york: mcgraw-hill. raymond, e. s. (2001). the cathedral and the bazaar: musings on linux and open source by an accidental revolutionary (revised ed.). sebastopol, california, usa: o’reilly & associates, inc. the seamonkey engineering bible (2003). retrieved april 21, 2003, from: http://www.mozilla.org/projects/seamonkey/rules/bible.html williams, d., collins, s., & blizzard, c. (2003). c++ portability guide, version 0.8. retrieved oct. 31, 2003, from: http://www.mozilla.org/hacking/portable-cpp.html working with the seamonkey tree (2002). retrieved oct. 31, 2003, from: http://www.mozilla.org/hacking/working-with-seamonkey.html yeh, c. (1999). mozilla tree verification process. retrieved april 11, 2003, from: http://www.mozilla.org/build/verification.html microsoft word ajis vol13-2 o.doc 167 australasian journal of information systems volume 13 number 2 may 2006 ajis featured theme: ethics in information systems theme editor: professor matthew warren school of information systems, deakin university, victoria, australia. email: matthew.warren@deakin.edu.au “the computer is now central to society. its attendant ethical issues are pervasive, complex and important. in response, at systems development level, new software engineering codes of ethics are being developed, for example, by ieee together with acm in the usa. even so, workplace computer systems can result in job loss, de-skilling, and employee monitoring and surveillance. new problems of intellectual property arise with digital storage and transmission of information. the ease of collecting, storing, retrieving and transmitting information challenges the human right to privacy. the internet raises issues regarding social interaction, censorship and standards of conduct. who can or should control the internet? the internet is global, yet who is setting global standards? does the usa dominate the internet? if that is cultural imperialism, does it matter? electronic commerce and electronic advertising raise ethical questions of privacy and security. computers are used for crimes, and for their prevention and detection. when does privacy become more important than the prevention and detection of crime?” the previous paragraph was written by professor john weckert, charles sturt university, australia in his foundation paper for the development of the australian institute of computer ethics (aice) in 1998. aice was set up to research these key issues and the way that they are impacting australia and the rest of the world. in order to promote their activities, aice has organised a number of conferences around australia focusing upon computer ethics and its impact upon australia. the aice2005 (geelong) conference follows on from the highly successful initial aiicec99 (melbourne), aice2000 (canberra), aice2002 (sydney) and aice2005 (geelong). the special edition focused upon two key areas, the impact of computer ethics upon business and the role of computer ethics impacting upon the use of technology. the paper by bowern et al, entitled “ict integrity: bringing the acs code of ethics up to date”, discusses how the australian computer society code of ethics should be developed for the future to meet the needs of its members. the paper by pye and warren entitled “striking a balance between ethics and ict governance” discusses the inter-relationships between computer ethics and ict governance and the way in which they could be implemented. following this the paper by john barlow entitled “computer simulations, disclosure and duty of care” discusses the way in simulations should be used in an ethical manner, especially in the realm of teaching and the final paper by pierce et al entitled “penetration testing professional ethics: a conceptual model” discusses the key role of ethics in security testing and the blurred line between right and wrong. professor john weckert, charles sturt university, australia recent research into the future issues of computers ethics focuses upon three major aspects problem areas, methodology, and its interdisciplinary nature. first, the problems of growing concern relate to computer networks, and in australasian journal of information systems volume 13 number 2 may 2006 168 particular to the internet. the issue of privacy will become even more important because of the technology's ability to gather, store, match and make inferences from data and information. this has undoubted benefits but also serious costs in terms of loss of individual privacy and possibly autonomy. regulation of the internet will require much more careful examination as that medium plays a larger role in communication, entertainment, business and so on. in both of these cases there will need to closer examination of the kinds of privacy that deserve protection and what kind of regulation is justified. general claims that individual privacy should be protected at all costs and that there should be no internet content regulation in any circumstances just will not do. in addition, more effort must be spent on considering how we want computer technology to be used. on the one hand, it should make life easier for all, and not just for a favoured few. on the other, how much easier do we want our lives to be? life cannot be satisfying without challenges. global ethics will also become more important. we will need to work harder at finding what is common between cultures, and then build on those commonalities. second, computer ethics must become more rigorous and develop a stronger theoretical base. simultaneously there must be close liaison between theoreticians and practitioners. computer ethics without theory is mere consciousness raising, useful, but not enough on its own. theoretical studies remote from professional practice can be mere intellectual exercise, again useful, but insufficient alone. if computer ethics is to be taken seriously there must be a rigorous and theoretically sound examination of practical problems, and it must propose answers within the parameters of the available technology. third, a number of disciplines must cooperate to a greater extent if thoroughly analysed and practical answers are to be provided. at a minimum, computing professionals and academics, philosophers, psychologists, sociologists and lawyers are required. an examination of these three aspects will give some guidance to the directions that computer ethics ought to take. these future ethical issues are the ones that aice and australia will have to deal with to ensure that the impact of technology does not become a nightmare. ajis special edition knowledge management december 2001 making the invisible visible: modelling social learning in a knowledge management context henry linger' school of information management and systems monash university caulfieldeast,vic3145 phone:+61-3-99032260 henry, linger@sims.monash.edu.au leoni warne joint systems branch defense science and technology organisation department of defense canberra act 2600 phone:+612 6265 8819 leoni.wame@dsto.defence.gov.au * author for correspondence abstract the articulation of knowledge management as an organisational strategy has occurred in the context of a radical shift towards an information based economy. the most significant aspect for organisations operating in the information economy is their ability to utilise the volumes of information that are now readily available without the constraint of media, geography or time. a critical factor for organisations is the speed at which they are able to productively process such information to enable the organisation to react rapidly to changes in their operating environments. in this context organisation needs to produce and re-produce knowledge. the shift from information to knowledge is an acknowledgment of the significant role of the human actor in the process of transforming information into effective organisational outcomes. social learning represents important processes that contribute to actors' ability to understand information, create knowledge from that information and share what they know. social learning is therefore intrinsic to knowledge management. in this paper we present a knowledge management architecture that supports a learning organisation. this architecture accommodates social learning and processes by which knowledge is internalised and externalised by individuals, work groups and the organisation as a whole. the architecture incorporates a model social learning based on the results of ethnographic studies and a model of learning derived from knowledge management case studies. the architecture is not domain specific but can be applied to activity that can be characterised as knowledge work in an organisational context. as such the architecture can play a variety of roles; as a conceptual framework, as a diagnostic tool to identify breakdown and as a design tool for organisational change. introduction the effective assimilation of knowledge into an organisation and the organisation's ability to learn and construct new knowledge is an imperative for many organisations in this 'knowledge millennium'. in addition organisations need to retain knowledge when individuals leave, or when situations change, to ensure the viability of the organisation in the longer term. this, in turn, has implications for the development of systems that manage organisational knowledge effectively. many different research efforts have been launched to investigate and facilitate these processes (brown & duguid, 1991; davenport & prusak, 1998; flood, 1999; hildreth, wright, & kimble, 1999; ward, 1998). knowledge exists in the minds of individuals, groups and in organisations. it is, therefore, linked to the human mind, its cognition, reasoning and understanding, but it is shaped by the goals of the organisation or group of which the individual is a member. knowledge management, therefore, is about the acquisition, construction, transfer, and sharing of the knowledge assets within an organisation, aimed at achieving the organisation's objectives. however, the term 'knowledge management' can mean different things to different people; from informal processes that mediate the transformation of an individual's implicit knowledge into explicit knowledge that can be shared; to the complex knowledge databases that facilitate formal organisational processes and capture, store and disseminate vital corporate knowledge. clearly there is a myriad of knowledge management interests that occur along the continuum between informal processes and technological artefacts. our position in this continuum is that knowledge management needs to develop organisational knowledge assets (mainly people), support them with information technology and other tools, and enhances the processes by which individuals, groups, and the entire organisation learns. in our view, knowledge management needs to be concerned with cultural, organisational, procedural, and human factors as well as with tools and technologies. together these form the basis of an organisation's integrated knowledge management strategy. 56 ajis special edition knowledge management december 2001 the purpose of this paper is to present a series of models that can represent the whole continuum, emphasising the process of organisational social learning that we believe underlies successful knowledge management in organisations. we present a general model of learning, a specific architecture for social learning and a generalised knowledge management architecture for a learning organization that represents the continuum discussed above. through these models the paper explores how social learning is incorporated into a knowledge management framework and how this extended framework contributes to the development of a learning organization. the models presented in this paper are the result of collaboration between the authors who have worked respectively in knowledge management and social research that focussed on social learning. the collaboration is based on our belief that social learning is tightly coupled to knowledge management. these models draw on knowledge management models derived from a number of case studies (linger et al., 2000; linger et al., 1998) and the ethnographic studies conducted by the enterprise social learning architectures (esla) research team at the defence science and technology organisation (dsto) (wame, 2000). in this paper we focus on activity that can be characterised as knowledge work that is not domain specific. the composition of this paper is the following. section 2 presents a model of learning and discusses the role of social learning in this model. section 3 introduces the esla task, models the finding as an architecture for social learning and maps the findings onto the learning model. the section also presents a summary of the esla findings. section 4 presents a knowledge management architecture that supports a learning organization while the following section links the three models. section 6 makes concluding remarks regarding the models. a model for representing social learning in this section, we introduce a model of learning that embodies social learning. this model is presented in figure 1 below. the model is based on the assumption that knowledge work underlies learning and that such work is focussed on a task or a specific activity. this changes the perspective from an organisational view to a focus on work practices. in this paper we use the term "actor" to denote an individual who is performing a particular role, emphasising our focus on work practices. the notion of "actor" defines work as a social activity, performed through human agency, but maintains the primacy of work roles in executing a task. implicit in our use of the term actor is a person who has the necessary skill and knowledge to perform the role. this leads to a definition of a community of practice that includes those actors who are engaged directly in performing some aspect of a task (wenger, 1998). the task boundaries are defined by the community, but authorised by the organization, thus allowing members of the community to be distinguished from other potential stakeholders in the task. organisationally however, actors engaged in a particular task can (and usually are) also members of other communities that exist either within a particular organisation or are independent of that organisation. the model involves three perspectives and three sites of discourse. the micro level discourse is the personal and private that allows each individual, or actor, to maintain their own individual/private tacit and explicit knowledge and understanding of a task or activity. the actor can use this framework to perform the task as well as to explore the boundaries of his or her knowledge. we consider all work to be social and represent this by showing more than one actor involved in the task. this allows a multitude of interpretations of how the task is performed, ranging from each actor needing to perform different aspects to collectively complete a single instance of the task, through to the same task being performed many times by the same or different actors and instances of the task being performed at different times and/or places. each actor then has the opportunity to publicise their understanding of the task by making their knowledge available to the community. the discourse at the meso level can be mediated with artefacts or through verbalisation that encodes aspects of each actor's private knowledge. the intention is to provoke discussion and debate, not to impose a particular view. it is the actor who determines when and what is made available to the community. this sharing with the community represents the personal and public perspective. the result of discussion within the community is a consensual perspective that is expressed through a minimal framework representing the current shared and negotiated understanding of the task that all actors would be required to apply to the performance of that task (levitt and march, 1988). the discourse at the macro level represents the organisation's formal and explicit definition of the task. 57 ajis special edition knowledge management december 2001 perspective personal/ private personal/ public consensual site of discourse micro/ individual meso/ community macro/ organisation figure 1. a model of learning (adapted from linger, 2000) social learning is represented by the arrows in figure 1. the arrow between the micro and meso levels indicates an actor's engagement in a social process that represents that actor's participation in a community of practice. the arrow that links the actors within the community represents the social learning that includes the varieties of social and cultural interactions through which actors share and acquire knowledge. in this context, social learning also includes formal aspects of learning, such as training and records management, as the focus is on processes rather than the activity itself. the arrows that link the meso level to the macro level define the mechanisms by which an organisation formalises social learning into sanctioned procedures to achieve organisational outcomes. these mechanisms also form part of the organisation's social learning as they are manifested in the organisation's decision making structure built around more formal social processes such as committees and governing bodies. a significant feature of this model is that social learning is represented by double-headed arrows. this indicates that there is feedback in. every learning process. this means that the actor makes a contribution to the social or cultural process and gains from their engagement with the process. in the context of social learning, the value for the actor is their evolving knowledge of the activity of the community of practice. the nature of the social learning mechanisms contributes significantly to the culture of the organisation and the prevailing organisational values. as a specific example, the feedback in social learning, indicated by the double-headed arrows in the model, define a learning organisation. without this feedback, or if it is ineffectual, the organisation reverts to a hierarchical bureaucracy. the boxes in figure 1. indicate a body of knowledge (bok) that is pertinent to the perspective adopted and the level of discourse (livari and linger, 2000). as shown in the diagram this knowledge has an internal structure represented by a task-based knowledge management framework (linger, 2000). this indicates that some aspects of the body of knowledge at each level of the model can be expressed as explicit artefacts. such representation allows the design of computer-based support systems to facilitate the learning processes as well as task performance. however, it needs to be stressed that the learning model is generic and is not dependent on this internal framework of the body of knowledge. applying the results of the esla studies to this model of learning, it is apparent that the learning processes and strategies that were identified in the studies correlate to the interaction between and within the levels of the model. this is discussed in detail in section 3 below. 58 ajts special edition knowledge management december 2001 the enterprise social learning architectures (esla) task the esla task is a four-year research study investigating social learning within the australian defence organisation. social learning, in this context, refers to learning done in or by a group, an organisation, or any cultural cluster and includes: • the procedures by which knowledge and practice are transmitted across posting cycles, across different work situations and across time; and • the procedures that facilitate generative learning learning that enhances the enterprise's ability to adjust to dynamic and unexpected situations and to react creatively to them. the immediate aim of this research is to understand the issues inherent in building learning, adaptive and sustainable organisations. the long-term objective, however, is to develop architectures that will support the development of information systems that guide and enhance organisational learning. in this way, social learning is inextricably interwoven with knowledge management. the esla study began in june 1998 and has acquired data from several different ado settings. the first setting was a 'tactical' single service headquarters, where a pilot study was conducted to determine the feasibility of the project's aim and methods. the research team returned to this headquarters in april and may 2001 to validate that the findings remained stable over time. the second setting was a joint (all three services and civilians) 'strategic' headquarters within the main australian defence headquarters. currently, further studies are underway in a single-service strategic headquarters. more information on the esla study findings is presented in a separate article in this issue (wame et al., 2001). the methodologies used and research conducted has been reported in depth elsewhere (see agostino, gori, & warne, 1999; ali et al, 2000; warne, 1999; warne, 2000) an emerging architecture for social learning the progressive findings of the esla studies, reported in other papers (see papers listed above), represent the collective research results of all the different settings studied to date. the focus is on the findings that are likely to be relevant to all organisations, not just military ones. these findings are multi-layered and allowed the research team to pinpoint a number of environmental and cultural issues, processes and strategies that facilitate effective social learning and knowledge management. in this paper we have conceptualized these results into the architecture represented in figure 2 below. the esla findings can be collectively conceptualised as a learning toolset of actions, processes and strategies that the organisation engages to facilitate its activities to achieve organisational outcomes. this toolset impacts social learning in four distinct ways; as motivators, enablers, challengers and inhibitors. our position is that each tool has a greater or lesser impact on each of these effectors of social learning. the organisational values arise from the balance of these organisational effectors, but in turn, the values inform these effectors and how they are balanced. these reciprocal and interdependent relationships emerge from the findings of the esla research. while motivators form a sound foundation and enablers provide the bricks or building material, the values are the mortar that binds them all together. challengers and inhibitors are environmental or personal factors that impede or erode the organisation's continued robustness. the architecture also indicates that the organisation's performance influences its values. this introduces a temporal dimension into the dynamic as there is often a temporal distance between an outcome and when it is reflected back unto the organization by the external environment. this is indicated in the diagram by the arrows originating with the organisational outcomes. an example of a study finding mapped to this architecture is inquiry and reflection. an important element of generative learning is for organisational members to be able to engage in dialogue that is open and is based on inquiry and reflection. a supportive communication climate is a prerequisite for such dialogue and it requires learning how to recognise defensive patterns of interaction that undermine learning(ali et al., 2000; senge, 1992). however, an additional and obvious requirement for such dialogue is having the time to engage in it. on numerous occasions, the esla team encountered comments from staff at both the strategic headquarters that there is little time to reflect, learn from experiences (irrespective of whether the experience was a successes or failure) and generally discuss work matters. workload was attributed as a factor preventing people from setting some time aside for thinking and reflection and such comments were often made with an indication of bitterness. on the other hand many of the processes at the tactical headquarters, from mission debriefs to tactics meetings and training, demonstrated that they recognised the importance of inquiry and reflection at a group level. as well as improving the quality of the reflection, group 59 ajis special edition knowledge management december 2001 reflection also contributes to the development of a common identity and a shared understanding amongst the participants. ef ectors / ^ motivators \ / enablers \ \ challengers r \ inhibitors /• organisational outcomes l ___ s figure 2. an architecture for social learning reflection as a tool is an enabler because it allows the actors to remove themselves from the actual performance of the activity. reflection provides the actors with the space and distance they need to better understand the activity they are engaged in. reflection as a motivator allows the actors to learn from their engagement in the activity, with the intention of instigating some form of improvement of the activity. this level of control is empowering. however, reflection requires time that takes the actor away from the actual performance of an activity. the challenge is how to accommodate reflection in the actors' work practices. reflection is also an inhibitor, as it is often an unacknowledged and unrewarded activity. the organisational outcome is determined by how these four effectors are applied in a particular situation. it is the net balance that defines the outcome. moreover the net balance also impacts on the overarching organisational values. for example, if, on balance, reflection is predominantly a motivator and enabler, it contributes to defining empowerment as an organisational value. simultaneously, empowerment, as a value, exerts an influence on how the tool, reflection, is viewed in the organisation. some of the other processes and strategies identified in the esla studies that can be considered part of the toolset in the architecture include: • timely and relevant handover and induction; • building a common identity; • strong goal alignment; • effective workplace design; • facilitating personal networks; • systemic understanding; • team building; • public recognition and award systems; 60 aj1s special edition knowledge management december 2001 • it infrastructure; • mutually beneficial performance management systems; • facilitating accurate perceptions of the organisation; • apprenticeships and legitimate peripheral participation; • peer review and cross-peer learning; • mentoring and/or buddying systems; • the use of social networks and social organisation; • the use of mediating artefacts and bridging agents; • bricolage 1; • the judicious use of protocols at meetings and briefings; and • effective formal information flows and records management. applying esla finding to the learning model it is apparent that the learning processes and strategies that were identified in the studies correlate to the interaction between and within the levels of the learning model discussed in section 2. it would appear that the social learning processes are dominant in the interaction between the individual and community level of the learning model, and at the community level itself. processes that enable new members of the community to gain access to the community's knowledge and its practices include: • timely and relevant handover and induction; • effective workplace design; • systemic understanding; • facilitating personal networks; • team building; • facilitating accurate perceptions of the organisation; • bricolage; • apprenticeships and legitimate peripheral participation; and • mentoring and/or buddying systems. the community also engages in processes that evolve its knowledge and practices so as to maintain the community's ability to perform its activities in changing circumstances. these processes include: • encouragement of dialogue based on inquiry and reflection; • building a common identity; • strong goal alignment; • effective workplace design; • facilitating personal networks; • systemic understanding; • team building; • public recognition and award systems; • it infrastructure; • mutually beneficial performance management systems; • facilitating accurate perceptions of the organisation; • peer review and cross-peer learning; • the use of social networks and social organisation; • bricolage; • the judicious use of protocols at meetings and briefings; and ', bricolage is the ability to "make do with 'whatever is to hand' " (brown and duguid 1991:10). even when working on formal tasks, people will use whatever resources, social and material, that are available to them. when 'standard procedures' no longer seem adequate, they will improvise. people will develop models and knowledge that will enable them to achieve their goals (brown and duguid 1991; agostino et al 1999). this also implies that the actors have been given enough autonomy to be able to improvise and adapt their practice, as the circumstances require. 61 ajis special edition knowledge management december 2001 • effective formal information flows and records management. while these processes have a community focus, they also serve an important cultural and learning function for the individual, especially a new community member. for the individual, these processes reinforce acceptable forms of behaviour, encourage knowledge sharing and reuse, reflective practice as well as constructing an identity based on membership of the community. a viable organisation requires effective interaction between its diverse communities of practice. the processes that support this are: • systemic understanding; • the use of social networks and social organisation; • the use of mediating artefacts and bridging agents; • the judicious use of protocols at meetings and briefings; and • effective formal information flows and records management. it is apparent that the social learning processes and strategies can, and are, used in different ways depending on which level in the learning model they are applied to. the esla studies highlight that such distinctions are somewhat abstract as in practice a learning process is simply used by the actor to address a variety of needs. the esla studies also clearly identify and distinguish new members of the community and the learning strategies adopted by the community to induct these novices as full members. it is instructive to note that the same processes are used by the community on an ongoing basis, albeit for a different purposes, and are not restricted to induction of new members. for example mentoring is an important strategy for inducting new members into the community. but equally it is a process that requires established members to articulate aspects of the community's knowledge. this process of "making the invisible visible" is in itself a learning process for established member and is indicative of the critical role of social learning in a learning and adaptive organisation that shares its knowledge. the esla studies also highlight that there are fewer learning processes that facilitate the interaction between the community level and the organisational level of the model. this deficit is problematic in that it calls into question the process by which the consensual view is developed. this has considerable impact operationally as it potentially establishes a conflict between the community's understanding of the task, based on experiential knowledge, and the organisational view derived from an abstract understanding of the task. from the perspective of social learning, the esla studies indicate that the organisation uses mediating artefacts and bridging agents, meeting protocols and formal information flows and records management as strategies to support interaction between the organisation and its communities. these strategies fit into formal structures and cannot work alone to support learning. the interaction between the community and organisational levels was not explicitly studied in the esla project to date, so that the observations above are an interpretation and extrapolation of the existing studies. a summary of esla findings the findings suggest that social learning will flourish where the influences of power, trust, forgiveness and sharing are positively applied to the construction of knowledge. these factors provide a fertile ground for learning by allowing people to speak honestly, share information and offer suggestions without fear of ridicule or retribution ((hoffmann & withers, 1995) the positive outcomes that arise from the effectors in the learning architecture nurture and are nurtured by the overarching organisational values that include: • empowerment (where empowerment of staff also makes them accountable); • trust (which entails mutual respect); • forgiveness (in terms of allowing personnel to take reasonable risks, forgiving mistakes and facilitating knowledge construction on the basis of lessons learnt) • cultural cohesiveness (in terms of common identity, shared goals and a shared understanding) • commitment (which includes a mutual commitment and loyalty from the employee to the organisation as well as the organisation to the employee) • openness of the decision making process and • a culture of information sharing. what has emerged from the esla project is the paramount need for the ado to facilitate knowledge transfer and the construction of objectified organisational knowledge. to this end the ado has adopted diverse social learning 62 ajis special edition knowledge management december 2001 processes that aim to engender a learning organisation. the studies that have been conducted indicate how central social learning is to the organisation's effective performance while also highlighting the difficulty in applying these processes to the diverse activities that are undertaken. the challenge for the ado is to adopt these values in the process of transforming the ado into a learning organisation. these values need to underpin the enculturation of individuals and communities within the ado as well as to be expressed in the formal structures and processes of the organisation. the diffusion of values from the macro level of the learning model to the meso and micro levels provides a consistent narrative that underlies social learning and knowledge management in the organisation a knowledge management architecture for a learning organisation a knowledge management architecture that supports a learning organisation, shown in figure 3 below, accommodates actors in their ability to act effectively as individuals, and members of a community of practice, within an organisation. it also incorporates processes that enable the actors to perform their tasks, learn, and engage effectively in knowledge work as well as the means to objectify and document their understanding of their tasks. private personal community organisational public figure 3. a knowledge management architecture for a learning organisation the significance of this architecture is that it represents two largely antithetical processes through which the organisation maintains its viability. the left side indicates the social, and cultural processes through which actors engage in knowledge work, including knowledge construction, reconstruction and sharing. to a large extent this incorporates the formal and informal social learning processes and strategies. the architecture shows the left loop as predominantly a process flowing from the individual to the community to the organisation. the processes however are bidirectional providing feedback between all levels. this loop is essential because it is the means by which individuals and communities are able to act effectively, by adapting the formal, authorised definition of the task to changing cir63 ajis special edition knowledge management december 2001 cumstances. moreover this loop is informed by the over-arching organisational values and facilitated by the toolset identified above. on the other side is the formal managerial command structure that establishes the authorised procedures through which activities are performed. on the whole, there is little if any input from individual actors into formulation of these procedures although the architecture indicates that communities could have a formal role. the importance of this loop is that it establishes the context in which activities take place and contributes to the implicit knowledge that actors bring to the activity in terms of their understanding of the organisational imperatives that underlie the task. without this understanding diverse communities that constitute the organisation would not be able to work toward identifiable organisational goals. this also represents the full knowledge management continuum referred to in the introduction to this paper. it is our contention that the social and cultural aspects of knowledge management predominate through the left-hand side of the model, and the formalised organisational systems and tools dominate the right-hand side of the model. the architecture borrows the 3 levels of the learning model to identify the sites of discourse that provide the granularity necessary for a rich, multi-layered narrative of an organisation. the level of discourse is represented by the body of knowledge (bok) that is relevant to that level. included in the representation are two of the dimensions of the bok that influence learning within this architecture. the private/public dimension is resonant with the learning model (figure 1.) in that it identifies what knowledge is available for sharing. at the individual level it refers to what knowledge is available to all actors. at the community level it defines what the community communicates to the organisation about its activities. at the organisational level, this dimension deal with what information the organisation releases to the public at large. this is consistent with the learning model in that internal processes must exist that decide what knowledge and information is private or public. the implicit/explicit dimension relates to making sense of the world, and changes in it, and expressing and objectifying that understanding. it represents processes necessary to internalise learning. in the context of this architecture, both the left and right loops interact with the bok as a whole rather than a particular point within these dimensions. nevertheless, it is relevant that social learning does in fact accommodate knowledge that is useful and can be shared (mainly public and explicit). on the other hand, the objective and formal information that is the command loop needs to be assimilated and internalised in order to make sense of these organisational imperatives (mainly private and implicit). the discussion of these processes is outside the scope of this paper. the ability of the loops to work synergistically appears to be dependent on the implementation of all components of both loops. as discussed above, social learning processes are not extensively employed between the community and the organisation. this weakens the left loop and results in an over-reliance on the right loop in determining how tasks are performed. this reinforces the hierarchical, bureaucratic organisational structure to the detriment of other possibilities implied by an adaptive learning organisation for example one of the esla findings was that the building of a common identity through cultural cohesion was one of the tools for social learning. at the personal level, this related to how closely actors identified with their workplace and working colleagues. it was clear from the differences in the single service and joint settings that there were much stronger bonds in the single service environments than in the joint environment. at the community level, this cultural cohesion impacted on the shared understanding of the work activity and a shared vision for the organisation. at the organisational level this transformed into mission statements and stated organisational values. these same statements and values form part of the public image of the organisation. this public image then feeds into the community level of the organisation motivating the community and influencing how they perceive their goals and objectives. for example, some of the negative public perceptions of the defence organisation that were aired in the media during the esla studies had an immediate impact on the morale and perceived self-value of communities within the ado. they perceived themselves to be powerless and under-valued, and this directly impacted on how they viewed the goals and the objectives of their community. although in organisations where the right loop is not transparent, the public image can be interpreted at the community and individual level as rhetoric that is at odds with experiential perception. in organisations where the rhetoric matches experience the right loop between the community and individual acts to improve morale and job satisfaction by improving the individual's understanding of their role in the organisation. not surprising, this is in itself empowering and can be seen as propagating the organisation's values. the importance of this interpretation of the knowledge management architecture is that it reveals the potential for contradiction that are inherent in organisations as well as the complexity that arises from multiple viewpoints. what it also identifies implicitly is the processes that are necessary to integrate these viewpoints and a synthesis for the contradictions between the viewpoints. 64 ajis special edition knowledge management december 2001 links between the three models the knowledge management architecture shown in figure 3 adopts an organisational perspective that is sufficiently granular to reveal the organisation's internal functioning. each level of this architecture implicitly incorporates both the learning model (figure i) and the architecture for social learning (figure 2), albeit with different interpretations at each level. at the individual level, the model of learning represents the actors' engagement with the organisation through their participation in communities of practice. the social learning architecture, when applied to the individual level, represents the actors' interpretation of the organisational values and identifies what subset of the learning toolset applies to them and how these tools impacts their ability to contribute to organisational outcomes. at the community level, the learning model identifies the members of the community and their contribution to the community as well as how the community influences the viability of the organisation. the social learning architecture allows the community to adapt the formal organisational values to their activities and to adopt a toolset necessary to meet their organisational requirements. at this level, the community needs to overtly balance the effectors to produce the defined outcomes. at the organisational level, the learning model identifies the communities and its members and their contributions. in terms of the social learning architecture, as discussed above, values are often formally and explicitly stated and the toolset focuses on the formal training and learning programs conducted by the organisation. the organisation also explicitly identifies the effectors, and their net effect, in terms of the stated aims, objective and such measures as key performance indicators. the explicit/implicit dimension of the bok in the knowledge management architecture relates to the information and knowledge that exists at each level and relates to the ability of the learning organisation to manage knowledge in support of both its ability to perform work and to learn. the interpretation of this dimension, in the context of the architecture, can be limited to encoded and informal information and knowledge as the focus is on the social and cultural processes, which contribute (positively or negatively) to learning. thus the emphasis is on knowledge that can be articulated in some way. the meaning of the public/private and explicit/implicit dimensions of the bok varies at each level of the architecture. for example at the individual level, implicit could refer to the experiential knowledge an actor acquires through work performance. by contrast, implicit at the organisational level could refer to the cultural significance of myth expressed through the re-telling of stories relating to episodes in the organisation's history (shanks, 1990) conclusion in this paper, we have presented a series of models that can represent the whole continuum of interests and issues related to the social and cultural aspects of knowledge management. this continuum ranges from the transformation of individuals' implicit knowledge to the implementation of formal processes and tools that store, support and provide corporate knowledge within organizations. the organisational social learning that underlies successful knowledge management in organizations was mapped unto these models. the esla studies are valuable in that they empirically reveal the social processes and cultural norms that make the organization viable. importantly the data highlights how significant social learning is to the organization, while ironically it shows that these processes remain largely unacknowledged in their contribution to organisational viability. the studies "make the invisible visible". 'architecture', like knowledge management, has a range of meanings and uses. it can simply mean a high level description (model) of the system to be built. to others, it is conceptual or logical as opposed to physical. to others still, 'architecture' is 'requirements' whereas to some, it is simply a set of 'principles' (zachman, 1999). to the authors of this paper, it is used as a tool to facilitate understanding of complex issues. it is apparent that the models and architectures presented in this paper play multiple roles in our research. they have been, or could be, used as: • a conceptual framework that informs the conduct of a study in any setting; • an analytic tool to organise and interpret data • a framework to summarise findings • a diagnostic tool to identify breakdown • a design tool for organisational change the longer term aim of our research is to utilise the models presented in this paper to define architectures for support systems that facilitate knowledge sharing, and particularly the ability to make more explicit the knowledge and practice of the community. such systems would be an integral aspect of the transformation of the organization into a learning organization. 65 ajis special edition knowledge management december 2001 references agostino, k., gori, r., and wame, l. (1999) the enterprise social learning architectures task: context and conclusions from the pilot study (restricted dsto-rr-0160), canberra: dsto. ali, i., pascoe, c., warne, l., gori, r., and agostino, k. (2000) enabling learning: a progress report on the enterprise social learning architecture task, proceedings of the 5th international command and control research and technology symposium, canberra, australia brown, j. s., and duguid, p. (1991) organisational learning and communities-of-practice: toward a unified view of working, learning and innovation, science, 2, 1,40-57. davenport, t. h., and prusak, l. (1998) working knowledge: how organisations manage what they know, harvard business school press, boston, massachusetts. flood, r. l. (1999) rethinking the fifth discipline: learning within the unknowable, routledge, london. hildreth, p., wright, p., and kimble, c. (1999) knowledge management: are we really missing something?, information systems the next generation: proceedings of the 4th ukais conference, york, uk, 347-356. hoffmann, f., and withers, b. (1995) shared values: nutrients for learning, in learning organisations: developing cultures for tomorrow's workplace, productivity press, portland, oregon. levitt, b. and march, j.g. (1988) organisational learning; annual review of sociology, 14: 319-340. livari, j. and linger, h., 2000, "the characteristics of knowledge work: a theoretical perspective", proceedings of the americas conference on information systems, amcis'2000, long beach california, august. linger, h., burstein, f., kelly, j., ryan, c. and gigliotti, p., 2000, "creating a learning community through knowledge management: the mandala project", proceedings of ifip wp8.3 conference on dss and knowledge management, stockholm, july. linger, h., 2000, "re-inventing the firm: organisational implications of knowledge management", 4th international conference on business information systems, bis'2000, poznan, poland. linger, h. burstein, f. zaslavsky, a. aitkin, c. and crofts, n; 1998, "an innovative organisational tool for epidemiological research"; european journal of epidemiology, 14:587-593 senge, p. m. (1992) the fifth discipline: the art and practice of the learning organization, random house, sydney. shanks, r. (1990) tell me a story: a new look at real and artificial memory, macmillian publishing co. new york, usa. ward, a. (1998) knowledge management, http:/www.zdnet.co.uk/itweek/brief/1998/27/knowledge/ http:/www.zdnet.co.uk/itweek/brief/1998/27/knowledge/. wame, l. (1999) a sociotechnical approach to social learning analysis in the australian defence force, in e. coakes, r. lloyd-jones, and d. willis (eds.), the new sociotech: graffiti on the long wall, springer-verlag for the british computer society, london. warne, l. (2000) understanding organisational learning in military headquarters: findings from a pilot study, australian journal of information systems, 7, 2, 78-87. warne, l., ali, i. m., pascoe, c., and agostino, k. (2001) "a holistic approach to knowledge management and social learning: lessons learnt from military headquarters" australian journal of information systems, special edition on knowledge management. wenger, e. (1998), communities of practice: learning, meaning and identity, cambridge university pres, cambridge, uk zachman, j. a. (1999) a framework for information system architecture, ibm system journal, 38, 2/3,454-470. 66 ajis vol. 8 no. 2 may 2001 echoes of semiotically-based design in the development and testing of a workflow system clarisse sieckenius de souza1, cecilia kramer vieira da cunha1, raquel oliveira prates1'2, simone diniz junqueira barbosa1 'departamento de informatica, pontificia universidade catdiica do rio de janeiro, brazil 2departamento de computacao, institute de matematica e estatistica, uerj, brazil abstract workflow systems are information-intensive task-oriented computer applications that typically involve a considerable number of users playing a wide variety of roles. since communication, coordination and decision-making processes are essential for such systems, representing, interpreting and negotiating collective meanings are a crucial issue for software design and development processes. in this paper, we report and discuss our experience in implementing qualitas, a web-based workflow system. semiotic theory was extensively used to support design decisions and negotiations with users about technological signs. taking scenarios as a type-sign exchanged throughout the whole process, we could trace the theoretic underpinnings of our experience and draw some revealing conclusions about the product and the process of technologically reified discourse. although it is present in all information technology applications, this kind of discourse is seldom analyzed by software designers and developers. our conjecture is that outside semiotic theory, professionals involved with human-computer interaction and software engineering practices have difficulty to coalesce concepts derived from such different disciplines as psychology, anthropology, linguistics and sociology, to name a few. semiotics, however, can by itself provide a unifying ontological basis for interdisciplinary knowledge, raising issues and proposing alternatives, that may help professionals gain insights at lower learning costs. keywords: semiotic engineering, workflow systems, information-intensive task-oriented systems, scenario-based design and development of computer systems, human-computer interaction introduction over the past few years workflow technology has been gaining territory in many organizations as a tool to improve coordination and enhance the quality of processes and products. traditional information systems have been integrated into this technology, allowing users to retrieve information for decision-making, to turn decisions into action, and then to generate more information relative to and derived from these very processes. however, one of the most important challenges associated to workflow technology is to avoid permanent commitments to current organizational practices that can eventually prevent processes from adapting to continuously changing contexts. research in this field has produced a number of alternative ways to face this challenge [khoshafian & buckiewicz (1995), dourish et al. (1996), heinl et al. (1999), malone et al. (1995), glance et al. (1996)]. as is the case in the field of human-computer interaction (hci) in general, two major paths to accommodating variety can be identified: • trying to capture the widest range of work patterns, actual and potential, and affording those to workflow systems users; and • allowing for customization and extension of implemented workflow systems. in this paper we will be addressing the first path to designing and implementing workflow systems'. we will be showing how semiotics has been used to inform and support the design and development of one particular webbased workflow application — qualitas. the goal of this system was to help maintain the quality of services certified by iso9001 standards in a large brazilian company. our customers expected the system to lead the employees involved in training, consulting, evaluation and coordination activities to abide to a set of certified norms, and to improve the efficiency of the overall process. their view of this project was that it should suffice to translate certified norms into computerized tools that would generate the same middle and top management indicators they currently used, and they would then reap the benefits of computer-based technology. an instance of such benefits would be the possibility to readily obtain indicators and export them to spreadsheet and database applications, and proceed much faster with decision-making processes. this view represented an extension and partial integration of information systems network installed in the company, but considerations about the actual work context in which people generated and used information failed to be explicitly formulated and discussed in their initial request for our participation in their project. a consistent analysis of how people would react to their job being exposed to others through the indicators presented in a new technological environment was also not explicitly carried out. thus, our first and foremost task was to discuss the various dimensions of workflow technology with our customers, so that the application we developed would have a greater chance of achieving their expected goals. we started by discussing what the new technology would be, compared to the existing information and decisionmaking systems in the company. both parties agreed that the technology should be a web-based informationour ongoing research about a semiotically-based approach to the design of extensible workflow systems is reported in [cunha et al. (2000)] 115 ajis vol. 8 no. 2 may 2001 intensive task-oriented system (wits), a special kind of workflow system. an emphasis on information within the context of organizational tasks was intended to communicate that our aim was explicitly one of informing work practices, not of automating them. subsequent steps involved a lengthy process of negotiation and design, around technologies, organizational goals and practices, and user studies, among others. throughout this project, semiotics has guided design and determined the quality of the final product. three basic concepts in this discipline have been used: • the peircean sign [peirce (1931, p. 99)]; • unlimited semiosis [eco (1979, p. 69)]; and • abduction [peirce (1931-5, pp. 151-156)] and pragmatics [peirce (1931-5, chapter 17)]. in peircean semiotics, a sign is anything that can stand for something else for some interpreter. although this definition seems to be extremely vague and broad, its very vagueness points to a central tenet of semiotic theory — what makes anything meaningful is the fact that there is somebody for whom it means something (in fact, anything). therefore, it is the observer or listener the interpreter who establishes the necessary and sufficient conditions for anything to be a sign [eco (1979, p.16)]. for example, in a work situation, everyday before he leaves, a tidy employee may be intentionally arranging his desktop in a certain way, in an effort to tell the cleaning staff how he expects to find it in the morning. however, to his disappointment, the cleaning staff doesn't realize there is an intentional arrangement of objects on the desk. but, what is a sign to the employee, isn't one to the cleaning staff, who rearrenges everything, everytime. by virtue of the other facet of the peircean sign, the employee may interpret the different arrangement of his desktop as a sign that the cleaning staff means to be irritating, regardless of what is actually the case. semiosis is the process of assigning meaning to an observed form or phenomenon. the structure of the peircean sign involves three elements: the object, the representamen (or representation of the object), and the interpretant (the ideas, actions, feelings, or other signs triggered by the observation of the representamen or of the object). the most relevant feature of semiosis is that it consists of a chain of interpretants, whose elements and length are unpredictable. in the example above, it may simply not be the case that the cleaning staff is trying to make any statement at all about the employee's desktop arrangement. however, the employee may be generating an elaborate and completely subjective interpretation for something that is totally unfounded. as long as interpretations don't depend on objective grounding, semiosis can be out of control for anyone who is generating what is taken to be the representamen of the subjective sign. finally, abduction is a reasoning method that has been previously characterized as inference to the best explanation [josephson & josephson (1994)]. the gist of this method is to create an explanatory hypothesis for an observed form or phenomenon, and to try and confirm it with reinforcing evidence. as long as there is no counterfactual evidence, the hypothesis holds and every interpretation of the associated phenomena may be taken as a reinforcement of the hypothesis. if at one point observed information challenges the validity of the hypothesis, it is revised and reformulated until another satisfactory hypothesis is found to accommodate the new facts. carrying our example one step further, suppose the employee has concluded that the cleaning staff is trying to annoy him. this is the meaning he has abducted from observing helplessly his rearranged desk every morning when he arrives at work. however, let us suppose that employee and cleaning staff meet one day, and that they talk to each other. the cleaning staff then tells the employee that they arrange all the desks in the same way, and that they thought they were doing a good job. they are sorry if they have annoyed him, and they will keep his desk as it is from now on. the employee is thus forced to revise his beliefs and conclude that his interpretation (or abduction) of facts was very wrong. the openness of interpretation in semiotic theory has challenged many semanticists, especially those that take a more platonic view of meaning as is the case in many logic approaches. nevertheless, peirce himself was a logician, and the founder of pragmatics, a discipline motivated by the need to explain how and why, despite of unlimited semiosis and the subjective nature of signs, people can achieve mutual understanding and interact productively. at the heart of pragmatics lies the notion of negotiating meanings, given the goals of speakers and interpreters in context. situations constrain, or if not as much at least motivate, semiosis. individual and collective goals determine how close or how far from each other speakers and listeners will intentionally allow their interpretants to be. to illustrate a pragmatic principle in action, we can carry our example still one step further and suppose that in the hypothetical dialogue where the cleaning staff is telling the employee why they were rearranging his desk every day, they say that: "we do this because of procedure p001c5, which we have been trained to execute in our company." unless the employee extensively knows the procedures for cleaning offices, it is very unlikely that he will be able to assign to procedure p001c5 the same meaning that the cleaning staff does. but this doesn't preclude them from achieving satisfactory mutual understanding. the employee's vague interpretant of the term (standing for some norm or procedure that has nothing to do with me personally) is pragmatically sufficient to help the conversation reach its goal and reestablish a good relationship between these people. semantically, it can not only be argued, but also probably shown, that these two parties were not talking about the same objective thing [frege(1892)]. 116 ajis vol. 8 no. 2 may 2001 concepts of the sign, semiosis, and abduction have been extensively used by our design and development team, who chose scenarios as an on-going representation of meanings that should be shared and agreed by customers, users, and developers. it should be clear from this introduction that the meaning of technology, of information that flows in it, and even the tasks that can be carried out in computer-supported environments, are not objective for humans, although they are so for symbol-processing systems. in other words, although managers, for instance, may pragmatically generate different interpretants for performance indicators they see on screen at two different points in time, any computer procedure embedded in the system that uses those indicators to produce others will assign to it one specific unchanging (ideal) programmed meaning. and this may affect the workflow in very important ways. a semiotically-based approach to design and development of computer technology will take these factors in consideration. in our specific case, we will be using different aspects of our own semiotic engineering approach [de souza (1993)] to help designers generate and users interpret the algorithmic meanings [nake and gabrowski (to appear)] embedded in wits. semiotic engineering principles that affect the design of wits can be summarized by the following: • interactive software is a one-shot message sent from system designers to system users about the range of interface messages users can exchange with the system in order to achieve a certain set of goals [de souza (1993)]. • the system is the designers' deputy inasmuch as it enacts all and only the conversations that the designers thought they would have with users given their interpretation of who the users are, what they need and want, what they like and prefer [prates, de souza and barbosa (2000)]. • every interactive system has one unique language that can be used in the communicative exchanges between users and system [de souza, barbosa and da silva (to appear)]. • it is critically important that this uniquely designed interactive language allows users to generate consistent interpretations of interface signs by letting them abduct its fundamental design principles [de souza, prates and carey (2000)]. in the other three sections of this paper we will show that scenarios have been chosen as the basic representamen for various semiosic chains that emerged throughout the development of qualitas. we will discuss some aspects of technology-based signs and the contributions of semiotic engineering to designing and developing wits. our focus will be placed on human-computer interaction, more specifically on the importance of helping users understand technological affordances and designers turn technologies into tools [adler & winograd (1992)]. signs and semiosis in technology as we have seen in the previous section, technology introduces new signs in the domain. in this section, we will be analyzing the nature of signs and how they are generated, taking into consideration facts about the obsolescence rate of processes, the turnover of employees, and role-switching perspectives on organizational discourse. wtts signs can be classified into: data, information, and knowledge. this is a traditional categorization, especially in the field of ai [newell (1982)], but we will highlight some semiotic distinctions between each class. from the system's perspective, data is the grounding upon which a number of abstractions can be built. it functions as raw material for all information processing inside and outside the system. the representations of data are the focus of important topics in programming languages, namely data structures. information is precisely what we gain as we abstract upon data. it is dependent on data models that reflect the domain and typically emphasize static relations among entities. knowledge, however, provides explanations about how information is derived from data. it makes use of an ontology of the domain, from which consistent representations and inference rules are derived. occasionally, if systems have meta-knowledge, they could also provide explanations about how knowledge itself is used. in sum, the major semiotic difference between these three classes is that data is an arbitrary symbol whose semantics is outside the system. information presents an abstraction that can be processed by the system. it is causally dependent on data (which in semiotic terms amounts to an index), but this causal relation cannot be manipulated by the system. at the knowledge level, the system is able to account for the processes that derive information. in other words, at the knowledge level the causal relations between information and data are computational symbols that the system can manipulate in inferential processes. these concepts may be illustrated by the following situation. 117 ajis vol. 8 no. 2 may 2001 services that do not conform to norm when training services are provided by employees in this company, an evaluation sheet is passed to the participants who rate the service on a scale from 1 to 5. the certified norms state that the acceptable quality of services is that at least 80% of the ratings fall between 4 and 5. otherwise, a process of investigation and reporting non-conformity to the standards must be started. in this case, the ratings are the data, the acceptability of the service is the information, and the remedial action that is required is knowledge about work practices. in qualitas, we have these three levels of signs. employees can inspect the individual ratings and be informed if there is a problem with the quality of services. whenever there is, the system itself triggers a series of actions that will lead to a decision relative to the problem that occurred. we can see that the complexity of the signs increases from a single number to a whole sequence of interactions and commitments. in systems without the knowledge level, users may have the diagnosis of the problem, but they will have to decide what to do and take action themselves. if the system doesn't even have the information level, then users may fail to diagnose the problem altogether. another interesting aspect of the above classification is related to symbol grounding. all the data imported or input is computationally improvable from the importing system's perspective, in the sense that the system takes it as true. on the one hand, although users may know that the data was derived from an elaborate causal chain of facts, as far as the system is concerned, it is an axiom. on the other hand, all the information the system derives from the data is computationally provable and, as long as the system has a knowledge level, it is also explainable. as a consequence, in knowledge systems, information is semiotically an index (a sign that has a contiguous or causal relation to its referent) that can therefore be negotiated, albeit in a limited way, among users and systems. obsolescence of signs one of the best-known problems of knowledge systems is that the cost/benefit ratio in certain domains is not acceptable. this is due primarily to the lack of established knowledge or practices. if knowledge isn't consolidated and is prone to "quantum leaps" as people gain new insights about what they're doing, the maintenance cost of knowledge systems isn't compensated by the benefits of its use. the rate of obsolescence is typically high. the benefits of using knowledge systems are only reaped when there is maturity and tradition in the domain. in this case, the bulk of knowledge is considerably stable, and the evolution is smooth. this allows designers to introduce knowledge acquisition and/or machine learning facilities in the system, which will ensure longer term usability of the system's knowledge base. from a semiotic perspective, non-consolidated domains present a high rate of sign obsolescence, whereas consolidated domains provide the basis for more long-lasting grounding of signs and meaning-negotiation processes. it may be argued that explanations are relevant for a fixed set of users only during a limited period of time. once they know why the system generates certain kinds of information, they no longer need clarifications, i.e., the knowledge becomes tacit. nevertheless, from an organizational perspective (and even as an inherent characteristic of certain domains), the set of users is typically not fixed, due to the turnover of employees. in this situation, explanations are crucially important to support training processes and to help newcomers acquire knowledge not only about the system, but about current practices as well. in semiotic terms, tacit knowledge is typically knowledge that needs not be abducted, which in artificial intelligence is often referred to as the common-sense knowledge of a certain community. experimenting the alter semiosis one of the greatest problems in human communication is the fact that we often misjudge how we are going to be interpreted by our listeners in conversation. we hope to make ourselves understood, but some facts that determine interpretation are out of our reach. in human conversation, there are a number of mechanisms that help us remedy misconceptions and misunderstandings that may arise. however, hi medium and large organizations, the very structure of the company makes it difficult, if not impossible, for employees to have a clear grasp of how their work practices and products are being interpreted by others. a great advantage of wits technology is that it contains a model of all the roles and group structures that are involved in organizational processes. as long as these models can be inspected, they allow an employee assigned to a certain role to experiment other roles (in demonstrational or educational simulations). he may realize how the kinds of signs that are generated by him when performing his assigned role are being viewed in a different context. for instance, let us suppose that his productivity indicators are exceptionally good, but that his colleague, whose indicators are not as good, is the one who gets a promotion. locally, this seems unfair to him. 118 ajis vol. 8 no. 2 may 2001 however, if he could play the role of a top manager, he would perhaps understand that, globally, his colleague's performance has had a greater impact on the company than his own. we refer to this possibility as experimenting the alter semiosis. outside the technological setting, the success of this experimenting is strongly dependent upon someone's abilities of detached observation, that is, of not letting his own biases play a distorting role in interpreting reality. with technology, we may reduce these distortions by providing enough information to engage users in someone else's role. we hope that the technological protocol that derives from modelled organizational practices provides a large volume of evidence to motivate semiosis that is consistent with the underlying models. from design to testing: signs and scenarios scenarios are stories about people and their activities (carroll (2000), imaz & benyon (1999)). they can be used throughout different stages of system design and development. what they contain, and sometimes lack, reflects a state of our own understanding of what the users do and want. it also keeps the focus in meaning negotiation processes. technological proposal scenarios at the early stages of the project, the goal was to explore and discuss the potential of the technology. after an introductory study of the domain (mainly through interviews and readings of existing documentation), technological proposal scenarios were created. they were a mock-up of our own vision about the future system, simulating a few activities considered relevant by the user community. this introductory level allowed for the creation of an initial set of shared meanings. these scenarios also supported the definition of the scope of the system, which involved a negotiation of the users' priorities in combination with technological constraints and organizational planning. at first, we mapped almost directly the existing paper forms onto html forms. the idea was to capture and record the data input in the various fields into a global database, from which indicators would be generated for management. however, at this very stage, some technological signs were introduced, as we can see in figure 1. in the first proposal we had a 1-to-l mapping from paper to html form. the ok and cancel buttons were technological signs introduced at this stage, and the ok button led users to visualize the complete filled-out form. this form's layout was a replica of the original paper form, in order to help users connect their new experience with their familiar practices. soon after that, technological constraints played a major role in leading us to a second proposal. the original paper forms contained too much information to fit in an interactively appropriate html form. if the first proposal were implemented, users would have to scroll through excessively long pages. because of this interactive restriction, we split the single html page into a number of pages. however, users then needed contextual information from one page to another, and also needed to be informed about the progress of the work. notice that, in the second proposal, the name and client fields from the first html page are copied as text to the next. moreover, a third technological sign, the view button, was explicitly introduced to allow access to partially filled-out versions of the original form. these scenarios try to capture not only the gist of work practices as they are currently performed and as they should be performed in the new technological medium, but also the kinds of additional knowledge users should be prepared to acquire. specifically, we conveyed in the technological proposal scenarios the idea that training users in web-based systems would be the minimum requirement for the whole project to succeed. this was important because the kinds of technology used by each individual in the community of users were very different. but because the technological proposal scenarios were typically run by one presenter for an audience of representatives of our customer, die alternation of roles and users (required for a full-fledged computer-supported collaborative system implementing those ideas) was not explicitly included in the mock-up as a designergenerated sign in the technological language under examination. that is, in many occasions the presenter would tell the audience, as she moved from one screen to the next, that the person who would be seeing the next screen was a manager, an employee or a company's client. 119 ajis vol. 8 no. 2 may 2001 html form (2nd proposal) paper form html form (1st proposal) r name: client: date: service name: client: date: name: client: name: john smith client: acme co. date: service: i name: client: date: service html visualization form (same layout as paper form) figure 1 an instance of evolution in technological scenarios. whereas this expedient had the undesired consequence that our mistaken interpretations about the roles of users in different tasks were not evident to our customers till later in the design process, it served to show them the difference between automating and informing people's work. although they had verbally agreed with us in previous meetings about what this difference meant, it was not until they were presented the technological sign for it (the mock-up scenario the presenter run) that they realized they did not know about a number of positive aspects of wits technology. other technological proposal scenarios allowed us to show some advantages brought about by the technology, such as the derivation of various indicators that would otherwise be too laborious to be generated. these indicators would also be new signs introduced in the domain, through the use of this technology. when presenting technological proposal scenarios, users should be made aware that the signs depicted in the mock-ups are temporary representations. users should able to have a "feeling" of what the technology is going to be like, although they must be warned that changes are going to happen in the future. in other words, we are talking about sign-types, although we are using a sign-token as a placeholder. for instance, the ok button had its label replaced by the name of the next stage in the process. user studies scenarios these scenarios extensively described the work processes. at first, we gathered information from studying existing documentation, which was mainly composed of the application of iso9001 regulations to this particular company and the corresponding paper forms already in use. the interesting point in these studies was the awareness that norms prescribe ideal processes, whose initial input is typically abstracted and omitted. the consequence for work practices is that people create their own habits for non-regulated processes, and this is precisely the reason why it is so difficult to introduce technology that will not only implement regulated processes, but also has to accommodate personal variations in carrying out jobs. what matters to the norm are the protocols through which information is fed into the process. technology designers have a choice to implement strictly the norm, or to support the surrounding processes that generate information that is protocoled to the norm. the risks involved in the first choice is the potential impedance between the quality of the regulated process and the quality of the information required for this process. this information may be produced under circumstances that cannot warrant an adequate degree of quality. in qualitas, we made the second choice. for instance, we observed that an occasionally lengthy preparation stage was being carried out informally by employees. the paper forms were used as a draft to gear the final documented version. drafts were used as a basis for negotiating contracts with clients, and only when a final agreement was reached was the final form produced and filed. we chose to include this early stage in qualitas, in order to provide better support for the negotiation processes with the client, and avoid some amount of rework involved hi transposing information from drafts to the protocoled form. figure 2 shows schematically the 120 ajis vol. 8 no. 2 may 2001 difference between the flow of processes before and after the introduction of the technology. informal before regulated processes record proposal analyze proposal after infomatized processes prepare proposal record proposal analyze proposal figure 2 users' work processes before and after the introduction of the technology. another problem with the norms was that they did not prescribe remedial actions for all kinds of exceptions that occurred in work situations. as a result, employees took relatively arbitrary decisions about how to proceed, and this again jeopardized the overall quality of the regulated process. since a major requirement of any software development process is to handle exceptions in an adequate way, the very introduction of technology triggered a knowledge elicitation process in the company. in it, the whole community of users was prompted to describe known exceptions, the way they handled them, as well as to envision other plausible exceptions and remedial actions for them. for instance, a portion of the norm stated that: "... the conditions of the contract should be formally communicated to the customer within five business days after the initial request of service..." however, it didn't explain how to proceed if, for some reason, the customer was not notified within the deadline. it was implicit that, in this case, a non-conformity report should be generated and the remedial process undertaken. but the very fact that the text didn't explicitly say so allowed for different interpretations of what a non-conformity really was. in order to explore the extent of latent practices, one of our scenarios explicitly introduced a technological contingency that would make a whole set of norms become completely inapplicable. we took the norms that referred to the procedures involved in preparing materials for a conference and used it to show them that the text was only applicable to very traditional conference settings. there were references to the quality of "overhead projectors", "slides", "xerox copies", and the like. we simply showed that in case of videoconferences, for instance, virtually 80% of the checklist imposed by the norm would be useless. what would they do then? reaction to these scenarios helped designers sense the latency of interpretants, which plays a major role whenever exceptions contexts arise. progressive implementation scenarios throughout the design and development process, many iterations of technologically-grounded scenarios were discussed with customers and users. they invariably led the participating teams to revise their beliefs and adjust interpretants about the company's goal, the employees' work practices, and the technological requirements. however, the most interesting aspect of these scenarios was that they helped us identify the emergence of new types of signs in the technological language. in the following, we will discuss two instances of this situation. the generation of a contract's history because wits must have internal representations for both information and tasks, it can generate logs of users' activities within the scope of the system. as a consequence, it was easy to generate a brief summary of how users carried out their jobs providing services involved in contracts they signed with their customers. an example of a summary can be seen in figure 3 although the text is in portuguese and the narrative is totally fictional, it shows the reader that even a bar chart is included in the document, depicting quantitative indicators relative to the service provided by the user. 121 ajis vol. 8 no. 2 may 2001 consuitona e'ducagao avaliagao articul, e coord. sumario do contrato n-: s 003/2000 0(a) tecnico(a) giupo tsdoc abriu em 21/07/2000 urn contrato de prtsucao de serri;o de consuitona, atendendo ao pedtdo feito em 19/d7/2000 poi oiupo tsdoc (multi-client^ em nome do 6rgao reduc. 0 contrato recebeu o n2 s-003/2000 e tern as seguintes caractensticas: titulo: tests 3 violeta requisites e expectativas do client*: tesle 3 violeta recursos diditicos e/ou de infraestzutura: teste 3 violeta duracao total: 2 he data inicial: 21/07/2000 hh x controto data final: 21/07/2000 i local de atendimento: teste 3 violeta parcerias: teste 3 violeta responsabilidade pelos recuisos financeiros: susema crono grama: fase 1: teste 3 violeta 21/07/2000 2 hs anajise critica a anilise critica do contrato, reakzada em21/07/2000 por orupo tsdoc cntetios de atendimento: *-001/2qoo 7-002/2000 ?-oq3/2000 7-go4/2000 m-001/2000 0-001/2000 q-ooj/2000 q-007/2000 il figure 3 an automatically generated summary of work processes. in this scenario we see that the availability of internal system representations used by developers to check for consistency and keep track of rum-taking in complex collaborative goals, in a way motivated the interpretant of a process history. designers saw internal representations as a sign of system behavior throughout the various wits activities. thus, they easily concluded that this could also be of interest to the users, themselves. the emerging sign was readily accepted and adopted by users, who recognized the contribution of technology at this point. performing technological signs as soon as qualitas was introduced in the company, even in its early alpha and beta versions, it became clear to customers and users that it was a performing version of the norms and procedures certified by iso9001. prior to qualitas, the only way for an employee whose role was to provide a consulting service to understand what it was like to be his coordinator was to read about the coordinator's role (in the documentation) or to talk to the coordinator about it and to wonder. however, after qualitas was running, a demonstration version of it (which later was used for educational purposes) allowed users to play the role of whoever they wished, and to gain a much wider understanding of what it meant to be someone else as far as work practices in that company were concerned. this is another example of the way the designers of qualitas used internal representations to communicate to users some interesting information they, designers, could "read" in the system. they had a global view of how roles were interconnected and how the overall process in the company was carried out by actors playing these roles. thus, they saw in this an opportunity to explicitly represent this emerging sign at the interface, for users to explore in a demonstration version. this may be considered a technological sign at a meta-level if compared to the summary sign, because it assumes that users will switch between application modes and contexts. test case scenarios test case scenarios have been, along with training and technological evolution scenarios, the most revealing ones in the context of wits. because qualitas was a kind of computer-supported collaborative system, test cases had to cover coordination and communication aspects of as large a portion of its operations as possible. the scenarios we generated for this stage were cast as a sort of role playing game (rpg). a group of approximately ten users was selected and assigned roles and tasks they should be engaged in acting out for nearly two weeks. the period of time was decided on the basis of plausible deadlines dictated by the enforced norms and procedures of the company. as these were the last batch of scenarios prior to the system's full installation in the company, they made use of all the range of signs emerging in the company's context — not only in the system's context. they referred to contracts, clients, divisions and managers who were known facts, people, places, occasions, and settings for that community. although the tasks people were performing in their rpg-like activity had no further consequence 122 ajis vol. 8 no. 2 may 2001 beyond the scope of a simulation, the realistic grounding of the game contributed to engage the users. the value of these scenarios, in addition to the familiar and expected indication of occasional bugs and opportunities for improvement in the software itself, was to serve as a testimony of the company's maturity acquired by virtue of new technological settings. a comparison between test case scenarios and the early versions of the certified norms and procedures, the technological proposal scenarios, and the user studies scenarios, showed how much knowledge had been gained in two years of intensive research and development work. both, our customers and us, had learned much about what we do and what the others do. training scenarios like test case scenarios, training scenarios have given the design team and the customers alike a unique opportunity to reflect upon work practices. the role played by signs in this context has been paramount. the most interesting of all instances is reported below. the flow of processes in the first training session for managers who were going to use qualitas from then on, the instructor presented them a schematic overview of the whole system, where simple graphic conventions were meant to show the audience the various processes covered by the system, the different professionals involved at each stage, and the dependencies between action taken at one point and the others. the graphic was prepared by the head of the design team, who was trained not only in hci and computer science, but also in semiotics and rhetoric. this led her to depict what she thought were the important abstractions and invariants of work practices across different areas of the company, the connections between these and the interactions afforded through the system's interface, and the projection of roles people played onto processes they were involved with in the overall organizational dynamics. her view was determined not only by the whole history of the design process she coordinated, but also by her aim to teach the audience how she thought the system reflected the essence of their work practices and introduced new ones that she thought would be beneficial for the organization. a picture of the flow of processes she used for didactic reasons is seen in figure 4: figure 4 the flow of processes carried out inside and outside qualitas the surprising effect of the diagrammatic sign appearing in one of the instructor's slide was that it caused an immediate reaction from the audience. the trainees related to the flow of processes with an intuitive understanding that helped them have an instant gestaltic perception of the technology. this was evidenced by the follow up questions they asked and the scenarios they built themselves to query the functioning of the system. they immediately asked for a copy of the diagram. discussion it should be concluded from previous sections that scenarios have played a central role as representamina that were used to induce a variety of interpretations throughout the development of qualitas. in this discussion we will call the reader's attention to the semiotic qualities we have explored in scenarios (for a comparison between the scenario-based software development process of qualitas and that of similar applications, see monteiro, barbosa and de souza (2000)). information technology has been often mistaken by both its producers and consumers in terms of the effects it can bring about in organizational environments. whereas it is undeniably true that there are many benefits in 123 ajis vol. 8 no. 2 may 2001 providing a systematic treatment of information, that can be thus integrated into larger systems and processes, some of the consequences of computer-mediated information and work flows have not always been adequately analyzed [suchman (1995), dourish et al. (1996)]. our choice of technological proposal scenarios in the form of a mock-up web-based qualitas environment was a (performing) sign of how certain tasks would be changed by the introduction of wits technology. as we said, our focus was not on interface widgets and screen layouts, which were intentionally left very sparse, but rather on the types of conversations that we expected to occur in the system. these conversations involved new languages and new signs which we specifically brought forth in order to trigger our customers' interpretants in unpredictable semiosic chains. the scenarios were told in these languages. since in our approach an achieved system is a one-shot message from designers to users, the ultimate purpose of our technological proposal scenarios was to foster an ongoing focused dialogue about what this one-shot message should eventually be. the approach helped us negotiate meanings in constantly evolving sign systems. misconceptions that went unnoticed in one sign system, often emerged and were remedied in a more elaborate one. the major contribution of semiotic theory dealing with technological proposal scenarios was to give the designers awareness of an emerging unique technological language (or sign system). the semantics of signs in this language is algorithmically embedded in the system. the words, phrases and conversational structures that will occur during interaction must engage users in semiosic processes that will pragmatically settle around interpretants that are consistent with programmed meanings and with culturally grounded work practices. another important contribution of semiotic theory is a conceptual framework or, depending on the designers' expertise in the field, an ontology of communicative phenomena that can be put to use in order to produce a fair characterization of the designer's deputy. the technological proposal scenarios function as a reification of a mutually agreed interpretant of what the technology should be. once the meaning of the proposal is settled, a thorough analysis of future users in their current situation must be carried out as part of requirements engineering and user modeling processes. user studies scenarios in qualitas were intentionally generated by the design team (and not by users), after numerous interviews, observation sessions, documentation analysis, and negotiation with customers. our aim was to produce another reification of a mutually agreed interpretant, but this time it should be thoroughly focused on the users' work practices, independently of considerations about technological change. user studies scenarios were told in a language that was totally familiar to users, unlike the case of previous scenarios, where the language was an innovative technological sign system. cases were described in natural language narratives, where fictional characters performed non-fictional tasks in the company. the tasks included situations that had actually occurred in the analyzed history of contracts, as well as some plausible variations on those. paper forms and standardized procedures were examined in these the scenarios, with a special emphasis on arbitrary interpretations the design team was aware it had generated due to incomplete information provided in the resources they had used. another emphasized topic in user studies scenarios, as seen in the previous section, was that of exceptional situations, which helped designers explore latent interpretants in the community of users. the contributions of semiotic theory for elaborating and interpreting user studies scenarios were twofold. on the one hand, the theory highlighted the abundance of arbitrary abductive processes followed by the employees (our future users), which not always settled around a consistent set of interpretants. this was a warning for our team, in that we should be able to let our users know as exactly as possible which of all registered interpretants the implemented system would be prepared to cope with. and although all interpretants were socially efficient prior to the introduction of qualitas, allowing the company to operate pragmatically in line with acceptable standards, the technological protocols embedded in qualitas could bring about major changes in that scene. on the other hand, our team was also faced with the situation that normative texts are a genre where argumentation is an inadequate rhetoric to use. therefore, the exploration about the reasons for norms, which would immensely help employees decide what to do in case of exceptions, was typically left out of documents generated from or about work practices. notwithstanding this aspect of norms, information systems are exactly the kind of technology devoted to helping users decide what to do when they find themselves in a situation they are unsure of their next step. therefore, it would not be appropriate to merely transpose the signs that occurred in the organizational discourse genre prior to the introduction of qualitas into the new system. a thorough interpretation and evaluation of signs in the new genre was called for, so that the users could the benefits of wits beyond and instead of fearing that their job would be automated and they would thus lose their status in the company. moving on to the progressive implementation scenarios, the encouraging contribution of semiotic theory in cases such as the one illustrated in the previous section was the basis it provided for multi-coded information presentation. the information content of the message conveyed by text and graphics presented in figure 3 was the same as that conveyed by a set of filled-out forms (identical to the paper forms used prior to the introduction of qualitas) offered by the system in one of many alternative visualizations of task-related information. once the 124 ajis vol. 8 no. 2 may 2001 system's algorithmic interpretant of tasks is adequately characterized, it becomes the semantic grounding of a number of alternative representamina — visual, textual and even procedural. a remarkable consequence of the emergence of the service summary sign in one of the progressive implementation scenarios was that the company revised its own paper forms in order to include explicitly a field where the employees were required to write a brief narrative of their contract's history. the second instance of progressive implementation scenarios, in the previous section, leads us to yet another valuable semiotic contribution to software design practices. systems are performing messages that can be told in many different codes and many different genres. the unique nature of computer systems is that they allow users to experience virtualities [winograd (1996)] and to gain perspectives that cannot, by any other means, be systematically communicated. an exploration of the relations among co-referential signs across codes and genres shows the designers a wealth of possibilities for intentional intertextuality [kristeva (1980)] that can eventually invest technology with meanings that go untapped. this feature is clearly related to experimenting the alter semiosis, where roles played by different people in a community can be better reified in signs that can be explored from an action perspective. the history of changing scenarios, documentation, practices and systems is equivalent to a semiosic process. along the historical chain in qualitas scenarios, we could see the reifications of interpretants resulting from a continuing abductive process that had led us to the final system implementation. quite importantly, semiotic theory warned us against the stability of semiosis and the definitiveness of meanings for human beings. this caused a major impact on our development team's conception and on the architecture of wits we eventually committed ourselves to using. we realized that the success of qualitas or any similar system can only be achieved through mechanisms that can allow users (or developers in narrow collaboration with users) to customize, adapt and extend wits [cunha et al. (2000)]. it also called out attention to the importance of introspection in software sub-systems, which dourish has referred as the system's accountability [dourish (1997)]. if software can account for its behavior, for instance by generating narratives of its inner workings at an adequate level of abstraction for users to understand, it can productively reveal its algorithmic interpretant. thus, even if users cannot make the system understand their innovative interpretants and use it appropriately, they can always examine the system's unchanging interpretations and try to compensate this inflexibility temporarily with novel ways to use information and do their work. as far as training scenarios are concerned, semiotics pointed to the importance of alternating codes in communication in order to convey complementary views on the same phenomenon. as shown in the examples provided earlier, the team saw that the flow diagram used primarily as pedagogical support should be added to the system's interface as a sign through which users would not only be able to have a rapid access to the various modules of the system, but also and more importantly be able to assess the temporal and causal relations between different stages in the processes covered by that technology. additionally, the diagram could easily be turned into a dynamic visual sign, that would be able to indicate how far or close the users are from task completion, whom their job affects and by whom it is affected, where the process bottlenecks are, and other contextual information. all of the latter are important resources that support opportunistic problem-solving practices that are most encouraged to ensure an organization's competitiveness in the market today. the noticeable point about this is that the emergence of the sign, in static form, and the opportunity to alter its nature to take advantage of underlying algorithmic interpretants by turning it into a dynamic sign, came directly from the ontological underpinnings of design choices made by design team participants. yet another kind of scenarios could be identified in the development process, although the haven't been fully explored to-date, the technological evolution scenarios. however, there are clear indications of what those would be. as was mentioned previously, one of the services provided by our customers is that of coordination. this amounts to organizing meetings and larger events, and acting as a leader in contingent group activities of various sorts. although there are some general norms and procedures relative to these tasks, they are in fact the least understood processes of all, because so much of it is dependent on unique situations that require different skills and styles from coordinators. our customers decided to postpone the specification of this part of qualitas to a future date when they hope there will be more established knowledge about work practices and information use. however, our design team decided to take a more proactive attitude and to propose technological evolution that basically embraces wits and sees coordination services as something will not be decided, planned and reported in qualitas, but will be actually carried out in the system. in other words, whereas educational activities, for instance, are analyzed and planned in qualitas, carried out outside the system (in some external physical setting), and then reported and evaluated back in the system again, coordination activities would be quite extensively achieved in qualitas through an elaborate set of communication and coordination tools. this proposal derives from all the work done in the field ofgroupware and collaboration environments, but there is an interesting semiotic addendum to it. in all other modules of qualitas an interested examiner (an auditor or a new employee, for example) can take qualitas as an active document [stefik (1995, pp.143-145)] that enacts (or performs) the discourse that is told about working practices. the structure of such discourse is partly determined 125 ajis vol. 8 no. 2 may 2001 by certified norms and procedures, partly by technological protocols, and partly by social protocols present in this community. the difficulty to produce discourse about coordination services is that it is fairly unstructured and deeply dependent on every different situation the employee encounters while doing his job. existing norms state that meetings, for instance, must be called by the coordinator, who is also responsible to propose the best date and venue for it, to prepare the meeting's agenda, and to make sure that it is achieved by the participants on the occasion the meeting takes place. whereas this is all true, the norm doesn't say much that somebody doesn't already know about coordinating meetings and, more crucially, it doesn't capture the coordination patterns adopted by the employees. these patterns are important indications of how efficiently the process is being carried out. the technological evolution scenario we are currently working with is one of adding tools that will, by necessity, implement algorithmic interpretants of the processes involved in such activities. by allowing users to obtain an account of these interpretants and to customize a considerable portion of the offered tools, we intend to generate a new technological sign — partial immersion. by this we mean the ability that users will have to progressively eliminate the distinctions between discourse and action. in particular, as teleconferencing facilities are added to wits architectures, once work is done in qualitas a series of sign variations about work (history, cases, patterns, tendencies, and many other indices), both static and dynamic, can be automatically generated. if group decisions can affect the way work is carried out, so that the most efficient practices are always known and considered for a fisrt choice, the whole purpose of norms is superceded by more agile information conveyed by a combination of social and technological organizational discourse. conclusion in this paper, we reported and discussed our experience in implementing qualitas, a web-based workflow system. semiotic theory was extensively used to support design decisions and negotiations with users about technological signs. taking scenarios as a type-sign exchanged throughout the whole process, we could trace the theoretic underpinnings of our experience and draw some revealing conclusions about the product and the process of technologically reified discourse. although it is present in all information technology applications, this kind of discourse is seldom analyzed by software designers and developers. our conjecture is that outside semiotic theory, professionals involved with human-computer interaction and software engineering practices have difficulty to coalesce concepts derived from such different disciplines as psychology, anthropology, linguistics and sociology, just to name a few. semiotics, however, can by itself provide a unifying ontological basis for interdisciplinary knowledge, raising issues and proposing alternatives, that may help professionals gain insights at lower learning costs. we have seen how such semiotic concepts as interpretant, semiosis and abduction were repeatedly invoked to provide the knowledge scaffolding needed to build explanations for observed phenomena and to generate solution to identified problems. the fact that systems can only produce algorithmic interpretations of signs helped us appreciate the extent to which this fixed semiosic process is equivalent to a reification of interpretants. the systematic repetition of interpretations by the system creates an imprint of itself that in time gains a concrete character, that is very different from natural human interpretations (that are seldom the same, even under nearly identical circumstances). this ontological contrast must be faced and dealt with by designers, not reduced to the idealization that if we study users well and long enough we will eventually be able to embed the correct processes and meanings in the technology we are developing. once the reifications are designed and implemented, a productive process of sign variations on a theme can be carried out to explore the inherent communicative qualities of different codes such as the textual, visual and procedural. we saw that in qualitas, demonstrational walkthroughs, natural language narratives and descriptions, filled-out forms, as well as histograms, pie charts and x-y graphs, all amounted to different renditions of a fixed set of meanings programmed in the system. this undeniably helped not only qualitas users, but also the development team and our clients, gain a wide perspective on what we were all doing and/or saying we were doing. in sum, the most important insights supported by the semiotic approach whose echoes we have tried to convey in this paper have to do with collaborative sense-making. this was the key underlying process taking place between us and our clients, between our clients and the employees who were going to be affected by their decision to introduce new technology in the company, between us and these employees, and finally between them and our system. the success of this multi-faceted collaborative sense-making process can be assessed by evaluation experiments of various sorts. the results of evaluations can be contrasted to every semiotically-motivated design choice and tell much about how well the application is communicating shared knowledge by means of the signs present in its interface. 126 ajis vol. 8 no. 2 may 2001 acknowledgments the authors would like to thank cnpq and tecgraf for the financial support that allowed this work to be developed. they are very greatful to all participants of the development team who have always engaged themselves in discussing relevant research issues associated to this project and contributed with valuable insights. references adler, ps. & winograd, t.a. (1992) usability: turning technologies into tools. oxford university press. new york, ny. brown, g & yule, g (1983) discourse analysis. cambridge university press. cambridge, uk. carroll, j.m. (2000) making use: scenario-based design of human-computer interactions. the mit press. cambridge, ma. cunha, c.k., de souza, c.s., quental, v. & schwabe, d. (2000) "a model for extensible web-based information-intensive task-oriented systems". in proceedings of hcp2000. sunderland, uk. de souza, c.s. (1993) "the semiotic engineering of user interface languages". international journal of man-machine studies. no. 39. pp. 753-773. de souza, c.s., barbosa, s.d.j. & da silva, s.r.p. (to appear) "semiotic engineering principles for evaluating end-user environments". to appear in interacting with computers. de souza, c.s., prates, r.o. & carey, t. (2000) "missing and declining affordances: are these appropriate concepts?". in proceedings of the iii workshop on human factors in computer systems, ihc'2000. gramado, rs, brazil. dourish, p., holmes, j., maclean, a., marqvardsen, p., zbyslaw, a. "freeflow: mediating between representation and action in workflow systems". in proceedings of computer supported cooperative work '96. cambridge ma usa. pp. 190-198. dourish, p. (1997) "accounting for system behavior: representation, reflection, and resourceful action". in kyng, m. & mathiassen, l. (eds.) computers and design in context. the mit press. cambridge, ma. pp.145-170. eco, u. (1979) a theory of semiotics. indiana university press. bloomington, in. frege, g (1892) "uber sinn und bedeutung" zeitschrift fur philosophic und philosoophische kritik 100, 2550. glance, n. s. and pagani, d. s. and pareschi, r. (1996) "generalized process structure grammars (gpsg) for flexible representations of work". in proceedings of computer supported cooperative work'96 (cscw'96); pp. 180-189. heinl, p., horn, s., jablonski, s., neeb, j., stein, k. and teschke, m. (1999) "a comprehensive approach to flexibility in workflow management systems" in proceedings of the international joint conference on work activities coordination and collaboration wacc'99 pp. 79-88. imaz, m. & benyon, d. (1999) "how stories capture interactions". in proceedings of human-computer interaction interact'99. ifip. josephson, j.r. and josephson, s.g (eds., 1994) abductive inference: computation, philosophy, technology. cambridge university press, cambridge. khoshafian, s. & buckiewicz, m. (1995) introduction to groupware, workflow, and workgroup computing. john wiley & sons. new york, ny. kristeva, j. (1980) "word, dialogue, and novel" desire and language. rudiez, l. (ed). new york. columbia university press. pp.64-91. malone, t. w. and lai, k. and fry, c. (1995) "experiments with oval: a radically tailorable tool for cooperative work". in acm transactions on information systems; vol. 13; no. 2; april 1995; pp. 177205 monteiro, c. de c., barbosa, s.d.j. & de souza, c.s. (2000) "the role of designer-generated scenarios in developing web applications: a case study". in proceedings of the hi workshop on human factors in computer systems, ihc'2000. gramado, rs, brazil. nake, f. & gabrowski, s. (to appear) "interaction is pseudo-communication". newell, a. (1982) "the knowledge level". artificial intelligence, vol. 18, 1. pp.87-127. peirce, c.s. (1931-5) philosophical writings of peirce. selected and edited by buchler, j. dover publications, 1955. new york, ny. prates, r.o., de souza, c.s. & barbosa, s.d.j. (2000) "a method for evaluating the communicability of user interfaces" acm interactions. jan-feb 2000. pp.31-38. 127 ajis vol. 8 no. 2 may 2001 stefik, m. (1995) introduction to knowledge systems. morgan kaufinann. san francisco, ca. suchman, l. (1995) "representations of work". guest editor in communications of the acm. vol. 38, no. 9. pp. 33-35. winograd, t. (1996) bringing design to software. acm press. new york, ny. 128 microsoft word special final.doc ajis special issue december 2004 21 interactive spaces towards collaborative structuring and ubiquitous presentation in domestic environments kaj grønbæk and marianne graves petersen center for interactive spaces department of computer science åbogade 34, dk-8200 århus n kgronbak, mgraves@interactivespaces.net abstract this paper analyses the use of media and material in private homes based on empirical studies in a project on designing interactive domestic environments. based on the analyses we propose a domestic hypermedia infrastructure (dohm) combining spatial, context-aware and physical hypermedia to support collaborative structuring and ubiquitous presentation of materials in private homes. with dohm we propose establishing new relationship between digital and physical hyperspaces, folding hyperspaces into the physical space of the household. thus we strive to combine the qualities of physical domestic materials and spaces with the flexibility and dynamics of digital hyperspaces. we propose a variety of new ubiquitous home appliances called mediawall, mediatable, mediatray and mediaport, which address these issues. keywords interactive spaces, augmented reality, context awareness, ubiquitous hypermedia, domestic technology, multimedia, physical hypermedia, interactive spaces designing interactive spaces is an effort which combines the competencies of architects and computer scientists. in center for interactive spaces, we take activities as the common starting point when we co-design space and technology in an iterative design process, where neither defines the other nor is taken for granted but both dimensions are manipulated and synthesized into innovative design concepts. designing interactive technology in this perspective has a number of implications. first, where much focus in the field of domestic technologies has been on exploiting technological possibilities to support awareness and communication amongst people who are not co-located (hutchinson et al. 2003, mynat et al. 2003), a focus on interactivespaces as described above implies support for interaction amongst people, who are co-located. in a domestic context, this aspect becomes crucial, as we currently see an increased digitization of domestic material, e.g. photos, movies, calendars, recipes, notes, banking, messages from school etc. increasingly become digitized and thus no longer have an inherent physical form. digital materials often reside on a personal computer, which provides single user access to materials rather than engaging multiuser experiences which are highlighted in an interactivespaces perspective. moreover, when studying how physical domestic materials are organised in homes, the actual physical layout of the home provides an implicit structuring mechanism for these materials. thus, a perspective of interactivespaces implies carefully considering the relationship between digital and physical spaces. in this work, we adopt a folding strategy where the digital spaces is folded into the physical space seeking to maintain the qualities of both types of spaces addressing a domestic space moreover has a number of consequences. historically, the workplace has undergone a similar transition of digitization as we currently witness in homes. however, the home is quite different from the workplace in many respects. activities in the home are less task-oriented (monk 2000); the rationalities of work in terms of production, efficiency and organization of labour do not necessarily transfer to the home (crabtree et al. 2002). moreover inhabitants continuously re-configure and appropriate their homes both to ajis special issue december 2004 22 express their identity to the outside world (palasmaa 1994), but equally to capture their own history and biography (silverstone et al. 1992). thus homes have their own aesthetics where the visible, physical “information material” often play a role in expressing identity e.g. in terms of the books being read or the music listened to by the inhabitants. these qualities of the domestic environment are important to understand when designing domestic interactive spaces. thus in order to design future domestic interactive environments, we argue that it is necessary to study the characteristics of the existing use of both physical and digital materials in homes today. while we are not the only one to take this point of stance, our focus is different from other studies. e.g. some groups have focused on understanding patterns of domestic routines (crabtree et al 2002a, 2002b, tolmie et al. 2002) where we have focused more broadly on eliciting the different roles domestic information materials may have, not restricted to patterns of routine. others have investigated what constitutes the home experience (eggen et al. 2003) more generally, but this study did not reveal how specific materials formed part of constituting the home experience. our studies are inspired by the studies by (crabtree et al. 2002a,2002b), who investigate how the physical space of the home is used to coordinate the handling of paper mail, but our focus is on materials more broadly than paper mail. compared to the jigsaw domestic component system proposed by humble et al (2003), we take a material centered approach. we have focused on the organization of domestic material and on how we can provide a seamless folding between the physical and digital material spaces. where humble et al.(ibid) focus on supporting transformations between digital and physical material, we focus on linking and integrating the physical and the digital. the paper is structured as follows: section 2 describes a qualitative approach to the study of 6 different types of households and section 3 outlines the challenges identified in this study for the design of domestic interactive spaces. section 4 introduces a domestic hypermedia infrastructure (dohm) which seeks to meet the challenges identified in section 3. section 5 gives examples of appliances, which fold hyperspace into the physical home environment. section 6 discusses related work. section 7 concludes the paper. empirical studies we have undertaken qualitative empirical studies in six private homes inhabited by different types of families, ranging from singles, over families with children living at home, to communes. we visited each household once and stayed for approximately one and a half to two hours each place. in each home, we asked the inhabitants to take us on tours around the house (petersen and baillie 2001). we focused on how people organize domestic information materials in general, i.e. media, letters, memo lists, newspapers etc., and we interviewed people about the rationale, and history of the physical placements of both physical and digital materials. we asked about ownership and possible conflicts around placement of materials (mynatt et al. 2003). we captured data from the homes through video recordings, and pictures. the six visits form part of a continued investigation into domestic technology use of our own (petersen et al. 2002, kjær et al. 2000) as well as others (crabtree et al. 2002b, mynatt et al. 2003). information material in homes taking tours in private homes reveals a wealth of information materials with hypermedia qualities, which are literally distributed all over the house. table 1 holds a non exhaustive list of domestic materials, which are grouped into categories with different characteristics. the purpose of this list is to illustrate the richness and heterogeneity of domestic materials, rather than providing an exhaustive and complete list. ajis special issue december 2004 23 types of material example material characteristics media movies, music, pictures, books, news, magazines, games, news, computer applications carefully chosen, bought and stored used for leisure purposes home administration tools calendars, memo lists, tickets, contact information, notes for other family members, letters from school, kindergarten etc ongoing administration used as tools for coordination domestic task recipes, manuals, handbooks support domestic tasks manifestations of social life invitations, greetings, personal letters, jokes maintaining social relations capturing history used as decoration communication with official authorities letters from bank, bills, information push often little affection towards material decoration materials pictures, souvenirs, seasonal objects capturing history expressing identity work materials electronic mail, word documents etc. support work tasks table 1: the heterogeneity of domestic materials and their use we do not wish to claim that the categories below are rigid and disjunctive, e.g. a calendar is also a manifestation of social life. the purpose is rather to illustrate that home contain a complex collection of various heterogeneous materials of very different nature. materials which are kept for different purposes, e.g. calendars, messages from school, shopping lists support the practical coordination of home life whereas various media are consumed and used for leisure purposes. media are also most often actively selected, bought and carefully stored, whereas letters of communication with official authorities are pushed regularly upon the household members. as suggested in the following, these different forms of information materials are also often treated quite differently. one tendency, which we do observe across the different types of materials, however, is an increased digitization of domestic materials. this happens with e.g. music (premkumar 2003), photos, and letters from bank. this has both negative as well as positive side-effects. the mission of this paper is to learn how to take the best from both sides and combine into visions on concepts for future domestic hypermedia systems. challenges to domestic hypermedia interfaces in the following, we illustrate how the context of the home provides new challenges to hypermedia structuring mechanism. while these challenges are specific to the domestic sphere, we suggest that they are fruitful provocations for new conceptions of hypermedia more generally, and thus are also worth investigating in other contexts. linking digital and physical material the homes contain a number of implicit links between digital and physical material that are currently not well supported. taking the case of digital photos we saw an instance where digital photos from a wedding had been developed onto paper in order to distribute them to others, and to make a physical collection of the pictures. in this case, the co-existence of the digital and physical form is important, however, there is no direct link between the physical and digital versions of the picture. e.g. there is ajis special issue december 2004 24 no easy way of tracing the digital version of a physical photo. the only means of creating a link is through annotating the physical and digital versions by hand. we saw no instances where this had happened in the homes we visited. we saw an example of this, however, with photos developed from negatives. as can be seen in figure 2, time-stamps on post-its with regular intervals in both collections provided means for linking the composites together. thus our studies suggest a need for linking support between digital and physical material, allowing for the co-existing of forms, rather than supporting transformation between forms as suggested by e.g. humble et al. (2003).. figure 1: left: unlinked digital photos and printed versions. right: recipe printed from the web with printed url an additional example is the case of recipes printed from the web, right picture in figure 1. here we find a one-directional link from the printed version to the digital material. again we see how the digital and physical versions co-exist in homes as the physical version is more robust in the context of the kitchen, than the mobile computer of the household. however, both when it comes to physical pictures and physical recipes there are problems with searching through the material for specific contents as this relies on the inhabitant to produce metadata or in other ways impose structure on the physical material. one of the people we visited explained that it was much easier to re-print the recipe from the web rather than trying to find the printed version in the unstructured collection of recipes. further, there are a lot of implicit links between physical materials in the home. figure 2: left and middle: postits with dates provides link between composites of physical photos and corresponding negatives right: calendar and notice board in kitchen e.g. figure 2, right shows an example where the picture on the notice board of the wedded couple is implicitly linked to a date on the nearby calendar. the same holds true for the football match schedule on the notice board. while both calendar and football schedule may well be digitized and linked together in terms of dates, this would miss the important quality of notification through the persistent visibility, which characterizes physical papers on a notice board in a busy place of the ajis special issue december 2004 25 home. thus as discussed in the following, key to designing successful hypermedia support is to understanding the roles of different domestic surfaces. understanding domestic surfaces crabtree et al.(2002a) were among the first to spell out in details how the specific surfaces in the home designate certain meaning and serve to coordinate domestic life. in their study they focused on paper mail (ibid), and described how if for instance a letter is found by the porch, it suggests that no other person has dealt with it yet. whereas if it is placed at the doorstep of the teenagers’ door, it signals that it has passed some other household member, e.g. a parent, who may have recognized a letter from the phone-provider and expects the teenager to deal with it. confirming the studies of crabtree et al., but focusing on domestic information materials more broadly than paper mail, our studies also suggest that the spatial layout of the home is indeed used to coordinate and structure domestic information materials, and thus is a resource in designing domestic hypermedia.to indicate how different surfaces of the home have different meaning, some example characteristics are listed in table 2 and examples from the homes are presented in figure 3 and figure 4. surface example material surface characteristics notice boards emergency phone number prescriptions pictures from a wedding appointment with dentist information which is ‘one click away’ contact information notifications memorable material table brochures from a recent visit in a theme park mixed pile of childrens’ book, adults’ book, and unused wrapping paper unfinished shopping list, commercials traces of earlier activities piles awaiting further distribution in the home mixed media activity spaces entrance hall a book which need to be returned to the shop a hat which must be worn when leaving the home things to remember to bring along when leaving the home shelves collection of selected cookbooks, music cds, computer games. illegal games are kept in a closed cupboard. only cool cookbooks are on display in the common-room. less cool books are stored in a room where visitors do not normally come media collections. not necessarily sorted optimally for searching or browsing but rather in terms of aesthetics, and in terms of what people wish to show to others. frames picture collages, posters, etc.. staging of personal memory table 2: examples of different surfaces and their characteristics ajis special issue december 2004 26 figure 3: left: dining table with brochure from a theme park recently visited. right: mixed pile of children’s’ book, adults’ book, and unused wrapping paper awaiting further distribution in the home thus as the list suggest, we also see how different places in the house have different meanings and are used to structure domestic information material. we also see how other interest than supporting browsing and searching come into play when placing materials in homes. e.g. as illustrated in figure 5, only selected cookbooks are placed so they are immediately visible to visitors whereas the rest of the collection is kept in a place where visitors not normally come. this is an example of what palasmaa has described as “home is a staging of personal memory. it functions as a two-way mediator personal space expresses the personality to the outside world, but, equally important, it strengthens the dweller's self-image and concretizes his world order”(palasmaa 1994, p. 6). as people behave this way, and their home become meaningful in this way, it is important to ensure that this is also supported in future domestic environments where domestic materials are digitized. figure 4: left: table in entrance hall with a pile of things to remember when leaving the house. figure 5: midlle: shelves in kitchen where selected cookbooks are on display. right shelves in home office, holding the other part of the cookbook collection. the highly spatial distribution of physical materials onto a multitude of surfaces in homes stands in market contrast to what happens, as domestic materials become digitized and stored on a personal computer in the home. the pc essentially provides centralized and individualized access to digital materials. it lacks the spatial distribution, the persistence and visibility of physical materials on display in the home, as we see numerous examples of in our empirical material. thus we need to learn from the qualities of physical material, which enable spatial information structuring in the home when designing domestic interactive spaces. ajis special issue december 2004 27 lazy structuring and sustainability of structures as evident from the pictures in figure 6, which are taken in the same home, the time invested in structuring material reflects relationship to material. on the left picture is a mix of different materials including bills and papers from the bank. right we see a collection of carefully collected and indexed cartoons. this leads to concerns on the sustainability of structures in everyday life of the home as some structuring mechanisms impose more effort on the user than others. an example of this is a case, where a person started to annotate individual digital photos with names of persons and places on pictures making this content directly searchable. however, the person who had explored this facility stopped after having annotated only a fraction of his collection, simply due to the conflict between the size of his collection and the time and effort in annotating individual pictures. figure 6: pictures from the same home. letters from the bank etc. versus favorite cartoons an additional example of breakdown of structuring mechanisms is the problem of scalability. as when for instance the collection of cartoons in left picture of figure 7 becomes too big to be browsed through in terms of author. in right picture of figure 7 we see an example where the collection is being re-structured. here we see three different classifications on the same material at the same time. one part is placed in labeled boxes in order to provide unique location identification for each magazine. the intention is to exploit peer-to-peer digital metadata on the magazines to provide computer supported searching and browsing. figure 7: different classifications on the same material – in this case cartoons another part is still ordered alphabetically by author. finally a third part consists of the most recent issues. they are piled waiting for the owner to find time and mood to place them in the new categorization. this is in no ways a unique case. in the homes we saw numerous examples of half finished structures. this calls for an open hypermedia approach to domestic hypermedia, and for supporting lazy structuring of domestic materials. an example of a half finished structures and lazy structure of the physical material in the home is the widespread use of piles as a structuring mechanism. all the homes we visited had piles. piles of bank papers and official papers waiting to be filed in binders. piles of books on the dining table, waiting to be put in place. thus piles are not as unstructured as it may first seem thanks to the meaning of different surfaces of places of the home, as ajis special issue december 2004 28 discussed above. some are piles of heterogeneous material, implicitly ordered by date, as sedimentations of materials. not an effective structuring mechanism for a fast search, but a mechanism, which with the least possible effort still put some structure on the material largely due to the specific location of the pile. the pile in the entrance hall in figure 4 is not sediment. it is carefully selected stuff which must be brought when leaving the home etc. thus the specific context of the pile provides an additional layer of meaning to the structure. a dimension which is lost, if most domestic material can only be distributed on one, digital desktop. figure 8: piles as structuring mechanism in homes the structuring experience while one way to be pragmatic with respect to everyday life is to support lazy structuring mechanisms, an alternative and complementary approach is to design for playful and engaging structuring experiences thus aspiring people to take time to impose structure on their materials. the structuring experience provided by traditional pc’s most often consists of individual manipulation of pull-down menus and dialogue boxes. given the nature of the home, it would be interesting to investigate how collective and more playful structuring experiences can be supported. domestic hypermedia – folding hyperspaces into the household the empirical studies showed a lot of different ways of organizing physical material in the home. however, collaborative spatial organization and persistent visual awareness of the material seemed to be common means for the handling of domestic materials. in this section we propose a hypermedia concept that supports these approaches to structuring digital domestic material. we propose a hypermedia concept where digital hyperspaces are folded into the physical space. the notion of folding is inspired by the work on folding within architecture. in architectural foldings the idea is to create more dynamic and open buildings by folding rooms of one type in between rooms of another type, e.g. public spaces are folded into more private spaces and vice versa. as described by lynn (1994) folding employs neither agitation nor evisceration but a supple layering. in such a way, that the characteristics of the individual layers are maintained. “a folded mixture is neither homogeneous, like whipped cream, nor fragmented like chopped nuts, but smooth and heterogeneous” (ibid p.9). ajis special issue december 2004 29 figure 9: folding physical and digital spatiality figure 10: dohm domestic hypermedia infrastructure we propose domestic hypermedia to fold digital spaces into physical spaces and vice versa in a way where the characteristics of both spaces are preserved, yet they are connected through layering. thus we envision a multitude of digitally enabled surfaces in the future home. each surface resides in a particular physical context, in the digital environment. the physical surfaces are represented by placereps (place representations), which can be instantiated graphically or by a name, as depicted in figure 12. the folding consists in the parallel continuous presence of an abstract physical home representation in the digital hyperspace, and the physical representation of the domestic hyperspace through the interactive surfaces. domestic hypermedia infrastructure the domestic hypermedia infrastructure is inspired by spatial (marshall and shipman 1997), open (grønbæk and trigg 1999), and physical hypermedia (grønbæk et al. 2003) approaches. research in spatial organization of digital information is of particular relevance to the domestic domain, given the importance of the spatial organisation of domestic materials. open hypermedia (grønbæk and trigg 1999) is another source of inspiration since the home similar to many work domains are characterized by being populated with a lot of different media which are not well suited for embedded structures. in particular we see a need for the ubiquitous open hypermedia systems, that deals with context-awareness (e.g. hycon (bouvin et al. 2003)) and augmentation of the physical world (e.g. the topos physical hypermedia (grønbæk et al. 2003). finally, the physical hypermedia approach may be tailored to the home environment in order to maintain relationships between physical material like souvenirs and digital photos, equipment and manuals, cd-covers and mp3s etc. dohm is in particular designed to support folding of hyperspaces into the physical space. the placere p home server web server web server mediatray place registrar dohm client mediagate hycon server place reg. service … storage service apps “surface” mediag ate service notificati on service portfolio service collage service portfolio … … rfid physical digitalphysical digital ajis special issue december 2004 30 empirical studies show that spatial organization and piling of material are very common structuring mechanism in homes as well as lazy or incomplete structuring is observed in many places. thus dohm in particular supports spatial hypermedia structuring integrating the connected physical surfaces. the dohm infrastructure also supports carrying digital material around both inside and outside the home by means of having a physical token (like a cell phone or a smartcard) representing a persons digital portfolio on the home server and other integrated servers. the dohm infrastructure is thus capable of handling the common structures that are used in homes, e.g. composites representing play lists, photo albums, and slideshows. meta-data associated with physical objects or digital material like photos are represented in rdf. a central user interface for dohm is the dohm client, which provides a 2d spatial hypermedia interface including the registered surfaces as objects. modelling home surfaces – presentation specifications and styles the various surfaces in the private home needs to be addressable from the point of view of the hypermedia system, such that particular material can be presented in different (or even multiple) physical locations in the home. as discussed in above, surfaces in the private home serve many different purposes and exist in different contexts. e.g. things to remember when leaving the home are places persistently and visibly in the hallway. whereas peripheral awareness of earlier activities is created through the traces of material left on dining table. this is implemented in the hypermedia model dohm through adding two central concepts to the hycon framework in order to handle representation and presentation. placereps represent the place or context of a particular domestic hypermedia surface in the home; and presstyles represent the style specification for a given object or surface. placereps may represent both a specific place like “the refrigerator door“ or an abstract place like “entrance” which may represent several physical locations in the home including, both hall, basement and terrace. thus in this way, information that must be recalled when leaving the home can be displayed in all the physical entrances. we wish to give the users maximum freedom in deciding how material in their home should be presented, to support the continuous appropriation of the home (silverstone et al. 1992). we use presentation styles to represent for the specific presentation style for a material object or a surface object. in light of the different nature of the different materials and surfaces i.e. some material on some surfaces should be able call for attention in some context, e.g. the appointment with the dentist on the notice board, which must be remembered, whereas others need to be discrete and aging, e.g. pictures from a recent weekend trip with friends. thus we wish to support the users in specifying different presentation styles for material and surfaces in the home. for instance, a teenage kid may prefer to have a certain skin presstyle associated with the surfaces reminding her of a favorite movie on every surface in the room. on the other hand, parents may want to associate an intrusive red colored notification presstyle with a reminder note that they place on every relevant surface in the home. in this case there should be a combination strategy that allows both the teenager to maintain her surface skin and to be notified about the important note from the parents. this is solved by a calculation of the cascade based on the cascade priority attributes of the involved presstyles. we currently explore different presentation styles, including persistent, aging, collage, highlight, surprise, animated, context-sensing. the presentation styles will be subject to further experimentation and evaluation in an iterative design process. supporting collaborative structuring and ubiquitous presentation in this section we describe some design concepts which are build upon the dohm infrastructure and which addresses the need for collaborative structuring and ubiquitous presentation identified in the empirical studies. ajis special issue december 2004 31 mediagates for piles of unsorted incoming material – supporting lazy structuring the dohm system provides client support for uploading of digital material and scanning of physical material to the home server. the gate hardware will typically be scanners (for paper), flashram or usb ports (for pictures and documents), firewire ports (for video). but email, sms/mms messages may also be transferred to gate space. the material being uploaded will be dropped onto a “gate” folder and canvas acting as temporary space for unsorted incoming material. this gate canvas can be associated with places in the home, where the family wish to create awareness of recent un-processed material. one such place may be the dining table which in homes is often used as a gate area for physical material in transition into other places in the home, e.g. see figure 3. in this way we may support the lazy structuring witnessed in homes, as seen in figure 7 and figure 8. mediatable: augmented collaborative dining table – supporting new structuring experiences in many families, the dining table is a central place for coordinating activities, exchanging messages, and for organizing incoming materials. in the studies we have seen how families leave physical material on the dining table or the common room table creating a shared awareness in the home, before it is put away on a shelf or pinned to a notice board. when physical material becomes digitized it is often uploaded to or received on a computer somewhere in the home by one family member without reaching the attention of the rest of the family. in a future home with many digital materials we wish to be able to support collective coordination and organization of materials similar to the physical case. we are thus designing the dohm client to support e.g. a dining table to be augmented with top projection and direct tangible interaction. the dohm client present itself to the user as a spatial hypermedia interface, implemented in svg (scalable vector graphics) thus allowing elements to be turned around to be visible from arbitrary positions around the table, see figure 12. the idea is that a common room table becomes a digital surface providing touch based interaction. the default view is the scratch area with incoming material and visual icons for the registered placerep objects. in this way, family members become aware of new materials like incoming messages, new photos, new mp3s etc. family members passing by can for instance drag a piece of material to a specific placerep icon to make the material visible one or more surfaces in the home shown with the default presstyle for these surfaces. in a situation where the family gather in the living room they may sit down or stand around the table and sort the piles of recent pictures into folders on the home server and drag some of them to specific placereps to have them rendered on specific surfaces. they may also associate specific digital material with physical objects by means of rfid tags as described in (grønbæk and trigg 1999). the objects may be printouts of pictures or souvenirs like a concert ticket or a kids’ doll, which then becomes a physical link anchor for a picture or a collection of pictures. thus the table also aim to support collective organization of material thus supporting new types of structuring experiences as suggested from the empirical investigations. ajis special issue december 2004 32 figure 11: a scenario with two people interacting around a mediatable figure 12: the dohm client as it looks when supporting collaborative picture sorting. the icons on the left are placerep icons for surfaces in the home. the floating toolbars contain tools for interaction with objects. mediatrays for physical link following the dohm client may fold out into the physical world as a display less mediatray (see figure 13), which can be used both as a normal tray and as an interface for activating links by means of rfid tagged physical material. in the living room environment, a mediatray placed on the sofa table may be used to start the playback of music on a connected hifi set based on a physical object which has been associated the music. it may also be used to invoke a picture slide show on the tv based on a single printed picture being placed in the tray. in a kitchen environment a special mediatray may be used to retrieve grocery declarations and recipes based on a piece of grocery material being put on the tray. the mediatray is a display less hypermedia interface; it needs to integrate seamlessly with the environment without having to be configured via menus and dialogs. thus the mediatray needs to take advantage of its context such as the nearest hifi set, tv set or other surface. if the tray is physically moveable, it just needs to be registered to a new location, and then it automatically takes advantage of the new context, similar to the physical hypermedia system for architects discussed by grønbæk et al (1999). figure 13: mediatray for physical hypermedia interaction. the right picture shows the rfid tagreader in the bottom. ajis special issue december 2004 33 mediawall: calm awareness support in order to exploit the potential of surfaces in the home, we have developed the concept of the mediawall. messages and notes often contain contents to be remembered and to be used at certain points in time. thus some sort of notification service is needed, as when the appointment with the dentist must be remembered. however, few people wish to have there home filled with notifications in terms of active alarm clocks. in contrast, the mediawall offers the quality of ubiquitous and persistently visible message. maintaining awareness about important material or critical events is supported in the dohm system. here we will focus on the discrete or calm (weiser and brown 1996) awareness based on discrete visual appearance. we are inspired by the notions of informative arts (holmquist and skog 2003, redström et al. 2000) as well as infocanvas (millar and stasko 2001). the idea is to provide awareness about important material through an artistic collage of the material on one or more surfaces. to support this we are taking advantage of the open hypermedia techniques for anchoring links in arbitrary web pages. here we let family members make selections on arbitrary web pages including the home server, submitting the corresponding anchors to a collage service together with a placerep with a ‘collage’ presstyle. the surface identified by the placerep will then display the selected text and graphics corresponding to the anchor according to some schema with an expression designed to fit the actual room and the inhabitant’s preferred level of calmness. the collage service is meant to maintain an ongoing awareness of the updated view of information identified by the submitted anchors, thus an active crawler is need to regularly collect information from servers, filtering the parts of the pages needed, making an artistic transformation before the representation is dispatched to the surfaces pointed out by the placereps that were associated with the anchor when submitted to collage service. conclusions the mediagate, the mediatable, the mediatray and the mediawall together presents a vision of a domestic hypermedia system, which takes into account the specific challenges of the household. the mediagate address the challenge of lazy structuring, the mediatable offers new structuring experiences, the mediatray provide new ways of linking digital and physical materials, and the mediawall takes advantage of the meaning and richness of the surfaces of the home. in this paper we have demonstrated how we apply the perspective of interactive spaces to maintain a focus on supporting interaction among people who are co-located. we have presented an empirical study of the use of primarily physical material in private homes and we have illustrated how taking the context of the home seriously implies certain challenges for domestic hypermedia. designing for the pragmatics of real domestic life implies for instance supporting lazy structuring of domestic materials. it implies using the rich set of surfaces of the home as a resource in design, and it implies exploring how collective and more playful structuring mechanisms can be developed. based on the challenges revealed in these studies we have described the design of a new domestic hypermedia infrastructure called dohm, which supports the folding of spatial and navigational hypermedia into the physical environment of a home. we have presented a couple of novel home appliances that take advantage of the dohm infrastructure and fold the hyperspace into the living room. we have established the basic dohm infrastructure and are initiating the initial experiments with users introducing the appliances being designed. we have developed the first lab prototypes of the appliances and the dohm infrastructure, and they will be evaluated both in our lab and in specific home settings within the coming months. ajis special issue december 2004 34 references bouvin, n.o., christensen, b.g., grønbæk, k., hansen, f.a. hycon: a framework for contextaware mobile hypermedia. 2003. to appear in nrhm journal volume 9. crabtree, a., hemmings, t. and rodden, t. (2002a) “coordinate displays in the home”, cscw workshop on public, community and situated displays, proceedings of the 2002 acm conference on computer supported cooperative work, new orleans: acm press. crabtree, a., hemmings, t., and rodden, t. (2002b) patternbased support for interactive design in domestic settings. in proc. of dis2002, acm press, pp. 265-275. eggen, b., rozendaal, m. and schimmel, o. (2003). home radio extending the home experience beyond the boundaries of the physical house. in proceedings of the home oriented informatics and telematics,university of california, irvine, april 6-8, 2003. grønbæk, k. & trigg, r.h. from web to workplace: designing open hypermedia systems. mit press, boston massechussets. july 1999, 424 pp (isbn 0-262-07191-6). grønbæk, k., ørbæk, p., kristensen, j.f., and eriksen, m.a. physical hypermedia: augmenting physical material with hypermedia structures. to appear in new review of hypermedia and multimedia (nrhm). vol 9. 2003. holmquist, l. e., and skog, t. (2003) informative art: information visualization in everyday environments. in proc. of the first intl. conference in computer graphics and interaction techniques in australia and southeast asia. humble, j., hemmings, t., crabtree, a., koleva, b. and rodden, t. (2003) “‘playing with your bits’: user-composition of ubiquitous domestic environments”, proceedings of the 5th international conference on ubiquitous computing, pp. 256-263, seattle: springer. hutchinson, h., mackay, w., westerlund, b., bederson, b. b., druin, a.,plaisant, c., beadoinlafon, m., conversy, s., evans, h., hansen, h., roussel, n., eiderbäck, b., lindquist, s., and sundblad, y. (2003) technology probes: inspiring design for and with families. in proceedings of chi 2003. acm press, pp. kjær, a., madsen, k. h. & petersen, m. g. (2000) methodological challenges. in the study of technology use at home. in home informatics and telematics: information, technology and society, sloane, a. and van rijn, f. (eds.) boston: klüwer academic publ., pp.45-60 lynn, g. (1994) architectural curvilinearity : the folded, the pliant and the supple. in folding in architecture. architectural design magazine. academy grp ltd., pp. 8-15. marshall, c., & shipman, f. spatial hypertext and the practice of information triage, in proc. tenth acm conference on hypertext (hypertext '97). (southampton, uk, apr, 1997), pp. 124133. millar, t., stasko, j. (2001) the infocanvas: information conveyance through personalized, expressive art. in proceedings of chi 2001, acm press, pp. 305-306. monk, a. (2000) user-centred design. the home use challenge. in sloane, a. & van rijn, f. home informatics and telematics. information, technology and society. kluwer academic publishers, pp. 181-190. mynatt, e., rowan, j., jacobs, a., and craighill, s. (2003) digital family portraits: supporting peace of mind for extended family members. in proceedings of chi 2001, acm press, pp. 333-340. pallasmaa, j. (1994) identity, intimacy and domicile. notes on the phenomenology of home. in finnish architectural review 1 / 1994. petersen, m. g., madsen, k. h. and kjær, a. (2002) usability of everyday technology – emerging and fading opportunities. in acm transactions on computer-human interaction, vol. 9, no. 2, june 2002, pp. 74-105. ajis special issue december 2004 35 petersen, m. g. and baillie, l. (2001) methodologies for designing future household technologies. in proceedings of the oikos2001 workshop, aarhus university press, pp. 47-49 premkumar, g. p. (2003) alternate distribution strategies for digital music. in communications of the acm, september, volume 46, issue 9. redström j, skog t & hallnäs l. informative art: using amplified artworks as information displays. in proceedings of dare 2000, designing augmented reality environments, acm, 2000, elsinore, denmark silverstone, r., hisch, e. & morley, d. (1992) information and communication technologies and the moral economy of the household. in consuming technologies. media and information in domestic spaces. routledge. london and new york. pp. 15-31. tolmie, p., pycock, diggins, t., maclean, a., and karsenty, a. (2001) unremarkable computing. in proceedings of chi2002, acm press, pp. 399-406. weiser, m. & brown, j.s. (july 1996) "designing calm technology", powergrid journal, v 1.01. copyright [author/s names] © 2004. the authors assign to ozchi and educational and non-profit institutions a non-exclusive licence to use this document for personal use and in courses of instruction provided that the article is used in full and this copyright statement is reproduced. the authors also grant a non-exclusive licence to ozchi to publish this document in full in the conference papers and proceedings. those documents may be published on the world wide web, cd-rom, in printed form, and on mirror sites on the world wide web. any other usage is prohibited without the express permission of the authors. microsoft word 13 1 marina.doc ajis vol 13, no. 1 september 2005 225 requirements engineering and software project success: an industrial survey in australia and the u.s. june verner empirical software engineering national ict australia australiantechnology park june.verner@nicta.com.au karl cox empirical software engineering national ict australia australiantechnology park karl.cox@nicta.com.au steven bleistein computer science and engineering, university of new south wales & national ict australia steven.bleistein@nicta.com.au narciso cerpa departamento de ingeniería de sistemas, universidad de talca, camino los niches km. 1, curicó, chile ncerpa@utalca.cl abstract because requirements engineering is recognized as critical to successful software projects we surveyed a number of software practitioners regarding their software development practices during recent software projects. relationships between requirements practices and software project outcomes enable us to better understand requirements issues and their relationship with project success. we asked three sets of questions directly related to requirements issues: 1) requirements practices, 2) the sponsor and customers/users, and 3) project management. our respondents were from business organizations in the u.s. and australia, and were almost exclusively involved in in-house software development. the most significant factors from each question set were: 1) the requirements were good, 2) there was a high level of customer/user involvement, and 3) the requirements were managed effectively. overall, the best predictor of project success was that the requirements were good together with the requirements were managed effectively (93% of projects were predicted correctly). our survey shows that effective project management is fundamental to effective requirements engineering. introduction requirements engineering (re) can be simply described as identifying a problem’s context, locating the customer’s requirements within that context and delivering a specification that meets customer needs within that context. there are many requirements methodologies that purport to do this, for example, soft systems methodology (checkland 1981), scenario analysis (carroll 1995), and uml (booch et al 1999). sometimes they work, sometimes they do not. the implication of such requirements methodologies, if we can label at least aspects of them as such, is that the application of ‘x’ method will produce the right requirements irrespective of the problem’s characteristics. this is conventional wisdom and unsurprisingly, the creators and vendors of requirements methodologies claim, with one exception (jackson 2001) that their approach is a hammer and all problems are nails. while there are many factors other than just application of a requirements methodology that influence the success or failure of software projects in practice, in this paper we focus only on requirements engineering. as davis and hickey state, we need as researchers, to be ajis vol 13, no. 1 september 2005 226 aware of what is really going on in practice to be able to position our research within that context (davis & hickey 2002). without this, we will forever practice our art in a contextfree bubble. to document practitioners’ views regarding software project success or failure and the practices they consider important to successful projects, we conducted wide ranging structured discussions with twenty-one senior software practitioners employed by a large u. s. financial organization. this discussion focused on software projects with which the practitioners had been recently involved. we later had similar discussions with another group of software practitioners from the u.s who worked in a variety of other commercial organizations and also developed in-house software. based on these discussions we developed a questionnaire in order to investigate those software development practices that lead to successful project outcomes. we chose a survey because of its simplicity and our wish to find relationships amongst variables; a survey gave us coverage of a greater number of projects at lesser cost than would an equivalent number of interviews or a series of case studies. the original practitioner group responded to our questionnaire by answering it twice, once for a project they considered successful and once for a project they considered a failure. our questionnaire was later distributed to the second group of practitioners. then the questionnaire was answered by a group of australian practitioners based in sydney. our sample is not random but rather a convenience sample of practitioners known to us. the questionnaire was organized into a number of sections covering the entire software development process. we also asked respondents if they considered the project they referenced, when answering the questionnaire, to be a success. only questions relating to requirements engineering and requirements management are considered in this paper. sections of the questionnaire, not considered here are discussed elsewhere e.g., (verner & evanco 2003, 2005). our perspective could be broadly considered a review of in-house requirements management practices. this perspective is important because most software engineering research has emphasized “technical matters above behavioral matters” (glass 2002). moreover, there has been a general lack of quantitative survey-based research regarding early non-technical aspects of software development. in addition, in-house software development failure is unlikely to receive the same attention as third-party software development failures with their attendant litigation and negative media coverage. our motivation is to show which requirements engineering practices are directly related to project success. the standish reports on the state of software engineering practice indicate that requirements engineering is critical to software success (standish 1999). when requirements are poorly defined and re processes are ad hoc, the end result is nearly always an unsatisfactory product or a cancelled project. a standish group report revealed that three of the top ten reasons for challenged projects or outright project failure were lack of user involvement, unstable requirements and poor project management (standish 1999). a more recent survey of twelve uk companies found that requirements problems accounted for 48% of all software problems [18]. another recent report from the u.k. stated that only 16% of software projects could be considered truly successful. “projects are often poorly defined, codes of practice are frequently ignored and there is a woeful inability to learn from past experience” (jobserve.com 2004). another survey of 150 companies in the u.s. showed that the majority requirements modelling technique of choice was “none” (neill & laplante 2003). in section 2, we discuss some details of the questionnaire responses, in section 3, the ajis vol 13, no. 1 september 2005 227 results of the questionnaire, and in section 4, the implications of our results. section 5 presents our conclusions and suggestions for further research. questionnaire responses we received completed questionnaires from 143 respondents, reporting on 164 distinct projects. as noted earlier, the majority of our respondents were developers involved with software for use within their own organizations (financial institutions, banks, pharmaceutical companies, insurance companies, etc.). the responses to the first set of 42 questionnaires described 42 projects, 21 regarded as successful and 21 unsuccessful. the second set of responses included descriptions of 80 unique projects reported from various companies in the northeastern u.s. the third set of responses, completed by developers working in sydney, australia, included descriptions of 42 unique projects a sample of 164 projects is a reasonable size for empirical software engineering research. sixty-six percent of projects were regarded as successful and 34% unsuccessful, 88% were development projects (63% successful), and 12% were large (in terms of effort) maintenance/enhancement projects (75% successful). the percentage of projects by number of full-time it employees is 1-4 = 39%; 5-9 = 24%; 10-19 = 19%; 20-29 = 5%; 3039 = 4%; 40-99 = 6%; and 100-180 = 8% (range 1-180, median 6). results and analysis the percentage of “yes” responses to the survey questions is shown in table 1. table 2 shows significant correlations with project success (<0.05) as well as some associations between responses to selected questions. we have classified our questions in these tables as follows: “c” refers to questions that deal with the project sponsor, customers and users; “r” to questions directly related to requirements; and “m” to questions related to the management of the development process. requirements questions although good project management necessitates that requirements are complete and consistent [28], gathering requirements with a specific methodology (r1) was not significantly correlated with project success (table 2). however, in 49% of our projects respondents did not know what requirements methodology was used. for the ones that did know, four projects used prototyping and eleven used jad sessions with prototyping; for the remainder of projects, interviews and focus groups were the main requirements gathering method. ajis vol 13, no. 1 september 2005 228 table 1: percentage “yes” responses to questions id question success 18 % yes failure 19 % yes total 20 % yes c0 were the stakeholders committed and involved? 66 57 63 c1 there was a high level of customer/user involvement 80 47 69 c2 there was a high level of customer/user confidence in the development team 70 29 56 c3 there was a low level of customer/user turnover 73 55 65 c4 senior level project sponsorship lasted right through the project 80 50 70 c5 you were affected by large numbers of customers/users 29 33 30 r1 were requirements gathered using a specific method? 53 50 52 r2 were requirements complete and accurate at project start? 47 25 40 r3 if not complete at start were requirements completed later? 80 23 56 r4 overall, were the requirements good? 81 28 66 r5 did the project have a well defined scope? 81 46 69 r6 did the scope increase during the project? 61 74 66 r7 customers/users made adequate time available for requirements gathering? 80 42 66 r8 was there a central repository for requirements? 77 44 66 r9 did requirements result in well defined deliverables? 79 37 64 r10 did the size of the project have a negative impact on requirements? 31 52 38 m1 the requirements were managed effectively 86 35 64 m2 was the project manager experienced in the application area 69 69 69 m3 was a defined development methodology used? 73 50 66 m4 was the methodology appropriate for the project? 81 46 65 m5 was delivery decision made with appropriate requirements information? 67 20 51 18 this column represents the percentage of “yes” answers to questions for successful projects. 19 this column represents the percentage of “yes” answers to questions for projects that were failures. 20 this column represents the percentage of “yes” answers to the questions for all projects. ajis vol 13, no. 1 september 2005 229 m6 the project manager was able to choose the development methodology 41 25 34 thirteen of the 15 projects using prototyping and/or jad were successful. eight u.s. projects used uml to document requirements; only four were successful; practitioners commented that there were “too many new things without a pilot” and “unfamiliar methods used”. this indicates that either uml imposes what might be considered non-requirements notations upon the requirements, or that some developers were unfamiliar with uml. also, the failed uml projects all had other problems, e.g., poor estimates, and no risk management, so their failures were not necessarily only due to use of uml. no australian projects used uml what appears to be more important than a defined requirements gathering methodology (r1) is that the project has a defined software development methodology (m3) that is appropriate for the project (m4), as both of these variables were significantly correlated with project success. surprisingly, one-third of projects did not have a defined development methodology. nearly half the projects began with incomplete requirements (r2). it is therefore not surprising that the scope was changed for many projects (r6); a s 2 test of r2 with r6 was significant. the scope was also more likely to change for larger projects. table 2: correlations of questions to project success and to other questions id question direction of success relation -ship significant correlation with project success s 2 correlation with other questions c0 were the stakeholders committed and involved? c4 c1 there was a high level of customer/user involvement + 0.000 c2, c4, r5 c2 there was a high level of customer/user confidence in the development team + 0.000 c1, r5 c3 there was a low level of customer/user turnover + 0.019 c4 senior level project sponsorship lasted right through the project + 0.000 c0, c1, r5 c5 you were affected by large numbers of customers/users r1 were requirements gathered using a specific method? m4 r2 were requirements complete and accurate at project start? + 0.006 r6 (-), m4, r5 r3 if not complete at start were requirements completed later? + 0.000 m4, r5 ajis vol 13, no. 1 september 2005 230 r4 overall, were the requirements good? + 0.000 m4 r5 did the project have a well defined scope? + 0.000 m1, m3, m4, m5, c1, c2, c4, r2, r3, r4, r6 (-), r7, r8, r9, r10 (-) r6 did the scope increase during the project? r2(-), r5 (-) r7 customers/users made adequate time available for requirements gathering? + 0.000 m4, r5 r8 was there a central repository for requirements? + 0.000 m1, m4, r5 r9 did requirements result in well defined deliverables? + 0.000 m4, r5 r10 did the size of the project have an impact on requirements? 0.000 r5 (-) m1 the requirements were managed effectively + 0.000 r8, m4, r5 m2 was the project manager experienced in the application area m3 was a defined development methodology used? + 0.007 m4, r5 m4 was the methodology appropriate for the project? + 0.000 r1,r2,r3,r4,r 5,r7, r8,r9,c4,m1, m5 m5 was delivery decision made with appropriate requirements information? + 0.000 m4, r5 m6 the project manager was able to choose the development methodology our results, shown in tables 1 and 2, indicate that requirements continue to be a big problem for software development (moynihan 1997, schenk et al 1998) and one of the most common causes of runaway projects (glass 2001). given that control over requirements is necessary to move from the lowest cmmi level, it is clear that many of the organizations in our sample are still at the lowest level (cmmi 2004). these results agree with (neill & laplante 2003), whose respondents thought that their companies did not do enough requirements engineering. while sixty percent of projects began with poor requirements, fewer than 10% of projects used a development methodology designed to deal with unclear requirements. not surprisingly, and consistent with observations made by glass (1998), we found that good requirements (r4), that were complete and accurate at the start of the project (r2), with a well-defined project scope (r5), resulting in well-defined software deliverables (r9), were all positively correlated with project success. the importance of user ajis vol 13, no. 1 september 2005 231 involvement in requirements gathering (r7) supports the observations of both clavadetscher (1998) and glass (1998). we found that if requirements were initially incomplete, completing the requirements during the project (r3) was positively correlated with project success. although boehm (1991) includes a “continuing stream of requirements changes” in his top ten risk items, we did not find that changing the scope during the project (r6) was correlated with project failure. also, being able to effectively manage requirements and any changes to them (m1) through a central repository (r8) was positively correlated with project success. the fact that only 66% of our projects used a central repository supports the suggestion that “we fail to use requirements management to surface (early) errors or problems” (clavadetscher 1998). when the size of a project impacted on requirements gathering (r10), project failure was more likely. this result agrees with (glass 1998), suggesting that project size hampers requirements gathering, and leads to unclear, incomplete, and potentially unstable requirements. large numbers of customers and users had no significant impact on project failure. using logistic regression with the responses to the requirements questions, the best predictor of project success was r4 (the requirements were good) which predicted 89% successes, 58% failures, and 78% of projects correctly overall. sponsor, customer and user questions a project that has customers/users who have a low turnover rate (c3), who have confidence in the development team (c2), and who have a high level of involvement in the project (c1), is likely to be a success. however, having a large number of customers and users (c5) was not correlated with project failure. evidence shows that a high level of customer/user involvement right through the project from requirements elicitation to acceptance testing is necessary for project success (standish 1999). the correlation between customer/user involvement (c1) with level of confidence they have in the development team (c2) is interesting and leads us to ask about causal effects. “are customers/users involved because they are confident in the development team or if they are involved do they become more confident in the development team?” we suspect that the answer is the former. this leads us to suggest that development teams who do not present themselves well to users and manage customer/user expectations properly may be sowing the seeds for failure. we were not surprised that a high degree of senior level sponsorship that lasted right through the project (c4) was significantly related to (c0) committed and involved stakeholders and (c1) a high level of customer/user involvement. using logistic regression with responses to the sponsor, customer and user questions, the best predictor of project success was c1 (there was a high level of customer/user involvement), with c2 (there was a high level of customer/user confidence in the development team) which predicted 90% successes, 51% failures and 78% correctly overall. on its own c2 (there was a high level of customer/user confidence in the development team) predicted 70% projects correctly overall. project management questions a pm experienced in the application area (m2) was not correlated with project success. “successful project managers are generalists, not technical specialists”; while a certain level of technical competence is helpful, managerial and interpersonal skills are more ajis vol 13, no. 1 september 2005 232 important (jurison 1999). a project that has a pm who manages requirements effectively (m1), and uses a well defined software development methodology (m3) that is appropriate for the project (m4) and that has estimates of effort and schedule made with appropriate requirements information (m5) is likely to be successful. good estimates of effort and schedule (c4) have a huge effect on project success (demarco & lister 2003). as early as 1975 brooks stated that more projects have gone awry for lack of calendar time than from all other causes combined (brossler 1999). optimistic estimation is still one of the two most common causes for runaway projects (glass 2001) with cost and schedule failures exceeding any other kinds of software failures in practice (glass 2003). boehm (1991) includes unrealistic schedules and budgets in his top 10 risk items. using logistic regression with the responses to the project management set of questions, m5 (making delivery decisions with appropriate requirements information), with m4 (the development methodology was appropriate for the project) and m1 (the requirements were managed effectively) was the best predictor of project outcome, predicting 86% successes, 77% failures, and 83% correctly overall. on its own m1 (the requirements were managed effectively) predicted project outcome correctly for 77% of projects. this result supports davis, who claims that requirements triage is critical: determining which requirements a product must have given a time constraint and resources available within that time frame (davis 2003). important correlations the most important project success prediction factors are that the requirements were good (r4) and that the requirements were managed effectively (m1). these two factors alone correctly predicted 93% of successful projects. having good requirements is highly correlated with a high level of customer/user involvement. it is difficult to get good requirements without customer/user involvement. we investigate two factors more thoroughly since they are discussed little in the requirements research literature. these are (r5) did the project have a well-defined scope, and (m4) was the development methodology appropriate for the project? as shown in table 2, both have significant correlations with many other factors. note that there are also many other significant correlations that we have not discussed in this paper nor are shown in table 2. scope a well-defined scope is critical to project success. we found that scope was significantly, positively correlated with a number of factors: • c1, a high level of customer/user involvement. without this level of customer/user involvement, it is not easy to identify the problem to be solved. without this identification it is impossible to define a project’s scope. asking, “what functions do you want?” and not asking, “what is this system for, who’s involved?” is not likely to help define scope accurately. you can only ask these questions throughout the project when you have a high level of customer/user involvement. • c2, there is a high level of customer/user confidence in the development team. c2 is significantly correlated with c1. it is interesting that this is an ajis vol 13, no. 1 september 2005 233 important factor though not particularly well addressed in the research literature. without a high level of confidence, one is less likely to elicit the right scope and from this weak starting place one is less likely to elicit the right requirements. • c4, senior project sponsorship lasted right through the project. this is a critical success factor. high level support induces greater cooperation that could be missing without such sponsorship. a high level sponsor may also be more aware of the wider scope of the project’s impact. • r2, the requirements were complete and accurately defined at the start of the project. there is a natural correlation to scope. with inaccurate scope or unmanaged scope creep, it will be difficult to identify a complete requirements set. • r3, the requirements were completed at some stage in the project. similar to r2, a well-defined scope even if it creeps, can still allow a complete requirements set at some point during development. so long as the project chunk being worked on at one time has well-defined scope and the requirements are complete, then project success is more likely. • r4, overall the requirements were good. given a well-defined scope, it should be easier to identify all the necessary requirements, i.e. requirements were good. • r6, did the scope increase during the project? this is negatively correlated; that is, the more scope increased, the less likely it was to be well-defined. • r7, customers/users made adequate time available for requirements gathering. exploration of the problem space with customers and users who have time to discuss this allows for better scoping of the project and of manageable chunks for development. • r8, there was a central requirements repository. this is a critical success factor. it is entirely necessary to have one, and only one, repository to store the requirements. this, of course, aids in scoping the project. it is easy for the development team to see the scope of their project and know that it is the agreed scope project-wide. • r9, the requirements resulted in well-defined deliverables. this is often difficult to do without a well-defined scope simply because the goal posts may keep shifting. • r10, the size of the project had an impact on requirements. this is negatively correlated; that is, the larger the project, the more important it is to define scope. it is also much more difficult to achieve. • m1, the requirements were managed effectively. a well-defined scope and decomposition of the project into related, manageable requirements chunks is difficult. good project management and in particular, requirements management, is essential for a successful project outcome. • m3, a defined development methodology was used. a development methodology appropriate to the problem enables a better scoping of the requirements in that there is more likelihood that the project is scoped according to the relevant aspects of the defined methodology. as an example, all requirements relating to an information system will be ajis vol 13, no. 1 september 2005 234 scoped together to fit an information systems method within the wider methodology. • m4, the methodology was appropriate for the project. this is very similar to m3 above. a project’s parts are significantly better scoped if the development methodology of choice is appropriate to the project’s parts. • m5, the delivery decision was made with appropriate requirements information. a well-defined scope will significantly improve the success of delivery decisions because without this knowledge it will be difficult to know what can be delivered as a complete piece of work within the project. appropriate methodology an appropriate lifecycle development methodology is shown to be significantly correlated with project success. there is some literature to support this notion, for instance (jackson 2001), though it appears vendors are happy to assume their one-size-fits-all does indeed fit. an appropriate methodology, m4, is significantly correlated with: • c4, senior level sponsorship lasted right through the project. a senior sponsor can enforce the right methodology and can equally defend a project manager or developer’s choice of methodology. this support is important to successful uptake of the approach. • r2, the requirements were complete and accurate at the start. naturally, complete requirements allow for an identification of sub-problem types within the a project, and then a choice of the appropriate methodology becomes more apparent. • r3, requirements were completed at some point in the development. as for r2, understanding aspects of the problem allows the right choice of method for that problem part. so though requirements might not be completed at the start, awareness of the types of problems being addressed allows for choice of the right methods. • r4, overall the requirements were good. requirements are often better achieved when they are developed using the appropriate method. • r5, see m4 in section 3.4.1. • r7, customers/users made adequate time for requirements gathering. when this occurs, it is easier to get the right requirements, to understand the problem and then to select the appropriate methodology. • r8, there was a central repository for requirements. this helps the appropriate selection of methodology simply because there is one location to look for requirements and therefore one place to organize the requirements appropriately. it is easier to select the methodology based on this structure and single point of information. • r9, did the requirements result in well-defined deliverables? an appropriate methodology and an appropriate, well-defined scope, allow for well-defined deliverables that are actually delivered according to their scope as defined. • m1, the requirements were managed effectively. requirements management is part of project management. methodological selection ajis vol 13, no. 1 september 2005 235 is simplified through good requirements management as it is easier to understand the problem to be solved, and from there select ways to do that appropriately. • m5, delivery decisions were made with the appropriate requirements information. this is correlated with an appropriate methodology. it is much easier to make these kinds of decisions when you can trust the approach you are using for development. there are many factors other than those we have discussed above. you can’t make accurate delivery schedules without scoping your project. you can’t get this information except for a combination of factors, including budget, which we have not considered at all. although this discussion appears simple, it is more complex than we portray. politics, for instance, is something almost entirely ignored in the requirements research literature. in the workplace, it is a highly significant factor to what requirements are actually delivered and what methodology is selected. discussion the developers we surveyed mainly develop in-house software for their organization’s use. their organizations have a heavy reliance on software for many business functions. while we would not assume that our results are typical of all organizations, we believe that they are reasonably typical of organizations that develop in-house software. surveys are of course based on self-reported data which reflects what people say happened, not what they actually did or experienced. because we surveyed software developers our results are limited to their knowledge, attitudes, and beliefs regarding the projects and pms with which they were involved. however, as the majority of projects are fairly small (63% employed fewer than 10 people and 84% fewer than 20), we believe that our respondents have a reasonable knowledge of most project events. the overall preponderance of small projects may however, bias our results. overall, the best logistic regression prediction equation using data from each of the three groups of questions, was r4 (overall the requirements were good) with m1 (the requirements were managed effectively) which predicted 93% successes, 59% failures and 83% correctly overall. we were surprised that so many projects started (and continued), with unclear requirements. why are pms prepared to go ahead with projects without either appropriate requirements or a development methodology able to deal with unclear requirements? it is common knowledge that good requirements lead to software development success so why are pms apparently so unaware that they are prepared to jeopardize project success in this fashion? poor requirements have a negative effect on the estimation process; this then leads to schedule and cost underestimates, inadequate staffing and then staffing itself becomes a major risk factor. many project management problems are in fact requirements problems in disguise, particularly those related to scheduling and effort estimation. the results suggest that senior management needs better education regarding the importance of adequate requirements, and that good requirements are necessary to produce appropriate schedule and effort estimates. while some consider that using uml for requirements modeling and management is helpful, in this research we find no supporting evidence. to the contrary, in at least one ajis vol 13, no. 1 september 2005 236 project the use of uml was forced upon the development team with no accompanying training; project failure was the outcome. it might be important to distinguish between scope creep and requirements creep more clearly. evolving requirements throughout a project tends to have no significant impact on success as long as requirements were considered complete at some point during the project. in contrast, scope increase was not correlated with project success. perhaps scope ought to be defined as the boundaries of the problem domain within which to seek requirements. it is important that these boundaries be defined clearly early in the project, whereas the requirements within those boundaries may evolve continuously. finally, having a central repository for requirements is clearly correlated with project success. this is good news, because it is relatively easy to do. in fact, it is difficult to understand why a project would not have a central repository for requirements given the technology available today. conclusions and further research we discovered that: 1) it is not the number of users involved that is important, but rather managing the size of the project in terms of functionality; 2) it is not the requirements methodology per se, but rather use of an appropriate software development methodology into which the requirements methodology fits; 3) it is not scope creep, but rather that scope is well defined when it creeps; 4) it is not a project manager experienced in the application area, but rather a project manager who manages requirements effectively; 5) it is not necessarily having complete requirements at the start of the project but rather completing the requirements at some stage during the project; and 6) projects that had a central repository for requirements were more likely to succeed. the most important correlations for project success are to get good requirements and to manage those requirements effectively. getting good requirements means a number of things. some that are important are a high level of customer/user involvement, high-level sponsorship throughout, to scope the project effectively and it is critical to have a good project manager who can manage, rather than one who just happens to know the application domain. table 1 shows that current practices are fair at best. there is much opportunity for improvement at the start of a project in the requirements area. this is very important if we wish to increase the quality and success of our software projects. analysis of our survey suggests further research is required in order to investigate: what kinds of pressures lead project managers not only to start projects with poor requirements, but also to actually complete them without really knowing what the requirements are? how might the requirements analysis activity be better integrated with scheduling and cost estimation? it may be important to distinguish more clearly between project requirements versus project ajis vol 13, no. 1 september 2005 237 scope. is it possible that a good definition of scope at the outset of a project enables project teams to better deal with loosely defined requirements that later evolve? customer involvement and customer confidence in the project team indicate better likelihood of success. how are these interrelated? do customers become more involved because they are confident in the team, or are they confident because they are involved? what motivates customer involvement and confidence? this research serves as a starting point in motivating continuing research in requirements practice in industry and project success factors. we intend to continue with this research in the future. references boehm b. w., software engineering economics, prentice hall, englewood cliffs, nj (1981). boehm b. w., “software risk management principles and practice”, ieee software vol. 8 no 1, 1991, pp32-41 booch, g., rumbaugh, j., jacobson, the uml user guide, addison wesley, 1999. brooks f. p. jr., the mythical man month. essays on software engineering, addison wesley, usa (1975). brossler p. “knowledge management at a software house: a progress report” proc workshop on learning organizations (1999), pp 77-83. carroll, j. (ed.), scenario-based design: envisioning work and technology in system development, wiley publications, 1995. checkland, p., systems thinking, systems practice, wiley publications, 1981. clavadetscher, c. “user involvement: key to success”, ieee software. vol. 15, issue 2 (mar/apr 1998) p. 30, 32. cmmi http://www.sei.cmu.edu/cmmi/ accessed 5 th july 2004 collier b, t. demarco and p. fearey, “a defined process for project postmortem review”, ieee software, vol. 13, no. 4 (1996), pp. 65-72 davis, a., 2003, “the art of requirements triage”, ieee computer, march, pp42-49 davis, a., and a. hickey, “requirements researchers: do we practice what we preach?”, requirements engineering journal, 2002, 7, pp107-111. demarco t. and t. lister, waltzing with bears, dorset house publishing new york ny, 2003 glass r “error-free software remains extremely elusive”, ieee software january/february, 2003, pp103-104. glass, r. l. “how not to prepare for a consulting assignment and other ugly consultancy truths”, communications of the acm. volume 41, issue 12 (december 1998) pp11-13. glass r. l., “frequently forgotten fundamental facts about software engineering” ieee software, may/june 2001, pp112, 110, 111. glass r. l., “project retrospectives, and why they never happen”, ieee software 2002, september/october, pp112, 111. t. hall, s. beecham, and a. rainer, “requirements problems in twelve companies: an empirical analysis” iee proceedings software, 149 (5), pp153-160, 2002 jackson, m., problem frames, addison wesley, 2001. jobserve.com, “uk wasting billions on it projects”, http://www.jobserve.com/news/newsstory.asp?e=e&sid=sid2598, 21/4/2004. ajis vol 13, no. 1 september 2005 238 jurison, j., , software project management, the manager’s view (1999) communications of ais volume 2, article 17. kerth, n. l., project retrospectives: a handbook for team reviews, dorset house publishing, new york 2001. keuffel, w., “planning for and mitigating risk” software development 7 9 (1999) pp8185 kwak, y. h. and stoddard j., “project risk management: lessons learned from software development environments”, in press, to appear in technovation, 2004 kwak, y. h. and ibbs, c. w. “calculating project management’s return on investment” project management journal 31, 2, 2000 pp38-47 moynihan t., “how experienced project managers assess risk”, ieee software, vol 14, (may/june 1997) pp35-41. neill c. j. and laplante p. a., requirements engineering: state of the practice” ieee software, november/december 2003 pp 40-45. osmundson j. s., michael j. b., machniak, m. j., and grossman m. a., “quality management metrics for software development” accepted for information and management, 2003. phan, dien d., douglas r. vogel and jay f. nunamaker jr., ”empirical studies in software development projects: field survey and os/400 study, information & management 28 (1995) pp271-280 pressman, r. software engineering: a practitioners approach, mcgraw hill (1996). schenk, k.d and vitalari, n. p., et al. “differences between novice and expert systems analysts: what do we know and what do we do?”, journal of management information systems, vol. 15, issue 1 (summer 1998) pp9-51. standish group “chaos: a recipe for success”, standish group international, 1999 verner j. m. and evanco w. m., “a investigation into software process knowledge” in managing software engineering knowledge, (eds) a. aurum, r. jeffery, c. wohlin and m. handzic, springer-verlag, 2003, pp29-47. verner j. m. and evanco w. m., “in–house software development: what software project management practices lead to success?” ieee software jan/feb vol. 22, issue 1, 2005, pp86-93 microsoft word ajis10-2finala.doc ajis vol. 10 no. 2 may 2003 105 challenging dualisms in female perceptions of it work jenine beekhuyzen sue nielsen liisa von hellens school of computing and information technology griffith university brisbane, australia email: j.beekhuyzen@griffith.edu.au; s.nielsen@griffith.edu.au; l.vonhellens@griffith.edu.au abstract this paper explores the way that professional women working in the it industry discuss the nature of their work and it is suggested that the way women talk about their work reinforces widely held impressions of the information technology industry. structuration theory illuminated how this talk is not always consistent with the womens' lived experiences. the interview data revealed contradictions in these dualisms, indicating that these polarised views of women and it work are being undermined by women in the it industry. this paper suggests that mentoring, interactions with professional it organisations, and professional it women talking to females in their it education years can give new ideas to the perceptions of it and thus challenge these dualisms. the interviews were coded using nud*ist, which allowed the researchers to explore the interviewees perceptions of it work. in this paper, the perceptions of the interviewees are discussed as structures of signification that need to be altered in order to successfully challenge these dualisms. it is suggested that exposing females in their it education years to the professional it women who are challenging these dualisms is an essential part of transforming these structures of signification. this research is part of an ongoing project (winit) commenced in 1995, which seeks to understand the declining female participation in it education and work. in order to have a better understanding of the way women help configure the institutional realm of it work, we propose that more qualitative studies of women at work in it as well as women talking about it are needed. keyword women in it, structuration theory, australia, nud*ist, mentoring introduction this paper uses some concepts from giddens’ structuration theory to examine the way that women talk about working in the information technology (it) industry. the research is part of an ongoing project (winit) commenced in 1995, which seeks to understand the declining female participation in it education and work. this research attempts to capture the experiences and perceptions of women who work in this very competitive, declining female workforce. the data was collected primarily through interviews with female it professionals, and a small number of male it professionals. statistics on the representation of women in the it industry are worrying. there has been a small, though noticeable, rise in female participation in the traditionally male-dominated professions of science, engineering and medicine. however, there has not been a corresponding trend in the information technology sector. at the end of 1999 women represented just 29 percent of the workforce in the us technology sector, according to a study by the white house council of economic advisers. that is down from 40 percent of the workforce in 1986 (council of economic advisors 2000). by the start of the last decade, the proportion of females in the it workforce was beginning to decline in most western countries (von hellens, pringle, nielsen et al. 2000). this is despite the idea that "women may prove to be a key resource of skilled technology workers for international it markets" (maitland 2001) . this is a complex issue and has attracted increasing research interest; for example, see the review by ahuja (2002). the winit research carried out to this point has sought to develop an understanding of complexity and social embededness of the problem of it recruitment. particularly, the research has focused on the cultural and gender factors affecting female recruitment (von hellens and nielsen 2001). a conceptual model of the identified factors was developed and such an approach was found to be useful in examining the situation with relation to high school students and first year university students (nielsen, von hellens, pringle et al. 1999). the research considers factors such as media representations of it, peer, school and parental influences. both male and female students lacked information about it work and education and did not demonstrate any understanding about the nature of it work. overwhelmingly the students perceived it as a masculinised domain and the female students mainly saw it courses as boring and difficult. our more recent research has focused on women working in the it industry and it academia, where the significance of informed choice is more evident (pringle, nielsen, von hellens et al. 2000; trauth, nielsen and von hellens 2000; von hellens et al. 2000). these women have resisted in some way the influences, which discourage young women from entering it and have persisted in developing their it careers. this paper presents early attempts to reexamine texts of professional women’s interviews using some concepts ajis vol. 10 no. 2 may 2003 106 from structuration theory. these concepts were initially established as relevant through a rereading of the interviews. until 1999, the second and third authors of this paper (the primary researchers of winit) had been primarily focusing the winit project on university and high school students. in 1999, the winit research was extended to cover female it professionals and initial results were published (pringle et al. 2000; von hellens et al. 2000). in collaboration with professor eileen trauth in 2000, the research focus further expanded to include more interviews with professional women in it. both parties carried out interviews with women from a variety of backgrounds. the authors of this paper first introduced the use of structuration theory to the interview data in our acm sigcpr/sigmis paper (nielsen, von hellens and beekhuyzen 2003), but in this paper we discuss the analysis of the data in more detail and the utilisation of nud*ist software. in the last two decades structuration theory has been applied to information systems research in a number of ways (walsham and han 1990; jones 1999; walsham 2002). this paper is in the growing tradition information systems research where “particular concepts from giddens’ writing are adopted”(jones 1999). most of these studies have focused on the nature of technology and its relationship to human agents in information systems development and use. we intend to extend this application to the problem of the relationships between the it education and commercial industries, social perceptions of it education and work and the human resource requirements of the it industry. theory of structuration is used in this paper as a basis for analysing the changing relationship between social structures (the it education and commercial industries) on the one hand, and human agency (students and it professionals) on the other hand. it is not our intention to describe giddens’ theory of structuration, or discuss its many critics. at this stage of the research we intend to apply some of the concepts of structuration to the interview data to seek further illumination of the complex topic of women in it. halford and leonard (2001) suggest that giddens offers “considerable promise for the development of fuller understandings between gender, power and organisation”. the research deals with two extremely complex areas. firstly the attitudes and motivations of young people, particularly females, choosing educational and career paths and the social factors which influence them. secondly, the nature of an industry which is difficult to compare to any other – being of fairly recent origin, not governed by the same professional and regulatory constraints as other complex industries (such as engineering, accounting and law) and subject to recent upheavals in terms of skills demands and shortages. there is a very large body of literature on the demand for it skills. however the primarily positivist approach taken in this research is not relevant with the structuration theory approach. from a structuration theory point of view, demand for it skills cannot be treated as a matter of social fact, but is implicated in the structuration of the it industry, and reinforced or challenged by industry participants. the interview data is discussed throughout the paper using the concepts of duality and dualisms, routinisation and interpretive schemes and signification. the focus is on structures of signification since the research has so far collected only interview and survey data. to use the other dimensions of giddens’ duality of structure it would be helpful to undertake observation and the examination of documentation to identify how domination and legitimation occurs through the use of power and sanctions. in this paper, the authors outline a mentoring program that we conducted, which can be designed to address the dualisms that construct the it field as something that is not appealing to young females. data collection interviews were conducted in 1999-2000 with thirty-two (32) female and two (2) male it professionals working in australia. the interviewees represent a range of ages, employment sectors, educational backgrounds and ethnic backgrounds. the males were interviewed primarily to provide insight into the female experience. working in technical areas, the interviewees are referenced to explore the duality of the it industry, as it is perceived by women working in the industry. the importance of operating within a male domain, including the extent of outside-work socializing with male colleagues (such as attendance at sporting events) as necessary for career progression has been identified previously (von hellens, nielsen and trauth 2001). the implication is that while women have been socialized away from it, the it field has been constructed as a masculine domain. this follow-up study by the collaborating authors in 2000 (trauth et al. 2000) enabled a deeper exploration of the current position of australian women working in technical positions in the it industry. this paper revisits the interview data discussed in the trauth et al. paper (2000) and in the von hellens et al papers (2000; 2001) from the perspective of structure and agency. it aims to explore perceptions of the it industry through the eyes of female it professionals. the data collection was a collaborative effort between von hellens and nielsen, australian-based researchers, and trauth, a us-based researcher. ajis vol. 10 no. 2 may 2003 107 semi-structured open-ended interviews of approximately 90 minutes in duration allowed interviewees to express their personal views and discuss their individual experiences in the australian it industry. four main areas were covered, including demographic information, personal history, general questions about gender and it, and recommendations regarding how society, the it profession and educational institutions might address the gender imbalance in it (von hellens et al. 2001). the interviewed women are not a random sample, they were gathered primarily from the ‘women in information technology’ (wit) whose has a mission to sustain an active network of women in the it industry, facilitating career development and promoting enhanced industry management in brisbane, australia (wit 2002). these women are both the subjects of our study and informed observers of the skills issues faced by women in the industry. this study does not consider the observed skills required by women in the industry such as todd et al., (1995), instead it explores the perceived perceptions of these women. the respondents were it practitioners working in a range of levels at a range of industries in various parts of australia and academics in it faculties. the respondents drew upon their own experiences to offer insights about the themes, which were explored in this study. the female subjects offered two perspectives on the topic; their experiences of entering and progressing in the it field, and helping other women enter and succeed in it (von hellens et al. 2001). data analysis initial sorting and analysis of the interview data was carried out with the aid of nud*ist, an australian qualitative software application. nud*ist is an acronym for non-numerical unstructured data, indexing, searching and theorising and is designed to aid users in handling non-numerical and unstructured data in qualitative analysis, by “supporting processes of coding data in an index system, searching text or searching patterns of coding and theorising about the data” (qsr 1997). from the outset, the data was messy, but the problem was clear: a lack of participation of females in the it industry. nud*ist was employed to help in structuring the large amounts of interview data collected from the it professionals. the processes of nud*ist support the principles of qualitative research; that enquiry is interactive, building on the results of previous inquiries and constructing new ideas out of old ones (qsr, 1997, p30). the core of the winit project strategy is to analyse data qualitatively, building on the results of previous inquiries over time. the authors believe this has worked effectively in the project so far. nud*ist was found to be helpful as it reminds the researcher where data is, eg, a conversation about harassment in the workplace is located under the node “harassment”. the software allows the researcher to connect a ‘term’ and a ‘concept’ which allows for enriched indexing. taking into consideration that expressions cannot be taken at face value, this issue relates to the notion of context which has been discussed in terms of the hermeneutic circle (klein and myers 1999). the systematic approach is efficient at indexing and searching however, in this experience the promise of theorising was not of much value. during analysis, the 1 st author developed an indexed database system which provided access to data in ways which paralleled the project aims and research methodology (interviews, questionnaire). after the initial coding of interview data at nodes, the data was searched manually for distinctive patterns of text. significant contradictions/oppositions were found in the data; for example, the contradiction of work/home (refer table 1). we discovered unexpected issues when the project moved from a single-researcher analysis using the tool to shared analysis. it became very important to make a note of any inconsistencies in the data, and to regimentally adhere to the standards of coding agreed to by all parties. we found a configuration management plan to be very useful. an obvious advantage for using this software is that it allowed the data to be analysed consistently among researchers. due to the longitudinal nature of this study, a number of researchers are likely to be involved in coding during its continuation. another advantage of using this software is that has allowed for easy back coding of data that was collected in previous years. as the project was established in 1995, there are many interviews that could benefit from being analysed using nud*ist. an examination of our interview data using concepts from structuration theory shows that a number of dualisms provide the interviewees with interpretive schemes through which they can interpret and make sense of their working lives. such dualisms may provide ontological security as representations of the routines which women in the it industry enact in their daily activities. the interviews of professional women reveal contradictions in these dualisms, indicating that these polarised views of women and it work are being undermined by women in the it industry. ajis vol. 10 no. 2 may 2003 108 duality and dualisms central to structuration theory is the notion of duality (giddens 1984). in keeping with modern trends in sociology, giddens rejected the notion of outright dualism, in which the fundamental forms of things are viewed as of two contrasting irreducible types (for example nature and nurture). according to many writers on giddens’ the notion of duality attempts to resolve the ‘agency/structure debate’ (walsham and han 1990). this debate concerns the paradox of social reality whereby “the properties of the elements of social phenomena obtain many of their characteristics from the larger phenomena of which they are a part, while the larger entities obtain their characteristics mostly from the relations between the parts of which they are composed” (cohen 1968). “structuration is an approach to understanding the social world that replaces the dualisms of social theory with a duality of reciprocal interdependencies” (boland 1996) giddens criticised contemporary sociology for treating the relationship between the individual and society as a dualism, in which individual and society are separable, and for producing social theories which focus on one side of the paradox; functionalism for example regarding the individual as produced and determined by society, whereas interpretivism regards the individual as the producer of social reality. in contrast to dualism the notion of duality proposes a dialectical interaction between conceptual pairs with no claim that that these concepts are ultimately irreducible. thus giddens’ approach is to propose a duality of structure in which “the structural properties of social systems are both the medium and the outcome of the practices that constitute these systems” (giddens 1979). the authors identified a number of dualisms in the way women typically represent their work experiences in the it industry (summarised in table 1 & 2). this paper briefly explores some of these dualisms and how contradictions in their expression seem to indicate that polarised views of women in it are being modified. in other words the dualisms are viewed as social structures which influence individual actions both in an enabling and constraining manner, but which over time may be reinforced, modified and transformed by individual actors. writers in women’s studies have shown how such dualisms are constructed (barrett 1992) depending on prevailing social tendencies (such as the dualism between individual and society as a product of the enlightenment). they have also shown how the notion of dualism enables the promotion of one type of experience over the other so that dualisms often consist of unequal conceptual pairs, (such as active/passive, public/private) associated with gender and power (porter 1991). a dualistic approach to the problem of female participation in it can be seen in much of the literature, particularly that proposing the idea of inherent differences (e.g. (venkatesh, morris and ackerman 2000) which identifies gender differences in adoption and usage of technology. dualisms of skills and attributes home/work it industry/career certainty it work/emotion intuition/analysis programmer/people focused soft skills/technical education solving problems/talking to people technology/business problems technical/management technical/people skills technology/communication skills table 1 dualisms in it work the dualisms in table 1 represent it work and skills as a set of either/or propositions. to be good at programming infers lack of communication skills. technological knowledge is distinct from business knowledge. to be a woman means having good communication skills but lesser technical skills. private or home concerns are separable from work. the dualisms in table 2 represent skilled it work and gender as a set of either/or propositions. to be assertive, to have political and networking skills and to be interested in continuous learning and rapid change infers masculinity. having a broad perspective and attention to detail are viewed as being feminine traits. ajis vol. 10 no. 2 may 2003 109 issue men women attention to detail uncommon common broad perspective uncommon common continuous learning and rapid change common uncommon assertiveness common uncommon communication uncommon common noisy vocalism common uncommon political skills common uncommon networking common uncommon left brain common uncommon programming common uncommon technical skills common uncommon table 2 gender specific dualisms in it work dualisms of it work dualism a number of dualisms of it work have been identified and these have been grouped as: hard versus soft skills, home versus work, the division of labour, routinisation, and interpretive schemes (nielsen et al. 2003). hard versus soft skills: one of the most interesting dualisms is the separation of soft and hard skills, not only in terms of their association with women and men, but also in terms of how they are learned and valued. this is expressed by one interviewee as follows; ‘skills like communication and organisational .. they’re things that you can train into yourself from a little child – your parents can help you with those. but things like technical skills like it would need … you will only get that at a tertiary institution.’ the dualism of home and work: another frequently occurring dualism is that of home/work and private/public life. this dualism is reinforced by women’s choices to delay or not have children and their need to accommodate their working lives to their private responsibilities and vice versa. this is obviously not unique to the it industry but the women interviewed perceive that it presents different problems where the majority of the work force is male and is not presented with this conflict. the workload associated with the need to keep up with the rapid rate of change is seen as difficult to reconcile with family responsibilities and makes women apprehensive about taking even short breaks from their careers, for example for maternity leave. dualisms and the division of labour: despite the recent work on new it enabled organisational forms, the interviews of professional women confirmed that without exception they perceive their workplaces and career paths in the it industry in terms of traditional work specialisations. the dualisms referred to above reflect a traditional division of labour into hierarchies of skills, associated with gender, power and authority. these hierarchies are reinforced through award systems and management evaluation criteria, which purport (claim) to be objective and equitable but which “reflect entrenched managerial values” (acker 1990 p.149). responsibility and complexity are defined according to existing social practices; hence the undervaluing of many femininised occupations such as nursing and child care which take responsibility for human lives but which are assumed not to require the same level of training as a profession requiring more ‘hard’ skills such as medicine. the gendering of it work in this regard seems to have three consequences, as revealed in the interviews. it requires both men and women to ‘adapt’ to a masculinised domain, and it discourages many women and men from choosing it as a field of study or career. both the men and women interviewed indicated that they found the working arrangements and valuing of specific skill sets in the it industry difficult. ajis vol. 10 no. 2 may 2003 110 routinisation: the dualisms listed in table 1 are expressed in the interviews in a taken for granted way. in structuration theory the habitual, taken for granted nature of everyday activities is called routinisation; according to giddens the “ the predominant form of day–to-day social activity … in the enactment of routines agents sustain a sense of ontological security” (1984 p282). moreover, routine is “integral to the continuity of the personality of the agent … and to the institutions of society.” (giddens 1984 p.60). the unselfconscious reference to and representation of traditional dualisms allows women to operate without continual self-consciousness or anomie about the nature of the work. anomie (lehmann 1994) may exist where there is a conflict between personal goals and the availability of structures to support those goals, or where there is uncertainty about organisational values. in an area such as it which is rapidly changing and which presents its professional workforce with continual challenges (including the recent dotcom crashes) the need for the ontological security provided by routinisation would be particularly strong. dualisms as interpretive schemes: the dualisms expressed above may be viewed within structuration theory as the interpretive schemes through which signification is achieved. interpretive schemes are “the stocks of knowledge that human actors draw upon in order to make sense of their own and others’ actions” (walsham and han 1990 p.54). giddens’ concept of interpretive schemes is useful in showing how these women deal with the contradictions in the way they conduct their daily working lives, “making that conduct appear rational, understandable and accountable to self and other.” (boland 1996 p.693). the emergence of contradictions points to the possible transformation of structures of signification. in structuration theory interpretive schemes are not immutable or monolithic. the meaning of particular expressions such as ‘communication’ needs to be interpreted within the social context. (giddens 1979) for example, many of the women interviewed identified networking skills as an instinctive skill at which men excel and which provides them with an advantage. male networking is carried out not only formally but also informally through common/social interests such as sports, after work drinks sessions and so on, whereas women were more likely to go home after work and take on household and family duties. however, more recently women are developing networks through formal purposeful activities such as women in it associations. one interviewee saw this as a means for enabling women to utilise their natural assets. ‘it’s all sharing and giving and more so than with guys’. issues of female participation in it structuration theory illuminates two problems in studying female participation in the it industry. firstly, to understand the production and reproduction of social structure requires a longitudinal study but the rapid rate of change in the industry including high staff turnover make this difficult. without a longitudinal study it is difficult to identify how individual agents reinforce, modify or transform institutionalised traditions within the it industry. secondly the rapid rate of change makes it difficult to conceptualise in a meaningful way what constitutes the it industry. definitions produced by industry organisations (such as the australian bureau of statistics) do not necessarily reflect the experience of people working in the industry. furthermore, staff in the it industry differ from other complex, highly paid occupations. although the it industry generally hires a significant number of university graduates, it lacks institutionalised forms such as required professional education, membership of associations, defined career paths and professional sanctions, which are characteristic of other complex occupations such as law, medicine, accountancy, architecture and engineering (encel 1970). attempting to establish why women are not attracted to it work is hampered by this lack of clarity. the women interviewed were themselves from a diverse set of backgrounds, with different educational qualifications and work histories. they proposed aptitudes and skills rather than specific professional education as important for success in the industry. they see the it industry as diverse, offering many different options for work and careers. the role of individual women in reinforcing, modifying and transforming the social structures is therefore difficult to identify. as individual actors they perceive that they have overcome barriers and succeeded in the industry. they also talk about themselves as different from other women they know, indicating that they are challenging the dualism of gender. they consider that they possess the ‘key skills going into the next millennium [21 st century]’. ajis vol. 10 no. 2 may 2003 111 mentoring mentoring, interactions with professional it organisations, and professional it women talking to females in their it education years can give new ideas to the perceptions of it and thus challenge these dualisms. the perceptions of the interviewees need to be altered in order to successfully challenge these dualisms. exposing females involved in it education to these professional it women and professional organisations such at women in it (wit) who are challenging these dualisms is an essential part of transforming these structures of signification. existing research on mentoring has focused mainly on mentoring programs within the workplace. the success of such programs depends on the level of support shown by the management of the organisation in which the program is being implemented. a mentoring scheme can help companies improve the gender balance of their staff and to develop a stable corporate culture (limerick et al., 1994). however, there is no clear australian evidence connecting mentoring to improvements in recruitment of students to specific areas of study. mentoring programs in schools are aimed at achieving the fullest potential of each student in emotional health, academic achievement, interpersonal relationships and vocational knowledge through a positive relationship with at least one adult (maccallum and beltman 1999). although there is a diverse range of models of mentoring programs operating in australian schools (maccallum and beltman 1999), there are no examples of students being mentored specifically on it career choices. it mentoring project as part of the winit project, an it mentoring project was conducted in 2001 by the authors to investigate whether mentoring programs have the potential to influence and change students’ perceptions and decisionmaking regarding it career choices and education. the project established a mentoring program that introduced high school students to the variety of career options in the it industry by providing successful it professionals as role models to provide realistic advice on the nature of it education and work. a mentoring program may play an important role in addressing the it skills shortage, increasing participation in it education by providing opportunities for informed decisions and making it education and work more appealing to students. the winit team collaborated with information and processing technology (ipt) teachers at a local secondary school to establish a mentoring program for 110 year 11 ipt students. a total of 28 mentors were sourced by the investigating team and consisted of it professionals, academics and recent it graduates. the role of the mentors was to assist small groups of students with the analysis and design stages of a programming assignment, as well as provide general advice about the nature of it work, the skills needed to succeed and the wide range of options available in the industry. the mentoring program ran for the first half of 2001 and the overall view was very positive. most mentors believed it was a worthwhile initiative and would be willing to support the program in the future. students were eager to meet their mentors face-to-face and used the time to find out more about it study and where it can lead, which helped fit the ipt subject into the bigger picture and give it a real-life context. the mentors were able to help the students with problem solving, teamwork and working the class assignment by sharing their expert knowledge on it project principles, risk management and business systems. one mentor highlighted the fact that the mentoring program was a good opportunity for female students to gain some insight into it as a career, as females are still outnumbered in the it industry. mentors can become role models for female students, offering general information and advice about the it industry and providing examples of the diversity of career paths within the industry and the variety of training options and skills that are considered valuable, all of which will positively promote careers in the it industry. female mentors, in particular, will project a strong positive image to female students and convey the fact that the it industry is no longer just a male domain. as this was a pilot program, there were problems encountered along the way. the main one was the lack of opportunity for students and mentors to meet before the program started. this would have provided a means of breaking the ice and perhaps facilitated communication and interaction during the program. some groups made little or no effort to contact their mentors, while on the other hand, some mentors did not respond to student emails. there were also problems associated with differing levels of expectation between students, teachers and mentors, and a lack of specific tasks within the assignment description to assist students to make full use of their mentor resource. surveys and interviews with students have been conducted to determine whether the mentoring program has been successful in influencing students’ perceptions of it education and work and feedback will also be ajis vol. 10 no. 2 may 2003 112 sought from teachers and mentors. many valuable lessons have been learnt from this study that can be used to produce guidelines for the establishment of future it mentoring programs. the investigators aim to continue the program at macgregor high school and expand it to include other high schools in the griffith university catchment area. discussion an important aspect of the mentoring project was its relevance to the gender issue. although it education is less attractive to students in general, females tend to be noticeably under-represented. this is due in part to the construction of the it as a male domain, whereas occupations such as teaching and nursing are considered more suitable for females. a mentoring program may provide female role models to dispel negative stereotypes and highlight the positive aspects of being a woman working in the it field. statistics on student applications to university and technical education level it courses indicate that it education is unattractive to the majority of high achieving students in general and female students in particular. prior research into high school and university students’ perceptions of it studies and work indicate not only considerable uncertainty about what the it work consists of and what skills are needed to succeed as an it professional, but also that it work is perceived as difficult, boring and solitary in nature, requiring little interaction with fellow workers or customers (greenhill, von hellens, nielsen et al. 1997; nielsen et al. 1999). as a result, female high school students lack interest in it degree studies and the number of female it graduates continues to be low, varying between 14%-20% of all students enrolled in it degree courses. the it industry figures are equally unsatisfactory. according to korn/ferry international, only one in ten board seats is occupied by a woman (burke 2000). the situation on australian government boards is better (30.9%) but still below the target figures (osw 2000). some companies have put initiatives in place to rectify the situation, e.g. ibm, where women made up 18% of their executive positions globally in 1999, whereas the figure was 10.5% in 1995, when they formed a women’s diversity task force (bensten 2000). this paper suggests that it mentoring could be a viable way to challenge the perceptions that females in it education have, and to make it a more attractive career option. conducting observations and an examination of the documentation is recommended to enable an insight into how domination and legitimation occurs through the use of power and sanctions. conclusion as with all social science research, this study involves what giddens refers to as ‘the double hermeneutic’ (giddens 1982 p.11-13). subjects can take up and reject or reaffirm the research on women in it. in this way research projects themselves may help to bring about change as more interviewees consider the implications of the questions they are asked. this is related to the idea of the meta level observer which is discussed in trauth et al. (2000). a limitation of the research so far is that it has relied on interviews and surveys with no data drawn from observation. overall, the participation of women in it remains low and this is a matter of concern, although there have been many success stories in attracting women into it education. this paper indicates that the way women talk about their work reinforces widely held impressions of the it industry and the use of structuration theory helps to show how this talk is not always consistent with the women’s lived experiences. mentoring is suggested in this paper as a way to challenge these dualism and structures of signification. through mentoring, interactions between professional it organisations, professional it women and females in it education can help to challenge these dualisms and therefore alter perceptions. in order to understand better how women help configure the institutional realm of it work, we propose that more qualitative studies of women at work in it as well as women talking about it are needed. it is suggested that a ‘proper’ application of structuration theory would require a longitudinal study. due to the dynamic nature of the it industry (unlike other, more traditional industries), a longitudinal study could give great insight into perceptions of women in it over time. references acker, j. (1990). "hierarchies, jobs, bodies: a theory of gendered organisations." gender and society 4(2): 139-158. ahuja, m.k. (2002). "women in the information technology profession: a literature review, synthesis and research agenda." european journal of information systems 11: 20-23. barrett, m. (1992). words and things: materialism and method in contemporary feminist analysis. destabislising theory: contemporary feminist debates. m. a. barrett and a. phillips. cambridge, polity press. ajis vol. 10 no. 2 may 2003 113 bensten, c. (2000). why women hate it: cover story, cio magazine. 2000. boland, r.j. (1996). "why shared meanings have no place in structuration theory: a reply to scapens and macintosh." accounting, organizations and society 21(7/8): 691-697. burke, f. (2000). women are coming on board but very slowly. the australian: 18. cohen, p. (1968). modern social theory. london, heinemann. council of economic advisors. (2000). opportunities and gender pay equity in new economic occupations. washington rep., may 11. [online] http://clinton3.nara.gov/wh/eop/cea/html/whitepapers.html encel, s. (1970). equality and authority. london, tavistock. giddens, a. (1979). central problems in social theory. berkeley, university of california press. giddens, a. (1982). profiles and critiques in social theory. london, macmillan. giddens, a. (1984). the consitution of society. cambridge, polity. greenhill, a., von hellens, l., nielsen, s.h. and pringle, r. (1997). australian women in it education: meanings and multiculturalism. 6th international ifip conference on women, work and computerization (wwc-97), bonn, germany. halford, s. and leonard, p. (2001). gender, power and organisations. london, palgrave. jones, m.r. (1999). structuration theory. rethinking management information systems. w. j. currie and r. galliers. oxford, oxford university press: 103-135. klein, h., k and myers, m.d. (1999). "a set of principles for conducting and evaluating interpretive field studies in information systems." mis quarterly 23(1): 67-93. lehmann, j.m. (1994). durkheim and women. lincoln, nebraska university press. maccallum, j. and beltman, s. (1999). international year of older person mentoring research project, murdoch university. maitland, a. (2001). a long-term solution to the it skills shortage. financial times: 9. nielsen, s.h., von hellens, l. and beekhuyzen, j.p., trauth, e. (2003). women talking about it work: duality or dualism? acm sigcpr/sigmis, philadelphia, pennsylvania, 10-12 april 2003. nielsen, s.h., von hellens, l., pringle, r. and greenhill, a. (1999). "students’ perceptions of information technology careers: conceptualising the influence of cultural and gender factors for it education." gates 5(1): 30-38. osw (2000). commonwealth office of the status of women (osw), resources: australian women. 2000. porter, e. (1991). women and moral identity. sydney, allen and unwin. pringle, r., nielsen, s.h., von hellens, l., greenhill, a. and parfitt, l. (2000). net gains: successful strategies for professional women in it. 7th international conference on women, work and computerization . charting the course to the future ifip tc9 wg 9.1, columbia, canada. qsr (1997). qsr nud*ist 4: user guide. melbourne, sage publications. todd, p.a., mckeen, j.d. and galllupe, r.b. (1995). "the evolution of is job skills: a content analysis of is job." mis quarterly 19(1): 1. trauth, e., nielsen, s.h. and von hellens, l. (2000). explaining the it gender gap: australian stories. 10th australasian conference on information systems, brisbane, australia. venkatesh, v., morris, m.g. and ackerman, p.l. (2000). "a longitudinal field investigation of gender differences in indiviudal technology adoption in decision-making processes." organizational behavior and human decision processes(83): 33-60. von hellens, l., nielsen, s.h. and trauth, e. (2001). breaking and entering the male domain. women in the it industry. acm sigcpr, san diego, usa. von hellens, l.a. and nielsen, s.n. (2001). "australian women in it." communications of the acm 44(7): 46-52. von hellens, l.a., pringle, r., nielsen, s.h. and greenhill, a. (2000). people, business and it skills: the perspective of women in the it industry. acm sigcpr, chicago, usa. walsham, g. (2002). "cross-cultural software production and use: a structural analysis." mis quarterly 26(4): 359-380. walsham, g. and han, c.k. (1990). structuration theory and information systems research. international conference on information systems. wit (2002). women in information technology wit home page/mission. [online] http://www.wit.org.au., 2002. ajis vol. 10 no. 2 may 2003 114 copyright jenine beekhuyzen; sue nielsen; liisa von hellens © 2003. the authors assign to auswit2003 and educational and non-profit institutions a non-exclusive licence to use this document for personal use and in courses of instruction provided that the article is used in full and this copyright statement is reproduced. the authors also grant a non-exclusive licence to auswit2003 to publish this document in the proceedings. those documents may be published on the world wide web, cd-rom, in printed form, and on mirror sites on the world wide web. any other usage is prohibited without the express permission of the authors. acknowledgements the authors gratefully acknowledge financial support from the australian research council large grants scheme which funded the collection of the interview data by the first two authors and professor eileen trauth. the authors are also grateful to professor trauth for her collaboration with the winit project and for allowing access to her interview data. aj1s vol. 8 no. 1 september 2000 evolution of web-based applications using domain-specific markup languages guntram graef, martin gaedke telecooperation office (teco), university of karlsruhe, vincenz-prlefinitz sir. 1, 76131 karlsruhe, germany, tel.: +49 (721) 6902-89, fax: -16, e-mail: {graej\gaedke}@teco.edu. url: http://www.teco.edu -orhttp://webengineering.org abstract the lifecycle of web-based applications is characterized by frequent changes to content, user interface, and functionality. updating content, improving the services provided to users, drives further development of a web-based application. the major goal for the success of a web-based application becomes therefore its evolution. though, development and maintenance of web-based applications suffers from the underlying document-based implementation model. a disciplined evolution of web-based applications requires the application of software engineering practice for systematic further development and reuse of software artifacts. in this contribution we suggest to adopt the component paradigm to development and evolution of web-based applications. the approach is based on a dedicated component-technology and component-software architecture. it allows abstracting from many technical aspects related to the web as an application platform by introducing domain specific markup languages. these languages allow the description of services, which represent domain-components in our web-component-software approach. domain experts with limited knowledge of technical details can therefore describe application functionality and the evolution of orthogonal aspects of the application can be de-coupled. the whole approach is based on xml to achieve the necessary standardization and economic efficiency for the use in real world projects. keywords: web engineering, webcomposition, domain component, domain engineering, evolution introduction by offering ubiquitous access to any kind of information and applications the world wide web (web) was able to establish itself as the dominant platform for the delivery of hypermedia applications. under the influence of increasing competition, especially in the area of electronic commerce, these applications are subject to constant change concerning their functionality, their application interfaces or the information they offer (cusumano & yoffie 1999). these applications, that we will also refer to as web-applications, can be strongly characterized by their underlying implementation model. due to the rapid speed of technological innovation common to the web the lifecycles of web-applications become very short. the applications have to undergo an evolutionary process that never stops. nevertheless, in the large majority of cases no disciplined approach is employed to address the increasing complexity of web application (barta & schranz 1998, gellersen & gaedke 1999). the insight that the development and evolution of web applications requires a dedicated support through models, methods and principles of software engineering, comparable to that employed during the development of traditional applications, seems to suggest that web application development should be based on a solid foundation of software engineering methodology. from a software engineering point of view the world wide web with its unique character is a new application domain (gaedke et al 1999a). this new discipline that during the last three years has established itself as web engineering offers both a cost reduction and an increase in quality during the development and evolution of webapplications (gaedke & rehse 2000). web engineering implicitly considers berners-lee's central demand for heterogenity of the system and autonomous administration of its resource (berners-lee 1990). this demand that we will refer to as the basic principles of the web is a major obstacle for current approaches to the development and maintenance of web-applications, which will become obvious in section 2 of this contribution. furthermore, a fine-grained and reuse-oriented implementation is necessary to allow for the federation of existing web-applications or application parts into new applications (gaedke & turowski 2000). the positive experiences with component-based software development and its advantages (mclure 1997, lim 1998, tracz 1995, szyperski 1997) make it desirable to be able to use a dedicated component technology for the development and evolution of web-applications. this is also a prerequisite to be able to fully take advantage of applying modem reuse oriented software engineering processes to web-technology. in the following sections of this contribution we will present a framework in which the evolution of a web-application can take place. in the third section we will describe the webcomposition markup language that can be used to describe web-applications as components. we will introduce the concept of domain specific markup languages in the fourth section and describe how it can be used to further 51 ajis vol. 8 no. 1 september 2000 facilitate the development and evolution of functionality for the web. a commercial application that is based on the described architecture will briefly be described in the fifth section. te last section contains a conclusion and an outlook on further research. evolution framework the requirements for a software system change as time goes by. it is obvious that many kinds of influences are responsible for this, such as new regulations, changes in corporate identity or an extension of functionality. such maintenance tasks are difficult to handle if the application has not been designed with the possibility of later changes and extensions in mind (gaedke & turowski 1999). to allow for a disciplined and manageable evolution of a web-application in the future it makes sense not to design the initial application on the basis of the concrete requirements identified at the start of the project. instead the initial application should be regarded as an empty application that is suitable for accommodating functionality within a clearly defined evolution space. this approach is based on domain engineering, which has been described as a process for creating a competence in application engineering for a family of similar systems (sei 1999). during an analysis phase the properties of an application domain are determined. during a design phase this information is transformed into a model for the application domain. from this the required evolution space can be determined and, during the implementation phase of the domain engineering process, the initial application can be constructed as a framework ready to accommodate any kind of functionality that lies within the evolution space of the domain. this view can be extended to several application domains. therefore the term basic evolution bus has been introduced in (gadeke & graef 2000) to describe the basic architecture of a web-application. the evolution bus is the initial application for all abstract application domains of a web-application. application domain application domain web-application evolution bus application domain application domain evolution by extending the domain set domain specific evolution by integrating domain specific services figure 1: dimensions of a web-application's evolution space it enables the management and collaboration of domain-components, i.e. components that implement specific application domains such as web-based procurement, reporting or user driven data exchange. these initial domain-components also represent prototypes for future domain-components of the same application domain. the evolution can take place in two clearly defined ways (figure 7): • domain specific evolution the extension of a domain through new domain-components, e.g. by prototyping an existing domain-component. another possibility is that the domain itself changes or that it receives more functionality, which requires the modification of the domain's initial component that serves as a prototype for other domain components. • evolution of the domain set the evolution of an application is also possible through the modification of the domain set. the extension of an application's functionality by adding a new application domain takes place e.g. when a shopping basket and corresponding functionality is 52 ajis vol. 8 no. 1 september 2000 added to a web-based product catalogue. the integration of a new domain is realized by connecting a new initial domain component to the evolution bus. a framework for implementing the evolution bus and the domain components can be developed with the webcomposition markup language that will be introduced in the following section. webcomposition-approach for component-based web engineering webcomposition component model the abstract webcomposition component model, introduced in (gellersen et al 1997), extends the implementation model of the web to circumvent its semantic limitations for the modeling of applications. it enables the development of components from which web-applications can be composed. such components are a code-abstraction of an arbitrary target language. via a mapping mechanism from the component model to the web implementation model it is possible to map components to web-resources that contain e.g. html, wml (wireless markup language) or scriptcode. in contrast to web-resources webcomposition components can encapsulate design artefacts of arbitrary granularity. this can range from simple html element properties such as the used font-type to complex business processes or the implementation of design patterns. it is also possible to create a new component as a composition of existing components. the webcomposition component model offers object-oriented semantics that can be employed to define relations between components such as inheritance, aggregation and polymorphism. in contrast to class based languages such as java or c++ webcomposition uses prototype instance inheritance as described in (ungar & smith 1987). hereby, new variants can be created from prototypes that can be modified by overloading individual properties. during the description of a new component each existing component of the model can be referred to as a prototype. the webcomposition component model addresses the problems related to reuse on the web and the mapping of fine-grained design entities to a document based implementation model by supporting the implementation of a web-application within a fine-grained component model. the evolution of web-applications takes place through the manipulation or the addition of components that are kept persistent and accessible in a component store during their whole lifecycle. webcomposition markup language the webcomposition approach suggests the webcomposition markup language (wcml) to support the development of components. this language has already been introduced in (gaedke et al 1999b) and will thus only be briefly described here. wcml is an application of the extensible markup language (xml) and therefore inherits some of its advantages. the syntax of wcml documents can easily be checked against a document type definition (dtd). wcml is platform independent and easy to process. the development of tools is strongly facilitated through the availability of xmlparsers and libraries in the public domain. furthermore a variety of existing tools can be used such as xml-editors. for a more detailed language description and further information the reader may refer to (gaedke 1999). components can be stored in a component store. this can be a database, but file-systems or webservers can also serve as component stores. via an uri-addressing scheme several component stores can be integrated into virtual component stores to allow for component reuse beyond a single component store. with the help of web-servers and virtual component stores, components can be directly reused worldwide via the internet. in (gaedke et al 2000) a compiler has been introduced that performs a mapping of wcml-components to web-resources, as visualized in figure 2. the compiler uses a freely available xml parser component and employs it to read wcml components from a virtual component store, to create a parse-tree and to perform a syntax check against a wcml dtd. then the compiler resolves all references and inheritance relationships between wcml components. finally the presentation methods of the components are executed and the required target resources are generated. this completes the mapping to the implementation model of the web (or another target platform). 53 ajis vol. 8 no. 1 september 2000 rle system database virtual component store <wcml> <component> <protype is="maincontent"/> </component> </wcml> xml beschreibung figure 2: use of the wcml-compiler description of services with wcml considering the business applications accessible via web-technology one can state that in general applications offer a set of services to a user in order to support him during various tasks. examples are the retrieval of information, the ordering of products or the issuing of a complaint. in a broader context via each service an enterprise offers a product to its customers that is not necessarily limited to the scope of the web-application system. it could also involve existing business applications or the whole enterprise. therefore in (gaedke et al 1999a) the concept of services as entities for modelling web-applications has been described in more detail. in that contribution a service is described as consisting of several components. first, a service contains information that an organization associates with it. this content is usually available in a media representation such as text or graphics. second, a service contains layout and navigation that are used to make the service content available. it also contains directives describing the user-interaction and the consequences triggered by user-actions. another important part of a service is the implementation of the business process i.e. the technical description of the interaction with the business application systems such as enterprise resource planning systems. technically a service can be represented as an object with an inner state that can be manipulated by the user. the audiovisual representation during run-time and the user-interface of a service are determined by web pages. the definition of the process is contained in process components in the back-end. data, presentation and process are separated. thus it is e.g. possible to accommodate different humanmachine or machine-machine interaction modalities by simply exchanging the presentation. in (gaedke & turowski 2000) it has been described how the most important aspects of a service can be encapsulated within several wcml components: • service content: a wcml component describing the properties and information content of a service (e.g. textual content). • service layout and navigation: wcml components defining with which layout a service should be displayed and how the information should be made available to the user. this includes how navigation between pages should take place. an example would be a component that implements the web design pattern decorator to display the service according to the corporate identity of an organization. furthermore the web design pattern guided tour may be used to provide a guided navigation through the pages of a service. • service user-interaction: a wcml component that controls the interaction process between a user and the service. this controls the access to the service content by the user-interface. • service processing: a sequence of calls to components to perform further processing in business application systems. these calls reflect the automated part of the business process. they use a middleware layer within the application framework to access (legacy) application systems. a service can be composed from these components and therefore in itself represents a component. a service can also serve as a prototype. this makes a service reusable by inheritance. 54 aj1s vol. 8 no. 1 september 2000 service example: ordering mobile phones the following example clarifies the definition of services with wcml. a service-component shall offer a product (in this case a mobile phone) for sale. the service must describe the phone (content), it must display the information in a browser (layout and navigation) and it must define the ordering process for the telephone (process). each wcml component can be identified via a unique name (uuid). the description of a service for selling mobile phones (sellmobilephone) is based on a component for the description of standard products. the component standardproduct (figure 3) defines the properties common to all products. a property can be an attribute or a parameterless function of the component. <?xmlversion='1.0' ?> <!doctype wcml system "wcm!2.dtd"> <wcml> <component uuid='standardproduct'> <propertyname='currency'>eur</property> <propertyname='unitname'>piece</property> <propertyname='unitnameplural'>pieces</property> <propertyname='quantity'>l</property> • •• </component> figure 3: content component for products now this component can be used as a prototype by the content component of the sellmobilephone service to add all data relevant for the mobile phone (figure 4). <component uuid='cellphonemodel 123'> <prototype is='standardproduct'/> <property name='bandinfo'>dual</property> <property name='price'>229.99</property> </component> figure 4: content component for a certain product the component cellphonemodell23 assumes the properties of the prototype through inheritance from standardproduct. with the inherited properties e.g. the currency of the provided price of 229.99 is now clearly defined as in euro (eur). the component userlnteraction234 describes the user-interaction. the code extract in figure 6 shows a simple user-interaction for the input of the quantity of a product order. the example makes use of ordered multiple-inheritance in wcml to access several prototype components at once. 55 ajis vol. 8 no. 1 september 2000 <componentuuid='userinteraction234'> <prototype is='user!nteraction'/> <prototype is^numericinputfield1^ <propertyname='datafield'>quantity</property> <property name='minvalue'>k/property> <propertyname='maxvalue'>99</property> ••• </component> figure 5: user-interaction component the presentation components and process components are also defined in wcml. all components are now composed to a service by a service component (figure 6). <component uuid='sellmobilephone'> <prototypeis='cellphonemodell23'/> <prototype is='userinteractionl23'/> <prototype is='layout'/> <prototype is='navigation'/> <prototype is='orderprocess'/> </component> ••• </wcml> figure 6: service component by exchanging the content component cellphonemodell23 the same service could be used for selling other types of mobile phones. by changing the process component the underlying business process could be changed, e.g. to offer the phone in a bundling action. by changing the layout component or the navigation component the presentation of the service can be modified. the processing of the business data that results from use of the various services takes place within the business application systems of the organization. these can be heterogeneous environments with several legacy systems. within the application framework each system is represented by an agent. the communication with these applications must take place via a simple and standardized distributed mechanism to make the services built above this mechanism reusable. for the business process a business-to-business communication between the applications of several organizations might be necessary. a suitable approach based on widely accepted communication standards has already been introduced in (gaedke & turowski 2000). domain-specific markup languages to further facilitate the domain specific evolution of an application we are going to introduce the concept of service domain markup languages (sdmls). sdmls are domain specific languages based on xml. they are used to standardize domain components and to support the interaction between domain components and the evolution bus. an sdml abstracts from those properties of a service common to all services of a service domain. it serves as a tool to easily specify individual services in the context of a service domain. the semantics and complexity of such a language corresponds exactly to the decision space available to a service developer working within a service domain. figure 7 shows an extract from the document type definition of a very simple sdml for the domain "product ordering" in the context of an existing e-commerce system. the labelling of the language elements (tags) has been based directly on the vocabulary used by experts in the corresponding domain. 56 ajis vol. 8 no. 1 september 2000 the language enables a domain expert without advanced programming knowledge to describe all aspects of a specific product ordering service. a service developer can describe the structure and content of an order via an order object (order) while he can define the user-interaction in the service_form part of the sdml document tree. with the message tag the developer can define a confirmation message displayed after the order has been accepted. ^sdml.dld -xml authority hs e3 ft!-t€d! .view. tdotec,v*idow help ' \^-'" .'"' '"• t • • • • ; . . ' r ' $ : '-" v ', v' • ' •' . ' • . • ' " • • ' • • • ;-: sdm. j — p order wfi—_«^?el?.«| osid7j name ? j shcf?t_descript10n? j information? j price ?j cijrrency.cooe? ]̂ tax? j quantity? j order_by? j oroer.for?^ bju-.to?,.,! type?y| — order line" <>! .. i i : i — oispj typej quantity j name j short_description (.j information j supplier j price j currency_cooe j tax j 1 — total j —( message ? s r— olsid?w| fexiname? !̂ type? j p?j value 7 j fadsize?^ maxlenoth?j columns? j rows?w| event ?j script ?j store value ? <,| naj.count?^ sge7«>| itemj — disabled? j figure 7: document type definition of an sdml figure 8 shows an extract of a service description for a concrete product ordering service for mobile phones. in the first part general properties of the service and the associated order such as service name, standard price and value added tax rate are defined. then various order lines are described within the order_line tag. the code example shows an order line for dl-telephone cards that can be included as accessories with a mobile phone order. inside the service_form tag the various user-interface elements are defined in the same order, as they will be displayed later. the example code given describes a text input field whose content is linked to the quantity-property of the order. 57 ajis vol. 8 no. 1 september 2000 <?xml version-1.0' encoding='iso-8859-l'?> <?xml:stylesheet type="text/xsl" href="servicecompiler.xsl" ? <!doctype sdml system "orderservice_sdml.dtd"> <sdml> <order> <guspid> <osid> <name> <short_description> <information> <price> <currency_code> <tax> <quantity> <order_by> <order_for> <bill_to> <type> </guspid> </osid> </name> <string>handyoiderservice</string> <string>handyosid</string> <html>handy ordeiservice</htmi> <html>service for ordering mobile phones</htmlx/short_description> <html>if you have any questions call 5214</html>^information> <number>199.00</number> </price> <string>eur</string> </currency_code> <number>16.00</number> <ftax> <number>1.00</number> </quantity> <variable>useriix/variable> </order_by> <variable>useriix/variable> </order_for> <variable>useriexvariable> </bill_to> <string></string> </type> <order_lrne> <olsid> <type> <quantity> <name> <short_description> <informat1on> <supplier> <price> <currency_code> <tax> <total> </order_line> <order_line> </order_line> </order> <service_form> <widget> <olsid> <fieldname> <type> <string>olsid-d1</string> <stringx/string> <number> 1 </number> <htmi>d 1 -card</html> <html>d1-telephone card</html> <htmlx/htmi> <string>deutsche telekom</string> <number>29.0</number> <string>eur</string> <number>16.0</number> <number>33.64</number> </olsid> </type> </quantity> </name> </short_description> </information> </supplier> </price> </currency_code> </tax> <value> <fieldsize> <maxlength> </widget> <widget> </widget> </service_form> <message>danke fur ihre bestellung!</message> </sdml> <string></string> </olsid> <string>quantity</string> </fieldname> <string>textfield</string> <ntpe> <string>handyosid.quantity</string></id> <string>1</string> </value> <string>5</string> </fieldsize> <string>4</string> <maxlength> figure 8: sdml description of an order service in this example the service developer can modify service content (order section) and user-interaction (service_form section). layout, navigation, certain aspects of content and user-interaction and the processing are the same for all services of this service domain and are therefore not part of the sdml. they are encapsulated within wcml-components of the framework and are maintained by experts such as web-designers and web-engineers. in other service domains the decision space might be quite different though. the sdml for another domain may e.g. contain processing statements or layout properties. service factory a service factory does the transformation of a service description into a functioning service. the concept of a service factory is based on the factory design pattern by (gamma et al 1995) and has been introduced in (gaedke et al 1999a). in our case a mapping is performed from an sdml to a service 58 aj1s vol. 8 no. 1 september 2000 component in wcml. that generated service component usually is tightly integrated with existing framework components via inheritance and composition operators. the mapping rules for a single sdml are encapsulated in a factory method that is selected and invoked from a control function. because both sdmls as well as wcml are based on xml such a factory method is easily implemented with either the extensible stylesheet language (xsl) [21] or by using special wcml language constructs for the definition of factory methods [22]. in both cases using xml as the underlying technology significantly contributes to the efficiency and flexibility of the described approach. virtual component store ^ sdml document ^ service editor sdml document wcml components service factory wcml components figure 9: system architecture domain-specific evolution domain-specific evolution takes place by the description of new services with an sdml and by the extension of an sdml itself together with the underlying wcml component framework. an example would be the extension of the previously described sdml for order services by a multi-lingual order confirmation. various sub tags could extend the sdml, e.g. the message tag and the framework component for the visualization of order confirmations could be extended with multi-lingual support. the factory method for the sdml would have to be extended by xsl mapping templates for the new xml tags added to the sdml. 59 ajis vol. 8 no. 1 september 2000 -n f ac to ry c on tro l f un ct io n =* order service factory method information service factory method • • • figure 10: internal architecture of the service factory evolution of the domain set developing a new application domain can extend the domain set. this is done by defining a new sdml and by extending the wcml component framework by adding a new service prototype component. to enable the mapping of sdml service descriptions to wcml components an xsl style sheet that implements a factory method for the service factory has to be added. tools using xml based markup languages for describing services facilitates the development of special development tools. various tools and libraries for dealing with xml documents are publicly available and xml itself provides a clearly defined structure that forms the basis for easy manipulation of documents. figure 11 shows an editor that can be used to visually program order services. it is based on the sdml introduced in the previous example. the editor maps the sdml bijectively to the graphical i/o-elements of the user-interface. it has been developed in java using an xml parser component that is freely available. such an editor provides a domain expert with a tool that allows her to develop simple services without sophisticated programming knowledge. a service developed with such a tool can also be used as a starting point for a web-engineer who may use the generated wcml components and extend them to develop a new service leaving the boundaries of the current domain. figure 9 shows how a service editor and a service factory work together with the system components. the exchange of design artefacts between the various instances takes place indirectly via the virtual component store that stores wcml components and a repository for sdml service descriptions. 60 ajis vol. 8 no. 1 september 2000 le'seivice facloiy |d:wvh<iridy180699b xml he efbt add sendee help 1 [sl̂ lhl1 1&~ hi ' isishimp i y i — 1 rage i &; j q radiobutton (olsid-d1)[kartej|p q radiobutton (olsld-d2)[karte) '.!,-•: q radiobutton (olsid-ep)[karte) | j:!i; r^ t lib! , [j checkbox (olsid-lt) 1 1| i q <coolbreak/> (paoebreak-1) j j| q selectfleld (olsid-handy) j| } q none (olsid-html-code) pjjjj: q unknown (olsid-mso) !;|| i 11 'c* k^ i p im l ... , 1 y " ". . •:*;; / . • • •• ĵ bhigh • • ' : fpropertfes (description) î swss ' " • j service productld. . (niustbeequeltodbid) i service name i . i short description i price-'.•'.''• ' . '•••' \ . • . , < • • • • • . ' j currency code. tax ' ' '." -•r^-' .s j chacqevatue of hew service ' ...„ — .. . .._, .., bee! '.: . " ' ' ' : handysld mobile phone order service select, configure and order mobile phoi; 199.0 ; . i fbjr 160 ' ' cancel ; — • ; / -. . figure 11: visual editor for order services application of the approach in a joint project between the telecooperation office (teco) at the university of karlsruhe and hewlett-packard (hp) a service-oriented e-commerce application, eurovictor, has been developed based on the webcomposition component model and domain-specific markup languages. the aim of the project was to develop a support system that enables development, management, maintenance and evolution of services within the heterogeneous environment of the european intranet of hp. an evolution bus was realized as part of the project. the evolution of the application took place through the integration of several domain specific services such as presentation of information, software orders or product purchase. these services serve as prototypes for the domain specific evolution of the application. figure 12 shows the web-application's start page. the left part contains a menu that allows for a selection of services. in the middle and on the right side two special services can be found: a service for the adaptation of the application to the behaviour of the current user and another service offering shopping basket functionality. the adaptation service demonstrates the flexibility the application gained due to the domain-component specific architecture it is based on. a large part of the domain specific evolution of the application has been triggered by its international scope. as an example new services are constructed through inheritance that are adapted to national layouts, languages or legal regulations. meanwhile in almost all european countries services are developed in a distributed and decentralized manner and made available via the eurovictor system. 61 ajis vol. 8 no. 1 september 2000 * victor home o ̂ ^ hardware order » software / accounts w telecommunication ^ telephone * installation * relocation + removal ^ telephone-display * change "r call redirection » new * add * delete ^ voice-mail * new * relocate * delete mobile-/homeoffice additional services news profile about hello guntram your last login was at 9/24/9910:11:02 am changes to your application recently you have used the services telephone: installation and yojgb-maillnbw. we added the following services to your application that might be related: telephone; relocation telephone: removal telephone-display: change gajl redirection^ new call redirection: add gall redirection^deleje voice-mail; relocate voice-mail: delete some services were removed from your application. click for more information. visibility of service shop^ssmant^content has been increased due to frequent use. visibility of service news: company stories, has been decreased due to infrequent use. el̂ , shop assistant your basket currently contains i item, copyright © 1«w hewtett-fedun! company j j figure 12: the euro victor system conclusion the web has been established as a major platform for applications, although the underlying implementation model complicates the development and evolution of web-applications. to manage the evolution of complex web-applications during their complete lifecycle it is necessary to use a development model that allows us to compose applications from reusable components of moderate complexity and arbitrary granularity. the webcomposition component model with the language wcml is suitable for that purpose. we further facilitate the development and evolution of web-applications by introducing domain specific markup languages for web-services and the related mechanism of a service factory. services are based on initial domain components that each serve as a prototype for a service domain. the prototypes of all service domains are based on an evolution bus as a basic framework. a webapplication can evolve within this evolution bus in two different ways. first, an evolution can take place within a service domain by modifying the initial service component or by adding a new service component to the component set of a domain usually by proto-typing. second, the domain set itself can evolve by adding new initial domain components and connecting them to the evolution bus. different aspects of a web-application can be separated and assigned to people with different roles working on different levels of abstraction. this ranges from the visual programming of a simple service by a domain expert to the modification of an initial domain component by a web-engineer. the approach described in this contribution has been successfully applied to a large and internationally distributed web-application at hewlett-packard. acknowledgements an earlier version of this paper was presented at the first xml conference (xml2000), heidelberg, germany, may 3-4, 2000. comments from participants of this conference are gratefully acknowledged. the authors would further like to thank u. stegemiiller of the hewlett-packard corporation, germany for their cooperation, thoughtful feedback, and for providing inspiration for some of the ideas developed in the approach presented here. 62 ajis vol. 8 no. 1 september 2000 examples wcml-compiler and code examples are available at http://webengineering.org references barta r.a. and m. w. schranz, "jessica: an object-oriented hypermedia publishing processor," computer networks and isdn systems, vol. 30(1998), pp. 239-249, 1998. berners-lee t., "information management: a proposal," , vol. 1998: cern, 1990. cusumano m.a. and d. b. yoffie, "software development on internet time," ieee computer, vol. 32, pp. 60-69, 1999. gaedke m., h.-w. gellersen, a. schmidt, u. stegemuller, and w. kurr, "object-oriented web engineering for large-scale web service management," presented at thirty-second annual hawaii international conference on system sciences (hicss-32), island of maui, usa, 1999a. gaedke m. and j. rehse, "supporting compositional reuse in component-based web engineering," presented at 2000 acm symposium on applied computing (sac 2000), villa olmo, como, italy, 2000. gaedke m. and k. turowski, "integrating web-based e-commerce applications with business application systems," netnomics journal, baltzer science publishers, vol. 2, pp. 117-138, 2000. gaedke m. and k. turowski, "framework for maintaining evolution of e-commerce applications in the web," presented at 12th international conference software and systems engineering and their applications (icssea '99), paris, france, 1999. gaedke m. and g. graef, "development and evolution of web-applications using the webcomposition process model," presented at 2nd web engineering workshop at the 9th international world wide web conference (www9-webe), amsterdam, the netherlands, 2000. gaedke m., d. schempf, and h.-w. gellersen, "wcml: an enabling technology for the reuse in object-oriented web engineering," presented at poster-proceedings of the 8th international world wide web conference (www8), toronto, ontario, canada, 1999. gaedke m., "webengineering.org homepage," , vol. 2000, 1999. gaedke m., c. segor, and h.-w. gellersen, "wcml: paving the way for reuse in object-oriented web engineering," presented at 2000 acm symposium on applied computing (sac 2000), villa olmo, como, italy, 2000. gamma e., r. helm, r. johnson, and j. vlissides, design patterns: elements of reusable objectoriented software. reading, mass.: addison-wesley, 1995. gellersen h.w. and m. gaedke, "object-oriented web application development," ieee internet computing, vol. 3, pp. 60-68, 1999. h.-w. gellersen, r. wicke, and m. gaedke, "webcompostion: an object-oriented support system for the web engineering lifecycle," computer networks and isdn systems, vol. 29, pp. 1429-1437, 1997. graef g. and m. gaedke, "an evolution-oriented architecture for web applications," presented at second nordic workshop on software architecture (nosa'99), ronneby, sweden, 1999. lim w.c., managing software reuse : a comprehensive guide to strategically reengineering the organization for reusable components. upper saddle river, nj: prentice hall, 1998. mcclure c.l., software reuse techniques : adding reuse to the system development process. upper saddle river, n.j.: prentice hall, 1997. tracz w, confessions of a used program salesman : institutionalizing software reuse. reading, mass.: addison-wesley pub. co., 1995. szyperski c., component software: beyond object-oriented programming. reading, mass.: acm press; addison-wesley, 1997. sei, "domain engineering," , vol. 1999: software engineering institute, carnegie mellon university, 1999. ungar d. and r. b. smith, "self: the power of simplicity," presented at oopsla '87, 1987. segor c. and m. gaedke, "crossing the gap from design to implementation in web-application development," presented at information resources management association international conference, anchorage, alaska, usa, 2000. 63 ajis vol 9 no. 1 september 2001 a balanced approach to capturing user requirements in business-toconsumer web information systems m. s. lane a. koronios department of information systems university of southern queensland toowoomba, australia email: lanem@usq.edu.au abstract the development of business-to-consumer web information systems pose special challenges in the requirements analysis phase. it is difficult to capture user requirements given that users are relatively autonomous and anonymous and there are no major incentives for users to become involved in the development of a web information system. the researchers reviewed traditional requirement elicitation techniques, marketing research techniques and web usage analysis techniques. current practice was assessed and the findings suggest that a balanced approach to user requirements capture will result in more complete and user centred requirements. this approach should lead to more effective business-to-consumer web information systems. keywords requirements elicitation, requirements capture, requirements analysis, business-to-consumer electronic commerce; web information systems introduction the growth of the world wide web has seen the emergence of web based electronic commerce. the development of effective mass information systems such as web information systems is critical to the success of web based electronic commerce (hansen 1995). moreover, capturing user requirements is particularly important in the development of web information systems where user acceptance is often critical to the success of a web information system (hansen 1995; bauer & chong 1999). this paper focuses on the capture of user requirements in the development of business-to-consumer web information systems. a review of requirements elicitation techniques, marketing research techniques and web usage analysis is conducted to evaluate the suitability of these techniques in the development of business-toconsumer web information systems. the paper proposes that a balanced approach to the capture of user requirements should result in more complete user requirements and the development of effective business-toconsumer web information systems. the paper is structured as follows. first, business-to-consumer web information systems are defined. then, the difficulty in capturing user requirements in these systems is highlighted. next, traditional requirement elicitation techniques, marketing research techniques and web analysis techniques are assessed and the researchers argue for a balanced approach to the capture of user requirements that combines these techniques. then, three cases are reviewed to provide a preliminary assessment of current practice's approach to the capture of user requirements in business-to-consumer web information systems. finally there are some concluding remarks for research and practice, and the researchers outline plans for future work. business-to-consumer web information systems business-to-consumer web information systems are information systems based on web technology that support consumer interaction such as on-line shopping through integration with databases and transaction processing systems (isakowitz, bieber & vitali 1998). these systems rely on the hypertext functionality and transfer mechanisms of the world wide web and are characterised by interactivity, dynamic updating, hypertextuality and global presence (scharl & brandtweiner 1998). in general, web information systems because of their open system platform look set to be the dominant information system for the next decade (lyytinen, rose & welke 1998). 61 ajis vol 9 no. 1 september 2001 challenge in capturing user requirements while much of the traditional information system development approaches are applicable to business-toconsumer web information systems, the requirements analysis phase poses special challenges for these types of systems (hansen 1995; poulymenakou, drakos, papazafeiropoulou & doukidis 1998; teubner & klein 1998). the increased organisational scope of business-to-consumer web information systems means that users are relatively anonymous and autonomous. the capture of user requirements is further complicated by the potential diversity that can exist in users as a key stakeholder group. for instance, users may vary from first time customers, regular customers, dissatisfied customers to the public at large. so how does one capture the requirements of users given that use is voluntary, the user is not readily accessible in person and there are no major incentives for the user to become involved in the development process? in order to address this question, the researchers reviewed current knowledge in order to identify and assess the suitability of existing requirements capture methods, techniques and technologies. traditional methods and techniques for capturing user requirements the researchers assessed a number of studies that have synthesised past requirements elicitation methodologies and techniques (fazlollahi & tanniru 1991; byrd, cossick & zmud 1992). fazlollahi and tanniru (1991) revised davis's (1982) contingency approach to the selection of a requirements elicitation methodology. the contingency approach is based on a three stage process. the first stage involves determining the level of uncertainty and the level of equivocality that exists in a system. technology, organisational context and external environment can affect the level of uncertainty and equivocality that exists in a system. the completion of this stage provides guidance in selecting the appropriate information acquisition strategy to capture the information requirements. and finally a requirements elicitation methodology is chosen to implement the selected information acquisition strategy and capture the necessary information (fazlollahi & tanniru 1991). in the revised contingency approach, the information acquisition strategies are mapped on the appropriate requirements elicitation methodologies. these methodologies are classified into the following approaches: asking the user, understanding the current system, the utilising system and experimentation approaches. the elicitation methodologies and information acquisition strategies are summarised in table 1. requirements elicitation methodologies asking the users closed questions open questions brain storming guided brain storming group consensus interviewing understanding current system documentation analysis utilising system normative analysis strategy set transformation critical factors process analysis ends/means analysis decision analysis socioeconomic analysis process analysis experimentation prototyping or heuristic methods information acquisition strategies asking the users special reports (controlled response) special reports (uncontrolled response) group meeting group meeting or integrators group meeting or integrators direct contact understanding the current situation rules and regulations; information systems utilising system information systems analysis; rule and regulations direct contact, group meeting; integrators planning; special reports, direct contact planning, information analysis, special reports planning, direct contact; information systems analysis direct contact; information systems analysis information systems analysis; rules and regulations information systems analysis; rules and regulations experimentation direct contact; planning uncertainty/ equivocality high equivocality t uncertainty equivocality continuum 4, high uncertainty table 1 mapping of requirements elicitation methodologies and information acquisition strategies (source: adapted from fazlollahi & tanniru 1991, p. 299) the level of uncertainty and equivocality existing in the problem domain of a system determines the suitability of different types of requirements elicitation methodologies. asking the user is more appropriate in situations where high levels of equivocality exists while experimenting with prototypes is more appropriate in situations where more information is required to reduce uncertainty. byrd et al. (1992) provided a comprehensive synthesis of requirements analysis techniques. the requirements 62 ajis vol 9 no. 1 september 2001 analysis techniques were classified into five distinction categories: observation techniques, unstructured elicitation techniques, mapping techniques, formal analysis techniques and structured elicitation techniques. observation techniques consist of behaviour analysis, protocol analysis and prototyping. unstructured elicitation techniques consist of teach-back interviewing, open interviews, brainstorming using the collective decisionmaking approach and the goal-oriented approach. mapping techniques consist of multi-dimensional scaling, cognitive mapping, and variance analysis. formal analysis techniques include techniques such as text analysis and repertory grids while structured elicitation techniques include techniques such as: card sort, scenario analysis, structured interviewing, critical success factors and future analysis. the different requirements elicitation techniques that fell within these five categories were mapped onto the problem domain, locus of control and communication obstacles in which these requirements elicitation techniques might be used most effectively. the mapping of these five categories is presented in table 2 and a discussion of the problem domain, locus of control, and communication obstacles follows. problem domain legend : ir = information requirements; pu = process understanding; bu = behaviour understanding; pfu = problem frame understanding technique category observation unstructured elicitation mapping formal analysis structured elicitation requirements elicitation method and problem domain behaviour analysis (ir, pu,bu) protocol analysis (pu, bu) prototyping (ir, pu) teachback interview (bu, pfu) open interview (ir, pu, bu, pfu) brainstorming collective decision making (pu, pfu) goal oriented approach (bu, pfu) multi-dimensional scaling (bu) cognitive mapping (pu, bu) variance analysis (pu, bu, pfu) machine rule induction (pu, pfu) text analysis (pfu) repertory grid (pu, bu) card sort (bu) scenario technique (pu, bu, pfu) structured interview (ir, pu, bu, pfu) critical success factors (ir, pu, bu, pfu) future analysis (pu, bu, pfu) locus of control user driven user driven analyst driven user driven user driven user driven analyst driven analyst driven user driven analyst driven machine driven analyst driven analyst driven analyst driven user driven analyst driven analyst driven analyst driven communication obstacle between between within/between between/among between within/among within/among between/among within/between between/among between/among between within/between within/between within/among between within/between/among within/between/among table 2 categorisation scheme for requirements elicitation methodologies (source: adapted from byrd et al. 1992) problem domain categories. many different elicitation techniques are used to gain understanding of the domain problem. in order to compare different elicitation techniques, these techniques have been grouped into the following problem domain categories (byrd et al. 1992): information requirements, process understanding, behaviour understanding, and problem frame understanding. information requirements is concerned with the information requirements of the proposed information system such as the information to be displayed to the user in soft copy (visually presented on screen) or hard copy (printed reports). process understanding is concerned with an analysis of the business processes that will affect or be affected by the proposed information system. this would include the facts, rules, beliefs, algorithms and procedures pertinent to business processes, factors that may inhibit design, development and implement of the proposed information system. behaviour understanding focuses on the dynamic nature of data and knowledge and the need to analyse and understand events in the environment that impact on data and knowledge recorded in the proposed information system (byrd et al. 1992). behaviour understanding involves mental models and operational models which are abstractions of problem domain from both user perspective and input/output processes perspective. problem frame understanding is concerned with understanding the overall context in which the system is being developed. for instance, the business objectives and goals that the proposed information system should address and the existing technological environment which will be used to support the proposed information system. locus of control in the requirements engineering process can be user driven, analyst driven, joint participation 63 ajis vol 9 no. 1 september 2001 (user, analyst), or automated (byrd et al. 1992). user driven means that requirements elicitation process is driven by the user. typically, the requirements elicitation process was driven by the system analyst. in more recent times there has been a move toward joint participation in requirement elicitation process in information systems that are socio-technical in nature (byrd et al. 1992). information systems that are socio-technical in nature imply the involvement of people in the day-to-day operations of an information system. there has also been some progress in the automation of the requirements elicitation process in certain contexts. for instance, the requirements of users in web information systems can be derived from the analysis of the web server log files. overall, there is a tendency for locus of control to be analyst driven in the development of web information systems where the analyst/web developer builds a system from client specifications without any input from the users (loh & pendse 1997). communication obstacles consist of three different obstacles: within, between and among. these three obstacles can inhibit the communication process in requirements elicitation. within obstacles involve the cognitive limitations of humans as information processors and problem solvers (davis 1982; valusek & fryback 1985). people specifying information requirements are subject to limited memory and recall as well as information processing biases such as selective perception or representativeness. elicitation techniques that focus on within communication obstacles concentrate on enhancing recall and understanding a user's innate information processing activities and decision processes. between obstacles. cognitive limitations also contribute to communication difficulties between system analysts and end users. these communication difficulties are referred to as between obstacles. the communication problems are not only caused by individual cognitive limitations but also by a lack of common language (davis 1982; valusek & fryback 1985). between obstacles are addressed by elicitation techniques designed to aid the participants communication process. such techniques are often an attempt to create a mutually understood context or provide a means to otherwise facilitate die interaction process. much of the anecdotal evidence suggests that there is little involvement of the users in the development of web information systems. communication between users and web developers is a neglected issue in the current web information systems development practices, but this is understandable because of the difficulties in involving users in the development process. among obstacles. even if these previous obstacles ate addressed one may still be confronted with balancing the needs of multiple stakeholders (byrd et al. 1992). a system designed for multiple stakeholders must attempt to accommodate each stakeholder's needs which can be in conflict with other stakeholders' needs or be in competition for limited system design resources. among obstacles require tradeoffs and weighting of multiple needs after determining individual requirements. the requirement elicitation techniques that address among obstacles incorporate the varying views of distinct stakeholders. these techniques also provide mechanisms by which stakeholders can negotiate and reach an consensus regarding a collective set of needs. the diversity and large number of stakeholders involved both directly and indirectly with a web information system means that among obstacles is important issue in the management of the requirements of a business-to-consumer web information system. managing the requirements elicitation process methodologies. the use of methodologies in requirements engineering is problematic in that there are a number of issues to consider in applying a suitable requirements engineering methodology. firstly, there are different categories of problem domains, which will require different requirements analysis techniques. secondly, the locus of control will vary depending on the technique(s) used and communication obstacles that are encountered (byrd et al. 1992). technical, organisational and environment aspects of the problem domain of a proposed information system will influence the level of uncertainty and equivocality confronting system developers (fazlollahi & tanniru 1991). the choice of the appropriate requirements elicitation methodology and accompanying information acquisition strategy will be dependent on the above mentioned factors. market approach to capturing user requirements more recently researchers have suggested that marketing research techniques provide a variety of approaches that address the difficulty in capturing user requirements in web information systems (hansen 1995; bauer & chong 1999). these include collection, processing and analysis of market data from primary and secondary sources to identify customer needs; prototyping; observation of and interviews with test users; test markets; statistical analysis of usage data; and periodical polls with questionnaires (hansen 1995). table 3 provides an assessment of the suitability of the marketing research techniques in terms of feasibility and cost for web information system development. the assessment of the different marketing research techniques is a subjective evaluation of their effectiveness in gathering the user requirements in the development of business-to-consumer web information systems. 64 ajis vol 9 no. 1 september 2001 techniques feasibility cost exploratory techniques interviewing techniques projective techniques observational techniques medium high medium high high medium problem solving research techniques longitudinal consumer panel mail surveys telephone surveys personal interviews on-line surveys personal observation mechanical and unobtrusive observation medium high medium low high low high low medium high high low high low table 3 assessment of the suitability of marketing research techniques for web site development (source: adopted from bauer & chong 1999, p. 421) similar to fazlollahi and tanniru 's (1991) classification, exploratory techniques are more suited to dealing with situations of high equivocality which require richer and not necessarily more information while problem solving research techniques are more suited to situations of high uncertainty requiring more information. there is considerable overlap between the requirements elicitation methodologies discussed in the previous section and the marketing research techniques that have been highlighted in table 3. it would appear that the use of requirements elicitation techniques that involve asking the user have been neglected due to the evolution of web information systems from a grass roots and spontaneous effort and the relative autonomy and anonymity of the users. mechanical observation relies on non-human devices to record people's behaviour. the internet and the web provide the opportunity to collect more information about the user than many realise. nevertheless the potential for collecting data using mechanical observation has been neglected and/or under-estimated by many firms (bauer & chong 1999). capturing user requirements with web analysis tools web analysis tools provide a valid way to capture and analyse user behaviour and activity in web information systems. these tools allow web developers to draw inferences from the analysis of user activity and behaviour on-line which can be used to develop and evaluate user requirements. technologies such as cookies, log-file statistics or click stream analysis can provide a clear picture of user behaviour right down to the individual. user information collected on-line tends to be more useful for the re-development and evolution of a web information system rather than for the initial start-up phase. there are a number of studies that have assessed the suitability of these tools in the development of web information systems (e.g. bertot & mcclure 1997; wu, yu & ballman 1998; bauer & scharl 1999). a discussion of these studies follows. bertot and mclure (1997) discuss analytical techniques that use web usage to determine user behaviour in relation to a web information system. web servers automatically generate and dynamically update four usage log files. these log files capture the following information in relation to user activity: (1) access log records the date, time, ip address and items accessed by the user; (2) agent log provides data on a user browsers, browser version and operating system; (3) error log records abort errors (4) referrer log indicates what other sites on the web are linked to the web information system. the following information can be obtained from the access log file: total number of daily and weekly accesses and hits; average daily accesses and hits; percentage of accesses and hits generated by web information system; percentage of accesses and hits experienced from within a country and from overseas countries; and specific path that a user takes through a server per session. a hit is a particular item downloaded from a web page whereas a access is recorded when a user downloads an entire web page. it is important to distinguish between the number of hits and accesses as the number of hits recorded in a access log file can be misleading. for instance, a firm may have their corporate logo image on every web page, hence that image is likely to be the most frequently downloaded hit item on a web site. from the agent log file, it is possible to determine the type and version of browsers used to access a web server and the type of operating system of the computer used to access the web server. this data is particularly 65 ajis vol 9 no. 1 september 2001 important as it indicates the access and display capabilities of the users. web developers should develop web information systems that meet the technical capability of the majority of users. with the referrral log file it is possible to determine the total and unique number of referring ip addresses; and the most frequently referring web sites. this data allows web developers to notify most frequently referring sites when it is necessary to update their links to avoid user encountered error messages. the error log file provides information on errors (e.g. dead linkserror 404 and aborted file downloads) that users encounter or initiate. analysis of the error log file can provide information such as the occurrence of missing files, erroneous links and aborted downloads. images, files and pages that are slow to download leading users to abort the file download are identified in the error log file as aborted downloads. this is important information as it allows developers to modify and correct content, thus decreasing the number of errors that a user will encounter in navigating the hypertext structure of a web information system. cookies and user login ids can be used to identify a user session when accessing a web system (wu et al. 1998). there is considerable resistance by some users to monitoring their activity online due to privacy concerns. the speed tracer is a web usage, mining and analysis tool that can record a user session without the use of cookies or a user id. speed tracer provides user-oriented statistics such as most frequent external referrals, most frequently visited pages, distributions of user session duration and the number of pages visited. there are ethical and privacy issues that may impact on these mechanical observation approaches. as the internet matures as a business medium, privacy legislation may restrict such approaches in the future. in australia, recent legislation has been passed that restricts the collection and use of online customer information without the consent and prior knowledge of the person involved. a balanced approach to requirements capture the researchers argue that in order to facilitate the capture and management of user requirements in business-toconsumer web information systems, is professionals need to combine traditional requirements elicitation techniques, market research techniques and web usage analysis techniques. many of the traditional requirements elicitation techniques and market research techniques are reliant on the cooperation and involvement of users. on the other hand, web usage analysis techniques use mechanical observation techniques for constant capture of user activity but these techniques are reliant on there being sufficient user activity in a system. and in the future, privacy legislation may curtail automated capture of user activity through web server log files and cookies. therefore, given the nature of web information systems and relative autonomy and anonymity of users, using a balanced approach that combines these techniques will facilitate capture and management of user requirements. traditional and marketing requirements capture techniques are used initially to identify the target market that exists online and periodic polling of the target market is necessary to ensure that marketing strategies are working effectively. mechanical observation techniques complement traditional requirements elicitation and market research techniques in that they provide continuous assessment of user requirements through analysis and inference of user activity (bauer & chong 1999). such a balanced approach addresses the equivocality and uncertainty faced in capturing the user requirements of business-to-consumer web information systems that are constantly evolving. the reiterative nature of requirements capture has been acknowledged by numerous researchers (loucopoulos & karakotsas 1995; somerville 1996) and given that the balanced approach advocated by the researchers includes sociological and technical techniques, the multiview methodology would accommodate such an approach (avison & wood-harper 1990). research issue and method this study was designed to investigate the techniques used to capture user requirements in business-to-consumer web information systems. the business-to-consumer web information systems pose special challenges regarding the capture of user requirements. the global scope and voluntary use of such systems, and the relatively autonomous and anonymous nature of users means that user requirements are difficult to capture. hence, the research issue is defined as "the assessment of requirement capture techniques used in the development of business-to-consumer web information systems". of particular interest to the researchers is the degree to which firms are using traditional requirements capture techniques, marketing research techniques and web usage analysis techniques in the ongoing development of business-to-consumer web information systems. the identification and assessment of requirements capture techniques will assist in identifying best practice in the capture and management of user requirements. a qualitative approach was deemed appropriate as this research is in the exploratory stages and there is a lack of empirical research on requirements capture and management in business-to-consumer web information systems (galliers 1994, yin 1994). in the first phase of this research, data collection involved analysis of three businessto-consumer web information systems with the aim of providing a preliminary assessment of requirements capture techniques currently used in practice. data was collected in a series face-to-face interviews with senior 66 ajis vol 9 no. 1 september 2001 management, marketing management and system/analysts in three firms that actively involved in business-toconsumer electronic commerce. assessment of requirements capture in three cases the results of these case interviews were analysed by the researchers. the details of the three cases are presented in table 4 and a brief discussion of each case follows. case product years in operation number of employees case a business software 13 30 caseb health products 20 100 casec film processing 23 78 table 4. details of the three cases case a is a small company that publishes, republishes and distributes business software to a number of large retailers. case a also sells software directly to customers by mail order and uses the internet as additional marketing channel. case b is a medium sized company which manufactures and direct markets health products by mail order. the web information system is seen as complementary to its existing marketing channels and case b wants to be strategically placed for the expected growth of the internet as a marketing channel. case b believe that their product is a good fit for the internet. case c is medium sized company that provides film processing services by mail order and is using the internet to complement this service. case c see the internet as an excellent delivery vehicle for their products as the film processing industry is starting to embrace digital processing. the development practices of the three firms were assessed with regard to requirements capture techniques and tools these cases were using in the ongoing development of their web information systems. the requirements capture techniques used by the three cases are presented in table 5 and a discussion of these techniques follows. locus of control communication obstacles traditional requirements elicitation techniques marketing research techniques web usage analysis techniques case a analyst/web developer within communication not addressed poor communication between web developers and users multiple stakeholders requirements poorly addressed goal oriented approach user feedback email online surveys isp web server log files caseb analyst/web developer within communication not addressed poor communication between web developers and users multiple stakeholders requirements addressed to some extent goal oriented approach critical success factors user feedback email online surveys own web server log files cookies identify user sessions casec analyst/web developer within communication not addressed poor communication between web developers and users multiple stakeholders requirements addressed to a greater extent goal oriented approach critical success factors user feedback email online surveys isp web server log files monitor bandwidth utilisation cookies and user id identify user sessions table 5 requirements elicitation techniques, market research techniques and web usage analysis techniques used in the three cases in all three cases, development was driven by the analyst/web developer which supports the findings of the study by lob. and pendse (1997). they found that there was a tendency for web developers to build web systems based on client specifications with little input from the users of such systems. this can be explained in part by the evolutionary nature of web information systems which have evolved from a simple information presence to supporting business transactions, communication and distribution. as these systems realise their commercial potential, it is becoming increasingly important to consider the user in the development of these systems. the lack of user involvement in these systems indicates that communication between the users and the web developers is limited. generally, there was a lack of involvement of users in the development process although cases b and c considered a wider range of stakeholders in business-to-consumer web information systems. in all three cases, the requirements of the web information system was driven by business goals and objectives. cases b and c also focused on the critical success factors for selling the products and services online as these 67 ajis vol 9 no. 1 september 2001 firms were much more proactive in selling online whereas case c saw their web information system as providing support for their products. although user involvement was limited, all three firms received some email feedback from users concerning the design and performance of their systems. the information received was of a voluntary nature, none of the firms had considered capturing requirements explicitly from users and were more interested in capturing marketing information from the users. user activity and to a lesser extent user behaviour in each case's web information system was analysed from web server log files. the log files were mainly used to analyse web traffic such as frequency of particular pages accessed and frequency of referring links. cases b and c were able to analyse individual user activity to a greater degree due to the use of cookies and user ids which allows them to identify a particular user session. at present, this information is mainly used for tracking orders placed by online customers. case c is also using the information to provide a personalised experience for the customer. preliminary findings and conclusions there appears to be little involvement of users in the definition and specification of the requirements in businessto-consumer web information systems. the requirements elicitation process appears to be largely analyst driven where analysts/consultants develop a web information system from the client's specifications namely senior management. the requirements elicitation methods define the system from the existing or current situation, recipients of the system, or through experimentation of a constantly evolving system. the user requirements were largely defined from senior management and web developer perspectives, by analysing web server log files and through limited voluntary feedback from users. there appears to be considerable scope for web developers to actively elicit requirements from users and for web developers to better utilise user activity and user behaviour which is implicitly recorded on web server log files. overall, traditional requirements elicitation techniques provide a rich and varied source of methods for capturing user requirements. the marketing research techniques complement the marketing nature of business-toconsumer electronic commerce and add another dimension to existing requirement elicitation techniques. the mechanical observation techniques such as web usage analysis techniques are important as they allow web developer to capture and evaluate user requirements by monitoring user activity on web information systems. a balanced approach to the capture of user requirements that incorporates traditional requirements elicitation techniques, market research techniques and web usage analysis techniques should provide a more complete set of user requirements and lead to development of business-to-consumer web information systems that are user centred and effective. future work further work is needed to fully assess the usage and viability of traditional requirement elicitation techniques, market research techniques and web usage analysis techniques in the capture of user requirements in business-toconsumer web information systems. the researchers will conduct a number of case studies across the different segments of the online retail market. this will allow the researchers to validate the usefulness of combining existing traditional requirements elicitation techniques and marketing research techniques with the mechanical observation of user activity and behaviour. the suitability of multiview methodology to support such an approach will also be evaluated. it is envisaged that an additional outcome of these confirmatory cases will be the development of a survey instrument to assess the requirements engineering process in web information systems. references avison, d. e. and wood-harper (1990) multiview: an exploration in information systems development, mcgraw-hill, maidenhead. bauer, c. and scharl, a. (29th november 1999) development of web information systems: conceptual models and examples from the retail financial services industry, proceedings of 3rd annual collecter conference on electronic commerce, wellington, new zealand. bauer, c. and chong, s. (june 1999) electronic markets development: using marketing research to determine user requirements, in: global networked organisations, proceedings of 12th bled electronic commerce conference, bled, slovenia, pp. 408-423. bertot, j. and mcclure, c. r. (1997) web usage statistics: measurement issues and analytical techniques, government information quarterly, 14:4, pp. 373-390. byrd, t., cossick, k. and zmud, r. (march 1992) a synthesis of research on requirements analysis and knowledge acquisition techniques, mis quarterly, 16:1, pp. 117-138. 68 ajis vol 9 no. 1 september 2001 davis, g (1982) strategies for information requirements determination, ibm systems journal, 21:1, pp. 4-30. fazlollahi, b. and tanniru, m. r. (1991) selecting a requirements determination methodology-contingency approach revisited, information and management, 21:5, pp. 291-303. galliers, r. (1994) choosing information systems research approaches, in: g r. henley on thames (ed) information systems research: issues, methods and practical guidelines, alfred walker, pp. 144-162 . hansen, h. r. (1995) conceptual framework and guidelines for the implementation of mass information systems, information & management, 28:2, pp. 125-142. isakowitz, t., bieber, m. and vitali, f. (july 1998) web information systems, communications of the acm, 41:7, pp. 78-80. lyytinen, k., rose. g and welke, r. (july 1998) the brave new world of development in the internetwork computing architecture (internca): or how distributed computing platforms will change systems development, information systems journal, 8:3, pp. 241-253. loucopoulos, p and karakotsas, v. (1995) systems requirement engineering, mcgraw-hill, berkshire, uk. poulymenakou, a., drakos, w, papazafeiropoulou, a. and doukidis, g (november 1998) towards new web application development practices, australian journal of information systems (special edition electronic commerce,), pp. 107-113. scharl, a. and brandtweiner, r. (1998) a conceptual research framework for analysing the evolution of electronic markets, electronic markets, 8:2, pp. 39-42. somerville, i. (1996) software engineering 5th ed, addison and wesley, new york. teubner, r. a. and klein, s. (november 1998) planning and designing web-based electronic commerce: a case study in the insurance industry, australian journal of information systems, special edition electronic commerce, pp. 86-96. ton, l. and pendse, s. an empirical evaluation of web page design and consumer behaviour, proceedings of 1997 coilecter conference, [online], available: http://www.collecter.org/cou97/toh.pdf, [accessed 8* may 2000]. valusek, j. r. and fryback, d. g (1987) information requirements determination: obstacles within, among and between, in: r. galliers (ed) information analysis: selected readings, reading, ma, addisonwesley.pp. 139-151 . wu, k. l., yu, p. s. and ballman, a. (1998) speed tracer: a web usage mining and analysis tool, ibm systems journal, 37:1, pp. 89-106. yin, r. k. (1994) case study research design and methods, thousand oaks, california, sage. 69 microsoft word ajisfinal2003.doc australasian journal of information systems special issue 2003/2004 4 information systems development as a research method helen hasan school of economics and information systems university of wollongong wollongong, australia email: :hasan@uow.edu.au abstract this paper takes the stance that some cases of information systems development can be considered knowledge creating activities, and, in those cases, information systems development can be a legitimate research method. in these cases not only is knowledge created about the development process itself but also a deeper understanding emerges about the organisational problem that the system is designed to solve. the paper begins with a brief overview of research in the design sciences and a comparison of research methods that are concerned with the design, and use, of information systems. this is followed by an assessment of the way systems development as a research method deals with the scientific research processes of data collection, analysis, synthesis and display. a case study, where the systems development research method was use, is described to illustrate the method and give the reader a better understanding of the approach. introduction information systems are modern entities that are distinctive in at least three respects. firstly, the computer hardware and software artefacts, on which they are based, are tools like no other in the history of mankind (kaptelinin 1996). not only can information and communications technologies (ict) enable the automation of intricate work processes but they also provide support for sophisticated ‘knowledge work’ (zuboff 1988, marcus et al 2002). secondly, information systems have a socio-technical composition with hardware, software, people and processes integrated into a complex, purposeful whole. thirdly, ict products and their use, are evolving at an unparalleled rate, with increases in power and capability matched by decreasing costs. information systems (is) as a field of study, draws its significance from the uniqueness of computer-based information and communication tools and their place in shaping recent human, social and organisational history. is researchers are distinguished by the fact that they have the difficult and challenging responsibility of understanding, creating and using information systems to best effect. to reach maturity as a discipline in its own right, the new field of is borrows research approaches from a wide variety of older disciplines, the closest comparative fields being the engineering traditions and the design sciences. engineering is a traditional discipline concerning the construction and use of artefacts. the design sciences aim, not only to develop knowledge for the design and realisation of artefacts, but also to improve the understanding of how to solve the social and organisational problems for which the artefact is designed. according to simon (1981) “design sciences do not tell how things are but how they ought to be to attain some ends”. similarly, advances in the field of is result from a better understanding of how to develop and use ict-based tools and what impact they have on the way we work, and live. the question then arises, as posed by gregor (2002 p12): what constitutes a contribution to knowledge when research is of this type? papers describing such research typically contain “no hypotheses, no experimental design and no data analysis” (ibid p13) and so often pose a dilemma for reviewers. this does not necessarily invalidate this type of research and the challenge is to conduct and report it in ways that identify the rigour and contribution of the research making it acceptable to journal editors and reviewers. this paper proposes that, due to their distinctive nature, information systems development can be a knowledge creating activity, when those systems relate to emergent knowledge processes (ekp) (markus et al 2002), and that, in those cases, information systems development is a legitimate research method. in the process of this type of information systems development, not only is knowledge created about the development process itself but also a deeper understanding emerges about the organisational problem that the system is designed to solve. the paper begins with a brief australasian journal of information systems special issue 2003/2004 5 overview of research in the design sciences and a comparison of a range of research methods that are concerned with the design, and use, of information systems. this is followed by an assessment of the way systems development as a research method deals with the scientific research processes of data collection, analysis, synthesis and display. a case study, where the systems development research method was adopted, is described and used to gain a better understanding of this approach. the paper then suggests a pragmatic way by which systems development research can be designed, conducted, with the results presented and justified. research in is information systems (is) is a developing and applied field, and members of the is community are questioning some of the more traditional ideas of what constitutes research and in what ways legitimate research can be conducted in the field. the characteristic of is that distinguishes it from other management fields in the social sciences is that it concerns the use of “artefacts in humanmachine systems” (gregor 2002). conversely the characteristic that distinguishes is from more technical fields, such as computer science and information technology, is its concern for the human elements in organisational and social systems. research can be defined as “diligent and systematic enquiry or investigation into a subject in order to discover facts or principles”, which are “accepted or professed rules of action” (the macquarie dictionary, 1981). the same source defines knowledge as “acquaintance with facts, truths or principles as from study or investigation” and the outcome of research adds to the body of knowledge of a discipline and it is appropriate to emphasise the strong conceptual link between research and the processes of knowledge discovery and creation. in is the quality of research is usually determined by the applicability of the knowledge that has been created. an important determinant of the quality of research is through its relationship to sound theory. is researchers have frequently borrowed and adapted theories and methodologies from older, more established disciplines. new theories and methodologies are also evolving appropriate to the unique socio-technical nature of the is field. kaptelinin (1996) describes the computer as a social and psychological tool unlike any other in the history of human endeavour because of its capacity to mediate human learning and communication. the objects of study in is research, information systems, are therefore a complex interaction of human, social and technical components that mediate organisational processes. while is researchers continue to rely on a wide variety of existing and new theories for the range of topics covered by the field, the identification of a single unifying or at least prominent, theory for is is proving to be difficult. in a classification of types of theories, among those for analysing, describing, understanding, explaining and predicting, gregor (2002 p12) proposes that a “theory for design and action” is most relevant for is research. this design and action theory is about the construction and use of artefacts, and about methodologies and tools for “how to do something” (ibid). this type of theory is dynamic so that it can be informed by other theories and can, in turn, provide feedback to augment traditional theories. to be able to identify what is, and what is not, research in information systems development, it is useful to distinguish between systems that automate and those that informate, using the term introduced by zuboff (1988). where the former are conceptually simpler than the latter, taking an existing manual process and replacing it with a computer-based system. many informate type systems are conceptually complex and the high failure rate of systems such as decision support systems, executive information systems, groupware and knowledge management systems is indicative that they are not well understood. markus et al (2002) have used the description “systems that support emergent knowledge processes” for these complex informate type applications and have shown that they require a new is design theory. examples of ekps are basic research, new product development and strategic business planning. these have the characteristics that there is no best structure or sequence to the process, and have an actor-set that is unpredictable and requires knowledge that is complex, evolving dynamically and distributed across a community of people. ekp involve innovation by sense-making, building knowledge through a recursive, participatory and australasian journal of information systems special issue 2003/2004 6 evolutionary manner (boland and tenkasi 1995). most knowledge intensive emergent processes involve high-level professional and technical personnel who have a high degree of autonomy and so have challenging information requirements, needing knowledge and expertise to apply this information. much of the early research in information systems has resulted in improved understanding of methods and techniques for the construction and use of automate or low-level informate systems. markus et al (2002) have developed a design theory for informate systems to support ekps however these are generally not standard and tend to be dependent on the specific context. what works in one organisation department or group may not work in another so that much knowledge about such systems is not transferable. many is projects concerned with ekp can be considered a piece of original research where the requirements, design and even implementation is original and contains new knowledge towards a general understanding of how to productively manage data and information in complex situations. many of these systems evolve through a series of prototypes, which are constantly evaluated with the results fed back into the systems requirements and design. through the activities of systems analysis and design and the programming of computers to be tools for complex social activities such as ekp, people are engaged in higher mental activities from which new insights and knowledge emerge. this is not a design theory but rather a grounded method of generating theory from a participatory action research process mediated by the unique characteristics of information and communications technology. systems development and related research methods following the lead of nunamaker et al (1991) this paper will adopt the term systems development method (sdm) for the design science research approach in is. however there are other related approaches, mostly in the realm of qualitative and interpretive research, some of which are action research, grounded theory, engineering, constructivism, pragmatism, tool-mediation of activity and developmental research. these are now presented in order to clarify and enlighten the discussion. a systems development method (sdm) for data collection, analysis and theory building has the reflective and iterative attributes often associated with participatory action research, combining research and practice in such a way that action brings about some situational improvement and research increases the broader understanding of the issue. however there are two particular characteristics that distinguish sdm from the general action research approach. firstly, in sdm there are always three inter-related domains where this research method can generate knowledge, those of: (a) the techniques of systems development, (b) the properties of system itself, and (c) the situation where the system is to be used. secondly, the research project is both constrained by the limits that current information technology places on the development of systems and is enabled by the uniqueness of this technology, which can, as a tool, mediate human learning and communication. sdm also has qualities found in grounded approaches to research. as explained in the award winning paper of orwlikowski (1995), a grounded approach enables is research to incorporate the criticality of organisational context in shaping technology use in organisations. grounded theory is an inductive, discovery methodology with the aims of generating a descriptive and explanatory theory while iteratively gathering rich data from one or more sites. concepts are suggested by the data rather than imposed from outside and are organised through the identification of recurring themes into theory. sdm however differs from traditional grounded theory research in the way data is coded and categorised. the term engineering-type research is mentioned by burstein and gregor (1999) who claim that this type of is research method is not always recognised as such and is comparatively poorly understood. engineering is relevant to is research that studies the design, delivery, use and impact of information and communications technology (ict) in organisations and society. however, ict can be considered more than physical artefacts or tools but rather ones that incorporate logic, in the form of software, and interact with people in a way no previous artefact has done. information systems australasian journal of information systems special issue 2003/2004 7 are both physical and mental tools so that in is construction can be considered a form of constructivism that relates closely to sdm. in the context of learning and development, constructivism is a theory, which came out of the work of piaget and is a philosophy of learning founded on the premise that, by reflecting on our experiences, we construct our own understanding of the world we live in. each of us generates our own rules and mental models, which we use to make sense of our experiences, much in the same way as models are constructed though the processes of systems analysis and design. as construction is a precursor of design, the discipline of engineering is strongly related to the field of design. niiniluoto (2001) and march & smith (1995) distinguish design science from the natural or descriptive sciences in research. march & smith (1995) recognise knowledge in design, and see the design process as one of creating knowledge, which is evident through the building and evaluation of the thing designed. the aim of design research is to explain how and why the constructs, models, methods and instantiations work. architecture, for example, can be viewed as a design science and one of the concepts emanating from architecture is the concept of patterns (alexander et al 1977), to explain how good designers conceptualise and reuse their ideas. the concept of pattern has more recently been applied to it development (lyon 2000). research methods, involving activities of design and construction, are akin to grounded approaches to research such as pragmatism, the philosophy that truth is what works in practice. in is activities of interest are found in the development of socio-technical systems involving computer-based tools. typical methods used in is research, where the design and construction of a system is involved, are observation, action or participant research often with various forms of prototyping (baskerville & wood-harper 1998). the evidence for validity of this type of research, in terms of knowledge creation, is usually referred to as proof of concept. it could be stated that there is a parallel between pragmatic, constructive research methodologies and the developmental methodologies used in practice for systems analysis, design and production. when using the term methodology in is it is often hard to determine where practice stops and research begins. rather than separate research and practice it may be more beneficial to investigate ways of combining and reconciling the two. a way to do this is by relating research to the concept of tool-mediate activity emanating from the original concepts of vygotsky (1978). vygotsky’s approach may be called social constructivism because he emphasized the critical importance of culture and the importance of the social context for development. vygotsky, and subsequent scholars in activity theory such as leoniev (1981) and engestrom (1987), saw learning, development and work as holistic human activity, which both mediates, and is mediated by, the tools used and the social context of the activity. this two-way concept of mediation implies that the capability and availability of tools mediates what can be done and the tool, in turn, evolves to hold the historical knowledge of how the community works and is organised. it is through the dynamic process of mediation that learning and development occurs, both in the individual and in the society as a whole (hasan & crawford 2002). the type and quality of the tools used for human work determines, to a much greater extent than they have in the past, the pattern and rate of development. new technology is driving changes to organisational structures and activities and this in turn is placing increasing demands on the capability and capacity of the technology. the changes that ict, and in particular the internet, has made to the way information is perceived and used in society today, is illustrative of this concept. it is proposed that research in this area is so closely related to an evolving practice that it must be embedded in, and immediately applicable to, that practice. an approach that incorporates this concept is development research, which is disciplined investigation conducted in the context of the development of a product or program for the purpose of improving either the thing being developed or the developer. it is therefore ideal for this investigation as it is both contextual and evolutionary, where a prototype model is constructed, used with the target group, which is observed and questioned before the prototype is revised. australasian journal of information systems special issue 2003/2004 8 the stages of systems development research sdm research incorporates many of the concepts mentioned above, design, construction, dynamic tool mediation and developmental aspects. these are found in the five stages of systems development research proposed by nunamaker et al (1991) namely, concept design, constructing the architecture of the system, prototyping, product development and technology transfer. these stages will be adopted in this paper, although in contrast to nunamaker et al, it is proposed here that the stages do not follow a linear progression but rather one that is interactive and dynamic as determined by the concept of tool mediation mentioned above. this suggests that what is being done, the research activity, is continually influencing, and being influenced by, the tools used in these five stages. therefore the distinction between the stages is blurred. they may be continually revisited or, sometimes, one or more may be left out of the process.: the five stages are as follows: concept design: in this first stage there needs to be an adaptation and amalgamation of current technical and theoretical advances in the area of interest. the researcher must find, synthesise, use, apply existing knowledge to identify gaps or limitations of existing systems and develop a meaningful research objective. this stage may involve a substantial literature review although the time taken to get research published probably means that the current state of the art is better gleaned from direct communication with practitioners and other researchers in the field. while this stage is more one of locating and synthesising existing knowledge, rather than discovering or creating new knowledge, it could result in the publication of a review paper on the topic. constructing the architecture of the system: the second stage is overtly one of new knowledge creation that should be accepted as genuine research. the researcher engages in the creative and innovative design activity of architecture development, defining components, models, algorithms and data structures. prototyping: this is the stage where proof of concept is often used to demonstrate that a system can be built based on the results of the previous stage. this may be done with a single working prototype or involve the iterative analysis, design and implementation of an evolving prototype. learning occurs through the evolutionary system building process where insight is gained about the problem and the complexity of the system. the evolutionary prototyping development process includes regular expert/user evaluation feeding back into the systems development process. in many cases of systems development the research stops at this stage because the system fails to meet expectation or is not feasible to be further developed for commercial use. (koskivaara 2002) product development: at the conclusion of the prototyping stage it is possible to freeze and formalise the systems specifications to build, test and evaluate a robust system. in some cases of systems development research there is a particular client sponsoring the research is interested in adopting the systems produced. if commercialisation occurs the new knowledge emerging from the project is often confidential and there is no public release of knowledge. technology transfer if the production stage is successful the product may interest a general audience. at this stage it may be possible to evaluate the use of the system with case and field studies or laboratory experiments, consolidating experiences learnt and even developing new theories of use. this may feed back into a new research cycle. a case study in systems development recent advances in ict have heightened organisational interest in communities (mclure-wasko & faraj 2000) and many organisations are looking to ict for solutions to such innovative knowledge work practices (schultze & boland 2000). the description of ekp, made earlier in this paper, is highly relevant to the complexities of communities of practice and is an area, therefore, where the systems development method is suitable for research. the study, described here, is part of ongoing research by the authors (see for example hasan & crawford 2003a,b) and will be used to exemplify the use of the sdm. it concerns the creation, maintainability and sustainability of communities of practice and learning. the design of this research was based on the evolutionary development of a australasian journal of information systems special issue 2003/2004 9 web-based groupware system, together with protocols for establishing and maintaining communities using it. the research was conducted through observations of a variety of disparate communities supported for the purpose of the research. the research originated with a community-base program of the australian photonics industry, which has a research centre and several small spin-off companies at the australian technology park. this pilot outreach program aimed to • increase awareness of the nature of photonics and career opportunities in the field • enable direct links between interested young people, scientists and entrepreneurial business development in the field. • provide extended and enriched learning experiences for students in schools, colleges and universities. • work cooperatively with teachers, students and parents to create e-learning experiences and learning materials that motivate and engage young people in the adventure of entrepreneurial research and development in photonics • facilitate pathways for people wishing to obtain qualifications and careers in the field. it was decided to run the pilot program as a community of representatives from the interested parties using a combination of face-to-face workshops and online project-based activities. technologists and research students at the technology park, under the guidance of experienced trainers, collected a suite of ict tools to support these endeavours, particularly the online community of learning and practice. a small web-based system was built to enable teams to have their own dedicated space to send messages, hold discussions and store files generated in the course of their team’s project. this was a rich learning experience for all, particularly the traniers involved in setting up the program. the subsequent research conducted by the authors, has developed and expanded the methods used in this program to study online communities in general. this will now be reported using the five stages outlined above. concept design and research questions there is a growing body of literature (boland & tenkasi 1995, engestrom 1999, toulmin 1999, wenger et al 2002), which promotes a view of socially-constructed, collective knowledge as the predominant source of learning, creativity and innovation. moreover this focus promotes knowing as an activity by specific groups of people in specific circumstances for a specific purpose. even in highly commercial firms, desirable outcomes are commonly achieved, not at the organisational or individual level, but at the group level in work units, cross-functional teams or informal groups of people who have come together with a common interest. it is not surprising then that the concept of ‘community of practice’, made popular by the work of wenger (wenger et al 2002) to cover an holistic, systemic view of community, has captured the attention of diverse researchers. distributed communities face challenges when coordinating their knowledge-sharing activities into the social and technical determinants of distributed communities of practice. innovative information and communications technologies have promised to enable such networks, but research shows their sustainability is still problematic. from the experience of the pilot photonics program it was realised that designing and building the system, not only the technology but also the protocols of the communities, was an effective way to conduct research into this problem. a project was begun to study the socio-technical determinants of different and diverse communities through the use not only of innovative groupware technology, but also of techniques, to establish understanding and trust between the participants to enable the community to thrive and achieve its purpose. an inductive developmental approach was taken in this research, recognising the mediating effect of new ict capability on such collective activity. as researchers we looked for any opportunity to study communities in regional areas where distance motivated the use of online facilities. a development research investigation was conducted in three regionally based communities that were in the process of being established. the specific concepts that became the object of the research were: • the nexus between learning and practice, (and similarly between knowledge and activity), australasian journal of information systems special issue 2003/2004 10 • the frequent location of knowledge creation activities in communities as distinct from individuals or formal organisations, and • the establishment of a viable social-technical model for systems to support communities, particularly those who predominantly communicate online. constructing the architecture of the system in the initial programs on photonics at the technology park, a multifaceted approach had been adopted where scientists, business developers, teachers, technologists, museums and business people contributed to: • intensive workshops at the technology park, • an online ‘community of interest’ that supports sustained creative activity as new materials are built and knowledge is exchanged. • community celebrations where young people show their creative work and explain their new learning and interest to members of the community including politicians, local government officials and the media. this framework helped to establish the architecture of the system, including the technical and social elements. in the three communities that were studied, a single day workshop was held to establish and build the community cohesion and trust. this workshop also determined what would be achieved and how. teams were established to work together in an extended online period on appropriate project designed to facilitate the desired outcomes, the progress of which was presented at occasional half-day face-to-face or virtual meetings. prototyping for the purposes of the research a prototype of an online support system was constructed modelled on the proprietary one used in the previous photonics project, which incorporated additional facilities to record all community activities and to regularly poll participants on their views. the interface and functionality of the system could be easily modified as the collected research data were analysed. the communities were each established several months apart so that each one benefited from experiences of the previous ones in an evolutionary process. preliminary results of this research have been published elsewhere (hasan & crawford 2003a,b). product development and technology transfer in addition to the afore-mentioned research publications, the result of this research is a viable product. as the research has progress, technical expertise has been used to ensure that the sociotechnical product, which has emerged from this research, is commercially robust. the technical system, together with the process of using it effectively, has reached a level of maturity and is in demand from a number of business and educational organisations. this activity is not, of itself, considered as part of the research. designing, presenting and justifying systems development research using sdm research a definition of research, as “diligent and systematic enquiry performed to discover rules of action”, is paraphrased from the introduction to this paper. a cursory overview of the development of informate systems that support ekp shows that the following can be considered research using an sdm: • creating group decision support systems added to the understanding of group planning and administration processes australasian journal of information systems special issue 2003/2004 11 • the growth of applications in the field of artificial intelligence led to the creation of the field of cognitive science and our understanding of both machine and human information processing. • the development of executive information systems using online analytic processing systems led to a better understanding of the executive management processes and their information needs. • the use of expert systems shells to set up knowledge bases has clarified the knowledge of many specific domains such as medical diagnosis. in the same vein it is claimed that the case described above uses sdm and conforms to the definition of research. is research aims to increase the understanding of organisational individual and social processes and how information and communications technology tools can support those processes. the output of stages 1 to 3 can be considered as contributions to new knowledge as follows: • from stage 1. this stage provided the research questions and plan for the project as a whole. • in stage 2. here emerged the design of a socio-technical model of sustainable online communities. • in stage 3 here a continuing research process takes place that is adding to the knowledge of how communities can use technological systems for their knowledge management and acting as “proof of concept”. research design although the progress of a systems development project is usually determined by the systems requirements, the fact that it is also research means that there must be a research agenda. in order to present systems development research as research, the researcher must state at some stage the research problem, the objective and the questions to be addressed in terms of what are the gaps in, or the limitations of, existing knowledge in the area. the researcher must then be able to interpret the findings from the research in terms of its contribution to knowledge. the contribution may be in the innovative nature of the product, its ability to improve performance in the workplace or in the illustration of a new method of product development. there must also be some way that this contribution can be verified, and in systems development research this can be done through the success of the system as proof of concept. this may be supplemented by evaluations of the systems concept or the usability of the system itself, as in the online community project described above. dealing with data in general any research project involves the collection of data by measurement, observation or other form of investigation. the research then consists of processes of data collection, analysis, presentation and either verification of hypotheses or the drawing of some conclusions that add to the accumulated knowledge of the field of study (miles & huberman 1994). to be accepted as valid research, the systems development approach must have either some methodical data collection and analysis or an alternative rigorous procedure appropriate for systems development research. data collected in research using sdm can be empirical, such as that from systems testing, qualitative, such as descriptions of the development process, or even implicit, in that the data of interest are embedded in a system’s design or implementation. the latter is somewhat unusual but must be considered if the systems development method is to be widely used in is. it should be noted that many of the techniques for the design of research and analysis of research data are similar to, and overlap, the is skills of analysis and design applied to systems including modelling the situation, documentation of the process, justification of choices, planning and conducting systems testing. presenting the results of data analysis may take different forms in sdm research. at stage two this may be the architecture itself or its justification. at stage three this may be the system itself as proof of concept. at stage four it may be the impact of the system on an organisation. from a research australasian journal of information systems special issue 2003/2004 12 perspective a valid outcome may be lessons learnt from a failure with knowledge of what doesn’t work. the test by which a systems development project can be considered valid research is by a demonstrable contribution to knowledge and a verifiable statement of what has been learnt. conclusion and observations is research has contributed significantly to the knowledge of how ict systems can be created and used effectively but this is often not recognised as research. even though the process of data collection, analysis and display are not as easily recognised and the distinctions between them are blurred, these three processes can be said to be present in the systems development approach to research. certainly the display is evident in the system itself but findings of the research are also evident in innovations to the way that the organisation conducts the activities for which the system was designed and consequent improvements to organisational performance. researchers with ict skills often use this approach but may not have an appreciation of how it constitutes research. we have observed that people with a purely technical focus place more emphasis in getting the product to work than in learning from the process. however it is probably not difficult to insist that systems development research papers contain an explicit statement of the research problem, objective and questions and conclude with a description of the outcomes in terms of the contribution to knowledge. there may often be no clear boundary between aspects of the systems development method and the research method, in many cases of this type of research. this is especially so with complex informate-type systems used to support ekp, but that should not mean that the research is any less valid if a contribution to knowledge can be demonstrated. it is hoped that a tradition in the use of the systems development research method can be established as legitimate research so that worthy papers can be readily approved for publication. references alexander c., ishikawa s., and silverstein m (1977) a pattern language: towns, buildings, constructions, oxford university press ny. baskerville r., wood-harper a.t. (1998) diversity in information systems action research methods, european journal of information systems 7. 90-107. blackler f. (1993) knowledge and the theory of organisations: organisations as activity systems and the reframing of management. journal of management studies 30/6 863-884. boland r. tenkasi r. 1995 perspective making and perspective taking in communities of knowing org science 6/4 350-372 burstein f., gregor s. (1999) the systems development or engineering approach to research in information systems: an action research perspective, proceedings of acis99, wellington, nz, 122-134. engeström y. (1987). learning by expanding: an activity-theoretical approach to developmental research, orienta-konsultit, helsinki. engestrom y. (1999) innovative learning in work teams: analysing cycles of knowledge creation in practice, in engestrom y. miettinen r. punamaki r. eds perspectives on activity theory, cambridge university press, cambridge uk. gregor s. (2002) a theory of theories in information systems, in s. gregor and d. hart, information systems foundations: building the theoretical basis, anu canberra, 1-20 hasan h. and crawford k. (2002) codifying or enabling: the challenge of knowledge management systems, journal of operation research, 54, 184-193 hasan h. and crawford k. (2003a) a multifaceted approach to distributed communities of learning and practice, proceedings of isglob03, ifip 8.2/9.3, athens. hasan h. and crawford k. (2003b) socio-technical systems for knowledge mobilisation in communities, proceeding of acis, perth. australasian journal of information systems special issue 2003/2004 13 kaptelinin (1996) computer-mediated activity: functional organs in social and developmental context. in nardy, b. (ed.). context and consciousness: activity theory and humancomputer interactions. cambridge, mass: the mit press kasanen e, lukka k siitonen a (1993) the constructive approach in management accounting research, journal of management accounting research (fall) 243-264. koskivaara e. (2002) design science approaches to information systems research, in s. gregor and d. hart, information systems foundations: building the theoretical basis, anu canberra, 205-216. leontiev a.n. (1981) problems of the development of mind moscow: progress lyon k.l. (2000) using patterns to capture tacit knowledge and enhance knowledge transfer in virtual teams, in y. malhotra ed, knowledge management and virtual organisations, idea group publishing, hershey pa. march s.t., smith g.f. (1995) design and natural science research on information technology, decision support systems, 15, 251-266. markus l. majchrzak a. gasser l (2002) a design theory for systems that support emergent knowledge processes, mis quartery 26/3 179-212. mcadam r. and mccreedy s. (1999) a critical review of knowledge management models, the learning organisation 6/3 91-:100. mclure-wasko m. and faraj s. (2000) “it is what one does”: why people participate and help other in electronic communities of practice, journal of strategic information systems, 9 155-173. miles m. b. and huberman a. m. (1994) qualitative data analysis, sage publications, thousand oaks, ca. niiniluoto i (2001) futures studies: science or art? futures 33 371-377. nunamaker j.f., chen m., purdin t. (1991) systems development in information systems research, journal of management information systems 7/3 89-106. orlikowski w (1993) case tools as organisational change: investigating incremental and radical changes in systems development, mis quarterly, 17/3. schultze u. and boland r 2000 knowledge management technology and the reproduction of knowledge work practices. journal of strategic information systems, 9 193-212 simon h (1981) the sciences of the artificial, the mit press, cambridge (ma). the macquarie dictionary (1981), macquarie university press, north ryde, australia. toulmin s. (1999) knowledge as shared procedures, in engestrom y. miettinen r. and punamaki r. eds perspectives on activity theory, cambridge university press, cambridge uk. pp 7086. wenger e. mcdermott r. snyder w. (2002) cultivating communities of practice, harvard business school press, boston. virkkunen j. and kuutti k. (2000) “understanding organisational learning by focusing on ‘activity systems’”, accounting, management and information technology 10 291-319. vygotsky, l.s. (1978) mind and society, harvard university press, cambridge, ma zuboff s (1988) in the age of the smart machine heinemann professional, oxford ajis vol 9 no. 1 september 2001 designing for web site usability chris procter judith symonds school of information science and humanities the open polytechnic of new zealand lower hutt, new zealand email: symjud@topnz.ac.n2 abstract this paper explores the customer perspective of world wide web (www) site design in the light of current www usability research. a usability study undertaken by spool et al. (1999) is replicated where testers search for specific information on a www site and answer a usability questionnaire. the search was carried out on new zealand winery www sites. the results are compared to the findings of spool in the areas of navigation, graphics and page layout. all three areas are found to be significant influences on www usability. additional recommendations concerning guidelines for usability are made. keywords web usability; user expectations; user interface; interface design introduction a substantial body of business world wide web (www) research investigates the business perspective relating to the purpose of www pages (banks 1999, lowry et al. 1999, hasan & tibbit 1999, lee et al. 1999, ridley & ridley 1999, spadaccini & burn 1999) and www page implementation problems (hasan & tibbit 1999, lane & cavaye 1999, lowry et al. 1999, ridley & ridley 1999, van akkeren & cavaye 1999). few researchers have approached business www sites from the perspective of the customer in australia and new zealand. ng et al. (1998) have found that the most frequent response to a question on future development of company www sites was 'more interaction with users'. such customer interaction can only be assured through research into their perceptions of current www sites. features of customer interfaces, such as navigation, that make it easier for customers to find information, positively influence the number of visitors to a site (lohse & spiller 1999). the aim of this research was to evaluate regional small business web sites from a customer usability perspective using the approach documented by spool et al. (1999). this paper presents a review of the relevant literature in the field of www usability design. this is followed by a description of a study of new zealand winery www sites that was undertaken. the paper concludes with key findings of the study concerning www usability design considerations and recommendations for future work. literature the objective of the literature search was firstly to define www usability, then to identify appropriate usability measures. significant work in this area has been conducted by nielsen (2000) and spool et al. (1999). one limitation of current usability studies is an understanding of the level of www experience testers brought with them to the study and the final part of this section presents relevant literature on factors that influence www experience. www usability defined there is no common definition for the term usability (corry et al. 1997). we use keevil's (1998) definition of www usability as 'a measure of how easy it is to find, understand and use the information displayed on a web site'. in addition, we add that web site usability is also about 'identifying the obstacles to users accomplishing their goals' (spool et al. 1999). usability studies do not necessarily attempt to measure the extent to which the developers have achieved their design objectives. in the usability testing described within this paper the researchers did not have information about the objectives of the developers of the web sites in question. thus this study is distinct from web site rankings and top ten lists since we do not set out to say which of the tested sites are more or less successful. although www usability measures originate from the field of software usability, www site usability is more important than that of traditional software because the internet provides users with user controlled navigation in a try before you buy environment (nielsen 2000). in addition, the www pages must invite, attract or pull the user in to engage in further exploration by its look and feel before the user has time to move on to the next link (singh & dalai 1999). after the event of initial access, site usability becomes critical. 92 ajis vol 9 no. 1 september 2001 identifying appropriate usability measures four recent instruments designed for customer evaluation of it can be identified in the literature. the instrument developed by goodhue (1998) has been designed to measure user evaluation of an information system. the instruments used by selz and schubert (1998) and srivihok (2000) are designed to measure customer evaluation of web commerce sites. the instrument used by spool et al. (1999) is designed to measure customer evaluation of web sites in general. table 1 lists a number of variables, indicating which are used within the respective user evaluation instruments. these variables are discussed below in order to identify appropriate measures of www usability. variable information accuracy compatibility navigation speed interface help trust friendliness currency appearance personal security user evaluation assessment instrument goodhue v v v v v x v x v v v x x selz x x x v v v x v x x x v v srivihok x x x v v x v x x v v x v spool x v x v v x v x x v v5 v6 x table 1: factors in user evaluation instruments analysis of the user evaluation instruments revealed that goodhue (1998) suggests three variables used to measure user evaluation of information systems that are not included in the other instruments. these variables are: compatibility, information (meaning the right level of detail) and friendliness (meaning ease of use of hardware and software). these variables are also not used within this study for the following reasons: • compatibility relates specifically to browser software and some plug-in software, which are not directly relevant to this study. • the level of detail on a www site is measured by the more specific variables such as navigation. • ease of use of hardware and software was not considered a relevant factor: the study was conducted on a reliable network with experienced users. none of the users subsequently commented on this issue. there are three variables used by selz and srivihok which are not considered appropriate in the winery usability study. firstly, security is not measured since this study did not involve the exchange of any sensitive information. secondly, the interface is measured in more detail by the usability instrument. thirdly, selz measured contact possibilities of ecommerce sites (listed above as trust). the contact possibilities variable has not been used since it is a measure of user trust rather than usability. the variables used by spool are not contradicted by any of the other instruments and appear to the most relevant measures of www usability. they are therefore selected as the most suitable instrument for the winery www usability study. properties affecting usability nielsen (1998) laments that most www sites have never been tested by real users before being published on the internet. the only test www sites get according to nielsen (1998) is when they are taken to the 'art director's office to look at the screen'. nielsen (2000) suggests that care should be taken with the presentation of the content of the www site. this includes writing for scanability using structured and meaningful headings, bulleted lists and using highlighting 5 includes appearance; graphics relevance; graphics quality; ease of reading; quality of language 6 includes fun; distractions and personal productivity 93 ajis vol 9 no. 1 september 2001 for emphasis. he also suggests using plain language and writing in the inverted pyramid style by placing the most important facts up front. legibility of the text can be improved by ensuring a high contrast between background text colours, using plain rather than patterned backgrounds, using larger fonts and ensuring text stands still instead of using animation or scrolling banners. arranging information in 'chunks' of short reference documents intended to be read non-sequentially is a practice used in technical and administrative documents that applies to presenting information on www sites (lynch & horton 1999). spool et al. (1999) conducted a study on the usability of eight well known corporate www sites. the data for this study was collected during 1997 using a set of 'scavenger hunt' tests. the participants of the study were observed finding information on the web sites. their ease of finding information was measured by use of both observation and questionnaires. as a result of this study spool presents three findings that he argues contradict accepted www design guidelines: 1) web sites that separate content form navigation make it harder to find information. 2) graphics appear to make no difference to usability with the exception of animation, which was very annoying. 3) it is not safe to assume that good page layout, as practised in hard copy publication, applies to web design. for example, spool argues that white-space and page readability on www pages makes information harder to find. although these three findings form the basis of the book by spool et al. (1999), the relationship between the findings and the evidence presented is limited. to date there has been some work on www usability testing using video tape to record scenario-based testing of sites (osterbauer et al. 2000). tester www experience the study by spool et al. (1999) does not attempt to gauge the prior level of www experience of the testers involved in their study, apart from providing testers with identical experience immediately prior to testing. for the purposes of the winery www usability study, prior www experience is considered a relevant factor. it can help researchers understand if usability obstacles are a factor of www literacy that will disappear when www users become more capable. in order to do this a definition of www experience was formed from the literature. www experience is influenced by a) demographic characteristics of the participants, b) past behaviour and c) access type. a) demographic characteristics such as race, gender, age, education level, household income, number of children at home, occupation, geographic region and gender by generation groups are all known demographic variables likely to influence www experience (katz & aspden 1997, kraut et al. 1996, hoffman et al. 1996). this study measured gender, age, education level and number of children. b) past behaviour is measured through usage history and the type of activities performed. usage history can be measured in monthly and daily frequency as well as the overall time since first becoming familiar with the www (hoffman 1996 & teo et al. 1996). the type of activities undertaken can be described using four categories of usage experience as described by teo et al (1996). the usage experience categories are: messaging, browsing, downloading and purchasing. we relied on respondents to report their past www behaviour and measured browsing and purchasing usage. data on messaging and downloading usage was not collected since these two activities relate to the internet and not specifically to www pages. c) access type can be described by type of access account, access location and home computer ownership (teo et al. 1996, hoffman 1996). we collected data on all three of these variables. method the aim of the wine web usability study was to explore www usability and to replicate the study by spool et al. (1999). firstly, a self-administered questionnaire was designed to collect data on experience in using the www, and experience with the subject area of the sites (wine). questions were developed to gather information on wine purchasing behaviour and interest in trying new wines. wine purchasing behaviour was measured by purchase frequency, purchase amount and purchase place. secondly, testers answered a set of questions about eight winery www sites. following this they completed a questionnaire, which included questions concerning their experience whilst searching the web site and their perceptions of www site usability. this questionnaire used the variable suggested by spool as previously discussed. however, the proportion of answers that were 'correct' have not been measured. 94 ajis vol 9 no. 1 september 2001 sample the population of www testers in the wine www usability study were the 93 members of the social club at a tertiary education institution. the social club was chosen because it is made up of people that represent the economic and social characteristics of potential customers of wine www sites. since www sites differ between industries, it was decided to concentrate on one industry in this study rather than risk being unable to compare between industries. winery www sites were chosen because they are generally small regional businesses well represented in new zealand with a product that is suitable for mail order and sale over the www. phase one www and wine experience the purpose of phase one of the wine www usability study was to determine the level of www and wine experience among the sample population. this was measured by means of a questionnaire. ninety-three questionnaires were distributed and 73 were returned giving a response rate of 79%. descriptive statistics were used to analyse the www and wine experience variables. phase two testing www usability the purpose of phase two of the wine www usability study was to rate the eight winery www sites based on usability measures and to record the testers subjective experience of using the www site. respondents of the www and wine experience questionnaire were asked to indicate their willingness to test www sites. of the 73 respondents of phase one, 50 were willing to test www sites and 40 testers actually completed the testing. the group of testers was statistically compared to the respondents of phase one and there were no significant demographic, www experience or wine experience differences between the groups. since each person tested three www sites, this stage resulted in 120 usable responses. the www sites were distributed randomly to control for the order that respondents did the testing. to test a www site, respondents answered four questions by seeking information from the site. the questions were formatted so that the answers were multiple choice, yes / no and short answer to allow the respondent to complete the questions quickly. in formulating the questions the content as suggested by spool et al. (1999) was used. the content of the four questions was as follows: 1. simple fact question 2. judgement question based on one fact 3. comparison of several facts question 4. judgement question based on several facts the answers to questions one and three could be found in the information on the web site. the answers to questions two and four required the participant to use the information on the web site to make a judgement. for example, testers were asked 'what variety of grapes are grown at the [b] vineyard?' although the answers to these winery questions are important, this paper does not deal with the analysis of the answers to these questions. participants were assured that the exercise was not a test for themselves, but a test for the web site. they were instructed that if they could not find the answer to one of the questions after a genuine search they were to note the question and move on. after each www site test, respondents completed a questionnaire consisting of questions relating to their experience using the site and the usability of the site. each of the questions used a seven point lickert scale where one was negative and seven was positive, for example one equals unacceptable and seven equals excellent. data from the www usability tests was analysed using the mean of the scores for each of the variables shown in the tables below. spool selected those variables representing the success of the search, then rated the www sites on their usability by multiplying these together and converting them into a percentage. in this research we use the total mean score for bom the experience and usability questions. this appeared to be just as accurate in indicating the usability of the www sites. analysis of the scores and tester comments are used to analyse the information further. 95 ajis vol 9 no. 1 september 2001 findings this section describes firstly the results of the www and wine experience questionnaire to give a profile of the testers. then the findings of the user experience and usability tests are presented. phase one: tester profile a profile of the testers was built up through analysis of the demographic, wine experience and www experience data collected. the www and wine experience analysis suggests that the testers are a moderately educated, sample of mainly middle-aged men and women. the majority of respondents purchase wine and have a general interest in finding out about wines. there is almost 100% access to the internet either at home or work amongst the testers and they have a high exposure to the internet in terms of opportunity and amount of use. the internet experience questions suggest that our study achieved a good representation of confident and non-confident www navigators. the testers are mainly search dominant customers with some www browsing interest. about half of the testers are likely to use the www for business. the testers were not accustomed to purchasing wine on the internet and few were likely to have researched wine on the internet before making a purchase from the store. phase two: usability responses to protect the confidentiality of the wineries, each of the wineries were assigned a letter of the alphabet from a to h. a mean score for the experience questions and the usability questions was calculated and these are displayed in table 2 and table 3 respectively. the site rankings were determined using the total of the mean scores. item 1. physical feeling right now 2. mental feeling right now 3. frustrated / always knew what to do 4. faster / slower than expected 5. quality of information 6. confident your found all the information 7. feeling now task is over total wine www sites b 3.46 2.31 2.23 3.00 1.77 2.08 2.08 16.93 a 4.31 4.31 4.38 3.77 4.38 3.08 3.77 28.00 h 3.87 4.27 4.40 3.93 4.40 4.33 4.33 29.53 e 4.41 4.53 4.71 4.06 4.94 5.41 4.24 32.30 f 4.89 4.83 4.89 4.33 5.53 5.56 4.11 34.14 d 4.53 5.53 5.07 4.80 5.93 5.73 4.80 36.39 g 4.88 5.50 5.31 5.13 5.50 5.62 4.50 36.44 c 4.77 5.54 5.31 4.92 6.08 5.85 5.31 37.78 table 2: experience mean scores for each www site arranged in ascending site rankings the data shows that testers did not have a good experience using web site b. generally testers rated the quality of information as very poor (1.77), a very low score on a seven point lickert scale. in contrast the mean score for quality of information for site c was very high (6.08). similarly there is a very big range of results for users confidence in finding all the information (2.08 5.85). 96 ajis vol 9 no. 1 september 2001 item 1. ease of finding specific information 2. ease of reading data 3. ease of concentrating on the data search (distractions) 4. logic of navigation 5. ease of search 6. appearance of site 7. quality of graphics 8. relevance of graphics 9. speed of data display 10. timeliness of data (is it current?) 1 1 . quality of language 12. fun to use? 13. explanations of how to use site 14. overall ease of use 15. completeness with which the site's subject is treated 16. your overall productivity with the site total winery www sites b 1.77 3.83 3.15 2.85 3.23 3.33 4.54 4.23 4.23 2.67 3.08 2.69 2.23 3.00 1.54 1.54 47.91 a 3.85 4.92 4.92 4.31 4.08 5.38 5.62 5.54 6.00 4.62 5.31 4.46 3.92 4.38 4.25 4.23 75.79 h 4.36 5.00 4.79 4.79 4.50 4.50 5.36 4.21 5.71 5.36 5.29 3.93 3.79 4.21 4.71 4.69 75.20 e 5.00 5.24 5.53 4.82 4.71 4.94 5.19 5.35 5.53 5.47 5.71 4.94 4.00 4.53 5.29 5.12 81.37 f 5.06 4.82 5.06 5.00 4.94 5.35 5.47 5.71 5.47 5.18 5.35 4.24 4.18 5.00 5.29 5.27 81.39 d 5.13 5.73 5.47 5.00 4.80 5.93 6.07 5.73 5.87 5.47 5.73 4.80 4.00 5.00 5.57 5.50 85.80 g 5.53 5.53 5.60 5.80 5.27 5.53 5.60 5.93 5.93 5.40 5.47 4.73 4.80 5.60 5.93 5.71 88.36 c 5.46 5.46 5.85 5.92 5.62 5.77 6.08 5.85 6.23 5.50 5.85 4.77 4.46 5.54 5.62 5.50 89.48 table 3: usability mean scores for each www site arranged in ascending site rankings each item from table 3 is a measure of usability and a potential obstacle to finding information on the www site. for example, insufficient 'explanations of how to use the site' present an obstacle to the user searching for information. the data shows that testers regarded web site b as being very 'incomplete' (1.54) and 'difficult to find specific information' (1.77). in contrast, site c was regarded as 'complete' (5.62) and 'easy to find subject information' (5.46). high ranking www sites rated well for all the variables and similarly, low ranking www sites rated poorly for all variables. this suggests that usability is an overall package rather than having some factors that can be done well and some that can be done poorly. discussion this section compares the results of the winery usability study with that of the corporate usability study undertaken by spool et al. (1999). then new issues raised by this research are discussed. finally, some general observations are made concerning the data. comparing the results with spool firstly we consider the issue of navigation matching content, then whether graphics make a difference to usability followed by the relevance of publishing concepts to www design. navigation matches content the winery usability study supports the findings of the spool's work that navigation is easier for users if the label or icon for the link reflects the content behind the link. www sites a and b, were rated poorly relative to the other scores for finding specific information (3.85;1.77), ease of searching (4.08;3.23) and logic of navigation (4.31;2.85) variables. they were also seen to use poor labelling for navigational links. analysis of tester comments on likes and dislikes of the www site also indicate that users value easy navigation. for example, one tester commented 'information was under (the) headings i expected'. similarly, users commented on navigation features that were less helpful such as 'no navigation at the top of pages only the bottom', 'lack of clear direction' and 'navigation buttons unclear to where they lead'. 97 ajis vol 9 no. 1 september 2001 graphics make a difference contrary to spool's findings, in the winery usability study, there is a close correlation between successful searching and the quality and relevance of graphics used on the design. furthermore there were twenty one comments out of the 120 tests done that indicated that the graphics of a site were important. for example, one tester when commenting on things they liked about the site responded 'colourful, good graphics, classy'. however, we agree with spool that animations can be off-putting for users. for example, one tester commented 'animations irritate me'. page layout and readability in this study there is a clear relationship between page layout and usability. www sites that rated poorly in ease of reading consisted of pages of wordy text with few headings and little emphasis on keywords. testers commented positively about www sites that featured clear text ('clean lines consistent page format') easy reading ('text clear and easy to read'), colour schemes ('warm colours of graphics') and the appeal of the site ('logical, simple, professional look'). testers were also clear about what they didn't like in terms of page layout and readability such as too much text ('too much text to plough through' and 'far too wordy') poor contrast in colours ('background too dark') inefficient set-out ('the look too much space on either side of the screen and between paragraphs') and they commented it was hard to find specific information because 'headings were too vague'. these findings support the ideas of nielsen (2000) as discussed in the literature section. they seem to contradict spool's findings concerning page layout and readability to some extent although he may be correct about www design requiring white space. this study suggests that traditional publishing guidelines as for example discussed by carroll et al. (1988) are relevant to www design. other issues across all the www sites the scores for the variables concerning how to use the site and fiin to use were relatively low. nothing in the tester comments indicated why the fun to use variable scored lower than other variables. testers commented positively on clear labelling of icons and links ('when you run the mouse over icon, a description is displayed'), use of simple language ('plainer language could have been used in parts') and intuitiveness ('not being able to read some information that stayed hidden until clicked'). currency of information is one area that testers picked up on. testers report 'frustration with the number of unavailable wines' and another tester noted that the 'news was not updated' as it still advertised an event which had passed. the testers applauded simplicity in design. this interest in simplicity was evident in comments such as 'reasonably straight forward whilst still elegant' and 'simple words no fancy graphics clear definitions'. testers also valued useful content about the wine process, the wine itself and the events related to the wineries. they were often frustrated by a lack of information in such comments as 'cost not with wine description would have to know to look for an order form'. the testers also commented negatively on the style of the information presented in comments such as 'imbalance of rave reviews was disconcerting'. testers rarely commented on speed of access if it was fast but were quick to note even the slightest hint of lag as in this comment 'a hint of slowness of access'. general observations the ranking of sites in the experience and usability tables are the same. when web site users have a good experience they regard the web site as usable or vice versa. for example, in a question relating to the overall atmosphere of the web site tested, one tester commented 'it was comprehensive, enjoyable and easy to use'. this quote includes elements of both usability and experience. in other words, because users had a high opinion of certain features of a web site such as the graphics, the logic of the navigation and the readability, they enjoyed the experience of working with the site. the link between these variables may be worthy of further research. reflection this study supports logical navigation and structure for www site design. it suggests that graphics do influence usability. it also suggests that publishing design principles do apply to www site design. several new factors in addition to those suggested by spool et al. (1999) are highlighted by this study including fun, self explanatory design, currency, simplicity, completeness and access speed. these findings suggest that if you want to design a usable www site you should consider the following points: • ensure that navigational links match the content behind the links 98 ajis vol 9 no. 1 september 2001 • use colourful, relevant, professional looking graphics • don't overdo white space but keep the text clean and clear and maintain contrast between background and foreground. • write for scanability by including useful headings and keywords • factor in a bit of fun • use a self explanatory design • design for currency to allow information to be readily updated • use simple designs • provide complete information in an objective manner • optimise access speed to the www site and between pages. value of usability variables the usability variables ranked the eight www sites in this study in the expected order. the variables were very useful in determining the obstacles users encountered in using the sites and the variables were supported by the comments of the testers. even though the testers were experienced in the use of the internet and in purchasing wine, the users still encountered obstacles to achieving their goal of searching for the answer to a specific question. this suggests that www site usability will continue to be an issue for www designers, particularly as more experienced users demand even better usability. it also suggests that the usability instrument used in this study is a valid and reliable measure of the obstacles for users in using www pages. recommendations for future research some interesting issues that are raised by this study include generalisations across industries, measuring the users initial impressions of the www site and the value of the tester comments in this exploratory study. it is suggested that the make-up of a business to customer web site depends, amongst other factors, on the particular industry that the business belongs to (spadaccini & burn 1999). it may be interesting to repeat the study in small business industries other than wineries, and to test the potential diversity, for example, those with prospective customers with more internet experience such is it professionals or under 25s. in addition to the usability of a particular www site, it may be important to measure the impression users pick up from different www site designs and look at the extent to which you can compare user expectations to owner priorities. although impression isn't totally related to usability, it generally is the deciding factor on whether a user remains at a www site after initial contact. a measure for this concept is more difficult to develop. preliminary work on customer impressions of www sites has already been undertaken by singh and dalai (1999) in applying the established principles of marketing to www design. although the site ranking process and the results for each of the variables is conclusive, much of the interesting analysis was drawn from the comments of the testers. further qualitative studies in this area, perhaps in the form of a series of case studies may allow for a deeper analysis of www site usability. conclusion the winery usability study has confirmed many of the findings of the corporate usability study (spool et al. 1999) and established that usability problems are not diminished by the relevant experience of the customer. the study has also provided rich data, which will be fed back to the individual wineries to allow them to improve the usability of their www sites. follow on work will analyse the answers to the four winery questions and feed back these findings to the wineries. the usefulness of the usability instrument in understanding obstacles users face in searching www sites has also been tested with a positive outcome. this study has provided an interesting and useful look at www sites from the perspective of the customer that provides a foundation to explore the issue of www design more deeply. references banks, d.a. (1999) who needs methodologies? a case study of the development of a web-based information system, 10th australasian conference on information systems (acis), wellington, new zealand. carroll, j.m., smith-kerker, p.l., ford, j.r. and mazur-rimetz, s.a. (1988) the minimal manual, humancomputer interaction, 3,123-153. corry, m.d., frick, t.w., and hansen, l. (1997) user-centered design and usability testing of a web site: an illustrative case study, etr & d, 45(4), 65-76. goodhue, d.l. (1998) development and measurement validity of a task-technology fit instrument for user evaluations of information systems, decision sciences, 29,105-138. 99 ajis vol 9 no. 1 september 2001 hasan, h. and tibbit, h. (1999) multiple perspectives on electronic business: a case study of a financial planning service, 10th australasian conference on information systems (acis), wellington, new zealand. hoffinan, d.l., kalsbeek, w.d., novak, t.p. (1996) internet and web use in the us, communications of the acm, 39, 36-46. jentzsch, r. and miniotas, a. (1999) the application of e-commerce to a sme, 10th australasian conference on information systems (acis), wellington, new zealand. katz, j. and aspden, p. (1997) motivations for and barriers to internet usage: results of a national public opinion survey, internet research: electronic networking applications and policy, 7(3). <http://www.emerald-library.com/brev/17207cbl.htm> (21 february 1999) keevil, b. (1998) measuring the usability index of your web site, proceedings of the 16th annual international conference on computer documentation, sept 24-26, quebec. kraut, r, scherlis, w., mukhopadhyay, j.m. and kiesler, s. (1996) the homenet field trial in residential internet services, communications of the acm, 39(12), pp. 55-63. lane, m.s. and caveye, a.l.m. (1999) the convergence of it and marketing in the ongoing development of an internet marketing web site, 10th australasian conference on information systems (acis), wellington, new zealand. lee, c.y., seddon, p. and corbitt, b. (1999) evaluating the business value of internet-based business-tobusiness electronic commerce, 10th australasian conference on information systems (acis), wellington, new zealand. lohse g.l. and spiller p. (1999) internet retail store design: how the user interface influences traffic and sales, journal of computer mediated communication, 5(2). <http://www.ascusc.org/jcmc/vol5/issue2/lohse.htm> (6 march 2000) lowry, g singh, m. and scollary, a. (1999) electronic commerce initiatives in australia: identifying opportunity, meeting challenges, and measuring success, 10th australasian conference on information systems (acis), wellington, new zealand. lynch, pj. and horton, s. (1999) web style guide, yale university press, new haven. nielsen, j. (2000) designing web usability, new riders publishing, indianapolis. nielsen, j. (1998) that mess on your web site, technology review, sep/oct. ng, h., pan, yj. and wilson t.d. (1998) business use of the world wide web: a report on further investigations, information research, 3(4). <http://www.shef.ac.uk/~is/publications/infers/paper46.html> (15 december 1999) osterbauer, c., kohle, m. and grechenig, t. (2000) web usability testing: a case study of usability testing of chosen sites (banks, daily newspapers, insurances), 6th australian www conference, cairns, australia. <http://ausweb.scu.edu.au/aw2kpapers/osterbauer/paper.html> (30 may 2000) ridley, j. and ridley, g (1999) the potential for adoption of electronic commerce in the tasmanian timber industry, 10th australasian conference on information systems (acis), wellington, new zealand. selz, e. and schubert, p. (1998) web assessment a mode for the evaluation and the assessment of successful electronic commerce applications, proceedings of the 31st hawaii international conference of system science, hawaii: ieee computer society,. singh, s.n. and dalai n.p. (1999) web home pages as advertisements, communications of the acm, 42(8), pp. 91-99. spadaccini, a. and bum j. (1999) on-line organisations they hype, myths and realities: a study of it enabled communications in organisations in western australia, 10th australasian conference on information systems (acis), wellington, new zealand. spool, j.m., scanlon, t., schroeder, w, snyde,r r. and deangelo t. (1999) web site usability: a designers guide, san francisco:morgan kaufmann publishers. srivihok, a. (2000) an assessment tool for electronic commerce: end user evaluation of web commerce sites, (internet) standing, c. (1999) managing and developing internet commerce systems with icdm, 10th australasian conference on information systems (acis), wellington, new zealand. teo, t.s.h. lim, v.k.g and lai, r.y.c. (1997) users and uses of the internet: the case of singapore, international journal of information management, 17(5), pp. 325-336. urquhart, c. and lennox, m. (1999) perceptions and adoption of it: conversions in agriculture, 10th australasian conference on information systems (acis), wellington, new zealand. van akkeren, j. and cavaye, a.l.m. (1999) factors affecting entry-level internet technology adoption by small business in australia: an empirical study, 10th australasian conference on information systems (acis), wellington, new zealand. 100 ajis vol 9 no. 1 september 2001 wong, j. and romm, c.t. (1999) culture, industry or structure? comparing organizational web site development strategy across industry and culture, 10th australasian conference on information systems (acisj, wellington, new zealand. 101 microsoft word ajisfinal 12 2 marina.doc ajis volume 12 no. 2 may 2005 147 the buywell way: seven essential practices of a highly successful multi-channel etailer mary tate beverley hope brent coker school of information management victoria university of wellington mary.tate@vuw.ac.nz beverley.hope@vuw.ac.nz brent.coker@vuw.ac.nz abstract after the dot-com bust there is considerable evidence that multi-channel retailers are more successful than purely on-line retailers. multi-channel retailing is becoming mainstream and considerable research exists on successful multi-channel strategies. despite this, some organisations are having more success than others with their multi-channel approach. we talked to the management of one of australasia’s most successful multi-channel apparel and home-ware retailers about the theory and practice of multichannel retailing, with the aim of building on existing theory in multi-channel e-commerce. keywords multi-channel, e-commerce strategy, active scanning introduction the demise of the dotcom’s has shown us the value having a physical retail presence in addition to an internet presence. increasingly, these ‘click-n-brick’ organisations are being supplemented with mail-order catalogues, kiosks, and contact centres to form multi-channel marketing and sales organisations. shorter product life-cycles, greater product choice, and increased fragmentation of customer segments are driving customers to select channels to suit particular purposes and preferences (goersch 2002). research indicates around one third of consumers use a combination of the internet, catalogues, and stores to do their purchasing. these multi-channel consumers spend more than consumers who purchase only from physical retail stores, and they are more loyal (forrester 2002; shop.org 2001). while there is a growing literature on e-commerce implementation and multi-channel strategy, some companies have clearly been more successful than others. the aim of this research was to study an exemplar organisation, with the aim of extending existing theory in multi-channel e-commerce. the research question is: “what extensions to existing multi-channel e-commerce theory can be made based on an exemplar organisation?” first we review the literature with respect to existing theory for success factors in multi-channel ecommerce. next we present the research method and results. the paper concludes with a summary of the findings, implications for future research, and contributions of the paper to research and to practice. literature review there has been much research on successful implementation of e-commerce into existing organisations (barua et al. 2001; ghoshal et al. 2002; goersch 2002; gulati et al. 2002; steinfield et al. 2002; weill et al. 2002; westerman 2002; willcocks et al. 2001a). some researchers have approached multi-channel e-commerce research from the perspective of buyer behaviour (balabanis ajis volume 12 no. 2 may 2005 148 et al. 2001; kaufman-scarborough et al. 2002; reardon et al. 2002), while others have focused on internet strategy (porter 2001; simons et al. 2002). based on this research we identified a number of success factors that we expected to be significant in our exemplar organisation. in this section we identify from the literature success factors (sfs) for a multi-channel e-commerce strategy. from synthesis of literature discussing e-commerce implementation into the organisation, multi-channel buyer behaviour, and internet strategy, six initial sfs were discovered: functional integration, channel synergy, brand management, information management, logistics management, and customer management. following our data gathering, and additional, post-hoc literature review was carried out to further explain our findings and provide additional insights in areas that our case organisation had identified as important to them. as a result of this, an additional sf, active scanning, was added based on marketing literature. sf1: functional integration functional integration refers to the degree to which channels share common organisational resources within a multi-channel environment. for example, channels would share functional departments such as finance, marketing, and logistics. (barua et al. 2001; goersch 2002; lee et al. 2001; schoenbachler et al. 2002; steinfield et al. 2002; willcocks et al. 2001a) there is a danger that, while achieving operational and marketing efficiencies, functional integration can result in loss of freedom and creativity for business units.(ghoshal et al. 2002; gulati et al. 2002; westerman 2002) one way to prevent this is for business units to also share “ emotional integration”, that is, to subscribe to a shared identity and meaning (ghosal et al 2002). this includes a desire to reach common goals supported by a shared organisational philosophy. in summary, functional integration is the sharing of organisational resources across channels, ideally bound by a shared identity and organisational philosophy. the benefits of functional integration are transferred to the customer through channel synergy. sf2: channel synergy channel synergy is defined as the degree to which channels consistently provide a seamless communication and purchasing experience for customers. (kaufman-scarborough et al. 2002; porter 2001; reardon et al. 2002; schoenbachler et al. 2002; simons et al. 2002; steinfield et al. 2002). it implies ease of movement across channels. for example, the internet becomes another customer contact point to facilitate the overall purchasing experience, regardless of where the final transaction takes place (goersch, 2002). a propensity to switch channels creates synchronisation problems for a firm as customers expect to be recognised on each channel. for example, customers buying on the internet expect to be able to return goods to the physical store. pricing and inventory information must be consistent across channels (day et al. 2002). displaying lower prices on one channel may attract customers to that channel, but it carries the danger of inciting resentment from customers loyal to another channel. by maintaining channel consistency, shoppers are encouraged to stay with the same retailer when they switch channels (goersch, 2002). in summary, channel synergy includes seamless purchasing and communications experiences for customers and consistency of pricing across channels. sf3: brand management brand management within the context of multi-channel e-commerce is defined as a firm’s ability to ajis volume 12 no. 2 may 2005 149 reinforce and leverage a consistent brand image across channels. (bitner et al. 2002; goersch 2002; kocas 2003; willcocks et al. 2001a). multi-channel firms have a distinct advantage over single channel firms by having more interfaces with customers, allowing increased brand visibility and economies of scale in advertising. the more familiar the customer is with a brand, the lower the perceived risk of purchasing across multiple channels (schoenbachler et al. 2002). building a strong service brand requires a track record of delivering on promises, for example, meeting promised delivery times. reliability across all customer contacts is important, because to the customer, the encounter is the service (bitner et al. 2000). in summary, brand management includes portraying a consistent brand image, and a track record of service delivery. sf4: information management information management is defined as the process by which information is collected, stored, analysed, and applied (willcocks et al. 2001a). good information management is particularly important in multi-channel firms because of the complexities involved in processing transactions from multiple sources and the need for channel synergy. this requires integration of information systems (barua et al. 2001; weill et al. 2002). for b2c business models, integration is needed in three areas: application infrastructure (payment transaction processing), internal and external communications infrastructure, and it management (weill et al, 2002). investing in an integrated information technology infrastructure allows up-to-date customer data from any channel to be made available through all other channels (goersch, 2002). furthermore, it allows organisations to track customer demand, thereby lowering costs and improving efficiency while enabling accuracy and speed of order fulfilment (lee et al. 2001). information captured through multi-channel shopping is frequently used as a basis for data mining. information from click stream data, catalogue addresses, and purchasing information can be used to develop automated customer services such as adaptive profiling, one-to-one marketing techniques, and mass customisation. in summary, good information management involves keeping information up-to-date and consistent across channels through integrated information systems. information captured from multi-channel shoppers can be used to develop a range of automated services. internally, integrated information systems are required to support logistics management. sf5: logistics management logistics in a multi-channel strategy involves the ability to effectively and efficiently fulfil orders arising from multiple sources. order fulfilment from channels other than a physical retail store is the most expensive and critical operation for multi-channel retailers (lee et al. 2001; pyke et al. 2001). the process must be matched to the product, the industry, and the business environment. large or heavy products sold through channels that require a firm to make delivery put pressure on profit margins. similarly, delivery of small orders or low value orders can squeeze margins. by contrast, products that can be digitalised such as music and software can be delivered over the internet at much lower costs than distribution of hard copies. in summary, logistics management involves the ability to process orders from multiple sources, and to provide an efficient logistics function to manage the cost of delivery. ajis volume 12 no. 2 may 2005 150 sf 6: customer management customer management is a firm’s ability to manage customer relationships, process customer information, and offer customer service in a manner likely to foster and retain customer loyalty (barua et al. 2001; kaufman-scarborough et al. 2002; lee et al. 2001; schoenbachler et al. 2002; simons et al. 2002). buywell went beyond the descriptions of best practice customer management in the literature. this led us to carry out a post-hoc literature search to assist in further explaining this dimension, and to propose the additional factor described as “active scanning”. literature also suggested that there would be particular challenges in customer management, particularly for the remote channels, because buywell’s products are mostly non-commoditised items such as clothing and home-wares. it has been suggested that these products often require personalised selling; the process of identifying customer needs and helping people find products to meet their needs. lack of opportunity for personal selling is considered a limitation for selling experience goods remotely (schneider et al. 2001). we were interested in exploring how our case organisation had managed this limitation. company profits can be dramatically increased from small increases in customer retention rates (reichheld, 1996 cited in (winer 2001)). to increase retention rates, firms must build strong relationships with customers based upon understanding who their customers are and what they want. a multi-channel strategy can provide this information through click stream and catalogue purchasing data. using this information, organisations are able to offer a high level of service through adaptive profiling and one-to-one marketing techniques. (willcocks et al. 2001b) sf 7 active scanning our post-hoc literature search found day’s commentary on becoming a “market-driven organisation” using “market sensing” most closely described the practices of buywell (day 1994). we were not able to find previous studies that had considered the application of aspects of market sensing in multi-channel e-commerce. market-driven organisations are more effective at meeting the needs of customers than non marketdriven competitors. they demonstrate “a superior ability to understand, attract, and retain the most valuable customers within the market” (day, 1994 p. 39). this provides the firm with more anticipatory capability on market conditions resulting in increased speed in responding to the market. market sensing is a distinctive capability of market-driven organisations. proposed by day in 1994, before the wide-spread diffusion of the internet, market sensing is “the ability of a firm to learn from their customers and competitors in order to be able to act on events and trends in current and prospective markets in a deliberate and systematic way” (day, 1994, p. 43). two important aspects of market sensing are active scanning and informed imitation. in the case of buywell, active scanning was particularly evident. active scanning allows a firm to capture detailed qualitative information in real time, which enables superior customer service and better alignment of a firm’s offerings with customer needs. this involves continuous communication with customers (day, 1994). active scanning requires employees be motivated to pass information gathered from customers to management. the types of information valuable to management are complaints, requests for new products or services, and feedback on the consequences of competitor activity. for many firms, this information from frontline employees is blocked (day, 1994). in summary, market sensing extends the traditional customer management to include active scanning. this is achieved by continuous communication with customers feeding into strategic decision-making. ajis volume 12 no. 2 may 2005 151 these seven success factors derived from literature: functional integration, channel synergy, brand management, information management, logistics management, customer management and active scanning; are used to anchor the findings of our case study. research method this research uses a case study methodology in which theoretical propositions, presented in the form of success factors, are compared with empirical materials collected from the field. this creates a link between theory and empirical data “providing a template against which to compare the results of the study” [pg 50] (yin 1993). this is important given that there is a significant body of existing literature. this also allows us to make explicit the areas where existing theory can be extended by the findings from the case organisation. the case site a single case site is used. the company, which we will call buywell, is australasia’s largest mail-order apparel firm with operations in new zealand and, more recently, australia. buywell is a privately owned company with approximately 650 employees serving 230,000 customers. initially, the firm sold only women’s apparel and only through mail order catalogues. more recently the product range has expanded to include men’s and boys’ wear, lingerie, footwear, house-ware, and oversized clothing, but women’s apparel still accounts for most of buywell’s sales. their multi-channel strategy now includes mail-order catalogue, contact centre, internet website, and five physical retail stores located across the north island of new zealand. buywell. most of buywell’s customers shop remotely through catalogues or the contact centre. between nine and eleven catalogues are mailed out each year, and most catalogue-based orders are received within twenty days of each new issue. the website has shown steady growth since its inception in 1998, accounting for 2% of sales by 2001 and 7% in 2003. buywell were selected for this research for three reasons. firstly, buywell has been very successful, showing strong growth over the past 10 years. consequently, they were expected to be a good exemplar of a successful multi-channel strategy. secondly, buywell represents a typical multi-channel strategy firm, both in size and channel mix. the average number of channels for b2c business models is 4.2 channels (day et al. 2002). buywell uses four channels, making it representative of multi-channel organisations with respect to number of channels. thirdly, buywell has a relatively mature channel mix in comparison to most other multi-channel organisations within new zealand. furthermore, buywell was a multi-channel organisation before wide-spread diffusion of the internet, making them similar to multi-channel firms studied in the literature and increasing compatibility for sf matching. data collection and analysis data were collected in two phases. in the first phase, semi-structured interviews were held with five key managers: the channel campaign manager, distribution centre manager, marketing manager, is manager, and managing director. these five positions were chosen because they represent leadership positions in areas contributing to the organisation’s multi-channel e-commerce strategy. interviews sought to gain insight into the participants’ understanding of how organisational elements influence the firm’s e-commerce strategy. interviewing five managers contributed to reliability by acting as verification on organisational memory and establishing a common understanding of multi-channel e-commerce strategy (earl 1993). ajis volume 12 no. 2 may 2005 152 interviews were transcribed in full and analysed using pattern matching with a list of categories based on the sfs identified from the literature. additional categories, not identified from literature, were created as required. as we discussed earlier, we then conducted a post-hoc literature review to further explain our new categories, and identify the areas where we were aiming to make extensions to existing theory. findings from this analysis were presented to buywell management for verification and comment. following that, we facilitated a focus group in which the management team responded to and elaborated on our findings. results as expected, all sfs were found to varying degrees. we have divided our results into three sections – areas where buywell confirmed but did not extend the practices expected from e-commerce literature, areas where buywell challenged or exceeded the practices suggested by existing ecommerce literature, and new factors we were not able to identify from existing literature. we first consider areas where buywell confirmed existing theory. confirmation of existing theory in these areas, buywell’s management found existing theory to be relevant to their business, but in our view, their business practices reflected, but did not significantly extend, existing theory. sf 1: functional integration in this area we found that buywell followed the best practice identified in literature, but did not appear to significantly extend previous findings. buywell is extremely well integrated with a structure organised around functions rather than channels. the main three main functions used as the basis for the organisation structure are purchasing, logistics, and sales and marketing, called by buywell “buying smarter”, “working smarter”, and “selling smarter”. the marketing manager explained: “we don’t want to structure marketing in one silo, or one channel... it’s all integrated within the role, and we work across the channels. we don’t have a catalogue manager or an internet marketing manager, we have a customer care manager or a loyalty manager, and they are there for the customer regardless of what channel.” respondents confirmed buywell had a strong, shared organisational philosophy. the general manager explained: “…they become accustomed to the ‘buywell way’ as we call it, from the customer service perspective…” in summary, buywell had a very high level of functional integration, and a clearly articulated shared identity and meaning across all functional departments. sf 2: channel synergy channel synergy describes consistency of pricing and a seamlessness purchasing and communications experience across channels. in this area buywell also demonstrated the best practice identified in literature. we did not find any insights from buywell’s practice that extended this individual factor significantly, although we did find a particularly close correlation between this factor and other factors, which is discussed later. the general manager explained: “we would confuse the heck out of those people if the website was totally different to the ajis volume 12 no. 2 may 2005 153 catalogue. so a lot of thought has gone into trying to provide consistency.” marketing over the internet is a lot cheaper than printing catalogues. to encourage customers to shop on the internet, a free delivery incentive is offered. by contrast, catalogue orders incur a delivery charge. this provides an incentive without inconsistent pricing. the channel campaign manager explained: “the only difference is free delivery on internet orders. the actual product pricing is consistent.” in summary, buywell’s channels were found to be synergistic, providing a seamless communications and purchasing experience. sf 3: brand management brand management includes portraying a consistent brand image and a track record of service delivery. the marketing manager commented: “a lot of customers trust us with their credit card details. they think, ‘oh, that’s buywell, they're ok’. we’re not a fly-by-night company or a dot-com. we’re here to stay… i think it’s the experience that the customer has in each channel. are we consistent? is our brand delivery consistent?” buywell has built a strong service brand by attention to quality control and consistency of service, building customer confidence across all channels. they are then able to leverage their brand across channels with a consistent brand image. we found that maintaining a service brand in multi-channel retailing of experience goods such as apparel had particular challenges, and was highly dependant on internal quality processes to maintain the consistency and veracity of the virtual experience. this was identified as an additional success factor for buywell, and is discussed in more detail later. for these factors, functional integration, channel synergy, and brand management, the practice at buywell confirmed existing theory. we now consider areas where findings from the buywell case challenged or extended existing theory. challenges or extensions to existing theory sf 4: information management information management involves keeping information up-to-date and systems integrated across channels. as expected, we found that buywell had highly integrated information systems, but in this area we also found some interesting innovations and exceptions to current thinking. integrity of the database has been identified in previous research as a critical first step to successful management of customer relationships (winer 2001). buywell understands this and constructively uses database information in interactions with customers. the channel campaign manager explained: “the best thing about our environment is that we have a customer database that is just fantastic. we have the details and behaviour of every customer who shops with us. so we can recognise that we sent mrs jones an email and then she shopped two days later with us. we are fortunate in that respect. you can never underestimate the value that a database provides. we have a customer relationship manager now, and a crm team. they are concerned with managing the database...” buywell have implemented several interesting innovations in information management. a major innovation is that buywell strives to identify face-to-face customers in the physical retail stores by ajis volume 12 no. 2 may 2005 154 encouraging them to provide their customer number, or to register if they are new customers. this assists in providing completeness of data regardless of channel, and provides and extraordinarily rich picture of multi-channel shopper behaviour. in an exception to much current research, we found that buywell, despite their commitment to maintaining rich and accurate data, were wary of over-dependence on quantitative data and automated tools. we found that as well as data mining from the database, buywell also had extensive processes in place for capturing qualitative information from customers and triangulating it with the quantitative data from their database. these processes are discussed in more detail in the section on “active scanning”. despite the high quality of their database, we found that buywell were cautious about extensive use of their database for automated customer service, and web techniques such as one-to-one marketing techniques, and mass customisation. “the website isn’t as good as the old csr. with a csr you [the customer] can have a conversation.” in summary, buywell places great emphasis on keeping their database up-to-date and accurate. an interesting innovation is their ability to integrate data on customer buying behaviour in physical retail stores with data from other channels. despite this, contrary to much current theory, buywell are wary of over-dependence on the database. they value the insights gained from personal customer contacts very highly, and use qualitative data extensively to complement the outputs of their information systems. sf 5: logistics management logistics management involves the processing of orders from multiple sources and provision of efficient and cost effective delivery, and is an essential component of a multi-channel strategy. as expected, we found that buywell had highly integrated and effective logistics processes. we also found that buywell had extended the best practices we were able to identify from literature for logistics management. in addition to attention to initial fulfilment, buywell pro-actively managed the returns process. as well as internal efficiencies, buywell had enhanced their logistics function with alliances. in addition, logistics excellence was used to further company growth. the general manager commented: “we’ve got an investment and infrastructure we can leverage. at one stage [a partner] had their own distribution centre, but we said ‘hey this is complicating things, why don’t you just come into here’”. to provide basic operational efficiency, the distribution system is tightly integrated with the main order system. orders from each channel are synthesised in a gateway before reaching the distribution centre. the distribution centre manager comments: “to the distribution centre it’s all the same. orders comes in by a variety of means, but once the order is processed it comes down to the distribution centre warehouse management system and we don’t know whether it’s been a phone order, a mail order, or an internet order.” returns and service recovery are frequently a problem with e-commerce. at buywell, pro-active management of the returns process extends the basic logistics process. quality processes within the company reduce the return rate, which further enhances the efficiency of the logistics function. buywell accepted that returns are a fact of life when selling apparel remotely, and aimed to manage them pro-actively: “in new zealand we do something we call easy returns. (the customer) only needs to affix a sticker that we send out to them to send an item back. they don’t need to go the post office to pay for anything. just put a sticker on and post it, and we debit their account $3.00.” ajis volume 12 no. 2 may 2005 155 as well as providing a service to customers, buywell also use the return process as a source of market intelligence: “on our returns form we have a section where people say why they are returning the item. obviously if 80% of people are saying the product is too long, we have a problem. so, comments are all analysed.” to enhance the efficiency of the logistics function, buywell has formed an alliance with new zealand’s leading postal service provider which now has a branch collocated with buywell. this improves cost-effectiveness of the delivery system. we found that as well as providing internal efficiencies, buywell’s logistics capability has become a point of differentiation that has enabled growth. several existing retailers with complementary products have formed co-branded partnerships with buywell. these have included a children’s wear company and a pharmaceuticals company who were interested in expanding their reach and channel range. the buywell brand is used to give confidence in the logistics and fulfilment offered by the remote channels, while the original brand provides product quality confidence. this provides a winwin situation as the general manager explained: “we could [act as an outsourcer] as a call centre business, but why do that…we prefer to share at the gross margin line…why earn a dollar of someone when we could earn $3.50 at the bottom line” in summary, as suggested by previous research, buywell are able to efficiently process orders from multiple sources. its logistics system is tightly integrated with the ordering system. in our view, buywell’s practice extends existing our understanding of the logistics function in a multi-channel organisation selling non-commodity items. buywell’s products are experience goods and therefore more likely to be returned than commodity items, so they have extended the logistics function to provide pro-active management of the returns process. an operational alliance with a courier or postal services provider can improve cost-effectiveness. finally, logistics capability has been leveraged to develop co-branded partnerships and contribute to company growth. sf 6: customer management we noted earlier that one aspect of buywell’s channel synergy is an integrated customer support through a contact centre. integrated contact centres are relatively common, but rather than seeing contact centres as a necessary evil, measured by call closures, buywell have developed their contact centre into the nerve-centre of the business. as part of their channel synergy, buywell actively encouraged customers of all remote channels to use their contact centre remote channels. buywell also engage in outbound communication to retain customers that have become inactive. we found that buywell’s customer management practices had effectively mitigated one of the disadvantages often cited for selling experience goods remotely – lack of personalised selling. once in touch with the contact centre, buywell’s customers are able to access empowered csr’s with excellent skills and resources for personalised selling, despite the lack of face-to-face contact. for example, buywell encourage customers to seek advice from csrs during a purchase decision via the catalogue or website. this might involve getting an opinion from the customer service representative (csr) such as whether one piece of clothing matches another or thickness of a particular fabric. portable racks along the walls of the contact centre display all apparel available from current catalogues, enabling csrs to check products in response to questions. so popular is this system, that some customers request a specific csr with whom they have formed a relationship. the marketing manager explained: “csrs can give information like the true colour of [an item of clothing], how good is it. a csr can also tell the customer if two items of clothing go together…the csr can be ajis volume 12 no. 2 may 2005 156 pro-active and say, “hey, it’s not exactly that colour, it’s a little lighter – or darker. do you still want it?” or, “this size 10 is actually larger than a normal size 10. you may want to go down a size”. the contact centre is also a key source of real-time market intelligence. the process for gathering that intelligence is described in the section on “active scanning”. in summary, we found that buywell extended existing understanding of customer management in multi-channel e-commerce, especially for non-commodity items such as clothing. in particular, considerable effort was made to empower csr’s to provide quality personalised selling. new insights into success factors for multi-channel e-commerce from an exemplar organisation this section describes aspects of buywell’s multi-channel strategy that were considered important to their success that we were not able to identify from previous studies in multi-channel e-commerce strategy. sf 7: active scanning to carry out market sensing, market-driven multi-channel organisations require the traditional e-commerce skills of customer management, including use of technology to manage customer relationships, process customer information, and offer enhanced customer service. but market sensing requires more it requires active scanning. buywell leads best practice in being market driven through its ability to closely align strategic decisions to customer expectations. customer expectations are garnered from extensive daily contact with them, as the general manager observes: “the great thing about this organisation is that we talk to every single customer maybe not online but we have a relationship with every customer either through the catalogue or through the contact centre. it’s quite unique.” and, further “we talk to upwards of 8000 (customers) every day. so we get a lot of real time feedback about how we perform, across all our channels.... major issues that are customer centric get dealt with very quickly by our contact centre manager, our sales and marketing manager, and our marketing department as appropriate. we spend a heck of a lot of time talking to our customers.” to be able to actively scan the market, employees must be motivated to pass to management the information gathered from customers. buywell uses incentives and training of csrs to provide the required motivation. the general manager explains: “…we have a structure in place where our customer service reps have a reporting process through their team leaders to the contact centre manage, and that filters up through the organisation. we are a fairly flat organisation anyway; we are not a hugely hierarchical place. we are training our staff constantly, and starting to incentivise them to get a bit of competition going.” information flows at buywell are shown in figure 1. information is obtained from customer complaints and praise are gathered from website and catalogue customers primarily through the csrs in the contact centres. these are passed on to team leaders and, where appropriate, to the contact centre manager or the sales and marketing manager, with significant information passing to the general manager. ajis volume 12 no. 2 may 2005 157 as a result of this flow of information through the organisation management gains insight into customer preferences, buying patterns, concerns, and areas for improvement. cus tom er s e rv ice r ep resen ta tive (c sr ) t eam leade r c on tac t c en tre m anage r g ene ra l m anage r s a les and m arke ting m anage r w ebs ite c us tom ers c a ta logue c us tom ers c on tac t cen tre c us tom e rs buywell feedback to customer via email or telephone customer feedback and information to buywell key: cus tom er s e rv ice r ep resen ta tive (c sr ) t eam leade r c on tac t c en tre m anage r g ene ra l m anage r s a les and m arke ting m anage r w ebs ite c us tom ers c a ta logue c us tom ers c on tac t cen tre c us tom e rs buywell feedback to customer via email or telephone customer feedback and information to buywell key: ajis volume 12 no. 2 may 2005 158 the contact centre does more than provide reactive customer management. the high level of customer contacts, and the personalisation and customer intimacy provided by the contact centre enable the practice of active scanning, as shown in figure 2. the incentives and value placed by the senior management on the market scanning provided by the contact centre further empowers and enhances the process of customer management. this creates a virtuous circle that provides a continuous flow of rich, real-time market information. figure 2: “virtuous circle” of empowered customer management and active scanning in summary, buywell have complemented their operational management of customer relationships across multi-channels with a sophisticated market-sensing mechanism, which is used to pass flows of information from customers to management. sf 8: process engineering we were not able to identify any studies of multi-channel e-commerce that specifically discussed this factor. this seems to be particularly relevant to the sale of non-commoditised items. goods such as apparel and home-wares have been considered difficult to sell over the internet because of the lack of ability to “touch and feel” (zeng et al. 2003). we found that an important part of the buywell brand experience was extensive quality controls to ensure consistency and veracity (for example, in colour and texture) between the virtual experience and the actual goods. buywell have a whole department dedicated to product consistency, particularly with respect to fit. for example: “we have different sized people who will try on garments. like, we’ll have a perfect size 12 who will try on a dress… and that is what we base our sizes on.” all buywell’s products are made to order to using measurements that are specific to the body shape of their target market, australasia. so, there is a good initial likelihood of fit. in a market where sizing of women’s apparel is extremely inconsistent, this offers confidence in the remote buying experience. the marketing manager explains: “i can be a size 10 in one retail store, an 8 in another store or a 12 in another. whereas at buywell we work really hard (on sizing consistency) and decrease returns by doing that. our size 10 is this size waist, this size hips, and that’s for new zealand. so we have buywell measurements.” attention to consistency increases customer confidence, and contributes to the brand image and the likelihood of repeat purchasing. in addition to consistency, a commonly raised objection to purchasing apparel remotely is the inability to check fabric colour and quality. buywell have quality control processes to ensure a realistic virtual depiction of the products, especially for catalogue production. quality controls are empowered and personalised customer management empowered and personalised customer management active scanningactive scanning ajis volume 12 no. 2 may 2005 159 exercised at all stages of the imaging and printing process. the marketing manager describes part of the process: “we have an established relationship with an imaging company. they are the ones who make sure (images) are correct. …and we’ll have samples of the clothing styles in every colour, so the imaging company can compare they can grab the garment and put it against the image.” in summary, attention to product consistency, and to veracity between the virtual presentation of products and the real-world versions is a key aspect of buywell’s multi-channel strategy that has received little or no attention in previous studies. this factor directly supports brand management, and also contributes indirectly to excellence in other areas, such as logistics, by reducing the return rate. this has significant implications for multi-channel retailers of other non-commodity items. summary buywell showed elements of each of the original sfs we identified from the literature: functional integration, channel synergy, brand management, information management, logistics, customer management and active scanning. for functional integration, channel synergy and brand management, we found that buywell confirmed but did not significantly extend the findings from previous studies. for logistics and customer management buywell’s practices extended the findings we were able to identify from previous studies, in particular by creating added value for the business as well as providing operational efficiency. for information management, we found that buywell were able to capture information about most face-to-face customer contacts, which provided a particularly rich picture of customer multi-channel behaviour. despite this, contrary to the findings of some previous research, buywell were wary about excessive dependence on their database at the expense of personal and qualitative insights and customer contacts. active scanning, and process engineering to provide consistency and veracity of experience were new factors added as a result of this study. active scanning is a concept from marketing literature, but we were unable to find any studies that related it specifically to multi-channel e-commerce. we also found relatively no previous literature considering the importance process engineering to provide consistency and veracity between real and virtual experience. this is particularly relevant to products such as apparel. table 1 provides a summary of the elements of each sf found in buywell and their relationship to previous research in multi-channel e-commerce strategy. discussion the findings of this case study confirm the criticality of sfs identified from the literature in a multi-channel retail apparel firm. some additional insights are offered to factors previously identified by research, and two new success factors – active scanning, and process engineering, to provide consistency and veracity of experience, are proposed. several limitations of this study should be considered. first, the channels tested are a subset of channels available to retailers. the study focussed on mail-order catalogue, internet website, contact centre, and physical retail stores. channels not included, though of increasing importance, include mobile technologies and in-store kiosks. second, this was a single case study, so the findings need to be further investigated in firms with characteristics similar to the case site. implications for future research may be drawn from these limitations. there appears to be some overlap and some correlation between the sfs. future research may determine the importance of each sf, and how each sf is related to others in its impact on firm performance. ajis volume 12 no. 2 may 2005 160 although buywell have a long and successful history as a multi-channel organisation, their internet channel is still at a relatively early stage of development. many multi-channel strategies are transferable across channels, but not all. future research involving a longitudinal study of the development of buywell’s internet could provide insights into how a successful multi-channel ecommerce strategy evolves as the internet channel matures. further, our study suggests that researchers in e-commerce strategy have largely ignored importance of process engineering. existing theory from other fields, such as marketing, diffusion of innovation, and information technology implementation could be examined for relevance and fit to e-commerce strategy. conclusions several contributions emerge from this research. we provide eight success factors for multi-channel e-commerce strategies incorporating the internet, mail-order catalogue, contact centre, and physical retail stores. we make a theoretical contribution by showing how customer management can be extended to market sensing in a multi-channel business model, and arguing for the importance of process engineering in providing a consistent experience between real and virtual channels. we provide best practice insights from an organisation that has proved highly successful in selling experience goods, a category often considered to be difficult to sell over the internet. in an era of increased customer purchasing options, the challenge is to keep customers within the channel mix of the organisation. buywell showed how this could be done through a focus on customers, a focus on process, and skilful use of technology. this research suggests that success factors in multi-channel e-commerce are not a “laundry list” of independent factors, but a set of highly interdependent processes and practices. further research is required to model these interdependencies. success factors may vary with the product line and the channel mix, buywell sell mainly experience goods, and the majority of their products are sold through remote channels. ajis volume 12 no. 2 may 2005 161 table 1: summary of success factor elements success factor sources strategies that confirm existing theory strategies that challenge or extend existing theory confirmation of existing theory functional integration barua, konana, whinston & yin (2001); ghoshal & gratton (2002); goersch (2002); gulati & garino (2000); lee (2001); schoenbachler & gordon (2002); steinfield (2002); westerman (2002); willcocks & plant (2001). shared organisational functions shared identity and meaning none channel synergy goersch (2002); kaufman-scarborough & lindquist (2002); schoenbachler & gordon (2002); porter (2001); reardon & mccorkle (2002); simons (2001); steinfield (2002). seamlessness of experience consistent pricing none brand management bitner et al (2002), goersch, 2002; kocas (2003); schoenbachler & gordon, (2002); willcocks & plant (2001). brand attribute leverage consistent brand image service delivery track record none challenges or extensions to existing theory information management barua, konana, whinston & yin (2001); goersch, 2002); lee et al (2001); weill & vitale (2002); willcocks & plant (2001); zettelmeyer (2000). data integrity systems integration ability to identify face-toface shoppers and integrate with data from other channels integration of qualitative and quantitative data about customers logistics lee & whang (2001); pyke, johnson & desmond (2001). ability to process orders from multiple sources partnership with postal services provider leveraging logistics capability to create alliances and growth active management of returns and return rate ajis volume 12 no. 2 may 2005 162 customer management barua, konana, whinston & yin (2001); lee (2001); kaufman-scarborough & lindquist (2002); schoenbachler & gordon (2002); simons (2001); willcocks & plant (2001). customer service customer information processing timely response to e-mail remote personalised merchandising empowered customer management “virtuous circle” of empowered customer management and active scanning new insights into success factors for multi-channel e-commerce from an exemplar organisation active scanning day (1994); day & hubbard (2002). active scanning a centralised contact centre providing rich and intimate personal contact supported by sophisticated processes to capture customer feedback can provide active market scanning for a multi-channel organisation process engineering none internal processes to ensure the consistency and veracity of real and virtual experience quality internal processes are required for multichannel retailing of noncommodity items mary tate, brent coker, and beverley hope © 2004. the authors assign to acis and educational and non-profit institutions a non-exclusive licence to use this document for personal use and in courses of instruction provided that the article is used in full and this copyright statement is reproduced. the authors also grant a non-exclusive licence to acis to publish this document in full in the conference papers and proceedings. those documents may be published on the world wide web, cd-rom, in printed form, and on mirror sites on the world wide web. any other usage is prohibited without the express permission of the authors. references balabanis, g., and reynolds, n.l. "consumer attitudes towards multi-channel retailer's web sites: the role on involvement, brand attitude, internet knowlege and visit duration," journal of business strategies (18:2) 2001. barua, a., konana, p., whinston, a.b., and yin, f. "driving e-business excellence," mit sloan management review (43:1) 2001. bitner, m.j., brown, s., and meuter, m.l. "technology infusion in service encounters," in: academy of marketing science journal, 2000. bitner, m.j., ostrom, a.l., and meuter, m.l. "implementing successful self-service technologies," academy of management executive (16:4) 2002. day, g.s. "the capabilities of market driven organisations," journal of marketing management (58:4) 1994. day, g.s., and hubbard, k.j. "customer relationships go digital," business strategy review (14:1) 2002. ajis volume 12 no. 2 may 2005 163 earl, m.j. "experiences in strategic information systems planning," mis quarterly (17:1) 1993. forrester "capturing cross channel dollars," 2002. ghoshal, s., and gratton, l. "integrating the enterprise," mit sloan management review (44:1) 2002. goersch, d. "multi-channel integration and its implications for retail web-sites," european conference on information systems, gdansk, poland, 2002. gulati, r., and garino, j. "get the right mix of bricks and clicks," harvard business review (78:3) 2002. kaufman-scarborough, c., and lindquist, j.d. "e-shopping in a multiple-channel environment," journal of consumer marketing (19:4) 2002. kocas, c. "evolution of prices in electronic markets under diffusion of price-comparison shopping," journal of management information systems (19:3) 2003. lee, h.l., and whang, s. "winning the last mile of e-commerce," mit sloan management review (42:4) 2001, pp 54-62. porter, m. "strategy and the internet," harvard business review (79:3) 2001. pyke, d.f., johnson, m.e., and desmond, p. "e-fulfillment: it's harder than it looks," supply chain management review (january/february) 2001, pp 26-32. reardon, j., and mccorkle, d.e. "a consumer model for channel switching behaviour," international journal of retailing and distribution management (30:4) 2002. schneider, g., and perry, j. electronic commerce thomson learning, boston ma, 2001. schoenbachler, d.d., and gordon, g.l. "multi-channel shopping: understanding what drives channel choice.," journal of consumer marketing (19:1) 2002, pp 42-53. shop.org "the multi-channel retail report 2.0," 2001. simons, l.p.a., steinfield, c., and bouwman, h. "strategic position of the web in a multi-channel market approach," internet research: electronic networking applications and policy (12:4) 2002. steinfield, c., bouwman, h., and adelaar, t. "the dynamic of click-and mortar electronic commerce: opportunities and management strategies," international journal of electronic commerce (7:1) 2002. weill, p., and vitale, m. "what it infrastructure capabilities are needed to implement e-business models," mis quarterly executive (1:1) 2002. westerman, g. "mixing bricks with clicks: organisation design for busineness-to-consumer electronic commerce in incumbent retailers," international conference on information systems, barcelona, spain, 2002. willcocks, l.p., and plant, r. "pathways to e-business leadership: getting from bricks to clicks," mit slaon management review (42) 2001a. willcocks, l.p., and plant, r. "pathways to e-business leadership: getting from bricks to clicks," mit sloan management review (42) 2001b, pp 50-59. winer, r.s. "a framework for customer relationship management," california management review (43:4) 2001, pp 89-104. yin, r.k. applications of case study research sage publications, thousand oaks, ca, 1993. zeng, m., and reinartz, w. "beyond online search: the road to profitability," california management review (43:4) 2003. microsoft word ajisvol10 1_final_edit.doc ajis vol. 10 no. 1 september 2002 29 organizational forms & knowledge types: a cognitive multi-actor approach rené j. jorna faculty of management & organization; university of groningen; po 800, 9700 av groningen, the netherlands. telephone: +31-50-3633625/3864 fax: +31-50-36362275 abstract in this article knowledge types are combined with organizational forms. knowledge is something individual actors have. this paves the way for a semio-cognitive approach to actors. organizations in which primary and organizational (secondary) processes are performed are viewed as multi-actor systems. organizational processes consist of tasks, such as planning, control and administration. these tasks are performed by individuals with knowledge. with regard to knowledge not only content, but also type can be discerned. three knowledge types are distinguished: tacit, coded and theoretical knowledge. in developing a conceptual framework for the cognitive analysis of knowledge management, i will illustrate and explain that the dominance of tacit, coded or theoretical knowledge matches wellknown organizational forms with labels such as machine bureaucracy, simple structure, professional bureaucracy and market. therefore, knowledge management is about the management of knowledge types and of representations of cognitive actors executing tasks in organizations. introduction three different perspectives on knowledge management can be distinguished. first, knowledge management can be seen as a part of human resource management. in that case the discussion is about the assessment of skills, abilities and intellectual powers of people (weggeman, 1997). in a second perspective knowledge management is a kind of upgraded information management. then knowledge management has a technical interpretation and refers to the relation between humans (and their knowledge) and information systems, such as management information systems, database systems, decision support systems and knowledge systems (laudon & laudon, 2000) a third interpretation of knowledge management is related to organizational studies in general and is about the control, the assessment, the maintenance, the coordination, the division and the spreading of knowledge (and learning) as such (jorna, 1998; sorge & warner, 2001). starting points in this interpretation of knowledge management are organizations and processes. it is an approach that works from the top, organizational structures and processes, downwards to people and tasks. it seems, however, that the debate about knowledge management resembles the well known box of pandora. organizational researchers started a debate of which the consequences could hardly be overlooked. knowledge and learning as such are not easy to define or operationalize. moreover, in most organizational (and information systems) literature a conceptual framework dealing with knowledge and learning is lacking, especially when one takes a perspective that goes from the bottom, individuals and tasks, up toward the top, namely organizations and processes. in this article, i want to argue that knowledge management is about the types and contents of knowledge (and learning) in which individual people with their minds, cognitive architectures and (mental) representations are the basic focus of attention. humans as information processing systems behave intelligently and acquire, integrate and use knowledge. organizational and societal principles are on the one hand the expression of thought processes of human individuals. on the other hand they are present, active and relevant because of the individuals that interpret and represent principles as beliefs, opinions and knowledge. this argument requires a conceptual framework at another level of aggregation than is usual in discussions of knowledge and knowledge management in organizations (laudon, 2001). instead of departing from organizational principles, coordination mechanisms and organizational forms, we will start from individual actors with their avowed and tacit knowledge and interpretations and representations. discussions about knowledge and knowledge management can be in terms of the content of knowledge or the type of knowledge. the content of knowledge will not be discussed in this article. what i want to emphasize is that the types (forms) of knowledge and their distribution among and dominance over people in relation to tasks in an organization are important. the conceptual framework of knowledge types that is presented in section 3 is used to operationalize knowledge management. it is also used to match, as i will argue, various organizational forms in organizations. in explaining this cognitive interpretation of knowledge management a conceptual framework dealing with (types of) knowledge within organizational settings is important. this framework is based on the view that an organization is a multi-actor system. i will discuss this view in section 2, in which also various kinds of actors will be categorized. this categorization has consequences for organizational forms. in section 3 i will discuss ajis vol. 10 no. 1 september 2002 30 various types of knowledge in detail. in section 4 i will relate the knowledge types to organizational forms. section 5 gives conclusions in relation to knowledge management, innovation and information systems. actors in a multi-actor perspective on organizations sorge (in sorge & warner, 2001) distinguishes two meanings of organization: 1. organization may refer to a social unity (or collectiveness) and 2. organization may refer to organizational properties. in other definitions of aorganization@ processes (and their coordination) are mentioned as cornerstones (thompson, 1967; gazendam, 1993). a process is aan action in its progress@ (van dale, 1995, p. 2378). i want to argue that in the analysis of processes that is the collection of progressing actions the executing entity in the form of the human information system often is neglected. a process needs a material carrier. a process runs on something, just like software runs on a computer. in production environments the carriers are the machines, the instruments, the tools, the computers and of course the employees. in service and administration environments the material carriers of processes are the employees and the computers. it is interesting to see, however, that both the details and the interpretations with regard to what the employees are and what mental mechanisms are involved in performing tasks are vaguely formulated. in many organizational studies the realization of the process(es) is often a highly abstract, almost metaphysical affair (see sorge &warner, 2001). from a knowledge (management) perspective in particular, this abstract meta-individual perspective is odd, unwanted and unnecessary. even if one views knowledge from a platonic supra-individual perspective, it still is directly connected to the human mind. individuals, alone or in groups, execute tasks by which they use interpretation, intelligence and knowledge. even coordination, planning and control as separate tasks normally called the organizational processes (see section 4) are, so to speak, in the minds (and hands) of individuals. furthermore, an organization originates from the individuals who are part of the organization and who can think of that organization. without thinking of an organization, there is no organization, even if there are constructs and artifacts as buildings and machines. as sorge (2001, p. 7) says aof course, there would not be any organization behavior without human behavior.@ my proposition is that the basic ingredients of an organization are the intelligent cognitive actors. this means that there exists a nesting of a) actors within organizations and b) organizations within actors. the first nesting means that an organization always consists of a collection of actors. this makes an organization a multi-actor system. the second means that a set of actors can form an organization, but only in the sense that an actor can think of and reason about the role of the other actors. the other actors in turn have opinions and beliefs, that is to say representations, about the first actor. this implies that an organization as a construct consists of the sharing of individual representations. the key notions, here, are representation and interpretation. an organization is a representation (and an interpretation) in the eyes of (intelligent) actors. this means that an organization as a human-made conceptual construct is not an object such as a car, a bridge or a computer. if we speak about actors with representations and interpretations, the actor is the concrete human individual and not some metaphorical entity. properties as awill@, acognition@, amemory@ and amotives@ apply to individuals. literally, or materially speaking, only individuals have these characteristics. organizations as such do not have a memory, knowledge or the ability to learn. in the remainder of this article an organization is conceived of as a multi-actor system consisting of natural actors and coordination mechanisms. if we look at actors we can ask what it means to be a cognitively and psychologically plausible actor. or more neutral, what kinds of actors can be discerned and how are they structured? later we will discuss the building blocks of actors in general and of intelligent actors in particular. in general three kinds of actors can be discerned: a) response function actors, b) representational actors and c) representational response function actors (see figure 1). an actor is an acting entity. we speak about a cohesive, structured and organized unit. this entity operates in an environment. this entity is an actor, because it is self-contained and strives toward continuation. looking at the psychological characteristics, it has perception and interaction including the possibility of learning in the sense of habit formation. this kind of actor does not have internal representations. its cognitive domain is absent or empty. i call this actor a response function system (rf-system). it can be compared with the ant in the sand (simon, 1969). in discussing complex behavior of systems simon stated that the behavior of an ant on the sand can be called complex, although not intelligent, because the behavior is a function of the complexity of the irregular sandy environment that the ant has to cross. another kind of actor is what i call a representational system 7-system). this actor has representations and is able to internally transduce external events into its cognitive domain. this representational system has representations, autonomy and transducer components. the interaction or the social context is problematic, which means that there is no device that semantically interprets causal input and output. as far as we speak about interaction it is merely a low level reaction to stimuli. if we look at present day cognitive science, most linguistically and logically oriented researchers have such a kind of actor in mind (phylyshyn, 1984). in most classical cognitive science the social environment is not the subject of research (clancey, 1997). representations ajis vol. 10 no. 1 september 2002 31 and reasoning mechanisms are investigated in the cognitive environment of the individual and not in its social world. the third kind of actor is the representational response function system (rrf-system). this actor incorporates a really intelligent, interactive and cognitive system. it is able to perceive, interact, represent and be autonomous. cognitive processes include symbols, operations and semantic interpretable response functions. rrf-systems behave on the knowledge level, as newell called it. "there exists a distinct computer systems level, lying immediately above the symbol level, which is characterized by knowledge as the medium and the principle of rationality as the law of behavior." (1982, p. 99) newell is proposing this knowledge level for natural (humans) as well as for artificial (computers) intelligent systems. the actor equipped with the integration of representations and responses has knowledge. "knowledge", says newell, "is whatever can be ascribed to an actor, such that its behavior can be computed according to the principle of rationality." (newell, 1982, p. 105) the principle of rationality is expressed in the belief that an acting person will undertake those actions by which his goals are reached (simon, 1947/76). figure 1; overview of kinds of actors the sub-division in kinds of actors is based on (cognitive) psychological components, namely: a) perception, b) interaction (including learning in the sense of habit formation), c) representation and interpretation (including learning in the sense of chunking and creation) and d) autonomy and self-consciousness (self-representation). with perception i mean that a system must be able to accept input in a general sense. this input may include visible, audible and tangible stimuli and the accepting system may vary from a lobster to a human being or even a computer system. interaction is the process by which a system has contact with its environment resulting in actions and motor behavior. stimuli as input in the system lead to output in the sense of responses. the reaction patterns of the system may result in learned behavior, that is the formation of habits. a system that internally symbolizes the environment is said to have and use representations. representations consist of sets of symbol structures on which operations are defined (jorna, 1990). examples of representations are words, pictures, semantic nets, propositions or temporal strings (kosslyn, 1980; anderson, 1990). a representational system learns by means of chunking mechanisms and by the creation, transformation and manipulation of symbols. details of the components can be found in cognitive science literature (posner, 1989; newell, 1990; stillings et al., 1995). a system is said to be autonomous, self-organized and self-conscious if it is able to have a representation of its own (physical and conceptual) position in the environment. this means that the system also has selfrepresentation. an autonomous system has reconstructing representational interaction patterns. actors in the form of response function systems, with only perception and (stimulus-response) interaction, are at the lowest level and can not be called intelligent actors. actors, such as representational response function ajis vol. 10 no. 1 september 2002 32 systems, have self-organization (self-representation), including perception, interaction and representations and are at the highest level. this form is what we regularly call an actor that is reflective, intelligent and thoughtful. human beings are instantiations of these intelligent actors. computers are said to have representations, but not self-organization and self-representation. if computers can do tasks for which it is indicated that humans perform these tasks with intelligence, computers according to cognitive scientists have intelligence, at least to a certain degree. the componential structure of actors is only one part of the conception of an organization as a multi-actor system. the other part consists of coordination mechanisms, including communication, that is used to regulate, control, attune and adjust the multitude of actors (gazendam, 1993). coordination mechanisms are realized in so called organizational forms. it should, however, be borne in mind that coordination mechanisms also exist as representations in the cognitive system of actors. various coordination mechanism can be found, such as the standardization of tasks or products, the top-down structuring of hierarchies or the implementation of trust and loyalty. i will return to the details of these mechanisms in section 4. coordination mechanisms also have to match with the various kinds of actors in the multi-actor system. we can make a distinction in homogeneous and in heterogenous multi-actor systems. homogenous means that all actors are of the same kind. if, for example all actors are response function systems, the actors have perception and interaction. to take up the example of simon's ant we are referring here to a group of ants perceiving and interacting with each other. coordination is only defined in terms of reactions to the behavior of other actors. to make a provocative statement, i state that although organizational theory and the larger part of economics speak about the coordination of multiple intelligent actors, in actual practice the actors behave as response function systems. interpretation, representation and cognitive components are not used to found the overt behavior of individuals (klos, 2000; van den broek, 2001). another example of a homogenous multi-actor system is the one where all actors are representational response function systems. each actor has perception, interaction, representation and autonomy and (inter)acts in a semantically rich and intelligent way. a collection of human information processing systems is an example of such a multi-actor system. this is what we normally call an organization in which various coordination mechanisms regulate the interaction. as can be imagined many multi-actor systems are heterogenous. most actors in such a system may be representational response function actors, however some are only representational and others are merely response function actors. my argument is that the only meaningful incorporation of actors in organizational theory is in the form of a representational response function system. this allows for the use and integration of cognition, knowledge, interpretation and sign (symbol) manipulation. this combination of terms has consequences for knowledge management and refers to cognitive science (posner, 1989) and to semiotics (nöth, 2000; michon, jackson and jorna, 2001). the coordination and communication between actors is expressed in signs. this implies that an organization is also a semiotic entity. it is semiotic as an entity, because of its participants, its processes and outcomes. with semiotic i mean that an organization as an artefact, a construct or a representation, is basically a sign type or sign token. the interesting thing about looking at an organization as a semiotic entity is that it reveals the many different sorts of signs turning up in the communication and information structures between the various actors. in the discussion of knowledge management in organizations the basic approach has to be to look at the actors as human information processing systems. managing knowledge not only means managing behavior, but it also means managing representations in one way or another. knowledge is inherent to cognitive systems. managing this knowledge is therefore related to suitable coordination mechanisms. not every coordination mechanism is suitable in every multi-actor situation. furthermore, managing knowledge is not in the first place related to the content of knowledge, but especially to the type of knowledge. managing the content of knowledge is inherent to the internal structure of the domain under consideration. managing types (or forms) of knowledge is essential in (the operationalization of) knowledge management. i will discuss the various types of knowledge in the next section. ajis vol. 10 no. 1 september 2002 33 figure 2: homogenous and heterogenous multi-actor systems knowledge and knowledge types knowledge, information and data are closely connected. the raw material of data is the unformatted unstructured material in the world (jorna & simons, 1992; schreiber a.o. 2000). data are such things as the various signals acoustic, visual, tactile and otherwise that are around us and that can be interpreted as (consisting of) information. the formal relation between data and information is that information is a structuring of data that reduces uncertainty. the more a message reduces uncertainty, the higher its information value. in a more informal way it could be said that information is interpreted data. knowledge is the interpretation of information of an individual using his own history, experiences and interpretation schemes. that is the reason why the same information may lead to different knowledge for different individuals. formulated more dynamically, the relation between data, information and knowledge is as follows: a person (a human information processing system) receives data, and through the knowledge he or she possesses, data are turned into information. ultimately, this process changes the former knowledge of the interpreting person. it is almost impossible to define types of knowledge unambiguously. traditionally conflicting epistemological, psychological and cultural categories can easily be distinguished (nooteboom, 2000). to avoid debates, we start from a semiotic perspective. we choose the semiotic stance because we believe that sign production and sign understanding are fundamental in human cognition and human communication. based on the information concepts of boisot (1995), we define a dynamic model for knowledge types (for details see: van heusden & jorna, 2001). we start with three types of (semiotically inspired) knowledge: a) tacit or perceptual, b) coded, and c) theoretical knowledge. this division in knowledge types refers to the number of semiotic dimensions involved in the representation. ajis vol. 10 no. 1 september 2002 34 the first type is tacit or perceptual knowledge. it starts from a perception of difference. the first semiotic step is always to recognize the situation in terms of a situation (or state of affairs) you already know. we find ourselves, at this point, at the level of asense making@ (eco, 1976; weick, 1995, choo, 1998). perceptual knowledge as one-dimensional representation underlies what michael polanyi has coined as apersonal knowledge@ (polanyi, 1966). he describes the process involved in this tacit knowledge as being aaware of that from which we are attending to another thing, in the appearance of that thing@ (p. 11). tacit knowledge often is bodily knowledge: awhen we make a thing function as the proximal term of tacit knowing, we incorporate it in our body or extend our body to include it so that we come to dwell in it@(p. 16). this type of knowledge can not be coded, it is about concrete experiences, and it can be shared only with those who are co-present. differentiation of this type of knowledge can be done by the measurement of detail; some of us have an acute sense of differences; they are intense perceivers (detailed tacit knowledge), while others tend to overlook most differences and concentrate on identities (broad tacit knowledge). knowledge of details is of course relative to a certain domain, and to the knowledge of others in the same field. thus a professional will be able to perceive more when looking at a certain activity than an amateur. one cannot rely upon verbal reports. personal knowledge has to be determined through the analysis of behavior, that is, of what someone is able to do in a certain situation. a knowledge type becomes two-dimensional when, out of the relation between the two events in the process of representation, a new dimension emerges. this new dimension is the dimension of the sign as code. codes range from an undefined set of icons or diagrams to a well-defined set of mathematical symbols. differentiation in codes is constituted by the size of the set, the well-definedness of the elements in the set, the concatenation rules and the degree of disambiguation. the two-dimensional sign requires communication and makes communication easier. the two-dimensional sign is therefore basically a social sign. although categorization in codes is (mostly) conventional and rests upon the grouping of features, this knowledge enables us to communicate about our experience. in other words, the diffusion of knowledge becomes easier where codes are available (boisot 1995). externalization and diffusion require coding. the twodimensional sign is also tied to a context, but this is not any longer the context of a concrete state of affairs. in terms of boisot (1995): the diffusion of the sign now takes place along the lines of a social community. codes can be differentiated by taking into account the number of elements and combination rules a code consists of, as well as the degree of ambiguity allowed. thus musical notation systems are more strongly coded than natural languages. at the lowest level of codification, codes tend to dissipate into concrete, that is, onedimensional perceptual knowledge. therefore, in the use of images and metaphors, coded knowledge comes closest to the non-coded concrete knowledge of the first phase. further details about the operationalization of weak and strong codes can be found in goodman (1968; see also jorna, 1990). the third type of knowledge, theoretical knowledge, emerges when to perceptual difference and codification a third dimension is added, which is that of the logical or formal relation. knowledge is theoretical when coded signs relate to the events represented, not on the basis of a convention, but on the basis of formal or structural qualities. basically, theoretical knowledge is knowledge about the necessary relations between events and categories of events. most scientific knowledge belongs here. it is often the result of answers to the question: why? it should be clear that theoretical knowledge makes diffusion even easier than does coded knowledge; theoretical knowledge is not, in principle, conventional, but universal. although codes are needed to communicate theoretical knowledge, the knowledge goes beyond coding. codes are the vehicle. however, theoretical knowledge is basically independent of historical and personal contexts. various attempts to differentiate theoretical knowledge can be found. one such a differentiation is in terms of levels of abstraction. the more entities belong to a concept the more abstract the concept is. mammal is more abstract than dog which is more abstract than poodle. another proposal is the more complex the knowledge is, the more abstract it is. details of the operationalization of theoretical knowledge can be found in bunge (1979). in van heusden & jorna (2001) we developed a knowledge space for the different types of knowledge (see figure 3: the k-space). all forms of sensory (tacit) knowledge are on the horizontal axis. this knowledge is not yet coded, it is not theoretical, and it ranges from the very rough or broad to the highly detailed. in the horizontal plane we find coded knowledge, which is not yet theoretical. however, codes presuppose sensory knowledge and therefore the horizontal axis is part of this plane. finally, in the three-dimensional cube we find all kinds of theoretical knowledge, ranging from the concrete to the highly abstract. again, as theoretical knowledge presupposes coding and sensory knowledge, the two axes are also part of the theoretical cube. movements through the knowledge space can now be schematized as movements along the one-dimensional linear axis of perceptual knowledge, movements through the two-dimensional plane of codification, and, finally, movements through the three-dimensional theoretical space. conversion of knowledge, in our view, occurs when one passes from one dimension into another, e.g., when tacit becomes coded. ajis vol. 10 no. 1 september 2002 35 insert figure 3: the knowledge space one can schematize different learning processes within this scheme: the learning of a skill means moving along the sensory knowledge axis. the learning of codes, that is, moving through the codification plane, eventually goes together with a movement along the sensory knowledge axis. the learning of abstractions is through a movement upward, eventually in combination with two other movements toward the right end of the sensory knowledge axis and the far end of the codification plane. scientific experiments, for instance, would be instances of highly detailed, highly abstract, but weakly coded knowledge; one would find this knowledge near the front on the right of the knowledge space. in reality tacit, coded and theoretical knowledge are not in a clear-cut way divided over actors executing tasks. it is the distribution of the types within and over actors and the dominance of one type in a task or a collection of tasks that matters. as indicated, theoretical knowledge builds upon coded knowledge which in turn builds upon tacit knowledge. this means that even if the dominant type is theoretical, tacit knowledge is always present. i repeat that fundamental in our approach is that humans as information processing systems possessing knowledge and cognition execute tasks, and that, irrespective of the content of the knowledge used in executing the task, the type of knowledge can be characterized. the k-space is a vehicle to assess types of knowledge. an individual executing a task with the aid of knowledge can move through the k-space over time if his/her knowledge type changes. the same holds for an organization as a multi-actor system. knowledge types and organizational forms the discussion about organizational forms refers to coordination, cooperation and the design of organizations. it is not primarily about the primary process within the organization. primary processes are about what an organization produces, yields or brings forth, such as hospitals, which through treating and curing patients, produce health. the way these primary processes are structured concerns the organizational processes, also called the secondary processes (porter, 1985). the organizational processes can be implemented in the organization in three different ways. first, they can be mixed up with the primary processes. secondly, they can be lifted out and separated from the primary processes in independent units. thirdly, the organizational processes can completely be cut off from the original organization and put into a new organization, whose primary process is then the execution of the organizational processes. for reasons of clarity i will treat the variations as similar: the organizational process. organizational processes can be described in terms of their constituting tasks: control, planning, administration, monitoring, communication, maintenance and contracting (daft, 2000). control concerns the task in which the right and power to command, decide, rule and judge is executed. planning is the task in which the courses of actions for staff, products, machines, vehicles and activities at a strategic, tactical or operational level are determined. administration is here meant to concern the task(s) by which all kinds of information regarding the primary (and organizational) processes is set down and established. monitoring means the task to follow and assess the progress of the various products, services or primary processes in the organization. communication is the task to remain in contact, orally or in written form, with the other members inside as well as outside the organization. maintenance concerns the support and up keeping of the means of production and services. ajis vol. 10 no. 1 september 2002 36 contracting is the task to manage and arrange the promises and appointments within and outside the organization in a formal (legal) manner. as indicated in section 3, the types of knowledge are tacit, coded and theoretical. although in practice all types of knowledge exist for every individual, this does not mean that uniform distributions exist. it is to be expected that with regard to various tasks one type is dominant. the determination of the dominance of a knowledge type can be accomplished for one individual, for all individuals and for the separate tasks out of which the organizational process exists. theoretically, eight possible combinations of dominance ( ) and subordination ( ) can be determined. however, the absence of any or the presence of all dominant type(s) are equivalent. seven combinations remain: from tacit: , coded: and theoretical: , to tacit: , coded: and theoretical: knowledge. now we will turn away from the knowledge types and organizational processes and direct our attention to the other end of the spectrum where the organizational forms are labeled. many labels can be found (sorge, 2001), but the most prominent ones are from thompson, mintzberg and boisot. with regard to forms of organizations, thompson (1967) describes coordination within an organization in terms of (task or process) interdependence, such as pooled interdependence, sequential interdependence and reciprocal interdependence. pooled interdependence concerns independent departments, that is to say a divisional structure. sequential interdependence relates to the situation where the output of a is the input for b. in reciprocal interdependence the output of a is the input for b and the output for b is the input for a. in describing the development of organizational forms, mintzberg (1983) enumerates five forms: a simple structure, a machine bureaucracy, a professional bureaucracy, a divisionalized form and an adhocracy. boisot (1995), who in a similar way as mintzberg deals with the evolution of organizations, makes a distinction in fief, clan, market and bureaucracy. boisot distinguishes organizations in terms of the codification, the concreteness and the diffusion of information. the organizational forms discussed by thompson, mintzberg and boisot are based on decomposition structures, ways of coordination and the characterization of information. other divisions take into account the authority relation for example, the subdivision into monarchy, bureaucracy, aristocracy, meritocracy, democracy or technocracy (see also sorge& warner, 2001) -, institutional factors (williamson, 1975) or organizational strategies (prospector, defender, analyzer, reactor; miles & snow, 1978). the question is how organizational forms fit into the distributions and dominance of types of knowledge. i will briefly characterize the various organizational forms by referring to them in terms of knowledge types. the combinations are not the result of empirical research. they are the result of analytical thinking and could be reformulated as hypotheses. a clan consists of a limited group of actors that cooperate on the basis of trust, sometimes justified by family or very close friendship relations. boisot (1995, p. 259) says that athe term clan refers to a non-hierarchical group of limited size transacting on the basis of shared intangible knowledge and values.@ these values are implicit and well-known by the members of the clan, but they are very difficult to formulate. clans are small and local. different clans have different interpretations of what trust, loyalty, responsibility and obedience mean. because of the condition of physical proximity a large clan splits into sub-clans. the organizational process within a clan, rooted in trust and loyalty, does not work in impersonal relations. clans do use coded knowledge, but the interpretation of the codes is guided by knowledge of the tacit type. theoretical knowledge is largely absent. questions about and explanations of awhy governance and coordination are what they are@ are neither posed nor given. the distribution of knowledge types in a clan is: ta: ; co: ; th: . the divisionalized form is only one of the many structures where tacit and coded knowledge are dominant and theoretical knowledge is subordinate. in a divisionalized form the middle line (in mintzberg=s (1983) terms) is responsible for the development of new businesses and the control of operations. often divisions are formed within the larger organization. these divisions operate independently, but also have to communicate with other divisions. the organizational process to coordinate, cooperate and communicate is by means of rules and procedures which are often available and used in coded form. the distribution of knowledge types is: ta: ; co: ; th: . in a machine bureaucracy coded knowledge is dominant. these bureaucracies are famous for their self-willed search for procedures, guidelines and scripts. everything within the organizational process has to be coded, otherwise it is not suitable and usable. tacit knowledge is avoided and if it turns up it will be converted into coded knowledge. theoretical knowledge is also absent because the rules are so called self-evident. explanations in terms of theories, models or scientific regularities or laws are not used. often these theories are considered as undermining the procedures and rules formulated in codes. the distribution of knowledge types is: ta: ; co: ; th: . in a professional bureaucracy (mintzberg, 1983) the operational core consists of highly trained and wellspecialized professionals. many of them may have an academic background. this means that the organizational processes of coordination, control and planning are executed in close relation to the domain structure of the professions. the knowledge the professionals use is coded in the sense that it is represented and documented in rules, procedures and scripts. the knowledge is also theoretical because in answer to aquestions why@ explanations, theories and alogical necessities@ can be formulated. very little knowledge is tacit, which can be ajis vol. 10 no. 1 september 2002 37 illustrated by the long explicitly structured training period the novices in this kind of organization have to undergo. examples of these organizations are hospitals, universities and ministries in certain fields (agriculture, economics or justice). the distribution of knowledge types is: ta: ; co: ; th: . in organizational studies (sorge & warner, 2001) no clear-cut organizational form matches the dominance of tacit and theoretical knowledge. in this situation coded knowledge is subordinate. if we search for organizational situations where the distribution: ta: ; co: ; th: , is applicable the most striking example is a research unit 7&d). the dominance of theoretical knowledge is clear in such units, because the development of theories is the reason for their existence. however, much of the knowledge that is developed is immature, provisional and tentative. this means that codification may be the result of the research, but it is not the knowledge type by which the organizational process works. the tacitness of knowledge is relevant because oral explanations, demonstrations, intuitive insights and sophisticated imitation are essential in hypotheses and educated guesses. everyone working in university research is familiar with the tension between the real research attitude (tacit and theoretical knowledge) and the red tape (coded knowledge) of the university officials. concerning the dominance of just theoretical knowledge, that is: ta: ; co: ; th: , it is very difficult to match this with one organizational form in particular. it should be an organization where theories dominate the coordination, cooperation and communication of and within the organization. tacit and coded knowledge are of course present but they are guided by the true theoretical insights. the only real organization that comes close to this division of knowledge types is, i guess, the roman catholic church. it is relevant not to identify theoretical knowledge with scientific knowledge. scientific knowledge is an example of theoretical knowledge, but not the other way around. although the bible as coded knowledge is very important for the roman church, the interpretation of this document is guided by theoretical knowledge. the islam could also be an example, but in contrast to the roman catholic church the islam does not have a centralized authority with regard to the doctrine. this example also makes clear that other principles besides the division and dominance of knowledge types should not be neglected. the roman catholic church has a hierarchical structure, whereas the islam has a network structure (beinin & stork, 1997). a market is characterized by many autonomous organizations. the coordination as an organizational process is brought about by means of an ainvisible hand@, it is realized outside the organization itself. the interaction is based on rivalry and competition. in theory the organizations are said to be equivalent, but in practice they are very different. the organizations internally work on the basis of local information, whereas external comparison is achieved in terms of prices. because in reality the organizations are so diversified, this implies that an organization with a great deal of tacit knowledge competes or negotiates with an organization with a lot of coded and theoretical knowledge. however, two organizations both possessing dominant coded knowledge may also negociate with one another. the variety of organizations involved in the market, explains the dominance of all types of knowledge (ta: ; c: ; th: ). it also explains the complexity of market situations. the various organizational forms are interpreted in relation to the organizational process. the forms are idealized in relation to the division and dominance of the knowledge types. if one goes one level deeper into the tasks that are the building blocks of the organizational processes, it might be the case that for control coded knowledge is dominant in the organization, whereas for planning in the same organization tacit knowledge is dominant. in order to test the way in which organizational forms and knowledge types match, it is necessary to collect data at the level of tasks, task execution and individual actors. in table 1 the combinations of dominance of knowledge types and organizational forms are summarized. i repeat that the combinations have to be conceived as hypotheses; until now no empirical data is available that could corroborate this matching. conclusion: knowledge management, organizations and empirical research the purpose of this article is to combine knowledge types with forms of organizations. conceptually, a big distance had to be bridged. knowledge is something individuals have within their cognitive system. organizations are social units or collectives consisting of actors who cooperate and coordinate their divided and distributed tasks as part of primary (and organizational) processes. i bridged this gap by first interpreting organizations as multi-actor systems. this paved the way for a semiocognitive approach to actors. second, i gave a behavioral and cognitive interpretation of the characteristics actors (might) have. based on these features various kinds of actors could be discerned. third, i argued that, besides the content of knowledge, also the type of knowledge is important. three types of knowledge were distinguished: tacit, coded and theoretical. finally, i analyzed the organizational process in terms of tasks, such as planning, control and administration, and i showed that various distributions (and dominance) of knowledge types match well-known organizational forms with labels such as machine bureaucracy, simple structure, clan or market. ajis vol. 10 no. 1 september 2002 38 organizational form tacit coded theoretical simple structure; adhocracy: clan; fief; family divisionalized form machine bureaucracy professional bureaucracy r&d department; university research no application immediately recognizable market, web, network (organizational processes are also external) no application immediately recognizable (or market, see text) table 1: organizational forms in terms of the dominance of knowledge types for organizational processes. i argue that this gives an operationalization of knowledge management with respect to the tasks that constitute the organizational process. if knowledge is used in the organizational process, and no one would deny that, then in order to be managed that knowledge has to become observable and ameasurable@. the point i also try to make here is that with regard to organizational forms there are, apart from random, also less random matches. i will formulate some conclusions in statements concerning knowledge management and organizations. i will end with briefly mentioning three kinds of empirical research that are now being conducted on the basis of the conceptual framework discussed in this article. knowledge management: knowledge management is related to the management of the representations of actors. management of knowledge means management of the content of knowledge as well as the type of knowledge. it is possible to discern various types of knowledge: tacit, coded and theoretical knowledge. the various types of knowledge are co-present. this holds for the primary processes as well as for the organizational processes. tacit knowledge or coded knowledge, but also the combination of coded and theoretical knowledge may be dominant. organization: organizations are multi-actor systems, consisting of actors and coordination mechanisms. with regard to human organizations, it can be bluntly stated that if there are no human information processing systems, there is no organization. the key terms for humans as information processing systems are cognition and representation. humans have, develop, change and use representations: of their colleagues, their tasks, the rules, routines and procedures in the organizations, of information systems, and coordination mechanisms. coordination mechanisms are incorporated in organizational forms. organizational processes are also executed by human actors. empirical research: on the basis of the foregoing conceptual framework empirical research is now being conducted in the area of medical product innovation, the implementation of new standards for information provision and the implementation of planning supporting software. with regard to knowledge management in medical product innovation, two innovation projects are followed over time and two already finished projects are analyzed in terms of the change of dominant type of knowledge. concerning medical product innovation various contents of knowledge are determined, such as clinical expertise, mechanical engineering, electronics, animal experiments and patents. for each content domain the change, the interaction and the diffusion of knowledge types are investigated. the goal of this research is to add a knowledge management instrument to existing project management methods. those methods generally lack a knowledge management component. regarding the information provision in the royal dutch army empirical research is being conducted with the aid of questionnaires and observation schemes. for two large units, the so called awhites@ (office people) and agreens@ (battle people), the use and implementation of information provision methods (itil) is assessed and analyzed (van der voort, 1999). the goal is to explain the different implementation rate and scope of itil in terms of the different distributions and dominance of knowledge types with regard to the various tasks for the various actors. concerning the implementation of planning software in a large hospital, we started a longitudinal empirical research (sjarbaini, 2001) in which we follow the change in the distribution and dominance of knowledge types during the implementation and use of planning support software. from a knowledge management perspective questions are awhy does part of the implementation possibly fail@ or awhy does the implementation last ajis vol. 10 no. 1 september 2002 39 longer than expected@? more than forty planners are questioned and observed in connection with their knowledge types for three sub-tasks of planning. three what we call aknowledge snapshots@ are taken: a) before the implementation of the planning software, b) during the implementation of and training with the software and c) 6 months later. we expect that for certain sub-tasks the knowledge will change from tacit into coded, whereas for other sub-tasks tacit remains tacit. the last situation may hamper the successful use and acceptance of the software (van wezel & jorna, 1998). the assessment of knowledge types gives an indication about the duration and speed of the implementation of innovation. innovation as a radical change in ways of working or task execution is bounded by the types of knowledge that are present in an organizational structure. if the types of knowledge are awrongly@ distributed or the awrong= one is dominant, the innovation process may be seriously impaired. references anderson, j.r. (1990). the adaptive character of thought. hillsdale, nj: lawrence erlbaum associates. beinin, j. & stork, j. (eds.) (1997). political islam. berkeley: university of california press. boisot, m. (1995). information space. a framework of learning in organizations, institutions and culture. london: routledge. bunge, m. (1979). treatise on basic philosophy (iv volumes). dordrecht: reidel. broek, h. van den (2001). on agent cooperation: the relevance of cognitive plausibility for multi-agent simulation models of organizations. som phd-thesis: university of groningen. choo, c. w. (1998). the knowing organization. how organizations use information to construct meaning, create knowledge, and make decisions. new york: oxford up. clancey, w.j. (1997). situated cognition: on human knowledge and computer representations. cambridge: cambridge university press daft, r.l. (2000). organization theory and design (7 th edition). cincinatti: south western thompson. dale, van. (1995). groot woordenboek der nederlandse taal (12 e druk). utrecht. van dale lexicografie. eco, u. (1976). a theory of semiotics. bloomington: indiana university press. gazendam, w.h.m. (1993). variety controls variety: on the use of organizational theories in information management. groningen: wolters-noordhoff. goodman, n. (1968). languages of art. an approach to the theory of symbols. indianapolis: bobbs-merrill. heusden, b. van & jorna, r.j. (2001). toward a semiotic theory of cognitive dynamics in organizations. in: kecheng liu, (ed.) organizational semiotics. amsterdam: kluwer. jorna, r.j. (1990) knowledge representation and symbols in the mind. probleme der semiotik. tübingen: stauffenburg verlag. jorna, r.j.(1998). knowledge management really is about representations. the semiotic review of books 9.2 : 5-8. jorna, r.j. & simons, j.l. (red.) (1992). kennis in organisaties: theorie en toepassingen van kennissystemen. muiderberg: coutinho. (knowledge in organizations: theory and applications of knowledge systems) klos, t. b. (2000). agent-based computational transaction cost economic. som phd-thesis university of groningen. kosslyn, s.m. (1980). image and mind. cambridge: harvard university press. laudon, k.c. (2001). organizational information and knowledge. in: sorge & warner, m. (eds.) the iebm handbook of organizational behavior. london: thomson learning (p. 297-307). michon, j.a., jackson, j.l. & jorna, r.j. (2001). semiotic aspects of psychology. in: r.posner, k. robbering and t.a. sebeok (ed.). semiotics: a handbook on the sign-theoretic foundations of nature and culture. berlin: de gruyter. (in press) miles, r.e. & snow, c.c. (1978). organizational strategy, structure and process. new york; mcgraw-hill. mintzberg, h. (1983). structures in fives: designing effective organizations. englewood cliffs: prentice hal. newell, a. (1982). the knowledge level, artificial intelligence, 18, 87-127. newell, a. (1990). unified theories of cognition . cambridge: harvard university press. nooteboom, b. (2000). learning and innovation in organization and economics. oxford: oxford university press nöth, w. (2000). handbuch der semiotik (2e auflage). stuutgart: metzler verlag. pylyshyn, z.w. (1984). computation and cognition. cambridge: the mit press. polanyi, michael (1966). the tacit dimension, london: routledge. porter, m.e. (1980). competitive advantage: creating and sustaining superior performance. new york: the free press. posner, m.i. (ed.). (1989). foundations of cognitive science. boston, mass.: the mit-press. ajis vol. 10 no. 1 september 2002 40 schreiber, g., akkermans, h., anjewierden, a., hoog, r. de, shadbolt, n., velde, w. van de & wielinga, b. (2000). knowledge engineering and management: the commonkads methodology. cambridge, ma: the mit press. simon, h.a. (1947/1976). administrative behavior . new york: macmillan. simon, h. a. (1969). the sciences of the artificial. cambridge, ma: mit press. sjarbaini, l (2000). a questionnaire for knowledge type assessment. internal report. university of groningen. sorge a. (2001). organization behavior. in: sorge & warner, m. (eds.) the iebm handbook of organizational behavior. london: thomson learning (p 3-20). sorge & warner, m. (eds.)(2001). the iebm handbook of organizational behavior. london: thomson learning. stillings, n.a., weisler, s.e., chase, c.h., feinstein, m.h., garfield, j.l. & rissland, e.l. (1995). cognitive science: an introduction, 2 nd edition. cambridge, mass.: the mit press. thompson, j. d. (1967). organizations in action . new york: mcgraw-hill. voort, a.g. van der (1999). wordt innovatie belemmerd door itil? informatie, maandblad voor de informatievoorziening, mei 1999, jaargang 41. (is innovation hindered by itil?) weick, k. (1995). sensemaking in organizations. thousand oaks (ca.): sage. wezel ,w. van & jorna, r.j. (1999). the sec-system: reuse support for scheduling system development. decision support systems: the international journal, 26, p.67-87. williamson, o.e. (1975). markets and hierarchies: analysis and anti-trust implications. new york: the free press. microsoft word ajis vol13-2 o.doc 17   australasian journal of information systems     volume 13 number 2    may 2006  ajis featured theme:    qualitative research in information systems    editors:    dr. liisa von hellens & jenine beekhuyzen  school of information and communication technology  griffith university,   brisbane, australia  email:  l.vonhellens@griffith.edu.au    dr donald kerr  department of management  griffith business school  griffith university  brisbane, australia      qualitative  research  attempts  to  understand  the  complexities  of  the  unstructured  nature  of  information  systems  implementations.    this  lack  of  structure  is  primarily  due  to  the  complex  interactions between social and technological factors within business.   qualitative research in these  areas  can  provide  insights  into  why  information  systems  implementations  have  not  been  as  successful as projected and can provide alternative solutions to many issues associated with political,  social and gender aspects of organisations. the challenges facing qualitative research are many and  varied,  from  the difficulty  in gaining access  to organisations  to concerns about  the validity of  the  techniques and the generalisability of the results.  the series of papers shown here attempt to address  some of these concerns.  the five papers in this section of ajis have been redeveloped from papers presented at the qualit  conference,  griffith  university  in  brisbane  on  the  23rd-25th  november  2005.  the  papers  were  selected for their impact and significance from the thirty-six research papers and seven research-inprogress  papers  presented  at  the  conference  (full  conference  information  is  available  at  www.cit.gu.edu.au/conferences/qualit2005). each of  the papers  illustrates how qualitative  research  helps solve an important practical problem.    the  paper  ì evading  technological  determinism  in  erp  implementation:  towards  a  consultative  social approachî  by clare archer-lean, jo-anne clark and don kerr provides a welcome critique to  enterprise  resource  planning  package  implementation.  the  paper  provides  an  illustration  of  the  technological determinism of these packages and of the particular challenges of implementation in a  government  owned  corporation.  a  new  approach  is  offered  for  interpreting  the  complex  clash  between  the push  to use  technology  to determine  the nature of an organisation and  the end userís  perspective that technology is a tool of an existing and rich culture. the insight of this paper is not  only  to  reject  technological determinist  approaches but  to  identify  the  technologically determinist  tendencies in erp implementation itself.    the paper ì demystifying a hermeneutic approach to is researchî  by phyl webb and carol pollard  describes  how  the  authors  went  about  defining  and  justifying  a  hermeneutical  approach  for  an  ongoing  study  on  the  prescribed  vs.  practiced  it  governance.    many  difficult  and  complex  descriptions of hermeneutics were considered until they gained a level of comfort and confidence in  australasian journal of information systems     volume 13 number 2    may 2006     18  its applicability and use  in  the qualitative study  in it governance.   examples are given of how  the  framework was used in data analysis.    the  paper  by kate crawford  and helen hasan  ì activity  theory  as  a  research  framework  in  isî   shows how activity theory can be applied to a variety of information systems studies.  the authors  discuss the specific challenges of each recent research example (vignettes) and how activity theory  helps  to  address  those  challenges.   the paper  includes  illustrations of how  these vignettes  can be  analysed according to the seven points of the framework.     in their paper ì the evolution of a business process theory - the case of a multi-grounded theoryî  the  authors mikael lind and gˆ ran goldkuhl propose the further development of the empirically driven,  inductive  grounded  theory  approach  to  data  analysis  by  complementing  it  with  a  theory  driven  analysis  of  a  seven  year  long  qualitative  study  of  business  processes  in what  they  call  a multigrounded theory.   although the paper provides a nice illustration of the multi-grounded theory it is  impossible to replicate the heated discussion it triggered during the conference. the paper certainly  made the audience reconsider the very definition of grounded theory and not everyone agreed with  the argument adopted  in  the paper, which  is an extension  to grounded  theory.   the paper  is also a  good example of a  theme paper, as after all  the qualit 2005 conference did provide a  forum  for  discussing challenges in qualitative research in the field of information systems.      microsoft word 13 1 marina.doc ajis vol 13, no. 1 september 2005 101 the management of computer systems in small business: a preliminary analysis of the issues of importance to managers evan patullock school of business la trobe university albury-wodonga campus parkers road wodonga, vic, 3689 abstract research investigating the relative importance of issues facing organisations in their management of information systems (is) has grown rapidly since the early 1980s. however, this research has largely focused on large businesses, with little comparable data collected in the context of small businesses. this paper reports preliminary results from a pilot project investigating the relative importance of various is concerns from the perspective of small business managers in the albury-wodonga region. initial results confirm the operational focus of small business managers despite evidence of some concern with more strategic issues. in light of the preliminary nature of these findings, the paper concludes by identifying areas for further research. introduction investigating the relative importance of issues facing organisations in their management of information systems (is) has been a popular research topic since the initial work of ball and harris (1982), and numerous benefits have resulted. for instance, we have been able to witness the evolution of information technology (it) in organisations through noting changes in the importance of particular issues over time, as well as see how the rating of issues can be influenced by culture (dekleva & zupancic, 1996; dexter et al, 1993; doukidis et al, 1992; moores, 1996; palvia et al. 2002; watson et al, 1997; yang, 1996). and, as watson (1989) states, “if is educators, consultants, professional societies and researchers are to serve the is community effectively, they must be aware of the key issues of is managers.” however, despite this extensive body of research, it is not without deficiencies. for example, much of the work in this field reports the views of is managers, which questions the validity of using findings as a measure of organisational concerns. as pervan’s (1998) early australian study found, chief executive officers (ceo) and is managers assign different ratings to a number of issues. the research also lacks a balance of coverage since it is predominantly focused on large businesses, with little comparable research having been conducted within small businesses. whilst this may have been understandable in the early days when few, if any, small businesses were able to afford a computer, the situation today is different. since the advent of the personal computer (pc), price has become less of a problem for even the smallest of businesses, and as a result ownership levels have continued to increase. in australia, for example, 94 percent of small businesses presently own a computer, a rise of 2 percent from the previous year (sensis, 2004). thus, similar issues research within small businesses is both feasible and overdue. more recently in this journal marshall and mckay (2003) and lawson et al (2003) have made some progress towards understanding the adoption behaviour of smaller firms in this context. more specifically, both have identified a range of impediments that constrain the ajis vol 13, no. 1 september 2005 102 acceptance and diffusion of e-commerce within small and medium-sized australian enterprises. similarly, dholaia & kshetri (2004) and lucchetti & sterlacchini (2004) have considered the adoption of internet technologies by small firms in the us and italy, respectively. notwithstanding this progress, the asymmetric research effort focussing primarily on larger firms remains. there are other reasons why greater research within small businesses is warranted. firstly, since a small business is not simply a scaled-down version of a large business (dandridge, 1979; delone, 1981; flavel, 1996; house of representatives, 1990; raymond, 1985; welsh & white, 1981), findings from large business research will not necessarily apply to small businesses. however, small businesses experience similar pressures to large businesses when it comes to it investment decisions (reimenschneider & mykytyn, 2000), even though they potentially face greater challenges in implementing computerised systems (stair et al, 1989). more specifically: • small businesses have a much smaller financial base from which to work (flavel, 1996), thereby having to potentially spend larger proportions of available funds to purchase computers. • unlike larger organisations that have had the opportunity to gradually introduce computers, and adapt to their use, over many years; small businesses have a significantly shorter timeframe in which to ‘catch up’ in order to remain competitive (yap et al, 1994). • to exacerbate the difficulties, small businesses usually have to implement technology without the support of dedicated it staff often employed by larger businesses (flavel, 1996; gable, 1991; thong & yap, 1994). and finally, the importance of the small business sector to a country’s economy (caree et al, 2002; hawkins, 1993) ensures that it warrants specific research attention. within australia, for example, australian bureau of statistics (abs) data shows that approximately 97 percent of all private sector businesses are small, and account for 49 percent of all private sector employment (trewin, 2001). similar figures are found within other oecd countries (schlıgl, 2004). smaller organisations have also under-pinned employment growth since the early 1980s (dist, 1995; hawkins, 1993; julien, 1995; o.e.c.d., 1993). obviously, a healthy small business sector benefits the economy as a whole, and by encouraging small enterprises to flourish, societies can realise a range of economic and social objectives (schlıgl 1994). therefore, by better understanding the issues small businesses have in managing their computer systems, we may be better placed to provide advice on their efficient and effective use (watson, 1989). this paper makes tentative progress towards identifying the relative importance of issues confronting small business managers in the context of managing their existing computer systems. preliminary in nature, the paper extends the range of issues identified by earlier work focusing on larger firms to the perceptions and attitudes of smaller australian enterprises. the paper itself is organised into seven main parts. section two provides a working definition of small businesses to underpin subsequent analysis. section three offers a synoptic review of pertinent literature and positions the present work in the context of earlier work. the research design framework and results from a survey of almost 200 regional firms are presented in sections four and five whilst section six provides a preliminary analysis of results. section seven offers some brief concluding remarks and identifies areas for future research. ajis vol 13, no. 1 september 2005 103 defining small business there is no universally accepted definition of small business (forsaith et al, 1995), with different countries adopting their own definition depending on a set of local factors and conditions (hall, 1995). commonly used definitions are either qualitative, which attempt to list criteria common to all small businesses, or quantitative, which specify a particular metric. qualitative definitions, whilst inadequate for classifying a business for statistical purposes (forsaith et al, 1995), provide useful insights into the nature of small business. for example: • small enterprises are often cash-starved, and not able to employ full-time specialist staff to manage all of the functions and demands upon the entity, (flavel, 1996; thong et al, 1996); • small enterprises are more dependent on external sources of technological assistance than larger ones (o.e.c.d., 1993); quantitative definitions frequently use employee numbers, assets or sales as a metric (forsaith et al, 1995; house of representatives, 1990). whilst employee numbers is the most common, there are non-trivial differences in the figure used, with values ranging from less than 20 up to 500 (forsaith et al, 1995). australia uses a combination of measures, depending on the particular industry sector (trewin, 2001), and as with other countries, the definition has undergone changes over time. originally small meant having less than 100 employees (house of representatives, 1990), whilst a later abs definition specified small as fewer than 20 persons irrespective of the industry in which it existed. in keeping with this latest trend, the latter definition has been employed within this research. previous research into small business computing whilst there is a growing body of research investigating computers within small businesses, most has focused on usage, training, and the purpose of implementation (ray et al, 1994; riemenschneider & mykytyn, 2000). other research informs us that not all small businesses employ computers, with usage appearing to increase in proportion to organisational size (sensis, 2004; thong & yap, 1995; van beveren & thomson 2002).as a result, some researchers deliberately exclude smaller organisations from their surveys in order to improve the chances of sampling from computerised firms (raymond, 1985; raymond, 1987). the take-up of computers in a small business can be inhibited by a range of factors, including a lack of resources (flavel, 1996), the absence of highly educated staff (lucchetti & sterlacchini 2004) the perception that existing manual systems are adequate for the task, and a belief that computers will not reduce costs or be as efficient as existing manual systems (chen, 1993; keeling et al 2000). some factors positively affect usage, such as the decreasing cost and increasing performance of hardware, the fear of being left behind as competitors take up technology, and an increasing awareness that computers can lead to a reduction in production costs (farhoomand & hrycyk, 1985; keeling et al 2000). management traits have also been shown to impact on usage. for instance, managers with a positive attitude to the adoption of computers, and those who are more innovative and/or knowledgeable about computers, are more likely to incorporate them within their business (thong & yap, 1995). managers’ perceptions of potential benefits have also been shown to ajis vol 13, no. 1 september 2005 104 be significant in this context (poon & swatman 1999). whilst the proportion of small businesses using computers has increased over time, little appears to have changed in the way they are used, with the main focus remaining on increasing operational efficiency through controlling day to day operational activities (cooley et al, 1987), primarily using either accounting or word-processing software (chen, 1993; cheney, 1983; lai, 1994; malone, 1985; nazem, 1990; nickell & seado, 1986). little priority would appear to be afforded strategic applications, such as using information to support management decision making (chen, 1993; levy et al, 1999), despite the argument that small businesses need to adopt a more strategic approach to technology in order to remain competitive (information industries taskforce, 1997). various propositions have been offered to explain the primarily operational focus. one is that managers of small businesses may perceive the use of computers differently to those in large business (doukidis et al, 1994) where a more strategic focus is adopted (pervan, 1996; pervan 1998; watson et al 1997). similarly, glynn & koenig (1995) suggest that many small businesses neglect the use of technology for strategic advantage because of resistance by management. they also propose that small businesses will only adopt it when forced to do so, or in order to remain competitive. this view is supported, in part, by dholakia & kshetri’s (2004) empirical work which showed a significant relationship between internet adoption by smes and perceived competitive pressures. however, cragg and zinatelli (1995) suggest that applications used by a small business will be determined primarily by the availability of packaged software, and that, if such software is not available, custom making it is usually not considered due to limited financial resources and low levels of internal it expertise. such a view is also supported by palvia et al (1994) and thong and yap (1994). by exploring the perceived relative importance of it issues confronting small business managers we may be able to trace changes over time. notwithstanding studies that have focussed primarily on adoption, to the knowledge of the author, comparatively few studies have directly addressed the relative importance of management issues confronting small business operators with their existing computing systems, particularly in australia. rivard et al (1988) surveyed is managers in canadian small and large organisations, and found managers of small firms were more concerned with operational issues than were managers of large firms. similarly, in their examination of us small businesses, alpar & ein-dor (1991) asked managers “what is your major concern in respect to the use of computers in your firm?”. whilst ‘no problems’ was the highest frequency response, concerns related to reliability, systems quality, change and cost. an additional us study sought open-ended responses from small firms by asking managers “what is the single most important thing you have learned about managing the use of information technology in your firm?” the most frequently mentioned factors related to keeping up to date with it, training and education, getting information quickly, and the importance of accurate data (riemenschneider & mykytyn 2000). a key issues framework and delphi methodology was adapted from niederman et al (1991) by pollard and hayne (1998) to identify the top three issues for a sample of canadian small firms: ‘using is for competitive advantage’, ‘improving is project management practices’ and ‘improving the effectiveness of software development’ were ranked highest. whilst these earlier studies offer some promise, in the present context their application to australian small businesses is problematic on two grounds. firstly, the classification of small businesses employed in these earlier studies varies markedly from that which defines small business in australia. for instance, pollard & hayne (1998) define small firms as employing less than 200 staff whilst riemenschneider & mykytyn (2000) use a threshold ajis vol 13, no. 1 september 2005 105 of 500 employees. secondly, the response solicitation techniques are likely to prove vexatious. for example, open-ended questions give rise to serious inconsistencies in the interpretation phase. by way of contrast, a closed-ended framework is likely to become outdated since issues of importance change over time, mainly as a result of technological developments (watson et al 1997). in addition, the delphi methodology employed by pollard and hayne (1998) results in significant burdens on respondents who must confront multiple rounds of questions. a way forward would appear to reside in deploying an issues framework with closedended responses that is sufficiently malleable to accommodate the context of small australian enterprises. this might then serve as a benchmark for future work. of particular promise is the framework developed by pervan (1998) which sought to understand the relative importance of it issues from the perspective of ceos in the us. the advantage of this framework is that it targets ceos rather than is managers, and is therefore more akin to the range of concerns confronting small business managers. however, pervan’s framework sought to measure the ‘critical value’ of issues. this was defined as the mean of the rating of an issues’ ‘importance’ and it ‘problematic’ status. in light of the preliminary nature of the present investigation a simple rating of ‘importance’ was judged likely to provide a useful starting point. research design a variety of methodologies have been employed in previous research into the importance of is management issues, including mail-out questionnaires, telephone and face to face interviews, and using both open-ended and closed questions. this project utilised a mail-out questionnaire containing a closed set of issues to be rated. accordingly, the approach broadly mirrors similar work in the context of larger firms (see, for example, pervan 1998) but in this instance the framework is deployed to gain initial insights in the context of australian smes. one of the drawbacks associated with mail-out questionnaires is the possibility of achieving low response rates, potentially in the order of ten to twenty percent (lambert and harrington, 1990), which in turn can lead to problems associated with non-response bias and self-selection (larson and chow, 2003; van kenhove et al, 2002). a review of previous research projects, particularly those using the delphi approach, showed response rates as low as 7 percent (pervan, 1998). whilst the delphi approach per se may not be totally responsible for such low rates, it does rely on participants continuing to provide responses over a number of rounds. therefore, based on the possibility that time-poor small business managers may not have been prepared to participate in multiple rounds, a single administration of the questionnaire was used as a means of maximising the response rate. the questionnaire comprised two sections. the first section sought information about the business such as its size, age, industry sector, its use of computers, and the like. in addition, this section contained four questions aimed at understanding how managers perceived the use of computers within their business. respondents were asked to use a 5point likert scale to indicate their ‘comfort’ with using computer technology; the perceived advantages of computer systems; the extent to which improvements were sought with the use of the computing system; and the extent of the firm’s reliance on the computer system. section two of the survey contained a list of twenty issues, with respondents being asked to rate the important of each issue on a scale of 1 to 10 (1 for least important to 10 for most important). ajis vol 13, no. 1 september 2005 106 the list of issues was developed from pervan’s (1998) framework and modified as a consequence of focus sessions with small businesses. this approach accords with watson et al’s (1997) observation that a re-think of the existing key issues research framework may be appropriate given that little has changed in respect of the methodology used and the actual issues being investigated since it first began. hence many of the issues in the earlier work are either not relevant for small businesses, or are expressed in a language which may not be comprehendible to a non-technical, small business manager. since the response rate can be influenced by whether or not a respondent can understand the language used (erdogen and tagg, 2003), the issues used in this research needed to reflect this. whilst every attempt was made to keep as close as possible to the original framework, a number of changes were made as a result of focus sessions with small business managers: • irrelevant issues were removed, such as “developing and managing distributed systems”. • others were re-worded to eliminate much of the jargon. for example “building a responsive it infrastructure” was changed to “having a ‘computing infrastructure’ that supports the needs of my business”. • the issue “recruiting and developing is human resources” was reworded and split into two separate issues. small businesses do not generally seek to employ designated computing staff (blili & raymond, 1993), and feedback from the focus sessions suggested that recruitment and staff development were two separate matters. hence the two issues included in the questionnaire were “recruiting ‘computer literate’ employees” and “developing ‘computer literate’ employees”. • issues based upon some of the qualitative definitions of a small business were included, relating to managing relationships with vendors, finding money for hardware and software, and keeping up to date with technology. • watson et al (1997) commented that the existing key issues framework contained few new and emerging issues. in this survey two such issues were included, one relating to implementing e-commerce and the other to controlling computer viruses and worms. as a result of this process, a total of twenty issues were identified, including both operational and strategic issues. following the work of pervan (1996; 1998), these were classified into four groups, as shown in table 1. ajis vol 13, no. 1 september 2005 107 table 1: classification of issues technology management (tm) have a ‘computing infrastructure’ that supports the needs of the business plan and manage computer networks measure the effectiveness and productivity of computer systems establish effective disaster recovery capabilities plan, implement and manage office automation select and integrate ‘off-the-shelf’ software manage and control computer viruses and worms find money for hardware and software keep up to date with new technology strategic management (sm) using computer technology to facilitate and manage changed business practices identify opportunities for the strategic use of computer systems outsource selected computing services develop appropriate funding levels for computer systems facilitate/manage computer systems that assist managers in making decisions plan and implement electronic commerce systems people management (pm) recruit ‘computer literate’ employees develop ‘computer literate’ employees manage relationships with computer vendors data management (dm) manage data and document storage improve information security and control the final change related to the wording of the preamble. in research undertaken within large businesses, respondents have been asked to consider the importance of each issue over a three to five year time period. small businesses, however, are more likely to engage in short-range planning (welsh & white, 1981), and therefore respondents in this survey were asked to rate the importance of each issue “at the present time, and for the foreseeable ajis vol 13, no. 1 september 2005 108 future”. a copy of the issues and the explanation that accompanied each can be found in the appendix a. administration of the survey the survey was conducted in the albury-wodonga statistical district, a region covering six local government areas in north-east victoria and southern new south wales, encompassing the cities of albury and wodonga where the majority of the population of approximately 100,000 resides. over 5,000 small businesses are known to exist within the region, employing approximately 19,000 people, and covering all industry categories in both the government and private sectors (investment albury-wodonga, 2003). the area was deemed suitable for this project since it contained a mixture of city and rural businesses. five hundred and fifty private sector small businesses were randomly selected from a register compiled by a local business development agency. since the register was over twelve months old at the time of its use, a manual check of business names was undertaken, resulting in the removal of seventeen businesses that could no longer be found the telephone directory. the questionnaire was therefore sent to 533 businesses. a variety of techniques aimed at maximising the response rate have been proposed for use in mail-out questionnaire research (larson and chow, 2003; erdogan and tagg, 2003; van kenhove et al, 2002; paxson, 1995). those used in this project included: • the questionnaire was professionally printed, incorporating a glossy cover page and the university logo to identify sponsorship. • a signed, department-headed covering letter, including the researcher’s name and position accompanied the questionnaire. • an assurance of total confidentiality was provided. • a summary of research findings was offered to all participants. • return envelopes were hand-stamped rather than using a reply-paid envelope. • the chance to win one of two monetary prizes was offered to those who responded. • a follow-up letter was sent out two weeks after the initial mail-out. survey results six questionnaires were returned as undeliverable, and a further three businesses made contact to advise that, whilst their office contained fewer than twenty employees, they were part of larger enterprises and thus were constrained in the way they operated. completed responses were received from 192 businesses, of which 8 indicated that they employed 20 or more people. the final sample was therefore 184 small businesses, equating to a response rate of 35.3 percent. using data published for businesses in the albury wodonga region (investment albury-wodonga, 2003), a χ 2 test confirmed that respondents were representative of the small business population in respect of their size and industry category. whilst there are no published data on the age of businesses, either within this region or nationally, the strong representation of ‘older’ firms in the sample is perhaps explained by the ‘survivorship’ principle that pervades smes generally. managers were asked to provide details of their use of computer, and their responses ajis vol 13, no. 1 september 2005 109 showed: • 91 percent used a computer in their business, with only 4 not owning their own equipment. • 75 percent were connected to the internet table 2 summarises the relationship between business size and computer usage, and size and internet connectivity within the sample data. table 2: computer and internet usage against business size business size use a computer internet connectivity 1 to 5 employees 86.7% 67.8% 6 to 10 employees 92.9% 76.8% 11 to 15 employees 100% 80.6% 16 to 19 employees 100% 100% usage figures mirror those published for australian small business (sensis, 2004), and whilst connectivity figures appear lower, it was not significantly different at the 5 percent level. both connectivity and usage would appear to be positively correlated with business size in this case. a preliminary analysis of management issues the ranking of the 20 issues in order of their perceived importance is shown in table 3 below table 3: ranking of issues of importance rank key issue issue category mean rating std devn 1 manage and control computer viruses and worms tm 9.27 1.71 2 establish effective disaster recovery capabilities tm 9.08 1.72 3 manage data and document storage dm 8.46 2.15 4 improve information security and control dm 8.38 2.27 5 have a 'computing infrastructure' that supports the needs of my business tm 8.13 1.97 6 keep up to date with new technology tm 8.07 2.19 7 find money for computer hardware and software tm 7.45 2.54 8 develop 'computer literate' employees pm 7.44 2.48 9 manage relationships with computer vendors pm 7.15 2.83 10 use computer technology to facilitate and manage changed business practice sm 6.98 2.40 11 plan, implement and manage office automation tm 6.83 2.64 12 measure the effectiveness and productivity of computer systems tm 6.80 2.48 13 recruit 'computer literate' employees pm 6.76 2.79 ajis vol 13, no. 1 september 2005 110 14 determine appropriate funding levels for computer systems sm 6.74 2.56 15 select and integrate 'off-the-shelf' software tm 6.65 2.83 16 identify opportunities for the strategic use of computer systems sm 6.53 2.69 17 plan and manage computer networks tm 6.41 3.27 18 facilitate/manage computer systems that assist managers in making decisions sm 6.36 2.67 19 plan and implement electronic commerce systems sm 5.71 2.93 20 outsource selected computing services sm 5.38 3.22 a number of observations can be made from the results. firstly, the range of mean ratings is relatively small, with even the lowest ranked issue achieving a mean score above 5. this may be an indication that all issues, irrespective of their rank position, are considered to be at least of moderate importance to small business managers. secondly, technology and data management issues tend to rate higher than strategic management issues. indeed, the highest ranking achieved for a strategic issue was tenth, with four of the six ranked amongst the bottom five positions. this is probably not unexpected, given previous research highlighting the operational focus of small business (see, for instance, dholakia & kshetri 2004). the results also appear to confirm that recruitment and development of staff within small business are two separate issues, with development assuming a higher level of importance. as previously noted, small businesses seek out people who have the requisite business skills rather than technology skills, and subsequently develop additional skills, such as computer-literacy, were necessary. a final observation is that there was general unanimity on the ranking of issues across a range of categorisations, such as business size, industry, and whether or not they used the internet. there was also agreement on the ranking irrespective of how the respondent perceived the use of computers within their business. this general agreement on the relative importance of issues across the sample is of interest since, a priori, variations may have been expected. for instance, one proposition from previous small business research was that managers with a positive outlook towards technology might rate issues differently (thong & yap, 1995). this was not supported by the data. it might also have been expected that finding money for hardware and software would be more of a problem for smaller businesses than larger organisations, but this was not evident in these data. and finally, previous research within large businesses found that the rating of issues varied across industry groupings (niederman et al, 1991; palvia et al 2002). no similar variation could be found in this survey. there were, however, two issues where the rank appeared to be influenced by the category: • managing computer networks was more important for larger-sized businesses than for smaller ones. one possible explanation is that organisations with more employees may require a greater number of computers, thereby increasing the potential for networking. • using computer systems to assist managers with decision making was rated higher for newer businesses than older ones (‘newer’ being defined ajis vol 13, no. 1 september 2005 111 as less than five years old and ‘older’ greater than five years). one explanation may be that many managers of older businesses were making decisions without the use of computers prior to their introduction, and are content to continue using such practices. on the other hand, newer managers, without such prior experience, may look to other tools to assist them, including the use of computers. top ranked issues issues ranked in the top ten places appear to reinforce the importance of computers within small businesses. given the high usage levels of computers and the internet, as well as managers’ responses to how they perceive computers within their businesses, it is not surprising that technology and data management issues, and to a lesser extent people management issues, would rate so highly. so, for instance, viruses and worms have been a concern for a number of years, and have the potential to negatively impact on organisational computer systems despite advances in anti-viral software and the availability of security software such as firewalls. putting in place disaster recovery procedures and ensuring the protection of valuable data are two ways of overcoming the risks associated with a high reliance on technology. at the same time managers need to keep up to date with the changing world of computers, to understand how they can be used to best effect within their business, and be able to find sufficient funds to purchase and maintain computing resources. when one considers that they are having to do all this, at the same time as running their businesses, this may explain why they have little time to consider other, more strategic, uses of their computers, especially without access to their own it support staff. put simply, the ‘here and now’ of operating small businesses constrains managers’ capacity to focus on strategic issues – including strategic it. strategic issues the low ranking of strategic issues differs with the results of research in large businesses (pervan, 1998; watson et al, 1997). whilst these findings are probably not unexpected, they are difficult to reconcile with other parts of the survey data. more specifically, a large number of respondents indicated in the first section of the survey that they actively sought new or improved ways of using their computer systems. respondents were not asked to provide details of what these ‘new or improved’ uses are, but it would appear from the results that they are not strategic. planning and managing e-commerce systems this was one of two contemporary issues included in the survey, and was ranked in 19th position overall, achieving a low ranking across all business categorisations, including those that use the internet. the relatively low ranking of the importance of e-commerce is of interest, especially given the proactive role played by australian governments to increase small business involvement. one explanation for this phenomenon may be the limited understanding of the concept amongst small business managers (see, for example, pease and rowe 2003). whilst a low rank position does not necessarily mean a lack of importance, the findings of this project provide at least some grounds for questioning the effectiveness of government programs in this domain. ajis vol 13, no. 1 september 2005 112 outsourcing this was the lowest ranked issue overall, and given that only four respondents did not own their computers, its position is not unexpected. however, the relatively high mean rating of 5.4 is of interest. this may support the view that outsourcing is irrelevant for the vast majority, but is of significant concern for the few engaged in outsourcing. concluding remarks stair et al (1989) proposed that small businesses would move through the same stages of technology adoption as larger businesses, albeit at a faster rate and with a different focus at certain stages. in addition, the highest stage reached in nolan’s (1979) adoption model is ‘maturity’ where the organisation perceives its technology as a strategic resource. the data collected for this research suggests that small businesses in australia fall well short of this lofty state. despite the majority of respondents indicating that they were looking for new or improved ways of using their computer systems, it appears that they place more importance upon managing the existing technology than looking towards applications that have a more strategic focus. this project is one of relatively few that have investigated the issues faced by small businesses in the management of their computer systems and attempted to establish the comparative importance of those issues in an australian context. as such, the data analysed herein should be considered as a prelude to future work in this field. any conclusions drawn from this study must also be considered against the limitations of the work. firstly, whilst the response rate in this project was much higher than in other published surveys, mail-out questionnaires cannot always ensure that representative data are captured. secondly, the choice of issues to include in this type of research requires further refinement. for example, the inclusion of the ‘managing viruses and worms’ issue, upon reflection, might be better encapsulated under a broader heading like ‘annoyances’ associated with using computers. similarly, additional issues could be integrated into this approach. in this regard pervan (1996, 1998) identified a fifth category of issues, namely end-user computing. whilst this category was excluded on the basis of the focus sessions that preceded this survey, its inclusion in future work may be warranted. hopefully, further research will resolve these problems and result in an enduring issues framework that can trace the behaviour of small businesses over time. ultimately, this might improve the policy responses to the challenges confronting small business and enhance performance. references alpar, p. and ein-dor, p (1991) “major is concerns of entrepreneurial organisations”, information and management, jan, vol 20, no 1, pp 1-11 ball, l. and harris, r. (1982) “smis members: a membership analysis”, mis quarterly, vol 6, no 1, pp 19-38 blili, s. and l. raymond (1993) “information technology: threats and opportunities for small and medium-sized enterprises”, international journal of information management, vol 13, pp. 439-448 caree, m., van stel, a., thurik, r. and wennekers, s. (2002) “economic development and ajis vol 13, no. 1 september 2005 113 business analysis: an analysis using data of 23 oecd countries in the period 19761996”, small business economics, nov, vol 19, no 3, pp 271-290 chen, j.c. (1993) “the impact of microcomputer systems on small business: england 10 years later”, journal of small business management, july, vol 31, no 3, pp 96-102 cheney, p.h. (1983) “getting the most out of your first computer system”, american journal of small business, april-june, vol 7, no 4, pp 50-60 cooley, p.l., walz, d.t. and walz, d.b. (1987) “a research agenda for computers and small business”, american journal of small business, winter, vol 11, no 3, pp 31-42 cragg, p.b. and zinatelli, n. (1995) “the evolution of information systems in small firms”, information and management, vol 29, n 1, pp 1-8 dandridge, t.c. (1979) “children are not little grown-ups: small business needs its own organizational theory”, journal of small business management, vol 17, no 2, pp 53-57 dekleva, s.m. and zupancic, j. (1996) “key issues in information systems management: a delphi study in slovenia”, information and management, october, vol 31, no 1, pp 111 delone, w.h. (1981) “firm size and characteristics of computer use”, mis quarterly, vol 5, no 4, pp 65-77 department of industry, science and technology (1995) fifth annual review – small business in australia, august 1995, commonwealth of australia, canberra, dexter, a.s., janson, m.a., kiudorf, e. and laast-laas, j. (1993) “key information technology issues in estonia”, journal of strategic information systems, vol 2 no 2, pp 139-151 dholakia, r. and kshetri, n. (2004) “factors impacting the adoption of the internet among smes”, small business economics, vol 23, no 4, pp 311-322 doukidis, g.i., smithson s. and naoum g. (1992) “information systems management in greece: issues and perceptions”, journal of strategic systems, march, vol 1, no 2, pp 63-75 doukidis, g.i., smithson s., and lybereas, t. (1994) “trends in information technology in small business”, journal of end user computing, fall, vol 6, no 4, pp 15-25 erdogen, b.z. and tagg, s. (2003) “the advertising agency manager’s response patterns to a mail survey and follow-ups”, marketing intelligence and planning, vol 21, no 6, pp 392-399 farhoomand, f. and hrycyk, g.p. (1985) “the feasibility of computers in the small business environment”, american journal of small business, spring, vo 9 no 4, pp 15-22 flavel, r. (1996) seeking to improve your business: a guide for owner-managers and business advisors, australian government publishing service, canberra, forsaith, d., fuller, d., pattison, b., sutcliffe, p. and vincente, w. “defining enterprises by size: australian emperical evidence on the interchangeability of alternative definitions at the industry level”, in dunlop, w. and gibson, b. (1995) “skills for success in small and medium enterprises”, proceedings of the 40th world conference of the international council for small business, june, newcastle gable, g.g. (1991) “consultant engagement for first time computerization: a pro-active client role in small businesses”, information and management, vol 20, no 2, pp 83-93 glynn, k. and koenig, m.e.d. (1995) “small business and information technology”, annual review of information science and technology, vol 30 , pp 251-280 hall, c. “skilling for internationalisation: findings from oecd research on the globalisation of sme’s”, in dunlop, w. and gibson, b. (1995) “skills for success in small and medium enterprises”, proceedings of the 40th world conference of the international council ajis vol 13, no. 1 september 2005 114 for small business, june, newcastle hawkins, dr b. (1993) “the strategic importance of smes in creating employment and wealth”, oecd world conference on smes and job creation: strategic issues for business and government, december house of representatives – standing committee on industry, science and technology (1990) small business in australia – challenges, problems and opportunities, australian government publishing service, canberra, information industries taskforce, the global information economy: the way ahead, department of industry, science and tourism, australian government publishing service, canberra, 1997, (http://www.dist.gov.au/itt/golds/html/execsumm.html) investment albury-wodonga (2003) albury-wodonga economic indicators 2002, albury julien, p. (1995) “new technologies and technological information in small business”, journal of business venturing, vol 10, no 6, pp 459-475 keeling, k. vassilopoulou, k. mcgoldrick, p. and macaulay, l. (2001) “market realities and innovation in small to medium enterprises: facilitators and barriers to the use of electronic commerce”, new product development and innovation management, vol 2, no 1, pp 57-70 lai, v. (1994) “a survey of rural small business computer use: success factors and decision support”, information and management, vol 26, no 6, pp 297-304 lawson, r., alcock, c. and cooper, j. (2002) “diffusion of electronic commerce in small and medium enterprises”, australian journal of information systems, vol 9, no 2, pp 123134 lambert, d.m. and harrington, t.c. (1990) “measuring nonresponse bias in customer service mail surveys”, journal of business logistics, vol 11, no 2, pp 5-25 larson, p.d. and chow, g. (2003) “total cost/response trade-offs in mail survey research: impact of follow-up mailings and monetary incentives”, industrial marketing management, vol 32, pp 533-537 levy, m., powerll, p. and galliers, r. (1999) “assessing information systems strategy development in smes”, information and management, november, vol 36, no 5, pp 247-261 lucchetti, r. and sterlacchini a. (2004) “the adoption of ict among smes: evidence from an italian survey”, small business economics, vol 23, no 2, pp 151-168 malone, s.c. (1985) “computerizing small business information systems”, journal of small business management, april, vol 23, no 2, pp 10-16 marshall, p. and mckay, j. (2002) “evaluating the benefits of electronic commerce in small and medium enterprises”, australian journal of information systems, vol 9, no 2, pp 135-147 moores, t.t. (1996) “key issues in the management of information systems: a hong kong perspective”, information and management, september, vol 30, no 6, pp 301-307 nazem, s.m. (1990) “sources of software and levels of satisfaction for small business computer applications”, information and management, september, vol 19, no 2, pp 95100 nickell, g.s. and seado, p.c. (1986) “the impact of attitudes and experience on small business computer use”, american journal of small business, spring, vol 10, no 1, pp 37-48 niederman, f., brancheau, j.c. and wetherbe, j.c. (1991) “information systems management issues for the 1990s”, mis quarterly, december, vol 15, no 4, pp 474-500 nolan, r. (1979) “managing the crises in data processing”, harvard business review, ajis vol 13, no. 1 september 2005 115 march-april, pp 115-126 organization for economic co-operation and development (1993) small and medium-sized enterprises: technology and competitiveness, oecd, palvia, p., means, d.b. jr and jackson, w.m. (1994) “determinants of computing in very small business”, information and management, vol 27, no 3, pp 161-174 palvia, p.c., palvia, s.c. and whitworth, e.j. (2002) “global information technology: a meta-analysis of key issues”, information and management, march, vol 39, no 5, pp 403-414 paxson, m.c. (1995) “increasing survey response rates: practical instructions from the totaldesign method”, cornell hotel and restaurant administration quarterly, vol 36, no 4, pp 66-73 pease, w. and rowe, m. (2003) “issues faced by small and medium enterprises (smes) and their take-up of e-commerce in australian regional communities”, proceedings of the 4 th international we-b conference, perth, 24-25 november pervan, g. (1996) “results from a study of key issues in australasian is management – 1996”, 7th australasian conference on information systems pervan, g. (1998) “a ceo view of the key issues in australian information systems management 1997”, australian journal of information systems, vol 5, no 2, pp 51-59 pollard, c.e. and hayne, s.c. (1998) “the changing face of information system issues in small firms”, international small business journal, apr-june, vol 16, no 3, pp 70-87 poon, s. and swatman, p. (1999) “an exploratory study of small business internet commerce issues”, information and management, vol 35, pp 9-18 ray, c.m., harris, t.m. and dye, j.l. (1994) “small business attitudes towards computers”, journal of end user computing, winter, vol 6, no 1, pp 16-25 raymond, l. (1985) “organisational characteristics and mis success in the context of small business”, mis quarterly, march, vol 9, no 1, pp 37-52 raymond, l. (1987) “validating and applying user satisfaction as a measure of mis success in small organisations”, information and management, vol 12, no 4, pp 173-179 riemenschneider, c.k. and mykytyn, p.b. (2000) “what small business executives have learned about managing information technology”, information and management, august, vol 37, no 5, pp 257-269 rivard, s., boisvert, c. and talbot, j. (1988) “key management issues in information systems: a comparative analysis”, proceedings of the annual conference of the asac 1998 conference, pp 32-42 schlıgel, h. (2004) “small and medium enterprises: seizing the potential”, the oecd observer, organisation for economic cooperation and development, may, pp 46-48 sensis (2004) “sensis e-business report – the on-line experience of small and medium enterprises” http://www.sensis.com.au stair, r.m., crittendon, w.f. and crittendon, v.l. (1989) “the use, operation, and control of the small business computer”, information and management, vol 16, no 3, pp 125-130 thong, j., yap, c. and raman, k.s. (1994) “engagement of external expertise in information systems implementation”, journal of management information systems, fall, vol 11, no 2, pp 209-221 thong, j. and yap, c. (1995) “ceo characteristics, organisational characteristics and information technology adoption in small business”, omega international journal of management science, vol 23, no 4, pp 429-442 thong, j.y.l., yap, c.s. and raman, k.s. (1996) “top management support, external expertise and information systems implementation in small business”, information ajis vol 13, no. 1 september 2005 116 systems research, vol 7, no 2, pp 248-267 trewin, d. (2001) small business in australia: 2001, australian bureau of statistics, australian government publishing service, canberra van beveren, j. and thomson, h. (2002) “the use of electronic commerce by smes in victoria, australia”, journal os small business management, vol 40, no 3, pp 250-254 van kenhove, p., wijnen, k. and de wulf, k. (2002) “the influence of topic involvement in mail-survey response behaviour”, psychology and marketing, mar, vol 19, no 3, pp 293-301 watson, r.t. (1989) “key issues in information systems management: australian perspective – 1988”, the australian computer journal, august, vol 21, no 2, pp 118-126 watson, r.t., kelly, g.g., galliers, r.d. and brancheau, j.c. (1997) “key issues in information systems management: an international perspective”, journal of management information systems, spring, vol 13, no 4, pp 91-115 welsh, j.a. and white, j.f. (1981) “a small business is not a little big business”, harvard business review, vol 59, no 4, pp 18-32 yang, h.l. (1996) “key information management issues in taiwan and the us”, information and management, august, vol 30, no 5, pp 251-267 yap, c.s., thong, y.j.l. and raman, k.s. (1994) “effect of government incentives on computerization in small businesses”, european journal of information systems, vol 3, no 3, pp 191-206 ajis vol 13, no. 1 september 2005 117 appendix a in your opinion how important is it to: 1. have a ‘computing infrastructure’ that supports the needs of my business? in order for small businesses to remain viable and productive they must have the appropriate hardware and software that not only supports the existing business, but also remains responsive to change. this task is made more difficult due to the continuing rapid developments in technology. 2. use computer technology to facilitate and manage changed business practices? to remain viable in today’s competitive business environment, many managers are looking for new ways to run their business. in some cases managers are looking to achieve this via use of their computer system, since the innovative use of such systems can often assist in this change process. this is proving difficult for many managers since this may be a new or foreign way of using their computers. 3. plan and manage computer networks? many businesses are connecting their computers together into networks to allow communication amongst employees, as well as to share hardware such as printers. although there are many advantages from using computer networks, it takes considerable planning and on-going management to maximise the advantages and reduce some of the negatives, such as possible security breaches. 4. recruit ‘computer literate’ employees? few small businesses have employees who are both ‘computer literate’ and understand the workings of the business. usually it is the manager who has to be competent with both. to make effective use of any computer technology, a business needs to recruit ‘computerliterate’ staff. 5. develop ‘computer literate’ employees? staff who both understand the operations of the business, as well as understand and can use technology, are vital to the success of any business. hence ways must be found to retain such staff, and one way to achieve this is to provide the means by which these people can continually update both their computing and business skills. 6. identify opportunities for the strategic use of computer systems? most small businesses only use their computers to perform routine business processes such as maintaining accounts, whilst others have seen that their computer systems may also be used strategically to possibly gain an advantage over their competitors. the difficulty is to identify such opportunities. 7. measure the effectiveness and productivity of computer systems? understanding whether, and how, computer technology impacts on the bottom-line of a business is important for justifying any future investment in technology. additionally, measuring the performance of computer systems is necessary for effective management of them. measurement of these systems is becoming more important as businesses attempt to reduce operating expenses in order to remain both viable and competitive. 8. outsource selected computing services? for businesses that don’t possess the skills to make effective use of computer technology, or those that do not wish to purchase either the hardware or software themselves, it might be more appropriate to have outside contractors provide these services. finding suitable service providers, and managing the relationship with them, can prove to be a difficult task for many managers. ajis vol 13, no. 1 september 2005 118 9. manage data and document storage? information stored in both electronic and paper-based form is accumulating at an accelerating pace. the challenge is find a means of storing this data that also allows for ease of access when it is required again. appropriate use of computer technology can assist in this process. 10. establish effective disaster recovery capabilities? many businesses rely to some extent on their computer systems, and there are risks to these organisations from the loss of such systems. effective recovery plans must be in place, and tested regularly, to ensure that losses are minimised in the event of such a disaster. as businesses become more reliant on their computer systems, the risks become greater. 11. determine appropriate funding levels for computer systems? there is generally no accepted way of determining how much money should be directed to computer systems relative to the other funding needs of the business. under-spending on these systems can be as detrimental to the business as over-spending, and a method needs to be established to allocate monies appropriately. 12. plan, implement and manage office automation? office automation is being implemented by many businesses to improve productivity. this is achieved by computerising certain routine office/business procedures. problems arise in determining how this should be done and what resources are required. careful planning needs to take place to ensure the process is successful. 13. improve information security and control? as businesses increase their dependence on computer systems, there are risks to the business from the destruction and alteration of data, the disclosure of information to outside sources, or a disruption due to the system ‘crashing’. to overcome these risks, tight security controls need to be put in place, and reliable, ‘fault-tolerant’ systems need to be devised and implemented. 14. facilitate/manage computer systems that assist managers in making decisions? increasingly small businesses are investigating ways in which computer software may be used to assist managers in the decision making process. however these efforts have often met with mixed success, and businesses need to carefully plan and manage how these will be used in future. 15. select and integrate ‘off-the-shelf’ software? few small businesses can afford to have computer software specially designed to suit their individual needs, and hence have to rely on packaged ‘off-the-shelf’ software. whilst this is the cheaper alternative, it may prove difficult to integrate all the different software that a business uses. additionally such software might not always meet the specific needs of the business, meaning compromises in its use and a resulting decrease in its benefit. 16. plan and implement electronic commerce systems? with the increasing globalisation of markets, and customers and suppliers looking to conduct business on-line, many small businesses are looking for ways in which to proceed with this new form of doing business. this often means looking in detail at the way they presently conduct their business, a re-appraisal of present practices, and an increased reliance on their computer systems. the task is made more difficult because there are few successful models for managers to use to guide implementation. 17. manage relationships with computer vendors? most small businesses have to rely on the advice of third-party computer vendors when it comes to anything to do with computer hardware and software, simply because there are rarely people with the necessary skills within the business itself. it is important, therefore, ajis vol 13, no. 1 september 2005 119 to develop good working relationships with these vendors in order to receive informed, reliable and independent advice based upon the needs of the business. 18. manage and control computer viruses and worms? despite the advances in anti-virus software in recent years, computer viruses still pose a problem. with more businesses becoming reliant on their computer systems and the internet, the potential for viruses to either destroy data, or at worst render such systems inoperative, means that managers must take steps to either eliminate the problem altogether, or at least manage the risks. 19. find money for computer hardware and software? although prices for computer hardware and software continue to fall relative to the capability of such systems, the purchase of a computer system still represents a significant investment for a small business. given the scarcity of funds, and the competing demands for these resources, it is often difficult to find money for such systems. 20. keep up to date with new technology? with the rapid advances being made in computer technology, many people feel threatened by it, whilst others often have difficulty keeping up with new developments. as businesses become more reliant on such technology, managers are under increasing pressure to stay abreast of developments, as well as apply be able to adapt it within their own business. microsoft word ajis12-1final repaired.doc ajis vol. 12, no. 1 september 2004 4 applying the australian and new zealand risk management standard to information systems in smes robyn a davidson, phd student, school of commerce, the flinders university of south australia, gpo box 2100, adelaide south australia 5001. telephone: +61 8 82013081 facsimile: +61 8 82012644 email: robyn.davidson@flinders.edu.au susan c lambert, lecturer, school of commerce, the flinders university of south australia, gpo box 2100, adelaide south australia 5001. telephone: +61 8 82012766 facsimile: +61 8 82012644 email: susan.lambert@flinders.edu.au nominated author for correspondence: robyn davidson abstract this paper advocates the use of the australia/new zealand risk management standard (sa/snz, 1999) in conjunction with of a modified version of birch and mcevoy’s (1992) structured risk analysis for information systems (sra-is) to identify information systems security risks in smes. the use of internet based commerce by smes exposes them to information systems security risks that they are ill equipped to recognise let alone mitigate. unlike the identification of some business risks, identification of risks associated with information systems requires certain technical expertise. the structure of the existing information system must be understood and modelled before risks can be identified and it is acknowledged that the required technical expertise may not be present in smes, thus the involvement of information systems consultants may be necessary. once the information system has been modelled little information systems expertise is required to complete the analysis, keeping consultant involvement to a minimum and maximising owner/manager involvement. introduction small and medium enterprise (sme) owner/managers recognise and deal with risks in many aspects of their business. risk in relation to destruction or theft of physical assets, theft or unauthorised use of intellectual property, financial risk associated with undertaking large contracts. “the need to manage risk systematically applies to all organisations and to all functions and activities within an organisation and should be recognised as of fundamental importance by all managers and staff” (knight, 1999). one such function is the provision of information systems. information is the lifeblood of any business. turn off the information system for a day and chances are productivity will come to a grinding halt. similar consequences will result from the system working ineffectively or from the integrity of data being compromised. the increasing use of networked information systems within smes can be the source of serious security problems (spinellis, kokolakis & gritzalis, 1999). the fact that the business is small does not mean that it will escape the notice of would be hackers. “hackers normally do not attack sites based on the profile of companies, but randomly target a range of internet protocol addresses and seek to infiltrate on a sequential basis…” (see cited in ravendran, 2001). smes need to understand that once connected to the internet they are equally at risk of attack as any other organisation. giannacopoulos (2002) states that, “somewhere, sometime, someone will target your company for attack...”. smes typically lack the technical expertise and resources to effectively identify and manage risk (spinellis et al, 1999). many smes are unaware of the risks that face their information systems so it is hardly surprising that they do not take adequate security measures. what is needed is a method ajis vol. 12, no. 1 september 2004 5 that can be applied by sme owner/managers to identify risks and measure them so that appropriate risk management decisions can be made. this paper shows how the australia/new zealand standard: risk management (sa/snz, 1999) can be used by sme owner/managers to identify and evaluate risks. the australian/new zealand standard: risk management (sa/snz, 1999) defines risk as “the chance of something happening that will have an impact upon objectives”. the risk materialises when a threat such as a hardware failure, data being tampered with or the system being destroyed by fire, is coupled with a vulnerability such as unrestricted access to premises. threats to information systems (is) can arise from accidental errors or mishaps such as power failures, accidental equipment breakage or equipment failure. threats can also arise from criminal attacks such as spying by competitors, impersonation to gather information or steal from users, denial of service attacks or malicious content attacks. these threats can come from inside or from outside the organisation, they may be intentional or unintentional but whatever the cause, the damage can be avoided or at least reduced by applying a methodical risk management plan and process. risk management and the security policy risk management begins with the development of a security policy that defines how risks will be identified, analysed, evaluated and treated. it should be an organisation wide policy that covers all aspects of the business. to this end the organisation needs to be divided into appropriate units for risk consideration such as customer relations, occupational health and safety, production, inventory and information systems. the security policy should clearly define what is to be secured, why it needs to be secured, and how it is to be secured. the intention of the policy is not to give the technical details of how security will actually be achieved but to state how the risk management process will be implemented. the technical aspects of how security will actually be achieved will depend on what risks and treatment options are identified and the cost of these options. the australia / new zealand standard: risk management (sa/snz, 1999) provides a “generic guide for the establishment and implementation of a risk management process”. this standard can be applied to a wide range of organisations for a number of applications, including risk management of information systems. the risk management standard provides a: “logical and systematic method of establishing the context, identifying, analysing, evaluating, treating, monitoring and communicating risks associated with any activity, function or process in a way that will enable organisations to minimise losses and maximise opportunities” (sa/snz, 1999, p.1). the process is illustrated in figure 1 and discussion of each stage follows. it is important that all assumptions, methods, data sources and results of the risk management process be carefully and thoroughly documented. the documentation will provide evidence of a systematic approach to risk identification and analysis, as well as forming the basis of a knowledge database of the recorded risks to the organisation. the documentation will also provide decision-makers with a basis for their decisions and will facilitate ongoing review, and communication of the information to stakeholders (sa/snz, 1999, p.21). the standard recommends that a structured method for risk identification and assessment be used but falls short of providing such a process. the problem smes find with risk assessment methods for information systems is that they are difficult to administer because they require technical expertise that is not present in house. the process often needs to be outsourced which means considerable expense. what the authors of this paper have endeavoured to design is a method that isolates the technical, information systems tasks and provides a technique for owner/managers to apply to their own information systems. the process recommended in this paper is a simplified version of birch and mcevoy’s (1992) structured risk analysis for information systems method (sra-is). in figure 1, the australian/new zealand standard risk management process has been divided into five phases and sra-is has been mapped onto each phase to show how it can be applied to each task. ajis vol. 12, no. 1 september 2004 6 phase 1: establish the context in which the risk management process will take place establishing the context involves defining the strategic context, the organisational context and the risk management context in which the risk management process (rmp) will take place. criteria against which risks will be evaluated and the structure of the analysis should also be defined at this stage (sa/snz, 1999, p.7). the strategic context provides a description of the organisation; where it sits within the industry, who controls it, who its customers are, how big it is, its employee profile and the major strengths, weaknesses, threats and opportunities that face the organisation (sa/snz, 1999, p.9). the organisational context refers to “the capabilities, goals and objectives of the organisation and the strategies that are in place to achieve them” (sa/snz, 1999, p.9). the organisational context in which the rmp is to be implemented must be defined and understood so that the rmp can be designed to complement, or at least not conflict with, organisational goals and objectives. the risk management context refers to establishing the “goals, objectives, strategies, scope and parameters of the activity, or part of the organisation to which the rmp is being applied” (sa/snz, 1999, p.10). determining the risk management context requires the definition of a “risk unit(s)” or part of the organisation or that function of the organisation to which the rmp is to be applied. the risk units of an organisation might include occupational health and safety, warehousing, property management or information systems. several rmps might exist in a single organisation making up the over-all rmp of the entire organisation. criteria against which risks are evaluated need to be determined (this relates to phase 4: evaluate risks). this enables the users to identify risks that are acceptable and those that are not acceptable. if the risk falls outside the acceptable range action needs to be taken to reduce the risk. it is up to the users to set the level of risk that they are willing to accept and the level that they cannot accept. for instance, users might decide to measure risks on a qualitative scale of “low” to “extreme” and that risks with a value of “high” or “extreme” are not acceptable. risks that fall into the “high” or “extreme” categories require immediate treatments to bring them down to at least the “moderate” level. the criteria used will depend on the method used to measure risks. this is discussed in phase 3: analyse risks. a structured method is needed to systematically identify all risks. in this phase, the structure to be used should be decided upon and documented. structured analysis techniques are discussed in phase 2: identify risks. phase 2: identify risks the aim of the risk identification stage is to generate a comprehensive list of all risks facing the organisational unit regardless of whether they are or are not under the control of the organisation. it is essential that a well-structured process be used, so that all significant risks are identified; if they are not identified at this stage they are excluded from analysis. (sa/snz, 1999, p.12). how can we be confident that all risks have been identified? certainly, something more than intuition and brainstorming is required. a properly applied structured analysis technique will ensures all risks are identified. this involves separating the activity into a set of elements, which provides a logical framework for identification and analysis (sa/snz, 1999, p.12). many structured analysis techniques have been developed. examples of these are the cobra risk consultant (c & a systems security, 2002), cramm (gamma secure systems, 2002), and structured risk analysis for information systems (birch & mcevoy, 1992) methods. these methods typically have a structured way of capturing all of the information necessary to support risk analysis that allows all risks to be identified and analysed. birch and mcevoy’s structured risk analysis for information systems (sra-is) method is adopted in this paper. all structured techniques require considerable skills from the users. this is a draw back for smaller organisations since it often requires extensive use of expert information systems security consultants, which translates to a prohibitively expensive exercise. the sra-is technique is no exception, however, the adaptations proposed in this paper require minimal technical expertise to implement. sra-is requires the organisation to build three models from which threats, ajis vol. 12, no. 1 september 2004 7 vulnerabilities and ultimately risks are derived. it is vital that these models accurately reflect the information assets, physical assets and the information flows and stores of the organisation. it is in building these models that technical skills are required. this is the only stage that requires specialist information systems skills and since it is such a vital stage of the process, it is recommended that the skills be outsourced if they do not exist within the organisation. the application of the models requires general business skills and knowledge of the business, skills that are likely to exist within the business. the models upon which all risks are derived are the information model (from which threats are derived), the technology model (from which vulnerabilities are derived) and the business model that links the information and technology models. risks are the manifestation of threats and vulnerabilities. it is useful then to identify threats and vulnerabilities and derive risks from these. a threat is something that will have an adverse effect on an organisation and exists whether or not there are any practical or apparent ways in which it might ever be manifested. threats to information systems can be divided into three categories as shown in table 1. birch and mcevoy (1992) define a vulnerability as a characteristic of a physical system, which allows a threat to be exploited, while being independent from any specific threat. is vulnerabilities are categorised by the type of threat to which they relate as shown in table 2. ajis vol. 12, no. 1 september 2004 8 figure 1: risk management process source: adapted from sa/snz, 1999, p.11 table 1: types of threats threat category explanation those that threaten data confidentiality private and sensitive data must be kept confidential. those that threaten data/system integrity data and programs should only be changed by authorised personnel. those that threaten data/system availability the is should operate effectively and efficiently to ensure service to authorised users. table 2: types of vulnerabilities m o n it o r a n d r e v ie w c o m m u n ic a te a n d c o n s u lt establish the context � the strategic context � the organisational context � the risk management context � develop criteria identify risks � what can happen � how can it happen evaluate risks � compare against criteria � set risk priorities accept risks treat risks � identify treatment options � evaluate treatment options � select treatment options � prepare treatment options � implement plans analyse risks determine existing controls determine likelihood determine consequenc es estimate level of risk yes no assess p h a s e p h a s e p h a s e p h a s e 5 p h a s e for structure, use birch & mcevoy’s sra-is. build sra-is models. identify threats, vulnerabilities and risks. estimate frequency of attacks and potential loss. calculate annual loss exposure risks. ajis vol. 12, no. 1 september 2004 9 threat vulnerability threats relating to data confidentiality can be exploited if… physical assets and communication links can be accessed by unauthorised personnel. threats to data/system integrity can be exploited if…. physical assets and communication links can be tampered with. threats to data/system availability can be exploited if… physical assets and communication links can be damaged. table 3 shows some possible vulnerabilities to a telephone line communication link and the threats to the data that travels through the line. table 3: an example of vulnerabilities and threats vulnerability relating to the communication link “telephone line” threat relating to the information asset “data” the telephone line may be accessed by using a wire-tap. data is viewed by unauthorised parties. loss of confidentiality of data. the telephone line may be tampered with by being redirected. data is lost or corrupted. data integrity is lost. the telephone line could be damaged by being cut. the system is inaccessible. data is unavailable. by cross-referencing the identified threats (what can happen to the information asset) with the identified vulnerabilities (how it can happen) a complete list of risks can be identified. an organisation is therefore at risk when there is a threat to the business and a vulnerability that may be exploited to realise that threat. once a complete list of risks has been identified they are analysed. information model identifying the threats using the birch and mcevoy (1992) method requires the user to build an “information model” that identifies all of the information assets of the organisation, i.e. all of the elements of the organisation about which information is generated. these elements include customers, suppliers, and transactions such as sales and deliveries and details of the goods and services traded by the organisation. a simple information model for a retail organisation is shown in figure 2. each information asset maybe subject to an integrity threat (i), a confidentiality threat (c), or an accessibility threat (a). each threat identified in the information model is catalogued in a table such as that shown in table 4 along with an estimate of the loss the business would suffer should the threat be realised. the loss estimate figure will depend on the type of analysis used. see phase 3: analysis. ajis vol. 12, no. 1 september 2004 10 figure 2: information model example 7 stock item 6 supplier delivery 4 purchase order 2 supplier 5 customer delivery 3 sales order 1 customer source: birch & mcevoy, 1992 table 4: threat catalogue extract inf. asset information asset name threat id description of threat to information asset loss (l) 1 customer i1 customer lost/corrupt 3 customer c1 customer disclosed 3 customer a1 customer not available 2 2 supplier i2 supplier lost/corrupt 4 supplier c2 supplier disclosed 4 supplier a2 supplier not available 3 3 sales order i3 sales order lost/corrupt 2 sales order c3 sales order disclosed 2 sales order a3 sales order not available 1 4 purchase order i4 purchase order lost/corrupt 1 purchase order c4 purchase order disclosed 1 purchase order a4 purchase order not available 2 source: adapted from birch & mcevoy, 1992 “customer” data from figure 2 is the “customer information asset 1” in table 4, “supplier” data from figure 2 is the “supplier information asset 2” in table 4, etc. table 4 shows that for every information asset there are three potential threats, e.g. there may be a threat to customer data integrity (i1), customer data confidentiality (c1), or access to customer data (a1). technology model the technology model identifies the physical information system assets of the organisation such as file servers, internet connections, personal computers, and local area networks (lans). a simple technology model relating to a retail organisation is shown in figure 3. ajis vol. 12, no. 1 september 2004 11 figure 3: technology model example 0 gothamwidgets 1.4 manual post 1.5 pstn 1.3 seriai link 1.2 factory lan 1.5 pstn 1.2 factory lan 1.2 factory lan 1.2 factory lan 1.4 manual 1.1 manual road/rail post dispatch pc 1 supp lier goods inward pc 2 customer factory pc 3 edi gateway pc 4 1.1 anual road / railroad/rail m source: birch & mcevoy, 1992 figure 3 shows four personal computers (pcs) connected by a local area network (lan). customers and suppliers can provide or get information by telephone (public switched telephone network – pstn), through the post, or with deliveries by road or rail. it is from these physical assets that vulnerabilities can be derived. each physical asset may be vulnerable to unauthorised access, tampering or damage. unauthorised access can threaten data confidentiality. tampering can threaten data and system integrity. damage to physical assets can restrict or prevent access to data and the system. each vulnerability is catalogued in a table such as that shown in table 5 along with an estimate of the frequency of the vulnerability being exploited. the frequency estimate, like the loss estimate, depends on the type of analysis used. see phase 3: analysis. table 5: vulnerability catalogue extract phy. asset physical asset name vul. id description of vulnerability of physical asset prob (p) 1 dispatch pc i1 dispatch pc tampered -2 dispatch pc c1 dispatch pc accessed 0 dispatch pc a1 dispatch pc damaged -1 2 goods in pc i2 goods in pc tampered -2 goods in pc c2 goods in pc accessed 0 goods in pc a2 goods in pc damaged -1 3 factory pc i3 factory pc tampered -3 factory pc c3 factory pc accessed -1 factory pc a3 factory pc damaged -2 4 edi gateway pc i4 edi gateway pc tampered -2 edi gateway pc c4 edi gateway pc accessed 0 edi gateway pc a4 edi gateway pc damaged -1 1.1 road/rail i1.1 road/rail tampered -2 road/rail c1.1 road/rail accessed 0 road/rail a1.1 road/rail damaged -1 source: adapted from birch & mcevoy, 1992 business model the next step is to determine the vulnerabilities that might cause a threat to be realised. for instance, access to customer order information will be threatened if the file server holding that data is damaged. if the organisation operates over the internet then the same threat could be realised if ajis vol. 12, no. 1 september 2004 12 the telephone line connecting the file server to the internet is damaged. a methodical structured process is required to ensure all vulnerabilities are matched to each threat. it is the business model that provides the required link between the information model (threats) and the technology model (vulnerabilities). the business model is used to expose risks by linking the threats identified from the information model and the vulnerabilities identified from the technology model. the business model shows the flows, sources and sinks of information, along with the information processing and retention centres (birch & mcevoy, 1992). figure 4 shows a simple retail business model. figure 4: business model example stock 0 gothamwidgets sales order customer delivery stock purchase order supp lier delivery stock purchase order request supp lier r cp o ess ro der 1 customer rp ocess c apur h se 2 manage stock 3 d1 stock source: adapted from birch & mcevoy 1992 in figure 4, the “customer” and “supplier” are external entities that provide (source) and receive (sink) data through the data flows indicated by the arrows. this is data about the sales order, customer delivery, purchase order, and supplier delivery. the square boxes represent processes that transform data. a process transforms the data in some way, such as updating stock levels, and sends the data to another process or to a data store. a data store is represented by a rectangle. coding of information assets, physical assets, and each element of the business model allows for cross-referencing and ultimately matching of vulnerabilities and threats and therefore risks. an example of how to determine the risks for the threat that customer information may be disclosed is illustrated in figure 5. this can be a tedious and error prone exercise but can be automated with the use of a database program. davidson (2000a; 2000b) developed a program that was tested on the information systems of small public accounting practices and it proved to be a successful method to determine risks. the elements of each model and the relationships between these elements are input into the program. every threat (lost/corrupt, disclosed, not available) and vulnerability (damaged, accessed, tampered) is then automatically generated. using the inputted loss and frequency estimates the program generates a complete list of risks with calculated loss exposure figures. phase 3: analyse risks in order to assess the significance of a potential risk each risk must be assigned a value. for each potential risk, an estimate must be made of the resulting consequences or loss should the threat be realised and an estimate of the likelihood or frequency of the vulnerability being exploited. estimates can be made based on historical evidence and from the judgements of personnel with knowledge of the relevant threats and vulnerabilities. the estimates can be quantitative, semiquantitative or qualitative (sa/snz, 1999, p.13). ajis vol. 12, no. 1 september 2004 13 figure 5: cross referencing between models to determine risks quantitative estimates are particularly useful when consequences can be expressed in dollar terms, and likelihood can be expressed as frequency per year. the risk level is then expressed as an annual loss exposure (ale) in ‘dollars per year’. the advantage of quantitative analysis is that it facilitates cost-benefit analysis and can help justify expenditure. however, any inaccuracy in estimates can lead to wildly inaccurate ale figures (dorey, 1991). qualitative estimates use descriptive terms. consequences can be measured on a scale such as “low to huge” financial loss and likelihood of the event occurring can be expressed as “rare to almost certain”. qualitative estimates are appropriate in situations where it is difficult to accurately estimate losses. a matrix is used to combine the consequences and losses resulting in a loss exposure of “low” to “extreme” (dorey, 1991; (sa/snz, 1999, p.14 & 34-35). semi-quantitative analysis gives numerical values to qualitative scales. this allows numerical values to be combined to produce a numerical risk level. the risk level does not represent an see table 4 identified potential risks customer disclosed by road/rail being accessed customer disclosed by manual/post being accessed customer disclosed by pstn (phone line) being accessed customer disclosed by factory pc being accessed a risk exists where there is a threat, and a vulnerability that will allow this threat to be realised. for each threat, select the information asset. see figure 2 threat catalogue threat: (c1) customer information cross-reference the information asset to the business model to determine which information flows and stores involve that asset. see figure 4 cross-reference the information flows and stores from the business model to physical assets in the technology model. see figure 3 select the relevant vulnerabilities (availability, integrity, or confidentiality) for the identified physical assets. see table 5 information model information asset: (1) customer business model information flows and stores: 1.1 sales order 1.2 customer delivery technology model physical assets: 1.1 road/rail 1.4 manual post 1.5 pstn 3. factory pc vulnerability catalogue vc 1.1 road/rail accessed vc 1.4 manual/post accessed vc 1.5 pstn accessed vc 3 factory pc accessed ajis vol. 12, no. 1 september 2004 14 intrinsic value such as that produced in quantitative analysis. semi-quantitative analysis has the advantage of allowing subjective assessments but ranks risks in more detail than achieved with qualitative analysis (dorey, 1991). birch and mcevoy’s (1992) sra-is method uses a semi-quantitative analysis which produces a detailed ranked list of risks. semi-quantitative analysis is appropriate where it is difficult to estimate intangible losses such as loss of reputation, loss of client data, and sales histories, but a more detailed ranking of risks is required than that provided by qualitative analysis. phase 4: evaluate risks this stage of the rmp compares the level of risk found during the analysis process with the risk evaluation criteria established in the “establish the context” stage discussed in phase 1 (sa/snz, 1999, p.15). an organisation using qualitative analysis may establish that “extreme” and “high” risks require immediate attention, “low” risks are acceptable and “moderate” risks will be considered on a case by case basis to determine their level of acceptability. an organisation using quantitative analysis can set levels of acceptability in numerical terms such as, risks with an ale of “x dollars per year” or less are acceptable. any risk with an ale in excess of “x dollars per year” requires immediate attention. semi-quantitative analysis allows an acceptable level of risk to be set similar to quantitative analysis. however, the figure does not represent a dollar value per year, but indicates the seriousness of the risk relative to the other risks. for instance, a risk with an exposure of 4 is more serious than a risk with an exposure of 3. appropriate treatment must be determined for those risks that are deemed unacceptable, whilst those risks determined to be acceptable should be monitored and periodically reviewed to ensure that they remain acceptable. phase 5: treat risks there are three steps in treating risks: step 1. identify treatment options; step 2. evaluate and select treatment options; and step 3. prepare and implement treatment plans. step 1: identify treatment options five treatment options are available: 1. avoid the risk; 2. reduce the likelihood of the occurrence; 3. reduce the consequences; 4. transfer the risk; or 5. retain the risk (sa/snz, 1999, p.16). risk avoidance involves eliminating the activities that generated the risk. for example, being connected to the internet poses a risk that an attacker could access sensitive files. this risk can be avoided by simply disconnecting from the internet. care must be taken however to ensure that the risk avoidance measures do not conflict with the strategic objectives of the organisation identified in phase 1. this requires the analyst to consider both the costs and benefits of applying the treatment. the likelihood of the risk occurring relates to the vulnerabilities of the is. therefore, to reduce the likelihood of an occurrence, the vulnerabilities can be treated so that the expected frequency of an attack falls to a level that generates an acceptable risk level. for instance, the risk of an outside party tapping into the customer data could be reduced through the use of a firewall to restrict access. the consequences, should the threat be realised, refer to the economic or other loss experienced should an attack take place. thus consequences can be reduced in many ways such as daily backups ajis vol. 12, no. 1 september 2004 15 of all files, or minimising the number of files stored on the computer that is connected to the internet. another way of reducing the loss or consequence of an attack is by transferring part or all of the loss to a third party through insurance. this is not always appropriate for is risks since often the damage to or loss of data cannot be monetarily compensated. retaining the risk is another option. individual risk profiles differ; therefore, some organisations will retain more risks than others. after treating risks to reduce them there will be residual risks or new risks arising from treatment. it is important that users revisit their original criteria to ensure they are comfortable with the levels in terms of both costs and benefits. step 2: evaluate and select treatment options the risk treatment options must be assessed in terms of the effect they will have on the organisation and the cost of implementing them. the benefits obtained from implementing the treatment options should outweigh the costs (both monetary and non-monetary). identifying appropriate treatment options requires both business and technical is skills and knowledge. treatment options can be aimed at decreasing the impact rating of the threat, i.e. decreasing the loss should an attack occur. for example, by backing up data twice daily so that if the hard drive crashes during the day only half a day’s data needs to be reconstructed. alternatively, the treatment option might be aimed at reducing the vulnerability by decreasing the likelihood of the physical item being accessed, tampered with or damaged. for example, by placing all telecommunication cables underground to decrease the likelihood of them being cut by vandals. the best options are those where large reductions can be obtained with relatively low expenditure. there may be cases of rare severe risks that will warrant treatment that cannot be justified on economic grounds alone, these cases need to be considered on a case-by-case basis. the australian/new zealand standard: information security management (sa/snz, 2000) provides a list of control objectives and the related controls that can be applied to suit the needs of various organisations. step 3. prepare and implement treatment plans the risk treatment plan documents the controls that have been chosen to treat the risks. it also states who has responsibility for implementing the plan, what resources are to be utilised, budget allocation and the timetable for implementation. the plan will also include details of how compliance with the treatment plan will be reviewed (sa/snz, 1999, p.41). monitoring and review monitoring and reviewing, as shown in figure 1, is an ongoing process that is part of every stage of the complete rmp. as well as monitoring the effectiveness of the risk treatment plan and how it was implemented, risks and their control measures need to be continually monitored, as few risks remain static. circumstances can change which affect the likelihood and consequences of an event, as well as the suitability of treatment options. by regularly repeating the risk management cycle it ensures that the management of risks remains relevant (sa/snz, 1999, p.20). communication and consultation throughout the rmp various stakeholders should be consulted and kept informed of findings and proposed actions. the stakeholders are those who can be affected by a decision or activity and include employees, management, insurance organisations, financiers, customers and suppliers. ajis vol. 12, no. 1 september 2004 16 risk management summary the australian/new zealand standard: risk management (sa/snz, 1999) describes a standard procedure for managing risks. this procedure takes the form of iterating through the five phases of establishing the context, identifying, analysing, evaluating, and treating risks. throughout these five phases there is continual monitoring and review of the processes and the stakeholders are communicated with and consulted. this is a generic procedure that can be applied to a wide range of organisations and activities. when implementing the rmp the risk assessor needs to have a good knowledge of the organisation and types of threats and vulnerabilities it faces to be able to identify, analyse and evaluate risks. the risk assessor needs to be able to accurately break the information structure down into elements in order to identify systematically all threats and vulnerabilities. recognition of the value of the information is crucial to enable accurate estimates of the loss that would arise should a threat be realised. similarly, historical evidence and good judgement is essential to accurately estimate the likelihood of vulnerabilities being exploited. the example given in this paper uses a simplified version of birch and mcevoy’s (1992) sra-is method. a more complex version systematically derives all risks and vulnerabilities from models of the organisation’s information system. it calculates an exposure level taking into account: • the loss to the organisation and gain to an attacker should a threat be realised, • the frequency of exploiting a vulnerability and the cost to an attacker to do so, and • the four different types of attackers. davidson (2000a) has written a computerised risk analysis program based on this method. this program simplifies the rmp by automating the generation of threats, vulnerabilities, risks and exposure levels. the risk assessor must also have a good knowledge of what risk treatment options are available. conclusion information system security is a serious concern to smes operating in networked information systems environments. before owner/managers can implement appropriate security measures however they must identify and evaluate the risks that they face. this paper has provided a method of identifying and evaluating information system security risks that is consistent with the recommendations of the australian/new zealand standard: risk management (sa/snz, 1999). the application of this structured risk analysis technique can be supported by a purpose built database. this, like any other information systems analysis does require accurate systems modelling, the skills for which may need to be outsourced. this should not be looked upon as a major obstacle but as the foundation of future systems security management. ignoring information system security could result in irrecoverable damage to the tangible and/or intangible assets of the business. applying ad hoc security measures may result in the misdirection of resources. a systematic approach to risk management will provide owner/managers with the information needed to make cost justified information systems security decisions. resources can then be allocated in a way that maximises the benefits to the organisation. references birch, d. g. w., & mcevoy, n. a., 1992, ‘risk analysis for information systems’, journal of information technology, vol. 7, pp. 44-53. c & a systems security ltd, 2002, ‘introduction to security risk analysis and the cobra approach’, available on-line from http://www.securitypolicy.co.uk/risk analysis/, last accessed 1/7/02. davidson, r. a., 2000a, ‘information systems risk analysis method’, computer program, available by contacting robyn.davidson@flinders.edu.au. ajis vol. 12, no. 1 september 2004 17 davidson, r. a., 2000b, ‘information systems risk analysis for smaller south australian public accounting practices’, honours thesis, school of commerce, flinders university of south australia. dorey, p. g., 1991, ‘risk assessment current methodologies and software products’, information security monitor, vol. 7, no. 1, pp. 5-7. gamma secure systems, 2002, ‘a practitioner’s view of cramm’, available on-line from http://www.gammass1.co.uk/topics/hot5.html, last accessed 28/6/02. giannacopoulos, p., 2002, ‘paranoia is good’, strategic finance, montvale, vol. 83, no. 8, pp. 2629. knight, k., 1999, a basic introduction to managing risk, (hb 142-1999), standards association of australia. ravendran, a., 2001, ‘low security awareness among smes’, computimes malaysia, new york, august 30. spinellis, d., kokolakis, s. & gritzalis, s., 1999, ‘security requirements, risks and recommendations for small enterprise and home-office environment’, information management & computer security, vol. 7, no. 3, pp. 121-128. standards australia & standards new zealand (sa/snz) 1999, as/nzs 4360:1999 risk management, standards australia & standards new zealand, homebush, aust., wellington, nz. standards australia & standards new zealand (sa/snz) 2000, as/nzs 7799.2:2000 information security management, standards australia & standards new zealand, homebush, aust., wellington, nz. ajis vol. 8 no. 2 may 2001 a method for reverse engineering of use case realizations in uml dragan bojic, dusan velasevic university of belgrade, faculty of electrical engineering bulevar revolucije 73,11000 belgrade, yugoslavia {bojic.velasevicj@buefil.etf.bg.ac.yu abstract we propose a novel method for recovering certain elements of the uml model of a software system. these include relationships between use cases as well as class roles in collaborations that realize each use case, identifying common functionality and thus establishing a hierarchical view of the model. the method is based on dynamic analysis of the system for the selected test cases that cover relevant use cases. the theory of formal concept analysis is applied to obtain classification of model elements, obtained by a static analysis of code, in terms of use case realizations. keywords object-oriented systems, uml modeling, reverse engineering, use cases introduction a support for reverse engineering a system model from its code has become a standard part of modern analysis and design tools to satisfy the needs for round-trip engineering, updating a model to the evolved code, and taking the legacy context into account. our method is devised to supplement a model obtained by a typical static analyzer. such models lack the support for reverse engineering of the use case architectural view. therefore, we have decided to address this issue by recovering use case realizations, which serve as a basis for classifying model elements in a hierarchically nested package structure that is normally a part of a design model as prescribed by jacobson (1999). we have adopted the approach to conduct reverse engineering guided by use cases (i.e. scenarios of system use). a use case is a means of describing system functionality, and represents the collection of interactions between the system and its environment or between various parts of the system for some specific use of the system, booch (1998). jarke (1998, 1999) reports a number of advantages of a scenario driven development or reengineering process: • use cases are easily comprehensible for users as well as developers so they represent a good communication medium. • design efforts are focused on the most important functionality of the old/changed system before the detailed design. • design decomposition is task-oriented, enabling the exploration of design alternatives, iterative development and reusability of design knowledge. while code reuse is certainly an attractive option to software developers, reuse of the most general specifications, i.e. use cases and interaction diagrams, will prove to be more effective. in particular, blok (1998) states that reuse of requirement specifications can lead to a significant decrease in development time and cost, as it usually leads to the reuse of work-products further down-stream in the development process. specifically, there exist techniques for storing and retrieving reusable uml use case specifications to and from a large collection of uml design components, and also, for automatically generating a framework design model from uml use case diagrams, class diagrams and sequence diagrams, kim (1998). clearly there is a need to have a use case view of the system model. the presented method takes into account the following practical considerations: • the cost of understanding and applying it must be much less that the cost of manually performing the recovery. user is required to have priory knowledge of the system functionality only, to define relevant use cases and corresponding test cases that execute those use cases. • the method uses only those kinds of data about the existing system, its structure and behavior, which is readily available in most contemporary commercial development tools, thus the cost of implementing it on various platforms is not very high. in fact, to construct a prototype tool to support our method, we only had to adapt the output of microsoft visual c++ profiler to conlmp concept analysis tool, burmeister (1998), and update the model extracted by rational rose design tool. ajis vol. 8 no. 2 may 2001 the proposed method there are several activities to perform when applying the proposed method: 1. identify a set of use cases for a given system or a part of it. 2. for each use case, define one or more test cases that cover relevant use case. 3. collect profiling information for each test case. 4. construct the context relation between use cases and covered system code entities. 5. construct the conceptual lattice (i.e. acyclic digraph) from a context relation by applying formal concept analysis. informally, each concept is a pair of two components: a set of use cases and a set of system code entities that is a shared part of realization of these use cases. if an edge connects two concepts in the lattice, then the parent concept represents "broader" functionality that in some way "uses" the functionality of the child concept. 6. estimate the quality of decomposition from the topology of the lattice, and if necessary, modify the test cases or apply some purification of the context relation, to eliminate the effects of interleaving functionality in code and other undesired effects, and repeat activities necessary to obtain the updated lattice. 7. construct the basic uml model using a static analyzer and decompose the model using the concept lattice. such analyzers have become an integral part of most general-purpose ooad tools to support round trip engineering. these activities are further elaborated in this chapter. in the next section we present the results obtained by applying the method to a small sample application. using a prototype tool on a small example we have constructed a prototype tool urca that supports the proposed method and uses four other tools. the c++ profiler from microsoft visual studio environment is used for data gathering, rational rose for extracting initial uml model of the system and also for presentation of the resulting decomposition. conlmp concept analysis tool is used for constructing a concept lattice, and ristancase development assistant for c (da-c) for viewing the lattice. the data flow graph of the system is represented in figure 1. data are exchanged through textual files, except for the communication between visual studio and other tools that is based on automation. tools other than urca dictate file formats. the components of urca are test recorder application written in visual basic, and ca_input, rose_input and dac_input command line applications written in c++. recently we replaced conlmp tool with our own implementation of concept analysis algorithm to overcome some conlmp's limitations. a batch process activates the latter three applications once test recording is done to prepare data for other tools. we shall now present the use of the prototype tool to analyze a sample application scribble step 3 tutorial application from microsoft developer network library (figure 2). it is an mdi application that supports elementary drawing, which can be saved and subsequently retrieved. it has a total of 34 classes and 401 member and non-member functions. mfc framework classes were excluded from the analysis. that leaves 11 user defined classes and 121 functions. we defined the following use cases that roughly corresponds to items from the main menu: ul. working with documents (generalization of u2 and u3) u2. creating, opening and saving u3. printing u4. working with multiple windows u5. drawing u6. configuring u7. help ajis vol. 8 no. 2 may 2001 application prepared for profilin c conlmp v*1 figure 1. structure of the prototype system we defined twelve test cases to cover these use cases (we subtracted execution counts of the application start and exit test case from the profiling data for other test cases). cribble document.scb ; &e £dit een figure 2. scribble sample application screenshot figure 3 shows a test recorder application screen of our prototype recovery system. this application is used to activate a profiling session in the visual studio environment for each test case and to collect profiling data (the number of executions of each function) using the automation interface. the user then selects use cases covered from a list of all use cases. this information and profiling data can be either saved to a file or memorized internally for test differing. a difference between previously memorized data and current data can also be saved to a file as a separate test case. for example, the first test case was to start the application, select file new from the menu, and then exit. it covered ul and u2. on the basis of coverage data, a so-called context matrix is computed by the ca_input application. this input is presented to the conlmp tool, which calculates the resulting description of the corresponding concept lattice. a concept lattice can be viewed using the development assistant for c tool (figure 4). urca dac_input application was used to create a da-c project from the concept description file. the lattice is represented in the form of c source code, each concept is a c function that calls functions for the successor concepts. the concept lattice, i.e. call hierarchy graph, for our example is shown in figure 5. the ajis vol. 8 no. 2 may 2001 information content of each concept (its use cases and functions) is present in the form of comments and can also be examined in da-c. generalized use cases are marked a, externally visible use cases (which are directly covered with test cases) bear no special mark, and included use cases (a common functionality revealed by a concept analysis) are marked #. in figure 5, included use cases were named after common functions that are contained in the corresponding concepts. urca tool cusp recorder s3e: m,1 0.0 afxloassinrr:afx.cl«ssinrr(stru ct cruntimeqass ") (soibdocobj) 1 00 abdnitioji;e(mt unsigned long) (appmodul ob|) 0 0 0 caboutolg::caboutdlg(void) (scribble obj) 0 0.0 caboutolg::dodataexchange(dass co ata exchange •) (scribble obj) 0 00 caboutolg::getmessagemap(void) (scribble.obj) 0 00 q-:*'i ftc ri!;bi£figure 3: test case recorder application j'ejlb-'.edit ^arch .brows*; fiioph,^metrics sjart eroject qptjons f i? |̂ m i* {jbtfag -g.a fe'lg sg help void printing(void) | /•*******.•*««use cases**** printing **********••••fuhctiohs**** cscribbleview::onbeginprini cscribbleview::oncndprinti: cscribblaviaw::onpreparepr. */ drawingview (); void workingwithmultiplewin^ /•*»«*»*«»««««use cases**** working with multiple windo cchildfraaa: :cchildframe (v cchildfraae: :createobject ( cchildframa: :precreatehind _ocmjldek«jna.^>.' scalar delet. figure 4: da-c application screenshot using the rational rose 98i visual c-hadd-in operating with visual c++ fuzzy parser, we have created a reverse engineered model of the scribble application. the urca rose_input application uses a concept lattice description to create a rose script file (figure 6) for updating the reverse engineered uml model of the scribble application. the updated model contains a package hierarchy as a logical view corresponding to a concept lattice. figures 7-10 present the partial logical structure of the model. for each concept, there is one logical package. a ajis vol. 8 no. 2 may 2001 lattice determines the containment relationship between packages. the package that corresponds to some concept u contains or references only those parts of classes relevant to the realization of the use case u, which are not already present in contained packages. those parts are shown in the pictures. a static analyzer that is a part of rose induced relationships between classes. now we shall discuss the decomposition process in detail. # figure 5: the concept lattice for scribble sub main dim co as category dim cl as category dim c2 as category dim dummy as classdiagram dim aclass as class set co = roseapp.currentmodel.rootcategory set cl = co.addcategoryo'help ") set dummy = cl.addclassdiagram("main") d = co.classdiagrams.getfirst("main").addcategory(cl) set c2 = co.addcategory("drawing") set dummy = c2.addclassdiagram("main") d = co.classdiagrams.getfirst("main").addcategory(c2) set c3 = co.addcategory("working with documents ") set dummy = c3.addclassdiagram("main") figure 6: part of rose script generated by urca identifying use cases each use case represents one functional requirement. for the reasons of efficiency for large systems, we are interested in those use cases that are "architecturally significant" as defined by jacobson (1999), so that separate use cases should not be used for too elementary system functions. relationships between use cases, as defined by booch (1998), are not introduced in this step. they will be determined automatically in the next phases. ajis vol. 8 no. 2 may 2001 working with multiple windows (from creating, opening and saving i! cscribbleview !! (from reverse engineered) ij#cscribbleview() cscribbleooc : (from reverse engineered) + getdocsize() ; j<<virtual»+precreatewindow(j) |:«virtual» + -cscribbleview() jkcreateobjectq i cmdichildwnd i (from frame windows) ] i i 1 „.. ; | updatingview j i j (from deletingdoccontents) j cchildframe ! ; jfrom reversejngineered) _| ' «virtual»+precreatewindowo i «virtual» + -cchildframe() i + cchildframe() + createobjectq i figure 7: working with multiple windows package creating, opening and saving (from working with documents ) l ,-,_:. ~~~ ; ctypedptiust<class cobust.class cstroke • working with i (from reverse engineered) multiple window^ „... . — _ . . _ , , " ' ~ ^ 1 + ctypedptrlistq +-ctypedptrtjsto -nn_strokelist ideletingdoccon i tents : ' (from drawing) i j replacingpen ! (from configuring) cscribbledoc (from reverse engineered) i#cscribbledoc() |# initdocumentq i |«virtual»+onopendocumento |«virtual»+onnewdocument() | ! «virtual» -f serializeo • j |«virtual» + -cscribbledoc() figure 8: creating, opening and saving package drawing (from logical view) destructelements(class cpoint', int) \ | destructetements(class cpoint'. int) h-•-----:--•---•-••••:•--•••-•-•:•--•:--•••-•-•••-•------•---• i cscribbledoc i (from reverse engineered) |+ getcurrentperx) •f newstroke() «afx_msg» * oneditqearall() «afx_msg» # onupdateeditclearajiq carray<class cpoint.class cpoint> (from reverse engineered) • +m_pointarray cstroke ! i (from reverse engineefed) j ucstrokeo ! i + cstrokeo | + finishstroke() #m_pstrokecur cscribbleview (from reverse engineered) :«atx_msg»#onlbuttonllp() j i «afx_msg» # onlbuttondown() ; : +m_strokelist \ i ctypedptrlist<dass coblist.dass cstroke *> i (from reverse engineered) !+addtall(j ! + removeheado deleting doccon tents help (from logical view) cscnbbleapp (from rewrse engineered) j «afx_msg»+ onappaboutq1 igetmessagema j ;l p caboutdig (from rewrae engineered) virtual» » dodataexchangeq +caboutdlgo figure 9: drawing package updatingview (from deletingdoccontents) cscribbleview (from reverse engineered) «virtual» # onupdate() i drawingview i i figure 10: help and updating view packages use case identification can be done manually, in a systematic manner similar to one described in noffsinger (1998). for example, for graphical user interface applications, each menu item, toolbar button and other items that initiate some function can be assigned one use case. this procedure can even be automated by analyzing resource file that contains definitions of such items (figure 11). resource descriptions can be extracted even from executable file for windows applications. ajis vol. 8 no. 2 may 2001 selecting and executing test cases a test case specifies one way of testing the system, including what to test with which input or result, and under which conditions to test. each test case can be derived from, and is traceable to a use case or a use case realization, to verify the result of the interaction between the actors and the system or between various components of the system jacobson (1999). in our case, the purpose of executing tests is not to verify system behavior, but to collect profiling data. for each use case defined in the previous step, we devise one or more scenarios of system use. a particular scenario can possibly exercise more than one use case, if they are indirectly related, but the exact kind of relationship need not be specified. each test case specifies how to test some specific scenario from the set of devised scenarios. when we obtain the initial architectural model, we can add more test cases that concentrate on some specific use case realization, to refine the model. the above methodology is related to high-level functional testing as described in kit (1995). each test case can be related to more than one use case. for example, a test case can cover general editing functionality and specific text replacement functionality; or a test can cover opening test file and then previewing that file for printing. the profiling information is then collected for each test case. we have chosen to collect executions counts at the function (class members) level (the code lines level seemed too detailed). using execution counts, and not just coverage information enables what we called "test case differing": suppose we conduct a test case in which a file is opened in editor, and another in which a file is opened and then previewed for printing. by subtracting execution counts of the first test case from those of the second, we obtain profiling information for just the print preview use case, which can now be treated as a separate test case. idr_mainframe menu preload discardable begin popup "&file" begin menuitem "&new\tctrl+n", id_file_new menuitem "&0pen...\tctr1+0" , id_file_open menuitem separator menuitem "p&rint setup. menuitem separator menuitem "recent file", menuitem separator menuitem "e&xit", id_app_exit end popup "scview" begin menuitem "stoolbar", id_view_toolbar menuitem "^status bar", id_view_status_bar end end id_file_print_setup id_file_mru_file1, grayed ort+n flpen~/r :ctrl-t-q ' print setup-.: exit file menu file new file open resource script file figure 11: identifying use cases from menus print setup introduction to formal concept analysis concept analysis provides a way to identify sensible groupings of objects that have common attributes burmeister (1998). a context relation indicate which object has which attributes. figure 12 present a sample context relation for four objects oi o* and eight attributes, ai-a8. o, 02 03 04 a! x a2 x a3 x x x a4 x x x a5 x x as x x a7 x x as x x figure 12. a sample context relation ajis vol. 8 no. 2 may 2001 for any set of objects o, let ca(o) represent the set of attributes which every object in o posses. similarly, for a set of attributes a, let co(a) represent the set of objects which posses all attributes in a. a pair (o,a) where a = ca(0) and o = co(a) is called a formal concept. such formal concepts correspond to maximal rectangles in the context relation matrix, where of course permutations of rows or columns do not matter. for a concept c = (o, a), the first component o is called the extent and the second component a is called the intent of c. a concept (x,, y,) is a subconcept of concept (x2, y2) if, and only if, x, is a subset of x2 (or, equivalently, y2 is a subset of y(). the subconcept relation forms a complete partial order (the concept lattice) over the set of concepts. an algorithm to compute a concept lattice can be found in siff(1997). the concept lattice for a sample context relation from figure 12 is presented in figure 13. let us now define the notions of object and attribute concepts. the object concept (e,i) that corresponds to an object o is the smallest (closest to the leaf) concept for which o e e. the attribute concept (e,i) that corresponds to an attribute a is the largest (closest to the root) concept for which a e i. figure 14 depicts the lattice from figure 13 annotated with this information. for example, c5 concept is the object concept for o3 and the attribute concept for a«, a7 and ag. = ({02,03,04}, {a3a.}) = ({0,}, {a,,a2}) = ({02)04}, {a3,a4,a5}) = ({03,04}, {a3,a4,a«,a7, = (ik (ai,a2, a3,a4,a5,a«,a7,a8}) figure 13. a sample concept lattice £" figure 14. object and attribute concepts applying formal concept analysis for our purposes, the set of objects o includes all functions, class member as well as non-member, in our application. the set of attributes a includes all use cases as defined by the user. context relation between functions and use cases is defined with a meaning "f is related with u if, and only if, f is a part of code that implements u". relation implements is established on the basis of profiling data obtained by executing test cases, and is defined as follows: f implements u, if, and only if, there exist at least one test case that executes f (at least once) and covers u. with this definition of context relation, the extent e of each concept c in a resulting concept lattice is a set of functions that are the shared part of implementation code of the set of use cases that forms the intent i of c. in a real world situation, two problems could possibly arise with this interpretation. first, if all test cases that cover a certain use case u| also cover some unrelated use case u2, than all functions that implement u| will be mistakenly related to u2 also. this problem can be eliminated by carefully choosing test cases. the second problem is code interleaving, a fact that the same function f can actually participate in implementing ajis vol. 8 no. 2 may 2001 two unrelated use cases u, and u2. the above definition works fine for this situation, relating f to both u, and u2, but a resulting concept lattice mistakenly relates together u, and u2. splitting the row f of the context matrix in two or more rows, one for each of the implemented use cases, and recalculating the lattice solves this problem. a tool can perform this operation automatically, without the need for the user to examine the context relation manually, although urca prototype does not support it yet. the second modification of context relation is the elimination of all functions that belong to standard library or foundation classes because they are used in many different contexts. this does not apply to classes that are inherited from these standard classes to perform some specific functionality. in this interpretation of concepts we use the term function concept instead of object concept, and use case concept instead of attribute concept. we have chosen to modify the resulting concept lattice by eliminating all such concepts that are neither function concepts nor use case concepts for at least one function or use case. these concepts have no contribution to the model decomposition, but only clutter the lattice. pairs of concepts in the original lattice that were connected via these eliminated concepts are connected directly to each other in the resulting lattice. usually, if test cases covered all functions and use cases, the root and the leaf concepts of the lattice would be eliminated. the remaining concepts can be classified in the folio wing categories (differently marked in figure 5): 1. use case concepts for generalized use cases (marked a in figure 5). generalized use cases are identified by the fact that they never appear alone in test cases, and that their corresponding concept is larger than the concepts of those other related use cases. 2. use case concepts for "ordinary" use cases (without marking in figure 5) are all other use case concepts. 3. function concepts that are not at the same tune use case concepts (marked # in figure 5) are usually lower in the lattice than use case concepts and represent some shared functionality between different use cases. updating model from the lattice from the concept lattice, we can determine various elements of a uml model. a particular concept can be use case concept for zero or more use cases. in the former case, a concept represents some common functionality that is not externally visible. in the latter case, if there is more than one use case corresponding to a particular concept, further test cases are needed to differentiate between those use cases. in case of one to one correspondence, each use case concept corresponds to the collaboration (use case realization) of the corresponding use case. extents of these concepts define analysis classes (parts of real, implementation classes that take role in the realization of relevant use cases). for every concept in a lattice, calculated by the rules described in the previous section, we introduce one package in a logical view (more precisely, in a distinct sub-view of this view). the relation between concepts in the lattice corresponds to the containment relation between packages. if there is a concept in the lattice with more than one parent concept, a corresponding package will be owned by one parent package and referenced from others. packages that correspond to use case concepts are named after the relevant use cases. each non-member function belongs to the package that corresponds to its function concept. in case of classes, there could be more than one function concept for its member functions. in that case, we assign the class to the package that corresponds to the concept that is function concept to most of its functions, and reference the class from other packages. it is well known that the use case and the logical view of a uml model are related with each other by interaction diagrams describing particular use cases, showing interaction (function member calls) of instances of classes from the logical view that are implementing these use cases. having only the information obtained from static analysis and execution counts, one cannot determine exact interaction diagrams but only model elements relevant for the implementation of each use case. they all belong to the package that corresponds to use case concept for a particular use case. the ordering relation on the set of concepts can be used to determine relationships among particular use cases. if the use case concept of some use case u is greater the use case concept of some use case u' then there exists a unidirectional relationship between them. if u corresponds to generalized use case, then u "generalizes" it. in other cases, the relation is either "includes" or "extends" but a deeper analysis is needed to establish which kind. the preceding rules are summarized in figure 15. actors (external entities) can also be introduced in the model by assigning them to some functions in code. good candidates are event handling functions and i/o library functions. if function f is assigned to actor a, and the object concept for f corresponds to the realization of use case u then we associate a and u in the model. how do modules, subsystems and components fit in this model? this hierarchy is independent to functional decomposition (it also includes other criteria: physical distribution, technology concerns, etc. 10 ajis vol. 8 no. 2 may 2001 figure 16 depict possible relationships between physical subsystems and use case realizations. if we wish to introduce this additional classification in our model we can extend the context relation to include containment relation between functions and subsystems. n^trnkp ! <x<virtual» serialize() <*finishstroke() ; <x3rawstroke() sgetboundingrect() • <cstroke() ! «cstroke() | drawing (from scnbble) +m_poirilahby ; : carray<cpoint,cpoint> : oadd() i <>assertvalid() :" «carray() ; <clementat() \^getsize() i operator 00 i o-carfayx) a package is introduced for each concept \ ctypedrrljst<coblist,cstroke'> #m_pstrokecur ! (from working with multiple windows) oaddtail() <>removeheado y+m_strokelist cscribbledoc (from working with multiple windows) cscribbleview z;:::::::::: : (from working with multipjewindows) ; <>getcurrentpen() . _. ., î «afx_msg» onupdateeditclearaiio ;^x<afx_msg» onlbuttonup() i ;^x<afx_msg» oneditclearaji() !9<>«afx_msg» onlbuttondown()!! «newstroke() figure 15. summary of rules for updating uml model from the lattice a method is k assigned to the ̂ lowest package in the lattice that contains it relationships l\ obtainted from '• _stetic_anajysis class role (the l\ part of class in ; this realization) usec^se • usecase realization a /realization b --"i—--•*• r-use case realization a use case / use case realization c realization d figure 16. relationships between physical subsystems and use case realizations related work numerous methods were proposed structuring models of software systems. structural clustering techniques define strictly structural criteria for decomposing a system in a hierarchy of subsystems minimal coupling between subsystems and maximal coupling between entities in the same subsystem. for example, mancoridis (1998) proposes sub-optimal algorithms based on search (hill climbing) and a genetic algorithm. wiggerts (1997) presents an overview of clustering algorithms that could be used for software re-modularization. clustering techniques that focus on data abstraction, consider the use of data of the same type as the grouping criterion. these techniques find their use in discovering candidates for object and classes in non-object-oriented software. lindig and snelting (1997) use a formal concept analysis on relation between procedures and global variables. however, experiments suggest a limited usability for discovering system structure (partly because of noise in input data). siff and reps (1997) also use formal concept analysis in a similar context, but also make use 11 ajis vol. 8 no. 2 may 2001 of negative information, e.g. function f does not use structure x. they claim more promising results than lindig and snelting (1997). plan recognition techniques, qulici (1995, 1998), need a library of predefined programming plans, which are matched against parts of code to build a hierarchy of concepts (abstractions such as complex data structures or a specific functionality). each plan consists of two parts: a pattern, used for matching and a concept that is recognized with the pattern. a pattern is a combination of components (the language items or sub-plans that must be recognized) and constraints (the relationships that must hold between these components to have the plan recognized). plan matching can be performed by a variety of techniques including deductive inference, constraint satisfaction or pattern matching, and could be combined with a top-down search of plan libraries for a plan that matches a particular concept. plan recognition could be applied to maintenance related program transformations, kozaczynski (1992). these techniques typically require the existence of a large plan database and at the present guarantee only partial model recovery. there is also a problem of varying code patterns that implement the same plan and a scaling problem when considering large systems. recently, a work has been done on identifying standard design patterns in code. for example, antoniol et. al (1998) present a multi-stage reduction strategy using software metrics and structural properties to extract some structural design patterns from code. besides automatic decomposition techniques, with a possibility for a user to guide a process in some phases, there exist a number of manual and semi-automatic techniques. these techniques allow collecting, filtering and presenting in a suitable form, data obtained by means of either static or dynamic analysis of the system. the user is, however, responsible for identifying and grouping system entities into subsystems, or for connecting implementation entities with a functional model of the system. rigi tool, described in miiller (1993), allows a user to create multiple views, and also has some graph arranging algorithms to cluster related system entities. dali tool from kazman (1997) uses a two-phase process in the view extraction phase, various elementary views are extracted from the system using static and dynamic analysis, and are stored in a view repository. in the view fusion phase, these views are joined under user manual control into more complex views. in richner (1998, 1999), static and dynamic information is modeled in terms of logic facts in prolog, which allows creating views with high-level abstractions using declarative queries. in software reflexion modeling technique, murphy (1995), users define a high-level model (a graph of logical subsystems) and specify how the model maps to the source model (a call graph) by assigning functions to logical subsystems. a tool then computes software reflexion model, a graphical representation that shows how well the connections between logical subsystems defined in high-level model agree with those actually found in source model. de hondt (1998) proposes an architectural recovery model based on software classifications. in his model, classifications are containers of software artifacts, and artifacts can be classified in multiple ways. among other methods, he proposes virtual classifications. there is an explicit (e.g. declarative) description which elements are intended to belong to a classification. finally, there has been some work on locating functionality in code using static or dynamic analysis. the software reconnaissance technique implemented in. %suds toolset, agrawal (1998) locates plans i.e. code fragments associated with a particular feature i.e. functionality by comparing traces of test cases that exhibit or do not exhibit a feature. every feature must be considered separately from others, and the focus is on small-scale features. wilde (1996) proposed a technique to partition a program code in equivalence classes such that all code in the same class has the same execution count in each of multiple test cases. the results of the case study were mixed. on the one hand several meaningful and non-trivial program plans were identified. on the other, a significant fraction of the equivalence classes did not represent meaningful program plans. slicing techniques make use of call and data dependency graphs to identify those parts of code that contribute the computation of the selected function, for all possible inputs, in case of static slicing, or for a given input, in case of dynamic slicing. a dynamic slicing technique presented in korel (1998) helps to understand large programs, by visualizing the control flow slices at the level of call graphs. in canfora (1998), a static slicing technique is used to identify user interface software layer. conclusions the proposed method is conceived to automate a part of task that a reengineer has to do manually after creating basic uml model with a static analyzer. we certainly do not imply that a presented decomposition strategy is "the best" strategy, in the sense that the original designer would create the same decomposition, or that it works equally well for all kinds of software systems. but it is our opinion that a functional decomposition is useful for the system-understanding task that faces a reengineer unfamiliar with system implementation. we applied the method to 10k line wordpad standard windows accessory application. we focused on essential functionality and devised about 25 use cases and roughly the same number of test cases that covered about 70% 12 aj1s vol. 8 no. 2 may 2001 of application functions. resulting lattice contains 35 concepts, 5 "generalized", 15 "ordinary" and 15 "included" ones. the decomposition quality was subjectively assessed as good, without making any transformations on the context relation. in the future, we also plan to demonstrate the usability of our method on even more complex applications. that might include a project the first author has been involved with for about four years. the application is an integrated development environment with reverse engineering capabilities that runs on windows platforms. its size is nearly 400.000 lines of c++ code, and it has many attributes of a legacy application: the original team left the firm, the application was initially developed without any explicit modeling or architectural considerations etc. a work is now being done to extend the method to include creation of interaction diagrams, on the basis of more detailed dynamic data e.g. a complete activation tree for the given test execution, produced by some tools. the presented method can be applied even to non object-oriented systems, in combination with techniques that identify candidate objects in the code, to obtain an object-oriented model of such systems. references agrawal, h & alberi, j. l. & morgan, j. r. & li j. & london, s. & wong, w. e. & ghosh, s. & wilde, n. (1998) "mining system tests to aid software maintenance", ieee computer, july, pp. 64-73. antoniol, g & fiutem, r. & cristoforetti, l. (1998) "design pattern recovery in object-oriented software", proceedings of the 6th international ieee workshop on program comprehension iwpc 98, ischia, italy, pp. 153-160. blok, m. c. & cybulski, j. l. (1998) "reusing uml specifications in a constrained application domain", asia pacific software engineering conference, taipei, taiwan, dec, pp 196-202 booch, g & rumbaugh, j. & jacobson, i. (1998) the unified modeling language user guide, addisonwesley. burmeister, p. (1998) formal concept analysis with conlmp: introduction to the basic features, arbeitsgruppe allgemeine algebra und diskrete mathematik, technische hochschule darmstadt, schlobgartenstr. 7, 64289 darmstadt, germany. canfora, g & cimitile, a. & de lucia, a. & di lucca, g a. (1998) "decomposing legacy programs, a first step towards migrating to client-server platforms", proceedings of the 6th international ieee workshop on program comprehension iwpc 98, ischia, italy, pp. 136-144. de hondt, k. (1998) a novel approach to architectural recovery of object-oriented systems, phd theses, vrije universiteit brussel. jarke, m. (1999) "scenarios for modeling", communications of the acm, january, vol. 42, no. 1, pp. 47-48. jarke, m. & kurki-suonio, r. (1998) "scenario management introduction to the special issue", ieee transactions on software engineering, vol. 24, no. 12, december, pp. 1033-1035. jacobson, i. & booch, g & rumbaugh, j. (1999) the unified software development process, addisonwesley. kazman, r. & carriere, s. j. (1997) view extraction and view fusion in architectural understanding, software engineering institute, carnegie mellon university, pittsburgh, pa 15213 kit, e. (1995) software testing in the real world, addison-wesley. korel, b. & rilling, j. (1998) "program slicing in understanding of large programs", proceedings of the 6th international ieee workshop on program comprehension iwpc 98, ischia, italy, pp. 145-152. kozaczynski, w. & ning, j. & engberts, a. (1992) "program concept recognition and transformation", ieee transactions on software engineering, vol. 18, no. 12, december, pp. 1065-1075. kim, d. k. & jung, h. t. & kim, c. k. (1998) "techniques for systematically generating framework diagram based on uml", asia pacific software engineering conference, taipei, taiwan, dec, pp. 203-210. lindig, c. & snelting, g (1997) "assessing modular structure of legacy code based on mathematical concept analysis", international conference on software engineering, boston, usa, pp. 349-359. mancoridis, s. & mitchell, b. s. & rorres, c. & chen, y. & gansner, e. r. (1998) "using automatic clustering to produce high-level system organizations of source code", ieee proceedings of 6th international workshop on program understanding iwpc'98, ischia, italy, june, pp. 45-52. muller, h. & orgun, m. & tilley, s. r. & uhl, j. s. (1993) "a reverse engineering approach to subsystem structure identification", software maintenance: research and practice, 5(4), december, pp. 181-204. murphy, g & notkin, d. & sullivan, k. (1995) "software reflexion models: bridging the gap between source and high-level models", proceedings of the acm sigsoft '95, washington, d.c., pp. 18-28. noffsinger, w. b. & niedbalski, r. & blanks, m. & emmart, n. (1998) "legacy object modeling speeds software integration", communications of the acm, vol. 41, no. 12, december, pp. 80-89. quilici, a. & yang, q. & woods, s. (1998) "applying plan recognition algorithms to program understanding", journal of automated software engineering , 5(3), pp. 347-372. 13 ajis vol. 8 no. 2 may 2001 quilici, a. & chin, d. (1995) "decode: a cooperative environment for reverse-engineering legacy software", proceedings of the second ieee working conference on reverse engineering, july, pp. 156-165. richner, t. & ducasse, s. (1999) "recovering high-level views of object-oriented applications from static and dynamic information", proceedings of the ieee international conference on software maintenance csmr'99 richner, t. & ducasse, s. & wuyts, r. (1998) "understanding object-oriented programs with declarative event analysis", 12th european conference on object-oriented programming, workshop on experiences in object-oriented re-engineering, brussels, belgium, july. sefika, m. & sane, a. & campbell, r. h. (1996) "monitoring compliance of a software system with its highlevel design models", proceedings of the 18th international conference on software engineering, march, pp 387-396. siflf, m. & reps, t. (1997) "identifying modules via concept analysis", ieee international conference on software maintenance, bary, italy, september. wiggerts, t. a. (1997) "using clustering algorithms in legacy system remodularization", working conference of reverse engineering wcre '97, pp. 33-43wilde, n. & gotten, m. & london, s. (1996) "using execution counts to identify delocalized program plans", tech. rpt. serc-tr-81f, software engineering research center, ose-301, university of florida, fl 32611. 14 microsoft word ajisfinal 12 2 marina.doc ajis volume 12 no. 2 may 2005 164 altered images: the relations between design representations and design practice ms susan keller, dr ross smith deakin university dr steve howard, dr jennie carroll university of melbourne school of information systems deakin university melbourne, victoria email: susan.keller@deakin.edu.au, ross.smith@deakin.edu.au department of information systems university of melbourne melbourne, victoria email: showard@unimelb.edu.au, jcarroll@staff.dis.unimelb.edu.au abstract as information systems move out of the office into the wider world and are merged with mobile appliances, buildings and even clothing, the representations traditionally used in any one discipline may not be adequate for understanding these new domains. design representations are ‘ways of seeing and not seeing’. despite the central role representations play in design, the information systems design community has little understanding of the relation, ideal or actual, between design practice and design representation. this paper reports on an extensive design case study that aims at increasing understanding of the nature and affordances of representations in the design process and argues for the need for information systems as a discipline to open up discussion of the design representations that may be required to effectively design systems that mix traditional is with disciplines such as industrial design, architecture and fashion design. keywords design representations, design process, affordances, case study, industrial design introduction advances in technology are leading information systems out of the office and into domains such as mobile appliances, buildings and even clothing. these new domains present design challenges which the current repertoire of design representations may not be able to adequately represent. already the boundaries between work/organisational, social and personal activities are becoming blurred: consider for instance the impact of mobile technologies (carroll et al. 2001). these changes mean we will increasingly place importance on cross-activity and cross-domain design in is and this will require an expanded mindset and a richer toolkit of representations. design, whether in is, urban planning, industrial design or architecture, requires designers to create and to give form to ideas through representations. representation has a profound influence on design (hevner et al. 2004) and it is the quality of the representational artefacts that largely determines the degree to which we can reflect on our designs (suchman 1994). yet compared to other design disciplines is has an impoverished set of design representations, especially design representations that allow reasoning of the design as a whole (stolterman 1999) and the information systems design community has little understanding of the relation, ideal or actual, between design practice and design representation. we argue that different design representations facilitate some designers’ activities and constrain others: the affordances, or material properties, of design representations may influence the particular representation choices designers make. further, we conjecture that examination of design representations across different design disciplines may provide theoretical understandings that could: enable the development of new representations that allow better design of mobile, wearable, tangible ajis volume 12 no. 2 may 2005 165 and ubiquitous computing systems; enhance cross-discipline communication; and provide a basis for the development of design tools to support design in these new contexts. the paper begins with a discussion of design and design representations then moves on to discuss the research approach and the case study findings from a first case which was undertaken at an industrial design site. such a site was chosen as a starting point because of the rich selection of design artefacts employed in a short life-cycle development process – features it is argued that might subsequently inform an enhanced understanding of is design. the paper presents several different ways of examining design representations. different representations allow consideration of certain aspects or views of the design; the views, identified in the industrial design case study are outlined. next a high-level description of the design process is presented and mapped against the views. this allows us to describe some of the affordances of particular design representations. finally, a brief reflection on the implications of the case study to the future world of designing ubiquitous is applications is presented. design and design representations there are many ways design can be conceptualised, but a useful one for thinking about design representations is offered by rosenman and gero (1998). they contend that as humans we exist in a physical world which gives rise to certain needs (for example shelter, warmth) but we also exist in a socio-cultural world of our own making. this socio-cultural world also gives rise to various needs and desires. the artefacts that are created as a result of the needs arising in these two worlds form the techno-physical world. design can thus be seen as a human endeavour that starts with an idea conceived to satisfy some human need or desire and ends with a syntactical description of an artefact that when realised is intended to satisfy that need: it proceeds from “a human socio-cultural environment to an artefact techno-physical environment” (rosenman and gero, 1998). design representations are an intrinsic part of design (perry and sanderson 1998). in fact some researchers define design as the production of design artefacts (dym 1994), (galle 1999). design representations allow reasoning about design issues and some authors (dym 1994; simon 1996) argue that problem solving in design is basically a matter of finding a representation that makes the solution obvious. consider for instance, chen’s (1976) development of the entity-relationship model. this representation, of the semantics of data, profoundly influenced how we reason about data in information systems (hevner et al. 2004). design representations can be defined as any ‘perceptible expression of a design idea, proposal or fact’ (saddler 2001). this definition includes conversations, sketches, scenarios, storyboards and schematic representations. brereton (2004) categorises representations in terms of: whether they exist only in the mind (internal) or are expressed in the world (external), whether they are transient, such as utterances, or durable, and whether they are generated during the process or appropriated ready-made to assist with the design. saddler (2001) offers a fine-grained analysis of the role of design representations in design including such purposes as ‘making ideas manipulable’, ‘summarising design decisions’ and ‘forcing things into the open’. these are not incompatible with the two roles galle (1999) offers: that of communication and exploration. galle (1999) includes ‘self-communication’ to cover those instances when the designer uses design representations to think through design issues. the social role of design representations is summed up by perry and sanderson (1998): design representations “are the foci of much of the social interaction, as contested symbols, objectifications of temporary agreements, and symbolic representations of potential productions.” design representations have been studied in different discrete domains including information ajis volume 12 no. 2 may 2005 166 systems (see for example (faro and giordano 1999; kyng 1995; perry and sanderson 1998; stolterman 1999)but there appears to be little research on the role of design representations in emerging contexts where the boundary between is and other types of design is being blurred. the research presented in this paper is a first step in addressing this gap. research design the present research seeks to build an enhanced understanding of the nature and affordances of representations in the design process and to argue the need for is as a discipline to open up discussion of the design representations that might be required to effectively design systems for the next generation of ubiquitous is applications mixing is with disciplines such as industrial design. an interpretive research approach has been selected because the project aims at building understanding of the role of design representations during the design process and klein and myer (1999) suggest that interpretive research facilitates understanding ‘human thought and action in social and organizational contexts’. a case study method has been selected because of the exploratory nature of the research (marshall and rossman 1995). a series of case studies; starting first in the world of industrial design, but then migrating to the world of is design, including designing for the world of ubiquitous is applications, is planned. the initial case study, which is the focus of this paper, was conducted at an industrial design company we shall call indesigns. an industrial design company was chosen because of the rich selection of design artefacts evident in their process, the emphasis on creativity in design and the short life-cycle of the development process. the fieldwork took place over a period of six months and involved the investigator spending at least three hours each week at the company and also one two week period full time. the weekly visits allowed observation of a variety of projects at different stages. this provided a good understanding of the types of representations produced and their uses. during the full-time period at the company one project was followed from inception, allowing a more complete understanding of the overall process. data collection involved participant observation supplemented with semi-structured interviews, and the examination of archived documents and artefacts used by designers. most of the interviews were taped as were many design team meetings. the data were analysed using standard inductive techniques such as coding and thematic analysis (babbie 2001; miles and huberman 1994). an initial conceptual framework, developed from the literature, was used to focus data collection and to provide initial categories for coding. these early categories were replaced with categories that arose from analysis of the data. finally, the concepts, derived from the analysis, were presented to the participants at the end of the case study period. two of the most experienced designers in the organisation attended and gave their support to the conceptualisations. findings from the case study in this section, the basic design process at indesigns is outlined. then the physical design representations that are created during the process are described. the conceptualisations arising from the analysis of the data are then presented, followed by a brief concluding reflection on some of the possible implications of the case study to the future world of is applications design. ajis volume 12 no. 2 may 2005 167 organisational background indesigns employs between three to six designers depending on work-load and it specialises in the design of electronic devices such as personal digital assistants, electronic monitoring devices and small kitchen appliances. indesigns typically has up to six projects going at any one time and it also manages the manufacturing side of the product design process. during this study six different designers were studied. two of these designers had less than two years experience and the other four had a minimum of eight years experience. design process at indesigns indesigns employs a process which they conceptualise as having three main phases: concept development; design development; and engineering design. during concept development the client’s idea is explored and different designs are put forward for discussion. during design development the selected design elements are further elaborated with the focus moving to the component parts of the product and how they will fit together. finally, during engineering design the detail design is further refined using a cad engineering package. at the end of this stage an exact prototype of the product can be built from the cad data. as in all design the process is not linear but involves much iteration, however the process moves the design from the socio-cultural to the techno-physical; from consideration of how the client idea can be designed to satisfy some human need or desire, to the various aspects of the design which will eventually enable a syntactic description of the artefact to emerge. the use of design representations is central to this process. design representations used table 1 below summarises some of the external, durable and self-generated (brereton 2004) design representations that have been observed in use during the design process. this is not to underplay the importance of the more transient representations such as conversations and gestures. the table characterises the observed design representations in terms of form (the physical form the representation takes), description (description of the content of the representation) and role/actors (the role the representation plays in the design process and who uses it). table 1: the observed form, description, role and actors of industrial design representations. form description role/actors brief (textual descriptions) high level product description making ideas and intentions tangible. client and designer negotiate. emails (textual descriptions plus sketches) information discussed verbally often accompanied by sketches summarising knowledge, confirming details, making less ambiguous. designers send to other stakeholders.. concept (presentation) boards attractive but vague pictures of various designs of the product summarising designers’ ideas. designers present to client. sketches – used for communication design elements whole designs communicate with other designers, clients, or manufacturers. ajis volume 12 no. 2 may 2005 168 form description role/actors sketches – used for designer’s thinking tasks. design features such as where an antenna will sit. allow designer to work through idea and problem solve visually. also allows for idea generation whether feature, theme or style. electronics printout size and shape of internal electronics printed out from cad in 1:1 scale so form can be drawn around it. designer use to understand one constraint on the form. 2d drawing technical drawings with measurements of dimensions product, wall thicknesses, clearances etc allow designers to check measurements 2d image attractive picture of product facilitate requirements elicitation. designer presents to client. allow visual, symbolic and emotional thinking 3d solid internal and external walls and components of product shown in solid form allow designer to check alignment of edges. designer presents to client in order to see the product in 3d. model can be rotated. 3d wire frame view in cad internal and external walls and components of product shown such that all lines are visible designer moves around in space to visualise where something can fit. foam mock-up hand made foam model of artefact. for designers: to work out various design features ie: where a particular led had to be situated. to overcome ‘scale’ problem. that is the problem that designers have of understanding the size of components when modelling on computer. foam mock-up computer generated cad information used to create exact representation of product. created for client to enable a concrete experience with the artefact: allows experience that can’t be found in drawings, for instance, how it feels in the hand. sla representation (basic) cad data is used to produce a polymer model of the product. for designers to see how 3rd party components fit into the model and to determine it the product can fit together properly. sla painted and weighted the sla is sanded down and painted to resemble the final product. for hand held devices the sla is weighted. for designers: to get a feel for the weight and balance of how it sits in the hand. for clients to fully experience the product. ajis volume 12 no. 2 may 2005 169 form description role/actors web cad presentation way of presenting final design concept. software allows client to see model as it is rotated. other software allows designer to annotate design in real time to highlight various features. used by designers with a phone hook up to explain design to client. power point presentation highlight design or aspects of the design. way to sequence and ration information used for long distance presentation to the client. models from silicon exact representation of product mould made from sla. also painted and finished. for client and client’s sale team to determine interest in product before production. animation animation of product highlighting functional aspects aim to generate a desire for product. client can use to sell idea to others. indesigns can use to sell extra features. quality assurance diagram line drawing to emphasise one critical detail. used by designers to emphasis critical manufacturing elements “the jpeg” captured screen dump from cad 3d solid showing internal surface. designers then import into photoshop and annotate with arrows and text. for instance: ‘front boss’, ‘back mounting plate’. used to facilitate communication with manufacturers. gets around ‘compass’ problem. that is, which way is up, where the front/back is located. discussion in this section three conceptualisations arising from the data analysis, presented in outline in table 1, are elaborated. the first of these is the notion of views of the artefact that different design representations focus designer attention in different ways. by examining views across different design contexts we may gain understandings that can benefit designers of new information appliances and systems. secondly, we provide a high-level description of the industrial design process and map this process against the views. finally, using the design process mapping, we are able to describe some of the affordances of particular design representations analysed. again this analysis, if taken across different design contexts may offer additional understandings and tools to designers striving to meet new design challenges. views although the design of an artefact needs to be considered as a whole, different representations are required to facilitate consideration of different aspects or views of the design (avison and fitzgerald 1995). these aspects or views cover both the socio-cultural and the technical and embody the ‘area of concern’ of a design. from observations and the examination of physical design representations the following views at indesigns were identified: context of use, aesthetics, marketing, function, manufacturing, materials and technical. brief descriptions of these are presented in column 1 of table 2. the first four ajis volume 12 no. 2 may 2005 170 (context of use, aesthetics, marketing and function) are predominately in the realm of the sociocultural. their concern is to enable thinking about the idea conceived to satisfy a human need or desire. the final three (manufacturing, materials and technical) are predominately in the realm of the techno-physical concerned with materialising ideas. although all views are considered concurrently by industrial designers, it has been observed that the emphasis shifts over time, with the sociocultural views predominating early and the more techno-physical views later in the process. from a broader perspective, it is interesting to note that there appear to be no views that enable industrial designers to reason about the social impact of their designs. however, one designer did mention his belief that reasoning about the environmental impacts of designed artefacts will become increasingly important. rosenman and gero (1998) also argue that different aspects or views represent stakeholder concerns and are implicit in the types of design representations created. thus, it follows that views will differ depending on the design situation because of differing stakeholder concerns. the conceptualisation of design views above is useful for two reasons. firstly, by gaining an awareness of such views across contexts we may be able to enrich all design practice. secondly, the view concept is useful in terms of allowing us to think about how design representations are used in the design process. this is explored in the next section. conceive, refine, breakdown cycle from a high level of abstraction, the industrial design process, that has been observed, can be conceptualised as a process of idea generation and refinement which is interrupted frequently. this was observed not only for the cental design concept as a whole but also for the myriad individual ideas that make up the final design. it has been observed that an idea is conceived and refined, before an interruption leads to rethinking of the concepts. such interruptions in the refinement process may be either intentional or unintentional, triggered by both socio-cultural and technical issues. alternatively, they may be deliberately sought as a means of improving some aspect of a design. these interruptions can be seen to occur within views or when the integrity of a design as a whole cannot be sustained. figure 1 depicts the notion of views and breakdowns. idea syntactic expression of artefact socio-cultural techno-physical breakdowns within views breakdowns across view view 1 view 2 view 3 view 4 figure 2: breakdowns along and across views. different design representations were observed in use in concept generation, refinement and to trigger or understand the interruptions that occurred. in concept generation high-level solution ideas are formulated. this involves exploration of the problem space and the generation and evaluation of various solutions all at a fairly high level of abstraction. however, sometimes discussion was observed to stall on technical issues and these were then explored in more detail. within the case study, concept generation was associated with a great deal of verbal discussion, ajis volume 12 no. 2 may 2005 171 the use of analogy and metaphor and throw-away sketches which are used to help communicate ideas or illustrate problems. once an idea is decided upon work then begins to refine and develop the idea. refinement is associated with a move to more detailed, formal representations. for instance, industrial form design (the design of the product shape and styling) moves from hand sketches to computer generated illustrations. reproducing a sketch using the more exact medium of a computer drawing package forces a refinement of the concept. breakdowns occur within views when there is an interruption in the flow of concept development. such interruptions may be deliberately sought because they can be seen to improve some aspect of the design. for instance, a designer may sketch a particular aspect of the form design hoping it may trigger a slightly different (better) idea. interruptions, along views may also occur because of sociocultural reasons, for instance becoming aware that the target market wants a ‘round’ aesthetic rather than a ‘pointy’ aesthetic, or technical reasons such as the planned manufacturing process being unavailable to meet the required time-line. similarly, industrial designers often use ambiguous representations as a way of opening up dialogue and eliciting design ideas from clients. many more of these types of interruptions occur early on in the process, in the socio-cultural realm, because of the need to develop a shared meaning around the desired product. during member checking (which involved presenting, to participants, the concepts derived from the analysis) designers confirmed the notion that they seek interruptions in pursuit of a creative leap. as one designer put it: “interestingly you can determine how many interruptions you want to have in many ways because you can go with the path of least resistance if you want to. ….(in this case) we might not use as many tools (representations) along the process because basically a number of them might offer the interruption we don’t necessarily want. (however) if you want to do something that is going to be a little more ground-breaking then you go down many paths, use many tools. this is in design, development, technical, functional, everything.” although industrial designers focus on a particular view they are aware of the implications of their decisions on other aspects of the design. breakdowns occur not only with representations used to focus on a particular view but because of the interconnection between views. for example, a designed product shape could not be manufactured from the originally selected material because of technical issues that were not known at the concept stage. breakdowns that occur when it is no longer possible to refine/develop the design idea as a whole have a far greater impact than breakdowns along a single view and these are not sought. for instance, in one observed product design, a breakdown occurred because it became apparent (through the refining process) that a particular gasket required considerable strength from the supporting structures to hold it in place. this meant that either the materials or build approach had to be re-conceptualised. breakdowns across views were characterized by a great deal of questioning, queries and the search for further information as well as many detailed, throw away sketches used to communicate and convince others of the problem. the breakdown stage, whether occurring along or across views led to another round of concept generation. table 2 below summarises the identified views, described in column 1, and the type of design representations used in concept generation and refinement. ajis volume 12 no. 2 may 2005 172 view concept generation refinement context of use (concerned with understanding the environment the product will be used in – not always modelled) discussions, sketches of envisaged area of use (or photos of existing area of use) , sketches of people using product photos or drawings of area of use with drawings overlaid with sketches of the product and people using them. test rigs: 3d working model used to test usability principles. aesthetics (concerned with form and style) discussions, throw away sketches, sketches of various foam concepts, electronics printout, handmade foam models, concept boards, metaphors and analogy. sketches of various design elements, 2d computer generated images, 3d computer generated images that can be rotated, computer generated foam models marketing (concerned with market positioning and branding) discussions, mood boards, branding, metaphors, hand sketches 2d computer generated images, 3d external form, rub downs (graphics), animations, painted and weighted prototype functional (what functions the device provides – designers also think about how user will interact with product) discussions, textual descriptions, analogy handmade form models, sketches of aspects of the design that the user will interact with materials (has implication for form, structure, texture, manufacture) discussions around the properties of materials and the implications for the design. discussions about source, availability, implications of use manufacturing (concerned with how the product will fit together) sketches of how the product will be assembled 2d technical drawings, qa diagram, 3d cad drawings (wireframe and solid view), sla prototype technical (concerned with issues such as ability to withstand weather conditions, shock, radio frequency shielding etc) discussions, textual descriptions, sketches standard tests eg. shock table 2: view and representations for concept development and refinement nature and affordances of design representations affordances involve the interaction between an actor and the properties of an environment (gaver 1991; norman 1999). in the context of social interaction affordances have also been used to describe the material properties of the environment that affect how people interact (gaver 1996) (kreijns and kirschner 2001). this is not to say that social behaviour cannot be accounted for in terms of ‘social conventions’ and ‘communities of practice’ but the affordance approach examines how social activities are shaped by the material environment (gaver 1996). in the same way we can examine the extent to which the physical properties of design representations shape the design process supporting both the socio-cultural and techno-physical aspects of the design. the case study data suggests that industrial design representations used for concept generation tend ajis volume 12 no. 2 may 2005 173 to be those that allow speedy generation, evaluation and exchange of ideas as well as providing high level abstractions and ambiguity. the design representations used for refinement tend to fall into two groups – throw-away representations created solely to allow the designer to work through some aspect of the design (for instance throw-away sketches and foam models) and representation that allow more formal, concrete expression of ideas (for instance hand-drawn sketches followed by 2d computer representations moving finally to 3d computer representations). concept generation as discussed above, during concept generation high-level solution ideas are formulated. this is a very fluid part of the design process and we can conjecture that design representations should facilitate speedy generation, evaluation and exchange of ideas as well as providing high level abstractions. ambiguity would also be useful. the case study data suggests that design representations used for concept generation do tend to have these features. verbal discussions and hand drawn sketches are fast and ambiguous, the use of analogy and metaphor “save lots of time in understanding what the client has in mind” (quote from designer). concept boards feature attractive vague pictures. one designer explained it like this: “concept boards don’t have any consideration towards manufacturing, no detail, they are very ambiguous. they just look nice. they have no detail. see this shape here, that could be a cut, it could be a different material, a different colour. at the moment it is just a shape.” the advantage of this ambiguity is that it fosters communication and requirement elicitation. the same designer explained it as follows: “we can then ask questions like: ‘what do you think if that was a different material?’; ‘what do you think if that was a sticker?’; ‘do you like the fact that it is a different colour or do you think it should be all the same colour?’. so it is just a talking point. we will throw in some different ideas.” refinement refinement is associated with a move to more detailed, formal representations. this part of the design process requires thinking through issues so high-level concepts can be made more concrete. we can conjecture that design representations need to facilitate designer thinking, and to enable more detail to be added. an analysis of the case study data suggest that design representations used for refinement do tend to fall into two groups – throw-away representations created solely to allow the designer to work through some aspect of the design and representations that allow more formal, concrete expression of ideas. for example, to help him think through an issue such as where a cord would come out of a product one of the designers used many sketches focussing on just that feature. in other situations, a designer may create a foam model by hand to help think through an issue. one designer described the reason he decided to make a foam model while he was trying to refine a design in modelling software. “this (the foam model) was built for me to get my head around how big it is because after building it yesterday in prodesigner (software package) i thought it looked pretty big, pretty bulky, the size looked awkward, so i built up this model this morning and it doesn’t seem as big as i thought”. the foam model provides the designer with something a computer-generated image can not. as this designer went on to explain: “the one thing that might be hard to understand on the screen is the physical size, because you can be working right in close and you get totally disoriented by the scale, although you know its dimensions. it is really hard to understand scale until you get a prototype back or make a foam model.” the goal in refining an idea is to bring it closer to a formal syntactic representation. there will be a number of representations used along this road. in form design for instance, a designer begins with ajis volume 12 no. 2 may 2005 174 hand-drawn sketches and as the next stage reproduces them as a 2d image in software. because the computer software is more exact than a hand sketch designers explained that it allows a first cut at refining the form concept. designers explained this in terms of: “it allows more detail, you can get it looking quite realistic”; “it gives more of a concrete idea”; “(it is) better to explain things like outlines, colours and materials.” the next stage of refinement for form design comes with the move to modelling the concept as a 3d surface drawing. breakdown breakdowns occur within views when there is an interruption in the flow of concept development. these interruptions may be sought as a means of improving understanding. interruptions along views may also occur because of socio-cultural or technical reasons. breakdowns were observed to also occur when it was no longer possible to refine/develop the design idea as a whole. these types of breakdowns are not sought and in the case study were often flagged by a ‘salient detail’ – one detail that could not be worked around and led to the need to reconceptualise part of the design. breakdowns across views occur because it is not possible to understand all the issues at the conceptual design stage. for instance, in one product design the light emitting diodes (leds), for aesthetic purposes, where not situated above the position of the leds on the printed circuit board (pcb). this is not unusual and in the design light pipes were used to bring the light from the pcb to the surface. however, it was not until testing that radio frequency leakage around the light pipes was discovered to be an issue. this technical issue lead to a breakdown that required the form design to be reworked. representations used around breakdowns included those that allowed communication of the problem in a convincing way as well as representations that allow analysis and discussion. representations included detailed sketches backed up by verbal discussions and often supported by textual explanations. characteristics of representations studied to summarise then, some of the characteristics of design representations noted were: speed of creation and change, level of abstraction and detail, amount of ambiguity and degree of context provided. other important characteristics of the design representations included: the ability of the representation to depict the physical size and feel of the intended artefact, the ability of the representation to enable reasoning about ‘the compass problem’, that is which way is up, down, front and back and the ability of the representation to elicit a positive emotional response from the client. implications for is design – some initial comments as previously indicated, the present research seeks to build an enhanced understanding of the nature and affordances of representations in the design process and to argue the need for information systems as a discipline to open up discussion of the design representations required to effectively design systems that might be required for the next generation of ubiquitous is applications. an initial case study has been reported, drawing upon the world of industrial design as a means of exposing a rich selection of design artefacts employed in a short life-cycle development – features that might inform an enhanced understanding of is design as it faces increasingly novel design issues and shortening development times. the analysis reported above has highlighted: • a rich classification of views of the artefacts; • an industrial design conceive, refine, and breakdown cycle, both within and ajis volume 12 no. 2 may 2005 175 across views; and • an interpretation of the application of design artefacts at various stages of the design cycle in terms of the nature of affordances. the implication of these emerging understandings for design in the is context remains. suffice to observe that the notion of a “catastrophe cycle”, possibly analogous to breakdown, has been observed in information systems design (nguyen et al. 2000). further, recent work in creativity and the is requirements engineering process (cybulski et al. 2003) has highlighted, the importance of creative insight in the conceptualisation of information systems – a process that may well be enhanced by further investigations of the notions of views and affordances of design representations opened up by the case study reported in this work. further studies in other contexts will follow this case-study, in particular contexts that facilitate understanding of if and how the insights gained from this first case study might translate to designing for the world of ubiquitous computing. from this we hope to develop a framework that will enable the links between the various characteristics of design representations to be drawn together. having such a framework for understanding the nature and affordances of design representations would provide a basis for investigating which representations are most effective to tackle specific design tasks. conclusions design is a complex human activity that requires consideration of both socio-cultural and technical issues. design representations are instrumental in the process of moving from an artefact idea to embodiment in the techno-physical realm. the socio-cultural realm, because it exists only as ‘human concepts and interpretations’ (rosenman and gero 1998), is where we can conjecture that there may be a great deal that different design disciplines can offer each other especially as we design for mobile, wearable, tangible and ubiquitous computing that radically alters the context of traditional information systems. this will require an expansion of our notions of the ‘area of concern’ that design must consider. many researchers are attempting to uncover the design issues in these new areas (see for example (dix 2000); (mcneese 2003; york and pendharkar 2004)). what is also required is an understanding of the nature and affordances of the design representations that will be required to enable reasoning in these new areas. as benyon (2002) puts it “the concepts and representations that we need for designing such experiences may take us a long way from where we are now.” references avison, d.e, and g. fitzgerald. 1995. information systems development: methodologies, techniques and tools. berkshire, uk: mcgraw-hill international (uk) limited. babbie, e. 2001. the practice of social research. belmont, ca.: wadsworth. benyon, d. 2002. "representations in human-computer systems development." cognition, technology and work 4: 180 196. brereton, m. 2004. "distributed cognition in engineering design: negotiating between abstract and material representations." pp. 83-103 in design representation: springer-verlag london limited. carroll, j, s howard, f vetere, j peck, and j murphy. 2001. "identity, power and fragmentation in cyberspace: technology appropriation by young people." pp. 95-102 in twelfth australasian conference on information systems. coffs harbour, nsw australia. chen, p.p.s. 1976. "the entity-relationship model: toward a unified view." acm transactions ajis volume 12 no. 2 may 2005 176 on database systems 1: 9-36. cybulski, j., l. nguyen, t. thanasankit, and s. lichtenstein. 2003. "understanding problem solving in requirements engineering: debating creativity with is practitioners." in pacific asia conference on information systems. dix, a. 2000. "exploiting space and location as a design framework for interactive mobile systems." acm transaction on computer-human interaction 7: 285-321. dym, c.l. 1994. engineering design: a synthesis of views. cambridge, uk: cambridge univerty press. faro, a., and d. giordano. 1999. "ontology, aesthetics and creativity at the crossroad in information system design." pp. 124-131 in creativity and cognition. loughborough uk. galle, p. 1999. "design as intentional action: a conceptual analysis." design studies: 57-81. gaver, w. w. 1991. "technology affordances." pp. 79-84 in chi'91. new orleans, lousiana: acm, new york. gaver, w. 1996. "affordances for interaction: the social is material for design." interactions 8: 111129. hevner, alan r., salvatore t. march, jinsoo park, and sudha ram. 2004. "design science in information systems research." mis quarterly 28: 75-105. klein, h.k., and m. myers. 1999. "a set of principles for conducting and evaluating interpretive field studies in information systems." mis quarterly 23: 67-94. kreijns, k., and p.a. kirschner. 2001. "the social affordances of computer-supported collaborative learning environments." pp. t1f-12 to t1f-17 in 31st asee/ieee frontiers in education conference. reno, nv: ieee. kyng, m. 1995. "making representations work." communications of the acm 58: 46-55. marshall, c., and g. b. rossman. 1995. designing qualitative research. thousand oaks, ca.: sage publications. mcneese, m.d. 2003. "new visions of human–computer interaction: making affect compute." international journal of human-computer studies 59: 33-53. miles, m. b., and a. m. huberman. 1994. qualitative data analysis: an expanded sourcebook. california, usa: sage publications. nguyen, l., j. carroll, and p. swatman. 2000. "supporting and monitoring the creativity of is personnel during the requirements engineering process." pp. 1-9 in proceedings of the 33rd annual hawaii international conference on system sciences, institute of electrical and electronics engineers. hawaii, usa. norman, d.a. 1999. "affordance, conventions, and design." interactions may + june: 38-42. perry, m., and d. sanderson. 1998. "coordinating joint design work: the role of communication and artefacts." design studies 19: 273-288. rosenman, m.a., and j.s. gero. 1998. "purpose and function in design: from the socio-cultural to the techno-physical." design studies: 161-186. saddler, harry. 2001. "understanding design representations." interactions july + aug: 17-24. simon, h.a. 1996. the sciences of the artifical. cambridge, ma: mit press. stolterman, e. 1999. "the design of information systems: parti, formats and sketching." information systems journal 9: 3-20. suchman, l. 1994. "preface." in hawaii international conference on systems sciences. hawaii, usa. york, j., and p. c. pendharkar. 2004. "human–computer interaction issues for mobile computing in a variable work context." international journal of human-computer studies article in press, corrected proof. ajis volume 12 no. 2 may 2005 177 copyright the following copyright statement with appropriate authors’ names must be included at the end of the paper [susan keller, ross smith, steve howard, jennie carroll] © 2004. the authors assign to acis and educational and non-profit institutions a non-exclusive licence to use this document for personal use and in courses of instruction provided that the article is used in full and this copyright statement is reproduced. the authors also grant a non-exclusive licence to acis to publish this document in full in the conference papers and proceedings. those documents may be published on the world wide web, cd-rom, in printed form, and on mirror sites on the world wide web. any other usage is prohibited without the express permission of the authors. microsoft word ajis vol 15-1 m.doc australasian journal of information systems volume 15 number 1 december 2007 113 user participation in contemporary is development: an is management perspective laurie mcleod auckland university of technology, new zealand email: laumcl88@aut.ac.nz stephen g. macdonell auckland university of technology, new zealand email: smacdone@aut.ac.nz bill doolin auckland university of technology, new zealand email: bdoolin@aut.ac.nz abstract user participation in is development has become an established practice perceived to improve both the development process and its outcomes. recently, however, markus and mao (2004) have highlighted the need to revisit user participation in light of the changing is development environment. a survey of new zealand organisations with 200 or more full-time employees was undertaken in order to obtain an updated assessment of the actual practice of user participation in is development projects from the perspective of is management. the results suggest that user participation continues to be a dominant aspect of is development for is managers. the paper provides empirical data on the common reasons for having users participate and the form and types of user activities in development. the responding is managers perceived user participation to be beneficial to is development in their recent is projects, and many intended continuing or increasing their current levels of user participation in the future. australasian journal of information systems volume 15 number 1 december 2007 114 introduction the participation of users in is development has been an important focus of interest within the is literature since the early 1980s (ives & olson, 1984), when a perceived ‘user relations problem’ (friedman & cornford, 1989) led to calls to better articulate user requirements or improve user influence, concerns with democratising is development, and attempts to align the social and technical aspects of development (knights & murray, 1994). for example, the development of a ‘participatory design’ philosophy in europe and scandinavia encouraged active user participation both to improve the system developed and to allow users to co-determine the effect of the system on their workplace (clement & van den besselaar, 1993; iivari & lyytinen, 1999; mumford, 1997). socio-technical approaches to is development, such as ethics (effective technical and human design of computer-based systems) (mumford, 1997, 2000) or multiview (avison, et al., 1998), made user participation an important part of design. other development approaches emerged that emphasised user participation, including joint application design (jad) (carmel, et al., 1993; davidson, 1999), prototyping (baskerville & stage, 1996; beynon-davies, et al., 1999), and information engineering (ie) (beath & orlikowski, 1994). in addition to these specific user-centred approaches, the notion of ‘user participation’ (as distinct from user involvement; barki & hartwick, 1989; barki & hartwick, 1994a) has become commonly used to describe the activities performed by users or their representatives in is development. extensive support for user participation in is development can be found in both the is academic and practice literature (e.g. johnson, et al., 2001; reel, 1999). indeed, it has become institutionalised in is development practice through education and training, standard reference texts, standard methods of is development, and standards or best practice prescriptions such as iso standard 13407 (iso, 1999), ieee standard 1233 (ieee, 1998) and swebok (bourque, et al., 2004). however, this institutionalisation of a somewhat abstracted perspective of user participation (as represented in standards, textbooks and the like) presents a rather simplified view of the concept. as symon & clegg (2005, p. 1142) caution, the term ‘user participation’ can be problematic as “different constructions of ‘user’ and ‘participation’ are possible and inevitable”. a number of authors have conducted meta-analyses of empirical studies examining the relationship between user participation and is success (cavaye, 1995; hwang & thorn, 1999; mahmood, et al., 2000). overall, it seems that while in some studies user participation was found to positively influence is outcomes, many studies were inconclusive. in terms of specific empirical studies, a number have identified a significant positive relationship between user participation and is success (coombs, et al., 1999; doherty, et al., 2003), user satisfaction or acceptance (foster & franz, 1999; lin & shao, 2000; lu & wang, 1997), project completion (wixom & watson, 2001; yetton, et al., 2000), project performance (aladwani, 2000), system impact (lynch & gregor, 2004) or data quality (zeffane & cheek, 1998). case study evidence also suggests that active user participation is an important component of successful is development (butler, 2003; wastell & newman, 1996; wilson, et al., 1997). similarly, lack of user participation is perceived to contribute to is failure or abandonment (clegg, et al., 1997; johnson, et al., 2001; keil, et al., 2002; newman & noble, 1990; pan, 2005; sauer, 1999; schmidt, et al., 2001). the perception that user participation can play an important role in influencing is project outcomes has meant that it has become an established practice within many organisations, with users playing an increasingly important role in is development. for example, howcroft & wilson (2003b) describe an organisation in which user participation became so entrenched in the is development culture that it was inconceivable that an is project would be developed without the participation of australasian journal of information systems volume 15 number 1 december 2007 115 users. in a survey of irish project managers kiely & fitzgerald (2002) found that users played a substantial part in development in two thirds of is projects. given the modern, dynamic is development environment of short-term business needs, rapid application development, internet, multimedia and e-commerce applications, packaged software and outsourcing (barry & lang, 2003; sawyer, 2001; schmidt, et al., 2001), the nature and extent of user participation may well be changing (markus & mao, 2004). empirical knowledge of the current forms of is development practice is needed (barry & lang, 2003; ljung & allwood, 1999), including knowledge related specifically to user participation. cross-sectional surveys are a useful tool for gathering descriptive information from a large sample on current practices (fitzgerald, et al., 1999; markus & mao, 2004). this paper reports the results of a survey conducted on is development practice in recent is projects in new zealand organisations. one of the primary objectives of the study was to obtain an updated picture of actual user participation practices in is development. the remainder of the paper is structured as follows. first, we review prior literature on the nature of user participation in is development, including its benefits and limitations. we then outline the development of our survey instrument and the data collection procedure used in this study. subsequent sections present and discuss the results of the survey, before some concluding remarks are made about the findings. the nature of user participation there are potentially many reasons why an organisation may choose to have users participate in is development. the decision may rest with one of the many groups within an interest in the project (such as top management, the is function, users, an external consultant, or an external development company). an organisation may have an organisational policy on user participation, or it may be established practice within the organisation. user participation may be a requirement of the standard method of is development being used or it may be appropriate given the characteristics of the project (wilson, et al., 1997). a number of authors have suggested that ‘user participation’ is not a homogeneous construct (e.g. hartwick & barki, 2001; iivari, n., 2004; ljung & allwood, 1999; palanisamy, 2001). for example, cavaye (1995) identifies various dimensions of user participation relating to the proportion of users who participate, the formality of users’ participation, their level of responsibility during development, the effect or influence of participation on the development effort, and the level of participation during different phases or aspects of development. accordingly, individuals may participate in an is project directly, or indirectly through user representatives (barki & hartwick, 1994a; butler & fitzgerald, 2001; ives & olson, 1984). for logistical reasons, is projects usually involve user representatives rather than all intended users of the is (cavaye, 1995). in a survey of swedish computer consultants, ljung & allwood (1999) found that half of the respondents did not consult the entire user population. some respondents felt that the complete user population was too large to participate, given the financial or time constraints present in many is projects. other respondents felt that smaller groups were important for ensuring that users were heard. the conventional wisdom is that user groups from all organisational levels and functions, and reflecting all levels of expertise, need to be considered for participation in order for that participation to be representative and to capture the breadth and depth of expertise in an organisation (damodaran, 1996; ives & olson, 1984; newman & noble, 1990; pan, et al., 2004; wilson, et al., 1997). australasian journal of information systems volume 15 number 1 december 2007 116 however, several problems have been identified within the is literature concerning which users actually participate in an is project. for example, the increasing numbers and types of affected or intended users in the changing is development context can increase the difficulty of securing appropriate participation (markus & mao, 2004). it may be difficult to identify or access specific users of an is, particularly in the development of packaged software (iivari, n., 2004; kujala, 2003; ljung & allwood, 1999). the people chosen to represent the users may not have the same job descriptions or work tasks as the staff they are meant to represent, particularly if they are not chosen by the users themselves. the same representatives may be involved in several projects within the organisation, because of their interests or prior experience, so that the is function loses direct contact with the wider user community (ljung & allwood, 1999). in short, users can be a diverse group with different, and possibly conflicting, interests and needs (asaro, 2000). user participation activities may be performed individually or as part of a group (barki & hartwick, 1994a). participation may involve formal groups or teams (e.g. steering committees, reference groups), individuals in a formal advisory capacity (as official liaison between the project team and the application domain), official meetings or seminars, or informal relations, discussions and tasks (barki & hartwick, 1994a; butler & fitzgerald, 2001; cavaye, 1995; damodaran, 1996; heinbokel, et al., 1996; ljung & allwood, 1999). user input may be sought through seminars, workshops or meetings, questionnaires, interviews, observation, email or bulletin board (butler & fitzgerald, 2001; gallivan & keil, 2003; ljung & allwood, 1999; palanisamy, 2001). some participation activities are believed to produce better is outcomes than others, e.g. using non-technical approaches in requirements determination or observing users at work (markus & mao, 2004). users or their representatives may have various levels of responsibility in is development (barki, et al., 2001; mckeen & guimaraes, 1997). they may participate in an advisory role (as a group or individually), as part of the project team, be given sign-off responsibilities at various stages, or be given full responsibility for development (butler & fitzgerald, 2001; cavaye, 1995; dodd & carr, 1994; heinbokel, et al., 1996; ljung & allwood, 1999; mckeen & guimaraes, 1997; palanisamy, 2001). in a survey of is users, hartwick & barki (1994) found that activities reflecting overall responsibility for development were more important than hands-on activities or activities reflecting users’ evaluation of work done by is staff, in terms of influencing users’ attitudes to the is. empirical studies have shown that organisations where users and is staff are held jointly accountable appeared to have higher perceived project success rates than the average (wastell & sewards, 1995). the influence that users can have on how the is project is managed and on the final design of the is product varies, ranging from no influence (where users are not asked, are unwilling to participate, or have no impact) to strong influence (where the user department pays for and/or controls the development) (damodaran, 1996; hartwick & barki, 2001; hunton & beeler, 1997; ives & olson, 1984; palanisamy, 2001). in between these extremes, their influence often depends on the role they play in development (e.g. project leadership, as part of the team, given sign-off responsibility, or in an advisory role only), how well they are represented (e.g. few or unrepresentative users participate), when and how frequently they interact with the project team, and the amount of consideration given to their contribution by the project team (e.g. whether their contribution is considered, taken seriously, ignored, misunderstood, or not even solicited) (butler & fitzgerald, 2001; cavaye, 1995; gallivan & keil, 2003; hartwick & barki, 2001; iivari, j., 2004; iivari, n., 2004; ives & olson, 1984; ljung & allwood, 1999; lynch & gregor, 2004; palanisamy, 2001; saleem, 1996). the stage(s) in the is development process at which user participation occurs can also impact on the project outcome (lin & shao, 2000; mckeen, et al., 1994; mckeen & guimaraes, 1997; saleem, 1996). empirical studies have shown that user participation in the early stages of development can australasian journal of information systems volume 15 number 1 december 2007 117 have greater impact on user acceptance of an is than participation at later stages (foster & franz, 1999; kujala, 2003; pan, 2005). participation throughout the entire development process may similarly increase the likelihood of user acceptance (butler & fitzgerald, 1999b; butler & fitzgerald, 2001). user participation is most likely to occur in early stages of (traditional) development, such as problem definition and requirements determination, and in latter stages of development, such as testing and installation. users do not usually participate in physical design and coding (butler & fitzgerald, 2001; cavaye, 1995; foster & franz, 1999; heinbokel, et al., 1996; mckeen & guimaraes, 1997; wastell & sewards, 1995). in the modern is development context, however, users may participate in a wider variety of technical and non-technical activities than may have been previously the case (e.g. business process redesign, it infrastructure development, project management or change management) (markus & mao, 2004). the benefits of user participation are well rehearsed in the is literature. in an early and influential paper, ives & olson (1984) argued that user participation can improve system quality by providing more accurate and complete user requirements, by providing knowledge of the business context, avoiding unacceptable or unnecessary system features, and improving user understanding of the is. further, they suggested that user participation may lead to greater user acceptance by creating realistic user expectations of the is, by providing a forum for conflict resolution about design issues, by fostering feelings of ownership and commitment, and by decreasing resistance to change. subsequent studies (some of which were based on empirical data) have reinforced these potential benefits, which are summarised in table 1. user or business participation is also emphasised as an important part of is development within the new zealand is practice literature (bell, 1998; newman, 2004; smith, 2003). user participation is not without its perceived limitations (table 2). for example, user participation can increase the complexity of the is development process, particularly where it is intensive, or may be difficult to implement or manage in practice. the greater the number of users or user groups involved, the longer it may take to reach agreement and the greater the amount of resources required. even with user participation, user resistance may still occur (butler & fitzgerald, 2001; howcroft & wilson, 2003a). it has been suggested that ignoring users’ suggestions may have worse consequences than not involving users in the first place (canel, et al., 1997; mckeen & guimaraes, 1997). however, drawing on a survey of project managers and users involved in is projects in large us organisations, mckeen and guimaraes (1997) found no evidence of dysfunctional effects of user participation, even in is projects where user participation was relatively unnecessary. in summary, while at an abstract level there is broad and long-standing support for user participation in is development, a more fine-grained analysis reveals some variation in opinion. furthermore, actual empirical analysis (as opposed to more speculative commentary) suggests a range of practices and attitudes in terms of user participation. given that the is development context continues to change, further empirical analyses should provide valuable insights into the stability or otherwise of these practices and attitudes. this provides the underlying motivation for the research now described. australasian journal of information systems volume 15 number 1 december 2007 118 user participation may … sources improve the quality of the is developed (butler & fitzgerald, 2001; canel, et al., 1997; hwang & thorn, 1999; kim & peterson, 2003; lin & shao, 2000; mckeen, et al., 1994; roberts, et al., 2000) avoid unacceptable or unnecessary system features from being developed (cavaye, 1995; kujala, 2003; lin & shao, 2000; mckeen, et al., 1994; mckeen & guimaraes, 1997; roberts, et al., 2000) provide a more accurate and complete assessment of user requirements (barki & hartwick, 1994a; butler & fitzgerald, 1999a; cavaye, 1995; foster & franz, 1999; kim & peterson, 2003; kujala, 2003; lin & shao, 2000; mckeen, et al., 1994; mckeen & guimaraes, 1997; roberts, et al., 2000; wixom & watson, 2001) provide developers and the is group with knowledge of the is context, particularly in relation to the organisation and functional unit it is to support (butler & fitzgerald, 1999b; canel, et al., 1997; lin & shao, 2000; mckeen, et al., 1994; mckeen & guimaraes, 1997; roberts, et al., 2000; yetton, et al., 2000) improve user understanding of the is and its features (barki & hartwick, 1994a; kim & peterson, 2003; kujala, 2003; lin & shao, 2000; mahmood, et al., 2000; mckeen & guimaraes, 1997; wixom & watson, 2001) increase user acceptance of the is (butler & fitzgerald, 1999b; butler & fitzgerald, 2001; foster & franz, 1999; kim & peterson, 2003; kujala, 2003; lin & shao, 2000; mahmood, et al., 2000; mckeen, et al., 1994; roberts, et al., 2000; saleem, 1996; wixom & watson, 2001) lead to user satisfaction with the is (butler & fitzgerald, 1999a; canel, et al., 1997; hwang & thorn, 1999; kujala, 2003; lin & shao, 2000; lu & wang, 1997; lynch & gregor, 2004; mahmood, et al., 2000; mckeen, et al., 1994; mckeen & guimaraes, 1997; yoon, et al., 1995) help to create more realistic user expectations about is capabilities (butler, 2003; lin & shao, 2000; mckeen, et al., 1994; mckeen & guimaraes, 1997; roberts, et al., 2000) provide a forum for bargaining and conflict resolution (amoako-gyampah & white, 1997; barki & hartwick, 1994b; butler, 2003; butler & fitzgerald, 1999b; canel, et al., 1997; foster & franz, 1999; jiang, et al., 2002; lin & shao, 2000; mckeen, et al., 1994; mckeen & guimaraes, 1997; roberts, et al., 2000) facilitate communication between participants, particularly users and developers (amoako-gyampah & white, 1997; hartwick & barki, 2001; markus & mao, 2004; mckeen, et al., 1994) australasian journal of information systems volume 15 number 1 december 2007 119 foster feelings of ownership of the is by users (barki & hartwick, 1994a; butler, 2003; butler & fitzgerald, 1999b; hartwick & barki, 1994; lin & shao, 2000; lynch & gregor, 2004; mahmood, et al., 2000; mckeen, et al., 1994; mckeen & guimaraes, 1997; roberts, et al., 2000; saleem, 1996; yoon & guimaraes, 1995) foster user commitment to the is (butler, 2003; canel, et al., 1997; cavaye, 1995; kim & peterson, 2003; lin & shao, 2000; mckeen, et al., 1994; mckeen & guimaraes, 1997; roberts, et al., 2000) decrease user resistance to change (amoako-gyampah & white, 1997; butler, 2003; cavaye, 1995; lin & shao, 2000; mckeen, et al., 1994; mckeen & guimaraes, 1997; roberts, et al., 2000) ensure the developed is meets user needs (butler, 2003; cavaye, 1995; kim & peterson, 2003; lynch & gregor, 2004; pan, et al., 2004; yetton, et al., 2000) table 1: potential benefits of user participation user participation may … sources increase project duration, particularly where there are large numbers of users or multiple user groups with differing interests (canel, et al., 1997; cavaye, 1995; iivari, n., 2004; kujala, 2003; ljung & allwood, 1999) increase project cost, particularly where there are large numbers of users or better informed users needing more support (cavaye, 1995; iivari, n., 2004; kujala, 2003; ljung & allwood, 1999) be difficult to manage or implement, particularly where there are time constraints, large numbers of users, competing user groups, or where users are unable or unwilling to participate. (butler & fitzgerald, 1999b; canel, et al., 1997; cavaye, 1995; heinbokel, et al., 1996; kujala, 2003; ljung & allwood, 1999) create or reinforce user resistance to implementation of the is, particularly where users perceive that their suggestions have been ignored. (butler & fitzgerald, 2001; canel, et al., 1997; markus & mao, 2004; mckeen & guimaraes, 1997) reduce developers’ influence in the is development process, as users exert greater influence. (cavaye, 1995; heinbokel, et al., 1996) table 2: potential limitations of user participation research approach our aim was to gain a detailed understanding of organisational practices in relation to user participation in is development. as such, it was decided that a cross-sectional survey would be the most appropriate mechanism to use in order to achieve the desired breadth of coverage. we chose to use a multi-page web-based design for our survey instrument. dillman’s (2000) principles for constructing web surveys were followed in order minimise the effects of measurement, nonresponse, coverage and sampling errors. the survey was composed primarily of a questionnaire that solicited information about aspects of is development practice in new zealand organisations, including a detailed sub-section addressing the participation of users in the is development process. australasian journal of information systems volume 15 number 1 december 2007 120 given the lack of an agreed understanding of the meaning of ‘user participation’ (symon & clegg, 2005), participants were not given a definition of the term (although ‘user’ was defined as ‘an employee of the organisation who interacts with the is on a day-to-day basis’). we wanted participants to consider user participation in a general sense and without undue influence or constraint from us, so as to be more inclusive of participatory practices that may have been in use. in the first part of the questionnaire, participants were asked to specify (or estimate) the total number of is projects that had been undertaken and completed by their organisation during a threeyear period. participants were asked to categorise these is projects in terms of their size (as measured by their cost), different forms of is development and acquisition, and different levels of user participation in the development process. for those projects in which users did not participate, participants were prompted to specify the reasons for this. participants were then asked a series of questions related to the nature of user participation in their is projects. this included the most common reason for user participation, the type and form of that participation, and the stages of is development in which it occurred. participants were also asked to indicate their level of agreement with a range of positive and negative statements about user participation drawn from the benefits and limitations of user participation identified in the prior studies reviewed above (tables 1 and 2). in answering these questions, participants could choose from a five-point likert-type scale comprising “strongly disagree”, ”disagree”, ”neutral”, “agree” and “strongly agree” options, or alternatively a “don’t know or not applicable” option. all participants were then asked to identify any anticipated changes in user participation in is development in their organisation in the three years to follow. finally, participants were asked to specify their official position and to classify their organisation in terms of business sector, organisational size, and the size and location of its is function. we pilot tested the survey with 20 organisations in march 2004, which produced minor modifications to question wording. the main survey was undertaken during april and may 2004. the target population was those organisations large enough to require is beyond that which could be achieved by standard desktop applications, to have an inherent need for systematisation and computerised integration of business functions, and be more likely to utilise up-to-date software innovations and development practices. overall, the survey was administered to 460 new zealand public and private sector organisations with 200 or more ftes. the manager responsible for is project work within the organisation (typically an is/it manager or cio) was targeted as the respondent in order to provide both an organisational view and one informed by knowledge of the organisation’s is development practice. although single-respondent managerial surveys have their limitations with respect to distance from actual development work (wynekoop & russo, 1997), managerial level respondents are more likely to be knowledgeable about organisation-wide issues (doherty & king, 2001). we acknowledge, however, that it is often management that defines the boundaries of user participation (howcroft & wilson, 2003a, 2003b), and that the survey findings represent an is management perspective on user participation. in order to ensure currency of the results and to ensure more accurate recall by survey respondents, the survey focused on is projects undertaken and completed (or substantially completed) in the three calendar years 2001 to 2003. the number of responses received was 113, for a response rate of 25%. seven responses were unusable, either because critical (demographic) data was missing or the reported organisational size was below 200 ftes. this left 106 usable responses that formed the basis of subsequent data analysis (although not all respondents answered every question). characteristics of the responding organisations are shown in table 3. the 106 organisations in the respondent population represent australasian journal of information systems volume 15 number 1 december 2007 121 17% of the target population and provide a reasonable match with respect to business sector and organisational size. just over half the organisations reported sizes of is function of fewer than 10 ftes, and in the majority of organisations (78%) the is function was located in one central unit. business category communications & media construction & engineering education, health & community services electricity, gas & water utilities finance, insurance & banking government & local government it, business, legal & property services manufacturing & processing primary industries tourism, accommodation & food services transportation, logistics & storage wholesale & retail trade organisational size (fte) 200 to 499 500 to 999 1000 to 1999 2000 or mor size of is function (fte) fewer than 4 4 to 9 10 to 19 20 to 49 50 to 99 100 or more location of is function centralised distributed mainly outsourced don’t know respondent’s role chief information officer is manager development manager project manager system administrator non-is manager % organisations (n=106) 2 8 16 3 8 12 8 24 1 3 6 11 % organisations (n=106) 43 25 13 19 % organisations (n=104) 23 30 9 13 16 10 % organisations (n=106) 78 12 8 1 % organisations (n=106) 22 45 9 8 5 10 table 3: characteristics of respondent organisations australasian journal of information systems volume 15 number 1 december 2007 122 results and analysis is projects data summarising the is projects reported on in the survey are presented in table 4. the number of projects completed (or substantially completed) over the three year period 2001-2003 ranged from 0 to 230 projects per organisation, resulting in an average of 7 projects per organisation per year. five percent of organisations did not undertake any is projects in that time, while 59% of organisations undertook between 1 and 10 is projects. just over half of the reported is projects (54%) were bespoke developments, while the remaining 46% involved the purchase of packaged software or applications. further detailed analysis of these responses and others associated with general is development practices can be found in mcleod et al. (2006). number of projects undertaken by an organisation 0 1-5 6-10 11-20 21-50 51-100 more than 100 type of is development/acquisition purchase of packaged software: with little or no customisation with in-house customisation with outsourced customisation bespoke is development: in-house outsourced project costs $1000 or less $1,001 $10,000 $10,001 $50,000 $50,001 $100,000 $100,001 $500,000 $500,001 $1,000,000 $1,000,001 or more don’t know the project cost % organisations (n=105) 5 39 20 13 12 7 4 % projects (n=2039) 18 13 15 42 12 projects (n=2215) 7 20 27 18 16 7 5 1 table 4: is project details australasian journal of information systems volume 15 number 1 december 2007 123 extent of user participation table 5 shows the extent of user participation in the is development process reported for the more than 2000 projects surveyed across 100 responding organisations. users participated in close to all of the reported projects (92%) for at least part of the development process, and for more or less all of the development process in 61% of the reported projects. this level of user participation is higher than that reported in kiely and fitzgerald’s (2002) survey of medium to large irish organisations, in which users participated in 65% of the projects and played a significant role in 56% of projects. all responding organisations who undertook is projects reported having at least some level of user participation in at least some of their projects. of these, 84% always had users participating, 46% for more or less all of the development process. users did not participate in at least some projects in 16% of organisations. user participation users did not participate users participated for more or less all of development users participated for only part of development users never participated users participated users participated for more or less all of development in at least some projects users participated for only part of development in at least some projects users did not participate in at least some projects % projects (n=2129) 8 61 31 % organisations (n=100) 0 100 77 50 16 table 5: extent of user participation the most common reason given for users not participating in particular is projects was that the projects were perceived to be of little or no relevance to users, usually because of their technical or infrastructural nature. of course, all is projects (even technical ones) will ultimately have some users, so presumably these respondents were referring to business users. for example, one respondent noted that “technical projects would involve mainly it professionals, whereas customerfacing projects would involve users”. users were also not involved in two projects where the is was packaged software requiring little or no customisation, consistent with findings reported by butler and fitzgerald (1999a). other reasons given for not including users were that development was done overseas; users were unavailable to participate in the development; and it was not organisational policy. in the latter case, the respondent noted that until recently the importance of user acceptance was not recognised by his or her organisation, a reasonably surprising comment given the relatively widespread adoption of user participation practices. in contrast, the first two reasons given are consistent with prior literature which suggests that users may not be able to participate in is development (or may only be able to participate in a restricted way) because of their geographical location, the architectural layout of the office buildings, or other job commitments (butler & fitzgerald, 2001; cavaye, 1995; nandhakumar & jones, 1997). australasian journal of information systems volume 15 number 1 december 2007 124 nature of user participation the most common reason given by respondents for having users participate in their is projects was because of its fit with the characteristics of the project (27%) (see table 6). this is consistent with the is literature that suggests that user participation is important with certain types of projects, such as large, technically complex or cross-functional projects (butler & fitzgerald, 2001; cavaye, 1995; howcroft & wilson, 2003b; lin & shao, 2000; mahmood, et al., 2000; mckeen, et al., 1994; yetton, et al., 2000). a further 19% of respondents practiced user participation because it was a requirement of their standard method(s), which typically support some degree of user participation (cavaye, 1995; damodaran, 1996; iivari, j., 2004; kirsch & beath, 1996; kujala, 2003; nandhakumar & jones, 1997; roberts, et al., 2000). overall, 31% of organisations reported that users participated in their is projects for organisational reasons, namely because of organisational policy or historical practice within the organisation. the influence of users was given as a reason for user participation by 18% of respondents. this may reflect organisations where the is function is regarded as a support service to the business, or where users are politically active or have sufficient influence to require participation. no organisations employed user participation because it was the choice of an external development company. most common reason for user participation fit with project characteristics requirement of a standard method historical practice in the organisation influence of users organisational policy choice of external development company other types of user participation mainly user representatives participated all users participated % organisations (n=101) 27 19 19 18 12 0 6 % organisations (n=99) 94 6 table 6: nature of user participation in almost all of the responding organisations (94%), user representatives typically participated in is development (table 6). this is consistent with other studies, which acknowledge the difficulties (or impossibility) of involving all users, particularly given the increasing numbers and types of affected users in the modern is development context (e.g. cavaye, 1995; ljung & allwood, 1999; markus & mao, 2004). in only 6% of organisations did all users typically participate in the is projects undertaken. the six organisations who responded in this way were relatively small (with 200 to 499 ftes) and undertook a relatively small number of projects (five of them undertook four or fewer projects over the three years). however, their projects ranged in size, with 41% costing more than $100,000, suggesting that in at least some of the projects full user participation may have involved significant logistical issues (cavaye, 1995; markus & mao, 2004). respondents were asked to indicate how frequently various forms of user participation occurred. the distribution of responses for this question is shown in figure 1, with the various forms of user participation ordered from left to right in increasing levels of participation (and, to some extent, increasing levels of responsibility conferred on the users). the first four forms involve relatively low levels of user participation and, apart from where users have sign-off responsibility, limited australasian journal of information systems volume 15 number 1 december 2007 125 accountability. in contrast, user participation and levels of responsibility are higher where users are part of the development team or they have full responsibility for development. 0 10 20 30 40 50 60 70 80 90 100 users were informally consulted during development (n=101) users participated in a formal advisory capacity as individuals (n=101) users participated in a formal advisory capacity as a group (n=101) users had signoff responsibility at various stages of development (n=101) users were part of development team (n=100) users had full responsibility for development (n=100) form of user participation % o rg an is at io ns always often sometimes never figure 1: various forms of user participation the proportion of organisations who reported using the latter two forms of user participation (development team membership or full responsibility for development) is noticeably less than the forms with lower levels of participation and responsibility. this is consistent with other empirical studies which report that user-led development is less prevalent than other forms of user participation (barki & hartwick, 1994b; dodd & carr, 1994; mckeen & guimaraes, 1997; wastell & sewards, 1995). all but one of the 101 organisations informally consulted users during the development process to some extent. the proportion of organisations not using a particular form of participation increases moving through the range of forms of participation from left to right in figure 1. notably, 65% of the organisations never gave users full responsibility for development. seventy-seven percent of the organisations informally consulted users during development on a regular basis (often or always). between 60% and 70% of the organisations regularly involved users in a formal advisory capacity and/or gave them sign-off responsibility at various stages in development. users were regularly part of the development team in 48% of the organisations and users regularly had full responsibility for development in 11% of the organisations. the proportion of organisations that always used a particular form of participation varied from 2%, where users always had full responsibility for development, to 34%, where users always had sign-off responsibility at various development stages. it should be noted that the form of user participation within a given project or organisation does not necessarily bear any relationship to attaining the benefits of user participation. within the is literature, it is generally accepted that users need to perceive that their participation (or that of their low participation high participation australasian journal of information systems volume 15 number 1 december 2007 126 representatives) is meaningful in order to realise such benefits (butler & fitzgerald, 2001; cavaye, 1995; hunton & beeler, 1997; lynch & gregor, 2004; markus & mao, 2004; saleem, 1996). while users are more likely to perceive that they have had greater influence where they have had full responsibility for development or been part of the development team, is projects in which users are consulted and their input seriously considered can still achieve the benefits of user participation. respondents were asked to indicate how frequently users participated at different stages of is development. the distribution of responses for this question is presented in figure 2. users most often participated in requirements determination, testing, training, and evaluation. the majority of organisations (between 80% and 90%) regularly (often or always) involved users in each of these phases of development. this is consistent with reported practice where, apart from the elicitation of requirements, users tend to be involved in the latter stages of is development. organisations less commonly involved users in the planning, design or installation of is projects. just over half of the organisations regularly involved users in planning (57%) and design (54%). fewer organisations (40%) regularly involved users in the installation of is projects. as might be expected, only 8% of organisations regularly involved users in the programming for is projects. these results are consistent with previous empirical is literature (butler & fitzgerald, 2001; cavaye, 1995; foster & franz, 1999; heinbokel, et al., 1996; mckeen & guimaraes, 1997; wastell & sewards, 1995). 0 10 20 30 40 50 60 70 80 90 100 r eq ui re m en ts de te rm in at io n (n =1 00 ) te st in g (n =9 9) tr ai ni ng (n =1 00 ) ev al ua tio n (n =9 7) pl an ni ng (n =1 00 ) d es ig n (n =1 00 ) in st al la tio n (n =1 00 ) pr og ra m m in g (n =1 00 ) stages of is development % o rg an is at io ns always often sometimes never figure 2: user participation in various stages of is development while markus & mao (2004) point out that in the modern is development context users may also be involved in a wider variety of activities than previously, this was only found to be the case in two organisations in this study, where respondents specified that users often participated in either configuration or organisational change management. australasian journal of information systems volume 15 number 1 december 2007 127 benefits of user participation respondents were asked to indicate their level of agreement with various positive statements about user participation in their is projects undertaken over the three-year time frame. these statements are shown in figure 3, in order of level of agreement. the mean ratings for all statements are above the neutral value of 3, implying that the respondents tended to agree with these benefits of user participation (the 5-point likert scale labels were assigned a value from 1 to 5). the majority of respondents (82%) agreed that user participation facilitated successful is development. as one respondent noted, “it is clear that the projects that had user participation … from the start have been far more successful”. meeting user expectations is perceived as an ongoing challenge by cios across new zealand (bhargava, 2006; hind, 2002), and the next five highly ranked benefits of user participation focused on aspects of user acceptance of the developed system. these included creating realistic user expectations of the is, ensuring the developed system met user needs, and generating user commitment to and satisfaction with the system. there was relatively less (although still high) support for the benefits of user participation in facilitating the process of is development, such as by facilitating effective communication and conflict resolution between developers and users, ensuring comprehensive user requirements and ensuring adequate developer knowledge of the is context. overall, respondents were least convinced that user participation avoided unacceptable or unnecessary system features, although over half of the respondents (56%) still agreed with this statement. 0 10 20 30 40 50 60 70 80 90 100 avoided unacceptable system features (μ=3.42,n=98) facilitated user-developer conflict resolution (μ=3.66,n=97) ensured developer knowledge of context (μ=3.70,n=96) ensured accurate & complete requirements (μ=3.73,n=99) led to delivery of high-quality system (μ=3.86,n=99) facilitated developer-user communication (μ=3.84,n=98) led to user satisfaction with system (μ=3.95,n=99) ensured the system met user needs (μ=3.98,n=99) led to user commitment (μ=4.02,n=99) ensured user understanding of system features (μ=3.93,n=98) created realistic user expectations (μ=3.98,n=99) facilitated successful is development (μ=4.02,n=99) % organisations agree neutral disagree figure 3: relative agreement with positive statements about user participation australasian journal of information systems volume 15 number 1 december 2007 128 limitations of user participation respondents were asked to indicate their level of agreement with various negative statements about user participation in their is projects undertaken over the three-year time frame. these statements are shown in figure 4, in order of level of agreement. the mean ratings for all statements are below the neutral value of 3, implying that respondents tended to disagree with these negative statements about user participation in the projects reported on in this survey. this is consistent with the overall beneficial perception of user participation identified above. 0 10 20 30 40 50 60 70 80 90 100 created user resistance to implementation of the system (μ=1.96,n=98) constrained developers' influence in the development process (μ=2.34,n=96) was difficult to manage or implement (μ=2.40,n=98) was time-consuming or costly (μ=2.54,n=98) % organisations agree neutral disagree figure 4: relative agreement with negative statements about user participation less than one in five respondents (17%) agreed with the statement that user participation in their projects was time-consuming or costly. in fact, over half the respondents (55%) disagreed with this statement. similarly, only 12% of respondents agreed that user participation in their is projects had been difficult to manage or implement. while some developers (or the is function more generally) may endeavour to protect their interests in relation to is development in a specific project, this was not regarded as an issue by most respondents in this survey, with only 12% agreeing that user participation in their is projects constrained developer influence in the development process. almost no respondents (3%) considered that user participation in their is projects actually created user resistance. proposed changes in user participation respondents were asked to comment on anticipated changes to user participation in is development in their organisations. of the 66 responses to this question (table 7), 44% indicated that there would be little or no change to the current level and form of user participation. two respondents (3%) commented that there would be less user participation in future. one of these specifically commented on the need for the is developers to have “better veto rights on scope creep”. while scope creep might be a function of user participation, it is not necessarily so – it might suggest difficulties in project management rather than being a direct criticism of user participation. in contrast, just over half the respondents (53%) indicated that more user participation in is development was expected to occur. some of these respondents also provided information on envisaged changes to the form of participation. analysis of these responses revealed a number of common themes (table 7). seven respondents (11%) anticipated a significant change towards ownership of is projects by users in their organisations. these respondents talked about users as increasingly becoming “owners”, “drivers” and “leaders” of is development. the language they used included terms such as “influence” and “empowerment”. business ownership of is projects ties in with the increased alignment of it with business reported earlier. both of these seem to have been topics of discussion australasian journal of information systems volume 15 number 1 december 2007 129 in the practitioner literature around the time that the survey was conducted (bell, 2003). for example, kumove (2003) argues that business owners should be held (at least jointly) accountable for is projects, on the grounds that when it is aligned with business, is projects are the is components of business projects. such arguments also appear in the more recent practitioner literature (e.g. bell, 2005; kpmg, 2005; paredes, 2005). for example, cramm (2005) argues that for it to be viewed as a business enabler, the it organisation needs to delegate to the business control over the “what” of it (as compared to the “how”). by this, she includes deriving it-enabled business strategies and plans, establishing priorities and service requirements, allocating funding and approving vendors. changes in user participation % organisations (n=66) little or no change less user participation more user participation impacts of changes in user participation greater user ownership of projects creating more formal roles for users increased use of user knowledge more development work by users more controlled user participation 44 3 53 11 6 5 3 3 table 7: changes in user participation three respondents (5%) suggested that participation in is development by users would increase due to the need to access their knowledge base. in one case, “this is because the nature of those projects demands extensive knowledge of detailed facets of the company's operating environment, and it will be necessary for us to tap into that knowledge to gain not only a better functional outcome, but also to encourage ownership at the user level.” in another case, it is “critical, given that we don't have an internal is development team to share and own some of the knowledge”. this latter comment suggests that if the outsourcing of is development increases, users may become more involved in is development because of the lack of institutional knowledge and memory among the external is developers. four respondents (6%) talked about creating more clearly defined or formal roles for users in the development process, often including formal approval or sign-off of is deliverables. two respondents (3%) suggested that new development tools would allow users to do more development work, including customisation, themselves. two other respondents (3%) wanted more user participation, but in a controlled way. in one case, apparently, “allowing users sign-off rights tends to slow things down too much”. conclusion this study has provided an updated assessment of user participation in is development based on empirical data from actual is projects in new zealand organisations. user participation was practiced to some extent by all organisations in the survey and in 92% of the reported projects. this level of user participation is higher than that previously reported by kiely and fitzgerald (2002). australasian journal of information systems volume 15 number 1 december 2007 130 the most common reasons given by responding organisations for having users participate in their is projects were organisational policy or historical practice (31%), fit with the characteristics of the project (27%), a requirement of their standard methods (19%), and the influence of users (18%). almost all organisations (94%), had user representatives typically participating in is development. the six organisations in which all users typically participated in the is projects undertaken were relatively small and undertook relatively fewer projects. while users were regularly part of the development team in just less than half (48%) of the responding organisations, the level of user-led development remains low, with users regularly having full responsibility for development in only 11% of the organisations, and 65% of the organisations never giving users full responsibility. in the majority of organisations users were regularly consulted, either informally or formally, and were regularly given sign-off responsibility at various development stages. users most often participated in requirements determination, testing, training, or evaluation. just over half of the organisations regularly involved users in planning and design. there was little evidence that the range of user activities was being extended in the modern is development environment (markus & mao, 2004). on the whole, it seems that the responding organisations that have users participate in is development are confident of the benefits of doing so. respondents perceived that user participation facilitated successful is development in their is projects, helped gain user acceptance of the developed system and assisted the development process. the results of this survey suggest that for most organisations the time, financial resources or effort required for user participation (cavaye, 1995; ljung & allwood, 1999) are generally accepted, rather than regarded as a negative consequence of user participation. only a small proportion of respondents felt that user participation in their is projects had been time-consuming or costly, or difficult to manage or implement. this was reinforced by the number of organisations intending to continue or increase their current levels of user participation in the future. as one respondent noted, “although participation could be seen as time consuming or costly, it is viewed within our organisation as a positive necessity”. while there is some indication in the is literature that user participation may produce user resistance in individual is projects, this would appear to be a rare occurrence based on the results of this survey. in fact, there was relatively strong agreement expressed by the survey respondents that user participation led to user satisfaction with and commitment to the systems developed. this would seem to support mckeen and guimaraes’ (1997) finding of a lack of evidence of dysfunctional effects of user participation. interestingly, one respondent noted that, in his or her organisation, there was a need for “more acceptance by users that it [user participation] is beneficial”. this comment is a timely reminder that users themselves may be reluctant to be involved or may be unconvinced of the value of their participation, and that managers and developers should not take the perceived benefits of user participation as self-evident for all stakeholders in the is development process. overall, the results of this study provide continued evidence for the widespread acceptance of user participation in is management practice. despite opportunities to comment on less conventional forms of user participation, the survey respondents generally appeared to reproduce the institutionalised and rather simplified view of user participation discussed in the introduction to this paper. consequently, more detailed and longitudinal research from a range of perspectives is necessary to explore the subtleties and complexities of user participation, in its various forms, as it is enacted in a changing is development context (markus & mao, 2004). for example, the implications for user participation of increased ownership of is projects by users, as noted in this survey, deserve closer attention. australasian journal of information systems volume 15 number 1 december 2007 131 acknowledgements we would like to thank all respondents to the survey reported in this paper. this research was funded through a top achiever doctoral scholarship by the tertiary education commission of new zealand. references aladwani, a.m. (2000) "is project characteristics and performance: a kuwaiti illustration", journal of global information management, vol. 8 no. 2, pp. 50-57. amoako-gyampah, k. & white, k.b. (1997) "when is user involvement not user involvement?" information strategy: the executive's journal, vol. 13 no. 4, pp. 40-45. asaro, p.m. (2000) "transforming society by transforming technology: the science and politics of participatory design", accounting, management and information technologies, vol. 10 no. 4, pp. 257-290. avison, d.e., wood-harper, a.t., vidgen, r. & wood, j.r.g. (1998) "a further exploration into information systems development: the evolution of multiview2", information technology and people, vol. 11 no. 2, pp. 124-139. barki, h. & hartwick, j. (1989) "rethinking the concept of user involvement", mis quarterly, vol. 13 no. 1, pp. 53-63. barki, h. & hartwick, j. (1994a) "measuring user participation, user involvement and user attitude", mis quarterly, vol. 18 no. 1, pp. 59-82. barki, h. & hartwick, j. (1994b) "user participation, conflict, and conflict resolution: the mediating roles of influence", information systems research, vol. 5 no. 4, pp. 422-438. barki, h., rivard, s. & talbot, j. (2001) "an integrative contingency model of software project risk management", journal of management information systems, vol. 17 no. 4, pp. 37-69. barry, c. & lang, m. (2003) "a comparison of 'traditional' and multimedia information systems development practices", information and software technology, vol. 45, pp. 217-227. baskerville, r. & stage, j. (1996) "controlling prototype development through risk analysis", mis quarterly, vol. 20 no. 4, pp. 481-504. beath, c.m. & orlikowski, w.j. (1994) "the contradictory structure of systems development methodologies: deconstructing the is-user relationship in information engineering", information systems research, vol. 5 no. 4, pp. 350-377. bell, c., (2003) "who owns it?" mis new zealand, 1 september 2003. bell, c., (2005) "has it lost its shine", mis new zealand, 1 june 2005. bell, s., (1998) "strengths and shortcomings", computerworld new zealand, 19 october 1998. beynon-davies, p., tudhope, d. & mackay, h. (1999) "information systems prototyping in practice", journal of information technology, vol. 14 no. 1, pp. 107-120. bhargava, v., (2006) "developing and maintaining successful slas is not so simple, says idc". http://www.idc.com.au/press/release.asp?release_id=206 accessed 17 may 2006. australasian journal of information systems volume 15 number 1 december 2007 132 bourque, p., dupuis, r. & tripp, l.l. (eds.) (2004) guide to the software engineering body of knowledge (swebok), ieee computer society press, los alamitos, ca. butler, t. (2003) "an institutional perspective on developing and implementing intranet and internetbased information systems", information systems journal, vol. 13 no. 3, pp. 209231. butler, t. & fitzgerald, b. (1999a) "the institutionalisation of user participation for systems development in telecom eireann", in success and pitfalls of information technology management, m. khosrowpour (ed.), idea group publishing, hershey, pa, pp. 68-86. butler, t. & fitzgerald, b. (1999b) "unpacking the systems development process: an empirical application of the csf concept in a research context", journal of strategic information systems, vol. 8 no. 4, pp. 351-371. butler, t. & fitzgerald, b. (2001) "the relationship between user participation and the management of change surrounding the development of information systems: a european perspective", journal of end user computing, vol. 13 no. 1, pp. 12-25. canel, c., mathieu, r.g. & das, s.r. (1997) "the successful development of information systems for fms: some useful lessons", industrial management & data systems, vol. 97 no. 8, pp. 316-319. carmel, e., whitaker, r.d. & george, j.f. (1993) "pd and joint application design: a transatlantic comparison", communications of the acm, vol. 36 no. 4, pp. 40-48. cavaye, a.l.m. (1995) "user participation in system development revisited", information & management, vol. 28 no. 5, pp. 311-323. clegg, c.w., axtell, c., damodaran, l., farbey, b., hull, r., lloyd-jones, r., nicholls, j., sell, r. & tomlinson, c. (1997) "information technology: a study of performance and the role of human and organizational factors", ergonomics, vol. 40 no. 9, pp. 851-871. clement, a. & van den besselar, p. (1993) "a retrospective look at pd projects", communications of the acm, vol. 36 no. 4, pp. 29-37. coombs, c.r., doherty, n.f. & loan-clarke, j. (1999) "factors affecting the level of success of community information systems", journal of management in medicine, vol. 13 no. 3, pp. 142-153. cramm, s., (2005) "share power to gain control", idg cio, 4 march 2005. damodaran, l. (1996) "user involvement in the systems design process a practical guide for users", behaviour and information technology, vol. 15 no. 6, pp. 363-377. davidson, e.j. (1999) "joint application design (jad) in practice", journal of systems and software, vol. 45, pp. 215-223. dillman, d.a. (2000) mail and internet surveys: the tailored design method, 2nd ed, john wiley and sons, new york. dodd, j.l. & carr, h.h. (1994) "system development led by end-users: an assessment of end-user involvement in information systems development", journal of systems management, vol. 45 no. 8, pp. 34-40. australasian journal of information systems volume 15 number 1 december 2007 133 doherty, n.f. & king, m. (2001) "an investigation of the factors affecting the successful treatment of organisational issues in systems development projects", european journal of information systems, vol. 10, pp. 147-160. doherty, n.f., king, m. & al-mushayt, o. (2003) "the impact of the inadequacies in the treatment of organizational issues on information systems development projects", information and management, vol. 41, pp. 49-62. fitzgerald, g., philippidis, a. & probert, p. (1999) "information systems development, maintenance, and enhancement: findings from a uk study", international journal of information management, vol. 19 no. 4, pp. 319-329. foster, s.t. & franz, c.r. (1999) "user involvement during information systems development: a comparison of analyst and user perceptions of system acceptance", journal of engineering and technology management, vol. 16 no. 3-4, pp. 329-348. friedman, a.l. & cornford, d.s. (1989) computer systems development: history, organization and implementation, john wiley & sons, chichester. gallivan, m.j. & keil, m. (2003) "the user-developer communication process: a critical case study", information systems journal, vol. 13 no. 1, pp. 37-68. hartwick, j. & barki, h. (1994) "explaining the role of user participation in information system use", management science, vol. 40 no. 4, pp. 440-465. hartwick, j. & barki, h. (2001) "communication as a dimension of user participation", ieee transactions on professional communication, vol. 44 no. 1, pp. 21-31. heinbokel, t., sonnentag, s., frese, m., stolte, w. & brodbeck, f.c. (1996) "don't underestimate the problems of user centredness in software development projects there are many!" behaviour and information technology, vol. 15 no. 4, pp. 226-236. hind, p., (2002) "forecast for management 2002", cio new zealand, 1 october. http://idg.net.nz/cio.nsf/unid/9476c90fa6c2a215cc256c4b007500e3?opendocument& highlight=2,new,zealand,companies. howcroft, d. & wilson, m. (2003a) "paradoxes of participatory practices: the janus role of the systems developer", information and organization, vol. 13 no. 1, pp. 1-24. howcroft, d. & wilson, m. (2003b) "participation: 'bounded freedom' or hidden constraints on user involvement", new technology, work and employment, vol. 18 no. 1, pp. 2-19. hunton, j.e. & beeler, j.d. (1997) "effects of user participation in systems development: a longitudinal field experiment", mis quarterly, pp. 359-388. hwang, m.i. & thorn, r.g. (1999) "the effect of user engagement on system success: a metaanalytical integration of research findings", information & management, vol. 35 no. 4, pp. 229-236. ieee, (1998) ieee standard 1233: ieee guide for developing system requirements specifications. ieee, new york. iivari, j. (2004) "enculturation of user involvement in software development organizations an interpretive case study in the product development context", in proceedings of the third nordic conference on human-computer interaction (tampere, finland), new york, acm press, pp. 287-296. australasian journal of information systems volume 15 number 1 december 2007 134 iivari, n. (2004) "exploring the rhetoric on representing the user: discourses on user involvement in software development", in proceedings of the 25th international conference on information systems (12-15 december, washington dc) pp. 631-643. iivari, j. & lyytinen, k. (1999) "research on information systems development in scandinavia: unity in plurality", in rethinking management information systems: an interdisciplinary perspective, w.l. currie & b. galliers (eds.), oxford university press, oxford, pp. 57-102. iso (1999) iso standard 13407: human-centred design processes for interactive systems. international organization for standardization, geneva. ives, b. & olson, m.h. (1984) "user involvement in mis success: a review of research", management science, vol. 30 no. 5, pp. 586-603. jiang, j.j., chen, e. & klein, g. (2002) "the importance of building a foundation for user involvement in information systems projects", project management journal, vol. 33 no. 1, pp. 20-26. johnson, j., boucher, k.d., connors, k. & robinson, j. (2001) "the criteria for success", software magazine, vol. 21 no. 1, pp. s3-s11. keil, m., tiwana, a. & bush, a. (2002) "reconciling user and project manager perceptions on it project risk: a delphi study", information systems journal, vol. 12 no. 2, pp. 103-119. kiely, g. & fitzgerald, b. (2002) "an investigation of the information systems development environment: the nature of development life cycles and the use of methods", in proceedings of the eighth americas conference of information systems (dallas), ais, pp. 1289-1296. kim, c.s. & peterson, d.k. (2003) "a comparison of the perceived importance of information systems development strategies by developers from the united states and korea", information resources management journal, vol. 16 no. 1, pp. 1-18. kirsch, l.j. & beath, c.m. (1996) "the enactments and consequences of token, shared, and compliant participation in information systems development", accounting, management and information technologies, vol. 6 no. 4, pp. 221-254. knights, d. & murray, f. (1994) managers divided: organisation politics and information technology management, wiley, chichester. kpmg, (2005) global it project management survey, kpmg international, switzerland. kujala, s. (2003) "user involvement: a review of the benefits and challenges", behaviour and information technology, vol. 22 no. 1, pp. 1-16. kumove, a., (2003) "why are organisations unhappy with it? part iii", mis new zealand, september. retrieved 17 may 2006 from factiva database. lin, w.t. & shao, b.b.m. (2000) "the relationship between user participation and system success: a contingency approach", information & management, vol. 37 no. 6, pp. 283-295. ljung, k. & allwood, c.m. (1999) "computer consultants' views of user participation in the system development process", computers in human behaviour, vol. 15 no. 6, pp. 713-734. lu, h.-p. & wang, j.-y. (1997) "the relationships between management styles, user participation, and system success over mis growth stages", information & management, vol. 32 no. 4, pp. 203-213. australasian journal of information systems volume 15 number 1 december 2007 135 lynch, t. & gregor, s. (2004) "user participation in decision support systems development: influencing system outcomes", european journal of information systems, vol. 13, pp. 286301. mahmood, m.a., burn, j.m., gemoets, l.a. & jacquez, c. (2000) "variables affecting information technology end-user satisfaction: a meta-analysis of the empirical literature", international journal of human-computer studies, vol. 52, pp. 751-771. markus, m.l. & mao, j.-y. (2004) "participation in development and implementation updating an old, tired concept for today's is contexts", journal of the association for information systems, vol. 5 no. 11-12, pp. 514-544. mckeen, j.d., guimaeres, t. & wetherbe, j.c. (1994) "the relationship between user participation and user satisfaction: an investigation of four contingency factors", mis quarterly, vol. 18 no. 4, pp. 427-451. mckeen, j.d. & guimaraes, t. (1997) "successful strategies for user participation in systems development", journal of management information systems, vol. 14 no. 2, pp. 133-150. mcleod, l., macdonell, s. and doolin, b. (2006) is development practice in new zealand organisations (unpublished manuscript, under review). mumford, e. (1997) "the reality of participative systems design: contributing to stability in a rocking boat", information systems journal, vol. 7 no. 4, pp. 309-322. mumford, e. (2000) "a socio-technical approach to systems design", requirements engineering, vol. 5 no. 2, pp. 125-133. nandhakumar, j. & jones, m. (1997) "designing in the dark: the changing user-developer relationship in information systems development", in proceedings of the eighteenth international conference on information systems pp. 75-88. newman, k., (2004) "strategic spin-offs", mis new zealand, 1 october 2004. retrieved 22 may 2006 from factiva database. newman, m. & noble, f. (1990) "user involvement as an interaction process: a case study", information systems research, vol. 1 no. 1, pp. 89-113. palanisamy, r. (2001) "user involvement in information systems planning leads to strategic success: an empirical study", journal of services research, vol. 1 no. 2, pp. 125-157. pan, g.s.c. (2005) "information systems project abandonment: a stakeholder analysis", international journal of information management, vol. 25, pp. 173-184. pan, g.s.c., pan, s.l. & flynn, d.j. (2004) "de-escalation of commitment to information systems projects: a process perspective", journal of strategic information systems, vol. 13, pp. 247270. paredes, d., (2005) "the new agenda setter", mis new zealand, 1 august 2005. retrieved 17 may 2006 from factiva database. reel, j.s. (1999) "critical success factors in software projects", ieee software, vol. 16 no. 3, pp. 18-23. roberts, t.l., leigh, w. & purvis, r.l. (2000) "perceptions on stakeholder involvement in the implementation of system development methodologies", journal of computer information systems, vol. 40 no. 3, pp. 78-83. australasian journal of information systems volume 15 number 1 december 2007 136 saleem, n. (1996) "an empirical test of the contingency approach to user participation in information systems development", journal of management information systems, vol. 13 no. 1, pp. 145-166. sauer, c. (1999) "deciding the future for is failures: not the choice you might think", in rethinking management information systems: an interdisciplinary perspective, r.d. galliers & w.l. currie (eds.), oxford university press, oxford, pp. 279-309. sawyer, s. (2001) "a market-based perspective on information systems development", communications of the acm, vol. 44 no. 11, pp. 97-102. schmidt, r., lyytinen, k., keil, m. & cule, p. (2001) "identifying software project risks: an international delphi study", journal of management information systems, vol. 17 no. 4, pp. 5-36. smith, p., (2003) "tools of the trade", mis new zealand, 1 october 2003. retrieved 22 may 2006 from factiva database. symon, g. & clegg, c.w. (2005) "constructing identity and participation during technological change", human relations, vol. 58 no. 9, pp. 1141-1161. wastell, d. & newman, m. (1996) "information system design, stress and organisational change in the ambulance services: a tale of two cities", accounting, management and information technologies, vol. 6 no. 4, pp. 283-300. wastell, d. & sewards, a. (1995) "an information system profile of the uk manufacturing sector", journal of information technology, vol. 10, pp. 179-189. wilson, s., bekker, m., johnson, p. & johnson, h. (1997) "helping and hindering user involvement a tale of everyday design", in proceedings of the sigchi conference on human factors in computing systems s. pemberton (ed.), acm press, pp. 178-185. wixom, b. & watson, h.j. (2001) "an empirical investigation of the factors affecting data warehousing success", mis quarterly, vol. 25 no. 1, pp. 17-41. wynekoop, j.l. & russo, n.l. (1997) "studying system development methodologies: an examination of research methods", information systems journal, vol. 7 no. 1, pp. 47-65. yetton, p., martin, a., sharma, r. & johnston, k. (2000) "a model of information systems project performance", information systems journal, vol. 10 no. 4, pp. 263-289. yoon, y. & guimaraes, t. (1995) "assessing expert systems: impact on users' jobs", journal of management information systems, vol. 12, pp. 225-249. yoon, y., guimaraes, t. & o'neal, q. (1995) "exploring factors associated with expert system success", mis quarterly, vol. 10 no. 4, pp. 83-106. zeffane, r. & cheek, b. (1998) "does user involvement during information systems development improve data quality?" human systems management, vol. 17 no. 2, pp. 115-121. stakeholder safety in information systems research australasian journal of information systems volume 14 number 1 nov 2006 stakeholder safety in information systems research dr. r.h. barbour school of computing and information technology unitec carrington road mount albert new zealand email: bbarbour@unitec.ac.nz abstract information communication technology (ict) researchers adapt and use tools from reference and cognate disciplines. this application of existing tools outside the context of their development has implications beyond the immediate problem context. ict researchers have access to a wide variety of data sources including newer ones, such as the internet, that may bring unexpected outcomes. ict research can impact on researchers, their institutions and the researched in unexpected ways. people so affected are the stakeholders in ict research activities. reputations, welfare and property may be put at risk by unplanned events described in this paper. legal aspects of ict research are broadly identified and linked to the tort of negligence. the social research association’s code for researcher safety is described and its application extended to include the internet as a potential data source. a common set of underlying ethical principles is identified suggesting that the ict researcher can refine particular research protocols for specific social contexts. keywords: stakeholder, researcher safety, research protocol, ethics, internet, social research, information communication technology, negligence, information privacy, information access, information ownership, intellectual property. introduction it is now twenty years since the publication of two papers, one by moor (1985) and another by mason (1986). for moor computer ethics in 1985 was characterised among other things by a policy vacuum in how computers should be used. for mason there were four issues: information privacy, information access, information as property and information accuracy. bynum’s (1999) paper is true to its title and provides a succinct summary of the history of ethics in computing. bynum attributes the term computer ethics to maner who used the term in the late 1970s. floridi and sanders (2002) suggest that there are at least five positions in the literature as to how computer ethics should be viewed. johnson’s (2004) paper in (floridi, 2004) advanced a view that computing introduced a new species of generic moral problems. the underlying ethical problems remain to appear in a new 43 australasian journal of information systems volume 14 number 1 nov 2006 computer related domain. other thinkers including maner (1980) believe that computing introduces radically new ethical issues that will eventually supplant conventional ethical issues as the newer issues come to predominate. this paper makes no case for any particular view but returns to each of mason’s four issues and addresses them within the perspective of stakeholder safety and in relation to modern societies that are becoming increasingly litigious. the particular motivation in this paper is to encourage the people using social research methods to develop protocols to protect the welfare of all stakeholders involved in the research process. the focus of information communication technology (ict) research is extending from interest in hardware and software to close interest in the human aspects of ict. researchers are more frequently carrying out in-depth research into human perceptions of, and responses to, ict uses and contexts. research interactions, discussed more fully below, involve risks to the human subjects of the research and the researchers who carry out the research. this work, often qualitative and using social science methods, exposes researchers to the hazards of mistaken intention, misinterpretation of instructions and behaviour, and claims of unacceptable actions that, if they arose, could be interpreted in law as negligence (cecil, 1991). specific examples where researchers may be at direct physical risk are: conducting interviews in remote or high-risk locations; delivering and collecting questionnaires from homes and flats in high-risk areas; and conducting interviews in social clubs or, in some situations, conducting focus group research in, and into, other cultures. protocols, describing actions to reduce the consequences of such hazards, take many forms and learning about them is part of the training of most professions. the goal of such training is to ensure that the researcher carries out the research in a way that ensures that the research activities do not result in damage or harm to either the public or the researcher. it is easy to understand why the medical professions have very clear protocols for medical people in their dealings with patients. it is not so clear why ict researchers should meet analogous requirements when dealing with respondents, particularly when the interactions occur in the world wide web through the medium of the internet. an indication of the recent emergence of concern for social science researcher safety is expressed in the form of guidelines for personal safety during work (kirklees community council, 2005) adapted from the social research association (sra, 2004). in (leonard, 2001) the gender specific responses to the hazards of field study in anthropology are reported as being passed down by word of mouth. the recent adoption of social science research methods in ict has resulted in a vacuum in tacit practice that might otherwise protect individual researchers. following moor’s (1985, and moor and bynum’s, 2002) concern over a vacuum in how computers should be used, this paper attempts to address an analogous issue of the vacuum related to how (ict) researchers and their institutions should act to keep stakeholders safe. in the first major section, the purposeful application of social science research methods in ict contexts is discussed. possible difficulties arising from potential failures in practice are discussed in the first sub-section. suggestions are outlined that could address research safety in the second and third sub-sections. i describe, in the second major section, a generic view of the law related to negligence and involuntary exposure to hazard. in the third major section, stakeholders are identified and strategies for resolving ict related research problems are discussed. in the fourth major section, the social research association (sra) code is described and some details discussed. in the last section, the internet-world wide web or cyberspace is considered as a data source and some special aspects of cyberspace research further considered. 44 australasian journal of information systems volume 14 number 1 nov 2006 purposeful research using social science reference discipline methods it is well understood, at the disciplinary level, that social science researchers see people and the data related to them as being worthy of a special duty of care. with respect to persons, subjects of research expect that they will retain the four rights of ownership of their own information (mason, 1986). those ownership rights include the ability to access information stored about them and to correct information that is incorrect. rights to personal information extend to privacy, that is the right to block access to information, and lastly, the right to have ownership and management of information regarded as intellectual property (moor, 1985). in essence, this position places people and their data as ‘ends’ in themselves, not means to the ‘end’ of research. at a pragmatic level, this view is held because continued goodwill in the form of public cooperation is essential for continued social ict research. the overarching purpose of social research is to better understand people and so implicitly, in some way, improve their welfare by reporting the outcome of the research. the metric for that enhanced welfare is the balance of the person’s perceptions of the good and bad consequences of the research as experienced. the actions of social researchers come under scrutiny if it can be shown that the experience of the people participating in research does not meet the expectations of the relevant discipline or profession. such expectations are described in codes of ethics and in the guidelines provided by institutional research committees for researchers seeking ethical approval for their research (johnson, 2004). in the context of research proposals encompassing research using human subjects, the deliberations of people serving on ethics committees serve three purposes. the first is to ensure that the respondents are not harmed by the research activity. the second is to ensure that either the activities or the outcomes of the research do not harm the good name of a discipline, reflecting the good name of the institution. the third, more recent, is to ensure that the welfare of researchers pursuing legitimate research (an academic activity) as paid employees, is not put at unacceptable risk. this third element in the trio of ethical responsibilities is a recent addition (johnson, 2004) to the concerns of academic employers who in turn have passed the responsibility on to academics themselves. the current disciplinary academic solution to the problem of reducing risk is to require research be supported by proposals that must be approved by an ethics committee responsible for ensuring that the proposed research will not result in a litigatable outcome. in this process, experienced academics who have already shown that they understand what is required to ensure that research risks are identified and their effects mitigated, help colleagues to plan and carry out appropriate research actions. such knowledge and experience is accumulated incrementally in disciplines as they grow and develop. ict academics, often coming from disciplines other than social science, have not had the length of development time required to ensure that, as academic supervisors, they have sufficient experience of newer social research methods to offer wise council to younger colleagues. lack of consultation with colleagues in the social sciences exacerbates a potentially avoidable situation. failure to provide appropriate advice on the consequences of proposed research exposes academic institutions to the possibility of claims of negligence discussed in the next section. 45 australasian journal of information systems volume 14 number 1 nov 2006 negligence as suggested, there is a wealth of experience in other disciplines describing research risks, identifying hazardous situations and proposing mitigating strategies. it is to this body of work that i turn to identify, in analogous contexts, those possibly unforeseen events sketched in the opening paragraphs that may affect the ict researcher. it is assumed that research is being carried out in a way that does not intentionally harm subjects. it is also assumed that the research has more than nuisance value. i begin by examining the worst possible outcome, that of a claim of negligence. in the research context, negligence is taken to mean failing to adhere to current discipline or employer protocols for the conduct of research. in law that might be seen as is reported by cecil (p 38, 1991) as: “getting wrong that which another practitioner more often than not would get right” successful claims of negligence have been shown to reflect five failures related to duty, reasonableness, causation, damage and remoteness. i describe them in the following section adapted from cecil (1991), and dealing with an english view of negligence in architecture. cook & gilbert (2004) deal with a new zealand point of view across a number of disciplines. the five situations leading to possible negligence claims are: duty (risks, the law and negligence) the law on negligence varies in detail from profession to profession and from country to country. most western jurisdictions require of citizens a duty of care in actions to protect other members of the community from unreasonable risk of harm. it is usually accepted that: • parents have a duty to care for their children. • landlords have a duty to tenants to keep a residence habitable. • professionally, engineers have a duty to ensure the bridges they build do not collapse. each duty is applicable to a particular responsibility in a context. professional activities require a higher standard of care than the average person in society. in the discussion that follows it is noted that case law is extremely limited as to research related academic negligence. jackson and powell (2002) a widely referenced series on professional negligence do not provide a separate chapter on either teaching or academic negligence. reasonableness problems arise when claims are made that a person has failed in their duty. the test is: would a reasonable person in a similar situation have done the same thing? it is usual to apply the criteria with respect to reasonableness in two ways. the first is to take a hypothetical person and ask what would they have done. the second is to ask whether the actions taken by the actual person were reasonable under the circumstances at the time. failure on either test is examined either in an ethics hearing or in a law court causation claims about research outcomes could be scrutinised on the basis of showing the damage was caused directly by negligently carrying out the research activities. research outcomes could also come under scrutiny because of a more distant consequential claim where it can be shown that the risks of harm were foreseeable. 46 australasian journal of information systems volume 14 number 1 nov 2006 damage damage can take many forms: physical, financial, psychological or social. remedies under law are complex and fall outside the scope of this discussion but are intended to offer compensation for the actions taken that caused harm. remoteness actions that cause damage must be part of normal expectations of the course of events. it must be reasonable, again, to have expected the damage under the circumstances. defences a ‘defence’ may remove claims under the law that harm was caused by negligence. in essence, in a research context, a defence would be advanced to show that the research outcomes were a result of actions outside the claims. defences may arise if it can be shown that the person making a claim took actions that contributed to the harm. a defence may arise if it can be shown that the person was informed of the nature of the research and had signed a consent form that identified the possibility of harm. lastly, a defence can take the form of a counter-claim that the harm was caused by the person’s actions in the context. the items listed above are well worth avoiding and share sufficient common ground to justify practises being developed that guide and protect researchers from the most obvious problems. policy statements in university research guidelines should identify the causes of negligence listed in general above and attempt to protect staff by insisting on approved and signed informed consent being sought from subjects. not only do researchers wish to avoid harm to other people but also to themselves, an issue examined in the next section. solutions to the problem of researcher safety the first area to look for information that will reduce harm to researchers is in the ethical guidelines of disciplines and organisations. ethics committees, charged with administering those guidelines provide three types of support in the form of: • accumulated professional and lay experience for researchers. • aggregate classes of problems for presentation to a group of individuals who have a shared memory about the conduct of research. • standard contexts for classes of research related-problems that do not easily lend themselves to administrative solution, such as check-list based evaluations. the second area to look for guidance is the codes of ethics in the professions. both ethics committees and codes of ethics have the primary focus of ensuring participant safety. as mentioned above, almost never, until recently, is there a stated concern with researcher safety (sra, 2004) or the safety of other people considered below in figure one. however, in the mid to late 1990s, concerns over work place safety (cfhs, 2005) of people who dealt directly with the public (bentley, 2005) led the social research association to consider developing a code of practice for social researchers that recognised the particular risks associated with carrying out social research. sra (2004) and cfhs list the following items as matters of concern: • risk of physical threat or abuse. 47 australasian journal of information systems volume 14 number 1 nov 2006 • risk of psychological trauma or consequences as a result of actual or threatened violence or the nature of what is disclosed during the interaction. • risk of being in a compromising situation in which there might be accusations of improper behaviour. • increased exposure to general risks of everyday life and social interaction: travel, infectious illness, accident. the cfhs (2005) lists the following item as a matter of concern: • risk of damage to, or theft of, equipment and data dealing with these threats is the responsibility of both the employer and the researcher. in most countries, an employer must meet work place safety requirements of protection from harm (kirklees community council, 2005). an employee is expected to adhere to instructions provided during employment to avoid embroiling the employer in legal claims as well as for the simple common sense reason of avoiding harm to oneself. before looking in more detail at the code, stakeholders in social research will be identified. the stakeholder view the extent of the impact of involvement in the consequences of research has been shown (mccosker, barnard & gerber, 2001) to be wider than previously thought. they show that data collection in areas of high stress or increased sensitivity may have impacts on all the people involved. mccosker et al.’s (2001) view is that researchers and other people involved in the research task may require support and counselling because of their involvement. people that may be affected include those providing data, entering and coding the data, reporting the research, supervising and reviewing the outcomes of the research process (see figure one). these newly identified elements have arisen from the context of studies of natural hazards, disasters, familial and other abuse and criminality, newsroom and chat room interactions. it is suggested that taking a stakeholder/participant view of research into sensitive issues will result in better social research and safer, more informed, communities. particular care is required where ict research involves children and other vulnerable individuals or groups. 48 australasian journal of information systems volume 14 number 1 nov 2006 data entry: transcribes voice to textsupervisor: guides researcher institutional oversight of research reviewer/examiner: assesses academic merits of research researcher: collects and reports data reader: lay public reads finished research suggested new focus of safety concerns current focus of safety concerns respondent’s group: ‘owns’ respondent: provides responses to research questions figure 1. participants potentially affected by it showing current (lower part of the figure) and suggested new foci of concern (including all stakeholders). (adapted from mccosker, barnard & gerber, 2001). safety of person and property concerns for researchers’ safety centres on security of person and property are outside those matters traditionally addressed by ethics committees. however, recent changes in law in many countries have gradually altered the nature of the workplace into one where responsibility for workplace safety is shared between employer and employee. ethics committees might reasonably be expected to request information about research protocols where the research context could place people or property at risk. the most useful contribution would be in the continuing provision of a single point of reference (an institutionally sanctioned but independent ethics committee) to which the acceptability of critical aspects of research protocols for staff and students could be confirmed. the sra code formalises the issues related to safety during research. the social research association code (sra code) the sra code (sra, 2004) describes research responsibilities, budgeting and planning, safe research designs, assessing risks, preliminary preparation for fieldwork, precautions during interviewing, handling risk situations, safety of the public and facilities for debriefing. the code is thorough, but not exhaustive, in considering aspects of the ict research protocols where human subjects are concerned. critical aspects of field research are considered below, then the internet as a data source is examined from a researcher’s point of view. 49 australasian journal of information systems volume 14 number 1 nov 2006 in the field a single researcher becomes more exposed to risk during the actual data gathering stages of research where the research takes place in the ordinary milieu of private spaces whether on-line or work or home spaces. concern for researcher safety may be heightened should the research be about sensitive topics, such as marginal or illegal uses of ict systems (stealing software, phishing, or hacking systems). institutions have a role in developing research protocols that protect employees from the worst excesses of other computer users. universities have a special duty of care to protect student researchers during their research activities (figure one). research supervisors have a role in ensuring that risk assessments are carried out and that people selected to collect data have adequate training in following accepted protocols. where research is carried out in public spaces risk assessments should include concerns over transport to and from data collection locations. where on-site work requires a lengthy period of stay away from home suitable safe accommodation should be identified. community leaders and police should be consulted for advice on local cultural norms and safe data collection locations should the topics or locale be deemed unsafe either generally or at particular times of the day. safety concerns begin during the development of research topics and in the formative stages of research proposals. choices are always open with respect to research questions, data collection methods, data sources, interview sites, paired data collection, timing of data collection. while risk can rarely be removed from data collection, care at the stage of developing research proposals is a prudent strategy that can remove potential risks through careful guidance in topic choice. choosing to interview in a public place during hours when many people are in the vicinity is preferable to interviewing in a home late a night. seeking background information about respondents and their life contexts can provide information that reduces the number of unknown elements during interviews. lodging schedules for data collection with supervisors and, in high risk areas, to the police ensures people who might need to be called on know broadly what is planned. research designs tied to the relevant reference discipline literature will alert researchers and their supervisors to potential problems resulting from the type of data being sought. special care should be taken to complete background reading in reference disciplines on the use and misuse of particular tools and techniques. some topics may invoke strong responses, especially when the respondent feels threatened, that cannot always be predicted. in any case, exit strategies should be considered by supervisors and discussed with researchers so that the consequences of unpredictable outcomes can be avoided. prior research into maps of the physical surroundings of the interview area as well as a visit with a friend to the site may be required. suitable exit strategies in more hazardous situations may include escape via known routes, moving to close proximity with a trusted associate, catching a waiting taxi or car. early visits to an interview area during the day may help establish styles and modes of dress that will enable blending in with the local cultural norms. valuable property, and equipment should be kept out of sight. carrying a cell phone, personal id, a covering letter on institutional letterhead and a personal alarm are all good practice in many contexts. cell phones can usefully be programmed with a single button press number recall in an emergency. a cell phone can be used to create a record of an interview if a call is placed to an answer phone. a schedule of planned activities can be lodged with trusted persons. regular calls to a base can help ensure that research activities are monitored. 50 australasian journal of information systems volume 14 number 1 nov 2006 researchers often spend time in the field simply observing the context and, without involvement, becoming familiar with possible subjects. additional briefing about local cultural norms can avoid behaviour that is provoking, or is inconsistent with gender expectations, gestures or language or expected social distance. co-supervisors from reference disciplines can provide essential guidance for less experienced researchers by working through plans for field research. while most social research is carried out either face-to-face or by paper, newer electronic media have particular attractions of ease of use and amenability to advanced computer based processing and analysis. the internet is considered in the next section together with some of the ethical and safety issues that may arise in ict research. the internet the internet fulfils an important role in connecting people and information across the globe in a more or less open fashion. what people write or are prepared to say in any medium may not be an accurate reflection of their actual views. the same view holds for internet accessible media as data sources. scepticism is a suitable strategy when evaluating internet sources as a research tool or information derived from the internet as suitable data for research purposes. internet based research the communications that make up the internet include email, chat rooms, discussion boards, on-line auctions, web sites and newsrooms. the major distinction that might occur is that between public and private space in cyberspace. that is whether the person making the communication intends it to be a public pronouncement or believes that the audience is restricted and therefore that the communication is essentially private (eysenbach & till, 2001). in the same paper eysenbach & till (2001) state: that where matters raised in cyberspace communications have potential to impact on the lives of people, including the researcher, steps must be taken to treat the emerging data as coming from a private space. on the other hand, people’s contributions in the public parts of cyberspace can be assumed to have the same status as letters to the editor, that is, just opinions. clues to private space are the requirement to ‘join’ the group by registration, or password, or other means of identification. some apparently open cyberspaces have clear expectations that the space is private and usually inform potential participants by announcement on the home page of the web page. public or private cyberspace the primary issue here is for the potential respondent(s) to determine whether their contributions are public or private. possible research uses of internet communications are described by (eysenbach & till, 2001) as: • passive analysis where the researcher simply records the contributions for subsequent study. the researcher is not identified or active in the communication in any way other than observing and recording that interaction. • active analysis where the researcher participates in the communications in order to confirm the accuracy of contributions. • public analysis where the researcher declares themselves to the group and gathers information, in the form of semi-structured “interviews”, on-line “focus groups”, or surveys, from the other participants. 51 australasian journal of information systems volume 14 number 1 nov 2006 key issues for researchers in cyberspace are: whether informed consent, privacy, and confidentiality issues require particular actions in the new media of the internet. if the standard of acceptance of research activities is to allay respondent’s concerns, it is clear that all three issues should be taken into consideration when designing cyberspace research. consent rarely can be waived except where privacy and confidentiality are rigorously protected by carefully anonymising data so that neither the individual nor their group can be identified in the results. gaining informed consent, where it seems a personal space is being investigated should be carried out in an open and straightforward manner. the approach may be direct, to an email address or indirectly, to the group as a whole. the list owner may help and would at least understand something of the cultural dynamics of the group members. eysenbach & till, (2001) advise caution in accepting internet contributions as there is no practical way of verifying contributions that are without attribution. even with attribution and email confirmation it is difficult to distinguish between actual and fabricated contributions. verbatim quotes can also be problematic as data sources because they may easily be traced back to original contributors. while the anonymising process may have been thorough, simply keying a string of words into a search engine is likely to return the internet resource and the author from which it was drawn. it is easy to make connections between data and persons that may then led to breaches of privacy, loss of confidentiality and ultimately, misuse of another person’s intellectual property (eysenbach & till, 2001). the three key points are: to keep distance from the data, distance between the data sources and distance between the people and the way the data is reported. clearly, the internet poses new challenges to people deciding how to interact with the rich data sources it offers. among the issues that eysenbach & till (2001) identify are: privacy, vulnerability, potential harm, informed consent and intellectual property. the issues of: data security, privacy, intellectual property, confidentiality of sources, right to correct data remain on the internet, as in any other context. the apparent ease of access to data does not remove responsibility from researchers considering possible harm, or from seeking consent to use other people’s contributions. because matching data sets is easy to carry out with publicly available search tools such as google, particular care is needed to ensure that string matching does not reveal personal details that would otherwise be inaccessible. researchers plan to publish, make public, the outcomes of their research. thus a special duty of care is required to ensure that internet participants’ rights to privacy are not trampled over in the process of carrying out research. equally, researchers considering using the internet as a data source should be mindful of the particular risks of providing their personal data during the research process. any data requested for verifying identity should be that related to the sponsoring university or institution or responsible committee where there is any possibility of negative outcomes. creating a hotmail disposable address can further anonymise email communications. google requests for web sites dealing with dos and don’ts on the internet provide more generic advice and are good discussion starting points when working through research protocols with supervisors and other researchers. such is the rapid change in societies in general and the internet in particular that any one approach is unlikely to find lasting acceptance or applicability. hence , in the literature examined no one protocol has been identified as being preferable. rather, current practice has seen the identification of a number of key considerations that should be addressed when protocols for a particular research data collection process is developed for a particular research context. eysenbach & till (2001) argue for the development of best practice protocols. the speed of growth and complexity of both global 52 australasian journal of information systems volume 14 number 1 nov 2006 society and the internet suggests a more agile approach is appropriate. a more contextually specific approach is proposed. this approach is aimed at using the established underlying ethical principles described above, supported by reasoned context specific protocols. research protocols would ideally be negotiated among stakeholders in full awareness of the special character of newer media, technologies and the actors in ict social contexts. in the next section, the major points raised in this paper are summarised. summary social research methods are being increasingly applied to many ict problem contexts where people and technology are involved. aspects of this extension, related to legal consequences, should matters prove unsatisfactory, were discussed. attention has been drawn to wider aspects of professional relationships with the community. the research process was examined by considering a stakeholder view of research activities. the extent of research engagement in the research context has raised issues related to whether participation is active or passive when internet sourced data is used. the issues relating to public and private cyberspace have been identified as concerns for both researcher and respondent. social research may harm some potential research subjects because of their particular vulnerabilities so special care is required when, for example, subjects are children. potential harm may extend to communities, so care is required where minorities are of particular research interest. informed consent is usually required from human subjects but may, exceptionally, be waived if public data is examined retrospectively in a passive anonymised way that does not cause harm to those stakeholders whose data is being researched. confidentiality is a crucial element in ensuring privacy for individuals. modern search engines enable global searches for text that may link text quoted in research to identify an individual person. where quotes are used from newer data sources, particularly from internet sources, informed consent to use those quotes is a must. people own their creations including personal data in any form of media. attribution is the usual means of recognising written intellectual property. seeking permission to use those contributions is also a usual requirement. the internet, as a data source, offers new possibilities for research yet few protocols have been developed to ensure stakeholder safety while the research is conducted. safer practices for researchers were identified and briefly discussed extending from matters of data security to personal safety to safety of equipment. the practices of social research will continue to be informed by public debate and researcher experience. universities, professions and research institutions will find proactive responses based on ethical principles more acceptable than reactive responses to the many ethical issues surrounding social science research in ict. taking a stakeholder view indicates that there is much work needed in the areas of developing local protocols acceptable to people in particular contexts in particular cultures as well as protocols for international contexts where data is gathered across cultures. 53 australasian journal of information systems volume 14 number 1 nov 2006 references bentley, j. (2005). suzy lumplugh trust. retrieved 27 nov, 2005, from http://www.suzylamplugh.org/home/history.shtml bynum t.w. (1999) a very short history of computer ethics. apa newsletters on philosophyand computers. spring feruary 1999, (last accessed wednesday, 10 may 2006) . http://www. southernct.edu/organizations/rccs/resources/research/introduction/bynum_shrt_hist.html cecil, r. (1991) professional liability. legal studies and services publishing ltd. the bath press, bath. cfhs. (2005, 18 november 2005). standard operating procedures. retrieved 25 nov, 2005, from http://www.ichs.qmul.ac.uk/research/sop/safety.html cook, p & gilbert, m., (2004) professional liability. new zealand law society seminar. june 2004. kirklees community council, k. m. (2005). research and consultation guidelines: researcher safety. retrieved 24 november, 2005 from http://www.digitv.org.uk/how_to_guide/ developing_a_ditv_service/step_by_step/research/researchersafety.asp eysenbach, g., & till, j. e. (2001). ethical issues in qualitative research on internet communities. british medical journal, 323, 1103-1105. floridi, l., & sanders, j. w. (2002). mapping the foundationalist debate in computer ethics. ethics and information technology, 4, 1-4. floridi, l. e. (2004). philosophy of computing and information: blackwell publishing ltd., oxford jackson, r. m. & powell, j. s. r., (2002). jackson and powell on professional negligence (common law library). london: sweet & maxwell. isbn 0421792205 johnson, d. (2004). computer ethics. (in floridi 2004., pp 65-75 ) leonard, d. (2001). a woman's guide to doctoral studies. buckingham: open university press, buckingham mason, r. o. (1986). four ethical issues of the information age. mis quarterly, 10(1), 4-12. maner, w (1978, 1980), starter kit on teaching computer ethics (self published in 1978. republished in 1980 by helvetia press in cooperation with the national information and resource center for teaching philosophy). mccosker, h., barnard, a., & gerber, r. (2001). undertaking sensitive research: issues and strategies for meeting the safety needs of all participants. forum: qualitative social research, 2(1). moor, j. (1985). what is computer ethics? metaphilosophy, 16(4), 266-275. moor, j., & bynum, t. w. (eds.). (2002). cyberphilosophy the intersection of computing and philosophy. malden, ma: blackwell publishing ltd., oxford. sra. (2004). a code of practice for the safety of social researchers. retrieved 24 nov, 2005, from http://www.soc.surrey.ac.uk/sru/sru29.html 54 http://www.southernct.edu/organizations/rccs/resources/research/introduction/bynum_shrt_hist.html http://www.southernct.edu/organizations/rccs/resources/research/introduction/bynum_shrt_hist.html http://en.wikipedia.org/w/index.php?title=special:booksources&isbn=0421792205 purposeful research using social science reference discipline methods negligence duty (risks, the law and negligence) reasonableness causation damage remoteness defences solutions to the problem of researcher safety safety of person and property in the field internet based research public or private cyberspace microsoft word ajisvol10 1_final_edit.doc ajis vol. 10 no. 1 september 2002 88 evaluating the knowledge assets of innovative companies maen al-hawari and helen hasan department of information systems university of wollongong wollongong australia maah33@uow.edu.au introduction knowledge has long been recognised as a valuable resource for organisational growth and sustained competitive advantage, especially for organisations competing in an uncertain environment (miller & shamsie 1987). in the current post-industrial society, knowledge is recognised as a primary source of a company’s wealth. however knowledge assets are much more difficult to identify and measure than are the physical assets with which we are much more familiar. (boisot 1998) as a company’s innovative capacity may be dependent upon its ability to take advantage of its knowledge assets, it is important to be able to identify and measure those assets. while large companies can afford extensive knowledge management projects, there is a acute need for a method by which managers in smaller organisations can easily and reliably locate, quantify and compare their knowledge assets in order to maximise their potential for innovation. the paper will begin with an overview of current thinking on the topic of knowledge management (km). it will then introduce the three functions of the knowledge life cycle (bhatt 2000, tan 2000), the four modes of conversion between tacit and explicit knowledge (nonaka 1995) and the five knowledge enablers (von krogh 2000). the research, reported here, aims to identify, from the literature, a set of knowledge elements that will give a balanced view of knowledge assets across the four modes and five enablers. an integrated model, which is the result of research to date by the authors, will then be defined. this model combines the functions of the knowledge life cycle and nonaka’s knowledge creation spiral with the notion of i-space, which has been used to classify information across three dimensions, to form a new model of k-space, which can be used to classify the knowledge elements. the paper will present this model and discuss the appropriateness of a set of knowledge elements as a means of measuring the knowledge asset of an organisation. it is suggested that the measure be verified by testing the outcomes against established indicators of innovation. the purpose of this research is to determine a practical way by which managers can determine the value of their knowledge assets and track the growth or decline of knowledge in their companies. knowledge management, technology and innovation knowledge management is a topic that is being addressed by many academic fields, including psychology, business, information technology, economics and many more. km is clearly an interdisciplinary research area and cross-functional in practice where there is disagreement as to whether km should be considered a technical issue, a human resources issue, a procedural issue or a part of strategic management (bollinger & smith 2001). it is undeniable that advances in information and communication technologies have heightened the interest in knowledge as a strategic resource (mclure, wasko & faraj 2000) and knowledge management could be viewed as the latest in a long line of applications of technology for the provision of business solutions in organisations (bollinger & smith 2001). however, as choo (1998) observes, the question of how organisations can use information and communication technologies for km is much harder to answer than it sounds. in this paper, knowledge management is defined as a set of organisational activities that positively influence knowledge creation, assisting relationships and communication between people, as well as the diffusion of local knowledge through the organisation and across the organisational boundary (von et al 2000). information and communication technologies can be considered as drivers or enablers of km. the authors come from the field of information systems and will therefore emphasise the role of information and communication technologies in these activities while recognising that there are other, non-technical perspectives on km as defined. as with researchers in most fields the authors assume the existence of some relationship between innovation and knowledge management. it is the newest information technologies that hold the most potential for innovation in an era characterised by knowledge as the critical resource for business activity (malhotra 2001). globalisation, created by new it, has placed businesses everywhere in a new and different competitive situation where knowledgeable, effective behaviour can provide a competitive edge. in this climate enterprises have turned to explicit and systematic knowledge management to develop the intellectual capital needed to succeed (wiig 1999). the so-called “productivity paradox”, which showed no relationship whatever between computer expenditure and company performance (ives 1994), was based on research into the older it. the disconnect between it expenditures and the firms organizational performance may change with an economic transition from an era of competitive advantage based on information to one based on knowledge creation (malhotra 2000). ajis vol. 10 no. 1 september 2002 89 much research in km grapples with the perceived need to pin down the “soft” concept of knowledge as has been done with the firmer concepts of “data” and “information”. a way forward has been to classify organisational knowledge into two forms: “tacit” and “explicit” (nonaka 1995, von et al 2000; kakabads & kouzmin 2001; earl 2001). identifying activities and technology that convert knowledge from one form to another, i.e. tacit to tacit, tacit to explicit, explicit to explicit or explicit to tacit, is significant for two reasons: 1. the activities and technologies, that create and transfer knowledge, are relatively easy to find and hence measure, 2. when seeking to retrieve lost knowledge, eg when employees leave the organisation, activities and technologies can be identifies to capture and manage their knowledge by making it explicit. small and medium-sized enterprises (sme) are often a fertile environment for knowledge creation and transfer and hence innovation. companies who have effective ways to manage their knowledge are much better prepared to face any changes in new economy, and thereby be innovative (harari, 1994; nonaka, 1994; west, 1992) and better decide how to invest and to compete (carneir 2000). smes are then suitable sites for the study, and adoption, of km. research design and method this research is being conducted in two phases. the first phase consists of a literature review and development of a suitable model. the model will be developed and justified in this paper, together with a body of literature used to classify knowledge-converting activities, and practical knowledge elements will be identified that can be inserted into the model. the second phase of the research, which will involve an extensive empirical study, aims to develop a practical indicator of knowledge growth in organisations. this study will use the relationship between knowledge management and innovation to test if there is merit in pursuing a balance in the set of activities converting knowledge between the tacit and explicit forms as has been shown in japanese companies (von et al 2000). the paper will discuss preliminary progress on this phase by describing the development of a suitable set of measurable knowledge elements. background concepts the aim of the first phase of the research is to identify aspects of sound knowledge management theory that could be used to provide an integrated framework for identifying different forms of knowledge and measuring the growth of knowledge in an organisation. the k-space (knowledge space) model is the result of this research. the contributing theories, and the justification for their integration into the k-space model, are now presented. the life cycle of knowledge: knowledge can be considered to pass through a number of function or phases in a cyclic fashion. the most commonly accepted are the creation, transference and management as shown in figure 1 (bhatt 2000). the core function for organisations in this cycle is management, as it could be said that without management the creation and transfer of knowledge has no direction. management may be defined in different ways, but common to most definitions used in km is that knowledge management implies directing, supporting and enabling processes that may be inherently uncontrollable or stifled by heavy-handed direction (von krogh et al 2000). figure 1 the knowledge life cycle. in the life cycle of knowledge, “knowledge management” is seen as a set of organisational activities that positively influence knowledge creation and assist relationships and conversations between people as well as the diffusion of knowledge inside or outside organization. in the context of knowledge, management is, in part, an enabling process so that “knowledge enablers” have been defined as follows: ajis vol. 10 no. 1 september 2002 90 knowledge enablers according to von krogh et al (2000 p8) there are five knowledge enablers: 1. instil knowledge vision cycle (ikvc) 2. manage conversation cycle (mcc) 3. mobilize knowledge activist cycle (mkac) 4. create the right context cycle (crcc) 5. globalise local knowledge cycle (glkc) knowledge enablers work in cycles to create and transfer knowledge because knowledge enabling should be thought of in an evolutionary manner, always aimed at continuously improving knowledge, creating and realising the potential of the company. this list is a practical set of processes that can be used in our research and emphasises the importance of the cyclic nature of knowledge processes. the four modes of knowledge conversion nonaka’s two-dimensional model for the creation of knowledge, shown in figure 2, is based on the idea that the constant interaction between tacit and explicit knowledge produces, or creates, new knowledge. tacit knowledge is highly personal and hard to formalise, making it difficult to communicate or to share with others (nonaka 1995 p8). explicit knowledge is knowledge can be easily processed by a computer, transmitted electronically or stored in database (nonaka 1995 p9). explicit knowledge is knowledge that has been externalised and exists in knowledge artefacts. tacit knowledge, on the other hand, is knowledge embodied within the minds of individual organisational members. nonaka named the four modes by which these two forms of knowledge interact as shown in figure 2. socialization is the process of converting tacit to tacit knowledge. externalisation is the process of converting tacit to explicit knowledge. combination is the process of converting explicit to explicit and internalisation is the process of converting explicit to tacit knowledge. this has led to the knowledge creation spiral of nonaka and takeuchi (1995) which views organisational knowledge creation as a process involving a continual interplay between the explicit and tacit dimensions of knowledge, cycling through the modes. in addition four levels of carriers of knowledge in organizations area are assumed, namely individual, group, organisational and interorganisational. the spiral moves and expands as it moves between these levels. figure 2 four modes of knowledge conversion knowledge elements for the purpose of this research, knowledge elements are defined as activities that managers use to manage knowledge in their organisation. these knowledge elements can be classified into four main categories by their support for the modes of conversion between tacit and explicit knowledge. knowledge elements thus are a set of activities, which can be classified as socialisation elements externalisation elements combination elements internalisation elements furthermore, each element can act as a force to move knowledge between the two different forms, tacit and explicit, and the five different knowledge enabler cycles described above. the four modes and five enabling cycles give 20 different categories and it is our assertion that a balanced assessment of these, offer a pragmatic basis for measuring the knowledge creation value of an organisation. one objective of this research is to test the socialisation externalisation internalisation combination tacit explicit tacit explicit from to ajis vol. 10 no. 1 september 2002 91 theory that innovation is best enabled by using all four different knowledge creation modes as equally possible (nonaka 1995). to reach this objective requires a means of identifying and determining the value of knowledge elements. it is planned to conducted empirical research to create and evaluate a survey instrument that could be used in organisations to determine the cumulative value of its knowledge elements and track these over time. standard deviation will be used a measure of how widely values are dispersed from the average value (the mean). the resulting measure of knowledge assets will be validated against established indicators of innovation such as that provided by the ander drejer model shown in figure 3, especially this model is defined innovation management in disruptive technology change situation. the expectation, based on the definition of innovation of nonaka discussed above is that company y will be more innovation than company x if it has a lower standard deviation of the value of knowledge elements average across the different modes. figure 3 innovation measures (ander drejer model ) an attempt has already been made by the authors to create a balanced list of knowledge elements but it was found that a more formal model was required if the survey instrument used to collect data on these elements was to yield a meaningful outcome. a solution was found in a spatial representation of knowledge elements as follows. 1.1.1...1.1 5. developing the k-space model objective knowledge figure 4: knowledge as an object knowledge as an object in 3d. in order to formalise this work is has been found helpful to consider knowledge as an object which can move in space. in the information systems paradigm an object has attributes and functions as shown in figure 4. the functions, which affect knowledge, are the knowledge elements, as defined above, and the attributes are the properties along the tacit to explicit dimension. the object moves through space driven by the effect of force. mathematically, a point in space is represented by three-dimensions. any object in space can be identified by its position; hence the properties of knowledge are the values on the dimensions of that space. each of the four modes of knowledge creation (figure 2) has been represented as a two-dimension rectangle and can be considered as a plane in space as show in figure 5. attributes ajis vol. 10 no. 1 september 2002 92 figure 5 nonaka’s four modes as planes it has been found to be relatively easy to classify knowledge elements into one of these four planes. the next step is to incorporate the dynamics of the five knowledge enablers. an information system perspective raises three issues: the challenge to codify and formalise, the need to communicate, share and transmit and the importance of retaining context when doing these. these provide three dimensions for a spatial representation of the knowledge enablers. the information space support for the view of knowledge as an object in space comes from the analogous concept of i-space, which was developed by boisot (95; 98) for information. i-space is a cube, shown in figure 6) that brings together the three essential dimensions of information; codification, diffusion and abstraction, with associated scale that ranges from codified to uncodified, from diffused to undiffused and from abstract to concrete (shariq 1998). figure 6: the dimensions of i-space (ashford 1997) from i-space to k-space: knowledge and information the concept of knowledge elements, as defined above, is now imposed onto the i-space model to create a corresponding model for knowledge objects, the k-space. this involves a mapping of the dimensions of tacit and explicit knowledge elements onto the dimensions of information in the i-space, ie: codification, diffusion and abstraction. there are many similarities in the processes associated with knowledge and information. both can be shared (eschenfelder et al 1998, cook 1999, byrne 2000) ,codified (haldin-herrgard 2000)) and encapsulated (hildreth, et al 2000), ohr 2000, baster et al 2001). in information systems terms, knowledge results from the processing of information just as information is a processing of data (hasan 2001). however knowledge could result form the processing of data where the result ajis vol. 10 no. 1 september 2002 93 is new knowledge and there is a constant dynamic of interchange between all three: data, information and knowledge (callioni 2002). for the purposes of building a model of k-space we content that the processes of sharing, abstracting and codifying information can be transferred to the concept of knowledge with some restrictions on the meaning of information and meaning of knowledge. in particular the importance of context is a feature that distinguishes knowledge from information. (brakensiek 2002, leonard & sensiper 1998). context is added to information through utility, as knowledge is often defined as information in action (sveiby 1997). utility as the context of knowledge utility is defined as the outcome of the management of knowledge. utility and context distinguish knowledge from information. knowledge affects the organisation at different levels including operational performance, the way that employees do their jobs and the way that the managers make their decisions. the utility of knowledge transcends organisational boundaries, to the life of employees and into society. knowledge increases proficiency in the performance of complex cognitive tasks (wyman et al 1998). moreover, sharing knowledge affects the working of a team and, as result, the team attitude becomes much more focussed on information sharing, transactive memory, group learning and cognitive consensus (mohammed & dumville 2001). costly errors are caused when knowledge is not shared (hoopes & postrel 1999). competitive advantage (yli-renko et al 2001), the influence of decisions (borg 2001) and effective management of the change (coffman 2000) are the result of knowledge exploitation. the current emphasis on knowledge management is greatly shifting the way which employees do their work and the way which leadership is defined (tyson 1999). knowledge helps employees to reduce the cycle time of doing their jobs (lynn 2000). indeed the ultimate utility of knowledge management in organisations is innovation. mapping the forms of knowledge onto the dimensions of information as shown in figure 6, the dimensions of information in i-space are represented as the level of coding (c=high and c=low), the level of abstraction (a=high and a=low) and the level of diffusion (d=high and d=low). these will now be discussed in terms of knowledge rather than information by adding the concepts of utility and context. figure 7: the k-space with dimensions of codification, diffusion and abstraction codification: the height of the cube represents codification, which relates to the level of classification, aggregation and analysis (ashford 1997). the need for codification is enhanced by information technology and varies with managerial level. there is a positive relation between high technology use and a manager’s ability to answer the common questions of “who”, “where” and “when” (choo 2000). change the methods used to manage the organization for innovation (gray 2001, abraham & knight 2001). increase performance (sircar et al 2000). based on this, management theory suggested that managers with the greatest ability to interpret codified knowledge should be at the highest level of the organisation. ajis vol. 10 no. 1 september 2002 94 diffusion: can be defined as the availability of information and knowledge for sharing, transmission and interchanging inside and outside organisation. abstraction can be defined as tendency of information or data to be free of the context of the community. 6. the place of tacit and explicit knowledge in the k-space nonaka’s four modes of knowledge conversion are based on the idea that knowledge is created as a result of interaction between tacit and explicit knowledge. in our model, the categories of tacit and explicit are further divided into two forms, based on the extent of diffusion and shown in table 1. 1.1.1...1.1.1 diffusi on 1.1.1...2 knowledg e form d (high) d (low) 1.1.1...3 tacit √ explicit √ semi-tacit √ semi-explicit √ table 1:forms of knowledge based on diffusion nonaka’s four tradition modes of knowledge conversion occupy faces on the k-space cube. the bottom face of the cube embraces the socialization plane (p1), tacit to tacit. the left side of the cube embraces the externalisation plane (p2), tacit to explicit. the top face of the cube embraces the combination plane (p3), explicit to explicit. the right side of the cube embraces the internalisation plane (p4). in all of nonaka’s forms the codification dimension is small even for explicit knowledge. (walter swap; dorothy leonard; mimi shields; lisa abrams; 2001). tacit knowledge in both forms is represented on the bottom face of cube. the points [cad,cad,cad,cad] delineate the tacit plane and the diffusion dimension distinguishes whether a knowledge element is semi or complete tacit. the level of abstraction determines by how much knowledge is removed from its context. when knowledge is removed from its context, more understanding is needed and knowledge is little more than information. we use “capital a” [cad,cad] for knowledge in its proper context. we are currently looking for knowledge enablers that will allow the organization to recreate the right context for codified knowledge. the prefix “semi” extends the classical definitions of tacit and explicit from nonaka’s work (1995) where tacit knowledge is highly personal and difficult to share but not impossible and explicit knowledge is easily processed and transmitted. tacit knowledge, which is undiffused, is highly tacit. in contrast, if tacit knowledge is highly diffused it can be thought of as semi-tacit as it takes on a more overt form as it is more widely known. on the other hand, explicit knowledge, which is undiffused, is only semi-explicit or relatively private. nonaka (1995) segmented tacit knowledge into two dimensions: the technical dimension and the cognitive dimension where the technical dimension is much easier to diffuse than the cognitive one. hence technology enables highly diffused tacit knowledge, i.e. “semi-tacit” knowledge that is easier to convert to an explicit form. on the other hand, completely tacit knowledge is less diffused when it is contained in mental models, beliefs and perceptions. this type of knowledge is not easy to diffuse by technology until it is codified. ajis vol. 10 no. 1 september 2002 95 figure 7: k-space showing the four planes on cube explicit knowledge in both forms are represented in the top of the k-space (p3) delineated by the points [cad,cad,cad,cad]. the diffusion dimension determines whether the knowledge element is semi or complete explicit and the abstraction dimension give the degree of context. this is so often the problem with information stored in traditional technical database systems where the context of the data is lost. the codification dimension in all forms on this plane is high, a quality that often distinguishes between tacit and explicit knowledge elements (marwick 2001). the right face of the cube (p4) represents the internalisation process and knowledge can often be treated as explicit or even just information (cad,cad, cad,cad) that is internalised by people and becomes tacit by their interpretation. technologies that aid this function are the internet, knowledge repositories or specialised information systems, which can often add some element of contextual information. internalisation adds context and knowledge without context will not added any value to organisations (merlyn & välikangas 1998). knowledge creation and the location of knowledge elements in k-space the knowledge creation spiral described by nonaka cycles around the cube as shown in figure 8. the cycle may be started at different points and take different paths. these correspond to the five different knowledge enablers described earlier. the “instil knowledge vision cycle” (ikvc), is one of those cycles which could start from (cad) or (cad). the “manage conversation cycle” (mcc) would probably start at the socialisation plane because this cycle is more related to generating knowledge by sharing it between people. figure 8: knowledge enabling cycles in k-space ajis vol. 10 no. 1 september 2002 96 the positions of elements on these planes are proving to be very important in determining where the knowledge elements are located in general. a practical set of knowledge elements is currently being refined. the first step in this process is to choose a balanced set of elements in each of the four planes equivalent to nonaka’s four mode of knowledge creation. the next step is to identify these elements within the five knowledge-enabling processes. some examples will now be given, firstly of elements, and tools to support them, in the four planes and then two examples of knowledge elements within the “instil knowledge vision cycle” (ikvc) knowledge enabler that are positioned on different planes. plane 1 in k-space (socialisation) represents the process of converting knowledge from tacit to tacit forms. knowledge elements located in this plane would be items that encourage participation in conversions between people. there may be techniques for including people with variety of educational backgrounds, ages, professional skills and functional responsibilities in a conversation or identifying the rituals that encourage entering conversation (von et al 2000). knowledge elements in this plane need systems that assist people in scanning knowledge or provide patterns or schemas to help people make sense of tacit knowledge. plane 2 in k-space (externalisation) is the process of converting tacit to explicit knowledge, with the starting point of undiffused tacit knowledge. these processes could be strongly affected by two dimensions: abstraction and codification. as the codification dimension increases, tacit knowledge is easily converted to explicit without other processing. thus any function that helps codification, such as sorting, classification and justifying would be located towards the top of this plane. lower on this plane are functions more akin to prototyping. here knowledge can be converted from tacit to explicit in a trial and error mode using computer aided design tools, case tools or end-user programming environments which do not necessarily yield a complete application. plane 3 in k-space (combination) is the process of systematizing the concepts in a knowledge repository (nonaka 1995). the coordinates of diffusion and abstraction determine the position of a knowledge element on this plane, as the codification is usually high and constant. most traditional information systems therefore play a significant role. plane 4 in k-space (internalisation) is the processing of knowledge from explicit to tacit. this plane is the antithesis of plane 2, with the diffusion value high, as the explicit knowledge is easily transmitted through electronic systems. knowledge elements found here vary in their degree of codification and abstraction. in the process of becoming tacit, knowledge losses its degree of classification and increases its degree of context. internets, intranets and portals may play roles to support knowledge elements here. ikvc example 1. “dedication to direction” is one knowledge element that can contribute to the knowledge enabler, the “instil knowledge vision cycle” (ikvc). in this approach, the vision of company is created from knowledge, which is highly specified and managers carefully and deliberately construct an explicit road map depicting the way to achieve their knowledge vision. when the vision is highly specified, the people in organisation can create and share knowledge in a highly codified form that is easy to convert into the electronic form of a machine as text or data. therefore “dedication to direction” can be classified as a “combination” knowledge element in plane 3. ikvc example 2. “commitment to generativity” in organisations is another knowledge element that can contribute to the knowledge enabler, the “instil knowledge vision cycle”. here the organisational vision is shared among employees who play a major role in helping the organisation to be successful (hodgkinson 2002.). this approach to determining the knowledge vision results in new thinking, new ideas, phrasing and actions from the people (von krogh & roos, 1997). consequently, the vision is created from human knowledge, which is difficult to formalise into codes or rules. generativity in organisation can be accomplished by stimulating the employees to consider how information is conceptualised in the organisational context. a knowledge creation and sharing process among them leads to diffuse knowledge that is the property of humans, and not necessarily authenticated. the conceptualisation of this form of knowledge is a changeable process, depending on source and context. knowledge created from documents, the internet or any highly diffused resource is transferred to teams or groups and then it can be converted to individual human knowledge. therefore, “commitment to generativity in organisation” can be classified as a “socialisation” knowledge element in plane 1. these two examples from the same knowledge enabling cycle illustrate the diversity that exists in the range of candidates for the set of knowledge elements. determining a comprehensive, yet workable, set of knowledge elements is the challenge of this research conclusion the k-space model explains the concepts of nonaka’s (1995) four knowledge creation processes and the five knowledge enablers of von krogh et al (2000 p8). both concepts are widely accepted as a valid basis for knowledge management theory. the k-space model is providing us with a formal framework for locating and giving a utility value to knowledge elements with a dynamic, contextual environment. these knowledge elements or manager activities are supported by human and technology tools that make knowledge management an integral part of the work process. ajis vol. 10 no. 1 september 2002 97 the next phase of the research will be to survey managers in organisations from different industries to both confirm and refine our set of representation knowledge elements and to place them on the k-space. the survey instrument is currently being design to cover all areas of the k-space. a triangulation method will synthesize results from three different complementary questionnaires for business managers, it managers and employees in the same organisations. this analysis will provide us with a balanced set of knowledge elements in the four by five (20) categories. furthermore, managers and it managers will be interviewed to see what are the most common tools they used in their company to manage innovation and knowledge and what they think the processes of innovation and knowledge management mean for them. the count of knowledge elements (ke) will be used as a measure of the knowledge assets of the company. a measure of innovation, as mentioned above, will also be determined and used to calibrate the measure of knowledge assets in each company. we anticipate that these measures will vary with industry. once the measure is calibrated it can be used in businesses as a measure of their overall knowledge assets. it will also be used to determine the extent to which these assets are balanced across the four modes and five knowledge enablers and to identify any knowledge gaps. the latter will be measured as (abs(total number of ke in a category – number of ke they already have in that category)). organisations can identify what tools can be used to support the identified knowledge elements gaps in the organisation. some companies have a variety of knowledge management tools in their organisation such as email, the internet, intranets and libraries but not all adequately support a balanced view of knowledge management process. this identification process can be done in any organisation and would be particularly useful in smes where the process of knowledge management is often not well understood. references ashford j. (1997) an information-space model for the development and application of computer–based tools in information creation and dissemination, journal of documentation 53/ 4 baster g. prabhudev k. jude s. e. (2001) business components case study of bankers trust australia limited, communications of the acm 44/5 92. bhatt g. d (2000) ;organizing knowledge in the knowledge development cycle, journal of knowledge management, 4/1 15-26. boisot m. (1998) knowledge assets, oxford press. bollinger a. s. and smith r.d. (2001) managing organisational knowledge as a strategic asset, journal of knowledge management 5/1 8-18. brakensiek j.c. (2002) knowledge management for ehs professionals occupational health and safety; waco; jan 2002; byrne r. (2001) employees: capital or commodity? the learning organisation 8/1 callioni p. (2002) keynote address, km standards challenge, australian standards international. carneir a. (2000) how does knowledge management influence innovation and competitiveness? journal of knowledge management, 4/2 2000 pp. 87-98 choo c.w. (1998) the knowing organisation, oxford university press, new york. coffman p. (2000) networking know-how: integral knowledge management (km) and the energy multinational; oil & gas journal, tulsa, 4-7. drejer a. (2002) situation for innovation management: towards a contingency model, european journal of innovation management 5/1 4-17. cook p (1999) i heard it through the grapevine: making knowledge management work by learning to share knowledge skills and experience, industrial and commercial training 31/3. earl m. (2001) knowledge management strategies toward a taxonomy, journal of management information system 18/1 215-233. eschenfelder k. heckman r. and sawyer s. (1998) the distribution of computing the knowledge markets of distributed technical support specialists, information technology & people 11/2. haldin-herrgard t. (2000) difficulties in diffusion of tacit knowledge in organizations journal of intellectual capital 1/4 hasan h. (2001) knowledge creation in organisations: the role of computer-based kms; european conference on knowledge management 2001 hildreth p. kimble c. and wright p. (2000) communities of practice in the distributed international environment, journal of knowledge management 4/1 hodkinson m. (2002) a shared strategic vision: dream or reality, the learning organisation: an international journal, 9/2. hoopes d. g., postrel s. (1999) shared knowledge, glitches, and product development performance, strategic management journal, 20/ 9 837-865. ajis vol. 10 no. 1 september 2002 98 ives b. (1994) probing the productivity paradox, mis quarterly 18 21-24 kakabads n.k., kouzmin a. (2001) from tacit knowledge to knowledge management: leveraging invisible assets, leonard d., sensiper, s. (1998) the role of tacit knowledge in group innovation, california management review, 40/3 112-132. lynn g. (2000) knowledge management in new product teams: practices and outcomes, ieee transactions on engineering management 47/ 2, pg. 22 marwick a.d. (2001) knowledge management technology, ibm system journal 40/ 4 mclure wasko m. and faraj s. (2000) “it is what one does”: why people participate and help other in electronic communities of practice, journal of strategic information systems, 9 155-173. merlyn p.r., välikangas l (1998) from information technology to knowledge technology: taking the user into consideration, journal of knowledge management 2/2 mohammed s., dumville b.c. (2001) team mental models in a team knowledge framework: expanding theory and measurement across disciplinary boundaries, journal of organizational behavior 22/2 89-106 nonaka i. (1995) the knowledge–creating company; new york, oxford university press. nonaka i. and takeuchi h. (1995) the knowledge creating company: how japanese companies create the dynamics of innovation, oxford university press, new york. ohr s. (2000) an overview :packaging issues-packaging plays key role in portable wireless, electronic engineering times p83 sircar s., turnbow j. l. and bordoloi b. (2000) a framework for assessing the relationship between information technology investment and firm performance, journal of management information systems, spring 2000 swan w. newell s. and robertson m. (2000) the diffusion, design and social shaping of production management information systems in europe, information technology and people 13/1 swap w., leonard d., shields m., abrams l. (2001) using mentoring and storytelling to transfer knowledge in the workplace journal of management information systems; armonk; summer 2001. tan j. (2000) managing knowledge: how to do it--a practical case study the british journal of administrative management, 19; 12-14. tyson s. (1999) how hr knowledge contributes to organizational performance, human resource management journal, 9/3, 42-52 von g. kazuo i. and nonaka i (2000) enabling knowledge creation wiig k. m. (1993) knowledge management foundations: thinking about thinking – how people and organizations create, represent, and use knowledge, schema press, arlington, tx, usa, wyman b.g., randel j.m. (1998) the relation of knowledge organization to performance of a complex cognitive task, applied cognitive psychology 12/ 3 251-264 yli-renko h., autio e., harry j., sapienza h.j. (2001) social capital, knowledge acquisition, and knowledge exploitation in young technology-based firms; strategic management journal, 22/ 6-7 587-613. microsoft word ajis12-1final repaired.doc ajis vol. 12, no. 1 september 2004 75 the significance of electronic commerce to firms’ operations in relation to business location: an empirical investigation uchenna c. eze dept. of economic geography nanyang technological university, singapore uc_chinwe@hotmail.com a. lee gilbert imarc, nanyang business school nanyang technological university, singapore algilbert@ntu.edu.sg abstract globalization of production and increasing competition spurs greater business use of innovative information systems. as globalization extends its reach over cities and regions, the positions of those places within the emerging global paradigms of regional economies is changing. only those regions and cities that can mobilize assets for local advantage would succeed. this research examines the implications of location for electronic commerce (ec) role in firm operations through the lens of managerial perceptions of ec systems, ec activities, agglomeration economies, firm-specific features and outputs relative to industry. the input-based view and industrial development frameworks provide the theoretical underpinning for this research. fully completed instruments from 106 firms in singapore and lagos financial services sector, respectively, are the basis of our analysis. our findings reveal varying results between industries across the two cities, supporting our propositions. firms with well-configured ec systems are more likely to experience efficiency in ec activities and outputs, given conducive operational conditions. however, firm-specific features were not linked to output, a finding inconsistent with prior studies. finally, ec business models that focus on operational efficiency strongly complement the historically relevant location variable in industrial operations. these findings provide basis for recommendations to policymakers, practitioners, and researchers. key words and phrases: electronic commerce, business location, information systems resource-based view, singapore, lagos introduction the relentless spread of economic globalization rests on increasing interdependence and interconnectedness among markets, policies, technology diffusion, and production, information, and distribution activities. our globe “shrinks” with each increase in the capacity and speed of physical and virtual communications, challenging our models of economic geography and conventional business structures. the integration of the world wide web (w3) with legacy technology platforms such as electronic data interchange (edi) affords firms the opportunity to revolutionize and restructure their business processes, reach new marketplaces and collaborate with other firms (zwass, 2003). as globalization extends its reach over communities, cities and regions, plummer and taylor (2000) note that, the positioning of those places within the emerging global paradigm of regional economies shifts. the conventional wisdom is that only those industries, cities and regions that can mobilize assets for local advantage will succeed (amin and thrift, 1997, p 155). this model prescribes that local firms must achieve internationally competitive productivity levels to be effective (plummer and taylor, 2000). yet, economic globalization processes may be locally determined and place-specific as well as global (plummer and taylor, 2001; porter, 2000). we apply the resource-based view and porter’s (2000, 1990) frameworks of industrial competitiveness to explore the influence of technology diffusion (in a specific locality) on firm performance. porter’s study of industrial structure and globalization predates the emergence of electronic commerce (ec). it is thus unsurprising that his model does not identify the type of industries that may benefit from the global diffusion of access and use of ec, nor does it suggest whether location plays a role in the potential benefits derivable from the implementation of this recent phenomenon. nonetheless, his concepts represent a robust background and structural guide for our analysis. ajis vol. 12, no. 1 september 2004 76 this exploratory research examines the influence of ec on business operations in relation to specific business environments. specifically, we developed a conceptual framework to examine management perceptions on the interplay among six constructs, namely; input (ec systems), output, business environment, agglomeration economies, firm-specific features, and ec activity (see figure 1). against this backdrop, our paper focuses on ec as a tool that could help industries, cities and regions weather the growing global competitive intensity, and adapt to structural changes in the global economic and geographic landscapes. figure 1: interplay among the primary constructs in the model the evidence in this paper draws from field experiences with ec applications among firms operating in two divergent geographic contexts: the equatorial city-state of singapore and the african city of lagos. their deep geographic and economic divergences illuminate the underlying forces we seek to explore. the strategic position the two cities occupy as regional hubs for local and international financial operations influenced the choice of city. consequently, the results provide relevant policy, research, and practice implications for firms including perhaps future directions for governments with respect to ec framework developments necessary for contemporary competition. to this end, we explore two main questions: (1) how significant are the structural differences/similarities (in terms of input, output, business environment, ec activity, and firmspecific features) between financial industries in singapore and lagos? (2) what sort of relationships exist among input, output, ec activity and firm-specific features of firms in the contexts (city and industry) of this research? the next sections present the conceptual framework, methodology, data analysis and findings, followed by our conclusions. the context of ec in singapore and lagos singapore lies 137 km north of the equator and connects to malaysia by a causeway across the narrow straits of johor. with a small population of about, 4.2 million (comprising chinese 76.7%, malay 14%, indian 7.9%, and others 1.4%) concentrated in a compact island of about 647.5 square kilometres (sq km.) (land: 637.5 sq km, water: 10 sq km.), singapore is slightly more than 3.5 times the size of washington dc and enjoys significant economies of density such as universal communications access for its entirely urban population 4.2 million. the small local market will not sustain capital-intensive ec applications, so local ec initiatives depend on regional markets for expansion and development. the small domestic market also deters firms from adopting new ec applications not already proven in advanced countries such as united states, united kingdom, japan, and australia, out of fear that they may be locked in to technological standards that become obsolete if these larger countries pursue a different standard. as a small nation-state in southeast asia, singapore is recognized for the quality of its infrastructure supporting the regional operations of multinational firms and more recently as a regional centre for international ec operations (debreceny et al., 2002; gilbert et al., 1999). this robust ec input e-commerce activity business environment firm-specific features output agglomeration economies ajis vol. 12, no. 1 september 2004 77 infrastructure, together with the connected links to major networks in international cities the nation has, and the strengths as a trading hub make singapore a favourable site for regional ec ventures. singapore’s ec infrastructure comprises e-payment, e-security, e-fulfilment portals including new generation portals and ec application services. the revenue generated by firms from ec infrastructure and supporting services is growing including revenue generated through ec transactions (tham, 2002). the singapore government actively boosts the rapid development of ec, encouraging firms to adopt online business models. the government agenda hopes to move over 85% of its business dealings with firms island-wide to the web by the end of 2004. nigeria is located at the southwestern coast of west africa, consists of 36 states, and covers an area of 923,768 sq km. nigeria shares borders with benin, cameroon, chad and niger (see figure 3.6). its population, estimated at 126 million (comprising about 250 recognized groups, many divided into subgroups of considerable social, economic and political significance), is sparsely and densely distributed in rural areas and cities (land: 910,768 sq km, water: 13,000 sq km). nigeria is about 1426.5 times the size of singapore or slightly more than twice the size of california which makes it extremely difficult to provide universal communications access to its entire population. lagos, the specific site for this research, is the industrial and commercial capital of nigeria in the equatorial region of west africa regionally known as the growth pole and centre for local/international financial institutions and it headquarters (ujah, 1999). private sector, particularly financial, oil, and telecommunication firms drive ec development in lagos, unlike singapore. quasi-governmental agencies such as the national communication commission (ncc) and the nigeria internet group (nig) provide statutory support for the ec initiatives in lagos. the government of nigeria is gradually developing a solid ec infrastructure to enhance operational activities among business partners. over 75% of manufacturing and trading firms in lagos use electronic data interchange (edi) as their key operating platform. with the rapid diffusion of the internet, greater numbers of firms are shifting their operations from the proprietary networks of edi to the w3. as in singapore, revenue generated from ec support services grows at a geometric rate. ec value was usd8.3 billion in 2002 up 22% from usd6.8 billion in 2001. projections for the 2005 ec value stand at usd15 billion, and with the liberalization of the telecommunication sector and the introduction of gsm services, experts predict a boom for ec activities in lagos. (ujah, 2001) conceptual framework and hypotheses the resourceor input-based view of the firm responded to the limitations of the structure-conductperformance paradigm of the industrial organization (io) view of the firm (bain, 1959; grant, 1991a). early input-based theories found the io view – that a firm’s external environment determines its success – to be unrealistic, and turned to the seminal work of penrose (1959) for inspiration. to counter the io view, prahalad and hamel (1990) developed an input-based theory around the internal competencies of a firm. in their version of input-based theory, competitive advantage lies within a firm, rooted in assets that are valuable and inimitable. a firm’s competencies and its managerial abilities to marshal these assets to produce superior performance are the source of its competitive advantage. in the drive to add depth and breadth to this internal view, these theorists noted but left rather vague the role of linkages to a firm’s external environment. barney (1991) addressed this issue by pinpointing the conditions under which a firm’s inputs become valuable by bringing the external environment into the input-based picture. in developing the notion of external environment, he noted that external factors alone do not generate value. however, by nurturing internal competencies and applying them appropriately to the external environment, a firm can create new value. thus, for a firm’s resource to become valuable, it must allow the firm to exploit opportunities and neutralize threats in the firm’s environment (barney, 1991). conner (1991) made the link between input-based theory, the competitive environment, and firm capabilities more explicit in her comparison of the input-based and io views. she observed that all three recognize the external constraints of demand, competition, and public policy. for conner, the task for input-based theorists is to discern the appropriate rent-generating inputs given external (e.g., demand, public policy, and competitor action) and internal (e.g., history, resource endowments, and corporate culture) constraints. thus, in its current state, the input-based view ajis vol. 12, no. 1 september 2004 78 addresses the fit between what firms have the ability to do and what they have the opportunity to do. the interplay with market forces determines the value of resources. a resource that is valuable in a particular industry or at a particular time might fail to have the same value in a different industry or chronological context. in this respect, ec deployment in a firm represents a resource while ec configuration with other resources in the firm is a feature, differentiating the ec capability and the operational ec strategy existing in a firm. when a firm competes in any industry, it performs discrete but interconnected value-creating activities such as operating a sales force, or delivering products, and these activities have points of connection with the activities of suppliers and customers (rowson, 1998). we consider ec as a complementary tool firms use to perform these activities more efficiently as opposed to the view characterizing ec as a cannibal that destroys all conventional business models (porter, 2001). porter’s (1990) view on industrial and national development provides additional theoretical underpinning for this paper. the conceptual model, with deep roots in the resource-based model, integrates much of earlier works in it and firm, and incorporates the strategic forces unleashed by the rapid diffusion of the internet. he argued that two sets of determinants affect industrial competitive advantage and, despite the rushing tide of strong globalization, the importance of these factors in determining national competitive and industry capabilities advantage has not diminished, but in fact has become more definite. among the set of fundamental determinants are factors of production, including human, physical, knowledge, capital resources, and infrastructure. table 1 depicts the variables used to build the conceptual framework and their respective sources. table 1: variables, descriptions and sources variables descriptions sources input: ec system functionality the richness and navigation functions of the web-page in terms of interactivity/interface, catalog/commercial application gebauer & scharl, 1999; zhuang, 2000; powell and dent-micallef, 1997 expertise ec application availability of requisite skills for managing ec systems and operations zhuang, 2000; gatian et al., 1994 business environment: perceived stability extent of predictability of political, economic and business situations in the location zaheer & zaheer, 1997; sundell, 1999 technology change extent of change in firm and external network systems zaheer & zaheer, 1997 output: responsiveness the quickness with which firms attend to customers zaheer & zaheer, 1997 customer retention expectation of decrease/increase in the share of customers in the marketplace gatian, et al., 1995 overall market growth general improvement in service provision to customers sundell, 1999 ec activity: business activities performed via ec systems zhuang, 2000; gebauer & scharl, 1999; literature firm specific features: annual revenue annual revenue of the firm for the fiscal year 2000 eze, 2004 ec expenditure expenditure on ec and related information systems eze, 2004 period of ec use period of ec use in the firm gatian et al., 1995 agglomeration economies sophistication in local assets which enables location of regional offices of firms and thus the interdependence of firms based on shared value systems plummer & taylor, 2001 sheppard, 2000; porter, 1998, 1990 no single study motivated our conceptual framework. previous researches have tested individual paths of the framework we propose. nonetheless, we place the constructs in a more comprehensive and integrated manner, tracking the course of the deployment and practice of ec in contexts in which it is not studied enough. figure 2 is a schematic representation of the conceptual framework in this research. there are three main parts in the framework and they are core drivers, representing the attributes of the firm in the industry, economic structure and ec influences, and firm-level outcome. ajis vol. 12, no. 1 september 2004 79 the core drivers, which include input, firm-specific features and business environment, are the independent variables while output denotes the dependent variable. in addition, the ec activity, the economic structure and agglomeration economies are the moderating variables. the firm attributes are core drivers because they influence the general ec context in the firm and industry, while input is defined by ec functionality and expertise and is hypothesized to directly impact output. ec activities comprise the routine and core activities. agglomeration economies are benefits that accrue from geographic proximate group of interconnected firms and associated institutions. any firm is only one part of a complex chain of production – a series of economic activities held together by linkages among firms (dicken and lloyd, 1990). agglomeration economies derive from shared value systems, location of firms’ production/distribution processes in strategic cities, and interdependence of firms for production activities, and we operationalized this variable with two items: the level of linkages of production and marketing activities among firms, and the degree of interdependence of firms in the performance of economic activities. figure 2 reveals the elaborate relationships that this research explores. the directions of arrows illustrate the influence of one variable on the other. propositions 2, 3, and 4 relate directly to the conceptual framework. in addition, we developed proposition 1 to understand the perceived contextual structures of the constructs and thereby addressing any differences/similarities between the industries in the study areas, which may have operational implications for the relationships among the variables. proposition 1: there would be significant positive structural (in terms of input, output, ec activity, firm-specific feature and business environment) differences between singapore and lagos financial industries, considering the economic and political paths of the two places. prior studies (sundell, 1999; zaheer and zaheer, 1997) provide extensive evidence that the influence of ec and information systems on output is mixed. nevertheless, the conceptual framework illustrates that firms deploying well-configured ec strategy would experience improvements in output in terms of responsiveness, customer retention and overall growth in service provision (bird and lerhman, 1993).the conceptual framework depicts that the firm output relates positively to conducive business environment factors in which the firm operates. prior studies indicate that a business environment characterized as stable and enabling tend to stimulate and encourage improved firm operations (afuah and tucci, 2001; ansoff and sullivan, 1993; kettelhut, 1992). also, previous research on the relationship between uncertainty and output suggests that the two variables are negatively correlated (downey and cool, 1982; waddock and isabella, 1989). in addition, firmspecific features such as annual revenue base, annual expenditure on ec system and the duration of ec usage in the firm would influence a firm’s output. there are extensive prior studies linking the two constructs (rai, 1997; zhuang, 2000). this research examines the possibility that adequate firmspecific features positively influence output in the financial industries in the two cities. the framework also depicts that business activities performed via ec systems may influence the output of a firm. existing research provides specific prior conclusions on the potential impact of ec systems on business activities, and consequently, specific success stories of ec to firms’ operations noted by zhuang (2000), and hamel and sampler (1998) served as the bases for this proposition. this paper, therefore, examines the aforementioned using data from two substantive domains singapore and lagos. hence the following proposition: ajis vol. 12, no. 1 september 2004 80 figure 2: the detailed conceptual framework proposition 2: efficiency in ec activity, firm-specific features, input, and business environment factors would likely have stronger positive influence on output among singapore firms compared to lagos firms, considering the economic and political paths of the two places. the conceptual framework depicts that enabling business environment in which a firm operates would positively influence the structure of input (ec systems and expertise) deployment. this conception contends that the nurturing and development of ec systems in the firm would be high if business environment factors are stable and enabling. prior research reveals extensive evidence that conducive structure in the business environment would influence positively the deployment of information systems. this paper attempts to provide an expanded understanding by examining the relationships of the two variables in the industries and cities. the conceptual framework also illustrates that firm-specific features such as annual revenue, annual expenditure on ec system and the duration of ec usage in the firm would influence a firm’s input. prior studies indicate linkages between the two constructs (bird and lerhman, 1993; trippi and salameh, 1989). we, therefore, investigate whether this proposition holds for the financial industries in singapore and lagos. the above descriptions set up the next proposition: core drivers economic structure and ec influences firm-level outcome shared value system production/marketing via regional offices routine activity core activity input ec functionality ec expertise annual revenue ec expenditure stability technology change responsiveness customer retention overall growth in service provision output business environment firm-specific feature ec activity ec use attributes of the firm in the industry economic agglomeration economies factor conditions support industries markets ajis vol. 12, no. 1 september 2004 81 proposition 3: efficiency in firm-specific features and business environment factors would likely have stronger positive influence on input among singapore firms compared to lagos firms, considering the economic and political paths of the two places. the conceptual framework also depicts that stability in the business environment would directly relate to efficiency in business activities and operations that a firm performs using ec. specific literature that demonstrates the relationships between these two variables is scanty. however, firm success stories and experiences from business and economic geography professionals (gebauer and scharl, 1999; hepworth, 1990; leinbach and brunn, 2001; porter, 1990) provide extensive illustrations and predictions regarding the link between ec activities and business environment factors. the conceptual framework also illustrates that firm-specific features would influence ec activity positively. the literature provides extensive review on prior studies linking the two constructs (gebauer and scharl, 1999; rai, 1997). we investigate this proposition using data from firms operating in financial industries in singapore and lagos. also, the framework depicts that the level of sophistication in input factors a firm deploys would have positive implications on the ec activities the firm performs. prior studies reveal that information systems deployment would positively influence activities an enterprise performs (gebauer and scharl, 1999; zhuang, 2000). this research provides expanded contextual insights on the relationship between ec systems and business activities. hence the following proposition: proposition 4: efficiency in input, firm-specific features, and business environment factors would likely have stronger positive influence on ec activity among singapore firms compared to lagos firms, considering the economic and political paths of the two places. as we noted earlier, business environmental conditions are major cornerstones in strategic international management, industrial geography and spatial planning. they provide key materials upon which firms conduct, organize their operations, and make concrete decisions on competition. the geographically proximate cluster of independent and informally linked firms and institutions represents a robust organizational form in the continuum between markets and hierarchies (plummer and taylor, 2000; sheppard, 2000). the mere presence of firms, suppliers, and institutions in a location creates the potential for economic value, but it does not necessarily ensure the realization of this potential. social glue binds clusters together, contributing to the value creation process. many of the competitive advantages of clusters depend on the free flow of information, the discovery of value-adding transactions, the willingness to align agendas and to work across firms, and strong motivations for improvement. hence, the following proposition: proposition 5: the lack of conducive and sophisticated environmental conditions in a locality such as expertise, capital and infrastructure would likely have a negative effect on agglomeration economies. methodology survey instrument we took several steps to ensure data validity and reliability including pre-tested the instrument with two academics and two business executives selected from the industry. the instrument was then revised for any potentially confusing items. before the administration of the final survey instruments, the authors called a random subset of 10 respondents for pilot testing to verify if any previously identified concerns with the instrument persisted. six instruments were returned for a response rate of 60%. suggestions made during the pre-test for improving the instrument and thus its content validity were addressed before the primary mailings. further, we used previously validated measurements items (gebauer and scharl, 1999; sundell, 1999; zhu, 2002) to ensure the validity of the measures; including multiple-item measures for some of the constructs to enhance content coverage. in addition, we computed cronbach’s alpha scores for the overall internal reliability and ajis vol. 12, no. 1 september 2004 82 consistency on the likert scale questions in the instrument pertaining to input, business environment, ec activities, agglomeration economies, firm-specific features, and output, resulting in alpha scores of 0.71 or higher, which indicates strong internal consistency. sample and participation we selected a sample of 289 (160 from singapore and 129 from lagos) for this research. the participants were mainly large firms randomly selected from the financial service industry, selected based on the knowledge that firms likely to be successful at obtaining advantages from the use of information systems were those that exhibit evidence of growth (lee and adams, 1990). the population frame for the financial firms in lagos were the list of registered businesses in the lagos stock exchange and the industry classification provided by statistics office of nigeria. the directory of financial institutions in singapore provided by monetary authority of singapore (mas) served as the population frame for the firms in singapore. the instruments were then mailed to cios of the 160 and 129 firms in singapore and lagos, respectively. the questions/statements were based, mainly, on a five-point (1-5) likert point scale. of the 289 firms surveyed, 106 useable responses were collected (i.e., a response rate of 36.67%). there were 58 valid responses from singapore, (i.e., a response rate of 36.25%), while there were 48 useable responses from lagos (i.e., a response rate of 37.21%). our total response rate of 36.67% compares favourably with similar firms mail surveys: e.g., mcdougall et al. (1994) had 11% response rate in a study of new technology-based firms; chandler and hanks (1994) had a 19% response rate in a study of manufacturing firms. it is also important to note that although individuals representing the firms provided responses to the instrument, we based the unit of analysis on the firm. in addition, we assessed the potential for non-response bias by comparing early versus late respondents, as advocated by uma (1992). measures and their psychometric properties we refined the groupings of the instrument items into six measures by a series of confirmatory factor analyses using a structural equations approach (see table 2). the motivation for the research design was to develop measures with desirable psychometric properties and to eliminate any items not measuring the constructs well (carmines and mclver, 1981). ajis vol. 12, no. 1 september 2004 83 table 2: standardized coefficients, estimates, average shared variance and composite reliability for the measurement model for singapore and lagos samples construct/indicator standardized path coefficients estimates average shared variance (%) composite reliability input: 67 (63) 0.81 (0.78) ec functionality 0.921 (0.671) 7.89** (5.98**) ec expertise 0.894 (0.594) 8.19** (6.33***) output: 72 (75) 0.87 (0.91) responsiveness 0.876 (0.496) 7.90** (7.46***) customer retention 0.795 (0.645) 6.89** (5.34**) overall growth 0.804 (0.657) 7.19** (4.87**) business environment: 83 (65) 0.91 (0.88) perceived stability 0.717 (0.733) 5.79** (6.22***) technology change 0.871 (0.481) 7.11** (5.39**) firm-specific features: 68 (54) 0.82 (0.87 annual revenue 0.891 (0.675) 6.35** (5.18**) ec expenditure 0.900 (0.551) 7.39** (5.33**) ec usage 0.880 (0.630) 6.73** (4.99**) ec activity: 75 (61) 0.86 (0.87) routine activity 0.650 (0.601) 5.23** (3.77**) core activity 0.710 (0.418) 7.01** (3.26**) agglomeration economies: 65 (59) 0.74 (0.78) production via overseas office 0.672 (0.672) 5.29** (5.73**) shared value system 0.613 (0.613) 3.95* (4.35**) values in parentheses are for lagos, **p<0.05, ***p<0.01 we performed each confirmatory factor analysis using calis (covariance analysis of linear structural equations) using maximum likelihood estimation. we also evaluated the fit of the confirmatory factor analysis by subjecting the variables to several tests. the goodness of fit index averaged 0.90, and when adjusted for the degrees of freedom, averaged 0.85. the root mean square residual was 0.05 on average, and the significant chi-square statistic had a value of 158.0 with 80 degrees of freedom. the corresponding normed chi-square value was 1.98. bentler’s comparative fit index was 0.96 while bentler and bonett’s (1980) normed and nonnormed and bollen’s normed and nonnormed indexes ranged from 0.89 to 0.96. generally, the fit of the confirmatory factor analysis to the data was satisfactory (anderson and gerbing, 1988). using the results of the final confirmatory factor analysis, we examined the psychometric properties of the measures, which indicate that the measures exhibit satisfactory average shared variance. table 2 also displays the composites reliability coefficients for the measures and demonstrate satisfactory composite reliability. we dew several conclusions from the discriminant analysis regarding the measures’ psychometric properties one of which is that all the measures displayed adequate convergence as measured by these values since the average percentage of shared variance for each measure is larger than 50%. in addition, the measures have satisfactory reliability, since all the reliability coefficients are 0.64 or higher for singapore and 0.57 or higher for lagos. these results satisfy convergent and discriminant validities (rainer and harrison, 1993). descriptive analysis table 3 reveals that 62.1% and 64.4% of the respondents from singapore and lagos, respectively, have worked with the firm for more than 5 years and thus can be considered well-informed on the firm/industry and associated issues this paper addresses (uma, 1992). also, the table depicts that 58.6% and 56.3% of the firms in singapore and lagos financial industry, respectively, invest 6% and over annually on ec systems, indicating relatively high percentages. an approximately equal percentage revealed using ec for more than 8 years, indicating a relatively high level of experience. ajis vol. 12, no. 1 september 2004 84 table 3: profile of respondents singapore lagos item value average percent of respondents average percent of respondents number of years with firm >5 years 62.1% 64.4% number of employees >500 58.3% 56.0% annual revenue >s$500m 62.1% 49% ec usage period ≥8 years 46.6% 48% annual expenditure on ec ≥6% 58.6% 56.3% number of cases: singapore = 58; lagos = 48 as table 4 reveals, perceptions of respondents in singapore on the issues were, generally positive and stronger compared with perceptions among respondents in lagos. however, while the percentage of respondents’ who perceive government’s policies on the specific issues to be encouraging were mostly lower among lagos firms, competition policy and r&d subsidies emerged short of expectation among firms in singapore considering the robust information infrastructure and the high awareness programmes associated with the city. table 4: regulatory and policy issues in singapore and lagos factors no effect discouraged encouraged telecom infrastructure: communication cost 10.3(20.8) 24.1(35.2) 65.5(44.0) network reliability 0(20.8) 20.7(20.6) 79.3(57.6) competition between operators 6.9(28.0) 25.9(35.2) 67.2(36.8) interoperability of networks 1.7(24.0) 17.2(39.2) 81.0(36.8) regulatory factors: transaction security 0(21.2) 25.9(20.0) 74.1(58.8) certification of payment 0(18.6) 0(20.6) 91.4(60.8) regulation of e-transactions 8.6(10.8) 3.4(36.0) 87.9(53.2) protection of intellectual property 15.5(34.8) 15.5(38.0) 69.0(27.2) policy issues r&d subsidies 1.7(27.6) 50.0(47.6) 48.3(24.8) awareness programs 0(34.0) 20.7(28.4) 79.3(37.6) competition policy 13.8(19.2) 32.8(42.4) 53.4(38.4) electronic government models 0(27.6) 17.2(34.8) 82.8(37.6) percentages in parenthesis are for lagos the nigeria communication commission responsible for injecting competition in the telecommunication sector respond to these concerns by adopting policies aimed at reducing the adverse impact of the monopolistic status of nigeria telecommunication (nitel) by issuing operating licenses to local and foreign investors willing and capable to increase the intensity of competition among the information technology operators. regarding singapore, ida has been able to develop relatively solid competitive conditions in the sector, which would yield dividends in the near future. table 5: means for variables by industry/city singapore financial industry mean lagos financial industry mean business environment perceived stability technology change 3.96 4.06 3.87 business environment perceived stability technology change 2.90 2.66 3.20 input ec functionality ec expertise 2.97 3.01 2.92 input ec functionality ec expertise 2.92 2.89 2.96 output responsiveness 3.72 3.74 output responsiveness 3.56 3.31 ajis vol. 12, no. 1 september 2004 85 number of customers overall growth 3.35 4.09 number of customers overall growth 3.27 4.17 ec activity routine activities core activities 3.64 4.28 2.98 ec activity routine activities core activities 3.52 4.18 2.85 agglomeration economies prod/marketing via overseas shared value system 3.92 3.74 4.10 agglomeration economies prod/marketing via overseas shared value system 3.10 2.80 3.40 table 5 depicts the composite mean values for five variables investigated in this study – business environment, ec activity, agglomeration economies, input, and output. perspectives on singapore and lagos industries reveal some similarities, especially, with regard to input, and differences between composite means for business environment, agglomeration economies, output and ec activity (see table 5). these results highlight the varying contextual features and underlying characteristics of the variables within singapore and lagos financial industries, particularly, on environmental factors. input factors for the industries in singapore and lagos emerged with relatively less disparity as in overall growth and core activities. findings this section presents correlation analysis to establish any relationship among the dimensions of the constructs including any underlying strength and orientation among the dimensions. it also provides evidence crucial for determining the applicability of multiple regression analysis. table 6 provides codes and descriptions of the factors we used in the preceding analysis. the results in table 7 reveal that firm-specific features, input, output, agglomeration economies, and ec activity, generally correlate positively. aexpec correlates with techang, which implies that a firm would not favour investment in ec systems if it were not receptive to changes in information systems. in addition, the results in table 8 reveal that the variable dimensions, generally correlate weakly. for instance, respon correlates weakly with ecact, which implies that a firm’s responsiveness to customers’ and suppliers’ requests and orders relate marginally to the extent and number of activities performed using ec. although there were significant positive relationships among the variables in both industries in singapore and lagos, the correlation coefficients for industries in lagos were generally weak compared to those for singapore industries. in addition, results of singapore sample reveal stronger relationships between the factors compared to lagos study. table 9 indicates that the structural difference between singapore and lagos samples is significant at p<0.05 with respect to output (t = 4.434; p-value = 0.000), ec activity (t = 2.321; p-value = 0.027), and business environment (t = 11.428; p-value = 0.000), and to a lesser degree, input. on the other hand, there was no significant difference between the two industries on firm-specific features at p<0.05 (t = -1.617; p-value = 0.109), which supports the descriptive result in table 3. the results in table 9 indicate structural patterns and contexts, which are not only determined by firm and industry characteristics, but also by the specific distinct backgrounds and trajectories of singapore and lagos economic and social development processes. ajis vol. 12, no. 1 september 2004 86 table 6: codes & descriptions of variables codes descriptions isinfin input for singapore financial industry osinfin output for singapore financial industry esinfin environment for singapore financial industry asinfin ec activity for singapore financial industry fsinfin firm-specific features for singapore financial industry ilagfin input for lagos financial industry olagfin output for lagos financial industry elagfin environment for lagos financial industry alagfin ec activity for lagos financial industry flagfin firm-specific features for lagos financial industry size annual revenue (firm-specific) ectime years of ec usage in the firm (firm-specific) aexpec annual expenditure on ec systems (firm-specific) ecact activities performed via ec (ec activity) perstab perceived stability in business environment (city-specific) techang technology change (city-specific) cusret customer retention (output) respon responsiveness (output) overal overall growth in service provision (output) ecsfun ec systems functionality (input) ecexp ec expertise in the firm (input) table 7: correlation matrix for firm-specific features, business environment, input, ec activities, agglomeration economies, and output sinfin variables 1 2 3 4 5 6 7 8 9 10 11 12 1. size 2. ectime 0.756 3. aexpec 0.835 0.722 4. ecact 0.669 0.565 0.659 5. perstab n. s 0.456 0.697 0.697 6. techang n. s 0.261 0.598 0.766 0.889 7. cusret n. s 0.863 0.491 0.783 0.641 0.656 8. respon n. s 0.769 0.754 0.811 0.738 0.578 0.714 9. overal n. s 0.784 0.689 0.644 0.772 0.717 0.622 0.599 10. ecsfun 0.761 0.822 0.887 0.564 0.692 0.687 0.659 0.676 0.748 11. ecexp 0.896 0.799 0.703 0.630 0.745 0.705 0.504 0.729 0.561 0.649 12. prore 0.723 0.592 0.601 0.432 0.679 0.665 n. s 0.432 0.346 0.395 0.276 13. values 0.632 0.610 0.545 0.534 0.712 0.541 n. s 0.329 0.290 0.223 0.439 0.657 all correlations are significant at p< 0.05, unless otherwise noted as not significant. number of cases = 58 the results of bivariate step-wise multiple regression analysis indicate that the perceived relationships among output, business environment, input and ec activity among singapore and lagos firms are mix (see table 10). eight of the nine path coefficients for propositions 2, 3 and 4 in singapore and lagos industries were significantly different from zero at 0.05% significance level (see table 10). all the outcomes had reasonably large standardized coefficients with the expected signs except for propositions 3b, 4a and 4c that indicate weak relationships. the results from the two ajis vol. 12, no. 1 september 2004 87 locations did not support the proposed relationship depicting output as a function of firm-specific features. in addition, proposition 5 emerged significant for singapore study at p<0.05 (r2 = 0.86; pvalue = 0.000) and lagos (r2 = 0.53; p-value = 0.013), respectively). the result for lagos emerged relatively weak, which is not surprising. these results indicate that firms’ ec systems and related resources have the potential to enhance the agglomeration ability that industries in a city can harness, which may improve the synergistic relationships among firms. by implication, that singapore’s strong economic structure and decades of government’s commitment towards promoting the island as a regional hub for financial and ict operations, markedly differentiates it from lagos. table 8: correlation matrix for firm-specific features, input, business environment, ec activities, agglomeration economies, and output – lagfin variable 1 2 3 4 5 6 7 8 9 10 11 12 1. size 2. ectime 0.343 3. aexpec 0.676 0.211 4. ecact 0.321 0.197 0.321 5. perstab n. s 0.103 0.429 0.345 6.techang n. s n. s 0.328 0.467 0.324 7. cusret n. s 0.511 0.467 0.625 0.425 0.329 8. respon n. s 0.612 0.679 0.522 0.326 0.317 0.211 9. overal n. s 0.421 0.662 0.456 0.446 0.226 0.450 0.345 10. ecsfun 0.451 0.674 0.712 0.345 0.302 0.428 0.477 0.476 0.435 11. ecexp 0.567 0.347 0.511 0.424 0.221 0.529 0.502 0.395 0.341 0.512 12. prore 0.342 0.495 0.267 0.324 0.435 0.532 n. s 0.432 0.435 0.300 0.387 13. values 0.453 0.542 0.475 0.322 0.511 0.397 n. s 0.546 0.346 0.492 0.401 0.702 all correlations are significant at p< 0.05, unless otherwise noted as not significant. number of cases = 48 table 9: test summary for proposition 1 proposition parts of proposition t p-value output for lag output for sin 4.434** 0.000 input for lag input for sin 2.442* 0.018 be for lag be for sin 11.428*** 0.000 fsf for lag fsf for sin -1.617 0.109 p1. there would be significant positive structural differences between singapore and lagos financial industries. eca for lag eca for sin 2.321* 0.027 lag= lagos; sin= singapore; fsf= firm-specific features; eca= ec activity; be= business environment these results are particularly significant because through the 1990s until to date, singapore has taken strong strategic steps to remain competitive in the regional and global marketplace by promoting the island as a centre for industrial operations (see earlier notes). nonetheless, it is insufficient for singapore to be a nodal point for the attraction of overseas capital, skills, knowledge and other resources, it is also crucial for singapore to expand and grow its regional market including africa and latin america. in addition to the strategies singapore government adopted to maintain the island’s competitiveness including the connected singapore initiative, this research highlights and advocates more aggressive deployment of ec technologies in firms. with the existing infrastructural capabilities in place, it [ec] would be a great asset to enhance and advance firms’ collaboration in the production and marketing of goods and services within the global marketplace framework. these results imply that the success of a firm and industry may depend on the economic processes in a particular societal context. hence, the playing field upon which places compete is uneven. cities, for example, have distinctive characteristics and histories, and situate differently within the larger ajis vol. 12, no. 1 september 2004 88 political economy based specifically on at least two dimensions, each of which tilts the playing field to favour some cities over others: enabling system and historical trajectories. first, firms found in places where the state favours capitalist approach, institutes policies that promote industrial competitiveness, facilitates well-trained and motivated workforce, and provides enabling ec and related infrastructure have greater possibilities to gain competitive advantages from the particulars of their systems (sheppard, 2000). secondly, each city occupies a unique geographical trajectory because of its historical role and location within the broader political and economic systems, a uniqueness that creates differences in the ability of individual cities to respond to economic and political restructuring. however, the total implications and directions of these results may be unclear. nonetheless, firm-specific features did not predict output in both study areas, although both variables correlate. the financial industry has a strong history and tradition of information systems implementation because of its task requirements. consequently, the industry would require concrete and strategic investment and usage of advanced information systems to maintain positive significant progress in operational and strategic efficiencies. ajis vol. 12, no. 1 september 2004 89 table 10: test summary for propositions 2-4 source r2 b, t-ratio p-value (a) std. coef. proposition 2a: output on input output contingent on input – sin 0.819 0.494 (15.893)*** 0.000 0.905 output contingent on input – lag 0.689 0.401 (16.494)*** 0.000 0.767 proposition 2b: output on business environment output contingent on be – sin 0.822 0.830 (16.092)*** 0.000 0.907 output contingent on be – lag 0.681 0.750 (16.204)*** 0.000 0.855 proposition 2c: output on ec activity output contingent on eca – sin 0.890 0.257 (21.332)*** 0.000 0.944 output contingent on eca – lag 0.785 0.245 (21.178)*** 0.000 0.881 proposition 2d: output on firm specific feature output contingent on fsf – sin 0.067 0.442 (1.023) 0.254 0.076 output contingent on fsf – lag 0.075 0.054 (1.296) 0.121 0.117 proposition 3a: input on business environment input contingent on be – sin 0.838 1.536 (17.047)*** 0.000 0.816 input contingent on be – lag 0.742 1.619 (18.787)*** 0.000 0.818 proposition 3b: input on firm specific feature input contingent on fsf – sin 0.501 1.356 (15.226)*** 0.000 0.510 input contingent on fsf – lag 0.642 1.459 (16.080)*** 0.000 0.472 proposition 4a: ec activity on business environment eca contingent on be – sin 0.852 3.096 (17.939)*** 0.000 0.923 eca contingent on be – lag 0.716 2.782 (17.613)*** 0.000 0.846 proposition 4b: ec activity on input eca contingent on input – sin 0.329 0.427 (2.702)* 0.019 0.410 eca contingent on input – lag 0.309 0.329 (2.309)* 0.020 0.297 proposition 4c: ec activity on firm specific feature eca contingent on fsf – sin 0.532 1.411 (11.101)*** 0.000 0.449 eca contingent on fsf – lag 0.385 1.209 (9.311)** 0.001 0.334 lag= lagos; sin= singapore; fsf= firm-specific feature; eca= ec activity; be= business environment discussion clearly, the patterns of relationships that emerge among the constructs for singapore and lagos firms demonstrate strong influence of the environment in which the firms operate evident from the strength of the relationships among the variables and the varying results between the two cities. in theory, location should no longer be a source of competitive advantage as open global markets, rapid transportation, and high-speed communications systems tend to equalize any differences. however, in practice, location and largely, regional imperatives remain central to national and global industrial competition. consequently, a city’s infrastructural capabilities in terms of high quality expertise, capital, sophisticated knowledge management tools and the ability of firm to integrate strategically these resources for potential benefits are key differentiating factors in contemporary and future global competition. it is pertinent to note, however, that technological factor is the key resource in this paper. the complementary nature of this factor, as zhuang (2000) observed, would be insufficient to explain all the possible outcomes in this comparison. overall, while firm resources are important determinants for operational activities, technological opportunities and local imperatives are equally significant. the structural differences between the two industries mirror the distinct trajectories of the cities and their influence on associated business performances. consistent with porter’s (1990) view, our findings demonstrate the importance of sophisticated factor conditions in a city, which enable accelerated development processes in the specific industry including the potential roles of ec. accordingly, although the ec phenomenon is global in character and certainly requires firms to implement effective international information infrastructure standards, the potential benefits of ec have a distinct geography as development processes, efficient ajis vol. 12, no. 1 september 2004 90 governance, information systems policies, and pervasive deployment of ec technologies differ across the globe. implications for research we examined empirically the paths among the constructs and received modest support, which could be useful resources for future research development in ec and firms. in addition, our finding is consistent with evolving academic models of technology-firm-environment framework expanded by zhu et al. (2002) in new substantive areas – singapore and lagos. in this regard, it extends the field of information systems, ec, and the geographic concentration of industries consistent with porter’s (1998) analysis. we are optimistic that as the subject area develops further particularly in theoretical and methodological approaches specific to ec research, future studies based on the procedures we used would receive greater empirical support. since ec is still evolving with just a few studies focusing on theory and framework building, our findings and design processes represent modest contributions to research, and may be useful in strengthening the theoretical and operational directions of future researches regarding the roles of ec in industrial developments of cities. although this research focuses on firms in the financial industries in lagos and singapore, our findings appear to have implications for financial firms operating in business environments, which share similar characterization as our sample. taking into account the effects of business tradition and the extent of ec deployment in firms, this research is replicable to financial industries in hong kong/johannesburg or accra/hong kong. the heightened notion of digital economy and its pervasive influence on the structure of industries makes a case for applicability in this research. implications for practice firms could, based on our findings, effectively configure their inputs for efficient business activities and functional associations among partners and customers. the results demonstrate significant positive relationships among some of the constructs suggesting that with adequate configurations of ec, business activities tend to improve given a favourable business environment. managers may use our procedure to assess the success of ec strategies in their respective firms by determining the strengths of the relationships among the various constructs. they can also assess their successes with each factor against corporate goals. further, managers may compare data from their firm against normalized data across related firms, which would enable them to compare the scores for their firms with those of the industry. in addition, this research provides an expanded understanding of zhuang’s (2000) resource-based view of the firm using ec applications as the key factors to address business operations, with specific reference to the cities and industries we investigated. this understanding would be useful to managers who may need to adapt the input-based view in making similar analysis, for instance, in identifying the strategic roles of ec for enhancing business operations and performance of firms in specific contexts. we are aware that correlation is not causation; yet, managers may still consider the quality of ec systems in their firms. perhaps, by examining the sub-dimensions of the constructs and the effects of ec under distinct conditions, managers may be encouraged to improve their ec strategies, which will probably translate into better overall industrial performance. implications for policy greater practical and effective government policies in the two cities, particularly lagos, would be important to make it easier and more affordable to firms in the private sector to deploy ec systems in their industries. policies that would encourage competition among internet service providers (isps) and related operators in the sector would consequently boost investment in the it sector, which may have spillover effects on industrial operations and overall economic performance. this, among other initiatives, would create attractive sites for regional business development, particularly small businesses, which could drive foreign direct investment. as globalization and competitive pressures force governments to rethink conventional modes of administration, there may be no other ajis vol. 12, no. 1 september 2004 91 innovative strategy for government agencies in lagos to achieve operational effectiveness but to employ adequately ec systems in key activities, particularly in the administration of public goods. agenda for future studies longitudinal investigations designed to answer critical questions on specific dynamics of the interplay of ec and the firms’ competitive functions in divergent business landscapes would be very useful to provide additional insights on the conceptual framework. in addition, this research relied primarily on perceptual data. future research should consider personal interviews as they might provide additional information on the variables. however, dess and robinson (1984) argue that subjective data tend to correlate strongly with objective data. a case study employing qualitative research approach would be useful for greater appreciation of the overall framework development, and new managerial insights on the nature of ec strategy. references afuah, a. & tucci, c.l. (2001) internet business models and strategies: text and cases, boston, ma: mcgraw-hill. amin, a. & thrift, n. (1997) globalization, socio-economics, and territoriality, in, r. lee, and j. wills (eds.), geographies of economies, london: arnold, pp 76-99. anderson, j.c. & gerbing, d.w. (1988) “structural equation modelling in practice: a review and recommended two-step approach”, psychological bulletin, vol 103, pp 411-423. ansoff, h.i. & sullivan, p.a. (1993) “optimizing profitability in turbulent environments: a formula for strategic success”, long range planning, vol 26, no 5, pp 11-23. bain, j.s. (1959) industrial organization, new york: wiley & sons. barney, j. (1991) “firm resources and sustained competitive advantage”, journal of management, vol 17, no 1, pp 99-120. bentler, p.m. & bonett, d.g. (1980) “significance test and goodness of fit in the analysis of covariance structures”, psychological bulletin, vol 88, pp 588-606. bird, a. & lerhman, w.g. (1993) “the effects of information technology adoption in japanese corporations”, japan and world economic journal, vol 5, pp 217-242. carmines, e. & mclver, j. (1981) “analysing models with unobserved variables: analysis of covariance structures”, in, bohrnstedt g. borgatta e. (eds), social measurement: current issues, beverly hills, ca: sage, pp 65-115. chandler, g.n. & hanks, s.h. (1994) “market attractiveness, resource-based capabilities, venture strategies and venture performance”, journal of business venturing, vol 9, pp 331-349. conner, k.r. (1991) “a historical comparison of resource-based theory and five schools of thought within industrial economics: do we have a new theory of the firm?”, journal of management, vol 17, no 1, pp 121-154. debreceny, r.; putterill, m.; tung, l.l.; & gilbert, a.l. (2002) “new tools for the determination of e-commerce inhibitors”, decision support systems, vol 34, pp 177-195. dess, g.g. and robinson, r.b. (1984) “measuring organizational performance in the absence of objective measures: the case of privately-held firm and conglomerate business unit”, strategic management journal, vol 5, pp 265-273. dicken, p. & lloyd, p.e. (1990) location in space: theoretical perspectives in economic geography, 3rd ed., new york: harper collins publishers. downey, h.k., & slocum, j.w. (1982) “managerial uncertainty and performance”, social science quarterly, vol 63, no 2, pp 195-207. eze, u.c. (2004) a contextual perspective of e-commerce across borders and industries: comparing perceptions of the it and financial services industries in singapore and nigeria, phd dissertation, nanyang technological university, singapore. ajis vol. 12, no. 1 september 2004 92 gatian, a.w., brown, r.m., & hicks, j.j.o.jr. (1995) “organizational innovativeness, competitive strategy and investment success”, journal of strategic information systems, vol 4, no 1, pp 43-59. gebauer, j. & scharl, a. (1999) “between flexibility and automation: an evaluation of web technology from a business process perspective”, journal of computer-mediated communication, vol 5, no 2, pp 1-18. gilbert, l.a., kendall, j., soh, c., & tung, l.l. (1999) “perceived barriers to the widespread commercial use of the web”, presented to the ausweb99, the fifth australian world wide web conference, ballina, nsw. grant, r.m. (1991a) “the resource-based theory of competitive advantage: implications for strategy formulation”, california management review, pp 114-135. hamel, g. & sampler, j. (1998) “the e-corporation”, fortune, 7, p 80. hepworth, m.e. (1990) geography of the information economy, new york: gilford press. ida. (2001) quarterly report on e-commerce development in singapore. kettelhut, m.c. (1992) “strategic requirements for is in the turbulent healthcare environment”, system management, vol 43, no 6, pp 6-10. lee, m.c.s. & adams, d.a. 1990 “a manager’s guide to the strategic potential of information systems”, information and management, vol 19, pp 169-182. leinbach, t.r. & brunn, s.d. (2001) “introduction: e-commerce: definitions, dimensions and constraints”, in, leinbach, t.r. and brunn, s.d. (eds.), worlds of e-commerce: economic, geographic and social dimensions, chichester: john wiley & sons. mahmood, m. & soon, s.k. (1991) “a comprehensive model for measuring the potential impact of information technology on organizational strategic variables”, decision sciences, vol 22, pp 869-897. mcdougall, p., phillips, p.; covin, j.g., robinson, r.b.jr., & herron, l. (1994) “the effect of industry growth and strategic breadth on new venture performance and strategy content”, strategic management journal, vol 15, no 7, pp 537-554. penrose, e. (1959) the theory of growth of the firm, new york: john wiley & sons. plummer, p. & taylor, m. (2001) “theories of local economic growth: concepts, models, and measurement”, environmental planning, a, no 33, pp 219-236. plummer, p. & taylor, m. (2000) “theory and praxis in economic geography: enterprising’ and local growth in a global economy”, presentation at the wisconsin economic summit, midwest express centre, milwaukee, wisconsin, november 29-december 1, pp 23-41. powell, t.c. & dent-micallef, a. (1997) “information technology as competitive advantage: the role of human, business, and technology resources”, strategic management journal, vol 18, no 5, pp 375-405. porter, m.e. (1990) the competitive advantage of nations, new york: free press. porter, m.e. (1998) “clusters and the new economies of competition”, harvard business review, boston. porter, m.e. (2001) “strategy and the internet”, harvard business review, march, pp 62-78. prahalad, c.k. & hamel, g. (1990) “the core competence of the corporation”, harvard business review, vol 68, no 3, pp 79-91. rai, a. (1997) “technology investment and business performance”, communications of the acm, vol 40, pp 89-97. rainer, r.e. & harrison, a.w. (1993) “toward development of the end user computing construct in a university setting”, decision sciences journal, vol 24, pp 1187-1202. rowsom, m. (1998) “bridging the gap from traditional marketing to electronic commerce”, direct marketing, vol 60, no 9, pp 23-25. sheppard, e. (2000) “competition in space and between places”, in, e. sheppard and t.j. barnes, a companion to economic geography, (eds.), massachusetts: blackwell, pp 171-186. sundell, r.a. (1999) relationship among types of information technology structures, business environments, electronic commerce and financial performance. d.b.a. dissertation, united states international university, san diego. ajis vol. 12, no. 1 september 2004 93 trippi, r.r. & salameh, t.t. (1989) “strategic information systems: current research issues”, journal of information systems management, vol 6, no 3, pp 30-35. tham, i. (2002) “e-commerce activities in singapore are bucking the recession trend, a government survey revealed”, (http://news.com.com/2100-1017-853304.html). ujah, e. (2001) “electronic commerce and small business development in nigeria”, nigeria business times, monday 17th, september. ujah, e. (1999) “keeping up with the growth in nigeria it industry”, nigeria business times, monday 2nd, may, p 12. uma, s. (1992) research methods for business, 2nd ed., new york: john wiley & sons. waddock, s.a. & isabella, l.a. (1989) “strategy, beliefs about the environment, and performance in banking simulation”, journal of management, vol 15, no 4, pp 617-632. zaheer, a. & zaheer, s. (1997) “catching the wave: alertness, responsiveness, and market influence in global electronic networks”, management science, vol 43, no 11, pp 1493-1509. zhuang, y. (2000) a resource-based view of electronic commerce, ph.d. dissertation, university of kentucky. zhu, k.; kraemer, k.l.; & xu, s. (2002) “a cross-country study of electronic business adoption using the technology-organization-environment framework”, presented to the twenty-third international conference on information systems, barcelona, pp 337-348. zwass, v. (2003) “electronic commerce and organizational innovation: aspects and opportunities”, international journal of electronic commerce, vol 7, no 3, pp 7-37. microsoft word 13 1 marina.doc ajis vol 13, no. 1 september 2005 287 diffusion of innovation – the adoption of electronic commerce by small and medium enterprises (smes) a comparative analysis wayne pease faculty of business wide bay campus university of southern queensland (pease@usq.edu.au) michelle rowe faculty of business and public management edith cowan university (m.rowe@ecu.edu.au) abstract this paper explores the issues that influence the diffusion of innovation as it relates to the adoption of e-commerce by small and medium enterprises (smes). it seeks to identify factors facilitating and inhibiting such adoption across contexts – regional, small city and large city. this analysis is cross cultural so the impact of differing economic and cultural issues also will be identified in this research. whilst it is generally accepted that the strategic use of information technology (it) is vital in the marketplace, the rate of such uptake between small and large businesses varies. this research seeks to identify the reasons for this variation. it is critical to understand such factors so that steps can be taken to redress inequity of uptake that might be identified. the paper endeavours to explore factors that are needed to facilitate and encourage it adoption and so positively influence user acceptance and use of it innovations in smes. reasons for such uptake as well as strategic approach to the adoption of e-commerce, and variations regarding same also will be considered. the paper examines existing theory as it pertains to the diffusion of innovation acknowledging the perspective of regional and urban smes in various cultural contexts. empirical investigation exploring this diffusion, the rate of and approach to the uptake by smes is planned using a case study methodology. introduction e-commerce is considered to provide substantial benefits to business, particularly small business. it enables new ways of working to emerge and facilitates an organisation’s reengineering. benefits from e-commerce can be argued to be greater for smes (small and medium enterprises) since traditionally they have operated in an uncertain and dynamic environment (gessin 1996; mcrea 1996; murphy & daley 1999; nooteboom 1994). smes have a critical role to play in the economy. despite advances in information technology and acceptance by large organisations of such technologies, the same level of adoption is not evident amongst smes (bode & burn 2002; knol & stroeken 2001). this paper focuses on the use of e-commerce by smes as it relates to their suppliers and other organisations such as intermediaries (b2b) and as it facilitates interaction with customers and potential customers (b2c). participation in e-commerce is important from the perspective of commercial transactions but more so in the way it encourages transformation of internal systems and the subsequent influences in terms of cost, responsiveness to customers, customization of offerings and the potential emergence of new products and services (crawford 1998, p.6). from this viewpoint it is considered that the study of the uptake of e-commerce by smes in differing contexts by way of city size and culture is important. this study addresses the ajis vol 13, no. 1 september 2005 288 research questions – • does the diffusion of innovation model apply to smes in differing contexts urban and regional and cultural contexts; • what are the factors that influence the adoption of ecommerce (an innovation) amongst smes in differing contexts; • is the rate of adoption slower for smes and to what extent is this a function of city size and location and culture; and • is there an inequity of the rate of diffusion in regional versus urban areas. these research questions will be addressed in the proposed research by way of case studies from the hervey bay region in queensland, smes located in perth, western australia and shenyang, a large industrialised city in north east china. these issues and a number of hypotheses identified in the paper will be tested by way of in-depth interviews and case studies of selected sme proprietors from these nominated cities. uptake of e-commerce by smes crawford (1998, p.20) argued that whilst access to the internet is relatively inexpensive and readily available, smaller enterprises have been slow in using this as a low cost introduction to e-commerce. the main reasons cited for this reluctance were a lack of time, lack of awareness and a lack of business opportunities. crawford also noted some impetus for change is vital for e-commerce to be considered. brown (2002) identified a number of factors limiting the adoption of e-commerce: • e-commerce was seen to be a distraction from core business; • significant perceived cost and risks associated with e-commerce; • lack of strategic vision; and • lack of realisation of value/benefits of e-commerce to individual smes the main reason the internet is used to purchase products is due to its convenience. this is an important factor for regional smes to consider, since it highlights an opportunity for them to effectively reach a broader market. flint & herbert (2000) noted that the smaller the business the less likely it is to have a computer that is used daily and that, regional businesses tend not to appreciate the opportunities the internet provides. most smes adopt e-commerce in a just by chance or casual manner, rather than as a consequence of systematic consideration and planning (engsbo et al 2001; scupola 2002). jocumsen (2004) describes the decision making process of smes as being much less complex compared with what is suggested by theoretical frameworks, with ‘extensive use of learned competencies in the form of ‘perceived’ rationality and the use of rudimentary analytical tools’ (jocumsen 2004, p.668). the lack of real specific awareness which was also identified by brown (2002) is considered to be a substantial obstacle to the uptake of ecommerce. stroeken and coumans (1998, p.227) note that there is a lack of strategic insight of entrepreneurs/proprietors in relation to ‘altered product options as a result of the introduction of it, the alternative market and customer approach, and the desired forms of co-operation’ required to make it work, and this they believe inhibits the application of it within smes. ajis vol 13, no. 1 september 2005 289 the diffusion of innovation model understanding how organisations decide to change and do something different (moseley 2000) is critical to making sure such change (and the methods by which this change is introduced), are effective. research into the diffusion of innovation model is vital since it indicates how barriers can render such implementation unsuccessful (baskerville & priesheje 2001). rogers (1995) developed the diffusion of innovation model which represents the rate of adoption and the stages through which one passes before adopting innovation. this model has been widely believed to ‘best’ explain such adoption of innovation. innovation is ‘an idea, practice or object that is perceived as new by an individual or other unit of adoption’ (rogers 1995, p.11). an innovation is not an invention – from a regional perspective it is ‘doing something which did not exist before in a particular territory or technical area’ (vuarin & rodriguez 1994, p.15). rogers (1995) commented that his six step model was limited and may not always hold true due to its linearity. it can be argued that innovation diffusion is an ‘unstructured emergent phenomenon’ (baskerville & pries-heje 2001, p.187) and is too complex to be expressed in a step-like model. an emerging model of innovation is based on the premise that the diffusion of innovation is unstructured and is stimulated by shock – internal or external to the organisation, rather than being a stepwise and rational process (baskerville & pries-heje 2001; van de ven, angle & poole 1989). wejnert (2002) posits that the adoption process is not uniform and differs based on the nature of the innovation itself, the innovators and the environmental context within which the organisation is placed. we hypothesise that for regional smes these factors differ markedly from those placed in urban regions, particularly with regard to the characteristics of the innovator (the proprietor) as well as the organisation including such things as organisational readiness, attitude to risk, knowledge and experience with e-commerce and so on. similarly we hypothesise that the economic and cultural context has a significant bearing on rate of and approach to adoption of ecommerce amongst smes. these hypotheses will be tested in the research. interactive models such as those developed by burgelman and sayles (1986) uses both technology-linking and need-linking to realise successful innovation diffusion (baskerville & pries-heje 2001). in this model a technical breakthrough achieves context within actual or potential market demand (lucas 1994). the applicability of this approach will be considered across the various contexts included in the study. from this perspective then for smes included in the study the question needs to be considered to what extent do market needs drive technology adoption or vice versa. it is our hypothesis that neither of these occurs for regional smes and this in part explains the lack of adoption of e-commerce for smes located in regional centres compared with urban centres. we also hypothesise that the ‘shock’ which precipitates such adoption for smes is reactive or is either a forced decision or occurs just by chance (engsbo et al. 2001) rather than being strategic in nature. diffusion is the ‘process by which an innovation is communicated through certain channels over time among the members of a social system’ (rogers 1995, p. 5). this existence of a social system is of great importance when considering the uptake of e-commerce generally and the relative rate of adoption within regional/rural areas compared with urban regions. it is hypothesised that there is a lower level of awareness and a different message ‘shared’ ajis vol 13, no. 1 september 2005 290 via the social system in which smes find themselves. this role of the milieu, taking into account the cultural context, and the subsequent communication within networks (which would include competition) is one that requires further investigation. rogers (1995) postulated that innovation was more likely to succeed and be more readily adopted if the relative advantage as a consequence of its introduction was evident; if it was compatible with the organisation, its operations and its view of the world; if it was not ‘too’ complex, and if it was trialable and could be observed prior to adoption. local initiatives in the hervey bay region have attempted to embody these factors (pease & rowe 2003a, 2003b; pease rowe & wright 2003). the diffusion of innovation model identified a number of categories of adopter, based on the speed with which they adopted the innovation. the question is what the profile of innovators and early adopters is, and to what extent are they likely to reside in regional areas, mid sized cities and large industrialised cities? griffith et al (1986) argued that organisations must possess certain traits if innovation was to have a greater chance of success. these included such things as skill, expertise and experience, management support, approach to risk and leadership. rogers (1995) and ruttan (1996) have found that there are three main characteristics that describe innovators and early adopters: • their business or professional circumstances: early adopters tend to have a pressing problem, i.e., a clear gap between the actual and the desired and have access to the resources necessary to change their current practice; • their personal attributes: early adopters tend to have good formal education, relatively high socio economic status, are socially mobile, open to new ideas and able to cope with risk and uncertainty; • their connections: early adopters tend to be exposed to a variety of media, to be well connected in interpersonal networks and to have good contacts with opinion leaders, change agents and other early adopters (moseley 2000). recent research focuses on the importance of the third of these factors, reflecting the view that ‘innovation is a social rather than a solely individual process…occurring among a variety of stakeholders rather than as a matter of transfer or dissemination of technology or ideas’ (engel 1997, p.125). networking and relationship building, for example, amongst smes is critical to this. analysis of this factor within urban and regional areas, and across cultural contexts will be explored in this research. essentially the diffusion is about reducing uncertainty ‘among members of the social system by means of communication’ (knol & stroeken 2001, p.229). for smes this is largely by way of interpersonal channels with learning being critical to the adoption and implementation of complex innovations. promotion of the adoption of e-commerce should include the use of appropriate media to transmit information, the encouragement of early adopters to converse with their peers, targeted training and education programmes, the deployment of facilitators to inform and persuade potential innovators, the identification and support of opinion leaders, the issue of incentives to promote some early trial adoptions, and the promise of ongoing support (moseley 2000). the extent to which these issues are at play will be considered in relation to the three ‘levels’ included in this study. it is important to note that the consequence of innovation may not all be positive and this need to be borne in mind by those promoting, in this case, the adoption of e-commerce. the extent to which such innovation is not positive will be investigated across these levels. ajis vol 13, no. 1 september 2005 291 the challenge in regional/rural areas rural development requires a mass of innovation decisions to be taken by individuals, households, firms and voluntary and public bodies in areas devoid of scale or agglomeration economies (equating to engsbo’s et al (2001) concept of critical mass), replete with problems relating to remoteness and an urgent need to restructure the economic base (moseley 2000). consideration will be given to local initiatives in regional areas to encourage such it adoption. such initiatives, where they exist, no doubt reflect the need to restructure the economic base of a region and provide access to infrastructure in regional areas. adoption of a new way of doing things by smes, amongst others, is arguably fundamental to bringing about change in regional areas insofar as it facilitates the emergence of the new economy. from another perspective the role of smes in emerging and developing economies such as china is also of interest – what is the role of smes similarly in bringing about change and facilitating the emergence of the new economy? from these points alone the issue of e-commerce uptake is worthy of study. overview of state of play there are numerous barriers to the adoption of e-commerce, particularly by regional smes. these are largely however internal in nature assuming that external initiatives have been put in place – this however may be quite diverse. nevertheless these are ‘real’ potential barriers which need to be addressed. the diffusion of innovations model is important to understanding the dynamics at play in relation to adoption of e-commerce amongst smes and this has been outlined above. empirical research now needs to be conducted to identify the current situation so that appropriate actions can be set in train. this is the intention of this research. research objectives the objectives of the research are to identify factors influencing the adoption of innovation, in this case e-commerce, by smes in regional and urban contexts. lessons from the diffusion of innovation model will be applied both from the point of view of actual adoption and diffusion of innovation as well as from the regional/urban and cultural context to ensure that initiatives undertaken to promote same are effective. best practice models and case studies then could be developed to act as a guide to smes as they consider adopting ecommerce and other ict in various contexts. this paper draws upon and reports part of a long term research project that aims to identify factors affecting the uptake of e-commerce by smes comparing urban and regional contexts, and introducing cultural issues. research methodology the study will be an ongoing one initially involving the case study methodology appropriate to the style of research questions posed that is concerned with ‘how’ and ‘why’ (morgan 1997; leonard-barton, 1988). this methodology was selected given the exploratory nature of the research and since it provided a non-threatening environment ajis vol 13, no. 1 september 2005 292 whereby participants, sme proprietors, could discuss relevant issues in-depth and interact via open discussion. the structured interview questions from which the case studies will be built will be shaped specifically to identify what smes understand by e-commerce, to identify factors triggering or impeding its adoption, to identify time take in considering whether to adopt e-commerce and to identify the benefits from, or expected from, and difficulties associated with ecommerce. the instrument will be pretested and modified to increase clarity, particularly regarding the understanding of questions given the diversity in cultural context. consideration will be given to follow up surveys of sme proprietors in the three contexts. the decision to proceed with these surveys largely is dependent on access to a database of participants in the chinese context. the findings from the in depth discussions will be analysed to identify the factors at play both triggering and hindering the adoption of e-commerce within the various contexts. models will be tested and best practice case studies developed to assist smes in various economic, cultural contexts and locations as they consider adoption of ict and ecommerce. the research seeks to identify variation of factors facilitating/inhibiting such adoption and hopes to achieve the following outcomes: • add to the body of knowledge so that smes are better able to consider and plan potential adoption of ict and ecommerce; • contribute to theoretical understanding of ict and ecommerce adoption by smes in various cultural and economic contexts; and • guide policy makers as they seek to facilitate the adoption of ict and ecommerce amongst smes. references baskerville, r. & pries-heje, j. (2001) “a multiple-theory analysis of a diffusion of information technology case”, information systems journal, vol 11, no 3, pp 181212. bode, s. & burn, j.m. (2002) “strategies for consultancy engagement for e-business development a case analysis of australian smes”, in managing information technology in small business: challenges and solutions, ed. s. burgess, idea group, melbourne, australia, pp 227-245. brown e (ed.) (2002) accelerating the up-take of e-commerce by small and medium enterprises, small enterprise telecommunications centre (setel), viewed 23 march 2003, <http://www.setel.com.au/smeforum2002/final_report.pdf>. burgelman, r.a. & sayles, l.r. (1986) inside corporate innovation: strategy, structure, and managerial skills, the free press, new york crawford, j. (1998) news –a project to get smaller enterprises online, department of industry, science and tourism, viewed march 23 2003, <http://www.noie.gov.au/publications/noie/sme/newsbk.pdf>. engel p.g.h. (1997) the social organisation of innovation: a focus on stakeholder interaction, amsterdam, royal tropical institute. engsbo, m., saarinen, t., salmi, h. & scupola, a. (2001) “a framework of adoption of ecommerce in networks of smes” in proceedings of iris 2001, ulvik, norway. flint, j. & herbert, r. (2000) “marketing, the internet & regional small business”, anzmac 2000 conference – visionary marketing for the 21 st century, facing ajis vol 13, no. 1 september 2005 293 the challenge, gold coast, australia, 28 nov – 1 dec. gessin, j. (1996) “impact of electronic commerce on small and medium sized enterprises”, management, jan-feb, pp 11-12. griffiths, j.m., ronald, g.h., ellen, a.s. & pat, c. (1986), diffusion of innovations in library and information science final report, rockville, md, king research. jocumsen, g. (2004) “how do small business managers make strategic marketing decisions?”, european journal of marketing, vol. 38, no. 5/6, pp 659-74. knol, w.h.c. & stroeken, j.h.m. (2001) “the diffusion and adoption of information technology in smalland medium – sized enterprises through it scenarios”, technology analysis & strategic management, vol 13, no 2, pp 227-246. leonard-barton, d. (1988) “implementation characteristics of organisation innovations, limits and opportunities for managerial strategies”, communications research, vol 15, no 5, pp 603-631. lucas, h.c. (1994) ”marketing and technology strategy in a medium-tech start-up”, information & management, vol 27, pp 247-257. mcrea, p. (1996) “reshaping industry with the internet”, management, jan-feb, pp 7-10. morgan, d.l. (1997) focus groups as qualitative research, (2 nd edition), sage, london. moseley, m. (2000) “innovation and rural development: some lessons from britain and western europe”, planning practice and research, feb-may, vol 15, issue 1/2, pp 95-115. murphy, p. r. & daley, j. m. (1999) “edi benefits and barriers: comparing international freight forwarders and their customers”, international journal of physical distribution and logic management, vol 29, no 3, pp 207-216. nooteboom, b. (1994) “innovation and diffusion in small firms: theory and evidence”, small business economics, vol 6, no 5, pp 327-347. pease, w. & rowe, m. (2003a) “e-commerce and small and medium enterprises (smes) in regional communities”, proceedings of the chartered institute of marketing inaugural conference, sydney, australia, august 20-22. pease, w. & rowe, m. (2003b) “factors affecting take-up of electronic commerce b small and medium enterprises (smes) in the regional community of hervey bay, australia”, international telecommunications society’s asia-australasia regional conference, perth, australia, june 22-24. pease, w., rowe, m. & wright, l. (2003) “ict and regional development in hervey bay”, international telecommunications society’s asia-australasia regional conference, perth, australia, june 22-24. rogers e. (1995) diffusion of innovations, 4 th ed, free press, new york. ruttan, v.w. (1996) “what happened to technology adoption-diffusion research?”, sociologica ruralis, vol 36, no 1, pp 51-73. scupola, a. (2002) “adoption issues of business-to-business internet commerce in european smes”, proceedings of the 35 th hawaii international conference on system sciences, hawaii, usa. stroeken, j. & coumans, j. (1998) “the actual and potential use of information technology in small and medium sized enterprises”, promethus, vol 16, no 4, pp 469-483. van de ven, a.h., angle, h.l. & poole, m.s. (1989) research on the management of innovation: the minnesota studies, harper & row, new york. vuarin p. & rodriguez m. (1994) “innovation and communication within leader”, leader magazine, vol 7, pp 13-16. ajis vol 13, no. 1 september 2005 294 wejnert, b. (2002) “integrating models of diffusion of innovations: a conceptual framework”, annual review of sociology, vol 28, no 1, pp 297-326. microsoft word ajis112final.doc australasian journal of information systems vol 11 no 2 may 2004 37 steps to internet adoption success: a study of four small regional organisations aileen cater-steel and shelly grist usq australia contact author: aileen cater-steel dept of information systems, faculty of business usq, toowoomba 4350 caterst@usq.edu.au abstract in response to concerns about the reluctance of australian small business to embrace e-commerce, this study describes the steps taken by four small regional organisations to establish web sites and enter into e-commerce. as well as mapping the actual adoption steps to models of e-commerce adoption, recommendations are made to guide small business through initial web site development. this study suggests that when adopting internet technologies, small organisations needs to focus on sound business objectives, plan for an evolutionary approach, and recognise the value of adapting a traditional systems development life cycle approach to web site development. introduction australians are recognised as early adopters of technology, however, take up rates of e-commerce by small organisations could be improved. to add to the concern, it has been reported that only a limited proportion of small to medium enterprises (smes) are interested in going online (castellari, 2003). for those that have established web sites, many are now disillusioned by their attempts as promised benefits have not materialized. according to engsbo, saarinen, salmi and scupola (2001), smes should be encouraged to adopt e-commerce to increase their efficiency, achieve competitive advantage, provide faster customer response and better service offerings. the aim of this research is to report on the adoption of e-commerce by australian smes, then, through four case studies (based in regional queensland), identify how internet-based systems were implemented to improve business processes. this research examines what happened in four cases when small regional organisations commissioned their first web sites. the cases are diverse: an agricultural engineering consultancy established an intranet to enable decentralisation of operations from a single site to three offices in three states; a ‘mum & dad’ import micro-business used the web to bypass the middleman and sell direct to their customers; a sporting organisation embraced the internet as the most cost-effective and efficient way to plan and promote an international event; and finally, a small, well-established real estate firm responded to the needs of a new ‘web customer’ by creating a web site for realty listings and staff details. this paper considers the important role of small businesses in australia, and reports their use of computers and e-commerce. then, detailed insights into the adoption of e-commerce by four small organisations are provided. the discussion section summarises the success factors across all four cases. finally, recommendations to sme managers are made, implications for research and practice are discussed, along with opportunities for further research, and the limitations of this study. internet adoption by australian firms the focus of this research is on small organisations, defined by the australian bureau of statistics (abs, 2002) as businesses employing less than 20 people. the abs recognises three categories within that definition: non-employing businesses (i.e. sole-proprietorships and partnerships without employees); businesses with one to four employees; and businesses with between five and 19 employees. the first two categories are sometimes referred to as micro-businesses. organisations with 20-199 employees are classed as medium sized businesses. therefore the term small to medium enterprise (sme) refers to organisations ranging from the sole operator to a firm employing up to 199 staff. australasian journal of information systems vol 11 no 2 may 2004 38 small business in australia although about 95 per cent of all businesses in australia are small businesses (nettleship, 2003), there is little research reported about their adoption of information technology in general and the internet in particular. the small business sector makes a very significant contribution to australia, employing half the australian private workforce, and contributing more than 30 percent of the gross domestic product (nettleship, 2003). australians are recognised as early adopters of technology, however, recent news articles claim that the take up rates of e-commerce by smes in australia are dismally low (castellari, 2003). use of computers and the internet by smes on one hand, australia’s unmetered phone access is recognised in an oecd report as providing a pricing structure favourable to internet access (paltridge, 2000), and helping place australia in the top six internet adopters of the 29 oecd countries reported. however, this advantage only applies to businesses with local access to their internet service provider, and internet connection high costs and slow speeds may be prohibitive for regional, rural and remote companies. in march 2002, the abs released statistics about the use of computers by small business. about two-thirds of the 1,162,000 small businesses reported use of computers in their business operations. as the size of the business increases, so does the likelihood that a business would have a computer, as shown in figure 1. figure1. use of computers, internet and e-commerce by small business (abs). as at june 2001, more than half of all small businesses had access to the internet. again, the use of the internet was more common in the larger businesses. the most common usage of the internet was for email and research. as shown in figure 1, almost 20 per cent of small businesses were involved in direct e-commerce, which was defined as ‘making or receiving payments via the internet’ (abs, 2003). researchers do not agree on a definition of e-commerce for smes. e-commerce is defined as ‘every type of business transaction in which the participants (i.e. suppliers, end users etc.) prepare or transact business or conduct their trade in goods and services electronically’ (docita, 1999); or 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% have a computer access to the internet involved in ecommerce non-employing 1 -4 employees 5 19 employees total small business australasian journal of information systems vol 11 no 2 may 2004 39 ‘the marketing, buying, and selling of products and services on the internet’ (awad, 2002); or ‘the use of internet technologies for internal business processes (intranet), for business relationships (extranet), and for the buying and selling of goods, services and information (internet)’ (oecd, 1998). therefore, the case of a sole proprietor who uses internet banking to bpay a telephone account would be considered e-commerce by the abs, but not by other researchers. consequently, it is difficult to compare adoption levels from different studies, and the abs adoption levels shown in figure 1 could be considered to be a generous estimate of in may 2002, the yellow pages conducted telephone interviews with 1,200 small and 600 medium sized businesses to determine the extent of their technology usage and e-commerce activities (yellow pages, 2002). the sample included 300 queensland-based businesses, 135 of which were located in non-metropolitan areas. this study, which provides a valuable snapshot of the adoption of e-commerce by australian smes, found that 79 per cent of small businesses had an internet connection, by way of a permanently connected modem (40%) or a dial-up modem (34%). more than half the small businesses interviewed reported that they did not have broadband access available. email to communicate with customers and suppliers was cited as the most important reason to use the internet. many organisations also recognised the importance of having a web site with one third of all small businesses reporting that they did have a web site, and a further 17 per cent stating their intention to get one within 12 months. for small businesses, 45 per cent had used a specialist web site designer to create their site, and the same proportion developed the sites themselves. as well as using the web to advertise their products and services (36%), many small businesses (38%) were purchasing business-related goods on-line, almost one third of all small businesses were taking orders online, and one quarter of small businesses received payments over the internet. theoretical perspectives in the 1970’s, a popular model was devised to represent the evolutionary nature of information systems development. nolan’s stages of growth model comprised initiation, contagion, control, and integration (nolan, 1973). mckay, prananto and marshall (2000) created a stages of growth for ebusiness (sog-e) model by taking galliers and sutherland’s adaptation of nolan’s model, and adding an internet commerce model comprising six stages: no presence; static online presence; interactive online presence; internet commerce; internal integration; and external integration. recently, in a similar vein, other researchers have proposed phases or steps to represent the evolutionary adoption of e-commerce by smes. a four phase model suggested by chau and turner (2002) starts from the position zero: conventional sme utilising no web-based e-commerce, to phase 1: incorporate a static web presence; 2: add dynamic and interactive functionality to ecommerce use; 3: engage in substantial re-engineering of business processes to accommodate ecommerce initiatives; and finally, phase 4: virtual business enterprise. in evaluating the adoption of e-commerce by regional victorian smes, van beveren and thomson (2002) identified the steps to e-commerce as email, website advertising and online ordering, forms availability, search facility and finally event information and/or links to other firms. the australian-based small enterprise telecommunications centre (setel) encourages smes with a five step process: 1: get on-line and get email; 2: get internet banking; 3: get a web-site, initially to advertise the business phone number and email address; 4: get an interactive dynamic e-commerce system integrated with traditional business systems; 5: get voice and data systems integrated’ (brown, 2002). however, these recipe-style prescriptions may not be universally practical for many organisations. this study therefore examines the first steps taken by four small regional organisations in their adoption of e-commerce. action research method the research method adopted involved both action research and interpretive case studies. action research ‘aims to contribute both to the practical concerns of people in an immediate problematic situation and to the goals of social science by joint collaboration within a mutually acceptable ethical framework’ (rapoport, 1970). one of the researchers involved in this study is a specialist web developer and created the web sites for the four organisations reported here, gaining valuable australasian journal of information systems vol 11 no 2 may 2004 40 insights into the motivation and the evolutionary process. after the web sites had been in use for some months, the researcher conducted structured telephone interviews with key personnel to ascertain the benefits derived from the systems implemented. the use of case studies is recommended as a valuable strategy for information systems research in a natural setting, as it provides an opportunity to understand the nature and complexity of the situation (benbasat, goldstein, & mead, 1987). the four cases reported here were selected because of their different characteristics, and because of the researcher’s familiarity with them they proved a rich and ready source of information (perry, 1998; yin, 1994). adoption case studies this section presents a brief description of each of the four organisations, details the steps they took to adopt e-commerce, and assesses the success of their efforts. case a: agricultural consulting firm fsa consulting is a nine year old agricultural consulting business firmly established in the southeast queensland area. when one of the principal partners moved to victoria, decentralising the business operations became a priority. a corporate online administration system was developed to allow project and client information to be accessed and maintained regardless of location. a staff section was created for consultants to track billable hours to ensure performance requirements were being met. the staff section also included a calendar area for staff to post their weekly schedules, coordinate meetings, and book a corporate vehicle for on-site client visits. as the administration system slowly became key to fsa’s business operations, additional components were added to make the operations independent of office location. some components subsumed local processes while others provided new or enhanced capabilities. a job history section was created for project members record detailed notes on current tasks, team meetings and discussions with clients. this allowed team members located in different states to coordinate their efforts and keep in touch with project developments. a positive by-product of maintaining the job history information was that it enforced quality assurances processes recommended by liability insurance suppliers; an important consideration which has lately become more critical for businesses of all sizes. standard forms were also added to the website which allowed staff members to fill out and complete standard office forms even while offsite visiting clients. with the business operation decentralised, fsa consulting could now expand its consulting area to attract new clients. in addition to the victoria and queensland offices, a business office in south australia was soon opened to provide consulting services across a third state. case b: craft goods wholesaler case b involves a micro-business that has used the web to bypass the middleman and sell direct to their customers – a perfect illustration of disintermediation. case b served as an importer/supplier of craft items for a major chain store. as the product range they supplied was popular, revenues and sales increased at a healthy rate. since the market showed such a positive growth rate, the chain store set up other suppliers in direct competition with case b who had been their sole source for this particular product range. this proved to be a valuable business strategy for the chain store since it allowed them to increase their margins from 100% to 300%. however, it was not so profitable for case b, who now saw its margins squeezed by the new competition. moreover, cash flow was also affected as the chain store invoice terms gradually changed from the original 30 days to 90 days. case b found itself in a very weak bargaining position. with only one customer they had little recourse when that customer chose to dictate new and less favourable terms. the owners of case b therefore decided to bypass the chain store and sell direct to customers. case b staff had researched other craft websites and magazines, and were sure that they could compete on product variety, quality and price. they commissioned a web developer (one of the authors of this paper) to build a site to achieve three key goals. firstly, high resolution images were required to display the quality of their products. a second priority was for clear, easy-to-follow navigation so australasian journal of information systems vol 11 no 2 may 2004 41 that the clients would not get discouraged. finally, some means of communicating with customers via the website was required. case b felt very strongly that another way to compete was by using email to build close customer relations, in other words, putting a face on what is basically a very impersonal transaction. to achieve this personalisation, a feedback form was included in the website for customers to register their interest in different product areas. all respondents received personalised email responses from case b. the company viewpoint was that if someone took the time to fill in the registration form ‘they were a potential customer waiting to be sold;’ and it was ‘very important to convert that interest into a sale’ (from interview with case b owner). with the website complete and meeting all key goals, case b launched their virtual store. initially, visits to the site were sporadic. case b had built a virtual storefront but no one knew they were there. a key turning point for case b’s online success was locating a portal which attracted all those customers they were trying to find. for a minimal yearly fee, an advertisement for case b’s website was added to the portal and suddenly they were ‘found’. from an initial two to three visits per day, site traffic is currently averaging 80 visits per day. initially customer order payments were by cheque, money order, or direct bank deposits. contrary to popular theory, case b’s customers expressed a clear preference for paying online by credit card. clearly these customers were comfortable with internet shopping and any initial security concerns were overshadowed by their strong desire to place their orders and pay for them in one step. after eight months online, case b has grown from one customer to 500 registered customers providing four to six orders per day. this case suggests that the internet can provide a level playing field. disintermediation allows david to compete against goliath, and on david’s terms. case c: sporting association the inaugural polocrosse world cup was held at warwick, queensland in april 2003. with less than a year to plan an international event, and a very limited budget, the world cup team embraced the internet as the most cost-effective and efficient way of communicating with participants and of providing information to spectators and the media. the polocrosse world cup website was designed to serve three primary target audiences. for competitors and participants, one section of the website provided detailed information on competition rules, the allocation of horses to teams, and the facilities available. for the media, the website was used to both recognise and encourage sponsorship from the public and private sectors. forging strong relations with media partners was recognized early on as key to attracting corporate sponsors and to building a multi-media event marketing campaign. to facilitate this, another section of the website provided varying levels of recognition to media and corporate sponsors. a news section was also created to provide fact sheets and news releases to keep media partners informed of breaking events. for the final target audience, spectators, an area of the website was designed to build excitement about the world cup event and to attract a new audience to the sport. as polocrosse is one of only three truly australian sports (along with australian rules and camp drafting), a brief history and introduction to the sport was provided on the website to educate the uninitiated. having the inaugural world cup in australia was seen as an ideal vehicle to generate interest and attract new fans. the website was launched in august 2002 and continued to grow and adapt to meet the needs of the world cup event. during the cup competition, new sections were added to the website to provide coverage of the competition. a description and the results of each match were displayed immediately following the match so spectators around the world could keep up to date. the contribution of the media partners enabled photos and videos of each match to be made available from the website. website statistics show that traffic grew from 324 sessions served in the week the site was launched to 6,129 sessions for the week of competition (may 4 th ). more than 25,000 people attended the inaugural polocrosse world cup, more than double the number hoped for by the organisers. one of the objectives of the event was to promote polocrosse to a new audience and exit research at the event showed that 20 per cent of the audience were newcomers to polocrosse and/or horse sports. many of these were day trippers who had accessed information about the event through reciprocal links from tourist and other sports websites. the internet delivered on its promise of providing a cost-effective and efficient means of publicising the event. australasian journal of information systems vol 11 no 2 may 2004 42 case d: real estate firm the internet is slowly changing even the most traditional ways of doing business. with over 50 years serving the real estate needs of the toowoomba area, peter snow & co has a solid understanding of their market and their customers. but the internet has provided a new type of customer with different goals and objectives. they are time poor, so they want to peruse property listings at their convenience. they want instant responses. they do not want to wait for thursday’s property listings in the newspaper; they want to go online and find out what properties are available right now. they prefer to do their own research. they will decide whether they think the real estate firm is trustworthy, and that the agent is someone they can do business with, based on their impression of the website, in lieu of a face-to-face meeting. peter snow & co has evolved to meet the changing needs of this new customer. a website was created for the business and all property listings are now available 24/7. their company’s sales pitch has been updated: ‘we’re an independent real estate firm with 53 years of experience in the area. we’ll promote your property to the local area and to the world by listing your property on our website.’ traditional customers are also pleased with some of the new services that the internet can provide. peter snow & co can nowprovide feedback such as ‘we listed your property at 4:00 in the afternoon and 12 visitors had viewed the advertisement by 9 a.m. the next morning’. table 1 summarises the characteristics and e-commerce adoption of the four cases. characteristic case a agricultural consulting case b wholesaler case c sporting association case d real estate firm # of employees 12 0 0 ~15 website www.fsaconsultin g.net withheld – confidential www.polocrosse.co m.au/worldcup www.petersnow.com. au connection adsl modem modem modem web host webcentral webcentral diamond media webcentral email � � � � web presence � � � � web forms � enquiry � register � � web search � google � � � web orders not applicable � � not applicable table 1. summary of characteristics of four case study organisations discussion the internet has many enablers for small businesses. case a illustrates the effectiveness of using an intranet to decentralise and streamline business operations. fsa’s administration system allowed the business to expand its market share and geographic range of operations, provided better coordination and dissemination of project tasks and information, and improved quality control processes that helped them address insurance liability issues. case b proved that disintermediation can be a powerful tool for small businesses. on the internet a small business can compete with the large companies. a virtual storefront can offer a product selection usually only available in stores in major cities that have a large population base to draw from. there is minimal cost difference between offering 20 products or 200 products when all of the floor space is virtual. the internet offers small businesses the ability to offer a rich description and selection of products with a wide geographical reach; their products are available to a global market. this combination of richness and reach was previously only available to large corporations. case c demonstrates that even non-profit organisations can effectively use the internet to promote their objectives. the polocrosse world cup committee orchestrated the efforts of their mostly volunteer staff to overcome the barriers of limited time, funding, and media access. in the past, non australasian journal of information systems vol 11 no 2 may 2004 43 -profit organisations have received little attention from information systems researchers, but this case proves that they also can benefit from e-commerce adoption. case d highlights that the traditional face of business is also evolving. practices that have been in place for fifty years changed to meet the new demands and needs that the internet created. the welcome by-product is the new services that can now be offered to traditional customers, to create and sustain customer loyalty. the stages of internet adoption of these four organisations are summarised in table 2 and mapped to the four models introduced earlier. this analysis revealed that the model proposed by chau and turner (2002) is the most accurate to describe the initial steps taken. the sog-e model may provide an accurate representation for medium and large businesses, but the six stages of the galliers and sutherland model, which constitute the lower half of the sog-e model, do not appear to fit the it maturity stages of the four case studies presented here. case steps to e-business best model fit a fsa consulting 1.email 2.static public web site 3.dynamic admin system 4.electronic newsletters and search added to public web site 5.admin system extended to include more business processes. the it maturity stages of the sog-e model are a good fit for fsa. b case b 1.email 2.static public web site with order and registration form 3.secure order form added with ability to process credit cards 4.shopping cart facility added to web site (proposed) case b is the only true ecommerce firm included in the case studies. its best fit is the chau & turner model. c polocrosse 1.static public web site with access to static ticket order forms. 2.electronic newsletters 3.competition results posted with minimal delay, also video and images of competition available from the web the chau & turner model is the best fit if step 3 is interpreted to include dynamic information for promotional purposes. d peter snow & co 1.email 2.web site with static company content and dynamic property listings this case has the lowest it adoption of the four case studies. chau & turner model is the best fit. table 2. summary of adoption stages of four case study organisations. recommendations this research recommends that building a successful e-commerce site starts with developing a clear understanding of the business objectives. the next step is to transform the list of objectives into an e-commerce site plan. this planning step requires an understanding of what is possible within the realm of e-commerce. such specialised skills are not always available within a small business. most smes at this point must decide whether to outsource their site development or hire/train staff to acquire these skills. the planning step, where business objectives are translated into an e-commerce site plan, is critical. based on the research presented here, it is recommended to extend the systems development life cycle (sdlc) approach proposed by laudon and travers (2004): • systems analysis/planning: identify business objectives, system functionality, and information requirements. part of the information requirements should include analysing the customer’s information/services needs to enhance customer service. time should also be spent determining how to measure the return on investment (roi). web sites, like any other business expense, should contribute to the bottom line. even if the site is nothing australasian journal of information systems vol 11 no 2 may 2004 44 more than an online version of a yellow pages advertisement, part of the plan should include measuring its success or failure. • systems design: develop the logical and physical layout of the web site to provide an efficient and intuitive user interface. questions such as “how many clicks does it take to find a contact number?” should be answered here. • building the system: design the graphical layout of the web site, develop the html and any dynamic pages, select a hosting environment. selection of an appropriate colour and style to represent the business is critical. according to fogg et al. (2002), the design look is the most important factor in the credibility of a web site. • testing: verify the functionality of the site and also that it meets the business objectives defined earlier in the planning phase. • implementation and maintenance: web sites are similar to a living entity. as the business grows and evolves, so too should the web site. web site statistics can provide a wealth of information about the effective features of the web site and also highlight areas in need of some tweaking or a major overhaul. web sites also need continual checking for outdated links, and updates to keep business information current. an outdated web site reflects very poorly on the business it represents. conclusions in all the cases mentioned, internet adoption was in response to a clear business objective, and has enabled each organisation to achieve its objective. it is recognised that the web sites developed for the three businesses will continue to evolve, but at an affordable rate, and in response to demands from customers, rather than for the sake of technology adoption. further research will continue to track the success of the firms reported here in terms of efficiency improvements and increased sales and profits. one possible limitation of this study, and of all action research projects, could be a degree of bias stemming from the involvement of one of the authors as the web developer and researcher. findings from this research make both theoretical and practical contributions. recent reports in the media (e.g. castellari, 2003) of the slow uptake of e-commerce by smes are shown to be out of date compared with abs and yellow pages statistics. this research warns that recipe-style prescriptions must be tailored to focus on achieving step-wise improvements to business processes. to assist information systems professionals and small business owners, recommendations are made, based on the positive experience of these four organisations. in terms of theory, researchers in this field of study need to recognise that many small businesses have overcome the hurdle of lack of awareness, and are looking at enhancing their basic e-commerce applications to reap further benefits. acknowledgement this paper is an extension of work presented by the authors at the itira conference held at caloundra in december 2003. the authors appreciate the comments and recommendations made by the itira reviewers and delegates. references abs.8127.0 characteristics of small business. retrieved 13 may, 2003, from the world wide web: http://www.abs.gov.au/ausstats/abs@.nsf/0/e49e3b4dc3595c92ca2568a900139377?o pen abs (2003). latest figures. media release 19/3/2003. retrieved 13 may, 2003, from the world wide web: http://www.businessaccess.vic.gov.au/web/sbv/sbvsite.nsf/pages/info_sheets_stats awad, e. (2002). electronic commerce from vision to fulfillment. upper saddle river, new jersey: prentice hall. benbasat, i., goldstein, d. k., & mead, m. (1987). the case research strategy in studies of information systems. mis quarterly, 11(3), 369-387. australasian journal of information systems vol 11 no 2 may 2004 45 brown, e. (2002). accelerating the up-take of e-commerce by small & medium enterprises: a report and action plan by the sme e-commerce forum taskforce: small enterprise telecommunications centre (setel). castellari, k. (2003, 11 march). transitions: techno fence divides business. australian. chau, s. b., & turner, p. (2002, december). an exploration of factors that influence the ability of small and medium enterprises to engage in electronic commerce: preliminary findings from 34 australian case studies. paper presented at the 13th australasian conference on information systems, melbourne. docita (1999). australia's e-commerce report card chapter 1. department of communications, it and the arts. retrieved 11 june, 2003, from the world wide web: http://www.dcita.gov.au/article/0,,0_1-2_1-4_13790,00.html fogg, b.j., soohoo, c., danielson, d., marable, l., stanford, j., & tauber, e. (2002, 29 october). how do people evaluate a web site's credibility?, persuasive technology lab, stanford university, retrieved 21 february, 2004, from the world wide web http://www.consumerwebwatch.org/news/report3_credibilityresearch/stanfordptl_abstract.h tm. engsbo, m., saarinen, t., salmi, h., & scupola, a. (2001, 11-12 august). a framework of adoption of e-commerce in networks of smes. paper presented at the iris24: the 24th information systems research seminar in scandinavia, university of bergen, norway. mckay, j., prananto, a., & marshall, p. (2000). e-business maturity: the sog-e model. paper presented at acis, brisbane. nettleship, d. (2003, april/may). small is beautiful unless you're a small business. information age, 14-16. nolan, r. l. (1973). managing the computer resource: a stage hypothesis. communications of the acm, 16, 339-405. laudon, k., & traver, g. (2004). e-commerce business, technology, society, 2 nd edn, addison wesley, boston. oecd (1998). smes and electronic commerce (dsti/ind/pme(98)18/final): oecd. paltridge, s. (2000). local access pricing and the international digital divide. oecd. retrieved 10th june, 2003, from the world wide web: http://www.isoc.org/oti/ perry, c. (1998). a structured approach to presenting theses. australasian marketing journal, 6(1), 63-85. rapoport, t. (1970). three dilemmas of action research. human relations, 23, 499-513. van beveren, j., & thomson, h. (2002). the use of electronic commerce by smes in victoria, australia. journal of small business management, 40(3), 250-253. yellow pages (2002). e-business report: the online experience of small and medium enterprises. yin, r. k. (1994). case study research: design and methods. thousand oaks, calif: sage publications. microsoft word ajis10-2finala.doc ajis vol. 10 no. 2 may 2003 144 the co-regulatory scheme for internet content regulation: operational effectiveness geoffrey a sandy school of information systems faculty of business and law victoria university, melbourne, australia e-mail: geoff.sandy@vu.edu.au abstract the debate about the likely effectives of a scheme to regulate the access to certain internet content during the hearings of the select senate select committee and during the parliamentary debates was characterized by a great diversity of opinion. this paper describes these opinions and then examines the claims made about the operational effectiveness of the co-regulatory scheme for internet content regulation since its inception on 1 january 2000. keywords internet content regulation, co-regulatory scheme, online services bill, censorship introduction recently, the department of communications information technology and the arts published a review of the operation of schedule 5 to the broadcasting services act (commonwealth of australia 2002). this refers to the co regulatory scheme for internet content regulation that became operational on 1 january 2000. the purpose of the scheme is to establish a framework for the regulation of the content of online services.). specifically, the scheme seeks to: 1. provide a means of addressing complaints about certain internet content 2. restrict access to certain internet content that is likely to cause offence to a reasonable adult 3. protect children from exposure to internet content that is unsuitable for them in a relatively open society like australia all political parties claim to support the principle that adults should be able to read, hear and see what they want. none would consider this as an absolute right to “freedom of expression”. it is how this is interpreted that gives rise to different views about the justifiable degree of government censorship. equally all would wish to protect children from unsuitable material but differ as to how this is to be defined, and as to the proper role of the government in this regard. the bill was prepared by the ruling conservative coalition but was opposed in part or whole by the opposition parties. this paper discusses the effectiveness of the scheme in terms of its specific objectives after two and half years of operation. it identifies the main arguments about its likely effectiveness of those who supported or opposed the bill at the time of its preparation and passing through the parliament. it then examines the claims made about its operational effectiveness since that time. it does not address the existing or proposed state or territory legislation regulating internet content. the primary source documents used for this research are: 1. the official committee hansard of the hearings from the submitters on the broadcasting services amendment (online services) bill 1999 by the senate select committee on information technologies 2. senate hansard of the second reading of the bill 3. house hansard of the second reading of the bill 4. the broadcasting services amendment (online services) bill 1999 5. department of communications, information technology and the arts, bi-annual reports of the scheme 6. official media releases throughout the paper reference is made to (1) above as the “hearings”, (2) and (3) as the “debates” and (4) as the bill. the abbreviated in-text citation used throughout for (1) is “ssc” for senate select committee, (2) is “sh” for senate hansard and (3) is “hh” for house hansard. full details are provided in the references section. acknowledgement is made of the reliance on the work of electronic frontiers australia (efa). efa is the premier civil liberties group regarding the internet in australia. its views are representative of those opposed to the bill and those who claim the scheme is ineffective. ajis vol. 10 no. 2 may 2003 145 providing a means of addressing complaints a specific objective of the bill is to provide a means of addressing complaints by australian residents or a body corporate that carries on activities in australia about certain internet content. on receiving a complaint about internet material the aba can require an internet service provider (isp) to take down x rated or refused classification material (rc) hosted onshore, and to take all reasonable steps to prevent access to x rated or rc material hosted offshore. in respect to restricted (r) classified onshore content the aba must be satisfied that restricted access arrangements are in place. no proposal is made in respect to offshore r classified material. the bill expressly specifies time frames for the take down process and penalties for non compliance. australia uses a classification system to determine the degree of censorship that is to be applied to a variety of offline media that includes films, videotapes, publications and video games. the classification board of the office of film and literature classification (olfc) undertake the classification. the state and territory governments also use this although some adopt a more rigorous interpretation. classification decisions apply criteria that are part of the national classification code. the most important “test” that determines a particular classification is the concept of a “reasonable adult”. such a person is defined officially as “possessing common sense and an open mind, and able to balance opinion with generally accepted community standards” (olfc 1999b). the bill envisaged the establishment of a community advisory body on internet content now known as netalert. in a media release minister alston (1999) said that “netalert will be responsible for researching new access management technologies and running national awareness campaigns to promote a safer internet for young people”. he saw netalert as “an additional measure that will play an important role by empowering parents to take control of the internet so that they can feel confident about their children going online”. the supporters of the bill claimed the legislation was in response to a major concern expressed by the community about internet pornography (sh 5218; hh 6907). during the debates the coalition claimed the bill had attracted little or no criticism from the public (sh 5218) and minister alston asserted that internet pornography is a major social problem for australia. indeed, some coalition members believe the internet is awash with pornography with the claim that it constitutes 60% of its content (ssc49; 73). internet industry representatives at the hearings reacted with incredulity to this claim (ssc 49; 73). no comprehensive study on community attitudes to internet pornography has been conducted in australia. both supporters and opponents of the bill provide anecdotal evidence to support their case (sh 5210; 5212). what empirical evidence exists suggests that internet pornography is not a major concern (ssc 260ff; 297299). however, few australian would support child pornography and minister alston (2001), in a media report cites a recent survey by australiascan that found 97% of australians are opposed to child pornography. it can be argued that the establishment of a formal mechanism to process complaints and publicizing that fact can act as a self-fulfilling prophecy. in other words the very act of establishing a mechanism to deal with a perceived social concern may create the social concern . sensational press stories about internet porn can also act to “manufacture” concern when it is not strong. restrict access by adults to offensive material the classic libertarian position is to treat freedom of speech as indispensable to the open society. if the government wishes to censor some type of speech, like pornography, then the onus of proof of demonstrable harm rests with the government. state intervention to censor pornography without empirically proven evidence of social harm is an illegitimate exercise of state power in an open society. during the hearings and the debates offensive and unsuitable material was almost invariable equated with pornography. surprisingly little was said about equating harm of the "reasonable adult" with feelings of being offended by an image and/or text. this is central to the governments system of censorship of any media. vital to a view on pornographic harm is the meaning and power of the pornographic image (sandy 2001). one view is that the pornographic image is a master text to practices it portrays. it does represent real (bad) attitudes and desires, and, it teaches the consumer these attitudes, and to expect to recreate these practices in reality. as indicated previously there is little empirical support for this view. many supporters of the bill accepted this view (ssc 108; sh 5212-5213). the opposing view is that the pornographic image is a signifying system and a fantasy scenario. what is portrayed is not the object of desire but a scenario in which certain wishes are presented. this may involve scenarios that are illegal as acts. fantasy is clearly a separate realm from reality and cannot be taken at face value. opponents of the bill tended to make this distinction an claimed the bill would violate the freedom of an adults (sh 5199;5207;5213). the representative of the australian computer society, for instance, spoke for many when he argued that the internet should remain an adult medium and should not be transformed into a media suitable only for children (ssc 72). this he claimed would be the effect of the legislation. ajis vol. 10 no. 2 may 2003 146 supporters of the bill asserted it did not suppress free speech (scc150; hh 6907; sh 5219). further, the coalition countered that the internet warranted no special treatment in regards to government regulation compared to the other media (sh 5219; hh 6970). what is censored offline is now to be censored online. however, this begs the question about whether censorship of other media is justified most research studies on pornographic harm are for media other than the internet. australia lacks empirical research in this area. as is common for any contentious area each group holding different views claim the research findings support their particular view. pornographic harm is no exception. sandy (2001) summarized these studies and they suggest there is no causative link between sexually explicit material and acts of violence (non consensual) or in a change of male attitudes (for the worse) towards women. this is whether the sexually explicit image is accompanied by violence or not. it appears that there is little empirical support to justify censorship of pornography in an open society on the grounds of it being harmful to adults. if this is accepted then this applies to all media and not just the internet. in regard to minors and pornographic harm there are no reported studies for obvious ethical reasons. in the case of images of pedophilia or images of incest with minors it is widely agreed that these be illegal because it is assumed that production of such images involve non-consensual acts. recently, the supreme court of the united states ruled that legislation that made morphed child pornography images illegal was contrary to the first amendment. efa claim that the scheme is one component of a wider strategy by the coalition and other conservative forces to curtail adult freedoms. the “evidence” cited includes: 1. tighter regulation that goes beyond pornography to adult themes, especially in regard to films (efa 1999) 2. failure to approve a change from x rated videos to non violent erotica (efa 2000c) 3. more restrictive treatment of computer games (olfc 1999a) 4. standardisation (more restrictive) of classification criteria used by the olfc (efa 2000b) 5. proposed “excessive and overboard” cybercrime bill 2001 (efa 2001a) 6. “tougher” telecommunications (interception) legislation amendment (efa 2001b) 7. proposed “profoundly flawed” new south wales internet censorship bill 2001 (efa 2002a) 8. proposed “profoundly flawed” south australian net censorship bill 2000-2001 (2002b) all of these examples pre-date september 11. libertarians are concerned that we now operate in a climate that is more conducive to legislating away adult freedoms. protection of children the issue of protection of children from unsuitable material was a dominant one during the hearings and debates. it remains the key issue in discussions about the operational effectiveness of the scheme. all parties agree that both parents (including teachers, librarians etc) and the government have a role in protection of children from unsuitable internet material. the main disagreement concerns the degree of freedom left to the parent to determine what material the child consumes. supporters of the bill argued that the government must be pro-active in doing all it could to assist parents and guardians in undertaking their parental responsibility. the coalition members argued that the scheme would support parents who lack the technical knowledge necessary to properly supervise their children’s access to the internet (ssc 51; sh 5219; hh 7977). much was made by them of portraying children as more technically competent than their parents, and because of this, are able to deceive their parents about their internet use. implicit in the arguments of some supporters of the bill was that it would counter irresponsible parents who did not properly supervise their children’s internet access (ssc 51; 157-158; sh 5219; 5212). the bill was considered necessary to protect children from their own (irresponsible) parents. coalition members at the hearings adopted a similar view towards library staff in regards to supervision of internet access by children in libraries. they suggested that library staff are too busy to be monitor breaches of policy, and given the tenor of questioning an implicit suggestion that the library policy on this matter borders on the irresponsible (ssc 192ff; 103). originally the government believed that unsuitable material could be successfully blocked at the isp or “backbone” provider level. those opposed to the bill argued that the best place to block content is at the receiver and not the isp or backbone provider (ssc 151ff; 243ff). the findings of a report by the commonwealth scientific and industrial research organisation (csiro 1999) were given prominence in the hearings and debates. it was treated as authoritative by both sides. the report considered blocking access at the isp or backbone provider level to be largely ineffective. instead it recommended the use of filtering ajis vol. 10 no. 2 may 2003 147 software by internet users. it evaluated the effectiveness of filter products and commented on their deficiencies. the coalition acknowledged the deficiencies of the filter products but looked forward to a future where technology had developed to such a state that it would make the bill more enforceable (hh 7978). until then the coalition responded by stating “the fact that some aspects of the bill are in the opinion of some in the industry, unlikely to succeed is not argument enough, for the government to ignore this issue and to allow the internet to go unchecked as a conduit to the world’s unsavory material for our children” (hh 7978). more recently the csiro (2001) has evaluated the effectiveness of 14 internet filtering software on behalf of the aba. it found that they vary considerably in their effectiveness in blocking targeted material. a common argument made by those opposed to the bill was that not only is the blocking software ineffective in protecting minors but the charge that the government is deceiving parents into believing the technology is trustworthy (hh 7970ff; ssc 257-258; 104). thus, parents are “lulled into a false sense of security” (ssc 245) and therefore “the bill could do more harm than good” (ssc 63). further, it was argued that the government by putting its faith in software that does not work results in a lost opportunity to educate and empower parents to take responsibility for supervision of the internet by their children (ssc 6914). the strongest claim about the likely ineffectiveness of the scheme was by industry spokespersons at the hearings (scc 63ff; 127ff) and of others in the debates (hh 7970; sh 5136). they argued that the legislation will not work because over 90 per cent of the content is hosted offshore. the supporters of the bill were also reminded of the ease with which domestic sites could move offshore to escape enforcement (hh 6913). some opposition members (sh 5199; 5210) and submitters to the hearings (ssc 48-49; 70) pointed out that the bill is not needed because the existing laws, especially the crimes act, are adequate to deal with illegal activity online. an industry spokesperson at the hearings succinctly put it “as far as we know there are no illegal activities that become less illegal on the internet” (ssc 209). however, the hearings were told that some uncertainty exists as to whether the crimes act does cover the internet (scc 123). the coalition indicated it would legislate to rid the act of any uncertainty (hh 6907). opponents of the bill stressed during the hearings and debates (ssc 244; 262ff;sh 5198ff) that a society, like australia, is composed of many communities of widely differing standards in relation to sexuality, gender relations and sexual expression. one of australia’s greatest assets is a social structure that is highly pluralistic. in reality, the “test” of the "reasonable adult" results in the majoritian norm being forced on all communities. in an open society like australia, speech that is deemed illegal or restricted should depend on “proving” its demonstrable harm. in the absence of this mere offensiveness to a group is no justification for censorship by the state. operational effectiveness of the co-regulatory scheme in an attempt to gather evidence of the scheme’s ineffectiveness efa in february 2000 lodged a foi with the aba seeking material banned and not banned under the scheme. it was seeking evidence to “prove our point that the system put in place will not work and cannot ever work. the whole system’s basic concept is flawed because most content on the net is hosted outside australia so there is nothing the government can do about it”(efa 2000a). six months later some documents were released by the aba. efa accused the aba of a “net cover-up” (jackson 2000). after examination of the released documents efa (2000a) made the following claims about the operational effectiveness of the scheme: 1. the blacked out urls were not only of banned pages but for papers not banned. 2. discrepancies exist between the information released and the aba’s first quarter report issued in april (aba 2000) and minister alston’s first bi-annual report on the scheme tabled in september (commonwealth of australia 2000). 3. the aba had gone beyond its legislature remit and made a value judgment as to what may harm if the urls for certain non-prohibited content are released. 4. the aba has acted on and prohibited content as a result of complaints received from non-australian residents. in media releases minister alston (2001a; 2000) claims the internet is safer for all australians because of the work of the aba of continual removal of illegal and highly offensive material on the internet particularly child pornography sites. he refers to the second and third bi-annual reports on the scheme (commonwealth of australia 2001; 2002). referring to the second bi-annual report he indicates that the aba completed 221 investigations of which 139 resulted in a location of prohibited or potentially prohibited content. he states “this represents a 49 per cent increase in the number of completed investigations which located such material in the first six-month period and demonstrates the scheme’s continued effectiveness in addressing public ajis vol. 10 no. 2 may 2003 148 concerns regarding the nature and accessibility of some internet content’. he believes “australian families will welcome the continued removal by the online content regulatory scheme of illegal and highly offensive material on the internet, particularly child pornography sites”. he also discusses the work of net alert in assisting industry to reach full compliance through the development of an information manual and a series of national industry compliance seminars targeting smaller isps. in addition he states that netalert and the aba are working on research and community education initiatives. specifically, netalert’ advisory services, tollfree national help line, information kit and website are mentioned. similar reference is made in the third biannual report. efa (2001c) and others claim that the scheme has failed to achieve its objectives. first, the aba has gone out of its way to portray the reported figures in a way that overstates the apparent effectiveness of the scheme. the aba’s internet content complaint form clearly states you can only complain about one item of internet content in each complaint form. however, the aba in the second bi-annual report indicates 67 “items” of prohibited content from 6 australian sites. in contrast 136 items of prohibited content were reported in respect of the 133 overseas sites. the third bi-annual report indicates that the aba issued 8 take-down notices to australian hosts covering 37 items of prohibited content. efa believes the aba is ramping up the “item count” in order to make the figures look respectable in relation to australian content. second, of the 67 items of prohibited content 45 items (child pornography) from 6 australian sites were referred to the state/ territory police. similarly, of the 37 items 23 that are child pornography were also referred. it would appear no successful prosecutions have been undertaken. efa claim most of these complaints relate to newsgroups. the australia institute (flood and hamilton 2003) believe that “not only the regulation of pornography on the internet is manifestly failing but the regulatory authorities themselves appear to have lost sight of their functions”. it claims the aba seems more concerned to promote the use of the internet than to protect children from its dangers. third, minister alston is criticized for “misleading political rhetoric” that grossly overstates the impact of the scheme. this lulls australian parents into a false sense of security. the claims made by him play on the emotive impact of the child pornography bogieman, without any supporting information about prosecutions of australians for publishing such material. fourth, efa estimates that the total cost of the scheme for the first 12 months of operation is $2.5m. and suggests the benefits are not cost effective. it suggests the impact in australia has been the cancellation of a number of newsgroups and a few adult australian web sites forced offshore (forbes 1999). further, it is claimed that netalert’s activities are equally questionable in terms of outcome. fifth, efa suggests that by encouraging australian citizens to report child abuse material to the aba, the process places complainants at risk of committing a criminal offence. the mere awareness of the existence of such material implies that it has been downloaded into the complainant’s computer. it is noted that there is nothing in the bill that grants immunity from liability for persons reporting illegal content. sixth, it is claimed that the aba fails to disclose what content has been removed and why. this is different from the treatment of offline content (taggart 2002). david flint, chairman of the aba counters by stating that the internet is different from the offline media and believes that making child pornography urls public will jeopardize the policing and legal processes (taggart 2002). recently, the administrative appeals tribunal (2002) ruled against efa and in favour of the aba. seventh, minister alston (2001b) claims that the success of the scheme will lead to an uptake of the new technology by families and this in turn will drive the growth of the new economy in australia. however, mcauliffe (2001) points out that the latest bureau of statistics figures show the number on internet subscribers in australia rose by 3.9 million at the end of the december quarter but there were 7,000 fewer household internet accounts and decrease in 307,000 free internet accounts. the bureau believes the decline is due to industry restructuring and isps going out of business. conclusion the key test regarding the operational effectiveness of the co-regulatory scheme is whether it has lead to significantly greater protection of children from unsuitable internet content. supporters of the scheme claim that this is the case and is evidenced by: 1. the issue of take-down notices and referrals of illegal material (child pornography) to the federal police of sites hosted overseas and to the state/ territory police of sites hosted in australia. 2. the work of netalert on research for example the technical evaluation of filters by the csiro 3. the work of netalert community education initiatives for example promotion of the australian families guide to the internet web site and presentations at various conferences. ajis vol. 10 no. 2 may 2003 149 opponents of the scheme claim the scheme is ineffective principally because overwhelmingly unsuitable content is hosted overseas and all the government can do if it is illegal is to refer it to the federal police. australian sites referred to the state/ territory police are few and there is no evidence of any successful prosecutions in relation to these. critics of the scheme remain unconvinced the scheme was necessary as such material was and is covered by the states/ territory’s crime acts. further they assert that parents may be lulled into a false sense of security believing the scheme to be effective in protecting children when it is not. at this stage the weight of evidence suggests the scheme has not afforded children significantly greater protection against unsuitable internet content. references alston r (2000) a safer internet for all australians, media release, 19 may. alston r (2001a) australia’s safer internet media release, 13 february. alston r (2001b) growing internet and computer use media release, 8 may. alston r (1999) internet content advisory scheme announced, media release, 26 november. australian broadcasting authority (2000) internet content complaints scheme – the first 3 months. http://www.aba.gov.au/about/public_relations/newrel_2000/27nr2000.htm date accessed 20 june 2000. commonwealth administrative appeals tribunal (2002) electronic frontiers australia inc and australian broadcasting authority, aata 449 (12 june 2002). commonwealth of australia (2002) a review of the operation of schedule 5 to the broadcasting services act 1992: issues paper, september. commonwealth of australia (2002) sixth-month report on co-regulatory scheme for internet content regulation january to june 2001 department of communications, information technology and the arts, february commonwealth of australia (2001) sixth-month report on co-regulatory scheme for internet content regulation july to december 2000 department of communications, information technology and the arts, april. commonwealth of australia (2000) sixth-month report on co-regulatory scheme for internet content regulation january to june 2000 department of communications, information technology and the arts, september. commonwealth of australia (1999) broadcasting services amendment (online services) act 1999. commonwealth of australia (1999) house hansard: broadcasting services amendment (online services) bill 1999 – second reading, canberra. commonwealth of australia (1999) official committee hansard: senate select committee on information technologies – broadcasting services amendment (online services) bill 1999, canberra. commonwealth of australia (1999) senate hansard: broadcasting services amendment (online services) bill 1999 – second reading, canberra. commonwealth scientific and industrial research organisation (1999) blocking content on the internet: a technical perspective. division of mathematical and information sciences, june commonwealth scientific and industrial research organisation (2001) effectiveness of internet filtering software, mathematical and information sciences, september. electronic frontiers australia (2002a) new south wales censorship bill 2001. http://www. efa.org.au/campaigsn/nswbill.html date accessed 28 march 2002. electronic frontiers australia (2002b) south australian net censorship bill 2000-2001 http://www.efa.org.au/campaigns/sabill.html date accessed 28 arch 2002. electronic frontiers australia (2001a) cybercrime bill 2001 http://www.efa.org.au/issues/security/ date accessed 28 march 2002. electronic frontiers australia (20001b) on-line surveillance http://www.efa.org.au/issues/privacy/surveillance.html date accessed 28 march 2002. electronic frontiers australia (2001c) regulatory failure: australia’s internet censorship regime http://www.efa.org.au/analysis/aba_analysis.html date accessed 26 october 2001. electronic frontiers australia (2000a) foi request on aba report on documents released/ denied. http://www.efa.org.au/foi/efa_foibarep1.html date accessed 9 september 2000. electronic frontiers australia (2000b) proposed changes to classification/ censorship act http://www.efa.org.au/issues/censor/classifrev01.html date accessed 15 april 2002. electronic frontiers australia (2000c) submission to the senate legal and constitutional legislation committee: the provisions of the classification(publications, films and computer games) amendment bill (no. 2) http://www.efa.org.au/publish/senate00.htm date accessed 6 may 2000. ajis vol. 10 no. 2 may 2003 150 electronic frontiers australia (1999) the net censorship dilemma: blinded by the smoke the hidden agenda of the online services bill 1999. http://rene.efa.org.au/liberty/blinded.html date accessed 17 april 1999. flood m and hamilton c (2003) regulating youth access to pornography, the australia institute discussion paper number 53, march. forbes m (1999) porn sites head offshore to beat law sydney morning herald 9 june. jackson d (2000) aba accused of net cover-up australianit 30 may. mcauliffe m (2001) oz home internet subscribers drop zdnet australia 1 may. office of film and literature classification (1999a) guidelines for the classification of computer games: amendment no. 1. office of film and literature classification (1999b) guidelines for the classification of films and videotapes: amendment no. 2. sandy g (2001) the online services bill: theories and evidence of pornographic harm. proceedings of the second international conference of the australian institute of computer ethics, 11-12 november 2000, australian national university, canberra. taggart s (2002) oz censor law still confuses wired news, 2 february. microsoft word ajis 11 1 final.doc ajis vol.11 no. 1 september 2003 76 product cost management structures: a review and neural network modelling jha p. p., glassey, j., montague, g.a. and mohan, p1 school of chemical engineering and advanced materials, merz court, university of newcastle upon tyne, ne1 7ru, u.k. phone: (+44) 191 2227275 email: pushkarjha@hotmail.com 1eli lilly and co., drop code tl 32, tippecanoe labs, post box 685, lafayette, indiana -47902, u.s.a address for correspondence: pushkar. p. jha 4045a cass business school 106 bunhill row london ec1y 8tz tel: +44(0)20 7040 8727 email: pushkarjha@hotmail.com abstract this paper reviews the growth of approaches in product costing and draws synergies with information management and resource planning systems, to investigate potential application of state of the art modelling techniques of neural networks. increasing demands on costing systems to serve multiple decision-making objectives, have made it essential to use better techniques for analysis of available data. this need is highlighted in the paper. the approach of neural networks, which have several analogous facets to complement and aid the information demands of modern product costing, enterprise resource planning (erp) structures and the dominant-computing environment (for information management in the object oriented paradigm) form the domain for investigation. simulated data is used in neural network applications across activities that consume resources and deliver products, to generate information for monitoring and control decisions. the results in application for feature extraction and variation detection and their implications are presented in the paper. keywords: activity based costing (abc), enterprise resource planning (erp), neural networks, self-organising maps, hopfield networks and object orientation. introduction product costing information has undergone numerous changes in the last few decades towards an increasingly strategic nature. in the data and process standardisation in enterprises this role is augmented by the appreciation of the aspect of process and inter-linkages, which has synergies with the developments in costing approaches. product costing structures are looking at activities as driving the cost and as the vital interface between resources (inputs) and products (outputs) (cooper & kaplan 1991). this issue is reviewed in the present contribution covering strategic approaches in product costing, including the object oriented paradigm that strives to improve product costing functionality and neural networks. neural networks are discussed as a technique that can prove useful in data modelling for information extraction. the advantages over conventional techniques, especially those of a non-parameterised approach and the comprehensiveness of this approach due to the wide range of properties and structures that can be applied on data, are also highlighted in this paper. the linkages across sub-systems within enterprises are appreciative of the influence each subdomain has on the other. the issue of modelling to accurately estimate and capture this influence however, is subject to constant improvement. the latest developments in modelling techniques were in the area of the multidimensional facets of systems. the growth in modelling techniques towards more comprehensive nature and the increasing use of intelligent systems to escape the limitations imposed by conventional statistics are indicators of this development. the basic issue in product costing is that of ‘estimation’ and ‘detection’ essentially for performance evaluation and control. these together with other multiple decision objectives are fed into strategy and planning. however, these areas have competing information demands that draw from various variable sets in product cost data. the accuracy, comprehensiveness and scope of techniques in analysis are thus critical. ajis vol.11 no. 1 september 2003 77 the applications for feature extraction and detection in this paper were carried out on simulated data using matlab. matlab handles multidimensionality found in the increasingly wider perspectives in costing approaches in a straightforward manner. expressions of external factors can be attached to approximate closely the real problem domain/ the real system within matrices that represent a ‘natural’ description of certain complex relationships. these relationships are always of concern in strategic planning, a key objective for which enterprises continuously seek better modelling solutions (sherwood 1983). cost information structures the costing systems are expected to provide information for numerous and sometimes competing decision objectives. advances in product costing to improve its characteristics as a strategic tool have made costing more open to implementation of uncertainty (influence of factors outside the firm’s control), non-linear influences and relevance of a wider temporal approach (costing is no longer limited to only the short run). “marketing, administrative, sales, distribution and deployment ‘events’ are considered to be cost drivers in the modern financial system” (dekker 2000) showing a growing complexity of processes and need for accuracy and comprehensive approach in associating costs to processes. also the modern day product cycle is characterised by products maturing over an increasingly shorter time frame and advantages due to innovations diminish quickly. the strategy implication of these characteristics is that of having a responsive strategy which responds quickly to changes in the dynamic market environment. the emphasis is on the process for both seeking competitive advantage through product and production strategies, and developing a response to a change in the factors that make a successful product. the erp systems have provided for process and data standardisation to aid this but generation of information out of these systems is subject to modelling techniques. thus the scope of neural networks in this regard is discussed in this paper after reviewing the growth in dimensions for product costing. it is essential to identify what a product is actually costing if business decisions are to be well informed. the cost allocation process is essentially a two-stage process as in the activity based costing approach. in this two-stage process separate cost centres or pools are created for each major activity. this is further traced using suitable measures or ‘cost driver rates’ to individual products. it is a manifestation of the fact that activities consume resources and product consumes activities (cooper & kaplan 1991). activities consume a variety of resources that are not always in relative or perfect proportion and the same applies for products consuming these activities. the trade-off problem between complexity (too many cost drivers) and loss of accuracy (too few cost drivers) is addressed in the comprehensive data and process standardisation in erp packages such as sap. pragmatic product costing, an extension of abc, tries to arrive at resource consumption by the processing time consumed for each activity. it works on the unit price of standard time. it is said to allow faster cost estimation as compared to process flows in standard abc (jiao & tseng (1999). however data and process standardisation, as in sap production planning modules, seem to address the needs of this approach and also any benefits it provides compared to the basic approach of abc. influences that are not covered in the ‘certainty’ of assumptions of traditional wisdom in costing have become more and more relevant over the years. with a growth in multiproduct nature of businesses, strategic approaches like ‘target costing’ complement the developments recognising the activity emphasis. here, the recognition of external market factors to plan costs also induces a ‘learning effect’ over time taking costing out of temporal bounds and giving it more of a strategic nature. ajis vol.11 no. 1 september 2003 78 in target costing one works from the selling price downwards to feasible cost levels. target costing is an approach made popular by the ‘japanese business strategy thinking’. until very recently the focus was limited to standard costing. the purpose of standard costing is on deviations between actual and expected results for a given manufacturing process. on the other hand, in target costing the target costs are set during the design phase and the reduction in standard costs is strived for to achieve the wider goal of target cost levels. additionally companies realise the best sources of cost savings before the product reaches the production stage (lagagne & discenza (1993). target costing is also said to improve motivation for performance, a manifestation of the modern view of integrating influences of management segments. the extension of costing during the customer revolution of the 1980s saw the reconstruction of customer as an investment, the customer being ‘chosen to invest in’ for returns. value to customer became more important in contrast to only the cost to customer in acquiring the product offered (boyce 2000). this was actually always there in any business strategy but the involvement of accounting practise in customer valuation presents another strategy dimension costing is supposed to provide information for. these objectives are ever increasing, as the business environment becomes more and more competitive. a competitive product must address factors such as cost, performance, aesthetics, schedule or time-to-market and quality as value to the customer. today, parametric estimating is usually applied to large systems that describe these factors (crow 1997). product cost modelling software most of the integrated software for modelling of product costs is subject to a very specific framework, which is developed to suit the needs of a particular industry. the basic data interface is always a step structure ‘or’ hierarchy of levels that have costs at the base level. these are then linked to the activity structure and cost pools to identify the cost drivers and subsequently allocation is performed. most models first begin by assuming the ‘nature of these levels’ based on an ‘industry type’ and then provide a cost estimation and forecasting technique. the data in these software packages is grouped across spreadsheets like for example in the four structures of resource, pool, activity and report (reed & goswami 1998). an example of this, but also the recognition of the need to extend modelling techniques, is the esa cost modelling software (ecom) developed for cost estimation in the area for space projects. ecom is able to escalate cost data to any given economic conditions (fatelnig 1997). to prepare reliable price estimates, the cost analyst ideally needs to know the product tree, the work breakdown structure and the work-package descriptions. the cost estimation uses a mix of approaches including resource consumption in work packages, analogy, using historical estimates, standard cost estimation functions and to some extent rules of thumb, essentially similar to use of expert knowledge or ‘the good guess’ (fatelnig 1997). very little application is reported in the field of product cost modelling using artificial intelligence and knowledge-based approaches. an example of this approach is seen in the building industry. combine2 within the joule programme of the european commission developed a prototype of a computer based integrated building design system. within this, there exists a costing tool that presents a model of life cycle costing of building design in different phases. the prototype tool for one of the phases of design has been programmed with the neural network application development software. all the cost effects have been cased by means of real building information or knowledge collected and calculated from real cases (lassila et al 1994). erp and the emphasis erp is popular in businesses of today due to increasing process efficiencies. “erps primarily automate the transaction processing requirements of financial, distribution and manufacturing business processes” (luettegen 2000). thus enterprise information and management packages like sap have become widely applied. all parts of the business (company, business areas, controlling area, operating concern) are linked together with common data access and real time information flow across management divisions for a better accumulation of and return on information. the prerequisite to sap is business process re-engineering, which implies a thorough analysis and coding of the sales, production, purchasing and finance and accounting events in a business process and how ajis vol.11 no. 1 september 2003 79 each action influences processes in the other segments (enterprise management technology 2000). this is essentially a well-defined networking to link business nodes by appropriate flows. the emphasis in enterprise resource management paradigms is analogous to the emphasis in costing i.e., ‘process’ and ‘tailored to enterprise need’. analogies also exist between these paradigms and the network based technique discussed later in the paper. these include networking, real time flows in information systems and also the growing need to eliminate the limiting assumptions in response to an increasing learning interface with environment. sap ag is the major erp package vendor for the fortune 500 companies. sap’s first two products operated on mainframe hardware; r/1 was batch oriented, then in 1981 it was replaced by an online system called r/2. sap has dominated the market with the help of its improved r/3 system in the last decade. the role of object oriented structure in enhancing functionality can be seen in the fact that r/3 currently contains over 1000 business processes that ‘have to be selected from’ for applications the object-oriented approach can reduce the complexity since in such design approach there is some sort of classification, that associates the methods operating on data structures and the system draws relationships across objects. functionality of different classes can be combined and objects stimulate each other to guide the system’s functionality as a whole. modifications are object based and not system based, allowing easy adaptability in adding new facets to the system. one can associate data and processes that operate on it to each object. systems, based on an object-oriented approach, are easier to understand because they are more closely related to reality. they have a small semantic gap between the model and reality. such systems are more flexible because they can be broken up into many small and independent entities (objects) that can be refined and evolved independently. because of this independence they can be used later in other similar designs, improving ‘code reuse’ (moreira & clark 1996). this relates to the efforts to make sap more adaptable/generic than it currently is and to improve its functionality. to illustrate this point, a cost object can hold information such as the name of the cost centre, the resource consumed, consumption rate, activity type, activity type prices e.t.c. (depending on how the overall classification is set up). it will have a set of operations to modify this data through operations of monitoring the activity and/or intimating uncalled for fluctuations in resource consumption with activity level. of course, it will also be connected to other cost objects, work in progress and inventory objects in the system. production order can be taken as the cost object, in effect it could be a class if different types of production orders were the reference point, (an object can be viewed in different ways depending upon the underlying relationships that constitute the structure). the data used to value manufacturing activity relies on master data in both the logistics and accounting modules. the product costing sub module lies in the sap set up for the production planning module where planning and control is carried out for manufacturing activities. an illustration of this is shown in figure 1. the figure gives an outline of the product costing module to illustrate the flows in a micro section of sap. clearly an object-oriented structure would resolve complexity by placing emphasis as much on ‘objects’ as on the flows between them. besides, the classification structure and associated properties allow for better automation and functionality in processing information the intelligent agent concept and data modelling solutions accurate information, accounting for anomalies in data structures and processing flexibility are always desirable. the intelligent agent concept extends the object oriented paradigm in to address there. the two distinct advantages in this extension may be noted as: � creation of smart business solutions, enabled but not necessarily driven by object technology. in these convergence to solutions despite inconsistency and anomalies in data or in case of incomplete knowledge is possible. � ability to extend the concept of information generation to knowledge and learning. (farhoodi & fingar 1997). the synergies can be summarised in figure2 ajis vol.11 no. 1 september 2003 80 figure 1: sap product costing sub-module 2 overhead (oh) cost center allocations from cost centre cost centre variances adm in oh allocation m achine oh allocation labour oh allocation m aterial oh allocation production o rder fin ished goods inventory raw materials inventory p r o d u c t i o n variances w ork in progress adapted (sap 1999) neural networks an overview in management science and operations research, neural networks have been applied to marketing, investment, banking, stock price prediction, property pricing, project management and exchange rate prediction besides other areas (sarle et al 1997). comparisons of neural network performance particularly with linear regression have been performed during the inception of interest in this domain for a wider investigation of product designs, process routings and cost data (zhang et al 1996) with results that are in favour of neural networks. in finance neural networks have been applied to the area of firm failure or bankruptcy prediction, where the emphasis is essentially on feature extraction. in solving classification problems for prediction of bankruptcy there have been results to suggest that though neural networks perform better than statistical models, their dependence on the nature of training data makes for an argument for their application in conjunction with statistical models (jain & nag 1997). this has already been tested in applications of non-linear partial least square regression using a neural network structure for other areas. this approach is being investigated in this research with applications on product cost data. other comparison studies have also been performed where these techniques were compared with the results from statistical techniques (zhang et al 1996). depending upon the data the results were mixed but neural networks performed as well or better than the statistical techniques in comparison. the emphasis is being placed on the ability of networks to identify patterns data (nonparameterised) which allows a better appreciation of external market factors. when these are taken 2 in following figure profitability analysis is not included which would then link into the other modules and further attachments would flow across the entire system. ajis vol.11 no. 1 september 2003 81 into perspective neural networks have been shown to have very good modelling potential. these results are for non-product cost financial data like market index forecasting (walczak 1999). product costs also involve this facet given the strategic approach of target costing (working from the selling price downwards) where market forces and expectations is a vital ingredient in decisions. this ‘external’ influence is also expressed in the ‘valuation of the customer’ in product costing decisions. examples of work in cost system design that make use of concepts similar to intelligent systems are given in the discussion on cost modelling software in this paper (fatelnig 1997). work on exploring the use of neural networks in product costing has been largely limited to research apart from some industry specific applications that have been mentioned earlier. feature extraction in the manufacturing process for costs has been explored in research using neural networks. this is for estimation during the design phase in keeping with the key idea pursued in life cycle costs, that of ‘design to cost’ so as to reduce unnecessary costs later on in the product life cycle (ou-yang & lin 1997). there are many kinds of neural networks. the developments in this modelling technique are marked by perpetual evolution. [24]. all the types vary in terms of the training methods they may use and further in terms of the learning law used, which comprises of a variety of characteristics (type of activation and learning rates). business process re-engineering pre-requisite to sap sap object oriented structures intelligent systems data & operations that create information improving functionality neural networks use of these techniques to carry out the operation synergies real time, parallel processing, non-parameterised, higher immunity to data anomalies,can incorporate varying properties to suit a particular domain in architecture and facilitated in an object oriented environment , can incorporate other intelligent systems to form suitable hybrids for carrying out the operation facilitate flexibility : inheritance, polymorphism: relational capacities figure 2: synergies for any application of nns the following properties need to be defined: � the topology (connectivity options) � the learning rate [learning made possible through attaching a local memory to the processing element so that results of previous runs can be used to modify weights. weights are adaptive coefficients that determine the intensity of the input signal (nelson, illingworth,1993)] � the approach to learning i.e. training � choices available in neuron functions of combination and then further using this as an activation function to allow outputs to vary with time before passing them on to the threshold or transfer function (a non-linear function, typically a sigmoid). ajis vol.11 no. 1 september 2003 82 these properties are not separate compartments of characteristics but particular sets of these are ‘complementary in selection of a model’. but what influences the selection? essentially it is the nature of the objective function, the nature of the variable set. contemporary research looks at how the use of neural networks can better capture the variability in costs i.e., feature extraction and detection. in their research, rohde and otzen of the copenhagen business school (1999), have discussed the abc approach and neural networks in conjunction. they have used terms of ‘abc networks’ and ‘full cost networks’ essentially based on the kind of input data used. the importance of the company’s cost allocation policy has been articulated and results indicated that such full cost networks are better at generalising and predicting than the abc networks. these results seem to be focussed on ‘calculatory precision’ and on ‘data issues’ in application of networks. in order to estimate of costs it is essential to accumulate activity information over the process. however, the information needs for decision-making needs may be very different to those for control. qualitative information viewed from processing the available data in a certain fashion may be required (borjesson 1994). for control purposes the variability in activity measures over time in a relative space and detection of unqualified deviations is important. it is this issue that is illustrated in the applications later on the present contribution. the better handling of feature extraction in neural networks compared to other techniques is established in numerous studies. the processing structures in neural networks throw some light on the reasons behind this besides the object oriented synergies that attach them well with the computational environment for information management and the outlook in sap. a very basic topology and learning algorithm-based classification of neural networks is as provided in table i. table i: basic classification of neural networks ajis vol.11 no. 1 september 2003 83 topology: feedforward: here the connections between nodes are not cyclical. due to this they provide a result faster relative to the feedback topology. feedback: here there are cycles in connections i.e. the network iterates for a relatively long time as compared to the feedforward set up before providing a response (past output included as current input). learning algorithm: supervised: the target values are given to the nn during training so that it can adjust weights (synaptic weights are ‘knowledge’ of the nn) to try and match outputs to these (performance parameterised by a tolerance level within which the output may be considered as good). there are two forms to this: autoassociative, where the target values are the same as inputs heteroassociative where the targets are different from inputs. neural networks can accept either categorical or quantitative data, supervised learning with quantitative data is called ‘regression’ (as estimating the best fit). a variant of supervised learning is ‘reinforcement learning’. here the total generalised error in the output is spread out over each incoming signal and the eligibility of update of a synapse is based on a probability relationship (of the output being correct). on the other hand pure supervised learning does not depend upon the updates in the last levels activation, it simply takes into account a specific synapse and does not take into account the global estimate of error. unsupervised: the nn is not provided with target values to match. these essentially perform data compression to reduce dimensionality. a form of learning that can be either supervised or unsupervised (implementable either through feedforward or feedback structures) is competitive learning. here there is a random specialisation, in the order of which output neuron produces the ‘closest signal’ to the target. to hypothesise the applicability, potential and synergy of neural networks one may look at the hebbian learning rule for neural networks. this postulates that if two neurons were very active at the same time the strength of connection between them would increase. this would imply that activities consuming resources could be mapped in a relative space. the ‘instar rule’ is a modification of the hebbian rule(to recognise an input) and it may be used in competitive networks of the som (selforganising maps: ‘feature extraction’) type. here weights of the winning neurons and neighbours both are updated (demuth & beale 1996). in other words activities consuming the resource would be ‘weighted’ in relative intensities. given more elaborate data, as in sap systems, comprehensive mapping of activities consuming resources could be performed for various decision objectives. the ‘data centered design’ of neural networks is in line with the object oriented approach (oo) that is popular in computing and automation today. the abstractness in classes and optimisation of methods that operate on the data in terms of their frequency of use has an analogy with generalisation and self-pruning respectively, in neural networks. the importance of the class interface and inheritance in such systems has synergies with the classification emphasis in sap, provides designing simplifications and also easier update as inherent in the object oriented approach. ajis vol.11 no. 1 september 2003 84 the extension of classes or their specialisation using the concept of inheritance and the property of polymorphism 3 are very analogous to the learning attributes and parallel structure of the neural networks. oos relate to the synergies of association, hierarchy, organisation and search across different intelligent systems like neural networks, expert systems and genetic algorithms. for example, an oo structure of the counterpropagation neural network (cpn, figure 3) indicates how a neural network system can be looked at under the object oriented system. here a particular property/ attribute to constitute the neural network structure can be called a data centered design where integration does not close options but allows for an ‘associative classification’ across objects and methods that operate on them. cpn attributes of cpn class k lvq g goe encoder lay er wins: no. of wins for each neuron outpu t of k inpu t to g noise range winners scaling figure 3: hierarchy and class in oo systems for cpns is a base ne ural net clas s *lvq and goe ment ioned in the figure refer to different training method s. (blum 1992) ‘the object-oriented paradigm has given a new dimension to integration technologies. it is leading to a faster and more reliable componentisation of sap (runge 2000). clearly having techniques of analysis that relate well with the generic nature of such systems shall ensure better efficiency. feature extraction using self-organising maps self-organising maps (som) are auto-associative, topological vector maps, which have possibly the largest applications among the basic neural network model types (backpropagation, hopfield, boltzmann machine, etc). these can be used in applications of classification and feature extraction, image processing and optimisation. it is the feature extraction property of som, which will be our area of interest in this section for use in analysis for product costing. som converts non-linear statistical relationships between high-dimensional data items into simple relationships expressed geometrically in a ‘relative mapping’. information is compressed preserving the most important topological relationships of the data items, in kind of abstract classes. given more elaborate data, as in sap systems, a comprehensive mapping of activities consuming resources for temporal comparisons and visualisation of the relative spread of activity intensities can be made available through use of soms. the application of feature extraction to product cost can be much wider than the activity analysis covered by this contribution. feature extraction based on the illustration here can be easily hypothesised to hold potential for the ‘design to cost’ concept and ‘value analysis’, both attributes of the discussed strategic role of costing in the businesses of today. ajis vol.11 no. 1 september 2003 85 self-organising networks are different from a classifier system where boundaries between known categories are adjusted (like in learning vector quantizer networks) by not relating to classes but unknown clusters (unsupervised). it is a mix of competitive learning and neighbourhood influences that constitute the som. before interpreting the application results, there is a need to discuss further the variety and the property selection issues in som and the underlying process. among other issues the number of neurons used for feature extraction and also the neighbourhood distance function to be used have to be selected. the som grid could theoretically cater for any number of samples and be as large as required. setting the number of nodes to approximately the number of input samples is generally acceptable when data sets are relatively small. the issues of accuracy for som show that neural networks are influenced by distributional assumptions, as when the map is to be used for new data, accuracy is assured only when the new data approximates the same distribution as the data set used in learning. distance functions used can also vary; layer distance functions are used to find distances between neurons in a layer. a combination of this with euclidean distance weight function (applying weights to input) can also be used. during the training phase, the som strives to approximate the density of the data. the reference vectors in the network weights move to the areas where the density of the input data is high. eventually, only few weight vectors lie in areas where the input data is sparse. in the learning process, sample input vectors selected at random are presented to the som. the weight vectors are updated based on the similarity between the two using the distance function. as the learning converges the neighbourhood radius (space) also reduces. the precision of the mapping is derived from the ‘average quantization error’, a measure of the accuracy of neuron responses to the given data set. the second measure of precision is that of topology preservation. this deals with the map structure and essentially how much the process has to twist the data set for the mapping (hodju & halme 1999). case study an activity record set was used in terms of activity measures. seven such activities driving the cost over a large set of multivariate columns were simulated within the matlab environment. the rows/drivers (input vectors) were as follows: a: set up hours; b: number of set ups; c: material handling hours; d: number of purchase orders; e: number of hours in delivery runs; f: maintenance hours g: inspection hours. these were taken as indicative of a production process for a bulk production process. the objective here was to use this in feature extraction to investigate the spread in intensity of the activities in a relative space and over time and to hypothesise the utility of this information. the weight graphs (figures 4, 5) show the intensity of each activity in terms of the update that the neurons have undergone. for the data set described above the best network topology was found to use six neurons. choosing a lower number of neurons essentially makes the activation less visible as can be seen in the weight graphs corresponding to the graphs showing the number of neurons chosen to constitute the som grid (figures 6,7). ajis vol.11 no. 1 september 2003 86 clock wise from top left: fig 4. six neuron feature space; fig 5. activation of the seven drivers; fig (s) 7 and 6 for three neuron feature space and activation, respectively however, since in sap there is already data and process standardisation, there will be records for all activities. thus there is no need to reduce to a suitable, smaller number of pools. so feature extractions for this purpose ‘may be obsolete’ given that sap is already in place. the argument to use this for ‘optimising sap’ rather than use it in the business process re-engineering has again a serious drawback. if some activity records were to be excluded, which now show a high degree of overlap with others, it would be impossible to map changes in intensity of activities overtime. however, for this purpose feature extraction could be of great value. if the data set was broken into four parts a change in pattern of activity intensities over time can be observed.. this particular approach in using feature extraction can be very useful in identifying rise or fall in intensity of activities in a relative space. use of the ‘outstar function’ (here weights are transpose of instar), could give insightful information. the analyst can investigate the impact of changing one of the activities. the outstar produces a modified prediction vector of remaining activities, accounting for changes in the investigated activity. 1 2 3 4 5 6 50 100 150 200 250 300 1 2 3 50 100 150 200 250 300 80 100 120 140 160 180 100 120 140 160 180 200 w(i,1) w ( i, 2 ) 100 120 140 160 180 100 120 140 160 180 w(i,1) w ( i, 2 ) number of neurons number of neurons v ar ia b le /a ct iv it y i n te n si ti es v ar ia b le /a ct iv it y i n te n si ti es ajis vol.11 no. 1 september 2003 87 a = 1 1 1 1 1 1 1 1 1 1 1 -1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 -1 1 1 1 1 1 1 1 1 1 1 b = 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 -1 1 1 1 1 1 1 1 c = 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 -1 1 1 1 1 1 1 1 d = 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 hopfield equilibrium points and detection of deviations hopfield networks are auto-associative structures with single layer lateral connections. here encoding is performed simply by adding the product of activation at both ends of the synapse to the weight stored. training is performed by presenting a pattern to the network and then letting it cascade back and forth till it stabilises to a steady retrieved pattern. an expel application of hopfield networks is the optimisation of the famous travelling salesman like problems. alternatively, the hopfield algorithm can be applied for linear programming problems. however, in the case of highly erratic non-linear mapping they are outperformed by other network models like backpropagation [17]. in some ways it can be argued that hopfield networks do not really learn. however they can be used for classification and detection in deviation from equilibrium in their basic form by the storing of equilibrium points. this is what is explored in the following application notes for product cost systems. the objective here is to detect process anomalies, taking activity vectors as inputs over time. this is performed using the hopfield networks to store a stable multivariate vector and then allowing the recall of the network (when presented with similar patterns) to identify unqualified relative variations in a particular activity in a given time period. subsequently similar sized sets are fed in to see whether the pattern was stored. it can be seen in the figures: 8 to 11 that the output corresponding to each of these sets marks out clearly the point of non-conformity in the activity consumption pattern. this can be very useful in spotting the exact point of nonconformity (unqualified variation). ajis vol.11 no. 1 september 2003 88 0 2 4 6 8 -1 -0.5 0 0.5 1 0 2 4 6 8 -1 -0.5 0 0.5 1 0 2 4 6 8 -1 -0.5 0 0.5 1 0 2 4 6 8 0 u n q u al if ie d v ar ia ti o n variable/ activity measure variable/ activity measure variable/ activity measure variable/ activity measure u n q u al if ie d v ar ia ti o n figure(s) 8,9,10,11 unqualified variation detection. clockwise from top left: set a, set b, set c, set d here since the data set of activities is being used, it can be seen that for the set ‘b’, an unqualified variation occurred for activity measure (a: set up hours; b: number of set ups; c: material handling hours; d: number of purchase orders; e: number of hours in delivery runs; f: maintenance hours g: inspection hours). ‘f’ (row 6) in the time period 7 (column 7). in set ‘a’, there are two such events and one in set ‘c’. for set d there are no unqualified variations implying that the consumption of activities in production is within the learned patterns. the same detection approach can be used for a different variable set where identification of unqualified variations is needed. conclusions product costing has taken a more process based outlook and a strategic role as is inevitable in the existing business environment where products mature very quickly and advantages of innovations are washed away very quickly. the need to monitor, control and optimise for product costs has thus found expression in fertile approaches that allow better organising and planning. specifically these include life cycle costing, target costing, activity based costing and industry specific software. in the integrative outlook to management in erp application software like sap, there exists a potential for ‘more expressive’ and non-parameterised data processing, to improve the quality of information and utilisation of increased data interfaces available. data modelling capacities of neural network, as nonparameterised systems, have a better reach in terms of capturing and expressing information by identifying patterns in data. the growth in concepts and systems in product costing within the integrated information structures have been very much parallel to the growth in modelling techniques of neural networks. the integrated information structures implied are two fold. first relates to the enterprise effort at achieving the best decisions through generating exhaustive information in real time and monitoring ajis vol.11 no. 1 september 2003 89 processes for efficiency. the second relates to the computational issues that are dominated by the object-oriented paradigm that promises solving the complexity and improving the functionality of erp structures like sap. the approach of neural networks is analogous and thus hypothesised to be adaptable for increasing functionality in the process emphasis and computational environment. the information needs to serve the multiple strategic decision objectives can be effectively provided by neural networks as discussed in application illustrations and also in discussion on numerous studies that have established their equivalence if not superiority to conventional statistical techniques. this is in light of both the synergetic and quality of output advantages that they have in relation to the conventional techniques. the future work will concentrate on a comparative analysis of conventional statistical estimation with neural networks and will include a discussion on and application of neuro-fuzzy hybrid systems. references blum, a. (1992) neural networks in c++: an object-oriented framework for buildingconnectionist systems, new york: wiley,, pp 6-55. börjesson, s. (1994) what kind of activity-based information does your purpose require? two case studies. int jnl operations and production management14(12), pp 79-99 boyce, g. (2000) valuing customers and loyalty: the rhetoric of customer focus versus the reality of alienation and exclusion of (devalued) customers. critical perspectives on accounting, vol. 11. no. 6, 649-689. cooper, r. and kaplan r., s. (1991) the design of cost management systems, text cases and readings, pp.3-7, 147-155. crow, k., drm associates. (1997) value analysis and function analysis system technique. http://members.aol.com/drmassoc/qfd.html dekker, s. management accounting for the year 2000. http://www.dtrakker.com/articles/frame_y2kmgmtacct.htm. demuth, h., and beale, m. (1996) neural network toolbox. mathworks inc, pp. 8-10 8-19. enterprise management technology.(2000) kania school of management. http://academic.uofs.edu/faculty/gramborw/sap/saptutorial.htm. farhoodi, f., and fingar, f.(1997) distributed object computing, "doc" magazine. http://home1.gte.net/pfingar/docmag_part2.htm (oct, nov 1997). fatelnig, k. p. (1997) esa cost modelling, a general description. http://www.estec.esa.nl/eawww/ecom/article/ecom.htm. hodju, p., and halme, j. (1999) the som algorithm. http://koti.mbnet.fi/~phodju/nenet/selforganizingmap/theory.html . jain, b., a., and nag, b., n. (1997) journal of management information systems, vol. 14 no2, (1997), pp201-217. jiao, j., and tseng, m. m. (1999) a pragmatic approach to product costing based on standard time estimation. international journal of operations and production management, (uk), vol. 9 no. 7, pp 738-756. lagagne, m., and discenza, r. (1993) new product costing japanese style-cpa in industry. the cpa journal http://www.nysscpa.org/cpajournal/old/14345315.htm. lassila, k., pallas, j., pulakka, s., and lee, s. (1994) a comparison of neural networks, non-linear biased regression and a genetic algorithm for dynamic model identification. international chemometrics internet conference (incinc'94). (1994). luettegen, r. (2000).erp/ package implementation: methods and considerations. enterprise software group, technology and business integrators, inc., woodcliff lake, nj. mcauley, d. (1997).the back propagation network: learning by example http://www2.psy.uq.edu.au/~brainwav/manual/backprop.html moreira, a.m.d., clark r.g. (1996). adding rigour to object-oriented analysis, software engineering journal, vol. 11, pp. 270—280 otzen, d., and rohde, c. (1999) (copenhagen business school) product design and cost estimation using neural networks a management accounting perspectivemanufacturing accounting research conference , university of southern denmark. ajis vol.11 no. 1 september 2003 90 ou-yang, c., and lin, t. s. (1997) developing an integrated framework for feature-based early manufacturing cost estimation. int jnl advanced manufacturing technology 13(9), 618-629. reed, j., and goswami, s. (1998) object-oriented integration in sap the bapi. http://www.softwarejobs.com/sap/sap88.html . runge pty ltd., xerax cost modelling software. http://www.runge.com/xeras/howwork.htm (1991-2000) sap labs, inc. sap ag . (1999). product costing scenarios made easy 2-1 to 2-5. http://wwwtech.saplabs.com/other/product_costing/02%20chapter%202.pdf sarle, w.s., wise, b. m., holt, b. r., and gallagher, n.b., ed., (1997)neural network faq, part 1-3 of 7 (introduction, learning, generalisation and applications): periodic posting to the usenet newsgroup comp.ai.neural-nets, url: ftp://ftp.sas.com/pub/neural/faq.html sharda, r., and rampal, r. (1998) neural networks and management science/operations research: a bibliographic essay. encyclopaedia of library and information science, vol. 61, supp. 24, , pp. 247-259. sherwood, d. (1983) financial modelling, a practical guide, gee and co. (publishers) ltd, pp 371-376. walczak, s. (1999) journal of management information systems. vol. 16, no2, pp 177-195. zhang y.f., fuh j.y.h., and chan, w.t. (1996) feature-based cost estimation using neural networks for packaging products. computers in industry, vol. 32, pp. 95-113. microsoft word ajisfinal 12 2 marina.doc ajis volume 12 no. 2 may 2005 79 “hybrids” and the gendering of computing jobs in australia gillian whitehouse chris diamond school of political science & international studies the university of queensland st lucia, queensland email: g.whitehouse@uq.edu.au abstract this paper presents recent australian evidence on the extent to which women are entering “hybrid” computing jobs combining technical and communication or “people management” skills, and the way these skill combinations are valued at organisational level. we draw on a survey of detailed occupational roles in large it firms to examine the representation of women in a range of jobs consistent with the notion of “hybrid”, and analyse the discourse around these sorts of skills in a set of organisational case studies. our research shows a traditional picture of labour market segmentation, with limited representation of women in high status jobs, and their relatively greater prevalence in more routine areas of the industry. while our case studies highlight perceptions of the need for hybrid roles and assumptions about the suitability of women for such jobs, the ongoing masculinity of core development functions appears untouched by this discourse. introduction a recurring theme in debates over the representation of women in computing professions has been the idea that opportunities for women would increase with the maturation of the it industry and the need to supplement technical creativity with social and organisational skills. as woodfield (2000, 2002) notes, problems in the 1980s and 1990s were often attributed to over-reliance on technical “wizards” in the drive for innovation, and a consequent lack of communication and management skills that hindered the ability to meet project deadlines and respond effectively to the needs of end users. the industry of the future, it was argued, would need multi-skilled managers with “peopleoriented skills” as much as technical expertise (morris 1989), with social aspects of technology such as user friendliness essential to profitability (see fountain 2000: 57). the optimal computing professional, therefore, was not simply a wizard or geek, but a “hybrid” able to bridge technical and social/management skills. the (expressly essentialist) extension of this position was that it was “women’s” skills that were necessary, and that women would make the best hybrids. this was seen as providing a “golden opportunity” for women to enter professional it occupations (deakin 1984), and the basis for a “second wave of optimism” about prospects for women in the industry (see woodfield 2000). regardless of the “truth” of essentialist assumptions about women’s skills (that is, in terms of whether, on average, women do have better communications skills, or make the best hybrids, or – more fundamentally – are less likely to be technical wizards), it is an empirical question as to whether recruitment and training practices have been influenced by a perceived need for hybrid roles, and if so, whether women are choosing to take up such positions. additionally, even if there is evidence of movement of women into educational programs and jobs that might be classed as “hybrid”, a further empirical question is how such skills are valued in the industry, and the extent to which this occurs along traditionally gendered lines. cockburn’s (1983, 1985) work on the ongoing processes of gendering skills continues to remind us that the reshaping of occupations and notions of skill in response to technological and other influences usually happens in ways that advantage men. thus a broadening of skills in the it industry could just as easily consolidate gendered divisions as ameliorate them. in particular, an extension of hybrid roles could allow the “unadulterated and prestigious” aspects of what grundy (1991: 356) has referred to as “abstract computing” to remain a ajis volume 12 no. 2 may 2005 80 masculine domain, while women move into the “messy” side of it work, or take up the more user oriented tasks that tend to be devalued as “soft computing” (fountain 2000: 57, mahoney and van toen 1990:320-2). this paper seeks to extend analysis of these possibilities, adding recent australian research to the existing body of evidence. our goals are relatively modest. we do not address fundamental questions about the social processes that shape the relationship between masculinity and it (as in, for example, wajcman 1991, webster 1996) or assess notions of gendered skill differences such as turkle’s (1984) comparison of “soft and hard mastery”; rather we simply examine evidence on employment patterns and organisational discourse relating to hybrid roles in computing employment. we adopt a combination of methods, drawing on survey research to examine the presence of women in occupational roles that most closely correspond with the notion of “hybrid” and elaborating this picture with qualitative analysis of a set of organisational case studies that examine the discourses around recruitment and valuation of different types of skills. such an approach is consistent with a feminist perspective (brannen 1992, chapman 1995, stoker 1995), providing both context and depth (greene et al. 1989), and has the advantage of reducing the influences of weaknesses inherent in any one method. findings from the survey research are reported first, with the rest of the paper devoted to the organisational case studies. hybrid roles: evidence from survey research the prevalence and feminisation of hybrid roles is illustrated here using data from a commissioned survey of employment patterns and pay rates for males and females in selected computing roles. the survey was undertaken in november-december 2003 by classified salary information services (csi), a private consultancy company specialising in monitoring market trends in pay for large it companies operating in australia. one hundred and eight organisations were contacted, and responses were obtained from 77 companies, ranging in annual turnover within australia from $2 million to over $5 000 million, and together employing over 63 000 staff. specific information was collected on 12 706 employees working in 106 designated occupational roles. this total included regular full-time employees and long term contractors, but excluded part-time, short-term contract, casual and expatriate staff. the sample was not representative of the it sector or computing professionals as a whole – as well as excluding some types of employees (in particular short-term contract staff who may be very highly paid), it did not include smaller it firms or organisations in which it was not the primary function. however, it did provide a comprehensive picture of regular employment in large it organisations, with its main advantage being the fine level of occupational detail available within these specialised firms. this detail is far superior to the occupational categories in the australian standard classification of occupations (used in australian census and labour force survey statistics), and it enabled us to focus on a number of different occupational roles that could be labelled “hybrid”. a complexity not elaborated in the introductory section of the paper is the lack of specificity about what exactly constitutes a hybrid role and the range of possibilities that can be envisaged, from highly paid professional jobs involving specialised design for individual clients to more routine tasks such as technical writing. the three sets of occupational roles we examine below are situated at different points on an innovation/support spectrum of tasks, as well as illustrating different points on a management hierarchy. they are: consultants (required to combine technical and business expertise in the delivery of individualised systems to clients); software project managers (required to have a detailed knowledge of developing technologies and user needs, plus oversee projects and manage staff); and technical writers (required to present technical information in a manner accessible to end users). in elaborating the occupational roles within these categories, we draw on ajis volume 12 no. 2 may 2005 81 csi’s position descriptions, which have been developed in consultation with the it companies that comprise its clientele. table 1 presents information on occupational roles within the first category – consultants. the position descriptions used by csi define the role of consultant as involving the development of industry specific systems services for clients, evaluating clients’ business needs and liaising with customers during installation and testing. consultants are expected to have a blend of “technical, commercial and consulting skills” and although an it degree may be part of their qualifications, many also have business qualifications, and at the higher levels, an mba. at the highest level, employees are expected to have around 20 years experience. levels of remuneration (represented in the third column of the table as the average nominal base salary of employees in each role as a percentage of the average nominal base salary for employees in the job role “programmer”) are high at the top of the scale, reflecting the importance of this role in shaping core business as well as maintaining client linkages. this is a conservative estimate of relative pay – if the ratio is based on total remuneration costs rather than base salaries, those at the top rung of this job ladder are earning over 400 per cent of a programmer’s average remuneration. occupational roles career level % female average pay relative to programmer n (employees) consultants associate consultant 2 31 105 346 consultant 3 31 150 1135 senior consultant 4 19 203 594 principal consultant 5 23 214 449 senior principal consultant 6 19 273 326 associate consulting director 6 15 371 115 table 1: consultants – female share and relative pay in large it organisations, 2003 while we are unable to gauge from our cross-sectional survey data whether the proportion of consultants has increased over recent years in these organisations, we can estimate levels of representation of women in these jobs in 2003. as column 2 in table 1 shows, although women make up close to one-third of the first two career levels represented in the table (compared with 22 per cent of employees in our sample as a whole), they comprise only around one-fifth to one-quarter of the next three levels, and only 15 per cent of the top rung. overall, the data suggest that although these jobs are consistent with the notion of hybrid, the upper echelons are the well paid managerial jobs usually associated with unbroken career patterns – that is, the types of jobs in which women are consistently underrepresented. information on the second group selected for examination – software project managers – is presented in the upper half of table 2. position descriptions for these jobs emphasise the need for a high level of technical expertise in software design, as well as experience in “people management” and an ability to understand the needs of users. the senior management position may also involve control of budgets and the recruitment and training of staff. as table 2 shows, these are high level career positions (from level 4-6), with relative pay ranging up to about two and one-half times that of a programmer. ajis volume 12 no. 2 may 2005 82 occupational roles career level % female average pay relative to programmer n (employees) software project managers project leader 4 32 145 523 software project manager 5 22 190 166 senior software project manager 6 15 249 79 software developers programmer 2 29 100 345 analyst programmer 3 27 143 909 senior programmer 3 22 158 778 system analyst 4 28 139 224 table 2: software project managers and software developers – female share and relative pay in large it organisations, 2003 the lower half of table 2 illustrates the group of software development roles beginning with programmer that are likely to provide the stepping stones into project management positions. in contrast with the software project management roles, the development roles are normally associated with “pure” or “hard” computing, although the move into “analyst” positions will involve some of the oversight and needs assessment skills associated with the notion of hybrid. in contrast to expectations about women’s entry to hybrid roles, the pattern is one of relative concentration of women more routine technical jobs – as illustrated in table 2, column 2, women make up a little under 30 per cent of entry-level programmers. from this position it appears they are slightly more likely to move into analyst type roles than senior programmer (which delivers the highest pay levels on average amongst the group of software development job titles in spite of not being rated at the highest career level), but are even more likely to be found in project leader positions. however, in a manner very similar to that illustrated for consultants, the proportion of women in project management roles falls sharply with steps up the career levels. the final hybrid category we examine is technical writing – obviously located more towards the “support” end of an innovation/support continuum and included under the umbrella of software project support in the csi classification. these jobs involve considerable technical knowledge as well as an ability to present information that is easily accessible to users, for example in manuals and online tutorials. the need for sophisticated communications skills distinguishes them from the more technically oriented test analysts that also fall within the broad project support group. table 3 shows that technical writing jobs are rated at a higher career level than test analyst jobs (although we note that highly skilled private contractors can earn extremely high salaries doing advanced level testing work – see grey and healy 2004), and this is similarly the case for the “senior” role in both groups. unfortunately, low numbers in the technical writer roles mean that the results for female share and average pay in these jobs may be unreliable, but there is a strong suggestion that they are jobs more likely to be performed by women, particularly at the initial level of technical writer. however, relatively high female share is not limited to the roles most closely linked with the notion of hybrid. the lower panel in table 3 also shows that the proportion of women in test analyst roles is relatively high (47 per cent compared with 22 per cent for the survey as a whole) and that women are more likely to be represented in the top career level of quality manager than they are in the equivalent career level positions for the jobs shown in tables 1 and 2. it also shows that the pay increment for quality manager is lower than for the equivalent career level jobs in the earlier tables, although relatively small numbers in this group mean the figure should be treated with some caution. ajis volume 12 no. 2 may 2005 83 occupational roles career level % female average pay relative to programmer n (employees) software project support technical writing technical writer* 3 82 113 22 senior technical writer* 4 41 146 17 testing and quality management test analyst 2 47 105 66 senior test analyst 3 41 127 78 quality manager* 5 32 172 28 table 3: software project support roles – female share and relative pay in large it organisations, 2003 *note: small numbers reduce the reliability of estimates for these occupational roles overall, these findings are consistent with a traditional picture of labour market segmentation, and echo patterns found in studies of computing professionals in other countries (see, for example, panteli et al 2001). our focus on hybrid roles suggests that those most likely to be taken up by women are at the more routine or support end of the it job spectrum, and that women’s representation in management roles decreases significantly with seniority. although the survey cannot inform us about trends such as the extent to which hybrid roles are becoming more prevalent or drawing more women into the industry, it has provided a background for the case study research and emphasised the need to consider a variety of hybrid roles within computing work. the following section of the paper turns to a qualitative analysis to try to uncover perceptions about the need for hybrid jobs, their suitability for women and the value of hybrid skills. hybrid roles and gendered skills: case study evidence the evidence drawn on in this section of the paper was gathered in a series of organisational case studies conducted in 2003-4. although our research spanned both public and private sectors and included organisations in which it was not the primary function, in order to be consistent with the population of organisations on which our survey data was based we focus here on evidence from four specialised it companies. all four were private sector companies selling it services and/or software. while they were all engaged in similar tasks and so employed staff in similar jobs, the scale of each organisation and the local product market in which it operated differed, allowing for useful comparison and contrast. table 4 provides information on the number of employees and types of jobs in these four organisations, as well as the labels we will be using to identify them in the following analysis. we note that the job titles listed will not match exactly with those used in the previous section of the paper – one of the ongoing difficulties of research in this field is the varied terminology used in different firms and by different data collection agencies. the problem is addressed as far as possible by elaborating the nature of the jobs we focus on in the course of the qualitative analysis. ajis volume 12 no. 2 may 2005 84 org. workplace size (employees) it jobs* it1 1131 consultants, support analysts, software developers, internal it consultants it2 266 technical analysts, programmers, testers, business analysts, developers, consultants it3 42 professional services consultants, software architects, programmers, technical consultants, analyst programmers, project managers it4 25 developers, database administrators, system analysts, web designers, it security, network administrators table 4: case study organisations *note: jobs titles are those used by the organisation themselves. information about the job tasks would indicate that similar roles have different names, and different roles have similar names, in different organisations. information from these organisations was gathered from senior managers, human resource managers and it workers themselves, using interviews and focus groups conducted according to semistructured interview schedules. these covered a broad range of issues including qualifications and skills, recruitment, training and careers, work organisation, pay, working time and flexibility, work/family, work regulation and organisational culture. the semi-structured format meant that the sessions were less question-and-answer and more like group discussions where respondents reflected on recent experience and interacted with each other. the sequence of questions, apart from initial introductory questions, was not crucial and most topics could be pursued as they arose. at it1, a senior hr manager was interviewed and then a focus group was held with seven staff. a senior hr manager and the senior it manager were interviewed at it2. another interview was then held with a line manager who had a long association with the company. a focus group of nine it staff was then undertaken. at it3 and it4, both of which were smaller companies, the researchers interviewed the md/ceo and then held focus groups with employees – eight and five it staff, respectively. each session was recorded and transcribed verbatim before being rechecked by one of the interviewers. the transcripts from these interviews were analysed using the miles and huberman (1994) thematic conceptual matrix. this analysis was initially undertaken manually, but the transcripts were subsequently re-examined with the assistance of nudist 6. our focus was on gendered perceptions of skill, particularly communication and people management skills and the extent to which these might be seen as “soft” skills in comparison with “pure” technical roles. the findings are reported in two sections: the first focusing on women and hybrids; the second on men and “pure” computing roles. the analysis utilises direct quotes as appropriate, setting these in organisational context and noting possible reasons for differences among the four cases. the organisations are labelled it1-4 (see table 4), and our citations give the sex and position (manager or employee) of the interviewee. page numbers refer to the typed transcripts. perceptions of women’s skills and hybrid roles the case studies provided examples of women in a wide range of hybrid roles from senior management to technical writing. in three of our four cases we interviewed a female senior manager, and while the jobs these women held did not directly involve technical work, the importance of a technical background in achieving a management position was often noted – as the female md of ajis volume 12 no. 2 may 2005 85 it3 explained, having a programming background provided her with credibility she would otherwise lack as a woman in the industry. in general, women were seen as most likely to progress through management rather than technical roles, “it’s not that they don’t have the technical capabilities but they seem to progress to management as opposed to technical specialists” (female manager, it2: 8). women in these organisations, however, faced the usual barriers in career advancement: “it’s the old boys’ network” (female employee, it1: 17); “…what i see happening is that it’s men in [it1] who are setting the requirements for people to aspire to executive roles, and are setting them according to a male set of criteria ” (female employee, it1: 14). at the other end of the it job spectrum, and in line with the data presented in the previous section of the paper, support roles such as technical writing were predominantly female. this hybrid role linking technical expertise with communication skill and a focus on end users, was seen as important because: … the big problem with the programmers is they write all the stuff down and it’s in programmer gobble-dee-gook and someone reads it and goes, “oh! i don’t quite understand that”. so they [the technical writers] basically take that and turn it into something that [can be understood by customers who can then] do their own configuration. (female manager, it2: 6) however, these jobs were seen as isolated from the “main business”. the only part-time worker at it3 was a female technical writer and it was explained that her role could be part-time because “her job is quite isolated” (female manager, it3: 18). another job in these organisations that linked technical and communication or administrative skills at the support end of the spectrum was database administration, and this was clearly seen as the kind of job likely to be performed by women: they keep the operational wheels turning … it’s a little bit tech, but not really tech. it’s sort of operational. so yes, more females there … they’re probably more conservative people. you know they’ve got families, they’ve got houses, they’ve got commitments. (male manager, it4: 14-15) the most positive ideas about hybrid roles that would offer more than routine jobs or support role opportunities for women emerged in discussion of “analysts” – a term used somewhat differently across the organisations to encompass varying degrees of proximity to the technical aspects of the software development process, although typically distinguished from the less technically engaged “business analysts”. in an echo of the literature cited in the introduction to this paper, the female manager we interviewed at it3 predicted an increasing representation of women in the industry in analyst roles: i think the importance in the analyst role will grow and grow and grow ... i think 20 years ago the it industry was full of men, and they decided that they knew best and people would get what they would give them because this was best for them to do … that world has changed. the end-user is much more sophisticated now … so the role of analyst, which i believe women are much better at, is a strong role. (female manager, it3: 16-17) in this manager’s view, women were innately suited to such roles – “i think this is a natural job for a woman, for a technical woman” (female manager, it3: 5). she felt that women performed better in these roles because of a capacity to think broadly, while men tended to think in a linear fashion. such jobs, however, did appear to remain separate from the core of “pure” computing in the organisations we examined. also somewhat detached from “pure” technical work were presales and consulting roles combining business and it skills – areas where women were reasonably well represented in it1 and it2. these were often seen as desirable jobs: presales is an area that perhaps a lot of consultants and other technical people would aspire to because it’s dealing directly … with the customers. it’s a little bit more … glamorous ajis volume 12 no. 2 may 2005 86 than the other roles. it’s more highly paid than the other roles. (female manager, it1: 5) in it1 there was a clear recruitment strategy to provide the organisation with some of these higher skill hybrid employees and enhance the level of communication and business skills in the organisation. as a manager at it1 explained: “we actually sponsor the [business/it dual] degrees of [two local universities], and we get involved in actually screening [school] leavers” (female manager, it1: 6). this meant that employees for the consulting group in this organisation: “…generally come out of a business/it type degree, have some technical skills … some functional as well. so that the mix is good”. moreover, our interviewee noted that this meant they were hiring more women – there were more women undertaking these types of degrees and their skill base was: “well rounded [with good] communication skills” (female manager, it1: 6). this approach to recruitment was clearly evident to the employees: “communication skills are definitely what they’re looking for … communication skills and customer focus” (female employee, it1: 11). in all four organisations there was some indication that recruitment into hybrid roles could also occur through non-it channels. while this was sometimes through masters level degrees in different fields, there were also examples of older women who had developed a talent for it in the course of their work and moved in that direction from administrative or other support roles. in the smaller organisations (it3 and it4), project manager and business analyst were singled out by managers as roles that did not require formal it qualifications. also, at it2 some staff members were recruited because of particular industry experience and knowledge of the product market, and a manager observed: “we’ve actually got a good mix of males and females across that range” (male manager, it2: 4). the range of hybrid roles and entry pathways was thus extremely varied, and appeared to depend to a large extent on the nature of the organisation, including its size and primary function. for example, it1’s main product was largely developed overseas, and the local need was mainly for adaptations for the ausralian market and development of a local clientele, making business and customer liaison skills a primary focus. while these skills were obviously important for all the organisations, it2 (although part of an overseas owned corporation) was primarily focused on a locally developed product and the technical development staff that would best ensure its competitive edge. similarly, the smaller organisations (it3 and it4) were mainly dependent on technical innovation. nevertheless, there was a widespread perception that a greater variety of jobs was emerging in the industry and that women would be well suited to a number of these. in it1 in particular, there was a clear focus on well-paid it/business jobs that were not located at the “routine” end of the spectrum. however, in many cases the technical component of the jobs under discussion was relatively limited, and the other side of the hybrid role – the communications and people management skills – sometimes appeared devalued and not compatible with technical skill. for example: this might sound like a generalisation but …[she has] more of the softer skills. she seems to be far more caring and ….yeah, she just seems to… have a better relationship with the people we work with than say i do. ... i think it seems if people have a problem in our department they’ll go and see [her]. unless it’s technical and then they’re at my desk. (male employee, it3: 30) moreover, assumptions about women’s skills sometimes flowed over into expectations that they would pick up basic secretarial and other support roles. as one of the female employees in it2 told us: … sometimes ... when you’re in a client meeting they say, “oh yes [respondent’s name] you can take the notes, … you … can type up the minutes”…[or] “you are the minute taker”; and, “oh [respondent’s name], can you organise this or can you organise that?” and it’s just like …right, okay... so we’re just pulling that hat back on. (female employee, it2: 25) ajis volume 12 no. 2 may 2005 87 a consistent theme alongside these perceptions of women’s skills and potential roles in the it industry was the notion that “pure” computing remained a masculine domain. ideas about men’s skills – including the lack of communications skills among technical experts – are examined in the following section. perceptions of men’s skills and “pure” computing in all our organisations the primary determinant of business success was the capacity for technical innovation, and jobs around this function were always prioritised. for example, in spite of the positive views reported above about analyst roles for women in it3, the most important role in that organisation was identified as “architect” – a crucial development function that was exclusively male in that workplace: “we don’t recruit men more than women on that role [architect] but i think that’s where they naturally fall” (female manager, it3: 1). in the small firms in particular it was clearly technical skill and inventiveness that was seen as providing the leading edge. the recruitment of technical experts was a highly focused process that tended to reproduce masculine culture and a male dominated technical workforce. in it4, the manager viewed it as essential to maintain the intimacy of the development team because: … the relationships between them and how everything should be done [is crucial]. and there’s a lot of … i describe it as technical religion. you know, where they believe things should be done a certain sort of a way (male manager, it4: 9) a new member must suit this existing orthodoxy: “they work in teams, and they’ll actually shift as teams – typically” (male manager, it4: 11). the recruitment process was therefore intense: i will do five or six interviews involving all of the [development] team because the environment they operate in together is very important. and to upset that environment is really costly … i’m looking for someone that can fit in and does subscribe to the same view, but is also willing to … question the given logic at any time. (male manager, it4: 910) the masculinity of this team, similar to that described by newton (2001), was also underscored by the manager’s observations that: the developers could easily find jobs but] it’s finding a job which is interesting to them, where they’re going to fit in, where they’re going to feel valued, where there’s an alpha geek that they … want to learn from. there’s a real alpha geek sort of a thing – you know, that is there a key person here that i really respect and i can learn from. (male manager, it4: 16) in this context, the (all male) development staff were the opinion leaders, and their lives revolved around it: … there’s a clear hierarchy … they know everything, they’re arguing about stuff, and they’re really opinionated and they just live it … they’ve all got, have lan at home with eight computers on it, and that’s all they do. … i let them do what they want to do…[and they] get paid a lot. (male manager, it4: 9, 14-15, 20) these observations suggest that any drive to recruit hybrids is likely to be focused on the types of roles that can be more easily detached from the design core of it, and that there is likely to be a cultural core in it organisations that devalues the status of hybrid jobs. this is in spite of a recognition of the benefits of enhancing communication and business skills in all our case studies, and widespread perceptions that technically adept men tended to lack good communications skills. for instance: i think these … really kind of pointy-end roles, the software architect and programmer type roles, i think will still attract more men. i don’t know why but that’s what i think …. these ajis volume 12 no. 2 may 2005 88 guys, our development guys don’t really get exposed to clients because they don’t have a lot of client skill. (female manager, it3: 17) similarly, the ceo of it4 elaborated his view of development and network engineering as “pretty blokey” with some observations on the communication skill deficiencies of these staff: … they can have, not the greatest sort of personal skills … they’ll sit in two rooms, one room apart, and they won’t go and talk to each other. they’ll do it all through email … there’re sort of strange stuff that goes on like that. (male manager, it4: 10, 13) these oddities were nevertheless tolerated – perhaps even expected as a trade off for technical brilliance (see also woodfield 2002 on perceptions of men’s communication skills in it work). they are a reminder that the basis for optimism about women and hybrid roles – the need for better communication within organisations and with end users – should be accepted with some caution. conclusion overall, our research has raised some questions over the idea that business pressures in computing firms will lead to the extension of hybrid roles involving both technical and communication or “people” skills, and that this will in turn increase the representation of women in it work. the survey data we examined was consistent with traditional patterns of gender segmentation across both hybrid and non-hybrid types of roles, although it did underline the existence of well paid roles in the industry that extended beyond purely technical functions. at the organisational level, we found some resonance with the idea that hybrid roles were important to the industry and perceptions of women’s skills that were consistent with their suitability for such jobs. in one case in particular, there was evidence of recruitment practices and links with tertiary programs that appeared to be assisting women’s entry to relatively well paid jobs combining technical and other types of skills. however, there was also evidence that business pressures were primarily driven by the need for technical creativity, particularly in the smaller organisations, and there was ongoing reinforcement of a masculine culture around these types of jobs and at least a tolerance of what was perceived as men’s lack of communication skills. thus although our findings suggest a more complex scenario than a clear-cut separation of women into devalued “soft computing” roles and men into “pure” technical roles, the research does illustrate aspects of the gendering of occupational roles and highlights the prevalence of essentialist ideas about gender and skill that are as likely to reinforce inequalities in the sector as they are to erode them. references brannen, j. (1992) “combining qualitative and quantitative approaches: an overview” in j. brannen (ed.) mixing methods: qualitative and quantitative research, avesbury, aldershot, uk. chapman, j. (1995) “the feminist perspective” in d. marsh and g. stoker (eds) theory and methods in political science, macmillan, london. cockburn, c. (1985) machinery of dominance: women, men and technical know-how, pluto press, london. cockburn, c. (1983) brothers: male dominance and technological change, pluto press, london. deakin, r. (1984) women and computing: the golden opportunity, macmillan, london. fountain, j. e. (2000) constructing the information society: women, information technology, and design, technology in society, 22, 45-62. greene, j., caracelli, v., & graham, w. (1989) toward a conceptual framework for mixed-method evaluation designs, educational evaluation and policy analysis, 11, 3, 255-274. ajis volume 12 no. 2 may 2005 89 grey, s., &healy, g. (2004) women and it contracting work – a testing process, new technology work and employment, 19, 1, 30-42. grundy, f. (1994) “women in the computing workplace: some impressions” in a. adam. j. emms, e. green and j. owen (eds) women, work and computerization, elsevier, amsterdam. mahoney, k., & van toen, b. (1990) mathematical formalism as a means of occupational closure in computing – why ‘hard’ computing tends to exclude women, gender and education, 2, 3, 319-331. miles, m. b., & huberman, a. m. (1994) qualitative data analysis: an expanded sourcebook, sage, thousand oaks. morris, j. (1989) women in computing, reed, london. newton, s. (2001) breaking the code: women confront the promises and the perils of high technology, women’s studies quarterly, 29, 3-4, 71-79. panteli, n., stack, j., & ramsay, h. (2001) gendered patterns in computing work in the late 1990s, new technology, work and employment 16, 1, 3-17. stoker, g. (1995) “introduction” in d. marsh and g. stoker (eds) theory and methods in political science, macmillan, london. turkle, s. (1984) women and computer programming: a different approach, technology review, 87, nov-dec, 48-51. wajcman, j. (1991) feminism confronts technology, polity, cambridge. webster, j. (1996) shaping women’s work: gender, employment and information technology, longman, london. woodfield, r. (2002) woman and information systems development: not just a pretty (inter)face? information technology & people, 15, 2, 119-138. woodfield, r. (2000) women, work and computing, cambridge university press, cambridge. acknowledgements this paper is based on research for the project the production of pay (in)equity for women: a study of emerging occupations which is funded through an australian research council discovery grant (dp0209261). the research team consists of gillian whitehouse (uq), meg smith (uws), alison preston (curtin), rosemary hunter (gu) and di zetlin (uq). we would like to thank michelle hayes for her research assistance. copyright [gillian whitehouse & chris diamond] © 2004. the authors assign griffith university a nonexclusive license to use this document for personal use provided that the article is used in full and this copyright statement is reproduced. the authors also grant a non-exclusive license to griffith university to publish this document in full in the conference proceedings. such documents may be published on the world wide web, cd-rom, in printed form, and on mirror sites on the world wide web. any other usage is prohibited without the express permission of the authors. ajis vol. 9, no. 2 may 2002 the impact of business size and business type on small business investment in electronic commerce: a study of swedish small businesses r.c. macgregor1 l. vrazalic1 s. carlsson2 d. bunker1 m. magnusson2 1 department of information systems, university of wollongong, australia northfields ave wollongong nsw 2522, australia 2 department of information systems division for information technology karlstad university se-65188 karlstad, sweden 3 school of information systems, technology and management university of new south wales sydney nsw 2052, australia abstract in the past, organisations relied on traditional quantitative metrics, such as return on investment (roi) to make decisions when investing in technology. with the advent of electronic commerce (ec), organisations have had to rethink their investment and acquisition decisions due to the strategic nature of electronic commerce. where roi measures have failed, they have been replaced with a plethora of organisational driving forces. this paper focuses on the driving forces behind ec adoption by small and medium enterprises (sme's) and aims to determine the impact of organisational factors such as size and type of business on ec acquisition criteria. the results of a research study carried out in sweden are presented and suggest that there exist high levels of significance between the size of the business and customer demand, reduced costs, developing new markets and improvement to marketing as driving forces, and the type of business and customer demand, pressure from competition, increased sales and improvement of relationship with business partners as driving forces for ec adoption. introduction prior to the advent of electronic commerce (ec) organisations utilised technology to enforce specific products and product boundaries. the products themselves were tangible, requiring physical inputs and processes that could be easily evaluated. boundaries around these products were defined by the products themselves and maintained within the operational level of the organisation such that organisational strategies were fixed and controllable. the technology used to support these products was standalone and, like the products it supported, could be evaluated by aggregated financial outcomes that related to the objectives of the organisation as a whole. as a result, decisions to invest in technology involved an examination of the return on investment (roi), a measure that was housed within strict strategic boundaries in the organisation. the advent of ec has not only radically altered the day-to-day procedures of the organisation (kuljis et al 1998) but has also altered the use of technology within the organisation (fuller 2000; kendall & kendall 2001). as the use of technology has changed, so have the corresponding investment decision making processes. instead of physical products and services, and their operational boundaries, other organisational factors have had an influence on strategic investment decision making. these changes have not only affected large organisations, but have had an impact on small to medium enterprises (sme's) as well. the aim of this paper is to examine the impact of organisational factors, including size and type of business, on the driving forces behind ec acquisition by sme's. this paper begins by examining some of the changes brought about by the advent of ec, in particular changes to the metrics used by organisations when considering investment in ec technology. the paper then discusses the nature of sme's, in particular their approach to acquisition, investment and use of ec technology. finally, the paper presents a study of 118 swedish sme's which investigated the major reasons for the adoption of ec. the study also endeavoured to ascertain whether organisational factors such as the size of the business and the type of business were associated with driving forces behind the adoption of ec. metrics adopted by organisations when considering ec technology as already suggested, prior to the advent of ec, technology was designed to embody existing organisational values and practices, power relationships and conventions, where retail and financial strategies were fixed and controllable. organisations utilised technology to enforce the use of specific products and product boundaries. 31 ajis vol.9, no. 2 may 2002 this was achievable by limiting the number and types of products and by placing the boundaries within the operational levels of the organisation. factors that might be termed informal social ties (customers, competitors, environmental trends) were 'shadows' to the formal organisation. the focus, instead, was on computerising procedures to achieve low-level operational competence. the role of technology was simply to increase efficiency, within the product boundaries, at a procedural level (schneider 1999), while decisions to invest in technology were primarily concerned with improvements to organisational efficiency and effectiveness. they were carried out in terms of strict strategic boundaries and were adjudged on rigid internal perspectives. an organisation considering the adoption of technology examined the roi of such technology where that roi was housed within a pre-stated set of strategic guidelines. a number of authors (kollman 1998; dugan 1999; hlava 1999; wilder 1999; violino 2000a; cameron 2000) have argued that with the advent of ec, traditional roi measurements have become obsolete or invalid. they suggest that roi is incapable of accounting for hidden costs and intangible strategic benefits, and that it cannot keep pace with the need to constantly adjust due to the fast pace of information technology (it) development. instead, there have been diverse attempts to apply different forms of evaluation methods and calculations in order to gauge the payback of it investments. indeed, wilder (1999) reports that many organisations are "tossing out conventional thinking about the need for a return on investment and focusing on how the initiatives advance their overall business strategy, whether it's to improve customer satisfaction, increase brand awareness, or open new sales channels". organisations have begun to rely on in-house, as well as off-the-shelf products for this purpose (moozakis & lewis 2000). some of the methods utilised include identifying sources of sales revenue generated through and attributable to e-commerce (violino 2000a), customer satisfaction surveys (violino 2000b), measuring the it performance of an e-commerce website as a whole, operational cost savings, observing customers' surfing and online purchasing habits to expand marketing strategies, determining the revenues generated during off-business hours, counting online transactions and the number of online visitors (lewis 2000), as well as benchmarking against competitors. where these methods fail, they are replaced by the instincts and experience of managers and decision makers (violino 2000a). not surprisingly, some organisations have begun viewing e-commerce outlays as absolutely necessary, regardless of cost, in order to remain competitive. cameron (2000) further confirms the abandonment of formal roi calculations by stating that it projects and investments can be justified for any number of reasons, without ever measuring any form of roi, highlighting the importance placed on e-commerce in the modern organisation. a number of reasons have been put forward for the inefficiency of roi as a measure of ec investment. wilder (1999) suggests that fundamental to the changes brought about by ec is the need for organisations to realise that as the line between product and product boundaries has become blurred, so too has the line between the operational and strategic roles of technology. indeed, as technology pervades the whole financial structure of organisations, such that web strategies become inexorably linked with overall business strategies (lederer et al 1997), it needs to be examined from a strategic standpoint rather than as just another technological adjunct. norman and zafar (1999) suggest that organisations need to clearly examine and articulate their business strategies in terms of e-commerce. brynjolfsson and hitt (1998) suggest that today investment in ec cannot exist without investment in organisational change and a re-examination of the products and the product boundaries. chircu and kauffman (2000) add that there has been a shift from technology that supports products to technology which is tailored to the customer. for the organisation, the 'shadows' of the past (customers, competitors, environmental trends) have become legitimate formal structures, pushing aside old organisational hierarchies. the implications of this development have led many organisations, including sme's which will be discussed next, to re-examine their it acquisition and investment strategies, in particular those relating to ec adoption. the nature of small businesses the special circumstances of small business have been the topic of many government committee findings as well as research initiatives. barnett and mackness (1983) suggest that small businesses should not be thought of as miniature versions of large businesses, but quite unique in their own right. small business tends to be more risky than its larger counterpart (brigham & smith 1967; walker 1975), it is subject to higher failure rates (klatt 1973; cochran 1981) and tends to keep less adequate records (markland 1974; rotch 1987). reynolds, savage and williams (1994) provide the following summary of features of small business generally not shared by larger organisations: • a small management team • strong owner influence • multi-functional management • a close and loyal work team • centralised power and control 32 ajis vol.9, no.2 may2002 • lack of specialist staff • informal and inadequate planning and control systems • lack of promotable staff • lack of control over the business environment • limited ability to obtain finance • labour intensive work • limited process and product technology • narrow product/service range, limited market share and heavy reliance on few customers • intuitive rather than rational decisions • reactive rather than innovative responses to change • personal rather than tasks oriented leadership • education, experience and skills practical but narrow • low employee turnover • product dedication rather than customer orientation • reluctance to take risks • management heavily swayed by the owner's personal idiosyncrasies • strong desire for independence • intrusion of family interests in line with general views concerning small business, the acquisition and subsequent use of computer technology in the small business environment cannot simply be considered a miniaturised version of the larger organisation. clearly, many of the features gleaned from studies of larger businesses can be applied to small business. however, the nature of small business operations presents a variety of factors which must be considered when computer technology is acquired in the small business environment. it is now appropriate to look at ec acquisition in light of the research carried out on sme's. acquisition of ec by sme's there have been a number of studies that have examined the involvement in ec by sme's. many of these studies (cragg & king 1993; thong et al 1996; kollman 1998) have concentrated on the implementation and post-implementation phases of technology use, however recent literature has begun to see the importance of the pre-acquisition decision making process. in a study of 308 ceo's, reimenshcneider and mykytyn (2000) nominated a variety of reasons, which they termed 'most important things' that prompted sme's into the adoption of ec. these included reduction of costs, increased productivity, faster feedback of information and access to customers. a similar study of the asia pacific region by pricewaterhousecoopers (1999) suggested factors such as improved customer service, customer information exchange, improved competitiveness, access to international markets, increased revenue, attracting new investment, as reasons put forward by sme's for their adoption of ec. similar studies by fuller (2000) and kendall (2000) have provided comparable lists of reasons for the adoption of ec by sme's. not only does the literature seem to support the views of authors such as wilder (1999), violino (2000a), cameron (2000), that there are alternatives to simple roi when ec is considered, recent studies (thong 1999; mcdonagh & prothero 2000; korchak & rodman 2001) suggest that these alternatives are affected by factors such as size of business and the type of business. indeed, in a study of sme's in the us, korchak and rodman (2001) found that 29% of sme's with a staff number above 20 were using ec in their day-to-day business activities compared to 13% for organisations with staff levels below 20. these figures are comparable to a study carried out in australia by the department of communication information technology (2000) which found that 28% of sme's with a staff greater than 20 were using ec, as compared to 17% of those with a staff level below 20. the australian study also examined the respondent group across business type and found that 14% of industrial sme's were using ec in their day-to-day activities as compared to 31% of those sme's involved in the service sector. a research study, which investigated the impact of these organisational factors on driving forces behind ec adoption in swedish sme's, will be presented next. survey instrument a questionnaire was developed for sme managers. a series of factors termed 'driving forces for the adoption of ec' were developed. the inclusion and exclusion of factors was based on previous studies, in particular those carried out by pricewaterhousecoopers (1999) and reimenschneider and mykytyn (2000). only those factors 33 ajis vol.9, no. 2 may 2002 which were reported as having a greater than 50% response as important were included in the survey instrument. the list of factors included in the questionnaire were: • customer demand • pressure by competition • pressure by suppliers • reduced cost • improvement to customer services • improvement to lead time • increased sales • increased sales improvement to internal efficiency • improved relationships with business partners • development of new markets • improved competitiveness • improved marketing • greater control of the business respondents were asked to rate each of these factors as pertaining to their adoption ec across a 5 point lickert scale where 1 indicates that the factor was unimportant to the decision to adopt ec, 2 indicates little importance to the decision to adopt ec, 3 indicates some importance to the decision to adopt ec, 4 indicates the factor was quite important to the decision to adopt ec and 5 indicates it was of greatest importance to the decision to adopt ec. respondents were also asked the number of employees in the organisation (none (self-employed); 1 to 9; 10 to 19; 20 to 49; more than 50) as well as the type of business (industrial, service, retail, finance). the questionnaires were distributed to sme's around three regional areas of sweden and sme's that are members of the chamber of commerce. analysis of responses a total of 1170 questionnaires were distributed. responses were obtained from 260 sme organisations giving a response rate of 22.4%. of the 260 responses, 118 indicated that they were using ec in their day-to-day business activities. tables 1 and 2 provide a description of the respondent group, while table 3 provides the overall responses for the major driving forces behind the adoption of ec. some respondents did not complete the questionnaire fully. this was taken into account in the statistical analysis. table 1 number of employees 0 1-9 10-19 20-49 >50 total respondents 37 131 36 27 26 ec users 9 49 21 19 20 table 2 type of business industrial service retail finance total respondents 59 95 48 54 ec users 31 49 19 5 34 no importance 48 36 57 13 6 little importance 25 20 23 12 1 some importance 20 32 19 28 16 very important 19 19 13 37 43 greatest importance 6 9 3 27 52 ajis vol. 9, no. 2 may 2002 table 3 factors which were the major driving forces behind the adoption of ec customer demand pressure by competition pressure by suppliers reduced cost improved customer services improvement to lead 29 14 23 30 19 time increased sales 17 13 35 25 26 newmarkets 18 15 21 33 30 development improved 13 5 18 41 39 competitiveness improved marketing 24 8 21 30 33 greater control of 25 16 28 29 19 business improved relationships 16 11 32 39 19 with business partners a series of chi-square tests were carried out to determine whether the number of employees or the type of business were associated with the rating of factors behind the adoption of ec. for brevity, only the significant data will be shown. tables 4 to 7 show that the number of employees was significantly associated with the rating of the following ec adoption factors customer demand, reduced costs, possibility of new markets and improved marketing. table 4 rating of the importance of customer demand as a criteria for the adoption of ec no of employees 0 1-9 10-19 20-49 >50 (p<.05) no importance 5 27 5 5 6 little importance 1 10 9 3 2 some importance 2 5 2 6 5 very important 1 5 5 4 4 greatest importance 0 2 0 1 3 35 little importance 0 9 0 2 1 some importance 1 13 2 6 6 very important 2 11 10 6 8 greatest importance 1 11 7 3 5 no importance 4 8 1 1 4 little importance 1 3 1 4 6 some importance 0 9 6 5 1 very important 3 8 8 8 6 greatest importance 1 21 5 1 2 ajis vol. 9, no. 2 may 2002 table 5 rating of the importance of reduced costs as a criteria for the adoption of ec no no of employees importance 0 5 1-9 4 10-19 2 20-49 2 >50 0 (p < .005) table 6 rating of the importance of developing new markets as a criteria for the adoption of ec no of employees 0 1-9 10-19 20-49 >50 (p<.001) table 7 rating of the importance of improvement to marketing as a criteria for the adoption of ec no of employees 0 1-9 10-19 20-49 >50 (p<-05) the data was also examined to determine whether the type of business was significantly associated with the rating of any of the ec adoption criteria. tables 8 to 11 show that the type of business was significantly associated with the rating of the following ec adoption factors: customer demand, pressure from competition, increased sales and improvement of relationships with business partners. table 8 rating of the importance of customer demand as a criteria for the adoption of ec type of business industrial service retail finance (p < -05) no importance 4 6 0 1 2 little importance 1 1 0 1 2 some importance 1 8 2 5 2 very important 2 11 11 8 9 greatest importance 1 22 8 4 4 no importance 13 21 • 9 0 little importance 5 12 5 1 some importance 5 7 4 1 very important 5 9 1 1 greatest importance 3 0 0 2 36 no importance 8 16 8 0 little importance 7 9 3 0 some importance 9 14 5 1 very important 5 8 3 0 greatest importance 2 1 0 4 no importance 2 10 1 1 little importance 3 8 1 0 some importance 13 14 6 0 very important 9 10 3 0 greatest importance 5 6 8 4 ajis vol. 9, no. 2 may 2002 table 9 rating of the importance of pressure from competition as a criteria for the adoption of ec type of business industrial service retail finance (p<.001) table 10 rating of the importance of increased sales as a criteria for the adoption of ec type of business industrial service retail finance (p<-05) table 11 rating of the importance of improvement of relationships with business partners as a criteria for the adoption of ec no type of business industrial service retail finance (p < -05) discussion before examining the data in detail, it is interesting to note that of the 260 respondents, only 118 (45.4%) indicated that they are using ec in their day-to-day business activities. this finding will be the subject of a study to be produced by the authors. furthermore, an examination of table 1 shows that those respondents who had a staff level above 10 were far more likely to adopt ec than those with less than 10 employees. this data is in accord with the studies carried out by department of communication information technology (2000) and korchak and rodman (2001). a number of authors and studies (wilder 1999; pricewaterhousecoopers 1999; violino 2000b; reimenschneider & mykytyn 2000) have suggested that factors other than strict roi are being used by sme's as criteria to be considered for the adoption of ec. a closer examination of table 3 would tend to support this, at least amongst the respondent group. while reduction of cost was seen as important as a criteria for the adoption of ec, by far the strongest reasons appear to be improvements to customer services, improvements to internal efficiency and improvements to organisational competitiveness. while these figures do not actually reject roi, they suggest that for sme's, roi is only one of several factors that need to be taken into consideration when adopting ec. based on recent studies (thong 1999; mcdonagh & prothero 2000; korchak & rodman 2000) which suggested that the size of the organisation had an effect upon the decision making processes surrounding the adoption of ec, the data was examined to determine whether the number of employees was associated with the rating of any of the factors behind ec adoption. the figures presented in tables 4 to 7 suggest that at least for some factors, there is a significant association to the size of the organisation. an examination of table 4 shows that for organisations with less than 10 employees, demand from customers was of little or no importance when ec was being considered. for those organisations with larger employee levels, the demand from customers had some importance in the decision making process. table 5, on the other 37 no importance 2 8 1 3 little importance 0 6 3 0 some importance 11 14 6 0 very important 12 15 3 1 greatest importance 6 5 6 1 ajis vol. 9, no. 2 may 2002 hand, compares the rating of the reduction of cost as a factor behind the adoption of ec, with the size of the organisation. as the data indicates, organisations with staff levels greater than 10 tended to place a higher importance on cost reduction than those organisations with a small staff group. indeed, over 50% of those organisations who had no staff placed no importance on the reduction in cost as a criteria for the adoption of ec. an examination of table 6 shows that medium sized sme's (1 to 20 employees) placed a higher level of importance on the use of ec to attract new markets than those organisations whose staffing level was greater than 20. finally, a closer look at table 7 indicates that, with the exception of single person organisations, all other respondents placed a great deal of importance on improvement to marketing through the adoption of ec. studies by pricewaterhousecoopers (1999) and department of communication information technology (2000) suggested that the type of business has a direct bearing upon the decision making process surrounding the adoption of ec. the data presented in tables 8 to 11 shows that the rating of at least some factors associated with the adoption of ec are significantly associated with the type of business of the respondent. demand from customers and pressure from competition were rated as being important only by finance organisations. as can be seen in table 8, with the exception of those businesses involved with finance, little importance was placed on the demand of customers in the adoption of ec, while table 9, indicating that the finance sector placed the most importance on pressure from competition, supports the findings of wilder (1999). an examination of table 10 shows that all respondents, with the exception of service organisations, placed a high emphasis on increased sales as a criterion for the adoption of ec. finally, table 11 also shows that all groups, except financial sector organisations, placed a moderately high level of importance on improving business relations with partners. 60% of the financial sector saw no importance in this factor. conclusion the advent of ec has made it necessary for sme's to radically re-think their priorities when making decisions as to whether to adopt the technology into their organisation. since ec has such wide ranging effects on the organisation, simple roi no longer suffices as the only justifying metric through which decisions can be made. not only is there a need to re-think the metrics involved in the decision making process, there is also convincing evidence that organisational factors, including the number of employees and type of business, have a strong bearing on the importance of many of the driving forces behind ec. the results of this study have led to two conclusions. first, it supports the findings of similar research indicating that sme's have already begun to think beyond simple roi as a sole mechanism to decide whether ec should be adopted. secondly, factors such as size of organisation and the type of business, appear to have some bearing upon the importance placed on certain factors when these decisions are being made. acknowledgements the authors would like to express their appreciation to sara berggren and anette stjarnkvist for their help in distributing and collating the questionnaires. references bamett, r.r. & mackness, j.r. (1983) "an action research study of small firm management", journal of applied systems analysis, vol 10, pp 63 83. brigham, e.f. & smith, k.v. (1967) "the cost of capital to the small firm", the engineering economist, vol 13 no l,pp 1 -26. brynjolfsson, e. & hitt, l.m. (1998) "beyond the productivity paradox", communications of the acm, vol 41 no 8, pp 49-55. cameron, p. (2000) "measuring up", cma management, vol 74 no 2, pp 26 28. chircu, a.m & kauffman, r.j. (2000) "limits to value in electronic commerce-related it investments", journal of management information systems, vol 17 no 2, pp 59 80. cochran, a.b. (1981) "small business mortality rates: a review of the literature", journal of small business management, vol 19 no 4, pp 50 59. cragg, p.b. & king, m. (1993) "small firm computing: motivators and inhibitors", mis quarterly, vol 17 no i,pp47-60. department of communication information technology, australia (2000) taking the plunge, 49 pp. dugan, s. (1999) "the revenue factors: strategies for maximizing e-commerce success", infoworld, vol 21 no 40, pp 70 72. fuller, t. (2000) "the small business guide to the internet: a practical approach to going online", international small business journal, vol 19 no 1, pp 105 107. 38 ajis vol. 9, no. 2 may 2002 hlava, m.m.k. (1999) "expanding partnerships for small-and medium-sized enterprises (sme's) needed: the right kind of support", bulletin of the american society of information science, vol 25 no 5, pp 7 9. kendall, k.e. (2000) "e-commerce aesthetics: e-commerce for e-commerce' sake", information resources management journal, vol 13 no 3, pp 3 4. kendall, j.e. & kendall, k.e. (2001) "a paradoxically peaceful coexistence between commerce and ecommerce", journal of information technology theory and application, vol 3 no 4, pp 1 6. klatt, l.a. (1973) small business management: essential in entrepreneurship, wadsworth, ca kollman, t. (1998) "marketing for electronic market places the relevance of two 'critical points of success'", electronic markets, vol 8 no 3, pp 36 39. korchak, r. & rodman, r. (2001) "ebusiness adoption among us small manufacturers and the role of manufacturing extension", economic development review, vol 17 no 3, pp 20 25. kuljis, j, macredie, r. & paul, rj. (1998) "information gathering problems in multinational banking", journal of strategic information systems, vol 7, pp 233 245. lederer, a.l., mirchandani, d.a. & sims, k. (1997) "the link between information strategy and electronic commerce", journal of organisational computing and electronic commerce, vol 7 no 1, pp 17 34. lewis, d. (2000) "pressure mounts to gauge e-biz roi", internetweek, url http://www.internetweek.com/transformation2000/issues/roi.htm. mcdonagh, p. & prothero, a. (2000) "eurolicking and the irish sme: prepared for e-commerce and the single currency", irish marketing review, vol 13 no 1, pp 21 33. markland, r.e. (1974) "the role of the computer in small business management", journal of small business management, vol 12 no 1, pp 21 26. moozakis, c. & lewis, d. (2000) "enterprises tailor roi to e-business", internetweek, url http://www.internetweek.com/netresults/netl21800.htm. norman, j.d. & zafar, s. (1999) "e-analytics: science and art", banking strategies, vol 75 no 2, pp 6 12. pricewaterhousecoopers (1999) sme electronic commerce study final report, 37pp reimenschneider, c.k. & mykytyn, p.p.jr (2000) "what small business executives have learned about managing information technology", information & management, vol 37, pp 257 267. reynolds, w., savage, w. & williams, a. (1994) your own business: a practical guide to success, itp. rotch, w. (1987) management of small enterprises: cases and readings, university of virginia press. schneider, p. (1999) "australia unbound", cio, vol 12 no 16, pp 40 45. thong, j.y.l., yap, c.s, & raman, k.s. (1996) "top management support, external expertise and information systems implementation in small business", information systems research, vol 7 no 2, pp 248 267. violino, b. (2000a) "it excellence 500", internetweek, no 754, pp 48 62. violino, b. (2000b) "payback time for e-business", internetweek, special issue 811. walker, e.w. (1975) "investment and capital structure decision making in small business" in walker, e.w. (ed) the dynamic small firm: selected readings, austin press, texas. wilder, c. (1999) "roi: e-business strategic investment", informationweek, no 735, pp 48 56. 39 microsoft word ajisvol10 1_final_edit.doc ajis vol. 10 no. 1 september 2002 20 exploratory analysis of the effect of consultants on the use of world wide web sites in smes sigi goode school of business and information management australian national university canberra, australia email: sigi.goode@anu.edu.au ph: +61 2 6125 5048 abstract there is little published research on the role of consultants in technology adoption. given the increasing popularity of the world wide web in commercial environments and the number of consultants now offering web development services, some analysis into the effects of their engagement would be of benefit. in an extension of an ongoing study, an existing sample of 113 world wide web adopters was used to examine the nature of world wide web site use with respect to consultant and internet service provider (isp) engagement. analysis was also conducted into the use of consultants and isps as developers and maintainers of these sites. this preliminary research finds a number of interesting outcomes. no significant relationship is found between consultant or isp engagement and world wide web site use, regardless of whether the consultant was engaged as site developer or site maintainer. the study raises a number of additional findings that are of interest but are not directly related to this study. these findings merit further research. introduction there is considerable published evidence of the substantial growth in commercial www adoption over the past four years (gallaugher 1999, ranganathan and ganapathy 2002). several authors note the fervency with which companies are seeking to make efficient and cost effective use of this technology (teo et al. 1997, tan and teo 1998, liang and lai 2002). the adoption of new technology, however, is not always smooth or successful. in some cases, the perceived difficulties of a new adoption, or the costs or risks of an unsuccessful adoption, are so great that some businesses may seek to supplement their existing abilities with those of an external consultant or consulting group (rassam and oates 1991). in particular, this may be the case where small business resource poverty means staff members must take on more than one role (palvia and palvia 1999). this, combined with the relative newness of www research, makes fertile ground for a preliminary study into the area. at this early stage, with research into www adoption and success still largely in its infancy, this study is interested in the basic research question: does a company’s engagement of consultants or isps affect that company’s use of their www site? this paper seeks to answer this question by comparing the site uses of businesses who have engaged consultants to those that have not. the rest of this paper is structured as follows: first, the paper examines the existing research into the use of technology consultants, and uses this analysis to derive some initial propositions for examination. the paper then examines the commercial uses of world wide web technology to determine the range of web technology applications with which a consultant may be able to provide assistance. the research method and sample are discussed, followed by hypothesis testing and discussion of results. finally, limitations of the study are discussed and conclusions are made. consultants and information systems an examination of the information systems research literature reveals a lack of published research into the role of consultants in commercial information technology environments. sturdy (1997) notes that the bulk of research in this area lies in the historical role of consultants in business, but argues that “little attention has been given to the wider…consequences of consultancy”. gable (1991), soh et al. (1992) and kole (1983) make similar arguments, observing the importance of studying the role of consultants in business, in particular with respect to smalland medium-sized organisations. existing research into the role of consultants in it is thin, however the topic has received some scholarly attention, notably from gable (1991). gable argues that it is important to study consultants so as to be able to manage them more effectively, to justify further use or discontinuance of external consultants, to assess the impact of the engagement on the organisation, to assess the appropriateness of the level of client dependence on consultants, and to provide feedback to the client. soh et al. (1992) note that the examination of the role of consultants in small business environments is important because small businesses are deficient in slack resources (or “resource poverty”), and may also lack technical support and/or experience. this lack of experience may also undermine the future of the firm, including developing a strategic plan (dou and dou 1999). soh et al. (1992) examine the topic of consultant engagement with respect to system success. the study found that consultant-engaging businesses had higher levels of computer usage. this, the authors argue, suggests that ajis vol. 10 no. 1 september 2002 21 these businesses may have better operating procedures, and have better access to critical information. this conclusion would be consistent with gable (1991) and fitzgerald (1975). this leads to the first research proposition: p1. businesses that engage consultants will have higher levels of system use than businesses that do not. soh et al. also examined the level of business experience of firms that engage consultants. while their study found no significant differences in experience levels, gable (1991), major (1999), sturdy (1997), tisdall (1982) argue that consultants are often engaged in instances where system implementation staff or business operations are inexperienced or immature. cavaye and cragg (1995) conducted analyses of case businesses in six different industries and found that inexperienced businesses outsourced their implementation “so that their system was built by experienced professionals”. this leads to the second research proposition: p2. businesses that engage consultants have less experience with information systems than those that do not engage consultants. it is important to note that the propositions made above are purely exploratory, intended to provide some initial assessment of the role of consultants in technology and website use. as such, the propositions are presented in the spirit of new research into an area. it is understood that there may be some degree of logical inconsistency between p1 and p2. businesses with more is experience for example, might have greater current system use (bajaj and nidumolu 1998). however, it is possible for a firm to have higher experience but less use: such would be the case for newer firms or new systems (doll and torkzadeh 1998), for instance. in this vein, exploratory research acknowledges that, for any phenomenon, there may be multiple objective points of view (malhotra and grover 1998). such perspectives may exhibit differences, however each may be intrinsically valid (walsham 1993) and is hence particularly useful in the early stages of research (dubin 1978). the world wide web and its commercial uses the www is a technology that is receiving increasing attention in both the research literature and popular media. businesses are apparently keen to adopt this new technology and its popularity as a marketing and information dissemination tool is increasing. while the technology can be easy to adopt and “unadopt”, the process can be financially costly and/or time-consuming for businesses that are unfamiliar with the technology (nambisan and wang 1999). businesses are faced with the choice of either self-tuition or the engagement of an external knowledge source such as a consultant or internet service provider (isp). a number of authors examine the business uses of world wide web technology. the literature suggests three broad uses, being the promotion of products and services, the provision of data and information, and the processing of business transactions. table i details literature references to these uses. table i: commercial uses of the world wide web in the research literature use/application referencing papers promotion of products and services swatman et al. (1995), lin (1996), lewis (1996), ravindran et al. (1996), jarvenpaa and todd (1997) koreto (1997) haynes et al. (1998) spiller and lohse (1998) gallaugher (1999) lindemann and schmid (1999) provision of data and information swatman et al. (1995) lin (1996) abell and lin (1996) jarvenpaa and ives (1996) liu et al. (1997) haynes et al. (1998) spiller and lohse (1998) lindemann and schmid (1999) processing of business transactions swatman et al. (1995) warkentin (1995) bhimani (1996) panurach (1996) feher and towell (1997) spiller and lohse (1998) lee (1999) ajis vol. 10 no. 1 september 2002 22 hypothesis development the propositions made above are now extended into testable hypotheses which relate the engagement of consultants to computer system and world wide web site use. p1. businesses that engage consultants will have higher levels of system use than businesses that do not. h1a consultant-engaging businesses have more website uses than businesses that do not engage consultants. h1b consultant-engaging businesses have more computers than businesses that do not engage consultants. h1c consultant-engaging businesses have more computers per employee than businesses that do not engage consultants. h1d consultant-engaging businesses have more computer applications than businesses that do not engage consultants. p2. businesses that engage consultants have less experience with information systems than those that do not engage consultants. h2a consultant-engaging businesses have more years of computer use than businesses that do not engage consultants. h2b consultant-engaging businesses have more years of web use than businesses that do not engage consultants. research method data for this research were obtained from an ongoing study into the adoption of www technology. these data were originally obtained via a survey questionnaire of 245 australian small businesses who were known adopters of www technology. the questionnaire instrument captured descriptive data on the business in addition to data concerning the business’ adoption of the www. questions regarding the identity of both the developer and the current maintainer of the website were included in the survey3. in addition, data were gathered concerning the business’ use of the website. where possible, questions were adapted from existing is research, in particular montazemi (1988) and delone (1981). space was included at the end of the question to allow the respondent to add site uses that may have been overlooked. once complete, the survey was pre-tested on a group of senior university staff, as advocated by dillman (1978). this process proved useful in checking spelling and grammar in addition to survey structure and flow. after this pre-test, a pilot test was conducted using two local businesses. this second stage proved useful in restructuring the wording of two questions. surveys were printed on paper bearing the official university letterhead and crest, and were accompanied by a letter assuring the respondents of confidentiality. the sample the survey yielded a final useable response rate of 113 from 245 businesses targeted, yielding a response rate of 46%. analysis of respondent data was compared to population profiles provided by the australian bureau of statistics (mclennan 1999): this analysis suggested a representative spread of businesses types, and showed that no businesses was unnaturally under or over represented in the respondent group. analysis of the existence of response bias was also conducted. as advocated by grover et al (1994), data on business size and respondent position were examined with respect to early and later respondents. this analysis showed no significant differences. filion (1975), however, notes that the determination of response bias may be error-prone. if the analysis of return dates provides an indication of response bias, the results shown in table ii would suggest that little bias, if any, exists. this is consistent with the high response rate obtained (church 1993), which is substantially higher than normal response rates in the information systems research literature (o’connor et al. 1997). 3 it was noted that there may be some biasing effect associated with the use of a consultant (perhaps some businesses may view consultant engagement as prestigious, for instance), so care was taken to structure the question in an open multi-selection format. indeed, one respondent listed the site’s developer simply as “daughter”. ajis vol. 10 no. 1 september 2002 23 table ii: mann-whitney analysis of response differences size posn mann-whitney u wilcoxon w z asymp. sig. (2-tailed) 1163.500 1829.500 -1.377 .168 1632.000 2028.00 -.161 .872 a. grouping variable: date received summary statistics for the respondent group are shown in table iii. most businesses were in the services sector, followed by retail and manufacturing. the majority of businesses employed less than ten staff members and were less than ten years old. these results suggest that most respondents were younger small businesses. table iv shows the respondent www usage summary statistics. almost half of the respondents stated that their website was developed by an in-house person or group, the next largest source of site development being an external consultant. while many isps also now offer site design services, they made up the smallest group of site developers. these data on the use of in-house development methods support the “champion” theory whereby new technology is introduced into the firm by a single person who can spearhead its cause (cavaye and cragg 1995). the use of an in-house development group may provide advantages in terms of cost, and also may support good requirements communication in that all members of the development process are familiar with each other. additionally, hiring a single person may require fewer resources than hiring a developmental team. the effect of this difference in resource requirement is unknown and, as a result, a distinction was made between these two groups. analysis of website maintenance data revealed an interesting trend. most respondents were now using an inhouse person to maintain their websites: use of other methods of site maintenance had decreased. this evidence suggests that the initial adoption of the technology process may be the most challenging and technologically demanding phase of the adoption process. a significant number of respondents (20%) still used the original consultants to maintain their website. most respondents used their site for the provision of company and product information. almost half of all respondents used their website for the provision of online sales. interestingly, only a small percentage kept pricing information on their site. these data suggest a discrepancy: only half of those sites who had used online sales mechanisms also featured prices on their site. apart from simple respondent error, there may be several explanations for this. first, it is possible that the sales mechanisms used on these sites also incorporated prices into their operation, hence there was no need to store price information elsewhere. second, it is possible that the products being sold were featured on a “price on application” basis (this would be consistent with the fact that most respondents were based in the service industry, where engagements may be priced on a case by case basis). no respondents supplied other uses for their website, which suggests that the uses provided in the survey were generally comprehensive. the provision of company information received the greatest use from respondents, supporting the contention that the www is used by many businesses at this stage as an alternative advertising medium. ajis vol. 10 no. 1 september 2002 24 table iii: business summary statistics demographic % industry services retail manufacturing 49.7 30.1 20.9 number of employees 0 – 5 6 – 10 11 15 16 – 30 31 or more 59.2 17.6 8.8 6.8 7.2 age of business 0 5 years 6 10 years 11 15 years 16 – 20 years 21 – 25 years 26 30 years 31 years or more 38.9 23.9 13.2 10.6 4.4 4.4 5.3 table iv: world wide web site characteristics demographic frequency percentage site age 6 months or less 7 12 months 13 18 months 19 24 months 25 32 months 33 months or longer 32 44 20 12 2 3 28.3 38.9 17.6 10.6 1.7 2.6 website usage company information product information sales customer information pricing information 84 77 64 60 35 69.4 63.6 52.8 49.5 28.9 website developer in-house person external consultant in-house team external internet provider 58 28 15 12 51.3 24.7 13.2 10.6 website content maintainer in-house person in-house team external consultant external internet provider 67 18 16 12 59.2 15.9 14.1 10.6 hypothesis testing table v shows the results of testing each of the hypotheses derived above. non-parametric statistical methods were used in the hypothesis testing process due to significant levels of skewness and kurtosis. newbold (1991) argues that in circumstances where a significant number of observations are available, non-parametric statistical methods are approximately equal in predictive strength to their parametric counterparts. ajis vol. 10 no. 1 september 2002 25 table v: summary of hypothesis test results hypothesis element mann-whitney u z score significance h1a website use 1296.00 -1.007 .314 h1b number of pcs 1182.00 -1.677 .094 h1c pcs per employee 1239.50 -1.332 .183 h1d number of applications 1373.00 -.536 .592 h2a years of computer use 1416.00 -.273 .785 h2b years of web use 1372.50 -.314 .754 a two tailed significance results and discussion does a company’s engagement of consultants or isps affect that company’s use of their www site? analysis of the results found no significant relationship between the use of consultants and any of the research items. the hypothesis test results raise a number of interesting outcomes and these are worth exploring on a case by case basis. the effect of consultants on web usage is integral to this study. the results found no significant difference between those companies that engaged consultants and those that did not. one possible explanation for this might be that consultants make no difference to the way in which commercial websites are used, however this is not necessarily the case. it is possible that had these businesses not engaged consultants, their websites would not have had as broad a range of uses. this finding is of substantial significance: the engagement of a consultant makes the site’s use no more or less varied than those of other businesses who do not engage external expertise. another interpretation of these findings is that, at this level of technological intervention, consultants do not have any significant impact on world wide web technology. second, the results suggest that businesses of any degree of experience are equally likely to engage consultants. this finding contrasts with evidence put forward by toole et al. (1996), who found that firms that received some form of management assistance were generally younger and less experienced in their particular business environments. one explanation for this outcome may be that while some experienced businesses understand the technology and can implement it without risk, others can predict when they will need to rely on external knowledge. inexperienced businesses may also know that they lack knowledge in a particular area, or they are wary of the introduction of new technology, hence the engagement of consultants. this argument would be consistent with soh et al. (1992) and turner (1982). a further explanation for this outcome concerns the political nature of consultant engagement as noted by sturdy (1997a) and bennett (1990), whereby consultants may be engaged by businesses not to assist in technology management but rather in order to defer or shift liability for project failure onto an external body, or to validate an existing or proposed course of action. third, the duration of system use also had no association with consultant engagement and use. consultants had been engaged by businesses regardless of the number of years for which the business had been using computer or www technology. if firms with substantial computer system experience were enlisting consultant help with www technology adoption, then this suggests that perhaps firms view www technology as markedly different to other technologies. this finding would be consistent with goode and stevens (2000), who noted differences between world wide web technology and more traditional information systems. if, as has been discussed earlier, the initial adoption is the most challenging and demanding aspect of the adoption process, it is possible that the consultants were not engaged in these projects for a long enough period of time. the finding that only 20% of respondents were continuing with their consultant engagement supports this conclusion, as does anecdotal evidence from the respondent group, a number of whom expressed dissatisfaction with their consultants. one respondent wrote, “if we’d had the choice again, we wouldn’t have used [the consultant]…we’d have tried it ourselves”. another respondent, whose site had been designed by and was hosted with an isp, wrote, “our website is failing because of [our isp]”. this would not be the first instance of consultant debility or failure: wilder (1987), nadel (1988), fincham (1999) have made similar observations. a number of respondents also expressed concern that the consultant had not transferred to the business the crucial knowledge they required to manage the website. this was evidenced in respondent anecdotal remarks, such as, “[the consultant] never told us what he was doing – it was a complete mystery”. apart from simple carelessness or lack of due attention, this may be because the consultant is hopeful for future engagement and is reluctant to surrender the intellectual assets that would bring this about. alternatively, the consultant may be concerned about the ease with which their competitive advantage in this arena could be eroded. in this regard, sturdy (1997a) notes, “it should be emphasised that consultants actively contribute to the conditions of their demand” and “[consultants] not only normalise problems, but elaborate them, and direct attention to new ones”. rassam and oates (1991) make similar observations. this findings in aggregate have implications for the ways in which businesses are using their websites. first, there appears to be a core group of site applications in common usage. these are the promotion of products and ajis vol. 10 no. 1 september 2002 26 services, the provision of data and information and the processing of business transactions. this list, and that provided in the questionnaire may not have been exhaustive, however – businesses may have used their sites for other, more technically demanding applications, such as order tracking or wireless communication. while space was provided on the survey instrument for respondents to add their own applications, few made use of this feature. this issue merits further analysis. an alternative interpretation of this finding is that consultants have more profound effects on the technical aspects of the website adoption project. it is possible that consultants are engaged not for their ability to supplement or augment site usage and content, but rather for their assistance in the site’s physical implementation (server installation, for instance), or the site’s coding (the implementation of more complex technology such as javascript, flash or server-side scripting languages in addition to more basic html). conclusions and further research this study has sought to contribute to the body of information system adoption research in general and www adoption in particular. the study is, however, subject to a number of limitations. it is possible that there are more site applications than those mentioned in the survey. additionally, the study was conducted in an australian context, and the degree to which the results are generalisable to other geographical areas is unknown. as with most studies involving survey methods, the research is open to self selection bias and this may distort the true facts. these limitations, whose overall effect is unknown, may mitigate the findings of the study. it is important to understand that while no relationship is found between consultant engagement and website usage, the finding does not provide enough evidence to support the re-evaluation or disengagement of the consultant role. the finding does, in conjunction with respondent comments, however, suggest that the relationship between consultant engagement and website use deserves further research attention. the findings also suggest that deeper examination of the propositions tested here would be welcome. for instance, prior research has suggested that less experienced organisations may be more likely to engage consultants, however it would also make for interesting analysis to examine conditions of consultant engagement when firms exhibit substantial experience with information systems and consultants. it is possible that such firms may be more familiar with the pitfalls of consultant engagement, as expressed by some of the organisations in this study. engagements in these circumstances may be more successful. finally, it should be noted that this study is of a purely exploratory nature and many factors still require examination. the range of website uses requires critical re-examination, as does the exact nature of the services offered by the consulting group. due to the field’s infancy, a case study approach may be the most scholarly rewarding approach for this. the large number of consulting groups in the marketplace, in addition to the frequency of their engagement suggests these firms must provide some tangible benefit to world wide web adopters. the finding of a lack of a relationship between the consultant’s engagement and the variability of site application appears to contradict this. these areas provide fertile ground for further research. references abell, w., lim, l. (1996) “business use of the internet in new zealand: an exploratory study”, ausweb 96 bajaj, a., nidumolu, s. r., (1998) “a feedback model to understand information system usage”, information & management, vol. 33, no. 4, 25 march, 213-224 bennett, r. (1990) choosing and using management consultants, kogan page publishers, london bhimani, a. (1996) “securing the commercial internet”, communications of the acm, vol. 39, no. 6, june, pp. 29 – 35 cavaye, a. l. m., cragg, p., b. (1995) “factors contributing to the success of customer oriented interorganisational systems”, journal of strategic information systems, vol. 4, no. 1, pp. 13 – 30 church, a. (1993) “estimating the effect of incentives on mail survey response rates: a meta-analysis”, public opinion quarterly, vol. 57, no. 1, pp. 62 79 delone, w. h. (1981) “firm size and the characteristics of computer use”, management information systems quarterly, december, pp. 65 77 dillman, d. a. (1978) mail and telephone surveys: the total design method, john wiley and sons doll, w. j., torkzadeh, g., (1998) “developing a multidimensional measure of system-use in an organizational context”, information & management, vol. 33, no. 4, 25 march, 171-185 dou, h., dou, j., (1999), “innovation management technology: experimental approach for small firms in a deprived environment”, international journal of information management, vol. 19, no. 5, october, 401-412 dubin, r. (1978) theory building, free press, new york,. feher, a., towell, e. (1997) “business use of the internet”, internet research: electronic networking applications and policy, vol. 7, no. 3, pp. 195 – 200 ajis vol. 10 no. 1 september 2002 27 filion, f. l. (1975) “estimating bias due to non-response in mail surveys”, public opinion quarterly, fall, pp. 482 492 fincham, r. (1999) “the consultant-client relationship: critical perspectives on the management of organisational change”, journal of management studies, col. 36, no. 3, pp. 335 – 352 fitzgerald, j. (1975) “in-house staff versus outside consultants”, proceedings of academy of management, august, pp. 113 115 gable, g. g. (1991) “consultant engagement for computer system selection”, information & management, 20, pp. 83 93 gallaugher, j. (1999) “challenging the new conventional wisdom of net commerce strategies”, communications of the acm, july, pp. 27-29 goode, s., stevens, k. (2000) “an analysis of the business characteristics of adopters and non-adopters of world wide web technology”, journal of information technology and management, vol. 1, no. 1, pp. 129 – 154 grover, v., lee, c., durand, d. (1994) “analysing methodological rigor of mis survey research from 1980 – 1989”, information & management, vol. 24, pp. 305 318 haynes, p. j., becherer, r. c., helms, m. m. (1998) “small and mid-sized businesses and internet use: unrealized potential?” internet research: electronic networking applications and policy, vol. 8, no. 3, pp. 239 – 235 jarvenpaa, a. l., ives, b. (1996) “introducing transformational technologies the case of web technology”, international journal of electronic commerce, vol. 1, no. 1, fall, pp. 95 – 126 jarvenpaa, s. l., todd, p. a. (1997) “consumer reactions to electronic shopping on the world wide web”, international journal of electronic commerce, vol. 1, no. 2, pp. 59 – 88 kole, m. a. (1983) “going outside for mis implementation”, information & management, 6, pp. 261 – 268 koreto, r. j. (1997) “when the bottom line is online”, journal of accountancy, march, pp. 63 – 65 lee, r. m. (1999) “distributed electronic trade scenarios: representation, design, prototyping”, international journal of electronic commerce, vol. 3, no. 2, pp. 105 – 135 lewis, t. (1996) “alice in wired world: wonderland or wasteland?”, computer, july, pp. 17 – 19 liang, t., lai, h., (2002) “effect of store design on consumer purchases: an empirical study of on-line bookstores”, information & management, vol. 39, no. 6, may, 431-444 lin, b. (1996) “managing in an information highway age: critical issues”, internet research: electronic networking applications and policy, vol. 6, no. 4, pp. 44 – 47 lindemann, m. a., schmid, b. f. (1999) “framework for specifying, building and operating electronic markets”, international journal of electronic commerce, vol. 3, no. 2, pp. 7 21 liu c., arnett, k. p., capella, l. m., beatty, r. c. (1997) “web sites of the fortune 500 companies: facing customers through home pages”, information & management, vol. 31, pp. 335 – 345 major, m. j. (1999) “after the merger: easing the transition”, tma journal, vol. 19, no. 5, pp. 54 – 57 malhotra, m. k., grover, v. (1998), “an assessment of survey research in pom: from constructs to theory”, journal of operations management, 16, 4, july, 407-425. mclennan, s. (1999), yearbook australia, australian bureau of statistics, government publishing service, canberra montazemi, a. r. (1988) “factors affecting information satisfaction in the context of the small business environment”, mis quarterly, june, pp. 239 256 nadel, r. b. (1988) “computer consulting for the smaller firm: opportunity or trap?” cpa journal, vol. 58, no. 4, pp. 18 – 29 nahavandi, a., chesteen, s. (1988) “the impact of consulting on small business: a further examination”, entrepreneurship theory and practice, fall, pp. 29 – 40 nambisan, s., wang, y. (1999) “roadblocks to web technology adoption?”, communications of the acm, vol. 42, no. 1, january, pp. 98 – 101 newbold, p. (1991), statistics for business and economics, 3rd edition, prentice-hall international, new jersey o'connor, m., bentley, j., calvert, c. (1997) “small business and the internet”, 8th australasian conference on information systems palvia, p. c., palvia, s. c., (1999) “an examination of the it satisfaction of small-business users”, information & management, vol. 35, no. 3, 8 march, 127-137 panurach, p. (1996) “money in electronic commerce: digital cash, electronic fund transfer and e-cash”, communications of the acm, vol. 39, no. 6, june, pp. 45 51 ranganathan, c., ganapathy, s., (2002) “key dimensions of business-to-consumer web sites”, information & management, vol. 39, no. 6, may, 457-465 rassam, c., oates, d. (1991), management consultancy – the inside story, mercury publishers, london ravindran, s., barua, a., lee, b., whinston, a. b. (1996) “strategies for smart shopping in cyberspace”, ajis vol. 10 no. 1 september 2002 28 journal of organisational computing and electronic commerce, vol. 6, no. 1, pp. 33-49 soh, c. p. p., yap, c. s., raman, k. s. (1992) “impact of consultants on computerisation success in small business”, information & management, vol. 22, pp. 309 319 spiller, p., lohse, g. l. (1998) “a classification of internet retail stores”, international journal of electronic commerce, vol. 2, no. 2, winter, pp. 29 – 56 sturdy, a. (1997) “the dialectics of consultancy”, critical perspectives on accounting, vol. 8, pp. 511 – 535 sturdy, a. (1997a) “the consultancy process: an insecure business?”, journal of management studies, vol. 34, no. 2, pp. 389 – 413 swatman, p. m. c., poon, s., green, d. g., payne, p. (1995) “overtaking on the information superhighway: strategic use of the internet”, australian conference on information systems tan, m., teo, t. s. h. (1998) “factors influencing the adoption of the internet”, international journal of electronic commerce, vol. 2, no. 3, spring, pp. 5 18 teo, t. s. h., tan, m., buk, w. k. (1998) “a contingency model of internet adoption in singapore” international journal of electronic commerce, vol. 2, no. 2, , pp. 95 – 118 tisdall, p. (1982), agents of change – the development and practice of management consultancy, heinemann publishing, london toole, h. r., sciglimpaglia, d., brkovic, n. (1996) “interest in the use of consulting services by small business: a comparison of gender and ethnicity”, proceedings of the international council on small business turner, j. s. (1982) “firm size, performance and computer use”, proceedings of the third international conference on information systems, december walsham, g. (1993) interpreting information systems in organisations, wiley publishing, chichester, uk warkentin, m. e. (1995) “competitive advantage on the world wide web: a webmaster's guide”, applied computing review, vol. 3, no. 2, pp. 25 32 wilder, c. (1987) “systems integrators”, computer world, december ajis special edition knowledge management december 2001 knowledge management audit a methodology and case study thomas w. lauer and mohan tanniru* school of business administration, oakland university rochester, ml 48304 usa lauer@oakland.edu tanniru@oakland.edu abstract the strategic importance of knowledge in today's organisation has been discussed extensively and research has looked at various issues in developing knowledge management systems. both the characterisation of knowledge and alternate models for understanding the acquisition and use of such knowledge have taken on significant prominence. this is due to the complexities associated with acquiring and representing knowledge, and the varied nature of its use in knowledge work. however, the role of the knowledge workers and the processes that guide their knowledge work as they meet the knowledge goals of an organisation have received little attention. this paper proposes a knowledge audit (an assessment of the way knowledge processes meet an organisation's knowledge goals) methodology to understand the "gaps" in the needs of a knowledge worker before one develops km systems the methodology also uses "process change" research to help build a socio-technical environment critical for knowledge work. the audit methodology is applied to a particular case and the implementation of the audit recommendations is discussed. future implications of such an audit arc also discussed introduction the argument for the strategic importance of knowledge in today's organisation has been made eloquently by a number of authors. for example, drucker (1978) asserts that making knowledge work productive will be the great management task of the century, and that, "...the most valuable asset of a 21st century institution...will be its knowledge workers and their productivity" (drucker 1999, p.79). others have noted the shift from an economy with a physical basis to one where knowledge and intellectual assets differentiate firms (hansen et al. 1999). a corollary observation is that an organisation's ability to learn and acquire knowledge rapidly may provide the only source of sustainable competitive advantage (winter 1995; dierick & cool 1989; lippmann & rumelt 1982). in a similar vein, changes brought on by technological development and global competition make rapid learning essential for the modem organisation (croasdell 2001). given the wider recognition of the critical role "knowledge" plays in organisations, early research has focused on different aspects of knowledge management (km). according to ackerman (1996) information technologies enable automating organisational memory systems by 1) making recorded knowledge retrievable and 2) by making individuals with knowledge accessible. to develop computer supported knowledge management systems, earlier research has focused on knowledge management strategy (davenport 1999), analysis and characterisation of tacit knowledge (nonaka and takeuchi 1995), technology support in design (eg. duffy 2000), and social aspects of knowledge use and sharing in implementations (eg. cook & brown 1999). given the nature of knowledge and the complexities associated with its capture and dissemination, research in cognitive science, systems development methodologies, socio-technical systems, and technology diffusion have become antecedents to much of the research on the km systems development. specifically, these research streams are used to help us better understand the constituent components of knowledge and how they are acquired and used by knowledge workers. one taxonomy of knowledge developed by nonaka (1994) explicates the tacit and explicit dimensions of knowledge and is based on the earlier work of michael polyani (polyani 1962; 1967) that has informed a significant body of research on knowledge based systems. an extension of this taxonomy was presented in spender (1996) in which he added an individual and social dimension to the explicit tacit distinction. for the individual, explicit knowledge is conscious whereas tacit knowledge is automatic. in the social realm, explicit knowledge is objectified. it refers to explicit policies and procedures. in contrast, tacit knowledge on the social level refers to the underlying organisational culture, the beliefs and values that are implicitly understood within the firm. one of the goals of the km system is to make an individual's tacit knowledge explicit on both the individual and the social levels. to help us better understand both the knowledge acquisition and use by knowledge workers, one model distinguishes between repository and network models (ackerman 1996; alavi & leidner 1999; fahey & prusak 1998; hansen et al. 1999). alavi describes the two models as follows. "the repository model views knowledge as an object that can be collected, stored, organised, and disseminated. as such, these systems focus on managing explicit knowledge and primarily focus on the creation and storage/retrieval aspects of organisational knowledge management." under this model, knowledge remains with the individual who possesses it and is transferred mainly through person-to-person contacts. key technologies are databases and intranets and a critical issue for the implementation of these systems is the codification of knowledge. an alternative, network models do not, "... attempt to codify and extract knowledge 23 ajis special edition knowledge management december 2001 from the individual who possesses it." network models support km processes that involve social interaction and direct communication among persons. this is in contrast to die repository model that involves person-repository and repository-person modes of knowledge transfer." while both models involve a socio-technical mix, the repository model throws its weight toward the technical system, while the network model emphasises the social system. in other words, the development of a socio-technical kms has to recognise the existence of both these models in an organisation and address them appropriately. the way in which knowledge is acquired and shared among knowledge workers is thus very complex and, hence, the knowledge work has more in common with work that is often associated with complex decision making than with operational or transaction-based work. thus certain parallels may be drawn from the way in which information systems are developed to support complex decision processes. the development of decision or executive support systems often called first for an understanding the problem solving processes of a decision maker. hence, before we embark on the development of km systems, we need to understand the processes used by the knowledge-workers. research has identified a few characterisations of processes associated with knowledge work [by probst, raub, and romhardt (2000), gupta and govindarajan (2000)]. by understanding the key processes associated with knowledge work and their inter-relationships, one should be able to effectively position the km system as one that support some of these knowledge processes. in other words, the focus of kms development will be one of support for knowledge work, thus ensuring that the goals as well as implementation strategies are articulated in terms of how they are aimed at support a knowledge worker. the focus of this paper is to conduct a "knowledge audit" an a priori analysis of knowledge processes and the degree to which they address the knowledge goals of both the organisation and the knowledge worker. just as "productivity goals" drive the development of transactional or operational systems and "decision quality goals" drive the development of a dss, "knowledge goals" should drive the way knowledge is processed by a firm's knowledge workers. the deficiencies or gaps detected from such an audit can then lead to the development of a socio-technical knowledge management system that supports knowledge work using a mix of "repository" and "network" models. the next section will discuss the knowledge audit methodology and the third section will illustrate the use of this methodology to a particular organisation. section four provides some recommendations and section five illustrates the changes made to the organisation since the audit recommendations were made. section six provides some concluding comments along with some general observations on the audit procedure. a methodology to conduct a knowledge management audit an audit is typically intended to ensure that the goals of the organisation are met. if the goals of an organisation are to replace one system with another to gain efficiencies and the system is fairly structured, the audit procedure may view the system as a "black box" and "audit around the system" compare the resources consumed and the reliability of the old and the new systems for viewing it from outside. if the system is complex, goals are somewhat illdefined, and reliability comparisons are difficult to make, then auditing may be performed by going "through the system" understand the key processes and their reasonableness in meeting the qualitatively defined goals. the auditing of expert systems, for example, when the knowledge is intended to support rather than replace experts has followed similar knowledge process analysis and reasonableness tests [agarwal, et. al, 1993]. the goal of the km audit here is to understand the processes that constitute the activities of a knowledge worker, and see how well they address the "knowledge goals" of the organisation. the knowledge processes discussed by probst, raub and romhardt [2000] are shown in figure 1 and these are used to perform the knowledge audit. it is top-down in that the knowledge goals provide the basis for evaluating various knowledge processes shown within the box. these processes are inter-dependent and consequently, one should not focus on one or a few of these processes to the exclusion of the others. both knowledge goals and knowledge assessment are outside these knowledge processes practiced within the organisation. knowledge assessment, similar to knowledge audit, is an evaluation of how well various knowledge processes are used to meet the knowledge goals. while an assessment can be frequent and internal to an organisation, an audit tends to be periodic and often driven by an external need. results of the assessment can feed into potential modifications to the knowledge goals or activities within the knowledge processes. in the remaining section, each of the elements in figure 1 is discussed in more detail. knowledge goals. knowledge goals give direction to knowledge management. in and of itself, the articulation of knowledge goals may provide a number of insights especially since this is likely to be a new activity for management. while the knowl24 ajis special edition knowledge management december 2001 edge goals should support the broader organisational strategy, the two should be distinct given that the knowledge goals should be concerned exclusively with knowledge processes. probst et al. identify three levels for these knowledge goals. normative goals create knowledge awareness and a climate for sharing. strategic goals define the organisation's core knowledge and identify skills that will be needed. operational goals are concerned with implementing knowledge management methods and making normative and strategic goals realisable. knowledge identification: this step calls for identification of the organisation's knowledge, which can be made known to members of the organisation on a need to know basis. many companies find it difficult to maintain a map of internal and external sources of knowledge and information. appropriate knowledge identification, known as knowledge transparency, enables members of the organisation to locate what they need without inefficiency or duplication of effort, and lessens the likelihood of uninformed decisions. knowledge acquisition. organisations can import knowledge from the 'outside'. relationships with customers, suppliers, competitors, and partners hold the potential of providing knowledge. it is also possible to buy knowledge from various firms. in this process, we need to identify opportunities for knowledge acquisition and find out various external sources, from which knowledge and expertise should be sought. knowledge development. knowledge development focuses on developing new skills and knowledge internally. it is an activity that complements knowledge acquisition. it asks how can an organisation generate better ideas, new skills, better processes, and new products. knowledge development includes all management efforts aimed at internally producing capabilities that do not currently exist in the organisation. knowledge sharing and distribution. this process seeks to apply knowledge where it is needed. to do this, it is necessary to develop means for transforming isolated experiences into something the entire organisation can use. given that it is not possible for anyone to know everything about everything, what can an organisation do to insure that knowledge of critical areas is gathered and shared where and when appropriate. an important activity is to understand how the transition of knowledge from the individual to the group takes place. knowledge retention. a number of organisational activities take place that impede the retention of important knowledge, and these include the loss of an employee, mergers or acquisitions, reorganisations, or a failure to update and store knowledge of future value. there are a variety of means for storing organisational knowledge. knowledge assessment. the way that knowledge management goals are established will directly affect the way they are measured. methods for knowledge assessment must be tailored specifically to the organisation in light of the way its knowledge management strategy is formulated. unlike areas such as finance or quality management, there are no set indices or measures for knowledge management. some creativity must be used to establish means of assessment that will measure whether knowledge goals are fulfilled and how they contribute to the organisational mission. the framework described in figure 1 is general and action-oriented in that it enables the implementation of knowledge processes using a variety of tools and techniques, so that the knowledge goals are met. in addition, the application of the framework will help an organisation articulate its knowledge goals in concert with its overall organisational strategy as well as support for knowledge work, and lead to the development of criteria for measuring success. given that the knowledge management activity is articulated as a set of processes that are used to reach a set of stated goals, the knowledge audit then looks at how well each of these processes address the goals. 25 ajis special edition knowledge management december 2001 knowledge knowled goals assessm i l 1 ' knowledge knowledge identification retention / \:/i v/ '• •••-. **• .•' : .̂ / \ ->c / : x. / v>c i \ * --. / k.... x \ / 1 -••knowledge ,." \ / / kno ° "--.... \ / ; •* acquisition " -.,,„ a / , -" uti \ '\, / \ ,--f * / ,, \ 1 / /, -" \. \ 1 / •̂•" --<i / knowledge ^ knowledge development distribution ge ent wledge ization figure 1 knowledge management framework the challenge for an organisation is to design a socio-technical system that will enable knowledge workers to be effective in performing various knowledge processes within the organisational context. levitt (1972) describes a model (see figure 2) that emphasises the importance of four related factors in designing a socio-technical system and these include, in addition to the task, the technology, organisational structure, and people. so, the knowledge audit needs to take into consideration the relationship of the current knowledge processes to the organisational structure, people and technology, before developing recommendations for process change. in a later section, the same model is also used in recommending process changes. this, hence, leads to a methodology for conducting a knowledge management audit. 26 aj1s special edition knowledge management december 2001 organizational structure task people figure 2. process change model a methodology for knowledge management audit articulate knowledge management goals, which include normative, strategic, and operational goals. describe each knowledge process (shown in figure 1) and its relationship to the organisational structure, people and technology (socio-technical environment). assess how well each process in this environment is fulfilling the knowledge goals articulated in step 1. next section will discuss an application of this audit methodology to an organisation. km audit of atib organization applied technology in business (atib) program is established by the school of business administration at a midwestern university. business students, who may come from any major are competitively selected to participate in this program and, upon completion, will be awarded an atib minor. the program's primary mission is to enable the students to gain the skills they need to apply information technology to solve real world problems that are provided by the program's corporate sponsors. the students are the "knowledge workers" that use "corporate projects'" as the primary vehicle to learn and apply many of the skills they acquire in the classroom related activities. in effect, "corporate projects " act as the key source of knowledge that is shared by the three primary stakeholders (students, faculty and industry). given the central role these projects play, this "project knowledge" becomes a key success factor. even though the atib program is established within an educational setting to support student learning, its need to provide "project knowledge" for workers and retain as much of that knowledge as possible before the students leave the program poses many interesting challenges that can be generalised to many industrial settings. the atib organisation had finished its third year, completed 150 projects, and graduated 34 students when the initial audit was performed. since that time, the program has completed 50 additional projects and graduated 16 more students. figure 3 illustrates how the knowledge workers (students) interact with various knowledge providers (director, faculty/staff, project sponsors, seniors and other students) to obtain "project knowledge" that is needed to complete their projects. the atib laboratory plays a crucial role in supporting the knowledge workers. this layout is shown in figure 4. the lab acts as a meeting place for many formal activities (such as workshop presentations), informal get-togethers and team meetings of students. in a way, the lab acts as a social setting for bringing many knowledge management activities together. the organisational structure is flat. all the students (junior and senior), faculty, and sponsors have a direct and open line of communication to the director. faculty and director together provide coordination on corporate projects from the academic side and a corporate sponsor provides coordination of the project from the industry side. senior students also coordinate and/or mentor projects of the juniors. 27 ajis special edition knowledge management december 2001 knowledge goals (normative, strategic and operational) at the time the knowledge audit was initiated, atib had not defined any of its knowledge goals. the following represents a first cut at inferring both the normative and strategic knowledge goals for atib. in order to support the program's mission [develop adaptive, technology literate, innovative business professionals], the knowledge organisation, has established the following normative goals: 1. create an inter-disciplinary orientation where the student (knowledge worker) develops a respect for diverse knowledge rather than viewing knowledge from the standpoint of a single 'functional silo'. 2. create an organisational environment where constructive sharing of knowledge and cooperation are practiced in support of the atib organisation. 3. create a learning culture that encourages seeking new knowledge and sharing it with others. inter-disciplinary orientation the strategic goals that support the creation of an inter-disciplinary orientation are: recruit students from multiple functional backgrounds and interests into a common program that has a single mission effective utilisation of it. form teams with students from multiple disciplines to solve problems of all types posed by corporate sponsors from different industry. select projects that require students to draw upon diverse knowledge. use case studies and class discussion to reinforce the need to solve problems with an organisation wide focus as opposed to functional focus. organisational orientation the strategic goals that support the creation of an organisational orientation are: engage students in activities and put them in positions in support of the organisation's objective of supporting each one's learning and meeting sponsor needs. create opportunities for each one to present their views and discuss their projects and challenges in a forum that encourages constructive dialogue and discussion. facilitate ease in sharing of knowledge, thus giving each one an opportunity to act as expert in some areas in support of all the others. learning culture the strategic goals that support the creation of learning and knowledge culture are: recruit program staff that recognises the need for constant learning and that have a willingness to share what they know with others. encourage learning from failures and successes by supporting the sharing of lessons learned with others. encourage mutual support and remove any inhibitors for individual growth. mapping of knowledge processes to program activities the program-related activities are mapped to the knowledge processes (shown in figure 1) in table 1. one noteworthy result is the absence of any program-related activities for knowledge acquisition. here, the knowledge acquisition refers to purchasing or otherwise acquiring knowledge from outside the organisation. since the main focus of the atib program is the development of students' skills and knowledge, there would be little if any knowledge acquisition. 28 ajis special edition knowledge management december 2001 table 1. atib activities addressing the knowledge processes a knowledge worker (student) perspective learn from class lectures learn from lab exercises learn from workshops/conferences be a leader of a project team be a member of a project team access project libraries complete a project prepare corporate project reports present at workshops/conferences update project libraries prepare tutorials vlentor other students interact with atib director interact with atib faculty/staff interact with seniors/other students interact with sponsors knowledge identification x x x x x x x x knowledge knowledge acquisideveloption ment x x x x x x x x x x x x x knowledge utilisation x x x x x x x x knowledge knowledge distribution retention x x x x x x x x x x x x one of the authors used graduate students from a km class he taught to conduct a knowledge audit of atib as a class project. the students were required to interview various atib stakeholders (including students, former students, faculty, and corporate sponsors) in order to understand how atib students performed the knowledge processes discussed previously. students were organised into teams to conduct the interviews in three phases. each student team consisted of four members. each interview was conducted by two students, one who asked questions, and the other who served as a scribe. the interviewers used e-mail for follow-up questions. during the first phase, a student team interviewed the director to determine normative, strategic, and operational goals. the second phase covered knowledge identification, acquisition, and development. the third phase covered knowledge distribution, utilisation, and retention. interviewees for the second and third phases included the director, current students, former students, faculty, and corporate sponsors. what follows in this section is based on the results of the final report from the graduate km class. audit observations knowledge identification much of the knowledge needed to complete a project is identified on an ad hoc basis (as dictated by the project), and the director, faculty and senior students play a significant role in this process. the physical layout and the flat organisational structure support a great deal of informal interaction that often facilitates serendipitous knowledge discovery. in addition, there are a number of functions that take place during the year that serve the dual purposes of socialising the students and familiarising them with the work of other student teams and the sponsors' business. in spite of the fact that the role of technology is somewhat limited, the culture permits networking and mutual project support for students seeking the needed knowledge from others. knowledge acquisition as noted previously, knowledge acquisition plays almost no role for the atib students since it refers to acquiring knowledge and skills from outside the organisation. 29 ajis special edition knowledge management december 2001 knowledge development while this process of the km framework is well developed, the dynamics of the field (it) and the diversity of projects makes the knowledge content incorporated into the program somewhat fluid. the role of faculty in the knowledge development process is not well articulated today, and the role of technology (such as the use of self-paced training material) is not well integrated into the program. also, the program today relies on the self-motivation of the knowledge workers and the requirements of the corporate projects for the overall knowledge development process. in the long run, a set of metrics are needed to ensure a pre and post assessment of the knowledge developed by the students. knowledge utilisation again, as hi the development phase of km, the degree to which a student utilises his/her knowledge is dependent on the project they are asked to complete. given that there is no scientific way the projects are assigned to students today, the degree of knowledge utilisation (other than general project management knowledge) varies with each student. knowledge distribution and sharing the role of technology for knowledge distribution is somewhat limited to storing status reports, email communications, powerpoint™ presentations, etc. while these provide a way to distribute knowledge to those in the future, it is not well indexed for easy search and dissemination. also, there are not many opportunities for abstraction and generalisation. knowledge retention while the mentorship program is an effective means of distributing knowledge from seniors to juniors, there is no concerted effort to preserve much of this knowledge in a static form that can be used by others. for example, there are no de-briefing sessions for seniors leaving the program. technology has not been incorporated as a means of storing and indexing lessons learned in a searchable format. in addition, it may prove to be useful to categorise other project documentation with sub-categories relating to features of the project, business or information technology. in order to support such a change to the technology for knowledge retention, it would be necessary to modify the affected parts of the way that projects are carried out and to insure that student inputs to the database were part of their evaluation. seeking synergy between knowledge goals, processes and socio-technical environment (steps 1 and 2) one of the critical aspects of a knowledge management audit is to identify disconnects between the socio-technical environment, the generic knowledge processes, and the knowledge goals. in this way, the audit points to changes in the socio-technical environment and the generic knowledge processes needed to support the knowledge goals in a way that is very similar to creating a change management culture that sustains process redesign effort for radically improving corporate performance during process redesign. both the knowledge sources and organisational structure (see figures 3 and 4) and the discussion above, highlight a few major program attributes (listed below) of the sociotechnical environment of the atib program that, we believe, will continue to support the knowledge processes and their meeting the stated knowledge goals: support of open interaction among students as well as among all the program's constituents: faculty, sponsors, alumni and students, using a variety of informal settings for communication and sharing of knowledge; support of new technology exploration without fear of failure and sharing that knowledge with no perceived threat of losing competitive edge (as evaluation is based on project success and individual learning rather than an evaluation on a curve); recruitment of students from multiple disciplines and assigning them to projects with diverse scope and coverage; and recruitment of sponsors from diverse industry to ensure that business and technology changes will continue to infiltrate into the program and keep it current and learning oriented. 30 ajis special edition knowledge management december 2001 next section will provide a summary of the audit observations and make recommendations for changes in the knowledge processes. figure 3. a knowledge worker perspective knowledge sources di reel or faculiy/staft project sponsors seniors others students corporate project successfully completed project class lectures lab projects project workshop review & research mentorship & training knowledge worker activities figure 4. physical layout director & administrative staff atib lab class & workshops atib support faculty atib sponsors (geographically dispersed within an average of 15 mile region recommendations from the km audit overall, the results of the km audit can be summarised within three categories: with respect to knowledge strategy, within the social system, and within the technical system. from a km vantage point, atib's social system functions well. consistent with this observation, atib relies on a network model in that knowledge is regarded as resting with the individual and is transferred primarily through person-to-person interchanges. the audit suggested weaknesses within the technical system and with regard to unarticulated knowledge goals. the audit suggested opportunities to use technology for knowledge development, knowledge use, knowledge sharing, and knowledge retention. prior to the audit, atib had not explicitly considered knowledge goals. consequently, it had not made knowledge goals integral to its stakeholders and its day-to-day activities. since the audit was based on an analysis of generic knowledge processes, changes aimed at improving them should be considered in light of research on "process change". process reengineering methodologies emphasise the importance of aligning information technology, organisational structure, job design, rewards and controls with the changed process. (see eg. hammer & champy 1993, kettinger et al. 1997, klein 1994, rummler & brache 1995). the importance of organisational culture and organisational values as part of the common ground for change is also noted by some of these methodologies. however, as davenport et al. (1996) note, there are significant challenges in applying traditional approaches to process change to knowledge work and knowledge workers. the challenges they report include: a) the variety and uncertainty of inputs and outputs, b) the unstructured and individualised nature of the work, c) the lack of clear separation among inputs, process, and outputs, d) the lack of measures of process effectiveness, e) worker autonomy, 0 high variability in performance across individuals and time, and g) lack of information technology support. in spite of these difficulties, there are 31 ajis special edition knowledge management december 2001 important features of reengineering methodologies that can be incorporated into a methodology for the analysis, design, and implementation of a knowledge management system. the emphasis on harmonising the social and the technical system is an important feature of bpr methodologies, and this may be carried over to knowledge management. however, the strong process centric orientation may be less suitable due to a number of differences between successful knowledge management projects and the typical successful bpr project [davenport et al. (1996)]. differences they noted included the absence of ambitious stretch goals for performance improvement and new designs that focus on features rather than actual processes. bpr methodologies based on the model described in figure 2 identify process (task) as the primary focus of change, and identify technology (usually it), organisational structure (how communication and reporting occurs), and people (job design, including rewards and training) as infrastructure factors, which must be addressed as part of this change. in other words, if a process is changed without complementary changes to these infrastructure areas, the process change will be ineffective. given the difficulties of process change in general, and knowledge process change in particular, a cautious incremental approach was favoured over attempting to institute radical change. changes were to be based on the following three recommendations. three major recommendations the audit shows that the knowledge organisation has been operating relatively successfully using a "network model" of informal links among various stakeholders such junior and senior students, sponsors, faculty and alumni to complete many of the knowledge processes, and the role of the "repository model" is relatively minor. while this alone is not inappropriate for a small organisation, over a longer period of time, as the knowledge base increases in size and the knowledge resources (people) move far from the knowledge organisation, a more formal "repository" model may be relevant even if the network model continues to play an important role in the success of the program. second, since the knowledge organisation has evolved over its three years (at the time of the audit), many of the goals, strategies, etc. have been evolving as well. while this is not atypical for a new organisation, these need to be articulated and disseminated effectively to various stakeholders. such a dissemination allows for a stakeholder understanding of the knowledge goals/strategies and their potential input in any refinement to these goals/strategies as the environment changes. this can also support an eventual distribution of some of the roles and responsibilities of running this organisation to others. lastly, while the program appears to be successful in accomplishing its goals, many of the comments related to these accomplishments are anecdotal. more formal metrics have to be developed and instituted to ensure that the accomplishment of goals, quantitative as well as qualitative, are tracked over time. next section will discuss the implementation of the three audit recommendations. implementation of audit recommendations repository support of project knowledge an atib portal was developed and implemented in may-august 2000 (four months after the audit) and several of its features are shown in figures 5-11. portal features: figure 5 provides a list of features that are available to the administrator, which include adding new sponsors, defining projects, assigning students to sponsors, etc. some other administrative features such as assigning grades, providing access to the portal to guests and sponsors, etc. are also included. 32 ajis special edition knowledge management december 2001 figure 5 opening panel and portal features 3 the pottal administrator home page • micioioft internet explo f>. £<8 v>w favorites look jjjlaek f.y^_ s!°p ' rehash i home £ search^ favorite* histoiy, _[ mai -p™*_ e« oocuk : :j adders |€] h«p://a(ib.!ba.oaklandeowthepoital'inside/nde»l asp •:&:\r-4f.vfftij.$-x.!i edit bulletin board add/edit projects add sponsor faculty add/edit inactive projects 1 edit calendar (id-at.b paf.-.ub) | student/project assignment | update sponsor [ guests report generator remove student change alumni status student passwords welcome mohan) have a nice spring/summer "| |tf inter** the porta. ;b]documanl3-...| 10-.30am project status reports: figure 6 provides a screen that lists status reports of all projects that are under execution in any given semester and by highlighting one of these, the students as well as a sponsor can get access to specific activities that have been completed under the project in all the weeks since the project started. these status reports are moved to archival storage at the end of each term and become a pan of knowledge base, which can be accessed by students at a later time. effective search mechanisms, besides the name of the corporation or team-leader are being explored. figure 7 provides a sample of the detailed status report for a week and figure 8 provides the project tracking report, ie. where a student is on a project based on a set of steps that were a-priori identified. 33 ajis special edition knowledge management december 2001 figure 6 project status reports search favorites histaji ' ha] print e« dwcuss ij address |€1 nip //atib.!ba.oaklandedu/thepoital/inside/ind<!»1 .asp unkj», rh^^jfc^?'<:\i^r;> •v*j v \f j sign out. « ,*, ̂ x -t,u.;.r. ,• . " ' s , ' f'^.''-".' f*'fl * "'' ' ' ^-l^, --— 7 ,, . tfiepor«^i. v 1 ^°| add documentation view all tutorials * 4» jmnx ** i please select one of the foil owing documents: t$ft*ftatl«0aii& 'access tutorial la f»oj«ltrac.tin$ < access tutorial ib r '̂ttoftfjfii&ljfte^ * * a-. -a -1 ^ retiftttw • '' access tutorial ic •"an , ̂ iltookten.»r«n asp tutoria| ..ta»î aefi«» , crystal reports tutorial *""*** how to make access • catehtaitt ' query seek inout from the tft.tropm ', ^^ ' 44swhw14i4*'" visual basic tutorial 1 i ^ f̂tdwlĵ vv 'visual basic tutorial 2 *" ^rv a t v*", ' svisuol basic tutorial 3 leam how to create a simple table in an access 97database learn how to create a simple query and form in an access database learn how to create a dropdown combo box in an access 97database short tutorial on asp technology for beginners short tutorial for beginners in crystal reports 7.0 it will allow the user to input values of a parameter, which will be used by a query to extract information from a data base part 1 of visual basic documentation. part 2 lesson on vb technology part 3 lesson on vb technology _!*. _j_zi_ . . . . . 3 €]catand» ifflnbmi outlook .. [ jgqmicrotoltpowa | bjjathecnrcnendat j @jdocurnent2-hk: .||cjthe portal .. figure 7 details of a status report associated with a particular project j 0e &fl view favaitej look help j assess ig] blip //abb sba oakland.edu/thepottal/inside/index1.asp the i taam members: ' \ brian meitzner ~ class of 200° julie maves " class of 1999 i michael nicolai class of 2000 accomplishments: . finished project. presented to eds sales team on 4/11/01. discussion forum .p^^* , • make final presentation to atib sponsers and students. cjutroom ' documentation eds markting and portfolio management status reports archive: " 4/19/01 status report 4/10/01 status report " 3/30/01 status report ;̂ j resumes • g<start) qlnbex quite. | hjmicrccthpo. | qpoouraentz||a]the porto... |qoocuwrt3-...| 34 aj1s special edition knowledge management december 2001 figure 8 project tracking ni id at 01 home page • miciosott inlernel txploi'3 ' t>c i'oilti jjj £fc e« y»» fawritw loch llob *• back 9' slop el refresh hone q sa "' smtc*i favnio pirn edt dbcun |lr*c» sign out. v. description: simulate the space allocation based on user requirements. support decision making related to surgical patient routing and holding area utilization. simulate the routing patterns for patient flow and identify effects on capacity. (see mark isfcen * for m trjchnologies in use: simulation package summary ^1 • project completion t;a project detail ^:iia^jgj*j ; s tfcafttav '•' '• : ' . ;•, "ri 'ttm*il.ij* * i ~ v > ' . . • • • , ••••;• ox-ii • : • ' f ' • ".' ;. ̂ : ccrtacu step name 1) develop a project work plan: detailed planning identify project goals: formulate conceptual idea of tools' function read history or project and simulation methods 2) deyalop functionality of the model _ • . . . number of days 4 7 7 60 ! "• i i status complete complete complete complete 0 internet i !^bsiart|'g^inbox oute. | [ajmiaotoll po. | g|0ocunert2 ,.||gjtho porto... igjooorong•..\ documentation: figure 9 provides a list of tutorials and special reports (including past project summaries) that have been developed each term. these can form the basis for self-paced learning and act as reference documents to refresh one's knowledge about a technology when it is called for. 35 ajis special edition knowledge management december 2001 figure 9 documentation '5 the portal administrator home page microsoft internet explorer hif-3 e3 £*» £* yjew fjvoita loob hjefc l̂ l̂ back ' -ui'<i'd " & el 4 i ® 23 0 l@r s* s . y stop flcheih ham | semch favmies histogi i ms) bint eeh oiku» j ajj<*e» [<gj htlp /valib *a«*land.edu/th(#ortal/imkfcaidcx1 .asp vw ;̂ ? ^ * i " % i j »/ •>$$& \. ^ mwnfe/'ljl suhjiroport* * |5r<i|cw3ms«rtdfi$|sw> -j \ oow««*jw ' t,' tsj* "s ' ^ v>. ^1 fii r̂ knufon foî ujlft > ••,*no«1«)9o&)iie' •: : ««*» j *mnit«>* ^ *• x ' j1^. » '1 vv * sigh out. «*reh j sj: j^go' jj lhu * 7 itie font®*.! •uq| add documentation view alii tutorials 1.j please select one of the following documents: .access tutorial la access tutorial ib [access tutorial ic i 1 asp tutorial learn how to create a simple table in an access 97database i leam how to create a simple query and form in an access database learn how to create a dropdown combo box in an access 97database short tutorial on asp technology for beginners 'crystal reports tutorial (short tutorial for beginners in crystal reports 7.0 how to make access 'query seek input from the visual basic tutoriall visual basic tutorial 2 ivisuat basic tutorial 3 it will allow the user to input values of a parameter, which will be used by a query to extract information from a data part 1 of visual basic documentation. ;' part 2 lesson on vb technology i part 3 lesson on vb technology ** fjcdbndar tt"i¥ qlnbox outook lahbrecomroendalj mk..||g] the portal -... ; ittmam knowledge base: figure 10 provides a list of projects that are being worked on for a particular sponsor. this information is used to get a general idea on the type of projects a firm may engage in, especially if a new sponsor is looking to define projects in an area similar to theirs for potential students. 36 ajis special edition knowledge management december 2001 figure 10 knowledge base back rowi'd slop rehatti hobo ! smch favofho hstoiy.; md print edt t~[~l* mo"* •v.v:..? ..•/ m^, .h.m-.n.'-i1'-*'''' ' • v.'v.':,'' . ' . . i s1gnout. ji'.af, '-'.i.••.;•;;] projects for eds: . ' tame 1 i project name '^;/w*.% •'•.. 1 '' global proiect i'̂ sfs^sr'svcj! mmanaseaierjijjsng : ' u's? v̂"*ku '?' jeds martctino and ''' !"""'rj«u>m* ~'/! •. ! portfolio management '• i*?t*,r?tk ^ i information solutions l î̂ ^̂ ^ku l̂̂ 'i :^ midwest sales ''•-,• ijw«mfe. 'rt '1 i web basecl '•'''r̂ ^s* •* " ' • i c°rt'mun'ca"on c *v'i '' 1 i.*0*'̂ '?'1^1"?''' •'] 1 streamlining the • v *ifciniju w»';;" "•" ^ reporting for gm ; :.;;;̂ r-*.v ''••?;?\ ; easier* jj—iiii... :'.-x. j . ' ..••::'-': a •:v;:.'. '-;:-:>;. :''•:?: .'•'.•:'-'•' , ;-:i-i'-::n::.-.v' . • . v' t : tnepoit date project description i develop an application on intranet that can capture 10/6/00 i project information for access by multiple participants ! across the globe. 1/11/01 ^ match customer characteristics with eds capabilities i for developing targeted marketing ; create a repository of value proposition for easy 1/11/01 : retrieval and use. i continue the project from last term., support web1/23/01 ; based interaction among various team members in a i project team. i analyze the current reports and provide better user 2/11/01 i interface using vbasic to generate and display various i reports to gm dealers. &lt j ^. igbslait| £3lito,-0ufcck . | |o)miciaaapomb. | bjdocunert2 me. ||el the portal . . iocs am discussion forum figure 11 provides a discussion forum that can be used by students and sponsors to exchange ideas on a topic of interest. questions can be posed by a student and can be answered by alumni, seniors or sponsors that have specific knowledge on the topic. note that not only does the discussion forum provide support for internal communication but it also supports one of the key features of the atib program: networking students with alumni and sponsors. the discussion on an advanced program in figure 11 is an exchange between a student, alumni and a sponsor. in addition to these features, the portal also provides a contact list that includes email, phone or fax, resume information, etc. 37 ajis special edition knowledge management december 2001 figure 11 discussion forum julie is a student, peggy is a alumni and john is a sponsor •'5 the poital administialoi home page miciosott internet exploier he3e3 jj fjte e« view fayoites loots help j back pv»prd stop heftesh home |! ;̂ g 23 .'^s" i seach favorites notary j address |£] htlp //affl) sfaa oakland eowtheportal/mside/indexl asp ' \ ^vw,*^ t" *v * c •. sign qth".j ' " " * ' ( ^ v 1 \ * 't. , from n. v|wi "* date *>jj$|liift»"i posting status reporu, _ j cmttmttfl *re«umeft ^ ! „ *,*¥ ' ,' ' <**,< john cipriani 11/3/00 '. : ,1 mai pint e<8 ~ discuss • _^j f>go j links" —""» ' i ne r*wi ca;i jii i have a ms in info systems management and have found that the masters of science approach combines the mba methodology with strong technical background. i would recommend it to those pursuing an it specific career versus the more generalist mba. remember that your _ foundation (educational or experiential) does not determine your success ; but rather what you do with it. ctutraom 5 from julie , date 10/1/00 , posting for the individual who has the technical experience and wants to focus on business management this seems to be the perfect solution even with 1 an mis background. i guess my concern is what will be the critical differences for student with an is background? would he be in their best i interest to go for the mba instead? ;, [ from peggy 'g] discussion foam ^9 start | ggunbojtquito| microsoft pp. [ |qdocuroent2-.|[qthe porta... @j document ...,| | \<& internet 10:19am streamline articulation and dissemination of knowledge goals and strategies in an evolving organisation, some of the goals and strategies may be initiated by those in-charge. even so, it is critical that these are shared among various stakeholders for review and refinement. this may enable the eventual redistribution of some of the roles and responsibilities of the organisation to others and the creation of some degree of redundancy to mitigate some of the risk to the organisation. a web-based framework was developed in april 2001 and will be used to communicate many of the program's goals, strategies, and processes to various stakeholders to get their reaction and increase their involvement. figure 12 provides a sample framework that is used to highlight various features of the program and its stakeholders, and the matrix provides hyperlinks to dissemination status, responsibility assignment, etc. 38 ajis special edition knowledge management december 2001 figure 12 atib continuous improvement accreditation process project features and stakeholders b«i at.fik.1 swbrte* ha» i scat* footo ham, mj . p*« tdl dikun director ^faculty students spcnsc i. mission. > i < '< a processes x ii. strategies x x x iii. curriculum ; x x x ;iv. physical settjng x | x x development of metrics since some of the strategic goals call for inter-disciplinary orientation, learning, and knowledge sharing, measuring the degree to which an individual knowledge worker has accomplished these goals is difficult to measure. while many of the strategic and operational goals are intended to support these normative goals, one still needs feedback mechanisms (formal as well as informal) to gather such information after a student leaves a program. three different mechanisms have been put in place to gather the needed information at different levels of granularity. first, a project survey instrument was developed and is now sent to students at the end of each project to assess their perception of success and to ascertain what features characterise the project (problem complexity, team maturity, faculty and sponsor coordination, etc.). the aim of the survey is to evaluate the degree of impact different project features have on project success and ultimately to influence the manner that students are selected for projects. a similar survey is now sent to corporate sponsors as well. second, a formal survey was developed and will be sent at year-end to various alumni and sponsors. its purpose is to determine their level of satisfaction with the program/projects in meeting their expectations. the survey is administered to get students' reactions after they leave the program and to measure long-range effects of the program on students' careers and sponsors' projects. third, a formal albeit qualitative forum has been established as a part of a yearly conference, enabling various alumni to reflect on their collective experience since they left the program and the way their current work relates to experiences in the program. this is yet another way to understand the role of the knowledge sharing and learning culture that was embedded in the program on a student's career in the long run. in summary, the knowledge audit conducted one year ago has highlighted several important and critical aspects of atib as a knowledge organisation. some aspects, the effective use of the network model and the cross-disciplinary culture, have contributed to the overall success of the program. it, however, has highlighted other measures the organisation should undertake to streamline various knowledge processes to support both repository related activities and continual assessment. many of these recommendations have been put in practice and a future audit may be used to assess their impact. discussion and conclusion some specific probes that have provided motivation for developing km systems in the past include the acclamation of new knowledge workers and mitigating the loss of intellectual capital through the attrition of key knowledge workers. by thinking of atib more broadly as a knowledge organisation, we gained insights into these two related problems. in addition, by conducting the knowledge audit, we are able to offer some observations concerning methodological issues: the usefulness of knowledge audits for establishing knowledge requirements, and the relationship between network and repository models. 39 ajis special edition knowledge management december 2001 acclimating a new knowledge worker to an organisation. atib is quite successful in its efforts to introduce new students to its program. the design of training experiences so that responsibility and independence are incrementally shifted from external sources such as faculty and sponsors to the student is a key factor in developing technology capable project workers. the use of seniors in a mentorship role with juniors is vital to the success of this incremental approach. atib supports productive social interactions through the physical design of the working space and through holding a number of quasi-social functions throughout the year. atib makes extensive use of workshops and conferences where students can share the results of their work with other students, faculty, and sponsors. firms interested in instituting knowledge management programs could incorporate such techniques. it can be socially rewarding, personally satisfying, and intellectually valuable to present the results of a successful project to one's peers. lessons learned conferences can serve as a means of preserving organisational knowledge while simultaneously enabling the development and strengthening of interpersonal networks. planning for the loss of critical intellectual capital. for atib stakeholders, thinking of itself as a knowledge organisation where the graduates represented a loss of intellectual capital, was a new concept. while atib's 50% annual turnover is more extreme than the typical organisation, modem firms, especially in the dot.com high tech world, are faced with the problem of preparing for the contingency of losing a key employee. atib has just begun to plan for this, but it appears that building redundancy among different students through mentorship will help to mitigate most adverse effects. there is a clear relationship between problems of attrition and those of acclimating new workers. industries characterised by high levels of worker mobility such as it will be faced with both problems on an ongoing basis. furthermore, trends towards 'free agency', the rise of the independent agent who sells some form of personal expertise, will make these problems more salient. effective solutions to this pair of problems can provide substantial benefit for knowledge organisations. incorporating a knowledge audit into a knowledge management methodology. alavi (2000) suggested two important issues for km system design: 1) the development of methodologies for identifying factors that influence the knowledge management requirements of the firm and 2) understanding the relationship between knowledge requirements and km model (repository or network). the knowledge audit has something to offer for both of these key questions. by forcing an articulation of knowledge goals, the knowledge audit moves stakeholders toward an understanding of organisational knowledge requirements. knowledge audits and assessments are iterative activities. thus the knowledge strategies as well as the generic knowledge processes undergo change as part of each iteration. ongoing evaluation of requirements and the means of satisfying them are part and parcel of each iteration and are kicked-off by the audit. because of the contrasting ontological assumptions about knowledge, one may be tempted to regard the repository and network models as being incompatible, that a firm can opt for one or the other. spender's (1996) model offers insight here. knowledge that is social and explicit is amenable to codification and storage in a repository. knowledge that is social and tacit is the basis for organisational culture, values, and beliefs. the network model enhances the exchange of this type of knowledge. the atib portal is an attempt to support both models by storing organisational/explicit knowledge in a repository and by enhancing and enlarging existing networks through discussion groups and providing extensive contact information. references ackerman, m.s. (1996). definitional and contextual issues in organizational and group memories, information technology and people, 9,1, 10 24. agarwal, r., kannar, r, and m. tanniru, "formal validation of a knowledge-based system using a variation of the turing test", expert systems with applications, 1993, vol.6, pp.181-192 alavi, maryam (2000). managing organizational knowledge in framing the domains of it management, robert w. zmud ed., cincinnati: pinnaflex educational resources, inc. alavi, m and d. leidner (2001). conceptual foundations of knowledge management and knowledge management systems, mis quarterly. cook, scott d. n., brown, john seely (1999). bridging epistemologies: the generative dance between organizational knowledge and organizational knowing, organization science, 10, 4, 381-400. 40 aj1s special edition knowledge management december 2001 croasdell, david t. (2001). it's role in organizational memory and learning, information systems management, 1 8 , 1 , 8 i i . davenport, thomas h., jarvenpaa, sirkka l., beers, michael c. (1996). improving knowledge work processes, sloan management review, 36, 53-65, summer, 1996. davenport, thomas h. (1999). knowledge management and the broader firm: strategy, advantage, and performance, in knowledge management handbook, jay liebowitz, ed., boca raton, fl: crc press. dierick, i. and k. cool (1989). asset stock accumulation and sustainable competitive advantage, management science, 35, 12, 1504-1514. drucker, peter f. (1978). the age of discontinuity, harper & row: new york. drucker, peter f. (1999). knowledge-worker productivity: the biggest challenge, california management review, 41, 2, 79 94. duffy, jan (2000). the km infrastructure, information management journal, 34, 2, 62 66. fancy, l. and l. prusak (1998). the eleven deadliest sins of knowledge management, california management review, 40, 3, 265 280. grant, robert m. (1996). toward a knowledge-based theory of the firm, strategic management journal, 7, winter special issue, 109-122. hammer, michael and champy, james (1993). reengineering the corporation, new york: harpercollins publishers hansen, m.t., n. nohira, and t. tierney (1999). what's your strategy for managing knowledge? harvard business review. march-april 1999, 106 116. kettinger, william j., teng, james t. c., and guha, subashish (1997). business process change: a study of methodologies, techniques, and tools, mis quarterly, 21,1, 55-80. klein, mark m. (1994). reengineering methodologies and tools, information system management, spring 1994. leavitt, harold j. (1972). managerial psychology: an introduction to individuals, pairs, and groups in organizations, chicago: university of chicago press. lippmann, s. a. and r. p. rumelt (1982). uncertain imitability: an analysis of interfirm differences in efficiency under competition, bell journal of economics, 13, 418 438. nonaka, i. (1994). a dynamic theory of organizational knowledge creation, organizational science, 5, 1, 14 37. nonaka, i. and takeuchi, h. (1995). the knowledge creating company, new york: oxford university press. pfeffer, jeffrey, sutton, robert i. (1999). knowing "what" to do is not enough: turning knowledge into action, california management review, 42, 1, 83-108. polyani, m. (1967). the tacit dimension. london: routledge and kegan paul. polyani, m. (1962). personal knowledge: toward a post-critical philosophy. new york: harper torchbooks. probst, gilbert, raub, steffen, romhardt, kai (2000). managing knowledge: building blocks for success, chichester: john wiley & sons ltd. rummler, g. and brache, a. (1995). how to manage the white space on the organization chart (2nd ed.), new york: jossey bass, inc. sena, james a., shani, a. b. (rami) (1999). intellectual capital and knowledge creation: towards an alternative framework, in knowledge management handbook, jay liebowitz, ed., boca raton, fl: crc press. spender, j.-c. (1996). making knowledge the basis of a dynamic theory of the firm, strategic management journal, 7, winter special issue, 45-62. winter, s. g. (1995). four rs of profitability: rents, resources, routines, and replications, in resource-based and evolutionary theories of the firm: towards a synthesis, c. a. montgomery ed. norwell, ma: kluwer, pp. 147 -178. 4! the australasian journal of information systems (ajis) began in 1993 and was based on the simple realisation that there was a need for an academic ‘outlet’ for is research in australia australasian journal of information systems volume 14 number 2 june 2007 113 the alignment of business and information technology strategy in australia jovita gartlan strategy & operations deloitte touche tohmatsu, australia email: jgartlan@deloitte.com.au graeme shanks department of information systems, the university of melbourne email: gshanks@unimelb.edu.au abstract the alignment of business and information technology strategy has long been recognised as a key issue for managers and has grown in importance as it has become strategically significant. previous studies have noted the elusiveness of alignment of business and information technology strategy and identified a number of factors that promote alignment. this study builds on previous work and categorises the factors as either people, process or organisational. a cross industry survey of australian organisations is then used to determine the perceived benefits of alignment and the perceived importance of each factor and how successful it was in promoting alignment. the findings of the study show that australian organisations perceive that alignment is important and can bring considerable benefits. furthermore the study highlights the relative importance of the factors and indicates where organisations should focus their attention in order to successfully achieve alignment. introduction achieving alignment between business and information technology (it) strategies has long been a crucial issue for many organisations. there was much interest in strategic alignment in the early 1990s as it became seen to be an integral component of organisations (broadbent and weill 1993, henderson and venkatraman 1993. keen (1991) notes that “it has become an important aspect of everyday business. it is potentially a key element in competitive positioning”. it had moved from being an operational function to being a critical strategic organisationsal tool or resource that should be shaped to deliver business needs. the alignment between business and it strategies again rose to prominence in the early 2000s with the rise of ebusiness (choe 2003, pollalis 2003). in a 2004 us mailto:linda.melinda@ecu.edu.au� australasian journal of information systems volume 14 number 2 june 2007 114 society of information management survey, the number one concern for executives was the alignment of business and it (luftmann 2005). despite this interest, luftman, papp and brier (1999) found that 42% of executives of fortune 500 companies in the usa stated that their business and it strategies were not aligned. very few studies of the alignment between business and it strategies have been conducted in australia. the most significant australian study was broadbent and weill (1993) which focused on the banking sector. given the renewed interest in and importance of the alignment between business and it strategies and the lack of australian studies, we undertook a survey in 2005 that explored the alignment between business and it strategies in australian organisations and this paper reports the outcomes. this paper is structured as follows. the next section reviews key research in business strategy, it strategy and alignment between business and it strategies and provides definitions of key concepts. a set of factors that promote alignment between business and it strategies is discussed. the third section describes the research design and explains how the survey instrument was developed and how data was collected and analysed. the next section presents the results of the survey and discusses their implications. the final section concludes the paper and presents some suggestions for future work. alignment of business and information technology strategy business strategy there are many views on what constitutes “business strategy”. definitions for business strategy and its features include the following. a business strategy … • should encapsulate a statement of an organisation’s mission or vision so that there is a clear and consistent point of focus (king, 1978); • provides the ability to understand competitive complexities through a systematic approach with the aim of achieving competitive advantage. organisations need to evaluate both their internal and external environment in order to determine a position and market approach (henderson, 1989; rigby et al, 2002; henderson & venkatraman, 1993; barney 1992a & b; black & boar, 1994, porter, 1980; porter, 1985); • provides a deliberate plan of action (henderson, 1989; kaplan & norton, 1996: henderson & venkatraman, 1993, king 1978); • can be used as a tool for accelerating change by defining the directions to be followed to change the current state (henderson, 1989); • is partly formulated by performing environmental scanning and may be used as an analytical tool to predict future business risks and opportunities (henderson, 1989; kaplan & norton, 1996; beinhocker, 1999; porter, 1991; peteraf, 1993; barney, 1993; learned et al, 1965); • can be used to allocate company resources as decision makers can refer to it to ensure decisions are inline with an overall company focus (henderson, 1989; rigby et al, 2002; henderson & venkatraman, 1993; beinhocker, 1999; porter, 1991; porter, 1980; porter, 1985; peteraf, 1993; barney, 1993); australasian journal of information systems volume 14 number 2 june 2007 115 • can be used as an internal assessment tool which identifies strengths and weaknesses as part of the internal scanning process during strategy formation (porter, 1991; peteraf, 1993; barney, 1993; black & boar, 1994; learned et al, 1965); • calls for strong senior management commitment in its formulation and during implementation (henderson, 1989; kaplan & norton, 1996); • should include outcomes and key performance indicators to determine if the strategy is successful (kaplan & norton, 1996); • should be adaptable to enable organisations to be nimble and guard their core competitive advantage from competitors (kaplan & norton, 1996, porter, 1996:62); • is a rich channel of information which lessens uncertainty and ambiguity through its use of the planning and analysis process (broadbent & weill; 1993). we use the following definition of business strategy based on a synthesis of the definitions and key features above. a business strategy is an analytical management tool used for planning a future business path. it addresses the internal and external business environment, the approach to competition, vision and allocation of company resources and which calls on strong commitment in its formulation and execution. it strategy it strategy is similar to business strategy although the focus is specifically on technology. from an it management perspective the following additional components are specific to it strategy. an it strategy … • addresses the management of organisational hardware and software resources and enables organisations to support planned change in future directions and resources(broadbent & weill, 1993; king, 1978; henderson & vankatraman, 1993; gadiesh & gilbert, 2001); • defines how it will be used to facilitate electronic communication to support business processes and needs (broadbent & weill, 1993; henderson & venkatraman, 1993; clark, 1989; gadiesh & gilbert, 2001); • defines the management of data including approaches for gathering, storing and presenting data (broadbent & weill, 1993); • defines internal and external business relationships in order to structure liaison between the business and vendors (henderson & vankatraman, 1993); • includes management of it human resources to ensure that these are inline with the firm’s it strategy (henderson & vankatraman, 1993); • acts as a strategic business tool reflecting the relationship between business strategy and it strategy making the inference that it strategy is indeed primarily a business tool (simons & davila, 1998; kantrow, 1980; haeckel & nolan, 1993; amit & shoemaker, 1993). we use the following definition of information technology strategy based on a synthesis of the definitions and key features above. an it strategy is a strategic business tool used to structure a future path and addresses the use and management of it resources, business it relationships both internal and external and the flow and storage of information throughout the organisation. australasian journal of information systems volume 14 number 2 june 2007 116 alignment between business and it strategy achieving alignment between business and it strategies is an elusive task for organisations. luftman, papp & brier (1999:12) found that of the 1,051 executive responses received, representing over 500 us fortune 1,000 organizations, 42% stated that their business and it strategies were not aligned. there are a number of different perspectives on alignment. broadbent & weill (1993) referred to alignment of business and it strategy as “the extent to which business strategies were enabled, supported and stimulated by information strategies”. king & teo (1996: 309) defined alignment as the “coordination between the business and is planning functions and activities”. luftman, papp & brier (1999) argue that “alignment focuses on activities which management perform to achieve cohesive goals across the organisation”. a general theme that emerges is that alignment is synonymous with cohesive and concurrent achievement of mutual goals between business and it. the strategic alignment model (sam) of henderson & venkatraman (1990) addresses four domains: business strategy, it strategy, organisational infrastructure and processes and information systems infrastructure and processes. each domain contains key components, with the overall model having a total of eight key components. sam also includes the two dimensions of strategic integration and functional integration. luftman (1996) later extended sam to twelve components that define alignment between business and it. the twelve components are categorised into four broad areas addressing areas such as business scope, distinctive competencies, business governance, administrative structure, processes, skills, technological scope, systemic competencies, it governance, it architecture, processes and skills. overall, the model is comprehensive in defining alignment and relates back to the above general theme of concurrent achievement of mutual goals. king (1978) provided a different perspective and argued that it strategy should be directly derived from business strategy and thus a hierarchy of strategies emerges. this was later found to be flawed (king & zmud 1981, baets 1992) by the same writer who argued that as business strategy is ever changing, the processes of business strategy formulations and it strategy formulation need to be integrated, although the degree of integration will vary between industries. overall, alignment of business strategy and it strategy has remained an important organisational issue that has plagued management for many years. we use the following definition of alignment based on a synthesis of the definitions and key features above. alignment of business and it strategies involves the cohesive and concurrent formulation of common business and it strategies, and the process of formulating the strategies is reciprocal in nature. successful alignment between business and it strategy the success of alignment between business and it strategy may be understood from a number of different perspectives. mintzberg (1978) argues that in order for alignment to be successful it must be realised, that is, where business strategy is evident in it decision-making as opposed to remaining in a document. chan et al (1997). rockart, earl & ross (1996) simply state that successful alignment is understanding business opportunities through it. australasian journal of information systems volume 14 number 2 june 2007 117 broadbent & weill (1993) provide a more comprehensive definition and argue that successful alignment is exhibited through the outward display of competitive advantage or successful achievement of business goals from the use of information or it. they show that in order to achieve alignment an is strategy needs to be consistent with business needs, be flexible and the formation process needs to be issue-oriented with a view of different organisational levels (broadbent & weill, 1993). henderson & venkatraman (1993: 472473) added a financial aspect to the definition of successful strategic alignment stating that it is achieved when “economic performance is directly related to the ability of management to create a strategic fit between the position of an organisation in the competitive product-market arena and the design of an appropriate administrative structure to support its execution”. they argue that strategic fit is dynamic and “not an event but a process of continuous adaptation and change”. we use the following definition of successful alignment between business and it strategy based on a synthesis of the definitions and key features above. successful alignment between business and it strategy is evident where both it and business strategy can demonstrate a planned alliance which then leads to tangible, successful, business-focused outcomes. importance of alignment between business and it strategy it is having an increasingly important influence on a business environment that becoming more complex and uncertain ((amit & schoemaker 1993, stopford 2001). porter and millar (1985) argued that it was changing the structure of industries in the economy, altering the rules and allowing organisations to create competitive advantage throughout all aspects of the value chain. keen (1991) claimed that “it has become an important aspect of everyday business. it is potentially a key element in competitive positioning”. keen (1991) also predicted seven key influences of it on business including online processing, image technology, changes to business relationships and achieving location independence. rockart, earl & ross (1996) added the concept of processes reengineering as a major it influence, noting that in 1996 “more than 50% of capital equipment invested in the united states was being devoted to information technology”. in 2003 alone australian organisations spent $20.189 billion on it services (gartner, 2004: 15). clearly the importance of it to business is increasing. the nature of the influence and role of it in organisations has also changed. rockart (1988) argued that in the 1950s information systems were used mainly for accounting purposes and batch processing. these later evolved to database-centred online systems supporting organisational processes and finally to it being the communication norm. it has moved from simply supporting operational functions to being a critical strategic organisational resource which should be shaped to deliver business needs (henderson & venkatraman, 1993; rockart, 1988; carr, 2003). businesses are highly dependant on it (broadbent & weill (1993). the business entrepreneur vinod khosla, cofounder of sun microsystems and it venture capitalist, stated “business is being completely reinvented” with the use of it (champion & carr, 2000). the influence of it has also been noted in the business board room. rockart (1998) stated that it can have a profound effect on business strategy. keen (1991:2) predicted that by 1993 it would be difficult for any fortune 1000 company “to define an effective business strategy that does not rely significantly on information technology”. overall, it has become a major organisational resource australasian journal of information systems volume 14 number 2 june 2007 118 for executing business strategy (rockart, earl & ross, 1996; feeny & willcocks, 1998; rangan & adner, 2001). given the important role that it plays in business strategy it is then vital that the two are in alignment. henderson & venkatraman (1993) argue that no it system on its own will deliver competitive advantage, the advantage comes from being able to exploit that functionality to achieve business goals. they further add that the inability to achieve these goals is due to lack of alignment between business strategy and it strategy. clearly, business leaders should be seeking ways to achieve successful alignment between it and business strategies. the alignment of business and it strategy has consistently been found to be a major concern for business executives (chan et al 1997, luftmann 2005). factors that promote alignment between business and it strategy a useful means of understanding how organisations can achieve successful alignment between business and it strategy is to identify factors that promote alignment. using a comprehensive analysis of relevant literature, ten factors that promote the alignment between business and it strategies have been identified (broadbent and weill 1993, burns and szeto 2000, rockart, earl and ross 1996, henderson and venkatraman 1993, choe 2003, pollalis 2003, earl and feeny 1994, kaplan and norton 1996, luftmann, papp and brier 1999). the ten factors are: firm wide active involvement; long term focus; meeting of the minds; clarity and consistency; management skill and capability; alignment facilitating processes; organisational structure; organisational culture; communication; it as an organisational tool. each of these factors is discussed below. firm-wide active involvement a two way relationship between the business and it functions is required during strategy formulation to achieve strategic alignment. this requires extensive firm-wide participation, information flow and interaction between business and it staff (henderson & venkatraman 1993; kantrow 1980, choe 2003; luftman, papp & brier 1999). in particular the chief information officer (cio) should be actively involved in all strategy formulation within the organisation (rockart, earl & ross 1996; ross et al 1996; keen 1991). in a study of sixty organisations earl & feeny (1994) found that a crucial part of a cio’s role is to assure that the organisation’s it function is deployed to achieve strategic advantage. it staff should feel like they are part of the organisation and not just the it industry (broadbent & weill 1993). if this is not the case then some critical decisions may be left to the it department alone (haeckel & nolan 1993). australasian journal of information systems volume 14 number 2 june 2007 119 long term focus both it and business strategies should hold a long-term focus on critical issues for successful alignment (broadbent & weill, 1993; kaplan & norton, 1996). when either it or business strategy lacks a long-term focus alignment is impeded. in particular, senior management attention is required when formulating it strategy to ensure a long-term focus (broadbent & weill 1993; rockart 1988). meeting of the minds management must reach consensus about firm-wide strategic issues between the business and it functions for successful alignment (broadbent & weill, 1993; rockart, earl & ross, 1996; kaplan & norton, 1996). if this “meeting of the minds” occurs then a clear understanding of key strategies spreads throughout the organisation (simons & davila 1998). consensus is best achieved when both the chief executive officer (ceo) and the cio share a clear vision and focus and communicate in a common language (slywotzky & morrison 2000). clarity and consistency both the business and it strategies need to be clear and consistent with business goals of the organisation for successful alignment (broadbent & weill 1993; henderson & venkatraman 1993; chan et al 1997; willcocks & plant 2001; choe 2003; pollalis 2003; kaplan & norton (1996). management skill and capability management skill and capability needs to be well developed in both it and business managers to facilitate alignment (broadbent & weill 1993; rockart, earl & ross 1996; choe 2003; luftman, papp & brier 1999; kaplan & norton 1996). kantrow (1980) added that the skill of understanding technology is not just needed at upper management but is central to business thinking at all levels. the need for it managers to understand business needs is particularly important (kaplan & norton 1996). alignment facilitation processes strategically-oriented decision-making processes that are maintained and managed by key stakeholders are important for successful alignment (broadbent & weill 1993; luftman, papp & brier 1999; kaplan & norton 1996). organisational structure having an organisational structure that provides mechanisms for accountability and ownership of strategy formulation is important for successful alignment (broadbent & weill, 1993; henderson & venkatraman, 1993; luftman, papp & brier, 1999). having such mechanisms in place not only ensures that it and business strategy will be formulated but also holds individuals accountable when alignment is not achieved and business goals remain unfulfilled. in their study of strategy formulation in four australian banks, broadbent & weill (1993) found that a rapid change in organisational structure from centralised to decentralised caused it systems to be incompatible, leading to a focus on operational issues rather than on formulating and maintaining it strategy. pollalis (2003) noted that those organisations with smaller it units were more flexible and therefore more readily able to the align business and it strategy more successfully than those who had larger it units. australasian journal of information systems volume 14 number 2 june 2007 120 organisational culture a supportive organisational culture is required to establish a strong working relationship between the business and it functions and achieve successful alignment (kantrow 1980). senior management support is important is the establishment of a supportive organisational culture (luftman, papp & brier, 1999). communication purposeful and strategically focused communication is important for successful alignment (broadbent & weill (1993); broadbent & weill 1997; choe 2003; king & teo 1996; kaplan & norton 1996). communication facilitates understanding and promotes mutually beneficial thinking about strategy (rockart, earl & ross, 1996 ;pollalis 2003). when communication is rewarded and recognised it became an organisational norm (kaplan & norton 1996). keen (1991: 214) noted that in order for communication to be effective organisations need “to replace old monologues by dialogue” and that much misalignment is due to poor communication. he further adds that the dialogue needs to start with top management with the view that it is a business resource that is used to achieve business needs. it as an organisational tool it needs to be perceived by organisations as a resource or business asset (willcocks & plant 2001; haeckel & nolan 1993; keen 1991; luftman, papp & brier 1999). this perception needs to be deeply instilled within organisations to ensure that new it opportunities that emerge are recognised and can impact business strategy formulation (henderson & venkatraman 1993). the ten factors vary across the individual, group and organisational levels, indicating that successful alignment is complex and requires transformation of both individuals and organisations. research design the purpose of the research is to explore the alignment of business and it strategies in australian organisations and identify ways that organisations can achieve successful alignment. we use factors as a way of defining how organisations can achieve successful alignment between business and it strategy. factors are easily understood and communicated and well suited to this purpose. the research design comprised three phases. the first phase involved an analysis of literature to synthesise a set of factors that promote alignment between business and it strategy. the second phase involved a series of expert interviews in order to confirm and refine the factors and ensure they could be readily understood by practitioners. the third phase involved a survey of australian organisations to identify the importance and degree of success of each factor in achieving successful alignment. the first phase is reported in the previous section of the paper and resulted in the identification of ten factors that promote alignment between business and it strategy. in the second phase six expert interviews were conducted with practitioners who had extensive consulting experience with business and it strategy. the interviews were semi-structured and comprised a number of questions based on the ten factors described earlier in the paper. interviewees were asked to comment on the definition and importance of each of the factors and how they related to each other. they were also australasian journal of information systems volume 14 number 2 june 2007 121 asked to nominate any further factors. as a result of the expert interviews, two changes were made to the list of factors: 1. the factors were grouped into three clusters – people, process and organisation – for presentation in the survey questionnaire; 2. several of the factors – management skill and capability, firm-wide active involvement, alignment facilitation processes and long term focus were split into two sub-factors, one relating to business and the other it. the other factors remained as single factors. thus a total of fourteen factors grouped into three clusters were used as the basis of survey questionnaire design. in the third phase the survey questionnaire was designed, piloted, administered and the results analysed. the survey comprised five sections together with a covering letter that explained the purpose of the research study and offered a copy of the results and their interpretation to those who responded. the first section of the survey asked for background information including size, structure, revenue, geographical segmentation or operations and client base, industry sector and overall perceptions of business and it strategy. the second and third sections concerned aspects of business and it strategy respectively and explore the perceived benefits of alignment. the fourth section asked respondents to use of 5-point likert scale to indicate their perceptions of both the importance and success in their organisation of factors that promote alignment between business and it strategy. the original survey included a fifth part that concerned factors that inhibited alignment between business and it strategy. due to space limitations this study focused on analysing and reporting the other parts of the survey. a copy of the survey may be found in the appendix the survey questionnaire was piloted by nine experienced business and it strategy consultants and several changes were made to the wording of the questionnaire to clarify meaning. the population of the survey was cios and ceos from large and medium sized australian organisations. it is very important that these senior people complete the survey questionnaire, as we need to capture their perceptions regarding organisational performance. the sample was obtained by purchasing a mailing list of the cios and ceos of the top 500 australian organisations. this gave a population sample of 1000 for the survey. however after filtering of organisations that were based in new zealand the population sample was reduced to 944. the survey was distributed by mail with an explanation letter and a reply paid envelop included. a total of 69 usable responses were received indicating a response rate of 7%. while this is relatively low, it is reasonable given errors found in the mailing list and the senior level of the respondents. results and discussion the results of the survey are presented according to the main sections of the survey questionnaire: organisation background information, business strategy, information technology strategy, and factors that promote alignment between business and it strategy. organisation background information the first section of the questionnaire concerned general information about the organisations and their broad perceptions of alignment between business and it strategy. of the 69 respondents, 25% indicated they were ceos and 51% indicated they were cios: the remaining 24% did not provide the information. respondents were from a variety of industry sectors with 19% from manufacturing, australasian journal of information systems volume 14 number 2 june 2007 122 17% from finance and insurance, and the remainder between 5% and 10% including agriculture, forestry and fishing, mining, wholesale trade, retail trade, transport and storage, property and business services and government administration. the organisations were either private, publicly listed or government in approximately equal proportions. just over half of the respondents indicated that all their operations within australia (54%) while almost all respondents (90%) indicated that 75% or more of their operations were within australia. similarly, 80% of respondents indicated that 75% or more of their client base was within australia. all respondents agreed that it was important or very important to their organisation’s operations. some interesting patterns emerge from the percentage of total capital expenditure that is spent on it per annum. 15% of respondents indicated that that they spend in excess of 50% of their annual capital expenditure on it and all of these respondents had annual revenue of $500m or more. 80% of privately owned organisations and 62% of publicly owned organisations indicated that they spend less than 20% of their annual capital expenditure on it. in comparison, 38% of publicly owned organisations and only 8% of private organisations indicated that they spend more than 50% of their annual capital expenditure on it. overall, this indicates that public organisations are more likely to spend large amounts of their capital budget on it than private organizations (assuming that capital budget is defined consistently by both public and private organizations). almost all respondents were involved or very involved with business strategy and it strategy formulation within their organisations. also, almost all respondents agreed that achieving alignment between business and it strategy within their organisations was very important. finally respondents were asked about what alignment of business and it strategy brings to their organisations. the three highest rated benefits were improved communication between business and it decision makers, improved relationships between business and it decision makers, and improved use of it to achieve organisational goals. the three lowest rated benefits were achieving a reduction in it costs, improving revenue and achieving a better perceived use of it. further analysis revealed differences in perceptions of cios and ceos. ceos believed more strongly than cios that alignment could improve revenue (59% vs. 46%), reduce it costs (71% vs. 60%) and increase competitive advantage in the marketplace (88% vs. 69%). cios focus more on the intangible benefits that alignment brings. different perceptions between large (annual revenue greater than $500m) and smaller organisations were also revealed. large organisations believed more strongly than smaller organisations that alignment would bring improved revenue but less supportive of the impact alignment would have on organisational brand and competitive advantage in the marketplace. business strategy and information technology strategy the second section of the questionnaire concerned more focused perceptions of alignment between business and it strategy. a significant majority of respondents (87%) either agreed or strongly agreed that their organisation has a well formulated business strategy and most business strategies were organisation-wide (80%). 74% of respondents either agreed or strongly agreed that their organisation has a well formulated it strategy and most it strategies were organisation-wide (78%). thus business strategies are generally perceived to be better formulated than it strategies. most (67%) of respondents either agreed or strongly agreed that their business strategy encapsulated their it strategy. cios were more likely than ceos to believe that their organisation’s business strategy encapsulated their it strategy (74% vs. 63%) and more likely to think the two strategies are australasian journal of information systems volume 14 number 2 june 2007 123 aligned (86% vs. 71%). a minority (22%) of respondents perceive their business and it strategies to not be aligned. a recent us study of fortune 500 companies found that 42% of executives thought that their business and it strategies were not aligned (luftman, papp & brier, 1999), suggesting that australian organisations may be more successful in achieving alignment than us organizations (however it is difficult to compare results of the two studies as they used different survey questionnaires and the respondents may have had different profiles). factors that promote alignment the fourth section of the questionnaire concerned perceptions of factors that promote alignment between business and it strategy. the factors were grouped into three clusters – people, process and organisation and are reported accordingly. respondents were asked to indicate how important they consider each factor to be in achieving alignment and how successful their organisation had been in performing against each factor on 5-point likert scales. results are reported for importance and success separately and then cross tabulations are used to examine the interaction of importance and success. for the cross tabulations a two-by-two matrix is used. the vertical axis displays the importance response as either important (likert scale score of 4 or 5 agree or strongly agree) or not important (likert scale score of 3 or less – neutral, disagree or strongly disagree). the horizontal axis displays the success response as either successful (likert scale score of 4 or 5 agree or strongly agree) or not successful (likert scale score of 3 or less – neutral, disagree or strongly disagree). in the first cross tabulation the number in each cell is the sum of likert scale responses. in the second cross tabulation a list of individual factors is provided in each cell, based on the average of likert scale responses. further analysis of responses is undertaken for each cross tabulation using the ceo/cio, public/private, and large/sme groupings. people factors the questionnaire asked respondents to rate the importance and their organisation’s success for the following people factors: meeting of the minds between business and it decision makers; management skill and capability of business decision makers; management skill and capability of it decision makers; communication between business and it decision makers; firm wide active involvement in business and it strategy formation; involvement of business decision makers in it strategy formation; involvement of it decision makers in business strategy formation. with respect to importance, all 7 of the people factors had average likert scale ratings between 4 and 5 (agree and strongly agree) indicating strong agreement that people factors are important in achieving alignment. the top three people factors in importance are communication between business and it decision makers, management skill and capability of business decision makers and achieving a meeting of the minds between business and it decision makers. with respect to success, all 7 of the people factors had average likert scale ratings between 3 and 4 (neutral and agree) indicating agreement about successfully performing people alignment factors. the three people factors most successfully performed are management skill and capability of business decision makers, management skill and capability of it decision makers and effective communication between business and it decision makers. figure 1 displays the first cross tabulation for people factors. the majority of respondents (283 62%) perceived people factors to be important and indicated that their organisation was successful at performing them. some respondents (132 29%) perceived people factors to be important but australasian journal of information systems volume 14 number 2 june 2007 124 felt that their organisations were unsuccessful at performing them. the remaining respondents (44 – approx.10%) perceived people factors to be not important. figure 1 people factors cross tabulation further analysis revealed that ceos generally perceived people factors to be both more important and that their organisations were more successful with people factors than cios. this indicates that ceos place more emphasis on people factors than the cios. there is little difference in the perceptions of large organisations and smes about people factors. both public and private organisations consider people factors to be equally important but public organisations perceived that they were more successful with people factors than private organisations, suggesting that private organisations find people factors more challenging. figure 2 displays the second cross tabulation for people factors and indicates that all of the people factors are perceived as important and successfully performed except for firm wide active involvement in business and it strategy formation, which is seen as important but not successfully performed. unsuccessful successful n ot im po rta nt im po rta nt people factors that promote alignment 32 12 132 283 unsuccessful successful n ot im po rta nt im po rta nt people factors that promote alignment 32 12 132 283 australasian journal of information systems volume 14 number 2 june 2007 125 figure 2 people factor individual cross tabulation the top three factors scoring 70% or above on the cross tabulation analyses are: the management skill and capability of the business decision maker; the management skill and capability of the it decision maker and; communication between the business and it decision maker. these findings are not entirely consistent with previous studies, which found that firm wide active involvement in strategy formation, management skill and capability, communication and a meeting of the minds were the most important factors in promoting alignment. this study suggests that the management skill and capability of the it decision maker is also perceived to be very important. the cross tabulation revealed that many respondents believed that firm wide active involvement in business and it strategy formation was not important in promoting alignment firm wide active involvement in strategy formation is predominately a feature of us studies and is a less important factor for australian organisations. further analysis revealed that while cios generally agreed with this observation, ceos generally disagreed indicating that while ceos believe that organisational buy-in is important in strategy formation, cios believe it is more of an individual task. surprisingly 15% of all respondents believed that it was not important to involve the it decision maker during the business strategy formation process, with cios providing stronger support than ceos. furthermore, 12% of all respondents believed that involving the business decision maker in it strategy formation was also unimportant, with ceos providing stronger support than cios. earl and feeny (1994) found that it was crucial that the ceo be involved in it strategy formation in order to ensure strategy business goals are deployed through the organisations use of it. it is surprising that some australian managers perceive that business and it decision makers need only be involved in their own strategy formation process. further analysis revealed that cios perceived that obtaining involvement from business decision makers in it strategy formation was the least successfully performed people factor. correspondingly ceos perceived that obtaining it decision maker involvement during business strategy formation was the least successfully performed people factor. this suggests that both ceos and cios have more work to do in developing trusting relationships within strategy formation. unsuccessful successful n ot im po rta nt im po rta nt people factors that promote alignment meeting of the minds business decision makers skill it decision makers skill communication between decision makers involve business decision maker in it strategy involve it decision maker in business strategy firm wide active involvement in business strategy formation unsuccessful successful n ot im po rta nt im po rta nt people factors that promote alignment meeting of the minds business decision makers skill it decision makers skill communication between decision makers involve business decision maker in it strategy involve it decision maker in business strategy firm wide active involvement in business strategy formation australasian journal of information systems volume 14 number 2 june 2007 126 process factors the questionnaire asked respondents to rate the importance and their organisation’s success for the following process factors: a process which promotes clarity and consistency; a process that ensures it strategy goals are linked with business goals; a process that ensures business strategy goals are linked with it goals; the availability of a formal process which facilitates alignment; formal communication processes in place between business and it decision makers; a formal process that ensures business strategy has a long term (5 year +) focus; a formal process that ensures it strategy has a long term (5 year +) focus. with respect to importance, only 3 process factors had average likert scale ratings between 4 and 5 (agree and strongly agree), while the other 4 process factors averaged just under 4. this indicates strong agreement that process factors are important in achieving alignment, although not as important as people factors. the top three process factors in importance are having a process that ensures it strategy goals are linked with business goals, having a process which promotes clarity and consistency and having a formal communication process in place between business and it decision makers. with respect to success, all 7 of the process factors had average likert scale ratings between 3 and 4 (neutral and agree) indicating agreement about successful performance with process alignment factors. the three process factors most successfully performed are the same as for importance above. figure 3 displays the first cross tabulation for process factors. many respondents (214 46%) perceived process factors to be important and indicated that their organisation was successful at performing them. some respondents (150 32%) perceived process factors to be important but felt that their organisations were unsuccessful at performing them. the remaining respondents (105 22%) perceived process factors to be not important. figure 3 process factors cross tabulation further analysis revealed that both ceo and cio perceptions were generally consistent with the overall cross tabulation data, however ceos were slightly more confident (10%) in indicating that their organisation was successful at performing in this category. unsuccessful successful n ot im po rta nt im po rta nt process factors that promote alignment 82 23 150 214 unsuccessful successful n ot im po rta nt im po rta nt process factors that promote alignment 82 23 150 214 australasian journal of information systems volume 14 number 2 june 2007 127 figure 4 displays the second cross tabulation for process factors and indicates that all of the process factors are perceived as important but only three are seen to be successfully achieved. figure 4 process factor individual cross tabulation the top two factors scoring 50% or above on the cross tabulation analyses are: a process that promotes clarity and consistency; a process that ensures it goals are linked with business goals. these findings are not entirely consistent with previous studies, which found that a long term focus, clarity and consistency and an alignment facilitating process were key factors. while this study confirmed that clarity and consistency were important, it also found the additional factor of having a process that ensures it goals are linked with business goals was very important in australian organisations. surprisingly, linking business goals to it goals was found to be the least important process factor. this suggest an asymmetrical relationship in which it strategy should be driven by business strategy, a view supported by king (1978) but later rejected by king & zmud (1981) and baets (1992) who argue that relationships in business are very dynamic rather than hierarchical. this study however finds that in australian organisations a hierarchical relationship between business and it strategy is evident and furthermore, is seen to be important in promoting alignment. a third of overall respondents believed that having a long term it strategy is not important and just under a third believe that a long term business strategy is not important. cios were more likely to agree with this finding than ceos (13%), suggesting that the it environment is changing too rapidly to allow for realistic long term strategy formation. larger organisations were more likely than smes to find a long term business strategy unimportant (21%), and also more likely than smes to find a long term it strategy as unimportant in achieving alignment (14%), indicating that smes place more emphasis on long term strategy formation. a third of overall respondents believed that having a formal process that facilitates alignment is not important. publicly owned organisations were more likely than privately owned organisations to agree. large organisations also agreed with the statement; however smes generally didn’t agree and found all of the factors important in achieving alignment. unsuccessful successful n ot im po rta nt im po rta nt process factors that promote alignment process which promotes clarity & consistency process that ensures it goals are linked with business goals process that ensures business goals are linked to it goals availability of a formal process that facilitates alignment formal communication processes between it and business decision makers formal process that ensures business strategy has a long term focus formal process that ensures it strategy has a long term focus unsuccessful successful n ot im po rta nt im po rta nt process factors that promote alignment process which promotes clarity & consistency process that ensures it goals are linked with business goals process that ensures business goals are linked to it goals availability of a formal process that facilitates alignment formal communication processes between it and business decision makers formal process that ensures business strategy has a long term focus formal process that ensures it strategy has a long term focus australasian journal of information systems volume 14 number 2 june 2007 128 over a third of respondents found that having a process that facilitates alignment was performed unsuccessfully. furthermore over a third of respondents found that implementing a process that promotes clarity and consistency and having a formal communication process between the business and it decision makers was also difficult to perform successfully. these responses are surprisingly high given that these factors are considered so important in the literature. organisational factors the questionnaire asked respondents to rate the importance and their organisation’s success for the following organisational factors: an organisational structure which facilitates alignment of business and it decision makers; an organisational culture which facilitates alignment between business and it decision makers; the view that it is an innovative organisational tool as opposed to a cost centre. with respect to importance, all 3 of the organisational factors had average likert scale ratings between 4 and 5 (agree and strongly agree) indicating strong agreement that organisational factors are important in achieving alignment. the top organisational factor in importance is having an organisational culture which facilitates alignment between business and it decision makers. with respect to success, all 3 of the organisational factors had average likert scale ratings between 3 and 4 (neutral and agree) indicating agreement about successful performance with organisational alignment factors. the organisational factor most successfully performed is having an organisational structure which facilitates alignment of business and it decision makers. figure 5 displays the first cross tabulation for organisational factors. many respondents (100 50%) perceived organisational factors to be important and indicated that their organisation was successful at performing them. some respondents (65 33%) perceived organisational factors to be important but felt that their organisations were unsuccessful at performing them. the remaining respondents (35 18%) perceived people factors to be not important. further analysis revealed that the perceptions of cios were generally consistent with the overall cross tabulation data. ceos however were more likely to perceive that organisational factors were important but difficult to perform successfully (21%) indicating that ceos are more optimistic about successfully performing organisational factors. figure 6 displays the second cross tabulation for organisational factors and indicates that all of the organisational factors are perceived as important and successfully performed except for the view that it is an innovative organisational tool as opposed to a cost centre, which is seen as important but not successfully performed. australasian journal of information systems volume 14 number 2 june 2007 129 figure 5 organisational factors cross tabulation figure 6 organisational factor individual cross tabulation the top two factors scoring 50% or above on the cross tabulation analyses are: an organisational structure which facilitates alignment of business and it decision makers; an organisational culture which facilitates alignment between business and it decision makers. these findings are consistent with previous studies, which found that creating an organisational structure that facilitates alignment is important as it provides a “mechanism for accountability and ownership” if alignment goals remain unfulfilled (broadbent & weill 1993). henderson & venkatraman (1993), in their us based study, found that the it function needs to be viewed as an organisational tool in order to promote alignment and when it is not viewed this way it is usually unsuccessful successful n ot im po rta nt im po rta nt organisational factors that promote alignment 25 10 65 100 unsuccessful successful n ot im po rta nt im po rta nt organisational factors that promote alignment 25 10 65 100 unsuccessful successful n ot im po rta nt im po rta nt organisational factors that promote alignment an organisational structure that facilitates alignment an organisational culture that facilitates alignment the view that it is an innovative tool instead of a cost centre unsuccessful successful n ot im po rta nt im po rta nt organisational factors that promote alignment an organisational structure that facilitates alignment an organisational culture that facilitates alignment the view that it is an innovative tool instead of a cost centre australasian journal of information systems volume 14 number 2 june 2007 130 due to deeply instilled perceptions which hamper “the extent to which new it opportunities emerge and are addressed by the organisation”. this study confirms this outcome for australian organisations. further analysis revealed that ceos were 33% more likely than cios to indicate that having an organisational culture that promotes alignment is both important and successfully performed. this may lead to problems with achieving alignment as porter (1996) notes that the greatest threat to strategy can come from within the organisation. furthermore public organisations were more likely than privately owned organisations to indicate that having an organisational culture that promotes alignment is both important and successfully performed (17%). similarly smes were slightly more likely than large organisations to indicate that an organisational culture that facilitates alignment is both important and successfully performed (11%). surprisingly 17% of all respondents indicated that the organisational factors were not important in achieving alignment. ceos were more likely than cios to perceive that organisational culture was not an important element in achieving alignment. large organisations (19%) were more likely than smes (11%) to perceive that organisational culture was not an important element in achieving alignment. a third of all respondents indicated that although an organisational culture facilitating alignment and an attitude that it is an organisational tool are important, they are difficult to perform successfully (37%). cios were more likely than ceos to perceive that treating it as an organisational tool was the least successfully performed factor (17%). they were also more likely to indicate that an organisational culture facilitating alignment was unsuccessful than the ceos and more likely than the ceos to indicate that an alignment friendly organisational structure was a challenge. overall, this suggests that the cio group were less optimistic than the ceo group with successful performance of organisational factors. smes were more likely than large organisations to find it difficult to change the perception that it is an organisational tool rather than a cost centre (26%). furthermore public organisations were slightly more likely than privately owned organisations to find it difficult to succeed with an organisational culture that facilitated alignment (14%). privately owned organisations on the other hand were more likely to find successfully performing an alignment friendly organisational structure difficult (20%). both public and privately owned organisations struggled with changing the perception of it as an organisational tool rather than a cost centre. high level factor comparisons overall respondents perceived that people factors were the most important and the most successfully performed, followed by organisational factors and process factors. organisational factors were found to be highly important but the most difficult to successfully perform. process factors were given the lowest importance rating. the large/sme, public/private respondent segments were consistent with these overall trends. both cios and ceos were consistent with the overall trend for allocating importance. however ceos perceived that process and then people factors were more difficult to successfully perform than organisational factors. australasian journal of information systems volume 14 number 2 june 2007 131 conclusion a number of important conclusions about the alignment of it and business strategy may be drawn from this study. respondents perceive that alignment brings both tangible and intangible benefits. when compared with previous research involving mainly us base organisations, it is apparent that australian organisations are perceived to be more successful than us companies in achieving alignment. respondents perceived that people factors were the most important and the most successfully performed, followed by organisational and process factors. organisational factors were deemed the hardest to successfully perform. people factors were generally found to be both important and successfully performed. of the people factors, the business decision maker’s skill and capability are the most important factors in achieving successful alignment. cios want to be more involved in business strategy formation however ceos do not always believe that to be necessary. cios would also like to see more ceo involvement in it strategy formation but the ceo group believes that may not be necessary. a surprising proportion of ceos perceive that business and it decision makers need only be involved in their own strategy formation process. furthermore, the study found that business decision makers may not always be comfortable being involved in a strategy formation process outside of their own area. process factors were found to be both important and successfully performed by a third of respondents. the most important process factors were having a process that promotes clarity and consistency and a process that ensures it goals are linked with business goals. furthermore a hierarchical relationship was found suggesting that it strategy tend to be driven by business strategy in australian organisations. surprisingly a third of all respondents believed that having a long term it strategy is not important and just under a third believe that a long term business strategy is not important. another surprising outcome was that a third of all respondents indicated that implementing a process that promotes clarity and consistency and having a formal communication process between the business and it decision makers was challenging. publicly owned organisations found that achieving clarity and consistency was particularly difficult to successfully perform and almost half the smes perceived that developing a long term it strategy was the least successfully performed factor. cios perceived that ensuring business goals are linked to it goals was difficult to successfully perform. this is consistent with the earlier finding that it strategy tends to be driven by business strategy. organisational factors were found to be important and successfully performed by a third of respondents. organisational structure and culture are seen to be most important factors in facilitating alignment. a third of all respondents indicated that an alignment promoting organisational culture and an attitude that it is an organisational tool are difficult to successfully perform. cios perceive that generating an organisational culture that promotes alignment and treating it as an organisational tool can be difficult to successfully perform. further research can be undertaken in two areas: the survey could be administered annually to determine trends over time in perceptions of the importance and success with alignment factors; in-depth case studies could be conducted to better understand how and why perceptions of respondents were formed and to strengthen generalisation of the findings. australasian journal of information systems volume 14 number 2 june 2007 132 in summary, australian organisations are heavily reliant on it and achieving alignment between business and it strategy translates into realising tangible and intangible benefits both internally and in the market place. alignment between business and it strategy allows organisations to exploit it functionality to achieve business goals. inability to achieve these goals is partly due to a lack of alignment. the findings of this study will help australian organisations to understand how to best achieve successful alignment and reap the benefits that alignment can bring. references baets, w. (1992) aligning information systems with business strategy, journal of strategic information systems vol 1 no 4, pp 205-213. bashein, b.j. & markus, l. (1997) a credibility equation for it specialists, sloan management review, vol 38 no 4, pp 35–44. beer, m. & eisenstat, r.a. (2000) the silent killers of strategy implementation and learning, sloan management review, vol 41 no 4, pp 29-40. broadbent, m. & weill, p. (1993) improving business and information strategy alignment: learning from the banking industry, ibm systems journal, vol 32 no 1, pp 162-179. broadbent, m. & weill, p. (1997) management by maxim: how business and it managers can create it infrastructures, sloan management review, vol 38, pp 77-92. chan, y.e., huff, s.l., barclay, d.w. & copeland, d.g. (1997) business strategic orientation, information systems strategic orientation and strategic alignment, information systems research, vol 8 no 2), pp 125-150. choe, j. (2003) the effect of environmental uncertainty and strategic applications of is on a firm's performance, information & management, vol 40 no 4, pp 257-268. earl, m.j. & feeny, d.f. (1994) is your cio adding value?, sloan management review, vol 35, pp 1120. feeny, d.f. & willcocks, l.p. (1998) core is capabilities for exploiting information technology, sloan management review, vol 39 no 3, pp 9–21. gadiesh, o. & gilbert, j.l. (2001) transforming corner office strategy into frontline action, harvard business review, may, pp 73-79 haeckel, s.h. & nolan, r.l. (1993) managing by wire, harvard business review, vol 71 no 5, pp 122-132. henderson, j.c. & venkatraman, n. (1993) strategic alignment: leveraging information technology for transforming organisations, ibm systems journal, vol 32 no 1, pp 472-484. kantrow, a, m. (1980): the strategy – technology connection, harvard business review august, pp 6-21. kaplan, r.s. & norton, d.p. (1996) translating strategy into action – the balanced scorecard, harvard business school press, boston, ma keen, p.g.w. (1991) shaping the futurebusiness design through information technology, harvard business school press, boston, ma australasian journal of information systems volume 14 number 2 june 2007 133 king, w.r. (1978) strategic planning for management information systems, mis quarterly, vol 2 no 1, pp 27-37 king, w.r., & zmud, r.w. (1981) managing information systems: policy planning, strategic planning and operational planning, proc. of the second international conference on information systems, boston, ma, pp 299-308 king, w.r., & teo, t. (1996) integrating between business planning and information systems planning, information and management, vol 30, pp 309-321 luftman, j.n., papp, r. & brier, t. (1999) enablers and inhibitors of business-it alignment, communications of the association of information systems, vol 1 no11 luftman, j.n. (2005): key issues for it executives 2004, mis quarterly executive, vol 4 no 2, pp 1-5 modzelewski, j. (2004a) alignment of business and information technology strategies: an australian approach, school of business systems, monash university modzelewski, j. (2004b) research design: alignment of business and information technology strategiesan australian approach, school of business systems, monash university modzelewski, j., (2004c) data collection & analysis: alignment of business and information technology strategiesa cross industry australian study, school of business systems, monash university minzberg, h. (1978) patterns in strategy formation”, management science, vol 24 no 9, pp 934948 pollalis, y.a. (2003) patterns of co-alignment in information-intensive organizations: business performance through integration strategies, international journal of information management, vol 23 no 6, pp 469-492 porter, m.e. (1996) what is strategy?, harvard business review, december, pp 61-98 rangan, s. & adner, r. (2001) profits and the internet: seven misconceptions, sloan management review, vol 42 no, pp 444-453 rockart, j.f. (1988) the line takes the leadership – is management in a wired society, sloan management review, vol 29, pp 5764 rockart, j.f., earl, m.j. & ross, j.w. (1996): eight imperatives for the new it organisation, sloan management review, vol 31 no 1, pp 43-55 ross, j.w., beath, c. & goodhue, d.l. (1996) develop long term competitiveness through it assets, sloan management review, vol 38 no 1, pp 31-42 ross, j.w. & weill, p. (2002) six it decisions your it people shouldn’t make, harvard business review, vol 80 no 11, pp 84–91 simons, r. & davila, a. (1998) how high is your return on management?, harvard business review, february, pp 71-80 slywotzky, a.j. & morrison, d.j. (2000) how digital is your business?, nicholas brealey publishing, london willcocks, l.p. & plant, r., (2001) pathways to e-business leadership: getting from bricks to clicks, sloan management review, vol 42 no 3, pp 50-59 australasian journal of information systems volume 14 number 2 june 2007 134 appendix 1 alignment of business and information technology strategy questionnaire alignment of business & information technology strategy a cross industry australian study part 1: background this series of questions aims to collect background information which will be used in the formation of industry clusters and subsequent data analysis. 1. in which industry sector is your organisation? agricultural, forestry and fishing communication services mining finance & insurance manufacturing property & business services electricity, gas & water supply government administration and defence construction education wholesale trade health & community services retail trade cultural & recreational services accommodation, cafes & restaurants personal services transport & storage professional services other 2. what is your organisation’s annual revenue? __________________________________ 3. what is your organisation’s ownership structure? private publicly listed government 4. how are your organisation’s operations geographically segmented? 100% of operations in australia between 76% and 99% of operations in australia and the remainder overseas between 51% and 75% of operations in australia and the remainder overseas between 26% and 50% of operations in australia and the remainder overseas less than 25% operations within australia 5. what is the geographic segmentation of your organisation’s client base? 100% of client base is in australia between 76% and 99% of client base is in australia and the remainder overseas australasian journal of information systems volume 14 number 2 june 2007 135 between 51% and 75% of client base is in australia and the remainder overseas between 26% and 50% of client base is in australia and the remainder overseas less than 25% of client base is in australia 6. how important is it to your organisation’s operations? not important at all not important neutral important very important 7. what percentage of your organisation’s total capital expenditure is spent on it per annum? between 0-10% between 11-20% between21-30% between 31-40% between 41-50% over 50% 8. what is your current title within the organisation? (please specify) __________________________________ 9. please answer the following two statements a) to what extent are you involved in business strategy formation within your organisation? very uninvolved uninvolved neutral involved very involved b) to what extent are you involved in information technology strategy formation within your organisation? very uninvolved uninvolved neutral involved very involved 10. how important do you think it is to achieve alignment between business and information technology strategy within your organisation? not important at all not important neutral important very important 11. do you think alignment between business and information technology strategy achieves the following within your organisation? strongly disagree disagree neutral agree strongl y agree improved relationship between business and it decision makers improved communications between business and it decision makers improved perception of the it function within the organisation improved use of it within the organisation improved utilisation of it resources to achieve organisational goals improved revenue reduction in it costs reduction in overall costs better it returns on investment better overall returns on investment perceived improved use of it innovation by the marketplace positive effect on organisational brand australasian journal of information systems volume 14 number 2 june 2007 136 increased competitive advantage in the marketplace part 2: business strategy business strategy in this questionnaire is defined as the strategy adopted for the entire organisation. 12. my organisation has a well formulated business strategy. strongly disagree disagree neutral agree strongly agree 13. the highest level of business strategy within my organisation is: organisation based business unit based divisionally based project based part 3: information technology strategy for the purpose of this questionnaire information technology strategy refers to the strategy adopted by the entire organisation in planning its use of information technology resources. 14. my organisation has a well formulated information technology strategy. strongly disagree disagree neutral agree strongly agree 15. my organisation’s business strategy encapsulates the information technology strategy. strongly disagree disagree neutral agree strongly agree 16. the highest level of information technology strategy within my organisation is: organisation based business unit based divisionally based project based part 4: factors that promote the alignment between business and information technology strategy previous studies have identified that there are several factors which promote alignment between business and information technology strategy. studies have noted that when these factors are present there is a greater chance of successful alignment. 17. how well aligned is the business and information technology strategy within your organisation? not well at all not well neutral well extremely well 18. how important are the following people factors in achieving the alignment between business strategy & information technology strategy within your organisation? australasian journal of information systems volume 14 number 2 june 2007 137 very unimportant unimportant neutral import ant very important meeting of the minds between business and it decision makers management skill & capability of business decision makers management skill & capability of it decision makers communication between business and it decision makers firm wide active involvement in business and it strategy formulation involvement of business decision makers in it strategy formulation involvement of it decision makers in business strategy formulation meeting of the minds between business and it decision makers 19. how successful is your organisation in performing against each of these people alignment factors? very unsuccessful unsuccessfu l neutral successf ul very successful meeting of the minds between business and it decision makers management skill & capability of business decision makers management skill & capability of it decision makers communication between business and it decision makers firm wide active involvement in business and it strategy formulation involvement of business decision makers in it strategy formulation involvement of it decision makers in business strategy formulation meeting of the minds between business and it decision makers 20. how important are the following process factors in achieving the alignment between business strategy & information technology strategy within your organisation? very unimportant unimportant neutral important very important a process which promotes clarity & consistency a process that ensures it strategy goals are linked with business goals a process that ensures business goals are linked to it strategy goals the availability of a formal process which facilitates alignment formal communication processes in place between business and it decision makers a formal process that ensures business strategy has a long term (5 years +) focus a formal process that ensures it strategy has a long term (5 years +) focus australasian journal of information systems volume 14 number 2 june 2007 138 21. how successful is your organisation in performing against each of these process alignment factors? very unsuccessful unsuccessful neutral successf ul very successful a process which promotes clarity & consistency a process that ensures it strategy goals are linked with business goals a process that ensures business goals are linked to it strategy goals the availability of a formal process which facilitates alignment formal communication processes in place between business and it decision makers a formal process that ensures business strategy has a long term (5 years +) focus a formal process that ensures it strategy has a long term (5 years +) focus 22. how important are the following organisational factors in achieving the alignment between business strategy & information technology strategy within your organisation? very unimportant unimportant neutral important very important an organisational structure which facilitates alignment of business and it decision makers an organisational culture which facilitates alignment between business and it decision makers the view that it is an innovative organisational tool as opposed to a cost centre 32. how successful is your organisation in performing against each of these organisational alignment factors? very unsuccessful unsuccessful neutral successful very success ful an organisational structure which facilitates alignment of business and it decision makers an organisational culture which facilitates alignment between business and it decision makers the view that it is an innovative organisational tool as opposed to a cost centre australasian journal of information systems volume 14 number 2 june 2007 139 biographical notes jovita gartlan is a strategy & operations consultant with deloitte and a masters of business systems student with monash university. with deloitte jovita has worked for a wide range of high profile clients providing advisory consulting services. her recent projects include m&a services, market entry strategy development, labour productivity review, operating model reviews, business transformation, business roadmap development, cost analysis and forecasting and detailed process reengineering. jovita has completed a bachelor of international trade and a bachelor of business management at monash university and was awarded a monash travel grant to study business mandarin in shanghai, china. jovita has a keen interest in australian academic research that addresses key business issues and provides fact based evidence for the business world. graeme shanks is a professorial fellow in the department of information systems at the university of melbourne, australia. he has also held academic positions at monash university and chisholm institute of technology. before becoming an academic, graeme worked for a number of private and government organizations as a programmer, systems analysts and project leader. his teaching and research interests include information quality, conceptual modelling, identity management, implementation and impact of enterprise systems, and decision support systems. graeme has published the results of his research in over 100 refereed journal and conference papers. he is on the editorial boards of the journal of data warehousing, the journal of database management, the international journal of data warehousing and mining and the journal of knowledge management research and practice. editorial board thomson scientific (isi) impact factor. using existing ranking lists polling eminent scholars stage one: identification of existing rankings stage two: initial ranking list drawn up stage three: feedback on the initial ranking list stage four: information and finalisation of ranking list level a journals level b journals premier professional journals abstract introduction the pending australian research quality framework (rqf) methodology sourcing and procuring the data examining the data for correctness results rq1 what journal outlets are used by australian academics to publish is-related research? rq2 which journals have been most preferred by australian academics for article publication as measured by the total number of articles published in each journal? conclusion limitations and future research socio-pragmatic constructionism social constructionism pragmatism socio-pragmatic constructionism representation simplification pragmatic orientation syntax semantics pragmatics implications for practice implications for theory passive dimension (perceptions); active dimension (conceptions) the aesthetic dimension is not characterized as “plug and play”. the aesthetic dimension is not described by graphical user interfaces. aesthetics occurs when the system elevates the work to a new level of effective vision. aesthetics may occur on different levels or from different perspectives. 1. rational goal seeking (intelligence(design(choice) 2. happenstance rationalization (design(choice(intelligence) 3. existential introversion (choice(intelligence(design) 4. deferred design (intelligence(choice(design) 5. situational sensemaking (design(intelligence(choice) 6. existential heroism (choice(design(intelligence) constructs output performance outsourcing experience contract establishment the feedback effect business strategy it strategy firm-wide active involvement long term focus meeting of the minds clarity and consistency management skill and capability alignment facilitation processes organisational structure organisational culture communication it as an organisational tool organisation background information factors that promote alignment people factors process factors organisational factors high level factor comparisons alignment of business & information technology strategy a cross industry australian study part 1: background part 2: business strategy part 3: information technology strategy part 4: factors that promote the alignment between business and information technology strategy documentation interviews researcher bias microsoft word ajis 11 1 final.doc ajis vol.11 no. 1 september 2003 2 editorial the australian journal of information systems has changed its name from australian to australasian. to mark the change, we have created a ‘new look’ journal. i am pleased to announce that the journal was recently voted in the top 25 is journals in the world and i would like to thank all of our reviewers and authors for their professionalism in helping achieve this position. our first paper is presented by george widmeyer, university of michigan and is entitled the trichotomy of processes: a phisosophical basis for information systems. this paper explores the principle of trichotomy, posed by charles peirce, to model the world and business applications in information systems. john haynes, university of central florida presents our next paper entitled the management of intuition. this paper examines a phenomenological approach to managing intuition. in particular, the paper examines the nature of heidegger’s notion of ding-an-sich (the thing-in-itself), a grasp of which, it is argued, is essential in coming to terms with the human capacity for intuition and creativity. our third paper is entitled a critical analysis of the knowledge creation process and is presented by stephen probert, cranfield university. the paper then explores how some of the work by foucault and adorno may provide a new direction for developing research approaches in knowledge creation and management. chatpong tangmanee from chulalongkorn university presents our next paper entitled patterns of programmers’ use of computer-mediated communications systems. this paper investigates computer-mediated communications (cmc) such as email, the world-wide web etc and provides the patterns for which programmers use these cmcs. elizabeth kemp, chris phillips and jamie alam, from massey university present our next paper entitled software engineering practices and tool support: an exploratory study in new zealand. this paper describes a preliminary investigation of the practices of software engineers within new zealand, including their use of development tools. our next paper is entitled investigating the partial relationships between testability and the dynamic range-to-domain ratio and is presented by zuhoor al-khanjari, from sultan qaboos university and martin woodward, university of liverpool. the propagation, infection and execution (pie) analysis technique has been proposed as a way of estimating the voas notion of testability. this paper reviews the various testability concepts and summarises the pie technique. p. jha, j. glassey, g. montague, university of newcastle and p mohan, tippecanoe labs present our next paper entitled product cost management structures: a review and neural network modelling. this paper reviews the growth of approaches in product costing and draws synergies with information management and resource planning systems, to investigate potential application of state of the art modelling techniques of neural networks. microsoft word ajis vol13-2 o.doc 69 australasian journal of information systems volume 13 number 2 may 2006 how to develop a multi-grounded theory: the evolution of a business process theory mikael lind university college of borås school of business and informatics s-501 90 borås, sweden email: mikael.lind@hb.se göran goldkuhl linköping university department of computer and information science s-581 83 linköping, sweden email:ggo@ida.liu.se abstract in the information systems field there is a great need for different theories. theory development can be performed in different ways – deductively and/or inductively. different approaches with their pros and cons for theory development exists. a combined approach, which builds on inductive as well as deductive thinking, has been put forward – a multi-grounded theory approach. in this paper the evolution of a business process theory is regarded as the development of a multi-grounded theory. this evolution is based on empirical studies, theory-informed conceptual development and the creation of conceptual cohesion. the theoretical development has involved a dialectic approach aiming at a theoretical synthesis based on antagonistic theories. the result of this research process was a multi-grounded business process theory. multigrounded means that the theory is empirically, internally and theoretically founded. this business process theory can be used as an aid for business modellers to direct attention towards relevant aspects when business process determination is performed. keywords: research method, grounded theory, business processes, dialectics introduction knowledge created within scientific disciplines are often codified and structured in theories. in information systems (is) research, there are growing efforts in developing theories. one approach often taken is to use an established theory from a reference discipline and redevelop and adapt it to the information systems context. the use of external theories such as actor-network theory (latour, 1992) and structuration theory (giddens, 1984) are examples of such endeavours; confer walsham (1997), orlikowski (1992) and rose et al (2003) for examples and analyses. such an approach can be seen as a deductive one. an established theory is used to categorize and classify is phenomena. a deductive approach to theory development is not the only one. an alternative approach often taken is an inductive analysis of empirical data for creation of a theory. the grounded theory (gt) approach (glaser & strauss, 1967) is a prominent representative of inductive approaches to theory development. during the later years gt has been used within is research; confer e.g. urquhart (2001) for example and analysis. australasian journal of information systems volume 13 number 2 may 2006 70 criticism has however been raised against grounded theory for a too strict inductive approach where existing theories are left out. an alternative approach, building on grounded theory, has therefore been put forward – the multigrounded theory (mgt) approach. goldkuhl & cronholm (2003) define a multi-grounded theory to be an empirically, theoretically and internally grounded theory. developing a multi-grounded theory builds on the interplay between grounding and generation in which empirical data and theories will have different roles as knowledge sources. in this approach the risk of “introvert theorizing” is avoided. the importance of acknowledging and utilising other theoretical sources is put forward as a strong argument for working in a multi-grounded manner. our main interest in this paper is the development of theories in information systems. how can such a development be performed? can theory development be performed in a combined inductive and deductive way? our research interest is towards research method, research process and research product. this is hence meta research. studies in research methodology are often performed as theoretical discourses. our approach taken here is more empirically oriented. we do not deny the need for formulating and assessing theoretical arguments (this will be done), but we claim that research on research should also be empirically oriented. we use a case study approach in the paper. we use a theory development process – a development of a business process theory – as our case (lind, 2002). the need for the development of this theory is based in the problem of how to divide organisational work into different business processes during business analysis. we have had access to material from this research process as well as to knowledge from the key researchers. one important aspect of a multi-grounded theory is that it should be cumulative and that the theory should be grounded in already existing theories. such theories can however be contradictory (antagonistic). this affords the possibilities of a dialectic approach. in a dialectic research contradictions between a thesis and an antithesis are resolved through the creation of a synthesis. the cumulative theoretical development is operationalised by letting the synthesis become a new thesis to be used in future dialectic interplays (skirbekk & gilje, 2001). in this paper we will analyse and describe the interplay between analysis of empirical data, and contradicting theories (as thesis and antithesis) as well as how complementary theories have been used in the evolution of a multi-grounded theory. empirical data from several business process analyses meet several theories both for the generation of and grounding of the business process theory. an answer will be given in what ways the business process theory is to be regarded as multigrounded and the fruitfulness of adopting a multi-grounded theory approach when performing theory development. in order to give such an answer focus will be put upon the data generation, the theoryinformed theory generation as well as the theoretical, empirical and internal grounding. this paper is structured as follows. in the next section we will introduce the different principles of the multi-grounded theory approach. in that section we will also relate multi-grounded theory approach to the origins in grounded theory. following that we will give a historical description of the research process concerning the development of the business process theory mentioned above. this historical description will be divided into three episodes. following that historical description we will show how this research process is an application of multi-grounded theory approach. the paper ends with some conclusions. 71 australasian journal of information systems volume 13 number 2 may 2006 principles of multi-grounded theory origin in grounded theory grounded theory (gt) is an established research approach for qualitative analysis in social research (glaser & strauss, 1967; glaser, 1978; strauss, 1987; strauss & corbin, 1990; strauss & corbin, 1998). as such, gt has also been established within information systems research (e.g. urquhart, 2001). gt is a method for analysing data, mainly qualitative data. gt is a method to support the, often difficult, work to go from large amounts of qualitative data via abstractions and conceptualisations to a condensed theory. the sociologists barney glaser and anselm strauss developed this approach during the sixties and published a seminal book “the discovery of grounded theory” (glaser & strauss, 1967). this book has had a great impact on qualitative social research. gt has been further developed and it has later turned into different “dialects”. in the beginning of the nineties there was a break between the two originators. glaser (1992) made a harsh critique of a book co-written by strauss (strauss & corbin, 1990). this dispute between the originators has been discussed by several scholars, e.g. babchuk (1997), bryant (2002), charmaz (2000), smit & bryant (2000) and urquhart (2001). gt is an inductive approach to theory development. it starts with analysis of empirical data. this data analysis should be performed in a way that is theoretically unprejudiced. the originators explicitly warn against reading literature concerning other theories before performing data collection and data analysis. “an effective strategy is, at first, literally to ignore the literature of theory and fact on the area under study, in order to assure that the emergence of categories will not be contaminated by concepts more suited to different areas.” (glaser & strauss, 1967 p 37). the analysis of data is performed in a series of coding steps. according to strauss & corbin (1990) three coding steps should be performed: open coding, axial coding and selective coding. in the open coding, the data are scrutinized and a first conceptualisation is made. categories are formulated, which in the axial coding are grouped into categorical structures. an action-theoretical paradigm model is used in this axial coding. originally in strauss & corbin (1990) this paradigm model was fairly complicated. later, in the second edition of their book, strauss & corbin (1998) simplified this action model (conditions � actions � results/effects). the controversy between glaser and strauss is concerned with the use of such an action oriented paradigm model; confer the harsh critique in glaser (1992). urquhart (2001 p 115) also criticizes the action paradigm model: ”put simply, i found it difficult to apply the coding paradigm, and the relationships between codes and categories hard to discover.” she has in her analyses given preference to glaser. it must however be noted that her critique is towards the more complicated action model in strauss & corbin (1990) first edition of their book. mgt is in adherence with the much simpler action paradigm model of the second edition (strauss & corbin, 1998) mentioned above. in the selective coding, the emerging theory is condensed and a main theoretical category is formulated. through these coding processes, categories and theory are generated from data and also automatically grounded in these data. a ‘grounded theory’ means theory grounded in empirical data. grounded theory, in its different variants, has been widely used in research endeavours. the gt approach has been assessed and criticized by several scholars; e.g. charmaz (2000), bryant (2002) and urquhart (2001). one criticism is the pure inductive approach to data analysis where relations to other theories are left out (e.g. bryant, 2002; cronholm, 2002). another criticism is a supposed unclear epistemological basis (e.g. bryant, 2002; charmaz, 2000). australasian journal of information systems volume 13 number 2 may 2006 72 revised to multi-grounded theory based on experiences and analysis of strengths and weaknesses (cronholm, 2002) a revised approach, called multi-grounded theory (mgt), has been formulated by goldkuhl & cronholm (2003); confer also axelsson & goldkuhl (2004), cronholm (2004; 2005) and goldkuhl (2004a; 2004b). mgt should be seen as an extension to or modification of gt. we have mainly followed strauss & corbin (1990; 1998) in our revision. we will comment on this preference below. the main principles of the multi-grounded theory approach will be presented below. gt favours a strict empirically driven analysis. start with the empirical data and then abstract and categorize is the motto. gt is as a pure inductive approach contrasted to a theory-driven deductive analysis. mgt is an attempt to combine certain aspects from inductivism and deductivism, as a kind abductivism (alvesson & sköldberg, 1999). it incorporates both an inductive analysis and a theoryinformed analysis of data (figure 1). multi-grounded theory "combined view" theory-driven analysis "deductivism" empirically driven analysis "inductivism" thesis antithesis synthesis figure 1: multi-grounded theory as a dialectical synthesis between inductivism (gt) and deductivism (from goldkuhl & cronholm, 2003) mgt includes a more systematic use of pre-existing theories than pure gt. besides empirical grounding, two more grounding processes (theoretical and internal) have been added. this means that it is an enhanced grounding perspective behind mgt: a multi-grounded perspective. a multigrounded theory is a theory grounded in: • “empirical grounding” empirical data (preferably through mainly an inductive approach) • “theoretical grounding” pre-existing theories (well selected for the theorized phenomena) • “internal grounding” an explicit congruence within the theory itself (between elements in the theory) these three grounding processes emanates originally from goldkuhl (1999), later also described in goldkuhl (2004b). one criticism raised against gt is that the analysis can be too unfocused both in empirical and theoretical phases (cronholm, 2002). the research questions may be too vague. in the mgt approach, a continual refinement of research questions is emphasised. “in a pragmatic spirit we think that it is often reasonable to think through one’s research questions to some depth at a start. it is however important to be open-minded during the research process and let empirical observations and theoretical insights influence the research interest. it is fully acceptable to let the research questions develop through the empirical and theoretical work.” (goldkuhl & cronholm, 2003). 73 australasian journal of information systems volume 13 number 2 may 2006 compared to ordinary gt the role of theories and research interest are emphasised more in the mgt approach. the research interest (operationalised in research questions) should evolve over time and one should use external theories in a constructive way throughout the research process (figure 2). theory development empirical data existing theories research interest multi-grounded theory figure 2: empirical data, research interest and existing theories informing the development of a multi-grounded theory (from goldkuhl & cronholm, 2003) the multi-grounded theory approach has been operationalised into a number of work steps as a method for data analysis and theory development (figure 3). it consists of the following main working areas : • research interest reflection and revision • theory generation • explicit grounding theory development theory generation i inductive coding conceptual refinement building catogorical structures research interest reflection and revision explicit grounding processes theoretical matching explicit empirical validation evaluation of theoretical cohesion theoretical generation ii theory condensation figure 3: working structure of the mgt approach (from goldkuhl & cronholm, 2003) australasian journal of information systems volume 13 number 2 may 2006 74 theory generation corresponds to the coding processes in gt (strauss & corbin, 1990) in certain ways. ‘open coding’ (in gt) corresponds fairly well to ‘inductive coding’ (in mgt). the openminded way to analyse data of gt is also emphasized in mgt (ibid). after the inductive coding there is a ‘conceptual refinement’ in mgt, which is not found in gt (see ibid and especially goldkuhl (2002; 2004a)). axial coding (in gt) corresponds to ‘building categorical structures’ in mgt (see goldkuhl & cronholm (2003) and especially axelsson & goldkuhl (2004)). the stage of ‘theory condensation’ (in mgt) corresponds to ‘selective coding’ in gt. the strong claim for one core category (in gt) is however not raised in mgt. theory condensation is a concluding stage in mgt. it should however be preceded by three explicit grounding processes: theoretical matching, evaluation of theoretical cohesion, and empirical validation. the two first grounding processes (theoretical and internal grounding) cannot be found in gt as such explicit processes. a ‘multigrounded theory’ means a theory grounded in empirical data, other theories and internally validated. mgt and the different variants of gt how is mgt positioned in relation to the two variants of gt; glaser vs strauss? several scholars within is argue for the importance to make an explicit stance in the gt dispute; e.g. urquhart (2001) and smit & bryant (2000). we agree with this and have declared above that mgt is more in line with strauss than glaser. what are the reasons for this stance? there are many good features of gt, not only the inductive way of building theories from data. another very important aspect is the pragmatic orientation of gt. already in the beginning of glaser & strauss (1967 p 1) such pragmatic features of a grounded theory are put forth: ”such a theory fits empirical situations, and is understandable to sociologists and laymen alike. most important, it works provides us with predictions, explanations, interpretations and applications.” in later parts of their book (especially chapter 10) this pragmatic orientation has come through even more elaborate. the link to american pragmatism and especially to dewey (1938) is clearly described. as bryant (2002) points out this pragmatic orientation has not always been recognized as an important part of gt. one can, however, read the different books by strauss (glaser & strauss, 1967; strauss 1987; strauss & corbin 1990) and clearly follow a pragmatic thread through his writings on gt. his foundation in american pragmatism has been most clearly been described in strauss (1993). the discourse of the gt dispute seems to have been set by glaser (1992). strauss is considered to be the renegade from original gt. e.g. charmaz (2000) and urquhart (2001) seem to follow glaser’s way of defining the dispute. there does not seem to be any explicit answers from strauss concerning glaser’s critique. this might be an explanation why some scholars tend to follow glaser’s arguments too easily. but is it really so that strauss is the renegade? the pragmatic orientation in original gt has clearly been maintained by strauss. such a clear pragmatic orientation cannot be found in glaser (1978; 1992). in this alternative way of defining the dispute, glaser can be seen to be the renegade! is is a science of the practical, and as such it is important that theories and other research products are useful for the is practice. there are several scholars, when arguing for such a practice orientation, who identify pragmatism to be a proper foundation for is research (e.g. goles & hirschheim, 2000; baskerville & myers 2004; goldkuhl 2004c). we agree with this and believe that gt in its pragmatic orientation is good analytic instrument for such is research. a good analytic instrument can however be even better. mgt should be seen as improvement of gt, still in a pragmatic vein. this discussion will be furthered below through the concept of practical theory. as claimed above, mgt is a research approach based on a dialectic interplay between deductivism as the thesis and inductivism as the anti-thesis. different versions of gt have been used as the anti 75 australasian journal of information systems volume 13 number 2 may 2006 thesis, but as claimed in this section the strauss way has made more important marks in mgt because of its pragmatic emphasis. we acknowledge the need for an inductive way of building theories, but there is also a need to involve other processes (as e.g. conceptual refinement and explicit grounding processes). we build on grounded theory, but since we see multi-grounded theory as a further refined approach (in relation to grounded theory) we have chosen to label the research approach as multi-grounded theory. glaser (1992) means that not even straussian gt should be called grounded theory. mgt differs even more from glaser’s ideal. the evolution of a business process theory the case study described in this paper was driven from the need to develop a business process theory to be used in business analysis when developing information systems. during the period 1994 2001 a research project concerning a business process theory as a basis for systems development was conducted by the authors of this paper. this means that we use our own research process as the main object for this meta research. we refer to ourselves as “the key researchers” in the following. the research project was heavily influenced by process orientation. in this section the research process concerning the evolution of this business process theory is described. result from this business process theory development is summarised in lind (2002). the theory development was inspired by the principles of multi-grounded theory without being a strict application of these principles. in fact mgt was evolving as an approach to research methodology during the same time. the evolving mgt approach has been applied in several research studies. one of them was the development of this business process theory. other studies were concerned with information systems architecture (axelsson & goldkuhl, 2004), usability of case tools (cronholm, 2004), information systems actability (ågerfalk, 2004), method configuration (karlsson, 2005) and knowledge management (braf, 2004). the research process is divided into three episodes, which will be reconstructed in the next section. each episode generated important results, which were important parts in building the multi-grounded theory (i.e. the business process theory). the description of the three episodes is followed by a reflection of in what ways this research process is related to the multi-grounded theory approach. episode 1: initialising the research process it was the year of 1994. business process reengineering (bpr) (cf. hammer & champy, 1993; davenport, 1993) had put forward a business process perspective on organisational work. traditionally, organizations have in their development efforts focused on their internal activities at the expense of focusing on their environment. in different approaches for organizational development, such as total quality management tqm (harrington, 1991), bpr and process management, there was an emphasis on horizontal work processes combined with a customer focus. common to all these approaches is that they focus on business processes in order to facilitate the value creation and need satisfying of the customers or clients of the organization. two researchers in information systems (the authors of this paper) watched this development trend in organisational management. these researchers had a strong emphasis in their research that information systems and practices need to be co-designed. this means that perspectives on organisational work need to influence the analysis performed of the workpractice (cf. analysis of the object system according to langefors, 1973) when developing information systems. questions such as “how will this management trend influence system development?”, “what characterises a process oriented system development?”, “how shall a process oriented system development be conducted and australasian journal of information systems volume 13 number 2 may 2006 76 how are existing methods affected?” arose. literature on business processes had at that time a strong emphasis on perspectives and concepts rather than methods. there were no guidelines on how to perform process oriented development work. the key researchers found a suitable empirical object, a steel company. this company had an ambition to work with the development of new information systems with a strong business process emphasis. a change analysis was conducted which included activity analysis, problem analysis, strength analysis, goal analysis and formulation of change requirements (goldkuhl & röstlinger, 2005). an important ambition with the change analysis was to achieve a high integration of production and administrative work. it was thus important to study details as well as holistic aspects of the business. early in this project it was identified that the organisation performed business in different ways, thus one could say that it existed different business processes. during the activity analysis separate series of process models were drawn covering the contents of each different business process. this kind of process model separation was made in order to reveal the diverse and specific logics of the different business processes. the insight that the business was performed in different ways created a need for knowledge regarding how to determine (= delimit and divide) business processes in relation to each other. a suitable process determination was performed of the company in order to distinguish different business processes. the notion of variant processes was coined. the key researchers studied the ways business processes had been delimited in the change analysis. the researchers reconstructed and abstracted the used criteria. two criteria were used to perform the determination: internal handling (as an expression for the content of the business processes) and customer relations (as an expression for how the exchanges between the steel company and the customer were conducted). two categories of customer relations were distinguished; separate ordering and frame contracting. by combining these criteria six variant processes had been distinguished. examples of identified variant process were ‘standard stock customer’, ‘frame contracting customer’, ‘trading’. the insights from this field study led the key researchers to refine the research question and the research purpose. they moved from the question of how to perform process oriented systems development to the question of how to determine business processes and criteria for such determination. when looking into the literature the key researchers also found arguments that this was an important problem. how is a business process constituted? how can a process be delimited and divided? these were issues not yet resolved theoretically or practically. one of the “process gurus” thomas davenport (1993, pp. 27-28) claimed that “considerable controversy resolves around the number of processes appropriate to a given organization. the difficulty derives from the fact that processes are almost infinitely divisible; the activities involved in taking and fulfilling a customer order, for example, can be viewed as one process or hundreds. the ’appropriate’ number of processes has been pegged out from two to more than one hundred”. the first case study gave a first tentative definition of the business process notion and also criteria for process determination. a need to direct attention towards process variants had been acknowledged. this episode of initiation of the research process had given fruitful conceptions for future research. this concerned especially the variant process concept and some preliminary criteria for process determination. 77 australasian journal of information systems volume 13 number 2 may 2006 episode 2: more empirical data and the introduction of antagonistic theories the further research was characterised by parallel empirical generation/validation and theoretical development (see figure 4). episode 1 empirical generation / validation theoretical development empirical studies provisional constructs and a refined research interest episode 2 figure 4: the two first research episodes the conducted empirical projects were oriented towards application of earlier generated knowledge (the notion of business processes and the determination criteria). approximately 15 action research projects were conducted. these projects were performed in many different settings. in these projects it was very important to arrive at practical results. an important foundation in these projects was the need to rather rapidly perform process determinations. requirements had been put upon the process theory to manage process determination in a well-founded way. this also meant that the research performed in the empirical field gave new insights regarding suitable ways of making process determinations. different settings gave new dimensions to process determination. during the empirical studies the key researchers understood the importance of managing different customers in different ways. one example of such situation was from a sawmill where a distinction was made between core customers and potential core customers as well as other customers (without the potential of becoming core customer). another aspect that was important, identified in the same empirical setting, was the differentiation between different types of products. the sawmill worked with three types of products; customer-unique products, standard products as well as waste products. different kinds of customers bought different kinds of products. these circumstances suggested that there existed a close relationship between the way the interaction between the customer and the company was conducted and the characteristics of the products that were offered. parallel to the empirical studies theoretical development was performed. in theoretical discourse on business processes it had been identified that there existed two conflicting families of theories (cf. e.g. keen & knapp, 1996). the predominant view regarded a business process as a phenomenon that takes one or more inputs and transforms these to an output that is of value to the customer. this view was expressed in much bpr literature. it originates from tqm and can be seen to be a traditional industrial view on business processes (e.g. harrington, 1991). tqm has a focus on transformation of raw material to finished product. within the bpr approach this view was then widened in order to include more commercial aspects. in contrast to this transformative view on business processes there was also the communicative view based on language/action-oriented theories; confer winograd & flores (1986) who base their work on the speech act theory of searle (1969). this perspective is based on the idea that communication is not just transfer of information. when you communicate you also act. as a reaction against the transformative view on processes there exist several process-oriented approaches for business australasian journal of information systems volume 13 number 2 may 2006 78 modelling11 based on the language/action perspective. by applying a communicative view on processes the organization’s establishment and fulfilment of commitments were emphasised. the key researchers acknowledged these conflicting views. they saw the need to position their emerging theories in relation to these theories. they felt that they could not adopt either view. a third alternative was to develop a new theory. the key researchers therefore adopted a dialectic approach to develop a new business process view as a synthesis. according to hegel’s dialectic thinking (skirbekk & gilje, 2001), knowledge development can be divided into three main phases: 1) the thesis as the original state, 2) the antithesis as the reaction against the thesis, and 3) the synthesis that resolves the oppositions between the thesis and antithesis. development of syntheses can be characterised by taking the good parts from the thesis and the antithesis and avoiding antagonistic parts through transformation and creation of something new. the transformative view on business processes was regarded as the thesis in this dialectical approach. important strengths of the transformative view on business processes were the focus on activity chains, value-adding activities, and on customers. weaknesses were a nearly total dissociation of coordination and communication. a communicative view on business processes was regarded as an antithesis in relation to the transformative view (the thesis) on business processes. strengths in the communicative view on business processes were a focus on communication as the backbone. one main weakness was however a total dissociation of material acts. in order to arrive at a comprehensive understanding of the thesis and the antithesis conceptual modelling was performed. the different concepts had to be clarified. a thorough analysis was performed in which a number of different concepts with relations to the concept of business process were identified. conceptual modelling as well as goal modelling also served as an important tool in deriving a congruent synthesis. aspects identified through empirical observations inspired the creation of synthesis. these aspects helped the key researchers in directing attention towards contradictions between the transformative view on business processes (the thesis) and the communicative view on business processes (the antithesis). by identifying these contradictions, strengths and weaknesses could be identified in the thesis and the antithesis. this basis was however not enough for developing the synthesis. the key researchers found that it was necessary to move to an ontological level. in order to determine which concepts and values to include in the synthesis the dialectical process was guided by generic action models together with empirical observations. these action models are in this process to be regarded as an ontological foundation (generic theory) used to secure that chosen concepts were possible to relate to each other. different types of theories were thus used; antagonistic theories and theories as the ontological foundation. workpractice theory and socio-instrumental action theory (goldkuhl & röstlinger, 2003) formed the ontological foundation (generic action models). four important forces have thus influenced the development of a new business process theory (see figure 5). 11 example of approaches based upon the language/action perspective are action workflow (medina-mora et al, 1992), demo (dietz, 1999), which are influenced by the conversation-foraction schema (winograd & flores, 1986). 79 australasian journal of information systems volume 13 number 2 may 2006 synthesis: a further developed notion of business processes thesis: a transformative view on business processes theories about practices and actions empirical observations antithesis: a communicative view on business processes figure 5: influences on and procedure for developing the synthesis the further developed notion of processes presented in lind (2002) and lind & goldkuhl (2006), was based upon a transformative and a communicative view on processes, empirical observations, and foundational theories (see also figure 6), implied that: • there was a need to separate work performed in the organization for potential respective particular clients; • processes could not exclusively be regarded as either transformation or communication; transformation needs to be regarded in an coordination context; • processes could not exclusively be regarded as sequentially related sub-processes; there exist variants of processes consisting of sequentially related sub-processes; • a one-sided focus on the customer not was enough; there was also a need to focus suppliers and other parties related to the business; • an asymmetric view on customer satisfaction not was enough. there was a need to acknowledge reciprocal relationships between customer and supplier and mutual satisfaction of both parties. client transformation process basis result agree upon assignment fulfil assignment conclude assignment clientbusiness figure 6: synthesis: transformation in an assignment perspective (from lind, 2002) the result from this episode, in which parallel theoretical development and empirical studies were performed, was a synthesised theoretical conception, i.e. further developed notion of business processes. this theoretical conception was improved by empirical observations. australasian journal of information systems volume 13 number 2 may 2006 80 episode 3: theoretical refinement and further empirical grounding the key researchers were now satisfied in having developed a business process theory that covered both transformative aspects as well as coordinative aspects of business processes. this further developed notion of business processes was now used to re-evaluate the process determinations made in earlier empirical studies. this investigation revealed new criteria to take into consideration when dividing organisational work into business processes. in order to avoid researcher bias, secondary empirical cases were analysed. secondary empirical cases consisted of empirical data not generated/collected by the key researchers. these new criteria could be divided into two dimensions of criteria that were combined when performing process determination. the first was the product dimension and included an analysis of different product characteristics (cf. goldkuhl & röstlinger, 2000) and the second dimension was the business relation dimension. the product dimension concerned different types and classes of products. the business relation dimension mainly distinguished between separate orders (one-time transactions) and frame contracting (long-term agreements); goldkuhl & lind (2004). these two dimensions of criteria were made the subject of several theoretical matching processes. to conclude, one can state that there had been a gradual interplay between theoretical studies and empirical studies in order to establish an appropriate business process theory with appurtenant criteria for process determination. important to acknowledge in this research process was the need for a foundational theory (such as socio-instrumental pragmatism) as an enabler for developing a consistent and transparent theory. different procedures therefore needed to be adopted. this is in line with blumer (1954) who reasoned "theory, inquiry and empirical facts are interwoven in a texture of operation with theory guiding inquiry, inquiry seeking and isolating facts, and facts affecting theory. the fruitfulness of their interplay is the means by which an empirical science develops". after this episode the key researcher felt that a full-blown multi-grounded theory (i.e. an empirically, theoretically and internally grounded theory) had evolved. the evolution from a mgt-perspective this description of the evolution of the multi-grounded business process theory with ancillary criteria for process determination is a reconstruction of how the theory development has been conducted. the research was not designed in this way from the beginning. it has rather been a continual design of the research process based on discoveries made as the evolution went on. this research process can be seen as an application of the multi-grounded theory approach (see section 2). the research process has been inspired by the evolving mgt approach. during the research process there has been interplay between empirical data and existing theories as a basis for refinement of the research interest. research questions and hypotheses have gradually evolved through empirical and theoretical analyses. during this research process there has been recurrent theoretical matching as well as recurrent empirical validation. the primary empirical data has been supplemented with additional empirical data (secondary empirical data). the primary empirical data has both been used for generating theory, i.e. a synthesised theoretical conception of business process, as well as a basis for reconsidering the empirical data in the light of a new theory and thereby generating criteria for business process determination. an important source for developing the business process theory has been the analyses of competing theories on business processes. the whole research process has been guided by certain ontological foundations based in work practice 81 australasian journal of information systems volume 13 number 2 may 2006 theory and social action theory. in the theoretical grounding theories for strengthening the explanatory power of different concepts and values have also been used (as e.g. in the properties of the two dimensions of criteria; product and business relation). in this mgt-development three different types of theories have been used; 1) antagonistic theories, 2) theories forming the ontological foundation, and 3) theories for strengthening the explanatory power of concepts and values. the analysis and development has been supported by performing modelling such as e.g. conceptual modelling and goal modelling in order to arrive at a conceptual cohesion of the evolving theory. these modelling activities are important to ensure an internally grounded theory. going back to figure 3 (describing the mgt approach) one can conclude that the evolution described in the episodes above can be understood according to the different steps in the working structure of mgt. one can conclude that all the different episodes are important for many of the processes of generation and validation. there has been a continual interplay between several of these processes in order to create a multi-grounded theory. it is however important to notice that the point of departure has been in inductive coding, conceptual refinement and building categorical structure (theory generation 1). this research study is thus a recognition of the need to let empirical data guide the continuous research process. the developed business process theory has been put into application in several occasions after the development reported in this paper (e.g. lind et al, 2003; lind & olausson, 2004; haraldson & lind, 2005; eriksson & lind, 2005; lind & goldkuhl, 2006). in several business analysis projects this full-blown multi-grounded business process theory has been used for determination of business processes in many different kinds of practices. the theory has shown to be a good support in directing attention towards essential aspects of the studied workpractice and thus performing wellfounded process determination. this practical use of the business process theory means a further empirical grounding. this is however a grounding that goes beyond observational grounding. it implies an application grounding (goldkuhl, 2004a). the evolving theory is not only a conceptualisation of an empirical domain. the concepts have shown to be useful in application for practical change. this process of putting the theory into application has also created incentives for further development. to create concepts and theory useful for practical change is in line with the notion of a practical theory (cronen, 2001). the main source of inspiration for the concept of practical theory put forth by cronen is pragmatist philosopher dewey (1938) with his logic of inquiry. this can be compared with what strauss (1987 p 6) says about the relation between gt and the pragmatism of dewey: ”for a fuller historical understanding of the background of grounded theory, it would be useful to read john dewey’s logic: the theory of inquiry [dewey, 1938]…”. the idea of application grounding (and practical theory) is actually in full accordance with the pragmatic claims in original gt (glaser & strauss, 1967) and strauss’ further writings. conclusion in this paper we have put the mgt research approach into application by studying the evolution of a business process theory with ancilliary criteria for process determination. important to acknowledge is that research methodological considerations have been made continually during this theory evolution. the evolution can be characterised as an interplay between empirical studies, theoretical analysis and building a congruent theory with appurtenant criteria. the business process theory has been internally, empirically and theoretically generated as well as grounded. the empirical studies have not just been about data collection – they have also been arranged data generation efforts based on certain theoretical needs. criteria for business process determination have been generated by having the theory put into application a kind of theoretical sampling. this business process theory australasian journal of information systems volume 13 number 2 may 2006 82 is also to be regarded as a practical theory (cronen, 2001), i.e. the theory has been applied in practical problem solving. there has been an active application of the principles of the business process theory in the different action research studies. the theory has not only been empirically generated through observation and inductive analysis. the theory has also been validated and modified through its application in different business process analysis settings. we believe that mgt is describing the way successful theory development actually is performed in a field such as information systems. it is important to pay attention to and interact with the empirical field without being theoretically naïve. that risk is present when applying a pure grounded theory approach in the research process. the multi-grounded theory approach builds upon a combined inductive, empirical and theory informing approach. this research approach allows also a continual refinement of research interest and research questions. our stance is also that research methods (such as for example gt and mgt) should be made objects for empirical research. in this paper we have put the multi-grounded theory approach into an empirical application by giving a historical description and analysis of an actual research process that has been conducted. we claim that many descriptions of research methods are just theoretical arguments. in this paper we have studied a research method by its application in an empirical example. the evolution of a business process theory has been seen as a case study of theory development. studies on research methods should be empirically based not only a theoretical discourse. research on the multi-grounded theory research approach should in itself be made in a multi-grounded way as we have tried to show in this paper! this paper is thus to be seen as a contribution to the validation of the mgt approach. other papers, as axelsson & goldkuhl (2004), cronholm (2004; 2005) and ågerfalk (2004), have also made empirical contributions to the multigrounded theory approach. references ågerfalk p (2004) grounding through operationalization: constructing tangible theory in is research, in proc of the 12th european conference on information systems (ecis 2004), turku alvesson m., sköldberg k. (1999) reflexive methodology: new vistas for qualitative research, sage, london axelsson k., goldkuhl g. (2004) theory modelling action focus when building a multigrounded theory, in proc of the 3rd european conference on research methodology for business and management studies, reading babchuk wa (1997) glaser or strauss?: grounded theory and adult education, midwest researchto-practice conference in adult, continuing and community education, michigan state university baskerville r, myers m (2004) special issue on action research in information systems: making is research relevant to practice – foreword, mis quarterly, vol 28 (3), pp 329-335 blumer h. (1954) what is wrong with social theory?, american sociological review, vol 19, pp 310 braf e (2004) knowledge demanded for action studies of knowledge mediation in organisations, phd diss, department of computer & information science, linköping university 83 australasian journal of information systems volume 13 number 2 may 2006 bryant a. (2002) re-grounding grounded theory, the journal of information technology theory and application, vol 4, no 1, p 25-42 charmaz k. (2000) grounded theory: objectivist and constructivist methods, in denzin n k, lincoln y s (eds, 2001) handbook of qualitative research, 2 ed, sage, thousands oakes cronen v. (2001) practical theory, practical art, and the pragmatic-systemic account of inquiry, communication theory, vol 11, no 1 cronholm s. (2002) grounded theory in use – a review of experiences, in proceedings of european conference on research methods in business, reading cronholm s. (2004) illustrating multi-grounded theory, accepted to the 3rd european conference on research methodology for business and management studies, reading cronholm s. (2005) multi-grounded theory in practice – a review of experiences from use, in proc of qualit2005, brisbane, australia davenport t. h. (1993) process innovation – reeingeering work through information technology, harvard business school press, boston dewey j (1938) logic: the theory of inquiry, henry holt, new york dietz j. l. g. (1999) understanding and modelling business processes with demo, proc. 18th international conference on conceptual modeling (er'99), paris eriksson o., lind m. (2005) “problems with communication quality in commitment management” in goldkuhl g., clarke r. j., axelsson k. (eds.) proceedings of the international workshop communication and coordination in business processes, linköping university and university college of borås, sweden giddens a. (1984) the constitution of society. outline of the theory of structuration, polity press, cambridge glaser b (1978) theoretical sensitivity: advances in the methodology of grounded theory, sociology press, mill valley, california glaser b. (1992) basics of grounded theory analysis: emergence vs. forcing, sociology press, mill valley, calif. glaser b., strauss a. (1967) the discovery of grounded theory, aldine, new york goldkuhl g. (1999) the grounding of usable knowledge: an inquiry in the epistemology of action knowledge, accepted to hss99, falun; also working paper 99:03 cmto, linköping university goldkuhl g (2002) anchoring scientific abstractions – ontological and linguistic determination following socio-instrumental pragmatism, in proceedings of european conference on research methods in business, reading goldkuhl g (2004a) conceptual determination when developing a multi-grounded theory – example: defining isd method, in proc of the 3rd european conference on research methods in business and management (ecrm 2004), reading goldkuhl g (2004b) design theories in information systems – a need for multi-grounding, journal of information technology theory and application (jitta), 6(2) p 59-72 goldkuhl g (2004c) meanings of pragmatism: ways to conduct information systems research, in proc of the 2nd intl conf on action in language, organisations and information systems (alois), linköping university australasian journal of information systems volume 13 number 2 may 2006 84 goldkuhl g., cronholm s. (2003) multi-grounded theory – adding theoretical grounding to grounded theory, in proc of the 2nd european conference on research methods in business and management (ecrm 2003), reading goldkuhl g., lind m. (2004) the generics of business interaction emphasizing dynamic features through the bat model, in proc of the 9th intl conference on the language action perspective (lap2004), rutgers university goldkuhl g., röstlinger a. (2000) beyond goods and services an elaborate product classification on pragmatic grounds, in proc of quality in services (quis 7), karlstad university goldkuhl g, röstlinger a (2003) towards an integral understanding of organisations and information systems: convergence of three theories, in gazendam h wm, jorna r j, cijsouw r s (eds, 2003) dynamics and change in organizations. studies in organizational semiotics, kluwer, boston goldkuhl g., röstlinger a. (2005) change analysis – innovation and evolution, invited paper to the 14th intl conf on information systems development, karlstad university goles t, hirschheim r (2000) the paradigm is dead, the paradigm is dead … long live the paradigm: the legacy of burell and morgan, omega, vol 28 p 249-268 hammer m., champy j. a. (1993) reengineering the corporation: a manifesto for business revolution. nicholas brealy, london. haraldson s., lind m. (2005) ”broken patterns” in goldkuhl g., lind m., haraldson s. (eds.) proceedings of the 10th international working conference on the language action perspective on communication modelling, linköping university and university college of borås, sweden harrington h. j. (1991) business process improvement: the breakthrough strategy for total quality, productivity and competiviness. mcgraw hill, new york. karlsson f (2005) method configuration method and computerized tool support, phd diss, department of computer & information science, linköping university keen p.g.w., knapp e.m. (1996): every manager’s guide to business processes – a glossary of key terms & concepts for today’s business leader. harvard business school press, boston langefors b. (1973) theoretical analysis of information systems, fourth edition, studentlitteratur, lund latour b. (1992) technology is society made durable, in law (ed, 1992) a sociology of monsters: essays on power, technology and domination, routledge& kegan paul, london lind m. (2002) dividing businesses into processes – foundations for modelling essentials. in liu k et al (eds, 2002) organisational semiotics: evolving a science of information systems, kluwer, boston lind m., goldkuhl g. (2006) “designing business process variants – using the bat framework as a pragmatic lens” in c. bussler et al. (eds.): bpm 2005 workshops, lncs 3812, pp. 408 – 420, 2006, springer-verlag berlin heidelberg 2006 lind m., hjalmarsson a., olausson j. (2003) "modelling interaction and co-ordination as business communication in a mail-order setting" in weigand h., goldkuhl g., de moor a. (eds) proceedings of the 8th international working conference on the language-action perspective on communication modelling, tilburg university, the netherlands lind m., olausson j. (2004) "balancing horizontal and vertical co-ordination in business transactions towards a clarification of the role of it-systems in an e-commerce setting" 85 australasian journal of information systems volume 13 number 2 may 2006 in goldkuhl g., lind m., cronholm s. (eds.) proceedings of action in language, organisations and information systems, linköping university, sweden medina-mora r., winograd t., flores r., flores f. (1992) the action workflow approach to workflow management technology, in: turner j., kraut r. (eds.) proceedings of the conference on computer-supported cooperative work, cscw’92, acm press, new york orlikowski w. j. (1992) the duality of technology: rethinking the concept of technology in organizations, organization science, vol 3 (3) pp 398-429 rose j., jones m., truex d. (2003) the problem of agency: how humans act, how machines act, in proc of the international workshop on action in language, organisations and information systems (alois-2003), linköping university searle j. r. (1969): speech acts. an essay in the philosophy of language. cambridge university press, london skirbekk, g., gilje, nils., (2001) history of western thought: an introduction to the philosophical roots of modernity. taylor & francis books ltd. smit j, bryant a (2000) grounded theory method in is research: glaser vs strauss, working paper, leeds metropolitan university strauss a (1987) qualitative analysis for social scientists, cambridge university press, cambridge strauss a (1993) continual permutations of action, aldine de gruyter, new york strauss a., corbin j. (1990) basics of qualitative research. grounded theory, procedures and techniques, sage, newbury park strauss a, corbin j (1998) basics of qualitative research. techniques and procedures for developing grounded theory, 2nd edition, sage, newbury park urquhart c. (2001) an encounter with grounded theory: tackling the practical and philosophical issues, in trauth e m (ed, 2001) qualitative research in is: issues and trends, idea group, hershey walsham g. (1997) actor-network theory: current status and future prospects, in lee as, liebenau j, degross ji (eds, 1997) information systems and qualitative research, chapman & hall, london winograd t. , flores f. (1986) understanding computers and cognition: a new foundation for design, ablex, norwood nj australasian journal of information systems volume 13 number 2 may 2006 86 microsoft word ajis vol13-2 o.doc 151 australasian journal of information systems volume 13 number 2 may 2006 an exploration of user interface designs for real-time panoramic photography patrick baudisch, desney tan, drew steedly, eric rudolph, matt uyttendaele, chris pal, and richard szeliski microsoft research one microsoft way redmond, wa 98052, usa email: {baudisch, desney, steedly, erudolph, mattu, szeliski}@ microsoft.com pal@ cs.umass.edu abstract image stitching allows users to combine multiple regular-sized photographs into a single wide-angle picture, often referred to as a panoramic picture. to create such a panoramic picture, users traditionally first take all the photographs, then upload them to a pc and stitch. during stitching, however, users often discover that the produced panorama contains artifacts or is incomplete. fixing these flaws requires retaking individual images, which is often difficult by this time. in this paper, we present panoramic viewfinder, an interactive system for panorama construction that offers a real-time preview of the panorama while shooting. as the user swipes the camera across the scene, each photo is immediately added to the preview. by making ghosting and stitching failures apparent, the system allows users to immediately retake necessary images. the system also provides a preview of the cropped panorama. when this preview includes all desired scene elements, users know that the panorama will be complete. unlike earlier work in the field of real-time stitching, this paper focuses on the user interface aspects of real-time stitching. we describe our prototype, individual shooting modes, and provide an overview of our implementation. building on our experiences with panoramic viewfinder, we discuss a separate design that relaxes the level of synchrony between user and camera required by the current system and provide usage flexibility that we believe might further improve the user experience. keywords: panorama, panoramic viewfinder, user interface, interactive, stitching, realtime, preview. introduction to allow digital camera users to take pictures with a viewing angle wider than the one supported by the built-in lens, researchers have proposed techniques for stitching multiple photographs (mann and picard, 1994; szeliski, 1996) or a stream of video (teodosio and bender, 1993; irani and anandan, 1998) into a single contiguous panoramic picture, or panorama. recent full-view stitching methods even allow automatic stitching of photos taken in arbitrary order and spatial arrangement (szeliski and shum, 1997; bergen et al., 1992). while some recent off-the-shelf cameras offer simple stitch assist modes (see related work section), stitching is still generally performed as a post-hoc step to picture taking. making flawless panoramas using a post-hoc stitcher can be challenging. we conducted an informal survey on an internal company mailing list for stitching users and received 26 responses out of 163 australasian journal of information systems volume 13 number 2 may 2006 152 mailing list members. each respondent had taken between 4 and 180 panoramas (median 20). all except one participant who had shot all his panoramas using a tripod had discovered at least one of the following flaws when stitching their pictures: • ghosting (88% respondents): objects that move between frames end up appearing translucent when the stitcher blends frames together. since cameras typically show only one picture at a time, ghosting can be hard to detect. • missing content (65% respondents): users may miss relevant areas or relevant areas may disappear when cropping the panorama to its final rectangular shape, as illustrated in figure 2. even if users are aware of the danger, the difficulty of understanding perspective projections (edwards, 1999) in combination with the particular geometry of panoramas (cylindrical or spherical projection) can make it hard for users to estimate what space needs to be covered. • stitching failed (38% respondents): the stitcher was unable to connect one or more frames to the rest of the panorama. reasons include lack of overlap between pictures, lack of the texture, or flaws in the pictures, such as poor focus or motion blur. a b c figure 2: common mishap when taking panoramas: (a) the user is trying to cover the seattle space needle, but does not cover enough of the sky around it. during cropping, the user has the choice between (b) including white fringes and (c) cropping relevant content. fixing these flaws requires retaking images, but by the time users discover the errors during stitching, it is often too late. 153 australasian journal of information systems volume 13 number 2 may 2006 while other researchers have looked at real-time previews generated using real-time stitching from a technical perspective (e.g. peleg and herman, 1997), we address the above issues by providing users with a novel user interface that allows them to utilize these previews in order to verify which areas of the scene have been successfully covered. the user interface of our system, called panoramic viewfinder, is designed to match the interaction model of existing digital cameras. in fact, we have constrained our designs to depend only a mode selector and a shutter button so that it may be ported to existing camera interaction mechanisms. we begin with a walkthrough of the system. we then go over related work and describe the implementation of our prototype. we describe our experiences with this prototype and discuss the broader space of solutions. we also propose a second design, which alleviates some of the shortcomings of our current design. we conclude with a summary of our findings and a description of future work. panoramic viewfinder panoramic viewfinder is an interactive system for panorama construction that provides users with a real-time stitched preview of the panorama while shooting (baudisch et al., 2005). figure 3 shows the user interface of our prototype system, here running on a sony u50 device with an attached webcam. panoramic viewfinder offers three main user interface elements designed to help complete the desired panorama, i.e., (1) the preview shows the panorama in its current state of completion, (2) the viewfinder shows what the camera sees at this moment, integrated at the correct location into the panorama, and (3) the real-time cropping frame shows the extent the panorama will have after cropping if the user stops shooting at this instant. australasian journal of information systems volume 13 number 2 may 2006 154 cropping frame: this area will survive cropping cropping frame: this area will survive cropping preview: overall area covered so far preview: overall area covered so far viewfinder: what the camera sees right now viewfinder: what the camera sees right now 1 23 figure 3: the panoramic viewfinder user interface on a sony u50 ultra portable pc walkthrough figure 4 illustrates the user’s interaction with the panoramic viewfinder using an example scenario. the user’s task is to take a panoramic picture of the front of a building complex 16 . this example features our system’s manual mode, in which the user manually triggers each photo. we explain the automated modes later in the paper. 1. the camera is ready, but the user has not triggered the shutter button. the lcd shows what the camera sees, just as in any other camera. 2. the user has pressed the shutter button and the camera has taken the first picture. from now on, the viewfinder is displayed in its relative location with respect to the panorama. this is made possible by continuously matching the viewfinder contents with the panorama (realtime stitching, see implementation section). 3. the user pans the camera and takes additional pictures. since these are automatically added to the panorama, the panorama grows. whenever necessary, the panorama preview is automatically re-scaled to fit the screen. the cropping frame is also updated continuously, and it always represents the largest rectangle that can be inscribed into the current 16 for a demo video of our system see http://www.patrickbaudisch.com/projects/panoramicviewfinder 155 australasian journal of information systems volume 13 number 2 may 2006 panorama. before stopping, the user checks the cropping frame and sees that stopping now would cause part of the building to be removed during cropping! 4. the user takes an additional shot of the top left corner. this completes the bounding box and the cropping frame now contains the entire building complex. 5. when uploading the photo to the pc, the panorama is restitched using our high-quality offline stitcher (brown et al., 2005) to obtain maximum image quality. when cropping the panorama to the rectangular shape, all the desired content is preserved. the user already knows that this will be successful, because the cropping frame has indicated that all the required material has been taken. while we expect that most users will choose to keep the default cropping frame, as displayed in the preview, we do permit users to modify the final cropping if they so desire. in this way, they can perform non-standard (e.g. non-rectangular) crops. whatever their final intent, they know from the preview that they have captured the desired material for their composition. in summary, panoramic viewfinder allows users to take pictures by first shooting the desired scene elements, then filling the bounding box around these elements. one key element in this process is the real-time cropping frame, which serves as a visual reference. the real-time cropping frame inverts the traditional cropping model: traditional post-hoc cropping requires users to narrow the image down to what is desired; using the real-time cropping frame, users keep adding content until the cropping frame has grown to cover the desired area. this approach has two main benefits. first, it avoids the need for providing the cropping frame with explicit controls, such as resize handles. this supports our goal of creating an interaction model that transfers seamlessly into a standard consumer camera, since operating a set of resize handles using the set of built-in buttons available on such cameras can be inefficient and frustrating. second, growing the cropping frame rather than narrowing it down increases users’ efficiency by helping them avoid taking undesired content in the first place. while the default action during upload is to crop each picture automatically as previewed using the real-time cropping frame, users can always override that choice by accessing the uncropped panorama and cropping it manually with resize handles. this works well in the offline mode because users have access to a mouse rather than having to use the camera as their input device. in addition to helping users take care of missing content, panoramic viewfinder can also help users detect ghosting, which typically becomes visible as an erroneous alignment between overlapping frames. finally, panoramic viewfinder provides support for the third problem, i.e., failure to stitch, by immediately notifying users whenever such a failure occurs. we describe this in the following section. dealing with stitching failure in order to place the viewfinder into the panorama preview, the system matches every new image received from the camera with the preview. in some situations, for example if the user moved to camera so quickly that the new frame cannot be matched, the system is unable to properly place the received image. if this happens, the viewfinder continues to update, but remains stationary and informs the user of the situation by changing the viewfinder frame from green to red and by playing a continuous error sound. australasian journal of information systems volume 13 number 2 may 2006 156 a b c d e figure 4: with panoramic viewfinder, panning the camera corresponds to a painting operation with a textured rectangular (actual screen captures with viewfinder and cropping frame emphasized to improve readability on b/w printouts) to fix the problem, users may pan the camera and take an additional shot that offers greater overlap with the existing panoramic image. alternatively, users may choose to zoom out to cover more of the image, and then zoom back in to fill in with higher resolution. as soon as the next match is found, the alert ends and the viewfinder continues to track within the preview. 157 australasian journal of information systems volume 13 number 2 may 2006 in addition to alerting users, panoramic viewfinder also provides a mechanism for reducing stitching errors in the first place. we first experimented with a video mode, i.e., adding each and every frame to the panorama. the implementation of such a video mode was simple, as every new frame is stitched temporarily anyway in order to properly place the viewfinder. while video mode eliminated most stitching errors, the large number of stored frames affected the frame rate and increased registration error (sawney et al., 1998). additionally, in order to optimize the stitching, we only attempted to match the current frame with the previous one (and not all others in the preview) in this mode. this meant that if a user lost track of the stitched preview, they would have to return the viewfinder to the location of the last frame taken, rather than being able to move it back to an arbitrary part of the existing image. we therefore created an automatic snapshot mode. in this mode, the system does not add frames to the panoramic mosaic until the stitcher loses track. when it does, the system automatically adds the last frame that it was able to successfully register. it then retries stitching the current frame which will typically now work, due to the extra frame in between. compared to video mode, automatic mode creates less data, is therefore faster, and leads to less accumulated registration error. it also allowed us to compare the current frame to the set of frames that constitute the entire panoramic picture. related work panoramic viewfinder touches three areas of related work: stitching, user interfaces designed to support stitching, and user interface and interaction research on small screen devices. we have already mentioned some approaches to stitching in the introduction. panoramic viewfinder uses a feature-based approach (brown et al., 2003). unlike image-based techniques (szeliski 1996; peleg and herman, 1997; szeliski and shum 1997) that attempt to match bitmaps, feature-based techniques reduce the complexity of the matching problem by using only a few salient image features, such as corners. these algorithms also tend to be more robust to scene motion and parallax induced by camera translations. the concept of real-time stitching was first explored by the videobrush system (peleg and herman 1997; sawhney et al., 1998; sawhney, hsu, and kumar, 1998). while videobrush offered most of the technical features of the real-time stitcher underlying panoramic viewfinder, it did not offer any particular user interface beyond displaying translated images on a computer screen. we believe that computing power embedded in mobile devices has now reached the point where we can begin to explore the user experience when such techniques are used in consumer electronics. in this paper, we propose a user interface concept to be added on top of real-time stitchers, such as videobrush, within the constraints of the interaction affordances in digital cameras. novel features include the scaling preview, the real-time cropping frame, warning when losing track, and the automatic snapshot mode. some current off-the-shelf digital consumer cameras, such as the canon powershot s230 (www.powershot.com), offer a stitch assist mode. to help users line up frames and to obtain the right amount of overlap between frames, the lcd shows the current viewfinder next to part of the last frame taken (figure 5a). in order to test the validity of the stitch assist concept for large panoramas, we implemented a 2d version of stitch assist (figure 5b). by showing cropped previous frames in all four directions, our 2d stitch assist allows users to add pictures in any of the four main directions. while we found this to work well for smaller panoramas, informal observations revealed that for larger panoramas, users quickly lost track of the shots they had taken and spots within the desired panorama that remained to be taken. additionally, since this scheme provides no feedback of australasian journal of information systems volume 13 number 2 may 2006 158 the quality of shots, users sometimes ended up with shots that were not ideal for stitching (e.g. shots that were out of focus or contained motion blur). a b figure 5: (a) the stitch assist mode, here from a canon powershot camera tells users where to take the next picture. the user pans the camera until the next picture on the right lines up with the piece of the previous picture on the left. (b) our 2d adaptation of stitch assist allows users to continue a panoramic picture in any of the four main directions. to address this latter problem, the hp photosmart r707 (www.hp.com) allows users to switch to playback mode and review panoramas on the camera’s lcd screen. although this allows users to see flaws while still on-site, fixing flaws still requires retaking the entire panorama. in contrast to both these schemes, panoramic viewfinder offers users more freedom by allowing them to take content in random order by “sweeping” camera over the desired scene. it also provides immediate feedback about the quality of the panorama so that the user can interactively fix flaws. the main interaction involved in using panoramic viewfinder, sweeping the camera over the desired scene to capture the relevant images, was inspired by various small-screen device interaction research projects. for example, early work by rekimoto (1996) exploring how users could interact with the digital world by directly manipulating the device itself led to interaction techniques in which users could control menus and scroll bars, as well as browse maps and 3d objects, by tilting the device. peep-hole displays (yee, 2003) and the chameleon system (fitzmaurice, 1993) treat the screen as a small viewport into a much larger world and allow users to explore digital and physical spaces by moving the display around. however, because of the domain in which these systems were designed, neither addressed the issues surrounding stitching and presenting an arbitrary space or image formed by the display sweep. in a parallel line of work, baudisch (1998) proposed paintable interfaces, which treat the cursor much like a paintbrush and allow users to manipulate large numbers of objects on screen simple by brushing over them. panoramic viewfinder uses the same painting or sweeping mechanism explored in all this work. rather than using mechanical sensing means, as the other techniques did, we use vision-based techniques to formulate a contiguous view of the world that the camera has seen (the eventual panoramic image). there are two views that must be shown to the panoramic viewfinder user: what the camera currently sees, as would be traditionally seen in the viewfinder; and the picture the camera has of the world it has seen, or the panorama at that point in time. displaying these two views on the same small screen could be accomplished in a variety of ways (e.g. overview plus detail, modally switching between the two, etc). we chose to use the focus-in-context approach, which displays the focal data, in this case the viewfinder image, in-place within the overview data, the panorama. this 159 australasian journal of information systems volume 13 number 2 may 2006 class of techniques has been shown to be cognitively less demanding than approaches that require users to switch attention between multiple views (baudisch et al., 2002). implementation our panoramic viewfinder prototype is written in c++ and runs under windows xp. we have two implementations of the front end, one using the gdi+ graphics library and one based on directx (http://microsoft.com/windows/directx). while we focus this paper on describing the interface innovation in this system, the technical details of our stitching engine can be found in (steedly et al., 2005). the algorithm is loosely based on the traditional offline stitcher described in (brown et al., 2005) and shares some libraries with it. similar to the offline stitcher, the real-time stitcher in panoramic viewfinder performs stitching in four steps: 1) extracting multi-scale oriented patches (mops) from each new image, 2) matching them with previous images, 3) estimating the orientation of the camera when the new image was taken, and 4) warping the new image onto the composite. mops are 8×8 pixel patches in the image, extracted at harris corner locations (figure 6), which occur at the centre of patches in the image that are easily localizable in both image dimensions. this is in contrast to lines, which can only be localized in one dimension, and uniform colour patches, which are hard to localize in any dimension. the corner locations are extracted with sub-pixel accuracy at multiple resolutions or scales. anywhere from fifty to several hundred features are extracted from a typical image. after extracting mops in a new image, the features are compared to features from previously stitched images. the comparison is made efficient by using a three-dimensional hash table (10×10×10) to store the features. in order to ensure that features are well distributed in the hash table, we perform a harr wavelet transform (jain, 1989) on the 8×8 patches and the harr coefficients are converted into a vector of length 64. the three hash table indices are the first three harr transform coefficients of the image patch. the features from each new image are matched to the features from previous images using the hash table. only two feature matches are needed to estimate the position from which the new image was taken. in practice, many more than two matches are typically found, over-constraining the position. hence, we use a robust least-squares optimization to estimate the new camera position. once we estimate the position of the new image, we display it on top of the composite of the previously captured images. if the user (or automatic snapshot algorithm) elects to keep the image, the mop features are permanently added to the hash table and the new image is permanently added to the composite. if not, the features and image are discarded. the entire process is then repeated for the next incoming frame. panoramic viewfinder is different from the offline stitcher in that it allows adding an image without reprocessing the images already stitched. to make this possible, panoramic viewfinder interleaves the four steps listed above. it finds the best location for each new image, but keeps all existing relationships fixed. while this leads to more error accumulation than the offline approach, which reminimizes errors between all pairs of images, it allows new frames to be added in roughly constant rather than quadratic time. australasian journal of information systems volume 13 number 2 may 2006 160 using the optimizations described above, our prototype reaches frame rates of up to 4 frames/sec on a 2ghz tablet pc with 1gb of ram. we obtain slightly slower performance on an ultra portable 1 ghz sony u50 (figure 3). figure 6: feature matching between two images: the white boxes show the locations of mops in each image and the black lines show the correspondences that the feature matcher detects. the corner patches and correspondences were not extracted by our code and are only for illustration. computing the cropping frame in real-time the user interface concept underlying panoramic viewfinder relies heavily on its ability to update the cropping frame in real-time. here we describe our algorithm that makes this possible. the real-time stitcher outputs the panorama as an alpha-matted composite bitmap image. in order to place the cropping frame, the system computes the largest interior rectangle that contains only fully opaque pixels. the problem of computing the largest inscribed rectangle in an arbitrary shaped polygon has been studied in the computational geometry community. however, existing algorithms (e.g. alt et al., 1995) require the geometry to be described as a polygon, which does not cover cases involving interior transparent regions. also, converting the rounded outlines of a panorama to a polygon can be computationally expensive. we therefore created an algorithm based on a pixelbased representation of the panorama. at the highest level, our algorithm tests all possible combinations of top-left and bottom-right corners thereby testing all possible rectangles that can be inscribed within a given region. the algorithm obtains real-time performance by eliminating top-left corners of boxes with expected areas lower than a moving upper bound and by limiting the search for the bottom right corners to a linear number of possible candidates. by doing this, we obtain a complexity of roughly o(n 2 ), with n being the width of the picture, which is suitable for real-time use. the detailed steps in this process are: step 1—down-sampling: for faster performance, our algorithm starts by creating a version of the panorama bitmap that is down-sampled by a factor of four. all further computation takes place on this down-sampled version. step 2—pre-compute span lengths: when computing rectangle surfaces, our algorithm uses information about the length of spans. since the same span information is used repeatedly, it is cached as follows. the algorithm creates two arrays called width image and height image. for each pixel, width image stores the length of the horizontal span right of this pixel, height image stores the length of vertical span below this pixel. both are created simultaneously by traversing the panorama from the last row to the first row and, for each of these rows, from the last column to the first one. since each span value is generated based on its right or bottom neighbor, this step requires o(n 2 ) operations. 161 australasian journal of information systems volume 13 number 2 may 2006 step 3—compute rectangle surfaces: now the algorithm searches for the largest interior rectangle. it initializes the largest found rectangle to area 0. then, it traverses the image top-down and left to right. at each pixel location (a) the algorithm computes an upper bound for the size of the largest possible rectangle whose top-left corner is at that pixel location. the upper bound is computed as the product of the horizontal span times the vertical span—both are looked up in the width and height images. (b) if the upper bound is smaller than the current best, this pixel cannot be the top-left corner of the sought rectangle and the algorithm moves on to the next pixel. (c) if the upper bound is larger than the current best, the algorithm computes the true largest rectangle with this particular top-left corner. in order to do so, it traverses down the current column of the width image, looking for the bottom left corner that results in the largest rectangle surface. during this traversal, it keeps track of the largest rectangle created by any bottom left corner so far. the algorithm also keeps track of the length of the shortest horizontal span seen so far in this column. the width of a rectangle with a given bottom left corner is the shortest span seen above. for each pixel in that column, the algorithm computes the surface of the rectangle that has the respective pixel as its bottom right corner as the product or that width times its distance from the top left corner. it keeps updating the best value until all pixels are traversed. in the worst case, step 3 of our algorithm takes o(n 3 ) operations. however, due its ability to terminate early, we find that in practice it is closer to o(n 2 ) and is suitable for our real-time requirements. interestingly, performance analysis shows that the algorithm spends 80% of its time in step 2. as a future optimization we therefore plan to improve performance further by updating the width and height of images incrementally as new input images are added. usage experience, limitations, and ideas for a revised design we designed the interaction described above to minimize stitching flaws by providing feedback as early as possible. we had initially assumed that instant notification would allow users to fix problems immediately, thus minimizing the overall impact of errors. after building and using panoramic viewfinder (baudisch et al., 2005), we have realized that the conclusions we had initially drawn are not as straightforward as we had thought. we had expected a seamless shooting experience when shooting in auto mode, but while using our system to take panoramic pictures, we experienced more “lost track warnings” than we had initially expected. most of these errors were caused by sweeping the camera too fast. because the underlying stitcher requires a certain minimum overlap between consecutive frames, the overall speed at which users can sweep the camera without losing track is determined by the frame rate, which is only about 4 frames/second in our current version. the problem is not only that this “speed limit” limits how fast users can shoot panoramas, but also that staying below that speed limit requires additional attention. in fact, receiving this warning requires users to focus on the display, which often breaks the “flow” of the interaction. while future code optimizations and increases in processing power might improve this situation by raising the speed limit, we contend that it is extremely difficult to completely eliminate all stitching failures. examples of problems likely to persist include lack of texture or motion blur in low light situations, which makes stitching hard. this brings up the question: does losing track need to break the flow? or, should we avoid breaking the flow by allowing users to temporarily be in a “broken” state? australasian journal of information systems volume 13 number 2 may 2006 162 similar problems and solutions have been discussed in other application domains, such as source code editing. many initial editors were batch-oriented and used a “compile, view error list, edit, repeat”-cycle, which could be considered similar to working with an offline stitcher. to remedy this somewhat disjoint process flow, some second generation source code editors, such as the microsoft visual basic editor began using a tight control loop. they continuously checked code syntax and prevented users from moving focus to a different line of source code until the current one was error free. however, enforcing this specific editing order on users made it hard for them to stay in the flow, which led to a series of usability issues. in retrospect, this evolution of solutions is analogous to that of the path we have already taken with panoramic viewfinder. to complete the analogy, more recent editors, such as microsoft intellisense or word have overcome some of these problems by using an “asynchronous” design. as the user is typing, squiggly underlines inform the user of issues that need to be addressed eventually. however, they do leave the control about order with the user and allow them to continue their train of thought. based on these observations, we propose a modified design to panoramic viewfinder, one that relaxes the synchrony between stitcher and user. there are many possible designs to embody such an asynchronous user interface concept. figure 7 shows a mock-up of one possible design. as an extension to the current instantiation of panoramic viewfinder, a separate viewfinder shows a live image coming from the camera (region 3). whenever the shutter is manually triggered, a shot is taken and added to the burst buffer, as indicated by a little gray arrow (region 4). in this instantiation of our design, users can either choose to start in automatic snapshot mode while shots are able to be stitched into the panorama and fall back to manual mode once tracking is lost, or they can choose to just do everything manually. the burst buffer is similar to the burst buffer offered by many of today’s digital cameras. it stores images without applying any processing, which allows shots to be taken at a very high rate. this process runs at the highest priority, which means that the other processes will be halted while the user is shooting to assure maximum shooting frame rate. a fast stitcher, as it is implemented in panoramic viewfinder, takes images from the burst buffer and tries to stitch them to the panorama. this thread runs at medium priority. if fast stitching succeeds, the respective frame is added to the main panorama and the panorama preview (region 1) is updated accordingly. note that the green frame inside the preview denotes the last successfully stitched frame, but unlike the previous version, this is now distinct from the view through the viewfinder. if fast stitching fails, the frame is moved to the recycle buffer (region 3). as a third and optional thread running at lowest priority, a “quality stitcher” similar to the offline stitcher re-stitches all frames in preview, burst-, and recycle buffer. this stitcher is slower, but since it compares all sets of frames to each other, it may find additional matches. when it succeeds, the respective frames are added to the preview and removed from the respective buffers. the preview always shows the largest sub panorama. if running the second stitcher creates a sub-panorama bigger than the one currently displayed in the preview, the new sub-panorama will instead be displayed in the preview. a typical case where this will occur is when the very first shot of a panorama was blurred or otherwise unsuccessful. when the preview is switched, the previously shown subpanorama goes back into the recycle buffer; its stitching information may be cached to speed up computation in case both sub-panoramas can be connected later. the asynchronous panoramic viewfinder offers audio feedback different from the original version. as before, taking pictures results in a shutter speed sound. instead of signaling stitching failure using a continuous error sound, however, the system now updates users on a per-event basis. successful stitches (transition from burst buffer to preview) result in a friendly ‘ping’ sound, unsuccessful stitches (transition from burst buffer to recycle buffer) in a ‘pop’ sound, and successful recoveries 163 australasian journal of information systems volume 13 number 2 may 2006 (transition from recycle buffer to preview) in a “charm” sound that is repeated if multiple frames were recovered. at the expense of offering a slightly more complex user experience, the asynchronous design allows users to shoot first and at any rate while fixing stitching failures on demand. the burst buffer within this design allows shooting at all times and at the highest possible rate. a typical usage would be for users to start a new panorama by shooting many shots very quickly. stitching failure, in particular during this early stage, does not affect the user’s ability to shoot, so stitching failures do not break the user’s flow anymore. panorama: largest sub panorama created so far panorama: largest sub panorama created so far 1 2 1 last frame added to the panorama last frame added to the panorama 2 viewfinder: what the camera sees right now viewfinder: what the camera sees right now burst buffer: # pictures taken, but not processed burst buffer: # pictures taken, but not processed 3 4 recycle buffer: # of frames not yet matched recycle buffer: # of frames not yet matched 5 figure 7: mock-up of an asynchronous panoramic viewfinder user interface conclusions in this paper, we have presented panoramic viewfinder, an interactive system for panorama construction that offers a real-time preview of the panorama while shooting. panoramic viewfinder makes ghosting, missing content, and stitching failures visible, thereby allowing users to fix them by immediately retaking necessary pictures. additionally, showing users the result of their work right away offers “instant gratification”, which we believe is one reason why many users today prefer digital photography to traditional photography. by automating the remaining steps in the process, i.e., high-quality restitching and auto cropping, panoramic viewfinder brings panoramic pictures one step closer to the user experience that users expect from regular digital photography, but which is not yet available with panoramic photography. australasian journal of information systems volume 13 number 2 may 2006 164 we have also discussed our usage experiences with panoramic viewfinder as well as usability issues and possible solutions to them. we propose a revised design that relaxes the forced synchrony between camera and user, and that we believe can further improve the user experience. as future work, we are planning to optimize our algorithms to increase the frame rate and to reduce accumulated error. we also plan on running a user study evaluating user’s performance and experience with panoramic viewfinder. acknowledgements we would like to thank the members of the interactive visual media group and the visualization and interaction group at microsoft research for their assistance and feedback on the work. references alt, h., hsu, d., and snoeyink, j. (1995). computing the largest inscribed isothetic rectangle. proceedings of 1995 canadian conference on computational geometry (cccg), 67–72. baudisch, p. (1998). don't click, paint! using toggle maps to manipulate sets of toggle switches. proceedings of 1998 acm symposium on user interface and software technology (uist), 65-66. baudisch, p., good, n., bellotti, v., and schraedley, p. (2002). keeping things in context: a comparative evaluation of focus plus context screens, overviews, and zooming. proceedings of chi 2002 conference on human factors in computing systems, 259-266. baudisch, p., tan, d., steedly, d., rudolph, e., uyttendaele, m., pal, c., and szeliski, r. (2005). panoramic viewfinder: providing a real-time preview to help users avoid flaws in panoramic pictures. in proceedings of ozchi 2005, acm international conference proceedings series, canberra, australia, november 2005 bergen, j. r., anandan, p., hanna, k. j., and hingorani, r. (1992). hierarchical model-based motion estimation. second european conference on computer vision (eccv), 237–252. brown, m. and lowe, d.g. (2003). recognising panoramas. proceedings of the 9th international conference on computer vision (iccv), 1218-1225. brown, m., szeliski, r., and winder, s. (2005). multi-image matching using multi-scale oriented patches. in ieee computer society conference on computer vision and pattern recognition (cvpr), volume i, 510-517. edwards, b. (1999). the new drawing on the right side of the brain. jeremy tarcher publishing, expanded edition. fitzmaurice, g. (1993). situated information spaces and spatially aware palmtop computers. communications of the acm, 36(7), 38–49. irani, m. and anandan, p. (1998). video indexing based on mosaic representations. ieee transactions on pattern analysis and machine intelligence, 86(5), 905–921. jain, a.k. (1989). fundamentals of digital image processing. englewood cliffs, nj: prentice hall. mann, s. and picard, r. w. (1994). virtual bellows: constructing high-quality images from video. proceedings of first ieee international conference on image processing (icip-94), 363–367. peleg, s., and herman, j. panoramic mosaics by manifold projection. (1997). proceedings of the 1997 conference on computer vision and pattern recognition (cvpr), 338. rekimoto, j. tilting operations for small screen interfaces. proceedings of 1996 acm symposium on user interface and software technology (uist), 167–168. 165 australasian journal of information systems volume 13 number 2 may 2006 sawhney, h.s., kumar, r., gendel, g., bergen, j., dixon, d., paragano, v. (1998). videobrush™: experiences with consumer video mosaicing. proceedings of ieee workshop on applications of computer vision (wacv), 56–62. sawhney, h.s., hsu, s., and kumar, r. robust video mosaicing through topology inference and local to global alignment. (1998). in proc. european conf. on computer vision, 103-119. steedly, d., pal, c., and szeliski, r. efficiently registering video into panoramic mosaics. proc. tenth international conference on computer vision (iccv 2005), volume ii, 1300-1307. szeliski, r. (1996) video mosaics for virtual environments. ieee computer graphics and applications, 16(2), 22–30. szeliski, r., and shum, h. (1997). creating full view panoramic image mosaics and environment maps. proceedings of siggraph 1997 31 st international conference on computer graphics and interactive techniques, 251–258. teodosio, l. and bender, w. (1993). salient video stills: content and context preserved. proceedings of acm multimedia 1993, 39–46. yee, k.p. (2003). peephole displays: pen interaction on spatially aware handheld computers. proceedings of chi 2003 conference on human factors in computing systems, 1-8. australasian journal of information systems volume 13 number 2 may 2006 166 microsoft word ajis10-2finala.doc ajis vol. 10 no. 2 may 2003 81 the australian government’s abandoned infrastructure outsourcing program: what can be learned? anne c. rouse department of information systems university of melbourne email: acrouse@forensys.com.au brian j. corbitt school of information systems deakin university abstract early in 2001, after a damning public report by the auditor-general, the australian federal government abandoned its highly promoted “whole of government” it infrastructure outsourcing initiative. this about-face was greeted in the press with reports that the initiative was a “fiasco”. yet a four-year case study conducted by the authors suggests a more complex picture. like many other “selective” outsourcers of it, the federal government had been led to believe that it was adopting a relatively low risk strategy that would, if well managed, lead to significant cost savings and operational benefits. instead, despite having implemented many widely promoted “best practices”, the federal government found a substantial discrepancy between what outsourcing promised to deliver, and what was actually achieved. in this respect their experiences were no different from those of many other large it organizations engaged in selective it outsourcing, who responded to a substantial contemporaneous survey. this case study examines why the government’s expectations were not achieved, and arrives at conclusions that have important implications for decision makers confronted with choices about sourcing it service delivery. keywords outsourcing of is el07, is strategic planning ef04, issues in organising is, eg03 is risk management el08, selective outsourcing, public sector, government outsourcing, case study, survey introduction on 24 july 2001 an article appeared in the influential it pages of the australian claiming that as a result of the “federal government it outsourcing fiasco” many public and private sector organisations were rethinking their attitudes to it outsourcing. the article stated that “concerns about the handling of federal outsourcing had turned private industry off the whole concept” (mitchell, 2001a, p 31). this article highlights the controversy surrounding the federal government’s outsourcing of its it infrastructure. six weeks later, the issue again hit the headlines in the australian with a report that the 2000 senate finance and public administration references committee had found the initiative was seriously flawed (mitchell, 2001b p 41). between 1998 and 2001 the initiative was rarely out of the headlines. this paper reports a four year case study of the federal government whole-of-government it infrastructure outsourcing initiative. the paper provides a brief outline of the case; reports the data sources used; considers several key observations that can be made about the case, and presents several implications these have for decision makers. case synopsis in april 1997, the australian federal government announced that by 1999 it intended to outsource the delivery of it infrastructure (mainframe, server, and desktop services, and in some cases telecommunications) for most federal agencies. at the time it was suggested that this initiative would lead to saving of up to $aud1billion; boost local industry development; and provide access to new technologies. savings were to occur through the use of external vendors, and the consolidation of requirements across government agencies, which would, in some cases, be clustered to form cross-agency contracts. the decision created some controversy, and was examined by an ongoing senate inquiry (sfparc, 1997). expecting some internal resistance to the initiative, and anxious that projected cost reductions not be dissipated, in the 1997 budget government deducted from line budgets the 15% savings expected to accrue from outsourcing it. in 1999, the federal auditor-general independently decided to review the outcomes of the first four contracts under this initiative. his september 2000 report (anao, 2000) found that the government’s ambitious timetable had not been met, and that the costs of market testing and contracting had blown out substantially (by over $aud26 million). most of these cost blow-outs were attributed to fees paid to specialist international consultants, including ‘big 5” accounting firms, outsourcing advisors, and the outsourcing legal specialists, shaw pittman. these advisors were engaged to facilitate the implementation of the initiative. the auditorgeneral also found that, contrary to government claims, only one of the first four contracts, the contract for “cluster 3”, was likely to deliver any substantial savings, and that actual savings after twelve months were less ajis vol. 10 no. 2 may 2003 82 than half of those projected at the time the contract was signed. at the same time the auditor-general found that outsourcing had a number of negative consequences on the operational business of agencies. the auditor-general’s report re-ignited community opposition to the initiative, and led the government, late in 2000, to commission an independent inquiry to examine the risks associated with the initiative (the humphry inquiry). in january 2001 the federal government released the humphry report (humphry, 2000), which recommended that the centralised and consolidated approach to it outsourcing be discontinued. however, agency line managers would still be required to evaluate the outsourcing of their it infrastructure services and justify their decision if they chose not to outsource. surprisingly, the humphry report did not address the issue of cost savings, but did find that the implementation risk (the risk to the agencies’ operations) associated with the consolidated approach were higher than had been expected. the auditor-general reviewed the first four contracts for the initiative, listed below: table 1: agencies analysed by the auditor-general’s whole of government it infrastructure outsourcing initiative review (anao, 2000) agency value won by date ‘cluster 3’ – dima*, aec* and four small agencies $160m csc mar-98 department of employment, training and youth affairs and employment national (deetya-en) $300m cancelled jun-98 australian tax office (ato) $490m ibm/gsa jun-99 ‘group 5’ – five small agencies $90m advantra jul-99 *dima: department of immigration and multicultural affairs, aec: australian electoral commission contrary to suggestions made elsewhere (e.g. seddon, 2001; willcocks’ submission to sfparc, 2001), although relatively large in scope, the whole of government outsourcing initiative was not an example of “total outsourcing”. instead, the initiative was an example of “selective outsourcing8”, involving only infrastructure it services and accounting for around half of it expenditure (sfparc, 1997; 2001). methodology this case study was analysed using hermeneutic textual analysis. hermeneutic analysis is a process used to identify contextual explanations for apparent contradictions found in qualitative text-based data. the textual material is repeatedly sifted and compared to find cross-verification and contradictions. understanding emerges from the comparison between new and previous information. the strategy is akin to the work of a historian sifting through competing accounts of past events and motivations. text sources used by the authors included: • notes from interviews conducted between 1997 and 2001 with 16 informants working for 9 federal agencies, or for their it service vendors. in most cases, informants were interviewed more than once. • a literature review based on press reports related to australian it outsourcing between 1997 and december 2001; • detailed analysis of a number of federal and state government reports, particularly the 1997, 1998 and 2000 senate finance and public administration references committee inquiries into outsourcing initiatives (sfparc 1997, 2001), the auditor-general’s report (anao, 2000); and the humphry report (cited as humphry, 2001); • walker and walker’s (2000) broad based analysis of government outsourcing in australia; • a study of the case by seddon (2001) that is drawn from the auditor-general’s and humphry reports; • analyses prepared for members of parliament by the parliamentary library politics and public administration group (verspaandonk 2001a, 2001b); • notes from a series of focus groups involving 56 respondents reporting on their experiences of it outsourcing; and • analysis (reported in rouse, 2002). of a university of melbourne it outsourcing survey of 1000 large sites in australia, distributed in 2000. this survey, which had a 24% response rate (n = 240) was 8 lacity & willcocks (1998) have defined “selective outsourcing” as outsourcing of between 20% and 80% of the it budget, in contrast to “total outsourcing” in which over 80% of the it budget is outsourced. they argue that total outsourcing is particularly problematic, and recommend strongly against it because of the risks. ajis vol. 10 no. 2 may 2003 83 conducted by a team of five at the university that included rouse, seddon, and willcocks (an initial report is found in seddon et al, 2000)9. key findings the authors’ analysis of the case revealed several key findings (discussed below) that provide a lens through which the case can be interpreted. the 15% savings target although the initiative encountered problems in a number of areas, it is unlikely that it would have been abandoned if it had delivered the 15% savings expected on the basis of expert advice received from vendors, industry specialists, consultants and academics. at the time vendor bids were assessed, detailed costings were carried out by oasito (the office of asset sales and it outsourcing, the agency tasked with managing the initiative) under the guidance of specialist consultants, and in consultation with agency management and it staff. these bids in turn were sought only after extensive research had been undertaken of the services likely to be required, and the costs of internal service delivery. projections at the time vendors were selected suggested savings of up to 28% were likely, and a target of 15% would have appeared achievable. however, the auditor-general’s analysis (anao, 2000) revealed that despite expert advice, the cost and savings projections for two of the arrangements (ato and group 5) were flawed. the business model projections had failed to take into account additional cash streams associated with equipment that would be available at the end of the contract if the services were not outsourced. they had also under-estimated the costs of leasing risks absorbed by the agencies. according to the auditor-general the consequence of these flawed projections was that only one agency (cluster 3) was likely to make any substantial savings from the outsourcing arrangements. reviewing the business model developed when the vendor was selected, the auditor-general determined that projected savings for that group should have been 28%, of which 5% were associated with one-off retrenchment costs. however, the actual savings in the first 12 months of the contract were only 12%.  3% short of the government’s apparently conservative 15% target, and less than half the ongoing savings the business model projected. because cluster 3 agencies were required to make up the 3% shortfall out of operating budget, this over-optimism had profound negative effects on their operational performance, and the 12% savings were seen in a negative light by all involved. however, examination of the empirical literature on outsourcing savings, as well as discussions with 56 it practitioners in the focus groups (rouse, 2002) suggests that savings of 12% are at the high end of what can realistically be expected from outsourcing it. the contemporaneous survey of it outsourcers conducted by the university of melbourne in late 1999/early 2000 revealed a quite different picture to that expected by government decision-makers, and instead confirmed the information coming from the focus groups. in that study only a minority of selective outsourcing cases (42.4%) reported any savings at all, and only 8.7% reported “substantial” savings. these ratios were statistically no different to those for the sample as a whole  42.1% and 7.4% respectively (rouse, 2002). the finding that most respondents failed to report any savings from outsourcing applied whether respondents were government or non-government organisations, medium, large or very large organisations, and whatever proportions of their it budget was outsourced (rouse, 2002). in light of this information, the experiences of the federal initiative can be seen as typical, rather than aberrant. other, smaller contemporaneous surveys  such as the two reported by lacity & willcocks, (2000, 2001) with a total of 77 outsourcing respondents; and that by aubert et al (1999) with 70 respondents  also found that it outsourcing did not result in substantial savings. cost increases, on the other hand, occurred in 22% of the respondents to the university of melbourne study, and in up to 49% of the respondents to aubert et al’s (1999). decision makers went to some trouble to follow recommended practices it has been suggested in the press and academic literature that the federal government initiative represents a failure by decision-makers to follow recommended practices. seddon (2001), for example, juxtaposes an account of the initiative with what he labels “ten commandments” for good practice, implying by doing so that these prescriptions were not followed. most of these prescriptions are based on a range of practices described as “proven” to lead to outsourcing success by lacity and willcocks in their most recent text (2001, 9 referred to in this paper as the university of melbourne survey. ajis vol. 10 no. 2 may 2003 84 pp 6-16). these practices were articulated in the mid 90s, and arose largely from lacity’s and hirschheim’s (1993; 1995) series of case studies. these authors had observed that many organisations that experienced substantial problems with outsourcing engaged in what could be described as naive behaviour, such as failing to carry out thorough investigations of their own and market costs; and signing long contracts (10+ years) with a single vendor for substantial proportions of their it budget, so increasing dependence on the vendor. as a result of these case studies lacity and hirschheim developed a set of prescriptions for good sourcing practices. these were further codified and augmented by lacity and willcocks on the basis of a post hoc examination of 61 case studies one or other author had conducted in europe and the us (described in lacity & willcocks, 1998; 2001). our investigation of the federal initiative suggests that “not following recommended practices” is an inadequate explanation for the failure. the fact that the government spent over $25 million on obtaining and implementing specialist consultant advice related to the design and implementation of the program (anao, 2000, p 17) indicates that decision-makers were keen to make the arrangement work well. government had also received advice from the it review group (itrg); the it&t policy advisory committee, panels of industry specialists, and from the australian industries commission. all of this advice suggested that if the process was carefully managed, the government could succeed in achieving the targeted 15% savings (and possibly much more) without compromising operational effectiveness. this message is, incidentally, prevalent in current mis textbooks, despite the fact that it has little empirical support. the evidence gathered by the authors is that the federal government spent considerable effort in planning for, and implementing the initiative, even though, in hindsight, the costs and time involved were greater than had been expected. in this respect they were typical of other australian organisations (seddon et al, 2000). the case study data gathered by the authors suggests that decision-makers followed almost all of the “proven practices” from the literature. of seddon’s “ten commandments” for it outsourcing success, there is evidence that only one was not followed. this practice relates to allowing the internal it services groups to bid against external vendors, which case informants suggested was an expensive and impractical proposition. to manage risks, government decision-makers adopted many practices recommended by lacity, hirschheim and willcocks, and summarised in lacity and willcocks (2001). these included: • selective outsourcing (at around 50% of it budget); • targeting services thought likely to be generic and “non core”, i.e. infrastructure services which lacity & willcocks (2001 p 9) and seddon et al (2000 p 4) have suggested are frequently outsourced successfully; • use of short contracts (3 to 5 years); • crafting of detailed and comprehensive contracts that were prepared with the advice of specialists; • retaining highly-skilled specialist staff to manage the arrangement; • involving both line and it management in the decision, even though, in the end, senior management (i.e. cabinet) made the decision; • conducting detailed and extensive analysis of vendor offerings and both internal and market prices; and • designing and implementing post-contract management structures and processes. regular formal and informal monitoring and sla negotiations were instituted as part of this process. looking at the management attention and resources devoted to the initiative, the case appears to be a textbook example of decision-makers responding to the best available advice. this observation raises a critical question: “how likely is it that organisations that do implement good management practices will “succeed” in it outsourcing?”. research, which influenced the decision, did not necessarily support the decision examination of the industry assistance commission’s report into contracting out, the senate inquiries and the citations used in discussion papers (e.g. verspaandonk, 2000b) indicates that government’s expectation, that 15% savings were realisable from it outsourcing, were influenced by several key studies. particularly important were findings by domberger, hodge, and lacity & willcocks. this research was cited by informants, and in public documents, as evidence that carefully-managed outsourcing frequently leads to substantial cost savings of 20% to 30% or more. however, closer analysis of this work reveals that a more complex relationship exists. for example, research by domberger (ctc, 1999) suggested savings of 30%+ from outsourcing were common, and this research played an important role in setting government expectations (rouse, 2002, walker & walker, 2000). walker and walker (2000) identify several flaws in some of this research, and careful examination of the most recent data (ctc, 1999) reveals that while such savings were common for simple outsourced services such as cleaning and catering, increases of around 8% were observed when it services were outsourced. ajis vol. 10 no. 2 may 2003 85 hodge (1996) carried out a meta-analysis of a large number of quantitative studies of general outsourcing by government agencies. his studies translated results from earlier studies to a common metric, so that data could be consolidated and re-analysed. he found savings of between 8% and 14% were possible but these were largely for simple services, like garbage collection, or cleaning (hodge, 1996). however, savings for corporate services (which would include it) increased on average by 5% (hodge, 1999). in relation to it services, lacity and willcocks (1998) reported cross case comparisons that suggested most outsourcing (70%+ of the 33 outsourcing cases with discernible outcomes) led to savings. they found that “selective” outsourcing of it services was particularly successful. for their selective cases reporting cost savings  a large majority (85%) of the selective outsourcing cases that reported an outcome  savings were, on average, a substantial 23.8% (data from lacity & willcocks, 1998, re-analysed in rouse, 2002). but the authors acknowledge in that paper that these cases are not statistically representative. the fact that these cases are not statistically representative means they shed no light on the success rates that might be expected in the wider population. furthermore, researchers often have difficulty gaining access to case study sites involving problems or failure. so the positive odds of success represented by the cases studied by lacity and willcocks (1998; 2001), and the substantial cost savings reported for selective outsourcing (1998) are not those that typical outsourcing purchasers can expect. in spite of these limitations, the success ratios reported by these authors were cited by a number of informants involved in the initiative as justification for their strong belief that their selective it outsourcing would succeed. contemporary evidence indicates that selective, well-managed outsourcing still involves considerable risks in following the various recommended practices discussed above, the australian federal government was typical of the large organisations studied in the university of melbourne survey. most of those respondents also followed recommended practices like selective outsourcing; outsourcing infrastructure services; employing short contracts; benchmarking it services; and detailing services and service levels in their contracts. despite this, most survey respondents were disappointed with their it outsourcing experiences, with only just over 1/3 (36%) reporting a positive score on a measure of satisfaction and value. less than a third of respondents reported positive scores for measures of strategic benefits, technology benefits, and economies of scale. when it came to cost savings, only 7.4% of respondents reported substantial savings, and only 42.1% reported any savings at all. one explanation for the problems experienced by the first four agencies in the federal initiative is that it outsourcing  even selective outsourcing involving short contracts and infrastructure services  is much riskier than has been recognised. the fact that previous research did not quantify levels of risks appears to have led to unrealistic expectations and over-optimism. much of the research on risks has been based on case studies (e.g. lacity and hirschheim, 1993) or subjective-argumentative research (e.g. earl, 1996). it may be that decision-makers discounted the risks described by researchers in the belief that they were rare, or only encountered by poorly managed organisations. that risks are substantial is evident in the responses to the university of melbourne study. analysis revealed that risks ranged from high to very high for both public and private sector organisations in that survey (rouse, 2002). negative outcomes like failing to get expected cost savings; inability to concentrate on core business, failure to avoid technology obsolescence, and inability to control costs were reported by the majority of organisations, while substantial problems (like unexpected cost increases, loss of organisational knowledge, and difficulties projecting future requirements) were encountered by many. the proposition that these risks were not expected is consistent with the low level of satisfaction expressed by respondents, as low satisfaction is usually the consequence when positive expectations are not confirmed by actual experiences (c.f. rouse, 2002). in light of this contextual information, the negative experiences reported in the reviews conducted by the auditor-general and by humphry no longer appear to be examples of flawed management failing to achieve what other, better managed organisations were achieving. instead they are relatively typical of the experiences of australia’s top private and public organisations discussion and implications attributions of the causes of outsourcing failure need to be examined carefully the humphry report recommended the abandonment of compulsory centralised and clustered it outsourcing for future arrangements, because of the potential negative impacts this approach had on operational goals. but as a proponent of the benefits of outsourcing, humphry did not suggest that it outsourcing itself was inherently problematic, or that achieving significant cost savings was unlikely. the fact that the auditorajis vol. 10 no. 2 may 2003 86 general had found that two of the first three arrangements would never achieve substantial savings was not acknowledged. instead, humphry cited the substantial savings observed for cluster 3 by the auditor-general as evidence that outsourcing can be an economic success. paradoxically, humphry also recommended abandonment of the clustering strategy that contributed to these savings. the tenor of humphry’s report suggests he accepted the (largely vendor-driven) proposition that wellmanaged outsourcing usually succeeds. he argues (p 24) that “the debate over cost savings has tended to obscure other benefits, which can arise from properly implemented outsourcing [emphasis added] such as wider access to technology and technical skills, strategic partnership in a dynamic technical environment and an opportunity to manage capital expenditures more effectively”. a belief in this proposition (which does not have empirical support) would have influenced him to seek evidence of poor management as a cause of the failure to meet expectations, as would the political pragmatics of such an inquiry. humphry did indeed suggest that the initiative had been compromised by poor management on the part of the central management agency (oasito). humphry reported that decision-makers (within oasito) had concentrated on contractual rather than operational issues, and had failed to ensure co-operation from line management. in addition, he suggested that agency management had been resistant and obstructive: “it is widely accepted that agencies’ inertia and resistance to change contributed significantly to the delays (p 9)... there is a need for [an] agency’s executive and its senior managers to demonstrate support for the process and commitment to its success (p 14). these explanations are not refutable, and so are not helpful. if decision-makers in oasito had not concentrated on legal and contractual issues, they would have been rightly criticised. and how could decisionmakers know “how much” attention to implementation risk was sufficient? furthermore, any problematic strategy (where key expectations are not being met) is likely to result in resistance from some line managers, particularly if the strategy hampers their operational goals (the situation described by both the auditor-general and humphry). hence the evidence of poor management and line management opposition becomes selfreferential: because the initiative failed, decision makers must have failed to gain “sufficient” co-operation of stakeholders and must not have devoted “enough” attention to operational issues. and if line managers expressed concerns about the initiative, this “lack of buy in” (p 11) must have contributed to the failure to achieve the original goals. yet when these propositions are examined, causal links are not strongly supported. another factor implicated by humphry was the clustering that occurred in two of the three contracts studied by the attorney-general. the attorney-general too had suggested that clustering had exacerbated difficulties encountered by agencies. although humphry defended the original clustering strategy, he suggested it had outlived its usefulness and was by 2000 no longer necessary (p 12). consequently, a key element of his recommendations was the abandonment of requirements that agencies form grouped contracts (i.e. cluster). yet clustering had allowed small agencies to benefit from larger agencies’ contract negotiation and management capacities, and attraction to vendors. significantly, the only contract of those studied by the auditor-general that was likely to succeed financially was cluster 3, which chose to renew its clustered contract with csc in august 2002. when this case is examined in depth, there is little evidence that it was clustering, the implementation flaws described by humphry, or the concerns expressed by some line managers that resulted in the abandonment of the initiative. much of the evidence cited by humphry about risks and downsides of outsourcing had already been presented to the 1997 senate inquiry (sfparc, 1997) but at that stage the government had decided that the risks were manageable. what had changed by 2000 was that the auditor-general had revealed that the downsides of it outsourcing (including the managerial attention that had to be devoted to the arrangement, and the difficulties of implementing within a short timeframe) were substantial, whether or not agencies were involved in clusters. furthermore, the risks that agencies would experience lock-in with the incumbent vendor at end of the contract were much more visible, as a result of the failure to attract more than one bidder to the fourth contract (deetya-en). at the same time, the likelihood of commensurate financial and industry development benefits was much lower than had been expected. consequently the risk/return ratio of the initiative was by then demonstrably, and publicly, poor. when the same arguments about risks and problems were presented to humphry in 2000, together with evidence that impact of these risks were indeed widespread and damaging, he determined that these risks were by then so significant that the initiative should be reconfigured. given the damaging findings of the auditorgeneral’s report, it is unlikely that he had any other choices open to him. it is tempting to see difficulties reported to humphry as peculiar to the particular form of implementation (e.g. use of a prime-contractor model, clustering, or exercise of senior-management fiat), or the result of poor management. however, the difficulties can also be seen as examples of an important, and largely inherent, trade-off first identified by lacity and hirschheim (1995): certain strategies (like requiring the line to adopt common standards and to consolidate their it requirements) whether implemented by vendors or internally, can lead to moderate cost savings, but these are often at the expense of the line’s operational goals. ajis vol. 10 no. 2 may 2003 87 as an earlier supporter of outsourcing, it is unlikely that humphry would have attributed the failure of the arrangement to an inherent trade-off between cost savings goals and operational needs, or to generally high risks that outsourcing benefits will fail to materialise (as revealed by rouse’s 2002 analysis of the university of melbourne survey). humphry had been one of the industry specialists on the it&t policy advisory committee that encouraged the government’s adoption of it outsourcing and the clustering approach (humphry, 2001, p 22). a reading of his report suggests a strong commitment to the intrinsic value of it outsourcing, and its capacity to produce demonstrable benefits if well-managed. humphry’s commitment to the potential benefits of outsourcing is understandable given the advice and research presented to government at that time the it&t policy advisory committee made its recommendations (1997) the fact that the initiative failed to achieve goals that, in hindsight, appear very unlikely to have been achieved puts the case in a different light. so does the evidence (from the university of melbourne survey) that many other organisations were also failing to achieve similar goals. this suggests that humphry’s attributions, and his targeting of resistant line managers and a less-than-competent oasito (which was consequently disbanded) could represent a search for scapegoats for the failure of a policy that was far riskier than was, at the time, recognised. some organisations will never discover that their cost expectations are not being met hodge (1996) observed that the more rigorous the evaluation studies of outsourcing, the lower the reported savings. this is because more rigorous studies included costs ignored by less rigorous ones, and controlled for other critical factors. an issue canvassed in walker & walker (2000) is the quality of cost and savings projections used to justify government sourcing decisions. yet when researchers gather data, where do informants obtain their figures when asked what savings (or dis-savings) they have made from a sourcing decision? as the auditor-general’s review reveals, to answer the question of what savings were actually obtained is complex, expensive, and requires substantial expertise. rolf jester, a gartner spokesman, has suggested (in the australian, 23 jan 2001), that after as little as 12 months it is not even possible to establish with any certainty what the savings were, an observation also made by the auditor-general. the process of determining savings requires comparison of four imprecise projections, each involving measurement errors and technical and financial assumptions. these four projections include: • projections made at the time vendors tender about the service bundle that would be needed over the life of the contract; • contemporary projections of the likely costs of supplying these services using in-house resources: • contemporary projections of the chosen vendor’s likely costs to supply these services and the difference between this and those projected for in-house delivery; • retrospective projections of what the services actually supplied by the vendor (which may be different to those projected at time of contract) might have cost if supplied in-house. even the calculation of the actual costs and services provided under the outsourcing arrangements involve some imprecision. as the case study has illustrated, this calculation too can depend on assumptions and interpretation of both the contract and accounting conventions. the auditor-general’s analysis revealed that there was a substantial difference between the services projected at the time the contract was signed, and those actually needed by cluster 3 over the first 12 months, as well as differences between projected and actual service volumes. these were not the result of ignorance or failure to attend to the issues, but rather a result of unexpected business initiatives (anao, 2000, p 196). as lacity & hirschheim (1993) revealed, such differences can rapidly consume projected savings. there were also substantial differences between the costs expected for these services, and the actual costs, as a result of different assumptions about how staff would respond to voluntary retrenchment offers; how contractual terms would be interpreted, how quickly certain technology upgrades would occur; as well as some minor errors in the complex financial models used to evaluate tenders (anao, 2000, pp .196-197). this is an important lesson for decision-makers, as many organisations could be expected to encounter similar situations. cluster 3 was the only agency in a position to even attempt the various comparisons discussed above. the clustered group 5 argued that to carry out this analysis would be prohibitively time consuming and difficult, and might lead to distortions because of the high level of error involved. ato was too early into its contract to measure savings at the time the attorney-general reported. it is likely that similar circumstances will prevent most other it outsourcers from conducting post-implementation reviews of actual cost savings achieved. this means that reported “savings” will often be based upon a very optimistic expectation, and that conflicting information may never be formally gathered to challenge the initial expectation. this is a particular problem for it outsourcing researchers, because unless purchaser organisations have in fact carried out a comprehensive post-outsourcing review, many “savings” figures reported to researchers are likely to be those projected at the time the contract is signed, and so to significantly over-state actual savings ajis vol. 10 no. 2 may 2003 88 the auditor-general’s analysis of the group 5 and ato business cases revealed further, much greater error margins in projected costs associated with failure on the part of the decision makers to spot flaws in the assumptions that underlay projections made largely by specialist consultants. according to the auditorgeneral these flaws meant that projected savings of 10% (group 5) and 18% (ato) would result in probable cost increases, or, at best, savings of 5.4% (anao, 2000, p 167) though these savings would quickly be eaten up by management costs and by unexpected changes. analytic flaws like these are not likely to be revealed even when post implementation reviews are carried out, unless reviews are performed by staff with specialist financial skills, such as those held in the office of the auditor-general. without the auditor-general’s independent (and to some extent, unwelcome) public review, government decision makers might not have discovered that their expectations were not being met for some years, if at all. but this situation is likely to apply to many other organisations involved in it outsourcing. detailed analysis, like that conducted by the auditor-general, may well reveal that savings are eaten up by the same factors that were at play with the initiative: changes in business requirements and volumes, misunderstandings about what is meant by complex contractual terms, delays associated with evolving technologies, incorrect assumptions, and in complex business cases, possibly errors. it outsourcing is expected to lead to multiple outcomes, some of which are antagonistic an examination of trade and consulting literature in the last few years reveals a new theme: only naive decision-makers expect to make substantial savings from it outsourcing: instead, sophisticated managers are seeking strategic benefits, like access to new skills and technologies and concentration on core competencies. this theme can be discerned by examining sites like the vendor-dominated outsourcing institute, or those of consultants advising on outsourcing. a cynic might be tempted to see this as a response to the failure over a decade to demonstrate widespread substantial savings from it outsourcing. however, another possible interpretation is related to the findings of the federal initiative. this is that it outsourcing is usually initiated to achieve multiple goals, some of which may need to be traded off. the federal government was seeking cost savings; better, or at least no worse impact on operational business; and industry development. the evidence from the various documents consulted was that they could achieve some of these goals, but not all. the ato, for example, has reported substantial technology improvements as a result of its outsourcing arrangement, but reports of substantial savings are noticeable by their absence. according to the auditor-general, these were always unlikely. humphry reported a number of negative consequences of agencies’ attempts to meet industry development goals (which were later downgraded as a result of the humphry review), and noted that these goals conflicted with agency’s operational goals ( p 36). elsewhere lacity and willcocks (2001) reported that the south australian government achieved substantial industry development, but evidence for substantial cost savings from that arrangement is slim. rouse’s (2002) analysis of the university of melbourne survey revealed that while the majority of purchasers reported obtaining new skills and positive vendor service from their outsourcing arrangements, this was at the expense of economic benefits (including cost savings); technology benefits, and strategic benefits. the noted economist shlomo maital (1994) has observed that only inefficient organisations can achieve organisational improvements without trading off different goals. the federal government had been led by advice to expect that they would not encounter unacceptable trade offs when outsourcing it, but they were unpleasantly surprised. when cost savings were achieved, the operational impacts were quite negative, while arrangements (like, reportedly, the ato) that succeeded in meeting technology and business goals did so at the expense of substantial savings. the “solution” now in place as a result of the humphry review (that is, requiring individual agencies to market test their service requirements) may not end up solving the problems revealed by the auditor-general’s and humphry reports. reducing complex, rich observations to simplified “recommended practices” has some limitations the failure of an organisation that went to such trouble to manage its risks raises questions about the “operationalisation” of the recommended practices oasito followed. these practices were drawn largely from the case studies carried out by lacity and hirschheim in the early 90s, and the post-hoc cross-case analyses done when lacity and willcocks began reviewing their combined research (c.f. lacity et al, 1996; lacity & willcocks, 1998). examination of the original case descriptions (for example those in lacity & hirschheim, 1993) reveals complexity and richness, yet this richness was eventually distilled (lacity et al, 1996) into a number of relatively simple prescriptions that may not have captured the underlying causal factors. one example is “selective outsourcing”, which has frequently been described as leading to success (lacity et al 1996; lacity & willcocks 1998; 2001). statistical analysis of the university of melbourne survey had ajis vol. 10 no. 2 may 2003 89 failed to find any differences in success measures between “selective” and “total” outsourcers (rouse, 2002). in seeking an explanation for this, it became clear that the definition of “selective” outsourcing is malleable. while the label was originally used to describe outsourcing between 20% and 80% of it budget (lacity et al, 1996); more recently (2001) it has been used by the authors to describe outsourcing as low as 10% of it budget. informants in the focus groups conducted by rouse (2002) tended to define “selective” outsourcing in terms of what it was not, that is “total” outsourcing. total outsourcing has been presented in the academic literature as a risky and discredited strategy. yet, as both lacity & willcocks (2000) and the university of melbourne study identified that only around 7% of organisations engage in “total” outsourcing, there are very few decision-makers who would not perceive themselves as being involved in the recommended “selective” outsourcing. as another example, the recommended practice “involve line management and it management in the decision” came about because some of the problematic cases studied by lacity and hirschheim (1993) excluded one or other of these groups from the decision. in the federal initiative, the government did involve both line management and it in their decisions. there was extensive consultation and discussion amongst oasito and line and it staff in all the agencies involved. not all agreed with the decision, but stakeholders had many opportunities to express their reservations through this consultation, and through the senate inquiry that took place during 1997. when seeking to reconcile this observation it becomes apparent that the term “involvement” is very general. it does not, for example, necessarily imply that active support from all parties is required for success. there are many examples in the is and management literatures of senior management instituting ultimately successful organisational programs that encountered disagreements and dissent. furthermore, such simplicity is not apparent in lacity and hirschheim’s original discussion of this practice, which was concerned, with the problems of excluding one or other viewpoint. again, the impact of this imprecision is that it is likely to be only in those (probably rare) organisations that actively exclude one or other group that decision-makers would not perceive themselves as involving both it and line management. on the basis of this case study, it appears that many of the recommended practices are necessary, but not sufficient for achieving success, particularly as the probability of their not being followed at all is low. for example, it is unlikely that an organisation involved in a multi-million dollar venture would not establish post management structures and processes of some kind, yet the federal initiative has illustrated that such processes, even when based on “international best practice” advice do not guarantee success. a similar observation can be made about careful market testing and crafting of detailed contracts. the conclusion that such practices are not sufficient to guarantee success implies that decision makers may be unwittingly led to under-estimate their own risks when outsourcing it, because most would be able to reassure themselves that they were following recommended risk-minimising practices. it is significant that there has been virtually no evidence provided about how well such practices protect against failure. the bulk of research conducted to date is either case study based, or subjective-argumentative, and so is not able to provide probabilistic estimates of success and risks, or the moderating effects that recommended management practices have on these. quantitative studies have generally been small, so that the observed relationships could easily have occurred purely by chance. so although there is convincing qualitative evidence that failing to follow these practices can lead to problems, decision makers have no information about what risks they run even when such management practices are followed. conclusions this paper has presented a very different view of the federal it infrastructure case than has been presented in the press and some academic literature. in light of the contemporary survey data, it is our view that the case study is not an example of poor decisions and implementation. instead it illustrates a decision predicated on research and advice that was overoptimistic about likely benefits if it outsourcing, and unrealistically confident that good management could overcome the associated risks. many of the outcomes for this “fiasco” are in fact typical of those of contemporary australian organisations, and probably of many large organisations. a key implication of the case is that decision-makers need to frame their decisions in light of recent research about the relative risks and returns from it outsourcing (eg rouse, 2002; aubert et al, 1999). the case reveals considerable uncertainty surrounding likely outcomes, and highlights the possibility that key assumptions about costs, savings, managerial effort, or the effects of outsourcing on operational performance might be incorrect. framing it outsourcing as a quite risky venture (in contrast to the comfortable and reassuring message provided by many vendors and consultants) demands similar strategies to those undertaken when examining any potentially risky venture. such framing suggests careful examination of assumptions, recognition of the high levels of uncertainty involved in estimates, sensitivity analysis, “risk-boxing” of initial forays, and, as was done by the auditor-general, early, and ideally, independent, evaluation of the achievement of goals set for the venture. ajis vol. 10 no. 2 may 2003 90 references anao australian national audit office (2000). implementation of whole-of-government information technology and infrastructure consolidation and outsourcing initiative. audit report no 9. canberra. aubert, benoit a.; rivard, suzanne, and patry, michel (1999). chapter 3.3: impartition des services informatiques au canada: une comparaison 1993-1997. in michel poitevin (ed) impartition fondements et analyse. montreal: canada: presses de l’universite laval, 203-220. ctc consultants (1999) government outsourcing: what has been learnt? sydney: ctc consultants. earl, michael j. (1996) the risks of outsourcing. sloan management review; 37(3):26-32. hodge, graeme a. (1996) contracting out government services: a review of international evidence. montech. hodge, graeme a. (1999) competitive tendering and contracting out: rhetoric or reality? public productivity and management review. 22(4):455-469. humphry, richard (2000). review of the whole of government information technology outsourcing initiative ('humphry report'). canberra: commonwealth of australia lacity, mary c. & hirschheim, rudy (1993). information systems outsourcing: myths, metaphors and realities. chichester, england: wiley. lacity, mary c. & hirschheim, rudy (1995). beyond the information systems outsourcing bandwagon: the insourcing response. new york: wiley. lacity, mary c.; willcocks, leslie, & feeny david f. (1996). the value of selective it outsourcing. sloan management review. spring; 37(3):13-25. lacity, mary c. & willcocks, leslie (1998)an empirical investigation of information technology sourcing practices: lessons from experience. mis quarterly. 22 (3):363-408. lacity, mary c. & willcocks, leslie p. (2000) inside information technology outsourcing: a state-of-theart report. oxford, uk: oxiim. lacity, mary c. & willcocks leslie p. (2001) global information technology outsourcing: in search of business advantage. chichester, uk: wiley. maital, shlomo (1994). executive economics. new york: free press. mitchell, selina. (2001a) outsource suspicion hits firms. the australian it. 24 july : 31. mitchell, selina (2001b) outsourcing’s still a problem, report says. the australian it 4 september : 41. rouse, anne c. (2002). information technology outsourcing revisited: success factors and risks. unpublished doctoral thesis. melbourne: department of information systems, university of melbourne. seddon peter (2001). the australian federal government's clustered-agency it outsourcing experiment. communications of the association for information systems. 5(13):1-33. seddon, peter; cullen, sara; willcocks, leslie p.; rouse, anne c., & reilly, colin (2000). report on information technology outsourcing practices in australia, 2000. version 1, october. melbourne,: department of information systems, university of melbourne. available from department. sfparc senate finance and public administration references committee (1997) contracting out of government services: first report: information technology. canberra: commonwealth of australia. sfparc (2001) rebooting the it agenda in the australian public service: final report on the government's information technology outsourcing initiative. canberra, australia: commonwealth of australia; 2001. verspaandonk, rose. (2001a ). politics and public administration group: the whole-of-government it outsourcing initiative [web page].; http://www.aph.gov.au/library/intguide/pol/it_outsourcing.htm. accessed july 1 2001 verspaandonk, rose (2001b) outsourcing for and against: current issues brief 18 2000/2001 [web page] . available at: http://www.aph.gov.au/library/pubs/cib/2000-01/01cib18.htm. accessed july 1 2001. walker, bob & walker, betty con (2000). privatisation: sell off or sell out? sydney: abc books. acknowledgements the research for this paper was funded by simsion bowles & associates, and by a university of melbourne david hays “writing up” award. the authors would like to express their gratitude to professor liz sonenberg, department of information systems, university of melbourne, and to david watson, department of marketing, monash university, for their helpful comments on earlier drafts of the paper. ajis vol 9 no. 1 september 2001 scalable fault-tolerant location management scheme for mobile ip jinho ahn, sung-gi min and chongsun hwang dept. of computer science and engineering, korea university 5-1 anam-dong, sungbuk-gu, seoul 136-701, republic of korea abstract as the number of mobile nodes registering with a network rapidly increases in mobile ip, multiple mobility (home or foreign) agents can be allocated to a network in order to improve performance and availability. previous fault-tolerant schemes (denoted by prt schemes) to mask failures of the mobility agents use passive replication techniques. however, they result in high failure-free latency during registration process if the number of mobility agents in the same network increases, and force each mobility agent to manage bindings of all the mobile nodes registering with its network. in this paper, we present a new fault-tolerant scheme (denoted by cml scheme) using checkpointing and message logging techniques. the cml scheme achieves low failure-free latency even if the number of mobility agents in a network increases, and improves scalability to a large number of mobile nodes registering with each network compared with the prt schemes. additionally, the cml scheme allows each failed mobility agent to recover bindings of the mobile nodes registering with the mobility agent when it is repaired even if all the other mobility agents in the same network concurrently fail. introduction a mobile ip based system extends an ip based distributed system to support mobility of nodes by providing mobile nodes (mns) with continuous network connections while changing their locations [ghosh (1998), johnson (1994), perkins (1996b), solomon (1998)]. in other words, it transparently provides mobility for nodes while backward compatible with the current ip routing scheme by using two kinds of mobility agents (mas), home agent and foreign agent. in the system, each home agent must maintain information about the current careof address of each mn registering with the home agent and forward packets, destined to the mn, to the care-of address for the mn. each foreign agent should offer a care-of address for each visiting mn and forward packets to the mn. if each mn uses a collocated care-of address, it must have the same function of a foreign agent. as mobile computing is increasingly gaining popularity, mas should serve a large number of mns. thus, they may be single points of failure and potential performance bottlenecks. especially, if a home agent fails, all the mns served by it can't communicate with other nodes. the problems can be solved by allowing multiple mas to be assigned to the same network. previous fault-tolerant schemes (denoted by prt schemes) [ghosh (1998)] to mask failures of mas in mobile ip use passive replication techniques. in the prt schemes, each ma must always maintain bindings of all mns registering with its network. moreover, if each ma receives a registration request message from a mn in these schemes, it should process the message, forward the message to its peers and then wait until it has received all the acknowledgement messages from them. thus, the prt schemes result in high failure-free overhead during registration process if the number of mas in the same network increases and are not scalable to the number of mns registering with each network. log-based rollback recovery schemes, which use checkpointing and message logging techniques, are inexpensive during failure-free operation compared with the schemes using passive replication techniques [alvisi (1993)]. in this paper, we present a new fault-tolerant scheme (denoted by cml scheme) for mas using checkpointing and message logging techniques. the cml scheme reduces the failure-free latency even if the number of mas in a network increases, and improves scalability to a large number of mns registering with each network compared with the prt schemes and provides fast recovery for taking over failed mas. additionally, the cml scheme allows each failed ma to recover bindings of the mns registering with the ma when it is repaired even if all the mas in the same network fail. the rest of the paper is organized as follows. in section 2, we describe the overview of mobile ip based systems and problems of the previous fault-tolerant schemes respectively. in section 3, we present our fault-tolerant scheme, explain the description of the scheme in details and prove its correctness. section 4 compares our scheme with others and then, in section 5, we conclude this paper. preliminaries system model in mobile ip, each mn must have a unique home address and a home agent on its home network. if a mn moves from its home network to a foreign network, the current location of the mn is identified as a care-of address (coa), and the mapping between the home address and the coa of the mn is called a binding [solomon (1998)]. whenever the mn enters into a new foreign network, it must register by sending a registration request message to a foreign agent (fa) in the new network. the request message includes the home address of the mn ajis vol 9 no. 1 september 2001 and the ip address of its home agent (ha). then, the fa sends a registration request message to the ha. the message contains the home address and coa of the mm, the ip address of the ha, a registration lifetime and an identifier which uniquely identifies the registration request message. when the ha receives the request message, it updates the binding of the mn, and then sends a registration reply message back to the fa. when the fa receives the reply message, it updates its own table and forwards the message to the mn. a mn in a foreign network can obtain a coa in one of two ways as follows. first, if there is a fa in the foreign network, the mn will attempt to obtain a care-of address from the agent by using an agent discovery protocol [johnson (1994), solomon (1998)]. in this case, the ip address of the fa is used as the coa of the node. second, if there is no fa in the network, the mn can obtain a collocated coa in the network using a dhcp-like protocol [droms (1993)]. if a correspondent node (cn), which may be a mn or a fn, sends a packet to a mn, this packet is routed to the home network of the mn when the mn is in the network. when the mn is not in its home network, its ha intercepts, encapsulates and then tunnels the packet to the fa in the foreign network where the mn is currently located [perkins (1996a), perkins (1996c), simpson (1995)]. then, the fa de-tunnels the packet to the mn. if the mn currently uses a collocated coa, de-capsulation of the packet must be carried out by the mn rather than the fa. however, this triangle routing scheme may be inefficient because the messages, destined to each mn, should be first routed to its ha. in order to solve the problem, the route optimization scheme [perkins (1996d)] was proposed in which the messages are routed directly to the coa of the mn. in this scheme, each cn maintains a binding cache containing the co as of the communicating mns. the drawback of the scheme is that it forces the cn to be aware of mobility of the mns. each fa or ha is connected by a fixed wired network, which provides reliable fifo delivery of messages. each mn can directly communicate with its local fa or ha via a reliable fifo wireless network only if the mn is in the network covered by the mobility agent we assume that nodes, including fas, has, other fns and mns, fail, in which case they lose the contents of their volatile memories and stop their executions, according to the fail stop model [schlichting (1985)]. multiple fas or has can be allocated to each network in order to serve a large number of mns on the network like in figure 1 [binkley (1997)]. separate nodes or a single node on a network may have the function of ha and fa. similarly, the existing ip routers or separate nodes on a network may implement either function or both. we assume that the communication network is immune to partitioning and there is a stable storage that every ma can always access that persists beyond processor failures, thereby supporting recovery from failure of an arbitrary number of processors [elnozahy (1992)]. the execution of each ma is piecewise deterministic [elnozahy (1999), strom (1985)]: at any point during the execution, a state interval of the ma is determined by a non-deterministic event such as delivering a registration request message. the &-th state interval of a ma/?, denoted by sip k (k > 0), is started by the delivery event of the )t-th registration request message m ofp, denoted by devp k(m). letp's state, sp' ={sip°, sip' sip}, represent the sequence of all state intervals up to sip'. therefore, givenp's initial state, sip, and the non-deterministic events, [devp, devp 2, ..., devp], its corresponding state sp is uniquely determined. all information needed for replaying the delivery event of a message during recovery is called determinant of the event. definition 1. sip is stable if a determinant ofdevp'(m) is saved on stable storage. definition 2. sip is volatile if it is not stable. definition 3. spis recoverable if the system has sufficient information for replaying its failure-free execution up to sip in any future failures. lemma 1. sip' is stable ifsp is recoverable. proof. we prove this lemma by contradiction. assume that sip is unstable, i.e., volatile if sp is recoverable, p can replay devp\m) in any failure by definition 3. to do so, it has to obtain the determinant of devp'(m) during recovery. the stable storage is the only one that survives in an arbitrary number of failures by the fail-stop model and the property of the storage. thus, the determinant should have been saved on stable storage in a previous failure free execution. therefore, sip is stable by definition 1. hence, this contradicts the hypothesis. ajis vol 9 no. 1 september 2001 •. •* .. cell wireless link figure 1 mobile ip infrastructure lemma 2. sp' is recoverable if v/t (0 £ k < i): sip* is stable. proof. the proof proceeds by induction on /', the index of the state interval of each mobility agent p. [base case] if sip° is stable in this case, sp° is trivially recoverable because sip° is the initial state interval of p and deterministic. [induction hypothesis] we assume that the theorem is true in case that i = n. [induction step] if sp" is recoverable and there is the determinant of devp n+l(m) on stable storage, the theorem is true forp in case that i = n +1. in this step, sp" is recoverable by induction hypothesis because v& (0 < k <, i): (si,* is stable), and sip n+l is stable. therefore, sp"*1 is recoverable. by-the induction, sp is recoverable if v* (0 £ k ̂ /): sip k is stable. problems of prt schemes prt schemes [ghosh (1998)] to mask failures of multiple mas in a network use passive replication techniques. in the prt schemes, each ma in the network must always maintain bindings of all mns registering with the network. moreover, if each ma receives a registration request message from a mn in the schemes, it should process the message and forward the message to its peers and wait until it has received all the acknowledgement messages from them. thus, the prt schemes result in high failure-free latency during registration process as the number of mas allocated to a network increases, and are not scalable to the number of mns registering with each network. additionally, in the schemes, each failed ma cannot recover bindings of the mns registering with the ma when it is repaired if all the mas in the same network fail. ajis vol 9 no. 1 september 2001 to illustrate the stated problems of the schemes, consider the examples shown in figure 2 and 5. in figure 2, there are three mas, ma], ma2 and ma} in network a. if ma, receives a registration request message from a mn, named ma,, it updates ma,'s binding using the message and then forwards the message to ma2 and ma3, respectively. after ma2 and ma3 have received the message from ma,, they update maj's binding using the message, and then send each an acknowledgement message to ma,. after ma, has received the acknowledgement messages from ma2 and ma3, it sends a registration reply message to ma,. in this figure, we can see that the total number of messages generated per registration request message and the number of messages on the critical path for registration operation in the previous schemes increases as the number of mas allocated to a network increases. thus, they result in high latency during registration process. additionally, in the prt schemes, each ma in a network should still maintain bindings of all the mas registering with the network. for examples, in figure 3, 6000 mns register with network a, and ma, serves 2000 among them, ma2 serves 1000 and ma3 serves 3000. however, ma,, ma2 and ma} should respectively maintain the bindings of 6000 mns registering with network a. therefore, we can see that the traditional schemes are not scalable to a large number of mns registering with the same network even if there are multiple mas in the network. moreover, in the schemes, if ma,, ma2 and ma3 all concurrently fail in figure 3, each of the three mas cannot recover bindings of the mns registering with it from any other ma when it is repaired. network a registration request message update binding of a mn acknowledgement message figure 2 in case of performing a registration process in the prt schemes network a 2000 1000 3000 the number of mns' bindings managed by each ma the number of mns served with each ma figure 3 in case of scalability problems of the prt schemes ajis vol 9 no. 1 september 2001 the cml scheme basic idea to solve the stated problems of the prt schemes in this paper, we present the cml scheme using checkpointing and receiver-based pessimistic message logging techniques. for example, suppose ha] is the home agent ofmnj and mn] currently obtains a care-of address from fa] on the foreign network as in figure 1. in this case, mtv/ must send a registration request message to ha] through fa,. receiving the message, ha/ authenticates the message. if the message is invalid, hat sends mtv/ a registration reply message for rejection. otherwise, hai concurrently performs the following two executions: logging the message into the stable storage and updating the binding of mtv/ using the message. after having completed both executions, it sends a/tv; a registration reply message for acceptance. this step ensures that even if ha] fails, one among other home agents on the network, named hah can recover the bindings of all the mobile nodes registering with hat by restoring the logged registration request messages from the stable storage and replaying them. moreover, each home agent should periodically save the bindings of all the mobile nodes registering with it on the stable storage and remove all the logged messages beyond the previous checkpoint from the stable storage. therefore, the cml scheme can reduce the failure-free overhead compared with the prt scheme presented in [ghosh (1998)] because in the cml scheme, each ma need not forward each registration request message to the other mas and wait for each an acknowledgement message from them. furthermore, the cml scheme improves the scalability to a large number of mobile nodes registering with each network compared with the prt scheme because each home or foreign agent need to maintain the bindings of only the mobile nodes registering with it. in mobile ip, each ma broadcasts an agent advertisement message via its wired or wireless network interface every few seconds. therefore, in the cml scheme, each ma detects if other mas fail or not by monitoring their agent advertisement messages. for example, if hat in network b fails in figure 1, live has on the network can detect its failure because they may receive no agent advertisement message from it. in the cml scheme, one among the live mas, namely haj, which currently has the minimum number of registering mobile nodes, takes over hai. this step ensures that the cml scheme is scalable even during takeover compared with the prt schemes using passive replication techniques. haj restores the bindings of all the mobile nodes registering with hai and obtains the logged messages for ha, from the stable storage. then, it can recover the latest bindings of all the mns, which haj has managed before it failed, by replaying the messages in receive sequence order. after that, haj performs a gratuitous address resolution protocol mapping ha is ip address to haj's hardware address to take over ha, [plummer (1982)]. then, haj serves the mns on behalf of ha,. therefore, the cml scheme provides the mns with transparency of their ma's failure and replacement. if a failed ma, named mah is repaired in the cml scheme, it should monitor any agent advertisement message, including its ip address, for a few seconds and perform a gratuitous address resolution protocol mapping its ip address to its hardware address. if no other ma has taken over the role of the repaired agent, it should restore the bindings of all the mns managed before it failed and obtain its logged messages from the stable storage. then, it can recover the latest bindings of the mns served in its pre-failure state by replaying the messages in receipt sequence order. if it receives an agent advertisement message including its ip address from a live ma, named mak, it should require from mak the bindings of the mns served in its pre-failure state. if mak fails during its recovery, ma, can recover the latest bindings of the mns using its checkpoint and logged messages on the stable storage. therefore, the cml scheme allows each failed ma to recover bindings of the mns registering with the ma when it is repaired even if all the mas in the same network fail. the description in this part, we will first describe our fault-tolerant scheme for home agents and then for foreign agents. the description for home agents (1) data structures every home agent in mobile ip has the following data structures for our scheme. • hatablef. it is a vector for saving the timer of each home agent clustered hi a network. the timer of each home agent is used so that hai detects whether the agent is alive or failed currently. the timer of each home agent is initialized to initjtime. • rsn,: it is the receive sequence number of the latest registration request message which was delivered to ha,. it is initialized to 0. • locjinfof. it is a vector for saving the bindings of mobile nodes registering with hat. ajis vol 9 no. 1 september 2001 • logjnfof. it is a set for saving every permitted registration request message delivered to haj beyond the latest checkpoint and rsn, of the message. its element is of the form e = (msg, rsri). it is initialized to 0. (2) checkpointing and message logging algorithm procedure register_mn(m, mn) if(mn's home agent is ha!) then { authenticate m ; if(m is permitted) then { parallel psections section save (m, rsty) into log_infot at the stable storage ; section update mn 's binding in loc_infai using m ; psections end parallel end send a registration reply message for acceptance to mn ; }else send a registration reply message for rejection to mn ; } procedure checkpointing!} save loc_infoj and rsnt into the stable storage ; remove all e e log_info, at the stable storage ; figure 4 procedures for haj's logging each registration request message during registration process and periodically saving bindings of mns registering with hat into stable storage figure 4 shows a formal description of our checkpointing and receiver-based pessimistic message logging algorithm. whenever a home agent haj receives a registration request message m from a mobile node mn, it calls procedure register_mn(), which first authenticates m. if m is valid, hai performs two executions in parallel: incrementing rsnt by one and saving (m, rsni) into log_jnfoi at the stable storage, and updating mn's binding in .loc_infai using m. after having completed both executions, it sends mn a registration reply message for acceptance. otherwise, hai sends a registration reply message for rejection to mn. if haj attempts to take its local checkpoint, it calls procedure checkpointing(). in this procedure, ha, saves loc_infoi and its current receive sequence number into the stable storage. then, it removes all the messages logged in log_jnfot beyond the previous checkpoint. (3) failure detection algorithm procedure recv_aam(/n,/) hatableilf] +-init_time; procedure faiiure_detect() for all k e other home agents in the same network of/£4, st (hatablet[k] > 0) hatable,[k] «hatable,[k] -1 ; for all k e other home agents in the same network of hai st (hatable^k] = 0) if(election_for_takeover(a) = 0 then take over hak; figure 5 procedures for ha,'s detecting failures of other home agents in the same network using agent advertisement messages the home agent failure detection algorithm using agent advertisement messages is given in figure 5. if a home agent hai receives an agent advertisement message from another home agent haj, it calls procedure recv_aam() to set the timer for haj in hatable, to init_time. whenever advertisingjnterval (2 ~ 3) seconds have elapsed, hai calls procedure failurej)etect(). in this procedure, it decrements the timer for every other home agent by one. if among the other home agents in the ajis vol 9 no. 1 september 2001 same network, there are ones for which timers expire, procedure election_for_takeover() is called so that the remaining home agents determine which live home agents take over the failed ones. in this procedure, among the remaining live agents, one, which serves the minimum number of mobile nodes, takes over a failed agent. (4) takeover and recovery algorithm figure 6 and 7 show a formal description of our takeover and recovery algorithm of a home agent, ha,. if hat takes over a failed agent haj after having completed election_for_takeover(), it calls procedure take_over() in figure 6. in this procedure, hat restores loc_fnfoj, log_jnfoj and rsnj from the stable storage and then, it updates all the bindings in loc_jnfoj to the latest by replaying each logged message in receive sequence order. afterward, it performs a gratuitous address resolution protocol mapping ha/s ip address to its physical hardware address. then it should serve the mns having registered with haj, send an agent advertisement message for haj to other home agents every advertising_interval seconds and respond to every address resolution protocol request message sent for haj. when a failed home agent ha, is repaired and rebooted, it calls procedure recoverq in figure 7. in this procedure, it invokes procedure listen_to() to listen to any agent advertisement message including its ip address for sometime (i.e., init_time x advertisingjnterval seconds). then, it performs a gratuitous address resolution protocol mapping its ip address to its physical hardware address. if it receives no agent advertisement message including its ip address, it restores loc_jnfoj, log_jnfoi and rsnt from the stable storage and then updates all the bindings in loc_lnfoi using log_infoi. if it receives an agent advertisement message including its ip address from hak, it requires loc_jnfoi and rsn, from hak by remotely calling procedure req_bindings() at hak. if hak has failed before completing the procedure, /£4, restores loc_fnfoh log_jnfoj and rsni from the stable storage and then updates all the bindings in loc_jnfoj using log_jnfoi. the description for foreign agents in mobile ip, failures of foreign agents are less critical than those of home agents. it means that if a mobile node has registered with a foreign agent in a network and the agent fails currently, the mobile node can re-register with another foreign agent in the same network [perkins (1996b)]. however, to do so, it should send a registration request message to and receive a registration reply message from its home agent through the new foreign agent. procedure take_over(/) restore loc_infoj, log_infoj and rsnj from the stable storage ; for ail e e log_infoj in e.rsn order update loc_jnfoj replaying e.msg ; perform a gratuitous arp binding haj's ip address to hafs physical hardware address ; figure 6 takeover procedures for a home agent hat ajis vol 9 no. 1 september 2001 procedure recoverq £<-listen_to(i); perform a gratuitous arp binding ha,'s ip address to /£4,'s physical hardware address ; (loc_infoi, rsn^ <remote call at hak : req_bindings(r) ; if(hak has failed before completing req_bindings(/)) then { restore loc_infot, log_infoj and rsni from the stable storage ; for all e g z,og_/«/b, in e.rsn order update loc_infoi replaying e.msg ; } } else { restore loc_infoi, log_infot and rsn, from the stable storage ; for all e g log_infot in e.rsn order update loc_infoi replaying e.msg ; } procedure req_bindings(/) return (loc_infoj, rsn/) ; figure 7 recovery procedure for a home agent ha, the re-registration process across the internet may require high latency for recovering binding of the mobile node. thus, this method forces other nodes not to communicate with the mobile node until the registration process is completed. to solve the problem of the traditional method, live foreign agents can recover bindings of all the mobile nodes having registered with failed agents faster than the traditional method by using the same scheme as the scheme for home agents described in the previous section. however, unlike a home agent, a foreign agent logs a registration request message, sent from each mobile node having registered with the agent, into the stable storage only after it has received a registration reply message for acceptance from the home agent of the mobile node. if the foreign agent fails, another live foreign agent in the same network restores bindings of the mobile nodes having served by the failed agent and logged messages. in this case, the live foreign agent need not forward each logged message to the corresponding home agent when replaying the message. if the failed foreign agent attempts to recover, but there is no live agent having taken over the failed agent, it replay each logged message in the same manner. correctness in this section, we prove the correctness of our checkpointing and message logging algorithms, takeover algorithm and recovery algorithm for mobility agents. theorem 1 our checkpointing and message logging algorithms ensure that the current state of/?, sp* (0 £ k), is recoverable. proof. the proof proceeds by induction on k, the index of the state interval of each mobility agent p. [base case] in this case, sp° is the initial state interval of p and deterministic. therefore, sp° is trivially recoverable by lemma 2 because sip is stable. [induction hypothesis] we assume that the theorem is true in case that k=n. [induction step] if there is the determinant ofdevp"*l(m) on stable storage because the algorithms allows sp" to be recoverable by induction hypothesis, the theorem is true f o r p in case that a=«+l. in this step, when/? receives the («+l)-th registration request message m beyond sp", it saves the determinant of devp (m) on stable storage by calling procedure registerjmnq. afterwards, ifp calls checkpointingo, it saves sp a+\ i.e., the current bindings of the mobile nodes registering with it on stable storage, and then removes all the determinants for p from stable storage. therefore, our algorithms allow sp a+l to be recoverable. by the induction, our checkpointing and message logging algorithms ensure that the current state of p, sp (0 < k), is recoverable. theorem 2 if there are n (2 < ri) redundant mobility agents in a network and even n-l among them fail concurrently, all the mobile nodes registering with the failed agents can communicate with other nodes in the system after a live mobility agent has completed our takeover algorithm. 10 ajis vol 9 no. 1 september 2001 proof: the proof proceeds by induction on «, the number of redundant mobility agents in a network. [base case] in this case, there are a failed and a live mobility agent, named mafati and mative, respectively. in our protocol, maiive detects the failure ofmafaii when ma,ive's tinier for mafaii expires, and then take over ma^n by executing procedure election_for_takeover(). then, it restores from the stable storage the bindings of all the mobile nodes having registered with mafail, the logged messages beyond the latest checkpoint for a£4/aw, and the receive sequence number of the latest logged message delivered to mafail. then, it can recover the latest bindings of the mobile nodes by replaying each logged message in receive sequence number order. afterward, it performs a gratuitous address resolution protocol mapping a£4/a//'s ip address to its physical hardware address. then it serves the mns having registered with mafttih sends an agent advertisement message for ma^t to other home agents every advertising_interval seconds and responds to every address resolution protocol request message sent for mafaiithus, maiivt enables the mobile nodes to communicate with other nodes in the system. [induction hypothesis] we assume that the theorem is true in case that n = k. [induction step] for n=k+\, there are k failed mobility agents and a live one hi a network. if the live mobility agent ma,lvt has only to take over the k-ih failed mobility agent mafaiui because mative can take over k-\ failed mobility agents by induction hypothesis, the theorem is true in case that n = k+1. in our takeover algorithm, malive can recover bindings of the mobile nodes having registered with mafaiu[, and then serve the mobile nodes on behalf of m4/a/tt in the same manner like base case. thus, mauve enables all the mobile nodes having registered with k failed mobility agents to communicate with other nodes in the system. by the induction, if there are n redundant mobility agents in a network and even n-l among them fail concurrently, all the mobile nodes registering with the failed agents can communicate with other nodes in the system after a live mobility agent has completed our takeover algorithm. theorem 3 our recovery algorithm allows every failed mobility agent to recover the latest bindings of all the mobile nodes served by it before it failed. proof. we prove this theorem by contradiction. assume that every failed mobility agent cannot recover the latest bindings of all the mobile nodes after having completed our recovery algorithm. when a failed mobility agent mafaa is repaired and rebooted, it calls procedure recoverq. in this procedure, it listens to any agent advertisement message including its ip address for some seconds and then performs a gratuitous address resolution protocol mapping its ip address to its physical hardware address. there are two cases: case 1: there is no live mobility agent having taken over mafau. in this case, the information is in the stable storage that is needed for recovering the latest bindings of the mobile nodes served by mafaii. the information consists of bindings of the mobile nodes, logged messages for mafaii and the receive sequence number of the latest message delivered to its ip address. in recoverq, mafaa restores the information from the stable storage and then recovers the latest bindings of the mobile nodes by replaying each logged message in receive sequence order. case 2: a live mobility agent m4hw has taken over mafait. in this case, mauve has the latest bindings of the mobile nodes served by mafatt. in recoverq, mafat, requires from mauvt the bindings of all the mobile nodes served by it and the receive sequence number of the latest message delivered to its ip address by remotely calling procedure req_bindings() at mativt. there are two sub-cases: case 2.1: mauve successfully completes req_bindings(). in this case, mafai, obtains the latest bindings of the mobile nodes from mauve. case 2.2: malne has failed before it completes req_bindings(). in this case, m4/m/ recovers the latest bindings of the mobile nodes in the same manner like case 1. hence, both case 1 and 2 contradict the hypothesis. table 1. parameters and their meanings parameter meaning nomn number of mobile nodes registering with a network noma number of redundant mobility agents in a network 11 ajis vol 9 no. 1 september 2001 comparisons in this section, we intend to compare the cml scheme with the prt scheme presented in [ghosh (1998)] briefly. generally, performance indices used for evaluation of scalability of the two schemes are the number of mns whose bindings are managed by each ma in a network (denoted by nomnma) and the latency time for its processing a registration request message from each mn (denoted by ltrr). for simplicity, we suppose that each ma in a network serves the same number of mns. it means that if the number of mns registering with a network is n and the number of mas in the network is m, the number of mns served by each agent is (n i m). first, we evaluate scalability of the two schemes with respect to nomnma during failure-free operation. table 1 shows the parameters, which nomnma depends on, and their meanings. ifnomnmas of the two schemes are calculated using the parameters respectively, nomnma of the cml scheme is nomninoma whereas that of the prt scheme is nomn during failure-free operation. figure 8 illustrates nomnma versus nomn when noma is 10. in this figure, nomnma increases as nomn increases in the two schemes. however, we can see that the increasing rate of nomnma in the cml scheme is significantly lower than in the prt scheme. figure 9 illustrates nomnma versus noma when nomn is 6000. as noma increases in this figure, nomnma of the cml scheme decreases whereas nomnma of the prt scheme is always equal to nomn. the reason for these results is that each mobility agent in a network must maintain bindings of the mobile nodes registering with all the redundant mobility agents in the same network in the prt scheme whereas it has only to maintain bindings of the mobile nodes registering with only it in the cml scheme. therefore, we can see that the cml improves scalability to a large number of mobile nodes managed by each mobility agent compared with the prt scheme during failure-free operation. 0 1000 2000 3000 4000 5000 nomn figure 8 nomnma versus nomn (noma=w) next, we evaluate scalability of the two schemes with respect to ltrr during failure-free operation. if each mobility agent receives a registration request message from a mobile node in the prt scheme, it should process the message and forward the message to its peers, and wait for receiving all the acknowledgement messages from them. 12 ajis vol 9 no. 1 september 2001 10 12 figure 9 nomnma versus noma (m>m7v=6000) thus, the total number of messages generated per registration request message in the prt scheme is (2 x (noma-l)) and the number of messages on the critical path is noma. however, in the cml scheme, it should process the message and require the stable storage server to save the recovery information of the message into stable storage, and wait for receiving an acknowledgement message from it. thus, the total number of messages generated per registration request message in the cml scheme is 2 and the number of messages on the critical path is 2. therefore, we can see that the total number of messages and the number of messages on the critical path generated per registration request message in the cml scheme, not in the prt scheme, are always constant. next, we evaluate overheads of the two schemes for taking over or recovering failed mobility agents. in the prt scheme, live mobility agents can take over failed agents fast because they always maintain bindings of all the mobile nodes registering with not only itself but also its peers. therefore, the takeover time of the cml scheme may be longer than that of the prt scheme because each mobility agent has only to maintain the bindings of the mobile nodes registering with it and live mobility agents should recover the bindings of failed ones from the stable storage. however, the takeover time of the cml scheme can be reduced using two methods. the first method is that each mobility agent maintains only the latest in the stable storage among registration request messages sent from each mobile node registering with the agent. this method decreases the number of logged messages that each live mobility agent should replay when it takes over a failed agent the second method is implementing the stable storage as a high-speed and scalable storage system such as storage area network (san) [ibm (1999)]. if there are live mobility agents in a network, the recovery time of failed and repaired agents are the same in the two schemes because they can recover their bindings from the live mobility agents in the schemes. however, otherwise, each failed agent can recover its latest bindings from the stable storage in the cml scheme whereas it cannot recover them in the prt scheme. conclusion in this paper, we identified the problems in the prt schemes using passive replication techniques; high failurefree latency during registration process if the number of mas in the same network increases and forcing each ma to manage bindings of all the mns registering with its network. then, we presented the cml scheme using checkpointing and receiver-based pessimistic message logging techniques. the cml scheme achieves low failure-free latency even if the number of mas in a network increases, and improves scalability to a large number of mns registering with each network compared with the prt schemes. additionally, the cml scheme allows each failed ma to recover bindings of the mns registering with the ma when it is repaired even if all the mas in the same network fail. however, the takeover time of the cml scheme may be longer than that of the prt scheme like in section 4. the takeover time of the cml scheme can be reduced using the two methods mentioned in section 4. 13 ajis vol 9 no. 1 september 2001 references alvisi, l., hoppe, b. & marzullo, k. (1993) "nonblocking and orphan-free message logging protocols", proc. the 23th symposium on fault-tolerant computing, pp. 145-154. binkley, j. r. & mchugh, j. (1997) secure mobile networking: sixth quarterly report winter 1997, portland state university. dixit, a & gupta, v. (1996) mobile-ip for linux (ver 1.00), technical report, suny binghampton. droms, r. (1993) dynamic host configuration protocol, rfc 1541. elnozahy, e. n. & zwaenepoel, w. (1992) "manetho: transparent rollback-recovery with low overhead, limited rollback and fast output commit", ieee transactions on computers, vol 41, pp. 526-531. elnozahy, e. n., alvisi, l., wang, y. m. & johnson, d. b. (1999) a survey of rollback-recovery protocols in message-passing systems, technical report cmu-cs-99-148, carnegie-mellon university. ghosh, r. & varghese, g (1998) fault-tolerant mobile ip, technical report wucs-98 -11, washington university. ibm corporation and international data group. (1999) survey says storage area networks may unclog future roadblocks to e-business, news release. johnson, d. b. (1994) "scalable and robust internetwork routing for mobile hosts", proc. the 14th international conference on distributed computing systems, pp. 2-11. perkins, c. (1996) ip encapsulation within ip, rfc 2003. perkins, c. (1996) ip mobility support, rfc 2002. perkins, c. (1996) minimal encapsulation with ip, rfc 2004. perkins, c. & johnson, d. b. (1997) route optimization in mobile-ip, mobile ip working group, internet draft. plummer, d. c. (1982) an ethernet address resolution protocol or converting network protocol address to 48 bit ethernet address for transmission on ethernet hardware. std 37, rfc 826. postel, j. b. (1981) internet protocol, internet request for comments rfc 791. rose, m. t. (1990) the open book: a practical perspective on osi, prentice hall, englewood cliffs, nj. schlichting, r. d. & schneider, f. b. (1985) 'tail-stop processors: an approach to designing fault-tolerant distributed computing systems", acm transactions on computer systems, vol.1, no.3, pp. 222-238. simpson, w. (1995) ip in ip tunneling, rfc 1853. solomon, j. d. (1998) mobile d?, the internet unplugged, prentice hall series. strom, r. b. & yemeni, s. (1985) "optimistic recovery in distributed systems", acm transactions on computer systems, vol.3, no.3, pp. 204-226. turner, p. (1990) netware communications processes, netware application notes, novell research, pp. 2581. 14 microsoft word ajisfinal 12 2 marina.doc ajis volume 12 no. 2 may 2005 59 an integrated management approach to leveraging knowledge innovation andrew l s goh international graduate school of management, university of south australia adelaide, australia email: andrewgoh1@hotmail.com abstract: knowledge management (km) is widely accepted as an indispensable management discipline. it has now emerged as the next likely source of sustainable competitive advantage. however, as to how knowledge management (km) practices can be harnessed to leverage the pursuit of knowledge innovation has yet to be fully explored. this article discusses the significance of knowledge by describing the transition from “information revolution” to “knowledge revolution” – which enables various forms of knowledge to be absorbed, assimilated, shared and utilised. next, it highlights the practice of knowledge management (km) and explains why km could enable knowledge innovations to evolve, and how information technology (it) should be exploited for this purpose. it then proposes a management framework for leveraging knowledge innovation (ki) by focusing on knowledge-centred principles, knowledge-sharing infrastructures and knowledge-based initiatives, to offer insights into integrating concepts of knowledge management (km) with innovation management (im). in conclusion, it outlines the challenges of knowledge innovation (ki) that merit attention for organisations to better exploit its full benefits. keywords: knowledge innovation (ki), knowledge management (km), innovation management (im), knowledge-centred principles, knowledge-sharing infrastructures, knowledge-based initiatives, information and communication technologies (ict). introduction emergence of knowledge management (km) in the last decade or so, with the significant role played by knowledge-intensive businesses in the economy, the term “knowledge management (km)” has generated a lot of keen interest in the corporate sectors. however, with no universally accepted definition for knowledge management (km,) the term is often conceived in the broadest context, for instance, as a process through which organisations generate value from knowledge-based assets. but in actual fact, km embodies the critical issues of organisational processes that seek synergistic combination of data and information processing capacity through the use of appropriate technologies, and the creative and innovative capacity of human beings (barth, 2000; amidon, 1997). given that the internet has been evolving steadily for over two decades since its origins in the us apranet project in the 1960s and now increasingly used as a km tool, the field of knowledge management has strongly emerged as a “hot discipline”. in the prevailing knowledge-intensive business environment, many see it as the next likely source of sustainable competitive advantage. at the same time, “innovation management (im)”which is a field of discipline that deals primarily with issues relating to how the innovation process could be managed effectively, has attracted much attention too. like knowledge management (km), innovation management (im) has been consistently demonstrated to be a key value creator in organisational growth and business performance (goh, 2004; harkema and browaeys, 2002; giget, 1997). with technological innovations as the mainstay of today’s business, innovation management (im) has become an ajis volume 12 no. 2 may 2005 60 organisation’s integral function. however, both km and im represent areas of management that seemed to reside in separate spheres of influence, with almost no impact or influence on one another. yet, one major area of management concern confronting today’s organisations lies in the efficient use of knowledge-based assets to create better, faster and more cost-effective innovations. integration of km and im the immediate concern, as global competition intensifies, appears to be more than just dealing with km or im issues separately. rather, it involves acquiring the ability to harness km practices for im processes as a deliberate management strategy that would bring about a higher level of organisational performance. in the past, organisations that rely on the success of innovations to sustain organisational performance often ask themselves the question: “how can innovation improve our organisational performance?” with this new strategic management focus, organisations are now asking the question: “how can the pursuit of knowledge innovations enable us to sustain our longterm competitive advantage in the new business world?” (goh, 2004; harkema and browaeys, 2002; clarke and rollo, 2001; davenport and prusak, 1998) given the heightened interest to draw lessons from the management of knowledge innovation (ki), it seems opportune to integrate the two strands of thinking behind knowledge management (km) and innovation management (im). to establish threads of potential integration between knowledge and innovation management concepts, this article explores the linkages between km and im issues to shed light on how they can be drawn closer together. with this purpose in mind, the subsequent sections offer viewpoints to foster a more profound understanding and better appreciation of knowledge innovation. figure 1 provides a pictorial view for potential integration of the two disciplines, namely: innovation management (im) and knowledge management (km) to enable organisations in adopting a more integrated approach towards the management of ki as a source of sustainable competitive advantage. figure 1: knowledge innovation (ki) as a competitive tool competitive advantage innovation management (im) knowledge assets knowledge management (km) management of knowledge innovation (ki) ajis volume 12 no. 2 may 2005 61 the significance of knowledge transition to knowledge revolution in the new economy, knowledge is the primary resource for economic development; and land, labour and capital – the economist’s traditional factors of production – do not disappear, but they simply become secondary (drucker, 1994). traditional factors of production are limited by a threshold of scale and scope as every marginal increase in land, labour or capital results in diminishing returns on additional investment. by contrast, the returns from knowledge are governed by a different law of economics: investment in every additional unit of information or knowledge created and utilised results in a much higher return. this is largely due to “externalities” of an information or knowledge network since the strength and utility of the network rises with increasing membership (yogesh, 2000). the “what” that impacts on traditional types of innovation has shifted from the “tangibles and physical assets” to “processes wherein various forms of knowledge are absorbed, assimilated, shared and utilised with the objective of creating knowledge innovation”. under such a scenario, knowledge capital naturally becomes as a source of sustainable competitive advantage for organisations. while innovation practitioners recognise the importance of knowledge, the apparent confusion between the value of “knowledge” and “information” has caused organisations to sink billions of dollars in information technology (it) investments that have yielded marginal economic results. for instance, it economist paul strassmann concluded that there is no relationship whatsoever between computer expenditures and company performance (strassman, 1997). this prevailing disconnect between it expenditures and the firms’ organisational performance could be attributed to an economic transition from an era of competitive advantage based primarily on information processing to one based on knowledge creation (malhotra, 2000a; 1997). in fact, the rising interest in knowledge innovation as a strategic lever in organisational performance is not entirely new. back as the mid-nineties, support for knowledge innovation was already prevalent. for instance, british petroleum’s ceo sir john browne said that: “anyone in an organisation who is not directly accountable for making profits should be involved in creating, distributing and utilising knowledge that the company may use to make profits.” in other words, he advocates that all employees in an organisation must participate, directly or otherwise, in knowledge processes to innovate for economic gains such a stance fundamentally places the role of knowledge innovation as important, if not more important, as the role of marketing, sales or any profit-driven activity. it has thus become increasingly evident to organisations that the “information revolution” has been superseded with “knowledge revolution” as depicted in figure 2. to succeed in the knowledge economy, organisations who are able to capitalise on the opportunities arising from the availability of knowledge-based assets and ultimately derive the most value from them will be the industry winners. but to harness the most value from these assets, organisations must identify the types of knowledge assets that benefit businesses and understand how km practices can be implemented effectively to improve operational efficiency, reduce costs, achieve higher productivity and boost revenues. hence, an important implication of km research is to contribute findings that would help practitioners develop and improve the use of knowledge-based assets. ajis volume 12 no. 2 may 2005 62 figure 2: towards a knowledge revolution the practice of knowledge management with the changing forms of knowledge in today’s society, the role of managing knowledge has become a key engine for innovation and growth. knowledge adds value to an organisation through its contribution to products, processes and people, while knowledge management (km) transforms information, data and intellectual assets into enduring value by identifying “useful knowledge” for management actions (o’dell, 1996). the high economic value placed on knowledge has prompted organisations to support the design and use of knowledge management systems to enable people to create and apply knowledge effectively. in recent years, the practice of knowledge management (km) has evolved to be of considerable interest to management writers, thinkers and practitioners. coupled with organisations’ need to innovate, this interest on knowledge management (km) practices has been propelled alongside that of innovation management (km). with better recognition placed on the value of knowledge management (km) practices to business, organisations are now investing in ad-hoc project initiatives to manage, harness and exploit knowledge for business use. so far, the contribution made by knowledge management (km) practices, to improve organisational performance, has been significant. one example of the escalating intensity of knowledge in products is the intelligent car, whose engine management systems can monitor the functions of vital engine parts and “knows” in advance which part needs servicing and thus improves the car performance. another example of knowledge in processes is the sharing of best practices, such as in high-tech semiconductor fabrication plants, which brought about huge savings in capital investments. in the case of people, the “skilled knowledge” of experienced persons in commercial transactions (e.g. negotiation) often contributes to whether knowledge-intensive businesses would succeed or fail. with the benefits of knowledge management (km) practices to business, it has resulted in a proliferation of km tools such as expertise access tools, e-learning applications, web portals, discussion and chat technologies, electronic message boards, synchronous interaction tools, and search and data mining tools. while the benefits of km correlate directly to bottom-line savings, there is also a risk associated with investments in knowledge management (km), as they do not necessarily lead to expected benefits due to failures in implementation (lindgren and henfridsson, 2002; storey and barnett, 2000; fahey and prusak, 1998). nevertheless, the economic impact of information revolution (ir) expert systems decision support systems knowledge revolution (kr) internet intranet intranet intranet transition … new km tools …. etc ajis volume 12 no. 2 may 2005 63 km practices to business is manifested in three areas of any organisation. they are, namely: products like patents or technology licences (intellectual capital), processes like financial procedures or manufacturing methods (structural capital) and people like skilled manpower or specialised talents (human capital) as illustrated in figure 3. figure 3: benefits of km practices products processes people intellectual capital structural capital human capital km practices ajis volume 12 no. 2 may 2005 64 evolution of knowledge innovation today’s organisations are in an unending struggle to differentiate themselves from relentless competitors as markets become saturated with innovations all the time. the ability to differentiate depends on the availability of knowledge-based assets and how organisations employ km practices to harness the value of these assets. thus organisations have been trying to differentiate themselves based on unique production processes, rare and distinct skills, creativity, and now on management initiatives such as supply chain management and customer relationship management (gold, malhotra & segars, 2001). as global competition intensifies in most corporate sectors, knowledge management (km) would have to play a more pivotal role in the pursuit of innovation. with mounting pressure to acquire km capability for innovation management (im), many organisations are also fast working towards being recognised as an exemplar of knowledge management (km) practices. for instance, organisations are implementing ad-hoc knowledge-based initiatives like setting up a corporate intranet or initiating communities of km practices or redesigning a core business function around knowledge sharing. however, a majority of these km programmes are still predominantly centred on building computerised databases or electronic repositories for documenting, storing and disseminating best practices. for organisations to earn a sizeable payback for their km efforts, the real payoff lies in applying knowledge to innovation. this requires a management focus on the processes associated with knowledge creation, transmission and sharing in fostering organisational innovation. to cite an example, nokia, an organisation that has consistently applied km practices in its business, has yielded considerable benefits in innovation-related and product development functions. nokia makes use of km practices extensively to understand market trends and customer requirements and puts useful knowledge into action for its innovation pipeline. it is thus not surprising that industry analysts reported that nokia continually delivers a new mobile communication product every 25 days! exploiting it for knowledge management as early as in the 1980s, there were great expectations that knowledge-based computer systems (e.g. expert systems or decision support systems) should be exploited as a knowledge management (km) tool to solve business problems. for close to two decades, the search for km tools was centred on stand-alone information systems (is) such as case tools or commercial expert system shells. but in retrospect, part of the problem was that system developers have focused too much, perhaps overly so, on developing “thinking machines”, rather than designing these machines to augment human thinking. in other words, it appears that the roles of information management and knowledge management should be separated and played by machines and humans respectively. researchers also came to realise that only human beings can take the central role in knowledge management, not computer systems even with their powerful information-processing capabilities (stewart, 1991; 1993; 1995; nonaka, 1991; nonaka and takeuchi, 1995). computers are just merely tools, however great their information-processing capabilities may be. it must therefore be acknowledged that while information generated by computer systems is a rich carrier of “interpretative knowledge” for potential action, “useful knowledge” resides in the user’s subjective context of action based on that information. for this reason, the exploitation of technologies for knowledge management (km) must be seen as a form of human activity, while being mediated by appropriate it tools, fundamentally motivated towards a specific purpose. yet, apart from the social dimension associated with knowledge creation, researchers have adopted ajis volume 12 no. 2 may 2005 65 varying interpretations that relate to mechanistic dimension such as codification or personalisation and the like. nevertheless, the economic-rational view of modern organisations requires a more integrated perspective involving technological, organic and humanistic means of leveraging knowledge innovation for organisational growth (hasan and handzic, 2003; harkema and browaeys, 2002; davenport and prusak, 1998). but ultimately, it is the role of empirical research to enhance knowledge management practice for the pursuit of innovation. an integrated management framework motivation currently, the mainstream km literature in relation to innovation does not deal adequately the management of ki as an area of concern. unlike innovation theorists, most writers concerned with knowledge management (km) issues tend not to exhibit the same degree of understanding on the economic significance of innovation. on the other hand, innovation writers are often less able to articulate how km practices can be applied as an effective management tool. yet, virtually every organisation is grappling with the opportunities presented by km, including new ways to acquire, assimilate and share knowledge in innovation pursuits. by now, it is evident that the emergence of knowledge innovation is offering not only immense potential for organisations to gain a sustainable competitive advantage over rivals, but also the ability to improve organisational performance. for knowledge innovation to be employed successfully as a competitive tool would require a focused business strategy that harnesses knowledge management (km) in ways that offer a new approach to innovation strategies (blumentritt, and johnston, 1999; skyrme and amidon, 1997). hence, instead of investigating “what goes into the bolts and nuts of knowledge innovation”, the current approach focuses on “which strategic aspects of management should knowledge innovation focus on” to offer insights into the strategic management of knowledge innovation (ki). understanding the strategic km issues in the context of im is likely to provide insights into how the role of knowledge innovation can be better managed in areas such as implementation of ict or utilisation of knowledge assets. thus an integrated approach is proposed to guide those strategic aspects of management that would affect the evolution of knowledge innovation in organisations. however, so far, the lack of an integrative view of knowledge innovation (ki) leads to a certain degree of confusion. to better understand the evolution of knowledge innovation, fresh perspectives of km are necessary. in particular, two areas of knowledge literature on ki have yet to be fully addressed. firstly, a definition of knowledge innovation (ki) needs to be specified to put in perspective of its impact on the success of enterprises, economy and society; and to identify the key elements that underscore the critical importance of knowledge-based assets and the management actions required for innovation. secondly, since a comprehensive model for understanding the nature of knowledge management (km) practices in innovation is not yet apparent in extant literature, integrating km and im within the same framework is required. however, two pertinent questions remain to be answered: one, what are knowledge-based assets? two, what does a knowledge innovation really constitute? what are knowledge-based assets? not all information can be considered as “knowledge-based assets”. very often, organisations determine what information qualifies as knowledge-based assets, depending on the context and business objective. besides being conceived as a “human” concept, the view of knowledge-based assets adopted in this article also includes the concept of “objects” that can be coded and stored, ajis volume 12 no. 2 may 2005 66 which in turn may be subsequently mobilised for innovation. in general, knowledge-based assets fall into two categories: explicit or tacit. included among the former are patents, trademarks, business plans and marketing research – any information that can be documented, archived and codified, often with the help of information technology (it). in the case of the latter, it is much harder to grasp as the information is contained in people's heads and the real difficulty lies in figuring out how to document, share and manage it effectively. hence, it means that a highly integrative approach, to managing the processes for creating, classifying, storing, disseminating and utilising knowledgebased assets for innovation, should be adopted (amidon, 1997; drucker, 1988). even if technologies like electronic mail, groupware, and instant messaging tools can be used to manage tacit knowledge, identifying it is itself a major hurdle in most organisations. what does a knowledge innovation really constitute? notably, it was drucker in the 1950s who first coined the term “knowledge worker”, but it was only until fairly recent that corporate leaders, policy-makers and entrepreneurs began to strongly acknowledge that successful innovations are increasingly knowledge-intensive. amidon (1997) has aptly described “knowledge innovation” as the creation, evolution, exchange and application of new ideas into marketable goods and services, leading to the success of an enterprise, the vitality of a nation's economy and the advancement of society. in gist, two key elements are present in the definition. one, it recognises that knowledge is a key component of innovation not technology or finances. two, the actions associated with managing the flow, use and handling of knowledge in an innovation process is another key component. however, it is less clearly delineated as to what extent the literature between km and im, as represented in figure 4, can be drawn as insights into the management of knowledge innovation (harkema and browaeys, 2002; miller and morris, 1999; rogers, 1995). figure 4: management of knowledge innovation strategic management of knowledge innovation with a diverse range of digitised repositories, it tools and communities of best km practices, organisations are racing to revolutionise their approaches to use knowledge-based assets for knowledge management (km) innovation management (im) knowledge innovation (ki) ajis volume 12 no. 2 may 2005 67 innovation. however, applying available km tools alone that ran the gamut from standard, off-theshelf computer packages to sophisticated software designed to support knowledge management (km) activities is inadequate. insofar as knowledge innovation is concerned, it often involves unique and highly professional skills – which are difficult to be trained and learned. the role played by knowledge-based assets in innovation management (im) is also undergoing changes. because of the tacit nature of an organisation’s knowledge and the need to sustain competitive advantage, current thinking points to the management of ki as an effective way to support organisational performance (gupta and mcdaniel, 2002; clarke and rollo, 2001; probst, raub and romhardt; 1999). focusing on the management of ki is strategic because it is created through human exchanges and investments in continuous learning; and knowledge innovations are not stagnant but continually adjust in response to changing environments and market conditions. hence, this article proposed that knowledge innovation should be guided by appropriate knowledge-centred principles, with supporting knowledge-sharing infrastructures to better facilitate knowledge-based initiatives. nevertheless, the inputs of human responses, investment decisions, technological options and past experience should always be taken into consideration for the management of ki. by employing an integrated management framework for knowledge innovation (ki) as shown in figure 5 – in terms of perspectives on principles, infrastructures and initiatives, it would enable organisations to better fulfil their roles in these areas. although the framework extends beyond any specific type of km practices, tools and physical systems, its conceptual development is primarily based on a comprehensive overview of km literature. furthermore, while the actual km implementation may be firm-specific and uniquely diverse, the framework nonetheless provides an integrative view of management thinking that would help organisations merge relevant concepts of km with innovation management. ajis volume 12 no. 2 may 2005 68 figure 5: an integrated management framework knowledge-centred principles if knowledge management (km) practices were to be incorporated into innovation management thinking as a competitive tool for supporting knowledge innovation, organisations must re-examine their role within a larger framework of km practices, built on a mindset of knowledge-centred principles to be practised by the entire community of knowledge workers. this would then place a stronger emphasis on long-term strategic issues relating to knowledge management and thereby better support different forms of knowledge creation in organisations. to make headway along this line of emergent thinking, knowledge-centred principles that typify new approaches of managing knowledge capital for innovation, are identified through a distillation process. but more importantly, these principles incorporate the critical elements of knowledge management relevant to the innovation process. in contrast with other areas of management, six knowledge-centred principles seem to distinguish themselves between the management of knowledge innovation from the other conventional management approaches as outlined in figure 6 (harkema and browaeys, 2002; davis and botkin, 1999; miller and morris, 1999; skyrme and amidon, 1997, davenport, 1993). knowledge innovation human intellectual capital structural capital knowledgecentred principles knowledge sharing infrastructures knowledgebased initiatives people processes products ajis volume 12 no. 2 may 2005 69 figure 6: knowledge-centred principles these knowledge-centred principles are summarised as follows: • understanding innovation value system (not value chain) value chain thinking is linear and static, while the innovation value system consists of non-linear and dynamic knowledge processes and represents interdependent relationships that need to be understood, considered and developed for successful knowledge innovation; • formulating collaborative knowledge strategy (not competitive information strategy) competitive information strategies create win-lose scenarios due to competition for a share of the same information pie, while collaborative knowledge strategies encourage win-win situations through symbiotic relationships by sharing knowledge assets and growing the knowledge pie for all stakeholders. • developing strategic knowledge networks (not strategic business units) strategic business unit management tends to apply isolated islands of information assets, while the strategic knowledge networks foster the flow of knowledge assets between partners, customers, suppliers, internal and external units and other stakeholders, including competitors, in the innovation process. • constructing hybrid human-technology km solutions (not machine-based km solutions) human beings are better at “knowledge skills” while computers are more adept at “information tasks” such as collecting and categorising highly structured information that changes rapidly. to interpret knowledge within a broader context or to combine it with other types of information, or to synthesise unstructured knowledge, humans and machines must complement each other. • fostering bottom-up knowledge process improvements (not top-down “reengineering” of knowledge management approaches) – highly creative and useful knowledge work carried out by autonomous knowledge workers often require less top-down intervention and more bottom-up spontaneity. • focusing on customer success (not customer satisfaction) customer satisfaction meets today's needs only, while a deliberate focus on customer success helps identify future requirements and unmet needs, which form the competitive innovation thinking: value system versus value chain strategy: collaborative knowledge versus competitive information management: knowledge networks versus business units solutions: human-technology versus machine-based process improvements: bottom-up versus top-down customer focus: success versus satisfaction ajis volume 12 no. 2 may 2005 70 forces for firm growth and business expansion. notwithstanding the recognition given to these knowledge-centred principles, many km tools could also facilitate, to a certain extent, the management of knowledge innovation (ki). but more importantly, managing ki should not be seen as a technology-centred concept. while technological tools may support its management, it must be borne in mind that management decisions relating to knowledge innovation are ultimately based on who (people), what (knowledge assets) and why (business objectives). for this reason, organisations that went about solely installing large-scale centralised it systems or any advanced it tools in the hope that they have thus established a successful ki programme may be disappointed in the long haul. knowledge-sharing infrastructures the driving force behind the rapid transformation to greater inter-connectivity, accelerated data transmission and reduced costs of communications is none other than information and communication technologies (ict). undoubtedly, the greatest benefit of ict is its reach and one of its most important roles in km is its knowledge sharing advantages. currently, of the ict available at today’s workplace that have the most profound impact on km practices is that of the internet and related technologies – which offers an incredible information source direct to the end-users without the need to involve an intermediary such as information professionals (cio, 1996). although the internet has been growing in stature and impact, its knowledge-sharing considerations are almost entirely confined to its information role. whether it really shares “usable knowledge” for developing innovations is still subject to debate. while the cost of building national level knowledge-sharing infrastructures may be very high, there are also hardly any equally cost-effective technological alternatives. in countries where the knowledge-sharing infrastructure is weak (no optical fibre link to world’s information superhighway, for instance), the dial-up access charges may be even higher than the internet usage costs itself. but once the infrastructure is established, the cost of propagation technology can be very low. nevertheless, knowledge sharing continues to be impeded by the digital barrier unless there is a universal access to ict in all parts of the world. internet heralds the way for collaborative sharing of knowledge-sharing assets. to effectively implement knowledge-sharing infrastructures for ki, organisations should consider, at a minimum, ample support for the processes of codifying and storing knowledge, creation of knowledge maps (or corporate directories), sharing of best practices, and developing knowledge networks (maryam and leinder, 2001; davenport and prusak, 1998). for organisations to better exploit the use of the internet in the management of knowledge innovation, an integrative approach to implementing its knowledge-sharing structures of ict is required. viewed from km perspectives, the knowledgesharing infrastructures of the internet and its related technologies should possess the following five characteristics (barth, 2000; miller and morris, 1999; strassmann, 1997) as shown in figure 7: • it uses a widely-supported communications standard protocol – which means that it is universally accessible from multiple locations and through different computer platforms; • it offers world-wide access, with increasingly more international service providers (isp) – which means that individuals, who travel a lot, can use the internet like a corporate network without building an in-house option; • it avails end-user software, such as electronic mail and browsers to be universally available at low cost (and often free) – which means that it is cost-effective to implement on an enterprise-wide basis; ajis volume 12 no. 2 may 2005 71 • it employs a high-speed, broadband, digital network based on optical fibre cables with limitless bandwidth – which means that it provides quick access at an affordable cost; • it provides a quick means of publishing information, through the world wide web, that can be shared world-wide – which means that the universal repository of information resources can be updated and widely shared at an attractive cost. figure 7: knowledge-sharing infrastructures additionally, due to the growing interest of organisations who want to share information, knowledge-sharing infrastructures have expanded to include the use of intranets for better accessibility in a corporate environment. the advantages of using intranets are similar to those that use the internet in external information access and communications. because end-users are familiar with browser interfaces, information can be shared across different local area networks and computer platforms, and published information is instantly available over the whole network. besides, information may be presented in different formats, like word-processed documents. furthermore, intranets can host transaction and database applications with the web browser being the universal interface to different “back-end” systems. whatever the format and content may be, an intranet essentially facilitates the sharing of internal information, while the internet gateways external information. knowledge-based initiatives differences in perception of km issues exist in various business activities – for example, how much information should be involved before we consider a business activity as “knowledge-intensive”? seemingly, what was more crucial is not whether information was utilised, but whether the business activities involve the right kind of knowledge-based initiatives that produce intended benefits. for a the internet intranets knowledge-sharing infrastructures • uses widely-supported communications standard protocol; • offers world-wide access; • avails end-user software; • employs a high-speed, broad-band, digital network based on optical fibres; • provides means of publishing information. ajis volume 12 no. 2 may 2005 72 right knowledge-based initiative to take place, quantity rarely equals quality, and indeed, the objective of it is to identify “knowledge gems” from a “sea of information”. whether a knowledge-based initiative contributes to an innovation process depends largely on the interpretation of knowledge innovation (ki) as depicted in figure 8. there are two distinct interpretations of ki that give rise to various knowledge-centred initiatives, which eventually can be translated to knowledge capital through value creation. the first interpretation, which centres on the traditional information-processing model of knowledge management, assumes a problem as given and the solution as based upon a pre-specified notion of the business environment. the second interpretation, which revolves around human imagination and creativity as a new mental model of knowledge creation, constructs the definition of the problem from the knowledge available at a certain point in time and context (malhotra, 2000a; 2000b; 1997; yoneji, 1990). based on a review of current literature on km practices centred on organisations which host information in the internet, nine knowledge-based initiatives have emerged to be of considerable significance to knowledge innovation in products, processes and people (amidon, 1997; skyrme and amidon, 1997; de long and davenport, 1996; skyrme, 1991). they are listed below: products • structuring and mapping knowledge – such as developing typologies or synthesising different knowledge types; • developing knowledge databases – documenting best practices, expert directories, market intelligence and so on; • embedding knowledge in new products and services – such as the introduction of smart products; processes • capturing and re-using information as knowledge such as utilising old project deliverables as source materials to develop specifications for a new project; • sharing of knowledge or lessons learnt about knowledge processes from one part of the organisation to another, through distribution, dissemination or personal interactions; • measuring and managing the value of knowledge assets – such as attaching an economic worth to the ownership of patents or managing the rights of these patents; people • creating knowledge or intellectual capital teams to help identify and audit intangible knowledge-based assets using people from multiple disciplines and to develop new practices of knowledge management; • forming people-oriented knowledge centres – establishing focal points for the development of knowledge skills, managing and enhancing knowledge databases and facilitating knowledge flow; • using collaborative technologies for knowledge exchange between people the implementation of intranets (internal internet), electronic mail, lotus notes, groupware for multiple-user access. ajis volume 12 no. 2 may 2005 73 figure 8: knowledge-based initiatives while knowledge-based initiatives need not necessarily be implemented via the internet, the relative ease of use of internet technology, its cost-effectiveness and immediate availability to a global audience of users in more than 190 countries offer many advantages. three advantages are cited here. firstly, the internet performs exceptionally well in facilitating remote and rapid access, through network connections and enhancing global communications at a low cost. secondly, with major technological developments on the internet occurring frequently, it offers quantum improvements in access efficiency using browsers, search engines and intelligent software that search the most relevant information for end-users. thirdly, since knowledge management (km) activities are no longer implemented through stand-alone point solutions, such as expert systems or group decision support systems, the internet offers a means for enterprise-wide knowledge-based initiatives to be better accomplished through groupware systems like lotus notes and intranets. future challenges for knowledge innovation while there may be extensive literature written in the two management areas, namely knowledge management (km) and innovation management (im) as separate areas of concern, limited research was done specifically on the management of knowledge innovation (ki). this article integrates the two management areas: km and im, each having a momentum of its own, into one singular focus to foster new thinking in identifying sustainable competitive advantage through knowledge innovation. it suggests that to effectively manage knowledge innovation in any organisation, three strategic aspects of management should be looked into as a whole. one, organisations should embrace a mindset of knowledge-centred principles to better maximise the value of their knowledge capital for innovation. two, organisations should implement knowledge-sharing infrastructures through the use of ict to better enhance the knowledge-based assets required for the pursuit of innovation. three, organisations should promote knowledge-based initiatives to better facilitate the creation of knowledge innovation. to address the management of knowledge innovation (ki), an integrated interpretation of knowledge innovation informationprocessing model new mental model of knowledge creation knowledge-based initiatives ajis volume 12 no. 2 may 2005 74 management framework is thus proposed to better fulfil organisational roles. but to fully exploit the benefits of knowledge innovation (ki), three challenges that merit attention have been identified as follows: firstly, as knowledge innovation encompasses the use of various types of knowledge assets; and social, economic and other forms of tacit knowledge, the innovation process requires the assimilation of human imagination, intuition and creativity at all levels to achieve success. to unleash the potential of knowledge innovation, the challenge is to permeate knowledge-based initiatives to various layers of society – industrial, organisational and humanistic structures. this would then enable organisations and individuals to utilise knowledge capital to participate in the core activities of knowledge innovation. secondly, although the objective of knowledge innovation is to improve organisational performance, it should not be viewed as the “magic cure” for ailing organisations. paradoxical as it may sound, the fewer km practices an organisation requires in its pursuit of innovation, is also a reflection that it has championed knowledge-centred principles adequately in its business. the challenge posed to organisations is to identify, create and acquire new knowledge assets all the time, and to make these assets readily available, transparent and freely mobile for individuals involved in knowledge innovation. thirdly, for knowledge innovation to flourish, it must be fostered within an enabling environment of collaborative enterprises. since knowledge innovation constitutes the “discovery of new knowledge assets”, oftentimes based on competencies and talents inside and outside an organisation, collaboration between enterprises should be recognised, encouraged and valued. ultimately, the challenge lies in strengthening the role of all stakeholders in a knowledge enterprise towards collaboration – embracing a knowledge-oriented culture and nurturing a knowledge-sharing ecosystem. after all, the success of any knowledge innovation depends on the extent of collaboration amongst individuals who have created the knowledge – the very trait that make knowledge useful, beneficial and valuable to society. n conclusion, the ability of an organisation to capture, develop and use knowledge-based assets for innovation constitutes a source of sustainable competitive advantage. however, the entire process of leveraging knowledge innovation is likely to be encountered with failure if the knowledge management aspects of innovation are left to chance. furthermore, like all types of innovation, knowledge innovation, too, is often managed with a business objective in the context of “imperfect conditions”. it should therefore be acknowledged that the ultimate goal of managing knowledge innovation effectively is perhaps one for all “knowledge workers” to strive for, but never to be completely addressed. nevertheless, it is clearly outlined in this article that an integrated management approach, depending on organisational context and business objective, with due considerations given to knowledge-centred principles, knowledge-sharing infrastructures and knowledge-based initiatives, stands a far better chance of success over the long-term. i references amidon, debra m. (1997). innovation strategy for the knowledge economy: the ken awakening, boston, ma: butterworth-heinemann. barth, s. (2000). “defining knowledge management”, crm magazine (4 july 2000), information today inc. blumentritt, r. and johnston, r. (1999). “towards a strategy for knowledge management”, technology, analysis and strategic management, vol. 11, no. 3, pp. 287-300. cio (1996). “librarians at the gate”, webmaster magazine, cio (september). ajis volume 12 no. 2 may 2005 75 clarke, t., and rollo, c. (2001). “capitalizing knowledge: corporate knowledge management investments”, creativity and innovation management, vol. 10, no. 3; pp. 177–188 davenport, t. (1993). process innovation: re-engineering work through information technology, boston, ma: harvard business school press. davenport, t. h., and prusak, l. (1998). working knowledge: how organisations manage what they know, boston, ma: harvard business school press. davis, stan and botkin jim (1999). “the coming of knowledge-based business”, harvard business review, pp.165-170 (september-october 1999). de long and davenport, b. (1996). “what is a knowledge management project?” ernst & young llp centre for business innovation, research note cbi311. drucker, peter (1988). “the coming of the new organisation”, harvard business review, pp. 4553 (january-february 1988). drucker, peter (1994). “the theory of business”, harvard business review, pp. 95-104 (september-october 1994). fahey, l. and prusak, l. (1998). “the eleven deadliest sins of knowledge management”, california management review, vol. 40, no. 3, pp. 231-246. giget, marc. (1997). “technology, innovation and strategy: recent developments”, international journal of technology management, vol. 14, no. 6, pp. 613-634. goh, andrew (2004). “enhancing competitiveness through innovation: issues and implications for industrial policy-making”, international journal of applied management and technology, vol. 2, no.2, pp. 88-113 (may 2004). gold, a., malhotra, a., and segars, a. (2001). “knowledge management: an organizational capabilities perspective”, journal of management information systems, vol. 18, no. 1, pp. 185-214. gupta, a., and macdaniel, j. (2002). “creating competitive advantage by effectively managing knowledge: a framework for knowledge management”, journal of knowledge management practice, vol. 3, no. 2, pp. 40-49. hasan, h. and handzic, m. (2003). australian studies in knowledge management, australia: uow press (october 2003). harkema, s.j.m. and browaeys, m. j. (2002). “managing innovation successfully: a complex process”, european academy of management annual conference proceedings, euram 2002, brussels. lindgren, r. and henfridsson, o. (2002). “using competence systems: adoption barriers and design suggestions”, journal of information and knowledge management, vol. 1, no. 1, pp. 65-78. malhotra, y. (1997). “knowledge management in inquiring organisations”, proceedings of 3 rd america conference on information systems (philosophy of information systems minitrack), indianapolis, in, august 15-17, 1997. malhotra, y. (2000a). knowledge management and new organisation forms: a framework for business model innovation, knowledge management and virtual organisations, hershey, pa: idea group publishing. malhotra, y. (2000b). “knowledge management for e-business performance”, information strategy: the executives journal, vol. 16, no. 4, pp. 5-16. maryam, m. and leidner, d. (2001). “knowledge management and knowledge management systems: conceptual foundations and research issues”, mis quarterly, vol. 25, no. 1, pp. 107-136. miller, w.l. and morris, l. (1999). fourth generation r&d: managing knowledge, technology and innovation, canada: john wiley & sons inc. ajis volume 12 no. 2 may 2005 76 nonaka, ikujiro (1991). “the knowledge-creating company”, harvard business review, pp. 96104 (november-december 1991). nonaka, ikujiro and takeuchi, hirotaka (1995). the knowledge creating company: how japanese companies create the dynamics of innovation, new york: oxford university press. o'dell, carla (1996). “a current review of knowledge management best practices”, knowledge management 96 conference, business intelligence, london (december). probst, g., raub, s. and romhardt, k. (1999), managing knowledge: building blocks for success, london, uk: john wiley and sons. rogers, e (1995). diffusion of innovations (4 th edition), new york: the free press. skyrme, david j. (1991). “knowledge networking”, the intelligent enterprise, aslib, vol. 1, no. 9/10, pp. 9-15 (november). skyrme, david j. and amidon, debra m. (1997). creating the knowledge-based business, london, uk: business intelligence. stewart, thomas a. (1991). “brainpower”, fortune, pp. 44-56 (3 june). stewart, thomas a. (1993). “your company's most valuable asset: intellectual capital”, fortune, pp.51-56 (3 october 1993). stewart, thomas a. (1995). “trying to grasp the intangible”, fortune, pp.157-161 (2 october). storey, j. and barnett, e. (2000). “knowledge management initiatives: learning from failure”, journal of knowledge management, vol. 4, no. 2, pp. 145-156. strassmann, p.a. (1997). the squandered computer: evaluating the business alignment of information technologies, new canaan, ct: information economics press, yoneji, masuda (1990). ’the information society as a post-industrial society’, institute for the information society, tokyo. translated into english as managing in the information society, basil: blackwell publishers. microsoft word special final.doc ajis special issue december 2004 102 using cultural probes to explore mediated intimacy jesper kjeldskov1, 2, martin r. gibbs1, frank vetere1, steve howard1, sonja pedell1, karen mecoles1, marcus bunyan1, 3 1 the university of melbourne australia 2 aalborg university denmark 3 charles sturt university australia abstract intimacy is a crucial element of domestic life that has received insufficient attention from humancomputer interaction (hci) researchers despite their rapidly growing interest in the design of interactive technologies for domestic use. intimate acts differ from other activities, and there are unexplored opportunities to develop interactive technologies to support these acts. this paper presents the first phase of a two-part study exploring the potential of interactive technologies to support intimate relationships. we contribute to this uncharted domain of hci research a literature review of concepts useful in understanding intimacy and methods for its investigation. we conclude with preliminary results and suggestive design ideas for interactive technologies intended to support intimacy. keywords domestic technology, personal technology, intimacy, cultural probes, contextual interviews introduction interest in the design of interactive technologies for domestic use has been growing within the human-computer interaction field (hindus 1999). recent research has fruitfully examined the instrumental activities involved in coordinating and scheduling family behaviour (harper 2003), produced rich studies of the multiple meaning attached to domestic routines (crabtree 2003), performed empirical and technological explorations of fun and leisure (blythe et al. 2003) and developed proposals for aids to help family members stay in touch (hofmeer 1999). inspired by this research, we have been investigating and exploring a crucial element of domestic life that has received relatively scant attention to date from hci researchers – intimacy. our research is motivated by a desire to understand how intimate relationships between close family members might be supported by interactive technologies with a view to designing domestic and personal technologies for this very purpose. intimate relationships are different from the kinds of relationships that have been typically studied by hci researchers such as those found in the workplace or amongst social networks of friends. intimate acts also differ from the domestic behaviours usually addressed in the literature (see harper 2003 for a review) and attempting to study acts of intimacy presents the researcher with a number of unique and interesting challenges. studying intimacy is challenging because intimate acts are ephemeral and transient yet ubiquitous and crucial to the ongoing life of an intimate relationship. they form the material and background of close personal relationships, yet occur in the doing and then often vanish unremarked. while the informational content of intimate acts may be low and seemingly trivial to outsiders, the act itself can be laden with emotional significance for those involved. intimate acts often entail self-disclosure, and thus privacy is a concern. much of what passes between intimates is unsaid and premised on deep knowledge and understanding of one another and occurs in the context of a rich, shared and sometimes idiosyncratic view of the world that may be difficult for others to fathom and comprehend. intimacy also involves assumptions about commitment and mutuality, and carries nuanced expectations for reciprocity and exchange that are negotiated and arrived at over many years, yet remain fragile and are occasionally misjudged leading ajis special issue december 2004 103 to misunderstandings and conflict. finally, unlike instrumental tasks (e.g. coordination of family activities), or leisure activities (e.g. games) there is no generally accepted language for describing and discussing intimacy, especially in relation to designing technologies for its support. in this paper we present our response to the interesting issues and challenges arising from our efforts to understand how intimate relationships can be supported with interactive technologies. our study of ‘mediated intimacy’ has been divided into two major phases. in the first phase, we have focused on understanding current practice. to this end, we have adopted the ‘cultural probes’ techniques developed by gaver et al. (1999) and the equator team (cheverst et al. 2003) and extended them with contextual interviews (beyer and holtzblatt 1998). we have used these methods to investigate how people appropriate and use the gamut of artefacts, devices and interactive technologies at their disposal to perform the various communicative acts that enable and sustain their intimate relationships. while we have adopted methods that are gaining prominence in the hci community for investigative work in the domestic environment, how best to explore and understand intimacy remains an open and unexplored question. as a result, the first phase of our study has also had a strong methodological flavour with the aim of developing and refining techniques suitable for investigating these, and similar, phenomenon. the second phase of our study will take the insights and understandings generated in phase one and use them to design interactive technologies to support intimacy. these insights will feed a range of design activities such as the development of use scenarios, participative design workshops, prototype development and evaluation. at the time of writing, the first phase of study has been largely completed and the second phase is about to begin. in the following section of this paper we present the results of a wide-ranging cross-disciplinary review of the intimacy literature. this review has two parts, the first being a theoretical discussion of the prominent and common themes and concepts found throughout the intimacy research literature. we also consider the use of artefacts to mediate intimacy, with a particular focus on interactive technologies. mundane artefacts and existing interactive technologies are considered. in addition, a selection of recent prototypes and exploratory interactive technologies that attempt to mediate family relationships are reviewed. we describe in detail our research approach because of its potential for inspiring and informing (crabtree et al. 2003) the design of interactive domestic technologies. we then present preliminary results from the first phase of our study of current practice, highlighting the key analytic themes that have emerged from our investigation of how interactive technologies are used within intimate relationships. based on these finding, we offer three, broad-brush and indicative suggestions for the design of interactive technologies to support intimacy. what is intimacy? although there are many interesting discussions and perspectives, there are no agreed-upon definitions of intimacy (register and henley 1992, moss and schwebel 1993, robson and robson 1998). in this section, we discuss what constitutes intimacy. we believe a clearer understanding of intimacy is important because it provides a basis for the design of artefacts that better support intimate relationships. with the purpose of understanding intimacy and its contributing factors, we conducted a literature review of conference papers, journal papers and book chapters. this section extracts some of the major themes from this review. according to cheal (1987), an intimate relationship consists of a private world of significant others, which needs to be continuously maintained. in intimate relationships the significant other is often reminded that “they are indeed significant”. people remind each other through gestures, actions and gifts, some of which may be routine and unremarkable. our study provides an opportunity to investigate these exchanges of intimacy. the literature provides a framework or ‘lens’ for conducting the observation. ajis special issue december 2004 104 while the literature provides no unanimous agreement about what constitutes intimacy, some common themes do appear. our review offers eight prominent aspects of intimacy. these are: physical intimacy, non-verbal communication, self-disclosure, presence, cognitive intimacy, affective intimacy, commitment and mutuality. physical intimacy plays a central role in people’s description of their own intimate behaviour (robson and robson 1998) and is acknowledged as an essential aspect of intimate relationships (register and henley 1992; battarbee et al. 2002). physical intimacy is the sharing of physical encounters ranging from close physical proximity to sexual contact (moss and schwebel 1993). physical intimacy is not only concerned with bodily contact. physical intimacy also includes the visceral experience of heightened awareness of ones own body or feelings of new bodily experiences (e.g. butterflies in the stomach, weak at the knees), arising from physical or mediated contact with another (register and henley 1992). non-verbal communication is identified by several researchers as a significant carrier of personal expression in intimate relationships. non-verbal communication is communication by means of actions, gestures, facial mannerisms, close physical proximity or touch (register and henley 1992, battarbee et al. 2002). non-verbal communication is the aspect of intimacy that is better expressed through sensory evocations rather than linguistic forms, and helps to avoid the confusion that is sometimes caused by words (register and henley 1992). self-disclosure is a key characteristic that often differentiates intimate from non-intimate relationships (register and henley 1992, moss and schwebel 1993, robson and robson 1998). selfdisclosure is the act of revealing private information, such as the personal feelings of one person toward another. self-disclosure includes the removal of boundary between oneself and an intimate other (physically and psychologically), getting inside the life of another, and/or allowing another to cross one’s personal boundary (register and henley 1992). self disclosure demands a certain degree of trust in the other and making oneself vulnerable. furthermore, disclosing personal details often leads to an increased level of self-disclosure from an intimate other (duck 1988). thus, selfdisclosure is an effective mechanism for maintaining and changing the level of intimacy in a relationship. disclosing too little or too much can either escalate or de-escalate a relationship (robson and robson 1998). presence is the subjective feeling of another person being present in either a physical and/or a nonphysical manner (register and henley 1992). the feeling of presence can be triggered by symbolic actions of the absent one(s) or the feelings can emerge spontaneously without any (objective) external cause. thus, while being very powerful and contributing strongly to the feeling of intimacy with another, the feeling of another person being present in absence is very complex, subjective, and sometimes highly irrational. a feeling of presence is not so much due to being physically co-located, but due to re-living the pleasure related to being in the company of another person (register and henley 1992). complementing this view, other researchers note that presence goes both ways. they stress the importance of feeling oneself being present to another (in either a physical or a nonphysical manner) for creating and maintaining intimacy (ijsselsteijn et al. 2003, biocca and harms 2002). cognitive intimacy reflects the depth of awareness and knowledge intimates have of one another (moss and schwebel 1993). cognitive intimacy is characterised by feelings of ‘knowing’ the other. intimate friends, dating partners and spouses typically develop deep cognitive understandings of each other, and often share a range of personal information and preferences. this includes knowing ajis special issue december 2004 105 one another’s principles, values, strengths, weaknesses, hopes, fears and idiosyncrasies (altman and taylor 1973). also, it has been shown that increasing the amount of cognitively exchanged information between spouses increases the level of intimacy they experienced. thus being able to establish and maintain a shared cognitive life is a major requirement for building and sustaining an intimate relationship. affective intimacy is the reception and expression of emotion (moss and schwebel 1993). affective intimacy involves a feeling characterized by a deep sense of love, caring, compassion and positive attraction for one another. affective intimacy reflects the depth of awareness intimates have of one another’s emotional world and the emotional exchanges they share. the level of affective closeness in friendships, serious dating relationships or marriage, is commonly reported as closely related to the level of intimacy of that relationship (levinger and senn 1967). also, affective intimacy is often highlighted as a key differentiator between close friendships and relationships involving romantic love (moss and schwebel 1993). commitment is the extent to which partners in a relationship perceive their relationship as ongoing for an indefinite period (chelune et al. 1984). commitment includes acts intended to grow or maintain intimacy. being in a committed relationship generates strong feelings of cohesion and connection (moss and schwebel 1993). commitment is an important foundation for intimate relationships and often a precondition for other aspects of intimacy (e.g. self-disclosure) to flourish (chelune et al. 1984). misunderstandings in the expression of commitment and changes in one partner’s belief in the commitment of the other may impede the growth of an intimate relationship or initiate its decline (chelune et al. 1984, duck 1981). thus, being able to convey and experience commitment constitutes a crucial requirement for building and sustaining an intimate relationship. mutuality is considered the centre of any intimate relationship (cheal 1987, chelune et al. 1984). mutuality is the assumption that intimate partners are co-engaged in a common cause. mutuality originates from a process of exchange, interdependence and reciprocal expectations (moss and schwebel 1993). mutuality exists when gifts or symbolic signs of value are exchanged (cheal 1987). mutuality is characterized by a sense of fairness shared by both partners in relation to the rewards and costs of their interactions (chelune et al. 1984). while mutuality implies joint and shared interactions, it does not necessarily require similar or identical patterns of interaction. rather, intimate relationships may involve both reciprocal interactions (partners showing similar behaviour) and complementary interactions (partners showing different behaviour that complements each other). we believe these eight aspects of intimacy are useful for understanding how intimacy is constituted. however, it is important to note that these components do not exist or work independently, nor do they individually satisfy the intimate experience. indeed, the themes overlap greatly and are highly interrelated (moss and schwebel 1993). the themes discussed above suggest that intimacy includes both a behavioural and an emotional level involving, on one side, actions caused by or causing a feeling of intimacy (such as mutuality, self-disclosure, non-verbal communication or physical intimacy) and, on the other side, feelings of intimacy (such as affective intimacy, cognitive intimacy, presence or commitment) caused by or causing these actions. thus if technology is to support the mediation of an intimate relationship, it needs to facilitate these behaviours and emotions. mediated intimacy people have always used artefacts to mediate their intimate relationships. intimacy has been mediated through symbols of affection such as flowers and rings, through public declarations of love scrawled on park benches, through secret notes and love letters, and more recently through text messages on mobile phones. these ways of mediating close relationships share various combinations ajis special issue december 2004 106 of our eight characteristics of intimacy. flowers are nonverbal, they express emotion (affective intimacy) and they often symbolise gratitude (mutuality). written letters afford self-disclosure, knowledge of personal secrets (cognitive intimacy) and commitment. when one partner returns to the park bench alone, the scratching is a reminder of their absent loved one (presence). it is not surprising then that some recent technologies have been quickly appropriated and added to the repertoire of artefacts for mediating intimacy. notably, the internet is used for matchmaking and exploring options for personal relationships (donn and sherman 2002). furthermore, sms (short message service) is increasingly used to forge new romantic relationships (byrne and findlay 2004), to coordinate activities with intimate friends (grinter and eldrige 2001) and to use text messages as gifts between close others (taylor and harper 2003). there are also an increasing number of prototypes and exploratory technologies that attempt to mediate close relationships between couples and family members. some of these support the often stated need to simply keep-in-touch. rather than using ‘intense’ technologies (such as telephones), researchers are exploring uni-modal technologies (such as a light or movement) and evocative materials to create a feeling of presence-at-a distance. these technologies generate a sense of presence by exploiting peripheral awareness (gaver 2002) and a prior understanding of routines. for example: a feather in a plastic cone floats when the distant partner picks up a picture frame of the couple (gaver 2002); a light ‘orb’ glows in australia when a family member in london walks into their apartment (tollmar and joakim 2002), a special bowl (‘gustbowl’) wobbles when an adult son living away from his parent’s house, arrives home and places his keys in a similar bowl (van der hoog et al. 2004). the wobble, signifying “i’m home”, reassures the concerned parents. traditionally, physical intimacy could only be achieved through physical contact. until recently, mediating physical intimacy over long distance was not technologically possible. however new technologies are exploring notions of telepresence to bridge the barrier of physical distance. for example: two sets of cylinders that roll and rotate in unison as they are manipulated by separated partners (brave and dahley 1997); wearable tokens that transmit touch directly as pressure (hindus et al. 2001) or through vibration (chang et al. 2002); a bed that senses body position and transmits warmth to congruent parts of a lovers bed (goodman and misilim 2003). these devices not only mediate physical intimacy, they also support mutuality; i.e. the interplay or reciprocity of exchanges. these exchanges need not be symmetrical. for example ‘satellites’ (battarbee et al. 2002) consists of two ‘huggable’ balls. one ball captures video and emits sound while the other captures sound and emits video. the asymmetric modalities provoke playful and creative communication. this mutuality, or sense of ‘other’ is generated through negotiating the constraints of the technology. probing intimacy this paper addresses the challenge of investigating intimacy and designing interactive technologies to support intimate acts. our approach is a combination of ethnographic techniques and participatory design approaches that allow us to collaborative explore the phenomenon with a small group of ‘participant-research’ subjects. to this end we have adopted the ‘cultural probes’ techniques developed by gaver et al. (1999) and the equator team (cheverst et al. 2003) and extended them with contextual interviews (beyer and holtzblatt 1998). cultural probes are a novel collection of techniques gaining prominence in interactive systems design. they are particularly suited to investigating people’s everyday life in situations difficult to reach with traditional social science methods such as questionnaires, interviews, focus groups or participant-observation. probes are designed to prompt and elicit information from people about their lives and ‘local culture’ (gaver et al. 1999). in particular, probes are designed to garner an understanding of the playful character of human life and the multifaceted ways people ‘explore, wonder, love, worship, and waste time’ (gaver 2001). cultural probes gather insight from within the site in question, with the full ajis special issue december 2004 107 cooperation and involvement of the participants concerned. the insights are gathered as activities are performed and while technology is in use, thus maintaining ‘fidelity to the phenomenon’ under investigation (crabtree et al. 2003). like deep sea or planetary probes, cultural probes are ‘sent-out’ by researchers and return fragmentary data over time. rather than relying on the presence and intervention of the researcher, cultural probes are designed to encourage and empower subjects to collect data themselves (arnold 2004). this allows the collection of data from situations where researcher presence is problematic by giving participants the means to record everyday activities as they occur or shortly afterwards. it also allows research materials to be collected over longer periods in multiple locations compared with resource intensive methods such as traditional ethnographic approaches. cultural probes we assembled a collection of cultural probes into a ‘probe pack’ (figure 1). the probe pack contained: • two scrapbooks providing the participants with an open format for creative and rich descriptions of both current practice and imagined future technologies • two diaries (one for each partner) allowing the participants to individually describe the temporal flow and routine of their lives throughout the period of study • a digital camera and a photo printer allowing the participants to capture, print, edit and annotate up to 180 photographs for the scrapbooks, diaries and postcards • catchphrases (e.g. “i feel lonely when…” or “i really love it when…”) printed onto sticky labels allowing the participants to stick them into the diaries or scrapbooks provoking reflection by the participants • various consumables including stamped pre-addressed postcards, coloured post-it notes, pens, crayons, paper clips, glue and scissors etc. for use with the scrapbooks. • information for contacting the researchers via landline, voice-mail, e-mail and sms throughout the study. probe packs were given to six couples. participants were asked to use the probes to articulate the role technology played in their relationship. probes were used to elicit: where, when, how and why they interacted; how they felt during interaction; and reflections on the usefulness of current technologies in these situations. figure 1: contents of the probe pack figure 2: scrapbook and diary from participants ajis special issue december 2004 108 the diaries were used individually to record daily communication and interaction activities. this included the form of communication (e.g. sms) and other details such as time, date, location, the content and the feelings associated with it (e.g. urgency or dissatisfaction). the scrapbooks were used collaboratively to tell rich and evocative stories about communicative events and to express the technology wants, desires, likes and dislikes surrounding these events. couples were encouraged to use coloured notes, pens, crayons, glue, photos, magazines clippings, drawings and so forth to form a montage of their intimate lives. the digital camera and printer were used to photograph and print significant events. the docking printer provided the immediacy of traditional polaroid photographs with the convenience of lasting digital images. participants were encouraged to take photographs of everyday artefacts or events that express some important dimension of their interactions: the answering machine at the time of receiving some unexpected news or of a child in a football final to share with an absent parent. participants were asked to print, to annotate and to cut and paste the photographs into their scrapbooks or diaries or postcard, as they saw fit (figure 2). finally, the participants were asked to read and reflect on the catchphrase labels, complete them, and stick them into the diary, scrapbook or on a post card. the stamped postcards could be used for short stories or images to be sent to the researchers or intimate other. it was stressed that nothing in the probe pack was compulsory. the instructions were only suggestions. all the materials could be used in whatever way they wished. no time requirements were made, but it was suggested that the participants should spend about 20-30 minutes each day using the probe pack materials. an additional probe element was introduced midway through the study. this new element consisted of small printed facsimiles of a variety of mobile device screens (e.g. mobile phones and personal digital assistants). this new element served to both refresh participants engagement in the study and encourage them to envision possibilities for future technologies to support their relationship. participants were invited to use these mobile device screen facsimiles to note design ideas and to insert them into the scrapbooks or diaries (see figure 4). contextual interviews our study combined cultural probes with a series of contextual interviews (figure 3). these interviews gave participants the opportunity to explain, clarify and expand upon the materials they had collected and collated through their work with the culture probes. in these interviews, the probes became a starting point for a conversation between researchers and participants that revolved around the participant’s relationship, and the roles played by technologies in mediating (or not) their intimacy. the probes recorded fragments that were used during interviews to prompt memory, seek explanation, and encourage reflection. through these interviews we were able to uncover previously unarticulated aspects of their relationship and intimate behaviours, routines and habits. participants and researchers worked together to develop a shared understanding of how the relationship ‘worked’ and the roles technologies played and didn’t play in mediating their engagement with each other. interviews also allowed regular contact between researchers and participants. these meetings were used to ‘tune’ participant’s engagement with the probe materials. meetings were used to check on participants understanding of the study, to gently steer them toward the intended focus of the study, to ask them to focus more on certain probe activities and to introduce additional tasks. returning from these interviews, researchers brought back a wealth of data that fuelled ongoing discussion, reflection and analysis by the research team. ajis special issue december 2004 109 figure 3: contextual interview with participants figure 4: participant’s children playing with facsimiles participants the study involved six mixed-gender couples in long-term, stable relationships. all couples cohabitated, although work related travel occasionally required periods of separation. the age of participants ranged from late 20’s to late 40’s. three couples had children, ranging in age from 18 months to 12 years of age. all couples used a variety of electronic media such as landline and mobile telephony, email, chat, sms and fax to communicate with each other, although the exact mix of technologies used by each couple varied markedly. procedure the couples were recruited through a screening process involving an initial, informal interview. couples were selected to give diversity in both family situation (no children, very young children, older children); the degree to which they used icts to communicate with one another (once or twice a day up to 20 or more times a day); and types of technologies they used. important in the selection of participants was the degree to which participants were willing and able to articulate, discuss and reflect on their relationship as well as their ability to engage in imaginative speculation about future icts. two researchers were assigned to each couple and were responsible for introducing the cultural probe pack and conducting all contextual interviews with the couple. interviews were predominantly conducted in the participants’ homes, but some interviews were conducted at the university and several were conducted in a café. choice of interview location was at the discretion of participants. the cultural probes were deployed for a period of seven weeks. at the beginning of this period, an initial interview was carried out at the participant’s homes. this included questions about the participants’ backgrounds, their relationship, their communication habits and their use of technology. following the interview, the researchers presented the cultural probe materials and instructed the participants on how to use them. after the first week of the study, the researchers visited the participants’ homes again for a second interview. the purpose of this visit was to ensure the participants were ‘on track’ with the use of the probes and to investigate the activities of the first week through a conversation about the data collected in the diaries and scrapbooks. following the second interview, participants were left to work with the probe pack for three weeks, at which time, a third interview was conducted. the purpose of the third interview was to examine and review the materials collected through the probes since the last interview. researchers and ajis special issue december 2004 110 participants discussed the materials accumulated in the scrapbooks and diaries, and participants were invited and encouraged to clarify, elaborate and reflect on the materials they had recorded and composed over the previous three weeks. at the end of the third interview session, researchers introduced the small printed facsimiles of mobile device screens. participants annotated these facsimiles to explore novel designs in situations where current technology inadequately supports personal communication and interaction (figure 4). after the third interview, the participants were again left to work with their probe materials for three weeks. in the fourth and final interview, researchers discussed the diaries, scrapbook and other materials composed over the previous three weeks. in addition, researchers and participants discussed design ideas produced on the mobile device screen facsimiles. this interview was also used to bring closure to the seven-week process. the researchers retrieved all materials gathered through the cultural probes at this time. it is our intention to return the probe pack diaries and scrapbooks to participants as a ‘gift’ after our analysis of the material is completed (copies will be retained for our records). results the contents of the scrapbooks, diaries and interviews were reviewed and discussed on a weekly basis by the research team. on the basis of our analysis of material collected in the first four weeks of the study, we present some preliminary findings. a theme occurring across all couples was a strong need to support presence in absence. this is a feeling that the other is present, even though they may be far away. this is closely related to the issue of presence discussed earlier. participants identified a desire to be in contact with each other while physically separated. this does not only happen when one partner is away for a long period of time, it also occurs during ordinary workdays, sometimes with only minutes separation. for example, couples who work in office buildings a few blocks apart or couples who work on different floors of the same building are in regular contact, up to 10 times a day, via phone, sms, or email. often the messages play the dual role of organising family affairs (e.g. “who is picking up the children?” or “what is planned for dinner tonight?”) and for declaring a caring presence (“i am here, thinking of you”). other examples of creating presence-in-absence are through indirect, non-verbal communication “by proxy”, for example, one partner preparing breakfast for the other before leaving for work. presence is not only created by exchanges of message. artefacts themselves carry presence. merely carrying a mobile phone, because it affords the opportunity for immediate contact if desired, creates a sense of presence and feelings of security and comfort. furthermore, absence need not be physical. in some instances, couples who were physically nearby but engaged in separate activities supported presence-in-absence by communicating with each other using sms from the lounge room to the bed room or email from one side of the room to the other. in these examples, physical presence did not compensate for emotional absence. the couples felt a desire to be more present than afforded by physical closeness. this desire was mediated by technology. our study suggests that much of the communication that passes between intimates is emotional rather than factual. while intimate communication is not ‘fact free’ it often plays a role more akin to ‘stroking and patting’ than verbal conversation. the messages are often information poor, but laden with significance. examples include spontaneous gift giving, messages of affection, flirtation, telephone calls to chat about nothing in particular. many of these interactions serve to confirm that a person is thinking about, caring for and aware of their partner. it is often the act itself, rather than the explicit message it carries, that is significant. intimate acts are often ambiguous and incomplete, suggesting and hinting rather than explicating in detail. they occur in the context of a rich, shared and sometimes idiosyncratic view of the world. this shared world view is vital to feelings of intimacy and is also a resource artfully drawn on by intimates during interaction. ajis special issue december 2004 111 it comes as no surprise that our study indicates that intimate acts are often private. our results suggest privacy has many dimensions. for example, intimacy entails self-disclosure. intimates reveal something of themselves to each other and they may feel vulnerable if their interactions are exposed to the gaze of outsiders. as a result, intimate acts are often constructed to be hidden from the view of others. participants in our study were selective in the communication media they choose for interaction, and partly based these decisions on their perception of the privacy and security of the various communication channels at their disposal. for example, some did not use work-based email for certain intimate exchanges because they were aware that system administrators can view their emails. similarly, intimate information was not disclosed while on a crowded but quiet train car or over the telephone in an open plan office. some participants, particularly those who regularly used workplace technologies to communicate with each other, reported taking efforts to obscure that they were engaged in communication with their partner. even when intimacy is displayed in public, these acts can be nuanced and imbued with private meaning difficult to see and interpret by outsiders. some participants developed private ‘codes’ and short-hands to communicate with one another, such as an sms of ‘146’ for ‘i love you’ or calling the home telephone and allowing it to ring thrice at certain times of the day to signify ‘i’m awake, i’m ok, i’m thinking of you’. others engaged in flirting in situations were flirting was inappropriate – such as knowingly sending provocative sms messages when their partner was in a work meeting – relying on the personal form of these messages to obscure their lack of proprietary. some participants reported drawing on their shared past and detailed knowledge of one another to privately communicate in the full view of others, using oblique references that were presumably ambiguous or meaningless to others, such as when one end of a telephone conversation could be overheard. more generally, participants often drew on a repository of anecdotes, past conversations, knowledge of events and running jokes in their interactions. indeed, much of what passes between intimates is unsaid and premised on deep knowledge and understanding of one another. actively constructing and maintaining a shared history was also important to participants. all participants collected mementos, photographs and other materials that evoked their past together. some participants invested significant effort in constructing and maintaining these records in scrapbooks and photo albums, and often collaborated in reviewing and updating these repositories. these mementos were at once a public declaration of ongoing commitment and artefacts through which past intimate feeling could be rekindled. with the advent of digital photography, some participants had begun keeping digital archives of photographs burnt to cd-rom for viewing and reviewing on the home dvd player, often as a joint activity. participants were also engaged in a common journey and shared the costs and rewards of lives together. where a shared history brought comfort from the past, a common journey generates hope for the future. both are related to the theme of commitment. features of the participants’ lives, such as raising children, caring for older family members, maintaining the household, or travelling to work, were all enterprises that were shared and, in the sharing, became vehicles for enacting, affirming and maintaining their relationship. joint responsibility was taken for domestic life, including activities such as paying the bills, transporting children and preparing meals. these activities were often coordinated on the fly using interactive technology such as mobile phones. the division of labour within these relationships had regular patterns but was also fluid and renegotiated over the course of a day. in addition, responsibilities also provided occasions for affirming the relationship. for example, the success of a child at school was affirming “our” achievement as “good parents”. even activities that were the sole responsibility of one member of the relationship became joint enterprises. participants reported drawing on their partner’s help and skills for work related tasks such as database development, setting up a web-based email account or writing a job ajis special issue december 2004 112 application. being able to help and share common tasks affirmed the relationship and sense of moving through the world together as a team, rather than as atomised individuals. above all, our participants reported the need to find time to be alone together away from the hectic schedules of family and work related responsibilities that dominated their lives. whether this time alone was found while driving a car with the children asleep, sharing a meal at the end of the day, or even working on separate projects but sharing the same physical space to do so, all felt that physical closeness was crucial to their relationship, and something that could not be adequately replaced or mediated by interactive technology. all our intimate relationships involve expectations and assumptions about reciprocity and exchange between partners. in our study, we found interaction was founded on a commitment from both parties to reciprocate in both content and form. these expectations were nuanced, negotiated and arrived at over many years, yet remain fragile and are occasionally misjudged and misinterpreted leading to misunderstandings and conflict. our study has also taught us that it is important not to romanticise intimacy. intimacy and the tight, emotionally charged bonds it entails, is fragile. while relationships may be robust, misunderstanding, and misinterpretations do occur. when they occur, these breakdowns can have serious repercussions; creating ill will and emotional hurt that can obstruct and undermine intimacy within the relationship. the fallout from a simple breakdown in understanding between partners can reverberate through the relationship for days, if not weeks. for these reasons, any interactive technology designed to support or mediate intimacy needs to mitigate against these forms of breakdown and allow for easy and rapid repair of them when they do occur. even though our results are still preliminary, they have given us a deeper understanding of how some of the themes identified in the literature contribute to intimacy, and how people use technologies to support and sustain them. our challenge is to translate our observations and extended knowledge of the central themes to designs. design ideas we now present three preliminary design ideas that have been distilled from the data. the designs are intended to exploit opportunities for technologically mediating intimacy. the designs are not necessarily futuristic. similar designs may already exist. it is not our intention to implement these design ideas in their current forms. rather we intend to use the design ideas to seed ongoing design activities such as participant design workshops and scenario-based acting out (howard et al. 2002). memorabilia manager: couples spend a great deal of time and energy organising their personal mementos. these are typically photographs and videos, but often include other items such as tickets (e.g. airline ticket from a honeymoon) and souvenirs (a sea shell from that special holiday). these have important value for intimate relationships. they signify common history and a shared journey and contribute to the broad theme of commitment. a ‘memorabilia manager’ should be very portable and allow the experience of creating and reviewing the memorabilia. it would allow couples to compose digital and non-digital forms into a meaningful mosaic. it would be a type of ‘family blog’ that facilitates simple recording of family events. constant touch: face-to-face communication was regarded by all participants as an authentic experience, while mediated interaction as somewhat impoverished, thin and ‘abstracted’. however, when face-to-face interaction was not possible (or practical) intimates desired connectedness and presence. ‘constant touch’ is akin to walkie-talkie or ‘push-to-talk’ devices (telstra 2004). it provides an open channel for constant updating throughout the day. it may contain a single point transducer (such as a light) that is activated when one wants to say “i’m thinking of you”. the form is configurable to suggest a physical presence as if “she is with me all the time”. ajis special issue december 2004 113 family digital assistant: if a family had children, then the children tended to dominate all activities – including intimate ones. children were an anchor to the family’s life and a key mediator for the expression of intimacy. rather than finding intimacy beyond family duties, it was through the routine of child-raising that intimacy was expressed. the ‘family digital assistant’ (fda) acknowledges that shared mundane experiences are part of the intimate experience. where a pda is for personal purposes, a fda coordinates family activities. we observed that poor coordination can lead to emotional hurt and bad tempers. the fda is a response to this need. conclusions intimate relationships involve an intricate and nuanced dance between partners involving a complex array of varied activities. many of these take place in close physical proximity and involve touch and face-to-face communication. others are mediated across space and time and involve activities such as talking on the phone or writing letters and postcards. while mediating intimacy is not a new phenomenon, new technologies influence how, when and why we interact with each other. however, little is known about the adoption and use of these technologies within intimate relationships and even less is known about how, when and why they provide good support for mediating intimacy and how, when and why they fail. perhaps more than the majority of domestic acts, intimacies display fragility when under examination. in this paper we have described an approach to the study of intimacy that stressed the need to empower participants; providing them with means to record intimacies during or soon after the acts themselves and means to describe intimacy in their own words. we have stressed the importance of hearing the ‘voice of the intimates’ given our current lack of understanding of this new domain of hci. although we have largely limited our report of our preliminary findings to the analysis and description of current practice, our approach has also aimed to be playful; we have worked to encourage and legitimise participants’ exploration of possible futures, rather than strictly limiting them to reporting on their current practice. the insights generated through this playful imagining will be the topic of subsequent work. references altman, i. and taylor, d.a. (1973) social penetration: the development of interpersonal relationships. holt, new york. arnold, m. (forthcoming 2004) the connected home: probing the effects and affects of domesticated icts, proceedings of pdc2004, toronto, july 27–31 2004, acm. astra project (2004) http://www.presence-research.org/astra/ (accessed may 7 th 2004) battarbee, k., baerten, n. and loeber, s. (2002) pools and satellites intimacy in the city. proceedings of dis 2002, acm, 237-245. beyer, h. and holtzblatt, k. (1998) contextual design: defining customer-centered systems, morgan kaufmann, san francisco. brave, s. and dahley, a. (1997) intouch: a medium for haptic interpersonal communication. extended abstracts from chi’97, atlanta, ga, usa, acm. byrne, r. and findlay, b. (2004) preference for sms versus telephone calls in initiating romantic relationships. australian journal of emerging technologies and society (issue 2: autumn 2004) biocca, f. and harms, c. (2002) defining and measuring social presence: contribution to the networked minds theory and measure. proceedings of presence 2002, acm. ajis special issue december 2004 114 chang, a., o’modhrain, s., jacob, r., gunther, e. and ishii, h. (2002) comtouch: design of a vibrotactile communication device. proceedings of dis 2002, london, england, acm. cheal, d. (1987) ‘showing them you love them’: gift giving and the dialectic of intimacy. the sociological review, 35(1):150-169. chelune, g.j., robison, j.t. and kommor, m.j. (1984) a cognitive interactional model of intimate relationships. in derlega, v.j. (ed.) (1984) communication, intimacy, and close relationships. academic press, inc., new york. cheverst, k., clarke, k., dewsbury, g., hemmings, t., kember, s., rodden, t. and rouncefield, m. (2003) designing assistive technologies for medication regimes in care settings. international journal of universal access in the information society (uais), 2003. crabtree, a., hemmings, t., rodden, t., cheverst, k., clarke, k., dewsbury, g., hughes, j. and rouncefield, m. (2003) designing with care: adapting cultural probes to inform design in sensitive settings. proceedings of ozchi 2003, brisbane, australia. crabtree, a. (2003) remarks on the social organisation of space and place. homo oeconomicus, xix(4), 591-605. donn, j.e. and sherman, r.c. (2002) attitudes and practices regarding the formation of romantic relationships on the internet. cyberpsychology & behavior, 5(2), 107-123. duck, s. (1988) relating to others. open university press, london. gaver, b. (2002) provocative awareness. computer supported cooperative work, 11: 475-493. gaver, b. (2001) designing for ludic aspects of everyday life, ercim news, 47. http://www.ercim.org/publication/ercim_news/enw47/gaver.html, accessed 16 june 2004. gaver, b., dunne, t. and pacenti, e. (1999) design: cultural probes. interactions, 6(1), 21-29. goodman, e. and misilim, m. (2003) the sensing bed. proceedings of ubicomp 2003, seattle washington. grinter, r.e. and eldrige, m.a. (2001) y do tngrs luv 2 txt msg? proceedings of the seventh european conference on computer-supported cooperative work, ecscw '01, bonn, germany. harper, r. (ed.) (2003) inside the smart home. springer-verlag, london. hindus, d., mainwaring, s., leduc, n., hagstrom, a., and bayley, o. (2001) casablanca: designing social communication devices for the home. proceedings of chi 2001, acm. hindus, d. (1999) the importance of homes in technology research. proceedings of the second international workshop on cooperative buildings (cobuild'99), pittsburgh usa, springer-verlag. hofmeer, k. (1999) the digital hug: keeping families together. interactions, 6(6). howard, s., carroll jennie, vetere, f., murphy, j. and peck, j. (2002) provoking innovation: acting out with contextual scenarios. proceedings of bcs-hci 2002, springer-verlag, london. ijsselsteijn, w., van baren, j. and van lanen, f. (2003) staying in touch – social presence and connectedness through synchronous and asynchronous communication media. proceedings of the 10th international conference on human-computer interaction 2003, crete, greece, lawrence erlbaum associates. levinger, g. and senn, d.j. (1967) disclosure of feelings in marriage. merill-palmer quarterly, 13, 237-249. blythe, m.a., overbeeke, c.j., monk, a.f., and wright, p.c. (eds) (2003) funology: from usability to enjoyment. kluwer, dordrecht. moss, b.f. and schwebel, a.i (1993) marriage and romantic relationships: defining intimacy in romantic relationships. family relations, 42(1): 31-37. register, l. and henley, t. (192) the phenomenology of intimacy. journal of social and personal relationships, 9, 467-481. ajis special issue december 2004 115 robson, d. and robson, m. (1998) intimacy and computer communication. british journal of guidance and counselling, 26(1):33-42. taylor, a., and harper, r. (2003) the gift of the gab?: a design oriented sociology of young people's use of mobiles. computer supported cooperative work, 12(3): 267-296. telstra (2004) what’s doing: telstra completes push to talk trial with services to be available in june http://telstra.com/countrywide/peninsula/localtelstra.asp?page=whatsdoing, accessed 15 june 2004. tollmar, k., and joakim, p. (2002) understanding remote presence. proceedings of nordichi 2002, aarhus, denmark, acm. van der hoog, w., keller, i., and stappers, p.j. (2004) gustbowl: technology supporting affective communication through routine ritual interactions. proceedings of chi 2004, vienna, austria, acm. acknowledgements the authors would like to thank all couples participating in the study and acknowledge the support of the smart internet crc and the danish technical research council (project reference 26-03-0341). copyright jesper kjeldskov, martin r. gibbs, frank vetere, steve howard, sonja pedell, karen mecoles, marcus bunyan © 2004. the authors assign to ozchi and educational and non-profit institutions a non-exclusive licence to use this document for personal use and in courses of instruction provided that the article is used in full and this copyright statement is reproduced. the authors also grant a non-exclusive licence to ozchi to publish this document in full in the conference papers and proceedings. those documents may be published on the world wide web, cd-rom, in printed form, and on mirror sites on the world wide web. any other usage is prohibited without the express permission of the authors. the australasian journal of information systems (ajis) began in 1993 and was based on the simple realisation that there was a need for an academic ‘outlet’ for is research in australia australasian journal of information systems volume 14 number 1 nov 2006 the state of information systems in australian universities tasmanian report dr gail ridley it control research group school of accounting & corporate governance university of tasmania. gail.ridley@utas.edu.au abstract this paper examines information systems at the university of tasmania. the study draws upon a theoretical framework reported earlier. in common with studies conducted elsewhere in the region, this investigation utilised data collected on five themes. the study aimed to characterise information systems at the university of tasmania, as well as to investigate the relationship between the impact of local contingencies on a discipline and its degree of professionalism, within the tasmanian context. data were collected through a qualitative survey with seven influential academics associated with the discipline at the university, and from statistical sources. the findings suggest that an inverse relationship exists between the impact of local factors and the degree of professionalism in this is setting. a surprising finding was that the relationship found varied for research and teaching issues. introduction the current paper reports upon a study of the state of the information systems (is) discipline in the only university in tasmania, and draws upon a theoretical framework outlined in an earlier chapter in this volume. the theoretical construct that guides the study has been presented in the earlier framework paper in this edition and will not be restated. this study forms a sub-study of a wider investigation, comprised of a number of levels. it is one of a series of similar studies undertaken in each australian state and the australian capital territory, and reported upon elsewhere in this volume. tasmania is a unique state in australia as it has only a single university, which is one of the oldest universities in australia. the university of tasmania was established in 1890 in hobart. in 1991 the university merged with the tasmanian state institute of technology in launceston, forming the second campus, in the north of the state. then in 1995 the north-west centre was opened in burnie, and renamed the cradle coast campus in 2005. although the university of tasmania has just over 12,000 students, it offers one of the widest selection of courses in australia (university of tasmania 2005), including 13 within the discipline of information systems. 211 australasian journal of information systems volume 14 number 1 nov 2006 aims of the study this study aims to both characterise the state of the information systems academic discipline in the university of tasmania (utas), and also contribute to a better understanding of the same discipline in universities throughout australia. consequently, the data gathered for the study will be analysed at three levels. table 1 sets out the three levels. the first level of analysis will be reported in this current paper and provides a descriptive examination of is at the university of tasmania from qualitative and quantitative sources. the second level of analysis will also be reported in this paper, and aims to examine in the tasmanian is academic context, the postulation contained within the theoretical framework that an inverse relationship exists between the impact of local contingencies on a discipline and its degree of professionalism. at the third level of analysis, some of the data gathered for the current study will be utilised for a cross-case analysis, and reported elsewhere. the cross-case analysis will draw upon the remainder of the theoretical framework, which contends that a discipline will establish over time with the development of mechanisms of control and a core body of knowledge. this component of the theoretical framework cannot be investigated in a single case, as cross-regional analysis is required on such issues as key is research and teaching topics, and whether a unique symbol set exists for the discipline. moreover, some australian longitudinal data will be needed at this third level. each state study, although guided by the theoretical framework developed for the australian project, will be influenced by the background and interests of the is academics within the state. however, consistent with the content of the other state reports, the following issues have been examined within tasmania is academia: • extent to which is is/was impacted by local contingencies • extent to which is is/was identified as a separate field at the university • the distinctive features of the is curriculum at the university • the distinctive features of is research at the university • perception of the key people in the region who have impacted is at utas covered in this paper level scope 1 descriptive analysis using quantitative & qualitative sources 2 examines whether an inverse relationship exists between impact of local contingencies on discipline & degree of professionalism 3 cross-case analysis drawing upon theoretical framework table 1 three levels of analysis discussed in study 212 australasian journal of information systems volume 14 number 1 nov 2006 background the reader is referred to an extensive review of the literature on the development of disciplines that has been provided in the earlier framework paper of this volume. however, a brief overview of the work of is researchers who have examined the state or development of is in world regions is presented below. little prior research of this kind in an australian setting has been undertaken before, apart from the work of ridley (for example, ridley 1996; ridley, goulding, pervan & lowry 1998), who examined the state of the is discipline in australia, using two quantitative surveys, to find that the discipline at that time was at an early state of maturity. not long after the research by ridley and co-authors was published, related work was undertaken in europe by avgerou, siemer and bjorn-andersen (1999), and in north america by watson, taylor, higgens, kandec and meeks (1999). avgerou et al. (1999) used a survey methodology and found that european is researchers were concerned that their discipline may be absorbed by another. watson et al. (1999) conducted telephone interviews with 17 academic is leaders to uncover some doubt on whether the discipline could maintain or expand its position. the leaders also shared the concern of european is academics about possible absorption by better established disciplines. the methodology used for the current study is presented next. methodology the case study method was used for this study, derived from the approach of yin (2003). an interpretive philosophy was utilised, seeking “rich insight” (walsham 1995). the tasmanian case study was based on a detailed case study protocol developed by the queensland study team members, with input from the study team members from elsewhere in australia. the protocol aimed to bring about comparability across the states, consistency across the individual state case studies and efficiency in data gathering. ethics approval for the study for all state teams in australia was obtained by the queensland university of technology, and later ratified by utas. the primary method used to gather data was a qualitative survey, using an open-ended questionnaire, which was emailed as an attachment to the majority of the participants in november, 2005, and then returned by email. the instrument was piloted in the school of information systems at utas. all participants were first contacted by phone or approached in person, with the exception of one participant who was located outside australia, and another who worked elsewhere at the university, and was no longer associated with the information systems discipline. where a response had not arrived within the timeframe indicated for the return of the questionnaire, contact was again made with the academic, either in person or by email, to encourage them to respond. 213 australasian journal of information systems volume 14 number 1 nov 2006 eight academics who were currently closely involved with is at utas, or had been in the past, were approached to participate in the survey. of these, five were currently involved in either is teaching or research, or both, at the university in the school of is. of the remaining three academics, two had been employed in the information systems discipline at the university in the past, and were working as an is academic outside tasmania, while the third was employed in an allied discipline elsewhere at utas at the time the data were collected. of the participants, four had been employed as a head of school or in a more senior position, either at the time they participated in the study, or in the past, while two had had an extended association with the school of is. all the three participants no longer working with is academics at utas were known internationally in is or an allied discipline. the participants were chosen for their long-standing connection with the school of is at utas and/or because their present or past position, or international links, made it likely that they would have a broad understanding of is. both male and female academics were contacted and responded, but there was a higher proportion of males in both categories. although the author of this paper was also a current member of the school of is at utas, her views were not included in the data collection and care was taken to try to exclude them from the analysis. this was done by having another person trained in is research methods categorise the data after it had been stripped of identifying characteristics, and where the classification differed, agreement was reached after discussion. the survey method was used as some of the participants were located in other states or nations, so that interviewing would have been difficult. an interview was conducted with one of the participants, using the same interview protocol, with the researcher recording written notes. some of the responses took participants up to two hours to complete, and the participants were encouraged to write as much detail as they needed, where they considered the questions relevant. consequently, the survey and interview captured broad perceptions of the state of is at utas, as well as points of differentiation and distinctive characteristics of is at that university. existing statistical and archival material was also collated to supplement the data obtained from the participants, and to facilitate some degree of triangulation. a summary of the statistical information was emailed to those participants currently employed at the school of information systems, with a request to advise where the information required updating. none of the participants indicated that they wanted to modify the summary of the statistical summary. the collated statistical information was not sent to those study participants currently employed outside of the school, as they had less capacity to verify current information about the school. a thematic analysis process was followed, allowing the integration of both the qualitative data collected from participants and the quantitative data collated from other sources (dixon-woods, agarwal, jones, young & sutton 2005). although thematic analysis can be data driven, in this study the themes were driven by those in the theoretical framework referred to earlier. the discussion reported in the findings section below was “weighted towards themes that appear to have a high level of explanatory value”, rather than reflecting how frequently the themes were reported (dixon-woods et al. 2005, p. 47). the notes made during the interview with one participant were recorded in electronic format, and then all the responses for each question were collated by copying the text forwarded by email. the text relating to each question was then examined for themes arising from the framework. the themes, and issues arising from the themes, have been presented below, grouped into the five major categories set out in section 1.1. statistical and archival data have been incorporated into the discussion relating to the themes, where relevant. the discussion of the themes in section 4.0 214 australasian journal of information systems volume 14 number 1 nov 2006 below presents the findings of the descriptive examination of is at utas from qualitative and quantitative sources, and forms the first level of analysis for this study. following discussion of the themes, two tables were prepared to analyse the data at the second level. the second level analysis examines whether an inverse relationship exists between the impact of local contingencies on a discipline and its degree of professionalism, in the tasmanian is academic context. in order to do this, the impact of local contingencies on is teaching and research at utas, and its degree of professionalism, were each independently categorised by two people trained in is research methods as high, medium or low. if the classification varied, agreement was reached after discussion. this analysis allowed an evaluation to be made of the relationship between the impact of local contingencies on is at utas and its degree of professionalism, using the meaning of each term set out in the previous framework paper. however, professionalism was regarded in the theoretical framework as having high task certainty, routinisation of activities and a clear division of labour. where a discipline was not highly professionalised, the control of work process would be decentralised, with limited routinisation of tasks (whitley 1984). after the paper was written, the original participants were invited to comment on it. five participants took up this invitation. analysis and findings: level one section 4.0 first presents demographic characteristics of the respondents, as well as a discussion of the administrative placement of the school of is and the size of the is presence at utas. then the findings and discussion follow, relating to the five common themes listed in the aims of this study. respondent characteristics and is administrative placement and size responses were received from all people contacted, with the exception of one researcher based at the school of information systems. some participants felt unable to respond to some questions, particularly where they no longer worked in the school of information systems. where a participant did respond to a question as relevant, the replies varied in length from two words to several long paragraphs. the longest response to the questionnaire ran to five and one-half pages of single-spaced text. many of the responses to the same question from different participants varied considerably in the way that it was answered. consequently, it can be seen that the replies were far from cursory, and enabled personalised, reflective and deep responses, beyond what was likely to be obtained if all the data had been collected via interview. the demographic characteristics of respondents are shown in table 2: gender academics with phd 5 male 5 female 2 discipline of phd information systems 3 computer science 2 age (estimated) current position 40–50 2 professor 3 51–60 3 assoc prof 2 61–70 2 snr lecturer 1 lecturer 1 215 australasian journal of information systems volume 14 number 1 nov 2006 table 2: respondent demographics utas introduced a major in information technology around 1987. however, it was only around 1992 that a recognisable information systems program was introduced in the university. prior to january 1997, the academic discipline of is was co-located with computing in the department of computer science, in a faculty of science and engineering. the school of information systems was formed in january 1997, and administratively located within a faculty of commerce (from 2006 the faculty of business). the school of information systems is one of four schools located within the faculty, the others being management, accounting and corporate governance, and economics and finance. while the other three schools in the faculty at the hobart campus are physically located together on different floors of a building on the steep slopes of mount nelson, the school of information systems is situated at the other end of the university campus, a 15-minute walk away towards the derwent river. the school moved into a purpose built building erected in 2002, which overlooks the university oval in the foreground, and the river in the middle distance. in 2005 the school delivered is units at six locations: hobart, launceston and the cradle coast campus in tasmania, shanghai and fuzhou in china and jakarta in indonesia. the first three locations are campuses of utas. the other three locations are campuses of other universities, where an agreement is in place for delivery of utas programs. at the time that the data were collected in late 2005, 17 staff members were employed either fulltime or part-time, and based in the school of information systems. of these, 13 were academic (one part-time), while two undertook research positions. the school had three administrative officers (one part-time), and one technical officer employed under a service level agreement with the university. of these staff members, all but eleven academic staff members were based in hobart, two were located in launceston and no school of is staff members were permanently based in the cradle coast campus, or in shanghai, fuzhou or jakarta. in addition, an honorary research associate carried out research and paid casual teaching in hobart, while a recently retired academic undertook considerable casual lecturing for the school from both the launceston and hobart campuses. school of is students based at the cradle coast campus were served either by visiting staff members from hobart or launceston, and/or by videoconference or other remote teaching approaches, particularly webct vista. delivery to shanghai, fuzhou and jakarta was undertaken through occasional block teaching by utas staff, often supplemented by local teaching staff and electronic delivery through webct vista. table 3 sets out the staff and teaching locations. in semester 2, 2005, the school had 1010 students based in the six locations, which represented around 800 eftsus in total. eight undergraduate programs existed at that time, four graduate coursework programs, and a phd program. further coursework graduate programs will be introduced in 2006. the programs on offer at the school of information systems as at the end of 2005 have been set out in appendix 1. utas teaching locations hobart launceston cradle coast shanghai, fuzhou, jakarta academic staff 11 2 0 0 general staff 4 0 0 0 table 3 summary of staffing and teaching centres 216 australasian journal of information systems volume 14 number 1 nov 2006 impact of local contingencies the following section sets out the extent to which is was believed by the survey participants to be impacted by local contingencies at utas. three broad issues are considered, namely the extent to which the is curriculum and research had been affected by local factors, the variation in both by campus, and an examination of how is was affected by local factors when compared against other disciplines at utas. curriculum and teaching issues the multi-campus operation of the university within tasmania, and the associated delivery of is in those three centres, was seen to have come about, at least in part, by the influence of tasmanian politicians. a participant referred to the difficulty in adjusting the staff profile to student demand, which varies by campus, and balancing this issue and others due to funding pressures. course advisory committees were another way that local perspectives had impacted on the courses in the past. these committees which met irregularly were established for the is programs when the school of is commenced in 1997. although course advisory committees had provided feedback on the curriculum of the bachelor of information systems and honours programs, they did not influence their design or development significantly. however the master of information systems (mis) program was both designed and introduced as a response to demand for graduate professional development in is from the tasmanian state government. the mis course advisory committee played a significant role in both the curriculum and delivery modes for that program between 1997 and 1999. in contrast, the bachelor of information systems degree curriculum was strongly influenced by the broader issues of national published curricula, which included is’95 (longnecker, clark, couger, feinstein & clark 1995) and irma (irma 1996). another participant commented on a need for greater involvement of course advisory committees in more recent years to update the curriculum, but noted the introduction of logistics to the undergraduate curriculum in the past few years as a response to external demand. two other respondents saw little evidence of the school’s curriculum responding to local external factors. the latter participants were more recent, or relatively recent arrivals in the state, and had not worked in tasmania at the time that most of the courses had been developed. however, one of these participants noted that the school had responded to international demands in a considerable way, making reference to how the curriculum had changed to accommodate dimia requirements and those of the international market. the nature of the is programs varied considerably, depending on location. traditionally the majority of the school’s students had been located in hobart, while launceston had the second largest proportion of students. few students were based in burnie, and it has not been possible to complete an undergraduate is degree there. however, since the school had started to deliver is into shanghai the distribution of students by campus had changed dramatically. for example, as at november, 2005, 75% of the total eftsu of 800 (1010 students) of the school of is were international students. although some of the international students studied on-shore at tasmanian campuses, 61% of the school’s total eftsu was enrolled off-shore in china. a total of 59% of the school’s eftsu was enrolled in a bachelor of information systems degree in shanghai while 38% were enrolled in a range of programs in hobart, that are listed in appendix 1. all graduate programs were delivered on-shore, although for the first time in 2006, it was anticipated that students from the first cohort to graduate with a utas undergraduate degree from shanghai may undertake an is coursework masters degree based at hobart. 217 australasian journal of information systems volume 14 number 1 nov 2006 in recent years the teaching had become more centralised into the hobart campus. it was not surprising, given the restricted number of programs offered beyond hobart, that only a minority of the units available were delivered outside the hobart campus. a limited number of units were offered at launceston, and less were available at the cradle coast campus. increased use has been made of videoconference and other electronic means for delivery into launceston and the cradle coast campus. however, the teaching focus within an individual unit did not vary by campus. no honours or masters programs were offered in centres outside hobart, although they had been offered in launceston in past years. since the school of information systems had first been established, emphasis had been placed on consistent teaching across the campuses, and moderation of assessment across the multiple campuses in tasmania. this aim was achieved by using a single point of course development for each unit, while employing teaching teams that operated across all campuses. moreover, either single points of assessment were utilised for each unit, or team marking was used that spanned the centres. although some divergence in delivery modes and assessment had occurred in shanghai and fuzhou since the is programs were introduced there in 2003, very similar content had been retained in all units across all centres. research issues the research program has not been centrally controlled within the school, resulting in some variation in the nature of the research conducted within tasmania. for example, there were few common topics between researchers in recent years other than an interest in e-commerce. the latter interest was seen by one participant to be related to the era rather than a consciously adopted focus. another participant made reference to an early emphasis on electronic commerce in the school of is, both from a curriculum and research perspective. the same participant identified that a driver for this emphasis was the establishment of the tasmanian electronic commerce centre, originally located within the school of is in hobart. a comment was made of a perceived need to focus on fewer research areas. a decision to emphasise qualitative research methods taken in the past was still apparent in 2005, with most honours and phd projects using qualitative techniques. however, in the same year, no research was undertaken by the school in the centres outside tasmania. one participant commented that utas did not have a distinct research identity, while several references were made to the need for a greater research culture in the school. this influence and others was seen to act against the development of research collegiality, which also hindered research. many staff were early researchers who were still working towards completing their phd or gaining expertise and recognition within their research area. other participants referred to the lack of qualified supervisors available in the school, and the situation where one or two staff members supervised relatively large numbers of phd students, in particular, dr paul turner. the appointment of staff who had studied within the school led to an emphasis on one research method, which was seen as problematic by one participant. another local contingency to act upon research undertaken by the school of is was the funding obtained by senator harradine and used in part to set up the telstra broadband laboratory. the laboratory became the focus of some of the school’s research for a period. one participant referred to the proposal to amalgamate all faculty research in a single school, while another participant pointed to the role of the faculty of business in identifying research areas. the appointment of the 218 australasian journal of information systems volume 14 number 1 nov 2006 woolworths chair in is in 2004 was described as a consequence of political pressure on the state government to relax shop trading hours. other evidence was provided for the way that the nature of research conducted in the school had been influenced by local contingencies. one participant pointed to the advantages of researching in a state capital with a small population. several references were made to the range of local industry contacts that the school drew upon, with many student projects being based within the community. the small size of tasmania had resulted in federal and state government agencies being relatively easily accessible. in the past both tiers of government have been supportive of research that used their organisations for data collection. as a consequence, the school of is focussed on applied projects with relevance to the industry and government bodies that participated in such research. as another example, the limited number of large private sector organisations in tasmania had directed research efforts to the sme and public sector environments. the minimal extent of manufacturing carried out in the state has also acted to limit research undertaken through the school in this setting. finally, it was claimed that it was more likely to be dealt with by accountants in the state rather than senior it staff, while greater use of consultants and outsourcing in tasmania were reported. all three of the latter characteristics were seen to impact on research undertaken in the school of is. comparison against other disciplines a degree of divergence was seen in the views of participants on whether is was affected by local factors to a greater or lesser degree than for other disciplines at utas. several participants saw is as being affected by local factors to a similar extent as the other disciplines within the university, while another commented that little attention was paid to all computing-related disciplines. two participants commented that in comparison to other disciplines at utas, the volatility of the is discipline meant that there was greater need to maintain the currency of the curriculum and to give students access to current technology, which had resourcing implications. is as a separate field this section considers the extent to which is has been identified as a separate field at utas. first the degree to which is has a separate identity will be considered, followed by a consideration of the factors that distinguish is units and research from those found in the allied fields of business and computer science. then the status of is academics relative to their colleagues in business and computer science will be examined, before finally a comparison will be made between the terminology used by is academics at utas against that used by their business and computer science colleagues. extent to which is has a separate identity the introduction of is within the department of computer science at utas was an initiative of dr michael rees. after his departure, dr chris keen (now professor keen) took over the role of is advocate. since then is has had a separate identity in the faculty of business, and communication with computing has been reduced, which, according to one participant, has been to the loss of both schools. 219 australasian journal of information systems volume 14 number 1 nov 2006 although is at utas was seen by the participants to have a separate identity, this characteristic had taken over ten years to achieve. information systems was formally acknowledged as a single field of study and research within the university since january 1997 with the formation of the school of is. from about 1994 information systems was mainly associated with a group operating within the department of computer science on the hobart campus. towards the end of that year is was formally identified as a distinct teaching field within the department of computer science. however, there was also a smaller is teaching and research group within the department of applied computing at launceston. referring to the time in which is was establishing itself at utas, a participant who had been associated with the school for many years commented that most discipline areas at the university were unclear about the nature of the field. another participant now employed outside tasmania saw relevance as the key for survival where confusion existed about the nature of the discipline, and both opportunities and the professional community were small. distinguishing is from business and computer science in 1996 extensive discussion occurred between the staff of the proposed school of information systems and what were at that time the departments of computer science and applied computing. a participant reported that teaching and research topics were readily grouped into those suggested at that time by the australian computer society. the topics were divided into the technical aspects of computer programming, software engineering, networking, computer systems engineering and computer science, and for is, the business and social aspects of the application of icts. the same distinction has largely continued until 2005. in general the participants saw computing and is as having different perspectives, with limited recent communication between the two disciplines. however, they saw less to distinguish is from business, with is being concerned with the management of it and not as broad-based as business. the is units offered limited technical content, which is found in the school of computing. one participant saw pressure on computing schools to shift towards information systems teaching areas, in response to falling student enrolments. as evidence of this claim, database management has been introduced by the school of computing in some units in recent years, delivered by their own staff members. one participant saw the demise of service teaching to be partly responsible for this last development. one participant pointed to current pressures that act against is groups continuing as independent schools at utas and elsewhere, suggesting that there is potential for is to be amalgamated with other disciplines, in particular from within business, or computing. comparative status of is academics four participants believed that is academics had a similar status to business and computing academics at utas. a comment was made that while is and business academics have the same status, academics from schools situated within the faculty of science and technology have higher status, due to the better research profile in that faculty at the university. the school of computing is located in the faculty of science and technology at utas. however, another participant compared the size of the schools of computing and is when considering the status of each. both the schools of computing and is had similar eftsu numbers as at late 2005, with the former being the third largest school and the latter the fourth largest school in the university at that time. finally, another participant considered is academics to have a lower status than their colleagues in computing or management at utas. 220 australasian journal of information systems volume 14 number 1 nov 2006 comparative use of terminology most of the participants who responded to this question considered that is terminology used at utas would be broadly familiar to their business and computing colleagues. however, one person believed that there was relatively little understanding of the conceptual base of is among many staff in business. another noted the limited contact between is and computing staff, which, it is assumed, would make it difficult to assess the commonality in the terminology used by each group. features of is curriculum and teaching in the next section, the is curriculum and teaching at utas will be characterised. in order to do so, the place of is service teaching will be considered, followed by an examination of whether any distinctive themes, teaching styles, tools, techniques or technologies are used for teaching is at utas, and other teaching issues. finally any future plans for teaching is at utas are examined. role of is service teaching as the school of is does not strongly differentiate between service and core teaching, there is less service teaching undertaken than in some other universities. for example, the same undergraduate is units are offered within the bachelor of information systems degree and its associated combined degrees, as are available across all other degrees offered by the university. a number of students undertake is units within business, economics, science, computing, arts and geomatics degrees. is degrees account for approximately 80% of the teaching load while other degrees account for approximately 20%. what is commonly regarded as service teaching only occurs with the provision of two is units within the geomatics degree. a pressure that acts against service teaching occurring is the motivation of other disciplines to retain funding, and, according to one participant, the limited value placed on the philosophies of other schools. there is no core is unit in the university’s mba program, which was seen as worthy of note. distinctive themes and teaching styles a focus on management and strategy was identified in the is teaching at utas, with less emphasis on technical issues. this focus incorporated the major themes of project management, data modelling, data management and electronic commerce. oracle database management is taught to oracle certification level. oracle is taught because of its use in the tasmanian government, a major employer of is graduates. at the undergraduate level there are three majors: systems development; e-business and management of is. the approach taken in earlier years at the school when professional development was integrated into the undergraduate teaching program was seen as distinctive. however, this focus had waned more recently. the business logistics unit taught at undergraduate level was also identified as distinctive. one participant commented that the is teaching program at utas was not significantly different from the range of topics taught elsewhere in australia. relatively few suggestions were made regarding distinctive features of is teaching at utas, while one participant stated that no features of the teaching program were distinctive. however, mention was made of the significant use of discussion workshops and case studies in the school. reference was also made to the individual attention provided to students, which was perceived to foster their 221 australasian journal of information systems volume 14 number 1 nov 2006 confidence and encourage them to continue on to higher degrees. another participant commented on the high quality of the flexible online support that was provided to support the teaching program. distinctive tools, techniques, technologies although one participant commented that there were no tools, techniques or technologies that were distinctive in the is teaching at utas, some techniques associated with teaching delivery were identified as distinctive, including what was perceived as an effective combination of face-to-face and online delivery. reference was made to the sequential delivery style for units from year 1 to 3, designed to facilitate the progression of student responsibility for their own learning. for example, in the later semesters of a degree many units de-emphasised lectures to communicate knowledge, and placed more reliance on online resources with consolidation of the knowledge in workshops. other teaching issues a range of teaching related issues were raised by the academics as important. declining local enrolments was one, and the associated very high international load was another. one participant linked declining enrolments to a perceived falling interest of young people in it, and a negative image for the discipline. reference was also made to the associated unbalanced distribution of the teaching program across six centres, particularly as the majority of the school’s students were located in china as at late 2005. several participants commented on the need to monitor that the programs delivered by the school of is remained relevant to current needs and trends. in particular, several participants remarked on the need to ensure that the school’s is graduates have access to technical skills expected from employers. it was noted that in past years the school’s staff had included people with expertise in advanced databases and networking, but more management oriented staff had been appointed since that time. very recently an attempt has been made to correct this perceived imbalance through recruitment of staff with specialist technical skills. one participant no longer working in the discipline suggested the school should consider a greater degree of re-integration with computer science. changes planned for teaching/curriculum in the next 3 years finally, few changes to the nature of the is teaching or the curriculum over the coming three years were identified. however, the new masters programs being introduced from 2006 were raised: mebusiness, me-business with a specialisation and m-logistics. another participant foreshadowed a move for the school in future years away from it and systems development, to operations management and a business process focus that will require greater coverage of business analysis methodologies. this change was regarded as a necessary response to falling enrolments. 222 australasian journal of information systems volume 14 number 1 nov 2006 features of is research the features of is research at utas are discussed below. this section considers research output and funding, the balance between, and incentives for, research and teaching, conference attendance and planned future changes for research conducted at the school of is. research output and funding although there was general recognition that the research output of school staff was relatively low, several participants identified that the contribution was skewed. the majority of publications were in refereed conferences, both national and international, with a limited number of quality journal publications. one participant no longer working at utas saw an opportunity for researchers to target high quality conferences and journals, in order to obtain quality feedback. for this to happen it was identified that broadly accepted academic leadership and a unifying research culture would be required, enabling the development of research groups and the emergence of research strengths for the school. one participant believed that in the late 1990s the school’s research focussed on rigorous interpretive research methodologies. at that time it produced a series of quality phd theses, evidenced by comments from staff from other schools in australia, and the later success of some of those phd graduates. the school did not have a strong history of attracting arc discovery funding. the majority of the research funding obtained by the school of information systems in recent years has come from external, national competitive grants from the smart internet and sustainable tourism crcs, and arc linkage grants. a limited number of the school’s staff had been involved in attracting these research funds. more recently one staff member in particular had attracted considerable industry funding to the school, particularly in the area of e-health. small internal research grants have been made available by the faculty of business, while the university-wide institutional research grants scheme is designed to prepare staff for the future submission of arc grants. balance between research and teaching and incentives for each as teaching loads were high, participants perceived that pressure existed to push the balance away from research towards teaching. some staff had a teaching and administrative load only, while several staff members had a 75–85% research load. however the majority of staff was expected to undertake teaching, research and administration. some participants perceived greater recognition was given for good teaching than for research at the university through processes such as teaching merit certificates, teaching development grants and evaluation of teaching and learning. however, at a faculty level there has been a recent attempt to redress this through the introduction of research awards. faculty funding is available to assist staff to travel to conferences, which needs to be combined with school professional development funding. a faculty scheme of seed funding to encourage a range of different forms of research was introduced several years ago. a comment was made that incentives for the conduct of quality research could be better defined and implemented. for example, the same participant perceived that highly productive research staff did not receive recognition, or incentives through the distribution of 223 australasian journal of information systems volume 14 number 1 nov 2006 research quantum income. the same participant considered that the dest point system to be ineffective in challenging staff to undertake better research. however, the new approach of the commonwealth government to research outcomes in australian universities may redress this situation. research and teaching targets for individual staff are set within the compulsory performance management scheme implemented at utas. promotion is linked to good research outcomes, and more recently, an alternative route to promotion has been introduced through the demonstration of quality teaching. conference attendance it was believed that active researchers in the school would attend on average one to two conferences a year. in the past the australasian conference of is (acis) has been popular with staff and research students because it provides researchers with opportunities to network. however, one participant commented that attendance at acis by school staff had declined in recent years. icis, ecis, hicss, gitma, irma and other topic specific national or regional conferences have been attended occasionally by at least one staff member, while healthcare and security conferences have been popular. changes planned for is research over the next 3 years finally, few comments were made on changes planned for research in the school of is in coming years. associated with the relatively recent appointment of a woolworths chair in is at the school, several participants referred to a new focus on it governance and business process management. this move was seen as having potential to benefit the school, and several comments related to a desire for the emergence of a collegial research culture. however, while a senior participant foreshadowed a limited research group culture for the future, based on the belief that it is not possible to “bureaucratise is research”, another saw progress of the is discipline at utas as being linked to the future research performance of the school. key people who have impacted is the last section in the analysis and findings for level one considers key people who have impacted is at utas. while a range of individuals was identified as having impacted is at utas, only two were mentioned by more than one participant. professor chris keen was the individual mentioned most frequently. he was seen to be responsible for the inception of the school of information systems, and acknowledgement was made of his considerable entrepreneurial skills and vision. reference was also made to his influence on some of the current staff members as their supervisor while they completed their phd. the role of professor arthur sale was also acknowledged for recognising the need to introduce a more applied course in ict at utas when he was head of the department of computer science. other individuals were mentioned once by participants. jeremy firth and stephen haynes from the tasmanian public sector were described as strong advocates for an is program at utas in the 1990s. reference was also made to the contribution made by cathy urquhart and her role in the 224 australasian journal of information systems volume 14 number 1 nov 2006 development of the first is course at utas, and john lamp, for his development of a web-based service for the australasian and international is community. bob godfrey’s role within the australian computer society and his involvement with the development of is curricula was also acknowledged. more recent contributions that were recognised included paul turner’s strong engagement with industry, peters marshall’s focus on research and roy barkas’s role within logica cmg in hiring is graduates and providing assistance with research. section 4 has presented the findings for level one, providing a descriptive examination of is at utas, drawing upon qualitative and quantitative sources. this section first considered the demographic characteristics of the respondents, the administrative placement of the school of is and the size of the is presence at utas. then the findings and discussion were presented for the five themes common to the reports from each region in australia. the following section sets out the analysis and findings for level two of this study, and examines whether an inverse relationship could be identified between the impact of local contingencies on the is discipline, and the degree of professionalism, within the utas setting. analysis and findings: level two figure 1b of the theoretical framework guiding the study postulated an inverse relationship between the impact of local contingencies on the is discipline, and its degree of professionalism. the reader is referred to the earlier chapter in this volume on the study’s theoretical framework for an extended explanation of these two terms, and the relationship between them. the impact of local contingencies at the school of is at utas was evaluated by categorising the level one data relating to both research and teaching issues, including the variation in both by centre, into high, medium or low impact. non-local factors were omitted. where the categorisation by the second researcher differed, discussion took place until agreement was reached. both qualitative and quantitative data were considered, where relevant. this analysis is presented below. area high medium low is curriculum & teaching issues 3-campus delivery of is in tasmania influenced by politicians oracle taught because of its use in the tasmanian government, the major employer of is graduates funding pressures make adjusting staff profile to student demand difficult role of course advisory committees, particularly for mis degree increased use of videoconf, webct for teaching nonhobart campuses some variation in delivery to shanghai & fuzhou teaching focus does not vary by campus moderation undertaken across multiple campuses team marking across campuses single points of course development for each unit very similar unit content across all centres teaching fairly centralised in hobart 225 australasian journal of information systems volume 14 number 1 nov 2006 is research issues research not centrally controlled few common topics early emphasis on electronic commerce (ec), influenced by tasmanian ec centre some research involvement in telstra broadband lab, enabled through funding from senator harradine it sometimes dealt with locally by accountants; greater use of consultants & outsourcing small size of state means easy access to supportive govt agencies limited large private sector & manufacturing organisations has emphasised sme & govt research limited research culture appointment of woolworths chair in is associated with political pressure on state government limited phd supervisors no research conducted in school centres outside tasmania common interest in electronic commerce in previous years; diverging justification for this provided emphasis on qualitative methods, but positive approaches used & taught faculty influence on research topics & structure table 4 classification of impact of local contingencies on research and curriculum/teaching overall, it can be seen from table 4 that local contingencies had a low to medium impact on curriculum and teaching issues, while they had a high to medium impact on research issues. research was particularly influenced by local factors, with no issues recorded in the low range, and few in the medium range. next, an analysis of the extent of professionalism of the is discipline at utas follows. the degree of professionalism for both research and teaching issues at the school of is at hobart was also evaluated using the level one data, employing the same method used for the evaluation of the impact of local contingencies. professionalism requires high task certainty, routinisation of activities and a clear division of labour. table 5, below, suggests that is curriculum and teaching activities had a high to medium level of professionalism at utas, while research activities had a low to medium degree of professionalism. 226 australasian journal of information systems volume 14 number 1 nov 2006 area high medium low is curriculum & teaching issues relatively centralised location of staff in hbt aids task certainty through opportunities for communication moderation of teaching across campuses similar teaching content across campuses high quality flexible online support is an example of clear division of labour in this area widespread use of workshops and case studies in teaching range of mechanisms exist to promote good teaching sequential delivery style used to develop student responsibility from years 1 to 3 focus on management & strategy in teaching four major teaching themes identified staff performance management scheme sets teaching targets little emphasis on service teaching means greater routinisation across degrees oracle certification is an example of high task certainty is research issues some emphasis on qualitative research methods perceived fewer incentives for research than for teaching some incentives provided by faculty/university to do research some emphasis on sme & govt research new focus on it governance, business process management majority of staff expected to undertake research research program not centrally controlled few common topics research culture hard to sustain, detracting from routinisation of research perceived lack of recognition for productive research staff limited history in attracting arc funds less attendance at conferences than in past perceived need to focus on fewer research areas limited quality journal publications reduces task certainty for publishing in these journals high teaching loads leaves little time to routinise research table 5 classification of features of is research and curriculum and teaching at utas by degree of professionalism 227 australasian journal of information systems volume 14 number 1 nov 2006 now that the level two analysis has been undertaken, the conclusions of this study will be considered next. conclusions this final section reviews the findings, and presents the study’s limitations and suggestions for future research. in the descriptive analysis of the school of is at the university of tasmania, the extent to which is was impacted by local factors, whether it was perceived to be a separate field at the university, the distinctive characteristics of both its curriculum and research and the key people who had impacted is at utas were considered. then an analysis was undertaken of the perceived degree of impact of local contingencies and the degree of professionalism for both teaching and research for is at utas. the level two analysis suggests that while local contingencies had a low to medium impact on curriculum and teaching, at the same time they also had a high to medium impact on research issues. however, there were indications that the reverse was the case for the degree of professionalism, in that curriculum and teaching issues had a high level of professionalism while research activities were assessed as low to medium. these findings for research issues are consistent with the view of is as a fragmented adhocracy, as explained in the framework paper earlier in this volume. however, the different findings for curriculum and teaching issues have not, it is believed, been identified before for is, and make a contribution to research on the development of this discipline. the level two findings are consistent with the postulation from the theoretical framework that an inverse relationship exists between the impact of local contingencies on a discipline and its degree of professionalism. however, as the findings were derived from a single case study, the inverse relationship found is best confirmed by case studies conducted in other areas of australia and elsewhere. moreover, if the findings are confirmed in different regions using different methods of analysis, this characteristic will act to strengthen the findings. additional future work planned is to examine the remainder of the theoretical framework developed for the study through a cross-case analysis of how the components of the is discipline developed over time. it is possible that the framework may need modification as a result of its application to further cases. once these level three findings are known, comparison can be made with the european and north american studies to see if the common findings of those two studies extend to australia at least six years later. 228 australasian journal of information systems volume 14 number 1 nov 2006 references avgerou, c., siemer j., & bjorn-andersen, n. (1999) “the academic field of is in europe”, european journal of information systems, vol 8 no 2, pp 136–153. dixon-woods, m., agarwal, s., jones, d., young, b., & sutton, a. (2005) “synthesising qualitative and quantitative evidence: a review of possible methods”, journal of health services research & policy, vol 10 no 1, pp 45–53. irma (1996) the information resources management curriculum model: an international curriculum model for a 4 year undergraduate program in irm. a joint activity of irma and dama, irma, harrisburg, pa. longnecker, h., clark, j., couger, d. feinstein, d. & clark, j. (1995) is’95: model curriculum and guidelines for undergraduate degree programs in information. a joint activity of dpma, acm, icis and ais, school of cis, university of south alabama, mobile, al. ridley, g., goulding, p., pervan, g., & lowry, g. (1998) “the australian information systems research community: an analysis of mainstream publication outlets”, australian journal of information systems, vol 5 no 2, pp. 69–80. ridley, g. (1996) “establishing australian is research traditions: identifying appropriate quality publication outlets and the significance of conference publications”, proceedings of the 7th australasian conference on information systems, hobart, vol 2, pp 567–578. ridley, g. (2006) “characterising information systems in australia: a theoretical framework”, australasian journal of information systems, vol 13 no 3, pp. 38-60. university of tasmania (1996) welcome by the vice-chancellor, available: http://www.utas.edu.au/uni/welcome.html accessed 22 november 2005. yin, r. (2003) applications of case study research (2nd edn), sage publications, newbury park, ca. walsham, g. (1995) “interpretive case studies in is research: nature and method”, european journal of information systems, vol 4 no 2, pp 74–81. watson, h., taylor, k., higgens, g., kadec, c. & meeks, m. (1999) “leaders assess the current state of the academic is discipline”, communications of the ais, vol 2. whitley, r. (1984) the intellectual and social organization of the sciences, clarendon press, oxford, uk. the author dr gail ridley is employed as a lecturer in the school of accounting and corporate governance within the faculty of business at the university of tasmania in australia. she is based at the hobart campus. her teaching interests include strategic information systems, information systems research methods, systems development methodologies and accounting information systems. her research interests include information systems research methods, it governance and it control frameworks. gail has published in both international and australian journals and conferences, and has worked as a consultant to government in the strategic information systems domain. 229 http://www.utas.edu.au/uni/welcome.html australasian journal of information systems volume 14 number 1 nov 2006 appendix 1 programs offered at the school of information systems, university of tasmania, as at semester 2, 2005 undergraduate bis * bis/bbus bis(hons) bis/bmusic bis/bteaching bis/blaws bfa/bis bis/dipit*** pg coursework mis gdis gdinfomgt ** gcis rhd phd * in 3 specialisations: management of information systems, electronic business, systems development ** in 3 specialisations: general librarianship, teaching-librarianship, information management *** the dipit component is delivered by tafe 230 respondent characteristics and is administrative placement and size curriculum and teaching issues comparison against other disciplines extent to which is has a separate identity distinguishing is from business and computer science comparative status of is academics comparative use of terminology role of is service teaching distinctive themes and teaching styles distinctive tools, techniques, technologies other teaching issues changes planned for teaching/curriculum in the next 3 years research output and funding balance between research and teaching and incentives for each conference attendance changes planned for is research over the next 3 years area high medium low is curriculum & teaching issues is research issues the author undergraduate pg coursework rhd microsoft word ajis vol13-2 o.doc 183 australasian journal of information systems volume 13 number 2 may 2006 computer simulations, disclosure and duty of care john barlow school of arts and sciences (nsw) australian catholic university strathfield, nsw, australia abstract computer simulations provide cost effective methods for manipulating and modeling 'reality'. however they are not real. they are imitations of a system or event, real or fabricated, and as such mimic, duplicate or represent that system or event. the degree to which a computer simulation aligns with and reproduces the ‘reality’ of the system or event it attempts to mimic or duplicate depends upon many factors including the efficiency of the simulation algorithm, the processing power of the computer hardware used to run the simulation model, and the expertise, assumptions and prejudices of those concerned with designing, implementing and interpreting the simulation output. computer simulations in particular are increasingly replacing physical experimentation in many disciplines, and as a consequence, are used to underpin quite significant decision-making which may impact on ‘innocent’ third parties. in this context, this paper examines two interrelated issues: firstly, how much and what kind of information should a simulation builder be required to disclose to potential users of the simulation? secondly, what are the implications for a decision-maker who acts on the basis of their interpretation of a simulation output without any reference to its veracity, which may in turn comprise the safety of other parties? keywords: modeling and simulation, ethics, disclosure, duty of care, responsibility. introduction in general terms a model may be described as a representation of the connected variables, or conditions, that are assumed to be operating within a system or an event while a simulation may be described as an attempted reproduction, using a model, of the conditions that define a system or event. however, in many instances though, both terms are used to mean the same thing. there are numerous types of models which are used for different purposes. in science, for example, barbour (1974, pp 29 – 30) suggests four: experimental models which can be constructed and used, logical models which are used to illustrate abstract systems, mathematical models which are generally symbolic representations of quantitative variables, and theoretical models which are described as “imaginative mental constructs invented to account for observed phenomenon.” all these models can be used as the basis for building computer simulations. computer-based simulations may, at one level, involve the use of 'unsophisticated' spreadsheet and database applications to simulate and predict events that can, in the main, be conveniently modeled using mathematical relationships. for many people however, a computer-based simulation often means a multimedia application, such as a virtual reality application which provides them with the opportunity to 'experience' various situations, using the computational power of modern computer australasian journal of information systems volume 13 number 2 may 2006 184 platforms, and their associated user and graphical interfaces, to generate interactive images of ‘very real’ situations. numerous authors have provided definitions of computer-based modeling and simulation, together with descriptions of where they are used in various human activities. essentially there are a number of themes that run through these definitions, the most notable of which is the notion that simulations provide cost effective methods for manipulating and modeling 'reality'. there are many instances that illustrate this view. for example, architectural design simulations, which permit prospective buyers to walk through a new home design, are now frequently used as marketing tools. metallurgists are cautiously hopeful that the combination of more sophisticated simulation algorithms combined with increased computer processing power will further enhance their understanding of crack propagation in metals. in reality, computer-based simulations are utilised in just about every human endeavour including playing games for pleasure and entertainment, modeling courtroom evidence and complex experimental data, graphing philosophical paradoxes, and providing sophisticated modeling mechanisms for predicting the weather! in addition simulations are used in many application areas, most notably defence applications where, for example, military scenarios can be modelled and evaluated without the need to subject military personnel to real and potentially hazardous combat conditions. markovsky and jin (2002, p 1), for example, suggest a model is a “set of variables and the functional relationship among them. … a model contains at least one independent variable and one dependent variable, where a change in the former is linked to change [or an outcome] in the latter”, while a computer simulation is a “program whose variables and functional relationships are intended by its author to map onto empirical phenomenon and processes.” shelly, cashman & vermaat (2002, p 6.43), suggest computer simulations are computer-based models of real-life situations. computer applications of models and simulations are used to support computer-aided decision-making associated with such activities as design, testing, planning, and control. ören (2000, p 166) suggests simulation can be defined as goal-driven experimentation with dynamic models. he argues that the reasons for using simulation vary with the type of area of study and briefly sets out a number of examples. these include analysis problems where simulation is used to provide insight, continuous simulation used as part of the design process (particularly in engineering design), and discrete simulation used in association with business applications. other instances include artificial intelligence applications used in simulation software, autonomous software agents, internet simulations using local, or network computational platforms, as well as defence applications as noted above. singh (1996, pp 560-561), has adopted the combined term, 'simulation model', and defines it as a concise framework for the analysis and understanding of a system that facilitates imitating the behaviour of the system over time. he asserts that simulation-modeling techniques are powerful and cost effective tools for manipulating time, system inputs, and the logic combinations associated with complex systems. moreover, when combined with visual animation equipment simulation modeling techniques provide an efficient means of learning about, experimenting with, and analysing real-life complex systems. one of the important considerations of singh’s definition relates to the observation that simulation is synonymous with imitation, and while his work is strictly focused in the context of complex computer integrated manufacturing and design systems, it has merit in the wider context of computer modeling and simulation. it is an important observation in that it underlines the important, if not cautionary point that computer simulations are not real, they are imitations of a system or event, real or fabricated, and as such mimic, duplicate or represent that system or event. this point is supported 185 australasian journal of information systems volume 13 number 2 may 2006 by markovsky and jin (2002, p 2) when they assert that computer simulations are “at best virtual experiments yielding virtual data”. the degree to which a computer simulation actually aligns with and reproduces the ‘reality’ of the system or event it is attempting to mimic or duplicate is dependent upon many factors. these factors include, but are not limited to, the efficiency of the simulation algorithms, the processing power of the computer hardware used to run the simulation model, and most importantly the expertise, assumptions and prejudices of those concerned with designing, implementing, and interpreting the output of models and simulations. in this context, boumans (1999, p 93) argues that models in particular integrate more than just theory and data. that is, the veracity of a model, and hence any simulation derived from it, is dependent upon a complicated amalgamation and integration of many competing influences, including theoretical notions, mathematical concepts, mathematical techniques, stylized facts, empirical data, policy views, analogies, and metaphors. moreover, it could be argued that these competing influences can impact on a model builder’s deliberations at both a conscious and perhaps more crucially, an unconscious level, particularly where the focus of the model and simulation building activity is restricted to demonstrating and applying ‘intellectual’ skills and knowledge. this implies the need for a more considered and reflective approach to the use of models and simulations more generally. codes of professional ethics and professional practice codes of professional ethics and professional practice of various organisations provide a set of guidelines for the conduct of their members. the two codes are notionally differentiated by their intent. codes of ethics are intended to provide fundamental guidance, which should permeate the conduct of an organisation’s members, while the code of practice is designed to provide more specific guidelines regarding acceptable standards of professional practice. essentially the development and adoption of codes of ethics and practice by organised groups reflect their desire, in part, to be seen and accepted as ‘professional’ entities in the wider community, and their member's as 'experts' in their chosen field. the expanding importance and scope of the various application areas of modelling and simulation has prompted the call for the establishment of a professional code of ethics for those involved in the development of computer-based simulations. mulvey (1994, p 54) argued in the context of computer modeling that since the actions of modelers can impact on others “codes of ethical conduct represent attempts to reduce harm when professionals carry out their business”. this view is supported by ören (2002a) who has suggested that, given the expanding importance of their work, simulators are now obliged to reflect on the professional and ethical implications of their work. ören (2002b) has asserted that a code of professional ethics should be formulated and adopted for simulators since ethics was the missing link between serious validation as well as verification studies of modelling and simulation. he argued that since simulation studies could affect numerous people as well as the environment in many different ways, those involved in any aspect of it should reflect upon their responsibilities. further, simulations are used in many disciplines, such as research, scientific and engineering applications, and business, where specific codes of professional ethics and practice already exist and as such simulators should, at the very least, be aware of such codes. he also suggested that various simulation societies should provide leadership in urging their members to adopt a code of professional ethics so that their members “can show the acceptance of their responsibilities and accountabilities.” a code of professional ethics for simulationists (simethics) has since been established by the society for modeling and simulation international. this is a voluntary code and is no doubt an important development given the nature of the interaction between computers, simulators, computer australasian journal of information systems volume 13 number 2 may 2006 186 applications, and the general public. ferguson (1992) [quoted by beder (1998, p 49)], for example, asserts that commercial software programs are becoming increasingly available for all sorts of design problems. these programs, argues ferguson, offer ‘illusions of certainty’ without reducing the need for human judgment requiring ‘intimate, first hand internalized knowledge’ of the technology being used and the impact that this might have on product outcomes, a view supported by girill (2002). the real issue of course in this instance is how such codes can be implemented and used to guide the professional practice of simulators. one issue worth exploring in this context is the need for the disclosure of information about a simulation and the associated duty of care implications. disclosure and duty of care the need for the disclosure of information associated with simulations is an emerging issue. that is, how much information should a simulator be compelled to inform their colleagues and potential users, both from within the professional and public domains, about the limitations and assumptions underpinning the development of a simulation? the quality of a simulation is very much dependent upon the quality of the model it has been developed from, which implies of course that it is fundamentally dependent upon the quality of the data used to develop and test both the model and simulation. in addition, there is also an argument relating to the right, or perhaps even more importantly responsibility, of a simulation user to know what the simulation package is actually designed to model, and how it should be interpreted and applied. this should include information about the context, circumstances, and design constraints associated with the development of the model, the range and type of testing data used to verify the validity and reliability of the simulation, as well as information indicating the intended applications of the simulation. girill (1999), for example, has noted, “software trends already underway make computer documentation the next major venue for the appropriate disclosure of relevant facts that once might have remained hidden.” he observed that sophisticated computer simulations in particular are increasingly replacing physical experimentation in many disciplines, such as engineering and medicine, and as a consequence are used to underpin quite significant decision-making. in essence the decision-maker is increasingly vulnerable to the quality and appropriateness of the assumptions the software programmer has made when developing the simulation. girill (1999) makes an important point when he asserts: disclosing hidden software assumptions, and spelling out their implications for the output of simulation runs, will have a major impact on how astutely engineers and physicians can make good judgements based on simulated, rather than physical analysis. users, and in particular uncritical users and third parties such as members of the ‘public’ not directly involved in the building, implementation and interpretation of models and simulations, are very much dependent upon the competence and professional expertise of simulation builders and users. these individuals would probably have not seen it necessary, or perhaps not even been aware that they should have, carried out some form of risk assessment in relation to the model, the simulation, the output, and the interpretation of the output by ‘experts’. they are thus forced into a trust relationship between themselves and the expert simulation builders and users, implying that they believe them to be honest and true, which also significantly implies a degree of dependency and vulnerability. this point has been noted by fullinwider, (1995, p 2) as one of three that distinguishes a profession, the other two being an orientation to public good, and a specialized knowledge and training, and has clear duty of care implications. 187 australasian journal of information systems volume 13 number 2 may 2006 duty of care requires all individuals to do what is reasonable to protect the wellbeing of others. for example, a legal duty of care is placed on all employers, their employees and any others, including contractors, to do everything reasonably practicable to protect the health and safety of all individuals from hazards in a workplace. a professional duty of care also involves harm minimization, acting in a client’s best interest and exercising discretionary power responsibly. in particular, ören (2002a) has suggested that the major components of professionalism in modelling and simulation are knowledge, activities, and behaviour. that is, according to ören (2002a), “knowledge is essential to perform the activities which in turn should be based on acceptable behaviour”. manufacturers and suppliers of consumable goods have a duty to provide products that do not compromise the safety of consumers. indeed, legislation imposes statutory warranties of fitness for purpose which are contractually binding between seller and purchaser. the use of any product though involves an inherent degree of risk. heckman and wobbrock (1998, p 394) assert: strict products liability recognizes that with modern technology and mass production, injuries will occur without intentional misdeed and despite reasonable care. by implication then individuals and organizations that provide computer models and simulations, also have a duty to provide products that maintain the safety and wellbeing of others. that said, the nature of the risks can vary significantly over a wide range of issues, including for example, an inherently defective product produced without reference to new methods and standards, misinterpretation of output, or inappropriate use occasioned by the provision of poor, inappropriate or no training of users. like other products, faulty simulation products can result in negligence actions. there are various legal options available for defending charges of negligence, and in relation to computer usage in particular there appears to be no clear standards (nissenbaum, 1994, 1996). however this should be no excuse. it would seem reasonable to argue that all builders and users of computer simulations, and in particular those who would identify themselves as experts and members of a professional body, should exercise as an inherent code of practice, a moral responsibility for their products. weckert & adeney (1997, p 89) argue that “experts have a responsibility to be careful, competent, and honest in their work” and that the word “responsible” has two important senses: causal (cause and effect) and accountability (liability). hart (1985) [quoted by coleman (2005)] has suggested a broader interpretation of responsibility identifying four senses. these are: � role-responsibility (the performance or fulfillment of the duties attached to a person’s social role), � causal-responsibility (a retrospective sense of responsibility relating cause and effect between a person’s actions and the consequences of them), � liability-responsibility (responsibility for causing harm in violation of the law), and � capacity-responsibility (the capacity to understand the conduct required by relevant societal norms, and then to act appropriately). coleman (2005) writing in the context of computer systems generally has suggested these senses ‘provide a framework for exploring computers and moral responsibility’. these include: a retrospective analysis of ‘who is responsible for computer use’, the anticipation and prevention of future problems, a consideration of whether computers could be responsible, and an assessment of the decisions computers should not be allowed to make. she also provides a very useful analysis of the difficulties associated with identifying who are the responsible parties in relation to computer use, which centres on four barriers to responsibility identified by nissenbaum (1994, 1996): � the problem of many hands, � bugs, australasian journal of information systems volume 13 number 2 may 2006 188 � blaming the computer, and � ownership without liability, as well as two others: � poor articulation of norms, and � assumption of ethical neutrality. a summary of coleman’s analysis is set out in table 1. barrier description problem of many hands complex computer systems are produced by groups making it difficult to identify who is responsible for errors and harmful consequences of use. bugs ‘bugs’ and ‘computer error’ are preferred to ‘programmer error’ discouraging humans from interpreting these errors as their own (gotterban, 2001). blaming the computer a computer is often the proximate cause of harm. people often attribute intentionality to computers. people have ‘delegated’ or ‘abdicated’ their decision making to computers (ladd, 1989, p 219). people have high expectations of computers (johnson & mulvey, 1995). once computers are blamed there is no need to investigate other human factors. ownership without liability extending the privileges and rights of ownership of software systems without demanding owners accept responsibility for their products (nissenbaum, 1994, 1996). poor articulation of norms a poor understanding of what each party in the creation, implementation, and use of a system is responsible for doing (johnson & mulvey, 1995). assumption of ethical neutrality problems and solutions are articulated and developed in a context in which their impact on humanity is less visible than it should be (gotterban, 2001). table 1: responsibility barriers discussion clearly disclosure and duty of care are complex matters. for example, not only how much and what type of information should be disclosed is important, but also to whom should it be reasonably disclosed (colleagues, independent referees, members of the public)? in addition issues associated with intellectual property, the copyright and patenting of new simulation systems, and the possibility of industrial and international espionage especially given the increasing reliance by the defence sector on the use of computer-based simulations are also worthy of attention. 189 australasian journal of information systems volume 13 number 2 may 2006 a further issue related to disclosure and candour centres on the substance of the detail contained in the information to be released. even partial disclosure of the rationale underlying a model might result in an increase in the authority and mystique associated with the use of and results generated by the simulation which may be totally unwarranted and which may lead to unintended harm. disclosure may provide a simulation with a ‘credibility aura’ well beyond its actual merits, particularly in the public domain where knowledge of the mathematical assumptions and other theories underpinning the simulation design and implementation is either non-existent or perhaps little understood. it may also result in the reverse. sterman (1991, p 209), observes that most people are not able to make judgments about the veracity of computer models in an intelligent and informed manner, and makes the point that computer models can be misused, accidentally or intentionally: thus there have been many cases in which computer models have been used to justify decisions already taken, to provide a scapegoat when a forecast turned out wrong, or to lend specious authority to an argument. a basic consideration here is who is ultimately accountable in these circumstances if a decision made on the basis of a simulation results in harm to an individual or group either directly or indirectly? where does the duty of care lie? weckert & adeney (1997, p 91) have set out three conditions for establishing accountability. these can be interpreted as: control, intention, and free choice. it would seem reasonably obvious then that the misuse of computer simulations might hold the individuals concerned liable to account for their actions, assuming of course that they have not been coerced. this is an important consideration given the quite often deliberate, or otherwise, assertions of 'experts' reporting the results of their work, or politicians and government spokespeople explaining a controversial public policy matter, emphasizing the use of computer modeling and simulations to substantiate and support their assertions and arguments. this may directly or indirectly provide their assertions and arguments with a degree of implied authenticity and authority. there is a possible argument for suggesting that uncritical public trust or confidence in the authority of an expert who suggests, directly or even indirectly, they are using a computer model to support and validate an argument or observation might be somewhat naive and misplaced. perhaps the question is really this: is it the implied or actual use of the computer, or the model and simulation, or both, which assigns credibility to the expert advice? the simple reality is, as already noted above, the quality of the output generated by a computer-based model or simulation is critically dependent on the quality of the data used and the manner in which it was designed and implemented, not necessarily because it was modeled in a computer, even allowing for the fact that modern super computers undoubtedly provide a means for very sophisticated and rigorous modeling and simulations in real time. sterman (1991, p 209) has argued that whether members of society like it or not they are all becoming consumers of computer models. importantly he observed: the ability to understand and evaluate computer models is fast becoming a prerequisite for the policymaker, legislator, lobbyist, and citizen alike. that is, all members of society will need, to some extent, be prepared to accept the responsibility of carrying out some form of appropriate due diligence. in order for this to occur though, a supportive and perhaps even enforceable structure needs to be developed. one way forward here might be the consideration of a 'boundaries of responsibility' concept. this could be based on agreed need to know (or entitlement to know) protocols, which set out the type of information that should be reasonably disclosed, by whom and to whom, at each simulation design and implementation stage. australasian journal of information systems volume 13 number 2 may 2006 190 a possible research focus might involve the development and implementation of a set of information 'boundaries' which outline the rights, entitlements, and responsibilities of simulators and various application users and consumers (including the public domain), so that all concerned are afforded the opportunity to make reasoned, independent and critical decisions about the merits, or otherwise, of the output and use of computer-based simulations if they so choose. clearly this may have significant professional practice implications for simulators, as well as those who use simulation 'tools' as part of their daily responsibilities, be they employment based or otherwise. at a basic level, the 'boundaries of responsibility' proposal looks very much like another code of practice, and might well be so. however it does suggest the incorporation of groups beyond the simulation profession, such as simulation users and public consumers, who also make decisions, both directly and indirectly, based on the work of simulators. coleman (2005) has provided three recommendations for overcoming the series of barriers to responsibility noted earlier which may help to support and regularize this process. these are: • ensuring understanding of (and assumptions about) responsibility is appropriate for the task by using the practice of responsibility to improve both practice and technology. • re-designing computer systems to reveal that they are not responsible. • articulating those norms most relevant to the creation, implementation, and use of computer systems. one final point: mistakes can and do occur. coyle (1977) [quoted by coyle and exelby (2000, p 32)] states ‘there are almost endless opportunities for making mistakes in any kind of model building’, but as heckman & wobbrock (1998, p 399) have asserted: “humans should be responsible for the actions of their creations.” notwithstanding they were writing specifically in the context of autonomous agents, their assertion is relevant to the moral and professional responsibilities the wider modeling and simulation community should be prepared to accept. conclusion in the context of professional practice, the need for disclosure and duty of care were examined as matters which needed further consideration in the development and implementation of computer models and simulations. it was suggested that all simulation builders and users should be prepared to exercise an inherent moral responsibility for their work. references andradóttir, s., healy, k. j., withers, d. h. & nelson, b. l. (eds) (1997) proceedings of the 1997 winter simulation conference. barbour, i. g. (1974) myths, models and paradigms the nature of scientific and religious language, scm press, london. barney, g. o., kreutzer, w. b. & garrett, m. j. (eds) (1991) managing a nation the microcomputer software catalog 2nd ed, westview press, boulder. beder, s. (1998) the new engineer. management and professional responsibility in a changing world, macmillan, south yarra. boumans, m. (1999) “built-in justification”, in morgan and morrison (1999), pp 66 – 96. 191 australasian journal of information systems volume 13 number 2 may 2006 coleman, k. g. (2005) “computing and moral resonsibility”, the standford encyclopedia of philosophy (spring edition), edward n. zalta (ed). (http://plato.stanford.edu/archives/spr2005/entries/computing-responsibility accessed 10 august, 2005) coyle, r. g. (1977) management systems dynamics, wiley, chichester, in coyle, g. & exelby, d. 2000. coyle, g. & exelby, d. (2000) “the validation of commercial system dynamics models”, system dynamics review, vol. 16, no. 1 (spring), pp 27-41. fullinwider, r. (1995) “professional codes and moral understanding”, res publica, 4, pp 1-6. girill, t. r. (1999) “the duty of candor in future software documentation”, 46th annual stc conference, cincinnati, ohio. (http://www.stc.org/46thconf/postconf/girill.html, accessed 4 november, 2003) gotterbarn, d. (1995) “the moral responsibility of software developers: three levels of software engineering”, journal of information ethics, vol. 4, no. 1, pp 54-64., in coleman, k. g. 2005. hart, h. l. a. (1985) punishment and responsibility, in coleman, k. g. 2005. heckman, c. & wobbrock, j. o. (1998) “liability for autonomous agents”, autonomous agents ’98, pp 392-399. johnson, d. g. & mulvey, j. m. (1995) “accountability and computer decision systems”, communications of the acm, vol. 38, no. 12, pp 58-64. ladd, j. (1989) computers and moral responsibility, in the information web: ethical and social implications of computer networking, gould, c. (ed), westport press, boulder, in coleman, k. g. 2005. markovsky, b. and jin, w. (2002) “theories, tests and computer simulations”, presented at the 2002 annual meeting of the american sociological association (http://home.sc.rr.com/bmarkovsky/papers/asasim02.pdf, accessed 29 june 2005). morgan, m. s. and morrison, m. (1999) models as mediators perspectives on natural and social science, cup, cambridge. mulvey, j. m. (1994) models in the public sector: success, failure and ethical behavior, in wallace, w. a. (ed), pp 58 73. nissenbaum, h. (1994) “computing and accountability”, communications of the acm, vol. 37, no. 1, pp. 73-80, in coleman, k. g. 2005. nissenbaum, h. (1996) “accountability in a computerized society”, science and engineering ethics, vol. 2, pp 25-42 in coleman, k. g. 2005. ören, t. i. (2000) “responsibility, ethics, and simulation”, transactions of the society for computer simulation international, 17, 4, pp 165-170. ören, t. i. (2002a) “growing importance of modeling and simulation: professional and ethical implications”, in proceedings of the icsc’2002 – the 5th conference on system simulation and scientific computing. ören, t. i. (2002b) “rationale for a code of professional ethics for simulationists”, in proceedings of the summer computer simulation conference. rogers, r. v. (1997) what makes a modeling and simulation professional?: the consensus view from one workshop, in andradóttir, s. et al (eds) proceedings of the 1997 winter simulation conference, pp 1375-1382. shelly, g. b, cashman, t. j. & vermaat, m. e. (2002) discovering computers 2003 concepts for a digital world, thompson course technology, boston, ma. singh, n. (1996) systems approach to computer-integrated design and manufacture, john wiley & sons, new york. australasian journal of information systems volume 13 number 2 may 2006 192 society for modeling and simulation international (2005) a code of professional ethics for simulationists (simethics), http://www.site.uottawa.ca/~oren/scs_ethics/ethics.htm). sterman, j. d. (1991) a skeptic’s guide to computer models, in barney, g. o., kreutzer, w. b. & garrett, m. j. (eds) 1991, pp 209-229. wallace, w. a. (ed) (1994) ethics in modeling, elsevier science, oxford. weckert, j. & adeney, d. (1997) computer and information ethics, greenwood press, westport. ajis vol. 9, no. 2 may 2002 mobile data technologies and sme adoption and diffusion: an empirical study on barriers and facilitators. jeanette van akkeren associate lecturer in information systems, faculty of business university of the sunshine coast sippy downs, queensland, australia jvanakke@usc.edu.au debra harker lecturer in marketing faculty of business university of the sunshine coast sippy downs, queensland, australia dharker@usc.edu.au abstract the technological environment in which australian smes operate can be best described as dynamic and vital. the rate of technological change provides the sme owner/manger a complex and challenging operational context. wireless applications are being developed that provide mobile devices with internet content and e-business services. in australia, the adoption of ecommerce by large organisations has been relatively high, however, the same cannot be said for smes, where adoption has been slower than other developed countries. in contrast, however, mobile telephone adoption and diffusion is relatively high by smes. will smes who have been reluctant to adopt ecommerce technologies in the past be more ready to go on-line with the merging of the internet and mobile data technologies? this exploratory study identifies attitudes, perceptions and issues for mobile data technologies by regional sme owner/managers across a range of industry sectors. the major issues include the sector the firm belongs to, the current adoption status of the firm, the level of mistrust of the it industry, the cost of the technologies, and the applications and attributes of the technologies. keywords: mobile data technologies, ecommerce, sme, adoption, internet introduction the technological environment in which australian smes operate can be best described as dynamic and vital. the rate of technological change provides for the sme owner/manger a complex and challenging operational context. whilst is literature discusses the adoption of new technologies amongst smes, the latest technology, the mobile data technologies and associated 'wireless' applications, presents new problems for adoption and diffusion for sme owner/managers. how, for example, can needs be assessed for a technology that does not yet fully exist? the primary appeal of these mobile data technologies apart from mobility is that associated 'services' are delivered on existing devices such as mobile phones, palm-tops, and personal digital assistants (pdas). in the literature little empirical work exists on applications and services that would encourage the adoption of mobile data technologies by smes. this study provides empirical evidence on attitudes of sme owner/managers in a regional setting to mobile data technologies, and identifies the most significant facilitators and inhibitors to adoption. the development of gateway technologies for service providers supporting wap are already available and on the market. further, wireless applications are being developed that provide mobile devices with internet content and e-business services. these mobile data technologies are expected to affect business in a similar fashion to the internet and world wide web a few years ago (semilof 1999). the major appeal of mobile data technology is that it provides information to the mobile user such as reading news, getting stock quotes, sending email, downloading data, locating other users, remote access of home and business sites, and making purchases on a device that consumers are comfortable with the mobile telephone. the adoption of mobile telephones in the past five to ten years for voice services and messaging, by both small and large organisations in australia, has been high. helping to accelerate the demand for the newer mobile data technologies in countries already embracing the technologies (such as japan) is the explosive growth of the internet and mobile computing (clever 1999). in australia, this provides somewhat of a conundrum. although adoption of ecommerce by large organisations in australia has been relatively high, the same cannot be said for smes, where adoption has been slower than other countries, such as singapore, the united kingdom, the united states of america and japan (forrester research incorporated 1997, yellow pages australia 1999). in contrast, however, mobile telephone adoption and diffusion is relatively high by smes. the conundrum is, therefore, will smes who have been reluctant to adopt ecommerce technologies in the past be more ready to go on-line with the merging of mobile voice and mobile data technologies? ajis vol. 9, no. 2 may 2002 literature review early studies addressing the adoption of it provide insight into reasons why decision-markers adopt or do not adopt innovations. the diffusion of innovation theory posited by rogers (1995) suggests that characteristics of innovations help to persuade potential adopters to embrace or reject an innovation. other researchers have since completed work resulting in modifications to rogers' original theory that provide numerous models that address it adoption and usage within an organisation. the technology acceptance model addresses it adoption, implementation and diffusion in terms of perceived ease of use and perceived usefulness based on behavioural intentions (davis 1989). belief about the system, perceived usefulness and perceived ease of use is posited as directly affecting attitude to use (rogers 1995, agarwal and prasad 1997). further studies suggest that behaviour is a direct function of behavioural intention and perceived behavioural control that will impact on decision-makers choosing whether to adopt an innovation azjen 1991, taylor & todd 1995). therefore in the adoption of mobile data technologies, will the same behavioural/psychological factors impact on the decision by owner/managers to adopt mobile data technologies? constructs used in the above models are generally based around perceptions, beliefs, attitudes and intentions of the decision-maker. more recent studies have identified other factors in addition to these that will impact on an owner/manager's decision to adopt new innovations such as mobile phones and internet technologies. these include economic factors such as return on investment and characteristics of the firm such as the size, sector and status, and the structural sophistication of the firm. adoption of it/ecommerce by smes empirical studies have identified a variety of factors thought to affect e-commerce/internet technology adoption in small business (mien & raymond 1994, brooksbank, kirby, and kane 1992, kirby & turner 1993, lacovou, benbasat, and dexter 1995, tayl;or & todd 1995, harrison, mykytyn, and rienenschneider 1997). from the adoption factors identified in earlier studies a model (figure 1) was developed based on the study on the adoption of e-commerce technologies thought to facilitate or inhibit technology adoption by sme owner/managers (van akkeren & cavaye 1999). it'was on this basis that attitudes and perceptions of sme owner/managers were assessed on the adoption of mobile data technologies for this current study. figure 1: framework of sme adoption of innovations eranager l . , ; '' '' ^ it adoption in small . * . s i z e , . i . • information intensity % : • • • ' • • . . • • • ' . • • • • • • r ' 1 ' ' ' • ! " • • • . ''••' (source: van akkeren and cavaye, 1999) the three categories presented in the framework, thought to impact on the adoption of it innovations, provided areas of discussion for the focus groups in the exploratory stage of this research. participants were encouraged to discuss their attitudes and perceptions of it adoption in general and mobile technologies in particular. ajis vol. 9, no. 2 may 2002 recent studies on reasons why sme owner/managers adopt or do not adopt information technology (it) and ecommerce technologies have highlighted both inhibitors and facilitators to adoption and are similar in content to the factors described above. sme adoption is discussed as being determined by decision-maker characteristics, information system (is) characteristics, organisational characteristics and environmental characteristics (thong 1999). in this study it was found that the need for is to offer better alternatives to existing practices is critical to adoption by smes. therefore, could the use of mobile data technologies provide the 'better service' that sme owner/managers seek? owner/manager characteristics, planning orientation, and the existence of alliances/networks are suggested by as influencing technology adoption by smes (mcgregor & gomes 1999). specifically in terms of alliances/networks, previous literature also suggests that the decision to adopt or not adopt it will be influenced by the network within which the small business operates, particularly if time, cost and operational efficiencies can be realised (piovesana, and raush 1998, lawrence 1998, sillence, macdonald, lefang and frost 1998). in contrast, in a study of smes in the uk, it is proposed that adoption of it will be improved through education of the benefits of the technologies, and by fostering a planning orientation to is design and implementation (bridge & peel 1999). in an australian study on it adoption by smes, adoption decisions are identified as having three distinct phases (fink 1998). these are: assessing it benefits, organisational culture and firm-suited it; assessing internal resources and procedures; and evaluating external environment, support and services. in terms of mobile data technologies, one could argue that owner/mangers would need to assess what, if any benefits can be derived by adopting the technologies and whether their firm was suited to their adoption. part of that evaluation would necessarily need to include both the internal and external environment in which the firm operates. adoption and diffusion of mobile data technologies major innovations may have to 'prove themselves' in new markets before they can displace other technologies (friar and balachandra 1999). it is the early adopters or innovators who will initially experiment with these technologies. in addition, the usefulness and ease-of-use will impact on owner/manager acceptance of the technologies (agarwal and prasad 1997). therefore the attributes of the technologies may improve their acceptance by sme owner/managers. the acceptance of web-based technologies is also influenced by ease of use and perceived usefulness in terms of current is sophistication, complexity of the new technologies, and perceived costs and benefits (nambisan and wang 1999). in australia, the adoption of internet/e-commerce technologies varies in different states, and further, between regional and city-based firms. however, the adoption of mobile phones is consistently high across states and regions within australia. mobile data technologies, which 'marry' mobile phones and e-commerce technologies, are seen as eliminating time and distance as barriers for regional businesses in their adoption of these technologies. estimates put usage of mobile data technologies as one billion worldwide by 2003 (greengard 2000), or us$1.3 trillion in relation to annual turnovers by 2003 in the us alone (thurston 2000). with such potential markets available to the vendors of mobile data services and devices, it is useful to understand reasons why the majority of potential end-users are so far resisting these new technologies. lack of speed is a barrier to adoption as mobile data technologies are slow and hence inefficient (taylor 1999, saunders, heywood, dornon, bruno and allen 1999). another barrier is the lack of standardised it environment for developing mobile data applications as impeding the growth of the mobile data market (harrison 1999, axby 1998). limited bandwidth, higher usage costs, increased latency, and a susceptibility to transmission noise and call dropouts are also possible barriers to adoption (duffy 1999, johnson 1999). it is possible therefore that adopters are 'sitting back' and waiting for at least some of these problems to be corrected before entering the mobile data market. another area of concern for end-users is that the wireless application protocol, the emerging technology used to send data to and from handheld devices, has no security mechanisms built into it osowski 1999, chan 2000). this is concerning not only to the business user, but to the customers of the business as well. capturing users requires 'transparency', that is, users want information or communication access whenever and wherever they need it, using whatever device is most convenient at that moment (osowski 1999). sme owner/managers do not buy technology; they buy business benefits (duffy 1999). mobile data technology benefits include easy communication through e-mail, ready access to information (wherever/whenever), entertainment, and improved lifestyle through e-commerce and home banking. clearly, the literature on mobile data technologies to date underlines the importance of highlighting the benefits of using the technologies, and the ease of use to potential users. sme owner/managers are not interested in the architecture, standardisation issues or the technologies themselves, instead they require a device that provides efficient, effective access and communication applications personalised to their individual needs. ajis vol.9, no. 2 may 2002 most literature on the adoption and marketing of mobile data technologies is not empirically based and is limited to discussing the technologies in terms of their application to business, rather than adoption barriers. a distinction that can be made when determining adoption by decision makers in australian smes can be based on firms in different states, and further, regional versus city-based firms. mobile computing is seen as eliminating time and distance as barriers for regional businesses in their adoption of these technologies, particularly in relation to the design of work and for reaching potential markets. however, many regional areas of australia have been less prepared to adopt internet and ecommerce technologies compared to their city-based counterparts. this current study will provide a deeper understanding of the facilitators and barriers to the adoption of mobile data technologies by sme owner/managers in a regional setting. insight will be provided on the types of applications that potential business customers require methodology two focus groups were conducted with respondents in each usage group, non-adopters, partial-adopters and fulladopters of information technology, covering a variety of industry sectors. descriptions of each groups are: • full-adopter: used computer and internet for business, email, ecommerce, website. • partial-adopter: used computer for business, some use of internet for business/home but no website. • non-adopter: no use of computer for business purposes. selection of respondents was based on two main criteria: current technology adoption status of the firm and the industry sector the firm belonged to (based on australian and new zealand industry classification (anzic) categories). as much as possible, a range of industry sectors was represented in each of the focus groups as different industry sectors have different needs in terms of technology adoption and usage (van akkeren and cavaye, 1999). every respondent was the owner/manager of the firm as they make all the management decisions relevant to the enterprise, including technology adoption and usage. attendance at the focus groups was vital to the integrity of the research, thus a rigorous procedure was adopted to increase response rates. a database of sme contacts on the sunshine coast was developed by research assistants and a recruitment screening questionnaire was crafted by the researchers. the questionnaire succeeded in categorising respondents into usage groups and industry sectors and an incentive of a $50 gift voucher, redeemable at a local coffee shop, or restaurant, or cinema was used to encourage people to attend. the groups were scheduled towards the end of the working day and in the evening, which were found to be the times that best suited this type of business-person. a letter of confirmation was mailed and faxed to willing participants on the day of recruitment and a telephone call made on the day of the group to ensure attendance and alleviate any concerns. great care was taken to ensure respondents had a positive experience of the focus group process. for example, maps were included in the mail out, ushers were on hand to escort small groups of respondents from a meeting place to the focus group room, welcoming beverages and finger food were provided, university promotional material was distributed, and networking was encouraged. one of the key procedures in the protocol was to ask respondents before the group began to select the provider of their gift voucher and to assure them that the gift voucher would be mailed to them within the week. this was an important step in the process as recruitment promises were fulfilled before the focus group began, thus building a feeling of trust amongst respondents. the groups were video-recorded and respondents were informed before commencement about the process and reasons were provided as to why it was necessary to record the group. respondents were then given the option of leaving the group if they were not happy being recorded. no one left. it is worthwhile to record this protocol for focus group recruitment because it provided a number of significant benefits. first, positive public relations were apparent for the university, nortel networks and marketing research practices in general. second, attendance at the groups was better than expected (table 1). it is common practice to over-recruit for focus groups as there is a natural drop-out rate of 3-4 people on the day of the group but this was not found in this project, using this protocol. ajis vol. 9, no. 2 may 2002 table 1: focus group attendance category non-adopters non-adopters partial-adopters partial-adopters full-adopters full-adopters group time 5pm wednesday 24. 11. 99 7pm wednesday 24. 1 1 .99 4pm monday 22. 11. 99 7pm tuesday 23. 11. 99 7pm monday 22. 11. 99 4pm tuesday 23. 11. 99 respondents 10 9 15 11 14 12 focus groups ran for approximately one-and-a-half hours and were video-taped with the permission of respondents. questions were asked about current technology usage and respondents were encouraged to discuss their experiences, attitudes and perceptions of information technologies. after showing a brief video that demonstrated mobile data technologies, respondents were then encouraged to discuss their reactions to these new technologies and possible applications to their industry sector. the focus group topic guide was informed by the model developed from the literature review and in collaboration with nortel networks. data analysis was conducted using the content analysis approach. gifts vouchers were mailed to respondents at a later date, together with a nortel networks pen and promotional material. findings1 problems with, and praise for, it after a brief introductory session, fulland partial-adopter respondents were asked to discuss any problems with, and praise for, it in general. problems with it whilst there was some overlap in the areas of problems with it cited by both fulland partial-adopters, certain issues were mentioned by only one group. for example, 'pricing and costs' were areas of concern for fulladopters only, and they perceived a link between it and these areas. combining new ways of trading (for example on the internet) with the move to relocate manufacturing plants to offshore locations, full-adopters felt that competition has risen to new heights: "there's always a cheap copy available somewhere ... people appreciate the quality but they 're not prepared to pay for it" (f, clocks, 3) "people will access the internet, find the cuckoo clock they want... press their button and they 've got it there!" (f, printing, 25) a second area of problems with it, recognised by both full and partial-adopters, related to the 'reliability and support' of it products and services. there was criticism about the rate of change in the area of innovation and, in more practical terms, strong feelings of frustration about the usefulness of it manuals: "the gap between the promises and the deliverables is quite huge" (f, health foods, 2) "it's the pace of change — six months and it's out of date " (f, retail, 2) "i'm on the phone constantly every 2 or 3 days, 'how do i do this? (p, convenience store, 2) "i'd like to be able to read a manual" (f, apartments, 2) "the information on how the hell the damn thing works is a nightmare!" (p, travel agent, 5) however, there were useful suggestions to address the latter concern: 1 where direct quotations are made from respondents, the reference is given as 'full-adopter' (f), 'partialadopter' (p), 'non-adopter' (n), followed by an industry descriptor and number of employees, 'partial-adopter' (p), 'non-adopter' (n), followed by an industry descriptor and number of employees. ajis vol. 9, no. 2 may 2002 "an instruction manual on a video disk ... instead of words there's a picture of someone " (f, electronics, 2) similarly, both fulland partial-adopters cited 'compatibility' as an area of frustration with it. this related to computers, consumables (such as printer cartridges), and attachments in email: "you get something that's sent by email and you can't open it up and that's a real pain " (p, signage, 3) a key difference between these two groups of adopters was the issue of 'fear'. full-adopters did not display any fear about working with technology, however partial-adopters realised they needed to use it to demonstrate the currency, and therefore dynamism, of their business, in spite of their reservations: "one good thing it gives is the appearance you know what you 're doing " (p, hairdresser, 5) "if i present something into one of my shops which is a little bit high tech the girls will look at it and think 'i'm a hairdresser, not a computer whiz' so they 're a little scared of this technology' (p, hairdresser, 5) being low users of technology, non-adopters in the groups were asked why they did not use it. the most important reason cited by them was 'cost', together with: • losing data • maintenance • obsolescence • lack of suitable training • no need/benefit • viruses • time consuming • break downs • power failures • impersonal understandably, there was also the issue of 'fear' with non-adopters; fear of buying the wrong technology, or about their own ability to learn new technology: "ifyou write things out you don't have to worry if you press the wrong button and something gets lost" (n, home maint., 2) all of these problems were couched in business terms, that is, all of the areas were felt to negatively affect business practices. praise for it many full and partial-adopter respondents were willing to tolerate the frustrations of technology, given the positive impact it made on their lives. speed was the main advantage mentioned about technology and this had implications for business practices. many cited the increased improvement and decreased cost of technology over the years: "we bought a computer at approximately one third of the price of the one we bought four years ago, it's probably ten times more valuable, fifteen times more storage " (f, health foods, 2) innovations respondents in all groups were asked to suggest the main innovations in it over the past five years. the following table displays the responses of all groups: ajis vol. 9, no. 2 may 2002 table 2 innovations innovation mobile phone: computer: internet: eftpos: desktop banking: fax: barcoding/ scanning: databases: networking: operating systems & software: printers: scanners: video cassette recorder: benefits freedom, flexibility, convenience, availability. speed, control (inventory, accounting), data storage, letters, retrieving information. information, world trends, check prices, website, competitor information, cost effective (e.g. sending samples of work via email), convenience, speed, current. speed, streamlined service, cashless society, enhanced security, keeping up with competitors, add-on sales, impulse buying, security, keep up with competitors. immediacy, ease of moving money around, payroll, bill paying, remote operation: speed, time and money saving, direct communications, visual. currency of information (e.g. stock levels), competitive advantage. tracking, reports, mailouts. linking pcs together, reduces costs, increases speed. moving from dos to windows and nt better for business. higher quality, cheaper now. time and cost of producing high quality work reduced now. demonstrate business (if appropriate) (only mentioned by non-adopters) typical comments "if you miss out on a deal, you miss out on money " (n, boutique, 2) "we used to have two rooms of books, now we have one shelf of cds " (f, solicitor, 4) "it 's more convenient than getting the yellow pages out" (f, resort, 9) "we rarely take cheques ... generally credit cards, and that means big business" (f, tv service, 14) "the money is immediately in your bank" (n, shoes, 3) "we operate shops in rockhampton and on payday ...just highlight the people we want to pay and then schunk!" (f, printing, 25) "you come in and the floor's full of it, all the orders " (n, boutique, 2) "we do all our work by fax" (p, dry clean, 9) internet usage and adoption behaviour full-adopters were internet users and most had their own website. they tended to use the internet as a business component and competitive edge, for example to sell their goods and services, rather than buy, and to generate enquiries: "we get a lot of enquiries on email in the holidays, from all over the world ... tasmania, ireland" (f, resort, 9) "people say they 've seen your web page ... that's a positive aspect" (f, mobile hairdresser, 2) by definition, partial-adopters were not significant users of the internet for business purposes. those that did use the internet did so in response to pressure from competitors, customers, or suppliers. non-adopters did not use the internet and so were asked about family use; many had children who used the internet, either as part of their education, or actually working in the it industry: ajis vol.9, no. 2 may 2002 "they 're brought up with it" (n, designs, 1) all respondents were asked to comment on their use or intended use of teletext and digital televisions. whilst full-adopters were the most aware (of all three types of it users) when it came to teletext, the were not supporters; indeed, those that had used it found it to be too slow and not interactive enough: "there are over 600 pages, but there's only three you ever want!" (f, apartments, 2) "it's very ... dull" (f, health food, 2) partialand non-adopters of it did not use teletext. when asked about digital tv, fulland partial-adopters were wary, waiting for government standards to be established before buying, and also for, what they felt would be, the inevitable price decrease after launch. nonadopters had no interest: "the price will be astronomical, it 'ii come down ...just like the old colour tvs ... it's like we 're changing from black and white to colour" (f, property developer, 2) "when they come down to about $1,000 after about 6 years " (p, car electrics, 4) product/service applications video the nortel networks video was shown in all groups and then responses elicited about product/service application needs and benefits. initial reactions full-adopters were visibly stimulated by the video, displaying knowledge of the area and being keen to discuss future applications: "we 're not very far away from a lot of these things ... i witnessed a digital camera take a photo and he hooked it up to his mobile phone and sent it to someone" (f, printing, 25) partial-adopters were unenthusiastic about the technology demonstrated in the video, being very wary, cautious and fearful: "i wouldn 't want one of those because it 'd be 'where are you?' and they would be onto me" (p, dry clean, 9) "i wouldn't like it" (p, signage, 3) the reaction of the non-adopters was somewhere between the fulland partial-adopters; when asked, half of the non-adopters wanted one of the handsets, mainly to stay in touch with the youth and technology in general, whilst the other half did not see a need. however, they were overwhelmed, initially, by the technology demonstrated: "we couldn 't take it all in!" (n, shoes, 3) respondents then participated in an exercise whereby the services demonstrated in the video were listed, benefits associated, and then the services ranked in order of importance. tables 3, 4 and 5 display this information for each adopter level: 10 ajis vol. 9, no. 2 may 2002 table 3 services (full-adopters) service 1. 2 way communication: video/voice/ voice recognition 2. prioritising & screening messages 3. remote access 4. online information 5. attachments, downloading 6. navigation 7. translation benefits relationship marketing, the power of face-to-face persuasion, personalised and improved customer service, speed, time, planning: delegation of work to others, freedom, cost savings on labour, rent overheads: ability to check home/business from another location, security, peace of mind, working smart: voice searching on the internet or specialist databases: ability to send information to others. directions, deviations, detours, speed, time, planning. useful for international transactions. typical comments "international business is a very personal thing" (f, exports, 17) "no one would have to be in one particular office space " (f, fencing, 6) "if it wasn 't okay i could push another button and get security guys there straight away " (f, property developer, 2) "i could check the chlorine levels in my pool" (f, apartments, 2) "like 'intestacy ' — what sort of documents do i need? " (f, solicitor, 4) table 4 services (partial-adopters) service 1 . remote access & security 2. 2 way communication: video/voice/ voice recognition 3. navigation 4. prioritising & screening messages benefits control speed, communication, mobility, immediacy, ease of use, improved communication means improved customer relations, build loyalty. information delays, detours, local knowledge, time and cost savings from better planning. screen messages, screen ads out. typical comments "you don 't have to go into work ... just put that machine on and see what 's happening " (p, coffee house, 12) "i don 't need to sit at the computer ... just walk around the store chatting to it!" (p, conv. store, 2) "you can do more than one thing at a time" (p, newsagent, 5) "the courier guy ... hasn 't had to think about it, he 's done it in half the time ... made his run" (p, festival org, 1) 11 ajis vol. 9, no. 2 may 2002 table 5 services (non-adopters) service 1. 2 way communication: video/voice/ voice recognition 2. navigation 3. remote access & security 4. shopping benefits sending samples, time saved on travel and answering calls, freight costs reduced, business networking opportunities save time. both business and home needs. typical comments "you can see who you 're talking to " (n, hairdresser, 2) "the map business ... i loved that, that was excellent! " (n, boutique, 2) full-adopters had the clearest thinking about how they would use this technology immediately. their discussions centred around how their top three ranked services would interact and the impact this would have on their business practices. for example, they were the only group of adopters who included 'prioritising messages' in their top three (table 6), with the key benefit of delegation of tasks. there was also discussion in these groups about screening of junk messages and to prioritise messages, suggestions included screening by time of day, and by different types of callers. table 6 top three rankings, all groups rank first second third full-adopters 2 way communication: video/voice/ voice recognition prioritising & screening messages remote access & security partial-adopters remote access & security 2 way communication: video/voice/ voice recognition navigation non-adopters 2 way communication: video/voice/ voice recognition navigation remote access & security the partial-adopters were not keen to have this technology at all but when asked to rank their services, they included the navigation service, as did the non-adopters. all groups raised concerns about the services on show. partial-adopters were concerned about the reliability of the system, confidentiality, having no free time, the civil liberty issue and concerns about 'big brother'. nonadopters mentioned the issue of invasion of privacy. security of such a system was a recurring theme amongst all groups: "what happens if you lose it (the handset)? " (f, property developer, 2) "someone else could access your home security before they go in and rob you " (f, windscreen, 4) remedies for the security concern included; using thumbprint recognition, retina scan or voice recognition. service delivery participants were asked the type of mobile data device they would favour most, what personalisation services they would prefer, and the most appropriate pricing of the device. handsets the nortel video used to demonstrate the new technologies showed users having two or three different devices. from a practical point of view, full and partial-adopters were in agreement that they would prefer only one handset for this type of service: "you 'd be carrying this around (diary), along with this (mobile phone) and the glasses and you don't want more than one (pda/handset) to carry around" (f, export, 17) 12 ajis vol. 9, no. 2 may 2002 "i find it hard enough to remember to take my bag, my purse, my car keys and get in the car with three kids, let alone three or four other devices!" (f, electronics, 2) a personalised service the ability to personalise a service was welcomed by most group respondents, across all it usage types. to be able to access certain websites, locations and services is appealing and respondents felt that this could impact on price. some respondents, most notably the partial-adopters, were willing to take extra advertising on thenhandset if this reduced the price. respondents demonstrated their distrust of the it industry by requesting a compatible service, one that works with other competing products and services in the marketplace, and one that is reliable: "telecom come out with one, vodafone too, and suddenly you 've got to be better so they add a little extra on and it's not so simple " (f, apartments, 2) "you 'd have to pay for a service package, but then you 'd have to have the service to go with it" (p, conv. store, 2) price many respondents made the analogous link between this type of new technology and the introduction of the mobile phone some years ago: "it's like mobile phones ... people gradually over time realised they 're not so bad' (p, accountant, 8) this lead to a discussion on pricing with a 'user pays' approach gaining most support amongst group members: "the way people pay for mobile phones has been instrumental in the wide acceptance of mobile phones in this country" (f, business supplies, 4) for example, people were keen to have a modular approach to price, whereby they could up/downgrade as and when they needed more or less services. thus, a rental or lease arrangement was popular, charging on a usage approach, rather than apriority of service basis, whereby they would be restricted to using the service at certain times of the day or night: "you pay another fee per month if you want to get to talk, change it in different languages, you pay another fee, you could pay $100 per month or $500 per month but if you don't want any-you only want to pay $50 per month' (f, property developer, 2) "leasing it... when your job changes or your business grows, you just hand that one in and say 7 need the next model'" (n, cafe, 15) "in business you 're probably going to want to be charged by use because you can't say 'i can't do that until after 10 o 'clock' or something" (n, gifts, 2) "say, an 18month contract but with the ability to upgrade easily" (p, art gallery, 2) full-adopters, comfortable with technology, were especially able to see the value and benefits of the proposed service, consequently they would be keen to acquire the service if it was affordable: "we 're going to rely on it more and more, that's going to be a great marketing tool and a great money-making tool for them (nortel) so it should be priced on usage" (f, printing, 25) discussion it is possible to argue that there are many different factors that will impact on an sme owner/managers' decision to adopt or not adopt mobile data technologies. the industry sector the firm belongs to, the current it adoption status of the firm, the level of mistrust of the it industry and the cost of the technologies are highlighted in this 13 ajis vol. 9, no. 2 may 2002 study as possible barriers or facilitators to adoption. the features of the mobile device, including the applications on offer (which directly relates to perceived business benefits of the technologies) are also raised as having a possible impact on adoption. the focus groups found significant differences in the way different types of people view and use technology. full-adopters of technology, for example, are very open to new ideas and innovations, compared to the partialadopters, although they raise concerns about the cost of keeping up with the rate of technological change. nonadopters, whilst being fearful of technology generally, have an open mind when presented with futuristic new technology, especially when compared to the partial-adopters. one reason for these differences could be that partial-adopters have been 'forced' to adopt technology before they were really prepared; this adoption may have been in response to customer, competitor, or supplier pressure. thus, they are fearful; of over-reliance on technology, of being hoodwinked by it companies, of being made to feel inadequate, and of being overtaken by others in terms of business practices. when asked about innovation in it in the past five years most discussion in the groups centred around the mobile phone, computers, the internet, eftpos, desktop banking and the fax machine. benefits associated with these innovations included freedom, flexibility, speed, convenience, increased information, competitive advantage, ease of use and direct communications. these reactions bode well for nortel and the future of wap, as the key features of wap provide all of these benefits on a mobile device. in terms of internet use by the group members, full-adopters have no fear and use the technology for business and pleasure. partial-adopters tend to have internet technology forced upon them by competitors, customers or suppliers, whilst the non-adopters commented on family use and pressure and demonstrated a desire to 'keep up' with the youth of today, and tomorrow. the nortel networks video that was used in the groups facilitated much discussion, excitement and fear, but also biased responses somewhat, especially in relation to the number of handsets or pdas people would tolerate. initial reactions to the video by respondents were mixed. full-adopters were visibly excited by the prospect of the technology demonstrated, partial-adopters were unenthusiastic, cautious and fearful, whilst non-adopters were evenly divided in their interest. full-adopters, many of whom could be considered early adopters and innovators, would use this technology tomorrow if it were available, as long as it was affordable. the service they would most like to 'buy' is the twoway communication with video/voice recognition. this group of adopters was the most focused in their intended use of the technology, being very business-focused and seeing a need for almost all of the services on display. the non-adopters also cited the two-way communication with video/voice recognition as their favourite service and both groups felt that this service would give them improved customer relations with the face-to-face persuasion, increased speed and better use of time. partial-adopters were reticent about the technology but felt that, if they had to choose, the ability to remotely access sites would be useful and the security aspect appealing. the greatest benefit for them would be control of the working environment. all three adopter groups were concerned with security of the system. the issue of personalising the service was attractive to most focus group members as they are busy running a business and not interested in unsolicited messages or advertising. people also linked the personalisation issue to price, feeling that a user-pays system was fairest, with the ability to upgrade when they were ready or saw a need for more services. group members also likened the new technology to the roll out of mobile phones in australia, with rental plans and price based on usage levels. respondents were not keen to have the services priced on a priority basis whereby they would be restricted to using the service at certain times of the day or night as this would impact negatively on their business practices. the model used as the basis for the development of the focus group topic guides was useful in predicting the types of issues different adopter groups raised. however the level of emphasis placed on different constructs listed in the model varied depending on the adoption status of the owner/manager. for example, the level of mistrust of the it industry was seen as an influencing factor by partialand full-adopters. however, non-adopters were less concerned about this issue. this is not surprising given they have not purchased technologies and therefore are not experienced with the it industry. perceptions of owner/managers of the benefits of mobile data technologies appear to be an important consideration, as do the purchasing and on-going costs of the technology. the model also predicts that computer literacy, external pressure to adopt the technology, customer/supplier dependency and the size, sector and status of the firm will impact on technology adoption. findings from the focus groups support these factors as influencing adoption of mobile data technologies. to conclude, there are a number of issues that are apparent from this exploratory study that need to be included in a larger, quantitative study with a significant sample from the wider population. limitations of focus groups such as the influence participants have on each other, and the intimidation some participants' experience within a group situation could be overcome with a quantitative study such as a survey or telephone interview. in addition, a greater cross-section of industry sectors would need to be included to improve the generalisability of this study. findings presented so far may not be generalisable to other australian capital cities. however including citybased small businesses in further work should overcome this problem. benefits are very important to sme 14 ajis vol. 9, no. 2 may 2002 owner/managers who do not have the luxury of time to train, research, or upgrade their technology, without losing business. thus, the quantitative study should focus upon technology use and associated benefits across different industry sectors, gauging levels of concern/comfort with technology. the study could then focus respondents on the new technology, gauge awareness, need, and services sought, building on the findings from this exploratory work. references agarwal, r. and prasad, j. (1997), 'the role of innovation characteristics and perceived voluntariness in the acceptance of information technologies', decision sciences, 28,3,557-582. ajzen, i. (1991), 'the theory of planned behaviour', organisational behavior and human decision processes,, 179-211 in taylor, s. and todd, p.a., 'understanding information technology usage: a test of competing models', information systems research, 6,2 (june 1995),144 -176. axby, e. (1998), 'creating a market for mobile data', telecommunications, 32, 9 september, 37-39. bridge j. and peel m. j. (1999), 'a study of computer usage and strategic planning in the sme sector', international small business journal, london, 17,4, 82. brooksbank, r., kirby, d., and kane, s. (1992), 'it adoption and the independent retail business: the retail newsagency', international small business journal, 10, 3 (apr-june, 53-61. chan, t. (2000), 'cracks in the wap', america's network, 104,3,26-27. clever, m. (1999), 'mass market solutions for mobile data', telecommunications, 33, 6,40-49. davis, f. (1989), 'perceived usefulness, perceived ease of use and user acceptance of information technology', mis quarterly, 13, 3, september, 319-340. duffy, r. (1999), 'wireless set to take the lead', telecommunications, 33, 11,24-26. fink, d. (1998), 'guidelines for the successful adoption of it in small and medium enterprises', international journal of information management, august, 18,4. forrester research incorporated (1997), forrester ranks world economics for ecommerce, 30 april, cambridge,, ma, [online] <http://www.forrester.com.>. accessed august 1998. friar, j. h. and balachandra, r. (1999), 'strategies for marketing new technologies', research technology management, 42,4, 37-43. gefen, d., and straub, d.w. (1997),, 'gender differences in the perception and use of e-mail: an extension to the technology acceptance model', mis quarterly, 21,4,389-400. greengard, s. (2000), 'going mobile', industry week, 249, 1,18-22. harrison, d.a., mykytyn, p.p. jr, and rienenschneider, c.k. (1997), 'executive decisions about it adoption in small business: theory and empirical tests'. information systems research, a journal of the institute of management sciences, 8,2, 171-195. harrison, h. (1999), 'wap: the key to mobile data', telecommunications, 33, 9,96-98. horn, d. (2000), 'e-business portals ease end-user service creation', telecommunications, 34, 1,43-45. lacovou, c.l., benbasat, i., and dexter, a.a. (1995), 'electronic data interchange and small organisations: adoption and impact of technology', mis quarterly, 19,4,465-485. johnson, a.h. (1999), 'wap', computerworld, 33,44 69. julien, p.a., and raymond, l. (1994), "factors of new technology adoption in the retail sector', entrepreseurship: theory and practice, 18,5,79-90. kirby, d., and turner, m. (1993), 'it and the small retail business', international journal of retail and distribution management, 21,7,20-27. lawrence, k.l. (1998), 'factors inhibiting the utilisation of electronic commerce facilities in tasmanian smallto medium-sized enterprises', 8th australasian conference on information systems,. mahajan, v. and muller, e. (1999), 'when is it worthwhile targeting the majority instead of the innovators in a new product launch', journal of market research, 35,4,488-495. mcgregor, j. and gomes, c. (1999), 'technology uptake in smes: some evidence from new zealand', journal of small business management, 37 3, 94-102. mobile data conference (1999), sydney, nsw. nambisan s. and wang y. (1999), 'roadblocks to web-based technology adoption', association for computing machinery, communications of the acm, new york,, 42,1, 98-101. osowski, k. (1999), 'unifying communications', communications news, 36,9, 18-20. piovesana, r. and raush, b. (1998), "the web help small players connect', bobbin, 39, 9, 50-51. riggs b. and bachelor b. (1999), 'vendors address issue of security for wireless devices', informationweek, 765,25-26. rogers, e.m. (1995), diffusion of innovations, 4* edn, the free press, new york. saunders, s., heywood, p., dornon, a., bruno, l. and allen, l. (1999), 'wireless ip: ready or not, here it comes', data communications, 28, 12 42-68. 15 ajis vol. 9, no. 2 may 2002 semilof, m. (1999), 'hitting the road? take the internet along', computer reseller news, 864, 1. sillence, j.a.a., macdonald, s., lefang, b. and frost, b. (1998), 'email adoption, diffusion, use and impact within small firms: a survey of uk companies', international journal of information management, 18, 4,231-242. taylor, m. (1999), "the need for speed', communications international, 26, 2, 41-42. taylor, s. and todd, p.a. (1995) 'understanding information technology usage: a test of competing models', information systems research, 6, 2, 144 -176. thong, j.y.l. (1999), 'an integrated model of i/s adoption in small businesses', journal of management information systems, 15,4,, 187-214. thong, j. and yap, c.s. (1995), 'ceo characteristics, organisational characteristics, and information technology adoption in small business', omega,, 23,4 429-442. thurston, c. (2000), 'economic waves wash onto all shores', global finance, 14, 1, 121-123. van akkeren j. k. and cavaye a. l. m. (1999), 'factors affecting entry-level internet adoption by smes: an empirical study', proceedings from the australasian conference in information systems, 2, 1716-1728. weiers r. m. (1988), marketing research, (2nd edn.), prentice hall,, nj. wexler, j. (1999), 'cut the cord', upside, 11, 11, 201. yellow pages australia (1999) small business index: survey of computer technology and ecommerce in australian small and medium business. 16 virtuous hackers: developing ethical sensitivity in a community of practice paula roberts and jenny webber university of south australia paula.roberts@unisa.edu.au abstract it is estimated that losses due to computer break-ins by malicious 'crackers' (who might be external intruders or disgruntled employees intent on personal gain or revenge) are costing companies billions of dollars each year. but former hackers are now assisting the computer security industry to track down such intruders, and to develop sound security practices in order to ward off future attacks. it is argued that in recent times computer programming has moved from a craft-based, bricolage activity to a scientific approach which has led to a knowledge gap developing between the former fraternity of hackers and the computer security industry. the current inadequacies of the security industry have made this co-operation with hackers necessary but problematic, that is, should hackers who have developed their unique skill by breaking into company and government systems now be used for the rightful purposes of strengthening computer security? however, this relationship might also suggest that the hacker ethos, which has developed through the membership of a 'community of practice', and which has as its cornerstone the moral custodianship of computers and the information they contain, may represent the best way of developing ethical practice in the computer industry. introduction some former hackers, the virtuosos from the heyday of thrill-seeking computer break-ins, are now assisting system operators to establish and maintain sound security practices by testing system vulnerability with their own specialized knowledge, thus helping to foil the activities of a minority of malicious, criminal hackers known as 'crackers'. the size of the computer security problem is difficult to quantify, although some estimates suggest that, in the united states, it might amount to billions of dollars per year (behar, 1997), or 'about 4% of the gnp in most industrialized nations ... (a)ccording to some lloyd's underwriters and several confirming research studies' (taylor, 1999: 75). these investigations take into account losses due to both external intrusion and internal sabotage by current employees. the motivation for such break-ins may have its origins in the technical challenge of breaching a company's information security, or the potential for personal gain, notoriety, revenge, or advancement of ideological beliefs. the detection and prosecution of hackers has always been difficult because many companies who have been subject to security break-ins are oblivious to such damage. other businesses which have detected breaches of their information systems, have chosen not to disclose such events, fearing public embarrassment and the commercial damage associated with a loss of consumer confidence if the weaknesses of their corporate computer systems are revealed. in the united states an earlier government response to hacking was to introduce increasingly punitive measures for those few hackers who were detected and brought to account (sterling, 1992: roush, 1995), and to institute often unwinnable legal proceedings in efforts to establish public scapegoats in the hacker fraternity (denning 1991; sterling 1992). these court encounters were not successful in curbing hacking, but did provoke legal and ethical 'freedom of information' debates between those who argued for the privacy of information and others who supported free access to computer systems and their information (denning 1991). behar (1997) notes that security issues have come into sharper focus as greater corporate dependence on e-mail and networks has been matched by increasing amounts of economic espionage from crackers. this increased vulnerability to attack has been paralleled by government requirements that companies take responsibility for keeping their own data secure, in the knowledge they will be held liable also for losses incurred by 'downstream' companies who are damaged by the host company's lax computer security. in response to these developments, former hackers are now being enlisted by corporations and security companies to test the vulnerability of corporate information systems (sprenger 2000). but the use of hackers, whose own penetration of computer systems has mostly been accomplished by stealth and in breach of the law, raises ethical issues, particularly the question of whether the seemingly unethical should lead the computing profession to a new-found ethical emphasis on computer security and the custodianship of information. in response to this dilemma, it could be argued that the former hackers have an ethos that is grounded in the ethical use of computers. for example, for no apparent pecuniary gain, some hackers have spent large amounts of their own time in obsessively tracking down malicious intruders and bringing them to account for the damage they have caused not only to organizations but to the ethos of the former hacking fraternity (see stoll, 1991; shimomura, 1996). both stoll who pursued the german intruders, and shimomura who tracked down the infamous kevin mitnick, have documented their individual quests which represent years of personal persistence in bringing these crackers to justice, in the face of denial and indifference from u.s. defense department 172 systems managers and the american f.b.i. here is evidence of a hacker-based ethics, not imposed by professional codes of conduct, but one which is based on an intrinsic set of values and beliefs, inspired by an inherent respect for computers and the information they contain, and the abhorrence of those who do not share this respect. this new stage in the folklore of hacking suggests a 'bottom-up' formulation of computer ethics that is based less on universal principles than on an ethical sensitivity which develops through the use of computers. this personal ethos indicates also the appropriateness of not an 'add on' approach in the teaching of computer ethics, but of an encouragement of a sense of moral responsibility for computer systems from the earliest stages of a student's use of computers (roberts 1994; roberts & webber 1999). there are parallels here in the training of artisans in pre-industrial times, a training embedded in a craftsman/apprentice relationship, which encouraged not only a transfer of skills but also the development and then guardianship of the ethos of the craft for future generations. thus, contemporary computing 'apprentices' should be assisted in developing a moral sensitivity for the systems they use, and for the safety of the information which computers store. it could be argued that the remnants of the former hacker fraternity are revealing what computer ethicists may well have overlooked, that this sensitivity might best grow in a community of practice which has a common respect for the tools of the computer age. the ethos of hacking: computing as a valued human practice levy (1984) and roush (1995) describe an earlier generation of hackers who, from the 1960s, enjoyed stretching the capabilities of programmable systems, and who then moved on to penetrating corporate and academic computer systems with the aim of testing 'how far down the establishment's electronic corridors they could creep before anyone would take notice' (roush 1995:36). this first flush of hacking paled in later years with the growth of the internet and electronic mail, when networking meant that if 'two or three sysadmins found out about an operating system hole or a common system weakness, pretty soon everyone who cared knew about it and plugged it' (roush (1995: 36). at the same time law-enforcement crackdowns became an increasing deterrent to hackers who had outgrown the adolescent computing culture and had moved on to real-life concerns such as career and family responsibilities, making the risk of imprisonment for the sake of an ideal, a possibility which could no longer be contemplated. roush (1995) interviewed a number of present-day hackers and found they displayed a similar ethos to those former computer virtuosos whose 'common badge' was a sense of superiority to the bureaucrats whose systems they could so easily infiltrate. likewise, roush contends these new hackers are inspired by the thrill of unearthing system weaknesses, but they also set strict personal limits for their on-line adventures and hold in contempt those crackers who aim to damage a system and benefit from its vulnerability. roush reports that 'knightmare', a member of this new hacker fraternity has, in his book secrets of a super hacker, defined a 'set of ideals' which constitute an ethos of mature, 'responsible' hacking. this ethic is explained as 'never harming any computer, software, system or person, nor profiting from a "hack", but instead informing computer managers of their systems' weaknesses. thus a 'true hacker' has 'the ability to steal money, information, software, and hardware and to commit sabotage and espionage, but chooses to do none of these things' (roush, 1995:35). in examining the evolution of this hacker ethos, it is interesting to consider how might this adolescent computing elite, (most often working in isolation and classed as 'loners'), have gained such a strong sense of community and group identity which would lead to a collectively formed set of values and ideals. it is likely that these commonly held beliefs grew from communication across the personal and electronic hacker networks and then became incumbent upon the members of the hacker fraternity. thus an ethos grew which was intimately related to the specialized, high level computing expertise of these elites, and developed both from a personal desire for expertise, and from an obsessive love of computers. sherry turkle, a professor of computing at the massachusetts institute of technology, has studied the behaviour of undergraduate hackers in the mit computer labs where they lived and programmed computers for 22 hours a day, unwashed and unhealthy, and giving every impression that 'machines had replaced people in their lives' (turkic, 1988). however, this dark side of the hackers' craft of programming has been described elsewhere by turkle and her colleague, seymour papert, as 'bricolage', that sensitive yet playful, hobby-like approach to gaining skill which is the mark of a true craftsman (turkle & papert, 1990). this passion for their computing craft was accompanied in these students by a contempt for government and corporate computer systems which, in the hackers' view, constituted a misuse of information technology by contradicting the constitutional rights of citizens in respect of the freedom of information. thus the hacker community's ethos has reflected both an intrinsic desire to preserve and extend its own conduct and expertise, and an external motivation to attack and expose the vulnerability of institutionalized computer data systems. this latter motivation is based on a passionate distaste for private data stores, and an ideal of the 173 public freedom of information; beliefs which hackers, and their advocates, have defended in the courts (denning 1991). arnold (1997: 14-15) discusses the concept of sport as a valued and virtuous human practice which involves membership of a community and the development of a group identity. his analysis is useful in understanding both the evolution and the virtue of the hackers' ethos. arnold argues that, despite sport's 'more recent perverted and unsavoury connections' ... '(\v)hen sport is pursued for its own sake, its rules willingly followed and its finest conventions upheld, sport becomes an enobling and worthwhile form of life'. blum (1994: 146) (cited in arnold 1997: 15) discusses the relationship between virtue and community in the writings of maclntyre (1984: 10) and argues that virtues can only be learned and sustained in a community of practice. thus, the ethics of virtue are not seen as an alternative to the ethics of universality, but as complementary, that is, while based upon universal ethical principles the commonly held views of the practice community are indigenous and characteristic to its particular activities. furthermore, 'a practice, like a profession, is characterized as much by the way its participants conduct themselves as in the skills they develop and the purposes to which they are committed'. thus, if hacking is perceived by hackers as an elite practice, with internal goals and standards which are pursued in a moral way for their own sake, then a member of the hacking practice-community would be expected to only ever apply their elite technical expertise to responsible hacking, and not to malicious hacking. members would also be required to track down and expose deviates (crackers), who by their behaviour were damaging both the integrity of the practice-community, and the wider society. the hacker crackdown sterling (1992: xiii) documents government attempts to curb hacking which in 1990 involved 'a nationwide crackdown on illicit computer hackers, with arrests, criminal charges, one dramatic show trial, several guilty pleas, and huge confiscations of data all over the united states'. the 'show trial' to which sterling referred was the united states vs. craig neidorf legal proceedings against the hacker, neidorf, which are detailed by dorothy denning (1991). denning is a professional cryptographer and computer security expert who had for some time studied the hackers of the digital underground from an anthropological perspective. in this research, according to sterling (1992: 286), denning discovered that these computer-intruding hackers, who had been characterized as unethical, irresponsible, and a serious danger to society, did in fact have their own subculture and their own rules. they were not particularly well-considered rules, but they were, in fact, rules. basically, they didn't take money and they didn't break anything. denning was engaged by neidorf s defence team as an expert witness in a trial which set out to find a public scapegoat in the hacker fraternity and which charged neidorf with '10 felony counts carrying a maximum penalty of 65 years in prison' (denning, 1991: 26). the neidorf case was as much about constitutional rights of freedom of speech as it was hacking, and involved charges brought against neidorf s electronic newsletter, phrack, in which details of the bell telephone system had been published. it was subsequently proved that the bell information was not classified, and, in fact, was freely available to consumers, and, in this regard, phrack did not give away any secrets. the charges that phrack was also a primer for breaking into computer systems were also shown to be false, as denning was able to point to other publications, for example stoll's 1991 publication, which gave much more explicit information about systems' entry. the case against neidorf collapsed on the third day, but although neidorf was freed of all criminal charges, the trial cost of $100,000 was incurred by him and his family (denning, 1991: 28). one outcome of the trial was the establishment of the electronic frontier foundation whose aim is the protection of constitutional rights within the electronic media. however, as denning cautions, all published data carries with it the need for legal and ethical judgement, particularly 'in the case of hacker publications (where) the majority of readers are impressionable young people who are the foundation of the future. articles which encourage illegal break-ins or contain information obtained in this manner should not simply be dismissed as proper just because they are protected under first amendment rights' (denning, 1991: 31). further outcomes of the hacker crackdown were the individual national legislative procedures such as the u.k. computer misuse act, which established punitive measures for a range of hacking offences, including the introduction of viruses. however, with so far no international agreement on legislation, and the unlikely possibility of enforcement of such legislation, as the love bug virus incident in 2000 showed, malicious or foolhardy hacking is not likely to be contained by legislative procedures alone. 174 sleeping with the enemy: the digital underground and the computer security industry roush (1995: 38) argues that as computer-related crime is growing more sophisticated, more varied, and more costly to society, information-security experts and law-enforcement agencies need to turn to the expertise of 'good' hackers as a significant knowledge source to assist their attempts to stem the onslaught of destructive computer intrusions. taylor (1999) contends that the computer security problem is more complex than simply recruiting hackers. firstly the vulnerability of current computer systems is due in great measure to the neglect by academia and industry of research and development in computer security. in particular, security flaws, (and the reasons they are not eliminated) reflect the gulf between programming's expectations and its achievements. crackers, taking advantage of this situation, are simply using trivial holes left by inexperienced programmers who were not taught the limitations of computer systems. in academia. where computer security is under-theorized apart from cryptography, the situation now is 'like teaching how to fly airplanes but not teaching pilots that engines sometimes fail', (cohen, in taylor 1999:87). also under-valued by industry and academia is practical knowledge of the vulnerability of computer systems, wisdom gained only by hands-on experience and experimentation. as computing has evolved from a craft culture to a scientific reliance on standardized procedures, deficiencies in computer security and software development have made apparent the problems caused by this change from a 'craft' to a 'scientific' approach in the education of computing personnel. the scientific approach emphasizes a formal design methodology and internal consistency that aims to prove the efficacy of a program at the design stage, while the craft approach, (typical of the attitudes towards computing of the early creative, if less-than-disciplined, hackers) is more attuned to the realities of the software's ultimate use. the departure from a craft approach may have contributed to marginalizing hackers away from a core of influence in computing as industry has moved to more rigorously researched knowledge, but the spurning of the hackers' potential contribution to computer security may relate also to a developing ethical sensitivity within the industry. whilst the knowledge of the early hackers was once valued despite its doubtful methods of acquisition, what was then accepted in the academic pursuit of knowledge appears to be no longer tolerable in the presentday academic or commercial environments of computing. a mainstay of the hacker ethos is that information should be free, and therefore illicit access to computer systems is not inherently wrong, but others question the ethics of this hacker activity, for example, thompson (1984: 763) argues 'the acts performed by these kids are vandalism at best and probably trespass and theft at worst'. however, others contend that the likelihood of harm from the hackers is exaggerated, and is outweighed by their potential to provide useful information about security weaknesses. for example, kapor (1991), murray (1992) and spafford (1992) argue for the ethical, instrumental value and use of the hackers' experience, particularly their knowledge of the psychology of crackers and other computer felons which will aid the security industry and thereby bring great benefits to society. these opposing views are reflected in enduring arguments at various levels, ethical and practical, which are common to information ethics in general, and are matters on which academia, industry and society are yet to reach a set of agreed standards. because of such disputes, stichler (1998: 180) notes that information ethics, has at times become moribund. in similar vein, maclntyre (1984: 6) has argued, 'it is precisely because there is in our society no established way of deciding between these claims that moral argument appears to be necessarily interminable... hence perhaps the slightly shrill tone of so much moral debate'. stichler (1998: 179-180) provides a pointer useful to the hacker issue: in the debate over the ethics of information distribution, utilitarian positions focus on maximising social benefits and tend to favour a market system based on private ownership; while deontological positions focus on the right to information access and tend to favour a more socialistic system emphasising equality of distribution and public ownership. in cases where rights conflict, where one person's (or corporation's) intellectual property rights conflict with another's right to access, deontologists often have no convincing way of deciding the right course of action and are thus forced to fall back on utilitarian considerations as the only practical means of reaching a decision. for the argument in this paper, it is useful to return to maclntyre (1984) and his discussion of the virtues, particularly its application for an understanding of the virtues of the good hacker community. maclntyre argues that 'the essential function of the virtues is clear. without them, without justice, courage and truthfulness, practices could not resist the corrupting power of institutions' (1984: 194). the origins of the activity of hacking itself stem from the craft-like, bricoleur quality of programming, and it is the hackers' holistic approach to computing which explains hacking's lasting appeal to subsequent generations (taylor, 1999:88). the fundamental disagreement over the implications of the craft aspects of hacking is one of 175 the underlying factors in the dispute between the digital underground and the computer security industry, and, at the same time, the strongest reason why hacking is likely to survive in some form or another, even as programming develops towards more science-based methods. more important still is hacking's contribution to an applied computer ethics which is based not only on universal principles but which has, as its cornerstone, a moral respect for computers and their information, which grows as computing expertise develops. this dichotomy mirrors the skills/ethos base of the craft guilds of pre-industrial times, which was handed down to future generations through the craftsman/apprentice relationship. this is a strong argument for an integrated approach to the teaching of computing ethics in mainstream computing subjects (roberts, 1994; roberts & webber, 1999), so that as a skill develops it is accompanied by a growing moral sensitivity for the social responsibility that skill also entails. conclusion : hacking visions of a new computer ethics? the titles of two early, but still prominent, primers on hacking, levy's hackers, heroes of the computer revolution (1984) and sterling's the hacker crackdown (1992) reflect the changing attitudes towards hackers over this period. from a former tolerant acceptance of adolescent precocious computing skill, attitudes hardened with an awareness in the late 1980s and early 1990s of the sinister threat to computer systems posed by a more recent development in hacking, the actions of malicious crackers. however, law enforcement and its increased penalties for hacking have proved ineffectual and very few cases of illegal computer entry are detected or reported. almost ten years after the publication of sterling's book, the computing profession and society are still faced with the ethical dilemma of whether the hackers' unique knowledge of the vulnerability of computing systems (which has been gained by the seemingly unethical acts of system break-ins) should be harnessed by the computer security industry for the common good. obviously, there is no easy theoretical solution to this quandary. however, in practice, former hackers are now working to halt computer crime by joining forces with the security industry (see, for example, sprenger, 2000). these efforts may not yet bring about public respectability for the hacker fraternity, but the hidden value of the hackers' personal ethos, which values computers and their ethical use, may provide the substance for a new computer ethics which transcends professional codes and guides the general computer user. this appears society's best hope in limiting the damage and cost of malicious computer break-ins. references arnold, p.j. (1997) sport, ethics and education, london: cassell. behar, r. (1997) 'who's reading your e-mail'. time, february 3, 64-67. blum, l.a. (1994) moral perception and particularity. cambridge: cambridge univ press. denning, d.e. (1991) 'united states vs craig neidorf: a debate on electronic publishing, constitutional rights and hacking'. communications of the acm, vol. 34, 3:24-32. kapor, m. (1991) 'civil liberties in cyberspace: when does hacking turn from an exercise of civil liberties into crime?' scientific american, september. levy, s. (1984) hackers: heroes of the computer revolution. harmondsworth: penguin. maclntyre, a.c. (1984) after virtue: a study in moral theory, 2nd edn. notre dame: notre dame university press. murray, w. (1992) 'on computer security and public trust' in t.w. bynum, w. maner, & j.l. fodor, (eds.) teaching computer ethics. new haven, ct: research center on computing and society. roberts, p.m. (1994) 'the place and pedagogy of teaching ethics in the computing curriculum' australian educational computing, may. roberts, p.m. & webber, j. (1999) 'visual truth in the digital age: towards a protocol for image ethics' australian computer journal, vol. 31,3: 78-82. roush, w. (1995) 'hackers: taking a byte out of computer crime'. technology review, april: 32-40. shimomura, t. (1996) take-down: the pursuit and capture of kevin mitnick, america's most wanted computer outlaw by the man who did it. new york: hyperion. spafford, e.h. (1992) 'are computer hacker break-ins ethical? journal of systems software. 17:41-47. sprenger, p. (2000) 'tiger teammates, hacking bright' information age, august/sept: 32. sterling, b. (1994) the hacker crackdown: law and disorder on the electronic frontier. harmondsworth, uk: penguin. stichler, r.n. (1998) 'ethics in the information market'. in r.n. stichler & r. hauptman (eds.) ethics, information and technology readings, pp. 169-183. jefferson, nc: mcfarland & co. stoll, c. (1991) the cuckoo's egg. new york: doubleday. taylor, p.a. (1999) hackers: crime in the digital sublime. london: routledge. 176 thompson, k. (1984) 'reflections on trusting trust'. turing award lecture. communications of the acm. vol. 27, 8: 761-3. turkic, s. (1988) 'computational reticence: why women fear the intimate machine'. in c. kramarae, (ed.) technology and women's voices. new york: routledge & kegan paul. turkic, s. & papert, s. (1990) 'epistemological pluralism: styles and voices within the computer culture'. signs: journal of women in culture and society, vol. 16 ,1 . 177 microsoft word 13 1 marina.doc ajis vol 13, no. 1 september 2005 239 the virtual shopping experience: using virtual presence to motivate online shopping carolyn ye-phern chin, south-east water, melbourne, australia email: carolyn.chin@sewl.com.au paula m.c. swatman, university of south australia, adelaide, australia email: paula.swatman@unisa.edu.au abstract online shopping has thus far tended to be a niche business – highly successful in selling digital products such as shares, software and, increasingly, music and films, it has been less successful in persuading the purchasers of ‘traditional’ goods such as cars, clothes, toiletries, or household appliances to forsake their physical retailers and move into cyberspace. in this wide-ranging review paper we investigate the issue of the virtual experience – endeavouring to understand what is needed for a successful ‘shopping experience’ online and what the possible obstacles or pitfalls along the way might be. we initially introduce the concepts of virtual presence (the sense of ‘being there’) and virtual reality, discussing the possible roles both can play in providing a solution to the problem of effective online shopping. we then consider the experience economy, a concept which encapsulates many of the issues related to the problem of online shopping and which suggests ways in which online retailers can enhance the effectiveness of their sites by means of a virtual ‘experience’. having set the scene for online shopping, we discuss etailing today in terms of direct product experience and the opportunities which cyber-shopping offers to replicate this process. finally, we identify some of the possibilities and problems of online shopping today, illustrating the current status of virtual presence in retailing with two micro-cases of success and failure. keywords: online shopping, innovative value added services, presence, virtual presence, virtual reality, experience economy. introduction although the internet has provided consumers with global access to an unlimited array of goods available for purchase, overall purchasing levels and, in particular, the online purchase of goods which normally require physical presence, such as clothing, footware, cars and other, similar ‘tactile’ products is a hurdle to the uptake of truly widespread online shopping. hou and rego (2002) note that the decision to shop online depends to a large extent on the shopping experience the consumer is willing to accept and point out that online shopping still lacks social interactivity. as far back as 1972, tauber extended mcclelland’s primary social motives (1961) into a more specialised set of social motives driving consumer purchasing behaviour: (i) social experience; (ii) communication with others; (iii) peer group attraction; (iv) status and authority; and (v) the pleasure of bargaining. indeed, the social aspects of shopping are a major attraction for many people. a qualitative study of shopper motivation in the uk (dennis et al., 2002) found that the social dimension was a very important factor for many of the study’s participants. as one respondent noted: “shopping has become going out, meeting friends, starbucks, pret à manger [a british restaurant chain]”. online shopping fits westbrook and black’s (1985) three shopping motivation concepts – diversion from routine activities, sensory stimulation and learning about new trends – to a large extent, but tauber’s five fundamental motives are more closely linked to human social contact and ajis vol 13, no. 1 september 2005 240 consequently harder to match. if it is to be truly successful and fulfil its promise, online shopping must find some way to fulfil the human needs currently met by face-to-face shopping activities. this review paper discusses the ways in which “virtual presence” might offer a partial solution to this problem, by enabling online shoppers to interact with and modify (or enhance) their shopping experience. we begin by introducing the literature relating to virtual presence and virtual reality, discussing the possible roles of these concepts in providing a solution to the problem of effective online shopping. we continue our discussion by considering the ways in which virtual presence affects and is affected by the online shopping ‘experience’ – which must provide something as satisfying as today’s physical shopping experience – and extend this discussion by considering the direct shopping experience, in which physical touch and test must now be replaced by its virtual equivalent. we next consider online retailing today, discussing motivations for buying online. we revisit the issue of the online experience, using two micro-cases of a high-profile failure and a less high-profile success to show what is possible in etailing today and, finally, we discuss the pros and cons of vr-enhanced cyber-shopping and the likelihood of its success in today’s environment. presence, virtual presence and virtual reality virtual reality and other emerging technologies are designed to give the user a kind of mediated experience which creates a strong sense of presence (lombard & ditton, 1997). ijsselsteijn (2002, p.245) points out that in one sense all reality is virtual, being constructed through the observer’s cognitive and sensory abilities: “reality is not ‘out there’, it is what we take to be ‘out there’ … one can produce illusions which the brain will be unable to discriminate from physical reality”. steuer (1992) defined virtual reality as a particular type of experience which involves the sense of “presence” – or the sense of being in a particular place. an enhanced sense of presence is central to the use of virtual reality in a commercial environment, leading to enhanced usefulness and profitability. furthermore, a better understanding of what presence is can save valuable time and money – and improve the end-product in the design of current and new media technologies (lombard & ditton 1997). but what might be needed to create an effective sense of presence? table 1 summarises the work of five groups of researchers who have investigated the factors which make up presence: ajis vol 13, no. 1 september 2005 241 kim & biocca, 1997 schubert et al., 1999 lessiter et al., 2000 enlund, 2000; knudsen, 2002 arrival the sense of “being there” in the virtual environment spatial presence – the sense of “being there” in the virtual environment physical space – the sense of “being there” in the virtual environment sensory environment – vividness, company of others involvement – concentration only on the virtual space engagement – intensity of the experience individual preconditions – suspension of disbelief departure – the sense of “not being” in the physical environment realness – believability of the virtual environment naturalness – believability of the virtual environment content characteristics – believability of the virtual environment negative effects – dizziness, disorientation, nausea table 1. a comparison of “sense of presence” factor analysis studies all five groups of authors found that presence is likely to be related not only to the user’s sense of a physical, spatial environment, but also to his/her personal evaluation of the appeal and believability of both the environment and its content (in other words, the entire experience). kim & biocca (1997), importantly, concluded that simply feeling present in the virtual environment was not sufficient to create a strong memory of the experience – only when users no longer felt present in their physical environment was there an effect on memory (and hence persuasion). unlike traditional in-store retailing, where shopping information is conveyed to consumers through several channels, including: the store environment itself, product trial, product offering or service (schiffman et al., 1977), b2c ecommerce relies solely on the web interface for communication and for managing customer relationships (jiang & benbasat, 2002; benbasat & desanctis, 2000). two issues are particularly relevant to relationship management in an online shopping environment: • para-social presence (jiang & benbasat, 2002; kumar & benbasat, 2002) – which relates to the distance between the customer and the company; and • product perception (jiang & benbasat, 2002; jarvenpaa & todd, 1997) – which refers to the distance between the customer and the product. while both these issues are clearly important in any overall discussion of relationship management, this paper focuses on product perception. although there has been much excitement over the potential of “interactivity” on the internet, very little is understood about exactly what creates a sense of presence or interactivity, and what impact this has on user behaviour. businesses are spending millions of dollars to add games, animation and personalisation tools to their web sites without really knowing what effect this has on their customers (klein, 2001). ajis vol 13, no. 1 september 2005 242 essentially, the question to be resolved is what is meant by “presence” in a web environment. despite the popularity and academic interest in this topic, the centrality and importance of presence has not yet been studied in much depth. lombard and ditton (1997) noted that the work which had thus far been done was fragmentary and unsystematic, partly because the people interested in presence came from many different academic backgrounds. they added that the more specific, focused research conducted by private industry and government had mostly remained proprietary. media alternatives can vary widely in terms of the quality and quantity of the product information they can transmit to consumers, compared with what is generally considered the consumer’s best source of product information – direct product experience (klein 2001). the traditional electronic media, such as radio and television, are unable to deliver customised product information, but the internet has the potential to deliver product information in both personally-customised and realistic ways, so that the receiver may interpret the information much more as though it came from their own direct experience than they can with advertising from traditional media sources (klein, 2001; hou and rego, 2002). in recent years, the improvement of ecommerce interface design has led to easier navigation, more attractive graphic interfaces and more powerful plug-in components (jiang & benbasat, 2002; lohse & spiller, 1997; gehrke & turban, 1999; kim & yoo, 2000; nielsen, 1999), although researchers are still doubtful about the effectiveness of online shopping (jiang & benbasat, 2002; wolfinbarger & gilly, 2001; rose et al., 1999; bhatnagar et al., 2000). the argument is that the web interface is constrained, since online consumers can only passively receive the product information presented, without being able to feel, touch, or try online products, which significantly limits their perception of the products they purchase in this way (jiang & benbasat, 2002). to address this issue, a variety of forms of improved product presentation have been suggested. for instance, online consumers are currently able to simulate direct product experience (jiang & benbasat, 2002; kempf & smith, 1998) either by means of 3d or virtual reality (jiang & benbasat, 2002; li et al., 2000; 2002a; 2002b; peek, 1997). virtual reality or virtual product experience allows the consumer to interact with online products and experience a much wider range of those products’ features (jiang & benbasat, 2002; ryan, 2001). earlier exploratory studies have suggested that virtual reality has the potential to improve the consumers’ product knowledge and brand attitude, including enhancing their purchase intentions (jiang & benbasat, 2002; li et al., 2002, 2002a, 2002b; daugherty et al., 2002). the use of virtual product models can be extended to product development, customisation, marketing, manufacturing, and selling of products for internet based virtual companies (pulli, antoniac & hickey, 1999). as we noted earlier, however, both knudsen (2002) and enlund (2000) recommend that the producers of virtual reality continue to concentrate on the technical skills upon which the improved sensory user-environment must be based, as well as on learning how to produce quality content which will support the creation of a strong sense of presence and reality. kim & biocca (1997) point out that when users feel “present” in a virtual environment, they are also more likely to be persuaded to purchase or use the product being sold. virtual prototyping models offer an entirely new perspective for ecommerce on the internet as they provide a strong sense of reality through meaningful interaction, which allows the model of a product to be tested. this is because all-digital prototypes resemble their physical counterparts as closely as possible with visual three-dimensional images, ajis vol 13, no. 1 september 2005 243 behavioural, haptic and auditory characteristics. online trial of products can also be offered where customers can browse in a virtual salesroom to test operations of a virtual prototype, simulated to mimic its physical counterpart (pulli, antoniac & hickey, 1999). of course, the success of such approaches depends, to a large extent, on how convincing the user’s experience with the virtual prototype is. how “realistic” is virtual reality today? the “experience economy” pine and gilmore (1999), in their seminal work which first brought the concept of the “experience economy” to public notice, observe that price competition is no longer sufficient to sustain growth and profitability – and conclude that companies must find new ways of adding value to their business. in its simplest sense, the concept can be described by saying that consumers are now buying experiences as well as goods and services. voss (2003, p4), in a report of a major piece of empirical work which identifies some of the key management and business trends in experience, states that: “creating outstanding and memorable experiences has become central … to an increasing number of businesses, as companies seek to build the emotional involvement that goes with them”. pine and gilmore believe that experiences are a fourth economic offering, following on from commodities, goods and services (see figure 1). each successive offering greatly increases in value, because the buyer finds each more relevant to what s/he truly wants. and because companies stage so many different kinds of experiences, they can more easily differentiate their offerings and hence charge a premium price based on the distinctive value provided. pine and gilmore add that businesses need a strategy for managing these four dimensions of value creation. voss (2003) points out that experiences create “emotional responses and connections with products and brands which are difficult to build in any other way”, adding that one of the opportunities thus provided is the chance to try out products, which allows the shopper to build a form of involvement with the product. pine and gilmore believe that experiences have always existed, but that consumers, businesses and economists previously lumped them into the service sector along with such unexciting activities as dry cleaning, auto repair, telephone access or wholesale distribution. pine and gilmore point out that when customers buy a service, they purchase a set of intangible activities carried out on their behalf. but when they buy an experience, they pay to spend time enjoying a series of memorable events staged by the company – as in a theatrical play – which engage them in a personal way. ajis vol 13, no. 1 september 2005 244 figure 1. the progression of economic value (pine & gilmore, 1999, p.22) the answer provided by the experience economy thus lies partly in technology, which powers so many experiences, and partly in the increasingly competitive intensity which drives the ongoing search for differentiation. bruns (2002) suggests that the experience economy not only offers practical insights into how business can respond more effectively to the reshaping of demand, but also provides a framework for thinking about how advanced technologies can be used in the online environment to enhance experience. these authors all highlight the need to engage online shoppers with a memorable shopping experience, rather than simply selling goods and services on the internet. the question which naturally emerges is just how that memorable experience is to be obtained by the online shopper. in the following sections, we discuss the possibility that virtual reality may be able to offer at least a partial solution to the problem of making online shopping into a pleasant and memorable experience. direct product experience direct product experience (also known as product trial), describes a consumer’s direct feel, touch and trial of a product. it is generally believed to be one of the most effective, yet most costly, ways to introduce a new product (jiang & benbasat, 2002; kotler, 1988). there are several reasons why product trial should be taken seriously by researchers interested in online retailing success factors (jiang & benbasat, 2002; kempf & smith, 1998; hoch & deighton, 1989). firstly, direct experience involves multiple sensory cues, such as vision, feel, smell, taste and sound. when combined together, these cues form a ajis vol 13, no. 1 september 2005 245 vivid, informative and impressive presentation of products. secondly, as consumers are involved in the direct experience, they will be more motivated to evaluate products and their attention and engagement should be more aroused (jiang & benbasat, 2002). in addition, as all product information is generated by the consumers themselves during product trial, it possesses greater credibility and trustworthiness than information supplied by manufacturers or retailers. learning from direct experience is a self-controlled experience and consumers can thus control the way they receive, store, and process product information, leading to more effective management of product information (jiang & benbasat, 2002). studies undertaken by smith and swinyard (1983; 1988) found that belief and attitude confidence was significantly higher when consumers were exposed to product sampling only, than when exposed to advertising only; while marks & kamins (1988) also found that attitude formed through direct product experience was held more confidently and was less susceptible to change than advertising. other research (jiang & benbasat, 2002) has suggested that the effect of direct product experiences is dependent on product attribute categories. nelson (1974) had previously identified these as: • search attributes: those whose information are conveyed better through second-hand sources, such as catalogues, advertising or word-of-mouth, than direct trial of products; and • experience attributes (with an obvious, though in this domain of research unstated, link to the work of pine & gilbert (1999), voss (2003) and others). making use of this categorisation, wright and lynch (1995) found that direct product experience led to higher belief confidence and accessibility of experience attributes than advertising, while advertising resulted in higher belief confidence and accessibility of search attributes. a growing number of researchers are now investigating the impact of 3d virtual product experience as a substitute for direct product experience, especially in relation to the influence of virtual advertisements (see, for example, daugherty et al., 2001; 2002; jiang & benbasat, 2002; li et al., 2001; 2002a; 2002b: peck & childers, 2003). the major questions which remain, therefore, are firstly how attractive online shopping is overall, whether online consumers are willing to experiment with virtual presentation of their shopping experience and, most importantly, whether this experience is sufficiently rich to provide a substitute for the “real” world. understanding online shopping behaviour despite all the hype surrounding internet shopping, a large proportion of online shoppers still do not purchase online (cgey, 2000). this cap gemini/ernst & young survey of canadian online shoppers found that the top five barriers to purchasing online goods or services were: (i) security concerns (22%), (ii) too impersonal (20%), (iii) time consuming/frustrating (13%), (iv) difficulty navigating sites (11%), and (v) not enough or too much information (10%) (cgey, 2000). a number of commercial studies have suggested that online shopping is simply not taking off as predicted. for example, van putten (2000) cites a forrester research study which found that 40%-60% of intended online purchases were abandoned once shoppers reached ajis vol 13, no. 1 september 2005 246 the check-out page; and a major study of 12,000 us consumers by the boston group (2000) found that 28% of attempted online purchases failed, and four out of five consumers purchasing online had experienced at least one failed purchase attempt. and this problem is still with us – elgin (2003) found that google’s new shopping service, froogle, was not showing the rapid growth its highly successful parent had expected and hoped for. why is online shopping growing so much more slowly than forecast, long after the dot.com crash? does the answer lie in the experience consumers seek when they shop online and the limited ability of today’s online shopping environments to fulfil those needs and wants? smith (2003) explains that understanding motivations is crucial when considering why people shop online, believing that we need an understanding of people’s basic motivations for shopping in general as well as, more specifically, the motivators for collecting information and shopping on the internet. he wonders why we make purchases – and asks whether all shopping activities are motivated by the same behaviour. tauber (1972) identified a variety of personal and social motives for shopping behaviour (see table 3). these motives are often disguised, or perhaps are even unknown to the shopper him/herself – but they have an effect on the times we choose to shop, the stores we choose to enter, and the brands of products and services we choose to purchase. personal motives for shopping social motives for shopping role playing social experience recreation communication with others self gratification peer group attraction learning about new trends status and authority sensory stimulation the pleasure of bargaining table 3. a summary of tauber’s (1972) personal and social shopping motives these motives are likely to be as relevant to online shopping as to “traditional” shopping activities. hoffman et al. (2003) noted that many of the well-known b2c retail failures had been attempting to replicate offline shopping patterns online, while success stories often included those etailers taking advantage of ‘webby’ applications such as mp3 file sharing (peer-to-peer exchanges), instant messaging, or c2c models. this finding is similar to that of albers et al. (2002), whose study of 10 successful online b2c companies indicated that profitable companies in this sector developed internet-oriented business models right from the start – offering digitalised products and significant added-value for consumers, rather than replicating offline business models. but before we write-off the idea of selling conventional products online completely, it is worth considering other possible influences affecting online retailing: • age-related online consumption behaviour suggests that when “younger surfers who have grown up on the net become full-fledged consumers, experiential benefits are likely to become more desirable” (wolfinbarger & gilly, 2001, p.52). peck & childers (2003) identified the high ‘need-for-touch’ (nft) group, who were more likely to purchase products they could touch – either physically, or by using haptic touchsubstitutes. • the online interface relates to the way in which firms such as amazon and autobytel have managed to expand globally by creating a standardised, transaction-oriented interface, so that users always know what to expect (cole & o’keefe 2000). since these pure-play online ajis vol 13, no. 1 september 2005 247 retailers primarily focus on the sale of ‘traditional’ physical goods, it would appear that their interface is generally successful – whether individual shoppers like it or not. a number of researchers have begun to investigate the impact which the web site’s user interface has on retail success: • lohse & spiller (1998; 1999) and spiller & lohse (1998) confirmed a strong link between a web site’s b2c interface and the volume of traffic and, more importantly, sales achieved by the site; • eroglu, machleit & davis (2001) found that web site ‘stickiness’ was influenced by the virtual store’s atmosphere; • wolfinbarger & gilly (2003) found that the four issues of website design, product fulfilment & reliability, web site privacy & security awareness, and customer service all have a strong positive link with the customer’s belief in the quality of the retail site; and with their satisfaction, loyalty and overall attitude toward the etailer; and • peck and childers (2003) research into the addition of haptic features to enhance the shopping experience suggest that visual cues alone are not sufficient for long-term success in cyber-shopping experiences. in discussing the role of ‘affect’ in human-computer interaction, hudlicka (2003) notes that: “in some research communities we no longer even speak of users and machines as separate entities, but rather of collaborative systems, integrated human-machine systems and joint cognitive systems” (p.2). she makes the very relevant point that online agents capable of interacting with humans in a seamless and adaptive manner will need to be able to recognise and respond to user affect. this, it seems to us, sums up the discussion about the most effective environment for online shopping rather neatly – only when online shoppers are positively affected by their virtual shopping experience will they turn their backs on ‘traditional’ retail environments and flock to the internet in huge numbers. virtual presence in retailing today the range of virtual reality-enhanced online shopping venues today is still comparatively limited, although a number of experiments in applying virtual reality to online shopping have already been attempted. indeed, this may be part of the problem – highly publicised failures, like that of boo.com (the online apparel retailer discussed below), have made many people wary of using sophisticated technologies like virtual reality to enhance online fashion retailing or, indeed, any form of online retailing. although the examples currently available are still limited, there is much to be learned from what is being achieved and from the way in which it is implemented – and perhaps even more to be learned from the failures. the following dot-points briefly describe two examples of online virtual presence: a spectacular failure, and a less well-known software solution being used by a number of fashion etailers to enhance the buying experience for their customers with considerable success: • boo.com – presented itself as a cool online fashion retailer which understood the needs of its young customers and could offer them not only the world’s largest online catalogue of fashion images (fully rotational and impressively vr-enhanced), but also the ability to “try” ajis vol 13, no. 1 september 2005 248 apparel items on personal electronic avatars; a sophisticated virtual shopping assistant to guide purchases; and links between purchasing history and new product arrivals. after a two-year development period, the site belatedly went live in november 1999 and, despite its many technical problems, was rated highly by the media and its customers (stockport, kunnath and seddick, 2001). however the strain of producing integrated systems ultimately proved too taxing. the launch date slipped further and further and, when the site did go live, it was still not really ready – too slow, too complicated and far too prone to error (stockport, 2000). boo.com closed its virtual doors only six months after first opening them. • my virtual model a more modest, but considerably more successful, approach to vr-assisted fashion etailing is my virtual model™ (mvm), which was initially released in 1997. more than three million models have since been created using mvm software and the company’s web site claims that shoppers using these solutions spend more, buy more and return far fewer items – resulting directly in reduced shipping and handling costs (itac, 2001). once a would-be shopper has created and registered his/her model, it can be re-used on the web sites of other mvm etailers. lands’ end, the best-known mvm user, says that the software has enabled the company to gain a better understanding of purchasers’ preferences and intentions, since these are automatically stored in a demographic database and can be re-used by other applications, including planning and procurement (edwards, 2000). the modest, but very real, success of my virtual model™ at all adopting web sites suggests that online shoppers are ready for a more interesting and exciting experience than they are currently able to obtain. conclusion online shopping is likely to be driven by the same (or similar) motives as is face-to-face shopping. beyond its obvious benefit in terms of convenience 21 , online shopping must satisfy the personal and social goals identified by tauber (1972) and westbrook & black (1985) – it must, in the words of pine & gilmore (1999) offer an ‘experience’ sufficiently desirable to warrant its use. virtual reality has the potential to make shopping just such an experience – not merely in terms of providing a virtual substitute for the human shop assistant, but also in terms of its ability to recognise and provide what each individual users seeks from his/her shopping. just how far that experience should go is something which will be limited by technology and by the user’s willingness to immerse him/herself in the experience. users are becoming increasingly sophisticated in their expectations of computer technology and hudlicka (2003) points out that this growing sophistication, combined with escalating technical progress, is leading to a redefinition of what is understood by effective and desirable human-computer interaction. in other words, miss boo may have been ahead of 21 this paper does not touch upon the potential benefits of online shopping for those temporarily or permanently unable to go shopping physically, although this is clearly a matter of no small importance to those affected. ajis vol 13, no. 1 september 2005 249 her time, technically speaking, but it will probably not be long before such virtual assistants become relatively commonplace in b2c etailing. how much further immersion needs to go is still unclear. peck & childers (2003) believe that without haptic extensions, many forms of online shopping are unlikely to be truly successful. they suggest that while those shoppers whose sense of touch is not strongly developed (low-nft shoppers, whose motivation and/or ability to obtain haptic information is minimal) may not require such enhancements, while high-nft shoppers “may be difficult to convert to users of non-touch media” (p.48). in this paper we have explored the elements of virtual presence and online shopping, seeking to find a way to replicate the essential shopping experience in a virtual environment. voss’s (2003) wide-ranging study of the experience economy found that successful companies across many sectors created “outstanding and memorable experiences” for their customers and users. the limited, but encouraging, success of my virtual model for apparel stores across the usa and europe suggests that, even in its present state, a virtual avatar offers a real advantage to those engaged in the very challenging task of selling clothing online. and apparel is by no means the only industry to make this discovery – car manufacturers and dealers have also been investing in 3d technology to help them break into the tough, but potentially lucrative, online automotive market. mercedes has been using vr for nearly four years to offer would-be buyers of its innovative geländerwagen suvs (www.gwagen.com) a virtual test drive (edwards, 2000). the california start-up realitybuy.com (www.realitybuy.com), beginning with the b2b market, has now successfully created cheap, readily-created virtual objects for b2c furniture and office suppliers, motor-bike dealers, computer manufacturers and jewellery stores (to name only a few of their applications). online consumers are finding the virtual shopping experience more and more effective. virtual reality alone is not yet sufficiently technically advanced to create a true sense of presence, of “being there”. but the addition of such features as simulations which show just what the product could do; recommender systems which show what other buyers have liked; and chat rooms which provide the opportunity to discuss the product with both the sellers themselves and with other buyers go a long way towards creating a memorable experience which will persuade online shoppers to buy. this literature-based study as yet just hints at the likely uses of vr for etailing –the next stages of this research project will involve the gathering and analysis of empirical data and will provide a solid foundation for predictions of what awaits online shoppers over the next few years. references albers, s., panten, g. and schäfers, b. (2002). die ecommerce-gewinner: wie unternehmen im web profitabel wurden. 10 erfolgsgeschichten aus erster hand, f.a.z.-institut, frankfurt, url: http://www.faz-institut.de/in/intemplates/fazinst/doc.asp?doc={d3da315f-a41347fa-85a1-614af0bf2af7}&rub={9a874892-b40d-4eed-b5bf-5781d1129973}, [last accessed: 23/08/2003]. benbasat, i. & desanctis, g. (2000). “communication challenges: a value network perspective”, in g. dickson and g. desantis (eds.): information technology and the future enterprise: new models for managers. prentice-hall, englewood cliffs, nj. bhatnagar, a., misra, s. and rao, h.r. (2000). “on risk, convenience, and internet shopping behaviour”, communications of the acm, vol 43 no. 11, pp98-105. ajis vol 13, no. 1 september 2005 250 biocca, f., & levy, m. r. (1995). communication in the age of virtual reality, lawrence erlbaum associates, hillsdale, nj. boston group (2000). “winning the online consumer: insights into online consumer behaviour”, boston consulting group research report, url: http://www.bcg.com/publications/files/winning_online_consumer_insights_rep ort_mar00.pdf [last accessed: 21/08/2003] bruns, b. (2001). “nanotechnology and the experience economy”, nanodot: http://nanodot.org/article.pl?sid=01/02/10/063220 [last accessed 22/5/03] cgey (2000). “canadians kick tires online – but few complete the deal”. url: www.ca.cgey.com/news/press_releases/pr_march21_00.htm [last accessed 22/5/03] cole, m. and o’keefe, r.m. (2000). “conceptualising the dynamics of globalisation and culture in electronic commerce”, journal of global information technology management, vol. 3, pp4-17. daugherty, t., li, h. and biocca, f. (2002). “experiential ecommerce: a summary of research investigating the impact of virtual experience on consumers learning”, in c. haugtvedt, k. machleit and r. yalch (eds) (2002) online consumer psychology: understanding influencing consumer behaviour in the virtual world, lawrence erlbaum associates. dennis c., patel, t., king, t., and hilton, j. (2002). “qualitative studies of shoppers’ motivations”, 9 th international conference on recent advances in retailing and services science, eirass, heidelberg, germany, august, url: http://dallas.brunel.ac.uk/modres_uploads/bs3416s/qualmotivation.doc [last accessed: 11/08/2003] edwards, j. (2000). “3-d finally gets serious”, cio.com.magazine, march 1, url: http://www.cio.com/archive/030100/et.html [last accessed: 23/08/2003] elgin, b. (2003). “google’s offspring: taking baby steps”, business week online, march 19, url: http://www.businessweek.com/technology/content/mar2003/tc20030319_7982_tc016.ht m, [last accessed: 21/08/2003]. enlund, n. (2000). “the production of presence – distance techniques in education”, publishing and art, acs’2000 proceedings, szczecin. eroglu, s.a., machleit, a.k. and david, l.m. (2001). „atmospheric qualities of online retailing: a conceptual model and implications”, journal of business research, vol. 54, issue 2, nov., pp177-184. gehrke, d. and turban, e.(1999). “determinants of successful website design: relative importance and recommendations for effectiveness”, proceedings of the 32 nd hawaii international conference on system sciences. hoch, s.j. & deighton, j. (1989). managing what consumers learn from experience, journal of marketing, vol. 53, iss.2; pp1-20. hoffman, d.l., novak, t.p. and duhachek, a. (2003). “the influence of goal-directed and experiential activities on online flow experiences”, journal of consumer psychology – special issue on “consumers in cyberspace”, vol. 13, nos. 1&2, pp317. hou j. and rego c. (2002). “internet marketing: an overview”, working paper university of mississippi, url: http://faculty.bus.olemiss.edu/crego/papers/hces0802.pdf [last accessed 11/08/03]. hudlicka, e. (2003). “to feel or not to feel: the role of affect in human-computer interaction”, international journal of human-computer studies, vol. 59, pp1-32. ajis vol 13, no. 1 september 2005 251 ijsselsteijn, w. (2002). “elements of a multi-level theory of presence: phenomenology”, mental processing and neural correlates, presence 2002 – 5th international workshop on presence, porto, portugal, 9-11 october, 245-259. itac (2001). “electronic commerce in canada success stories: my virtual model inc”., electronic commerce task force of the information technology association of canada, url: http://i-com.ic.gc.ca/english/ stories/mvmsucc.html [last accessed: 31/08/2003]. jarvenpaa, s.l. and todd, p.a (1996). “consumer reactions to electronic shopping on the world wide web”, international journal of electronic commerce, winter 1996-97, vol. 1, no. 2, pp 59-88. jiang, z. & benbasat, i. (2002). “virtual product experience: effects of visual & functionality control of products on perceived diagnosticity in electronic shopping”, university of british columbia. kempf, d.s. & smith, r.e. (1998). “consumer processing of product trial and the influence of prior advertising: a structural modelling approach”, journal of marketing research, vol. xxxv, pp.325-337. kim, t. and biocca, f. (1997). “telepresence via television: two dimensions of telepresence may have different connections to memory and persuasion:”, journal of computer-mediated communication, vol. 3, no. 2, september, url: www.ascusc.org/jcmc/vol3/issue2/kim.html#factor [last accessed 14/08/2003] kim, j. and yoo, b. (2000). “toward the optimal link structure of the cyber shopping mall”, international journal of human-computer studies, vol. 52, pp 531-551. klein, l.r. (2001), creating “virtual experiences in computer-mediated environments”, rice university http://www.bepress.com/cgi/viewcontent.cgi?article=1001&context=roms [last accessed 22/5/03] knudsen, c.j. (2002). “video mediated communication”, royal institute of technology, sweden. http://kmr.nada.kth.se/papers/telepresence/vmc_knudsen_banff_2002.pdf [last accessed 22/5/03] kotler, p. (1998). marketing management, prenctice-hall, englewood cliffs, nj. kumar, n. and benbasat, i. (2002). “para-social presence: a re-conceptualization of “social presence” to capture the emerging relationship between a web site and her visitors”, proceedings of the 35 th annual hawaii international conference on system sciences (hicss), hawaii. lessiter, j., freeman, j., keogh, e. and davidoff j. (2000) “development of a new crossmedia presence questionnaire: the itc-sense of presence inventory”, presence – 3rd international workshop on presence, technical university of delft, delft, netherlands, march 27-28. li, h., daugherty, t. and biocca, f. (2001). “characteristics of virtual experience in electronic commerce: a protocol analysis”, journal of interactive marketing, vol. 15, no. 3, summer, pp13-30. li, h.; daugherty, t. and biocca, t. (2002a). “the role of virtual experience in consumer learning”, journal of consumer psychology. li, h. daugherty, t. and biocca, t. (2002b). “impact of 3d advertising on product knowledge, brand attitude, and purchase intention: the mediating role of presence”, journal of advertising, vol. 31, no. 3, pp 43-57. lohse, g. l. and spiller, p. (1998). “electronic shopping: how do customer interfaces ajis vol 13, no. 1 september 2005 252 produce sales on the internet”, communications of the acm, vol. 41, no. 7, pp81-87. lohse, g.l. and spiller, p. (1999). “internet retail store design: how the user interface influences traffic and sales”, journal of computer-mediated communication, vol. 5, no. 2, available online url: http://www.ascucs.org/jcmc/vol5/issue2/lohse.htm [last accessed: 22/08/2003]. lombard, m. and ditton, t. (1997). “at the heart of it all: the concept of presence”, temple university. url: www.ascusc.org/jcmc/vol3/issue2/lombard.html#abstract [last accessed 15/3/03] marks, l.j. & kamins, m.a. (1988). “the use of product sampling and advertising: effects of sequence of exposure and degree of advertising claim exaggeration on consumers’ belief strength, belief confidence, and attitudes”, journal of marketing research, vol. xxv, august, pp. 266-281. nelson, p. (1974). “advertising as information”, journal of political economy, 83, july/august. nielsen, j. (1999). designing web usability: the practice of simplicity, new riders publishing, december. peck, j. and childers, t.l. (2003). “to have and to hold: the influence of haptic information on product judgments”, journal of marketing, vol. 67, no. 2, pp35-56. peek, r. (1997). “is the web ready for 3d? should you be?”, information today, medford, dec, vol. 14, issue 11. pine ii, j.b. & gilmore, j. h. (1999). the experience economy – work is theatre and every business a stage, harvard business school press, boston, usa. pulli, p.; antoniac, p. & hickey, s. (1999). “mobile telepresence services for virtual enterprise”. url: http://paula.oulu.fi/publications/submited/ice99.pdf [last accessed 22/5/03] rose, g., khoo, h. and straub, d.w. (1999). “current technological impediments to business-to-consumer electronic commerce”, communications of the association for information systems, vol. 1, article 16, june. ryan, c. (2001). “virtual reality in marketing”, direct marketing, vol. 63, issue 12, april. schiffman, l., dash, j. and dillon, w. (1977). “the contribution of store image characteristics to store-type choice”, journal of retailing, vol. 53, summer. schubert, t., friedmann, f. and regenbrecht, h. (1999) “embodied presence in virtual environments”, in ray paton & irene neilson (eds.) (1999) visual representations and interpretations, pp. 268-278. london: springer-verlag. smith s.m. (2003). “internet marketing: an online text”, url: http://marketing.byu.edu/htmlpages/courses/490r/chapters/text.html [last accessed: 14/08/2003] smith, r.e. & swinyard, w.r., (1988). “cognitive response to advertising and trial: belief strength, belief confidence and product curiosity”, journal of advertising, vol 17, no.3, 1988, pp3-14. spiller, p. and lohse, g.l. (1998). “a classification of internet retail stores”, international journal of electronic commerce, vol. 2, no. 2, pp29-56. steuer, j.s. (1992). “defining virtual reality: dimensions determining telepresence”, journal of communication, vol. 42, no. 4, pp73-93. stockport, g.j. (2000). boo.com – the path to failure, european case clearing house, case 300-168-1. stockport, g.j., kunnath, g. and sedick, r. (2001). “boo.com – the path to failure”, ajis vol 13, no. 1 september 2005 253 journal of interactive marketing, vol. 15, no. 4, pp56-70. tauber e.m. (1972). “marketing notes and communications: why do people shop?”, journal of marketing, vol. 36, october, pp46-59. van putten, c. (2000). “the top 5 barriers to smooth online shopping”, flash commerce news, url: http://flashcommerce.com/articles/00/03/03/120430320.htm [last accessed: 21/08/2003]. voss c. (2003). the experience profit cycle: trends in the experience and service economy, research report, london business school, july, p4. westbrook r.a. and black w.c. (1985). “a motivation-based shopper typology”, journal of retailing, vol. 61, no. 1, pp78-103. wolfinbarger, m. and gilly, m.c., (2001). “shopping online for freedom, control and fun”, california management review, vol. 43, no. 2, winter; pp34-55. wolfinbarger, m. and gilly, m.c., (2003). “etailq: dimensionalizing, measuring and predicting etail quality”, journal of retailing, vol. 79, no. 3, fall, pp183-198. wright, a.a. and lynch, jr., j.g., (1995), “communications effects of advertising versus direct experience when both search and experience attributes are present”, journal of consumer research, vol. 21, march, pp708-718. microsoft word ajisfinal2003.doc australasian journal of information systems special issue 2003/2004 63 supporting topic map creation using data mining techniques witold abramowicz, tomasz kaczmarek, marek kowalkiewicz department of management information systems the poznan university of economics, poznan, poland abstract there is an increasing interest in automating creation of semantic structures, especially topic maps, by taking advantage of existing, structured information resources. this article gives a preview of the most popular method – based on rdf triples, and suggests a way to automate topic map creation from unstructured information sources. the method can be applied in information systems development domain when analysing vast unstructured data repositories in preparation for system design, or when migrating large amounts of unstructured data from legacy systems. there are two innovative methods presented in the paper – term crawling (tc) and clustering hierarchy projection (chp), which are applied to build a topic map based on free text documents from local repositories and those downloaded from the internet. the methods originate from data mining techniques for knowledge discovery. a sample tool, which uses described techniques, has been implemented. the preliminary results that have been achieved on the test collection are presented in concluding sections of the article. background in today’s enterprises, documentation is spread throughout the whole organisation. the locations may include corporate portals, document management systems, users’ private folders, web servers and many others. in order to provide employees with access to all relevant documents one should consider using a common data structure which is able to identify location of any (or almost any) document in the enterprise. one such structure, discussed in this paper, can be topic maps. however, introducing a new data structure imposes a requirement to enter data about all the documents which should be made available, and fill in all the necessary attributes. due to heterogeneity of data sources, automated techniques are still more a concept than reality. on the other hand, just supporting of implementation may be helpful. when using topic maps as a data structure describing the whole document repository, one may apply concepts from the field of information retrieval. after providing background information, we further explore selected techniques and suggest an extension of some. topic maps topic maps as a structuring mechanism for repositories are becoming more and more popular. the phenomenon of topic maps can be observed during many conferences, such as xml europe, in newsgroup discussions (the oasis society), and in practice – in many applications throughout the world (rittershofer 2002; andersen 2003). the basic idea of topic maps has been described in “the tao of topic maps” (pepper 2000), a more detailed (and standardized) view has been presented in iso 13250 (iso 2000) specification, and one of the first attempts to sketch the idea of using topic maps for structuring content has been made in “topic maps for repositories” (ahmed 2000). we focus in this article on the topic map technology, because we believe it is a potentially interesting data structure for information systems development. topic maps as semantic structures can be used in information systems development in several areas. they may be used as a tool supporting information systems design. the semantics carried by a topic map may be used to understand the domain and analyse the requirements for the design. one way of using topic maps would be to identify subjects and patterns that may exist in unstructured data analysed for system modelling. the tools, supporting topic map visualization, help to analyse domain and design database. the other potential application area is migration from unstructured data or text repository to the structured one. automatically created topic maps provide insight into unknown data structure thereby imposing partial structure constraints. the latter application is further explored throughout this article. australasian journal of information systems special issue 2003/2004 64 topic maps enable modelling and representation of knowledge in an interchangeable form, and at the same time they make up a uniform framework for knowledge and information resources management (pepper 2002a,b). topic maps constitute a model that is applicable by a wide range of industries. they are helpful in organization and navigation of continuously growing information pools. current topic maps architecture bases on the following elements (after steve pepper, we call it the tao of topic maps): topics – the term topic refers to an element in the topic map that represents the subject being referred to. topics can be categorized. they can have zero or more topic types and can also have names. the standard names for topics are: base name, display name and sort name. each topic can have facets – attributes for storing additional information, for example topic profiles. topic types determine structure hierarchy, allowing, among other things, construction of either top-level ontologies, domain specific ontologies, or application specific ontologies based on the iso 13250 tm standard. topic names – among wide range of names we use in day-to-day activity, the iso standard describes three of them as follows: base name – characterises the topic for internal purposes; it is a required name, display name is used for external representation purposes and is optional, sort name – allows constructing indexes of topics, useful for sorting purposes and facilitate searching. associations – a topic association is a link element, showing relationships between two topics. association can have types (for example influenced by, required by, written in etc.) and roles (for example influencer, influenced, prerequisite, result, document, language). association types – similarly to topics which are grouped by other topics called topic type, associations are grouped by associations types also represented as topics in the topic map. even with large and complicated structures, navigation, searching, and clarity of user interface functionality are preserved by the possibility of creating custom types of association. one may say that topic types and association types play the same role in the topic map. association roles – each topic that is involved in an association is characterised by association role i.e. influencer and influenced. occurrences – occurrences link topics to one or more relevant information resources. an occurrence can be anything, most often it is a uri (universal resource identifier), or document (article, picture, video etc.). occurrences can have roles and role types (web based training, computer based training, ms word document, flash animation, knowledge base etc.). additionally, the iso specification of topic maps defines the following: identity – during the mapping or merging of large scale maps it may occur, that two or more topics with different base names are describing the same concept or fact. making topic maps structure portable we have to take into consideration such situations. there are properties called public subjects (identity) that describe theme of the topic in a standard way. in case of conflict situation topics with the same identity are merged. scope – the extent of the validity of a topic characteristic assignment: the context in which a name or an occurrence is assigned to a given topic, and the context in which topics are related through associations. facets – facets basically provide a mechanism for assigning property-value pairs to information resources. a facet is simply a property; its values are called facet values. facets are typically used for supplying the kind of metadata that might otherwise have been provided by sgml or xml attributes, or by a document management system. this could include properties such as “language”, “security”, “applicability”, “user level”, “online/offline”, etc. once such properties have been assigned, they can be used to create query filters producing restricted subsets of resources, for example those whose language is “italian” and user level is “secondary school student”. australasian journal of information systems special issue 2003/2004 65 the topic maps standard is constantly being developed. they are used for knowledge structuring, for example in digital encyclopaedias, where entries can be linked very flexibly. navigation in such structure is much easier than navigation in traditional tree-based catalogues. topic maps technology is being introduced in web search engines (for example http://www.webbrain.com/) enhancing their efficiency (pepper 2002a,b) . at the beginning, the idea, as a descendant of the docbook (walsh and muellner 1999) standard, was to create topic maps from scratch. therefore, the process of creating topic maps would be to identify resources, try to describe them as it was needed, and link subjects with the other ones (a good introduction to that can be found in the mentioned work of steve pepper). one of challenging tasks would be to identify all required occurrences of each topic, and then to associate them with this topic. recently, the topic maps community has focused on automated topic map creation from already existing resources. the automated process is more “occurrence driven”, which means that building mechanisms first analyse the source base (for example a set of html files), and then create a topic map. in this approach, occurrences finally play a smaller role – their content is treated as a base for creating topics and associations in the map, and afterwards source occurrences can even be omitted. as it is explained later in the text, in order to create such a map, one needs to have access to some structured sources, or mechanisms for natural language processing (nlp). the aim of this article is to suggest a method for automated topic map creation for unstructured sources with no need to use nlp this makes the process more flexible (e.g. mostly independent from language of source base). data mining (berry and linoff 1997) define data mining as the process of exploration and analysis, by automatic or semiautomatic means, of large quantities of data in order to discover meaningful patterns and rules. they support this definition in their later publications, such as (berry and linoff 2000). according to this, data mining is mainly used for the following six activities: classification – assigning analysed objects to predefined classes based on its features. this is mainly used for classifying tuples in relational databases. one has to prepare well defined classes, and afterwards, every attribute of a tuple is tested against the definition. definitions of classes are made based on an analysis of a training set consisting of preclassified examples. examples of classification include assigning categories to filtered text documents, credit scoring, assigning customers to predefined segments. estimation – as opposed to classification, which results in discrete outcomes, estimation creates continuously valued outcomes. analysing input data, one comes up with a value for output data, being a continuous variable. estimation may also be used for classification purposes, when one decides, that output values are compared against a classifying threshold. for examples bank scoring rules may state, that a bank customer will get an equity loan if his score – as derived by data mining rules – will be above 70 points. other examples of estimation include: estimating the number of children in a family, estimating the value of a piece of real estate. prediction – being either classification or estimation, but with one specific attribute – there is no way of checking, whether outcome is true or false at the time of deriving the outcome. one has to wait, because at the derivation time the information needed to prove or falsify the rule is unavailable. examples include predicting most profitable customers within the next quarter, predicting which customers will order broadband internet. affinity grouping and association rules – its task is to determine facts that occur together. the classic example is one of determining which products are bought together by supermarket customers. affinity grouping may be used to arrange items on shelves in a store or in a catalogue. australasian journal of information systems special issue 2003/2004 66 clustering – is used to segment a group of objects into a number of smaller subgroups (clusters). clustering does not rely on predefined classes (as opposed to classification), and there are no examples. objects are grouped together based on their self-similarity. cluster descriptions may be proved by data miner. clustering is often used as a first step, before applying some other form of data mining. the list of six methods of data mining is not a complete list. one may suggest some other forms, which will be used, either separately or in cooperation with other methods, to identify rules in underlying data repository. in further parts of this document authors propose an original method of data mining (or, more specifically, text mining) – term crawling. automated topic map creation the issue of automated topic map creation has been widely discussed in many publications in 2002, two years after publishing an iso standard document for the topic maps. there is currently a number of running projects focusing on creating topic maps from existing data sources, one such example is the ontopia’s mapmaker toolkit. the approach can be described as “identify – describe – create topic map” procedure, whilst specific solutions differ. here, we will illustrate the topic map creation process by analysing usable data sources, propose four step procedure for its creation, and show the three approaches to using the procedure. data sources in most cases, there is no need to create topic maps from scratch. existing information resources in organizations can be used as a critical mass which will leverage the process. topic maps, in its simplest form, can be described as a collection of topics, associations and occurrences, the so called tao of topic maps. therefore, the most efficient information resources are those, that can be effortlessly converted into subjects, relations between them, and subjects’ instances. examples of such sources include: • relational databases – where primary keys describe topics, fields within one record can be occurrences, and relations help to establish associations • web sites – where urls identify topics, webpage contents are occurrences, and hyperlinks show associations with other web pages • directory systems – where directory objects point at topics, directory schema describes topic types, objects themselves are occurrences, and associations are derived from the treebased structure of directory • content management systems – which are similar to the websites, but store smaller units of data (paragraphs) and often provide more detailed descriptions • files in file systems – which can be treated similarly to directory systems, and again, files of specific types can contain metadata, which can afterwards be used in preparing more detailed topic map other way to enumerate the data sources – a more general one – would be based on source characteristics. and therefore we can identify: • structured knowledge – ontologies and classification systems, database schemas, document type definitions (dtds) and xml schemas, metadata schemas • structured document content – with emails, newsgroup messages or accounting documents as examples • unstructured document content – where preparations must involve more sophisticated techniques, such as natural language processing with named entity recognition, concept extraction, and taxonomic classifications. most nlp based processing tools require only raw text, therefore document transitions are not complicated australasian journal of information systems special issue 2003/2004 67 • document metadata – analyses include properties stored in a file (such as microsoft office’s properties, rdf-pdf or html dublin core) or externally stored properties (rdf, mpeg 21, document management system metadata) procedure the procedure of automated topic map creation can be split into four steps. the procedure bases on a common assumption, that before creating a final map, there is a rdf (resource description framework) model prepared, which is used as input data afterwards. the four steps are as follows: 1. subject recognition 2. information extraction and preparing 3. rdf modelling 4. mapping rdf model into a topic map whilst the fourth step can be done using available tools (such as those from ontopia), the previous three are most interesting, and let us experiment and develop methods for converting source data into a resource description framework model. in this part we analyse current routines, further in the paper we will propose a method based on the two experimental concepts: term crawling and clustering hierarchy projection. both concepts, aside from abstract considerations, are also undergoing implementation tests – sample results are presented in the text. subject recognition in order to identify potential topics and say something about subjects, one has to locate data sources – subject occurrences. this is highly dependent on content type. for example, in relational databases this will mean analysing a database schema and deciding, which tables contain candidate entities (one can also create his own queries based on selected tables); in document repositories the subject recognition will mean selecting a set of documents, which will be processed later on. once we have identified the subjects and their occurrences, we should prepare unique uris. the most common method is to use one’s own registered domain name to create universal resource identifiers. example uri would then be http://www.mydomain.org/uri/apps/msword which would identify the microsoft’s application. information extraction and preparing after the subjects are recognized, one has to extract data needed for processing. when analysing structured sources, such as xml documents, emails etc. this is quite obvious – one has to decide which document properties are important and, optionally, to prepare them. preparing includes data conversion, value normalizing, splitting single values into multiple values, aggregating multiple values into single values, traversing hyperlinks to collect additional data etc. for semi-structured and unstructured data, this can be trickier, and advanced information processing techniques, such as shallow text processing (abramowicz and piskorski 2003), natural language processing (named entity recognition, concept extraction, and taxonomic classification) have to be used. further parts of the paper will show that term crawling and clustering hierarchy projection techniques can help here as well. rdf modelling rdf models describe objects, which correspond to topics in a topic map; their properties correspond to occurrences or associations with other topics. a rdf model consists of statements (often called triples), which have three parts: subject, property, and value. subject describes a resource, the statement is about; property describes the property type assigned to subject; value contains a specific value of the property of the subject. subject, property, and value can contain uris, value can also contain other data types, such as strings, integers etc. australasian journal of information systems special issue 2003/2004 68 mapping rdf model into a topic map this step, performed mostly by automatic tools, involves analysing rdf triples, and deciding whether a triple describes a topic, an association, or an occurrence. preparing topics, associations and occurrences is the final step in the topic map creation, and further activities focus only on refining the map. processing approaches the described procedure of automated topic map creation can be used in a variety of environments, using a wide spectrum of source data. depending on expected application, one of three approaches can be chosen from: one-time processing, repeated batch processing, and continuous processing. one-time processing one time move from legacy system to topic map is very effective, because the legacy system is no longer used and full power of topic maps can be used from the beginning. the disadvantages include need to roll out legacy indexes or supporting users who have not rolled out, and therefore do not have access to the latest data. repeated batch processing the repeated batch processing can be triggered or scheduled. it allows for using existing, legacy indexes, and topic map at the same time. however this procedure is more resource consuming, less reliable (especially when source data schemas change), and does not guarantee that topic map is up to date. continuous processing continuous processing, or wrapper around existing system, is a most complex technique. it lets users use existing tools and indexes and at the same time it updates the topic map, so that it is always up to date. however, if the existing system changes, significant development efforts may be required in order to maintain operability. the approach proposed in this article can be applied in all the approaches mentioned above. it is also feasible in the last, most complicated, case. when a document collection is changing (new documents are added), the topic map should change in order to resemble new structure. in our approach it is possible to track new vocabulary which may be introduced with new documents, and include it in one of the dictionaries used in the procedure described below (clustering hierarchy projection and term crawling). selected knowledge discovery techniques methods of automated topic map creation, sketched in the previous part of the text (see section 0), are very efficient for structured and semi-structured data. however, when we try to apply the described procedure to unstructured data, such as collections of documents from the web, a number of questions arise. there is no easy way to point out subjects, and associations. one, previously mentioned, way would be to use natural language processing techniques, however they require significant effort to build rules for different languages (as, obviously, not only english documents may be processed), associations proposed by those techniques base only on documents’ contents (and therefore overlook assumedly well known relations – contextual information). australasian journal of information systems special issue 2003/2004 69 recent work on ontology building (which is a shared and formal specification of the vocabulary and assumptions about its use describing certain, limited reality) gives hints on which techniques can be useful when we deal with unstructured data (maedche 2002). these techniques originate from data mining and knowledge discovery, as described in section 0. we adopted three of them to propose new approach towards topic map creation. the first one – term crawling – will let us automatically identify relations between concepts, the second one – clustering hierarchy projection – is based on hierarchical clustering of documents, thus allowing to identify additional subjects in a document collection and hierarchical relationships. the third one – association rules discovery – is used to extend topic map with non-hierarchical associations. these methods, jointly used, provide a framework for creating rdf triples from unstructured data, and eventually creating a topic map, which would not be possible when using standard methods. this section provides background on data mining techniques used in proposed approach to topic map creation. clustering the basic idea behind clustering is that documents can be grouped according to their content similarity without any prior knowledge or assumptions concerning this content. there are various approaches to clustering as described in (bhatia and deogun 1998) and (steinbach, karypis et al. 2000). usually the techniques are divided into k-means clustering and hierarchical clustering. the former is based on the following procedure: 1. select initial k points among documents (each document is represented by vector of its terms frequencies) – these points are called centroids. the mathematics defines centroid of n point masses mi located at points xi as a centre of mass with the formula like in eq. (1) ∑ ∑ = == n i i n i ii m xm x 1 1 (1) in our case the x would be terms frequencies and m – their weights. 2. assign all remaining points to the closest centroid therefore creating clusters 3. recompute centroid for each cluster. 4. repeat 2 and 3 until centroids don’t change. however this approach proved to give worse clusters than the hierarchical clustering, which is usually described as below: 1. create primary set of clusters where each document is represented by single cluster. 2. compute similarity between all clusters using selected similarity measure – this creates similarity matrix. 3. reduce the clusters number by merging the closest clusters. 4. update the similarity matrix. 5. repeat steps 3 and 4 until the desired number of clusters is reached. there are two approaches to creating a hierarchical tree of clusters (maedche 2002) – either bottomup, starting with individual objects and grouping the most similar ones into clusters, or top-down, starting with one large clusters, containing all objects, and dividing such cluster into smaller ones. the three functions used are sim, coh and split. the sim function returns similarity measure between any two given documents. there is a number of similarity measures to choose from, according to the model. here, the cosine measure may be a good choice. the coh function measures cluster coherence, and the split function splits one cluster into more objects. australasian journal of information systems special issue 2003/2004 70 algorithm 1.: hierarchical clustering algorithm – bottom-up (maedche 2002) require: a set { }nxxx ,,1 k= of objects, n as the overall number of objects, a function sim: ( ) ( ) ℜ→× xrxr ensure: the set of clusters k (or cluster hypobook) for i:=1 to n do ki:=xi. end for k:={k1,...,kn} j:=n+1 while |k|>1 do ( ) ( ) ( )vukkkknn kksimkk vu ,maxarg:, ,21 ×∈= 21 nnj kkk ∪= { } { }jnn kkkkk ∪= 21 ,\: 1: += jj end while algorithm 2: hierarchical clustering algorithm – top down (maedche 2002) require: a set { }nxxx ,,1 k= of objects, n as the overall number of objects, a function coh: ℜ→)(xr a function split: )()()( xrxrxr →× k:={x}(=k1) j:=1 while 1.. >∈∃ ui ktksk do ku:=arg min kkv ∈ coh(kv) (kj+1,kj+2)=split(ku) k:=k\{ku}∪ {kj+1,kj+2} j:=j+2 end while hierarchical clustering comes in different flavours depending on the similarity measure taken. average linking is based on the mean of documents vectors – the centroid (also defined as a centre of a cloud of points – in most approaches it is simply a vector of average term frequencies from all documents in the cluster). complete linking uses the dissimilarity measure which is the greatest distance among points in compared clusters. single linking uses dissimilarity in the opaque way – here the dissimilarity is defined as the minimum distance between any points in compared clusters. various distance metrics are used to compute similarity or dissimilarity but the most common is the cosine measure which is dot product of the two vectors divided by their lengths product. ∑∑ ∑ ∈∈ ∈∈ = yyxx yyxx yx xy yx 22 , ),cos( rr australasian journal of information systems special issue 2003/2004 71 in a tool described in this article we also use this measure. hierarchical clustering is particularly interesting in our approach because the algorithm generates hierarchical tree of clusters as it merges them into bigger clusters. the figure 1 shows the sample hierarchy generated by hierarchical clustering algorithm. the initial clusters (cluster 1, cluster 2 … or c1, c2 etc.) are merged during the subsequent iterations of the algorithm, based on their similarity. the bottom part of the picture shows inclusion of the primary clusters in their conglomerates. figure 5. the hierarchy tree generated by a clustering algorithm term crawling the aim of term crawling method is to build a network of associations between terms. the semantic network is built by analysis of concurrent term presence in documents on the world wide web. assuming that there is a tool for gathering information from internet sources and assessing their relevance, term crawling aims to indicate, that even unstructured data of limited trust can be utilized in information systems. the network of associations, created by term crawling will then be used in creating associations between clusters created by chp (clustering hierarchy projection). 1. in order to build a semantic network of associations between terms, we have to specify a starting point – this will be the primary term. 2. the primary term is used as a query, and – through google api (dornfest 2002) – submitted to the search engine, which replies with a list of web pages relevant to the query. 3. web pages are downloaded 4. value of web pages for the process is assessed – this includes length, format, and language analyses. 5. downloaded web pages are tokenised, and tokens weights are estimated – this leads to creating n-dimensional matrix, where n is equal to number of aspects of each token (at least its uri and weight). 6. the matrix is then processed. the processing techniques are still in the experimental phase and we are looking for the most efficient technique. currently we create intersections of term sets from each webpage and as a result, after applying stop-word list, we get a list of terms related to the primary term. 7. in that phase, the network of associations can be updated. if related terms are already contained in a network, then new associations are created, otherwise new nodes and associations are created. 8. then, according to user’s needs, recursive searches can be deployed. breadth-first search attempt is preferred. continuous running of term crawling mechanism leads to building large semantic networks of associations between terms. apparently, such algorithm will result in gigantic maps, if not constrained by depth boundaries. the boundaries should be specified beforehand. c4 c5 australasian journal of information systems special issue 2003/2004 72 figure 6. network created with term crawler – “xml” as an initial term figure 2 shows a sample map created with term crawling algorithm, with crawling depth limited to two levels and stop list applied. as it can be observed, not all results are valuable for further processing (see “april” – a term associated with “information”), but overall structure looks very promising. affinity grouping and association rules affinity grouping and association rules are useful for identifying coexisting facts. when analysing large document repositories, they may be used – as in case presented in this article – for identifying coexisting concepts. an effective way of identifying important rules is by representing them using first order logic. a rule may be represented as: ( ) ( ) ( ) ( ) ( )nnnkkkjjj varvarvarvarvarvar ,,),(,,, 111222111 ∧∧∧⇒∧∧∧ kk where: ai is an attribute vi is a value (simple or compound) ri is a predicate the left side of a rule is called rule body, the right side – rule head. the sentence above may be either true (confirming the rule, when rule body and head are both true) or false (violating the rule, when rule body is true, and head is false). every rule has two measures: support (s) and confidence (c) which are defined below in equations (2) and (3). rule support is the ratio of the number of objects holding the rule to the number of all objects in analysed dataset. confidence is the ratio of the number of the objects where both rule body and head hold to the number of objects where rule body holds. t t s h = (2) b h t t c = (3) where: t is a set of all objects in the analysed dataset t b is a set of objects where rule body holds t h is a set of objects that hold the rule (which implies that bh tt ⊆ ) australasian journal of information systems special issue 2003/2004 73 topic map creation process topic map creation is a two faceted process – first, there should be created an initial topic map, basing on a document collection. after that, there is a place for a continuous process of updating topic maps (as described in point 0 – processing approaches). the following part of the paper focuses on a first facet – creating an initial topic map from a document collection. we highlight six main steps of the process: preparing document collection, generating base dictionary, performing term crawling and clustering (those two subprocesses can be run in parallel), identifying associations, and preparing rdf triples for further processing. document collection the collection that we use for creating topic map is gathered from the cnn.com portal – it is a set of several hundred news stories from main categories (e.g. health, politics, travel, technology). the documents are partially described with metadata (publication date, title and category). the most interesting results of clustering hierarchy projection and term crawling can be obtained for collections of documents of different subjects – this is also an advantage of topic maps. still, in order to use thesauri efficiently, the documents should belong to some general domain. topic maps, chp, and tc are not necessarily the best solution for document collections of highly specific domains. base dictionary the base dictionary, which contains selected words from the document collection, may be divided into domain specific dictionaries. the standard stoplist for english language is used to clear the dictionary, and typos are removed. then the dictionary is split into smaller dictionaries (this is handmade). the number of child dictionaries depends on the main topics that we would like to distinguish in our topic map. the most basic approach would be to divide the dictionary according to main categories found in a document collection. more sophisticated approach would be to identify topics based on geographical names or economic entity names. the dictionaries used for subsequent passes of clustering algorithm do not have to be disjoint. however, the above remarks are only heuristics, and the final decision concerning rules of the dictionary split depends both on the collection (its specialization and range) and the desired result. sometimes repeated experiments are necessary to obtain satisfying results. in fact, expert’s skills are required to asses the aspects of the collection that should be identified and used in the procedure. this is a potential subject of future publications. term crawling after the base dictionary has been created, term crawling mechanism is used to identify and store relations between terms. each term is taken from the dictionary, and related terms are searched. depth of term crawling should be set manually, but two levels (as in figure 2) seem to be efficient for most applications. the term crawler filters only stop words – association network may contain non-dictionary terms as well. as an outcome of term crawling, there will be created a network with n vertices (where n is a number of terms in dictionary incremented by other related terms found out by tc) and m edges (term relations, as indicated by tc). this network can be used to identify associations in the topic map (see below). the term crawler is implemented in visual basic .net and uses extensively google api, based on web services. the semantic network, created by term crawler may be stored in xml files or relational database. this is out of the scope of this paper, but it is worth mentioning, that the semantic structure can be successfully used for user query modification (broadening queries and australasian journal of information systems special issue 2003/2004 74 including related terms), which can be of a great value for e-business. catalogue browsers can for example propose products that are in some way related to those specified by customer (moped instead of a motorcycle etc.). the semantic network can be also treated as a preview of present-day associations between subjects, with respect to their evolution (which can also be represented in the network). clustering hierarchy projection clustering proved to be valuable in dividing large document collections into smaller, meaningful parts (i.e. clusters represent certain topics). the rationale behind our approach is to use clustering to assign certain topics, created by the clustering algorithm, to clustered documents. we use the hierarchy of generated clusters to create hierarchy of topics and their occurrences – the documents. except from using hierarchy of clusters (which is usually discarded in clustering applications) we use several dictionaries in the algorithm. classical approach to the clustering uses single cleared dictionary. the stoplist is used to eliminate junk-words and in some approaches the words are stemmed (according to the algorithm first described in (porter 1980). in our approach we split this dictionary to obtain smaller dictionaries for specific aspects of a domain that the documents regard. the clustering of the same documents set using different dictionaries produces different hierarchies of clusters. this can be intuitively viewed as a projection of n-dimensional term space (where n is the number of terms in the whole dictionary) to a set of mdimensional spaces (where m<n). one could argue that we loose some information because clustering is done with smaller dictionary. but in exchange we gain a general view on the document collection and we can build the topic map based on several hierarchies and associations between them, which emerge from co-occurrences of the same documents in each hierarchy. we use bottom-up hierarchical algorithm with each dictionary that has been selected. every iteration of the algorithm is logged. this information is used to create the hierarchies of clusters. the hierarchies may differ in depth, depending on the technique used to create the tree. the sample tool allows for adjusting that. for example, when merging two clusters of similar size (measured in number of documents in each cluster) they form a new, more general cluster. on the other hand, merging single-document cluster with a cluster containing several documents does not produce a new cluster. the single-document one is joined with the bigger one. cluster can be characterized by a set of terms that appear in its documents and in the dictionary used for clustering. these terms can be ranked according to their frequency in the whole cluster or according to their input to the similarity of the documents in the cluster (the smaller is a distance between documents computed using single term frequency, the more similar documents are and therefore the better given term describes the whole cluster). the most popular measure for weighting terms is term frequency – inverse document frequency measure, denoted dttfidf , and defined in equation (4). this measure is used throughout the topic map creation process and therefore is readily available for choosing terms that describe clusters best. )log(*, , t dt dt df d d freq tfidf = (4) where: tfidft,d is tfidf measure of term t for document d freqt,d is the number of occurrences of term t in document d |d| is the number of all terms in document d |d| is the number of documents in the document collection dft is the number of documents that term t occurs in. in current stage these terms (with the highest tfidf) are used by human author of a topic map to give the topic name. automatic assignment of a topic name can be achieved using thesaurus which is discussed in section 0 of this paper. australasian journal of information systems special issue 2003/2004 75 clustering also provides linking topics to occurrences – the documents in a document collection. such link has a form of uri pointing at a given document. although, using our technique this is achieved in an obvious way, this becomes a non-trivial task when creating a topic map without automating tools. identifying all occurrences of a given topic in an enormous collection may be challenging for a human author, but is easily achieved (with certain degree of accuracy) by a retrieval machine. the outcomes of the clustering mechanism are several hierarchies of clusters that can be mapped to topics in an arising topic map. the next step is to create or interpret the existing associations. associations the preceding procedure produces data structures that suggest existence of certain associations between topics. these associations can be divided into: 1. generalizing – these associations are derived from hierarchies generated during the execution of clustering algorithm. merging two clusters into one more general allows creating association of a type: generalization, which connects smaller cluster with the bigger and more general one. 2. specifying – these are the opposite of the “generalization” associations, but they are created from more general to more specific clusters, down the hierarchy tree. 3. unnamed associations between different hierarchies – these are based on the co-occurrence of the same documents in the clusters belonging to the hierarchies created with different dictionaries. the hierarchy trees are compared and the list of suggested associations is created. the list is created in the following manner: the more common documents appear in compared clusters, the stronger the association is assumed to be, and therefore it appears higher on the list. affinity grouping and association discovery techniques (described in paragraph 0) are used to identify unnamed associations. to find associations between two clustering hierarchies generated with different dictionaries and thus between separate topics one has to apply the following procedure: for each document in both hierarchies get clusters that the document belongs to – they form rules’ body (of course the bottom clusters containing single documents are not taken into account, as well as top cluster – if it contains the whole document collection) find clusters in the second hierarchy that the document belongs to – they form rules’ heads given set of objects generated above select unique set of rules and compute support and confidence measures for them rules with high confidence measure are particularly useful since they indicate strong relationship between topics from separate hierarchies. it is crucial to take into account relationship’s direction and apply the above procedure in both directions analysing both hierarchies as a source for rules’ bodies. this provides that associations of clusters with similar documents and documents number are found to be the strongest. the ordered (according to confidence) set of associations is then presented to the topic map engineer, who can discard an association, if he considers it unnecessary, or give it a name. for example if one hierarchy was obtained with the dictionary containing corporations’ names and the other with the dictionary containing product names, the association name could be “is a product of”. 4. unnamed associations between clusters as indicated by term crawling – each of clusters is described with a term (or set of terms). those terms, further converted into topics in a topic map, are compared with a network created by term crawling mechanism. the comparison lets us identify even more relations between clusters – depth of association network searching can be decided upon each time a topic map is generated. association discovery using support and confidence measures can be performed with term crawling if the australasian journal of information systems special issue 2003/2004 76 procedure described in paragraph 0 is to be modified. instead of taking into account only these terms that appear in all the analysed documents, one could measure the number of occurrences of a certain term – this could be used to compute the confidence measure for the rule: query term � found term. similarly to unnamed associations between different hierarchies, unnamed associations between clusters produce a list of suggested relations, which can be processed by human authors afterwards. rdf triples the resource description framework model and syntax specification defines the rdf data model, and basic serialization syntax. it became a w3c (world wide web consortium) recommendation in 1999. the data model is basically a directed graph. its elements include entities and binary relationships. the relationship is represented by rdf statement (also called rdf triple). a statement can be represented by two nodes and a directed arc between them: • subject – the resource the statement is about (uri) • property – the property, being assigned to the subject (uri) • value – value assigned to the property (uri or string literal) in our case, most popular rdf triples would be for example: • (http://my.org/news/1, #topic, http://my.org/thesauriitems/54) – representing occurrence of a specific term • (http://my.org/thesauriitems/23, #generalizing, http://my.org/thesauriitems/66) – representing generalization of topics (terms) • (http://my.org/thesauriitems/66, #specifying, http://my.org/thesauriitems/23) – representing specification of topics (terms) • (http://my.org/thesauriitems/38, #suggested, http://my.org/thesauriitems/95) – representing suggested association between terms, based on term crawling, and identifying associations based on coexistence of documents in different clusters (as described above) after creating rdf triples, the process of creating a topic map becomes obvious, and existing tools can be used for that purpose. future work the proposed solution, although not fully automating the topic map creation process, has proven to be useful when creating a topic map describing a document collection. the approach could be further enhanced in several areas. there is significant effort towards identifying topics in plain text files. these topics may be both single words or short phrases and whole text parts or chapters devoted to specific subject. identifying topics is not enough, one have to name them. thesauri structures can be used to support it at a larger scale. these structures contain information about words synonyms, hierarchies of meaning and phrases, which may be useful when naming topics. a serious effort in the information retrieval and document understanding is aimed at automatic topic recognition in texts. here, the word topic stands for a larger part of a given text, devoted to some specific subject. such techniques may be helpful when identifying topics for topic maps. attempts have been made to use the described method to support continuous processing of the document collection. as mentioned above it involves modifying the topic map as new documents (potentially significantly different from others in the collection) appear. the approach involves creating new dictionary from the new terms found and adding topic hierarchy generated to the already existing. some associations can be provided by term crawling, but further research in this field is necessary. the most troublesome on this level of generality is association creation and interpretation. in the existing solutions associations are created based on examined language structures. in our approach a number of general associations is created, however naming the unnamed ones is currently not solved in our solution. australasian journal of information systems special issue 2003/2004 77 the applications of topic maps in the domain of information systems development demands further research. the insight into unstructured data or text semantics that may be obtained using automatic topic map generation can be useful for system design, as pointed in the introduction. however this requires further improvement both concepts and tools. references abramowicz, w. and j. piskorski, 2003, "information extraction from free-text business documents." effective databases for text & document management 2003: 12-23. abramowicz, w., kowalkiewicz, m., and zawadzki, p., 2002, tell me what you know or i’ll tell you what you know. skill map ontology for information technology courseware, mehdi khosrow-pour (ed.), issues and trends of information technology management in contemporary organizations, information resources management association international conference, seattle, usa, 2002, information science publishing. abramowicz, w., kowalkiewicz, m., and zawadzki, p., 2003, ontology frames for it courseware representation. in e. coakes (ed.), knowledge management: current issues and challenges. irm press ahmed, k., 2000, "topic maps for repositories." xml europe conference. andersen, s. q., 2003, forskning.no. 2003. baeza-yates, r., and ribeiro-neto, b., 1999, modern information retrieval acm press, addison wesley longman limited, usa berry, m. j. a. and g. linoff, 1997, data mining techniques for marketing, sales and customer support. new york, john wiley & sons inc. berry, m. j. a. and g. linoff, 2000, mastering data mining. the art and science of customer relationship management. new york, john wiley & sons inc. bhatia, s. k. and j. s. deogun, 1998, "conceptual clustering in information retrieval." ieee transactions on systems, man and cybernetics: 427-436. ding, c., and he, x., 2002, cluster merging and splitting in hierarchical clustering algorithms, 2002 ieee international conference on data mining, maebashi, japan dornfest, r., 2002, google web api, the o'reilly network. gómez-pérez, a., 1999, evaluation of taxonomic knowledge in ontologies and knowledge bases, proc. of the knowledge acquisition workshop grønmo, g. o., 2000, creating semantically valid topic maps, xml europe conference, paris, france grønmo, g. o., automagic topic maps, retrieved april 26, 2003 from: http://www.ontopia.net/topicmaps/materials/automagic.html gruber, t. r., 1993, toward principles for the design of ontologies used for knowledge sharing, international workshop on formal ontology, padova, italy iso, 2000, information technology sgml applications topic maps. iso/iec 13250. geneva. knight, j. r., 1996, discrete pattern matching over sequences and interval sets, ph.d. dissertation, department of computer science, the university of arizona ksiezyk, r., 2000, answer is just a question [of matching topic maps], xml europe conference, paris, france maedche, a., 2002, ontology learning for the semantic web. boston, kluwer academic publishers. moore, g., 2001, rdf and topic maps – an exercise in convergence, retrieved april 26, 2003 from: http://www.topicmaps.com/topicmapsrdf.pdf oommen, b. j., and de st. croix e. v., 1994, string taxonomy using learning automata, ieeetsmc: ieee transactions on systems, man, and cybernetics pepper, s. 2000. "the tao of topic maps." xml europe conference. pepper, s., 2002a, ten theses on topic maps and rdf, retrieved april 26, 2003 from: http://www.ontopia.net/topicmaps/materials/rdf.html pepper. s., 2002b, the ontopia mapmaker, retrieved april 26, 2003 from: http://www.ontopia.net/topicmaps/materials/mapmaker_files/frame.htm australasian journal of information systems special issue 2003/2004 78 porter, m. f., 1980, "an alogrithm for suffix stripping." resource description framework (rdf) model and syntax specification, feb. 1999. w3c recommendation. rittershofer, a., 2002, lernen mit topic maps. 2003. sowa, j. f., 2000, knowledge representation: logical, philosophical, and computational foundations, brooks cole publishing co., pacific grove, ca steinbach, m., g. karypis and k. v., 2000, a comparison of document clustering techniques. 2003. walsh, n. and l. muellner, 1999, docbook: the definitive guide, o'reilly & associates. wrightson, a., 2001, topic maps and knowledge representation, retrieved april 26, 2003 from: http://www.ontopia.net/topicmaps/materials/kr-tm.html zhao, y., and, karypis, g., evaluation of hierarchical clustering algorithms for document datasets”, retrieved april 2003 from: http://citeseer.nj.nec.com/zhao02evaluation.html microsoft word ajis 11 1 final.doc ajis vol.11 no. 1 september 2003 37 software engineering practices and tool support: an exploratory study in new zealand elizabeth a. kemp, chris phillips, jaimee alam, institute of information sciences and technology college of sciences massey university palmerston north email: e.kemp@massey.ac.nz, c.phillips@massey.ac.nz, jaimee_2@hotmail.com abstract this study was designed as a preliminary investigation of the practices of software engineers within new zealand, including their use of development tools. the project involved a review of relevant literature on software engineering and case tools, the development and testing of an interview protocol, and structured interviews with five software engineers. this paper describes the project, presents the findings, examines the results in the context of the literature and outlines on-going funded work involving a larger survey. keywords: project management, case tools, lifecycles, software engineers introduction research is currently being undertaken into the development of computer aided software engineering (case) tools to assist software engineers. a necessary first step is to find out about real users so that they can be provided with the kinds of tools that they find useful. there are few studies that attempt to model the activities of software engineers. singer et al. (1997) commented on the paucity of studies in this area noting that little effort had been expended in understanding how software engineers work. previous studies had focused solely on programmers who were usually university students working on small programs. they were themselves particularly interested in designing tools to help software engineers maintain legacy systems. groves et al. (2000) have carried out a survey of software requirement specification practices in new zealand but were primarily concerned with the requirements gathering activity. this study has a wider focus, with the project management procedures followed, the software process models adopted, and notations used all of interest. in addition, the case/i-case tools provided for staff and reasons for the use/nonuse of such tools were also of concern. a preliminary study of the practices of software engineers in new zealand, therefore, has been undertaken to inform debate in this area. the project involved a review of relevant literature on software engineering and case tools, the development and testing of an interview protocol, and structured interviews with five software engineers. this paper describes the project, presents an analysis of the data, examines the results in the context of the theory described in the literature, and outlines on-going funded work involving a larger survey background software engineering is a discipline involving the study of a systematic, quantifiable approach to the development, operation and maintenance of software (ieee 1993). software engineers can be considered as problem solvers (pfleeger, 1998) aiming to produce quality software. generally, software projects fall into the following categories: concept development, new application developments (e.g. meeting customer requests), application enhancements (corrections or upgrades), application maintenance or re-engineering legacy systems (pressman, 2001). another way of looking at a project is to determine what groves et al. (2000) call the kind of development undertaken by a company. they distinguish between specific products for customers (one-off contracts or mass production), in house software to support the running of the organisation and product support where software is included in goods sold by the company. an organisation can be involved with more than one kind of development. ajis vol.11 no. 1 september 2003 38 project management it is necessary to have guidelines for the efficient development and evolution of software (robillard et al., 2003). typical project management activities include planning and scheduling, estimating costs, resource allocation, software configuration management, measurement, risk management, software quality assurance and formal technical reviews. there are no general sets of tasks that apply to every project. however, task sets (work tasks, milestones and deliverables) can be associated with various project types. the larger and more complex the application is, the more rigorous the set of management tasks should be (wasserman, 1996). three kinds of rigour can be defined (presssman, 2001): strict, structured and casual. strict refers to the situation where all lifecycle and project management activities are undertaken. structured indicates that all the lifecycle and appropriate project management procedures necessary to produce high quality software are undertaken. finally, casual does not imply that procedures are ignored but that all the lifecycle activities and a minimum set of project management tasks are undertaken. various factors impact on the degree of rigour applied: • customer/developer communication • performance constraints • size of project • maturity of applicable technology • embedded or non-embedded characteristics • project staff rada and craparo (2000) observe that the effective management of software engineering projects is an important issue for organisations. whether or not the standards are developed internally, a company should follow any standard that it has selected. it may be a mistake to assume that all software engineers follow the precepts of the many writers on software engineering. behforooz and hudson (1996) believe that software engineering practices and principles are not fully recognised by either academia or industry. humphreys (1998) claims that the general practices of software engineers are poor by almost any measure. he substantiates this by describing the large number of reported systems (with defects) that are delivered late and over budget. he may well be correct but he offers no evidence, however, from empirical studies. software engineers, moreover, work for organisations with substantially different cultures (wasserman, 1996; tellioglu and wagner, 1999). what a software engineer does may depend not only on their knowledge and skills but also on what they are asked to do. the role of the organisation, therefore, also contributes to the success of a project. there have been studies of the important factors, which should be taken into account. as a result of analysing unsuccessful projects, kavanagh (2000) identified eight important lessons for organisations: • analyse and understand full implications of proposed systems • specify system taking account of business and user requirements • look at scale and complexity to see if project can be broken up into a series of small projects • involve senior management • provide high quality project management • draw up risk management and contingency plans • review project • plan to train staff keil et al. (1998) acknowledge that software projects are notoriously difficult to manage. in a study, experienced software project managers identified what they saw as the main risks to a project. three panels were set up of practitioners from the usa, finland and hong kong. all three panels identified the same factors even if they attributed varying levels of importance to them. these factors were: lack of top management commitment to the project, failure to gain user commitment, misunderstanding their requirements, lack of adequate user involvement, failure to manage end users expectations, changing scope/objectives, lack of frozen requirements, introduction of new ajis vol.11 no. 1 september 2003 39 technology, insufficient/ inappropriate staffing, and conflict between user departments. overall, the issues for organisations are much the same in both these studies. software process models and associated notations those who write about the discipline of software engineering describe the lifecycle activities that should be undertaken (lending and chervany, 1998; pfleeger, 1998; schach, 1999; sharma and rei, 2000; pressman, 2001; sommerville, 2001). life cycle activities depend upon the software development process followed such as the waterfall model, prototyping, data-centred or the evolutionary approach. the traditional lifecycle or waterfall model is typically described in a sequential fashion moving from stage to stage involving requirements specification, software design, coding and testing, each requiring a set of deliverables (sommerville, 2001). iterations between phases are required, however, to enable revisions to be made whilst an application is under development. the traditional lifecycle is still used in industry particularly for larger systems engineering projects (sommerville, 2001). in the data centred approach the focus is on the data rather than function and the specific method of entity relationship modelling (sallis et al., 1995). prototyping an application begins with some requirements gathering and moves on to the development of some part of the system. it assumes that the customer can communicate their requirements more effectively once there is something to look at (yourdon, 1994). the developer may undertake prototyping when there are no clearly stated requirements. it is also appropriate when the developer has little experience of the application domain or of the tools and languages that will be used to develop the system (yourdon, 1994). the subsequent prototype can be discarded as in throw away prototyping or retained as the basis for further development as in evolutionary or incremental prototyping (schach, 1999; pressman, 2001). problems can arise if the prototype is taken to be a working version of the application. there are other iterative and evolutionary approaches. evolutionary in this context means that increasingly complete versions of the software are developed (pressman, 2001). one such is the spiral model (boehm, 1988), which uses prototyping to reduce risks as well as incorporating the phases of the classic life cycle (pressman, 2001). another evolutionary approach involves component-based development and is centred round the object-oriented (o-o) approach. the unified process (jacobson et al., 1999) and rational unified process (kruchten, 1999) are examples of this. currently, there is also a move to using what are termed agile methods (beck, 1999; fowler, 2001). such methodologies “attempt a useful compromise between no process and too much process, providing just enough process to gain a reasonable payoff.” (fowler, 2001, p2) these methodologies emphasise the frequent delivery of software, regular (daily) contact with the customer and minimising documentation. some of the main advantages and disadvantages of this and other software process models (sallis et al., 1995; schach, 1999; fowler, 2001; pressman, 2001; sommerville, 2001 ) are shown in table 1. once a life cycle has been selected a software engineer has to make a decision about what notations to use to model and document the system. these make it easier to understand a problem and shape a solution (robillard et al., 2003). a large number of notations are available including entity relationship, data flow and data structure diagrams (jordan and machesky, 1990) and the nine notations specified in the unified modelling language, use case, class, sequence, collaboration, activity, state, component, deployment and package diagrams (booch et al., 1999). robillard et al. (2003) observe that gains in uniformity, reliability and productivity are made when notations are consistent across the various phases of the lifecycle. it is important though that staff have tools to support them and are trained in their use. ajis vol.11 no. 1 september 2003 40 process model characteristics advantage disadvantage iterative waterfall iterative appropriate when requirements understood iterations costly data centred iterative development of good data model less emphasis on function prototyping evolutionary establishes requirements danger of prototype becoming final system component-based evolutionary software re-use comparatively untried agile methods evolutionary close involvement with the customer not suitable for large projects table 1: comparison of process models case tools to assist software engineers to produce high quality software, case tools, which automate manual activities, have been developed. case tools are available to assist at all stages of the lifecycle and can be used for many tasks (sommerville, 2001). pressman (2001) provides a taxonomy of case tools by function, identifying 24 different categories (table 2). they can be as simple as a single tool for supporting particular activities or as complex as a complete environment involving a database, hardware, networks, operating systems, and myriad other components. those which bring together a suite of tools are referred to as integrated case (i-case) tools. business process engineering analysis and design process modelling and management pro/sim project planning interface design and development risk analysis prototyping project management programming requirements tracing web development metrics and management integration and testing documentation static analysis system software dynamic analysis quality assurance test management database management client/server testing software configuration management re-engineering table 2: classification of case tools (based on pressman, 2001) the proposed benefits of i-case tools include the following (pressman, 2001): • smooth transfer of information (models, programs, documents) from one tool to another and from one software engineering phase to another. • reduction of effort required to perform umbrella activities e.g. configuration, quality assurance and documents produced. • an increase in project control achieved through better planning, monitoring and communication. • improved co-ordination among staff members. the use of i-case tools also creates important challenges for tool developers. integration demands that relevant information be represented consistently, that standardized interfaces are available on tools, that a homogeneous way of communicating between tools and developers is available and that it is easy to move among versions of operating systems and hardware platforms (pressman, 2001). i-case users often experience a productivity decrease for the first 6 months, and it may take 12 to ajis vol.11 no. 1 september 2003 41 18 months before productivity gains are visible. introduction to any new technology could contribute to this productivity decrease. ideally, automated tools should support both lifecycle and project management activities. sharma and rei (2000) developed a framework to assess case tool usage, which essentially integrates both types of activities. the framework has three processes: production, coordination and organisation. the production process is decomposed into subprocesses related to lifecycle activities such as the representation of objects, relationships and processes, and testing/validation tasks such as test data generation and automatic restructuring of program code. the coordination process comprises the control and cooperation subprocesses. control is associated with tasks such as the enforcement of policies, resource management and auditing whilst the cooperation subprocess is concerned with enabling people to exchange information (either electronically or face to face). finally, the organisation process is split into support tasks to assist users to understand and use case technology effectively and infrastructure tasks (such as the development of a repository) to enable sharing of knowledge about a project. sharma and rei surveyed over a thousand organisations about their usage of case tools. only a seventh of those who responded had adopted case tools. the percentage of those adopters who actually used case tools for the purposes of production, coordination and organisation are 68%, 45% and 66% respectively. overall 60% of tasks were supported to some degree by case. the figure for coordination is surprisingly low. there were also some discrepancies within the categories. there was a very high case tool adoption rate for production activities such as the representation of objects, relationships and processes (95% of the respondents) with a much smaller proportion of adopters using case for testing and validation purposes (39%). similarly, within the “organisation” category, 86% used case for infrastructure activities but only 46% for support activities. overall, gray et al. (2000) concluded that it was hard to provide evidence of productivity gains or increases in the quality of the product from the use of case tools. further research into the practices of software engineers was required in order to build usable tools. reasons behind non-use of case and i-case tools whilst there are a large number of tools on the market they are not necessarily widely used. there is an extensive literature on the adoption of case tools which focuses on the factors that play a part in whether such tools are accepted or not. these can be categorised according to organisational, interface, people and tool related issues. organisational issues are concerned with cost (jarzabeck and huang, 1998; finnegan et al., 2000), vendor support (mcchesney and glass, 1993; finnegan et al., 2000) training in the use and benefits of such tools (mcchesney and glass, 1993; iivari, 1996; sorensen, 1993), use of appropriate life cycle methodology (holt, 1997), introducing case into the workplace (misra, 1990; orlikowski, 1993), compatibility with other tools (misra, 1990) and imposition of standards, for example making tool use compulsory ((iivari, 1996). case tool functionality has many dimensions. at a high level the issues to be considered include the complexity of the features provided as well as whether the tool supports multiple users and facilitates reverse engineering (lending and chervany, 1998; gray et al., 2000). the smooth transition between the analysis, design and implementation components is another important consideration (lending and chervany, 1998). some more detailed areas relate to the level of customisation available, the speed of access to the data dictionary and the quality of reports (misra, 1990). closely related to the functionality offered by a tool is its usability, that is its ease of use and ease of learning (lending and chervany, 1998; gray et al., 2000). ease of navigation as well as assistance with layout are concerns that have been mentioned in this regard (misra, 1990). the provision of adequate graphics primitives, suitable error messages and on-line help have also been discussed. interestingly, finnegan et al. (2000), found that people did not find case tools easy to use even though they were perfectly happy with the provision of help, ease of navigation etc. this is in line with several other studies (gray et al., 2000). users find the notations and editors inadequate for their purpose, proving to be an obstacle rather than an aid. sommerville (2001) also notes that case tools do not offer a great deal of support for members of a team. ajis vol.11 no. 1 september 2003 42 there are also factors associated with the needs of the people who have to use such tools. jarzabek and huang (1998) consider that case tools do not support problem solving and creativity. they believe that if a given methodology is enforced too strictly, software engineers are likely to spend more time fitting their ideas to the methodology rather than actually solving the software problem at hand. a more ‘free-style’ approach to software engineering, at the initial stages, at least, is likely to inspire software engineers to be more creative and therefore more successful in their endeavours. there is the requirement to build case tools that “bridge the conceptual gap between a computer system and human thinking” (p95). gray et al. (2000) also believe that case tools should complement the users’ creative problem solving processes. another important people factor relates to high expectations of case tools by users. such expectations that i-case should work on any situation and with any methodology are a major reason for failure (aaen, 1994; jarzabek and huang, 1998; chmura and crocket, 1995). finally, system developers prefer high autonomy but the icase tool does not allow for this (lending and chervany, 1998). developers do not feel motivated. lending and chervany conducted a tool usability measurement and discovered that neither intrinsic motivation (tool is fun to use) nor extrinsic motivation (tool is perceived to be useful) was high. orlikowski (1993) also pointed out that the attributes of people also played a part in developers reaction to case tools. she suggested that developers with considerable investment and experience of traditional systems development practices were more reluctant to use case tools than those with less time and experience in systems development. research methodology the objective of the study was a preliminary investigation of the practices of software engineers within new zealand, with particular reference to their use of development tools. it was decided to employ a qualitative research approach using structured interviews, rather than a quantitative method such as a questionnaire, because of the exploratory nature of the study and the complex processes involved. in particular, interviewing permits the collection of rich data for analysis as well as providing the flexibility for asking follow up questions (scott, clayton and gibson, 1991). the case study method is suitable where there are a large number of variables (yin, 1993). a multiple rather than a single case approach was chosen to provide greater diversity. once the research objectives were determined, the relevant literature on software engineering practice and the use of case tools was reviewed. this was used as the basis for developing the interview protocol which was subsequently revised after testing it out in a pilot interview. the focus was on determining the activities of the developer and other members of the team on a current project. the final version of the protocol had three sections: general questions to make the participant feel comfortable; in-depth questions on the project lifecycle, management activities and the tools used on a particular project; and, finally, there were some general questions concerning the use of case tools. a copy of the questions was sent in advance to the developers. purposive or theoretical sampling was used to select the participants (patton, 1990). five people were selected who, it was anticipated, would provide rich data about working as software engineers. all of them were required to be currently developing software and using tools. it was also seen as important to cover different environments rather than interview several developers from an organisation. the people selected were working on quite different kinds of projects and for companies of different types and sizes. three of the developers were male and the other two female. however, to preserve confidentiality for them and their organisation, all the developers are referred to as “he” in this paper. all the interviews were recorded and transcribed. the transcript and summary were sent to the interviewees for validation. as a result some small changes were made. the data for each case was analysed by inspection of the interviews with results entered into a table for each developer. in an iterative process these results were scrutinised and revised. summary tables were then derived to show organisational, project and tool usage information. the results for each case could then be compared and contrasted before similarities and differences with the literature were identified. threats to reliability and validity were minimised by careful case selection, using multiple researchers, employing purposive sampling, mechanically recording data, having subjects review the interview summaries, scrutinising in detail the results of the analysis and using constructs defined in ajis vol.11 no. 1 september 2003 43 the literature (patton, 1990; yin 1993). whilst there were only five cases, as perry and staudenmayer (1994, p37) comment, “data on real users, even if the sample is small, is revealing.” fred brooks (1988), too, notes the importance of obtaining real data on user behaviour for progress in software development to take place. results table 3 shows the overall information about the developers and the companies for which they worked. the companies in the study varied widely in size. a company size of 1 to 50 members is regarded as small, 50 to 150 as medium and 150 plus as large, in a new zealand context. four of the developers had extensive experience in the computing industry. developer a on the other hand had spent only eighteen months working in software development (following on from 5 years university study in it). he was employed by a commercial software development firm (organisation a), with a little over twenty staff, which handled new technologies with an emphasis on real time and embedded systems software. the owner of this company suggested developer a as a candidate for interviewing since, despite his lack of experience, he had a good understanding of the processes and procedures followed by the organisation. this proposal was accepted since it provided the opportunity to obtain data about software engineering practice in a small firm. developer employee title employee time in software industry size of company generic area a systems analyst limited small real time and embedded systems b analyst/ programmer extensive large administration c development manager extensive medium business applications d senior software engineer extensive large administration e software development manager extensive medium real time and embedded systems table 3: developers and their organisations the other developers all worked for larger companies. developers b and d were employed in the it department of very large companies in new zealand terms (organisations b and d respectively) with the responsibility for building administrative systems. developer c worked on a variety of projects, dictated by the contracts the company obtained for software development. finally, developer e was employed in an organisation which delivered real time and embedded systems software. these four people were all in charge of the project they described. whilst they had to work within the framework of company policies they had some ability to influence decisions. in this study, the project undertaken by organisation a was a concept development project where new technology was being applied (table 4). the applications in organisations b, and d would be classified in the pressman typology as new developments, responding to customer requests, that in organisation c a re-engineering project and in organisation e an application enhancement. there was also an element of concept development in the project undertaken by organisation e. the kinds of product development undertaken were one-off contracts (organisations a and c), in house development (organisations b and d) and mass production in organisation e. ajis vol.11 no. 1 september 2003 44 project developer project size project type kind of development developer a medium concept development one off contract developer b medium new development in house developer c large application maintenance one off contract developer d medium new development in house developer e medium application enhancement mass production table 4: projects in organisations the size of a project can be categorised, in a new zealand context, according to the number of people involved with the development process where less than three is small, from three to nine medium, and greater than ten is large (groves et al., 2000). given this classification scheme, the only large project was undertaken by organisation c with all the others medium in size. the project management and lifecycle activities that were engaged in, together with tools used to support them are described below. this is followed by an extensive discussion of case tool issues concerning adoption, selection and integration. project management activities and tool support project planning and scheduling was undertaken for all five projects. microsoft project was used for these purposes in organisations a, b and d. gantt charts were seen as particularly useful by developer d although he also mentioned that the resource allocations could become corrupted. the other management activities were more extensive in the case of organisations a, c and e. company a held the project plan, risk assessment, minutes of meetings, tasks allocated, hours per staff member etc in the “projects database” in lotus notes. project reports were produced from this data so that the project coordinator could review progress. niku was used in organisation c to manage the set up of the contract, the conditions, resources, time recording, invoicing etc. microsoft project was available in the company for those who preferred to use it for planning. finally, an inhouse project management framework, which provides templates, checklists, role descriptions, good practice guidelines and milestones, was used to direct such activities in organisation e with relevant information held in word. version control software was used on all the projects except that managed by developer d. whilst version control software was available within the company, its use was not seen as essential on this medium sized project. project management procedures were very strictly enforced in organisation a. all work tasks were performed within a group environment and the methodology, tools, outputs and related activities were prescribed. these standards were formally monitored and audited on a regular basis. in contrast, developer b used less formal procedures for developing systems. there were some means of enforcing consistency through the standards agreed by all the team members for project documentation such as data models. progress was monitored, too, on a regular basis. much though was left to the individual’s preferences with regards to team organisation to deliver systems on time. procedures were clearly laid out on the project managed by developer c. these were agreed in advance with the client. it was vital in this case to have clearly defined procedures as several people were working on the project including many from outside the company. there was some latitude, however, with regard to the choice of tools. where the number of developers on the project were small as in the case of organisation d there was not quite the same requirements for formal standards to be imposed. finally, organisation e had clearly specified procedures although the tools on a project could be changed, within limits. to classify these projects according to the degree of rigour applied to the selection of the task sets, organisations a, c and e are at the strict end of the spectrum with organisations b and d towards the structured/casual end. various tools were used for project documentation in these organisations including word, powerpoint, excel, and lotus notes. the documentation produced was usually extensive except in the case of developer b who stated that “documentation is minimal. the documentation that is most useful probably gets written in the comments of the code.” ajis vol.11 no. 1 september 2003 45 software process models four of the developers (a, c, d and e) followed what they described as an iterative waterfall approach. as one of the developers noted, the same stages (analysis, design etc) apply whether an organisation adopts the waterfall or other approach. in only one case, organisation d, was this decision made at the discretion of the developer otherwise it was tailored to the needs of the organisation. developer b used a data-centred methodology in association with the prototyping of customer requirements. mockups of the system were built and reviewed with customers. each mockup involved all the phases in the lifecycle. the prototype could be considerably revised once agreement on requirements had been reached. nearing the completion deadline, additional staff members could be added to work on the coding and delivery phases. lifecycle activities and tool support with regard to lifecycle activities undertaken, in all five cases the relevant requirements were obtained and appropriate models developed. a variety of tools were employed to construct the models. developer a used visio for drawing data flow diagrams (dfds) and entity relationship diagrams (erds). the creation of erds was seen as a central activity by developer b who used power designer for this purpose. developer b also drew flow charts in word to assist with customer communication. dfds and structure charts were sketched by hand if seen as necessary to clarify complex issues. class, collaboration, component and use cases were developed by organisation c using rational rose. state charts were drawn in visio as this was the preferred option of staff on the project. developer d used powerpoint as well as visio for drawing relevant diagrams – workflow and interaction diagrams. early prototyping of the interface was carried out in frontpage. finally, class diagrams and state charts were produced for organisation e initially in smart draw but rational rose was later used for this purpose. in organisation a, programmers developed the code based on a design specification (text and models) stored in lotus notes. code was written in visual basic and c++. version control was provided by cvs. once ready for alpha testing the code was stored in the “software release” database in lotus notes. developer b based his code on his understanding of the system as defined in the erd models previously drawn. he developed the system in sql using ingres application by form. unix tools were available for searching, editing and limited version control. data was loaded into the database via a spreadsheet. other team members assisted with coding. they attended project meetings and were aware of the documentation and models developed. a set of specifications in word incorporating all relevant diagrams was the basis of the coding for organisation c since developer c believed that the output from rational rose was insufficient to act as a design specification. jbuilder was the environment used for generating java code with wincvs for version control. developer d also ensured that programmers were provided with a specification in word which included all the important diagrams. vb6 was the language used for coding. in organisation e, the design documentation had to be completed before release for coding which was carried out using visual c++/studio. clearcase was replaced with cvs for version control purposes by developer e as it was more flexible and easier to use. testing was carried out on all projects. in organisation a, the test cases were stored in the project database. developer b set up a user training database for testing. customers were able to enter their own data in order to check out the system. the whole testing process was very complex in the case of organisation c. on the project in question, regression testing was carried out manually because the customer had no methodology for this. rational robot was available for this, however, within the organisation. functional testing was also largely manual. if there were no problems, the software was checked by a simple test script. it was then released for testing by an independent team. organisation c also used software developed in house for performance testing and simulating the loading environment as well as clear quest for defect management. the testing process was simpler in the case of organisation d where test plans were stored in word/excel and a defect tracking tool used internally to capture and assign defects. finally, automated tools were used to a limited extent by developer e. boundschecker was used to go through c++ code to look for ajis vol.11 no. 1 september 2003 46 memory leaks whilst smartbits set up traffic patterns on a network. on the whole though, a simple terminal emulator sufficed for testing purposes. for a summary of the tools used on these five projects for management and lifecycle activities see table 5. learning to use tools developer a thought that better use could be made of the tools provided in the company and that “everything is a learning curve.” developer b made it clear that there was a long familiarization process associated with powerdesigner and that staff did not always get the time they needed to come up to speed with it. completing the project on time had the highest priority. there was not always time for training new staff who joined the project in organisation c. instead, they were expected to learn on the job, with guidance provided by experienced team leaders. in organisation d, the staff were supposed to teach themselves vb6 by reading a specified text. no training for tool use was provided in organisation e where staff were expected to learn by using tools and accessing information provided by the tool or from the web. tools used (developer a) tools used (developer b) tools used (developer c) tools used (developer d) tools used (developer e) project management lotus notes niku word (in house framework) microsoft project microsoft project microsoft project cvs unix tools win cvs cvs clearcase documen tation microsoft word lotus notes microsoft word microsoft word microsoft word powerpoint excel microsoft excel analysis and design visio power designer rational rose visio powerpoint front page visio rational rose smart draw program ming visual basic and c environments ingres abf jbuilder vb6 sql visualc++/ testing lotus notes ingres in house tools clear quest word excel in house tool bounds smartbits table 5: tools by activities tool adoption, selection and integration organisation a enforced the usage of case tools from the beginning to the end of a project. developer a believed that their use helped the staff to meet deadlines. the current tools, however, did not support the transition from phase to phase of the lifecycle. developer a also recognised the complexity of case tools and the problem of ensuring that their power could be used effectively. he suggested that the reasons why i-case tools might not be used in the organisation included cost, training, major changes to the existing infrastructure, failure to appreciate the benefits of case and the need for a suite of products. the use of case tools was not always enforced for developers in organisation b. a tool might not be used if it was hindering application development. there was a long learning curve, for example, associated with power designer. transition between stages of a project took place manually. ajis vol.11 no. 1 september 2003 47 problems arose because there was no comprehensive data dictionary. moreover, developer b commented, without an i-case tool repository, an organisation must rely on institutional memory. this made it difficult for anyone to understand the whole system. the developer believed that the lack of i-case tools in the organisation could be explained by their cost and the time required to learn them. developer c set up an environment for the project where most of the tools were specified but there was some small element of choice eg. staff could use visio instead of rational rose for drawing state charts. organisation c was one of the two companies in this study to be using i-case tools such as rational rose. this had been selected in preference to togetherj because of its larger company base in new zealand. whilst rational rose fitted in at a high level with the process they followed during the project, it was used in an agile way and not to integrate information from each phase. there were issues, also, to do with complexity, cost, supposed benefits, and practicability. the large number of features provided made such products in the opinion of developer c bloated and difficult to use. this then generated a requirement for support. he also wondered whether the usage of rational rose warranted the cost since it was being used primarily as a repository for models. only a small number of its features were being used and it was thought that the same benefits may be obtained by using visio. the cost of licenses was also an issue. since the organisation had only been able to afford a limited number of licenses a developer sometimes had to wait to gain access to the software. developer c saw the benefits of i-case with respect to quality and productivity as overblown. he thought it might be possible to set up a good process with simple tools that could provide similar quality and productivity. even if round trip engineering (involving generation of code from models, reverse engineering etc) was undertaken, developer c wondered whether it was practical or possible to use an i-case tool for this purpose. in organisation d, the developer was able to choose (within budgetary constraints) all the tools for use on the project with the exception of vb6 which was the standard ide in the organisation. icase tools were not being currently used in the organisation for reasons of cost and size of development team. given, the small number of developers on a project, developer d maintained that there was only a limited requirement to communicate in an integrated fashion. if a larger project demanded this facility though then i-case would be considered with the costs and benefits evaluated. finally, in organisation e there was some scope to change tools, as occurred with the move from clearcase to cvs. organisation e was the only other company to use i-case. rational rose was again the company choice and developer e was considering whether to purchase a version that supported the development of real time systems. rational rose was not used for code generation although it was used for reverse engineering purposes. unfortunately, large, complex and cluttered class diagrams resulted from this process. there were also issues concerning the flexibility/complexity, cost, and appropriateness of i-case. developer e thought that the preference currently was for flexible tools which could be plugged together as needed by a project, e.g. program editor and compiler, rather than having a single large integrated environment, which would be complex, expensive and resource consuming. he believed that cost was a significant consideration in moving to a new tool since single simple tools tend to be relatively cheap to obtain (us$200 or less) and it was possible to purchase several of them for the cost of a complex integrated tool. finally, developer e also stated that, to be successful, a tool must help without getting in the way – hence the shift to cvs, and the caution in moving to an integrated case tool. most software engineers want to be producing code not drawing diagrams. discussion the results described above will be discussed in the light of the literature on this topic. important issues relate to the choice of project management activities, the selection of lifecycle and notations, and the use of tools. ajis vol.11 no. 1 september 2003 48 project management all the projects involved some project management activities (pressman, 2001; sommerville, 2001) but the balance between the activities differed significantly between the companies. it is worth considering what factors contributed to the rigorous processes followed in organisations a, c and e. the project type partly explains this (pressman, 2001). in the case of organisations a and e the project involved concept development whilst the project in organisations c was a demanding reengineering application. there were many other reasons though for the strict procedures followed. two of these organisations were producing software for customers on a one-off basis whilst the third was involved in mass production. their financial viability depended upon their success in meeting the requirements of customers. not surprisingly they had extremely centralised procedures for managing a project and enforcing standards. all these companies had a major requirement to track their progress to date and their expenses. organisations a and e also were set up to deal with the situation where staff worked on multiple projects and the costs had to be assigned to the appropriate account. other factors that are said to play a part when deciding upon the set of project management activities include maturity of the applicable technology, performance constraints, embedded versus nonembedded, size of the project, customer/developer communication and staff expertise (pressman, 2001). both organisations a and e were faced with similar problems to do with the use of emerging technologies and the development of embedded systems with performance constraints. they had developed procedures to ensure that projects of this type were properly delivered. developing an application in jdee in the case of organisation c was a difficult undertaking, given the lack of maturity of the technology. the project managed by developer c was also a large one involving staff from the company, the customer and an independent testing team. this, too, required that rigorous procedures be employed. customer/developer communication was also a critical issue when determining the set of project management tasks. organisations a and c who were both involved in one off contracts made great efforts to ensure that they always had the relevant information about the project available for the customer. the expertise of staff did not, however, play a part in the selection of project management procedures in organisations a, c and e. in organisation a this issue was irrelevant given the highly centralised procedures imposed. the tasks to be carried out were also clearly laid out in organisations c and e. as was noted earlier, the degree of rigour with which project management activities were applied by developers b and d was rated as structured/casual. even though the standards for documentation were determined by the team in organisation b, these were still followed (rada and craparo, 2000). it is interesting to consider the reasons, though, why the process framework could not be classified in these organisations as completely structured given the importance of the applications. firstly, the financial pressures were not so great for developers b and d working in it departments of large companies to produce in-house software. whilst in organisation b there were procedures in place for scheduling activities and reviewing, the documentation produced was limited. planning could be quite fluid. if a project was falling behind schedule then additional staff members could be added to the programming team. these programmers attended project meetings and were familiar with the documentation. this strategy was possible because staff were available and no extra costs had to be passed on to a client. secondly, for developers b and d, since the customer was on site, it was easy to involve representative users in prototyping and testing activities. thirdly, developers b and d were both experienced team leaders who were given considerable autonomy. developer d could also rely on the competence and knowledge of staff who had experience with a variety of languages and tools. fourthly, developer d only had small numbers of people to work with at each phase of the project. microsoft project sufficed in these circumstances. version control software was available within the company but its use on this project was not seen as necessary. commercial pressures, however, do not wholly explain the difference between the highly structured way in which, for example, organisation a worked when compared with the more informal work standards of organisation b. nor does the nature of the applications (concept development versus ajis vol.11 no. 1 september 2003 49 customer request) since the customer requests were of a fairly urgent nature and impacted on the business of the organisation. there were many reasons why a more thorough approach to project management would have been beneficial for organisation b. it had a large number of staff dedicated to software development (65 as opposed to less than 20 developers in organisation a) with teams to deliver applications. in addition there were considerable pressures to deliver quality systems in a timely fashion. the reason why it had chosen to implement less strict project management procedures relates partly to a difference in philosophy. management in organisation b decided to give autonomy to experienced staff who were expected to bring the project in on time, providing them with additional resources for programming if required. this is essentially a cultural difference between organisations b and a in particular (wasserman, 1996; telleglu and wagner, 1999). advice is given in the literature to assist organisations cope with the difficult task of project management (keil et al., 1998; kavanagh, 2000.) many of these important principles were followed by these organisations. all of them, for example, emphasised the importance of the customer. developer a worked closely with the customer for instance and developer c’s team worked on the customer site. the main weakness would be with regard to training staff for a particular project (kavanagh, 2000; robillard et al., 2003). developer a thought that better use could be made of tools, implying that training was inadequate. developer b made it clear that there were time constraints on training. it was only possible for new staff to be given on the job training for the project managed by developer c, with time again a major limitation. in organisations d and e staff were expected to upskill themselves. this was seen as possible because the staff employed were believed to be sufficiently well-educated to learn from texts, on-line tutorials etc. time constraints, therefore, had an impact on the training opportunities offered to staff on a project. it was not always possible to train staff in the use of basic let alone advanced features of tools. software process models lifecycle activities are a critical component of the software developers’ activities (pfleeger, 1998; pressman, 2001; sommerville 2001). in this study, four of the organisations used an iterative waterfall system tailored essentially to the requirements of the project (d) or the organisation (a, c and e). there were object-oriented aspects to the projects developed in organisations c and e (development of use cases, classes, state charts etc) but experience dictated that there was no need to move to a strictly o-o approach given that the lifecycle followed catered to the requirements of the organisation for managing application complexity. organisation b built administrative systems centred around databases so that a data-centred lifecycle developing erd diagrams for analysis and design purposes was an appropriate choice in these circumstances (sallis et al., 1995). to handle the problem of identifying functionality (sallis et al., 1995), prototyping, usually throwaway, was important for stabilising customer requirements (schach, 1999). the overall process followed was an iterative one. for none of the projects in these five organisations was an evolutionary process seen as applicable. perhaps the comparative recency of the rational unified approach, for instance, may be an obstacle as there are not enough people with experience using it. none of the organisations used an agile methodology (fowler, 2001) although it was clear that philosophically organisation b was trying to avoid too much process. documentation, for instance, was kept to a minimum. the customer was also very much involved in the testing process. lifecycle activities and tools used in all the organisations that were studied, well-accepted notations were used to model the system at the analysis and design phases entity relationship, data flow, data structure, class diagrams, state charts. these were usually drawn using an appropriate package: power designer (1 user), powerpoint (1 user), smartdraw (1 user) rational rose (2 users), and visio (3 users). the ease of use of visio for drawing was emphasised by all those who produced analysis and design diagrams (including state charts) with this tool. developer b’s preference for producing some drawings by hand will be discussed in the section on tool adoption. only developers c and e who used rational ajis vol.11 no. 1 september 2003 50 rose for class diagrams on their projects could ensure consistency across the notations used to describe the project as suggested by robillard et al. (2003). developing a complete design specification for the coding phase was an important activity for all but developer b. developer e observed that it was company policy that design documents had to be up to date before release for coding and this required effort and discipline when the tools were not integrated. the design specification was also kept for the purposes of customer reference in organisations a and c who had their clients to satisfy. it has already been noted that, philosophically, organisation b was committed to less formal procedures than the other companies. testing was carried out on the projects described with virtually no use of automated tools. rational robot was available in organisation c but could not be used on the project since at the time, the customer had no methodology for regression testing. performance was also checked by software developed in-house in organisation c. this was useful in both the implementation and maintenance phases of the lifecycle. it was possible to check the application could deal with the likely volume of transactions and ensure that when the system was up and running that the service level agreement was adhered to. comparison of case tool usage with the sharma and rei framework the framework used for case tool usage provided by sharma and rei (2000) encompasses both life cycle (production) and project management activities (coordination, organisation). all of the organisations used case for representation and analysis purposes. case tools only played a limited role with regard to testing (performance testing, generation of network traffic) but no organisation used them for automatic generation of tests or for analysing the program structure. with regard to project management, the coordination activities were supported fully by case tools in organisations a, c and d and to a limited extent in b and d. finally, all the organisations used case tools for support and infrastructure activities. it can be seen that case tools were not widely used for testing purposes. interestingly, one of sharma and rei’s conclusions from their study was that the adoption and infusion levels of case was lowest for testing and validation purposes. enforcing tool use case tools were provided for their users by all the companies although only organisation a made use of them all compulsory (iivari, 1996). organisation c in which large applications were usually built on the client’s premises set up an environment for each project. in this case, however, staff were at liberty to use the tools they preferred for project planning or drawing state charts. the other organisations on the other hand allowed their very experienced staff more autonomy. it was not mandatory for developer b or his team to use power designer. developer d was even allowed to choose some of the tools used on a project within budgetary constraints. this was only practicable because of the breadth of experience with tools and environments which existed among the software development group. developer e was permitted to replace a tool that was seen as inadequate. tool adoption all the developers noted problems for staff in coming to speed with the functionality of the case/icase tools. there was a major problem with using rational rose for drawing state charts in organisation c, for example. this is in line with the findings of lending and chervany (1998) that such tools can be time-consuming to learn and difficult to use. all the developers had problems moving from one phase of the lifecycle to another, even developers c and e who were using icase tools. producing design specifications, therefore, involved integrating diagrams and text either in a lotus notes database (a) or in word (c, d and e). people factors were also significant. developer b in order to understand the system and not be constrained by case tools chose to draw some models by hand. this is similar to the freestyle approach, advocated for the initial stages of a project by jarzabek and huang (1998). attributes of individuals also played a small role with respect to the use of case (orlikowski, 1993.) developer ajis vol.11 no. 1 september 2003 51 b, the most experienced of those studied and well-versed in traditional methods, showed the least enthusiasm for case tools although they were used for some activities. developer b also enjoyed the high degree of autonomy permitted (lending and chervany, 1998). the case tools that were used had to be suitable for their purpose (lending and chervany, 1998.) developer c selected wincvs as a far more reliable tool than the proposed alternative. developer e changed to cvs from a much more powerful tool that offered far too many features. if a suitable tool was not available then sometimes one was written in house that was tailored to the company’s requirements, such as performance tracking software (organisation c) and a defect management tool (organisation d). finally, the situation also occurred where a tool (rational robot) could not be used because it did not fit in with the client’s procedures. purchase of i-case only organisations c and e purchased an i-case tool. whilst recognizing these as effective drawing tools, there were some reservations. the cost (jarzabeck and huang, 1998; finnegan et al., 2000) and effectiveness of such tools were questioned. both developers saw them as overly complex (lending and chervany, 1998). developer c thought that i-case had too many features and could be difficult to use. rational rose had been used for reverse engineering by developer e who was not impressed with the overly complex class diagrams produced. in neither of these organisations was the tool used to its full potential (sorensen, 1993). organisational factors played an important role in determining whether or not to purchase i-case. cost (jarzabeck and huang 1998) was cited as one of the reasons why i-case tools were not purchased. organisations b and d, however, had they seen a real need for such tools, could easily have afforded them. other reasons include the time to train staff (mcchesney and glass, 1993) and introducing i-case into busy workplaces (misra, 1990). when several employees have to be brought up to speed with a new technology and possibly a new methodology then problems arise. the infrastructure may be missing and it may be difficult to match preferences for methodologies/languages with suitable tools. there was also a concern expressed by developer d that the introduction of such a tool would slow down the production of code. the functionality and usability of i-case tools were also seen as an obstacle. the lack of flexibility of such tools was seen as a problem by developer a. there was, as yet, no requirement to support multiple users in organisation d. claims for i-case it is interesting to consider pressman’s (2001) claims for i-case in the light of the results reported here. only two organisations had an i-case tool and neither used it to integrate models or as a repository. developer c believed that the system was too large for all its aspects to be defined within a single repository. developer e thought that a heavy commitment was needed to learn the tool and take advantage of the integrated facilities. on the other hand the project repository of organisation a in lotus notes enabled it to reduce effort to perform umbrella activities such as configuration management, increase its project control and improve co-ordination without i-case. the small number of developers on the project headed by developer d meant that there was not quite the same need to co-ordinate activities. developer b noted the problem that arose in organisation b without an i-case repository, that is the lack of documentation about what had happened previously. further work the findings reported in this study are the result of only five case studies. exploratory studies of this type are required, though, so that the software engineering community can start to find out how practitioners actually work. it is essential to find out about real users so that they can be provided with the kinds of tools that they find useful. however, if further progress is to be made towards improved software engineering processes and methods, better computer-aided software engineering (case) tool support, and improved training of software engineers, additional research is necessary. ajis vol.11 no. 1 september 2003 52 a questionnaire has been piloted and sent out to several hundred software developers in new zealand. conclusion in this study the practices of five software engineers were analysed with the results compared to the software engineering literature. the projects reported on varied in type, kind of development and, to some extent, size. project planning and management was undertaken by all the developers studied but the associated task set could be described as rigorous for three companies and structured/casual for the other two. all organisations followed iterative processes, in four cases a waterfall model and in the fifth a prototyping approach. suitable notations were used but only two organisations benefited from using the same notations for analysis and design. all the companies provided a selection of case tools for their staff. overall, various choices could be made about project management procedures, life cycle process and tools except in organisation a where all of these decisions were determined by organisational policies. the fact that developer a was relatively junior whilst the others managed the projects they described has no impact on the results reported here. organisation a was highly centralized under its ceo for reasons that have already been outlined. the preferences of the managers had some impact in the other cases. with regard to the software model, developers b and d were able to choose an approach they thought suited the project. the situation pertaining to tools was more flexible since some degree of choice was permitted to developers b, c, d and e. overall, there is an interesting tension in the results reported here between centralised and decentralised processes. on the one hand, one developer was proud of the organisation’s centralised project management activities and, on the other hand, there were two developers who enjoyed the autonomy that they were given in developing appropriate processes. because of time pressures, training in the use of case/i-case tools was limited and they were not always used to their full potential. tools could be rejected if they did not seem to meet the perceived need. in only two of the five organisations was an i-case tool used. cost and other organisational issues such as introducing i-case when a team was under pressure to develop systems were seen as obstacles to investment in this area. the preference seemed to be for flexible, light-weight tools. despite the various differences reported, all of the participants in this study followed appropriate procedures for developing software. there was no instance where the process was ad hoc. perhaps this was because no one was involved in small projects. many factors appeared to play a part in accounting for the differences reported: the project type, the kind of development, the financial situation of the company, the culture of the organisation, the size of the project, the experience of staff, and the developer’s preference. overall, there appears to be a trend to use more highly formalised procedures (organisations a, c and e) when companies are under considerable commercial pressure and dealing with new technologies. more autonomy (within a defined framework) was granted to developers b and d who were delivering in house software. acknowledgement the authors gratefully acknowledge funding from the new zealand foundation for research, science and technology to carry out part of this research. references aaen, i. (1994) problems in case introduction: experiences from user organisations, information and software technology, 36, 11, pp 634-654. beck, k. (1999) extreme programming explained: embrace change, reading, ma: addisonwesley. behforooz, a. & hudson, f. (1996) software engineering fundamentals, new york: oxford university press. ajis vol.11 no. 1 september 2003 53 boehm, b (1988) a spiral model for software development and enhancement, computer, vol. 21, 5, pp 61-72. booch, g., rumbaugh, j. & jacobson, i. (1999) unified modeling language user guide, reading mass: addison-wesley. brooks, f. (1988) plenary address, proc computer-human interface conference, new york: acm press, pp 1-13. chmura, a. & crocket, d. (1995) what’s the proper role for case tools? ieee software, 12 (2), pp 18-19. finnigan, d., kemp, e. & mehandjiska, d. (2000) towards an ideal case tool, smt 2000, wollongong, ieee computer society press, pp 189-197. fowler, m. (2001) the new methodology, http://www.martinfowler.com/articles/newmethodology.html, updated nov 2001. gray, j.p., liu, a. & scott, l. (2000) issues in software engineering tool construction. information and software technology. 42, pp 73-77. groves, l., nickson, r., reeve, g., reeves, s. & utting, m. (2000) a survey of software requirements specification practices in the new zealand software industry, proceedings aswec 2000: australian software engineering conference 2000. (ed.) d. d. grant. ieee computer society. pp 189-201. holt, j. (1997) current practice in software engineering: a survey, computing and control engineering journal, 8 (4), pp 167– 172. humphreys, w. (1998) why don't they practice what we preach? carnegie mellon university: software engineering institute. ieee (1993) ieee standards collection: software engineering, ieee standard 610.12 – 1990, ieee. iivari, j. (1996) why are case tools not used? communications of the acm, 39, 10, pp 94-103. jacobson, i., booch, g. & rumbaugh, j. (1999) the unified software development process, reading, mass : addison-wesley. jarzabek, s. & huang, r. (1998) the case for user-centered case tools, communications of the acm, 41, 8, pp 93-99. jordan, e.w. & machesky, j.j. (1990) systems development: requirements, evaluation, design, and implementation, boston: pws-kent publishing company. kavanagh, j. (2000) project sick-list, the computer bulletin, march 2000, pp 24 –26. keil, m., cule, p., lyytinen, k. & schmidt, r (1998) a framework for identifying software projects, communications of the acm, vol. 41, no 11, pp 76-83. kruchten, p. (1999) the rational unified process, an introduction (2nd edition), reading, mass: addison-wesley. lending, d. & chervany, n.l. (1998) case tools: understanding the reasons for non-use, computer personnel, april 1998, pp 13-24. mcchesney, i.r. & glass, d. (1993) post-implementation management of case methodology, european journal of information systems, 2, 3, pp 201-209. misra, s.k. (1990) analyzing case system characteristics: evaluative framework, information & software technology, 32, 6, pp 415-422. orlikowski, w. j. (1993) case tools as organizational change: investigating incremental and radical changes in systems development, misq, vol. 17, no.3, pp 309-340. patton, m. q. (1990) qualitative evaluation methods. beverly hills, ca: sage publications. perry, d., staudenmayer, n. & votta, l. (1994) people, organisations and process improvement, ieee software, july, pp 36–45. pfleeger, s. l. (1998) software engineering: the production of quality software, new york: prentice hall international. pressman, r. s (2001) software engineering: a practitioner’s approach, 5th ed., singapore: mcgraw-hill. rada, r. & craparo, j. (2000) standardizing software projects, communications of the acm , vol. 43, no 12, pp 21-25. robillard, p., d’astous, p., & kruchten, p. ( 2003) software engineering process with the upedu, boston: pearson education. ajis vol.11 no. 1 september 2003 54 sallis p., tate, g., & macdonnell, s. (1995) software engineering practice, management, improvement, sydney: addison-wesley. schach, s. (1999) classical and object-oriented software engineering, new york: mcgrawhill. scott, c.a., clayton, j.e. and gibson, e.l. (1991) a practical guide to knowledge acquisition, reading, mass: addison-wesley. sharma, s. & rei, a. (2000) case deployment in is organisations, communications of the acm, 43, 1, pp 80-88. singer, j., lethbridge, t., vinson, n. & anquetil, n. (1997) an examination of software engineering work practices, proceedings of cascon '97, toronto, pp 209-223. sommerville, i. (2001) software engineering, reading, mass: pearson. sorensen, c. (1993) what influences regular case use in organisations? – an empirically based model, scandinavian journal of information systems, 5, 1, pp 25 –50. tellioglu, h. & wagner, i. (1999) software cultures. exploring cultural practices in managing heterogeneity within systems design, communications of the acm, vol. 42, number 12, dec. 1999, pp 71-77. wasserman, a. i. (1996) towards a discipline of software engineering, ieee, 13(6), pp 23-31. yin, r. k. (1989) case study research: design and methods, london: sage publication. yourdon, e (1994): object-orientated systems design, usa: prentice hall international editions. microsoft word 13 1 marina.doc ajis vol 13, no. 1 september 2005 159 creativity, requirements and perspectives oliver hoffmann10, david cropley11}12, arthur cropley13 lemai nguyen14, and paul swatman15 abstract is there room for more creativity in information systems? this article grew out of an awre’04 panel discussion on creativity in requirements engineering, and the impact of requirements engineering on creativity in systems engineering and systems use. both panel and article were motivated by the goal of identifying a framework for understanding creativity in a larger context and thus establishing a potential structure for future research. the authors’ research backgrounds differ widely and, at times, our views conflict – occasionally, quite sharply. we make underlying world views our own and those of relevant disciplines – explicit; identify the paradox caused by the need to be functionally creative while leaving room for creativity in successive stages; and argue for a multiparadigm framework for resolving this paradox. keywords: information systems, creativity, requirements engineering, world views introduction creativity is widely acknowledged as vital to society. higgins (1994) lists ten “primary challenges” that business faces in the twenty-first century, and suggests that innovation and creativity are the mechanisms to address these challenges. there is also an important link between creativity and social welfare. disciplines such as business, industry, trade and commerce, international relations, science, medicine, engineering and, of course, information systems all suggest responses to those challenges. the link between the challenge and these disciplines is not simply a matter of social or educational philosophy: the importance of creativity to prosperity, even economic survival, is stressed by many authors. “the economic importance of creativity is clear because new products or services create jobs” (sternberg & lubart 1992). to take a particularly striking example of the concrete importance of creativity, the present era of global terrorism requires that technological solutions to the problems posed by terrorist activities exhibit high levels of creativity, if they are to withstand the efforts of terrorists to circumvent them.how do information systems and creativity relate? according to the united states national academies committee on information technology and creativity (2004), the effects would be mutually supportive: “just as the engagement of it helps shape the development of inventive and creative practices, so also can inventive and creative practices positively 10 oliver@hoffmann.org 11 david.cropley@unisa.edu.au 12 systems engineering and evaluation centre, university of south australia 13 acropley@bigpond.net.au, emeritus professor of educational psychology, hamburg university 14 lemai.nguyen@deakin.edu.au, school of information systems, deakin university 15 paul.swatman@unisa.edu.au, computer and information science school, university of south australia ajis vol 13, no. 1 september 2005 160 influence the development of it”. the committee on information technology and is u seis dev e lo pm en t cre a tiv e p ra c tise en g in ee rs u se rs in fo rm ation s y s tem figure 13: is and creative practise: potential mutual benefits creativity (2004) refers to “creative practices”, primarily arts related, but the claim of mutually beneficial interaction between information technology and creativity is made in a quite general way. does the increased use of information systems enhance human creativity? does increased engineering creativity improve the quality of information systems? what do we have to know about information systems in order to make more room for creativity? although the topic of creativity would, in general, include “creative information systems”, where the system itself displays creative behaviour, we will exclude this aspect from our article and focus, instead, on human creativity in relation to information systems. roughly speaking, there are two actor-sets involved: those developing information systems (whom we might term “engineers”) and those using them. we will introduce a distinction within the former category, dividing the group of engineers into requirements engineers and system designers. even though these three groups do not necessarily interact with each other directly, they are all linked by their involvement with information systems, and the actions of each influence the creative potential of all. we will analyse what is known about the role of creativity in these three loci, and how differences in their understandings and conceptualisations might inform a more general framework. we will start our analysis with the last group: what expectations do creative users have of information system properties? ajis vol 13, no. 1 september 2005 161 system designers users information system re figure 14: actor-sets linked by their involvement with is: three loci for human creativity what creative users expect supporting creativity is not necessarily the same as increasing productivity. rather than simply making our routine activities more efficient (essentially a quantitative change), information technology can achieve qualitative effects by extending creative potential, thus helping to “reframe and redirect the expenditure of human effort, generating unanticipated payoffs of exceptionally high value” (committee on information technology and creativity 2004). on the other hand, the use of computing can also decrease creative potential: computer software always relies on specific metaphors, and enforcing the wrong metaphor can inhibit users from exploring creative alternatives, leading to solutions that are rich in detail but lack core creativity (lawson 1999). as with all tools, the use of information systems introduces some new capabilities, while restricting others. but information systems are not like other tools. being programmable and adaptable, information systems should be less likely to impose a particular way of working than other tools.to creative users, information systems might not seem like tools at all, but rather like instruments (beynon et al. 2001): they present obvious benefits and limitations, both of which must be studied and mastered, and which can both be used in creative ways. eventually, creative masters can achieve a form of harmony or flow (csikszentmihalyi 1997) when interacting with their instrument. but are information systems built with such a use in mind? as we will discuss below, increasing information systems development creativity might actually decrease potential creative use: information systems engineers display their creativity via the power of their creations, and typically, more powerful computing is regarded as equivalent to empowering computer users. for creative information systems use, however, this might not always be the case. any information systems user can be creative. but for some, like artists, creative designers, scientists developing a new theory, or engineers working on innovative solutions, creativity is an essential goal. like a magnifying glass, these groups of users expose the role of information systems in creative processes. so what do creative users want information ajis vol 13, no. 1 september 2005 162 systems to do for them? in some ways, they expect the same basic services as all users: computation, storage, and communication. “tasks for helping people be more creative more of the time” shneiderman (2002) can be classified according to the services they depend on (shneiderman’s tasks in boldface): is service for creative users 1. modelling. the ability to compute on existing information can serve as the basis for various kinds of models, which can be used for visualisations of objects and processes, exploration of scenarios and of complete works from various fragments. is service for creative users 2. book keeping. not surprisingly, the ability to store information is one of the key reasons to use an information system, including for creative purposes. this service helps with tasks such as reviewing previous steps in a process and searching for compatible material. is service for creative users 3. communication. human creativity frequently (some argue always) involves some sort of collaborative effort, and is can provide means for relating to others or disseminating results.some creative computer users are also computer experts, but most are not. subsequently, most see the computer as a black box, and are not interested in intricate system performance details, only in overall system behaviour towards them. from this perspective, behaviourist criteria for the needs of creative users can be formulated, for instance those compiled by candy & edmonds: the system must enable the user to is behaviour criterion 1. adopt an holistic perspective on the task or problem to achieve this, the user should have access, for example, to overviews, multiple views and alternative representation of the data in respect of the developing solution. is behaviour criterion 2. keep several channels of exploration open in parallel to achieve this, the user should be able to alternate between tasks freely, fluently and quickly. this implies having simultaneous access to different forms of visual data and methods for knowledge interaction. is behaviour criterion 3. explore and evaluate existing design knowledge in relation to other heterogeneous knowledge sources, generate and evaluate new concepts and apply constraints as appropriate. candy & edmonds derive these criteria from studies of what they call creative knowledge work, primarily scientists’ use of information systems for analysis and theory development. hewett (2005) uses the term creative problem solving environments for information systems supporting such work. similar observations have been made in studies on artists (edmonds & candy 2002) and their use of information systems: while it is important that information systems help with modelling (is service for creative users 1), it is equally important that the choice of model and representation is always left to the user, enabling a holistic overview (is behaviour criterion 1). in other words, information systems ajis vol 13, no. 1 september 2005 163 computing power has to be balanced with user control. with every new information systems capability, ways of maintaining user capability for informed choice has to be provided. clearly, creative users expect information systems engineers to provide systems with such properties. whether engineers are able to fulfil this expectation will in part depend on their understanding of creativity that of information systems users and their own. creativity in engineering according to horenstein (2002), engineering is “a career full of discovery, creativity and excitement”. a simple definition of engineering is that it harnesses both natural resources and the laws/forces of nature to provide society with products, systems and services, for the good of humankind. but what is engineering creativity? burghardt (1995) put it plainly: “technology is the manifestation of engineering creativity”. the products of engineering creativity are physical objects, complex systems such as a submarine or a business information system, or processes in the sense of a service, technique or method (a manufacturing process, a control process, a logistics service). what makes products creative? it seems more or less self-evident that the first characteristic of creativity is novelty—as the psychologist bruner (1962) put it, creativity must create surprise. however, surprisingness is not sufficient on its own. if it were, every absurd suggestion would be creative. thus, creative solutions must be not only novel, but also relevant and effective (bruner 1962). in artistic creativity or any form of non-functional creativity, surprisingness alone may be regarded as automatically being effective, or effectiveness may be judged according to purely aesthetic or expressive criteria. in the case of engineers, however, effectiveness means, in the first instance: “will it work?”. other functional criteria of effectiveness are whether or not a solution can be financed, whether it can be produced with available resources of skills and money, whether it it is consistent with a firm’s culture, whether it will be accepted by customers or the public, and so on. if it will not, no amount of novelty, even novelty that does the job, can be inserted into what already exists, without great risk. the consequences of generating novelty that lacks effectiveness are quite different for information systems engineers and information systems users. for an artist or author, for instance, it might mean embarrassment, damage to the reputation, or loss of income, although it might also mean praise for being avant garde. for engineers in general, however, it might well mean bankruptcy for a firm, endless lawsuits, or even a disaster causing great loss of life, such as the collapse of the west gate bridge on october 15, 1970, australia’s worst industrial accident. thus, paradoxically, in engineering creativity effectiveness takes precedence over novelty, as absurd as that may seem. novelty without effectiveness is simply too dangerous! this means that for engineering, there is a tension between the need to produce novelty and the necessity of effectiveness, with the latter likely to be emphasized at the cost of the former. cropley & cropley (2005) refer to creativity possessing this particular property as functional creativity, where the emphasis is not only on characteristics such as novelty, but also on the important engineering concept of effectiveness, forming “…creativity with a purpose” (burghardt 1995). this is not to say that non-engineering creativity is devoid of purpose: as mentioned above, creativity in general is essential for progress and growth, which in itself is enough to give ajis vol 13, no. 1 september 2005 164 all creative efforts purpose. but in contrast to engineering creativity, general creativity cannot be judged against predefined criteria of relevance or effectivity. to enable creative users to achieve a more holistic view as they desire information systems engineers would have to have a clear understanding of the differences between their own creative process and the needs of their clients. but as we will show, the tension between engineering precision and human creativity extends to the process of formulating such needs as well. creativity in requirements engineering engineering is focused on meeting the needs of users or customers humans or humankind (voland 2004). from a pure engineering point of view, the purpose of requirements engineering (re) is to translate that need (often ill-defined and vague) into a formal, rigorous statement, so that engineers can build a satisfactory solution to it. when aiming at translating creative user needs for information systems designers, one would therefore expect re to do the job. but as in the general engineering case, this runs contrary to how requirements engineers see their own work. like other kinds of engineering, re is typically seen as a problem solving process. “re can be defined as the systematic process of developing requirements through an iterative co-operative process of analysing a problem, documenting the resulting observations in a variety of representation formats, and checking the accuracy of the understanding gained.” (loucopoulos & karakostas 1995). based on this definition, re problem solving is described as an application of engineering methods and techniques to analyse and solve a requirements problem. a large proportion of re research efforts has been placed on the development and evaluation of various engineering modelling methods and techniques to express and specify requirements more completely and precisely. what place does creativity have in re? many re authors and especially re practitioners acknowledge and agree on the importance and the role of creative techniques and heuristics in applying proposed engineering principles to solving requirements problems. but how creativity occurs in this engineering application has somehow been neglected in re research. how should requirements engineers incorporate the needs of creative users if their own creativity is hardly accommodated? requirements engineers’ creativity enables them to deal with new problems/domains, unique contexts, or novel applications and combinations of existing methods and techniques. these abilities are assumed to be based either on “mysterious” natural intuitions, heuristics which accumulate over years of experiences, and/or a deliberate application of creativity techniques such as brainstorming, mind mapping, scenario simulation, and so forth (gause & weinberg 1989; batra & davis 19r92; shanks & simsion 1992) . creativity in re has recently received increasing attention in the re community. nguyen et al. (2000) reported that requirements engineers occasionally re-conceptualise the users’ requirements and restructured their requirements model as a result of unexpected insight. maiden & gizikis (2001) pointed out the lack of and calls for more research into supporting creativity in re. since then creativity workshops and tutorials have been continually held at awre’02, re’03, re’04, awre’04 and re’05 conferences. recent research effort into creativity in re can be classified into the following main streams: understanding and facilitating the creative problem solving process (nguyen & swatman 2003; pennel & maiden 2003; maiden et al. 2004; nguyen & swatman 2006), developing brainstorming techniques to undertake requirements elicitation (mich et al. 2004), incorporating creativity ajis vol 13, no. 1 september 2005 165 in re education (armarego 2004), and investigating context factors and socioorganisational context within which re creativity is exercised (cybulski et al. 2003; dallman et al. 2005). in our view, to effectively foster and support creativity in re it is vital to understand how the re process occurs to define the creative path from requirement problem to solution. action research conducted by nguyen & swatman (2000, 2003) has yielded some unexpected findings on the creative path from requirement problem to solution: viewed from the problem solving perspective, creative steps appear as catastrophic interruptions. consequently, the re problem solving process can be described as a catastrophe-cycle process consisting of cycles of building up an understanding and structuring of the problem and occasionally re-conceptualising and restructuring it though opportunistic gaining of accidental knowledge about the problem. this process can be explained through an evolution of three types of complexity in the requirements model: essential, incidental and accidental complexity. essential complexity represents the intrinsic understanding of the requirements problem gained, agreed upon and embedded in the requirements model. ideally, this type of complexity grows over time towards an objective completeness of the problem complexity. essential complexity is rather subjective and may be assessed as agreed knowledge about the real world problem between different stakeholders. overall, we can assume that our understanding of the problem must not be decreasing as the problem space is explored and new requirements are uncovered by the requirements engineer. therefore, essential complexity grows over time as learning takes place, the problem space is explored, and different creative ideas and thoughts about user needs are elicited, discovered, invented, expressed and added into the requirements model. incidental complexity represents complexity of representation rather than substance in the model. it shows the poor fit between the structure of the requirements model and the need that the model attempts to mirror. as new essential understanding is continuously gained and agreed upon, new requirements are added to the model. the more requirement components are added into the model, the more difficult it becomes to add new components into the increasingly complex existing model. therefore, incidental complexity grows exponentially as the structure of the model grows. as new model components are expressed, added and modified in the requirements model, incidental complexity increases due to inevitable entropy and a growing gap between the requirements model and the real world need. therefore, the overall complexity grows consisting of both essential understanding gained, and incidental complexity growth. during this process, creativity plays a role in requirements expressions and in modelling and evaluation of different modelling choices using various criteria (for example see shanks & simsion 1992). however, the most significant creative steps occur when the entire requirements model is transformed. at some unexpected “aha!” moment, the requirements engineer re-conceptualises the problem, and restructures the requirements model based on the new essential knowledge formed. from the classical problem solving perspective, this moment appears as a disastrous disruption. accidentally discovered essential knowledge can be referred to as accidental complexity to denote a significant change in the model structure which now reflects a new essential understanding of the real world problem. accidental complexity represents hidden knowledge in the requirements model which becomes explicit only as a result of re-conceptualisation insight at the crisis points. after the model is restructured, the accidental complexity becomes a part of the essential complexity which continues to grow ajis vol 13, no. 1 september 2005 166 over time. as a result of the re-conceptualisation insight, the model is restructured and the essential complexity increases in the form of accidental complexity while incidental complexity decreases. the newly restructured model becomes the new basis for the next catastrophe cycle of the construction and restructuring of the requirements model. this type of engineering creativity disguised as disastrous disruption was discovered while attempting to uncover cognitive behaviours of requirements engineers. a number of authors (such as guindon 1990; visser 1990; khushalani 1997; carroll & swatman 1999; nguyen et al. 2000) suggest that the re process be understood as an insight-driven and opportunistic rather than a systematic, top-down and balanced development process. however, how to trigger insight or make restructuring happen seem to remain an unanswered question: “a restructuring event has an involuntary character; it is experienced as something that happens, rather than as something the problem solver does” (ohlsson 1984). this may explain why insight and support for insight have received limited attention in re. the evaluation stage therefore is either missing or not treated explicitly in the literature. recently, maiden and his colleagues (2004) utilised creative workshops to simulate and structure the re creative process on the basis of wallas’s (1926) model discussed below. nguyen & swatman (2003, 2006) associate the preparation and incubation stages with the creative and reflective structuring of the requirements model. they suggest that various creativity techniques, such as brainstorming, idea generation, scenarios and simulations, and ad hoc design rationale could be used to support and promote creativity during these stages. cognitive processes the above explanation of the re catastrophe cycle process is consistent with wallas’s (1926) description of the creative process. according to him, the creative problem solving process consists of four stages: preparation, incubation, illumination (insight), and the verification of illumination. hadamard (1954) added that creativity involves an unconscious mental process and insight is seen as a breakthrough by unconscious ideas. the growth of essential and incidental complexity of the requirements model can be seen as a result of the preparation and incubation stages. the insight (or aha! moment) and its restructuring role in problem solving are often discussed and recognised in gestalt psychology. the insight concept is based on the illumination stage in wallas’s creative process model. restructuring is often associated with the occurrence of insight, a sudden, unpredictable flash of ideas which often involves a surprise and a solution. according to gestalt psychologists, problem space restructuring is crucial in problem solving and reveals a new way of looking at the problem, often from a broader perspective. where does novelty come from? two concepts are very useful here: • non-algorithmic thinking: the first modern psychological analysis of creativity (guilford 1950) laid down the basis for understanding creative thinking. he distinguished between convergent thinking, aimed at finding the one and one only best or correct answer through the application of conventional logic to existing knowledge (for instance: “what number comes next in the series 2, 4, 8,…”), and divergent thinking, aimed at producing a variety of alternative answers to open problems (for instance: “give as many uses as you can for a tin can”). the essential idea is that ajis vol 13, no. 1 september 2005 167 of a contrast between thinking that follows “the rules” of logic, good sense, order, and so on, and thinking that breaks away from the usual patterns by going beyond the obvious, abandoning the known and branching out, crossing boundaries, making unexpected combinations, seeing remote associations, building networks of ideas, and so on. resnick (1987) used the term non-algorithmic thinking to refer to the latter. however, psychological research has differentiated the simple equation: divergent thinking = creativity, and current research has reestablished the value of convergent thinking in creativity (e.g. (cropley 2006)). • nuanced judgement: subsequently, psychologists have analysed creative thinking in terms not just of the kind of thinking (see the examples just given), but also in terms of metacognitive processes, i.e., the “executive processes” in thinking, that allow people to organize and keep track of their own non-algorithmic thinking. metacognition may be defined as awareness of one’s own cognitive processes rather than the content of those processes together with the use of that self awareness in controlling and improving cognitive processes (biggs & moore 1993). flavell, one of the first to introduce the concept of metacognition, suggests that metacognitive knowledge (knowledge about one’s own cognition) includes three interacting variables: knowledge of strategies, of person (self and others) and of tasks. metacognitive processes include reviewing one’s own knowledge, consciously directing attention, identifying and marshalling one’s own cognitive resources, selecting a promising line of attack, designing a workable solution strategy, evaluating one’s own progress, recognizing and avoiding dead ends, recognizing promising alternatives, being able to make an effective change of course when it is needed, and so on. such control processes involve what resnick called nuanced judgement. this requires controlling thinking not just on the basis of logic and correctness/incorrectness, but also of hunches, personal inclinations, subjective decisions about probability, intuition, and so on. 16 non-algorithmic thinking and nuanced judgement are facilitated by an open definition of the task, minimal specification of the solution pathway, and the loosest practicable definition of the solution. precision, in as much as it constrains choice, is, in simplistic terms, anti-creative. to enable creative solutions, the engineer would have to tolerate living with uncertainty, ambiguity, openness, and show personal flexibility. unfortunately, such conditions are difficult for both engineers and clients to tolerate, because they expose both partners to risk, personal risk (ridicule, etc.), vocational risk (loss of the job), financial risk, and so on! thus engineers typically dislike ambiguity, at a personal, cognitive level, so that the needs of creative engineering are at odds with the psychological profile of engineers. it is not hard to see why this situation might have arisen. aside from the pressures of those who occupy the space later in the life cycle of systems 16 it is extremely important to note that nuanced judgement does not mean no judgement, or do whatever you please. it involves rigour, self-criticism, search for excellence, etc..., but on the basis of a different way of thinking. ajis vol 13, no. 1 september 2005 168 (the designer, builder etc.), requirements engineers are likely to feel more comfortable reducing the ambiguity of a need by jumping in to constrain the solution. this seems to fit the general psychological profile of engineers. educational implications fortunately, non-algorithmic thinking and nuanced judgement can be taught and learned in university classrooms. the same is true of risk acceptance and tolerance of uncertainty. what is needed is • systematic thematisation of the issues in terms of understandable concepts, • repeated exposure of students to uncertainty, and • assessment based on rewards for dealing productively with ambiguity, taking risks and the like. cropley & cropley (2000) give a specific example of how to do this in an engineering class. encouraging the development of creative traits in students is challenging for teachers. when creative behaviour leads to students being less easy to manage, teachers tend to perceive them less as creative than as a nuisance: teachers’ rating of their favourite students is negatively correlated with creativity (edwards 2001). but the resistance against non-algorithmic thinking goes both ways, and is a long-standing problem, as the following examples illustrates. in 1903 there was an unusually high failure rate among western australian students who took the university of adelaide’s matriculation examination, and this led to an intense public discussion, for instance in the morning herald (1904). the complaint was straightforward: the university had suddenly set assignments such as “to what extent can sir walter scott’s novels be classified as historically accurate?”, whereas what the parents and protesting teachers wanted was “list three examples of historical errors in the novel ivanhoe”. the complainants specifically criticized the fact that it would not be possible to learn the answer to the “to what extent …” example by heart, and that students would have to use their own judgement, with all the risks associated with this, including the possibility of being wrong! protesting parents and educators made it plain that they were not demanding an easy examination, but one that they saw as predictable, objective and fair. as the former anecdote refers to students of the humanities, one can imagine how strong resistance to non-algorithmic thinking is in engineering, in students, teachers and practising engineers alike. where too much algorithmic thinking can lead is demonstrated by an example in scientific rigour on part of a german-latvian microbiologist and pathologist. eugen semmer worked in the institute of veterinary medicine in riga. in 1870 he published a paper reporting on the strange recovery of two horses that were dying from what we would now call “infections”. at the veterinary clinic the horses were accidentally exposed to spores of the mushroom penicillium notatum and inexplicably got better, despite a grave prognosis. semmer correctly identified the mushrooms as the cause of the horses’ recovery, but saw this as a problem that impeded his research on the pathology of disease: how can you conduct microbiological studies in pathology if the patients regain their health? neither semmer himself nor the readers of his paper recognized that he had, in fact, discovered antibiotics, presumably because this had not been part of the project specifications! the world had to wait another 70 years for alexander fleming to discover penicillin. ajis vol 13, no. 1 september 2005 169 the tension between reliable functionality and creative potential is not unique to information systems engineering, but the players involved requirements engineers, information systems designers and information systems users are particularly heterogeneous and to a large degree isolated from each other. in fact the only thing that connects them is technology the information systems they all refer to and therefore the ways they influence each other might easily go unnoticed. the creativity paradox the dimensions of a dilemma emerge: the more engineers do their job as traditionally expected, the less room remains for creativity. this dilemma manifests itself in the understanding of re creativity, the relationship of re with information systems design, and of information systems design and implementation with information systems use. good requirements practice dictates that we should avoid specifying the solution prematurely (i.e. we should start by saying what the system must do, and delay specifying how it should do it for as long as possible). the requirements, in other words, should be solution-free. but effectiveness takes precedence over novelty, and engineers know that eventually someone has to cut metal (or code), which requires clarity, precision and money. thus requirements engineers seek to describe the task precisely, specify the pathway to the solution, and define in advance what will be an acceptable solution. in an effort to facilitate the next phases of the engineering process (design and synthesis), requirements engineers are under pressure to be too helpful to the designers. they are over-applying their own creative skills. too many non-functional requirements (constraints) are introduced in the requirements process, to the point where all effective choice is removed, and subsequent creativity is restricted. the same paradox applies to information systems designers: while trying to empower information systems users with increased functionality, they might actually decrease the potential for holistic use of information systems, and thereby restrict information systems user creativity. while seeking to promote creativity, requirements and their implementation may, because of the justified and necessary emphasis on effectiveness, act as a gatekeeper, holding creativity at bay. the paradox involves requirements engineers, engineers in general, and end users for engineered products, and it effects the creativity of all three groups. thus, we call it the creativity paradox: over-applying the problem solving notion may inhibit requirement engineers creativity, over-applying requirements engineering may inhibit system design creativity, and over-applying engineering may inhibit product user creativity. resolving a paradox typically involves introducing a new frame of reference. in this case, we start by taking a closer look at the notion of problem solving used in information systems engineering. throughout re and se literature, problems in re are often described as ill structured, complex, and domain specific (guindon 1990; batra & davis 1992). the ill structuredness of re problems lies in the ambiguity, vagueness and incompleteness of human wants and needs and the possibility of multiple solutions. the complexity in re manifests itself in different levels of abstraction and varying aspects of requirements (data, process, object, method, etc.) and various formats of representation ranging from informal, semi-formal to formal. in addition, requirements are rather volatile and evolve during the project partly due to our exploration in the problem space, partly due to the dynamics of a business environment formed and reformed by the interactions of the stakeholders. thus, this kind of problems in re makes sense only in a specific domain and often unique context. interestingly, solutions to such ill-structured, complex, and domain specific ajis vol 13, no. 1 september 2005 170 problems are expected to be well defined, precisely expressed and completely specified, so that they will serve as input to systems design and the implementation phase. considering all these properties, is problem solving the right metaphor for describing this activity? on the one hand, coyne (2005) argues that in fact all problems are wicked, or illdefined. on the other, the fundamental usefulness of the problem-solving metaphor is placed in question. robertson for instance suggests that requirements engineers should be creative and even invent requirements based on their understanding of the organisation’s competitive business goals and context. due to subjectivity in problem perception, formulation, judgement and acceptance of suggested solutions, a clear understanding of insight is important. none of the original elements for the classic definition of problem solving can be upheld: neither the problem space is clearly defined or stable, nor are there objective criteria for testing solutions or stating the problem itself. in addition, problem solvers themselves are significantly changed during the problem solving process, by learning and therefore transforming their own perception. the path from a problem to a possible solution in re is created and crafted by requirements engineers when they explore the problem space from subjective descriptions of the problem by different stakeholders. this involves continuously going backwards and forwards between different expressions of needs and wants to develop a problem space as agreed by the problem owner(s) and solver(s). the problem solver plays an active and creative role in shaping and crafting the problem through consulting and helping the problem owner(s) understand and express their needs and possible requirements (gause & weinberg 1989; maiden et al. 2004). this is a learning process during which both problem owner(s) and solver(s) learn about the problem. it is expected that re solutions will make sense in the systems world (checkland & scholes 1990) and will be expressed in the form of a requirements model/specification. the solution (in the systems world) has to mirror the need (in the real world). consequently, the assessment of solutions is rather subjective and contextual. pohl (1994) refers to this as the agreement dimension in re. this dimension is concerned with disparities and inconsistencies between different views of different stakeholders such as systems owners, customers, end-users, managers, requirements engineers, developers, and so forth. it deals with communicative and social activities among these people to support the resolution and integration of different viewpoints (checkland & scholes 1990; goguen 1994; coughlan & macredie 2002). therefore, problems and solutions “exist” in the eye of the beholder(s) and are grounded in their subjective views. world views why use metaphors like problem solving at all, if their fit with engineering activities is so poor? the explanation might be related to underlying world views dominating engineering. world views are the basis of all our models, understanding, and knowledge, and as such also the basis of our scientific approaches. names for world views differ, and world view categorisations are overlapping: there is no objective way of describing all our world views, and, by the very nature of these views, there never will be. but with respect to academic life, acceptance of world views is split into two distinct groups. gould (2000) calls one group realist and the other group relativist. the realist group is also linked to positivist, objectivist, rationalist and other related views, the relativist group is linked to interpretivist, subjectivist and other views. these views are all distinct from each other. ajis vol 13, no. 1 september 2005 171 their emphasis lies on different core beliefs and goals. however, for the purpose of this article, we will loosely group them into the realist and relativist camp. by virtue of their positivist scientific education, and by virtue of rationalist engineering practices, engineers judge their experiences and their own contributions within a positivist/rationalist framework. in the case of the problem solving metaphor, one can identify both positivist and rationalist elements: the notion of problem space can be identified with the positivist view of pre-defined reality, and the understanding of the solving act with the rationalist view of improving or controlling conditions by applying the power of rational intervention. of course engineers are not entirely restricted by their world views. thinking style and world view are related, but different. a relativist world view would appear to support divergent thinking more than a realist world view, by virtue of its inherent acceptance of subjective understanding and context-relative re-interpretation. but in practice adoption of relativist views indirectly motivates developing convergent thinking: transcending individual differences and arriving at shared understanding will be even more important for relativists, since he would have to produce trans-subjective meaning rather than assuming its a priori existence. but eventually, due to fundamental differences in implicit assumptions, alternative world views would lead to completely different models of information systems creation and use. the interpretivist view favoured in the humanities would for instance explicitly stress the subjective knowledge of the human stakeholders involved. these alternative views do find their way into information systems research, but they occupy niches left over by the dominant view, and research results compatible with these alternative views do not lead to a re-interpretation of core realist engineering models. therefore, notions like problem solving are upheld in spite of their deficiencies. is it, then, time for a paradigm shift (kuhn 1962) towards relativist models? unfortunately, replacing the present view with another will not improve our practices, or the power of our understanding. it will only replace one area of weakness with another. the realist views have been adopted in science/engineering for good reason: they are the views best adapted to understanding nature/engineering. replacing them will not improve our situation. merging relativist views with realist elements is also not a real option: world views are self-sufficient, they offer a complete set of beliefs, a framework for understanding any situation. as mentioned above, world views can not be proven wrong. as the literature on re demonstrates, models based in a particular view can always be extended to absorb inconsistent empirical results: extending the rationalist notion of search in a problem space to that of an ill-defined wicked problem space that transforms itself to accommodate subjective perception and relies on problem solver intuition is already a substantial extension, and there is no reason to believe this notion cannot be stretched further. in fact, the expectation of functional creativity placed upon engineers leaves no other choice but upholding notions of problem definition, problem solving and objective functionality. so the predominant views are justified, necessary, and self-sufficient. some alien elements (like stakeholder’s subjective knowledge) might be imported, but only as add-ons to the existing models. merging entirely heterogeneous world views scientifically is not feasible. but as much as these views exclude each other in an ontological sense, we are able to compare and synthesise them as individual observers as we just did. coming back to the initial analysis of creative information systems use, we can detect expectations grounded in different world views: in is behaviour criterion 1, the dominant world view is clearly relativist: multiple views incorporating multiple forms of knowledge representation can be used without restriction, ajis vol 13, no. 1 september 2005 172 implying that the relationship between knowledge and knowledge representation is flexible as well. on the other hand, the computer is expected to somehow enable the switch from one type of knowledge representation to another, implying an objective way of translating between these representations, firmly grounded in realist expectations. a similar relativistrealist contradiction is true for is behaviour criterion 2: relativist freedom to mix and match tasks at will is contrasted with the realist expectation of keeping individual tasks and knowledge interaction consistent. as a mirror image of the above, is behaviour criterion 3 is primarily realist, but with relativist undercurrents: exploring implications of existing knowledge representations and evaluating them implicitly assumes a realist link between knowledge representations and knowledge, but dynamically including heterogeneous knowledge sources or new constraints implies relativist freedom in linking subjects, knowledge, and goals. conclusion we have emphasized in this paper that information systems and creativity involves both engineered products and human players. the differences between the two are so significant as to justify using two different sets of world views to describe them. however the attempt to reconcile these two world views leads to a paradox: innovative thinking is grounded in qualities best understood in relativist terms, but information systems as engineering products are best understood in realist terms. processes can best be understood in terms of relativist transformation of both problem and solution, but results are best understood in realist terms like precision and accuracy. in this paper we have begun to work out the pathway towards resolving the paradox: in order to be of benefit to both information systems engineering and information systems use, our models of creativity will have to be capable of switching from realist to relativist views and back again. unfortunately, this involves a trade-off: information systems cannot be expected to deliver perfectly reliable realist services while, at the same time, leaving maximum relativist freedom. the task is to find a balance. creativity of the developer does not necessarily benefit or inhibit potential creativity of the user. in some situations, flexibility or other information system properties can be expressed in realist terms. in other situations, abandoning realist expectations and leaving relativist choice for later stages is more appropriate. in conceptualising, designing and building a bridge, for instance, what we have called “reliable realist services” may be far more important than relativist considerations. in the case of the sydney opera house, however, experience has shown that a relative lack of exact detail in specifying the task has ultimately produced a building that is almost universally praised, despite the fact that the original requirements were not met. whenever realist expectations are maximised, information systems designers will have to take control of the process, and information systems user creativity will be restricted. where user creativity is maximised, realist expectations will have to be reduced. in order to address the socio-technological complexity of information systems and their use, pluralist research approaches and practices are required. from the point of view of education, this means that information systems designer creativity can be improved by teaching requirements engineers how to leave the door open for relativist freedom in later stages of engineering, and information systems user creativity can be improved by teaching information systems designers how to offer relativist freedom to users. finally, because of the trade-off mentioned above, information systems usability for creative users can also be increased by teaching them to make tough choices in relation to their own expectations and ajis vol 13, no. 1 september 2005 173 their own role in the production process. the design of educational experiences that will achieve these goals (teaching and learning methods, assessment procedures, and so on) now need to be considered. references armarego, j. (2004), “learning requirements engineering within an engineering ethos”, in proceedings of 9th australian workshop on requirements engineering awre’04, adelaide, australia. batra, d. & davis, j. g. (1992), “conceptual data modelling in database design: similarities and differences between expert and novice designers”, international journal manmachine studies, vol. 37:pp. 83–101. beynon, m., ch’en, y.-c., hseu, h.-w., maad, s., rasmequan, s., joe, c., rungrattanaubol, j., russ, s., ward, a., & wong, a. (2001), “the computer as instrument”, in m. beynon, c. l. nehaniv, & k. dautenhahn, eds., cognitive technology: instruments of mind : 4th international conference, ci 2001. biggs, b. j. & moore, p. j. (1993), the process of learning, prentice hall, englewood cliffs (nj). bruner, j. s. (1962), “the conditions of creativity”, in h. gruber, g. terrell, & m. wertheimer, eds., contemporary approaches to cognition, pp. 1–30, atherton, ny, usa. burghardt, m. d. (1995), introduction to the engineering profession, addison wesley, ny, usa. candy, l. & edmonds, e. (1997), “supporting the creative user: a criteria-based approach to interaction design”, design studies, vol. 18(2):pp. 185–194. carroll, j. & swatman, p. a. (1999), opportunism in the requirements engineering process, school of information systems, deakin university. checkland, p. & scholes, j. (1990), soft systems methodology in action, john willey & sons ltd, chichester , uk. committee on information technology and creativity (2004), beyond productivity, national academy press. coughlan, j. & macredie, r. d. (2002), “effective communication in requirements elicitation: a comparison of methodologies”, journal of requirements engineering, vol. 7(2):pp. 47–60. coyne, r. (2005), “wicked problems revisited”, design studies, vol. 26:pp. 5–17. cropley, a. j. (2006), “in praise of convergent thinking”, creativity research journal, vol. 18. cropley, d. h. & cropley, a. j. (2000), “fostering creativity in engineering undergraduates”, high ability studies, vol. 11:pp. 207–219. cropley, d. h. & cropley, a. j. (2005), “engineering creativity: a systems concept of functional creativity”, in j. c. kaufman & j. baer, eds., faces of the muse: how people think, work and act creatively in diverse domains, pp. 169–185, lawrence erlbaum, hillsdale, nj, usa. csikszentmihalyi, m. (1997), creativity: flow and the psychology of discovery and invention, harperperennial, ny, usa. cybulski, j., nguyen, l., thanasankit, t., & lichtenstein, s. (2003), “understanding problem solving in requirements engineering: debating creativity with is practitioners”, in pacis 2003 proceedings of the seventh pacific asia conference on information systems, ajis vol 13, no. 1 september 2005 174 adelaide, australia. dallman, s., nguyen, l., lamp, j., & cybulski, j. (2005), “contextual factors which influence creativity in requirements engineering”, in proceedings of 13th european conference on information systems ecis 2005, regensburg, germany. edmonds, e. & candy, l. (2002), “creativity, art practice, and knowledge”, communications of the acm, vol. 45(10):pp. 91–95. edwards, s. m. (2001), “the technology paradox: efficiency versus creativity”, creativity research journal, vol. 13(2):pp. 221–228. flavell, j. (1979), “metacognition and cognitive monitoring: a new area of cognitivedevelopmental inquiry”, american psychologist, vol. 34(10):pp. 906–911. gause, d. c. & weinberg, g. m. (1989), exploring requirements: quality before design, dorset house publishing, new york, usa. goguen, j. a. (1994), requirements engineering: social and technical issues, chap. requirements engineering as the reconciliation of social and technical issues, pp. 165– 199, academic press, london, uk. gould, s. j. (2000), “deconstructing the ‘science wars’ by reconstructing an old mold”, science magazine, vol. 287(5451):p. 253. guilford, j. p. (1950), “creativity”, american psychologist, vol. 5:pp. 444–454. guindon, r. (1990), “designing the design process: exploiting opportunistic thoughts”, human-computer interaction, vol. 5:pp. 305–344. hadamard, j. (1954), the psychology of invention in the mathematical field, dover publications, new york, usa. hewett, t. t. (2005), “informing the design of computer-based environments to support creativity”, international journal of human-computer studies, in press. higgins, j. m. (1994), 101 creative problem solving techniques: the handbook of new ideas for business, new management publishing company, winter park, fl, usa. horenstein, m. n. (2002), design concepts for engineers, prentice hall, upper saddle river, nj, usa. khushalani, a. j. (1997), modelling and supporting opportunistic design problem solving, ph.d. thesis, school of computer science and software engineering, swinburne university of technology, melbourne, au. kuhn, t. s. (1962), the structure of scientific revolutions, the chicago university press. lawson, b. (1999), ““fake” and “real” creativity using computer aided design: some lessons from herman hertzberger”, in proceedings of creativity & cognition 1999, pp. 174–179. loucopoulos, p. & karakostas, v. (1995), system requirements engineering, mcgraw-hill book company, new york, usa. maiden, n. & gizikis, a. (2001), “where do requirements come from?”, ieee software, vol. 18(5):pp. 10–12. maiden, n., manning, s., robertson, s., & greenwood, j. (2004), “integrating creativity workshops into structured requirements processes”, in proceedings of the 2004 conference on designing interactive systems, acm press. mich, l., anesi, c., & berry, d. m. (2004), “requirements engineering and creativity: an innovative approach based on a model of the pragmatics of communication”, in proceedings of requirements engineering: foundation of software quality refsq’04, riga, latvia. morning herald (1904), “adelaide university examinations”. nguyen, l., carroll, j., & swatman, p. a. (2000), “supporting and monitoring the creativity of is personnel during the requirements engineering process”, in proceedings of 33rd ajis vol 13, no. 1 september 2005 175 hawai’i international conference on system sciences hicss-33. nguyen, l. & swatman, p. a. (2000), “essential and incidental complexity in requirements models”, in proceedings of 4th ieee international conference on requirements engineering icre 2000. nguyen, l. & swatman, p. a. (2003), “managing the requirements engineering process”, requirements engineering, vol. 8(1):pp. 55–68. nguyen, l. & swatman, p. a. (2006), rationale management in software engineering, chap. promoting and supporting requirements engineering creativity, springer. ohlsson, s. (1984), “restructuring revisited: summary and critique of the gestalt theory of problem solving”, scandinavian journal of psychology, vol. 25:pp. 65–78. pennel, l. & maiden, n. a. m. (2003), “creating requirements: techniques and experiences in the policing domain”, in refsq’2003 workshop proceedings, velden, austria. pohl, k. (1994), “three dimensions of requirements engineering: a framework and its application”, information systems, vol. 19(3):pp. 243–258. resnick, l. b. (1987), education and learning to think, national academy press, washington, dc, usa. robertson, j. (2002), “eureka! why analysts should invent requirements”, ieee software, vol. 19(4):pp. 20–22. shanks, g. & simsion, g. (1992), “automated support for creative database design”, in b. srinivasan & j. zeleznikov, eds., research and practical issues in databases, pp. 293–305, world scientific publishing. shneiderman, b. (2002), “creativity support tools”, communications of the acm, vol. 45(10):pp. 116–120. sternberg, r. j. & lubart, t. i. (1992), creative giftedness in children, ablex, norwood, nj, usa. visser, w. (1990), “more or less following a plan during design: opportunistic deviations in specifications”, international journal of man-machine studies, vol. 33:pp. 247–278. voland, g. (2004), engineering by design, pearson education, upper saddle river, n j, usa. wallas, g. (1926), the art of thought, jonathan cape. microsoft word special final.doc ajis special issue december 2004 36 a design approach for tangible user interfaces bernard champoux (associate researcher) dr sriram subramanian (assistant professor) department of computer sciences human computer interaction lab university of saskatchewan saskatoon, canada bjc489@mail.usask.ca sriram@cs.usask.ca abstract this paper proposes a mechanism to design tangible user interface (tui) based on alexander’s (1964) design approach i.e. achieving fitness between the form and its context. adapted to the design of tuis, the fitness-of-use mechanism now takes into consideration the potential conflicts between the hardware of the artifact (electro-mechanical components) and the form of the user’s control (physicalergonomics). the design problem is a search for an effortless co-existence (fitness-of-use) between these two aspects. tangible interface design differs from traditional graphical interface design as unsolved conflicts between hardware and ergonomics can deeply affect the desired interaction. here we propose a mechanism (in the form of eight questions) that support the design by defining the boundaries of the task, orienting the hardware (electro-mechanics) and ergonomics of the design space for various sub-tasks and finally fitting the different components of the hardware and physicalergonomics of the artefact to provide a component level fitness which will delineate the final tangible interfaces. we further evaluate the effectiveness and efficiency of our approach by quantitative user evaluation. keywords tangible user interface, tangible computing, design approach, collaborative design. introduction a research direction in human-computer interaction that has re-surfaced in the last decade is the use of physical real-world artefacts to represent and control digital data. tangible user interface is the popular term used to refer to such computing systems that use physical artefacts as representation and control of digital data (ullmer and ishii, 2000; dourish, 2001). research in tangible user interface has broadly focused on developing systems for various application domains and proposing different frameworks to classify the different systems. systems have been developed to exploit tangible user interfaces for desktop metaphor (e.g., neurosurgical props from hinckley, 1994), virtual reality metaphor (e.g., cubic mouse from frolich and plate, 2001) or mixed reality metaphors. (e.g., datatiles from rekimoto, 2000). frameworks have been proposed based on the type of interaction supported (continuous vs. discrete) (ullmer, 2002) and level of mapping between the physical artefact and digital data (wensveen et al, 2004). however, to make tangible user interfaces a viable real-world interface by truly targeting users’ needs, a mechanism to facilitate the design of tangible interfaces and their prototyping is required. while software engineers have mature software thinking, referring to the functionalities’ structure and software interface in the desktop space, a tangible thinking, referring to the dynamics’ structure in the real-world space, is still in its infancy. purpose of the design mechanism the problem in the development of tuis based on desired user interaction is to maintain, during the process, a balance of the various trade-offs between the appropriate set of interactions required for ajis special issue december 2004 37 the task and their compatibility with the available sensor technology. how to combine hardware and their usability constraints to create a tangible interface? the mechanism proposed in this paper, when used by a team of experts, aims to provide a workflow in this process and would result in a set of clear and coherent decisions, i.e. clear specifications and/or a functional diagram that works coherently with the necessary tasks. structured through eight questions in three phases: defining the boundaries (bo 1-2-3-4), orienting the components (oc 5-6) and fitting the components (fc 7-8), the mechanism will ease the location of problems that occurs during the development, facilitate their identification with the mechanism terminology (boundaries, orienting and/or fitting problem) and help their formulation i.e. asking to the right expert the right question. finally, a completed process will be the starting point to craft a s-type (specifications) prototype to verify the co-efficiency between the required tasks, their interaction and the needed mechanicals components. the design team will then know how well the fitness-of-use has been achieved and will be able to use their answers to the questions as a checklist. we believe this approach to be an excellent solution to propose “tangible thinking” to students and to help experts in managing their projects. figure 1 summarizes the eight questions and the next section presents the fitness-of-use problem. bo 1: what should the user----------- experience? bo 2: what are the human tasks? bo 3: what would the artefact -----------represent and control? bo 4: what are the conventions? • physical ergonomics • electromechanical oc 5: a) what is the nature of the interaction for each ------------sub-task? • continuous • discrete • assembly b) what are the electro-------------------mechanical and physical ---------- ergonomic constraints for -----------this task? oc 6: does the sub-task need any ----------relational interaction? fc 7: what are the relations between the objects and ---------the actions? fc 8: what is the task order when using the-artefact? figure 1: the eight questions origin of the 8 questions designing a computer-enhanced object with a precise task in mind can become quite complicated in the absence of a specific workflow, especially when working in teams. these eight questions arise from literature reading and experiences of designing interactive systems and/or smart objects. people are sometimes very tempted to design a form right away. there is, a priori, nothing wrong with this as long as the designer is aware of the specific electromechanical constraints involved in the ajis special issue december 2004 38 development of the artefact. however, when working with an idea, dealing with a functional diagram is far more easier (and cheaper) than constantly changing the real things. the fitness-of-use problem every design problem begins with an effort to achieve an appropriate fit between two entities: the form in question and its context (alexander, 1964). a form is the desired solution to the problem and the context defines the problem (see figure 2). the appropriate fit between the two entities results in an object with symmetry of use properties: effortless contact between the form and the context. (see figure 3). figure 2: the original fitness problem figure 3: symmetry of use: frictionless co-existence between a form and its context. in tuis, this symmetry of use should occur between the physical artefact (form) that represents and controls information and the application (context and the suitable interaction) that defines the digital information: fitness-of-use (see figure 4). this is also evident from the mcr-pd interaction model proposed by ishii and ullmer (2000), which highlights a bridge between the physical world of atoms and the digital world of bits. figure 4: the fitness-of-use problem adapted to tui design ajis special issue december 2004 39 this fitness-of-use problem is confounded by the potential conflict between the hardware (electromechanical components like sensors and actuators that go into the physical artefact) and ergonomics (how the user will use and control the artefact: the interaction) of the design space. the problem is to find out what sensors fit into the artefact and how to fit them so that the ergonomics of the artefact are not compromised. the user’s ability to control the artefact is dictated by the ergonomics, which in turn are dictated by the interactions required by the task. in contrast, the ability of the artefact to control the digital information is dictated by the electromechanics. thus, the fitnessof-use problem is also transformed into finding a frictionless co-existence of the physical ergonomics of the artefact and its electromechanics. achieving the fitness-of-use in tui design aims to create “a vehicle by which the user acquires/constructs a meaning” of the application (kaptelimin et al., 1981). in order to do so, our mechanism will define the core structure of a physical artefact by addressing the fitness-for-use problem between the appropriate set of interaction needed for the task and their compatibility with the available sensor technology. the potential interactions that occur during the use of the physical artefact highly influence the core structure of the device. any change in the interaction style signifies a change in the core structure. figure 5: the physical artefact’s core structure. figure 6: the physical artefact’s core structure changed because of a different interaction style. in this paper, we investigate the conflicting spaces of tangible interaction to achieve this fitness-ofuse through eight questions in three phases: defining the boundaries (bo 1-2-3-4), orienting the components (oc 5-6) and fitting the components (fc 7-8). the desired fit needs to be defined through the attributes of the interaction, such as information about the tasks, user experience and context (physical and digital) of interaction. the first phase defines the various electromechanical ajis special issue december 2004 40 and physical ergonomic requirements of the artefact. these are then oriented depending on whether continuous or discrete interaction will be supported for each sub-task. this orients the fit between the various electromechanical components and the desired artefact ergonomic. in the third phase, fitness between the electromechanics and the ergonomics of artefacts is achieved at the component level, which is subsequently prototyped for testing and validation purposes. related work brensen (2001), proposed the modality theory to allow developers to decide which input and output modalities to use for interaction. the modality theory provides a comprehensive analysis of all available modalities in the media of graphics, acoustics and haptics and can suggest a set of input and output modalities for a given application. however, the theory does not address the design of the final realization. bellotti et al., (2002), pose five questions that any developer of a sensing systems should address, for improved usability of their ubiquitous or tangible computing system. these questions, which relate to initiating an interaction, specifying the actions and overcoming misunderstandings, are mostly focussed on the final outcome of the interaction and do not deal with specifying the requirements for designing the dynamics of the interaction. scenario-based design (carroll et al., 1998) provides an advanced vision of an application. it creates a set of interaction requirements that can be used to design the application. however, it does not provide a means to factor in electromechanical constraints that might occur when designing tangible user interfaces for the application. the mechanism defining the boundaries one of the fundamental requirements is the ability to layout the parameters of the design space. these parameters specify the boundaries within which the tui solutions are sought. to layout these parameters, we need to define the desired user experience, the resulting human tasks, and the digital information that will be represented and controlled. all of the team members have an opinion on how a task should be accomplished and what are the required interactions. once completed, this phase will enable the team to verify the coherency between the interaction and the chosen technology. user experience the desired user experience is usually the starting point for such a design process. on most occasions this comes from observations of everyday environments and tasks, with a desire to augment the environment with sensors that will modify and enhance the user’s experience in interacting with them. for example, in mediablocks (ullmer et al., 1997), one of the experiences sought was to use real-world gestures to manipulate digital media (multimedia presentation, video etc.) without the need for an explicit computer. human tasks based on user observation we can build a hierarchical task analysis tree, like the one in figure 7, for the application. usually, this tree is based on current practices in performing the task. one of the goals of the tui designer is to incorporate the desired user experience by adapting this tree through subtle changes to certain tasks. while this can be a challenge, it also lays out the boundaries of our design space. with reference to mediablocks, this means creating a task analysis tree of how users ajis special issue december 2004 41 currently interact with digital media and identifying possible tasks that can be modified. ullmer et al., (1997) chose to modify the current practice of capturing, transporting and reviewing pieces of digital media to achieve their desired user experience. the task analysis tree also suggests the possible ergonomic constraints the application might impose on the design. artefact representation and control by analysing the user experience the designer can also identify the different digital information that needs to be represented and controlled by the user. the task analysis tree also specifies details of the digital information that will be controlled by each sub-task. based on this information, the designer can identify the specific sensors and actuators needed for physical control of the digital information. this will also suggest how impractical a physical control might be for certain digital information, thereby laying out the electromechanical constraints of the design space. at this stage, the sensors used are specified at a broad level (i.e., the designer knows s/he needs a force sensor and not a light sensor) without knowledge of the specific details of the sensor (what kind of force sensor, how many newtons etc.) conventions a consequence of analysing the design space is a realization that we make many assumptions about the user expectations and interactions. it is always useful to step out of the design process to reflect on the possible conventions that might be taken for granted. these conventions might impose newer constraints on the ergonomics or electromechanics of the design. for example, in mediablocks, the user is assumed to carry the mediablock. this imposes weight and size restrictions on the block. while in this case the constraint is fairly easy to deduce, it might not always be the case. usually, defining the boundaries of our design space is an iterative process. the user experience desired will inevitably be refined after due consideration to the human tasks and the desired physical control. the following four questions can be used as a general guideline for laying out the boundaries of the design space. 1. what should the user experience? 2. what are the human tasks? 3. what would the artefact represent and control? task sub-tasks from original task analysis sub-task to be modified for desired user-experience figure 7: the task analysis tree ajis special issue december 2004 42 4. what are the conventions? the next step in the design process is to take a closer look at the sub-tasks that need to be modified. these electromechanical and the ergonomic constraints have to be aligned with each other based on the nature of the tangible interaction suited for these sub-tasks. orienting the components for desired fitness-of-use ullmer (2002) has proposed a framework for tangible interfaces that identifies three classifications: interactive surfaces, token+constraint and constructive assemblies. this classification is partly based on the varying degree of support for continuous and discrete forms of interaction. more specifically, interactive surfaces support continuous interaction; token+constraint systems support a combination of (constrained) continuous and discrete interactions, while constructive assemblies almost never support continuous interaction but are instead aimed at discrete interactions. nature of the interaction this part of the mechanism establishes the nature of physical interaction for each sub-task. the user control imposes newer constraints on the specifics of the sensors and actuators required for the electromechanics of the physical artefact. when designing physical artefacts for continuous interaction, the sensors will need to collect data at a higher frame-rate than when dealing with constructive assemblies. similarly, depending on the kind of user action supported, the physical artefact might need to be wireless. this part also provides newer insights into the ergonomics of the final design. for example, when designing for continuous interaction on the table top, the weight of the physical artefact might not be a concern whereas the ability of the artefact to slide on the table-top might be an important physical ergonomic constraint. similarly, when designing physical artefacts with discrete control, the designer might have to provide mechanisms to restrict the user’s movement of the physical artefact to discrete grids. in the case of mediablocks, a token+constraint approach is used to design the interactions. the following two questions may be used as a guideline for orienting the dynamics of the interaction: 5. what is the nature of the interaction for each sub-task? will the sub-task require continuous or discrete interaction? what are the electromechanical requirements for this task and what are the ergonomic constraints for this task? 6. does the sub-task need to have any relational interaction? if two or more physical artefacts are combined to perform the sub-task, will the combination be interpreted in a relational manner: two artefacts together result in adding the corresponding digital information? at this point one has identified the various electromechanical and physical ergonomic components for the tangible interface. the next step to gather all the elements: required tasks and context and to verify how they fit together. this is achieved by integrating the sub-tasks with the rest of the tasks in the task analysis tree with the available technology (questions 7 and 8). fitting the components: integrating the sub-tasks to the task analysis tree a crucial part of the fitness is to integrate the modified sub-task with the rest of the tasks in the task analysis tree of question 2. this integration can be achieved by assessing how the user will initiate and terminate interactions for this sub-task. in general it might be useful for the designer to carefully address the five questions raised by bellotti et al., (2002). these questions not only integrate the subajis special issue december 2004 43 tasks but also provide detailed questions for a better fit between the electro-mechanics of the artefact and the ergonomics. the five questions are • how does the user initiate (and terminate) interaction with the artefact? • how does the artefact inform the user that it realizes that the user has initiated interactions with it? • what are the user actions to which the artefact can react? • how does the artefact inform the user that it has understood the user’s actions and is responding to it? • how can the user recover from a mistake or misunderstanding? task order vs. artefact use order at this point one should have the entire task analysis tree with the modified sub-tasks and the relevant electro-mechanical components identified for these sub tasks. the desired artefact ergonomics have also been identified for the sub tasks providing a component level fit between the electromechanical components and their artefact level ergonomics. in this stage it is important to make sure that the orders in which the various artefact components will be used reflect the order in which the tasks will be executed by the user. the following two questions can be used to provide a component level fitness to the design problem: 7. how should the sub-tasks that use tangible interfaces be integrated with the other tasks in the tasks analysis tree? 8. is the order in which the user is expected to use an artifact similar to the order in which the tasks occur? if the design has been carried out properly then this will be satisfied and so this question acts as a useful parity check. this is the most difficult part of the mechanism because it gathers all the needed elements required to perform the task described in the bo 1-2-3-4 section. the developers can then see what fits and what does not fit. in order to solve some of the problems that occur in this final phase of the development, some expertise from several fields might be required. study and evaluation the next step is to apply the guidelines methodically to an existing system and see where the final design differs from the existing systems. this will give us a handle to evaluate the effectiveness of this approach in solving real-world tangible interface problems. user performance scenario: paint inc is a company that makes tools for drawing. paint inc has recently heard about this new idea of smart/tangible computing and is exited about incorporating these ideas into its product line. they’ve hired you to design a tool for young children to experience, enjoy and learn about different colors by painting in a canvas without the need to mix colors or get their hands dirty with the different colored paints. objective: it’s your goal to design a smart tool which allows young children (aged 4 and up) to experience the different styles and forms of painting without having to mix and match real paint. you have chosen to make the paint brush a smart tool. methodology: the 8 questions below have been developed to help designers develop tangible interfaces. you are expected to use this methodology to guide your design process. ajis special issue december 2004 44 user 1 user 2 user 3 user 4 defining the boundaries bo 1 completed i completed i completed i completed i bo 2 completed i completed i completed i completed i bo 3 completed i completed i completed i completed i bo 4 completed i completed i completed i completed i orienting the components oc 5 completed i half completed i completed i completed i oc 6 completed i not completed i completed i completed i fitting the components fc 7 half completed i not completed i not completed i not completed i fc 8 half completed i not completed i not completed i half completed i table 1: user performance of each phase. observations: it is clear that all the users were able to answer the first four questions. the answers to these questions are very subjective. users 1, 3 and 4 were able to complete questions 5 and 6. these are technical questions that require a knowledge that engineers and computer scientists have. as for user 2, s/he was able to define some of the nature of the interaction for each sub-task but was not able to define any relational interaction that could occur during the course of the task. as the case study was performed in an average time of 2.5 hours, it is not surprising to observe that questions 7 and 8 were not completed (except for user 1 and 4 who half completed the task). again, these questions (especially question 7) required the subjects to relate each component to a specific action needed for the task(s). this is the step where the fitness-of-use is achieved and therefore, constitutes the core of the physical artefact. user 1 provides a short description of this relation without specifying any components. table 2 show the answer to question 7 based on the real i/o brush with user 1 description. ajis special issue december 2004 45 objects (components) action spring-based flexible/bendable touch sensors bouncing like a real brush ccd video camera capture surface light bulbs provide light for the camera 150 optical fiber confirm the capture of a surface drawing canvas (wacom®) surface on which user brush over. dial selecting the “ink mode” user 1 description: the brush is the integration fulcrum. it emulates information from other tangible devices and then transfers information to the software “canvas” component (?) emulate component (?) transfer table 2: i/o brush’s relations between objects and action (fc 7) as for question 8, user 1 and 4 could provide an overview of the task order. since none of them had suggested any components, their results remain impossible to corroborate. table 3 presents their results with the real i/o brush task order. overall, the test shows that a workflow was initiated among the participants. obviously, further investigation has to be undertaken to produce specifications that work coherently with the required task i.e. the scenario provided with the case study and the participants interpretation of it: bo 1,2,3 and 4. achieving the fitness-of-use is in the details on how bo 1-2-3 and 4 will be created mechanically and later on, ergonomically. for example, in the i/o brush, characteristics such as “bendable and flexible” touch sensors to emulate the bouncing of a real brush were added. also, optical fibers were interlaced or blended with the actual brush tips to confirm the capture of a surface in order to provide feedback to the user (like an ok button in gui). in our opinion, the only element missing in the i/o brush to achieve a complete fitness-of-use is the capacity to function wirelessly as recommended by the authors (kimiko et al.2004). in our case study, two users out of four (the computer engineer and the psychologist) had prescribed wireless capacities for their device. i/o brush tasks order: needed components: f pick up the artefact brush f switch the brush on switch on/off f choose the ink mode dial:texture, color, movement f brush over the surface wacom user 1 tasks order: 1clean brush brush 2add paint component (?) 3place paint on palette component (?) 4paint on canvas component (?) user 4 tasks order: ajis special issue december 2004 46 1pick up brush brush 2select print component (?) 3make a mark component (?) 4save/finish picture component (?) table 3: i/o brush, user 1 and 4 task order the use of our mechanism has also pinpointed where problems occur, before any kind of prototyping or form. this feature is advantageous when one considers how costly prototyping can be. here are some problems identified in the case study: user 1: fitting problems: no components are suggested (fc7) and task order is difficult to corroborate realistically (fc-8). user 2: orientation problems: the tasks are defined but not related along with the electromechanical/physical ergonomics constraints (oc 5-6). fitting components problem: as no components were mentioned, providing a task order coherent with the appropriate technology was then difficult (fc 7-8). user 3/4: fitting problems: a clear lack of knowledge for the appropriate technologies (fc 7-8). in a real life situation, (we take into account that user were all by themselves) it would remain possible for user 1,2,3 and 4 to consult an expert, or the pass it on to another member of the team that has the appropriate knowledge to fill the blanks. concerning the effectiveness of the mechanism, user 1 and 4 (engineers) are less satisfied than user 2 and 3. however, regarding the efficiency aspect, user 4 remains unsatisfied while his/her colleague has a different point of view. figure 8: effectiveness ajis special issue december 2004 47 figure 9: efficiency conclusions tui design as a fitness problem in this paper we adapted the original fitness problem proposed by alexander (1964) to the design of tui’s. this approach allows the designer to work simultaneously with the potentially conflicting hardware and physical ergonomic requirements of the tangible interface design. here we reduce the complexity of the hardware and physical ergonomic requirements and finally identify a component level fitness-of-use between the hardware and the physical-ergonomics. this component level fitness is a form of requirement specification that may be used by the designer to identify the final form of the tangible interface for the given application. guidelines for tui design our analysis of the conflicting requirements based on the fitness approach led to eight specific questions that might be used as guidelines for the design of tui’s. figure 1 summarizes our eight questions. the first four questions define the space within which the desired tangible interface is sought. the next two questions orient the tangible interaction based on ullmer’s framework and finally, we layout the details of initiating and terminating the interaction using bellotti’s ideas. it might not always be easy to answer these questions, but in our opinion these questions can lead the designer to think about aspects of the design they may otherwise neglected. in the recent years, a lot of groundwork has been done in the field of tangible computing. we believe it is now time to take it to the user level: instead of finding out the tasks that can be accomplished with a specific technology, we are now entering an era of looking more closely at how the user perform a task and how technology can enhance that task/experience. in addition, there is as well an enormous gap between the idea of a physical artefact and its complete, coherent and functional diagram. physical artefacts that work seamlessly and effortlessly with the intended interaction are not ajis special issue december 2004 48 easy to design. therefore, the use of this mechanism will help to accurately identify the problems and next location in workflow while providing the kind of expertise needed in order to solve them. above all, this mechanism can provide a means to collaborate and communicate tui design between scientists, designers and engineers without hindering their ideas and /or vision. future work in the future we intend to extend the checklist into a set of heuristics and then attempt to chart the cost-benefit tradeoffs at a component level. this chart can then lay the foundations for a predictive model for tangible interface design, in the lines of goms (card et al., 1983) and critical path analysis (baber and mellor, 2001). such models permit early evaluation of designs (prior to building and testing of prototypes) by employing predictive models of user performance to evaluate competing designs. references alexander c. 1964. notes on the synthesis of form, harvard university press, cambridge, massachusetts. baber, c.; mellor, b., (2001), “using critical path analysis to model multimodal human-computer interaction”, in international journal of human computer studies, 2001 54(1), 613-636. bellotti, v., back, m., edwards, w. k., grinter, r. e., henderson, a. and lopes, c. (2002), “making sense of sensing systems: five questions for designers and researchers”, in proceedings of chi'02 conference on human factors in computing systems, 415–422. acm press. bernsen, n. o. (2001), “multimodality in language and speech systems from theory to design support tool”, chapter in granström, b. (ed.): multimodality in language and speech systems. dordrecht: kluwer academic publishers. card, s; thomas m and allen n (1983), “the psychology of human-computer interaction”, hillsdale, new jersey: lawrence erlbaum carroll jm, rosson mb, chin jr george, koenemann jürgen (1998) "requirements development in scenario-based design", in ieee transactions on software engineering, 1156-1170. dourish p. (2001) where the action is: the foundation for embodied interaction, cambridge, mit press. fitzmaurice, g. w., ishii, h., & buxton, w. a. s. (1995) “bricks: laying the foundations for graspable user interfaces”, in proceedings of the chi’95 conference on human factors in computing systems, 442–449. acm press. frohlich, b. and plate, j. (2000), “the cubic mouse: a new device for three-dimensional input”, in proceedings of chi'00 conference on human factors in computing systems, 526 – 531. acm press. gorbet, matthew g, orth m, ishii h. (1998), “triangles: tangible interface for manipulation and exploration of digital information topography”, in the proceedings of chi '98 conference on human factors in computing systems, 1-8. acm press. greenberg, s. and fitchett, c. (2001), “phidgets: easy development of physical interfaces through physical widgets”, proceedings of the uist 2001 14th annual acm symposium on user interface software and technology, 209-218, acm press. hinckley, k; pausch, r; goble, j. and kassell, n. (1994), “passive real-world interface props for neurosurgical visualization”, in proceedings of chi'94 conference on human factors in computing systems, 452-458. acm press. ajis special issue december 2004 49 kaptelimin v., nardi b, macaulay c. (1999), “methods & tools: the activity checklist: a tool for representing the “space” of context”, interaction, july-august 1999, 27-39. quote from the book: leont’ev, a. n. problems of the development of mind. progress, moscow, 1981. kimiko, r., marti s., and ishii h. (2004), “i/o brush: drawing with everyday objects as ink”, proceedings of chi'04 conference on human factors in computing systems, 452-458. acm press. rauterberg, m., fjeld, m., krueger, h., bichsel, m., leonhard, u. and meier, m. (1997), “buildit: a computer vision-based interaction technique for a planning tool”, in proceedings of hci '97, 303-314. springer rekimoto, j., ullmer, b., & oba, h. (2001), “ datatiles: a modular platform for mixed physical and graphical interactions”, in proceedings of the chi’01 conference on human factors in computing systems, 269–276. acm press. ullmer, b., ishii, h., and glas, d (1998). “ mediablocks: physical containers, transports, and controls for online media”, in proceedings of siggraph'98 conference on computer graphics, july 19-24, 1998, 379-386. acm press. ullmer, b., & ishii, h. (1997), “the metadesk: models and prototypes for tangible user interfaces”, in proceedings of the uist 1997 10th annual acm symposium on user interface software and technology, 223–232. acm press. ullmer, b. and ishii, h. (2000), “emerging frameworks for tangible user interfaces”, ibm systems journal, 393(3) 915-931. ullmer, b. (2002), “tangible interfaces for manipulating aggregates of digital information”, unpublished doctoral dissertation, massachusetts institute of technology. wensveen s.a.g., djajadiningrat j.p., overbeeke c.j. (2004), “interaction frogger: a design framework to couple action and function through feedback and feedforward”, in proceedings of dis 2004, in press. acm press. ajis special issue december 2004 50 appendix this section tabulates how some of the existing systems address the fitness problem. we’ve tried to define the fitness desired and orient the nature of the fitness. the last two questions relating to the component level fitness are not discussed here since they relate to the designers idea of the solution and different designers might provide different component level fitness. 1.1 what should the user experienc e? what are the human tasks? 1.2 what would the artefact represent and control? what are the conventi ons? what is the nature of the interactio n for each subtask? does the sub-task need to any relational interaction? 1.3 mediablock s (ullmer et al, 1998) 1.4 -record and carry data without using computer interface. -display data -putting info into a container. -carry the information. -sequence the information. represent: container control: media’ sequence ergonomic: -easy to carry electromechanical: 1.5 connect to several devices. 1.6 media sequences device (discrete interacti on) 1.7 1.8 1.9 sub-tasks performed in combination with: video camera video display printer digital white board. 1.10 triangles module (gorbet et al. 1998) manipulate, connect and combine information chunk. -grasp and manipulate digital information. build 2 and 3d shapes represent: chunk of information to be connected and assembled. control: the arrangement of the information by triggering events in the digital space based on a user manipulation of a physical element. ergonomic: must allow the construction of 2 and 3d shapes. electromechanical: easy to connect and disconnect. must produce the manipulated information in real time the information are connected to each other both physically and digitally (constructive 1.11 inter acti on) the information physical arrangement must be connected to a display. ajis special issue december 2004 51 built-it system (rauterberg et al. 1997) planning a plan layout and simultaneously the result in real time. navigating positioning represent: the object within the space. control: positioning and rotating the object. ergonomic: simultaneou s 2d and 3d view. electro mechanical: displaying positioning 3dof the information is displayed in real time. (continuous interaction) printing 1.12 table 1 copyright mr. bernard champoux/sriram subramanian © 2004. the authors assign to ozchi and educational and non-profit institutions a non-exclusive licence to use this document for personal use and in courses of instruction provided that the article is used in full and this copyright statement is reproduced. the authors also grant a non-exclusive licence to ozchi to publish this document in full in the conference papers and proceedings. those documents may be published on the world wide web, cd-rom, in printed form, and on mirror sites on the world wide web. any other usage is prohibited without the express permission of the authors. microsoft word ajisfinal 12 2 marina.doc ajis volume 12 no. 2 may 2005 50 the effect of gst on farm management information systems and business management skills name(s): dr tony lewis & dr christopher bell affiliation(s) address(es): school of business systems, po box 63b, monash university, clayton vic 3800 telephone: (03) 9905 9692 facsimile: (03) 9905 5159 email: tony.lewis@infotech.monash.edu.au abstract in 1989 new zealand farmers were confronted by the introduction of a gst. despite the short to medium term difficulties, many farmers have benefited from the experience. the introduction of the gst forced many new zealand farmers to improve their record systems as they were required to submit more extensive and accurate information to comply with their new gst requirements. this increase in sophistication of their record systems also meant that farmers had a larger store of more accurate information available to support their farm business management decision-making. it is expected that the introduction of gst and payg reporting requirements in australia is also acting as a catalyst in the evolution of dairy farm record systems and increase in dairy farmer’s store of business management skills. this paper reports the results of a survey that describes the characteristics of dairy farm management information systems and indicates the business management skills that dairy farmers perceive they need to acquire in the short term in order to improve their farm management information systems and comply with their gst requirements. overall, the importance of bookkeeping/ accounting skills is strongly related to bas. however, the results also show that as the level of sophistication of dairy farm record systems grows the demand for business skills shifts from accounting/ bookkeeping skills to computer and analytical skills. keywords: information systems, gst, business management skills. introduction tax reform, culminating with the introduction of goods and services tax (gst), pay as you go (payg) and other new compliance requirements from the 1 st july 2000 has forced many dairy farmers to critically review their farm record systems and their own store of business management skills. in 1989 new zealand farmers were confronted by new reporting requirements bought about by the lange government’s introduction of a gst. despite the short to medium term difficulties, many farmers have benefited from the experience. the general experience of new zealand farmers with their gst was typified in a report screened on the abc’s landline program (fullerton, 1999). the farmer interviewed stated that following improvements made in his accounting system to comply with the gst, he received an unexpected benefit – better quality information. this farmer now has an ongoing and increasing demand for quality information and has planned to make ongoing improvements in his farm record system in order to provide it. this paper reports the results of a survey that describes the characteristics of dairy farm management information systems and indicates the business management skills that dairy farmers perceive they need to acquire in the short term. ajis volume 12 no. 2 may 2005 51 background the ability of a farm management information system (fmis) to provide quality information is inhibited by the inadequacy in scope and structure of the underlying accounting system. although many farm accounting systems are based on “commercial” manual recording systems or computer software cashbook/ accounting packages, they do not inherently generate accurate, relevant and consistent information. initially, a farm business manager must design a “chart of accounts” that adequately reflects the true nature their own specific farm business. they must know how to properly identify, classify and record the various types of transactions related to their farm business entity. the farm business manager must also possess an adequate store of analytical skills in order to maximise the potential value of information generated by the farm accounting system. a farm business manager must therefore possess relatively high levels of business management knowledge and skills, especially bookkeeping/ accounting and analytical skills, in order for their farm businesses to realise the maximum benefit offered by a fmis. from a business management perspective, farm businesses are quite similar to other business types that have an underlying family social structure. these other business types commonly use the same popular cashbook or accounting packages used by farm businesses. for example, the common cashbook computer package quicken is used by a variety of small business types including farmers. quicken was promoted among farmer groups several years ago, and it is the most popular cashbook package used by victorian farmers (lewis, 1997). however, some serious inadequacies have been noted in client designed quicken reports forwarded by a variety of small businesses to their accountant (frank, 2001): • lack of appropriate account structures • transactions not correctly classified • missing or duplicated transactions • personal/ family transactions included with business transactions • basic accounting rules broken, i.e. debits recorded as credits • report structures that do not conform to accepted accounting standards one melbourne accountant (frank, 2001) reported that some of his clients using quicken had requested him to reduce his fee for completing their tax returns. they stated that they expected a reduction as they were now doing most of his work for him. in fact, the inadequacies in the quicken reports rendered them useless for tax compliance purposes, and the accountant was forced to reconstruct accurate accounts before he could complete their tax returns. there is a procedure in the accounting profession called “shoebox” accounting. clients place bank statements, cheque butts, invoices, receipts and the like, into a literal shoebox. every year this shoebox would be upended on their accountant’s desk. the accountant’s task was to sort out the mess, construct accurate accounts and prepare the tax return. many clients use of the quicken program results in nothing more that an “electronic shoebox.” the implication is that if output from these packages is inadequate for tax compliance purposes, then it is most probably inadequate for business analysis and business decision-making tasks because of a lack of scope and accuracy. in the past, compliance reporting such as income tax was mainly the realm of accountants (lewis, 1997). farm business taxation information was commonly reconstructed from bank statements and other documents. therefore, there was little incentive for farm business proprietors to keep complete and accurate records in respect to their compliance responsibilities. while it is true that a substantial number of dairy farmers maintained farm record systems that provided specific farm production or financial information, many lacked the scope and structure that would provide accurate and reliable information suitable for other business decision-making and reporting tasks. again, there was little incentive for farm business managers to increase the scope and sophistication ajis volume 12 no. 2 may 2005 52 of their farm record systems, and the potential for these systems to enhance business outcomes was restricted accordingly. based on the new zealand experience, it is anticipated that gst and payg reporting requirements in the business activity statement (bas) will actually encourage farm business managers to maintain more sophisticated records (fullerton, 1999). a substantial increase in annual accounting fees is a keen incentive for farm business managers to establish and maintain their own improved farm record systems (paterson, 2000), and there are severe penalties for inaccurate figures on the bas. many farm business managers may perceive that they currently lack specific skills needed to make improvements in their farm record systems. therefore, gst and payg reporting requirements are also encouraging dairy farmers to review their personal store of business management skills. the range of business management skills to be examined in the current study will fall into three basic groups. these are: bookkeeping and accounting skills, computer skills and analytical skills. it should be noted that computer skills are inclusive of internet skills. it is also important to note that farm business managers will be asked to indicate the skills they believe they currently lack in order to successfully and efficiently complete their bas without outside (the farm business) assistance. so, these will be the skills they perceive they need, which may or may not be the same as the skills they actually need. details of the survey the survey sample was based on a random selection of 343 dairy farmer members of dairy food safety victoria. this sampling frame included the majority of registered dairy farmers operating in victoria. questionnaire forms were mailed to the selected dairy farmers in january 2002, from which 85 usable replies were received. this represents a response rate of approximately 25%. questionnaires were addressed to the “prime decision maker” within each dairy farm business. this person is conceptualised as the one individual who has the primary task of making most business decisions, or where joint decision-making is common, the individual who normally has the casting vote. in an attempt to minimise the risk of bias in the results, the role of prime decision maker was not presumed to be either task or gender related. in order to provide a context for the study, and to explore any relationship between skills and the dairy farming environment, several variables are used to generally describe the dairy farms in the study population. these include size of farm, number of years operated as a dairy farm business, minimum and maximum cows and maximum beasts maintained in the year preceding the questionnaire date, the type of operation i.e. whether a family business or not, and the number of people operating the dairy farm business. also described are the accounting and production records maintained by individuals in the farm businesses or “outsiders” such as accountants and bookkeepers. details include the decision-making purposes to which information from these records is put. a number of personal demographics related to prime decision makers and their spouse are used to round out the description. the model used to describe farm businesses is the farm management information system (fmis) model (lewis, 1998) (see fig. 1). this model segments the study population according to the relative level of complexity and sophistication of their fmis. a fmis is defined as either accounting or production records maintained by individuals on the farm that provide information that is used to support farm management decision-making. there are four levels of complexity of fmis. the first is no fmis – where neither accounting nor production records provide information to support decision-making. the second is manual fmis – where accounting and/ or production records are maintained manually, without the aid of a computer system. the third is computerajis volume 12 no. 2 may 2005 53 supported fmis – where some or all accounting or production records are maintained on a computer system, and the forth is use of the internet – where a computer system is used to gain access to the internet to perform one of several tasks that include sending and / or receiving email, location of sources of information, and the purchase goods and services. this model depicts increasing sophistication in the underlying accounting or production record systems. movement through the various levels is relatively contiguous, although there are exceptions. these exceptions are in the minority and do not form a significant group. results of the survey the study population (those respondents with usable replies) are domiciled in all major dairy areas in victoria, with 94% living on their farms. average size of dairy farms is 203 hectares (minimum 25, maximum 1100) carrying an average minimum of 94.6 cows and an average maximum of 236.6 cows within an annual cycle. prime decision makers are generally well experienced in dairy farming with an average involvement of 22.4 years. however, not all respondents see a future in dairy with 34.5% stating that given the opportunity they would seek some other activity outside the dairy sector. a majority, 98.8% of prime decision makers are involved in a family type business. included in the definition of “family business” are sole operators, single-family operations, partnerships involving relatives and companies involving family members. there was only one non-family business respondent involved in a partnership of non-relatives. farms are operated either by family members or a combination of family members and hired hands. average age of prime decision makers is 53.2 years, with a minimum of 29 years and maximum of 79 years. this statistic is consistent with other farmer groups, and indicates that, as a working group, people involved in dairy farming tend to be older than the average general working population. prime decision makers tend to be male (78.6%), but a significant proportion (21.4%) are in fact female. many past studies presumed the respondent farmer to be male and the prime decision maker. that is not to say that the current respondent females are the primary farmer – this presumption was carefully avoided in the construction of the questionnaire. but, is true to say that they are the prime decision makers within their own farm businesses. ajis volume 12 no. 2 may 2005 54 the majority of prime decision makers (61.9%) have a secondary education, but trade/ certificate (13.1%) and diploma courses (15.5%) are also significant in the results. the interpretation is that while formal eduction remains at about the secondary level, more vocationally oriented postsecondary education, such as that available at tafe and agricultural colleges is becoming more common. farm record systems can be composed of a range of various records and reports. each record system will be somewhat unique, but there is a commonly observable set of record and report types, a number of which can be found in all systems. table 1 details frequency and percentage results for each accounting or production record or report type. table 1 accounting records record type ourselves outsiders no. % no. % cash book 59 69.4 5 5.9 reconciliation 52 61.2 9 10.6 other important ledger a/cs 35 41.2 1 2.8 a general ledger 32 37.6 6 7.1 budgets 42 49.4 14 16.5 performance report 26 30.6 10 11.8 profit & loss statements 28 32.9 34 40.0 balance sheets 25 29.5 33 38.8 tax return 11 12.9 67 78.8 production records record type ourselves outsiders no. % no. % livestock records 78 91.8 4 4.7 cropping or paddock management 68 80.0 1 1.2 n = 85 prime decision makers were asked to indicate if a particular record or report type was maintained by individuals within the farm family or by hired hands working on the farm (“ourselves”), or by “outsiders” such as accountants and bookkeepers. if a particular record or report type was not maintained, then a missing value was recorded in the results. results in table 1 detail the number and percentage of farm businesses with a particular record or report type maintained either by “ourselves” or “outsiders.” table 1 shows that the simpler forms of accounting records are maintained primarily by “ourselves” while the more complex record type has an increasing proportion maintained by “outsiders.” the majority of tax returns were prepared by “outsiders” mainly accountants. in general, accounting records are maintained by individuals within the farm business. of course, it is possible that for a specific farm business some records may be maintained by “ourselves” and others, particularly tax returns, are prepared by “outsiders.” ajis volume 12 no. 2 may 2005 55 there are two types of production records detailed in table 1. livestock records cover such things as details about individual animals, feed rations, breeding details or quantity or quality of production. cropping information or paddock management includes records recording information such as cropping history, seed sown, fertiliser, herbicide or insecticide use. many dairy farmers raise fodder crops and most have sown pasture in their paddocks. the majority of production records are maintained by individuals within the farm business. in order for a farm management information system to be deemed to exist, information from a farm record system must be used to support farm business decision-making. prime decision makers were also requested to indicate how information is used. for accounting records, four uses for information were detailed: (1) to show how much we have spent, or how much we have to spend, (2) to show how the farm business has run in the past so we can decide how to run it in the future, (3) to provide information to help us make big decisions, like purchase of equipment or whether to buy the farm next door, and (4) to provide information to help us decide what to do when problems come like drought, sickness, low prices and so on. for production records, four uses were also detailed: (1) to provide information to help decide which animal to breed, or which to cull, (2) to help decide which animals to market, (3) to provide information to help decide what to grow or when to grow it, and (4) to decide how much seed, fertiliser, herbicide or insecticide to use. results indicate that accounting and production records commonly do provide information to support farm business decision-making. the strong results for “how much spent, and how much we have to spend,”(78.8%) and “to help decide which animals to breed, which to cull” (82.2%) appear to reflect the more universal nature of such decisions. results for the model of farm management information systems are detailed in table 2. results show that the majority of farm businesses own a computer system, with the majority of computer owners having access to the internet. the result of 48.2% for “use of the internet:” was larger than expected, but not surprising owning to the fact that the majority of new computers now ship with modems, and connection to the internet via a telephone line is a relatively simple procedure. it is interesting to note that only 7.9% of computer owners purchased a computer to gain access to the internet – the majority 92.1% owned and operated a computer system before internet connection. this result tends to support the notion of contiguous movement along the model, that is, the notion of gradual increase in sophistication of farm management information systems. table 2 level of sophistication of farm management information systems level of sophistication frequency % no fmis 1 1.2 manual fmis 26 30.6 computer-supported fmis 17 20.0 use of internet 41 48.2 n = 85 prime decision makers were asked to indicate which of four skills that they believe they immediately need to improve in order to more easily complete their business activity statement (bas). these four skills are defined as follows: (1) accounting/ bookkeeping skills – e.g. keep cash book better, expand your farm records, improve budgets, produce more accurate profit and loss statements, ajis volume 12 no. 2 may 2005 56 produce more details balance sheets etc., (2) computer skills – e.g. learn new software packages, use existing software better and so forth, (3) internet skills – e.g. learn to find more information on gst and bas, and similar activities, and (4) analytical skills – e.g. how to perform more advanced calculations using the information you already have available. prime decision maker’s responses are detailed in table 3. table 3 management skills management skill frequency % accounting/ bookkeeping 41 54.7 computer 19 25.3 internet 2 2.7 analytical 13 17.3 n = 85 system missing = 10 the raw results in table 3 suggest that less technically oriented skills of accounting/ bookkeeping are the most needed for completion of bas. however, results are in part related to the level of complexity of farm management information systems. table 4 repeats the results for table 3, but broken down by the level of complexity of farm management information systems. table 4 management skill by level of sophistication of farm management information system no fmis management skill frequency % accounting/ bookkeeping 1 100.0 manual fmis management skill frequency % accounting/ bookkeeping 21 95.5 analytical 1 4.5 system missing = 4 computer-supported fmis management skill frequency % accounting/ bookkeeping 9 60.0 computer 2 13.3 internet 1 6.7 ajis volume 12 no. 2 may 2005 57 analytical 3 20.0 system missing = 2 use of internet management skill frequency % accounting/ bookkeeping 10 27.0 computer 17 45.9 internet 1 2.7 analytical 9 24.3 system missing = 4 n = 85 despite there being only one observation for “no fmis” results show that accounting/ bookkeeping skills are the most important to operators of non-computer supported fmis, which tapers off as the level of sophistication of fmis grows. conversely, the importance of computer and analytical skills grow as the level of sophistication increases. the interpretation given to these results is that individuals within farm businesses that have relatively sophisticated farm management information systems have, in the main, acquired an adequate level of accounting/ bookkeeping skills, and other, more technically oriented skills become relatively more important. the most important set of skills for the completion of bas appears to be accounting/ bookkeeping skills. the ability for farm businesses to complete their bas from their own resources does appear to be related to the level of complexity of their farm management information systems. summary and implication the data suggests that the ability of victorian dairy farm businesses to complete their bas form from their own resources appears to be related to the level of complexity of their farm management information systems. in particular, accounting/ bookkeeping skills appear strongly related to bas completion. the implication of the results relates to farm business decision-making – if it is accepted that larger and more sophisticated farm management information systems are better able to support farm business decision-making, then the introduction of bas requirements is acting as a catalyst in improving farm management outcomes. results also highlight the importance of accounting/ bookkeeping skills. if the aim is to help increase the level of sophistication of farm management information systems, particularly the adoption of farm computer systems, then the transfer of accounting / bookkeeping skills should be made a priority for individuals in farm businesses that operate less sophisticated farm management information systems. for operators of the more sophisticated farm management information systems the transfer of computer and analytical skills should be made a priority. references fullerton, ticky (1999) “gst new zealand” landline abc tv june 6 th frank, graeme (2001) interview, melbourne may 9 th ajis volume 12 no. 2 may 2005 58 lewis, tony (1997) “factors influencing the adoption of computer-based management information and decision-support systems by australian farmers” ph.d thesis swinburne university of technology, victoria, australia lewis, tony (1998) “evolution of farm management information systems” computers and electronics in agriculture, vol 19, pp 233-248 paterson, jane (2000) “nff head has faith in gst education” landline abc tv march 26 th ajis vol 9 no. 1 september 2001 customers' beliefs behind business-to-consumer electronic commerce nenalim commerce department university of queensland brisbane, australia email: nlim@commerce.uq.edu.au abstract with the rising importance of electronic commerce, it is important for researchers and company executives to understand customers' beliefs behind online shopping behaviour. this proposed study extends the technology acceptance model (tam) to test the influence of experience, self-efficacy, perceived risk, and social influence on customers' acceptance of the business-to-consumer (b2c) e-commerce. keywords electronic commerce, technology acceptance model, system use, adoption, perceived risk, experience, user acceptance, online buying behaviour introduction based on the technology acceptance model (tam), the present study examines the following research question: in what ways do experience (davis et al., 1989; karahanna et al., 1999; taylor & todd, 1995), self-efficacy (bandura 1977; 1982), perceived risk (cox & rich, 1964; jarvenpaa & todd, 1996), and social influence (salancik & pfeffer, 1978; fulk et al., 1990) affect customers' acceptance of the business-to-consumer electronic commerce? acceptance of business-to-consumer e-commerce refers to when a customer prefers to buy online instead of buying through other marketing channels, such as store-based retailing and telephone shopping. according to the electronic commerce association, electronic commerce (e-commerce) is "any form of business or administrative transaction or information exchange that is executed using any information and communications technology" (till, 1998, p. 9). with the rising importance of e-commerce, some people expect it to bring sociodemographic changes as well as business advantages. for business people, e-commerce is a unique two-way communication approach to a direct marketing channel (jarvenpaa & todd, 1996). drucker (1999) suggests that e-commerce might even lead to an information revolution. in monetary terms, beer (2000) estimates that australian businesses will spend over a $2.64 billion on web technology in 2000. activmeida (1999) reports that us $95 billion revenues were generated from global ecommerce in 1999. revenue is expected to exceed us $220 billion in 2000 (activmedia, 1999), and reach us $6.9 trillion by 2004 (nua, 2000). a common approach is to classify e-commerce in two ways: business-to-business (b2b) and business-toconsumer (b2c). b2b e-commerce refers to paperless exchange of business information over the internet. it involves activities such as electronic data interchange (edi) and electronic funds transfer (eft). b2c ecommerce involves companies selling goods or services to customers online. this study focuses on b2c ecommerce. it uses the terms b2c e-commerce and online shopping interchangeably. business-to-consumer (b2c) electronic commerce b2c e-commerce gets much publicity through companies, such as yahoo.com and amazon.com (mis australia, 2000). studies show that customers purchase online because it is convenient, without sales pressure, and saves time (gvu, 1999; henderson et al., 1998). a spate of studies by various consulting companies indicate that online shopping has a huge, relatively untapped market. according to a boston consulting group report on b2c e-commerce, north-american retailers generated us $33.1 billion revenues in 1999. the 2000 revenue figure is expected to jump to us $61.1 billion (boston consulting group, 2000). european retailers obtained us $2.8 billion from online sales in 1999 (conlin, 2000). they are expected to generate us $8.9 billion by the end of year 2000 (boston consulting group, 2000). regarding australia, horey (1999) estimated that the online revenue was a $920m in 1999. the number of australian online shoppers is expected to reach 3.8 million by the end of 2000 (consult pty, 2000). although increasing results seem to suggest that online shopping is growing rapidly, predictions on b2c business could be optimistic (switzer, 2000). for example, the 1999 north-american online revenue figure (us $33.1 billion) is still relatively small, because it represents only 1.4 percent of all retail sales (boston consulting group, 2000). charlton et al. (1999) found that the general public in the united kingdom exhibits little interest in e-commerce. moreover, customers often browse items online but with no intention to buy (gvu, 1999). in their study on gender and internet shopping, fram & grady (1997) suggest that women only have moderate 70 ajis vol 9 no. 1 september 2001 interest in online shopping. in addition, 65 to 75 percent of online shoppers in two different studies abandoned their purchases before the purchases were completed (boston consulting group, 2000; vu et al., 1999). in short, on the one hand, growing studies (e.g., activmedia, 1999; boston consulting group, 2000; nua ltd., 2000) suggest that b2c business will grow substantially. on the other hand, evidence shows that many customers may still not want to purchase online (boston consulting group, 2000; charlton et al., 1999; gvu, 1999; vu et al., 1999). because of the increasing financial investment by organisations (beer, 2000) and potentially large profits from b2c e-commerce (activmedia, 1999; horey, 1999; boston consulting group, 2000), it is important for researchers and company executives to understand more about online shopping behaviour. with the objective to enhance the acceptance of online shopping, this study examines the beliefs behind customers' behaviour. technology acceptance model (tam) evaluation of the success of information technology or information systems has been an important topic for researchers and practitioners (delone & mclean, 1992). among the various evaluation criteria proposed by researchers, user acceptance of a new technology or system is considered to be a vital and useful criterion. based on the theory of reasoned action (fishbein & ajzen, 1975; ajzen & fishbein, 1980), the technology acceptance model (tam) focuses on two perceptions, perceived usefulness and perceived ease of use, to predict users' acceptance of new technology (davis, 1989; davis et al., 1989). because the tam is parsimonious and well developed (davis & venkatesh, 1996), many researchers have adopted this model to assess users' acceptance of different technologies, such as microcomputers (igbaria et al., 1995), word processing systems (adams et al., 1992; davis & venkatesh, 1996), spreadsheet systems (adams et al., 1992; davis & venkatesh, 1996; hendrickson et al., 1993; mathieson, 1991), and database systems (hendrickson et al., 1993). over the years, many researchers have added constructs to the tam to try to enhance its predictive power. some studies elaborated external variables that affect perceived ease of use and perceived usefulness in tam to better understand the motivation behind users' behaviour. for example, gefen & straub (1997) studied the impact of gender on usage of email. gefen & keil (1998) focused on the effect of perceived developer responsiveness on perceived ease of use and perceived usefulness. karahanna & straub (1999) examined the effect of four types of external variables, namely, social presence, social influence, availability of user training and support, and perceived accessibility, on an email system. studies on the world wide web/electronic commerce literature indicates that many existing studies on the world wide web (www/web) or e-commerce pertain to human-computer interaction (hci). these hci studies focus on the physical features, such as web interface design and the navigation pattern of surfers (e.g., grose et al., 1998; mcdonald & stevenson, 1998). results of these hci studies give information, such as how to distinguish a good web page from a bad one. unfortunately, they do not tell company executives why customers buy or do not buy online. instead of focusing on hci issues, some studies examine the acceptance of the web technology by focusing on the beliefs and motivations of customers (jarvenpaa & todd, 1996; agarwal & prasad, 1997; morris & dillon, 1997; fram & grady, 1997; fenech, 1998; agarwal & prasad, 1998; parthasarathy & bhattacherjee, 1998). most web-related studies adopted the tam or the innovation diffusion theory as the basic research model. innovation diffusion theory (rogers, 1983; moore & benbasat, 1991) is similar to the tam. this theory postulates that researchers can predict innovation diffusion by examining users' belief over various characteristics of the innovation. the eight perceived characteristics of innovations reported by moore & benbasat (1991) are voluntariness, relative advantage, image, compatibility, complexity, trialability, visibility, and result demonstrability. table 1 is a summary of these web-related prior studies. 71 ajis vol 9 no. 1 september 2001 year 1996 1997 1997 1997 1998 1998 1998 1998 1999 author ! , jarvenpaa & todd agarwal & prasad fram & grady morris & dillon agarwal & prasad fenech henderson et al. parthasarathy & bhattacherjee korgaonkar & wolin focus of study electronic shopping on www www for learning cyber shopping pattern information searching using netscape www information services on the internet www in general electronic supermarket online services www in general methodology open-ended survey; focus group experiment field study (survey on the web) field study (survey) field study (survey) survey experiment (followed by focus groups); survey field survey focus group; survey sample 220 shoppers 73 mba students 254 women 76 students 175 part-time mba students 150 students 57 families; 64 users 145 subscribers 6 focus groups; 401 consumers theory + additional variables nil innovation diffusion theory + perceived voluntariness nil technology acceptance model (tam) tam + personal innovativeness tam + self-efficacy tam + enjoyment + peergroup norms + usability + experience + individualsituational factors innovation diffusion theory + external influence + interpersonal influence + network externality media uses and gratification theory table 1 summary of studies on acceptance of the web/electronic commerce problems of existing web-related studies a few problems in the following aspects can be observed from table 1: • focus of study • theory testing • modification of the technology acceptance model first, except for henderson et al. (1998), all the web-related studies in table 1 examine users' overall impression of the internet or the web. unfortunately, because of the broad nature of the internet and the web, results of these studies are not especially useful for practitioners. second, some studies in table 1 are open-ended survey type studies. such studies are exploratory in nature, and do not contain any theory or research model (fram & grady, 1997; jarvenpaa & todd, 1996). for instance, although jarvenpaa & todd (1996) classify customers' responses into four categories based on studies in storebased retailing and direct marketing, their results simply represent a list of concerns of the customers regarding online shopping. their study suggests that experience, and perceived risk may play important roles in predicting web acceptance. third, for those web-related studies that specified research models, most applied the simple tam approach or innovation diffusion theory (e.g., agarwal & prasad, 1997; morris & dillon, 1997). nonetheless, because of the difference between ordinary information systems and e-commerce, a simple tam or innovation diffusion approach is unlikely to predict acceptance accurately. for instance, the scope of e-commerce is much wider than ordinary information systems as it extends across numerous people and organisations. moreover, e-commerce carries financial consequences for the users. therefore, from the customers' or users' viewpoints, risk becomes a vital factor. as more than 50 percent of customers abandon purchases before purchases are completed (boston consulting group, 2000; vu et al., 1999), it is an interesting question to find out whether self-efficacy of customers contributes to this behaviour. to tackle the problems of the existing studies, this study adds potentially important constructs to the tam, and examines the effect of these constructs. moreover, this study focuses on certain types of goods and services in online shopping. fram & grady (1997) found that women often purchase software, books, and music online. 72 ajis vol 9 no. 1 september 2001 across different countries, gvu (1999) also reported that hardware, software, books, music, and travel are the five most common items that customers intend to buy when they browse the web. to enhance the predictive power of the proposed model, this research focuses only on online customers' purchasing behaviour in relation to these five items. this study does not distinguish between multichannel retailers, such as airlines, and online-only retailers, such as e-store.com.au and dstore.com.au. conceptual model this research examines the factors that motivate acceptance of online shopping from the customers' viewpoint. figure 1 describes the research model for this study. both agarwal & prasad (1997) and agarwal & prasad (1998) reported an adjusted coefficient of determination that is below 0.5. it is expected that the constructs proposed in the model will be able to explain a larger proportion of variance in online shopping behaviour than existing studies. to focus on the subjective perceptions from customers' viewpoints, however, this study does not consider objective factors that may affect online shopping, such as speed of transmission lines, quality of products, and prices of products. the following sections describe the potentially important constructs and the proposed hypotheses. h2 online shopping behaviour figure 1 b2c e-commerce adoption model experience previous studies on tam have shown that experience potentially influences users' acceptance of new technology (adams et al., 1992; chau & hui, 1998; davis, et al., 1989; igbaria et al., 1995; karahanna et al., 1999; szajna, 1996; taylor & todd, 1995; thompson et al., 1994). ratnasingham (1999) found that companies are more secure when they trade with existing edi partners. in a study regarding telephone shopping, cox & rich (1964) also found that prior shopping experience can reduce uncertainty. thus, it is likely that customers will feel more comfortable buying online if they have good online buying experience. experience was reported as vital in two of the studies in table 1 (henderson et al., 1998; jarvenpaa & todd, 1996). this study defines experience as online shopping experience. to make the results more meaningful, this study focuses only on prior experience with b2c e-commerce. it is not concerned with general computer experience. this study measures experience in three ways: (a) the number of times a customer tried to buy goods or services online; (b) the number of times a customer bought goods or services online; and (c) the number of times a customer felt satisfied with completed online shopping. it is expected that, with more practice, a user will learn new skills and possibly increase his or her self-efficacy. compeau & higgins (1995a) argue that previous experience of technology helps to enhance users' self-efficacy in using the technology. on the other hand, venkatesh & davis (1996) report that self-efficacy does not change over time with experience. in view of this contradictory result, the present study does not specify the direction of the association: • hi a: experience is associated with self-efficacy. with prior online shopping experience, this study postulates that perceived risk of customers over b2c ecommerce will be reduced. referring to edi, ratnasingham (1999) indicates that organisations get a higher level of security if they have previous experience with the business parties. similarly, if customers buy from the same 73 ajis vol 9 no. 1 september 2001 vendors, perceived risk will be reduced. • hlb: experience is negatively associated with perceived risk. karahanna et al. (1999) and thompson et al. (1994) found that social influence had a greater impact on inexperienced users than experienced users. with experience, users may be better able to form their opinions on the new technology instead of relying on or being affected by the comments of other people. thus, the following hypothesis is proposed: • hlc: experience is negatively associated with social influence. hubona & geitz (1997) conclude that external variables do not always affect usage behaviour through mediating belief constructs. their results show that the direct linkage between experience and usage is significant. moreover, thompson et al. (1994) report that experience has a direct effect on the behaviour of personal computer usage. the results also indicate that the indirect effect of experience through users' beliefs is much smaller than the direct effect. therefore, this study hypothesises that: • hid: experience is positively and directly associated with online shopping behaviour. self-efficacy the concept of self-efficacy originated from bandura's social cognitive theory (bandura, 1977; 1982). selfefficacy is about perception of ability. it refers to how confident users are in doing certain tasks, such as using new technologies. in this study, self-efficacy is defined as an individual's perceived ability to buy goods or services on the web. pooling data on different technologies from different organisations, compeau & higgins (1995b) and compeau et al. (1999) found that computer self-efficacy has an important impact on users' affect, anxiety, and usage. moreover, compeau & higgins (1995a) reported that self-efficacy affects computing performance, whereas davis et al. (1989) and compeau et al. (1999) showed that self-efficacy perceptions affect the perceived ease of use of users. in spite of the importance of self-efficacy, only fenech (1998) has examined the impact of self-efficacy in the studies shown in table 1. fenech (1998) reported that the predictability of tam was greatly improved by the addition of computer self-efficacy because adjusted goodness-of-fit index jumped from 0.556 to 0.89. venkatesh & davis (1996) suggest that users with a higher level self-efficacy tend to perceive new technology as easier to use. users with high self-efficacy are more willing to learn a new technology. therefore this study hypothesises that: • h2: self-efficacy is positively associated with perceived ease of use. perceived risk researchers in psychology have widely studied risk theory, especially in relation to gambling behaviour (e.g., pollatsek & tversky, 1970; jia et al., 1999). traditionally, developers of information systems do not pay much attention to security or risk (straub & welke, 1998). security and risk issues have always been described as one of the critical factors, however, for e-commerce success (arnum, 1995; fram & grady, 1997; rebel & koenig, 1999; till, 1998). for instance, fram & grady (1997) report that online customers are concerned about credit card fraud. they normally purchase only products with low purchasing risk. the fragility of the web and e-commerce was evidenced in february 2000. in that incident, traffic to websites such as yahoo, amazon, cnn, and ebay were jammed with junk messages (richtel & robinson, 2000). a telephone survey conducted after the incident indicates that 70 percent of online shoppers were worried about the attack thirty percent of the respondents said that they would be less likely to buy online (schwartz, 2000). new technologies and protocols, such as digital signatures, encryption (arnum, 1995), and wireless application protocol (zampetakis, 2000), are used to improve internet security. moreover, online companies obtain a webtrust seal of assurance to enhance customers' confidence in online shopping. a webtrust seal indicates that a company has fulfilled certain security principles and criteria as specified by professional accounting bodies (icaa, 1999). in spite of the calls for better security, a study done by the georgia institute of technology indicates that security is not the most important determinant for online shopping (gvu, 1999). moreover, although jarvenpaa & todd (1996) found that risk is an important factor in electronic shopping, their results also indicate that consumers consider product perceptions, shopping experience, and customer service as more important an objective of this study is to examine the extent to which perceived risk is a deterrent to online shopping. in the marketing literature, cox & rich (1964) argue that customers who use telephone shopping perceive risk because they cannot personally inspect the products or compare the quality, size, or style of products. customers may also perceive risk because time may be lost or frustration may result where the purchases are unsuccessful. 74 ajis vol 9 no. 1 september 2001 following cox & rich (1964), this study defines perceived risk as "the nature and amount of risk perceived by a consumer in contemplating a particular purchase action" (p. 33). in their study of mail-order shopping, simpson & lakner (1993) identify four types of perceived risk, namely, economic, performance-related, physical, and social/psychological. based on simpson & lakner (1993), jarvenpaa & todd (1996) identify the following five types of risk in online shopping: • economic risk: the risk of monetary loss arising from online shopping. it is a result of the unreliability of vendors. • social risk: social risk involves acceptance of the shopping behaviour by other society members, such as family, and friends. • performance risk: performance risk is concerned about whether the purchased goods or services are able to meet customers' expectations. • personal risk: the possibility of any harm to the customers because of the purchase behaviour. for example, customers may suffer because credit card information is compromised. • privacy risk: customers suffer from privacy risk if they lose their privacy. this may be a result of the fact that information is collected about the customers during the shopping process. jarvenpaa & todd (1996)'s results indicate that personal risk and performance risk are more important than other types of risk. this study adapts jarvenpaa & todd's classification and proposes four types of perceived risk regarding b2c e-commerce. the privacy risk in jarvenpaa & todd (1996) is combined with the personal risk in this study because these two types of risk are similar. in relation to b2c e-commerce, customers find online shopping useful if they can get the shopping done fast or the information provided on the web can help customers make purchasing decisions. if customers perceive high levels of risk in online shopping, it is likely that they will find b2c e-commerce less useful. however, if customers are willing to continue to purchase online, it implicitly reflects that customers find online shopping useful. therefore, this study hypothesises that: • h3: perceived risk is negatively associated with perceived usefulness. social influence the idea of social influence originated from social psychology (salancik & pfeffer, 1978; fulk et al., 1990). in some cases, researchers used different terminology to describe social influence. for example, the theory of reasoned action (fishbein & ajzen, 1975; ajzen & fishbein, 1980; karahanna et al., 1999) used the term "subjective norm", whereas chau & hui (1998) used "opinion leader." social influence in this study refers to the influence of colleagues, friends, and relatives on customers. kelman (1961) identifies three stages in social influence: compliance, identification, and intemalization. at the compliance stage, a person is under social influence because he or she simply wants to get a favourable reaction from others. at the identification stage, a person actually sees some advantages in the social influence, although the main motivation is still to meet others' expectations. at the final stage, a person internalises other people's values or opinions. this study includes social influence because recent studies suggest that this factor has an impact on new technology usage (gefen & straub, 1997; henderson et al., 1998; jasperson et al., 1999; venkatesh & morris, 2000). jasperson et al. (1999) found that social influence could lead to three types of responses (conformance, imitation, and mutual discovery) in new technology usage. so far, tam researchers have different opinions on the usefulness of social influence. davis (1989), adams et al. (1992), and szajna (1996) did not examine this construct in their research. most studies that consider this construct report that it is significant (karahanna & straub, 1999; venkatesh & morris, 2000; agarwal, & prasad, 1997). parthasarathy & bhattacherjee (1998) found that social influence is more important than external influences such as advertisements. nonetheless, some studies found social influence to be insignificant (e.g., davis et al., 1989; mathieson, 1991). schmitz & fulk (1991) and karahanna & straub (1999) found that social influence has a significant impact on behavioural intention through perceived usefulness. however, the theory of reasoned action posits that social influence has a direct effect on behavioural intention (fishbein & ajzen, 1975; ajzen & fishbein, 1980). karahanna et al. (1999) also report that potential adopters' intention to adopt new technology was solely affected by social influence. their model indicates a direct relationship between social influence and behavioural intention. hence, this study tests the following two hypotheses: • h4a: social influence is positively associated with perceived usefulness. • h4b: social influence is positively and directly associated with online shopping behaviour. 75 ajis vol 9 no. 1 september 2001 perceived ease of use and perceived usefulness prior studies found that perceived ease of use only has an indirect effect on usage through perceived usefulness (gefen & keil, 1998; karahanna & straub, 1999). moreover, research to date consistently shows that perceived usefulness is a significant predictor of system usage (davis, 1989; davis et al., 1989; karahanna & straub, 1999). therefore, this study examines the following two hypotheses: • h5a: perceived ease of use is positively associated with perceived usefulness. • h5b: perceived usefulness is positively associated with online shopping behaviour. conclusion this proposed study reconstructs the boundary of tam in the current web-dominant situation by adding experience, self-efficacy, perceived risk, and social influence. moreover, this study can help businesses adjust their e-commerce strategies. for instance, businesses can learn from the results, and decide whether it is worthwhile to spend more money on upgrading security over the web. this in turn will affect the future development of security of the web. moreover, if the results support the importance of experience on online shopping, this outcome should motivate companies to spend more effort, such as giving special discounts, to attract new customers. if social influence is found to be significant, managers may need to think of ways to create a positive social influence over customers. the research methodology of this study is survey. it is assumed that this model is in equilibrium. therefore, it is appropriate to examine the associations among constructs in a cross-sectional manner. the target participants are university students and employees of organisations involved in e-commerce. these two groups of people are believed to represent an important portion of the e-commerce customers because of their easy access to the internet. the next step of this research is to prepare survey instruments. existing instruments of perceived ease of use, perceived usefulness, and self-efficacy will be adopted (davis, 1989; compeau & higgins, 1995b), whereas new instruments will be developed for the other constructs. a pilot study will be performed to confirm the validities and reliability of the instruments before actual data collection. references activmedia research llc(1999) real numbers behind 'net profits', url http://research.acti\tnedia.com/reports/rn99exec/rn99execsummary.pdf adams, d.a., nelson, r.r. & todd, p.a. (1992) "perceived usefulness, ease of use and usage of information technology: a replication" mis quarterly, 16(2), 227-247. agarwal, r., & prasad, j. (1997) "the role of innovation characteristics and perceived voluntariness in the acceptance of information technologies" decision sciences, 28(3), 557-582. agarwal, r., & prasad, j. (1998) "a conceptual and operational definition of personal innovativeness in the domain of information technology" information systems research, 9(2), 204-215. ajzen, i., & fishbein, m. (1980) understanding attitudes and predicting social behaviour, prentice hall: enlewood cliffs, nj. arnum, e. (1995) "doing business on the internet-a question of balance" business communications review, 25(8), 35-37. bandura, a. (1977) "self-efficacy: toward a unifying theory of behavioural change" psychological review, 84(2), 191-215. bandura, a. (1982) "self-efficacy mechanism in human agency" american psychologist, 37(2), 122-147. beer, s. (2000) "e-commerce to double internet spending" the australian financial review, 4 april, 45. boston consulting group (2000) the state of online retailing 3.0, shop.org, url http://www.shop.org/research/default.htm charlton, c., gittings, c., leng, p., little, j., & neilson, i. (1999) "bring the internet to the community" interacting with computers, 12, 51-61. chau, p.y.k., & hui, k.l. (1998) "identifying early adopters of new it products: a case of windows 95" information & management, 33,225-230. compeau, d.r., & higgins, c.a. (1995a) "application of social cognitive theory to training for computer skills" information systems research, 6(2), 118-143. compeau, d.r., & higgins, c.a. (1995b) "computer self-efficacy development of a measure and initial test" mis quarterly, 19(2), 189-211. compeau, d., higgins, c.a., & huff, s. (1999) "social cognitive theory and individual reactions to computing technology: a longitudinal study", mis quarterly 23(2), 145-158. conlin, r. (2000) "european e-commerce poised for boom" e-commerce times, march 28. url http://www.ecommercetimes.com/news/articles2000/000328-7.shtnil 76 ajis vol 9 no. 1 september 2001 consult pty ltd. (2000) australian online shopping report, july december 1999, url http://www.consult.com.au/index.html cox, d.f., & rich, s.u. (1964) 'terceived risk and consumer decision making-the case of telephone shopping" journal of marketing research, 1(4), 32-39. davis, f.d. (1989) "perceived usefulness, perceived ease of use, and user acceptance of information technology" mis quarterly, 13(3), 319-339. davis, f.d., bagozzi, r.p., & warshaw, p.r. (1989) "user acceptance of computer technology: a comparison of two theoretical models" management science, 35(8), 982-1003. davis, f.d., & venkatesh, v. (1996) "a critical assessment of potential measurement biases in the technology acceptance model: three experiments" international journal of human-computer studies, 45(1), 19-45. delone, w.h., & mclean, e.r. (1992) "information systems success: the quest for the dependent variables" information systems research, 3(1), 60-95. drucker, p.p. (1999 oct.) "beyond the information revolution" atlantic monthly, 47-57. fenech, t. (1998) "using perceived ease of use and perceived usefulness to predict acceptance of the world wide web" computer networks and isdn systems, 30,629-630. fishbein, m., & ajzen, i. (1975) belief, attitude, intention and behaviour: an introduction to theory and research, addison-wesley, boston, m.a fram, e.h., & grady, d.b. (1997) "internet shoppers: is there a surfer gender gap?" direct marketing, 59(9), 46-50. fulk, j., schmitz, j, & steinfield, c.w. (1990) "asocial influence model of technology use", in j.fulk and c.w. steinfield (eds.) organisations and communication technology, newbury park, ca: sage publications, 117-140. gefen, d., & keil, m. (1998) "the impact of developer responsiveness on perceptions of usefulness and ease of use: an extension of the technology acceptance model" data base for advances in information systems, 29(2), 35-49. gefen, d., & straub, d.w. (1997) "gender differences in the perception and use of e-mail: an extension to the technology acceptance model" mis quarterly, 21(4), 389-400. grose, e., forsythe, c., & ratner, j. (1998) "using web and traditional style gudtes to design web interfaces", in c. forsythe, e. grose, & j. ratner (eds.) human factors and web development, lawrence erlbaum associates: new jersey, 121-136. gvu (graphic, visualization, & usability center). (1999) gvu's tenth world wide web user survey. georgia institute of technology, atlanta, ga, usa. url http://www.gvu. gatech.edu/gvu/user_surveys/survey-1998-10/tenthreport.html henderson, r., rickwood, d., & roberts, p. (1998) "the beta test of an electronic supermarket" interacting with computers, 10, 385-399. hendrickson, a.r., massey, p.d., & cronan, t.p. (1993) "on the test-retest reliability of perceived usefulness and perceived ease of use scales" mis quarterly, 17(2), 227-230. horey, j. (1999 nov. 30) "well, i'll be it was b2c" the australian, 8. hubona, gs., & geitz, s. (1997) "external variables, beliefs, attitudes and information technology usage behaviour" in proceedings of the 32nd hawaii international conference on system sciences. igbaria, m., guimaraes, t., & davis, g (1995) 'testing the determinants of micro-computer usage via a structured equation model" journal of management information systems, 11(4), 87-114. icaa (institute of chartered accountants in australia). (1999) assurance for business-to-consumer electronic commerce. url http://www.icaa.org.au/icaa/assurance/centreset.htm jarvenpaa, s.l., & todd, p.a. (1996) "consumer reactions to electronic shopping on the world wide web" international journal of electronic commerce, 1(2), 59-88. jasperson, 'j(s)., sambamurthy, v., & zmud, r.w. (1999) "social influence and individual it use: unravelling the pathways of appropriation moves" proceedings of the twentieth international conference on information systems, prabuddha de, & janice i. degross (eds.). charlotte, north carolina, 113-118. jia, j.m., dyer, j.s., & butler, j.c. (1999) "measures of perceived risk" management science, 45(4), 519-532. karahanna, e., & straub, d.w. (1999) "the psychological origins of perceived usefulness and ease-of-use" information & management, 35(4), 237-250. karahanna, e., straub, d.w., & chervany, n.l. (1999) "information technology adoption across time: a crosssectional comparison of pre-adoption and post-adoption beliefs" mis quarterly, 23(2), 183-213. kelman, h.c. (1961) "process of opinion change" public opinion quarterly, 25, 57-78. korgaonkar, p.k., & wolin, l.d. (1999) "a multivariate analysis of web usage" journal of advertising research, 39(2), 53-68. mathieson, k. (1991) "predicting user intentions: comparing the technology acceptance model with the theory of planned behaviour" information systems research, 2(3), 173-191. 77 ajis vol 9 no. 1 september 2001 mcdonald, s., & stevenson, r.j. (1998) "navigation in hyperspace: an evaluation of the effects of navigational tools and subject matter expertise on browsing and information retrieval in hypertext" interacting with computers, 10, 129-142. mis australia. (2000) "b2b booms" march, 88. moore, g.c., & benbasat, i. (1991) "development of an instrument to measure the perceptions of adopting an information technology innovation" information systems research, 2(3), 192-222. morris, m.g, & dillon, a. (1997) "how user perceptions influence software use" ieee software, 14(4), 58-65. nua ltd. (2000) forrester research: usd? trillion in ecommerce revenues by 2004, url http://www.nua.ie/surveys/?f=vs&art_id=905355736&rel=true parthasarathy, m., & bhattacherjee, a. (1998) "understanding post-adoption behaviour in the context of online services" information systems research, 9(4), 362-379. pollatsek, a., & tversky, a. (1970) "a theory of risk" j. math. psych., 7,540-553. ratnasingham, p. (1999) "implicit trust in the risk assessment process of edi" computers & security, 18, 317321. rebel, t.f. and koenig, w. (1999) "ensuring security and trust in electronic commerce" in doing business on the internet: opportunities and pitfalls, edited by fay sudweeks and celia t. ronun. springer: london, 101-112. richtel, m., & robinson, s. (2000 9 feb.) "several web sites attacked following assault on yahoo" new york times, p. al. rogers, e.m. (1983) diffusion of innovation. the free press: new york., third edition. salancik, gr., pfeffer, j. (1978) " a social information processing approach to job attitudes and task design" administrative science quarterly, 23,224-253. schmitz, j., & fulk, j. (1991) "organizational colleagues, media richness, and electronic mail: a test of the social influence model of technology use" communication research, 18(4), 487-523. schwartz, j. (2000) "poll: hack attacks dent e-confidence". washing post, 2 march 2000, £08. url http://washingtonpost.com/wp-dyn/business/a56622-2000marl.html simpson, l., & lakner, h.b. (1993) "perceived risk and mail order shopping for apparel" journal of consumer studies and home economics, 17,377-398. straub, d.w., & welke r.j. (1998) "coping with systems risk: security planning models for management decision making" mis quarterly, 22(4), 441-469. switzer,p. (2000) "hype jumps on, survey finds". the australian, 3 april, 37. szajna, b. (1996) "empirical evaluation of the revised technology acceptance model" management science, 42(1), 85-92. taylor, s., & todd, p.a. (1995) "assessing it usage: the role of prior experience" mis quarterly, 19(4), 561570. thompson, r.l., higgins, c.a., & howell, j.m. (1994) "influence of experience on personal computer utilization: testing a conceptual model" journal of management information systems, 11(1), 167-187. till, r. (1998) "transforming the way we do business", in t. nash (ed.) electronic commerce: directors and opportunities for electronic business, director pub. ltd. venkatesh, v., & davis, f.d. (1996) "a model of the antecedents of perceived ease of use: development and test" decision sciences, 27(3), 451-481. venkatesh, v., & morris, m.g (2000) "why don't men ever stop to ask for directions? gender, social influence, and their role in technology acceptance and usage behaviour" mis quarterly, 24(1), forthcoming. vu, t., smith, p., & bennett, t. (1999) bizrate.com, npd joint survey reveals 75 percent of online customers are abandoning shopping cart. bizrate.com press release, url http://www.bizrate.com/press/press_room/release_49.xpml zampetakis, h. (2000) "digital ids pave wireless web way" the australian financial review, 8 april, 19. 78 ajis vol. 8 no. 2 may 2001 maritime work and communication peter bagh andersen, department of computer science, university of aalborg introduction this paper presents findings from the project "elastic systems" under the danish center for human machine interaction. the project aims at developing methods for analyzing dynamic maritime work processes and for designing flexible instrument interfaces that will support changing work environments. here i only address the former issue, the design issues being published elsewhere, e.g. in andersen(1999) and andersen & may (2001). more data can be found in andersen(2000). the purpose of the present paper is twofold: (1) to present characteristics of maritime work and communication, and (2) to suggest a conceptual framework that covers communicative as well as non-communicative acts. the latter purpose is motivated in two ways. theoretical motivation: we know that language is self-referential, so that it can speak of non-linguistic entities like ships and waves as well as its own properties, such as the correct wording or style. the easiest way to accomplish this is for language to treat both domains in a similar way. the other motivation is more practical: it is a fact that communication and physical actions are interwoven in maritime work, and a theory that builds on a sharp distinction between these two kinds of behaviors will miss this basic characteristic. the data is from a voyage aboard sally maersk from algeciras to goteborg and back to rotterdam. we recorded 60 hours of high quality video, and the paper builds upon a 16 hours trip from felixstowe to rotterdam, supplemented by data from simulated voyages in the simulator at the danish maritime institute. the conceptual framework is based on lind 1994 and lind 2000 and distinguishes between the following types of action types: (1) prevent(suppress, avoid): if i hadn't done it, then an undesirable state would have developed. prevent drifting by using auxiliary rudder. (2) maintain: if i hadn't done it, then a desirable state would have disappeared. maintain course. (3) help: a positive state would have developed anyway, but i/something made it happen faster/more efficiently. the wind helped us moving the ship to quay. (4) create(produce): if i hadn't done it, it wouldn't have happened. turn on the thrusters. (5) destroy (remove): if i hadn't done it, a undesired state would have persisted. stop engine. (6) let: it will happen even if i don't do anything. let the ship slide into berth without using power. the meaning concept implied here is a modal one, since all action types are defined by means of counterfactual statements: what would happen, had we not done anything, a course of events termed "passive projection" in ryan 1991. this modal conception of meaning is among the stock of trade in structuralist linguistics: meaning units are constituted by means of differences and are defined negatively against units that are possible now, but not actualized. saying that something is blue is to negate that it is green, brown, yellow, etc. the german sociologist niklas luhmann has a meaning definition very much in agreement with this. to luhmann, meaning is a specific mode of experience: namely that human experience, although always filled with a specific actual content, is still aware of the possible contents that are left out: experience experiences itself as variable [...]. it does not find itself closed and self-contained, not restricted to itself, but is always referring to something that is at the moment not its actual content. [...] the problem of integrating the actuality of experience with the transcendence of its other possibilities remains inescapable, and inescapable, too, is the form of experience processing that accomplishes this. it is this that i call meaning. luhmann 1990: 25 a linguist would say that meaning is structured in paradigms only one member of which can be chosen, while the rest of the paradigm is present for the consciousness and affects its actual contents. meaning is also characterized by its immense complexity that cannot be handled by the individual system: the most important feature of the differentiation between actuality and potentiality found in experience resides in the character of the overabundance of possibilities, which by far exceeds what can be realized through action or actualized in experience. luhmann 1990: 26 therefore, actualizing a specific meaning presupposes negating the rest. the constitution of this world of not-yet-actualized potentialities, which is constantly given and accompanies all experience, rests on the specifically human capacity for negation..[...] negation appears [...] to actually constitute the universality, i.e. the world reference, of all practical life — especially where experience or action is directed 83 ajis vol. 8 no. 2 may 2001 positively toward some particular meaning and intends it under the form of "is" or "ought". luhmann 1990: 2728 an overview of maritime work and communication in this section, we give a broad overview of characteristics of maritime work and communication, as it can be observed on the bridge. the next sections give examples and suggest ways of analyzing the findings by means of the framework in section 1. work: • purpose. the purpose of bridge work is to move the ship from departure to destination (produce) in a safe (prevent) and efficient way and with cargo intact (maintain). from this formulation particularly three concepts seem relevant: goal (arriving at destination), maintenance (maintaining cargo and ship), and avoidance (avoiding accidents). • tasks. bridge work fall into two intertwined tasks: maneuvering is the art of pitting the controllable forces (such as rudder and propeller) against the uncontrollable ones (such as wind and current) in order to achieve one's purpose (dmi1995: 1). navigation is the art of fixing the correct position of the ship. • division of labor. there is a strict division of labor defined by the hierarchy (2nd, 1st and chief officer, captain) and the watch schedule. however, planning and execution are not disjoint since the captain who is responsible for the planning is also hi charge in critical situations. • reciprocity. the officer must be able to reconstruct the intentions of the captain on the foreign ship, and imagine how the foreign ship conceives of himself. control • control chains. bridge work involves chains of control that change regularly during the voyage and which involve human and mechanical components. control changes can be simple or complex. controlactions can be the target of other control actions, such as maintaining telephone lines in order to prevent accidents. communication • uniformity: conversations are heterogeneous with respect to register, time and medium. greetings of the pilot could occur in any register whereas the highly specific rudderand course commands belong to the maritime domain. some conversations take only a few seconds, others last several hours. some conversations are oral, others go through walkie-talkie or intercom, whereas still others take place over vhp radio. • standardization. conversations are to some degree event-driven. the structure of some utterances is not primarily controlled by language-internal rules, but by events outside the control of the crew, or by the physical working process itself. conversations range from fixed protocols to more free interchanges. • time-structure. conversations are intertwined and concurrent. the crew often maintains several threads of conversations at the same time and when the current conversation is interrupted, another can be resumed. • verbalization. work, including communication itself, is verbalized. you must say what you are doing. it is important that the crew maintains an updated overview of the state of affairs (carstensen & nielsen 1998), and for this reason, crew members must verbalize their actions. it is also important that errors do not go unnoticed, and it is particularly important that authority does not suppress criticism from subordinates of wrong decisions. • meta-communication. metaand object-communication — communicating about communication and communicating about the voyage — are mixed. there are chains of communication where a speaker request another speak to ask a third speaker to.... • reduction of complexity. complexity is reduced by focusing only on a few choices in the individual conversation. complexity is also reduced by relying on fixed protocols. work as mentioned in section 2, three kinds of actions are important in maritime work: bringing the ship to its destination, maintaining the cargo intact and avoiding casualties. 84 ajis vol. 8 no. 2 may 2001 purpose and task structure if we look closer at the six action types in section 1, we discover that they imply different set-ups of auxiliary forces. produce and destroy are actions that change the world into a state that would not obtain, had we not intervened. they therefore imply that certain means are employed to effect this change. maintain and avoid make the opposite assumption, namely that there are forces in the world that will change it, if we do not do something. furthermore, the means we employ in these action types must be able to counteract the forces. in figures 3.4 3.9 below we use the three types of relations shown in figures 3.1-3.3 as building blocks. (in many cases, the relations in fig. 3.2 and 3.3 can be analyzed as control actions in their own right. solid line: the actor's intention. dashed line: the course of events, had the actor not acted.) b b fig. 3.1. a changes into b unless an action is performed which will result in the outcome c. fig. 3.2. a furthers the outcome of state b or the proper working of object b fig. 3.3. a counteracts the outcome b or the proper working of object b the six action types can then be diagrammed as shown in figures 3.4 3.9. for example, to produce is to see to it that a state that would otherwise be absent in fact becomes present. and to destroy is to cause something otherwise present to become absent. presence absence means absence'> absence fig. 3.4. produce absence countermeasure a presence*^ means presence^^ presence fig. 3.5. destroy presence countermeasure * absence obstacle absence fig. 3.6. maintain fig. 3.7. suppress presence a means presence absence*-' fig. 3.8. let happen. fig. 3.9. help happen. countermeasures can work in two ways. either the force of the obstacle is weakened as shown in figs. 3.6 and 3.7: an example is the building of breakwaters in harbors to counteract the force of the waves. or the countermeasure influences the same state variable as the obstacle, but in the opposite direction. an example is countering side-wind by means of permanent rudder. in this case both obstacle and countermeasure influences the same state-variable, namely the course of the ship. as an example of a authentic configuration of forces, consider the maneuver discussed in text 3.1. 85 ajis vol. 8 no. 2 may 2001 (c) (a) (c) (a) (c) (a) (c) (a) (c) (a) look, now you must also take care, look, now, we reduce speed, right, then the wind gets more power over the ship and ... yes ...and starts to, so...there is possibly current too, so just give it about five degrees drift to port. we are already high up, we steer seventy, so ... yes and the course in the map says seventy eight....but it did take hold when i came over noordhinder yes, but it changes all the time here, right, but particularly, particularly when you reduce, reduce speed, then you of course get more sensitive to ... yes both wind and current that is evident text 3.1. complex control. when speed is reduced, the rudder is less effective since the force of the rudder is dependent on the water flowing past. at the same time, the movement of the ship is influenced by currents which necessitates auxiliary rudder. so when speed is reduced, more auxiliary rudder must be given in order to stay on course. or, using our building blocks (fig. 3.10): we want to maintain the course, but the wind will change it if we do not intervene. we can use the rudder as a means for counteracting the force of the wind, but, at this time, we are slowing down to kill time, so the low speed will decrease the force of the rudder. in such cases, we can use the engine to speed up a bit to counteract the effects of low speed (a more detailed analysis will show that what really happens is that faster rotation of the propeller increases the flow of water along the rudder and thereby increases its effect). rudder engine course deviation low speed wind coursecourse fig. 3.10. maintaining course. speeding up to increase the effect of the rudder in order to counteract the force of the wind. many maritime maneuvers can be represented by the configuration shown in fig. 3.11. the means aims a producing the goal state and may simultaneously produce the negative side-effect. the side-effect and the obstacle suppress the achievement of the goal, and the countermeasures suppress the effect of obstacle and sideeffect. 86 ajis vol. 8 no. 2 may 2001 obstacle. countermeasure absence ^ absence fig. 3.11. goal, means, side-effects, obstacles and countermeasure as a set of connected control actions. fig. 3.12 shows an example of fig. 3.11: the rudder is used to counteract a turning moment produced during a crash stop that is achieved by putting the propeller in reverse. sideeffect (turning moment) reduced speed & same course a means (propeller) countermeasure (rudder or thruster) speed same speed fig. 3.12. using rudder to counteract turning moment produced during crash stop effected by putting the propeller in reverse. real maneuvering and navigation consist in handling a large set of concurrent control actions where the same objects and states participate in several control actions simultaneously. the countenneasures in fig. 3.11 does no only show up in maneuvers, but also in the layout of the bridge and in communication protocols. therefore, merely knowing the purpose of an instrument does not suffice, one also has to know the type of accidents it is designed to prevent. for example, both bridge and engine can control the engine power. this shared control is regulated in order to prevent failures. in a cargo ferry, engine control could be exercised from five different places: the main distinction was the opposition between bridge and engine room; if control were on the bridge, it could be allocated to four different places: either to the stern bridge or, on the front bridge, to the central console or to the port or starboard consoles. see fig. 3.13. the bridge crew could itself physically distribute control to the four "bridge-locations but not to the engineer. in the latter case, the bridge can only request that the engine takes over control. the engineer acknowledges the request and itself makes the physical transfer of control in the engine room (fig. 3.14). why this protocol? the reason is that the ship's safety is jeopardized if the engineer happens not to be present in the control room of the engine at the moment control is transferred. in this case, nobody would be in control of the ship. 87 ajis vol. 8 no. 2 may 2001 fig. 3.13. control transfer panel on bridge. fig. 3.14. control switch in engine room some of the communicative patterns conform to the pattern in fig. 3.11 as well. for example, commands are normally repeated (text 3.2), and this is true also of reports via vhp or intercom (text 3.3). the purpose is to avoid misunderstandings due to noise. (l) and the bow to port please () (c) bow full to port (l) yes ... dead slow astern (c) dead slow astern text 3.2. repeated orders during berthing in rotterdam. l = pilot, c = captain. (a) (c) (c) (l) and then the stern is you've got the stern the stern is inside the inside the corner of the quay (radio) inside, ok (radio) corner yeah, okay, okay text 33. repeated reports during berthing in rotterdam. l = pilot, c = captain, a = 2nd officer on the stern deck. fig. 3.15 shows how the task structure is repeated in the communication protocols. noise may hinder the utterance in producing the desired action, and repetition counteract this negative effect. misunderstandings due to e.g. noise action a utterance t repetition of utterance no action action fig. 3.15. goal, means, side-effects, obstacles in communication. the importance of obstacles and negative side-effects is due to the fact that many forces working on the ship are uncontrollable. for the experienced officer, some configurations of these forces are predictable while others are not, some being chaotic (in the mathematical sense). text 3.1 gave an example offerees that can be partially predicted, but inside harbor the forces may turn out to be unpredictable: (b) it is possible it was that which, or...the pilot talked about it being deeper the last distance to the quay,(...) than it really is out in the channel (...) he said that at arrival (...) he kept saying that strange things happen which you do not exactly know, (...) before you are in it, so there is only one way, and that is to keep your eyes open, and then... _^____ text 3.4. unpredictable forces. 88 ajis vol. 8 no. 2 may 2001 planning and execution apart from being responsible for the planning and the decisions, the captain also takes part of the execution since he is in control during departure from and entrance to harbors (the helmsman is taking care of the wheel, the captain wields the machine telegraph, and the pilot guides both). this is unique for the maritime domain, since in most organizations planning and execution are done by different persons. this means that the officers participate in two rather different kinds of conversations about the same event, planning and execution. let us first look at the planning bit. (c) and then .. i, i guess we go in the normal way and .. (c) and turn around in the beer canal.. (l) that's right (c) and and back her in (l) back her in yes yes (c) yeah yes (l) (...) procedure here to back (...) (c) that's more or less normal procedure now (...) (l) that is normal procedure for this type (...) (l) yeah yes yes (c) yeah yes yes (l) we we could turn here but yeah (c) i have tried it with the k-class, to turn it here, get the bow up, and well, it was nice and calm so ... no problem (l) that is right for the wind is pushing your all the way back towards the buoy here so ... (c) yeah... yeah (l) now i prefer to do it with this these type always here in time (...) (c) yeah you don't get anything (l) no you don't get anything (c) no no (c) so what do you know, what kind of wind we have in there, we have about what here, eight meters per seconds out here now (l) yeah it has been force five from northeast something like that, i think so yes (c) can we guess, um can we berth her without a tug? (l) arh i think one aft (c) one aft (l) yes (c) yeah text 3.5. planning the entrance to rotterdam at 10.50. at the planning stage at 11 o'clock, the maneuver is characterized by the opposition, turn and back, and is presented as the normal and safest way, "you don't get anything". the reference to "normality" is probably not a coincidence, but may be a way of reducing meaning complexity in luhmann's sense (section 1). both pilot and captain have tried this before many times, and by referring to a "standard", many decisions have been taken without needing discussion. the text has expressions for probability interspersed since neither captain nor pilot want to impose on the other: i guess, we could, i prefer, can we berth her without a tug? arh i think one aft. and is in the future or the past (i have tried it with kclass). they cooperate about the planning, supporting and supplementing the utterances of the other, and taking care to be polite and supportive. the captain may have wished to leave out the tug to save money, but does not insist when the pilot recommends one. the execution of this plan looks quite different four hours later. 89 ajis vol. 8 no. 2 may 2001 (l) and the stern thruster to starboard captain (c) stern thruster to starboard (p) (l) stop engine (c) stop engine (p) (c) four knots (l) yeah that is okay (c) yeah ... that's okay (l) is she always working? (c) the thruster? (l) the thruster yeah (c) yeah (l) to starboard, ah the stem thruster to starboard and the bow to port (c) bow to port yeah (p) (l) mmm she is not swinging that much (c) no only (l) ah few degrees only (c) few degrees only (l) all right... hard .. hard to port please (c) hard-a-port (l) dead slow ahead (c) dead slow ahead (p) (a) and then the stem is inside the corner of the quay (radio) (c) you've got the stern inside, ok (radio) (c) the stern is inside the corner (l) yeah, okay okay (l) see the wind have a lot of effect on the ship arh? (c) yes ... it is also (...) up now fifteen m/sec, that is thirty knots (l) yeah (p) (c) and it is nearly on the beam still (l) yeah (p) (l) okay stop the engine (c) stop engine (l) midships the wheel (c) midships (p) (l) and stop the stern thruster captain (c) stop the stern thruster (p) (c) stern thruster stopped (l) okay (l) stop the bow thruster (c) stop the bow (p) text 3.6. execution (turning the ship) at 14.11 the two tasks, turn and back, are dissolved into many small steps characterized by rudder, thruster, and machine telegraph orders (midships the wheel, stern thruster to starboard, dead slow ahead), and occasional reports of 90 ajis vol. 8 no. 2 may 2001 the state of the ship (is she always working? she is not swinging that much). probability has turned in necessity, and arguments have vanished. verbs are normally omitted (except in stop engines) and only sentence parts designating instrument settings (dead slow astern) remain. the tense has become present. the utterances follow a rigid order + accept schema, and are intermingled with physical manipulation of controls and observation of instruments and environment. reciprocity during departure from felixstowe (text 3.7), captain and pilot identify a vessel and begin guessing at his course (what course he is gonna take, he is most on, on the western side of the fairways). predictability seems to be highly valued, since they complain about ships that cut straight over, and the captain enquires whether she will have a pilot aboard, implying that a pilot will ensure normal and predictable behaviour. (l) okay... ferry coming in there (c) yeah he is right here (l) small ship there (c) yeah (l) and the dredger (c) no that is the dredge.. (l) that is the dredger, and there is a small ship (...) coming in or.. i'm not quite sure what course he is gonna take, but we will se in a minute (p) (c) yeah its looks like, he is most on on the western side of the fairways (l) yeah (c) yeah (p) (c) but some (...) when when the draft allow us into the (...) (l) (...) cuts straight over and all sorts of thinks yeah (c) yes i have seen that happened (l) yeah (p) (c) is he coming in without a pilot.... (l) he is coming without a pilot yeah, he is gonna pick up a pilot... in here, to go to ipswich, which is sometimes a bit of a problem cause they don't always ... (c) yeah i can imagine (l) realise exactly where they should be in the channel (c) mmm mmm (p) (l) nah, well as long as he keeps heading like that, it will be all right text 3.7. guessing the intention of a small ship. departure from felixstowe. the reason is that the captain and pilot keep imagining scenarios for their own near future, and in order to do that, they have to make guesses about the intentions of other ships. most of the times they succeed, but sometimes the guesses are wrong. in text 3.8, the foreign captain believes that sally will pass her head-on, whereas we are in fact turning, so that the action will be "overtaking". the pilot does not realize the error, but the captain does. originally they believed that the foreign captain believed that they intended to pass by "overtaking", but the captain realizes that the foreign captain believed them to intend to pass head-on. s.atlantic standing by zero two one six[vhf] pilot okay [vhp] master who was that? pilot the outgoing ship, sealand atlantic, she want to pass port to port master how, how can we do that? [pass the sealand atlantic port to port] c. officer (...) 1. officer yes pilot we are steering around slowly, slowly pause 1. officer i presume he means that he wil l . . . 91 ajis vol. 8 no. 2 may 2001 master yeah, but how can he ...(...), he will over take us probably.... pilot yeah, i don't know master ...yeah, no, so urn, port to port pilot sealand atlantic, the sally masrsk [vhp] s.atlantic sally masrsk, sealand atlantic [vhp] pilot um, you want to pass us on our starboard side, on the north side? [vhp] s.atlantic roger, 111 like to ...i, i heard you were turning to your port to go back in, i (...) in with the drencher, otherwise we are going red to red, over [vhp] pilot i think red to green with us [vhp] pilotst sealand atlantic ,pilot maas [vhp] s.atlantic pilot maas [vhp] pilotst the, the sally masrsk is turning to port, so you can proceed her north of her, over [vhp] s.atlantic she is gonna turn to port, okay thank you, thank you, sally maersk [vhp] [notice that the sealand atlantic thinks they are talking to the sally masrsk—when they are in fact talking to the pilot station maas approach]. pilot yeah (...) port to port, red to red not possible master that is not possible pilot his red side to our green side text 3.8. beliefs about the beliefs of own intention. the accident was prevented by the captain's being able to recursively having beliefs about the other's belief about his own intentions. the suggestion that meaning is recursive in this way can also be found in posner (1993) and ryan (1991). control it is characteristic of maritime work that it contains long control chains consisting of intermingled humans and non-humans that work in real time. two simple control elements form a chain if the controllee of one equals the controller of the other. the reason for these chains is that location is often a hindrance for performing a certain job on a ship. for example, during berthing, the captain must be on the bridge where the helm and machine telegraph are located, but from the bridge he cannot see the ship's sides, and he cannot manipulate the wires. in the old days, the problem was solved by cooperation and communication, so we had pure social chains of control. during the past decades, crew size has been halved, and remote control has replaced cooperation in many places, yielding mixed chains of control. the machine telegraph is an example: originally it was a device used by the captain to signal commands to the chief engineer. nowadays it allows the captain to control the machine directly. we characterize a chain by 5 elements: 1. controller: the item from where control originate. human or piece of machinery. 2. mediator: the item through which control is administered. language or piece of machinery. 3. controllee: the item controlled. human or piece of machinery. 4. behavior: the controlled aspect of the controllee. verbal or non-verbal behavior. 5. effect (optional): the ultimate causal effect of controlling the behavior. an example of a complex heterogeneous chain is shown in fig. 4.1. the pilot controls the helmsman's hands by means of language, and the helmsman controls the rudder engine by means of the wheel. the ultimate effect is to control the ship's course. f pilot 1 i j language helmsman: | nanos i wheel . f rudder engine: ^ ^1 ruaaer angle i fig. 4.1. complex chain: pilot guiding helmsman. 92 ajis vol. 8 no. 2 may 2001 text 4.1. exemplifies the language part of the chain. (l) one zero zero please (k) one zero zero (l) yeah (p) (k) one zero zero (l) thank you text 4.1. complex, heterogeneous chain: pilot guiding helmsman another characteristic is that the control chains change regularly during entry and exit of harbor. in open water, there is only one person on the bridge who handles everything, whereas during entry, the complexity of cooperative arrangements increases: the pilot embarks and guides the captain, a helmsman is called to handle the rudder, the chief officer is called in, and finally control is moved into the wings where the captain finishes the approach. fig. 4.2 shows that the control chain changes from simple to complex when the helmsman is called in. (rudder engine!) ^ rudder angle j "* fig. 4.2. assigning and dismissing helmsman text 4.2 and 4.3 exemplify assignment and dismissal of the helmsman: (p) (c) yes o, will you take the wheel, now we will soon have some action (o) (...) (c) zero nine five (o) zero nine five (c) and it is yours (l) okay (c) yeah hand steering (l) yeah (c) you have got hold of it, right, it works and everything. (o) yes (c) good text 4.2. captain assigns helmsman to duty. (c) yes, k, i will let you off the hook, i have got it here now (k) (...) text 43. captain dismisses helmsman. the different types of control change are verbalized differently: change of medium (e.g. between wing and central bridge) is seen as a movement of control (captain: yes, you can take the steering in, and steady it at a hundred and thirty seven, one three seven. officer: i have taken thruster and telegraph in) which is not the case when humans are involved. here the social control structure influences the way the change is treated: captain pilot exchanges are like a negotiation where the parties must agree, whereas captain helmsman exchanges are more like ordering as in text 4.2; and in a pilot 1s' officer exchange, the pilot suggests hand steering to the officer, who subsequently asks the captain if this is ok. in all cases, the change must be marked verbally in a clear and public way. communication conversations are heterogeneous with respect to register, time and medium. heterogeneity 93 ajis vol. 8 no. 2 may 2001 the highly schematized captain-helmsman commands in text 4.1 are unique to the tasks of the maritime domain, whereas the polite greeting of the pilot in text 5.1 could be heard in any social setting where a newcomer enters. (l) good morning sir (c) good morning sir (l) how are you (c) c is my name, welcome onboard (l) thank you (l) you allow smoking or not captain (c) you are very welcome to smoke (l) thank you sir (c) i do not hate you for that reason sir text 5.1. greetings. the short rudder commands (text 4.1) take only a few seconds, whereas a planning and re-planning discussion, due to the fact that the ship occupying our berth was delayed, took 11 hours (the start is shown in text 3.5). triggers some types of communication are triggered by internal linguistic rules. this is true of oral genres such as presentations and speeches, and of most written genres. there is nothing to interrupt the speaker/writer who can undisturbed compose his speech or text. the situation is different in maritime communication: although linguistic rules do regulate local parts of the conversation, so that answers follow questions, and acknowledgements follow orders, the main structure of the conversation is triggered by outside events and the progress of work. this means that the coherence of communication resides outside language. longer stretches of communication are triggered by the environment and not by linguistic rules. this is obvious with the many comments on ships appearing and disappearing: such comments can only be made after the vessel is in view, and are uninteresting when it has sailed past. (l) okay... ferry coming in there (c) yeah he is right here (l) small ship there (c) yeah text 5.2. comment on passing ferry. (d) i have him out in four [degrees?] to the starboard (c) mmm (d) there is one coming towards us (c) yes (d) cpa at zero comma four text 53. comment on approaching ship. (c) we can se her coming in the canal now or... (l) yes .. big container ship coming out with ... if it's gudrun i don't know.. we will see text 5.4. comment on approaching ship. thus, maritime communication is event-driven, i.e. it depends upon events that are outside the control of the crew. this means that there is no guarantee that one conversation can end before another one begins. in fact, the typical situation is that a conversation is interrupted by another one, and later resumed. in text 5.5, the captain is involved hi a long conversation concerned with unfastening the moorings (times normal), but it is interrupted by a reminder to the 2nd officer (times italic) and a planning conversation with the pilot (courier). 94 ajis vol. 8 no. 2 may 2001 mooring critique mooring planning mooring text 5.5. (a) (c) (c) (a) (c) (a) (c) (d) (c) (c) (l) (c) (d) (a) (c) (c) (l) (c) (l) (c) (l) (c) (l) (c) (l) (c) (l) (c) (l) (c) (l) (c) (l) (c) (l) (c) (a) (c) three and we are ready astern (radio) you are ready astern (radio) (p) a, while i remember it, when you check the bridge, then remember to finish with the stern horn, so that it is switched to the stern horn, because i do not always check it in the morning, and then someone in the stern may be in for a surprise if we use it (radio) yes, okay, it is understood (radio) it is just a small detail which the firsts officer and his people would appreciate (radio) maybe it becomes my turn some day (radio) because it bloody often ... and if you don 't see it, and it is just a bad habit... and we are ready fore (radio) you are ready fore (radio) okay we can start singling up yes sir you single up to one and one we single up to one and one (radio). (p) then we should be ready to run out of here and sir have you received our pilot chart, we have it ready for you here okay will there be any ... the same as usually ... as also under departure her ... coming out yes usual route out and everything yes (p) let's see ... yeah it is just the ..the speed we have to decide on coming around this spit () here, what would that be? ... coming around eight knots? (...) dead slow and then just push her up a little bit (...), and then, coming around about eight knots (...), that's fine captain keep her on a low speed, so we can always give her a kick give her a kick if we need to, yes sir ... yeah you fully agree with my ideas and ... yeah yeah and normal routine out (...) just normal routine now (...), there shouldn't be too much (...) traffic no (...) you don't expect any heavy traffic in, inbound no sir very good there might be the old ferry or something, but nothing too major yes and we have clear way around the corner, and the stone barge will be somewhere, but we will get him out in the channel (...) very good (...) before we get there that is all understood (p) and one and one astern (radio) it was one and one astern (radio) concurrent conversations verbalization maritime work must be verbalized and verbalization comes in two forms: 1 . it is important that the crew maintains an updated overview of the state of affairs, and for this reason, crew members must verbalize their actions. 2. it is important that errors do not go unnoticed, and it is particularly important that authority does not suppress criticism from subordinates of wrong decisions. 95 ajis vol. 8 no. 2 may 2001 the two forms differ with respect to control: hi the first case, the control action is to maintain a state of knowledge that will deteriorate if communication stops — many pieces of information on the bridge (e.g. the current course and speed, the vessels outside, etc.) loose their validity automatically as the voyage progresses, and must therefore be maintained. in the second case, the problem is not that information becomes invalid, but rather that it is suppressed because of the speaker's fear of his superior. the control action, effected through the ship-owner's guidelines, is thus to suppress this suppression: in summary the owners' require superiors as well as subordinate to be assertive and subordinates must speak up when orders in their considered opinion are contrary to rules and regulations or will obviously give results contrary to the superior's intent. maersk 1999: section 2.02:1-2. as shown in the italicized words above, the control actions seem useful for describing these important facts too. fig. 5.1 diagrams the first case, maintaining an up-to-date overview. verbalization obsolete knowledge the movement of the ship ./ f y x x up-to-date knowledge -^ ^up-to-date knowledge fig. 5.1. maintaining overview. however, such norms can themselves be the victim of obstacles such as forgetfulness or sloppiness, and therefore themselves need to be maintained: maintain in order to maintain. we see this in the next example where the 1st officer notices that the pilot manipulates the autopilot himself. this is not allowed on this ship, since pilots do not verbalize their actions, and thereby violates the rule of verbalization. therefore the officer notifies the captain of the breach of regulations, and the captain asks the pilot (politely: nah, it's easier just to lets us now) to ask the crew to change the course. (d) now at zero nine zero (c) zero nine zero, well he doesn't say anything about it (d) no, maybe we should ask him to (c) yes (p) (d) when you infor... when you changed the course (...) inform (...) just tell me which course you go to please () (l) okay (d) thank you very much (c) nah it's easier just to lets us now, then we will do it so .... yeah (l) okay fine text 5.6. maintaining the rule of verbalization. in the next example, the helmsman is criticized for not verbalizing when the ship has reached the new course: (c) and then you sing out, k, when you are on the new course, right, (..) when it is there, then you say one one five, when it is mere, then we know, then we know that it is there (k) i did say that the last time (c) well, i just didn't hear it (k) (...) (c) okay, (...), its okay...just keep doing it text 5.7. maintaining the rule of verbalization. 96 ajis vol. 8 no. 2 may 2001 n the second norm (fig. 5.2) the crew want to produce relevant criticism if he possess the relevant knowledge. however, the authoritarian tradition endangers this by suppressing producing. a countermeasure is to suppress suppressing producing where the two suppressions are hoped to cancel one another. authoritythonty / i\ speech guidelines t knowledge silence /^-silence fig. 5.2. preventing the suppressing effect of authority. although formally, produce = suppress + suppress + produce, the two expressions do not denote the same state of affairs, since in the last case suppression of suppression is not just "cancelled out", but is realized as expensive training courses and written guidelines. focus and background in any communicative situation, some features are taken for granted, whereas others are the topic of discussion. we shall say that the former constitute the background, the latter \hefocus. backgrounded information is either not expressed at all, • referred to via unstressed pronouns or definite nouns that recur identically during the conversation, or • referred to via certain backgrounding constructions, such as adnominal adjectives and clauses. information can be backgrounded for two reasons: either it is irrelevant and can be represented by indefinite pronouns, such as someone or somewhere, or it is already fixed to a specific item, in which case it can be referred to by definite pronouns, such as he, she or //. we can in fact define a set of communicative functions by means of particular combinations of backgrounded and focused categories of information. the definition can be made very precise by considering a typical sentential schema we assume are used to interpret utterances about physical processes. subject someone tense, aspect, modality does verb acton object something manner in some way time, place sometime schema 5.1. the basic sentential schema. a sentential schema consists of a set of slots following one another in a more or less fixed sequence. each slot can be filled with a particular kind of linguistic material, e.g. nouns, verbs or adverbials. a set of items that can fill the same slot is normally called a paradigm. if we assume that each field can be backgrounded or focused, then there are 26 = 64 different communicative functions. if we furthermore distinguish between three modes of communications, indicative, interrogative and imperative, there are 192 in total. of these, only a small number are actually found. for example, it does not seem to make sense to have all slots backgrounded. below is a list of examples of communicative functions defined by different combinations of background and focus (boldface) paradigms (adapted from andersen 1997). focus paradigms are filled with an question pronoun, indicating an empty placeholder that must be filled. background paradigms come in two varieties: they can also be filled by a indefinite pronoun indicating that the filler of the slot is irrelevant or is postponed to later. the reason for this is the nature of planning. plans are seldom fully specified but contain undecided elements that are only later filled in. the speaker is consciously vague or currently uninterested in these pieces of information. however, it can also be filled by a definite pronoun indicating that the choice is already made, either by default or previously in the conversation. 97 ajis vol. 8 no. 2 may 2001 imperative: | who | should begin | acting on | it | in some way | when | schema 5.2. work distribution: who should begin doing it when! example: writing out the watch schedule: who should take the watch at which time. i who i should | act on | it | in some way | sometime | schema 5.3. resource allocation: who should do it? example: assigning responsibilities to crew: the engineer is responsible for maintenance of the ship. i he [ should begin | acting on [i t | in some way | when | schema 5.4. scheduling: he should begin doing it when! example: fixing the time for a job: the 2nd officer and a sailor should be on the deck at 3 o'clock to assist the pilot embarking. | he | should begin | acting on | what | in some way | now | schema 5.5. object scheduling: he should begin doing it to what now? example: fixing the exact sequence in which objects are handled: the captain pointing out the individual wires to let go in leaving the quay. | he | should begin | acting on | what | in some way | sometime | schema 5.6. task allocation: what should he do to what! assigning duties to the philippine maintenance crew: you should paint the fore deck now. | he | should | act on [ something | how | sometime | schema 5.7. instruction: how should he do it? example: rudder and course orders: the helmsman should turn the wheel which degrees now? interrogative: | he | is | acting on | it | in some way | now | schema 5.8. supervision: is he doing it? [ he | has | acted on [it | in some way | now | schema 5.9. person reporting: has he done it? checking whether the 2nd officer has contacted the pilot. | someone | has | acted on | it | in some way | now | schema 5.10. object reporting: has it been done to it? checking whether the combination has been taken in after pilot departure. [someone | has | acted on | it | how | now ] schema 5.11. control check: has it been done to it in the correct way? in work distribution, the question concerns the actor and the time, while the action and the object are given (who should begin doing it when!). the problem is the who, not the manner and not the what. therefore the actor and time paradigms should contain many members, while the manner slot is irrelevant. a good maritime example is the watch schedule that determines who will be on the bridge at which time. conversely, in instruction the implied question concerns the manner (how should he do it?), and the actor and time slots are irrelevant. an example is the captain's rudder and course commands (text 5.8), but also his instruction to the helmsman to remember to report back when the new course has been reached. scheduling, which is very frequent in maritime operations, differ from work distribution in that the actor too is backgrounded, and the only remaining problem is to fix the time he is to do it. the difference between person and object reporting is that in the first case, we are interested in the actor's behavior and not in the particular work object, e.g. because we want to know if he is finished and free to do another job, whereas the object's state, and not the person's, is important in the latter case. the purpose of using background and focus paradigms is to reduce complexity in the sense of luhmann (section 1). if the utterance is to specify an action that can be performed, then all paradigms must contain exactly one member that connects to work operations. one cannot make the ship sail "some" course. the course must be "200°, 201°, etc.". so the purpose of the conversation is to fix all focus paradigms to exactly one. but if we can choose between, say 10 items in each slot, there are 106 possibilities to choose from, and we will never get around to do anything before the ship is grounded. 98 aj1s vol. 8 no. 2 may 2001 however, if we do neither have to bother with the subject (the helmsman does it), the time (it is now), the action (turn), the object (the wheel), complexity is reduced to the manner slot with a few degrees of freedom (1-30 degrees to each side). text 5.8 shows an example and schema 5.12 shows the instruction sentential schema that is used to interpret it. pilot starboard twenty helmsm what? pilot starboard twenty helmsm starboard twenty text 5.8. rudder commands. subject helmsman modality should verb turn object wheel manner starboard or port , 1-30 time, place now, at the helm stand schema 5.12. instruction: how should he do it? if we look at maritime conversations we discover that typically most of the paradigms are backgrounded by means of the watch schedule, the formal hierarchy, and the voyage plan, so that very few paradigms are focused. the most frequent are the manner (cf. schema 5.12), the time and the aspect/modality paradigm. many maritime conversations are about the detailed parameters for handling instruments, the time for doing an action, and whether an action is imminent, executing or ended (aspect). • aspect: is ready (about) to do it, begins doing it, is doing it, has done it. aspect is extremely important in maritime communication. many tasks are explicitly run through all four phases: for example, when the lines are taken during berthing (text 5.5), the captain is informed (1) when the deck crew is ready for the operation, and he orders them (2) to start the operation. he may be informed while they are doing it (3), and the crew reports back when they have done it (4). • tense and time: past, present, future (will). tense and time is just as important. the future time of the main operations is regularly discussed. • modality: must (is required to), may (is allowed to), must not (is prohibited to), may not (is not required to), possible, and impossible. course-orders are imperatives, whereas reports are indicatives. relief from duty is of the "allowed to" variety. there are three important processes connected with focus: (1) opening the relevant focus paradigms in order to line up the relevant options; (2) closing the focus paradigm in order to reach a decision, and (3) re-focusing (converting background paradigms into focus paradigms) when it is discovered that backgrounded information is erroneous. text 3.8 illustrates the latter process. the assumed background action is "passing head-on" and the focused manner slot contains two members, red to red and red to green, from which they have to choose. sealand atlantic makes the wrong choice red to red although she ought to know that sally is turning port and that the action will be "overtaking". the captain begins to re-focus the modality paradigm (how can we) and the correct action (he will overtake us probably) and the pilot quickly realizes his error (not possible) — although he does not admit to it! in order to follow the shifting members of focus and background paradigms, we can draw a diagram where the x-axis represents time, and the y-axis represent the slots to be filled out. we add the various fillers of the slots and record how long they are active. as long as two or more fillers exist for the same slot, the slot is a focus paradigm whose contents have not been decided. when the lifetime of all but one filler has ceased, the slot is unambiguously filled with one option. consider again the discussion about the future maneuver in rotterdam harbor (text 3.5). what is constant in the sentential schema is the subject which is "sally maersk", the place which is "rotterdam harbor", and the verb which is "turn". these are background slots. the discussion is concerned with the process of entering the harbor, either turning in the beer canal or turning inside the basin. there is no question of whether to "turn" or not, the choice concerns the location of the turn, which makes part of the place-adverbial, "inside/outside basin", the focus slot (fig. 5.3). 99 ajis vol. 8 no. 2 may 2001 sally maersk turn in beer canal inside basin rotterdam subject verb place fig. 5.3 focus and background. closing the place slot. putting it all together we can now combine the focus-analysis with the action analysis. in both action and communication people try to achieve a desired state or avoid an unwanted one, and in both cases, meaning is a modal concept where the actual is pitted against the possible. however, in communication, the state people try to control is not a physical state, but their shared opinion or consensus. the actions available are not physical actions but communicative moves, and the influence of actions upon the state is not a causal effect, but rather an indirect perturbation of the participants' states of mind. the state that will persist if i do not say anything (the passive projection) is the background paradigms, whereas the change in the focus paradigms is the result of my linguistic action. under this interpretation, it is easy to find communicative variants of the six basic action types: (1) prevent(suppress, avoid): if i hadn't said it, then an undesirable opinion would have developed. preventing the dangerous effect of authoritarian suppression in section 5.3. (2) maintain: if i hadn't said it, then a desirable opinion would have disappeared. reminding the 2nd officer of his duty to call the pilot. (3) help: a positive opinion would have developed anyway, but i/something made it happen faster/more efficiently. captain and pilot cooperating abut the correct maneuvers into the harbor in text 3.5. (4) create(produce): if i hadn't said it, the opinion wouldn't have appeared. captain commenting on new ships he notices, text 5.2-5.4. (5) destroy (remove): if i hadn't said it, an undesired opinion would have persisted. captain criticizing bad habits. (6) let: it will happen even if i don't do anything. the speaker keeps silent because he agrees to the lines of arguments so far. as an example of a pure linguistic application, consider the anaphor chain in text 6.1. 1 p okay ... ferry coming in there 2 c yeah, he is right here 3 p small ship there... 4 c yeah 5 p ...and the dredger 6 c no, that is the dredger.. 7 p that is the dredger, and there's a small ship coming in 8 c yes text. 6.1. handling references. as shown in fig. 6.1, captain (c) and pilot (p) are on their way out of felixstowe harbor and the captain has just begun to plot traffic ships on the radar. both pilot and captain keep lookout for traffic via the radar and the window. 100 ajis vol. 8 no. 2 may 2001 fig. 6.1. "ferry coming in there in (1) the pilot creates an association between a vessel noun and a reference (possibly by pointing). in (2) the captain maintains this reference by means of the pronoun he, and confirms the observation. in (3) the pilot creates yet another reference, and in (5) the pilot maintains a reference from a previous conversation by means of the definite article (the dredger—unfortunately, i can't find the introduction of the dredger on the tape). however, in (6) the captain suppresses this reference again: the ship pointed to is not the dredger. the latter two operations are displayed in fig. 6.2. the interpretation is that first the captain wants to remind himself and the pilot of the previous identification of the dredger that may have been jeopardized by forgetfulness. the pilot thinks that this is a misidentification that should not be allowed to live on, and he therefore suppresses it. "and the dredger" 1 x is not the dredger * t / * /' forgetfullness y is the dredger x is the dredger ^~ x is the dredger 'no that is the dredger" •• x is the dredger fig. 6.2. maintaining and destroying references. i have chosen a "technical" subject like construction of anaphor chains to substantiate my hypothesis that the action concepts presented may in fact be refined to account for bonafide linguistic processes too. to summarize, the claims of this paper are: (1) meaning is a modal concept articulating the possible against the actual; (2) both physical actions and communications are controlled by six action types that oppose the passive projection of the current state against active interference; (3) the six action types imply different configurations of helping and opposing forces; (4) the state controlled by communicative actions is represented by means of focus and background paradigms; (5) the framework can describe essential properties of maritime maneuvers, maritime instrumentation and communicative functions. the paper does not say anything definite about how communication interacts with its social, psychic and physical environment. the focus/background distinction, however, suggests that two processes are important: (6) variety controls variety, i.e. the linguistic choices of the focus paradigms are used to control the choices in the physical and social world; (7) the environment is controlled by reducing its complexity, i.e. the background paradigms represent features of the environment that are taken for granted, and only a small remaining fraction is presented as choices in the focus paradigms. 101 ajis vol. 8 no. 2 may 2001 references andersen. p. b0gh & m. may(2001). tearing up interfaces. in: information, organisation and technology. studies in organisational semiotics. eds. kecheng liu, rodney j. clarke, peter b0gh andersen, ronald k. stamper. kluwer: boston/dordrecht/london: 299-338. andersen. p. b0gh (1997). a theory of computer semiotics. semiotic approaches to construction and assessment of computer systems. cambridge: cambridge university press, 1990, paperback edition 1997. andersen. p. b0gh (1999). elastic interfaces. maritime instrumentation as an example. proceedings of the csapc'99, universite de valenciennes, valenciennes france, 35-41. andersen. p. b0gh (2000). communication and work on maritime bridges. chmi-1-00. http://www.cs.auc.dk/~pba/reportlist carstensen, p.h. & m. nielsen (1998). safeco wp ii. 1.2: task analysis of navigator performance. dmi/riso national laboratory. danish maritime institute (1995). ship handling, course manual. lyngby: danish maritime institute. lind, m. (1990). representing goals and functions of complex systems. institute of automatic control systems. dtu. 90-d-381. lind, m. (1994). modeling goals and functions of complex industrial plants. applied artificial intelligence 8: 259-283. lind, m. (2000). actions, functions and failures in dynamic environments. chmi-8-00. http://www.cs.auc.dk/~pba/reportlist luhmann, n. (1990). essays on self-reference. new york: columbia university press. maersk (1999). guidelines for navigators. maersk vessels. technical organisation, level 2 document. nau610. version april, 1999 posner, r. (1993). believing, causing, indenting: the basis for a hierarchy of sign concepts in the reconstruction of communication. in: jorna, van heusden & posner (eds.) signs, search and communication. berlin: walter de gruyter: 215-270. ryan, m-l. (1991). possible worlds, artificial intelligence and narrative theory. bloomington & indianapolis: indiana university press. 102 microsoft word ajis vol13-2 o.doc 87 australasian journal of information systems volume 13 number 2 may 2006 ajis featured theme: human-computer interaction in information systems theme editors: sandrine balbo university of melbourne email: sandrine@unimelb.edu.au todd bentley monash university accident research centre penny collings university of canberra the fields of information systems and human-computer interaction (hci) are intertwined through their interest in successful systems design; this involves acknowledging the human and technical dimensions of work (and leisure) in systems design and evaluation. indeed there are many similarities within the fields, namely the promotion of participative systems design approaches and the analysis of user-needs in order to understand how the system should perform. specifically, hci practitioners and researchers are interested on how to best address human needs and desires within the context of system use and performance. to achieve this, hci draws upon knowledge from the field of information systems, as well as psychology, graphical design, sociology, computer science, and ethnography (among others). with the increase in technology used in ‘everyday life’, understanding human needs within the system has become even more challenging – there is an increasing demand to understand not just how the person will use the product in question, but also how the overall context surrounding them impacts upon this and how the person perceives the quality of interaction. two recent areas of development and focus within the field of hci are to understand the use of ubiquitous computers and mobile devices. another recent avenue of interest is to understand how products should be designed so that they can be used by anyone regardless of their personal capacities (e.g. older users; blind; children; etc.). again, these areas highlight the need to understand both the human and technical aspects in order to develop an optimal system – a common goal shared by hci and is. ozchi is the annual conference on (hci) run by chisig, the computer-human interaction special interest group of the ergonomics society of australia. starting in the late 1980s, ozchi offers a national and international forum for the presentation of current research and industry practice in hci through papers, panels, workshops, tutorials and posters. the papers included here were judged to be the five best presented at ozchi 2005. for publication in ajis, they have been extended to provide more theoretical background than was initially published through ozchi. each paper was initially blind reviewed through the ozchi review process, then again for the award of the gitte lindgaard prize during the conference, and finally for this journal. our appreciation extends to the reviewers involved in all stages. these papers serve to highlight several current topics within the hci field: the need to look at ‘nontraditional’ methods in order to assess how well a system will work within a specific context (lin et al., graham et al.); designing systems to aid interaction between people in addition to just between a australasian journal of information systems volume 13 number 2 may 2006 88 human and a computer (graham et al., tran et al.); and the research and design implications of moving into a mobile world (hagen et al., baudisch et al.). the editors gratefully acknowledge the work of the revierers in the preparation of this ajis feature theme: ash donaldson, different solutions pty ltd (australia), susan hansen, ict centre, csiro (australia), keith cheverst, computing department, lancaster university (uk) and anxo roibas, school of computing, mathematical & information sciences, university of brighton (uk). ajis vol. 8 no. 2 may 2001 synthesizing structuration and institutional theory: three cases jason bennett thatcher college of business florida state university tallahassee, fl 32306-110 telephone: (850) 644-0916 fax: (850) 644-8225 e-mail: ithatche@mailer.fsu.edu mark srite school of business administration university of wisconsin milwaukee p.o. box 742 milwaukee, wi53201 telephone: (414) 229-5502 fax:(414)229-5999 e-mail: msritefaiuwm.edu manolis diakourakis college of business florida state university tallahassee, fl 32306-110 telephone: (850) 644-0916 fax: (850) 644-8225 e-mail: edd3323@.gamet.acns.fsu.edu david kuhlmeier college of business florida state university tallahassee, fl 32306-110 telephone: (850) 644-4417 fax: (850) 644-8225 e-mail: dbk9613@.eamet.acns.fsu.edu abstract research suggests that it is difficult to predict the influence of information technology (it) on organizations. using the structuration approach, theorists like orlikowski suggest technology's effect is context dependent. within organizations, structuration theory suggests it's use and understanding reflects the interaction of individuals, social structures, and features of the technology. however, in the public sector, external institutions play an important role in shaping the development of internal structures. this paper develops a conceptual framework that links institutions and organizations social structures to the use and understanding of information technology in the public sector. to develop a more accurate image of the structuring process, it suggests theory should explicitly model the influence of regulative, normative, and cognitive institutions on how technology is used and perceived in government organizations. modeling technology in public organizations since the early nineties, reformers have advocated using information technology (it) as a means for enabling citizen participation and effectively managing information. for example, when florida was experimenting with absentee voting over the internet, sandra mortham, then secretary of state, suggested the electronic voting initiative would guarantee citizens the "opportunity to fully participate in florida's democracy" (florida department of state 1997). due to perceived benefits, advocates contend that public agencies should invest in technologies and re-engineer processes (gore, 1993). despite huge government investments in it, public information systems often fail to realize projected benefits (madden and miranda, 1998). a notable example occurred in 1985 when the internal revenue service's computer system faltered under its workload and failed to process tax returns (kelman, 1990). when explaining systems' inadequacy, critics frequently point to features of public organizations like procurement systems and their institutional context such as legislative oversight (e.g. gore 1993). they suggest that the institutional and organizational environment limits the efficacy of it in the public sector. given it's potential for improving public sector service delivery, it is important to understand how the organizational and institutional context affects the understanding and use of it within public agencies. the purpose of this paper is to present a conceptual model linking institutions, social structures, and technology in public organizations. first, it briefly reviews support for deterministic views of it-induced change in public agencies. next, the paper outlines how the structuration approach informs our understanding of the implications of information technologies. then, it extends the structuration approach to the public context. finally, through a 32 aj1s vol. 8 no. 2 may 2001 brief review of three cases, the paper illustrates how institutions influence the social structuring of information technologies within government and non-profit organizations. government and technological change advocates of public sector reform emphasize information systems' (is') role in improving public organizations' responsiveness to their clients' needs (anderson and danziger, 1995). osbome and gaebler (1993) suggest that modern "hierarchical, centralized bureaucracies . . . simply do not function well in the rapidly changing, information-rich, knowledge intensive society and economy of the 1990s" (p. 12). through strategic applications of is, they assert that leaders may build organizations capable of responding to dynamic environments and of meeting changing client expectations. the national performance review (gore, 1993) echoes this logic. it suggests that positive reforms hinge on how managers implement new technologies and change processes in public agencies. if they adopt an it, reformers assert that public agencies will become more efficient, effective, and customer oriented. however, research suggests that it's introduction and use does not necessarily lead to more efficient public organizations. in the urbis studies, danziger, dutton, kling, and kraemer (1982) found that public organizations used it to supplement, not reengineer, existing processes. in the context of mainframe computing, it was used to automate data management systems. although the technology was new, processes were not fundamentally altered. indeed, it may reemphasize the need for hierarchical structures (gurbaxani and whang, 1991). it use increases available information and complexity in the organization and environment. to navigate complex situations, firms will require more middle managers to coordinate a firm's activities (wynne and otway, 1983). clearly, it use may not yield organizational change. towards a dynamic model: the structuration approach given unanticipated results, theorists have sought more dynamic explanations of technology's influence on organizations. when compared to deterministic approaches, researchers (e.g. orlikowski, 1992; desanctis & poole, 1994) suggest that using structuration theory to examine information technology yields a broader understanding of it's influence within organizations. emergent approaches, such as structuration, suggest it1 inked outcomes reflect the interaction of diverse environmental features such as organizational culture and a technology's attributes. structuration theory: a brief review this section briefly reviews structuration theory and its applications to the understanding of it within organizations. it suggests that integrating institutional and structuration approaches will yield a deeper understanding of the social structuring of information technologies within organizations. structuration theory provides a dynamic explanation of how humans construct and interact with their environment. within organizations, social structures may be perceived as rules or norms that guide human understanding and action. they are created by the interaction of human action, existing social structures, and artifacts such as it (giddens, 1979). social structures should be perceived as ways of understanding that persist over time. they define parameters of human interaction and applications of knowledge within organizations. (see orlikwoski (1992) for a more detailed review of structuration theory). structures are transitory in nature. at any given moment, people, rules, and resources interact as part of a recursive process (desanctis & poole, 1994). a structure may be reshaped, reinforced, or undercut by changing perceptions, technologies, or experiences. giddens (1979) suggests that structures should be perceived as having a dualistic nature in that they enable and limit human action. consider a group. values and understanding embodied in structures shape group decision-making processes. by shaping group processes, structures limit choices. simultaneously, they allow group members to organize and make sense of information. by doing so, they enable group decision making processes and legitimize ensuing actions. within organizations, structuration theory suggests it's influence reflects interacting social structures. outcomes are a product of the recursive interaction of an organization's structures, individuals' understanding, and an artifact's features (orlikwoski, 1992). an organization's formal and informal structures such as the division of labor, decision-making processes, or experiences influence how individuals perceive and select a technology. once they adopt an it, individuals' use of a technology's features contributes to new social understanding. in turn, the newly enacted social practices shape the organization's structural features. through users, organizational structures and technology interact. at any given point in time, the interaction between individuals and structures contributes to shifting definitions and perceptions of the technology's potential. as use of a technology shifts, users' perceptions and structures change. within this model, users mediate the interaction of technology and an organization's structure. research using the structuration approach has emphasized endogenous determinants of technology use in organizations. throughout their work, leading "structuration theorists" like orlikowski (1991, 1992) and barley (1986) have consistently emphasized the importance of 33 ajis vol. 8 no. 2 may 2001 structures within groups and organizations. institutions as social structures when exploring the structuring of technologies, scholars have left relatively unexplored the broader institutional context in which organizations are embedded (king et al 1994). institutional theory examines how organizations' choices and their understanding are influenced by the social structures embedded in the environment (scott, 1995). not unlike internal structures, institutions reflect external "rules, norms, and beliefs that describe reality for the organization, explaining what is and what is not, what can be acted upon and what cannot" (hoffman, 1999, p. 351). scott (1995) suggests institutions may be comprised of three distinct types of social structures: regulative, normative, and cognitive. regulative features build on laws and regulations. organizations comply with regulative institutions to avoid punishments such as lawsuits. normative institutions reflect the social understanding embedded in decision-making rules or procedures. organizations comply with these rules to meet perceived social obligations rooted in external value systems such as professional codes of ethics. cognitive institutions embody the symbols and cultural understanding of reality and the frames "through which meaning is developed" (hoffman, 1999, p. 353). social structures are cognitive institutions embedded in firms and broader organizational fields. as a group, regulative, normative, and cognitive institutions frame how issues and artifacts are understood and used in organizations (fligstein, 1990). extending structuration theory to the public context when applied to public organizations, structuration-based models of technology need to explicitly link the broader regulative, normative, and cognitive structures to internal "social structuring" (see figure 1). laws, which can be seen as regulative features, often dictate specific processes for planning, purchasing, and implementing new information technologies. normative features can be seen through limited empirical evidence which suggests that, in comparison to private firms, public agencies' use of information technologies reflects more extensive political interdependencies, more "red tape", purchase decisions more concerned with due process than the "bottom line", more external budgetary constraints, and information management positions situated lower in the organization (cats-baril & thompson, 1995; caudle, gorr, & newcomer, 1991). these results generally support the proposition that the use and understanding of information technologies reflects the degree of discretionary authority extended to managers (kraemer, 1989). finally, organizational "publicness," that is, the extent to which the organization's activities are owned, funded, and overseen by legal governmental authority rather than market forces (e.g., rainey, backoff, & levine, 1976), influences the development of cognitive institutions, i.e. social structures, within organizations. within the public sector, regulative institutions shape structures of domination linked to information technology. in terms of domination, regulative institutions assign decision-making authority for developing strategic it plans and acquiring technology. for example, in florida, a central coordinating agency (state of florida technology office, 1997) guides public technology policies. the coordinating agency's authority is rooted in florida statutes. also, legal institutions articulate rules for the formal technology acquisition processes. for example, in federal agencies, statues spell out requirements for request for proposals, decision-making criterion, and appropriate payment methods for purchasing information systems (kelman, 1990). further, laws underpin perceptions of appropriate uses of information technology. for example, florida public employees are expected to restrict internet use to work-related activities. in numerous cases, employees have been sanctioned or fired for violating guidelines on appropriate uses of technology. clearly, regulative values constrain selection decisions and the range of applications of information technology within public organizations. 34 ajis figure 1. vol. 8 no. 2 may 2001 institutionalism structuration regulative normative cognitive it is important to note that structures and events within organizations influence the broader institutional framework. for example, during the late 1800s, perceived abuse of purchasing systems by public officials led to the strengthening of institutional controls over the purchasing and hiring processes (kelman, 1990). critics suggest that institutional controls have undercut agencies' ability to purchase cutting edge technologies or to develop ongoing relationships with vendors (globerman and vining, 1996). due to constraints, they suggest agencies have not been able realize the perceived efficiencies linked to it use. as a result, reformers have called for the loosening of regulative controls on it purchasing. normative institutions can be seen to influence structures of legitimation linked to information technology. many scholars suggest that employees in the public and nonprofit sectors are motivated differently than their private sector counterparts (perry, 1997). they suggest that values such as equity concern for client needs, or service shape civil servants' behavior (perry, 1990). when evaluating it, normative institutions will influence the perceived legitimacy of information systems. consider the role of stakeholders in evaluating system performance. in new zealand, despite realizing economic efficiencies, stakeholder criticism led the department of education to abandon a successful payroll processing system (myers, 1994). reflecting normative values, public employees' obligation to respond to stakeholders played an important role in de-legitimizing an information system within the agency. cognitive institutions influence structures of signification within organizations. scholars suggest that information technology use may parallel shifts in broader cultures (fletcher and taplin, 1999) and governments (norris and kraemer, 1996). consider information systems design. in a comparative study of scandinavia and the united states, kraemer, anderson, and perry (1994) found approaches to design reflected cultural differences across countries. in scandinavia, information systems were designed to meet needs across agency and government boundaries. within the united states, public information systems' development reflected distinct environmental pressures felt by different agencies. cognitive institutions influenced the development of distinct system design processes and outcomes. where scandinavian agencies emphasized values such as consistency and gradual development, u.s. agencies information systems' development processes reflected values such as responsiveness and stakeholder participation in defining organizational needs. in either country, cognitive values shaped perceptions of appropriate organizational rules for systems design. implications of institutions and structuration approach: three web development case studies in this section, structuration theory is used to extend our understanding of institutions' influence on it use in the public sector. three cases are used to illustrate the influence of institutions on the social structuring of internet technologies within organizations. to protect the anonymity of the organizations, pseudonyms have been used in each case. case one: regulative institutions and website design florida possesses among the most rigid right to information and access laws in the united states. the sunshine laws require two essential tasks: information provision and notification of meetings. chapter 119 of the florida 35 ajis vol. 8 no. 2 may 2001 statutes requires any record created or received by publicly commissioned agencies while conducting official business to be available for review unless specifically exempted by the legislature. the definition of public records includes paper and electronic documents including sound recordings, electronic communications, and photographs (florida attorney general 1997b). chapter 286 of the florida statutes establishes the right of access to most board, commission, or other public governing body meetings (florida attorney general 1997a). public organizations are required to provide access to any meetings where two or more governing members will discuss business. florida agencies' application of internet technologies reflects political leaders' legal authority and stringent information provision requirements. oversight is a regulatory commission charged with overseeing the service delivery of a large state-agency. they are charged with responding to citizen concerns, monitoring compliance with operating standards, and evaluating a large state agency's performance. within oversight, regulative institutions shaped the acquisition process and the features of their proposed website. the impetus to develop a site stemmed from the legal authority vested in board members. the politically appointed chairman directed staff to contract out the design and implementation of the website. they were instructed to develop a site that notified stakeholders of meetings and provided documents such as minutes, press releases, and reports. reflecting sunshine law requirements, oversight staff established three criteria for evaluating the website's design. to ensure compliance with information provision requirements, the staff wanted a wide range of users to be able to access information. the website had to be easily navigable via a wide range of browsers ranging from text-based browsers such as lynx to graphical browsers such as netscape. also, to ensure that reports could be viewed and printed by stakeholders, the staff required reports to be provided as text-based and through widely available document readers such as adobe acrobat or microsoft word. to ease compliance with notification requirements, the staff wanted an easily updated site. clearly, the site's evaluation reflected information provision and notification requirements. as implemented, the website's simplicity reflected the staffs concern for ease of access and use. to ensure compatibility with a wide range of browsers, important links were provided in text and image based formats. to broaden access, important documents such as meeting notifications were posted in a text-based format. to ease site navigation, access to important information, like that in reports, was embedded no more than two links from the homepage. in this case, regulative institutions shaped the structuring of the internet site's design process. legal authority, invested in the commission chairman, led to the website's development. more importantly, after the initiating decision was conveyed, staff members' understanding of the technology's potential reflected their understanding of regulative institutions. evaluation criteria emphasized information provision requirements embedded in the regulatory framework. rather than employing "state of the art" features, the website ensured that users with limited access to new software or technology could view and retrieve information. by establishing the legal context, regulative institutions framed structures of domination embedded in the website's design. the website's design processes and outcome were influenced by oversight's regulative environment. case two: normative institutions and information provision over the web acting on values of the reinventing government movement (frederickson, 1991), stakeholders pressured florida agencies to reform basic business processes. for example, florida taxwatch, a watchdog organization, has emerged as an influential monitor of government spending. it has issued numerous reports identifying opportunities for the streamlining of public service provision. while honoring traditional commitments to open government, groups such as taxwatch have argued mat public agencies should provide more and better services. reformers' successes are evident in new performance measurement standards. in 1994, the florida legislature passed the government performance and accountability act. it required agencies to develop, report on, and apply performance measures to service delivery (florida taxwatch, 1999). in 1999, the legislature amended the act to require that agencies provide unit cost information (florida taxwatch, 1999). by focusing attention on unit costs, stakeholders have increased pressure for realizing more effective business processes. shifts in normative institutions re-framed how agencies integrated new technologies. provider is a large state agency charged with notifying interested parties of requests for proposals (rfps) and of regulatory changes. in the past, provider employees have had to collect, format, and share information in a weekly publication. because statutes limited the cost per page of the publication (florida attorney general, 1997a), the agency could not fully recoup the high labor and production costs tied to the publication. to streamline production processes, provider moved from a printed standard to an electronic solution for delivering the publication. electronic submissions have to conform to pre-specified standards for inclusion in the electronic document. documents are then made available at the first of each week on a website. subscribers may download documents or view them through web browsers. by selecting an internet-based solution, the agency shifted production costs to peer agencies and delivery costs to subscribers. by moving to an electronic solution, provider conformed to pressure to develop efficient and effective business processes. it is important to note that provider has not 36 ajis vol. 8 no. 2 may 2001 made a complete transition to an electronic publication. because many citizens lack computer access, managers expressed a concern that all citizens would not be able to view the information. due to these concerns, provider distributes both print and electronic versions of the document. competing normative institutions provided cultural legitimacy to providing both electronic and paper versions of the publication. responding to shifting normative institutions, provider developed a system that streamlined service provision. by doing so, it reduced publication costs and eased stakeholders' access to information. through the use of it, the agency complied with normative institutions' emphasis on cost-effective service delivery. at the same time, provider's understanding of it's limits were influenced by perceived obligations to provide access to information. despite being less cost-effective, the agency used a paper-based alternative to ensure that a wide range of stakeholders could obtain copies of the documents. in this case, normative institutions influenced the structuring of how the agency perceived limits on the legitimate uses of electronic publications. case three: cognitive institutions and website implementation conserve is a large non-profit environmental organization with over one million members in the united states (us) alone and is the largest private international organization of its type in the world. the nature of the organization is that of an information provider. recently conserve implemented a website. the website design involved representatives from every department in its planning, design, and implementation. its features reflect cognitive institutions' influence on the organization. the website was used to provide external and internal stakeholders information on the organization. interviews with conserve staff indicated that perceived task turnaround time dropped and that the use of internet technology resulted in high performers doing more work. professional staff was able to produce more in terms of both quantity and quality. this increase of information coupled with an increase in output can be explained through the use of cognitive institutions. individuals with greater access to information can better determine the internal representation of their environment and increase their work output. as noted by scott "social construction of actors also defines what they see as their interests" (1995, p. 43). in this case the increase in information allowed the actors (conserve personnel) to better define what they saw as their interests (doing a better job). the introduction of the website also radically changed the way the way conserve handled the enforcement of rules and regulations. in the past rules and regulations were bound and distributed to each department. due to the cumbersome process of having to manually look up a particular regulation enforcement was less stringent. the introduction of the website allowed increased and easier access to the rules and regulations. consequently, enforcement increased. easier access to information allowed individuals to better internalize the rules and regulations, which resulted in more widespread enforcement. this influenced members' perceptions of cognitive institutions on rule enforcement. implications and directions for future research this paper has several important implications towards our understanding of technology in organizations. first, it suggests that scholars should attempt to identify the influence of institutions on the structuring of technologies. by identifying external sources, scholars may identify patterns of structure across industries and organizational fields. by doing so, studies may better inform our understanding of how technology influences organizations. the paper also extends our understanding of how external institutions such as laws or professional values may influence the structuring of technologies. laws constrain human action and the understanding of the range of possible actions. by framing how we intially perceive a technology, external institutions limit the range of possible applications of that technology. to fully understand it as a social structure, more detailed research is needed to examine external institutions' influence within organizations. for practitioners, this research suggests that attempts to sell or implement a technology need to be positioned in a manner consistent with an organization's institutional context. for example, when marketing a technology to a public agency, a firm would be well served to research not only legal issues, but also which institutions and social structures influence managerial decisions. by gaining a thorough understanding of how technology is structured, individuals will increase the likelihood of successful sales or implementation efforts. as presented, the model reflects a limited view of how external structures interact with organization specific structures. a more comprehensive study might examine links between stakeholder perceptions, technology, and public agency activity across levels of government. examining institutions' influence within public organizations should be perceived as a starting point for broader studies of technology's definition across sectors. future research might explore how statutory reporting requirements influence decision-makers' it selection and implementation within firms. also, this study provided a limited view of how organizations' structures influence the development of institutions. future research should examine how processes within a 37 ajis vol. 8 no. 2 may 2001 single organization or group of organizations contribute to broader social and industry-wide understanding and applications of technology. references anderson, k.v. and danziger, j.n. (1995) "information technology and the political world: the impact of it on capabilities, interactions, orientations and values", international journal of public administration, vol. 18, pp 1693-1746. barley, s. r. (1986) "technology as an occasion for structuring evidence from observations of ct scanners and the social order of radiology departments", administrative science quarterly, vol. 31, pp 78-108. cats-baril, w. and thompson, r. (1995) "managing information technology projects in the public sector", public administration review, vol. 55, pp 559-566. caudle, s.l., gorr, w.l. and newcomer, k.e. (1991) "key information systems management issues for the public sector", mis quarterly, vol. 15, pp 171-188. danziger, j.n., dutton, w.h., kling, r., and kraemer, k.l. (1982) computers in politics, new york: columbia university press. de sanctis, g, and poole, m.s. (1994) "capturing the complexity in advanced technology use: adaptive structuration theory", organization science, vol. 5, pp 121-147. fletcher, d.s. and taplin, i.m. (1999) "organizational evolution: the american life cycle", national productivity review, vol. 18, pp 29-36. florida attorney general (1997a) "florida's government in the sunshine law," available on-line: http://legal.firn.edu/sunshine/general.html florida attorney general (1997b) "most frequently asked questions on florida's open government laws", available on-line: http://legal.firn.edu/sunsnine/faq.html florida department of state (1997) "mortham announces pilot program for voting on the internet", available on-line: http://www.dos.state.fl.us/press/new/internervoting.htm florida taxwatch (1999) "pb2 "diamond in the rough" or just a "zirconium bauble"?" available on-line: http://www.floridataxwatch.org/ frederickson, gh. (1991) "comparing the reinventing government movement to the new public administration", public administration review, vol. 56, pp 263-270. fligstein, n. (1990) the transformation of corporate control, cambridge: harvard university press. giddens, a., (1979) central problems in social theory: action, structure and contradiction in social analysis, berkeley, ca: university of california press. globerman, s. and vining, a.r. (1996) "a framework for evaluating the government contracting out decision with an application for information technology", public administration review, vol. 56, pp 577-584. gore, a. (1993) the national performance review, washington, d.c.: u.s. government printing office. gurbaxani, v, & whang, s. (1991) "the impact of information systems on organizations and markets", communications of the acm, vol. 34, pp 59-73. hoffman, a.j. (1999) "institutional evolution and change: environmentalism and the u.s. chemical industry", academy of management journal, vol. 42, pp 451-371. king, j.l., gurbaxani, v, kraemer, k.l., mcfarlan, f.w., raman, k.s., and yap, c.s., (1994) "institutional factors in information technology innovation", information systems research, vol. 5, pp. 139-169. kelman, s., (1990) procurement and public management: the fear of discretion and quality of government performance, washington d.c.: aei press. kraemer, k.l., anderson, k.v., and perry, j.l. (1994) "information technology and transitions in the public service: a comparison of scandinavia and the united states." international journal of public administration, vol. 17, pp 1871-1906. kraemer, k.l., king, j.l., dunkle, d.e. and lane, j.p. (1989) managing information systems: change and control in organizational computing, san francisco: jossey-bass. madden, m. and miranda, r. (1998) "contracting for enterprise financial software: the methodology of parallel negotiations", government finance review, vol. 14, pp 33-39. myers, m.m. (1994) "a disaster for everyone to see: an interpretive analysis of a failed is project", accounting, management, and information technology, vol. 4, pp 185-201. norris, d., and kramer, k.l. (1996) "mainframe computing in american cities: myths and realities," public administration review, vol. 56, pp 568-576. orlikowski, w.j., (1991) "integrated information environment or matrix of control? the contradictory implications of information technology", accounting, management and information technology, vol. 1, pp 9-42. orlikowski, w.j. (1992) "the duality of technology: rethinking the concept of technology in organizations," organization science, vol. 3, pp 398-427. 38 ajis vol. 8 no. 2 may 2001 osborne, d., and gaebler, t. (1993) reinventing government, how the entrepreneurial spirit is transforming the public sector, new york: plume. perry, j.l. (1997) "antecedents of public service motivation", journal of public administration research and theory, vol. 7, pp 181-197. perry, j.l. and wise, l.r. (1990) "the motivational bases of public service", public administration review, vol. 50, pp 367-373. rainey, h., backoff, r. and levine, c. (1976) "comparing public and private organizations," public administration review, vol. 36, pp 233-244. scott, r.s. (1995) institutions and organizations, thousand oaks, ca: sage. state of florida technology office (1997) http://mail.irm.state.fl.us wynne, b. and otway, h.j. (1983) "information technology, power and managers," office, technology, and people, vol. 2, pp. 43-56. 39 microsoft word ajis vol13-2 o.doc 193 australasian journal of information systems volume 13 number 2 may 2006 penetration testing professional ethics: a conceptual model and taxonomy justin d. pierce 1 , ashley g. jones 2 and matthew j. warren 1 1: school of information systems deakin university geelong, victoria, australia email: jdpierce@deakin.edu.au 2: information risk analyst abn amro level 27, 367 collins street, melbourne vic australia abstract in an environment where commercial software is continually patched to correct security flaws, penetration testing can provide organisations with a realistic assessment of their security posture. penetration testing uses the same principles as criminal hackers to penetrate corporate networks and thereby verify the presence of software vulnerabilities. network administrators can use the results of a penetration test to correct flaws and improve overall security. the use of hacking techniques, however, raises several ethical questions that centre on the integrity of the tester to maintain professional distance and uphold the profession. this paper discusses the ethics of penetration testing and presents our conceptual model and revised taxonomy. keywords: penetration testing, computer security and computer ethics. introduction the adage ‘knowledge is power’ can be used to illustrate the gap between information security professionals and ordinary end-users. to laymen, information and especially computer security is a clandestine unknown against which they feel powerless. bereft of security knowledge, end-users succumb to fear, uncertainty and doubt (fud). the unbridled pace of today’s business has underpinned an explosive growth in the adoption of information and communications technologies (icts). in an increasingly uncertain and competitive business environment, product life-cycles are shortened and their outputs pushed to market quicker in an effort to maximise profits margins: there are few industries immune to this cycle. software companies, for example, are pressured to release applications faster and this often results in haphazard software testing practices. in fact, the underlying philosophy of the rapid application development (rad) ‘phased development’ methodology centres on finishing core functionality quickly and then implementing other functionality (including security) and rigorous testing in subsequent software versions (dennis et al., 2002). the literature shows little convergence enumerating the volume of vulnerabilities routinely discovered every week but commercially the trend remains to be playing a game of ‘catch-up’ in patching vulnerable software. australasian journal of information systems volume 13 number 2 may 2006 194 in a predominantly networked society these defective icts are transmitting vast quantities of sensitive data across relatively unsecured communications media. moreover maturing technologies such as encryption can be used to make sensitive data useless to interceptors but it is essentially a patch in itself: the internet was designed to survive a nuclear assault during the cold war and not for the widespread commercial use it has adapted to today nor with security in mind. the question facing many network administrators then is ‘how secure is my network?’ assessing the security of a network can be achieved using controlled hacking techniques. penetration testing, as it is termed, can provide security assurances to network administrators. the practice tends toward the patching trend noted but by its very nature—hacking—raises several ethical concerns. this paper seeks to discuss such ethical concerns and present a conceptual model and taxonomy to show their relationships. we start by discussing the limited body of literature before presenting a categorised set of penetration testing ethics. the paper is then concluded with directions for future research. contemporary penetration testing in the literature ‘quality assurance and testing organizations are tasked with the broad objective of assuring that a software application fulfils its functional business requirements.’ (arkin et al., 2005, p.84) the major theme in the penetration testing literature tends toward describing how the tester should conduct their tests according to a plethora of differing methodologies and philosophies using a growing collection of labyrinthine automated tools. the commonly accepted definition of penetration testing is the ‘[sanctioned] illegitimate acquisition of legitimate authority.’ (geer and harthorne, 2002, p.1; logan and clarkson, 2005; thompson, 2005) geer and harthorne (2002) point out that penetration testing should be considered an art rather than a science. the distinction is based on the commonly accepted limitation that penetration testing cannot prove the absence of network vulnerabilities, only the presence of them: therefore a penetration test that fails to uncover any vulnerability is not necessarily a good penetration test result (arkin et al., 2005). whereas science relies on the disproving of null hypotheses, penetration testing can at most be a science of insecurity as opposed to a science of security (geer and harthorne, 2002). in this same vein geer and harthorne (2002) suggest that if a penetration test fails to uncover network vulnerabilities then it is more likely to create value for the client. the possibility for clients to misunderstand the so-called ‘science of insecurity’ is thus illustrated and an important question of ethics is uncovered by the bye: the chance of misrepresenting penetration testing and its potential to guarantee security. geer and harthorne (2002) go on (p.3) with a cynical view of penetration testing, predicting that it will evolve into more of a quality assurance regime using checklists rather than the art of discovering known and unknown vulnerabilities and providing realistic assessment of software security posture. to the contrary, there seems to be a movement in the literature toward separating security testing from software quality assurance. in the context of bug reports and quality assurance, however, arkin et al. (2005, p.85) suggest that [people] often use penetration testing as an excuse to declare victory. when a penetration test concentrates on finding and removing a small handful of bugs (and does so successfully), everyone looks good: the testers look smart for finding the problem, the builders look benevolent for acquiescing to the test, and the executives can check off the security box and get on with making money. unfortunately, penetration testing done without any basis in security risk analysis leads to this 195 australasian journal of information systems volume 13 number 2 may 2006 situation with alarming frequency. by analogy, imagine declaring victory by finding and removing only the first one or two bugs encountered during system testing! haphazard penetration testing is giving way to engineering approaches emerging from the literature and will wane when best-practice standards mature and the distinction between security professionals and hackers becomes even more prominent. for example thompson (2005) describes a novel approach to operational penetration testing where a threat model resembling a tree-like flowchart is developed. network vulnerabilities are then tested according to reusable and evolving threat models. further, beznosov and kruchten (2005, pp.49-50) describe modern tactics for assuring software quality: a fundamental practice in the assurance business is to keep developers and security evaluators “at arm’s length” from each other so that they do not affect each other’s ideas. since security assurance must be completely neutral and objective, its practitioners and the developers should not become too closely involved except during their information gathering sessions. this leads to developers often focusing on the functional development with a “tunnel vision” that becomes quite blind to security flaws. recently, universities have ventured toward offering security testing courses. while still in its infancy this branch of information management is evolving into a specialised profession that will soon require undergraduate qualification like mainstream computer science. in this instance, it would seem that, academia has caught wind of industrial certification bodies such as the international information systems security certification consortium (isc 2 ), the information systems audit and control association (isaca), the institute for security and open methodologies (isecom), cisco and microsoft who offer specialist security certification. (the business models are slightly different for academia and the cited industrial certification bodies that exact revenue from the continued subscriptions required to maintain certification.) nonetheless opportunity exists for academia and industry to collaborate on educating tomorrow’s security professionals, yet there are several problems to iron out. in early pedagogical case studies tikekar (2003) and logan and clarkson (2005) point out that until recently, students were not required to study computer ethics to graduate. security professionals who are not educated in computer ethics poses perhaps the fundamental question as logan and clarkson (2005, p.159) ask ‘what happens when universities, such as marshall university, west virginia house the state’s digital evidence lab for the state police, as well as student computer forensic training labs?’ without computer ethics curricula, graduates might be less informed as to what constitutes abuse and misuse of their skills. this scenario could extend to the possibility of alqaeda (and similar) recruits enrolling in such courses with the predisposed intention to later launch co-ordinated terrorist attacks on critical information infrastructure. indeed it has been revealed that al-qaeda used strong encryption algorithms (pgp) to hide their communications (oz, 2006). penetration testing is an evolving practice and the small but growing body of literature shows that there are many arising issues that need to be addressed before it can mature fully. the literature is rich with methodologies and frameworks (see pierce et al., 2005a; pierce et al., 2004), but lacks longitudinal studies that prove their merits. for example, the literature lacks case studies that demonstrate how penetration testing fits into business and military continuity planning. furthermore, although equivalent international standards such as iso 17799-2000 are in place a formal australian standard for penetration testing as yet does not exist. the following section demonstrates the ethical concerns that arise from penetration testing and shows how they converge on six major themes. australasian journal of information systems volume 13 number 2 may 2006 196 the ethics of penetration testing the open source security testing methodology manual (osstmm) (herzog, 2003) outlines its ‘rules of engagement’ which are essentially a set of rules designed to restrict unethical penetration testing practices. while the osstmm does not discuss the ethics of penetration testing explicitly, its scope is limited to outlining a methodology for penetration testing. we discuss the ethics of penetration testing in this section using the rules of engagement and those identified in the literature review as reference points. the six major themes of penetration testing ethics as identified in the literature centre on integrity, which branches out to serving and protecting the client and preserving the security profession. these objectives are met by avoiding conflicts of interest, the provision of false positives and false negatives, and finally legally binding testers to their ethical obligations in the contract. by analogy, let us imagine a profession-wide vision and mission statement: we will strive to provide security assurance consultation with professional integrity. fulfilling the mission statement would be achieved by adopting two important and virtuous strategies: upholding the profession, and serving and protecting the client. policies for implementing the strategies include: avoiding conflicts of interest, avoiding the provision of false positives and false negatives, and legally binding the tester’s and the client’s ethics. we discuss the ethical considerations in each category below. serve and protect the client and uphold the security profession testing should not be performed without the expressed written permission of the client. whereas in the hacking community attacks occur non-consensually, contractual arrangements must be in place to provide a degree of separation between hackers and security professionals. there is general consensus in the literature that testers should not rely solely on automated tools but also on their skills. notwithstanding the tester should be well-versed in computer security and know how their tool arsenal works. tools should be tested themselves in an isolated laboratory prior to being used in production. the use of past client’s data, with or without permission, should not be used to promote the services of the tester. while it may provide a false positive to the potential client, it could also damage the reputation of the implicated organisation. the tester should notify the client at the first instance of discovering highly vulnerable flaws as in the case of those that endanger human life. the notification should contain appropriate countermeasures to correct the flaw and minimise dangers to human life and the organisation in general. the principle objective of penetration testing is to test security measures in a network: there is little point to testing systems known to be highly vulnerable. testing should not commence until appropriate security has been applied to the system. the results of social engineering tests should be delivered only in summarised and statistical format so as not to implicate individuals. the emphasis is on protecting the client and insulating unknowing 197 australasian journal of information systems volume 13 number 2 may 2006 employees that might be subject to subsequent embarrassment or termination of employment as a result of the test. the delivery of the report should be preceded by a notice of delivery. the client, upon receiving the report, should acknowledge that they are in receipt of the report. the curtesy underscores the importance of client confidentiality. the report will contain an appropriate level of detail of the tests performed, the results and the steps the client should take to improve overall network security. false positives and false negatives and conflicts of interest penetration tests that fail to uncover vulnerabilities should not be passed up as free services (geer and harthorne, 2002; herzog, 2003). this represents false positives by misrepresenting the penetration testing practice as an assurance of the absence of vulnerabilities. the tester is ethically bound to serve the customer. this holds true even if it is in the best interest of the customer to engage a different testing company. in that event, the tester should not recommend any particular company so as to avoid the possibility of a conflict of interest or the perception of one. the promoting of public hacking or trespass contests for security assurance is unethical because it implies a false security guarantee. contests also draw unnecessary attention to the client network as a perception of a ‘fair game’ target will endure in the hacking community far longer than the expiration of the testing contract. when new vulnerabilities ensue the client’s network may be the target of continuing non-solicited attacks. as alluded to in the previous sub-section, clients should behave in a manner that does not encroach on the tester or interfere with a test in a manner that may alter its outcome. this includes deploying additional security during a test. as testing provides a snapshot in time of the security posture of a network, changing the security environment could lead to false positives or false negatives. further, the client should notify only key internal personnel of the penetration test. the extent to which people are kept inside the circle is at the discretion of the client but it must be stressed that widespread knowledge of the test will alter behaviour and affect the outcome of the test thus promoting false positives or false negatives. if white-box or in-house testing is requested, the tester should first perform black-box testing offsite. this concern draws attention to the use of internal security auditors that could develop tunnel vision (as insiders know the target network very well) and lead to false positives and false negatives. statistically (auscert, 2005; csi/fbi, 2005), however, system compromise originates in greater frequencies from within the organisation than outside the organisation. in this light the use of internal security auditors presents some merit. legally binding ethics and other considerations the tester should work non-disclosure and limited liability clauses into the contract. the use of these clauses in the contract legally binds the tester to his ethical obligations. non-disclosure is common practice, but the tester should also assume limited liability generally not exceeding the cost of the test for inadvertent damages incurred by the client as a result of negligent testing or malpractice. therefore, it is in the best interests of the tester to have sharpened their skill base to provide the said degree of separation between hackers, script kiddies and security professionals. australasian journal of information systems volume 13 number 2 may 2006 198 ethically, the contract should scope the tests and indicate emergency contacts as well as the ip addresses from which the tests are originating. additionally, the contract should specify failsafe procedures such as recovering from dos attacks. it has been suggested that using fud to sell penetration testing services is unethical. the osstmm explicates that crime facts and figures should not be used to promote security testing. academics, however, routinely use crime statistics – such as those found in the recent auscert (2005) and csi/fbi (2005) computer security surveys – to justify the problems with computer security, necessitate new research, and sell security courses at the university and practitioner levels. the question of using statistics as an ethical concern should be met with a balanced view of the need to educate end-users in security versus the instilment of fud. the ethical concerns presented in the paper centre on five interrelating themes: serving and protecting the client; upholding the security profession; conflicts of interest; false positives and false negatives; and, legally binding the tester to their professional ethics. a common factor, integrity, can be seen to tie these categories together as in the following figure. the tester’s integrity should compel them to serve and protect the client while behaving ethically to preserve the integrity of the profession. figure 1 illustrates our conceptual model of penetration testing ethics and how they correlate with a central theme of the integrity of penetration testers. figure 8: a conceptual model of penetration testing ethics critics of an earlier version of this paper (pierce et al. 2005b) rightly corrected that figure 1 was not, definitively, a taxonomy: it shows no specific categorisation of penetration testing ethics. of interest in figure 1, however, is how we modelled all the ethical constructs to be related to integrity: we maintain this assertion here too. although figure 1 shows that avoiding conflicts of interest, avoiding false positives and false negatives and legally binding ethics are prerequisites for integrity, and that integrity is a prerequisite for upholding the profession and for serving and protecting the client, we having modelled the ethics in a slightly different way in the following figure. our revised taxonomy is a bottom-up hierarchical representation of the relationships between ethical categories as discussed in the paper. it shows upholding the profession and serving and protecting the client built upon the foundation of integrity. additionally, avoiding conflicts of interest, avoiding false positives and false negatives and legally binding ethics are built upon a foundation of serving and protecting the client. avoid false positives and false negatives integrity binding ethics legally avoid conflicts of interest serve and protect the customer uphold the security profession 199 australasian journal of information systems volume 13 number 2 may 2006 avoid conflicts of interest avoid false positives and false negatives legally bind ethics uphold the profession serve and protect the client integrity figure 2: a taxonomy of penetration testing ethics the penetration tester should act with integrity at all times. in his endeavours, as noted, the penetration tester should strive to maintain a degree of separation between the criminal hacker and the security professional; thereby to uphold the profession. the penetration tester can be seen to be acting with integrity if they can be seen to be upholding the profession. to further clarify figure 2 suppose the penetration tester builds non-disclosure and limited liability into the testing contract, they can ergo be seen binding their ethics legally and thereby ethically serving and protecting the client. to ethically serve and protect the client is to act with integrity. integrity is therefore the synergising foundation from which the professional ethics of penetration testing extend. if the penetration tester refuses to engage with the criminal hacking fraternity they can be seen to be using their skills for commissioned tests only and therefore upholding the profession. conclusion we have seen that the business world is experiencing unparalleled growth due to its unbridled speed. no organisation is immune to the rapid changes in technological development that can be seen to fuel a cycle of business needs driving technological development enabling business needs. the pressures facing the business environment are also felt by software companies that resultantly cull application security testing to release the product quicker and then patch later. the problem lies in that vulnerable software is introduced to a hostile commercial environment. penetration testing provides organisations with a means of assessing their security stance at a given moment in time. testers and clients must behave ethically: clients so as not to alter test outcomes and testers so as to separate them from the hacking community. the ethics of penetration testing centre on integrity; serve and protect the client and uphold the security profession by behaving ethically. the small body of literature tends toward presenting methodologies and frameworks for conducting penetration tests, but seldom integrates penetration testing into an overall business model. there seems to be confusion as to how organisations can best gauge value from the services of a penetration test. risk analysis is a sister topic that is gaining incredible momentum in the literature and should also be integrated with penetration testing to produce an overall model of organisational security testing. looking to the future, the authors will investigate and propose methods of integrating penetration testing with the better established risk analysis discipline. we will also look at ways of providing pseudo-dynamic security assurance using automated approaches. australasian journal of information systems volume 13 number 2 may 2006 200 references arkin, b., stender, s. and mcgraw, g. (2005) ‘software penetration testing’, ieee security and privacy, january/february 2005, pp.84-7. australian computer emergency response team (auscert) (2005) ‘2005 australian computer crime and security survey’ vol. 1, auscert, brisbane, australia. beznosov, k. and kruchten, p. (2005) ‘towards agile security assurance’, proceedings of the 2004 workshop on new security paradigms, nova scotia, canada. computer security institute / federal bureau of investigation (csi/fbi) (2005) ‘tenth annual 2005 computer crime and security survey’, vol. 1, csi, san francisco, usa. dennis, a., wixom, b.h., and tegarden, d. (2002) ‘systems analysis and design: an objectoriented approach with uml’, wiley, new york, ny, usa. geer, d. and harthorne, j. (2002) ‘penetration testing: a duet’, proceedings of the 18 th annual computer security applications conference (acsac’02). herzog, p. (2003) ‘open source security testing methodology manual’, isecom, usa. international standards organization (2005) ‘iso/iec 17799:2005 – information technology – security techniques – code of practice for information security management’, iso, switzerland. logan, p.y. and clarkson, a. (2005) ‘teaching students to hack: curriculum issues in information security’, proceedings of the 36 th sigcse technical symposium on computer science education, st. louis, missouri, usa. oz, e. (2006) ‘management information systems’, 5 th ed, thomson, boston, ma, usa. pierce, j.d., warren, m.j. and corray, x.j. (2005a) ‘in pursuit of a standard penetration testing methodology’, journal of information warfare, 4(2), pp.26-39. pierce, j.d., jones, a.g. and warren, m.j. (2005b) ‘a taxonomy of penetration testing ethics’ in proceedings of the 4 th international australian institute of computer ethics (aice) conference, september 26 th 2005, waurn ponds, victoria. pierce, j.d., warren, m.j. and corray, x.j. (2004) ‘a critical review of penetration testing methodologies’, in proceedings of the 5 th australian information warfare and security conference, 25-26 th november 2004, fremantle, west australia. thompson, h.h. (2005) ‘application penetration testing’ ieee security and privacy, january/february 2005, pp.66-9. ajis vol.9, no. 2 may 2002 consensus building through systems thinking the case of policy and planning in healthcare kambiz e. maani school of business and economics the university of auckland email: k.maani@auckiand.ac.nz abstract this paper discusses a methodology and a case study using qualitative system dynamics to create consensus, team learning and shared vision in a public organisation. the case involves determining planning priorities for a division of the ministry of health in new zealand, leading to the creation of a business plan. the methodology involves systems thinking using group model building (gmb) a three-step process starting with structured brainstorming using the partial kj (jiro kawakita) technique to identify priority areas and then clustering them into 'affinity' groups. next, the priority clusters are used by the participants to construct causal loop diagrams representing 'systems of priorities'(in contrast with list of priorities). finally, through a group process, leverage points or key priorities are identified as the basis for a business plan. while the above process was well agreed upon by the participants at the outset, strong group resistance was encountered when managers attempted to reduce the number of priority areas. systems thinking approach was used to create consensus and the commitment to the outcome. the gmb approach offers significant promise in using qualitative system dynamics with non-systems experts. the methodology can be applied to change management initiative and complex decisions such as restructuring, reengineering, and supply chain design. the expected outcomes are greater commitment and shared vision. key words: group model building, team learning, consensus building, qualitative system dynamics, change management, and planning introduction the case study reported here is based on a ministry of health (moh) project involving operational and business planning for one of its divisions. the division employs staff with diverse professional and policy backgrounds and varying periods of tenure with the division. the main question of the planning exercise was the selection of key priority areas (6-7) to focus the limited resources and efforts of the division. while the project brief required the use of systems thinking approach, it also specified that this had to be conducted in an indirect and implicit manner. given the short time periods allotted for planning workshops and the lack of familiarity of the participants with systems thinking modelling, this posed a significant facilitation challenge. therefore, in consultation with division manager, a series of workshops were designed to accommodate division's requirements. this paper describes the policy and priority setting process for this division and highlights the use of systems thinking modelling to transform contentious issues into mutual agreement and commitment. before the process is described, a brief review of relevant literature follows. literature review lack of commitment to decisions is often a key factor in resistance to policy change and organisational initiatives. as the experience has shown, change management projects are fraught with stories of resistance to change, sapping morale and adverse consequences in organisations. the likelihood and magnitude of the adverse consequence are greatest where divergent groups, ie, different divisions, department or units are involved. 'messy' problems are defined as situations in which there are large differences of opinion about the problem or even on the question of whether there is a problem (ackoff, 1974; 1979). messy situations make it difficult for a management team to reach agreement. system dynamics (sd) modelling with groups known as group model building (gmb) is a powerful tool for dealing with these. sd and gmb are especially effective in dealing with semi-structured and ill-structured decision situations. gmb offers an opportunity to align and share piecemeal mental models (huz et al. 1997) and create the possibility of assimilating and integrating partial mental models into a holistic system description (vennix 1995; 1996). gmb and sd can help uncover 'illusions' that may occur due to the fact that the definition of a problem may be a socially constructed phenomenon that has not been put to test. vennix (1999) identifies two sources of messy problems, namely, the individual, and the group/team that give rise to the existence of messy situations. limited information processing capacity (vennix 1990), and perceptions and reality constructions (schutz 1962) are the main contributors to the individual sources of messy problems. increasing the information processing capacity not only affects the dynamics of a system but also it's causal feedback structure (dorner 1980). one of the implications of this individual source on gmb is that both qualitative and quantitative modelling are important (coyle 1999; vennix et al. 1993). 84 ajis vol. 9, no. 2 may 2002 the group sources of messy problems relate to deficiencies in group interaction, and the self-fulfilling nature of reality construction in groups. people not only construct reality in their minds, but their behaviour also causes their mental model to become reality in their environment. deficiencies in group interaction are in the form of mixing up of cognitive tasks (rogers & roethlisberger 1988), lack of critical investigation (janis & mann 1977), and the way team members communicate (argyris, 1990). the above points highlight the need for a group facilitator in the gmb process. a facilitator is a person who acts as a 'role model' for the group, a person who can avoid the common deficiencies in group interaction, which negatively affect the quality of the decision (vennix et al. 1993). systems thinking interventions will be much more effective if sd and mb tools are skilfully combined with adequate facilitation (senge, 1990). critical characteristics of an effective facilitator include a primary concern with procedure and process and only indirectly with the content, i.e. with the how rather than what. both attitudes and skills are important characteristics of the 'ideal' facilitator (vennix 1996,1999). some of the facilitation attitudes are a helping and inquiring attitude (asks questions rather than provide answers), which, at the same time is neutral with regard to the content of discussion. a facilitator should be able to foster reflection and learning in a team by discouraging defensive communication, while maintaining his/her own integrity and authenticity. other important facilitation skills are a thorough knowledge of sd and mb as well as group process techniques. the latter requires an awareness of the existence of various cognitive tasks that a group can encounter. conflict handling and efficient two-way communication are other important facilitation skills (vennix 1999). group model building need not lead to model quantification and simulation. vennix (1999) argues that, due to existence of persistent cognitive and social barriers to learning (argyris 1990; 1994; senge 1990), simulations are not ultimate solutions. in fact, quantification will either add to understanding the issue or will be dangerously misleading (wolstenholme 1992; 1999; coyle 1999). active construction of the model is just as important as playing it. thus, it is important to recognise that in a number of cases it is not always useful or even desirable to go through the whole model-building cycle. in summary, when conducting interventions through gmb, one should be aware that cognitive limitations, differences in perceptions (leading to multiple realities) and ineffective communication patterns (which block productive discussion of these multiple realities) play a key role in the success of the intervention. at the educational level, this requires teaching facilitation skills and group dynamics in sd programmes to engender appropriate attitudes and skills for effective gmb facilitation (haslett et al. 1999). the following sections describe a case study using group model building and facilitation in a real life situation. issues identification and methodology the first step in the division's priority and planning process was to establish a common ground and create a safe space for group dialogue. this required a shared understanding of what the real and perceived barriers to progress were. the key issues arose out of an initial round of "checking ins" whereby the participants shared their thoughts and feelings about various issues. the methodology used in this step was a modified kj or affinity procedure. the methodology is described in systems thinking and modelling (maani and cavana, 2000, p.34): the affinity diagrams method, also known as kj, is a process of mapping creative group thoughts and ideas. the aim of this process is to allow new thought patterns and breakthroughs emerge naturally from a large pool of 'raw' ideas. the affinity/kj technique has three main applications: • formulation of vague problems • synthesis of non-numerical data (also called semantic or language data) • teamwork and consensus building. there are 7 steps in constructing an affinity diagram.1 assemble the right team phrase issues to be considered generate and record ideas (each on a separate card, or post-it notes) randomly lay out completed cards sort cards into related groupings create header (or label) cards construct the affinity diagram by placing the headers and sub-headers (on top of the page) with all the cards beneath them. this should form a column for each cluster. using the kj methodology the following question was posed for brainstorming. 85 ajis vol.9, no. 2 may 2002 "what is preventing us from making faster progress?" a total of fifty 'raw' statements were generated (five per participant). the participants then clustered these statements into nineteen headings or key issues as shown in figure 1. this exercise served as a lead or mental preparation for the next step, namely, identification of priority areas. the question of priority setting in organisations is often contentious as it implies trade-offs need to be made, hence creating winners and losers. often, this results in open challenges and, even worst, silent resentments that lead to loss of commitment by individuals and groups. it is therefore critical that the issue of priority setting is managed in a systemic (holistic) manner. priority selection in the next step, a second workshop was devoted to the following question. the question was phrased as follows: what are the priorities in health policy in terms of where the division should be placing its greatest efforts? again the group used the kj methodology to identify 'raw' priorities by each individual participant and then cluster these in fewer priority areas. the kj method is very beneficial in this process. not only it avoids awkward disagreements and contradictions, the method converges very quickly yielding visible group consensus. furthermore, the participants often see the process as 'fun', adding another impetus towards teambuilding. again as the group was diverse in terms of organisational hierarchies and professional backgrounds, it was important that no priorities were missed or reduced early on and prematurely. the kj process thus ensured that all contributions were included. this process resulted in 42 'raw' priority statements, which were clustered into 19 priority areas as shown in figure 2. as the management had desired to have only 6-7 priority areas, this seemed to be too large for any practical purposes. therefore, it felt necessary to reduce the initial set of 19 to 6-7 areas. in order to proceed, a set of criteria was developed by the participants for priority selection. the group later refined these criteria. the final criteria list is shown in table 1 below. table 1 criteria for priority selection 1. realistic can we marshal the resources? 2. is it reasonable and compatible with the government's direction/ political environment? 3. impact direct impact on maori health 4. quick visible results within a few weeks or months (maximum 6 months) 5. alignment with division's mission and other stakeholders 6. fundamental cause cause not symptom focus 7. existing initiatives capitalises on existing initiatives 8. planning horizon short, medium or long in order to minimise subjectivity, it was deemed pertinent to apply the above criteria objectively. it was agreed by the group to use a priority matrix to rank order the priorities. thus, a scaled ordinal ranking of 1-5 was adopted where 1 indicated lowest priority and 5 denoted highest priority. it was also decided that, initially, relative weights should not assigned to the criteria. the group then proceeded to rank each of the 19 priority areas against the seven criteria stated above. the outcome of this process, or the priority matrix, is shown in table 2. according to the priority matrix, a clear set of rank ordered priories emerged. as all the participants had agreed and freely contributed the process to this point, it was expected that top priority areas would be selected from the priority matrix. contrary to this expectation, most participants showed strong resistance to this outcome! this was both surprising and enlightening. it is important to bear in mind that in groups where diversity of tasks and purposes are present this resistance exists whether or not it is voiced. in this case no individual participant was prepared to 'let go' of his or her area of work. of course, at this stage it was possible for the manager to intervene and use her authority to 'force' or coerce the opposing 'camps' into acceptance. but it soon became apparent that any 'reduction' of priority areas would be counterproductive and damaging to the group's integrity and unity. 86 ajis vol.9, no. 2 may 2002 from priority selection to a system of priorities to break this impasse, it was agreed to adopt all priority areas. however, this was an impracticable solution and contrary to management's initial objective. at this stage, in order to resolve this apparent conflict we offered to use the systems thinking approach. the underlying philosophy of systems thinking is the primacy of the whole and relationships rather than the individual parts. it was explained that this concept was contrary to treating priorities as independent and isolated elements, as they were originally perceived. rather, the group needed to view them as part of a priority system, where all priority areas were regarded as indispensable elements of the system (i.e., the business plan). in such a system, while all elements are important for the working of the whole, relative importance and 'timing' of the parts are nevertheless recognised and acknowledged. this is done through the identification of 'leverage' points in systems thinking. thus, the group identified areas that were deemed to have a fundamental (or cause) effect on the whole as leverage areas. in the implementation phase, the division started with the 'leverage areas' first and as a team, thereby differentiating the priority areas by timing rather than perceived importance. these interventions in turn would introduce a positive chain effect in the system as a whole. this approach necessitated adopting a new mental model, which transformed the question of 'priority selection' from a reductionistic perspective to a systemic mind set. having accepted this philosophy and approach, the group converted the priority matrix into several plausible causal loop diagrams (cld) of priorities. the group then chose one of the clds as their preferred one. this cld is shown in figure 3. it is interesting to note that the selection of the preferred cld, unlike the previous priority selection case, was rather quick and unanimous. in this cld (figure 3), variables identified by 'l' indicate leverage points in the systems, where earlier attention and focus will be devoted. conclusion contentious organisational issues, such as priority selection, program, policy and direction changes often create resistance to change and unspoken resentment. the conventional change management techniques are unable to cope with the dynamic complexity and unintended consequences of such actions. this paper illustrates the application of systems thinking and group model building in a division of the ministry of health in new zealand. the driving question of the planning exercise was to determine a reduced set of priorities for the creation of the division's business plan. the methodology involved a three-step process: (1) brainstorming to identify key issues and priority areas, (2) developing a priority matrix, and (3) creating causal loop diagrams representing 'systems of priorities' (in contrast with list of priorities). using the priority clds leverage points or key priority areas were identified as input to the business plan. the paper illustrates how systems thinking methodology could create consensus, team learning and shared vision in today's complex organisations. this approach helped transform 'piecemeal mental models' to shared understanding, acceptance, and commitment to resolving issues and the challenges facing the organisation. group model building offers significant promise in using qualitative system dynamics with novice users. the methodology can be applied to change management initiatives and complex organizational decisions such as restructuring, reengineering, and supply chain design. the expected outcomes are greater commitment, mutual acceptance and shared vision. references ackoff ra. (1974) redesigning the future: a systems approach to societal problems, new york: wiley. ackoff ra. (1979) the future f operational research is past. journal of the operational research society vol30,no2,pp93-104. argyris c. (1990) overcoming organizational defenses, facilitating organizational learning, boston: allyn and bacon. argyris c. (1994) good communication that blocks learning: a theory of action perspective, reading, ma: addison-wesley. coyle g. (1999) qualitative modelling in system dynamics or what are the wise limits of quantification? keynote address to the conference of the system dynamics society, wellington, new zealand dorner d. (1980) on the difficulties people have in dealing with complexity, simulation and games vol 11, nol,pp87-106. 87 ajis vol. 9, no. 2 may 2002 haslett t, barton j, sarah r. (1999) the use of group modelling techniques as a teaching tool, in proceedings of the 1999 international conference of the system dynamics society, wellington, new zealand. huz s, andersen df, richardson gp, boothroyd r. (1997) a framework for evaluating systems thinking interventions: an experimental approach to mental health system change, system dynamics review, vol 13, no 2, pp 149-169. janis il & mann l. (1977) decision making: a psychological analysis of conflict, choice and commitment, new york: the free press. maani k & cavana r. (2000) systems thinking and modelling understanding change and complexity, new zealand: prentice hall. rogers cr, roethlisberger fj. (1988) barriers and gateways to communication, in: john j. gabarro, people: managing your most important asset, harvard business review, special edition of articles, pp 19-25. schutz a. (1962) collected papers i: the problem of social reality, the hague: martinus nijhoff. senge p, (1990). the fifth discipline: the art and practice of the learning organization, new york: doubleday:. vennix jam. (1990) mental models and computer models: design and evaluation of a computer-based learning environment for policy making, ph.d. dissertation, university of nijmegen, netherlands vennix jam, scheper w, willems r. (1993) group model building: what does the client think of it? in the role of strategic modelling in international competitiveness, proceedings of the 1993 international system dynamics conference, sepada e., machuca j. (eds). cancun: mexico; pp 534-543. vennix jam. (1995) building consensus in strategic decision-making: system dynamics as a support system, group decision and negotiation, vol 4, no 4, pp 335-355. vennix jam. (1996) group model-building: facilitating team learning using system dynamics chapter 5, chichester: wiley. vennix jam. (1999) group model building, system dynamics review vol 15, no 4, pp 379-401. wolstenholme e.f. (1992) the definition and application of a step wise approach to model conceptualisation and analysis, european journal of operational research, vol 59, pp 123-136. wolstenholme e.f. (1999) qualitative vs quantitative modelling: the evolving balance, journal of the operational research society, vol 50, pp 422-428. 88 ajis vol.9, no.2 figure 1: issues clusters may 2002 other health determinants maori development maori health barriers with internal & external relationships team work inability to capture team strengths & to acknowledge & address weaknesses lack of staff & stabilisation leadership lack of direction, leadership & communication provide the leadership compass not leading many of our own projects public health ability to influence lack of resource to support govt. 's priority for maori health out of decision making loop get into decision making loop focus on making a difference make waves desire to share power reluctance to share power those in power govt. process constraints maori health or development maori health status cant be addressed by health alone no consensus re maori health gain vs maori development are we captured by bureaucracy? public or personal health? too focused on personal health intervention personal vs public health? turbulent environment turbulence change fatigue lack of follow through personal health vision of maori health lack of clear definition of maori health status mhb don't really believe we can do it identify a common goal being too passive & not analytical enough time pressure work pressure limits ability for staff development lack of time to develop as a team too much to deal with clear priorities not sure what to do lack of focus on priorities lack of shared focus lack of vision in direction ad hoc work priorities unable to prioritise work (lack of guidance) moh-taking a problem (solving) approach most of the time not enough attention & time given to big issues or questions too reactive, not enough proactive lack of effective collaboration with key groups relationships with others health is part of other things responsibility of health status is beyond the health sector failure to identify (maori) health statushealth sector responsibility failure to capture the complexities of the health sector (first up) 89 ajis vol.9, no. 2 figure 2 priority areas may 2002 prevention public health prevention strategies project-public health approach information to better manage care (prevention & treatment) role models whanau capacity health information for consumers incentives for hapu, whanau capacity of all providers treaty of waitangi raise ideas/model for parallel maori structures rural health maori rural health needs accountability cycle accountability arrangement documents incentives for funders evaluate monitoring & evaluation tools incentives on providers integrated care development disease management diabetes social economical & cultural economic/social determinants highlight how health dependant on education, employment, housing etc identify structural factors causing increase of maori in poverty increasing the maori health identify funding allocation clarify maori healt model/pbiloso measures maori health 90 ajis vol. 9, no. 2 figure 2 priority areas (cont'd) may 2002 maori provider development advocate funding (at least 5 years) for maori health providers maori provider workforce development high priority diseases biggest killers most common morbidities diabetes high priority health issues affecting maori maori workforce capacity maori workforce nzmc rep-moh establish maori advisors in each branch of moh health service-access barriers, $, cultural, geographical -research, service utilisation by income levels child & youth youth health maori youth, mental health, suicide profile child health the future nga tamariki tamariki ora focus yr 1 capacity & capability of moh to respond tpk review of moh communication & relationship & collaboration maintaining communication with providers support maori providers manage relationship with hfa on maori health regular moh publications health sector developments collaborate with hfa, tpk, treasury, social services etc... stakeholder collaboration (effective) 91 a.tts vnl. 0 nn. t. table 2: priority matrix inn? priority area <\ccouma witty cvcle developing maori models evaluate monitoring & evaluation tools rural health maori provider develodment maori workforce opacity communication, relationship & collaboration increase maori hearth putea disease mngnt high priority diseases incentive on providers capacity & capability of moh to respond social. cultural & economic child & youth access barriers « ha nau capacity treaty of wartangi capacity of all providers prevention, public hearth realistic s 4 4 5 5 4 5 $ 2 2 3 3 3 4 2 3 3 3 4 impact on maori health 4 3 4 4 4 4 3 3 5 5 2 2 4 5 5 3 4 3 4 quick visible results 5 3 $ 2 2 2 5 1 2 2 2 3 1 4 2 2 3 2 3 alignment 5 5 5 3 5 5 5 5 5 5 2 4 5 5 5 5 s 3 4 fundamental cause 2 5 2 3 3 3 2 4 3 4 3 2 5 5 4 5 5 3 5 capitalise on other initiatives s 4 4 4 s 5 5 4 3 3 3 3 5 4 3 3 3 3 4 planning time s s s i s.m.l s.m.l score 26 24 22 19 24 23 25 20 20 21 15 17 23 27 21 21 23 17 24 0 scale 1=low5=hlgh l=no5=yes s=shorttbrm m=medium term l=long term 92 ajis vol. 9, no. 2 may 2002 figure 3 priorities cld accountability cycle maori development treaty of waitangi (l) developing maori models (l) impact on maori health maori health putea communication, collaboration & relationships (l) moh capacity to respond (l) the engine r evaluate & monitoring (l) social, cultural, economic 1. seven management tools, goal/qpc, 1991 93 microsoft word ajis10-2finala.doc ajis vol. 10 no. 2 may 2003 60 information systems development contracts: an exploratory study of australian legal practitioners’ and information systems professionals’ perceptions phil joyce school of information technology swinburne university melbourne, victoria, australia email: philj@it.swin.edu.au ross smith faculty of business and law deakin university geelong, victoria, australia email: smithr@deakin.edu.au kathryn hodges telstra pty ltd victoria, australia email: kathryn.hodges@team.telstra.com.au abstract is professionals are primarily concerned with the development of a project to meet the goals of a client. unfortunately, is professional do not always pay close attention to the contract or to its management and are often unaware of the legal implications of the contracts to which they are assenting. legal advice can be sought during pre-contractual processes to help ensure that the contract meets the expectations of all the parties. if the project does not meet stakeholder’s expectations, thus causing a contract to come into dispute, lawyers may be called in to review the contract and to seek a settlement. this paper reports an exploratory study of the differences in perceptions of is development contracts that exist between is professionals and legal practitioners. keywords: is contracts, is professionals introduction ed yourdon, respected commentator on information systems (is) and information technology (it) issues, has recently observed that “lawyers and courtrooms are probably the last thing that your average it manager would think about during a typical systems development project” (yourdon 2000). the is professional’s primary focus is the development of products that meet a client’s needs. in order to achieve this the is professional goes through a process which includes the definition of requirements, the use of design principles and implementation guidelines to build the product and a review of the final product against the requirements. a step in this process for the is professional is the preparation and development of the contract. is professionals often view the contract as part of the normal processes that have to be gone through when developing a product. often, however, the is manager is unaware of the legal consequences of assenting to a contract. in contrast, the legal practitioner sees contract development as a crucial step in building relationships. legal practitioners focus on reducing risks in the contract, including ensuring the clarity of requirements’ specifications and negotiating aspects of liability. unfortunately, the legal profession is often brought in to review the contract only when there exists a significant difference in opinion of the product to be provided. resolution of the disputation will require intervention by legal professionals to provide a remedy for the situation. when examining an is development contract in dispute a judge will take into account a number of factors. first, are there precedents that have been set in previous cases? second, is there contract law that applies to the is development contract? within contract law both express terms and implied terms will be examined. against the express and implied terms of the is development contract a judge will need to consider the professional standards that should have been met by an is professional and the qualifications that a professional is expected to hold. software development practices will also be examined. if either the express or implied terms have been breached then damages could be awarded. in summary then, clearly at the heart of yourdon’s observation is a generalisation, but nevertheless it is a concern that often the perceptions that is/it professionals hold of the is development contract may be different from those of legal professionals. in response, in this paper we address the question: are there differences in the perceptions held by legal practitioners and is professionals of is development contracts? specifically, in sections 2 and 3 we explore the extant literature, which provides the context for a subsequent semi-structured interview program. in ajis vol. 10 no. 2 may 2003 61 section 2 we review relevant literature related to codes of ethics and professional standards, software development process quality standards, and the process of developing an is development contract, its structure and issues surrounding contract management. in section 3 we briefly highlight some of the relevant aspects of extant australian contract law. in sections 4 and 5 we then report an exploratory program of semi-structured interviews with a small cohort of select practitioners, probing their perceptions of the process of developing, and the content of, is development contracts. finally, in section 6, some brief final comments are included, addressing the scope of research to date and the way forward for research of this type. is professionals and is contracts is professionals are the people who enter into is development contracts, usually to provide a piece of software for a client. in developing this software it is expected that they follow professional standards, both in regard to their ethical conduct and professional behaviour, as well as appropriate software implementation standards. codes of ethics and professional standards the professional standards used in developing software attempt to reflect wider community expectations. the wider community within which developers operate needs to have confidence in the is profession, that they will do their job in an ethical manner taking into consideration the community’s interests, and that they possess relevant skills and qualifications. if the community were to lose faith in the is profession, government may feel compelled to step in and legislate. it might be argued that it is not in the is profession’s best interests that the government might so legislate, so restricting their practice, their flexibility, and in consequence increase costs due to expenses incurred by compliance. it may be argued that the is development industry needs to show that it is capable of self-regulation. in response, a number of professional associations have developed codes of ethics and codes of professional conduct (anderson et al. 1993). the is industry by its nature is multi-disciplinary and therefore attracts employees from a wide range of backgrounds. due to the dramatic growth in is many people have gained employment within the is industry and have become, by definition, “is professionals”. a number of these people have not undertaken formal study in the is discipline, and many have not been formally exposed to the ethical considerations of professional practice. moreover, there is a significant difference between being made aware of the ethical issues and becoming a responsible ethical professional (martin and welt 1999). gotterbarn (1999) suggests that members of the computing profession have to take a stand to ensure it is clear that it is no longer acceptable to do inferior work or to do work that damages the computing profession’s basic values. the ieeecs and the acm have identified that software engineering should be classified as a profession. as a profession, members are expected to act ethically and competently, and the professional society is expected to support their ethical behaviour and to hold them accountable when they deviate from endorsed codes and behaviours (rosenberg 1998). if ethics are to be part of is development then wood-harper et al. (1999) prescribes two principles to which one should adhere: • principle 1 ethical reasoning should be conducted throughout the life of an information system, including inception, testing, distribution, modification and termination. • principle 2 the design of every is should include the design of its ethical use, the design of its ethical distribution, the design of its ethical risk, and the methods of justifying ethical criteria. ethics are behavioural guidelines that parallel certain legal standards. ethical standards have long existed in the computer field and have been incorporated in various agreements and contracts for many years (ulrich 2000). ulrich suggests that by incorporating ethical standards into is contracts, corporations, vendors and individuals have specific guidelines that can be cited in a court of law should a dispute arise. quality standards quality standards involved in an is development contract include not only the professional practice standards that should be followed, but also include the standards that should be followed when developing the product. to prove that a contract has been met, the software needs to be developed not only to the specifications but also to an expected level of quality. it can be argued that if an expected level of quality is to be achieved, one should require a developer to have some kind of quality management process in place. quality standards serve two purposes: they provide guidance to organizations installing quality process systems; and they are the basis for the certification of quality systems (van der pijl et al. 1997). the guidance that the quality standards provide seeks to help the is profession to meet the quality expectations that are required within a contract. the certification that can be obtained provides reassurance and recognition to the ajis vol. 10 no. 2 may 2003 62 wider community. three prominent software quality process widely recognised by the is profession are: iso 9000 series accreditation, the capability maturity model (cmm) and spice (see spice 1999). there is no clear consensus on the relative value of these three quality process standard assessment mechanisms. indeed, the more organisations use quality standards the more it is possible that parties participating in software contracts will use different quality standards (van der pijl et al. 1997). further, one might expect that standards will keep being developed to meet different needs. the problem with this is that the public will not know which standard should be used and confusion may well further grow. such confusion may well surface in the courts. is development contracts an is development contract is designed as a legal instrument that the client and vendor sign to assent to the development of a product and/or service. central to the contract are the terms and conditions it contains. these terms and conditions address who is to develop the product and/or service and the solution that is to be provided. the terms and conditions within the contract have implications for the general public, the is industry and the legal profession. the public wants to ensure that the system developed will not do them harm. the is profession wants to ensure that the software product is developed in accordance with the terms in the is development contract. the legal practitioners are involved with the is development contract either at the beginning, with the negotiation and drawing of the contract, or if it comes into dispute and the parties seek legal redress. to try to prevent the contract from coming into dispute is professionals need to understand the processes to be followed before entering into a contract, what is required in a contract, and how to manage that contract. processes before entering into an is contract the first part of the contract process is to determine the type of relationship that is required. nam et al. (1996) outline four main types of relationships: support; reliance; alignment and alliance. the type of relationship that is decided upon determines the expressed terms that are needed to be included in the contract. once the type of relationship is decided the next step is to choose a contractual partner. binstock (1999) identifies that in choosing a partner it is important to make sure that they have software development certification (e.g., iso 9000 or at least cmm level 2 or 3). the contract should also contain provision for termination at short notice if the wrong party has been selected and their performance is not satisfactory (bramson 1990). this is to ensure that if the wrong partner has been selected then the contract can be terminated before either party suffers significant loss (frost 1999). contents of an is development contract in the development of the contract it is important the contract contains detailed specifications and schedules (binstock 1999, frost 1999). detailed specifications are important, especially in an industry where clarity is, some would argue, a scarce commodity. hollington (2000) identifies two significant problems with specifications: • gaining an understanding of what is wanted; and • articulating what you want so that others can understand it. dakin (1996) points out that the process of drawing up a written contract is intended to assist in this understanding. however, when negotiating something as technical as is development there is room for error. the first step in designing a good contract is to ascertain the true objectives of each party. bragg and allen (2000) identify a number of terms that should be addressed, including: defining intellectual property rights; establishing pricing; defining the customer’s responsibility; identifying deliverables; defining acceptance criteria / procedures; providing limited indemnity; and defining an exit strategy / dispute resolution process. in association with a contract a requirements document needs to be prepared. this document has the greatest potential impact on litigation avoidance, as it is a source of a statement of the functions the software is to deliver. bragg and allen (2000) and binstock (1999) point out that clarity of communication is essential to a well developed is development contract. lawrence (2000) supports the need to communicate, but goes further to indicate that communication in is development projects needs to outline the value of the project; what problem is being solved; product design and plans; project status; and product status. welch (1999) identifies that, as part of building the contractual relationship, it is important that there is a transfer of knowledge to the client. confusion or ambiguity in any of these areas can be fatal to a project (lawrence 2000). ajis vol. 10 no. 2 may 2003 63 contract management contract management is clearly different to project management. is professionals often get contract management and project management confused. project management can be defined as the systematic approach of analysing, organising and completing a project (norris et al. 1996). contract management involves looking at and reviewing the contractual document (dakin 1996). davis (1995) states that good management is more important than good technology. it is considered by some to be good management to organise the contract into small jobs and then sign each off before moving on to the next one (bramson 1990). this notionally ensures that both parties to the contract are aware of the present stage of a development, and whether the contract is being met. mah and stone (2000) discuss four early warning signs of is development contract breakdown. the first is identified as ‘a shot gun wedding’ where contracts are entered into quickly as the parties believe they will fix the contract later. the second is a poor or inadequate metrics framework often contracts do not contain metrics. these metrics are vital for project estimation, service level determination and productivity benchmarking (dakin 1996). the third is treating outsourcing as a transaction. a theme in soured contracts is that the relationship failed. relationship management is an area where parties experience a breakdown. outsourcing deals are business transactions but are also relationships. the final warning sign lies in contractual dispute resolution clauses. when you see dispute resolution clauses you often don’t see processes for managing conflicts what you find are ways to escalate them. all of the previous discussion looked at how to stop things from going wrong and where they go wrong. lawrence (2000) discusses why they go wrong. specifically, lawrence suggests that the parties: • start out unprepared; • underestimate the complexity involved; • choose the wrong design strategy; and • leave the requirements vague enough that any interpretation would be plausible. • legal practitioners and is contracts we turn now from a consideration of professional behaviour, quality and quality processes, and the is contract itself, to the law that underpins the drafting and interpretation of software development contracts. the laws discussed in this section relate to the laws of the commonwealth of australia. contract law is predominantly based upon law that has been established by judges in appellate cases. this is the way common law develops. legislation increasingly plays a role in augmenting or changing common law. an is development contract is a contract in the full legal sense of the word and may therefore be reviewed by a judge in the event of dispute. when disputes arise it is usually as a result of a perceived breach of the contract. for this reason this brief review of relevant literature focuses on disputes which involve breaches of contract. establishing a contract in judgement of a dispute, the court, when looking at a contract, will establish whether or not a contract has been formed. to form a contract there must be an offer, acceptance, consideration and an intention to create legal relations. these four elements must be established to prove that there was an is development contract in place. if a contract exists then it will be necessary to assess what is in the agreement. the terms of the agreement may be express or implied. express terms of an is contract generally, contracts of the kind being discussed in this paper will include express terms; express terms are the terms that are specifically stated within the contract, either orally or in writing. identification of express terms is simply a matter of examining the contractual document (if the contract is in writing, which is the case for most is development contracts). in the course of negotiations, parties will often refer to various matters that pertain to their particular situation. some statements made during the negotiation may be intended to be a term of the contract while other matters or points may not be considered as important. a term is a contractual statement which amounts to an undertaking or guarantee by the maker of the statement, that it is true or that the maker had reasonable grounds for making it. importantly, the terms must be delineated and expressed. difficulties arise when the parties have differing views, or representation, of what forms the basis (or terms) of the is contract. a representation that does not accord with the true facts (past or present) may render a contract voidable. this representation may include statements of fact, advice, or opinion made in negotiation, which are inaccurate or misleading. at this ajis vol. 10 no. 2 may 2003 64 point it is important to distinguish a term from a representation, insofar as contract law is concerned. if a representation is untrue then the only common law remedy is rescission (though there are no extensive statutory remedies under the trade practices act 1974 (cth) (tpa) and fair trading acts (of the various states)). if a pre-contractual statement becomes a term then the remedy of damages will be available, and depending on the seriousness of the breach, the possibility of terminating the contract. therefore, it is important to establish whether a statement made during negotiations is a term that was agreed upon by both parties. within an is development contract there are a number of express terms that should be included. they include terms which deal with: supply and distribution; research and development; options and rights of first refusal; joint venture relationships; system integration; licence; specifications; installation and tests; price and payment; records and other administration; warranties; maintenance; support and spare parts; suspension; termination; after termination; subcontracting; relationship; assignment; severance; and waiver terms (calvert 1995). implied terms of an is contract implied terms are part of a contract, even though they are not stated, and merely reading the document will not expose them. there are two types of implied terms: implied by fact; and implied by law. as stated in secured income real estate (australia) ltd v st martins investments pty ltd1, there are five requirements to be satisfied before a term can be implied by fact. specifically the term should: • not contradict any express term of the contract; • be reasonable and equitable; • be necessary to give business efficacy to the contract — that is no term will be implied if the contract is effective without it; • so obvious it goes with saying; and • be capable of being clearly expressed. these requirements ensure that implied terms are reasonable and will not interfere with the contract. the significant point behind this is that terms will be implied when they are necessary to make the contract work in the way the parties intended. the more extensive the negotiations and the more detailed the resulting contract, the less likely it is that a term will be implied because it will be difficult to convince the court that a necessary term was left out (codelfa construction pty ltd v state rail authority of nsw2). a term may be implied at common law, in order to express a generally accepted incident of the contract, or to complete an incomplete contract (carter 1994). in these types of situations the terms are incidents of the contracts because judicial decisions have indicated that it is to be expected of the parties. where there is not such a situation, due to unique facts, then a term may be implied as a matter of justice and fairness (carter 1994). a term may also be implied by statute. those with the greatest relevance are consumer protection legislation and sale of goods legislation. consumer protection legislation, such as the trade practices act (tpa), can imply terms. a representative case is caslec industries pty ltd v windhover data systems pty ltd3. in this case a computer installation was in dispute. the main thrust of the case is that after the software was installed expectations were not met as to what the system was to do. hence there was a disagreement as to the terms of the software agreement. the court found that not only was there a breach of the contractual terms but also the implied terms, warranty and being reasonably fit for purpose under s 69(1) of the tpa. the consumer protection provisions of the tpa that are the most relevant for technology suppliers and consumers are (calvert 1995): • implied terms into contracts; • imposed liability on manufacturers directly to consumers for goods of unmerchantable quality (for where goods are not purchased directly from the manufacturer); and • imposed certain minimum standards of conduct (prohibiting misleading and deceptive conduct). for the tpa to be implied into a contract there must be a consumer. section 4b of the tpa defines a consumer as a person who shall be taken to have acquired particular goods as a consumer if the price of the goods did not exceed $40,000: • where the goods exceeded $40,000, the goods were of a kind ordinarily acquired for personal, domestic or household use; and 1 (1979) 144 clr 596 at 605-6 2 (1982) 149 clr 337 3 (federal court of australia, gummow j, no g627/90, 13 august 1992, unreported) ajis vol. 10 no. 2 may 2003 65 • were not acquired for the purpose of re-supply or for the purpose of using them up or transforming them. unfortunately the $40,000 limit is very difficult to apply, as it is not clear what is involved in calculating the price. it might involve the sale of 20,000 of the same item at $10.00 each (hence total contract price is $200,000 but it is likely to be a sale of goods to a consumer because the unit price is below $40,000). however in the case of maintenance, where the contract might specify it to be 10% of the system price of $1,000,000, it is more problematic. if the fees are paid monthly then it is less than $40,000 each month. where do you draw the line? the tpa defines goods and services very broadly. most is development contracts will come under one or the other. however sometimes it is very hard to categorise technology. while it is clear that a computer system, hardware and software provided together can be goods, it is not clear whether goods includes software supplied separately (toby constructions products pty ltd v computa bar (sales) pty ltd4). categorisation of goods and services is important, as the conditions and warranties implied for each category are different. the terms implied for goods are more comprehensive. this remains unclear as the courts have not come to a definite answer themselves as in caslec industries pty ltd v windhover data systems pty ltd5 where the court considered software to be services, whereas in st albans city council v icl6 the court considered software to be goods. the requirement of the consistency between the implied and expressed terms of the contract applies to all types of implied terms, as has been seen in secured income real estate (australia) ltd v st martins investments pty ltd7, where one of the requirements to give business efficiency is that it does not contradict express terms. a term implied by law will also not be implied if it was found to be inconsistent with the express terms. therefore it is not easy to imply a term into an is development contract. the term, if it is to be implied, must serve a significant purpose. research population: is professionals and legal practitioner cohort an is development contract negotiation involves two key stakeholder groups: is professionals; and legal practitioners. for the purpose of this exploratory research, representatives of both groups were identified in the melbourne and geelong areas of australia. a total of 10 professionals were approached (4 legal practitioners and 6 is professionals). the is professionals selected were all members of middle to senior management in their organisations, working within the information technology industry. the legal practitioners held senior positions within their firms, specialising, or having a significant interest and experience in, information technology contracting practice. participants in the interview program were selected initially for an interview, based upon their position titles, with particular interest in positions related to the negotiation, preparation and/or oversight of is development contracts. the principle question to be addressed in the present research program is: are there differences in the perceptions held by legal practitioners and is professionals of is development contracts? to proceed, this question was decomposed into four sub-questions: • what is the accepted legal opinion of how contract law applies to is development contracts? • what are the perceptions that is professionals hold of is development contracts? • what are the perceptions that legal practitioners hold of is development contracts? and • are there differences between the perceptions held by the is professionals and the legal practitioners? the first of these questions has been addressed, in part, in the preceding section 3 of this paper and is canvassed further below. findings to date related to the remaining three questions, drawn from the program of semi-structured interviews, are also reported. research findings — difference in perceptions the aim of this research was to conduct an exploratory study of the differences in perceptions of is development contracts held by the key stakeholder groups (is professionals and legal practitioners). the semistructured interview covered a number of key issues, with perceptions been uncovered spanning matters such 4 (1983) 2 nswlr 48 5 (federal court of australia, gummow j, no g627/90, 13 august 1992, unreported) 6 (1994, unreported) 7 (1979) 144 clr 596 at 605-6 ajis vol. 10 no. 2 may 2003 66 as: pre-contractual processes; contents of a contract; contract management; aspects of is professional practice; expectations of quality; the specification of a quality process; contract management; common legal advise; awareness of case law and major problems and issues of is contracts (hodges 2000). this section has been structured in terms of the four sub-questions, as outlined above with tentative conclusions being drawn against each of these sub-questions. what is accepted legal opinion on the application of contract law to is development contracts? contract law provides a number underlying principles that are used as guidance in the development of an is contract. an is contract as a legal instrument a contract is a legally binding document that contains terms and conditions that state the obligations of the parties to that contract. if these obligations are not met then a breach of contract has occurred. the injured party may seek legal advice that may lead to the process of mediation, arbitration, or litigation and therefore obtainment of an appropriate remedy. therefore to form a legally binding contract the contract must consist of four elements: offer; acceptance; consideration; and the intention to create legal relations. express and implied terms of an is contract a contract has both express and implied terms. both must be met to fulfil the obligations of the contract. the express terms explicitly state the obligations of the parties to the contract and are clearly delineated. these include the requirement specification, intellectual property rights, pricing and acceptance criteria. in contrast, the implied terms are not defined within the contract. instead, they can either be implied by fact or by law. therefore, implied terms are not only hard for the courts to establish, but also for the legal and is profession to define. often community, industry and government expectations establish the implied terms of a contract. these can include professional practice and quality. what are the perceptions that is professionals hold of is development contracts? a number of perceptions have been uncovered from interviews with the is professionals’ cohort, spanning matters such as: entering into a contract (e.g., pre-contractual processes, contract management); aspects of is professional practice; expectations of software quality specification and quality management; common legal advice; and awareness of case law. entering into a contract entering into a contract can be seen as a two-step process, involving the organisation (pre-contractual processes) and the drafting of the contract. it was perceived that the pre-contractual processes should include: requirements specification; obtaining a quotation of tender; vendor investigation; and negotiation. the is practitioners, however, believed that the processes they themselves identified were not universally followed. pre-contractual processes can form the basis of the main terms of the is development contract. is professionals perceive that the contract should include: pricing; requirement specification; intellectual property rights; and acceptance criteria. entering into the contract is not the last stage the contract also needs to be managed. to emphasise this, one is professional stated that: “its critical. you need to have a framework that facilitates openness so that you can exchange information quickly because it doesn’t take long for a project to get off the rails”. is professional membership membership of a professional association was not seen as an important component of an is development contract by the is professionals. however the is professionals reported that adherence to a code of ethics as well as a code of conduct should be specifically incorporated in an is development contract. a possible interpretation of this is that is professionals tend not to rank membership of a professional association as essential, or indeed maybe they consider it irrelevant, to the day-to-day conduct of is development activities. ajis vol. 10 no. 2 may 2003 67 software quality management systems there was a perception that an expectation of quality should be expressed within an is development contract. there was, however, a diversity of perceptions of what the requirements of the quality process should be. it was encouraging, however, that the is professionals were able to name some key quality expectations and processes that could be followed in the execution of an is development project (e.g., the iso 9000 series and cmm). legal advice to the is professional legal advice is usually given in relation to interpretation, although is developers tend to avoid seeking legal advice. this was supported with the following statement from an is professional: “we try to avoid that like the plague”. indeed one is professional clearly stated “it’s a big point as to the value seeking legal advice brings – they tend to introduce as many new difficulties as they can.” is professional legal knowledge four of the six is professionals claimed to be aware of relevant case law. their awareness, however, appeared to be based only on is cases reported in the popular press (e.g., cases involving ibm, microsoft, and apple) often cases which actually had little to do with testing the application of contract law. what are the perceptions that legal practitioners hold of is development contracts? a number of perceptions have been uncovered from the interviewed legal practitioners, spanning matters such as: entering into a contract; main terms of an is contract; contract management project management; aspects of is professional practice; and is professional membership. entering into a contract the legal practitioners perceive that pre-contractual processes should include: requirement specification; and vendor investigation. however, the legal practitioners believed that these processes were not universally followed within the is industry. main terms of a is contract the main terms in an is development contract were identified as: requirement specification; and intellectual property rights. contract management and project management although the is profession enter into contracts, legal practitioners did not believe that the contracts were managed after signing. one of the legal practitioners made the statement that: “almost by definition they come to us too late, and they really do have a very big problem by the time they come to us.” further, the major problem and issue associated with is development contracts raised by legal practitioners was requirement specification. incorporation of is professional membership into an is contract there was diverse opinion as to the significance of incorporating association membership in an is development contract. legal practitioners certainly acknowledged that there should be adherence to a code of ethics, but they did not believe that a code of conduct should be specifically incorporated in an is development contract. are there differences between the perceptions held by the is professionals and the legal practitioners when examining is contracts? central to this work has been the comparison of the two cohorts and their responses. the main differences between the two cohorts covered: pre-contractual process; main terms of the is contract; contract and project management; professional membership; code of conduct and ethics in an is contract; and software quality management. ajis vol. 10 no. 2 may 2003 68 pre-contractual process with regard to the pre-contractual processes, is professionals perceived the quotation of tender and negotiation as important. in contrast, the cohort of legal practitioners did not rate this as highly. both cohorts said that the pre-contractual processes they identified as important were not universally followed. main terms of an is contract the is professionals identified the establishment of pricing and acceptance criteria as the main terms of an is development contract while the cohort of legal practitioners did not rate these as highly. contract management and project management a significant difference was in the perception of contract management. the is professionals believed contract management was routinely carried out, whilst the legal practitioners believe it was rarely practiced by is professionals. this may be attributable to is professionals identifying project management, which presumably they do routinely carry out, with contract management. these are viewed, one might argue quite correctly, by the legal profession as distinctly different activities. incorporation of is professional membership in an is contract there was a diverse response to questions concerning the incorporation of professional association membership in to an is development contract from the legal practitioners. the is professionals believed that it was of no contractural significance. an interpretation is that is professionals tend not to rank membership of a professional association as essential, or indeed maybe consider it irrelevant, to the day-to-day conduct of is development activities. code of conduct and professional ethics in an is contract there was also a difference in perceptions in relation to adherence to a code of conduct being incorporated in an is development contract. is professionals supported its inclusion, while the legal practitioners were against the notion. software quality management the requirement of specifying quality processes in an is development contract received a diverse response from the is professionals, while the legal practitioners supported the idea. this may suggest that the is profession is somewhat ambivalent to the whole notion of quality software development processes. the is professionals were, however, able to name several quality processes that might be used, while the legal practitioners found it difficult to specify any. conclusion although this is a limited exploratory study, there is tentative evidence of a number of fundamental differences in the perceptions of is development contracts held by is professionals and legal practitioners, as highlighted in this paper. further research, with more substantial cohort sizes, is recommended to confirm and further delineate these differences. references anderson, r. e., d. g. johnson, et al. (1993). “using the new acm code of ethics in decision making.” communications of the acm 36(2): 98 102. binstock, a. (1999). “outside development partners.” informationweek (757): 133-140. bragg, t. and r. allen (2000). “litigation avoidance: a lifecycle approach.” cutter it journal 13(9): 411. bramson, r. s. (1990). “the dotted line.” information strategy: the executive's journal 7(1): 44-47. calvert, m. (1995). technology contracts. melbourne, reed international books australia pty limited trading as butterworths. carter, j. w. (1994). outline of contract law in australia. melbourne, reed international books australia pty limited trading as butterworths. dakin, k. (1996). “keys to engineering a workable contract.” ieee software(january): 99-100. ajis vol. 10 no. 2 may 2003 69 davis, a. m. (1995). 201 principles of software development. sydney, mcgraw-hill, inc. frost, l. i. (1999). “you get what you ask for.” the journal of business strategy 20(3): 11-14. gotterbarn, d. (1999). “specifying the standard make it right.” computers and society (september): 13-16. hodges, k. (2000). "an exploratory investigation of stakeholder perceptions of information systems development contracts," honours thesis, school of management information systems, deakin university. hollington, p. (2000). “you don't always get what you want: but, you do usually get what you ask for!” cutter it journal 13(9): 26-30. lawrence, b. (2000). “learnings of an expert witness.” cutter it journal 13(9): 31-35. mah, m. and d. stone (2000). “it relationship management: four early warning signs of breakdown.” cutter it journal 13(9): 12-19. martin, c. d. and e. yale welt (1999). “from awareness to action : integrating ethics and social responsibility into the computer science curriculum.” computers and society (june): 6-14. nam, k., s. rajagopalan, et al. (1996). “a two level investigation of information systems outsourcing.” communications of the acm 39(7): 36-44. norris, m., p. rigby, et al. (1996). the healthy software project: a guide to successful development and management. chichester, john wiley & sons. rosenberg, r. s. (1998). “beyond the code of ethics : the responsibility of professionals societies.” computers and society acm policy '98: 18-25. spice (1999). “an introduction to the documents.” last accessed: 10th may 2000. http://www.sqi.gu.edu.au/spice/suite/intro.html ulrich, w. (2000). “it ethics: heading off legal challenges by doing the right thing.” cutter it journal 13(9): 20-25. van der pijl, g. j., g. j. p. swinkels, et al. (1997). “iso 9000 versus cmm: standardisation and certification of is development.” information and management 32: 267-274. welch, a. (1999). “communication plays vital role in outsourcing partnerships.” computer dealer news 15(36): 31-32. wood-harper, a. t., s. corder, et al. (1999). “ethically situated information systems development.” the australian computer journal 31(3): 66-71. yourdon, e. (2000). “introduction” cutter it journal 13(9): 2-3 microsoft word 13 1 marina.doc ajis vol 13, no. 1 september 2005 5 editorial this is the first issue of volume 13 of the australasian journal of information systems and is the last issue that i will be editing. our first paper is presented by graeme j. byrne and lorraine j. staehr, from la trobe university and is entitled women in the australian it industry: where are we now. the purpose of this paper is to assess the status of women in the it industry in 2001 relative to their position in 1996 and in comparison to their status in other industry sectors. among other findings, the paper shows that women participate in the it industry at much lower rates than women in many other industries and that they have not achieved pay equity with men. our next paper, entitled rule mining for dynamic databases is presented by a das and d k bhattacharyya , from tezpur university, india. this paper presents two modified version of the borders algorithm called modified borders. experimental results show that the modified version performs better than the borders algorithm in terms of execution time. our next paper is entitled don’t leave patch management to chance and is presented by ralph defrangeso, usa. this paper attempts to introduce conditional probability into the patch management process giving the reader a tool to evaluate where best to spend their time, analysis or testing. bayes’ theorem is used to help predict the outcome of a patch management example. james gibson, university of wollongong presents our next paper entitled a research strategy for investigating business process management approaches. this paper defines the research framework for effective academic research enabled by co-ordinating research based on the primacy of the business model and its resultant effective representation in executable systems. sigi goode, australian national university, presents our next paper entitled a longitudinal study of isp reactions to australian internet content regulation laws. this paper discusses a longitudinal study which surveys a targeted selection of australian isps to determine both initial and subsequent effects of and attitudes towards the legislation. our next paper is presented by emmanuel ikart, university of wollongong and is entitled executive information systems and the top-officers’ roles: an exploratory study of user-behaviour model and lessons learnt. this paper reports on research in progress in australia on the adoption and usage of eis by executives. the results suggest a higher degree of eis usage by middle managers than top-level managers. evan patullock, la trobe university, presents our next paper entitled the management of computer systems in small business: a preliminary analysis of the issues of importance to managers. this paper reports on preliminary results from a pilot project investigating the relative importance of various is concerns from the perspective of small business managers in the albury-wodonga region. initial results confirm the operational focus of small business managers. graham pervan and hilangwa maimbo, curtin university, present our next paper entitled a credit union case study of the business value of is/it. this paper investigates the business value of is/it investments in the financial services sectors (fss) in australia using a case study methodology to develop and refine a conceptual model of the relationship between is/it investment and organisational performance. our next paper is presented by wei ge, university of wisconsin, marcus rothenberger, university of nevada and edward chen, germany. the paper is entitled a model for an electronic information marketplace. this paper presents a modified business model for ajis vol 13, no. 1 september 2005 6 information trading, which uses a reverse auction approach together with a multiple-buyer price discovery process. our next paper is entitled securing xml query processing storage and is presented by charles shoniregun, university of east london, oleksandr logvynovskiy, london south bank university and kevin lu, brunel university. this paper presents a new structural join algorithm called sequence join algorithm, for sequential regular path expressions in securing xml query processing storage which exploits information about position of the elements in a product to skip generation of redundant intermediate lists. our next paper is presented by oliver hoffman, david cropley, university of south australia, arthur cropley, university of hamburg, lemai nguyen, deakin university and paul swatman, university of south australia and is entitled creativity, requirements and perspective. the paper examines the paradox caused by the need to be functionally creative while leaving room for creativity in successive stages. sharman lichtenstein, lemai nguyen and alexia hunter, deakin university present our next paper entitled issues in it service-oriented requirements engineering. this paper suggests that in the new it services era, new techniques and approaches are needed for eliciting and determining provider and customer requirements that involve key stakeholder groups equitably and more closely negotiate the sometimes-conflicting provider and customer needs. oliver minor, university of koblenz-landau and jocelyn armarego, murdoch university present our next paper entitled requirements engineering: a close look at industry needs and model curricula. this paper examines the requirements component of model curricula in the disciplines of computer science, information systems and software engineering. the results reveal that the model curricula address the expectations for requirements activities, practitioners have mentioned. our next paper is entitled consensus making in requirements negotiation: the communication perspective and is presented by jenny price and jacob cybulski, deakin university. as requirements elicitation involves rich communication between project stakeholders, this paper explores negotiation and consensus making from the communication perspective. the resulting model assists in the understanding of the communication factors that influence the consensus process during requirements negotiation. june verner and karl cox, national ict australia, steven bleistein, university of new south wales & narciso cerpa, universidad de talca, present our next paper entitled requirements engineering and software project success: an industrial survey in australia and the u.s. this paper showed that the best predictor of project success was that the requirements were good and were managed effectively. carolyn ye-phem chin and paula swatman, university of south australia present our next paper entitled the virtual shopping experience: using virtual presence to motivate online shopping. this paper identifies some of the possibilities and problems of online shopping today, illustrating the current status of virtual presence in retailing with two micro-cases of success and failure. our next paper is entitled the relevance of value net integrator and shared infrastructure ebusiness models in managing chronic conditions a working paper and is presented by susan lambert, flinders university. this paper looks to ebusiness models for assistance in understanding the requirements of general practitioners (gps) in providing multidisciplinary team care to patients with chronic conditions. ajis vol 13, no. 1 september 2005 7 daniel walsh, carole alcock, lois burgess, university of wollongong and joan cooper, flinders university present our next paper entitled a pda based point of care e-health solution for ambulatory care. this paper discusses phase one of a larger, four-phase project which aims to develop information access applications at point-of-care for ambulatory care services. the initial phase of the research (phase one) involves workflow analysis, requirements specification and the development and testing of a system prototype. janice burn and paul alexander, edith cowan university present our next paper entitled a capabilities analysis of e-fulfilment business: transformation in the logistics industry. this paper reports on the evaluation of practices of 48 leading e-fulfilment suppliers in the uk. initial findings suggest that while a comprehensive model of e-fulfilment is validated these organisations are also providing services not previously recognised as e-fulfilment capabilities by the literature. our last paper is entitled diffusion of innovation – the adoption of e-commerce by small to medium enterprises (smes) a comparative analysis and is presented by wayne pease, university of southern queensland and michelle rowe, edith cowan university. this paper explores the issues that influence the diffusion of innovation as it relates to the adoption of e-commerce by small and medium enterprises (smes), in particular factors facilitating and inhibiting adoption across contexts – regional, small city and large city. i would like to thank the many authors, referees, board members and technical experts that have helped me over the 13 years that i have been editor of ajis. i believe that the new editor will find that the many willing hands that are available will make his/her job that much easier. i hope that ajis continues to grow in strength and continues to support the is community. dr rob macgregor microsoft word ajis10-2finala.doc ajis vol. 10 no. 2 may 2003 115 the extent to which information communication technology careers fulfil the career ideals of girls. judy young school of information systems university of tasmania jf_young@infosys.utas.edu.au abstract the aim of this research is to make a contribution to knowledge in relation to women as a minority group in the information communication and technology (ict) industry. the research presented in this paper represents a minor component of a much larger research project conducted under the auspices of women in it tasmania (wiitt) and supported by funding from the intelligent island. the focus of the current research is on year nine students throughout tasmania. it will involve a detailed examination of the results in relation to the perceptions of year nine girls and boys in relation to the career characteristics of their ideal careers and their views of careers in ict. the objective is to identify areas that need to be specifically promoted to girls to attract them to careers in ict. in addition existing research has shown that parents and relatives have a considerable influence on girls when they are making a career choice. this can mean that if the information girls obtain from this advice is inaccurate or out of date, then many could reject a career in ict as a viable career option. accordingly a secondary objective is to consider the means by which such information can be most effectively distributed. key word gender differences, perceptions of it careers, is career. introduction historically, worldwide, women have been a minority in the information communication and technology (ict) labour force. even though a chronic shortage of skilled information systems (is) personnel has plagued the ict industry, women have continued to represent a largely untapped human resource. despite increased opportunities for employment and the changing nature of employment in the ict industry, typically the extent of the involvement of women has remained constant at around 20%. the research presented in this paper is based on the findings of the recent women in it tasmania (wiitt) report (young 2002). the objective of this report was to determine the ‘buying behaviours’ of girls associated with ict careers. the wiitt project was founded on the premise that a gap existed in the body of existing knowledge in relation to the under representation of women in ict careers. that is, the bulk of previous research had been conducted in larger population centres while the situation in regional areas has been largely ignored. to address this deficiency in knowledge the wiitt project was located in tasmania, the small island state of australia. this setting for the project was seen as a perfect opportunity to take a large sample of the identified population across a broad range of rural and suburban regions. one of the main findings from the wiitt research showed that only five percent of the tasmanian year nine students surveyed intended to establish careers in the ict industry. further, of those who indicated this intention, boys out numbered girls by four to one. this result highlights a real need for action to promote ict careers to girls in smaller population areas. in addition the wiitt research found that when choosing a career, girls more so than boys, sought the advice of their parents and relatives. this result confirms the findings from earlier research that showed parents exert considerable influence on girls when career decisions are made (multimedia victoria, 2001). however if these people, when acting as advisors are unaware, ill informed or hold out of date views of the nature of ict careers, then many girls may reject careers in the industry based on misinformation. previous research in the area has been found this often to be the case (multimedia victoria 2001). potentially this could mean that now obsolete features of the industry such as a heavy emphasis on technology still persist to preclude a larger number of young women from entering the industry. the reality is that in the modern business environment the nature of ict has broadened with a need now for personnel capable of providing business, people and creative skills. this implies that many of the historical negative barriers that portray the industry as solitary, male dominated and ‘nerdy’ are no longer appropriate. from this background of the findings of the wiitt report it became apparent that to encourage more young women to take up careers in the industry there is a need for information that accurately portrays ict careers in a positive light. background for more than a decade the problem of women as a minority group in the ict industry has attracted the interest of researchers and a considerable body of knowledge has been assembled (frenkel 1990; spertus 1991; igbaria and baroudi 1995; camp 1997; teague 1998; the acm committee on women in computing 1998; gurer 1999; meares and sargent 1999; information industries bureau 2000; newmarch, taylor-steele ajis vol. 10 no. 2 may 2003 116 et al.. 2000; ahuja 2002). however, the situation is a complex one (queensland government, 1998) that so far has defied any workable solution. the existing research can be broadly divided into being based in either an industry or an education perspective. the industry perspective has encompassed women already working in the industry (frenkel 1990; swanson and wise 1996; selby 1997; teague 1998; nielsen, von hellens et al.. 2000; nielson, et al. 2000; von hellens, pringle et al. 2000; von hellens, et al. 2001; ahuja 2002), women lecturing in computing or it in universities (frenkel 1990; levenson 1990; selby 1997; o'leary 1999; cuny and aspray 2000; pringle, nielsen et al. 2000; trauth, et al. 2000; multimedia victoria 2001) as well as women teaching it courses in the tafe post compulsory education system (delaney and dyson 1998; multimedia victoria 2001). the educational aspect of the research has focussed on girls as students in the school education system (chan, stafford et al. 1998; nielson, et al. 2000; moon 2001), young women enrolled in tafe it training (newmarch, 2000; information industries training board, 2001) or studying it related courses at university (levenson 1990; bana and hassoun 1997; margolis and fisher 1997; selby, young et al. 1997; cuny and aspray 2000; newmarch, taylor-steele et al. 2000; nielson, et al. 2000; jewell and maltby 2001; staehr, martin et al. 2001). much of the existing education research has been focussed on the secondary school level of education as this has been identified where the initial barrier that precludes many girls from ict careers is established (chan, stafford et al. 1998; nielson, et al. 2000; moon 2001). this has given rise to the notion of the ‘shrinking pipeline’ (camp, 1997). what this implies is that a large proportion of girls discontinue studies in computing subjects at this point of their education. in addition, as young women move through tertiary education and beyond, the proportion that remain involved in ict continues to decline. from the education stream of research three major themes can be identified as having a negative impact on the perceptions of girls in relation to ict. these are girls lack of confidence, the impact of the teaching and learning environment and the lack of accurate ict career information. girls lack of confidence the dominance of boys in relation to access and usage of computers in the school system has long been recognised (margolis and fisher 1997; selby, young et al. 1997; newmarch, taylor-steele et al. 2000). potentially, partially due to the dominance of boys, girls have been found to be less confident in their computing abilities. in addition, it has been determined that many teachers react differently to girls in ict classes. this has often precluded them from demonstrating their capabilities in a positive light and so has served to perpetuate the lack of confidence on the part of girls (chan, stafford et al. 1998). as a result, the issue has been a recurring theme to an extent it has been described as a ‘ghost that haunts our subject’ (margolis and fisher 1997:4). the impact of the teaching and learning environment there is considerable evidence in the literature to suggest that the content and delivery of ict in the secondary school system has been a major disincentive to interest girls to continue studies in the subject (selby 1997; newmarch, taylor-steele et al. 2000). in particular the teaching and learning environment has been found to be non-conducive to encourage the participation of girls (selby, 1997). while generally students have reported ict be a boring subject (teague 1998; bowes, cowley et al. 2001; multimedia victoria 2001), a recent study conducted in the tasmanian state education system found twice as many girls than boys perceived it courses to be boring (bowes, cowley et al. 2001). in part this has been attributed to the fact that because many courses focus on gaining familiarity with software packages, it has communicated an incorrect message to girls that ict involved secretarial careers (chan, stafford et al. 1998; moon 2001; newmarch, taylor-steele et al. 2000). in addition, it has been concluded that many teachers appointed to deliver ict courses have little or no knowledge or skills in the subject (bowes, cowley et al. 2001; centre for international economic report, 2001). the lack of accurate ict information a lack of understanding of what careers in ict entail has been identified as a considerable influence contributing to the imbalance of women in the industry (selby, young et al. 1997; teague 1998; newmarch, taylor-steele et al. 2000; nielsen, von hellens et al. 2000; moon 2001). this implies that outdated, negative images of the ict industry have continued to be influential deterrents. the representation of ict as a male ajis vol. 10 no. 2 may 2003 117 dominated, highly technical, solitary field and the domain of ‘geeks’ and ‘nerds’ has been effective in precluding women in larger numbers from careers in the industry. the reality is that in the modern business environment there is a wide range of ict activities and roles to which women are eminently suited (queensland government 1998). further, the mix of skills required is ever changing to support the continuous developments in the industry (it&t skills hub 2000). in the current ict business arena, while a focus on technology remains, it no longer represents a dominant activity. equally, there is also now a need for personnel with people, business and creative expertise (swanson and wise 1996; juliff 1998). a number of approaches have been recommended to achieve the widespread and effective distribution of ict career information to girls. these include workshops, job shadowing, mentoring (chan, stafford et al. 1998), industry visits, attachments or summer internships (newmarch, taylor-steele et al. 2000). in addition, it has been proposed that an effective way to overcome the lack of ict career information could involve setting up links between schools, tertiary institutions, the ict industry, government agencies and professional organisations (selby, young et al. 1997; von hellens, et al. 2001). generally, however, these approaches focus on girls and young women and do not take into account the substantial influence that parents and friends can exert when career decisions are made. this implies that to promote the uptake of ict careers among young women, up-to-date career information is not only made available but distributed to a wide audience. research objective from the perspective of the lack of understanding on the part of women in relation to modern day ict careers, the aim of this paper is to determine the types of career characteristics that need to be promoted to attract larger numbers of them to careers in the industry. a secondary objective is to recommend the means by which the distribution of such information can be most effectively achieved. assumptions the research reported in this paper is based on the assumption that if girls lack understanding of what modern day ict careers entail, they will be unable to recognise any associations between this form of career and what they see as important for their ideal career. at the same time if outdated myths are not identified and dispelled, they will persist as influential in precluding many young women from careers in ict. research scope this research draws on the findings from the wiitt report and examines in further detail the results from two questions contained in the questionnaire administered within that research. for the purposes of this paper the scope of the research is limited to year nine students attending secondary schools throughout tasmania. based on a five point likert scale the first question asked participants to indicate the extent to which they agreed or disagreed that sixteen career characteristics represented their ideal career. the second question repeated this approach but in this instance in relation to careers in ict. the aim of this approach was to ‘tease’ out the issues of special importance to girls in order to demonstrate the extent to which ict careers can achieve their ideal career goals. these career characteristics were drawn from existing allied research (selby 1997; multimedia victoria 2001; swift 2001). the application of these previously established characteristics provides a basis to identify particular aspects that need to be promoted while at the same time to provide feedback of areas where special attention needs to be made to redress incorrect information. method the aim of this section is to establish the sample population and the approach to data collection used in this research. the process of data collection provides details of the pilot test and the administration of the main questionnaire. sample the research presented in this paper is based on a sampling of year nine students enrolled in schools throughout tasmania. the sampling encompassed district high schools, high schools, catholic and noncatholic independent schools. these represented single sex schools and co-educational schools. enrolment in information processing (the term used for computer studies at this level in the tasmanian education system) as a subject at was not a prerequisite for taking part in the research. ajis vol. 10 no. 2 may 2003 118 three reasons support the primary focus of this research on this level of education group. firstly, in the tasmanian education system it is at this point in their education when students are able to choose the non-core subjects they wish to study. this implies an ability to make a decisive choice to enrol in ict related courses. secondly, year nine is also the stage where it has been found that peer pressure is exerted on girls and as a consequence many drop out of ict as a subject (frenkel 1990). thirdly, it has been widely acknowledged that the process of making a career choice commences around year nine in the education system (moon 2001). data collection the research was based on a cross sectional sampling and involved the distribution of questionnaires to the schools that agreed to participate in the study. the questionnaire developed and administered within this project was based on instruments applied in earlier research that shared a similar focus to the present study (selby 1997; multimedia victoria 2001; swift 2001). these instruments had been used in research conducted in new zealand, australia and canada. consequently the wiitt survey incorporated previously tested and validated instruments that had been applied in a range of countries. pilot survey prior to the administration of the main questionnaire, the instrument was subjected to a pilot round of sampling. the aim of this process was to ensure that the language and content of the survey were appropriate for tasmanian year nine students. it was based on a forum conducted with forty-eight students attending a department of education school located in a north hobart suburb. both boys and girls completed an individual paper based brain storming activity of the key aspects proposed in the draft survey. on the basis of this pilot round no changes were necessary to the questionnaire and accordingly it was deemed as appropriate for application in the main survey. main survey the potential sample size of the research was governed by the ability to gain support for the project from individual schools, parental consent and the willingness of students to participate. following approvals from the relevant governing educational bodies a letter outlining the proposed research and an invitation to participate was mailed to sixty-four secondary schools throughout tasmania. a draft survey was included with this correspondence. this contact was subsequently followed up by telephone. as a result twenty-six schools agreed to participate. results this section reports the results obtained in the more comprehensive wiitt research that relate to the specific focus of this paper. the reporting of these results will be preceded by some demographic results to establish a profile of the schools and students who took part in the research. the main section of the results will report the outcomes from the statistical analyses conducted from three perspectives of the career characteristics with respect to ideal careers and then those associated with careers in ict. these will involve a series of comparisons based on sixteen given career characteristics. within these initially the views of boys and girls in relation to their ideal careers will be examined. this process will then be repeated with respect to careers in ict. the final comparison will solely focus on girls and the extent any match or mismatch their views between their ideal career and careers in ict. demographic results twenty of the twenty-six schools that originally agreed to support the research returned questionnaires. although reminder letters were sent to the outstanding schools no further responses were forthcoming. in the context of the sampling it was not feasible to attempt to calculate the actual response rate. to do this would have necessitated obtaining figures related to parental dissent and student absences on the day the questionnaire was administered or the number of students unwilling to participate in the survey. the quantification of this information would have placed an added burden on the schools concerned and so may have proved counter productive to enlist their support for the study. table 1 provides a summary of the school type attended and gender distributions of the 1634 tasmanian year nine students who participated in this research. as shown, slightly more girls (53%) than boys (47%) completed questionnaires. overall nearly two thirds of the responses were received from high schools. a further 22% of responses came from non-catholic independent colleges, 9% of students attended catholic ajis vol. 10 no. 2 may 2003 119 colleges while 4% came from district high schools. in high schools a larger proportion of girls (39%) than boys (27%) contributed data to the research. by gender the distribution was the reverse in non-catholic independent colleges and catholic colleges with slightly more boys that girls taking part in the survey. boys and girls were equally represented in the district high schools that took part in this research. table 1 breakdown of year nine responses by school type and gender main results in this section of the results from two questions applied in the wiitt questionnaire are reported. these will consider the outcome following a statistical analysis from the three defined perspectives. that is, it will include an examination of the responses obtained from girls and boys to the given 16 careers characteristics with respect their ‘ideal’ careers and then in relation to careers in ict. the final perspective of results will reconsider these two career forms focussing solely on girls. the perceptions of career characteristics for ideal careers from a gender perspective the first result reported in this section is the outcome when girls and boys were asked to indicate the extent of agreement or disagreement with the sixteen proposed features to describe their ideal careers. table 2 shows the results when independent sample t-tests were applied to the data. this revealed little agreement between what girls and boys regard as important in their ideal careers. the variation mean differences were found to be statistically non significant in only four of the sixteen proposed characteristics. these were interesting challenges, travel, flexible hours and job security. for interesting challenges, the mean scores of 4.13 and 4.08 were shown to be non significant (t1483=. 348, p>.05). this was also true for travel (t1483=. 460, p>.05), flexible hours (t1479=. 828, p>.05) and job security (t1460=. 373, p>.05). the t-tests revealed that the differences in the mean scores for the remaining twelve characteristics were statistically significant. this result serves to highlight the extent to which girls and boys place different emphasis on issues that constitute their ideal careers. girls are more inclined to want to be involved in interesting work and working with people. the extent of the standard deviation for these two career characteristics (.79 and .84) supports that generally there was considerable consensus among girls on these issues. more girls than boys strongly agreed that creativity, responsibility, socially useful work and independence were characteristics they perceived as important in describing their ideal careers. boy participants indicated that a high salary, job security, high status, a cool image and the opportunity for self-employment were preferred characteristics in their ideal careers. on the basis of gender, the differences in the mean scores significant in relation to working alone and working from home. within these results the responses from girl respondents tended more towards strongly disagreeing that these were features seen as desirable for their ideal careers. district high school high school non-catholic independent college catholic college total % % % % % female 1.7 38.9 9.5 3.2 53.3 male 1.7 26.8 12.6 5.4 46.5 3.4 65.7 22.1 8.6 100 n=1634 ajis vol. 10 no. 2 may 2003 120 table 2 mean differences ‘ideal’ careers of year nine girls and boys girls boys levene test t n= mean sd n= mean sd f sig. sig. interesting work 810 4.52 .79 684 4.27 1.00 56.367 .000* .000* working with people 814 4.21 .84 693 3.94 1.14 27.327 .000* .000* interesting challenges 804 4.13 1.07 681 4.08 1.08 3.335 .068 .348 creativity 796 3.99 1.12 674 3.76 1.25 32.719 .000* .000* responsibility 801 3.97 1.05 678 3.83 1.22 35.721 .000* .020* high salary 803 3.96 1.10 685 4.13 1.03 .148 .700 .002* socially useful work 799 3.92 1.06 672 3.68 1.22 44.834 .000* .000* travel 803 3.89 1.23 682 3.84 1.24 .166 .684 .460 flexible hours 805 3.80 1.21 676 3.82 1.23 1.453 .228 .828 independence 792 3.68 1.19 670 3.50 1.33 30.301 .000* .005* job security 794 3.56 1.24 668 3.62 1.30 1.357 .244 .373 high status 786 3.51 1.20 672 3.73 1.21 1.803 .180 .001* cool image 798 2.83 1.40 677 3.24 1.41 .122 .727 .000* self-employment 791 2.70 1.26 678 3.06 1.40 19.965 .000* .000* working alone 791 2.23 1.21 676 2.51 1.37 37.020 .000* .000* working from home 789 2.20 1.19 671 2.55 1.38 57.878 .000* .000* *∂ = <.05 1=strongly disagree 5=strongly agree the perceptions of career characteristics for ict careers from a gender perspective table 3 reports the results when the same career characteristics were used to assess the extent to which girls and boys agreed or disagreed with these as features of ict careers. the results highlight that there is considerable alignment in how girls and boys perceive ict careers. based on levene tests and subsequent appropriate t-tests, on the basis of gender the differences were found to be not statistically significant for twelve of the sixteen career characteristics. regardless of gender the participants tended to strongly agree that interesting work, interesting challenges, high salary, working with people, creativity, socially useful work, flexible hours, high status, working alone, travel, working from home and self employment were features of ict careers. there were four career characteristics in relation to ict careers where there were significant differences between the mean score results. within these while girls more strongly agreed that careers in the industry involved responsibility (t-3.405=.001, p<.05) independence (t-3.4.861=.000, p<.05) and job security (t2.592=.012, p<.05) more so than boys they disagreed that ict careers involved a cool image (t-5.627=.000, p<.05). ajis vol. 10 no. 2 may 2003 121 table 3 mean differences ict careers of year nine girls and boys girls boys levene test t n= mean sd n= mean sd f sig. sig. responsibility 784 3.79 1.13 670 3.58 1.27 30.702 .000* .001* interesting work 786 3.77 1.29 679 3.99 1.26 5.569 .018* .22 interesting challenges 777 3.77 1.14 675 3.75 1.25 11.349 .001* .751 independence 776 3.71 1.15 666 3.39 1.28 26.804 .000* .000* high salary 782 3.67 1.12 671 3.75 1.22 3.753 .053 .183 working with people 793 3.62 1.20 686 3.52 1.33 18.243 .000* .121 job security 785 3.57 1.13 667 3.41 1.29 30.460 .000* .012* creativity 782 3.54 1.26 673 3.44 1.36 12.601 .000* .127 socially useful work 784 3.48 1.22 663 3.56 1.25 .002 .966 .181 flexible hours 780 3.37 1.21 674 3.43 1.30 4.800 .029* .401 high status 774 3.29 1.22 666 3.42 1.26 .456 .499 .052 working alone 781 3.20 1.32 670 3.08 1.37 1.365 .243 .099 travel 780 3.10 1.37 676 3.16 1.42 1.983 .159 .06 working from home 769 3.06 1.30 668 3.09 1.38 3.758 .053 .687 self-employment 778 3.00 1.28 667 3.12 1.35 5.370 .021* .088 cool image 780 2.59 1.29 666 2.98 1.36 2.830 .093 .000* *∂ = <.05 1=strongly disagree 5=strongly agree a comparison of the perceptions of girls between ideal careers and ict careers to determine the extent of match between the view of girls in relation to their ideal careers and careers in ict, paired t-tests were conducted on the sixteen career characteristics. because the analysis involved paired results in some areas the numbers will vary to those presented in the two earlier tables. the reason for this is that missing data may have precluded some cases being included in the test. the results are provided in table 4. this shows that in only two career characteristics, independence (t-.157=.876, p>.05) and job security (t.076=.939, p>.05), the differences in the mean scores proved to be statistically non significant. this implies that girls tend agreed to the same extent that these are characteristics of both their ideal careers and careers in ict. statistically significant differences were found between the mean results for ideal careers and ict careers for the remaining fourteen career characteristics. a comparison of the mean scores shows that girls that were more inclined to perceive interesting work, working with people, interesting challenges, creativity, responsibility, high salary, socially useful work, travel, flexible hours, high status and a cool image as components of their ideal careers than features of careers in ict. in addition, as shown by the mean results, girls associated self-employment, working alone and working from home with ict careers of least importance in their ‘ideal’ careers. ajis vol. 10 no. 2 may 2003 122 table 4 a comparison of the results from girls between ideal and ict careers ideal career ict career n= mean sd mean sd t sig interesting work 781 4.51 .79 3.78 1.20 14.412 .000* working with people 789 4.20 .85 3.62 1.21 11.729 .000* interesting challenges 771 4.14 1.02 3.77 1.14 7.123 .000* creativity 769 3.99 1.12 3.54 1.27 8.148 .000* responsibility 770 3.98 1.03 3.79 1.12 4.240 .000* high salary 770 3.96 1.10 3.67 1.12 6.108 .000* socially useful work 770 3.94 1.05 3.47 1.21 8.880 .000* travel 775 3.90 1.23 3.09 1.37 13.155 .000* flexible hours 772 3.80 1.21 3.37 1.21 8.111 .000* independence 764 3.70 1.18 3.71 1.15 .157 .876 job security 769 3.57 1.24 3.56 1.12 .076 .939 high status 753 3.52 1.20 3.29 1.21 4.443 .000* cool image 767 2.83 1.40 2.58 1.28 4.620 .000* self employment 767 2.71 1.26 3.00 1.28 -5.413 .000* working alone 765 2.22 1.21 3.19 1.32 -16.416 .000* working at home 754 2.21 1.16 3.06 1.30 -15.372 .000* *∂ = <.05 1=strongly disagree 5=strongly agree discussion and conclusions this research has determined that while girls have different ideas to boys as to what they want in their ideal careers, these students shared highly similar views in their perceptions of ict careers. in addition when the results between ideal careers and ict careers of the girl students were compared, then this demonstrated that generally they failed to recognise the extent to which a career in ict could fulfil their career goals. with respect to their ideal careers girls placed different values than boys on a majority of the career characteristics applied in this research. this outcome has provided the basis to determine the features that need to be promoted to attract the interest of girls towards careers in the ict industry. girls most highly valued interesting work, working with people and interesting challenges so accordingly the message needs to be conveyed that these are integral components of careers in the modern ict business arena. the primary focus on these three features as representative of ict careers could also provide the means to diffuse the common view of the industry as a highly technological and solitary field. as such this could reduce the dominance of technology and social isolation as key barriers preventing many women from embarking on careers in the ict field. while working alone, working from home and self-employment are options within the industry, the research has shown these were not highly regarded as components in the ideal careers of girls. possibly because of the importance girls placed on working with people, working alone and working from home were the least desired features of the their ideal careers. the implications are that the notion of ict as a solitary career could still persist to dissuade girls from considering careers in ict as a viable choice. in addition while traditionally self-employment has represented a large aspect of ict careers, the research has shown that girls were clearly not attracted to this form of employment. the results for these career characteristics supports that any promotional ict career information needs to address the potential negative effects of these aspects of the industry on the career choice of girls. this implies that while these need to be acknowledged as features of ict careers, an emphasis must made that engagement in these forms of career involvement is optional and up to individual choice. the results from perceptions of careers in ict on the basis of gender showed that generally girls and boys held similar views. however there were four exceptions. girls, more so than boys believed that ict careers involved responsibility, independence and job security whereas boys supported that this form of career presented a cool image. the comparison of the responses from girls between ideal careers and careers in ict showed a considerable mismatch. from the results only two career characteristics, independence and job security, were similarly regarded in both their ideal careers and careers in ict. however, these were only low priorities as features of their ideal careers. as a result of this comparison the scope of the issues that need to be promoted to more fully inform girls about careers in ict have been determined. clearly girls need to be made aware that much of the work performed by ict professionals is interesting and challenging and involves working with people. what is important also is that all aspects need to be addressed even though girls do not necessarily perceive some of them to be ideal in their future careers. ajis vol. 10 no. 2 may 2003 123 a secondary objective of this research was to consider the means by which information about ict careers can be distributed. while a number approaches have been proposed in the literature what is important is that such information should reach a wide audience and not just target secondary school girls. this is essential to ensure that when girls call on parents and friends for career advice, those offering this support are aware the scope and diversity modern day ict careers have to offer. a natural follow on from the wiitt research would be to use tasmania as a test case. an initial aim could be to distribute true to life information about the roles and activities women undertake in the course of their employment as ict professionals to the widest possible audience. this could take the form of a weekly diary appearing as a newspaper column that details the day to day work of women currently engaged in ict careers. ideally this will provide an opportunity to emphasise the extent to which they interact with people, the challenges they face and the interesting nature of their work. a potential benefit of this approach is that because of their participation in the wiitt research, many tasmanian secondary school girls will have a heightened awareness of ict careers. on this basis they will be more inclined to read this information and discuss it with their parents and friends. a second stage initiative could be for a professional organisation, such as wiitt or the australian computer society (acs), to arrange a series of visits by women ict professionals to schools especially during careers expos. the measurement of the effectiveness of these approaches could then form the basis for future research into the buying behaviours of girls in relation to careers in ict. references ahuja, m. k. (2002). women in the information technology profession: a literature review, synthesis and research agenda. european journal of information systems 11: 20-23. american association of university women (2000). tech-savvy: educating girls in the new computer age. url: http://www.aauw.org/2000/techsavvybd.html accessed november 14, 2000 bana, s. and s. hassoun (1997). improving the graduate school environment for women in computer science. url: http://www.cs.washington.edu/homes/soha/gh/list.html accessed may 18, 2002. bowes, j., t. cowley, price, k. (2001). tasmanian student readiness for the it industry. hobart, tasite. camp, t. (1997). the incredible shrinking pipeline. communications of the acm 40(10): 103-110. centre for international economics. (2001) breaking the skills barrier-demonstrating the benefits of investment in ict higher education in australia. centre for international economics. sydney. chan, v., k. stafford, klawe, m, chen, g. (1998). gender differences in vancouver secondary students' interests related to information technology careers. url: http://taz.cs.ubc.ca/swift accessed february 14th, 2002. cuny, j. aspray, w. (2000). recruitment and retention of women graduates students in computer science and engineering. washington, computer research association. delaney, dyson, b.c. (1998). women: creating the connection, strategic planning and research division, office of training and further education. foster, j. j. (2002). data analysis using spss for windows. sage publications. london frenkel, k. a. (1990). women and computing. communications of the acm 33(11): 34-46. gurer, d. (1999). testimony to the committee on workforce needs in information technology. url: http://www.acm.org/women/work-shortage.shtml accessed september 14, 2000. ho, r. (2002). handbook of univariate and multivariate data analysis & interpretation: spss approach. central queensland university publishing unit. queensland. igbaria, m. baroudi, j.j. (1995). the impact of job performance evaluations on career advancement prospects: an examination of gender differences in the is workplace. misq 19(1 march): 107-125. information industries bureau (2000). queensland ict industry profile 1999/2000, queensland government innovation and information economy jewell, h. maltby, j.r. (2001). female involvement in information technology degrees: perception, expectation and enrolment. proceedings of the tenth australasian conference on information systems, southern cross university. juliff, p. l. (1998). it skills essence of any future success. the australian: 44. levenson, n. g. (1990). educational pipeline issues for women. lundy, s. k. (1999). 21st century women. address to fitt luncheon, july 14. margolis, j. fisher, a. (1997). geek mythology and attracting undergraduate women to computer science.url: http://www.cs.cmu.edu/~gendergap/papers/wepan97.html. accessed november 9, 2000. ajis vol. 10 no. 2 may 2003 124 meares, c. a. sargent, j.j.f. (1999). the digital workforce: building infotech skills at the speed of innovation, us department of commerce. moon, s. (2001). set for life. canberra, university of canberra. multimedia victoria (2001). reality bytes an in-depth analysis of attitudes about technology and career skills. melbourne, victorian state government. newmarch, e. taylor-steele, s. cumpston, k. (2000). women in it what are the barriers?, department of education training and youth affairs. nielsen, s. von hellens, l. wong, s. (2001). the male it domain: you've got to be in it to winit. proceedings twelth australasian conference on information systems, coffs harbour. nielsen, s. von hellens, l. wong, s.. (2000). the women in information technology project: uncovering the pride and prejudices. proceedings of the sixth australian women in computing workshop, brisbane. nielson, s., von hellens, l. wong, s. (2000). the game of social constructs: we're going to winit. proceedings of the international conference on information systems, brisbane. o'leary, d. (1999). accessibility to computer science: a reflection for faculty members. url: http://www.cs.umd.edu/~oleary/faculty.whole.html. accessed april 6, 2002. pringle, r., s. nielsen, et al.. (2000). net gains success strategies of professional women in it. proceedings of the 7th international conference of women, work and computerization, ifip tc9 wg9.1, british columbia. queensland government (1998). it&t skills in the smart state. url: http://www.iib.qld.gov.au/skills/it&t20skills%20paper/it&t%20skills/ index.html. accessed february 13, 2002. roepke, r. p. agarwal, r. ferratt, w. (2000). aligning the it human resource with business vision: the leadership initiative at 3m. misq (forthcoming). selby, l.young, a. fisher, d. (1997). increasing the participation of women in tertiary level computing courses: what works and why. proceedings of ascilite97, perth. selby, l. m. (1997). a critical analysis of factors affecting the participation of women in tertiary level computer science and information systems courses. science and mathematics education centre. perth, curtin university of technology. spertus, e. (1991). why are there so few female computer scientists?, mit. staehr, l., m. martin, et al.. (2001). computer attitudes and computing career perceptions of first year computing students.informing science(june): 502-509. swanson, g. and p. wise (1996). digital futures: women's employment in the multimedia industry. brisbane, australian key centre for cultural and media policy, griffith university. swift (2001). swift career choice report. british columbia, university of british columbia. url: http://www.taz.cs.ubc.ca/swift/. accessed january 31, 2002. tasmanian department of state development (2000). industry audits: information technology, communications and advanced technology. url: http://www.dsd.tas.gov.au/inaud2.htm. accessed january 31, 2002. teague, j. (1998). recruiting computing professionals expanding the recruitment base. working paper 198/3. deakin university. the acm committee on women in computing (1998). computer gender gap prompts major study by acm. url: http://www.acm.org/announcements/acmwnsf.html. accessed november 9, 2000. trauth, e. nielson, s. von hellens, l. (2000). explaining the it gender gap: australian stories. proceedings of the tenth australasian conference on information systems, brisbane. von hellens, l. nielson, s. trauth, e. (2001). breaking and entering the male domain. women in the it industry. proceedings acm sic cpr, san diego. von hellens, l. a. pringle, r. greenhill, a. nielson, s. (2000). people, business and it skills: the perspective of women in the it industry. proceedings acm sigcpr, chicago. young, j. (2002). increasing the participation of women in information, communication technology careers. women in it tasmania. available on line at http://www.acs.org.au/tas/wiitt/wit_report_v4.pdf. acknowledgements the author wishes to acknowledge women in it tasmania (wiitt) and the intelligent island program. this project was undertaken by wiitt with the support of the intelligent island program. the intelligent island program is funded by the commonwealth government and is being developed in partnership with the tasmanian government. ajis vol. 10 no. 2 may 2003 125 copyright judy young © 2003. the author assigns to auswit2003 and educational and non-profit institutions a nonexclusive licence to use this document for personal use and in courses of instruction provided that the article is used in full and this copyright statement is reproduced. the author also grants a non-exclusive licence to auswit2003 to publish this document in the proceedings. those documents may be published on the world wide web, cd-rom, in printed form and on mirror sites on the world wide web. any other usage is prohibited without the express permission of the author. microsoft word special final.doc ajis special issue december 2004 133 does touching a projection augmented model and interacting with it using a spatiallycoincident device, affect a user’s perception of its size? emily bennett brett stevens department of information systems and computer applications university of portsmouth uk emily.bennett@port.ac.uk abstract a projection augmented model (pa model) is a novel type of display. it consists of a real physical model, onto which a computer image is projected to create a realistic looking object. pa models provide their users with whole-hand haptic feedback and support spatially-coincident haptic interaction devices. this paper reports on an experiment that investigated the effect these factors have on a user’s perception of the size a pa model. results showed that touching a pa model increased the accuracy of size estimates; however using a spatially-coincident haptic interaction device had no effect. keywords (projection augmented models, haptic, tangible user interface, size perception) introduction figure 1: paris (main picture), paris without the projected image (inset) a projection augmented model (pa model) is a type of projection based augmented reality display. it consists of a physical three-dimensional model, onto which a computer image is projected to create a realistic looking object (stevens et al. 2001). for example, paris (projection augmented reality information system) consists of smooth white plaster models of objects commonly found in a garden shed. the image projected onto these objects provides colour and visual texture (figure 1). the current paper reports on a user study that investigated what effect touching a pa model has on the perception of the size of its features. it is important to note when this paper refers to ‘touch’ or ajis special issue december 2004 134 ‘haptic’ it is actually referring to the situation when a pa model is being simultaneously viewed and touched. to put this study in context, it is first necessary to review the capabilities, applications and limitations of pa models. projection augmented models to create a pa model, the image that is normally viewed on a pc monitor is projected directly onto a physical model. the projector and the physical model must be carefully aligned, so the projected image is correctly registered onto the physical model. technology has been developed that semiautomates this process (raskar et al. 2001), however it is possible to do this manually. multiple projectors can be used to illuminate the pa model on all sides, and thus allowing the pa model to be viewed from any direction. the projected image can change the perceived colour of the physical model. it can also make it appear to be made from a different material. for example, a dull clay vase can appear to be made out of a shiny ceramic material (raskar et al. 2001). different lighting conditions can also be simulated (raskar et al. 2001). additionally, by projecting an animated image onto the static physical model, apparent motion can be simulated (raskar, ziegler & willwacher 2002). text or other information can be added to the projected image. for example, the image projected onto a pa model of a fossil can include annotations (bimber et al. 2002). examples of applications for which pa models are currently being developed include product design (verlinden et al. 2004), geographical visualization (piper, ratii, & ishii 2002) (clark et al. 1998) and museum exhibits (exhibition at the museum of london 2004) (bimber et al. 2002). pa models have several advantages over other types of displays. a pa model gives a threedimensional physical shape to a two-dimensional computer image. people can touch the surface of a pa model with their bare hands. therefore pa models provide whole-hand haptic feedback, without the need to wear any additional equipment. additionally, multiple people can simultaneously view a pa model. each person has their own unique view, which is dependant on their physical location. this is because pa models are real physical objects therefore each person positioned around it, perceives it from a different direction. for example, people seated on the left side of paris (figure 1) would see the left side of the objects (e.g. plant pots and bricks), whereas someone standing up and looking down would see the top of the objects. this can be contrasted with other displays such as a standard monitor, which present an image from a single viewpoint, and everyone sees the same image regardless of their location. some displays can support multiple users, with each user having their own unique position-dependant viewpoint. however, the number of users these displays can support is limited by the current technology, whereas the number of multiple users that a pa model can support is only limited by the physical space around the model. despite the potential benefits and applications of pa models, they have some limitations. these limitations highlight the ways in which pa models may differ from their real life counterparts. the most notable limitation is that pa models normally have a fixed physical shape. although deformable pa models have been built e.g. (piper, ratii, & ishii 2002) (iwata et al 2001), they can only change shape within a limited range. a second limitation is that there is a perceptual mismatch between visual and haptic feedback for some properties of a pa model. for example, a pa model cannot simulate properties such as physical texture. this is because to enable it to be possible to project onto a pa model, the physical model must have a smooth diffuse surface. therefore when a person touches a pa model, its visual texture (from the projected image) does not match its physical texture. for example, when a person touches a pa model that looks like it is made from metal, they realize it does not feel like metal (raskar et al. 2001). however, it is worth pointing out that this limitation is only relevant when a pa ajis special issue december 2004 135 model represents an object on a real life scale. this is much less likely to be an issue for pa models that represent objects on a different scale to real life. a further problem is that when a person touches a pa model, the projected image appears on the back of their hand, and their hand casts a shadow onto the pa model. multiple projectors and computer vision techniques can reduce this problem (mukaigawa, nagai & ohta 2004). this problem can be overcome completely by using a back-projected pa model e.g. (clark et al. 1998). research project and research question the research presented in this paper is part of a larger project. to put this paper in context, it is important to outline the aims and objectives of this project. project overview the research into pa models has focused on developing the enabling technology; very few studies into how people use pa models have been conducted. it is important to carry out user studies on a new type of display early in its development (lampton, bliss & morris 2002). when conducting human centred studies with new displays that do not have a main application area, it is advisable to evaluate them in terms of ‘basic’ tasks, such as perception and memory (hale and stanney 2004 ; kirkpatrick & douglas 2002). a main application area for pa models has not yet emerged; therefore this project will focus on ‘basic’ tasks to allow guidelines to be created that can be generalised to many applications. a review of pa models has shown that they provide extensive haptic feedback because the user can touch them with their bare hands. however, it has been noted that touching a pa model may actually reduce its realism. the feedback provided by pa models is of particular interest because it could both benefit and hinder the user, depending on the task they are performing. pa models should be considered as part of a user interface system, any user interface system needs both a display and an interaction device (latta & oberg 1994). therefore both haptic feedback and haptic interaction are important and they should be studied together. the aim of this research project is to create guidelines that will enable a person who is interested in using a pa model for a particular application, determine the best way of making use of the haptic feedback and the haptic interaction devices that pa models can support, to meet the requirements of that application. these guidelines will indicate which tasks benefit from touching and haptically interacting with a pa model. they will also suggest the factors for which touch and haptic interaction may have a detrimental effect. ajis special issue december 2004 136 figure 2a: interacting with a pa model using a mouse, figure 2b: interacting with a pa model using a tui to clarify the aims of this project further; the term ‘haptically interacting’ refers to interaction devices that enable the user to perceive the haptic properties of a pa model. such devices can be referred to as spatially-coincident devices. an example of a spatially-coincident device is a tangible user interface (tui). a tui allows a person to use small physical objects to interact with digital information (ullmer & ishii, 2001). these physical objects are referred to as phicons (physical icons). a tui can be used to interact with a pa model by allowing the user to place a phicon directly on its surface (figure 2b). spatially-coincident interaction devices can be contrasted with spatially-separate devices. when using a spatially-separate device, the action performed by the user is in a different location from the pa model, thus the pa model is not haptically perceived. an example of a spatially-separate device is a mouse. when using a mouse, the user moves it in a separate location to the mouse pointer, which is projected onto the physical model (figure 2a). aims of the first study and scope of this paper as part of this project, the first study was conducted to investigate the effect that touching a pa model and interacting with it using a spatially-coincident device, has on users’ perception of its size and their sense of object-presence. object-presence is the feeling that an object exists in a person’s environment, when that object does not (stevens et al 2001). this paper focuses on the size perception results. it is important to note that object-presence was investigated using a post-test questionnaire, and a between-participants experimental design was used. therefore the measurement of object-presence could not have affected the size perception results described in this paper. hypotheses the research question this paper investigates is: what effect does touching a pa model and interacting with it using a spatially-coincident device, have on users’ perception of its size? separate hypotheses can be made about the effect of touching and the effect of using a spatially-coincident interaction device. ajis special issue december 2004 137 hypothesis 1 the first issue this study investigates is concerned with the effect touching a pa model has on the accuracy of the users’ size estimates. it has been shown that vision is the dominant sense for perceiving size (ernst & bülthoff 2004). this suggests that touching a pa model will not increase the accuracy of the perception of its size. however, there is some conflicting evidence, for example it has been shown kinesthetic feedback dominates over vision for the perception of depth (beers, woolpert & haggard 2002). moreover, it has been demonstrated that perception is more accurate when both visual and haptic feedback are combined. several models have been proposed which all suggest that more accurate perception can be achieved by combining the senses, see (ernst & bülthoff 2004) for a review. for example, the optimal integration model predicts that the way in which humans combine sensory information depends on the situation, and in some cases more accurate perception can be achieved by using multiples senses (ernst & banks, 2002). this model is supported by empirical research that found the addition of haptic feedback to visual feedback increased the accuracy of size estimates (gepshtein & banks, 2003). therefore it is hypothesized that the accuracy of size estimates will be higher when a pa model simultaneously viewed and touched, compared to when it is just looked at. it is important to note that it may appear that this hypothesis has already been tested, however the visual feedback provided in gepshtein et al’s (2003) experiment only contained the visual depth cue of stereopsis, it did not contain other depth cues such as texture gradient. it is possible that if more depth cues were provided, the haptic feedback may not have had such an effect on perception because the visual information would have been sufficient for the participants to accurately perceive size. given that a pa model provides feedback for all visual depth cues, gepshtein et al (2003) does not provide conclusive evidence that touching a pa model will increase the accuracy of perception. indeed, it has been argued that it is important that research into multi-sensory integration and perception (such as this study), should be ecologically valid (de gelder & bertelson 2003). therefore it is necessary to conduct empirical research with actual pa models, opposed to relying on previous work. hypothesis 2 the second issue that this study is investigating is concerned with the effect that interacting with a pa model using a spatially-coincident interaction device has on the accuracy of the users’ size estimates. to operationalise this question, the two interaction devices that this study focuses on are a mouse (spatially-separate) and a tui (spatially-coincident) (see section 3.1 for details). spatiallycoincident interaction devices such as tuis, require the user to physically touch the surface of the pa model. this means that the users haptically perceive the pa model’s physical shape, while they are interacting with it. thus spatially-coincident devices such as tuis, provide kinesthetic (limb position) feedback and single-point force feedback. this may have an effect on how accurately the users’ can estimate the size features of a pa model. the decision was made to investigate a priming effect because it more accurately reflects how user interfaces are used in real life. in real life, a user may be asked for information (e.g. a size estimate) about the display that they have been interacting with. in making this estimate, they may implicitly use information they gained previously while they were interacting with the display. however, it is unlikely they would actually use the interaction device to make their estimate. therefore this hypothesis is concerned with size estimates made a short time after using a particular device. (opposed to the first question in section 4.1 that is concerned with size estimates made almost simultaneously to the visual or haptic feedback being perceived). ajis special issue december 2004 138 the same research into the effect of combining visual and haptic feedback, which lead to the first hypothesis (section 4.1), is also relevant to this question. this research suggests that using a tui is likely to result in people making more accurate size estimates. this is supported by empirical evidence that is directly relevant to this question. it has been demonstrated that combining visual and kinesthetic feedback results in a more accurate estimate of the position of a finger, compared to using visual feedback alone (beers, woolpert & haggard 2002) (beers, stittig & denier 1996). this suggests that the kinesthetic feedback gained from using a tui will enable people to made more accurate size estimates, compared to if they used a mouse that does not provide kinesthetic feedback. however, caution must be taken in applying previous research to this study. this study is concerned with size estimates that are made a short time after the interaction has taken place. however, the research into the effect of combining sensory modalities is concerned with measures taken whist or directly after the stimulus is perceived. therefore memory is also a factor that needs to be considered. according to the information processing approach, the perceptual system and memory are linked, which suggests that if a stimulus is more accurately perceived, then it will be more accurately remembered (eysneck & kean 2001). this indicates that previous research which demonstrated combining modalities results in more accurate perception, also indicates that memory will be more accurate. therefore it is hypothesized that the accuracy of size estimates will be higher after a tui is used, compared to after a mouse is used. empirical study method a total of 80 participants (55 male 25 female) were used. they were all students on computingrelated degree courses. the mean ages were 24 for males and 20 for females. an equal number of male and females were used in each condition (+/-1). paris (projection augmented reality information system) (figure 1) is the pa model that was constructed for this experiment. this pa model consisted of white plaster models of various objects that are commonly found in a garden shed. the image projected onto these objects gives them colour and visual texture. paris represents objects that are likely to be familiar to the participants. although this was necessary for the part of the study that investigated object-presence (not reported in this paper), it is a potential problem for this part of the study. participants’ size estimates may be affected by their previous knowledge. however, this is unlikely to be an issue because apart from the bricks, none of the objects in paris have a standard size in real life. moreover, most of the size estimates were of distances between objects or of parts of objects. haptic visual mouse tui f e e d b a c k i v device iv figure 3: experimental conditions ajis special issue december 2004 139 a 2 x 2 between-participants factorial design was used (figure 3). there were two ivs; device and feedback (figure 3). figure 4a: mouse condition, figure 4b: tui condition, figure 4c: size estimate markers the device iv was split into two levels of mouse and tui. the device conditions were to prime the participant with either using a tui or a mouse (hypothesis 2). the participants were required to complete a manipulation and target acquisition task. in the mouse conditions this involved the manipulation of a projected green square to select various parts of the pa model (figure 4a). in the tui conditions they had to use a phicon (plastic snail) to select various parts of the pa model (figure 4b). the feedback iv was split into two levels of visual and haptic. participants in both conditions were seated in front of the apparatus. they were asked to make size estimates of various features of paris. pairs of markers were projected onto paris (figure 4c). the participants had to give an estimate of the distance between them (from their furthest most points). participants in the visual condition made their estimate by just looking at the markers. participants in the haptic condition made their estimate by both looking at and touching the markers. precisely how the participants gave their size estimates is explained in section 5.11. the procedure of the experiment is as follows: 1. participants completed the priming task by interacting with paris using either a tui or the mouse (device iv). 2. participants then made size estimates of various parts of paris. they did this by either looking at it, or by both looking at it and touching it (feedback iv). their size estimates were recorded. 3. finally, participants completed object-presence questionnaires (not reported in this paper). measures certain restrictions on how participants made their size estimates were necessary to ensure the experiment was valid. the participants in the haptic conditions were told to touch the two dots with their thumb and index finger. if the distance between the dots was too great, they were told to use the index fingers on both hands. this created a slight problem because the participants all had different size hands. hence, there were some sizes which some participants made with one hand and others which they made with both hands. this problem was over come by using range of sizes, which meant all participants made some judgments with one hand and some with both hands. once they had touched the markers, they had to place their hand flat down on the table, to ensure they lost the position of their fingers. ajis special issue december 2004 140 a further restriction was imposed on the participants in the haptic conditions. they were not allowed to touch the markers and hold their fingers in a fixed position, and then rotate their hand to get a better view of the distance between their fingers. if they did this, they would actually be making judgments based on visual feedback. it could be argued that this restriction is not ecologically valid because in real life people would be free to do this. however, by allowing people to do this, it would mean the experiment would really be investigating the effect of changing the viewpoint, as opposed to the effect of haptic feedback. this leads to a further restriction that was imposed in both the haptic and the visual conditions. participants were instructed not to move forward or change their angle of view when making the estimates. this instruction aimed to ensure that everyone perceived all of the sizes from roughly the same viewpoint, thus minimize the difference in the visual feedback received by participants. the decision of how participants should actually give their size estimates was problematic. it has been argued that when comparing visual and haptic senses, the results can be affected by the method used to record the participants’ response (welch & warren, 1980). it is therefore necessary to explain the measures used in this experiment. perception of size can be measured using either relative or definite estimates. the procedures required and the type of data generated from relative estimate tasks, were not suitable for this experiment (bingham & pagano 1998). definite estimates can be measured using either verbal or action estimates. verbal estimates involve giving a verbal estimate in specific units, for example in centimetres. however, it has been found that verbal estimates vary considerably from person to person (bingham et al., 1998). this problem can be overcome to some extent, by placing a calibrating device such as a ruler near the object that is being perceived (bingham et al., 1998). however, it has been shown that visual feedback dominates when this method is used (heller, calcaterra, green & brown, 1999). this suggests that this method is not suitable when studying multi-sensory perception because it may not be sensitive enough to pick up information gained through the haptic sense. therefore a verbal estimates measure was not used. action estimates are the other type of definite judgment that can be used to measure the perception of size. an action estimate typically requires the participant to view an object, and then give a size estimate by making an action, such as reaching out an arm (pagano, grutzmacher & jenkins, 2001). importantly, the participant cannot see their hand or arm while making their estimate. it has been argued that action estimates are a more valid way of measuring perception than verbal estimates (see bingham et al., 1998 for a review). however, it has been found that the haptic sense dominates when this method is used (heller et al., 1999). this suggests that this type of measure may be more sensitive to haptic feedback than to visual feedback. therefore, if this method was used and size was more accurately estimated in the haptic conditions, it may actually be because the method was very sensitive to haptically perceived information. an alternative to this type of action estimate is to allow participants to see their hands while they are making their judgement. however, this brings the opposite problem to the one described above. it has been argued the reason that many studies have shown vision to be the dominant sense in the perception of size, is because this type of method was used (heller & schiff, 1991). it has been suggested that this type of method is not sensitive enough to detect haptically perceived information (heller et al., 1991). therefore, if this method was used and there was no difference in the accuracy of the size estimates between the haptic and visual conditions, this may actually be because the method is not sensitive enough to detect haptically perceived information. given these problems with both types of action estimate, if either estimates were used alone, it would not be possible to gain a full and accurate picture of the effect ivs may have had. therefore two different action estimates of size were used; one where the participant could not see their hands (nosight), and another where they were able to see their hands (sight). the participants were required ajis special issue december 2004 141 to make a pincer movement with their fingers to indicate their size estimate. the experimenter recorded the distance between their fingers. a total of 20 different sizes were used (20 pairs of markers). for each pair of markers the participant gave a sight and a no-sight estimate, thus two sets of data were obtained. the markers were displayed while the participants gave their estimate. roughly an equal number of height, width and depth estimates were used. if the size was too big, they were instructed to use both hands when giving their estimates. importantly, participants in the haptic conditions were instructed to use the same hand when giving their estimate, as they used when perceiving the size. this is because there is no automatic perceptual connection between the hands (lange, klatzky & ernst, 2004). results two sets of size estimates were obtained; sight estimates and no-sight estimates (section 5.11). for both data sets, the accuracy of the participants’ estimate for each distance was calculated by taking away their estimate from the correct distance. thus, a positive number indicated an overestimate and a negative number indicated an underestimate. from both the sight and no-sight data sets two values will be investigated; an absolute value and a signed value. the absolute for each participant is the mean of his or her 20 accuracy values, after turning all underestimates into positive numbers. this gives an overall indication of how accurate a person was. thus higher values indicate a higher level of error in their estimates. a very brief summary of the absolute values results has been published previously (bennett & stevens 2004). the signed value for each participant is the mean of his or her 20 accuracy values, when the signs of the estimates are not changed. this gives an indication of the direction of the error. positive signed-values indicate on average the participant is making overestimates, and negative signedvalues indicate on average the participant is making underestimates. the results from the two data sets are presented separately; the sight estimate results are presented first (both absolute and signed values), then the no-sight estimate results (both absolute and signed values). sight estimate feedback hapticvisual e rr o r (c m ) 3.2 3.1 3.0 2.9 2.8 2.7 2.6 device mouse tui haptic visual mouse tui f e e d b a c k i v device iv mean: 3.12 s.d: 0.53 mean: 2.71 s.d: 0.45 mean: 3.09 s.d: 0.49 mean: 2.77 s.d: 0.35 mean: 0.51 s.d: 0.63 mean: 0.13 s.d: 0.47 mean: 0.02 s.d: 0.54 mean: 0.17 s.d: 0.63 figure 5a: graph of absolute values of sight estimate results, figure 5b: sight estimate results – absolute values above diagonal, signed values below diagonal the absolute values and the signed values for the sight estimates were analysed separately, using a two-way between-participants anova. analysing the absolute values (figure 5b) found a ajis special issue december 2004 142 significant effect for feedback (f(1,76)=12.92, p<0.001), no significant effect for device (f(1,76)=0.02, p=0.90) and no significant interaction (f(1,76)=0.22, p=0.64). this effect is illustrated in the graph in figure 5a. analysing the signed values (figure 5b) found a significant effect for feedback (f(1,76)=4.26, p<0.05), but no significant effect for device (f(1,76)=0.87, p=0.35), and no significant interaction (f(1,76)=3.20, p=0.08). no-sight estimate feedback hapticvisual e rr o r (c m ) 4.2 4.0 3.8 3.6 3.4 3.2 3.0 device mouse tui haptic visual mouse tui f e e d b a c k i v device iv mean: 4.13 s.d: 0.64 mean: 3.26 s.d: 0.55 mean: 3.74 s.d: 0.40 mean: 3.17 s.d: 0.57 mean: 0.14 s.d: 0.81 mean: 0.12 s.d: 0.67 mean: 0.21 s.d: 0.51 mean: 0.06 s.d: 0.45 figure 6a: graph of absolute values of no-sight estimate results, figure 6b: no-sight estimate results – absolute values above diagonal, signed values below diagonal the absolute values and the signed values for the no-sight estimates were analysed separately, using a two-way between-participants anova. analysing the results from the absolute values (figure 6b) found a significant effect for feedback (f(1,76)=34.38, p<0.001), no significant effect for device (f(1,76)=3.73, p=0.06) and no significant interaction (f(1,76)=1.54, p=0.22). this effect is illustrated in the graph in figure 6a. analysing the results from signed values (6b) found no significant effect for feedback (f(1,76)=0.21, p=0.65), no significant effect for device (f(1,76)=0.33, p=0.57), and no significant interaction (f(1,76)=0.01, p=0.97). discussion the discussion first focuses on the absolute values for both the sight and the no-sight estimates because it is the absolute values that determine if the hypotheses are supported. it then examines the signed values for both the sight and no-sight estimates. for both the sight (section 5.2.1) and no-sight estimates (5.2.2), it was found that the absolute values of the estimates that were made with haptic feedback, were significantly lower than those made with only visual feedback. this supports the hypothesis that size is more accurately estimated when the pa model is touched, compared with when it is just looked at (section 4.1). for both the sight (section 5.2.1) and the no-sight estimates (section 5.2.2), there was no significant difference between the absolute values of the estimates made after using a tui, compared to those made using a mouse. therefore the hypothesis that the accuracy of size estimates will be higher after a tui is used, compared to after a mouse is used, is not supported. this can be explained by previous research. it has been found that combining visual and haptic feedback results in more accurate perception of size (section 4.1). however, there is also a large body of research that has shown the visual sense dominates during perception of size (ernst & bülthoff 2004). this suggests that the effect haptic feedback has on perception may depend on the type of haptic feedback that is being provided. thus the single point-of-contact haptic feedback ajis special issue december 2004 143 provided by a tui, may not be sufficient to increase the accuracy of the perception of size. this prediction is supported by research that found haptic feedback perceived through a single point is not sufficient for a person to accurately perceive the size and shape of an object (klatzky & lederman 2004). this has also been shown to be true for single-point-of-contact haptic devices, such as a phantom (kirkpatrick & douglas 2002). memory may also have been a factor that lead to no significant difference being found between the tui and the mouse conditions. the size estimates were made a short time after the participants completed the priming (interaction) task. it is possible that the haptic feedback gained in the tui conditions, was lost due to the time delay. the results can be investigated in more depth by looking at the results for the signed values. the signed values indicate the direction (overestimate or underestimate) of the results in each condition. for both the sight and the no-sight data sets, in all four conditions the signed values are positive, therefore in all conditions on average participants slightly over estimated sizes. when the no-sight estimate was analysed, there was no significant difference between signed values. this indicates that for this measure, neither iv has an effect on the direction of the estimates. however, when the sight estimates were analysed, there was a significant effect for feedback, with lower overestimates being made in the haptic conditions. this suggests that for the sight estimates, haptic feedback can reduce the tendency to overestimate sizes. taking the pattern of the results for the no-sight and the sight estimates together; they suggest that people can be affected by the sight of their hands when making size estimates. a similar pattern has been found in previous research (heller et al. 1999). returning to the original research question that this study is investigating. i.e. what effect does touching a pa model and interacting with it using a spatially-coincident device, have on users’ perception of its size, when compared to not touching it and using a spatially-separate device? (section 3.2) the effect of touching a pa model has been tested directly (see above). the effect of using a spatially-coincident device or spatially-separate device was operationalised using a tui and a mouse. therefore these results suggest that using a spatially-coincident device does not affect the users’ perception of size, compared to using a spatially-separate device. the question investigated in this study is part of a larger project. the projected aims to suggest a number of guidelines that will enable a person who is interested in using a pa model for a particular application, determine the best way of making use of the haptic feedback that pa models can support, to meet the requirements of that application (section 3.1). the guideline that can be derived from these results is; for applications for which size perception is important, the users should be instructed to touch the pa model with their bare hands, and it is not important whether a spatiallycoincident or spatially-separate device is used. it is important to note that these are preliminary guidelines; more work is needed to determine how touch affects other factors. conclusions this study found that size estimates made when a pa model is simultaneously viewed and touched are significantly more accurate than those made when it is just looked at. this study also found that there was no significant difference in the accuracy of estimates of size made after interacting with a pa model using a spatially-coincident device, compared with after using a spatially-separate device. this research is part of a larger project that aims to suggest a number of guidelines that will enable a person who is interested in using a pa model for a particular application, determine the best way of making use of the haptic feedback that pa models support, to meet the requirements of that application. these results can be used to contribute to the formation of these guidelines. the findings from this study are also related to the wider area of haptic perception and haptic displays. they support previous work, which found haptic feedback increases the accuracy of the ajis special issue december 2004 144 perception of size. the results of this study also support previous work that found single-point haptic feedback is insufficient in the perception of size. references bennett, e. and stevens, b. (2004). the effect that haptically perceiving a projection augmented model the perception of size, (poster). in proceedings of international symposium of mixed and augmented reality (ismar04). in press. beers, r., woolpert d. and haggard, p. (2002) when feeling is more important than seeing in sensorimotor adaptation. current biology, elsevier, 12, 834-837. beers, r., stittig, a. and denier, j. (1996). how humans combine simultaneous proprioceptive and visual position information. experimental brain research, springer, 111, 253-261. bimber, o., gatesy, s., witmer, l., raskar, r. and encarnacao, l. (2002) merging fossil specimens with computer-generated information. ieee computer. 35(9), 25-30. bingham g. and pagano, c. (1998) the necessity of a perception-action approach to definite distance perception: monocular distance perception to guide reaching. journal of experimental psychology, 1998, 24(1), 145-168. clark, d. mckeon, r., marciano, r. and bailey, m. (1998) rear-projecting virtual data onto physical terrain: an exercise in two senses being better than one. in proceedings of ieee visualization, ieee press, 451-454. ernst, m and banks, m. (2002) humans integrate visual and haptic information in a statistically optimal fashion. nature, 415, 429-433. ernst, m and bülthoff, h. (2004) merging the senses into a robust percept. trends in cognitive sciences, elsevier, 8(4), 162-169 eysneck, m and kean, m cognitive psychology, (4 th edition). 2001, psychology press. exhibition at the museum of london. available online at: http://www.museumoflondon.org.uk/molsite/exhibits/bodies/bath.htm, accessed may 2004. de gelder, b and bertelson, p (2003) multisensory integration, perception and ecological validity. trends in cognitive science, elsevier, 7(10), 460-467. gepshtein, s. and banks, m. (2003) viewing geometry determines how vision and haptics combine in size perception, current biology, elsevier, 13(6), 483-488, 2003 hale k. and stanney, k. (2004) deriving haptic design guidelines from human physiological, psychophysical, and neurological foundations. ieee computer graphics and applications. 24(2), 33-39. heller, m. calcaterra, j. green, s and brown, l (1999) intersensory conflict between vision and touch: the response modality dominates when precise action-riveting judgments are required. perception and psychophysics, 61(7), 1384-1398. heller, m. and schiff, w. (1991) psychology of touch. lawrence erlbaum associates new jersey, usa. iwata, h., yano, f. nakaizumi, and kawamura, r. (2001) project feelex: adding haptic surface to graphics. in proceedings of international conference of computer graphics and interactive techniques (siggraph '01), acm press, 469-476. kirkpatrick, a. and douglas, s. (2002) application-based evaluation of haptic interfaces. in proceedings of 10th symposium on haptic interfaces for virtual environment and teleoperator systems, ieee computer society press, 32-40. klatzky, r. and lederman, s. (2003) touch. in a. healy and r. proctor (eds.) handbook of psychology, volume 4: experimental psychology. wiley, new york. ajis special issue december 2004 145 lampton, d., bliss, j. and morris, c. (2002) human performance measurements in virtual environments. in k. m. stanney (eds.) handbook of virtual environments: design, implementation, and applications, lawrence erlbaum associates, new jersey, usa. lange, c., klatzky, r. and ernst, m. (2004) bimanual size estimation: no automatic integration of information across the hands. (in press) available online: http://www.kyb.mpg.de/publications/pdfs/pdf2722.pdf accessed june 2004. latta, j. n. and oberg, d. (1994) conceptual virtual reality model. ieee computer graphics and applications, 14(1), 23-29. mukaigawa, y., nagai, h. and ohta, y. (2004) space relighter: technical report of the institute of electronics, infomation and communication engineers. www.image.esys.tsukuba.ac.jp/~mukaigaw/papers/prmu2004-spacerelighter.pdf, accessed june 2004 pagano, c., grutzmacher, r. and jenkins, j. (2001) comparing verbal and reaching responses to visually percieved egocentric distances. ecological psychology, lawrence erlbaum associates, 13(3), 197-226. piper, b., ratii, c. and ishii, h. (2002) illuminating clay: a tangible interface with potential grass applications. in proceedings of open source geographical information systems – grass users conference. raskar, r., welch, g., low k. and bandyopadhyay, d. (2001) shader lamps: animating real objects with image-based illuminations. in proceedings of the 12 th eurographics workshop on rendering, springer. raskar, r. ziegler r. and willwacher, t. (2002) cartoon dioramas in motion. in proceedings of international symposium on non-photorealistic animation and rendering (npar), acm press. stevens, b., jerrams-smith, j., heathcote, d. and callear, d. (2002) putting the virtual into reality: assessing object-presence with projection-augmented models. presence: teleoperators and virtual environments, mit press, 11(1), 79-92. ullmer, b and ishii, h. emerging frameworks for tangible user interfaces. carroll, j. m. humancomputer interaction in the new millennium. boston, usa: addison-wesley professional; 2001; pp. 579-601 verlinden, j., esker, w., wind, l. and horváth, i. (2004) qualitative comparison of virtual and augmented prototyping in handheld products. in proceedings of international design conference – design 2004, dubrovnik. welch r. and warren, d. (1980) immediate perceptual response to intersensory discrepancy, psychological bulletin, 88(3), 638-667. copyright emily bennett and brett stevens © 2004. the authors assign to ozchi and educational and nonprofit institutions a non-exclusive licence to use this document for personal use and in courses of instruction provided that the article is used in full and this copyright statement is reproduced. the authors also grant a non-exclusive licence to ozchi to publish this document in full in the conference papers and proceedings. those documents may be published on the world wide web, cd-rom, in printed form, and on mirror sites on the world wide web. any other usage is prohibited without the express permission of the authors. aj1s vol. 8 no. 1 september 2000 an empirical analysis of the effect of criticality, complexity, and organisational influence on software reliability paul l bowen, ph.d., cpa1, jon heales phd2 anne e. speed, mis3 'department of commerce university of queensland brisbane, queensland 4072 australia phone: +61 7 3365 6584; fax: +61 7 3365 6788 internet: bowen@lorien.commerce uq.edu au 2the university of queensland brisbane, queensland 4072 australia phone: +61 7 3365 6433; fax: +61 7 3365 6788 internet: heales@commerce.uq.edu.au ^the university of queensland brisbane, queensland 4072 australia abstract this paper is motivated by a desire to improve software reliability, in particular, the reliability of software that impacts the operational and financial viability of organisations. we examine the effects of criticality, complexity, and organisational influence on information systems reliability. questionnaires were used to gather quantitative data for statistical analysis. post-hoc in-depth interviews were used to help explain results of the statistical analysis. surprisingly, no associations were observed between reliability and criticality or between reliability and complexity. a positive relationship was found, however, between system reliability and organisational influence. the interviews indicated that organisations mitigated the potential negative effects of complexity through additional planning, and achieved more reliable software by assigning more competent project managers they managed criticality by assigning more competent project managers to more critical systems. the significant relationship between system reliability and organisational influence indicates that is managers respond to internal political pressures. this result implies that senior management should take steps to ensure that excessive organisational influence does not cause is managers to misallocate resources. for example, for each major project, the is steering committee can determine the desired level of reliability, appoint project mangers with the appropriate skill set, and periodically communicate with these project managers about the activities used to achieve each system's desired level of reliability. the authors are grateful to fiona rohde, ron weber, the anonymous reviewers, and the editor for their helpful comments. introduction this paper is motivated by the desire to improve levels of software reliability', in particular, software that impacts the operational and financial viability of an organisation. software errors2 incur three types of costs: the cost associated with the error itself, the subsequent cost of correcting the error, and the cost associated with the loss of stakeholders' confidence and goodwill3. because of the economic impacts information systems have on organisations, a greater understanding of factors affecting reliability can help minimise costly errors resulting from inadequate levels of software reliability. project managers control many factors that affect an information system's quality during development, e.g., they allocate resources, exercise management control, and determine testing regimes. when making decisions about these control factors, the project managers take into account a number of factors. this paper examines the effects of three of these factors on information systems reliability: software reliability is the probability that the software will execute without failure or will perform successfully on demand (rook, 1990). shooman (1983) defined software errors as problems in the external operation of a system caused by internal software faults. for example, westpac bank blamed insufficient software testing for a fault that caused automatic teller machines to allow customers to overdraw their accounts. this error cost westpac several million dollars, incurred unexpected software maintenance costs, and resulted in unfavourable publicity (software engineering notes, 1991). ajis vol. 8 no. 1 september 2000 criticality4, complexity5, and organisational influence6. the effects are examined at three levels: is management level, project management level, and programmer level. project managers, as agents of the steering committee and the system owners, should ensure that information systems critical to the organisation's success have high reliability. a positive relationship between criticality and reliability indicates that organisations can potentially improve their benefit/cost relationships by allocating more resources to improving the reliability of critical systems. that is, information system costs would increase but would be outweighed by the lower risk of losses or missed opportunities. second, greater software complexity has long been associated with lower reliability (see, e.g., halstead 1977, 84-91; ropponen and lyytinen, 2000). this paper explores this relationship and increases our understanding of the relationship between complexity and reliability. management can use this understanding to improve information systems reliability in increasingly complex systems environments. third, in a perfect world, organisational influence should not affect reliability, i.e., subject to complexity constraints, criticality rather than politics should determine the reliability of organisational information systems. a positive relationship between organisational influence and reliability indicates a potential misapplication of the organisation's resources that senior management should address. surveys and interviews in three organisations were used to gather evidence about the associations between information systems criticality, complexity, organisational influence, and reliability. questionnaires were used to gather quantitative data for statistical analysis, and post-hoc in-depth interviews were used to supplement and explain the results of the statistical analysis. surprisingly, no associations were observed between reliability and criticality or between reliability and complexity. a positive relationship was found, however, between system reliability and organisational influence. post-hoc interviews were undertaken to try to explain the unexpected statistical results of the survey. for example, the post-hoc interviews revealed that mitigating factors may explain the lack of a statistical association between complexity and reliability. for example, the organisations had, by additional planning, achieved more reliable software by assigning more competent project managers. that is, they managed criticality by assigning more competent project managers to more critical systems. the research found a statistically significant relationship between system reliability and organisational influence that indicates is managers respond to internal political pressures. theory and hypotheses complexity, criticality, and organisational influence have very different types of relationships with software reliability. complexity has a direct technical association with software reliability, i.e., more complex software is typically less reliable, or equivalently, organisations must expend more computing and information systems resources to achieve a fixed level of reliability for more complex software. the link between criticality and software reliability is primarily economic, i.e., economic rationality dictates that organisations should expend more resources on reliability for software that is critical to organisational success. organisational influence and software reliability exhibit a behavioural relationship, i.e., like other members of an organisation, information systems personnel are subject to political pressure and are likely to expend greater efforts on information systems for people or groups they perceive as more powerful or influential. reliability musa et al. (1987, 15) define software reliability as the probability of failure-free operation of an information system in a specified time frame and environment. they note that their view of reliability represents a user-oriented view of software quality (musa et al. 1987, 5). chillarege (1996) takes a similar view when he states that software failures occur when users' expectations are not met or the users are unable to perform useful work with the software. furthermore, pitt et al. (1995) and watson et al. (1998) assert that reliability, defined as the dependable and accurate performance of the promised service or function, is one of five major dimensions of information systems effectiveness. criticality refers to the importance of the software to the organisation. for example, reservation systems are extremely critical to airlines because the airlines cannot conduct business without them. 5 complexity depends on the size, intricacy, and sophistication of the software. organisational influence refers to the power structure within the organisation, i.e., the capacity of an individual or group to modify the conduct of the other individuals or groups. ajis vol. 8 no. 1 september 2000 is management control of reliability the capability maturity model (sallis, tate et al., 1995; herbsleb et al. 1997) presents an overall approach that management can use to improve software quality. for example, key practices used to determine an organisation's position on the software process maturity scale include projecting design errors, test errors, and remaining errors; analysing error causes; controlling design changes, requirements changes, and code changes; training developers and review leaders; and maintaining and analysing numerous metrics about software process improvement (dekleva and drehmer 1997). software reliability is determined by events that occur during all phases of the systems development life cycle.7 case tools can help reduce errors throughout the process (orlikowski 1993). for example, upper case tools facilitate communication, documentation, and co-ordination during the requirements elicitation and definition stages. lower case tools can help reduce errors translating the requirements specifications into the target programming language. modern programming techniques, including structured programming (linger et al. 1979) and program verification (dyer 1992), reduce initial software faults. organisations use various forms of testing to detect remaining faults, and to ensure adequate levels of software quality and reliability (beizer 1984). increasing software reliability by using rigorous programming techniques or performing extensive testing may increase the costs of developing that software but will reduce the expected costs of software errors (zhao and xie 1993; yang and chao 1995). complexity software complexity refers to those characteristics that make the software difficult to create, understand, or change (curtis et al. 1979). increasing complexity places progressively greater cognitive demands on people and reduces their performance (campbell 1988). the most cited software complexity measures are those proposed by mccabe (1976) and halstead (1977). mccabe's cyclomatic complexity computes the maximum number of linearly independent paths in the program, i.e., it focuses on the number of decision points in the program. halstead defined a number of complexity measures all of which are related to the number of operators (keywords of the programming language) and operands (variables). of particular relevance to this study, he demonstrated that his complexity measure for mental discriminations (programmer effort) is positively related to program error rates (halstead 1977, 84-91). more recent empirical studies also provide evidence of the negative impacts of complexity. grady (1993) found that post-release defect density was highly correlated with structural complexity. one reason for this relationship was that significant portions of the code were not tested prior to release because of the difficulty of testing all possible paths through complex code. card and glass (1990) found a close correlation between defect density and design complexity.8 furthermore, empirical studies at a large commercial bank and at a mass merchandising retailer provide compelling evidence of the negative effects of complexity on maintenance activities (banker et al. 1993; banker et al. 1998). ropponen and lyytinen (2000) found that managing project complexity was a component of software development risk in both scheduling and timing, and in resource usage and performance. thus, software complexity poses a major obstacle to producing and maintaining reliable software. this gives rise to the following hypothesis: hi: a negative relationship exists between software reliability and the complexity of that software. criticality information systems are critical to the success of most organisations. indeed, many organisations attempt to use their information systems to create competitive advantages (see, e.g., mata et al. 1995). the phases of the systems development life cycle (sdlc) are problem definition, feasibility study, analysis, general systems definition, detailed systems definition, implementation, and evaluation and maintenance. in this view, the implementation stage includes programming and (module) testing, and systems and acceptance testing (leitch and davis, 1992). design complexity is a function ol structural complexity, data complexity, and procedural complexity. ajis vol. 8 no. 1 september 2000 for example, chrysler corporation depends on its edi system to achieve benefits of over $100 per vehicle thereby helping it remain competitive and profitable (mukhopadhyay et al. 1995). perhaps the epitome of the criticality of information systems is sabre, american airline's reservation system (copeland and mckenney 1988). not only the airlines themselves, but also related organisations such as travel agencies, are totally dependent on the continuous, accurate, and reliable functioning of the reservation systems. sherer and paul (1993) developed a model for assessing the expected financial consequences of software errors. their model is a function of the expected financial consequence of hazards that can result from each possible use, weighted by the probability each hazard will occur when the information system is employed for that use, and adjusted for the extent to which the system is expected to perform that use during the time period of interest. ceteris paribus, the expected net benefits of reliability are greater for systems that are used by more stakeholders and for systems that relate to core functions of the organisation. hence, wealth maximising organisations would seek to increase the reliability of those software applications that are most critical to their success, i.e., the information systems most closely aligned with the organisation's goals, objectives, and critical success factors. this gives rise to the following hypothesis: h2: a positive relationship exists between software reliability and criticality of that software to the success of the organisation. organisational influence another possible obstacle in producing software with the appropriate level of reliability is the influence of organisational sub-units. organisational influence is the capacity of individuals or groups to obtain the outcome they prefer in particular situations (salancik and pfeffer 1977). persons or groups with organisation influence can induce others to perform tasks they might not otherwise perform (lansbury and spillane 1991, 98). although organisational sub-units should align themselves with the goals of the entire organisation, the possibility exists for organisational sub-units to exert their influence in a manner that is in conflict with the goals of the organisation as a whole (raghunathan and raghunathan 1992). for example, organisational units with substantial influence can cause the information systems department to expend resources on projects that do not match the optimal priorities of the organisation as a whole. alternatively, organisational sub-units with less influence may not be able to obtain information systems department resources for projects that, if implemented, would make substantial contributions to the objectives of the overall organisation. is personnel may expend more effort or exercise greater care when developing software for clients with greater organisational influence. a positive relationship is likely to exist between software reliability and the organisational influence of the requesting department. this gives rise to the following hypothesis: h3: a positive relationship exists between software reliability and the organisational influence of the requesting department. figure 1 shows the effect on software reliability of complexity, criticality, and organisational influence. complexity criticality organisational influence software reliability +ve figure 1. the effects of complexity, criticality and organisational influence on software reliability. ajis vol. 8 no. 1 september 2000 methodology to obtain information on software reliability, organisations considered likely to develop rather than purchase a substantial portion of their application software were identified. is managers from three organisations agreed to participate. details of the organisations, projects, project managers, and programmers are provided in table 1. the land titles department provides an accurate, secure system for recording property ownership and other interests in freehold and state leasehold land. the system enables protection of the rights and interests of proprietors and the community. it also supplies efficient, readily available services for accessing land registry information. the insurance company offers a wide range of insurance options including home and its contents insurance, motor vehicle and compulsory third party coverage, boat coverage (both at sea and ashore), life insurance, income protection and trauma insurance, and commercial and rural business insurance. the government it service organisation provides a wide range of services and consultancies across the full spectrum of it activity from designing and executing multimedia web pages to providing fully integrated it solutions for an entire organisation. it is the largest information technology services provider in the state. table 1: organisations, projects, and personnel surveyed it managers projects project managers* programmers land titles department 1 5 5 7 insurance company 1 6 2 8 government it service organisation 1 4 2 5 totals 3 15 9 20 *some project managers were in charge of more than one project two forms of analysis were used. first, quantitative analyses were performed on questionnaire responses to statistically test the significance of the associations in the model (see figure 1). the questionnaires are summarised in appendix a. second, because of the small sample size and to obtain a richer data set, in-depth interviews were conducted to supplement and explain the quantitative results. these interviews were conducted with the three is managers and five project managers. the project managers were identified by the is managers.9 at least one project manager from each organisation was interviewed. information was obtained from more than one level to provide different views and perceptions of the effects that criticality, complexity, and organisational influence may have on software reliability (gough, 1993; moymhan, 1990; nath, 1989). each interview lasted approximately one hour and consisted of open-ended questions eliciting each participant's opinions about: • the reliability of the software they develop, • the complexity and criticality of the software, • their programming techniques and testing procedures, and • the factors that would affect the programming techniques, testing procedures, and overall reliability of individual software applications. all participants allowed their interviews to be tape-recorded, agreed to be available for follow-up questions,10 and requested copies of the results of the study. after each survey interview, each participant was asked to complete a set of questionnaires (see appendix a). each participant read the questionnaires and any questions the participant had were answered. is managers completed questionnaires about their organisation's programming techniques and testing procedures, and about the ability of each of their project managers. they also completed questionnaires about the criticality, complexity, organisational influence, and reliability of several application software projects. the project managers completed questionnaires about their organisation's programming techniques and testing procedures, and about the ability of each of their programmers. the project managers also 9 interviewing project managers identified by the is managers introduces the possibility of bias and care must be taken when placing reliance on the results. 10 two follow-up phone calls were made. ajis vol. 8 no. 1 september 2000 completed questionnaires about the reliability, criticality, complexity, and organisational influence of each application software project they managed. all participants returned completed questionnaires. the three participating organisations returned questionnaires on a total of 15 separate application software projects. the participants rated the reliability, criticality, complexity, and organisational influence associated with each software project on a seven-point likert scale (see appendix a). the is managers rated the project managers in terms of thenability to manage software projects on the dimensions of reliability, criticality, complexity, and organisational influence of the requesting department." project managers rated their programmers on the reliability of their software, and the programmer's ability to handle critical projects, complex projects, and projects for influential members of the organisation. responses for these ratings were also on seven-point likert scales. to identify any weaknesses in the questionnaires12 including any difficulties by the participants, the responses on each questionnaire were compared with the qualitative statements made by each participant during their interview. figure 2 shows the summary data for each variable by organisation. table 2 lists the constructs and variable names. because of the small sample size, the data were also analysed using non-parametric statistics. both pearson (parametric) and spearman (non-parametric) correlation coefficients are shown in appendix b. results from these two methods are essentially identical and are consistent with the regression analyses reported in the results section. mean values of variables t.w j h s g09 & cm _4 ^> 1t1 ^^ f™| o co co i-l h m >— t 2 * pl, }j oh o, l c o m bh j u-s oh j 2 h 2u 2 oh g c o m oh j o ph figure 2. summary data of variables by organisation. table 2: table of constructs and variable names variable construct srel system reliability scrit system criticality scomp system complexity sinfl system organisational influence plrel project manager reliability plcrit project manager criticality plcomp project manager complexity plinfl project manager organisational influence prel programmer reliability quantitative measures have been developed for both software reliability (e.g., jelinski and moranda, 1972; littlewood, 1981; musa, 1975; schick and wolverton, 1973) and complexity (e.g., halstead, 1977; mccabe, 1976). unfortunately, none of the organisations contacted had the necessary data to calculate either of these measures. prior to their use in this study, the questionnaires were reviewed by three persons each having a minimum of four years of is experience in diverse industry groups. ajis vol. 8 no. 1 september 2000 pcrjt programmer criticality pcomp programmer complexity pinfl programmer organisational influence results the relationships of reliability to software criticality, complexity, and organisational influence were analysed using ols regressions. table 3 contains the results of analysing the model: ' reliability = (j0 + pi*criticality(+) + p2*complexity(-) + p3*organisational influence(+) criticality the analysis did not reveal the expected positive relationship between reliability and criticality (hi). a review of the transcripts of the interviews disclosed that all three organisations used structured programming techniques and rigorous testing methodologies. although the managers in one of the organisations stated that critical projects would cause them to pay closer attention to detail, managers at all organisations indicated that programming and testing procedures were essentially the same for all projects. the is managers did, however, make comments indicating that they considered criticality when assigning project managers and other is personnel to specific projects. that is, the is managers assigned more capable project managers to critical projects to enhance the reliability of critical software systems. hence, comments by the is managers tend to support the hypothesised positive relationship between reliability and criticality but the relationship is not supported by the statistical analysis. a number of reasons may explain this apparent conflict. first, the correlation analyses in appendix b reveal significant positive relationships between criticality and complexity for is managers, project leaders, and programmers. therefore, the lack of a positive statistically significant relationship between reliability and criticality may be because the critical software is more complex. second, this research may not have used sufficiently accurate measurements of reliability or criticality to reveal statistically significant results. third, using essentially the same programming and testing procedures may mitigate the greater attention to detail and the assignment of more capable personnel. table 3: effect of criticality, complexity, and organisational influence on software reliability'4 source model error criticality complexity org. influ. df 3 11 1 1 1 mean square 0.5682 0.1828 0.0006 0.0157 1.4449 f value 3.11 0.00 0.09 7.91 p r > f 0.0708 0.9551 0.7752 0.0169 parameter estimate -0.0068 0.0256 0.3073 r' = 0.46 complexity quantitative analysis of the questionnaire data also did not reveal the expected negative relationship between reliability and complexity (h2). post-hoc interviews revealed that although the organisations the direction of the expected impact of each independent variable is in parenthesis. reported mean squares are based on type iii sum of squares. ajis vol. 8 no. 1 september 2000 followed the same type of programming and testing procedures for all projects, two of the organisations stated that complex projects required more planning. this additional planning included increased attention to analysis and construction of specifications and more careful planning and organisation of programming tasks before actually beginning coding and testing tasks. one manager stated, "with a complex program, you need to map out the program and write a test plan first." managers also indicated that they considered size and complexity when assigning personnel to projects. using personnel assignments to compensate for size and complexity helped explain the lack of a statistically significant relationship between complexity and reliability. other possible explanations include measurement problems, or the greater attention to planning and organisation. organisational influence the expected positive relationship between reliability and organisational influence (h3) was highly significant (p-value < 0.02, two-tailed test). project managers at one of the organisations were adamant that organisational influence did not change their programming and testing procedures. the is manager at that organisation, however, does pay more attention to projects for influential clients and frequently mentioned organisational influence during the interviews. managers at the other two organisations stated that they altered the nature or extent of their programming and testing procedures for projects requested by clients with high levels of influence. one of these organisations was explicitly market driven with the is manager stating that project managers would perform additional testing and other reliability enhancing procedures if the client was willing to pay for it. the effect of organisational influence appears to be different in the three organisations.15 post-hoc analysis of organisational influence the relationship between reliability and organisational influence (h3) was supported both statistically and by the case study interviews in all three organisations. the level of influence that clients were able to exert, however, did vary across organisations, i.e., influence was important in all three organisations but was more important in some of the organisations than in others. the interaction of influence with organisation is also significant (p-value = 0.0319) for a regression with reliability as the dependent variable and organisational influence, organisation, and the interaction of organisational influence and organisation as the independent variables. post-hoc analysis of project manager competency a recurrent comment from is managers was that, to increase software reliability, they assigned more competent project managers. the results of an ols regression of the relationship of software reliability with project managers' perceived abilities to produce reliable software and to deal with criticality, complexity, and organisational influence support this view (see table 4). because is managers assigned more competent project managers to improve reliability, one might expect project managers to follow a similar strategy with programmers. the results of an ols regression of the relationship of software reliability with programmers' perceived abilities to produce reliable software and to deal with criticality, complexity, and organisational influence did not indicate any statistically significant relationships (see table 5, f(4il0) = 1.10, p-value=0.4084). follow-up conversations with the participants indicated that one organisation tried to assign more competent programmers to obtain more reliable software and in another organisation programmers were assigned from a pool. the third organisation considered coding as a generic process and relied on the project manager and other procedures to obtain the desired reliability levels. the pearson correlation coefficient between reliability and influence for the first organisation is 0.4677 but 0.6124 and 0.9956 for the other two. 10 ajis vol. 8 no. 1 september 2000 table 4: effect of project managers' abilities on software reliability source model error reliability criticality complexity org. influ. r2 = 0.67 table source model error reliability criticality complexity orgjnflu. df 4 10 1 1 1 1 5: effect df 4 10 1 1 1 1 mean square 0.6231 1.2226 0.5591 0.7422 0.4216 0.0072 of programmers' mean square 0.2837 0.2580 0.1210 0.3744 0.0111 0.1535 f value 5.10 4.57 6.07 3.45 0.06 abilities on f value 1.10 0.47 1.45 0.04 0.59 p r > f 0.0168 0.0582 0.0335 0.0930 0.8132 parameter estimate 0.6258 -1.1407 0.7409 0.0629 software reliability p r > f 0.4084 0.5091 0.2561 0.8397 0.4584 parameter estimate 0.2685 0.3220 -0.0650 -0.3366 r2 = 0.31 future research the research has focussed on the relationships in the model outlined in figure 1. the case study data has helped to explain in more depth the relationships in the model. on a broader front, there are a number of organisational issues that could impact on the model, and the results of this research. future research can, for example, examine the impact on organisational influence and/or criticality that the following factors may have: • management attitude and how it activities are managed; • the impact of outsourcing, resource allocation, and project management; and • the type of organisation and its maturity in it terms (cmm stage); and the degree of centralisation/de-centralisation. conclusion this study investigated the relationship between application systems reliability and its criticality, complexity, and organisational influence. no statistically significant relationship was found for either criticality or complexity. a statistically significant association was found between organisational influence and reliability. interviews and follow-up conversations indicated that organisations mitigated the potential negative effects of complexity by additional planning and that they achieved more reliable software by assigning more competent project managers. 11 ajis vol. 8 no. 1 september 2000 because of the lack of a significant relationship between reliability and criticality and because reliability is managed at project manager level by the assignment of project managers with varying degrees of capability, senior management (e.g., via a steering committee) need to determine the criticality of application software projects so that they can control reliability through the assignment of project managers. these managers should install and monitor control systems to ensure critical software applications attain appropriate levels of reliability. the lack of a significant relationship between reliability and complexity indicates that is managers have developed effective strategies for controlling software complexity. because a primary component of these strategies includes assigning more competent project managers, organisations need to devote significant effort to attracting and developing high quality project managers. organisations might also examine the possible benefits of assigning more competent programmers to important software projects. the significant effect between reliability and organisational influence indicates that is managers respond to internal political pressures. this result implies that senior management should take steps to ensure that excessive organisational influence does not cause is managers and staff to misallocate resources. management should ensure that individual software clients align their goals with those of the entire organisation and that is managers achieve appropriate levels of reliability for software projects requested by clients with differing amounts of organisational influence. the appointment of project managers by senior is management indicates their concern for reliability. this concern is not reflected by the appointment of programmers by project managers. senior is managers and project managers need to be aware of the need to control software reliability at the programmer level. where reliability is a concern, steps should be taken at the project management level to assign programmers with the appropriate level of expertise. furthermore, is managers, project leaders, and programmers should ensure that the development methodology and all components of the sdlc are appropriate for the level of reliability desired. references banker, r.d., s.m. datar, c.f. kemerer, and d. zweig. 1993. "software complexity and maintenance costs," communications of the acm, vol. 36, no. 11 (november) 81-94. banker, r.d., g.b. davis, and s.a. slaughter. 1998. "software development practices, software complexity, and software maintenance performance," management science, vol. 44, no. 4 (april) 433-450. basili, v. r. (1979), in shooman, m. l. (1989) software engineering: design / reliability / management beizer, b. 1984. software testing techniques. new york: van nostrand reinhold. campbell, d.j., 1988. "task complexity: a review and analysis," academy of management review, vol. 13, no. 1,40-52. card, d. and glass, r. measuring software design quality. prentice-hall, englewood cliffs. n.j., 1990. 55. chillarege, r., 1996. "what is software failure?" ieee transactions on reliability, vol. 45, no. 3 (september) 354-355. copeland, d.g., and j.l. mckenney, 1988. "airline reservation systems: lessons from history," mis quarterly, vol. 12, no. 3 (september) 353-370. curtis, b., s.b. sheppard, p. milliman, m.a. borst, and t. love. 1979. "measuring the psychological complexity of software maintenance tasks with the halstead and mccabe metrics," ieee transactions on software engineering, vol. se-5, no. 2, 96-104. dekleva, s., and d. drehmer, 1997. "measuring software engineering evolution: a rasch calibration." information systems research, vol. 8, no. 1 (march) 95-104. dyer, m. 1992. the cleanroom approach to quality software development. new york: john wiley & sons. gough, c.a. 1993. business and information strategy alignment: a study of queensland government agencies. unpublished thesis: university of queensland. grady, r. b. 1993. "practical results from measuring software quality" communications of the acm. 36(11), 62-69. halstead, m.h. 1977. elements of software science. new york: elsevier north-holland. herbsleb, j., d. zubrow, d. goldenson, w. hayes, and m. paulk, 1997. software quality and the capability maturity model, vol. 40, no. 6 (june) 30-40. jelinski, z., and p.b. moranda. 1972. "software reliability research." in statistical computer performance evaluation. (w. freiberger, editor) new york: academic: 465-484. 12 ajis vol. 8 no. 1 september 2000 lansbury, r.d., and r. spillane. 1991. organisational behavior: the australian context. melbourne: longman cheshire. leitch, r.a., and k.r. davis. 1992. accounting information systems: theory and practice. 2nd ed. englewood cliffs, n.j.: prentice hall. linger, r.c., h.d. mills, and b.i. witt. 1979. structured programming: theory and practice. reading, mass.: addison-wesley. littlewood, b. 1981. "stochastic reliability growth: a model for fault removal in computer programs and hardware design." ieee transactions on reliability. 30(4): 313-320. mata, f.j., w.l. fuerst, and j.b. barney, 1995. "information technology and sustained competitive advantage: a resource-based analysis," mis quarterly, vol. 19, no. 4 (december 1995) 487505. mccabe, t.j. 1976. "a complexity measure." ieee transactions on software engineering. 2(4): 308-320. moynihan, t. 1990. "what chief executive and senior managers want from their it departments." mis quarterly (march): 15-25. mukhopadhyay, t., s. kekre, and s. kalathur, 1995. "business value of information technology: a study of electronic data interchange," mis quarterly, vol. 19, no. 2 (june) 137-156. musa, j.d. 1975. "a theory of software reliability and its application." ieee transactions on software engineering. 1(3): 312-327. musa, j.d., a. lannino, k. okumoto, 1987. software reliability: measurement, prediction, application. new york: mcgraw-hill. nath, r. 1989. "aligning mis with the business goals." information and management (16): 71-79. orlikowski, w., 1993, "case tools as organizational change: investigating incremental and radical changes in systems development," mis quarterly, vol. 17, no. 3 (september) 309-340. pitt, l.f., r.t. watson, and c.b. kavan, 1995. "service quality: a measure of information systems effectiveness." mis quarterly, vol. 19, no. 2 (june) 173-187. raghunathan, b., and t.s. raghunathan, 1992. "the relationship between organizational factors and accounting for information systems costs," journal of information systems, vol. 6, no. 2 (fall) 115-126. rook, paul ed. (1990): software reliability handbook. elsevier applied science, london. ropponen, j. and k. lyytinen (2000). "components of software development risk: how to address them? a project manager survey." ieee transactions on software engineering, vol. 26, no 2:98-111 salancik, g.r., and j. pfeffer. 1977. "who gets power and how they hold on to it: a strategiccontingency model of power." organizational dynamics (winter). sallis, p., g. tate, et al. (1995). software engineering: practice, management, improvement, addison-wesley sherer, s.a., and j.w. paul, 1993. "focusing audit testing on high risk software modules: a methodology and application," journal of information systems, vol. 7, no. 2 (fall) 65-84. schick, g.j., and r.w. wolverton. 1973. "assessment of software reliability." proceeding operations research. wurzburg-wien: physica-verlag: 395-422. shooman, m.l. 1983. software engineering: design, reliability, and management. new york: mcgraw-hill. yang, m. c., and chao, a. 1995. "reliability-estimation & stopping-rules for software testing, based on repeated appearances of bugs." ieee transactions on reliability 44(2): 315-321. watson, r.t., l.f. pitt, and c.b. kavan, 1998. "measuring information systems service quality: lessons from two longitudinal case studies," mis quarterly, vol. 22, no. 1 (march) 61-79. zhao and xie, m. 1993. "robustness of optimum software release policies." ieee transactions on reliability: 42(2): 218-225. 13 ajis vol. 8 no. 1 september 2000 appendix a questionnaires the following appendix contains relevant extracts from the questionnaires provided to edp managers, project managers, and programmers. questions were asked of it managers, project managers, and programmers, and about the projects with which they were associated. the questionnaire is titled "the effect of programming and testing methodologies on software reliability" the following questions were asked of it managers (n=3), project managers (n=5), and programmers (n=20): question how effective do you believe these methodologies are in producing reliable software ? how important is the programming process for ensuring software reliability? how important is the testing phase for ensuring software reliability? list the major departments or groups and circle the number corresponding to their level of organisational influence. (likert scale for each department) scale 7 point likert 7 point likert 7 point likert 7 point likert the following questions were asked of it managers and project managers: how reliable is the resulting software? (does it have a low failure rate?) rank their (subordinates) ability to handle projects critical to the organisation rank their ability to handle large or complex projects, where complexity is associated with lines of code or function points. rank their ability to handle projects for very influential members of your organisation 7 point likert rank rank rank the following questions relating to a total of 15 projects were asked of it managers, project managers, and programmers: how reliable was the resulting software? how critical was the application? how complex was the application? list the major departments or groups and circle the number corresponding to their level of organisational influence. 7 point likert 7 point likert 7 point likert 7 point likert 14 ajis vol. 8 no. 1 september 2000 appendix b correlation analyses the following table shows the pearson parametric correlation coefficients and p-values for the data variables. table 2 above shows the legend for the variable names. table b1. pearson correlation coefficients srel pvalue scrit pvalue scomp p-value sinfl p-value plrel p-value plcrit p-value plcomp p-value plinfl p-value pgrel p-value pgcrit p-value pgcom b p-value pginfl p-value srel 1.000 0.000 0.040 0.889 0.206 0.462 0.674 0.006 0.630 0.012 0.459 0.085 0.191 0.496 0.135 0.632 0.447 0.095 0.484 0.068 0.382 0.160 0.335 0.222 scri t 1.000 0.000 0.505 0.055 -0.098 0.729 -0.129 0.646 -0.285 0.303 -0.453 0.090 -0.009 0.975 0.092 0.745 0.311 0.259 0.320 0.246 0.263 0.344 scom p 1.000 0.000 0.205 0.464 -0.150 0.594 -0.091 0.746 -0.046 0.870 -0.239 0.392 0.213 0.447 0.310 0.261 0.552 0.033 0.504 0.055 sinfl 1.000 0.000 0.628 0.012 -0.199 0.477 0.025 0.929 0.065 0.818 0.308 0.264 0.336 0.221 0.364 0.182 0.269 0.332 plre l 1.000 0.000 -0.118 0.676 -0.053 0.850 0.538 0.039 0.287 0.300 0.328 0.233 0.159 0.571 0.141 0.617 plcri t 1.000 0.000 0.862 0.000 0.257 0.356 -0.670 0.006 -0.413 0.126 -0.504 0.055 -0.360 0.188 plcom p 1.000 0.000 0.089 0.752 -0.522 0.046 -0.316 0.251 -0.421 0.119 -0.284 0.305 plinfl 1.000 0.000 -0.153 0.586 0.160 0.569 -0.158 0.575 -0.012 0.965 pgre l 1.000 0.000 0.827 0.000 0.878 0.000 0.836 0.000 pgcrj t 1.000 0.000 0.869 0.000 0.882 0.000 pgcom p 1.000 0.000 0.893 0.000 15 ajis vol. 8 no. 1 september 2000 the following table shows the spearman (non-parametric) correlation coefficients and p-values for the data variables. table 2 above shows the legend for the variable names. table b2. spearman correlation coefficients scrit pvalue scomp pvalue sinfl p-value plrel p-value plcrit p-value plcomp p-value plinfl p-value pgrel p-value pgcrit p-value pgcom p p-value pginfl p-value srel scri scom sinfl plre plcri plcom plinfl pgre pgcrj pgcom t p l t p l t p 0.071 0.800 0.261 0.638 0.348 0.011 0.741 -0.019 0.271 0.002 0.947 0.328 0.578 -0.226 -0.172 0.493 0.024 0.418 0.540 0.062 -0.242 -0.073 -0.328 -0.122 0.414 0.125 0.385 0.797 0.233 0.664 -0.354 -0.032 -0.049 -0.081 0.855 0.148 0.599 0.196 0.911 0.863 0.773 0.000 0.117 0.049 -0.121 -0.033 0.495 0.308 0.141 0.679 0.863 0.668 0.907 0.061 0.264 0.617 0.736 -0.090 0.140 0.596 0.526 -0.641 -0.432 -0.169 0.002 0.751 0.620 0.019 0.044 0.010 0.107 0.548 0.675 0.018 0.156 0.376 0.578 -0.364 -0.279 0.317 0.821 0.006 0.949 0.579 0.167 0.024 0.182 0.313 0.250 0.000 0.707 0.067 0.505 0.500 0.341 -0.412 -0.340 -0.066 0.790 0.743 0.003 0.811 0.055 0.058 0.213 0.127 0.215 0.816 0.001 0.002 0.538 0.025 0.455 0.266 0.264 -0.216 -0.161 0.049 0.702 0.774 0.818 0.039 0.929 0.088 0.338 0.341 0.439 0.566 0.862 0.004 0.001 0.000 16 ajis vol. 9, no. 2 may 2002 editorial this is the second issue of volume 9 of the australian journal of information systems. as editor i am pleased to report the continuing interest in the journal both in australia as well as overseas. jeanette van akkeren and debra barker from the university of the sunshine coast present our first paper mobile data technologies and sme adoption and diffusion: an empirical study on barriers and facilitators. this is the first of two papers presented by these authors and identifies attitudes, perceptions and issues for mobile data technologies by regional sme owner/managers across a range of industry sectors. debra harker and jeanette van akkeren present the second paper, which follows on from the first study. this paper is entitled mobile data technology for small business: needs, uses and adoption. this paper shows that overall the needs of regional sme's for mdt can be conceptualised into three areas where the technology will assist business practices; communication, ecommerce and security. our next paper is entitled the impact of business size and business type on small business investment in electronic commerce: a study of swedish small businesses by rob macgregor and lejla vrazalic, university of wollongong, deborah bunker, university of nsw, sten carlsson and monika magnusson, karlstad university, sweden. this paper focuses on the driving forces behind ec adoption by small and medium enterprises (sme's) and aims to determine the impact of organisational factors such as size and type of business on ec acquisition criteria. ajay saxena, s.d. bhatnagar and p.k. saxena, dayalbagh educational institute, india present our next paper entitled integrated multimedia based intelligent group decision support system for electrical power network. in this paper, an integrated multimedia based intelligent group decision support system (imigdss) is presented, and its main components are analysed and discussed. in particular attention is focused on the data base, model base, central black board (cbb) and multicriteria futuristic decision process (mfdp) module. gulender gencoglu, graeme altmann, ross smith and david mackay, deakin university present our next paper using soft system methodology to address supply chain management problem. this paper reports an investigation of if, and how, soft systems methodology (ssm) might be used to facilitate better management of industry supply chains. john davies, victoria university of wellington, new zealand presents our next paper entitled models of governance a viable systems perspective. this paper surveys alternative conceptualisations of governance that have surfaced in the academic and practitioner literature and suggests a framework for examining governance issues spanning the corporate, public and nonprofit sectors. our next paper information warfare: using the viable system model as a framework to attack organisations is presented by bill hutchinson, edith cowan university and mat warren deakin university. this paper uses a proven systemic, analytic framework the viable system model (vsm) in a functionalist mode, to analyse the vulnerabilities of an organisation's information resources to this form of aggression. l.houghton and p.ledington, the university of he sunshine coast present our next paper entitled the evolution of confusion: soft systems methodology and social theory revisited. this paper reveals apparent weaknesses in the research debate about ssm and social theory, and shows how the evolution of ssm has apparently been affected by that debate. kambiz maani, the university of auckland, new zealand presents our next paper entitled consensus building through systems thinking the case of policy and planning in healthcare. this paper discusses a methodology and a case study using qualitative system dynamics to create consensus, team learning and shared vision in a public organisation. karen manley, queensland university of technology presents our next paper the systems approach to innovation studies. this paper presents an up-to-date overview of this fast moving field, with a view to assisting public policy makers and business managers in designing more effective innovation processes. our next paper is entitled a virtual organisation model for e-government by janice burn and greg robins, edit cowan university. this paper looks at the implementation of a new customer value alliance model in egovernment. sherah kurnia, the university of melbourne, john berts, monash university and robert johnston, the university of melbourne present our next paper entitled understanding the diffusion of efficient consumer response: an australian survey study. differences in barriers, perceptions, and benefits experienced between manufacturers and retailers discovered in this study suggest that australian retailers are leading manufacturers in ecr implementation and that they experience more benefits than manufacturers. robyn lawson, university of western sydney, carole alcock and joan cooper, the university of wollongong present our next paper entitled diffusion of electronic commerce in small and medium enterprises. this paper presents research results from two regional areas which indicate that while most manufacturing smes use email, very few are involved in electronic commerce activities. major barriers are concern about security and privacy of transactions, cost of consultants, and lack of it expertise of staff. ajis vol. 9, no. 2 may 2002 our next paper is evaluating the benefits of electronic commerce in small and medium enterprises by peter marshall, mt eliza business school and judy mckay, monash university. this paper outlines and analyses the behaviours of small and medium enterprises with respect to the evaluation of electronic commerce investments, and the subsequent realisation of the anticipated benefits from those investments. barbara begier, poznan university of technology, poland presents our next paper entitled quality of goals a key to the human-oriented technology. this paper examines the social and ethical impacts of new technologies. don gotterbara, east tennessee state university presents the next paper entitled reducing software failures: addressing the ethical risks of software development life cycle. this paper presents a process which expands the concept of software risk to include social, professional, and ethical risks that lead to software failure. helen richardson, university of salford, uk and kate richardson, manchester metropolitan university, uk present he next paper entitled customer relationship management systems (crm) and information ethics in call centres 'you are the weakest link. goodbye!'. this paper catalogues the rise and rise of call centres in the north west of england, uk and their use of crm systems. paula roberts and jenny webber, the university of south australia present the next paper virtuous hackers: developing ethical sensitivity in a community of practice. this paper argues that in recent times computer programming has moved from a craft-based, bricolage activity to a scientific approach which has led to a knowledge gap developing between the former fraternity of hackers and the computer security industry. our next paper is entitled systems of the information society, simon rogerson, de montford university, uk. this paper discusses some of the contextual issues which need to be addressed to realise this goal. there is a need to strive continually to develop adaptive systems which satisfy the dynamic needs of people. our final paper is presented by joe thomas and andy bissett, sheffield hallam university, uk and is entitled meeting ethical demands in a multi-agency project. this paper looks at various health service and voluntary sector agencies dealing with drug misusers in a large city in the north of england have set up a task group to coordinate their assessment and referral process. an it solution to their needs was believed to be desirable but raises serious issues concerning confidentiality, and the suitability of an it solution to the particular information problem. once again i would like to pass on my thanks to the authors and consulting editors for the high quality of material in this issue. i would like to thank lily soh and diniz da rocha for their continued assistance with the journal and the maintenance of the home page. rob macgregor editor http://www.uow.edu.au/ajis/ajis.html the australasian journal of information systems (ajis) began in 1993 and was based on the simple realisation that there was a need for an academic ‘outlet’ for is research in australia australasian journal of information systems volume 14 number 1 nov 2006 key aspects of the history of the information systems discipline in australia roger clarke xamax consultancy pty ltd. canberra act australia roger.clarke@xamax.com.au abstract the information systems discipline is 40 years old, in australia, as in scandinavia, the u.s.a., the u.k. and germany. this paper presents what may be the first formally published attempt at a history of the discipline in this country. it identifies the precursors to the discipline, and its emergence in colleges and universities. it identifies institutions and individuals who were present at the birth, and traces key steps in organisational and political history. more controversially, it provides interpretations of the nature of the discipline and its key intellectual themes, and identifies threats to its survival. introduction the natural starting-point for this special issue is a review of the origins and foundation years of the information systems (is) discipline in australia. preparing such a paper is an undertaking that anyone would enter into only with considerable trepidation. there is a very limited literature on the history of the discipline as a whole, and almost nothing appears to have been published in formal outlets on the discipline in australia. it is timely that an initial contribution be offered, because many of the early players have already reached retirement age and the rest are approaching it. the author has been active in the discipline since 1968. the project was accordingly commenced by utilising the author's own archives and memory to construct a basic framework of times, places, people and topics. this had the advantage of resource efficiency, and of course the disadvantage that a personal perspective may have unduly limited the subsequent information-gathering steps. the project continued with archival research. no formal histories of the discipline in australia were identified, and few of the discipline elsewhere. the only formalised departmental histories that were unearthed were greig & levin (1989) regarding computing at caulfield/chisholm (196588) and dreyfus (2004) regarding the university of melbourne (1994-2004). interviews were performed face-to-face, by telephone and by email with a selection of individuals who were known by the author to have long standing in the field, or were identified by early interviewees. 123 australasian journal of information systems volume 14 number 1 nov 2006 a working paper was prepared and circulated within the project team and to further selected individuals who it was anticipated would be able to contribute further information and to identify errors, anomalies and omissions. the acknowledgements section lists the individuals on whom i have placed greatest reliance. this paper was developed by extracting key aspects from the revised working paper with the intention of providing a 'scene-setter' for the issue as a whole. this paper is lightly referenced, with the intention that some of the trappings of scholarship be sacrificed in favour of readability. the underlying working paper addresses most aspects more fully, and provides further references. access to the working paper and to collections of further resources is provided at the end of this paper. the organisation of a paper of this nature could be approached in several ways. a chronological recitation would lack depth, and so too would a focus on australia to the exclusion of the rest of the world. on the other hand, a purely thematic discussion would involve continual and confusing reversions in time. accordingly, a blend of the chronological and the thematic has been adopted. this provides a counterpoint to the ‘disciplinary developmental’ approach adopted in ridley (2006), elsewhere in this special issue. the paper commences by identifying events and individuals that appear to have been key to the passage of the discipline from its embryonic state towards a mature community of scholars. that is followed by a brief discussion of the nature of the information systems discipline. a number of intellectual themes are then picked out that appear to have been the subject of heated, and in many cases recurrent, debate. a brief overview of the organisational settings in which the is discipline has been practised precedes an examination of the political dimensions of the discipline – which in turn sets the scene for the evaluation of the current state of play that is undertaken by the papers in the body of this special issue. some caveats are important. the paper is sole-authored, rather than being the product of a substantial collaboration or task force. the author is not a professional historian. and, far from being a disinterested observer, the author was a participant and contributor throughout the period under review, and was and remains a protagonist in some of the debates. the invitation from the editor of the special issue was for a paper that was brief rather than comprehensive. as a result of all of these factors, it is inevitable that the reader will detect selectivity of topics, mannerism in their treatment, and intrusion of the perspective of the individual or of 'schools of thought'. if the reader considers that the paper's deficiencies outweigh its contributions, they should regard this document as merely a first attempt, and a stimulant for their own efforts. foundations of the discipline in australia this section provides a rendition of the years prior to and following the establishment of the first departments of is in australia. emergence – to 1965 some disciplines are blessed with a single name from the outset (e.g. history, engineering); others have a dominant title that migrates seamlessly at some point in time (e.g. philosophy of science to science and technology studies); and yet others have a primary name with variants (e.g. computer 124 australasian journal of information systems volume 14 number 1 nov 2006 science). one major challenge that confronts someone studying is is that the appellations applied to it have been highly varied both cross-sectionally and longitudinally. contrary to the predictions of 'cultural cringe' theory, the is discipline in australia did not follow on from developments overseas, but emerged in parallel with them. not only did early patterns elsewhere not determine the directions of development of the discipline here, they did not even greatly influence them until the end of the 1970s. nonetheless, it is convenient to commence by briefly outlining the beginnings of the discipline in key locations overseas. predecessor professional activities can be readily found in the design of manual and automated data processing systems through the first half of the twentieth century, but no academic discipline can be discerned during that period. broad consensus exists that the primary stimulus for the emergence of is was the administrative application of computers, commencing in 1951 with the lyons tea company in the u.k. (caminer et al. 1998, land 2000), and univac for the u.s. census (e.g. johnson 2006). even then, there was a considerable gestation period. the foundation of the is discipline is most conveniently identified as being in 1965, with the earliest contributions that can be unequivocally identified being those of borje langefors in sweden and gordon davis and bill king in the u.s.a., followed very shortly afterwards by enid mumford, frank land and peter keen in the u.k. and dan couger, jim emery and warren macfarlan in the u.s.a. the major regions that contributed to its establishment and maturation have had substantially different flavours. very roughly and inadequately, the driver of the discipline in the u.s.a. (where it has always been referred to as management information systems – mis) has been value to the organisation and management, and it has tended towards managerialism and managerial rationalism. scandinavia and the u.k. have from the beginning had a much stronger association with human behaviour within organisations and hence with what might be termed 'constructive looseness'. in germany (where it was and remains wirtschaftsinformatik – wi, whose sense is approximately 'business applications of computer science'), there has been a sustained and strong orientation toward data processing and software development. in australia, an immediate predecessor to the discipline was the commonwealth public sector's programmer in training (pit) scheme, which ran primarily in canberra, sydney and melbourne from 1963 to 1970. (many commonwealth agencies were located outside canberra until the whitlam era, 1972-75). about half of the pit scheme's syllabus addressed systems analysis and design, and it produced hundreds of graduates who not only populated, and continue to populate, senior positions in the public sector nationwide, but also filtered out into leadership positions in the private sector. establishment – 1965-1973 departments were established in colleges of advanced education (caes), commencing in 1965, in order to assist in the application of computers to business and government. in some cases, the need was at the low level of fostering computer awareness and providing basic training in their use, and in others at the trade and professional levels of operating computers, and especially developing application software to run on them. hence edp and software development were early terms applied to clusters of is staff. a few concentrations of is staff emerged in accounting departments and were a little further removed from machines and a little more concerned with (particular kinds of) information. until as late as 1990, a great deal of the time of many is academics was committed 125 australasian journal of information systems volume 14 number 1 nov 2006 to imparting basic computer-usage skills to students, most of whom were not otherwise enrolled in is units. the first specialist department, called electronic data processing (edp), appears to have been established at caulfield institute of technology in 1965. (during successive reorganisations, caulfield was later re-named chisholm, and finally absorbed into monash university in 1987). the courses that it offered combined instruction about technology with teaching about how to apply it. programming was a central feature, because all applications had to be custom-built. the transfer of the pit scheme into several caes at the beginning of the 1970s marked a key shift in education and training, as 'train your own' gave way to a formalised educational system. departments were also established in the institutes of technology, qit (now qut), nswit (now uts), rmit, sait (now unisa) and wait (now curtin). meanwhile, is topics emerged in university accounting departments. it appears that the first identifiable unit of study, in edp, was established by len dunn in the department of economics & commerce at the university of tasmania in 1965 (using algol as a teaching vehicle). this was closely followed by unsw and the university of queensland c. 1966-68, and then wollongong college of unsw and the university of melbourne in 1970. sydney and melbourne were major world cities, and wollongong was one of the major centres in the then-very-large steel industry. hobart's early activity was presumably stimulated by the large installation at the tasmanian hydroelectric commission. by the beginning of the 1970s, honours theses in is topics began to emerge from the unsw and university of queensland accounting departments. it is instructive to compare the pattern of development in is with the emergence of computer science (cs). although cs units emerged from the late 1950s in departments of physics, electrical engineering, mathematics and statistics, the growth was very slow until the mid-to-late 1960s (i.e. at much the same time as is). offerings in computer science in most cases migrated from postgraduate diplomas back to final-year undergraduate, eventually expanding into full majors (which mirrors cae offerings in edp, whereas in most universities is migrated forward from undergraduate service topics and units to sub-majors, majors and only later postgraduate teaching and research). the first full computer science majors became available only in 1975, at the universities of melbourne and tasmania. by this time, demand had ensured that many universities offered is service units, is majors had been launched at unsw, uni of tasmania and qut, and some of the caes had begun to offer full undergraduate awards to complement their postgraduate diploma courses. a critical difference between the two disciplines was that, by the end of 1973, there were at least 6 professors of computer science, but none of is. as cs gathered momentum, it had the opportunity to claim applications of computing as its own, which would have denied vital space to is; but its leaders almost unanimously chose not to do so. during these early years, most is academics were loners, and they mostly presented their papers at conferences run by computer scientists (in particular the australian computer conference series run from the early 1960s onwards) or by accountants. the computing in caes conference series emerged as a focal point and ran c. 1970 to 1990, but there appears to be no formal archive of the papers presented at them. 126 australasian journal of information systems volume 14 number 1 nov 2006 consolidation – 1974-1990 in 1974, unsw appointed the first professor of is, cyril brookes, and formed the first university is department, almost a decade after the cae sector had started to form departments of computing and data processing. the first masters dissertation appearsto have been completed in 1976 (by roger clarke), then in 1978 the first australian was awarded a phd in is (ron weber at minnesota, under gordon davis' supervision). the first phds awarded in is in australia were in 1982 and 1984 at the university of queensland, supervised by ron weber (errol iselin and iris vessey) and in 1986 at unsw, supervised by cyril brookes (ross jeffery). ron weber was the second professor, but his post was only partly in is, and he did not take it up until 1981, by which time there were close to a dozen professors of computer science. it was only in the immediate post-dawkins era at the very end of the 1980s that additional full professorial positions were created, notably at monash, which, when it absorbed chisholm institute of technology, took over the mantle from qit/qut as the largest concentration of is academics in australia. the number of professorial posts surged during the 1990s, with about 20 created. by 2005, there were over 30. until at least the mid-1980s, there was no clear body of knowledge for the is discipline in australia. the software development life cycle (sdlc) and database management had both emerged in the late 1960s. but it required years of experimentation and refinement before they matured and merged into structured analysis and design. only then was a sufficiently comprehensive framework available for the management of software development projects. in addition, no prior student knowledge of technology could be assumed, so a considerable amount of time had to be spent on introductory computing topics. published curricula emerged early in the u.s.a. (the first being couger 1973), and some time later in the u.k. although informative, they were not well-fitted to the australian context. they were comprehensive, and were oriented towards either computer science or the specifically u.s. form of graduate schools of business. because limited time was available within is service units, topics had to be selected, and integrated into local course environments. as a result, curriculum development in australia was largely localised and in many cases insular. the first local text-book, brookes et al., appeared only in 1982. a list of leading texts of the mid-1980s is in clarke (1987). the formation of the australian computer society (acs) in 1966 owed much to the efforts of academics in foundation disciplines such as mathematics and physics; but its primary role quickly became that of a professional association. its most direct relationship with the is discipline has been as an accreditation body, assessing the suitability of courses as a basis for professional membership of the society. yet such was the dominance of computer science in the organisation, and such was the variability in the philosophies and curricula in is courses, that it took until 1990 before is was formally recognised within the accreditation guidelines (clarke & lo 1989). the change was substantial, however, with the core body of knowledge established by the acs in 1992 (maynard 1992, underwood 1997) evidencing a 50-50 split between cs and is topics. through the 1980s, the early emphasis on custom-built applications came to be complemented by the customisation and implementation of pre-written packages, and the orientation toward software development gradually gave way to a focus on systems integration. in parallel with that, the closed shop of large mainframe installations was gradually giving way to 'personal computers' and then to local area networks and client-server architecture. 127 australasian journal of information systems volume 14 number 1 nov 2006 the orientation in universities was most commonly away from programming and towards the analysis, design and implementation phases of development and integration, reflecting the realisation that relevant and effective information systems depended on fit with and embedment within human processes. there were parallel developments in is management, in decision support, and later in the strategic management of it and of information itself. over time, information management became a distinguishable body of knowledge, and intellectual relationships developed with library science. through the late 1970s and the 1980s, the vigour of the discipline in the usa resulted in that country establishing leadership in, and for many years even outright dominance over, the is discipline. specialist journals began to appear, the first being misq and database c. 1977, together with specialist conferences, particularly icis from 1980. during this period, ifip tc8 (technical committee 8 of the international federation for information processing), whose locus was primarily europe, began to run conferences. australians took advantage of these publishing and networking opportunities, and ran well-attended international conferences here, the first in 1984. a period of active debate about philosophies and methods of research took place through the 1980s. the establishment of the association for information systems (ais) in 1994 was associated with the emergence of a genuinely international discipline. u.s. numbers and energy have dominated publishing output throughout, but recent years have seen somewhat more accommodation among the pluralism of world views. it might be an exaggeration to speak of tolerance; but widespread if somewhat grudging recognition has emerged that there are multiple flavours of the discipline, and that it is appropriate that there be many outlets for research. some venues intentionally reflect a broad array of approaches, whereas others are more focussed in their orientations. relatively few topic-areas and treatments are unable to find a refereed venue, although the half-dozen journals that are generally regarded as being at the top end of the discipline would be seen by many as being less open than others. by the end of the 1980s, some stability had been achieved in the conception of what is was about. in their submissions to a review of what the government styled the 'computing studies and information sciences disciplines', the acs and the australian national university utilised a graphic which was prepared by this author and is reproduced at exhibit 1 below, which sought to convey the scope of is and its relationships with adjacent disciplines. it was also published as attachment 5 of maynard (1992). is was depicted as occupying vital space between the technical and business disciplines, encompassing a range of applied and instrumentalist topics, and interacting closely with many other disciplines and sub-disciplines. 128 australasian journal of information systems volume 14 number 1 nov 2006 exhibit 1: location of the is discipline, as perceived in 1991 the nature of the discipline most disciplines achieve a reasonable degree of unanimity about the nature and scope of their field, thereby marginalising 'deviant' thinkers, but leaving space for debate about what properly is and is not within-scope of the discipline. the definition of the is discipline has been especially fraught. rather than canvassing the array of possible definitions and getting caught up in nuance, this section briefly reviews the nature of the topics that the discipline involves. one fundamental tension has existed throughout, and is reflected in the 'edp in caes' and 'is in universities' differentiation already noted in the previous section. at a gross level, two primary flavours of the is discipline can be distinguished: • a 'technology' flavour, which is tempered to a lesser or greater degree by business considerations; and • a 'rational management' flavour, which views technology as a collection of interventions into organisational settings. the technology that supports the handling of information was referred to as 'computing' until about 1980, and initially encompassed computers, their uses, and application software including its development, functionality and user interfaces. subsequent to the marriage of computing with communications, it has been conventionally referred to as 'information technology (it)' or 'information and communications technology (ict)'. the endeavour to provide intellectual under-pinnings for applications led early theorists to place 'information' and 'systems' at the heart of the discipline. the personal and psychological dimensions computer information engn science cs systems is engineering graphics electronic materials & devices software engineering artificial intelligence computer architecture programming systems analysis individual & oragnisational behavior financial & cost & mgmt accounting modeling tools digital circuitry operating systems formal language theory algorithms data analysis decision support systems information mangement micro ecs engineering design signal analysis robotics computationa l theory data structures database management it management audit materials science control & communication theory concurrency compiler construction networks & communications knowledge based systems project management marketing production management numerical computing logic graphics bus. law discrete maths n.b. the topic areas are indicative only and are not exhaustive. ??? model showing information technology areas and disciplines and interfaces with other professional bodies computer systems engineering cse commerce bus. admin. 129 australasian journal of information systems volume 14 number 1 nov 2006 of perception and cognition attracted attention, but much more effort has always been focussed on the organisational contexts within which systems are used. during the last few years, there has been a movement to focus on 'the it artefact', a concept whose meaning continues to be debated, but that emphatically denies 'information' and 'system' as being at the heart of the discipline, and substitutes a technology-dependent idea. a considerable amount of mechanistic thinking has entered the discipline, as teachers sought to simplify, reduce and automate. that has generated counter-movements imbued with interpretivist and critical perspectives, that are concerned with information-in-use and technology-in-use, and that deny that success can be achieved with simple formulae. ethics has made an appearance from time to time, but the discipline has paid very limited attention to the impacts and implications of the interventions that information systems and information technologies represent. users of systems and technologies are most commonly regarded as agents of the corporations and government agencies that employ them, and seldom as principals or even participants. the notion of 'usees' – individuals affected by systems and technology, such as the people whose data is collected and managed – is barely detectable in is teaching and research. journal editors actively deny that there is any policy dimension to the discipline. building on this necessarily brisk depiction of the discipline's nature, the following section seeks out important themes during the discipline's life so far. intellectual themes is grew out of, and drew heavily from, a number of disciplines and schools of thought. critical among them were accounting and the applied end of computer science. also of importance, in some departments more than others, were organisation & methods (o&m), operations management, operations research (or) then management science (ms), management accounting, management theory, business strategy, general systems theory (gst) and cybernetics, and socio-technical systems. is researchers have for the most part avoided the creation of new theory, and have mostly drawn on theories that already existed in underlying disciplines, mostly in the social sciences. these are conventionally referred to as 'reference disciplines', although a critical observer would find it difficult to avoid using the term 'derivative' to describe a moderate proportion of the work in the discipline. the approaches adopted were diverse from the outset, and no single text-book or school of thought dominated. the ideas of langefors (1963, 1966), despite being translated into english at a very early stage, did not take strong root in australia. perhaps the single most influential book during the early years was gordon davis' 'conceptual foundations' (1974, 2nd edition as davis & olson 1984). this provided a broad and liberal set of perspectives, and could be held up as an antidote to the more mechanistic and narrowly rationalist, often technology-driven, approaches. u.k. perspectives, particularly those in mumford & banks (1967), checkland (1981), mumford (1983) and wood-harper et al. (1985), had considerable influence in some departments but little or none in others. 130 australasian journal of information systems volume 14 number 1 nov 2006 cataloguing the diversity of topics and approaches is a terrifying endeavour, but any discussion of a discipline's history simply cannot afford to overlook attempts at the encyclopaedist's approach. early endeavours to define the scope included mason & mitroff's (1973) 'program for research on mis' and ives et al.'s (1980) 'framework for research in computer-based mis'. local contributions included a 'manifesto for australian-based research' (galliers 1987), and a special issue on 'current research directions in is' (jeffery & lawrence 1990). reviews of the research undertaken in is include culnan (1986, 1987), alavi et al. (1989), alavi & carlson (1992), avgerou et al. (1999), galliers & whitley (2002) and banker & kauffman (2004). studies conducted in australia include galliers (1987), ridley et al. (1998), pervan & cecezkecmanovic (2001), pervan & shanks (2004) and pervan & shanks (2006). these all attest to the enormous breadth of topics addressed within the is discipline. the diversity arises in several dimensions: • cross-sectionally, reflecting: • the diversity of origins; • the diversity of host disciplines and co-located disciplines; and • longitudinally, as drift has occurred over time, driven by changes in technology, in fashion in management and in management disciplines, and increasingly in fashion within the is discipline itself. barki et al. (1988, 1993) attempted to enumerate and provide structure to the topics that were within-scope of is. their proposed taxonomy of c. 2,000 terms was used for some years, but has since been abandoned because the rate of change is too high and no-one has the energy and authority to maintain it. rather than adopting the barki scheme, exhibit 2 suggests a clustering of themes and topics in is in australia since 1965. exhibit 2 depicts a clustering, not a taxonomy. as is apparent from the appearance of such topics as adoption, systems analysis and project management in multiple clusters, there has been continual cross-feeding, and co-evolution of thinking. the list of themes also evidences enormous diversity of research domains. these are viewed through the lenses of many theories, most of them imported, and imported from a wide variety of 'reference disciplines'. that the discipline's internal and external boundaries are so ill-defined and porous can be readily seen as an intellectual strength, because it implies adaptability and preparedness to focus on current needs. it can also be seen as an intellectual weakness, because it raises the question as to whether is has the cohesion and substance required of a discipline. as will be discussed later, this also represents a political weakness. technology as enabler and driver, including computers, edp, computer usage, applications development, the software development life cycle (sdlc), later the systems life-cycle (slc), systems design, systems analysis, project management, usability, technology adoption, technology acceptance, and impediments to adoption; applications of technology, including accounting / fmis / erp, inventory / supply chain, management and executive applications (dss, eis, business intelligence and knowledge management), crm, ecommerce, ebusiness, industry-sector-specific applications; data management, including file-handling, data base management systems (dbms), data modelling, data dictionaries, entity/attribute/relationship (ear) modelling; organisations, as the primary context within which information systems are developed and operated, and for whose purposes they are applied, including organisational behaviour, 131 australasian journal of information systems volume 14 number 1 nov 2006 requirements elicitation and analysis, business process analysis, project management, change management, usability, technology adoption, and impediments to adoption; systems thinking, including o&m, gst and cybernetics leading from sdlc to slc, sociotechnical theory, soft systems methods, incorporation of human factors, usability, adoption, and merging with human behaviour and communications into semiotics and perhaps ontological foundations; business school thinking, including operations management, or/ms, management accounting, controls and audit, project management, management of dp/is/it/ict, strategic is, information management, usability and adoption; and information management, which grew in part out of library / information science, and delivered a vital appreciation of the role of metadata in enabling the discovery and management of information. exhibit 2: themes and topics in australian is the diversity is as apparent in method as it is in research domain. the way to recognition and respect has been perceived by many academic leaders to be via commitment to methodological rigour. the natural byproduct has been harm to relevance, as researchers focus on the empirically researchable. an example of this is that electronic data interchange (edi) has continued to be a focus in refereed publications in the ecommerce literature for years after it was rendered largely irrelevant by the explosion of the internet. this was because edi was stable and measurable, and hence could be subjected to more rigorous analysis than phenomena that were rapidly mutating. whether and when the discipline will recover a healthier balance between rigour and relevance remains an open question. linked with that problem is the tendency to view the primary audience for the work by is researchers as being one another. there is a dire risk of withdrawal to the 'ivory tower', and of becoming self-referential and self-serving. a healthy discipline of course involves active discourse among its members, but it also focuses on well-defined external constituencies. serving those constituencies demands that much of the research not be 'pure', but be actively and willingly 'tainted' by applied notions (applying tools to problems) and by instrumentalist notions (tackling problems, and applying whatever tools appear likely to assist in the process). the is discipline in australia could be seen to have slid over the last four decades from an excess of pragmatic relevance to a dangerous obsession with purist rigour. organisational settings it was noted above that early is staff were loners. for many is staff, that has continued to be the case, because, unlike most of the established disciplines, is is frequently not a department in its own right. the most reliable information available (which is of at best moderate quality) is in the directories of is academics published in 1988-94 (clarke 1988, 1991, gable & clarke 1994, 1996), and integrated into the worldwide is directory when it was launched by isworld in 1995. unfortunately, the data's quality since that time has dropped enormously, because no systematic follow-up has been undertaken. nor has the potential for auto-updating from university web-sites been exploited. by the end of the 1980s, almost all institutions had specialist organisational units focussed on is. but the staff-count in those departments represented little more than half of the total, with many people out-posted in other departments. a significant difference from the patterns in the u.s.a. has 132 australasian journal of information systems volume 14 number 1 nov 2006 been the relatively very limited involvement of and interest in is by australian graduate schools of business, especially until the mid-1990s. year count of institutions count of named is depts count of depts %age of depts named is count of individuals count of professors 1988 41 9 55 16 175 2 1991 39 22 76 29 521 7 1994 38 32 84 38 640 13 1996 39 39 88 44 630 15 2005 42 28 103 28 692 30 exhibit 3: institution and staff statistics exhibit 3 provides a statistical summary of the data in the directory at various points in time. the growth surge apparent between 1988 and 1991 may be exaggerated by under-reporting in the first edition, but probably not greatly. generally the data quality is likely to be highest in the figures for 1991, 1994 and 1996, which are based on project-managed surveys. no snapshot is available between 1996 and 2005. if one were, it would have been likely to show peak counts of individuals and professors, and possibly also of departments. the counts of individuals and professors in 2005 is under-stated because many new appointees have failed to create entries, but that is somewhat counter-balanced by the existence of undeleted entries for staff who have departed. during the 1980s, is staff with relevant doctorates were an aspiration for most departments, rather than a reality. there has been an increase in the number of degree-granting institutions, but many are not in a position to provide effective supervision for more than a small number of doctoral candidates, and few departments have articulated doctoral programs. australian representation in international refereed conferences has been consistently strong since the late 1980s, but the increased professionalism is only slowly converting into impact in international journals. this can be attributed to strong competition arising from the ongoing high productivity of american scholars, the higher level of journal-publication productivity of european scholars in recent years, and the explosion in doctoral programs in other countries. political dimensions from 1965 to c. 1990, communications within the discipline in australia were informal and somewhat haphazard. an early step to draw the scattered individuals and groups together was the development of a directory (clarke 1988, 1991, gable & clarke 1994, 1996). a critical initiative was the establishment of the national conference, the australian conference in information systems (acis). the first was held at monash in 1989, and it has run annually since then. during the first few years, the standing committee comprised ross jeffery, ron weber, roger clarke, peter weill 133 australasian journal of information systems volume 14 number 1 nov 2006 and igor hawryszkiewycz. the committee was then migrated to the icis pattern of rotating membership involving recent, current and near-future organisers. the mid-1990s saw maturation of the is discipline at the international level. as the internet was grasped as an opportunity for international communications and publication, the isworld mailinglist and web-site were established, both in 1994. the international association for information systems (ais) was also formed in that year. the online worldwide directory consolidated the three regional printed directories in 1995. the regional fora pacis (from 1993), ecis (from 1993) and amcis (from 1995) emerged to provide a broader geographical frame for acis. australians have been very active contributors to pacis, isworld, the ais and the directory project, and to ecis, ifip tc8 and other international conferences. meanwhile, the national specialist journal the australian journal of information systems (ajis) was established in 1994. liaison among professors and departmental heads had been emergent for some time, and was formalised through the australian committee of professors and heads of information systems (acphis) in 1995. an isworld page for australia was established by this author in 1996. some years later, a chapter of ais was established (aais, formed 2001). despite the gradual development of these cultural institutions, is in australia has remained marginalised, in many cases perceived as a mere service discipline, or as an interloper. for example, two 'sandstone universities' created departments only c. 1995 (melbourne) and c. 2000 (sydney), and adelaide and u.w.a. have never had significant departments. the strongest employers were the 'red brick' universities, the institutes of technology, and specific colleges of advanced education. in almost all cases, the big players had a strong orientation towards the technology flavour of the discipline, although some had sufficient critical mass to enable multiple perspectives to be nurtured, and to be reflected in coursework as well as research. at the end of the 1980s, re-structuring of the tertiary education sector was forced by labor minister john dawkins. the revolution was multi-pronged. it disestablished about 40 caes and 25 other smaller organisations and forced their amalgamation variously into the existing 19 universities and 6 sometime institutes of technology, or into one of about 15 new combines (avcc 2004). the changes destroyed the highly valuable distinction between institutions with industry-oriented mission statements and those with primarily academic orientation, resulting in wasteful confusion as caes struggled to become strong research-oriented institutions. the success of the commonwealth bureaucracy in imposing the changes gave rise to a culture of interventionism by the relevant agency, most recently the department of education, science & training (dest). flurries of additional administrative responsibilities have been imposed on universities, drawing resources away from teaching and research. per-student funding was slashed, and universities were forced to become entrepreneurial at short notice. the nominal objective of achieving administrative efficiencies was argued from the outset to be a mirage, and transpired to be just that. lacking skills, and forced to learn them very quickly, many universities made ill-judged forays into crowded and difficult export markets, many suffered further financial damage as a result, and some teeter on bankruptcy. managerialism rapidly replaced collegial structure and processes, and profit-making, or at least financial survival, dislodged all of the other objectives that universities had once had. teaching standards plummeted under the assault of much-increased student/staff ratios, lower entry-points as institutions converted themselves into degree-factories for the upper half of matriculants, and huge increases in the numbers of fee-paying foreign students whose preparation and whose english were mostly below the intakes of earlier years, and in some cases well below. 134 australasian journal of information systems volume 14 number 1 nov 2006 as a minor political player in an increasingly politicised environment, is has suffered. the proportion of departments containing is that were named is or similar grew steadily from the late 1980s to the mid-1990s to nearly half, but has plummeted since the end of the 1990s. pearcey wrote nearly two decades ago (1988, p. 125) that "the demand for people with computing expertise has always outstripped the capacity of the tertiary sector to supply it, and the situation seems unlikely to change". based on the author’s experience, that was the case for at least three decades, from the late 1960s until the end of the 1990s. about 2000, two factors conspired to cause dramatic change. demand for is graduates had become over-heated during the 'dot.com bubble' of the mid-to-late 1990s. investors then made the remarkable discovery that most start-ups were presaged on establishing dominant market-share in a market that not only did not exist, but would not come into existence in time, if ever; and that hence almost all start-ups would fail. this resulted in a sudden loss of investor confidence, and the bubble burst. the other key factor was that offshore outsourcing had been progressively extending from data capture to programming and even detailed design work, reducing demand for local graduates. the publicity accompanying this very large 'market correction' was followed by substantial reductions in enrolments from domestic students although it appears to have had a smaller impact on foreign fee-paying numbers. a further factor in the disappearance of is departments has been the external financial pressures on universities, which have encouraged the imposition of departmental amalgamations in the hope that this will result in cost-savings. the 2005 staff head-count figures in exhibit 3 are believed to reflect the position after the majority of the slashing had been undertaken. the timings of the down-sizing varied, but the quantum appears to have been mostly reasonably even across the country, with 20-30% decreases the norm, although a small number were as high as 50%. as is the way with 'the invisible hand' so beloved of economists, it appears very likely that the slump will have been an over-correction, and that there will soon be shortages in trained staff. the damage done to the is discipline following the 'dot.com bust' in 2000 is therefore in part understandable, and in part interim. in part, however, it reflects the discipline's ongoing political weakness. this meant that it was less able to gain control over the financial resources attributable to it during the boom times. subsequently, during the recent downturn in student interest, it was less able to resist closures by demanding cross-subsidy, as has long been the norm among humanities and science disciplines that fall on hard times. further evidence of seriously inadequate political influence was the lack of formal recognition in the grants scheme (originally australian research grants council – argc, later australian research council – arc). the coding scheme has only included specifically is terms since about 1998, and is representation on the expert panel was only achieved in 2001. it might have been feasible for the discipline to bolster its position within individual institutions and with government by drawing support from industry and government users. it has almost entirely failed to do so, however, because its links with employers of its graduates have always been tenuous and sporadic. few departments have even created, let alone sustained, programs targeted at executives, and most employers avoid, and even deny, any responsibility to contribute to the preparation of the people they hire. meanwhile, most professional development training continues to be product-specific and offered by suppliers, or somewhat more generic but offered by 135 australasian journal of information systems volume 14 number 1 nov 2006 consultants. there is very limited linkage between on-campus postgraduate studies and employersponsored workplace training. conclusions the is discipline in australia has grown from a standing start in 1965 to a body of c. 700 people, including c. 30 professors, representation in every university, and a comprehensive set of cultural institutions. on the other hand, the discipline has always been characterised by porousness, derivativeness and unmanageable diversity, and has interleaved focus on discipline with a 'multidisciplinary' concern about research domains. these have translated into organisational fragmentation, ongoing self-doubt, and a lack of political power. is is really a discipline? and does it matter if it isn't? is there a core? is it so heavily dependent on technology and management fashion that it can never have the stable core necessary for a recognised discipline? is is not a discipline, but merely a research domain that needs to be viewed through the lenses of a variety of genuine disciplines? put more colourfully, are is academics destined to wander forever, as rosencrantz and guildenstern to hamlet, backstage bit-actors to host-discipline leads, under ongoing threat of irrelevance, and even execution? has the value of the is discipline been ephemeral? does it need to be absorbed by broader disciplines either side of it? does it need to continue to exist much as it does now, but with less energy wasted on existential angst? or is existential angst simply a necessary rite of passage for a growing discipline? during recent years, debate has raged about ‘the core of the discipline’, and the risk exists that there may be a reduction in scope from ‘systems that handle information’ to some notion of an ‘it artefact’. a further concern is that the is discipline needs to extend its scope beyond exploratory, descriptive, explanatory and predictive outcomes to embrace the normative as well. rather than ceding policy-relevant research to other, less timid disciplines, it is essential that is declare needs associated with human values, and consider system features that assist in and that militate against the satisfaction of those needs. this would force maturation in the naïve notions of rigour and objectivity that have dominated the discipline in recent years. this special issue is published soon after the 40th anniversary of the discipline in australia, and as the generation of people active in the early years hand over the reins to succeeding generations. in a context as dynamic as that which afflicts is it would be foolhardy to commit too much time to purely historical studies. on the other hand, careful study and consideration of the threads of history may provide the perspective needed in order to avoid further repetition of harmful errors. the first four decades of the is discipline in australia saw progress and growth achieved, but in a context of multi-dimensional change, uncertainty and adversity. the next decade clearly promises more of the last three. whether further progress and growth will be achieved, and even whether the discipline will survive in its present form, depends on the choices that today's leaders make about the discipline's scope, core, orientation, and attitude to the relevance-rigour balance. 136 australasian journal of information systems volume 14 number 1 nov 2006 references ackoff r.l. (1969) 'management misinformation systems' mngt. sci. 14, 4 (december 1967) 147-156 alavi m., carlson p. & brooke g. (1989) 'the ecology of mis research: a twenty year status review' proc. int'l conf. infor. syst., 1989, p. 198 alavi m. & carlson p. (1992) 'a review of mis research and disciplinary development' j. mngt. infor. syst. 8, 4 (1992) 45-62 avcc (2004) 'australian higher education institutions' australian vice-chancellor's association, 4 november 2004, at http://www.avcc.edu.au/documents/universities/australianhemergesamalgamations.pdf avgerou c., siemer j. & bjorn-andersen n. (1999) 'the academic field of information systems in europe' euro. j. infor. syst. 8 (1999) 136-153 banville c. & landry m. (1989) 'can the field of mis be disciplined?' commun. acm 32, 1 (january 1989) 48-60 barki h., rivard s. & talbot j. (1988) 'an information systems keyword classification scheme' mis qtly 12, 2 (june 1988) barki h., rivard s. & talbot j. (1993) 'a keyword classification scheme for is research literature: an update' mis qtly 17, 2 (june 1993) benbasat i. & zmud r.w. (2003) 'the identity crisis within the is discipline: defining and communicating the discipline’s core properties' mis qtly 27, 2(june 2003) 183-194 brookes c.h.p., grouse p.j., jeffery d.r. & lawrence m.j. (1982) 'information systems design' sydney, prentice hall, 1982 caminer d., aris j., hermon p. & land f. (1998) 'leo the incredible story of the world's first business computer' mcgraw hill, new york checkland p. (1981) 'systems thinking, systems practice' wiley, chichester, 1981 clarke r. (1987) 'the computing professional's bookshelf' austral. comp. j. 19, 4 (december 1987) 222-223 clarke r. (ed.) (1988) 'australian information systems academics: 1988/89 directory' australian national university, november 1988 clarke r. (ed.) (1991) 'australasian information systems academics: 1991 directory' australian national university, april 1991 clarke r. & lo b. (1989) 'accreditation requirements for information systems courses for the australian computer society' australian computer society, november 1989 couger j. (ed.) (1973) 'curriculum recommendations for undergraduate programs in information systems' commun. acm 16, 12 (december 1973) 727-749 culnan m. (1986) 'the intellectual development of management information systems, 1972–1982: a co–citation analysis' mngt. sci. 32, 2 (1986) 156–172 137 australasian journal of information systems volume 14 number 1 nov 2006 culnan m. (1987) 'mapping the intellectual structure of mis, 1980–1985: a co–citation analysis' mis qtly 11, 3 (september 1987) 341–353 davis g.b. (1974) 'management information systems: conceptual foundations, structure, and development' mcgraw-hill, 1974 davis g.b. & olson m.h. (1984) 'management information systems: conceptual foundations, structure, and development' 2nd ed., mcgraw-hill, 1984 dearden j. (1972) 'mis is a mirage' harv. bus. rev. january-february 1972, 90-99 gable g. & clarke r. (eds.) (1996) 'asia pacific directory of information systems researchers: 1996' national university of singapore, 1996 gable g. & clarke r. (eds.) (1994) 'asia pacific directory of information systems researchers: 1994' national university of singapore, 1994 galliers r.d. (1987) 'information systems planning: a manifesto for australian-based research' austral. comp. j. 19, 2 (june 1987) 49-55 galliers r.d. & whitley e.a. (2002) 'an anatomy of european information systems research ecis 1993-ecis 2002: some initial findings' proc. 10th euro. conf. infor. syst., gdansk, poland, june 2002 ives b., hamilton s. & davis g.b. (1980) 'a framework for research in computer-based management information-systems' management science 26, 9 (september 1980) 910-934 jeffery d.r. & lawrence m.j. (eds.) (1990) 'current research directions in information systems' special issue, austral. comp. j. 18, 4 (december 1986) 157-158 johnson l.r. (2006) 'coming to grips with univac' annals of the history of computing 28, 2 (april-june 2006) 32-42 keen p.g.w. (1980) 'mis research: reference disciplines and a cumulative tradition' proc. 1st int'l conf. infor. syst., philadelphia, december 1980, 9-18 keen p.g.w. (1991) 'relevance and rigor in information systems research: improving quality, confidence, cohesion and impact' , in nissen h.e., heinz k.l. & hirschheim r. (1991) pp. 27-49 land f. (2000) 'the first business computer: a case study in user-driven innovation' j. annals of the hist. of computing 22, 3 (july-september 2000) 16-26 langefors, b. (1963). some approaches to the theory of information systems, bit, bind 3, hefte 4, 229-254 langefors b. (1966) 'theoretical analysis of information systems' lund, sweden: studentlitteratur, 1966 lyttinen k. & king j.l. (2004) 'nothing at the center?: academic legitimacy in the information systems field' j. assoc. infor. syst. 5, 6 (june 2004) 220-246 mason r.o. (2005) 'putting 'systems' back into information systems: an essay in honor of gordon davis' working paper, may 2005 mason r.o. & mitroff i.i. (1973) 'a program for research on management information systems' management science 19, 5 (january 1973) 138 australasian journal of information systems volume 14 number 1 nov 2006 maynard g.b. (1992) 'report of the task force on the acs towards 2000' australian computer society, 1992 mumford e. & banks o. (1967) 'the computer and the clerk', routledge kegan paul, 1967 mumford e. (1983) 'designing human systems', manchester bus. sch., 1983 nissen h.e., heinz k.l. & hirschheim r. eds. (1991) 'information systems research: contemporary approaches and emergent traditions' north-holland, amsterdam orlikowski w.j. & baroudi j.j. (1991) 'studying information technology in organizations: research approaches and assumptions' infor. syst. res. 2 (1991) 1-28 orlikowski w. & iacono s. (2001) 'desperately seeking the 'it' in it research – a call to theorizing the it artifact' infor. syst. res. 12, 2 (june 2001) 121-134 pervan g. & cecez-kecmanovic d. (2001) 'the status of information systems research in australia: preliminary results' proc. 12th austral. conf. in infor. syst., december 2001 pervan g. & shanks g. (2004) 'is research activity in australia: results of the 2004 acphis research survey' presentation to austral. conf. infor. syst., december 2004 pervan g. & shanks g. (2006) 'the 2005 survey of information systems research in australia' austral. journ. of infor. syst., 2006 ridley g. (2006) ‘characterising information systems in australia: a theoretical framework’ austral. journ. of infor. syst., 2006 ridley g., goulding p., lowry g. & pervan g.p. (1998) 'the australian information systems research community: an analysis of mainstream publication outlets' austral. l. infor. syst. 5, 2 (1998) 69-80 underwood a. (1997) 'the acs core body of knowledge for information technology professionals' australian computer society, 1997 weber r. (2003) 'still desperately seeking the it artifact' mis qtly 27, 2 (june 2003) iii-xi wood-harper a.t., antill l & avison d.e. (1985) 'information systems definition: the multiview approach' blackwell, oxford, 1985. acknowledgements guy gable provided the stimulus for this paper, and he, bob smyth and other members of the 'is in oz' project team provided important input to the work. the paper has benefited greatly from interviews with the following key players in is, listed in alphabetical order: cyril brookes, bill caelli, frank land, stewart leech, gerry maynard, graham pervan, and ron weber. at least a dozen further senior members of the discipline made important contributions, including dick mason. responsibility for the errors, the omissions, the unfortunate mis-phrasings, and the judgementally impregnated expressions, rests with the author. 139 australasian journal of information systems volume 14 number 1 nov 2006 appendix the working paper from which this paper was developed is: clarke r. (2006) 'a retrospective on the information systems discipline in australia' xamax consultancy pty ltd, april 2006, at http://www.anu.edu.au/people/roger.clarke/sos/aishist0605.html the resources developed as part of the project are accessible at http://www.anu.edu.au/people/roger.clarke/sos/aishistapps0605.html. they comprise: 1. bibliography 1a: historical and reflective articles on is in australia 1b: early articles on is in australia 1c: curriculum-related publications 2. resource sites 2a. resources – international 2b. resources – regional 2c. resources – australian – publications 2d. resources – australian – organisations 3. timelines 3a. timeline – international 3b. timeline – australian 4. early australian professors in is 5. early australian phds in is 6. the international impact of australian is it is intended that the working paper and the resource-set be progressively corrected, refined and expanded, and archived in ajis, aais and acs repositories. contributions should be sent to the author. the author roger clarke is principal of xamax consultancy pty ltd, based in canberra. he completed bcomm (hons i) and mcomm degrees parttime at the university of new south wales (unsw) from 1967 to 1976, and taught is there 1973-76 and in 1983. he was reader in information systems at the australian national university (anu) from 1984 to 1995, during which time he was awarded a phd by supplication. he is a visiting professor in the department of computer science at the anu, in the e-commerce programme at the university of hong kong, and in the cyberspace law & policy centre at unsw. during the last 20 years, his primary interests have been in ebusiness, information infrastructure, and dataveillance and privacy. © xamax consultancy pty ltd, 2006 140 acknowledgements appendix the author ajis vol. 6 no. 2 may 1999 editorial this is the first issue of volume 6 of the australian journal of information systems. as editor i am pleased to report the ongoing interest in the journal both in australia and overseas. our first paper preliminary insights into the implementation of standardised it in franchise systems angele cavaye, department of information systems, university of southern queensland, colin mccosker & chad perry, department of marketing, university of southern queensland uses case studies to investigate it implementation in franchise systems. the paper examines both franchiser and franchisee factors and shows that both impact on it implementation. our second paper refinement of research surveying in software methodologies by analogy: finding your patch eugene doroshenko, department of computer science and software engineering, university of melbourne & felicity lear, school of information systems, university of tasmania introduces a model which can indicate field maturity. the paper examines and describes the use of this model. dieter fink, school of management information systems, edith cowan university provides our next paper business students' perceptions of electronic commerce will they join the revolution. this paper shows that the main concern of business students is the insecurity of ec in such areas as computer viruses, theft of information, and unauthorised access to data by intruders. as a consequence, the trust the students currently have in ec is relatively low. our next paper is integrating is and hci using activity theory as a philosophical and theoretical basis helen hasan, department of business systems, university of wollongong. this paper describes the basic principles of activity theory as they apply to is and hci and uses a case study to show that an integrated approach would have improved the project. our next paper practical and tacit knowing as a foundation of information systems john haynes, department of information systems, ucol, new zealand explores the importance of the human subject, with its inherent intuition, as a foundation for information systems. the paper examines intuition from several philosophical standpoints. our next paper applying quality costs in a software development environment i.p. hollingsworth, department of commerce, university of birmingham, w. keogh aberdeen business school, the robert gordon university & m.h. atkins, department of management studies, university of aberdeen shows that quality costs can be a measure of software quality. the paper examines several models of quality cost and discusses the major issues involved in setting up a quality cost programme. elizabeth kemp, institute of information sciences and technology, massey university provides our next paper object oriented analysis: a voyage of discovery. using the experiences of a business analyst on several object-oriented projects, suggestions are made for organisations migrating to and object oriented environment. our final paper is the origin of the year 2000 date problem: an alternative hypothesis richard kingsford, parliamentary information systems office, parliament of australia & leone dunn, school of information technology and computer science, university of wollongong. this paper proposes that the fundamental origin of the problem lay in pre-existing western culture and that this cultural practice set western society on a collision course with the technology of the future. once again i would like to pass on my thanks to the authors and consulting editors for the high quality of material in this issue. i would like to thank lily soh, ted gould and deborah bunker for their continued assistance with the journal and david dodds for the maintenance of the home page. rob macgregor editor http://www.uow.edu.au/ajis/ajis.html ajis vol. 8 no. 2 may 2001 aspects of text miningfrom computational semiotics to systemic functional hypertexts alexander mehler university of trier mehler@uni-trier.de abstract the significance of natural language texts as the prime information structure for the management and dissemination of knowledge in organisations is still increasing. making relevant documents available depending on varying tasks in different contexts is of primary importance for any efficient task completion. implementing this demand requires the content based processing of texts, which enables to reconstruct or, if necessary, to explore the relationship of task, context and document. text mining is a technology that is suitable for solving problems of this kind. in the following, semiotic aspects of text mining are investigated. based on the primary object of text mining natural language lexis the specific complexity of this class of signs is outlined and requirements for the implementation of text mining procedures are derived. this is done with reference to text linkage introduced as a special task in text mining. text linkage refers to the exploration of implicit, content based relations of texts (and their annotation as typed links in corpora possibly organised as hypertexts). in this context, the term systemic functional hypertext is introduced, which distinguishes genre and register layers for the management of links in a poly-level hypertext system. introduction the set of documents available online increases rapidly. this is evident when regarding the availability of text, image, audio and video documents as part of intranets, extranets or the internet. the majority of these documents are natural language texts lacking schematic structures resulting from the usage of forms or text building blocks.2 due to the enormous increase of documents for the textual representation of business and public organisations, an immense problem of information processing emerges, which is tied to the following exemplary questions: which documents are relevant for which task in which context? how does an organisation's member find these documents? has any document been omitted, and if so, in which sense? these questions cannot be answered solely with the help of information retrieval techniques. information retrieval is concerned with the problem of retrieving indexed texts from pre-processed document collections on the basis of structured queries as part of standardised retrieval languages. in contrast to this, the problem of information processing outlined above is characterised by its uncertainty or more specifically: by its vagueness and ambiguity4 as well as by its contextsensitivity and dynamics. among other aspects, vagueness is a type of uncertainty referring to the impossibility of making precise distinctions and assignments, whereas ambiguity denotes the existence of diverse, dissonant, nonspecific, and possibly competing alternatives. with regard to the information processing, these types of uncertainty can be characterised as follows: given a task t connected with a possibly vague information need / in context c, it may not be categorically decidable, whether a document x is relevant for t depending on c, or not (contextsensitivity and vagueness).5 in addition, there are possibly many documents that are assumed to be relevant for t in context c (ambiguity in the sense of nonspecificity). since there may exist a collection of texts, each of which cannot be categorically assigned to the given task, vagueness and ambiguity are types of uncertainty, which besides context sensitivity simultaneously apply to the task of text mining. beyond context sensitivity and uncertainty, the dynamics of the relation of task, context, and text has to be considered: as organisations, their tasks and contexts change, the evaluation of documents as being relevant changes, too. to summarise: there is not only a fuzzy many to many relation between texts, contexts, and tasks (the same text can be relevant for the same task to varying degrees in different contexts), but this relation has a further parameter: time. due to the existence of 2 clearly, this "lack" does not indicate a deficient mode of textual organization. on the contrary, natural language texts represent the most efficient information structure for the dissemination of knowledge in human communities. 3 see sparck jones (1999:7). 4 see zadeh (1997) for a constraint-based formalization of different terms of uncertainty. see also klir/folger (1988) for information theoretical definitions of these terms. 5 the term task as well as the term organization is used in the following rather unspecifically. a task is a generic term for any target, which demands a specific state to be realised by members of a given organization. in case of economic organizations, the general operating purpose of the corresponding enterprise (determined by the production of certain goods or the provision of services) determines its tasks. for a text to be relevant for performing a specific task means for example that processing this text may help to reach the goal defined by the task. in a more elaborate model these organization theoretical terms and their relations need to be defined in more detail. 129 ajis vol. 8 no. 2 may 2001 varying contexts for judging a text as task relevant, which cannot be explicitly coded in the document collection under consideration, an information processing problem emerges, whose degree of virulence correlates with the grade of uncertainty of the dynamic mapping of texts, contexts, and tasks. in any case, a risk to overlook relevant or to receive irrelevant documents emerges.6 now, three problem categories of increasing complexity can be distinguished:7 • the retrieval problem: a known already structured information space s is confronted with determinate, structured information needs. this problem category characterising the classical field of application of information retrieval and database systems is associated with questions of the following type: which information units of the already structured information space s have the features f\,ufn? in information retrieval, features are index terms and s is a collection of indexed documents. in the area of database systems, features are constituents of a database query, and s is a relational or object oriented database. though the mapping of information units and features fh i e {!,...,«}, may be uncertain (as in case of fuzzy databases or graded relevance judgements of retrieved documents), the set of information units and their features is supposed to be determined. contexts can only be modelled by means of enumerable, static features ft. the retrieval problem presupposes the solution of categorisation tasks: • the categorisation problem: an unknown, unstructured information space s is confronted with determinate structured information needs.8 this scenario is characterised by questions of the following type: to which features f\,t\fn do the information units of s belong? this question goes beyond the domain of the retrieval problem since it does not presuppose that the information units are part of an already structured information space. though the set of features ft is supposed to be known (e.g. as being elements of a thesaurus, a set of index terms, or a set of topic categories), it is unknown, which information units have which features. the categorisation problem is addressed in the area of text classification. it clearly arises during the automatic build-up and maintenance of ir systems. from the perspective of an organisation's member, a categorisation problem emerges when she is able to describe her information need, but does not know how to connect her description with the unknown information space. • the exploration problem: an unknown, unstructured, dynamic information space is confronted with an uncertain, unstructured information need. this problem category is connected with the characteristic question: which information units are or become relevant for which task in -which sense under which circumstances? regarding the categorisation problem, the presupposition is abandoned that the relevant features describing the information need are known, static, context insensitive, and unstructured. from a member's perspective, this problem arises when she is not conscious of her information need and does not know much or even anything about the documents available, their contents and respects that make them relevant for her task. this is the kind of information processing problem described above with recourse to the terms of uncertainty, contextsensitivity, and dynamics. these three interdependent problem categories can be illustrated as follows: suppose, a member of an organisation deals with a task 7] described by a set of directly available documents xi as part of the information space s. if the organisation disposes of an information retrieval system and the member knows to describe her information need based on the corresponding query language, she can formulate a query in order to select task relevant documents x d xt. hence, a retrieval problem arises.9 if there is no retrieval system or if it only covers a subset of s, the content based connection of 7} and 5 has to be detected, first. suppose that there exists knowledge about characteristic features of 7), e.g. with respect to the type to which it belongs. in this case, a categorisation problem arises: those documents, which are categorised by the known features of 7), have to be determined. in the case of missing knowledge, an exploration problem arises that can be solved approximately as follows: maybe there exists a task 7} (not necessarily known to the user), which is connected with a document set xj (for the description of 7} and its realisation) and is somehow related to task 7} (maybe, 7} is a sub-, subsequent or concurrent task of 7), etc.). in this case the (type of) relation of 7} and 7} has to be discovered and documents serving for describing both complexes have to be linked. since the document set xt is supposed to be the only available information-describing task 7}, it serves (among others) as an informational basis for exploring such 6 users do not survey the documents available, their contents and the respects under which they may become relevant for their tasks (information needs). 7 these three categories only describe extreme cases of a continuum of more subtle tasks. 8 the case of a structured information space (e.g. an indexed document collection of an information retrieval system) which is confronted with an unstructured information need will not be taken into consideration. it may be seen as a learning problem of how to use the corresponding query language in order to select information units from the information space. 9 in this context, the property of the information space to be known means that the documents which the member is seeking are supposed to be an already indexed part of/, so that they can be requested using the retrieval language. 130 ajis vol. 8 no. 2 may 2001 content based, text-spanning relations. this is illustrated in figure (1) by means of a commuting diagram, where r denotes the relevance relation between tasks and documents, d is the type of dependence of 7} and 7}, and <-» parallels this dependence on the level of documents. on the background of the dependence of t, and 7} it is explored that the set of documents xj is related to 7}. an alternative, bottom-up strategy would be to explore xt in order to derive features describing 7] and then to look for documents in s, which are fuzzily categorised by these features. as tasks tt, 7} change, this procedure needs to be repeated. figure 1: the (intertextual) dependence of documents on the background of two tasks. the solution of the retrieval problem presupposes that all items which can be retrieved are determined by the indexation of the information space s. items that are not indexed cannot be retrieved. the retrieval capacity is determined by the antecedent process of indexation, and does not include processes of reconstruction or even exploration. in contrast to this, the exploration problem arises from the twofold uncertainty, contextsensitivity and dynamics of information need and information space: not only does the contextsensitive connection between information need and information units have to be explored, but it also varies as the organisational context evolves. in other words, the exploration problem cannot be solved by indexation techniques, because the universe of (social-semiotic) contexts cannot be extensionally represented ex ante. the categorisation problem has an exceptional position in the sense that a solution of the exploration problem includes a solution of the retrieval problem, which on its part requires a solution of the categorisation problem. the automatic solution of these three tasks primarily demands the use of minimal knowledge resources: instead of presupposing a broad range of knowledge derived from human expertise, a sparse, procedural model is required, which is able to learn relevant knowledge from natural language texts, task or context descriptions. this is a prerequisite for the re-usability, adaptability, and transferability of the same solution in an area of rapidly changing tasks, contexts, and domains of interest. text mining text mining is a technology that is especially suitable for solving problems of the type of the exploration task outlined above. objects of text mining are natural language texts, which lack the explicit, schematic organisation of formal data structures. instead of providing a nominal definition of text mining, it is operationally outlined by specifying the order of typical text mining procedures.10 given any text corpus c, the following text mining steps can be distinguished: 1. reconstruction: it is the reconstruction of linguistic knowledge underlying the usage of text components and their intra-, intertextual and exophoric relations that occurs first. "reconstruction" means that knowledge is discovered and not simply matched with predefined schemata or patterns that was (or has been) constitutive for the production (reception) of texts in c." instead of starting with predefined, fixed and restricted knowledge domains, reconstruction aims at replacing factual by procedural knowledge of how to acquire/learn (domain or task) relevant knowledge. this processing stage is not bound to the analysis of corpus c, but may include the processing of a (possibly much larger) training corpus. for this purpose, methods of computational linguistics, machine learning, knowledge engineering (data mining), statistics and many other fields are applied. 2. systematisation: secondly, the systematisation of the structures reconstructed in the previous stage occurs. one has to examine, which types (classes or clusters) are to be distinguished based on which kind of text function. a type allows to subsume linguistic units showing similar or homogeneous usage regularities 10 see hearst (1999) for a "negative" definition of what she calls text data mining, which she distinguishes from information retrieval, data mining, and corpus-based computational linguistics. 11 see also hearst (1999). 131 ajis vol. 8 no. 2 may 2001 with respect to a given function (in the corpus under consideration). systematisation aims at reconstructing higher level linguistic or conceptual units, such as types of rhetorical structure, schematic structure, etc. reconstruction and systematisation are not strictly ordered. they may be organised with the help of a feedback mechanism so that relations of types (classes) are reconstructed on their own, which are used for evaluating (confirming/modifying) representations of relations of their instances. 3. exploration: subsequently, heretofore unknown, task relevant, structural or referential relations, which were not (necessarily) constitutive for the texts under consideration, are explored}2 this can be exemplified as follows: suppose a situation, in which n persons produce texts, each of which is only known by the corresponding author. tthere may be relationships of semantic similarity helping to facilitate, clarify or augment the reception of these texts so that the reader gains information which she cannot obtain from processing any of these texts in isolation. as in case of the distinction of categorisation and exploration tasks, reconstruction, systematisation and exploration are not categorically separated. 4. extraction, annotation and visualisation: finally, implicit relations that were discovered during the first three steps are made explicit with the help of annotation and visualisation techniques. this step has the aims of making explicit the information reconstructed or explored before so that a user can easily skim the value-added information inherent to the texts under consideration. examples are the annotation of attributes (and their values) as well as relations (links) of text components (using a standardised annotation language like xml), their visualisation in hierarchies or networks, the summarisation of single texts, or the response to natural language queries regarding a text base. obviously, there cannot be a knowledge free discovery approach. rather, text mining relies on sparse, transparent, automatically adaptable and extendable resources. its procedural knowledge applied during the reconstruction stage primarily amis at supplying the subsequent mining procedures with cotextual and contextual knowledge derived from a natural language corpus of texts without the need to refer to introspectively predefined knowledge. text mining should not be confused with data mining. data mining is a technology that aims to extract relevant, previously unknown information from relational or object oriented databases.13 in contrast to this, text mining aims at exploring relevant, previously unknown knowledge from natural language texts, which are characterised by their implicit, indeterminate organisation which makes reconstruction an indispensable part of text mining. text mining can be seen as covering the following tasks:l4 • text knowledge engineering aims at the maintenance and acquisition of domain-specific knowledge (organised as conceptual taxonomies) from natural language texts. hahn/ schnattinger (1998) explicitly separate text knowledge engineering from information extraction: in contrast to the latter, the former does not aim at simply instantiating predefined templates with factual knowledge, but at automatically learning and enhancing conceptual, taxonomically organised structures based on (partial) processing of natural language texts.15 • text categorisation aims at classifying texts by mapping them to a set of predefined categories reflecting aspects of text content, genre, authorship, etc.16 hearst (1999) argues that text categorisation is no text (data) mining, since it simply assigns texts to predefined categories without exploring "new information". the author shares this evaluation. but if the requirement is abandoned that these categories are predefined and if they are explored on their own, a text mining task can be seen to arise. • text summarisation has the aim of reducing textually coded information to its "essential points" based on intermediate, content based text representations.17 the resulting summarisations serve as a starting point for generating summary texts. in the following a fourth type of text mining task is described which is called text linkage. this is done by first specifying semiotic aspects of text mining, namely by characterising the complexity of its special object: natural language texts. 12 see hearst (1999). the difference of structural and referential relations points to alternative conceptions of sign meaning. 13 see fayyard et al. (1996). 14 the subsequent enumeration is due to the variety of text mining approaches and their aims discussed in literature. a comprehensive overview of these different approaches is lacking as yet. 15 see cowie/lehnert (1996), grishman (1997), and wilks/catizone (1999) for overviews of information extraction. 16 see aas/eikvil (1999) for an overview of methods for text categorisation. 17 see endres-niggemeyer (1998). 132 ajis vol. 8 no. 2 may 2001 semiotic aspects of text mining natural language texts differ fundamentally from the categorically, clear-cut defined data structures of data mining. data mining procedures generally operate on relational (possibly normalised) data structures (tables) in which every attribute (column) has a unique meaning explicitly determined by its interand intrarelational dependencies. arguing that texts are (in comparison to these units) unstructured, as numerous text mining approaches suggest, is misleading. evidently texts do not represent sets of words or sentences, but have semantic structure beyond the sentence level. the organisation of texts their texture is based, among other things, on cohesion and coherence relations of text-forming resources which are used to connect text components in order to express semantic as well as pragmatic continuity.li whereas cohesion refers to the characteristics of a text being a linguistic unit, coherence comprises situational, cognitive and social aspects of text processing. according to the system theoretical approach of strohner/rickheit (1990), cohesion and coherence are not strictly separated so that the unifying term coherence can be used to cover both aspects of textuality. as a rule, coherence relations, which indicate constraints for the interpretation of texts and their components (with respect to a given context of situation and culture), are not surface structurally explicit.19 this aspect of uncertainty and context sensitivity is accompanied by the "structural plasticity" of coherence relations:20 to varying degrees, coherence relations exist between units of different levels of linguistic resolution, not necessarily restricted to text components of a specific type (e.g. sentence or paragraph level), size or text distance.21 this sort of structural plasticity, which formal data structures lack, serves for the constitution of texts as the most efficient "data structure" for the dissemination of information.22 beyond this intratextual dynamics23, a second kind of variability, which relates to the mutual dependence of texts and their social contexts, has to be distinguished. this can be outlined following systemic functional linguistics (sfl, see halliday 1991) which views natural languages as dynamic, probabilistic systems persisting on the background of permanent interaction within their social environments, where texts form the primary unit of interaction. as natural languages are embedded into larger social semiotic systems, texts are produced/received depending on social contexts stratified into the context of situation and cultureconnected via genres and registers with the language system. registers manifest variety according to use depending on situational context. they are differentiated according to the dimensions of field (i.e. course of events), tenor (social roles of the participants), and mode (symbolic organisation of the situation), see halliday 1978). genres correspond to the staging of social processes as part of the context of culture and may be linguistically (though not deterministically) manifested by means of generic, schematic structures (see martin 1992).24 halliday (1978) defines registers as manifestations of the correlation of semiotic context and semantic system. a register is a configuration of meaning potentials (and linguistic resources), that are typically connected with a certain situation type, that is a specific configuration of field, tenor, and mode. as patterns of semantic resources typically associated with one or more situation types in a given speech community, registers and genres serve to connect texts with their social contexts. in other words, the constitutive dependence of social and linguistic system is paralleled by the interdependence of context and text as instances of situation types and text types (as linguistic patterns for encoding realisations of genres and registers), respectively. thus, texts always have two contexts: the (sub-)system of meaning potentials underlying their production/interpretation (mediated through one or more text types) as well as the (abstract) situation type(s), whose (real) instances form the social context for texts functioning as communication units. the semiotic 18 see halliday/hasan (1976). 19 in this sense, it is more appropriate to speak of natural language texts as "implicitly" structured entities than to misleadingly suggest that texts are "unstructured". 20 this specific complexity of coherence is tied to its dependence on the dynamics of text production/reception. 21 see halliday (1994) for examples of these kinds of coherence relations. 22 the term efficiency is used in the sense of frege (1994), who states that natural languages are in contrast to formal, artificial languages adaptable/flexible in order to cope with new, unforeseen situations. as texts are the primary linguistic unit by means of which language is realised, they must realise this property, too. 23 this is not to claim that coherence relations are restricted to texts. on the contrary, they embed texts into contexts. 24 to separate between genre and register is not to negate the possibility that genre can be subsumed under mode as one dimension of context of situation, as halliday (1978) suggests in contrast to martin. this dispute does not affect the model proposed in the following. 133 ajis vol. 8 no. 2 may 2001 complexity of this twofold contextual embedding relates to the fact that texts make not only use of meaning potentials and linguistic realisation patterns, but also confirm, modify or even constitute such regularities. furthermore, because of the interdependence of social and language system text processing also affects what is conceived as social context.25 to summarise, coherence relations do not only connect components of the same text constituting intratextual cotexts (i.e. textual contexts) for their interpretation but also embed texts into social contexts by realising generic structures and registers. in other words, coherence relations do not only allow the same text component to be interpreted differently in different texts, but also the same text to function in different contexts. finally, the co-realisation of the same or similar genres or registers is a source for intertextuality26 so that coherence relations can be "exploited" for exploring intertextual relations. as the scientific discipline for the study of signs, their constitution, and usage, semiotics analyses the complexity of signs in general.27 on the background of the considerations outlined so far, the following aspects of complexity can be distinguished: • toward the dynamics of sign meanings: depending on different (cognitive, situative, or social) contexts, the same sign can have different meanings (context sensitivity). as a result of processes of stabilisation/perturbation of such regularities, signs, their properties and relations are enforced, inhibited, they emerge or disappear (variability with respect to language subsystems). regarding the same text, the value of a quantitative text characteristic may vary in the course of text production/reception (variability with respect to single text systems). as peirce (1983) points out, signs have a tripartite, relational organisation, according to which they enter into recursive processes of sign interpretation, called semiosis. the interpretation relations involved in such processes determine the signs' vague identities, their borders, properties and relations (vagueness). • polyfunctionality and synergy: the same (type or class of) sign(s) can serve for different (text forming) functions. similarly, the same function can be served by different (types or classes of) signs (polyfunctionality). signs cooperate and compete with respect to their (text forming) functions. the function of a (type or class of) sign(s) may enforce or suppress the function of other (types or classes of) signs (synergy).28 • realisation and covariation: text signs realise genres and registers mediating between social and language system. as outlined above, sfl does not conceive this dependence as unidirectional instantiation, but as a sort of co-evolvement referred to as redounding. in other words, changes in context are correlated with changes in texture; there exists a systematic covariation of contextual and textual features (see lemke 1995). as a consequence, context is seen as a semiotic entity in sfl, which does not simply serve for the disambiguation of textual components, but whose constitution depends on its part on sign usage. from a semiotic point of view, processes of sign usage can modify those conditions, underlying their realisation. as a consequence, semiotics focuses on processes of sign (meaning) constitution instead of presupposing linguistic knowledge for the description of signs. according to the specific complexity of (text) signs, a significant difference of methods applied in text and data mining is expected. furthermore, a methodically concretised computational semiotics appears to serve as a convenient framework for the specification (and, where appropriate, solution) of the task of text (sign) mining. computational semiotics (cs) has been proposed as a scientific discipline in order to account for the intersection of semiosis and computation.2 a computational semiotic model of sign processing is a procedural model reflecting the aspects of complexity outlined above. it comprises a representational format which • separates representations of language system, social contexts, and texts, • allows to model dynamics of context and language change so that its representations are modified, if the underlying data basis is changed, and • enables to model the vagueness of semiotic phenomena by distinguishing gradual memberships of linguistic and contextual relations. 25seehalliday(1978). 26 see lemke (1995). 27 see peirce (1983). 28 the synergetic aspect of self organization based on cooperating/competing linguistic processes which aim at adapting the properties of functional equivalents in order to optimally serve for cooperating/competing language needs is analysed in synergetic linguistics. see kohler (1990). 29 see clarke/mehler (2000). 134 ajis vol. 8 no. 2 may 2001 models in cs have to be formalised in algorithmic terms (procedures), which allow to delegate them to a computer. results produced by these implementations are bound to the cotextual and contextual information underlying their processing. one does not claim to produce "representative" models regarding the totality of the (sub-)languages under consideration since any model of semiotic phenomena may change immediately if its context is changed. cs models are falsifiable regarding their theoretical background, procedural claims, and the structures they produce. with respect to the task of text mining, computational semiotics seems to be an adequate framework because it addresses the exploration of sign usage regularities in a computational model.30 this insight can now serve as a semiotic reconstruction of text linkage as a task in text mining. systemic functional text linkage text linkage refers to the exploration of implicit, content based, context sensitive relations of texts and their annotation as typed links in corpora organised as hypertexts. the aim of this section is to show that text linkage can be seen as a further task in text mining which naturally gives rise to a semiotic understanding as proposed in the last section. evidently, the task of automatically exploring intertextual relations and their representation as links in hypertext has not been solved yet. this lack correlates with the flexibility of human information processing: depending on a user's varying information needs as well as cognitive, situative, and social contexts the same text can be interpreted or at least evaluated differently with respect to its relevance. as a matter of principle, this context sensitivity of information processing implies that a text corpus does not have a predefined, deterministic hypertext structure:31 different users with different information needs and contexts may find different intertextual relations to be relevant to be explicitly represented as links. nevertheless, automatically creating text links is a crucial demand, because the amount of documents available online is rapidly increasing, so that an exploration problem (as described in the introductory section) arises. text linkage is an explorative task in the sense that it aims at answering the question which (type of) intertextual relation holds between which texts in which (type of) context. with respect to the variation of tasks (as a dynamic user perspective on the same text collection) text linkage aims at answering the question which texts are relevant for supporting which task (goal, or information need) subject to which context. 30 computational semiotics could be understood as a theoretical basis of text mining as a technological discipline. 31 this has already been stated elsewhere. see mehler (1999). 135 ajis vol. 8 no. 2 may 2001 schematic patterns xhtext type | }i . 1 social system ( language system figure 2: a simplified model of text linkage as an explorative task hi text mining.32 in order to illustrate this, the data model in figure (2) comprises relevant categories affected by text linkage.33 the basic building blocks are (social) context with children context of culture (c.o.c.) and context of situation (c.o.s.), further: genre and register, text, text type, and task:34 genre and register, which are dependent on c.o.c. and c.o.s., respectively, are related with text type. an instance of text type is a system of linguistic patterns for realising one or more genres and/or registers (see schematic patterns and thematic patterns for modelling a many to many relation between text type, genre, and register).35 the entity text type serves for the representation of thematic and generic heterogeneity of texts and its linguistic manifestation. in diagram (2) each text (see entity text) is seen to belong (to varying degrees see attribute mu representing membership values d) to one or more text types (see relation realizes). in the framework of text 32 the conceptual model in figure (2) uses boxes and lines to represent entities and relationships, respectively. lines are either straight (unique/one) or branching (multiple/many) in order to model the complexity of relationships. furthermore, circles are used to represent optional relationships, whereas vertical bars are used to model mandatory relationships. different styles of (shaded or dashed) boxes are used for highlighting objects. to exemplify this: the relationship between genre and schematic patterns means that a genre has n > 0 schematic patterns, whereas a schematic pattern belongs (is dependent on) exactly one genre (there is no schematic pattern without a corresponding genre). this example shows that the conceptual model in figure (2) simplifies entities and their relationships that need to be specified in a more detailed model. 33 for the sake of simplicity, attributes have been skipped. 34 situation refers to instances of context, i.e. real situations as instances of situation types. in the same situation one or more texts may be processed (as discourse). 35 the entity thematic patterns models ideational, interpersonal, and textual meaning components, associated with field, tenor, and mode as dimensions of situation types. for the purpose of clarity, these and many other details are skipped in diagram (2). 136 aj1s vol. 8 no. 2 may 2001 linguistics36, realizes can be seen as an extensional representation of text sorts, whereas text type serves as an intensional characterisation of texts. in contrast to genres and registers, text types are pan of the language system. this is indicated by a vertical dashed line that separates the social system from the language system. genres and registers serve to bridge both systems as indicated in diagram (2). further, there is an entity task related to user and context (via depends on). from the perspective of text mining, the most important entities are intertextual relation and relates, which serve to model any kind of homogeneous or heterogeneous intertextual relation classified by instances of type. role serves to determine the arguments of instances of intertextual relation (in the case of similarity relations, role allows for example to model instances of asymmetric similarity). with respect to the entity type, at least two types of intertextual relations can be distinguished: 7 1. similarity: two texts may be intertextually related, if they are similar with respect to field, tenor, mode or generic structure (staging) they are realising. suppose, for example, two registers relating to similar fields (e.g. "natural disaster" and "environmental disaster"). because of the similarity of their fields, both texts are expected to realise similar lexicogrammatical structures (e.g. similar lexical choices like "damage", "destruction", "number of the dead", etc.). this situation is illustrated in figure (3) by means of a commuting diagram, where 0 denotes similarity, v stands for the realisation of register /?, by means of text xit and «-> denotes the intertextual relation of*,, *,-. 2. dependence: on the background of the same (macro) genre or register (realised by a system of intertextually linked texts) two texts may be linked because of their generic or thematic dependence. suppose for example an organisation dealing with the management of working flows (social context) accompanied by technical documentation (generic structure). suppose now a text describing a specific working flow in general and a second text dealing with any stage in this process. obviously, processing the latter document depends on processing the former. this kind of intertexhiality goes beyond similarity, since it is based on generic integrity of both texts: they are integrated into the same social process. the same holds for text linkage based on macro registers, where several texts are linked based on their contribution to the same field (tenor and mode) for realising the same instance of an homogeneous register.38 the differences between both types of intertextual relations can be exemplified as follows: • suppose two scientific texts in the area of cognitive linguistics which describe different experiments (comprising the following stages: hypotheses, tests, results, prospect, etc.) with unrelated contents. in this case, an intertextual relation holds between both texts based on their generic similarity (see figure 3.b). suppose now two texts of the same area are given: whereas the first is supposed to describe an experiment, the second is a theoretical statement concerning the topics dealt with in the first text. in this case, a register based similarity is observed (because of related, though not identical fields) and hence a topic based intertextual relation is stated (see figure 3.a). although both texts are possibly realised by totally different linguistic means (wording, etc.), they are seen to be semantically similar due to the realisation of related registers. clearly, both kinds of similarity can be combined. • suppose now, a macro genre of series of experiments and two texts dealing with consecutive experiments of the same series is given. in this case, we observe a (generic) dependence relation between both texts. as a text mining task, text linkage aims at answering the question which texts are connected by which type of relation, possibly dependent on which task and which context. the results of this exploration are represented by instances of intertextual relation (and its companions), which is optionally related with task and context. moreover, text linkage aims at exploring the relation of genre, register and texts. it asks which texts realise which registers/genres and whether there is a new register/genre to be introduced or already established registers/genres have to be merged. because of the dependence of text links on higher level genre and register relations and since these entities are not (strictly) presupposed -, text linkage can be seen as a special kind of "text categorisation" in which the system of "categories" is explored and dynamically maintained during processing of text corpora. furthermore, genres and registers are not just unstructured categories, but have structure for the classification of textual macrostructures. in this sense, text linkage radically departs from text categorisation. 36 see heinemann (2000). 37 this enumeration clearly does not claim to be complete, neither with respect to the range, nor to the details of possible text relations. 38 if there is knowledge about the relation depends on between task and context, the dependence of different tasks see the introductory section can be mediately explored by means of the exploration of relations of registers (or genres). 137 ajis vol. 8 no. 2 may 2001 («) (b) figure 3: intertextual dependence of two texts based on realising dependent registers (a) or genres (b). as outlined so far, text linkage can be seen to serve as the explorative basis for automatically creating hypertexts from text corpora of heterogeneous genres and registers. furthermore, being based on sfl, it allows the development of a linguistic typology of links. bucher (1999) uses a generalised schema, in which a link l connects a node a with a node b dependent on c, thereby pointing to different tasks of link generation as outlined in table (1). <eaik'-'vt < i r > identification reference sequel typology »•&*" '•"•. • •• -v}*-' • ' ••• • '•• •• •' "' **'•characteristic question '•• :, , " * • • which elements serve as a link source? which aspect of the source document is linked? which aspect of the target document is linked? which type of link is given? coiistituent l a b c table 1: dimensions of links in hypertexts according to bucher (1999). of special interest is constituent c, which refers to the question of what type of link l is supposed to instantiate. based on the understanding that links in hypertexts are realisations of intratextual and intertextual relations, the task of typology can be reformulated in terms of the (coherence providing) linguistic resources, the type of context involved, and the integrity or stability of links.39 in other words, links in hypertext are seen to be stratified with respect to the linguistic organisation (cohesion) as well as situational (register based), and generic coherence of the texts to be linked. in this sense, links in hypertext can be seen as intertextual "coherent ties", whose strength depends on the threefold coherence of the texts linked. as links enter into hypertext paths, a further dimension of link typology has to be taken into account: purely associative links of text pairs need to be distinguished from chains of links, which constitute higher level hypertext structures.40 consequently, the typology for hypertext links distinguishes at least two dimensions: 1. the dimension of contextuality refers to the linguistic, situational and cultural basis of links. 2. the dimension of path sensitivity refers to the scope of links. it relates to the question of whether they are only used to link (associate) text pairs or whether they serve to build coherent paths or chains of interlinked texts. dimensions associativity (of text pairs) path sensitivity (of text chains) cohesion; ... (1) associative link (4) cohesive path register (2) register associativity (5) situationally coherent hypertext path genre (3) schematic associativity (6) generically coherent hypertext path table 2: a two-dimensional systemic functional typology of text linkage. 39 the link typology outlined in the following is not restricted to intertextual links, but comprises intratextual links as well. 40 see kuhlen (1991), who already discusses higher level link structures in hypertext for the representation of thematic and/or rhetoric progressions. 138 ajis vol. 8 no. 2 may 2001 on this background, a provisional link typology can be outlined as given in table (2). at the lower end of this matrix purely associative links are distinguished the typical case of links produced in the area of automatic hypertext construction which neither take path, nor the context of register or genre into account. the construction of (lexically) cohesive as well as path sensitive links is described in mehler (2000). the main idea of this approach is: instead of manifesting context-free associations, so called cohesion trees are used in order to model context priming effects based on the lexical organisation of consecutively linked texts. as a consequence, cohesion trees allow the perspective evaluation of numerically represented text similarities. both kinds of links (i.e. link type (1) and (4) in table 2) are based on automatic corpus processing neglecting higher level genre and register dependencies. more coherent links are produced if these context layers are taken into account. the combination of genre or register with the concept of path sensitivity refers to links as constituents of hypertext paths connecting texts realising the same genre or register (see link types (5) and (6) in table 2). in the course of automatically generating hypertexts from newspaper articles, an example of generic coherent hypertext paths would be a sequel of dialogically ordered thesis and replies of a small set of participants discussing the same topic. an example would be the philosophical debate about gene technology in german feature pages initiated by the philosopher sloterdijk). obviously, associative, path insensitive links are of lowest stability as they vary according to the underlying text corpus: to enlarge this corpus may rapidly modify the strength of text links as the representations of meaning regularities of text components change due to the processing of new texts. in contrast to this, the combination of register and generic coherence extends a link's validity/stability. thus, if hypertext construction aims at a higher level of readability or comprehensibility, the combination of cohesion, register, and genre stable links is needed. but since registers and genres are "shaped" by means of texts, the enlargement of the corpus base may lead to a modification of such links as well in the longer run. these considerations induce a four-layer architecture for the construction of so called systemic functional hypertexts: first, the reconstruction of genre structures occurs. on this level, the constituency, staging as well as association and dissociation of genres is modelled. second, on the level of registers, networks of registers as well as accessibility constraints of situation types are described. third, the language layer models classes of texture forming resources, their paradigmatic organisation, and linguistic realisation patterns. fourth, the text layer describes context sensitive intraand intertextual relations represented as hypertext links. hypertexts of this kind are stratified hypertexts: the first two layers consist of interlinked nodes, which dominate sets of texts as realisations of the corresponding genre or register representation. as explained above, this dominance relation does not induce an equivalence relation over the corpus of texts. requirements analysis as a theoretical background, computational semiotics points to specific requirements for solving the problem of text linkage as a task in text mining. in this context, the following requirements for any implementation of a text linkage procedure can be distinguished: • text meaning and coherence: as a starting point for text linkage, representations of aspects of text meaning are needed, which reflect coherence relations as the fundamental organisation unit of natural language texts. these representations have to make explicit to which degree they reflect certain types of textual coherence and certain aspects of text meaning. • implicit text relations: the mining procedure should be able to link texts, even if they share only few or no components at all, but deal with related or similar topics or realise related genres or registers, respectively. the significance of this requirement can be exemplified by looking at texts in different languages, which are totally different in terms of surface structure, but may be similar regarding their content. • reconstructive, explorative corpus analysis: the decision to link texts is based on knowledge reconstructed or even explored from the corpora to be converted into hypertext. any text linkage has to take the specifics of signs and their usage in the corpora into consideration. this does not mean that, initially, text linkage is knowledge free since it possesses at least procedural knowledge of how to acquire relevant information. furthermore, any factual knowledge that the system has already disposed of, should be the result of reconstructive, explorative corpus analysis of its own.41 • representational format and context model: the specific complexity of texts requires a representational format, which allows to model the vagueness of text relations with any resolution, disposes of context correlates and is adaptable to the analysis of texts of previously unknown registers and genres. in addition, the format has to be sensitive to new cotextual and contextual evidence, which may justify modifications of the representations already produced. 41 this requirement is certainly a difficult, if not an unrealistic one at present. nevertheless, there exist computational models, which aim at reconstructing morpho-syntactic knowledge from large text corpora. see for example medina urrea (2000). 139 ajis vol. 8 no. 2 may 2001 the fulfilment of these requirements enables transparent knowledge acquisition, since factual (extensional) knowledge, whose acquisition is tied to human expertise, is replaced by intensional, procedural specifications. prospects the considerations outlined so far are rather programmatic: they have tried to clarify the task of text mining from the perspective of computational semiotics with special focus on the task of text linkage as an explorative basis for the automatic construction of hypertexts. it was the aim of the paper to show that text mining cannot rely on classical methods of information retrieval or knowledge engineering since its object, i.e. natural language texts, have a complexity which goes beyond the complexity of classical data structures. clearly, this aim could only be tentatively, motivationally realised leaving the proof of all necessary evidence and examples to forthcoming papers. thus, future work will aim at giving procedural definitions of genres, registers, text types, and text forming resources. special emphasis will be on the corpus analytic reconstruction of these entities. moreover, a formal model is needed for the complete formalisation of systemic functional hypertexts, thereby making formally explicit such (more or less vague) terms as context sensitivity, variability, vagueness, polyfunctionality, realisation and covariation. finally, the model has to prove its computability and applicability. references aas, k.; eikvil, l. (1999): text categorisation: a survey. norwegian computer center, report no. 941,1-37. bucher, h.-j. (1999): die zeitung als hypertext. verstehensprobleme und gestaltungsprinzipien fur onlinezeitungen. in: lobin, h. [ed.]: text im digitalen medium: linguistische aspekte von textdesign, texttechnologie und hypertext-engineering. opladen: westdeutscher verlag, 9-32. clarke, r.; mehler, a. (1999): theorising print media in contexts: a systemic semiotic contribution to computational semiotics. in: proceedings of the 7th international congress of the iass-ais: international association for semiotic studies sign processes in complex systems. university of technology, dresden, germany, october 6-11,1999. cowie, j.; lehnert, w. (1996): information extraction. in: communications of the acm, 39(1), 80-91. endres-niggemeyer, b. (1998): summarizing information. berlin [a.o.]: springer. fayyard, u.; piatetsky-shapiro, g; smyth, p. (1996): the kdd process for extracting useful knowledge from data. in: communications of the acm, 39(11), 27-34. frege, g (1994): funktion, begriff, bedeutung. fiinf logische studien. 7. edition; gottingen: vandenhoeck & ruprecht grishman, r. (1997): information extraction: techniques and challenges. in: pazienza, m. t. [ed.]: information extraction: a multidisciplinary approach to an emerging information technology; international summer school, scie-97, frascati, italy, july 14-18, 1997. berlin [a.o.]: springer, 10-27. hahn, u.; schnattinger, k. (19-°): towards text knowledge engineering. in: proceedings of the 15th national conference on artificial ^intelligence & 10th conference on innovative applications of artificial intelligence, madison, wisconsin, july 26-30, 1998. cambridge: aaai press, mit press, 524-531. halliday, m. a. k. (1978): language as social semiotic. london: edward arnold. halliday, m. a. k. (1991): towards probabilistic interpretations. in: ventola, e. [ed.]: functional and systemic linguistics. berlin [a.o.]: de gruyter, 39-61. halliday, m. a. k. (1994): introduction to functional grammar. london: edward arnold. halliday, m. a. k.; hasan, r. (1976): cohesion in english. london: longman. hearst, m. a. (1999): untangling text data mining. in: proceedings of acl '99: the 37th annual meeting of the association for computational linguistics, university of maryland. heinemann, w. (2000): textsorte textmuster texttyp. in: brinker, k.; antos, g; heinemann, w.; sager, s. f. [eds.]: textund gesprachslinguistik. linguistics of text and conversation. berlin [a.o.]: de gruyter, 507-523. klir, g j.; folger, t. a. (1988): fuzzy sets, uncertainty, and information. englewood: prentice hall. kohler, r. (1987): systems theoretical linguistics. in: theoretical linguistics 14(2/3), 241-257. kuhlen, r. (1991): hypertext: ein nichtlineares medium zwischen buch und wissensbank berlin [u.a.]: springer. lemke, j. l. (1995): intertextuality and text semantics. in: fries, p. h.; gregory, m. [eds.]: discourse in society: systemic functional perspectives. meaning and choice in language: studies for michael halliday. norwood: ablex publishing, 85-114. lobin, h. (2000): informationsmodellierung in xml und sgml. berlin [a.o.]: springer. martin, j. r. (1992): english text. system and structure. philadelphia [u.a.]: john benjamins. medina urrea, a. (2000): automatic discovery of affixes by means of a corpus: a catalog of spanish affixes. 140 aj1s vol. 8 no. 2 may 2001 in: journal of quantitative linguistics 7(2), 97-114. mehler, a. (1998): toward computational aspects of text semiotics. in: proceedings of the 1998 ieee isic/cira/isas joint conference on the science and technology of intelligent systems. piscataway: leee/omnipress, 807-813. mehler, a. (1999): aspects of text semantics in hypertext. in: tochterman, k.; westbomke, j, wiil, u. k., leggett, j. j.: hypertext '99. returning to our diverse roots. proceedings of the 10th acm conference on hypertext and hypermedia. darmstadt: acm, 25-26. mehler, a. (2000): text mining with the help of cohesion trees. appears in: proceedings volume of the annual conference of gfkl at the university of passau, germany, 2000. peirce, c. s. (1983): phanomen und logik der zeichen. frankfurt am main: suhrkamp. rieger, b. (1989): unscharfe semantik: die empirische analyse, quantitative beschreibung, formale representation und prozedurale modellierung vager wortbedeutungen in texten. frankfurt am main: peter lang. sparck jones, k. (1999): what is the role of nlp in text retrieval? in: strzalkowski, t. [hrsg.]: natural language information retrieval. dordrecht [u.a.]: kluwer, 1-24. strohner, h.; rickheit, g (1990): {cognitive, kommunikative und sprachliche zusammenhange: eine systemtheoretische konzeption linguistischer koharenz. in: linguistische berichte 725, 3-24. wilks, y.; catizone, r. (1999): can we make information extraction more adaptive? in: pazienza, m. t. [ed.]: information extraction. towards scalable, adaptable systems. berlin [a.o.j: springer, 1-16. zadeh, l. a. (1997): toward a theory of fuzzy information granulation and its centrality in human reasoning and fuzzy logic. in: fuzzy sets and systems 90, 111-127. 141 ajis vol. 7 no. 1 sept 1999 editorial this is the first issue of volume 7 of the australian journal of information systems. this issue is divided into two sections. the first section contains papers submitted through the normal channels of the journal, the second contains two papers judged the best papers from the recent efis 99 conference. our first paper discourse strategies model: an initial phase for discovery of the fact-based statements from descriptive text bruce calway & ross smith, swinburne university of technology examines the resource of fact modeling and its use to the systems analyst. the paper provides a discourse strategy model which processes descriptive text and provides an example of its use. steve howard, swinburne chi laboratory, swinburne university, provides our second paper entitled using qfd to tame wicked information system's requirements. the paper argues that information system's requirements can be in tension with both themselves and their context and presents a response to that problem using quality function deployment. a case study is used to illustrate the application of qfd. our third paper strengthen and support the maintenance of object oriented software ming-chi lee, national ping tung institute of commerce, taiwan, timothy shih & teh-sheng huang, department of computer science and information engineering, tamkang university examines the concept of inheritance hierarchies in object-oriented languages. the paper shows that the difficulties often lie in the lack of effective detection tools for inheritance and provides mechanisms for detection and resolution of redundant inheritances. out fourth paper advances in research direction in data-warehousing technology mukesh mohania, school of computer and information science, university of south australia, sunil samtani, department of computer science telecommunications, university of missouri-kansas city, john roddick, school of computer and information science, university of south australia, yahiko kambayashi, department of social informatics, kyoto university presents an overview of data warehousing, multi-dimensional databases, olap and data mining technologies. the paper also discusses recent developments in view modeling, indexing schemes and parallel query processes. jeanette van akkeren, department of information systems, university of the sunshine coast & angele cavaye, department of information systems, university of southern queensland present our next paper confusion with diffusion? unravelling is diffusion and innovation literature with a focus on sme's. this paper examines various models of adoption and diffusion of it and their relevance to sme's. an editorial for the last two papers is provided by s. conrad, magdeburg university, w. hasselbring, tilburg university & g. saake, magdeburg university. uwe hohenstein, corporate technology, siemens presents our next paper a generative approach for building database federations. this paper presents a specification-based approach to building database federations. the approach centres on a number of generators which provide flexibility with respect to schema modification. our last paper constructing multidatabase collections using an extended odmg object model adrian skehill & mark roantree, interoperable systems group, dublin city university examines the odmg and cnaonical models and extends these into a more powerful collection mechanism allowing the construction of federated collections from distributed data sources. a book review of 'information rules a strategic guide to the network economy' by carl shapiro & hal varian is presented by tim coultman agsm. once again i would like to pass on my thanks to the authors and consulting editors for the high quality of material in this issue. i would like to thank lily soh, ted gould and deborah bunker for their continued assistance with the journal and david dodds for the maintenance of the home page. rob macgregor editor http://www.uow.edu.au/ajis/ajis.html ajis vol. 9, no. 2 may 2002 information warfare: using the viable system model as a framework to attack organisations bill hutchinson school of management information systems edith cowan university western australia, 6018 tel: +61 (0)8 9273 8283, fax: +61 (0)8 9273 8332 email: w.hutchinson@cowan.edu.au mat warren school of computing and mathematics deakin university geelong victoria australia 3217 tel: +61 (0)3 5227 2536, fax: +61 (0)3 5227 2028 email: m.warren@deakin.edu.au abstract information is the glue in any organization. it is needed for policy, decision-making, control, and co-ordination. if an organisation's information systems are disrupted or destroyed, then damage to the whole inevitably follows. this paper uses a proven systemic, analytic framework the viable system model (vsm) in a functionalist mode, to analyse the vulnerabilities of an organisation's information resources to this form of aggression. it examines the tactics available, and where they can be used to effectively attack an organisation. introduction the concept of information warfare (iw), which until recently was restricted to military circles, has entered the civilian world. in this information age, with its dependence information and its associated technology, the phenomenon of iw has taken on added significance. in 1993, the increasing importance of iw led the national defence university in the usa to set up the school of information warfare and strategy (schwartau, 1996, p.8). according to the us department of defence (1996) over 120 countries are developing iw techniques. in the contemporary world, the commercial, social, governmental, and military infrastructures are tightly intertwined. commercial and government organisations are vulnerable to this type of attack, and some already have been affected by it. for example, in november 1998 (barton, 1998), it was reported that a hacker wiped out more than 4500 new zealand web sites. cobb (1998, p.26) cites an australian bank's compliance and fraud officer, who admits that the cost of information attack to be in excess of $500 000. denning (1999) cites many more examples of larger impacts on organisations, and the potentially disastrous effects on the social fabric and economy they can pose. it is an issue of concern for both large and small organisational systems. this paper outlines possible modes of information attack using the viable system model as a framework. it is an attempt to use a systemic tool to expose vulnerability of the information infrastructure in all organisations. its emphasis is on the attack process, not on counter-attack, counter-measures, or detection. these latter two elements are the province of the security function. of course, knowledge of attack strategies inevitably increases the effectiveness of counter-measures. information warfare (iw) iw is the use of information, or information systems to disrupt or destroy an organisation perceived to be an 'enemy'1. the over-riding objective being to coerce the target to act in a way that is favourable to the attacker2. the use of information to gain advantage over an adversary is not new. however, the magnitude of the impact of these tactics has increased in this information age. society as a whole is feeling the impact and accompanying massive changes as information increasingly becomes a source of power, influence, and economic gain. this paper will examine what schwartau (1996) calls 'class 2: corporate information warfare'. it includes areas such as industrial espionage, knowledge theft, as well as the more conventional competitive practices of organisations. firstly, possible tactics for information warfare will be discussed. secondly, a brief description of the viable system model (vsm) will be given to illustrate the functions of a viable organisation espoused by this model. finally, the tactics and the vsm will be brought together to illustrate possible attack strategies using the vsm as a framework. 67 ajis vol.9, no. 2 may 2002 tactics there are a number of ways information, or information systems can be used to gain advantage over (or disadvantage) another organisation. some aggressive tactics (developed by the author) are listed below: • information can be manipulated, or 'created' (disinformation) to provide the target or its environment (for example, clients) a perception that develops behaviours detrimental to the target, or beneficial to the attacker. at one level, this can be viewed as advertising, and at another, deliberate deception. • information can be intercepted, thus giving the interceptor an advantageous insight into the target's strengths, weaknesses, and intentions. this information can be gained legitimately, or illegitimately. • information flows in the target organisation can be disrupted, or stopped, thereby interfering with the normal processes of the target producing an advantage for the attacker. • a target organisation can be 'flooded' with information, thereby slowing or stopping effective processing or analysis of the incoming information. • information can be made unavailable to a target organisation by destroying the storage medium, or cutting off the information source. • disrupting their availability, or making them produce incorrect/dubious output can lower the credibility of information systems. • confidential or sensitive information can be exposed to the public, clients, government agencies, and so on, thereby embarrassing or in other ways harming the organisation. • knecht (1996, p. 168) describes means of attack with a more military emphasis. these are listed below: • physical attacks on the components of the information infrastructure. • physical attacks on the components containing the information infrastructure. • physical attacks on or the subversion of the people (witting or unwitting) who operate elements of the information infrastructure. • physical destruction of information (erasure or overwrite) without harming the infrastructure components. • logic (malicious code) attacks on the components of the information infrastructure. • logic attacks on computer-controlled components supporting the information infrastructure. • attacks on the information provided via the information infrastructure to specific functions. • corruption of information using logic or digital attacks without harming the components of the information infrastructure. • combined attacks on the information infrastructure or supporting components to mask other types of attacks, or to obtain the benefits of a combined attack. obviously, many of these tactics are not pertinent to the contemporary business world (at least, not any ethically based corporate strategy) but they do give an idea of the range of possibilities open to an attacker. the form of attack can be varied. it depends on such factors as the medium of information transfer and storage (for example, electronic, telephone, verbal, facsimile, etc.), the legality of the attack, and the technology used (for example, electronic surveillance, computer viruses3, and human senses). rathmell et al (1997) give a number of likely means for penetrative attacks which can be found in table 1. other more physical means, such as directed energy weapons, can be used to destroy hardware (waltz, 1998). 68 ajis vol. 9, no. 2 may 2002 attack tactic compromised trusted user acquisition of user's password trojan horses software bombs viruses worms comments most software attacks and computer crimes are carried out by trusted users. can be achieved by packet sniffers, and password crackers. installed after penetration. mimic actions of system utilities. useful for sabotage, extortion, and blackmail. similar to trojan horses. planted with some mission critical software, and triggered by date/time. many variations. almost all computer systems have been infected at some time. replicate themselves and consume system and network resources. table 1: methods of attacking a computer system/network (summarised from rathmell et al, 1997) motivations for attacks can have an organisational purpose, or be purely malicious. attacks can be by individuals or groups. attackers can be internal to the organisation, or come from individuals or groups external to it. not all the information involved needs to be confidential, much of it can be in the public domain. rathmell et al (1997) posit the three main reasons for terrorist groups to use information warfare as raising funds, propaganda campaigns (mostly through web sites), and to attack the organisation's information infrastructure. denning (1999) lists five classes of resources involved in information warfare. they are: • containers, eg computer and human memories • transporters, eg humans, telecommunication systems • sensors, eg scanners, cameras, microphones, human senses • recorders, eg disk writers, printers, human processes • processors, eg microprocessors, humans, software each of these elements, or groupings of them, can be the foci of attacks. thus, the range of targets can vary from public opinion to a microwave link. each combination of attack factors for example, information and communication medium, type of attacker, tactic, and determination of attacker (opportunistic/long term) make protection against every factor difficult. the points of vulnerability in an organisation will now be examined using the viable system model (vsm) and its associated functions as a framework for attack. the viable system model (vsm) the viable system model (vsm) developed by stafford beer (1985), using the principles of cybernetics has been successfully used to diagnose existing organisational structures and design new ones. as espejo (1993, p. 522) states "vsm provides a language in which to appreciate the complexity of organisational tasks and the communication mechanisms underlying people's interactions". it is the generic nature of the vsm that allows it to be used in a myriad of circumstances for organisational analysis. it is a metaphor for a robust organisation (system), and can be used to analyse organisational health, and other systemic problems. however, its insights into the requirements of maintaining a viable organisation can also provide clues to the mechanisms for destroying the same. this paper will use this model as a framework to analyse potential vulnerabilities to an organisation's information systems. its ability to be used in this manner demonstrates vsm's insights into the requirements of a functioning system. before using the vsm, it is essential to understand the dynamics of its applicability and a diagrammatic representation is shown at figure 1. the vsm consists of five subsystems, which have the following functions: implementation (si): this function consists of semi-autonomous units, which carry out the operational tasks in the system. these are the functions that are basic to the existence/purpose(s) of the system. they interact with their local environment, and each other. each unit has its own local management, which is connected to wider management by vertical information flows. this function is the 'doing' part of an organization. the vsm has a recursive element, and each si has another vsm embedded in it. 69 ajis vol.9, no. 2 may 2002 co-ordination (s2): this function co-ordinates the s1 units to ensure that each s1 unit acts in the best interest of the whole system, rather than its own. this could be represented by something as simple as a timetable, or as subtle as morale among the workforce. internal control (s3): this function interprets policy information from 'higher' functions (s4), and 'lower' functions. it is the function, which controls the operational levels. its function is not to create policy, but to implement it. information arriving from the si function must periodically be audited for its quality and correctness. this is the s3* audit function. intelligence and development (s4): this function acts as a filter of information from the s3 function and the overall outside environment. its purpose is to ensure that the policy making function (s5) is adequately briefed, and decisions are transmitted to s3. strategy and policy (s5): this function is responsible for the direction of the whole system. it must balance internal and external factors. beer (ibid.) contends that all the above functions must be adequately performed in an organisation to keep it viable. it should again be noted that the model has a recursive element. each s1 unit has embedded in it, another vsm. hence, the local environment now becomes the total environment for that system, and so on. more detailed descriptions of the vsm can be found in beer (1984, 1985), jackson (1991), and flood and jackson (1991). the vsm concentrates on functions but provides an effective tool for specifying information flows throughout the organisation. it explicitly states what needs to 'go on' in a healthy system, and hence the information channels needed to ensure it. it illustrates the need for both internal information for stability, and external information for survival in its environment. the comprehensive nature of the model enables it to be used to design information system strategies, and examine any missing components. obviously, these qualities can be used to build and expose weaknesses in existing systems. however, these same characteristics can be used to attack organisations, and exploit weaknesses. attacking the organization a planned attack can be made at the operational, tactical, or strategic level. roughly, these three levels can be mapped against the si, s2/s3, and s4/s5 functions of the vsm respectively. also, they can be equated with the efficiency, effectiveness, and vision/purpose of the organisation. another perspective can be based on time. in this case, the objectives may be short, medium, or long term. in a world of a ubiquitous mass media, and instant, universal communications "the distinctions between the tactical, operational, and strategic tend to blur into insignificance" (dearth and williamson, 1996, p. 25). for example, a campaign of misinformation claiming food poisoning caused by a particular product picked up by the mass media can change a local affair to a strategic one as the producer's total environment becomes involved. each specific function will now be examined. however, the recursive nature of the vsm, with each s1 having vsm embedded in it, should be noted. 70 ajis vol. 9, no. 2 may 2002 (note: env = environment, mgnt = management) figure 1: the viable system model (after beer, 1985) 71 ajis vol. 9, no. 2 may 2002 attacking the fundamental operating units (si) the operating units can be disrupted by: • denying them their operating (local) environments, • disconnecting them from other s1 units, • separating them from the management functions. information can be used to misinform both the local environment and the s1 units. this can cause a decrease in the beneficial relationship between the two. disrupting the relationships between si units can result in the fragmentation of the whole operations function. this can be achieved by giving different information to each s1 either internally, or from the environment. this will cause a misreading by si management units of both their own local environments, and the performance of other s1 units. attacks on the information flows from s2 and s3 can lower the effectiveness of s1 management. attacks on the si units are intended to decrease the efficiency of the whole organisation by disrupting its operational (production) functions. it can be viewed as analogous to the classic, military aim of 'breaking the line'. here the attacker's objective is to disperse the enemy by such things a concentrated attack on a specific part of the line, or striking at its flanks. in information warfare terms, this means using information tactics to cause chaos to a specified si unit (a direct attack), or disrupting the information between sis and/or higher s2/s3 functions (a flanking move). attacking the coordinating function (s2) the purpose of attacking the co-ordinating function (s2) is to destroy the cohesion of the operating units. such simple things as a timetable, production schedule, or more abstract entities such as staff morale can bring about cohesion. therefore, the aim of the attacker is to manipulate, change, or deny information to make this coordinating function ineffective. thus, the activities of s1 units would be uncoordinated at the very least, and working against one another to the point of complete disruption in a highly successful attack. an example might be to spread misinformation designed to create negative perceptions amongst staff thereby causing a loss of morale. the coordinating function can be disrupted by direct attacks on communication or information systems causing the malfunction, or more indirectly by subtlety changing pertinent information causing the integrated functions of the si units to fail. classic psychological operations can be used against the victim's staff (as it can be in the local si environment, and more general organisational environment to change perceptions of customers, suppliers, authorities, and the public). attacking the controlling function (s3) the main point of attacking the control function is to use information to disrupt the interpretation of policy. thus, the instructions passed down to the si units would not be commensurate with the intentions of the policies created by s5. altering information coming into s3 from si and s3*, and leaving s3 for s4 will also disrupt or misinform s4. hence, the formulation of policy will be corrupted by 'bad' information, or be less effective because of lack of information. attacking the s3 function should disrupt or destroy effective co-operation between the planning/policy aspects of the organisation and its operating functions. the main intention is therefore is make the operational units either non-functional, or to function in a way that is at odds with the policy making function, and to the benefit of the attacker. therefore the destruction, or probably more effectively, the corruption of information going in and out of the s3 function is the attacker's aim. destroying the 'brains' and 'senses' of an organisation (s4/s5) the purpose of the s4 level is to be the interface between the external and internal environments by processing and communicating information to s5 and s3. s5 produces policy from the information sent by s4. these two functions can therefore be seen as the 'brains' and 'senses' of the organisation. therefore, the purpose of an information attack at this level is to create false perceptions of both the internal organisational position, and the external environment. thus, the aim is to create policies and strategies that are inappropriate for the organisation. the ultimate aim is the organisation's demise. as well as falsifying, or depriving information, emergency (algedonic) signals can be withheld, delayed, or made to occur erroneously (or very frequently) so they will be ignored in the future. s4 could also be 'flooded' with, or fed erroneous information so causing confusion and mistrust of its validity. 72 ajis vol. 9, no. 2 may 2002 according to richelson (1993, p.3), the main tasks of an intelligence systems (s4) is to collect, process, analyse and produce, and disseminate information. waltz (1998, p.51) elaborates on this and classifies the levels of 'information' as: • data (measurements and observations) • information (data placed into context, indexed, and organised) • knowledge (information understood and explained) • wisdom (knowledge effectively applied) therefore, the task of the attacker is to disrupt, deny, or manipulate data to provide the target with no information, or misinformation. if the target obtains data then the attacker's function is to provide misinformation to manipulate the context. for example, to imply that the data is incorrect (or, in fact, to imply incorrect information is correct). at the knowledge level, the attacker could provide further misinformation to interfere with the deductive and inductive processes needed understand information. a classic example of this can be illustrated by the programme of deception executed by the british before the d-day landings in normandy in 1944 (cave brown, 1975). at later stages, the attacker needs to deny the effective application of the target's knowledge (or, encourage detrimental applications). conclusion this brief outline using the vsm4 has illustrated the vulnerability of organisations to attack using information, or its associated systems of storage and transmission. whilst the security function within an organisation has the responsibility to combat such attacks, an effective defence must entail continual monitoring and intelligence to ensure a dynamic response. information systems tend to be viewed as synonymous with computer systems, yet information is received, stored, analysed, and acted upon in many forms within an organisation. therefore, information attacks can occur on many elements in organisational systems. this paper has given a framework for an attacker, and thus has also provided a framework for a defence system. organisations can be vulnerable at many levels, and organised attacks can have different objectives. an overall attack strategy should attempt to disrupt, manipulate, deny, and destroy information resources at all functional levels of an organisation. obviously, a robust system would not be a passive victim in this process, but knowledge of possible aggressive tactics will give the victim added abilities to counteract them. thus the vsm no only gives a framework for attack and but one for defence as well. references anonymous. (1998). maximum security second edition, sams publishing, indianapolis. barton, c. (1998). computer hacker destroys 4500 web sites. in: new zealand herald, 19 nov, 1998. beer, s. (1984). the viable system model: its provenance, development, methodology and pathology. in, espejo r, harnden r.(eds.), the viable system model john wiley & sons, chichester. pp.211-270. beer, s. (1985). diagnosing the system for organisations. wiley, chichester. cave brown, a. (1975). bodyguard of lies. harper and row, new york. cobb, a. (1998). thinking about the unthinkable: australian vulnerabilities to a high-tech risks. research paper 18 1997-98. department of australian parliamentary library, canberra. dearth, d.h., williamson, c.a.. (1996). information age/information war. in: campen, a.d., dearth, d.h., thomas goodden, r (eds). cyberwar: security, strategy, and conflict in the information age, afcea international press, fairfax, pp.13-29. denning, d.e. (1999). information warfare and security. addison-wesley, reading. department of defence. (1996). information security: computer attacks at department of defense pose increasing risks. (testimony, 05/22/96, gao/t-aimd-96-92) espejo, r. (1993). domains of interaction between a social system and its environment, systems practice, vol 6, no 5. flood, r.l., jackson, m.c. (1991) creative problem solving: total systems intervention. wiley, chichester. jackson, m.c. (1991). systems methodology for the management sciences. plenum press, new york. knecht, r.j. (1996) thoughts about information warfare. in campen, a.d., dearth, d.h., goodden, r.t. (eds).: cyberwar: security, strategy, and conflict in the information age, afcea international press, fairfax, pp. 161-174. kuehl, d.t. (1996). strategic information warfare and comprehensive situational awareness. in: campen, a.d., dearth, d.h., goodden, r.t. (eds). cyberwar: security, strategy, and conflict in the information age, afcea international press, fairfax, pp. 185-195. 73 ajis vol.9, no. 2 may 2002 rathmell, a., overill, r., valeri, l., gearson, j. (1997). the iw threat from sub-state groups: an interdisciplinary approach, presented at third international symposium on command and control research and technology, june 17-20, 1997. (available on-line at: http://www.kcl.ac.uk/orgs/icsa). richelson, j.t. (1995). the u.s. intelligence community third edition. westview press, boulder. schwartau, w.(1996). information warfare second edition. thunder's mouth press, new york. waltz, e. (1998). information warfare principles and operations. artech house, norwood. 1. there are numerous other definitions of iw. for example, knecht (1996, p. 165) offers "the preparation for and use of physical or logic-based weapons to disrupt or destroy information or information systems in order to degrade or disrupt a function(s) that depend upon the information or information systems". 2. the conventional information systems (is) function is to exploit information for the organisation's benefit, whilst the security function should protect it. the information warfare function is to deny, disrupt, or destroy a target's is function. waltz (1998), although concentrating mostly on military and national iw, gives a good background to the topic. 3. there are many good texts on computer attack methods such as spoofing, worms, viruses, and spamming. a recent comprehensive publication can be found in the references section (anon, 1998). 4. of course, there are many organisational models that could be used. for instance, warden's concept of rings and systems (cited in kuehl, 1996, pp. 187-189) can be used. here the target is modelled in a series of concentric rings. each represents (from the outer to inner ring):the fighting mechanism (for example is security staff), the population (for example, the general staff), the infrastructure (for example, power, communication systems, and networks), the organic essential (for example, money, and production inputs), and leadership. each attack can be aimed at any of these rings. classic theory would have the inner 'leadership' ring as the target, which would be a 'war winner'. 74 microsoft word ajis112final.doc australasian journal of information systems vol 11 no 2 may 2004 70 community sector organisations: a digital divide in communication and knowledge creation? tom denison centre for community networking monash university australia monash university po box 197 caulfield east vic 3145 email: tom.denison@sims.monash.edu.au abstract in 2002, the centre for community networking commenced a research programme aimed at improving the understanding of the use of communications and information technology (ict) by community and third sector organisations within australia. the immediate goal of this research was the creation of the monash community ict index. this paper represents the first attempt to provide a more in-depth analysis of the data obtained in that survey, setting the results in the context of research undertaken by zook (2001), castells (2001) and adamic (2001), who consider that not only is the internet a natural fit with “winner-take-all” markets but reinforces existing patterns of geographic centralisation in the information society and economy. it will pay particular attention to factors such as the location and size of organisations, and whether there is any support for the notion of a digital divide in the social communications networks and creators of knowledge within the australian community sector. introduction in 2002, the centre for community networking (ccnr) at monash university commenced a research programme aimed at improving our understanding of the use of communications and information technology (ict) by community and third sector organisations within australia. the immediate goal of this research was the creation of the monash community ict index, designed to be a national longitudinal data series providing indicators of use of ict by community sector organisations, describing patterns of use and barriers. that survey was conducted by telephone between august 29 and the end of october 2002 and the initial findings have been made available (ccnr 2003). the impetus for the research came from a perceived lack of general statistical information which adequately described the current australian situation and a suitable theoretical framework in which to interpret it. the creation of the index was seen as having significant potential benefit for the public sector for planning, policy development and national co-ordination as well as providing a basis for further research and analysis. the monash community ict index itself was intended to provide a quantitative snapshot of the state of ict usage within the australian community sector. future surveys should supplement it by providing evidence of trends, while further qualitative studies will allow not only exploration of questions of take-up and barriers to take-up but also the way in which community sector organisations operating in an online environment can make a positive contribution to social capital, defined by putnam as the “features of social organisation, such as trust, norms and networks, that can improve the efficiency of society by facilitating coordinated actions” (putnam et al. 1993, p. 167). as elsewhere, for example in the uk where the office of the e-envoy sees non-profits as important partners in the delivery of government services (2002), it is hoped that the research will inform government policy in the area. in this paper, i explore one aspect of that broader research project, examining some of the research around the nature and development of the internet, to determine whether there might be evidence of any environmental factors which may impinge on the development of web sites by community sector organisations or their ability to use them effectively. i will argue that the interplay of external pressures and the nature of the internet, particularly in regard to factors such as place and organisational size, can have a significant effect on the success of the internet strategies of community sector organisations. i will also argue that the effective use of the internet by most organisations is impeded by the presence of large sites which weaken the ability of smaller sites to promote themselves, so weakening their position with regard to their target audience. the consequences of these factors may well be the loss of local input and/or power and the placing of australasian journal of information systems vol 11 no 2 may 2004 71 unrealistically high demands on sites because of the need to respond to external forces. support for this will be garnered from a closer examination of the monash community ict index than has been made previously, but it must be seen as part of a broader research in progress. these questions will need to be explored in a qualitative sense in a series of future case studies, the design of which will be influenced by these findings. research into the nature of the internet despite the apparent potential of the internet to enhance democratic participation and to eliminate place as a factor in information creation and distribution, there is a significant body of research that suggests that this is not occurring. the recent work of adamic and huberman (2000, 2001), based on an analysis of america online (aol) log files, provides evidence that rather than leveling the playing field for content providers, the internet strongly conforms to a power law distribution, typical of ‘winner takes all’ market models. while agreeing that the internet’s dispersal allows for greater accessibility to the content published by anyone, their results effectively demonstrate that patterns of usage follow a logarithmic scale, and at the two extremes, a handful of sites have millions of visitors while millions of sites attract only a handful of visitors. notably, they found that 1% of all sites accounted for 55% of unique visits and, surprisingly, that when the analysis was restricted to .edu domains, the same distribution was found, with only slightly less drastic consequences – 5% of all sites accounted for 59% of unique users. the implication is that new sites will join the millions of ‘invisible’ sites and that, while some may advance over time, the majority will stay in the lower ranks. this sits well with the idea of early advantage and continuing dominance – high ranking sites are difficult but not impossible to dislodge, but even if the ranking of an individual site changes, the overall pattern remains the same. this suggests that the internet tends toward reinforcing and increasing the gains of leading information creators and providers to the detriment of smaller players. it also has implications for all organisations trying to use the internet, whether commercial or non-commercial in nature. adopting a different approach, zook (2001) undertook a massive study based on the registration addresses for both cone and cc domain names, on the assumption that this was the best method of determining the location of the production and organisation of internet content. his results indicate that, far from being a dispersed medium, there is a striking centralisation of internet publishers worldwide. in particular he notes that while the top 100 cities (only 46 of which are outside the us) contain 6.7% of the world’s population they contain over half of the world’s internet domains. while he believes that it is too early to judge the long term pattern and economic impact, he argues that ‘1) the supply and demand for internet content remains overwhelmingly within oecd countries; and 2) the production and supply of internet content is primarily concentrated in the world’s urban areas’ (zook 2001, p. 1680). christie and hepworth (2001), quoting research undertaken by boldlygo.com in 2000, report a similar trend in the united kingdom, where 80% of all dotcoms are based in south-east england (60% in london alone) while other major centers such as silicon glen in scotland and silicon fen in cambridge accounted for a mere 4% and 5% respectively. these findings support the notion not only that the internet naturally favours the development of large-scale sites but that place, in the sense of the domination of key urban centres, is important. characterising the internet as a communications facilitator, castells (2001) builds on this work by considering the digital divide from two perspectives – access to the internet and control of information creation. he believes that in terms of access the digital divide is of diminishing significance but that these trends indicate that the question of control of information creation is much more problematic. he goes so far as to postulate that those who aren’t a part of the internet world will be marginalised and that, because of the nature of the internet and its global and geographic characteristics, those who don’t use it effectively will also be marginalised. this is not as dramatic as it sounds a focus on the structure can mask an ongoing rearrangement of the players and a cultural interplay that is not always one-way. for example, in discussing the music industry and its domination by five international conglomerates, colista (1998) argues that while the australasian journal of information systems vol 11 no 2 may 2004 72 overall structure of the sector has important consequences for the industry and the services it provides, it says little about the specific instances of cultural flow or changes to content that exist within that structure. ‘independent labels in the music industry may mediate some of the effects of such domination. these companies seek more innovative styles of music, often hoping to find the next big trend. regardless of motivation, the existence of independent labels, which have also sought out international musics, serve as a source of diversity for listeners and as a recording vehicle for unknown acts’ (colista 1998, p.190). the importance of this is that in many ways the structure of the international music industry parallels that of the internet and may provide useful insights into the consequences of that structure. nor is this a new phenomenon. studies of the book industry document similar patterns of activity in both the 19th and 20th centuries, that is, centralisation and geographic concentration based on largescale mergers aimed at both economies of scale and market dominance, coupled with continual entry into the marketplace of numerous new small publishing houses which feed the cycle (coser 1982, lyons 2001). one important impact of this, which resonates strongly with the work of adamic, is succinctly summarised by john brooks, speaking for the authors guild, when he states: it is not that fewer books are being published than formerly, or even, at least not provably, that books of exceptional merit are going unpublished. it is, rather, that such books are increasingly being squeezed out of sight by the conglomerate-hyped and –packaged best sellers, and not reaching the public that they deserve, and that deserve them. (coser 1982, p. 29) again, there would appear to be a strong parallel with internet publishing. why is this happening? noam (2001) argues that the internet thrives on economies of scale, and large organisations are best placed to achieve them. elsewhere, he has made the claim that the ‘internet today is more concentrated than the rest of the information sector’ (noam 2003). castells (2001) argues this is because early adopters set the cultural and social context for the internet, and so dominate both the provision of information and the form of its communication. this assists in perpetuating their position. adamic (2000) agrees that the early mover advantage is paramount and there is also support in standard texts on the information industry. for example, varian and shapiro (1999) argue that not only do early adopters indeed have an advantage overall but that these patterns are reflected, with distinct settings, within each major industry sector. another perspective is provided by research on the advantages of technological conglomerates such as technology parks which provide benefits through improved information flows, mutual learning and economies of scale. (piore 1984). this also appears to be a factor in the success of broader geographic regions which are both strong in basic resources and can successfully harness aspects of social capital. for example, silicon valley’s success vis a vis it’s major competitor, the route 128 corridor outside of boston, is often claimed to be because of the informal horizontal networks established between industry players, whereas the route 128 corridor relied on an ‘i’ll succeed on my own’ philosophy, which resulted in poor performance in relative terms and its eventual eclipse. (putnam 2000, saxenian 1994). thus, access to skills and a culture of sharing and innovation, based on close geographic proximity, would appear to provide a significant advantage in harnessing the potential of the internet. if this is indeed the case, it would give a clear advantage to urban organisations. even if the overall concentration of internet publishing is clear, the question remains as to whether this work is more related to business imperatives than those that drive community sector organisations. if it does apply, how does it manifest itself and what are the implications? after all, many community sector organisations do not want to be major institutions and are more interested in operating at the local level – but the question comes back to whether there are factors subjecting community sector organisations to the same pressures to enlarge, whether this is at the cost of local control and, from the point of view of the broader research, whether there are any consequences for social capital. australasian journal of information systems vol 11 no 2 may 2004 73 change, community sector organisations and the inte rnet there is no doubt that community sector organisations are undergoing change. within australia, lyons (2001) has demonstrated that there are many (often sector specific) reasons for this –such as the reduced membership base of trade unions and churches, changing demographics and the need for increased size in order to be able to work effectively with government. these pressures are not new. with regard to the use of it specifically, an early acoss (1996) study into the use of it by australian community sector organisations made the point that many organisations are looking to it so that they are able to enter into a partnership with government provision of community services on an equal footing. increasingly this will mean taking up economies of scale in order to facilitate the adoption of e-commerce. similar issues have been documented overseas, by burt and taylor (2002) in the uk, and surman (2001) who notes the problems of fewer volunteers, decreased funding, increased need for coordination and higher demand for services in canada. the community sector is undergoing changes and there are a number of pressures on organisations forcing them to attempt to expand, or at least reverse declines in their membership base. it is not unreasonable to assume that their use of the internet would be seen as one opportunity to address some of the issues, and to assume that there will be some reflection of that in the type of presence they adopt, and the expectations of what that presence might achieve. at a broader level, giddens (1996) argues that the welfare state is under threat and at the very least undergoing restructuring and that ‘the concept of a ‘civil society’ lying between the individual and the state… is a suspect one when applied to current social conditions’ (giddens 1996, p. 229). both he and castells (2001) consider that globalisation is posing both threats and opportunities and suggest that at the very least that community sector organisations are in the process of change and undergoing shifts in their structure. certainly it is not hard to find other examples in this regard, which may require them to change in order to respond effectively e.g. the un call for the world summit on the information society on the one hand (wsis 2003), and the growing criticism of nongovernment organisations by the new right on the other (lobe 2003). these are global influences and certainly there will be pressure within countries to develop organisations that are capable of responding to these challenges. once again, this implies larger organisations, groups of organisations or peak bodies that have sufficient resources to respond to the challenges. thus there are a number of external pressures on community sector organisations, at all levels, which are pushing towards a restructuring of the sector and which can be expected to have different consequences for organisations with different characteristics. use of the internet no doubt forms part of the response to these issues but, as can be seen, this will add to the pressure for larger organisations. the question requires further analysis and it may be that the impact on community sector organisations will be determined by a number of factors such as the size of organisation in question, its geographic location and constituency, and the nature of its activities. for example, the requirements of small localised groups with limited constituencies (e.g. the local kindergarten) should be simple enough for them to make effective use of the internet without the need for economies of scale. medium to large organisations with a more diverse geographic constituency may be forced into increasingly large operations in order to meet new challenges and benefit from the economies of scale necessary to achieve their objectives. thus, they may be forced into competing with each other, adding to pressures for a restructuring of the sector. this may well have a range of implications for the development of community sector organisations and their use of the internet. it may also have implications for the maintenance of social capital. a pertinent example of how such restructuring may result in the atrophying of community life can be found in heying’s study (1997) of organisations in atlanta. he argues that the economic transformations that occurred between 1961 and 1991, principally the delocalisation of corporate leadership consequent on the increasing nationalisation and globalisation of us business, broke the nexus of interlocking sectoral memberships by removing senior executives and the logic of their philanthropic contributions from ‘place’ to a more strategic way of thinking, leading to a decline in local civic contribution in atlanta. in a similar vein, putnam (2000) suggests that accelerating nationalisation and globalisation of us economic structures may be having a negative impact on levels of social capital – ‘replacement of local banks, shops and other locally based firms by far australasian journal of information systems vol 11 no 2 may 2004 74 flung multinational empires often means a decline in civic commitment on the part of business leaders’. being based on the us experience, these examples may reflect a stronger business/community nexus that has traditionally been the case in australia, but they are nonetheless illustrative of the types of connections that exist and the ways in which they can be disrupted. this should be of significant concern to those interested in levels of social capital in rural and regional australia, particularly when considering the effects of other developments, such as those illustrated by the economic modeling commissioned by the national office for the information economy (noie). that work indicates that while the overall effect of e-commerce on regional australia should be positive, it will come at the cost of significant restructuring, particularly in the area of local retail activity and banking presence (noie 2000). add to this the potential for corresponding changes in the community sector, and the situation will require close monitoring. in summary, it would appear that use of the internet has real potential to reinforce, and perhaps exacerbate, existing patterns of activity, increasing the importance of place and the benefits of size, regardless of the type of organisation involved. castells may be overstating the case that organisations will be marginalised because of the internet, but it is true that community sector organisations face a range of pressures and that failure to make effective use of the internet in response to those pressures only weakens their position. harnessing the internet in at least a partial response to these pressures may well reinforce their general thrust, and at a general level may well contribute to a delocalisation of the sector, with consequent losses in social capital. at a practical level – specifically in terms of the use of the internet – this will have implications for the effectiveness with which web sites can contribute to the goals of their organisations, and presumably the levels of satisfaction with their performance. this analysis suggests a number of research questions which would need to be considered to determine whether there is any support on the ground for it. for example, is location an important factor? are some community sector organisations better placed to take advantage of the internet, and so respond to the challenges they face, based on where they are located? is there evidence that particular community sector organisations are developing services so much faster and to such an extent that there control of the medium and domination of the field will lead to their expansion at the expense of smaller organisations and leading to a corresponding ‘delocalisation’ of those organisations, with a consequent decline in local social capital? these are very broad issues and would need qualitative studies and trend analysis to explore them thoroughly. monash community ict index that work remains to be done, but a closer look at the results of the monash community ict index should contribute to our understanding. as mentioned, this was a nationwide survey of community sector organisations, the definition of which was drawn from the work of the australian bureau of statistics (abs) and included ‘any non-government, non-commercial organisation or group with a formal structure. it may be as large as a national charity or as small as a local book club. purely ad hoc, informal and temporary gatherings of people do not constitute organisations…’ (australian bureau of statistics 2001). so, for example, community welfare organisations, sporting bodies, unions, business associations, churches, clubs, childcare centers, were included. organisations and agencies directly funded by government, such as schools and public libraries, were excluded, as were large financial institutions such as credit unions. the sample was drawn at random from two national directories the yellow pages and that of ourcommunity.com. a total of 2,915 organisations were approached and 923 usable responses obtained, representing an overall response rate of 31.7% and an accuracy of +/4% at the 95% confidence interval. drawing heavily on the model used in the abs business use of it surveys (2002), the survey aimed to collect base data about the take-up of communication and information technologies, and to provide information on the how well community sector organisations thought they were using those technologies to meet their needs. to that end, it was structured around the following broad questions: • what proportion of community organisations currently use computers? • is that computer usage adequate to their needs? australasian journal of information systems vol 11 no 2 may 2004 75 • what proportion currently have access to the internet? • is that access adequate to their needs? • what proportion currently publish a web site? • does that web site meet their needs? • how will the situation vary over the coming 12 months? the key findings of the survey together with the detail of the methodology have been published separately (ccnr 2003). basically, the results confirmed trends identified in earlier surveys, (acoss 1996, vcoss 1997), in terms of both the rapid take-up of communication and information technologies and the barriers to that take-up. for example, while the proportion of organisations using computers necessarily remained fairly static over the period at 97.1%, the proportion with access to the internet increased dramatically from approximately 40% to 89.7%. the survey also found high levels of satisfaction with the performance of the technology. 77.8% of respondents agreed or strongly agreed with the statement that the “organisation’s use of computers met their organisation’s needs”, rising to 84.9% when respondents commented on the situation expected in one year’s time. similarly, 76.5% of respondents agreed or strongly agreed that current internet access was meeting the organisation’s needs, rising to 85.6% when considering the situation a year into the future. however, the findings relating to the publishing of web sites were not as positive. the overall proportion publishing a web site was found to be 61.1%, with only 55.8% of those agreeing or strongly agreeing that the organisation’s web site was meeting its needs. location was also found to be significant. organisations located in the capital city of a state or territory were much more likely to have a web site – 73.2% as compared to only 45.6% of those located in other areas. they were also more likely to be satisfied with the performance of their web site – 59.7% of city-based organisations were satisfied as opposed to 48.6% of those in other areas. significant differences also emerged in the capacity of organisations to adopt the technology when size of the organisation is taken into account. 49.1% of the smallest organisations publish web sites as opposed to 77.6% of the largest. it is clear from figure 1. that when a combination of these two factors, location and size, were taken into account, small (that is those with annual revenues of less than $1million) regional organisations were found to be significantly less likely to publish a web site. figure 1. organisations with web sites 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% $0 -$ 25 ,0 00 $2 5, 00 0$1 00 ,0 00 $1 00 ,0 00 -$ 1, 00 0, 00 0 $1 ,0 00 ,0 00 + city regional australasian journal of information systems vol 11 no 2 may 2004 76 overall, the reasons for not publishing a web site were quite narrow. 28.7% of organisations reported lack of skills as a significant barrier, followed by concerns about cost at 24.3%. 26.8% reported other concerns, and while these dealt with a variety of issues, most could be said to relate to resource issues and the current priorities of the organisations concerned. the number indicating that web publishing was not suited to their organisation’s needs was comparatively high at 35.7%. once again, the result obtained when the factors of size and location are combined is quite revealing: small regional organisations were least likely to have a web site and lack of relevance to the organisation’s needs was found to be the major factor, while for small city-based organisations it was the cost that was most important. some of the results relating to levels of satisfaction have been mentioned above, but in addition there was also found to be a significant relationship between the size of an organisation and levels of satisfaction – however, as can be seen from figure 2., it was that the level of satisfaction with the web site decreased with the increase in the size of the organisation: 67% of organisations with an annual revenue of less than $25,000 pa were satisfied or very satisfied with their web site, while the figure dropped to 42.7% for those organisations with revenue of $1 million pa or more. figure 2. web site meets organisational needs (by revenue) 0.00% 10.00% 20.00% 30.00% 40.00% 50.00% 60.00% 70.00% 80.00% $0 -$ 25 ,0 00 $2 5$1 00 ,0 00 $1 00 -$ 1,0 00 ,0 00 $1 ,0 00 ,0 00 + series1 as can be seen from figure 3., city-based organisations would appear to be better at taking advantage of the medium, and while the use of sites to collect information and to conduct financial transactions register as significant differences between city-based and regional organisations, the reality is that most organisations put their sites to a fairly limited range of uses. related results include the fact that organisations with an annual revenue of over $1 million are significantly more likely to use their web site for financial transactions, and that organisations which provide access to online services or two-way communications (forums) on their sites are significantly more likely to be satisfied with the performance of their sites. australasian journal of information systems vol 11 no 2 may 2004 77 conclusion at first glance, many of these results would appear to support the analysis offered at the start of this paper. city-based organisations are significantly more likely to have web sites and are significantly more likely to be satisfied with them. there is clearly a relationship between the size of an organisation and the likelihood that it will have a web site – a pattern that holds true in both the cities and regions. once again, city-based organisations are far more likely to have a web site than regional organisations of comparable size, although the difference for the largest organisations is not significant. finally, there is a definite inverse relationship between the size of an organisation and its level of satisfaction with its web site, suggesting that perhaps the expectations on the performance of their web sites is much higher. in the two major areas – location and size – the results are as suggested by the literature, and it would be easy, though premature, to accept that the mechanisms discussed earlier in this paper are at work. in fact, it may prove that these are only temporary divisions whose significance will diminish over time that can only be established through the creation of a time series and further quantitative analysis. it may also be that there are other factors at work, and it will be the purpose of the previously mentioned case studies to explore that possibility, specifically by examining issues such as the aims of the organisation, any environmental issues it must deal with such as access to skills and innovation or pressures to enlarge, and the strategic aims of their internet presence. in conclusion, an examination of the research surrounding the nature of the internet and the pressures on community sector organisations suggests that, in apparent opposition to its potential, the internet is more likely to reinforce, and perhaps exacerbate, existing patterns of activity, increasing the importance of place and the benefits of size. while failure to make effective use of the internet in response to those pressures will weaken their position, attempts to use it in at least a partial response to those pressures may well reinforce the general thrust of those pressures, and at a general level may well contribute to a delocalisation of the sector, with consequent losses in social capital. at a practical level – specifically in terms of the use of the internet – this has implications for the effectiveness with which web sites can contribute to the goals of their organisations, and presumably the levels of satisfaction with their performance. while an analysis of the results of the australasian journal of information systems vol 11 no 2 may 2004 78 monash community ict index would appear to offer support for this argument, additional research must be undertaken in order to clarify the real situation. references acoss (1996). electronic communication and the community sector, final report. acoss paper no. 81, australian council of social services. (http://www.acoss.org.au/papers/current accessed: 7/3/03) adamic, l. a., & huberman, b. a. (2000). the nature of markets in the world wide web. qjec, 1(1): 5-12. adamic, l. a., & huberman, b. a. (2001). the web’s hidden order. communications of the acm, 44 (9): 55-58. australian bureau of statistics (2001). 4441.0 voluntary work, australia. australian bureau of statistics (2002). 8129.0 business use of information technology burt, e. & j. taylor (1999). information and communication technologies: reshaping the voluntary sector in the information age? centre for the study of telematics and governance, glasgow caledonian university. (http://www.brunel.ac.uk/research/virtsoc/reports/voluntarymain.htm accessed 23/3/03) ccnr. (2003). the monash community information and technologies index (cict). school of information management and systems, monash unversity: clayton, vic. castells, m. (2001). the internet galaxy: reflections on the internet, business and society. oxford university press: oxford. christie, i. & hepworth, m. (2001). towards the sustainable e-region. in wilsdon, j. digital futures: living in a networked world. (pp. 140-162) earthscan publications ltd.: london. colista, c. & leshner, g. (1998). traveling music: following the path of music through the global market. critical studies in mass communication, 15(2): 181-194. coser, l. a., kadushin, c. & powell, w. w. (1982). books: the culture & commerce of publishing. basic books, inc.: new york. giddens, a. (1996). in defence of sociology. polity press: cambridge, uk. heying, c. h. (1997). civic elites and corporate delocalisation: an alternative explanation for declining civic engagement. the american behavioural scientist, 40(5): 657-668. lobe, j. (2003). u.s. conservatives take aim at ngos. commondreams. june 12, 2003. http://www.commondreams.org/headlines03/0612-09.htm (accessed 19/6/03). lyons, m. (2001). third sector: the contribution of nonprofit and cooperative enterprises in australia. allen & unwin: crows nest, nsw. lyons, m. & arnold, j., (2001) a history of the book in australia 1891 – 1945: a national culture in a colonized market. university of queensland press: st lucia, qld. noam, e. m. (2001). will the internet be bad for democracy? talk – camden, maine, usa. http://www.citi.columbia.edu/elinoam/articles/int_bad_dem.htm (accessed 10/6/2003). noam, e. m. (2003). the internet: still wide open and competitive? talk – oxford internet institute 21/3/2003. http://www.oii.ox.ac.uk/seminar210303.html (accessed 10/6/2003). noie (2000). e-commerce across australia. national office for the information economy: canberra, act. office of the eenvoy, u. (2002). e-enabling the voluntary and community sector. (http://www.eenvoy.gov.uk/publications/reports/e-enabling/summary.htm accessed: 6/12/02) piore, m. j. & sable, c. f. (1984). the second industrial divide: possibilities for prosperity. basic books: new york. putnam, r., leonardi, r. & nenetti, r. (1993). making democracy work: civic traditions in modern italy , princeton university press: princeton, nj. putnam, r. (2000). bowling alone: the collapse and revival of american community. simon & schuster: new york. saxenian, a. (1994). regional advantage: culture and competition in silicon valley and route 128. harvard university press: cambridge, massachussetts. surman, m. (2001). from access to applications: how the voluntary sector is using the internet.( http://www.volunteersonline.ca/news/environmentalscan.htm accessed: 15/3/02). australasian journal of information systems vol 11 no 2 may 2004 79 varian, h. r. & shapiro. c. (1999). information rules: a strategic guide to the network economy. harvard business school press: boston, massachusetts, usa. vcoss (1997). bridges and barriers research project. the council of adult education for the victorian council of social services.(http://www.skills.net.au/download/bbrpt.doc accessed: 7/3/03). wsis. (2003). world summit on the information society. http://www.geneva2003.org/wsis/index_c01_2_07.htm (accessed 19/6/03). zook, m. a. (2001). old hierarchies or new networks of centrality? – the global geography of the internet content market. american behavioural scientist, 44 (10): 1679-1696. the australasian journal of information systems (ajis) began in 1993 and was based on the simple realisation that there was a need for an academic ‘outlet’ for is research in australia australasian journal of information systems volume 14 number 1 nov 2006 information systems in western australian universities professor craig standing school of management information systems edith cowan university. c.standing@ecu.edu.au professor janice burn school of mis edith cowan university perth, wa j.burn@ecu.edu.au dr. chad lin school of management information systems edith cowan university c.lin@ecu.edu.au abstract the authors of this paper examine the current state of the development of information systems (is) teaching and research within western australia (wa). a brief overview of the wa environment is followed by an exploration of teaching and research in the four main universities. this is examined against the framework for the study and, in particular, the impact of social processes (ariav et al, 1987; klein et al, 1991) and local contingencies (culnan et al, 1993; checkland and howell, 1998), which are found to be of relevance to historic developments. introduction western australia (wa) is the largest state in australia, spanning 2400 km from north to south and encompassing 2.5 million square kilometres. the state is fairly isolated from the rest of australia and still maintains something of a ‘frontiersman’ mentality. this is compounded by the low population, with only 2 million inhabitants, approximately 73% of whom live in the capital city of perth, which has a current population of 1.5 million. the rest of wa is populated by small country towns which are typically remote from other centres and many of which lack basic infrastructure such as health and education services. in particular, broadband access is variable and information and communication technology services can be poor. 251 australasian journal of information systems volume 14 number 1 nov 2006 the capital city of perth has the dubious reputation of being the most remote city in the world, being as close to singapore as it is to sydney and a five hour flight from either. this has compensations since perth is on the same time zone as the asian centres of hong kong and singapore and hence attractive for international collaboration in business and indeed in education. however, the remoteness of the state has meant that few businesses headquarter in wa and, as such, collaboration with local industry becomes a problem since they tend to favour states where they are headquartered. the state also derives most of its revenue from the minerals and oil industries, with the majority of large companies associated with mining in some form. these location factors have impacted on the development of wa universities, since while all four public universities are based in perth, they all have a remit to service the whole state and so typically have remote campuses across wa. additionally, all universities have substantial external offerings which initially targeted the distant wa population but now recruit from a global community. wa population has remained fairly stable but is now forecast to grow; however, the growth in university students has come largely from out of state enrolments and specifically from asia. overview of wa universities the five universities in wa (ordered by number of enrolments from largest) are: curtin university of technology edith cowan university (ecu) the university of western australia (uwa) murdoch university notre dame university table 1 shows overall enrolment figures for 2004 and comparative statistics from dest selected higher education statistics. university total % overseas % postgrad % research % school leavers curtin 36,064 39.7 22.2 4.3 43.1 ecu 23,887 17.8 19.6 2.6 33.1 uwa 16,806 17.7 25.8 11.3 75.0 murdoch 12,655 17.5 17.8 6.5 34.9 notre dame 3000+ n/a n/a n/a n/a* * notre dame university is a private catholic university table 1 shows overall enrolment figures for 2004 and comparative statistics. a brief historical overview of these five helps to explain the particular unique focus of each institution and the positioning of information systems within the universities. university of western australia uwa was the state’s first university, established in 1911, and is a member of the ‘group of 8’ – referring to the eight most prominent research oriented universities in australia, often referred to as the ‘sandstone universities’. uwa has proportionately the highest intake of school-leavers nationally (national average was 44.3%) and consistently recruits over 80% of the state’s top 5% of school leavers. the good universities guide states “uwa has more young, bright full-timers than 252 australasian journal of information systems volume 14 number 1 nov 2006 any other university in the country it is at the top of the wa academic totem pole”. typically all undergraduate students would be full-time. with this profile of students, the most popular disciplines are the traditional ones and information systems plays only a supporting role. originally is was part of the department of information management and marketing (dimm) founded in 1993. a full major of is units was first offered in 1998. the im discipline group is now embedded within the school of economics and commerce; itself one of two schools in the uwa business school. curtin university of technology curtin university of technology is wa’s largest university and evolved from the western australian institute of technology (wait) in 1986. wait itself had been created in 1966 from the former perth technical college, which in 1969 also merged with the wa school of mines, the muresk agricultural college and schools of physiotherapy and occupational therapy. between 1966 and 1976, student enrolments expanded from 2,000 to over 10,000 and wait consolidated around three large schools of: • health sciences • business and administration • arts and architecture wait’s business school was the largest in australia and, in 1975, the school of computing and quantitative studies was formed with a prime focus on is. in 1986, when curtin university was formed, the school was renamed the school of information systems and remains within the business faculty. there is a separate school of computer science in a different faculty. curtin has a main campus and two additional metropolitan campuses in perth and regional campuses in kalgoorlie and northam. in addition, they have an off-shore campus in sarawak malaysia and a new campus in sydney central business district. curtin has 13 education centres within wa and 15 overseas education centres. curtin has the largest international student population in the state with around 40% high by national standards. these students will typically be full-time and their australian counterparts a mix of full-time and part-time. edith cowan university ecu was granted university status in 1991 and evolved from a number of education colleges formed as early as 1902. ecu remains the state’s major provider of teacher education. while the state’s second largest university, it actually has the largest enrolment of wa students. ecu focuses on the service professions and defines its particular strengths as: • education • nursing • business • computing • communications • creative and performing arts 253 australasian journal of information systems volume 14 number 1 nov 2006 information systems was part of the school of information systems and management science in the 1980s and became the department of management information systems in 1988 within the faculty of business. a separate school of computer science is in a different faculty. ecu had many campuses but has recently consolidated and now has its main campus 28km north of the city centre in joondalup – one of the fastest growth areas in australia. they have a second city campus and a regional campus at bunbury in the south. over 50% of the students at ecu are ‘mature’ entrants and typically part-time students. with this profile is students would normally come from within the it industry – as it has suffered a decline so the student population has declined. murdoch university murdoch university was formally constituted as wa’s second university in 1973 and, while a research-based university, it has a very strong emphasis on good teaching. information systems has had a chequered history as part of an information technology group. originally the department of it was part of the school of mathematics, business and sciences and then in 1997 it was moved to business information technology and law. in 2003, the school of information technology was moved into the division of arts. murdoch university has three campuses with two regional campuses and one city campus. notre dame university the university of notre dame was founded in 1990 and is a private catholic university. despite its small size it has eight colleges and the school of information technology resides within the college of science and technology. interestingly, all the courses offered in it are offered in online mode and the university spawned a separate company to promote the development of such online material. there is no is group as such and the it school has only two permanent staff members. for the purposes of this study only the public universities are considered in depth. notre dame does not have an information systems department. university unit is staff is ug students is pg students curtin school of is 24 ~ 160 ~ 200 ecu school of mis 18 ~ 150 ~ 110 uwa school of it 7 ~ 100 ~ 25 murdoch school e & c 8 ~ 70 ~ 10 table 2: an overview of the approximate numbers of is staff and student numbers within the four public universities. within these staffing numbers, curtin university has three is professors, edith cowan has three is professors and uwa and murdoch, no is professors. 254 australasian journal of information systems volume 14 number 1 nov 2006 the study interviews were conducted with senior academics at the four public universities during april and may 2005. the data summarised below represents the picture at that time. it should be noted, however, that in all cases, numbers were falling and continue to fall, with a severe decline in student enrolments and commensurate reduction in staff numbers. the dramatic upswings and downswings in student numbers in the is discipline over the years would suggest that the ‘discipline’ is very tightly aligned with the perceived industry needs of the moment rather than being viewed as a substantive ‘professionalised’ academic discipline in its own right (ruscio, 1987). courses curtin university cover the whole spectrum of it, ec and is with a recent emphasis towards some of the more technical is subjects; ecu focuses on the business applications of is, ec and im with a specialisation in erp at post graduate level; uwa clearly identifies information management as their specialisation with a focus on support for other discipline majors such as accounting and marketing and murdoch focus on the more technical aspects of is such as is development and use of multimedia. ecu is also targeting a new market with a masters degree in strategic project management which they hope will appeal to a broader market base. university ug courses pg courses curtin bachelor of commerce (is) bachelor of commerce (it) bachelor of commerce (ec) grad dip/certificates post grad dip/certificates master (is) master of commerce phd ecu bachelor of business (is) bachelor of business (ec) bachelor of business (im) bachelor of is grad dip/certificates (is, ec, erp) master (is) master strategic project management dba (is) phd uwa bachelor of commerce (is) master information management master of commerce (im) phd murdoch bachelor of science (is dev) bachelor of science (bis) bachelor science (multimedia) bachelor of science in is grad dip masters is phd table 3 shows the wide variety of courses offered across all four universities. despite some low enrolments in the ug and pg programmes, phd programmes continue to thrive in all four is groups with a total of around 60 doctoral students by research. in addition ecu offers a unique dba programme in is. this is a partially taught programme with a shorter thesis component and has a further 20 doctoral students mainly drawn from senior managers in the wa community. in general enrolments in general degrees such as bachelor of business with is specialisations remain fairly steady but enrolments on specialist is degrees are dropping radically. 255 australasian journal of information systems volume 14 number 1 nov 2006 interest in multimedia has declined significantly and electronic commerce no longer holds the appeal it once did. international recruitment has had a substantial decline impacting on the postgraduate market. research research focus within the four universities is also very different and this may be one of the reasons that all interviewees identified the low level of collaboration between wa universities. indeed, at the time of writing there were no joint projects and staff in each university seemed unfamiliar with research being conducted in other wa institutions. funding was variable with the onus on staff to generate funding through successful grant applications both internally or externally or through collaborative partnerships with industry. curtin has an industry sponsored professor which comes with considerable prestige in addition to research funds. research output also varied considerably. at uwa research activity of staff was largely interdisciplinary and with a relatively low proportion contributing to strictly is-related research to a high output overall at ecu with around 90% of staff research active. curtin followed the more typical path with 20% of staff providing 80% of output. it was stated that curtin had shifted emphasis towards teaching over research in recent years. identified themes for ongoing research are shown in table 4. university themes groups funding curtin technology adoption information technology management health informatics relational ontology worldwide databases grid supercomputing electronic commerce soft systems methodology individuals + phd groupings around professors grant based + research performance index funding system ecu electronic markets collaborative commerce in smes is evaluation is in construction industry is education and is in education 5 formal research groups led by the professors/ass profs + individual grant based + research activity index funding system uwa knowledge management virtual communities e-government transport modelling compliance monitoring systems individuals and interdisciplinary and inter-faculty collaborations grant-based and individual research support from school funds 256 australasian journal of information systems volume 14 number 1 nov 2006 murdoch human factors in is knowledge management e-learning decision support business strategy alignment individuals grant based + individual allocation from school table 4 research themes curtin and ecu both operate a research performance monitoring scheme which is directly linked to research funding (curtin university – research performance index, ecu – research activity index). in this way staff are rewarded with a $ value applied to the number of points they have generated towards the research assessment quantum. this scheme provides considerable incentive for staff to become productive but can favour quantity over quality. however, at ecu the recent trend has been to decrease the funding per point being allocated to staff and to the use the money to provide competitive internal research grants. all four universities support both qualitative and quantitative research and employ multi-methods. case studies and interpretive research were commonly applied. the overall research picture in wa fits closely with the view of a ‘fragmented adhocracy’ (whitley, 1984; culnan et al, 1993; checkland and holwell, 1998) with low researcher mutual dependency and very different contributions from each of the four universities reflecting local political pressures from within their different institutions but also from within the state where all four compete for highly limited funding. is leadership the particular environment of wa has strongly influenced is leadership since west australians are often loath to leave the state because of the rather laid back life style and tropical weather, giving a core academic staff remaining fairly static over the long term. in addition west australia is seen as a desirable location for international immigrants and hence the state attracts a fair share of international academics. it is rare to see australian academics from the eastern states choose to relocate west. this has had two consequences, wa academics have had to strive for excellence within the australian arena; wa has gained an international profile through its international academics. richard watson (originally wa home grown), bob galliers and janice burn have all contributed to the development of is within the state, coming from an international perspective, and west australians graham pervan and bernie glasson have had significant input both nationally and internationally. all six current professors at curtin and ecu are well known researchers with national and international reputations. interviewees stressed the real need for is leadership and active involvement in is research by the professoriate. is groups without a professor tended to have significantly lower profile in their home university. conclusions the main is strength in wa lies in curtin and ecu, where separate is schools have been in existence for over 30 years. in 2007, the is school at ecu will merge with the school of management to form a new school but it will still retain its identity as a discipline. is is recognised as a discipline in uwa and murdoch but as a sub-set of other disciplines and has a somewhat lower 257 australasian journal of information systems volume 14 number 1 nov 2006 status in these two institutions. different value systems and funding models inhibit collaboration between the four institutes. in 1998 curtin took the lead to organise the wa workshop in is research (wawisr) and this was repeated each year, hosted by each university in turn until 2001, when enthusiasm to organise it was exhausted. sadly, one of the reasons for this may be the increased emphasis on recognised research output, lessening the motivation for activities which would solely be regarded as community building. similarly, in 1999 ecu ran the first working for e-business (web) conference based in perth and continued to organise this until 2004. this conference attracted some local attendees but many more came from across australia to forge national collaborative links. with respect to the development of is as a discipline, social processes and local contingencies seem to have had considerable impact. wa is the most distant state in australia and hence looks inward to local industry for direction but also outwards from australia to forge international reputations. mechanisms of control are very much still in a developmental mode although this applies to the country as a whole, with is representation on the national research funding body – the australian research council (arc) only being achieved in 2000 with the appointment of professor janice burn as the is discipline representative on the arc research panel for mathematics information and computer sciences. the core body of knowledge taught at each of the four universities tends to reflect local allegiances with industry, services and professions and varies in line with local developments and needs rather than necessarily responding directly to demands from within the discipline itself. all interviewees commented on the decrease in numbers in is and diminishing staff numbers, which suggests that the picture painted in this study may be a very different one over the next few years. it was also interesting to note that is was seen to be a discipline grouping which could easily be transposed from within one discipline grouping to another, with residual homes having been found in business, sciences, mathematics and even engineering at one time or another across the four institutes. from the wa perspective it would appear that is is still ‘a perspective’ (fitzgerald, 2003) rather than a discipline and a perpetually shifting perspective at that. references ariav, g., de sanctis, g. & moore, j. (1987) “competing reference disciplines for mis research”, proceedings of the eighth international conference on information systems, pittsburgh, pp 455-458. checkland, p. & holwell, s. (1998) information systems and information systems – making sense of the field. wiley and sons, chichester. culnan, m. j., swanson, e. b. & keller, m. t. (1993) “mis research in the 1980s: shifting points of work and reference”, proceedings of the 26th hawaii international conference on systems sciences, vol 3, pp 597-606. fitzgerald, g. (2003) “information systems: a subject with particular perspective, no more, no less”, european journal of information systems, vol 12 no 3, pp 225-229. klein, h. k., hirshheim, r. & nissen, h. (1991) “a pluralist perspective of the information systems research arena”, in nissen, h. e., klein, h. k. and hirshheim, r. (eds). information systems research: contemporary approaches and emergent traditions, elsevier, amsterdam. 258 australasian journal of information systems volume 14 number 1 nov 2006 ridley, g. (2006) “characterising information systems in australia: a theoretical framework”, australasian journal of information systems, vol 13 no 3, pp. 38-60. ruscio, k. (1987) “many sectors, many professionals”, in b. clarke (ed). the academic profession, university of california press. whitley, r. (1984) “the development of management studies as a fragmented adhocracy”, social science information, vol 23 no 4/5, pp 775-818. acknowledgements thanks are due to the following interviewees: professor janice burn, ecu dr nick letch, uwa professor graham pervan, curtin professor craig standing, ecu dr fay sudweeks murdoch mr danny toohey, murdoch the authors janice burn was appointed foundation chair of information systems at edith cowan university in perth in 1997. she had previously held positions in hong kong, canada and uk. her research interests centre on the strategic use of is and its impact on organisational effectiveness in a wide variety of global contexts. janice has published extensively in these areas and now enjoys a well earned semi-retirement in the south of france where she continues to supervise her doctoral students as a virtual professor. craig standing is professor of strategic information management and head of school in the school of management information systems at edith cowan university, australia. his current research interests are in the areas of electronic markets and mobile commerce. he has published in the european journal of information systems, ieee transaction on engineering management, information & management and european journal of operational research. dr chad lin is a post-doctoral research fellow in school of management information systems at edith cowan university in australia. he has published many international research articles on a range of topics including outsourcing, e-commerce, is/it adoption, and evaluating the effectiveness of is/it. 259 australasian journal of information systems volume 14 number 1 nov 2006 260 introduction overview of wa universities university of western australia curtin university of technology edith cowan university murdoch university notre dame university courses the authors microsoft word 13 1 marina.doc ajis vol 13, no. 1 september 2005 8 women in the australian it industry: where are we now? graeme j. byrne and lorraine j. staehr school of engineering and mathematical sciences, la trobe university, po box 199, bendigo, australia, 3552 abstract the purpose of this paper is to assess the status of women in the it industry in 2001 relative to their position in 1996 and in comparison to their status in other industry sectors. to achieve this a special tabulation of 2001 and 1996 australian census data relating to women’s position within the australian it industry is analysed. the issues of participation, pay equity and representation at senior management levels are examined. this reveals a fairly negative picture and indicates the need for an intensified effort to rectify the situation. it is shown that women participate in the it industry at much lower rates than women in many other industries and that they have not achieved pay equity with men. also, women are not well represented in senior management roles. background the ongoing under-representation of women in it in australia is well documented (e.g. newmarch, taylor-steele and cumpston, 2000; von hellens and nielson, 2001; apesma, 2004). this problem is not confined to australia but is also found in other western countries e.g. the us and the uk (mai tland, 2001). in australia this under-representation starts in the education system and persists today despite efforts over the last fifteen years by the government and universities to address the problem. as far back as 1990 the australian government announced a national objective of 40% women in non-traditional courses in its discussion paper 'a fair chance for all' (deet, 1990). this resulted in financial support from the government through the higher education equity programme (heep) for universities to encourage more young women into undergraduate information technology (it) degrees. programmes were devised to both encourage the participation and retention of women in these courses. an overview of some of the programmes offered in australian universities can be found in craig, fisher, scollary and singh. (1998), and specific details of one such programme at la trobe university in staehr, martin and byrne (2000). so, how successful have these programmes been? there is evidence to suggest that although women’s participation rate in university degrees may not have reached the targeted 40% level, there has been an increase in the number of women studying it. the number of women in the it industry is not only affected by the number of women gaining it qualifications from universities, but also from technical and further education (tafe) colleges and private providers (e.g. computer power training institute). in addition, there will be women both already in, and entering the it industry without formal qualifications in it. it is only since 1997 that membership of the australian computer society (acs) at the highest level normally requires formal it qualifications. accenture1, a high profile it consulting company, advertises for graduates from commerce, hr, business, and engineering as well as it. therefore, there are multiple pathways for women to enter the it industry in australia. the number of women with post secondary school qualifications in 1 http://careers3.accenture.com/careers/australia/graduateshome/youquestionsanswered/default.ht m ajis vol 13, no. 1 september 2005 9 it (this includes university, tafe and private providers) increased from 18,986 in 1991 to 46,707 in 2001 (abs, 2003). however, the percentage of women relative to men actually decreased by 6.3%. the australian government has legislated in an attempt to secure equal opportunity for women in the workplace. the federal equal opportunity for women in the workplace agency2 was established in 1986 (eowa). this agency administers the equal opportunity for women in the workplace act 1999. the act requires companies with 100 or more employees to establish a workplace program to remove the barriers to women entering and advancing in the workplace and to report regularly to government. the sanctions for noncompliance are that the company will be named in parliament and the company will be ineligible to tender for government contracts and industry assistance. companies with particularly good policies are eligible to win awards and have their names publicized in the media, providing incentives for companies to have woman-friendly workplace policies. currently the requirement is only to have policies in place but there is no compulsion for companies to ensure that the policies are actually achieving benefits for women. the main purpose of this paper is to report on the status of women in the australian it industry. a special tabulation of australian census data for 1996 and 2001 (abs, 2002a) is examined, focusing on women’s participation and remuneration in the it industry. much of the evidence in the literature for the low participation rates of women comes from sample data. although there will be some inaccuracies (~1%), census data is far superior to sample data where errors are often much larger (5 to 10%). in addition the data used in this paper is disaggregated to the individual occupation level. although some data is available for 1991, it is not possible to match occupation groups from this census to those in the 1996 and 2001 censuses due to changes in definitions and so 1991 data is not considered. in this paper the best quality data available to date is used due to the revised australian standard classification of occupations (asco) codes. this allows for a much more detailed examination of specific areas within the it sector so the true extent of the problem can be highlighted. in the following sections the nature of the data is described, data analysis with an integrated discussion is presented, followed by some concluding remarks. the nature of the data in july 1996, the asco scheme was introduced (abs 1996). since then, the australian bureau of statistics (abs) and numerous other agencies (dest, 2002) have used this standard when classifying employment information. asco uses a 6-level coding system that classifies an occupation according to criteria such as; the common and alternative job titles, the skill level of the job and the common tasks carried out. this results in over 1,100 minor occupation groups of which eleven are relevant to it occupations where significant skill levels are required. these are the minor groups considered in this paper. occupation data in both the 1996 and 2001 censuses were coded using the asco system, which allows detailed examination of the participation of women in it occupations. the data set obtained for this study covers all of australia and contains identifiers for 11 occupation groups, 17 industry sectors, and gender for the 1996 and 2001 censuses. 2 http://www.eowa.gov.au/about_eowa.asp ajis vol 13, no. 1 september 2005 10 occupation descriptions the occupation descriptions in table 1 are based on those provided by the abs (1996) and dest (2002). the associated asco codes are also displayed. the first digit of these codes indicates the broad occupation group to which these minor groups belong and skill level required decreases as the number increases. a first digit of 1 indicates the broad group managers and administrators, a 2 indicates the broad group professionals and 3 indicates the broad group associate professionals. the descriptions of these broad groups based on asco definitions (abs, 1996) are • managers and administrators head government, legislative, industrial, commercial, agricultural and other establishments, or departments within these organisations. they determine the policy of the organisation or department, and direct its functioning, usually through other managers, and coordinate economic, social, • technical, legal and other policies. • professionals perform analytical, conceptual and creative tasks through the application of theoretical knowledge and experience in the fields of science, engineering, business and information, health, education, social welfare and the arts. • associate professionals perform complex technical and administrative support functions that require an understanding of the underlying theories and methods of a particular field and significant practical skills. tasks are often performed in support of professionals. in 2001 within the it industry there were; 27,198 persons working as managers and administrators of which 6,097 (22.4%) were women, 135,685 persons working as professionals of which 30,339 (22.4%) were women and 29,250 persons working as associate professionals of which 8,972 (30.7%) were women. the minor groups chosen for this study, with the exception of computing support technician, all have a minimum entry requirement of a bachelor degree or experience considered equivalent. electronic engineers were not included as they can have one of three specialisations: biomedical engineer, computer engineer or mechatronic engineer. computer engineers are included in the asco code 2-23113 above. note also that the data does not include it research and teaching professionals within the education sector. where education is mentioned as an industry sector it only includes the it occupations that would be associated with a computer support centre. ajis vol 13, no. 1 september 2005 11 table 1: asco defined it occupations and their descriptions (abs, 1996) asco code job title job description 1-22411 information technology manager plans, administers and reviews the acquisition, development, maintenance and use of computer and telecommunications systems within an organisation. 2-22213 sales representative (information and communication products) represents companies in selling information and communications equipment, systems and solutions to businesses. 2-23100 computing professionals, nfd3 designs and prepares software to meet specific requirements in all aspects of the computing environment. also controls and audits the operation of computing facilities within an organisation. (this occupation group is assigned when no further definition (nfd) of a more specific nature is recorded on the census form) 2-23111 systems manager controls the operating effectiveness of information technology systems, and arranges system maintenance and enhancements to meet user requirements. 2-23113 systems designer conducts research into computer and communications systems, and develops techniques for their effective application and use. 2-23115 software designer designs and modifies the operating environment software which links computer software and hardware 2-23117 applications and analyst programmer writes tests and maintains computer programs to meet the application needs of end users of computer systems. 2-23119 systems programmer writes, maintains and updates programs that control the overall functioning of computers. 2-23121 computer systems auditor designs and monitors control systems, ensures the integrity and security of data and reviews the organisation's computing environment and usage of computer facilities. 2-23179 computing professionals, nec3 this occupation group covers computing professionals not elsewhere classified and includes occupations such as multimedia developer, business (systems) analyst and 3 abbreviations; nfd = not further defined and nec = not elsewhere classified, refer to persons who could not be classified into a more definite category. ajis vol 13, no. 1 september 2005 12 web designer/developer. 3-29411 computing support technician provides technical advice and support to users of computer software and hardware. data analysis and discussion according to the 2001 census of population and housing, there were 153,183 persons in australia with a post-secondary qualification in the it area. of this number, 30.1% were women, which represents a decrease of 6% on the same figure in 1996. given that the total number of men and women with it qualifications increased by almost 70% between 1996 and 2001, this decline indicates a significant worsening of the trend away from it education by women. also, as shown in table 2, the total number of persons working in it occupations in 2001 is 192,132, compared with 126,375 persons in 1996 (a 52% increase). during the same period, women’s participation in it occupations decreased by 2.3%. thus, no matter how you look at it, the it industry is failing to attract/retain women and the situation is getting worse. possible causes have been identified by von hellens and nielsen, (2001) and include; misconceptions about the nature of it work, unfriendly male dominated workplaces and the fact that the nature of it work makes it difficult to balance work and home responsibilities. industry sectors this section presents a detailed examination of women’s participation in it occupations focusing on industry sectors in 2001, and changes during the inter-censual period. table 2 below shows the numbers of men and women in each of the 17 industry sectors, along with the participation rate of women for the 2001 census. of the 192,132 persons employed in it occupations in 2001, the great majority (45%) are employed within the property and business services industry with finance and insurance, manufacturing, communication services, government administration and defence, wholesale trade, retail trade and education together employing a further 45% in it roles. from a women’s participation rate perspective, the best performing industries are health and community services, personal and other services, finance and insurance, agriculture, forestry and fishing, and government administration and defence. interestingly, the industry with the highest number of it employees (property and business services) has one of the lowest female participation rates at 20.8%. overall, the participation rate of women in it occupations is 23.6%, which is less than half the rate for all occupations across all industry sectors (abs, 2002b). it is probably not surprising that the health and community services has by far the highest participation rate as traditionally this industry has employed mainly women. perhaps this industry sector provides a more supportive environment for women in general, and therefore also for women in it roles compared with other industry sectors. ajis vol 13, no. 1 september 2005 13 table 2: numbers employed in it occupations classified by industry sector and gender in 2001. numbers employed in it occupations women’s participation rate industry women men persons it occupations all occup ations accomm., cafes and restaurants 136 385 521 26.1% 56.1% agriculture, forestry and fishing 131 315 446 29.4% 30.3% communication services 2923 10573 13496 21.7% 34.0% construction 340 1180 1520 22.4% 13.3% cultural and recreational services 726 2020 2746 26.4% 47.9% education 2353 6404 8757 26.9% 68.7% electricity, gas and water supply 517 1893 2410 21.5% 19.8% finance and insurance 5290 11637 16927 31.3% 56.3% government admin. & defence 3791 9124 12915 29.4% 44.9% health and community services 1408 2475 3883 36.3% 78.5% manufacturing 2818 10888 13706 20.6% 26.6% mining 243 805 1048 23.2% 13.6% non-classifiable economic units 213 776 989 21.5% 43.3% not stated 117 421 538 21.8% 47.4% personal and other services 556 1205 1761 31.6% 47.2% property and business services 17911 68086 85997 20.8% 46.3% retail trade 2268 6520 8788 25.8% 52.5% transport and storage 1035 2971 4006 25.8% 25.6% wholesale trade 2632 9046 11678 22.5% 32.3% all industries 45408 146724 192132 23.6% 45.2% occupation groups this section presents an examination of women’s participation in it focusing on the minor occupation groups and their changes during the inter-censual period. figure 1 below shows the percentage breakdown for all persons in the it occupation groups at the 1996 and 2001 censuses. in both census periods, applications and analyst programmers, computing support technicians, information technology managers, systems managers and software designers were the major occupation groups. also, while the total numbers in all occupations except systems programmers increased (see table 3), this growth was not uniform with the percentage share in occupations such as systems managers and computing support technicians falling by around 2% while the percentage share for software designers and information technology managers increased by 4.1 and 2.4% respectively. table 3 shows the numbers employed and women’s participation rates (wpr) in 1996 and 2001 in each of the minor it occupation groups defined by asco (see occupation descriptions above). in 2001, women’s participation in it occupations was highest for computing support technicians (30.7%) with sales representatives, systems managers and computer systems auditors making up 26-28% of their respective occupation groups. the 2001 census (abs, 2002b) puts women’s participation in the broad occupation groups (see occupation descriptions above); managers and administrators, professionals and associate professionals at 28%, 53% and 44% respectively across all industry sectors. as ajis vol 13, no. 1 september 2005 14 mentioned earlier, the participation rates within the it sector for these broad groups are 22.4%, 22.4% and 30.7% respectively. clearly, women managers, professionals and technicians are underrepresented within the it industry compared with most other industries. this is especially true for computing professionals where women’s participation is less than one half of the value across all industries. thus, although many professions such as medicine and law have been successful in attracting women in at least equal numbers to men, the it profession has a long way to go if equity is to be achieved on this measure. 1996 2001 0.0% 5.0% 10.0% 15.0% computing support technician computing professionals, nec computer systems auditor systems programmer applications and analyst programmer software designer systems designer systems manager computing professionals, nfd sales representative information technology manager 17.2% 15.2% 8.3% 9.2% 0.4% 0.4% 4.1% 2.3% 16.7% 15.6% 7.4% 11.4% 8.6% 8.9% 14.3% 12.1% 4.6% 5.7% 6.7% 4.8% 11.7% 14.2% figure 1: persons in each it occupation group as a percentage of all it occupations 1996 and 2001. changes over time also shown in the last column of table 3 is the inter-censual change in women’s participation (�wpr). while there are some encouraging signs with participation increasing by 2.3% for computing professionals nfd, 2.7% for software designers, 2.1% for systems managers and 1.2% for information technology managers, all other occupation groups exhibited no change or a decline in women’s participation between 1996 and 2001. overall there was a 2.3% decrease in women’s participation in it occupations in the 5-year period to 2001, the worst situation being for computing support technician where an 11.7% decline occurred. there is no obvious explanation for this large decline and further research on this issue is required. the overall 2.3% decrease is worrying, but not surprising, particularly in light of the partially successful efforts by government and the universities to increase the participation and retention rates of women in it education (newmarch et al., 2000). ajis vol 13, no. 1 september 2005 15 table 3: numbers employed and women’s participation by asco occupation groups in 1996 and 2001. 1996 2001 women total wpr women total wpr �wpr information technology manager 3142 14808 21.2% 6097 27198 22.4% 1.2% sales representative 2378 8440 28.2% 2584 9191 28.1% -0.1% computing professionals, nfd 1129 5850 19.3% 2369 10989 21.6% 2.3% systems manager 4686 18042 26.0% 6553 23319 28.1% 2.1% systems designer 1910 10852 17.6% 2928 17195 17.0% -0.6% software designer 1307 9346 14.0% 3681 21997 16.7% 2.7% applications and analyst programmer 5106 21097 24.2% 6928 29944 23.1% -1.1% systems programmer 1011 5186 19.5% 861 4421 19.5% 0.0% computer systems auditor 165 560 29.5% 223 864 25.8% -3.7% computing professionals, nec 2647 10451 25.3% 4212 17764 23.7% -1.6% computing support technician 9252 21743 42.6% 8972 29250 30.7% -11.9% group total 32733 126375 25.9% 45408 192132 23.6% -2.3% income data in this section, median incomes of men and women are compared across all minor it occupation groups. the abs has determined that the median is the most appropriate measure of typical income for an occupation group (abs, 2001). figure 2 shows that median incomes of women are below that of men in all minor it occupation groups. census data (abs, 2002b) show that far more women work on a part time basis than men and this may explain some of the disparity in income between men and women. however general workforce data from the actu (2002) suggest that even after adjusting for part time work effects, a significant disparity remains. also, the percentage of women engaged in part time work in the it industry is less than in other occupations (apesma, 2004) due to the relatively high skill levels required for entry into this profession. thus it appears that women actually are paid less than men in similar occupations in the it sector. given that government legislation in australia mandates equal pay for equal work regardless of gender the foregoing observation raises serious questions about what is actually happening in the australian workplace. kirner and rayner (1999, p. 26) provide some evidence suggesting that women do not do as well financially when negotiating individual contracts such as australian workplace agreements. the scatter plot in figure 3 is an attempt to view the two issues of participation and income equity jointly. it shows the median income for women and men for each minor it occupation group (indicated by asco code) plotted against the corresponding participation rates. if equity in both remuneration and participation rates were to exist, the points for women and men for each occupation group should coalesce along the 50% line at identical income levels. ajis vol 13, no. 1 september 2005 16 female male $0 $500 $1,000 $1,500 median income computing support technician computing professionals, nec computer systems auditor systems programmer applications and analyst programmer software designer systems designer systems manager computing professionals, nfd sales representative information technology manager $724 $830 $984 $1,192 $1,010 $1,349 $966 $1,128 $991 $1,092 $1,121 $1,286 $1,164 $1,306 $836 $1,108 $960 $1,227 $791 $1,006 $1,341 $1,500 figure 2: comparison of men’s and women’s median incomes in it occupations 2001 � male � female 0% 25% 50% 75% 100% participation rate $500 $750 $1,000 $1,250 $1,500 m e d ia n i n c o m e � � � � � � � � � � � � � � � � � � � � � � 122411 222213 223100 223111 223113 223115 223117 223119 223121 223179 329411 122411 222213 223100 223111 223113 223115 223117 223119 223121 223179 329411 figure 3: median income of it occupations versus participation rate the substantial horizontal distance between the points along with the generally lower vertical positions of the female points relative to the corresponding male points provides a measure of the extent of gender inequity within the it industry. this diagram also illustrates that higher participation rates of women are associated with lower incomes while for men the situation is reversed. ajis vol 13, no. 1 september 2005 17 conclusion unfortunately there is little good news. the analysis presented in this paper clearly shows that the position of women within the australian it industry did not improve during the 1996 to 2001 period, and in many cases actually worsened. the main results show that the ratio of men to women is about 3 to 1 and women participate in it occupations at about half the rate found in the general workforce. on an industry basis the health and community service sector appears to provide an environment that supports women in it occupations but even here the participation rate is far below the figure for other health related occupations. few women have made inroads into management roles in it with and those that do appear to earn significantly less than their male counterparts. most occupations experienced no change or a decrease in women’s participation between 1996 and 2001. notable exceptions to this were computing professionals (nfd), systems managers, and software designers where gains of between 2% and 3% occurred. the occupation group with the highest participation rate is computer support technicians however there was a severe downward trend of nearly 12% between 1996 and 2001. income disparities between men and women occur in it occupations as in most other occupations. also, women are concentrated in the lower income occupations of the it sector. employment in the it industry has been falling since 2001 (houghton, 2003). more recent survey data on women in it suggests that women have borne the brunt of the financial constraints that accompanied this decline in the sector (apesma, 2004). it is clear that much work needs to be done if women are to participate and receive financial rewards commensurate with men in the australian it industry. there is some evidence that government has recognised the problems and is moving to address them. recent initiatives include the establishment of the victorian women in ict network and the victorian government’s new realities careers awareness campaign that are designed to attract women to it careers and provide support for those already in an it career (see http://www.mmv.vic.gov.au/home). also the australian federal government’s women in ict summit advisory group established in 2005 (see http://www.dcita.gov.au/ict) is designed to increase the number of women studying and working in it. the next australian census is due in 2006 and it will provide another opportunity to review the situation. if the trend by women away from it occupations continues then government, universities and industry will need to re-evaluate their strategies to achieve gender equity in it. references australian council of trade unions [actu] (2002) “pay equity”, url: http://www.actu.asn.au/public/futurework/payequity.html, accessed 31 july 2005. association of professional engineers, scientists and managers, australia [apesma] (2004) “women in the professions – survey report 2004”, url http://www.apesma.asn.au/women/survey/women_in_professions_2004.pdf, accessed 27 july 2005. australian bureau of statistics, (1996) “1221.0 –information paper: australian standard classification of occupations”, second edition, canberra: abs. australian bureau of statistics (2001) “income imputation census fact sheet”, url: http://www.abs.gov.au, accessed 10 dec 2002. ajis vol 13, no. 1 september 2005 18 australian bureau of statistics (2002a), “2001 census (special tabulation)”, canberra: abs. australian bureau of statistics, (2002b) “cdata 2001; your census at work” (cdrom), canberra: abs. australian bureau of statistics (2003) “2001 census, basic community profile –time series profile category no. 2003.0, table 11”, canberra: abs. craig, a., fisher, j., scollary, a. and singh, m. (1998) “closing the gap women education and information technology courses in australia”, journal of systems software, vol 40, pp 7-15. department of employment, education and training [deet] (1990) “a fair chance for all: higher education that's within everyone's reach”, discussion paper, canberra: australian government publishing service department of education, science and training [dest] (2002) “job guide 2002”, url: http://jobguide.dest.gov.au/, accessed 13 dec 2002. department of education, science and training [dest] (2001), higher education equity scheme, url http://www.dest.gov.au/highered/programmes/heep.htm, accessed: 10 dec 2002. houghton, j. (2003) “australian ict industries update” (summary only), melbourne: centre for strategic economic studies, url: http://www.cfses.com/documents/ict_industries_update_2003_summary.pdf, accessed 31 july 2005. kirner, j. and raynor, m., (1999) the women’s power handbook, ringwood, australia: viking. maitland, a., (2001) “where’s the new it girl?”, financial times (22 february), p. 9. newmarch, e, taylor-steele, s and cumpston, a, (2000), “women in it – what are the barriers?”, detya, url: http://www.dest.gov.au/research/pubs/womeninit.htm, accessed: 25 january 2003. staehr, l., martin, m. and byrne, g. (2000) improving the retention rates of women in computing: an intervention program, journal of college student retention: research, theory and practice, vol 2, pp 127-140. von hellens, l. and nielsen, s., (2001) “australian women in it”, communications of the acm, vol 44, pp 46-52. acknowledgements the school of business and technology, la trobe university, bendigo provided financial support to purchase the census data. ajis vol. 9, no. 2 may 2002 using soft systems methodology to address supply chain management problems gulender gencoglu, graeme altmann, ross smith and david mackay school of management information systems faculty of business and law deakin university, geelong, victoria, australia 3217 abstract this paper reports an investigation of if, and how, soft systems methodology (ssm) might.be used to facilitate better management of industry supply chains. in two workshops involving supply chain managers from the textile clothing and footwear (tcp) industry and industry facilitators, ways in which ssm techniques might supplement existing supply chain management (scm) workshop approaches have been explored. specifically, the placement of ssm techniques within a workshop setting, reactions to the techniques, perceived reasons for using ssm, together with strengths and difficulties encountered, have been examined. keywords: soft systems methodology (ssm), supply chain management (scm) introduction in the world of ecommerce, supply chain management (scm) is now mission critical; get it right and a business prospers, get it wrong and it can threaten the very survival of the organisation. scm involves both optimisation of the delivery of goods and services, and optimisation of information flows. to the customer, optimisation means that the supplier knows what the customer needs and understands the timing of the delivery of goods and/or services. to the supplier, optimisation means that the right goods and/or services are available in the right quantities at the right time, when the customer needs them, without requiring the supplier to carry excess inventory or maintain excessive production capacity. interest in scm has grown steadily since the 1980's, as firms have come to recognise the benefits to be gained from collaborative relationships within and beyond their own organisations. it has been realised that it is not possible for companies to effectively compete in a climate of isolation from their suppliers and other key stakeholders; cooperation brings success for all (lummus and vokurka 1999). the role of technology as an enabler of scm has facilitated the initial development of relationships between organisations, however technology alone is not sufficient. businesses are volatile and their operation is dependent upon a chain of cooperating people. researchers are really now only in the early stages of understanding and modeling the social, cultural and political dimensions of the supply chain, and in particular, understanding how the people focused challenges of managing the supply chain might best be addressed. it remains an open question how supply chain managers might accommodate various stakeholder perspectives of supply chain problems, the cultural, social and political forces at play, the diversity of individual skill and knowledge bases, and diverse individual value systems. in this paper we report an investigation of if, and how, an existing systems-based problem solving approach, soft systems methodology (ssm), might be used to facilitate better management of industry supply chains. in two workshops involving supply chain managers from the textile clothing and footwear (tcp) industry and industry facilitators, ways in which ssm techniques might supplement existing supply chain management (scm) workshop approaches have been explored. specifically, the placement of ssm techniques within a workshop setting, reactions to the techniques, perceived reasons for using ssm, together with strengths and difficulties encountered, have been examined. initially in this paper, the scene is set with a brief review of some areas of scm concern, and recent trends. ssm is then briefly introduced, and the objective of the present research program is established. the research approach is then outlined, including a brief background to the workshop participants and their situation. some of the key results are then outlined, including insights into the way in which ssm might be integrated with an existing scm workshop technique used for some years now by the quick response (qr) group within business victoria. some of the perceived strengths of the use of select ssm techniques in this context are reported. finally, some possible future paths for this research are discussed. supply chain management—areas of concern and trends areas of current scm concern have been highlighted by hakanson (1999), and include: • improving customer service having the right product at the right place at the right time to reduce the number of people purchasing elsewhere; 49 aj1s vol. 9, no. 2 may 2002 • saving money and reducing costs; and • achieving better cash utilisation the time it takes to be paid for products delivered. according to li (2000), recent trends in supply chain management are now pushing companies to more diverse products, shorter product life cycles, increased competition and ever-increasing globalisation. these trends challenge companies seeking to achieve supply chain efficiencies. li argues that in order to address such challenges stakeholders in the supply chain must convene regularly to discuss matters of mutual concern. one might observe that the above concerns and trends are but symptoms of a fundamental scm principle. forger (2000) has suggested that what makes one supply chain better than another is collaboration. collaboration allows companies to predict supply chain events, as well as permitting suppliers and customers to plan around potential glitches while maximizing their return on supply chain opportunities. further, the impact of collaboration is not limited to improvements in the inter-organisational chain, but can have profound effects within the four walls of each company making up that chain. in summary, examination of the scm literature reveals difficulties with the lack of communication between supply chain partners, cultural inconsistencies, and difficulties with the integration of systems between key stakeholders in the supply chain. in short, if one is to take collaboration seriously, to achieve the full potential of a supply chain, one must be able to analyse the social, cultural, and political dimensions of a scm situation. soft systems methodology it is beyond the scope of the present paper to review soft systems methodology (ssm) (see checkland and scholes (1990) and papers therein). suffice to say that ssm has been widely used as a general systems-based problem-solving framework but little has been written on its application to scm problems, although rigby et al. (2000) makes oblique reference to some perceived shortcomings of ssm when describing research undertaken in the field of agile supply chains (see also day (1999)). developed in the uk and used around the world, ssm provides a set of procedures and notations for investigating organisational problem situations and taking action to improve those situations. ssm has a welldocumented set of processes, supported by a rich set of techniques for finding out about problems and generating systems-based models that can be used to generate systemically desirable and culturally feasible organisational change. even though ssm has not been widely applied to addressing problems in supply chain management, the very nature of ssm appears to make it a suitable vehicle to address the social, cultural, and political issues commonly associated with scm problems. what is evident from a search of the scm literature is that scm problems of a human nature are overwhelmingly more common than problems of a technical nature. in this sense, organisations are social systems, as well as business systems thus the social construction of problematic situations in organisations are complex to understand and manage. research objectives reflecting the above observations based upon the extant literature, the aim of this study is to address the research question: can ssm be used to complement an existing workshop-based approach for thinking about supply chain management problems by conceptualizing and modeling the key activities of suppliers, retailers and other stakeholders? as such, can ssm techniques such as rich pictures, catwoe analysis, root definitions and conceptual models be used to reveal issues that were previously unrecognized by those involved in the management of supply chains? research approach given the exploratory nature of the present research, a case study approach has been used to address the research question. yin (1985, cited by rubin and babbie (1997)) defines a case study as an empirical inquiry that: 'investigates a contemporary phenomenon within its real-life context; when the boundaries between phenomenon and context are not clearly evident; and in which multiple sources of evidence are used'. participants in the present case study were a senior supply chain manager, a production planner, a scm facilitator and two ssm facilitators. two half-day workshops were used to collect the data. the scm facilitator was from business victoria, a victorian government initiative for improving supply chain management processes within the textile clothing and footwear industry (tcp) industry. the facilitator had extensive experience in running scm workshops but had no knowledge of ssm. the senior supply chain manager and the 50 ajis vol. 9, no. 2 may 2002 production planner were from a large australian company in the tcp industry, specifically a producer of socks and hosiery. as a producer of socks and hosiery, the organisation manufactures locally as well as importing products that can no longer be cost effectively produced within australia. the company purchases yarn from local producers and imports yarns. the company sells to the major retail discount stores such as woolworths, coles myer and big w. they also concentrate on selling their own brands in smaller boutique stores around australia and are looking at the prospect of exporting niche products, particularly by using merino wool because it can be marketed as a unique australian fiber. the company had been previously involved in meetings initiated and facilitated by a victorian government scm facilitator who had spent a considerable amount of time working with tcp companies, forming cluster groups. those that made up the cluster groups were typically a major retailer, a tcp manufacturer and one or two suppliers to the manufacturer. the aim of convening the cluster groups was to achieve mutual gain by improving their total effectiveness in servicing each other and the end consumer. the format of these earlier meetings was an approach outlined below (see the next section of this paper). it was against the experience of participants in those previous workshops that select ssm concepts and techniques were introduced in the two half-day workshops conducted as part of the present research. data collection from the two ssm-based workshops took the form of: • some pre and post workshop semi-structured interviews with some of the workshop participants; • drawings and written responses to questions asked in the workshop situation; and • audio recordings (each workshop was audio recorded with the permission of participants). data from the above collected sources were analysed to: • document supply chain management processes currently employed by workshop participants; • identify scm issues and problems; • identify initial reactions to the ssm techniques introduced in the sessions; and • collect reactions to the experience of using ssm techniques in the workshops. the integration of ssm with existing scm workshop approaches as indicated above, the workshops discussed in this paper built on previous workshops that had been run over several years by the quick response (qr) group within business victoria. this qr program has been documented by perry et al. (2000) and is summarised on the left-hand side of figure 1. specifically, in these workshops senior managers meet regularly in newly formed cluster groups consisting of one or two representatives of major retailers, manufacturers and suppliers. the processes involved in the qr workshops can be summarised, as shown on the left-hand side of figure 1, as involving four key features: 1. the generation and circulation of company scm 'wish lists' within each cluster group; 2. discussion of the wish lists and the development of group and company action plans; 3. the injection into each group of a consultant selected by the various companies to provide assistance with in-house quick response implementation activities; and 4. follow-up strategic and tactical planning amongst each cluster group. results from these qr workshops (perry et al. 2000) reveal that unproved communication and information flow result, as well as improved performance within the supply chains involved. prior to the workshops discussed in this paper the present authors speculated on points at which ssm techniques might be injected into the qr workshop process model of perry et al. (2000). the initial view that was formed is represented by the additional steps 6a,b and 7a,b shown on the right of figure 1. specifically, it was felt that the ssm technique of rich picture building and reflection upon those rich pictures might facilitate the preparation of the scm 'wish list' (ie. steps 6a,b). further it was felt that the use of catwoe analysis, ssm root definitions and conceptual modeling of each node in the supply chain (ie. a "bottom up" development of conceptual models of each node in the supply chain) might provide a useful means of facilitating the identification and specification of in-house improvements (ie. steps 7a,b). this possible use of ssm in conjunction with the established qr workshop process model described by perry et al. (2000) was introduced to workshop participants, and reactions were collected. results and findings from these workshops follow. 51 ajis vol.9, no. 2 may 2002 results the placement of ssm techniques within the workshop before looking at the reactions of participants to the individual ssm techniques, it is worth reporting the reactions of those participants to the suggested placement of those techniques within the workshop process. in summary, significant doubt emerged from the participants concerning the suggested placement outlined in figure 1. instead, a preferred sequencing, outlined in figure 2, emerged. 1. quick response briefing/education 2. quick response readiness audit at 3. support and involvement of top-level management 4. formation of supply chain partnership 5. appointment of in-house quick response representative or champion 6. selection of a neutral quick response facilitator i 7. dissemination of 'wish lists' l 8. selection of a quick response consultant for in-house improvements 9. development of an effective supply chain oartnershid culture 10. regular supply chain workshops/meetings 11. strategic planning and 12. monitoring of action plans 6a. preparation of rich pictures. 7a. bottom-up conceptual model figure 1: the qr workshop process model of perry et al. (2000) (left side), with additional possible points for the inclusion of ssm techniques (right side) 52 ajis vol. 9, no. 2 may 2002 1. quick response briefing/education 2. quick response readiness audit at 3. support and involvement of top-level management 4. formation of supply chain partnership 5. appointment of in-house quick response representative or champion 6. selection of a neutral quick response facilitator 7. dissemination of 'wish lists' 8. selection of a quick response consultant for in-house improvements 9. development of an effective supply chain partnership culture 10. regular supply chain workshops/meetings 11. strategic ^ . . . ,^ planning and w loa. preparation of rich pictures lob. top down approach to conceptual model loc. catwoe serving as a mission statement. 12. monitoring of action plans figure 2: the qr workshop process model of perry et al. (2000) (left side), with the placement of additional ssm techniques preferred by workshop participants (right side) in summary, the participants argued that: • the scm 'wish list' generation step did not require support from ssm rich picture building (see further discussion under 'workshop techniques' below). instead, the wish list approach had the advantage of yielding quickly high value insights that build important enthusiasm for the workshops within participants. use of graphic ssm rich picture building and reflection at this very early stage in the life of the workshop group was seen as unnecessarily delaying the realisation of these quick returns. instead the workshop participants argued that the rich picture techniques would be best introduced subsequently, as part of the cycle of regular supply chain workshops/meetings, where it might be used as a technique to identify some of the less readily apparent scm issues and concerns (figure 2, step loa). • ssm catwoe analysis and conceptual model building was seen also as best introduced as part of the subsequent cycle of regular supply chain workshops/meetings (figure 2, step lob). further, it was believed that the value of the conceptual models came from examining the supply chain from 53 ajis vol.9, no. 2 may 2002 a top-down perspective, rather than the bottom-up approach proposed in figure 1. the catwoe technique was seen also as a useful means of reviewing/reasoning about the mission of supply chain participants (figure 2, step 1 oc). workshop techniques as noted above, the workshop participants strongly supported the retention of the use of wish lists, a central component of the previous qr workshops. these were seen as a fast way of focussing workshop participants thinking on the main supply chain issues and problems, yielding quick returns to the participants. the wish lists typically contain requests for improved working relationships between partners, improved response to customer orders and reduced pipeline waste and delay. in a sense, the wish lists were simply lists of things participants would ideally like to happen in the supply chain in order to improve their own performance and profitability. it was observed that using wish lists seemed to 'break the ice' with participants. in the course of the workshop, when the scm facilitator asked the senior supply chain manager to come up with a wish list, as many as 15 'problems' were produced in a manner of minutes. the qr facilitator stated reasons as to the usefulness of the wish list approach. they were: • it gives people ownership, therefore it's not just the facilitator coming in and stating that he is going to rework the supply chain; • it is the day-to-day things that people are anxious to fix; • the senior supply chain managers know intimately all the details, thus the solutions are usually fairly readily achieved; and • it leads the people in the workshop to realise that problems can often be solved by a simple alteration to a requirement or a procedure in another cell. this is not to say that the ssm techniques, when introduced to the workshop participants, were not appreciated. indeed, the scm facilitator commented on the fact that the use of ssm substantially added to the wish list approach: 'after deriving wish lists you end up with informal results. what you want to do then is to add structure, ...now ssm techniques can be used to add that structure.' what did emerge was that it was felt that the ssm techniques were better introduced after the quick gains achieved by using the wish list approach had been realised. in this regard it is worth noting that the participants observed that it was not a matter of whether it would be useful to apply ssm to scm, rather, where and how ssm could be applied. the collected observations of participants on their exposure to the ssm rich picture building, catwoe and conceptual modeling techniques are recorded as reasons for using ssm (table 1), identified strengths (table 2) and difficulties raised (table 3). it is beyond the scope of the present paper to discuss each of these in detail. suffice to say that they provide support for the use of ssm in this context, and some insight into potential difficulties that may need to be addressed in future ssm-enabled scm workshop sessions. reason acknowledges different ideas and perspectives promotes understanding of scm concepts prompts questioning, thinking, and critical analysis encourages formal consideration of political and social environment provides a disciplined, formal way to proceed in an unstructured situation iterative nature of the approach table 1: reasons cited by participants for using ssm in the scm context 54 ajis vol. 9, no. 2 may 2002 strength conceptual nature of the approach adds structure to wish lists focuses on the purpose of the chain, systems, needs, outputs ... practical promotes creativity and builds confidence provides discipline and a formal structure systematic nature of the approach able to deal with unstructured situations participative and collaborative in nature prompts thinking allows for questioning of the current supply chain arrangements takes account of social, political and cultural issues table 2: strengths of ssm in the scm context cited by participants difficulty learning how to begin to draw rich pictures problems in reaching group consensus learning how to represent human activities using ssm conceptual models table 3: difficulties encountered in the use ssm in the scm context cited by participants discussion and conclusions drawing upon the results reported above, four central themes related specifically to the introduction of ssm techniques and associated concepts have emerged from the research to date: • it appears that ssm allowed participants to achieve a better understanding of supply chain management processes because it introduces techniques that allow them to quickly see the views and perspectives of other stakeholders. participants easily take to the use of rich pictures to visually show supply chain processes. rich pictures accommodate different peoples' image of the chain, and therefore eliminate pressures on the manager to come up with verbal descriptions of systems and processes. • when an approach involving using catwoe analysis as a tool for the development of root definitions or scm 'mission statements' was canvassed, participants felt that by incorporating the elements of customers, actors, transformation, worldview, ownership and environment, they could see and reanalyze the cooperative role of the supply chain. • participants saw that the value of conceptual models in the scm context comes from examining the supply chain from a top-down perspective. rather than beginning with each participant in a supply chain developing their own activity model and capturing their perceptions of the current sequence of events, it was argued that it was better to start at the top-level and develop fresh views in order to overcome the danger of locking oneself into current practices. • ssm techniques were seen as generally useful in identifying shared and conflicting views. by having the parties come together in a workshop situation they are able to discuss key processes through the use of universal icons and remove those icons that are not appropriate. this allows inhibitors to a smooth functioning supply chain to be identified so that they can be removed. it should be noted, however, that the experience reported in this paper is limited to two workshop sessions and associated interactions with the workshop participants. it is therefore important that the findings reported be confirmed by the conduct of a future, richer series of such sessions, with participants drawn across all tiers in supply chains, operating in a number of industrial settings. results to date, however, are promising. references checkland, p. & scholes, j. (1990). soft systems methodology in action, chichester: john wiley & sons. day, m. (1999). buyer-supplier integration in the u.k. tableware supply chain. unpublished phd thesis, university of keele. forger, g. (2000). collaboration-the supply chain's defining factor? supply chain management review. 55 aj1s vol. 9, no. 2 may 2002 hakanson, b. (1999). supply chain management: where today's business competes, ascet. li, y. (2000). supply chain update, grainger center for supply chain management. lummus, r.r. & vokurka, r.j. (1999). defining supply chain management. industrial management & data systems. perry, m. & sohal, a.s. (2000). quick response practices and technologies in developing supply chains a case study. international journal of physical distribution & logistics management 30(7/8), pp. 627639. rigby, c., day, m., forrester, p. & burnett, j. (2000). agile supply: rethinking systems thinking, systems practice. international journal of agile management systems, 2/3, pp. 178-186. rubin, a. & babbie, e. (1997). research methods for social work, brooks/cole publishing company, u.s.a. 56 microsoft word ajisfinal.doc ajis volume 12 no. 2 may 2005 3 editorial this is the second issue of volume 12 of the australasian journal of information systems. our first paper is presented by dr v. saxena of babasaheb bhimrao ambedkar university and mr m darbari of babu banarasi das national institute of technology. their paper is entitled a diagrammatic tool for representing user interaction in uml. uids have proven to be a valuable tool to gather requirements since they describe the exchange of information between the system and the user in a high level of abstraction, without considering specific user interface aspects and design details as in other uml diagrams. this paper presents a diagrammatic tool to represent the users/system interaction called user interaction diagram (uid). chris phillips, elizabeth a. kemp and duncan hedderley from massey university, present our next paper entitled software development methods and tools: a new zealand study. the focus of this study is on the methods and tools used by software developers and the factors affecting the use and take-up of those tools in the workplace. tony lewis & christopher bell, from monash university present our next paper entitled the effect of gst on farm management information systems and business management skills. this paper reports the results of a survey that describes the characteristics of dairy farm management information systems and indicates the business management skills that dairy farmers perceive they need to acquire in the short term in order to improve their farm management information systems and comply with their gst requirements. our next paper is entitled an integrated management approach to leveraging knowledge innovation and is presented by andrew goh from the university of south australia. this paper discusses the significance of knowledge by describing the transition from “information revolution” to “knowledge revolution” – which enables various forms of knowledge to be absorbed, assimilated, shared and utilised. gillian whitehouse and chris diamond from the university of queensland present our next paper entitled “hybrids” and the gendering of computing jobs in australia. this paper presents recent australian evidence on the extent to which women are entering “hybrid” computing jobs combining technical and communication or “people management” skills, and the way these skill combinations are valued at organisational level. supriya singh, kylie cassar bartolo and christine satchell from rmit university/smart internet technology cooperative research centre present our next paper entitled grounded theory and user requirements: a challenge for qualitative research. this paper discusses the methodological implications of moving from grounded theory to user requirements for the design of information and communication technologies. our next paper is presented by jennifer patashnick and michael rich, children’s hospital boston and is entitled researching human experience: video intervention/prevention assessment (via). this paper examines the value of qualitative inquiry and why human experience rarely is investigated directly. an analysis of a sample via data is presented to demonstrate how, by utilizing grounded theory and qualitative analysis, one can derive rich and unique information from human experience. leoni warne, defence science and technology organisation, celina pascoe, university of canberra and irena ali, department of defence present our next paper entitled painting with and without numbers: the use of qualitative and quantitative methods to study social learning. this paper describes the process by which the qualitative methods in this study of knowledge processes were expanded to include quantitative methods. it focuses on how this combination of data collection methods evolved, and the ways in which it was capitalised on to provide a much more enriched set of findings than would have been the case if qualitative or quantitative methods had been used alone. ajis volume 12 no. 2 may 2005 4 bruce rowlands, griffith university presents our next paper entitled grounded theorising applied to is research — developing a coding strategy. this paper provides an example of developing a coding strategy to build theory of the roles of methods in is development. the research seeks to identify and understand how system development methods are used in an is department within a large australian bank. our next paper is presented by david wilton, massey university. it is entitled a tale of two projects: why it projects fail (and why they sometimes actually succeed). this paper describes two similar outsourced is projects that took place in wellington, new zealand (nz) during the period 1992-99, with emphasis on the reasons why one project failed (the prime contractor repudiated the contract and the project was abandoned) and why the other succeeded. mary tate, beverley hope and brent coker, victoria, university of wellington present our next paper entitled the buywell way: seven essential practices of a highly successful multi-channel etailer. this paper examined multi-channel retailing with the aim of building on existing theory in multi-channel e-commerce. susan keller and ross smith, deakin university and steve howard and jennie carroll, university of melbourne present the next paper entitled altered images: the relations between design representations and design practice. this paper reports on an extensive design case study that aims at increasing understanding of the nature and affordances of representations in the design process and argues for the need for information systems as a discipline to open up discussion of the design representations that may be required to effectively design systems that mix traditional is with disciplines such as industrial design, architecture and fashion design. david arnott, monash university, graham pervan, curtin university and gemma dodson, monash university present our next paper entitled a descriptive analysis of decision support systems research between 1990 and 2003. this paper is the first major report of a project that is investigating the theoretic foundations of decision support systems (dss). the findings indicate that dss research is more dominated by positivist research than general information systems (in particular experiments, surveys, and descriptions of specific applications and systems), is heavily influenced by the work of herbert simon, is poorly grounded in contemporary judgement and decision-making research, and falls down in the identification of the nature of clients and users. our final paper is presented by peter marshall, judy mckay, university of tasmania and adi prananto, monash university and is entitled business value creation from it investments: towards a process theory of it. a number of models have been put forward to describe the relationship between it expenditure and business value. this paper considers one such model, the process model of soh and markus, and examines its features and deficiencies. i would like to thank ms cathy nicastrti for her assistance in preparing this issue of the journal. dr rob macgregor editor microsoft word ajisfinal 12 2 marina.doc ajis volume 12 no. 2 may 2005 112 painting with and without numbers: the use of qualitative and quantitative methods to study social learning leoni warne defence science and technology organisation fern hill park department of defence canberra act 2600 phone +61 2 6256 6219 fax +61 2 6256 6233 email: leoni.warne@dsto.defence.gov.au celina pascoe school of information management and tourism university of canberra act 2601 phone +61 2 6201 2912 fax +61 2 6256 6233 email celina.pascoe@canberra.edu.au irena ali defence science and technology organisation fern hill park department of defence canberra act 2600 phone +61 2 6256 6252 fax +61 2 6256 6233 email irena.ali@dsto.defence.gov.au introduction the enterprise social learning architecture (esla) team of the defence science and technology organisation (dsto), conducted a four-year research study investigating social learning within the australian defence organisation (ado). the immediate aim of this research was to understand the issues inherent in building learning, adaptive and sustainable systems. the long-term objective was to develop architectures that would support the development of information systems to guide and enhance organisational learning and facilitate knowledge management. in this paper we will discuss the methodologies used by the esla team to gain understanding into effective social learning and the organisational and cultural factors that support such learning. also, the paper will discuss the lessons learned from methodological approaches to this study as well as support tools used to analyse large volumes of qualitative data. there has been an increasing emphasis in the past decade on investigating the social and organisational factors that may underpin successful information system development and usage (butterfield and pendegraft, 1996; davenport and prusak, 1992; delone and mclean, 1992). investigation of these issues necessitates a sound understanding of organisational culture, human social interactions, communication and relationships, and reflects an increasing awareness of the importance of the social aspects of socio-technical systems that people work and operate in. this paper describes the process by which the qualitative methods in this study of knowledge processes were expanded to include quantitative methods. it focuses on how this combination of data collection methods evolved, and the ways in which it was capitalised on to provide a much more enriched set of findings than would have been the case if qualitative or quantitative methods had been used alone. the paper also focuses on pitfalls that arose in the use of the various methods, including those associated with the use of software for qualitative data, and the means by which a successful methodological blend can be achieved with research relating to knowledge and its management. ajis volume 12 no. 2 may 2005 113 background to the study social learning the study investigated social learning in a range of settings within the australian defence organisation (ado). social learning is usually associated with albert bandura (1977), and emphasises the vital role of observing and modelling the behaviours, attitudes, and emotional reactions of others in the learning process. like lave and wenger’s (1991) situated learning and vygotsky’s (1978) social development theory, social learning considers social interaction to be at the heart of learning. for lave and wenger, social interaction is critical to situated learning because the learning process is seen as unintentional rather than deliberate, and as a function of the activity, context, and culture in which it occurs. similarly, for vygotsky, social interaction plays a fundamental role in the development of cognition because the observational and imitation aspects of learning particular behaviours precede their incorporation into the individual’s personal repertoire. with its emphasis on attention, memory, and motivation, social learning spans both these behavioural and cognitive frameworks of learning. this study adds to the work of bandura and others by investigating adult social learning in organisations. informal, activity-based learning is inherent to all human activities. it is this capacity to learn from one another that leads to the advancement of the human race. workplaces are full of learning opportunities and in work life, socially based learning is occurring all the time. interaction occurs between peers, genders, functional groups and ages, and across hierarchies and it happens in ways not normally recognised as learning. it is through learning that we see ourselves in a different context and this transformation of oneself through learning is particularly important (jordan, 1996). information systems can and should be developed to facilitate this process, though many systems developed in the past have been shown to be so prescriptive they can inhibit learning and stifle innovation. for the purpose of the esla research, social learning is defined as learning occurring in or by a group, an organisation, or any cultural cluster and includes: the procedures by which knowledge and practice are transmitted across posting cycles, across different work situations and across time; and the procedures that facilitate generative learning – learning that enhances the enterprise’s ability to adjust to dynamic and unexpected situations and to react creatively to them. the use of the ‘social’ in learning reflects that organisations, organisational units, and work groups are in fact social clusters as are study groups and task groups and thus learning occurs in a social context. it is therefore, necessary to examine how individuals learn in an organisational context and what it means for an individual to learn, to understand how people work most effectively together and what their support requirements are. individual learning is an important component of social learning. however, this study focused primarily on the social context and cumulative effects of learning, rather than on individual learning alone. description of study settings the esla team completed three studies into social learning processes. the first of which was a pilot study. the pilot study was conducted over a six-month period in 1998 in the tactical environment at an air base. it involved five field trips. the pilot study had two purposes: firstly to see if it was feasible to observe, understand and document social learning processes, particularly in command and control situations, and secondly to trial the use of ethnographic techniques for this purpose. ajis volume 12 no. 2 may 2005 114 the second study took place in a strategic environment, at the australian defence headquarters (adhq). that study initially involved one branch of the adhq and commenced in june 1999. after three months, this study was extended to a fuller research study at the strategic headquarters. that setting was of particular interest as its personnel were distributed across different geographical locations, different services, and different functional branches where work outcomes were heavily reliant on the prevailing economic and political climate, and a number of information systems were already in use. this contrasted with the tactical headquarters where the environment was very structured. the work environment in the strategic headquarters lacked the same degree of structure and had a much higher degree of ambiguity. the third setting for the study took place in a single service strategic headquarters and the focus was to find out whether social learning constructs differ in a single service strategic headquarters as opposed to a joint service environment. study methodologies the following section describes the methodological approaches to the research involving qualitative and quantitative methods. the methodological approach was initially qualitative only, but it evolved to include quantitative aspects. the qualitative methods comprised ethnographic observations and semi-structured interviews, whereas the quantitative method consisted of a survey. qualitative methods ethnographic observations the methodology employed in all phases of the study was based on ethnography. researchers are increasingly employing qualitative methods, specifically ethnography, to gain an understanding of social, organisational and information systems interactions (myers, 1999). ethnographers acquire data that provides a rich context because they do so by immersing themselves in the situation – this enables them to gradually see and understand the key concepts that influence the setting. the research team used ethnographic techniques in the form of fieldwork, which entailed observing the work taking place in different settings, and using directed questioning to clarify issues. according to harvey and myers (1995), ethnography is ideal for providing information systems researchers with rich insights into the human, social and organisational aspects of information system development and implementation. it is appropriate where a key aspect of the research is to analyse, or at least take into consideration, various aspects of the context and the social process. given the exploratory nature of the pilot studies, as well as the importance of context in understanding the social process of learning, ethnography was deemed the most suitable methodological tool. the advantages of ethnography for this particular study are depicted in the following diagram (figure 1): ajis volume 12 no. 2 may 2005 115 figure 1 benefits of ethnography as a data collection method prior to the commencement of the research study, the original team members were thoroughly briefed on the principles and ethics of ethnographic research by gitte jordan and bill clancy (then from the institute of research and learning associated with xerox parc) who helped to popularise the use of ethnography in industrial settings. like every research method, ethnography has some limitations. it is time consuming, both in terms of fieldwork, writing up of fieldnotes and in their analysis. it does not offer much breadth as only one organisation or one culture is studied at a time. the esla team addressed this issue by conducting research in a variety of settings. with ethnography there is also a danger of influencing the subject’s behaviour, and, furthermore, according to atteslander cited in fredrichs, (1975, p26) “we only believe what we see; unfortunately we only see what we want to believe”. in order to overcome this constraint, to better understand the setting under study and to gain insights into the social processes of learning, the esla team used orientation as an initial step in the strategic settings. during this stage, no data was gathered but researchers sat in on meetings, work areas, gathered documentation, and generally made themselves an occasional part of the work place. only after it appeared that the researchers’ presence was no longer regarded as an unusual distraction, did observations begin. the issues were further addressed by working in teams, whenever possible, of two researchers from different professional backgrounds and specialties. the multidisciplinary composition of the esla team shaped the type of ethnography that took place, as well as providing a richer picture of the setting under study. for most of the study, the team consisted of four researchers: one social scientist, one interpersonal immersion in setting being studied observational field-work observing work in different settings directed questioning to clarify issues richer picture of setting being studied observe day-to-day work practices (actual vs perceived) can see key concepts influencing that setting evidence/information provided to challenge assumptions capture of insiders’ perspectives ethnography ajis volume 12 no. 2 may 2005 116 communication specialist, one information management/information seeking specialist, and one researcher from an information systems/organisational studies background. the multidisciplinary nature of the team meant that the data collection and analysis was enriched by the members’ different perspectives, expertise and experiences, thus providing reliability for the findings. team meetings were held regularly to corroborate and consolidate the findings as the work unfolded, and to identify emerging key social learning issues. due to the nature of the tactical environment, ethnography proved to be a very successful research and data gathering tool in that first setting. however, that setting was the only place where the esla team felt that ethnographic techniques alone were sufficient and timely enough. semi-structured interviews in addition to ethnographic observations, the esla team undertook extensive, semi-structured interviews with a sample of staff in the two strategic settings. a stratified sampling technique was used to ensure that an adequate representation was achieved. the specific characteristics of interest were branch and directorate affiliation, gender, rank, whether military or civilian, work location, and duration of placement. in total, 15 interviews were conducted in the joint strategic setting and 48 at the more populous (about 115 staff) single service strategic environment. the duration of the interviews ranged between an hour and an hour and a half. the semi-structured interviews proved to be a very useful tool for capturing perceptions about the work-place, and in some instances, was clearly cathartic for interviewees who felt that their opinions were not sought or valued in their immediate environment. while there was a consistent set of topics covered in each interview to guarantee uniformity of data across the whole sample, each interviewee was also given full rein to discuss what they perceived to be related topics, or other burning issues, so that new concerns emerged and the researchers were able to build on their understanding of the setting being studied. the following set of topics was covered by the semistructured interviews: personal background and career history, familiarity with duty statements and job expectations, job handovers, match between personal knowledge / skills and knowledge / skills required for the position, on-the-job learning, information seeking strategies, communication climate, networking, records management, corporate knowledge, work environment / physical location, gender issues, and other issues that emerged during the interviews. the interviews were transcribed and coded according to a thesaurus of terms developed by the esla team. these coded interviews and the coded field notes from observations were entered into the qualitative analysis database n’vivo and were subsequently rigorously examined and crosschecked against the quantitative results. quantitative the strategic settings under study were characterised by a highly politicised and unstructured climate and the areas relied on input both from internal parties and from outside agencies. furthermore, the outcomes of work were dependent on the vagaries of the economic and political climates. in the first of these settings, the esla researchers encountered a new problem. the personnel being observed sometimes interpreted occasional passing comments made by the ethnographer as statements reflecting the research findings. in one instance, one of the ethnographers commented to a director that ethnographic work in his area had slowed because his subordinates were so busy they were rarely at their desks. the director’s response was to send an email to all his staff to say that “dsto says you attend too many meetings”. to counter such misinterpretations, a survey questionnaire was constructed and implemented. it served to move the ajis volume 12 no. 2 may 2005 117 participants’ attention from ‘what dsto thinks’ to ‘what staff member’s think’ (bailey, 1982; harvey and myers, 1995; myers 1999). however, the survey also served as a validation of the observational data. the first survey administered in the joint strategic setting consisted of three parts part a comprised forty-seven likert scale questions, several of which acted as consistency check questions; part b comprised six ‘open ended’ questions, inviting respondents to give a brief statement on each of them. these questions were prompted by, and further explored, the data collected through ethnographic fieldwork. finally, part c was designed to gather some demographic details about the respondents. similarly, the survey administered to staff of the single service strategic environment consisted of three parts, however, the questionnaire (part a) was more in depth and comprised ninety two questions; in part b respondents were asked to rank listed attributes (or add others if considered necessary); and part c gathered demographic data. this same survey (with minor changes to organisational acronyms) was also applied at the follow up visits to the tactical setting and the joint strategic environment, giving a quantitative means of comparison of all three settings. the response rate to the first survey in the joint strategic setting was 96.7%; for the single service strategic setting it was 73%; for the tactical setting, a much smaller sample, the response rate was 92% and a repeated survey in the joint strategic setting – applied to ensure findings were consistent over several years was 73%. the survey became a part of the methodological toolbox used by the esla team and also served to facilitate the comparison of results across different settings, and across time in the same setting. the combination of qualitative and quantitative techniques was facilitated by the use of a qualitative analysis software tool – n’vivo. n’vivo database n’vivo is a database designed for analysis, storage and retrieval of qualitative data and includes a facility for reducing qualitative data to a quantitative matrix. the esla team used n’vivo version 2.0 which allows users to edit, annotate, code and link data as well as import rich text format documents and link directly to multimedia, document files and websites external to the database. this facilitates the indexing and co-location of multiple sources of data producing a rich picture of the setting under study. by using n’vivo, the researchers had at their disposal a wide range of protocols for linking data and ideas, and combining processes as the data and the method required, allowing findings to emerge logically. the document browser facility is a rich text editor, a coder and a coding viewer. as concepts were developed, they were gathered in a flexible system of nodes, and the node browser displayed the coded material as required. the coding in n’vivo was done by dragging and dropping selected text to a previously devised node(s) or node(s), or for those members of the team that were not co-located, by marking up transcripts that were later added into n’vivo by a research assistant. coding could also be done automatically by section coding or using the search tool, however, this facility was not used by the researchers. to analyse and explore relationships in the data the esla team used n’vivo’s integrated show tools. these integrated tools support qualitative searching and the program allows users to scope searches as specifically as required by filtering or combining search terms, nodes, attributes or text strings. searches could be expanded or refined, asking the same question of a different combination of data or a new question of the same material. the results of searches could be coded, so they could be built on to explore other questions if required. the esla research team underwent extensive training on the use of n’vivo, but still feel they have not managed to exploit its full potential. the tool has proved to be an invaluable analysis utility. ajis volume 12 no. 2 may 2005 118 lessons about methodologies and methodological tools the multidisciplinary nature of the team meant that there were, initially, multiple understandings about the most appropriate methodological approaches to the research. the pros and cons of various methodologies were vigorously debated before a consensus was reached. the lessons learned from this included: the desirability of having a personal and collective ‘faith’ in the approach to the research before beginning the work, and the need for ensuring that all team members have the knowledge and skill necessary to apply the chosen methodology, and furthermore, if a consensus of confidence in the work cannot be reached despite training, discussion and team building exercises, the team should be disbanded or dissenting members should move on to other areas of research. the research team was aware that they were engaged in work that was largely alien to the ‘hardscience’ approaches in dsto. in order to maintain the credibility of the research, it was important to ensure that the chosen methodology was rigorously applied and robust. to ensure this was the case, the team set out to: collaborate with universities where the chosen methodological approach was more commonly applied; and to expose the research to consistent peer review through presentation at conferences and publication in journals. while this proved to be a very successful tactic for re-enforcing confidence in the rigour of the research, it also resulted in a frequently excessive publishing workload. by raising the profile of the work, it also increased requests for presentations and collaboration which, in itself, also added to the team’s workload. upon reflection, it became evident that the team undertook what might be considered an overambitious approach. qualitative research is both time and labour intensive, and it is important not to over-burden it with nugatory processes, like over interviewing and over-coding, and in some cases, repetitive or non-constructive discussions about code terms or emerging findings. in the quest of conducting thorough research and validating all the data, the team probably conducted too many interviews, particularly at the single service strategic setting. data analysis indicated that, after a certain point, the team was not getting any new insights or richness of data and, as it turned out, the conducting of further interviews proved to be futile. while the sample of interviews conducted at the joint strategic setting was adequate at 22% of total staff, at single service strategic hq, 44% of the staff were interviewed. the lesson derived from this is that: a sample of 20-30% of total staff can be an adequate representation for semi—structured interviews. if the team had taken more time to reflect on the adequacy of the data already gathered this might not have occurred. another aspect of conducting qualitative research is to carefully think through and plan how the data is to be coded and subsequently retrieved. the coding of data, in particular, is a very time consuming process and the team did not pay enough attention to how the coding would impact on retrieval of information, concentrating instead on lengthy, detailed discussions on the nuances of code terms. this was largely due to their unfamiliarity with the qualitative software package n’vivo, but also their concern that some important finding might be overlooked. the end result of this concern was that there were too many code terms used and multiple code terms were applied to individual paragraphs, thus making the retrieved data very cumbersome. this was exacerbated by the fact that the team attended the n’vivo training course much too early, long before needing to ajis volume 12 no. 2 may 2005 119 apply it. therefore the software was not utilised to its full potential. lessons learnt here include: know your software packages before using them – schedule training for when it can be backed up by using it, and avoid over-coding –bearing in mind that the purpose of coding is the retrieval of data that is effectively categorised (and not too cumbersome to be meaningful). furthermore, the primary lesson reinforced here is the first rule of research which is violated by the inexperienced and experienced alike: know exactly how you are going to analyse your data before you collect it. conclusions the successful blend researchers frequently combine different methods of data collection when studying the same social issue in a process known as triangulation. this concept has been discussed intensively in the area of social and information systems research and is a recognised way of validating research data (silverman, 1985; miles & huberman, 1994). the esla research team was able to strengthen the key findings from the qualitative techniques by triangulating with quantitative data from the survey (kidder, 1981; bailey, 1982). surveys can provide an accurate description of a real world situation from a variety of perspectives (galliers, 1992) and the survey used by the esla team extended its understanding of the factors under investigation through the collective perceptions of the personnel involved. this blend of both qualitative and quantitative methods of data collection was used in all settings studied after the first pilot study. the combination of qualitative and quantitative research methods provides data and findings that are much richer than those derived from one of these methods alone. qualitative methods are commonly used as a first step in quantitative studies (tucker et al, 1995), and the esla team used observational data to formulate quantitative surveys as well as to focus researchers’ attention on pertinent issues that were further discussed in semi-structured interviews. furthermore, the combination of methods enabled complementary perspectives of each of the settings. the observations and interviews provided not only data that offered the insiders’ point of view but also shed a light on the unique aspects of the various social settings that were studied (pondy et al, 1983). on the other hand, the quantitative surveys enabled generalisations across the settings. it is this combination that has given the esla research strength and validity. not only was the research study triangulated by method (observations, interviews, and quantitative surveys), but additionally by data source (data had been collected at different times throughout the study and from varying settings). triangulation was further facilitated by the multi-disciplinary composition of the research team. the esla team took the opportunity to discuss the methodological approach to this study at numerous seminars and special interest group’s meetings. feedback from these was that the methodology was rigorous and the findings well triangulated and valid. the effectiveness of the methodology used by the esla team and the strength of its findings add weight to the view that the dichotomy of a qualitative-quantitative separation is now archaic (miles & huberman, 1994). it is hoped that the lessons learned from this study will be of benefit to those who are planning to undertake research using qualitative methods or a combination of qualitative and quantitative approaches. references atteslander cited in fredrichs, j. (1975) participant observation: theory and practice. franborough: lexington books, p 26. ajis volume 12 no. 2 may 2005 120 bailey, k. d. (1982) methods of social research. new york, free press. bandura, a. (1977) social learning theory. new york: general learning press. butterfield, j., & pendegraft, n. (1996) cultural analysis in is planning and management. journal of systems management 47: 14-17. davenport, t., eccles, r., & prusak, l. (1992) information politics. sloan management review 34(1): 53-65. delone, w. h., & mclean., e. r. (1992) information systems success: the quest for the dependent variable. information systems research 3: 60-95. galliers, r. d. (1992) choosing information systems research approaches. information systems research: issues, methods and practical guidelines. r. galliers. oxford, blackwell scientific. harvey, l., & myers, m. d. (1995) scholarship and practice: the contribution of ethnographic research methods to bridging the gap. information technology & people 8(3): 13-27. jordan, b. (1996) transforming ethnograthy reinventing research. groupware software. j. schiestl and h. schelle. marburg, germany, tectum verlag.: 200-212. kidder, l. h. (1981) research methods in social relationships. new york, holt, rinehart and winston. lave, j., & wenger, e. (1990) situated learning: legitimate periperal participation. cambridge, uk: cambridge university press. miles, m. b., & huberman, a. m. (1994) qualitative data analysis: an expanded sourcebook. london, sage. myers, m. d. (1999) investigating information systems with ethnographic research. communications for the association for information systems 2(23). pondy, l., frost, p., morgan, g., & dandridge., t. (1983) organisational symbolism. greenwich, ct, jai press. silverman, d. (1985) qualitative methods and sociology. aldershot, gower pub. co. tucker, m. l., sterkel, p. k., & meyer, g. d. (1995) qualitative research in business communication: a review and analysis. journal of business communication 32(4): 383400. vygotsky, l. s. (1978) mind in society. cambridge, ma: harvard university press. note: this paper is not an original work. it is based on a research report that the esla team prepared at the conclusion of the social learning study: warne, l. ali, i. pascoe, c. (2003) "social learning and knowledge management a journey through the australian defence organisation: the final report of the enterprise social learning architectures task", dsto rr 0257 ar 012 854, defence systems analysis division, dsto information sciences laboratories, south australia. microsoft word ajis vol13-2 o.doc 105 australasian journal of information systems volume 13 number 2 may 2006 an empirical study of relationships between traditional usability indexes and physiological data tao lin, atsumi imamiya, masaki omata, wanhua hu department of computer science and media engineering, university of yamanashi, takeda 4-3-11, kofu, yamanashi prefecture, 400-8511, japan email: lintao@hci.media.yamanashi.ac.jp abstract task performance data and subjective assessment data are widely used as usability measures in the human-computer interaction (hci) field. recently, physiology has also been explored as a metric for evaluating usability. however, it is not clear how physiological measures relate to traditional usability evaluation indexes. in this paper, we investigate the relationships among three kinds of data: task performance, subjective assessment and physiological measures. we found evidence that physiological data correlate with task performance data in a video game: with a decrease of the task performance level, the normalized galvanic skin response (gsr) increases. in addition, physiological data are mirrored in subjective reports assessing stress level. the research provides an initial step toward using physiology as a complementary or an independent usability measure for hci evaluation. introduction usability is implicit in a design and manifests itself through interaction with a product. it may be measured by the extent to which the product affords an effective and satisfying interaction to the intended users, performing the intended tasks at an acceptable cost. in traditional usability evaluation methods, task performance is measured as an important element, yet user cost is often neglected. performance-based measures usually use the end results to infer the continuous interaction process, but they can not reflect how tasks impact upon users physically. although users are able to complete many tasks at the same performance level, they still may have different opinions about the usability because of different user costs. some may feel almost no discomfort, while others report that they experienced considerable stress. therefore, user cost should also be considered as an important element in usability evaluation methods. in 1993, a study has proposed a human-computer interaction (hci) evaluation framework of task performance, user satisfaction and user cost (sweeney, maguire and shackel, 1993). recent studies assessing multimedia quality also revisited the framework (wilson and sasse, 2000a; wilson and sasse, 2000b). task performance and user satisfaction were employed to determine the levels of user effectiveness and satisfaction. the user cost referred to the level of user investment required to achieve and maintain high levels of the task performance and satisfaction, and it may be demonstrated in terms of the level of physiological and /or mental effort or stress incurred. at present, subjective approaches to measure the user cost have been proposed, but they have a fundamental problem. when used in isolation, they may not be reliable due to the cognitive mediation. for example, wilson and descamps reported that users rate video quality lower when performing a difficult task than when performing an easier one (wilson and descamps, 1996). this evidence suggests that contextual variables may influence users australasian journal of information systems volume 13 number 2 may 2006 106 assessment. knoche et al. also argued that subjective assessment is fundamentally flawed, as it is not possible for users to register what they do not consciously perceive (knoche, de meer, and kirsch, 1999). in addition, as the scale of an experiment becomes larger, accurate subjective results require a larger number of subjects and more time for analysis, with greater associated costs. as hci researchers, we recognize that task performance and subjective assessment are essential methods of usability evaluation, but they are not enough. an objective method for measuring the user cost should be integrated into the traditional usability evaluation framework. the ultimate goal of our research is to create a new usability evaluation method based on traditional usability evaluation indexes (task performance and subjective assessment) and physiological data for measuring user costs. the evidence from physiology has suggested that physiological measures (e.g., skin conductance, heart rate, pupil size, respiration, blood volume pulse) reflect reactions of autonomic nervous system (ans) and are therefore difficult to ‘fake’ (andreassi, 2000). physiologists have used these measures as objective identifiers of human emotions such as anger, grief and sadness (ekman, levenson and friesen, 1983), while researchers in human factors have used them to determine mental effort and stress (vicente, thornton and moray, 1987). moreover, recent improvements in technology have made it possible to provide continuous, high-resolution data sources for hci fields. physiological measures are being explored in several studies as an independent or dependent usability metric. they show potential, for example, in assessing multimedia quality and measuring presence in a virtual environment. we believe that physiological data will be a valuable complement to traditional usability evaluation measures. however, there is still a lack of investigation into relationships between physiological data and traditional usability indexes. this paper investigates relationships between task performance data, subjective data and physiological data. it is important to disentangle the relationships to testify the efficacy of physiological measures for hci evaluation and understand user states during interactions. it provides an initial step toward establishing a new usability evaluation method that uses physiology as a complementary measure or as an independent measure for hci evaluation. in addition, we examine physiological responses to frustrating events for clues to help explain the differences between physiological and task performance data. in our experiment, we created a video game environment to elicit task performance, subjective assessment and physiological data, and analyzed the links and correlations among them. stress and physiological measures in the study, stress was measured as user cost by physiological measures. historically, stress has been defined as a reaction from a calm state to an aroused state (cannon, 1927). the reactions are described as “fight or flight” not “fight or joy”. some researchers made distinction between “eustress” and “distress”, where eustress is a positive type of stress, such as joy, and distress is a negative type of stress (eisenhofer and goldstien, 1988). the definition of stress in the present study will only cover “distress” or stress with a negative bias. based on previous research, we chose the physiological properties of galvanic skin response (gsr), blood volume pulse (bvp) and heart rate (hr), because they can be measured non-invasively and are good indicators of arousal. the gsr signal is an indicator of skin conductance. in 1956, seyle linked gsr to stress and ans arousal (seyle, 1956). recent research also showed that skin conductance varies linearly with the overall level of arousal and increases with anxiety and stress (picard, 1997; healey, 2000), and can also reflect both emotional response and cognitive activity (boucsein, 1992). the bvp signal is an 107 australasian journal of information systems volume 13 number 2 may 2006 indicator of blood flow. bvp increases with negatively valenced emotions such as fear and anxiety, and decreases with relaxation (picard, 1997; healey, 2000). heart rate is also considered to be a good indicator of overall activity levels, with a high heart rate associated with an anxious state and a low rate with a relaxed state (frijda, 1986). heart rate was automatically calculated from bvp with the biograph software in our experiment. related studies on using physiology as a metric of hci evaluation several studies reported by wilson and sasse show a novel method for assessing multimedia quality in the context of networked applications: physiological responses to degradations in media quality (audio and video) are taken as an objective measure of user cost (wilson, 2001). they found significant increases in gsr and hr, and significant decreases in bvp for video shown at 5 frames per second versus 25 frames per second (wilson and sasse, 2000a), even though most subjects didn’t report noticing a difference in media quality. another main finding of this research is that subjective and physiological results do not always correlate with each other (wilson and sasse, 2000b). these discrepancies between physiological and subjective assessment support the argument for a 3-d approach to evaluating multimedia quality and other hci evaluation areas. ward et al. used several physiological measures (hr, bvp, and gsr) to assess users’ responses to well-designed and poorly designed web pages. no significant differences were found between users viewing the two types of web pages, in part due to large individual differences. however, distinct trends were seen between the groups when the data were normalized and plotted. participants using the poor interface showed higher levels of arousal (ward and marsden, 2003; ward, marsden, cahill and johnson, 2002). their study also provides an example of how physiological data can be fit into usability evaluation. meehan et al. used physiological measures (gsr, hr, skin temperature) to evaluate presence in stressful virtual environments. the experiments found that the change in hr can satisfy the requirements for a reliable, objective measure of presence, and that change in gsr does to a lesser extent; change in skin temperature does not (meehan, insko, whitton and brooks, 2002). in the domain of entertainment technology, an experiment was conducted to test the efficacy of physiological measures as evaluators of collaborative entertainment technology (mandryk and inkpen, 2004). their results suggest that there are different physiological responses when a user is playing against a computer than when playing against a friend. these results are mirrored in the subjective reports provided by the participants. ravaja et al. examined phasic psychophysiological responses indexing emotional valence (i.e., facial electromyography) and arousal (i.e., inter-beat intervals and gsr) to different game events in a video game and suggested that information on the emotion responses elicited by game events and event pattern may be applied in the game design (ravaja, saari, laarni, kallinen, and salminen, 2005). affective computing is one of the active areas in hci field, whose research are aimed at giving machines skills of emotional intelligence, including the ability to recognize, model, and understand human emotion, to appropriately communicate emotion, and to respond to it effectively. the new developments in the hci field are narrowing the gap between the human and the machine and machines are increasingly to sense, or infer user attributes. to the extent that distinct emotions prepare the organism for distinct behaviour (e.g. approach vs. avoid, and fight vs. flee at the most fundamental level), they ought to be reflected in distinct physiological signatures. this is the basis for using specific signatures along these signatures to recognize a particular affective state. while australasian journal of information systems volume 13 number 2 may 2006 108 debate continues regarding the specificity of these signatures for particular emotions, and the degree of affective differentiation possible, particularly when only ans signals are considered, some research has suggested that with sufficient data, appropriate baseline and normalizing procedures, and subsequent pattern recognition algorithms, it is possible to differentiate among a number of emotions (davidson and ekman, 1994; picard, vyzas and healey, 2001). scheirer et al. applied a pattern-recognition strategy known as hidden markov models to gsr and bvp data to detect states of frustration deliberately induced by a slow computer game interface (scheirer, fernandez, klein and picard, 2002). experiment design to obtain task performance, subjective and physiological data, and investigate their links or correlations, we chose a video game as the experimental task. participants were required to play a popular 3-d video game called super mario 64, which was manufactured by nintendo ® . the gsr data were collected at 64 hz; bvp and hr data at 128 hz. questionnaire data, including subjective assessment of stress, participants’ statistics and task performance data, were collected into a data file and analysed using spss 11.0 software. participants fourteen male and four female university students aged 19 to 31 participated in the experiment. before the experiment, all participants were required to fill out a background questionnaire about their experience with the game, average game times and personal information such as sex, age and handedness. nine of the 18 participants were experienced with the game, while the other nine were somewhat experienced or completely inexperienced. task in our experiment, participants were required to play three parts of super mario 64 in a random order as quickly and accurately as possible. task one was to run all regulated paths up to a mountain and defeat king bo-bomb, which took about 140 seconds for an average skilled player. task two was to pound a wooden post into the ground while avoiding an attack from a monster. it generally took 55 seconds to play that part of the game. in task three, participants were required to go through a snow slide without falling from it. this usually took about 50 seconds. screen shots of the three tasks are shown in figure 1. figure 1: screen shots of the experimental tasks (task one, task two and task three) participants played each task continuously for 10 minutes, regardless of results. that is, participants could repeat a task many times in ten minutes. in order to ensure consistent experimental conditions, game settings were not changed during the course of each experiment. 109 australasian journal of information systems volume 13 number 2 may 2006 experimental apparatus and protocol the experimental tasks were performed in an hci laboratory. super mario 64 was played on a nintendo 64 and viewed on a 25-inch television screen. physiological signal data were collected with the procomp infiniti system and biograph software from thought technologies tm . to measure gsr, two sensors were placed on the left fingers. bvp and hr were simultaneously measured using a sensor on the right fingers. the bvp sensor is sensitive to movement, which is the most likely cause for noisy data. therefore, participants were required not to move the finger attached to the bvp sensor while playing. additionally, game output was recorded in order to synchronize it with the screen showing physiological data. an experimental scene is shown in figure 2. figure 2: an experimental scene the experiment was divided into four phases: a welcome phase, a practice phase, a game phase and a debriefing phase. during the welcome phase, participants signed a consent form with a detailed description of the experiment, its duration and its research purpose. each participant also filled out a background questionnaire. during the practice phase, instructions were read to each individual, describing the game rules, as well as a brief tutorial on how to complete the game tasks. participants were then allowed to practice for approximately three minutes for each game task. at the outset of the game phase, a 10-minute resting baseline (gsr, bvp, and hr) was gathered. then, participants played three tasks, and each task lasted 10 minutes. after completing a task, each participant had about 15 minutes to rest and completed a questionnaire to assess levels of task difficulty and stress caused by the game. during the course of playing the game, their task performance data and physiological data were collected. participants were neither encouraged nor discouraged to talk. at the end of the game, participants discussed their impressions of the experiment. results and discussion australasian journal of information systems volume 13 number 2 may 2006 110 in this section, subjective data assessing stress and physiological data are first described and analyzed. then the correlations between task performance and physiological data are investigated. finally, physiological responses to frustration events are examined. subjective data and physiological data participants’ stress was assessed subjectively on a unidimensional scale, which derived from the rsme (rating scale mental effort) (zijlstra, 1993). ratings of perceived stress are indicated by a cross on a continuous line. the line runs from 0 to 150 mm, and every 10 mm is indicated. along the line, at several anchor points, statements related to perceived stress are given, for example, almost no stress or extreme stress. the scale is scored by the measurement of the distance from the origin to the mark in mm. means for the stress scores were analysed using an anova analysis to determine whether there is difference in stress perceived for the three tasks. the result shows significant differences (f=8.25, df=53, p=0.001) (see figure 3). task threetask twotask one m e a n s tr e s s s c o re 70 60 50 40 30 figure 3: mean stress score of three tasks task three caused the highest stress level, followed by task two and task one. although the difference in stress-score means has statistical significance, the pattern is not always consistent. all the participants stated consistently on the questionnaire that they perceived the least stress during task one. however, four participants reported that task two caused the highest stress level. the questionnaire for evaluating task difficulty could provide partial reasons for the discrepancies. all participants reported that task one was easy, and five participants thought that task two was the most difficult, including the four participants who perceived the highest stress level during task two. in addition, at the end of the tasks, when the four participants were asked why task two caused more stress than task three, they also explained that task two was very difficult to play. the results suggested that the discrepancy with the pattern could be attributed to task difficulty. we analysed physiological data across three tasks. the extremely individual nature of gsr (there is no “normal” baselines for people) makes it virtually impossible to directly compare gsr measurements across individuals (bersak, mcdarby, augenblick, mcdarby, mcdonnell, mcdonal and karkun, 2001). to minimize the influence of individuals, we normalized gsr data using the formula (signal-baseline) / baseline and used the gsr change over a certain time period as our gauge 111 australasian journal of information systems volume 13 number 2 may 2006 of stress during that period. we found that there were significant differences in mean normalized gsr among the three tasks by using an anova analysis (f=11.6, df=53, p<0.001) (see figure 4). task threetask twotask one m e a n n o rm a liz e d g s r 3.5 3.0 2.5 2.0 1.5 1.0 figure 4: mean normalized gsr of three tasks the results are mirrored in the subjective report for assessing stress levels. as previous research suggested, gsr data can indicate the different levels of stress caused by the tasks. moreover, gsr data from all participants were consistent with the pattern: task three caused the greatest normalized gsr, followed by task two and task one. the four participants who perceived the highest stress levels during task two in subjective report were not affected by the task difficulty. we also examined hr and bvp data but didn’t find significant differences among the three tasks. this may be caused by bvp signal with lots of noisy data. most subjects could not avoid moving their fingers attached to the bvp sensor while playing. task performance data and physiological data participants’ success times for a task were defined as their task performance index. the more successes participants got, the higher task performance level was. the sum of success times for three tasks was analysed as overall task performance. among participants, participant 7, who had 22 successes, reached the highest task performance level. in order to examine the relationship between task performance data and physiological data, we classified participants into three groups with different task performance levels according their success times: low (0-7 successes), middle (8-15 successes) and high (16-22 successes). the mean normalized gsr across the three levels was shown in figure 5. australasian journal of information systems volume 13 number 2 may 2006 112 task performance lowmiddlehigh m e a n n o rm a liz e d g s r 4.0 3.5 3.0 2.5 2.0 1.5 1.0 figure 5: mean normalized gsr across three task performance levels an anova analysis showed that there were significant differences among the three groups with different task performance levels (f=3.72, df=17, p= 0.04). the group with low task performance level experienced the greatest normalized gsr, followed by the group with middle task performance level and the group with high task performance level. as gilleade et al. suggested, skilled players responded less physiologically compared with inexperienced players (gilleade, dix and allanson, 2005). yerkes-dodson law (yerkes and dodson, 1908) also suggested that everyone has an optimal level of arousal for performance and going over this level will result in an impairment of task performance and possibly stress. there seemed to be an inherent amount of arousal in every task and the mismatch between task demand and user ability resulted in stress. we also analysed normalized gsr and success times for each task separately (see figure 6). figure 6: mean normalized gsr across success times for three tasks 113 australasian journal of information systems volume 13 number 2 may 2006 for task one, there is a consistently decreasing trend in normalized gsr with an increase in success times. normalized gsr shows statistically significant differences across different success times (1, 2, 3 and 4 successes) (f=10.12, df=17, p=0.001). in addition, success times and normalized gsr also show a negative, statistically significant linear correlation (r=-0.77, p<0.01). for task two and task three, mean normalized gsr doesn’t decrease consistently with an increase in success times, but they still show an overall decreasing trend with an increase in success times. correlation analysis for them shows that normalized gsr of task two negatively and significantly correlates with its success times. (r=-0.60, p =0.005), and task three does not show significant correlations between normalized gsr and success times. there seem to be reasons for the inconsistencies in the trend and the lack of statistical significance in task three. the first is that the number of participants was not large enough to distinguish subtle task performance difference when tasks became more difficult. moreover, there are larger differences in physiological responses between individual participants during difficult tasks than during easy task, which would also have contributed to the lack of statistical significance. physiological response to frustration events stress is a subnet of emotion and often associated with negative emotional states, such as anxiety and frustration (lazarus, 1993). in the study, we inspected physiological response to frustration by examining small periods of time surrounding frustration events. in lawson’s theory of frustration (lawson, 1965), frustration is described as “the occurrence of an obstacle that prevented the satisfaction of a need.” we examined frustration events during task one and task two by reviewing the recorded tapes, which mainly included the following kinds of events: being attacked by enemies, accidentally falling from a mountain road or a bridge, and being hit by an unexpected bomb. we measured gsr of frustration events and when collected at 64 hz, it provides fastresponse, high-resolution and continuous time-series data. there were 355 frustration events in all. gsr data were windowed 5 sec prior to the frustration events and 10 sec after. for 256 frustration events, participants experienced more than 5% increases in gsr when frustrated. an example of a frustration event and its corresponding gsr response is shown in figure 7. figure 7: a participant was frustrated when attacked and experienced a large increase in gsr we also investigated the relationship of frustration events and task performance. figure 8 shows the distribution of frustration events across success times for task one and task two. the results suggest a trend: with an increase in success times, the mean number of frustration events decreases. participants with poor performance experienced more frustration. australasian journal of information systems volume 13 number 2 may 2006 114 figure 8: distribution of frustration events across success times we also analysed participants’ gsr response to their failures in task three, because once participants fell from the snow slide, they failed and had to start again. ninety percent of participants’ failures produced more than a 5% increase in gsr. the link between success times and frustration events provides a reason for explaining the correlation of task performance and physiological data, although it is not comprehensive. when a frustration event happened, participants had to invest extra time and effort to deal with it. consequently, task performance decreased and gsr responses changed. conclusions while the results of this study are preliminary and warrant further investigation, there are three findings that indicate the necessity and promise for using physiology to evaluate usability. the first is that physiological measures are consistent with subjective measures and show significant sensitivity to changes in stress levels. another finding is that, while we have been not able to disentangle clearly the cause-effect relationship between task performance and physiological data, we did find a correlation between the two kinds of data. in the study, gsr data was used to measure stress as a non-specific response of the body and the finding suggests when an individual comes under stress, his performance may be adversely affected. although stress may have a positive effect on performance in individual cases, the subjects reported that stress produced negative consequences on non-task performance dimensions. for example, high levels of stress led to emotional exhaustion. in addition, we found that most frustration events produced remarkable physiological responses, and the participants with poor performance tended to experienced more frustration. the poor task performance and greater physiological response could be attributed to these frustration events. these results show the potential value of physiological data as a data source for usability evaluation. physiological data not only provide a real-time window into users’ state and make it possible to evaluate usability at a more detailed level, but also provide some clues to explaining differences in task performance. the study takes an initial step toward establishing a new usability evaluation 115 australasian journal of information systems volume 13 number 2 may 2006 method using physiology as a complementary measure or as an independent measure for hci evaluation. future work the ultimate purpose of our research is to establish a new usability evaluation method based on three kinds of data: user subjectivity, task performance data and physiological data, which not only evaluates the system’s effectiveness and efficiency, but also takes physiological reactions into account. this paper presents an initial examination of the correlation between physiological data and the traditional index of usability evaluation. more steps are required to create this new usability evaluation method. the experimental results showed that gsr data correlate to task performance data, but we don’t conclude that there is a cause-effect relationship between the two kinds of data. we need more rigorous experimental conditions and analytical methods to understand the correlation. in the experiment, bvp signal failed to index stress levels because of lots of noisy data. we will use more reliable ekg signal instead of the bvp sensor in the next step. there are also other physiological measures that may be useful for usability evaluation in hci. for example, hr variability has been studied extensively. it has been used as an indicator of the extent of task engagement in information processing requiring significant mental effort (sirevaag, kramer, wickens, reisweber, strayer and grenell, 1993; tattersall and hockey, 1995; wilson, 1993), and has been used to detect rapid transient shifts in mental workload (kramer, 1991). eye tracking is another well studied measure. a major use of eye-tracking technologies in hci is to ascertain what is being processed based on what is being looked at. the use of eye tracking enables usability evaluation to be conducted at a more detailed level, pinpointing specific areas within a display that may be causing usability problems and indicating how such issues change over time. in our usability evaluation framework, we will synchronize physiological responses with eye movements and establish a connection between elements of the interface and a user’s current state, through which we are able to understand where usability problems are and how users react to them. in addition, eye tracking has potential to help explain task performance and to detect user fatigue and strain. for example, the number of fixations overall is thought to be negatively correlated with search efficiency, and longer fixations are an indication of difficulty in extracting information from a display (goldberg and kotval, 1999). it also has been found that pupil diameter decreases with fatigue (hess, 1972; lowenstein and loewenfeld, 1964), and pupil diameter changes are related to positive effects (pupil dilation) and negative effects (pupil constriction) (partal, maria and surakka, 2000). in our experiments, different levels of task performance produced different physiological responses. eye tracking could provide clues about explaining the differences in task performance and physiological responses. to create a new methodology, more subtle experimental manipulations should also be explored. for instance, experiments should be extended to different domains and a variety of tasks. moreover, ensuring a sufficient number of participants is needed to increase the statistical power of the study. australasian journal of information systems volume 13 number 2 may 2006 116 acknowledgements we thank the members of the hci group at the university of yamanashi for their support of the research. this study was supported in part by the grants-in-aid for scientific research of the japan society for the promotion of science and by the riec of tohoku university awarded to a. imamiya. references andreassi, j.l. (2000) psychophysiology: human behavior and physiological response (4th edition.). mahwah, nj: lawrence erlbaum associates bersak, d., mcdarby, g., augenblick, n., mcdarby, p., mcdonnell, d., mcdonal, b. and karkun, r. (2001) “biofeedback using an immersive competitive environment”. online proceedings for the designing ubiquitous computing games workshop, ubicomp 2001. boucsein, w. (1992) electrodermal activity. new york: plenum press. cannon, b.w. (1927) “the james-lange theory of emotions: a critical examinations and alternative theory”, american journal of psychology, vol. 39 pp 106-124 davidson, r.j., ekman, p., (1994) “afterward: is there emotion specific physiology” in:ekman, p.,davidson, r.j. (eds.), the nature of emotion. oxford university press, oxford. eisenhofer, i.j.k.g. and goldstien. d (1988) “sympathoadrenal medullary system and stress”, mechanisms of physical and emotional stress, plenum press. ekman, p., levenson, r.w.and friesen, w.v. (1983) “autonomic nervous system activity distinguishes among emotions”, science, vol. 221(4616), pp 1208-1210. frijda, n. h. (1986) the mmotions. cambridge : cambridge university press. gilleade, k. dix, a. and allanson, j. (2005) “affective videogames and modes of affective gaming: assist me, challenge me, emote me” online proceedings of digra'2005. goldberg, j.h., kotval, x.p. (1999). “computer interface evaluation using eye movements: methods and constructs”, international journal of industrial ergonomics, vol 24, pp 631–645. healey, j.a.(2000) wearable and automotive system for affect recognition from physiology, phd thesis pp.22-45, massachusetts institute of technology. hess, e.h. (1972) pupillometrics. in greenfield, n.s., and sternbach r.a. (eds.). handbook of psychophysiology, pp. 491-5310, new york: holt, rinehart and winston. kramer, a. f. (1991) “physiological metrics of mental workload: a review of recent progress” in d. l. damos (eds.). multiple-task performance pp 279-328. london: taylor and francis. knoche, h., de meer, h.g. & kirsch, d. (1999) “utility curves: mean opinion scores considered biased”, proceedings of 7th international workshop on quality of service, 1-4 june 1999, university college london, london, uk. lawson, r. (1965) frustration: the development of scientific concept. new york: macmillan. lazarus, s.r. (1993) “why we should think of stress as a subset of emotion”, in the handbook of stress (ed leo golderberger and shlomo breznitz). the free press, new york. lowenstein, o. and loewenfeld, i.e. (1964) “the sleep-waking cycle and pupillary activity”. annals of the new york academy of sciences, vol. 117, pp 142-156. mandryk, r.l. and inkpen, k. (2004) “physiological indicators for the evaluation of co-located collaborative play”, proceedings of computer supported cooperative work (cscw 2004), chicago, il, usa. meehan, m., insko, b., whitton, m. and brooks, f. (2002) “physiological measures of presence in stressful virtual environments”, proceedings of the 29th annual conference on computer graphics and interactive techniques , san antonio, texas july 23 – 26. partal, t., maria, j. and surakka, v. (2000) “pupillary responses to emotionally provocative stimuli”. proceedings of etra 2000, florida, usa. 117 australasian journal of information systems volume 13 number 2 may 2006 picard, r.w. (1997) affective computing, cambridge, ma: mit press. picard, r.w., vyzas, e., and healey, j., (2001) “toward machine emotional intelligence:analysis of affective physiological state”. ieee pami, vol 23 (10), pp 1165–1174. ravaja, n., saari, t., laarni, j., kallinen, k. and salminen, m. (2005) “the psychophysiology of video gaming: phasic emotional responses to game events”, online proceedings of digra 2005. scheirer, j., fernandez, r., klein, j., and picard, r. (2002) “frustrating the user on purpose: a step toward building an affective computer” interacting with computers, vol. 14, no. 2, pp 93118. seyle, h. (1956) the stress of life, new york: mcgraw-hill. sirevaag, e. j., kramer, a. e, wickens, c. d., reisweber, i., strayer, d. l., and grenell, j. f.(1993) “assessment of pilot performance and mental workload in rotary wing aircraft”, ergonomics, vol 36, pp 1121-1140. sweeney, m., maguire, m. and shackel, b (1993) “evaluating user-computer interaction: a framework”, international journal of man-machine studies, april, vol.38, pp 689-711. tattersall, a., and hockey, g. (1995). “level of operator control and changes in heart rate variability during simulated flight maintenance”. human factors, vol. 37(4), pp 682-698. vicente, k.j., thornton, d.c. and moray, n. (1987) “spectral analysis of sinus arrhythmia: a measure of mental effort”, human factors, vol. 29(2), pp 171-182. ward, r.d., marsden, p.h., cahill, b. and johnson, c. (2002) “physiological responses to welldesigned and poorly designed interfaces”, proceedings of chi 2002 workshop on physiological computing, minneapolis, mn, usa. ward, r.d. and marsden, p.h. (2003) “physiological responses to different web page designs”, international journal of human-computer studies, vol. 59(1/2), pp 199-212. wilson, g. f. (1993) “air-to-ground training missions: a psychophysiological workload analysis”, ergonomics, vol 36, pp 1071-1087. wilson, f. and descamps, p. t. (1996) “should we accept anything less than tv quality: visual communication”, international broadcasting convention, 12th – 16 september 1996, amsterdam. wilson, g.m. (2001) “psychophysiological indicators of the impact of media quality on users”, proceedings of chi 2001 doctoral consortium, pp 95-96. seattle, wash., usa: acm press. wilson, g.m. and sasse, m.a. (2000) “do users always know what's good for them? utilizing physiological responses to assess media quality”, proceedings of hci 2000: people and computers xiv usability or else! pp 327-339, sunderland, uk: springer. wilson, g.m. and sasse, m.a. (2000) “investigating the impact of audio degradations on users: subjective vs. objective assessment methods”. proceedings of ozchi 2000: interfacing reality in the new millennium, pp 135-142, sydney, australia. yerkes, r.m. and dodson, j.d. (1908) “the relation of strength of stimulus to rapidity of habit formation”, journal of comparative and neurological psychology, vol 18, pp 459-482. zijlstra, f.r.h. (1993) “efficiency in work behavior. a design approach for modern tools”, phd thesis, delft university of technology. delft, the netherlands: delft university press. australasian journal of information systems volume 13 number 2 may 2006 118 microsoft word ajisfinal2003.doc australasian journal of information systems special issue 2003/2004 54 modeling the dynamics of an information system jacek unold the wroclaw university of economics komandorska 118/120 53-345 wroclaw poland unold@han.ae.wroc.pl abstract the article concentrates on the nature of a social subsystem of an information system. it analyzes the nature of information processes of collectivity within an is and introduces a model of is dynamics. the model is based on the assumption that a social subsystem of an information system works as a nonlinear dynamic system. the model of is dynamics is verified on the indexes of the stock market. it arises from the basic assumption of the technical analysis of the markets, that is, the index chart reflects the play of demand and supply, which in turn represents the crowd sentiment on the market. introduction the area of information systems (is) is often seen to be about technology. however, this area is multidisciplinary and although information technology (it) is certainly relevant, so are human and organizational aspects (e.g., vidgen et al., 2002). all information systems, from manual to informal to computer-based, are designed, operated and used by people in a variety of organizational and environmental settings and contexts. the ubiquitous processes of globalization and virtualization are constantly changing our understanding of the human factor involved in the operation of information systems. no longer can we perceive the “people” component of is as independent individuals. the users of local, regional and global telecommunication networks create a specific form of a “virtual crowd”, accessing the same sources of information and reacting to the same sets of stimuli. this phenomenon creates a new challenge and defines new research areas. the article concentrates on the nature of a social component of an information system, a component determining the is dynamics. it proposes an innovative approach to the analysis and modeling of collective information processes and mechanisms of collective behavior. this is the first attempt of this kind in relation to the above-mentioned social issues in the is area. the following research question has been formulated against the background of the state-of-the-art: • are information processes of collectivity and the resulting collective behavior, which make the is dynamics, phenomena of exclusively qualitative nature, impossible to structure? to answer this question a model of is dynamics is introduced. the model is based on the assumption that a social subsystem of an is works as a nonlinear dynamic system. generally, the adopted research methodology consists of the following stages: identification, analysis, synthesis, exemplification, verification, and interpretation. the structure of the presentation helps illustrate the modeling process. in the following chapters the article introduces the notion of a social subsystem within an information system, identifies determinants of the is dynamics, analyzes the phenomenon of self-organization of collective behavior within an is, synthesizes the research findings in a model representation of the is dynamics, and presents its exemplification, verification, and interpretation. social component of an information system all organizations have information systems. according to vigden et al (2002, p. 2), an information system is a set of interacting componentspeople, procedures, and technologies – that together collect, process, store, and distribute information to support control, decision-making and management in organizations. benson and standing (2002, p. 5) identify the following components of an is: people, data/information, procedures, hardware, software, communications. in the next approach, avison and fitzgerald (2003, p. 19) define an is as a system which assembles, stores, australasian journal of information systems special issue 2003/2004 55 processes and delivers information relevant to an organization (or to society), in such a way that the information is accessible and useful to those who wish to use it, including managers, staff, clients and citizens. the authors stress that an information system is a human activity (social) system, which may or may not involve the use of computer systems. this definition is very useful in that it emphasizes the human and organizational aspects of information systems, and makes clear that not all information systems use it. the area of is might include simple manual (paper-and-pencil) information systems, informal (word-of-mouth) information systems or computer-based information systems. despite slightly different approaches and definitions, people are required for the operation of all information systems. this is the social component of all information systems. avison and fitzgerald (2003, p. 5, 6) outline two examples of an is, at somewhat different ends of the spectrum. one is a payroll system, which was one of the first applications to be computerized. the other example system is very different. it is an electronic auction house, such as ebay. it is relatively new (ebay started in 1995) and exciting, and uses the world wide web as its user interface. yet, essentially, it is just an information system. it matches buyers with sellers utilizing an auction concept. the electronic auction enables buyers and sellers to be geographically distributed across the world. the electronic auction is an is, comprising people, rules, procedures, technology, software, communications and allied services. such electronic houses reflect the shift from marketplace to marketspace in modern business environment. exactly the same understanding of an is can be applied to a modern electronic and virtual stock exchange, where people, distributed geographically, can perform market transactions via telecommunications technologies. thus, a social subsystem of a stock exchange is is a chosen research area for this analysis. collective behavior and is dynamics the next concept, which has to be discussed is that of dynamics and is dynamics. one of the meanings of the word dynamics (gr. dynamikos – strong, forceful) refers to the motivating or driving forces, physical or moral, in any field (webster’s, 2002; p. 445). by the dynamics of an organization we understand the dynamics of its information system, considering the subjectivity and crucial role of its social subsystem. such a notion of dynamics represents changes in the various types of knowledge, in the learning, and unlearning processes (eden and spender, 1998; p. 15). in this approach, the basic determinants of the is dynamics are: knowledge, learning and unlearning. the research shows that collective knowledge cannot be understood without paying attention to the communication processes going on among the group’s members (weick and roberts, 1993; p. 358). it follows that information processes of collectivity are another determinant of the is dynamics, and learning and unlearning processes constitute a specific category of those processes. according to turniansky and hare (1998; p. 112), organizational learning is a metaphor to focus attention on the ways an organization adapts to its environment. organizational learning occurs when knowledge, acquired and developed by individual members, is embedded in “organizational memory”. this process can be identified at four levels of learning: individuals, groups, organizations, and populations of organizations. cook and yanow (1993) present “a cultural perspective” of organizational learning, which is understood as the acquiring, sustaining, or changing of intersubjective meanings through the collective actions of the group. this concept directly refers to the generic notion of dynamics, understood as the ability to act. hence, collective behavior is the next determinant of the is dynamics. the central concept embracing individual and collective knowledge, both explicit and implicit, is that of the “activity system” (eden and spender 1998; p. 15). the key to the nature of activity system is the dynamics, which leads to changes in various types of knowledge and its information processes (including learning and unlearning). this concept points to the next determinant of the is dynamics: collective mind. the collective mind is located in the practice of the activity system. any entity that has capacities for generating and absorbing information, for feedback and self-regulation, possesses mind. weick and roberts (1993; p. 377) suggest that the key feature, which distinguishes the different forms or types of organization, is the degree to which they facilitate or inhibit the australasian journal of information systems special issue 2003/2004 56 development of collective mind. many authors use the notions of collective mind and organizational culture interchangeably (eden and spender, 1998). the notion of collective mind offers a means to contemplate organizational intelligence. it is interesting that an organism (system) does not even need a brain in order to be intelligent. intelligence is a property that emerges when a certain level of organization is reached, which enables the system to process the information (wheatley, 1999; p. 98). the concepts of collective mind and organizational intelligence add a crucial qualitative dimension to the systems analysis. they add the missing internal social dimension to the technical or mechanistic dimension, which is the focus of classical theory of systems and organizations. these concepts also allow for further analysis of the is dynamics, which is understood as a phenomenon directly linked with collective behavior within “a human activity (social) system,” in another words, within an information system. self-organization of collective behavior in is to identify the basic characterstics of a social subsystem of an is it is helpful to analyze the issue of alternative organizational forms. contemporary literature points to four basic forms (turniansky and hare, 1998; p. 101): • the bureaucratic organization, • the post-bureaucratic, interactive organization, • team-based organization, • self-organization. the fourth category, self-organization, is especially important for further analysis. the challenges of the future – globalization and virtualization of business activity – are indicating that organizations will probably have to move much closer to the model of a complex adaptive system. some of the characteristics of complex adaptive systems (freedman, 1992), which are relevant to this discussion include: a) they consist of a network of agents acting in a self-managed way without centralized control; b) the environment in which the agents find themselves is constantly changing and evolving since it is produced by their interactions with other agents; c) organized patterns of behavior arise from competition and cooperation among the agents; d) this cooperation produces structures arising from interactions and interdependencies. the processes in a complex adaptive system are those of mutual adjustment and self-regulation. the structure that emerges is not simply an aggregation of individual actions, but has unique properties not possessed by individuals alone. self-organization can not be imposed from outside, but operates from within the system itself. organization is not designed into the parts, but is generated by the interaction of those parts as a whole. the self-organizing form has implications for organizational learning and creativity. self-organizing systems are characterized by system resiliency rather than stability. when such a system has to deal with new information (information shock), it reconfigures itself to adapt to the new situation. at the same time, the life of a complex adaptive system depends on the access to new information. in a seemingly paradoxical way, openness to the environment, to information from outside, leads to higher levels of system autonomy and identification. another principle fundamental to self-organizing systems is self-reference. in response to environmental disturbances that signal the need for change, the system changes in a way that remains consistent with itself in that environment. the chosen research area for this analysis is an information system of a stock market or, to be more specific, its social subsystem. modern stock exchange operates analogously to the electronic auction house, cited previously as an examplary is. what is important, unlike other collectivities, the behavior of a stock market collectivity is reflected by relatively simple and concrete indicators. these are: price changes shown by the index chart, and some “mechanical” indicators of collective australasian journal of information systems special issue 2003/2004 57 activity, such as volume (the total number of shares changing hands during a session) and total turnover (money engaged on either side during a session). our initial analysis shows that such defined social subsystem of a stock market’s is reveals all the attributes of a complex adaptive system. this system consists of a network of agents (here: investors) acting in a self-managed way without centralized control (attribute a). the environment in which the investors operate changes and evolves constantly, which is a result of continuous fluctuations in economy and the market situation. the investors try to follow these changes, which produces interactions among them, reflected by individual orders placed during a session (b). the daily activity of the market is based on competition and cooperation among the investors, which leads to a consensus, reflected by the prevailing market trend. this trend proves the existence of organized patterns of collective behavior (c). the analysis of the fourth attribute (d) points to the emergence of a natural dynamic structure of such a system, which is linked to the organized patterns of behavior. recapitulating, these phenomena reflect the dynamics of the information system of a stock market. the analysis of self-organization leads to the issue of system complexity. unlike traditional science, which studies “ideal” phenomena, complexity theory studies the phenomena most common in the real world: turbulence and disequilibrium, self-organization, adaptation, system learning. these are some of the “emergent behaviors” which crop up again and again not only in biological systems, but also in technological, computational and economic systems. these behaviors are characteristic of nonlinear, complex adaptive systems (battram, 1999; p. 16). complexity theory offers a range of new insights into the behavior of social and economic systems. the idea of self-organization and emergence can be used to identify and explain the dynamics of individual and collective behavior on the stock market. thousands of independent and difficult to observe transactions, carried out by individual participants of the market, generate the emergence of specific and predictable patterns of collective behavior. these phenomena can only be identified on the higher (collective, not individual) level of social organization. kauffman’s (1995) famous phrase “order for free” describes this process of “crystallization”, also known as the emergence of complexity in complex adaptive systems. model representation of is dynamics the concept of a model representation of the is dynamics is based on the theory of cycles, which allows for a graphic illustration of the discussed phenomenon. the crowd is created by information capable of uniting single individuals into a group. the group, then, lives its own life, a life, which depends on the exchange of information with the environment. the most significant symptom of this phenomenon is the collectivity’s fluctuation during this exchange, and it reflects its dynamics. according to the theory of cycles such stable fluctuations between a system and its subsystem can be presented in a model form as a bounded cycle (plummer, 1998). as far as capital market collectivities are concerned, those fluctuations have two basic determinants, price changes (y) and changes of the crowd sentiment (x) caused by (y) (fig. 1). figure 1. formation mechanism of a spiral of the collectivity adaptation process: a bounded cycle australasian journal of information systems special issue 2003/2004 58 source: adapted from (plummer, 1998) [legend: x axis crowd sentiment; y axis – prices] the bounded cycle is a basic mechanism through which complex adaptive systems react to the fluctuations of the environment. because this cycle is stable, it does not represent all adaptive processes. in reality, the flow of information is not a continuous process. so when unexpected information appears (information shock), the collectivity tries to conform to the new conditions by `changing its dynamic structure. it is expressed by a sudden “jump” from the cycle path. in the case of financial market collectivities, a jump in both prices (y) and moods (x) occurs (fig. 2). some time later, the collectivity tries to return to the basic cycle path and this phenomenon can be expressed by a spiral of the adaptation process (fig. 3). in the next step the observations concerning the nature of adaptive processes of collectivity will be supplemented with the issue, what kind of a spiral represents these phenomena? figure 2. formation mechanism of a spiral of the collectivity adaptation process information shock and “jump” from the cycle path source: adapted from (plummer, 1998) figure 3. formation mechanism of a spiral of the collectivity adaptation process spiral of the adaptation process. source: adapted from (plummer, 1998) australasian journal of information systems special issue 2003/2004 59 mathematical and graphical dimension of the is dynamics there are many different spiral movements, so it is necessary to define a mathematical base for this specific model curve, which reflects the dynamics of collectivity. this stage of the analysis is grounded in a well-known assumption that a collectivity also forms a natural system (e.g., frost and prechter, 1999). thus, this mathematical base can be found in the world of nature and we can look for analogies between the two natural systems: collectivity and nature. the most common curve in nature is a logarithmic spiral. the tail of a comet curves away from the sun in the spiral. distant galaxies, hurricane clouds, ocean waves and whirlpools swirl in spirals, as do many other natural phenomena. the construction of the logarithmic spiral (fig. 4) is based on the fibonacci ratio φ=1.618, known as the golden ratio or golden mean. it defines the ideal proportions. r2 : r1 = r3 : r2 = r4 : r3 = ... = rn : rn-1 = 1.618 d2 : d1 = d3 : d2 = d4 : d3 = dn : dn-1 = 1.618 (d1 = r1 + r3, d2 = r2 + r4 and so on) arcxy : arcwx = arcyz : arcxy, itd. = arcxz : arcwy = 1,618 figure 4. fibonacci ratio (1.618) and logarithmic spiral source: (unold 2003, p. 188) the fact that a collectivity also forms a natural system is a basic premise allowing us to look for analogies in the behavior of collectivity and natural phenomena. this natural law, permeating the universe and described by the fibonacci ratio φ=1.618, should refer to the dynamics of collective behavior as well. since adaptations to the exchange of information spiral, and financial markets reflect psychology and the dynamics of the crowd, the spiral identified in price formations also should be logarithmic. the case study confirms that, indeed, the top of each successive wave of higher degree on the index chart is the touch point of the logarithmic expansion, as presented in fig. 5 . australasian journal of information systems special issue 2003/2004 60 figure 5. logarithmic spiral on the warsaw stock exchange index (wig) source: author’s research the next diagram (fig. 6) confirms that the phenomena described in this analysis are identified in any market behavior, both on the emerging polish stock market (fig. 5) and the developed german one. figure 6. logarithmic extension of the trend on the german stock exchange (dax): january 1995 – april 2003 source: author’s research also the next diagram, which identifies the logarithmic extension of the market trend in japan (the nikkei index), confirms the universal nature of crowd behavior. this observation allows for the generalization of market behavior, identified as a research example of is dynamics. australasian journal of information systems special issue 2003/2004 61 figure 7. logarithmic spiral at the nikkei index in japan: november 1978 – november 1995 source: author’s research conclusions the model of is dynamics presented in this article provides an approach to help us understand the nature of social phenomena identified within an information system. the model posits that the social subsystem is a nonlinear dynamic system, hence the research area is the collectivity of stock market investors, constituting the “people” component of the chosen is. the proposed model reveals the basic mechanism forming a spiral of collective adaptation processes. this spiral is isomorphic and self-similar. this observation is confirmed in the practical part of the research – the logarithmic extension is identified in any trend on any stock market. the revealed nature of the is dynamics shows that the decision-making process of collectivity is adaptive and follows specific patterns found in nature. therefore, unlike the decision-making process of an individual, it can be expressed mathematically and is predictable. in other words, individual behavior – which is often irrational and unpredictable composes an adaptive, spiral, and, thus, predictable process of collective decision-making. the achieved outcome also confirms the possibility of a deterministic approach to certain qualitative factors encountered in an information system. these parameters, considered indeterministic so far, refer to the dynamics of group behavior within an is. this should help increase the efficiency of management through better understanding, and even predicting the behavior of large groups of people. besides the domain of is, the potential beneficiaries include the areas of finance, communications, marketing, cyberspace, distant learning, and politics. the disclosure of a mathematical expression of is dynamics (φ=1.618) is the first significant step in this direction. references avison, d., fitzgerald, g. (2003) information systems development: methodologies, techniques and tools, mcgraw-hill, new york. battram, a. (1999) navigating complexity, the industrial society, london. benson, s., standing, c. (2002) information systems: a business approach, john wiley & sons, milton. australasian journal of information systems special issue 2003/2004 62 cook, s. d. n., yanov. d. (1993) “culture and organizational learning”, journal of management inquiry, 2. eden, c., spender, j. c. (1998) managerial and organizational cognition: theory, methods and research, sage publications, london. freedman, d. h. (1992) “is management still a science?” harvard business review, nov.-dec. frost, a., prechter, r. (1999) elliot wave principle, john wiley & sons, new york. kauffman, s. (1996) at home in the universe, oxford university press, oxford. peters, e. e. (1996) chaos and order in the capital markets. a new view of cycle, prices, and market volatility, john wiley & sons, new york. plummer t. (1998) forecasting financial markets. kogan page limited, london. turnianski, b., hare, a. p. (1998) individuals in groups and organizations, sage publication, london. unold, j. (2003) information systems dynamics and adaptive rationality (in polish), the wroclaw university of economics, wroclaw. vigden, r., avison, d., wood, b., wood-harper, t. (2002) developing web information systems, butterworth-heinemann/elsevier science, oxford. webster’s encyclopedic unabridged dictionary of the english language (2002), portland house. weick, k. e., roberts k. h. (1993) “collective mind in organization: heedful interrelating on flight decks”. administrative science quarterly, 38. wheatley, m. j. (1999) leadership and the new science. discovering order in a chaotic world, berrett koehler publishers, san francisco. microsoft word special final.doc ajis special issue december 2004 146 online communities: researching sociability and usability in hard to reach populations jenny preece , dean college of information studies university of maryland 4105e hornbake building, south wing college park maryland, md 20742, usa preece@umd.edu abstract overview thousands of social gatherings online communities – occur via icts across the internet. they use listservers, bulletin boards, instant messaging, blogs, social network and meeting facilitation systems and purpose-build platforms. each community is the product of a complex set of interacting variables. some of these variables come from usability of the software but social factors have a strong influence. direct cause and effect relationships are often difficult to tease out and identifying criteria for successful online communities can be difficult. furthermore several disciplines own this topic, each with its own theories, research methods, community of researchers and publication outlets. this is good for multidisciplinary research but it can be challenging. in this paper i discuss two research projects. each focuses on community populations that are hard to reach. the first analyses why people do not actively participate in online discussions, i.e., lurk. the second is a new project to develop cross-cultural online book communities for children across the world who do not speak each other’s languages. i present the key findings from each project and suggest ways of working with these hard to reach populations. keywords online community, networked community, ict, social computing, usability, sociability, lurking, children, digital library introduction online communities are sources of information, support, entertainment and places to ‘hang-out’ and socialize for millions of people. these communities support patients coping with diseases, students discussing homework projects, hobbyists pursuing their passions, and teens chatting about their lives. scholars use them to track academic topics, lawyers seek legal information, and professionals exchange business knowledge. a variety of software facilitates information exchange and communication including listservers, bulletin boards, instant messaging, blogs, social network and meeting facilitation systems and purpose-build platforms. these communities are known as ‘communities of practice’ (cops) and ‘communities of interest’ (cois) depending on their purposes. however, the term ‘online community’ is controversial for a number of reasons; not least because most online communities also have physical off-line components. for example, many online communities start as face-to-face communities before going online. others start online and then subgroups realize that they live near enough to meet face-to-face. some efforts to define ‘online community’ encourage deeper understanding. for example, wenger says that online communities comprise “people with a shared interest who engage in collective learning that creates bonds between them” (wenger, 1998). here the emphasis is on the role of shared interest in bringing people together. learning promotes bonds that hold people together in the ajis special issue december 2004 147 community. depending on the nature of these bonds they can give rise to weak-tie and strong-tie communities (granovetter, 1973). each of us belongs to tens if not hundreds of weak-tie communities; email lists to which people subscribe can be thought of as weak-tie communities (wellman, 1997). weak-tie communities are wonderful for disseminating information. in contrast strong ties are harder to come-by. these are the people to whom we turn in times of real need, for a loan, a ride to the hospital, or to pick up your child from school when an important meeting is running over-time. we can further define a community by how people take and give back, that is, by reciprocity, or by the amount of trust reported by its members, or by networks of members’ relationships. communities are often described by their focus of interest. while every community, like every person has its own characteristics or personality, communities drawn together by similar interests tend to be more similar than those drawn together by widely different interests. online health communities differ from education communities, and from religious and political communities (maloney-krichmar and preece, 2004 in press; preece et al., 2003; 2004). furthermore, many online communities share members with other communities that meet in other online spaces or off-line (maloney-krichmar and preece, 2004 in press). technically-oriented researchers tend to describe online communities in terms of their supporting technology; for example, a bulletin board community, an instant messaging (im) community (preece, 2000). we know that the nature of the social space provided by technology impacts community activity. small differences between platforms can strongly influence social interaction. for example, personal information presented in pre-defined tabular format creates different perceptions from free-form descriptions. another way of thinking about community is from an operational perspective. an online community is “a group of people with a common purpose whose interaction is mediated and supported by computer systems, and governed by formal and informal policies” (preece, 2000; 2004). of course, when we delve beneath the surface of the four highlighted components people, purpose, computer systems and policies we see a much more complex set of attributes. people have identities that are determined by how and what they reveal about themselves online, which may be influenced by the design of the software, their expertise in using it or by something as fundamental as personality and motivation. people’s skills and expertise contribute to their reputations that, in turn, influence how they behave online and how others perceive them, trust them, cooperate or challenge them (preece, 2004). using the “people, purposes and policies” components provides “handles” for understanding sociability, which can be mapped to usability design (preece 2000; de souza and preece, 2004). online community researchers experience challenges that are well-known in human-computer interaction (hci) research (preece, rogers, sharp, 2002). for example, how do you get to know your users? sometimes these problems are tricky in online communities as the two example studies reported below indicate. the goal of the first study was to get lurkers to speak out about why they don’t actively participate so that community developers and managers can develop strategies to encourage them to participate when appropriate. the second study investigates ways of supporting community among children aged 7-11 from different cultures who may not speak the same language. project 1: why don’t people participate in online discussions? new posts raising questions, giving answers or offering support, jokes and other comments encourage vibrant online discussion. if there is no new content community members will leave and seek to get their needs satisfied somewhere else. this was particularly evident during the dot.com era. business entrepreneurs thought that they could add a bulletin board to their e-commerce site and that this “community” would draw-in customers and expand their sales. they were wrong. adding ajis special issue december 2004 148 communications software is not the same as creating an online community. communities evolve nurtured by their members and leaders. some questions that intrigue commercial web developers and online community researchers are: what added value do online communities bring? why do so many people prefer not to participate publicly (i.e., lurk)? how can we get people to actively contribute? our research has investigated this question from a variety of perspectives (andrews et al., 2003; nonnecke et al., 2004; 2004a; 2004b; preece et al, 2004). we know that lurking varies across different kinds of communities (nonnecke and preece, 2000) and that the number of lurkers is influenced by community size, moderation style, and software support, among other things. from an online survey that generated 1188 responses from posters and lurkers from 375 msn bulletin board communities, 219 lurkers spoke out about their reasons for not posting. we know from this study that while lurkers do not participate publicly, they did seek answers to questions. however, lurkers’ satisfaction with their community experience is lower than those who post. data from 19 checkbox items and over 490 open-ended responses were analysed in order to identify the main reasons why lurkers lurk. the reasons given were categorized under five types: (i) being able to get what they needed without having to post; (ii) needing to find out more about the group before participating; (iii) thinking that by not posting they were being helpful and not cluttering-up the conversation; (iv) not being able to make the software work (i.e., poor usability); and (v) not liking the group dynamics of the community (preece et al, 2004). while some lurkers can be thought of as free-riders who take with out reciprocating (smith and kollock, 1999), the vast majority lurk for other reasons, some of which are altruistic. by far the largest number of lurkers were unaware about the unsociable impact of their behaviour on others. from these findings it is clear that there are many ways that might improve online community experiences for both posters and lurkers. some solutions require improved software and better tools, but better moderation practices and strategies for facilitating social interaction are needed, particularly for welcoming newcomers and during the early life of online communities. doing this research has also engaged us in finding solutions to a variety of challenges concerned with reading this ‘hard-to-reach’ user population. first we needed an operational definition of “lurking”. second, lurkers are by definition people who do not reveal themselves, so we had to find ways of getting lurkers to respond to our survey. third, getting a representative sample from the unbounded and constantly changing populations in internet communities is challenging (andrews et al., 2003). in addition there are ethical issues associated with research on open internet communities, particularly patient support communities, so getting irb approval took time. we then experienced harassment from a small but vigorous group of sports community participants who tried to hack our server twice and sent obscene messages to our research assistant. project 2: international children’s digital library book communities the second project is quite different. in this project we are researching and developing software to support online communities for children who access the international children’s digital library (icdl) (druin, 2002; druin, 2004 in press). these children are geographically scattered across the world and the goal of this project is to support communicative exchanges that include games, story telling, chatting with friends, collaborating on projects and doing history, geography, math and reading. the children come from different countries and most do not speak the same language as the children that they meet online. nor do they share the same culture. indeed, another important goal of this research is foster cross-cultural understanding and tolerance. the international children’s digital library communities (icdlcommunities) project extends the work of allison druin’s and ben bederson’s icdl team. the project team includes specialists from ajis special issue december 2004 149 information studies, computer science, education, art and psychology and children aged 7-11 working together to build a digital library. the goal of their project is to put at least 100 children’s books online for each of 100 countries across the world. in all there will be over 10,000 carefully selected children’s books in the icdl. each book will be presented in the original language in which was written. understanding the technical, educational, recreational and learning needs of young children requires different techniques from those traditionally used with adults. consequently, allison druin and her team have devoted considerable effort to developing appropriate techniques. they successfully use an intergenerational design methodology in which children and adults work together (druin, 2002). this methodology has two broad components. a form of contextual inquiry enables the team to explore the design space, followed by participatory design in which the children and adults invent new designs for reviewing and testing. working with children in this way involves negotiating new power structures in which the ideas and skills of both children and adults are respected. different approaches to knowledge elicitation are also needed. children like to use crayons and pens to draw their ideas. glue, scissors, colored paper and various other artefacts are also useful for making models. this approach contrasts strongly with that of most adults who write, construct tables and other complex representations to describe their ideas. the online icdlcommunities project will enable children’s communities to develop around the book collection. children will talk about the books that they read online, create their own books and create and share their own stories aided by teachers and parents. using the intergenerational design methods mentioned above, we are working with children to understand their community needs. what do they like to do online? how children engage with other children using technology? what do they talk about? what kinds of activities motivate the children? we know that for children’s technology to be successful it must engage them, put them in control and give them appropriate feedback to support their social engagement with peers (druin, 1999; druin and inkpen, 2001; bruckman and bandlow, 2002). an added and non-trivial challenge for developing children’s cross cultural book communities is that the children will meet others who do not speak their language. already we are exploring prototype tools that will enable children to make their own drawings, develop picture dictionaries, exchange messages (mostly in the form of drawings or drawings that have been labelled), ask and answer questions. there are many challenges but to our surprise lack of a common language is not the most serious. understanding each other becomes a game. it is fun to guess what the other children’s pictures are about. as well as having to develop new ways of understanding this ‘hard-to-reach’ user group of international children, we are also identifying issues that are known to be important in adult’s online communities. for example, early on our hungarian children’s team asked to send photos of themselves to the american children’s team. they were intrigued to see what the other children look like, how they use technology, what they do at school, what kind of sports they play, which holidays they celebrate, and what kinds of foods and candies they like. identity and culture are important both collectively for the community and for individual children, just as they are for adults. conclusions: some lessons learned the findings from these projects suggest some sociability and usability principles for designing successful online communities and techniques for reaching ‘hard-to-reach’ user populations. these are summarized in the following ten suggestions for improving sociability and usability in online communities. ajis special issue december 2004 150 10 sociability and usability suggestions • enable communication tasks needs to be motivating and meaningful for participants (particularly for children) • devise software, moderator and community-run strategies for encouraging online participation (e.g., mentoring, tours, meeting areas for new-comers) • make participants aware of norms and responsibilities (particularly the need to contribute as well as take) • reward contributions and service to the community • provide reliable software (reliability wins over state-of-the art with many coi participants) • provide 24/7 access (particularly for support communities) • make the community’s purpose clear (descriptive name, purpose statement, window onto the communication) • make communication software obvious (do not expect users to notice a link hidden in text and other links, provide a clear link or window onto the communication) • support online identity and development of trust and empathy (allow people to describe or show themselves through stories and pictures) • devise ways to make children’s online activities as safe as possible (e.g., engage parents and teachers) the ‘take away’ message from both research studies is that “while usability is always important it is not sufficient. we must pay careful attention to sociability too”. doing community-base research may require us to expand old techniques and develop new ones so that we can reach participants in “hard to reach” community populations. in our studies we had to (i) operationalize slippery concepts (e.g., lurking defined so that it was measurable); (2) adapt sampling techniques to be as rigorous as possible for researching unbounded internet populations; and (3) develop community-centered participatory design methodologies for working with children from different cultural and language groups across the world. references andrews, d., nonnecke, b., preece, j. (2003) electronic survey methodology: a case study in reaching hard to involve internet users. international journal of human-computer interaction. 16, 2, 185-210. bruckman, a. and bandlow, a. (2002). hci for kids. in j. jacko and a. sears (eds.) handbook of human-computer interaction. hillsdale, nj: lawrence earlbaum associates. de souza, c., s., preece, j. (2004) a framework for analyzing and understanding online communities. interacting with computers, the interdisciplinary journal of humancomputer interaction, 16, 3, 579-610. druin, a. (2002). the role of children in the design of new technology. behaviour and information technology, 21(1), 1-25.(tech report version at: ftp://ftp.cs.umd.edu/pub/hcil/reports-abstracts-bibliography/99-23html/99-23.pdf) druin, a. (ed.). (1999) the design of children's technology. san francisco, ca: morgan kaufmann. (book at: http://www.umiacs.umd.edu/~allisond/books.html) druin, a. (in press). what children can teach us: developing digital libraries for children. library quarterly. (tech report version at: ftp://ftp.cs.umd.edu/pub/hcil/reports-abstractsbibliography/2003-39html/2003-39.pdf) ajis special issue december 2004 151 druin, a. and inkpen, k. (2001) when are personal technologies for children. personal technologies, 5, 3, 191-194. granovetter, m. (1973) the strength of weak ties. american journal of sociology, 78,1360-1380. maloney-krichmar, d. and preece, j. (2004) a multilevel analysis of sociability, usability and community dynamics in an online health community (accepted, transactions on humancomputer interaction) nonnecke, b. and preece, j. (2000) lurker demographics: counting the silent. proceedings of acm chi’2000, hague, the netherlands, 73-80. nonnecke, b., andrews, d., preece, j. (2004) online lurking: current and past. association of internet researchers (aoir) conference, brighton, uk, september. nonnecke, b., preece, j., andrews, d. (2004a) lurking and public participation in discretionary online communities. (accepted electronic commerce research journal.) nonnecke, b., preece, j., andrews, d., voutour, r. (2004b) online lurkers tell why. american conference on information systems. (amcis) new york, august 5-8. preece, j. (2000) online communities: designing usability, supporting sociability. chichester, uk: john wiley & sons. www.ifsm.umbc.edu/onlinecommunities preece, j. (2004) etiquette online: from nice to necessary. communications of the acm, 47, 4, 56-61 preece, j., abras, c., maloney-krichmar, d. (2004) designing and evaluating online communities: research speaks to emerging practice. international journal of web-based communities. 1, 1, 2-18. preece, j., maloney-krichmar, d. and abras, c. (2003) history of online communities in karen christensen & david levinson (eds.), encyclopedia of community: from village to virtual world. thousand oaks: sage publications, 1023-1027. preece, j., nonnecke, b., andrews, d. (2004) the top five reasons for lurking: improving community experiences for everyone. computers in human behavior. 20, 2, 201-223. preece, j., rogers, y. & sharp, h. (2002) interaction design: beyond human-computer interaction. new york, ny: john wiley & sons. www.id-book.com smith, m. and kollock, p. (1999) communities in cyberspace. london: routledge. wellman, b. (1997) an electronic group is virtually a social network. in s. kiesler (ed.) culture of the internet. mahwah, nj: lawrence earlbaum associates, 179-205. wenger, e. (1998) communities of practice. cambridge, uk: cambridge university press. acknowledgements i thank my colleagues from the community participation project: dorine andrews, blair nonnecke, russell, voutour and the international children’s digital library online communties project team: allison druin, aaron, elkiss, anita komladi, philip resnik, enrique stanziola, roya swartz, lingling zhang. nsf (grant no: nsf 0328823) is supporting the international children’s digital library online communties project and the community participation project is partially supported by microsoft research and partially by university of maryland baltimore county. copyright jenny preece © 2004. the authors assign to ozchi and educational and non-profit institutions a non-exclusive licence to use this document for personal use and in courses of instruction provided that the article is used in full and this copyright statement is reproduced. the authors also grant a non-exclusive licence to ozchi to publish this document in full in the conference papers and ajis special issue december 2004 152 proceedings. those documents may be published on the world wide web, cd-rom, in printed form, and on mirror sites on the world wide web. any other usage is prohibited without the express permission of the authors. ajis vol. 9, no. 2 may 2002 integrated multimedia based intelligent group decision support system for electrical power network ajay kumar saxena s. 0. bhatnagar p. k saxena faculty of engg. faculty of soc. sc. faculty of engg. dayalbagh educational institute, dayalbagh, agra-282005, india e_mail: aksaxena61@hotmail.com abstract electrical power network in recent time requires an intelligent, virtual environment based decision process for the coordination of all its individual elements and the interrelated tasks. its ultimate goal is to achieve maximum productivity and efficiency through the efficient and effective application of generation, transmission, distribution, pricing and regulatory systems. however, the complexity of electrical power network and the presence of conflicting multiple goals and objectives postulated by various groups emphasized the need of an intelligent group decision support system approach in this field. in this paper, an integrated multimedia based intelligent group decision support system (im1gdss) is presented, and its main components are analyzed and discussed. in particular attention is focused on the data base, model base, central black board (cbb) and multicriteria futuristic decision process (mfdp) module. the model base interacts with electrical power network load forecasting and planning (epnlfp) module; resource optimization, modeling and simulation (romas) module; electrical power network control and evaluation process (epncaep) module, and mfdp module through cbb for strategic planning, management control, operational planning and transaction processing. the richness of multimedia channels adds a totally new dimension in a group decision making for electrical power network. the proposed imigdss is a user friendly, highly interactive group decision making system, based on efficient intelligent and multimedia communication support for group discussions, retrieval of content and multi criteria decision analysis. introduction electrical power network plays a unique role in the development and supporting most of the major activities for the overall growth of a country. in many developed and developing countries, today's socio-economic environment has expanded the role of the technical managers, who, operates and design the electrical power network. organizations and sectors capable of recruiting technical managers with sound technical expertise, interpersonal skills and management education are more likely to be successful in managing their limited resources for planning and controlling activities. the electrical power network (epn) can be divided into five sub-systems: power generating sub-system (pgss)include elements normally associated with the generation of power, such as generators and steam turbines and other accessories. transmission sub-systems (tss)consists of all substation activities including transmission lines, control houses, switchyards and transformers at power stations. distribution facilities sub-station (dfss) -include feeder lines, safety devices and distribution transformers at service drops. network control and protection facilities sub-system (ncpfss)-comprises of the energy control center, the supervisory control and data acquisition (scada) system and some elements that provide network protection at the center. auxiliary facilities sub-station (afss) include storage yards, maintenance facilities and radio communication facilities for network maintenance. since the last few decades, the global trends towards liberalization of electric power market open up to give customers the opportunity to purchase energy at more favorable prices. this has resulted in restructuring of epn. the generation is separated from transmission to enable competition among power plants to arrive at the cheapest production price. the high voltage system becomes the network simply to transmit energy from the generating plant to the regional networks. the transmission system takes over the role of meeting contractual agreement among different parties for delivering power. in the new environment, greater investment in the epn will be made only if direct and short-term benefits can be expected through the reduced cost of generation, transmission and distribution. several literatures are available in the field of power generation, transmission and distribution issues. arriaga, rudnic and stadlin (1995), hissey (1994) and vojdani, imparato, saini wollenberg and happ (1996) have discussed various aspects of transmission system and pricing of transmission transaction in open access. torre, feltes, roman and merrill (1999) and tabros (1994) have suggested various management and pricing aspects of transmission system in deregulated environment. to determine the economically adapted transmission system in open access a genetic algorithm based dynamic transmission planning methodology is formulated by rudnic, palma, cura and silva (1996). handschin, heine, koing, nikoden, seibt and palma (1998) have developed an object-oriented software for transmission planning issues. parikh and chattopdhyaya (1995) have discussed a multi area approach for economic aspects of an indian transmission system. bell, daniel 40 ajis vol. 9, no. 2 may 2002 and dunn (1999) have discussed the various security aspects on the distribution network from the operator's points of view. djordje, mcgillis, galiana, cheng and loud (1997) have developed an integrated knowledgebased model of the power-system planning process and a generic approach for power-system design. they used this model and approach to develop a system that can solve the most common planning and design problems in power systems. although their tool will not replace experienced human experts but it has the potential to automatically generate a wide set of design alternatives for a given power-system problem, while justifying its reasoning process. hart, uy, green, laplace and norosel (2000) have suggested various options for distribution automation in the context of current deregulated environment. khare and christie (1997) have discussed problems of dispatching active and reactive power on a large inter-connected power system to maintain security. it models operator's heuristics in the decision-making process by means of fuzzy sets. for various strategic, tactical and operational planning and control, decision support systems have been widely used. in large organizations and sectors, a single person rarely makes important decisions, more commonly the decisions are taken by a group of senior executives. a group decision support system (gdss) is an interactive computer-based system, which facilitates solution of unstructured problems by a set of decision-makers working together as a group. lim, raman and wei (1991) have studied the effects of group decision support systems (gdss) on decision making in various contexts, involving important variables such as group task and group size. tavana, kennedy and joglekar (1996) have presented a framework to help a group of decision makers define and articulate a hierarchy of hiring criteria and sub-criteria and rate each of the candidates on that hierarchy. hatcher (1992) presented a design of a geographically distributed group decision system that was used by the us army. they remarked that the ability to both see and hear a colleague either in real time or via a cdrom recording enhances familiarity as well as the quality of the decision process. teng (1993) proposed a functional taxonomy on two dimensions, content support and process support. the validity of the taxonomy was then demonstrated by fitting a large number of implemented or proposed system in gdss research onto one of the 16 types. saini, saxena and kalra (2000) described the application of internet technology for efficient management of the operational organizations for creation of synergistic intelligent group decision making and support systems. the designed group decision support system sigdmss is a user friendly, highly interactive intelligent group management system, based on inexpensive and efficient internet communication support, for group discussions, retrieval of content and expeditious decision analysis. the sigdmss is designed with synergistic integration of a collection of neoteric intelligent and pragmatic, conventional paradigms and techniques to develop a state-of-art decision-making and support contraption. most of the present group decision support systems contain the capabilities of communication technologies, computer technologies, database technologies and decision technologies to support the identification, analysis, formulation, evaluation, and solution of semi-structured or unstructured problems. the communication function results in any-place capability. due to lan, wan, teleconferencing etc. distances are becoming irrelevant, allowing users to share the same electronic workspace, whether in a face-to-face interaction or physically distant from each other. the data-base technology adds any-time capability. it permits users to work in same-time mode as well as in different-time mode (asynchronous), and in addition establishes an organizational memory. this paper adds another important dimension i.e. use of multimedia technologies in group decision support system to generate virtual (work) environments. multimedia is a new, exciting field, which represents the merging of three important industries: computing, communication and broadcasting. a multimedia environment consists of various kinds of media data such as image, graphic, audio and video. the complexity of multimedia applications creates a stress on all the components of computer system. a multimedia operating system should be able to support new data types and provide synchronization of multiple date streams via real time scheduling and fast interrupt processing. multimedia has many applications in various fields such as distance learning, electronic news distribution, medical application, poi/pos system, remote assessment, clinical information and messaging systems. hatcher (1995) explored the ideas about a multimedia tool kit and provided a focus for development of a theoretical framework to assist in guiding the applications in multimedia supported group/organizational decision systems. in this paper, an integrated multimedia based intelligent group decision support system (imigdss) for electrical power network is presented and its main components are analyzed and discussed. epn in recent time requires an intelligent, virtual environment based group decision process for the coordination of all its individual elements and the interrelated tasks. its ultimate goal is to achieve maximum productivity and efficiency through the efficient and effective application of generation, transmission, distribution, pricing and regulatory systems. group decision making process the key factor in a group decision-making, which is absent in individual decision making is the need for interaction between the group members during the decision making process. this has led researchers to focus on information exchange issues during group meetings in the design of group decision support systems. a group decision support system (gdss) also known as an electronic meeting system (ems) is a set of software, 41 ajis vol. 9, no. 2 may 2002 hardware, language components and procedures, that support a group of people engaged in a decision related meeting and its major function is to support the three common group activities information retrieval, sharing and use. the degree to which a gdss facilitates the communication or information exchange, has been used by desanctis and gallupe (1987) to classify gdsss. there is also a great need for intelligence sharing between group decision members (gdm) due to several factors. interaction is very much essential when a complex decision problem is divided into hierarchy of sub-decision problems, each of which is solved by an individual within the group. to make a meaningful attempt at solving or understanding the overall decision problem, a member might need the intelligence of another member to solve or interpret criteria of complex decision problem. radermacher (1994) discussed the scope of decision support system starting from the three words, decision, support and system that make up the term. based on a decision-theory point of view, the notion of decision and related cognitive concepts were discussed, particularly in connection with the idea of supporting human decision makers in making decisions. multimedia platform multimedia platform is now becoming a critical element in multi criteria decision making and group decision making. advances in hardware and software are serving as catalysts for multimedia applications. multimedia can easily be associated with computer-aided exchange or transformation of group information that involves simultaneous and coordinated use of different communication media such as video, audio, graphic and animation furht (1994). a multimedia operating system (mos) should be able to support new data types and provide synchronization of multiple data streams via real time scheduling and fast interrupt processing. in multimedia, if the end user sees that a multimedia application is allowed to control what and when the elements are delivered, it is called 'interactive multimedia'. in multimedia if end user is provided a structure of linked elements through which he can navigate, it is hypermedia. multimedia project is called "linear" if it starts at a beginning and runs through to an end and is called "nonlinear" if users are given navigational control to watch through the content at will and results in interactive multimedia. multimedia elements are put together into a project using software tools called 'authoring tools'. in addition to, providing a method for users to interact with the project, most authoring tools also offer facilities for creating and editing text and images, and have extensions to derive digitized video disc (dvd) players. essential multimedia peripherals, multimedia software tools [for painting and drawing-canvas, designer image studio, super paint etc; for cad and 3d macro model, auto cad, 3d studio, super 3d etc.; for image editing color it, colour studio, photoshop etc; for sound editingalchemy, audioshop, audiotrax, encore etc; for video and movie making-movie pak, animator pro, premiere, supervideo etc.; for accessoriescapture, image pals, photo disc etc.], multimedia authoring tools, query by image control (qbic) project, retrieval of multimedia [rom] object, continuous media storage etc. are essential for multimedia based group decision making. for effective and efficient group decision-making, multimedia environment must use virtual world tour guide (vwtg), multimedia group communication (mgc), multimedia use-friendly directory (mufd) features. description of emigdss electrical power is the engine of growth of any developing country. no major economic activity can be sustained without adequate and reliable supply of power. epn plays a unique role in supporting most of the infrastructure systems and the general public because of their dependence on electric power. thus it needs a very careful planning and control actions. the operation of such a complex system requires many decision attributes. the network control function of epn is performed from centralized energy control centers (eccs). these facilities may monitor generating facilities, transmission facilities and the distribution facilities. to monitor and control the epn, these facilities utilize scada systems and other management systems. a control center coordinates the operation of bulk power system component and is responsible for operating the power system within a geographic region called a control area. one or more utility make up a control area. a control center is connected to other control centers with transmission tie lines. through proper communications (metering and telemetry), the control center is constantly informed of generating plant output, transmission lines and ties to neighboring systems, and system conditions. a control center uses this information to ensure reliability by following reliability criteria and to maintain its interchange schedule with other control centers. the proposed imigdss is based on a black board problem solving architecture, where intelligent multicriteria futuristic decision process (mfdp) module based on multi criteria futuristic decision making methodology (singh & saxena, 1999) and multimedia platform associate to different epn application components to interact through a central black board (cbb). the basic components of the proposed imigdss architecture are presented in fig. 1. it underlines the main modules relating to the electrical power network load forecasting and planning (epnlfp); resource optimization modeling and simulation (romas); electrical power network control and evaluation process (epncaep). each module is customized with data base management 42 ajis vol. 9, no. 2 may 2002 system (dbms) and technology base (tb). the interaction between the group decision makers (gdm) and the global database takes place through a mfdp module for strategic planning, management control, operational planning and transaction processing with the tasks of defining the decision context, the main objectives, the criterias, the sub-criterias and the scenarios. the group facilitator coordinates the group use of technology and there is a dialog management system for use for both the facilitator and the gdm. the source code of imbigdss is written in 'c'. the system has many special features, which are very essential in a well-designed software package: there is no limit to problem dimensions other than the computer's physical memory and it has context-sensitive on-line help. it can be very easily converted to any other programming language version (including graphics, text output). it has full-screen editors in all stages of problem development. it is device independent. it automatically selects the best (highest resolution) graphics mode to display charts and other relevant information and the package can run on all ibm compatible machines. the dbms system the main task of this system is the simplification of the preparation and input of data in epn analysis programs and also control and verification of the bulk of data required by the application software for strategic planning management control operational control transaction processing application program epnlfp romas epncaep internal data external i data database management system dbms) •4—> t—r model base system technology base intelligent mfftp multimedia platform central black board dialog management system links to other nodes /jla group decision members fig. 1: imigdss for electrical power network 43 aj1s vol.9, no. 2 may 2002 application programs. this database supplies data to various other modules. the main advantages of dbms are: privacy, since access to the system is strictly protected via a series of password locks. protection against system corruption, through a backup and recovery mechanism. economical storage combined with satisfactory speed of retrieval. data independence from both hardware and application module through an appropriate architecture. integrity of data, because they are logically and syntactically checked at the time they are entered via built-in validation procedures. the fact that every data item is stored in a unique memory portion guarantees data consistency. very user-friendly operation through the use of interactive graphics and selection from menus of options. user-comprehensible organization of the data by the use of appropriately constructed tables and the implementation with relational model. the dbms is two-dimensional and the gdm are working with tables that have lines and columns like a spreadsheet. the gdm can easily travel through the data by the use of full-screen editor that permits them to scroll the data in all directions. the basic database of epn consists of generation, transmission and distribution data, affects on the environment, various cost and tariffs, maintenance scheduling and other charges. electrical power network load forecasting and planning (epnlfp) module the importance of accurate forecasts in generation and transmission planning is that it ensures the availability of supply of electricity, as well as providing the means of avoiding over and under utilization of installed capacity and making the best possible use of capacity. load forecast in transmission planning enables us to determine the amount of power to be transmitted to various regions from a generating station. large errors in forecasting can lead to bad planning which can be costly. in generation forecasting too high forecast lead to operation of more plant capacity than is required and unnecessary capital expenditure whereas too low forecast prevent optimum economic growth and lead to the maintenance of many costly and expensive-to-run generators. these additional costs will finally be borne by the consumers. transmission and distribution facilities, sales, profit, level of inventories, peak energy demands, nature and location of additional capacity of generation would depend on the forecasts. epn forecast can be classified as: short term forecasts (stf)the stf is necessary in planning the level and mix of generating capacity that will be used to support actual demand, and the sequence in which power stations are brought in operation. this will affect the fuel requirements and the associated overall costs. medium term forecast (mtf)the mtf is used in preparing operating plans, financial planning and tariff setting. mtf is used to develop final details of the next plant in terms of the unit size and station size etc. long term forecasting (ltf)the ltf of electricity consumption and demand is used in the planning of investment in generating capacity and the development of fuel supply. ltf is used for strategic decisions like optimum combination of plant types, determining the pattern of future expansions. different important social indicators like population growth, employment requirements, technological development, population densities, economic growth rate in agricultural and industrial sectors etc. are taken into account. generation planning for epn is normally based on following crucial factors: project locationthe location of site for a project will have a vital bearing on efficient and economic operation of the system. project management and layoutlayout and installation of a new generation unit will involve co-ordination of thousands of inter-related activities entrusted to various specialized agencies so as to meet the specified project schedule. delays in execution of a project will not only result in cost overruns, but will also hamper the national development. mode of generation and unit sizesduring generation planning one is confronted with possible alternatives either as regards mode of generation or unit sizes. in all such cases, the choice has to be finally made on economic considerations. once the choice is made, detailed plan for financing the project has to be proposed using financial analysis techniques. types of plantsbase load plants are nuclear, hydro and thermal with high load factor, high initial cost but lower operating cost. it is a better decision to consider installing more peaking units or plants with capacity to work in two-shift operation. timingtiming depends upon construction period, reliability desired, selection of size and type, pattern of expansion, assistance available through interconnection and expansion timetable of interconnected system. 44 ajis vol. 9, no. 2 may 2002 transmission planning for power system is based on the following factors: transmission management and layoutif a new line is to be constructed, it is very necessary to find out the route. before selecting a route we must see the existing population living in that area and other geographical aspects. the layout and installation of a new transmission line or upgrading of existing facility will involve coordination of thousands of inter-related activities entrusted to various specialized agencies so as to meet the specified project schedule. modes of transmissiondepending upon the amount of power to be transmitted, area to which power is to be transmitted and the cost involved in transmission we must select the mode of transmission. the mode of transmission may be ac transmission or dc transmission, it may be overhead line or underground cable transmission system. the criteria for choice will have to be based on economic, safety and security considerations. timingtiming depends upon construction or up-gradation period, reliability desired, selection of route and type of conductors required. the finances available and nature of expansion planning is also of great importance. distribution network planning and design requires decision on the following factorsdata base collection systemthe database of the distribution network includes the customer class, behavior, financial conditions, metering information, geographical lay out of the area and the expected demand in a specific period. system layoutthe manner in which the distribution system are split and re-split on their way from the substation to the consumer premises will affect the efficiency and economic operations of the distribution system. selection of switching and protectionswitching stations and substations are used to transform the electrical energy to a different voltage level. adequate protection equipment must be installed at the substations and at the consumers' premises. communication systemsthe communication system gives the electric utility an effective method for interrogating devices on the distribution system such as customer meters, switches, reclosures and fuses. the protocol selected for communication network for future integration, automation strategy and communication medium will have to be decided. timingtiming depends upon construction period, reliability and safety desired, selection of right-of way, pattern of expansion in a particular consumer class. resource optimization, modeling and simulation (romas) module in romas, various possible alternatives for epn layout are generated and designed in terms of optimal values in respect of economy, safety, reliability and quality of supply. then, various alternatives have to be compared with respect to a set of criteria defined in the epncaep module and the policy objectives established in the electrical power network mfdp module. intelligent mfdp module is used to allocate optimal resources and facilities. the romas module is based on group decision priority ranking (gdpr) model (singh & saxena, 2000) to give optimal group consensus ranking solutions and maximizes the overall futuristic decision priority. then a sensitivity analysis, based on the most extreme sets of parameter value combination is performed. the results of the model are relatively sensitive to changes in estimates used for such parameter values. the basic aim of romas module is the development and the application of tools for representing and measuring the impacts of the proposed objectives and criteria. from the point of view of optimization, modeling and simulation tools in a epn support system environment, there is full integration between different hierarchical levels and environment levels to provide users with an integrated framework. electrical power network control and evaluation process (epncaep) module the basic functions of epncaep module are commissioning, testing and checking the performance of the network against the expected or anticipated values, and in the event of performance being below the expected level taking corrective action to obtain improved performance. epn control is mainly achieved through generation scheduling, budgetary control and inventory control. scheduling of generation of different units is done to meet the set demand such that total generation is minimum and also deciding unit commitment during light load periods are important application of control. production and consumption will have to be simultaneous which demands co-ordination of power generation, transmission and distribution functions to achieve optimum efficiency. a master plan is developed for power plants, which is further developed into monthly and daily generation schedules. in a capital scarce economy like ours it is essential to ensure that expenditure increased in a project is at all times, proportional to actual progress made. private power handling utilities and state electricity 45 ajis vol.9, no. 2 may 2002 boards have the responsibility to develop and maintain an economic system of power supply so that only the minimum necessary capital is invested. budgetary control helps in realizing such economics and also furnishes information regarding actual position of capital commitments. installation and maintenance of sophisticated inventory control system in a power plant is vital as stock out cost is prohibitive in power industry and reliability is the test of efficiency for power utilities. an epn requires a huge store to handle a large volume of components and critical spares. the stock out cost being of very high order. the evaluation process in epn decision-making is a multistage and multilevel process based on the estimation and measurement of impacts according to time scales. electricity generation, transmission and distribution are continuous process. any laxity in these sets off adverse chain reactions culminating in the consumer dissatisfaction and suffering. different power system evaluation categories must be set according to the decisionmaking problems and context such as operational analysis or socio-economic evaluation. the aim of an operational analysis is to verify if the epn fulfills technical requirements and is reliable and effective with respect to detailed and specific technical objectives. in this case, alternatives correspond to different technical options and the evaluation criteria are technical indices. the corresponding impacts and effects are measured by impact analysis. the performance of epn is judged by the following standards: power supply facilities should be planned and executed in such a manner that, besides being uninterrupted supply, at no point of time shortage of power occurs. maximum production could be achieved from the installed power supply facilities by taking all measures, so that there is no labor or personnel unrest leading to disruption in power supply. ensuring adequate and stable supply of power throughout the service area as per the need of the users is then tacit obligation of the monopolistic nature of the power supply industry. to earn sufficient revenue not only to become self sufficient from revenue expenditure point of view but also to finance substantial part of new schemes. consumer's voltages to be maintained as close to the declared values as possible. the aim of socio-economical evaluation is to determine the final utility for the people affected. in this case, criteria do not refer to technical indices but to social gains and losses. central black board (cbb) a central black board model for epn is a conceptual architecture having independent and interactive knowledge based agents as data completion, data analysis and prediction and control functions. it stores a representation of the epn structures and services. the designed imigdss is based on a black board problem solving architecture where intelligent mfdp module and multimedia platform connected to various epn components interact through a cbb model. in this model database, local knowledge base and specialized technology base sources act upon a common globally shared database known as central black board, according to a strategy aiming at building a group problem solution, both cooperatively and opportunistically. the cbb model is well suited for real-time systems and is basically communication medium that enables information to be shared among the group users. this modular and distributed computing environment allows a kind of parallel processing and reasoning with the integration of intelligent mfdp module and multimedia platform, specialized for each of the different module. multicriteria futuristic decision process (mfdp) module mfdp module is a multi-person, multi-objective, multi-disciplinary, multi-level, multi-period, multidimensional futurologistic, group decision ranking module based, on feedback of multi disciplinary functional groups (mdfg) of electrical power network decision making group (epndm), electrical power network expert (epnep) group and inter-disciplinary futuristic (if) group; to simulate decision problems. mfdp module is a non-linear framework for carrying out both deductive and inductive intelligent thinking that allows the consideration of several factors at a time, along with a feedback mechanism and numerical trade-off, without the use of syllogism. the mfdp module is primarily based on following main principles: • the electrical power network current decision plan (epncdp) for the goal is developed by epnep group with the aid of the possible pictures of the future, characterized asa negative picture, a neutral picture and an optimistic picture. • the epncdp generated by outside in approach (oia) and inside out approach (ioa) is clustered into multi futuristic decision factors (mfdf) by epnep group. • the mfdf are prioritized by a structure group communication (sgc) technique to accomplish controlled feedback of if group, in and out contributions and flow of information and knowledge, assessment of the group judgment or view, opportunity for if group to revise views and degree of anonymity for the if group. 46 aj1s vol.9, no. 2 may 2002 • the prioritized mfdf of epn are pulled together through the principle of hierarchic composition to provide the overall futuristic decision priority (fdp) weights of the available mfdf of the system. • the stability of the outcome to changes in the importance of the mfdf is determined by testing the best choice against "what if type of change in the priorities of the mfdf. mfdp module allows groups to lay out a problem as they see it in its complexity and to redefine its definition and structure through interaction. through a mathematical sequence, multi disciplinary functional groups (mdfg) members synthesizes their futuristic judgments into an overall estimate of the relative futuristic priorities of alternative courses of action. it tracks inconsistencies in the multi disciplinary functional groups (mdfg) members futuristic judgments and preferences, there by enabling group decision makers to access the quality of their futuristic assessments knowledge and the stability of the solution. conclusions an interactive multimedia based intelligent group decision support system (imigdss) for electrical power network is presented in this paper. in particular emphasis is focused on database, model base, central black board and multicriteria futuristic decision process (mfdp) module. mfdp module provides a comprehensive framework for group decision making of multi-criteria problems of electrical power network by choosing the best one in a set of competing futuristic alternatives. the imigdss facilitates the exchange of information, data, processes in text, audio, video, animation and graphics form between the distribution network group members without the need for a meeting or direct communication. this model also incorporates the feedback from the end group users. the system can test existing as well new plans and several "what-if' situations and evaluate the associated impacts. it will help in operating at strategic, tactical and operational level, through the best use of the existing facilities. references arriaga i.j.p., hugh rudnick and walter o. stadlin (1995), "international power system transmission open access experience", ieee trans, on power systems, 10(1), 554-561. bell k.r.w., a.r.daniel and r.w. dunn (1999), "modeling of operators heuristics in distribution for security enhancement", ieee transactions on power systems, vol. 14, no. 3. desanctis.g. and r.b.gallupe (1987), "a foundation for the study of group decision support systems", management science, 33(5), 589-609. djordje, a., d.t. mcgillis, francisco d. galiana, john cheng, and lester loud (1997), "an integrated knowledge-based model for power-system planning", ieee expert/intelligent systems & their applications, vol. 12, no. 4. furht, b. (1994)," multimedia systems: an overview ", proceedings ieee, 1,47-59. hart, d.g., d.uy, j.northcote-green, c.laplace, d. norosel (2000), "automated solutions for distribution feeders", ieee computer applications in power, 25-30. handschin e., michael heine, dieter konig, torsten nikoden, thomas seibt and rodrigo palma (1998), "object-oriented software engineering for transmission planning in open access schemes", ieee trans. on power systems, 13,94-100. hatcher, m.e. (1992), "a video conferencing system for the united states army: group decisionmaking in a geographically distributed environment", decision support systems, 8,181-190. hissey t. (1994), "international experience in transmission open access", ieee power engineering review, 14(12), 3-18. khare, r. and r.d.christie (1997), "prioritizing emergency control problems with fuzzy set theory", ieee transactions on power systems, vol. 12, no. 3, 1237-44. lim, lai-huat, k.s.raman and kwok-kee wei (1991), "interacting effects of gdss and leadership", decision support systems, 12, 199-211. montfort, b. and p.lederer (1986), "generation planning at electricite de francea sharp focus for the decade", electrical power and energy systems, 8(2). parikh j., d. chattopadhyay (1995), "a multi-area linear approach for analysis of economic operation of the indian power system", ieee trans, on power systems, 11(1), 52-58. rudnick h., rodrigo palma, eliana cura and carlos silva (1996), "economically adapted transmission system in open access schemes applications of genetic algorithms", ieee trans, on power systems, 11(3), 1427-1440. radermacher, f.j. (1994), "decision support systems: scope and potential", decision support systems, 12, 257-265. 47 ajis vol.9, no. 2 may 2002 saini r., p.k.saxena and p.k.kalra (2000), " internet enabled synergistic intelligent systems and their applications to efficient management of operational organizations", information sciences, 127,45-62. singh preetvanti and p.k. saxena (1999), "futuristic decision priorities to eradicate tribal population by the year 2020 ad", advances in bio sciences, 18(1), 15-22. singh preetvanti and p.k. saxena (2000), "group decision priority ranking model for transportation planning", working paper, mhrd/engg.-pv, faculty of engg., dei, dayalbagh, agra, 1-12. tabors r.d. (1994), "transmission system management & pricing: new paradigms and international comparisons", ieee trans, on power systems, 9, 206-215. tavana, m., d.t.kennedy and p. joglekar (1996), "a group decision support framework for consensus ranking of technical manger candidates", omega, 24(5), 523-538. teng, j.t.c. and k.ramamurthy (1993), "group decision support systems: clarifying the concept and establishing a functional taxonomy", esfor, 31(3), 166-184. teofilo de la torre, j.w.feltes, t.g.s.roman and h.m.merrill (1999), "deregulation, privatization and competition: transmission planning under uncertainty", ieee transactions on power systems, 14(2), 460-465. vojdani a.f., c.f. imparato, n.k. saini, b.f. wollenberg and h.h. happ (1996), "transmission access issues", ieee transactions on power systems, 11(1), 41-51. 48 microsoft word ajis 11 1 final.doc ajis vol.11 no. 1 september 2003 3 the trichotomy of processes: a philosophical basis for information systems george r. widmeyer university of michigan school of business administration 701 tappan street ann arbor, mi 48109-1234 usa phone: 1-734-763-5808 fax: 1-734-764-3146 email: widmeyer@umich.edu abstract the principle of trichotomy from the american philosopher charles s. peirce can be used to categorize processes into the triad of transactional, relational, and informational. the usefulness of these categories is explicated by a comparison with structuration theory and control theory, and elaborated with a consideration of democracy in a knowledge economy. these three example applications of the process triad show the generality of the conceptual categories and provide a natural way of bringing ideas from social and ethical theories into information systems design. modeling the world and understanding business applications through the use of the trichotomy of processes should facilitate the development of more valuable information systems. keywords: business processes, information systems theory, conceptual modeling, ontology, peirce, open society introduction various frameworks for understanding the business processes of an organization have been proposed. there are linear, sequential models such as the porter value chain (porter 1980) or the plan, source, make, deliver, return of the supply chain operations reference-model (supplychain council 2000). there are cyclic models such as the customer service life cycle (ives and learmouth 1984) or the knowledge management life cycle (birkinshaw and sheehan 2002). each of these models starts with an examination of the goods or services that are delivered to a customer of an organization and, as such, are mostly grounded in a view of the flow of physical entities in an organization. this paper proposes an approach for information systems conceptual modeling based on a metalevel categorization of business processes. it uses the principle of firstness, secondness and thirdness as developed by charles s. peirce (american philosopher, 1839 – 1914) to generate a triad of categories for business processes, which is referred to in this paper as the principle of trichotomy. the resulting categories are identified as the transactional, relational, and informational. whereas peirce developed his principle as a metalevel distinction for generating categories by viewing entities from different perspectives this paper uses it to generate a triad of business processes. the rest of this paper is structured as follows. the next section describes peirce’s concept of firstness, secondness, and thirdness and the principle of trichotomy. this is applied to processes with the result being the trichotomy of processes. section three addresses social process from the perspective of the triads of giddens (1993), ouchi (1979), and popper (1966). this is followed by the application of the process triads to e-business and virtual organizations. the final section is some conclusions and suggestions for future research. trichotomy of concepts a fundamental concept in philosophy is ontology, which is the study of the categories of things that exist or may exist in some domain (sowa 2000, p 492). this section uses peirce’s principle of trichotomy to generate a triad of categories for business processes. ajis vol.11 no. 1 september 2003 4 trichotomy the ancient greeks, the scholastics, and many other philosophical traditions categorized the nature of things. immanuel kant (1787) developed a table of categories in four groups of three. kant believed that the triadic pattern was fundamental: “in every group, the number of categories is always the same, namely, three. that is remarkable because elsewhere all a priori division of concepts must be by dichotomy. furthermore, the third category always arises from a combination of the second category with the first.” (p b:110) the german philosophers continued this development. in particular, hegel’s (1831) development of a dialectical reasoning involved the triadic pattern of thesis, antithesis, and synthesis. peirce was critical of hegel’s logic and at the same time he was intrigued by his triads of categories (peirce 1998, p 428). peirce states in his cambridge conference lectures of 1898 that he had been “a passionate devotee of kant” but he also perceived “that there was something wrong with kant’s formal logic” (peirce 1992, p 124). his own analysis led him to conclude that some, but not all of kant’s triads reflected more basic categories, which he called first, second, and third. three conceptions are perpetually turning up at every point in every theory of logic, and in the most rounded system they occur in connection with one another. they are conceptions so very broad and consequently indefinite that they are hard to seize and may be easily overlooked. i call them the conceptions of first, second, third. first is the conception of being or existing independent of anything else. second is the conception of being relative to, the conception of reaction with, something else. third is the conception of mediation, whereby a first and a second are brought into relation. (peirce 1891, p 175) there are two key features of his categories: first, second, and third are of equal status; and, it is not necessary to go beyond the three, because fourth, fifth, and higher-order relations can always be defined in terms of the three. peirce’s principle can be used to generate differing perspectives for viewing entities (sowa 2000). firstness is the properties inherent in something. if x is a variable of type entity, then firstness can be defined by the monadic predicate p(x), which describes the entity x by its property p. secondness is a relation or reaction between x and another entity y, which is the dyadic relation r(x, y). thirdness involves mediation between two entities by a third, which can be represented by the irreducible triadic relation m(x, y, z). peirce emphasized that the third could not be represented using two seconds. sowa (2000) uses the principle of trichotomy to generate his three primitive categories for entities of independent, relative and mediating. as an example of the principle of trichotomy consider the diagram in figure 1 that would be common in a three-factor research model. figure 1. example of a three-factor research model factor x factor z factor y ajis vol.11 no. 1 september 2003 5 the properties of factor x are firstness and likewise for factors y and z. the relationship between factor x and y is secondness. factor z mediates the relationship between x and y; and hence, the third order relationship is thirdness. note that x, y and z are the entities, and the properties and relations are the firstness, secondness, and thirdness. most readers who know of the work of peirce are probably familiar with his three characteristics concerned with the functioning of a sign—the sign itself, its object, and its interpretant. peirce started his development of a theory of signs, which he called semeiotic, in 1867 and wrote many papers on it throughout his life (1906). his early ideas started with the trichotomy of icon, index, and symbol, but he later recognized that this relational trichotomy is based on secondness—the relationship between a sign and its object (sowa 2000, p 396). given his belief in the principle of trichotomy he then developed trichotomies based on the firstness and thirdness of signs (peirce 1906). this approach could be pursued for processes, but i will adopt peirce’s phrasing to state that my conception on this “is not yet quite free from mist” (peirce 1906, p 505) and hence this paper only presents a trichotomy (singular) for processes. a key idea from peirce is that the same sign can be called an icon or an index or a symbol; that is, it can function in multiple ways. if this idea is applied to processes then it means that a specific business process can have some properties that are considered firstness, and some secondness, and some thirdness. trichotomy of processes most ontological studies consider the properties and relations of things or entities. a different view of what exists in the world takes processes as its fundamental concept. in the book process and reality, whitehead (1929) develops a process-oriented ontology and eight categories of existence. these eight categories constitute two peircean triads, supplemented with two extra categories for generating combinations (sowa 2000). this paper adopts the process-oriented view and proposes the triad of transactional, relational, and informational as presented in table 1. the first row is from peirce (1891), the second row is from sowa (2000), and the third row is the result of applying the principle of trichotomy to processes. firstness secondness thirdness independent relative mediating transactional relational informational table 1. trichotomy of processes the transactional category for processes contains the basic property of all processes. a process has a state and transitions between states are events (arbib 1987). transactions cause the transitions between states, and how faithfully or reliably those transitions occur is a characteristic of the processes. this is firstness because it is a property of a process being or existing independent of any other processthe relational category for processes represents the relationship of one process with another. this category captures the properties of the relationship between two processes. this relationship need not be symmetrical; that is, it may be viewed differently from one process versus the other. also, it may specify a potential or possible non-negative mode of being between the processes. this is secondness because there is a dyadic relation between each process and each process is viewed relative to the other. the informational category for processes captures the creation of a relationship between two other processes. the strength of the relationship between the two processes is mediated by the third. this irreducible triadic relation between three processes specifies a pragmatic or intentional understanding of a process and provides a context for interpreting the data exchanged between two processes. this is thirdness since each of the three processes is necessary and the third mediates the relationship between the first two. consider the process of making a commitment. a possible property of the commitment process is its reliability, which is firstness. it takes two people or organizations to make a commitment and each participant in the commitment needs to be able to observe and respond to the other, the contracting relationship is secondness. thirdness is the property of a process that mediates two other processes. we could take “working together” as this process, and the strength of commitment is shown by how well the two people or organizations work together in achieving a goal. ajis vol.11 no. 1 september 2003 6 triads in social processes organizational research often uses dichotomies in the form of two-by-two frameworks. there are also three-factor or three-dimensional models but these are usually inductively developed from social or organizational experience rather than deductively from first principles. this section applies the philosophically(ontologically-) grounded trichotomy of processes to two particular organizational frameworks and elaborates the concept through a consideration of democracy in a knowledge economy. structuration theory the structuration theory of anthony giddens (1993) provides a connection between human action and social structure. the work of desanctis and poole (1994) and orlikowski and robey (1991) are examples where the concepts from giddens are used in information systems research. this section of the paper maps giddens’ triad of structures to the trichotomy of processes. the key principle in structuration theory is that of duality of structure—human action is enabled and constrained by structure, but structure is also the result of human action. this duality of structure can be understood as follows: agents communicate, exercise power, and sanction their own behavior and that of others; in doing so they produce and reproduce (with possible transformation) structures of signification, domination, and legitimation (orlikowski and robey 1991). this is a dynamic process and involves three modalities. the dimensions of social interaction can be represented as in table 2. process transactional relational informational structure domination legitimation signification modality facility norm interpretative scheme interaction power morality communication table 2. structuration theory and the trichotomy of processes the concepts on the first line are the categories for processes based on the peirce analysis from the previous section. the triad of structure based on giddens (1993) is listed in the second row. note that the order of the entries has been changed from the usual way that they are listed in tables and the order in which they are discussed by giddens (1993, p 129-30) and other authors based on his work. the concepts on the fourth line refer to the properties of interaction that occur between members of a community. modality refers to the mediation of interaction and structure. the use of power in interaction involves the application of facilities whereby participants are able to generate outcomes through affecting the conduct of others. the moral constitution of interaction involves the application of norms. these norms are based are based on structures of legitimation. finally, the communication of meaning in interaction involves the use of interpretative schemes. these are cognitive schemes that depend upon a shared understanding by a community. giddens (1993, p 130) states that “just as communication, power and morality are integral elements of interaction, so signification, domination and legitimation are only analytically separable properties of structure.” this is similar the peircean constraint on his principle of trichotomy in that first, second, and third are of equal status and it is not possible to reduce a third to a pair of seconds. the duality of structure and process is evident in table 2. the transactional properties of a process are related to the ability or facility to change the state of the process. structures of domination control the resources of the process and can bring about changes through the exercise of power without the influence of other processes. this is firstness. the relational category of a process is related to the use of norms in the exercise of authority. structures of legitimation control the use of moral rules of the process. the morality of the interaction determines the type of relationship between processes by sanctioning their own behavior and that of others. this is secondness. the informational category of a process is related to the use of interpretative schemes in communication. structures of signification control the semantic rules of the process. information ajis vol.11 no. 1 september 2003 7 can be defined as an interpretation of data. the relationship between the process and the semantic rules provides the meaning to the participants in the communicative interaction. this is thirdness. the mapping of the triad of structuration theory of giddens to those of the trichotomy of processes is consistent with giddens’ duality of process and structure. this mapping helps in the explication of the transactional, relational, and informational characteristics of processes. the next section repeats this approach using a framework for the three modes of control of ouchi. the social prerequisites of control the design of organizational control by ouchi (1979) provides three control mechanisms: market, bureaucracy, and clan. he arranges the three modes of control along two dimensions: the informational requirements necessary to operationalize each control type, and the social underpinnings necessary to operate each control type. the social requirements are a set of agreements between people which are necessary for a form of control to be employed. according to ouchi (1979, p 838) a market cannot exist without a norm of reciprocity that assures that if one party in a market transaction attempts to cheat another, then the cheater, if discovered, is punished by all members of the social system. he goes on to state that a bureaucracy contains not only a norm of reciprocity, but also agreement on legitimate authority. finally, the clan form requires not only a norm of reciprocity and the idea of legitimate authority, but also social agreements on a broad range of values and beliefs. “in reality, of course, we will never observe a pure market, pure bureaucracy, or a pure clan. real organizations will each contain some features of each of the modes of control” (ouchi 1979, p 840). therefore, this paper just maps the social prerequisites of control to the trichotomy of processes as shown in table 3. process transactional relational informational social requirements legitimate authority norm of reciprocity shared values and beliefs table 3. social prerequisites of control and the trichotomy of processes the transactional properties of a process are related to the exercise of legitimate authority that is recognized as the right of higher organizational roles to command and to audit or monitor those holding lower organizational roles, within some limits. the relational properties are based on norms of reciprocity that require information for their exercise. finally, the depth of the level of common agreement on values and beliefs mediates the strength of the relationship between processes. these three social prerequisites are parallel to the structures of giddens described in the previous section. the open society the consideration of the role of democracy in a knowledge economy can lead to an appreciation of karl popper’s (1966) concept of the open society. the open society and its enemies by karl popper was written during “the grave years” of 1938 to 1943 “when the outcome of the war was uncertain” (popper 1966, p viii). he provides a critical view of the philosophy of politics and of history with volume one having plato and volume two having hegel as “enemies” of the open society. “the magical or tribal or collectivist society will also be called the closed society, and the society in which individuals are confronted with personal decisions, the open society” (popper 1966, p 173). as an example of characteristics of the open society he quotes pericles’ ideal of democracy. our administration favors the many instead of the few: this is why it is called a democracy. the laws afford equal justice to all alike in their private disputes, but we do not ignore the claims of excellence. … we consider a man who takes no interest in the state not as harmless, but as useless; and although only a few may originate ajis vol.11 no. 1 september 2003 8 a policy, we are all able to judge it. [emphasis in popper] (popper 1966, p 186) the idea of an open society can be founded on the concepts of fallibility, reciprocity, and the diversity of values, which conform to the peircean triad of firstness, secondness, and thirdness. we want our processes to be reliable, but we must accept that they are fallible. much conceptual effort is devoted to the design of business transactions that are reliable. if a business transaction fails then the process is generally designed to “rollback” to a previous state, but social transactions cannot be so easily rolled back. social organizations fall short of perfection and must be designed on processes that are always held open for improvement. a key property of these processes then must be fallibility, which is firstness. this concept of business transactions as having the property of fallibility is consistent with popper’s concept of falsification in science (popper 1989). it simply means that we can err in our choice or fall short of the truth. this does not mean that our process are irrational, only that we cannot be certain. he states that we should consider that “… all the known historical examples of human fallibility—including all the known examples of miscarriage of justice—are examples of the advance of our knowledge (popper 1966, p 375-6). our consideration of democracy in a knowledge economy should include the condition of fallibility in our processes, but this recognition of fallibility is a necessary but not sufficient condition. we must combine it with some degree of concern for our fellow human beings based on the norm of reciprocity (soros 1998). the relational category of business processes involves monitoring and reciprocity. if we go back to ouchi’s social prerequisites of control described in an earlier section, then we see that norms of reciprocity require data for their exercise. there must be a category of processes that includes these types of processes. at the same time, information is an interpretation of data, which means that it has a subjective nature that is partially determined by the social context of the process. the informational category of business processes depends on values and beliefs that mediate the strength of the relationship between processes. whereas ouchi (1979) describes these as “shared values and beliefs,” i use the phrase “diversity of values” that i think is more in line with popper’s open society concept. certainly, shared values hold a society together, but acceptance of a diversity of values keep a society open to improvement (soros 1998). the strength of the relational nature of our processes is mediated by the diversity of values and these provide a context for understanding social interactions. the consideration of democracy in a knowledge economy can be framed in terms of the concept of the open society or organization. information systems enable the business processes of an organization and this paper proposes that these should be categorized into transactional, relational, and informational. the trichotomy of processes provides the categories of different settings in which democratic principles can be explored, but rather than continue the next section focuses on business domain modeling to support the development of information systems. application to information systems information technology enables process improvement, but many people can only think of ways to automate business processes. if they would also think of the relational and informational aspects of business process then they would have a much richer view of the value of information technology. this section describes two application areas in business that can benefit from the conceptual foundation provided by a trichotomy of processes. electronic marketplaces on the internet markets play an important role in both the traditional economy and the new economy, which is characterized by electronic networks and knowledge workers. the markets do not require a physical location and participants can be located globally and yet interact (almost) simultaneously. consider the functions of an auction-based market system (bakos 1998). there needs to be systems of communications to match buyers and sellers, which are based on shared interpretative schemes for product representation and pricing. there needs to be systems of workflow to facilitate the ajis vol.11 no. 1 september 2003 9 transactions. these support the fulfillment processes of an exchange of property rights between participants. the relative power of the participants in the market system generally determines the design of the workflow processes. finally, there needs to be systems of authority to provide the institutional infrastructure, which legitimates the efforts of the actors and supports dispute resolution. these functions can be related to the three c’s of electronic commerce: content, commerce, and community. these relationships are summarized in the table 4. process transactional relational informational market functions facilitation of transactions institutional infrastructure matching buyers and sellers e-commerce commerce community content table 4. electronic markets and the trichotomy of processes the transactional properties of market processes are related to the facilitation of the transaction. the acceptance of community norms determines the interaction between participants. the institutional infrastructure relates the participants and this secondness cannot be eliminated from the market without it ceasing to function. the informational relationship depends on the content of the process. information is needed about both the buyers and the sellers in order to do the matching and the understanding of the content mediates the strength of the relationship between the buyers and sellers. virtual organizations the trichotomy of processes provides a starting framework for conceptualizing information systems that support a virtual organization. most recent literature identifies the two general perspectives of structure and process in describing the characteristics of a virtual organization, but generally the process view focuses on the process of change (saabeel, verduijn, hagdorn, and kumar 2002). an alternative view is to start with the fundamental transactional, relational, and informational processes of any organization and then ask how these are different in a virtual organization. the word “virtual” as an adjective means having the function but not the form. a virtual organization performs the function and has the processes of a non-virtual organization, but the form (structure) is different. the result should be evident in the behavior of the organization. any set of processes generated by the trichotomy of processes principle should be evident in both types of organizations. my conjecture is that the information systems designed to support such processes are basically the same in both the virtual and non-virtual organizations. what may be different to those companies that compete most vigorously in a knowledge economy is the relative emphasis on the three types of processes, but this emphasis should depend less upon the virtualness of the organization and more on the knowledge intensity of the processes. research contribution triads of categories are evident in many of the groundings that have been used in information systems research. this paper explores how these can be unified through a philosophical foundation for information systems. it specifically uses just one concept from the work of charles s. peirce; albeit, a concept that he considered fundamental to most of his philosophical thinking. any “philosophy of information technology” needs to include ontology, and the trichotomy of processes provides a starting point. a second contribution of this paper is that it uses the principle of trichotomy to generate a framework that unifies an understanding of organizational communication and information systems. the triad of transactional, relational, and informational provides a framework for viewing the giddens’ (1993) triad of structuration theory: domination, legitimation, and signification. it provides a view of ouchi’s (1979) triad of the social requirements of control: legitimate authority, norms of reciprocity, and shared values / beliefs. a third example of the proposed process triad is to the three c’s of e-commerce: commerce, community and content. these three example applications of the process triad show the generality of the conceptual categories and ajis vol.11 no. 1 september 2003 10 provide a natural way of bringing ideas from social and ethical theories into information systems design. finally, a practical contribution of the trichotomy of processes is that it can provide principles for business process improvement. the trichotomy is useful in thinking about it-enabled knowledge work. consider the processes that need to be supported by a web-based portal for an organization. as a first principle—the design of the portal needs to provide services for transactional, relational, and informational processes. as a second principle—the design of the portal needs to support fallibility in the transactional processes, reciprocity in the relational processes, and diversity of values in the informational processes. these principles build on the argument presented in this paper, but they need to be tested in practice; even so, it is hypothesized that starting information systems design from these two principles should facilitate the development of more valuable information systems. conclusions this paper lays a foundation for information systems conceptual modeling based on a categorization of processes into the triad of transactional, relational, and informational. the paper is conceptual in that it describes the three categories of processes and explicates them using giddens’ (1993) structuration theory, ouchi’s (1979) social prerequisites of control, and popper’s (1966) concept of the open society. the categories of processes can be applied to the design of information systems such as e-business applications or in the design of applications to support virtual organizations. stakeholders must be able to classify phenomena in the modeled domain and the generic concept of process could be refined with the trichotomy of processes. this can make it easier for stakeholders to model and understand the domain for the purpose of information systems development. the extensions to current conceptual modeling grammars and the development of new methods can both benefit from the ontological considerations presented in this paper. the limitations of this current research is that it is conceptual and does not propose actual constructs for a grammar or how to extend current grammars such as data flow diagrams (dfd) or entityrelationship diagrams (erd). another area of future research could be directed to the development of new methods of using a current conceptual modeling grammar (wand and weber 2002) that is informed by the trichotomy of processes. references arbib, m. a. (1987) brains, machines, and mathematics, 2nd edition, springer-verlag, new york. bakos, y. (1998) “the emerging role of electronic marketplaces on the internet,” communications of the acm (41:8), pp 35-42. birkinshaw, j., and sheehan, t. (2002) “managing the knowledge life cycle,” mit sloan management review (44:1), pp 75-83. desanctis, g., and poole, m.s. (1994) “capturing the complexity of advanced technology use: adaptive structuration theory,” organization science (5:2), pp 121-147. giddens, a. (1993) new rules of sociological method: a positive critique of interpretative sociologies, 2nd ed., stanford university press, stanford, california. hegel, g. w. f. (1831) science of logic, translated by a. v. miller, humanities press international, atlantic highlands, new jersey, 1989. ives, b., and learmonth, g. p. (1984) “the information system as a competitive weapon,” communications of the acm (27:12), pp 1193-1201. kant, i. (1787) critique of pure reason, edited p. guyer & a. w. wood, cambridge university press, cambridge, england, 1998. orlikowski, w.j., & robey, d. (1991) “information technology and the structuring of organizations,” information systems research (2:2), pp 143-169. ouchi, w. g. (1979) “a conceptual framework for the design of organizational control mechanisms,” management science (25:9), pp 833-848. peirce, c. s. (1891) “the architecture of theories,” monist (i: 2), pp 161-176. ajis vol.11 no. 1 september 2003 11 peirce, c. s. (1906) “prolegomena to an apology for pragmaticism,” monist (xvi: 4), pp 492-546. peirce, c. s. (1992) reasoning and the logic of things: the cambridge conferences lectures of 1898, edited by k. l. ketner, harvard university press, cambridge, massachusetts. peirce, c. s. (1998) “pragmatism,” in the essential peirce, volume 2, the peirce edition project (eds.), indiana university press, bloomington and indianapolis, indiana. popper, k. r. (1996) the open society and its enemies, 5th edition, princeton university press, princeton, new jersey. popper, k. r. (1989) conjectures and refutations: the growth of scientific knowledge, 5th edition, routledge, london, england. porter, m. e. (1980) competitive strategy: techniques for analyzing industries and competitors, free press, new york. saabeel, w., verduijn, t. m., hagdorn, l. & kumar, k. (2002) “a model of virtual organization: a structure and process perspective,” electronic journal of organizational virtualness (4:1), pp 1-16. soros, g. (1998) “toward a global open society,” the atlantic monthly (28:1), pp 20-32. sowa, j. f. (2000) knowledge representation: logical, philosophical, and computational foundations, brooks/cole, pacific grove, california. supply-chain council. (2000) supply-chain operations reference-model: overview version 4.0. wand, y., and weber, r. (2002) research commentary: information systems and conceptual modeling—a research agenda. information systems research (13:4), pp 363-276. whitehead, a. n. (1929) process and reality, corrected edition edited by d. r. griffin and d. w. sherburne, the free press, new york, 1978. microsoft word ajisfinal 12 2 marina.doc ajis volume 12 no. 2 may 2005 103 researching human experience: video intervention/prevention assessment (via) jennifer patashnick michael rich video intervention/prevention assessment (via) children’s hospital boston 300 longwood avenue boston, massachusetts 02115 email: jennifer.patashnick@childrens.harvard.edu abstract human experience is a critical subject for research. by discussing video intervention/prevention assessment (via), a patient-centered health research method where patients teach their clinicians about living with a chronic condition through the creation of visual illness narratives, this paper examines the value of qualitative inquiry and why human experience rarely is investigated directly. an analysis of a sample via data is presented to demonstrate how, by utilizing grounded theory and qualitative analysis, one can derive rich and unique information from human experience. introduction human lives are continuous streams of experience. at every moment of every day, our senses are bombarded by a variety of stimuli. we synthesize experience by filtering and selecting stimuli of importance, then organizing those perceptions into concepts that can be related to others and on which we can act. as infants, our experiences fundamentally alter and shape our brains into an architecture that can interact effectively with the environments that influence their development. experience is the basis of learning; synthesizing and communicating experience is the basis of teaching. historically, all information was communicated through the use of stories, transmitting knowledge from person to person and from one generation to the next. we understood and trusted narrative as a means of conveying important data to each other. however, with the development of the scientific method and the descartian view of the world, we became enamored with our ability to measure, quantify, and be “precise.” we came to the belief that to be truly scientific, information had to be numerical. positivism regards only what is quantifiable to be of scientific value (comte 1851/1865). if one cannot assign a numerical value to an observation, one should not attempt analysis because non-numerical data cannot be manipulated and compared. without a standard value against which to compare, measurement is imprecise and the validity of the observation is questionable. so how does one investigate and analyze life experience in a way that is not only useful but also scientifically valid? in striving to be evidence-based, medicine often leans toward a positivist evaluation of outcomes. while the number of hospital admissions or total cost of antibiotic therapy can be useful for evaluating effectiveness of health care delivery or for standardizing treatment, it does not address the core issues of illness, health, and health care. because issues such as quality of life or why patients do not adhere to medical plans are difficult to measure, much of the scientific community has agreed that translating experience into such quantifiable factors is the best means of evaluating health and health care. this approach is indirect and the outcomes are selected as much for their measurability as for their salience. ajis volume 12 no. 2 may 2005 104 experience as data a frequent concern about the analysis of human experience using narrative is reliability of recall. studies have shown that individuals asked to provide details about certain events, such as what they ate that day, do not exhibit perfect recall nor report everything they consumed (armstrong et al. 2000). however, what individuals do remember is that which is the most salient for them about the experience (fries et al. 1995). in medicine, it is the patient’s experience, understanding, and beliefs about the origins and meaning of illness that most strongly determine his/her response to illness and adherence to medical plans (rich et al. 2002). yet only recently have questions in the biomedical realm been approached with narrative research methods, and this work is far from universally accepted or encouraged. another reason for this lack of acceptance may be the time required for each patient to present his/her own individual perspective on the illness to the clinician. however, by understanding the emotional and psychological components of the patient’s life experience and considering them in developing the treatment plan, the clinician may be able to more effectively address how the patient’s life affects the patient’s illness and improve his/her prognosis. the distinction between disease and illness is a crucial one. disease, in this case, can be defined as the biological dysfunction that caused the patient to seek medical care (kleinman 1988, morse & johnson 1991). illness is how the patient experiences disease in the context of his/her own life (rich et al. 2000c). three patients with identical disease states (as measured using universally accepted clinical criteria) might experience three distinct illnesses, with differing symptomatology, levels of disability, and quality of life. a considerable amount of research on the mind-body connection indicates that one’s mental state can dramatically affect one’s health (melmed 2001). so why is the patient’s experience of illness so frequently ignored? for some, the use of human experience as data may challenge the perception of the objectivity of medical diagnosis. using evidence based medicine, doctors base treatment plans on neutral “scientific” data (rosenberg & donald 1995). in western medicine, recording measurable data, such as a patient’s heart rate, blood pressure, and temperature, is a staple of clinical assessment. clinicians rely on these measurements to determine a patient’s health status in comparison to known healthy values. an abnormally elevated temperature can indicate the presence of an infection. increased temperature is a sign of disease, a relatively predictable, common response of human physiology. psychological response to illness, however, can have a profound effect on one’s perception of health, one’s response to disease, and one’s ultimate outcome. these responses are individual, unpredictable, and cannot be quantified as easily as body temperature. in order to understand a patient’s illness, a clinician must learn from the patient what s/he experiences, what meaning s/he has made from that experience, and what significance s/he has assigned to the experience. by inviting the patient into a health care partnership, the clinician and patient together can target treatment strategies to address the features of the illness most important to the patient and develop a medical management plan to which the patient is motivated to adhere. patients’ experiences of their health and illness introduce subjectivity to the medical evaluation, which is difficult to measure. one way to investigate experience is to collect as many data as possible from the constant stimuli of sights and sounds to which an individual is exposed, while simultaneously providing the individual with an opportunity to narrate the scene. the documentation of stimuli preserves the experience for later reflection and analysis, while the accompanying narration allows the incorporation of the individual’s subjective, first-person point of view. then, an analytical strategy for assigning importance to the stimuli and narrative must be developed. traditional scientific and biomedical protocol would suggest that one start counting events and ajis volume 12 no. 2 may 2005 105 assigning numbers to quantify data. indeed, video intervention/prevention assessment (via) data include quantifiable information such as individual demographics or numbers of observed events. however, none of these measurements effectively relate the illness experiences or the individuals’ responses to those experiences. testable hypotheses about much of this quantitative data can be developed before the study begins. in contrast to traditional hypothesis-driven research, grounded theory seeks to collect and organize data until a theory emerges, constantly testing and modifying the theory to fit the data. grounded theory allows researchers to collect a broad range of data, discovering critical information for which they did not go looking. in this way, the researcher need not predict what might be found by a specific line of inquiry, but can let the data “speak for itself” (glaser 1992, glaser & strauss 1967, strauss & corbin 1990). the via method allowing subjects to represent and document themselves and their experiences is not a new idea in academia. numerous studies in a multitude of disciplines including anthropology (worth & adair 1997), psychology (furman 1990), and education (urban 1989) have demonstrated the value of the individual’s own perspective on their lives. when patients at children’s hospital boston are enrolled in via, they are asked to “teach your doctor” what it is like to live with a chronic condition. via seeks to improve the heath care of chronically ill patients by more actively including the patient’s life experience in his/her treatment. with the elimination of house calls, clinicians are no longer seeing patients in the context of their everyday lives. further, since limits are imposed upon clinical visits by insurance companies and other external forces, actual time spent with each patient has been reduced in some cases to mere minutes. data collection via was started in 1994 at children’s hospital boston by michael rich, md, mph in response to what was perceived to be an ever-increasing gap in communication between clinicians and their patients, particularly children and adolescents (rich & chalfen 1999, rich et al. 2000b). via bridges this void by loaning participants video camcorders and encouraging them to record anything and everything they feel is pertinent to their lives with their condition, particularly those factors about which doctors may not ask or be aware. after informed consent is obtained, via participants are loaned lightweight, handheld camcorders, given access to an unlimited supply of blank tape, and encouraged to videotape as much as possible over the course of several months. in this way, large quantities of data may be accrued. a joint decision between the participant and a via researcher determines when the participant’s visual narrative has been completed. some participants have collected only a few hours of videotape; others’ visual narratives constitute several days’ running time. nearly all is valuable for its unique ability to convey the participant’s view of his or her life. however, none of it can be analyzed effectively using quantitative methods. once a participant has completed his/her visual narrative, it is logged in detail. these logs are used as guidebooks to the data, so that a multidisciplinary team of physicians, anthropologists, social workers, and psychologists can review any or all of the audiovisual data for information salient to the study being conducted. this yields a multidimensional perspective on living with a chronic condition in the context of the patient’s own life. as an example, consider the following scene: js, an 18 year old male with sickle cell disease, is talking with his friend jerry in a bathroom. js, clearly inebriated, is drinking beer and smoking a cigarette while he discusses his initial experiences with alcohol and tobacco. jerry is skeptical of js’s claim that he could quit smoking and drinking at any time. js then says he could quit, but doesn’t ajis volume 12 no. 2 may 2005 106 want to. jerry tells js that he should stop because of his sickle cell disease. js resents this, curses sickle cell, and says, “i want to live my life like i never had sickle cell.” js suggests that if he didn’t have sickle cell disease, then he wouldn’t drink or smoke so much, while jerry predicts js would drink even more. although the scene is only three minutes long, it contains many distinct themes and layers of meaning. the complexity and density of important information in these visual illness narratives are what led the via research team to turn to computer-assisted qualitative analysis. the pilot via project on asthma (rich et al. 2000a) was originally analyzed on paper. however, as sophisticated software was developed for qualitative analysis, we explored the use of several programs to determine which could facilitate our process most readily. the analysis which can be done on a computer is much more comprehensive than that which can be performed manually, but many qualitative analysis programs are ill-equipped for what we wished to accomplish: a more complete and textured understanding of patient experience. previous studies using audiovisual material analyzed brief, “micro” sequences of human behavior where each frame of video was of interest and predictable events, such as a child kicking a ball in a playground, could be quantified (tronick et al. 1991). via, however, is concerned with “macro” features of human experience that could not be predicted and needed to be elicited from hundreds of hours of visual narrative that participants collected. via is less interested in how many times the child kicks the ball than it is in why the child is at the playground, what he does, how he interacts with other children, and if and how the child’s medical condition affects these behaviors. the quantity of videotape collected represented a significant data management challenge. additionally, the density and complexity of detail in the audiovisual data allowed for significant nuance of meaning, with overlapping layers of events, personalities, and emotions present in each scene. each observer of human behavior, on tape or otherwise, has their own interpretation of the event, the primary criticism of those who question the validity of this type of research. of paramount concern was standardization and reproducibility of via analysis, particularly in the initial stages of analysis. this intricate combination of factors caused the via research team to develop an unprecedented analytical method using text logs and nvivo (1998). because no current qualitative analysis software program can directly handle the quantities of audiovisual data collected by via easily and flexibly without requiring immense computer resources, via visual narratives are “transcribed” into logs, text intermediates which function as a “road map” of the video data. the creation of the log is a detailed process. since a picture really is “worth a thousand words”, the logs must convey in text the content of the visual narrative without losing the richness and dimensionality of the original video. the via research team realized that there are two simultaneous streams of information in all audiovisual scenes: objective and subjective. loggers are trained to readily identify and document each in a specific text format. objective elements are those which are directly observable and emotionally neutral, descriptions that could be generated by and agreed upon by any observer. returning to the scene previously described, an example of objective data is the observation that js is holding a bottle and leaning against a door. in the development of the logging process, the via research team had multiple loggers review the same tapes, and had each logger “transcribe” the tapes several times. little variation was found between the logs, particularly in the case of the objective observations. therefore, via has both synchronic and diachronic reliability. subjective data are elements such as the emotional tone or content of the scene and may vary based on how much of the context is known by the observer, how the observer relates to those in the scene, and even the age and gender of the observer. in our example scene, the logger subjectively notes that jerry seems to be acting as js’s conscience “like jiminy cricket [from the animated film of pinocchio]”. this analogy, as descriptive and apt as it was, may or may not have occurred to a ajis volume 12 no. 2 may 2005 107 different observer. each logger’s subjective observations serve to further describe the emotional tone and enhance the psychological elements of the narratives. figure 1 shows the completed log of the example scene. note that objective observations appear in normal text, while subjective observations are in italics. (for further details of the logging procedure, please refer to rich & patashnick 2002.) figure 1: example scene logged in via format data analysis once the log of the visual narrative is created, it is imported into nvivo, a software program designed by qsr for the management, analysis, and structuring of qualitative data. nvivo allows via researchers to draw meaning from the vast quantity of visual narratives through coding. following the edicts of grounded theory, coding starts with a clean slate where no themes are anticipated or predicted from the data in advance. any theme which emerges from a review of the log can be assigned to a node. nodes are labels for all data which share a common theme or code. for our purposes, a node can contain pieces of data that are as short as a single word or as long as several pages of text, encompassing any time interval of the original video from a frame to the length of the tape. each passage of text, analyzed from various perspectives, can be assigned to an 04:01/00:31:21 jerry films js, who is leaning against the wall in a bathroom, holding and drinking from a bottle of beer. jerry says, “you see what js’s got in his hand…a man with sickle cell drinking that shit.” js says “i’ve been doing this since i was…how old was i?” jerry says “i don’t know, but i think you’re a little drunk.” js says that he was twelve years old when he first had alcohol and that he didn’t like it. jerry asks, “why’d you do it?” js responds “just to try it. and i ended up throwing up.” js adds it wasn’t peer pressure that convinced him to try drinking. he doesn’t let others pressure him into things, but he ends up participating anyway maybe to add excitement to his life? 04:01/00:32:08 jerry asks if js would make the same decisions about alcohol if he could go back in time. js says no. jerry asks the same question about cigarettes and js says “no, hell no!” js smokes a cigarette during this conversation and says he is addicted. he seems to be aware that these substances are bad for his health and doesn’t want to have these habits, straightforwardly admitting regret. his friend jerry seems insightful, and acts like js’s conscience, almost like jiminy cricket! 04:01/00:32:38 jerry suggests that there are nicorette patches and js says that if he wanted to quit, he could quit. jerry says, “no you couldn’t…that’s bullshit.” js says if he really wanted to, he thinks he could quit cold turkey, but he just doesn’t want to. js says “i don’t want to quit cigarettes and i don’t want to quit drinking and f****** smoking weed.” jerry says “but you should though, that’s the whole point.” js interrupts jerry and says, “why? why why why? because i got sickle cell?” jerry responds “exactly.” very clear from js’s tone that he resents sickle cell ruling his life. 04:01/00:33:25 js continues in a raised voice “no, but the thing is, just ‘cause i have sickle cell, i want to live my life like i never had sickle cell.” jerry says, “that’s just it…because you have sickle cell, you’ve got to do different things.” 04:01/00:33:35 js waves his middle finger to the camera and says “f*** sickle cell!” he laughs. jerry says “that’s too bad because it’s flowing through your bloodstream right now.” 04:01/00:33:43 js continues “i don’t know; that’s the problem. i got sickle cell. you know, that’s it. that’s what pisses me off. that’s the whole fact of the whole shit is i got sickle cell. that’s it. that’s it. if i didn’t have sickle cell, do you think i’d be doing this? maybe. maybe not.” jerry responds “actually, i think if you didn’t have sickle cell you’d have one in that hand and one in that hand right there,” as he films the bottle of beer js is holding. ajis volume 12 no. 2 may 2005 108 infinite number of distinct nodes which can seamlessly overlap. a selection of codes for the example scene are presented in figure 2 as coding stripes. the stripes show which sections of the log have been labeled with various codes. figure 2: logged scene with nvivo coding stripes nvivo keeps track of the nodes as they are created in a list format. as each participant’s log is coded, the list of nodes expands. this growth is rapid at the start of coding and tapers off throughout the process as codes that characterize the patients’ experiences are defined and standardized. at the conclusion of coding (or several coding passes as necessary), more complex questions can be asked of the coded data. for example, one could ask in what ways participants tried to normalize their illness experience. to do this, one would create a query that would identify all the passages of the logs where a specific reference to an illness and a code of “normalcy” occurred concurrently, as they do in figure 2. one may be looking for the answers to any or all of the following questions: is js the only participant who seeks to normalize his illness through his risk-taking? in what other “typical” adolescent behaviors do adolescents with chronic illness choose to engage? if one were to analyze these same data by hand, each question would cause the ajis volume 12 no. 2 may 2005 109 researcher to have to re-watch hundreds of hours of video or re-read thousands of pages of logs. instead, nvivo keeps track of all codes, and one needs only to ask the questions of the data to be presented with examples that fit the search criteria. this is not to say that using qualitative software is without challenge or that the computer does the thinking for the researcher. certainly, one needs to employ appropriate strategies when querying the data to avoid nonsensical search results. simply put, one must know what one is asking. codes must be consistently applied and defined, and searches carefully constructed. for verification of a code, the via research team can go back to the original visual narrative at any time, and view the sections they are coding to gain even more context from the scene to confirm the applicability of a particular code. nvivo allows researchers easy, direct access to the original audiovisual data through the use of databites linked to sections of video that can be embedded in the appropriate sections of the log document. the ubiquity of experience and the value of via to date, via has focused solely on medical conditions in pediatric, adolescent, and young adult populations, but the versatility of the via method allows the application of the technique to virtually any human condition. via has the potential to explore social issues such as violence, teen pregnancy, or homelessness and life events such as the college experience. with widespread use of video cameras by the general public for the creation of home movies, and the increase of so-called “reality” television shows, one might question why what via is doing is considered so distinctive. first, what sets via apart is the intent of each project and the motivation of the participants. the goal is for clinicians to gain a better understanding of their patients’ lives with chronic conditions. in the traditional doctor-patient interaction, doctors hold the position of power. yet, it is only when patients take ownership of their condition and their day-to-day medical management that their health can improve. by asking patients to teach doctors about a health condition and giving them a camcorder, the power shifts quite literally into the hands of the patients, the true “experts” on their own lives. the knowledge and experience that the patient contributes will help clinicians to partner effectively with their patients by communicating with them as equals to create treatment plans with which both parties are comfortable and to which both are committed. second, most “home movies” are created because of an event such as a birthday party, graduation, or vacation. since these events do not happen every day, people consider these occurrences “special” and, therefore, worth capturing on tape, while the rest of their days generally happen without a thought toward documentation. even “reality” television programs rarely show everyday events for their own sake. instead, they focus on a constructed quest, conflict, or dare, something considered “dramatic” enough to be “worth taping”. for via, every moment counts. one of the biggest challenges the via field coordinator faces is how to shatter the participants’ expectations of an “mtv view of the world.” she must remind participants that their view of their lives is more interesting than attention-grabbing, artistic shots, and reinforce via’s perspective that no part of their life is “boring.” even what they may think is dull or routine is of value to our research. we really do want to see how they live their everyday lives, and that includes getting up in the morning, eating breakfast, brushing their teeth, going to school or work, relaxing, participating in sports, surfing the web, taking medications, going to doctors’ appointments, and all other parts of their day, regardless of whether they are ordinary or extraordinary. third, with via, participants are in complete control of what they choose to record. participant control of the camera lends a subjective, first person viewpoint on their lives, which is crucial to understanding the how the participant perceives and lives within his/her life context. in home movies, “reality” television, and even documentaries, it is usually an uninvolved third person who ajis volume 12 no. 2 may 2005 110 decides what, how, and when to record. this fundamentally changes the nature of the data collected, since the gaze of the camera observing an event from the outside can not only influence the event, but has a different perspective on the outcome. via gives all control and decision-making power to participants, asking them to show clinicians how they live, what they need, and what their priorities are. this perspective allows clinicians to approach the situation through the patient’s eyes, instead of relying on what the clinician might hypothesize or expect the patient would want or need. conclusion via, when used in conjunction with nvivo, applies media and information technology toward a scientifically rigorous investigation of human experience. the development of inexpensive, lightweight consumer electronics combined with the increasing flexibility and user-friendliness of qualitative analysis software programs has created the opportunity to return control of many aspects of health to the real experts on medical conditions – individuals who live with illness on a daily basis. although traditionally understood as a weakness, via makes the subjective view of the participant into one of the greatest strengths of the data. by asking participants to draw on their experience and teach their clinicians what is salient to them about their illnesses, via shows clinicians what the patient sees as most important to the healing process. human experience is a universal but frequently overlooked source of data on health and illness because it is perceived as difficult to measure. however, when examined with via’s rigorous analytic method based on grounded theory and qualitative analysis, these complex data can reveal aspects of the patient’s illness experience of which clinicians were not previously aware. the experience and understanding of the patient can be coupled with the knowledge and skills of the clinician allowing them to become equal partners in more effective and humane health care delivery. references armstrong, a., macdonald, a., booth, i.w., platts, r.g., knibb, r.c., & booth, d.a. (2000) errors in memory for dietary intake and their reduction. applied cognitive psychology, 14(2), 183191. comte, a. (1865) a general view of positivism (j.h. bridges, trans.). london: trübner and co. (original work published 1851). fries, e., green, p., & bowen, d. j. (1995) what did i eat yesterday? determinants of accuracy in 24-hour food memories. applied cognitive psychology, 9(2), 143-155. furman, l. (1990) video therapy: an alternative for the treatment of adolescents. arts in psychotherapy, 17(2), 165-169. glaser, b. g. (1992) basics of grounded theory analysis. mill valley, ca: sociology press. glaser, b. g. & strauss, a. l. (1967) the discovery of grounded theory. chicago: aldine. kleinman, a. (1998) the illness narratives: suffering, healing, and the human condition. new york: basic books. melmed, r. (2001) mind, body, and medicine: an integrative text. new york: oxford university press. morse, j. m., & johnson, j. l. (1991) the illness experience: dimensions of suffering. newbury park, ca: sage publications. nvivo (version 2.0) (1998) [computer software]. victoria, australia: qualitative solutions and research. rich, m., & chalfen, r. (1999) showing and telling asthma: children teaching physicians with visual narratives. visual sociology, 14, 51-71. ajis volume 12 no. 2 may 2005 111 rich, m., lamola, s., amory, c., & schneider, l. (2000a) asthma in life context: video intervention/prevention assessment (via). pediatrics, 105(3), 469-477. rich, m., lamola, s., gordon, j., & chalfen, r. (2000b) video intervention/prevention assessment: a patient-centered methodology for understanding the adolescent illness experience. journal of adolescent health, 27(3), 155-165. rich, m., & patashnick, j. (2002) narrative research with audiovisual data: video intervention/prevention assessment (via) and nvivo. international journal of social research methodology, 5(3), 245-261. rich, m., patashnick, j., & chalfen, r. (2002) visual illness narratives of asthma: explanatory models and health-related behavior. american journal of health behavior, 26(6), 442-453. rich, m., taylor, s.a., & chalfen, r. (2000c) illness as a social construct: understanding what asthma means to the patient to treat the disease. the joint commission journal on quality improvement, 26(5), 244-253. rosenberg, w., & donald, a. (1995) evidence based medicine: an approach to clinical problemsolving. british medical journal 310, 1122-1126. strauss, a., & corbin, j. (1990) basics of qualitative research: grounded theory procedures and techniques. newbury park, ca: sage publications. tronick, e., beehly, m., fetters, l., & weinberg, m. k. (1991) new methodologies for evaluating residual brain damage in infants exposed to drugs of abuse: objective methods for describing movement, facial expressions, and communicative behaviors. methodological issues in controlled studies of prenatal exposure to drugs of abuse. national institute of drug abuse research monograph (114, pp. 262-291). washington, dc: department of health and human services. urban, m. (1989) video biographies: reading, researching, and recording. english journal, 78(8), 58-59. worth, s., & adair, j. (1997) through navajo eyes: an exploration in film communication and anthropology. albuquerque: university of new mexico press. acknowledgements the authors wish to thank the organizations that have supported via, including the national institute of child health and human development, the researchers and interns who have left their imprint on via, and especially each via participant who has made via what it is by so graciously sharing his or her life with us. copyright jennifer patashnick and michael rich  2004. the authors assign griffith university a nonexclusive license to use this document for personal use provided that the article is used in full and this copyright statement is reproduced. the authors also grant a non-exclusive license to griffith university to publish this document in full in the conference proceedings. such documents may be published on the world wide web, cd-rom, in printed form, and on mirror sites on the world wide web. any other usage is prohibited without the express permission of the authors. microsoft word ajis vol13-2 o.doc 19 australasian journal of information systems volume 13 number 2 may 2006 evading technological determinism in erp implementation: towards a consultative social approach dr clare archer-lean centre for social change queensland university of technology carseldine campus brisbane, queensland, australia email: clare.archerlean@shafston.edu jo-anne clark school of information and communication technology griffith university, nathan campus brisbane, queensland, australia email: jo-anne.clark@student.griffith.edu.au dr donald kerr department of management griffith business school griffith university, logan campus brisbane, queensland, australia email: d.kerr@griffith.edu.au abstract enterprise resource planning (erp) systems are implemented in business in the hope of obtaining benefits in the form of improved communications and increased efficiency through the standardization of information technology (it) across functional business areas. the benefits, and in some cases problems, associated with implementation have been well documented, however there is little information available on their effectiveness in a different form of organization. this paper looks at erp implementations in government owned corporations (goc) and discusses implementation issues by looking at the way we perceive such organisations. a goc case study is presented and explored in terms of the erp and goc literature in the context of existing social research approaches. this paper examines why end users in a goc appear to be neither complying with, nor fully exploiting the potential of the erp. the paper contends that discourses by staff at the operational level are different to that of staff at the managerial level. the paper also confirms previous research stating the importance of end user input for effective it systems and erp implementation processes. introduction over the past decade, the number of organizations adopting erp (enterprise resource planning) systems has increased significantly (beekhuyzen, 2006). the erp promise involves a standardisation and streamlining of all business functions across the organisation to enable better communication, thereby increasing efficiency (davenport, 1998, scheer & habermann, 2000, bagchi, kanungo & dasgupta, 2003). this streamlining also occurs at an external level, in that a motivating factor in implementing erp is the need to standardise the enterprise’s processes as they australasian journal of information systems volume 13 number 2 may 2006 20 are benchmarked against other corporations, thereby increasing their own competitiveness (scavo, 2005). this paper builds on hobson et. al.’s statement that past erp research has been primarily technologically determinist and that the research has largely ignored what these authors term “the social system” (hobson, et.al, 2005:3). an australian government owned corporation provides a fruitful case site to explore the problems of a technological determinist implementation that fails to synthesize with the existing social system. a cultural analogy is used to allow the rethinking of the causes of erp implementation failure. this paper attributes erp failure to a clash between the push to use technology to determine the nature of an organisation rather than as a tool of an existing and rich culture. the key area of the case explored as evidence for the source of implementation difficulties is the training stage of implementation. some multi-disciplinary approaches are employed to provide insightful ways to improve the implementation process of erp systems. context and background erp definitions and existing research erp can be viewed as a pre-packaged software product that can be bought off-the-shelf and then tailored by an organisation in order to integrate and share its information and related business processes within and across functional areas (davenport, 1998). the erp can replace an organisation’s disparate systems with a tightly integrated system of standard software modules that embodies all functional areas (ross, 1999; skok & döringer, 2001). this type of multifaceted system promises a seamless integration of company wide information which includes financial, accounting, human resource, supply chain and customer information (davenport, 1998); although it demands a large technical architecture that has huge storage needs, networking requirements and specific hardware/software requirements (scheer & habermann, 2000). the reasons driving such wide spread erp implementation are clear. according to skok & döringer (2001:3) the key drivers for the adoption and growth of erp can be attributed to increased globalisation of businesses, business process reengineering, a focus on standardisation of processes e.g. iso9000, scaleable and flexible emerging client/server infrastructures, the trend for collaboration among software vendors and the rapid emergence of e-business. erps are a way for organisations to replace their ageing legacy systems, improve strategic business advantage, and improve system architectures. (markus, petrie & axline, 2000). the clear intent to replace old it systems can potentially permeate into an attempt to replace the wider organizational culture. the fixed requirements can also cause problems for end users in terms of adoption and erp usage as by its very nature erp imposes its own logic on an organisation’s existing structure (davenport, 1998). information systems implementation in general has been, as reported in the research, notoriously difficult. so it is therefore not surprising that erp implementations are especially difficult to implement, due to their size, scope and complexity (alvarez and urla, 2002). some of the most serious problems associated with erp implementation include limited functionality, a narrow scope for decision support, upgrade and maintenance difficulties and total cost of ownership (markus, petrie & axline, 2000). along with the erp’s track record of unsuccessful implementations is the often overlooked issue of end user support. end user’s needs are frequently missed by technological solutions (beekhuyzen, 2006). throughout this paper, the end user’s role in erp implementations is discussed. hobson et al. (2005) contend that the existing heavily positivist research into erp implementation is inadequate as it does not take into account the social system of the end user. their study supported this hypothesis with a literature review of the use of key words associated with a positivist outlook. this paper seeks to understand why end users in a goc are neither complying with nor fully exploiting the potential of erp. the case study is selected because of the demonstrable history of 21 australasian journal of information systems volume 13 number 2 may 2006 problems in erp implementation within non-standard commercial organisation, particularly the government owned corporations (goc).4 the authors suggest that a multi-faceted qualitative focus is necessary in researching the issues involved in erp implementation. this paper seeks to begin discussion of erp implementation issues in goc by changing the way we perceive such organisations. government owned corporations and erp implementation erp systems are ideally placed to service the private sector. a brief overview of the historical development of erp suggests that its foundations lie heavily in assisting manufacturing companies in planning and scheduling (the united states army, 2005).5 in addition, some of the most noteworthy australian success stories in erp implementation feature private sector companies rather than public sector organisations. for example, meadow lea’s implementation of erp facilitated the fast tracking of data on the company’s grocery outlet sales across the nation. this information — hitherto unavailable to the company — was vital in effective financial and marketing strategies. erp can facilitate the development of strategic alliances. for example, erp standardisation allowed the formation of the meadow lea / canadian canola association and international fats and oils companies’ alliance (dooley, 2004). current political trends of economic rationalism require management of goc to increasingly draw on the competitive success strategies of the private sector, including use of advanced information systems, specifically the erp, to streamline both internal data processing and retrieval and external business engagements. but goc are remarkably different organisations to private sector corporations and therefore require information systems or at least end user training and implementation strategies that are tailored to their specific needs. the government owned corporation (goc) is a special case in erp implementation because of the inherent paradoxical properties of the goc. in previous eras, industries such as energy and transport provision existed primarily in the public sector, in clear distinction to the private sector. currently, services are provided by goc that operate in terms of private process and financial impetus but also through public management and guidelines. a goc is defined as a corporatised commercial entity that is freed from the budget sector but that is managed by governmental ministers (queensland's government owned corporations act, 1993). this type of corporation acts as a publicly-owned trading enterprise which conducts activities and provides services in a commercially-orientated environment (office of government owned corporations, 2003). the office controlling gocs was established by the queensland government in 2000. the role of this office was to “manage the state government's shareholding relationship with gocs”. the office is responsible for: • negotiating outcomes of the annual performance contract or statement of corporate intent (sci) as well as the five-yearly corporate plan; • monitoring the performance of gocs and determining dividend payments; • assessing major investment proposals; • goc board appointments; • ensuring goc compliance with relevant queensland government policies; and 4 identity protected. 5 the mrp and mrp2 systems were specifically designed for the manufacturing sector. australasian journal of information systems volume 13 number 2 may 2006 22 • administering the government owned corporations act 1993. (government owned corporations act, 1993). this list demonstrates the goc yoking of financial viability and government policy compliance; suggesting that at any given time a goc is subject to a number of potentially contradictory impulses. while increasing reporting requirements and privatisation do put pressure on goc to be defined through their commercial entity, goc have different requirements in terms of profit to a lone standing private enterprise. a recent state government report has revealed that many goc are in fact running at a loss rather than yielding any profit returns. administrative functions have been blamed for the long term lack of solvency of the corporations involved (smart business smart queensland, commerce queensland blueprint: smart policy – smart business, 2005). these different financial operations are due to the fact that goc are also crucial service providers for the public interest. a goc must therefore consider employee rights, public interest and voter backlash and profitability: like private companies, they engage in profit-seeking commercial activity, often in competition with private sector businesses. like public sector agencies, they are required to execute government policies, often in the form of delivering non-commercial services or “community service obligations” (government business enterprises and public accountability through parliament, 1999-2000). this tension between public and private drives makes fast tracked changes more complicated than would be the case in a purely private corporation (brown, ryan, and parker 2000). in erp adoption and training of end users, there is no facility for gradual change and the push for fast change actually results in freezing of the process. as stated in davenport (1998) it is inherent in the very nature of an enterprise system that is imposes its own logic and structure on a company’s strategy, organization, and culture. it can push a company towards full system integration when this may not be in the organization’s best interest (davenport, 1998). the public sector also operates at a much slower rate because of the structural and bureaucratic levels necessary in such organizations: “if a company rushes to install an enterprise system without first having a clear understanding of the business implication, the dream of integration can quickly turn into a nightmare” (davenport, 1998:121). a final complicating factor in the implementation of erp in goc lies in their service areas. most obviously, state and national goc are involved in the two areas of energy and transport. these industries are very mature and often involve an organisational structure and culture with deeply entrenched ideologies and practices. so a goc is split in terms of its corporate identity. it is public and private; mature and culturally established yet desirous of technological innovation and change. it is fundamentally categorised by its incorporation of a number of different practices and motivations held in delicate balance. the erp technological determinist push for institutional transformation to the requirements of the technology, potentially threatens the tropes that categorise the organisation itself. the research site the specific identity of the case study has not been revealed because of the ethical necessity to comply with anonymity clauses implicit in the research project. the goc will therefore be referred to simply as case. the case organisation used for this study is a large goc operating at a state level. like all essential utility services within australia, the case has been operating for well over a century and is 23 australasian journal of information systems volume 13 number 2 may 2006 a very mature industry. the case organisation has a global reputation for providing innovative and performance driven services. it has a turnover of over $a 2 billion and has over 13,000 employees. the case goc had already replaced a sound operational system with sap r/2. recently management made the decision to upgrade to sap r/3. the initial motivation for this implementation was to facilitate new reporting requirements and the implementation process involved conversion from sap r/2 to sap r/3 for 6,000 users. the modules involved in the implementation were: financial; material management; logistics; forecasting and planning; materials resources planning (mrp); human resources; information systems, including executive information systems; project management; and office integration. the objective of the research project was to find ways to optimize the supply chain of the goc and to this end, 33 interviews with key stakeholders along the entire supply chain were conducted to determine important elements in each individual’s job and how they related to the supply chain in its entirety. interviews ranged from human resource personnel all the way to operational staff and included the supply chain partners such as suppliers of raw materials and manufactures. over 500 pages of transcripts were recorded. analysis of case: social systems vs technological systems technological determinism is frequently a latent assumption in it research and can be defined as a focus on the technical system and its success rather than the social system that must operate it. many researchers “have long rejected the overly simplistic notion that technology is an autonomous force that determines the course of society” (smith & marx, 1994). whilst acknowledging the ongoing nature of this study, the key goal of this paper is to better explain end user non-compliance with erp and open up ways of perceiving a goc’s it implementation requirements by reexamining end users’ experience. as hobson et al. (2005) state, erp research is frequently positivist in nature and as a consequence technologically deterministic. when the social system/end users are explored, it is in terms of their deficiencies in responding to training packages and in operation of the software. in contrast to this approach orlikowski (1991) stresses that technology should be viewed as “the product of human action” [emphasis added]. often erp implementation causes so much stress on an organisation that human capabilities are unable to realistically keep up or meet its requirements, for example in the hilton corporations upgrade of peoplesoft the vice president of the hilton's corporation stated that “employees got paid, but only after round-the-clock efforts by the company's payroll staff. it took three weeks to completely catch up” (anthes, 2003). the us army enterprise solutions competency center (2006) view erp implementation fundamentals as a plethora of requirements that “can make governance structures look more like engineering diagrams than organization charts and slow the decision making process to the point where it cannot effectively support its programs.” in erp implementations there is often a misalignment between the human/business system and the new technological system (davenport, 1998). the authors concur with orlikowski (1991) in her statement that technology has “a dual nature, being both constructed and enacted by human agents, as well a material force that shapes human action and social practices”. to date, social focused approaches to erp implementation exist but are limited in numbers. new social approaches will improve future use of erp if they focus on the language of the users. one way in which this can be done is to focus on the reports of the users themselves. interviews then are a vital tool, if viewed through a qualitative lens (yin, 1994). in order to avoid a positivist australasian journal of information systems volume 13 number 2 may 2006 24 concentration on systems analysis and data collation, the language should be seen as part of an organic and dynamic social system. a positive analogy for understanding the organic and dynamic social system of the goc is that of a series of entrenched sub-cultures. the metaphor of an organisation as a culture is not new. the perception that organisations do not simply possess cultures which must be subsumed to the larger organisational missions but actually are entrenched culture/s with shared values, beliefs, language and social systems of operation is well documented in the literature (smircich 1983; bright and cooper 1993). culture is a diverse term and must be defined. geertz’s “culture concept” emphasizes the importance of historically ingrained and inherited meaning carried through symbols within cultures. these patterns of symbols and linguistic tropes within culture both reflect and, importantly, shape understanding of reality (geertz 1966 in varenne, 2002). in the context of an organization using technology the relationship between cultural language and symbols and perception and use of the technology is clear. this is so much so that we must distinguish as orlikowski does between the physical construction and use of technology and the social use and interpretation of technology (orlikowski, 1991). that technology is socially constructed “by actors through the different meanings they attach to it and the various features they emphasize and use” is clear in the case with the end users’ frustration in a supposedly beneficial system. here, we posit that the case goc consists of not only diverse missions and operators, but also cultures, even sub cultures, with historically entrenched forms of communication and understanding of information. such embedded cultural mores cannot simply be re-determined by an externally applied technological infrastructure. the impediments to implementation are a result of the technologically determinist inability to gauge the complexity of the existing culture. these complexities are largely expressed in symbolic and non-self-conscious forms which in turn shape responses to innovations within the culture. the key users of the supply chain of the erp within the case corporation each operate out of inherent values and serve particular functions while using a specific discourse and carrying out particular rituals. this was emphasized through the strong culture that existed in the goc with the experienced “long-time workers” consistently referring to new employees as not “understanding the history and culture of the organization”. these “new workers” were treated with suspicion and a certain amount of mistrust as they often referred to the need to “employ people with a higher level of education and a greater understanding of the modern corporation”. the case does not represent a homogeneous and unified culture ready to cohesively adapt to change. erp implementation does not take into account the intricacies of the culture and it is therefore unable to shape reality or shape itself to the existing reality, thereby failing to deliver the promised increases in effectiveness. one of the glaring examples of the failure to synthesize erp implementation with the existing culture of the case occurs within training. erp implementation clearly requires mandatory processes to make the system operational. like any information system’s implementation it follows a chosen life cycle and there are optimal recommendations associated with the implementation stage such as planning, user training, conversion strategies and follow up (alter, 2002). the design of the training need not be entirely determined by the technology. end user training provides an area to explore the usefulness of social focused methodology in improving erp implementation. much research in erp implementation has identified lack of or insufficient training as a key barrier to success. we contend that there is only ostensible compliance, at the level of initial training and that training packages are often computerised manuals with ‘successful’ training achieved through the user working their way through each module. it did not take long for end users in the case to realize that repeated pressing of the next or enter key could speed up the process considerably without the need to learn how the system was working. people were inclined to do this because the training was not done inthe context of the every day work routine of each individual and the output 25 australasian journal of information systems volume 13 number 2 may 2006 produced during training did not relate to their own work environment. repeated pressing of the ‘enter’ key ultimately leads to the end user being awarded a certificate, qualifying her/him to use erp. this form of information transfer does not align with the entrenched forms of practical and “on the job” (case study data) training and communication in the pre-existing operational staff culture. the erp training process, while it may, through the work of training organisations, attempt to find an example meaningful to the end user, ultimately is a set package that is largely mechanized hoopjumping. the users feel imposed upon rather than assisted, and as a result there is much anecdotal evidence with the case for ostensible rather than actual compliance with the training package. motivation is core part of successful implementation of any organizational change. in the case the staff motivation for erp implementation is absent. there is a general feeling (revealed in interviews) that the system is ultimately about higher management and accounting compliances than ease of use and assistance for users in the supply chain. the original sap r/2 system replaced a useable system for operational staff because management was more concerned with integrating it resources and having a system that they could use for their own reporting requirements. it appears that management is supporting a technological determinist approach that is impeding with their goal of efficiency. the politicized opposition within the work space of us (end user) and them (management and erp system) was of course compounded by the inefficiencies resulting from the failures of the non-meaningful training package. repeatedly, those interviewed throughout the supply train used language and phrasing that symbolically indicated strong feelings of incapability between their own work culture and identity and erp usage. many expressed frustration at being compelled to adapt their work processes to fit the system. for example, one middle manager who was forced to use the new system stated: “sap comes along, far from helping you manage your processes, it tells you you’ve got to change your processes to suit the information system” (case study data). apart from meaningful training and clear motivation for change, another core reason for the erp implementation problems appears to be differences in language styles between the erp system and the existing culture. the workers have a long history in manual processes and practical application of skills. the language they require to successfully complete these tasks is necessarily practical and concrete, “information from the right people is more current and accurate than what’s in the system” (case study data). the processes they use are also necessarily concrete and hands-on. and yet the erp demands a language and process that is abstracted from the lived reality of their normal work functions. the response is very human: a complete lack of trust in the new process and ‘language’ required to implement it. this in turn then results not in increased efficiency, the key promise of the erp system, but doubling up of process. for example, operational managers often keep physical quantities of product away from the centralised information system (sap). an operational manager at the final point of the supply chain states that: “engineer says – how can we get this [product] in an emergency. we tend to know where [product] is available in what we call “grass stocks” and we can find them in an emergency” (case study data). “grass stocks” are products that are concealed from the main inventory and made available (by people who know where they are located) in times of an emergency. the fact that product is concealed from the main sap system indicates that in this particular case the operational manager has a mistrust of the centralised inventory system. this mistrust of the information system could be due to the operational level employees feeling more comfortable knowing and physically sighting inventory rather than trusting the information system. the end users do not place information “grass stocks” into the erp system because of their impression/fear that the erp may compromise emergency processes. the organizational culture, far from being replaced and determined by the new technology, is actually re-emerging in forms outside the control of the system. australasian journal of information systems volume 13 number 2 may 2006 26 in effect what is happening is that the use of the erp system is complied with only ostensibly, and the users see it as an unnecessary hoop through which they now must jump. there is a clear understanding that the erp is used only by a few operational supervisors and administration people have a concern that data may be incorrect. for example, a manager who took the time to learn how to extract reports from sap at a more detailed level stated: “they cannot audit their admin people on what they do because they (managers) do not have a clue about sap so they rely on their admin people totally. they could be plugging in the wrong numbers” (case study data). beyond withholding information from the erp system, users also develop workarounds (alter, 2002) to aid end user access to the data embedded within the erp. as is repeatedly cited in the research, the ‘ever present knowledge gap among implementation personnel is usually significant along with [gaps] in organisational member’s experience’ (soh, kien & tay-yap, 2000). an example of this is shown in the quote below where the respondent is having problems with the technology and can only receive useful help from his/her line supervisor who happens to have the necessary level of expertise: “you have got to keep your eye on the whole ballgame otherwise nothing happens. the computer system is something i have always struggled with […] you basically learn on the job, basically my supervisor […] is very good on computers and so he’s helped me out from that point of view. so if i have a problem with anything i mean there are people in the office [who can help], i say i have a problem with this, i can’t get this [done]. show me how to do it. (case study data) […and…] that’s what i find mind boggling is that there are managers out there don’t have the faintest [idea] about sap and are at the whim of their admin people” (case study data). this sporadic rather than complete use of the erp system within the case organization hinders rather than aids efficiency. it creates institutional divisions and results in failure to fully use the system. the existing culture is refusing to be determined by a system that has neither meaning nor motivation for them. a technologically determinist attempt to replace existing practices and languages will necessarily fail as the cultural naturally and organically evades such supplanting strategies. in order to overcome the results of a technologically determinist implementation, we contend that the culture of operational level staff and managerial staff is significantly diverse and that effective it systems require the input of the end users. in terms of the symbolic messages sent and read within the goc culture, the refusal by upper management to enact consultative processes in implementation represents mistrust by upper management in their own workers. there is no attempt at open interactive communication concerning the erp product or implementation and there is a push to supplant existing structures. this is evidenced in the case where a sound operational system was replaced with sap r/2 so managers could get reports that suited their requirements. this mistrust does not go unnoticed and the workers respond with reticence and non-compliance. some even prefer to use people to obtain data, rather than trust the system as evidenced by the statement by an operational staff member: “i understand the people i work with but not the systems. you meet a person and you know that you need only tell them once or sometimes twice but with a system it is different and i have not worked that out yet” (case study data). it is very clear that the operational staff feel disempowered by their positions and their current engagement with the system. frequently managerial instructions and even erp operations are phrased as ‘commands’ or ‘told’ instructions and the speaker’s response is recorded as what they ‘know’, don’t know’, or ‘see’ (case study data). the use of verbal process verbs for managerial instructions as opposed to mental process verbs for understanding demonstrates some of the latent emotions at work in this situation and also the loss of agency felt by end users. the use of personal pronouns also demonstrates power relations. there is repeated use of non-specified ‘us’ / ‘we’ / ‘you’ versus ‘they’ ‘them’ dichotomies again indicative of the perception of unbalanced power relations with the narratives of the interviewees. 27 australasian journal of information systems volume 13 number 2 may 2006 the focus on the people within culture, as opposed to the literal, positivist and technologically determinist functions is vital in understanding problems within erp implementation. language and the communication forms that people use are not simply a means to help convey a pre-existing reality; rather, they affect perception of reality. language then is not just content, it is form; paralanguage, adjective and noun choices, and verb process choices all operate to affect that which we communicate and embedding latent messages and even ideologies (goatley, 2000). erp implementation has, at a symbolic level at least, involved ideological competition and communication breakdown within the goc social system because the intricacies of the existing communication and cultural patterns within the goc have not been valued or assessed. returning again to the training stage of the implementation process as an example; it is evident that end users are not simply failing to absorb and understand the content of the training package. they are actively bypassing the information communicated as not meaningful to them. erp research must avoid technologically determinist emphasis on the benefits of the product. the product, be it training manual or erp system itself is only functional once incorporated into the culture of the end user. this is true whether the erp is being implemented into a goc or a new forming corporate company. but since goc are more closed and mature and therefore entrenched cultures than young profit seeking organisations it is necessarily harder, and potentially counterproductive, to attempt to change the existing culture rather than adapt to it. and the existing culture must be evaluated and researched according to detailed observation of the complex nuances of the communication practiced and the denotative (literal) and connotative (associated and culturally produced) levels of engagement (fiske, 1990). the development of erp implementation, training models, for example, must seek to harmonise with the connotative levels of the language of the existing culture. this is possible through detailed observation of the existing culture and interpretation of verbal feedback through focus on discursive practice and connotative signification. significantly, one of the reasons that end users within the goc are particularly unable to abandon their existing work practices and cultures, thereby inhibiting smooth erp standardization, seems to lie in the way in which many of the staff link identity and job position. many of the interview scripts taken can be best understood through what can be termed narrative frameworks. as alvarez points out, people frequently use narratives to provide unity and structure to their lives and also to characterize, even define, their own identities as the protagonists of their own life story. alvarez has observed that in the past analysts conducting interviews with end users seek to rush through or evade the story telling/narrative style responses to their questions but in so doing lose key information (alvarez, 2002). understanding the transcripts of users in our case as embedded narratives coming from people within an established and embedded culture allows furthering of our understanding of why noncompliance occurs in erp implementation and use. firstly this is because many operational level workers do not separate their own subject positioning and feelings of worth from the achievements of the organisation. the position is not simply a job to be done, separate from their culture and identity; it is a culture informing their identity, we can see this implied in the interview transcripts: ‘i enjoy coming to work. there’s sense of most occasion there’s a sense of achievement, we have certain commitments to get out. when we do that i feel good. when i don’t do that, i don’t feel so good. we meet “better than requirements” for our dispatches for the month, i will take that as a personal pride. if we don’t i will take it as something i should have done better” (case study data) the impediments to job requirements caused by the failed erp system are meaning that the end user does not ‘feel good’ therefore compounding the attitudinal problems causing the erp implementation difficulties. secondly, is the importance of linguistic difference realizing difference ‘genres’, or “language used in performance of particular social practice” (chouliaraki and fairclough 1999: 56). the interviews demonstrate repeated moments of the end user constructing meaning australasian journal of information systems volume 13 number 2 may 2006 28 through narrative style genres, as said above. this is in stark contrast to the technologically determinist ‘mandated course of action’ style discourse used in erp implementation processes supported by managerial discourse. this communication dissonance seems to harm end user willingness (even ability) to enter into the system. it is apparent that the workers also feel an opposition between the technologically determined culture that is being imposed and the existing ‘human’ culture. operational workers also feel the necessity in valuing their existing ‘cultural’ structure, rituals and processes in improving communication. there is some mistrust, fear even, that the technology, far from increasing visibility across the board, will create barriers between ‘real’ people and impede successful process: “once you separate people in time and space you have lost a lot so i am not an advocate of technology per se. going back to the things we mentioned before about personal contacts and trust i believe for communication to be effective, we [need to] communicate face to face, there are a whole lot of other things that are subliminal that affects trust and clarifies things” (case study data). the end users are stressing the importance of human communication within the existing social system; research into erp implementation must follow that lead. recommendations from the initial interpretation of the interview transcript data from the goc, it is evident that the problems with erp implementation include: low training absorption/comprehension; low end user compliance, including work around processes and withholding data; and dissonance between upper managerial and operational levels. this paper has suggested the mature and complex existing culture and communication break down as key reasons for these issues. the authors also stress that it is not fruitful or even possible to resolve these issues by coercing fundamental changes in the existing culture. the research team proposes that further analysis is necessary to clearly define the key components of the existing culture. this will enable better understanding of the it and it implementation requirements of the case goc, and potentially other similar organisations. this involves detailed observation of the complex nuances of the communication, practiced and literal and symbolic levels of interaction and meaning production. one example of the outcomes of this process might be the development of training models with detailed and meaningful programs and formats developed in consultation with end users. this might involve the use of narratives and analogy over computerised chains of information delivery. there is also a need to ensure that, at all levels of the implementation life cycle, the end users see the relevance and value of the erp to work practices and that the change is being integrated into the mores of the existing culture. implementation must use the symbols of the existing culture to communicate management’s valuing of the end user if implementation is to be openly received. conclusion existing erp research is strongly positivist in nature and inadequate in terms of providing the perfect solution for corporations to successfully implement erp systems. this paper contributes to the field of is research in that it offers a new approach to interpreting the complex clash between the push to use technology to determine the nature of an organisation rather than as a tool of an existing and rich culture. technological determinism has been explored in terms of its impact on research into erp implementation. the insight of this paper is not only to reject technological determinist approaches but to identify the technologically determinist tendencies in erp implementation itself. the push to replace an existing organizational culture rather than consult and assist that culture lies 29 australasian journal of information systems volume 13 number 2 may 2006 at the heart of many erp implementation failures. evidence for this hypothesis can be found through careful analysis of interviews with end users. the interviews must be interpreted not just through their blatant ostensible content but through the latent meaning revealed through a range of multi-disciplinary approaches which reveal the rich and potentially productive complexity of the existing culture. in researching erp implementation in goc a new approach allows for the rethinking of the issues themselves and promotion of a better understanding of the causes of implementation failures and impediments. future research will need to adopt cultural studies analysis for the interpretation of end user feedback (interviews) and input on erp in order to aid prediction of problems and assist future systems development. references alter, s. (2002) information systems: the foundation of e-business, prentice hall, usa. anthes, g. h. (2003) hilton checks into new suite, http://www.computerworld.com/printthis/2003/0,4814,82522,00.html, accessed may 25, 2005. alvarez, r. (2002) discourse analysis of requirements and knowledge elicitation interviews proceedings of the 35th hawaii international conference on system sciences, 2002. alvarez, r., and urla, j. (2002) using narrative analysis to examine information requirements interviews during erp implementation database for advances in information systems, new york:winter 2002. vol.33, iss. 1, pg. 38-52. bagchi, b., kanungo, s., and dasgupta, s. (2003) modeling use of enterprise resource planning systems: a path analytic study, european journal of information systems. basingstoke: june 2003. vol. 12, iss. 2; pg. 142. beekhuyzen, j. (2006 in press) an australian case of erp quality: do individual users matter? in measuring information systems delivery quality, e. duggan & reichgelt (ed). idea group publishing, pennsylvania, usa. bright, k., and cooper, c.l. (1993) organisational culture and the management of quality: towards a new framework” journal of managerial psychology 8 (6) : 21-27. brown, k., ryan, n. & parker, r. (2000) new modes of service delivery in the public sector: commercializing government services. the international journal of public sector management vol. 13 no. 3, 2000, pp206-221. chouliaraki, l. & fairclough, n. (1999) discourse in late modernity: rethinking critical discourse analysis edinburgh university press, edinburgh. davenport, t.h. (1998) putting the enterprise into the enterprise system, harvard business review, vol. 76 no. 4, pp. 121-31 dooley, l. (2004) agribusiness awards: a report for the rural industries research and development corporation monash university october 2004 http://www.rirdc.gov.au/reports/glc/w04-156.pdf, accessed 28 may, 2005. fiske, john. (1990) introduction into communication studies routledge: london. gerrtz, c. religion as a cultural system. in his the interpretation of cultures. new york: basic books.1973 [1966] in varenne, h.(2002) “culture definitions” communication and culture, department of international and transculturual studies, teachers college, columbia university, http://varenne.tc.columbia.edu/hv/clt/and/culture_def.html#ftop accessed july 10, 2005. goatley, a. (2000) critical reading and writing. routledge, london. government owned corporations act (1993) http://www.legislation.qld.gov.au/legisltn/current/g/goowncorpa93_06c_030328. pdf, accessed may 10, 2005. australasian journal of information systems volume 13 number 2 may 2006 30 goc (2005) office of government owner corporations (queensland government) http://www.ogoc.qld.gov.au/ accessed june 25 2005. government business enterprises and public accountability through parliament, (1999-2000) http://www.aph.gov.au/library/pubs/rp/1999-2000/2000rp18.htm, accessed may 10, 2005. hobson d.p., kerr d., burgess, k. & houghton, l. (2005) the inefficiencies and excess’s of enterprise resource planning systems (erp), the 2005 standing conference on organisational symbolism xxiii, excess and organisation held in stockholm, sweden. markus, m. l, petrie, d. and axline,s. (2000) bucking the trends: what the future may hold for erp packages, information systems frontiers, 2, 2 ( september): 181-193. orlikowski, wanda j. (1992.) the duality of technology : rethinking the concept of technology in organizations. organizational studies, 3(3):398-427, 1992. ross, j. (1999) the erp revolution: surviving versus thriving, cisr working paper no. 307, http://mitsloan.mit.edu/cisr/r-papers.php, accessed may 10, 2005. scavo, f. (2005) benchmarking in buying and implementing enterprise systems http://www.computereconomics.com/article.cfm?id=1028, accessed may 10, 2005. scheer, a.w., & habermann, f. (2000) making erp a success, association for computing machinery communications of the acm new york, april 2000 vol. 43, issue 4, p57-61. skok, w., and doeringer, h. (2002) potential impact of cultural differences on enterprise resource planning (erp) projects. electronic journal of information systems in developing nations. 7. 5. 1-8. soh, c., kien, s. s. & tay-yap, j. (2000) cultural fits and misfits, is erp a universal solution?, 2000 communications of the acm april 2000/vol. 43, no. 4 smart business smart queensland, commerce queensland blueprint: smart policy – smart business (2005) http://www.commerceqld.com.au/docs/smartbusinesssmartqueenslandfull.pdf, accessed may 10, 2005. smith, m. r., and marx, l. (1994) does technology drive history? the dilemma of technological determinism. cambridge, mass.: the mit press. smircich, l. (1983) concepts of culture and organisational analysis administrative science quarterly 28 (3): 339-358. the united states army, (2005) www.army.mil/aeioo/erp/overview1.html, accessed may 30, 2005. the us army enterprise solutions competency center (2006) erp resource center, http://www.army.mil/aeioo/erp/gov.htm, date accessed 13 february, 2006. yin, r.k. (1994). case study research: design and methods. thousand oaks, ca: sage publications. microsoft word ajis10-2finala.doc ajis vol. 10 no. 2 may 2003 151 australian hackers and ethics m.j.warrenφ and w.hutchinsonω φschool of information technology, deakin university, geelong, victoria, australia. ωschool of computing & information science, edith cowan university, mount lawley, western australia, australia. e-mail contact: mjwarren@deakin.edu.au abstract the aim of the paper is to look at the way hackers act and ways in which society can protect itself. the paper will show the current views and attitudes of hackers in an australian context. the paper will also include a case study to show how a hacking incident can develop and how technology can be used to protect against hacking. keywords: computers and society, australia and hacking. introduction we have seen a rise in computer misuse at a global level, it is generally thought that ‘hackers” are responsible for these attacks. hackers are perceived as being adolescent males, in dark bedrooms being able to cause massive damage across the world just by the use of their computers. a more romantic perception portrays them as being determined: cyber knights with a code of conduct to live by just like the great arthurian knights. this paper looks at hackers, their ethical viewpoint and the role and impact of hackers within australia. computer hacker the definition of hacking according to bruce sterling (1993) in his book titled ‘the hacker crackdown’, the term “hacking” is the act of intruding into computer systems by stealth and without permission (lopez-fernandez and warren, 2002). however, this name is used routinely today by almost all enforcement officials with any professional interest in computer fraud and abuse to describe any crime committed with, by, through, or against a computer. moreover, ‘hacker’ is what computer-intruders choose to call themselves, not as a criminal pejorative, but as a noble title given to those “soaked through with heroic anti-bureaucratic sentiment.” (sterling, 1993). hacking then, can describe the determination to make access to computers and information as free as possible. hacking can involve the heartfelt conviction that beauty can be found in computers, that the fine aesthetic in a perfect program can liberate the mind and the spirit (levy, 1984). state of australian it security and australian hackers a recent auscert survey (auscert, 2002) has focused upon the state of it security within australia, the following is a summary of the main results: • 67% of all organizations surveyed have been attacked in 2002 twice the 1999 level and 35 per cent of these organizations experienced six or more incidents; • 98% of companies had experienced either computer security incidents / crimes or other forms of computer abuse (such as network scanning, theft of laptops, employee abuse); • of australian organisations who were victims of computer incidents, 65% of these attacks were from internally parties within the organisation and 89% came from external sources; • 43% of australian organizations were willing to hire ex-hackers to deal with security issues, three times more than in the us. the survey showed that it security and computer misuse are a major problem within australia. the survey showed that external attacks were the source of the majority of attacks. perhaps of interest is the willingness of australian organization to use hackers to improve their security. ajis vol. 10 no. 2 may 2003 152 hacker motivation a recent hypotheses put forward has been regarding hacking motivation is that they are suffering from asperger syndrome (dreyfus, 2002). aspies typically have an almost obsessional approach to solving problems and are often oblivious to their peers' view that a given problem is ‘unsolvable’. both are often prerequisites to becoming an elite-end hacker. there does not appear to be any in-depth research linking illegal hacking and asperger syndrome. however, one of the world's leading asperger syndrome experts, australian clinical psychologist tony attwood, believes some hackers may share characteristics with "aspies", as they refer to themselves (dreyfus, 2002). "it's the sheer challenge rather than any (criminal intent). it's the pursuit of knowledge and truth with different priorities and perceptions. they see it as an intellectual challenge and a prize, (and) they look at the success of what they have done rather than the consequences of the lives of people they have affected" (dreyfus, 2002). perhaps technology and deception combined into ‘honeypots’ and ‘honeynets’ can offer protection against such individuals. honeypots & honeynets a honeypot is a ‘pretend’ server with the aim of tracking black-hats (an unauthorized person trying to get access to a system) (spitzner, 2000a) in the act of probing and compromising a system. the aim is to deceive the black-hat into thinking they are attacking an actual real life server (software examples include systems by cohen 2000, and network associates, 2000). the aim of the honeypot is to monitor the black hats by a number of means (spitzner, 2000a), they are: • tracking the honeypot firewall logs • analysis of honeypot system logs to determine what the kernel and user processes are doing. • using a sniffer on the firewall that ‘sniffs’ any traffic going to or from the honeypot. the advantage of a sniffer is that it picks up all keystrokes and screen captures. • using a tripwire on the honeypot. a tripwire tells the system administrator what binaries have been altered on a compromised system (such as a new account added to: /etc/passwd, or a trojaned binary). the aim of the honeypot is to attract the black-hats, monitor them, let them gain root access to the system, and then eventually log them off the system, all without any suspicion being aroused. once black-hats gain root access, they are monitored for several days in order for the system administrator to learn what they were doing. the biggest problem is how to limit the black-hats offensive actions (spitzner, 2000b). this is done by using the honeypot firewall, and implementing a rule base schema that allows access from the internet to a honeypot’s firewall, but limits outbound network traffic. it is important that the black-hat is allowed enough outbound traffic so as not to arouse suspicion. the results of these honeypot assessments are made public (http://project.honeynet.org/) so that network administrators can access the information and ensure that they are protected against common hacker attacks and techniques. the work by spiztner developed into expanding the honeypots into honeynets. spitzner (2000c) identified that the honeypots needed to be expanded for the following reasons: • to be able to determine attacks upon switches, routers and different operating systems of a network • generate information from several sources (for example, honeypots) in order to provide information in greater detail. • detect new attack patterns such as vulnerability scanning and how black-hats progress from one system to another. the result was grouping a number of honeypots together to form a honeynet, so a hacker would feel that they were gaining access to a much large networked system. an ideal solution to stop someone suffering from the asperger syndrome to cause harm is by the use of honeypots and honeynets. a major issue is whether the use of honeypots and honeynets are ethically acceptable. is it ethically acceptable to deceive an attacker who is trying to hack into a computer systems? australian hackers research in the early 90’s within australia showed that computer crime and hacking was a problem. victoria was the first australian state to implement state law to outlaw hacking in 1988 and the commonwealth ajis vol. 10 no. 2 may 2003 153 followed in 1989 (hughes, 1990). an analysis of computer crime in 1991, showed that within australia between 1990 and 1991 there had been 497 computer abuse incidents and 31 incidents related to hacking (around 6% of incidents) (kamay & adams, 1992). research at this time also indicated australian perception towards computer crime was influenced by cultural precedents (coldwell, 1995). other studies at this time also looked at the australian perception of computer crime, a study was undertaken looking at teachers’ perception of hacking and found from a sample group that 60.2% thought hacking was unacceptable and 39.8% thought hacking was acceptable (coldwell, 1994). in 1997 “underground” was written which described the history of australian hackers during the early nineties, the development of the ‘wank virus’ and ‘plans’ to destroy nasa computer systems (dreyfus, 1997). since that time, most of the australian hacking community seem to have disappeared, no well known australian hacking groups or even australian hacking conventions are in existence. the only large australian hacking group is ”2600 australia”, (http://www.2600.org.au/) this group is based upon the famous us hacking group 2600. the philosophy of 2600 australia is “2600 australia is a loose-knit group of people interested in computer security, electronic gadgetry, communications and just technology exploration in general” and in terms of their activities can be best described as a computer club. no research has been undertaken to determine the numbers of hackers within australia. since the mid nineties there has been no major hacking incidents involving australia or australian hacking groups, the millennium bug period and olympic game passed without any publicized incident. the most recent famous australian hacking case was to do with sewage. in october 2001, vitek boden was convicted of 30 charges involving computer hacking of the maroochy shire council sewerage system. the attacks, which commence in late 1999, involved using remote radio transmissions to alter the actions of the sewerage pumping stations and caused hundreds of thousands of litres of raw sewage to be pumped into public waterways (kingsley, 2002). in the year 2002, does hacking by australian hackers and hacking groups pose a real problem? an example of an australian hacking case the following is example of a typical australian hacking case in chronological order. the case study shows how a hacking incident can develop and how the press become focal point (lopez-fernandez and warren, 2002): carr defends mp in hacking case (australian financial review 07 aug 2001) the nsw labor mp at the centre of a hacking scandal said yesterday he had once trained as a computer programmer, after initially saying he ``wouldn't know the first thing about hacking into a computer". it also emerged yesterday that the computer of a senior liberal mp, mr peter debnam, had been unlawfully accessed at parliament on a public holiday. a computer belonging to the labor mp, mr tony kelly, was seized by nsw police yesterday after allegations that confidential files belonging to the opposition were found on a computer in the parliamentary office of the state government's leading upper house strategist mr tony kelly. office ban on computer mp's son (sydney morning herald 07 aug 2001) the son of the labor mp at the centre of computer hacking allegations at state parliament was barred from his father's parliamentary office last month, the herald has been told. the upper house mp, mr tony kelly, who admitted training as a computer programmer in the 1970s and 1980s, refused to comment on reports his son had extensive computer skills. it is understood mr john kelly has been a regular visitor to his father's office. hacking skills denied (illawarra mercury 07 aug 2001) the nsw labor mp at the centre of a parliamentary computer hacking scandal has revealed he had been a computer teacher at a tafe college. political espionage (sydney morning herald 08 aug 2001) sometimes security is only noticed when there is none. the discovery that a state government mp's office computer may have been used to hack into opposition computer files has shaken the customary quiet sense of security that pervades parliamentary life. mp in hacking affair steps aside (sydney morning herald 08 aug 2001) the controversy surrounding the alleged hacking of an opposition mp's computer deepened yesterday as the carr government politician at the centre of the allegations was forced to stand aside from his parliamentary positions. ajis vol. 10 no. 2 may 2003 154 hacking claims: mp steps aside (illawarra mercury 08 aug 2001) the nsw labor mp at the centre of a parliamentary computer hacking scandal stood aside from his upper house duties yesterday as the search for the hacker continued. nsw labor mp steps aside during inquiry into hacking (australian financial review 08 aug 2001) the nsw labor mp mr tony kelly stood aside from parliamentary duties yesterday amid a police investigation into computer hacking at state parliament. labor mp steps down from duties (newcastle herald 08 aug 2001) the nsw labor mp whose office computer is at the centre of a parliamentary computer hacking scandal stood aside from his upper house duties yesterday as the police investigation continued. hacking software found in mp's computer (sydney morning herald 09 aug 2001) a computer in the office of the labor mlc mr tony kelly was loaded with password ‘sniffing’ software that could have been used to break into the personal files of the liberal mp mr charlie lynn, a consultant hired to investigate hacking allegations inside the nsw parliament has found. the herald has confirmed that the 12-page preliminary report by a melbourne firm, esec, commissioned by parliamentary staff and handed to police on tuesday, recommends a more detailed analysis of the computer files. staff kept suspicious software under wraps (sydney morning herald 10 aug 2001) the nsw parliament ‘hackergate’ controversy deepened last night when parliamentary staff revealed they had covered up for nine days the discovery of suspicious software on an mp's computer. carr denies labor not cooperating (illawarra mercury 13 aug 2001) nsw premier bob carr has denied labor members were unwilling to cooperate with the police inquiry into state parliament's computer hacking scandal. mps' house rules frustrate police hunt for hackers (sydney morning herald 14 aug 2001) police investigations into computer hacking allegations at the nsw parliament are being frustrated by parliamentary privilege. hacker squad get the go-ahead on mps' files (sun herald 19 aug 2001) detectives from the commercial crime agency will return to parliament house in macquarie street tomorrow following a major breakthrough in the computer hacking investigation. mp clear in hack inquiry (illawarra mercury 31 aug 2001) nsw labor mp tony kelly was cleared yesterday of any criminal activity by police investigating allegations of computer hacking at state parliament. police clear mp of hacking allegations (sydney morning herald 31 aug 2001) the mystery surrounding the nsw parliament ``hackergate" controversy remained yesterday when police cleared the upper house labor mp mr tony kelly. they found that there were computer files belonging to liberal party mlc mr charlie lynn on a computer from his office. mp's son admits: `i loaded software' (sun herald 02 sep 2001) john kelly, son of embattled labor mp tony kelly, has told police investigators that he loaded hacker software on to his father's parliament house computer. parliament insecurity (sydney morning herald 03 sep 2001) the police inquiry into state parliament's so-called ``hackergate" controversy has done nothing to restore faith in a system that should guarantee mps unconstrained freedom in representing the public effectively. after a month-long investigation police have confirmed that unauthorised copies of computer files belonging to the liberal mp, mr charlie lynn, were found on a computer in the parliamentary office of the state government's leading upper house strategist mr tony kelly. this is a serious finding… it blamed for secret downloads (sydney morning herald 04 sep 2001) ajis vol. 10 no. 2 may 2003 155 questions continue to be raised in the mystery over the nsw parliament hacker scandal after a report blamed parliamentary it staff for accidentally loading confidential files belonging to the opposition mlc, mr charlie lynn, on to the computer of the nsw labor mp, mr tony kelly. kelly no computer hacker (illawarra mercury 04 sep 2001) nsw labor mp tony kelly has demanded an apology from opposition leader kerry chikarovski after being cleared yesterday of hacking into confidential liberal party computer files. mp's son loaded `hacking software' (illawarra mercury 05 sep 2001) the nsw labor mp embroiled in a computer hacking scandal has confirmed his son was responsible for loading suspect software on his pc. revealed: how mp's son used computer in hacking scandal (sydney morning herald 05 sep 2001) the son of mr tony kelly, the labor mp at the centre of hacking allegations, was using a computer in his father's parliamentary office late one friday night to run software that can scan computer networks for security weaknesses while his father was overseas on parliamentary business. no evidence of hacking, says clerk of parliament (sydney morning herald 07 dec 2001) the clerk of the nsw parliament did not contact police after security software was found on an mp's computer because he had no evidence that any offence had been committed, he revealed last night. in a final report on the hacker controversy sparked after suspicious software and files were found on the computer of legislative council member tony kelly in july the clerk of the nsw parliament, john evans, said it would have been inappropriate of him to assume an offence had occurred without independent evidence. the mini case study shows the main aspects of a hacking crime: • the actual attack and determination that an attack had taken place; • the response to the attack by the organisation; • involvement of legal authorities; • outcome of investigation. from an ethical viewpoint it is interesting how the press reported the incident and raised unconnected issues e.g. an mp had been a computer teacher at a tafe college. the mini case study showed two main ethical issues: • the mps son has access to his father’s parliament house work computer and was able to install computer software. the aim of the software was to scan network for security vulnerabilities e.g. the network at parliament house. an ethical solution would be to ensure that users do no allow other people to use their computers; • the accusation that it staff had accidentally downloaded sensitive computer files upon another users computer. the ethical dilemma is if it happened, why? if the it staff did download the material accidentally then it is an issue of professionalism, if they did with intention to cause harm it is an issue of unethical behaviour. at the end of the day no criminal charges were placed and the matter was resolved. if some simple ethical guidelines had been applied the whole series of events would never have occurred in the first place. conclusions as stated before in regards to the auscert survey, computer crime is a problem within australia and to resolve this some organisations are looking to hackers to solve their security problem. is this ethical? ethics is an extremely important component of information security, but the problem is that information security tends to just concentrate on internal processes of access and amendment rights. the introduction of deception techniques (honeypots & honeynets) to trap hackers can technically be effective and has been proven to work. however, from an ethical viewpoint should deception techniques be used to capture hackers? ajis vol. 10 no. 2 may 2003 156 references auscert (2002) 2002 australian computer crime and security survey, university of queensland, australia. coldwell r (1994) perceptions of computer crime, conference proceedings crime against business, australian institute of criminology, march, 1994. coldwell r (1995a) australian attitudes toward legal intervention into hacking. communications of the acm, 38, 11. cohen, f (2000). deception tool – kit, url: http://all.net/dtk/dtk.html dreyfus s (1997) underground hacking, madness and obsession on the electronic frontier, random house, australia. dreyfus s (2002) cracking the hackers code, the age, melbourne, 20 th august. hughes, g (1990) computer crime: the liability of hackers, the australian computer journal, 22, 2, may. kamay v and adams t (1992) the 1992 profile of computer abuse in australia, acarb, centre for computer abuse research at rmit, melbourne, australia. kingsley p (2002) queen v boden, acisp 2002 forensic computer workshop, melbourne. levy, s (1994). hackers:heroes of the computer revolution. unknown. lopez-fernandez m. and warren m.j (2002) an overview of australian hacking (2001), school of it technical report – trc 12.08, geelong, australia. network associates (2000) cybercop sting . url: http://www.nai.com/asp_set/products/tns/ccsting_intro sterling b (1993) the hacker crackdown: law and disorder on the electronic frontier, mass market paperback, usa. spitzner, l. (2000a) to build a honeypot. url: http://project.honeynet.org/papers/honeypot/ spitzner, l. (2000b). know your enemy: a forensic analysis, url: http://www.securityfocus.com/focus/ih/articles/foranalysis.html spitzner, l (2000c) to build a honeynet, first (forum of incident response & security teams) conference 2000, chicago, usa. ajis vol. 5 no. 2 may 1998 efficient consumer response (ecr): a survey of the australian grocery industry john k. harris paula m.c. swatman electronic commerce research group department of information systems monash university e-mail: paula.swatman@is.monash.edu.au abstract efficient consumer response (ecr) is a u.s. supply chain management strategy which attempts to address the inefficiencies which have led to excessive inventory and unnecessary costs at all levels within the grocery industry supply chain. this paper discusses the traditional grocery store format, the supermarket, and the ways in which inefficient business practices developed in the u.s. grocery supply chain; and discusses the major business activities needed for successful implementation of ecr. the paper then presents a brief summary of the results of a survey of ecr knowledge and usage within the australian grocery industry, which is the initial phase of a long term research project whose main purpose is to evaluate ecr as it applies to that industry. keywords food industry, efficient consumer response, supply chain management, exploratory research, strategic is, strategic alliances, business process re-engineering. introduction efficient consumer response (ecr) originated in the united states in 1992 as a direct result of threats from alternative store 'formats' (or types) and their supply chains (mckinsey & co. 1992) which highlighted major inefficiencies within the traditional grocery store format, the supermarket, and its supply chain (kurt salmon associates 1993). the concept on which ecr is based originated from the quick response (qr) strategy already existing in the textile and apparel industries (cooke 1994) which, in turn, were based on the manufacturing justin-time (jit) concept (ellram, la londe and weber 1989; knill 1990). ecr is concerned with changing, or reengineering, the grocery supply chain from a 'push system' to a 'pull system' — where trading partners form new alliance relationships and the replenishment of store products is initiated by the point of sale (pos) data. the jit concept was simple: to deliver raw material to production areas in the exact required amount at the precise time it was needed. the use of raw material pulls new raw material into the production process. during the mid 1980s the jit manufacturing concept was applied to the u.s. textile and apparel industries in an attempt to combat market penetration by overseas manufacturers. this textile and apparel industries initiative was termed 'quick response' and attempted to reduce the amount of inventory held within the apparel supply chain. quick response required the retailer to share point-of-sale-scanned data with manufacturers to improve the flow of product through the supply chain. the grocery industry noted the success of the quick response approach to managing supply chain data and proposed a similar stock replenishment system called ecr (cooke 1994; ellram, la londe and weber 1989). grocery industry supply chain practices and the implications of inefficiency the traditional grocery retail store, the supermarket, developed during the 1920's in the u.s. the supermarket is a large, departmentalised retail store (pintel and diamond 1991) which operates on a self serve basis — selling primarily grocery products and, occasionally, a small assortment of non-grocery merchandise. during the 1930's supermarkets experienced explosive growth due to the economic depression and the mobility provided by the automobile — consumers were no longer restricted to using the local, limited-line food store, but could travel to more distant but cheaper supermarkets (hayward and york 1976). the supermarket with its high operating efficiencies quickly became the popular and dominant retail grocery format, deposing the (limited line) food store by selling goods at lower prices (morgenstein and strongin 1992). during the 1940's and 1950's the supermarket became the major food distribution channel in the u.s. during the 1950s the u.s. became saturated with supermarkets, which led to the development of innovative ways to maintain profits. these included the use of private brand labels, stamps and games. large supermarket chain stores were the primary initiators of private brand labels as they were able to reduce product, distribution and selling costs to such an extent that these products could be sold at a considerable discount to consumers and still 32 ajis vol. 5 no. 2 may 1998 return a greater profit to the retailer (national commission of food marketing 1971). stamps were issued to customers in amounts based on the volume of grocery purchases. the collection of a suitable number of stamps could then be exchanged for a variety of products such as household goods, toys and recreation equipment (mayo 1993). the use of games was introduced during the early 1960s by supermarket chains in an attempt to build consumer loyalty by encouraging shoppers to purchase more regularly from the same store to increase their chance of winning a prize. consumers were given free game slips based on the purchase value at the check-out station (allvine 1971). discounting, based on low price and high-pressure advertising, began replacing the use of stamps and games as a promotional tool during the 1970s. it is estimated that such consumer promotional activities added as much as 4% to the costs of food items and as much as 20% to the overhead costs of operating a supermarket (mayo 1993). other supermarkets discounted a large number of well known leading items in an attempt to attract customers to the store, who would then purchase other non-discount items or buy items on impulse. discounting resulted in larger and larger physical store sizes, as profits accrued only through huge volume, frequent sales (forbes magazine 1971). to attract additional higher sales the discount supermarket added additional or larger departments such as pharmacy, bakery and general merchandise in an attempt to create a one-stop-shopping store. during this same period supermarket operators began using hard-hitting newspaper advertisements with pricing comparisons and shopper testimonials. these advertisements would often include coupons — which could be used either by manufacturers to boost the sale of a particular product, or by retailers to improve sales of a particular product within a particular store. the presentation of the coupon by the consumer at the supermarket checkout results in a discount for the coupon item being purchased (peak and peak 1977). couponing has remained the most prevalent form of consumer promotion in the u.s. grocery industry today, although the use of stamps and games as a consumer promotional tool has almost disappeared. couponing is highly paper intensive and costly to administer, with only around 2.6% of coupons being redeemed annually (kurt salmon associates 1993). despite its continued use, the coupon is thus considered an inefficient promotional tool. changes in demographic patterns caused a slowing of overall u.s. grocery sales during the 1970s and 1980s: population growth decelerated, following the post-war boom; increasing numbers of consumers began to eat out and, in consequence, spent less of their disposable incomes on food prepared and eaten at home (garry 1995; sansolo 1994); and alternative store formats began to market grocery items aggressively in new ways (newton 1993). at the same time, grocery industry organisations sought to increase their profits at the expense of their supply chain trading partners, leading to adversarial relationships between sellers and buyers (michelson 1995). the complexity of deals between grocery trading partners and the performance measurement systems used led to large inventory hodlings throughout the supply chain, with sellers (manufacturers, distributors and/or brokers) continually offering deals to buyers (wholesales, distributors, brokers and/or retailers) to 'push* products into the supply chain (sansolo and garry 1992; schlossberg 1992; teinowitz 1993). in most transactions, the seller is trying to sell as much as possible at the highest possible price, while the buyer is trying to buy only what is needed as cheaply as possible (liesse 1994); rather than developing trading partner relationships based on mutual benefit. the traditional key performance measurement systems used for selling and buying fuelled these adversarial relationships, since the measurement system for the seller was sales and shipments and that for the buyer was gross margins (hastings 1994). the sellers would start with very high selling prices which would later be discounted for trade promotional activities or to meet selling objectives within a given time period. buyers would seek to increase their gross margins by buying additional inventory during the discount period and selling at regular prices after the discount period had ended ('investment' or 'forward' buying) or by re-selling it immediately at a profit to buyers in other regions which had not been offered the discount (known as 'diverting') (harding 1995). as forward buying and diverting became more widespread, sellers (particularly manufacturers) realised that a growing proportion of their trade promotion spending was not reaching the consumer in the way of reduced prices, but instead was subsidising the buyer's business operation. sellers, in an attempt to force buyers to pass on the savings offered by trade promotions to the consumers, continued to offer more and more complex deals (kurt salmon associates 1993). it was the complexity of these deals between grocery trading partners which led to excessive inventory throughout the supply chain, with suppliers continually offering deals to wholesalers/distributors and/or retailers to 'push' products into the supply chain (schlossberg 1992; teinowitz 1993). while margins remained high, supermarket operators tolerated these inefficient practices — despite their tendency to create increased transportation, administration and warehousing costs. but as more and more of the u.s. grocery supply chain engaged in complex deals, forward buying, diverting and promotional activities, any real competitive advantage which might have existed originally was lost, although the additional costs continued (and increased). it was clear that a more efficient means of moving goods through the supply chain was needed — in particular, one which would encourage the growth of improved trading partner relationships, while 33 ajis vol. 5 no. 2 may 1998 reducing inventory holdings. efficient consumer response ecr calls for the creation of a timely, accurate and paperless flow of information — relying heavily on electronic data interchange (edi) and strategic alliances between supply chain members (sansolo 1993). the goal of ecr is to take out of the supply chain costs which do not add consumer value (robins 1994). ecr is about producing efficiencies in the grocery supply chain within the four core business process areas of efficient store assortment, efficient replenishment, efficient promotions and efficient product introductions (kurt salmon associates 1993). ecr business activities/initiatives to achieve these four efficiencies, ecr requires the following major business activities or initiatives (deroulet 1993): (a) continuous replenishment program (crp); (b) computer assisted ordering (cao); (c) flow-through distribution (cross-docking); (d) activity-based costing (abc); (e) category management; (f) integrated electronic data interchange (edi). (a) continuous replenishment program (crp) continuous replenishment, usually managed by the manufacturer, is a program used to control and monitor the movement of goods from the manufacturer to the warehouse/distributor (garry 1994c). crp involves the manufacturer (rather than the retailer's warehouse) taking responsibility for replenishing the warehouse inventory, with the buyer supplying actual warehouse inventory withdrawal data and data on 'stock-keeping units'(individual line items) to the manufacturer (cross 1993). crp programs reduce costs in distributors' inventory, but can increase some costs, such as transportation costs, if the manufacturer ships smaller truck loads more frequently (garry 1994b). successful crp implementation is dependent on effective trade relations, requiring shared business practices and information systems which rely heavily on edi. (b) computer-assisted ordering (cao) computer-assisted ordering, also known as 'computer-aided ordering' (fensholt 1992; garry 1992; thayer 1991; weinstein 1995), covers the second half of the overall inventory supply chain — the movement of goods from the warehouse/distribution centre to the retail store. the aim of cao is to generate store replenishment orders automatically, with minimal management intervention, based on such things as current and historical pos scan data, delivery data and sales forecasts. the benefits of cao have been identified as labour savings and dependability, warehouse and shipping improvements, and inventory reduction (garry 1994c). traditionally, stores have based their orders on the re-order clerk manually inspecting the store shelves and scanning the shelftag bar codes for those items with limited stock on the shelf (anderson 1996). the re-order amount entered by the clerk is based on the actual shelf amount and the ideal shelf quantity. the re-order clerk is not in a position to take into account pos data, inventory which has already been scheduled for delivery, or likely future trends based on forecasting. integrated cao systems are designed to minimise (and even eliminate) these problems. (c) flow through distribution (cross-docking) according to garry (1994a) the purpose of flow through distribution is to hasten the flow of products from the supplier to the retail store by reducing storage and handling of products at the distribution centre or warehouse. it involves the breaking down of pallets at the distribution centre, reassembling them for store delivery and then shipping them to the retail store without ever storing the product in the warehouse. this requires significant investment in technologies such as edi, bar coding and scanning of pallets, and cases; and warehouse design changes such as lower ceilings and less racking. the key edi transaction required for cross-docking is the advanced shipping notice (asn), to inform the distributor of the merchandise that is about to arrive. the automation of the warehouse inventory management system using bar codes means that these inaccuracies can be eliminated. 34 ajis vol. 5 no. 2 may 1998 (d) activity-based costing (abc) activity-based costing provides the cost and operating information necessary to support innovative management improvement initiatives such as ecr. the focus of abc is on accurate information about the true cost of products, services, processes, activities, distribution channels, customer segments, contracts and projects (miller 1996). abc supplies information about profits (where the money is being made) rather than about costs. traditional accounting systems use gross margin calculations that spread operating costs across all products based on unit purchase price regardless of the actual value chain (porter 1985) the product passes through. abc focuses management's attention on controlling the source of costs, decisions that create activities, rather than squeezing budgets. therefore abc as part of ecr can increase the profitability of the supply chain by removing or reducing those cost activities that do not add value. this can not be done with traditional systems because they do not reflect costs accurately (weinstein 1993). (e) category management the term category management first appeared in 1987 (smith 1993) when certain organisations, such as procter and gamble (mathews 1995a), began moving from 'brand' management to management 'by category'. category management has evolved to mean a process that involves managing product categories as business units and customising them on a store-by-store basis to satisfy consumer demands (gnau, richardson and dippold 1992). a category is a group of products having a common consumer end use (hofler 1996) and include such things as household cleaners, dairy and frozen foods, paper products, health and beauty care products, soft drinks, etc. category management allows the category manager to operate a category like a business so as to identify optimal product mix; and to stock each store with specific products that demographic and pos information indicate customers wish to purchase. (f) integrated electronic data interchange (edi) electronic data interchange is the computer-application to computer-application communication of structured, formatted messages based on international standards, using electronic transmission media with no manual intervention (brawn 1989; swatman 1993). edi is a technology which allows structured information to be shared among organisations in the supply chain resulting in significant reductions in transaction costs and enabling the organisations to adopt new and more effective and efficient business strategies (emmelhainz 1990; gilmour 1993; klima 1993; spence 1994), such as ecr. edi is viewed as the essential effective enabler of the ecr management strategy because it focuses on achieving integration across organisational functions and between organisations (swatman 1993) in the grocery supply chain. an australian ecr survey at the start of this research project, little u.s. and almost no australian research into ecr could be discovered. we identified three academic research projects on ecr or continuous replenishment (hoban 1993; clark 1994; mathews 1995) in the u.s. and in australia we found only a single industry report undertaken for the grocery manufacturers of australia by coopers & lybrand in 1995. we therefore decided to commence the overall research project (which involves a number of researchers and is expected to take several years) by investigating australian attitudes to ecr and existing practices by members of the grocery industry in this country — an investigation whicih we believed was most appropriately addressed by means of a mail survey. participants were chosen from those organisations directly involved in the grocery supply industry and which considered themselves to be members of die grocery supply chain. the questionnaire was posted to the most senior executive of 1500 grocery industry supply chain members, whose mailing lists were obtained from grocery industry journals (retail world pty ltd 1995; white, craig, wiland et al. 1995); and an effective response rate of 30%, or 450 organisations, was obtained after two mailouts of the questionnaire. the research question under examination was "can australian grocery industry supply chain members beefit from the efficient consumer response supply chain management strategy"? industry supply chain members were surveyed to: • determine the extent of inefficient business practices; • identify the current use of business activities (supply chain initiatives) necessary for the adoption of the ecr strategy; • identify those organisations that are aware of the ecr strategy to discover whether their 35 ajis vol. 5 no. 2 may 1998 organisation (and the supply chain as a whole) can benefit from the ecr strategy; • determine the reasons for adopting ecr by those organisations which have decided to pursue such a strategy. inefficient business practices figure 1 shows that 'diverting' is little used within the australian grocery industry, as just 17% of survey respondents indicated that they use some form of diverting, and only 2% stated that they used this practice the majority of the time. deal selling/buying which results in forward buying, however, is prevalent within australia and, as a result, there are likely to be excessive inventory holdings throughout the australian grocery supply chain. despite the greater than optimal inventory holdings, these figures suggest that inefficient promotional activities are used to a much lesser extent than in the u.s. and are therefore far less important as a motivator for ecr in this country. 100% deal selling/buying diverting foiward buying supply chain practice promotional activities ecr business activities figure 2 illustrates the various ecr initiatives and shows that category management (59%) and edi (49%), have the greatest number of respondents committed to their implementation, while cross-docking (19%) and continuous replenishment (23%) have the smallest number. even though category management (59%) is more widely used than the other initiatves, only 10% of respondents are committed to the implementation of this business activity as part of their ecr strategy. edi (17.1%) implementation as part of an ecr strategy has by far the greatest commitment in terms of ecr strategy, while activity-based costing (4%) and cross-docking (5%) have the lowest respondent commitment as part of their ecr strategy. these results suggest that although australian grocery industry members are actively engaged in a number of the ecr initiatives, they view each activity on its own merit — and have, thus far at least, shown little interest in integrating their improved supply chain management practices into an holistic ecr approach. 36 ajis vol. 5 no. 2 may 1998 100% tj c o ain 2 80%•"34tc 231 29% 59% i 0% electronic data interchange continuous replenishment cross-decking computer assisted ordering activity-based costing category management ecr related business activity committed to implementation committed to implementation (ecr) ecr awareness table 1 shows that just over two-thirds (61%) of respondents were previously aware of the ecr strategy, while another third (34%) were unaware of ecr before receiving this survey. two-thirds of the group of 'aware' organisations are actively engaged in pursuing an ecr strategy and, in table 2, we have summarised the reasons why these respondents are actively pursuing this strategy. this list makes it clear that the 'threat from alternative store formats' which motivates so many u.s. supermarkets is of little importance to australian grocery industry members. ecr status not aware of ecr aware of ecr: actively pursuing ecf not actively pursuing ecf unknown if actively pursuing ecf other percent 34% 61% 409! 169? 59? 5% table 1. respondents ecr status. reason improve efficiency of your organisatior pressure from trading partner^ cost savings to your organisatior improve efficiency of overall supply chaii cost savings to overall supply chaii threats from alternative store format! othei percent 60% 46% 45% 45% 33% 8% 6% table 2. reasons for actively pursuing and ecr strategy. figure 3 summarises respondents' views of the likely benefits of ecr to the australian grocery industry and shows that the majority of those respondents who are aware of ecr believe the australian grocery supply chain can gain significant benefit from ecr in terms of improved efficiency (76%) and reduced costs (61%); and that the respondent's own organisation can benefit (61%) from pursuing an ecr strategy. of the total group of respondents, 47% believe that ecr will improve the efficiency of the grocery supply chain, while 37% of all respondents believe that ecr will remove costs from the supply chain and also benefit their own organisation. 37 ajis vol. 5 no. 2 may 1998 improve efficiency of supply chain remove costs from supply chain respondent ecr opinion benefit respondents organisation ecr aware respondents (61 %) | all respondents (100%) discussion and conclusions our research into ecr within australia provides very strong evidence in two areas: • that the inefficient business practices of deal selling/buying and forward buying (figure 1), which inadvertently push excessive inventory into the supply chain, exist within the australian grocery industry although not necessarily to the same extent as in the u.s. • that australian organisations have begun implementing business activities and initiatives that are essential for the adoption of the ecr strategy. these results suggest that the australian grocery industry is a 'push system', like the u.s., where inventory is continually pushed into the supply chain by supplier deals, which in turn lead to forward buying by customers. there is considerable interest in ecr in australia, with 61% of respondents being aware of ecr and 40% of respondents actively pursing an ecr strategy (table 1). despite this interest in ecr, we are concerned that there is no real commitment to its implementation. this is borne out by the fact that while 40% of respondents are actively pursuing an ecr strategy (table 1) only 17% (edi) and 4% (abc) respectively of respondents are actually committed to the implementation of the essential business activities (figure 2) necessary for the successful implementation of ecr. a review of the australian grocery industry (commercial economic advisory service of australia 1991) identified inefficiencies within the australian grocery industry which ecr seeks to address. this review even identified threats from alternative store formats which were likely to have an adverse effect on the traditional grocery store, the supermarket. given the findings of this review, we find it somewhat surprising that respondents to our survey did not indicate threats from alternative store formats as a main reason for adopting ecr, and that there is not a stronger commitment to implement the necessary ecr business activities/initiatives. one of the main reasons given for adopting ecr is pressure from trading partners (table 2). if organisations are pressured and/or coerced into using an ecr strategy we do not believe that they can be committed to its implementation and therefore gain the potential benefits from such implementation. these results, which differ significantly from the (comparatively limited) evidence available from the u.s. suggest that the australian motive for engaging in ecr may well be quite different. indeed, the likelihood is that australian grocery industry members appear to be 'encouraged' to engage in ecr by large and powerful customers (supermarkets) — a situation which is reminiscent of the australian experience with edi. there is thus a real danger that, as with edi, organisations may become involved only to the extent required by their larger and more powerful trading partners, so that the full benefits of a strategic approach may be lost. the survey reported in this paper provides the first 'hard' evidence of australian ecr activities and attitudes. one thing which has become clear from the results is that, as with all surveys (which provide a 'snap-shot' of a respondent group at a point in time) we have made some fascinating discoveries, but opened up some even more intriguing possibilities which the data cannot answer satisfactorily. the next step is clearly to engage in case studies of australian and overseas organisations involved in ecr to discover whether our suspicions about the motivations for ecr involvement are correct. this project is being undertaken by another member of the monash electronic commerce research group and our serendipitous discovery that the member nations of the european union have nominated ecr as a major area of interest for the next few years encourages us to hope for an enthusiastic response from prospective case study participants. 38 ajis vol. 5 no. 2 may 1998 references allvine, f. (1971). the future for trading stamps and games. food marketing and distribution. d. mclaughlin and c. mallowe. new york, chain store publishing corporation, 169-183. anderson, b. (1996). the art and science of computer assisted ordering: methods for management westport, connecticut, quorum books. brawn, d. (1989). edi developments abroad and how they impact australia. idc conference: edi the key to profitability in the 1990s, sydney, australia, idc conferences. chain store age (1971). behind the new wave discount super revolution. food marketing and distribution. d. m. a. c. mallowe. new york, chain store publishing corporation, 131-139. clark, t. (1994) linking the grocery channel: technological innovation, organizational transformation, and channel performance, harvard university: phd thesis. commercial economic advisory service of australia (1991). grocery industry review: june 1991. st leonards, commercial economic advisory service of australia. cooke, j. (1994). logistics quality: part in beyond quality... speed. traffic management 33(6), 32-37. cross, g. j. (1993). continuous replenishment planning: an untapped gold mine. transportation and distribution. dec. 34(12), 49. deroulet, d. g. (1993). ecr: better information cuts costs. transportation & distribution. 34(10), 63. ellram, l. m., b. j. la londe and m. m. weber (1989). retail logistics. international journal of physical distribution & materials management 19(12), 29-39. emmelhainz, m. (1990). electronic data interchange: a total management guide new york, van nostrand reinhold. fensholt, c. (1992). presenting the 1992 supermarket business award for excellence to shaw's supermarkets, inc. supermarket business. 47(12), 23-25,81. forbes magazine (1971). the grocery business: maybe the stockholders should be picketing instead. food marketing and distribution. d. mclaughlin and c. mallowe. new york, chain store publishing corporation, 60-70. garry, m. (1992). how's your scan data? progressive-grocer. 71(10), 83-90. garry, m. (1994a). a blueprint of the future. progressive grocer. 73(10), 63-66. garry, m. (1994b). is there life after crp? progressive grocer. 73(9), 73-78. garry, m. (1994c). the stepping stone to ecr. progressive grocer. 73(6), 59-60. garry, m. (1994d). where's the beef. progressive grocer. 73(4). garry, m. (1995). industry snapshot progressive grocer. 74(1), 11. gilmour, p. (1993). integrated logistics management. logistics management: an australian framework. p. gilmour and r. lansbury. london, longman cheshire, 3-51. gnau, k., t. richardson, j. dippold (1992). nielson category management: positioning your organisation to win. chicago, ntc business books/american marketing association. harding, p. (1995). perspective: qr and ecr, kurt salmon associates. hastings, p. (1994). capabilities will supersede brands. purchasing and supply management (logistics supplement dec, 4-5. hayward, g. and g. york (1976). the food store distributor: a retail sales guide new york, chain store publications corporation. hoban, t. (1993) efficient consumer response (ecr): perceived barriers and opportunities, raleigh, north carolina state university: phd thesis. hofler, r. (1996). glossary of grocery industry terms. stamford, progressive grocer associates. klima, g. (1993). edi at supervalu "past • present future". the sixth international conference electronic data interchange & interorganizational systems, bled, slovenia. knill, b. (1990). quick response: now for the hard part. material handling engineering. 45(3), 67-78. kurt salmon associates (1993). efficient consumer response: enhancing consumer value in the grocery industry, american meat institute, food marketing institute, grocery manufacturers of america, national food brokers association, uniform code council. liesse, j. (1994). the nitty-gritty of ecr systems: how one company makes it pay. advertising age 65(19), s-l. mathews, r. (1995a). the power of category management. progressive grocer. 74(8), 12-14. mathews, r. (1995b). spartan pulls the plug on vmi. progressive grocer. 74(11), 64-65. mayo, j. (1993). the american grocery store: the business evolution of an architectural space. westport, greenwood press. 39 ajis vol. 5 no. 2 may 1998 mckinsey & co. (1992). evaluating the impact of alternative store formats. chicago, supermarket industry convention, food marketing institute. michelson, p. (1995). the emerging shape of retail logistics. managing retail logistics conference, melbourne, australia, ibc conferences pty ltd. miller, j. (1996). implementing activity-based management in daily operations. new york, john wiley & sons. morgenstein, m. and h. strongin (1992). modern retailing: management principles and practices. englewood cliffs, regents/prentice hall. national commission of food marketing (1971). private branding by retailers. food marketing and distribution. d. mclaughlin and c. mallowe. new york, chain store publishing corporation, 151161. newton, d. j. (1993). nontraditional retailers challenge the supermarket industry. foodreview. 16(1), 2-7. peak, h. and e. peak (1977). supermarket merchandising and management englewood cliffs, prentice-hall. pintel, g. and j. diamond (1991). retailing. englewood cliffs, prentice-hall. porter, m. (1979). how competative forces shape strategy. harvard business review. march-april, 137-145. porter, m. (1985). competative advantage: creating and sustaining superior performance. new york, the free press. retail world pty ltd (1995). grocery industry marketing guide 1995. rozelle, retail world pty ltd. robins, g. (1994). sailing into ecr's uncharted waters. stores. 76(10), 43-44. sansolo, m. (1993). ecr. progressive grocer. 72(11), 47-50. sansolo, m. (1994). consumer spending shrinks. progressive grocer: 61st annual report of the grocery industry. 73(4), 11. sansolo, m. and m. garry (1992). buyer/seller relations: the rumors of peace remain unfounded 59th annual report of the grocery industry. progressive grocer supplement 71(4), 24-26. schlossberg, h. (1992). grocers seek peace treaty in war with manufacturers. marketing news. 26(12), 1617. smith, k. (1993). no brand too small. progressive grocer. 72(12), ss4-ss5. spence, m. a. (1994). edi quickly becoming the norm. cma magazine 68(5), 14-16. swatman, p. (1993). integrating edi into existing organisational structure and internal application systems: the australian experience. school of computing., curtin university, phd thesis. teinowitz, i. (1993). grocery trade deals set to enter new era. advertising age 64(21), 3. thayer, w. (1991). computer-aided ordering is ready . . . should you care? progressive grocer. 70(3), 8186. weinstein, s. (1993). on the path to better selling. progressive grocer. 72(11), 54-55. weinstein, s. (1995). on the cutting edge. progressive grocer. 74(8), 34-40. white, m., e. craig, e. wiland, et al. (1995). the food week industry yearbook and directory of suppliers 1995 edition sydney, ian huntley pty ltd. wood, p. k. (1996). preparing for ecr at the store level. chain store age 72(5), 230. weinstein, s. (1993). on the path to better selling. progressive grocer 72(11), 54-55. weinstein, s. (1995). on the cutting edge. progressive grocer 74(8), 34-40. white, m., e. craig, e. wiland, et al. (1995). the food week industry yearbook and directory of suppliers 1995 edition. sydney, ian huntley pty ltd. wood, p. k. (1996). preparing for ecr at the store level. chain store age 72(5), 230. 40 the australasian journal of information systems (ajis) began in 1993 and was based on the simple realisation that there was a need for an academic ‘outlet’ for is research in australia australasian journal of information systems volume 14 number 1 nov 2006 a review of information systems programs in universities in victoria carol pollard appalachian state university boone, north carolina, usa pollardce@appstate.edu elsie s. k. chan australian catholic university victoria, australia elsie.chan@acu.edu.au abstract this paper outlines the current situation of information systems (is) programs within the state of victoria, australia. it reports on how victorian universities are addressing the challenges associated with reducing local and international student demand, and hence enrolments, at a time when is in particular and ict in general are seen by the business sector as necessary components contributing to organisational success. transcripts of interviews with 14 academicians at nine universities throughout victoria are analysed to give a current profile of is programs and identify the trends in their development over time. first, a profile of the state of victoria, its education system and its ict industry is provided to place this work in context. next, the interview sample is described and a number of relevant topics of interest are identified and discussed, comparing and contrasting the various programs. finally, a summary of the findings is provided in light of the framework used to guide all of the studies of the australian is programs described in this special issue. keywords: is programs, higher education, is enrolments, is research introduction tertiary education in computing in australia originated in the ‘institute of technology’ sector and only moved into the more “traditional” universities following the major changes to the australian university system initiated by labour’s minister for education john dawkins in 1987. in turn, this led to the ‘traditional’ universities beginning to offer degrees and post-graduate qualifications in information systems (is), originally referred to as electronic data processing (edp) and business computing (tatnall, 1993). information systems is a fascinating and pervasive discipline that has 231 australasian journal of information systems volume 14 number 1 nov 2006 struggled over the past three decades to establish itself as a distinct scientific discipline. this has proved somewhat difficult and elusive possibly in part due to the fact that is is not confined to only business activities it also profoundly affects our social activities (see, for example, buckingham et al., 1987; desouza et al., 2006; lee, 2001; lo, 1989; fielden, 1990; ang, 1992; ang and lo, 1991; avison, 1993; acm, ais and aitp, 1997; clarke, 1999; tatnall, 1999). ...the information systems field examines more than just the technological system, or just the social system, or even the two side by side; in addition, it investigates the phenomena that emerge when the two interact. (lee 2001, iii-vii) in a recent communication from sid huff, chair of is at university of auckland, to an international listserv, he referenced a recent it governance institute report that reported on the high importance of is to the business value of organisations. this paper reports on a study within a larger study of is in australia that reports on the evolution of is teaching and research programs throughout australia. first, a profile of the state of victoria, its education system and its ict industry is provided to put the work in context. next, the interview sample is described and a number of relevant topics of interest are discussed comparing and contrasting the various programs. finally, a summary of the findings is provided in light of the framework used to guide all of the studies of the australian is programs described in this special issue. demographics of victoria the state of victoria is located in the south-eastern corner of the mainland of australia. it is the smallest mainland state in area, representing only 3% of the australian land mass, but it is the most densely populated and urbanised state. victoria began as a farming community in the 1800s. in june 2005, victoria's population reached an estimated 5,022,300 making it the second most populous australian state, after new south wales (wikipedia, the free encyclopedia, 2006). currently, nearly 90% of victoria’s residents live in cities and towns. melbourne is victoria's capital and largest city, with more than 70% of all victorians living there. the sprawling melbourne metropolitan area is home to an estimated 3.6 million people. other important victorian urban centres include geelong, ballarat, bendigo, shepparton, warrnambool and the latrobe valley. education in victoria currently, victoria has nine public universities (table 1). the oldest, the university of melbourne, enrolled its first student in 1855. the largest, monash university, has an enrolment of nearly 56,000 students more than any other australian university in 2004. two of victoria’s universities (university of melbourne and monash university) are members of the group of eight (the go8 represents australia's leading universities), the total number of students enrolled in victorian universities was 241,755 in 2004, an increase of 2% over the previous year. international students make up 30% of all enrolments. the largest number of enrolments were recorded in the fields of business, administration and economics, with nearly a third of all students, followed by arts, humanities, and social science, with 20% of enrolments (australian bureau of statistics, department of education and training (victoria), department of education, science and training (commonwealth), national centre for vocational education research cited in wikipedia, the free encyclopedia). 232 http://en.wikipedia.org/wiki/australia http://en.wikipedia.org/wiki/new_south_wales http://en.wikipedia.org/wiki/melbourne http://en.wikipedia.org/wiki/geelong%2c_victoria http://en.wikipedia.org/wiki/ballarat%2c_victoria http://en.wikipedia.org/wiki/bendigo%2c_victoria http://en.wikipedia.org/wiki/shepparton http://en.wikipedia.org/wiki/warrnambool http://en.wikipedia.org/wiki/latrobe_valley http://en.wikipedia.org/wiki/latrobe_valley http://en.wikipedia.org/wiki/university_of_melbourne http://en.wikipedia.org/wiki/monash_university http://en.wikipedia.org/wiki/business http://en.wikipedia.org/wiki/administration http://en.wikipedia.org/wiki/economics http://en.wikipedia.org/wiki/arts http://en.wikipedia.org/wiki/humanities http://en.wikipedia.org/wiki/social_science australasian journal of information systems volume 14 number 1 nov 2006 students may obtain the victorian certificate of education on completion of secondary school education. they receive a tertiary entrance ranking, or enter score, to determine university admittance. from 2005, higher education providers are funded under the commonwealth grant scheme (cgs) to deliver a specified number of commonwealth supported places each year. table 1 shows the agreed number of fully funded places, the minimum number of undergraduate places and the cgs payment to each higher education provider in 2005. name of university website abbreviation student places1 (2005) under grads2 (2005) cgs payment3 $ million australian catholic university4 www.acu.edu.au acu national 6,671 6,183 47.7 deakin university www.deakin.edu.au deakin 13,963 13,606 97.2 la trobe university www.latrobe.edu.au la trobe 14,375 13,552 103.3 monash university www.monash.edu.au monash 21,780 20,469 160.7 royal melbourne institute of technology www.rmit.edu.au rmit 13,396 12,977 109 swinburne university of technology www.swinburne.edu.au swinburne 5,322 5,309 39.4 the university of melbourne www.unimelb.edu.au melbourne 18,595 17,285 150.9 university of ballarat www.ballarat.edu.au university of ballarat 3,305 3,210 261.6 victoria university www.vu.edu.au victoria university 9,537 8,807 62.6 table 1 australian government fully funded places and cgs payment by university in victoria, 20055 sophisticated ict industry in victoria victoria is home to a vibrant and sophisticated information and communication technology (ict) industry which boasts a strong component of locally grown companies. australian-owned firms account for approximately 69% of the industry in victoria. it is a centre for research and development which is leading the australian (and often global) research and development programs across a broad range of industry sectors. in 2001-02, victoria’s globally focused ict industry had a turnover of aud$19.8 billion (us $15.3 billion) and a skilled and creative workforce of 60,000. with export revenue of aud$615 million (us $476.4 million) and r&d expenditure of aud$303 million (us $235 million), the ict 1 a commonwealth supported place is one equivalent full time student load (eftsl). 2 total commonwealth supported places excludes research training scheme places. 3 cgs payments include enabling loading, regional loading, and medical student loading. 4 acu figures in table 1 are for acu multi-state campuses, not only victoria. 5 source: commonwealth of australia 2005, pp. 45-46 233 http://www.acu.edu.au/ http://www.deakin.edu.au/ http://www.latrobe.edu.au/ http://www.monash.edu.au/ http://www.rmit.edu.au/ http://www.swinburne.edu.au/ http://www.unimelb.edu.au/ http://www.ballarat.edu.au/ http://www.vu.edu.au/ australasian journal of information systems volume 14 number 1 nov 2006 industry is at the heart of the modern victorian economy. although victoria accounts for less than 25% of australia’s population, it is home to 31% of all australian ict jobs. these comprise 48% of all jobs in hardware manufacturing, 41% of jobs in software engineering and 35% of all ict consulting jobs (government of victoria, 2005). leading israeli it services and solutions provider ness technologies, inc. will open their australia new zealand headquarters in melbourne creating up to 50 new jobs, announced by the minister for information and communication technology, marsha thomson in april 2006 (government of victoria, 2006). high internet usage rates and world-class infrastructure make victoria a test-bed for e-commerce companies and, as a result, victoria has emerged as australia's leader in business-to-business and business-to-consumer e-commerce. recently, it was reported that melbourne in particular has a critical mass of creativity and skills to develop a competitive ict hotspot (newcomersnetwork.com, 2006). these statistics provide a natural conduit for a discussion of the evolution of the many strong ict programs that exist within victoria universities. research method walsham (1993; 1995) recommends case studies for interpretivist research, although this is by no means the only way in which case studies can be used, as clarified by yin (2003). a case study is used to explore or describe a particular issue within a specified unit of study (benbasat et al. 1987; shanks et al. 1993; miles & huberman 1994 and yin 1994). hence a qualitative, interpretivist approach was chosen to conduct the research upon which this paper is based. within and between case analysis was performed to offer a rich description and comparison of is programs at the nine universities represented, as indicated in table 1. face-to-face interviews were conducted with 14 senior academics in 2005. the number of interviews conducted at each university ranged from one to three, depending on availability of participants and included one key person from each university as the primary source of data. the semi-structured face-to-face interviews were primarily based upon the standardised interview protocol developed for use in the larger is-in-australia study. at the outset of each meeting, the researcher opened the session with a set of standard introductory remarks designed to (1) indicate the importance and purpose of the interview, (2) give assurance of anonymity and confidentiality to the participant and (3) establish rapport. each interview began with elicitation of demographic information (name, title, department) and then seeking information on the following topics of interest: • relative size and administrative placement of the is presence at their university • the extent to which is at the university is impacted by local contingencies • the extent to which is is identified as a separate field at their university • distinctive features of the is curriculum at their university • distinctive features of is research at their university • the key people who have impacted is in universities in victoria throughout the interview, spontaneity and tangential discussion were invited, to encourage participants to reveal possibly useful anecdotal data. mintzberg (1979) advocated the use of this technique and described its importance: for while systematic data create the foundation for our theories, it is the anecdotal data that enable us to do the building. theory building seems to require rich description, the richness that comes from anecdote. we uncover all kinds of relationships in our 234 australasian journal of information systems volume 14 number 1 nov 2006 hard data, but it is only through the use of this soft data that we are able to explain them. (p. 583) with agreements from the interviewees, most interviews were tape recorded in their entirety to minimise possible interviewer bias. upon transcription, interviewees were provided with a copy of their transcript and asked to provide feedback; revisions were made by the researchers as necessary. available documentation and archival material was also collected and analysed to provide some triangulation of data (denzin and lincoln, 1998). information systems programs in universities in victoria data was collected at nine universities across victoria, on the topics listed above based on an analysis of the interview transcripts and the available documentary evidence, each of these areas is explored in detail in the following sections. relative size and administrative placement of the is presence the total number of is tertiary students in victoria is approximately 7,000. a comparison of the size of the is presence ranges from 76 students at australian catholic university (melbourne campus) to approximately 1,000 at university of ballarat and monash as shown in table 2. full time is academic staff number from 3 to 50. however, it was noted by those interviewed at monash that their faculty size had been culled significantly since 2004 due to falling student numbers over the past several years. and, in a message from rodney turner to aisworld listserv isworld, dated may 16 2006, it was announced that victoria university was reducing its is staff from 26 to 14 (53.8%) as a beginning measure in staff reduction. name of university faculty department, school or group full time is academic staff in 2005 no. of is students in 2005 acu national arts and sciences business and informatics 3 76 deakin business and law science and technology information systems; information technology 30 38 150 3155 la trobe science, technology and engineering computer science and computer engineering 16 252 monash information technology berwick, caulfield, clayton, gippsland 150 1000 rmit business science, engineering and technology business information technology; computer science and information technology 41 51 622 235 australasian journal of information systems volume 14 number 1 nov 2006 swinburne information & communication technologies astrophysics & supercomputing; computer science & software engineering; information systems; telecommunications 52 500 melbourne science engineering information systems; software engineering 45 400 university of ballarat n.a. information technology & mathematical sciences 54 1,010 victoria university business and law health, engineering and science information systems; computer science and mathematics 26 35 800 table 2 – is presence in universities in victoria table 2 shows that while all nine universities in victoria represented in the study offer is programs, their location within the university structure differ by institution. as a whole, is programs in universities in victoria can be categorised between business and science \ technical \ engineering faculties. six universities have departments located within faculties of science \ technical \ engineering, whereas deakin, rmit and victoria university offer information technology programs both through the business and science \ technical \ engineering faculties. the extent to which is has a separate identity within the university the data revealed a trend in an overwhelming majority of universities in victoria to recognise information systems as a separate entity (table 3). in universities where this was the case, this separate identity was expressed as “a clear demarcation between is subjects and others in the faculty” (rmit, victoria university), “a strong reputation in industry” (swinburne) and “is has long had a strong separate identity” (monash). a specific example is that for many years in the 1970s and 1980s, caulfield and the chisholm institute of technology (subsequently merged into monash) pioneered is education in victoria. an early activity was the "programmer in training" (pit) scheme of the commonwealth government. the issue of “competing” with business, information technology and computer science departments was also raised by some in connection with the existence of a separate identity. overall, the existence of a separate identity for is was viewed positively, although at melbourne it was indicated while is has a separate identity as a department, “not many people know about our department”. of the two universities (university of ballarat and la trobe) that do not afford is a separate identity, la trobe has experienced some problems with this status, including the lack of representation of is journals and conferences in a recent journal and conference ranking exercise undertaken to improve the department’s (computer 236 australasian journal of information systems volume 14 number 1 nov 2006 science and computer engineering) research profile. at university of ballarat, is does not have separate administrative status and is lecturers teach into other areas of the school of it and mathematical science. this is also true for acu national, as one of the three is lecturers teaches the business units and is does not have separate administrative status. despite the generally positive perception of having a separate identity, those senior academics interviewed were relatively evenly divided between “same” and “less” in their views on status as shown in table 3. only one senior academic at monash felt that is academics were viewed more highly than their colleagues in other departments. at the four universities in victoria where it was perceived that is academics saw themselves as having a lesser status than their colleagues a number of reasons were given. in some cases it appeared to be due to the relatively new and evolving nature of is departments in australia (melbourne, rmit, victoria university) and one of the respondents at swinburne felt that the perception of is faculty was that they are “less prominent in research than their colleagues”. at those universities where the perception was that there was no difference between the status of is academics and their colleagues no elaboration for this perception was forthcoming. interestingly, table 3 shows that status and separate identity are not necessarily correlated. for example, academics at university of ballarat considered their status “no more or less” and those at la trobe were said to have mixed perceptions of status. in contrast, at those universities that had a separate identity, many viewed themselves as having less status for the reasons described above. university separate id status acu national no same deakin yes same la trobe no same/less monash yes same/higher rmit yes less swinburne yes less melbourne yes less university of ballarat no same victoria university yes less table 3 – is identity and status in universities in victoria the extent to which is is impacted by local contingencies while it appears that universities in victoria are seeking increased collaboration with the local community and industry as part of their strategic vision, as in the strong industry-based learning degree programs at swinburne and monash universities, established around 1990 as an initiative of the business council of australia, the majority of universities felt the influence of local industries was negligible in terms of impacting their curriculum. however, efforts to increase interaction with external partners are currently underway and, in some cases, are being promoted by top university officials. in the case of victoria university it is the vice chancellor who is spearheading these initiatives. victoria university was one of the earliest universities to link the program to systems in the market. the faculty of business and law and sap signed an agreement in march 1998 to enable the university to develop courses and conduct research based around sap's enterprise resource planning system referred to as sap r/3. in the case of monash, the dean of is and full 237 australasian journal of information systems volume 14 number 1 nov 2006 professors were primarily instrumental in this area. at swinburne the faculty as a whole were attempting to increase ties to industry to “(be) alert to any possible ‘competitive advantage’ in responding to special local needs”. where local influences were present, the most influential industries named were manufacturing and consulting services. at acu national, is curricula are not affected by local factors but in fact more by the ethical influence in the catholic mission as well as ethical and social responsibilities which will influence the curricula to a certain extent. national influences, on the other hand, are affecting is curriculum across victoria. for example, monash appears to be impacted more strongly by national rather than local community and industry influences, or by individual influences within the university attributed to staffing changes over the years. likewise, swinburne saw national influences as being most influential. the cyclic nature of demand for is courses was seen, in most cases, to be the factor that influenced is enrolment most. some respondents emphasised it was not always factors “local” to their australian location that affected their is curriculum. for example, rmit’s is program was impacted by factors at their offshore hong kong location. this is not surprising since many of the universities indicated that international students accounted for a large percentage of their enrolments at both graduate and postgraduate levels. distinctive features of the is curriculum universities in victoria offer a wide range of undergraduate and postgraduate is course and programs. table 4 summarises the programs offered at the various institutions. the greatest majority of the nine institutions represented offer the standard suite of bis and bis (hons), graduate certificate, graduate diploma and masters level programs. within victoria, all universities offer phd level is programs although not all universities currently had phd students enrolled. is/it programs offered at undergraduate level and masters by course work in universities in victoria are listed in appendix 1. university bis bis (hons) bit bbis mis mbi s mit mim phd acu national x x x deakin x x x x x x x la trobe x x x x x monash x x x x x x x x rmit x swinburne x x x x x x university of ballarat x x x x x melbourne x x x x x victoria university x x x x legend: bis bachelor of information systems bis(hons) bachelor of information systems with honours bit bachelor of information technology bbis bachelor of business information systems 238 australasian journal of information systems volume 14 number 1 nov 2006 mis master of information systems mbis master of business information systems mim master of information management and systems phd doctor of philosophy table 4 – diversity of is programs offered in addition to the traditional bachelor of information systems or information technology, bachelor of business information systems and the masters of information systems, a few institutions offer cross-disciplinary courses. examples of these cross-disciplinary course are shown below. university courses acu is and business (accounting, marketing or human resource management) latrobe bachelor of computing with applied science bachelor of computing with visual arts victoria university joint degrees with arts and science and engineering faculties masters in enterprise resource planning marketing/e-commerce table 5 – examples of cross-disciplinary courses in universities in victoria it is observed that in 2002 acu national students tended to study a single degree, bachelor of information systems. starting from 2004, students preferred to study a double degree. the reason for this change may be due to the perceptions of graduates, who believe graduates with double degrees could find jobs more easily than those with a single degree. many of the universities have a large population of international students the majority of whom are from asia, with only a small representation from europe. offshore is programs are offered on campuses located in hong kong, thailand, singapore, china, and malaysia. generally speaking, the courses taught are less technical than would be found in a department of computer science or information technology. the distinctive themes taught within many of these programs varied considerably as shown below. university distinctive theme deakin electronic commerce latrobe educational theory and practice monash decision analysis and information management rmit security, decision support, usability, interface design swinburne design and support of business processes univerity of ballarat global information systems victoria university erp table 6 distinctive themes at universities in victoria it is also interesting to observe that universities offer many is/it related masters programs nowadays compared to a decade ago. as mentioned in keen (1996 pp.129-132) there were only three universities offering is/it related masters programs by course work. they were master of information systems at monash, master of business (it) at rmit and master of business in computing at victoria university of technology (now renamed as victoria university). 239 australasian journal of information systems volume 14 number 1 nov 2006 is research programs in universities in victoria turning from the teaching profile and diversity of programs to that of research, an interesting picture emerges. table 7 demonstrates the diversity of research streams underway in universities in victoria and recognises two successful formal research centres. those universities who have active phd programs have been successful in varying degrees in attracting and maintaining phd students to support, in part, their research programs. overall, despite the large number of diverse topics being studied, is research output in universities in victoria was seen as being lower than in other departments. it should be noted, however that for the most part is research output appears to have been increasing over the past 3-5 years. efforts are underway to bolster research output. comments indicative of this include one at monash to the effect that “all is academics will be expected to be ‘research active’ by 2008” and at university ballarat, where it is a current requirement that “all academics in the school are expected to undertake research”. the mode of is research in universities in victoria is predominantly interpretive. only university ballarat reported using “multi-method, with an emphasis on quantitative techniques”. university areas of research formal research groups no. of phd students acu national is education e-commerce education 1 deakin requirement engineering it security knowledge management supply chain management software engineering e-commerce supply chain management (scm) and business-tobusiness (b2b) ecommerce 20 la trobe computational intelligence is education equity issues of women in it it adoption and impact 2 monash knowledge management systems development decision support systems information management it management e-business conceptual modelling centre for decision support and business intelligence research 60 rmit strategic is e-learning e-commerce knowledge management e-business 30 240 australasian journal of information systems volume 14 number 1 nov 2006 swinburne is governance process modelling is project management health informatics centre for information technology research 1 melbourne e-commerce technology adoption usability design security knowledge discovery 40 university of ballarat it education data mining and informatics mathematics and statistical analysis distributed simulation centre for informatics and optimisation (ciao) 0 victoria university semantics webs business process modeling negotiation support legal systems in is ethics, privacy and censorship portals for medical and aged care support erp electronic commerce research 16 table 7 – research streams in universities in victoria although research is considered a high priority at almost all universities in victoria, available funding appears to have a negative correlation with the avowed importance of researching. perceptions of “very little funding”, “dwindling funding” and “having trouble attracting arc and other external funding” were evident in the data. where funding has been obtained, it is primarily in the form of competitive grants from university internal sources or associated with established research centres, with some funding forthcoming from industry. a relatively few is departments have been successful in securing external grants from granting agencies such as arc. however, there appears to be a trend towards encouraging external grants rather than relying on internal funding, which in almost all cases was seen to be increasingly difficult to secure. respondents stressed that rewards, when available, are increasingly being awarded for research that results in journal publications rather than conference papers. victoria government support for is programs the victorian government is aware the number of students in is has dropped. in order to encourage more students to take is courses in the universities, in 2005 multimedia victoria initiated a project, the industry & universities collaboration pilot program, which provides funds to universities in victoria to promote and improve is courses. all universities in victoria will benefit from this program 241 australasian journal of information systems volume 14 number 1 nov 2006 the hon. marsha thomson, minister for information, communication and technology in the victorian state government convened a meeting on 26 may 2006 to discuss matters relating to an apparent shortage of ict skills in australia, and notably in victoria. key people who have impacted is in universities in victoria a number of key individuals were recognised for their contributions to is in victorian universities. predominantly, these individuals were long-standing academic leaders. examples included tony adams at rmit, who was named as the champion for the establishment and direction of the is department, along with marianne broadbent, head of the former im school in arts who worked with prof. adams to establish the rmit is research and coursework frameworks. robert johnson was identified as an influential force at melbourne as was gerald murphy at swinburne and angela scollary for her tireless work to establish the is school at victoria. the respondent from university of ballarat recognised prof. sid morris, their head of school as one who “has had a huge amount to do with the success we are having” and acknowledged the important contributions of prof. wayne robinson, university of ballarat’s deputy vice-chancellor (academic and research), together with that of prof. alex rubinov, director of ciao. a long list of influential individuals was forthcoming from respondents at monash. these included gerry maynard for his leadership in establishing the is department, jack greig, current head of is and the person responsible for the introduction of systems analysis units, phil steele, david arnott, graeme shanks, frada burstein, julie fisher, ron weber and ian martin, an academic involved in industry-based learning at the clayton campus less frequently mentioned, but no less important, are those non-academics who were seen to have influenced course design and skill requirements, such as the representatives at bendigo bank who have been supportive and influential at la trobe. summary despite declining enrolments and dwindling research support, from a teaching perspective is programs in the universities in victoria appear to be well placed to provide top calibre is graduates at the undergraduate and postgraduate levels to meet the projected increased demand from industry and is programs projected over the next 5 years. programs at all tertiary education levels are varied and carefully crafted, influenced by national rather than local impacts. it is encouraging to see that the research output and quality of the is community in victoria have improved greatly over the last few years. a large number of is academics have gained phds, published in leading is journals, joined editorial boards of prestigious journals and some have obtained arc linkage and discovery grants. while research topics are diverse within the state, methods are not. is academics in victoria appear to align themselves more closely with interpretivist european research traditions than with the quantitative, positivist approach more commonly found in north america. despite the increased importance that research appears to have in measuring performance, funding is dwindling and success in securing external competitive grants such as those offered by the australian research council appears to continue to elude the majority of is researchers in victoria. 242 australasian journal of information systems volume 14 number 1 nov 2006 in relating the victoria data to the underlying framework that guides the is in australia study as proposed by whitley (1984), it would appear that the victoria data do not fully support is as a distinct scientific discipline. each of whitley’s criteria will be addressed separately in light of the victoria data to demonstrate the conclusion reached. the first of whitley’s three conditions that must be met in order for an area of study to be considered a “distinct scientific field” is a social process that results in scientific reputations becoming socially prestigious and control critical rewards. mingers and stowell (1997) suggest this can be evidenced through publications and success in attracting research funding. clearly, victoria researchers view themselves as being somewhat less “respected” than their counterparts in other disciplines with only one respondent feeling that is researchers were of higher status than those in other departments. on the other hand, documentary evidence shows that a number of senior is academics in victoria have attained status as full professors and are recognised as being as qualified as their peers in other more mature disciplines. the deficiency in meeting this criterion is perhaps more telling in regard to “attracting research funding” where the data clearly demonstrate that external funding support for is research continues to be elusive and is researchers appear to be losing ground as they struggle with dwindling internal funding. the second of whitley’s criteria is the need to establish standards of research competence and skills. here, the victoria data add to the long-standing discussion on whether is is a discipline (dickson, benbasat and king 1982; benbasat and weber 1996; boudreau, et al. 2001) and the current perception that is continues to align itself more closely with a “fragmented adhocracy” as suggested by checkland and howell (1998) and kanungo (2004) than a distinct discipline. for example, while there were pockets of successful grant applications and a limited number of research centres throughout the state, the data revealed an overall lack of success in attracting research funding which could be construed as a negative reflection on research competence and skills that appear to be limited to interpretivism and lacking in the application of the more diverse, blended approach usually evident in more mature disciplines. the third and final criterion is one that requires the existence of a unique symbol system to allow exclusion of outsiders and unambiguous communication between initiates within the field. only 50% of victoria is programs are recognized as a separate entity and the research topics under scrutiny would clearly demonstrate a heavy reliance on reference disciplines with little or no discussion of the use of an is theory. this would suggest that this criterion has not been met. in conclusion, is programs in victoria appear to be evolving to meet the demands of industry from a teaching perspective, but somewhat lacking in the area of research output vis-à-vis their counterparts in other departments. however, while some universities have yet to establish is as a separate entity, research output is increasing along with efforts to successfully win external competitive grants. it would appear that the main challenges being faced include the significant drop in students studying is and the expected drop in full-fee overseas students, coupled with the commonwealth government’s new research quality framework (rqf), which will focus attention on the quality of is research. opportunities include building on recent successes in is research in victoria that should lead to a stronger is research base in the future. 243 australasian journal of information systems volume 14 number 1 nov 2006 acknowledgements the authors would like to express their gratitude to angela scollary for her valuable contribution to the data collection phase of this research project and to graeme shanks for his insightful comments on an earlier draft of this paper. carol pollard was affiliated with the school of information systems, university of tasmania, australia when this research began. references acm, ais and aitp (1997) is'97 model curriculum and guidelines for undergraduate degree programs in information systems. http://webfoot.csom.umn.edu/faculty/gdavis/curcomre.pdf accessed 30 may 2006. ang, a.y. (1992) “australian information systems curricula: a comparison between the views of universities and tafe colleges”, proceedings of third australian conference on information systems, wollongong, australia, 5-8 october, 747-758. ang, a.y. & lo, b.w.n. (1991) “changing emphasis in information systems curricula: an australian industrial perception”, proceedings acc'91 mosaic, adelaide, 6-10 october, 11-28. avison, d.e. (1993) “research in information systems development and the discipline of information systems”, proceedings of 4th australasian conference on information systems, queensland, australia, 28-30 september, 1-27. benbasat, i., goldstein, d.k., & mead, m. (1987) “the case research strategy in studies of information systems”, mis quarterly, vol 3, pp 368-386. benbasat, i. & weber, r. (1996) “research commentary: rethinking ‘diversity’ in information systems research”, information systems research, vol 7 no 4, pp 389-399. boudreau, m., gefen, d., & straub, d. (2001) “validation in information systems research”, mis quarterly, vol 25 no 1, pp 1-16. buckingham, r.a., hirschheim, r.a., land f.f. & tully,c.j. (1987) “information systems curriculum: a basis for course design”, in buckingham et. al. (ed.) information systems education: recommendations and implementation, university press, cambridge, great britain, pp 114-133. checkland, p., & holwell, s. (1998) information, systems and information systems – making sense of the field. university of illinois press, urbana, illinois. clarke, r. (1999) “comments on information systems curriculum”. http://www.anu.edu.au/people/roger.clarke/sos/iscurric.html accessed 30 may 2006. commonwealth of australia (2005) higher education report 2004-05, http://www.dest.gov.au/nr/rdonlyres/136cdc4e-9770-464d-bdcf72e56b031c6a/8092/highered_report_fullversion11.pdf accessed 30 may 2006. denzin, n.k. & lincoln, y.s. (1998) the landscape of qualitative research, sage publishing, thousand oaks, ca. 244 australasian journal of information systems volume 14 number 1 nov 2006 desouza, k.c., el sawy, o.a., galliers, r.d., loebbecke, c. & watson, r. (2006) “beyond rigor and relevance towards responsibility and reverberation: information systems research that really matters”, communications of the ais, vol 17, pp 341-353. dickson, g.w., benbasat, i. & king, w.r. (1982) “the mis area: problems, challenges and opportunities”, database, vol 14 no 1, pp 7-12. fielden, k. (1990) “facts, skills and creativity: an innovative approach to learning in information systems”, proceedings of first annual conference on information systems, monash university, melbourne, australia, 6 february, 2-14. government of victoria (2006) “melbourne home to new headquarters for leading israeli it company 27 april”. http://invest.vic.gov.au/news/news/ness+technologies.htm accessed 30 may 2006. kanungo, s. (2004) “on the emancipatory role of rural information systems”, information and people, vol 17 no 4, pp 407-422. keen, c. (1996) “a survey of course work masters in information systems in australia”. in d. arnott, k. dampney and a. scollary (eds.), the australian debate on information systems curriculum, proceedings of the australian in formation systems curriculum working conference, 24-25, sep. monash university, australia, 119-132. lee, a. (2001) “editor’s comments”, mis quarterly vol 25 no 1, pp iii-vii. lo, b.w.n. (1989) “a survey of information systems educational programmes in australian tertiary institutions, working paper series no. 1”, department of information systems, the university of wollongong. miles, m.b. and huberman, a.m. (1994) qualitative data analysis, sage publications, london. mingers, j. & stowell, f. (eds.) (1997) information systems: an emerging discipline? maidenhead, berkshire, uk: mcgraw-hill. mintzberg, h. (1979) “an emerging strategy of “direct” research”, administration science quarterly, vol 24, pp 580-589. newcomersnetwork.com (2006) “business in melbourne, victoria, australia – facts, figures, information, statistics, industries, manufacturing, biotechnology, ict, innovation and more” http://www.newcomersnetwork.com/mel/oursay/expertadvice/bimv.php accessed 30 may 2006. ridley, g. (2006) “characterising information systems in australia: a theoretical framework”, australasian journal of information systems, vol 13 no 3, pp. 38-60. shanks, g., rouse, a. and arnott, d. (1993) ‘a review of approaches to research and scholarship in information systems’, working paper 3/93, school of information management and systems, monash university, melbourne, australia. tatnall, a. (1993) “a curriculum history of business computing in victorian tertiary institutions from 1960-1985”, masters thesis, deakin university, geelong. tatnall, a. (1999) “innovation and change in the information systems curriculum of an australian university: a socio-technical perspectives”, phd thesis, central queensland university. walsham, g. (1993) interpreting information systems in organization, wiley, chichester. 245 australasian journal of information systems volume 14 number 1 nov 2006 walsham, g. (1995) “interpretive case studies in is research: nature and method”, european journal of information systems, vol 4, pp 74-81. whitley, r. (1984) the intellectual and social organization of the sciences, clarendon press, oxford, uk. wikipedia, the free encyclopedia (2006) “victoria (australia)”. http://en.wikipedia.org/wiki/victoria_(australia) accessed 30 may 2006. yin r. k. (1994) case study research, design and methods, sage publications, beverly hills, california, us. yin, r. k. (2003) case study research: design and methods, 3rd edition. sage publishing, thousand oaks, ca. the authors carol pollard is associate professor of information systems. her research interests include decision support systems, impact of emerging technologies and technology transfer. her current research focuses on the healthcare sector and small to medium-sized firms. carol has published her research in mis quarterly, journal of management information systems and information and management and has presented at numerous national and international conferences. carol is currently research director, center for applied research on emerging technologies (caret) at appalachian state university; vice president, international of gitma; chair, acis executive committee and an advisory board member of teradata university network. elsie chan currently plays a dual role in australian catholic university. she is a lecturer in information systems as well as the course coordinator of business programs in school of business and informatics, melbourne campus. she obtained her msc degree in artificial intelligence at university of essex, united kingdom in the late 1980's. since then she has been lecturing in various universities in hong kong, china and macau before she migrated to australia. she obtained her phd at deakin university, melbourne. her research areas are electronic commerce / electronic business and information systems education, marketing and web development 246 australasian journal of information systems volume 14 number 1 nov 2006 appendix 1 undergraduate and masters by coursework programs in information systems and information technology offered by universities in victoria australian catholic university (melbourne campus) bachelor of information systems bachelor of information systems (hons) bachelor of business / bachelor of information systems deakin university bachelor of information technology (it security) bachelor of information systems/bachelor of information technology bachelor of information systems bachelor of information systems (hons) bachelor of engineering/bachelor of information technology bachelor of information technology (multimedia technology) bachelor of information technology (computer science and software development) bachelor of information technology (games design and development) bachelor of information technology (web and mobile technologies) master of arts (professional communication)/master of information technology master of information technology master of information technology (professional) master of information technology/master of commerce master of information technology/master of information systems master of accounting information systems master of commerce/master of information systems la trobe university bachelor of information technology (computer networks) bachelor of information technology (information systems) bachelor of information technology (software development) bachelor of information systems/bachelor of business master of information technology (computer networks) master of information technology (intelligent systems & internet computing) monash bachelor of business information systems bachelor of information technology and systems bachelor of arts / bachelor of information management and systems bachelor of commerce / bachelor of business information systems master of applied information technology master of business administration/master of information management and systems 247 australasian journal of information systems volume 14 number 1 nov 2006 master of information management and systems master of information management and systems professional master of information systems master of information technology master of information technology (minor thesis) royal melbourne institute of technology bachelor of business (business information systems) bachelor of applied science (honours) (computing and internet technology) bachelor of applied science (information technology) bachelor of applied science (honours) (computing and internet technology) master of business (business information technology) master of applied science (information security) master of applied science (information systems) master of applied science (information technology) master of engineering (information technology) master of technology (information technology) master of technology (internet and web computing) swinburne university bachelor of information technology (honours) bachelor of science (information technology) bachelor of information systems (honours) bachelor of information technology bachelor of business information systems / bachelor of business bachelor of business information systems master of information systems management / master of accounting master of information systems management master of information systems management /master of business administration master of technology (information technology) master of information technology the university of melbourne bachelor of information systems bachelor of information systems (honours) master of business and information technology master of information systems master of information technology master of information technology in education university of ballarat bachelor of information systems bachelor of information technology 248 australasian journal of information systems volume 14 number 1 nov 2006 bachelor of information technology (professional practice) master of business information systems master of information systems master of information technology master of information technology studies master of information technology studies and master of business administration master of information and communication technology victoria university bachelor of business (accounting/information systems) bachelor of business (information systems) bachelor of business (tourism management/information systems) bachelor of arts/bachelor of business (information systems) bachelor of business (information systems) (honours) bachelor of science in information technology master of business in information systems master of business in information systems and enterprise resource planning systems 249 australasian journal of information systems volume 14 number 1 nov 2006 250 demographics of victoria education in victoria sophisticated ict industry in victoria information systems programs in universities in victoria relative size and administrative placement of the is presence the extent to which is has a separate identity within the university the extent to which is is impacted by local contingencies distinctive features of the is curriculum is research programs in universities in victoria victoria government support for is programs key people who have impacted is in universities in victoria acknowledgements the authors appendix 1 aj1s special edition knowledge management december 2001 a holistic approach to knowledge management and social learning: lessons learnt from military headquarters leoni wamc', irena all ' , celina pascoe2 and katerina agostino' 'defence systems analysis division defence science and technology organisation department of defence canberra act 2600 lconi.warncffidslo.dcfcncc.gov au h ena.alid3ldslo.defence.gnv.au " division of communication and education university of canberra act 2601 celina.pascoetscomedu.canberra.edu.au 'land operations division defence science and technology organisation department of defence edinburgh sa 5 1 i i katerina.aeostino@dsio.defence.gov au abstract this paper reports on research conducted by the enterprise social learning architecture (esla) team of the defence science and technology organisation. the esla team is investigating collaborative social learning within the australian defence organisation (ado). social learning is tightly coupled to knowledge management. three studies in three different settings have been conducted to date. the studies have provided multi-layered findings about social learning, and validated the use of ethnography for this purpose. preliminary findings arc discussed in this paper in terms of identified enablers and motivators for effective social learning and knowledge management. although the paper deals with the defence environment, the findings can be generalised to other organisational settings, as the study deals with understanding the issues inherent in building sustainable and adaptive learning organisations. introduction this paper reports the preliminary findings of research conducted by the enterprise social learning architecture (esla) team of the defence science and technology organisation. the esla task is currently in the third year of a four-year research study investigating social learning within the australian defence organisation (ado). a focus of the study has been to determine the factors that facilitate social learning and knowledge management within military command and control settings and, in so doing, to ascertain the elements of an environment that foster a seamless union between working and learning. to date, studies have been conducted in three different settings: a single service tactical headquarters, a joint strategic headquarters (all three services and civilians), and a single service strategic headquarters (where the service is different to the service in the first study). social learning is defined as learning occurring within a group, an organisation, or any cultural cluster and includes: the procedures by which knowledge and practice are transmitted across posting cycles, across different work situations and across time; and the procedures that facilitate generative learning learning that enhances the enterprise's ability to adjust to dynamic and unexpected situations and to react creatively to them. the immediate aim of this research is to understand the issues inherent in building learning, adaptive and sustainable organisations. in this way, social learning is tightly coupled to knowledge management. the esla research team interprets knowledge management to be about the acquisition, construction, development, transfer, and sharing of the knowledge assets within an organisation in order to achieve the organisation's objectives. knowledge management strategies develop organisational knowledge assets (people), support them by information technology and other tools, and enhance the processes by which individuals, groups, and the entire organisation learns. although there are many tools and technologies, which may become an integral part of an organisation's knowledge management strategy, knowledge management is primarily concerned with cultural, organisational, procedural,and human factors (ali, 2000; warne, 2000). the esla research looks specifically at the cultural, organizational, procedural and human factors as applied to social learning.findings have been used to derive a preliminary architecture that organisations might use to facilitate effective social learning for their staff. results to date have identified a number of processes and strategies that enable and facilitate knowledge management and a selection of these factors is discussed in this paper. the long-term 127 ajis special edition knowledge management december 2001 objective of this research, however, is to develop architectures that will support the development of information systems that guide and enhance organisational learning. methodology qualitative research techniques, in particular ethnographic approaches, have been used predominantly for this research because they are appropriate where a key aspect of the research is to analyse, or at least take into consideration, various aspects of the context and the social process (harvey & myers, 1995). ethnographers 'immerse' themselves into a situation for sufficient time to gradually see and understand the key factors that influence the setting being studied. it is most useful when researchers are attempting to understand complex functioning systems from a holistic perspective (jordan, 1993) as it deals with real world practices in real world situations. ethnography is often used, for example, to provide information systems researchers with rich insights into the human, social and organizational aspects of information system development and implementation (harvey & myers, 1995). given the exploratory nature of the research, as well as the importance of context in understanding the social process of learning, ethnographic techniques were considered to be the most suitable methodological tools for the study (ali, 2000; warne, 2000). the primary form of ethnography used was 'field work', which entailed observing the work taking place in different settings, and using directed questioning to clarify issues. currently the team comprises four researchers, with a wide range of backgrounds and specialisations, thereby enriching the collection and analysis of the data, and providing reliability for the findings. regular team meetings ensure that findings are corroborated, consolidated and that emerging social learning issues are identified. to ensure validity, the collected data has been triangulated by source (different times and places), and, as the research progressed, by method (observations, interviews, surveys). the methodologies used have evolved as the research has progressed from one study setting to another. while the methodology for the first setting was largely ethnographic, in subsequent settings, ethnographic observations were supplemented with quantitative surveys, and semi-structured interviews with a stratified sample of personnel in the area. all interviews were recorded and transcribed. the transcripts were then coded according to the thesaurus of terms specially developed by the esla team for coding of field observations and transcripts. the individual team members embedded these code terms, or textual labels, within the text of field notes, collected documents and interview transcripts. these embedded terms help the researcher to develop and map an analytical structure onto the data. coding also allows the raw field data to be gathered together into more meaningful chunks so that they can be retrieved and analysed together (miles & huberman, 1994). the overall aim is to discover the underlying patterns and regularities in the settings being studied. findings to date the esla team has identified a number of environmental or cultural factors, processes and strategies that, when positively applied, facilitate social learning and knowledge management within the ado. in this context, the environmental and cultural factors can be termed "motivators" and the processes and strategies termed "enablers". the motivators and enablers work together to generate an organisational environment that actively sustains and cultivates effective social learning. this environment both nurtures, and is nurtured by, a number of overriding organisational values that foster effective social learning. there is a great deal of overlap and interlock among these factors, such that some enablers also motivate, some motivators enable and the organisational values both enable and motivate. the research team continues to seek better ways of representing these relationships, as evidenced in the paper on "making the invisible visible" (linger & warne, 2001), also in this issue. the research team has concluded that all of these broadly ranging enablers and motivators should ideally be operating together, in order to optimize social learning and knowledge management. collectively, these factors give an organisation more capability and power than the sum of each of their component parts. the research team also found that some of the same motivating and enabling factors could also act as inhibitors or challengers of social learning when they are not thoughtfully applied. in this way, while positive and appropriate application of motivators, enablers and values work together to form a holistic and effective social learning environment, the absence or inappropriate application of some of these factors can have a destabilizing and destructive impact on the effectiveness of organisational learning and knowledge assets. this is illustrated in the diagram below. 128 aj1s special edition knowledge management december 2001 inhibitors overarching values challengers figure 1 factors in social learning enablers and motivators the enablers and motivators discussed in this paper are representative of those identified from the data gathered so far and are by no means exhaustive. for ease of discussion, the identified enablers and motivators have been assigned to seven categories: common identity, problem solving, team building, access to information, development of individual expertise, communication, and, finally, induction and enculturation. based on the esla research, these categories, or elements, are believed to be essential constructs for effective social learning (ali, 2000; wame, 2000). further, the order in which they appear does not imply level of importance or value to social learning; however, each enabler was observed to have an impact on effectiveness of learning processes occurring in the settings under study (figure 2). 129 ajis special edition knowledge management december 2001 1 common identity 1 goal alignment cultural identity language morale workplace design problem solving networking perceptions of the organisation systemic understanding reflection enablers & motivators ii team building leadership team-based morale . performance management valuing skills peer review workplace design i access to information record keeping networking protocols at meetings info exchange at meetings workplace design it infrastructure 1 development of individuals 1 career trajetory professional training mentoring i | communic'n inducti encultu communic'n climate info flows: formal & informal inquiry and reflection workplace design cor '( \ 1 figure 2 social learning constructs it is important to point out, that several of the motivators and enablers appear in more than one category as they play a role in supporting a number of social learning constructs (for example, 'workplace design' and 'inquiry and reflection'), and all of the identified factors appear to be inextricably intertwined (ali, 2000; wame, 2000). common identity the esla findings identified a common identity as one of the important outcomes of having the following enablers and motivators in place. a common identity implies a similar way of describing and making sense of the world, of determining what is significant and important, and of how to use resources in the environment. whilst a common identity and its sub categories are discussed as independent features in this paper, in reality they are not mutually exclusive since they significantly impact on one another. goal alignment motivates and enables goal alignment is an important process within the work culture, particularly in terms of social learning as it impacts on work processes and on group cohesiveness. at the tactical environment studied, the team observed a strong and consistent degree of goal alignment within the community. in contrast, at the joint strategic headquarters facility, the research team found nothing uniform about the ways in which goal alignment takes place and that cohesiveness and work relationships varied in accordance to social positioning, hierarchy, and rank. cohesive work arrangements were very much in evidence at the small team level, but less so as the teams expanded up the hierarchy of the organization. the esla team found that trust rather than goal alignment becomes more important as working environments become more risky and uncertain. cultural identity motivates the term cultural identity refers to a staff member's sense of self in relation to the specific 'tribe' and 'tradition' to which they belong and how this distinctiveness applies in their workplace. cultural identity is another important motivator for social learning. it impacts on the extent to which members of staff feel that they are part of the system or alienated from it. for the australian military, cultural identity may mean army, air force, or navy cultures; while for civilians, their cultural identity may be as accountants, computing professionals, or public servants. at times, the esla team observed a clash of values occurring as cultural shifts took place. this threatened the extent to which 130 ajis special edition knowledge management december 2001 staff felt that they were a part of the system and at times resulted in higher levels of alienation, and thereby reduced cultural identity. a common language motivates and enables language is another important factor fundamental to the overall social learning processes. by reflecting the social and political relationship between various members, language can impact on common identity. at the joint strategic headquarters re-organization workshop for example, it was stated that the defence executive had chosen the term 'staff rather than 'division' to refer to sections within this workplace because the latter is seen as denoting separation and division between sections within the branch. language is also important in terms of creating a shared understanding among workers and their relationship to the wider organization. the importance of language was highlighted at the joint strategic headquarters induction, when one presenter said: "words are bullets. never, never use imprecise language (when appraising performance). it can mean a career hit for somebody.... there are subtle differences in language use between the services. words that may signify praise in the navy or air force may be poison in army." good morale motivates morale has been a significant focus in the overall study because the research team found evidence of low morale being coupled with higher levels of alienation towards senior management. such alienation has obvious implications for a common identity and thus for social learning. perception of low morale has frequently been coupled with comments about not understanding the motivation or agenda of more senior staff. the team also found this lack of understanding not only affects morale, but also has an impact on trust, organizational cohesiveness, goal alignment and common identity, and consequently, on opportunities and motivation for social learning and the development and sharing of knowledge. workplace design enables a particular problem within the joint strategic environment was that its staff was scattered amongst several sites within the city, a consequence of the reorganization that created it. the dislocation had impacted on many of the social learning constructs. there was strong evidence of reduced morale, problems in sharing information because some sites were unable to access file servers and classified networks, as well as the impact that the reorganization had on individuals' perceived power bases. many joint strategic personnel interviewed by the esla team stated that they enjoyed the spontaneity of face-to-face discussion on work related issues as they crop up, and so this became problematic for them when teams were not colocated. the converse also applied. staff located in the smaller outposts, who often tended to be of junior ranks, often felt little identification with their organization. several 'dis-located' staff members expressed significant confusion about their cultural and common identity, in some cases identifying more with the workplace with which they were based than their division within the joint headquarters workplace. this has serious implications for the effectiveness of social learning in these areas, and seriously inhibits knowledge sharing and knowledge transfer. problem solving for knowledge workers in any organization, problem solving is a core activity. importantly the process of problem solving fosters knowledge generation and thus social learning. for instance, routine tasks often need to be done slightly differently in different circumstances and in doing so involve an element of problem solving that requires generative learning (lave & wenger 1991). both the tactical and strategic settings studied provided numerous examples of this. the gap between formal or routine procedures and the ways in which people actually do their work was highlighted many times. networking enables problem solving an individual's network is one of their most important resources, as personal and social networks are an important means of acquiring, propagating and sharing knowledge. the individuals who comprise the network can make available their own knowledge, expertise and experience. in this way, the knowledge resources available to any one per131 ajis special edition knowledge management december 2001 son, in their work and when problem solving, can be greatly increased. staff at all settings studied relied upon their networks for problem solving. personal networks can also play a pivotal role in the propagation of knowledge. a few members of staff in the settings studied were seen to be systematic in passing on relevant knowledge to colleagues, and many others told the research team that, time permitting, they would pass information onto colleagues. as davenport and prusak(1998) claim, when those who are in a position of 'know-how' share their expertise they contribute to problem solving. hence, personal networks were seen to function as channels supporting both 'information pull' and 'information push'. a good deal of effort can go into generating, maintaining and obtaining value from these networks. the research team was told that often, the primary benefit in attending a course or a conference was the new contacts that were made. accurate perceptions of the organization enable individual and shared perceptions of the organization, and how they operate, provide an essential backdrop to problem solving within an organizational context. these perceptions may consist of deeply ingrained assumptions, generalisations, or even pictures or images that influence how people within an organization understand their organizational world and how they should act within it (senge 1992). the importance of these perceptions cannot be stressed enough, because they directly influence the construction of individuals' knowledge and understandings that they draw upon in their day-to-day-activities. one general example is appreciating the ways in which an organisation's formal rules and processes can be bent to achieve a desired outcome. this class of knowledge can empower people to solve problems by expanding the range of solutions that may be available, and by giving them confidence to improvise or innovate. conversely, a lack of knowledge or incorrect perceptions will constrain the types of solutions that can be found and inhibit social learning. systemic understanding enables similarly, effective problem solving often requires a systemic understanding of organizational and interorganizational issues. achieving a systemic understanding requires a holistic view of an organization and its interrelationships, an understanding of the fabric of relationships and the likely effect of interrelated actions (senge 1992). such systemic understandings are becoming more important for organizations as they position themselves to derive benefit from a constantly changing environment. this systemic understanding of the workplace and its context was effectively applied to enhance work practices, in tandem with social learning, by a number of staff members observed in the research, but not by most. the military posting cycle and high staff turnover ensure a continuing need to make this class of information both readily available and to ensure that it is brought to the attention of all staff. hence the need remains to develop, encourage and nurture systemic understandings of the relevant parts of the organization. the esla team was told on numerous occasions that an understanding of where one's job or a project fits into the big picture is an important element in job satisfaction and hence morale. inquiry and reflection enables and motivates inquiry and reflection together are a powerful means of enhancing social learning and knowledge creation. inquiries, or questions, can be triggered by problems or needs that require solutions or explanation. reflection allows time for examination, contemplation and, often, resolution of the inquiries. to use a common metaphor, it is perhaps the best means for distinguishing between the forests and the trees of everyday working life. inquiry and reflection involving more than one staff member is an opportunity for both creating and sharing knowledge. some of the processes at the tactical setting studied, such as the mission debrief held after each mission, were seen to include elements of inquiry and reflection. in contrast and with very few exceptions, most of the staff interviewed at the strategic level headquarters regretted that their high workloads meant they had insufficient time for inquiry and reflection at the workplace, either individually or as a group. one pertinent comments was: "we spend so much time in the detail on the process that we're in, that we rarely give ourselves the opportunity to think strategically about what we're doing and to really determine the priorities of where our effort is worthwhile and where it isn't. i think you need to do that both individually and at a group level again with the people that you work with." 132 ajis special edition knowledge management december 2001 in order to effectively solve problems and innovate, time for inquiry and reflection must be factored into the workplace. staff are generally more motivated to do so if they know that inquiry and reflection is recognised as a valid and valuable use of their time. team building very few people work by themselves and achieve results by themselves. so the people who interact together and yet have different tasks and responsibilities need to understand what each of them are trying to do, why they are doing it, how they are doing it, and what results to expect. in this relationship of interdependencies, communication and trust play vital roles (drucker, 1999). good leadership enables and motivates in general, the leaders and managers of the areas studied were innovative and they motivated and developed their staff. the research team witnessed many meetings where leaders used inclusive language, were receptive to new ideas and openly shared their thinking processes and imperatives. they empowered others and the team's mutual respect was evident regardless of rank. "i've tried to engage people, involve them in the decision making process, keep people informed because i think people who are informed will be better participants and contribute better to whatever it is that we are trying to achieve, and where appropriate collegiate. sometimes however, a more directive style is necessary." the esla team spoke to many staff members who expressed their appreciation for the accessibility of their leaders, their non-judgemental approach and their courage in admitting their shortcomings. such qualities and atmosphere build up trust, and encourage collaboration and teamwork. the military leadership also exhibited two other motivating dimensions: a willingness to demonstrate that staff are highly valued and a willingness to acknowledge expertise and knowledge regardless of rank. this was particularly evident in the tactical setting. strong team-based morale motivates team spirit' and 'team cohesiveness' are both important values within the work culture and work ethic in the settings studied. in some of the settings studied, the attitude of 'them and us', as well as a feeling of being undervalued, was clearly prevalent. these individuals did not identify themselves as team members of the unit they belonged to, and some stated they were not encouraged to operate in a coordinated way to support organizational goals. this was because they did not see the significance of their particular tasks to the overall goals of the organization and this feeling was exacerbated by the geographic distribution of team members. this was a serious de-motivator, and these staff members were not keen to learn or share their knowledge with other team members. however, some examples of effective teamwork and team spirit were evident in the settings studied. the researchers observed instances where teamwork was well integrated into daily work and where people worked collaboratively. such teams were goal oriented and bound by achieving good results. there were not only teams in structure, but in spirit. they were usually formed within individual sections and were led and energised by a leader who saw his or her role as fostering team cohesiveness. members of these teams were co-located and made information (such as calendars, contacts, and computer work folders) available to one another. this not only contributed to knowledge sharing among team members and aided communication, but it also emphasised trust within the team. in these areas, morale was perceived to be high. people worked together happily and looked forward to socialising together. regular social functions were organised for the workplace teams. constructive performance management enables assessment, reporting and performance management form a significant part of the overall management of military personnel throughout their careers; however, if not handled constructively it may have adverse impacts on team spirit and thus on social learning. the outcome of a performance report often determines the prospects of one's career progression. the esla team was told that a poor performance rating, at a critical point in one's career, would severely reduce the prospects for promotion. this strong emphasis on individual performance management may be divisive as it may influence a proportion of individuals to focus on achieving their individual goals at the expense of assisting their team to achieve its goals. team-based performance management systems are being discussed, but have yet to gain wide acceptance. 133 ajis special edition knowledge management december 2001 for the military, the performance cycle is annual. it was reported to esla team members that there was often a lack of clear communication about performance expectations. some of the people interviewed were unclear about what was expected of them and what was stated in their duty statements. an annual performance appraisal appears to be too long to wait for recognition of good work and too late to correct a performance problem. clear communication about performance expectations can contribute not only to achieving organizational goals but also to appropriately directed social learning and knowledge development (morgan, 1989; wood, 1989). valuing skills motivates and enables at the tactical setting studied, the skills of individuals were seen to be highly valued. current expertise and competence were often seen to take precedence over rank. this type of interaction is acknowledged as an important cultural requirement for organizations that wish to further their generative learning (bokeno, 2000; isaacs, 1999). the researchers were told that the attitude had changed over previous years. before that change, senior officers believed that they must maintain the pretence that they were superior in all spheres of performance, to maintain their authority. however, informants explained that the results of that attitude were counterproductive. a senior officer stated that: "there has been a huge change with the culture regarding senior and junior officers. previously, junior officers would not go out of their way to help them the attitude was, if you knew it all, then do it yourself. it couldn't be more different now. they will offer help when they see it is needed. you simply need to say what you require. they respect the fact that we have other jobs to do and are not as current as we once were." one way of valuing skills and increasing morale is to publicly acknowledge outstanding work. key informants stated that public recognition of good work was scarce in the strategic settings studied. many personnel told the research team that a written or verbal word of praise, a pat on the back often means more, for example, than a pay raise "praise is better than money". peer review enables for the military, peer review is an important way of sharing expertise and problem solving. when done constructively, it also has a positive effect on team building because it demonstrates that colleagues are willing to contribute to achieving an individual's tasks and goals, and peer review helps to build and sustain relationships and to generate trust. peer review was seen to form an essential part of team building at the tactical headquarters, and to a lesser extent, at the strategic settings. at the tactical setting, the mission debrief is a major component of peer review. each mission is analysed to see how well its objectives were met, and how each individual's actions contributed towards the success or failure of that component of the mission. furthermore, this is done without singling out individual's failures or assigning blame. the importance of peer review was stressed by a senior officer who said "interaction with peers is an important way of getting feedback, constructive criticism and validation". certainly, constructive peer review enables knowledge sharing and social learning. effective workplace design enables workplace design was seen to have many negative impacts on social learning. staff located at small isolated outposts often felt excluded by their colleagues. the esla team was told about a number of instances when these workers were overlooked or not informed about meetings or other events. several staff members expressed confusion as to who was in their line management, or even their place in the ado. those teams where staff was co-located demonstrated the highest team-based morale and team spirit. however, whilst collocation can be helpful, it is insufficient on its own to guarantee successful team building. access to information the easy availability of corporate information has a direct input into knowledge acquisition and generation, and thereby, social learning. information, therefore, is an important organizational resource which, if properly managed, can lead to improved decision making and increased productivity. 134 ajis special edition knowledge management december 2001 good records keeping enables the research team observed that general familiarity with corporate records keeping procedures in the settings studied was quite poor and adherence to formal process was almost non-existent. records management and access to information contained in paper records posed a problem. some people had developed their own personal records keeping systems but there was little uniformity in these, and no adherence to file naming conventions and standards. consequently, there seemed to be little faith in the integrity of organizational records. those areas where there was good local practice in records management were seen to be very much the exception. as one informant stated: "i believe that physical files in the ... are no longer managed well because their management has been farmed out to outside bodies. with the file clerks there was consistency of procedures but the file clerks are no longer part of the procedure." furthermore, the preference for accessing and transferring information electronically seems to be growing, and the use of electronic tools for communication and decision making is prevalent at all levels. the issue of electronic records, particularly e-mail messages containing evidence of business transactions, posed problems not only in the setting studied but also in the ado at large. research reported in the professional literature on records management (enneking, 1998; henricks, 1999) suggests that the capture of email messages into a records management system offers the best solution to this problem. however, in the absence of organization-wide protocols, technology is only as good as the people using it: "i think we have problems with passing on information in the organization as a whole... you know we go upstairs and ask people where they save their work to and you ask five people you'd find different answers. we have an information management problem right from the outset. where do you save your e-mails? well i save them somewhere. where do you save yours? i put mine in a public folder. where do you save yours? i put them into private folders. it's corporate information...we have an information management problem and therefore passing on information becomes quite difficult because everybody does it in a different way." personal networking enables personal networks from previous postings, as well as newly acquired contacts, play a pivotal role in knowledge propagation. new knowledge often begins with the individual and making personal knowledge available to others is the central activity of knowledge creating organizations. through conversations people discover what they know, what others know and in the process of sharing, new knowledge is created. technologies such as e-mails, faxes, and telephones are invaluable aids in the process of knowledge sharing, but they are only supporting tools. knowledge sharing depends on the quality of conversations, formal or informal, that people have. webber (1993) describes this process: "conversations not rank, title, or the trappings of power determine who is literally and figuratively 'in the loop' and who is not." protocols at meetings enable meetings are another means of accessing information and those that were observed in the different settings studied varied significantly in format and the protocols in place. the meetings in the tactical environment were seen to be more conducive to social learning than those at the strategic environment. at the tactical headquarters, mission related meetings were observed to be excellent opportunities for learning. strict protocols were observed at such briefings allowing participants to discuss errors or problems encountered during missions without assigning blame or shame to individuals. as well as providing access to information, this facilitated the sharing of mistakes and the sharing of responsibility for solutions. even briefings where the main aim was information transfer from senior to junior staff ensured that staff were invited to speak to topics, discuss tactics or to identity potential objectives. the esla team had been told that, prior to the introduction of these protocols, the debriefs commonly became very heated and it was not uncommon for fighting to take place. a similar meeting protocol, although not as consistently adhered to, was observed in the single service strategic headquarters. however, there were few equivalent meetings at the joint strategic headquarters, other than some induction meetings and briefings. in an environment which, in many ways seemed the most complex and ambiguous, learning how to do the job was not given quite the same priority. 135 ajis special edition knowledge management december 2001 effective information exchange at meetings enables many meetings at the settings studied were held with the stated aim of propagating and exchanging information. information transfer is generally considered to be an uni-directional process, while information exchange is a two-way or multi-directional process. while meetings observed in the joint strategic environment tended to have a basic structure, their purpose appeared to be more for information transfer than for information exchange or information sharing. conversely, in both the single service settings, meetings usually presented ample opportunities for information exchange, brainstorming and group reflection. however, the esla team observed a vast difference among different sub-units in the way information was disseminated at meetings, so this protocol was very much dependent on local practice, which was generally determined by the manager. some managers did not have regular meetings at all. they conveyed information passed on from their superiors to their subordinates on an ad hoc basis, expecting them, in turn, to disseminate information to their respective subordinates. the research team was told that this 'multi-step' means of disseminating information compromised the content (inevitably filtering of information took place) as well as accuracy of information. one survey respondent stated that: "whilst at a social level members of... interact well, there is very poor group cohesiveness in the work environment. i believe that this is in part due to the failure to conduct regular progress / section meetings at lower levels and the failure to clearly identify who is doing what." whilst it is sometimes necessary to have meetings for information transfer only, knowledge transfer and social learning is more effective when there are opportunities for an open information exchange and dialogue at meetings and where questions can be asked, views expressed and problems raised without fear of blame or ridicule. workplace design enables workplace design is also an enabler for the category of access to information because it impacts on the ways by which individuals can access information. in the first instance, it is important that individuals know that some types of information exist. it is more likely to be used if it is readily accessible rather than if it is hidden or scattered across many offices. this problem is exacerbated by geographical distribution of units within the organization. this leads to a greater reliance on personal networks and on obtaining information from people rather than from documents and centralised sources. in the worst cases, it can lead to a loss of interest in information seeking or information verification. good it infrastructure and protocols enable the esla team observed that information access due to failings in the it infrastructure inhibited access to information within the strategic settings. hardware crashes and system failures were common. it was especially a problem for those individuals who were not co-located in the central headquarters. they lacked access to classified networks and to the main information resources stored on shared hardware drives; similarly, those staff located within the central headquarters had difficulty accessing some shared information resources of outposted staff. another issue that was often brought to the team's attention was the difficulty in finding information on the shared drive. since there was no specific person responsible for maintaining information on the shared drive and for naming folders, it was left to the discretion of the document originator where each document would be stored. the research team observed that in some units, people were more precise in giving explicit names to documents and folders but this practice was not uniform across the whole of the headquarters. development of individual expertise the acquisition and development of individual expertise is an integral part of knowledge development and social learning and there are a number of enablers that have been observed to facilitate this. 136 aj1s special edition knowledge management december 2001 career trajectories enable an individual's career trajectory describes the positions, roles and experience that he or shehas accumulated, up to and including the position currently held. whilst not excluding personal experiences outside of a work or training context, a well-designed career trajectory generally equips an individual with the skills, experience, maturity and personal networks needed to successfully fill a particular position or role. appropriate career trajectories facilitate social learning because they provide a foundation of knowledge upon which an individual can become fully productive more quickly, and as a consequence they are more able to construct new knowledge. good career management and thoughtful recruitment and selection procedures could serve the same purpose in civilian organizations. professional training enables and motivates appropriate and timely professional training is a significant component of the development of individual expertise and, therefore, a fundamental enabler for acquiring and generating new knowledge. generally, the military excel at supporting their officers through in-house, graduate and postgraduate training. training courses are also important to furthering individuals' expertise, as well as for forming the personal networks that subsequently develop. however, in times of budgetary constraints, training money is often the first to go, with damaging consequences for the organization's ability to learn and develop their knowledge. the esla team observed that numerous individuals at the strategic settings studied believed there was a lack of funds available for training. this perceived shortage of funding could have a direct impact on the trust that staff have for management. over many years, researchers (for example: mccauley & kuhnert, 1992 and argyris, 1973) have found that a general trust in management is associated with professional development opportunities at work. the implication is that an employee's sense of trust is fostered when the supervisor provides career growth opportunities because it authenticates the supervisor's commitment to that employee's professional development. this is important to social learning because trust plays a key role in collaboration and teamwork. as it turned out, in one of the strategic settings, there were ample funds available for training, but the availability of these funds was not effectively communicated to those who would have taken advantage of them. mentoring enables and motivates a degree of mentoring may be an important element in the development of individual expertise. mentoring is regarded as an effective method of assisting staff development, especially for junior staff. at the tactical headquarters, for example, a degree of mentoring was seen to be built into the training program. in terms of developing a career trajectory, the knowledge acquired through mentoring may also be important when individuals want to prepare themselves for specific roles in the future. the active management of junior officers' careers in the tactical environment provided another avenue for mentoring. the esla team, however, saw little evidence of systematic attempts at mentoring within the strategic environments although a mentoring program was on the 'wish list' of many staff members: "...it might range from ex-politicians, ex-secretaries, senior members of the strategy committees, or something like that mentors that have previously been through the system, just saying, 'hey, look, this is these are the hidden rules about how you do the thing". communication communication is one of the social learning constructs essential to effective learning and knowledge management within an organization, and to the effective functioning of any workplace. a supportive communication climate enables and motivates supportive communication climates are characterised by an open and free exchange of information, constructive conflict management procedures, and a high degree of worker involvement in solving organizational problems (larsen & folgero 1993). a supportive communication climate includes a culture of sharing knowledge, treating each other with respect, and breaking down cultural barriers arising out of inaccurate stereotyping. research has established the link between supportive organizational communication climates and generative learning as well as with higher levels of organizational commitment (bokeno 2000; ruppel 2000; guzley 1992). . 137 ajis special edition knowledge management december 2001 in both the single service environments studied, a supportive communication climate predominated. in the joint environment, however, the research team more often observed examples of defensive communication behaviour. findings from semi-structured interviews with these staff also indicated lower levels of organizational commitment. in such settings, social learning and knowledge sharing is inhibited. effective formal and informal information flows enable within a tactical environment such as the single service tactical headquarters studied, the communication flows, both formal and informal, are the lifeblood of the organization. it simply could not function without the meetings, briefings, email, telephone calls and conversations that take place. these activities provide information, align goals between individuals and different parts of the organization and help to coordinate their activities. when communication channels and information flows break down, as was observed several times at a high-stress military exercise, the consequences may be severe. the situation is similar in the strategic environments even though many of the time pressures are not as dire. the data gathered suggests that formal information flows were not always effective within the strategic organizations. information flowing from the top levels of the organization to the lower levels was perceived to be excessively filtered and fragmented in the process. but information flows from the lower level to the senior ranks, and lateral information flows, were seen to be particularly ineffective. informal information flows, however, seem effective in all the settings studied. on numerous occasions, it was pointed out that informal meetings, such as, morning teas or lunches provided an invaluable forum for exchange and transfer of information. in fact, these informal gatherings were a preferred means for communication and exchange of views by many staff. apart from socialising, the team was told, a shared lunch provided a forum where people could talk informally about work and decisions were often made on the basis of information acquired at these lunches. people were regretful, however, that under current workloads, a lunch hour is often used to catch up on work and thus there is less and less networking between colleagues. inquiry and reflection motivates and enables an important element of generative learning is for organizational members to be able to engage in dialogue, which is open and is based on inquiry and reflection. a supportive communication climate is a prerequisite for such dialogue and it requires learning how to recognise defensive patterns of interaction in teams that undermine learning (senge 1992). however, an additional and obvious requirement for such dialogue is having the time to engage in it. as stated earlier, on numerous occasions, the esla team encountered comments from the strategic headquarters staff that there is little time to reflect, learn from experiences, whether they be successes or failures, and generally discuss work matters. the comments were often made with an indication of bitterness and workload was attributed as a factor preventing people from setting some time aside for thinking and reflection. many of the processes at the tactical headquarters, from mission debriefs to tactics meetings and training, demonstrated that they recognised the importance of inquiry and reflection at a group level. as well as improving the quality of the reflection, group reflection also contributes to the development of a common identity and a shared understanding amongst the participants. workplace design enables the issue of workplace design and its impact on team and network building, and on accessing information necessary to getting one's job done, arose repeatedly during the study. numerous interviewees were aware that physical location and proximity to each other had the potential to promote the transfer of pertinent knowledge. indeed, the point was even made that in addition to more quickly obtaining answers to questions about particular tasks, an open plan workplace enabled one to tap into pertinent information by overhearing others' conversations. officers at the tactical headquarters settings believe that opportunities for learning have significantly decreased, citing the decrease of open plan areas as a factor: "these days this has all but gone out of the window. the crew rooms are no longer used in the same way. nowadays, they all have desks and work separately. this change has meant that the learning, which used to take place informally in the crew room, happens less frequently." however, as davenport and prusak (1998) point out, co-location in itself does not guarantee the sharing of knowledge; a common training or experience, or at least a common language, is essential. unless individuals are prepared 138 ajis special edition knowledge management december 2001 to ask and answer questions of one another, or to even just chat with each other, the knowledge advantage provided by open plan workplaces will be lost. an example of this was brought to the research team's attention when told that two workers had been co-located for three months before they realised that they were both working on the same project. conversely, many staff members of the strategic headquarters setting considered the open plan arrangement noisy and an inhibitor of effective communication, as the following comment illustrates: "if the environment was such that a group of people who worked together were located together and had some form of privacy to do their work as a group, that would be fine but when you're all lumped together and everybody can hear what everybody is saying and everybody's saying a hundred things at the same time, sometimes it can be an absolute nightmare." induction and enculturation orientation of new employees is often considered to be one of the most overlooked aspects of employee training (ganzel, 1998; tyler, 1998). like appropriate career trajectories and professional currency, effective induction and enculturation programs facilitate social learning by providing a foundation of knowledge upon which the individual can become fully productive more quickly and as a consequence they are more likely to assimilate new knowledge. good induction, however, is more that just an introduction to new job and workmates; it is a way of helping people find their feet and shape their attitudes and expectations (dunford 1992; george & cole, 1992). there are numerous advantages that come from good induction programs. these include morale building, minimisation of misunderstanding (because rules and regulations have been clearly explained), establishment of good working relationships, reduction of anxiety, and reduction of inefficiency. comprehensive, timely, induction programs enable induction, or the perceived lack of it, was a problem in the joint strategic setting studied. although not everybody interviewed was explicitly critical about the lack of job induction, some felt frustrated because often they had to labour to find obvious organizational information required for their work. this "discovery learning", as it was referred to, was regarded as very time consuming and seemed to lead to poor morale, frustration, and it negatively influenced people's perceptions about the organization. the plight of some informants was made clear in the following remarks: "...there was just no guidance, no handover, nothing about where this paper was at and what i should do with it." "we had no ... handover in terms of the status of projects that we were going to assume responsibility for. no handover in relation to file or information management within the section. so we just foraged. we're still foraging." timing is one of the most important elements of employee job induction. if the employees have to wait for weeks to be introduced to the job and the organization, they have been largely unproductive for that period of time. an issue that emerged from the interviews with staff in the strategic headquarters was the relationship between meaningful and timely induction and subsequent job satisfaction. what was also interesting was that those who were not properly inducted or enculturated into the organization saw no need and responsibility to actually prepare any form of handover for anyone who may take over their position in the future. if learning is to occur, a timely and comprehensive induction program would provide a solid ground to begin this process. 139 ajis special edition knowledge management december 2001 'buddy' and mentoring systems enable and motivate although highly desirable, when a new posting cycle begins, it is not always feasible to conduct an induction program that all new starters can attend. this is even harder to do in a civilian organization where new staff can start at any time during the year. in the interim, a 'buddy' or 'mentoring' system could fill in the gap. a 'buddy', would be an assigned, experienced workmate who could be available to answer questions and assist the orientation of new members during the initial few weeks. some interviewees mentioned that a colleague acted informally as a 'buddy' when they first came to the headquarters, and that they found this to be invaluable to settling into a new job and to effective learning. for instance, one person commented: "well, because [name] did the job before, and was pretty much my bible for the first three months so i was relying on [name] for a lot of stuff'. similarly, mentoring is regarded as an effective method of staff enculturation and development. its advantages include helping to align staff with organizational goals, and providing a context for the transfer of knowledge between senior and junior staff (davenport & prusa prusak 1998). early training enables the esla team was repeatedly told that early training is an important part of effective induction and enculturation. it is an opportunity to learn the explicit knowledge that is taught as part of formal training. it is also an opportunity to be exposed to the attitude and cultural perceptions of colleagues and peers. organisational values the organisational values that arise from the identified motivators and enablers, in turn facilitate the effective application of the same motivators and enablers. this recursive relationship is characteristic of the architecture that is emerging from the findings of this research. while motivators form a sound foundation and enablers provide the bricks or building material, the values are the mortar that binds them all together. furthermore, each of the factors is intertwined with, and dependent on, the others for the organization's social learning success and continued robustness (ali, 2000; warne, 2000). the overarching organisational values that emerged from the research nurture, and are nurtured by, the motivators and enablers identified. these values include: • empowerment (where empowerment of staff also makes them accountable); • trust (which entails mutual respect); • forgiveness (in terms of allowing personnel to take reasonable risks, forgiving mistakes and facilitating knowledge construction on the basis of lessons learnt) • cultural cohesiveness (in terms of common identity, shared goals and a shared understanding) • commitment (which includes a mutual commitment and loyalty from the employee to the organisation as well as the organisation to the employee) • openness of the decision making process and • a culture of information sharing. conclusion the implication of this study is that organisations seeking to improve information sharing and knowledge generation need to develop a greater awareness of the processes and strategies of organisational learning. organisational knowledge is distributed across functional groups and its generation and continual existence is dependent on the overall communication climate, which is embedded in the organisational culture. change and information overload are part of the normal operating conditions of most organizations today, and are likely to remain so for the foreseeable future. organisations need to be able to renew themselves constantly and respond creatively to the market forces of the information economy while maintaining their knowledge assets. an organizational culture recognising the value of knowledge and its exchange is a crucial element in whether knowledge work is successfully carried out or not. such a culture provides the opportunity for personal contact so that tacit knowledge, which cannot effectively be captured in procedures or represented in documents and databases, can be transferred. in a culture that values knowledge, managers recognise not just that knowledge generation is important for organizational success but also that it can be nurtured with time, and space (davenport & prusaprusak, 1998). 140 ajis special edition knowledge management december 2001 while the research described in this paper is continuing, and new findings may emerge, it is possible to reach some conclusions about effective social learning in the ado, as identified to date. the esla research has identified a substantial number of complex and interwoven social learning constructs, and the motivators, enablers and values that support social learning and facilitate effective knowledge management holistically. the results of these studies strongly suggest that the interplay between human, social and organizational issues within an organization must be considered first, to effectively facilitate social learning and knowledge management. the study findings to date indicate that information sharing and subsequent knowledge generation is successful where interactive environments are cultivated before technological solutions are considered. technology alone, however well designed it may be, is unlikely to produce effective knowledge management solutions in the absence of appropriate attention to such issues. although some of these issues are specific to a military organization, most are equally applicable to any organization that is attempting to improve its learning capability and knowledge management. acknowledgements the authors wish to acknowledge the vital contribution made to this work by dr jennie clothier whose vision made it possible and to all the defence personnel who participated in this study. thanks are also due to the two reviewers of this paper who gave helpful and constructive advice. copyright commonwealth of australia (c) 2001. leoni warne, irena ali, celina pascoe and katerina agostino for, and on behalf of, the commonwealth of australia assign a non-exclusive licence to use this document for personal use and in courses of instruction provided that the article is used in full and this copyright statement is reproduced. references ali, i., pascoe, c., wame, l., gori, r., and agostino, k. (2000). enabling learning: a progress report on the enterprise social learning architecture task, proceedings of the 5th international command and control research and technology symposium, canberra, australia. argyris, c. (1973). on organisations of the future. beverley hills, ca: sage. bokeno, r. m. and v. w. gantt (2000). "dialogic mentoring." management communication quarterly, november, vol 14, no 2, pp 237-270. davenport, t. h. and l. prusak (1998). working knowledge: how organisations manage what they know. boston, massachusetts, harvard business school press. drucker, p. (1999). "managing oneself." harvard business review, march-april, pp 65-74. dunford, r. w. (1992). organisational behaviour: an organisational analysis perspective. sydney, addison wesley. enneking, n. e. (1998). "managing email: working toward an effective solution." records management quarterly, vol 32, no 3, pp 24-43. ganzel, r. (1998). "elements of a great orientation."training, vol 35, no 3, pp 56. george, c. s. and k. cole (1992). supervision in action: the art of managing. sydney: prentice hall. guzley, r. m. (1992). "organizational climate and communication climate: predictors of commitment to the organization." management communication quarterly, vol 5, no 4, pp 379-402. harvey, l. and m. d. myers (1995). "scholarship and practice: the contribution of ethnographic research methods to bridging the gap." information technology & people, vol 8, no 3, pp 13-27. henricks, m. (1999). "sorting out electronic filing." office systems, vol 16, no 2, pp 30-34. isaacs, w. (1999). dialogue and the art of thinking together. new york: currency doubleday. jordan, b. (1993). ethnographic workplace studies and computer supported cooperative work. interdisciplinary workshop on informatics and psychology, scharding, austria. larsen, s. and i. s. folgero (1993). "supportive and defensive communication." international journal of contemporary hospital management, vol 5, no 3, pp 22-25. lave, j. and e. wenger(1991). situated learning: legitimate peripheral participation. cambridge,uk: cambridge university press. 141 ajis special edition knowledge management december 2001 linger, h. and warne, l. (2001). "making the invisible visible: modelling social learning in a knowledge management context", australian journal of information systems, special edition on knowledge management. mccauley, d. p. and k. w. kuhnert (1992). the professional manager. new york:mcgraw-hill. miles, m. b. and a. m. huberman (1994). qualitative data analysis: an expanded sourcebook. london: sage. morgan, t. (1989). performance management the missing link in strategic management and planning. public sector policies for the 1990's. d. c. corbett. melbourne, public sector management institute, faculty of economics and politics, monash university, pp 243 250. ruppel, c. p. and s. j. harrington (2000). "the relationship of communication, ethical work climate, and trust to commitment and innovation." journal of business ethics, vol 25, pp 313-328. senge, p. m. (1992). the fifth discipline: the art and practice of the learning organization. sydney: random house. tyler, k. (1998). "take new employee orientation off the back burner." hr magazine, vol 43, no 6, pp 49-57. warne, l. (2000. "understanding organisational learning in military headquarters: findings from a pilot study"", australian journal of information systems, vol 7, no 2, pp 78-87. webber, a. m. (1993). "what's so new about the new economy?" harvard business review, jan-feb, pp 24-42. 142 microsoft word ajisvol10 1_final_edit.doc ajis vol. 10 no. 1 september 2002 2 editorial this is the first issue of volume 10 of the australian journal of information systems. as editor i am pleased to report the continuing interest in the journal both in australia as well as overseas. our first paper is presented by pierre-jean charrel, institut de recherche en informatique de toulouse. the concept of viewpoint is studied in the field of the modelling and the knowledge management concerned in the upstream phases of a designing process. two models of viewpoint and correlation are proposed. sigi goode, school of business and information management, australian national university presents our next paper entitled – exploratory analysis of the effect of consultants on the use of world wide web sites in sme’s. this preliminary research finds a number of interesting outcomes. no significant relationship is found between consultant or isp engagement and world wide web site use, regardless of whether the consultant was engaged as site developer or site maintainer. our next paper is presented by rene jorna, university of groningen, the netherlands. in developing a conceptual framework for the cognitive analysis of knowledge management, the paper explains that the dominance of tacit, coded or theoretical knowledge matches well-known organizational forms. mikko siponen, department of information processing, university of oulu, presents the next paper entitled – database security and the problem of polyinstantiation: a moral scrutiny. this paper examines the morality of polyinstantiation from the viewpoint of moral philosophical frameworks. our next paper is presented by peter slade and jeanette van akkeren, faculty of business, university o he sunshine coast. this paper looks at which factors were important in impacting on small firms’ decision making in respect of information technology and e-commerce adoption. findings indicate that generally, the more a firm was concerned about its competitive position, so such a firm was likely to develop a web site. the final section of his issue is devoted to the area of sme’s. the first paper in this section is entitled – have sme’s benefited from e-commerce? and is presented by simpson poon, charles sturt university. this paper looks at a number of studies of ec and sme’s. tim beal, victoria university of wellington and moha asri abdullah, universit sains malaysia present our next paper entitled information and communication technology in asia pacific countries: implications for small and medium enterprises in malaysia. this paper explores global and asia pacific ict usage and the major factors influencing smes. it focuses particularly on malaysia where the development of ict usage amongst smes is one of the main policy items in the agenda of the malaysian government. maen al-hawari and helen hasan, university of wollongong present our next paper entitled evaluating the knowledge assets of innovative companies. the paper will present an integrated model and discusses the appropriateness of a set of knowledge elements as a means of measuring the knowledge asset of an organisation. colin jones, school of management university of tasmania, rob hecker, school of management university of tasmania and peter holland, department of management, monash university present our next paper entitled small firm internet adoption: a market oriented approach. the focus of this paper is the empirical testing of a normative web-based commerce adoption model developed from a review of the extant literature related to electronic marketing, the internet and the diffusion of new innovations. our final paper entitled electronic commerce and market focus: some findings from a study of swedish small to medium enterprises is presented by lejla vrazelic, university of wollongong, deborah bunker, school of technology and management, university of new south wales, rob macgregor, university of wollongong, sten carlsson, karlstad university and monika magnusson karlstad university. this paper presents the findings of a study of 118 swedish small to medium enterprises (smes) that have adopted ec. the results of the study suggest that improvements to customer service, internal efficiency and organisational competitiveness have become equally important when making ec investment decisions. finally helen hasan, university of wollongong presents a book review examining the ronald maier book entitled knowledge management systems: information and communications technologies for knowledge management. once again i would like to pass on my thanks to the authors and consulting editors for the high quality of material in this issue. i would like to thank lily soh and diniz da rocha for their continued assistance with the journal and the maintenance of the home page. rob macgregor editor http://www.uow.edu.au/ajis/ajis.html the australasian journal of information systems (ajis) began in 1993 and was based on the simple realisation that there was a need for an academic ‘outlet’ for is research in australia australasian journal of information systems volume 14 number 2 june 2007 65 information technology and aesthetics: passive and active dimensions john d. haynes school of information technology faculty of technology charles darwin university john.haynes@cdu.edu.au david b. paradice management information systems department college of business florida state university dparadice@cob.fsu.edu abstract this paper is concerned with examining and recognizing aesthetics in an information system (an organization incorporating both humans and information technology). information systems designers need to consider the aesthetics of their systems. to do this, they need some conception of what aesthetics is and is not. this paper constructs such a concept. is designers must avoid thinking of a specific system component such as a graphical user interface as defining the aesthetic dimension of their work. aesthetics emerge from the wholeness of things, not from specific parts or components. as such, aesthetics may naturally be considered in “systems”, and we propose that an effective manner of thinking of aesthetics is in terms of “themes”. we also argue that one must consider not only aesthetics that may be perceived by the senses (a passive dimension), but also aesthetics that may be conceived in the mind (an active dimension). we also consider the important differences between “definitions” (passive dimension) and “themes” (active dimension). finally, the study of aesthetics in is has greater implications than simply recognizing and appreciating beauty and art. the very human capacity for recognizing and appreciating beauty and art, the active aesthetic dimension, is also the same capacity for effective creativity and happiness. it follows that if an information system encouraged and provided the enabling circumstances for the human capacity of thematic recognition (as found in the human appreciation of art and beauty) it thereby also provides the grounds for effective creative and visionary organizational management. australasian journal of information systems volume 14 number 2 june 2007 66 key words: aesthetics, information technology, information systems, active aesthetic dimension, thematic recognition, ethics, passive aesthetic dimension, perception, conception, c. west churchman, heidegger. introduction to begin to consider the humanly appreciated notion of aesthetics in an information system is to already begin to consider more closely the human component of information systems. from this perspective we do not look literally at the information system, but rather we look at it indirectly through the lens of human being. in so doing, we thus expose limitations in what it means to have a literal “systems view” closed to the ramifications of the genuine interaction between humans and information technology. wilber (1998) describes – this kind of closed – systems theory as a “flatland holism” focused on objectifiable processes but lacking any connection to beauty, poetry, desire, love, moral wisdom, or artistic expression. churchman (1971, 1982, 1994), a leading systems theorist, repeatedly made calls for the expansion of management science from literal, functional, closed, process considerations to the openness of the “thinking designer”; as he put it “to transform the success of the other functions into thinking” (churchman, 1971, p 262). churchman wanted management science to be infused with the ‘living’ vitality of human thinking. it follows that human thinking is far more “alive” when humans themselves are in a state of care and creativity. while mitroff and denton (1999) have identified the influential role of spirituality in the workplace in general, armour (2002) identifies the “spirit” of projects as being the deciding factor in information systems success. information systems are used to support decision-making processes used in resolving problems, and mitroff and linstone (1993) note that aesthetics play a fundamental role in the means we use to address problems. kuhn (1962, p. 158) observes that sometimes "it is only personal and inarticulate aesthetic considerations" that can lead scientists to believe that a new proposal is on the right track. dalal (2002) asks flatly, “why do we not refer to the aesthetics of information systems as we do of works of art?” information systems must be designed with aesthetics in mind for dalal’s request to be realized. this requires is designers to have some conceptualization of aesthetics. recognizing aesthetics is an important and crucial first step, but understanding aesthetics as it may apply to is may be truly problematic. this paper provides a construction of the aesthetic concept that is designers may employ. we feel that recognizing aesthetics is tantamount to recognizing what it is to be open for inspiration. human inspiration and creative design become all the more effective when they are openly recognized to be important. such self knowing enshrines and empowers the is designer to become the “thinking designer”. one crucial outcome from the self knowing of human creative design and genuine human inspiration is the realization that it is important in its own right. it is open recognition that allowed ethics to emerge as a crucial consideration in the design of information systems. ethics in information systems continues to receive attention, particularly as it relates to the proper handling and storage of information (mason, 1986; paradice, 1990; banerjee, 1998; davison, 2000). ethics continues to be an important field precisely because it is of direct and recognized concern to the “thinking designer”. the continued genuine interest in ethics and information systems does not arise because ethics is seen as an object in some systems process, but because it is recognized to be inherently part of what it is to be and what it is to continue to be human. as other humans (clients, customers) recognize the ethical embrace in certain information systems they relate in a more holistic, genuine way to the information system and thus, especially in service industries, purchase more of its products. ethics provides for the care of the human, and in a australasian journal of information systems volume 14 number 2 june 2007 67 state of care humans are open to creativity, genuine action and also appreciation, among other purchase considerations, for a product produced in an ethical way. while ethics appeals to the heart (the senses of care and preservation), aesthetics appeals to both the heart (in this case, the passions) and the mind (our sense of beauty). in the fields of philosophy and art, where aesthetics are normally considered, there is much debate over exactly what aesthetics encompasses and how it may be defined. the central conceptual construction that we pursue in this paper is to transform thinking about aesthetics in the context of information systems from a passive approach that relies on definitions and perceptions, into a vital “living” experiential open “thematic” and conceived active approach abundantly suited to c west churchman’s “thinking designer”. passive approach: definitions: active approach: theme definitions are a crucial starting point for any endeavor, but they only provide a part or a glimpse of what they are describing. the risk is that they are somehow treated more than as a literal beginning and so are treated as an end in themselves. because they are only a glimpse (or a part) of what they describe we refer to definitions as passive. to shift from the reliance on definitions of aesthetics (a passive approach) into a thematic approach (an active approach), thereby makes a crucial transformation, namely, to begin to act out (to re-cognize; to cognize again; to draw upon past experience). in an active approach, we take responsibility for thinking through past experiences of, in this case, aesthetic pleasures in relation to a current aesthetic circumstance. but when we compare a current experience with a prior one recognized as aesthetic, what we compare are themes. we do not literally compare each part of the past experience with each part of the new experience. we compare the overview, or theme, of each. this capacity for thematic comparison is important for this paper since thematic comparison is active and reflective. the more thematic comparison is achieved the more likely it is to become an effective capacity for active vision generally. the more an information system reflects aesthetic dimensions, in what ever way it contextually does, the more opportunities there are for effective visionary output and management. aesthetics and the aesthetic dimension of life aesthetics is defined in a standard dictionary as “a branch of philosophy dealing with the nature of beauty, art, and taste and with the creation and appreciation of beauty.” when we rely solely on definitions (a passive approach), we note an immediate limitation of subsequent discussions of aesthetics. these discussions run the risk of being unproductive simply because there are many different and seemingly unrelated definitions of beauty, art, and taste. on the other hand, while not defining art, the philosopher stephen pepper does identify four themes (an active approach) of beauty, each corresponding to one of his four world hypotheses (yanal, 2001). • mechanism, associated with santayana, holds beauty to be the experience of immediate pleasure, and ugliness to be the experience of immediate displeasure. • contextualism, which pepper associates with dewey, holds beauty to be “voluntary vivid intuitions of quality.” australasian journal of information systems volume 14 number 2 june 2007 68 • organicism, associated with bosanquet, holds beauty to be an “integration of feeling”, and the degree of beauty varies with “the degree of integration and the amount of material integrated.” finally, • formism, primarily associated with aristotle, holds beauty to be the conformity of an object to the “norm” of the class of which it is a member. pepper implies that one’s worldview (or weltanschauung) is integrally related to one’s conception of beauty. like pepper, those persons who are moved by beauty are moved in terms of themes which resonate personally with themselves. these themes are ways for them to connect with beauty. such a connection cannot be made with a definition. a thematic approach more closely and directly connects to our hearts and our minds (we connect with a lived experience), whereas definitions appeal in an indirect way only to our minds through our senses. we perceive definitions, but we conceive when we relate with themes. similarly we initially perceive works of art and, then in relation to their aesthetic capacities, we conceive them thematically. we can see how the difference between a perception and a conception plays itself out in the following example. consider works performed in a manner different from the original context for which they were intended. for example, consider a musician performing a composition created by another musician where the other musician’s work is considered a work of art. consider an extreme case, for example, the elementary school child who plays a composition and who faithfully executes each and every note of the composition. the same composition, also faithfully executed by a master musician, may be considered a work of art, yet the schoolchild’s effort is not. the reason for this is that the child has not yet made the shift from perception to conception. the schoolchild is still working at a definitional stage, defining the notes and carefully reproducing them without fully comprehending the active power of the theme released by the notes. the master not only perceives the notes and the melody; but, conceives the full power of the theme of the music and thus is able to render the playing of the music with the full active vigor of that conception. similarly for designs in information systems, opportunities will be precluded if those designs limit themselves to perceptions of how the information system works in terms of defining its operations or in comparison with other information systems operations. perceptions are based on what already is the case, and in this sense they are present-oriented, or passive. but when design principles embrace the aesthetic dimension of an information system, there is a focus towards what is not yet the case, and new questions emerge such as, “how can we provide the enabling circumstances for humans to be creative?” and “how can we provide the enabling circumstances for humans to be visionary?” these questions are questions of aesthetics. they are questions that relate to the human capacity to be creative: namely, a human’s ability to conceive of thematic connections. haynes (1999) points out, themes cannot be perceived, but rather, they can only be comprehended through conception. we perceive colors and brush strokes and the outlines of, for example, a painting but we conceive of the theme that the painting portrays. a work of art portrays and offers its theme for inspection in much the same way that the concept of ethics offers its theme for consideration. in the case of the work of art, the beauty, taste, and artwork arise out of the whole state of the piece, the whole presentation, the whole accomplishment, which is itself thematic of its beauty, taste, artistry, etc. the beauty, taste, and art are not a part of the whole piece, but integral with it. we conceive these out of the whole in a thematic way. similarly when ethics is genuinely embraced in an information system, it is not the sets of rules of what it means to be ethical that portrays that ethic, but rather how ethics is conceived of by the is organization, and how each member acts according to his or her own conception. we can perceive a mission statement on ethics, but that does not tell us if the is is ethical. we can only come to know that by seeing how australasian journal of information systems volume 14 number 2 june 2007 69 those rules are adopted. the adoption of those rules requires human interpretation. interpretations are conceived rather than perceived. art is a product of culture and it is through art that a culture expresses its life (iep, 2001). similarly the culture inherent in an information system expresses something vital and living. if not, then there is an absence of culture in an information system in which case there is an absence of vitality. thinking designers thrive in conditions of vitality. like the theme in a work of art, a culture expresses itself as a theme and so it is not able to be strictly perceived (we perceive only its outward manifestation); but, rather it is more that the theme of the culture moves the thinking designer into action. comprehending such movement entails conception, rather than perception. passive dimension (perceptions); active dimension (conceptions) on the passive side are the experiences we have when we take in the external beauties of the world. the passive aesthetic is defined in terms of what people see, hear, smell, touch, and taste. in other words what they perceive. on the active side are the experiences we have when we consider things that people do. the active aesthetic includes performances as a musician, athlete, or creative problem solver. in other words what we conceive. a deeper conception of the aesthetic experience requires something well beyond the excitement of the senses, well beyond perception. that is to say, when the aesthetic experience is typically considered, the focus may be on how some art object looks or feels in its totality, or in the case of music: how it sounds in terms of the total performance. the perceptions of the senses and the conceptions of the totality of what is experienced cannot be divorced from prior experiences (iep, 2001; van gerwen, 1993). a particular work of art is first perceived as an object by the senses, and then is conceived to be beautiful in comparison with other objects – as a totality that have been experienced. indeed, to appreciate the “beauty” in an imaginative solution to a problem requires extending the concept of the aesthetic experience to include significant cognitive processes beyond simply touching, seeing, tasting, hearing, and smelling. two philosophers who gave aesthetics a great deal of attention were immanuel kant and john dewey. according to leddy (2001), kant distinguished (in our terms) thematically between two types of beauty: free beauty and adherent beauty. free beauties included birds, trees, music-not-setto-words, and even wallpaper. adherent beauties, on the other hand, included people, horses, and buildings. the difference in the two categories is that adherent beauty depends on a notion of the purpose of the thing being considered. for some reason, kant presupposes a notion of perfection for adherent beauties, and hence the beauty of them depends on their relationship to that ideal. consequently, kant can admire line art in one form, but dismisses the artwork of tattoos on humans because they interfere with the “perfection” of the human body. kant’s moral philosophy emphasizes that pleasure cannot be the impetus for moral action. kant’s position is that pleasure varies from individual to individual, and so pleasure cannot be a basis for society-wide agreement on moral action (donohoe, 2001). agreement on the universal validity of a moral proposition could never be achieved if it were dependent on endorsement only when it provided personal pleasure. kant therefore distinguishes between upper and lower faculties of desire. the lower faculty is driven by the pursuit of pleasure, the upper faculty by the pursuit of moral actions. humans are partially sensuous and partially rational. since all humans are driven by pursuits of pleasure, kant introduces the idea of a moral “command” to do good, the categorical imperative. tightly integrated in these notions is the idea that humans have the capacity of free will. it is the human ability to choose to do good that ultimately produces the pleasure associated with making moral choices. consequently, pleasure is not the motivator for performing moral acts, but pleasure may be attained from doing them. australasian journal of information systems volume 14 number 2 june 2007 70 understanding kant’s concepts of moral behavior is critical to understanding his concept of the aesthetic experience and aesthetic judgment. according to guyer (see donohoe, 2001), kant describes two separate acts of reflection in the making of an aesthetic judgment. the first act recognizes the pleasure attained. the second act uses the experience of the first act as evidence upon which to make an aesthetic judgment of the experience. it is in the second act of reflection that the original pleasure is categorized as either a satisfaction of a desire or the result of an aesthetic experience (donohoe, 2001; van gerwen, 1993). dewey also took a thematic approach and stressed the importance of recognizing the significance and integrity of all aspects of human experience (iep, 2001). the roots of aesthetic experience lie, dewey argued, in commonplace experience, in the consummatory experiences that are ubiquitous in the course of human life. dewey felt that the creative work of artists was not unique: any intelligent use of materials or ideas to create imaginative solutions to problems could afford the same type of immediate satisfaction one feels when viewing a great work of art. therefore, the aesthetic experience could be broadened to include dimensions of life not typically associated with the “creation and appreciation of beauty.” dewey also rejected the notion of the audience as a passive recipient of art (iep, 2001). the one appreciating the art must assimilate the artist's work, once again in the context of that person’s prior experiences. the appreciator must compare the current experience with prior experiences, integrate it into their personal experience, and perhaps discriminate among many possible types of prior experiences in order to assess the current feeling about the art being considered. why an aesthetic dimension can be considered in information systems although it has hardly been examined in the is literature, there is precedent for considering aesthetics in system design. bansler et al. (2000) touched on the topic in a passing comment about web designers who perceived a need to improve the “look and feel” of an intranet environment. markus and mao (2001) note that developers may have a goal of producing an aesthetic design, but these researchers do not provide any indication at all of what the phrase “aesthetic design” means or implies. as an extension of these earlier considerations, one of the central conclusions of this paper is that a “look and feel” and an “aesthetic design” can only be genuinely (actively) considered in a thematic way. a genuine aesthetic design implies its own thematic recognition. a real capacity (a conception rather than a perception) for improving a “look and feel” in an it environment comes from a thematic ability to connect who looks and feels with what is looked at and felt. wells (1996), in his examination of postmodernism and information technology, notes that heidegger characterized the modern technology age as lacking in philosophical thought. we can take this to mean that the design of information systems are lacking in “thinking (philosophical) design”. we suggest that consideration of a distinctly philosophical concept, aesthetics, can enhance the way information systems are “thought-fully” designed and consequently “thought-fully” discussed. however, heidegger (heidegger, 2001) also provides an incentive for pursuing the goal of this paper. consider the following quote from the question concerning technology (section 35): because the essence of technology is nothing technological, essential reflection upon technology and decisive confrontation with it must happen in a realm that is, on the australasian journal of information systems volume 14 number 2 june 2007 71 one hand, akin to the essence of technology and, on the other, fundamentally different from it. such a realm is art. we are not so much interested in the realm of art, but what constitutes essential reflection upon art, and to draw upon the capacity of such reflection, which is an aesthetic capacity, in the design of information systems. we have noted accordingly that such reflection comprehends a theme and it does so not by perception, but by conception. what is akin then to the essence of information systems is the theme of an information system. we can conceive of a theme in general or a theme in particular. in terms of the particular design of an information system, we direct our attention to aesthetic considerations, namely: what is thematic or could be thematic in a particular information system? information technology is only part of an information system, and so in terms of an aesthetic thematic approach we must pay particular attention to the relationship between humans and information technology. it is crucial to see that relationship in a thematic way, that is, as information technology being used by humans. by providing for the enabling circumstances for the emergence of that kind of thematic viewing or aesthetic sense, in an information system, humans can draw inspiration from a relationship with information technology. there are distinct dangers in not providing for the emergence of an aesthetic sense in an information system. this danger was noted by wells who observes that marcuse believes the nature of technology is to create a universal abstraction that alienates people from the world. coyne (1995) observes how information technology can marginalize ethics, decontextualize the human experience, and provide a means for one person or group to dominate another. introna (2002) argues that ethical considerations are becoming more difficult as information technology increasingly disassociates the consequences of decision makers’ actions from the people affected by those decisions. clearly, there are those who believe information technology is in need of some kind of “humanizing” experience. consideration of providing for the enabling circumstances for an aesthetic dimension in information systems can address these concerns. to summarize the above discussion and draw some implications: • definitions of beauty and art are not critical to our purpose. one need only recognize that there are thematic passive and active experiences that give pleasure, and that pleasure contributes to the aesthetic dimension of living. many have known the frustration (lack of pleasure) in being unable to acquire information in a timely manner, or in dealing with an information system that does not produce output in a form needed. when information is available as needed, when needed, and in the form needed, the information system has greater potential for providing pleasure on some aesthetic plane which in turn stimulates more effective action. • aesthetic experiences go beyond the five primary senses (perception) and include “performance” activities, including creative problem solving (conception). indeed aesthetic experiences, as we have noted, are directly related to our human capacity for comprehending themes. if we no longer were confronted, or even limited in its expression, by beauty in any of its multifaceted forms, what would happen to our aesthetic sense? we have also noted that our aesthetic sense is directly connected to our capacity for effective long term vision. creativity is a hallmark of human capability. but creativity, or creative thinking, is distinct from aesthetics. all human creativity is an aesthetic experience drawn from the well-springs of the subtle capacity of conceiving and either that conception arises from an inner aesthetic mood or an outer aesthetic inspiration. in this sense we believe that conceiving (which contributes to the welfare of australasian journal of information systems volume 14 number 2 june 2007 72 humans) is directly connected with the experience of aesthetics. effective creativity requires excitement and a spirit of adventure where the imagination is, to a large extent, unrestricted. in simple terms if a workplace is drab that is, unaesthetic then creativity may be curtailed. if ideas are drab and ideas also provide the basis for an aesthetic experience then creativity will suffer accordingly. creativity is a necessary prerequisite for good design and good design is directly connected with aesthetics: a good design is aesthetic. thus effective creativity in particular produces aesthetics and the absorbed inspiration of aesthetics in general provides a ground for the possibility of effective creativity. in this way creativity is indirectly connected with aesthetics. churchman observes that “design is one of the most fascinating” activities of man, (churchman, 1971, p. 3) and that mankind's ability in creating art, science, religion, and politics is paramount. the design of information systems is no less a creative act. in fact, it may be that if we do not comprehend what it is to provide for effective human creativity in the design of information systems, we run the serious risk of limiting such precious human capacities in the future. • according to kant, if people are motivated by a will to perform moral acts, people are endowed with the necessary and sufficient prerequisite to cultivate the aesthetic dimension of life. if kant is correct, we must provide the enabling circumstances to allow for an aesthetic dimension to emerge and all the more so (for the above reasons) in an information system. information systems workers are no less predisposed to moral action than any other professional group. in fact, studies have shown that information systems personnel conduct themselves in a manner consistent with concern for the well being of society (udas et al., 1996; paradice and dejoie 1991). there is no reason to presuppose, therefore, that information systems workers are not capable of considering and ultimately enhancing the aesthetic dimension of information systems. • living a “good life” involves fulfillment on all dimensions of life. people interact with information systems in many ways. certainly, information systems pervade the workplace. people also utilize information systems when traveling (in the form of transportation schedules or the dynamic output of road signs), when relaxing at home (in the form of on-line entertainment guides), and when shopping (either on-line or to determine prices on items actually in stores or during the purchasing process) among other ways. incorporating an aesthetic concern into information systems can have far-reaching implications for enhancing the ability to live a “good life”. why is living a “good life” thematic? because what is of the good is thematic precisely because a person with good judgment conceives (comprehends the totality of it as theme) it to be so. you cannot have for example, a theme of evil. evil is not capable of being thematic. it does not have sufficient quality to integrate itself into a whole; it is always fragmented. the beholding of beauty as an expression of living the “good life” on the other hand has the quality of being able to be unified and thus “exhibits or performs” itself thematically. if it is thematic it can be conceived by humans who have the wisdom and experience to do so and wisdom is fertile ground for “thinking designs”. • the roots of the aesthetic experience are in everyday activities. the ubiquitous nature of information systems certainly plays a role in everyday experiences. if an appreciation of an aesthetic dimension can be incorporated into information systems design, a widespread impact on the aesthetic dimension of people’s lives can be realized. • aesthetics involves recognition of a pleasurable experience. we need to carefully distinguish between fleeting pleasurable experiences and those pleasurable experiences that are based upon real past experiences. aesthetic recognition entails drawing upon an experience and comparing that experience to the current experience. we have noted that this is an australasian journal of information systems volume 14 number 2 june 2007 73 aesthetic capacity for comprehending themes. such aesthetic experiences are pleasurable individually because our own experiences are involved. when we witness art at the level of perception and conception we are an audience to that art. information systems, like most complex entities, may be perceived and conceived on many levels. in information systems, the audience may be users or information systems professionals. it follows that either user or professional, or both, will engage better if they are undergoing a pleasurable experience, and it will be all the more enduring if it is a pleasurable experience based upon conception. that is, the pleasurable experience was not just fleeting through perception, but connected – and thus conceived to past pleasurable experiences. as further confirmation of what we have previously elaborated as the need to distinguish between perception and conception, and the caution that it is conception that entails the aesthetic experience and not purely perception, sibley (1959, p. 426) notes that something may be described in nonaesthetic terms “as fully as we please”; but, we are not thereby put in a position of having to admit that the thing is, in fact, aesthetic. in other words, we have a perception not yet confirmed by conception. we find evidence of this pervasively in advertising. for example, a sign gives us immediate pleasure by perceiving it, but it becomes fleeting if it is not grounded in previous lived experiences that genuinely gave us pleasure. passive aesthetics in information systems perception the passive aesthetic of information systems design includes those aspects that are detected by our senses: seeing, touching, hearing, tasting, and smelling. the last two, tasting and smelling, are not normally excited by information system characteristics, and therefore will not be considered in depth here. however, a broader examination of how information systems technology is used in other domains could lead one to reasonably consider characteristics of taste and smell later. much has been studied that is related to what people sense when interacting with information systems. beginning with the study of tabular data representation versus graphical data representation (desanctis, 1984; jarvenpaa and desanctis 1988), the information systems literature has a long history in studying what may be more generally called media richness (markus, 1994; elshinnawy and markus 1997). with regard to sight, the judicious choice of color combinations, use of fonts, use of effects such as blinking, use of transitions, use of special effects such as curled corners, and other visual cues has gained greater appreciation (flanders and willis, 1998) with the advent of multimedia support in systems. the sense of touch is excited in the use of the keyboard, mice, track balls, pointer sticks in notebook keyboards, track pads, stylus, light pens, and similar input devices. in the design of information systems, most users have little control over the design of these hardware devices. information systems professionals and academics may, however, need to give thought to the location of icons in an information system’s interface since the location determines where one points. information systems professionals and researchers need to pay attention to the concept of “focus” in interface design. an object in an interface “has focus” when it is ready to receive input. users will likely find a design that moves focus from object to object in a natural progression to be more aesthetically pleasing than a design in which the focus moves in a seemingly random order. with regard to hearing sensations, users typically experience sounds associated with events such as warnings, incoming mail, file deletion, and so forth. where sounds are utilized in this manner, users need control over the sound used and its volume. lack of such control, while not implicitly leading to a lack of aesthetic beauty in a system, could be an obstacle in attaining passive aesthetic quality. australasian journal of information systems volume 14 number 2 june 2007 74 it is obvious that passive aesthetics related directly to users’ and professionals’ perceptions have undergone radical changes over the past several decades, precisely because passive aesthetics are a crucial design feature of current and future computer technology and systems. this trend will continue and all the more so with a better understanding of the relationship between human perceptions and human conceptions. active aesthetics in information systems conception turning to the active aesthetic of information system design, focus must move to something more than the technology itself. attention must turn to the information system user’s objective in using the system. the need for an organization's information system strategy to be coordinated with its business strategy is accepted (mata et al., 1995). unlike the passive dimension, in which “measures” of aesthetically pleasing characteristics rely on the senses being excited, in the active aesthetic dimension the mark of aesthetic pleasure may occur when the system goes unnoticed. to put this point another way, much as a well-officiated sporting event is characterized by officials who essentially do not interfere with play (i.e., when the official does not interfere with the theme of competition), an information system that does not interfere with the user's goals is very likely to be viewed as pleasing by that user. why? because such a situation of minimal interference frees the user to use her imagination, and in so doing releases the user’s conception, as an active aesthetic connection, with her past lived experiences. for example, consider a small aspect of today's constantly changing business environment: telephone area code and postal code changes. while they are relatively stable, these codes do change. many systems fill in these values automatically based on the entry of an address. unfortunately, many shipping systems do not allow a manual override of these codes. in this case, the system has interfered with the goal of the user (efficient processing). the result can be that a customer is not notified in a timely manner about a change in an order status or a package may arrive late. these events may engender frustrations in the customer that ultimately become frustrations for the organization. the customer is then left with an inability to fulfill her goal. thus, conception was blocked and a perception of frustration was created in its place. another aspect of the active aesthetic can be realized in the manner in which the information system organizes its information vis-à-vis the user's mental model. wells and fuerst (2000, 2001a, 2001b) have studied the role of the metaphor used in an information systems interface. they describe how metaphors are human derived models that apply tangible, concrete, recognizable objects (from a source domain) to abstract concepts and/or processes (in a target domain). metaphor use is an effective means to improving a user’s ability to understand an interface and use a system. clearly, the use of metaphor is an activation of conception in the user (haynes and mahfouz, 2001). wells and fuerst discuss the advantages of, for example, the use of an airplane schematic in seat selection while making on-line ticket reservations or the ability to place furniture in a virtual room while furnishing an office or home. however, problems can arise when an incorrect metaphorical model is used, when the metaphor is unrealistic, or when a mixed metaphor occurs. when the metaphor is appropriate, the user’s learning curve is flatter and productivity, information retention, satisfaction, and accuracy increase. wells and fuerst claim that deriving the interface from the appropriate domain is attractive because it couples the user’s view of the domain with the designer’s view, thereby reducing the likelihood of employing an incorrect metaphor. the proper metaphor leads to an aesthetically pleasant experience that has elements of both the passive and active dimensions. the passive aspect is inherent in the perceived connection a user will likely make between the system’s intent and its interface. the active aspect comes into play when the user is australasian journal of information systems volume 14 number 2 june 2007 75 better able to execute the tasks at hand, because conception now comes into effect, due to the better “fit” of the system and the task domain. a final way to consider the active aesthetic in information systems occurs in the design of knowledge-based systems. of interest here is the more general class of systems that are inquiring systems (churchman, 1971; courtney, 2001; courtney et al., 1997 and 1998; haynes, 2001; kienholz, 1999; richardson et al., 2001). according to churchman (1971, p. 8), “inquiry is an activity which produces knowledge.” at the heart of any inquiring system should be an epistemology, a theory of knowledge. the essence of this theory is how people answer the question of how certain things are known to be true and other things to be false (churchman, 2001). clearly here we have a high level of conception tried and tested over, in some cases, lifetimes of experience. churchman outlines five archetypal inquiring system designs based on his understanding of the works of five philosophers (leibnitz, locke, kant, hegel, and singer). the designs are not exact accounts of how each philosopher conceived the theory of knowledge. rather, the designs are derived from a reconstruction of each philosopher's ideas in the language of the design of an inquiring system (churchman, 1971, p. 18). in each case, in our terms, conception is the key. consider the use of conception in the following example for a leibnizian-type enquirer. in evaluating an information system the evaluator will tend to work through assumptions and proposed facts, examining each for validity vis-à-vis her weltanschauung. as each assumption or proposal is validated to our user, she will build an ever-enlarging fact net designed to “prove” some point. suppose the system, on the other hand, is based on a guarantor of consensus. a new “fact” to this system can only be considered in terms of prior occurrences of the same category of facts for which some consensus has been achieved. thus, on the first occurrence of a new fact, the system can only make note of its occurrence. there is no consensus (yet) regarding the fact. so, while the user may deduce the truth of the new fact using leibnizian-type inquiry, her system fails to produce similar results. when the underlying epistemology of an inquiring system is not congruent with the user's epistemology, the inquiring system will be considered less aesthetically pleasing than when the user and the system share similar concepts of knowledge production. in other words, when the conception of the user does not evoke from prior experience a match to the current experience the consequences are not pleasing; when there is a match there is an aesthetically pleasing outcome. information systems design: passive dimensions and active dimensions we have examined the passive and active dimensions of an information system in relation to aesthetics, respectively, in terms of: 1. definitions and themes. 2. perceptions and conceptions. 3. passive aesthetics outputs based on perceptions, and active aesthetics outputs based on conceptions. our focus throughout the paper has been on examining and recognizing two distinct aspects of aesthetics in relation to humans in the context of an information system. these two primary differences are in human perceiving and human conceiving, and both allow us to see the crucial importance of aesthetics for an information system. effective visionary computer technology design australasian journal of information systems volume 14 number 2 june 2007 76 focuses on the realm of human aesthetic perception, but first the designer must have conceived this to be the case. that is, the designer must have conceived that users are personally pleased to be bodily involved at the level of their senses. effective visionary management design, which is at the same time ethical, draws its inspiration from the realm of aesthetic conception. in both cases conception is the key, and aesthetics is a crucial element. moreover, the effective use of conception requires an environment that nurtures that conception. vital, active designs are “thinking” designs, but they cannot be thinking designs without aesthetics and without conception. with a sense of aesthetics and a capacity to draw upon conception, and with both in full measure, the designer becomes an artist. when the designer becomes an artist, that artistry transforms technology and gives it an appropriate place in the context of that information technology’s association with humans. guidelines for considering the aesthetic dimension in closing, there are several guiding thoughts that may be put forward with regard to considering an aesthetic dimension of information systems. the aesthetic dimension is not characterized as “plug and play”. in other words, there is no aesthetics module that can be added to a system to make it aesthetically pleasing. there is no component that can be inserted into a system to address the aesthetic dimension. similarly, the lack of a particular component or characteristic does not imply a system lacks aesthetic beauty. the aesthetic dimension reflects itself in the holistic concept of human conception. the aesthetic dimension is not described by graphical user interfaces. perhaps this is a corollary to the preceding point. when it was novel, the (generic) graphical user interface may have contributed to a system's aesthetic quality. a desk-top metaphor may be completely inadequate for a task that involves surgery, for example, or operating a complex machine. aesthetics occurs when the system elevates the work to a new level of effective vision. singer (1923, p.54) argued that "beauty is that quality of a work of art which makes its tragic turning a moment inspired by an heroic mood of new creating." his argument is that only when one is struggling in some situation, and then confronts something that elevates that person beyond her struggle, does one recognize aesthetic quality. he continues (1923, p. 54), "the value of beauty to life is its inspiration to a soul in travail...". all vision is born of struggle, and it is individually the struggle to reconcile one’s past experiences of creating moments of joy with one’s current ones. aesthetics may occur on different levels or from different perspectives. all complex systems are characterized by multiple stakeholders with often differing perspectives. information systems are no different. there is a technical perspective that is reflected in the design and construction of the system. there is a personal perspective that is reflected in the manner in which a user interacts with the system. there is a domain perspective reflected in the manner the system accomplishes its purpose in a specific task domain. the algorithm employed could be aesthetically pleasing while the system itself (as perceived by the user) may not. australasian journal of information systems volume 14 number 2 june 2007 77 according to philosophers, humans seek happiness on several dimensions, one being the aesthetic dimension. aesthetics evolved as the “philosophy of art”, but it actually extends beyond the art world. in fact, when people experience aesthetic pleasure in the routine of their lives, they truly begin to “live the good life." churchman (1971, p. 199) notes: “the contented life is the complete life, made up of all those aspects of life that make it meaningful.” one “lives the good life” when one is successful in finding happiness on all dimensions of life, and when one creates opportunities for others to do likewise. since information systems permeate modern life, developing the concept of an aesthetic dimension of information systems can have a great impact on the development of the aesthetic dimension of life in general. conclusion we have demonstrated in this paper the construction of the concept of aesthetics in terms of two dimensions. the second dimension, the active, goes well beyond our perceptions (the passive dimension) in the sensations we feel when one of the five senses is stimulated. aesthetic pleasure can also be experienced in various areas of performance, including creative problem solving as well as the everyday function of people's lives where all of these areas draw on our lived experiences; thus, enriching aesthetically the conception of our lives and by effect the lives of others. therefore, two areas in which the aesthetic dimension of information systems exist have been identified and explored. one is the passive aesthetic dimension, which focuses on the parts of the system seen, touched, and heard (namely perceptions) and we also articulated this dimension in terms of “definitions”. the second is the active aesthetic dimension, which focuses on the performance of the system and, in the case of inquiring systems, on the system’s underlying epistemology and we also explicated performance in terms of conception and further, its connection to creativity. we have also explored the view that the active aesthetic dimension can apply to the art of management, insofar as a comprehension of aesthetic considerations, together with ethical considerations, can provide a basis for visionary management. recognizing the need to attain fulfillment through happiness, and the role of the aesthetic dimension in doing so, can be a guiding force in the evaluation of information systems. information system designers must incorporate ways for users to modify and control system features that fall in the passive aesthetic dimension, so that users may customize these features in ways that please them. information system designers must be sensitive to the active aesthetic dimension, recognizing that the way a system functions must be consistent with both human and business information processing in order to create an opportunity for a pleasurable aesthetic experience for users. designers, however, must not fall prey to believing that aesthetic quality can be added to a system by simply programming in a particular feature. "success" on the aesthetic dimension requires that the information system elevate a user to a new and better environment and way of life. the more information systems professionals are able to consider this goal, and it can only be considered effectively thematically, that is actively, the more likely their systems will contribute to a better life for everyone. such considerations arise from a recognition of the importance of aesthetics, which itself arises from out of a deliberate organizational provision for the enabling circumstances of care and creativity; a deliberate provision for providing human organizational members with the capacity for thematic recognition. in our view, it is a necessary provision, because the relationship between a human being and information technology, in an organization, is always imminently aesthetic and therefore in that imminence, there is always the potential for evoking effective creativity. australasian journal of information systems volume 14 number 2 june 2007 78 references armour, p. (2002) “the spiritual life of projects”, communications of the acm, vol 45 no 1, pp11-14. banerjee, d. (1998) “modeling it ethics: a study of situational ethics”, mis quarterly, vol 22 no 1, pp 31-60. bansler, j.p., damsgaard, j., scheepers, r. havn, e. and thommesen, j. (2000) “corporate intranet implementation: managing emergent technologies and organizational practices”, journal of the association of information systems, vol. 1, paper 10. churchman, c. (1971) west. the design of inquiring systems, basic books. churchman, c.w. (1982) thought and wisdom intersystems publications, seaside, ca. churchman, w. (1994) “management science: science of managing and managing of science”, interfaces, vol 24 no 4, pp 99-110. churchman, c. west. (2001) “a philosophy for complexity, from managing complexity”, http://www.haas.berkley.edu/~gem/essays/complex.html. courtney, j. f. (2001) “decision making and knowledge management in inquiring organizations: toward a new decision-making paradigm for dss,” decision support systems, vol 31 no 1, pp 17-38. courtney, j.f., croasdell d.t. and paradice, d. b., (1997) “lockean inquiring organizations: guiding principles and design guidelines for learning organizations,” proceedings of the americas conference on information systems, indianapolis, indiana, august 15-17, pp 323325. courtney, j.f., croasdell d.t. and paradice, d. b., (1998) “inquiring organizations,” australian journal of information systems, vol 6 no 1, september, pp 3-15, and foundations of information systems: towards a philosophy of information technology,http://www.cba.uh.edu/~parks/fis/fisart.htm. coyne, r. (1995) designing information technology in the postmodern age. cambridge, ma: the mit press. dalal, n. (2002) “an integral perspective of information systems”, proceedings of the eighth americas conference on information systems, dallas, pp 1710-1713. danto, arthur c.(2001) “art, philosophy, and the philosophy of art”, http://www.csulb.edu/~jvancamp/361/361_r1.html. davison, r.m. (2000) “professional ethics in information systems: a personal perspective”, communications of the association of information systems, 3(8), pp 1-34. desanctis, g. (1984) “computer graphics as decision aids, directions for research”, decision sciences, vol 15 no 4, pp 463-487. donohoe, b., (2001) “an examination of moral action and aesthetic judgment in kant’s critical philosophy”, http://www.janushead.org/jhspg99/donohoe.cfm. el-shinnawy, m. and markus, m. l. (1997) “the poverty of media richness theory: explaining people’s choice of electronic mail vs. voice mail”, international journal of humancomputer studies, vol 6 no 4, pp 443-467. http://www.cba.uh.edu/~parks/fis/fisart.htm� http://www.csulb.edu/~jvancamp/361/361_r1.html� http://www.janushead.org/jhspg99/donohoe.cfm� australasian journal of information systems volume 14 number 2 june 2007 79 flanders, v. and willis, m. (1998) web pages that suck: learn good design by looking at bad design. haynes, john d. (1999) “practical and tacit knowing as a foundation of information systems,” australian journal of information systems, may, no 6, pp 57-64. haynes, john d. (2001) “churchman’s hegelian inquiring system and perspectival thinking,” information systems frontiers, vol 3 no 1, pp 29 – 39. haynes, john. d. and mahfouz, a. (2001) "objects that create their own context: the inclusion of conception in the mental model of the user in interface design," proceedings of the computer human interaction conference, ucol & massey university, palmerston north, new zealand, pp 39 – 43. heidegger, m. (2001) the question concerning technology, http://www.centenary.edu/~balexand/cyberculture/question1.html. iep (2001) the internet encyclopedia of philosophy, http://www.utm.edu/research/iep/. introna, l.d. (2002) “the (im)possibility of ethics in the information age,” information and organization, vol 12, pp 71-84. jarvenpaa, s. and desanctis, g. (1988) “standard scaling in decision graphs: an empirical investigation”, ieee transactions on systems, man, and cybernetics, vol 18 no 5, pp 824831. kienholz, alice (1999) “systems rethinking: an inquiring systems approach to the art and practice of the learning organization,” http://www.cba.uh.edu/~parks/fis/fisart.htm. kuhn, thomas. (1962) the structure of scientific revolutions. chicago: university of chicago press. leddy, t. (2001) “kant’s aesthetics: tattoos, architecture, and gender-bending”, http://aestheticsonline.org/ideas/leddy.html. markus, m. l. (1994) “electronic mail as the medium of managerial choice”, organization science, vol 5 no 4, pp 502-527. markus, m.l. and mao, j.-y. (2004) “participation in development and implementation – updating an old, tired concept for today’s is contexts”, journal of the association of information systems, vol 5 no 11-12, pp 514-544. mason, r.o.(1986) “four ethical issues of the information age”, mis quarterly, vol 10 no 1, pp 5-12. mata, f., fuerst, w.l., and barney, j. (1995) “information technology and sustained competitive advantage: a resource-based analysis,” mis quarterly, vol 19 no 4, december, pp 487506. mitroff, i. and denton, e. (1999) “a study of spirituality in the workplace”, sloan management review, summer, 83-92. mitroff, i. and linstone, h.a. (1993) the unbounded mind: breaking the chains of traditional business thinking, new york: oxford university press. paradice, d.b. (1990) "ethical attitudes of entry-level mis personnel." information and management, 18, pp 143-151. http://www.centenary.edu/~balexand/cyberculture/question1.html� http://www.utm.edu/research/iep/� http://www.cba.uh.edu/~parks/fis/fisart.htm� australasian journal of information systems volume 14 number 2 june 2007 80 paradice, d.b. and dejoie, r.m. (1991) “the ethical decision-making processes of information systems workers.” journal of business ethics, 10, pp 1-21. richardson, s. m., courtney, j.f., and paradice, d.b., (2001) “an assessment of the singerian approach to organizational learning: cases from academia and the utility industry,” special issue of information systems frontiers on philosophical reasoning in information systems research, vol 3 no 1, pp 49-62. sibley, f. (1959) “aesthetic concepts”, the philosophical review, 68(4), october, pp 421-450. singer, e.a. (1923) on the contented life. new york: holt & company. udas, k., fuerst, w.l., and paradice, d.b. (1996) “an investigation of ethical perceptions of public sector mis professionals.” journal of business ethics, 15(7), july, pp 721-734. van gerwen, rob. (1993) “kant’s regulative principle of aesthetic excellence: the ideal aesthetic experience”, presented at the annual conference of the british society of aesthetics, http://eserver.org/philosophy/aesthetic-excellence.txt, london, september. wells, john d. (1996) “postmodernism and information technology: philosophical perspectives and pragmatic implications,” proceedings of second americas conference on information systems, phoenix, pp 602-4. wells, john d. and fuerst, william l. (2000) “domain-oriented interface metaphors: designing web interfaces for effective customer interaction,” proceedings of the thirty-third hawaii international conference on system sciences. wells, john d. and fuerst, william l. (2001a) “domain-oriented interface metaphors: an alternative approach to customer interface design,” washington state university, working paper. wells, john d. and fuerst, william l. (2001b)“deriving interface metaphors from the business domain: an empirical investigation,” washington state university, working paper. wilber, k. (1998) the marriage of sense and soul, new york: random house. yanal, robert j., (2001) “pepper on definitions and aesthetics”, http://www.sunyit.edu/~harrell/pepper/pep_yanal.htm. http://eserver.org/philosophy/aesthetic-excellence.txt� http://www.sunyit.edu/~harrell/pepper/pep_yanal.htm� editorial board thomson scientific (isi) impact factor. using existing ranking lists polling eminent scholars stage one: identification of existing rankings stage two: initial ranking list drawn up stage three: feedback on the initial ranking list stage four: information and finalisation of ranking list level a journals level b journals premier professional journals abstract introduction the pending australian research quality framework (rqf) methodology sourcing and procuring the data examining the data for correctness results rq1 what journal outlets are used by australian academics to publish is-related research? rq2 which journals have been most preferred by australian academics for article publication as measured by the total number of articles published in each journal? conclusion limitations and future research socio-pragmatic constructionism social constructionism pragmatism socio-pragmatic constructionism representation simplification pragmatic orientation syntax semantics pragmatics implications for practice implications for theory passive dimension (perceptions); active dimension (conceptions) the aesthetic dimension is not characterized as “plug and play”. the aesthetic dimension is not described by graphical user interfaces. aesthetics occurs when the system elevates the work to a new level of effective vision. aesthetics may occur on different levels or from different perspectives. 1. rational goal seeking (intelligence(design(choice) 2. happenstance rationalization (design(choice(intelligence) 3. existential introversion (choice(intelligence(design) 4. deferred design (intelligence(choice(design) 5. situational sensemaking (design(intelligence(choice) 6. existential heroism (choice(design(intelligence) constructs output performance outsourcing experience contract establishment the feedback effect business strategy it strategy firm-wide active involvement long term focus meeting of the minds clarity and consistency management skill and capability alignment facilitation processes organisational structure organisational culture communication it as an organisational tool organisation background information factors that promote alignment people factors process factors organisational factors high level factor comparisons alignment of business & information technology strategy a cross industry australian study part 1: background part 2: business strategy part 3: information technology strategy part 4: factors that promote the alignment between business and information technology strategy documentation interviews researcher bias microsoft word 13 1 marina.doc ajis vol 13, no. 1 september 2005 139 a model for an electronic information marketplace wei ge school of business administration university of wisconsin – milwaukee p.o. box 742 milwaukee, wi 53201 usa telephone: +1-847-894-7891 e-mail: weige@uwm.edu marcus a. rothenberger department of mis university of nevada las vegas 4505 maryland pkwy. las vegas, nv 89154-6034 usa telephone: +1-702-895-2890 e-mail: marcus.rothenberger@ccmail.nevada.edu edward chen spessart str. 18 65812 bad soden germany telephone: +49-171-4911-356 e-mail: edchen68@hotmail.com abstract as the information content on the internet increases, the task of locating desired information and assessing its quality becomes increasingly difficult. this development causes users to be more willing to pay for information that is focused on specific issues, verifiable, and available upon request. thus, the nature of the internet opens up the opportunity for information trading. in this context, the internet cannot only be used to close the transaction, but also to deliver the product desired information to the user. early attempts to implement such business models have fallen short of expectations. in this paper, we discuss the limitations of such practices and present a modified business model for information trading, which uses a reverse auction approach together with a multiple-buyer price discovery process. introduction information is being published on the internet from countless information sources in such unprecedented speed that people can find almost anything online. as the amount of available information and the number of sources increase, however, efficient retrieval and quality assessment become more and more difficult. although most information on the internet is available for free, locating it requires substantial effort – sometimes more than what a single information seeker is willing to invest. at the same time, it is equally difficult for an individual information-seeker to assess the credibility and true value of the accessed information. the need for efficient and reliable information retrieval, therefore, may constitute an opportunity for customized information trading. such a service can build on the existing notion of information intermediaries, which are “e-commerce companies leveraging the internet to unite buyers and suppliers in a single, efficient virtual marketplace to facilitate the consummation of a transaction” (grover and teng 2001). ajis vol 13, no. 1 september 2005 140 the literature has long supported the notion of information intermediaries in an electronic market environment (bakos 1998, benjamin and wigand 1995, grover and teng 2001, lenz et al. 2002, mahadevan 2000, rose 1999). even though the nature of the internet lends itself to distributing information as a virtual commodity, most information intermediaries reported in the literature are restricted to tangible goods and traditional services such as the sale of books, airlines tickets, and hotel reservations (grover and teng 2001, luking-reiley 2000, rohm and pernul 2000). in addition, most intermediaries treating information as a virtual commodity, such as search engines and product catalogue aggregators, provide only storage and dissemination of meta-information; they don’t provide a means to customize information to the individual information seeker’s requirements (mahadevan 2000). the revenues of this line of business, therefore, are primarily generated from advertising or membership/subscription fees, but not from information trading itself. the few exceptions include web service marketplaces (yarom et al. 2004) and information marketplaces which will be discussed later in this paper; these business models generate their revenue streams from the sale of the virtual commodities. with the economy changing from a focus on the production of physical goods to communication (schmid 2001), business models involving for-profit dissemination of virtual commodities may continue to gain importance in the future. although most information is available for free if retrieved using search engines, the search cost and credibility of non-rated sources may be so high that an information-seeker would rather pay for quality-rated customized information about a specific subject domain. grover and teng (2001) classified information intermediaries according to their relationship with buyers and suppliers. today’s widespread search engines maintain an open relationship to both groups. as information intermediaries compete on their ability to deliver information and provide other value-added services to buyers, they would ultimately evolve towards greater buyer affiliation (grover and teng 2001). hence, a new form of information intermediary that provides a mechanism allowing trading of customized information in a similar manner as tangible goods may be a step in this direction. information market the online facilitation of information trading cannot be a simple copy of an existing business model developed for tangible goods. the unique cost structure of information requires a different approach to trading. the economics of information production is driven by the electronic dissemination of information goods. this means that although the production of the first copy of the information may be expensive, the cost for the creation of additional copies tends to approximate zero (rose 1999, shapiro and varian 1998). in the context of producing customized information targeted to the information seeker’s requirements, an information supplier may spend a substantial amount of time searching, acquiring, and customizing the information, but once he or she has produced the answer, the cost of selling another piece of the same information is close to zero. nevertheless, the timeliness of information constrains the potential resale value. depending on the type of information, its value may diminish rapidly if it is out of date. qualified information producers who can create quality information in a specialized knowledge domain are often incurring the highest cost (e.g., lawyers, physicians, etc.). the fixed cost for producing this type of information may be more than an individual information consumer is willing to pay. a model must take advantage of the low variable cost of reproducing information to find multiple buyers for high-priced information. ajis vol 13, no. 1 september 2005 141 meanwhile, the credibility of anonymous information suppliers in a specialized knowledge domain is crucial to information-seekers. there is a need, thus, to investigate how the distinct properties of information can be translated into a suitable business model. in this article, we first discuss the few existing information intermediaries that represent earlier attempts to implement variations of such a business model. we outline their weaknesses and the possible pitfalls of trading information on-line. we will then propose a business model that addresses these issues. existing information intermediaries over the past years there have been several attempts to establish electronic markets for customized information trading. table 1 summarizes existing business models. askjeeves.com and freeadvice.com are advertisement-based information markets. the revenue for these web sites comes from advertisement, rather than from fees for providing information. both web sites are subject to two major limitations. first, for an information seeker who is looking for quality answers, the credibility of anonymous information sources is a major concern, which becomes even more critical in a specialized knowledge domain. second, because of a lack of incentive for information suppliers to provide quality information, many posted questions remain unanswered (over 50 percent of total posted questions). table 1: existing intermediaries for custom information google answers this service allows users to post custom questions on a virtual board together with the price the user is willing to pay for the answer. everyone can access the board and read posted questions and answers. a number of pre-screened experts can reply to the question if they feel that the compensation offered meets their needs. google answers recommends setting the price according to the complexity of the question. if the price offered is too low considering the time it takes to answer the question, chances are, none of the experts is willing to answer the request. thus, there is an incentive to offer the compensation that is appropriate for the question. the web site charges a us$0.50 non-refundable listing fee and retains 25% of the compensation paid to the expert. in addition to the promised compensation, users can tip the expert who answered the question; the web site does not retain a percentage of the tip. users get a refund if they are not satisfied with the answer; they also rate the expert. several poor ratings may cause an expert to be excluded from the service. ajis vol 13, no. 1 september 2005 142 inforocket.com the service allows users to post questions about various subjects and then matches these with users who can provide answers. the information seeker names a price he or she is willing to pay for the answer and the information suppliers bid to offer answers. based on ratings and selfreported expertise, the information seeker selects the most convincing information supplier to answer the question. the web site charges a percentage of the transaction price for the service. in addition, an information store is available for instant purchase of answers generated in previous transactions. we are arguing that inforocket’s business model was not sufficiently tailored to the economic properties of information. in deed, the web site discontinued its service in october 2001. it introduced a new business model using the new name liveadvice.com. liveadvice.com the website allows users to search for experts in various areas. the experts are available for advice through traditional phone services – offering recordings and live conversations, both for a fee. the web service only supports the search for the experts’ phone numbers and facilitates a rating forum; it does not deliver the information. the web site operator receives a share of the revenues generated through the phone service. since 2003, three separate web sites are covering distinct areas based on the same business model: keen.com for astrology and psychics, ingenio.com for professional advice, and niteflirt.com for social/entertainment contacts. by 2004, keen.com, ingenio.com and niteflirt.com have abandoned their information trading approach. the urls are now used for other services: keen.com is a psychic web site, ingenio.com is an advertising network, and niteflirt.com is a traditional dating site. askjeeves.com the website is a popular search engine that provides an additional service named “ask other people”, where users can post questions about various subjects in order to receive free answers from experts. this web site differs from the business model we are proposing, as revenues do not come from the q&a forum, but through advertising, as it is the case with most search engines. freeadvice.com the website hosts the ask a question on law forum providing a place for users to obtain answers to legal questions. the purpose of this web site is to get users to hire traditional lawyers. the forum provides simple answers to attract people to the web site. for more detailed legal help, lawyer contact information is provided by region. also, users can post a legal case to a lawyer that will be selected by the web site provider. the source of revenue is the advertisement of legal services. inforocket.com was one of the few pay-per-answer services; however, the model did not succeed. after several attempts to refocus the concept, all succeeding web-sites have abandoned their information trading approach and have reverted to more traditional business models (see table 1). based on a survey of inforocket.com during its operation, we found that approximately 400 questions were posted per day and most requests paid about us$10 with few items up to us$75. although nielsen (2003) predicts that users will be willing to pay for quality content on the internet in the near future, at present it seems that it is taken for granted that most information on the internet is free. the lack of ajis vol 13, no. 1 september 2005 143 willingness to pay a higher price for an answer ultimately affects the quality of the information provided. google answers is a new commission-based information trading service that started in 2005. google answers does not allow for information resale, as it makes past questions and answers freely available. thus, the user originally posting the question must provide the entire compensation for the expert that answers a question, making quality information expensive. low-priced questions can hardly attract well-paid professionals with a specialized expertise, such as lawyers or physicians, to spend their time providing an answer. the concern about the quality of answers, consequently, affects the willingness of information seekers to pay for answers and will lead to the relatively small volume of transactions. although a working resale market for information is desirable in order to maximize compensation for the expert, yet limiting the cost for the individual answer seeker, the inforocket.com model failed to address that most information is time-sensitive and hence information quality (accurateness, timeliness, relevancy, etc.) changes over time. answers from previous transactions cannot be updated to keep the information current, thus limiting the potential resale. figure 1 summarizes the inforocket.com pay-per-answer information-trading model. we argue that there is potential for an improved business model that addresses above issues. a variation of a reverse auction model can be employed in the context of information trading. the nature of individual consumer-oriented marketplaces for information trading implies that the information needs of a particular group of information seekers are specific. hence, information suppliers must know information seekers’ requirements before they can provide answers. the value added by information suppliers largely comes from customizing information for a particular group of information seekers. however, there are also some concerns about the reverse auction model. for example, well over 80% of the product requests were never fulfilled on priceline.com because the company was not able to find a willing supplier (mcclellan 2000). one important implication in priceline’s case is that buyers tend to set the desired prices too low, thus decreasing the chances to get the item requested. therefore, as we investigate any business models that promise consumer empowerment through a reverse auction, we need to realize that they inevitably involve the risk of a low bid rate. ajis vol 13, no. 1 september 2005 144 figure 1: model of a commission-based information intermediary (inforocket.com) aggregate buying as mentioned above, the questions posted on the inforocket web site were priced mostly around us$10. clearly, the information suppliers with specialized knowledge have little incentive to supply information at such low rates. rafaeli and raban (2003) have shown that information sellers typically require a higher price to provide the information, than information buyers are willing to pay; they argue that information trading models must take this into account. thus, an information intermediary facilitating information trading between individual consumers, must ensure that the price for information attracts individuals or companies with professional knowledge in order to generate a stable revenue stream. based on the distinctive cost structure of information, the first copy is expensive, while subsequent copies are cheap (rose 1999, shapiro and varian 1998), there are opportunities to aggregate information seekers and thus minimize a single buyer’s cost while increasing the total price of an answer. this works well if the nature of the information is specialized, yet of interest to a group of people (e.g., immigration information relevant to foreigners with a particular visa status in a certain country [provided by a lawyer], information on studies conducted on the effectiveness of a particular medication [provided by a physician], etc.). as information in specialized domains may be of interest to more than one person, such an approach can solve the dilemma between the cost incurred to provide the information and the individual buyers’ spending limits. currently, neither of the two payper-answer business models discussed in this paper (inforocket and google answers) incorporate a mechanism that allows multiple information-seekers to share the cost for an answer. informati on store informati on suppliers ’ rating informati on seekers’ profile information seeker 1 information seeker 2 information seeker m information supplier 1 information supplier 2 information supplier n answ er market information intermediary question 1 question 2 question m answ er answer pool m ajis vol 13, no. 1 september 2005 145 we propose that an information seeker can either join an ongoing question or post a new question. an information seeker must provide the maximum amount he or she is willing to pay for the answer, the time window, as well as the minimum acceptable rating of information suppliers. by allowing information-seekers to set the price individually, we address the need for price discrimination for the sale of information goods. shapiro and varian (1998) propose to facilitate such price discrimination by offering different versions of the information goods at different price-levels. the ability of the information-seeker in our model to choose the price he or she is willing to pay, before the answer is provided, allows for the implementation of such price discrimination without the need for different versions of the information good. the information-seeker faces a trade-off between offering a low price and being able to obtain the information quickly: the more he or she offers, the sooner the price required for the transaction will be met, as fewer information-seekers must join to meet the price. thus, an information-seeker has an incentive to offer an amount that is close to the value that the answer represents to him or her, in order to obtain the information without unnecessary delay. figure 2: dynamic information pricing time t2 t1 price n of info-seekers bid p1 p2 infosupplier 1 infosupplier 2 ajis vol 13, no. 1 september 2005 146 the model is a dynamic pricing system; the final price of an answer is determined by the aggregated price and the minimum requirements for the information suppliers’ rating, as specified by all information seekers joining a particular question, the rating, and professional qualification records of available suppliers. when the aggregated price of all information seekers who signed up for a particular question reaches the offer price posted by an information supplier, and if the same information supplier also meets the minimum rating and professional qualification required by the information seekers, the auction closes successfully (figure 2). if more than one information supplier meets the requirements (both the bid price and the rating) at the same time, then the one with the highest rating wins the bid. information seekers joining a particular question are required to pay the previously specified amounts to the supplier. as demonstrated by sites like ebay, and amazon, the rating system has proved to be an effective way to build trust in a quasi-open market (luking-reiley 2000). however, sometimes serious information seekers looking for high quality answers in some specialized domains (such as legal advice) will be putting more stress on the qualifications and reputation of the information suppliers in those particular domains. if this is the case, a rating system alone may not satisfy all the criteria of those information seekers. thus, in addition to the widely applied rating system in most c2c auction models, the proposed model requires each information supplier in a specialized domain to provide his or her professional background and sometimes a certification from a third party to prove his or her qualifications in a particular specialized domain in order to enter in a bid for supplying information. the more people sign up for a particular question, the higher will be the total price that is paid for an answer. the cost for obtaining an answer to the individual information seeker is lowered, while the total revenue for the information supplier is maximized through the aggregation of buyers. information seekers, information suppliers as well as the information intermediary are all winners. an important implication of the model is that it generates a strong incentive for information suppliers with a professional knowledge base to offer high-priced information. more high-priced quality information such as legal advice, health care information, etc., may generate larger volumes of transactions and hence higher revenue for information intermediaries. figure 3 summarizes the aggregate buying model. figure 3: aggregate information buying model … information seeker group aggregated price i’ information seeker i information supplier 1 information supplier 2 information supplier n information seeker 1 information seeker m questio n i ajis vol 13, no. 1 september 2005 147 information store the distinctive cost structure of information also enables the information supplier to generate additional revenue by reselling information that has been generated through previous transactions. existing information trading models have implemented online stores to allow experts to resell answers they have created in earlier transactions; information seekers can then purchase such information instantly. however, existing models assume that information can be resold with little or without change and do not account for the information quality (accurateness, timeliness, relevancy, etc.) decreasing over time, rendering most information obsolete or outdated within a short time span. thus, we propose that information suppliers must be able to maintain and modify their answers in the information store. information suppliers, therefore, will have a strong incentive to update the answers in the information store as updates increase the likelihood of reselling the information. information requests usually are specific and unique to a particular group of information seekers. this may imply a low degree of reusability of the information gathered corresponding to some specific questions. however, the same question can be asked in a more general way to increase its resale potential. for example, an information seeker searching for information on immigration information to the united states can phrase her or his questions in different ways: • “i have applied for an adjustment of status from f1 (student) to h2b (temporary skilled worker). can i leave the country while the application is being processed? what documentation will i need to reenter the us?”, or • “can an alien living in the us leave the country and reenter, while the immigration and naturalization service is processing an adjustment of status? does it depend on the visa status? if yes, please specify the requirements that must be met to travel outside the us during the adjustment of status process.” the answer to the second question would apply to more potential information seekers than the answer to the first. the second answer may include information on various visa statuses, thus applying to any alien who is changing status. the first question can only attract information seekers that are changing from f1 to h1b status, because it has been phrased in an unnecessarily constraining way. if an information seeker desires to attract other information seekers for aggregate buying, he or she will post the question in a general fashion. however, if aggregate buying is not in the intention of the information seeker, another incentive is needed to encourage posting a question in a general fashion with a high resale potential. it is in the interest of the information intermediary to maximize the resale likelihood of every posting. therefore, our model pays a percentage of the resale revenue not only to the information supplier, but also to the information seeker who initially posted the question. this stream of revenue can offer an incentive for information seekers to post more valuable questions, making resale more likely. ajis vol 13, no. 1 september 2005 148 conclusion we have discussed the potential of an electronic information marketplace. the review of existing business models has shown that there are several shortcomings in the current practice. we have argued that an information-trading model that incorporates the distinctive cost structure of information can offer a viable alternative to the existing information intermediary strategies that are mostly based on advertising revenue. through providing the services of aggregate buying and an information store, the model exploits the potential opportunities created by the high fixed and low variable cost of creating information. while acknowledging the uniqueness of some specific information requests, we have provided concrete examples for specialized information that is of interest to specific groups of users. it is this type of small-scale reusable information for which the proposed model would work best. we are not arguing that the model is a silver bullet for any type of information trading on the web. however, we are arguing that there is a need to expand the existing set of information intermediary models to include pay-per-answer services for specialized subject domains. although initial attempts at commission-based information intermediaries existed (e.g., inforocket.com), these early models did not prevail. we have enhanced the existing notion of information trading by linking the unique properties of information to the requirements of such a business model. the authors believe that the model presented in this article may prompt the web community to take a second look at commission-based information trading. google has recently taken a step in this direction; however, its new google answers service neither incorporates an information store nor the concept of aggregate buying. references: bakos, y. (1998) “the emerging role of electronic marketplaces on the internet”, communications of the acm, vol 41 no 8, pp. 35-42. benjamin, r. and wigand, r. (1995) “electronic markets and virtual value chains on the information superhighway”, sloan management review, vol 36 no 2, pp. 62-72. figueiredo, j. m. de. (2000) “finding sustainable profitability in electronic commerce”, sloan management review, vol 41 no 4, pp. 41-52. grover, v. and teng, j.t.c. (2001) “e-commerce and the information market.”, communications of the acm, vol 44 no 4, pp. 79-86. lenz, m., zimmermann, h.-d., heitmann, m. (2002) “strategic partnerships and competitiveness of business-to-business e-marketplaces: preliminary evidence from europe”, electronic markets, vol 12 no 2, pp. 100-111. luking-reiley, d. (2000) “auctions on the internet: what’s being auctioned, and how”, the journal of industrial economics, vol 48 no 3, pp. 227-252. mahadevan, b. (2000) “business models for internet-based e-commerce: an anatomy”, california management review, vol 42 no 4, pp. 55-69. mcclellan, j. (2000) “browsing for buyer power. they are auctions with a twist: you name your price and sellers try to match it.” the guardian, 16 march. nielsen, j. (2003) “the case for micropayments”, http://www.useit.com/alertbox/980125.html, accessed november 2003. rafaeli, s. and raban, d.r. (2003) “experimental investigation of the subjective value of information trading”, journal of the association for information systems, vol 4, pp. 119-139. ajis vol 13, no. 1 september 2005 149 rohm, a.w. and pernul, g. (2000) “cops: a model and infrastructure for secure and fair electronic markets.” decision support systems, vol 29, pp. 343-355. rose, f. (1999) the economics, concept, and design of information intermediaries. physika-verlag, heidelberg and new york. schmid, b. (2001) “what is new about the digital economy?”, electronic markets, vol 11 no 1, pp. 44-51. shapiro, c. and varian, h.r. (1998) “versioning: the smart way to sell information.” harvard business review, november-december, pp. 106-114. yarom, i., topol, z., and rosenschein, j.s. (2004) “decentralized marketplaces and web serivices: challenges and opportunities”, workshop on agent mediated electronic commerce vi. microsoft word 13 1 marina.doc ajis vol 13, no. 1 september 2005 176 issues in it service-oriented requirements engineering sharman lichtenstein sharman.lichtenstein@deakin.edu.au lemai nguyen lemai.nguyen@deakin.edu.au alexia hunter alexia_h@bigpond.net.au school of information systems, faculty of business and law deakin university, 221 burwood highway, burwood, australia 3125 abstract corporate information technology (it) management is increasingly service-oriented, offering continuous evaluation and improvement of application, communication, delivery and support services to internal and external customers. service-oriented requirements engineering (sore) plays a significant role in identifying and specifying service requirements, formally defined through service-level agreements (slas). however, the new frameworks and approaches emerging to guide these developments have not yet addressed how requirements for such services can be effectively developed, nor identified the diverse issues involved. we report a case study of a web services team development of requirements for an internal service desk service. the study revealed five main issues of concern when developing service provider requirements: service roles, responsibilities and accountability; service performance metrics; resolution of conflicting stakeholder service requirements; customer acceptance of service change; and service provider team structure. this study suggests that in the new it services era, new techniques and approaches are needed for eliciting and determining provider and customer requirements that involve key stakeholder groups equitably and more closely negotiate the sometimes-conflicting provider and customer needs. keywords: it service management, it services, web services, socio-technical, requirements engineering acknowledgement: this paper was presented at 9 th australian workshop on requirements engineering, awre’04, 6-7 december, adelaide, australia. introduction in an increasingly service-oriented economy, corporate information technology (it) management is evolving towards service-based models that offer continuous evaluation and improvement of application, communication, delivery and support services to internal and external customers (mayerl et al., 2000; niessink & vliet, 1998; peppard, 2003; steen et al., 2005). forecasts for it services growth are strong in the us (idc, 2004) while slower but still positive in australia (hayes, 2004). given the demand for improved it services, there have been calls to identify approaches to developing it service requirements, as the functionality previously offered by application software is now re-located within infrastructure (van eck & wieringa, 2003; 2004). toward that end, this paper aims to explore issues of concern for service providers attempting to develop it service-oriented requirements. requirements engineering (re) for it services is located within the wider context of it service management frameworks such as it infrastructure library (itil) and strategies that encompass people, processes, knowledge and technology within a business context (rands, ajis vol 13, no. 1 september 2005 177 1992, steen et al., 2005). information systems strategies can also support the new it service paradigm by aligning it with business and reflecting a service focus (edvardsson et al., 2000). it is within such service-focused frameworks and strategies that service requirements must be identified, and defined as service-level agreements (slas) and the semi-automated delivery of component-based application systems (layzell et al., 2000; mcbride, 2002). while researchers have developed various service-oriented models and techniques to guide the development and operation of it services (e.g. budgen et al., 2004; garschhammer et al., 2001; niessink & van vliet, 1998; trienekens et al., 2004), very little attention has been given to investigating the diverse issues involved in it service requirements analysis. illustrating the need for such research, trienekens et al. (2004) noted recently: “there still exist many unsolved questions in the service management area, eg regarding the definition of services and the specification of slas ” (p. 44). three rationale highlight the importance of an investigation of the issues involved in service-oriented re. first, from more than 600 uk companies deploying it services only 38 per cent had established performance metric requirements or a timeframe for future performance evaluation (barker, 2004). second, according to trienekens and colleagues, service specifications can often be unclear, incomplete, negative or static (trienekens et al., 2004)1. third, as has traditionally been the case for systems requirements analysis, there may be significant as-yet-unexplored socio-technical issues experienced in negotiating conflicting customer and provider service requirements (trienekens et al., 2004). interestingly, van eck and wieringa (2003) suggest that two new types of re will emerge to complement the it service movement: re for service consumers (customers) and re for service providers (providers), while trienekens and colleagues (2004) point out the need for consensus between provider and customer in the sla specification (trienekens et al., 2004). clearly, issues involved in specifying and negotiating service oriented requirements need to be explored in greater detail. underlining an emerging interest by the re community in the it services sector, a workshop on service-oriented requirements engineering sore’04 was recently held for the first time (baresi, 2004). in this paper, we report findings from a case study of a web services team engaged in developing requirements for a new service for an internal service desk. we contribute to the emerging area of service-oriented re by providing new understandings of the issues involved in specifying and negotiating it service-oriented requirements. the paper is structured as follows. we first provide a backdrop to the empirical findings by reviewing some of the literature in it service management. we then review existing research in service-oriented re, suggesting that additional research is needed, and provide a socio-technical framework of service-oriented re. the case study methodology employed is then briefly described, and the empirical results presented. implications for theory and practice are discussed and conclusions drawn. theoretical foundations in this section, we first review it service management and demonstrate that new approaches to developing and managing services may be needed. second, by reviewing representative approaches to the elicitation and determination of provider and customer service requirements, we highlight that this is an important but under-researched area. ajis vol 13, no. 1 september 2005 178 it service management and guidelines it service management is aimed at improving the quality of it services and the alignment of these services with customer needs (van bon, 2002). it exploits existing software by offering new improved services such as performance and availability support, customer and help desk support, and education and maintenance (trienekens et al., 2004). experts are still deliberating the precise nature of an it service. we embrace the following simplified definition: “an it service is defined as a set of related functions provided by it systems to support one or more business areas” (mingay et al., 1998). it service management follows an “internal vendor of services” model in which there is an increased reliance on slas for negotiating specific service requirements and associated budgets. the importance of slas is far-reaching, as they are later used to monitor whether a service is satisfactorily performed by checking aggregated performance statistics obtained from tapping web server logs, process time and other information against service performance rules in the sla (sailer et al., 2004). currently, in order to support it service management, a number of service-oriented frameworks and themes have been developed. however, these guidelines lack some of the key principles, phases, and steps available in traditional systems development frameworks. in the itil framework, principles of project management are absent (servicetalk, 2004) while the softer issues such as organisational culture (mcbride in servicetalk, 2004) and customer involvement are also missing. recently, steen and colleagues emphasised the need for interdependency between layers of environment, business, applications and technology in it service architecture, highlighting the potential influence of socio-technical issues on it services (steen et al., 2005). due to paper size constraint, it service management is not further discussed here; however, it will be the subject of a future publication. a further poorly addressed area in existing guidelines is the elicitation and determination of provider and customer service requirements. this is important especially argued also by bohmann et al. (2003) as one of the core features of applications hosting services is heterogeneous customer requirements. this topic will be discussed in more detail in the next section. requirements engineering for it services in this section, we review key literature addressing the development of requirements for improved it services. van eck and wieringa (2003) assert that re enables the specification of not only the desired it functionality but also the services to be provided. as mentioned earlier, these authors suggest that slas play a central role in both types of re: re for service customers and re for service providers. according to the researchers, customers will focus on identifying the tasks that need support, whereas providers will focus on achieving economies of scale such as offering a new service for multiple customers, or applying a specialised skill to a common problem. this means that providers will likely prefer to offer a standardised service which will not suit all customers. accordingly, customers may have to adjust their processes to suit the services offered. trienekens et al. (2004, p. 46) suggest the need for service requirements negotiation, in that the requirements determined for the sla must address not only customer needs, but provider needs as well: “in software management science, the importance of a consensus based approach between stakeholders (e.g. customers) and developers is emphasized ajis vol 13, no. 1 september 2005 179 (boehm and ross, 1989), in particular regarding requirements determination. these concepts, such as ‘stakeholder win-win’, are applicable to sla specification, in particular to improve the usage of an sla to serve as a communication aid and a conflict prevention instrument.” van eck and wieringa (2003) point out that while requirements engineers may well ask customers what services are needed, they may also have to assist the customer in adjusting their business processes to services already provided (by providers) or to more general services suiting a wider customer base. bohmann and colleagues highlight the key dilemma faced by the service provider as “while striving for economies of scale in service operations through standardization of services, from a customer’s point of view, the service provider has to offer services which represent individual solutions that are customized to customer needs” (bohmann et al., 2003). in recent years, researchers have begun to develop techniques that could be employed by requirements engineers to identify service requirements specified in slas. we review four representative examples to illustrate the state of research in this emerging area for re (bohmann et al., 2003; van eck & wieringa, 2003, 2004; niessink & van viet, 1998; trienekens et al., 2004): • bohmann et al. (2003) argue that one of the key features of applications hosting services is the heterogeneity in customer requirements for example, different computing platforms and communication network, the integration of different business applications and sophisticated business characteristics. the researchers introduce modular service architectures to provide standardised yet customisable it services. the key objective is to assist service providers to address heterogeneity in customer requirements through matching and mapping required service features and factors during re and design phases. • van eck and wieringa (2003) propose a “service blueprint” requiring the specification of key issues such as customer self-service tasks, service delivery events, generic and customer-specific services, and visibilities of sub-processes. these design decisions influence important choices such as the degree of support provided by the service provider. in a recent paper, the authors further suggest that re might comprise the definition of required services in terms of inputs and outputs, as well as the workflow design needed to deliver the services (van eck & wieringa, 2004). a common shortcoming in the above two approaches is their primary focus on customer requirements, while paying comparatively little heed to the very real needs of providers, who will also be affected by the new service as they may have operational roles. • trienekens et al. (2004) provide a phased process for specifying a sla. in the first phase, introductions take place between an sla expert, service provider, contract expert and customer representative. in the second phase, the customer service needs are identified. in the third phase, services are defined, qualified and metrified. during this phase there is collaboration between provider and customer during which their separate service needs are negotiated. in the fourth phase, the contract is implemented. trienekens et al. (2004) also provide a model termed the service management lemniscate which advocates an iterative cycle ajis vol 13, no. 1 september 2005 180 between (i) service level management that accounts for business needs and leads to defining the sla, and (ii) service process management which involves design, implementation and monitoring. the researchers conceive it service as under continual improvement. • niessink and van vliet (1998) develop it service cmm as a maturity growth model – similar to the software cmm – focusing on service process assessment and improvement. they also discuss ways to identify changed service requirements, such as analysing logged incidents, and negotiating customer requests for changed service levels or new services. importantly, while this early maturity growth model and trienekens et al.’s (2004) approach suggest involving both customer and provider requirements, they fail to explore this aspect in sufficient detail. furthermore, from the perspective of an effective agreement, the negotiation of the service agreement is a critical step, and thus greater guidance is needed as to how this can be achieved. a key weakness in the above four approaches to service-oriented re is the absence of an in-depth consideration of the socio-technical issues involved. in traditional re, macaulay (1996) identified poor communication between stakeholders as the key factor limiting or enabling effective re. sutton (2000) and walz et al. (1993) narrowed communication factors to three key issues: articulation, misunderstanding and conflict. as technology and systems are embedded within socio-organisational contexts and processes, strongly sociotechnical approaches to (traditional) re are required (see for example, mavin & maiden, 2003; kotonya & sommerville, 1998, goguen, 1994, 1997; nuseibeh & easterbrook, 2000). if we extrapolate and apply the above socio-technical concepts to service-oriented re, we would expect significant stakeholder involvement in the re process, which should develop realistic customer expectations about provider and service capabilities and needs, provide bargaining and conflict resolution between customer and provider about design issues, reduce both customer and provider resistance to change, and commit both customers and providers to the new services. according to steen and colleagues, “a service description and a protocol of collaboration and negotiation are the only requirements for shared understanding between a service provider and a service user” (steen et al., 2005). we suggest there are likely to be socio-technical issues which arise in service provision which must be managed. these issues will be highlighted in the case study described later. ajis vol 13, no. 1 september 2005 181 emerging from the literature review presented in this section is a theoretical framework to understand and study service-oriented re (figure 1). within this framework, the re process aims at producing a specification of it service requirements (in practice, the sla, however we have provided an abstraction). this process involves addressing both customer and provider requirements. it is guided by it service management principles (as briefly introduced in section 2.1) and is embedded within a socio-technical environment. both customers and providers influence, and are influenced by, the socio-technical issues emerging in such an environment. in the context of this framework, our study explores two key questions: what issues do service providers experience eliciting and determining it service requirements, and how and why do these issues shape the requirements specification? this paper explores these questions by means of a case study, introduced in the next section. research method as part of an extensive project investigating the use of web and other technologies to support knowledge sharing (other findings are reported in hunter, 2003; lichtenstein et al., 2004; lichtenstein & hunter, 2005), we conducted a case study based on a web services team in the australian headquarters of a large multinational it company – glotech (real name withheld). a single case was deemed useful to explore the issues because the area of service-oriented re is new and relatively unexplored, and a single case can yield revelatory results (galliers, 1992). for this paper, we focused on an analysis of issues occurring in the web services team’s development of requirements for a planned itil-based web service – an adaptation of service desk. service desk solutions provide detailed business service definitions as well as a central repository to store all relations between customers, business services, slas and support level objectives. the planned service desk initiative at glotech was aimed at improving the current processes of internal customer publication to the corporate intranet – processes that, at the time, were enabled by e-mail messages between internal customers and the web services team. while glotech had supplied technologies to customers for it service management requirements engineering socio-technical issues customers providers it service requirement figure 1. service-oriented requirements engineering ajis vol 13, no. 1 september 2005 182 over a decade, the company did not yet have a procedure or approach for improving it services or developing it service requirements. the team comprised information systems and it university graduates. thus, the case study was of particular interest to our study in that it could potentially provide insights about the types of issues faced by service providers in attempting to identify it service requirements. the primary data collected comprised the transcript from an audio-taped two hour web services team meeting. during this re meeting, the team – in the role of the service provider – collectively drafted the requirements for the new service desk service. substantial secondary data were also collected from ten semi-structured audio-taped one hour interviews with web services team members, the corporate systems administrator and the marketing intranet manager who was responsible for authoring and submitting marketing intranet content. the questions included an examination of the issues involved in intranet publication processes. in the interview with the marketing manager, her concerns regarding this process were explored in depth, providing us with considerable knowledge of the marketing customer’s new service requirements. in the other interviews, many sociotechnical issues surrounding the existing intranet publication process were explored, providing additional contextual data used to enhance the primary data collected from the re meeting. meeting and interview data were supported by observations and document collection. coded categories and concepts discovered in the re meeting transcript were inductively developed (mayring, 2000) and cross-checked via a content analysis of the interview transcripts that focused on identifying issues relating to the services of internal intranet publication. additional insights gained from observations and documents were used for additional validation and enhancement of the themes identified. in the next section, we present the findings from the study. interpretation of research findings the web services team comprised eight fairly young it developers, three of whom were part-time contractors for strategic sourcing reasons. the team was headed by a full-time non-technical manager but was largely self-managing, with the manager remaining in touch with projects through weekly catch-up sessions with individual team members. the web services manager reported to the australian it operations manager. the team’s main role comprised developing, performing and maintaining web services needed by the australian subsidiary, including a range of intranets authored by different groups. existing web services intranet publication service at the time of study, the intranet publication service provided by the web services team for internal customers was co-ordinated through email messages and attachments exchanged between members of the web services team and the customers (although a few technical groups published directly using publishing software). the main processes currently performed by the intranet publication services comprised: • initial customer email request to intranet coordinator for content publication; • assignment by intranet coordinator of the entire publication task to one of the part-time contractor web services team members responsible for intranet publication duties; ajis vol 13, no. 1 september 2005 183 • internal review of content request; • preparation of publication proofs; • review and approval of proofs by customer (content provider); • live publication/update of intranet content. a new it service initiative had originated from a request by the marketing intranet manager for an improved intranet publication service due to a range of concerns, discussed next. marketing unit concerns with existing intranet publication service the five main concerns of marketing with the intranet publication service leading to a request for improved service are summarised, following. these were reported in an indepth interview of the marketing intranet manager conducted by the researcher. the first key problem related to the poor turnaround performance in respect of the elapsed time between initial request for content publication and content going live. a revealing comment by the marketing intranet manager was: “if an article is dated the 8th of august, then it has to be published on the intranet on the 8 th , otherwise the news is old.” the second key problem was the inability to track intranet publication requests. for example, the marketing intranet manager did not know whom to contact to find out the status of her publication requests. a related third key issue was that a single point of contact was sought for all requests. this concern was exacerbated by the fourth key concern which was the transitory nature of the service provider team structure. some of the web services team were contractors who only worked part-time, handing over intranet publication tasks to others on their weekly departure: “i also have a problem with the handing overs of tasks by the members of the web services team as often requests are not completed in time and sometimes they are completed in the wrong priority. this is also a result of the web services contractors coming in irregularly...” the fifth key concern was the need for flexible slas in publication service delivery, as some sections of the intranet required more frequent updating than others. this arrangement would enable improved scheduling by the web services team members and allow them to focus on the sections needing faster publication turnaround: “therefore i need a flexible service level agreement with the web services team.” from analysing the problems of customers, we can see that there were five key requirements which, apart from other issues which may arise in a formal requirements analysis, should be accommodated in the planned service desk service. we now explore the analysis of the web services team meeting in which draft new service desk requirements were developed entirely by the team, with neither the customer nor the team manager in attendance. ajis vol 13, no. 1 september 2005 184 shaping of requirements by web services team in this section, we describe how the web services team inadvertently shaped new service requirements by considering their own and customers’ tensions and concerns. the web services team met at least once a week to report progress and make new plans. at the re meeting, the team – without the marketing team or team manager present – briefly discussed the various customer problems that had been reported to them at different times in phone calls made by the customer when using the email-based method of servicing intranet publishing to marketing (and other corporate customers). the team appeared aware of the first three concerns reported in section 4.2, but not the fourth and fifth concern. throughout the meeting, there were many reinforcements of the decision to initially meet without the customer present in order to, according to one team member, “get the story straight” before presenting draft service requirements to marketing later. the planned initiative to resolve customer (and team) concerns consisted of designing and piloting a new service desk service for the marketing customer, and later extending that service to other internal customers. clearly, there was a degree of tension between the marketing customer and the web services team over the existing and planned service. for example, team members referred to current difficulties ‘losing’ customer emails – losses that had apparently been hidden from the marketing unit. a second conflict revolved around the protracted request turnaround which the web services team felt the marketing team had unfairly attributed to the web services team. the providers recognised the need for performance improvement, but felt that some of that improvement was needed in the content provision or reviewing stages (where they suggested the marketing team caused delays), rather than in their own responsibilities. a third tension was the marketing team’s demand for a single point of contact, which would place the entire burden of constant availability and responsiveness on one member of the web services team. summarising, the main web services team’s concerns with the existing publication service were: • being held responsible, unfairly, by managers and customers for lengthy end-to-end response times; • being unable to prove they were performing well as there was no recorded performance data; • losing emails as there were so many incoming and outgoing for service requests and work-in-progress; and • experiencing difficulties with tracking customer requests. during the meeting, the team developed an initial set of requirements for an adapted service desk service by which the customer could submit content for publication by a web form and uploaded attachments. a draft process model was drawn on a whiteboard, augmented by a set of publication request states and various notes. elements of design were included (e.g. web form, method of uploading attachments). the draft new service requirements model comprised the processes: • customer submission of a request by web form and uploading of an attachment for intranet publication of content; • opening of a ticket for the customer, internal review of proposed content by corporate intranet reviewer; • reviewer approval; ajis vol 13, no. 1 september 2005 185 • return to customer for action if not approved; • customer changes to content; • customer approval of final version; • live publication; and • closing of ticket. the team also prepared and refined a set of “compelling business reasons” to present to the marketing customers later to help “convince” them to accept the new service requirements. the “compelling business reasons” comprised: • provide a formal process for tracking publication requests; • enable more efficient provider processing of publication requests as each request will be followed by one person from the provider team; and • provide a backup system for publication requests to prevent loss. five key issues in eliciting and determining provider requirements – issues which clearly shaped the draft new service requirements model – were highlighted by the findings: service roles, responsibilities and accountability; service performance metrics; resolution of conflicting stakeholder service requirements; customer acceptance of service change; and service provider team structure. we discuss these issues next. service roles, responsibilities and accountability team members were very sensitive to the allocation of roles and responsibilities in the requirements, in recognition of likely ramifications for workload, accountability and performance measurement. for example: “if they (marketing) know it is ‘pending user action’ they will know it is their responsibility.” “re-opening a ticket is very bad for a service desk. it means that the people who were handling it (web services team) didn’t do their job properly, and it reflects in the statistics.” “all i’d do (planned service queue manager) is wait on content. instead of me (as single point of contact) being in charge of all those tasks and getting you guys to do them, you’d have your own list of tasks (as job point of contact) and basically you’d be timed on how quickly you could do them. it would free me up as a resource. i’d have my own list of tasks.” service performance metrics the need for the web services team to generate good service performance statistics regardless of customer performance was a constant theme. for example: “you guys might want to find out what kind of stats (statistics) we get (from service desk software), as if there is time between each comment added by the user, and three hours spent by the user on a ‘pending user action’, then we want that stats, not just opening and closing stats.” the proposed slas for publication turnaround time were discussed frequently for the same ajis vol 13, no. 1 september 2005 186 reason. for example: “the proposed slas are 1 day for urgent requests, 3 days for normal requests, and 5 days for special requests. nothing can be forced on us, though. they can’t put everything in 1 day requests.” resolution of conflicting stakeholder service requirements the requirements took into account the web services team’s concerns while attempting to accommodate most known customer concerns. a constant theme was the awareness that the customer had requirements that conflicted with provider needs, or requirements that were as-yet-unknown. team members often acted as proxies for the absent marketing unit in order to resolve conflicts and accommodate other needs. for example, a key customer requirement was acknowledged and its omission noted with regret: “marketing wants a single point of contact and they won’t be happy with our plan.” in many comments, the customer was perceived as potentially creating difficulties for the provider team. for example: “if there are stats (statistics) about their content delay, that could work against their acceptance of this if they get the idea that if they don’t do enough (content contribution, content review etc), they will get penalised.” “if suddenly they get measured, they are going to panic and think ‘oh <expletive>, we are going to have to submit stuff, and we have to do all these tickets.” however, genuine efforts were made to accommodate many perceived customer needs: “what if they just want to find a page they’ve done?” moreover, there were frequent reminders that the customer requirements had not yet been sourced and that this still had to take place. for example: “keep it simple for now so they can tell us what they want and we will add that.” “can this be changed if marketing doesn’t want it?” it was made abundantly clear by team members – especially by the individual who would likely have to take that role – that the ‘single point of contact’ requested by marketing would be too taxing for whoever had that role. nevertheless, concern was expressed about how the customer would react to being informed that this key requirement would not be met. plans were also made to manage perceived risk: “we need a plan b for marketing to publish with attachments (should the service desk fail). let them send email with attachments and a ticket number. it is easier to email with a ticket number and attachment (than use service desk)” this last comment indicated uncertainty about the effectiveness of the planned service desk. ajis vol 13, no. 1 september 2005 187 customer acceptance of service change a major concern was how to influence the customer to accept the planned new service. there was considerable anxiety about the future need to merge as-yet-unknown customer requirements with the draft model. the change acceptance process was merged in the team’s mind with eliciting and determining the customer requirements. for example: “the whole point is to have the users accept it. one of the strategies is to find influential people in marketing and get them onto your side. then they can influence the rest of the group.” “if she (marketing intranet manager) sits in on the presentation of this planned service she will feel like she was part of it.” “what are we going to call ourselves? the pitch team?” <laughter> “we are basically just coming up with a pitch.” service provider team structure the fact that the team was fairly young and included short term contractors highlighted their difficulties in relating to more senior marketing people and negotiating their own service needs. however, this type of team is not unusual for today’s it service organisation. when discussing how to approach marketing customers with the planned new service and convincing them to use it, it was suggested that early adopters could be approached: “how are we going to find that out (who are the early adopters in marketing)?” “i can’t do it as don’t really know marketing all that well as i have only been here six months so i can’t help.” “we can’t ask the contractors to help us (do that) as they are not paid to spend time with people.” members also expressed their concerns that they would not be heard by marketing because they were young and the marketing team comprised older people. discussion and conclusion underpinned by a socio-technical framework in service-oriented re (figure 1) which emerged from a literature review, we conducted a pioneering attempt to study serviceoriented re, helping to fill a gap in the literature and yielding important initial findings. in particular, the case study revealed five key issues of concern for service providers in service-oriented re, which shaped the draft service requirements. the issues are: service roles, responsibilities and accountability; service performance metrics; resolution of conflicting stakeholder service requirements; customer acceptance of service change; and service provider team structure. while we cannot generalise from one case study alone, these results are indicative. further research is needed to explore the issues identified for confirmation, validation and extension purposes. our study also provided a new understanding of service-oriented re in the evolving it service management organisational ajis vol 13, no. 1 september 2005 188 setting. the findings suggest three main implications for service-oriented re in practice: • first, businesses should recognise that although it service provision must clearly accommodate the customer’s needs, the service may have a significant impact on providers as well. if their concerns are overlooked, providers may not be fully committed to providing a quality service and may stymie service success by configuring the service requirements – including performance measurement, roles and responsibilities – to compensate. given the power represented in their technical knowledge and service configuration role, such manipulation is clearly feasible, as indeed was demonstrated in our study. • second, the provider concern outlined above is exacerbated by the key measure of service success – service performance. although the provider clearly plays a role in service performance, the customer also plays a role through their responses, which can contribute to measures such as end-toend response time and other performance metrics. therefore, the need to assign performance responsibility must be carefully and fairly handled in service-oriented re. • third, the social issues have been shown to be critical to a successful service-oriented re approach. stakeholder groups must be able to communicate and negotiate their conflicting it service needs. the composition of provider teams needs to be reconsidered in this light, raising new organisational issues. experienced personnel with strong negotiation skills may be needed. alternatively, facilitation of re meetings can be enabled by an independent trained facilitator. provider teams may also need training in the skills required for negotiating service requirements with customers. our study suggests that in the new it services era, new techniques and approaches are needed for eliciting and determining provider and customer requirements. new methods should involve key stakeholder groups equitably and more closely negotiate the sometimesconflicting provider and customer service needs. references baresi, l. (2004) call for papers, sore'04 (service-oriented requirements engineering) workshop, japan, available on world wide web at: http://serl.cs.colorado.edu/~serl/seworld/database/4128.html barker, d. (2004) itsmf – itil best practice: are we getting the message? servicetalk journal of it service management forum, 66, april, p. 3, graeme burn productions, available on world wide web at: http://www.itsmf.ro/downloads/april-04.pdf boehm, b.w. & ross, r. (1989) theory-w software project management principles and examples, ieee transactions on software engineering, 15(7): pp 902-916. bohmann, t., junginger, m. & krcmar, h. (2003) modular service architectures: a concept and method for engineering it services, proceedings of 36th hawaii international conference on system sciences (hicss 2003), ieee society press. budgen, d., brereton, p. & turner, m. (2004) codifying a service architectural style, ajis vol 13, no. 1 september 2005 189 proceedings of 28th annual international computer software and applications conference compsac 2004, hong kong, september 28-30. coughlan, j. macredie, r. d. (2002) effective communication in requirements elicitation: a comparison of methodologies. journal of requirements engineering, 7(2): 47-60. edvardsson, b., gustafsson, a, johnson, m. d. & sandén, b. (2000) new service development and innovation in the new economy, studentlitteratur, lund. galliers, r.d. (1992). choosing information systems research approaches. in r.d. galliers (ed) information systems research: issues, methods and practical guidelines, blackwell scientific publications, oxford, pp. 144 162. garschhammer, m., hauck, r., hegering, h.g., kempter, b., langer, m., nerb, m., radisic, i., roelle, h. & schmidt, h. (2001) towards generic service management concepts – a service model based approach. in g. pavlou, n. anerousis, and a. liotta, eds., proceedings of the 7th international ifip/ieee symposium on integrated management (im 2001), seattle, washington, usa, may, ieee publishing. goguen, j. a. (1994) requirements engineering: social and technical issues. academic press. goguen, j. a. (1997) towards a social, ethical theory of information, in social science research, technical systems and cooperative work: beyond the great divide, geoffrey bowker, les gasser, leigh star and william turner (ed.), erlbaum, pp. 2756. hayes, s. (2004) services defy gloomy forecast, australian it news, july 24, available on world wide web: http://australianit.news.com.au/common/print/0,7208,10254746%5e15317%5e%5enbv%5e 15306,00.html hunter, a. (2003) utilisation of a corporate intranet for knowledge sharing and retention, unpublished honors thesis, school of information systems, deakin university, melbourne, australia. idc (2004) worldwide web services software forecast, 2004-2008: cautious adoption continues, report #31079, idc. available on world wide web at: http://www.idc.com/getdoc.jsp?containerid=pr2004_04_13_090643 kotonya, g. & sommerville, i. (1998) requirements engineering: processes and techniques, john wiley and sons. layzell, p., bennett, k.h., budgen, d., brereton, p., macaulay, l. & munro, m. (2000) service-based software: the future for flexible software, proceedings of asia-pacific software engineering conference, 5-8 december, singapore. lichtenstein, s., hunter, a. & mustard, j. (2004) utilisation of intranets for knowledge sharing: a socio-technical study, proceedings of australasian conference on information systems (acis 2004), tasmania, hobart. lichtenstein, s. & hunter, a. (2005) considering the receiver in knowledge sharing: when the receiver seems ready the sharer appears, in proceedings of 7th australian conference on knowledge management and intelligent decision support (ackmids 2004), monash university, melbourne, australia. macaulay, l. a. (1996) requirements engineering, springer, london. mavin a. & maiden n.a.m. (2003) determining socio-technical systems requirements: experiences with generating and walking through scenarios, proceedings 11th international conference on requirements engineering, ieee computer society ajis vol 13, no. 1 september 2005 190 press, pp. 213-222. mayerl, c., nochta, z., muller, m., schauer, m., uremovic, a. & abeck, s. (2000) specification of a service management architecture to run distributed and networked systems, lecture notes in computer science and proceedings of 3rd international ifip/gi working conference on trends in distributed systems: towards a universal service market, pp. 258 – 269. mayring, p. (2000). qualitative content analysis. qualitative social research forum, 1(2). available on the world wide web at: http://www.qualitative-research.net/fqs-texte/2-00/2-00mayring-e.htm mcbride, n. (2002) the conceptual development of organisational it structure and strategy with a service focus, proceedings of the 7th annual conference of the uk academy of information systems, university of leeds, 10-12th april. mingay, s., furlonger, j., magee, f. & andren, e. (1998) the five pillars of organizational effectiveness, gartner: r-06-6660. niessink, f. & vliet, h.v. (1998) towards mature it services, software process improvement and practice, june, 4(2): pp. 55-71. nuseibeh, b. a. and easterbrook, s. m. (2000) requirements engineering: a roadmap, in a. c. w. finkelstein (ed.) the future of software engineering, (companion volume to the proceedings of the 22nd international conference on software engineering, icse'00). ieee computer society press. peppard, j. (2003) managing it as a portfolio of services, european management journal, 21(4): pp. 467-483. rands, t. (1992) information technology as a service operation, journal of information technology, 7(4): pp. 189-201. sailer, m., hanemann, a. & schmitz, d. (2004) variety of qos the mnm service model applied to web hosting services, proceedings of 11th annual workshop of hp ovua – hpov 2004, hp openview university association, june 20-23, paris. servicetalk (2004) servicetalk, june. available on world wide web at: http://www.itsmf.be/itsmfnewsite/documenten/servicetalk/june-04.pdf steen, m.w.a., strating, p., lankhorst, m.m., ter doest, h. & iacob, m.e. (2005) serviceoriented enterprise architecture, in service-oriented software system engineering: challenges and practices, stojanovic, z. & dahanayake, a. (eds.), idea group, chapter 7. sutton, d. c. (2000) linguistic problems with requirements and knowledge elicitation, journal of requirements engineering, london england, 5(2): pp.114–124. trienekens, j.j.m., bouman, j.j, & van der zwan, m. (2004) specification of service level agreements: problems, principles and practices, software quality journal, 12(1): pp: 43-57. van bon, j. (2002) it service management: an introduction, itsmf. van eck, p. & wieringa, r. (2003) requirements engineering for service-oriented computing: a position paper, proceedings of first international e-services workshop, icec 03, pittsburgh, usa, available on world wide web at: http://is.cs.utwente.nl/pubs/2003/eck_wieringa_icec03ws.pdf van eck, p. & wieringa, r. (2004) web services as product experience augmenters and the implications for requirements engineering: a position paper, proceedings of the international workshop in service-oriented requirements engineering, kyoto, japan, september 6. available on world wide web at: http://wwwhome.cs.utwente.nl/~patveck/papers/eck_wieringa_sore04_final.pdf ajis vol 13, no. 1 september 2005 191 walz, d. b., elam, j. j. & curtis, b. (1993) inside a software design team: knowledge acquisition, sharing, and integration, communication of acm, 36(10): pp. 63–77. ajis vol. 8 no. 2 may 2001 it takes more than ethics chris r. simpson co-director of the australian institute of computer ethics (aice); lecturer at the school of it, swinburne university of technology (sut). email: csimpson@swin.edu.au abstract recent positive developments in ethical outlook are explored, initially within the information and communication technology (ict) profession, and then broadened into other disciplines and the community in general. to understand why there has been a growing ethical problem in the first place, ethical attitudes of university students, ict exponents and people in other disciplines have been observed and noted. the search for practical ethical guidelines continues by questioning why, if professionalism indicates an adherence to a code of ethics that seeks high standards, do we still have trouble with the concept of ethics? ethics differ from one group to another. furthermore, ethics keep changing, as is evident in the latest codes, in which 'public good' now comes before the more inward-looking 'good of the profession'. so, how could an ethical code be more than an isolated, somewhat ineffective and temporary set of guidelines? how can it be freed of boundaries, of context and of time? how effective or relevant is the education of university students in practical ethics? how effective are the professional and ethical bodies? some answers are proposed and along the way, together with some simple but powerful notions and tools, that facilitate ethical understanding in university education and in professional practice. it is then argued that as work is a part of life, then a similar range of ethical options would be available in every context of life, be it as a private individual, a society, an employee, a small business, a corporation or in whatever discipline or role. in each situation, a similar range of lifestyle choices exist, for example, wasteful (careless), indulgent (selfish), sustainable (prudent), long-term view (responsible) or a perpetual view (meaningful and truly progressive). in other words, by agreeing to adopt 'the community ethics', one has taken on a dutiful role. for it to be more meaningful and more fruitful, life further demands such things as care of others and environment, along with some sense of humility, giving, gratitude, purpose, distant vision and wardenship. how can ethical guidelines transcend into being 'inspirational' guidelines, independent of occupation? is this not the realm of higher values? and if so, is this essential to the fundamental progress of human-kind? it may well be so. a few more simple observations and tools are offered that could be of assistance. keywords business, change, codes, computer, communications, community, consequences, education, ethics, grassroots, ict, information, it, is, practical, professional, responsibility, technology, tool, university, values. introduction this relatively short essay reports my views of the progress of ethical considerations in the field of information and communication technology (ict), and more broadly, in recent times in australia. at present, ict is not yet overtly values-driven: not in its design phases, nor its implementation, nor its application nor its use. i would dearly like to think that it could be. optimism or idealism aside, let us examine what would need to happen in order to say otherwise. time will tell, of course, but the good news is that mounting activity is occurring that is worthy of note. indeed, much has happened in the last few years that renders the prospect of utopia ever closer. let us trace some aspects of recent changes in ict outlook, then explore various prevailing external influences and finally propose broad and useful strategies that may encourage positive progress in the future, not only in ict, but in every avenue of life. ict education where does an appreciation of moral behaviour begin? at home? in school? at church? at scouts or guides or other service organisations? certainly before the 1970's it did. now, however, these institutions have either almost disappeared from public interest or waned significantly in influence as far as overt morals education goes. even at home, parental influence has on average diminished, as both (or a single parent) are commonly working full-time or are disinterested in morals or both. morals training was stripped out of school curricula in the mid 1970's. i have written extensively about this hi my paper given at ethicomp99 [simpson, 1999]. many students have been coming to their final year of university having seldom if ever formally thought about ethical issues relating to computing or, in more cases than expected, relating to any facet of life. only in the last two or three years has there been any hint of a return to certain schools of some formal morals coaching, and a few of these students are approaching final year university now in 2000. a big difference, noticeable over the last five or so years, has been the initial declared outlook of students as they enter my final-year computer ethics subject. let me explain that the subject, once called computing in the human context and now renamed professional issues in it (put), is compulsory in all undergraduate degree courses in the school of it at swinburne university of technology. whereas five years ago a significant number of students were cynical, and some remained so for the whole semester, now there is a much more 50 ajis vol. 8 no. 2 may 2001 willing and accepting outlook, along with a realization that such studies are important for their professional careers ahead. moreover, quite early in the semester, some are questioning why such a subject is not part of courses run by other departments of the university. the put subject stands alone in our it/is courses [simpson, 1998]. it has changed, developed and matured since 1990. while still serving to fulfill the accreditation requirements of the australian computer society (acs), it also attempts to redress the gross imbalance between technical and non-technical course-work that is completely out of step with the actual demands of professional practice. most significantly, it now serves well as a last-minute band-aid attempt to bridge the ethics gap. the gap is so wide that it is necessary to include in the syllabus countless opportunities and strategies to promote deep reflection and self-evaluation and to write about them. several aspects are introduced such as "what has ethics to do with me?"; generational differences; cultural differences; working in a team; ethical issues in artificial intelligence, in computer crime, in it law, and in globalisation; professional associations and codes; and responsibility. recent years have seen ethics-oriented subjects appear in it or is curricula at other universities. for example, in the school of computing and mathematics at deakin university, which has a long history in distance education, there exists an equivalent subject. it is called computers and society and professional ethics and is run by dr. jo coldwell [coldwell, 2000]. it uses a completely different approach, being largely web-based and organised as an on-line "conference". small group work is possible in asynchronous "sub-conferences", in which the results of group discussions upon each topic are posted on plenary pages. both this approach and that of swinburne's put subject are highly dependent upon students' personal involvement and introspection. these two approaches involving personal development and practical ethics are in stark contrast with various other implementations of an equivalent subject, in which for example, a bit of applied ethics (aristotle & co) plus some law cases plus some basic sociology or psychology theory may be included. i feel that this more theoretical approach would constitute a reasonable second rung should it exist in a subject stream, but that students are simply not ready for it without a solid notion of practical ethics, that is, who they are, what they believe, how they fit into the world and how they can take on moral responsibility. ict professionalism and practice although some would criticize by asserting that it professional bodies still have "no teeth" and that few practising ict (or it or is) people display committed adherence to a society or its code of ethics and practice, the ict profession, worldwide, has taken quite a few positive steps in recent years. several ict or software engineering codes of ethics and practice have been developed or are now updated to include a refreshing outward-looking attitude that no longer places as first priority the interests of the profession, but instead, the greater good. for example the acs code includes first under "values and ideals", in section 4.3.1: "priorities: i must place the interests of the community above those of personal or sectional interests. [acs, 2000.a]. an excellent new software engineering code was adopted jointly by ieee-computer society and the association for computing machinery (acm) late in 1999 [ieee-cs, 1999]. this code also re-prioritises first place to "the public interest" and embodies some extraordinarily released statements. relevant professional bodies around the world have continued to improve their position. for example, in australia, acs demands a certain level of competency in ethical studies for university courses to be accredited, and have developed a requisite "body of knowledge" and a professional recognition program, which is "supported by the increasing number of organisations declaring a preference for acs" [acs, 2000.b]. since january 1999, acs embraces a computer ethics taskforce (within the acs community affairs board's economic, legal and social implications committee) [acs, 2000.c]. yet the criticism of "no teeth" is not yet overcome. how can ict be recognized in the same light as (say) the medical profession, when the bulk of ict exponents do not have an accredited qualification, and of those that do, only a modest percentage hi australia have been members of the acs, and of those, only those who joined in the last few years will have taken requisite ethical studies. there is no national register of computer consultants. there is no hippocratic oath or its modern-day equivalent, nor a significant track record of successful tribunal challenges for malpractice, nor the overwhelming cooperation of the discipline's following. education at all levels is a major key to the problems just stated. education and advice is precisely what some ethical organisations (such as aice), which are external to the professional bodies, wish to offer in support of existing internal programmes for computer professionals. in addition, external organizations can offer public education and advice aimed at stimulating all sectors of the community to recognize and acknowledge ethical professional behaviour. 51 ajis vol. 8 no. 2 may 2001 ict practice new tools for some time, the practice of ict has been aided by software tools, project management tools and business packages. not until very recently have there emerged tools that go beyond purely technical design considerations; tools that begin to consider value statements and flow-on effects beyond the client and into the community and environment, and that pay attention to "distant vision": all of these being hallmarks of excellent and ethical design. one notable example assists with a style of risk evaluation that goes beyond technical issues into overtly ethical regions. the notion of preparing a software development impact statement that looks into the consequences of a project beyond client organization economics is quite new. a package called sodis [gotterbarn, 1999] is currently in its final test phase, although it will most likely be revised as more experience is accumulated. its operation is explained in don gotterbarn's paper [ibid]. basically, it takes the project task list and asks thirtytwo ethical impact questions about every task in turn of each of the relevant stakeholders in the project, both close and distant, in turn, whether there are ethical consequences of that task for that stakeholder (yes or no). a map of responses is generated and an ethical risk profile may be generated. establishing a complete list of stakeholders is crucial to the success of this method, as some stakeholders are anything but obvious initially. another instance of a software engineering tool with built-in ethical considerations has been proposed in a recent paper by rogerson, weckert and simpson. the structured systems analysis and design method (ssdam) is an established design methodology. it is a "hard" systems approach, which is shown can be enriched by incorporating various principles from a code of ethics, in this case the acs code of ethics [rogerson, 2000]. ict ethics bodies over the past few years, parallel to the rapidly increasing public use of the word "ethics", interest in computer ethics has also risen quickly in the community, particularly within academic circles. ethics bodies have been formed that have high ideals and aim to promote them. the australian institute of computer ethics (aice) is but one that runs seminars and international conferences and ties into academic programmes. there are many other australian bodies such as internet industry association (iia) and electronic frontiers association (efa) that have sectional interests within the overall technology, and who are all contributing strongly to a new level of public grasp of ethical issues. it is much the same globally. aice has a distributed presence in several locations around australia. it operates differently in each of its chapters and accumulates valuable experience and resources along the way to redistribute as the opportunities arise. it is alive by virtue of the energies of its members (or more correctly, "friends") and its network of collaborating and supporting bodies. strong and productive links exist between aice and similar bodies all over the world, such as ccsr in uk, rccs and seeri in usa and fict in the netherlands, as well as connections with individual gurus and friends in the field. the network is even stronger considering the personal links of our members, particularly through aice's other co-director john weckert. beyond the ict fraternity it is pointless to limit our attention exclusively to exponents of ict. most computer ethics issues extend to users in all disciplines, occupations (work) and in fact, every aspect of life. work is a part of life, so what is good for life should be good for work, although thousands might initially disagree. in support of this assertion, we should enquire: who else has recently taken up the cause of "ethics"? other professions and the business world the older professions have rigorous sets of controls that have used peer pressure to good effect. the entry conditions are extremely tight; the oath and/or code are binding; a practising register is maintained; fellowship is highly sought after; any minor blot on one's ethics of practice is remembered for ever. why then is the community at odds with certain decisions of such professional bodies over some recent ethical issues? perhaps there are shortcomings in the code relating to protection, or maybe there are unknown processes behind the scenes? i am merely pointing out that community ethics are continually modifying, and that even the most respected professions need to be conscious of these changes, particularly the currently perceived need for transparency of process and "public interest". many citizens are losing respect for outdated, expensive and unfair practices. i believe that the budding new ict profession is now and will try very hard in the future to maintain a close watch on the public interface and ethical aspects of its operations. success will depend upon its members and their perception of "ethics". the world of business is undergoing huge change. that is good news, because the recent trend has been towards annihilation of the world's resources and dehumanisation of its people! 52 ajis vol. 8 no. 2 may 2001 many would assert that 'business is unprincipled'. but what are we talking about here? managers, economists, accountants? no, i do not think so. has not the same prevailing narrow set of values been enforced upon all professionals and, in fact, anyone in almost any organisation? a notable factor is that the stakes are higher as the promotion ladder is ascended. like ict, "business" pervades most of the community. unfortunately, the limited set of values, summed up by "the bottom line", goes with it and affects us all. another external force that has considerably tightened its business-oriented grip in the last few years is that loosely called "globalisation". originally the thought of 'hands across the seas' was most attractive to generoushearted people, who cared about (say) third world inequities. however, it was seized upon by trans-national "gnomes" as an excuse to become two faced providing work in underdeveloped countries, but in reality exploiting them pitifully and worsening the inequity. able to thumb their noses at national governments, they have further disdained border controls and taxation responsibilities by applying recent developments of internet technology and exploiting its relative global freedom as well. what then is the change? is it all for the worse? on the contrary, in the last two or three years, there have begun to emerge practical expressions of what has been yearned for by many and talked about for the past decade by bodies like the st james ethics centre (see below). the concept of a "triple bottom line" is now being taken up by some corporations both in australia and around the globe. this term infers a value system that broadens the old and lonely economic values by adding human (social) values on one side and environmental (sustainability) values on the other. the net effect is that better decisions can be made with this system. it also has the effect of reducing overt greed, which, i believe, has ultimately caused the rampage alluded to above. how has this positive change come into being? grass-roots action by ethical bodies has set out to convince "business" that in every sense there are better results available by adopting broader values for decision-making. instrumental ethical bodies were not confined to the ict group! of course there have been other bodies, together with outspoken individuals, not to mention the prophets of a decade or two ago. other ethics bodies within the last five years particularly, many ethics groups with broader or overlapping spheres of interest have been created. i shall mention but a few that have immediate proximity. through john weckert, who leads the charles sturt university (csu) chapter of aice, intimate ties exist with the australian association for professional and applied ethics (aapae). this is a formal, academic association based at csu that has recently won collaborative grants to support high levels of research activity. the business ethics research unit (beru) is based at victoria university and runs frequent seminars, concentrating upon ethics in the business context. beru has existed for about two years, is already productive in research and has member-level links with aice. the ethical enterprise network (een) aims to "help our members adopt ethical practices within their own enterprises; and build awareness in the community about ethical, sustainable and just business practices." een also aims to build networks between ethical enterprises through membership, a regular newsletter, meetings and sharing of information about ethical activities, internal and external. [een, 2000] the st james ethics centre is based in sydney and has successfully run for more than 10 years, having attracted excellent patronage and an effective team of consultants. it has a very broad sphere of interest and has solid connections with the business world. aice has personal links with a melbourne project office that was opened in february 2000. a cooperative organization with whom aice has a personal link is "borderlands", founded in 1997, which "is a place where people can meet, talk, reflect, learn and teach, read and study, organise, administer and manage their networks or activities in and from. [a place] where consultation, consulting and counselling can happen, where a broad spectrum of basic resources are made available and accessible and which thus would become a node of various intersecting local, national and international networks." [borderlands, 2000]. it aims to involve citizens and groups at a grass-roots level of ethical and ecological concern. when do "ethics" cease to be useful? a number of positive and most releasing ideas are embodied in the terms "best practice", "greater good", "public good", "public interest" and "ethics". however, all of them have become embarrassing through the number of interpretations being made of them. there are almost as many definitions of these phrases as there are sectional interests! let us examine "ethics" in particular. ethics, as suggested already, is a problematic notion: ill-defined and either good or bad. specifically, ethics differ between different social groups and cultures and are subject to change. they are expressions of group norms, being the expectations of behaviour in that group, whether aspirational or destructive, often ending up as the lowest common denominator! "good ethics " need to be defined too. a number of definitions may be found in the various professional and corporate codes of ethics and practice, which frequently go only so 53 ajis vol. 8 no. 2 may 2001 far towards providing behavioural guidelines, as they are intended for use within a defined context. the best codes therefore look outwards. they define and prioritise "greater public good" before all else. likewise, the tenets are broad in nature. however, even this marvellous breakthrough of outward-looking codes is not ultimately sufficient why? because no code, to my knowledge, can adequately cope with boundaries that exist between national, cultural or sectional interests, especially when put to the acid test. furthermore, what is the greater public good? are there alternatives? yes, there are alternatives, because life is greater than this. true and effective human communication transcends boundaries of all kinds, be they professional, sectional, cultural or national. every professional person, or any person per se, therefore needs much broader and more universal guidelines than just those internal to a profession or group. what might these be? maybe the highest shared values those that are common to all mankind thereby transcending all boundaries. if so, what are they? modal axis managing and designing our life systemic axis figure 1: multi-modal systems framework some tools for viewing higher values rushworth kidder wrote a charming book, entitled "shared values for a troubled world" [kidder, 1994]. in it, he sets out his plan in the first chapter and then reports captivating conversations with twenty-four people, each well respected and each of widely differing race, creed, context and circumstance. his aim was to establish what values each of these people felt were the most important to them. kidder then ranked the most common of the values. there emerged eight shared values common to all: love, truthfulness, fairness, freedom, unity, tolerance, responsibility, respect for life. all of these pass the test for being "on the highest rung of the visionary ladder" [ibid p323]. that is, if one asks why, for example "why does truthfulness matter?" there is simply no sensible, worldly answer. he sums up the book with: [ibidp324] "so what good is this code of values? it gives us the way to build downward to the level of goals, plans, and tactics, where things really happen and the world really changes. it unifies us as we move around on the ladder, giving us a home territory of consensus and agreement. and it gives us a way not the way, but a way to reply when we're asked, "whose values will you teach?" answering this last question as we tumble into the twenty-first century with the twentieth century's sense of ethics, may be one of the most valuable mental activities of our time." being universal and perpetual, these higher values of mankind can be adopted personally just as easily as by a group or profession. further, being absolute, they are paradoxically impossible to categorise, as they know no boundaries. donald de raadt writes about an interesting way to analyse one's activities [de raadt, 1997]. the "multi-modal systems framework" is a social analysis tool that looks at everyday activities via a wide range of modes, thus assisting the clarity of one's thought processes. the model uses modes that range from: 54 ajis vol. 8 no. 2 may 2001 logical, numeric, spatial, to economic, aesthetic, juridical, ethical, & credal. (see figure. 1) to view any social instance is made much simpler by viewing each mode separately, by using its appropriate telescope that filters out all but one mode of behaviour. this exciting model acknowledges the higher values in the so-called credal plane, in which activities are pursued out of faith, rather than out of duty as in the ethical plane. the model continues to be developed at the centre for technology and social systems (cts) a joint swedish and dutch collegium, sometimes called 'swehol'. responsibility measurement a call for papers for the international society for the systems sciences, 42nd annual meeting, atlanta georgia, usa, 19-24 july 1998 stated: "a distinguishing feature of today's world is that technology has built the house in which we all live. more and more, our lives are lived within the confines of its walls. yet this implies that technology entails far more than the material artifacts surrounding us. technology has in fact become a very complex system that confronts us with new dimensions of human responsibility". i show my students a way to measure responsibility, certainly a useful tool for a professional person. we talk about responsibility relating to context, commitment, the bearing of consequences and values. this leads us to propose that the level of responsibility could be measured in four ways: commitment (intended duration/nature of responsibility); moral motive behind taking on the responsibility; context within which the responsibility is carried out and; scope of (project) consequences. these could be represented, visually graded, on a composite table of four independent columns: t t level i life-long of i long-term respon-1 medium-term sibility | short-term i ad hoc i once only i half-hearted commitment (project) t spiritual awareness ethical vision creativity compassion professionalism practicality duty t wardenship social responsibility community workplace home personal development (leave to someone else) environment community users client suppliers employer oneself moral motive context scope of consequences figure 2: four aspects of measuring responsibility the imperfect but useful graphical ideas given here in figure 2 reflect both kidder's results and the multi-modal model shown in figure 1. i go on to suggest to my students that responsibility is an attitude, a moral commitment to a way of life. bearing in mind that nothing tangible will be achieved until one actually does something. it is no good to sit on one's bottom, glowing with "responsibility". one needs to be proactive and actually take responsibility that a certain something happens. but then there is a process. first, one needs to perceive the problem. the need is usually thrust upon us by circumstance. once recognised, it means that one is ready to take on the task, as one's experience, maturity and sensitivity has rendered it visible. next comes the decision whether to make a commitment, and this is often the most difficult step to take. once the decision is made, one may identify where one lies in each of the four columns of the table in figure 2. appropriate action is usually clear, but will often require great courage to implement, because of implied consequences. some consequences are predictable, some unexpected and others may never eventuate. although one must be brave and firm in the faith that one's decision is the right one, if in retrospect it turns out that it was not the right decision, one should be prepared to make good. 55 ajis vol. 8 no. 2 may 2001 conclusion discussion so far has noted the recent general upsurge in ethical awareness, which is indeed most welcome and necessary. it reflects the excellent work that is being done to facilitate a broader understanding of what is good. many ethically concerned bodies have evolved that can assist organizations and corporations with, for example, the creation of codes of ethics suitable for their various contexts. but it has also been argued that guidelines of a more universal nature are required, that will transcend the limitations of ethics per se, being naturally sectional and ever-changing. the shared higher values of mankind need to be identified and not only wound into corporate codes and guidelines, but also absorbed into the very being of every individual, particularly those who must assume responsibility, such as professional people. some simple tools have been noted already that assist in understanding the existence and practicality of these higher values by placing them in graphs and priority lists of simple tools. idealism is a wonderful motivator, but to put ideals into practice clearly requires awareness, positive attitude, practical tools, skill to use them and both willingness and courage to match the task by taking on requisite responsibility. but the ultimate question is still unanswered: howl in the mist of today's confusing, chaotic world of competing interests and thundering "white noise", it is easy to be distracted from higher ideals. but a cogent argument could be made here: • that corporations and associations are just groups of individuals; • that the ultimate decisions are those of individuals; • that whatever guidelines an individual adopts is ultimately an individual choice; • that this choice bears upon his or her whole life, and upon the lives of all around about, and upon the environment itself. therefore vital corporate or public (or any) decisions become issues of personal attitude that include responsibility and caring for others, the humility to recognize one's limitations and the passionate will to broaden one's strengths and "positive ethical conduct" [dalai, 1999]. but as we have already asked, how do we obtain these qualities and insights? how might the "greater good" be recognized? perhaps there really is an urgent need in the community for: quietness; contemplation; questioning competition and growth; waiting patiently but proactively; listening; consideration of others; embracing wisdom; and faith. such tools as these for developing one's personal higher-value skills have been around for millennia and do not need explanation here. they have become progressively suppressed and denigrated at least since the industrial revolution, and suffered especially during the past two decades in the shadow of recent "business best practices". their reintroduction and general acceptance is long overdue. yet, the good news is that all this is beginning to change. and clearly, it is taking more than ethics! as a footnote, it is comforting to note that the dalai lama appears to have similar thoughts. his messages of "happiness" and "positive ethical conduct" are prominent in his recent book [dalai 1999]. references: australian computer society (2000.a) acs code of ethics, ref. via http://www.acs.org.au/national/pospaper/acsl31 .htm [accessed 15/9/00]. australian computer society (2000.b) acs professional recognition program, ref. via http://www.acs.org.au/national/profrecog.htm [accessed 15/9/00]. australian computer society (2000.c) australian computer society (acs), community affairs board (cab), economic, legal and social implications committee (elsic), computer ethics taskforce (get), ref. via http://www.acs.org.au/boards/cab/elsictr.htmltfet [accessed 15/9/00]. borderlands home page http://www.borderlands.org.au/ [accessed 15/9/00]. coldwell, jo (2000) subject: scc383 computers and society and professional ethics deakin university school of computing and mathematics, ref. via http://www3.cm.deakin.edu.au/~ioio/ [accessed 15/9/00]. dalai lama (1999) ethics in the new millenium, riverhead books august. de raadt, j donald r (1997) a sketch for humane operational research in a technological society; systems practice, august 1997, vol.10, no.4, pages 421-442. een (ethical enterprise network)(2000) home page http://home.vicnet.net.au/~eenet/ [accessed 15/9/00]. gotterbarn, donald (1999) promoting ethical responsibility in software development with a software development impact statement, proceedings of aicec99, melbourne, july; (also downloadable via http://www.aice.swin.edu.au/events/aicec99/webabstractsindex.html [accessed 15/9/00]). ieee-cs (computer society of ieee)(1999) ieee-cs / acm software engineering code of ethics (vers 5.2): ref. downloadable via http://www.computer.org/tab/swecc/code.htm [accessed 15/9/00]. 56 ajis vol. 8 no. 2 may 2001 kidder, rushworth m (1994) shared values for a troubled world, jossey-bass, san francisco. rogerson, s, weckert, j, simpson, c (2000) an ethical review of information systems development the australian computer society's code of ethics and ssadm; information, people and technology vol.13 no.2. simpson, c and burmeister, 0 (1998) new professionals, new measures of worth, new ethic of collaboratioa proceedings of ethicomp98 erasmus university, rotterdam, march. simpson c.r. (1999) could computer ethics spark a new moral generation? -an australian perspective, proceedings of ethicomp99, rome, november. 57 ajis vol. 8 no. 2 may 2001 ethics in research by mark campbell williams, faculty of business and public management, school of management information systems ecu (edith cowan university) churchiands,wa6018 email: m.williams@cowan.edu.au abstract in order to reflect on some ethical improprieties which i had committed during the data collection phase of an information systems research study, i conducted an heuristic and psychologically-oriented self-study. as part of this heuristic reflection, i engaged in a number of self dialogues in the form of a conversation between various characters. reported in this paper is one of these dialogues, concerning broad issues of ethics and research and discussing the notion of wisdom, maturity, meaning, and virtue. ethical considerations are always of primary importance, and i would assert that this is even more so when considering research investigating and using new media, such as the world wide web, in which acceptable ethical practices have yet to be established and consolidated. keywords: ethics, research, dialogue, wisdom, virtue introduction in this paper, which builds upon my presentation at the 1999 australian conference of information systems (williams, 1999), i present an autobiographical account of my five years as a teacher and researcher investigating a teaching reform of the first-year undergraduate business computing course for the bachelor of business degree at the university at which i was employed from 1991 to 1995.1 was progressively a tutor, then a lecturer, then the course coordinator facilitating the reform of the course. my research began in late 1991 as a qualitative investigation with the implicit hypothesis that open discourse could balance technicism in the university business computing classes taught by myself and three colleagues. the basic idea of the reform was to encourage communication and discourse about the meaning and purpose and wider societal implications of business computing in addition to the narrowly technical aspects. i present a description of the research and a conversation concerning the general issue of ethics in research, structured loosely around an idea of wisdom with some keywords of maturity, meaning and virtue. this discussion is presented in the form of a conversation between several persons with which i coordinated an exchange of comments concerning qualitative research. overall, i considered that my responsibility, to myself, to the participants and to the research community, was to tell my story, with as much maturity, meaning and virtue as i could. open discourse and technicism open discourse is communication concerning personal and social meaning, purpose, orientation, values, goals, concepts, ideas, feelings and emotions relevant to, but distinct from, the narrow subject matter, technique and administration of university business computing education. in a university business computing tutorial, for example, open discourse could occur in the initial introductions of the participants or in initial statements of the meaning and purpose of the subject to students' life situations or goals. any discourse specifically concerned with matters of computing techniques or the terminology of the text would not be open discourse (although open discourse may occur if the discussion broadens to include personal, historical or societal issues relating to the meaning or purpose of the techniques). open discourse does not necessarily have to be formal small group or whole group discussion. it may even be a passing comment to an individual student after the end of the computing laboratory, or an informal chat to a group of students in the campus grounds, or even a note on assessing a student's work (written communication can also be discourse). the teacher and students can be autonomous, free, well-rounded human beings able to engage in meaningful and balanced discourse even in spite of a technicist curriculum (taylor and campbell williams, 1993b). technicism is an over-emphasis on technical, instrumental or strategic techniques or actions, to the detriment of wider human communication, human values or human purpose (adomo & horkheimer, 1990; dryzek, 1990; ellul, 1990; habermas, 1972; marcuse, 1964, 1969). persons thinking in technicist ways would tend to define all human problems in terms of rational and technical solutions, thus leading to an undue emphasis on the science, instrumental rationality and technology of the modem age (bowers, 1988, 1993 a, 1993 b; ellul, 1964, 1973; habermas, 1971). 68 ajis vol. 8 no. 2 may 2001 the teaching reform the essence of the reform was to encourage open discourse (a form of communicative rationality) by introducing teaching and learning strategies such as group work, a dialogical communication process in the tutorials and mass lectures, and requiring students to keep personal learning journals. the first phase of my research, from 1991 to 1993, was an interpretive qualitative investigation with the implicit hypothesis that the teaching strategies would encourage communicative rationality through open discourse and balance the dominating instrumental rationality of technicism. however, in 1993, through a component of self-study in the research, i came to understand that i, personally, had succumbed to a form of unbalanced and inappropriate instrumental rationality (a strategic rationality) in my conduct of the research. i had imposed the teaching strategies on the students and tutors without fully informing them and without fully gaining their consent. in order to reflect on my unethical practice, i conducted a heuristic and psychologically-oriented self-study from 1993 to 1996. the research question was "what was the underlying reasons for the ethical shortcomings of my flawed investigation of 1991 to 1993, and how could i address this?". thus reporting on a dialogical interplay between outer-oriented and subjectively inner-oriented aspects of technicism and discourse, i explore that dimension of the self-study of teaching which includes the psychologically-oriented study of self. background theory for more than 20 years, in australia and overseas, i have been involved in high school teaching, then technical and further education, then industry computer based training consulting and now university computing education. my work in education has been immeasurably enriched by the advent of microcomputers and i have seen many students, especially the handicapped and those who do not score well in scholastic tests, display uncharacteristic enthusiasm for their education and sometimes achieve remarkable results through using this technology. so, regarding the increased use of microcomputers in education, one part of me is positive and enthusiastic. there is another aspect of myself, however, that remains concerned and critical about some aspects of the impact of computing technology on the teaching-learning process. the keyboard enables us to give orders without personal contact. we have a collection of individuals without personal contact. we have a collection of individuals without interaction (except in terms of technique or by means of it). (prof. jacques ellul, 1990,) jacques ellul's 40 years of (rather french) polemic against the dangers of "la technique" (ellul, 1964; 1990) forms a consistent critique of technicism and the sort of scenario portrayed by lewis. for ellul, technique is a ubiquitous and hegemonic force that dominates modernity and is reductionist in reducing human experience to matters of method and efficacy. pithily describing the essence of technicism, ellul (1990) comments wryly in regard to the optimism of a technicist economist: "whenever a difficulty arises, 'technical progress will deal with it'" (p. 21). understood in this way, technique can be seen as more than just technological, but as a social and psychological commitment with the added dimension of a faith in technical progress ellul understands technique as a spiritual force (1990, pp. 405-411). technique acts to close off all avenues of human resistance to what ellul prophesies as the spiralling descent of modernity's commitment to technical progress. it acts to create a monolithic evangelising mindset which stifles any criticism or true debate about other ways of knowing or being: "the intellectual and cultural tragedy of the modern world is that we are in a technical milieu that does not allow reflection" (1990, p. 145). ellul continues: are we then shut up, blocked, and chained by the inevitability of the technical system which is making us march like obedient automatons thanks to its bluff? yes, we are radically determined. . . . not really . . . we profit from the little cracks of freedom, and install in them a trembling freedom which is not attributed to or mediated by machines. (prof. jacques ellul, 1990,) in his last major work, the technological bluff (1990), ellul considers that education has been taken over by what he terms "terrorism in the velvet glove of technology" (pp. 389-394). in most major national political agendas, ellul perceives a non-reflective and hegemonic push for technique-oriented pedagogy in the form of computer-mediated education which "in some sense falsifies everything that traditional culture has slowly developed" (p. 391). in the quotation which begins this section, he ended the book by alluding to a possible hope for action in a technically determined future. his hope was that, by radically critical reflection, by a critique of the bluff of technological propaganda, by exploring "little cracks of freedom", individuals may be able, by what he terms "the sole aptitude of astonishment", to "truly invent the new thing" (1990, p. 412). 69 ajis vol. 8 no. 2 may 2001 a conversation concerning ethics in this paper, i present a conversation concerning the general issue of ethics in research, structured loosely around an idea of wisdom with some keywords of maturity, meaning and virtue. this discussion is presented in the form of a conversation between several persons. i coordinated an exchange of comments, concerning qualitative research, between mr craig standing, dr peter taylor, dr peter standen and myself. with their permission, i extensively edited, expanded, and added to the dialogue to change the themes and content to suit my own purposes. to protect my colleagues, and to clearly signal the fictive nature of the dialogue, i use the names bill, bob, ben. mark: it seems to me that some folk, including those that are normally wise and ethical, act in an unusually 'loose' manner when using computers and the web. i have experienced cases when researchers, not only students, act with gay abandon in copying and plagiarising material from the web. it is my opinion that ethical procedures and practices take considerable time to become established and consolidated in new media and new situations. also, some researchers are so concerned about research rigour that, it seems to me, they sometimes neglect important qualitative or intuitive human concerns. rather than using the word "ethics", perhaps we should begin by considering the concept of wisdom. bill: when using the word wisdom, i take it that you are talking from a value-laden framework that you think is preferable and can be used across multiple frameworks to guide the process of researching. i would like to hear more about this metaphor for i am thinking along the same line. too many of us seem to be under a great burden, perhaps under the name of rigour, to achieve acceptability in academia. mark: in your first insight you are correct, and i agree with your second comment. i follow what is known as a virtue theory especially as it has been expounded by alasdair maclntyre (1984) in his book beyond virtue: a study in moral theory. as i understand him, maclntyre asserted that wisdom can only be lived out in a virtuous life, informed by multiple ways of knowing and learning, in community. as bowers commented, community is: "based on some conception of how a good person acts" (1993, p. 87). wisdom can be passed on only through a kind of apprenticeship in a way analogous to a bird teaching its young to fly. wisdom, itself, seems to be part of the warp and weave of life bound up with constitutive forms of knowledge rational and non-rational, conscious and unconscious, cerebral and somatic, logical and intuitive, emotional and holistic these and more. linguistically, it is primarily in the form of moral, metaphorical narratives which represent the community's stored understanding and wisdom. c. a. bowers is much taken by these ideas, quoting maclntyre's comment: "the narrative of any one life is part of an interlocking set of narratives" (cited in bowers, 1993, p. 87). bowers (1993) compared this understanding with the limited understandings of some computing experts. he asserted that these forms of knowledge (what i term wisdom): "cannot be made explicit and re-encoded to fit the digital technology required for computer-mediated communication" (p. 86, 87). when i use the term wisdom, i am thinking of the virtue of wisdom in this broad sense. allow me to present pictorially some key words which may stimulate our discussion (see fig. 1). bob: one must strive for objectivity. it is difficult for the social sciences to achieve the miracles of the physical sciences, but researchers must strive to use the methods of the physical sciences. ben: what about that research in the social sciences which is difficult to replicate and, due to inherent subjectivity, almost impossible not to subtly influence the outcome? i am no positivist. in fact, i speculate that the whole idea of academic positivism became a device to lead the researcher to maturity and morality. in other words, to promote a type of wisdom. in the physical sciences, quantitative research was sufficiently limited and narrow that other researchers could replicate the experiment to test its validity. however, in the social sciences very few interventions could be reproduced in a way that could attest to the validity of the original research. perhaps the intellectual community evolved elaborate rules to restrict the issue and help researchers resist the temptation to use immature rhetoric (or to lie for recognition, or whatever). the hope may have been that, if students were trained in academic (sometimes esoteric) rules, then they would develop their intellect and thus grow in wisdom (outworked in maturity and morality). however, we are all aware of the history of academic and scientific fraud. i have some anecdotal evidence in that, of the eight or so academics with whom i have spoken informally regarding their theses, three openly said that they committed academic fraud (and this was in three of the most respected universities in the western world). there are other ways to lead people to wisdom for example, the adoption of a moral framework which is often the outworking of a religious commitment. 70 aj1s vol. 8 no. 2 may 2001 wisdom in teaching, learning, researching & writing experience and knowledge together with the power of applying them critically or practically i meaningmaturity being duly careful and adequate as a developed person especially 1. balance: a stability of mind; an equilibrium of instrumental action with communicative interaction 2. thorpughness: not superficial; done with care, completeness and context 3. rigour: strictness and logical exactitude pursuing that which is considered significant and importantespecially 1. purpose: intention, resolution, and a reason for endeavour 2. direction: a way or course for endeavour 3. interest: a feeling of curiosity or concern, can be a passion virtue virtuous character or behaviour; a good moral quality, as justice especially 1.truthfulness: esp. make known or reveal one s opinions, biases, interests, motivations and agendas (incl. unconscious) 2. honesty: truthful, fair and just in character and behaviour 3. commitment: dedication and application in a pledge or undertaking figure 1. thinking about of wisdom in teaching, learning, researching and writing mark: whatever the case, the post-enlightenment thinkers who formulated the academic rules developed a rigid system of experimental quantitative rigour (bowers, 1992). this restricted approach has been seriously questioned originally by neo-calvinist philosophy in the early twentieth century (doowerweerd, 1979) and then by postmodernism, poststructuralism, feminism (caraca & carrilho, 1994) and neo-marxism (bowers, 1992). i like the viewpoint of caraca and carrilho (1994) when they commended us to "think of knowledge and particularly scientific knowledge not as the application of a previously established method aimed also at achieving previously determined ends, but as an invention of strategies for discovery, legitimation and communication, for which the rhetoric of science may perhaps come to provide a rather unexpected picture for many" (p. 786). in my researching, i endeavour to explore ways to be rigorous and yet to be passionate. as eisner (1992) recommended: "when you have a conviction about what you believe is important to study or how you think it should be studied, my advice to you is to pursue that conviction." bob: you have conveniently ignored the history of the last 500 years here! scientific method, because it is an open system of knowledge acquisition, became popular precisely because it liberated us from moral philosophy that enslaved. remember galileo at the spanish inquisition? what about the role of religious 'wisdom1 in increasing the happiness of the human races from the ritual sacrifices of the aztecs through the dark ages to its peak in the inquisition and the enslavery of the third world in the 1500-1600s? scientific method was one of the major ways we rid the world of these tyrannies, by enabling anyone to come up with openly debatable and verifiable answers to fundamental questions such as whether the earth is flat, or is the centre of the universe. the realities espoused by wise folk such as witchdoctors, prophets and popes were found wrong. what other method could have achieved this? ben: bob, i am surprised that you have not grasped the deeper currents of history. critical theory, postmodernism and deep ecology have clearly revealed the deeply-embedded violence inherent in scientific consciousness. it is open only in so far as one treads a narrow epistemology. you should remember that it was the secular political inquisition which was the most barbaric (people often deliberately blasphemed at the crucial moment so that they would be tried by the more 'reasonable' religious inquisition). and both these pale before the terrible inquisitions of the enlightenment french revolution terrors through to the nazi holocausts and right up to pol pot. the enlightenment project, with science as its talisman, was originally empowering, but the seeds of its enslaving agenda were within its original mandate to dominate nature. ghastly destruction has been wrought by scientistic positivism not only in its application for the weapons of war. in one way, scientific and technological instrumental rationality has proven to be more 'wrong' and more disastrous for planetary ecology than all the witchdoctors, prophets and popes put together! mark: i think that there is something important in what you say, ben. adomo and horkheimer (1990), ellul (1964), marcuse (1969), and a host of other major social philosophers from nietzsche (1968) onwards drew attention to the way in which enlightenment thinking is now a debased and enslaving hegemony of modernity. but bob is surely true in pointing us to the social and personal benefits of free critical thinking, in freeing 71 ajis vol. 8 no. 2 may 2001 generations of people from the enslaving hegemonies of the debased authoritarian religious and monarchist institutions of the eighteenth century right up to the present. bill: this may be true, but these historical and philosophical currents are rather beyond the scope of our discussion at the moment. i think that both ben and bob are arguing for a balance. eisner (1992) reminds us that "scientific research writing, in the end, is a construction and the more artistic in character, the better." the debate is leading us to move from the narrow metaphor of rigour in research writing back to the fundamental questions of why we research at all. of course, for a lot of academics and researchers, researching is a job and not much more you do what you can get away with and what gives you promotion and acclaim. ben: i also have found instances of this lazy approach, but other persons have something more like a. faith in the scientific or enlightenment project (goudzwaard, 1979). they believe that this rational scientific method will progressively solve our social, cultural and even personal problems all we need is more progress in scientific knowledge and concomitant technological wizardry! this results in a type of positivism which is close to a religious faith (adorno & horkheimer, 1992). mark: i prefer to place my faith elsewhere. bill: hmmm . . . perhaps we should return to the wisdom diagram (fig. 1). using this wisdom hi teachinglearning-researching and writing metaphor, we should ask some key questions. mark: the central question becomes: "does the teacher-learner-researcher-writer demonstrate wisdom?". the corollary questions become; "does the teacher-learner-researcher-writer demonstrate maturity (i.e. balance and thoroughness and rigour) and meaning (a sense of purpose and direction and even, i would argue, a controlling and controlled passion at least an interest for the subject) and virtue (not only in not lying honesty but in 'coming clean' in self-disclosure (moustakas & douglas, 1985, p. 50), revealing one's consciously understood world-view belief systems and interests in the subject as well as what can be gleaned of one's unconscious drives, emotions and motivations; and in having the commitment to continue). at certain times and in certain areas of researching it would be wise to subsume certain areas of maturity (rigour) for the greater purposes of meaning and morality rigour is only part of maturity which is, hi turn, only a part of wisdom. however, as one reviewer commented, someone might say that i use the previous sentence for politically-oriented strategic action to authorise careless research. in the end, i have come to the conclusion that the wise choice is the one that is consistent with what guba and lincoln (1989) term quality, or goodness of the research, or, for heuristic research what moustakas (1990) terms validity. concluding discussion as a result of my heuristic reflection, i did change my teaching practice hi my university business computing courses. i now endeavour to provide rich learning environments in our classes by continuing the initiatives i had incorporated in the first phase, but in a more relaxed and gentle manner. i also incorporate the stretching and mental relaxation exercises suggested by von wodke (1993) into the computer laboratories. o'connor's (1981) research suggests that many people between the ages of 35 to 45 years go through a transitional stage in which attention to psychological development would be of considerable benefit. if teachers or researchers, or anyone for that matter, find themselves operating unethically, perhaps they need some form of inner work. however, i think that most researchers and teachers would benefit by doing some form of reflexive practice and basic self-management training, especially interpersonal communication skills. my tentative conclusion is that the deeper levels of the human psyche, if not recognised and honoured hi dialogue, will (paraphrasing norris, 1992, p. 180) disrupt even (especially?) our most noble and well-framed attempts at educational reform. could it not be that respecting and dialoguing with the deeper levels of the human psyche could lead to a fuller understanding of, and lay the foundations for, more psychologically whole and balanced teaching, researching, writing, and learning (and going wrong?). it is interesting that post-modernism and critical social theory, two challenging influences on recent systems thinking (flood, 1988; flood and jackson, 1991), both draw heavily on freud's metapsychology (osborne, 1992). i understand from postmodernist deconstruction, that any text is open to be understood on many levels. it is not a matter of understanding subconscious motives of the author, but rather attention to the text (or drawing) itself, rather than to the author (poole, r. 1988, pp. 205, 206). deconstruction of a text should not cause the author to feel uncomfortable. it is hi the play of language itself that other meanings in texts inevitably surface. in his book the postcard: from socrtes to freud, derrida (1987) deconstructs a medieval drawing of socrates and plato to bring to the surface the strange ways hi which the human mind defies reductionist understanding. i emphasise that this paper is largely tentative. i autobiographically describe understandings and experiences of 72 ajis vol. 8 no. 2 may 2001 technicism and open discourse in the outer world of the university business computing teaching reform, and in the inner world of my own pedagogical self. this is a complex notion. jung (1968) contends that the dialogue between the outer and inner worlds, the conscious and the unconscious, although of vital importance in almost every area of human pursuit, is often neglected, usually at a cost. i explored this dialogue in the context of reflecting heuristically on a teaching reform in a university business computing course. i think that this conscious-unconscious dialogue is present in any research, and is pervasive in academia, but largely ignored, usually at a cost. for science in general, some would contend that lack of reflectivity, including lack of awareness of the unconscious, lies near the heart of some failures, perhaps the failure, of the scientific project of modernity (adorno & horkheimer, 1990; bowers, 1993 b; habermas, 1972; marcuse, 1964, 1969). what price is the self? or the soul? for some researchers the cost may be high. for myself, i allowed my zeal for teaching reform, based on critical theory, to mask the voices of my students and colleagues, and the voice of the self. i wonder: am i the only teacher or researcher to have succumbed to this temptation? references adorno, t, & horkheimer, m. (1992). the dialectic of enlightenment. london: verso. barry, d. (1996). artful inquiry: a symbolic constructivist framework for social science research. qualitative inquiry 2 (4): 411-438. barry, d. (1994), making the invisible visible: using analogically-based methods to surface unconscious processes in organizations. organizational development journal 12 (4): 37-48. bowers, c. (1988). the cultural dimensions of educational computing: understanding the non-neutrality of technology. new york: teachers college press. bowers, c. (1993a). critical essays on education, modernity, and the recovery of the ecological imperative. new york: teachers college press. bowers, c. (1993b). education, cultural myths and the ecological crisis: towards deep change. albany. new york: state university of new york press. campbell williams, m. (1995). the use of symbo-constructive pictures to explore the shadow side of our teaching-learning. in j. abbott & l. willcoxson (eds.), proceedings of the forum for teaching and learning in higher education (pp. 179-184), murdoch university: perth, western australia. checkland, p. & scholes, j. (1990). soft systems methodology in practice. london: wiley. derrida, jacques. (1987). the post card: from socrates to freud and beyond. (alan bass, trans.). chicago: university of chicago press. dryzek, j. s. (1990). green reason: communicative ethics for the biosphere. environmental ethics 72(3), 195-211. ellul, j. (1964). the technological society. new york: knopf. ellul, j. (1973). the new demons. oxford: alden press. ellul, j. (1990). the technological bluff. michigan, u.s.a.: eerdmans. flood, r. l. (1988). situational complexity, systems modelling and methodology. transactions of the institute of management and control, 10 (3), 130-138 flood, r. l. & jackson, m. c. (1991). creative problem solving. united kingdom: wiley. habermas, j. (1971). towards a rational society (jeremy j. shapiro, trans.). london: heinemann habermas, j. (1972). knowledge and human interests (jeremy j. shapiro, trans.). london: heinemann. hicks, m. (1991). problem solving in business and management. london: chapman and hall. johnson, r. a. (1986). inner work: using dreams and active imagination for personal growth. new york: harpercollins. jung, c. g (1968). approaching the unconscious. in c. j. jung (ed.), man and his symbols (pp. 1-95). aldus books: london. jung, c. g (1989) memories, dreams, reflections (rev. ed.) (richard & clara winston, trans.).new york: vintage books. lewis, c.s. (1978). the abolition of man. london: fount. lewis, c.s. (1979). the great divorce. new york: macmillan. marcuse, h. (1964) one dimensional man. london: routledge and kegan paul. marcuse, h. (1969) eros and civilisation. london: allen lane penguin. moustakas, c. (1990). heuristic research: design, methodology and applications. newbury park: sage. norris, c. (1992). deconstruction, postmodernism and philosophy: habermas on derrida. in david wood (ed.), derrida: a critical reader (pp. 167-192). oxford: blackwell. osborne, richard. (1992). philosophy for beginners. ny: writers and readers. poole, r. (1988). deconstruction. in a. bullock, o. stallybrass & s. trombley (eds.), modern thought: (rev. ed.). glasgow: fontana. standing, c. & campbell williams, m. (1993 a). methodological pluralism for managing departmental change: an interpretive action-research case study within a university information systems department. in a. 73 ajis vol. 8 no. 2 may 2001 herrman & colin latchem (ed.), proceedings of the forum for teaching and learning in higher education (pp. 173-184). perth, western australia: curtin university. standing, c. & campbell williams, m. (1993 b). business re-engineering from an is perspective. paper presented at the information technology update'93 conference, edith cowan university, perth, western australia. standing, c. & campbell williams, m. (1993 c). mernodological pluralism in a rapidly changing it environment. in b. glasson & v. pervan (eds.), proceedings of the international federation of information processing professionals conference (pp. 23-25). perth, western australia: curtin university. taylor, p. & campbell williams, m. (1993). discourse towards balanced rationality in the high school mathematics classroom: ideas from habermas' critical theory. in j. malone & p. taylor (eds). constructivist interpretations of teaching and learning mathematics (pp. 135-148). perth, western australia: curtin university of technology. von wodke, m. (1993). mind over media: creative thinking skills for electronic media. new york: mcgrawhill. williams, m. campbell (1999). soft systems pictures: reforming a university information systems course: discussing a failure and an unusual research approach. in hope, b., & yoong, p. (eds). proceedings of the tenth australasian conference of information systerms, wellington, new zealand, victoria university of wellington. weizenbaum, j. (1977). computer power and human learning. san fransisco: w. h. freedman. 74 microsoft word ajis12-1final repaired.doc ajis vol. 12, no. 1 september 2004 18 securing xml documents charles a. shoniregun school of computing and technology, university of east london, longbridge road, barking, dagenham essex, rm8 2as, uk e-mail: c.shoniregun@uel.ac.uk, cshoniregun@acm.org, c.shoniregun@infonomis.org.uk oleksandr logvynovskiy business, computing & information management, london south bank university, borough road, london se1 olu uk e-mail: a.logvynovskiy@lsbu.ac.uk, a.logvynovskiy@infonomis.org.uk abstract xml (extensible markup language) is becoming the current standard for establishing interoperability on the web. xml data are self-descriptive and syntax-extensible; this makes it very suitable for representation and exchange of semi-structured data, and allows users to define new elements for their specific applications. as a result, the number of documents incorporating this standard is continuously increasing over the web. the processing of xml documents may require a traversal of all document structure and therefore, the cost could be very high. a strong demand for a means of efficient and effective xml processing has posed a new challenge for the database world. this paper discusses a fast and efficient indexing technique for xml documents, and introduces the xml graph numbering scheme. it can be used for indexing and securing graph structure of xml documents. this technique provides an efficient method to speed up xml data processing. furthermore, the paper explores the classification of existing methods impact of query processing, and indexing. keyword: xml data, xml document, numbering scheme, securing graph structure introduction the web and database communities have developed several approaches to resolve this problem. one is to convert semi-structured data into structured. this requires a pre-defined schema and a nontrivial translation of the data. another option uses a partial schema to convert the xml data into structured data, where a partial schema is extracted from the data by using data mining techniques. data that does not fit into this schema, will be stored and queried separately. an xml query combines searching by value and structure. the structure search incorporates navigating through relationships among the elements. the major difficulties of the structure search are full xml document traversals and presence of cross-references that may require a closure computation. as the costs of these operations are significantly high, it is especially important to employ suitable indexes to speed up the searching operations. the processing of xml documents may require a traversal of all document structure and, therefore, the cost could be very high (logvynovskiy and lü, 2003). a strong demand for a means of efficient and effective xml processing has posed a new challenge for the database world. related work within xml databases there are usually two kinds of indexes: value (data) indexes and structure (path) indexes. value indexes deal with coercing values and types of elements while structure indexes are meant to provide access to nodes reachable through certain paths. the lore project uses four indexes: two for value and two for structure. the structure indexes were used to simplify searching of paths of elements and provide access to parents of nodes. the value indexes were designed for type coercion. lore also uses a structure called dataguide as a path index. dataguide (or strong dataguide) is a concise labelled graph representing each path in the data exactly once. dataguide is used for query optimisation and as a path index, though it requires several index lookups (mchugh, 1999), while the stored system applies data mining techniques to discover a ajis vol. 12, no. 1 september 2004 19 partial schema and then converts semistructured data into relations. for the parts of the semistructured data that do not fit the schema are stored separately in native form. cooper et al (2001) have considered an approach of indexing xml paths as strings. they use a structure, called index fabric, analogous to a b-tree, to store paths encoded as stings. this technique has been implemented over relational database management systems (dbmss) and has shown better results than traditional indexing schemes. li and moon (2001) proposed the xiss system to index and store xml data. there are two indexes used for value related data and three indexes used for structure related data. these three indexes (value / structure related data) are element, attribute, and structure indexes. along with the identifier of each node these indexes contain a pair of numbers assigned to the node – extended traversal number and size of the node, which are later used by a selection algorithm for computing of ascendant-descendant relationship without tree traversal. within the xiss system indexes are used for speed up processing regular expression queries. a complex path expression is decomposed into several subsequent simple ones, which are then processed by one of the three join algorithms. the indexing scheme makes path query processing much faster than conventional algorithms. one unique feature of this approach is able to determine the ancestor-descendant relationship quickly and without traversal of the data tree. however, this scheme only deals with data that have tree-like structure. this means the data model of xml documents has to be a tree, i.e. the branches of the tree are independent, and there are no links between nodes of different branches. the structure of the xml document can be considered as either syntactical or logical. in the first case a document is treated as a tree, and in the latter case the document can be viewed as a graph. however, only in some simple cases does an xml document have a logical tree structure where each of its elements has exactly one parent element. in general, the logical relationship between different elements within an xml document is a graph. various models have been proposed with little difference (mchugh, 1999). if directly applied the numbering scheme proposed by li and moon (2001) on xml documents with graphs as their data models, requires significant additional query costs. for example, 0b shows a query that involves selecting node 2 and all of its reference nodes. if the numbering scheme proposed was used then 4 nodes need to be accessed. however, if the reference links can be modelled by the numbering scheme, then only two nodes need to be searched. indeed, we have extended the numbering scheme to handle xml documents with various types of data models, including graphs and trees. extended xml numbering scheme securing graph structure describes the motivation for introducing an extended numbering scheme for xml documents and the details of the algorithms designed to maintain the indexes based on the extended numbering graph scheme. each xml document consists of a number of elements. each element has its name, attributes, and content. names of the elements are represented by tags, which are delimiters of the elements. the ancestor-descendant relationship is represented as nesting of the descendant element within the tags of the ancestor element. due to the fact that the source data have a tree structure, where nodes of the tree correspond to the elements and labelled arcs stand for tags. since such a tree represents the containment relationship between nodes of the document, it is referred to as a containment tree. an example of xml data and its interpretation as a tree are shown on 0a, 1b, and 1c. the containment tree does not cover all semantic relationships between nodes but additional relations among elements can be defined through references or links. the name of the link describes its semantic role and the value of the link contains a path to the referenced node. the path sequence of names provides all ancestors of the node in the containment tree (w3c, 2001). links are defined as the content of value of the attribute or the element. depending on whether one needs to deal with the links as values or relations (arcs), the xml document can be treated either as a tree or as a graph (papakonstantinou et al., 1995)0. ajis vol. 12, no. 1 september 2004 20 a) b) <db id=1> <a id=2> <b id=3 ref=7> </b> <d id=4 ref=1> </d> </a> <a id=5> <b id=6> <c id=7> </c> </b> <d id=8 ref=5> </d> </a> </db> db a a b c b d d r r r 1 2 4 6 8 7 3 5 с) db a a b c b d d 1 2 4 6 8 7 3 5 figure 1 xml source data and its tree and graph representations tree numbering scheme first, let us consider the trees that are involved in the representation of our approach. the idea of using numbering scheme to determine the ancestor-descendant relationship between any two nodes of a tree was firstly introduced by dietz (1982)0. each node of a tree is associated with a pair of numbers. these numbers are calculated during pre-order and post-order traversals of a tree. it is stated that given nodes x and y and their preand post-order traversal numbers (nx pre, nx post) and (ny pre, ny post) respectively, x is an ancestor of y (y is a descendant of x), iff nx pre < ny pre and nx post > ny post. 0, shows an example of numbering of a tree. such an approach to find the relationship between nodes is very useful for indexing of xml trees, since it avoids tree traversals during query execution. the preand post-order numbers of each node can be calculated and stored in a database. there is one disadvantage of this scheme: when a node is inserted, removed, or updated, the recalculation of all numbers will be required. to overcome this drawback, dietz’s numbering scheme was modified li and moon, to calculate and compare pairs of extended pre-order numbers and ranges of descendants (size of sub-tree) of any two nodes within a tree. “extended” means that extra space are reserved for future possible insertions. deletions do not require recalculation either, increasing only the available space within the parent node range. the proposition was: given nodes x and y and their extended pre-order and size numbers (nx order, nx size) and (ny order, ny size) respectively, x is an ancestor of y (y is a descendant of x), iff nx order < ny order and nx order + nx size ≥ ny order + ny size. the extended numbering scheme is applied to the same example shown on 0, b. for simplicity, each node on the example is assigned to the pair (norder, norder + nsize) instead of (norder, nsize) – one can be obtained from another through simple arithmetical transformation. a bar originating from each node visually represents the range of the node. ajis vol. 12, no. 1 september 2004 21 1 2 4 6 8 7 3 5 a) b) 1 2 4 6 8 7 3 5 (1, 100) (10, 40) (15, 19) (30, 35) (50, 90) (60, 75) (65, 71) (80, 87) (1, 8) (2, 3) (3, 1) (4, 2) (5, 7) (6, 5) (7, 4) (8, 6) figure 2 numbering and extended numbering of the xml tree. this numbering scheme was used to implement indexes in xiss project and had shown improved performance for processing queries. graph numbering scheme we consider the same xml document as a logical graph (it is illustrated in 0, a). the graph is constructed based the numbered tree by interpreting links among its nodes. such a numbering system could not be able to explicitly specify the ancestor-descendant relationship. for example, the link o4 → o1 would create a cycle in the graph that makes the node o1 an ancestor and a descendant of the node o4 simultaneously. 1 2 4 6 8 7 3 5 a) {(1, 100)} {(10, 40)(1, 100)} {(15, 19)(65, 71)} {(50, 90)} {(60, 75)} {(65, 71)} {(80, 87)(50, 90)} b) 1 2 4 6 8 7 3 5 (1, 100) (10, 40) (15, 19) (30, 35) (50, 90) (60, 75) (65, 71) (80, 87) {(30, 35) (1, 100)} figure 3 numbering and extended numbering of the xml graph. to resolve this problem, we propose a xml graph numbering scheme (xns) that assigns a set of order-range pairs to a node instead of the single pair. the numbering set {(nx order, n x size)} of each node x of the graph consists of: • a primary numbering pair (nx order, nx size) that is assigned to the node x during extended numbering of the xml containment tree. this pair is denoted as (n x x order, n x x size); • a linked numbering pair is referred to as a primary numbering pair (ny order, ny size) of any node y that is linked by x directly or indirectly (through its descendants). such a pair is denoted as (n x y order, n x y size). the new linked pair (ny order, ny size) will be included into the numbering set only if it refers to the range not reachable through any other pair in the set, i.e. interval [ny order, ny order + ny size] is not contained in any interval [n x i order, n x i order + n x i size] from the set {(n x order, n x size)}. note that the ajis vol. 12, no. 1 september 2004 22 numbering set of each node will never be empty and consists at least of one numbering pair, the primary numbering pair. the ancestor-descendant relationship between two nodes is determined as follows. given nodes x and y and their numbering sets {(n x order, n x size)} and {(n y order, n y size)} respectively, x is an ancestor of y (y is a descendant of x), iff ∃i, (n x i order, n x i size) ∈ {(n x order, n x size)}, that n x i order < n y y order and n x i order + n x i size ≥ n y y order + n y y size. in other words, there exist such a node i, which is referenced by x and the ancestor of the node y. an example of the graph numbering is given on 0, b. the primary pairs are shown in bold. node o8 has the linked pair (50, 90), corresponding to node o5 and representing the link: o8 → o5. nodes o2, o4 have the linked pair (1, 100), corresponding to node o1 and representing the reference o4 → o1 (node o2 as parent of o4). lastly, node o3 has the linked pair (65, 71), corresponding to node o7 and representing the reference o3 → o7. nodes o2, o4 are ancestors of the node o3 but do not have the linked pair (65, 71) as they can reach node o7 via their linked pair (1, 100). the root node o1 has not any linked pairs: every node of the graph is reachable through its primary pair. algorithms this section describes algorithms designed to maintain index records for cross-references (links) of xml graph. each record of the index has such a form [ox, oy, (ny, sy), add_info], where oy, ny, sy are the identifier, the extended pre-order, and the size of the node y corresponding to the linked pair (n x y order, n x y size) of the node x. and add_info is the additional information stored in index, like ids of the parent, first children, sibling etc. for simplicity, we omit add_info in the following discussion. the process of creating an index includes into two steps: 1. create the index of the xml graph nodes from the source data. this step will produce index records for primary pairs of each node of the graph; 2. create index records for linked pairs of each node of the graph. the pseudo-code of the algorithm is described in 2 below: algorithm 1 create index input: source xml data file output: index of the numbered xml tree //building tree index (records for primary pairs of each node) 1: for each input element of the xml file do generate id, extended pre-order and size numbers oi, ni, si add index record [oi, oi, (ni, si)] end for; //resolving links (records for linked pairs of each node) 2: for each node oi of the tree do for each link oi → oj do insert link oi, oj end for end for. in the first step, the input source xml document is treated as a containment tree. the id o, the extended pre-order number n and the size s are generated for each node. then the record [o, o, (n, s)] is stored in a table. this record corresponds to the primary pair of the node. in the second step, the data is considered as a graph. each node of the graph will be iteratively visited to resolve every reference (link) of the node. the appropriate records for the linked pairs of the node will be recorded. the insert link algorithm is designed to carry out the operations involved in this step, which is described in figure 3 a. each of the nodes in the graph has its primary pair. let us start with adding link o3 → o7. first, ancestors records of the node o3 (including itself) are selected, including a = {[o1, o1, (1, 100)], [o2, o2, (10, 40)], [o3, o3, (15, 19)]}. then select the set of linked pairs of the ajis vol. 12, no. 1 september 2004 23 node o7: d = {[o7, o7, (65, 71)]}. for each ancestor a ∈ a, a record [a, o7, (65, 71)] is to be added in the index, if the node o7 is not already a descendant of a. thus the records [o2, o7, (65, 71)] and [o3, o7, (65, 71)] are added, and no record for the node o1 will be added, since o7 is already its descendant. next, let we add link o4 → o1. the ancestors records of the node o4 (including itself) are selected, including a = {[o1, o1, (1, 100)], [o2, o2, (10, 40)], [o2, o7, (65, 71)]}. then select the set of link pairs of the node o1: d = {[o1, o1, (1, 100)]}. for each ancestor a ∈ a, a record [a, o1, (1, 100)] is to be added in the index, if the node o1 is not already a descendant of a. the records [o2, o1, (1, 100)] and [o4, o1, (1, 100)] are added, and no duplicate record for the node o1 need to be added. the record [o2, o7, (65, 71)] will be removed since it is overlapped by the record [o2, o1, (1, 100)]. when we add the link o8 → o5, the ancestors records of the node o8 (including itself) are selected, including a = {[o1, o1, (1, 100)], [o5, o5, (50, 90)], [o8, o8, (80, 87)]}. then select the set of link pairs of the node o5: d = {[o5, o5, (50, 90)]}. for each ancestor a ∈ a, a record [a, o5, (50, 90)] is to be added in the index, if the node o5 is not already a descendant of a. thus, only the record [o8, o5, (50, 90)] is added, and no records for nodes o1 and o5 need to bee added. the final result is shown on 0, b and the algorithm 2 below shows the insert link. algorithm 2 insert link input: index, referencing node oi and referenced node oj output: index updated with records referencing node oj //select ancestor records of the node oi //note, that this selection includes oi 1: select index records for ancestors of the node oi: a = {[op, or, (nr, sr)]}, such that op is the parent of oi; //select records of the node oj 2: select index records for of the node oj: d = {[oj, ok, (nk, sk)]}; // 3: for each record [oj, ok, (nk, sk)] ∈ d do // for each record [op, or, (nr, sr)] ∈ a do // ancestor ← op; addnode ← true; // while ancestor = op do // if (or is parent of ok) then addnode ← true; // if (ok is parent of or) and (op ≠ or) then delete record [op, or, (nr, sr)]; end while // if addnode = true then insert record [op, ok, (nk, sk)]; end for end for ajis vol. 12, no. 1 september 2004 24 the algorithm for deletion the link oi → oj is straightforward: to delete all records of the form [op, oj, (nj, sj)], where op is the ancestor of oi. this is to remove the records for links from parents of the node oi to oj and from oi node itself to oj. the pseudo-code of the algorithm is described in algorithm 3 delete link input: index, referencing node oi and referenced node oj output: index updated with records referencing node oj 1: select index records for ancestors of the node oi: a = {[op, oj, (nj, sj)]}, such that op is the parent of oi //select records of the node oj 2: for each record [op, oj, (nj, sj)] ∈ a do // delete record [op, oj, (nj, sj)]; end for; analysis to assess the usability of this indexing method, we focused on the growth of index depending on the number of cross-reference links in xml documents. because if the indexes growth dramatically and require enormous storage space, this method would not be applicable. we have chosen two realworld data sets (digital bibliography and library project (dblp), and internet movie database (imdb)) to estimate the number of records in their indexes: • dblp contains computer science bibliography information. each dblp file represents a single publication. all the documents are grouped into classes of publication (book, conference paper, journal article, etc.). • imdb is a highly cyclic semistructured database with information about movies, actors, directors, producers, writers, etc. imdb includes 49 files (>500mb). we chose a subset of two files (movies, actors). the characteristics of the dblp and imdb data sets are summarised in error! reference source not found. below. data set size (mb) files elements links dblp 96.6 251,520 2,095,552 25,745 imdb 88.1 2 2,221,423 1,624,311 table 1 xml data sets to estimate the index size s for a database, it is necessary to find numbers of records corresponding to primary nprimary and linked nlinked numbering pairs of each node: s = nprimary + nlinked. the number of primary pairs nprimary is equal to the number of elements in the database n. the number of linked pairs nlinked depends on the number of referencing nodes nref and how many ancestors precede these referencing nodes (according to the 0). the number of preceding ancestors of any node is equal to the path length of this node li. consequently, the number of linked pairs nlinked equals: nlinked = σ li, i = [1, nref]. if the path lengths of nodes are approximately equal to each other, the previous statement turns into the following: nlinked = l · nref, ajis vol. 12, no. 1 september 2004 25 where l is the average path length of referencing nodes; nref is the number of referencing nodes. the number of referencing nodes nref depends on the topology of the graph, since every link increases the number of ancestors for the referenced node. for example, the number of ancestors of the node o7 in 0: numbering and extended numbering of the xml tree, a is equal to 3 (nodes o1, o5, o6), while the number of ancestors of the same node o7 in the 0: numbering and extended numbering of the xml graph, a is equal to 7 (nodes o1 to o6). first, we consider the topology of dblp data. a dblp document contains information about the type of publication, the title, the authors, and so on. it may also include a reference to other publication in the database. each reference is given as an absolute path from the entry directory. the length of any path does not exceed 3 and has form «class of publication/issue title/document name». so the average path length of referencing nodes is ldblp = 3. the approximate topology of the dblp dataset is shown on 0, a and b. it is assumed that there are no cross-references between any two documents, i.e. if book a has reference to book b, then book b can not have references to the book b. “class of publication” “issue title” “document name” “cite” l n a) b) “actor” “movie” … lmovie lactor n nactor nmovie figure 4 approximate topology of dblp data the index size s for the dblp database is: s = ndblp + ldblp · nbdlpref, where ndblp, ldblp, nbdlpref are the number of elements, the average path length of referencing nodes and the number of referencing nodes in the dblp data set. as a result of the number of elements, the increase of the index size of dblp is a linear function and directly proportional to the number of references. the statistical results regarding the dblp data set are summarised in table 2 below: data set number of primary pairs records, nprimary = ndblp number of references, nref number of linked pairs, nlinked (ldblp = 3) growth of index, % dblp 2,095,552 1,000 3,000 0.1 publications:251,520 10,000 30,000 1.4 100,000 300,000 14.3 25,745 (present in db) 77,235 3.7 table 2 estimation of index growth for dblp data set since the average path length of referencing nodes in dblp is low, the index size tends to grow slowly too. ajis vol. 12, no. 1 september 2004 26 now consider the topology of imdb data. there are several different types of imdb documents: movies, actors, directors, etc. each of the types has different structure and references to the documents of other types. the database contains many cross-references between documents. for example, “the matrix” movie has links to the actors “keanu reeves” and “laurence fishburne”. these actor documents have cross-reference to the “the matrix” movie. we have selected the subset of the imdb of two types (movie titles and actors). the approximate topology of the imdb subset is shown on 0, b. each type of documents has its average path length of referencing nodes lmovie and lactor, and number of referencing nodes nmovie and nactor respectively. if an average number of actors in each movie is denoted nactor, then each of the nactor actor nodes will have (nactor – 1) linked pairs to other actors and linked pairs to the movie. the movie node will have nactor pairs for the actors. thus, the number of linked pairs for one movie node and its ascendants is lmovie ·nactor and the number of linked pairs for one actor node and its ascendants is lactor · ((nactor – 1) + 1) = lactor · nactor. the total of linked pairs is: nlinked = nmovie (lmovie ·nactor + lactor · n 2 actor), where nmovie, lmovie are the number of elements, the average path length of movie referencing nodes; nactor, lactor are the average number of actors and the average path length of actor referencing nodes. even in the case of two types of documents, the index size grows non-linear. the statistical results regarding the dblp data set are summarised table 3 below: data set number of primary pairs records, nimdb number of references, nmovie number of linked pairs, nlinked (nactor= 20, lmovie= 5, lactor = 3) growth of index, % imdb 2,221,423 1,000 1,215,000 54.7 movies: 229,180 10,000 12,150,000 546.9 actors: 367, 932 100,000 121,500,000 5469.4 1,624,311 (present in db) 1,973,537,865 88841.1 table 3 estimation of index growth for imdb data set conclusion our results showed that the indexing method introduced in this paper is efficient in cases where the number of cross-reference is relatively low as a result of securing the graph structure. however, in the case of circular references or where a large number of cross-referencing is involved, the number of index records will be increased at a non-linear rate. indeed, the problems of internet security cannot be ignored by companies as this would result in the loss of competitive advantage in the market place and what the future holds for internet security technology cannot be predicted to the rate technology is advancing (shoniregun, 2002). references b. f. cooper, n sample, m. j. franklin, g. r. hjaltason, and m. shadmon (2001) a fast index for semistructured data. in proceedings of the 27th vldb conference, roma, italy p. f. dietz (1982) maintaining order in a linked list. in proceedings of the 14th annual acm symposium on theory of computing, san francisco, california. r. goldman, j. widom (1997) dataguide: enabling query formulation and optimization in semistructured databases. in vldb. q. li, b. moon (2001) indexing and querying xml data for regular path expressions. in proceedings of the 27th vldb conference, roma, italy. o. logvynovskiy , k. lü (2003), structural sequence join for xml regular path expressions, prep’03, exeter, uk, april. ajis vol. 12, no. 1 september 2004 27 j. mchugh, j. widom (1999) query optimization for xml. in proceedings of the 27th vldb conference. y. papakonstantinou, h. garcia-molina, j. widom (1995) object exchange across heterogeneous information sources. in proceedings of the 11th international conference on data engineering. shoniregun c.a. (2002), the future of internet security, acm ubiquity: information technology (it) magazine and forum, volume 3, issue 37, oct 29. w3c (2001) xml query language (xquery) 1.0. w3c recommendation, december 20, http://www.w3.org/tr/xquery/ (assessed date 21 may 2002). microsoft word ajis12-1final repaired.doc ajis vol. 12, no. 1 september 2004 103 exploring virtual enterprises using activity theory ronald c beckett the reinvention network sydney, australia e-mail: rcb@reinvent.net.au abstract from joint industry – university research into the key success factors that sustain virtual enterprises (ve’s), it has been observed that conditions for the establishment of a ve do not necessarily sustain it. transactions that are valued by the participants, a balance between similarity and complementarity attributes of the participants and effective utilisation of time are seen as important sustaining factors. different communication and ict tools are utilised in different circumstances, and successful collaborations develop formal or informal “rules” underlying their operation. considering such attributes helps us understand what happens within a successful ve, but not necessarily how. two case studies describing long-standing collaborations – one relating to a manufacturing network, and the other to a professional virtual community are presented. a third case study relating to the development of ict tools to be used in a ve is also presented. activity theory is used as a framework for discussion of the organisational attributes associated with these cases and how they operate. it is suggested that exploration of the nature of tools, communities etc is helpful in understanding ve operations, and applied at multiple levels, activity theory is also useful in understanding the evolution of those tools, communities etc. keywords virtual enterprise, activity theory, collaboration introduction network forms of organisation within large firms and inter-firm collaborations to form virtual enterprises (ve) are becoming more common as a means of competing more effectively in a globalised business world (eg wiendahl, englebrecht and hamacher, 2001), but the success rate of some of these initiatives seems low (eg marceau, 1999). from previous research into the key success factors that sustain such organisations, it has been observed that conditions for the establishment of a ve do not necessarily sustain it. transactions that are valued by the participants, a balance between similarity and complementarity attributes of the participants and effective utilisation of time are seen as important sustaining factors (beckett, hyland and sloan, 2003). identifying common goals, establishing consistent ways to do business and successfully engaging with the external environment are seen as essential establishment factors (marceau, 1999). different communication and information and communications technology (ict) tools are utilised in different circumstances, and successful collaborations develop formal or informal “rules” underpinning their operation. there are a lot of factors that characterise a ve, potentially making comparison between difference instances and their design difficult as there would seem to be many contingent variables. in this paper we look at one approach, activity theory, that links attributes such as goals, tools and rules to consistently make such a characterisation. two case studies describing long-standing collaborations – one relating to a manufacturing network, and the other to a professional virtual community are presented, along with a third case study describing an organisation set up just for one project: to conduct research into ve management methodologies and ict tools. activity theory is used as a framework for discussion of the organisational attributes associated with these cases. it is suggested that exploration of the nature of tools, communities etc is helpful in understanding ve operations, and can be applied at multiple levels. in this latter context, activity theory is also useful in understanding the evolution of those tools, communities etc and how many intersecting networks might link together. ajis vol. 12, no. 1 september 2004 104 case studies a manufacturing network a study of australian toolmaking innovations pty ltd (ati) was carried out using both a questionnaire and interviews. both the questionnaire and the interview structure built on previous work by fulop and kelly (1997). ati is a virtual enterprise acting as a shopfront for a network of toolmakers, and it has been in existence about eight years. the primary goal of the ati network was acquiring production orders from overseas. as the individual companies could not do this successfully due to their limited size, they formed a network to enter the export market as a larger entity. a secondary goal was to make the members stronger in the domestic market. the member firms in this network are all smes with between 20 and 100 employees. the driving force in the network was one of the two larger firms in the network. the latter is larger than the average firm because of activities other than toolmaking that are undertaken in that firm. as the age of the firms in the network varies from 19 to 35 years this can be regarded as a network of well-established firms. the managers interviewed described the domestic market growth potential of their toolmaking businesses as limited. only the lead firm was expanding its factory floor for toolmaking. some of the other network members have been downsizing their toolmaking businesses and have been looking for growth in other divisions. there are presently 5 firms in the network, all located in the southern sydney region. members find the area where the network members are located important mainly due to the ease of communication, and secondly for inter-firm transportation of products. there is a full time manager who operates independently, but is funded by the network partners. a board of partners provides general direction and strategy. there have been three different managers to date. the network members describe similarities in respect of the following characteristics: • product or service: all members do the same type of tooling (specifically plastic tooling) of high quality. within that sector each has a speciality. • clientele or customers: the type of customer is similar for all members the majority are automotive companies. members have their own customer base and rarely have the same company as a customer. customers all are demanding and want a wide range of services. • mode of operation: the members are the biggest toolmakers in the region and all very progressive in toolmaking. the mode of operation of the members is very much the same in the way that they all quote for orders, have similar operating methods (for example they all use cad/cam), and operate the same types of tools and machines. all members are privately owned companies. • territory or market: for the ati network two types of market can be distinguished. these are the domestic market and the export market. all members operate in the same domestic market (sydney, new south wales, and victoria). they have acquired their own overseas markets, although these are of a minor scale. the firms aimed to expand into the international market in collaboration using the network’s capacity to win large orders. • relationships that developed between participants resulted in joint purchasing, sharing of resources and capacity to meet peak demands of long-term domestic customers, and in information sharing on new technologies. the members have complementary capacity and machines. every member has some specialization in a market sector. members can share work with each other in confidence of quality and reliability. this gives the network a wide range of services to offer in tooling. whilst the network has reduced its internal costs, and increased revenue by sharing some domestic market projects, the level of revenue hoped for from export work was not initially achieved. however, in recent years, following deployment of the full time network manager to china for lengthy periods, business has improved, and the ve is profitable in its own right on the basis of export orders. the experience has lead to the partners considering joint ownership of assets and new business ventures. experience gained in collaboration has also resulted in one of the partners to call on the others to provide capacity in support of entry into a new market sector. ajis vol. 12, no. 1 september 2004 105 a professional virtual community this network was studied using a questionnaire as a checklist to adopt a consistent approach to individual interviews. it is an informal innovation network of industrial designers, engineers, tool designers and tool manufacturers tending to specialise in the development of unique plastic product designs that might also require the development of unique production processes. activities commenced in 1986 with just a few companies, and now it has about 15 participants distributed across five states in australia. each participant has less than ten employees, and the main criteria for participation seems to be demonstrated excellence in a specialisation, ease of collaboration and demonstrated contractually compliant performance. new business opportunities may be identified by any member who may then take the lead position for that project, or an opportunity may be referred to a more appropriately specialised partner for follow-up and leadership. the network arrangement enables members to offer turnkey product design and tooling packages to clients rather than just part of a package. the maintenance of a reputation for innovation and reliability is an important consideration for each individual participant, as in taking a lead position; they become interdependent on the others to maintain that reputation. the network itself is not a visible entity. membership has changed over time as more specialisations have been added, and as difficulties have arisen with either ease of collaboration or contract compliance with past members. one of the members who has a strong orientation towards collaborative arrangements arising from prior experience in europe appears to be the network champion, and he sets standards and protocols (eg everyone uses microsoft project software for project management). although the network structure is informal, projects are formally operated with clearly defined roles and tasks. this network is based on accessing and sharing intellectual assets rather than physical assets, but it is in place to expand the market access of the participants rather than to share knowledge. participants have noted some expansion of personal networks as a benefit. technology intelligence and operational intelligence are seen as the main beneficial knowledge transactions. some members of the network are considering establishing a more formal arrangement to develop their own products and brand. an international research project in january 2000 an industry-led international collaborative research project was launched by about 20 organisations in australia, europe and japan to identify ict tools and methodologies supporting inter-enterprise integration and collaboration in one-of-a-kind production. the project participants had some common interest in this kind of production, and there were various complementary skills amongst the participating organisations. the author was a participant in this project for some time, and this description draws on project publications. the project was completed in december 2002. concepts of appropriate reference architectures, applications in distributed engineering, and sales and service, some issues related to planning and scheduling, and perceived partner benefits from participation were all presented in a final symposium (globemen, 2002). some of the project participants had worked together on a previous project, but some had not been involved in such an activity before. some participants withdrew, and some new ones joined the team over the life of the project. a comprehensive project implementation plan was agreed upon prior to project launch, with clear definition of work packages, lead partners for each work package, participant contributions to each work package, deliverables and progress milestone/reporting requirements. this had been a requirement of both the umbrella project organisation (www.ims.org) and one of the funding authorities. one of the participants established a web site with both public access and member-only access arrangements that formed a data repository for formally submitted project discussion papers and deliverable documents (http://globemen.vtt.fi). each of the three geographic regions took a turn in providing a secretariat function to facilitate the organisation of events, production of deliverables and progress reporting. face-to-face workshops were held about every six months, with informal regional meetings in between. this provided a venue for participants to get to know each other better and for debate of different views. ajis vol. 12, no. 1 september 2004 106 tools and methodologies to support people networking in a global multicultural environment were sought with a focus on: • inter-enterprise information exchange and control • interaction with customers and users involving global product life-cycle management • optimisation of the delivery chain through inter-enterprise resource planning • distributed concurrent engineering in the one-of-a-kind environment being investigated, a new virtual enterprise had to be constructed every time a large project was undertaken, and through the life of the project responsibility for interaction with the team and the client might change, for example as the emphasis moved from engineering design to after-sales support of a manufacturing plant constructed by the partners. this led to the adoption by the research team of a project life cycle oriented virtual enterprise reference architecture and to the use of some standard workflow analysis tools. whilst the project focus was on the development of tools and methodologies, a number of human factors kept arising, such as how to develop trust between new partners when a new virtual enterprise was rapidly formed. a formal process was established for measuring the success of the research project. project performance indicators relating to deliverables and participation were developed, and individual partner success indicators were documented. during operation of the research project, benefits were noted from the requirement to report on success factors – participants were forced to reflect on what was being achieved and where it was leading; but it was also noted that the activity could be timeconsuming. the participants were each asked to identify success indicators from the viewpoint of their individual organisations. this action, in effect, sought to establish what value-adding transactions were expected. discussion some research related to ve’s has focussed on ict tools that facilitate sharing data and knowledge between remote partners. other research has focussed on human factors such as trust. one approach being used to better understand human – computer interaction is to adopt the application of activity theory, and in the following discussion this will be applied to the three cases to illustrate the utility of such application. activity theory (vygotsky, 1978, engestrom, 1987) is used in this paper to provide a framework for characterising the interplay of behaviours and tools that may be associated with successful virtual enterprise operations. a virtual enterprise is formed to do something as a collective activity that is achieved through the efforts of individual organisations and people. activity theory derives from studies of how people think in the context of undertaking an activity, and it is argued that their environment influences the way they think and act. in undertaking broad activities driven by some motivation, [object] people decide to take an action [subject] to achieve a goal requiring operations [community] to be implemented through a series of tasks. how this is enacted is influenced by [tools], [rules] and the [division of labour]. such a socially distributed activity system is illustrated in figure 1. ajis vol. 12, no. 1 september 2004 107 tools object rules division of labour outcomesubject community a model of work activity figure 1:a socially distributed activity system (from engestrom, 1987) in table 1 below, each of the case studies has been mapped against each component of the system presented in figure 1 to provide a holistic view of the cases. the third case is different in that it was never intended to be an enduring arrangement, but its outcomes were intended to be used as tools for use in subsequent ve’s. kutti and molin-juustila (1998) used the notion of hierarchy to explore linked activity systems associated with each element of a greater system. for example, in the third case, the topic “tools” is made the object, and illustrates how the development of ict tools, and the rules, community etc associated with that object/motive can be described in the same way as the first two cases. in a similar way, the topic “community” could be made the object to explore how single project virtual enterprises might be formed from a pre-existing network of potential partners, or the topic “division of labour” could be made the topic to understand how a project implementation plan might be developed. ajis vol. 12, no. 1 september 2004 108 activity theory element manufacturing network professional virtual community globemen international research project object -the motivation is to access scale to remain competitive as a strategic imperative -the motivation is to access complementary assets to access broader market opportunities the motivation is to identify and demonstrate methodologies and ict tools that will support the rapid creation and operation of a global virtual enterprise to undertake one-of-a-kind projects subject -action is through individual direct or indirect project opportunities that utilise the toolmaking knowledge of the partners and their supply chains -action is through individual projects and crossreferral opportunities generally related to specialised plastics product design and development action through research and development activities undertaken by individual partners according to assignments in the project implementation plan community -five established toolmaking companies undertaking individual projects either through the ve as project manager, or by direct negotiation between individual partners -14 to 20 informal participants establishing individual projects involving some of them under the umbrella of an individual lead company nineteen participants from australia, europe and japan, some from business enterprises, some from research institutes and some from universities. some participants had worked together before. plenary meetings where individuals could interact at the personal level were seen as important to both team cohesion and to the debate of different views tools -partner project meetings and teleconferencing facilitates interaction with remote clients -cad-cam technologies support exchanges of complex data -e-mail & teleconferencing -microsoft “project” -cad-cam files -data transferred using step or pdf files e-mail used extensively. a server set up in finland as a repository for all; project data. common formats for deliverable documents determined. common virtual enterprise reference architecture and workflow modelling tools adopted rules -formal ve company structure requires board meetings, financial reporting etc -competency –based trust -relationship – based trust -informal rules developed around: ----competency – based trust -contractual – based trust -relationship – based trust project sponsors required standard forms of agreement and a comprehensive project implementation plan. key events were scheduled to support direct interpersonal interaction between the participants. project success factors were identified up front and mapped as the project proceeded. division of labour ati manager acts as project manager, determining who will do what tasks, depending on task technical requirements and on available capacity. the ati manager may contract outside the network if necessary. the client may be internal or external. -lead company with the client interface responsibility determines who will do what, depending on the competencies needed and the availability of resources a project implementation plan defined who was doing what and when. how things were done was left up to the individual participants within the agreed rules. a coordination / leadership function rotated between regional partner groups table 1: a characterisation of two enduring virtual enterprises using activity theory element 109 one issue observed by the author in previous research into network organisations is the need to find some common “language” and a consistent way of doing things between partners. it is suggested here that an activity theory based approach may be used to provide such a framework, and its application by the author to the development of a new network currently in the formative stages looks promising. whilst potential industry participants first baulked at the expression “activity theory”, once discussion of how the issues arising could be classified as relating to tools/community/rules etc, the concept was readily accepted. comparing the three cases using this framework, a number of similarities are evident: the object is to do something that the partners could probably not do on their own; subject action is via the efforts of individual partners working on well defined projects; the subjects engage a wider community in their background activities; there is a common use of it enabling tools; there are formal or informal rules related to governance and partner expectations, and whilst there are different approaches to the division of labour, leadership in relation to its organisation is seen as important. concluding remarks virtual enterprises ve’s) have few resources of their own, but draw on the combined resources of participants and their networks to achieve a shared goal. in previous work (beckett, hyland and sloan, 2003) a study of the nature of transactions between the participants had proven helpful in understanding ve success factors. other views (eg vesterager, larsen, pedersen, tolle, m and bernus, 2001) consider project oriented reference architectures to define a framework for operations, or consider the actors, their resources and linkages (eg todeva, 2001). these views help understand what is typically happening in the operation of ve’s, but only partially reveal how this happens. it is suggested here that the application of a socially distributed systems framework developed from the concepts of activity theory can help understand how different kinds of ve’s operate, drawing out similarities and differences between them. in addition, an example of hierarchy is discussed drawing on the ideas of kutti and molin-juustila (1998) illustrating how the framework can be used to describe how each element of the higher level one, such as tools, might be developed. there is a lot of enthusiasm for the use of sophisticated ict tools to enhance intra and interenterprise collaboration, and the case examples indicate they are important to the operation of a ve. but the success or failure of a ve is not generally due to the tools used. success is related to a focus on common goals and people dynamics. failure is more often related to a failure to develop sensible working rules, or poor division of labour, or failure to engage with the internal or external community. an activity theory approach supports the consideration of these factors. the author is currently using this approach in the design of new collaborations, with promising results to date. references beckett, r.c; hyland, p and sloan, t (2003) “mapping collaborative transactions in networks that yield business benefits” proc 19 th annual imp conference, university of lugano, switzerland, 4-6 september (isbn 82-7042-576-1) engestrom, y. (1987) “learning by expanding: an activity-theoretical approach to developmental research” helsinki: orienta-konsultant foss, n. (1999) networks, capabilities and competitive advantage. scandinavian journal of management 15 pp1-15 fulop, l and kelly, j. (1997) “a study of business networks in australia” sydney: australian business chamber hasan, h; gould, e; larkin, p and vrazalic, l (eds) (2001) “information systems and activity theory: volume 2 theory and practice” university of wollongong press, australia (isbn 086418-6967) kuutti, k and molin-juustila, t (1998) “system support for ‘loose’coordination in a network organisation: an activity-theory perspective”in “information systems and activity theory: tools in context” edited by h. hasan, e.gould and p.hyland. university of wollongong press, australia (isbn 0 947127 02 x) ajis vol. 12, no. 1 september 2004 110 marceau, j., (1999) ‘networks of innovation, networks of production and networks of marketing: collaboration and competition in the biomedical and toolmaking industries in australia’ creativity and innovation management, vol 8, no 1, march, pp20-27 simmie, j., sennett, j., wood, p. and hart, d. (2002) innovation in europe: a tale of networks, knowledge and trade in five cities. regional studies, vol 36 no 1 pp 47-64 todeva, e., (2001) ‘the explanatory power of network theory and its application to multi-national business relationships’ proceedings of apros (asia-pacific research in organisation studies), hong kong baptist university, 3-5 december vesterager, j; larsen, l.b; pedersen, j.d; tolle, m and bernus, p (2001) “use of geram as basis for a virtual enterprise framework model” in mo, j.p.t and nemes, l (eds) “global engineering, manufacturing and enterprise networks”, kluwer academic publishers, the netherlands, pp61-65 vygotsky, l.s. (1978) “mind and society” harvard university press wiendahl, e.h, englebrecht, a and hamacher, o (2001) “from single enterprises to complementary networks” in mo, j.p.t and nemes, l (eds) “global engineering, manufacturing and enterprise networks”, kluwer academic publishers, the netherlands, pp 66-73 microsoft word ajisvol10 1_final_edit.doc ajis vol. 10 no. 1 september 2002 66 have smes benefited from e-commerce? simpson poon spoon@csu.edu.au charles sturt university wagga wagga, nsw 2650 abstract recently there has been a surge in research projects on how small and medium enterprises (smes) use ecommerce/e-business for business purposes. these include a mixture of academic, industry and government studies. the outcomes have presented a mixed message. while there are anecdotal success stories, smes found their ecommerce/e-business endeavours successful to a point but face difficulties in achieving many speculations portrayed by the media and visionaries. in this paper, i try to address the issue based on published works in the domain of ecommerce/e-business6 adoption among smes. the answer may lie in a more refined research strategy and consolidation of research outcomes. lessons from edi adoption might be useful to formulate impetus to further adoption. introduction over the last few years, finding out how smes can benefit from e-commerce/e-business has been a popular research and discussion topic. academe (e.g., poon, 1999; tetteh, 1999; mehrtens, et al., 2001), industry (e.g., brulles, 2001; yellow pages business index, 2002) and governments (e.g., oecd, 1998; noie, 2002) are all interested in facilitating the diffusion of e-commerce/e-business among this group of firms due to their strategic importance to many national economies. since the dotcom crash in the year 2000, many of the myths about e-commerce/e-business, its future and the potential impacts on the business world were re-evaluated. the exponential growth of adoption, the everlasting growth of market share and infinite price/earnings ratios are no longer true. coupled with the huge losses of wealth in the share market and the massive it investments, ecommerce/e-business has become a ‘taboo’ in many parts of the business world. for small businesses who were already sceptics after their initial try-out of e-commerce/e-business, many became very withdrawn from adopting e-commerce/e-business. some small firms were enthusiastic in the beginning and engaged consultants to promote e-commerce/e-business but only ended up being disappointed (bode, 2002). viewed differently, the downturn of the e-commerce/e-business might have been tragic from the financial sector perspective, but it is a golden opportunity for small businesses to accelerate the uptake of e-commerce/ebusiness. there are a number of reasons for this: the costs of adopting e-commerce/e-business have gone down and this includes technical talents, hardware and software. what needed to be custom-built is now readily available as solutions. the downturn of the it sector also means a larger pool of affordable it talents is now available. this is important, as many smes do not have the same amount of resources for e-commerce/e-business as their large corporation counterparts. second, more lessons have been learned over the last few years about what e-commerce/e-business can achieve and the barriers/costs of achieving it. these lessons help smes to avoid paths and solutions which are problematic and infeasible. consequently, smes can strategically allocate resources to target workable solutions based on others’ lessons learned. third, more programs and resources are now available to support further adoption of e-commerce/e-business. although first-mover advantage had been cited to explain the success of a handful of e-commerce/e-business ventures (e.g., e-bay), in the case of smes, first-movers to implement full-fledged e-commerce/e-business solutions had proven to be a costly, sometimes fatal business exercise. not only first-movers would have paid a high-price for their solutions, but also they became obsolete quickly (except if their e-commerce/e-business solution was merely email and a webpage). bode (2002) mentions how some small firms engaged so called ecommerce/e-business consultant to implement solutions only ended up being over-charged and disappointed. this should not be a consequence of adopting e-commerce/e-business. for the rest of this paper, i discuss why e-commerce/e-business adoption among smes has been a mixed success and what might be needed to further diffuse e-commerce/e-business and encourage more advanced deployment among smes in the future. 6 instead of defining the differences between e-commerce and e-business, the author treats the two as a combined term. ajis vol. 10 no. 1 september 2002 67 smes – an homogenous group with strong heterogeneity studies on e-commerce/e-business use and adoption among smes often started on different assumptions of what smes are. on one hand there are definitions by governments (e.g., holmes and gibson, 2001) which define smes based on number of persons employed, turnover and/or ownership structure of a business followed by further sub-classifications such as industry sector (e.g., manufacturing vs. non-manufacturing, agricultural vs. non-agricultural). by assuming smes within these categories are largely similar in their business processes, market orientation, management structure and having a similar supply chain often led to variation in results. for example, smes with 20 persons in a manufacturing industry, targets only the local city, might have a very different information technology (it) investment profile compared to a 20-person firm in the software industry and has a worldwide market. the e-commerce/e-business orientation, the experience and perception of ecommerce/e-business success will logically be different. to ensure results are comparable between different studies, it is increasingly important to have smes samples not just classified by number of employees, but also other factors such as percentages of goods exported, management philosophy and background, roles played on and the characteristics of the supply chain, among others. without these added dimensions of sampling, results from e-commerce/e-business studies will be difficult to rationalise for extension of theories or generalisation. conceptualising and interpreting e-commerce/e-business another difficulty to obtain a comprehensive understanding on how smes can benefit from e-commerce/ebusiness is the conceptualisation and interpretation of what e-commerce/e-business is. while there are formal definitions of what e-commerce and e-business is, these definition are often over-simplistic and context independent. for example, a commonly presented difference between e-commerce and e-business is ebusiness encompasses a wider scope of business activities (beyond transactions) which are supported using the internet and associated technologies (johnston, 2001). while such definitions provide a broad understanding and a common language when discussing e-commerce/e-business activities, specific context of such activities are often not elaborated. for example, the july 2002 yellow pages® small business index™ e-business report (yellow pages business index, 2002) reveals that 79% of small businesses have a pc connected to the internet and 75% identified email as the most essential application. but such information does not provide any context in understanding how small businesses have gain benefits (strategic or otherwise) from the internet, let alone ecommerce/e-business. not dismissing the encouraging adoption rate of internet access, true-spirited ecommerce/e-business goes beyond checking email to gaining strategic benefit. in fact, some developed countries starting to show despite a high adoption rate of internet access, there is only relatively low adoption rate of e-commerce/e-business (hksar government, 2001). figure 1 is another piece of evidence supporting adoption of internet access and use of email do not equate to e-commerce/e-business success necessarily. consequently, a more holistic approach to study how small businesses actually benefited from e-commerce/ebusiness is most important in order to unravel the reality. contextual understanding of e-commerce/e-business success an alternative is to understand e-commerce/e-business success by qualifying success based on specific business contexts. being a heterogeneous group, success factors applicable to one individual or group of smes do not guarantee they will work for another group. moreover, the perception of success differs from individual to individual. research has shown that industry characteristics, nature of goods (kumar and loebbecke, 2000), market scope, management’s contribution (poon, 2002) among others all have a contributing effect on the outcome of e-commerce/e-business success. it is also noted that there are multiple stages of e-commerce/ebusiness adoption and implementation (poon, 1999; mehrtens, et al., 2001; chau and turner, 2002). in order to combine the understanding on the real issues leading to e-commerce/e-business success, more elaborate research frameworks are necessary based on a holistic instead of partial view approach. one such framework is shown in figure 2. ajis vol. 10 no. 1 september 2002 68 figure 1 french smes failing to benefit from e-commerce/e-business quantiative selection criteria: (e.g., no. of employees, turnover, industry sector, net worth, etc.) qualitative selection criteria: (e.g., management style, market orientation, powerrelationship on the supply chain, etc.) selection criteria sample of smes who adopted e-commerce/ebusiness profiling research methodologies (e.g., statistical analysis, positivist approach, etc.) detailing research methodologies (e.g., case study, in-depth analysis, multi-facet approach, etc.) a n a ly s is & in te rp re ta ti o n more comprehensive understanding of smes ecommerce/ebusiness generate figure 2. a suggested research framework for ongoing smes e-commerce/e-business research as there are more research outcomes published in the area, i believe the area is mature enough to enter a consolidation phase. instead of providing more fragments, the area will benefit further if a consolidation of these understandings is performed. latest initiatives to facilitate e-commerce/e-business adoption among smes smes e-commerce/e-business adoption has reached the agenda of many governments and regional organisations. the internationalisation and globalisation of many sectors forces some smes to be part of a global alliance. e-commerce/e-business is often the it platform which keeps such geographically dispersed ajis vol. 10 no. 1 september 2002 69 alliances operating (oecd, 2002). the asia-pacific economic co-operation (apec) has a specific steering group which hosted a workshop7 dedicated to the understanding and development of smes e-commerce/ebusiness in 2000. in australia, the national office for information economy (noie) has recently furbished a series of case studies8 detailing how smes have benefited from e-commerce/e-business plus action plans for others to follow. in addition, individual countries in the region such as singapore and hksar-china9 also have initiatives to encourage the adoption of e-commerce/e-business among smes. among all these initiatives, it is apparent that resources are moving from helping individual smes to adopt ecommerce/e-business in isolation to setting up collaborations, networks, clusters etc., (d’orazio, et al., 2001) so smes can be part of a trading or industry e-network or e-cluster (brown and lockett, 2001). this shift of focus has the following advantages: the first wave of government and business effort to bring e-commerce/e-business to smes has achieved awareness and preliminary adoption. in developed countries, it means most smes now at least have an internetready pc, a browser and basic knowledge and experience of e-commerce/e-business. however, to have smes to ascend to the next stage(s) of adoption/implementation is more challenging than it was originally thought. one reason is that the amount of resources and investment needed grows exponentially. figure 3 is an illustration of the level of difficulty and investment varies with the comprehensiveness and scope of implementation. to further the adoption of e-commerce/e-business among smes, it might need the further commitment of large organisations. this might sound paradoxical but past experience is an important reference here. in the era when electronic data interchange (edi) was maturing, there were visionaries suggesting how edi might transform how business was done. surely there were pockets of success and among these, businesses had transformed and gained strategic advantage. however, most of the business community did not benefit from edi as extensively as it was first predicted. retrospectively, many explanations were given, by both the academe and the industry, why edi was not as successful as once thought. i believe there are important lessons here to learn when furthering e-commerce/e-business adoption among smes. in the case of edi, often the most successful implementations were the ones driven by large corporations. the reason being that they had the financial and it resources to build up the infrastructure. while there were debates on whether this type of ‘hub-and-spokes’ approach was healthy from a power-relationship perspective, they no doubt provided smes a ready-to-go platform to adopt edi. e-commerce/e-business adoption by smes is now facing a similar crossroad. while no one owns the internet and the basic tools (e.g., the browser) are often free, but true business activities requires the integration of backend systems and eventually across the supply chain. in many industry sectors, the most powerful players on the supply chain are still the large companies. therefore, an e-cluster (brown and lockett, 2001) formed by large organisations, regulatory bodies and smes seems to have delivered positive outcomes in some cases. of course time will tell if such an e-cluster approach is sustainable. incidentally, quite a few of the latest projects10 support by noie’s itol program are of the nature of building such e-clusters, either at technological or business process level, for various industry sectors. 7 http://www.apec2000.gov.bn/ecomm/index.htm 8 http://www.noie.gov.au/projects/casestudies/ecommerce/shortform/industry/index.htm 9 http://www.info.gov.hk/digital21/eng/sme/sme_intro.html 10 http://www.noie.gov.au/projects/ecommerce/itol/round_6/index.htm ajis vol. 10 no. 1 september 2002 70 sustainability of competitiveness l e v e l o f te c h n ic a l a n d o rg a n is a ti o n a l b a rr ie r e-brochures online transaction fully integrated with backend systems integrate with multiple business exchangesfully integrated into a supply-web in v e s tm e n t investment of resources figure 3. level of difficulty vs. sustainability of e-commerce/e-business solution lessons and implications for smes management smes management should be aware of the successes and failures experienced by their peers in adopting ecommerce/e-business. this will help a sme to increase its chances to be successful when they further their initiatives. existing case studies such as those presented by noie are useful resources. smes should also be aware of the impact of the new economy, particularly the importance of knowledge and information to the strategic position of an sme (lee, 2002). this will help management of smes put e-commerce/e-business into perspective. resources and commitment to build customised e-commerce/e-business solution in isolation can be enormous once gone past the initial stage. tools such as the ready reckoner11 offered by noie would be a good start to decide what is involved in adopting e-commerce/e-business. however, it is important to gain understanding on the pathway to full-fledged e-commerce/e-business solution (see figure 3) and obtain a good estimation on the resources commitment before engaging a provider of solution. e-commerce/e-business strategy and decision-making should be business driven and follow business logic. although the ‘build-it-and-it-will-come’ logic works for a small number of smes, for the majority, resources should be allocated based on predicted business outcomes which are well quantified and qualified. such business logic has prevailed in smes and traditional wisdom of priority should be applied. it is possible that the more feasible e-commerce/e-business solution is to use an e-cluster infrastructure. often adopting an application service provider model, most e-cluster applications require only an internet-connected pc and a 11 http://www.noie.gov.au/projects/casestudies/ecommerce/readyreckoner.xls. ajis vol. 10 no. 1 september 2002 71 standard browser. i believe some of the e-cluster applications will be the affordable e-commerce/e-business solution for many smes in the near future although the power-relationship issues will surface again. conclusion despite the many research efforts performed on the topic of smes e-commerce/e-business, anecdotal evidence pointing to both success and failure. frameworks for predicting smes e-commerce/e-business success are still in their infancy. among many smes reported to have benefited from e-commerce/e-business, some are of a short-term or marginal nature. there is no-doubt that most national economies can benefit from the success of widespread adoption of e-commerce/e-business. however, sustainable success can only be resulted from an advanced and complex state of adoption as illustrated in figure 3. smes should move forward adopting a cautionary and evolutionary approach. i do not believe there is turning back for e-commerce/e-business, but knowing how to move forward will determine the winner. despite the economic uncertainties at the time of writing, smes should take advantage of the lower costs and availability of e-commerce/e-business expertise to further their initiatives. also, it is important for smes to monitor developments supported by government and industry to ensure they can be part of the industry e-clusters when built. finally, an e-commerce/e-business strategy should be business driven. there are risks involved and by taking calculated risks in a well-informed manner, e-commerce/e-business should deliver benefit of a large scale to smes. references bode, a.s. (2002) strategies for consultancy engagment for e-business development – a case analysis of australian smes. in: burgess, s (ed.) managing information technology in small businesses: challenges and solutions. idea group publishing, pp. 227-245. brown, d.h. and lockett, n. (2001) ‘engaging smes in e-commerce: the role of intermediaries within eclusters’. electronic markets, 11(1), pp. 52-58. brulles. g. (2001) ‘the internet and smes: current situation and example of a business platform – world trade area’. in: presentations of the 28th international small business congress. 16-19 september, 2001, stockholm, sweden. available: http://www.isbc2001.org/home/gemma%20brulles.pdf. accessed: 5/09/2002 12:52 pm. chau s.b. & turner p. (2002) ‘a framework for analysing factors influencing small to medium sized enterprises (smes) ability to derive benefit from the conduct of web-based electronic commerce (ec) 34 australian case studies’. in: proceedings of the 10th european conference on information systems ecis 2002. june 6-8, 2002, gdansk, poland. d’orazio, r., hunter, a. and zanoni, e. (2002) smes clustering into dynamic networks: an effective way to promote technology development. available: http://europa.eu.int/ispo/ecommerce/research/papers.html. accessed: 5/09/2002 2:06 pm hksar government (2001) digital 21: it infrastructure – survey on information technology usage (results summary). available: http://www.info.gov.hk/digital21/eng/milestone/download/itsurveysummary2001.pdf. accessed: 3/09/2002 3:18 pm. holmes, s. and gibson, b. (2001) the definition of small business. available: http://www.setel.com.au/smeforum2002/tp/bp01a.pdf. accessed: 3/09/2002 12:26 pm. johnston, i. (2001) ‘background paper ep06 – e-commerce and e-business’. sme e-commerce/ebusiness roundtable think tank. 7th september, 2001. sydney. available: http://www.setel.com.au/smeforum2002/tp/bp06.htm. accessed: 3/09/2002 1:56 pm. kumar, k. and loebbecke, c. (2000) ‘preface: digital trade of intangible goods: technology, applications and business models’. journal of end-user computing, 12(2), p.3. lee, b-c. (2002) ‘competition and innovation: small and medium enterprises in the new economy’. in: proceedings of the smes in the global economy conference (eds. c. harvie and b-c. lee). 12-13th july, wollongong, australia. 28 lee-rev1.pdf. mehrtens, j., cragg, p. and mills. a. (2001) a model of internet adoption by smes. journal of information & management 39(3), pp. 165-176. noie (2002) advancing with e-commerce – short-form case studies. available: http://www.noie.gov.au/projects/casestudies/ecommerce/shortform/industry/index.htm. accessed: 5/09/2002 1:14 pm. oecd (2002) global industrial restructuring: implications for small businesses. science, technology and industry working paper 2002/4. available: ajis vol. 10 no. 1 september 2002 72 http://www.olis.oecd.org/olis/2002doc.nsf/43bb6130e5e86e5fc12569fa005d004c/37e9f62cbc41ebcdc1256b6 1002f647f/$file/jt00120875.pdf. accessed: 5/09/2002 5:18 pm. poon, s. (1999) ‘small business internet commerce – what are the lessons learned?’ in: doing business on the internet: opportunities and pitfalls (eds. c. romm and f. sudweeks), springer, pp. 113-124. poon, s. (2002) ‘management’s contribution to internet commerce benefit – experiences of online small business’. in: managing information technology in small business: challenges and solutions (ed. s. burgess). idea group publishing, pa, usa. pp. 279-298. tetteh, e.o. (1999) ‘from business networks to virtual organisations: a strategic approach to business environment transformation in online small and medium-sized enterprises’. in: proceedings of the 10th australasian conference on information systems. pp. 980-992. yellow pages business index (2002) e-business report – the online experience of small and medium enterprises. july, 2002. available: http://www.sensis.com.au/internet/static_files/smeiypbibi_jul02.pdf. accessed: 3/09/2002 3:42 pm. oecd (1998) smes and e-commerce. paper by the working party on small and medium-sized enterprises. paper no. dsti/ind/pme(98)18/final. available: http://www.olis.oecd.org/olis/1998doc.nsf/4cf568b5b90dad994125671b004bed59/25d3d51808273c06c1256 6be0037c7b6/$file/11e88491.eng. accessed: 6/09/2002 3:44 pm. microsoft word ajis vol13-2 o.doc 169 australasian journal of information systems volume 13 number 2 may 2006 ict integrity: bringing the acs code of ethics up to date michael bowern, oliver k. burmeister committee member of the australian computer society's committee on computer ethics charles sturt university wagga wagga, nsw, australia email: mike.bowern@anu.edu.au don gotterbarn director software engineering ethics research institute chair acm committee on professional ethics professor of computer science, east tennessee state university, usa john weckert school of information studies, center for applied philosophy and public ethics charles sturt university wagga wagga, nsw, australia abstract over the last 20 years the ict industry has seen many developments and changes, such as faster and smaller computers, many new digital devices, the convergence of this digital technology, the internet, and operational aspects such as outsourcing. governments and professional societies have discussed the issues related to these changes in the media, yet the australian computer society’s code of ethics has not changed. this paper discusses the need to change the acs code, the issues to be addressed, and procedures to be followed, if changes are to be made. these issues include, amongst others, the lack of specificity of the code, and the absence of a way to decide between ethical principles which may conflict in some situations, and how to resolve such conflicts. with the focus on the australian context, the acs code’s relationship with international standards and codification of ethics are considered, and the paper discusses what lessons can be learned from overseas experience in modifying codes. the paper concludes by outlining the next steps to bring the acs code up to date. keywords: codes of ethics, codes of conduct, codes of practice, acs, ifip, ict ethics, professional societies. introduction recognition of professionalism in information and communications technology (ict) in australia requires that members of the australian computer society (acs) adhere to a code of ethics (acp, 2005). systems using ict are not produced in an ethical vacuum. the values of many stakeholders are involved. typically these stakeholders are suppliers, vendors, employees, contractors, trade australasian journal of information systems volume 13 number 2 may 2006 170 unions, clients, and those who are affected by the delivery of the final product. increasingly, especially in globalisation and offshore issues, professional societies and politicians can also be involved. one way that professional societies address the diversity of values of so many different stakeholders is by requiring their members to adhere to their code of ethics. major functions of professional codes of ethics include the requirement to articulate ethical standards of the profession; to educate practitioners and the public about ethical obligations; and to provide guidance to resolve ethical quandaries (anderson et. al., 1993). philosophical dialogue about alleged difficulty with codes of ethics includes works by luegenbiehl (1983), ladd (1995), fairweather (2001) and tavani (2004). there are numerous terms in the literature for professional codes of ‘ethics’. in this paper we follow the guidance of the international federation for information processing (ifip) (berleur, et. al, 2004), in discussing two types of codes. the first type, which berleur et al refer to as the code of ‘ethics/conduct’, has a set of high level statements concerning such issues as honesty and integrity. this code governs ‘how the person to whom it applies conducts him [sic] or herself in an ethical manner’ (berleur, 2004, p 11). to avoid confusion, this type will be referred to as the code of conduct in the rest of this paper. the second type of code refers to a ‘code of practice’ for professionals, which ‘governs how the person to whom it applies carries out his or her work technically’ (berleur, 2004, p 11). this code includes a set of detailed statements related to the professional’s particular occupational environment. these statements of practice are more specific and more likely to change over time, than are the conduct statements. for example, the desire for honesty is less likely to change over time, than is the interpretation of the way in which honest dealings in the occupational environment are to function. typically, a professional society would define both types of code for its members in a single document, a good example being the software engineering code of ethics and professional practice. this is ‘the standard for teaching and practicing software engineering by the acm and ieee-cs’ (secepp, 1999). in this paper, reference to a ‘code’ or a ‘code of ethics’ means a document which includes both types of code, covering conduct and practice. other references will be made to a specific type of code under consideration, for example a ‘code of practice’. the problem we address in this paper is one in applied philosophy. presuming that a professional code of ethics can usefully serve the education and guidance functions, there are three critical questions. 1. how can one identify if the changing technology has reduced the ability of a code of ethics to be useful in ethical assessment? the code is to act as a guide. it should not be subject to frequent change, as each new technology or ict work process comes along. yet the inevitability of change in the ict industry leads one to recognise that certain changes may require a review of a code of ethics, particularly with respect to its code of practice. 2. given a code that no longer addresses some of the significant issues, what can be done to consistently revive the ability of the code of practice to serve this function without undermining the existing strengths of the code of conduct? 3. for codes which do not specifically distinguish the practice and the conduct elements, how can one modify the practice elements without also modifying the conduct standards? these concerns are not new. each professional society has had to grapple with them at one stage or another. ifip has recently suggested a series of high level guidelines to assess the strengths and 171 australasian journal of information systems volume 13 number 2 may 2006 weaknesses of codes of ethics (berleur, 2004). perhaps these guidelines could also help with the first of the above points, namely to re-evaluate the efficacy of an existing code, once significant change has occurred in the ict industry? rapidly changing technology can have a negative impact on the adequacy of its professional codes; this paper describes that impact, and ways to ameliorate it. in the ict industry one has always had to deal with the relationship between ethical assessment and technology. a brief examination of the evolution of ict professionals' codes will show that at earlier stages modifications were made to the codes to meet technological changes; and a desire to include their expanded sense of professional responsibility and behaviour occasioned by some of these advances. for example, a major revision of the acs code occurred in 1985, to address the concerns over the rise of computer based crime (coldwell, 1987). many changes to the ict industry have taken place since the last major revision of the acs code, 20 years ago. this paper discusses some of the factors which could be considered when reviewing the need to change the acs code. the paper also briefly examines reasons for the modifications to the acm/ieee ethical standards. the paper then focuses on changes currently planned and in progress by the acs. the paper identifies what were perceived as the major ethical shortcomings, and indicates the common elements required to meet the impact of changes in industry practice, in the education and technical functions of the acs code. these common elements may be used as guidelines in the next generation of ethical code reform. the paper ends by looking at how the acs plans to progress the redrafting of its code of ethics and its associated documents. when do changes in ict necessitate changes to the code of ethics? in a domain which changes as frequently as that of ict, a change in work practice, or the advent of some new technology, should not of itself constitute grounds for changing a code of ethics. examples of the types of change that could make us consider revising or reviewing our codes include the rise of the internet, the increased use of outsourcing and off-shoring, and the arrival and impact of micro/nano computing. however, codes cannot be reviewed with every change that occurs. to avoid ineffective continuous code reviews, procedural guidelines are needed within a professional society, that ensure regular review of its code or codes. how frequently should such a review be undertaken? given the logistical difficulties involved, and based on observations of previous changes to codes in australia and the usa, the authors contend that codes ought to be reviewed at least every 10 years. the acm code was reviewed and significantly revised in 1992 and reviewed again in 1998, but not since. the acs code was reviewed in 1975, and again in 1985, but not since then. by this reckoning a review of the acs code is long overdue. the review procedure ought to also allow for reviews that are determined by major technology changes. the advent of a biological computer implanted in the brain might be the sort of thing that justifies immediate code review rather than waiting a prescribed period of time. changes to codes of ethics have been motivated by a desire to include their expanded sense of professional responsibility and behaviour occasioned by some technological advances (gotterbarn, 1996). other changes have been made to include an aspirational component in the code. australasian journal of information systems volume 13 number 2 may 2006 172 why aspirational? to appeal for right behaviour when enforcement is not possible. in medicine and law, a breach of the code of ethics can mean loss of ability to practice one’s profession. not so in ict. an acs member can be held accountable, through the acs disciplinary procedures. but 80% of ict practitioners in australia are not acs members. also, a member facing the acs disciplinary procedures could simply resign. then s/he can continue to behave unethically; they simply can no longer claim to be an ict “professional”, which is a right that the australian council of professions (acp, 2005) has restricted to use by acs members only. gotterbarn (2000), writing about experiences in the development of two codes of ethics in the usa, found that a critical issue is that of specificity, that is how prescriptive and detailed the code should be. specificity has partly been addressed in the acs code of ethics by recent work (bowern, 2003) which resulted in a set of case studies identifying issues related to each of the clauses in the code. some of the cases were drawn from published material (burmeister, 2000; and burmeister and weckert, 2003); others were based on actual incidents known to the contributors, and a few were invented to complete the set. this exercise has revealed some shortcomings in the acs code. for example, the clause stating ‘i must distance myself professionally from someone whose membership of the society has been terminated because of unethical behaviour or unsatisfactory conduct’ is unfair and unworkable (bowern and weckert, 2005). another approach to specificity is in the clauses of the code itself. in some instances the code of conduct does not change, but changes in ict practice mean that the application of the code, as seen in the code of practice, is different. this task is simpler in codes where the code of conduct is distinguished from the code of practice, but it is much harder where there is no clear distinction in the document between ethics clauses and practice clauses. lessons from the evolution of the acm/ieee code the solution to code modification is not purely technical. in the acm/ieee software engineering code (secepp, 1999) it was important to include a broader sense of ethical reasoning in it. in evaluations of that code by philosophers such as herman tavani, the decision making guidance of the preamble was considered one of its strengths. the international task force that developed the software engineering code of ethics and professional practice (secepp) was aware of a number of previously identified weaknesses of professional codes and made a conscious effort to address those in their code. major motivations for writing the secepp were to document the professional responsibilities of software engineers, and those aspiring to be software engineers, in a way which could be used to educate practitioners and the public; and to facilitate ethical decision making in accordance with these responsibilities. there have been two major problems in attaining these broad goals. one is based on the overly specific content of a code, in which the code attempts to define precisely a complete list of all of the ethical behaviour of a professional. precisely defined codes are almost out of date the minute they are approved. this is especially true in professions related to industries as dynamic as those covering ict. on the other hand codes which are too general, which treat ethical judgement at its most abstract level, have been criticised for their failure to provide adequate guidance. this attack is often generalized into a simplistic criticism of all codes, because codes can never be complete and anticipate every possible ethical situation (fairweather, 2001). such criticisms of codes are easily made, but are not very useful for they do not distinguish an incompleteness which is a shortcoming, from an incompleteness which is a strength. the secepp attempts to steer a middle ground between 173 australasian journal of information systems volume 13 number 2 may 2006 code imperatives which are too vague to give useful guidance, and the numbing precision of detailed imperatives which are locked to a particular stage of technology. instead of appealing to a particular technical standard such as structured programming, which will change, the code appeals to the changing standards of the profession to address specific technical issues. by appealing to current best practices rather than naming a specific practice, as the standards change the particular technical items referred to in the code also change. this is a way to build a code which keeps current with the particular best standards of the profession. the secepp code differs from that of the acs, in that it also includes some general principles on ethical decision making to help guide the utilization of the specific clauses. for example, the preamble uses everyday english to advocate basic ethical approaches to decision making, and asks software engineers when making a decision guided by the specific principles in the code, also to: • consider broadly who is affected by their work (utilitarianism); • examine if they or their colleagues are treating other human beings with due respect (kantianism); • consider how the public, if reasonably well informed, would view their decisions (gert and others); and • analyse how the least empowered will be affected by their decisions (john rawls). the international secepp taskforce stated that ‘without the aspirations, the details can become legalistic and tedious; without the details the aspirations can become high sounding but empty; together the aspirations and details form a cohesive code.’ (gotterbarn, 1999, p103) another concern is that in many cases it appears as if principles in a code could point in conflicting directions and thus the code itself does not direct the final decision. secepp admits it is incomplete, but does not suffer from issues of vagueness, because it provides general guidance for ethical decision making as indicated above. this still leaves open the possibility that the general principles may be in tension, in particular circumstances. secepp has a clear hierarchy of values that facilitates the reduction of the instances of ethical tension. first, its eight principles are listed in an hierarchical order. if this does not help in the final decision making, then the code has an overriding principle – that a concern for health, safety and welfare has an overriding primacy. secepp consists of a set of principles (code of conduct statements) and details or examples for each of the principles (code of practice statements). its preamble contains some guidance on understanding and using the code. it is the preamble which facilitates it addressing some of the older philosophical criticisms of professional codes. this structure of the ieee-cs/acm code has raised other comments, whereby some philosophers have taken the secepp taskforce to task for including guidance about using the code as part of a document called a code of ethics and professional practice. as the following paragraph shows, this criticism has not deterred the adoption of the secepp by a significant number of software engineers. the secepp taskforce identified a set of common elements to the profession of software engineering. they did this without identifying a particular professional society’s preference for a particular culture’s ethical style. the evidence for that is in the number of professional societies throughout the world that have adopted the code and translated it into their native languages. currently the code has been adopted by professional societies in argentina, australia, canada, china, croatia, england, italy, israel, japan, mexico, spain, and the united states. australasian journal of information systems volume 13 number 2 may 2006 174 current codes of the acs the acs code of ethics (acs codes, 2005) comprises two sections. the first section is a simple policy statement plus a declaration of six values and ideals; which are then expanded in the second section, a detailed set of 37 statements, which are called the standards of conduct. this code of ethics is defined as part of the acs national regulations, to emphasise the importance of the code, and to ensure its prominence in the acs body of documentation. supplementary to the code of ethics is the code of professional conduct and professional practice (acs codes, 2005). this code was developed to provide more practical guidance in the day to day activities of ict professionals. it is not part of the national regulations, which means that it is easier to amend and update. changes to the national regulations require a vote by the national council, followed by a vote by all members of the acs. this has implications for future amendments to the codes. figure 1 illustrates the codes. national regulations clause code of professional conduct and professional practice 4. acs code of ethics 4.1 and 4.2 policy statement 4.3 values and ideals introduction 4.4 to 4.10 standards of conduct section a code of professional conduct there is no identified code of professional practice in the national regulations there is overlap between the standards of conduct and the code of professional conduct sections b to l code of professional practice figure 1: the “conduct” overlap. the code of professional conduct is ‘intended as a guideline for acceptable personal conduct for each it professional practicing in the industry’, and as such is complementary to the values and ideals and to the standards of conduct. there is some overlap between the standards of conduct and the codes of conduct. the code of professional practice is ‘intended as a guideline for acceptable methods of practice within the it industry’. the guideline is generic and addresses a range of aspects of the product life cycle, and acquisition, development, implementation and support processes. the code of professional conduct and professional practice has not been updated since its adoption by the society. the acs code of ethics (comprising a policy statement, the six values and ideals, and the standards of conduct) is a general code applicable to virtually anyone in the ict industry; this is what has hitherto been referred to as a code of conduct, in this paper. there is however also the ieee/acm code (secepp) which is aimed specifically at software engineers. this second code was adopted in 2004 by both the acs (davidson, 2004), and the institution of engineers, australia. 175 australasian journal of information systems volume 13 number 2 may 2006 the focus of this paper continues to be on the acs code, with lessons on the development of the secepp code being drawn on, for recommendations for changes to it. in 2003 the acs established a national committee on computer ethics (cce). amongst its terms of reference is ‘to develop and propose relevant codes of conduct’ (acs cce, 2003, p 5). the acs code is in need of change (bowern, 2003; burmeister, 2000) because it is dated, and does not reflect ethical issues arising from the technological developments which have occurred since the last major revision in 1985. consequently it does not reflect the ethical issues of the widespread adoption of the internet, ubiquitous problems like y2k, the human-computer interaction issues created by nanotechnology, nor, for example, does it adequately address the issues of increased use of outsourcing and off-shoring. the secepp code development has yielded lessons to be heeded. the extensive consultation process engaged in by that taskforce needs to be emulated and followed by the acs. ifip also advocates an extensive consultation process, arguing that the ‘process used to develop a code is as important as the code itself’ (berleur, 2004, p13). in summary, future versions of the acs code of ethics should: • incorporate the code of professional conduct, to ensure consistency with the standards of conduct, to produce a code consistent with the ifip code of conduct; • update and maintain the code of professional practice as an equivalent to the ifip code of practice; and • rationalise the way that a part of the code is incorporated in the national regulations to mandate its use, and the way that other parts of the code can be more easily updated. deficiencies in the acs code a recent acs report, including a small survey of acs members active in the computer ethics field, has identified some deficiencies and potential improvements, as follows, in the acs code (bowern, 2003). • the meaning and use of the code needs to be clarified, to explain exactly what role the code does, and should play, as a way to provide guidance and education. • the role and activities of the disciplinary committee in the acs should be reviewed and amended, if required. • consideration is required as to whether the code should take into account the fact that acs members come from different cultural backgrounds, and that they may interpret some of the clauses in different ways. • the code should be consistent with international standards since, although the software is developed or designed in australia, it has international consequences. • an editing process to resolve these existing issues should be established, ensuring that the code is maintained to reflect the changing nature of the ict industry. the acs cce have identified the following additional deficiencies: • in 1985 the "c" of ict was not part of the self-description of the acs membership. the code only indirectly addresses "c" type issues at this time. • unlike some other codes around the world, the acs code has no system of prioritisation, for the inevitable situations of conflict between clauses in the code. • the power of sanction (disciplinary committee) relationship to the code is poorly defined. • there is a need to resolve issues to do with a multiplicity of codes of ethics. australasian journal of information systems volume 13 number 2 may 2006 176 consider this last point. as mentioned already, in addition to the acs code of ethics, the acs has adopted the secepp code. then too, there are many people in the ict industry who belong to specialist groups (management consulting, graphic design, software engineering, systems administration, human-computer interaction, and more). in some of these there are codes of ethics specific to that group. for example, the systems administrators guild of australia formed their own working group on ethics, because they saw the acs code as too general, lacking specificity for their work. so they came up with their own code of ethics (lance, 1994). work needs to be done to define the relation of the acs code to these other codes, and to consider the following issues. should there be one single code for all ict professionals? what about in situations like secepp, a second code adopted by the acs, what happens if there is conflict between such a code and the acs code? disciplinary committee the roles and responsibilities of the acs disciplinary committee are described in the society’s rules and regulations. ifip argues that ‘no code has any value in terms of public duty unless it is associated with a power of sanction such as disciplinary procedures’ (berleur, 2004, p12). however, anderson, et al (1993) have argued that codes as education serve a useful function in educating and guiding decision making. whilst a laudable aim, the authors contend the ifip view is not currently achievable. professionalism in ict, certainly in australia, is still not at the same level as in engineering, medicine and law. as shown above, in australia it is possible for a member of the professional society, who has been called to account for a disciplinary matter, to simply resign their membership. upon their resignation, no further action by the professional society can be taken. however, in other professions, such as medicine, such opting out of the society is not possible. for this reason, it is the view of the authors that the greater emphasis ought to be on incentive and education, rather than on discipline, in regards to a code of ethics in ict. cultural aspects australia has a significant multi-cultural population, which is also reflected in its ict workforce. certainly the code’s audience consultation process should include members who come from different cultural backgrounds. one debate is whether the code should contain clauses reflecting the cultural differences of its members. part of this debate is whether the clauses of the code should have a common interpretation, or allow for contextual and cultural variances. ifip has argued (berleur, 2004), on the basis of kant’s ‘categorical imperative’, for a universalisation, in which a code contains necessary 'minimum criteria, conditions, and requirements' applicable to all members of a professional society, regardless of cultural, social and/or legal context. this is very different from the secepp approach (cited above) which argues against a code of practice as a statement of minimal standards and argues for a code including aspirational elements directing a striving for quality. the authors argue that it is not an issue of cultural debate whether testing reduces the risks of software failure, for example. the ethical responsibilities of a practising professional, embodied in a code of ethics, are dictated by that profession and its technology. the profession knows the best standard (its code of practice) to satisfy these responsibilities. 177 australasian journal of information systems volume 13 number 2 may 2006 national aspects gotterbarn (1997) describes the membership of the task force for software engineering ethics and professional practices. it comprised people predominantly from north america and europe, with a few other members, including one from australia. during the development of the draft codes gotterbarn found that north american contributions to the codes predominantly followed obligations/rights ethics, whereas the bias in europe was towards virtue ethics. his study identified that middle eastern and australian views did not easily fall into either of these categories. to the authors of this paper, the important point is to recognise that there are different approaches to ethics and to ensure that they are considered in the development of codes for ict ethics. these different approaches do something because it is the right thing, or do something because it is the will of some deity, or do something because it will produce the greatest good address the basis for a particular belief. the affirmation of intellectual property, for example, can be based on any of these approaches to ethics; but once ip is affirmed in a code it is not subject to cultural relativism. the acm/ieee secepp sought principles, consistent with each type of ethics, that were standards of software engineering. international aspects the acm and ieee are international organisations and the task force was established to recognise that international character. an objective of the task force was to establish a code which would be accepted internationally. the acs does have international members, and has entered into reciprocal agreements with a number of overseas computer societies, including several in south east asia. however, the acs does not have the same sort of international ambitions as those of the acm and ieee. therefore it might be argued that any redevelopment of the acs code would generally focus on the needs and issues relating to australian ict professionals, although those of the international members should not be forgotten. however, the development of computer systems and software are international activities, and have international impact; and those aspects must be reflected in codes of ethics. if the acs is to meet the needs of the profession, it is the needs of the international ict profession that must be met. if the acs comes up with a principle that is uniquely australian, then we should question whether it really was a principle of the profession. guidance and education further work is needed to identify how the acs code can best provide guidance and education to its members. current attempts by the cce at accomplishing this are mainly through better communication and publicity of the code to acs members. some members of the cce have produced a set of case studies related to each of the clauses in the acs code (acs cases, 2004), which have been publicised to members. since late 2004 the cce has arranged for a regular column in information age, the acs bi-monthly magazine for members and other professionals in the industry. the column seeks to promote the code, and discuss the ethical aspects of current ict news items or scandals. the case studies are one source of material for these articles. australasian journal of information systems volume 13 number 2 may 2006 178 the cce will also seek to arrange regular sessions at the acs annual conferences at which industry and academic speakers can address issues of the code of ethics. already greater use of the acs web site has been made for this purpose. advice should be provided on how the code would apply to the wide range of acs members, some of whom are not directly involved in systems development, for example ict professionals dealing directly with customers, such as some empirical software engineers. if the acs code is to cater to the widest possible interpretation of ‘ict professional’, it must be examined for its applicability to all aspects of the profession. an excellent example has been set through the acm/ieee software engineering code, that has recently been adopted by the acs for its members who are software engineers (davidson, 2004). the adoption of the software engineering code is another contribution to the specificity of codes for the acs. a possible approach to code revision moor (1999, p65) defines policies as ‘rules of conduct ranging from formal laws to informal, implicit guidelines for action’. so a code of ethics could be considered as a set of policy statements about how a professional should behave, in ict in this case. the on-going development of technology coupled with the malleability of computers means that there will always be a need to develop new policies. one framework that will aid acs deliberations is james moor’s just consequentialism. moor (1999, p65) comments on the problems rising from conflicting ethical theories and believes that ‘ethics needs more unifying theories that call upon the various strengths of the traditional approaches to ethics’. his just consequentialism theory, or framework, is discussed and summarised with respect to cybertechnology, in tavani (2004, pp59-60). the framework consists of two steps: • deliberate over various policies from an impartial point of view to determine whether they meet the criteria for being ethical policies (for example, they do not cause unnecessary harms, and support individual rights); • select the best policy from the set of just policies arrived at in the deliberation stage by ranking ethical policies in terms of benefits and (justifiable) harms. this approach would appear to be one way to consider the issues described above, and the cce will consider its use when revising the acs code of ethics. the way forward for the acs the next steps in this process of the review of the acs code of ethics, including the standards of conduct and the code of professional conduct and professional practice, involve wide consultation with acs members. the cce has already received input from various individual members, as well as from member representatives. one example of the latter is that of the director acs-w (representing the views of women members of the acs), raising aspects of professional practice in relation to discrimination. in 2006 the cce is engaging in more deliberate involvement of the wider membership of the acs. input into the process has also been received from the wider non-acs ict community. the latter has been achieved through the close ties between the cce and the australian institute of computer ethics (aice), for example, the topic of the acs code review process was the subject of a monthly 179 australasian journal of information systems volume 13 number 2 may 2006 meeting of the melbourne chapter of aice, in november 2005, with the director of aice, karen mather, attending the session from interstate (nsw). the cce is basing the next review on the method used in the 1985 review of the code, which used extensive consultation with acs fellows. the acs has various membership grades, the highest being that of fellow. in 2006 the cce will conduct focus group sessions with each of the eight branches of the acs. there is one branch for every state and territory in australia. each focus group will consist of up to 12 participants comprising fellows and other members nominated by the branch executive committee. six of the groups will consider one of the values and ideals described in the current ideals of the code, and the other two will focus on more general topics. other interested people, particularly directors and members of boards, will also be consulted. these focus group sessions will be completed by mid 2006. in july 2006 professor rogerson, the director of the centre for computing and social responsibility (ccsr) at de montfort university uk, will be visiting australia. simon rogerson was the first person to be appointed as a professor of computer ethics in europe, and he made a substantial contribution to the review of the british computer society’s code of ethics. the cce seeks to draw on this experience as they bring together the findings of the focus groups, and develop the revised code. professor rogerson will facilitate a meeting of the cce to help synthesize the results of the focus groups and other submissions, including the proposal by the acs-w. in addition to this, acs will be kept informed about the process through the acs bi-monthly publication, information age. it is envisaged that the redrafted code will be put forward to the november 2006 meeting of the acs council. lastly, this review process is deemed to be significant to the wider ict community as well. just as the review process draws on similar work from overseas, in the usa and in the uk, so it is expected that the process will help inform other societies working through similar issues. the latter will be achieved through disseminating information about the process of the review through working groups of the international federation for information processing (ifip), a process that has already started, given two members of the cce are ifip working group members, and more particularly, are part of the ifip working group charged with responsibility for developing frameworks on ethics, which has as one of its responsibilities, helping member societies formulate their codes of ethics (ifip sig 9.2.2). conclusion just as a motor vehicle should have regular services, so should a code of professional ethics. with a vehicle there is typically a major service infrequently, and more regular minor services. in the case of the acs code of ethics, there have been numerous minor services and lots of tinkering, since the last major service in 1985. it is long overdue for its next major service. the use of a suitable code of ethics is necessary for the successful development and implementation of new applications of ict. it is also necessary for the promotion of public trust in the professionalism of those in ict. codes are a tool for assessing the ethics of new technologies, such as nanoizing technology, and new ways of working within ict, such as the treatment of participants in a testing process, and in outsourcing services to an overseas organisation. change in ict is a fact of life. australasian journal of information systems volume 13 number 2 may 2006 180 professional societies need to put in place procedural mechanisms to ensure regular (at least every 10 years) reviews of the codes, to ensure their ongoing relevance. the acs requires ict professionals to keep up to date with changes in the industry. no less should be required of the professional code of ethics. how codes of ethics addressing ict practices are changed requires deliberate thought and planning. one way is to appeal to current best practices rather than naming a specific practice, as technical standards change the particular technical items referred to in the code also change. this is a way to build a code which keeps current with the particular best standards of the profession and overcoming the risk of the code becoming out of date by rapidly changing technology. in changing a code, there is a need to put metrics in place to ensure the efficacy of those changes. ifip has developed high level guidelines to assess the strengths and weaknesses of a code of ethics. such guidelines can help in this process, though more work is needed to turn these metrics into a reliable code assessment tool. references acp (2005) australian council of professions, http://www.professions.com.au/ accessed on 21 february 2006. acs cce (2003) cce submission to council, terms of reference for the national committee on computer ethics (cce), as defined by the community affairs board of the australian computer society, sydney: acs, nov. acs cases (2004) case studies, australian computer society, http://www.acs.org.au/publication/docs/acs_casestudiesfinal.pdf accessed on 21 february 2006. acs codes (2005) australian computer society codes, http://www.acs.org.au/ accessed on 21 february 2006. anderson, r. johnson d, gotterbarn d, and perrolle, j (1993) “using the acm code of ethics in decision making”, communications of the acm, oct. berleur, j., duquenoy, p., holvast, j., jones, m., kimppa, k., sizer, r., and whitehouse, d. (2004) “criteria and procedures for developing codes of ethics or of conduct”, international federation for information processing, ifip-sig9.2.2, ifip framework for ethics of computing, sep. bowern, m.e. (2003) report to the acs management committee on the acs code of ethics project, sydney: acs, dec. bowern, m.e. and weckert, j. (2005) “ethics: when a team member embezzles”, information age, june, viewed 21 february 2006, http://www.infoage.idg.com.au/index.php?id=1381484222 burmeister, o. k. (2000) “applying the acs code of ethics”, journal of research and practice in information technology, 32(2), 107-120. burmeister, o.k., and weckert, j. (2003) “applying the new software engineering code of ethics to usability engineering: a study of 4 cases”, journal of information, communication & ethics in society, 3(3), 119-132. coldwell, r.a. (1987) “non-professional practices in computing: some thoughts on the next decade or so”, the australian computer journal, 19(4), sydney: acs, november, 215-218. davidson, p. (2004) “acs and ieaust jointly adopt software ethics, practice code”, information age, apr, viewed 22 february 2006, http://www.infoage.idg.com.au/index.php?id=912845172. 181 australasian journal of information systems volume 13 number 2 may 2006 fairweather, n.b. (2001) no papa: “why incomplete codes of ethics are worse than none at all”, in readings in cyberethics (eds. r.a. spinello and h.t. tavani), jones and bartlett publishers, sudbury, ma. gotterbarn, d. (1996) “establishing standards of professional practice”, in the responsible software engineer, ed. colin meyer, springer verlag. gotterbarn, d. (1997) “the professionalization of software engineering and its significance for ethics education”, ieee, frontiers in education. proceedings 1997. gotterbarn, d. et al. (1999) "software engineering code of ethics is approved" in communications of the acm 42:10 102-107 gotterbarn, d. (2000) “two computer-related codes”, perspectives on the professions, 19(1). ladd, j. (1995) “the quest for a code of professional ethics: an intellectual and moral confusion”, in computers, ethics and social values (eds. d.g. johnson and h. nissenbaum), prentice hall, englewood cliffs nj. lance, k. (1994) “crafting a code of ethical conduct”, sage-au code of ethics, systems administrators guild of australia annual conference. luegenbiehl, h.c. (1983) “codes of ethics and the moral education of engineers”. business and professional ethics journal, 2(4), 41–61. moor, j. (1999) “just consequentialism and computing”. ethics and information technology 1(1), 65–69. secepp, (1999) ieee-cs/acm software engineering code of ethics and professional practice, version 5.2. tavani, h.t. (2004) ethics and technology: ethical issues in an age of information and communication technology. john wiley and sons, inc, hoboken, nj. australasian journal of information systems volume 13 number 2 may 2006 182 using artificial neural networks and function points to estimate 4gl software development effort g.e. wittig and g.r finnic school of information technology bond university gold coast, queensland 4229 abstract hie value of neural network modelling techniques in performing complicated pattern recognition and nonlinear estimation tasks has been demonstrated across an impressive spectrum of applications. software development is a complex environment with many interrelated factors affecting development effort and productivity. accurate forecasting has proved difficult since many of these interrelationships are not fully understood. an attempt to capture the significant attributes of the software development environment to enable improved accuracy in forecasting of development effort is made using backpropagation artificial neural networks. the data for this study was gathered from commercial 4gl software development projects, across a large range of sizes. as is typical of software developments, the range in productivity and other development factors in the data set is also large, accentuating the estimation problem. despite these difficulties the neural network model predictions were reasonably accurate in comparison with other published results, indicating the potential of the use of this approach. introduction reliable prediction of size and effort in software development projects is a necessary prerequisite to developing reliable cost and schedule estimates (de marco, 1982). the size and development effort measures, such as function points or lines of code developed per person-month, act as technical productivity and performance indicators that facilitate the tracking and control of software developments. unfortunately it is by no means easy to estimate development effort accurately and much effort has been expended to develop metrics which attempt to measure size and complexity of programs and systems (conte et al, 1986). many of the early metrics were developed in the procedural third generation language (3gl) environment. the mainly non-procedural fourth generation language (4gl) developments, with their high level query languages, screen writers, report generators, and application generators are substantially different (martin, 1982). function point analysis (fpa), which is technology independent (dreger, 1989; rudolph, 1986), has been used for both 3gl and 4gl developments to estimate development effort. an analysis of some commercial 4gl software developments from several organisations indicates the presence of large variations in productivity which must be accommodated to accurately estimate development effort (ferens & gurner, 1992; wittig, 1991). interrelationships between the various factors affecting development effort are complex, not fully understood and have made development cost estimation difficult and sometimes inaccurate (kemerer, 1987). these factors affecting development effort should not be analysed individually in isolation (kitchenham, 1992), as it is necessary to ascertain their combined effect. intuitively for example the proportionate difference in total development hours between a system developed by an expert programmer team and an average programmer team is significantly influenced by the complexity of the system being developed. in an analysis of 65 cases (banker et al, 1991) to identify factors influencing maintenance productivity, the technique of stochastic data envelopment analysis was used and the authors noted that linear models are not likely to make adequate representations of the development process. the value of neural network modelling techniques in performing complicated pattern recognition and non-linear estimation tasks has been demonstrated across an impressive spectrum of applications (white, 1988). a recent survey (rumelhart et al, 1993) shows that the recent growth of neural network applications has been quite remarkable. it was just four years ago that the only widely reported commercial application outside the financial industry was the airport baggage explosive detection system. since that time scores of industrial and commercial applications have become known. a few 87 ajis of these applications include telecommunications, particle accelerator beam control, credit card fraud detection, machine and hand-printed character recognition, cursive handwriting recognition, mass spectra classification, quality control in manufacturing, petroleum exploration, war on drugs, medical applications, financial forecasting and portfolio management, and loan approval. the details of many are considered corporate property and shrouded in secrecy. this growth in neural network applications is in part due to the availability of an increasingly wide array of dedicated neural network hardware, in the form of accelerator cards for pc's and workstations. complementing the hardware are scores of commercial software packages (rumelhart et al, 1993). this study uses backpropagation artificial neural networks to examine whether they are capable of adequately capturing software development complexities in their weight space, to enable them to make accurate estimates. research methodology data collection for this study development data from 15 commercial systems, developed by information systems professionals, was recorded. the size of the systems was measured in unadjusted function points as it appears to be a more consistent measure than source lines of code (jeffery & low 1990; kemerer, 1987; low & jeffery, 1990). development effort was measured in development hours. all activities, starting from the specification stage and through to that stage where the product is ready to be delivered to the end-user are included in the development time. the time spent on documentation is therefore also included. excluded is the time required for the formal user acceptance tests, as well as end-user training. the definition of a development hour is the actual time spent on the project, and also includes all time spent on attending meetings directly related to the software development, but excludes times such as public holidays, leave, illness, and development staff training. the development attributes which were included in the study were selected after reviewing some previously published research (albrecht & gaffney, 1983; boehm, 1981; conte et al, 1986; jeffery, 1987; jones, 1986; vessey, 1986). the selection was refined by interviewing the information systems managers of 10 commercial organisations to establish which factors they with their development experience considered to have a significant impact on development productivity. precise descriptions of the various development attributes were developed, to attempt to ensure data consistency across the data set (wittig, 1991). no interrater reliability study was conducted (kemerer, 1993), as some of the systems are very large and this would have been costly. in the determination of the average size of the development team both analysts and programmers are included in the calculation, as well as project managers and program librarians directly involved with the development. research data the systems ranged in size from a small 29 function point system to a system of 4669 function points. if uncommented 4gl source lines of code (sloc) is used as a size measure, the systems ranged in size from 600 to 571 000 sloc. the development effort required to develop these systems ranged from 40 to 81 270 hours. a system of less than 300 function points would be considered a small system, while a medium system would be between 300 and 800 function points, a large system between 800 and 1000 function points, and a very large system would be greater than 1000 function points (dreger, 1989). a summary of these details is given in table 1. the size unit of unadjusted function points (ufp) is used. there has been some criticism regarding the choice and weighting of the function point adjustment factors (jones, 1991; kitchenham, 1992; symons, 1991), and as it is easy to include these into the neural network model, the technical complexity adjustment to generate adjusted function points was not made. for this study software development productivity is defined in the economic sense (jones, 1986) and is expressed as the amount of output produced per unit of input, and the unit used is unadjusted function points per development hour. may 1994 88 gl system 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 table 1 dev hours 5027 1680 13300 98 588 450 40 81270 35000 240 100 4864 1200 2500 1400 data set lines code 88325 44204 170 557 1535 2019 1600 1400 298843 562500 600 3241 25000 30000 571000 sizeufp 1842 905 4191 208 342 164 29 4113 3486 286 214 2758 1913 4669 850 productivity ufp/hr 0.37 0.54 0.32 2.12 0.58 0.36 0.73 0.05 0.10 1.19 2.14 0.57 1.59 1.87 0.61 an examination of the productivity range indicates that this varies from 0.05 to 2.14 function points per hour. data with such a large size and productivity range, with significantly different development attributes, and which inherently contains a lot of noise, further complicates effort estimation. to convert the size of a system measured in function points to a development effort estimate, a productivity factor has to be applied. productivity varies not only by programmer, but also by project size (dreger 1989). this implies that the effect of project size on productivity has to be quantified to enable an accurate estimate of development effort to be made (finnie and wittig 1993). with a limited sample size multiple regression analysis proved difficult to derive accurate effort estimates, and attempts have been made using the analytic hierarchy process (finnie et al 1993) to prioritise the effect of the various development factors. neural network model for this study, backpropagation artificial neural network models were used. backpropagation networks are the most generalised neural networks currently in use (nelson & illingworth, 1991) and this approach was chosen in preference to hopfield and kohonen networks. nelson and illingworth (1991) discuss some of the many networks which have been developed and give guidelines for possible applications. as software development estimation is not a time series problem, approaches such as finite impulse response (fir) and recurrent networks were not considered the backpropagation network requires data from which to learn. to learn the network calculates the error, which is the difference between the desired response and the actual response, and a portion of it is propagated backward through the network. at each neuron in the network the error is used to adjust weights and threshold values of the neuron, so that at the next epoch the error in the network response will be less for the same inputs. this corrective procedure is called backpropagation and is applied continuously for each set of inputs or training data. the training data should consist of as much relevant data as possible. in practice one does not usually have the luxury of a perfect training data set. with limited data one has to consider the trade-off between having as large a training set as possible and still leaving sufficient data points to test and validate the model. for this project the data were divided into three sets. the training set comprised ten developments, the test set three, and the validation set two. the data for each category were randomly chosen, except that the data in the test and validation sets was not allowed to be larger or smaller than the largest and smallest developments respectively in the training set. this was done so that predictions were not made outside the data range on which the network had been trained. the inputs were unadjusted function points, average development team size, systems analyst capability, systems analyst experience, the level of requirements volatility, the level of processing 89 ajis complexity, whether reusable code was developed or not, the required processing reliability, and the capability and the experience of the programmer team. all the inputs except for the average development team size and the unadjusted function points reflecting the system size, were converted to a binary notation. the target against which the network was trained was the development hours of the systems in the training set. the accuracy of the development effort estimate was taken as the root mean square error (rmse), proportionate to the size range of the systems in the data set to avoid over-training the network, the dangers of which are discussed later, and to be able to monitor the generalisation capability of the network, the training error and the prediction error were recorded at 100 epoch intervals. as the initial randomly generated starting network weights affect network performance these were saved, and the network was then re-run using the same starting weights, and stopped it when it had reached its minimum prediction error. analysis of data network models were developed with various combinations of inputs selected from the attributes mentioned above. the results were not particularly encouraging and the prediction errors were erratic and not satisfactory. an examination of the results showed that the network appeared to consistently overestimate the size of the very small systems, as well as to consistently underestimate the size of the very large systems. for the remaining systems, the estimated development effort was more accurate. network parameters to try and improve the network performance, the learning rate and momentum were varied, as was the network architecture. models with one through to six hidden layers were developed. consistently the models with just a single hidden layer performed better, while the models with multiple hidden layers in many instances did not converge. various activation functions were tried, and the popular sigmoid function consistently gave the best results. to solve the problem of the network not training and predicting well on such a large target range, the natural logarithm of the development hours was used. this compressed the range and improved the network performance. there is no clearly defined theory which allows for the calculation of the ideal parameter settings and as a rule even slight parameter changes can cause major variations in the behaviour of almost all networks (schoneburg, 1990). it is through a process of trial and error and experience that settings are selected which will result in a reduced average prediction error. the settings of the learning rate and momentum control the way in which the error is used to correct the weights in the neural network for each training case. when the learning rate is set to high values (close to 1) there is the possibility of unstable behaviour, as evidenced by widely varying average error values. when the learning rate is set lower, the possibility of unstable behaviour is reduced, but training times are increased and there is a greater probability of getting stuck in local error minima. the higher the momentum, the larger the percentage of previous errors that is applied to the weight adjustment in each training case. for example, when the momentum is set at 0.5, then so percent of the weight adjustment will be due to the current error and so percent will be the weight adjustment applied in the previous case. for this set of data a learning rate of 0.1 and a momentum of 0.7 gave good results. a neural network architecture of a single hidden layer using a sigmoid activation function tended to result in the lowest average prediction error. the best results were obtained with a 23-4-1 architecture, and table 2 shows the results up to 2000 epochs. the average training error is reduced steadily as the network trains, as is the prediction error. for this network the lowest average prediction error was obtained at about 2000 iterations. with further training the training error is further reduced, but the network does not generalise well, and from this point the average prediction error increases. the reason for this is that the network tends to curve-fit the training data, giving a low average training error, but this then leads to poor generalisation. may 1994 90 training iterations 100 200 300 400 500 600 700 800 900 1000 1100 1200 1300 1400 1500 1600 1700 1800 1900 2000 results average err 0.2748 0.2060 0.1504 0.1210 0.1063 0.0971 0.0892 0.0820 0.0757 0.0709 0.0673 0.0646 0.0626 0.0609 0.0598 0.0585 0.0576 0.0569 0.0562 0.0556 prediction iterations 100 200 300 400 500 600 700 800 900 1000 1100 1200 1300 1400 1500 1600 1700 1800 1900 2000 results average err 0.1548 0.1429 0.1422 0.1413 0.1374 0.1307 0.1205 0.1091 0.0967 0.0865 0.0788 0.0692 0.0589 0.0486 0.0386 0.0315 0.0238 0.0168 0.0129 0.0126 table 2 training and prediction errors network performance a larger data set would have been preferable, but this type of information is difficult to gather. other studies such as kemerer (1987) (15 cases) and (1993) (27 cases) are also limited by data availability. a result of a small data set was that the network performance was significantly influenced by the initialisation weights which are randomly generated. some sets of initialisation weights resulted in better convergence and a reduced average prediction error. as there is currently no known theory on the allocation of starting weights to optimise the network performance, and these are generated and allocated randomly, it meant that from repeated trials the weight set which resulted in the lowest average prediction error was selected. the results of the network which gave the lowest average prediction error are shown in table 3. the output has been normalised by converting it back from the natural logarithm and then rounding it both tile actual system size and that predicted by the artificial neural network are shown. the first ten data sets comprise the training set the next three comprise the test set, and the final two the validation set the error in the training set is not important as mentioned above, by training the network further, this error could be further reduced, but this would have degraded the model's generalisation ability, resulting in an increased prediction error. examining the results in table 3 reveals that neither the very smallest, nor the largest systems were included in the testing and validation set, as this would have meant that they would have been excluded from the training set, and would have resulted in having to predict outside this range. training was stopped when the prediction error was at its lowest the prediction capability of a network is judged by the test and validation sets. the three systems in the test set took 100, 1200 and 4864 person-hours to develop. the model predicted 89, 1184 and 4429 for the three systems respectively, resulting in error predictions of 11.1%, 1.3% and 9.0%a 91 ajis 4gl system 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 table 3 actual development hours 5027 1680 13300 98 588 450 40 81270 35000 240 100 4864 1200 2500 1400 prediction results estimated development hours 5015 1652 13085 269 377 262 62 62236 34832 240 89 4429 1 184 2144 1396 percentage error -0.24% -1.68% -1.62% 174.41% -35.89% -41.69% 55.42% -23.42% -0.48% 0.11% -11.08% -8.95% -1.30% -14.24% -0.27% category training training training training training training training training training training test test test validation validation the two systems in the validation set required 1400 and 2500 person hours to develop. the prediction for the smaller system was 1396 person-hours, which is within 1% of the actual development time. for the other system the error was 14.2%. discussion within the limited data set, backpropagation artificial neural networks appear to indicate the potential to be developed into good software size estimation models. in examining the model's performance the following factors need to be considered: 1. these systems were developed in an uncontrolled and natural (not artificial) environment only the data was gathered and there was no control over the development environment. 2. the software development environment is complex, with many and often interrelated factors affecting development effort. currently available size estimation models have on occasions not performed well when applied to systems which were developed outside of a strictly controlled environment (ferens & guraer, 1992; kemerer, 1987; mukhopadhyay et al, 1992). 3. the range of the system sizes is large, as is the variation in productivity. despite these difficulties the models performed well in terms of current level of prediction accuracy (ferens & gumer, 1992; kemerer, 1987; mukhopadhyay et al, 1992) 4. the research experience with the small data set highlighted the importance of the initial weights allocated to the network weight space. this makes the model development more difficult. additional data, which is difficult to obtain in large sets, should reduce the influence of the starting weights and make training networks more stable (lendaris, 1993). 5. the model is not difficult to develop and has the flexibility of being able to incorporate additional attributes as input if special circumstances warrant their inclusion. further research the model has produced good results, as noted above, in being able to predict on average within 10 percent of the actual software development effort in the data set to which it was applied. neural networks have the ability to capture knowledge of the complex interrelationships in their weight matrix to enable them to make predictions. it would be a distinct advantage if the model was also able to provide explanation to clarify the results of the reasoning process. even if the weight matrix is accessible, it is difficult to interpret, since the user has no way to 'decompile' the weights. neural networks have no explicit, declarative knowledge structure which allows the representation of explanation structures, such as reasoning paths, and explanation of expectation failures. with a small training set as was used in this study it is not possible to have full confidence in training. it would may 1994 92 require a large set of training examples (of a few thousand instances for example) to allow the network to learn a 'complete' domain theory (diederich, 1992). several proposals have been made for explanation components in plain neural networks as well as structured connectionist systems (diederich, 1992). further research will be conducted to study causality and explanation in neural network models. in this regard connectionist semantic networks (diederich, 1992) and cascade correlation networks (hoehfeld & fahlman, 1991) are of interest acknowledgement the authors would like to thank the anonymous reviewers for their considered comments and helpful suggestions. references albrecht, aj. & gaffhey, j.e. (1983) "software function, source lines of code, and development effort prediction: a software science validation", ieee transactions on software engineering, vol 9, no 6, pp 639-648. banker, r.d. datar, s.m. & kemerer, c.f. (1991) "a model to evaluate variables impacting the productivity of software maintenance projects", management science, vol 37, no 1, pp 118. boehm, b.w. (1981) software engineering economics, englewood cliffs, nj: prentice hall. conte, s.d. dunsmore, h.e. & shen, v.y. (1986) software engineering metrics and models, menlo park: benjamin/cummings. de marco, t. (1982) controlling software projects: management, measurement and estimation, new york, ny: yourdon press. diederich, j. (1992) "explanation and artificial neural networks", international journal manmachine studies, vol 37, pp 335-355. dreger, j.b. (1989) function point analysis, englewood cliffs, nj: prentice hall. ferens, d.v. & gurner r.b. (1992) "an evaluation of three function point models for the estimation of software effort", ieee national aerospace and electronics conferencenaecon 1992, vol 2, pp 625-642. finnic, g.r. & wittig, g.e. (1993) "effect of system and team size on 4gl software development productivity", south african computer journal, to be published 1994. finnic, g.r. wittig, g.e. & petkov, d.i. (1993) "prioritising software development productivity factors using the analytic hierarchy process", journal of systems software, vol 22, no 2, pp 129-139. hoehfeld, m. & fahlman, s.e. (1991), "learning with limited numerical precision using the cascade-correlation algorithm", carnegie mellon university report, cmu-cs-91-130. jeffery, d.r. (1987) "a software development productivity model for mis environments", journal of systems software, vol 7, pp 115-125. jeffery, d.r. & g.c. low, (1990) "calibrating estimation tools for software development, july, software engineering journal, pp 215-221. jones, c. (1986) programming productivity, new york: ny: mcgraw-hill. jones, c. (1991) applied software measurement new york: ny, mcgraw-hill. kemerer, c.f. (1987) "an empirical validation of software cost estimation models", communications of the acm, vol 30, no 5, pp 416-429. kemerer, c.f. (1993) "reliability of function points measurement", communications of the acm, vol 36, no 2, pp 85-97. lendaris g.g. (1993) professor of system science and electrical engineering, portland state university, personal communication 2 december 1993. low g.c. & jeffery, d.r. (1990) "function points in the estimation and evaluation of the software process", ieee transactions on software engineering, vol 1, no 1, pp 64-71. kitchenham, b.a. (1992) "empirical studies of assumptions that underlie software cost-estimation models", information and software technology, vol 34, no 4, pp 211-218. martin, j. (1982) application development without programmers, englewood cliffs, nj: prentice hall. 93 ajis mukhopadhyay, t. vicinanza, s.s. & prietula, m.j. (1992) "examining the feasibility of a casebased reasoning model for software effort estimation", mis quarterly, vol 16, no 2, pp 155-171. nelson, m.m. & illingworth, w.t. (1991) a practical guide to neural nets, reading, addisonwesley. rudolph, eje. (1986) "productivity in computer application development", auckland university research report, auckland university, auckland, 1986. rumelhart, d.e. widrow, b & lehr, m. (1993) "applications of neural networks in industry, business and science", stanford university, unpublished. schoneburg, e. (1990) "stock prediction using neural networks: a project report", neurocomputing, vol 2, no 1, pp 17-27. vessey, i. (1986) "on program development effort and productivity", information and management, vol 10, pp 255-266. white, h. (1988) "economic prediction using neural networks: the case of ibm daily stock returns", ieee conference on neural networks, vol 2. wittig, g.e. (1991) "an analysis of 4gl software development productivity", masters thesis, university of natal. may 1994 94 the australasian journal of information systems (ajis) began in 1993 and was based on the simple realisation that there was a need for an academic ‘outlet’ for is research in australia australasian journal of information systems volume 14 number 1 nov 2006 user behaviours associated with password security and management kay bryant department of management griffith university k.bryant@griffith.edu.au john campbell school of information sciences and engineering university of canberra john.campbell@canberra.edu.au abstract control mechanisms established on the boundary of an information system are an important preliminary step to minimising losses from security breaches. the primary function of such controls is to restrict the use of information systems and resources to authorized users. password-based systems remain the predominant method of user authentication despite the many sophisticated and viable security alternatives that have emerged from research and development. however, the literature shows that passwords are often compromised through the poor security and management practices of users. this paper examines user password composition and security practices for email accounts. the results of a survey that examines user practice in creating and using passwords are reported. the results show that many users know about the risks of hackers, viruses and so on and take preliminary steps to combat them such as having passwords longer than eight characters. however, this appears to be as far as many users are willing to accede to the probability that their information and computing resources can be compromised. this paper makes some recommendations for the education of users in creating and maintaining their passwords. the responsibility for these educational programs can be shared between governments, organisations, educational institutions at all levels, and software vendors. keywords: password management, password security, user behaviours, user authentication, password composition 81 australasian journal of information systems volume 14 number 1 nov 2006 introduction computers and information systems are pervasive in our modern world. the ready availability of the internet and its continuing growth on a global scale only extends access to widespread computing and communication networks. any communication sent over the internet travels across unsecured channels, raising the spectre of security breaches for all who use the internet. user identification and authentication along with encryption key distribution is one of the important functions provided by communication network services. in today's internet-based environment, the reach of organisational networks and its external connections is significant. as such, the use of specialized software and hardware such as firewalls is used to implement basic access control functions (oppliger, 1997). controlling access to system resources is usually a three-step process. firstly, users identify themselves to the mechanism, then they must authenticate themselves and the mechanism authenticates itself. lastly, users request information technology resources and the actions they will take and the mechanism will either permit or deny the request based on information held on files that denote the resources and actions a user is permitted to undertake. the means by which users make themselves known to the system is typically through a unique identifier such as a name or an account number. once the access control mechanism establishes that it has a valid user, authentication of that user is undertaken. establishing security on the boundary of a system is the first step in minimizing losses. access controls are the usual type of control implemented on the boundary of a system (weber, 1999). the function of these controls is to restrict the use of systems and resources to authorized user, as well as limiting the type of actions that a user can perform. there are three main approaches to user authentication: something the user has such as a smart card or other token; some physical characteristic of the user such as a fingerprint, retinal image, voiceprint or facial pattern; or something that the user remembers such as a password or pin (furnell et al., 2000). each approach has its advantages as well as its intrinsic flaws. regardless of the approach selected by the organisation, there is a trade-off between the value of the resources being protected and the effectiveness and cost of implementing and maintaining it. while significant advances have been made in graphics-based approaches (see for example man et al. 2004 and wiedenbeck et al. 2005) and in biometrics and visualization-based approaches (for example, see jain et al. 2004, de paula et al 2005), passwords remain the most common means of authenticating a user. despite their potential weaknesses, password-based systems prevail because they can provide effective protection if they are used correctly used. however, they are conceptually simple for both system designers and especially end users who often compromise password security by forgetting them, writing them down, sharing them with other people and selecting easily guessed words. research has shown that users are considered to be one of the main risks to the effectiveness of security measures designed to counter information system threats (rhodes, 2001; aytes and connelly, 2004). individuals are particularly vulnerable to security threats as many do not have adequate knowledge to recognise the risks in the first instance nor to implement appropriate protection mechanisms in the second instance. this study focuses on password issues by examining behaviours exhibited by user when creating and managing passwords. the aim of this study is to assess the attitudes and awareness of users to password security issues, and to gain insight into password composition, reuse, and management practices. this paper outlines the major problems associated with passwordbased authentication systems. computer security issues associated with passwords are discussed in the following section. 82 australasian journal of information systems volume 14 number 1 nov 2006 security issues associated with passwords the password-based approach to user authentication has a number of shortcomings that can undermine the efficacy of computer system security (see for example jobusch and oldehoeft 1989, furnell et al. 1999, conklin et al. 2004, carstens 2004, ives et al. 2004). there are many different methods used to compromise password security, some of which are unsophisticated requiring little or no technical knowledge while others require a high level of technical expertise. unsophisticated techniques include guessing, observing, viewing written records, being told, tricks and artifice and even sifting through rubbish bins. other methods that require a high level of expertise include keyboard monitoring, packet interception, keystroke interception, host emulation and so on. several studies have examined the ease with which passwords can be determined or ‘cracked’. in one of the earliest empirical studies, morris and thompson (1979) found that a personal computer could guess 86 percent of passwords in less than one week. subsequent replications of the study by klein (1990) and spafford (1992) found that password selection had improved over time with only 21 percent being guessed in a week. unfortunately, the software tools that can deduce passwords have become even more powerful and seditious in recent years. a modern program designed to crack passwords by brute force can discover a randomly generated 5-character alphabetic password in less than two seconds and an 8-character one in a little under ten hours (keith et al 2006). however, one saving grace is that longer passwords take longer to discover. notwithstanding this, if the password is commonly used or is a dictionary word, the time taken to discover it by brute force attack is significantly reduced (keith et al 2006). many researchers have suggested various strategies to overcome the threat of software to break passwords. the focus has been placed on various algorithms for encryption (bishop and klein 1995; liao et al 2006), the use of biometrics (walton 2005, jain 2004; ratha 2001) and graphics (man et al. 2004 and wiedenbeck et al. 2005). researchers have also been recommending strategies to overcome inherent limitations in password systems, most focused on the user (bishop and klein 1995). the major strategies for overcoming the inherent weaknesses in password usage include the following: • password lengths of at least 8 characters: longer passwords increase the time taken by software cracking programs to determine it. • passwords with mixed case/symbols: including both upper/lower case and symbols (!£$% etc.) in passwords requires attacks to use brute force methods and increases the number of character permutations that must be tried. • non-dictionary words: selecting non-dictionary passwords prevents the use of dictionarybased attacks. such attacks can identify a password in less than 20 minutes even on dictionaries with up to one million words. the only way to identify non-dictionary passwords is using a brute-force approach (testing every combination of characters for every length of password). • password ageing: should an intruder obtain a valid password, most systems will allow them to continue to access the system until the intrusion is noticed. users need to change their passwords regularly, thus forcing the intruder to identify the new password. while these strategies may help improve password security, these restrictions make the composition and memorizing of passwords a complex and unintuitive exercise. the rapid proliferation of electronic commerce and other public access systems requires users to manage a large number of passwords on a day-to-day basis. unfortunately, typical users are capable of managing a relatively small number of passwords – typically less than five unique passwords (adams and sasse, 1999). consequently, users may be tempted to reuse passwords to access multiple systems. the analysis of 83 australasian journal of information systems volume 14 number 1 nov 2006 data collected from a survey of 884 computer users reveals some interesting characteristics surrounding user practices in creating and maintaining passwords. a survey of email password security password reuse can compromise the security of all of the password systems that a user might access. cognitive limitations mean that many users will choose passwords that are easy to remember; typically these passwords are based on some meaningful combination of names and/or numbers (bishop and klien, 1995; brown et al 2004). if the security of one system is breached, then all other password-based systems and other computing assets might become vulnerable to unauthorised access and malicious damage. for example, a list of usernames and passwords gained from one system might then be used in brute-force attacks on other password-protected systems. many systems protect against brute-force attacks by restricting the number of unsuccessful logon attempts before denying further access. however, this type of control offers no protection from brute-force attacks that use datasets of different username and password combinations. electronic mail (email) is universally the most widely adopted password-protected application and affects the daily life of almost every working person in the industrialized world (rudy 1996, bälter 2000). it was concluded that email systems would prove a useful application domain to use as a research context because of its importance and widespread social and organisational impact. the aim of this study was to assess the attitudes and awareness of users to password security issues, and to gain insight into password composition and management practice. a copy of the survey instrument is contained in the appendix. the first section of the questionnaire collected demographic data about the participants and their computer and email usage. it also sought to ascertain the extent to which participants shared passwords across applications and their awareness of password cracking techniques. the second section focused specifically on password composition and management practices. study data undergraduate level students from an australian university business faculty were chosen to be the research participants. this sample can be considered as representing practicing, non-specialist computer users whose typical password security behaviours are indicative of those that potential employees might be expected to bring with them into organisations. the survey was administered across three campuses located in close proximity to one another (that is, no two campuses are more than 50 miles apart). all of the students surveyed were in the business school and in their first year of study. participation in the survey was entirely voluntary. in all, 884 students volunteered to participate in this study. table 1 shows the relevant demographic details for the participants. there were marginally more males than females in the sample1. the majority of the participants were under 26 years of age; 213 participants were under 18 and 584 between 18 and 25. the remaining 87 participants were mature aged (greater than 25 years of age). most of the participants were enrolled at the university on a full time basis (811) and 59 were enrolled on a part-time basis. 1 on one of the campuses sampled offered programs that typically attract a higher proportion of female students. therefore, it was not surprising that there were significantly more female participants than males from this particular campus. subsequently, tests were conducted across all of the study variables for introduced bias. these tests revealed that this phenomenon had no significant impact on any of the findings reported here. 84 australasian journal of information systems volume 14 number 1 nov 2006 thirteen participants did not respond to this question and one participant was auditing the course and therefore was not formally enrolled. similarly, most of the participants were either not employed (257) or employed on a part-time basis (533). sixty-five were full-time employees, while 29 participants did not respond to this question. the majority of participants had used computers for more than 5 years; 480 had used computers between 6-10 years and 252 for longer than 10 years. only 25 participants had used computers for less than 2 years, while 126 had used computers between 3 and 5 years. one participant did not respond to this question. variable category total % * gender male 378 42.8% female 505 57.1% no response 1 0.1% age < 18 years 213 24.1% 18 – 25 years 584 66.1% 26 – 35 years 55 6.2% 36 years + 32 3.6% enrolment full-time 811 91.7% status part-time 59 6.7% not enrolled 1 0.1% no response 13 1.5% employment full-time 65 7.4% status part-time 533 60.3% not employed 257 29.1% no response 29 3.3% computing 0 – 2 years 25 2.8% experience 3 – 5 years 126 14.3% 6 – 10 years 480 54.3% > 10 years 252 28.5% no response 1 0.1% total participants: 884 100.0% * percentage totals may exceed 100% due to rounding. table 1: demographic details of participants analysis participants were asked to indicate for what purposes they used computers; selecting as many options as was relevant. more than 83% of participants indicated their main use was for internet (92.9%), email (90%) and home use (83.4%). banking (50.2%) and work use (47.7%) formed a second grouping and other areas of use (e.g. study and research; entertainment including games; and online purchasing and selling) accounted for 15.0%. participants were also asked to indicate what their email usage was. personal email use was most prevalent (95.5%), followed by university use (84.7%) and work-related use (24.9%). the majority of participants had either two or three email accounts; 49.4% had two and 27.9% had three. the remaining participants had either one account (11.7%), or they had four or more email accounts (11.0%). almost half of the participants accessed their email at least once a day, with another 27.7% accessing several times a week. sixty-one participants did not answer question. given that more than 95% percent of participants reported that they used email for personal communication, serious implications for organisational security are raised, especially if users reuse passwords across email 85 australasian journal of information systems volume 14 number 1 nov 2006 and other computer applications. this is because password reuse and poor security practices increase the likelihood that a password might be deduced thereby increasing the vulnerability of other systems where this password had been used. participants were then asked questions concerning the composition and choice of passwords – see table 2a and b. the average length of a password was 8.3 characters, and ranged between 1 and 25 characters. the majority of participants had passwords of greater than 5 characters in length. participants typically used 8 characters in their password (29.2%), and 7.4% of participants had passwords exceeding 11 characters. approximately 39.4% used only alphabetic characters in their passwords, while 42.3% used alphanumeric characters. the remainder either used numerals only (6.4%); added symbols (4.1%); or did not respond to the question (7.5%). typically, their choice of password contained meaningful data (43.1%) such as a name, street, preferred word, nickname, registration number and so on. a few selected pronounceable words (5.2%). another 23.8% combined meaningful data items to make up their passwords. only 10.7% choose a random combination of characters. very few participants had their passwords chosen for them (1.6%), while another 8% selected their password by some other means. there were 61.9% of participants who never changed their password and a further 19.8% who changed it no more than three times a year. participants were divided with respect to admitting whether they had forgotten their password – 60.9% said they had not forgotten it compared to 30.4% who had; 8.7% chose not to answer this question. 86 australasian journal of information systems volume 14 number 1 nov 2006 variable & category total % ^ password length 1-5 characters 31 3.5 6 characters 126 14.3 7 characters 93 10.5 8 characters 258 29.2 9 characters 104 11.8 10 characters 68 7.7 11 characters 34 3.8 > 11 and <26 characters 65 7.4 no response 105 11.9 password composition 1. alphabetic only 348 39.4 2. numeric only 57 6.4 3. alphanumeric 374 42.3 4. includes symbols 36 4.1 5. other 3 0.3 no response 66 7.5 choice of password 1. meaningful data 381 43.1 2. combo meaningful data 210 23.8 3. pronounceable word 46 5.2 4. random characters 95 10.7 5. not self-chosen 14 1.6 6. other 71 8.0 no response 67 7.6 frequency of changing password 1.never 547 61.9 2. less than once a year 119 13.5 3. 1-3 times a year 56 6.3 4. 4-6 times a year 79 8.9 5. once a month 10 1.1 6. several times a month 6 0.7 did not respond 67 13.5 forgotten password 1. yes 538 60.9 2. no 269 30.4 did not respond 77 8.7 ^ percentage totals may exceed 100% due to rounding. table 2a: participant practices relating to password composition and management since the survey data is categorical, nonparametric statistics were employed in analysing the data across three variables: gender, age and employment status. table 2b provides details of the results of the nonparametric analysis conducted on participant practices related to password composition and management. the results from the analysis on the number of characters in a password have not been reported, as there were no significant findings for any of the independent variables. there were significant differences in the type of characters used in the composition of passwords for all independent variables; employment status and age group at the 1% level, while gender was significant at the 5% level. females are more likely to choose alphabetic or numeric characters only, while males will choose a combination of characters, including symbols. full-time employees and those who are unemployed tend toward combination of characters while those who are employed on a part-time basis are more likely to choose alphabetic or numeric characters only. with respect to 87 australasian journal of information systems volume 14 number 1 nov 2006 age group, participants aged 25 years and under are more likely to choose alphabetic or numeric characters only, while the older participants will choose alphanumeric combinations that may include symbols. there is a significant difference in the method of choosing passwords for gender only. females are more likely to choose more meaningful detail or some combination thereof, while males tend towards pronounceable passwords or a random combination of characters. the frequency of changing passwords was only significant for age groups. older participants are likely to change their passwords more often than those who are younger. there was only a significant difference in whether the participant had forgotten their passwords for age group. it appears that the older the participant, the more likely they are to forget their passwords. password composition groups count mean rank p-value gender male female 346 470 430.54 393.10 0.014 ns age groups under 18yrs 18-25yrs 26-35yrs 36 and over 202 541 48 26 347.91 418.05 515.03 524.91 0.000 employment full/time part/time not employed 63 499 230 413.23 378.03 432.00 0.004 choice of password groups count mean rank p-value gender male female 346 470 438.36 386.52 0.001 age groups under 18yrs 18-25yrs 26-35yrs 36 and over 202 541 48 26 393.46 412.32 389.48 530.12 0.290 ns employment full/time part/time not employed 62 497 231 388.98 399.82 387.96 0.761 ns frequency of changing password groups count mean rank p-value gender male female 347 469 412.77 405.34 0.593 ns age groups under 18yrs 18-25yrs 26-35yrs 36 and over 201 541 49 26 383.33 410.73 470.46 402.91 0.042 ns employment full/time part/time not employed 62 497 231 433.20 388.80 399.79 0.204 ns 88 australasian journal of information systems volume 14 number 1 nov 2006 forgotten password groups count mean rank p-value gender male female 339 467 391.45 412.25 0.125 ns age groups under 18yrs 18-25yrs 26-35yrs 36 and over 200 534 48 25 374.41 404.00 471.25 488.92 0.002 employment full/time part/time not employed 61 492 229 383.29 386.07 405.63 0.383 ns ns = not significant p > 0.01 table 2b: results of non-parametric analyses of number of password composition and management table 3a and 3b provide relevant details about password reuse; specifically those passwords associated with email accounts and other computer applications. over half of participants used the same password (24.9%) or a slight variation of that password (31.2%). more than one-third used passwords that were very different (36.3%). sixty-seven participants (7.6%) did not answer this question. the participants were also asked whether they used other applications that required the use of passwords. approximately 60% of participants use passwords for other applications. there were three predominant groups: (1) banking, (2) other university applications and (3) communication applications such as chat rooms, messenger services and forums. when asked whether they reused the same passwords across other applications, 37.5% reported using the same password (17.4%) or a slight variation (20.1%). approximately 40% of participants did not answer this question. variable & category total % password reuse across email accounts 1. same password 220 24.9 2. slightly different 276 31.2 3. no similarities 321 36.3 no response 67 7.6 password reuse across other applications 1 same password 154 17.4 2. slightly different 178 20.1 3. no similarities 194 21.9 no response 358 40.5 percentages have been calculated in terms of the total number of participants. table 3a: participant practices related to password reuse 89 australasian journal of information systems volume 14 number 1 nov 2006 password reuse across email accounts gender groups count mean rank p-value male female 349 467 391.98 420.85 0.065 ns age groups under 18yrs 18-25yrs 26-35yrs 36 and over 198 541 51 27 396.39 412.50 418.20 423.40 0.905 ns employment full/time part/time not employed 61 490 238 465.02 399.72 367.34 0.005 password reuse across other applications gender groups count mean rank p-value male female 227 298 259.79 265.44 0.653 ns age groups under 18yrs 18-25yrs 26-35yrs 36 and over 116 344 43 23 247.19 261.87 286.15 315.39 0.178 ns employment full/time part/time not employed 49 325 152 303.49 249.85 251.71 0.039 ns ns = not significant p > 0.01 table 3b: results of nonparametric analyses of password reuse password reuse was tested using nonparametric analyses with gender, age group and employment status as independent variables. table 3b shows that there is no significant difference in reusing passwords across email accounts that is associated with age. however, there is a significant difference with respect to employment status (at 1% level). full-time employees have the least similarity, while unemployed participants have the most. there is only a significant difference due to gender at the 10% level, where females are more likely to have less similar passwords than males. with respect to sharing passwords across other computer applications, there is no difference due to gender and age group. there is a significant difference due to employment (5% level). full-time employees are more likely to have less similar passwords for other computer applications than parttime employees or the unemployed. 90 australasian journal of information systems volume 14 number 1 nov 2006 gender no one sib-ling parent partner/ spouse other relative close friend colleague other blank total male female no response 242 271 0 26 47 1 12 24 0 41 93 0 6 6 0 36 75 0 4 7 0 4 6 0 34 37 0 405 566 1 age under 18 18-25 yrs 26-35 yrs 36 and over 122 338 34 19 12 58 3 1 9 26 1 0 21 94 12 7 2 8 1 1 45 65 1 0 4 6 1 0 5 4 1 0 12 47 6 6 232 646 60 34 employment full-time part-time unemployed no response 39 295 164 15 6 48 17 3 3 25 7 1 15 82 33 4 1 5 6 0 4 82 21 4 1 6 3 1 1 6 2 1 3 39 27 2 73 588 280 31 totals 513 74 36 134 12 111 11 10 71 972 totals exceed 884 due to selection of multiple options table 4a: participant practices related to password sharing participants were also asked to indicate their personal practices related to sharing passwords and whether or not they kept written or electronic copies of their passwords. participants were asked to select as many options as were relevant to them. tables 4a, 4b and 4c show the results across the variables of interest, namely, gender, age and employment status. these outcomes are very promising. table 4a shows that over half of the participants (52.78%) reported that they did not share their passwords with others, and another 7.3% of participants did not respond to this question. the tendency to not share their password was very similar for males (24.90%) and females (27.88%). within the age and employment groups, 18-25 year olds (34.77%) and part-time employees (30.35%) respectively, were the ones least likely to share their passwords. for those who did report sharing their password, spouses or partners (13.79%) was the preferred choice followed by close friends (11.24%) and siblings (7.61%). table 4b reports on whether participants kept a handwritten record of their password. nearly threequarters (74.4%) of participants did not, while 8.57% did not respond to this question. females (41.43%) were less likely to keep handwritten records than males (32.86%). the 18-25 year old group (49.89%) and part-time employees (46.70%) were also less likely to keep handwritten records. for those who did report keeping a handwritten copy of their password, diaries (5.49%) was the preferred choice. other choices included notebooks (2.64%), drawers (2.53%) and desks (2.09%). 91 australasian journal of information systems volume 14 number 1 nov 2006 table 4b: participant practices related to handwritten records of passwords similar results were apparent for participants reporting on whether they kept an electronic copy of their password (table 4c). nearly 80% of participants did not keep an electronic record, but 10.95% did not respond to this question. as with handwritten copies, females (46.59%) were less likely to keep and electronic record than males (32.96%). similar results are also apparent with age and employment groups; 18-25 year olds (53.30%) and part-time employees (49.72%) reported not keeping electronic records. the mobile phone (4.80%) was the option of choice for those who did keep an electronic copy. overall, 640 participants (72.40%) did not keep any kind of record of their passwords, while 8.14% did not answer either question. a few of the participants did not keep an electronic copy but kept a handwritten copy in their wallet (4.07%), a drawer (1.81%) or in their desk (1.13%). others had no handwritten record but kept a copy on their mobile phone (2.38%), while another 24 participants (2.71%) did not indicate whether they kept an electronic record. table 4c: participant practices related to electronic records of passwords 92 australasian journal of information systems volume 14 number 1 nov 2006 participants were also asked whether they were familiar with password cracking techniques. again, participants were asked to select as many options as were relevant to them. details organised by gender, age and employment status are shown in table 5. table 5: participant practices related to electronic records of passwords gender not aware worm virus program file other blank total male female no response 4 6 0 192 161 1 235 292 1 147 103 1 36 2 1 96 155 0 710 719 4 age not aware worm virus program file other blank total under 18 18-25 yrs 26-35 yrs 36 and over 3 5 0 2 85 236 20 13 131 350 29 18 65 163 15 8 10 26 2 1 65 157 19 10 359 937 85 52 employment not aware worm virus program file other blank total full-time part-time unemployed no response 1 3 4 2 33 208 102 11 38 329 145 16 21 152 69 9 3 25 2 9 18 153 73 7 114 870 395 54 totals 10 354 528 251 39 251 1433 totals exceed 884 due to selection of multiple options table 4c: participant practices related to electronic records of passwords the data show that the majority of participants (81.79%) are familiar with at least one technique for cracking passwords. only 10 participants (0.70%) reported not being aware of any such techniques, while 17.52% did not respond to the question. the technique selected most often by participants was viruses (36.85%), followed by worms (24.70%), program files (17.52%) and other techniques (2.72%). there was very little difference between awareness of females (50.17%) compared to males (49.55%). as was the case with sharing passwords and keeping records of passwords, the 1825 year old group (65.39%) were more aware than other age group categories, while those employed part-time (60.71%) were most aware for the employment status group. lastly, participants were asked whether they had changed their password because they believed someone else had had discovered it. the majority of participants responded in the negative (585 or 66.18%), 228 (25.34%) said they had and 75 (8.48%) did not respond to this question. discussion email accounts are heavily used with approximately 30% of participants checking their email several times a week and a further 50% who check one or more times a day (table 2). what is concerning, is the reuse of the exact or similar passwords for different email accounts and other applications (table 3). many of the participants responding to these two questions used the exact same password or had passwords with a slight variation. one promising factor was that half of the passwords contained a combination of alphabetic, numerical and symbol characters and was on average 8 characters in length. another positive fact was that password length ranged between six and 25 characters (table 2) with 60% of participants having passwords of eight or more characters in length. password of this length increase the time taken for password detection software to determine them. however, while these results are positive, the fact that almost three-quarters of the passwords contained meaningful detail, a combination of meaningful details or pronounceable 93 australasian journal of information systems volume 14 number 1 nov 2006 words reduces its impact. this implies the passwords are easier to guess, especially by those who have even a modicum of personal knowledge about the user. a serious lack of concern with password security is indicated when this outcome is coupled with the fact that over three-quarters of the participants, never changed their password or changed it no more than three times a year. this finding is similar to that of the aytes and connelly (2004) study who found that participants rarely, if ever, changed their passwords voluntarily. it appears that while most participants have some rudimentary knowledge of good password practices, for example, password lengths of 8 or more characters, this is as far as the participants were prepared to go to secure their passwords. the extent of password reuse is an issue that organisations appear to be addressing, given that full-time employees are less likely to reuse passwords than unemployed or part-time employees. in an attempt to increase security, organisations may be taking the necessary security measures. nonetheless, it would appear that without the influence of full-time work environment, participants are either not sufficiently informed of the risks they are facing through poor password practices or they do not believe that they are at risk even though most are aware of one or more of the techniques used to break passwords. it also appears that age is a contributing factor to poor password practices. younger participants are more likely to have simplistic passwords; that is, those derived using alphabetic and numeric characters. further, they don't change their password as frequently as older participants, but they are also are less likely to forget their password. this finding may simply be due to the fact that younger participants change their password less frequently so familiarity has an impact. in addition, they do not have to remember complex passwords. it would appear that the younger generation is more in need of education programs. gender differences seem to balance the various behaviours. both males and females have poor practices, although in different areas. females are more likely to have simplistic passwords that are meaningful, but they also are less likely to reuse them. males, on the other hand, have pronounceable words using combinations or symbols and characters but are more likely to reuse them. neither situation is ideal, so it can be concluded that both females and males have equally poor practices when it comes to password management. overall, participants appear to be unconcerned about the risks associated with poor password composition and management practices. this outcome is of concern, given that over 82% of the participants can be considered as experienced computer users since they had been using computers for six or more years. it would appear there is a need for a better education process on password composition and management for users. the education process should also focus on the risks of not having appropriate password practices and what the consequences are for failure to adhere to such practices. this issue of who should provide this education is one yet to be investigated. governments are a likely provider especially since the youngest and the older participants have been shown to have weakest password practices. education can take the form of information provided online, but many older members of the community do not have access to the internet. newsletters are another alternative, but this approach relies on the user reading and understanding the information it contains. information sessions can be held, but again not all users would be able to or even willing to attend. this option would be the most expensive one and thus least likely to be adopted by governments. a tutorial available on cd, dvd or online is another strategy that could be used to educate users. this technique has been used with some degree of success (gips, 2001). gips reported that the product delivered awareness messages to users via animation and graphics on screensavers. further, gips reported that while user awareness increased, it was necessary to change 94 australasian journal of information systems volume 14 number 1 nov 2006 the screensaver regularly so they did not become bored with the message it was portraying. while animated tutorials may be expensive to develop initially, provided they are set up using the latest information and rules for designing strong passwords – ones difficult to crack – means the product can be used for some time and so reduce the overall cost involved. such a product could be provided free online or sent on cd or dvd to households. education could also be provided as part of study programs. tertiary institutions can provide information on good password practices during a student’s first year of study. however, given the results of this study, tertiary education is perhaps too late. since children are using computers and accessing the internet from an early age, primary schools may be the best option for providing education on password composition and related behaviours. good practices can be developed from the outset of the child’s use of computers rather than trying to correct bad habits some years later. video clips, animated tutorials or games would most likely hold the child’s attention better than printed sheets of information or formal classes. teachers can introduce the learning material and allow the children’s skills to develop over time. these types of activities could be extended into their secondary education. follow-up ‘lessons’ can be part of the curriculum for all students undertaking secondary education. equivalent classes can be established for the adult learner at tafe colleges, adult education classes and even provided through local libraries. software vendors are another likely provider of this educational material. vendors who supply relevant software such as operating systems, firewalls and other security software, email systems and such like, could also build in tutorials or rule sets for constructing and maintaining passwords. these components should be an integral part of the software suite and not considered as an add-on. further they should, by default, be active in the application ready for when the user needs to change their password. as a user creates their password, rule sets can be provided so they can better understand what they are required to do and why. the strength of the created password can be assessed and, if it does not reach a predetermined level, the password can be rejected. feedback is provided to the user so they fully understand why their selected password was a security risk. rewards can be provided to those users who create strong passwords on their first attempt. the rewards need not have a high monetary value; they can be a straightforward as officially recognising appropriate behaviours. rewarding positive and desired behaviours is more likely to achieve results than negative or fear-based strategies. these are a few strategies for providing users with appropriate information to develop good password practices and security awareness. all of these strategies have merit, so some combination of them would provide the broadest possible coverage to the general public. organisations and businesses can also take a role in reinforcing this knowledge, rather than assuming their employees are aware of the risks of poor password behaviours. simply educating their employees on the reasons why security and password practices have been implemented rather than keeping them ignorant due to misguided sense of “the need to know” can move users towards adopting good password and security practices. one organisation reported punishing its employees for poor security behaviours by requiring them to attend extremely mind-numbing courses on good password practices (tuesday, 2001). the punishment for continued poor practices was attendance at even longer mind-numbing courses. 95 australasian journal of information systems volume 14 number 1 nov 2006 conclusion the aim of this study was to assess the attitudes and awareness of users to password security issues, and to gain some insight into password composition, reuse, and management practice. this study explored aspects of user password management practice within the context of email usage by profiling email account usage and password reuse and management practice. the results from this study provide important insight into ongoing issues relating to the creation and management of userbased password management systems. the poor practices evidenced in the survey results support our initial focus on email account management as an important end-user application context. participants typically operate two or more email accounts as well as a host of other computer-based applications that require the use of passwords. as anticipated, this created password management difficulties for users and encouraged password reuse across different email accounts, and/or other computer-based applications. further, users do not appear to fully realise the risks of their poor password practices, nor are they aware of many of the consequences associated with breaches in security. the poor password composition practices adopted by many of the participants clearly highlight this problem. our results show that the vast majority of users are choosing passwords that are based on meaningful personal details that can be more readily guessed by others. it does appear that being a full-time employee is a mitigating factor, but this is more likely due organisational security requirements rather than better personal practices by these employees. however, further research is needed to fully explore this finding. younger participants appear to have the worst practices in terms of password management and so are the most at risk. any educational programmes should be targeted towards this group in the first instance. these programmes should highlight the risks of such poor practices in terms younger people can relate to. several options for providers of this education have been suggested including governments, educational institutions and software vendors. some combination of all options would provide the broadest possible coverage. while there have been significant technological developments in online authentication methods, user behaviours associated with password practices is an area that remains under researched. the results of this expanded study show that on the whole, the majority of users do not adopt secure management practices. this in turn exposes them, the organisations for which they work and their service providers, to higher levels of risk and potential breaches in security. future research needs to build upon this understanding and aim to gain further insight into how user practices can be improved. the research can build on this study's findings to determine the underlying reasons why younger people tend to have the poorest password practices. it would be useful to discover at what point these practices were established and determine if early intervention during secondary or even primary school would have payoffs. further research could also be targeted towards full-time and part-time employees in order to examine whether it is organisational requirements that is driving their password practices. any research within organisations should also discover whether security requirements actually interfere with the functions employees are required to undertake as part of their jobs. such interference may be a factor in explaining why poor security behaviours are still being practiced. any knowledge gained from such research can then be used as a basis for sound educational programs that focus on improving personal password practices as well as helping the organisation understand why users seek to circumvent security measures. 96 australasian journal of information systems volume 14 number 1 nov 2006 references adams, a. & sasse, m.a. (1999) "users are not the enemy", communications of the acm, vol 42 no 12, pp 41-46. aytes, k. & connolly, t. (2004) "computer security and risky computer practices: a rational choice perspective”, journal of organisational and end-user computing, vol 16 no 3, pp 22-41. bälter, o. (2000) "how to replace an old email system with a new", interacting with computers, vol 12 no 6, pp 601-614. bishop, m. & klein, d.v. (1995) "improving system security via proactive password checking" computers & security, vol 14 no 3, pp 233-249. brown, a.s., bracken, e., zoccoli, s. & douglas, k. (2004) "generating and remembering passwords", applied cognitive psychology, vol 18, pp 641-651. carstens, d.s., mccauley-bell, p., malone, l.c. & demara, r.f. (2004) "evaluation of the human impact of password authentication practices on information security", informing science journal, vol 7 no 1, pp 67 – 85. conklin, a., dietrich, g. & walz, d. (2004) "password-based authentication: a system perspective", proceedings of the 37th hawaii international conference on system sciences, hawaii. de paula, r., ding, x., dourish, p., nies, k., pillet, b., redmiles, d.f., ren, j., rode, j.a. & filho, r.s. (2005) "in the eye of the beholder: a visualization-based approach to information system security", international journal of human computer studies, vol 63 no 1-2, pp 524. furnell, s.m., dowland, p.s., illingworth, h.m. & reynolds, p.l. (2000) "authentication and supervision: a survey of user attitudes", computers & security, vol 19 no 6, pp 529-539. gips, m.a. (2001) "plugging into awareness", security management, vol 45 no 11, pp 21-22. ives, b., walsh, k.r. & schneider, h. (2004) "the domino effect of password reuse", communications of the acm vol 47 no 4, pp 75-78. jobusch, d.l. & oldehoeft, a.e. (1989) "a survey of password mechanisms: part 1", computers & security, vol 8 no 7, pp 587-604. jain, a.k., ross, a. & prabhakar, s. (2004) "an introduction to biometric recognition", ieee transactions circuits systems video technology, vol 14 no 1, pp 4–20. keith, m., shao, b. & steinhardt, p.j. (2006) "the usability of passphrases for authentication: an empirical study", international journal of human-computer studies, forthcoming. klein, d. (1990) "a survey of, and improvements to, password security", proceedings of the usenix second security workshop, portland, oregon, august 1990: pp 5-14. liao, i.e., lee, c.c. & hwang, m.s. (2006) "a password authentication scheme over insecure networks", journal of computer and system sciences, vol 72 no 4, pp 727-740. 97 australasian journal of information systems volume 14 number 1 nov 2006 man, s., hong, d., hayes, b. & matthews, m. (2004) "a password scheme strongly resistant to spyware", proceedings international conference on security and management, las vegas, pp 94-100. morris, r. & thompson, k. (1979) "password security: a case history", communications of the acm, vol 22 no 11, pp 594-577. oppliger, r. (1997) "internet security: firewalls and beyond", communications of the acm, vol 40 no 5, pp 92-105. ratha, n.k, connell, j.h. & bolle, r.m. (2001) "enhancing security and privacy in biometricsbased authentication systems", ibm systems journal, vol 40 no 3, pp 614-634. rhodes, k. (2004) "operations security awareness: the mind has no firewall", computer security journal, vol 16 no 2: pp 27-36. rudy, i.a. (1996) "a critical review on research on electronic mail", european journal of information systems, vol 4, pp 198-213. sherman, r. (1992) "biometrics futures", computers & security, vol 11 no 2, pp 128-133. spafford, e.h. (1992) "opus: preventing weak password choices", computers & security, vol 11 no 3, pp 273-278. tuesday, v. (2001) "human factor derails best-laid security plans", computerworld, vol 35 no 18, p 52. walton, r.w. (2005) "combining biometric measurements for security applications", computer fraud & security, vol 4, pp 7-13. weber, r. (1999) information systems control and audit, prentice-hall, upper saddle river, nj. wiedenbeck, s., waters, j., birget, j.c., brodskiy, a. & memon, n. (2005) "passpoints: design and longitudinal evaluation of a graphical password system", international journal of humancomputer studies, vol 63 no 1-2, pp 102-127. zviran, m. & haga, w.j. (1999) "password security: an empirical study", journal of management information systems, vol 15 no 4, pp 161-185. 98 australasian journal of information systems volume 14 number 1 nov 2006 appendix 99 australasian journal of information systems volume 14 number 1 nov 2006 100 user behaviours associated with password security and management introduction study data analysis composition and management password reuse across other applications appendix microsoft word 13 1 marina.doc ajis vol 13, no. 1 september 2005 120 a credit union case study of the business value of is/it graham pervan and hilangwa maimbo school of information systems curtin university of technology gpo box u1987, perth, australia 6845 graham.pervan@cbs.curtin.edu.au ; hilangwa.maimbo@standardbank.co.za abstract this paper investigates the business value of is/it investments in the financial services sectors (fss) in australia using a case study methodology to develop and refine a conceptual model of the relationship between is/it investment and organisational performance. the conceptual model has four main components, is/it investment, organisational performance, considerations for strategic information systems planning and managerial effectiveness. a case study of a credit union indicates support for components of the conceptual model. further, the analysis reveals the existence of intermediary factors, operations, product delivery, customer service and staff. it is concluded that directing is/it investments to these areas, particularly when it is done in concert, is most likely to have a positive impact on organisational performance. lastly, debate on the issue of an is/it investment threshold and its applicability to organisations is presented. keywords information systems, information technology, investment, organisational performance, case study, financial services introduction this article presents results from a case study that examines the relationship between information systems/information technology (is/it) investment and organisational performance within the financial services sector (fss) in australia. the case study was conducted in a single medium sized credit union. financial institutions (fis) are heavily reliant upon is/it in all facets of their business, from purely operational aspects to strategic applications that are designed to gain competitive advantage and provide leverage in the market place (apte & vepsalainen 1993). consequently, fis tend to allocate significant portions of their total capital expenditure budgets to acquiring new technologies and to maintaining existing is/it portfolios (carrington, llanguth & steiner 1997). as the fss tends to be regulated (lilja 1999), sometimes heavily, these expenditures may be discretionary (driven from within the organisation) or mandatory (required for regulatory compliance). in addition, the fact that revenues are derived from a variety of channels further complicates the is/it investment and organisational performance relationship. clearly is/it investments are deemed to contribute to organisational performance, but how they do that and where they have the most impact remain subjects of much debate (harker & zenios 2000b). given that total is/it expenditure represents a significant cost to any fi, the need for managers to understand this relationship has become increasingly important, as has the need to demonstrate that potential and existing investments in is/it are or will be beneficial to the organisation (dos santos 1991). thus if fis are to use is/it as a viable competitive weapon, their respective senior management teams (and indeed all staff) need to appreciate and understand the potential impact of is/it, not only on their organisations, but on the external environment as well. this in turn is complicated by the need to balance pressures to report and demonstrate good short-term financial performance against longer-term growth and investment strategies in modern fis. consequently some authors argue that ajis vol 13, no. 1 september 2005 121 is/it investments are benefiting consumers rather than fis in that the efficiencies brought about by these investments create a huge opportunity cost resulting in reduced profits and shareholder returns (carrington, llanguth & steiner 1997). although the foregoing issues are being viewed from the perspective of fis in particular, it should be pointed that issues relating to the business value of is/it in general have been the subject of much research in the past (kivijarvi & saarinen 1995). closer examination of this field of study reveals a number of themes as exemplified in table 1. it is also well worth noting that much of this previous research appears to have been conducted in what may be considered to be less information-intensive industries such as manufacturing. paradoxically, the fss, which is typically regarded as a highly information-intensive industry appears to have received less attention. theme sample studies effect of is/it at an industry level sager (1988); doll (1989); palmer & markus (2000); andersen (2001) performance of is/it department borovits & gilardi (1993); raghunathan, gupta & sundararaghavan (1989); miller & doyle (1987); miller (1993); slevin, stieman & boone (1991) information systems performance and productivity scudder & kucic (1991); menon, lee & eldenburg (2000); floyd & wooldridge (1990); banker, kauffman & morey (1990); panko (1991) is/it investment and performance sethi, hwang & pegels (1993); li & ye (1999); weill (1992); dos santos, peffers & mauer (1993); dewan, michael & min (1998); thatcher & oliver (2001) is/it evaluation/value tam (1998); tam (1992); davern & kauffman (2000); dos santos (1991) is/it benefits realisation/management lin & pervan (2003); silk (1990); ward, taylor & bond (1996); jurison (1996) table 11: summary of research themes and studies in the business value of is/it literature the rest of this article is laid out as follows: the next section reviews the is/it investment and organisational performance literature after which a brief overview of the credit union sector of the fss in australia is then presented. following that the research methodology and design are discussed and the results presented after which a summary is given. finally, some preliminary conclusions are put forward which are then followed by a discussion of the limitations and directions for further research. is/it investment and organisational performance lucas (1993) identified three main objectives for the growing interest in understanding the is/it investment and organisational performance relationship: • the need to better understand how is/it is implemented and used • to better understand the impact of is/it • for justifying new and continued investments in is/it by demonstrating ajis vol 13, no. 1 september 2005 122 perceived benefits. although there have been a number of studies of this relationship over the years, careful consideration of this body of research and literature reveals three issues: • a lack of theory (alavi & carlson 1992; gregor 2002) • problems with measurement (kauffman & weill 1989), and, • lack of generalisability of results, possibly due in part to (a) and (b) (kauffman & weill 1989). theory issues kaplan and duchon (1988) observed that the dominant approach to research in the field of mis in general has been based on the positivist (quantitative) philosophy and that most studies measure quantitative outcomes such as technical impact, economic value, improvements in effectiveness/efficiency and financial performance. although lucas (1993) noted that researchers have in fact used theory from a number of fields/disciplines in an attempt to better understand this relationship (table 2) there still appears to a lack of strong testable theory on the business value of is/it (markus & soh 1993). discipline theories microeconomics theory of the firm production function efficiency economics information economics finance/accounting capital asset pricing model time value of money determination of interest rates options theory behavioural science organisational behaviour cognitive science table 12: disciplines that contribute to this research (adapted from lucas (1993)) consequently, in much of this earlier research, there appears to have been a tendency to ‘test’ hypotheses using a variety of statistical analytical techniques applied to collected data using models derived from and developed using these contributing theories (kauffman & weill 1989; capon, farley & hoenig 1990). markus and soh (1993) identified and lamented the apparent lack of consensus in the is/it investment literature and noted the lack of theory in this field of study. indeed, there have been many calls to build strong testable theory on the business value of is/it. further, it may not be enough to simply adopt a theory from another discipline, no matter how sound, without first developing a deeper understanding of the phenomenon that is under investigation. there needs to be a structured approach to building a cumulative tradition of research into the business value of it if we are to better understand this relationship. it is partly in recognition of this need to lay the theoretical foundations that this study utilises case study as the methodology (yin 1994; cavaye 1996) by adopting a theory development approach to build a conceptual model of this relationship. this in turn should lead to further testing of the components of the model and thus a better understanding of the business value of it. ajis vol 13, no. 1 september 2005 123 methodological problems kauffman and weill (1989) provide a well-structured review of literature in this area in which they identified a number of weaknesses in previous research; • equivocality on the contribution of is/it investments to the value/performance of organisations. • predominant use of statistical data analysis techniques. • predominance of the positivist paradigm. • lack of consideration of context (which is recognised as an important issue when considering performance). • lack of explicit consideration of actual system performance. • application of theory from other disciplines (economics, marketing, organisation theory etc). • use of different measures. the apparent lack of a tradition of using commonly accepted measures may lead to inconsistencies in results obtained and therefore affect the generalisability of the results. while the above review only covers literature prior to 1989, tt may be argued that this is still a definite weakness in the literature (gregor & hart 2002) and there is a need to build theory by using methods such as case study. given the lack of consistency of reported results from previous studies, the generalisability of these results is considered low due to differences in research methods and measures employed. from a purely contextual perspective (trauth 2001), one might also argue that both the relative differences in time periods when the studies were conducted and the locations/regions in which they were conducted would have an influence on the results obtained. in order to overcome these issues, this research project adopted and used case study methodology to develop a better understanding of the business value of is/it in context and so that findings may more be generalisable, thus contributing to the existing body of knowledge on this issue (cavaye 1996). the is/it investment threshold it is generally agreed within the is/it domain that organisations need to maintain a level of expenditure in order to remain competitive (willcocks & lester 1999; harker & zenios 2000a). however, there appears to have been little work done in the field to determine what might be considered an appropriate level of investment. whilst conducting research in the insurance industry, bender (1986) observed that there appeared to be a threshold of investment in is/it that these organisations needed to maintain in order to remain competitive. comparing information processing expenses with total expenditure, bender (1986) noted from his sample of 132 companies that 22 companies had a ratio of between 5% and 10%, 35 companies had a ratio of between 15% and 20% and concluded that there seemed to be an optimum level or threshold of is/it investment of around 20% to 25%. the establishment of such a threshold could have implications for fis as it could be used as a benchmark by which fis could compare their respective is/it expenditures. determining such a threshold for an industry could in turn contribute significantly to the strategic information systems planning (sisp) process in that the threshold could be viewed as both an enabler when justifying expenditure relative to other fis or a limiter when used by ajis vol 13, no. 1 september 2005 124 organisations to cap is/it expenditures. developing the conceptual model in order to better understand the business value of is/it, a model of this relationship, grounded in the literature, is proposed. the conceptual model suggests the existence of a relationship between the level of investment in is/it, as depicted by the is/it portfolio (markus & soh 1993), and organisational performance. the model does not in any way suggest causality between the two major components. to illustrate how the model was developed, consider early models of is investment and organisational performance. beginning with a simple cause and effect model and using the organisation as the unit of analysis (mckeen & smith 1993b), it may be argued that is/it investment leads to better organisational performance as shown in figure 1. figure 1: a simple cause and effect view of the business value of is/it this view, though intuitive, is simplistic and fails to take into account the effect of factors such as the role of management, the environment and the fact that organisational performance has both an internal; and an external aspect. in addition, this view assumes that all is/it investment is one and the same, yet organisations invest in different types of technologies and build a portfolio of these technologies that together constitute their total technological capability. a broader perspective is therefore necessary if we are to better understand this relationship. trice and treacy (1986) suggested utilisation as the link between the two major components and that other factors, such as is budgets, personnel and is efficiency, may also influence the relationship. figure 11: an expanded view of is/it investment and organisational performance (trice & treacy 1986) mckeen and smith (1993, p 625) tested this model and concluded “there is a strong linear relationship between it investment and it usage”. their study and its result, though encouraging, had limited generalisability due to a number of weaknesses, most notable being the fact that the study was based on a specific technology (main frames). in addition, the model is open-ended and does not suggest any form of feedback or secondary interaction between organisational performance and it investment. this poses an interesting question, how can the investment be judged to have brought about any purported increases in performance if this secondary link is not explicitly considered? since then, a number of models have been proposed, each being a slight variation from the theme. some, such as weill (1992) and markus and soh (1993) went further and suggested the need to it investment organisational performanceit usage other factors other factors it investment organisational performanceit usage other factors other factors it investment organisational performanceit investment organisational performance ajis vol 13, no. 1 september 2005 125 consider the impact of different types of is/it investment within a given organisation and introduced the idea of the role and effect of management. others studies suggested the need to consider the effects of time lags and the impact of learning and growth (brynjolfsson 1993; devaraj & kohli 2000). therefore, the proposed conceptual model of this relationship has four main components, the is/it portfolio (infrastructure, transaction processing and management information systems), organisational performance (internal & external), managerial effectiveness and considerations for strategic information systems planning (maimbo & pervan 2002) and organisational context. this research was focused on the fss in and it is proposed that the conceptual model described applies to fis. the largest single industry within the fss, in australia, is the credit union industry. an overview of australia’s credit union industry now follows. credit union industry in australia crapp and skully (1985, p1), define a credit union as; “a democratically controlled voluntary co-operative society of individuals, bound together by a common bond for the pursuit of the economic welfare of members through the receipt of funds from members and other, and the provision of loans and other forms of credit and financial services to members” the credit union movement in australia began after the second world war, when the credit union concept was observed in canada by australian royal air force personnel and as a direct response to the perception that traditional banks were unresponsive to customer needs (cooke 1989). in australia, most credit unions are members of a larger body, credit union services corporation australia limited (cuscal), which serves as both an industry body and a service provider to its members. cuscal has approximately 181 member unions with a 32.45% share of the market as at the end of 2001 (woccu 2001). at an international level, cuscal is a member of the world council of credit unions (woccu), a worldwide body that has members and affiliates around the world. with representation in over 90 countries, woccu represents 37,000 credit unions and over 112 million members (woccu 2003). credit unions in australia comprise the largest single part of the financial services sector with a combined asset base of approximately aud$24 billion and approximately 3.5 million members. growth within this sector has continued to be positive and in 2001/2002 credit unions posted a combined growth of 9% (cuscal 2002). credit unions provide a wide range of products and services including loan products, atm access, internet banking services, credit card services and insurance. in fact, it has been noted that credit unions in australia have been very innovative in the development of financial products and service and consequently have been noted for being the first to introduce some of the services that consumers today often take for granted. some of these achievements include (crapp & skully 1985); • free bill payment facilities • free payroll deductions facilities • free life insurance coverage on credit/borrowings • first to have on-line computer operations • first to introduce atms and pos • first to introduce internationally linked atms and also to provide access ajis vol 13, no. 1 september 2005 126 to visa’s international worldwide atm network as the first non-usa based fis • first to offer high interest cheque accounts • first to offer computer based personal financial planning • first to introduce home banking as interest in customer service continues to grow, the issue of service quality (duncan & elliot 2002) gains even more prominence. consequently, the future for credit unions as an alternative to commercial/retail banks is strong, if they can continue to remain relevant by providing comparable or better financial services, at lower cost. for credit unions this is an area that is/it can provide competitive advantage if applied and utilised effectively and efficiently. for this reason, it is proposed that the conceptual model described above be investigated in credit unions and the approach to this investigation is set out below. research methodology and design due to the exploratory nature of the research and the need to further the development of the conceptual model, a highly structured case study methodology was chosen. thus, recommendations from yin (1994) and miles and huberman (1994) were adopted in order to address methodological issues including rigour and validity within the research project. further, to simplify manageability of an otherwise complex research project, the research design incorporated a phased approach built around the framework proposed by eisenhardt (1989) by grouping tasks into phases of model development, model testing and model refinement. one of the key aspects of the research design was the development of a comprehensive case study protocol (yin 1994). this protocol outlined the procedures and rules that governed the conduct of the researchers and the research project. the case study protocol defined the type of data that would be collected, contained the research instrument used to collect data (a detailed and structured interview guide) and provided a guide for data analysis. the primary data was collected through a series of interviews with senior management involved in the is/it investment decision-making process. contact with the credit union case reported here was initially made informally through the chief executive officer. a meeting was arranged during which the scope and objectives of the research were discussed and this was later followed up by a formal request. once potential interviewees had been identified, interview sessions were then scheduled. all in all, four interviews were conducted, each approximately 60 – 90 minutes long, over a twoweek period. in this case, the participants were the ceo, and three department heads, mcs (manager, corporate services), smfa (senior manager, finance & administration) and itm (it manager). each interview was recorded and when later transcribed averaged around 25 pages. in addition, a large amount secondary data in the form of internal financial data, annual reports, enterprise architecture schemas, meeting notes, strategy/planning documents, web based documents, industry statistics and other documentation/data were also collected. the purpose of collecting such a diverse range of data facilitated the creation of a database of information on the case and enabled triangulation and convergence of information (yin 1994) in order to get a clearer view of the issues at hand. the data were analysed using a qualitative content analysis approach that utilised a variety of techniques including inductive, deductive, comparative and pattern matching (miles & huberman 1994). ajis vol 13, no. 1 september 2005 127 results background decu8 (pvt) ltd is a medium sized credit union operating in australia. it has over 50 000 members and a fairly widespread branch network within the metropolitan area of a state capital. due to its size, decu has a relatively flat organisational structure built along functional lines with the corporate head office providing administrative & back office functions and points of representation (pors) such as branches that act as the front office thus providing contact with customers/members. like most credit unions in australia, decu (pvt) ltd is a member of the credit union services corporation australia limited (cuscal) and has outsourced its central processing to co-processing pty ltd, a company that runs the integrated data processing centre (idpc). co processing pty ltd was established in 1993 as a separate company owned by its founding members (credit unions) with the purpose of providing a cost effective centralised data processing environment (tsw 2003). consequently, and by virtue of its membership, decu enjoys both economies of scale and scope, and it is able to utilise powerful core computing services and systems that it would otherwise be unable to afford on its own. given that decu not only competes with other credit unions but also competes with much larger commercial/retail banks, its reliance on is/it is therefore significant and its ability to have access to such services critical to its existence. decu provides a wide range of services and products to its members including automated teller machines (atms), electronic funds transfer/point of sale (eft/pos), internet banking, loans, mortgages, term deposits, savings accounts, insurance, credit cards, financial planning, share trading, estate planning, travel and many more. over the past two years decu has experienced a profitability growth of over 60% and much of this is has been actually attributed to previous and ongoing investments in information technology. decu prides itself on its ability to deliver a wide range of products and services both effectively and efficiently to its members through state of the art technology. decu’s is/it investment threshold figure 3 is a summary of key ratios of is/it investment against expenditure and revenue. as can be seen, decu appears to have an is/it investment threshold of around 5%. although this figure appears to be consistent with some of the results reported by bender (1986) and therefore encouraging with respect to establishing the concept of an is/it threshold, caution needs to be exercised in the interpretation of this result before any generalisations can be made. this is due to differences in research design, industry, period and region between this current study and the earlier study. 8 pseudonym, the real name has been suppressed for confidentiality reasons ajis vol 13, no. 1 september 2005 128 figure 12: key is/it investment ratios for decu (pvt) ltd level of investment (information systems/technology portfolio) decu (pvt) ltd has a diverse is/it portfolio that is quite complex. in addition, decu has a number of relationships with external third party providers as it outsources much of its is/it. in terms of the conceptual model, all three components of the is/it portfolio were observed in this organisation. one very interesting observation made during the study of this organisation were the different views (table 3) of the senior management team in terms of the relative importance of the components of the is/it portfolio. ratings of importance participant infrastructure transactional dss/mis ceo 3 3 4 mcs 5 5 2 smfa 3 5 4 itm 4 3 1 table 13: participant ratings of relative importance of is/it portfolio components these rather diverse views of the relative importance of systems that comprise the is/it portfolio, coupled with a rather loose definition of is/it by each of the participants seemed to point to the lack of a deep appreciation of the roles that each part of the is/it portfolio plays within the organisation. consequently, this may lead to a lack of appropriate attention being given to components of the is/it portfolio when expenditure decisions need to be made. this observation was further reinforced with comments made by itm that there was a certain element of ‘scope creep’ due to the lack of a more formalised definition of what constitutes is/it with the result that the it function was having to take on everything that is ‘technology’. further, discussions with mcs identified the need to develop more formalised policies for the management of is/it, but in order to do this, more formalised definitions of what constitutes is/it in the organisation will be required (frenzel 1992). despite these apparent issues, there is a consensus that is/it investments as represented 0% 1% 2% 3% 4% 5% 6% percentage 1997 1998 1999 2000 2001 2002 year is/it expenditure ratios is/it expenditure as a percentage of overall expenditure is/it expenditure as a percentage of revenue ajis vol 13, no. 1 september 2005 129 by the is/it portfolio are both important and critical to the organisation operation as highlighted by mcs; “there’s two things which we are, we are a retail organisation and we rely enormously on it… if it doesn’t work we have people sitting there twiddling their thumbs.” much of decu’s investment in is/it has been directed at process improvement, reducing cost whilst growing the business. according to the ceo; “…there’s a lot on money to be saved on processes, our back office area has shrunk by about a tenth over the last five years while we have grown our business, probably doubled the size of our business but we’ve shrunk our back office” whilst the above may be true, and the organisation is always seeking ways to cut or maintain costs at a manageable level, it is pertinent to note that decu, like many other credit unions, still has a relatively high cost to income ratio (around 80% for credit unions in general (duncan & elliot 2002)). such a high cost to income ratio may have negative implications for decu in the following key areas: • customer service – may negatively impact decu’s ability to service customers cost effectively. • long term profitability – being a mutual organisation, decu is driven primarily by member service and not profit and shareholder return from the ceo’s comments, we can see that decu certainly views its is/it investment as part of its strategy to deal with the above issues. other members of the senior management team concurred with this view and agreed that is/it investments to date have contributed positively to organisational performance, although as mcs put it: “i’ve got a view of organisations like this that, there is very little of what we do in this organisation where you can put your finger on something and say that is the thing that makes the difference. the question is does it make a difference and i’d argue very strongly, yes, but is it 10% or 20% or 50%, i would not know.” in terms of the ability of existing systems to meet the organisation’s needs, we observed that although much of the decu’s infrastructure has been modernised, the core processing system is still considered to be the area that needs the most improvement primarily because of its age. according to smfa: “so the core system, while it is a good banking system that does everything it is supposed to do in terms of a banking system, the sort of data it produces is a little bit antiquated for what we need, particularly for things like marketing.” in addition: “its getting a little bit cumbersome, it can’t handle financial products that have become fashionable in the market.” given the importance of core processing systems (rated higher in importance on average than the other components of the portfolio), we surmised there is a need to consider the future of these systems in the long run. further discussion with smfa revealed that some alternatives have been considered (through and by cuscal) but caution is warranted due to the significant impact that any changes would have on member credit unions as whole. clearly, the need to understand the business value of is/it is an issue whose importance has been recognised by decu as it strives to maximise on existing investments whilst at ajis vol 13, no. 1 september 2005 130 the same time planning for future investments in is/it. organisational performance one of the main objectives that this research aims to achieve is the identification of suitable key indicators that can be used to benchmark organisational performance in the conceptual model. duncan and elliot (2002), in their study of customer service and quality, identified interest margins, return on assets and capital adequacy as useful performance indicators in the fss. campbell (1992) suggested that organisational performance in the fss may be best understood using what he termed the four main income drivers for banking institutions, net interest income, non interest income, operating expense and credit quality. clearly, the diversity of performance indicators could be problematic as it makes benchmarking and comparison of organisational performance difficult (palmer & markus 2000). although it is generally accepted that most industries have commonly accepted sets of performance indicators that are used, not only for performance reporting, but also for benchmarking purposes. the caveat is that even where such indictors are used, there may still be a need to normalise them in order to ensure that they all measure the same thing. an example of such would be depreciation where different rates of depreciation may be applied which then affects the ‘value’ of is/it assets reported. this may then affect any performance ratios or indicators that incorporate this item, such as return on assets (roa). the fss is no exception in this regard. stakeholder theory (freeman 1984; key 1999) provides some interesting insights and a probable explanation as to why this diversity exists. the analysis showed the key indicators of organisational performance to be: • cost to income ratio • return on equity • interest income • non-interest income • interest margins the indicators were cross-referenced against industry statistics as reported to apra (apra 2003) in order to confirm their validity and suitability as organisational performance indicators for use by credit unions. interesting enough, mcs suggested that organisational performance indicators might not always be considered when it came to making is/it related decisions. this was further corroborated by smfa who suggested that although cost might be mitigating factor it was certainly not a limiting factor and that it was really the perceived benefits that were the deciding factor when it came to is/it investments. in some respects, these arguments appear to support observations in the literature regarding the difficulty of relating is/it investments to financial measures (bresnahan 1986) and suggest that sometimes is/it investments may actually result more from qualitative assessments and even ‘gut feel’ or ‘hunches’ than anything else. in addition to these mainly financial (external) performance indicators, our research found that decu regularly measures its non-financial (internal) performance using surveys directed at both customers and staff. the benefits of using these management tools can be quite significant as the organisation develops a clear understanding of how it is perceived and the relevance of its products and services in the market place (duncan & elliot 2002). thus, having a clear set of indicators for organisational performance is necessary to better understand the business value of is/it. this may also include incorporating some qualitative measures as well. additionally, by using a consistent set of indicators ajis vol 13, no. 1 september 2005 131 researchers may be better positioned to understand the business value of is/it (palmer & markus 2000). this provides support for the organisational performance component of the conceptual model. managerial effectiveness five factors have been identified in the literature as comprising the managerial effectiveness component of the conceptual model (weill 1992; markus & soh 1993). these factors were identified as: • senior management commitment to is/it • experience of organisation with is/it • user satisfaction • political environment • organisational structure the research has found that the factors are present in decu. most striking in decu was the degree to which decu exhibits the first factor. decu’s senior management appear to exhibit a high level of commitment to is/it. discussions with itm provided the following comments with respect to senior management commitment, i.e.: “definitely absolutely, 100%.” and: “i think with all senior management, with all management… seeing the benefits of it, it has to have a positive effect…i think most managers realise that it is critical to them doing their jobs.” in fact, the ceo candidly admitted to not having a strong is/it background and not having been particularly in favour of is/it as enabler. however, he credits his change of perspective to his second-in-charge, mcs, whom he says has helped him to learn more about and appreciate benefits of utilising is/it: “and i will be honest to say that as the ceo, i would not have had such a strong contribution from is/it if it wasn’t for mcs and his ability to convince me to adopt this path” by taking up the challenge to educate the ceo on the benefits of is/it, mcs succeeded in bridging the gap between senior management and the is function, thus aligning their possibly divergent perspectives and in the process has facilitated the creation of an environment in which is/it use is encouraged through greater senior management commitment (peppard 2001). this willingness to learn about is/it, particularly at such a senior level, seems to be a strong cultural issue that senior management appears to be cultivating within the staff on the back of the realisation that most if not all of the work performed is via computers. management therefore plays an important role in determining and encouraging the effective utilisation of is/it resources in the organisation. as smfa said; “purely on the basis that if managers and senior managers are using the technology effectively, then everyone will use it as well. there’s are a lot of things that must necessarily come from the top.” thus, staff were actively encouraged to use computers and to learn about the various ajis vol 13, no. 1 september 2005 132 products and services so that they may better serve members. one cultural issue that has been identified in the literature as having a possible negative effect was the factor relating to the political environment (stacey 1993). decu however did not exhibit any such negative effects. this may be related to, and mediated by, two issues, the organisational structure (factor #5) and the apparent cohesiveness of the management team. in decu’s case, both these issues may be a direct result of the size of the organisation (stacey 1993). this leads to the conclusion that the impact of the managerial effectiveness construct on the determination of level of investment (and eventual utilisation) of is/it cannot be under estimated. as suggested in the literature the more effective the management process, the better the utilisation of is/it within an organisation (harris & katz 1989; sohal & ng 1998). considerations for strategic information systems planning (sisp) decu do not have any formalised process for sisp neither does it have a formal documented is/it strategy. according to mcs; “…we do actually have an it strategy, we just don’t have it written down. so we’ve actually started to put that together.” to this the ceo adds; “i think in reality we don’t really have a strategic plan for our it. we tend more to have a strategic plan for the business, ah and then obviously identify issues within that are relevant to the it area.” further enquiries revealed that decu does instead have a structured corporate planning process through which its corporate objectives and financial targets are set. as things stand management appear to be happy with the rather informal nature of the planning process for is/it. though this approach appears to have suited decu in the past and is currently in use, we are of the opinion that as decu continues to grow, more structure will have to be built around the sisp process. it may be that this informal approach has already had an effect particularly on how management views the relative importance of components of decu’s is/it portfolio as discussed earlier. this may have important ramifications for decu moving into to the future when it comes to determining is/it investment priorities (grover, teng & fiedler 1998). organisational context one of the major weaknesses identified in earlier research has been the lack of a deliberate attempt to consider the organisational context when designing research projects on the business value of is/it (kauffman & weill 1989) although there has been a growing call for researchers to consider the issue of context in all aspects of is research (trauth 2001). this, in part, has led to an increased use of interpretive approaches in is research. organisational context is important and it is important that is professionals and academics develop an appreciation of the organisational context in which such decisions are made in order for those decisions to be relevant. this issue of context assumes a higher level of significance for the fss as it is a regulated industry. ajis vol 13, no. 1 september 2005 133 two major organisational issues were observed. the first led to the conclusion that there are two main types of is/it investment decision for fis, discretionary and mandatory. in the context of the fss, discretionary investments are those is/it investments driven from within the organisation and targeted at any level of the is/it portfolio to achieve or deliver a given set of benefits to the organisation. conversely, mandatory is/it investments as those is/it investments driven by the need to attain regulatory compliance as set by the industry’s regulatory authority from time to time. the second observation was that it is possible for organisations to make investments into is/it that may have such an impact on the environment as to change the way the organisation competes9. this was particularly interesting and lends support to mcfarlan's (1984), and porter & millar's (1985) views on is/it and competition. from that perspective, organisations may make investments into is/it in order to gain competitive advantage, however, the corollary to that is that is/it alone will not give an organisation long term competitive advantage as it can easily be duplicated (willcocks, feeny & islei 1997). thus, is/it investments need to be made in the context of a broader strategy in order to achieve sustainable competitive advantage (barua, kriebel & mukhopadhyay 1991). summary decu (pvt) ltd provided an interesting case study on the business value of is/it. as mentioned earlier this case is fact part of a series case studies conducted in the fss with the aim of developing a model on the relationship between is/it investment and organisational performance. consequently, we observed some new themes/factors that will enable the extension and refinement of the conceptual model. emergent themes the following themes or factors were identified in the case as being those areas to which is/it investment is targeted and that in turn have a direct impact on organisational performance as follows; • operations (alpar & moshe 1990)* – banking operations relating to front office functions (points of representation) and back office functions (including head office). • product delivery (carrington, llanguth & steiner 1997)* – channels through which customers access products and services. • customer service (delone & mclean 1992)* – activities relating to the satisfying customer needs and managing relationships with customers. • staff (delone & mclean 1992)* – ensuring that staff have adequate access to is/it resources and are able to fully utilise them. *indicates support in the literature for each factor the analysis suggests that this set forms a group of intermediary variables and leads to the conclusion that there will be some interaction between the variables themselves. organisations that focus their is/it investment in these areas (in concert) are more likely to gain positive performance benefits. further, the level of focus on each area will be different 9 specific examples cannot be provided here, as that would identify the organisation in question. ajis vol 13, no. 1 september 2005 134 for different organisations. thus, it may be concluded that fis will invest in an is/it portfolio in such a manner that they will target each of these areas for improvement with express aim of achieving improvements in organisational performance. however, at this stage causality between such is/it investments and organisational performance is not yet proven as much more work needs to be done in refining and testing the model. conclusions financial institutions are by their very nature information intensive organisations (porter & millar 1985). organisations in this sector therefore present an excellent opportunity for study into the business value of is/it. utilising case study methodology the research was conducted in a medium size credit union and the results so far indicate support for the conceptual model and its components. in addition, observations indicate the presence of a set of intermediary variables at which fis appear to be directing is/it investment. comparison with the literature has provided preliminary support for the set of intermediary variables. therefore, it is concluded that decu needs to focus is/it investments in these areas, as that is likely to lead to performance improvements provided these investments are made in the context of a broader corporate strategy. for this to be effective, decu needs to develop polices for is/it including, but not limited to, a definition of what constitutes is/it in decu and who is responsible for which aspects of that is/it. such policies will also ensure a common understanding of the relative importance of the different parts of the is/it portfolio so that is/it investment priorities can easily be identified. in addition, a more formalised process of sisp is recommended for decu as the current informal process may not be sustainable as the organisation grows and competition intensifies. as part of this process, decu will need to more carefully consider the implications of their is/it investment decisions by monitoring their level of is/it expenditure and benchmarking against competitors using performance indicators such return on equity (roe), non-interest income, interest income and cost to income ratio. limitations and directions for further research the findings for this case study are very encouraging and may lead to a better understanding of the relationship between is/it investment and organisational performance. however, more work still needs to be done to verify the emergent themes thereby refining and testing the conceptual model, before these results can be generalised across the fss. as mentioned earlier, this is but one case in a broader research project involving multiple case studies and utilising a structured case study methodology. the findings reported here will be used for both “within case” analyses and “cross case” comparisons to and with other findings in the fss as the research program progresses. it is hoped that this further refinement and testing will lead to a more generalisable conceptual model of the business value of is/it. references alavi, m. & carlson, p. 1992, 'a review of mis research and disciplinary development', journal of management information systems, vol. 8, no. 4, pp. 45-62. alpar, p. & moshe, k. 1990, 'a microeconomic approach to the measurement of information technology value', journal of management information systems, vol. 7, no. 2, pp. 55-69. andersen, t. j. 2001, 'information technology, strategic decision making approaches and ajis vol 13, no. 1 september 2005 135 organisational performance in different industrial settings', journal of strategic information systems, no. 10, pp. 101 119. apra 2003, australian prudential regulatory authority statistics page, available: [http://www.apra.gov.au/] (01 july 2003). apte, u. m. & vepsalainen, a. p. j. 1993, 'high tech or high touch? efficient channel strategies for delivering financial services', journal of strategic information systems, vol. 2, no. 1, pp. 39 54. banker, r. d., kauffman, r. j. & morey, r. c. 1990, 'measuring gains in operational efficiency from information technology: a study of the positran deployment at hardee's inc', journal of management information systems, vol. 7, no. 2, pp. 29-54. barua, a., kriebel, c. & mukhopadhyay, t. 1991, 'an economic analysis of strategic information technology investments', mis quarterly, pp. 312 331. bender, d. 1986, 'financial impact of information processing', journal of management information systems, vol. 3, no. 2, pp. 232-238. borovits, i. & gilardi, r. 1993, 'evaluating cost/utilisation of organisations' information systems and users', information and management, no. 25, pp. 273 280. bresnahan, t. f. 1986, 'measuring the spillovers from technical advance: mainframe computers in the financial services.', american economic review, vol. 76, no. 4, pp. 742-755. brynjolfsson, e. 1993, 'the productivity paradox of information technology', communications of the acm, vol. 35, pp. 67-77. campbell, r. j. 1992, 'does technology deliver?', bankers monthly, vol. 109, no. 3, p. 17. capon, n., farley, j. u. & hoenig, s. 1990, 'determinants of financial performance: a meta analysis', management science, vol. 36, no. 10, pp. 1143 1159. carrington, m., llanguth, p. & steiner, t. d. 1997, 'the cost of technology', the banker, vol. 147, no. 861, pp. 49-50. cavaye, a. l. m. 1996, 'case study research: a multi-faceted research approach for is', information systems journal, no. 6, pp. 227-242. cooke, r. 1989, 'aussie style: member service and employee development get top billing down under', credit union management, no. august 1989, pp. 22 26. crapp, h. & skully, m. t. 1985, credit unions for australians, george allen and unwin australia pty ltd, north sydney. cuscal 2002, annual report 2002, credit union services corporation australia limited. davern, m. j. & kauffman, r. j. 2000, 'discovering potential and realised value from information technology investments', journal of management information systems, vol. 16, no. 4, pp. 121 143. delone, w. h. & mclean, e. r. 1992, 'information systems success: the quest for the dependent variable', information systems research, vol. 3, no. 1, pp. 60-95. devaraj, s. & kohli, r. 2000, 'information technology payoff in the health-care industry: a longitudinal study', journal of management information systems, vol. 16, no. 4, pp. 41 67. dewan, s., michael, s. c. & min, c.-k. 1998, 'firm characteristics and investments in information technology: scale and scope effects', information systems research, vol. 9, no. 3, pp. 219 232. doll, w. j. 1989, 'information technology's strategic impact on the american air travel service industry', information and management, no. 16, pp. 269 275. dos santos, b. l. 1991, 'justifying investments in new information technologies', journal of management information systems, vol. 7, no. 4, pp. 71-90. ajis vol 13, no. 1 september 2005 136 dos santos, b. l., peffers, k. & mauer, d. c. 1993, 'the impact of information technology investment announcements on the maket value of the firm', information systems research, vol. 4, no. 1, pp. 1 23. duncan, e. & elliot, g. 2002, 'customer service and financial performance among australian retail financial institutions', journal of financial services marketing, vol. 7, no. 1, pp. 25 41. eisenhardt, k. m. 1989, 'building theories from case study research', academy of management review, vol. 14, no. 4, pp. 532 550. floyd, s. w. & wooldridge, b. 1990, 'path analysis of the relationship between competitive strategy, information technology, and financial performance', journal of management information systems, vol. 7, no. 1, pp. 48-64. freeman, r. 1984, strategic management: a stakeholder approach, ballinger, boston, ma. frenzel, c. w. 1992, management of information technology, boyd and fraser, boston, ma. gregor, s. 2002, 'a theory of theories in information systems', in information systems foundations: building the theoretical base workshop, eds. gregor, s. & hart, d., school of business and information management, australia national university, australia national university, canberra. gregor, s. & hart, d. (eds.) 2002, information systems foundations: building the theoretical base, school of business and information management, australia national university, canberra. grover, v., teng, j. t. c. & fiedler, k. d. 1998, 'is investment priorities in contemporary organisations', communications of the acm, vol. 41, no. 2, pp. 40 48. harker, p. t. & zenios, s. a. (eds.) 2000a, performance of financial institutions, cambridge university press, new york. harker, p. t. & zenios, s. a. 2000b, 'what drives the performance of financial institutions?', in performance of financial institutions, eds. harker, p. t. & zenios, s. a., cambridge university press, new york, pp. 3-31. harris, s. e. & katz, j. l. 1989, 'predicting organisational performance using information technology managerial control ratios', in proceedings of the 22nd annual hawaii international conference on systems sciences, cambridge university press, new york, pp. 197-204. jurison, j. 1996, 'toward more effective management of information technology benefits', journal of strategic information systems, no. 5, pp. 263 274. kaplan, b. & duchon, d. 1988, 'combining qualitative and quantitative methods in information systems research', mis quarterly, pp. 571-586. kauffman, r. j. & weill, p. 1989, 'an evaluative framework for research on the performance effects of information technology investment', in proceedings of the tenth international conference on information systems, boston ma, butterworthheinemann, 313 washington st, newton, ma, newton, ma, pp. 377-388. key, s. 1999, 'toward a new theory of the firm: a critique of stakeholder 'theory'', management decision, vol. 37, no. 4. kivijarvi, h. & saarinen, t. 1995, 'investment in information systems and financial performance of the firm', information and management, no. 28, pp. 143 163. li, m. & ye, r. 1999, 'information technology and firm performance: linking with environmental, strategic and managerial contexts', information and management, no. 35, pp. 43 51. ajis vol 13, no. 1 september 2005 137 lilja, p. 1999, 'structural changes in the banking sectordriving forces and consequences', quartely review sveriges riksbank, no. 4, pp. 43-63. lin, c. & pervan, g. 2003, 'a public sector case study on evaluating and managing the benefits is/it', in technologies and methodologies for evaluating information technology in business, ed. davis, c. k., group publishing, hershey, usa. lucas h. c. jr.,1993, 'the business value of information technology: a historical perspective and thoughts for future research', in strategic information technology management: perspectives on organisational growth and competitive advantage, eds. banker, r. d., kauffman, r. j. & mahmood, m. a., idea group publishing, harrisburg, pa, pp. 359-374. maimbo, h. & pervan, g. 2002, 'a model of is/it investment in the banking industry', in ninth european conference on information technology evaluation, eds. brown, a. & remenyi, d. s. j., universite paris dauphine, france, 15 16 july. markus, l. m. & soh, c. 1993, 'banking on information technology: converting it spending into firm performance', in strategic information technology management: perspectives on organisational growth and competitive advantage, eds. banker, r. d., kauffman, r. j. & mahmood, m. a., idea group publishing, harrisburg, pa, pp. 375 403. mcfarlan, f. w. 1984, 'information technology changes the way you compete', harvard business review, vol. may-june 1984. mckeen, j. d. & smith, h. a. 1993a, 'linking investment with it usage'. mckeen, j. d. & smith, h. a. 1993b, 'the relationship between information technology use and organisational performance', in strategic information technology management: perspectives on organisational growth and competitive advantage, eds. banker, r. d., kauffman, r. j. & mahmood, m. a., idea group publishing, harrisburg, pa. menon, n. m., lee, b. & eldenburg, l. 2000, 'productivity of information systems in the healthcare industry', information systems research, vol. 11, no. 1, pp. 83 92. miles, m. b. & huberman, a. m. 1994, qualitative data analysis: an expanded source book, 2nd edn, sage publications inc., california. miller, j. 1993, 'measuring and aligning information systems with the organisation', information and management, no. 25, pp. 217 228. miller, j. & doyle, b. a. 1987, 'measuring the effectiveness of computer based information systems in the financial services sector', mis quarterly, pp. 106 124. palmer, j. & markus, l. m. 2000, 'the performance impacts of quick response strategic alignment in specialty retailing', information systems research, vol. 11, no. 3, pp. 241 259. panko, r. r. 1991, 'is office productivity stagnant?', mis quarterly, pp. 191 203. peppard, j. 2001, 'bridging the gap between the is organisaiton and the rest of the business: plotting a route', journal of information systems, no. 11, pp. 249 270. porter, m. e. & millar, v. e. 1985, 'how information gives you competitive advantage', harvard business review, vol. 63, no. july august, pp. 149 160. raghunathan, t. s., gupta, y. & sundararaghavan, p. s. 1989, 'assessing the impact of is executives' critical sucess factors on the performance of is organisations', information and management, no. 17, pp. 157 168. sager, m. t. 1988, 'competitive information systems in australian banking', information and management, no. 15, pp. 59 67. scudder, r. a. & kucic, a. r. 1991, 'productivity measures for information systems', information and management, no. 20, pp. 343 354. ajis vol 13, no. 1 september 2005 138 sethi, v., hwang, k. t. & pegels, c. 1993, 'information technology and organisational performance', information and management, no. 25, pp. 193-205. silk, d. j. 1990, 'managing is benefits for the 1990s', journal of information technology, no. 5, pp. 185 193. slevin, d. p., stieman, p. a. & boone, l., w. 1991, 'critical success factor analysis for information systems performance measurement and enhancement', information and management, no. 21, pp. 161 174. sohal, a. s. & ng, l. 1998, 'the role and impact of information technology in australian business', journal of information technology, no. 13, pp. 201-217. stacey, r. d. 1993, strategic management and organisational dynamics, pitman publishing, london. tam, k. y. 1992, 'capital budgeting in information systems development', information and management, no. 23, pp. 345 357. tam, k. y. 1998, 'the impact of information technology investments on firm performance and evaluation: evidence from newly industrialised economies', information systems research, vol. 9, no. 1, pp. 85-98. thatcher, m. e. & oliver, j. r. 2001, 'the impact of technology investments on a firm's production efficiency, product quality and productivity', journal of management information systems, vol. 18, no. 2, pp. 17 -45. trauth, e. m. 2001, 'the pursuit of information technology in context', in australasian conference on information systems, eds. finnie, g., cecez-kecmanovic, d. & lo, b., coff's habour, nsw, australia. trice, a. w. & treacy, m. e. 1986, 'utilisation as a dependent variable in mis research', in proceedings of the seventh international conference on information systems. tsw 2003, co-processing pty ltd, available: [http://www.copro.com.au/overview.html] (29 may 2003). ward, j., taylor, p. & bond, p. 1996, 'evaluation and realistion of is/it benefits: an empirical study of current practice', european journal of information systems, vol. 4, pp. 214 225. weill, p. 1992, 'the relationship between investment in information technology and firm performance: a study of the valve manufacturing sector', information systems research, vol. 3, no. 4, pp. 301-332. willcocks, l., feeny, d. & islei, g. 1997, managing it as a strategic resource, mcgrawhill international (uk) ltd, berkshire, england. willcocks, l. & lester, s. 1999, beyond the productivity paradox, john wiley and sons, chichester, uk. woccu 2001, 2001 statistical report, world council of credit unions inc. woccu 2003, woccu members, available: [http://www.woccu.org/about/memlist.htm] (29 may 2003). yin, r. k. 1994, case study research: design and methods, 2nd edn, sage publications inc, thousand oaks, california. microsoft word ajisfinal2003.doc australasian journal of information systems special issue 2003/2004 14 vernacularism in software design practice: does craftsmanship have a place in software engineering? paul r. taylor (ptaylor@csse.monash.edu.au) agentis international, level 2, 33 lincoln square south, carlton, 3053. abstract convention dictates that an information discipline matures from an informal shared practice to a defined and repeatable process through the externalisation and formal expression of its underlying theory. the inevitability of this progression should not be allowed to over-shadow the essential values, skills and knowledge transfer mechanisms of the superseded vernacular or craft-like practices. this paper examines the tension between software engineering’s professionalisation of the software design role—exemplified by the software architect—and its antithesis, the software craftsperson, a characterisation that continues to emerge despite attempts to suppress reliance on individual skills and abilities through software engineering process. in other design disciplines, the professionalisation of design marks a distinct progression from ad hoc, unrepeatable, unselfconscious craft to a selfconscious, demarcated type of design found in most forms of engineering. software engineering has partially failed to make this transition and this failure undermines the exclusivity of the engineering metaphor and engineering-based process models as a model of practical software design. software methods must acknowledge and find ways of incorporating vernacularism and informality if the creative act of design is to be correctly characterised, supported in methods, and taught. “traditional vernaculars are themselves the great mass media; that is, specialized frames and vehicles of experience” — marshall mcluhan. design and progress an historian who took the time to examine the chronicles of software engineering could be excused for concluding that ‘design’ has always had the status of second class object. in software engineering, design finds expression in the techniques of first-class things such as programming languages, architectures, modelling techniques and the mechanical modus operandi of software production. we comprehend design and exercise our design knowledge and skills indirectly via the domain-specific filters of common software design techniques—user interface prototyping, patterndriven architecture evolution, and use case analysis, for example. unlike other disciplines in which design also plays a central role—architecture and industrial design being the two exemplars— software engineering has not yet elevated design beyond its expression in techniques to professional specialty, to a communication medium for the explicit reflection of local or societal values, or as a fulcrum for social or economic change. questions about the relationships between individual practice, theory and collective professionalism are at the core of what it is we do as software designers. design in the software domain primarily serves the purpose of development and delivery, or of bringing product to market, and it must be presumed that the discipline, industry and community make this so. identifying possible reasons for this necessitates both a specialised understanding of the nature of software and its design as well as a broad exposure to design beyond any one particular domain. the relative immaturity of software design at the system and architectural levels repeatedly observed. shaw (1996), among others, claims that software architecture is an ad hoc, immature craft-like activity, applied only in an idiomatic fashion. ad hoc design activity is generally discredited in software engineering, but not so in design theory outside software (jencks & silver, 1973), where the ‘creative leap’ of synthesis, or the enigmatic assembly skills of the bricoleur can explain outstanding acts of design. these ‘designerly’ behaviours have interesting implications for a perspective of design in general (louridas, 1999), as well as software design (taylor, 2000a). the need to progress from craft to science is taken for granted in almost all applied sciences. this attitude may be seen to have its roots in the human struggle for ascendancy, the basis of science and the scientific method (weatherall, 1979). those who have questioned the need and appropriateness of this progression in design (such as christopher alexander in architecture) have at times been australasian journal of information systems special issue 2003/2004 15 branded brave, stupid or both (grabow, 1983). in the software discipline, the need to professionalise design has clearly been motivated by economics—software development processes prone to bottlenecks at key design points did not scale and were therefore not feasible in the emerging software application marketplace of the 1970s, and the force of these economic drivers has scaled several orders of magnitude since. whereas a craftsman designs and makes a one-off piece of furniture, jewellery or linen in isolation, the software designer’s artefact is more the scale of a house, bridge or office tower and the complexity of a thousand such domestic or decorative objects. the analogies are rich but can equally be specious (baragry & reed, 2001). the progression towards ever-increasing levels of design professionalism presents two important questions—is it the right progression in the case of software, and what are the implications of this progression for the ways we design software? the first question challenges the canon, even the status quo, with all of the attendant risks. an attempt at distilling the design consequences of an alternate metaphor, one that counters this notion of inevitable progress has been started (mcbreen 2002)(taylor, 2001). the remainder of this paper examines the second question—how has software design been shaped by the force of professionalisation through software’s pursuit of engineering rigour, and how has the performance of software design changed as a result? from vernacular to rational design design, both noun and verb, is so pervasive that it risks losing common meaning from its expression in so many objects and activities. essentially, it refers to the idea at the root of a work (hauffe, 1998). software design is pervasive, creative, and multi-dimensional (glass, 1999). it is widely regarded as a foundation of all systems development (booch, 1994). design problems of the class that are typically tackled by software projects have been referred to as ‘wicked’ problems (budgen, 1994, rittel & weber, 1984), meaning they have non-linear, unpredictable and unmanageable characteristics. a predictor of how the role of design in software creation has and may still change is the historical trajectory of design through the industrial revolution, which saw vernacular design practices replaced by the rational models and approaches that now underpin engineering and the architecture of the built world. self-conscious and unselfconscious design processes some observers detect this progression in the designer’s self-awareness. alexander (1964) introduced a broad distinction between two kinds of design processes, one he called ‘unselfconscious’, the other ‘self-conscious’. roughly speaking, the unselfconscious process is that which goes on in primitive societies or in the traditional craft or architectural vernacular contexts, while the self-conscious process is that which is typical of present-day, educated, specialised professional designers and architects. the way that the design and the production of useful objects is taught provides the best differentiator. in the unselfconscious case, the teaching of craft skills is through demonstration and imitation of skilled craftsman. thus the novice learns by practising the actual skill. in the ‘selfconscious’ process, the techniques are taught by being formulated explicitly and explained theoretically. in the unselfconscious culture, the same form is repeated over and over again, and the individual craftsperson must learn how to copy a prototype. but in self-conscious design there are always new contexts of use arising for which traditional solutions are inappropriate or inadequate. these necessitate some degree of theoretical understanding in order to be able to devise new forms to meet the new needs. in software creation, the unselfconscious designer—perhaps working in isolation to merge experience with skill, evolving the function, structure and aesthetics of a complex software architecture or component in piecemeal fashion—is a believable, if not suppressed image. a fundamental question is whether we as methodologists and design theorists believe that a model or paradigm should dictate, or reflect practice. a more manageable question is which model—or what overlay of models—best describes how software design is known to be performed. australasian journal of information systems special issue 2003/2004 16 the value of vernacularism the techniques, tricks, mores and means passed from building to building via builders are informal, causal and astute. design as a communication medium—or the ability of a building or artefact to harbour representations of its designer’s intentions beyond basic function—has long been recognised by architectural theorists (eco, 1997), and vernacular design does this with maximum efficiency. ‘vernacular’ is a term borrowed from linguists by architectural historians to mean ‘the language of a region’. vernacular buildings are those not designed by architects or professional designers. familiar examples include the cotswold villages, cape cod houses, and any other style that evolved locally under the collective influences of isolation, climate, locally available materials and cultural stability. in architectural terms, vernacular buildings are seen as the opposite of whatever is contemporary, high style, or polite. dormer (1988) writes that “the craft aesthetic is antitechnology, anti-science and anti-progress” (p. 135). it is little wonder that, in their clamour for engineering credibility, early software methodologists rejected vernacularism outright. vernacularism, however, has much to recommend it. vernacular buildings evolve extremely well. as each successive generation of design imitates its predecessor, the best buildings, techniques and uses of materials are copied while the less ideal are forgotten. according to brand’s (1994) analysis of how buildings ‘learn’ (that is, how they evolve to embody fit-for-purpose design over time), the heart of vernacular design is about form, not style. the evolutionary development of a vernacular style influences rooms and rooflines rather than colours or stylistic appliqué. the most striking feature of brand’s vernacularism is the effect of human use and habitation over time as the predominant selector of designs—“style is time’s fool; form is time’s student” (p. 133). others have written on theoretical aspects of the evolution of designs (hull, 1988, steadman, 1979, stebbins, 1971) and design knowledge (gero, 1996, kaplan, 2000). some related work focuses on evolution in software design (foote & opdyke, 1995, kemerer & slaughter, 1999, lieberherr & xiao, 1993, taylor, 2000b). vernacularism appears to be robust and irrepressible. it is wrong to assume that the professionalisation of design expunges all traces of vernacularism—rather, it exists to a degree in all design contexts. even as the most prominent and extreme modernists attempted to rid early twentieth century design of its human foibles by seeking a new kind of beauty in the ultimate elevation of function over form and aesthetics, a kind of ‘industrial vernacular’ was created (berman, 1988). the semiotic language of modernist design was sparse and minimalist but a distinct, readable language nonetheless, with touches of regional colour and recognisable personal idiosyncrasies (venturi et al., 1977). some software design theorists regard vernacular design as the only kind that has actually been seen to work. borenstein (1991), for example, thinks that anecdotes (programmer’s stories, word of mouth, narratives and case studies) convey more software design knowledge than any theory or method. a primary motivation of the software design patterns movement is to express common software designs that are known to have occurred in at least three different contexts (coplien, 1996). the so-called agile methods and extreme programming promote less formal approaches to design and knowledge transfer over ‘hard’ theoretical ones. using roles to illustrate the transition from vernacular to rational design if vernacularism cannot be eliminated in the pursuit of rationalism, it must exert its influence by degrees. a litmus test for the presence or absence of vernacularism in a design context is the relationships between stakeholders — the designer, the maker and the user. mayall (1979) views these relationships between the pre-industrial designer or craftsman and the user as a simple closed loop—the craftsman designs and makes for the patron directly. its vital characteristic for healthy australasian journal of information systems special issue 2003/2004 17 design is unhindered communication and a tightly bound, responsive feedback loop. this simple closed loop relationship still exists between designers and craftspersons and their patrons today. it also occurs between many architects and their clients, and consultants and their clients. it is so pervasive because it works so well. its simplicity leads to some obvious problems—it does not scale well, the customer may not know what to request or may interfere too closely with the designer’s work causing constant re-direction; or the customer may get too involved in the design process or otherwise not comprehend the represented complexity. walker and cross (1976) elaborate on the vernacular ‘mode of design’ in figure 1. these diagrams are read as follows: the rectangles with rounded corners represent roles, the product is shown as a circle, and dialogue is represented as a triangle. the exertion of pressures and needs (i.e. requirements) is represented by a jagged line superimposed over the thick arrow that represents the normal interactions during design and construction activity. in walker and cross’ vernacular design (a), user, client, designer and maker are one, and the craftsperson is unimpeded in her control of the object. this is a representation of classic craftsmanship, and if you substitute ‘object’ or ‘component’ for ‘product’, it is equally applicable to the software developer’s routine work of detailed design, coding, testing, and evolving a software artefact. if the object has no visibility to other developers or to the software system’s user, and is a consequence of the software developer’s particular design, the illusion is complete, and the cameo of the software craftsperson emerges. when the designer designs a product for someone else (or when the developer iterates the development of a component that provides user interface or services to a system user) the designer is still free to use vernacular solutions and techniques but must select, synthesise and arrange them to meet the negotiated requirements of the patron (b). design product predeliberations user client designer maker design product maker/designer (craftsman) discussion patron figure 1: two representations of vernacular design; (a) vernacular design; (b) the empirical exchange (walker & cross, 1976, p. 58). the separation of designing and making the craftsman’s direct experience of materials, patterns of use and clients is in sharp contrast with the isolation of post-industrial product designers from the manufacture, use and users of their designs. software methodologists may legitimately blame the inability of vernacular design to scale, or to be managed in contemporary business environments, as the primary reasons for its rejection. but in industrial design it was the emergence of new materials in standard sizes, forms and quantities that motivated the advent of designer as separate from manufacturer. australasian journal of information systems special issue 2003/2004 18 design designer client/user (patron) product maker brief figure 2: direct patronage (walker & cross, 1976, p. 57). in walker and cross’ depiction (figure 2) a new token appears—a rectangle that represents ‘the design’ in a form that the maker can interpret. the design is the designer’s interpretation and transformation of the brief from the client, user and/or patron. the fact that the patron and user interact only with the designer and the product, and never with the maker, reinforces the commodification of making in this (post-industrial) paradigm. the separation of designing and making opened the door for professionalisation, as typified by the architecture profession. now, one group laid claim to the high ground of the conceptual and aesthetic dimensions of design whilst delegating the detailed engineering and structural design to others. this division of responsibilities split concept from detail, image from functionality, and momentary perception from long-term occupation. the emergence of rational design processes sacrificed responsiveness and contextual relevance for uniformity and universality. each additional identification and separation of roles in time and space brought additional interfaces, and greater reliance on specification and process. the success of professionalised design in modern architecture and industrial manufacturing appeared as a glimmering chimera to those in other design domains looking for proven models to adopt. design (drawings/models) designerconstructor clientsusers brief (schedule/ specification) pressures (demands/needs) artefact (marketed product) figure 3: the rational network (walker & cross, 1976, p. 56). rational design the success of the separation of designing and making in industrial manufacturing should not be australasian journal of information systems special issue 2003/2004 19 interpreted as a universal justification for the elimination of vernacular design in software development—rationalism in design brings costs and consequences. mayall’s description of the emergence of rational design emphasises the loss of feedback loops. according to mayall, machinebased industrialisation caused the break-up of the designer/craftsperson into the four roles. the management–design–manufacturing split was a direct consequence of the growth of specialisation stimulated by the industrial revolution and larger, international markets that depended upon highvolume manufacture. the breakdown of communication between designer and user followed from the introduction of new types of product and manufacturing processes that could not be judged or assessed until they had been tried out for some time. feedback loops became strikingly elongated. walker’s depiction of rational design completes the picture that we are so familiar with today (figure 3). users are depicted as a separate stakeholder group from the designer’s clients. ultimately, users create the forces that motivate the design and construction processes, and users evaluate the results. the three major artefacts of the rational design process are the brief (requirements), the design (drawings and models), and the artefact or product itself. most contemporary software methodologies assume this model of the context and function of design, and these roles and relationships dictate the methodological framework by which software artefacts are produced (truex 2000). problems with the rational design paradigm historically, experience with the rational design paradigm suggests some consistent themes. these include the disconnection of design from manufacture, the disconnection of design from the context of use, the elongation of critical feedback loops, and the repression of ‘natural’ informal and vernacular design practices. a good place to start a critique of rational, post-industrial design is with the claim that rational design actually works on realistic problems—that it delivers workable solutions. jean claude garcias, writing on france’s changing cities (myerson, 1993) observed that the myth underlying the thinking of le corbusier and his colleagues was that of zoning—that by carefully re-ordering the city, by separating its functions, it would be possible to solve social ills. but functional decomposition, or master-planning as it is known outside of software, has not worked absolutely, and many observers now claim that modernist design principles found in many of the world’s modern cities left a legacy of structural and habitation problems (koestler 1998). the rational master-plan failed to recognise that the most vibrant, stimulating and functionally coherent cities are those in which the city’s structure allows for localised control and expression, and that these properties are emergent rather than consequential. this assertion has been made about other designed structures, including software structures (coplien, 2000a, foote & yoder, 1999). the rational designer must work hard to avoid over-commitment to a priori master-planning. in designing complex structures embedded in larger, non-deterministic social systems, a solution structure can only designed for the context as it is perceived. this reality of rational ‘snapshotdesign’ ameliorates the fiction of successful rational design for all eventualities within a given planning horizon. inevitably, in software and in other structured systems, today’s solution becomes tomorrow’s problem. the obvious antidote to one-shot design is evolutionary or emergent design process, but the rational paradigm is diametrically opposed to evolutionary design. during the analysis and design phases of a rational system development process, aspects of social complexity are modelled using a variety of tools, techniques and methods of abstraction. this methodological approach, often sold in the form of packaged product-methodologies and encased within a project culture, is inherited from science, drawing on formalism and notions of optimality. scientific positivism impels the rational methodologist toward universal approaches to a wide family of problem situations, reducing them to an abstract set of symbols (often diagrammatic), thereby allowing a series of semi-autonomous transformations to deliver a solution. at a philosophical level, this approach aims to assure that we develop credible knowledge about the present and possible future states of the world (lycett & paul, 1998)—it is as much about effecting risk management as it is about predictable, repeatable design. australasian journal of information systems special issue 2003/2004 20 the methodological approach to systems development (lycett argues) assumes social structures, mechanisms and processes as ‘invariant regularities’ that only have to be revealed to be understood. in the long term, methodological systems will—in most important cases—disappoint, as they do not allow internal variety to evolve in line with the environment. they are prone to represent temporal snapshots, ultimately leaving us with static systems that must operate in a dynamic world. lycett concludes that design should be an inherently ongoing process, not one where the designer is required to predict or control change and articulate every requirement of a design. continuous design was also recognised by design theorists outside of software, where a transition from a focus on product to process was advocated some time ago (thackara 1986) (jones, 1988, mitchell, 1988). in the software engineering literature, a growing band of authors have been questioning the exclusivity of the rational basis of the predominant metaphors and methods. blum (1996) systematically deconstructs the current software engineering paradigm using the failures of positivism and universal science to jump the gap to the social sciences, and the situatedness of observed contextual design as evidence, and argues for an adaptive paradigm of software design. truex (2000) reaches similar conclusions after applying his deconstruction of the ‘privileged text’ of is development methods. gabriel’s feyerabend project (2001) meets regularly to enact exercises and workshops to similarly deconstruct preconceptions of software construction. baragry (2001) has claimed that software must abandon its associations with inappropriate metaphors (architecture and engineering) in order to be able to define a more appropriate philosophical basis. many outside of software (feyerabend, 1993) have expressed similar views. those of us working in the theories and paradigms of software design are at risk of repeating design history through ignorance. as a reaction against industrialisation’s suffocation of design quality and craftsmanship, the arts and crafts movement originated in great britain and spread to america. artisans who worked in the arts and crafts style attempted to revive the medieval guild system, reestablishing high quality standards of workmanship, and instilling the idea of ‘truth to materials’ as the basis of design (naylor, 1990). recent writings in software methods witness a similar reaction. coplien (2000b) draws a direct parallel between the arts and crafts reaction and the replacement of large-scale industrial software processes (such as the capability maturity model) by developercentric lightweight methods. beck’s (2000) ‘extreme programming’ and scrum (beedle et al., 2000) are methods based on pair programming and time-boxing that attempt to relocate design responsibilities on all levels of abstraction back to the individual developer. these ‘agile’ methods attempt to empower the individual developer within a larger scale project management methodology by separating project management and risk aversion responsibilities from design responsibilities. this difficult balancing act illustrates the essence of the critical challenge facing software methodologists—to find ways of combining rational and vernacular design modes within contemporary software design contexts. in defence of the craft metaphor for software creation craft, as a metaphor, illustrates the process of making. it is concerned with the non-industrial acts of continuous design, creation, fabrication and evaluation. craftsmanship implies highly developed skill, deep knowledge of the fabric and material, and an intimate integration of the craftsman’s knowledge and expertise with the crafted object. craftsmanship holistically integrates designing, making and using, and equally intermeshes conceptual and detailed design with all aspects of fabrication. the performance of this kind of making involves transparently and continuously moving between modes of designing, making, evaluating and reflecting, being totally immersed in and consumed by the work, using tacit and explicit skills and knowledge, often in ways that result in knowledge becoming embedded within the artefact. the public looks to craft for perfection, mastery, and evidence of rare innate skill. clear criteria and a widely, almost universally held appreciation of what makes up good work define craft. in stonecraft of all kinds the craftsmanship is appreciated universally across time (today’s stonemasons learn from greek and roman ruins). the basis of this common understanding is the material itself— because stone, for instance, has never changed its basic building characteristics, examples of its use australasian journal of information systems special issue 2003/2004 21 for building are evident across the centuries. wherever the material has the capacity to act as a medium of communication that transcends language, culture and time, a craft exists. craftsmen collect around crafts-worthy materials—those that embody characteristics that reflect skill and attention to detail back to the worker and are malleable and pleasurable to work with. on the face of it, craft has much in common with the personal software development processes and practices used by experienced developers during intensive software development and evolution. how does the software engineering literature reflect upon this inherent informality? most interpretations are shaped by the positivistic roots of the applied sciences, and in any doctrine based upon functionalism the only alternative is to regard it as a shameful lack of control (hauffe 1998). clearly, pragmatic informality must be countered with the discipline of methodology. but the elimination of informality inherent in modernist architecture lead to a form of authoritarian ‘design control’ that is evident to a degree in most software methods. on the face of it, dormer’s earlier comment (that ‘the craft aesthetic is anti-technology, anti-science and anti-progress’) seems like an irreversible demolition of the metaphor as it might apply to software development. but if we parallel dormer’s crafts and industries with the state of software industry, it can be argued that the metaphor gains strength rather than losing it. software craft can only be called anti-technology, anti-science and anti-progress when the software industry approaches the level of determinism, automation, and maturity as the manufacturing industry has over the last century. by most comparisons, the software industry’s current ability to ‘manufacture’ software goods places it somewhere in the analogical vicinity of the manufacturing industry’s deployment of steam. in the light of this comparison, the craft ethic, aesthetic and metaphor in the software industry have relevance. professionalism—a third force on design practice figure 4 illustrates a view of the sources of the competing forces that this discussion has surveyed. those forces that shape software design practice that do not explicitly fit rational method or vernacular craft performance are represented as ‘professionalism’. the diagram of the left summarises current—albeit idealised—attitudes to rationalism, professionalism and vernacularism in software design thinking. this paper has argued that vernacular design is a profoundly natural and in many ways suitable perspective from which to think about design in the software fabric, but that the incumbent rational paradigm of software design has no place for vernacularism. addressing this oversight creates the intersection that appears on the right of figure 4, which raises important questions about which characteristics of software design belong in each of the set intersections, and why. the influence of professionalism is a force not to be underestimated in shaping how the rational and vernacular elements of design might be brought together. a shared sense of professionalism filters practices and legitimates some techniques over others. but the emergence of techniques within the context of a standard body of knowledge is different to the professionalisation of design. professionalisation selects models and modes of designing over time, and is subject to market pull and economic push. professionalism draws selectively from rationalism, using rational arguments to support behaviours that suit the individual or the enterprise. it groups a set of largely socially constructed phenomena. the intersection of rationalism and professionalism must be carefully scrutinised in every software design landscape to discern which rational methods and techniques are appropriate and contribute genuine value and which are there for other reasons. professionalism-meets-vernacularism is an under-recognised and under-rated juncture. this intersection is typified by the use of the professional or collegiate network, reflection, altruism, mentoring, and enculturation. here, designers seek out peers to source and validate potential solutions and experience for personal reasons. software engineering methodology has not acknowledged the value of this behaviour, and as a result, some informal, apparently ad hoc designer behaviours are unnecessarily under-valued. australasian journal of information systems special issue 2003/2004 22 rationalism-meets-vernacularism is a non sequitur in design theory and history. this intersection, to the degree that it exists at all, is best enumerated by countless accounts of the domination of rational design over vernacular craft. as noted, reactionary software design methods have begun to address this. idealised software design vernacularism professionalismrationalism vernacularism professionalismrationalism actual software design figure 4: representation of idealised and actual software design. the intersection of professionalism, rationalism and vernacularism represents the greatest challenge to the applied field of software creation. no one would sensibly claim that rationalism in contemporary software design is fatally flawed. however, as has been argued, many observers of industrialisation, both within and outside of software, perceive problems in a unilateral devotion to rational philosophy. as the discipline of software engineering has attempted to transform software craft into a fully-fledged member of the engineering family, the role of software designer (amongst several others) has evolved. careful observation of its changing features serves to illustrate the ever-present dangers of over-correction. the formalising forces of professionalisation of the craft of software development have had similar effect. of interest to software engineering professionals and methodologists is the degree to which these problems reflect on treatments of design in contemporary methodology and what, if any action should be taken to mitigate the associated risks. conclusion in summary, a number of objections to rational design have been raised. these include the disconnection of design from manufacture (a distinction that makes little sense in software construction); the disconnection of design from the context of use (a problem that has achieved more prominence in information systems literature than in software engineering); elongation of critical feedback loops, and repression of ‘natural’ informal and vernacular design practices. the problems that surface as a result of these mismatches stem from the underlying tension between the incumbent rational model of software design and other paradigms of thought and action that influence how practitioners approach software construction. this tension has the potential to retard progressive thought and innovative software design practice. marshall mcluhan’s insightful prediction from 1970 has been demonstrated repeatedly by the pioneer software creators at microsoft, sun, novell and countless others: “in terms of, say, a computer technology we are headed for cottage economics, where the most important industrial activities can be carried on in any individual little shack anywhere on the globe” — marshall mcluhan. suggestions for paths forward range from incremental methodological evolution (as much of the current software engineering research would seem to suggest) to full-scale kuhnian paradigm shift. either way, the environmental factors that will select the methodological variants that prosper in the next chapter of the history of software creation—the global economy’s insatiable demand for australasian journal of information systems special issue 2003/2004 23 software; the need for dramatic increases in quality, responsiveness and flexibility to name a few— will significantly test our current notions of rationalism, vernacularism and professionalism. references alexander, c. (1964) notes on the synthesis of form (new york, harvard university press). baragry, j. & reed, k. (2001) why we need a different view of software architectureworking ieee/ifip conference on software architecture (amsterdam, the netherlands, beck, k. (2000) embracing change: extreme programming explained (cambridge, cambridge university press). beedle, m., devos, m., sharon, y., schwaber, k. & sutherland, j. (2000) scrum: a pattern language for hyperproductive software development, in: n. harrison, b. foote & h. rohnert (eds) pattern languages of program design 4, vol. 4 (reading, massachusetts, addison-wesley). berman, m. (1988) the experience of modernity, in: j. thackara (ed) design after modernism (london, thames and hudson). blum, b. (1996) beyond programming: to a new era of design oxford university press). booch, g. (1994) object-oriented analysis and design with applications (redwood city, california, benjamin cummings). borenstein, n.s. (1991) programming as if people mattered: friendly programs, software engineering and other noble delusions (princeton, new jersey, princeton university press). brand, s. (1994) how buildings learn: what happens to them after they're built (new york, penguin). brooks, f.p. (1987) no silver bullet: essence and accidents of software engineering, ieee computer, 20(4), pp. 10-19. budgen, d. (1994) software design addison-wesley). coplien, j.o. (1996) software patterns(new york, lucent technologies, bell labs innovations). coplien, j.o. (2000a) architecture as metaphor coplien, j.o. (2000b) patterns and art, c++ report, 12(1), pp. 41-43. dormer, p. (1988) the ideal world of vermeer's little lacemaker, in: j. thackara (ed) design after modernism (london, thames and hudson). eco, u. (1997) function and sign: the semiotics of architecture, in: n. leach (ed) rethinking architecture: a reader in cultural theory (london, routledge). feyerabend, p. (1993) against method (london, verso). foote, b. & opdyke, w.f. (1995) lifecycle and refactoring patterns that support evolution and reuse, in: j.o. coplien & d.c. scmidt (eds) pattern languages of program design addison-wesley). foote, b. & yoder, j. (1999) big ball of mud, in: n. harrison (ed) pattern languages of program design 4, vol. 4 addison-wesley). gabriel, r.p. (2001) the feyerabend project (http://www.dreamsongs.com/feyerabendw1.html) gero, j.s. (1996) creativity, emergence and evolution in design: concepts and framework, knowledge based systems, 9(7), pp. 435-448. glass, r.l. (1999) on design, ieee software, mar/apr 1999, pp. 104-103. grabow, s. (1983) christopher alexander: the search for a new paradigm in architecture (chicago, university of chicago press). hauffe, t. (1998) design: a concise history (london, laurence king publishing). hull, d.l. (1988) science as a process: an evolutionary account of the social and conceptual development of science (chicago, the university of chicago press). jencks, c. & silver, n. (1973) adhocism: the case for improvisation (new york, anchor books). jones, j.c. (1988) softecnica, in: j. thackara (ed) design after modernism (london, thames and hudson). australasian journal of information systems special issue 2003/2004 24 kaplan, s.m. (2000) co-evolution in socio-technical systemscomputer supported cooperative work 2000 (philadelphia, acm). kemerer, c.f. & slaughter, s. (1999) an empirical approach to studying software evolution, ieee transactions on software engineering, 25(4), pp. 493-509. lieberherr, k.j. & xiao, c. (1993) object-oriented software evolution, ieee transactions on software engineering, 19(4), pp. 313-343. louridas, p. (1999) design as bricolage: anthropology meets design thinking, design studies, 20(6), pp. 517-535. lycett, m. & paul, r.j. (1998) information systems development: the challenge of evolutionary complexity in: w.r.j. baets (ed) sixth european conference on information systems (aix-en-provence, france, euro-arab management school, granada, spain.). mayall, w.h. (1979) principles in design (new york, van nostrand rienhold). mcbreen, p. (2002) software craftsmanship: the new imperative (addison wesley professional). mitchell, t. (1988) the product as illusion, in: j. thackara (ed) design after modernism (london, thames and hudson). myerson, j. (1993) design renaissance: selected papers from the international design congress in: j. myerson (ed) international design congress (glasgow, scotland, open eye publishing). naylor, g. (1990) the arts and craft movement: a study of its sources, ideals and influence on design theory (london, trefoil publications). rittel, h.j. & weber, m.m. (1984) planning problems are wicked problems, in: n. cross (ed) developments in design methodology wiley). shaw, m. & garlan, f. (1996) software architecture: perspectives on an emerging discipline steadman, p. (1979) the evolution of designs: biological analogy in architecture and the applied arts (cambridge, cambridge university press). stebbins, g.l. (1971) processes of organic evolution (englewood cliffs, new jersey, prenticehall, inc.). taylor, p. (2000a) adhocism in software architecture perspectives from design theory in: j.g.a.p. croll (ed) international conference of software methods and tools (2000) (wollongong, ieee computer press). taylor, p. (2000b) evolution of software design knowledge in: f. burnstein (ed) australian conference on knowledge management and intelligent decision support (2000) (melbourne, monash university). taylor, p.r. (2001) patterns of software craft in: j. noble & n. harrison (eds) second australian conference of pattern languages and programs (koalaplop 2001) (melbourne, university of wellington). thackara, j. (1988) beyond the object in design, in: j. thackara (ed) design after modernism (london, thames and hudson). venturi, r., scott brown, d. & izenour, s. (1977) learning from las vegas (cambridge, massachusetts, the mit press). walker, d. & cross, n. (1976) design: the man-made object the open university press). weatherall, m. (1979) scientific method (london, the english universities press ltd.). dormer, p. (1988). the ideal world of vermeer's little lacemaker. design after modernism. j. thackara. london, thames and hudson. hauffe, t. (1998). design: a concise history. london, laurence king publishing. koestler, h. (1998). home from nowhere. mcbreen, p. (2002). software craftsmanship: the new imperative. boston, addison-wesley. mcilroy, d. (1969). mass produced software components. first nato conference on software engineering. thackara, j. (1986). new british design. london, thames and hudson. truex, d. (2000). amethodical systems methodology. australasian journal of information systems special issue 2003/2004 25 venturi, r., d. scott brown, et al. (1977). learning from las vegas. cambridge, massachusetts, the mit press. microsoft word special final.doc ajis special issue december 2004 75 “help me pull that cursor” a collaborative interactive floor enhancing community interaction peter gall krogh institute of design aarhus school of architecture, denmark pkrogh@interactivespaces.net martin ludvigsen, institute of design aarhus school of architecture, denmark martinl@interactivespaces.net andreas lykke-olesen institute of design aarhus school of architecture, denmark alo@interactivespaces.net abstract in this paper we describe the development, experiments and evaluation of the ifloor, an interactive floor prototype installed at the local central municipality library. the primary purpose of the ifloor prototype is to support and stimulate community interaction between collocated people. the context of the library demands that any user can walk up and use the prototype without any devices or prior introduction. to achieve this, the ifloor proposes innovative interaction (modes/paradigms/patterns) for floor surfaces through the means of video tracking. browsing and selecting content is done in a collaborative process and mobile phones are used for posting messages onto the floor. the ifloor highlights topics on social issues of ubiquitous computing environments in public spaces, and provides an example of how to exploit human spatial movements, positions and arrangements in interaction with computers. keywords interactive floor, library, ubiquitous computing environments, spaces as interface, social computing, interaction design, designing for community interaction, video tracking. introduction / motivation much design and research effort within hci have been put into supporting distributed communities e.g. family members being away or living apart from one another (hutchinson 2003), geographically distributed fellow, or systems supporting the establishment of contact and formation of groups and communities between people not knowing one another on beforehand (nielsen 2002). there is a growing interest in how to support and encourage social interaction between collocated people and how the physical surroundings e.g. (wilde 2003) can be exploited in this regard. however, little work addresses how ubiquitous computing environments will go beyond spatially arranged devices as in e.g. (streitz 1999) and take into account and exploit spatial qualities of physical rooms, spaces and places. the work and the prototype presented here are a step towards realising the concept of interactive spaces. the user population addressed in the work presented here are the visitors of a central public library; people that do not necessarily know one another on beforehand, but might be interested in informal conversations and contact with other citizens. as one of the most profound institutions of a democratic society the public library primarily serves as a place giving every citizen unrestricted access to catalogued information and providing facilities for self-initiated life-long learning. but in doing so it also indirectly serves as a social space enabling awareness of fellow citizens and the pluralism that is equally important in maintaining a vivid democratic society. ajis special issue december 2004 76 in the research project denoted “the future hybrid library”, which is an initiative in the multidisciplinary research centre for “interactive spaces” mixing competences within architecture, engineering and computer science, we’ve been exploring how ubiquitous computing environments that take into account specific spatial qualities could support and enhance the social aspects of the library. as we would like to have as many potential users as possible to be exposed to the developed system the entrance hall of the local central public library was picked as framework for the design of the system as well as location for a later three week period of evaluation. on the basis of the spatial qualities and functions of the room we chose to explore this by means of an interactive display on the floor. background (and challenges) in his book, e-topia (mitchell 2000), william j mitchell starts out with a tale of the well and its importance in cities in the early days of civilisation. beyond being water reserve the well also had a strong social attraction, as people would go there just to meet up, exchange goods, arrange marriages etc. to enable the cities to physically expand and to minimize the risk of diseases, water was put into pipes suffusing the whole city. having direct access to water in each household made the practical role of the well obsolete, whereas the social needs remained the same. in response to these new social places e.g. cafes emerged. by means of the tale mitchell asks the polemic question: “what will happen when information by means of broadband access to the internet is “piped” into our households?” will this be the death of e.g. the library? which services and installations in the physical library will support the institution in maintaining its attractiveness and social role? in recent years libraries throughout the world have focused on delivering web-based services for its users. along with the web-based services there has been a growing focus in the physical library on meeting the needs of the individual user, and her ability to find relevant material. activities and approaches as these have resulted in a starvation of the attractions of the physical library beyond erecting evermore-impressive buildings and architectural statements of the importance of the physical library. through user studies, interviews with librarians and statistics on numbers of visitors (magistratens 4. afdeling, 2004, only in danish) at the local municipality library, we’ve learned that the by-product of these activities among others has been a decreasing number of visitors at the physical library. many needs of the user in regard of finding relevant material can be met through advanced search and reservation facilities on libraries web sites. in addition to this the general facilities on the internet: search engines, user groups, etc. also seemingly meet the users’ needs for finding information. furthermore, the social space in the physical library has taken a swing towards being more focused on the individual. independently, some users and librarians in our user studies even talked about the local library as having developed a supermarket-like atmosphere where people are indifferent to the whereabouts of others. probably this can be traced back to the increased focus of the library on serving the individual user supporting individual learning rather than providing a space for social gathering, activities and public awareness. to summarize, the experienced social value of the public library shows tendencies to decrease. visitors and librarians are becoming more and more focused on serving individual needs, efforts and tasks that obscure the view and attention to support and develop the social role of the library. to regain and strengthen the democratic role of the library it is necessary to devote more effort into the social aspects and activities of the library. new architectural monuments of the library and enhanced ways of efficiently organising the collection of materials are not enough to ensure the continued community-integrating role of the library. the design of computer systems also has to address the social challenges that the library is expected to meet. ajis special issue december 2004 77 related work in order to address the challenges mentioned above the work presented here draws upon work in three areas: architectural design, ubiquitous computing and designing for community interaction. furthermore, the interrelations of the three areas are explored in order to pursue the realisation of interactive spaces. from a perspective of architectural design, the design of physical spaces, most of the work within ubiquitous computing has either worked on furniture-sized installations (grønbæk 2001) following the concept of roomware (streitz 1999) or generally applying the scheme of “tabs, pads and boards” initially described in (weiser 1999). the only architectural element in these prototype environments that aims to transgress the affordances of furniture to become actual room-sized elements and interfaces are walls embedded with display facilities e.g. (johanson 2002). the vertical orientation of large displays positioned to be touch by hand, makes it fairly easy to adopt many of the wellestablished ideals within hci e.g. direct manipulation (schniederman 1987) whereas interfaces that are hard to touch directly by hand e.g. ceilings and floors are less explored as interactive surfaces. interactive floor surfaces are typically experimented on in dance and performances like set-ups e.g. the prototype magic carpet (paradiso 1997) and litefoot (fernstrom 1998). the prototypes are sensor intensive environments for the tracking of people’s movements of feet and in the case of the magic carpet the sensor floor has been supplemented with sensor technologies for tracking the movements of the upper body and arms. to serve different shaping and sizes of an interactive floor the z-tiles concept (mcelliot, et al. 2002) was developed. as the above-mentioned systems the ztiles interactive floor is based on sensor technologies. input from the interaction technologies is used to control and manipulate sound providing the idea of playing an instrument with your body movements. another system exploring multi-user spatial interaction by means of an interactive sensor-based floor is the virtual space project (leikas et al. 2001). the sensor technologies are in this case used to enable spatial interaction and control of a computer game projected on a vertical positioned display. another approach to enable spatial interaction is the use of video tracking e.g. the commercially available eyetoy game. http://au.playstation.com/ps2/hardware/eyetoy.jhtml. most of the work with designing for community interaction uses computer interfaces to mediate distributed activities enabling awareness of distributed people’s interests and whereabouts e.g. (büscher et al.). increasing effort has been directed towards enabling community interaction among collocated users (churchil 2004). as computers and computer interfaces for community interaction become an increasingly influential part of our everyday lives (grinter 2002) we see a need for addressing issues related to the experience of the social qualities related to the experience of collocated users. these concerns are intensified as computer interfaces transgress traditional interfaces and become embedded in the physical surroundings and spaces enabling groups of collocated people (huang 2004) to simultaneously interact with the systems accessible or as experienced in awareness facilities in office spaces. from our point of view this body of work lacks to take the spatial qualities of the physical environment sufficiently into account in regard to of how systems might be operated and appropriated in use. the new seattle central library (koolhaas 2004) is, a part from being an impressive architectural statement of the library as institution, also a refreshing example of how the need for social spaces and their nature can have just as strong impact on the design of the building as ensuring efficient organisation of the collection of media typically has when designing libraries. though the use of computer systems and their physical materialisation in the new seattle central library is fairly traditional with huge amounts of personal computers, floor spaces as “the living room” on 3rd floor and “the mixing chamber” on 5th floor, indicate an increased awareness of the social role of the library. but how would these floor spaces have materialised architecturally if they from the early design stages were thought of as ubiquitous computing environments, encouraging social interaction. ajis special issue december 2004 78 we see a need for including spatial concerns in the design of systems and interfaces for community interaction that goes beyond the mediation of distributed activities to include the experience of the collocated user. furthermore, there is an unexplored potential in including more playful aesthetic ways for interacting with these systems. from our point of view experiments taking on this challenge could be informed by approaches to aesthetic interaction as promoted by (petersen et al. 2004). design rationale and aims in the following we will present our rationales and aims of the prototype design. the development of this prototype has two basic aims. the first is to facilitate a space for communication and collaboration, based on the exploitation of user knowledge and curiosity. the system should not announce itself as a community supporting system. it should merely be a trigger for collocated people to start talking and engaging with one another. the second aim is experimenting with collaboration on interactive floor surfaces with no need of special input devices apart from being present in the physical library. this is done in order to design an interactive system that can be appropriated by most of the broad range of people visiting the public library every day. these users include all ages, genders, races and professions e.g. children, disabled, students, mothers with bags full of books etc. for more advanced interactions we will experiment with mobile phones as 90% of all danish families have minimum one mobile phone (statistics denmark 2004) the prototype is to be located in the entry area of the local central library, a transit space that is approached by the users from all directions. to keep the spatial qualities of this transit area it is important to ensure visual overview of the adjacent spaces. in addition to this public spaces are characterized by people with many different doings and different attitudes towards disruptions in the public realm (gehl 1987). to comply with this we will experiment with floor interaction that in many aspects will not alter the physical space. designing for public space requires certain considerations regarding the robustness of the system, which should result in an interactive system without any direct contact with the hardware. by introducing the prototype we want to inspire to the change of the communication style of the library. today most library users have interpersonal interactions with only the librarians, asking them for advice or references to books. the local central library is fairly visionary with respect to introducing technologies to make the daily handling of books and other materials more efficient. thus they have introduced self-service check inand out of materials, and recently also self-service check-out of reserved items. this means that the interactions between users and librarians have been minimized, and the library is decreasing its need for interpersonal interaction. in the design process library staff and mangers expressed the need to counter this trend by creating facilities and a space for informal exchange of knowledge among users, and encourage the transformation of the library into a more vivid social space. addressing this became one of the key design issues of the developed prototype. the prototype on the basis of the above identified challenges and needs a prototype was developed: an interactive floor for communication between users of the library named ifloor. we will describe the ifloor by looking at separate elements of the design: the general setup, the technology, the graphic user interface, bodily interaction and interaction by adding content. basically the ifloor is an interactive floor that affords users individually or in collaboration to browse and discuss projected questions and answers produced by the users themselves, through the movement and position of their body. by the use of a mobile phone or an email client questions and answers can be posted on the floor. ajis special issue december 2004 79 figure 1: the ifloor in use. several users discussing and interacting with the floor and the man on the opposite side of the floor is writing a message to the ifloor using a mobile phone technical setup the system consists of a remote server for receiving and handling sms’ and emails and administering questions and answers. furthermore, a projector mounted on the ceiling is connected to a local computer for the display on the floor. due to the requirement for system robustness, tracking technologies like e.g. interactive tiles (e.g. richardson) were rejected because this would require the installation of a technology-intensive floor that would be vulnerable in the public space. instead the floor interaction works on the basis of a video tracking system software (valli 2004) analysing the rim of the interface based on a video feed from a web-cam mounted on the ceiling. the tracking of people’s position and movement are sent from retina to macromedia flash and translated into magnetic forces attracting the cursor. this solution has the advantage that all fragile parts are mounted on the ceiling and thereby removed from direct access. using projections in bright daylight caused problems regarding the visibility of the display and the tracking. this was solved by using a powerful projector to project the graphic interface onto thin white pvc boards on the floor creating a clear projection image and giving a good sufficient contrast in the video feed for the blob analysis and threshold tracking done in retina. bodily interaction and the interface due to the novel interaction explored in the prototype and the unprecedented facilities offered by the prototype the graphic user interface had to be very simple and clear. technical tests showed that the precision in tracking we got from a simple web cam was enough to keep track of at least 15 people at one time in a 5 meters by 4 meters rectangle. diminishing the possibility of users entering the projection and thereby casting shadows on the content we only used data from the tracked persons in a one meter band surrounding the display. by using a visual feedback in form of a projected string connecting the cursor and the user while being in the tracked ajis special issue december 2004 80 area, people could see that they were taking part in the interaction. as soon as they entered the projection on the floor the string would disappear and they would have no influence on the system. we used the coordinates from the tracked persons to calculate the movement of a graphical element representing a shared cursor. the cursor had its initial position in the centre of the floor but as soon as the camera detected a person in the tracked area, the string would connect user and cursor and the cursor would start moving towards the user. stepping out of the tracked area would instantly make the cursor move towards the centre. in this way the system got very responsive supporting various playful ways of interacting with the cursor both as an individual and in collaboration e. g. by spreading out hands and feet one can obtain up to five strings giving five times the power of a user standing straight. figure 2: the unfolding of a question. a tool-tip provides the relevant instructions for replying. the green answer-box is an approved answer from a librarian. the maximum number of questions on the floor was set to 15 due to screen space and text readability. each question carried a time stamp that was used to exchange new questions with the oldest time stamp on the floor. we designed colour coded graphic elements as containers for the questions and answers. the fact that the floor was approached from all directions challenged the common layout of interfaces traditionally approached from one direction. to comply to that the questions were arranged in a circular array around the centre of the floor to provide equal access and readability of the display from all directions along the rim. intentionally, this encouraged users to walk around the floor in order to read all questions or ask others about questions on the other side of the floor. in experimenting with an alternative to double click on the floor we used the event where the cursor entered a question to trigger an animation unfolding the question and revealing its specific id number. if the question had been responded to earlier, these would fold out from behind the question enabling the user to read up to five different suggestions. the questions could be read at all times but users had to move and negotiate the cursor around to read the answers related to the questions. this was deliberately chosen to encourage users to communicate and negotiate on the movement of the cursor. by designing the cursor to slow down when entering a question we gave users time to read the question and related answers and get the id number. when leaving the unfolded question the cursor speed raised and an animation contracting the question and answers was triggered. during the evaluation period in the library we made ongoing improvements and changes to respond to interviews and user observations. the changes were made both to respond to expressed user needs as well as to try out new features. two changes are mentioned in particular here. in order to make the prototype more self-explanatory tool tips-like would evolve from the questions when the cursor would enter a question. the tool tips described exactly in three bullets how to answer the specific question either by mobile text messaging (sms) or e-mail. another change was made to encourage the librarians to take ownership of the prototype and to give their answers more integrity by developing an approved answer – colour coded in green that could be posted through the same web interface that was used to clean up the floor for offensive questions and answers. ajis special issue december 2004 81 to experiment with different ways of having users interact with the system we tried making the cursor respond to movement instead of position. this resulted in a dance-like performance to get the attention of the cursor but making precise interaction very difficult. finally, we tested the implementation of cursor-speed adjusting facilities mediated by icons. by adding small symbols in the corners of the display representing two speed up and two speed down facilities, users could move the cursor to an icon and through feedback from a counter see the change in cursor speed. this was tried out to add a higher degree of interaction but the feature also had a tendency to create chaos on the floor by unfolding all questions in seconds because of the speed. adding content to compose new questions users were requested to use either an email client or their mobile phone as input device. by using mobile phones and sms we could enable people to interact with the system without any specially designed gadgets supported by the library or the prototype itself. questions could be posed by sending a sms containing a question on max 120 characters to the ifloor phone number. about ten seconds after sending the question the user would get feedback in form of a sms saying “thank you for your question, you will receive answers as soon as they arrive, regards ifloor”. after that the oldest question and related answers disappear from the graphic interface making room for the new question to emerge on the floor during an animation clearly showing that new content has entered. responding to a question is done by entering the id number revealed from an unfolded question in a sms or e-mail followed by the answer. this creates a new answer that is nested to the question. in order to give visual feedback to users around the floor on changes in content and to promote curiosity in exploring the floor animations were implemented. figure 3: as the cursor enters the questions they unfold and reveal the id-number and related answers. too high cursor speed will activate neighbouring questions and responses creating a confusing overlapping pool of messages. evaluation the evaluation with users in the library context was conducted with emphasis on qualitative interviews and observations of users. in our observations we focused on relevance of the prototype and on learning what users experienced when interacting with the system, on the expense of statistics. during the three weeks of evaluating the prototype, we had several users experimenting with the installation. sometimes we stayed in the background to leave the users to figure out the interaction by themselves, and at other occasions we introduced the prototype to the users in order to engage in dialogue about how the installation was perceived. this way we had both informal talks with users and formal interviews conducted after users had worked or played with the prototype. in the course of the three weeks we also made observational videos of the space where the prototype was installed in order to see how people reacted to it, and how it affected people moving through the space. ajis special issue december 2004 82 another source in the evaluation was the feedback from the library staff on how they experienced the support of users interacting with the floor and their reflections on how it contributed to the traditional library context. along with the feedback from the user interviews, the librarians’ feedback was the starting point of redesigning and altering aspects of the floor during the three weeks and in order to enhance a walk-up-and-use experience and make the prototype more self-explanatory, as explained in the prototype section. as a part of the qualitative evaluation we also invited a 7th grade school class for a one hour quiz game. we posed the questions and they were supposed to find the answers by using the traditional search facilities provided by the library. with 25 pupils in the class it was a good way of seeing how the system would react to a maximum load of input. the experiences from this test emphasized the fact that even though the technology in theory could support an unlimited number of participants, the social and physical space set a certain limits to that number. the negotiation part of interacting with the prototype and trying to figure out how the tracking worked could not surpass the competitive element of the quiz game, so the prototype was often blocked by too many inputs. if the video tracking software can see users diagonally across from each other or, as in this case, all the way around the periphery of the floor, the cursor simply stops in a dead zone of equally balanced pull. the kids in the class were especially fast at adopting the system, and quickly used the floor beyond its intended use, as they started to post teasing remarks and other messages to the school mates. this way the floor expanded into a different kind of communication surface of graffiti-like tagging and personal exhibition. reflection the ifloor challenges several interaction paradigms (svanæs 2000) as they are difficult to translate these into interacting with a floor. in the three week period of evaluation and redesign of the prototype at the local central library we learned that the questions posed typically would be in the category of seeking advice or tips and tricks on every day things as, how to un-lock ones mobile phone, or where to find the best and cheapest printer. it came to work at bit like the notice board found in local supermarkets; however, in this case the communication was not about runaway cats and baby carriages for sale but knowledge exchange between users at the library. furthermore, as the floor was residing in the entrance and exit part of the library it came to work like a “sleeping policeman”, causing people to stop and to start chatting about the questions and answers on display, and taking the opportunity to participate in the process by posing new questions and responding to the ones in the display. furthermore, the playful way of navigating the interface was found very intriguing by the visitors and definitely also facilitated the process of making people talk to one another. figure 4: the ifloor in the library. after having discovered the interaction mode and the idea of the system, most people are enthusiastic about the technology and idea of use. when asked, most users were capable of thinking of many other places where a physical/digital bulletin board-like functionality could be helpful or ajis special issue december 2004 83 interesting. e.g. a group of student teachers had an idea for using it as a means of teaching because of its synergetic effect between play and serious content. spatial interaction proximity using the body as the means for interaction is to many people a very provocative and challenging idea. it seems the single-user-in-front-of-a-desktop paradigm has been thoroughly accepted by the general public. through the ifloor prototype we are experimenting with new principles for interacting with augmented floor surfaces. walls and all kinds of furniture (e.g. dunne 2001) have been used as interactive artefacts or displays, but using the floor surface as a collaborative multi-user surface sets design challenges as to how to point click and select content on the floor. being out of reach in most situations, the floor is not a surface for direct manipulation (schneiderman 1987). on the other hand, a potential input that is always available is if a user is able to view the floor he will be in the physical vicinity of it, so by using video tracking we can relate the floor to the user and afford interaction. this physical proximity is translated and used to orient the interface to the user. the question boxes are rotated in relation to the centre of the floor to ensure readability no matter from where the user will approach the display. at the same time this rotation encourages the user to walk around the floor to read the other questions, thereby discovering that the cursor is attached to her by a string and responding in real-time to movements. this sets of an exploration of the interaction and in several cases we saw users in the library trying out different ways of enhancing their ability to attract the cursor by e.g. joining other users or spreading out arms and legs. from private to public displays sending questions and answers to the ifloor are done by sms or via emails. this causes an interesting flow from using a public display in collaboration to using your private mobile phone or walking away to a computer, sometimes while still discussing the answer with a friend around the floor. the mobile phone becomes a remote control or a wireless keyboard for the shared display on the floor, causing a mix of private and public space. utilizing the mobile phone in the ifloor concept also means that the library extends its sphere of influence into the city, as you can pose questions or receive answers anywhere. the knowledge sharing that we try to establish in the library goes beyond the physical constraints of the library. however, the core of the community is still the collocated people at the library, since it is only the people present round the floor that can read and get access to the id-numbers and thereby answer the question. social issues social navigation during the first days of setting up in the library, we walked around the floor to ensure it was running properly. this attracted many users who interestedly started using the system. but when we stepped away from the floor the interest decreased. not for the individual users currently engaged in interaction but as a general trend for people arriving at the library. the floor, we decided, did not convey its potential use strongly enough. it looked attractive and interesting at first glance but potential users rapidly lost interest as they were unable to figure out how to use it and what the point of the interaction was. we changed the set-up by placing a3 sized posters on the three sides of the ajis special issue december 2004 84 floor where people would approach it. thereby we allowed an opportunity to observe and learn what the system was about before stepping onto the floor and interacting. even with this stepping stone approach, users were still hesitative to jump onto the floor and we sometimes had to jumpstart interaction simply by being present on the floor conveying a use pattern and the fact that the system was safe (and even fun) to use. when on its own, it seemed to us, the prototype was too unfamiliar for most people and the rate of how many walked up and experimented with the floor more than halved. as described in (höök et al. 2003) it is very important for a physical place as well as for a digital to convey it use context through the social interaction taking place. when designing a place that is both physical and digital and unfamiliar to the potential users, it is important somehow to provide the users with a possibility to gradually approach the system in a socially safe way. from observing how many people are using the floor and especially what they are doing, one can decide whether or not to participate. as a general rule when people are having fun or are deeply engaged in interacting it seems far more interesting for outsiders to try for themselves. negotiation and collaboration when users were experimenting with the prototype, we often saw problems with sharing the cursor. in most cases when the users did not know each other, the negotiation that had to take place in order to move the cursor to the desired location was limited. if a user finds a question interesting he will have to ask the other users around the floor to work with him to move the cursor to the question or tell them to get off the floor, which happened almost as often as the first approach. contrary to this, users were quite helpful explaining how to interact with the system to other users approaching the floor. in one case an elderly man had experimented with the floor and read the poster beside it, and was able to introduce the conceptual idea and use of the floor to a family of three. after this introduction they explored both what could be done with the video tracking and when using the sms-service. the single-user perspective is so deeply imbedded into users when it comes to using digital technology, that it seems surprising that a display and a cursor is addressing several users in 360 degrees. when approaching the ifloor a user discovers her influence on the system as a string is “attached” to her and follows her around as she walks round the floor to read the questions. when entering the floor, users will inadvertently disturb anyone else who might be doing something purposeful with the cursor. in this way the interaction with ifloor invites or even demands users to collaborate in order to control the interface. by setting up such “forced” collaboration, we wanted to create a hidden opportunity for library users to establish contact with each other, however transitory these contact might be. finally, we observed several users, who did not know each other beforehand, discussing possible answers to questions. as with a dinner table, it is seemed socially acceptable to be talking with the people you are sharing a point of attention with, and collaboratively solve the problem. play as users become more familiar with the interaction and figure out how the system is controlled, they can use the ifloor in a more playful manner. the ifloor can be manipulated to give more pull to a single user if he or she discovers that spreading out arms and legs will give more pull, as the video tracker sees more blobs and therefore “attracts” more strings to the cursor. this can give two or three users an opportunity to play with the floor and compete on who has the power to control the cursor. this game was often initiated when users were walking around the floor and noticed e.g. that they had strings “attached” to both their feet. the game became almost sport-like as it is your agility and stamina that will decide who can attract the most strings and win the cursor over. often these games appeared almost like dance performances as users really got captivated and expressive in their effort ajis special issue december 2004 85 to attract the cursor. making teams could also be a way of competing. sometimes users would also insist on the efficiency of something that did not have any effect on the system, like stomping or dragging the feet backwards as if the strings were physically attached to them. this, of course, makes sense in a direct translating of a physical string’s affordances, and this was often how users experimentally appropriated the prototype. future work in continuation of the “future hybrid library” project a related project has been launched called “the children’s interactive library”. in this work we will advance the exploration of spatial interaction paradigms in relation to the social spheres in the children’s library. in further development in the library context we will address the following issues for improving the ifloor prototype: • visual aging of questions to create a fast overview of new and old questions. • the relation to the physical surroundings e.g. using the floor for directing users to bookshelves related to their questions. • explore the use of the third dimension down in floor. • using gestures to improve the bodily interaction. as the technical solution proved to be fairly robust we believe that similar installations could be applied in other domains of the public space with a projectable floor material and the proper light conditions to do the video tracking e.g. as an interactive informative city map in transportation transit spaces supporting newcomers to get the latest information on events and happenings. conclusion the ifloor prototype, as we have presented in this paper, experiments with new interaction modes for using floors as collaborative display and interaction surfaces. novel interaction was achieved through the use of video tracking to extract the positions of the users around the ifloor, and thereby placing a shared cursor to navigate the posted messages. questions and responses were posted onto the floor by using a mobile phone as remote keyboards. in order to support community and informal interpersonal interactions in the library the ifloor encourages users to collaborate and negotiate when interacting with the cursor and browsing questions. the installation of the ifloor contributed to the library’s desire to change the library into a more social and communicative space. through user studies we evaluated and improved the prototype and found that in addition to fulfilling the design aims from the library, the ifloor also supported users’ curiosity through playful and spatial interaction. references büscher,m., kramp,g, & krogh,p (2003) in formation: support for flexibility, mobility, collaboration, and coherence. first international conference on appliance design, bristol, uk, 6-8 may 2003 churchill, e., girgensohn, a., nelsen, l., lee, a.: blending digital and physical spaces for ubiquitous community participation. in communications of the acm february 2004/vol. 47, no. 2 dunne, a. & fiona r., design noir – the secret life of electronic objects, august / birkhäuser, 2001 fernström, m., griffith, n: litefoot – auditory display of footwork. proceeding of icad’98, glasgow, scotland (1998) ajis special issue december 2004 86 gehl, j., life between buildings using public space, van nostrand reinhold, new york, 1987 grinter, r. e., palen, l., i m everywhere: instant messaging in teen life, proceedings of the 2002 acm conference on computer supported cooperative work, november 2002 grønbæk, k., gundersen, k. k., mogensen, p., & ørbæk, p. interactive room support for complex and distributed design projects. in proceedings of interact 2001, tokyo japan, july 2001. huang, e., russel, d.m., sue, a.e: im here instant messaging on large, shared displays for workgroup interactions. in proceedings of chi2004, vienna. pp 279 – 286 hutchinson, mackay, westerlund, bederson, druin, plaisant, beaudouin-lafon, conversy, evans, hansen, roussel, eiderbäck, lindquist, sundblad. technology probes: inspiring design for and with families, in the acm chi 2003 proceedings (2003) höök, k., benyon, d., munro, a.j., designing information spaces: the social navigation approach, springer-verlag london, uk, 2003 johanson, b., fox, a., winograd, t.: the interactive workspaces project: experiences with ubiquitous computing rooms. ieee pervasive computing magazine 1(2), april-june 2002. koolhaas, r., content, taschen verlag, köln 2004 isbn 3822830704, krogh, p., grønbæk, k.: (2001) architecture and pervasive computing – when buildings and design artefacts become computer interfaces, nordic journal of architectural research vol. 14 no. 3 2001, århus dk, pp. 11 – 22 magistratens 4. afdeling, 2004 forandringer og transformationer : oplæg til debat om indhold og struktur i de fysiske biblioteker i århus i fremtiden : bibliotekspolitisk redegørelse. århus : århus kommune leikas, j., väätänen, a. & räty, v. 2001: virtual space computer games with a floor sensor control : human centred approach in the design process. in: brewster, stephen & murray-smith, roderick (eds.) haptic human-computer interaction : first international workshop, glasgow, uk, august 31 september 1, 2000, proceedings. (lecture notes in computer science; vol. 2058) berlin: springer-verlag. pp. 199-204. mcelliot, l., dillon, m., leydon, k., richardson, b., fernstrom, m., paradiso, j.: forse fields – force fields for interactive environments. in proceedings of ubiquitous computing 2002: 4th international conference göteborg sweden pp. 168 – 175 nielsen, r. collaborative spaces: inhabiting virtual 3d worlds, 2002, in: virtual space: the spatiality of virtual inhabited 3d worlds. ed. lars qvortrup. paradiso, j., abler, c., hsiao, k., reynolds, m: the magic carpet physical sensing for immersive environments. proceedings of chi’ 97, atlanta, ga, usa (1997) petersen, m., iversen, o. s., krogh, p., ludvigsen, m., aesthetic interaction a pragmatist’s aesthetics of interactive systems, proceedings dis 2004 p.270-278, boston, ma., usa richardson, b., leydon, k., fernstrom, m., paradiso, j.a.: z-tiles: building blocks for modular, pressure-sensing floorspaces april 2004 extended abstracts of the 2004 conference on human factors and computing systems schneiderman, b.: designing the user interface: strategies for effective human-computer interaction addison-wesley publishers, reading, ma) (1987), statistics denmark, (in danish), durable consumer goods 2004, nyt fra danmarks statistik, nr. 183, 2004 streitz, n.a., et al. roomware for cooperative buildings: integrated design of architectural spaces and information spaces. in cobuild '98, cooperative buildings integrating information, organization, and architecture. 1998. darmstadt, germany.: springer: heidelberg. streitz, n.a., geißler, j., holmer, t., konomi, s., müller-tomfelde, c., reischl, w., rexroth, p., seitz, p., steinmetz, r. : i-land: an interactive landscape for creativity and innovation. in: ajis special issue december 2004 87 acm conference on human factors in computing systems (chi'99), pittsburgh, pennsylvania, usa, may 15-20, 1999. pp. 120-127. acm press, new york svanæs, d. understanding interactivity: steps to a phenomenology of human-computer interaction, monograph, computer science department, isbn 82-471-5201-0, ntnu, trondheim, 2000. valli, a., retina video tracking software available at http://alessandrovalli.com/retina/ (2004-0618) weiser m., seely brown j.; designing calm technology. mark weiser and john seely brown. the 100 show: the eighteenth annual of the american center for design. edited by therese rutowski. new york, ny: watson-guptill publications, 1996; pp. 159-163. weiser, m., "some computer science problems in ubiquitous computing," communications of the acm, july 1993. weiser, m., the computer for the 21st century, acm sigmobile mobile computing and communications review, volume 3 , issue 3 (july 1999), special issue dedicated to mark weiser, p. 3 11, 1999 wilde d, harris e, rogers y, randell c, ( 2003 ) the periscope: supporting a computer enhanced field trip for children. 1ad (first international conference on appliance design), may 6-8th, 2003, hp labs, bristol. also to appear in personal and ubiquitous computing journal acknowledgements we would like to thank colleagues in the centre for interactive spaces for comments on the prototype and in particular kaspar rosengren nielsen for programming the interface, niels olof bouvin for programming and administering the server part. we would also like to thank kaj grønbæk for useful comments on the paper. copyright [peter gall krogh, martin ludvigsen and andreas lykke-olesen] © 2004. the authors assign to ozchi and educational and non-profit institutions a non-exclusive licence to use this document for personal use and in courses of instruction provided that the article is used in full and this copyright statement is reproduced. the authors also grant a non-exclusive licence to ozchi to publish this document in full in the conference papers and proceedings. those documents may be published on the world wide web, cd-rom, in printed form, and on mirror sites on the world wide web. any other usage is prohibited without the express permission of the authors. microsoft word ajisvol10 1_final_edit.doc ajis vol. 10 no. 1 september 2002 99 small firm internet adoption: a market oriented approach colin jones dr rob hecker dr peter holland university of tasmania university of tasmania monash university school of management school of management department of management po box 152-16 po box 152-16 faculty of business and sandy bay sandy bay economics tasmania 7005 tasmania 7005 wellington road australia australia clayton phone: +61 (0) 3 6226 2826 phone: +61 (0) 3 6226 1174 victoria 3800 fax: +61 (0) 3 6226 2808 fax: +61 (0) 3 6226 2808 australia colin.jones@utas.edu.au rob.hecker@utas.edu.au phone: +61 (0) 3 990 52329 fax: +61 (0) 3 990 55412 (*) = contact author peter.holland@buseco.monash.edu.au abstract fundamental to the development of new customer value offerings via web-based commerce is a small firm’s ability to strategically acquire and exploit knowledge. the focus of this paper is the empirical testing of a normative web-based commerce adoption model developed from a review of the extant literature related to electronic marketing, the internet and the diffusion of new innovations. a preliminary test of the model’s theoretical contentions lent support to its overall focus, but found that the firm’s existing learning capabilities were diminished during the adoption of webbased commerce. consequently, sub-optimal adoption outcomes were associated with insufficient knowledge development. key words: web-based commerce, small firms, internet adoption, and market orientation introduction since the mainstream adoption of the internet by australian consumers (post 1996), there has been continual and marked increases in household participation, with 33% of all australian households having internet access as at may 2000, a 53% increase over may 1999 (abs, 1999). at july 2001, 75% of small firms, defined as having 19 or less employees (mclennan, 1999) were similarly connected to the internet (yellow pages business index, 2001). despite the high ‘internet implementation’ rates by small firms, only 27% utilise the medium to advertise their products online, a mere 19% accept online orders, and only 13% offer an on-line payment service for their customers (yellow pages business index, 2001). this is in stark contrast to small firm usage of the internet for e-mail (82%) and general research purposes (65%) (yellow pages business index, 2001). interestingly, and despite the apparent low actual usage rates for small businesses to date, 62% of australian small firms yet to introduce the internet into their operations, stated their imminent intention to become an online business (yellow pages business index, 2001). given these base statistics, there appears to be a significant discrepancy between the levels of commercial internet adoption and its contribution to incremental firm growth. a review of the literature indicates that although the potential benefits of web-based commerce are well documented (e.g. armstrong & hagel, 1996; hamill, 1997; hoffman & novak, 1997), there is a paucity of research dealing with the generic issue of how to effectively implement web-based commerce into a small firm’s operations. six specific areas deemed central to the adoption and effective implementation of web-based commerce were identified. the six areas, integral to the proposed normative web-based adoption model are as follows; 1) web-based commerce adoption drivers; 2) market orientation; 3) cooperative behaviours; 4) web-based business models; 5) value chain reconfiguration; and 6) web-based value. through a synthesis of the literature, a normative and generic adoption model for small firm web-based commerce was developed. proposed normative web-based commerce adoption model the model sought to combine components available to small businesses, therefore increasing possible acceptance of the model beyond specific industry characteristics. as such, this research sought to make two specific contributions. firstly, to provide a generic adoption model for small firms with regard to the strategic implementation of web-based commerce into their overall business strategy. secondly, to identify sources of potential difficulty and inefficiency in the actual implementation of a web-based commerce strategy for small firms. web-based commerce adoption drivers previous research into small firm adoption of web-based commerce has identified the medium’s perceived benefits as the primary motivator (e.g. poon & strom, 1997; poon & swatman, 1999). rogers (1995) argues that ajis vol. 10 no. 1 september 2002 100 successful adoption of complex innovations requires a knowledge base beyond ‘mere awareness’ of perceived benefits. during stage one (see figure 1 below) of the of the adoption model, it is predicted that firms gain an awareness of the perceived benefits of web-based commerce through exposure to the hype that surrounds the medium, and change agents (such as internet service providers [isps]) who promote it’s virtues. figure 1: the normative web-based commerce adoption model stage 1 however, complex innovations, such as the internet, require a knowledge base far exceeding awareness knowledge to (1) appreciate its possible application, and (2) reduce the possibility of misuse. rogers (1995) states that adopters must acquire a higher knowledge base comprising ‘how to’ and ‘principle’ knowledge. ‘how to’ knowledge provides web-based commerce adopters with an understanding of how to use the innovation effectively, and ‘principle’ knowledge refers to the theoretical underpinnings of the innovation. therefore, while awareness of the innovation may provide a possible ‘e-vision’ (sawhney & zabin, 2001), it is the acquisition of ‘how to’ and ‘principle’ knowledge that underwrites its successful implementation (rogers, 1995). market orientation the literature supports the reliance upon market orientation during the adoption of technological innovations by firms to enhance organisational learning (glazer, 1991; hoffman & novak, 1997; morgan, katsikeas & appuhadu, 1998). therefore, during stage two (see figure 2 below), it is argued that market orientated firms will access the required ‘how to’ and ‘principle’ knowledge needed to successfully adopt web-based commerce. figure 2: the normative web-based commerce adoption model: stage – 2 knowledge of: � consumers ‘how to’ & ‘principle’ k change agents motiva need created perceived benefits for firm firm behaviour limited access ‘how to’ & ‘principle’ knowledge change agents motiva need created perceived benefits for firm firm behaviour market orientation knowledge of: � consumers � market opportunities ajis vol. 10 no. 1 september 2002 101 the seminal work of hoffman and novak (1997) associates market-orientated firm behaviour with the development of a future web-based competitive advantage due to its ability to provide firm access to customer, market and technology intelligence. glazer (1991) also proposed the use of a firm’s market orientation to facilitate increased access to intelligence in information intense markets. in proposing market orientation as a defendable resource system, hunt and morgan (1995, p 11) define a firm’s market orientation to be: (1) the systematic gathering of information on customers and competitors, both present and potential, (2) the systematic analysis of the information for the purposes of developing market knowledge, and (3) the systematic use of such knowledge to guide strategy recognition, understanding, creation, selection, implementation, and modification . our contention is that such knowledge processes are a prerequisite for successful adoption of a complex technological innovation. in summary, stage two suggests that without access to specific knowledge of customers, technology and the marketplace, the firm’s adoption of web-based commerce will be less than optimal. cooperative behaviours stage three, as illustrated in figure 3, proposes the engagement of cooperative behaviours to share information through which the business’s knowledge base is expanded. mcwilliams and gray (1995) and lado, boyd and hanlon (1997) propose the use of cooperative strategies (e.g. logistics, payment & referrals) to overcome resource weaknesses that restrict the implementation of resource strengths. small firms in the possession of ‘how to’ and ‘principle’ knowledge may require assistance to access the benefits of web-based commerce due a possible lack expertise and/or time and financial resources. figure 3: the normative web-based commerce adoption model: stage – 3 . hoffman and novak (1997) propose the development of a cooperative rather than competitive approach to competitors to maintain a market orientation during web-based commerce adoption. complementary relationships with other firms are recommended to create additional value through new and innovative resource structures (brandenburger & nalebuff, 1995). rayport and jaworski (2001) also support the use of partnerships and strategic alliances which allow web-based firms to focus on their core competencies while developing resource clusters that enhance overall firm capabilities. however, during this process, the firm’s value chain and business model may be subject to a process of reconfiguration (timmers, 1998), requiring increased knowledge of customers’ desired benefits, technological capabilities and markets (rayport & jaworski, 2001). ‘how to’ & ‘principle’ knowledge change agents motiva need created perceived benefits for firm firm behaviour cooperative market orientation shared information increased knowledge knowledge of: � consumers � market opportunities ajis vol. 10 no. 1 september 2002 102 stage three demonstrates the use of the firm’s market orientation to share information and acquire new knowledge of customer needs, technological potential, and marketplace opportunities. this increases the ability to acquire the ‘how to’ and ‘principle’ knowledge required for the adoption of web-based commerce. cooperation at this stage, especially for small firms is vital given that the acquisition of information for webbased commerce may well represent a novel and therefore challenging area of knowledge development with regard a firm’s absorptive capacity (cohen & liventhal, 1990). web-based business models as a result of additional cooperative strategies, two issues related to the firm’s business model/s arise. firstly, the introduction of new firm strategies will require consideration of the existing operating structures, and secondly, the potential reconfiguration of the value chain may impact on a firm’s ability to access vital sources of intelligence. as illustrated in figure 4, stage four contends that as strategy changes, so must structure (mintzberg, 1990), and the firm’s market orientation is the key resource system upon which the new business model is determined. rayport and jaworski (2001) state that a high quality web-based business model should meet the following criteria; it must be unique, provide links between capabilities and benefits, support links between firm activities and capabilities, be mutually reinforcing, provide a link between the physical world and the virtual world, and lastly, the resource must be capable of supporting a sustainable advantage. as such, a firm’s market orientation may potentially provide an efficient resource to small firms whom typically have scarce resources (chappell & feindt, 1999) and simple operating structures (peterson, 1989; sanchez, 1997; chau & pederson, 2000). figure 4: the normative web-based commerce adoption model: stage – 4 consequently, such a web-based business model would be well placed to use market intelligence to develop capabilities that supported the value propositions so central to its function. stage four illustrates that as knowledge is gained and used cooperatively, the firm’s structure is altered to accommodate and support new firm capabilities. therefore, knowledge and the coordination of the resources are the focal drivers of business model transformation. ‘how to’ & ‘principle’ knowledge change agents motiva need created perceived benefits for firm firm behaviour market orientation market intelligence defendable structure business model cooperative knowledge of: � consumers � market opportunities ajis vol. 10 no. 1 september 2002 103 value chain reconfiguration stage five, as illustrated in figure 5 below, proposes the introduction of new value creating activities to satisfy the identified desired benefits of the target market/s. in line with market orientation theory, the challenge for firms during the introduction of web-based commerce, is to remain connected to customers and responsive to market opportunities (day, 1999). figure 5: the normative web-based commerce adoption model: stage – 5 under conditions of technological change, there is a greater likelihood that the link between firm offerings and customer needs may become divergent (slater & narver, 1994; enders & jelassi, 2000). to develop value offerings that complement the business model, timmers (1998) and afuah and tucci (2001) argue for the reconfiguration of the traditional value chain (porter, 1985) to access to new sources of web-based value. the model to date suggests that market orientated firms who reconstruct their value systems, may create value through the alignment of business strategy and structure with an explicit understanding of the buyers of their offerings (see drucker, 1985; slater & narver, 1998). during such a process, business model development may liberate ‘trapped value’ i.e. value gained from market and value system efficiencies, and/or introduce ‘new-tothe-world value’ i.e. new value through customisation and personalisation (rayport & jaworski, 2001). in summary, the model assumes the firm has now acquired ‘how to’ and ‘principle’ knowledge through its market orientation, identifying desired customer benefits which require new firm capabilities, resulting in the need to reconfigure the value chain, to access new web-based value. knowledge of: � consumers � market opportunities ‘how to’ & ‘principle’ knowledge change agents motiva need created perceived benefits for firm firm behaviour market orientation new value possibilities identified new value sources business model cooperative value chain ajis vol. 10 no. 1 september 2002 104 web-based value within the normative adoption models development thus far (see figure 6 over page), the firm’s market orientated behaviours are posited to be central to developing a web-based value proposition. that is, in terms of the development of firm capabilities through cooperative actions and access to new value creating activities/architecture to supplement the ‘traditional’ activities of the value chain. four sources of web-based value have been identified as efficiency, complementarities, lock-in and novelty (amit & zott, 2001). the value sources are dependent upon two important factors. firstly, a synergistic relationship exists between the sources requiring a need to develop all four to enhance the value of each individual source. secondly, their contributions to a competitive advantage are premised upon mutual gains between the firm and customer. therefore, all are dependent on increased levels of strategic customer, market and technology knowledge. figure 6: the normative web-based commerce adoption model given the noted discrepancy between internet implementation and its integration into activities supportive of incremental growth, a clear opportunity to empirically test the developed normative best-practice adoption model. ‘how to’ & ‘principle’ knowledge change agents motiva need created perceived benefits for firm firm behaviour market orientation unique value proposition business model cooperative value chain web-based value competitive advantage knowledge of: � consumers � market opportunities new value � efficiency � complementary ajis vol. 10 no. 1 september 2002 105 research method due to the exploratory nature of the research, a multiple case study design (yin, 1994) of in-depth interviews with five small firms to explore the adoption of web-base commerce within a real-life context. the research aims were to observe the degree of congruence between firm adoption of web-based commerce and the proposed model, and to identify difficulties experienced and value received. given the proposed importance of market orientation to the adoption model’s construction, pelham and wilson’s (1996) small and medium firm specific market orientation scale was used to measure the degree of pre(traditional commerce) and post-adoption (webbased commerce) market orientation. a judgemental sampling approach (babbie, 1999) enabled the cases to be selected via a selection criterion that enhanced the research aim of observing the influence of an intangible firm resource (market orientation) during the adoption of web-based commerce and the development of new value creating activities. the firms met the following criteria: all had served domestic, interstate and international markets in excess of five years, web-based commerce supplemented their existing operations, and their web site facilitated customer service, marketing communications and the exchange of a physical product. the findings the findings of this preliminary research are that the five firms did not follow the normative adoption model arising from the synthesis of the literature review. there was however support for the theoretical contentions of some discrete stages within the model. most notably, the influence of perceived benefits. table 1 illustrates that the relative degree of market orientation measured (with the exception of one) was less for web-based commerce in contrast to ‘traditional’ commerce, ranging from –2.25% to –28%. table 1: description of firms firm a firm b firm c firm d firm e industry book children's winemaker whisky knitwear publishing footwear distiller manufacturer adoption year 1996 1997 198 1995 1998 initial motivation change agent change agent change agent hype hype traditional market orientation 4.56 4.11 4.33 4.11 4.89 web-based market orientation 3.56 4.22 3.11 3.56 4.78 difference -22.00% 2.70% -28.00% -16.00% -2.25% however the actual presence of market oriented behaviours was not evident in any of the firm’s web-based commerce behaviours. not suprisingly, the firms did not access the ‘how to’ and ‘principle’ knowledge essential to the development of a web-based commerce competitive advantage. one firm did demonstrate delayed (three years post adoption) market orientated behaviours. they engaged in cooperative strategies and business model development and were able to clearly articulate the needs and wants of their web-based customers. across all the firms, resource constraints related to time and finances were offset through the acceptance of adoption incentives/assistance during their initial adoption of web-based commerce development. however, the most prevailing difficulty experienced across the firms was access to information sources through which ‘how to’ and ‘principle’ knowledge could be obtained and incorporated into the development of web-based commerce. the ability of local isps to act as ‘technology linkers’ was clearly beyond the scope of their capabilities, thereby providing tacit support for plume (2001) who notes a significant challenge confronts webbased commerce in the form of knowledge integration. specifically, this challenge is to the ability of firms (or external persons, such as isps) to successfully integrate traditional marketing practices with new technological opportunities to create value that is supportive of a sustainable competitive advantage. by and large, access to new value creating activities was not observed. despite the firms demonstrating the capability to develop a sustainable competitive advantage in ‘traditional’ commerce through the involvement of customers and channel members, such parties were reduced to mere spectators during their initial development of web-based commerce. it was observed in one case (which also demonstrated superior market-orientated behaviours) that when customers were transformed from spectators to participants, an increased understanding of the potential of web-based commerce did in fact occur. however, the ajis vol. 10 no. 1 september 2002 106 web-based value observed (communication & research efficiencies), was internalised within the firms, not shared with customers or channel members. while web-based value received exceeded the perceived cost of adoption (time & effort), it was insufficient to lay the foundation to a future competitive advantage. discussion this research sought to identify possible reasons for the marked discrepancy between small firm implementation of web-based commerce and its ability to create incremental value for firms and their customers. the research findings identify several major implications for academia and small firm practitioners engaged in web-based commerce. firstly regarding academia, the application of a firm’s market orientation, its nature, measurement, and value during web-based commerce adoption require further consideration. secondly, the unsuccessful acquisition, and therefore conversion of adoption information into adoption knowledge by small firms represents a major hurdle to the optimal adoption of web-based commerce. the role of market orientation as the central driver of the adoption model, market orientation was argued to be responsible for developing a rich knowledge base, reflective of customers needs (expressed and latent), marketplace opportunities, and the technologies that will connect the entities. without such knowledge, firms may not sense the desired benefits of their present and future customers required to identify value-creating activities supportive of a competitive advantage. a clear discrepancy was obtained between recorded market orientation measurements using pelham and wilson’s (1996) scale and observations of actual firm behaviours. the firms, while claiming to have developed strategies based upon an understanding of their web-based customers behaviours (surveys), demonstrated no strategic actions related to developing knowledge of the customers, technologies and markets associated with web-based commerce (observations). the first concern relates to the application of the firm’s market orientation. as a pre-existing intangible resource, a firm’s market orientation must be maintained and applied to contribute value to the firm. market orientation, intrinsically reflective of a firm’s culture (hunt & morgan, 1995), is a source of value through application, rather than merely through possession (prahalad & hamel, 1990). the absence of observable market-oriented behaviours restricted the empirical testing of the normative adoption model’s application. the apparent failure to transfer the observed market-oriented behaviours from ‘traditional’ commerce to web-based commerce would appear related to the firm’s lack of direction and knowledge developing web-based capabilities. the very of nature of a firm’s market orientation may well be challenged by new learning processes inherent to web-based commerce. slater (2001) suggests that a firm’s established market orientation must evolve to ensure greater market sensing in the face of significant market change. slater notes that a future challenge for firms desirous of developing an e-market orientation is to stay connected to their customers. this echo’s the literatures consistent ‘listen to the customer, understand the customer, and provide a solution for the customer’ proclamation for market-orientedness. the assumption being that market orientation is dependent upon a oneway linear relationship between customers, marketing, and the technologies employed to provide solutions. however, the research findings lend support to the proposition of wrenn (1997) that the construction and application of the market orientation construct may not be so straightforward. the firms found difficulty gaining access and understanding of their customers’ needs and wants. as such, the traditional interpretational role of marketing simply did not occur. consequently, technologies, be they products, processes, or a combination of both, did not eventuate to solve customer needs and wants. wrenn further asserts that a non-linear relationship between the customer, marketing and technologies (kotler, 1997) is required to translate technological attributes into customer benefits to sustain a market orientation during times of complex technical change (i.e. the internet). the suggestion is that a far more dynamic role for marketing may be required to maintain or establish market-orientated behaviours within an environment influenced by rapid technological change. it appears the role of market orientation during stage two was negatively impacted by the exclusion of customer’s needs, wants and consumer behaviour information from the adoption process. the absence of consideration for customer needs during the development process may produce sub-optimal outcomes, limiting internal efficiencies rather than the incremental long-term value. however, the activities of one firm over the eight months preceding the data collection period provides evidence of the valuable role customers contribute to market orientated firms in their adoption of web-based commerce. the firm developed close relationships with web-based customers in the north american market. in addition to being the only firm to record an increased web-based market orientation, they were also the only firm to articulate a clear understanding of their web-based customers’ preferences and buying behaviour, market opportunities and the application of technology for web-based commerce. along with an increased web-based market orientation (the survey), obvious market-oriented behaviours were observed. for example, the firm ajis vol. 10 no. 1 september 2002 107 developed a database through which valuable customer information of preferences was recorded and regularly updated, providing the firm with a longitudinal record for customer profiling. the firm also engaged in cooperative strategies to acquire ‘how to’ knowledge through sharing the cost of technical training. as such the firm’s adoption behaviours (albeit three years post adoption) provide partial support for stages two and three. the firm is using market-oriented behaviours to acquire a greater understanding of their customers and using cooperative practices to enhance such behaviours. finally, the case method produced an inconsistency between measured and observed market oriented behaviours. the recorded levels of web-based market orientation appear to reflect ‘anticipated’ future actions, rather than ‘actual’ behaviours. the firms appeared unable to separate their beliefs concerning what they actually do in a traditional sense compared to what that actually do in a web-based sense. given the prominence of quantitative methods that employ surveys to measure market orientation, the findings cast a shadow over the accuracy and reliability of such methods. the findings lend support to the contentions of rouse and daellenbach (1999) that a greater understanding of the actual contribution of such intangible firm resources should occur from within, rather than outside firms. knowledge acquisition and development difficulties a recurrent process throughout the web-based adoption model is the acquisition of market intelligence related to target markets, marketplaces and technologies and its conversion into knowledge. with the exception of one firm’s delayed behaviours, the firms have not acquired such information, and therefore lacked a sufficient knowledge base upon which to build a web-based commerce platform. as previously discussed, the firms did not utilise their observed traditional market oriented behaviours during adoption of web-based commerce. three interrelated factors appear to have contributed to the firm’s insufficient knowledge base. firstly, an inability to visualise new sources of value and alternative structures that would be complementary to their existing value chains. secondly, an over reliance upon they’re respective isps/advisors to perform a technology linking role, and therefore an apparent lack of enthusiasm to explore the new web-based landscape themselves. lastly, the firm’s individual lack of absorptive capacity appears to have hampered what little efforts have been made to learn about web-based commerce opportunities. the lack of vision sawhney and zabin (2001) posit that many firms may be constrained by assumptions, inherited from their past that restrict their view of the future. despite the literature’s expectation that market oriented firms will challenge past assumptions through generative learning processes, an e-vision was not visualised during adoption. as such, no consideration was given to visionary architectural change, but rather, reliance was upon modifying the past. the result reflects an ignorance as to the need for ‘how to’ and ‘principle’ knowledge. overall the initial approach appears very much a case of “if you don’t know where you’re going, any road will take you there” (sawhney & zabin, 2001, p 11). the findings are consistent with the conclusions of chaston, badger, mangles, and salder-smith, (2001) that perhaps small firms underestimate the need for planning prior to adopting webbased commerce. the quasi-technology linker the firms relied heavily on external isps/advisers to facilitate the development of web-based commerce at the expense of using their own knowledge of existing customers needs. subsequent interviews with the relevant isps confirmed that they had neither the ability, nor the desire to provide ‘how to’ and ‘principle’ knowledge. therefore, the opportunity to develop a value proposition derived from customer participation was lost, along with the value of the firm’s pre-existing market orientation. the lesson to be learnt would seem to be to empower customers as co-architects of future value, and the systems that deliver that value. this would allow the isps to implement what is possible from an architectural perspective, rather than what is current thinking from an engineering perspective. individual absorptive capacity absorptive capacity can be defined as the “routines and processes by which firms acquire, assimilate, transform, and exploit knowledge to produce a dynamic organizational capability” (zahra & george, 2002), or simply, the ability to acquire and strategically use ‘how to’ and ‘principle’ knowledge. the firms lacked intensity, speed, and direction in their observed efforts to acquire external knowledge to the detriment of their ability to take possession of ‘how to’ and ‘principle’ knowledge. this appears to indicate the difficulty in transferring a market ajis vol. 10 no. 1 september 2002 108 orientation from ‘traditional’ to web-based commerce without a guiding e-vision. it also perhaps reflects the difficulty of acquiring knowledge from such a novel and challenging domain. conclusion in summary, the overall research findings support the research of chau and lawrence (1998) who found little evidence of enthusiasm to actively pursue literature, or advice, regarding web-based commerce by tasmanian firms who had adopted web-based commerce. without engaging in ongoing market oriented behaviours (gathering, dissemination and strategic use of information), the value of a market orientation is significantly decreased. mere prior possession of the resource is not sufficient to contribute future value, it must be engaged to ensure knowledge is first gathered and disseminated within the firm and it’s strategic partners. this research, through the development of a normative web-based adoption model sought to examine the influence of a firm’s market orientation to assist in the development of a web-based competitive advantage. it was theorised that market-orientated firm behaviours were essential requirements to optimising the development of web-based commerce in small firms within which resource (time, finances and knowledge) constraints commonly occur. that only one firm was satisfied with their adoption of web-base commerce, and that they were the only firm to exhibit market-orientated behaviours (albeit three years post adoption), provides partial support for the role of market orientation during web-based commerce adoption. this research has highlighted the difficulties in maintaining and/or developing a market orientation in environments influenced by rapid technological change. given the significant contribution of small firms to the australian economy (gare, 2001), it is imperative further research examine the learning processes that contribute to the development of specific intelligence upon which incremental growth and future web-based competitive advantages are dependent. references afuah, a. and tucci, c. l. (2001) internet business models and strategies, sydney: mcgraw-hill irwin. amit, r. and zott, c. (2001) “value creation in e-business”, strategic management journal, vol 22 (6-7), 493-520. armstrong, l. and hagel, j. (1996) “the real value of on-line communities”, harvard business review, vol 74 (3), 134-141. australian bureau of statistics. (1999) communications and information technology: household use of information technology, http://www.abs.gov.au, catalogue number 81146.0, (accessed 21 october 2001). babbie, e. (1999) the basics of social research, melbourne: wadsworth publishing company. brandenburger, a. m. and nalebuff, b. j. (1995) “the right game: use game theory to shape strategy”, harvard business review, vol 73 (4), 57-71. chappell, c., & feindt, s. 1999. analysis of e-commerce practice in smes, http://www.kite.tsa.de/, (accessed 12 september 2001). chaston, i., badger, b., mangles, t. and sadler-smith, e. (2001) “the internet and e-commerce: an opportunity to examine organisational learning in progress in small manufacturing firms”, international small business journal, vol 19 (2), 134-147. chau, s. and lawrence, k.l. (1999) electronic commerce knowledge, capabilities, and utilisation: the tasmanian perspective, proceeding of collector’ 98, university of new south wales. cohen, w.m. and levinthal, d.a. (1990) “absorptive capacity: a new perspective on learning and innovation”, administrative science quarterly, vol 35, 128-152. day, g.s. (1999) “creating a market-driven organisation”, sloan management review, vol 41 (1), 23-34. drucker, p.f. (1985) innovation and entrepreneurship, new york: harper and row. enders, a. and jelassi, t. (2000) “the converging business models of internet and bricks-and-mortar retailers”, european management journal, vol 18 (5), 542-550. gare, s. (2001) “herded, bossed and cajoled, then milked”, the australian, 14 july 2001. glazer, r. (1991) “marketing in an information intensive environment: strategic implications of knowledge as an asset”, journal of marketing, vol 55, 1-19. hamill, j. (1997) “the internet and international marketing”, international marketing review, vol 14 (5), 300-323. hoffman, d. l. and novak, t. p. (1997) “a new marketing paradigm for electronic commerce”, the information society, vol 13 (1), 43-55. hunt, s.h. and morgan, r.m. (1995) “the comparative advantage theory of competition”, journal of marketing, april, vol 59, 1-15. kotler, p. (1997) marketing management: the millennium edition, sydney: prentice hall. ajis vol. 10 no. 1 september 2002 109 lado, a.a., boyd, n.g. and hanlon, s.c. (1997) “competition, cooperation, and the search for economic rents: a syncretic model”, academy of management review, vol 22 (1), 110-141. mclennan, w. (1999) small business in australia, abs catalogue no. 1321.0. mcwilliams, a. and gray, s.r. (1995) “understanding quasi integration”, journal of business strategies, vol 11, 171-195. mintzberg, h. (1990) “the design school: reconsidering the basic premises of strategic management”, strategic management journal, vol 11 (3), 175-195. morgan, r.e., katsikeas, c.s. and appuh-adu, k. (1998) “market orientation and organisational learning”, journal of marketing management, vol 14, 353-381. pelham, a.m. and wilson, d. (1996) “a longitudinal study of the impact of market structure, and market orientation culture on dimensions of small-firm performance”, journal of the academy of marketing science, vol 24 (1), 27-43. peterson, r.t. (1989) “small business adoption of the marketing concept vs. other business strategies”, journal of small business management, vol 27 (1), 38-46. plume, h. (2001) survival of the e-fittest, http://www.capcollege.bc.ca/admin/formedia/2001releases/ebiz.html. (accessed 5 august 2001). poon, s. and strom, j. (1997) an australia-uk study of small firm internet use, proceedings of the 1997 international management development association conference, seoul, korea, 316-323. poon, s. and swatman, p.m.c. (1999) “an exploratory study of small business internet commerce issues”, information & management, vol 35, 9-18. porter, m.e. (1985) competitive advantage, new york: free press. prahalad, c.k. and hamel, g. (1990) “the core competence of the corporation”, harvard business review, vol 68 (3), 79-91. rayport, j.f. & jaworski, b.j. (2001) e-commerce, sydney: mcgraw-hill/irwin. rogers, e.m. (1995) diffusion of innovations, 4 th edition, new york: the free press. rouse, m.j. and daellenbach, u.s. (1999) “rethinking research methods for the resource-based perspective: isolating sources of sustainable competitive advantage”, strategic management journal, vol 20 (5), 487494. sanchez, r. (1997) “preparing for an uncertain future: managing for strategic flexibility”, international studies of management & organisation, vol 27 (2), 71-95. slater, s.f. (2001) “market orientation at the beginning of a new millennium”, managing service quality, vol 11 (4), 230-232. slater, s.f. and narver, j.c. (1994) “does competitive environment moderate the market orientation performance relationship”, journal of marketing, january, vol 58, 46-55. slater, s.f. and narver, j.c. (1998) “customer-led and market-orientated: let’s not confuse the two”, strategic management journal, vol 19 (10), 1001-1006. sawhney, m. & zabin, j. (2001) the seven steps to nirvana, sydney: mcgraw-hill. timmers, p. (1998) “business models for electronic markets”, electronic markets, 8 (2), 3-8. www.electronicmarkets.org, (accessed 2 june 2001). wrenn, b. (1997) reality based market orientation, proceeding of marketing management association, chicago, il: marketing management association and department of marketing, school of business, indiana state university. yellow pages business index. (2001) e-commerce and computer technology, http://www.corporate.pacificaccess.com.au, (accessed 13 october 2001) yin, r.k. (1994) case study research design and methods, 2 nd edition, thousand oaks, california: sage publications. zahra, s.a. and george, g. (2002) “absorptive capacity: a review, reconceptualization, and extension”, academy of management review, vol 27 (2), 185-203. ajis special edition knowledge management december 2001 harnessing intranet technology for organisational knowledge creation jan damsgaard weatherhead school of management case western reserve university 10900 euclid avenue, cleveland, oh 44106 phone: +1 216 368 4640, fax: +1 216 368 4776 e-mail: jxd86@po.cwru.edu rens scheepers school of information technology, swinburne university of technology, po box 218, hawthorn, victoria 3122, australia phone: +61 3 9214 8439, fax: +61 3 9214 5501, e-mail: rens@it.swin.edu.au abstract many organisations have embraced intranets with the intent of harnessing the technology to support knowledge management initiatives. despite the promise that intranet technology holds in this regard, many of the early research studies indicate rather disappointing results. in this paper we propose a model that organisations can use to conceptualise and reflect on their intranet applications with a view towards more fruitful results, specifically in terms of knowledge creation. we do so by drawing upon nonaka's well-known framework of knowledge creation and combining that with a taxonomy of five intranet use modes. for each of nonaka's four knowledge creating activities we associate and describe the corresponding primary intranet use mode that we argue can foster the knowledge creation process. we illustrate the arguments with findings from our own empirical intranet field studies and other documented intranet-related knowledge management research. we conclude with some implications of the model and we suggest avenues for further research. introduction the mid 1990s marked the global adoption of the internet, and especially the web in many spheres of society. in organisational contexts in particular, web technologies have opened up opportunities for much wider participation by workers without the need for specialist it skills. intranets in particular have been embraced in record time by many organisations. from being a largely unknown technology only a few years back, a vast number of knowledgeintensive organisations have now embarked on large-scale intranet implementations. organisations have implemented intranets for a variety of reasons including to share information, to enhance organisational communication in general, and to support collaboration across departmental, functional and regional boundaries (bernard, 1996; scacchi & noll, 1997; mcnaughton et al., 1999). a recent development is the organisational application of intranet technology to support knowledge management (km) processes (scott, 1998; alavi & leidner, 1999; newell et al., 1999; alavi & leidner, 2001). the ideas that underpin km are not fundamentally new (spiegler, 2000) and can be traced to concepts such as the learning organisation (senge, 1990) and organisational memory (huber, 1991). in addition, the role of information technology (it) in enabling and supporting knowledge work and knowledge workers in organisations has been well documented in the past (eg. ciborra, 1996; davenport et al., 1996; sviokla, 1996). however, the more recent mass organisational adoption of intranets, combined with some unique features of the technology (such as its hypermedia capabilities) has stimulated fresh research interest into the application potential of intranet technology for km. there have been some descriptive case studies that document the various approaches being followed by organisations (especially the professional services firms) with regard to their in-house knowledge management systems (kms) development (eg. alavi, 1997; ezingeard et al., 2000; gottschalk, 2000). considerable uncertainty remains as to how organisations can successfully harness these technologies to foster their km processes (as reflected by the number of current suggested research agendas pertaining to kms (eg. hahn & subramani, 2000; alavi & leidner, 2001). although attention is directed towards specialised knowledge management systems, alavi & leidner (1999) found that companies that pursue km initiatives most often do so by relying on an intranet (as opposed to an isolated kms). despite the promise that intranet technology holds in this regard, there are however few documented examples of successful intranet-supported km initiatives. in fact, some of the available research studies on intranet supported km initiatives report rather discouraging findings (eg. newell et al., 1999). we suspect that there are two reasons for such disappointing findings. many intranet studies to date have focused mainly on only one popular application of the technology, namely publication of organisational information. we are convinced that richer application modes of the technology beyond publication may indeed be conducive to knowledge creation. a further reason for these disappointing findings may relate to a fairly static view of intranet technology as a "given", packaged technology with universal characteristics and features (lyytinen & damsgaard (2001) warn against this stationary view). instead we ajis special edition knowledge management december 2001 advocate the idea that to a large degree an intranet takes form after the context in which it is implemented and the organisational culture (schein, 1996), history and traditions of earlier technology adoptions (kling, 1980) are important when seeking to understand its manifestation. thus seen, the intranet is not a "packaged" technology with fixed attributes, but rather a learning intensive and highly malleable technology that is moulded and shaped according to the social forces at play in the organisation (hughes, 1987; williams & edge, 1996). this explains why the same technology can manifest adversely in different organisational settings (as observed by bansler et al., 2000). while some other researchers have explored the potential of intranets to support km in broad terms (eg. scott, 1998; gottschalk, 2000), we focus here specifically on how intranets may be used to foster the creation of new organisational knowledge. in particular we distinguish a number of intranet application modes that can be exploited to support various knowledge creation processes. we do so by proposing a combined model of knowledge creation and intranet use modes. although this paper is speculative in nature, our research is empirically founded. briefly summarised, we conducted an in-depth interpretive field study of intranet implementations in four large organisations (two in denmark and two in south africa) over a period of five years from 1997 to 2001. formal and intensive rounds of data collection were interspersed with periods of informal data collection. semi-structured formal interviews formed the basis of the study and in each case we interviewed a wide variety of organisational actors who were involved with the intranet. data was also collected on a more informal basis by means of regular e-mail and phone contact, lunch meetings, discussions and by attending presentations made by the case organisations. such data included notes from discussions, emails, policies, reports, demonstrations, intranet usage statistics, and our own inspection of the intranets. details of the empirical field study have been published elsewhere (see damsgaard & scheepers, 1999; bansler et al., 2000; damsgaard & scheepers, 2000). this paper is outlined as follows. firstly, some features of intranet technology are highlighted by drawing upon a taxonomy of five primary intranet use modes. thereafter nonaka's well-known model of knowledge creation is briefly presented. the model of knowledge creation and the taxonomy of intranet use modes are then combined. the combined model is illustrated with findings from our own empirical intranet field studies and other documented intranet-related km research. some implications of the combined model are discussed and avenues for further research are suggested. intranet technology defined technically, intranets are the application of internet technology (and specifically the world wide web service) for a prescribed community of users (typically members of an organisation). well-understood and widely available internet technology and standards (web servers, browsers, protocols) are employed, but access is restricted exclusively to specified organisational members, typically by means of passwords and/or firewalls (oppliger, 1997; laudon & laudon, 2000). the technical set-up of an intranet is relatively straightforward and the first information content can be quite easily created. in the following subsections we address some specific features of intranet technology. intranet technology characteristics intranet technology is multi-purpose, richly networked and integrates text, graphics, sound, and video (bernard, 1996; hills, 1997; damsgaard & scheepers, 1999). intranet technology supports both structured and unstructured data, mostly by means of html (hypertext mark-up language) documents as the common language of the web (lyytinen et al., 1998). since html describes the presentation of data independent of any specific computing platform, it enables information exchanges between diverse computing environments within the organisation and across functional boundaries. such exchanges are facilitated through departmental intranet servers located behind the organisational firewall which organisational members can readily access using a standard web browser (chellapa et al., 1997). unlike most it, intranets do not exclude the presence of other it systems (as a new inventory system often excludes the former). instead intranet technology is the unifier that can integrate existing it systems and provides "legacy systems" with a new graphical interface. therefore intranets are often referred to as "glueware" or "middleware" (lyytinen et al., 1998). as such, intranet technology can unify various computer-based systems in the organisation into one rich "system" with the web browser as the universal interface. ajis special edition knowledge management december 2001 intranet technology usage the organisational application of intranet technology tends to evolve and increase in sophistication over time (scheepers & damsgaard, 1997; romm & wong, 1998). this pattern is not imposed by the technology itself, but exhibits the organisational learning involved in applying the technology (attewell, 1992). initially, the technology is typically used for publishing "static" information (eg. departmental home pages, technical documents, product information). provided the supporting technical infrastructure exists, setting up a simple intranet website to publish information does not involve a major learning or financial commitment (ciborra & hanseth, 1998). by creating intranet sites with information that employees can readily access via the browser, organisations can save the obvious costs associated with printing, publishing and distributing paper-based information to employees. intranet-based publication also ensures that everyone uses the most recent version of information (compared to the alternative of physically distributing new and removing old copies of some document). as the organisation becomes more familiar with the technology, it may be applied for more advanced purposes (bansler et al., 2000). intranet technology can be applied in different "use modes" simultaneously (damsgaard & scheepers, 2000). these range from simple use modes such as publishing discussed above, to more advanced use modes such as organisational-wide searching for information; transacting with functionality on intranet pages and other organisational computer-based information systems; interacting between individuals and groups in the organisation; and even the recording of the computer-based "organisational memory". the five intranet use modes and typical application examples are summarised in table 1. table 1: a summary of intranet technology use modes (damsgaard & scheepers, 2000) use mode publishing transacting interacting searching recording description using the technology to publish information (eg. home pages, newsletters, technical documents, product catalogues, employee directories). using the technology to transact with functionality on intranet pages and other organisational computer-based information systems eg. via web forms. using the technology to interact with other individuals and groups in the organisation (eg. via discussion groups, collaborative applications). using the technology to search for organisational information (eg. via search engines, indexes, search agents). using the technology to record the computer-based "organisational memory" (such as capturing best practices, business processes, frequently asked questions). a model of intranet-facilitated knowledge creation we now propose an integrated model of intranet-facilitated organisational knowledge creation. we do so by combining nonaka's well-known model of knowledge creation (nonaka, 1994; nonaka & konno, 1998) with the taxonomy of intranet use modes as described above. to position the model, we briefly explore facets of knowledge, knowledge management and knowledge creation processes. the combined model is illustrated with findings from our own empirical intranet field studies and other documented intranet-related km research. facets of knowledge and knowledge management processes a detailed examination of the nature of knowledge itself and of the complexities of organisational km is beyond the scope of this paper. instead we highlight some facets of knowledge and its organisational management that are fundamental in conceptualising any proposed information technology support for the process. one of the central aims with organisational km is to leverage the knowledge of individuals or teams so that this knowledge becomes available as a resource for the entire organisation. this resource should ultimately not be dependent on particular individuals and should survive the originating individual (davenport & prusak, 1998). km is often subdivided into four highly intertwined organisational processes (wiig, 1993; alavi & leidner, 2001). the first process is knowledge creation and involves the creation and addition of new knowledge to the organisation's knowledge repository. the second is knowledge storing/retrieval and involves the identification and access of relevant knowledge from the knowledge repository. this is also referred to as "accessing the organisational memory" (huber, 1991). the third process is the transferring of knowledge whereby organisational actors share and dissemiajis special edition knowledge management december 2001 nate knowledge within the organisation. lastly, knowledge application refers to the steps where knowledge is translated into action. in this respect, alavi and leidner (2001) point out that improved organisational performance hinges not on knowledge itself, but on the ability to translate knowledge into effective action. knowledge is commonly separated into tacit and explicit knowledge. explicit knowledge can be expressed in words and numbers and shared in the form of data, scientific formula, specification and manuals (nonaka, 1994; nonaka & konno, 1998). this kind of knowledge can readily be transmitted between individuals formally and systematically, and consequently also through modem information infrastructures such as an intranet (alavi & leidner, 1999). tacit knowledge on the other hand is not easily visible and expressible, and it is hard to transfer because it cannot be stated explicitly (davenport & prusak, 1998). tacit knowledge is personal and difficult to formalise, which also makes it hard to communicate and share with others. tacit knowledge is deeply rooted in an individual's actions, skills and experience as well as in their ideals, values and emotions (wiig, 1993). according to davenport & prusak (1998) knowledge originates and resides in the minds of knowers, but it also becomes embedded in organisational documents, repositories, routines, processes and norms. knowledge creation nonaka and konno (1998) model knowledge creation as a process of interactions between explicit knowledge and tacit knowledge. their model incorporates four knowledge creation modes based on the conversion processes between tacit and explicit knowledge: socialisation, extemalisation, combination and intemalisation (see figure 1). in table 2 we summarise nonaka and konno's four knowledge creation modes. tacit knowledge tacit knowledge explicit knowledge explicit knowledge figure 1: the model of nonaka and konno of organisational knowledge creation processes ajis special edition knowledge management december 2001 table 2: a summary of four knowledge creation modes (nonaka & konno, 1998) knowledge creation mode socialisation extemalisation combination internalisation description the exchange of tacit knowledge between individuals by means of joint activities and social interaction (eg. the traditional notion of apprenticeship). the expression of individual tacit knowledge into comprehensible codified forms that can readily be understood by others (eg. expressing ideas, images as words, concepts, figurative language, and visuals). the processing and combination of explicit knowledge into more complex sets of explicit knowledge (in useable formats). this involves the capturing, combination, dissemination and the systematisation of knowledge (eg. the development of business plans based on various existing data sources). the conversion of explicit knowledge into the organisation's tacit knowledge. this requires individuals to access the knowledge of the group or the entire organisation (eg. via organisational training programs, learning-by-doing exercises) and identify knowledge relevant to their own roles. intranet-facilitated organisational knowledge creation much of the km literature is based on an information systems perspective and the belief that km systems, for example intranets, can be used to capture and store organisational actors' knowledge and make it available to others (newell et al, 1999). due to the characteristics of the technology, intranets are an efficient mechanism for distributing codified knowledge, but we argue here that intranets also have the potential to redraw the borderline between tacit and explicit knowledge domains. the cheap and proven intranet technology makes it economically attractive and feasible for organisations to codify a larger portion of their tacit knowledge base. however, it has been pointed out that this does not diminish the importance of tacit knowledge (david et al., 1999). on the contrary, it stipulates the need for high skill levels and competence when selecting the appropriate codified knowledge. in the following we associate intranet use modes to each of the knowledge creation modes in nonaka and konno's model. our model is depicted in figure 2. for analytical reasons, we associate only the primary intranet use mode to each knowledge conversion process, with intranet publishing underpinning all the use modes. we stress that we make such distinctions only for analytical purposes. we believe that it is indeed the integration of all the intranet use modes that will unleash the true potential of intranets to facilitate knowledge creating processes. ajis special edition knowledge management december 2001 tacit knowledge tacit knowledge explicit knowledge explicit knowledge figure 2: primary intranet use modes for facilitating knowledge creation socialisation (facilitated by intranet interaction) traditionally, socialisation involves capturing knowledge through physical proximity (nonaka & konno, 1998). the process of acquiring knowledge is largely supported through direct interaction. information is accessed at the actual job site within the company and the latest available information is collected and interpreted as collective action. disseminating tacit knowledge is another key aspect of socialisation. the process of transferring one's ideas directly to, or have them challenged by, colleagues is a means to share and create personal knowledge. in short, the key to acquire tacit knowledge is through experience and social interaction. though it may be more efficient, face-to-face social interaction is not necessarily a prerequisite to acquire tacit knowledge. indeed, the socialisation process can be richly facilitated electronically. in this regard, ngwenyama & lee (1997) demonstrate that even a "lean" communication medium such as email has a rich capacity for exchanging tacit thoughts. for example, the evolution of emotions like the "smiley" (:-)) in emails shows how simple ascii characters may be used to convey shared connotations between members of a social system. we see the primary intranet use mode for socialisation as interaction. intranets can support various forms of personal interaction, thereby connecting knowledgeable individuals to each other. in its simplest manner it may be in the form of one-to-one interaction. another form that is supported can be one-to-many interaction where one individual shares his/her ideas or views using the intranet. the most complex form is many-to-many interaction where a group of people interacts with another group of people using the intranet. for example, the marketing department may interact with the production department about recent sales and the need to reschedule production plans. popular intranet applications to support such interaction include threaded discussion groups, employee home pages, project pages, group calendars, various types of collaborative applications such as shared workspaces and even concurrent engineering (coleman, 1997; scherer, 1997). a site focusing primarily on interaction does not necessarily have to reside physically on an organisation's web server, but might as well be hosted somewhere else and be accessed via the intranet (eg. groove.net). compared with face-to-face, the intranet is neither an obvious nor the best facilitator of interaction-based knowledge conversion. however when physical distance, time differences or working conditions make face-to-face interaction impossible or difficult, the intranet can be a viable alternative. for example, in one large south african company we found an interactive intranet discussion group called "the wall" where employees can express their opinions anonymously by painting graffiti slogans on an html background that resembles a brick wall. the wall was a tremendous success among the employees, who stated that it was aj1s special edition knowledge management december 2001 an attractive way to "let off steam". the management could also read the graffiti on the wall and gain a feel of the general atmosphere in the company. not all intranet-facilitated knowledge conversion need necessarily be "desired". in one case, we noted the lack of interaction in intranet based cross-functional discussion groups at a large danish company. the management recognised that information exchanges rarely took place between functions (resulting in functional knowledge "silos"), but had hoped that by implementing an intranet this would change. however, in our interviewees' opinion it became clear that the intranet made the lack of cross-functional interaction even more visible. interviewees often cited the empty intranet discussion groups and this effectively reinforced this undesired behavioural pattern in the organisation (for similar observations see scheepers & damsgaard, 1997; newell et al., 1999; lamb & davidson, 2000). externalisation (facilitated by intranet recording) in practice externalisation is supported by two key factors (nonaka & konno, 1998). the first is the articulation of one's own tacit knowledge through techniques that help to express ideas or images as words, concepts and figurative language (such as metaphors, or analogies). the second factor is the translation of others' tacit knowledge into readily understandable forms. an example of this from the systems development domain is the altering between analytical and experimental modes of inquiry to determine system requirements (boehm, 1988; mathiassen & stage, 1992). we see the primary mode of intranet use for externalisation of knowledge as recording. the ease with which information from a variety of quarters in the organisation can be assimilated and integrated using intranet technology, means that the intranet can ultimately become the medium for the "organisational memory" (as huber (1991) envisioned the concept). by using the intranet as "definitive record" for organisational processes as they unfold, the intranet may mature into a rich record that can be "excavated". in this manner key events could be "replayed" to (re)create knowledge by reflecting on previous solutions to problems, recurring problems, rationales for decisions etc. in comparison to having such information locked away in filing cabinets, the availability of a rich, electronically searchable record holds great potential in this regard. in one of our cases, a senior r&d program manager commented that they meticulously record all their project documentation, customer meetings, minutes, and their informal project notes and comments on the intranet. he elaborated: "(since the inception of the intranet-based project repository) i don't think i have ever asked a project leader what's going on in his project. the only papers that have arisen out of the project are the contracts that had to be signed. we just don't generate other paper around the project" [programme manager, january 1998] the recording use mode can capture a rich account of factors that may become valuable only in hindsight. for example, when carefully examining several projects ex post some pattern may emerge that could not be observed from within a project or observed by examining one project alone. although we have seen many examples of such diligent intranet recording, we have not yet seen a similar level of activity in "excavating" this record carefully. combination (facilitated by intranet searching) nonaka and konno posit that combination mode of knowledge creation relies on three processes. capturing and integrating new explicit knowledge is pivotal. this includes collecting externalised knowledge from inside or outside the organisation and then combining this to create new knowledge. the dissemination of explicit knowledge is based on the process of transferring this form of knowledge directly eg. by means of presentations or meetings. hereby, new knowledge becomes readily accessible to other members of the same social system. the systemisation of explicit knowledge involves the editing or processing of explicit knowledge into plans and reports to make it more useable. in the combination process, justification occurs allowing the organisation to formulate practical action steps. we see the primary mode of intranet use in the combination of knowledge as searching. intranet searching can take on a number of forms. a "flat search" could be to navigate from perhaps the intranet home page by following hyperlinks to the desired information (hills, 1997). this of course requires that the searcher knows what she is searching for and that an appropriate navigation path exists to this information. apart from intranet home pages, other intranet "portals" may be created to serve as a starting point or entrance for searches (bhattacherjee, 1998). an intranet portal can be an index or directory page, or an intranet search engine. for example, the marketing department may decide to create such an internal portal to various marketing-related information in the organisation. similarly, many other portals to the same information space can be set up (eg. with a customer, departmental, product or project view in mind). in addition to these searches, we may see even more ad10 aj1s special edition knowledge management december 2001 vanced searching on intranets (for example as agent technologies mature (caglayan & harrison, 1997)). a necessary prerequisite is the use of more powerful mark-up languages than basic html; here xml (extensible mark-up language) appears a promising candidate due to its powerful tagging feature and the useful separation of content and presentation. by combining codified knowledge from various repositories scattered throughout the intranet, new avenues are opened up for the creation of cross-functional knowledge that is required for example in process innovation and new product development (scacchi & noll, 1997; cecez-kecmanovic et al., 1999; mcnaughton et al., 1999). searching the organisational intranet helps to identify potential useful knowledge, which in combination with other knowledge might bring about valuable new knowledge. in this regard, a vice president in a large danish company outlined a typical problem that he expects their intranet will help alleviate: "people sitting only a few 100 meters away from each other do not know they are working on similar things ...1 am often amazed how much people re-invent, not because they want to but because they don't know it exists." [vice president, october 1998] the potential for creating new explicit knowledge by combining existing codified knowledge, indicates that it is certainly worthwhile for organisations to invest in advanced search features on their intranets. in our field study numerous users have criticised the lack of sophistication of their organisation's current intranet search functionality. internalisation (facilitated by intranet transaction) nonaka and konno argue that in practice internalisation relies on two dimensions. firstly, explicit knowledge has to be embodied in action and practice. thus the process of internalising explicit knowledge actualises concepts or methods about strategy, tactics, innovation or improvement. for example, training programs help the trainees to understand the organisation and their role in the whole. secondly, there is a process of embodying the explicit knowledge by using simulations or experiments to trigger learning-by-doing processes. new concepts or methods can thus be learned in virtual situations. we see the primary intranet use mode for internalisation of knowledge happening through transaction with intranetbased knowledge repositories. using html forms and relying on the standard internet protocols, many existing organisational computer-based information systems can be accessed via the intranet (provided of course that the necessary linkages are introduced, eg. by web-enabling "legacy systems" and other repositories) (ressler & trefzger, 1997). in the same manner, embedded intranet-based functionality in scripts and applets on home pages of various departments, groups and employees can be accessed. using the browser as a standard front-end, the intranet enables users (often for the first time) to gain direct access to systems and repositories of information located in other parts of the organisation. prior to intranet technology, such cross-functional information access typically required the user to master complex system interfaces or the intricacies of disparate systems. by having the ability to interpret information from various quarters in an organisation, even "non-technical" employees can effectively use the intranet as their own virtual learning environment. in this respect the intranet can especially enable the (new) employee to make sense of her surroundings. this is illustrated by the following: "(the intranet) is a nice place to find information and saves a lot of time. in my old job we never had something like that. as a new employee, i wanted to know what i can find out without asking too many questions" [intranet user, october 1998] in one company we studied, going through old project reports or the quality handbook that was made available on the intranet helped employees to gain access to organisational knowledge about what was considered best practice. in the same company we also found that pictures, positions, responsibilities and other information of employees that were made available on the intranet, helped others to familiarise themselves with peoples' appearance and backgrounds prior to meeting them in person. ajis special edition knowledge management december 2001 discussion in the following, we outline some aspects of our model in terms of its conceptualisation and application potential. behavioural aspects pertaining to intranet use are central to the model's application potential. in the following we list some intranet use patterns emerging from our field studies that we believe are conducive to the knowledge creation processes in the model we have proposed. unlike many information systems and technologies where the target users are often known, this is not necessarily the case with an intranet. to a large extent this means that the intranet user can read and inquire without being identified (ie. anonymous use). compare this to a telephone call or an email where the inquirer reveals her identity in the process of asking. in this respect, many employees reported to us that the intranet provided them with a very attractive alternative to learn about their working environments without running the risk of revealing their ignorance when asking about specific issues (such as the comment above from the new employee). a further behavioural aspect concerns the question whether individuals would voluntarily put information onto the intranet (alavi & leidner, 2001). indeed such an activity was reported as an add-on to an already busy schedule by some interviewees. however, we encountered numerous examples of intranet "exhibitionists" who would dedicate hours of extra work to publish information onto the intranet. instead of an add-on, these individuals viewed the intranet as a means to gain some organisational visibility and repute (similar findings are also reported by (davenport & prusak, 1998)). the following quotations from our field study illustrate this behaviour: "people are joking with me, but i put my name on all the intranet pages i create. i want to go further and get a post as a programmer so i can have a budget for software." [technical official, january 1998] "as secretary, you usually just type what other people think. now all of a sudden i have an identity of my own." [secretary and intranet content provider, august 1998] the exhibitionist behaviour supplements organisational initiatives for km such as incentive schemes or mandated intranet use (eg. requiring the establishment of a project website prior to resource release). we have also noticed some changes in information distribution and seeking behaviour in our cases. instead of sending out all information "just-in-case" people need it (and thus placing the onus upon interested recipients to maintain their own information repositories eg. e-mail folders, file systems, etc.), the intranet is beginning to evolve into a "definitive organisational record". this also fosters a "just-in-time" information seeking behaviour: "we create content agendas, minutes and background information and just park it on the intranet. we don't email it out to everyone anymore. it's their responsibility to go and fetch it." [quality and information manager, september 1997] although much of knowledge creation and exchange processes we indicate in our model occurs "in public" on the intranet within the organisation, this does not preclude private exchanges between communities within the organisation. in this respect we have noticed the use of security mechanisms such as passwords and even internal firewalls within the larger intranet to restrict access to sites focussing on sensitive or community-specific issues. the model suggests that the availability of rich intranet content is conducive to knowledge creation. however, content creation behaviour resulting in information overload and intranet "chaos" is detrimental to the process. these dangers are captured in the following quotations from our fieldwork: "the biggest challenge is when people search information on the intranet they get drowned in information how do you sort it?" [vice president, october 1998] ".. .we are drowning in information, but starved of knowledge." [it manager, october 1998] to foster knowledge creation effectively (especially the combination mode), organisations would need to revisit their intranet content generation and retrieval strategies. attention should be paid to effective intranet content generation (perhaps by laying down minimum content standards). at the same time effective search and indexing approaches should be explored. the emergence of intranet-enabled document management products, search agents, automated indexing of intranet content and concept-based retrieval technologies (eg. 80-20.com) hold promise in this regard. returning to the disappointing findings reported by some authors regarding the potential of intranets for km (as outlined in the introduction), we argue the following. as indicated by our model, organisations need to set their sights beyond pure publication if they seek to harness the full potential of intranets for knowledge creation. although pub12 ajis special edition knowledge management december 2001 lication can indeed be instrumental in facilitating knowledge distribution, we have argued that it is vital to integrate the other intranet use modes to foster knowledge creation processes. as such, interaction, transaction, recording and searching become key levers with regards to knowledge creation. due to the malleability of intranet technology, organisations need to carefully consider how their intranet should be deployed so as to reap the maximum benefit in terms of knowledge creation. we are witnessing a large number of kms applications being offered by third parties (compared to in-house development) (karlsbjerg & damsgaard, 2001). such applications can be purchased and installed on the company intranet or they might be offered and operated by a third party asp (application service provider). these applications are typically second or third generation kms tools focusing on advanced features such as context sensitive searching or virtual team support. what is common for all of them is that they piggyback on an already well-established and expanded intranet. they can only be added to a mature intranet where the level of implementation is well beyond creating a critical mass of users and content (damsgaard & scheepers, 2000). despite these tools' complexity, once reduced to their simple parts they can readily be housed within our taxonomy of intranet use modes. some of these tools encompass several of the use modes simultaneously. it should be emphasised that km is not at all limited to intranet technology, as only a fraction of km creating activities can and should be supported via technology. there are many opportunities for developing web based km tools in other contexts. one promising avenue for researching critical km technology application currently not well understood is informal know-how trading among peers (even across competing firms) (von hippel, 1987). in this respect, further research is necessary to understand how intranets will merge with other distinct technologies (eg. extranets, erp, and internet) in forming ubiquitous (wireless) computing environments in the near future. one particular challenge here is that different technical interfaces (monitors, televisions, mobile devices) call for completely different presentations of the knowledge domain. a further challenge is how a particular knowledge domain, captured by a web of information sources on an intranet, could be represented on these platforms. the context free hypertext format (html based) that currently makes up the majority of existing intranets is not flexible enough to allow for such representation. xml is a promising improvement over html, but we expect that xml will have to fight an uphill battle in substituting html in the intranet context. the co-existence of both standards nullifies many of the benefits that the enhanced capabilities of xml offer. again, it seems that (as in many other cases) the first technology to attract a critical mass of usage sets the stage for subsequent technology innovation. our intention here was not to depict some universal intranet knowledge creation model that can be approximated to fit any organisation and intranet application. rather, we have attempted to establish a vocabulary for intranet managers and implementers to debate and reflect upon their intranet progress and to formulate development and implementation strategies in the light of knowledge creation processes. accordingly, we believe that when assessing the proposed model it is crucial to evaluate its application ability as experienced by managers and practitioners, instead of pursuing some rigid theoretical approach to validate the model itself. conclusion in this paper we have proposed a model for how intranets may be applied to create knowledge. our model is based on nonaka's well-known 2-by-2 matrix of knowledge creation and a taxonomy of intranet usage modes. for each of the four knowledge creating activities, we have associated the corresponding primary intranet use mode that is conducive to knowledge creation and we have described popular applications that we believe can foster the knowledge conversion processes. we attribute many of the early disappointing research findings on intranet-supported km processes to a fixation on publication as the primary intranet use mode and a failure to exploit other use modes of the technology. computerbased information sharing alone is not sufficient for knowledge creation; even worse, this may exacerbate the information overload occurring in many organisations today. we argue that it is only by purposefully combining intranet publication with the interaction, transaction, searching and recording use modes, that knowledge conversion processes can effectively be fostered between knowers in the organisation. our approach here was to propose and illustrate a model for knowledge creation fostered by intranet technology. we did not attempt to empirically validate the model here and therefore future research should test and refine this model empirically, but also in terms of its practical value in supporting intranet and knowledge managers with their intranet strategy formulation. we are currently engaged in such follow-up research in denmark and australia. 13 ajis special edition knowledge management december 2001 acknowledgements an earlier version of this paper was presented at the 9th european conference on information systems in bled, slovenia. the research was supported in part by funding from the danish research agency under the pitnit project grant number 9900102. references alavi, m. (1997). kpmg peat marwick u.s.: one giant brain, case 9-397-108: harvard business school. alavi, m., & leidner, d. e. (1999). knowledge management systems: issues, challenges, and benefits. communications of the association for information systems, 1 (article 7). alavi, m., & leidner, d. e. (2001). review: knowledge management and knowledge management systems: conceptual foundations and research issues. mis quarterly, 25 (1), 107-136. attewell, p. (1992). technology diffusion and organizational learning: the case of business computing. organization science, 5(1), 1-19. bansler, j. p., damsgaard, j., scheepers, r., havn, e., & thommesen, j. (2000). corporate intranet implementation: managing emergent technologies and organizational practices. journal of the association of information systems, 1 (article 10). bernard, r. (1996). the corporate intranet. new york: john wiley & sons. bhattacherjee, a. (1998). management of emerging technologies: experiences and lessons learned at us west. information & management, 33, 263-272. boehm, b. (1988). a spiral model for software development and enhancement. computer, 21 (5), 61-72. caglayan, a., & harrison, c. (1997). agent sourcebook. new york: john wiley & sons. cecez-kecmanovic, d., moodie, d., busuttil, a., & plesman, f. (1999). organizational change mediated by e-mail and intranet an ethnographic study. information, technology & people, 12 (1), 9-26. chellapa, r., barua, a., & whinston, a. b. (1997). intranets: looking beyond internal corporate web servers. in kalakota, r. & whinston, a. b. (eds.), readings in electronic commerce (pp. 311-321). reading, massachusetts: addison-wesley. ciborra, c. u. (ed.). (1996). groupware & teamwork. new york: john wiley & sons. ciborra, c. u., & hanseth, o. (1998). from tool to gestell: agendas for managing the information infrastructure. information, technology & people, 11 (4), 305-327. coleman, d. (1997). collaboration on the internet and intranets. in sprague, r. h. (ed.), proceedings of the 30th hawaii international conference on system sciences, maui, hawaii, 350-358. damsgaard, j., & scheepers, r. (1999). power, influence and intranet implementation: a safari of south african organizations. information, technology & people, 12 (4), 333-358. damsgaard, j., & scheepers, r. (2000). managing the crises in intranet implementation: a stage model. information systems journal, 70(2), 131-149. davenport, t. h., jarvenpaa, s. l., & beers, m. c. (1996). improving knowledge work processes. sloan management review (summer 1996), 53-65. davenport, t. h., & prusak, l. (1998). working knowledge: how organizations manage what they know. boston, massachusetts: harvard business school press. david, p., cowan, r., & foray, d. (1999). the explicit economics of knowledge codification and tacitness. research memoranda 027. maastricht: merit, maastricht economic research institute on innovation and technology. ezingeard, j.-n., leigh, s., & chandler-wilde, r. (2000). knowledge management at ernst & young uk: getting value through knowledge flows. in orlikowski, w., ang, s., weill, p., krcmar, h. & degross, j. i. (eds.), proceedings of the 21st international conference on information systems, brisbane, australia. gottschalk, p. (2000). strategic knowledge networks: the case of it support for eurojuris law firms in norway. international review of law, computer & technology, 14 (10), 115-130. hahn, j., & subramani, m. r. (2000). a framework of knowledge management systems: issues and challenges for theory and practice. in orlikowski, w., ang, s., weill, p., krcmar, h. & degross, j. i. (eds.), proceedings of the 21st international conference on information systems, brisbane, australia. hills, m. (1997). intranet business strategies. new york: john wiley and sons. 14 ajis special edition knowledge management december 2001 huber, g. p. (1991). organizational learning: the contributing processes and the literatures. organization science, 2 (1), 88-114. hughes, t. (1987). the evolution of large technological systems. in bijker, w., hughes, t. & pinch, t. (eds.), the social construction of technological systems (pp. 51-82). cambridge, massachusetts: the mit press. karlsbjerg, j., & damsgaard, j. (2001). make or buy a taxonomy of intranet implementation strategies, proceedings of the 9th european conference on information systems, bled, slovenia, 579-592. kling, r. (1980). social analyses of computing: theoretical perspectives in recent empirical research. computing surveys, 12 (1), 61-110. lamb, r., & davidson, e. (2000). the new computing archipelago: intranet islands of practice. in baskerville, r., stage, j. & degross, j. i. (eds.), proceedings of the ifip 8.2 working conference, organizational and social perspectives on information technology (pp. 255-274): kluwer academic publishers. laudon, k. c., & laudon, j. p. (2000). management information systems: organization and technology in the networked enterprise (sixth ed.). upper saddle river, new jersey: prentice-hall, inc. lyytinen, k., & damsgaard, j. (2001). what's wrong with the diffusion of innovation theory? the case of a complex and networked technology, proceedings of the ifip tc8 working group 8.6 conference, diffusing software product and process innovations, banff, canada, 173-190. lyytinen, k., rose, g., & welke, r. (1998). the brave new world of development in the internetwork computing architecture (internca): or how distributed computing platforms will change systems development. information systems journal, 8, 241-253. mathiassen, l., & stage, j. (1992). the principle of limited reduction in software design. information technology & people, 171-185. mcnaughton, r. b., quickenden, p., matear, s., & gray, b. (1999). intranet adoption and inter-functional coordination. journal of marketing management, 15, 387-403. newell, s., swan, j., galliers, r., & scarbrough, h. (1999). the intranet as a knowledge management tool? creating new electronic fences. in khosrowpour, m. (ed.), proceedings of the information resources management association international conference, managing information technology resources in organizations in the next millennium, hershey pa, usa. ngwenyama, o. k., & lee, a. s. (1997). communication richness in electronic mail: critical social theory and contextuality of meaning. mis quarterly, 21 (2), 145-167. nonaka, i. (1994). a dynamic theory of organizational knowledge creation. organization science, 5(1), 14-37. nonaka, i., & konno, n. (1998). the concept of "ba": building a foundation for knowledge creation. california management review, 40 (3), 40-54. oppliger, r. (1997). internet security: firewalls and beyond. communications of the acm, 40 (5), 92-102. ressler, s., & trefzger, b. (1997). the development of the nist virtual library. ieee internet computing, 35-41. romm, c. t., & wong, j. (1998). the dynamics of establishing organizational web sites: some puzzling findings. australian journal of information systems, 5 (2), 60-68. scacchi, w., & noll, j. (1997). process-driven intranets: life-cycle support for process reengineering. ieee internet computing, 42-49. scheepers, r., & damsgaard, j. (1997). using internet technology within the organization: a structurational analysis of intranets. in hayne, s. c. & prinz, w. (eds.), proceedings of the international acm siggroup conference of supporting group work, phoenix, arizona, 9-18. schein, e. h. (1996). three cultures of management: the key to organizational learning. sloan management review (fall 1996), 9-20. scherer, a. (1997). supporting concurrent engineering using an intranet approach. in sprague, r. h. (ed.), proceedings of the 30th hawaii international conference on system sciences, maui, hawaii. scott, j. e. (1998). organizational knowledge and the intranet. decision support systems, 23, 3-17. senge, p., m. (1990). the fifth discipline: the art and practice of the learning organization. sydney: random house. spiegler, i. (2000). knowledge management: a new idea or a recycled concept? communications of the association for information systems, 3 (article 14). sviokla, j. j. (1996). knowledge workers and radically new technology. sloan management review (summer 1996), 25-40. von hippel, e. (1987). cooperation between rivals: informal know-how trading. research policy, 16, 291-302. wiig, k. m. (1993). knowledge management foundations: how people and organizations create, represent and use knowledge. arlington, texas: schema press ltd. williams, r., & edge, d. (1996). the social shaping of technology. research policy (25), 865-899. 15 microsoft word ajis 11 1 final.doc ajis vol.11 no. 1 september 2003 12 the management of intuition john d haynes management information systems department college of business administration university of central florida, orlando, usa email: jhaynes@bus.ucf.edu abstract human intuition and the creativity afforded by that intuition is an aspect of the essential human-ness of any human being. heidegger refers to this essential humanness as dasein. collectively and individually therefore, dasein and from that, intuition should be an organization’s greatest asset. it is accordingly, at least initially, difficult to understand why the management of almost all organizations do not pay more attention to the development of this asset. this paper examines a phenomenological approach to managing intuition. in particular, the paper examines the nature of heidegger’s notion of ding-an-sich (the thing-in-itself), a grasp of which, it is argued, is essential in coming to terms with the human capacity for intuition and creativity. the paper also offers a number of possibilities for managing intuition under the general heading of thematic management. keywords: thing, thing-in-itself (ding-an-sich), intuition, phenomenology, dasein, reflexivity, subjectivity, objectivity, thematic management. introduction this paper is an attempt to focus on the hidden aspects of intuition and creativity and not treat them, or worse their creators, as “information packets” to be managed, but rather as crucial human assets to be nurtured and cared for. in many respects the proper care of intuition and creativity is seeing their possibilities in the context of an organization and guiding that transition from implicit conception to explicit recognition as a perceivable product. but humans who are (considerably) creative cannot always make the organizational contextual connection themselves, and in that sense intuition and creativity in an organizational context cannot be successfully self-managed. it is from this perspective that this paper is seeded. there needs to be a type of management that “contextually sees the possibilities of creativity and intuition” without at the same time interfering with the process of intuition and creativity. to do that this type of management needs to comprehend both the process of intuition and creativity as a thing-in-itself and to come to terms with the fact that this process is implicit or hidden, and at the same time recognize that this process must thematically connect with the goals and aspirations of the organization. thematic management the type of management we speak of above we refer to, in this paper, as thematic management precisely because that type of management needs to take the journey into what constitutes a thingin-itself and to relate that understanding thematically to the organization. in order to achieve that goal more precisely, we restrict ourselves, in this paper to heideggerian phenomenology and thereby lead the way toward a thematic management approach. in seeing this connection management discovers new possibilities. more particularly that kind of management shifts from perceiving (and manipulating) things to seeing the possibilities inherent in the conception of things-in-themselves. lucas introna [introna, 2002, p 224] comments that “heidegger refers to the work of the artisan [to make this point abundantly clear]. the artisan did not make or manufacture ‘things’ as such. the relationship between the skills of the artisan and the material being transformed was one in which the artisan drew on his tacit understanding of the world – the referential whole – of the ‘thing’ being made”. the referential whole is a thing-in-itself and its relation to the ‘thing’ being made is thematic. this is precisely why “theme” and thematic” is of importance in this paper, because we are linking together ajis vol.11 no. 1 september 2003 13 factor (thing) and process (thing-in-itself). we cannot link them directly, that would be to commit a category mistake, so we do so, just as the artisan does, indirectly, by gleaning the theme inherent in the thing-in-itself and linking that to what is thematic in the thing in question. the artisan intuits the theme as pure design (thing-in-itself) and makes the recognition of that thematic as an art product (thing). but just as the artisan expresses him or herself in this way, so too could management, provided, of course, that the management in question comprehends the crucial relation between thing as factor and thing-in-itself as process. in terms of thing as factor (fragment) we could say that this is organizational perceiving, and in terms of thing-in-itself as process (wholes) we could say that this is organizational conceiving. management needs to clearly see the relation between perceiving and conceiving in order to apply a “thematic approach” to the organization. why? because a theme of some work or process arises out of the reconciliation of the composition of the work or process as part and whole. this reconciliation is necessary to form a synthesis – a general grasp. on the basis of comments by david bohm [bohm, 1980, p 3] thinking in terms of referential wholes is what mankind has moved away from, to, generally, a state of fragmentation. consider specifically bohm’s point as follows: “being guided by a fragmentary world-view, man then acts in such a way as to try to break himself and the world up, so that all seems to correspond to his way of thinking. man thus obtains an apparent proof of the correctness of his fragmentary self-world view though, of course, he overlooks the fact that it is he himself, acting according to his mode of thought, who has brought about the fragmentation”. consider this fragmentation from the perspective of the development of the machine (computer). it is so easy for management to relinquish control to the computer, or worse “the instrumentality of the machine appears to transform the entire universe into a means to humankind’s own ends. but in this moment of apparent triumph, the will to mastery turns on itself and becomes self-destructive” [taylor, 2001, p 83]. if mankind continues to develop ways of management that lack a sense of intuition, then in the end our very capacity for intuition will begin to diminish. the notion of theme and its connection to intuition in history of the concept of time, [heidegger, 1992, p 99] heidegger says, “every phenomenological analysis of acts considers the act in such a way that the analysis does not really go along with the act, does not follow its thematic sense, but rather makes the act itself a theme”. this notion of making the act itself a theme is precisely why opportunities are open for all who adopt this phenomenological approach. they are open because in seeing the theme in the act itself we are availing ourselves of our intuition. on the one hand we have the ‘thingness’ of the act (that is the object component in the act; the measurable aspect of the act) and on the other hand, its referential connection to, or context with, the rest of the world, either limited in terms of the organization or in a wider viewing. but what allows for a thematic viewing, what discloses itself for that viewing, is the character or essence of the act, it is the thing-in-itself of that act. seeing that thematic connection, by having a sense of the thing-in-itself, is invoking intuition. heidegger’s dasein dasein, heidegger’s special term for a human being’s essential human-ness or “spark of spirit” [caputo, 1987, p 157], which belongs specifically to humans, is indicative or expressive of a human thing-in-itself. the human body or content is its thing-ness and dasein is indicative of its thing-initself. our human-ness is made expressible through our body and our thoughts. but our body and our thoughts are things, whereas our human-ness is the ‘form’ in which our essence as a human being expresses itself via the content of the human body and human thoughts. things are often seen in terms of content in space and time and things-in-themselves are seen in terms of form or concept (neither of which have spatial or temporal characteristics, yet they both ‘appear’ in space and time). following this view, some things have content as well as form, for example humans. some things have content but have no form, for example tables. ‘form’ is not viewed here (in this paper) in terms of space and time, but rather in a similar way in which heidegger treats dasein (a term he himself coined). while dasein (form) applies specifically to the content of humans, we can certainly conceive of a form that is not dasein. ajis vol.11 no. 1 september 2003 14 the point that phenomenology takes at issue with other philosophies is that phenomenology tries to articulate the essence (the thing-in-itself) in things. if, of course, the ‘thing’ in question possesses one, see in particular heidegger [heidegger, 1967]. insofar as phenomenology does this it is interested in the phenomenon itself rather than the word, or label, that points to it. in much the same way as heidegger is interested in dasein (not the word, but the phenomenon that it points to, i.e., human-ness) rather than which type of human, man or woman, who possesses it. therefore heidegger is being considerably more interested in the phenomenon that its definition. definitions for heidegger obscure our openness to the phenomenon, a phenomenon that may be elusive, but nevertheless is able to be grasped conceptually. moreover, in all of heidegger’s (considerably prodigious) work he never mentions either of the words ‘man’ or ‘woman’. this was no oversight. he replaced the word ‘man’ and ‘woman’ with dasein. heidegger coined the word ‘dasein’ to describe a human’s essential human-ness. one clear reason for the introduction of dasein is the provision for the shift from the need to define things-inthemselves. indeed, he wanted to get away from the need to define any thing or any ideal, rather he chose to reveal. more particularly the subjective side of humans, as reflected in their feelings and their thoughts is implicit and indicative of implicit knowledge which is not, as such, definable, and is often unintelligible, anticipating some future explicit action or destination in intelligible explicit knowledge. it might of course seem questionable whether revelation in the heideggerian sense could actually be achieved when starting from a concept such as the thing-in-itself. he is, after all, introducing severe constraints on objectivity. heidegger actually declares objective knowledge illusory. the confusion dissolves through recognition of heidegger's primary target. he proposes to aid in revealing dasein as a thing-in-itself. it is first and foremost dasein which cannot be defined as a closure but should instead be appreciated as revealing opening. the notion of intuition the acceptance of the ‘form’ or the concept or the idea of being (as distinct from the thing-ness or content of being or being-in-the-world) is an example of the implicit nature of intuition. almost all of heidegger’s work is the attempt to reveal, and render in all its beauty, the form or idea of being in its relation to being. he states again and again that our being shines forth in the (good) actions of our being or being-in-the-world. it is evident, also from heidegger's own actions, for that matter, that good-ness is not intrinsic. introducing value, i.e. good-ness, turns his program of revelation into an ideology. the intensity of heidegger only confirms that dasein cannot escape ideological revelation. existence always involves responsibility. man's responsibility is grounded in his intuition. intuition is not a thing and is therefore not able to be perceived as such. the process of intuition is hidden from our sense perceptions. generally at the level of perception things are open to perception (by which they may, or may not, be perceived), but, importantly, we are not able to perceive an idea or concept – simply because neither have any identifiable ‘thingness’ about them to enable them to be perceived. however, in the mode of conception, the invisibility of ideas, (or concepts, or form) provides the ground from which we can conceive such ideas, concepts, form. another phenomenological (albeit generically) writer of note on this general area of tacit or implicit knowledge (but beyond the scope of this particular paper) is michael polanyi [polanyi, 1967]. the mode of conception and the level of perception the notion of conception itself is an idea hidden from dasein's perception and can only be comprehended in terms of its own hidden process of coming-into-being. in this sense conception is its own becoming, insofar as the idea that is conceived is its being conceived. that is how we come to know the idea, by conceiving of it, including conception itself. what emerges from that hiddenness is the manifest idea, the resultant knowing that has transpired in that implicitness that (the level of) perception cannot comprehend. as indicated in an earlier section, organizational perceiving relates to things as factors (as fragments), and organizational conceiving relates to thingsin-themselves as process (wholes). it follows then that management needs to clearly see the crucial ajis vol.11 no. 1 september 2003 15 relation between perceiving and conceiving in order to apply a “thematic approach” to the organization. the shift to conception the way in which we conceive any idea or concept is the same thematic way in which we conceive of the idea of conception itself. this is known as reflexivity: the way in which phenomenologically we articulate thought in general is the same way-in-process in which we articulate phenomenology (the process) itself. and this conception of conception itself is at one with itself. it should be noted that interpretations, for example, are characteristic of the level of perception, but interpretations are absent from conception, there is no ‘thing’ to interpret. the shift to conception, before conception is reached, can be regarded as a perception-in-gestalt as haynes [haynes, 1999, p 17] indicates: that bird’s eye view or perception-in-gestalt is a mode that diffuses, like conceiving taken as a mode, a multiplicity of so-called interpretations into a modal shift. if one or the many so-called interpretations do not conform to a conception, i.e., leaving behind the perceiving of the [words as] objects or things, then we have a so-called interpretation, which is still operating at the level of perceiving. but if we consider the case of an interpretation that is conceived, then we have made one modal shift to that of conception. .. this arises out of the experience of undergoing the perception-in-gestalt. so there are three distinct stages in the comprehension of a central idea .. (1) perception, moving to (2) perception-in-gestalt, shifting to (3) conception. from the above quote we can see that reflexivity, like conception itself, is a three-step process. initially we have an interpretation at the level of perception, then we have a reflection, of seeing the duality of things, the beginning of the gestalt of things, until we move to the ‘hiddenness’ of conception. out of the darkness of conception comes a new light, a light to illuminate what has been conceived, a light that turns the implicit knowledge of conception into the explicit knowledge of the product of conception (an objectified thing or subjective knowledge of some type). the case in point in this paper is that the intuition inherent in conception leads to an outcome that is contextually accurate, not measurably accurate. heidegger’s comment “inquiry, as a kind of seeking, must be guided beforehand by what is sought, so the meaning of being must already be available to us in some way” [heidegger, 1987, p 24] is of importance here. it points to a resolution of phenomenology with its lived intuitional experience in being-in-the-world, which is its so-called pragmatic consequence. intuition's pragmatic, or behavioural, import comes into play through perspective. thus the concept of essence shifts to become perspectival, implying a departure from the heideggerian thing-in-itself to the notion of intuition that is able to satisfy heidegger’s requirement of having “the meaning of being already available to us” (that is intuitionally). hirschheim, klein and lyytinen’s [hirschheim, klein and lyttinen, 1995, p 62] also recognize the need for such a shift in their question applied to the reconciliation of subjectivity and objectivity. the question hirschheim et al pose is “is it possible to conceive of a third perspective on data modeling which combines both objectivist and subjectivist principles?” the solution they explain is in terms of the actor’s imagination. at some point that imagination must be intuitional, that is, at some point it must be able to imagine itself, which entails that the object of imagination is itself the subject of imagination, and in that sense it is “already available” to itself. the management of implicit intuition: thematic management the upshot of the forgoing is that in taking a phenomenological approach we look for evidence of things and things-in-themselves, rather than attempt to define such apparent ‘structures’. in seeking evidence we need to experience and/or re-experience what is in view, until the ‘evidence’ co-incides with our own experiences. then we experience what is really (in the deeper sense) in view for the ‘first time’ (of perhaps many times). we experience the mode of meaningful intuition that arises out of conception, and we experience the level of significant intuition (the end result or product of ajis vol.11 no. 1 september 2003 16 implicit intuition) that arises out of perception. what is implicit in meaningful intuition may eventually be made explicit as a product in significant intuition. at some stage in early history the invention of the wheel was implicit intuitionally in some human’s mind, now it is generally rendered as a significant product of that intuition, but that is not to say that one cannot take the journey again to re-experience it as implicit intuition, and further, to conception itself. in order to do that we need (again, like the original conceiver) to see stones not as objects but as possibilities for being “wheels”. there is a continual potential cycle from implicit intuition to its own counterpart as an explicit product of intuition and back again to implicit intuition. what is emerging from our discussion is a theme. it is the theme of the recognition of the journey, the process, the becoming, that takes place in the movement of the implicit to the explicit and back again to the implicit. this is the same journey that takes place in the recognition of a work of art, of music, of mathematics, of philosophy, of management. at some point the “experiencer” sees (or many experiencers see) that work for the very first time. in that experience of seeing the ‘artistry’ of a work of art, in that becoming, is the seed of the process of conception, and hence, potentially, the movement toward the ‘centre’ of creativity itself. in that potential is the distinct possibility of the centre of intuitional creativity because what is being conceived is conception itself. we see the work in its being conceived. the fact that someone else conceived it in its original ‘form’ before us is, of course the greater conception, but it nevertheless provides us with a crucial experience that when explored allows us to see not only the thing in itself (that which is really viewed) but the process of conception itself, the becoming itself, that sees it. with the discovery of the process that sees conception comes the realization of the need to care for and preserve opportunities for conception itself. the process of conceiving cannot, in the long run, be self-managed. what manages this implicit intuition is a type of management that recognizes the crucial difference between a thing and a thing-in-itself. the becoming of creativity in conception is a thing-in-itself. it is not a structure that can be defined. it is a reflexive process that can only be experienced. heidegger’s “the essence of technology is not itself technological .. technology is a way of revealing” [heidegger, 1977, p 4] is of importance here. the essence of reflexivity is not itself reflexive (there is no logical mechanism in ‘there’ doing it), but rather, it is a becoming (a nonlogical way) of itself as reflexive. its essence then is that (uniquely instantiated) becoming, and that (particular) becoming can be grasped in thought as a thematic movement, and with further reflection to discover that there is a one-ness about this becoming in general. we see that theme in the case of reflexivity itself as we see it elsewhere. we take the next step to a general thematic movement, when 'becoming' becomes its own sign. a sign that so many good thinkers come to know. the good thinker who is asked 'what is your identity?' does not turn to his occupation, or to his name badge or his social security number, but to the fruits of his reflexivity of himself. there may not be an answer that can be logically or rationally created in order to be communicated to others, but there is a response to the question itself, a deep response that allows the question to be (to let it be), and to come into being in all its myriad forms to be asked again and again. in that asking; in that exploration, creativity expresses itself. and that creativity requires genuine management. stanislav grof [grof, 1990, p 19] suggests, in another context, namely psychology, that states of consciousness such as reflexivity, which he incorporates as part of a third level of consciousness generally under the umbrella of the holotropic mind, suggests that we need to “develop brand new research projects, exploratory tools, and methodologies for discovering the deepest nature of the human psyche” in relation to human capacities such as reflexivity. but in the absence of those new techniques and in the absence of thematic considerations management will try to freeze that shift, to grasp it as a structure, to define it and to measure it. this is the whole problem with definitions. definitions freeze such a shift, and in freezing that which is so defined, dissolves that which is defined in its essential idea-hood; in its articulation in thought as an idea; in its grasping. so how could a management of implicit intuition proceed? such a management needs to recognize the crucial difference between a thing and a thing-in-itself. out of this recognition comes a need to break an organization into aspects that allow for these differences. the conceivers and the perceivers could constitute the major division, but such a division need not be categorical. by this i mean that the division itself could be thematic; it could be two thematic organizational aspects. all humans ajis vol.11 no. 1 september 2003 17 both conceive and perceive. some choose to stay at the level of perception. but they often stay there unwittingly. the management could attempt to emulate the process itself of conception by rotating the organizational members from in and out of organizational aspects of conceiving and perceiving. what could constitute such aspects between conceiving and perceiving in an organization? that would be apparent given the context of the organization in question. in this paper we have referred to this type of management as thematic management. as conception often brings together apparently contradictory elements, so too would thematic management. in any conception questions of alternatives are, if the conception succeeds, resolved. at the level of perception alternatives abound, hence, at that level, we have many interpretations, and, such interpretations also entail many viewpoints that apparently contradict each other. thematic management is in a position to resolve those differing interpretations. consider for example, the notion of ‘creative’ errors: errors that lead to a new discovery. if a strictly perception level approach to management were adopted the possibility of reflection upon an error of this kind would largely be ruled out, but if a sense of discovery inherent in the mode of conception approach to management flourished in an organization, then it is very likely that the ‘error’ in question would lead to an investigation and exploration that in turn may lead to a new discovery. and this, in turn, is a nurturing management approach to intuition. consider another example. apparently, “nothing succeeds like success”, but if the succeeding is in an area that the person has no real competence in, then the success will sooner or later sour into failure, and often, if the position “succeeded to” is a prominent one in the organization, then that eventual failure is often at the expense incurred from the departures of many good and valued employees. thematic management would largely preclude such a possibility, because the enthusiasm of discovery inherent in a culture of conception arises from a competence in what is being done, and so in this scenario incompetence is, or should be, difficult to disguise. conclusion we have seen how the distinction between a thing (ding) and a thing-in-itself (ding-an-sich) is essential to phenomenology, and therefore, in part, to coming to terms with intuition. we have seen how a thing-in-itself is not a structure with spatial and temporal characteristics yet it makes its presence felt it reveals itself in time and space. we extend this very notion of thing-in-itself and in extending it experientially realize that it is implicit intuition. from an exploration of implicit intuition we come to realize that it is conception that allows us to conceive of itself as conception and conceive of the presence, hidden though it is, of intuition. to know conception we must experience conception: what is conceived is its being conceived. in the absence of that knowing we are bereft of a most crucial human quality of mind: intuition and the creativity that follows. finally, we compared the mode of conception with the level of perception and arrived at a strategy for managing implicit intuition namely thematic management. references bohm, david (1980) wholeness and the implicate order, routledge, london and new york. caputo, john d. (1986) the mystical element in heidegger’s thought, fordham university press, new york, usa. grof, stanislav the holotropic mind – three levels of human consciousness and how they shape our lives, harper books, san francisco, usa haynes, john d, (1999) meaning as perspective: the contragram, thisone and company pty ltd, new zealand. heidegger, m. (1967) what is a thing? english translation by w.b. barton jr. and v. deutch. henry regnery and co. chicago. heidegger, m. (1977) the question concerning technology and other essays, translated by w lovitt, garland publishing, inc, new york. heidegger, m. (1987) being and time, translated by macquarie, j. & robinson, e., basil blackwell. heidegger, m. (1992) history of the concept of time, translated by theodore kisiel, indiana university press, usa.. ajis vol.11 no. 1 september 2003 18 hirschheim,r.a., klein, h.k and lyytinen, k. (1995) information systems development and data modeling: conceptual and philosophical foundations, cambridge university press. introna, lucas, (2002) “the question concerning information technology: thinking with heidegger on the essence of information technology”, chapter xiv, in haynes john d (editor) internet management issues: a global perspective, idea group publishing, usa. polanyi, m. (1967) the tacit dimension. anchor books, double day and company, garden city, new york. taylor, mark. c, (2001) the moment of complexity – emerging network culture, the university of chicago press, usa. microsoft word ajis10-2finala.doc ajis vol. 10 no. 2 may 2003 23 market makers’ recognition of key success factors in electronic marketplaces rosemary stockdale r.stockdale@ecu.edu.au craig standing c.standing@ecu.edu.au school of management information systems edith cowan university joondalup western australia abstract this study examines the recognition and use of critical success factors by market makers in electronic marketplaces. a content analysis of e-marketplace websites enabled an examination of how these factors have been incorporated into marketplace sites. evidence of market makers’ awareness of the success factors was found in all the sites although there remain questions and issues to be addressed. awareness of the need for critical mass and privacy were very evident, but the key factors of security, technological infrastructure and neutrality were identified as areas of concern. evidence of an awareness of the importance of trust by market makers was found, but more effective signalling of trust to buyers and sellers within the marketplaces is required. introduction the development of internet-based electronic commerce has led to great changes in the way business to business commerce has been conducted over the last few years. the internet has enabled businesses to connect with each other in a way that requires new thinking in every area of commerce, requiring greater flexibility and speed in decision-making, responsiveness to customers and ability to assimilate new learning (downes & mui, 1998). this is particularly true with the development of electronic marketplaces and the impact these are having on the complex area of supply chains. e-marketplaces have been developed in virtually every manufacturing and service industry, and market makers offer participants reduced transaction costs and the ability to buy and sell a vast variety of goods and services to a wider array of potential customers than ever before. the proliferation of electronic marketplaces, holding the promise of extensive benefits, has been rapid and widespread with an estimation of over 1000 global marketplaces in existence (hurwitz, 2000). however, the number of existing e-marketplaces cannot be sustained and the forecasts of considerable consolidation taking place in the next two to three years (forrester research, 2000) are becoming evident. these rapid changes in the marketplace environment raise the question of how participants might anticipate which market makers are likely to survive and for what reasons, requiring some measures of success to be recognised. against a background of consolidation and change, this paper uses the critical success factors of emarketplaces that have previously been identified from the literature (standing, 2001) and applies them to 12 mini-cases. this enables an examination of how these factors have been incorporated into marketplace sites. by gaining empirical evidence of market makers’ recognition of these critical success factors, it has been possible to highlight questions and issues that remain to be addressed in the electronic marketplace environment. electronic marketplaces in 1987, malone et al. predicted the increased use of electronic markets allowing some firms to become ‘market makers’ in the electronic environment. this development is well underway, but the profusion of electronic marketplaces, the speed of transition to the electronic environment, the variety of business models and the varying requirements of different industries and service sectors cloud an already confused marketplace picture. there is no consensus on how the marketplaces will develop, beyond that it is hard to predict their impact. business publications tend towards an optimistic view of e-marketplace development (downes & mui, 1998; modahl, 2000; raisch, 2001; sculley & woods, 2001), which is more cautiously supported in academic papers (bakos, 1998; ordanini & pol, 2001; senn, 2000) with some dissension on both sides (friesen, 2000; wise & morrison, 2000). despite the uncertainty there is immense pressure on companies, not least from their shareholders and directors (banham, 2000), to move quickly to the electronic marketplace. this they often do without a full understanding of what benefits they can accrue from participation and how they should recognise the factors that contribute to the development and success of e-marketplaces. this section proceeds ajis vol. 10 no. 2 may 2003 24 with an overview of electronic marketplaces to provide a context against which to discuss the key success factors. what is an electronic marketplace? there are several definitions of an e-marketplace from the broad description offered by malone of electronic markets as ‘information-technology-based governance mechanisms’ (malone, yates, & benjamin, 1987) to the more dynamic ‘interactive business communities providing a central market space where multiple companies can engage in b2b e-commerce and/or other e-business activities’ (brunn, jensen, & skovgaard, 2002). the latter definition takes into account the multiple stakeholders involved in an e-marketplace and notes that a range of activities beyond buying and selling can take place. descriptions of e-marketplaces as either horizontal or vertical offer an uncomplicated differentiation, although this distinction is rapidly blurring. for example some vertical marketplaces such as quadrem (http://www.quadrem.com) have moved towards offering ‘a complete solution’ to procurement needs, while horizontal marketplaces such as freemarkets (http://www.freemarkets.com) enable the purchase of industry specific goods. no coordinated business model exists for e-marketplaces, with different models evolving according to the type of ownership and levels of service offered. ownership the market makers many companies formed e-marketplaces to gain early mover advantage and to establish a critical mass of buyers and sellers to maintain a competitive edge in an increasingly fluid market environment (bakos, 1991). many of the early market makers were either intermediary companies, financed by venture capital, such as freemarkets (http://www.freemarkets.com), or individual, technology companies such as ibm (http://www.ibm.com). more recently there has been an increase in industry consortia marketplaces, resulting from unprecedented cooperation between rival companies; for example the creation of covisint (http://www.covisint.com) by a consortia of automobile companies including general motors, ford and daimlerchrysler. other sites have developed from associations between intermediaries, technology companies and industry participants such as egreencoffee (http://www.egreencoffee.com). the trend towards more industry coalition e-marketplaces is affecting the survival of some of the earlier intermediary businesses (spiegel, 2001) and leading the latter to re-invent their presence in the market either as service companies or consortia partners. pressure has also been caused by the unsustainable proliferation of e-marketplaces; for example over 17 major e-marketplaces were identified in the mining industry in january 2001 (ludeman, 2001) and 28 in the forest and building products industry (industry canada, 2000). while monopolies of marketplaces in individual industries would not be tolerated by governments, an optimal number of sites is likely to emerge from the current situation. the projection that less than 200 emarketplaces would survive within two to three years (forrester research, 2000) is becoming a reality. this raises the question of which e-marketplaces will survive and what criteria will lead to the success of the survivors as well as what issues will arise from this consolidation affecting both market makers and their customers. what do electronic marketplaces offer? previous studies have identified the different levels of service that are offered by e-marketplaces, developing from trading hubs that support the identification of potential trading partners to more complex models offering selection and, increasingly, execution services (choudhury and hartzel, 1998). models that have developed offer the following methods of trading: � catalogues either individual vendor or multi-vendor � auctions buyer or seller driven � exchange comparable to trading exchanges with a bid and ask system � negotiation the marketplace acts as intermediary for transactions such as requests for quotes (rfqs) in addition, e-marketplaces may offer the option of hosted storefronts where participants maintain an open webpage within the marketplace, offering easy access to other participating buyers and sellers. the potential for value-add services is an area that has not yet been fully exploited by market makers, but as technological capabilities develop, the ability to offer more of these services is becoming widespread. valueadd capabilities include transaction related services (e.g. order tracking, custom forms, escrow services and data warehousing,) and information related services (e.g. industry news, legislation changes, financial statistics and futures quotes). the latter are anticipated to offer the greatest opportunity for marketplaces to ajis vol. 10 no. 2 may 2003 25 gain competitive advantage (bakos, 1991) and develop extensive information capabilities to attract both buyers and sellers (brunn et al., 2002; wise & morrison, 2000). benefits of the electronic marketplace the reported benefits to companies that are trading through e-marketplaces are compelling and suggest that cost savings being experienced by companies are considerable. lucking-reiley and spulber (2000) report that online transactions could ‘reduce costs by a factor of five or ten or more.’ the following list of cost saving efficiencies was collated from a search of current periodicals and journals: � lower procurement costs � lower search costs � reduced administration costs � reduced development time � integrated global suppliers � cuts in inventory holdings � up-to-the-minute order tracking � a strengthening of relationships with commercial partners � exposure to a global market the scope of the individual marketplace and the commitment of the participating buyers/sellers will influence the level of these benefits that can be realised by the participants of a particular marketplace. key success factors of e-marketplaces an examination of the key success factors of e-marketplaces first requires due thought be given to the what is meant by success and the question of what constitutes a successful e-marketplace. simplistically, a successful marketplace is one that is profitable, but market makers, suppliers and buyers may place greater emphasis on achieving significant value; for example cost savings or market share (brunn et al., 2002), or gaining first mover advantage (bakos, 1998). success can be seen as a multi-dimensional concept where the different objectives of the stakeholders result in disparate perceptions of whether the marketplace can be deemed successful. in addition, in a dynamic electronic environment where consolidation of marketplaces is underway, profitability is not a guarantee of sustainability. a clear strategy that shows an understanding of stakeholder needs and the key issues affecting the online marketplace is more likely to enable market makers to survive the ‘consolidation crunch’ (brunn et al., 2002). however, gaining such an understanding of these needs and key issues is a difficult task in an immature and fast moving environment. the academic press has contributed significantly to the theory of electronic marketplaces (bakos, 1991, 1998; burton & mooney, 1998; kaplan & sawhney, 2000; malone et al., 1987) and research is ongoing into specific aspects of their development (bakos, 1997; choudhury, hartzel, & konsynski, 1998; deeter-schmelz, bizzari, graham, & howdyshell, 2001; dominique, 1998; ordanini & pol, 2001). however, it is the business press that offers current information on how market makers are structuring their companies in an effort to attract participants and which records the failures and successes of emarketplaces. business publications often source their articles from business consultants and it is often these consultants “that shape how managers think about it” (davenport & markus, 1999), while articles “from harvard business review, the sloan management review, computerworld and cio magazine” often form the basis of much is student course readings (davenport & markus, 1999). judicious use of such articles contributes to practitioner relevance in is knowledge and can inform on current issues in dynamic environments where an established body of academic literature has not yet formed. it was the business literature, which reflects a cross section of the dialogue being held within the business community, that formed the literature base for an earlier assessment of key success factors in electronic marketplaces (standing, 2001). using an analysis of over 150 business articles and reports, standing (2001) identified the key characteristics of e-marketplaces that are perceived by practitioners to have achieved a measure of success. our paper extends the study by the use of 12 mini-cases to determine the extent to which these characteristics, or key success factors, are being addressed by market makers. the previously identified key success factors are summarised in table 1. academic sources have been identified to support the business literature used by standing and table 1 reflects the factors as discussed within the academic and practitioner communities. ajis vol. 10 no. 2 may 2003 26 aims and methodology this study analyses electronic marketplace websites to determine the extent to which key success factors, identified from the literature, are being addressed. although the analysis of websites is a recent area of study, many disciplines have moved towards an examination of them particularly in regard to judging the purpose of specific websites and assessing the effectiveness of design (gibson and ward, 2000). this study however, is concerned with identifying the presence of specific factors on the websites and a content analysis approach was selected as the best method for identifying the presence of such data (berg, 2001). as a detailed content analysis was conducted on each of the selected websites, these websites can be regarded as mini-cases and are thus referred to in this paper. an awareness of both manifest and latent content analysis strategies (berg, 2001, p243) enables a more interpretive approach to be taken, to account not only for the physical presence of elements, but also for an awareness of the underlying meanings that may exist. a coding scheme was developed using concepts as the unit of analysis. an initial list of words, word groups and concepts was drawn up from the literature and refined by applying them to e-marketplace websites (not included in the mini-cases) to test the relationship between the critical success factor and the coding scheme. table 2 lists the concepts sought in an examination of the 12 marketplace sites. the use of these concepts allows for verification of the analysis while not confining it to rigid constraints that ignore the apparent differences in the diversity of the various marketplaces. ajis vol. 10 no. 2 may 2003 27 key success factor notes critical mass sufficient number of participants to ensure effective running of an emarketplace. low entry costs, designed to attract new participants also encourage memberships of multiple e-marketplaces which has implications for income streams as it disperses the transactional value of suppliers’ activities. (brunn et al., 2002; raisch, 2001; sculley & woods, 2001) income streams identified income streams include: transaction related fees membership/licensing fees sales of industry information value-add service fees advertising and marketing sales of accumulated marketing data. (gallaugher, auger, & barnir, 2000; sculley & woods, 2001) security a secure environment in which to exchange commercially sensitive and financial information. (mott, 2000) level of independence the ability to offer a neutral environment in which participants can trade without fear of compromise. the federal trade commission reports that over-inclusive ownership by industry consortia could raise concerns of exclusion practices (ftc, 2000). (kaplan & sawhney, 2000) technology infrastructure issues of: i. technological complexity resulting in additional costs to participants (e.g. hardware, software, training etc.) ii. software standards (croom, 2000; mott, 2000) relationship management: trust privacy trust can be signalled using criteria based on those outlined by smith et al (1999) in relation to business-to-consumer emarketplaces: i. online community facilities allows interaction and the sharing of positive references (kollock, 1999) ii. links from other trusted sites iii. unbiased product information from third parties iv. existing reputation (brand name). commercial firms also offer authentication, validation and transaction services. privacy raises issues of maintaining industry relationships and safeguarding sensitive information. fulfilling participants’ needs (value-add) market makers need to achieve a form of competitive advantage to attract participants and maintain profits (bakos, 1991). in identifying and incorporating a relevant range of facilities to meet, or exceed, participants’ needs, e-marketplaces can increase viability. the advantages of community sites are well-documented (hagel & armstrong, 1997) and the use of community site facilities has the capacity to create trust and enables the e-marketplace to act as a one-stop portal for industry participants. (brunn et al., 2002; wise & morrison, 2000) table 1: key success factors ajis vol. 10 no. 2 may 2003 28 critical success factor concepts sought critical mass names of major industry suppliers as participants, percentage growth rate, percentage of participant market, average transaction size, ‘we have critical mass’, number of participants, number of product lines income streams fees, transaction fees, licensing fees, commission, advertisements, percentage cut security ‘highest levels of security’, security statement, security guaranteed, security company participation (e.g. logo), secure environment/site level of independence statement of ownership, no affiliations, not affiliated to, independent, neutral technology infrastructure technology standards, industry standards, technology partnerships, infrastructure relationship management: trust privacy community facilities, links to other trusted sites, third party endorsement, validation, feedback, maintenance of partner relationships privacy statement fulfilling participants’ needs (value-add) any words or word groups that relate to facilities offered in addition to buying and selling transactions e.g. news, chat rooms, what’s new etc. table 2: concept coding scheme in a study of this kind the optimum number of websites to analyse is open to argument. a detailed analysis of twelve sites ensured coverage of a range of different types of marketplaces. a list of e-marketplaces, trading on the internet, was compiled from a search of the business press, academic journals and the use of the altavista (http://www.altavista.com) and yahoo (http://www.yahoo.com) search engines. the twelve sites were selected to represent a range of criteria (table 3) to ensure that no one business model was evident in the research. some industries are better served by electronic marketplaces than others and therefore a diverse range of industries, serving a cross section of market sectors, was selected to give a broader view of the prevalence of the recognition of critical success factors. ajis vol. 10 no. 2 may 2003 29 market sector industry ownership model entry barriers geographical range languages quadrem industry-wide suppliers/ buyers mining, minerals and metals industry industry consortia open global english, french, spanish, portuguese covisint manufacturers and their suppliers automobile industry industry consortia open global english, japanese converge wholesale distribution hi-tech goods industry consortia open global english appliance zone wholesalers/ high st. retailers consumer appliances intermediari es open europe english, french, german paper exchange industry wide suppliers and buyers pulp and paper industry intermediari es open global english, french, german, italian, spanish, finnish, swedish egreencoffee commodity traders coffee trade intermediari es open global english,spanis h, portuguese e-greenbiz small/medium wholesalers green industry products independent open usa english cargofinder service sector transportation (shipping) independent open dutch based (global) english 58k medium to large scale service sector printing independent industry registered printers only usa europe english, french telemerc reselling products and services telecomms. “vendor neutral” open global english retailers market xchange small businesses convenience stores and small business retailers intermediari es open usa english ecfood ‘large industry players’ food and beverages intermediari es certified suppliers and invited buyers global english table 3: selected electronic marketplaces representative marketplaces of intermediary and consortia type ownerships were selected and entry barriers, where evident on the websites, have been listed. all the remaining marketplaces are subject to registration but appear to have no other entry barriers. the geographical range is given as stated by the marketplaces although in some cases it is evident that the marketplaces are not truly ‘global’ in scope as can be seen from the number of global marketplaces that do not offer language options. the language options noted have an affinity to the industry sector and reflect the countries where the industry is particularly strong. for example, covisint, the automobile consortia site offers japanese and english while the paper and pulp industry marketplace offers a wide range of languages including finnish and swedish. the selection of the mini-case marketplaces to represent a broad range of criteria enabled a consideration of the recognition of critical success factors across all models of e-marketplace. as the marketplace environment matures, an assessment across industry sectors or specific business models will become more relevant. to further define the profile of each of the e-marketplaces, their methods of transacting business are given in table 4 and the type of service they offer is discussed below. ajis vol. 10 no. 2 may 2003 30 auctions storefronts negotiations catalogues exchange www.quadrem.com x x x www.covisint.com x x x www.converge.com x x x x www.appliancezone.co m x x www.paperexchange.c om x x x x x www.egreencoffee.com x x www.e-greenbiz.com x x www.cargofinder.com x www.58k.com x x www.telemerc.com x x x x www.retailersmarket xchange.com x x www.ecfood.com x x x x table 4: transaction mechanisms offered by the electronic marketplaces four e-marketplaces offered identification and selection services with execution of transactions being completed externally (usually via email) and five sites offered full execution services with financial and logistical support. three marketplaces were unclear, but it is thought that they offered, or were developing, execution services. all but one e-marketplace offered two or more methods of transacting business. the most common facility offered for transacting business was negotiation, closely followed by auctions (no distinction has been made between types of auction) and catalogues. the exchange facility, which is used for commodities and commodity-like products, was offered by only a third of the e-marketplaces and storefronts by a quarter. findings the findings are reported under the individual critical success factor headings: critical mass market makers recognised the need to achieve critical mass, and to convince prospective participants that their marketplace was a relevant force. nine sites showed figures indicating the number of registered buyers/sellers, the growth rate of transaction turnover or the number of product lines available. for example, 58k.com in which 3,680 printers participate, has a ‘growth rate of 3% daily’ and e-greenbiz.com has over 4,000 companies with 46,000 product line. only three e-marketplaces did not give figures or directly address the subject of critical mass, although each of them gave details of the size of the market they were addressing. income streams four of the identified income streams were found in eight of the websites; details are shown in table 5. the remaining four sites did not explain how income was raised and it was not possible to ascertain what income model they used. details of income generation through the sales of accumulated market data was not available. however, it is evident from the privacy statements given on two sites that market data is accumulated in this way and therefore this method of income generation has been added to the table. five of the sites use multiple income streams. it is likely that other forms of income generation are being developed and used, but these cannot be identified without fuller access to the e-marketplaces (for example, income from hosting storefronts for suppliers). ajis vol. 10 no. 2 may 2003 31 transactio n fees licensin g fee advertisin g value-add service fees sales of industry informatio n www.quadrem.com www.covisint.com x x www.converge.com x x x www.appliancezone.com www.paperexchange.com x x www.egreencoffee.com x x www.e-greenbiz.com www.cargofinder.com currently free x www.58k.com x www.telemerc.com currently free x www.retailersmarketxchange.com x x www.ecfood.com table 5: income streams identified in the e-marketplaces security the structure of the marketplace had an influence on how the security of the sites was addressed by the companies. e-marketplaces that offered only identification and selection processes (and therefore did not carry information on financial transactions) addressed the question of privacy rather than security. however, the larger industry consortia sites had strong statements assuring participants that security was ‘paramount’ and two offered security help for ‘company it desks’. seven sites relied on strategic partnerships with expert security companies to indicate their commitment to security issues. two e-marketplaces carry the verisign logo (http://www.verisign.com) although it is interesting to note that the status of one is ‘revoked’ and the other is ’expired’, which must raise concerns about the commitment these marketplaces have towards security. level of independence only three sites did not carry statements asserting their independence as an e-marketplace. the remaining nine companies all carry statements ranging from ‘e-greenbiz.com does not own any products nor is affiliated to any industry participant’ to appliancezone’s statement that they are an independent, neutral company funded by venture capital. industry consortia sites were particularly clear in stating that, although ownership is shared amongst industry players, the e-marketplace companies were completely independent. technology infrastructure four marketplaces mentioned technological infrastructure or addressed the question of standardization, three of which gave statements regarding the latter. one smaller company showed an awareness of the technical complexities that might concern its less technologically advanced customers, while two of the largest marketplaces state that their sites are standardized on xml. one consortia company is developing its own technology standards. relationship management two facets of relationship management were considered: trust and privacy. (i) trust. three sites were found to have incorporated an online community aspect to their marketplaces. two marketplaces incorporated links to other sites which were established within the relevant industry, although it was not possible to trace links back from other sites to the e-marketplaces. two marketplaces requested feedback from visitors, encouraging unbiased product information. five marketplaces had no identified criteria for signalling or engaging trust, although two sites recorded other aspects which could signal a measure of trust (validation of suppliers through an existing industry register and participation in ajis vol. 10 no. 2 may 2003 32 community-based initiatives). no direct use of existing reputation was discernible although all the emarketplaces gave lists of their strategic partnerships to give credence to their sites. (ii) privacy statements, relating to protection of data, were given on all the sites. four e-marketplaces included a facility to enable buyers and suppliers to maintain relationships (both new and established) by communicating and transacting in private within the market site. while several of the e-marketplaces gave assurances that information accumulated on participants would not be disclosed to a third party, there were few assurances that amalgamated data would not be collated. indeed, at least two of the major industry consortia sites have statements that visitors will be blocked from their sites if they disenable the cookies which allow data collection. fulfils participants’ needs (value-add) a wide variety of value-add facilities were identified, some of which occurred in a number of sites (e.g. industry news) and others (e.g. completion of customs paperwork) in only one site. e-marketplaces that incorporated an online community site had the largest range of added facilities such as tutorials, book sales, chat rooms and industry information. the larger companies offered more value-add in respect of transactionrelated facilities: for example, data warehousing, logistics and financial services, transaction tracking and virtual project workspace. the more specialised sites had industry-related value-adds. for example a commodities site offered futures quotes both in real-time (paid for) or delayed time (free) while the food industry orientated its value-add towards requirements in the food industry relating to health and safety, ingredient listings and standardized ingredients discussion the level of recognition of the key success factors by market makers was high with evidence of two factors (privacy and value-add) being addressed by all the marketplaces and a further four addressed by at least two thirds of the marketplaces (table 6). less focus was given to the remaining two factors, one of which (technology infrastructure) was ignored by eight of the marketplaces. the implications of these levels of recognition are discussed to identify questions and issues that remain to be addressed in the development of emarketplaces. ajis vol. 10 no. 2 may 2003 33 key success factor evidence of recognition critical mass 9 marketplaces gave numbers of participants or daily growth rate of the site. 3 marketplaces gave only the size of the market they were addressing. income streams 4 marketplaces showed no information on income streams 5 marketplaces had more than one income stream 3 marketplaces had a single income stream (with 2 currently waiving transaction fees) security 7 marketplaces used strategic partnerships with security firms 2 marketplaces used third party verification (verisign) 3 marketplaces showed no recognition of this factor level of independence 9 marketplaces carried statements asserting their independence technological infrastructure 4 marketplaces addressed technological infrastructure or standardisation relationship management i. trust ii. privacy i. trust 7 marketplaces showed at least one signal of trust 5 marketplaces had no identifiable signals ii. privacy all the marketplaces carried privacy statements fulfilling participants’ needs value-add all the marketplaces had at least one value-add facility table 6:level of recognition of critical success factors by market makers key success factors the importance of critical mass to the survival of an e-marketplace is self evident: if the marketplace does not have sufficient buyers and sellers it will not attract more participants. an awareness of this ‘chicken and egg’ situation was evident in the mini-cases examined where three quarters of the sites used a variety of signals to show that they had sufficient participants for an effective trading site. these signals varied from a straightforward statement of the number of participants to details of membership growth rates and the number of product lines carried by sellers in the marketplace. interestingly, the use of brand names to attract participants was not strongly promoted, even on the consortia sites where well known, multinational organisations were active. the use of brand names was restricted to strategic partnership lists that were not prominently displayed and would unlikely to be of interest to smaller prospective participants. the three marketplaces that only gave the size of the market they were addressing were indicating the potential of their marketplaces, but were not effectively signalling that a viable trading site had been established. however, all the sites were encouraging new participants through a number of initiatives, including an initial ‘try before you buy’ phase, free transactions, site tours and lengthy faq sections. the most common model of income generation, in eight sites, remained the charging of transaction fees although two sites had temporarily waived their fees in an attempt to attract critical mass. in all but one marketplace the fees were supported by additional forms of income generation. in the two sites that waived fees, income was generated by advertising revenues and it is possible that the market makers intended to use this as the major source of income. advertising revenue, although a potentially valuable income, needs to be weighed against any negative images of neutrality or quality that site advertisements may generate. it is not possible to predict which of the many different income streams may be viable sources of revenue (for example, it is not clear if the sale of futures quotes generates profit or is a value-add facility) and what factors affect participant preferences. a marketplace that attracts frequent transaction use by participants might better serve those participants by a move to licensing fees, while a site such as cargofinder that hosts fewer transactions but attracts multiple visits by firms seeking shipping details may benefit more from a choice of advertising revenue. ajis vol. 10 no. 2 may 2003 34 the use of multiple income streams by the market makers may indicate that they are uncertain of the potential of each income stream and are still in an experimental stage. alternatively, it is a likely indication of the need to generate revenues. it may also show a determination to retain flexibility thereby enabling participants to choose their level of financial commitment to the marketplace. such flexibility is a major advantage in attracting new participants to add to the critical mass of the marketplace. only one of the marketplaces, covisint, has introduced licensing fees. as a consortium marketplace they have the advantage of large multinational organisations as owners, which attracts a multitude of potential suppliers thereby ensuring a level of critical mass that few of the other marketplaces can match. for covisint, licensing fees are an additional source of income that is unlikely to have an effect on critical mass. security remains an issue of concern in several areas of electronic commerce (mott, 2000) and ways of ensuring the security of sites carrying sensitive commercial data and financial records remain a problem. statements regarding security were found on sites that offered transactional facilities beyond simple identification and selection processes (choudhury et al., 1998). these sites used strategic partnerships with security companies to deal with the issue. while this places the issue of security within specialist companies and ensures a level of expertise, it is unlikely that it will be enough to secure confidence unless breaches of security are avoided. the commitment of some market makers to security also comes into question where third party security firm ‘stamps of approval’ are not valid as was the case with the two marketplaces displaying the verisign logo (www.verisign.com). the three sites that did not address the security factor did not carry transactional data or appear to offer financial services of any kind. these sites were more concerned with the issue of privacy between buyers and sellers and carried clear privacy statements detailing their commitment to maintaining privacy within the marketplace. such statements were carried by all the sites. nine of the e-marketplaces declared their independence with emphatic statements notwithstanding their ownership, although in the case of the industry consortia it remains an open issue subject to close scrutiny by government bodies (ftc, 2000). independence is more of an issue for the consortia sites (all of whom carried statements of independence) which are comprised of normally competitive organisations within an industry collaborating in the creation of a marketplace for procurement purposes. such collaboration necessarily raises questions of neutrality for new entrants to the marketplace. the issue becomes more complex where consortia such as quadrem are developing buying and selling facilities for participants who are not owner-participants. the key success factor of technological infrastructure was, perhaps surprisingly, recognised by only four market makers. there are two issues of concern here. firstly, the additional costs of hardware, software, training and personnel remain a barrier to electronic commerce for many companies. the internet and its availability has alleviated some of these problems, but the more complex e-marketplaces cannot offer a full range of services without additional applications that increase the cost of participation and require a degree of commitment from participants in terms of technological investment. this impinges on the second issue of concern, the setting of software standards. without established software standards, commitment to investment in one marketplace affects the ability to participate in other marketplaces and indeed, the ability to connect marketplace activity with existing back-end systems. the establishment of common software standards will depend to a degree on the willingness of market makers with significant market share and economic power to co-operate. covisint has already established its own standards and other powerful marketplaces are likely to follow suit. one of the intermediary market makers showed awareness of technological issues for smaller companies wishing to participate online and offered an advice service. this awareness of the needs of small to medium size businesses (smes) has the potential to substantially increase the critical mass of marketplaces. ‘smes are a vast sector within the global economy’ (stokes, 2000) and many governments are funding a wide range of initiative to increase smes use of internet commerce (poon & swatman, 1999). few market makers have yet shown much awareness of the potential business that smes can bring to the e-marketplace. relationship management is very much an issue of trust which is a widely discussed aspect of business to consumer e-commerce, where several signals of trust have been developed and refined since early user groups began trading online (essig & arnold, 2001; geroski, 1998; humphrey & schmitz, 1998; kollock, 1999; smith, bailey, & brynjolfsson, 1999). business to business issues have lagged behind those of the consumer sector in electronic commerce and this was evident from the examination of the mini-cases. there was limited evidence of the recognition of the importance of signals of trust in the marketplaces although smaller companies utilised user friendliness and community orientation, while the larger companies used guided tours and presentations to display openness. none of the e-marketplaces used feedback from customers to promote trust with new participants. trust is a difficult issue to quantify and the building of trust is a long term and subjective process. however, it is being recognised as an essential factor of e-business success (keen, 2001) and requires more evidence of this to be shown by market makers. the remaining success factor is the e-marketplaces’ ability to fulfil participants’ needs and market makers showed that they are very aware of the need to provide more services than purely transactional ones. some companies were developing the online community model where the value-add covers a much wider range of ajis vol. 10 no. 2 may 2003 35 facilities than more functional sites which offered minimal value-add. the community model can include a wealth of facilities related to both transactional and information needs that can be developed according to the evolving needs of the community (hagel & armstrong, 1997). however, other e-marketplaces serve the needs of complex, fast moving industries where greater functionality is required and the community aspect holds less value. it is the market makers’ ability to anticipate and evolve their value-add facilities to meet participants’ needs that will enhance the attractiveness of the marketplaces once the initial participation stage is passed. switching costs although there has been some anticipation that the internet will reduce switching costs (porter, 2001) there is some evidence that dominant companies will raise switching costs to retain participants within a marketplace. for example, covisint, a marketplace with dominant market players charge a one-off licensing fee which has switching cost implications for small suppliers. they have also introduced their own technology standards which further reduces suppliers’ ability to participate in several e-marketplaces and binds them more closely to the larger industry players. the lack of common technological standards has switching cost implications for all electronic marketplace participants who will need to invest in diverse technologies to access different trading sites, so reducing flexibility in the electronic environment. however, commitment to one marketplace may become more common if the anticipated consolidation of marketplaces is as concentrated as forecast. summary the key success factors are all inter-related. critical mass is essential if the marketplace is to function effectively and retain both buyers and sellers by offering a choice of trading partner. however, other factors involved in achieving and retaining critical mass include generating and maintaining income streams that allow the marketplace to function as a commercial enterprise but that do not raise barriers to entry or impinge on trust issues through concerns about sales of private customer information. trust in the independence of the marketplace is also necessary, as overt displays of bias will alienate participants. technology barriers can deter the creation of critical mass, but low or diverse technology standards will not support the retention of participants or allow the marketplace to evolve its services to enhance its offered benefits. security and privacy, defined by keen (2001) as customer safety, often do not become an issue until there is a breach, which would destroy trust and seriously damage the viability of the marketplace. however, an assurance of security and privacy is essential to attract participants and requires the market makers to “define, implement and monitor policies and procedures to guarantee the maximum degree of customer safety as rigorously as your firm does for its finances.’ (keen, 2001) without participant faith in the security and privacy of an e-marketplace there can be no trust. trust is therefore closely related to security, privacy and the maintenance of relationships and is an essential factor in the retention of marketplace participants. if relationships cannot be built and retained, then critical mass will not be sustained. an important contributing factor to building and retaining relationships with participants is a marketplace’s ability to develop value-add facilities that reflect the needs and anticipates the wants of those participants. it is these value-add facilities that are predicted to contribute to the competitive advantage of e-marketplaces that survive the era of consolidation (brunn et al., 2002; wise & morrison, 2000). conclusions using the seven key success factors that were identified for this study, it has been possible to see that emarketplaces are recognising many of the requirements they need to develop their potential in the diverse ways that are suited to attracting and retaining their target market participants. however, several issues and questions remain. resources are being directed at building critical mass by encouraging new participants. multiple income streams have been identified by the e-marketplaces and a commitment to ensuring privacy is evident. questions remain concerning the issue of trust where there is a need to identify and develop more effective strategies. this is particularly important in the current state of proliferation of e-marketplaces, where participants have more choice of new sites if their trust is compromised. those markets that are able to develop and retain trust will gain competitive advantage in the imminent era of consolidation. in the areas of security, technology infrastructure and independence there remain concerns. as more emarketplaces support complete execution of transactions, involving confidential financial and logistical data, the issue of security needs to be further resolved to the satisfaction of participants. there will be an impact on the technological infrastructure of e-marketplaces if the larger companies develop their own standards rather than work towards common ones. a fragmentation of standards will lead to greater switching costs and ajis vol. 10 no. 2 may 2003 36 reduce the ability of participants to maximise their benefits from the electronic environment. a similar result may occur if the question of neutrality of e-market companies remains ambivalent. this is being addressed by government trade bodies in several countries as more industry consortia enter the arena, but the problem is likely to be exacerbated during the forthcoming consolidation period. the key success factor of meeting participants’ needs through value add has been well recognised by e-market companies and there are many examples of these companies being innovative in their approach to value add without compromising the more basic needs. once the other key success factors have been fully addressed, it is perhaps the value add which will encourage participants to favour one e-marketplace over another and play a major role in contributing to survival when the anticipated consolidation of the marketplaces begins. references bakos, j. y. (1991). a strategic analysis of electronic marketplaces. mis quarterly, vol 15(3), pp. 295-310. bakos, j. y. (1997). reducing buyer search costs: implications for electronic marketplaces. management science, vol 43(12) pp 1676-1692 bakos, j. y. (1998). the emerging role of electronic marketplaces on the internet. communications of the acm, vol 41(8), pp 35-42. banham, r. (2000). the b-to-b virtual bazaar. journal of accountancy, vol 19(1), pp 26-30. berg, b. (2001). qualitative research methods for the social sciences (4th ed.). needham heights, ma: allyn & bacon. brunn, p., jensen, m., & skovgaard, j. (2002). e-marketplaces: crafting a winning strategy. european management journal, vol 20(3), pp 286-298. burton, a. f., & mooney, j. g. (1998). the evolution of electronic marketplaces: an exploratory study of internet-based electronic commerce within the american independent insurance agency system. australian journal of information systems, special ed., pp 20-42. choudhury, v., hartzel, k. s., & konsynski, b. r. (1998). uses and consequences of electronic markets: an empirical investigation in the aircraft parts industry. mis quarterly, vol 22(4), pp 471-507. croom, s. r. (2000). the impact of web-based procurement on the management of operating resources supply. the journal of supply chain management, vol 36 (1) pp. 4-13. davenport, t., h, & markus, l. m. (1999). rigor vs relevance revisited: response to benbasat and zmud. mis quarterly, vol 23(1), pp 19-24. deeter-schmelz, d. r., bizzari, a., graham, r., & howdyshell, c. (2001). business-to-business online purchasing: suppliers' impact on buyers' adoption and usage intent. journal of supply chain management, vol37(1), pp 4-10. dominique, n. p. (1998). exchange costs as determinants of electronic markets bearings. electronic markets, vol 8(1), pp 3-6. downes, l., & mui, c. (1998). unleashing the killer app: digital strategies for market dominance. boston: harvard business school press. essig, m., & arnold, u. (2001). electronic procurement in supply chain management: an information economics-based analysis of electronic markets. journal of supply chain management, vol 37(4), pp 43-49. forrester research. (2000). the e-marketplace shakeout, forrester research. available: http://www.forrester.com/er/baseline (accessed 19 april 2001). friesen, b. g. (2000). online markets are not for everyone. consulting to management, vol 11(2), pp. 4850. gallaugher, j. m., auger, p., & barnir, a. (2000). revenue streams and digital content providers: an empirical investigation. information and management, vol 38, pp 473-485. geroski, p. a. (1998). thinking creatively about markets. international journal of industrial organization, vol 16, pp. 677-695. gibson, r., & ward, s. (2000). a proposed methodology for studying the function and effectiveness of party and candidate websites. social science computer review, vol 18(3), pp. 301-319. hagel, j., & armstrong, a. g. (1997). net gain: expanding markets through virtual communities. boston: harvard business school press. humphrey, j., & schmitz, h. (1998). trust and inter-firm relations in developing and transition economies. the journal of development studies, vol 34(4), pp 32-61. hurwitz. (2000). e-marketplaces: issues, risks, and requirements for success. hurwitz group white paper. framingham, ma: hurwitz group inc. industry canada. (2000). canadian business information: business information by sector available online at: http://strategic.gc.ca (accessed 21 february 2001). ajis vol. 10 no. 2 may 2003 37 kaplan, s., & sawhney, m. (2000). e-hubs: the new b2b marketplaces. harvard business review, mayjune. keen, p. (2001). welcome to the trust economy. intelligent enterprise, vol 4(3), pp 36-39. kollock, p. (1999). the production of trust in online markets. advances in group processes, vol 16, pp 99123. lucking-reiley, d. (2000). auctions on the internet: what's being auctioned and how? the journal of industrial economics, vol. 48(3), pp. 227-252. ludeman, f. (2001). mining's b2b e-marketplaces --get ready. mining engineering, vol 53(1). malone, t. w., yates, j., & benjamin, r. i. (1987). electronic markets and electronic hierarchies. communications of the acm, vol 30(6), pp 484-497. modahl, m. (2000). now or never. how companies must change today to win the battle for the internet consumer. london: orion. mott, s. (2000). the second generation of digital commerce solutions. computer networks, vol 32, pp 669683. ordanini, a., & pol, a. (2001). informediation and competitive advantage in d2b digital marketplaces. european management journal, vol 19(3), pp. 276-285. poon, s., & swatman, p. (1999). an exploratory study of small business internet commerce issues. information and management, 35, 9-18. porter, m. e. (2001). strategy and the internet. harvard business review, march, pp 63-78. raisch, w. d. (2001). the emarketplace. strategies for success in b2b ecommerce. new york: mcgraw hill. sculley, a. b., & woods, w. a. (2001). b2b exchanges. the killer application in the business-to-business internet revolution. new york: harpercollins. senn, j. a. (2000). business to business e-commerce. information systems management, spring, pp 2332. smith, m. d., bailey, j., & brynjolfsson, e. (1999). understanding digital markets: review and assessment. in e. brynjlfsson & b. kahin (eds.), understanding the digital economy. boston: mit press spiegel, r. (2001). gotcha! traditional industries stole the e-marketplace initiative from dot-coms. now what? ecommerce business wednesday 28 february. available online at: http://www.ecommercebusinessdaily.com/archive/ecbfa121800-93154asp (viewed 1 march 2001) standing, c (2001), the characteristics of successful e-marketplaces. proceedings of the pacific asia conference on information systems stokes, a. (2000). small and medium-sized enterprises and the environment: business imperatives. international small business journal, 19(1), 100-102. wise, r., & morrison, d. (2000). beyond the exchange: the future of b2b. harvard business review, november-december, pp 88-96. ajis vol. 4 no. 2 may 1997 experimental comparisons of entity-relationship and object-oriented data models peretz shoval information systems program, department of industrial engineering and management ben-gurion university of the negev, beer-sheva 84105 israel (e-mail: shoval@bgumail.bgu.ac.il; fax: 972-7-6472958) abstract the extended entity-relationship (eer) model is being "threatened" by the object-oriented (oo) approach, which penetrates into the areas of system analysis and data modeling. the issue of which of the two data models is better for data modeling is still an open question. we address the question by conducting experimental comparisons between the models. the results of our experiments reveal that: a) schema comprehension: ternary relationships are significantly easier to comprehend in the eer model than in the oo model; b) the eer model supasses the oo model for designing unary and ternary relationships; c) time: it takes less time to design eer schemas; d) preferences: the eer model is preferred by designers. we conclude that even if the objective is to implement an oo database schema, the following procedure is still recommended: 1) create an eer conceptual schema, 2) map it to an oo schema, and c) augment the oo schema with behavioral constructs that are unique to the oo approach. introduction the object oriented (oo) approach has spread into various areas of computing that include not only programming, but also systems analysis and design, database management, among others. there is no doubts about the advantages of the 00 approach for programming, in that it supports software reuse and information hiding,. however, the superiority of oo approach in earlier stages of software development, i.e., system analysis and data modeling, has not as yet been proven. i concentrate on data modeling (or conceptual design), an activity aimed at creating a conceptual schema. this schema is usually represented in a diagrammatic form, as it serves as a communication tool between developers and users. once approved by users, it is converted into a specific database schema, depending on the data model and dbms used for implementation. this conversion is considered simple, being an algorithmic, automatic process. the major problem, however, is to create a good conceptual schema that is semantically correct and comprehensible. for many years, entity-relationship, with its many extensions, generally termed eer, has been the most widely used model for conceptual design. recently, the oo model has been threatening to replace the eer model. in addition to static (structural) constructs of the model whose representation of data structure is somewhat equivalent to eer representation (e.g., object classes considered equivalent to entity and relationship types) the oo approach models system behavior through "methods" that are attached to the object classes, with messages passing among them. while this extra capability, unlike eer, provides more than just a data structure model, it does not necessarily mean that eer should be replaced by oo, since eer may indeed be "better" than 00 for modeling the data structure. taking into consideration that an eer schema can easily be mapped to an oo schema (gogola et al 1993, kornatzky & shoval 1994, nachouki et al 1992, narasimhan et al 1993), it may be worthwhile to begin the process of data modeling with the design of an eer conceptual schema, then convert it to an oo schema (if the objective is to implement an oo dbms), and finally add the behavioral constructs. such an "indirect" strategy for creating an oo schema could be considered when the eer model is regarded a "better" than the oo model. "better" can be judged from various perspectives. i address this issue from four perspectives: a) comprehension: understandability of schemas; b) design quality: correctness of schemas; c) time to complete design tasks; and d) designers' preference of models. we have conducted two experimental comparisons which compare the two models on these dimensions. the next section discusses related studies, the following two sections overview the underlying models, then i describe the experiments and their results, and the final section summarizes the results and conclusions. related studies various studies on the evaluation and comparison of data models have been conducted. the earlier studies mainly compared record-based models (hierarchical, network and relational). later on, eer became the most frequently studied model. eer has been compared with relational and other recordbased models, as well as with other semantic models (e.g. batra et al 1990, juhn & naumann 1985). some studies compared the data models from a designer perspective in an attempt to find out which yields more accurate and precise schemas. other studies compared the query languages of the data models in order to determine the languages by which the designers compose queries with greater accuracy and speed. yet, other studies took a user perspective, attempting to determine ease of 74 ajis vol. 4 no. 2 may 1997 comprehension. a survey of earlier studies can be found in batra & srinivasan (1992). although the results of these studies are not always clear-cut or consistent, there is a tendency to agree that her is superior to other, record-based and conceptual models. one of their general conclusions was that usability of data models should be evaluated by their ability to model relationships, a point that is stressed in our experiments recently we have begun to find studies which evaluate the oo model on the basis of controlled experiments. for example, agrawal and sinha (1992) examined the influence of designers' experience in functional analysis and task characteristics on quality of 00 schema design. they found out that the oo model does not always lead to good results; the quality of result depends on task characteristics. palvia et al. (1992) compared oo, er and dsd models from an end-user perspective, to see which schema is more comprehensible. their subjects were mis students, and their database schemas were trivial in terms of size and complexity. each subject received a version of a database corresponding to one data model, along with 30 questions that evaluate comprehension of the database. the total score on correct answers was used as a measure of comprehension. they concluded that users of the oo model showed a significantly better understanding of the data than users of the other two models. they admit, however, that their conclusion is weak, because comprehension was measured on overall terms only, and suggest that future research be directed to studying specific aspects of comprehension, e.g., different types of relationships. bock and ryan (1993) reported on a comparison of oo and eer models from a designer perspective. they examined correctness of design of eight types of constructs, and measured correctness according to a grading scheme developed by batra et al. (1990). their results indicated that eer is better for representing attribute identifiers, unary 1:1 and binary m:n relationships, while there are no significant differences for other dimensions. also, they found no difference in time to complete the tasks. although these results are interesting, in our opinion they have limited external validity, mainly because the oo model they used (as in kroenke 1992) represents relationships only by reference attributes in such a way that does not enable to distinguish between an object that represents a binary relationship from an object that represents a ternary relationship. the underlying eer and oo models the objective of this section is to overview and exemplify the models we use for experimentation. since each of the models has many variants, we confine ourselves to two specific types, that are sufficiently general to validate the results of the comparison. the eer model: as eer is more "standard" than oo, we only present an example (figure 1) using a diagrammatic notation as in elmasri and navathe (1994) and teorey et al. (1986). the oo model the oo model is still evolving and as yet no standard has been defined. the model that we use is based on o2 (deux et al 1991) and ode (agrawal & gehani 1989). by extending the model we are able to show all the details of the oo schema on diagram, as would be expected at the conceptual design stage. (since we focus on modeling data structure and not system behavior, methods are not included.) an example of the oo schema is shown in figure 2, representing the same reality as the eer diagram in figure 1. 75 ajis vol. 4 no. 2 may 1997 ar sport tru figure 1: eer diagram an object has attributes. (for brevity, we omit "class" and simply say "object".) an attribute may be atomic or multi-valued, in which case the attribute name is preceded by "set". an identifier attribute of an object is underlined. an attribute may refer to another object, in which case the referenced object name in written in [ ] brackets next to the respective attribute name. this actually implies a bidirectional relationship between the objects, which is signified in two complementary ways: a) reference-attributes are included in each of the involved objects; b) a link connects the two objects, with an indication of the relationship cardinality. in the case of an 1 :n relationship between two objects, the object in the "1" side has a set reference attribute to the other object, and in case of an m:n relationship both objects have set reference attributes. an attribute may be a tuple consisting of multiple (value or reference) attributes, and signified by { } brackets. for example, the set producers {[producer], number of plants} of city. an m:n relationship can also be represented as a separate object, with reference attributes to the related objects. a ternary or higher order relationships is represented as a separate object, linked to the other objects involved in the relationship. the reference attributes in all objects are included. in the example, we have store facts about agents selling vehicles to persons. therefore, we used the objects person, agent and vehicle, and for the relationship we created the object sales. each of the three original objects has a set reference attribute to sales, while sales has a tuple consisting of three reference attributes to those objects. the use of tuple of reference attributes in an object that represents a relationship enables to determine the degree of the relationship represented by that object (e.g., whether it is binary or ternary). finally, object hierarchies are signified by links from the subtypes to the supertypes, similar to the eer notation. 76 ajis vol. 4 no. 2 may 1997 producer name address set wtiere_producing {[city], number of plants) set vehicles_produced [vehicle] 1 n vehicle licence no. model set sold_by/to [sales] produced_by [producer] agent name address set vehicles_sold [sales] sales sale ([agent],(person], [vehicle]) price ($) date_ol_sale city name state set producers ([producer], number of plants) set inhabitants [person] mayor [person] 1 person ssn name birth_date set tetephone.numbers lives_in [city] mayor_of [city] set vehicles_bought [sales] car oilconsumption (ipm) sport max_speed (mph) truck capacity (tons) figure 2: oo diagram the user comprehension experiment and results this section provides a summary of the user-comprehension experiment and its results. for more details (see shoval & frumermann 1994). the experiment we examined user comprehension as follows: two groups of users, each studying a different model, were given a diagram of the model they studied, and a set of identical questions (statements) about facts in the schema. the users were students of behavioral science and management, having similar backgrounds and experience. they all took the same three computer related courses; therefore, they may be considered as "sophisticated users", i.e., users who can interact with professional analysts, express information needs and approve their products. the subjects were randomly divided into two groups, each studying one model. the studies emphasized comprehension of the schema diagrams (rather than design of diagrams). the same instructor taught both models (to avoid bias in teaching quality), and the same amount of time 1.5 hours was allocated to teach each model. in order to motivate the students, they were told that their performance in the experiment would be considered as part of the final course grade. we prepared a questionnaire that consisted of 48 "true" and "false" statements about facts in the conceptual schemas. the purpose was to find out not only if there is a difference in overall comprehension between the two schema diagrams, but also if there is a difference in the specific construct types. therefore, we classified the statements according to different construct types: a) attributes of entities/objects , b) binary-relationships, c) two binary relationships, d) ternaryrelationships, and e) other facts with no direct relationships. we prepared two equivalent schema diagrams, one for each model. the diagrams included exactly the same facts. the her diagram included 12 entities, 7 binary-relationships, 2 ternary-relationships and various attributes. in the oo diagram there were 14 objects some of which representing binary and ternary relationships each with various types of attributes: atomic, multi-valued, reference-attributes and tuples. each participant was given a schema diagram, according to the model s(he) studied, and a questionnaire. each user was expected to review the diagram and to mark each statement "true" or 77 ajis vol. 4 no. 2 may 1997 "false". the level of comprehension was measured by counting the number of correct answers within each group for each of the categories of statements, and for all statements together. analysis of results the results of the experiment are summarized in table 1. for each category (including "all") and model it shows the mean grade in absolute values and in percents, and the standard deviation. the last two columns indicate the significance of difference between mean grades, determined according to twotail z tests at 0=0.05. as it shows, we cannot reject the null hypothesis that overall, there is no difference of comprehension between the two models, although there is a slight difference in favor of the her model. there is, however, a significant difference in favor of her for "ternary-relationships": an average of 91.1% for eer vs. only 67.8% for oo. on the other hand, there is an advantage to oo for "other facts": 83.5% to oovs. 74.1%toeer. table 1; summary of the results for comprehension type of construct attributes binary relationships two relationships ternary relationships other facts all no of questions 9 11 9 9 10 48 model eer oo eer oo eer oo eer oo eer oo eer oo mean score 7.83 7.73 9.98 10.05 6.73 6.95 8.20 6.10 7.41 8.35 41.02 40.03 mean in % 87.00 85.89 90.73 91.36 74.78 77.22 91.10 67.80 74.10 83.50 83.71 81.69 stds 1.34 0.99 0.96 0.94 1.23 1.08 0.80 1.60 0.89 1.46 2.49 3.64 z-value -0.37 +0.32 +0.83 -8.55 +3.47 -1.4 which? (a=0.05) no no no eer oo no the insignificant differences between the models with respect to "attributes", "binary-relationships" and "two relationships" can be explained by the similarity in diagrammatic representation. as would be expected, in both models statements involving two binary-relationships are more difficult to comprehend than those involving just attributes or single relationships average scores were only 74.78% for eer and 77.22% for 00. since we found a significant difference between the models in the case of ternary-relationships, we further examined these results by distinguishing between m:n:l relationships and m:n:n relationships. in both cases we found a significant difference in favor of the eer model. for m:n:l, the average score of eer was 93.33, compared to an average score of only 78.33 for oo. for m:n:n, the average scores were 90.67 and 62.17, respectively. the advantage of eer on ternary relationships can be explained by the lucidity of the diamond symbol which connects the three entities. it seems that this symbol is clearer and more visible than the "normal" object rectangle which is used in oo for any type of object. moreover, in oo, a ternary-object includes sets of tuple attributes, which seem to be more complicated to comprehend compared to eer, where the diamond representing the ternary-relationship has only its own attributes. with respect to "other facts", oo turned out to be superior. this is actually not a specific category as it includes different statements which deal with entities or objects that exist in the diagram but have no direct relationships. oo may be easier to comprehend for this category because here most of the information is encapsulated within the objects, as opposed to eer where it is "spread-out" among the entities and relationships. the design quality experiment this section provides a summary of the design-quality experiment and its results. for more details )see shoval & shiran 1996). the experiment in this experiment we compared the models from a designer perspective, addressing three dimensions: a) design quality, i.e., correctness of schemas; b) time to complete design tasks, and c) designer 78 ajis vol. 4 no. 2 may 1997 preferences of the models. with respect to design quality we defined null hypotheses that there is no difference in correctness of modeling these constructs types: 1) entities/objects, 2) attributes of entities/objects, 3) inheritance relationships, 4) unary 1:1 relationships, 5) binary 1:1 relationships, 6) binary l:n relationships, 7) binary m:n relationships, 8) ternary m:n:l relationships, and 9) ternary m:n:p relationships. with respect to time and preference, we hypotheses that: 10) there is no difference in time to complete design tasks, and 11) there is no difference in designer preferences of the models. we defined two design tasks, both similar in size and complexity (involving the same types and numbers of various constructs). we created a narrative description for each task, on which basis each subject was asked to design a conceptual schema diagram using one of the two models. the subjects included 44 students, all majoring in information systems, with a similar background and course of studies. all subjects were trained to use the two models. the same amount of time (six hours) was devoted to study each model, and the same instructor taught all the students. in order to motivate the subjects, they were told that their performance in the experiment would be part of the final grade. the subjects were randomly divided into two groups: subjects in one group designed the first task with oo and the second task with her; subjects in the other group switched models and tasks. the starting and ending times of each task were recorded. at the end, each subject was given a short questionnaire, using 7-point scales, to express an opinion on the ability of each model to design a conceptual schema, and to indicate the relative preference of the models. correctness of each schema was measured according to the evaluation scheme described in batra et al. (1990). each construct type, in each model, was scored separately by subtracting a certain number of points for each error type (distinguishing between minor, medium and major errors). then we averaged the scores of all subjects on each construct type for each of the two models. analysis of results the results are summarized in table 2 and discussed herein. table 2: summary of results for design quality, time and preference no 1 2 3 4 5 6 7 8 9 hypotheses entity/class attributes inheritance unary 1 : 1 binary 1:1 binary l:n binary m:n ternary m:n:l ternary m:n:p eer model average 99.15 95.46 99.43 88.07 94.32 82.58 81.25 85.23 94.32 stdv 2.63 3.37 3.77 30.26 22.14 27.13 28.08 31.59 16.05 oo model average 99.62 94.23 99.43 70.45 94.32 88.64 78.69 67.61 76.14 stdv 1.51 4.29 3.77 34.64 20.01 18.86 28.35 36.80 38.86 significance t-value -1.22 1.95 0 3.86 0 -1.44 0.42 3.18 2.93 p-value .23 .08 .0004 .157 .68 .0028 .0054 which? n n n eer n n n eer eer 10 [time (minutes) lla how good is model (1-7)?* 53.50 5.77 16.71 1.05 68.00 4.70 i 15.44 2.82 5.16 <.0001 tfleer, 3.45 <.0001 eer lib relative preference ** oo 7 1 1 6 5 , 4 ii 1 eer 3 | 2 1 2.82 eer .--.* * the is based on the wilcoxon test; 3.45 is z-value of the test. ** 7 = absolute preference of oo; 1 = absolute preference of eer a. correctness of design (hypotheses 1-9): we used the t-test to measure the difference of mean scores for each of the nine types of constructs. the last columns indicate the significance of the results. the "which?" column indicates if the results are significant (at o=0.05) and which model is favored. we found no significant difference for six of the constructs, namely entity types or object classes (1), attributes (2), inheritance, or supertype-subtype hierarchies (3), and the three types of binary relationships (5-7). these results are explained by the similarity of the models when dealing with these 79 ajis vol. 4 no. 2 may 1997 constructs. the slight differences in the symbols that represent entities or objects, as well as inheritance links, turned out to be insignificant for design quality. this is evidenced by the high scores on those constructs above or close to 90. interestingly, in both models the scores for 1:1 relationships were higher than those for l:n relationships, which, in turn were higher than those for m:n relationships. these consistent results strengthen their validity. contrary to the above, we found significant differences for unary (4) and ternary (8 and 9) relationships; all favored the eer model, as explained herein: unary 1:1 relationships: representation of unary relationships may be complicated, especially in the oo model, because there are different options by which to represent them. for example, the 1:1 unary "marriage" relationship, which appeared in the experiment, can be represented in oo model as a reference attribute in employee, or as a separate object (marriage) related to employee. as it turned out, in both models many errors occurred, but in oo we found more errors in cases where the designers chose to represent the relationship as a separate object. ternary m:n:p relationships: there were only a few errors in modeling m:n:p relationships with the eer model, but many errors with oo model. typical errors in oo occurred when instead of a separate object, the relationship was represented as (complex) attributes of other objects. for example, to represent a ternary relationship between worker, project and skill, some designers defined a tuple of reference attributes {[skill], [project], work_hours} in the object worker. ternary m:n:l relationships: here too, the eer model was, indeed, better. the results indicate that in both models m:n:l relationships are more difficult to model than m:n:p relationships, but oo designers made more errors. a typical error was representation of a ternary as two binary relationships (e.g., a ternary relationship between worker, project and city was represented by some designers as two binary relationships: one between worker and project, another between project and city). when we note the similarities and differences between our results and the results obtained in (bock & ryan 1993), we see that the oo model was not superior for any dimension. in both studies eer was better than oo for unary 1:1 relationships; but in (bock & ryan 1993) eer was better for binary m:n relationships, whereas in our study eer was better for ternary relationships. b. time for completing the design task (hypothesis 10): as seen in table 2, it takes significantly less time to design eer schemas. although the time factor in itself is of minor importance, the fact that this result is consistent with the results on correctness of design, strengthens the validity of the experiment. c. preference of models by designers (hypothesis 11): for this dimension as well, eer turned out to be significantly better. this is evidenced both by the scores given to each model independently (row 1 la), and the relative preference of the two models (row 1 ib). the average of 2.82 on a 7-point scale indicated a significant preference of the eer model. even if we agree that preference is not important as performance, the consistent results further validate the experiment. summary and conclusions in summary, we found that the eer model is superior to the oo model for the following reasons: 1. comprehension of schemas: eer ternary relationships are easier to understand, whereas for oo, only non-specific ("other") facts are easier to understand, with no significant difference for the other constructs. 2. design quality: complex (unary and ternary) relationships are designed more correctly in eer, whereas there is no significant differences when designing other modeling constructs. 3. time: it takes less time to design an eer schema. 4. preferences: designers prefer to use the eer model. therefore, we conclude that even if the objective is to design and implement an oo schema, the following strategy is recommended: a) design an eer schema and validate it by users/ management; b) 80 ajis vol. 4 no. 2 may 1997 map it to an equivalent 00 schema (using an appropriate mapping algorithm or tool; and c) augment the oo schema with the behavioral constructs ("methods" and "messages"). references agrawal, r. and gehani, n. (1989) ode (object database and environment): the language and the data model, acm sigmod intl. conf. on management of data, pp. 36-45. agrawal, r. and sinha, a. (1992) an empirical study of the effects of experience and task characteristics on object-oriented modeling, in: wits'92 workshop on information technology and systems, pp. 225-234. batra, d. (1993) a framework for studying human error behavior in conceptual database modeling, information & management, vol. 25 ,pp. 121-131 batra, d., hoffer, j. and bostrom, r. (1990) comparing representations with relational and eer models, comm. of the acm, vol. 33 (2), pp. 126-139. batra, d. and srinivasan, a. (1992) a review and analysis of the usability of data management environments, int'l j. of man-machine studies, vol. 36, pp. 395-417. bock, d. and ryan, t. (1993) accuracy in modeling with extended entity relationship and object oriented data models, j. of database management, vol. 4 (4), pp. 30-39. deux, o. et al. (1991) the o2 system, communications of the acm, 34 (10), pp. 34-48. elmasri, r. and navathe, s. (1994) fundamentals of database systems, 2nd ed., benjamin/ cummings,. gogola, m., herzig, r., conrad, s., denker, g. and vlachantonis, n. (1993) integrating the er approach in an oo environment, int'l conference on er approach, pp. 376-389. juhn, s. and naumann, j. (1985) the effectiveness of data representation characteristics on user validation, in: proceedings of the sixth int. conf. on information systems, pp. 212-226. kerlinger, f. (1973) foundations of behavioral research, holt, rinhart and winston. kornatzky, y. and shoval, p. (1994) conceptual design of object-oriented schemas using the binaryrelationship model, data & knowledge engineering, vol. 14, pp. 265-288. kroenkc, d. (1992) database processing, 4th ed., macmillan publishing. nachouki, l., chastang, m. and briand, h. (1992) from entity-relationship diagram to an objectoriented database, int'l. conference on entity-relationship approach, pp. 459-481. narasimhan, b., navathe, s. and jayaraman, s. (1993) on mapping er and relational models onto oo schemas, int'l conference on er approach, pp. 402-413. palvia, p., lio, c. and to, p. (1992) the impact of conceptual data models on end-user performance, j. of database management, vol. 3 (4), pp. 4-15. shoval, p. and frumermann, i (1994) oo and eer conceptual schemas: a comparison of user comprehension, j. of database management, vol. 5 (4), pp. 28-38. shoval, p. and shiran, s. (1996) entity-relationship and object-oriented data modeling an experimental comparison of design quality, data & knowledge engineering, forthcoming. teorey, t., yang, d. and fry, j. (1986) a logical design methodology for relational database using extended entity-relationship model, acm computing surveys, vol. 18 , pp. 197-222. 81 microsoft word ajis10-2finala.doc ajis vol. 10 no. 2 may 2003 1 editorial this is the second issue of volume 10 of the australian journal of information systems. as editor i am pleased to report the continuing interest in the journal both in australia as well as overseas. our first paper is presented by bruce rowlands, school of computing and information technology, griffith university. . until quite recently, there has been little available in the is literature to guide the interpretive researcher to build theory of is practice. this paper provides guidance and an example for carrying out research using an interpretive framework. rosemary stockdale and craig standing, school of management information systems, edith cowan university present our next paper, market makers’ recognition of key success factors in electronic marketplaces. this study examines the recognition and use of critical success factors by market makers in electronic marketplaces. our next paper is presented by jennie carroll, steve howard, department of information systems, the university of melbourne, jane peck, novell pty ltd and john murphy, department of information systems, the university of melbourne. this paper reports a research project that examines young people’s appropriation of mobile technologies. thirty young australians were supplied with a free wap phone for a month and were tracked from their expectations, the initial encounter and the first month of use of a wap phone. diana gray and jeanette van akkeren, faculty of business, university of the sunshine coast, present our next paper, entitled e-commerce experiences in the real estate industry. a preliminary study in regional queensland. computer systems have become commonplace in most smes and technology is increasingly becoming a part of doing business. in recent years, the internet has become readily available to businesses; consequently there has been growing pressure on smes to take up e-commerce. this paper presents three indepth case studies from the real estate industry in a regional setting. phil joyce, school of information technology, swinburne university, ross smith, faculty of business and law, deakin university and kathryn hodges, telstra pty ltd present our next paper, information systems development contracts: an exploratory study of australian legal practitioners’ and information systems professionals’ perceptions. is professionals are primarily concerned with the development of a project to meet the goals of a client. unfortunately, is professional do not always pay close attention to the contract or to its management and are often unaware of the legal implications of the contracts to which they are assenting. this paper reports an exploratory study of the differences in perceptions of is development contracts that exist between is professionals and legal practitioners. our next paper is presented by helen richardson, information systems and technology, the open polytechnic of new zealand and beverley hope, school of information management victoria university of wellington and is entitled the role of information systems and technology in case management: a case study in health and welfare insurance. this study reports the role of information system and technology (ist) in supporting case management at the accident compensation corporation (acc) of new zealand. the findings of this study show that for acc, ist’s most valuable role is enhancing the relationship between client and case manager rather than replacing it. anne rouse, department of information systems, university of melbourne and brian corbitt, school of information systems, deakin university present our next paper. early in 2001, after a damning public report by the auditor-general, the australian federal government abandoned its highly promoted “whole of government” it infrastructure outsourcing initiative. a four-year case study conducted by the authors examines why the government’s expectations were not achieved, and arrives at conclusions that have important implications for decision makers confronted with choices about sourcing it service delivery. our next paper is presented by afzaal h. seyal and mohd. noah abd. rahman department of computing and information systems, institute of technology brunei and is entitled student use of the internet: an extension of tam in technical & vocational institutions in brunei darussalam. the paper surveys 266 students in 4 vocational colleges. jenine beekhuyzen, sue nielsen and liisa von hellens, school of computing and information technology, griffith university, brisbane, australia present our next paper, challenging dualisms in female perceptions of it work. this paper explores the way that professional women working in the it industry discuss the nature of their work and suggests that the way women talk about their work reinforces widely held impressions of the information technology industry. our next paper is presented by judy young, school of information systems, university of tasmania. the aim of this research is to make a contribution to knowledge in relation to women as a minority group in the information communication and technology (ict) industry. the focus of the current research is on year nine students throughout tasmania. the last five papers in this issue examine the importance of ethics in information systems. ajis vol. 10 no. 2 may 2003 2 the first of these papers is doing ethics: a universal technique in an accessibility context by christopher r. simpson, liddy nevile and oliver k. burmeister, swinburne university of technology. this paper presents techniques that allow the illustration of the value of ethics. our second paper is entitled considering the problem of insider it misuse by steven furnell and aung htike phyo, network research group, university of plymouth, plymouth, united kingdom. this paper focuses on the problem of insider misuse, the scale of it, and how it has effected the organisations. shona leitch and matthew warren, school of information technology, deakin university, present our next paper, designing systems for e-commerce. this paper discusses how ethical matters can affect the design of e-commerce systems and how a framework designed for e-commerce can be used to create and deliver effective, ethical e-commerce systems. the debate about the likely effectives of a scheme to regulate the access to certain internet content during the hearings of the select senate select committee and during the parliamentary debates is the subject of our next paper, entitled the co-regulatory scheme for internet content regulation: operational effectiveness, by geoffrey a sandy, school of information systems, faculty of business and law, victoria university. our final paper is entitled australian hackers and ethics, by m.j.warren, school of information technology, deakin university and w.hutchinson, school of computing & information science, edith cowan university. the aim of the paper is to look at the way hackers act and ways in which society can protect itself. the paper shows the current views and attitudes of hackers in an australian context. once again i would like to pass on my thanks to the authors and consulting editors for the high quality of material in this issue. i would like to thank lily soh and diniz da rocha for their continued assistance with the journal and the maintenance of the home page. rob macgregor editor http://www.uow.edu.au/ajis/ajis.html microsoft word 13 1 marina.doc ajis vol 13, no. 1 september 2005 19 rule mining for dynamic databases a das & d k bhattacharyya department of information technology tezpur university, napaam 784 028, india e-mail : dkb@tezu.ernet.in, anjan_sh@rediffmail.com abstract association rules identify associations among data items and were introduced in 1993 by agarwal et al.. most of the algorithms to find association rules deal with the static databases. there are very few algorithms that deal with dynamic databases. the most classical algorithm to find association rules in dynamic database is borders algorithm. this paper presents two modified version of the borders algorithm called modified borders. experimental results show that the modified version performs better than the borders algorithm in terms of execution time. to address the scalability issue, the paper also proposes a distributed version of the borders algorithm, called distributed borders. keywords: rule mining, itemset, frequent items, support, confidence, border sets, promoted border sets. introduction association rule is one of the most vital areas of research in data mining and was introduced by agarwal et. al in 1993 [agarwal(1993)]. association rules are of the form “80% of the customers who buy bread also buy butter". in other words association rules find the influence of one set of items on another set of items. association rules have got numerous applications in real world such as decision support, understanding customer behavior, telecommunication alarm diagnosis and prediction, etc. the departmental stores also can use association rules in many fields such as catalog design, add-on sales, stored layout etc. the terms most frequently used in relation to association rules are itemset, support, confidence, frequent itemsets . an itemset means a non-empty set of items. the support of an itemset x is defined as the % of transactions/records in a database that contains x. an itemset with support greater than the some pre-defined minimum support is called a frequent or large itemset. an association rule between two disjoint and frequent itemsets x and y exists, if x ∪ y is frequent and confidence is at least some pre-defined value. confidence of an association rule between x and y is defined as the percentages of transactions/records that contain x also contain y. finding frequent itemset is one important step in association rule mining. there are some influential algorithms to find the frequent itemsets from large databases. the most classical algorithm is aprior (agarwal(1994)). some other important algorithms are aprioritid [agarwal(1994)], fp-tree [han(2000)], partition algorithm[savasere (1995)], dic algorithm[brin(1997)] etc. one general assumption in all the above-mentioned algorithms is that database is static. however in reality, most of the databases are dynamic and are updated frequently i.e. new records are added, old records are deleted and existing records are modified frequently. so the itemsets, which are frequent, may not be frequent when the database is updated and the itemsets, which were not frequent, may become frequent when the database is updated. moreover, new database may contain some new interesting rules, which were not present in the old database. one obvious technique is re-running association rule mining algorithms ajis vol 13, no. 1 september 2005 20 in the updated database to find the frequent itemsets in the updated database. however, this is not the optimal solution because of the following reasons. 1. it will require running the algorithms on adding, deleting or updating a small number of records. 2. it will take too much time because it will scan the same database every time. 3. it will generate most of the itemsets repeatedly literature shows that there are some algorithms to find frequent itemsets in dynamic databases. some of them are fup [cheung(1996)], fup2[cheung(1997)] ,deli[lee(1998)], maap[ezeife(2002)]. some more works can be found in [thomas(1997), feldman(1997)]. the main requirements of a dynamic association rulemining algorithm are 1. it should be able to use the already discovered frequent itemsets to discover new frequent itemsets. 2. it should not have to scan the old records/transactions. 3. it should scan the new records/transactions as few number of times as possible. the most popular and important algorithm, which follows the above criteria, to find frequent itemsets in dynamic database is the borders algorithm [feldman(1999)]. this algorithm has used the concept of border and promoted border set to update the frequent itemsets. however, the algorithm suffers from scalability problem and cannot be used in distributed environment. to address these problems, this chapter presents a modified version of borders algorithm, which takes less time than that of borders algorithm. this chapter also presents a distributed border algorithm, which is meant for distributed dynamic databases. symbols given in table 1 are used to explain the algorithms. borders algorithm feldman et. al. [feldman(1999)] proposed borders algorithm, which uses the concept of border set. there are three variations of borders algorithm : addition of transactions/records; addition and deletion of transactions/records ; changing of minimum support threshold. the main characteristics of the algorithm are as follows. entire database is scanned only when new candidate sets are generated. there are only few candidates for which support is counted even if entire database is scanned. ajis vol 13, no. 1 september 2005 21 table 1: symbols d database |d| number of records/transactions in the database d. tid transaction id. x ,y itemsets. s(x) support of x in % si the site i n number of sites. α minimum support in % β positive real number ( < α ) told i old database at the site i tnew i incremental database at site i told the old database ( ∪ told i ) t new the newly added records/transactions (∪tnew i ) tdel records/transactions to be deleted. twhole i told i ∪ tnew i twhole the whole database (told ∪ t new) tdel lold set of frequent itemsets in told (with local support) bold set of border itemsets in told (with local support). bold / first border set in told bold // second border set in told bwhole / first border set in twhole bwhole // second border set in twhole.\\ lwhole i frequent itemsets in twhole at the site i lwhole set of frequent itemsets in twhole bwhole set of border itemsets in twhole b i promoted border itemsets at the site i b promoted border set (∪ bi ) f i frequent itemsets in the updated database at the site i f frequent itemsets in the updated database (∪ f i). l whole (i) x, x ∈ l whole and |x| = i b whole i border itemsets in t whole at the site i b(i) b ∈ b and |b|=i li frequent itemset of size i c i candidate itemset of size i c set of all candidates. c a candidate itemset. s(x) y support count of the itemset x in the database y x.sup.new i support of x at t new i x.sup.new support of x at t new x.sup i local support of the itemset x at the site i x.sup global support of x s(x) y i support of x at the site i for the database y. ajis vol 13, no. 1 september 2005 22 figure 1 : border set actually, manila and toivonenn gave concept of border set. an itemset x is called a border set if x is not frequent, but all its proper subsets are frequent. so collection of border sets forms the border line between the frequent sets and non-frequent sets. an itemset that was a border set before the database was updated and has become frequent after the database has been updated is called a promoted border set. borders algorithm also uses the same concept and maintains support counts for all the frequent sets as well as border sets. input : t new , told α, lold and bold output : lwhole and bwhole scan t new and increment the support count of x ∈ lold ∪ bold b :={x | x ∈ bold and s(x) twhole ≥ α} lwhole := b ∪ {x | x ∈ lold and s(x) twhole ≥ α } b whole = {x | ∀ x ∈ x, x -{x} ∈ l whole } m= max {i | b(i) ≠ φ } candidate-generation : l0 = φ, i =1 while( li ≠ φ or i ≤ m) do ci+1= { x = s1 ∪ s2 | (i) |x|=i+1, (ii) ∃ x ∈ s1, s1 -{x}∈ b(i) ∪ li (iii) ∀ x ∈ s2, s2 -{x} ∈ l whole(i) ∪ li } scan twhole and obtain s(x) for all x ∈ ci+1 li+1 = { x | x ∈ ci+1 and s(x) ≥ α} ajis vol 13, no. 1 september 2005 23 l whole = l whole ∪ l i+1 b whole = bwhole ∪ ci+1 l i+1} i = i+1 enddo figure 2: borders algorithm (addition) the algorithm is based on the observation that a set is required to be considered as a candidate set only if it has a subset that is a promoted border. the following lemma proves this observation. in addition to the lemma, the paper [feldman(1999)] also proved that borders algorithm is correct. lemma: if x is an itemset which is frequent in twhole and not frequent told , then there exist a subset y ∈ x such that y is promoted border. proof : let y be a minimal cardinality subset of x , which is frequent in t whole , but not in t old. so all proper subsets of y are frequent in t whole. however, by minimality of y, none of these subsets is a new frequent set in t whole. so y is a border set in told. given l old and b old, the task of the borders algorithm is to find l whole and b whole. the algorithm (addition) is presented in figure 2 . the algorithm assumes that l old and bold are known with their respective supports. l old and bold can be found by any association rule mining algorithms like apriori. the algorithm starts by making one pass over the new database t new and counts the supports of the itemsets of l old ∪ bold for tnew . during this pass the algorithm calculates b and l whole. if b is null, then l whole contains all the frequent sets of twhole. however, if there is at least one promoted border (i.e. b is not null), then the algorithm generates new candidate sets ci+1 (steps 8-10 ) and scans over the entire database to count the support of them (steps 11-13 ). then it finds the frequent itemsets li+1 based on the support count and updates lwhole and b whole. this process continues as long as new candidates can be generated. so the algorithm scans the whole database, if there is some promoted border set. otherwise, it does not require scanning the whole database. an example let us take told and tnew as given in table 2 and table 3 respectively and the support be 40%. a1 a2 a3 a4 a5 1 1 1 0 0 0 0 0 1 1 1 0 1 0 0 0 0 0 0 1 1 1 1 0 0 ajis vol 13, no. 1 september 2005 24 table 2 : told a1 a2 a3 a4 a5 0 0 1 0 0 0 0 0 0 1 1 0 0 0 0 1 1 0 0 1 1 0 1 0 0 table 3 : tnew then, lold is {(a1,a2,a3,a5), (a1a3), (a1a5), (a3a5), (a1a3a5)} and bold is { (a4), (a1a2), (a2a3), (a2a5)}. next, t new is added with t old. now, t new is scanned to calculate b, lwhole and b whole. b = {(a4)} l whole = {a4 , a1 , a5, a1a5)} b whole = {(a1a4), (a4a5), (a1a5)} here, we have got one promoted border (a4). so, we have to generate new candidate itemsets. new candidates are generated in level-wise fashion as in apriori. candidate 2-itemsets are c2 = { (a1a4) , (a4a5)} candidate 3-itemsets are c3 = { (a1a4a5) } now the whole database t whole is to be scanned to update l whole and b whole. discussion : the borders algorithm is robust enough to find the frequent itemsets in a dynamic database. however, from our experimental study, it has been observed that • with the increase in the volume of the told and t new, the cost of the scanning of t whole in the every iteration becomes too expensive. • it suffers from scalability problem. to overcome the above problems, this paper proposes two enhanced versions of the present borders (addition) algorithm. these are modified borders and distributed borders. modified borders has used two border sets to reduce scanning of entire database. on the other hand, distributed borders is the extension of borders in distributed environment. modified borders algorithm as it is mentioned above, borders has to scan entire database when there are some promoted borders. however, scanning entire database is very expensive, particularly, when the database is large. scanning entire database can be avoided if new candidates are not generated frequently. new candidates are generated, if there is even one promoted borders because borders are not included to generate candidate in the old database. so, if borders are included to generate candidates in the old database, then there will be no new candidates. however, it will be infeasible to include all the borders to generate candidates in the old database. this concept led us to include some of the borders, which are likely to become promoted border, to generate candidate sets in the old database so that if those borders become promoted, no new candidates will have to be generated. new candidates ajis vol 13, no. 1 september 2005 25 will be generated only when some borders, which were not included to generate candidates in the old database, becomes promoted. based on the above discussion, borders has been modified by including two border sets. the first border set is bold / and the second border set is bold // . bold / is calculated as { x | ∀ x ∈ x, x-{x} ∈ (l old ∪ b old / ); s(x) ≥ β and s(x) < α }. bold // is calculated as {x | ∀ x ∈ x, x-{x}∈ (l old ∪ b old / ) ; s(x) < β }. bold / and lold take part in candidate generation, whereas the elements of bold // are not used in candidate generation. another requirement in the algorithm is that all the subsets of an itemset x ∈ (bold / ∪ bold // ) must be ∈ ( lold ∪ bold / ). obviously, bold / contains the itemsets with higher probability of becoming promoted when new transactions are added. new candidate sets will be generated only when any of the elements of the bold // becomes promoted. if new candidate itemsets are generated, one scan over the whole database is required to find supports of the new candidates. the algorithm the algorithm works as follows. lold, bold / and bold // are assumed to be known with their respective support counts. the algorithm starts by making one pass over the new database tnew and updates supports of the elements of lold ∪ b old / ∪ bold // . during the pass, the algorithm generates four categories of itemsets b / , b // , b /// and b //// . if b // is null, then no new candidate set is required to be generated. if b // contains at least one itemset, then new candidate sets are required to be generated. if new candidate sets are generated, the algorithm makes one pass over the entire database to count the support of the new candidate sets. at the end, the algorithm generates lwhole, bwhole / and bwhole // , which are counterparts of the lold , bold / and bold // respectively, for the whole database twhole. the algorithm is presented in the figure 3. input : tnew , told , α, β , lold and bold / , bold // output : lwhole and bwhole / , bwhole // scan tnew and increment the support count of x ∈ (lold ∪ bold / ∪ bold // ) b / = {x | x ∈ bold / and s(x)t whole ≥ α} b // = {x | x ∈ bold // and s(x)t whole ≥ α } lwhole = b / ∪ b // ∪ {x | x ∈ l old and s(x)t whole ≥ α } b /// = {x | x ∈ bold // ; ∀ x ∈ x, x -{x} ∈ lwhole; s(x) twhole ≥ β and s(x)twhole < α } b //// = {x | x ∈bold / ∪ lold ; ∀ x ∈ x, x -{x} ∈ lwhole ; ( s(x)twhole ≥ β and s(x)twhole < α )} bwhole / = b /// ∪ b //// bwhole // = {x | ∀ x ∈ x, x -{x} ∈ lwhole and s(x) < β } if b // ≠ φ then m = max {i | b // (i) ≠ φ } candidate-generation: l0 = φ , b0 = φ, k=2 while (lk-1 ≠ φ or bk-1 ≠ φ or k ≤ m+1) do ck = φ l = b // ( k-1) ∪ lk -1 ∪ b /// (k-1) ∪ b k-1 m = l k-1 ∪ l whole (k-1) ∪ b whole / (k-1) for all itemsets in l1 ∈ l do begin ajis vol 13, no. 1 september 2005 26 for all itemsets in l2 ∈ m do begin if l1 [i] = l2[i] ( 1 ≤ i ≤ k-2 ) and l1[k-1] < l2[k-1] then c = {l1 [1], l1[2], …l1[k-2], l1[k-1], l2[k-1]} ck = ck ∪ c end for end for prune ck : all the subsets of ck of size k-1 must be present in m ; scan twhole and obtain support s(x) for all x ∈ ck lk = {x | x ∈ ck and s(x) ≥ α } l whole=lwhole ∪ lk b k = {x |x ∈ (ck lk ); ∀ x ∈ x, x-{x} ∈ l whole; s(x) ≥ β and s(x) < α) bwhole / = bwhole / ∪ bk bwhole // = bwhole // ∪ {x |x ∈ (ck lk); ∀ x ∈ x, x-{x} ∈ lwhole ; s(x) < β ) k = k+1 ; enddo figure 3: modified borders algorithm an example let us consider told as given in table 4 and assume α = 40% and β = 30%. a b c d e 1 1 0 0 1 0 0 1 0 0 0 0 0 1 0 0 0 0 1 0 0 1 0 1 0 1 1 1 0 0 1 0 0 0 0 0 0 0 0 0 1 1 0 1 0 table 4 : told : sample market -basket database now, lold, bold / , bold // are obtained as given below. lold = {a, b, d , (ab)} bold / = {(c)} bold // = {(e), (ad , bd, ac, bc, cd} suppose, tnew (table 5 ) is added to the t old ( table 4 ). when tnew is scanned, b / , b // , b /// b////, lwhole, bwhole / and bwhole // are obtained as {c}, {e}, φ, φ, {a, b, d, (c), (e), (ab) }, φ and φ respectively. since b// is not null, new candidate itemsets will have to be generated. new candidate itemsets, after pruning, will be {(ae,be,ce,de), (abe) }. then, entire database is scanned to find the support count of the new candidates. ajis vol 13, no. 1 september 2005 27 a b c d e 0 0 1 1 1 1 1 0 0 1 1 1 1 0 1 0 0 1 0 1 table 5 : tnew experimental results we compared modified borders with borders algorithm using two synthetic databases and one real database. both the algorithms were implemented on a intel piv based ws (with 256 mb sdram). lold, bold , bold / and bold // had been computed separately. test data : we used synthetic databases, which were generated using the technique given in [agarwal(1996)], and the connect-4 dataset, which were downloaded from uci machine learning repository(www.ics.uci.edu). all the synthetic datasets contained 100k records and dimensionality of each record is 255. other parameters for synthetic databases are shown in the table 6 . for all the experiments, initial sizes of the synthetic databases and connect-4 were taken as 80k records and 47557 records respectively. data set |t| [i] |d| t20i4100k 20 4 100k t20i6100k 20 6 100k table 6 : parameters used for synthetic data generation we executed each of the experiments several times. tables 7 through 9 show average number of full database scan required in borders and modified borders as the value of β increased from 3.5% to 4.5% and size of incremental database is increased from 5k records to 20k records. the value of α (minimum support) was taken as 5 % for all the experiments. average execution times over all increments are given in figures 4 through 6. table 7 : database:t20i4100k table 8 : database:t20i6100k increment borders modified (β =3.5%) modified (β =4%) modified ( β =4.5%) 5k 1 0 0 1 10k 3 0 1 2 15k 2 0 0 2 20k 3 1 1 3 increment borders modified (β=3.5%) modified (β =4%) modified (β =4.5%) 5k 0 0 0 0 10k 3 0 0 2 15k 2 1 1 1 20k 3 0 1 2 ajis vol 13, no. 1 september 2005 28 table 9 : database:connect4 figure 4 : average execution time increment borders modified (β = 3.5%) modified (β=4%) modified ( β =4.5%) 5k 0 0 0 0 10k 1 0 1 1 15k 1 0 1 1 20k 2 1 1 1 ajis vol 13, no. 1 september 2005 29 figure 5 : average execution time ajis vol 13, no. 1 september 2005 30 figure 6 : average execution time observations following are some observations made from the experimental results. • experimental results (tables 7, 8 & 9) show that borders algorithm requires whole scan of the database several number of times, whereas modified borders requires whole scan of database a few number of times. • the value of β has a great effect on the performance of modified borders algorithm. as the value of β increases, number of whole scan also increases. in our experiments, we found that when β =4.5%, number of full scan of the database is almost same in both the algorithms. • as far as execution time is concerned, modified borders takes much less time than that of borders when β is small (figures 4, 5 & 6 ). as the value of β increases, modified borders tend to take little more time. this is because number of full scan tends to increase with the increase of value of β. on the selection of the value of ββββ : value of β plays an important role in the algorithm. when β → α, the algorithm tends to become borders algorithm. with the decrease in value of β, performance of modified borders becomes better than borders algorithm in terms of execution time. however, ajis vol 13, no. 1 september 2005 31 with the decrease in value of β , the memory requirement increases due to the additional candidate sets. so value of β cannot be decreased too much. this cost of additional memory requirement is quite negligible in comparison to the requirement of full database scanning, particularly when database is very large. so there should be some trade off in choosing the value of β . if there is not enough memory and the database is dense, the value of β can be set to a higher value. for sparse databases, β can be set to a lower value. here is one simple method to adjust the value of β. for the first increment of the database, β can be set to a lower value. in the subsequent increments, β can be increased a little until number of candidates is manageable and gives desired result. distributed borders algorithm the borders algorithm is sequential in nature and meant for centralized database. however, most of the databases are distributed in nature. distributed borders algorithm is basically meant for distributed dynamic databases. however, it can also be used in a centralized database by partitioning the database and placing the partitions in different nodes of distributed systems. this process reduces the number of candidate sets to a great extent resulting in high flexibility, scalability and low cost performance ratio [cheung(1996),31-42]. distributed algorithms posses some problems such as locally frequent or border sets may not be globally frequent or border sets. again, message passing being a costly affair, processing should be confined in the local sites as much as possible. to explain the algorithm, it is assumed that databases of similar structure are distributed in different sites which are networked. let us consider a transactional database, where each record is a transaction in a supermarket made by the customers. each transaction is of the form <tid,1 ,1 ,0...0, 1> . here tid is the transaction id, which is unique for each transaction. 1 and 0 represents the corresponding item has been bought and not bought respectively in the transaction. it is also assumed that the database is horizontally partitioned and allocated in n sites si (i=1, 2, 3...n) in a distributed system. now, the task is to maintain the global frequent itemsets and global border sets in this distributed environment when the database is updated. distributed algorithm for maintaining frequent itemsets in dynamic database here, we have examined the borders algorithm in the distributed environment. let told be the old transaction database distributed in n sites. told i is the old transaction database at the site i (i =1, 2, 3...n). tnew and tnew i are the new transactions to be added to the whole database and to the transactions at the site i respectively. x.sup and x.sup i are the global support count and local support count at the site i respectively of the itemset x. for a given minimum support threshold α , an itemset x is globally frequent in the old database(updated database) if x. sup ≥ α |told| ( x.sup ≥ α |twhole| ). similarly, an itemset x is locally frequent in the old database(updated database) at some site i, if x.sup i ≥ α |told i | (x.sup i ≥ α |twhole i | ). like the borders algorithm, this algorithm also uses the concept of border set and promoted border set. the only difference is that all the concepts have been used in the context of the distributed environment. an itemset x is a global border, if x is not globally frequent, but all its subsets are globally frequent. an itemset x becomes globally promoted border on adding the new transactions, if x is a globally border in the old database and globally frequent in the updated database. lold is the global frequent itemsets in the old database and bold is the global border sets in the old database. ajis vol 13, no. 1 september 2005 32 given lold and bold, the problem here is to find updated frequent itemsets lwhole and border sets bwhole for the updated database twhole. the main purpose of the distributed borders algorithm is to reduce the number of candidate sets and in turn reduce the number of messages to be passed across the network and execution time. to reduce the number of messages, we used polling technique as discussed in dma [cheung(1996,911-921)]. some interesting observations, which are listed below, can be made relating to frequent, border and promoted border sets in large database in distributed environment. some of these observations were discussed in [cheung(1996,911-921)]. 1. every global frequent itemset x must be frequent in at least one site si. 2. if an itemset x is locally frequent at some site si, than all its subsets are also locally frequent in the site si. 3. if an itemset x is globally frequent at some site s i, then all its subsets are also globally frequent at the site si. 4. if an itemset x is globally frequent or promoted border, then x must be frequent in at least one site i. proof: this is obvious because if an itemset x is small in all the sites, it cannot be frequent in the whole database.  5. if an itemset x is a global promoted border set, then x must be frequent in tnew i for some site si. proof: let an itemset x is a promoted border set in the updated database twhole. then s(x) < α |told | and s(x) ≥ α |twhole|. since twhole = told ∪ tnew , s(x) ≥ α |tnew|. therefore s(x) ≥ α ∑ |tnew i .| . hence, s(x) ≥ α |tnew i | for at least one site si.  6. if x is global border set, then there exist a y ⊂ x so that y is local border in some site si. proof: if x is a global border set, then x must be small/infrequent in at least one site si. therefore, there exist at least one subset of x, which is a local border set in the site.  7. if a new candidate set c ∈ twhole has to be frequent or border in twhole , then either c or one of its immediate proper subsets must be locally frequent in one tnew i . proof : if c is a new candidate set there can be two possible cases: a) c is frequent in the updated database twhole : in this case c must be frequent in tnew i.e. c must be frequent in tnew i for some i. b) c is a border set in the updated database twhole : in this case c will be small and all of its subsets will be frequent in the updated database twhole. so there exists at least one c / ⊂ c, which was small in the old database told. otherwise, c would have been generated in the old database told. this c / will be frequent in the updated database twhole. so c / must be frequent in the tnew i.e. c / must be frequent in tnew i for some i.  8. if a candidate set x in the updated database is either frequent or border set, all of its immediate proper subsets must be either ∈ (f ∪ b) or frequent in at least one site. proof: there can be two possible cases: a) x is frequent in the updated database: if x is frequent, then all the subsets of x must also be frequent in the updated database. let y ⊂ x. ajis vol 13, no. 1 september 2005 33 then, y is either a candidate set or y ∈ f ∪ b . if y is a candidate and y is frequent, then y must be frequent in at least one site because y cannot be frequent if it is small in all the sites. b) x is border in the updated database: in this case, all the subsets of x must be frequent in the updated database. so, by first option, all the subsets are either ∈ ( f ∪ b ) or frequent in at least one site.  local pruning using the above observations many unnecessary candidates can be pruned locally. if an itemset x is locally small in all the sites, then x cannot be frequent globally. that is why, itemsets are first checked if they are locally frequent or not. their global support are found only when they are locally frequent in at least one site si. similarly some promoted border sets also can be pruned away locally using observation 4. so, if a border set x is not frequent in any site, then it is not tested for global promoted border. observation 7 is also very significant in pruning away candidate sets locally. after the candidate sets are generated, support of the candidate sets are counted in the incremental part tnew i . if any candidate set or at least one of its immediate proper subsets is not frequent in at least one tnew i , it can be pruned away because by observation 7 it can be neither frequent nor border. observation 8 is also helpful in pruning away unnecessary candidate sets. input: lold , bold, tnew i , told i and α output: updated lwhole and bwhole repeat the following steps at each site i distributively. scan tnew i and count the support of all the itemsets x ∈ ( lold ∪ bold ) and find b i = {x|x ∈ bold and s(x)t new i ≥ α | tnew i | } lwhole i = b i ∪ { x | x ∈ lold and s(x)twhole i ≥ α |twhole i | } (by observation 4) broadcast x ∈ lwhole i to other sites along with their supports. prune lwhole i : lwhole i = { x | x ∈ lwhole i and s(x)twhole ≥ α |twhole| } compute b = ∪bi and lwhole = ∪ lwhole i bwhole = {x | ∀ x ∈ x, x-{x} ∈ lwhole } generate candidates: m = max { i | b(i) ≠ φ } i=1 while (li ≠ φ or i ≤ m) do ci+1 = { x = s1 ∪ s2 | (i) |x|=i+1, (ii) ∃ x ∈ s1, s1{x} ∈ b(i) ∪ li , (iii) ∀ x ∈ s2, s2-{x} ∈ l whole (i) ∪ li } scan tnew i and compute x.sup.new i for all x ∈ ci+1. remove any candidate set x ∈ ci+1, which is or at least one of its immediate proper subsets is not frequent in tnew i (by observation 7). scan told i and find the support x.sup i for all x ∈ ci+1 ( tnew i has already been scanned) ci+1 = {x ∈ ci+1 | ∀ x ∈ x, y= x-{x}, y ∈ lwhole or y.sup i ≥ α |twhol e i | } (by ajis vol 13, no. 1 september 2005 34 observation 8) find the global support for all x ∈ ci+1 as x.sup = ∑ x.sup i li+1 = {x | x ∈ ci+1 and x.sup ≥ α } lwhole = lwhole ∪ li+1 bwhole = bwhole ∪ (ci+1 li+1) i=i+1 enddo return lwhole and bwhole figure 7 : : distributed borders algorithm explanation of the algorithm it is assumed that the lold and bold are available with the local support to all the sites. the algorithm starts with scanning the incremental portion tnew i and finds local support for all x ∈ lold ∪ bold. this is because frequent sets and border sets, which are frequent locally in at least one site, can only be frequent globally. then comes the step 2, which finds the global support for lwhole i this can be done by simply broadcasting the local support of x ∈ l whole i . if all the items are broadcast to all the sites, then for each item x, o(n 2 ) messages will be required, where n is the number of sites. so polling techniques as described in [cheung(1996,31-42)] can be used. this technique reduces the number of messages to o(n) for each itemset. step 3 prunes away all x ∈ lwhole i , which are not globally frequent. then the step 4 just broadcasts the lwhole i to other sites and receives the same from other sites to compute lwhole and b. it can be noted that, all the sites will be having the same set of lwhole and b. steps 6-11 are responsible for generating the candidate sets. the candidate sets are generated in the step-wise method like apriori. some kind of pruning techniques are required to prune away some unnecessary candidate sets. observation 4 helps prune away some candidate sets. steps 13-15 are basically pruning steps. it scans the tnew i and finds the x.sup.new i for all the candidate sets. according to observation 4, a candidate x can be neither frequent nor border, if neither x is frequent nor at least one of its immediate proper subsets is frequent in any tnew i . so the candidates, which do not conform to observation 4, can easily be removed from the candidate sets. step 16 finds the global support for all the candidate sets. polling technique as given in [cheung(1996)] can be used here also. steps 17-19 find frequent itemsets and border sets for the updated database. finally, step 22 returns the global frequent itemsets and global border sets. experimental results we simulated the algorithm on a share-nothing environment. a 10/100 mb lan was used to connect six piii machines running windows nt. each machine had 20gb disk space and 256mb memory. the dataset used in the experiments were t20i4200k and t20i6200k, which were generated using the technique given in [agarwal(1996)]. each datasets contained 200k tuples (transactions). each dataset was partitioned and corresponding partitions were loaded in the machines before the experiments started. data set |t| [i] |d| t20i42000k 20 4 200k t20i6200k 20 6 200k table 10 : parameters used for synthetic data ajis vol 13, no. 1 september 2005 35 we carried out three experiments. in the first experiment, we used three machines (sites). the purpose of the experiment was to find the execution time and number of candidate sets for different minimum support. each machine initially contained 63k transactions and 3k transactions were added to each machine as incremental database. the results are given in figure 8. the second experiment was the scale up experiment. the testbed of the second experiment was same as that of first experiment. here also we used three machines(sites). the purpose of the second experiment was to find the effect of the database size on the execution time. three machines initially contained 30%, 30% and 25% transactions respectively. size of incremental database was 5% for each of the machines and minimum support was 1%. the results are given in figure 9. the third experiment was the speedup experiment. the speedup factor is defined as s(n)=t(1)/t(n) and efficiency is defined as s(n)/n, where t(n) is the execution time with n sites. here, we increased the number of machines(sites) from 1 to 6. sizes of initial database and incremental database were taken as 80% and 20% respectively. initial and incremental database were divided equally among the machines and minimum support was taken as 1% . when we used 1 machine(site), it was the sequential run time of the borders algorithm. the results are given in figure 10. discussion the result of the first experiment was obvious and straightforward. in some cases execution time did not decrease significantly with the increase of minimum support. this was because whole scan of the database might be required in some sites. it was evident from the second experiment that execution time increased with the increase of the size of initial database and incremental database. . however, it increased linearly. third experiment measured speedup and efficiency of the algorithm. we found average efficiency of 63% and 67% for t20i6200k and t20i4200k respectively, which showed that the algorithm achieved sub-linear speedup. however, as with other distributed algorithms, performance of this algorithm also depends on the factors such as database types, distribution of data, skewness of data, network speed and other network related problems. conclusion this paper has presented two enhanced versions of the borders algorithm : modified borders and distributed borders. modified border tried to reduce the execution time by avoiding the full scan of the database in most of the cases. on the other hand, distributed borders is the modification of borders algorithm to make it suitable for distributed dynamic databases. ajis vol 13, no. 1 september 2005 36 figure 8 : execution time ajis vol 13, no. 1 september 2005 37 figure 9 : execution time for different database size ajis vol 13, no. 1 september 2005 38 figure 10 : execution time for different number of sites references: agarwal r, imielinski t and swami a(1993) “mining association rules between sets of items in large databases”, acm sigmod international conference on management of data, may. agarwal r and shrikant r (1994) “fast algorithms for mining association rules in large databases", proceedings of 20 th international conference on very large databases , august-september , santiago, chile, agarwal r, mannila h, shrikant r, toivonen h and verkamo a i(1996). fast discovery of association rules. mit press. brin s, motwani r, tsur d and ullman j(1997), “dynamic itemset counting and implication rules for marketbasket data", proceedings of 1997 sigmod , montreal, canada, june. cheung d w, han j, vincent t ng, w fu ada, and fu y(1996) “a fast distributed algorithm for mining association rules”, proceedings of 4th inernational conference on parallel and distributed information systems , pages 31–42. cheung d w, ng v t, ada w fu and fu y(1996) “efficient mining of association rules in distributed databases”, ieee transactions on knowledge and data engineering , december , 911-921. cheung d w, han j, ng v t and wong c y(1996) “maintenance of discovered association rules in large databases: an incremental updating technique”, 12th ajis vol 13, no. 1 september 2005 39 international conference on data engineering, new orleans, louisiana. cheung d w, lee d w, and kao s d(1997). “a general incremental technique for maintaining discovered association rules”, in proceedings of the fifth international conference on database system for advanced applications, melbourn, australia. ezeife c i and y su(2002). “mining incremental association rules with generalized fp tree”, proceedings of 15 th canadian conference on artificial intelligence, ai2002, calgary, canada, may. feldman r, aumann y, amir y, and mannila h(1997) “efficient algorithms for discovering frequent sets in incremental databases” , proceedings of acm sigmod workshop on research issues on data mining and knowledge discovery(dmkd). feldman r, aumann y and lipshtat o(1999) “borders : an efficient algorithm for association generation in dynamic databases” , journal of intelligent information system, pages 61–73. han j, pei j and yin y (2000) “mining frequent patterns without candidate generation”, proceedings of the 2000 acm-sigmod international conference on management of data, dallas, texas, usa. lee s d, cheung d w and kao b(1998). “is sampling useful in data mining? a case in the maintenance of discovered association rules”, data mining and knowledge discovery. savasere , omiecinski e and navathe s (1995), “an efficient algorithm for mining association rules in large databases”, proceedings of 21st conference on very large databases, zurich, september. thomas s, alsabti k , bodagala s, and s ranka(1997) “an efficient algorithm for the incremental updation of association rules in large databases”, proceedings of the 3rd international conference on kdd and data mining(kdd’97), new port. ajis vol.4 no. 2 may 1997 ethical issues in network system design duncan langford computing laboratory the university canterbury kent, ct2 7nf d.langford@ukc.ac.uk abstract today, most desktop computers and pcs are networked — that is, they have the ability to link to other machines, usually to access data and other information held remotely. such machines may sometimes be connected directly to each other, as part of an office or company computer system. more frequently, however, connected machines are at a considerable distance from each other, typically connected through links to global systems such as the internet, or worldwideweb (www). the networked machine itself may be anything from a powerful company computer with direct internet connections, to a small hobbyist machine, accessing a bulletin board through telephone and modem. it is important to remember that, whatever the type or the location of networked machines, their access to the network, and the network itself, was planned and constructed following deliberate design considerations. in this paper i discuss some ways in which the technical design of computer systems might appropriately be influenced by ethical issues, and examine pressures on computer scientists and others to technically control networkrelated actions perceived as 'unethical'. after examination of the current situation, i draw together the issues, and conclude by suggesting some ethically based recommendations for the future design of networked systems. introduction consensus definitions in this field are sometimes difficult. for example, information systems specialists may argue that any computer network is an information system; while hardware engineers, approaching the same point from a different direction, might consider physically linking one or more computers in such a way that they can share data forms a computer network automatically. computer networks themselves, of course, might be further classified according to their geographical extent: local area network (lan), metropolitan area network (man), wide area network (wan). networks may also be grouped according to the specific protocols used, and indeed by a wide variety of other measures. for our purposes, however, such specialist distinctions are an unnecessary complication. we may consider computer networks as principally falling into just two groups, one a sub-set of the other. these groups, discussed below, are closed networks, and open (or internet linked) networks. to most users, networking design decisions are essentially invisible. in reality, user consideration of networked computer systems operation is probably limited to visible problems — complaints about connection speeds being among the most common. such an emphasis is quite understandable, as designers of networked computer systems have always been principally concerned with technical issues. systems analysis, which provides explicit design criteria, may take on board user pressure for facilities, but traditionally — legal questions aside — designers of networked systems normally only respond to non-technical issues if such consideration is directly requested by a client. focusing of this kind is certainly comprehensible. limited budgets and heavy pressures on development time combine to reduce space for consideration of wider issues, especially problems that are not clearly an essential part of the technical specification. however, whether or not appreciated by client or development team, there are wider questions that have an important role to play in the development of modern networked systems. increasingly, society is no longer prepared to allow the free development of networked computer systems — especially globally networked systems. the definition of 'network design' used in this paper is consequently broad. it covers not only the physical arrangement of connections and links, but also the negotiated steps through which individual users must pass to obtain networked communication. as an example, consider a user employing a service provider in order to obtain access to an internet hub. to obtain a view of their local-to-user facilities, aspects such as links to service provider, links from service provider to hub — and, of course, the internet itself— all need to be examined. distance between computing systems is not considered (as this is a communications media problem); the size of the computing system is similarly irrelevant. my intention is to look beyond the immediate appearance of a networked system, to what lies behind it. 127 ajis vol. 4 no. 2 may 1997 closed networks the simplest form of networking is probably micro-micro linking. this is usually in the form of an uncomplicated connection of one pc to another, so each may exchange information and data. further machines may be added, in the form micro-micro*n (where n is a potentially large number). the result is the linking of a group of computers together, so that each may communicate and share information with any or all of the others. such a connection, which typically permits local email (electronic mail) together with exchange of data and files, is probably the most common type of office system. figure one, below, represents one typical way in which a closed group of computers may be linked. 1 id 1 c 1 ed figure one: atypical 'closed' network the original small loop may of course be extended, to take in further machines located elsewhere in a building. additionally, a series of such network loops may themselves be linked together, or even (using telephone or other links) connected with machines and similar networks in different geographical locations. although this may appear to be moving beyond the definition of a closed network, the essential aspect of a closed system remains: however large a 'closed' system may become, no connection is ever possible to machines outside the organisation. both closed and open networks may need a 'server', which in this context is a dedicated machine used to handle communication issues. the client-server communication is normally in the form of request and response messages — a client machine sends requests to the server, and reads responses returned by the server. this dialogue permits, for example, the exchange of electronic mail, as well as other communication matters, such as file transfers. exact details of the request and response formats are less important than that they occur. it is particularly important to note that in these circumstances communications are invisibly handled by a computer that is outside the direct control of the user. closed networks, then, are those where a potentially large number of computers are linked together. the size of the network and the physical location of the involved computers are irrelevant, provided no link has been made which extends communication beyond the limits of the concerned organisation. ethical issues in closed networks it must be remembered that to ensure effective operation, all computer networks must be administered, or serviced. by definition, the individual carrying out administrative duties has to be allowed complete control of 'their' system. this inevitably means that an individual user is totally dependent upon the ethical views of their administrator. if a relaxed view is taken on examination of data, no file can be private; no mail safe. if an organisation has not established clear rules and guidelines, there is little beyond an individual sense of responsibility to prevent a network administrator misusing their privileges. such misuse is, of course, quite invisible to individual users, who may be quite unaware of any interference with their data. sometimes, a company itself may rely on unethical network systems administration: dave was employed as a computer specialist by a medium-sized company, principally to support their network and electronic mail. it became clear that the technical director, who had originally set up the mail system, maintained a log of all email messages, and made copies of all private mail of interest to him. dave learned he was expected to continue monitoring mail (although, interestingly, was told not to actually read it himself), and to pass on copies of mail from and to selected employees. (langford, 1995) 128 ajis vol.4 no. 2 may 1997 the ethical issues involved in monitoring of networks are clear — users of closed network systems surely need to be aware of any external oversight of their work. it is insufficient and impractical to leave such issues undefined, in the hope that individuals will eventually work something out for themselves. in these circumstances, it is the responsibility of management to clarify company expectations of employee behaviour. if the sort of monitoring experienced by dave is felt to be necessary, management should at least be open about it. secret monitoring of network use can be hard to justify ethically. open networks as would be expected, an 'open' network provides all the resources of a closed network system, but with a significant added link. this link, normally made through an internet service provider (isp), gives an additional channel that allows communication to the internet — and hence to other computers and networks throughout the world. figure two illustrates a typical arrangement. the original computer (or closed network) located at (a) establishes links with a 'proxy server', (b). this server can act as a local store of external information — to prevent frequent reloading of the same data. it is directly controlled by company (a), who may monitor and set limits on its use. the use of a proxy server has an additional positive role, in providing a 'firewall', helping prevent attack by external 'hackers' on company systems. a. b. c. d. figure two: atypical 'open' network this server in turn is connected to an external 'service provider'. there has been an explosive growth in service providers — companies who exist in order to sell internet connections. they are very roughly analogous to those businesses who sell portable telephones, and who, without themselves owning a network, arrange connection to one of the mobile 'phone networks. their activities are beyond the immediate control of those purchasing their services, and even further beyond the individual user of a company computer. all internet service providers must make a further connection, to a national 'backbone site'; this site (which is, again, under the control of a different set of administrators) is directly connected to the wider internet. this series of network links is always followed, except in the case of a single user. here, rather than using a proxy/server, the proxy stage is often omitted. personal computer and small company users normally establish a direct link to their service provider, generally by telephoning a special modem number. ethical issues in open networks we may have a situation where an individual company user may wish, for example, to access a wordwideweb site in another country. figure two shows the steps that must be progressed; but what does passing through this succession of links mean in practice? first, only the original computer is under the direct control of the user. (in the case of a company computer network, of course, even this may not be true.) all other stages in the connection of user to internet are, inevitably, dependent upon others, and may consequently generally be controlled and influenced by factors that are external to the user. such control can be, and normally is, invisible. secondly, because access to the internet involves access to, and distribution of, information and data, it is important to appreciate that specific access might be limited or controlled. thirdly, the technical limitations inherent in a networked system can themselves restrict access. for example, there is a straightforward relationship between the volume of traffic carried by a network, and its functional efficiency. the dramatic growth of internet traffic has resulted in difficulties in speed and access to many users. such problems — despite increases in network bandwidth — can only grow worse. the demands of large 129 ajis vol. 4 no. 2 may 1997 numbers of new internet users are compounded by technical advances, such as the bandwidth demands of full motion internet video. routing algorithms and hardware decisions taken to offset these problems must inevitably affect individual users. specific issues so far i have discussed the way in which a simple closed network is controlled, and have examined the stages through which information must pass when a closed network becomes 'open', through connection to the wider internet. i will now look briefly at each level of network control, identifying specific points that may repay further ethical investigation. local network management a local network administrator has the ability to access, modify and delete all files of all local users. it is therefore important that the responsible managers have clearly defined their expectations of their administrators, and that individual administrators are working within clear and understandable rules and guidelines. furthermore, it is essential that users of the networked system are aware of measures that are taken to control the system they use, and are told, for example, if specific monitoring procedures are in place. while it is reasonable for a company to ensure its technical resources are not being used to support the private interests of its staff, secret monitoring and logging of staff access to web sites may be hard to justify ethically. a local proxy/server has the ability to keep a detailed log file of its operations — monitoring of use is consequently trivially easy. should employers monitor this information? james derk works for the indiana based evansville courier. his views are those of a concerned and informed professional: the original post (on the subject of newsroom internet monitoring) said nothing about reading personal e-mail. it talked about looking in the netscape cache, which i agreed was potentially slimy. however it is certainly less an invasion of privacy--on a company owned computer-than reading e-mail, which i can't condone under any circumstances. it's also dreadfully easy. (ire mailing list, 1996) company oversight of employee network use is clearly an ethical issue; but there is an additional way in which a local proxy may be used to control individual users. this is by forbidding them access to certain sites that are felt 'unsuitable'. this restriction is directly comparable with the employment, by an individual user, of an application such as 'net nanny'. (one of a number of commercial applications that interfere with the operation of an internet browser.) individual network users may wish to prevent, for example, their children accessing pornography. restriction of this sort, whether by employer or parent, is understandable, and may be perfectly justifiable, with certain provisos. not the least important proviso is that the censoring process is 'open', and the justification for it made clear. however, use of nanny-type applications may not be without further ethical problems. recent disturbing allegations concerning the nature of these programs have emerged from the usa. commercial gate keeping applications must, inevitably, contain a list of 'forbidden' internet sites and news groups, those which they consider should not be accessed. although this information is coded, following a 'hacker' cracking lists for most of the popular gate keeping applications have been revealed. it appears lists of forbidden sites may not be limited to pornography. political and other censorship may also be taking place, all the more disturbing because it is hidden. whether it is true or not — i have been assured by a us journalist of its truth — this case illustrates the relevance of ethical oversight. use of guardian applications and control of local proxy hosts surely need informed consideration and evaluation. service provider (isp) issues internet service providers (isps) simply act as distribution points (and collection points) for internet data. they are in effect acting as a clearing house for information, and, as a commercial service, have to convince customers that their provision is preferable to that of another company. because the service itself— internet connection — is virtually identical, service providers have to maintain a distinction in other ways. the chief ethical problem they face is probably that of the distribution of 'pornography' and other material perceived by some as 130 ajis vol.4 no. 2 may 1997 inappropriate. here they are in a classic cleft stick, bound to upset some customers whatever their choice. if they operate a restricted service, forbidding access to 'undesirable web sites and refusing to relay 'inappropriate' internet news groups (such as the infamous alt.sex. hierarchy) then a large proportion of prospective customers, aware of the free speech ethos of the internet, will just go elsewhere. on the other hand, if such material is made freely available, then the provider is liable to be the focus of public anger, and perhaps legal action. for example, in february 1996, the state prosecutors of bavaria and baden-wuerttemburg were concerned over internet dissemination of neo-nazi ernst zuendel's views. the major international isp compuserve, under threat for providing access, responded by preventing their german subscribers from viewing certain usenet news groups. unfortunately, as there was no easy way of screening out german users, this effected compuserve subscribers globally, and resulted in massive protest. however, despite the enormous inconvenience compuserve's attempted censorship caused, the attempt to completely block access could not have worked without additionally closing down the whole of deutsche telekom. german users could otherwise just use ordinary modems and standard voice lines to access any alternative isp in any country. of course, viewing censorship as a solution to network information problems is not limited to sensitive german authorities. as guardian journalist jack schofield put it: the problem with censoring the internet is that somebody somewhere objects to almost everything. if local german authorities can remove all the right wing content, and all the pornography, then why shouldn't the chinese remove all traces of capitalism? why shouldn't american fundamentalist states ban any sites concerned with evolution? why shouldn't the vatican demand the removal of all references to birth control? indeed, a few countries with radical views could quickly remove all religious, scientific and economic debate from the internet, then all we'd need is a militant vegetarian state to finish things off.... (guardian, 1996) if outright banning is not possible, what about monitoring and checking data, to ensure no illicit material is being conveyed? sadly, this too is impossible. searching the enormous quantity of data that flows through an average isp is not currently remotely practical — even expensive routers can hardly keep up with forwarding the data. there is certainly no spare capacity to comb traffic for suspect material, even if the inevitable delays this would cause were acceptable. it is barely possible that this situation may change as technology develops further, although, of course, so far use of the internet is increasing at a far faster rate than technology. a further ethical problem is related to the 'permission' which isps automatically give to their customers to use the internet, more specifically the ability this provides to easily send limitless quantities of electronic mail. some of this mail is merely uninformed, and annoying to other users. however, some is intended purely for commercial purposes, and sent very widely — 'spamming', in net speak. traditionally, the internet has had a non-commercial philosophy, but even the most tolerant of users can very rapidly grow tired of endless junk email. can such proliferation of mail be controlled? perhaps not easily, although one contributor to a discussion on the uk government's ethics mailing list felt it might, by hitting the isps where it hurt: ...i do see a role for contractual sanctions after the event however. i would suggest that writing an enforceable contract with financial penalties for say, unsolicited commercial email, or commercial spam would be quite straightforward. if enough isp's took the position that such activities were a no-no, those that continued to allow them could simply be declared rogue. if any one who connected through them couldn't access any major services, they wouldn't last long. (uk ethics news group, 1996) acting as the focus for users to obtain access to the internet, isps perhaps inevitably provide a focus for ethical debate, too. their responsibilities, to the communities they serve, the wider population, and the internet itself, have not yet been the subject of informed debate. such debate is perhaps overdue. 'backbone site' issues all practical aspects of operating a national backbone site are related to keeping it operational. several times, speaking to those involved, i was told that the issue is not how to analyse or monitor transmitted data, but how to keep the system running at all. backbone sites are very much at the hard edge of technological network development. backbone sites are even less open to monitoring and control than isps. as discussed above, the only viable option open to a government intent on control of the internet would be to close down their entire communications system — an impossible choice, given the dependence of modern society on international 131 ajis vol.4 no. 2 may 1997 networking thorough internet links. even then, of course, the action would be comparable to sawing off a small branch in an attempt to fell a large tree. summary in this paper i have examined the stages through which an individual user may obtain access to the internet. looking first at the mechanics of setting up a 'closed' network, i urged that the role of a network administrator is considered carefully, and that their supervision of 'their' network is carried out under clearly defined rules and expectations. i then looked at the expanding of 'closed' networks into 'open' ones — networks with an onward connection to the wider internet. the part played by a proxy/server, which provides an essential link between networks, was significant here. a specific issue was use made of data passing through a company owned server — particularly data monitoring by employers. finally, i looked at the part played in network communication by isps — internet service providers. they have a difficult task, in both facilitating their customer's access to internet material, and, potentially, limiting access, too. the way in which this is done is clearly important. it is ethically dubious to secretly restrict users from obtaining desired links, particularly if the censorship is carried out under a political agenda. conclusions although there may be immediate points to be made, there seems a clear case for debate. should the design of a networked system, at whatever scale, have as part of its technical design cycle, deliberate consideration of ethical issues? ought a company which purchases and installs a closed network consider first how that network should be used? when a provider of an internet link has the ability to restrict access in any way, would it be reasonable to insist that such a restriction is made explicit? can we depend upon politicians and government to appreciate the technical impossibility of co-ordinated global internet censorship — the only censorship that could possibly work? the ease with which it is possible to generate such questions shows a debate is long overdue. i suggest that at their heart is the need for acceptance of a single issue — serious consideration of non-technical issues in the design and development of networked computer systems. references ethics group (1996). ethics@ccta.gov.uk, 26 april. guardian (1996). guardian online, 26 january. ire mailing list (1996). investigative reporters & editors mailing list (www.reporter.org), 14 february. langford, d. (1995) practical computer ethics, mcgraw-hill, london. 132 the australasian journal of information systems (ajis) began in 1993 and was based on the simple realisation that there was a need for an academic ‘outlet’ for is research in australia australasian journal of information systems volume 14 number 1 nov 2006 case study: the state of information systems in queensland universities robert w smyth school of information systems queensland university of technology r.smyth@qut.edu.au guy g gable school of information systems queensland university of technology g.gable@qut.edu.au abstract the state of information systems (is) in queensland reflects the highly decentralised nature of the state. relative to its population, queensland has a large number of universities, each of which is engaged in information systems teaching and research. the study reveals little evidence of a distinctive queensland-flavour of information systems. rather, there is a diversity of curriculum approaches and an equally broad range of research foci and approaches to research. two of the state’s regional universities are notable for the relative strength of their is presence, in terms of number of is staff, number of is students and range of campuses across which is is taught. the breadth of topics and approaches to is in queensland is evidenced by the existence of separate, competing is groups in two of the largest universities; in each case, one of the is groups is highly technical in orientation while the other is business oriented. across the nine queensland universities there is wide variability in terms of the administrative location of the information systems academic staff in the university structure. the study assesses the state of is in queensland universities in relation to criteria indicative of the maturity of a discipline. measured against these criteria, information systems in queensland universities cannot be considered a mature, distinct academic discipline. 163 australasian journal of information systems volume 14 number 1 nov 2006 introduction for the information systems community worldwide, this is a period of great turbulence. there has been over the past three years an unprecedented downturn in demand for information technology skills, resulting in a sharp decline in student entry to university it courses. information systems courses have been among those affected by the downturn. following the falloff in demand for information systems courses has been a major re-examination of information systems. old insecurities about the status of information systems as a separate academic discipline have emerged. in universities in queensland, as elsewhere in the world, information systems curricula have been re-analysed with a view to repositioning information systems in response to the falloff in student demand. the changes to information systems curricula have been accompanied in some instances by administrative restructuring of information systems academic staff, driven by the goals of rationalising and economising. in this setting of introspection and change, there is value in analysing the current state of information systems in our universities. from a position of soundly-based understanding of the current status, and some feeling for how this status was arrived at, information systems academics are better placed to plan to take advantage of emerging opportunities and to minimise the impact of identified threats to the information systems discipline, to information systems academics, and to the programs of teaching and research guided by these academics. this study reports on such a study into the state of the information systems discipline in universities, in this instance, universities in the state of queensland. purpose of the queensland study the queensland study aims to document current characteristics of information systems programs and information systems research across universities in the state. as with the broader study, this one also seeks: to assess the strength of the is presence in the state’s universities, to evaluate the maturity of is as an academic discipline, to identify emerging trends in is, and to identify main influences on is in the state’s universities. these aims are to be seen in the context of debate about what information systems is and, significantly, whether is can legitimately claim to be a distinct academic discipline. the research method the queensland study utilises the case study method. the specific case study method applied draws heavily on the approach suggested by yin (2003), incorporating some of the ideas of walsham (1995). in particular, this version of the case study method seeks an interpretive approach, directed at what walsham calls “rich insight”. consistent with yin’s recommendation, the queensland case study utilises a detailed case study protocol. this protocol was developed by the queensland study team members with input and consensus from the all australian study team members. major objectives of the detailed case study protocol were to facilitate: • comparability across the states, • consistency across the individual state case studies, and • efficiency in the data gathering process. 164 australasian journal of information systems volume 14 number 1 nov 2006 the principal data gathering method used in the case study was interviewing. existing documentary and archival material was also gathered to supplement the interview data and to provide some triangulation of observations. the interviews were semi-structured, of about one hour’s duration, with emphasis on broad perceptions by the interviewee on the state of information systems in his/her university, points of differentiation, and distinctive features of information systems in that university and in other queensland universities. theoretical framework guiding the study there is a body of knowledge that suggests that many of the characteristics of information systems are consistent with those observed across emerging disciplines in the early stages of their development. for example, in the early evolution of management as a discipline, some of the characteristics that manifested themselves at that time have been seen more recently in the development of information systems. the framework proposed by ridley (2006) is based on two constructs: (1) degree of professionalisation as a discipline and (2) maturity as a scientific field. both are derived from whitley’s theory of scientific change (1984a, 1984b). the first construct concerns the degree of “professionalisation” of the discipline, which is expected to increase as the impact of local contingencies decreases. where a discipline is not highly professionalised, local contingencies such as political pressures, have high impact. consequently, the degree of professionalisation of is can be indicated by the extent of variation in the nature of is curriculum and research. the second construct has been derived from whitley’s three conditions for the establishment of a distinct scientific field: • scientific reputations both become socially prestigious and “control critical rewards” i.e. those in the discipline have the potential for prestige and power through prominence in that discipline; • standards of research competence and skills become established; • a unique symbol system is developed that allows the exclusion of outsiders and unambiguous communication between initiates within the discipline. the universities in this study in this study, data was gathered from all universities in queensland. there are nine universities in queensland, of which four have their main campus in brisbane. all nine universities teach information systems on at least one campus. the universities and the campus locations where information systems is taught are shown in table 1. university campus locations for is australian catholic university mcauley college brisbane bond university gold coast central queensland university rockhampton; bundaberg; gladstone; mackay; emerald; (international colleges: melbourne; sydney; gold coast; brisbane) griffith university brisbane; logan; gold coast 165 australasian journal of information systems volume 14 number 1 nov 2006 james cook university townsville; cairns; mackay queensland university of technology brisbane (2 campuses); caboolture university of queensland brisbane; ipswich university of the sunshine coast sunshine coast university of southern queensland toowoomba; wide bay table 1 campuses of queensland universities where is is taught relative size of the is presence in the queensland universities in total size, three of the queensland metropolitan universities, university of queensland, queensland university of technology, and griffith university, rank with the largest universities in australia. both university of queensland and qut have total student numbers around 40,000, while griffith student numbers are about 32,000. at the other end of the range, bond university has about 2,600 students, university of the sunshine coast has about 3,200 students, while the queensland branch of the australian catholic university, mcauley college, also has fewer than 5,000 students. the information systems presence at queensland universities, in terms of the number of is students, numbers of is academic staff, and the extent of degree programs and research activities is, not surprisingly, roughly aligned with the size of the universities; larger universities tend to have a stronger is presence than smaller universities. some indication of the size of the is presence in each queensland university is given in table 2. university no. of is academics no. of is students australian catholic university < 5 < 100 bond university 5-10 < 100 central queensland university 15-20 > 1,000 griffith university > 30 > 1,000 james cook university < 5 < 100 queensland university of technology > 30 > 1,000 university of queensland 20-25 > 1,000 university of the sunshine coast 5-10 100-200 university of southern queensland 25-30 > 1,000 table 2 the size of the is presence in qld universities table 2 shows, for each of the universities, a range to indicate the number of staff and to indicate the number of is students. the objective is to provide some relativities in relation to the size of the is presence across the state’s universities. the use of broad ranges also serves to overcome inconsistencies in terms of what respondents deem to be “is academics” and “is students”. in some instances, an is academic may also teach subjects in related disciplines; sessional staff may or may not be reported in full time equivalents. similarly, is students may be seen by some to include only those students who are studying an is major, or equivalent, while others will include all students studying even the equivalent of a minor in is. the number of is academics at a university is perhaps the better guide to the size of the is presence. 166 australasian journal of information systems volume 14 number 1 nov 2006 in general, it can be said that information systems has a strong presence in queensland universities. while the larger universities tend to have the larger is presence, two of the regional universities, which are far smaller than the three large brisbane-based universities, stand out for the relative size of their information systems groups. these two are university of southern queensland (usq) and central queensland university (cqu). at usq, the department of information systems is by far the largest of 7 departments within the faculty of business, with almost 25% of the faculty’s enrolment. another indicator of the relative size of information systems at usq is that “the department of information systems has more students than the whole faculty of engineering”. while the relative size of information systems at cqu is not quite as great as at usq, it is still disproportionately large in terms of student numbers, staff numbers, and range of programs/courses offered, compared with the overall size of the university. the common feature at usq and cqu, largely absent in the other queensland universities, is a very strong external-studies offering in information systems. in fact, about 70% of usq information systems students are external students. both usq and cqu have long pursued a strategy of placing heavy emphasis on external studies. the consequence of heavy external enrolments, allied to a policy of setting up study centres in the australian east-coast capital cities, means that only very small proportions of their information systems students attend at the home campuses of toowoomba and rockhampton respectively. the strength of external studies in information systems at these two regional universities can be seen as a distinctive feature of the state of information systems in universities in queensland. the administrative placement of is in queensland universities across the nine queensland universities there is remarkable variability in terms of the administrative location of the information systems academic staff in the university structure. table 3 shows, firstly, the administrative entity with which the information systems academics at that university are affiliated. it should be noted at this point that it is common to find academics involved with aspects of information systems across a range of departments, schools and faculties in a given university; for instance, some health informatics in a health faculty, information systems auditing in a school of accounting, and so on. in this study, data was collected in connection with only the groups who self-identify as teaching and/or researching information systems in each university. a feature of information systems in queensland universities is that during the period late-2004 to mid-2005, many of the information systems groups had either just participated in restructuring or were about to commence a restructuring process. cqu, griffith, james cook, and usq information systems academics were about to be involved in a review process. bond university information systems staff were involved in restructuring in june 2005 and qut information systems academics were in the process of a restructuring. the downturn in demand for it courses was cited as a major stimulus for restructuring. it can be seen from table 3 that five of the nine queensland universities have information systems academics in a separate school or department. interestingly, no two of these five use exactly the same terminology to describe their information systems administrative entity. the remaining four universities, bond university, james cook university, university of queensland, and university of sunshine coast, have is academic staff placed in administrative entities within other departments, schools or faculties. in these four instances, the administrative entities are identified as discipline groups. 167 australasian journal of information systems volume 14 number 1 nov 2006 university administrative entity home faculty australian catholic university school of business and informatics arts and sciences bond university informal is group within school of information technology business central queensland university school of information systems informatics and communications griffith university school of information communications and technology/informal is group in school of management engineering and information technology/business and law james cook university informal is group in school of business law, business and creative arts queensland university of technology school of information systems information technology university of queensland information systems cluster in business school; data and knowledge engineering division in school of information technology and electrical engineering business, economics and law/ engineering, physical sciences & architecture university of the sunshine coast information systems discipline group in faculty of business business university of southern queensland department of information systems business table 3 administrative placement of the is group in qld universities, june 2005 in turn, the “home” faculty for each information systems group also shows some variability across the nine queensland universities. at jcu, bond, usc and usq, the home or parent faculty for the is academics is business (or some variant on that title). at qut the home faculty for is academics is a faculty of information technology. both griffith university and university of queensland are distinctive in two respects in relation to the administrative placement of their information systems academics: firstly, both griffith university and university of queensland are characterised by two separate information systems groups. at griffith, the smaller of the two groups is within the school of management in the faculty of business and law. the second, and larger, information systems group at griffith university is distinctive in being administratively alongside engineering, in a faculty of engineering and information technology. at university of queensland, the larger of the two information systems groups is in the business school, while the second is group, somewhat akin to griffith, is within a school of information technology and electrical engineering. the placement of the is group at the australian catholic university within a faculty of arts and science appears anomalous until it is observed that acu has only three faculties, the other two being health and education. 168 australasian journal of information systems volume 14 number 1 nov 2006 in terms of independent status for information systems groups in queensland universities, there is evidence of two contradictory trends. on the one hand, at such universities as cqu, griffith and qut, information systems academics have moved into separate, identifiable schools of information systems. on the other hand, at usq a separate school of information technology, formed in 1990 to include both information systems and computer science, was broken up in 1993, with information systems returning to the business faculty. similarly, the faculty of information technology at bond university, which incorporated the information systems group (and computer science) was disbanded in june 2005, with all information technology being absorbed, as a new school of information technology, within the faculty of business. so, while some information systems groups have been moving out of business faculties in queensland, others have been moving back into business. sherer (2002) asserts that both the theoretical basis of the discipline and the curricular needs of the professional community influence the organisational placement of the information systems group. the assertion seems most plausible. if we assume that: 1. theoretical basis of the discipline, and 2. perceived curricular needs of the local community, are two significant determinants of the administrative placement of is groups, it is possible to make inferences about these factors from the differing placements in universities across the state. these inferences can provide useful insights regarding the maturity of information systems as a distinct academic discipline. this matter of maturity of the discipline is analysed in a later section of the report. distinctive features of information systems curriculum the queensland universities offer a wide range of both undergraduate and postgraduate information systems courses/programs. table 4 summarises the main information systems courses currently presented in the queensland universities. it is clear from the table that there is much variety in information systems courses across the state. at the coarsest level of analysis, it can be seen that the information systems courses bear a wide range of nomenclatures. within these obvious differences in degree names, in the queensland universities there is considerable further variability in information systems curriculum content. one obvious area of variability in curriculum content relates to the amount of “technical” emphasis in different courses. at qut and bond university, for instance, at least twenty of the twenty-four units in the undergraduate information systems degree course are from the faculty of information technology. the information systems units offered within the school of information technology and electrical engineering at university of queensland are characterised by the fact that every one of them has a technical focus. by contrast, the information systems undergraduate degrees in the universities where the is group is located within a business faculty tend to include a number (typically 4-6) of compulsory business units. the same variability in is curriculum content is to be observed in the coursework postgraduate information systems courses across the queensland universities. university undergraduate courses/programs postgraduate courses/programs australian catholic university bis mis bond university bis (discontinued after school’s move to faculty of business) bit(is) graddipit mit(prof) mit(exec) mit(hons) mitm mec 169 australasian journal of information systems volume 14 number 1 nov 2006 central queensland university bbus(is) bec bit (bio-informatics) gradcertis graddipismanagement mis gradcert health informatics graddip health informatics master of health informatics phd griffith university bit bbus(commercial comp) mit mit(advanced) mecom mis mis (advanced) msofteng mstrategicinfsysmgt phd james cook university bcom mcom mba-minftech phd queensland university of technology binftech(is) binftech(hons) graddipinftech minftech minftech(advanced) phd university of queensland bcom becom bbusman binftech mcom(is) mcom(ecom) msc(comp sc) meng minftech phd university of the sunshine coast bbus(is) bict graddipis minftech(research) phd university of southern queensland bbus bit graddipinftech graddipinfsys minfsys mit(research) mit(prof) phd table 4 information systems courses in queensland universities another aspect of variability of information systems curricula across queensland universities is in the demarcation between information systems and related discipline areas in relation to which discipline area has curriculum responsibility for specific topic areas. for instance, at cqu no programming subjects are controlled by information systems staff, being, instead, the domain of computer science staff. yet cqu has strong representation in its course curricula from such topic areas as management support systems and health informatics, topic areas covered by other 170 australasian journal of information systems volume 14 number 1 nov 2006 faculties in some other queensland universities. topic areas such as database and data communications, as well as a range of web-related topics, are other examples of curriculum areas that lie within the ambit of the information systems academic departments at some queensland universities and, yet, with other departments at other universities. distinctive features of information systems research as with information systems curriculum, diversity is a feature of information systems research in queensland universities. table 5 summarises major foci for information systems research in queensland universities and the organisational approaches to promote research. again, the diversity of topics highlights the breadth that appears to characterise information systems. once again also, the research foci of the individual universities do not appear to be closely related to geographical factors local to each university. instead, the research areas appear to reflect the specific interests and skills of the academic research leaders. again, there is evidence from some of the queensland universities of a policy to deliberately focus on global information systems issues in preference to purely local ones. nor is there any consistent pattern in the organisational approaches of the universities in seeking to promote and support information systems research in queensland universities. while some universities have established formal groups and programs and research centres, others, like the two information systems groups at university of queensland, which has a long tradition of research, rely on voluntary collaborations. some of the groupings have a tight information systems focus, while others, like james cook university, foster broad cross-discipline collaborations. university areas of is research focus is research groupings australian catholic university business requirements definition; software quality assurance; management of information systems; systems modelling and simulation individual bond university smart supply chain; business intelligence smart enterprise centre central queensland university health informatics; group solutions (gdss); teaching and learning; multimedia no research centres; research clusters from across schools in the faculty griffith university software quality; packaged software; decision support systems; programming methodology; gender and it; information systems analysis, design, development and implementation; information systems security; information systems strategy; end user issues; knowledge management; rural information systems software quality institute; institute for integrated and intelligent systems; is group in department of management james cook university people, identity and place: intellectual, social, economic and cultural dynamics; gender and it collaborative; crossfaculty 171 australasian journal of information systems volume 14 number 1 nov 2006 queensland university of technology it professional services: knowledge management; enterprise systems success factors; information technology sourcing ; the management consulting process ; information management in business processes; erp life cycle knowledge management; business process management; workflow patterns; workflow tools; web service design & implementation centre for information technology innovation: it professional services (itps) research program; business process management (bpm) research program university of queensland data quality; e-commerce; it governance; mobile communications (security aspects); knowledge management; computer forensics; ontological analysis; enterprise computing (workflows,…); spatial databases voluntary collaborative groupings university of the sunshine coast knowledge management; data mining faculty research centre, scribe university of southern queensland eclectic: e-business; is education; knowledge management; is development methodologies ebarc – electronic business advisory & research centre, facultywide table 5 is research foci and groupings for is research in queensland universities key figures who have influenced is in queensland universities senior academics in information systems in queensland universities cite a wide range of individuals as having been significant to the development of information systems teaching and research in their universities. in most instances, prominent information systems academics in the early days of information systems at the university in question were viewed as having had the greatest impact e.g. ed fitzgerald at usq. less commonly, outside figures were cited. at bond university, british academic frank land, an early advisor to bond, was proposed as a most influential individual. at qut, it was john puttick, a prominent it proprietor and practitioner, and chair of the faculty advisory committee. at usc, ron weber and guy gable were nominated as significant influences because of their prominence internationally as information systems researchers, and alan underwood, another queensland is academic, who had made a significant contribution to the australian computer society. the status of is as a distinct discipline in queensland universities to analyse the status of information systems as a distinct discipline, we turn initially to the two constructs from whitley (1984a; 1984b) outlined earlier in this report viz. professionalisation, and 172 australasian journal of information systems volume 14 number 1 nov 2006 conditions for acceptance as a scientific field. where a discipline is not highly professionalised, whitley argues that local contingencies have high impact. from the data collected in queensland universities, there is little evidence of information systems curriculum or research effort being focused on accommodating local community characteristics. nonetheless, as has been pointed out in an earlier section of this report, within the queensland universities there is considerable variability in curriculum content. it can be argued that this variability in information systems content is primarily attributable to a specific local contingency factor. that factor is the information systems leadership in each university. even within a single university, sharp changes in curriculum content can be seen where leadership of the information systems group changes. an example of this is university of the sunshine coast, where the curriculum moved abruptly from a focus on soft systems approaches to a much more “technical” curriculum following a change in the information systems leader at that university. as observed in the earlier discussion on the information systems curriculum in each of the queensland universities, there is a lack of consistency across the universities with regard to subject areas that properly “belong” to information systems staff as opposed to staff from computer science, multi media, business, or some other academic group. this observation of the absence in queensland universities of a “core body of knowledge” defining information systems, is consistent with the wider observations of fitzgerald (2003). fitzgerald concludes that the absence of such a core body of knowledge relegates information systems to “a subject with a particular perspective” rather than a discipline. certainly, this lack of clear boundaries on what constitutes information systems counts against acceptance of information systems as a mature, distinct academic discipline. review of findings while the study points to a considerable diversity in information systems across the nine universities in queensland, there was little evidence of sharply different “philosophies” of information systems as was reported from a survey of information systems in eighteen european countries (avgerou et al, 1999). the european study had, for instance, highlighted the strong sociotechnical stance of the scandinavian countries, in contrast to a pragmatic technical emphasis in german universities. in the curricula and research emphases of the queensland universities there was evidence of an eclectic melding of the various european and american approaches. only at university of queensland and at griffith university, both of which maintain two strong information systems groups, one “technical” and the other “business-focused”, is this blending of approaches to information systems not in place in a single administrative unit. in relating the queensland data to whitley’s (1984b) three criteria for the establishment of a distinct scientific discipline, it has to be said that the evidence from queensland universities does not fully support information systems being recognised as a distinct discipline. whitley’s first criterion, that those in the discipline have the potential for prestige and power through prominence in that discipline would appear to be met. at several of the queensland universities, senior information systems academics are recognised by their universities as professor of information systems. these individuals are recognised, by virtue of their achievements in information systems, as warranting the same prestige and decision-making power in the university as their colleagues in longestablished disciplines. in relation to the second criterion, that standards of research competence and skills become established, there is ongoing debate within the information systems community itself, world-wide. some would argue that the lack of a sound body of accepted information systems theory is evidence that these “standards of research competence” have not been met. again, in 173 australasian journal of information systems volume 14 number 1 nov 2006 relation to whitley’s third criterion, that a unique symbol system is developed that allows the exclusion of outsiders and unambiguous communication between initiates within the discipline, there is limited evidence from the queensland data to suggest that information systems academics in queensland share such a common unique symbol system. the wide variability in curriculum content and research foci would, alone, argue against this unique basis for communication among the information systems academics. in summary, the features of the information systems activities observed in queensland universities have much in common with features reported by whitley (1984a) in relation to the early stages of the development of management as a discipline area viz.: • a heavy reliance on reference disciplines • a paucity of theory specific to the discipline • a perceived lower status than for established disciplines, leading to the adoption of methods from the higher status disciplines • limited numbers of textbooks that review the discipline • poor definition of the boundaries of study • incorporation organisationally as a sub-set of an established discipline. whitley uses the term “fragmented adhocracy” to describe this immature stage of the development towards a distinct discipline. limitations, and future research the queensland study draws only patchily on historical data related to the development over time of information systems in the nine queensland universities. in some universities there was historical data readily available, while in others there was not. where historical data was available to the researchers, there was a richer context for analysis to understand the current situation. the value to the researchers of such limited historical data as was available, points to improved insights from an extension of this study, incorporating a fuller historical analysis. a feature highlighted in the execution of this study was the dynamic state of information systems in queensland universities at the time of the study. hence, the study represents a snapshot of a rapidly changing scene. to capitalise on the findings of this study, there is an imperative to replicate it over time. a longitudinal view of the state of information systems in queensland universities will tell much about the maturing of information systems as a discipline. general learnings from the queensland study in conducting this queensland study, there was recognition by the researchers of the intention to reflect on the procedures followed and the outcomes achieved; this and the other australian studies were to inform subsequent, broader studies into the state of information systems. a similar study into the state of information systems in universities in the countries of the pacific-asia region had been planned to follow the australian study, as mentioned earlier in this paper. a useful learning from the queensland study relates to the approach to data gathering that had been proposed in the study protocol. the intention had been to have interviews with at least one key person from each university in the state as the prime source of data. these interviews were planned to take about one hour each. in the event, arranging face-to-face interviews at two of the 174 australasian journal of information systems volume 14 number 1 nov 2006 queensland universities proved impractical. for both james cook university and the australian catholic university telephone interviews were used, followed up by interview notes and other exchanges by e-mail between the researcher and the interviewees. for future studies elsewhere, a revised study protocol has been prepared to accommodate telephone interviews. in addition, out of recognition of the large time requirements for interviewing where there are many universities involved in the study, a survey instrument has been prepared as a substitute for each interview. the instrument attempts to provide some of the richness of an interview by minimising questions seeking specific numerical responses, in favour of questions teasing out the distinctive characteristics of the university. in recognition of a study environment where, on the other hand, there are very few universities in the study, a guideline for conducting focus groups, based on the standard data gathering framework, has been added to the study protocol. the use of a theory framework to guide the data gathering and analysis, based on “the emergence of a discipline”, proved most helpful to the conduct of the queensland study. in fact, the framework adopted for the queensland study was based on an early version of the framework outlined by ridley earlier in this volume. progressive refinement of this framework, in light of the experiences in the queensland study, and other state studies, will prove valuable to similar future studies. references avgerou c., siemer j., & bjørn-andersen n. (1999) “the academic field of information systems in europe”, european journal of information systems, vol 8 no 2, pp 136-153. fitzgerald, g. (2003) “information systems: a subject with a particular perspective, no more, no less”, european journal of information systems, vol 12 no 3, pp 225–228. ridley, g. (2006) “characterising information systems in australia: a theoretical framework”, australasian journal of information systems, vol 13 no 3, pp. 38-60. sherer, s.a. (2002) “academic departments of information systems faculty in the u.s.”, journal of information systems education, vol 13 no 2, pp 105-116. walsham, g. (1995) “interpretive case studies in is research: nature and method”, european journal of information systems, no 4, pp 74-81. whitley, r. (1984a) “the development of management studies as a fragmented adhocracy”, social science information, vol 23 nos 4/5, pp 775–818. whitley, r. (1984b) the intellectual and social organization of the sciences, clarendon press, oxford, uk. yin, r.k. (2003) case study research: design and methods, 3rd ed., sage publications, thousand oaks. 175 australasian journal of information systems volume 14 number 1 nov 2006 the authors dr bob smyth is a research associate within the it professional services research program at queensland university of technology. he holds an msc from aston university and a phd from queensland university of technology. his main research interests include it professional services, computer aided software engineering and enterprise systems. he has a particular interest in the case study method. in recent years he has been heavily involved in the guidance of phd candidates in information systems. professor guy g. gable chairs the it professional services research program, faculty of information technology, queensland university of technology http://sky.fit.qut.edu.au/~gable/. his phd is from university of bradford and mba from ivey business school. he is senior editor journal of strategic information systems, associate editor misq, and on the editorial boards of journal of the ais, information systems frontiers, and australasian journal of information systems. key research interests include it professional services, it research methods, enterprise systems, and it evaluation. career emphasis is on research collaboration with industry and the professions chief investigator ($2.0m+) on grants with accenture, sap, institute of management consultants, and queensland treasury. he has published >100 refereed journal articles, conference papers and books. 176 http://sky.fit.qut.edu.au/%7egable/ purpose of the queensland study the universities in this study relative size of the is presence in the queensland universities the administrative placement of is in queensland universities distinctive features of information systems curriculum distinctive features of information systems research key figures who have influenced is in queensland universities the status of is as a distinct discipline in queensland universities general learnings from the queensland study the authors ajis vol. 7 no. 1 sept 1999 a generative approach for building database federations uwe hohenstein corporate technology siemens ag, zt se 2 d-81730 monchen (germany) e-mail: uwe.hohenstein@mchp.siemens.de abstract a comprehensive, specification-based approach for building database federations is introduced that supports an integrated odmg2.0 conforming access to heterogeneous data sources seamlessly done in c++. the approach is centered around several generators. a first set of generators produce odmg adapters for local sources in order to homogenize them. each adapter represents an odmg view and supports the odmg manipulation and querying. the adapters can be plugged into a federation framework. another generator produces an homogeneous and uniform view by putting an odmg conforming federation layer on top of the adapters. input to these generators are schema specifications. schemata are defined in corresponding specification languages. there are languages to homogenize relational and object-oriented databases, as well as ordinary file systems. any specification defines an odmg schema and relates it to an existing data source. an integration language is then used to integrate the schemata and to build system-spanning federated views thereupon. the generative nature provides flexibility with respect to schema modification of component databases. any time a schema changes, only the specification has to be adopted; new adapters are generated automatically. introduction federated database systems (dbss) provide solutions to give a uniform and integrated access to data stored in several autonomous sources (sheth and larson (1990), conrad et al. (1997)). a unified and consistent view of the stored data resolves discrepancies (sailor et al. (1992)) and conflicts (spaccapietra and parent (1994)) between database schemata, which result from representing real world situations in different ways. users are not aware of the location of items in a particular system. with all that, the autonomy of the constituent systems is preserved. each dbs still exists, and especially existing applications are not affected. research in the field of federated dbss has brought up several prototypes (e.g., rafii et al. (1991), busse et al. (1994), kuno and rundensteiner (1996)) and results (kambayashi et al. (1991), ims (1993)) that tackle fundamental problems such as global transaction management and query processing. several papers describe integration methodologies (reddy et al. (1994) or schmitt and saake (1996)) and languages for view definitions (kaul et al. (1990)). despite the variety of approaches, some important points are often neglected. 1. only few approaches such as pegasus (rafii et al. (1991)) incorporate object-oriented systems, and if they do, they use an own system instead of arbitrary commercial ones. iro-db from busse et al. (1994) is one exception as it incorporates the commercial systems o2 and ontos. indeed, plugging object-oriented systems in an even object-oriented federation framework is not as trivial as it seems to be. for instance, it is not clear how to handle databases that use object versions. there are no solutions discussed so far for those advanced concepts. moreover, incorporating files is almost neglected in spite of the relevance of huge amounts of file data. 2. the adapters (e.g., in huck et al. (1994), radeke (1995)) for incorporating existing data sources must be hand-coded for any schema again and again. this is particularly a problem in case of schema evolution. any change of a local schema requires a corresponding re-implementation of the adapter. in case of proprietary file systems, parsers must be implemented for each file (abiteboul et al. (1993)); the parser has furthermore to be enhanced with semantic actions that build objects at the federation layer. 3. a lot of approaches such as kaul et al. (1990) rely on querying only, or use own stand-alone manipulation languages. for example, pegasus uses an extension of osql, the language of hp's object-oriented system, for manipulation. data manipulation embedded in a standard language such as c++ is neglected. there are two important points that cause trouble: first, federated schemata must be represented in the programming language. this is quite critical in c++, because its semantics is sometimes odd. second, problems with view updates must be avoided. in iro-db (busse et al. (1994)) federated views are defined by means of the query language oql of odmg (cattell and barry (1997)). it is extremely doubtful, whether updates on those views can be executed unambiguously. our research prototype fihd flexible integration of heterogeneous database systems) provides a comprehensive solution to these aspects. fihd is a tightly coupled approach in the sense of sheth and larson 70 ajis vol. 7 no. 1 sept 1999 (1990). an open federation framework allows plugging in relational and object-oriented dbss as well as files. the overall approach of fthd is generative. section 2 describes how generators produce adapters that "wrap" a data source and provide a uniform access according to the odmg2.0 standard (cattell and barry (1997)) for object-oriented dbss. those adapters are implemented automatically on top of the component systems (cf. point 2). there are generators to wrap relational and object-oriented databases as well as file systems (point 1). a homogenization layer is achieved in this way. all these generators require input, some mapping information that describes an odmg schema and relates it to the local schema. section 3 is concerned with specification languages that serve this purpose. we introduce three languages according to the kind of data source. a specification language for relational databases enables remodeling tables in the odmg2.0 object model, thereby using object-oriented concepts intensively in order to express semantics as much as possible. this language incorporates ideas developed in the context of reverse engineering and semantic enrichment in the sense of castellanos (1993), hainault et al. (1993), chiang et al. (1994), or premerlani and blaha (1994). another language allows defining odmg views on top of objectoriented dbss, handling specific concepts that are not available in odmg. a third language enables one to specify an odmg view of files and to describe file contents in a grammar-like fashion. section 4 presents a specification language for defining federated schemata that yield system-spanning views. a corresponding generator creates federation layers giving a uniform odmg access to the federation. each layer supports an object manipulation and querying, seamlessly done in c++ (point 3). this is especially important in view of manipulating federations in c++. in the conclusions, we summarize our ideas and outline some future work we are planning to do. generative approach to database federation our approach follows sheth and larson (1990) who define a reference architecture for federation frameworks. two steps in their architecture are essential for us, homogenization and data integration. homogenization eliminates syntactic heterogeneity resulting from different data models and access interfaces. each local schema, expressed in its native data model, is translated into a canonical data model. we rely on odmg2.0 (cattell and barry (1997)) as canonical model because it defines a complete database interface consisting of an object model, an object definition language (odl), an object manipulation language (oml), and an object query language (oql). hence, each homogenized schemata offers manipulation and querying according to this standard for object-oriented dbss. data integration then deals with integrating such homogenized schemata into federated ones. a federated schema provides a unified, consistent, and transparent view of all the integrated data. particularly, semantic discrepancies between homogenized schema as described by sailor et al. (1992) or spaccapietra and parent (1994) are eliminated. we again define federated schemata in the canonical model, in odmg2.0. from a functional point of view, the homogenization layer requires adapters that convert local data into the canonical model and map operations onto the local systems. data integration has to support special operational aspects such as global transaction management and global query processing. in our federation approach, homogenization and data integration are done in a generative manner, i.e., the homogenizing adapters and the federation layers are generated automatically by means of generators. each generator requires user input that defines a homogenized or a federated odmg schema. to this end, we adopt the odmg object definition language (odl) and extend it to capture explicit information according to the respective purpose. languages odl, define a homogenization of relational databases (x = 'r'), files (x = 'file') and object-oriented systems o (x = 'o'). any odl, specification defines how the specified schema is related to the underlying data source. data integration is supported by an integration language odli,,,. odli,,, provides means to merge homogenized schemata into federated ones defining an object-oriented and system-spanning view. in fact, the integration language helps dissolve semantic heterogeneity between schemata, i.e., resolves conflicts between homogenized schemata. figure 1 illustrates the process of generation. given as input an odl, homogenization specification, corresponding generators gen, produce odmg interfaces lnterface_x automatically. each interface provides an odmg wrapper that maintains an odmg view of the local data and implements oml operations on top of local systems. other federation approaches generally require implementing a hand-coded adapter for each database to be incorporated in a federation. a generator genin, produces a corresponding interface interfacejnt to operate on the federation. 71 ajis vol. 7 no. 1 sept 1999 integration layer homogenization layer figure 1: generative approach similar to roantree and murphy (1997), the generators use a meta database to maintain information about all the specifications. the meta database is pre-filled with data from the dictionaries of local dbss, for example, what tables are available in a relational database and what types are in an object-oriented one. parsing a specification, each generator fills the meta database with information about how homogenized schemata are related to local ones, and how federated schemata are composed of homogenized ones. the real kernels of the generators read the meta information and generate the odmg2.0 conforming interfaces. the outcome consists of c++ classes including methods for manipulation. naturally, the implementations of interfaces for relational, object-oriented databases and file systems must call the original component source, while interfacejnt is implemented on top of all those homogenizing interfaces. each of the interfaces (interface_r etc.) builds an adapter that already supports oml/oql for the heterogeneous data sources. these adapters can be plugged together in order to access several databases in parallel with one common manipulation language. using the homogenization layer, an application can open the databases and access them in a homogeneous odmg way. however, the individual databases must be handled separately according to the respective disjoint schemata. there is no integrated view on all the data. it is the task of federated schemata to provide a system-spanning view with an integrated interfacejnt in addition. homogenization homogenizing relational databases with semantic enrichment the task of homogenizing relational databases is to convert relational tables into odmg schemata and to provide an odmg conforming access to tables. relational dbss, since having rather primitive modeling structures, do not carry much semantics. this is rather bad, since data integration is a complex task, as it requires a deep knowledge about the semantics of data. wrong semantics could lead during integration to inadequate and defective federated schemata. in our approach homogenizing relational systems is thus combined with ideas developed in the context of semantic enrichment (castellanos (1993), hohenstein and korner (1995)) and reverse engineering (hainault et al. (1993), cacm (1994), chiang et al. (1994), premerlani and blaha (1994)). implicit knowledge is expressed explicitly by using object-oriented concepts extensively. the price we pay for comprehensive remodeling capabilities is an explicit specification. indeed, the advantage of our approach lies in the fact that semantic enrichment is precisely specified. this gives us the opportunity to regain any implicitly given semantics and to remodel schemata in object-oriented terms in various ways. in automatic types of reverse engineering, as against, there is a danger of expressing wrong semantics, a fact we definitively want to avoid. we adopt the object definition language odl of odmg2.0 and extend it to capture explicit remodeling information. our specification language odlr enables one to remodel relational tables in an object-oriented manner. figure 2 presents a relational schema representing a part hierarchy: it models atomic and complex parts, the latter may be composed of either parts. table a contains atomic parts, and c complex ones. since atomic and complex parts may be part of a complex part, there is a table p that consists of the keys of a and c and maintains 72 ajis vol. 7 no. 1 sept 1999 common properties such as name and the build date. the hierarchy is expressed by a foreign key father in p that refers to the cid of the father part in c. semantic enrichment should produce the odmg schema presented in uml. there are an object type part and two subtypes atomic and complex. the composition hierarchy of parts is explicitly modeled by means of a relationship components/partof. figure 2 also presents a corresponding odlr specification. object types are defined by interface declarations, as usual in odmg odl. the clause from relation is an extension to odl. it relates object types to tables. type part is built from table p directly, [pid] denotes the relational key of p. each tuple, which is uniquely identified by its key value, refers to one object. composite keys are possible and denoted as [pid.name,... ]. complex : part from relation c[cid = p.pid] specifies that subtype complex is basically found in table c. moreover, each tuple in c is related to a tuple in p by means of equal id values. [cid=p.pid] is in fact a join condition that is necessary to access inherited p attributes. an equation long mid = m.mid connects an object type's attribute mid to a relational attribute mid of table m. the build date of a part is stored in several relational attributes day, month and year, in tables a and c, since both contain parts. date builddate = (a.day,a.month,a.year) + (c.day.c.montfi.c.year) combines those relational attributes to a predefined odmg type and merges the values obtained fot the tables. tabl a es: pid 4 5 6 7 name abd abe abf acg day 4 5 6 7 month 4 5 6 7 year 1994 1995 1996 1997 father 2 2 3 3 machine 20 20 30 30 material steel silicon steel wood c | pid name day month year father machine assembly 1 2 3 a ab ac 1 2 3 1 2 3 1999 1998 1998 null 1 1 10 20 30 glue weld screw m mid type 10 20 30 aaa bbb ccc object-oriented schema: \ / machine \ produces producedby 0..1 1..* cmateriaj^ atomic s \ , part /\ / partof i..* 0..1 comf1 components -ex — (^assembjy^ 73 ajis vol. 7 no. 1 sept 1999 odlr specification: interface part from relation a[pid] + c[pid] ( extent parts key id) { attribute long id = p.pid + c.pid; attribute string name = a.name + c.name; attribute date builddate = (a.day, a.month, a.year) + (c.day, c.month, c.year); relationship machine producedby inverse produces = (mi m.mid = a.machine + c.machine); relationship complex partof inverse components = (ci c.cid = a.father + c.father); }; interface machine from relation m[mid] (extent machines key mid) { attribute long mid = m.mid;... relationship set<part> produces inverse producedby = {a+ci a.machine + c.machine = m.mid}; }; interface complex: part from relation c[pid] { attribute string assembly = c.assembly; relationship set<part> = components inverse partof = {a+c i a.father+c.father = c.pid}; }; interface atomic: part from relation a[pid] { attribute material = a.material; }; figure 2: specification for semantic enrichment the relationship components is expressed by set<part> components inverse partof = {a+c i a.father + c.father = c.cid}. the set of component parts consists of those tuples in a and c that have the complex parts' cid as value of father. the inverse relationship producedby is analogously computed. round brackets convert a tuple in c into a corresponding object of type complex, while curly brackets convert a set of tuples into a set of objects. composite attributes are again possible to establish associations. the specification language offers more concepts to express semantics. for instance, several relational representations for subtypes (vertical, a horizontal and a complete materialization strategy as well as a flag approach) can be rebuild. for more details about the specification language, the reader is referred to hohenstein and korner (1995) as well as hohenstein (1996). homogenization of object-oriented databases in contrast to the one language odlr handling all relational dbss, there are several languages odlo because each odbms possesses some peculiarities that must be taken into account. each specification language odlo is naturally simpler than odlr since it is easy to express how odmg types are built from existing object types. but every object-oriented dbs provides special modeling concepts such as versioning that are not available in odmg. even if commercial systems completely supported the odmg standard, they would surely offer specific add-ons to beat competitors. those advanced constructs cause trouble. there may be existing databases that use those concepts, e.g., databases that contain versioned objects. in particular, some concepts possess a specific semantics that must be maintained by odmg operations at the homogenized level! we have to find odmg ways to remodel peculiarities adequately without falsifying semantics. in the following, we discuss some problems that occur when homogenizing the commercial object-oriented dbs objectivity/db. however, the critical points are similar for other systems. we present an adequate specification language odlobjy that reflects advanced concepts in odmg2.0. storage hierarchy objectivity has a storage hierarchy that consists of federated databases 2, containers, and objects. a federated database contains several databases and gives them a common schema. the databases can be located on different sites in a network. databases are partitioned into containers. newly created objects are assigned to one of the containers. containers are thus useful for clustering objects. 2 please note that the term "federated database" is here used in an objectivity context. 74 ajis vol. 7 no. 1 sept 1999 concerning homogenization, federated databases and databases do not give rise to problems. an objectivity application program can open only one federated database in a process. consequently, the names of a database and its associated federated database can be considered together as a database name from an odmg point of view. opening a database at the odmg level requires then opening both the federated database and the database. but containers cause some trouble. if we did not support containers, we would give up clustering, an important aspect of tuning object-oriented databases. a possible solution could be to define new types ct in the odmg schema, one for each container. objects of type t in a certain container are put into a type tg that is subtype of both t and ci. accessing type t yields all objects, while type tcj contains only those that are in container cj. unfortunately, the amount of containers is dynamic, as they can be created and deleted in objectivity programs. hence, the schema must be modified every time a new container is created and deleted. this is an inadequate solution! we suggest establishing a predefined type d.container as a subtype of odmg type d_database: each container becomes an instance of d_container. containers can thus be created and deleted dynamically, as in objectivity. being objects of supertype d_database, containers can be used instead of db when creating objects with new(db). clustering becomes possible. moreover, d_container inherits the typical d_database functionality such as open and close, now having the objectivity specific container semantics. certainly, we have slightly extended odmg, as there is a new predefined type, but the real user-defined schema is not affected. supporting clustering is worth it anyway. short relationships objectivity's object model offers so-called short relationships that reduce storage. short relationships must refer to objects in the same container. if short relationships are not supported in the homogenized schema, establishing new relationships may violate this restriction regarding containers. since objectivity recognizes violations and issues an error message, it is not necessary to designate short relationships in an odlotjy specification. complex objects objectivity allows one to define propagation of operations copy, delete, and lock. this is done by means of socalled relationship specifiers: the operation is propagated to objects related by that relationship. we suggest no propagated copy and lock, because odmg does not support any explicit copy and lock operations; this means no great loss of functionality. but otherwise we would extend oml. if propagation of delete is not supported at the odmg level, the contents of an objectivity database may be corrupted: propagation of delete may define an implicit integrity constraint; an object cannot exist without participating in a relationship. a propagated deletion can be achieved by changing the effect of odmg delete_object. the odlobjy syntax should indicate propagation so that the implementation of delete takes propagation into account. interface article from object type objyart { relationship order orderedby = objyart.orderedby: propagate delete; ); versioning in objectivity, any object in the database may occur in several versions. versionable classes need not be defined in the schema. objects can dynamically become versionable by invoking a method setversstatus(on/off). if versioning is enabled for an object, any modification (more precisely, any invocation of ooupdate in order to explicitly notify an update, comparable to odmg's markjnodified) creates a new version. a version graph maintains all versions of an object and possesses predefined relationships to navigate in the graph, i.e., going to previous or next versions. there exists a predefined genealogy class the objects of which are surrogates for version graphs. besides normal relationships pointing to a concrete version of an object, it is possible to establish relationships referring to genealogies, i.e., whole version graphs. those relationships are floating in a certain sense. if the target is a genealogy, the current version within the graph can be accessed by means of predefined methods. we cannot omit the concept of versioning if there are existing databases containing versioned objects and floating relationships. we propose a predefined class d_versiongraph. this class supports special methods to 75 ajis vol. 7 no. 1 sept 1999 handle versions, e.g., to turn versioning on and off, to designate a referenced version in a version graph as current one etc. any versionable type must inherit from d_versiongraph. objects then inherit versioning methods. furthermore, it is possible to mark relationships as floating. finally, the semantics of markjnodified is altered to create new versions if versioning is switched on. since all these points affect the "semantic enrichment" of object-oriented database systems, odlobjy must reflect them in the following way: interface article: d_versiongraph from object type objyart { } ; interface order from object type objyorder {... relationship set<article> orders = objyorder.orders floating; }; please note the whole functionality is concentrated in one additional predefined class. neither the schema nor oml is really extended. object manipulation homogenizing implies emulating odmg oml on top of objectivity. it is no problem to bridge the differences in both manipulation languages. but again, there are advanced objectivity manipulation concepts that exceed odmg2.0. omitting those manipulation features causes a lack of objectivity functionality. anyway, this will not exclude existing databases from a federation. we propose in general omitting those functions that do not endanger usability, e.g., copy propagation. sometimes simulating operations is easily possible by giving odmg operations an extended effect (propagational delete with delete_object, creating new versions with markjnodified). keeping advanced functionality can also be done by adding new classes. the extensions are then encapsulated in classes, staying outside the original oml. this principle was applied previously to support versioning and containers, e.g., classes d_versiongraph and d_container. similarly, we introduce a subclass d_longtransaction of d_transaction to support objectivity's checkln/checkout functions. we strictly avoid introducing new oml methods. hence, we do not support scoped queries in the sense that searching objects can be done in a particular container, in one database, or in the whole federation. this would require a new type of query method in odmg. homogenization of file systems the specification language odlf,ie has the goal to define a "schema" for file contents in a certain sense. the syntax of odlfl)e is a mixture of the odmg odl and an enhancement of yacc. odl is used to describe the schema of the data file in an object-oriented way. the yacc-part defines a grammar. this eases the later generation of a parser by means of this compiler-compiler. parsing a file, the collected information has to be organized according to the odl schema. hence a third part of odlflle relates the grammar to the schema. figure 3 presents the three parts of an odlrie specification. we assume a data file containing machines and the parts they produce. lines starting with "100" denote machines, those starting with "200" mark parts. please note the sequence of the records within the data file is relevant; it defines an implicit relationship between the objects: all parts following a machine record are produced by that machine. 76 ajis vol. 7 no. 1 sept 1999 file content: 100,caster,1 200.ab.4 200.abc.5 200.abd.6 100,press,2 200.acg.7 //part 1 ofodlfuf specification: odl schema interface product { attribute short no; attribute string name; interface machine { order by name asc; attribute short no; attribute string name; attribute set<product> partlist; //part 2 ofodlfne specification: grammar startsymbol(rle); // start rle: data[1-]; // rules of the grammar data: machine part(1-]; machine: '100' .comma mname_comma mnum _eol; part: '200' .comma pname .comma pnum .eol; pname: .identifier; // nonterminals using pnum: .number; // predefined terminal mname: .identifier; mnum: .number; //part 3 ofodlfiit specification: assignment rules machine class2use(machine); mname export(name,et_ascii); mnum export(no,et_ascii) condition(short,et_ascii,lt,10); data export(partlist); part class2use(product); pname export(name,et_ascii); pnum export(no,et_ascii); figure 3: odlrue specification interface definition with odl subset the first part of an odlfdc specification defines an odl-schema. the file content is here modeled as two object types machine and product. nested structures can be expressed: each machine produces a partlist, a set of products. it is important to express those nested structures because they occur quite often in files as shown by ashish and knoblock (1997). in addition to odmg odl, it can be specified that an object type must be ordered in a certain way. here, machines occur in the file with ascending names. it is essential to express this, because otherwise the adapter has no chance to write back the file correctly in this order! in other words, the specification would be incomplete. 77 ajis vol. 7 no. 1 sept 1999 definition of the file grammar the second part of the specification consists of a grammar for the data file. the grammar rules describe, how the data file is structured. figure 3 also shows the grammar part for our example. the syntax of rules is similar to the rules of the compiler-compiler yacc. nonterminal: item"! item2 i item3 item4...; is a rule that defines a nonterminal symbols by means of items. alternatives in rules are denoted as 'i' similar to yacc. each item can be either a nonterminal that must be defined by other syntax rules, or a terminal symbol. terminal symbols can be defined in two different ways: • embedding the characters in quotation marks such as "100". • usage of predefined terminal symbols. terminal symbols represent the characters in the data file. there are several predefined terminal symbols. for example _number stands for an optionally signed integer number. similarly, .identifier represents a sequence of letters, _byte any byte character, _comma a comma, _1 the digit "1", and so on. the predefined terminal symbols are provided because they ensure shorter specifications and thus improve readability. furthermore, fixed-sized attributes can be described easily. this is necessary to effectively describe files with fixed-sized records. the specification language offers the possibility to use repetition groups to bypass recursions. this clearly improves the readability and the maintainability of a specification and also has advantages for unparsing a file. for example, file: data[1-] specifies that data can occur more than once. in some cases, a data file contains a certain set of tokens which can occur in any order. it is cumbersome to describe this variable order by only using alternatives, as it has to be done with yacc. that is why odl,rie provides an operator &: article: author & title & journal & year; specifies that an article consists of an author, a title, a journal and a year, whereby these components may occur in any order within one record of the data file. assignment rules the grammar only describes the structure of the data file. a parser can then collect the information from the file. assignment rules now relate the parsed information to objects in the odl schema. a nonterminal always has a certain value. this value is a string, which arises as a concatenation of the values of the nonterminals on the right side of the rule: the value of machine is composed as follows: "100" is a terminal symbol which naturally has the value "100". _comma is a predefined terminal symbol with the value ",". mname and mnum are other nonterminals which are later assigned the values "caster" and "1" after analyzing the first machine record. hence, machine has the value "100,caster,l". the values of the nonterminals are assigned to the attributes of the objects in the following way. a nonterminal is connected with an object type by the keyword class2use. for example, machine class2use(machine) specifies that the nonterminal machine is related to the object type machine. that is, the value of the nonterminal is used to build machine-objects. an export rule fills an attribute of the current object type with the value of a nonterminal. thereby, it has to be specified how the data in the file is encoded. it is a difference, whether an integer number is encoded in a binary format or as an ascii-number. odlfiie offers a possibility to distinguish between different encodings. for example, et.ascii can be used for ascii-text, while etjeee is used for ieee binary encoding. pnum export (no, et_ascii) then specifies that the parsed pnum-value should be ascii-decoded before assigning to the no attribute of the current product-object. similarly, the set-valued attribute partlist is filled by using the data rule that describes the nesting. condition rules contain simple comparisons in order to be able to filter certain records. in figure 3, machine records are only exported if the value of mnum is lower than 10. a condition rule always effects a whole object type. the current record is only exported, if all condition rules affecting the appropriate object evaluate to "true". condition rules need a data type, since the comparison value need not be part of the object type. 78 ajis vol. 7 no. 1 sept 1999 condition rules are very useful because some file formats use a flag to mark a record as deleted in order to increase performance. using the condition rules, a specification can be built which only exports undeleted records to the dbs. such an odlf,ie specification is input to a generator. this generator produces a yacc program that is able to parse the data file. moreover, the yace-program contains semantic actions that build objects according to the odl schema. this releases one from the tremendous task of building yacc programs as it is necessary in abiteboul et al. (1993). the approach is similar to ashish and knoblock (1997), but more general. at the moment, the file adapter is not directly embedded in the federation framework; the adapter stores file data into a relational database it has previously installed. but the relational database can be included into the federation as described in subsection 3.1. an unparser is generated that writes data from the database back to files. data integration a flurry of activities developed languages to specify federated views for heterogeneous dbss. however, most proposals rely on relational and functional canonical models. others like radeke (1995) do rely on objectorientation, but present only simple examples, not showing how to mix subtype hierarchies, as discussed in schmitt and saake (1996) from a methodological point of view. some approaches seem to be too powerful with regard to view updates. for example, busse et al. (1994) use odmg oql queries to describe federated schemata, a natural and flexible mechanism. although they claim in huck et al. (1994) to support odmg oml, it is not clear how they map updates onto local databases. as far as we perceived, updates are a general problem of federation approaches. queries are often preferred. what is neglected, is how to represent federated views in a language such as c++, and how to embed object manipulation. these points led us designing a good compromise that is powerful enough, but is also able to support updates invoked from c++. odmg2.0 helps us since it defines a c++ binding for the oml. what we still have to do is to care for a c++ representation of federated views. the specification language odlim we propose, provides syntactic constructs to handle typical problems (reddy et al. (1994)) and aspects of schema integration such as: • naming conflicts such as homonyms/synonyms. • type conflicts of attributes. • scaling conflicts (dollar vs. dm). • structural discrepancies (spaccapietra and parent (1994)), e.g., if some unit is modeled by an attribute in one component schema, but as an object type elsewhere. • interdatabase connections, i.e., to specify logical links between so far disjoint databases, i.e., objects are built by "joining" objects from different databases. • objectification of values and relationships (busse et al. (1994)). • generalization to bring together objects of same type, but from different, heterogeneous databases, disjoint or overlapping. • relating types of different databases in a subtype hierarchy, in general merging subtype hierarchies etc. (schmitt and saake 1996). the language follows the way we pursue for homogenization. given an odli,,, specification as input, object classes, which represent the federated schema and provide oml, are automatically generated. in the following, we stress important aspects that are mainly centered around subtyping. we feel subtyping has not been investigated enough, especially because c++ possesses an odd semantics in this respect, impeding a class representation. let us take the homogenized schemata defined in figures 2 (schema si) and 3 (schema s2). we first want to "join" parts and products. we suppose that only complex parts are produced. hence, the semantics for integration is as follows: complex = product. we define an interface joined_part that generalizes complex and product. 79 ajis vol. 7 no. 1 sept 1999 interface part from part@s1 ld = part@s1.ld;... interface joinedj>art: part from product® s2[no] = complex® s1 [id] attribute name = complex@s1.name = product@s2.name; attribute assembly = complex@s1.assembly; relationship machine producedby inverse produces = ( machine@s21 product in machine®s2.partlist); similar to the odl* homogenization languages, the basic bricks are existing types (related to homogenized schemata by means of '@'), means to identify objects ('[ ]'), and set operators '+' (disjoint union), 'u', and others. the from clause defines how the (virtual) extent is built. at first, type part is directly taken from schema si. objects of type joined_part are essentially found in complex, however, information is also needed from product. this is particularly necessary to retain the attribute assembly from complex and the relationship producedby from product: an object of type joined.part should possess both. the clause ... from product@s2[no] = complex@s1[ld] indicates this fact and demands for an equality of extents. [no] and [id] are used to relate objects in both types. this is indeed some kind of instance integration that is absolutely necessary, but almost forgotten by other approaches. if the keys are not homogeneous with regard to data type and value, functions can be defined to relate the objects. since objects of product and complex are "joined", common attributes must be handled. the names of products and complex parts should be the same in both schemata. hence, we require an equation complex@s1.name = product@s2.name. if value conflicts between names were allowed, we could omit the part = product® s2.name, just saying to take the name of complex. if the names were homonyms, we could specify namel = product@s2.name name2 = complex@s1.name the relationship produces is computed by using the set-valued attribute partlist. the embedded partlist of machine is converted to an explicit relationship. a second example shows a generalization of types: we now suppose part and product be disjoint, and we want to generalize them to one type partj5um the extent of which should receive all the objects: interface part.sum from part@s1[ld] + product@s2[no] (extent sum) attribute no = part@s1.ld + product®s2.no; please note keys [...] are not necessary here. if they are omitted, part and product will be assumed to be disjoint due to '+', but there is no mean to control it then. key specifiers [id] and [no] allow the federation layer to check for disjoint sets of numbers. let us now suppose part and product overlapping: there are parts that possess the same id and no, stored in both databases. the following interface definition is sufficient at a first glance: interface partjjnion from part@s1[ld]uproduct@s2[no]...; interface part_s1 : partjjnion from part@s1... interface product_s2: partjjnion from product@s2... 'u' now allows non-disjoint unions, in contrast to '+'. [id] and [no] must specify how to relate identical parts in part and product. partjjnion allows handling all parts uniformly, independently of their location, interfaces can be added in order to access not only all parts, but also the parts in si (as part.s1), and the parts in s2 (as product.s2), too. 80 ajis vol. 7 no. 1 sept 1999 but this specification is only partly correct, owing to the odd semantics of c++, and odmg2.0, too: subtypes are always disjoint in c++ with regard to real instances. even if part and product contain common objects, the federated view is not aware of them. there is no means in oml, no object type, to insert one instance in both part_s1 and product_s2. this problem can be solved by introducing an additional subtype partjntersection of part.s1 and product.s2: interface partjntersection : part_s1, product_s2 from part@s1[ld], product @s2[no] ... partjntersection is now the type to insert common objects; it has the purpose to hold the intersection of part and product. hence an identification [ ] of objects in both types is demanded. considering the shallow extents, partj31 contains the real parts (that are not products), productj32 the real products, and partjntersection all the parts that are also products. the shallow extent of partjjnion is empty as there no instances that are neither parts nor products; nevertheless, the deep extent contains all the parts. other approaches such as busse et al. (1994) and radeke (1995) that also rely on odmg2.0 do not discuss this important fact! only approaches that use their own manipulation language, e.g., kaul et al. (1990), can make things easy to handle disjoint and overlapping generalizations. mixing several subtype hierarchies is another problematic case that requires special concepts. the literature contains methodologies how to mix hierarchies, for example schmitt and saake (1996). but less effort has been spent on languages to define mixed subtype hierarchies and their relationships to the original ones. let us assume two databases: a university library (uni) contains a type publication with a subtype journal. a computer science library (cs2) has a type publ possessing a subtype proceedings. the following conditions should hold with regard to extents: publ@un1 2publ0cs2 2journal@un1, publoun1 2 publ@cs2 2 proc@cs2, journal@un1 n proc@cs2 the following specification defines an integrated subtype hierarchy that covers these conditions. interface unlpublfrom publ@un1 ... interface cs.publ : unlpublfrom publ@cs2[isbn=publ@un1.isbn]... interface journal :cs_publ from journal@un1 [isbn=publ@cs2.isbn) ... interface proc : cs.publ from proc0cs2 ... interface j_p : journal.proc from journal@un1[isbn],proc@cs2[isbn] ... types of different schemata can be put in a subtype relationship by "join" conditions such as [isbn=publ@un1.isbn]. this is necessary to access inherited attributes. the definition of federated schemata must obviously be done in accordance with the extents of the existing types. in principle, we can derive different federated schemata from one and the same set of subtype hierarchies. methodologies such as schmitt and saake (1996) and spaccapietra and parent (1994) help finding the right semantics. to sum up, odlin, has the power to handle complex situations. anyway, the semantics is compatible with c++ so that manipulations become possible. the above examples can certainly give only an impression of the power and flexibility of our approach. conclusions in this paper, we suggested a comprehensive approach to database federation. building federations is done in a completely generative manner: generators automatically implement access layers to a set of heterogeneous data sources and provide integrated views and uniform access conforming to the odmg standard (cattell and barry (1997)). the approach is based on a set of specification languages that build the input to code generators. a first set of languages is concerned with homogenization. each object-oriented database system acquires a language of its own due to the diversity of systems and concepts. relational systems are handled by one single language. so are files. the languages allow making semantics explicit. an additional language allows one to merge the homogenized schemata to federated ones. a federated schema then defines a system-spanning, integrated and consistent view of all the databases. giving specifications in those languages as input, corresponding generators produce code to provide the schemata with an odmg conforming object manipulation. defining a homogenization specification, a generator produces a schema-dependent adapter that automatically maps odmg2.0 operations and queries onto 81 ajis vol. 7 no. 1 sept 1999 the data source. each adapter can be used stand-alone. for instance, the adapter for a relational database provides an object-oriented access to relational data. a set of generated odmg adapters can be used to access several, heterogeneous databases in parallel, without implementing any glue! a programmer just needs to compile and link the homogenized schemata into an application program in order to operate in an odmg manner on the data sources. plugging adapters in a global federation framework provides full functionality. there is now a database-spanning view, and global querying and transaction management. the generative nature of database federation has a significant advantage with regard to schema evolution: any time a local schema is modified, the corresponding generator produces a new adapter for the local database without any implementation effort. just the specification has to be adapted. the presented generative approach has partly been implemented. at the moment, there are generators for a homogenization of files, any relational database system and the commercial object-oriented databases objectivity/db and versant. the specification language for defining federated schemata and the corresponding generator for producing federation layers is still under development. future work will be dedicated to some improvements. homogenizing file systems should take into account directory information as discussed in hoding (1996). furthermore, file systems must be integrated into the federation framework directly, avoiding the detour via relational databases. finally, we feel the need for a comfortable graphical support for defining schemata similar to hohenstein and korner (1995). anyway, some more support is necessary for guiding users in the integration process. references abiteboul, s., cluet, s. & milo, t. (1993) "querying and updating the file", in proc. of conf. on very large databases (vldb) 1993 ashish, n. & knoblock, c. (1997) "wrapper generation for semi-structured internet sources", acm sigmod workshop on management of semi-structured data, tucson (arizona) 1997, superseded by acm sigmod record 26(4), dec. 1997 busse, r., fankhauser, p. & neuhold, e. (1994) "federated schemata in odmg", proc. of 2nd east/west database workshop 1994 cacm (1994) "reverse engineering", special issue of communications of the acm 37(5), 1994 castellanos, m. (1993) "semantic enrichment of interoperable databases", in ims (1993) cattell, r. & barry, d. (1997) (eds.) "the object database standard: odmg2.0", 2nd edition, morgankaufmann publishers, san mateo (ca) 1997 chiang, r., barron, t. & storey, v. (1994) "reverse engineering of relational databases: extraction of an eer model from a relational database", data&knowledge engineering 12, 1994 conrad, s., eaglestone, b., hasselbring, w. , roantree, m., saltor, f., schonhoff, m., strabler, m. & vermeer, m. (1997) "research issues in federated database systems", sigmod record 12/1997, 26(4) conrad, s., hasselbring, w., heuer, a. & saake, g. (1997) "proc. of the int. caise97 workshop engineering federated database systems efdbs'97", barcelona 1997 hainault, j.-l., tonneau, c., joris, m. & chandelon, m. (1993) "schema transformation techniques for database reverse engineering", 12th int. conf. on entity-relationship approach, karlsruhe 1993 hoding, m. (1996) "an approach to integration of file based systems into database federations", in proc. of 10th european research consortium for informatics and mathematics (ercim'96) on heterogeneous information management, prague 1996 hohenstein, u. (1996) "bridging the gap between c++ and relational databases", proc. of 10th european conf. on object-oriented programming (ecoop'96), linz (austria) 1996, springer lncs 1098 hohenstein, u. & korner, c. (1995) "a graphical tool for specifying semantic enrichment of relational databases", 6th ifip wg 2.6 work. group on data semantics (ds-6) "database applications semantics", atlanta 1995 huck, g., fankhauser, p., busse, r. & klas, w. (1994) "iro-db: an object-oriented approach towards federated and interoperable dbms", in: advances in databases and information systems (adbis'94), moscow 1994 ims (1993) "proc. of conf. on research issues in data engineering: interoperability in multidatabase systems" (rtoe-ims'93). vienna 1993 kambayashi, y., rusinkiewicz, m. & sheth, a. (1991) (eds.) "proc. of 1st int. workshop on interoperability in multidatabase systems", kyoto (japan), 1991 kaul, m., drosten, k. & neuhold, e. (1990) "viewsystem: integrating heterogeneous information bases by object-oriented views", proc. 6th int. conf. on data engineering, los angeles 1990 kuno, h. & rundensteiner, e. (1996) "the multiview oodb view system: design and implementation", theory and praxis of object systems 2(3), 1996 markowitz, v. & makowsky, j. (1990) "identifying extended er object structures in relational schemas", 82 ajis vol. 7 no. 1 sept 1999 ieee transactions on software engineering 16(8), 1990 pitoura, e., boukres, o. & elmagarid, a. "object-orientation in multidatabase systems", acm computing surveys 27(3), 1995 premerlani, w. & blaha, m. (1994) "an approach for reverse engineering of relational databases", communications of the acm 37(5), may 1994 radeke, e. (1995) "efendi: federated database system of cadlab", acm sigmod conf. on management of data 1995, sigmod record 24(2) rafii, r., ahmed, r., desmedt, p., kent, b., ketabchi, m. & litwin, w. (1991) "multidatabase management in pegasus", kambayashi et al. (1991) reddy, m., prasad, b., reddy, p. & gupta, a. (1994) "a methodology for integration of heterogeneous databases", ieee transactions on knowledge and data engineering 8(6), 1994 roantree, m. & murphy, j. (1997) "an architecture for federated database metadata", in conrad et al. (1997) saltor, f., castellanos, m. & garcia-solaco, m. (1992) "overcoming schematic discrepancies in interoperable databases", in o.k. hsia, e.j. neuhold, r. sacks-davis (eds.): proc. of the ifip wg 2.6 database semantics conf. (ds-5) on interoperable database systems, lome (australia), 1992 schmitt, i. & saake, g. (1995) "integrating of inheritance trees as part of view generation for database federations", 15th int. conf. on conceptual modeling (er'96), cottbus 1996, springer lncs 1157 sheth, a. & larson, j. (1990) "federated dbss for managing distributed, heterogeneous and autonomous databases", acm computing surveys 1990, 22(3) spaccapietra, s. & parent, c. (1994) "view integration: a step forward in solving structural conflicts", ieee transactions on knowledge & data engineering 1994, 6(2) 83 ajis vol. 4 no. 2 may 1997 an empirical examination of project leadership style in an information systems environment mohan thite and pamela simmons swinburne university of technology po box 218, hawthorn3122 melbourne, australia email: mthite@swin.edu.au; psimmons@swin.edu.au abstract after establishing that leadership is one of the critical success factors in is projects with the support of literature, this study examines the most appropriate leadership style at the project management level for the successful execution of is projects. the suitability of transformational leadership style in an is project environment is tested in a survey of australian is projects. the leadership behaviour of more and less successful is project managers as perceived by themselves and their subordinates is analysed in conjunction with other contingency factors influencing project implementation. the results support the proposition that there is a difference in the leadership styles of more and less successful project managers as perceived by their subordinates and that the successful managers exhibit significantly more of transformational leadership characteristics. introduction there is a widespread dissatisfaction about the performance of information systems (is) projects. inordinate delays, excessive budget over-runs, post-implementation testing, user dissatisfaction, late deliveries, poor reliability, maintenance problems... the list of woes continues (kapur, 1989; abdelhamid, 1990). considering the critical importance of is, the failure rate is quite disturbing. researchers have endeavoured to explain the causes of is project failure. significant efforts have been made to identify critical success factors so that is developers can focus their attention on these factors to ensure success. research has established that when it comes to project management, technology is more often a secondary issue behind management, particularly of human resources (sauer, 1993). leadership is critical in any human endeavour. it is generally recognised that technical/scientific professionals lack managerial skills, including leadership skills. many project managers are perceived as having average or less than average managerial skills (anderson, 1992). in a job satisfaction survey of is professionals, poor management was the most commonly cited source of frustration for is employees (kapur, 1989). pulk (1990) opines that the most obvious cause of the increased cost (of software development) is the lack of effective leadership within software development projects. therefore, it is important to look at the leadership behaviour of is project managers as a critical success factor. this study aims at finding a suitable leadership style that would make a significant contribution to the success of is projects. transformational leadership is a new paradigm in leadership research. bass & avolio (1991) have made a seminal contribution to the research on transformational leadership and this study tests their model in an is environment to examine whether it adequately differentiates between successful and unsuccessful projects and whether more successful project managers exhibit more of transformational leadership characteristics than their less successful counterparts. critical success factors there does not appear to be a consensus of opinion on the criteria for judging project success and the factors that influence that success (wateridge, 1995). pinto & slevin (1987: 3) presented a comparative analysis of various studies and identified the following nine critical success factors in these studies on project implementation: clearly defined goals; competent project manager; top management support; competent project team members; sufficient resource allocation; adequate communication channels; control mechanisms; feedback capabilities; and responsiveness to clients. they empirically developed a framework of the project implementation process and a diagnostic instrument called the project implementation profile (pip) which identifies ten critical factors: project mission; top management support; project schedule/plan; client consultation; personnel; technical tasks; client acceptance; monitoring & feedback; communication; and trouble shooting. however, pinto & prescott (1988) found out in another study that the relative importance of various critical success factors are subject to change at different phases of the project implementation process. according to wateridge (1995), within the is domain there has been little research and testing of project-success criteria and his research pointed out that there is a vast difference between the perceptions of users and those of project managers on the factors that contribute to successful is projects. deutsch (1991), through an exploratory analysis relating to the software project management, 92 ajis vol. 4 no. 2 may 1997 derived that the "residual management power factors", such as management of business risk, technical risk, external interface, multiple user needs, problem scope and planning & feedback correlate to successful project outcomes. brown (1994) identified organisational commitment, complete functional design and competent project management as critical success factors for large is projects. cash & fox (1992) opine that management's involvement, leadership, staffing and control & reporting are elements often found in successful implementation of computer systems. leadership in is projects the literature on is project management provides ample evidence on the importance of non-technical issues as critical success factors (phan et al., 1988; laudon & laudon, 1990). although there is a paucity of empirical evidence on the causes for project failure, sauer (1993) suggests that a consensus has been reached amongst researchers that failure is generally caused by the neglect of the behavioural and social factors. the role of the project manager in is projects is therefore even more challenging and vital than in other types of projects where these issues arc less critical. leadership taken in the context of projects presents special challenges (cleland, 1995: 86). in is departments, project managers are faced with increasingly complex tasks which require more than a single set of management skills (carter, 1988). that is why, the concept of "hybrid" managers, particularly in the context of is environment is gaining prominence (palmer & ottley, 1990). successful projects almost always have a "champion" who either by past experience or by persistent determination provides needed leadership to members of the project team to see that results are achieved (cash & fox, 1992). a project's success or failure is the result of the leadership of the project's stakeholders (cleland, 1995: 85). bridging the gap between software and product development requires effective leadership and project management within the software design teams (pulk, 1990). the key ingredient to effective project management is good people management and leadership skills. the more complex the people situation, the more critical a project manager's leadership capabilities become (bates, 1994). in today's corporations, tremendous responsibility lies with the is project manager, who must take the corporation's vision and translate that vision in to systems that support the company's strategic direction (geaney, 1995). technical leadership skills compared to other occupational groups, the management and leadership of technical employees pose unique challenges (thite, 1997). leading achievement-oriented technical professionals in a turbulent business environment requires non-traditional leadership skills and styles. weinberg (1986), an acknowledged expert on technical leadership, opines that outstanding technical leaders exhibit the problem-solving leadership style by concentrating on three major areas: understanding the problem, managing the flow of ideas, and maintaining quality. a study by rosenbaum (1991) found many common patterns related to effective technical leadership. successful leaders coach for peak performance, remove organisational obstructions from the path to innovation, orchestrate the professional development of their subordinates, expand individual productivity through team work, and facilitate self management. the study concluded that technical leaders in a hightechnology world need to be "high-touch", that is, adept at communicating, influencing and motivating. within the is environment, several studies have highlighted the essential qualities and skills of is project managers to ensure success. table-1 gives a comparative picture of such studies. they highlight the importance of is manager's ability to manage people, technology, stress, emotions, organisational politics and obstacles, communication etc. table-1: essential qualities and skills of is project managers rahn, 1987 bander, 1986 geaney, 1995 bloom, 1996 willingness to take risks, willingness to commit time, ability to handle stress well, deposition to people management, right emotional disposition, communication skills, ability to handle politics problem solving, managerial identity, achievement orientation, strong influence visible leadership, flexibility, sound business judgement, trust worthiness, active listening skills technical competence, political & people skills, handle obstacles, maturity to accept full project responsibility, stamina & sense of humour 93 ajis vol. 4 no. 2 may 1997 transformational leadership the concept of leadership can be traced back to antiquity and is, in a sense, terra incognita (unknown territory). today's new ideas about leadership are testing and threatening many of our traditional notions of what leaders do (cleland, 1995). transformational leadership, transactional leadership, and laissez-faire form a new paradigm for understanding both the lowerand higher-order effects of leadership (bass and avolio, 1989:1). earlier leadership theories confined themselves to transactions based on initiation (task orientation) and consideration (people orientation) which were not sufficient to explain the full range of leadership behaviours commonly associated with the best and also the worst leaders (bass & avolio, 1993). bass (1985) described transformational leaders as those who: • raise followers' level of awareness of the importance of achieving valued outcomes and the strategies for reaching them. • encourage followers to transcend their own self-interest for the sake of the team, organisation, or larger policy. • develop followers' needs to higher levels in such areas as achievement, autonomy, and affiliation, which can be both work-related and non work-related. according to bass & avolio (1993), transformational leaders exhibit the following key leadership behaviours: 1. charisma: followers identify with and emulate these leaders, who are trusted and seen as having an attainable mission & vision. this is further subdivided in to attributed charisma & idealised influence. sample item: instils pride in being associated with him/her. 2. inspirational motivation: provides symbols & simplified emotional appeals to increase awareness and understanding of mutually desired goals. sample item: articulates a compelling vision of the future. 3. intellectual stimulation: encourages followers to question their own way of doing things or to break with the past. sample item: seeks differing perspectives when solving problems. 4. individualised consideration: followers are treated differently but equitably on a one-toone basis. sample item: spends time teaching and coaching. transactional leaders motivate followers by appealing to their self-interest and exert influence based on bureaucratic authority, rather than influence based on exchange or inspiration (burns, 1978). according to bass & avolio (1993), transactional leadership behaviour consists of: 1. contingent reward: it involves a positively reinforcing interaction between leader & follower that emphasises an exchange. the emphasis is on facilitating the achievement of objectives agreed to by followers. sample item: makes clear what one can expect to receive when performance goals are achieved. 2. management-by-exception (mbe): only when things go wrong will the leader intervene to make some correction. punishment and discipline are likely to be evidence of such behaviour. many studies have found that mbe can be split into an active and a passive factor. the active manager arranges to monitor and correct deviations (sample item: focuses attention on irregularities, mistakes, exceptions, and deviations from standards); the passive manager waits for them to occur before taking corrective action (sample item: shows that he/she is a firm believer in 'if it ain't brake, don't fix it'). finally, laissez-faire is a non-leadership behaviour where leadership is absent and any intervention is avoided. bass and avolio (1993) view transformational and transactional leadership as distinct but not mutually exclusive processes and the former can "augment" the effectiveness of the latter as illustrated in figure-1. 94 ajis vol. 4 no. 2 figure-1: the augmentation model of transactional and transformational leadership may 1997 transformational leadership charisma inspiration intellectual stimulation individualise d transactional leadership management -bvcontingent reward expected effort extra effort expected performance performance beyond (source: bass & avolio, 1991) the relevance of transformational leadership model in the technical/scientific fields has been empirically tested by several studies (keller, 1995 & 1992; howell & higgins, 1990; beatty & lee, 1992). it is also said to be effective in a team environment (atwater and bass, 1994). the model has been empirically validated in an australian setting in hospital & educational institutions(parry & saros, 1994) and in the banking industry (carless, 1995). research on is project leadership the authors tested the suitability of transformational leadership model in the australian is environment. the research examined whether the project managers of more successful is projects exhibited more of transformational leadership behaviour than transactional leadership behaviour. the research was aimed at improving the quality of is project management by focusing on project leadership in conjunction with other critical success factors. sample design the target population was is companies and is divisions of other companies in australia which were engaged in software development activity in a project team environment. the list of organisations which were invited to participate in the survey was derived from a listing of top 50 software companies (kennedy, 1996), selected listing of top 100 it/is using organisations (conners, 1995), and selected member organisations of australian information industry association (aiia). the survey was supported by the aiia, the australian computer society (acs), the australian institute of management (aim). as per the research design, a senior manager of the respondent company, to whom the it project managers reported, selected two recent it projects in his/her company to participate in the survey: one which was more successful in his/her opinion (in terms of technical quality, cost performance, deadlines, customer satisfaction, and overall project performance) and another which was less successful. both these projects were at least half way through their duration (life cycle). the sampling frame consisted of the is project managers and their subordinates. the project manager indicated his/her leadership behaviour and its perceived effectiveness. the project team members also separately indicated the nature of the leadership behaviour of their project manager and their satisfaction with and the perceived effectiveness of the leadership style. 95 ajis vol. 4 no. 2 may 1997 research instruments the multi-factor leadership questionnaire (mlq) (form 5x) (bass and avolio, 1991) was used to examine leadership in this research. the questionnaire comprised seventy eight items that identify and measure key leadership behaviours. it assessed four transformational leadership factors (attributed charisma, idealised influence, intellectual stimulation and individualised consideration), three transactional leadership factors (contingent reward, management-by-exception: active and passive), and three outcome factors (satisfaction with the leader, individual and group effectiveness, and extra effort by followers). additional items (named as technical factors) such as ability to fulfil subordinate's desire for autonomy, implement change, preventing organisational bureaucracy, encourage champions, facilitate career development etc. were added to the mlq to specifically assess the leadership behaviours of is managers and were derived from the meta analysis of the existing literature on technical leadership. leadership effectiveness was measured in two ways: one, by subordinate satisfaction ratings and two, by the project performance data provided by the senior manager of the participating organisations. apart from leadership, the study also examined other contingency factors which are critical to the success of is projects. the project implementation profile (pip) (slevin & pinto, 1986) was used to examine the contingency factors. the pip identifies ten factors which are critical to the successful project implementation out of which three factors have been further identified as more critical than others in the "execution phase" of project life cycle in r&d projects (including computer software and hardware development) (pinto & slevin, 1989). these factors are: project mission, top management support and technical tasks. the survey respondents evaluated these factors with reference to their projects. sample the research began in april, 1995. the survey questionnaires were posted between the period october 1995 to february 1996. in all, 158 organisations were invited to participate in the survey. however, 30 (19%) organisations informed that they were not suitable, leaving a net total of 128 out of which 20 (15.6%) refused to participate and 72 (56.2%) did not respond at all. ultimately, 36 organisations participated in the survey yielding a net response rate of 28.1%. considering that 19% of organisations indicated their unsuitability for the survey, it is quite likely that many organisations which did not respond at all might have been unsuitable too and consequently, the response rate would have been higher. a higher response rate would have been preferred, however a response rate of 28% is reasonable, especially in light of the nature of the content of the questionnaire (steeh, 1981) which specifically asked the organisations to identify and involve unsuccessful projects, a politically difficult task. participant profile out of 36 organisations which participated in the survey, the data from one was unusable leaving 35 organisations comprising of 70 is projects (equally divided between more and less successful projects), 70 project managers and 228 project team members. in terms of performance as measured by the senior managers of the organisations on a scale of l(low) to 5 (high), more successful projects had an average score of 4 while the unsuccessful ones scored 2. the participating organisations had a total of nearly 12,000 is employees. on an average, each is project covered comprised of seven members out of whom three participated in the survey. the average duration of these projects was 8 months. 74% of the participating project managers and 65% of project team members were male. 70% of project managers were at least graduates. there was not enough information available about the qualification of team members. interestingly, both the managers and their subordinates had the same average is experience (15 years) and had spent the same time, on an average, in their current projects (11 months). results table2 shows the comparison of subordinates' ratings of project managers of more and less successful projects. the results show that the internal consistency (cronbach alpha) values for all the scales are significant i.e. >0.7. two-tail test of significance values (p) for all the scales are also significant, i.e. <0.05. these values indicate that there is a significant difference between the leadership styles, outcomes, and contingency scales between more and less successful projects, as predicted. 96 ajis vol. 4 no. 2 may 1997 table2: comparison of more and less successful projects: subordinates' ratings of project managers on leadership, outcome & contingency scales (scale: 1 = high; 5 = low) scale transformational factors 1 . attributed charisma 2. idealised influence 3. intellectual stimulation 4. individualised consdn. total more successful projects mean 18.27 22.91 23.38 22.91 87.47 sd 6.38 7.97 8.33 9.59 28.34 less successful jrojects mean 21.77 27.44 27.31 26.68 102.2 sd 6.95 9.64 8.66 8.50 30.32 t-value 2.83 3.88 3.49 3.11 3.79 df 226 226 226 226 226 p 2 tail .005 .000 .001 .002 .000 a .86 .89 .91 .92 transactional factors 1 . contingent reward 2. mbeactive 3. mbepassive 24.84 21.45 25.56 9.87 5.89 6.36 28.23 22.80 23.49 9.33 5.49 7.17 2.64 1.77 2.28 225 226 210 .009 .078 # .023 .88 .70 .88 technical leadership 27.05 10.65 31.59 11.25 3.12 225 .002 .81 outcome factors 1 . extra effort 2. effectiveness 3. satisfaction 8.38 9.03 4.22 3.59 2.79 1.99 10.07 11.06 4.97 3.47 3.11 2.23 3.52 5.19 2.67 216 225 225 .001 .000 .008 .90 .86 .93 pip 1 . project mission 2. top mangt. support 3. technical tasks total 8.88 11.44 10.47 30.60 2.46 3.98 3.13 7.86 10.17 12.74 12.02 34.69 3.35 4.09 3.16 8.78 3.35 2.42 3.69 3.71 226 223 225 226 .001 .016 .000 .000 .77 .81 .75 notes: 1 n = for more successful projects, 123 subordinates & 35 managers & for less successful projects, 105 subordinates & 35 managers. # since the direction of research was specified, 1-tail test of significance can be applied, the value of which is significant. the scale for leadership, outcome and contingency factors was 1 = high & 5 = low which means lower the mean, higher the score. the mean scores for the transformational leadership scales, outcome scales, technical leadership scale and the contingency scales (pip) for more successful projects are significantly lower than those of less successful projects. these results clearly indicate, as predicted, that in the opinion of the subordinates, the leadership behaviour of a more successful project manager is characterised more by transformational leadership qualities such as charisma, intellectual stimulation and individual consideration and such behaviour produces better outcomes in terms of extra effort, effectiveness and satisfaction. the success of such a leader is also attributable to other contingency factors covered by the study, viz., clear project mission, top management support and availability of required technology and expertise. as regards transactional leadership scales, more successful project managers show less of managementby-exception (passive) behaviour (with higher mean scores) indicating in line with the model that mbe (p) is the only negative leadership trait, whereas contingent reward and mbe (active) are positive traits though not as effective as transformational leadership traits. the results reinforce the assertion made in the model that a leader who 'augments' his^e^ transactional behaviour with transformational behaviour tends to be more successful. table3 compares the leadership ratings with outcome ratings on their correlation co-efficient values (r). all scales of transformational leadership are significantly correlated with outcome scales, as predicted, indicating that subordinates are more satisfied with transformational leaders who encourage them to produce extra efforts and are more effective. amongst transactional leadership scales, 97 ajis vol. 4 no. 2 may 1997 contingent reward is significantly correlated. however, mbe active scale shows a weak correlation whilst mbe passive shows strong but negative correlation, as predicted. based on the strength of correlation, the results indicate that in the opinion of the subordinates of more successful projects, the three most important leadership behaviours of their managers are: attributed charisma, individualised consideration and technical leadership. it is interesting to observe that for these subordinates, the contingent reward behaviour of their managers is more important than intellectual stimulation in terms of successful project outcome. the managers of more successful projects regard technical leadership as more important than transformational leadership in terms of the outcome of their leadership behaviour. table-3: correlation co-efficients: comparison of leadership ratings with outcome ratings scale transformational 1. attributed charisma 2. idealised influence 3. intellectual stimulation 4. individualised consideration transactional 1. contingent reward 2. mbe active 3. mbe passive technical leadership more successful projects subordinates' ratings .81 .72 .63 .75 .71 .48 -.29 .74 managers' (self) ratings .52 .68 .54 .72 .37 .00* -.33 .78 less successful projects subordinates' ratings .70 .62 .58 .71 .66 -.00* -.63 .56 managers' (self) ratings .64 .45 .39 .48 .63 .29 -.44 .48 notes: 1 n = for more successful projects, 123 subordinates & 35 managers & for less successful projects, 105 subordinates & 35 managers. 2 considering the sample, r values >.19 are considered significant 3 all the correlations are significant beyond the .000 level, except those marked with the asterix. 4 the sub-scales of outcome scale are: extra effort, satisfaction and effectiveness. the results of the maximum likelihood factor analysis with varimax rotation show that some of the transformational leadership scales (attributed charisma, idealised influence and intellectual stimulation) are not empirically distinguishable. however, the transactional leadership scales come out as distinct factors. these results are yet to be interpreted in conjunction with confirmatory factor analysis and the partial least squares analysis. to further understand the nature and importance of leadership in is projects, 18 senior is executives of the participating organisations were recently interviewed. these interviews are now being content analysed. conclusions the results of the study indicate that: • there is a significant difference in the leadership styles of more and less successful project managers as perceived by themselves, their subordinates and their senior managers. in other words, leadership and leadership style significantly influence the project outcome. • more successful managers exhibit significantly more of transformational leadership characteristics, namely, attributed charisma, idealised influence, intellectual stimulation and individualised consideration. they also exhibit more of technical leadership characteristics, such as the ability to fulfil subordinates' desire for autonomy, implement change, preventing organisational bureaucracy, encourage champions, and facilitate career development. 98 ajis vol. 4 no. 2 may 1997 • the bass & avolio's leadership model is significantly successful in differentiating between more and less successful is managers based on their leadership behaviour and adequately capturing the key behaviours of successful is project managers. the contingent reward behaviour seems to be more important in the context of an is project environment than some of the transformational leadership behaviours. the results also suggest that there are some other important characteristics of a successful is or technical managers which are not covered . by the model, such as the ones described in the study as technical leadership characteristics. the managers, their subordinates and their senior managers have rated technical leadership behaviours as important, if not more, than some or all of the transformational leadership behaviours. • while the study recommends transformational leadership style to an is manager to enhance leadership effectiveness, it has been rightly pointed out by the senior is managers interviewed for this study that there is no one style which is effective in all projects and it should change depending on the specific situation. however, transformational leadership can be an underlying but flexible leadership style for enhanced effectiveness. • the contingency factors considered in the study are well supported. a leader requires the back up of other critical success factors such as project mission, top management support and technical expertise to ensure success. thus, the study produces evidence that leadership is a critical success factor in an is project environment and that the leadership skills and styles of the project manager significantly influence the outcome of is projects. the research clearly indicates that those is project managers who are charismatic, individually considerate, intellectually stimulating and inspiring are more likely to produce successful project outcomes. references abdel-hamid, tarek, k. m, stuart, e. (1990) the elusive silver lining: how we fail to learn from software development failures, sloan management review, 32(1), 39-48. anderson, s. d. (1992) project quality & project managers, international journal of project management, 10(3), 138-144. atwater, d. c. & bass, b. m. (1994) transformational leadership in teams in bass, b. m. & avolio, b. j. (eds.) improving organisational effectiveness through transformational leadership, sage publications, california. bander, d. (1986) building a better project manager, computerworld, 20(21), 69-78. bass, b. m. and avolio, j. a. (1993) transformational leadership: a response to critiques, leadership theory and research: perspectives and directions, academic press inc. bass, b. m. & avolio, b. j. (1989) (1991) transformational leadership development: manual for the multi-factor leadership questionnaire, consulting psychologists press, california. bass, b. m. (1985). leadership and performance beyond expectations, free press, new york. bates, w. s. (1994) strong leadership crucial, computing canada, 20(22), 32. beatty, c and lee, g. (1992) leadership among middle managers an exploration in the context of technological change. human relations, 45 (9), 957-989. bloom, n. l. (1996) select the right is project manager for success, personnel journal, jan 1996, 6-9. brown, s. (1994) critical success factors in information technology projects, unpublished minor thesis for the degree of master of engineering, rmit, melbourne. burns, j. m. (1978) leadership, harper & row, new york. carless, s. a. (1995) transformational leadership and teams: an examination of the bass and kouzesposner models. unpublished doctoral dissertation, university of melbourne. carter, n. (1988) the project manager: an emerging professional, journal of information systems management, 5(4), 8-14. cash, c. h. & fox, r. (1992) elements of successful project management, journal of systems management, 43(9), 10-12. cleland, d. i. (1995) leadership and the project-management body of knowledge, international journal of project management, 13(2), 83-88. conners, e. (1995), mis 100, strategic publishing group, sydney. deutsch, m. s. (1991) an exploratory analysis relating the software project management process to process success, ieee transactions on engineering management, 38(4), 365-375. geaney, m. m. (1995) the right skills for the job, computing canada, 21(24), 52. 99 ajis vol. 4 no. 2 may 1997 howell, j. m. and higgins, l. a. (1990). champions of technological innovation, administrative science quarterly, 35 (2), 317-341. kapur, g. (1989) running in crisis mode?, computerworld, 23(16), 89-93. keller, r. t.(1995) transformational leaders make a difference, research technology management, 38(3), 41-44. keller, r. t. (1992) transformational leadership and the performance of research and development project groups, journal of management, 18(3), 481-501. kennedy, l. (ed.) (1996). the software 50. summer, 1996, idg communications, st. leonards. laudon k. & laudon j. (1990) management information systems: a contemporary perspective, maxwell maxmillan. palmer, c. & ottley, s. (1990) from potential to reality, a report by the british computer society task group on hybrids, january 1990. parry, k. w. & sarros, j. c. (1994) transformational leadership in australia: how different from the united states?. unpublished manuscript. pinto, j. k. & prescott, j. e. (1988) variations in critical success factors over the stages in the project life cycle, journal of management, 14(1), 5-18. pinto, j. k. & slevin, d. p. (1992) project implementation profile, xicom inc., new york. pinto, j. k. & slevin, d. p. (1989) critical success factors in r&d projects, research technology management, jan-feb 1989, 31-35. pinto, j. k. & slevin, d. p. (1987) critical factors in successful project implementation, ieee transactions on engineering management, em-34(1). phan d, vogel d & nunamaker j. (1988) the search for perfect project management, computerworld, september 1988,95-100. pulk, b. e. (1990) improving software project management, journal of systems & software, 13(3), 231-235. rahn, d. (1987) projects need 'right stuff, computerworld, 21(14), 79-80. rosenbaum, b. l. (1991) leading today's technical professional, training and development, 45 (10), 55-66. sauer, c. (1993) why information systems fail: a case study approach, alfred waller, uk. slevin, d.p. and pinto, j.k. (1986). the project implementation profile: new tool for project managers, project management journal, september 1986. 57-70. steeh, c. (1981). trends in non-response rates 1952-1979, public opinion quarterly, 45,40-57. thite, m. r. (1997) managerial skills in technical professionalsan exploration of it project leadership. in barta, b. et al. (eds.) the place of information technology in management and business education, london: chapman & hall. wateridge, j. (1995) it projects: a basis for success, international journal of project management, 13(3), 169-172. weinberg, g. m. (1986) becoming a technical leader, dorset, new york. 100 microsoft word 13 1 marina.doc ajis vol 13, no. 1 september 2005 40 don’t leave patch management to chance ralph m. defrangesco 1505 breckenridge place harleysville, pa. usa 19438 rdefrangesco@hotmail.com abstract patch management is a costly but necessary part of server management. should more time be spent on patch evaluation and analysis or testing and implementation? this paper attempts to introduce conditional probability into the patch management process giving the reader a tool to evaluate where best to spend their time, analysis or testing. bayes’ theorem will be used to help predict the outcome of a patch management example. introduction today, servers host mission critical applications that are highly reliable and available 24 hours a day, 7 days a week (treese, 2002). customers usually establish server uptime in a service level agreement (sla). server uptime can vary from 99%, for not very critical systems, to 99.999% for extremely critical systems (forsythe, 2005). an sla specifying a server to be up 99.999% of the time equates to 5 minutes of downtime a year (treese, 2002). it is very challenging to apply patches to a server with this type of sla. in an environment where patches get released weekly, how can we expect to analyze and test them thoroughly enough so there is no impact to production? one solution is to use a hot standby for failover purposes and to apply the patches to. this is an expensive option since the server sits around waiting to be used (treese, 2002). systems engineers and administrators need a tool that can help predict the outcome of patch analysis, testing and installation. with time frames shrinking to analyze, test and install patches, how much time should we spend on each activity? systems administrators are caught between a scylla and charybdis, a rock and a hard spot, because management wants it all, but with minimal impact and cost. this paper will interject the use of conditional probability theory, specifically bayes’ theorem as a tool to help predict the outcome of a patch management example. conditional probability allows us to specify the degree of belief in some proposition based on the assumption that some other propositions are true. this paper takes a top down approach meaning we look at the probability an event will occur given a prior probability. bayes theorem bayes’ theorem basically involves calculating a new event on the basis of earlier estimates which come from empirical data (bayes, 2002). we can explain bayes’ theorem with the following equation: in a key feature of bayes’ theorem, we combine our initial probability with new sample data and calculate a revised probability distribution. this new distribution now becomes the ajis vol 13, no. 1 september 2005 41 prior probability for a new sample. the intent is that we make better decisions based on empirical data. application of bayes theorem let’s look at an example. servers crash for a multitude of reasons. in our example, experience has shown us that when a server does crash, it’s due to system administrator error twenty percent of the time and usually can be tied back to an incomplete or untested patch bundle. if a patch bundle is incomplete, there is a five percent probability that it was due to a systems administrator not installing a pre or post requirement. on a similar note, if the bundle was incomplete, there is a fifteen percent probability that it was not tested sufficiently enough. what we need to know is, what is the probability that a bundle is complete and tested but caused a server to crash? for illustrative purposes we decided that our server has crashed ten times during the course of a year and that we spent fifty hours on patch evaluation and analysis. figure 1 uses the bayesapplet (yudkowsky, rovner, 2004) to calculate the posterior probability. let p(a) our prior probability = the percent time that a server crashes due to administrator error, .20%. let p(x|a), our first conditional probability, = the percent of the time a patch bundle is incomplete, .05%. let p(x~a), our second conditional probability, = the patch bundle was not tested sufficiently, .15%. using bayes’ theorem, our equation is given by in so that p (a|x) is the posterior probability of event x given that a is known is equal to the prior probability of x given a (.05), multiplied by the probability of a (.2) divided by the probability of x given a (.05) multiplied by the probability of a (.2) plus the probability of x given (not) a (.15) multiplied by the probability of (not) a (.8). a & x – complete (analyzed) and tested a & ~x – complete (analyzed) and not tested ~a & x – not complete and tested ~a & ~x – not complete and not tested ajis vol 13, no. 1 september 2005 42 figure 1 what we can conclude from figure 1 is that with prior and conditional probabilities as defined in our example, our posterior probability calculates to be 7.7%. we interpret this as meaning that although a patch bundle was complete (analyzed) and tested by a systems administrator that there is still almost an eight percent chance that a server could crash. in our example, it shows up as zero because of rounding (a & x). in figure 2, we are trying to show what the posterior probability would be if a patch bundle was complete but not tested. figure 2 again using the bayesapplet, we calculate the posterior probability to be 21.8%. this tells us that if a patch bundle is complete but not tested, there is a twenty-two percent chance that it will cause the server to crash. the applet rounds the probability to twenty percent (a & ~x). in analyzing both examples, we would be better off spending less time evaluating and more time on testing (~a & x) .10. now that we have both posterior probabilities, the next time we run the model, we must update our prior probabilities with our posterior probabilities because the prior probability was our initial belief. we added new evidence, conditional probabilities, and now we have a final belief. ajis vol 13, no. 1 september 2005 43 both of these examples can be expanded to see if there is any benefit to increasing the time we spend evaluating and analyzing versus testing and implementing. lets say, using the above as an example, that with fifty hours of evaluating and analyzing we end up with a posterior probability of 7.7% as in figure 1. we decide to increase our evaluation and analysis time to one-hundred hours and only reduce the probability to 6.7%. would it be worth the additional time to decrease the posterior probability by one percent? probably not since a two fold increase in hours only yields a one percent decrease in the probability that a server might crash. conclusion we can conclude that patching servers is a risk intense activity. systems administrators do not have a cadre of tools to predict the outcome of applying patches to critical servers, but yet are expected to keep their servers up and running without impact. this paper introduced the concept of applying probability theory to the task of analyzing patches and predicting patching outcomes. in our example, we saw that there may be only incremental benefits to spending more time analyzing rather then testing and implementing. we used this example to demonstrate how to apply conditional probability and the benefits to using this type of theory in patch management. we were able to demonstrate that even though we spent fifty hours analyzing patches, indicating the bundle was complete, we still crashed the server twenty percent of the time. from this, we can conclude that our time would be better spent testing rather then analyzing because our server crash rate is only ten percent. in a paper by kwiatowska (2003), he described probability theory as the following, “probability is also used to quantify unreliable or unpredictable behaviour, for example in fault tolerant systems, communication protocols and computer networks, where properties such as component failure and packet loss can be described probabilistically.” future work this work can be expanded to break out the difference between the amount of crashes caused by cots versus in-house developed software, the probability of backup failures and hardware or operating system failures. the markov process is very similar to bayes’ theorem in that it looks at a number of events and tries to guess the likelihood of the event based on current data. the output will look similar in nature to the original input (deshpande, karypis, 2004). references bayes. (2002. bayes original essay. retrieved on june 7, 2005 from http://www.stat.ucla.edu/history/essay.pdf deshpande. m, karypis. g. (2005). “selective markov models for predicting web page accesses”, acm transactions on internet technology, vol. 4, no. 2, may 2004, pp. 163-184 forsythe, (2005). five 9’s, retrieved on june 7, 2005 from http://www.forsythe.com/forsythe/infraservices/servers/server_five9s.jsp kwiatkowska. m. (2003). proceedings of the 18 th annual ieee symposium on logic in ajis vol 13, no. 1 september 2005 44 computer science (lics’03) treese. w. (2002). building big, reliable web sites, (part 1). net worker, vol. 6, issue 1, pp. 15-18 march 2002 yudkowsky.e., rovner.c., an intuitive explanation of bayesian reasoning, retrieved june 6, 2005 from http://yudkowsky.net/bayes/bayes.html microsoft word ajis112final.doc australasian journal of information systems vol 11 no 2 may 2004 80 adoption of online purchasing in communities, and i ts socioeconomic implications in regional central queenslan d, australia w. j. taylor, g. x. zhu, j. dekkers and s. marshall central queensland university abstract this paper analyses the general trends of online purchasing in central queensland (cq) communities during 1999-2002 and identifies the socio-economic factors affecting online purchasing activities. the online purchasing indicator, defined as a combination of percentages of online purchasers and of regular purchasers (>one item/month) within a group, is applied to compare these activities between these two groups. the study identifies that four factors, namely ‘personal attributes’, ‘knowledge’, ‘trust’ and ‘need’ may play important roles in online purchasing decisions. the research found that regional economic bleeding associated with low local adoption failing to provide justification for local business to adopt electronic purchasing support has not yet reached significant levels. introduction the last decade has witnessed a considerable increase in the use of e-commerce by a wide range of businesses, organisations and institutions in developed economies. the international telecommunications union (itu) claimed that global e-commerce was valued at $37 billion in 1998 and was expected to grow by 80% to us$707 billion in 2003. it also estimated that the value of business-to-business (b2b) e-commerce in 2001-2002 was in a range from us$100 billion to us$400 billion, and that the value of business-to-consumer (b2c) was in a range from us$25 billion to us$90 billion (rashid & al-qirim, 2001). to date much of the research on adoption of ecommerce has focused on national and international trends. the factors affecting the adoption of ecommerce in regional areas, particularly in australia are not well documented. a recent report in the united kingdom (national statistics 2002) indicated that ‘security concerns’ and perceptions of ‘lack of necessity’ are the main reasons for non-adoption of e-commerce. in north america, the national telecommunications and information administration (ntia 2002, p.31) found that geography has little impact on the selection of online activities. however, they found that other socio-economic factors, such as gender, age, race and income do have some relationships with internet user’s selection of online activities including e-commerce. since the establishment of the australian national office for the information economy (noie) in 1997 as australia’s lead agency for information economy issues, the government has given high priority to the improvement of national information and communications technologies (ict) infrastructure as the means to maximise the gains from e-commerce (noie 2001a, p. 6). as a result, australia has been ranked 2nd (behind the usa) among 60 countries in its development of an ebusiness environment (noie 2002, p. 37). noie (2001b) also reported that e-commerce could increase efficiency savings in business during the early adoption stages. it also found that there were significant overall net benefits to the nation from the adoption of e-commerce, and predicted that it could increase australia’s gdp by 2.7% by 2007 (dcita 2000a). despite the low population density across vast areas of australia, the australian government wants the benefits of ict to be spread evenly across the country (dcita 2000b). however, the adoption of ict and e-commerce in regional australia has until recently not been well researched or documented. central queensland (cq), covering approximately 141,936 km2 and with a population of 183,753 has had reasonable and growing access to ict infrastructure over recent years. engaging in e-commerce at the community level in this low population density region has social and economic significance for the regional development. whilst analysing socio-economic factors affecting home internet usage patterns, taylor et al. (2003) found people within 25-39 year old age cohort, and those with higher education levels tended to use internet more for online purchases. other than this research, the adoption of e-commerce at the community level in this low population density region has remained unexamined. researchers (devaraj et al. 2002, gefen & straub 2000, jiang et al. 2000) have employed adaptations of the technology acceptance model (tam) as originally described by davis et al., (1989) to examine the variety of external variables related to the adoption of e-commerce. based on the tam, stylianou et al. (2003) suggested the evaluation of individual perceptions about eaustralasian journal of information systems vol 11 no 2 may 2004 81 commerce should consider four variables: usefulness, ease of use, importance and security. as an extension of the tam, zhu et al. (2003) proposed that these variables are actually internal factors/barriers and could be interrelated, and removal of any one of these barriers could produce a chain reaction that leads to adoption. e-commerce has been traditionally defined as commercial transactions taking place over electronic networks, particularly the internet. the two main forms of transactions being: business-to-consumer (b2c), and business-to-business (b2b) (barnes & hunt 2001). this paper aims to: 1) create a universal indicator to make e-commerce activities comparable between groups within and between communities; 2) analyse socio-economic factors affecting online purchasing activities, and their development trends across 1999-2002; and 3) examine some likely superficial impacts of community e-commerce on the regional economy and b2b engagement within the region. in order to assess the commercial activities thorough the home internet, the term ‘online purchasing’ is used to provide a focus on consumer-to-business (c2b) perspective. whist the transaction value for b2c and c2b is virtually same, the transaction objectives are different, because b2c looks at the profit for the business whilst c2b looks at the cost saving when compared to the similar products at local prices and customer satisfaction after the transaction occurred. methodology data collection central queensland social survey was administrated through a ten-station computer-assisted telephone interviewing (cati) system (mummery and schofield 2002). the cati system ensures that the interview sample reflects the socioeconomic profile of the region. the survey was conducted in november each year across 1999-2002. a random selection approach was used to ensure that all respondents had an equal chance to be contacted by telephone. the survey sample was drawn from the telephone database by using the cati program to select interviewees, and only one eligible person (≥18) was selected as a respondent of a household for the thirty-minute interview. the nine socioeconomic profiles/factors (see fig. 1) drawn from the survey were used in this analysis. this is similar to the survey methods used by the australian bureau of statistics (abs 2001). figure 1 outlines the relevant abstraction from the question framework used to obtain adoption patterns for online purchasing in the region definition of purchase frequency and the community online purchase indicator (opi) online purchasers in a given community can be classified as irregular and regular purchasers. in this study, irregular online purchasing was defined as purchasing less than one item per month and regular purchasing defined as purchasing more than one item per month. the online purchase indicator (opi) is introduced here to compare the overall online purchasing activity between groups within each of socio-economic factors identified above in the central queensland communities surveyed. for a given community group, an opi is defined as a combination of the percentages of irregular online purchasers and the percentage of regular purchaser. it can be expressed as opi (%) = % of online purchasers × % of regular purchasers × 100. australasian journal of information systems vol 11 no 2 may 2004 82 figure 1 cqss computer/internet use question flowchart each year from 2000 to 2002 data analysis this paper analyses the general trends of online purchasing of irregular and regular online purchasers. the nine factors outlined above were analysed individually for their associations with the overall online purchasing activities expressed as opi. graphic demonstrations were used to explain the trend of online purchasing for the period 1999-2002. as the data collected were nominal (frequency) data, non-parametric methods (chi-square tests) were employed to examine significant associations between opi and groups of each a variable/factor (kinnear & gray 1998, p. 110). instead of tables, graphs are used to demonstrate these associations/differences each year and the trends across 1999-2002. however, some of these factors could be interrelated. therefore, binary logistic regressions with categorical covariates were applied to identify factors contributing to opi without the interference of other factors. an odds ratio (or) along with regression coefficient (knoke & bohrnstedt 1994, p. 180) was used to measure the weight of associations of each factor with these percentages. an or value of 1.00 indicates that two variables were unrelated; an or>1.00, indicates the positive covariation of the variables (eg, age and the percentage of online purchasers); an or<1.00, indicates the negative or inverse covariation. as the covariates (factors) are categorical, the first category of each factor was used as a reference (or = 1.00) to detect the association of category variation (eg, from younger to older age groups) with the percentages of online purchasers or regular purchasers. results general trends of online purchase in central queensland table 1 shows that percentages of online purchasers over the total population surveyed are relatively low; just over 16% in 2002. however, figure 2 also shows that there was a steady growth of online yes no usage patterns others e-commerce do you have a computer at home? yes do you access the internet from home? personal profiles location gender age education levels martial status children at home dwelling ownership combined household income employment status no australasian journal of information systems vol 11 no 2 may 2004 83 purchasing through home internet use over the period. the rate of this growth slowed down dramatically in 2002 from over 100% from 1999 to 2000, to 25% from 2001 to 2002. if this growth trend model is valid beyond 2002. it is predicated that by 2005, the online purchase rate will reach about 19.1% of total population in cq. however, the model predicts that growth rate will drop to less than 1 % from 2005 onwards. from 1999 to 2001, there was a slight increase in the percentage of irregular online purchasing and decrease in regular online purchasing in central queensland as shown in figure 3. this indicates that the majority (c. 80%) of central queenslanders had purchased less than one item per month online during that period. however, there was a noticeable increase in purchasing frequency in 2002, with 24% of home internet users being regular purchasers, a 20% increased from 2001. year 1999 2000 2001 2002 sample no 1138 1237 1216 1240 % of home internet adoption 23.6 36.1 42.1 44.7 % of online purchasers with adopters 17.7 23.5 31.6 37.2 % of online purchasers over total population sampled 4.2 8.5 13.3 16.6 table 1 home internet access and online purchasing behaviour in central queensland figure 2 the general trend of online purchasers’ behaviour and percentage of annual increase from 1999 to 2002 in central queensland. year 1999 2000 2001 2002 o nlin e pu rc ha se r (% ) 0 2 4 6 8 10 12 14 16 18 in cr ea se ( % ) 0 20 40 60 80 100 120 on-line purchase increase from previous yr y = 19 .19/( 1 + ex p -(x -2.22)/0 .96 ) ( r 2 = 0.999 , p = 0.000 1) australasian journal of information systems vol 11 no 2 may 2004 84 figure 3 comparisons of online purchasing patterns (irregular vs. regular) in central queensland – 1999-2002 the effects of socioeconomic factors on online purchasing. geographic location figure 4 shows that except for 2000, people living in townships have a more active role in online purchasing when compared to those living in cities and rural areas. it is interesting to note that in 1999, people living in townships and rural areas were much more active online purchasers than those in cities. it can be inferred that this indicates that those living away from city centres have stronger desire to use online purchasing because of the relative inconvenience of shopping associated with geographical location. nevertheless, the adoption of online purchasing was very slow until 2002 when there was a dramatic increase amongst those living in different locations (figure 4). year 1999 2000 2001 2002 ir re gu al r pu rc ha se rs ( % ) 0 75 76 77 78 79 80 81 82 r eg ul ar p ur ch as er s (% ) 0 19 20 21 22 23 24 25 irregular purchasers regular purchasers australasian journal of information systems vol 11 no 2 may 2004 85 figure 4 trends in adoption of online purchaser for people living in cities, townships and rural areas, 1999-2002 figure 5 the effect of gender on adoption of online purchaser in central queensland, 19992002 gender the effect of gender on adoption of online purchasing is displayed in fig 5. males were consistently higher users of the internet from home for online purchasing compared to females. this may suggest that men are more adventurous and have a more confidence and trust in purchasing on line. it may also suggest that there is a greater range of items suitable for males than females available online. year 999 2000 2001 2002 o p i ( % ) 0 2 4 6 8 10 12 city town rural area year 1999 2000 2001 2002 o p i ( % ) 0 1 2 3 4 5 6 7 8 9 10 11 male female australasian journal of information systems vol 11 no 2 may 2004 86 age figure 6 displays the effect of age on online purchasing. except for 2000, people within the 25-39 age cohort were most active in purchasing online from home, followed by those in the 40-55, 18-24 and >55 age cohorts. however, the online purchasing behaviour of those in the 18-24 age cohort steadily increased from 2000. the growth in online purchasing of the seniors (>55 age cohort) is uncertain due to lack of an opi measure in 1999 and 2001, even though its opi matched the youngest age group and was higher than its adjacent younger age group in 2002. figure 6 the effect of age on adoption of online purchasing in central queensland, 1999-2002 education levels figure 7 shows that for the period 1999 – 2000, people with university degrees had a higher opi than those with lesser qualifications. year 1999 2000 2001 2002 o p i ( % ) 0 2 4 6 8 10 12 18-24 25-39 40-55 >55 australasian journal of information systems vol 11 no 2 may 2004 87 figure 7 the effect of education levels on the adoption of online purchasing in central queensland, 1999-2002 the next highest opi was for those with tafe (technical) qualifications. even after 2000, the opi for people with university degrees was higher than those with tafe (technical) qualifications. in addition, the opi of the people with university degrees increased linearly when compared with those with lower qualifications who did not display a discernable pattern of usage. marital status generally, unmarried people were greater users of online purchasing from home when compared to their married counterparts (fig. 8). whilst the causes for this are unknown, the patterns may reflect a greater availability of products of more interest to unmarried people and/or a greater disposable income within the unmarried people. children at home figure 9 demonstrates that there was no consistent difference in opi between people with and without children. however, the opi for those without children increased dramatically at the end of the four-year period when compared to those with at least one child at home. the reasons for this are not clear year 1999 2000 2001 2002 o p i ( % ) 0 2 4 6 8 10 12 14 <secondary tafe uni australasian journal of information systems vol 11 no 2 may 2004 88 figure 8 the effect of marital status on home adoption of online purchasing in central queensland, 1999-2002 figure 9 the effect of children at home on the adoption of online purchasing in central queensland 1999-2002 dwelling ownership except for 1999, people who rented their dwelling were less active in online purchasing when compared to those who own their properties (figure 10). this suggests that people who rent may have less money, but this conclusion may also be impacted by education, marital status and age. year 1999 2000 2001 2002 o p i ( % ) 0 2 4 6 8 10 12 not married married year 1999 2000 2001 2002 o p i ( % ) 0 2 4 6 8 10 12 no children at least one child australasian journal of information systems vol 11 no 2 may 2004 89 combined household income figure 11 displays the effects of combined household income on online purchasing behaviour. prior to 2002, people with lower household income (<au$400 per week) had very low opi when compared to their counterparts with higher income. this was especially noticeable in 1999, when the online purchasing activity for this lower income group was virtually non-existent. however, as evidenced by the matched opi between two income groups in 2002, over the later period there was a growth of opi in the lower income group. this may be related to the general rise in the proportion of people who have home based internet access. figure 10 the effect of dwelling ownership on adoption of online purchasing in central queensland 1999-2002. figure 11 the effect of combined household income on adoption of online purchasing in central queensland, 1999-2002. year 1999 2000 2001 2002 o p i ( % ) 0 2 4 6 8 10 <400 pw >400 pw year 1999 2000 2001 2002 o p i ( % ) 0 2 4 6 8 10 rent own australasian journal of information systems vol 11 no 2 may 2004 90 employment status as may be expected, those fully employed had higher opi across the 2000-2001 period when compared to those who were unemployed (fig. 12). there was also steady increasing trend in opi for those fully employed people across the period, whereas there was no clear increasing trend for those semiand unemployed. however, it is also notable that the online purchasing activities of the semiand unemployed groups increased dramatically during 2001-2002 period, suggesting that financial capacity in itself may not be a factor limiting people engaging in purchasing online. figure 12 the effect of employment status on the adoption of online purchasing in central queensland 1999-2002. factors determining the participation of e-commerce in cq it can be seen that the factors affecting online purchasing behaviours varied during the period 1999 – 2000 (table 2). three external reasons were considered as possible rationale to contribute these variations: other external factors, such as the development of e-commerce market; internal factors, such as people’s understanding of, and attitudes towards e-commerce which could be different across years; and statistical factors, such as sampling errors. however, some of the factors, such as location, age, education level, marital status and household income, appeared more than once across the 1999-2002 period. this could suggest that these factors were less affected by environmental changes during this short period of time, but they had determinant effects on people’s decision for online shopping. therefore, any variation of these factors could influence people’s decision to engage in online purchasing activities. further examination of the data presented in table 2 resulted in the following interpretations. year 1999 2000 2001 2002 o p i ( % ) 0 2 4 6 8 10 fully employed semi-employed unemployed australasian journal of information systems vol 11 no 2 may 2004 91 online purchasers regular purchasers factor category 1999 2000 2001 2002 1999 2000 2001 2002 location city town rural area 1 2.07 0.91* 1 10.41 20.65 1 4.95 2.34 gender male female 1 0.69 age 18-24 25-39 40-55 >55 1 3.49 5.76 7.24 1 0.92* 3.22 1.40* 1 0.07 0.04 0.03 1 0.15 0.20 0.28 education levels secondary tafe uni 1 2.31 1.08* 1 1.52 1.77 1 2.48 1.62 1 4.24 2.41 martial status not married married 1 0.37 1 0.53 1 0.27 children at home no children > one child 1 3.19 dwelling ownership rent own house/flat 1 0.18 household income <au$400 >au$400 1 0.28 1 0.33 1 0.35 1 0.35 (*-indicates no significant relationship with the reference of category 1) table 2 odds ratios abstracted from binary logistic regressions analysis to compare factors determining online purchasing 1999-2002 in people living in townships (as opposed to cities or rural areas) were more likely to engage in purchasing online (both as irregular and regular purchasers) when compared to those in the cities. those older than 25 were more likely to make online purchases when compared to those in the 1824 age cohort. however, the 18-24 age cohort was more likely to become regular online purchasers once they had decided to adopt online shopping. people with tafe or higher education were more likely to engage in online purchasing when compared to those with lower qualifications. people who were not married were more active in online purchasing when compared to their married counterparts. people with lower household incomes were more likely to engage in e-commerce for cheaper commodities. discussion the introduction of opi made it possible to compare e-commerce activities between consuming groups within and between communities. the extended application of this concept is that it can be applied to compare e-commerce activities between regions and countries as well. however, the opi needs to be refined for future use, for example by the inclusion of purchasing/trade values. the reason for the current exclusion of purchasing values in the calculations of opi is that the present ecommerce research is community based. the inclusion of purchasing value would inevitably involve b2b transactions, e.g., by farmers who may also claim to be home internet users, and this would not fit our definition of community e-commerce. comparison/contrast analysis within a single factor indicated that except for 2002, people with higher household income had higher opi than lower income counterparts (fig.11). regression analysis (table 2) indicated that household income was one of the factors affecting home adoption of online purchasing, and that those with lower income were more likely to engage in online purchasing. this difference does not indicate a contradiction. this is because the single factor analysis did not consider other interrelated factors which could be the main contributors to adoption. australasian journal of information systems vol 11 no 2 may 2004 92 for example, the difference of opi in fig.11 may not have resulted from the difference of household incomes, and it could be due to differences of age and education levels (table 3) which are two major factors affecting the adoption of online purchasing (see table 2). the regression analysis was an essential process to determine the interrelated factors that did contribute essentially to the dependent variable (adoption of online purchasing). for example, employment status was a factor in the present regression analysis. it is necessary to consider a comparison of online purchasing as a component of e-commerce adoption in cq with other regions. at the present time there are no data available for comparisons regarding the adoption of online purchasing at either national, and regional levels in australia, or at regional levels for other countries. compared to the limited data available from the reports from usa (ntia 2002, p. 30) and uk (national statistics 2002, p. 6), it seems that adoption of community e-commerce in cq lags behind both the usa and uk (table 4). however, these differences do not necessarily suggest low adoption levels of e-commerce in cq. this is because in the usa research, differences between online purchasing and e-banking is not made clear. whereas, the research reported here, is confined to home based internet use. it does not include personal e-commerce conducted from public access points or from workplaces. education levels age groups income <secondary >tafe 18-24 25-39 40-55 <au$400 pw 65.4 30.3 45.2 16.1 76.8 >au$400 pw 34.6 69.7 54.8 83.9 23.6 p <0.0001 <0.0001 table 3 associations between household income and the corresponding adoption percentages in education level and age group in central queensland, 2002 aug. 2000 (%) sept 2001 (%) oct. 2002 (%) usa 13.3 (includes online purchase and banking) 21.0 (includes online purchase and banking) uk 18.1 cq 8.5 13.3 16.6 table 4 comparisons of adoption of e-commerce between the usa, uk and cq (aust) in addition, it is assumed that, for these already connected home internet, no extra costs are occurred for online shopping when compared to other internet activities. people may engage in online purchasing not only for convenience, but also save costs. nevertheless, the online purchasing in communities could embrace a wider range of activities. this research focused on online purchasing of commodities and services, and the related frequency of purchasing. other activities, such as online banking, online sales from small home business or online trading of household commodities, could also be considered as e-commerce within a community setting. finally, it has to be mentioned that the negative impact of community e-commerce may cause regional economic bleeding because the online shoppers may neglect local business in favour of distance shopping because of price or availability. the present survey found that no online purchases were made from businesses inside the cq region. however, we should look at this positively based on two facts: 1) the current volume of this trade is minimal; and 2) it stimulates b2b transactions between regional and metropolitan levels. for example, from the data collected in 2002, it is estimated that 17% (table 1) of households are online shoppers in the region, and 24% are regular shoppers. as a result, about 4% of total population are regular on-line shoppers in the region. the community e-commerce may push the local business to provide better prices and service through b2b engagement. this can be reflected by comments of an electronic shop owner who did not want to be named, “we realised that some local people buy something from the internet for cheaper price, but we also have our own special purchasing channels either through the internet or phone order. actually, we are doing online shopping for local consumers. we even can get the cheaper price because we order goods in bulk quantities. those who have no internet connection at australasian journal of information systems vol 11 no 2 may 2004 93 home, no credit cards, or just simply don’t trust online shopping, still buy locally. they may have to pay a little bit higher price, but they can get worry free by seeing the real products, having more choices of varieties and having one-year free labour service”. however, the owner did not mention that he recently sacked a storeman not because of a reduction in client numbers, but because of a reduction in the volume of stock due to the adoption of b2b commerce. what is to be blamed for the job loss: e-commerce, ict, or simply the emergence of any new technology? the fact is that due to the adoption of ict, while some jobs have disappeared in the conventional service industries, more job opportunities have been emerging in ict-related industries. therefore, analysing the impact of e-commerce on regional economy and employment is an important task for researchers in regional development. conclusion this paper concludes that some socioeconomic factors, such as location, age, education level, marital status and household income, have constant influence on people’s decision for online shopping, as these factors are less affected by environmental changes. the imbedded attributes of these factors can be interpreted as ‘personal attributes’, ‘knowledge’, ‘trust’ and ‘need’. the introduction of opi provides a tentative ruler to compare e-commerce activities between groups within and between communities. the significance of this concept may lead to more comparative research in online purchasing in the future. moreover, due to these limiting factors, the limited range of commodities available and problems with post-sale service provided by electronically enabled purchasing, the adoption of online shopping is understandably slower than home internet adoption (table 1). in reality it is currently at early stages of maturity and adoption and is considered as just one of the shopping options only for local consumers. thus, the effect of regional economic bleeding due to online purchasing external to the region is not high at this stage. the adoption of online purchasing by community may force the local business to engage b2b transactions for maintaining a competitive position and retaining local consumers. however, high levels of adoption of online purchasing in regional areas may cause employment shift and further research needs to be undertaken on this potential. online purchasing behaviour also needs to be studied in terms of the types of products purchased, the way in which electronic processes are used to aid the purchase decision as well as the socioeconomic factors affecting these behaviours within a community. references australian bureau of statistics (2001). use of the internet by householders november 2000 (no. 8147.0). canberra: abs. barnes, s. and hunt, b. (eds.) (2001) e-commerce & v-business. london, butterworth heinemann davis, f. d., bagozzi, r. p., & warshaw, p. r. (1989). user acceptance of computer technology: a comparison of two theoretical models. management science, 35, 982-1003. devaraj, s., fan, m., & kohli, r. (2002). antecedents of b2c channel satisfaction and preference: validation e-commerce metrics. information systems research, 13(3), 316-333. dcita-department of communications, information technology and (2000a) 'e-commerce: beyond 2000'.canberra, australian government publishing service dcita-department of communications, information technology and arts (2000b) e-commerce across australia.. canberra, australian government publishing service. gefen, d., & straub, d. w. (2000). te relative importance of perceived ease-of use in is adoption: a study of e-commerce adoption. journal of the association for information systems, 1(8), 127. jiang, j., hus, m. k., klein, g., & lin, b. (2000). e-commerce use behavior model: an empirical study. human systems management, 19(4), 265-277. kinnear, p.r. and gray, c.d. (1998) spss for windows made simple. hove, uk psychology press. knote, d. bohrnstedt, g.w.(1994) statistics for social data analysis. illinois, f. e. peacock publishers, inc.: itasca australasian journal of information systems vol 11 no 2 may 2004 94 mummery, k and schofield, g. (2001) 'central queensland social survey 2001.' population research laboratory, centre for social science research. rockhampton, central queensland university. national statistics (2002) internet access-households and individuals. uk national statistics, london. noie-the national office for the information (2001a) b2b e-commerce capturing value online. canberra, australian government publishing service. noie-the national office for the information economy (2001b) advancing with e-commerce-a summary of 34 case studies of small business e-commerce ventures. canberra, australian government publishing service. noie-the national office for the information (2002) the current state of play-australia's scorecard. canberra, australian government publishing service. ntia-national telecommunications and information administration (2002) a nation on line: how americans are expanding their use of the internet. us. department of commerce, washington. d. c. rashid, m. a., & al-qirim, n. a. (2001). e-commerce technology adoption framework by new zealand small to medium size enterprises. research letters in the information and mathematical sciences, 2, 63-70. stylianou, a. c., robbins, s. s., & jackson, p. (2003). perceptions and attitudes about ecommerce development in china: an exploratory study. journal of global information management, 11(2), 17-31. taylor, w. j., zhu, g. x., dekkers, j., & marshall, s. (2003). socio economic factors affecting factors affecting home internet usage patterns in central queensland. paper presented at the informing science and it education (insite) conference, 24-27 june, pori, finland. zhu, g. x., taylor, w. j., marshall, s., & dekkers, j. (2003). why don't these people connect to the internet at home? -analysing the internal barriers causing digital divide in central queensland, australia. paper presented at the the itira 2003 conference, caloundra, qld. australasian journal of information systems vol 11 no 2 may 2004 95 the australasian journal of information systems guidelines for submission of papers 3 copies of the paper should be submitted title of paper title of paper should start at the top of the margin and be centred and bold, followed by authors name and authors address. major headings major headings should be capitalised, bold, left-aligned and no numbering. sub headings sub headings should have initial capitalisation, bold and left-aligned. document formatting papers should be fully justified. there should be no indenting of paragraphs. leave a double space between paragraphs. all text must be typed in the times roman font in 12 pitch. papers should be double on a4 paper. margins are 2.5 cm top, bottom, left and right. pages should not contain page numbers. references should appear in the text as smith (1970) or smith (1970, p 120). the full references should be typed on separate sheets at the end of the manuscript. the following rules should be adopted: monographs smith, x.y. (1970) advanced accounting, sydney: cobb publishing co. periodicals smith, x.y. & smyth, a.b. (1970) "the current cost controversy again", journal of the new school, may, vol 1, pp 10-32. footnotes should be numbered consecutively with superscript arabic numerals. footnotes must appear on the page they are referred to and not as a separate sheet at the end of the manuscript. diagrams should be ink drawn/laser printed. paper should contain a 200 word abstract. a covering page should be submitted with the following details: o title of paper o authors name o authors address papers should be no longer than 20 pages in length, including tables, bibliography and figures. however the editor reserves the right to consider longer articles of major importance [1] ajis vol. 5 no. 2 may 1998 going soft on information systems evaluation barry jackson & sony sulaksono department of information systems, massey university, pajmerston north new zealand b.x.jackson@massey.ac.nz abstract the way organisations do business has changed dramatically over the last twenty years and so have their information systems as they attempt to support and promote the business enterprise. the information system has a key role to play in the success or possible failure of the business yet, despite its importance, there is little evidence that the components of the information systems evaluation process have kept pace with its changing role as they tend to rely on techniques based on narrow tangible/objective methods rather than on more holistic approaches which, in addition to the more traditional methods, include social, political and cultural perspectives more in keeping with the complex interactions associated with modern organisations. the paper charts the recent progress of organisational change, the increasing role of information systems within the organisation, and the information systems evaluation processes. it concludes with a proposal as to how the evaluation process could be widened to include the views and interests of a larger range of stakeholders by drawing upon elements of the soft systems methodology. introduction the last twenty years have seen a tremendous change in the way companies do business. much of this change can be attributed to the so called information revolution. the integrated interaction between information technology and business processes has not only changed the way organisations operate but how society responds to the new environment. the role of information systems within business organisations has changed from its initial function of automating clerical tasks (eg payroll, inventory) to providing pertinent information for operational, managerial and executive groups within the organisation (eg. decision support systems, executive support systems). in more recent times information systems have supported some organisations as they transform or re-engineer their operations in the face of a rapidly changing business environment. the significant benefits accrued by a company as a result of the efforts of its information system group are not necessarily tangible in nature; in fact most appear intangible and as a result are difficult to quantify and measure in a satisfactory manner. nevertheless the cost of information gathering and subsequent analysis is a significant burden for many organisations who, in the face of increasing competition and reduced profit margins, seek to reduce costs and improve efficiency and effectiveness. most information systems regimes appear to focus on the methods that favour tangible and objective assessment ignoring to some degree the organisational and social context in which the system resides. this paper draws upon the interpretivist framework of evaluating information systems based on the idea of content, context and process (serafeimidis & smithson, 1996; avgerou, 1995; symon, 1994) and proposes a soft systems approach to establishing evaluation criteria and strategies. soft systems methodology is ideally suited as an evaluation methodology as it acknowledges cultural, political and social perspectives and, through innovative and meaningful and structured and collaborative debate between accommodating stakeholders, seeks to create an environment where appropriate action can be taken. changing role of is scott morton (1991) argues that established organisations in their attempt to respond to the constantly changing business environment have seen their information systems pass through three phases or generations; automate phase, informate phase and the transformate phase. automate phase the automate phase is associated with attempts by the organisation to reduce costs by replacing repetitive clerical tasks with automated computer-based systems. given this role of information systems to automate existing processes by labour substitution (loveman, 1994; remenyi, money & twite, 1993) traditional cost benefit analysis can be applied to evaluate their performance. in addition to these financial benefits there are further perceived benefits such as greater speed, improved accuracy and so on. 41 ajis vol. 5 no. 2 may 1998 informate phase zuboff (1988), and yates and benyamin (1991) claim that most information systems in the late 1980s and early 1990s did not confine themselves solely to automating business activities but also generated, usually as a byproduct (scott morton, 1991), large quantities of data previously unavailable to the organisation. remenyi et al (1993) and eason (1988) propose that during this informate phase the information systems is intended to facilitate more effective management and control over the organisation's activities and empower users to improve their overall performance. schein (1994) suggests that information systems in the informate phase are also used to make critical organisational processes visible and understandable to other members of the organisation. information systems in this phase of operation would normally expect their performance to be measured against criteria such as; time to arrive at a decision, return on investment, systems usage or user satisfaction. transformate phase remenyi et al (1993), cash et al (1992) and o'brien (1996) claim that in some cases it is possible to radically change the way the organisation does business or even change the nature of the organisation itself through the way information systems are employed. scott morton (1991) warns that transformations of such magnitude may cause a redistribution of power and control within the organisation and as a result may be very threatening for those people involved. quite clearly the transformation phase is associated with people, cultural and political issues. this proposition is supported by schein (1994) who suggests that the cultural assumptions about the nature and use of information systems will themselves be crucial determinants of how such systems will be used to create further innovation. information systems evaluation approaches the changing role of information systems is invariably associated with considerable investment in both human (scott morton, 1991) and physical resources (willcocks, 1994). the increasing level of information systems investment has made the evaluation of such systems an important issue for the organisation (serafeimidis & smithson, 1996; willcocks, 1994; kumar, 1990; hirschheim & smithson, 1988). however, a formal evaluation of information systems is seldom done and if such evaluation is carried out it is usually concerned with those aspects of information systems that are regarded as important by the systems builders and information systems' managers (avgerou, 1995; symon, 1994; kumar, 1990). the information systems literature contains a plethora of articles on information systems evaluation each with its own assessment framework. summarising the literature hirschheim and smithson (1988) claim that most evaluation processes concentrate on technical rather than human, social or organisational aspects of systems. this view is supported by serafeimidis and smithson (1996) and bacon (1994) who surveyed 80 companies in the usa, uk, australia and new zealand and found that 79% of them placed technical systems requirements as the primary evaluation criteria. the difficulties of quantifying qualitative benefits of information systems has led to the use of more subjective techniques but even so such intangible benefits are very difficult to measure (willcocks, 1994; bacon, 1994; dos santos, 1994). however, hirschheim and smithson (1988) suggest that it is possible to classify evaluation techniques and methods into three zones, efficiency zone, effectiveness zone and the understanding zone. these three approaches are not distinct but can be thought of as a continuum ranging from highly objective and rational to very subjective and political. efficiency zone the basic underlying assumption of the efficiency zone is that the systems' functions and goals of evaluation are non-controversial and that the achievement of more precise measurement of these functions is the overall aim of the evaluation process (symon, 1994; hirschheim & smithson, 1988). hirschheim and smithson also claim that methods and techniques such as hardware performance and software maintainability can be classified in this zone. tangible benefits such as these are invariably evaluated through efficiency metrics (parker, benson & trainer, 1988). ginzberg and zmud (1988) argue that most evaluation techniques in common use today were developed at a time where the primary use of information systems was limited to automating clerical tasks within the administrative function of the organisation. not surprisingly therefore most information systems evaluation processes deal with efficiency metrics (saunders & jones, 1992). 42 ajis vol. 5 no. 2 may 1998 effectiveness zone as the role of information systems changes the resulting benefits cannot be easily quantified (wolstenholme, henderson & gavine, 1993; eason, 1988). from an evaluation perspective mahmood and mann (1991) suggest that as the information systems become more embedded within the organisation's business activities it becomes more and more difficult to isolate the information systems factors from other interrelated factors such as norms, beliefs, culture, working environment and so on. efforts to evaluate these systems by using such traditional cost benefit analysis or return on information systems investment metrics have been unsatisfactory (dos santos, 1994; wiseman, 1994; tegginmath, 1993). understanding zone the understanding zone represents a major shift in the evaluation process. it attempts to deal with the benefits that are associated with political and social activities. quite clearly such evaluation will be relative to a particular set of interests which may or may not represent the position of all the stakeholders (sauer, 1993). in the worst scenario the outcome of this evaluation may disproportionately affect some groups leading to a power imbalance within the organisation (avgerou, 1995). hirschheim and smithson (1988) suggest that in order to incorporate social and political aspects of any information systems evaluation a deeper understanding of the nature and the process of information evaluation itself is required. the content, context and process of evaluation symon (1994), avgerou (1995) and serafeimidis and smithson (1996) have replied to this plea from hirschheim and smithson (1988) by proposing an interpretivistic framework to evaluate information systems. this framework is based on the idea of content, context and process of organisational change as developed by pettigrew (1985). the content element provides the central kernel of what is to be evaluated and the process describes how this should be done, while the context investigates the organisational background and environment. these are not discrete independent elements. serafeimidis and smithson (1996) and symon (1994) argue that the elements are strongly linked and interact with each other. the context of evaluation. some researchers are adamant that an information system cannot be evaluated without due regard to its organisational and social context (serafeimidis & smithson, 1996; avgerou, 1995; symon, 1994). cash et al (1992) have illustrated this viewpoint with examples on how organisations which have successfully implemented systems in a narrow technical sense have experienced unforeseen disastrous organisational consequences because they appear to have ignored the context in which the business resides. davis and hamann (1988) go further and claim that the existing business and industrial context helps define what is expected of an information systems. willcocks and margetts (1994) and serafeimidis and smithson (1996) propose that the context of evaluation can be classified into two categories: • external context covers factors such as the economy, government policy, market demands, business competition, availability of supplies, external procedures, funding arrangements. • internal context covers factors such as strategy, organisation structure, rewards system, corporate culture, human resources, individual roles, industrial relations. the content of evaluation flood (1994) suggests that activities performed within an organisation must satisfy some need or purpose. globerson, globerson & frampton (1991) propose that by analysing the purpose of the organisation it is possible to identify and develop relevant criteria which can be used as the basis for measuring the success of an organisation. delone and mclean (1992) used this approach and developed six evaluation measures: • system quality measures the processing system itself in terms of speed, log on time, reliability. • information quality measures the information systems output in terms of clarity, conciseness. 43 ajis vol. 5 no. 2 may 1998 • information use measures the use of the output of an information systems in terms of acceptance of reports, frequency of use. • user satisfaction measures the use of the output of an information systems in terms of user satisfaction, enjoyment. • individual impact measures the effect of information on the behaviour of the recipient in terms of quality of decision analysis, cost awareness. • organisational impact measures the effect of information on the organisational performance in terms of profit performance, return on investment. the process of evaluation serafeimidis and smithson (1996) suggest that the process of evaluation covers how the evaluation is done and how the issues are perceived. from this proposition it can be seen that the process of evaluation is to some extent determined by how the stakeholders view the evaluation process. in fact, as pointed out by yates and benyamin (1991), organisations that take into account the human issues when they adopt an information systems, especially by encouraging participative involvement at multiple levels, frequently consider their investment successful. on the other hand organisations that adopt information systems without taking into account human and organisational issues were not be able to realise the system's full potential. serafeimidis and smithson argue that the process of evaluation should be regarded as a means of encouraging the involvement and commitment of the stakeholders. avgerou (1995) also claims that the evaluation process should be participative allowing all the stakeholders to openly express their views. finally, the process of evaluating an information system should be regarded as a learning process, mediating between the context and content (ward, taylor & bond, 1996; serafeimidis & smithson, 1996; symon, 1994; powell, 1992; etzerodt and madsen, 1988). evaluation is never value free as it invariably contains assumptions that are based on stakeholders' perspectives (symon, 1994). saunders and jones (1992) found that senior managers put greater emphasis on managerial data than did the information system managers, while the latter put more stress on attitudinal criteria and other financial criteria relating to budget status and project completion. gregory and jackson (1992) and powell (1992) argue that the way the stakeholders perceive their organisation will be reflected through the forms of evaluation methods adopted by them. they claim that the objectivist type of stakeholder has the following characteristics: • believes in tangible realities and wants to prove everything • puts more emphasis on the well-being of the organisation as a whole • leans towards conformity at all times and has little regard for individual freedom • concentrates on the end products of employees' labours • tends to view decision making as a burden of those in authority and considers that cooperation of colleagues should be assured by using appropriate rewards and punishments. • prefers quantitative information resulting from surveys and structured interviews. this type of stakeholder is likely to seek to quantify system inputs and outputs in order to attach values to the systems (powell, 1992). the subjectivist type of stakeholders is characterised as follows: • more interested in individual perceptions • tends to be more concerned with the well-being of the individual members of the organisation and the implications of the organisational policies on their lives • capitalises on the unique abilities of the individual • tends to be more concerned with the actual processes conducted to achieve the desired results • tends to view decision making as a participative interaction. this type of stakeholders is likely to adopt subjective methods which recognise the imperfection of computed values and rely instead on the attitudes and opinion of the users and the system builders (powell, 1992). gregory and jackson (1992) argue that both approaches are legitimate. indeed, the policies for evaluating information systems performance are probably shaped over time by integrating common interests and requirements of various information system executives, top managers and key users within the organisation (saunders and jones, 1992). although both subjectivist and objectivist approaches are legitimate approaches toward evaluating information 44 ajis vol. 5 no. 2 may 1998 systems, remenyi et al (1993) suggest that even in the more objectively oriented approaches the determination of the efficiency metric will almost invariably be based on subjective criteria and any suggestion that the method is totally objective should be resisted. soft systems approach to information systems evaluation the context, content and process of evaluation framework outlined above does not provide the analyst with sufficient support to construct an effective evaluation system. it is argued here that soft systems methodology (ssm) could be used by evaluators to gain a better insight into the evaluation framework, context, content and evaluation processes. soft systems methodology consider the modified form of ssm shown in figure 1 (checkland and scholes, 1990). real-world problem situation would-be improvers of the problem situation the situation as a culture analysis of the intervention 'social system' analysis 'political system' analysis levant systems models situation a a a stream of cultural analysis differentiate between models and the real world changes: systematically desirable culturally feasible 4 cction to inrprcvethesitudicn logic-base stream of analysis figure 1 45 ajis vol. 5 no. 2 may 1998 (i) finding out, history and would-be improvers would-be-improvers enters the real world problem situation with the intention of improving matters. entry into the situation is acceptable as historical information has suggested that there are a number of conflicting issues which need examining. (ii) streams of enquiry two interacting streams of structured inquiry are undertaken in order to clarify the issues and seek ways to debate them. the intention is that the process will lead to meaningful and feasible changes being made that will improve the situation. the cultural stream of enquiry consists of three analyses of the problem situation; intervention of the would-be improvers, situation as a social system and situation as a political system. the logical stream of enquiry identifies relevant systems that address both primary-tasks and issues. checkland and scholes (1990) suggest that a primary-task system is normally visible in the real world and represents some organised purposeful action whereas an issue-based system is essentially a system concerned with reducing the effects of human conflict. such systems are not normally mapped into the real world. these systems are expressed using closely worded root-definitions which describe the system's transformation process. the definitions also acknowledges the worldview (weltanschauung), stakeholders' views and opinions and system constraints. a conceptual model is then constructed which reflects each root definition and includes three measures of performance, efficiency (eg. have we been prudent in the use of resources?), effectiveness (eg. have we achieved the goal set out in the rd's transformation?) and efficacy (eg. have we done what we set out to do?). (iii) differences between models and the real-world the conceptual models are then used to ask questions about the real world. such questioning leads to debate. the aim of the debate is not to improve the model but to find accommodation between the different interests in the problem domain. achieving accommodation requires cultural knowledge acquired through actions performed in the cultural stream of enquiry. in fact, both streams of enquiry support each other's learning by recursive interplay. (iv) changes: systematically desirable and culturally feasible seeking to identify desirable and culturally feasible changes to improve the situation requires reference to the various world-views (weltanschauungen). (v) action to improve the situation agreed action is taken, learning takes place and the process is then repeated. relevance of ssm to information systems evaluation ssm has relevance to information systems evaluation for a number of reasons. • ssm has been used for some years and it is a well known and credible methodology. • the overall aim of ssm is to take subjectivity seriously rather than intuitively so that a meaningful purposeful action can be taken. • an information system involves many stakeholders each of whom possesses a different perspective and level of commitment within the problem domain. • soft systems methodologies encourage those concerned to openly declare their judgments and views and, by using root definitions and conceptual models, matters of concern can be debated. • a key component of the methodology is for the stakeholders to reach accommodation. one of the ways of focusing on accommodation is to examine the purpose of the system. this can be achieved through building a model of purposeful activity and declare the worldviews on which the model is built (checkland, clarke & poulter, 1996). • ssm forces the users to analyse political and social dimensions • ssm is a methodology for learning. • an integral feature of ssm is that it provides a mechanism for the elements of the system to be monitored and controlled. checkland & scholes (1990) argues that in monitoring one must look at certain features related to standards of good and bad performance and therefore the monitoring activity is contingent upon defining measures of performance. globerson et al (1991) state that performance is an expression of the level of expected achievement or behaviour. in addition, they go on to argue, setting standard of performance is an ongoing process, depend on the nature of the organisation. therefore, it can be said that as the efficacy, efficiency and effectiveness criteria 46 ajis vol. 5 no. 2 may 1998 change over time, the criteria against which the information systems is evaluated will also change over time. evaluation using ssm (i) scenario consider an organisation which has passed through the automate phase. let us assume that would-be improvers of the situation have entered the problem domain and have performed several of the ssm activities and have identified several relevant systems. one relevant task-based system suggests that the organisation would like to evaluate its information system from a wider perspective than it had done previously. this decision is based on the observation that the information system provides both tangible and intangible benefits to its operational, managerial and executive staff. in addition to the task-based issue a cultural issue has surfaced that would suggest that some information systems specialists are sceptical perhaps even antagonistic towards the broadening of the evaluation process. on the basis of the 'finding out' phase two relevant systems were identified (rsi, rs2): rs1. a system to identify broad-based information systems evaluation criteria. rs2. a system to change the views of the information systems specialist so that they will accept the idea that interpretivistic evaluation criteria are acceptable and effective measures of systems performance. (ii) root definition (rd1) and catwoe the following root definition and catwoe capture the essence of the first relevant system. the second relevant system is not pursued as this stage. quite clearly it is important but does not contribute directly to the evaluation process. rd1. a company-owned and operated system, staffed by members of the information systems group, which identifies appropriate (information systems) evaluation methods by using elements of soft systems methodology to enable the information system better fulfil its role in supporting the business function. the system is to operate within the current company's structure and resources. catwoe the elements of catwoe form a useful mnemonic (c customers, a actors, t transformation, w weltanschauung, o owner, e environmental constraints). it allows the developers to check their root definition and conceptual model for completeness. the key element in catwoe is the transformation process while the other elements describe the people and environment in which the transformation takes place. c information systems management, users, management a information systems staff, users, audit team t need for appropriate evaluation methods representative of the role of information need met * systems within the organisation w a belief that management is willing to expand the scope of evaluation to include subjective criteria o management e company's structure, financial and time constraints it is expected that choosing different combinations of catwoe would result in different perspectives of the problem domain being exhibited. this would lead to an increase in the richness of the understanding of the situation and would enhance debate and discussion which in turn could lead to an acceptable accommodation by the parties involved. these catwoe components imply: • evaluation will benefit the organisation as a whole as well as individual or groups of stakeholders (customers). • evaluation of information systems needs the support of the stakeholders (customers, owners of the system (owners), and those who carry out the evaluation process (actors). •. the system must contains a transformation process. 47 ajis vol. 5 no. 2 may 1998 • the information system resides within an environment where the organisation and its members have certain beliefs, norms and values (hirschheim and smithson, 1988; avgerou, 1995). this is captured by the weltanschauungen (w) and while not a constraint in the normal meaning of the word does affect whether changes to the systems are culturally desirable and acceptable. the root definition and its catwoe components should be defensible and reasonable to the organisation. (iii) conceptual model the conceptual model (figure 2) shows how the transformation in the root definition could be achieved. it too should reflect the catwoe components. understand the aims, \ values andactivities of j the organisationunderstand the context of evaluation understanding the role information systems has in supporting the organisation (three phases)/ understand the i content of v evaluation identify the benefits associated with the role of information systems understand the process of evaluation identify realisable evaluation criteria from within and without the organisation 7 create lists of evaluation criteria 11 monitor 1-810 define: efficiency effi 12 take control actio figure 2 the model is a representation of a purely abstract idea and cannot be validated against any real world situation and is only one model out of many relevant to debating the real world situation. the model is used to generate 48 ajis vol. 5 no. 2 may 1998 debate by comparing it with a real world situation to highlight some changes that are culturally feasible and systemically desirable so that a meaningful action can be taken. it can be seen from the above model that the system and its sub-systems are themselves monitored and controlled. monitoring and controlling the evaluation process will enable it to learn and improve its performance. embedded in these criteria are organisation resources, political activity, social values, etc. summary the changing role of information systems within enterprising organisations has brought about the consequence that information systems evaluation can no longer rely exclusively on objective methods if it is to gain any determination of the value of the information systems. the political nature and social condition have become crucial determinants in any evaluation of information systems. the need to effectively evaluate information systems has never been greater. the cost of developing and implementing such systems has risen alarmingly and it has become good business practice for organisations to scrutinise all their investment to ensure that their investments do indeed provide added value. ssm coupled with an interpretivist framework should provide evaluators with a methodology that not only identifies contextual and content based evaluation criteria but establishes a learning cycle that monitors and controls the entire evaluation process. ssm takes account of the subjective nature of human interactions and treats these interactions rigorously rather than intuitively. ssm adopts a dual approach to domain assessment by examining the interplay between the logic and cultural aspects of the problem domain. the cultural stream draws upon the social and political dimensions while the logical stream features tasks and activities within the problem domain. the resulting relevant systems, root definitions and conceptual models promote debate among the stakeholders so that the learning process is enabled and accommodation can be reached. the process culminates in an agenda for change that results in desirable and culturally feasible action to improve the situation. ssm also pinpoints the importance of having a system that is able to learn from its experience and from new external knowledge. references avgerou, c. (1995). "evaluating information systems by consultation and negotiation" international journal of information management, vol 15,6, pp 427-436. bacon, j. (1994). "why companies invest in information technology." in l. willcocks (ed.), information management: the evaluation of information systems investments, chapman and hall cash, j. i., mcfarlan, f. w., mckenney, j. l. (1992). corporate information systems management, richard d. irwin. checkland, p. b., clarke, s. & poulter, j. s. (1996). "the use of soft systems methodology for developing hiss and im & t strategies in nhs trusts." in b. richards & h. de glanville (eds.), healthcare computing 1966: proceedings from the perspective in healthcare computing conference, pp. 223233, harrogate, united kingdom. checkland, p. b. & scholes, j. (1990) soft systems methodology in action, wiley, chichester. davis, g. b. & hamann, j. r. (1988). "in-context information systems assessment: a proposal and an evaluation." in n. bjorn-andersen & g. b. davis (ed.), information systems assessment: issues and challenges. noordwijkerhout, north holland. delone, w. h., mclean, e. r. (1992). "information systems success: the quest for the dependent variable," information systems research, vol 3,1, pp 60-95. dos santos, b. (1994). "assessing the value of strategic information technology investments," in l. willcocks (ed.), information management: the evaluation of information systems investments, chapman and hall eason, k. (1988). information technology and organisational change, taylor and francis. etzerodt, p. & madsen, k. h. (1988). "information systems assessment as a learning process." in n. bjomandersen & g. b. davis (ed.), information systems assessment: issues and challenges. noordwijkerhout, north holland. flood, r. l. (1994) "total systems intervention (tsi): a reconstitution," journal of the operational research society, vol 46, pp 174-191. ginzberg, m. j. & zmud, r. w. (1988). "evolving criteria for information systems assessment." in n. bjornandersen & g. b. davis (ed.), information systems assessment: issues and challenges. noordwijkerhout, north holland. 49 ajis vol. 5 no. 2 may 1998 globerson, a., globerson, s., frampton, j. (1991). you can't manage what you don't measure, avebury gower publishing company ltd. gregory, a. j. & jackson, m. c. (1992). "evaluation methodologies: a systems for use," journal of the operational research society, vol 43,1, pp 19-28. hirschheim, r. & smithson, s. (1988). "a critical analysis of information systems evaluation." in n. bjomandersen & g. b. davis (ed.), information systems assessment: issues and challenges. noordwijkerhout, north holland. kumar, k. (1990). "post implementation evaluation of computer-based information systems : current practices". communication of the acm, february, vol 33, 2, pp 203-212. loveman, g. w. (1994). "an assessment of the productivity impact of information technology." in m. s. scon morton, (ed.), the corporation of the 1990s. oxford university press. mahmood, m.a. & mann, gj. (1991). "measuring the impact of it on organisational strategic performance: a key rational approach." proceedings of the twenty fourth hawaii international conference on systems science, vol 4, pp 251-258. o'brien, j. a. (1996). management information systems: managing information technology in the networked organisation, richard d. irwin parker, m. m., benson, r. j. & trainer, h. e. (1988) information economics, new jersey, prentice hall. pettigrew, a. m. (1985). the awakening giant: continuity and change in ici, oxford, blackwell. powell, p. (1992). "information technology evaluation: is it different?" journal of the operational research society, vol 43, 1, pp 29-41. remenyi, d. s. j., money, a. & twite, a. (1993). a guide to measuring and managing it benefits, 2nd edition, tech trans limited. sauer, c. (1993). why information systems fail: a case study approach, alfred waller limited. saunders, c. t. & jones, j. w. (1992) "measuring performance of the information systems function," journal of management information systems, vol 8,4 pp 63-82. schein, e. h. (1994). "innovative cultures and organisations." in m. s. scott morton, (ed.), the corporation of the 1990s. oxford university press. scott morton, m. s. (1991). "introduction." in m. s. scott morton, (ed.), the corporation of the 1990s. oxford university press. serafeimidis, v. & smithson, s. (1996). "the management of change for information systems evaluation practice experience from a case study," international journal of information management, vol 16, 3, pp 206-217. symon, v. (1994). "evaluation of information systems investments: towards multiple perspectives." in l. willcocks (ed.), information management: the evaluation of information systems investments, chapman and hall. tegginmath, s. (1993). "a psychometric evaluation of a measure of user information satisfaction." working paper, department of management and information systems, university of auckland. ward, j., taylor, p. & bond, p. (1996). "evaluation and realisation of is/it benefits: an empirical study of current practice," european journal of information systems, vol 4, pp 214-225. willcocks, l. (1994). "introduction: of capital importance." in l. willcocks (ed.), information management: the evaluation of information systems investments, chapman and hall willcocks, l. & margetts, h. (1994) "risk and information systems: developing the analysis". in l. willcocks (ed.), information management: the evaluation of information systems investments, chapman and hall. wiseman, d. (1994). "information economics: a practical approach to valuing information systems." in l. willcocks (ed.), information management: the evaluation of information systems investments, chapman and hall wolstenholme, e. f., henderson, s., gavine, a. (1993), information systems evaluation methodology, chichester, john wiley and sons. yates, j. & benyamin, r. i. (1991). "the past and present as windows on the future." in m. s. scott morton, (ed.), the corporation of the 1990s. oxford university press. zuboff, s. (1988). in the age of the smart machine: the future of work and power, new york, basic books. 50 reducing software failures: addressing the ethical risks of the software development lifecycle by don gotteibam software engineering ethics research institute, east tennessee state university, usa abstract a narrow approach to risk analysis and understanding the scope of a software project has contributed to significant software failures. a process is presented which expands the concept of software risk to include social, professional, and ethical risks that lead to software failure. using an expanded risk analysis will enlarge the project scope considered by software developers. this process also is incorporated into a software development life cycle. a tool to develop software development impact statements is also discussed. introduction software developers and software engineers have been evolving and refining techniques to mediate risks of developing software products that meet the needs of their clients. the risks focused on include: missed schedule, over budget, and failing to meet the system's specified requirements. in spite of this attention to risks, a high percentage of software is being delivered late, over budget, and not meeting all requirements, leading to software development being characterized as a "software crisis". software risk the problem in this paper i correct the meaning of "software failure", or more precisely, focus attention on some overlooked meaning of "software failure". software fails even when it is produced on schedule within budget and meets the customer's specified software requirements. software has been developed which, although meeting stated requirements, has significant negative social and ethical impacts. by ethical impact i mean those impacts of software which positively or negatively the circumstances, experiences, behavior, livelihood, or daily routine of others. the ethical stakeholders in software are those who are so affected. the aegis radar system, for example, met all requirements that the developer and the customer had set for it. the system designer's did not take into account the users of the software nor the conditions in which it would be used. the system was a success in terms of budget, schedule, and requirements satisfaction, even so, the user interface to the system was a primary factor in the vincennes shooting down an iranian commercial airliner killing 263 people. there are two factors that contribute to these professional and ethical failures; both related to those who have something to gain or lose as a result of the software project — system stakeholders. first, there is significant evidence that many of these failures are caused by limiting the consideration of system stakeholders to just the software developer and the customer. this limited scope of consideration leads to developing systems that have surprising negative affects because the needs of relevant system stakeholders were not considered. in the case of the aegis radar system the messages were not clear to the users of the system operating hi a hostile environment. second, these types of failures also arise from the developers limit the scope of software risk analysis just to technical and cost issues. a complete software development process requires 1) the identification of all relevant stakeholders and 2) enlarging risk analysis to include social, political, and ethical issues. i propose to add a process to a standard life cycle model that will help identify the relevant stakeholders and broaden the scope of risks anticipated. software quality and its risks the primary goal of software developers is the production of quality systems that meet the needs of the user. "software quality" is defined in terms of customer satisfaction. "risk" is understood as any potential threat to the delivery of a quality product. to meet the goal of quality software, developers focus on particular risks including: project and schedule slips, cost increases, technical and quality risks, the timeliness of the product, risks that the final product will not fit the business for which it was designed. projects are managed focusing on these risks. tools used to help identify and manage these risks include: risk tables, and lists of risks categorized by type, probability and impact. the checklist process is reminiscent of the process pilots go through before take off. as airline passenger we are made more comfortable by the fact that they go through this procedure. but unlike pilots, developers choose to ignore some risks. risks levels are 155 determined based on the anticipated impact of the risk and its probability of occurring. only risks above limited specified levels are addressed. software development life cycles (sdlc) and risk there are several models for developing software that reduce planning risks. all of these models contain similar phases: develop a statement of the customer's desires the requirements phase; design how to achieve those desires the design phase; code and test what was designed the implementation phase; and determine that the requirements are satisfied by the system developed system testing phase. the ordering and content of the steps through these phases called the system development life cycles (sdlc). many sdlcs are linear and require the documented completion of a one phase before going on to the next phase and are directed at the satisfaction of the customer's explicit requirements. only a few sdlcs include any risk analysis or expand the number and type of system stakeholders considered. spiral lifecycle and risk barry boehm's "spiral lifecycle" is one of the few sdlcs that specifically address risk. following his model, software is developed in a series of incremental releases. each iteration through the spiral includes tasks related to: customer communication, planning, risk analysis, engineering the development of the next level, construction and release and, customer evaluation and assent. each incremental element of the product that passes through these phases has undergone risk analysis and evaluation by the customer. although this model introduces a focus on risks, those risks are limited to the risks identified above. lifecycle measurement (lime) and stakeholders lime [ buglione, 1999] introduces a multidimensional analysis of quality and performance during software development. this model defines quality in terms of an economic dimension from the managers' viewpoint with particular attention to cost and schedule drivers; a social dimension from the users viewpoint, and a technical dimension from the developer's viewpoint with particular attention to technical quality that has a different impact during each sdlc phase. this model examines the role of a stakeholder in all phases of the project development. although this method includes a consideration of a stakeholder in all phases of project development, the stakeholder is the user for whom quality is achieved by the satisfaction of the specified requirements. the real problem limitations of lime and the spiral sdlc both of these models are improvements over earlier sdlcs that either had no risk analysis or had a very limited view of system stakeholders. the spiral model incorporates risk analysis and lime incorporates a consideration of an additional stakeholder throughout the development process. the risks considered by the spiral model are the technical risks identified earlier and lime expands the risk analysis to include risks to one stakeholder in terms of requirements specification. all of these methods attempt to anticipate and avoid all potential threats to a software project. the negative possibilities are those that would delay the delivery of the software that performs the desired functions in a timely and cost effective manner. however, none of these methods consider the ethical issues that need to be identified and addressed during the planning stages and re-considered throughout the development process. failure research recent research has confirmed that inadequate identification of project stakeholders and how they are affected by a project is a significant contributor to the project's failure. establishing the right project scope is essential in defining project goals. the stakeholders determine the scope of consideration. normally, the stated needs of the customer are the primary items of concern in defining the project objectives. investigating 16 organizational isrelated projects led [farbey et al, 1993] to conclude that regarding evaluation of it investment, "... the perception of what needed to be considered was disappointingly narrow, whether it concerned the possible scope and level of use of the system, [or] the range of people who could or should have been involved ...". they discovered, with the exception of vendors, all stakeholders involved in evaluation were internal to the organizations. the reason for this restricted involvement is that these are the only stakeholders originally identified in the traditional project goals or system requirements. we should not limit our consideration of 156 stakeholders to those who are financing the project or politically influential. stakeholders are individuals or groups who may be directly or indirectly affected by the project and thus have a stake in the development activities. those stakeholders who are negatively affected are particularly important. negative effects include both overt harm and the denial or reduction of goods. so obviously the development of medical software that delivers erroneous dosages of medicine that killed patients would have a negative effect; but we would also include as having a negative effect software that limited people's freedom of expression. limitations on positive ethical values and rights are negative effects. many companies have gone out of business because they have only emphasized short-term efficiency and productivity. the quantity and cost of major product recalls in terms of dollars and company reputation is evidence of this mistaken emphasis on short-term goals. when considering software development we need to .consider the impact of the system as a whole. in the past, the developers have restricted their involvement in the development of a product to the technical elements of a piece of software. this self-imposed limitation has contributed to the development of software that has been inferior and has had negative consequences for others: software that is not socially sensitive. the systems we develop perform tasks that affect other people in significant ways. the production of quality software that meets the needs of our clients and others requires both the carefully planned application of technical skills and a detailed understanding of the social, professional, and ethical aspects of the product and its impact on others. frequently the failure to consider social, ethical, and other risks has led to the delivery of unacceptable software that should be recalled and modified. because the process of recall and modification is too expensive for the developer, the product remains on the market. the scope of a project needs to be identified in terms of its real stakeholders. the expansion of the scope of a project to include all relevant stakeholders will also broaden the types of risks considered. many companies have gone out-of-business because they have only emphasized short-term efficiency and productivity. the quantity and cost of major product recalls in terms of dollars and company reputation is evidence of this mistaken emphasis on short-term goals. when considering software development we need to consider the impact of the system as a whole. in the past, the developers have restricted their involvement in the development of a product to its technical elements. this self-imposed limitation has contributed to the development of software that has been inferior and has had negative consequences for others. the systems we develop perform tasks that affect other people in significant ways. the production of quality software that meets the needs of our clients and others requires both the carefully planned application of technical skills and a detailed understanding of the social, professional, and ethical aspects of the product and its impact on others. stakeholder identification some of these software development methods distinguish between direct system stakeholders—[those who]"receive services from the system and send control information to the system"—and indirect stakeholders— [those who] "have an interest in some of the services that are delivered by the system but do not interact directly with it". these would include the passengers on the iranian airliner or the driver of an automobile whose brakes are controlled by a computer program. unfortunately 1) these methods do not provide an ethical or philosophical foundation for this distinction to reach beyond identifying those who have a business relation to the customer. they would not have identified as indirect stakeholders the 47 people killed by falling debris from a patriot missile. these methods also fail to 2) provide a method of identifying the social and ethical impacts on the indirect stakeholders. we need to extend the traditional software project stakeholder list from customers and corporations or shareholders to include all those who will be affected by the software and by its production. this enlargement of the domain of stakeholders has been implicitly endorsed by professional societies in the paramouncy clause — " protect public health, safety, and welfare" in their codes of ethics. this extension has been explicitly adopted in several legal decisions in the united states. this extended domain of stakeholders includes: users of the software, families of the users, social institutions which may be radically altered by the introduction of the software, the natural environment, social communities, software professionals, employees of the development organization and the development organization itself. given such a range of stakeholders, how is one ever to learn how to identify the relevant and significant stakeholders? software development impact statement funded research has been done on the development of a risk management process employing software development impact statements. the software development impact statement (sodis), a modification of an environmental impact statement, is a way of addressing the need to modify project tasks in a formal way. a sodis, like an environmental impact statement is used to identify potential negative impacts of a proposed 157 project and specify actions that will mediate those impacts. a sodis is intended to reflect both software development process and the more general obligations to various stakeholders. we can divide software project development into three distinct phases. they are: the feasibility phase that includes considerations of preparedness to start a project and managing action items needed to start the project; the requirements phase that defines the specifications of a system and identifies and manages potential risks with each requirement; and the detailed phase that uses a detailed software project management plan to manage each task on system development. each of these phases has its own peculiar risks. the purpose of the sodis is to identify these risks in a pre-audit of each phase. in the requirements phase, we can develop a high level analysis of the expected impacts of a project. a detailed sodis is developed from a preliminary software development plan. the goal of the sodis process is to identify significant ways in which the completion of individual tasks may negatively affect stakeholders and to identify additional project tasks needed to prevent any anticipated problems. a detailed sodis is developed from a preliminary gantt chart. the process of developing a sodis encourages the developer to think of people, groups, or organizations related to the project (stakeholders in the project) and how they are related to each of the individual tasks that collectively constitute the project. the goal of the sodis process is to identify significant ways in which the completion of individual tasks may negatively affect stakeholders and to identify additional project tasks needed to prevent any anticipated problems. . although all software projects have some unique elements, there are significant similarities between projects so that a generic practical approach can be taken to refocus the goal of a project to include a consideration of all ethically relevant stakeholders as well as all technically relevant stakeholders jmsodis project auditor welcome • fifa took hefc -i |x| isodis project auditor for windows risk analysis and tracking version 30 evaluation copy : copyright-2001; don.gcflerbam arri srion roserson u.s.̂ and foiwpipatentspehdngvy1 '-7:;; v ' . . ' 'welcome to the sodis project audita. select the.stage of analysis on which you want to work. 'cnoosc roft&tto^y andystt ,• ncqurontcnts amlysts f of uctoocd afvsyos vu picss ok to ptocg&q, o'beg»»eea*a>f|y analysis use feaabiity analysis to determine if a proposed project is ready to begn. .use idetaited analysis to identify risksiin a project's management pian figure 1 to aid with the major clerical task of completing this process for every task and for every stakeholder a tool the sodis project auditor was developed. the sodis project auditor keeps track of all decision make about the impact of project tasks on the relevant project stakeholders and it enables a proactive way to address the problems identified. 158 sodis stakeholder identification a preliminary identification of software project stakeholders is accomplished by examining the system plan and goals to see who is affected and how they may be affected. when determining stakeholders, an analyst should ask: whose behavior, daily routine, work process will be affected by the development and delivery of this project; whose circumstances, job, livelihood, community will be affected by the development and delivery of this project, and whose experiences will be affected by the development and delivery of this product. all those pointed to by these questions are stakeholders in the project. stakeholders are also those to whom the developer owes an obligation. the imperatives of the software engineering code of ethics and professional practice and similar codes define the rights of the developer and other stakeholders. these imperatives can be used to guide the stakeholder search. the process of identifying stakeholders also identifies their rights and the developers' obligations to the stakeholders. many of the computing codes have similar imperatives. these have been reduced and categorized under five general principles in the sodis process and incorporated into the sodis project auditor. on a high level, the sodis process can be reduced to four basic steps: (1) the identification of the immediate and extended stakeholders in a project, (2) the identification of the tasks or work breakdown packages in a project, (3) for every task, the identification and recording of potential ethical issues violated by the completion of that task for each stakeholder , and (4) the recording of the details and solutions of significant ethical issues which may be related to individual tasks and an examination of whether the current task needs to be modified or a new task created in order to address the identified concern. the sodis process also includes a consideration of other phases of an sdlc. some risks can be identified when a project is first conceived or can be identified at an intermediate stage when the customer's desires are being specified in the requirements phase. the sodis project auditor also provides a pre-audit for diese two project phases. a complete sodis process 1) broadens the types of risks considered in software development by 2) more accurately identifying relevant project stakeholders. the utilization of the sodis process will reduce the probability of the types of errors identified by farbey. the sodis should be part of a sdlc. identification of stakeholders the identification of stakeholders must strike a balance between a list of stakeholders that includes people or communities diat are ethically remote from the project, and a list of stakeholders that only includes a small portion of the ethically relevant stakeholders. the system provides a standard list of stakeholders that are related to most projects. this standard list of stakeholder roles changes with each change of project type. for example, a business project will include corporate stockholders, while a military project will not have stockholders as a standard stakeholder role. the system also enables the sodis analyst to add new stakeholders roles. the stakeholder identification form (figure 2) contains a statement of work that helps remind the analyst of the project goals and facilitates the identification of relevant stakeholders. the stakeholder form and the sodis analysis form are dynamic and enable the iterative process. if while doing an ethical analysis, one thinks of an additional stakeholder he/she can shift to the stakeholder identification form, add the stakeholder, and then return to the sodis analysis that will now include the new stakeholder. rogerson & gotterbarn [1997] proposed a method to help identify stakeholders based on gert's moral rules [gert 1988]. gert gives 10 basic moral rules. [gotterbarn 1991] these rules include: don't kill, don't cause pain, don't disable, don't deprive of freedom, don't deprive of pleasure, don't deceive, don't cheat, keep your promises, obey the law, and do your duty. these rules carry with them a corresponding set of rights such as the right to liberty, physical security, personal liberty, free speech, and property. how can these rules be used to identify stakeholders? 159 qflsodis pioject auditor ji.'bto edft.ioote, fiepprt :jle|p tdngjjpnt'l s^ftjft&jda^ li anajgg^^vmll&f!*te» mjjj.-. ^ project lnfamalib . wane: : cyclone launch ' ; .; ;• "fji :'•'"'"• ;r"d*ter project statement of work: a general description of the project to aid analysts think of stakeholder ' 1. behavkuavoik process wi be affected by the development 01 detveiy of this project •z -cinxanstance/job w8 be affected.by the development ot desywy of this ptoject torcfcfivef)>.o'>wsp'oject . . name:[customer figure 2. stakeholder identification a matrix can be set up for each ethical rule such as "don't cause harm the column headers of the "don't cause harm matrix" are the stakeholders, such as the "developer" and the "customer", and there is a row for each major requirement. the sodis analysts then visits each cell in the matrix asking, for each requirement whether meeting this requirement violates that obligation to the stakeholder. because the analysis as described is organized by particular software requirements, it will be easy to identify those requirements that generate a high level of ethical concern. thus, the list will also be used to determine if particular requirements have to be modified to avoid significant ethical problems. this method can be used at this stage to give a composite picture of the ethical impact of the entire project from the point of view of these stakeholders. might the completion of this requirement cause harm to the stakeholder? ('y' indicates that the task may cause harm to the stakeholder group) req\stakeholder requirement 1 requirement 2 requirement 3 customer n n y developer n n n user n n y community n y y additional stakeholders... this process can be used to both identify additional stakeholders and to determine their rights the first phase of the stakeholder identification should have identified some areas of broader ethical concern and some additional 160 stakeholders. the primary stakeholder analysis is repeated for these newly identified stakeholders. even if there were no new stakeholders identified, at a minimum the analysis should include software users, related cultural groups, and society as potential stakeholders. identification of tasks most software project management models proceed by decomposing the project into component tasks called "work breakdown packages" that only address the technical issues. these individual task descriptions are used in the reviewing and monitoring of the project. all of these tasks are ordered in a hierarchy of dependency on one another. each of these individual tasks may have significant ethical impact. the specific sodis is used to help the developer responsibly address the ethically loaded potential of each work breakdown package. this is accomplished by including a sodis analysis in the standard descriptive elements of a work breakdown package (figure 3). ijmsodisproiectaud.tm '£ite;̂ '';i«*:'itofpit--h ĵ. '.^7'. l:'?f:""': '^v •>!>))1 ; "1; f i* } i $ ) .1 ) f h j! li i] t 1 kthjcft̂ jtstakiî jcfe f̂ekjbn is v^summarylf wbpdetalj) sodis arab's jana|isis overview j angsts octal] ••^;"^^"-'••:•.•'••»,.-..'.--,•.•••:••'•*" ', • -f>-fr» • . • • . -«• ' • •£ , " . • l ~~ & cyclone launch s-corpcomm -corpcommkickoff •comm. plan delivered -packaging datasheets reseller kits competitive comparison demo script •working model q advertising d public relations : -working model b beta test i r-^^^^^^^5§] ;: mail beta copies !-• provide technical support ; -•dose beta i elinternal communications j sihardware relationships \ hprograms j release to manufacturing 1 -manufacture product i lproject announced ] i identification number: name: completion status: assignment status: description: product category: predecessors): successor(s): duration: hours of effort: cost estimate: estimated resources: location: priority: completion criteria: acceptance criteria: risk analysis/resolution: sodis analysis/resolution: package manager: organizational structure: notes: activity codes/flags: maintenance concerns: 23 . ; v : •• •-. ' • develqpjetalis! ' .1; • give to staff 5 june i i need j list of 20 organizations to do testing i edit 1 edit ji view j1 view 16800 i start date: jw1 5/1 993 8:00:., if 0 end date: 1^4/19985:00:0 1 0 1 ll view| | 4 "• ' " • • ' . , • ' • ,;' j review list for knowledgable testers |, edit j ' written approvl by customer of beta testers jj edit | jj"""v ~ " i new concept, few contacts in this area j[anak>ze 1 i not started jj analyze | j i ] | ) ^..__.^..___.___..____.^_ ^^^ , update list consistent with technology il..-f!jla;j r _ '" " ""* i ,. ^=^^^^^^^ ,',7'v . ' . ' • • • ' : figure 3 wbp detau screen; the sodis analysis process also facilitates the identification of new tasks or modifications to existing tasks that can be used as a means to mediate or avoid identified concerns. the identified tasks need to be incorporated into the software project management plan. the early identification of these software modifications saves the developer time and money and leads to a more coherent and ethically sensitive software product. this phase of the sodis process is a pre-audit of a detailed project plan that is developed late in a software development life cycle. 161 identify potential ethical issues this stakeholder identification process has been modified in the sodis project auditor. gert's ethical principles have been combined with ethical imperatives from several computing codes of ethics to reflect the professional positive responsibility of software developers. these principles have been framed as a set of 32 questions related to stakeholders in a software project, and to generalized responsibility as a software professional. these questions are placed in the bottom frame of the sodis analysis screen (figure 4). there may be some special circumstances that are not covered by these 32 questions so the system enables the sodis analyst to add questions to the analysis list. when the analysis is complete there are several usage statistics reports that give various snapshots of the major ethical issues with the project. |̂̂ ^ .̂e»jaijf sodis analysis j a^sj£0 îwj_an^y«i$ de wbps/tasks mighlthbtask rowstakeholdw b cyclone launch t$! corpcomm ^•advertising bpublic relations working model bbetatest ^^^ deyetop beta ost \fj^ ma8 beta copies i i •• provide technical support • -dose beta $• internal communications eghardware relationships [pprograms release to manufacturing | manufacture product ; project announced i cause ham to ji^li} customer lo': of property. propcq cause the unwanted modification or destruction:̂ :;; files and programs owned or in use by the ;.'-'•* stakehojder ,". cause the unnecessary expenditure of the •;?«<' resources of the stakeholder involve the design or approval of software whrcl;$' may lower the quality of life of the stakeholder fail to take into consideration the needs of the stakeholder discriminate against the stakeholder the reading audience the confetence omantzeis '(might develop beta list' fiupscate cause loss of information, loss of property, property damage, or environmental impacts that affect the stakeholder the audience today? guidance yes 'o no' . ji • :noforaii i -stakeholders figure 4 sodis analysis screen; when an ethical concern has been identified, the analyst gets an ethical concern form (figure 5) that asks the analyst to record their concern with the task and record a potential solution. the most critical part of this process is on this form, where the analyst is asked to assess the significance of their concern with the work breakdown package being analyzed. if the problem is significant then they have to determine whether the problem requires a modification of the task, deletion of the task from the project, or the addition of a task to overcome the anticipated problem. it is these adjustments to the software requirements or management project plan that complete risk analysis. 162 lymsoois project auditor f8e e&t toojs report' help ! inslwdions ji sta^ghddei iclentificatkin j ,1 . [ wbpsttasks *"" sodis analysis might this task bofe/siakehouei bcyclone launch corp comm osadvertising eb public relations working model b beta test mail beta copies provide technicals close beta ,' ej3internal communicator i bhardware relationship |j-programs « release to manufactuiji manufacture product project announced | ethical concern1 vii' it he r eading audience 'fft i the conference organizers involve the design or approval of software whicr may lower the quality of life of the stakeholder ^ .< ** ^ once a concern has been added,it cannot be edited please enter a brief description of your concern: testing must be finished before the presentation otherwise the audience !̂ will have wasted their time r urgent ' r.confidenual critical jdrticat save i'! save & enter solution i cancel ' stakeholder̂ not relevant to this might develop beta list' cause the unnecessary expenditure of the resources of >! concerns/solutions. the audience today? yes no s" -'jiiofoi~m~ ff o i "stakehokters figure 5 concern /solution screen; gotterbarnfig5.jpg the process of developing a sodis requires the consideration of ethical development and the ethical impacts of a product the ethical dimensions of software development. the sodis analysis process also facilitates the identification of new requirements or work breakdown packages that can be used as a means to address the ethical issues. the identified work breakdown packages need to be incorporated into the software project management plan. the early identification of these software modifications saves the developer time and money, and leads to a more coherent and ethically sensitive software product. the sodis analysis process also facilitates the identification of new tasks or modifications to existing tasks that can be used as a means to mediate or avoid identified concerns. the identified tasks need to be incorporated into the software project management plan. the early identification of these software modifications saves the developer time and money and leads to a more coherent and ethically sensitive software product. this phase of the sodis process is a pre-audit of a detailed project plan that is developed late in a sdlc. the sodis process also includes a consideration of other phases of an sdlc. some risks can be identified when a project is first conceived or can be identified at an intermediate stage when the customer's desires are being specified in the requirements phase. the sodis project auditor also provides a pre-audit for these two project phases. a complete sodis process 1) broadens the types of risks considered in software development by 2) more accurately identifying relevant project stakeholders. the utilization of the sodis process will reduce the probability of the types of errors identified by farbey. the sodis should be part of a sdlc. integration of the sodis and an sdlc barry boehm's modified spiral model the win-win model (ww) comes close to meeting 1) the stakeholder identification problem and 2) the risk limitation problem. his win-win spiral software development technique is used to elicit project requirements for all stakeholders. at each phase of a project's development the analyst 163 identifies the stakeholders for that stage, determines the win conditions for each new stakeholder, and then negotiates to have these new win condition requirements fit into a set of win-win conditions that have already been established for all concerned. there is a set of win conditions for the aegis radar customer. these conditions would be identified and a process developed to meet those conditions. then new stakeholders would be identified, for example the sailor's using the system on the vincennes, and their win conditions would be identified. they would consider it important to be able to clearly determine if an approaching aircraft were hostile. this win-condition would be incorporated, via negotiation, into the existing process plan. although this approach is similar to rawl's wide reflective equilibrium in deriving a coherent set of requirements through negotiation, the ethical element is missing from boehm's method. there is no methodology to identify ethically relevant stakeholders nor is there an ethical foundation for the negotiation process. the method is also limited in that it assumes all stakeholders are equal and that they will equally be aware of and able to describe their own win conditions. the negotiation amongst stakeholders will be unjust and will likely lead to failed systems, unless, contrary to fact, each stakeholder has such an equal identification and descriptive skill of their own win conditions. there is also an implicit assumption that all requirements are negotiable. as the method is constructed, all requirements have equal status—none are rejected because they are morally impermissible or required because they are morally mandatory. this model has two strengths. first it comes closer to properly expanding stakeholder identification than other software engineering methodologies. unlike all of the other approaches that presume the impact analysis is done as a single process, the win-win model is iterative requiring a re-identification of system stakeholders at each stage of the development process. the negotiation activities include: • identification of system stakeholdersdefined as "anyone that has a direct business interest in the system or product o be built and will be rewarded for a successful outcome of criticized if the effort fails" • determinations of the stakeholder's win conditions. • negotiation of the stakeholder's win conditions to reconcile them into a set of win-win conditions for all concerned. although risks and stakeholders are incorporated into the model, negotiations activity 1 does not identify a broad base of stakeholders. the sailors using the aegis radar system and the passenger who lost their lives would not be stakeholders. 2. rdpntify stakeholders van rr>nrlirkin? 5 deiir>e r»e»l level oi product and process • including partitions . ^ , 3 reconcile*m slahenolders ^^/""^.l / conditions establish "9*1 level directives, constraints, alternatives. 7 hn\iew. _ 4. evaluate product ana 6. validate product .s^-—^^ ^* process alternatives ano process oetimtions [boehm, 1998] the types of risks are also limited. the project risks include: difficulties associated with budget, schedule, personnel (staffing and organization), resources, and customer. technical risks include technical uncertainty, technical obsolescence, and difficulties in interfacing, maintenance, design and implementation. the business risks include: market risk (the product no one wants), product line skew (product does not match product line), and management risk (change in physical management or focus). these risks do not include risks to the users of the aegis radar system. in a further modification to his model, boehm adds three process milestones to help mark the completion of one spiral. these three "anchor points" life cycle objectives, life cycle architecture, and initial operational capacity provide technical information used to make decisions about whether and how the project might proceed. 164 the win-win model has the ability to analyze risk and incorporate stakeholders. the inclusion of a sodis analysis as a starting point for the project and as the fourth anchor point then the win-win model would be an effective sdlc which addressed the technical and ethical risks of software development. the sodis process facilitates the expansion of software risk analysis to reduce software failures. using this preaudit process in test in the uk and the usa facilitated the early identification of project risks. using a sodis process will make producing software of high quality and producing software that is ethically sensitive second nature to the software engineer. this research was partially funded by nsf grant 9874684 references: boehm, b.,"using the winwin spiral model: a case study," computer july 1998. buglione, l and abran, a (1999) lime: a three-dimensional measurement model for life cycle project management" proceedings of the international workshop on software measurement september 1999. farbey b, land f and targett d (1993) how to assess your it investment, butterworth heinemann. gert b (1988) morality, oxford university press. gotterbarn d (1991) computer ethics: responsibility regained, national forum, the phi kappa phi journal, vol71no3. gotterbarn d (1999) "promoting ethical responsibility in software development," proceeding of the aice computer ethics conference gotterbarn d and miller k and rogerson s (1999) software engineering code of ethics, communications of the acm. 1998 http://computer.org/computer/code-of-ethics.pdf green r m (1994) the ethical manager, macmillan publishing. mccarthy j (1996) dynamics of software development, microsoft press. rogerson s and gotterbarn d "the ethics of software project management", in ethics and information technology, ed. goran collste, new academic publisher, delhi, 1998 165 meeting ethical demands in a multi-agency project joe thomas & andy bissett school of computing & management sciences, sheffield hallam university, england si 1wb abstract various health service and voluntary sector agencies dealing with drug misusers in a large city in the north of england have set up a task group to co-ordinate their assessment and referral process. an it solution to their needs was believed to be desirable but raises serious issues concerning confidentiality, and the suitability of an it solution to the particular information problem. the approach is informed by social informatics and utilises soft systems methodology within the framework of joint application development. introduction this paper describes a piece of action research undertaken as an msc dissertation project between june and september 2000 [thomas, 2000]. the project involved the provision of an it solution for a task group of health service and voluntary sector agencies involved in providing services to drug misusers in a large city in the north of england. the task group was charged with co-ordinating the various agencies assessment and referral process. this was considered to be desirable as many of the clients of these services are involved with several agencies, either simultaneously or serially, and each agency assessed their clients separately, and may then refer them on to other agencies to be assessed again. the nature of this process was found to be repetitive and contributed to potential clients failing to seek help from the agencies (there is some anecdotal evidence that a small minority of drug misusers upon being convicted of an offence would rather accept a custodial sentence than a suspended sentence on condition that they use the agencies' services). although each agency has a different role (counselling, medical care, residential rehabilitation, advisory etc.) the core of the information they collect is essentially similar. the task group was set up to work towards a common assessment tool to allow information to be gathered at the clients entry point to the services, and to be passed on and added to as they progressed through whichever agencies were appropriate for them. in addition to streamlining the process to provide a better service to the drug misusers, the task group wished to co-ordinate the process in order to allow a centralised collection of information in order to produce accurate statistics regarding the nature and size of their client base to assist with planning and resourcing the services more efficiently. however the need to produce summary information could not be allowed to compromise client confidentiality, so, although the records must be identifiable (to prevent double-counting) they could not identify individuals. the ethical dimension of this problem arises mainly around issues of confidentiality, as each agency has its own agreements and understandings with its clients. often clients divulge information to one agency that they do not wish to be passed on to any other agencies they become involved with, even if they are referred. legislation such as the data protection act had proven to be a stumbling block for similar projects undertaken by dats (drug action team a local co-ordinating body) in other parts of the country, and regulations about the gathering and confidentiality of medical records created a divide between health service and voluntary sector agencies. in addition to this the clients' own agendas and permissions regarding the sharing of information needed to be addressed. also in need of consideration was how the assessment and referral process would integrate with their existing administrative duties. the health service agencies have particular difficulties in this area as they already have statutory paperwork requirements. in addition they do not collect all the information at the same time and it is performed under a number of different roles. the example given was that the reception staff would collect personal details like name, address, date of birth etc., and a nurse would complete separate paperwork for risk assessment, substance misuse history and so forth. the approach adopted by the researcher was informed by the field of social informatics [kling, 1996; 1998; 2000] and used checkland's [checkland, 1981; checkland & scholes, 1990; checkland & holwell 1998] soft system methodology as an analytical tool, and joint application development [wood & silver, 1995] as a framework to structure contact with the agencies. this paper will consider the project as a piece of action research, before discussing the epistemology of systemic thinking and social informatics. there follows a discussion of the methodology applied and the successes and failures of this methodology in an ethically charged and complex problem area. 183 the project as action research action research the research project was essentially a piece of action research, as defined by checkland and holwell [checkland & holwell, 1998]. the epistemological basis on which checkland and holwell build their definition of action research is defined as interpretivist, as opposed to positivist, or functionalist. their account of action research requires the researcher to adopt this interpretivist approach, and accepts the common notion that action research will entail the researcher becoming actively involved with the problem situation and more importantly, the people in the situation. emphasis is placed on understanding the social practice of the people involved, and the role that the problem situation plays for them, how it is integrated with their other processes and practices. action research was selected as the mode of enquiry for the project as it provides useful concepts for understanding of the complex social situation existing between the members of the task group. the interpretivist approach offered by action research represents a less problematic understanding of the problem domain than a positivist approach could. the focus of the project is the discovery of the requirements for a proposed system, and an analysis of the feasibility of implementing these requirements, not the technical problem of designing and implementing the system, although obviously the two are not easily separable. the building of a prototype, for example, by necessity addresses design issues, but its primary function and importance is as a confirmation that the requirements are being addressed. epistemology systemic thinking and social informatics soft systems methodology (ssm) was selected to provide an appreciation of the 'culture' of the group and its member organisations, as soft systems thinking is considered to be particularly effective in analysing complex organisational problems. social informatics was selected in combination with systemic thinking to provide a strong scepticism regarding the assumptions the group had already made, that the answer to their information systems problem was an it solution. as kling states [1999] of the critical orientation of social informatics, "the critical orientation refers to examining information technologies from perspectives that do not automatically and 'uncritically' adopt the goals and beliefs of the groups that commission, design, or implement specific information technologies." social informatics also incorporates concepts that are familiar from the discussion of action research and ssm. the holistic approach of social informatics seems to fit comfortably with the action research (and ssm) ethic of considering the social and organisational factors surrounding an information system problem in as great a depth as any proposed information technology solution. methodology: a combination of jad and ssm joint application development and soft systems methodology the research was conducted along the lines of an informal joint application development (jad) process,[wood and silver, 1995]. the jad process was adopted for a number of reasons, the foremost being the assembly of the task group, in effect this group was the jad group as it consisted of managers from all the relevant bodies. the authority existed within the group representatives to instigate and implement any findings from the project research, especially with the involvement of the area co-ordinator. the application of this process was somewhat problematic. the literature refers to the jad process as involving departments within a single organisation, and although the group is relatively cohesive; the fact that they are separate, or disparate, entities can lead to a lack of unified direction. however, the group does have a common interest: the best possible provision of services to drug users in the area, although they may not always agree on what this means, and all will naturally focus on the services their own agencies provides. the researcher filled the role of scribe and facilitator. the major issue with this selection is one of authority and the groups perception of the researchers role it became apparent there were different perceptions of the role the author should play. the group felt they required a research student to act as an it consultant, and to take their requirements away and produce a database. the action research and epistemological approach selected challenged this definition, and required a deeper investigation into their requirements and culture. as a research student (and lacking an executive sponsor to appeal to for authority), the facilitator did not have the authority to lead the group, in the way wood and silver would perhaps advise. however, the group are quite informal and friendly, and were keen to receive advice, there was no point at which major issues of authority arose. regardless of positive experiences in this study it remains a point worthy of consideration in future research. 184 the methodology in action an issue was raised about the paper records kept by most of the agencies. it was felt that a system that required a worker to check computer records and paper records would actually be more time consuming and less efficient than the existing practice of simply locating a client file in a filing cabinet. in addition, the usefulness of the paper form that had been prepared as a template for a database was not as immediately clear to the author as it appeared to be to the task group. the form contains several very long text fields in the later pages, and these was causing concern amongst the workers that it could be labour intensive, especially for those with slow typing speeds. not all the agencies have administrative staff to complete these duties, and those that do would still require a worker to complete a paper form to pass on for data entry. long text fields would also be of very limited use in producing statistics and searches for possible entries would need to be unpractically exhaustive. the issues of administration time and cost were also raised. the interviews conducted with the agencies had exposed differences in the methods of information gathering. doctors at the primary care clinic, already used a specially designed software package, and input the information interactively with his clients, during consultations. other agencies interviewed their clients in meeting rooms or in the clients' homes, in areas without access to computers. there was a general consensus amongst the voluntary sector agencies that the use of computers (desktop or laptop) in meetings could be distracting or intrusive, and in some cases may represent a security risk, in addition to representing a large additional budgetary consideration. the representative from the primary care clinic's suggestion that the other agencies adopt the health service practice of using pcs interactively with their clients was not considered to be wholly practical for the voluntary sector agencies. a local drugs project manager, in an interview with the author, had suggested that the practice was more viable for health service agencies due to the difference in the clients' expectations of how health service and voluntary sector agencies' records would be kept. clients would expect medical records to be scrupulously recorded and available across the health service, but would view the same practice with extreme suspicion if conducted by counselling and residential care agencies, where care is based more heavily on personal relationships and trust. perceptions (by the clients) of how the information would be used could also be problematic, as the recording of information electronically could create uncertainty as to the scope of access to it. assurances that access to a stand-alone system was purely within the agency that recorded it, and that information would only be transferred at referral with the client's consent was not, in his opinion, necessarily sufficient to allay this concern. there are obviously ethical issues involved here. if the voluntary service agencies were unwilling to complete computer records in the presence of the client (because this could undermine the trust that the information is confidential), is it ethical for them to enter the records when the client is not present, and perhaps not aware? the use of ssm to analyse the problem situation requires a root definition drawn from the catwoe mnemonic, which draws attention to the requirements in a slightly different manner, in the below example. c customers: the agencies involved in providing services to drug users in the local area, and indirectly their clients a actors: the research student and the agencies represented in the task group t transformation: information about clients recorded in varying formats but not shared or analysed -» information about clients recorded in a unified format, shared where appropriate and analysed at an agency and regional level w weltanschauung: social informatics used to address systemic thinking. sharing and analysing information will result in a better provision of services. o owners: the agencies involved in the task group and agencies not involved in the task group but involved in providing services to drug users e environmental client confidentiality, budget and time available for constraints the msc project, expertise of the research student, the existing it in place at the agencies, the agencies working practices, the attitudes of the clients from this mnemonic a root definition can be drawn as follows: "a system to record, share and analyse client information presented in a uniform format across the agencies that address issues of drug misuse in the local area in order to provide a better service to the clients." the mnemonic also draws attention to a previously un-stated assumption of the project: that the recording, sharing and analysis of client records will enable the agencies to provide a 'better' service. the assumption seems sound based on the information provided by the group, but is extremely subjective. it is not difficult to imagine that some clients enjoy the anonymity that a system that does not share information provides, as this enables them to represent themselves as they wish each time they come into contact with a new service. for clients who 185 wish to misrepresent themselves, perhaps to manipulate the services in their favour, sharing of information is not beneficial at all, from their perspective. issues of trust have been raised previously, and the possibility that clients may perceive wider access to their personal details as intrusive, even under the controlled circumstances the system would allow, is very real, and could form a serious barrier to the personal bond between worker and client. an example of one of the ssm rich pictures prepared during the course of the project appears below. complex organisational rich picture: requirements & issues "we need to formulate a city action plan, we need figures, statistics etc" "we have our own paperwork we still need to re-assess clients" black drugs project (absent "what about process for young people" young peoples drug project "is there the make it work?" "how much more work will it create?" , t local drugs project *^ church organised drugs project "can we trust the quality of voluntary sector assessments for medical purposes, we can't accept referrals from all agencies" primary care clinic "we need a *r networked system" operate with each other to provide the best possible service drugs and alcohol project the task group, suffering from 'project fatigue' v "we need statistics etc to apply for funding" "we need to track situations where children are at risk" i / child protection unit \ gp representative "we need to work with the voluntary sector to provide a complete package of services" will it create more work?" service agencies 186 the research stage yielded a great deal of insight into the very different contexts in which a system would be used. most of the voluntary sector agencies perceive their work as meeting clients and forming a personal relationship (this is an over-simplification, intended to serve as an illustration of the culture) with the record keeping serving as a subsidiary function. the health service by practice, tradition and statute requires scrupulous and comprehensive records without which it cannot function. whilst the practices of the two are becoming more closely intertwined this is still a significant distinction. the voluntary sector agencies tend towards a higher number of workers per pc, and are not in a position to consider it central to the services they provide. this is especially significant when considering the theories behind social informatics, as this environment (or social context) is far less conducive to efficient use of an it system. the session itself, phase 4 of the jad process, took place at the task group meeting 26 my 2000. the key elements of the session were adhered to: • a discussion of the assumptions that had been made by the various agencies, and how these differed from each other, e.g. the difference in practice between health service and voluntary sector agencies in use of pcs • a discussion of precisely what data was required. interviews had led to the proposal not all of the data from the assessment form be included, but that whatever data was entered would need to be updated and stored, a recording of assessment information only had not been considered viable by interviewees or by the author. • a discussion of the "business processes'. this overlapped heavily with the discussion of data requirements. the research had attempted to place the process the group wanted to automate (record keeping for assessments) in the context of the wider processes (record keeping about the client) the session yielded fruitful discussion, but also caused concern that the group had agreed to all the general proposals made by the researcher. as they were receiving 'free' advice a possible bias towards acceptance of the proposals, as this would result in further possible benefits without costs, may have been present, and prevented free expression of further, or more complex requirements. as a mitigating factor, the viability of a networked database (which had been the groups original aim) had been dismissed due to the project timeframe and budget, and it was proposed that an access database be implemented, either for use in maintaining and sharing records or to demonstrate the requirements a networked database would have. the group made provisional agreements (that the prototype should be developed) without committing themselves to using it. one of the agencies involved in the task group is currently piloting a 'beta version' of the database, with a view to rolling out the application to the other agencies once any outstanding issues have been identified and resolved. technologically, the application falls deliberately short of the distributed database the group initially believed they required; this compromise allows them to meet their confidentiality requirements whilst implementing a relatively low cost system. conclusion as is a common problem with action research, the interpretivist approach adopted makes it difficult to gauge what conclusions can be drawn that are not inextricably linked with the context of the particular area of study. it seems clear that checkland and holwelps prescribed 'immersion' in the problem area can be a poor definition of the boundaries of a project and 'scope creep' can easily ensue. however, given that software development is an area in which project failures or high cost maintenance problems are often attributed to an inadequate analysis of what is required at the initial stages, an approach which attempts to rigorously scrutinise the users stated needs and the environment in which the system will be used seems appropriate. it appears that the deviation from the jad process to concentrate more heavily on gathering and clarifying requirements and generating options using ssm resulted in a richer understanding of a complex and ethically charged situation, providing a basis for a system that took a more balanced consideration of the needs of its various stakeholders than an early acceptance of the group's stated requirements. taking ethical needs into account may result in a proposed solution that is not so technically optimal as the technologically driven discourse of it systems development might suggest. references checkland, p (1981) systems thinking, systems practice john wiley & sons ltd checkland, p & holwell, s (1998) information, systems and information systems john wiley & sons ltd checkland, p & scholes, j (1990) soft systems methodology in action john wiley & sons ltd kling, r (1999) "what is social informatics and why does it matter?" d-lib magazine january 1999, 5 (1) accessed at http://www.dlib.org/dlib/january99/kling/01kling.html, 27 june 2000 187 kling, r, crawford, h, rosenbaum, h, sawyer, s & weisband, s (2000)learning from social informatics: information and communication technologies in human contexts (draft) centre for social informatics, university of indiana accessed at http://www/slis.indiana.edu/kling thomas, j (2000) feasibility study for a multi-agency client assessment and referral database, sheffield hallam university wood, j & silver, d (1995) joint application development (2nd edition), john wiley & sons inc 188 cyclomatic complexity: theme and variations brian henderson-sellers, yagna raj pant, june m. vemer school of information systems, university of new south wales kensington, australia abstract focussing on the "mccabe family" of measures for the decision/logic structure of a program, leads to an evaluation of extensions to modularization, nesting and, potentially, to object-oriented program structures. a comparison of rated, operating and essential complexities of programs suggests two new metrics: 'inessential complexity" as a measure of unstructuredness and "product complexity" as a potential objective measure of structural complexity. finally, nesting and abstraction levels are considered, especially as to how metrics from the "mccabe family" might be applied in an object-oriented systems development environment. introduction though software complexity research began around the 1970s, a significant number of problems related to mastering complexity and significantly boosting programming productivity remain to be solved. indeed, failure to master complexity is the cause of many of present problems in software development. these problems are manifested by a high demand for human resources (a precious commodity), product deficiencies, late delivery, etc. traditional complexity measures like those of mccabe (1976), whilst not incorporating the human side of software development and maintenance, do provide some measure of the logical structure of a program. such logic structure measures, together with measures of data structures (e.g. number of variables), size measures (e.g. lines of code, total number of tokens), style measures and internal cohesion measures offer the capability of describing the intra-module or procedural complexity as one type of product metric. (for present purposes, the words measure and metric are used interchangeably. for more precise definitions, see melton et al. 1990). a second type of module metric is semantic cohesion which is strongly influenced by human cognition and knowledge descriptors. system-level measures are a third type of structural complexity measure which describe inter-module coupling (figure 1). in this paper, we focus on the "mccabe family" of measures for the decision/logic structure of a program, which are those encapsulating the cyclomatic complexity (e.g. myers, 1977; piwowarski, 1982; sagri, 1989) including extensions to describe modularization (henderson-sellers and tegarden, 1993), nesting (piwowarski, 1982) and object-oriented program structures although we focus here on the simplest case of single-entry, single-exit modules. (extensions to multi-entry/multi-exit modules are described in henderson-sellers and tegarden, 1993). in the following section we review two definitions of software complexity. rated, operating and essential complexities of programs as defined by mccabe (1976) and sagri (1989) are discussed together with the introduction of two new metrics: inessential complexity and product complexity. in the final section we consider the application of these measures to modularization and nesting and some of these ideas are then discussed from an object-oriented viewpoint. complexity there have been many attempts to define software complexity (e.g. curtis, 1979; basili, 1980). such definitions generally identify three important elements: descriptions of tasks/problem space, resources and interacting systems. the description of complexity depends on the nature of the interacting systems which in turn is related to the resources that have to be expended in this process. the interacting systems, for example, can either be computers or programmers, or other software. when the interacting system is a computer, parameters like cpu time, the number of decisions that are to be evaluated, the number of disk accesses for performing a given task and the memory required to perform the computation all become important. these are the accidental difficulties of software that today attend its production but are not inherent in the software itself (brooks, 1987). they may be 24 ajis termed "computational complexity" (e.g. zuse, 1991 and figure 1). on the other hand, when a programmer is interacting with software, the ease of performing tasks such as coding, debugging, testing, or modifying the software are of prime consideration. these tasks aim to address some aspects of the complexities of software a construct of interlocking concepts and many competing and often contradictory requirements (brooks, 1987) including both people and artefacts (programs). this is the area of "psychological complexity" (e.g. zuse, 1991) (figure 1). except for problems related to temporal logic, management of psychological complexity (understandability, readability, maintainability, etc.) can be extremely difficult, encompassing, as it does, problem characteristics, programmer characteristics and logical and syntactic structure within and between modules (figure 1). complexity computational complexity psychological complexity -l problem (functional) complexity programmer characteristic a/d/c jlmay be v_^influenced by module metrics d/c product/document compl !-lexity jral mplexity d/c = structural co: ' intermodule metrics (system design) e module metrics procedural module metrics semantic semantic cohesion d uenced by coupling metrics d a analysis d design c code —__ is-a-kind-of relationship • is-composed-of relationship figure 1 with the enormous gains in performance : price ratio in hardware technology in the last thirty years, execution efficiency is not normally a major issue over the entire life cycle when compared with the difficulties posed by specification, design and testing while developing and maintaining software (brooks, 1987). melton et al. (1990) caution against equating structural complexity with the difficulty september 1993 25 of understanding a piece of software. one may informally relate structural complexity with "synthesis': the creation of design and code; whilst psychological complexity relates to v analysis': dissection of existing code in order to comprehend as a prelude to maintenance. whilst psychological complexity describes the difficult area of human-software interaction, one of its components is more easily discussed objectively: the inherent complexity of the software product (or "structural complexity" — figure 1). here we concentrate solely on some of the available product metrics and suggest some extensions; focussing primarily on procedural complexity mainly at the intramodule level. procedural complexity measures mccabe (1976) deduced from graph theory that, for a directed acyclic graph (dag) extracted from the code, the cyclomatic complexity, v(g), could provide a measure of the decision structure and hence assist in evaluating the testing/validation effort required. despite the unjustified extrapolation by some authors of v(g) to a prognostic tool for reliability and effort (cf. shepperd, 1988), the mccabe cyclomatic complexity measure remains useful as a quantification of the complexity of testing paths and may provide some insights into cognitive complexity — although such a link has not yet been satisfactorily established, most propositions being purely speculative, not scientific (baker et al., 1990). v(g) is defined for a single dag by v(g) = e n + 2 (1) where e is the number of edges in the graph and n is the number of nodes. for p disjoint graphs, mccabe suggests v(g) = e n + 2p (2) whilst henderson-sellers (1992c) and henderson-sellers and tegarden (1993) recommend vli(g) = e-n + p + l (3) this latter formula measures the total number of possible decisions (= vlj(g) 1) i.e. the number of simple if statements in the whole program compound boolean expressions with m predicates are weighted as m decisions (mccabe, 1976). the original use intended by mccabe was to take a single flowgraph representing a code module for which the value of v(g) could be calculated. as a consequence, the basis for testing paths could be evaluated. a second use is to apply this approach to the whole program (using either equation (2) or (3)) to get a value of v(g) or vlj(g) respectively for the whole program and then use this to suggest as an heuristic to divide the program into subroutines following the classic decomposition principles as propounded by parnas (1972). however, essentially as a result of asymmetric nestings on these decision branches, the minimum number of decisions taken at runtime may be less than the value suggested from consideration of v(g). sagri (1989) introduces this notion as the "operating complexity", v0, calling mccabe's original number the "rated complexity", vr. these two numbers represent the minimum and maximum number of possible individual decisions. he argues that the interval (v0, vr) provides a more useful assessment of program complexity. however, no account is taken of the distribution of values (i.e. can all values in the interval be realised), a characteristic which might also be of interest (monarchi, 1992, p.c.). the value of vq is a measure probably more related to computational complexity than to structural complexity (figure 1), suggesting that the interval may not be as useful a measure of structural complexity as originally envisaged. furthermore, it should be stressed that the number of decisions discussed here is not the same as the number of paths through the module, as might be required for constructing testing procedures (see also nejmeh, 1988). this value may be as high as 2n where the program contains n binary decisions. sagri (1989) analysed five graphs gl~g5 (figure 2) constructed from different arrangements of three if-else statements (two further possible arrangements of three if-else statements are included as graphs 26 ajis g6 and §7) and found the minimum and maximum number of possible decisions that have to be made in each graph (table 1) to arrive at the following conclusions: 1. during execution, since the number of decisions that have to be made in graphs g2 and g4 is smaller than in graph gl, it is concluded that the nested and concatenated ifelses (graphs g2, g4) are less complex than un-nested if-elses (graph gl). the complexity ordering of these graphs is g4 < g2 < g3, g5 < gl. 2. the distinction between rated and operating complexity helps in comparing two alternative programs with the same cyclomatic number. graph el g2 g3 g4 k5 g6 g7 no. of run-time decisions min+1 4 2 3 3 3 2 2 max+1 4 4 4 3 4 4 4 v(g) 4 4 4 4 4 4 4 table 1: comparison of v(g) with minimum and maximum decisions taken during execution table 1 also suggests that the concepts of rated and operating complexity cannot distinguish graph g3 from g5 or graph g2 from g6 or g7. sagri (1989) specially noted that in graph gl, it is necessary to evaluate 3 decisions during run time irrespective of the path taken. in other words, it has a constant complexity in the sense that all runs make the same number of decisions. this gives it the highest overall operating complexity, reflecting the fact that this is a measure of computational resources used on average and has little to do with comprehension since graphs g2 and g3 are unstructured (section 3.1) and graphs g4 and g6 contain nested structures, commonly regarded as adding complexity (magel, 1981; myers, 1977; piwowarski, 1982) (see also discussion in section modularization and nesting). whilst relying on "intuition" to support his case (as do other authors e.g. myers (1977) cf. shepperd (1988)), sagri (1989) fails to comment on three distinct anomalies of the mode of presentation of "three ips" in figure 2. firstly, graphs g2 and g3 do not and cannot represent structured code (see below). secondly, the arguments rely solely on the graphical design resulting from the construction of a dag from 3 decisions. this is contrary to mccabe's presentation of v(g) in which he stresses the use of v(g) in extracting a dag from code not from design. this (mis)application of v(g) to the dag, using it as a surrogate to infer code complexity, was not intended by mccabe (1976). such analyses of dags, made by e.g. piwowarski (1982) and sagri (1989), frequently result in apparently well-designed logical structures which are unimplementable in a structured fashion, requiring highly complex boolean predicates and duplicate code or else the use of unconditional gotos (for example, magel's (1981) example dag number 9). september 1993 27 figure 2 finally, such discussions (as cited above) which select one graph among many indirectly suggest that it is possible to convert one graph to another. this seems highly unlikely since all the graphs presented implement different functions or algorithms and the question of favouring one among the others does not arise. the cyclomatic number of all the graphs is the same because it is a function of only the total number of if-else statements and not of their arrangement. revising the algorithms and logic embodied in one graph (say gl) to a more nested structure akin to g4, g5 or g7, say, is unlikely to be feasible without changing the overall cyclomatic number (see following section). in other words, the logical constructs of the seven presented dags do not map into each other necessarily. this is one of the drawbacks of mccabe's cyclomatic complexity number because design complexities are often dependent on the details of the implementation. in other words, the design use of v(g) may mask 28 ajis unstructured constructs such that the implemented code (when reverse engineered to a dag) has a higher value of v(g). code structuredness it is perhaps worth reintroducing a third structural complexity measure here: that of "essential complexity", ve, as defined in mccabe's original paper. this is the complexity remaining after reduction of the dag by removal of all closed cycles. thus, ve v(g) and should equal unity for a fully structured program. perhaps a better measure is the"inessential complexity", vj, defined as = 1 v. v(g) 1 (4) which has values in the range [0,1], if v(g) is not equal to 1. when ve = 1, the ratio v(g) ve has a value of unity and thus v, = 0. since vj = 0 corresponds to a well-structured program, non-zero values of vj reflect the degree to which a program has unstructured characteristics. sagri's graph gl g2 g3 g4 g5 new g6 new g7 v(g) vp. vi knots 4 1 0 0 4 4 1 8-9* 4 4 1 2 4 1 0 0 4 1 0 0 4 1 0 0 4 1 0 0 table 2: different measures of the graphs gl-g7 (^dependent on coding chosen) out of sagri's five original graphs, two are non-structured by mccabe's definition of containing a branch into a decision (mccabe, 1976, p316) and hence have a non-zero value of vj (equation (4)) see table 2. since the mccabe approach is to extract a dag from code, we must presume that sagri's graph g3 (used as the example here) was extracted from unstructured code containing at least one goto as in figure 3. as noted by sagri, this code (and the dag) has a value of v(g = 4 calculated either 1. directly from equation (1); 2. as the number of decisions (3) plus one (either from code or dag); or 3. as the number of regions (mccabe, 1976; bollobas,1979) = 4 (labelled a, b, c, d in figure 2). if a then 1 _ elseifbthen 2 ~~"~~ gotol3 l3 endif if c then 3 else 4 endif 5 figure 3 september 1993 29 however, what is not stated, perhaps because it is obtainable only from the code and not from the dags, is that this program (figure 3) has a non-zero value for the number of knots1 (table 2) a value long recognized as making programs less maintainable despite the fact that it adds nothing to v(g). in other words, it does not affect the testability of the code. thus sagri appears to confuse, implicitly, the complexity of testing (as indicated by mccabe's v(g)) with the complexity of comprehension (not discussed here). we should also note that the two measures, knots and vj, both measure the extent of unstructuredness; albeit on different scales. graph g3 original i ii iii iv no. of single decisions v(g) knots v(g) + knots 3 4 2 6 7 8 0 8 4 5 0 5 8 9 0 9 4 5 0 5 table 3: complexity values for the five coded versions for the implementation of the graph g3 of figure 3. as a further corollary, it is interesting to reverse the normal direction of mccabe's discussion of v(g) and ask how we might implement, for example, sagri's g3 graph if we were presented with this graph as the design. in addition to the valid (but non-structured) implementation of figure 3, we might well attempt to implement it in fully structured code with no unconditional goto statements. this can be accomplished in many ways (for example, four ways are shown in figure 4). however, all of these must utilize four or more simple predicate decisions (table 3). although all have a knot value of zero, their cyclomatic complexity v(g) is greater than that of the original unstructured code. a combination f v(g) and knots seems to offer a rationalization insofar as codes i and iii have a value of v(g) (=7) identical to the value of [v(g) + knots] for sagri's original graph. whilst acknowledging that the structured/non-structured divide was a more prevalent question at the time of mccabe's original proposition (1976), although still worthy of discussion today (prather, 1984) the discussion here has highlighted: l.the important asymmetry in translating from code to dag compared to implementing a dag in code; 2. the apparent increase in complexity (if measured solely by v(g)) when moving from unstructured to structured code (also noted by myers, 1977); and 3. the need to characterize non-structuredness by measures other than v(g) (see for example the large value of knots for graph g2 table 2). this discussion also lays the groundwork for our discussion of modularization and nesting (following section ) and the role of abstraction, especially in complex structured and object-oriented software systems. modularization and nesting modularization, by removing one or more control structures and replacing them by module "calls", does not, per se, change the value of v(g) of the overall system (as calculated by equation (1)) (henderson-sellers, 1992c). however, it does discretize the program into a larger number of cogniu'vely comprehensible smaller chunks — an idea to be pursued in our discussion of objectoriented systems in the following section. however, if this process also eliminates duplicate code containing at least one control structure, then the structurally-related value of v(g) will generally decrease (shepperd, 1988; henderson-sellers and tegarden, 1993), although the cognitive effort of such a process may be different (a discussion of which is beyond the scope of this present paper). 'the value of knots (woodward et al., 1979) is calculated as shown in figure 3 by drawing lines between if/else/endif structures and loops and also between goto statements and their target 30 ajis i if a then 1 if c then 3 else 4 elseif b then 2 3 elseif c 3 endif if nota and note and notc then 4 endif 5 5 ips equivalent to 7 decisions code block 4 duplicated and block 3 appears 3 times ii if a then 1 if c then 3 else 4 endif elseif b then 2 3 elseif c then 3 else 4 endif 5 4 ips code block appears 3 times and block 4 twice iii if a then 1 elseif b 2 3 endif if nota and b then blank elseif (a and c) or (notb and c) 3 else 4 endif 5 4 ips equivalent to 8 decisions . code block 3 duplicated iv if a then 1 if c then 3 else 4 endif else if b 2 3 elseif c 3 else 4 endif endif endif 5 4 ips code block duplicated and block 3 triplicated figure 4 september 1993 31 figure 5 nested control structure adds to the cyclomatic complexity on the other hand, the addition of a nested control structure adds to the cyclomatic complexity (piwowarski, 1982), whilst leaving the essential complexity value unchanged (figure 5). un-nested ifelses and loops will be simpler to construct for a programmer than nested if-elses/loops, contrary to the conclusions of sagri (1989), because it is easier mentally to handle one independent if-else/[oop at a time. in other words, only one level of abstraction needs to be "remembered" at one time. thus the control structure forms a single, easily understood "chunk" (cant et al., 1992). embedding a second control structure inside this top-level chunk disrupts the flow of cognition by requiring "tracing" — viz. a change of abstraction level — visualized by the landscape diagram of cant et al. (1992). disregarding the computations embedded within the block, it is obvious that one independent if— else/loop commonly requires less effort for cognitive processes like chunking and tracing because of constraints of our short-term memory (cant et al., 1992). on the other hand, nested if-elses also contain implicit ands, and thus create distinct paths (and naturally more side effects), which the programmer has to take into account while writing or modifying programs. similarly, in a doublynested loop structure the programmer/analyst needs to bear in mind the values of 2 control variables chunking and tracing between the two levels of abstraction continually. it is obvious that the logical structure of nested control structures is difficult to assess fully without the inclusion of programmer cognition. in this present paper, we retain a focus on artefacts measurable independently of cognitive complexity issues (cf. cant et al., 1992). in this context, we re-evaluate v(g), vq and ve from the "mccabe family", together with "nesting complexity (n)" designed specifically to address the questions of nesting (magel, 1981; piwowarski, 1982). piworwarski's sequential nested unstructured no. of single decisions v(g)v*(g) operating complexity (v^ essential complexity (vp) n* (equation 6) piwowarksi's n knots pi p9 a b 2 3 3 1 0 3 0 3 3 2 3 3 1 0 3 0 3 3 c d 2 3 2 1 1 4 0 4 4 2 3 3 1 1 4 0 4 4 e f 2 3 2 3 0 5 1 5 9 2 3 3 3 0 5 1 5 9 table 4: values for the 6 programs/graphs of piwowarski. (here v(g)v*(g)). also indicated are values of pi and ?2 given by equations (7) & (8) in table 4 are presented values for these three complexity measures and piwowarski's (1982) n for his six example dags (figure 6). the value of n is given by n = v*(g) + n* (5) 32 ajis where v*(g) is an adjusted cyclomatic complexity number, in which case/switch structures are treated as a single predicate (similar to myers1 (1977) lower bound) and n* represents the effect of nested control structures and is given by n* = i p*(i) (6) where p*(i) is the nesting depth of the ith predicate defined as the number of predicate node scopes overlapped or contained by the ith predicate node. b d 4 figure 6 piwowarski's six examples piwowarski's (1982) intentions are to derive a measure which gives high values for unstructured programs, intermediate values for structured programs with nested control structures and lowest values for sequential code with only top level control structures embedded. as such, it penalizes programs, such as e and f in figure 6, in which knots exist. thus n reflects not only the cyclomatic complexity but also the nesting levels and the degree of unstructuredness. the measure could alternatively be expressed n = v*(g) + n* + 2 k (7) september 1993 33 (where n* is redefined as n* = i p(i) (8) where p(i) is the straightforward nesting depth of the ith predicate and k is the knot value), i.e. the sum of the adjusted cyclomatic complexity, allowing for nesting (each nesting level in each control structure adding one to n), and twice the knot value. the former provides an additional weighting to evaluate nesting complexity whereas the latter addresses the very different issue of unstructuredness (see above). we recommend maintaining these two descriptions of disparate characteristics as separate values: 1. v*(g) modified by true nesting values (p(i) not p*(i)) and 2. knots. sagri's graph gl g2 g3 g4 g5 g6(new) g7(new) v(g)v*(g) v*(g) + (pi) knots piwowarski's n vp vn pi p-7 4 4 0 4 1 0 4 4 4 4 8-9* 20-22 4 1 7 16 4 4 2 8 4 1 7 16 4 6 0 6 1 0 6 6 4 5 0 5 1 0 5 5 4 6 0 6 1 0 6 6 4 7 0 7 1 0 7 7 table 5: values of various complexity metrics for the seven graphs based on sagri's (1989) disussion (*dependent on coding chosen) applying this philosophy to sagri's graphs gives the values shown in table 5. it can be seen that the "least complex" of the three nested graphs (g4~g7) is identified, using n, as g5; that the two unstructured dags have large, non-zero values for knots and a large (fractional — actually unity) value for vj; and that the basic cyclomatic complexity number reflects the number of decisions. as a tentative indication of the modelling approach we intend to pursue, we note that the factors ve, n* (equation (8)) and v(g) all appear to be monotonic increasing functions of "complexity". an appropriate additive model might therefore be = al v(g) + a2 n* + a3 (ve (9) (the subtraction of 1 from ve being necessary since ve = 1 is a well-structured program adding no additional complexity); a multiplicative model might be = a3ve[a1v(g) (10) where the weights, a^, a2> 33 will depend upon cognitive comprehension issues, as well as the task being performed. that this is a reasonable initial step (until dimensionality, meaning and cognitive concerns can be adequately evaluated) is seen in the last two lines of tables 4 and 5 where both "product complexity metrics", pj and p2 are given with the (unjustified) simplistic approach that aj = a2 = 33 = 1. the overall distribution is seen (figure 7) to reflect the broad differences we might anticipate between nested, sequential, structured and unstructured programs. 34 ajis 3 decisions (table 5) piwowarski (table 4) unstructured nested 2 sequential 2 1 1 2 3 4 2 1 2 5 6 2 1 7 unstructured nested sequential 2 1 2 1 2 1 3 4 5 6 7 8 2 9 10 16 figure 7 the role of abstraction abstraction provides a powerful tool in both analysis and design, as reflected in the recent surge of interest in the use of abstract data types (adts) and object-oriented methodologies (hendersonsellers, 1992b; mcgregor and sykes, 1992). the use of abstraction permits the rationalization of complexity by permitting the designer to concentrate sequentially on the design at different abstraction levels. this is closely associated with and in some sense opposite to the process of "prime decomposition" (e.g. fenton, 1991, p!75) and ties in closely with the chunking concepts of the cognitive complexity model (cant et al., 1992). for example, the un-nested if-elses represents a series of simple abstractions, while the outer if-else in a nested if-else (e.g. graph g4 of figure 2) can be considered to be at a higher level of abstraction subsuming, as one of its immediate lower-level abstraction, a single if-else. figure 2 (graph g7) adds one more level of nesting so that this graph can be viewed at any of three abstraction levels. the innermost control block can be considered independently; then at the next higher level of abstraction, this whole block can be viewed as a single node (figure 8); and so on. such a utilization of abstraction levels elaborates on the process, described by mccabe (1976) for reducing a flow graph to evaluate its essential complexity, ve. control bock at lowest abstract ion level figure 8 the innermost control block can be considered independently in object-oriented systems, abstractions are manifest in the form of encapsulated object classes connected through networks. indeed, object modelling focusses on the use of various levels or degrees of abstractions as a crucial tool (henderson-sellers, 1992a; selic et al., 1992). as we move to higher levels of abstraction, we tend to concern ourselves with progressively smaller volumes of information, and fewer overall items. as we move to lower levels of abstraction, we reveal more details, typically encountering more individual items, thus increasing the volume of information with which we must deal. further research on object-oriented structural complexity is clearly warranted, although we note that initial investigation of the usefulness of an unmodified mccabe cyclomatic number (tegarden et al., 1992) suggests that direct application of v(g) can at least highlight the impact of polymorphism september 1993 35 and inheritance although its use for detecting problems in these areas awaits evaluation since v(g) and vjj(g) are best suited to describing control flow structures not architectural configurations. summary and conclusions one of the major problems in complexity research is that of finding an objective assessment of the term complexity. whilst it is increasingly recognized (e.g. fenton, 1991) that the term cannot have a single, all-encompassing interpretation, it may be possible to define the term in very specific domains. for instance, the use of flow graphs discussed here can be evaluated with respect to numbers of decisions and control flow but cannot evaluate complexity due to convoluted variable plans, to linear algorithms nested between the decision nodes of the flowgraph or due to cognitive difficulties. we can conclude, therefore, that flow graph complexity measures, such as those of mccabe (1976), piwowarski (1982), sagri (1989) and henderson-sellers (1992c) can measure decision routes and hence may be applicable to maintenance and testing as originally envisaged by mccabe (1976). mccabe's (1976) original cyclomatic complexity, measuring decisions, supplemented by metrics for nesting and knots, has been rationalized in terms of new "product cyclomatic" metrics, pj and ?2however, the application of cognitive complexity models (cant et al., 1992) may prove useful in this context and is the subject of ongoing research (cant et al., 1992). flowgraph-based complexity measures may also have some validity in design, to the extent and with the caveats indicated above. however, in design, a graph such as g5, g6 or g7 (figure 2) with extensive use of nesting reflects more accurately recent software enginering techniques based on abstraction techniques which are reflected in the increasingly popular object-oriented paradigm. further work on complexity, including acknowledgement of psychological considerations and human factors, is required to clarify the operational role of complexity measures in both a structured and object-oriented software development environment. acknowledgements we wish to acknowledge the support of a small research grant (grant number c450.301) from the australian research council. we also wish to thank graham tate and david monarchi for their useful comments on the draft of this manuscript. references baker, a. l., bieman, j. m., fenton, n., gustafson, d. a., melton, a., whitty, r., (1990), a philosophy for software measurement, j. syst. soft., vol. 12, no. 3, pp. 277-281 basili, v. r., (1980), qualitative software complexity models: a summary, in tutorial models and methods for software management and eng, ieee press bollobas, b., (1979), graph theory: an introductory course, springer-verlag, new york, p. 180 brooks, f. p., (1987), no silver bullet essence and accidents of software engineering, ieee computer, vol. 20, no. 4, pp. 10-19 cant, s. n., jeffery, r. d., henderson-sellers, b., (1992), a conceptual model of cognitive complexity of elements of the programming process, centre for information technology research report no. 57, 1992, university of new south wales, sydney, australia (also submitted for publication) conte, s. d., dunsmore, h. e., and shen, v. y., (1986), software engineering metrics and models benjamin/cummings, menlo park, ca, p. 386 curtis, b., (1979), in search of software complexity, workshop on quantitative software models pp. 95-106 fenton, n. e., (1991), software metrics: a rigorous approach, chapman and hall, 337pp henderson-sellers, b., (1992a), object-oriented information systems: an introductory tutorial, australian computer j., vol. 24, no. 1, pp. 12-24 henderson-sellers, b., (1992b), a book of object-oriented knowledge, prentice hall, sydney, 297pp 36 ajis henderson-sellers, b., (i992c), modularization and mccabe's cyclomatic complexity, commun. acm, vol. 35, no. 12, pp. 17--19 henderson-sellers, b. and tegarden, d. p., (1993), the application of cyclomatic complexity to multiple entry/exit modules (submitted for publication) magel, k., 1981, regular expressions in a program complexity metric, acm sigplan not vol. 16, no. 7, pp. 61-65 mccabe, t. j., (1976), a complexity measure, ieee trans. soft. eng., vol. 2, no. 4, pp. 308-320 mcgregor, j. d. and sykes, d. a., (1992), object-oriented software development: engineering software for reuse van nostrand, reinhold, new york, p. 352 melton, a. c., gustafson, d. a., bieman, j. m., and baker, a. l., (1990), a mathematical perspective for software measures research, software eng. j., vol. 5, pp. 246-254 myers, g. j., (1977), an extension to the cyclomatic measure of program complexity, acm sigplan not vol. 12, no. 10, pp. 61-64 nejmeh, b. a., (1988), npath: a measure of execution path complexity and its applications, commun. acm vol. 31, no. 2, pp. 188-200 pamas, d. l., (1972), on the criteria to be used in decomposing systems into modules, commun. acm vol. 15, no. 12, pp. 1053-1058. piwowarski, p., (1982), a nesting level complexity measure, acm sigplan not., vol. 17, no. 9, pp. 44-50 prather, r. e., (1984), an axiomatic theory of software complexity measure, comp. j., vol. 27, no. 4, pp. 340-347 sagri, m., (1989), rated and operating complexity of program an extension to mccabe's theory of complexity measure, acm sigplan not., vol. 24, no. 8, pp. 8-12 selic, b., gullekson, g., mcgee, j., and engelberg, i., (1992), room: an object-oriented methodology for developing real-time systems, in case'92 fifth inl workshop on computer-aided software eng. montreal, quebec, canada shepperd, m., (1988), a critique of cyclomatic complexity as a software metric, software eng. j vol. 3, pp. 30-36 tegarden, d. p., sheetz, s. d., monarchi, d. e., (1992), effectiveness of traditional software metrics for object-oriented systems, hicss-92, ieee, san diego pp. 359-368 woodward, m., hennell, m., and hedley, d., (1979), a measure of control flow complexity in program text ieee trans. soft. eng., vol. 5, no. 1, pp. 45-50 zuse, h., (1991), software complexity measures and models, de gruyter and co. september 1993 37 ajis vol. 8 no. 1 september 2000 plugging files in database federations uwe hohenstein & andreas ebert siemens ag, corporate technology, zt se 2, d-81730 munchen, germany e-mail: {uwe.hohenstein,andreas.ebert}@mchp.siemens.de abstract in order to use files in database federations, we propose to migrate file data into a relational database. integrating the database in a federation is then well-understood. this procedure has the advantage that a ,/eal" database is handled supporting transactions and sql; files obtain a high-level interface for free. this paper presents a specification-based approach to support the process of migrating file data into a relational database, and vice versa. a specification language provides powerful concepts to describe the contents of files and directories. in contrast to similar work, directory structures are taken into account, because they contain useful semantics. given some descriptive specification of file contents, an adapter is generated moving data from several files into the database, and the other way round. introduction federated dbss provide solutions to handle data stored in several autonomous sources (sheth and larson (1990), conrad et al. (1997), conrad et al. (1999)), giving a user the illusion of a homogeneous ..database system" (dbs). a unified and consistent view of the stored data resolves discrepancies (saltor et al. (1992)) and conflicts (spaccapietra and parent (1994)) between database schemata, which result from representing real world situations in different ways. a transparent access ensures that users are not aware of the location of items in a particular system. with all that, the autonomy of the constituent systems is preserved. research in the field of federated dbss has brought up several prototypes (huck et al. (1994), kuno and rundensteiner (1996), roantree et al. (1999)) and results such as integration methodologies (reddy et al. (1994), schmitt and saake (1996)), languages for view definitions (kaul et al. (1990)), global transaction management and query processing. most effort is dedicated to ..real" database systems; incorporating files is neglected. but there are huge amounts of interesting data kept in ordinary files. for example, engineering applications could not live with the bad performance of relational dbss for handling complex structures. in absence of object-oriented dbss, those applications were forced to keep persistent data in files, and they still do due to the legacy problem. besides legacy applications, files also get new importance in the context of the world-wide web (www): web pages are stored in files, and generated pages can also be seen as files. other examples for maintaining persistent data in files are electronic documents and genome databases. these all are important information sources, and thus candidates for being plugged in database federations. according to the architecture of sheth and larson (1990), any data source in a federation requires an adapter that is responsible for homogenizing the data according to a canonical data model. an adapter maintains a view of the data in the canonical model. moreover, the adapter has to provide a corresponding interface which must be implemented on top of the local source (roantree et al. (1999)). implementing ah adapter for files is very cumbersome because files are somewhat different to databases: they do not possess an explicit schema, and they provide only a simple interface for reading and storing data. nowadays as object-oriented canonical data models are frequently used, e.g., huck et al. (1994), radeke (1995), hohenstein and ebert (1999), roantree et al. (1999), the gap to bridge has become even huger; particularly if the semantic power of the canonical model is used, and if a complete interface such as odmg oql (cattell and barry (1997)) is supported. papers such as hoding (1996) emphasize the necessity to take files into account for federations and analyze the problems that occur. but there are still no adequate solution to ease the integration of files. abiteboul et al. (1993) show how to generate a parser that provides an object view of file data. but this parser must be manually enhanced with semantic actions that collect information in a structured way and build objects. this is a complex task that requires a lot of knowledge in compiler technology, even if compiler-compilers like yacc are used. so far, this is a manual task to be done for each file again. other helpful solutions are coming from the field of semi-structured data. most work here relies on light-weight data models (buneman et al. (1997), nestorov et al. (1997)) that define a merely untyped node structure being less strong than classical models. in order to organize file data according to that data model, files are wrapped. a manual implementation of wrappers is often impractical if the format of sources changes frequently, as it is especially the case for die www. hence, ashish and knoblock (1997) suggest a semiautomatic way to generate wrappers around internet data sources. keywords are taken as tokens of 100 ajis vol. 8 no. 1 september 2000 interest, and the nested document structure is derived from the source. ashish and knoblock (1997) present regular (lex) expressions for detecting usual representations such as headings and emphasizing tags. these are heuristics, whereupon a wrapper is generated. the wrapper then allows users to query web documents in a database-like fashion. however, the overall approach is tailored very much to internet sources. kushmerick et al. (1997) propose wrapper generation, too. but the paper makes more assumptions about the data to be looked for. a powerful approach has also been developed in the tsemmis project (papakonstantinou et al. (1995), hammer et al. (1997)). on the basis of specified templates that model file data in oem (object exchange model), wrappers are generated. they claim that regular expressions are simpler to use for specifying templates. sattler and hoding (1999) propose a similar solution and discuss how to extract relational schemata from web sources. besides being tailored to www sources, all those approaches lack of two important points: • they take into account single files and do not cover the semantics of directory structures. but sometimes, records in several files constitute an object type of interest. moreover, directories are often used to express relationships. • they stress on querying (mostly the web) and are not concerned with updates and transactional support. this limits their usage in database federations. this paper now provides an effective solution to incorporating files in a database federation thereby abolishing the above deficits. we suggest using a relational database as intermediate storage for handling files. this seems to be an unnecessary indirection, but has some obvious advantages: • provided we store file data in a fine-granular manner in tables, the federation can use sql to query and manipulate the data. similarly, the transactions of the database system can be taken. users of the federation are synchronized by the dbs, as the database is just a ,,cache" for file data. certainly, external users of files must be locked until modified data is written back to files. that is, although the locking unit is a file, federation users can work concurrently. using dbs features thus makes the implementation of an adapter much easier. • furthermore, the step of expressing the semantics of file data in a high-level canonical data model can be divided into two simpler steps: (a) giving file data a simple relational structure first, and then (b) enhancing the semantics in a semantic enrichment process (hohenstein and komer (1995)) afterwards, the federation framework of hohenstein and ebert (1999) can take benefit from the higher semantic level. the remainder of this paper discusses both steps (a) and (b) in more detail and presents an adequate support. the next section elaborates on the underlying generative principle, which relies on two corresponding generators: the first one produces a file wrapper that converts file data into relational tables in a structured manner. the wrapper is also responsible for writing data back into files after modifications (abiteboul et al. (1995)). this releases one from the tremendous task of building yacc programs as it is necessary in abiteboul et al. (1993). a second generator produces an odmg adapter for the relational database to be plugged in our federation approach (hohenstein and ebert (1999)). both generators perfectly cooperate and provide together a real object-oriented view of data and a corresponding odmg access to files. however, the generators need some input. this is done by means of logic-based specifications. the third section presents the essentials of a first specification language odlf,ie for specifying file contents. an odlfik specification is input to the generator that produces the migration into the database. several examples will demonstrate the expressiveness of the specification language. relevant file formats are discussed, and it is shown how to handle them. in addition to our previous work in ebert et al. (1999), we introduce some new features to incorporate several files and their directory structure. it is important to keep this kind of semantics. furthermore, we present concepts to handle www sources. in contrast to other proposals, e.g., ashish and knoblock (1997), and hammer et al. (1997), our language is very general and suited for any kind of files. the fourth section presents a second language odlr that enables specifying semantic enrichment of relational databases. we discuss how the language enhances the semantic level of a relational database. the focus lies on concepts that are important for giving tables with file data a further upgrade. in the conclusions, we summarize our ideas and outline some future work we are planning to do. 101 ajis vol. 8 no. 1 september 2000 generative principle figure 1 gives an overview of the principle of having a generator for migrating data from files in a relational database and plugging the database in a federation. there are two generators with corresponding specification languages. a generator genfii,. produces a file adapter, consisting of a parser and an unparser. the generator requires some input given by a specification. a corresponding specification language odlfik has the goal to define a ,,schema" for file contents in a certain sense. the syntax of odlfiie is a mixture of the odmg odl (cattell and barry (1997)) and an enhancement of yacc. 1. a subset of odl is used to describe the schema of data files in an intuitive way. file contents are modelled independently of the physical file structure. a yacc-part then defines a context-free grammar. the grammar precisely describes the contents of files and is the basis for a parser. using a yacc-style eases the later generation of a parser by means of this compiler-compiler. we extended yacc to ease the writing of specifications. for example, the specification language offers repetition groups known from sgml, and concepts to express hyperlinks between files. parsing a file, the collected information has to be organized according to the odl schema. socalled assignment rules now relate the parsed information to objects in the odl schema. it is important to note that objects can be built from several files in several directories. assignment rules handle that, too. 2. /odu7____xge^ /spec/ / odmg oml/oql i 1 h r installs schema /uulpik / \ v /spec / -*3enflb> l r z. /-— yacc program+ semantic actions -f yacc generation access figure 1: generating parser and unparser the odl part (1) of an odlfae specification is used to derive adequate relational tables. it may be surprising why we use the higher odl to format file data in relational tables, and although odl is used, why not the full power but only a subset. on the one hand, we rely on odl because multi-valued attributes are necessary to specify nested structures. using only flat tables, the association between nesting and foreign keys is difficult to express, as we will see later. on the other hand, we use only ..complex objects"; subtypes and explicit relationships between object types are omitted. although useful from the point of modeling file data, we cannot really benefit from the object-oriented schema, because we have to map it afterwards onto flat tables. moreover, subtypes and relationships are difficult to handle because their representations in files are much more manifold than those discussed by castellanos and saltor (1991) and chiang et al. (1994) in the field of semantic enrichment of relational databases. such an enrichment of files should better be done by a successive semantic enrichment step, e.g., using the specification-based approach discussed later in this paper. 102 ajis vol. 8 no. 1 september 2000 the generation of tables is very straightforward. there are no big problems as an odlfue schema does not contain subtypes and relationships. each object type is mapped to a base table, and any multivalued attribute gets an additional table in the sense of normalization. this table takes the values entry by entry. each entry possesses a foreign key to its related base table. the generator uses the grammar (2) of the specification and produces a yacc program that is able to parse files. but a parser is not enough, since it only complains if a file does not satisfy the specified grammar. assignment rules (3) provide some abstract information that allows the generator to enrich a yacc-parser with semantic actions. those semantic actions build objects from parsed file records according to the odl schema. afterwards transferring complex odl objects into the relational database takes place. unfortunately, there is no tool support for building unparsing. consequently, the part of genfik for generating unparsers must be implemented from scratch. aa second generator genr produces an odmg adapter that presents a real object-oriented view of the relational data and provides an oml/oql access. in order to control the building of the view, a semantic enrichment specification in a language odlr is necessary. the odmg adapter can be plugged in a federation as discussed in hohenstein and ebert (1999). the data collected by genfiie and genr is stored in a meta-database similar to (roantree and murphy (1997)). the meta-data consists of any information found in specifications, i.e., odl schemata, the structure of the grammar, the assignments as well as the mapping of an odl schema to a relational schema. this makes the implementation of the generators easier, as producing the yacc program does not need to be done during parsing a specification. specification language odlme this section presents several schematic examples that sketch out in a systematic manner how to handle important file formats by odlpiie. we start with commonly used organizations of objects in a single file and then switch to directory structures, before discussing some special topics such as hyperlinks. handling single files a) table structures in files we start with a single file that contains records in tabular form to outline the basic principle of odlfi|e. the most common format is the ,,dbase" format, which stores objects horizontally value by value. the first line of the file introduces the structure of the data: a name ol for the table and three attributes attrl, attr2, and attr3. each row is then an object* = (valueil, valuei'2, value/3). a marker "-" should denote records that are deleted. file: ol attrl valueil value21 value31 attr2 value 12 value22 value32 attr3 value 13 value23 value33 specification 1: /* odl part + assignment rules */ interface o\from ot1[ok!="-"] { attribute int attrl = attr1 ; attribute string attr2 = a ttr2 ; attribute int attr3 = attr3 ; /* grammar */ file(ol.tab,datal); data1: _id " "_id " " _id " " _id _eol ot1[1-]; ot1: ok[0-1] ""attr1 "" attr2 " " attr3 _eol; ok: "-"; attr1: _number; attr2: _id; attr3: .number; 103 ajis vol. 8 no. 1 september 2000 the odl schema defines a corresponding interface ol. the structure of ol represents the table directly. this part is the logical view of file contents. it does not depend on how objects are organized in the file. according to our standard mapping strategy, the odl part is mapped on a relation ol(attrl int, attr2 varchar, attrs int) directly. the grammar defines the structure of the file. it first introduces a start symbol data1 for the file ,,ol.tab" by means of a file clause. data1 possesses a rule data1: ... that defines the file contents, starting with _id and a sequence of " " _id just to skip the headline; the _ids are not used as information. after an end-of-line (_eol), one or more ([!-]) ot1-records can occur. the rule for ot1 describes the structure of a single row of the table. such a record consists of an optional marker "-" (ok), a number (attr1), a string (attr2), and a number (attrs), all separated by a blank " ". terminal symbols denote the characters found in the data file. they are embedded in quotation marks. furthermore, there are several predefined non-terminal symbols. for example, _number stands for an optionally signed number. similarly, _id represents a sequence of letters, _digit any digit, _byte any byte character, _eol an end-of-line, and so on. the predefined non-terminals are useful because they make specifications shorter and thus improve readability. we extended yacc in order to ease the writing of specifications. the specification language offers repetition groups known from sgml. <non-terminal>[n-m] determines that <non-terminal> may occur at least n, but at most m times, [n-] means at least n times, [-m] almost m times, and [n] exactly n times. the optional marker can thus be expressed by ok[0-1]. assignment rules now relate the parsed information to objects in the odl schema. the rules extend the odl schema by means of a from clause and attribute equations. these parts are put in italics in the above example. ol from ot1 determines that the non-terminal ot1 characterizes ol objects. technically speaking, any time the rule for ot1 is passed by a parser, a new object of type ol is created. such a rule fixes an object as ..current". equations then describe how to fill attributes of the current object by using the values of non-terminals. each non-terminal has a certain value. this value is a string, which arises as a concatenation of the values of the non-terminals on the right side of its rule: the value of attr1 is exactly the number that is parsed, e.g., "11" after analyzing the second row. this value is exported to set the attribute attrl of ol by means of attrl = attr1. the value of ot1 (if used) is computed by concatenating the value of attr1, a blank (which naturally has the value " "), the value of attr2, a blank, the value of attr3, and an end-of-line. the approach allows one to filter records. filtering is important as there are some file formats that do not really remove records. instead they only mark records in a file as deleted to increase performance. consequently, it is no good to export those records because they should not be there. odlfae offers special conditions that restrict exporting records. for instance, ot1[ok!= "-"](int) determines to export only those records that have no marker "-". each condition is related to a non-terminal and requires a data type, since the comparing non-terminal is usually not used as an attribute value. please note the above file content has a style similar to defining tables in html: <table... > <tralign=left><th> attrl </th><th> attr2 </th><th> attrs </th></tr> <tr align=left> <td> valuel 1</td> <td> valuel2 </td> <td> valuels </td> </tr> </table> a specification must handle <table> etc. as terminals in addition to specification 1. just to demonstrate the power of our approach, we show another representation of tables that stores objects vertically. an object/ = (value/1, value/2, value/3) is built from one column. this table form sometimes occur in html files, if the number of entries is fixed, but the number of columns is subject to changes. this form has advantages as the record structure can grow vertically. file: attrl attr2 attrs value 11 value 12 valuels value21 value22 value23 values 1 value32 valuess specification 2: interface ol from attr1 { attribute int attrl = attr1; attribute string attr2 = attr2; attribute int attrs = attrs; 104 ajis vol. 8 no. 1 september 2000 file(vertical.tab,file); file: _id " " attrlfl-j _eol _id " " attr2[1-] _eol _id " " attr3[1-]; attr1: number " "; this specification is a little tricky because the creation of ol objects is bound to attr1: any time attr1 is passed, a new ol object has to be created. b) several objects of several types in one file there are several possibilities how records of different structure or type can be found in files. the next example discusses several ways to separate records (objects) and fields. we assume a file f 01 ...ol 02 ... o2 03 ... o3 where objects are arranged according to their type, ol should be an object of type ol, o2 an object of type o2 etc. specification 3: interface ol from ot1 { attribute int attr 1 = attr11; attribute int attr2 = attr 12; }; interface o2 from ot2 { attribute string attr 1 =attr21 [ebcdic]; attribute int attr2 = attr22; attribute string attr3 =attr23; }; interface o3 fromot3 { attribute int attr 1 = attr31; attribute int attr2 = attr32; }; file(f,file); file: ot1[1-] _eol ot2[1-] _eol ot3[1-]; ot1: attr11 " " attr12 ""; // (1) separator blank ot2: attr21 attr22 attr23; //(2) fixed-size ot3: val31&val32[0-1]; //(3) token-based attr11: _number; attr12: _number; attr21: _byte[4]; attr22: _digit[5]; attr23: _letter[2]; val31: " attr31:" _colon attr31 _eol; attr31: _number; val32: "attr32: " _colon attr32 _eol; attr32: .number; the odl schema models three object types ol, o2 and o3, each one possessing certain attributes. the non-terminals ot1, ot2, and ot3 directly correspond to the object types: a rule ot1 = ... describes the structure of ol objects, and so on. the example presents three approaches for separating fields in a record. again, each interface o; describes a corresponding table oi. 1. using separators, e.g., blanks (" ") between values for ot1; the blank is necessary because otherwise we do not know where the first integer stops. 105 ajis vol. 8 no. 1 september 2000 2. ot2 records have a fixed size: attr21, attr22, attr23 have a fixed length defined by _byte[4], _digit[5] and _letter[2]. that is, predefined terminal symbols are useful to describe fixed-sized records. 3. ot3 is token-based: tokens "attr31:" etc. denote that the subsequent values belong to this attribute. typical examples for this style are bibtex files and www pages. '&' expresses that the ordering of occurrences is irrelevant, i.e., val31 and val32 may occur in any order within the data. it is cumbersome to describe this variable order by only using alternatives '|' as provided by yacc. the separation of records in the file is just done by ordering the objects and indicating a switch of type by means of _eol. if the structures defined by ot1, ot2 and ot3 characterize the records of each type uniquely, then no _eol is necessary. above, the records are not identifiable because a record 111d22223d 18 causes trouble: is it an object (111,22223) of type ot1, or an object (" 111d",2222, "3d") of type ot2? if several clusters of object types occur in a file, e.g., "ol" ol...ol "o2" o2...o2 "ol" ol...ol "o3" o3...o3, a token has to introduce the change of object type. a specification then looks like: file: data[1-]; data: type1 & type2 & type3; type1: "ol" ot1[1-]; again, if the structure of types is unique, then no special tokens are needed to distinguish the objects in the file. sometimes, brackets are used to separate records, e.g.: file:"(" ot1[1-] ")("ot2[1-] ")("ot3[1-] ")". those context-free dependencies are easily expressible in a grammar. this type occurs in html files that for instance mark tables by means of <table>... </table>. odlpue offers a possibility to distinguish between different encodings such as ascii or ebcdic. an assignment attrl = attr21 [ebcdic] could specify that the parsed attr21-value should be ebcdic-decoded before being assigned to attrl of the current ol-object. ascii-decoding is the default. c) nested objects the following example shows a file that expresses a nested structure as it occurs quite often (ashish and knoblock (1997)). file: ol o2 o3o3...o3 o2 o3o3...o3 ol specification 4: interface ol from ot1 (key attrl) { attribute int attrl = ...; attribute set<o2> seto2 = data2[1-]; }; interface o2 from ot2 (key attrl ) { attribute int attrl = ...; attribute set<o3> seto3 = ot3[1-]; }; interface o3 from ot3 18 the symbol 'd' should denote a blank. 106 aj1s vol. 8 no. 1 september 2000 { attribute int attrl = ...; }; file(f,file); file: data1[1-]; data1: ot1 data2[1-]; data2: ot2ot3[1-]; ot1: ... ot2: ... ot3: ... the indentation denotes the nesting, i.e., ol contains several o2 objects that in turn possess o3 components. the odl schema defines the nesting by means of multi-valued attributes seto2 and seto3, each one referring to a set of subordinated objects. any of the collections (set, bag, list, array) can be used as attribute domains. the grammar uses repetition groups, e.g., data2[1-], to express the nesting. those terms are directly taken in assignment rules for filling the set-valued attributes. hence, seto2 = data2[1-] means that the value of data2[1-] is taken to fill seto2. the value itself is computed by concatenating the data2 values. this concept is very powerful, as we are able to express nested structures in any depth. for example, it is easy to analyze the nesting of headings in www pages or \(sub)section{...} in tex files. the generated tables to keep the file data will then look like ol (attrl int, ...), o2 (attrl int, ..., seto2fk int), and o3 (attrl int, ..., seto3fk int). seto2fk is a foreign key that refers back to the key of table ol to establish the relationship set02, and similar for seto3flc. in order to insert values for foreign keys in tables, we need a primary key. that is why interface ol and o2 must fix a key attribute by means of key. the key can certainly be composite. here we see why we use complex objects in the odl part instead of flat relational structures. it is difficult to specify a nesting by having some kind of ,,flat table" specification. a first try may look in pseudo notation like table o2 from ot2 { attribute int attrl = ...; attribute int seto2fk = ??? } we are confronted with the problem of how to find the foreign key value for seto2fk since the context of rule ot2 is progressed too far; no right side of ot2 carries this information. using multi-valued attributes is thus much easier. handling file systems so far, we have considered one file containing the objects of one or more types. then it is sufficient to specify a grammar for the file, and to associate the filename with this grammar. but it might happen that the objects of one type are spread over several files. in order to collect objects from several files and to take into account the directory structure for modeling file contents, advanced concepts are required. as far as we perceived, the literature has not presented a solution to use this kind of semantics. a) one file = one object type we start with a simple extension of the previous discussion. there are isolated files, each one containing the objects of one type. hence, a file represents one object type. files: fl :ol ...ol £2: o2 ... o2 o: o3 ... o3 specification 5: interface ol from ot1 {...}; interface o2 from ot2 {...}; interface o3 from ot3 {...}; file(fl,filel); file1: ot1[1-]; ... file(£2,file2); file2: ot2[1-]; ... 107 ajis vol. 8 no. 1 september 2000 file(o,file3); files: ot3[1-];... the above lines present the principle of a specification. each file fi obtains a starting symbol of its own, defined by the file clause. the objects of each type may certainly be complex (in the sense of specification 4). then oti must represent the complex structure in the known manner. b) one file = one object in contrast, the objects of one type can be spread over several files. we assume that there are files in a common directory dir; each file should contain one complex object. the directory denotes de-facto an object type. files: file fl: olo2o3...o3o2o3...o3... file £2: olo2o3...o3o2o3...o3... file f3: olo2o3...o3o2o3...o3... specification 6: interface ol from ot1 { attribute string filename = _name; file(dir/*, data1); data1: ot1 data2[1-]; data2: ot2 ot3[1-];... since the objects possess the same structure, one grammar can be used for handling several files. the file clause above determines to take all the files in dir into account. they are all parsed by the same grammar starting with non-terminal data1. the grammar extracts exactly one complex object of type ol from each file, including subobjects of types o2 and o3. the general form of the file-clause is file(<file>, ..., <file>, <non-terminal>). it determines the files to be analyzed by the start symbol <non-terminal>. each <file> possesses the form <directoryspec>/<filespec> with <directoryspec> describing an absolute path /<directory>/<directory>/... . <filespec> and <directoryspec> can be scoped by usual unix wildcards '*' ( any sequence of symbols) and '?' (an arbitrary symbol) to denote several files. for example, file(/dir?/*/p, data1) qualifies all the files that start with an "f and are in any directory beneath /dill, /dir2,..., /dira, /dirb,.... this is a simple, but very effective mechanism. furthermore, there are predefined functions for accessing meta-information such as file and directory names. they can be used for assignments in the same way as non-terminals. in the above example, each object of type ol receives the name fi of its file as value for the attribute filename; a predefined function _name computes the filename. this enables us to capture this important source of additional semantics. a function _dir determines the directory of the current file, _path gives out the complete path as a string etc. these functions enable one to navigate in the directory structure. naturally they can be appended, e.g., _dir._name to compute the name of the directory of the current file. c) one directory = one object type this is an extension of b). it considers several directories din", each one containing several files. each file fy in directory din represents a (complex) object of type oi: files: dirl= {fll,...,flm} dir2= {f 21,...,£2n} specification 7: file(/dirl/*, data1);... file(/dir2/*, data2);... 108 ajis vol. 8 no. 1 september 2000 the essential feature here again is the file clause. all the files in directory din are parsed by the same grammar with start symbol data;'. data/ defines the (complex) structure of type cm. d) directories representing relationships we found some file systems where a directory represents some kind of relationship between the objects contained in its files. let us assume several directories dirk that all contain three files fl, f2 and d each one related to a corresponding type ol, o2 and o3, respectively. those directories dirk can represent a ternary relationship, the ,,cross product" between the objects in files: any object in fl is in relationship with any object in fl and in d. files: dirl = { f l ,c , d } d i r 2 = { f l , f 2 , o } specification 8: interface 01 from ot1 { attribute attrl = ...; attribute dirname = _dir._name; }; interface o2 from ot2 { attribute attrl = ...; attribute dirname = _dir._name; }; interface o3 fromot3 { attribute attrl = ...; attribute dirname = _dir._name; }; file(dir*/fl,datal); data1:ot1[1-];... file(dir*/f2, data2); data2:ot2[1-];... file(dir*/f3, data3); data3: ot3[1-];... please note that the relationship is not directly expressed in the odl schema, because it is difficult to describe the association between a relationship and its directory representation. nevertheless, we use the directory name as an attribute dirname. using dirname, we can join the tables to compute the relationship in sql. below we present some sample files and how they result in tables. dirl: dir2: fl: records 2,3 fl: records 3,4 £2: records 1 f2: records 3,4,5 o: records 1,2,4 f3: records 3 ol id 2 3 3 4 dirname dirl dirl dir2 dir2 02 id 2 3 4 5 dirname dirl dir2 dir2 dir2 o3 id 1 2 4 3 dirname dirl dirl dirl dir2 let us now consider a typical constellation with directories representing some hierarchical relationship. 109 ajis vol. 8 no. 1 september 2000 files: /dir building 1 building2 lathe 1.mac iathe2.mac cutterl .mac building.lay lathes.mac specification 9: interface machine from machine { attribute int mid =_name._tail(1); attribute string type =_name._front(1 jlen-1); attribute string building =_dir._name; }; interface layout from layout { attribute string name =...; attribute string building =_dir._name; }; file(dir/building*/*.mac, machine); machine: ... file(dir*/building*/*.lay, layout); layout: ... there are several directories building/. each of those directories defines a building with layout data (file extension .lay) and machines (.mac) to be placed in the building. the machines have a certain type (e.g., lathe or cutter) and a number. both together form a file name. the file clause determines that all files with an extension .mac (independent of their directory) should be analyzed by the same grammar with starting symbol machine. similarly, the files with extension .lay are handled by layout. specification 9 defines interface's for machines and layouts, essentially using the machine and layout non-terminals, respectively. since there are several machine files, each one results in a machine object of its own. in order to extract the type of a machine, we take the file name by means of _name and decompose the name in two parts, type and number, by predefined functions. for both machines and layouts, we use the directory name of each file as an attribute building. this enables us to express the relationship by attribute building. the object types are directly mapped onto tables machine(mid int, type varchar, building varchar), layout(name,..., building varchar) with the same structure. hyperlinks in files a final example discusses the main principles of handling hyperlinks. we assume a file with several entries of the form country cities eol where any cities (underlined) is a link to another file containing the names of cities in this country (and maybe some other information). the following lines describe the html source: specification 10: interface country from country { attribute string cname = country; attribute set<city> cities = cities; }; interface city from city { attribute name = city; 110 ajis vol. 8 no. 1 september 2000 file(f.html, file); file: data[1-]; data: country " <ahref=http:" _ref cities ">" link "</a>" _eol; country: _id; cities: cityfl-]; city: _id; link: _id; //name of the link in browser hyperlinks occur as <a href=http:address > in www pages. it is not enough to describe such a hyperlink in a grammar by means of "<a href=" cities "> ..." with a non-terminal cities defining the structure of the addressed file. the parser will not be able to recognize the cities part, because it is part of a different file referred to by http:address. the parser tries to map the address to cities. this fails! we introduce a predefined keyword _ref to remedy this: _ref cities tells the parser to take cities as a string, the http address, during parsing. later on, cities is used as a starting symbol for the addressed file. then the rule cities: city[1-] is applied for this file. semantic enrichment having migrated data from files in a relational database, the database can be plugged in a federation. in the framework fihd of hohenstein and ebert (1999), this is done by generating an odmg adapter that homogenizes the relational database. homogenizing means to remodel tables in the odmg object model. in the fihd approach, we incorporate ideas developed in the context of reverse engineering and semantic enrichment (chiang et al. (1994), premerlani and blaha (1994), castellanos and saltor (1991), hohenstein and korner (1995)). hence, tables are not just converted to object types that possess the simple table structure, i.e., object types without relationships and without subtyping. instead, the semantic level of the relational schema is upgraded: implicit knowledge is expressed explicitly by using higher modeling concepts. this is advantageous, since data integration is a complex and ambitious task. providing an integrated access to dbss requires a deep knowledge about the semantics of data so that inter-database semantic relationships can be detected. any support that can be provided previously during homogenization is useful to let the integration concentrate on its real task. the overall approach of homogenization with semantic enrichment resembles the one for our file handling. it again uses a generator, relying on a specification language called odlr (hohenstein and korner (1995)): the language allows specifying how to homogenize the relational database in the odmg model. the advantage of our approach lies in the fact that semantic enrichment is explicitly precisely specified. this gives us the opportunity to remodel schemata in various ways, making any semantics explicit. the generator produces a corresponding odmg oml/oql wrapper to access the database. since the odmg adapter provides an object-oriented view of relational data including relationships and subtyping, the semantics is usable in form of high-level odmg operations, manipulating objects and relationships instead of isolated tuples. the concepts for semantic enrichment are very powerful. for instance, several attributes can be joined together to make an attribute of a predefined odmg type or an object type nested into another. new object types may be introduced, e.g., to bring together common attributes in a generalized object type. it is possible to remodel typical forms of relational representations for relationships and subtypes. particularly, several representations for subtype hierarchies can be handled. in hohenstein and korner (1995) we discuss a vertical, a horizontal and a complete materialization strategy as well as a flag approach. each one has pros and cons in regard to fast access, redundancy, and easy update. these substantial strategies are orthogonal and can be applied in a mixed manner within one subtype hierarchy. for more details about the specification language, the reader is referred to hohenstein and k6mer(1995). the following specification gives an example of enriching the tables obtained by specification 9. let us assume we want to see an additional object type building, two subtypes odmgcutter and odmglathe of machine according to the type of machines, and some explicit relationships. interface odmgmachine from relation machine[mid,type] { attribute int id = machine.mid; 111 ajis vol. 8 no. 1 september 2000 interface odmgcutter : odmgmachine from relation machine[type= "cutter"] {} interface odmglathe : odmgmachine ... from relation machine[type= "lathe"] ... interface odmgbuilding from relation group(machine[building]) { attribute string name = key; relationship set<odmgmachine> machines = { machine | building = key }; } interface odmglayout from relation layoutfname] { attribute string name = ...; relationship odmgbuilding belongsto = group { machine | building = layout.building } ; in contrast to odlfiie, the full odmg odl is used here to express semantics. we again introduced some amendments to express connections between object-oriented and relational schemata. the clause from relation relates the specified object types to tables. it is defined in what table the deep extent of a type (including objects of subtypes) is found. the easiest case is odmglayout from relation layout[name]: an object type is built from one relation directly, name is the relational key of layout. keys are necessary to identify objects in the object-oriented runtime system: the key value is directly taken as object identifier. each tuple, which is uniquely identified by its key value, refers to one object. tables without keys cannot correspond to object types. they can be used as bags nested in other types, nevertheless. composite keys are possible and denoted as (mid,type), cf. odmgmachine. the objects of subtype odmgcutter are also found in table machine, which contains all the machines, cutters and lathes. however, cutters are marked in the table by a flag type which is used to separate cutters from other machines: [type="cutter"]. hence, subtyping by flag is made explicit. concerning buildings, the attribute building of machine is ,,objectified". building objects are built from table machine. all the tuples in machine are grouped by building, and each such group builds an object. the keyword group expresses such a grouping. the number of objects corresponds to the number of different building values, building is the key of a group and can be referred to by key. equations '=' occurring behind the attributes relate attributes and relationships of object types to relational attributes. the simplest form is int id = machine.mid and relates an object type attribute id of odmgmachine with domain int to a relational attribute mid of table machine directly. relationships are specified similarly. odmgbuilding belongsto = group { machine | building = layoutbuilding } expresses a reference to odmgbuilding for interface odmglayout. remember, buildings are part of table machine, and the relationship is relationally represented in tables machine and layout by an attribute building. belongsto is computed as follows: take the set of tuples in machine that have a building-value equal to the layout's building, and then group the set according to building (that is the key of odmgbuilding). similarly, the relationship machines is expressed as all the tuples in machine that possess a building equal to the key. conclusions in this paper, we presented a specification-based approach to incorporate files in a federation approach based on the odmg standard (cattell and barry (1997)). the procedure consists of two steps: migrating data from several files and directories into a relational database, thereby giving the data a first structure. and then building an odmg adapter on top of the database; the adapter can be plugged in a federation. the two steps together make intensive use of the odmg object model, i.e., the federation layer can in total benefit from a real object-oriented view of file data. both steps are supported by specification languages and generators. a first language allows making semantics of files and directories explicit, i.e., giving files some structure. such a specification is input to a generator that produces a file adapter. the adapter is responsible for parsing and unparsing 112 ajis vol. 8 no. 1 september 2000 (modified data is stored back into files). moreover, an adequate relational database is installed, the tables of which are automatically filled with parsed file data. providing an odmg adapter for the relational database is done by a generator, too. defining semantic enrichment in a specification language makes the semantics of relational tables explicit by using objectoriented modeling concepts. a real object-oriented view, including subtyping and relationships, is achieved. a generator produces an odmg2.0 conforming manipulation and query interface. this gives the federation the opportunity to see relational data in an object-oriented way, to manipulate relational data in terms of c++ objects. either step is applicable for querying the www by now using sql or odmg/oql, respectively. the approach is embedded in a federation framework fihd (flexible integration of heterogeneous data sources), which is described hohenstein and ebert (1999). similar to tsimmis (papakonstantinou et al. (1995)), information manifold (kirk et al. (1995)), efendi (radeke (1995), oasis (roantree et al. (1999)) or sigmarob (hoding et al. (1999)), fihd has the goal to provide a global view of several, heterogeneous data sources, and to support an easy access to all the data without knowing the exact source and type of source. relying again on a generative principle, our federation approach uses the odmg adapters that homogenize databases and plugs them in a federation framework, which then gives a transparent odmg access. generators have been built for producing adapters for commercial systems such as oracle, sqlserver and informix, or object-oriented database systems such as objectivity/db and versant. future work will be dedicated to a comfortable graphical support for defining specifications similar to hohenstein and korner (1995). integrating file systems into the federation framework directly, avoiding the detour via relational databases, is another point. we also think of other types of front-end interfaces for the federation framework. owing to our fihd architecture, it is easily possible to generate any type of interface instead of odmg2.0, e.g., activex components, html pages, or xml to visualize information obtained from the federation. references abiteboul, s., cluet, s. & milo, t. (1993) ..querying and updating the file", proc. conf. on very large databases (vldb) 1993 abiteboul, s. cluet, s. & milo, t. (1995) ,,a database interface for file update", proc. acm sigmod'95, san jose, ca usa 1995 ashish, n. & knoblock, c. (1997) ..wrapper generation for semi-structured internet sources", proc. of acm sigmod workshop on management of semi-structured data, tucson (arizona) 1997, superseded by acm sigmod record 26(4), dec. 1997 buneman, p., davidson, s., fernandez, m. & suciu, d. (1997) .adding structure to unstructured data", proc. of int. conf. on database theory (icdt97), delphi 1997 busse, r., fankhauser, p. & neuhold, e. (1994) .federated schemata in odmg", proc. of 2nd east/west database workshop 1994 cacm (1994) ..reverse engineering", special issue of communications of the acm 37(5), 1994 castellanos, m. & saltor, f. (1991) ,,semantic enrichment of database schemas: an object-oriented approach", in y. kambayashi, m. rusinkiewicz, a. sheth (eds.): proc. of 1st int. workshop on interoperability in multidatabase systems, kyoto (japan), 1991 cattell, r. & barry, d. (1997) ,,the object database standard: odmg2.0", 2nd edition, morgankaufmann publishers, san mateo (ca) 1997 chiang, r., barron, t. & storey, v. (1994) ..reverse engineering of relational databases: extraction of an eer model from a relational database", data&knowiedge engineering 12, 1994 conrad, s., eaglestone, b., hasselbring, w., roantree, m., saltor, f., schonhoff, m., strafller, m. & vermeer, m. (1997) ..research issues in federated database systems", sigmod record 12/1997,26(4) conrad, s., hasselbring, w., heuer, a. & saake, g. (1997) ..proc. of the int. caise97 workshop engineering federated database systems efdbs'97", barcelona 1997 conrad, s., hasselbring, w., saake, g. (1999) ,,proc. of the 2nd workshop engineering federated information systems", kuhlungsbom (germany) 1999 ebert, a., hohenstein, u. & hoding, m. (1999) ,,an approach for generating file interfaces", proc. of database systems for advanced applications (dasfaa), taipeh 1999 113 ajis vol. 8 no. 1 september 2000 hainault, j.-l., tonneau, c., joris, m. & chandelon, m. (1993) ,,schema transformation techniques for database reverse engineering", 12th int. conf. on entity-relationship approach, karlsruhe 1993 hammer, j., garcia-molina, h., cho, j., aranha, r. & crespo, a. (1997) extracting semistructured information from the web", proc. of acm sigmod workshop on management of semistructured data, tucson (arizona) 1997, superseded by acm sigmod record 26(4), dec. 1997 hoding, m. (1996) ,,an approach to integration of file based systems into database federations", proc. of 10th european research consortium for informatics and mathematics (ercem'96) on heterogeneous information management, prague 1996 hoding, m., schwarz, k., conrad s. et al. (1999) ,,sigmafdb: overview of the magdeburg-approach to database federations", in conrad et al. (1999) huck, g., fankhauser, p., busse, r. & klas, w. (1994) .jro-db: an object-oriented approach towards federated and interoperable dbms", advances in databases and information systems (adbis'94), moscow 1994 hohenstein, u. & ebert, a. (1999) ,a comprehensive toolkit for building database federations", australian journal of information systems, vol.7, no. 1, september 1999 hohenstein, u. & korner, c. (1995) "a graphical tool for specifying semantic enrichment of relational databases", 6th ifip wg 2.6 work. group on data semantics (ds-6) "database applications semantics", atlanta 1995 hohenstein, u. & pleber, v. (1997) «a generative approach to database federation", in d. embley, r. goldstein (eds.): 16th int. conf. on conceptual modeling er'97, 1997, los angeles, springer lncs 1331 ims (1993) ,,proc. of conf. on research issues in data engineering: interoperability in multidatabase systems" (rtoe-ims'93). vienna 1993 kaul, m., drosten, k. & neuhold, e. (1990) ,,viewsystem: integrating heterogeneous information bases by object-oriented views", proc. of 6th int. conf. on data engineering, los angeles 1990 kambayashi, y., rusinkiewicz, m. & sheth, a. (1991) (eds.) ,,proc. of 1st int. workshop on interoperability in multidatabase systems", kyoto (japan), 1991 kirk, t., levy, a., sagiv, y. & srivastava, d. (1995) ,,the information manifold", proc. of the aaai spring symposium series, march 1995 kuno, h. & rundensteiner, e. (1996) ,,the multiview oodb view system: design and implementation", theory and praxis of object systems 2(3), 1996 kushmerick, n., weld, d. & doorenbos, r. (1997) ..wrapper induction for information extraction", int. joint conf. on artificial intelligence, nagoya (japan) 1997 markowitz, v. & makowsky, j. (1990) ..identifying extended er object structures in relational schemas", ieee trans, on software engineering 16(8), 1990 nestorov, s., abiteboul, s. & motwani, r. (1995) ..inferring structure in semistructured data", proc. of acm sigmod workshop on management of semi-structured data, tucson (arizona) 1997, superseded by acm sigmod record 26(4), dec. 1997 papakonstantinou, y., garcia-molina, h. & widom, j. (1995) ..object exchange across heterogeneous information sources", proc. of ieee conf. on data engineering, taipeh 1995 pitoura, e., boukres, o. & elmagarid, a. (1995) ..object-orientation in multidatabase systems", acm computing surveys 27(3), 1995 radeke, e. (1995) .jefendi: federated database system of cadlab", acm sigmod conf. on management of data 1995, sigmod record 24(2) reddy, m., prasad, b., reddy, p. & gupta, a. (1994) ,a methodology for integration of heterogeneous databases", ieee trans, on knowledge and data engineering 8(6), 1994 premerlani, w. & blaha, m. (1994) ,,an approach for reverse engineering of relational databases", communications of the acm 37(5), may 1994 roantree, m. & murphy, j. (1997) ,,an architecture for federated database metadata", hi conrad et al. (1997) roantree, m., murphy, j. & hasselbring, w. (1999) ,,the oasis multidatabase project", acm sigmod record 28(1), march 1999 saltor, f., castellanos, m. & garcia-solaco, m. (1992) ..overcoming schematic discrepancies in interoperable databases", in d.k. hsia, e.j. neuhold, r. sacks-davis (eds.): proc. of the ifip wg 2.6 database semantics conf. (ds-5) on interoperable database systems, lome (australia), 1992 sattler, k.-u. & hoding, m. (1999) ..adapter generation for extracting and querying data from web sources", proc. of 2nd acm sigmod workshop webdb'99, 1999 114 ajis vol. 8 no. 1 september 2000 schmitt, i. & saake, g. (1995) integrating of inheritance trees as part of view generation for database federations", 15th int. conf. on conceptual modeling (er'96), cottbus 1996, springer lncs 1157 sheth, a. & larson, j. (1990) ..federated dbss for managing distributed, heterogeneous and autonomous databases", acm computing surveys 1990, 22(3) spaccapietra, s. & parent, c. (1994) "view integration: a step forward in solving structural conflicts", ieee transactions on knowledge & data engineering 6(2), 1994 115 ajis special edition knowledge management december 2001 impediments to ester-firm transfer of best practice in an enterprise systems context greg t timbrell natalie m andrews guy g gable information systems management research centre queensland university of technology g.timbrell@qut.edu.au abstract a survey of major issues with enterprise systems was administered within the state government of queensland, australia and a subset of these issues mapped against szulanski's model of inter-firm best practice transfer stickiness (szulanski, g. 1996. exploring internal stickiness: impediments to the transfer of best practice within the firm. strategic management journal, 17(special issue), 27-43). the paper discusses impediments to knowledge transfer within this context, presents findings from the a posteriori application of szulanski's model to the survey responses, and suggests future research directions. keywords: erp, enterprise systems, knowledge management, knowledge transfer, consulting introduction a parent study, "cooperative enterprise system life-cycle knowledge management" (gable et al, 1998) explores means of better managing knowledge across the three key players involved in enterprise systems life-cycle support the client, vendor and implementation partner. herein we explore knowledge transfer issues in the enterprise systems (es) context. an early stage of the parent study entailed a study of major issues within the queensland state government in australia. through a modified delphi approach, personnel that had been involved in implementation and management of the common es sap r/3, were surveyed to determine what major issues they had experienced with the system. implementation partners were also asked to furnish major client-issues. the survey was piloted in five agencies (for discussion of methodological issues related to the pilot study see chang et al, 2000), then extended to the rest of the queensland government. from the extension study 479 knowledge transfer related survey responses were identified, then mapped against szulanski's model of intra-firm transfer of best practice (szulanski, 1996) in order to explore barriers to knowledge transfer on es implementations. this paper firstly describes why knowledge transfer in an es context is important. we then provide a literary overview of knowledge transfer, including a description of szulanski's model. after briefly recounting the pilot mapping exercise to szulanski's model and some preliminary results we lastly suggest future research directions. knowledge transfer in the enterprise systems context enterprise systems a new class of packaged application software has emerged over the past decade, ostensibly consolidating under a single banner, a multi-billion dollar industry that includes sap ag, the world's fourth largest software vendor, several other of the largest software firms and the world's largest management consulting organisations. variously called enterprise resource planning systems (erp), enterprise-wide systems, or simply enterprise systems (es), these comprehensive, packaged software solutions seek to integrate the complete range of a business's processes and functions in order to present a holistic view of the business from a single information and it architecture (gable et al, 1998; klaus et al, 2000). according to chan (1999), es implementations require a wide range of knowledge including project knowledge (how to implement es, business process engineering, change management, training and education), technical knowledge (such as programming, system and database administration), product knowledge (specifics of the es) and business knowledge (... of the business, it's culture and people). where an organisation does not have the requisite knowledge, it will often seek this implementation knowledge from third-party providers. implementation resources are predominantly knowledge based. this knowledge may be sourced from a (typically large) consulting firm (knowledge vendor), which acts in the capacity of implementation partner. sharing the joint objective of successful 116 ajis special edition knowledge management december 2001 implementation and in some cases, ongoing management, an alliance is formed between the client organisation, the es software vendor, and this third-party. having engaged a suitable implementation partner, the client completes the implementation process, goes live with the es and moves into the post-implementation maintenance and upgrade cycle. at the end of the implementation phase, the consultant usually withdraws from the organization. responsibility for managing the es falls back to the client. continuing success of the es becomes reliant on the client's skill and knowledge in operating, maintaining and upgrading the es. in order to keep the es 'live' and relevant, the client must either draw from their es capabilities transferred-in during the implementation period or seek expert support (knowledge) externally. such external support is usually available from the vendor, the implementation partner and other third parties and is often expensive. organisations planning to support the es in-house (insourcing) face the issue of attracting or developing, then retaining staff with the necessary knowledge. where the client plans to outsource its es to an application service provider (asp), post-implementation es knowledge self-sufficiency may not be necessary. asp vendors cite this alleviation of 'future skills risk' as one of their competitive advantages (bennett and timbrell, 2000). on the other hand, should the organisation follow an insourcing strategy, then it will often aspire to post-implementation es knowledge self-sufficiency, in order to reduce reliance on third-party support and the associated high costs of that support. knowledge management, and knowledge sharing in particular, is considered to offer significant commercial and practical benefits throughout the es life-cycle (gable et al, 1998). it can be argued, therefore, that knowledge transfer from vendor and implementation partner to the client organisation is an important factor in es life-cycle management. in the case of a client organisation outsourcing their es to an asp, feeny and willcocks (1998) suggest that consideration be given to what knowledge or competencies should be developed or retained in-house. according to davenport (2000), client organisations often experience poor es implementations because they regard the project as a one-time exercise and so fail to attend to es knowledge management issues, such as requesting (contracting for) knowledge transfers from consultants, or adequately maintaining the transferred knowledge. while the current literature falls short of empirically demonstrating causal links between effective knowledge transfer and successful es implementations, the expectation is that knowledge transfers leave the client organisation better positioned to maintain and evolve their system, and to generate returns from the es investment (davenport, 2000). this project and its umbrella research program attempt to bridge this gap in the literature. however the mere possession of transferred knowledge is unlikely to be sufficient to meet a client's independence objectives. effective knowledge transfer requires not only transmission, but also knowledge absorption and use (davenport and prusak, 1998). developing the capability to effectively maintain and leverage knowledge will depend on the client's ability to either integrate new es knowledge with knowledge of the business or combine it with existing capabilities. this knowledge must then be used to create appropriate routines and capabilities in support of business objectives (andreu and ciborra, 1996; kogut and zander, 1993; inkpen and dinur, 1998). organisational capability emerges over time through a process of organisational learning (levitt and march, 1988). it is informative then, to examine the literature regarding the transfer of knowledge into and within the firm for purposes of organisational learning. knowledge transfer according to nonaka (1994), organisational learning depends fundamentally on the internalisation of knowledge where explicit (formalised, explicated) knowledge is converted into tacit forms such as individual know-how and organisational routines. the knowledge is literally absorbed through action or experience. inkpen (1996) extends this notion by arguing that capability development in an alliance-context is therefore dependent on the integration of internalised knowledge from external sources into organisational routines. thus, in an es implementation, knowledge sourced from the implementation partner and the vendor must be translated, adapted and combined with knowledge of the organisation's business processes, then internalised into new organisational routines specific to the organisation's context. the resource-based view of the firm emphasises leveraging of the firm's resources to develop organisational capabilities, and exploiting these to the firm's advantage. andreu and ciborra (1996) discuss the importance of situated learning and knowledge accumulation in the capability development process. their work is notable for its focus on the role of context in organisational learning loops. they posit that the more path-dependent the learning process to develop work practices, routines and capabilities, the more idiosyncratic they are to the firm, and therefore, the less generalisable and transferable to different organisational contexts. over time, business environments and their apposite legacy or es, develop firm specific peculiarities. 1 1 7 ajis special edition knowledge management december 2001 constantly faced with implementing es in unique and distinctive business environments, consultants sieve their experiences in a systematic way to extract idiosyncrasies from their es client encounters thereby providing broad methodologies for general application to future es clients. consulting firms go to great lengths and expense to externalise es knowledge in order to achieve a comparative advantage and to leverage their costly people (timbrell and gable, 2001). vendors also are adopting this approach incorporating industry templates and reference process models into their product offerings. both consultant and vendor are looking to record their implementation experiences for the development of best practice implementation methods. these methods increase consulting firms' net value-add during future implementations by improving completeness and quality, expediting implementation (reducing client costs), and/or allowing the deployment of less experienced (less expensive) consulting staff. szulanski's model a different approach to knowledge transfer underpins research by szulanski (1995) who uses a (shannon and weaver, 1949) communications metaphor to analyse intra-firm transfer of best practice in a manner analogous to the transmission of a message from a source to a recipient within a given media or context. szulanski argues however that while knowledge transfer is a distinct experience rather than diffusion, best practice transfer should be regarded primarily as a process, rather than a transaction or event. best practice transfer thus unfolds over four stages through which organisational routines are replicated. initiation is defined as comprising ".. .all events that lead to the decision to transfer"; implementation "...begins with the decision to proceed"; ramp-up "...starts when the recipient starts using transferred knowledge"; and integration "...begins after the recipient achieves satisfactory results with the transferred knowledge" (szulanski, 1996, pp.28-29). the significance of szulanski's work is his analysis of the effects of certain characteristics of the context, the source, the recipient, and the message itself (ie. the knowledge or practice) in generating "noise" at various stages of the transfer process. szulanski argues that the successful transfer of best practices should begin by assessing the origins of transfer difficulty and then incorporating selective interventions aimed at the most significant barriers. an understanding of the types of mechanisms that have been empirically and theoretically demonstrated as potential barriers and facilitators of knowledge transfer may thus prove instructive when identifying the key issues at play in the particular case of es knowledge transfers. to provide structure to this analysis, we draw from knowledge management and other literature, using szulanski's model as a framework against which four categories of knowledge transfer mechanisms are examined. beginning from the premise that the transfer of best practices inside the firm can be "sticky" or difficult, szulanski explored the origins of internal stickiness in an intra-firm context (1995, 1996, 2000). his model is illustrated in the following diagram. characteristics of the transfer context barren organisational context arduous relationship characteristics of knowledge source lack of source motivation lack of perceived reliability of source characteristics of knowledge unproven knowledge causal ambiguity characteristics of knowledge recipient lack of recipient motivation lack of recipient absorptive capacity lack of recipient retentive capacity table 1: szulanski's origins of knowledge 'stickiness' characteristics of the knowledge causal ambiguity signals the absence of know-why ... why something is done, and why a given action results in a given outcome (szulanski, 1995). szulanski found causal ambiguity to be a significant origin of stickiness through all phases of the transfer process and particularly important during the first three stages. lippman and rumelt (1982) argued in their theory of uncertain imitability that causal ambiguity may impede the precise replication of knowledge in a different setting by obscuring effects of idiosyncratic features of the new context in which the knowledge is used. difficulty in replicating a capability can arise from ambiguity about the elements of success or failure and how they interact. szulanski (1995) also found that proof of the usefulness of the source's knowledge is most important during the first two stages of the knowledge transfer process. the less substantiated the knowledge or the greater the speculation about claims it will solve the recipient's problem ("unproven knowledge"), the more difficult it will be to induce the recipient to engage in the transfer. 118 ajis special edition knowledge management december 2001 characteristics of the source of knowledge szulanski's research also supported the notion that stickiness may derive from lack of source motivation to engage in knowledge transfer. the source may be reluctant to share for fear of losing ownership or privilege, they may perceive inadequate rewards for sharing, or they may be unwilling to commit time and resources to the transfer. confirming arrow's (1971) view, szulanski also found lack of perceived reliability of the source to be an important origin of stickiness early in the initiation stage. szulanski draws on persuasion theory in associating reliability with expertise and trustworthiness and notes that where these are perceived as lacking, transfer may be sticky and the source's advice challenged and resisted. davenport and prusak (1998) support this argument, noting that people judge knowledge according to the status and reputation of its source. characteristics of the recipient of knowledge szulanski's research confirmed the popular notion that a lack of recipient motivation to accept knowledge from an external source and engage in particular activities that require its use will present stickiness. interestingly, he found not only that knowledge-related variables were more important barriers than motivation, but also that a motivated recipient may add to transfer difficulty. this is because impatiently enthusiastic recipients embraced transfers without meticulous planning, resulting in cost and time overruns. lack of absorptive capacity of the recipient was strongly supported in szulanski's research as a major barrier to knowledge transfer, particularly during the implementation phase. cohen and levinthal (1990) found that a recipient's stock of prior related knowledge determines their absorptive capacity for new knowledge, and deficiencies can render the recipient unable to successfully exploit new knowledge. aside from basic skills, important prior knowledge may include a shared language, previous relevant experience and knowing where to find complementary expertise. the recipient's ability to retain transferred knowledge through institutionalizing its use, reflects retentive capacity. if this ability is lacking then difficulties in integrating knowledge may result in failure to persist in using it and even reversion to the status quo (zaltman et al, 1973). szulanski (1995, p. 122) confirmed lack of recipient retentive capacity as a cause of stickiness and argued that overcoming this barrier may require unlearning routinised use of prior knowledge. characteristics of the transfer context recognising the importance of organisational context to the gestation and evolution of a transfer initiative, szulanski described as 'fertile' or 'barren' those contexts that either facilitate or impede the number and fate of knowledge transfer attempts. his research confirmed barren organisational context as an important origin of stickiness during the ramp-up and integration phases. davenport and prusak (1998) discuss several cultural factors that may inhibit knowledge transfer, such as lack of trust; different cultures, vocabularies and frames of reference; lack of time and meeting places; a narrow idea of productive work; status and rewards accruing to knowledge "owners"; "not-invented-here" syndrome; and, intolerance of mistakes or need for help. above all else, they emphasize the importance of trust and common ground in facilitating knowledge transfer. "the closer people are to the culture of the knowledge being transferred, the easier it is to share and exchange" (p. 100). szulanski's research found that an arduous relationship "laborious and distant" (1996, p.32) between source and recipient increases difficulty during the implementation phase of knowledge transfer, when interaction is at its most intense. this has notable implications for tacit knowledge transfer, which may necessitate numerous individual exchanges (nonaka, 1994). overall, szulanski's (1995) empirical results from study of the origins of internal stickiness suggest that the major barriers to intra-firm knowledge transfer were lack of absorptive capacity on the part of the recipient, causal ambiguity with regard to the knowledge itself and an arduous relationship between the source and the recipient. data collection and methodology a survey was conducted of staff in queensland government agencies who were involved in the implementation of sap r/3, the government-wide standard es. staff were asked "what do you consider have been the major issues in implementing, managing and/or supporting the sap financials in ... (your government) agency?" 119 ajis special edition knowledge management december 2001 we classified these survey responses using a rules-based method (see appendix one) where individual responses were coded to szulanski's stickiness and transfer phase constructs. szulanski's constructs break down into a series of statements that were modified for the purposes of this analysis. in adapting szulanski's (intra-firm) instrument we removed statements purely relevant to internal transfers to establish its relevance to an inter-firm transfer context. for example, szulanski's original statement "at «company» despite structural differences units can always learn from one another", was changed to "low acceptance for copying and adapting knowledge or practices". to further assure this process, szulanski's original literature review was systematically revisited in order to confirm the theoretical basis for each statement. in szulanski's original instrument he included an open-ended question about the most important difficulty experienced in transfer (1995, pp.127-128). some example responses from this open-ended question were incorporated in our coding instrument. for instance, "nobody fully understood the concept" was classified by szulanski under causal ambiguity and so we added this to our coding table (see appendix one). a final adaptation of szulanski's instrument was the modification of his statements so that each was expressed as a barrier to knowledge transfer in our coding method. for example, "the limits of the knowledge or practice are fully specified" was changed to "limits of the knowledge or practice were not fully specified". consequently, the survey responses (which express issues related to erp) were coded to transfer phases or stickiness origins only if they conveyed elements of difficulty or stickiness in transfer. our modified codification instrument is attached as appendix one. a relatively simple procedure was used to classify the responses against szulanski's framework. each issue raised was considered in the context of its accompanying narrative description. codes were assigned (e.g. ca 5 meaning causal ambiguity reference statement 5) according to the constructs reflected and the instrument reference number applicable to the issue. the aim was to assign at least one code to represent the transfer phase to which the issue referred, and to identify at least one stickiness origin, on the premise that a knowledge transfer is defined as sticky when it is "worthy of remark" or "eventful" (szulanksi, 1995a, p21). there was no upper limit on the number of codes assigned per issue, this being driven by the data itself. ultimately, each original issue was associated with (coded to) one or two transfer phases; and one, two or three stickiness origins. this process effectively "decomposed" compound statements reported within a single "issue", thereby providing a means of addressing respondents' differing writing styles (i.e. reporting multiple items against the one "issue" versus reporting them as separate "issues"). on occasion it was necessary to interpret a remark based on other issues raised by the same respondent. for example, "availability of skilled sap resources" was classified under barren organisational context when considered in relation to further comments by the same respondent. in this instance other issues cited by the same respondent related to the barren organisational context and included "awareness of the project and sap in the organization " and "communication within the project". the codes assigned to each issue were then entered into a relational database and data analysis was subsequently performed to identify the incidence of stickiness origins reported for each transfer phase. the rules-based approach employed in this mapping exercise is summarised forthwith: • is it a knowledge related issue? • is there a direct, potential or inferred barrier? when did this happen? ie. fixing the potential barrier would have resulted in a better [lifecycle] phase. • is it relevant to knowledge transfer? • to what phase / outcome does this issue apply? • to what characteristic of knowledge or transfer participant or context does this issue apply? • reference the data set of other issues reported by the same respondent if further clarification is required. findings four hundred and seventy-nine (479) survey responses were assigned codes representing potential stickiness origins using the rules-based classification method. many of the survey responses contained multiple issues as well as issues that were be assigned multiple codes for both the transfer phase and the stickiness origin. the following table sets out the totals of phase and stickiness origin assignments: 120 aj1s special edition knowledge management december 2001 coding assignments 1 st transfer phase 2nd transfer phase total 1st stickiness origin 479 81 560 2nd stickiness origin 233 48 281 3rd stickiness origin 64 20 84 total 776 149 925 table 2: assignments of transfer phase and stickiness origins to issues stickiness instances were then mapped against their associated transfer phases. table 2 compares szulanski's findings with findings from this study and suggests the order of importance of the stickiness origin within each transfer phase based on counts. following we discuss only extreme cases of either accord or discord between szulanski ranks and our study counts. note that there are insufficient counts from our mapping of issues to draw any conclusions about es knowledge transfer in the initiation phase. this is because there was only a limited number of people within the queensland government involved in the initiation (selection, planning) of the common sap r/3 erp. using regression analysis, szulanski calculated the standardized beta coefficients for each of his stickiness origins within each transfer phase. in example the regression equation for the 'implementation' phase is ... stickiness implementation = bl *knowledgecausal ambiguity + b2 "knowledge unproven knowledge + b3*source lack motivation + b4*sourcelack perceived reliability + b5*recipientlack motivation + b6*recipientlack absorptive capacity + b7*recipientlack retentive capacity + b8*contextbarren + b9*relationship arduous. table 3 (adapted from szulanski, 1995:pp89&91) displays the ranks of only the beta coefficients szulanski found to be significant in each phase (ranks being based on the relative size of the betas with the largest beta being ranked 1). from szulanski's ranks in table 3 we see that causal ambiguity was found to be an important origin of stickiness across all transfer phases (included in the regression equation for all four phases and ranked 3rd, 2nd, 3rd and 4th most important of the 9 origins). our counts however do not support this finding (based on this study's counts, causal-ambiguity is not ranked in the top 4 in any transfer phase). it is suggested that in an es setting the rationale for an action and the explanation of the resultant outcome are usually clear. in the absence of such rationale the es processes would be highly questionable in an organisational setting. transfer phases stickiness origins knowledge: causal ambiguity unproven knowledge source: source lacks motivation source not perceived as reliable recipient: recipient lacks motivation recipient lacks absorptive capacity recipient lacks retentive capacity context: arduous relationship barren organisational context 1 2 3 4 5 6 7 8 9 n average initiation t # r na 2 na 1 na 1 na 3 na 2 na na 1 na 4 na 14 s r b 3 0.17* 2 0.27 1 0.28 168 . . . . implementation t # r 26 5 19 7 22 6 51 4 64 3 76 2 15 8 9 9 82 1 364 s r b 2 0.22 4 0.16" 3 0.20 1 0.50 2 0.22 169 40 . . . ramp-up t # r 27 6 49 2 7 8 14 7 31 5 73 1 45 3 5 9 33 4 284 s r b 3 0.22 3 0.22 4 0.18 5 -0.16* 1 0.50 2 -0.44 4 0.18* 165 32 integration t # r 12 6 24 5 6 7 4 8 33 4 44 3 73 1 1 9 66 2 263 s r b 4 0.16* 3 0.22 1 0.38 2 0.23 166 29 t=timbrell etal, s=szulanski, #=counts, b=beta coefficients (p<.01or 'p<.05), r=ranks (top 4 timbrell ranks bolded) table 3: incidence of stickiness, by origin, by transfer phase unproven knowledge was found by szulanski to be important in the first two transfer phases (2nd and 4th). our counts suggest that unproven knowledge is important in the 3rd phase i.e. ramp-up (ranked 2nd). it is during this stage that users first experience information from the system through reports. if this information is (or is perceived to ajis special edition knowledge management december 2001 be) inaccurate, users begin to doubt the usefulness of the es and speculate on its ability to solve their information needs. source lacks motivation was found to be an important origin of stickiness during the implementation (3rd) and rampup (3rd) stages in szulanski's study. our counts do not support this finding suggesting that the main knowledge sources, vendor, consultant, business areas and implementation team, are sufficiently motivated to share their knowledge in order to meet the organisation's objective of an effective es. source not perceived as reliable was found by szulanski to be 1st and 4th most important in the initiation and rampup phases respectively, whereas our study counts suggest this to be the 4th most important origin of stickiness during implementation and relatively unimportant during other phases. a possible explanation is that users did not regard some junior consultants as sufficiently expert to implement or support sap in a complex organisational context. szulanski's ranks (5th and 3rd) and those from this study (5th and 4th) for recipient lacks motivation are highly similar for the ramp-up and integration phases. unlike szulanski's study, in this study, this stickiness source ranks high (3rd) in the implementation phase. during implementation the recipient client organisations demonstrated thenlack of motivation in several ways: insufficient resources allocated to the project, rushed and unplanned es introduction and poor stakeholder and executive support. we note especially strong correspondence between the szulanski findings and our study counts on recipient lacks absorptive capacity which on szulanski's data ranks 1st in each of the latter 3 transfer phases, and ranks 2nd, 1st and 3rd based on our study counts (counts of 76, 73 and 44). this might be explained by the relative complexity of the es, an issue that consistently arose in the major issues study conducted in the queensland government. szulanski ranks recipient lacks retentive capacity high (2nd) in ramp-up only. our data suggest this source of stickiness is important (1st) during integration as well. we note that users experienced much difficulty unlearning old processes and system functions. barren organizational context was ranked 1st in our counts during the implementation stage, 4th during ramp-up and 2nd in the integration phase. while there is close correspondence with szulanski's ranks during ramp-up and integration (also 4th and 2nd) our finding for the implementation stage conflicts with szulanski's finding. drivers for this measure include poor executive and stakeholder support, poor planning, training and change management. finally, szulanski ranks arduous relationship 2nd during implementation, whereas our counts are low on this origin, for all transfer phases. a common indication from both studies is the problem of deficient absorptive capacity of knowledge recipients across the phases of implementation, ramp-up and integration. this suggests we concentrate our further analytical efforts on the recipients of knowledge, and specifically their preparedness and ability to accept and absorb es knowledge. origins of stickiness frequently cited in the current study (high counts), but not ranked highly by szulanski include source not perceived as reliable (implementation) and recipient lacks retentive capacity (integration). the former may reflect the serious dearth of es expertise at the time of most queensland government implementations and the concurrent general insufficiency of product knowledge across all involved (including vendor and implementation partner staff). it would appear that both studies identify lack of retentive capacity as a major problem in ramp-up and integration. this problem would not be as apparent in the earlier transfer phases, we speculate may be endemic to the enterprise system life-cycle. again the findings suggest a research focus on the recipients of es knowledge and their ability to retain that knowledge as well as unlearn legacy system behaviors. finally our counts indicate the importance of organizational context, support from management in both resources and endorsement of the es, in enabling knowledge transfer. conclusion while szulanski's research empirically tested the constructs, the data collection and analysis methods followed in the present study diverged markedly from szulanski's approaches, with uncertain effect on the validity of the findings as they are presented above. szulanski's method was based on a detailed questionnaire with a multi-item ordinal scale limiting the responses particular to each construct. in contrast the present study generated qualitative data from a single open-ended question. szulanski's measurement items for each construct were modified for applicability to the present study's context and data collection approaches. no explicit testing of the internal or external validity of the modified scale used in the present data analysis method was undertaken. the effects, if any, that these factors may have had on the validity of the coding instrument for classifying the openended qualitative responses from the survey are unknown. because of the care taken in making the modifications described in part 4 the effect is expected to be minimal. it is acknowledged that the delphi approach was deliberately chosen to generate issues important from the participants' perspectives. 122 aj1s special edition knowledge management december 2001 further aspects of the analysis method raise uncertainty regarding reliability. classification of the issues raised in the responses was inherently a subjective process open to analyst bias, since it involved the interpretation of qualitative data. to enable review by other analysts, raw data was coded to a level of detail that depicts the particular measurement items considered relevant in identifying each construct chosen for an issue. re-coding of the data set by a range of other analysts would reveal the degree to which the results are consistent when the process is repeated. there was no comparative method for rating stickiness strength or weakness, nor the degree of a particular stickiness origin's influence. in summary, further analysis of validity and reliability of the analysis methods would be appropriate before generalising to other es knowledge transfer situations. nonetheless, the findings as presented are interesting in two respects. firstly, they reveal important insights into the variables that may have directly or indirectly presented barriers to knowledge transfer in the es implementations responsible for the source data. secondly, the pattern of apparently significant barriers at each transfer stage suggests interesting similarities and differences when compared with szulanski's findings of intra-firm transfer stickiness. references andreu, r. & ciborra, c. (1996) "organisational learning and core capabilities development: the role of it," jour, nal of strategic information systems vol 5, pp 111-127. arrow, k. (1971) classificatory notes on the production and transmission of technical knowledge, amsterdam, holland. bennett, c. & timbrell, g. (2000) "application service providers: will they succeed?" information systems frontiers, vol 2, no 2, pp 195-211 chan, r. (1999) "knowledge management for implementing erp in smes," proceedings of the 3rd annual sap asia pacific conference, singapore. chang, s., gable, g., smythe, e. & timbrell, g. (2000) "methods for distilling key issues using a delphi approach", proceedings of the llth australasian conference on information systems, 6-8 december, brisbane, australia. cohen, w. m. & levinthal, d. (1990) "absorptive capacity: a new perspective on learning and innovation," administrative science quarterly, vol 35, no 1, pp 128-152. davenport, t. h. (2000) mission critical: realizing the promise of enterprise systems, boston, ma, hbs press. davenport, t. h. & prusak, l. (1998) working knowledge: how organizations manage what they know. boston, ma. harvard business school press. feeny, d. & willcocks, l. (1998) "re-designing the is function around core capabilities", long range planning, vol 31 no 3, pp 354-367. gable, g. g., scott, j. & davenport, t. (1998) "cooperative ews life-cycle knowledge management", proceedings of the ninth australian conference on information systems, 29 september 2 october, sydney, australia, pp 227-240. inkpen, a. c. (1996) "creating knowledge through collaboration", california management review, vol 39, no l,pp 123-140. inkpen, a. c. & dinur, a. "the transfer and management of knowledge in the multinational corporation: considering context". working paper 98-16. carnegie bosch institute. 1998. klaus,h., rosemann, m. & gable, g.g. (2000) "what is erp?", information systems frontiers, vol 2, no 2, pp 141-162. kogut, b. & zander, u. (1992) "knowledge of the firm, combinative capabilities and the replication of technology", organization science, vol 3, no 3, pp 383-396. levitt, b. & march, j.g. (1988) "organizational learning," annual review of sociology, vol 14, pp 319-340. lippman, s. a. & rumelt, r. p. (1982) "uncertain instability: an analysis of interfirm differences in efficiency under competition", bell journal of economics, vol 13, pp 418-438. nonaka, i. (1994) "a dynamic theory of organizational knowledge creation", organization science, vol 5, pp 14-37. shannon, c. & weaver, w. (1949) the mathematical theory of communication, university of illinois press, chicago, il. szulanski, g. (1995) appropriating rents from existing knowledge: intra-firm transfer of best practice,umi dissertation, fontainebleau: insead. szulanski, g. (1996) "exploring internal stickiness: impediments to the transfer of best practice within the firm," strategic management journal, vol 17: special issue, pp 27-43. 123 ajis special edition knowledge management december 2001 szulanski, g. (2000) "the process of knowledge transfer: a diachronic analysis of stickiness", organizational behavior and human decision processes, vol 82, no 1, pp 9-27. timbrell g. & gable, g. (2001) "the sap ecosystem: a knowledge perspective", proceedings of the information resources management association international conference, 20-23 may, toronto, canada zaltman, g., duncan, r. & holbek, j. (1973) innovations and organizations, wiley, new york, usa. 124 ajis special edition knowledge management december 2001 appendix one modified codification instrument constructs (szulanksi, 1995a) measurement items causal ambiguity "depth of knowledge." ambiguity about what factors lead to a successful outcome and/or how they interact. ambiguity about how components of the knowledge work when implemented in a different environment. limits of the knowledge or practice were not fully specified. not clear what skills, resources and prerequisites are necessary to successfully perform the practice. lack of adequate documentation exists regarding operating procedures. procedures do not describe exactly what people actually do. nobody fully understood the concept. unproven knowledge "degree of conjecture on the utility of the transferred knowledge." lack of a proven record of the past usefulness or success of the knowledge or practice. the knowledge or practice is not seen as contributing significantly to the organisation's successful outcomes. perceived lack of robustness or legitimacy of knowledge or practice. source lacks motivation "motivation of the source unit to support the transfer". source did not measure own performance. unwilling to share knowledge and what its knowledge limits are. poor efforts at communication, planning, documentation, implementation, training, troubleshooting. unwilling to devote time and resources to the transfer. source perceives a lack of adequate rewards for transferring knowledge. source not perceived as reliable "degree to which the ... [source] is perceived as reliable". perceived lack of expertise and trustworthiness. validity of source questioned (whether speaking from experience or is relaying what someone else has told them). source unable to accommodate needs of recipient into the knowledge or practice being transferred. perceived hidden agenda. perception that source is unable to achieve superior results. perceived instability. lacks necessary resources to support the transfer. lacks a history of successful transfers. advice or demonstration is challenged or resisted by recipient. recipient lacks motivation "motivation of the recipient unit to support the transfer". recipient did not measure own performance. recipient did not adequately communicate its needs to the source. recipient did not plan and implement adequate supporting arrangements. recipient did not assign personnel full-time to the transfer. recipient did not assign personnel to be trained. recipient did not troubleshoot. recipient did not ensure that people know their jobs & consent to continue doing them. users or staff reluctant to accept knowledge from outside. not invented here syndrome. recipient lacks absorptive capacity "ability of the recipient... to identify, value and apply new knowledge." inability to value, assimilate or apply transferred knowledge due to limited preexisting stock of knowledge. lack of a common language/terminology to deal with the knowledge or practice. lacks vision of what trying to achieve through the transfer. 125 aj1s special edition knowledge management december 2001 lacks information on state of the art practice. lacks clear division of roles and responsibilities to implement. lacks necessary skills to implement. recipient lacks the technical and managerial competence to absorb the knowledge or practice. recipient does not know who can help solve specific problems or provide specific information. recipient did not cope with the training methodology. recipient did not develop adequate equipment and processes. recipient did not hire the right people. relative inexperience of recipient. recipient unable to understand a different approach. inability to locate relevant process owner. recipient lacks retentive capacity "ability of the recipient... to routinize the use of new knowledge." lacks ability/persistence to retain transferred knowledge. lacks ability to institutionalise use of the new knowledge. discontinued use of the transferred knowledge. reversion to the status quo. personnel were not periodically retrained. recipient lacks mechanisms to detect malfunctions of practice. personnel uncertain how they will be rewarded for good performance. lack of a clear focal point in the organisation for the knowledge. ability to unlearn old routines and superceded knowledge. barren organisational context "degree to which the organizational context supports the development of transfers." organisational context hinders knowledge transfers. lack of detailed, timely and comparable performance measures. standard policies about the practice not enforced. lack of constant pressure to improve performance. difficult to justify time and money spent assimilating the transferred knowledge. low acceptance for copying and adapting knowledge or practices. loss of status if ask for help or need to learn from someone else. external help or advice not valued or encouraged. specific knowledge transferred was not valued. lack of opportunities for transfer of the needed knowledge. ^^^ arduous relationship difficulties with the "ease of communication and intimacy of the relationship". lack of easy or open communication btw source and recipient. laboriousness of interaction. language problems. collaboration avoidance. 126 the australasian journal of information systems (ajis) began in 1993 and was based on the simple realisation that there was a need for an academic ‘outlet’ for is research in australia australasian journal of information systems volume 14 number 1 june 2007 3 ajis editorial ajis publishes high quality contributions to theory and practice in the global information systems discipline. while research is global, ajis contributions should have an emphasis on the australasian context of the work. the journal welcomes papers on research and conceptual development based in a very wide range of inquiry methods, ways of thinking and modes of expression. in this edition the ajis featured theme examines the ranking of academic journals, including this one. the debate on journal ranking has been sparked by the australian government's push to change the way research is funded by its introduction of a research quality framework. the topic is current in the socio-political aspect of the discipline, although it is on the periphery of the information systems knowledge domain. the debate, however, opens up the bigger issue of the role of information systems in knowledge creation and mobilisation (e-research). and that is a core topic in our field. scholarly publishing is a subsystem in that activity and the editors of ajis have a continuing interest in the potential for information systems in e-research. the obvious first step was to put ajis online (just google ajis). it is now a fully open-access journal with web-based workflow management for article submission and tracking. the australian computer society is managing the hosting of ajis through its digital library initiative with the public knowledge project at simon frazer university and the university of british columbia. tom worthington deserves our thanks for setting up the initiative as does steve elliot for his support in the acs. all back issues have now been loaded into the system. this was a large task supported by the university of canberra thanks to marina materne. the journal being online improves access to articles and more work is being done now to improve indexing. of course, the more ajis is used the better. next we are looking into a range of eresearch ideas that can enhance scholarly work. in particular we are exploring the possibilities of: public reviewing of offered articles (like the british medical journal) and discussion forums associated with articles; curation of data (in concert with other repository projects) and research instruments that support articles but that are rarely published; an is concept management system (perhaps a wiki) to facilitate ongoing development, and coherent usage, of is concepts; 'living' articles; explicitly positioning ajis with other journals (eg, those that address specific fields, approaches or aspects of the is domain). in short, we aim to address the journal from a knowledge management perspective. australasian journal of information systems volume 14 number 1 june 2007 4 future featured themes include enterprise architecture, systems governance, design science and highlights from the 2007 australasian conference on information systems. please approach the editor with additional suggestions. my thanks to the co-editor, john campbell, and the paper authors. thanks too to the school of information sciences and engineering, university of canberra, for its significant ongoing support. craig mcdonald associate professor of information systems university of canberra, act, australia email: craig.mcdonald@canberra.edu.au june 2007 editorial board thomson scientific (isi) impact factor. using existing ranking lists polling eminent scholars stage one: identification of existing rankings stage two: initial ranking list drawn up stage three: feedback on the initial ranking list stage four: information and finalisation of ranking list level a journals level b journals premier professional journals abstract introduction the pending australian research quality framework (rqf) methodology sourcing and procuring the data examining the data for correctness results rq1 what journal outlets are used by australian academics to publish is-related research? rq2 which journals have been most preferred by australian academics for article publication as measured by the total number of articles published in each journal? conclusion limitations and future research socio-pragmatic constructionism social constructionism pragmatism socio-pragmatic constructionism representation simplification pragmatic orientation syntax semantics pragmatics implications for practice implications for theory passive dimension (perceptions); active dimension (conceptions) the aesthetic dimension is not characterized as “plug and play”. the aesthetic dimension is not described by graphical user interfaces. aesthetics occurs when the system elevates the work to a new level of effective vision. aesthetics may occur on different levels or from different perspectives. 1. rational goal seeking (intelligence(design(choice) 2. happenstance rationalization (design(choice(intelligence) 3. existential introversion (choice(intelligence(design) 4. deferred design (intelligence(choice(design) 5. situational sensemaking (design(intelligence(choice) 6. existential heroism (choice(design(intelligence) constructs output performance outsourcing experience contract establishment the feedback effect business strategy it strategy firm-wide active involvement long term focus meeting of the minds clarity and consistency management skill and capability alignment facilitation processes organisational structure organisational culture communication it as an organisational tool organisation background information factors that promote alignment people factors process factors organisational factors high level factor comparisons alignment of business & information technology strategy a cross industry australian study part 1: background part 2: business strategy part 3: information technology strategy part 4: factors that promote the alignment between business and information technology strategy documentation interviews researcher bias ajis vol. 8 no. 2 may 2001 assessing software quality through visualized cohesion metrics timothy k. shih* ming-chi lee** teh-sheng huang* lawrence y. deng* *dept. of computer science and information engineering tamkang university taiwan, r.o c e-mail: e8190070@lkgis.tku.edu.tw **dept. of business administration national pingtung institute of commerce taiwan, r.o.c abstract cohesion is one of the most important factors for software quality as well as maintainability, reliability and reusability. module cohesion is defined as a quality attribute that seeks for measuring the singleness of the purpose of a module. the module of poor quality can be a serious obstacle to the system quality. in order to design a good software quality, software managers and engineers need to introduce cohesion metrics to measure and produce desirable software. a highly cohesion software is thought to be a desirable constructing. in this paper, we propose a function-oriented cohesion metrics based on the analysis of live variables, live span and the visualization of processing element dependency graph. we give six typical cohesion examples to be measured as our experiments and justification. therefore, a well-defined, well-normalized, well-visualized and well-experimented cohesion metrics is proposed to indicate and thus enhance software cohesion strength. furthermore, this cohesion metrics can be easily incorporated with software case tool to help software engineers to improve software quality. keywords: software metrics, cohesion, software quality, live variables introduction the quantitative measurement of software has been defined as one of the functions of engineering disciplines. the establishment and the use of software measurement still make a great deal of extensive discussion. software production is often out of control due to the lack of measurement [demarco, t. ( 1982)]. without software metrics, software would be error prone, expensive and with a poor quality. therefore, suitable software measures are essential for software development. modularization of software represents a significant software engineering activity, which continues to receive considerable research attention. module cohesion is defined as a quality attribute, which seeks for measuring the singleness of purpose of a module. cohesiveness is a measure of an individual module's internal strength, which is the strength of the interrelationship of its internal element [martin, j. mcclure, c. (1983)]. module cohesion indicates how closely a module's internal components are related to others. cohesion is a critical assessment of software design quality. a software module with low cohesion will be error prone. fifty percent of the highly cohesive procedures are fault free, whereas only eighteen percent of procedures with low cohesion are fault free [fifty_percent, 1998]. cohesion is an attribute that can predict properties of implementations such as "ease of debugging, ease of maintenance, and ease of modification" [yourdon e. and constantine l.l. (1979)]. cohesiveness is one of the possible measurements for program modifiability [stevens, w., myers, g and constantine, l. (1974)]. a modifiable program is made up of modules that have high cohesiveness. therefore, well-defined and well-proved cohesion measures can support software engineer to design a higher cohesion module and enhance software quality, maintainability and reusability. this implies that the software quality can be improving by maximizing the degree of module cohesion. however, stevens, myers and contantine first define the notion of cohesion (smc cohesion), on ordinal order of seven levels that describes the degree of modules. the factors are functional, sequential, communicational, procedural, temporal, logical and coincidental [stevens, w., myers, g and constantine, l. (1974)][yourdon e. and constantine l.l. (1979)]. the cohesion strength of a specific procedure is increasing from coincidental to functional. a highly cohesive software module has one basic function and is indivisible [fenton, n. e. (1991)]. and, the coincidental is the least desirable but \hefunctional is the most desirable cohesion level. in the past decade, a few researchers work on cohesion measures. biemen's cohesion measures is based on program slicing. a program slice is a portion of program text that affects specified program variables [bieman, j. m. and ott, l. m. (1994)]. the program slice based measures depends on data flow and control flow of a procedure [bieman, j. e. and kang, b.k. (1998)]. for complex program functionality, the program slicing method can produce slices that are either too large to understand, or too simple but not significant to comprehend [lucia, a.d and fasolino, a.r. (1996)]. on the other hand, the mechanism does not consider the global data variables effect and the analyzed scope is not a formal module scope. lakhotia developed a logical analysis method to investigate cohesion types based on logical analysis [lakhotia, a. (1993)]. although, the result of the logical analysis meets the smc cohesion levels logically. but, it does not provide numerical system to compute cohesion strength. ott and thuss's approach is based on slice profile and processing element flow graph to express four types of module cohesion [ott, l. m. and thuss, j. j. (1989)]. but, this approach does not consistent 15 ajis vol. 8 no. 2 may 2001 with the smc cohesion and the fenton's cohesion spectrum. also, this method does not express a numerical system how to compute module cohesion. fortunately, the well-known henry-kafura measurement measures the relationship between procedures based information flow [henry, s. and kafura, d. (1981)]. but, it fails to measure the cohesion attribute of intra-procedure. live variables describe the extent of variables to be refereed within a module, while variable span captures the range of variable to be used in a module [conte, s. d., dunsmore h. e. and shen, v.y. (1986)]. generally, instance variable is used to derive higher-level inference on cohesion [metr_dis, 1998]. module cohesion aims to measure how tight between output processing elements [ott, l. m. and thuss, j. j. ( 1989)][bieman, j. e. and kang, b.k. (1998)][bieman, j. m. and ott, l. m. (1994)]. in this paper, we propose a function-oriented cohesion metrics, which is based on an analysis model of live variable semantics. we examine the function-oriented cohesion metrics (focm) for each output function, the most tightest function-oriented cohesion metrics (mtfocm), the least tightest function-oriented cohesion metrics (ltfocm), and the average tight functionoriented cohesion metrics (atfocm) of a specific procedure. in general, we always strive for a high cohesion. although the mid-range of the cohesion spectrum is often acceptable [pressman, r.s. (1988)]. and, a module may exhibit more than one type of cohesion. however, we will show that the proposed metrics do not only closely match smc cohesion and fenton's cohesion strength spectrum [fenton, n. e. (1991)], but also meet nonlinear cohesion scale of cohesion that was stressed by pressman and sommerville in [pressman, r.s. (1988)][sommerville, i. (1996)]. the main contributions of this paper are that we propose a formal and visualize analysis model, scientific basis, well-normalized, wellexperimented, and well-evaluated function-oriented cohesion measures to improve software quality. the rest of the paper is organized as follows. we address basic works about live variable semantics and formal definitions in section 2. we examine visualized analysis model and the proposed cohesion metrics in section 3. section 4 gives six typical cohesion procedure examples to make empirical study. to discuss and analysis our experimental results is expressed in section 5. finally, in section 6, the conclusion and future research are presented and discussed. basic works and definitions a module is a contiguous sequence of program statements that are bounded by boundary elements, and has an aggregate identifier [yourdon e. and constantino l.l. (1979)]. a module is usually defined as a segment of codes that is compilable independently [conte, s. d., dunsmore h. e. and shen, v.y. (1986)]. however, many researchers define a module to be a compilation unit of code, a view of program, or a procedure. by functionality, the processing elements in a procedure could be categorized into the following four categories: • input variable (iv): a set composed of the variables that are the input arguments to a procedure. • internal variable (inv): a set of local variables of a procedure. • global variable (gv): a set of global variables. • output variable (ov): a set of variables that are the output functions of a procedure. in general, variables in an individual module can be in assignment statement, initial statement, decision predicate, output function and iteration constructor, boolean semantics which could construct the main body of the whole procedure. a module will contain specific function semantics. in practice, it will be the key issue to develop a suitable model to construct the overall reference scenarios of the function in the procedure scope. live variables of a statement in module are the processing elements that are referenced in the statement. for sake of explaining the concept of live variables, a procedure example is given as follows: 16 ajis vol. 8 no. 2 may 2001 1 procedure sumandprod(n:integer; var sum; prod:integer) 2 var i:integer 3 begin 4 sum:=0; 5 prod:=l; 6 for i:=l to n do begin 7 sum:=sum+i; 8 prod:=prod*i; 9 end 10 end figure 1. a procedure example as sumandprod listed in figure 1, sum in line 4 is a live variable. both / and n are live variables in line 6. on the other hand, sum is only referred at two statements in the procedure. should software engineers concern sum in other statements other than line 4 and 7? the answer is "yes". however, software engineers constantly have to keep in mind where sum may be referred between statements 4 and 7. software engineers always realize that in each iteration sum has a different status in the for_loop, sum will be incremented, although the statements do not refer to sum in line 5 and 6. finally, the summation operation will exit. similarly, prod has the same scenario like sum. and, comprehending software artifacts is an important software engineering activities. a significant amount of time of a software engineer is spent in looking at the source code to discover information during testing, review and code inspection [canfora g and lakhotia a.(1999)]. we also know a programmer must be aware of the status of a number of data items during the programming process [conte, s. d., dunsmore h. e. and shen, v.y. (1986)]. thus, the more data items what a programmer must keep track of when constructing a procedure, the more difficult it is to construct. based on the above issues, live variables of a statement are not limited to the number of variable reference in that statement. in the past years, there are several possible scopes of live variables to be defined inevitably [curtis b. ( 1981)][ conte, s. d., dunsmore h. e. and shen, v.y. (1986)]. the first is the live variable computes from the beginning of a procedure to the end of the procedure. according to this definition, the computation is simple, but this violates the live variable abstract. the second is at a particular statement, only if variable is referenced a contain number of statements before or after that statements. depend on the second definition, we may calculate lifespan. but there is no agreement on what "certain number of statements" should be and no successful use has been reported [conte, s. d., dunsmore h. e. and shen, v.y. (1986)]. the third one is from its first reference to its last reference within a procedure. however, the third definition not only meets the principles of live variable abstraction, but can be counted algorithmically as well. we believe that the third is a suitable domain for computing module cohesion. therefore, we will adopt the third as the definition of live variables in this paper. for precise definition and helping software engineer to understand software easily, we use mathematics to define the live variables and use live span to visualize a live variable scope. therefore, we denoted the lifespan of a variable to be the reference domain that begins at the first occurrence and extends through the last occurrence. definition 1: the lifespan of a variable in a procedure is a set, denoted by ls(var_nam). the elements of the set are the name of the variable from the first (/-th line) to the last (/'-th line) referenced, ls(var_nam)={var_nami,var_nami+i ,var_nam^. and, the size of lifespan, | ls(var_«am) | =j-i+\. as an example in figure 1, we investigate the live span of variable sum. the set ls(sum)={sum4jums,sum6^um?}, with a size of 4. the ls set also represents a mapping from the references of sum to different locations (i.e. lines 4,5,6 and 7). definition 2: the live variables (lv) of a specific procedure (sp) is the set union of the lifespan of each variables which belongs to iv, inv, gv or ov. we denote lv(sp)= u ls(var_«a/w/) where var_namt e iv u inv u gv u ovandien. in figure 1, we could construct lv'(sumandprod). first, we know the processing elements are iv={w}, ov={sumj>rod},gv=0, and inv={/} respectively. next, ls(inv)={i6j7j8} where /v/ielvn, ls(ovy={sum4jum!ijum6jum7j>rod}j>rod6j)ord7j)rods} where oveov, ls(gv)= 0 where gvegv, ls(/v)={a^(s} where we iv. finally, lv(sumandprocd)=ls(iv) u ls(ov) u ls(gv) u ls(/nv). for simplicity, we can describe the live variables and live span of the procedure as figure 2. 17 ajis vol. 8 no. 2 may 2001 line 4 5 6 7 8 lv(sumandprod) sum4 sum5j>rods sum6j}rodgj 6jj6 sum7j)rod7j7 prodsjs count 1 2 4 3 2 12 live span — suztl, prod, i, figure 2. live variables and live span of sumandprod the live span and live variables of figure 1 are illustrated in figure 2. for instance, sum is firstly referenced in line 4, the last reference is in line 7. according to definition 1, the software engineer have to keep the reference of sum from line 4 and line 7, although sum does not appear in statement 5 and statement 6. therefore, line 4 to line 7 are the live span of variable sum and we use a symbolic "[" to describe the live span of a variable. together, a live span could clearly be viewed as a visualized scope of live variables in a specific procedure. visualization and cohesion metrics in this section, we propose function-oriented cohesion measures that are based on the analysis of live variables and live span abstraction. according to the previous section, we know that not all variables involve the computing of output functions in a specific procedure. logically, a module is the function that processes input, internal and global variables to produce a result. the output functions in ov(sp) are on behalf of the result of the module. in fact, not all elements of live span of a variable contribute a result of the output function of the procedure. for instance, the ls(prod) consists of prod;, prodg, prod-/, and prods, in which no element influences the result of output sum. but bothprods andprodg may involve the value of the output function prod through the assignment statement. as of the result, prods andprodg are the function-oriented live variables of output function prod. similarly, variables i6, is, n6 are the function-oriented live variables of prod. both /,; and nf may change the value of the output function and compute the multiplication via i8 indirectly. therefore, function-oriented live variables of prod is a set, denoted {prods, i6, n6, prodg, is}. therefore, the folv and fols of the procedure sumandprod are shown in figure 3. however, it is not enough formal that we express function-oriented live variables and live span just rely on the above description. therefore, firstly we will define direct variable and indirect variable of an output function before giving the definitions of function-oriented live variables. definition 3. direct variable of an output function is an element of lv(sp), which may influence the result of the output function in the same statement. for instance, variable i7 contributes the value of the sum7 directly. thus, i7 is the direct variable of the output function sum7. according to definition 3, output variable surely is the direct variable by itself. line 4 5 6 7 8 folv '(sumandprod) sum4 prods i6j*6 sum7j7 prodsjs count 1 1 2 2 2 8 fols — suin^ ^ sum, -prodi rle nfi rlprod. li, figure 3. the folv and function-oriented live span of sumandprod definition 4. indirect variable of an output function is an element of lv(sp), which may decide the execution of the output function or perform the result of the output function via direct variables. for instance, variables 16 and n6 of iteration statement in line 6 contribute the value of the sum7 indirectly. so i6 and n6 are the indirect variable of the output function sum7. definition 5. the function-oriented live variables of an output function in the specific procedure is a set, denotes folv(ov,)={/v e lv(.sp) | iv is direct or indirect variable of ov,; ov,eov(sp)} definition 6. the junction-oriented live span of an output function in the specific procedure is a set, denotes fols(ov,)={/v e ls(sp) | iv is direct or indirect variable of ov,; ov,eov(sp)} 18 ajis vol. 8 no. 2 may 2001 according to definition 5 and 6, the function-oriented live variables and function-oriented live span of sum and prod are depicted in figure 4. variables sum i n prod folv(swm) sum 4, sum7 16, f 7 n6 5 folv(s«ro)nflov(pro</) /6 nt 2 ¥olv(prod) /«,/« n6 prodsj)rodg 5 figure 4. the folv of sum, prod and their intersection in figure 4, the folv of the output function sum is the set {sum4,16, n6, sum7,17}. the size of the folv(sum) is 5. on the other hand, the folv of the output function prod is the set of {prods, i6, n6,prod7,18}, the size of the folv(s«»j) is 5. we know each element in ¥olv(sumandprod) will appear either in folv(iwm) or in folv(prod). which is defined by folv(sum) u folv(prod) as shown in figure 3. more importantly, the function-oriented live span could help software engineer to friendly visualize the scope of function oriented live variables in the specific procedure sumandprod. in general, the size of the folv(ov,) is the number of elements which belongs to lv(sp), and influence the value of o\>j directly or indirectly. then, the size of folv(ov,) implies the number of live variables in the specific procedure, shall contribute to the result of the output function ov. more precisely speaking, the proportion of |folv(ov,)| in |lv(sp)| can be viewed as the cohesion strength which just restricts on ov, in a module. therefore, the junction-oriented cohesion measure restricts on ov, is defined as follows: focm(ov,)= | folv(ovj)| / \lv(sp)\ for each ov, e ov(sp) from the perspective of the cohesion, the common shared function-oriented live variables of sum and prod are the most critical elements. module cohesion aims to measure how tight between output procession elements. and, the relation levels of output function pairs determine a cohesion level of a module [bieman, j. e. and kang, b.k. (1998)]. we define the most light function-oriented cohesion measure of the specific procedure (sp) as follows: mtfocm(5p)=|nfolv(ov,)|/|lv(s/>)| for each ov, e ov(sp) in a specific procedure, some elements of live variables really may not involve in influencing the result of output functions. this implies that not all of live variables have function-oriented relation with each ov. obviously, the live variable vl(sp) excluding the function-oriented live variable folv(sp) is a subset of live variables, which is defined as vixs/?)-folv(.sp). therefore, we define the least tight function-oriented cohesion measurement as: ltfocm(s/7>= | lv(sp) u folv(ov,)| / \lv(sp)\ for each ov, € ov(sp) however, it is also an important reference point for software measures that the average strength of all live variables involves the results of output functions. we propose the average tight function-oriented cohesion measurement of the specific procedure as follows: atfocm(s/>)= | u folv(ov,) | / 1 lv(sp) \ for each ov,€ ov(5p) consequently, we proposed four function-oriented cohesion measures in a specific procedure. the values of the four proposed cohesion measures are in the range between 0 and 1. therefore, the numerical systems of the function-oriented cohesion measures is a well-normalized cohesion. practically, it is necessary to express the folv using a dependency graph to visualize the function-oriented relationships between processing elements for helping software engineer to easily comprehend the folv. definition 7. the function-oriented live variables dependency graph of an output function ov in a specific procedure is a directed graph, denoted as folvdgov(v,e), where v is a finite set of elements called vertices which consist of folv(ov), e is a finite set of elements called edge which is defined as (/v, ov), where /v is direct or indirect variables of ov, oveov(sp) and /v€folv(ov). we use directed arrow with 'id' and '</' to describe the indirect or direct function-oriented live variables of an output function, respectively. furthermore, a function-oriented dependency graph of a specific procedure may express the semantics of direct 19 ajis vol. 8 no. 2 may 2001 or indirect dependency between function oriented live variables and output functions. definition 8. the function-oriented live variables dependency graph of a specific procedure is a directed graph, denoted as folvdg^(n,e)= ufolvdgov(n,e), for all oveov(sp). figure 5. the folvdgs of sum, prod and sumandprod folvdgs of sum, prod and sumandprod in figure 5 are the visualizations of folv of sum, prod and sumandprod respectively. it shall be very useful that the visualizations of folv may help software engineer to investigate the dependency relations between folv and output functions. typical examples and experiments in this section, there are six typical cohesion procedure implementations to be given to make an empirical study using our proposed cohesion measures. the purpose of measuring the distinctive cohesion procedure examples is to check the results of experiment of the proposed function-oriented cohesion measurements and to estimate whether they are compatible with the cohesion strength spectrum. six specific procedure implementations are coincidental, logical, procedural, communicational, sequential, and functional cohesion examples. we believe that the experiments will not affect the completeness of the empirical study although the six implementations will not include the temporal one. a system initialization module can be considered as a typical temporal cohesion. cohesion is a measure of the strength of the interrelatedness of statements within a module. in general, there are many definitions of the seven levels of cohesion in [fenton, n. e. (1991)][modularity 1999] [cohesion 1999][intramodule cohesion 1999]. a functional cohesion is the strongest cohesion in the cohesion strength spectrum. therefore, the functional cohesion is the most desirable. coincidental cohesion is to estimate whether a module performs more than one function, and whether there are unrelated [fenton, n. e. (1991)]. in the coincidental cohesion example, there is no significant relationship between the output elements. it is only by coincidence that they are together. similarly, it is hard to describe module purpose. however, a typical coincidental cohesion example and its abstract function diagram are given in figure 6. 01 procedure sumandprod(n: integer; var sum, prod: integer; arrl,arr2:int_array); 02 var i: integer; 03 begin 04 sum:=0; 05 prod:=0; 06 for i:=l to n do 07 sum:=sum+arrlfl]; 08 for i~l to n do 09 prod:=prod* arr2[i]; 10 end module 0 function inarrl i n arr2 abstract function diagram information flow diagram figure 6. a typical coincidental cohesion implementation by definition 1 and definition 2, we know and \ix(sumandprod)\=n. we express lv and ls of the coincidental cohesion example in figure 7. on the other hand, tolv(sum)={sum4j6jf6jum7,arrl7j7}, folv(proay={prodsjs^sj)rod9,arr29j9} and nfolv(ov)={lv e folv(smffi) and iv e ¥olv(prod)\ iv e lv }=0. they are shown in figure 8. in figure 9, the folv of sum, prod and their intersection are illustrated. foldgs of output functions and coincidental cohesion example are depicted in figure 10. 20 ajis vol. 8 no. 2 may 2001 line 4 5 6 7 8 9 lv(sumandprod) sum4 sumsjjrod; im orod j jvov* or or 0 sum 7j)rod7,arrl 7^7 prodsjm prod9,arr29j9 count 1 2 4 4 3 3 17 ls psum. sum sunrv sum, prod prod. prod, prodg -prod, r-i^ na i, am, i. no _i9 arr29 figure 7. the lv, ls of a coincidental cohesion example line 4 5 6 7 8 9 folv(sumandprod) sum< prod5 16^6 sum arrl j7' 7»*7 isn/, prodg,arr2gjo count 1 1 2 3 2 3 12 fols psum. ^ sum. -prodj — pro dp r1* n« 1, arrl , ib n. -i, arrz, figure 8. the folv and fols of a coincidental cohesion example variables sum i n arrl arr2 prod folv(s«ro) sum 4, sum 7 i6j7 n6 arrl 7 6 folv(swffi) r\folv(prod) 0 folv(prod) i*j9 n8 arr2i) prodsj)rodg 6 figure 9. the folv of sum, prod and their intersection figure 10. the folvdgs of output functions and coincidental cohesion example according to the function-oriented cohesion measures presented in section 3, the values of function-oriented cohesion measures of coincidental cohesion example are given as follows: focmtsuroh folv(smm) | / 1 lv(sumandprod) |=0.353 focm(prod)= \ folv(prod) \ i \ lv(sumandprod) (=0.353 mt¥ocm(sumandprod)=\r£olv(ov)\/\lv(sumandprod)^=q ltfocm(sumandprod)= \lv(sumandprod) ufolv(ov) | / 1 lv(sumandpmd) (=0.29 avfocm(sumandprod)=\u¥olv(ov) \ i \ lv(sumandprod) (=0.71 in this example, there are no common processing elements used to produce both output functions. and, the value 21 ajis vol. 8 no. 2 may 2001 of mtfocm of the coincidental cohesion is equal to zero. the coincidental cohesion type is the lowest cohesion level. logical cohesion is to check whether the module performs more than one function, and whether there are related logically [fenton, n. e. (1991)]. in a logical cohesion example, the module performs some related functions. one or more of them are selected by calling module. in other words, there is a dynamic function selection in the module. however, a typical logical cohesion example and its abstract function diagram are shown as follow. 01 procedure sumandprod(n: integer; var sum, prod: integer;arrl,arr2:int_array); 02 var i, j,flag: integer; 03 begin 04 sum:=0; 05 prod:=l; module06 if(flag>l) 07 for i:=l to n do begin 08 sum:=sum+ arrl [i]; 09 else 10 forl:=l to n do begin 11 prod:=prod* arr2[i]; 12 end by 0 function -•*• icontrol path abstract function diagram in an-2 information flow diagram figure 11. a typical logical cohesion implementation definition 1 and definition 2, ={.s«m4,sm/w5,prod5,sm7hsjto^ iojfioj>rodn,arr2]jj]i }, and \lv(sumandprod)lf=24, which are diagramed in figure 12. in other way, we may have folv(sum)={sum4jlag6j[7fl7,sums,arrlgjs}, and |folv (sum)\=7, folv(prod)={prodsjlag6j<>fl9j)rodi0,arrl10j,0} and |folv(pnw/)|=7, and nfolv(ov)={lv € folv(sum) and iv e tolv(prod)\ iv e lv}={flag7} and |nfolv(ov)|=l, and they are shown in figure 13. in figure 14, the folv of sum, prod and their intersection are illustrated. foldgs of output functions and logical cohesion example are illustrated in figure 15. line 4 5 6 7 8 9 10 11 lv(sumandprod) sum4 sum^prods sum6.prod6jlags sum7j>rod7j7jv7 sum nrod arrl j jvosf7 oi o**or o prodg, i9jj9 prodiojtqjniq prod,,,arr2nj[ll count 1 2 3 4 5 3 3 3 ls ^suitl4 sum2 sum. sum7 ^sume -prodj prode flage prod, prods prod. prod,0 lprodn -i, k, ia arrl 8 n8 i» n, i,o nio _ i,, arr2u 24 | figure 12. the lv and ls of a logical cohesion example line 5 6 7 8 9 10 11 12 ?olv(sumandprod) sum 4 prod} flag6 i7jj7 sum8,arrl8j8 linjfm prod,harr2,,j,1 count 1 1 1 2 3 0 2 3 fols -sum^ -sume _prod3 fl«8« _prod,, -i, n, ib arrla ilo istio -i,, arrz,, 13 figure 13. the folv and fols of a logical cohesion example 22 ajis vol. 8 no. 2 may 2001 variables sum i n flag arrl arr2 prod folv(sum) sum^umg hjg n7 flag6 arris 1 folv(sum)nfolv(prod) fag6 1 folv(prod) i io> in nio flag6 arr2,i prods, prod a 7 figure 14. the folv of sum, prod and their intersection prods id /^l10 arr2. figure 15. the folvdgs of output functions and logical cohesion example according to the definitions of function-oriented cohesion measures, the values of function-oriented cohesion measures of logical cohesion example are given as follows: focm(sum)=\ folv(sum) | / 1 lv(sumandprod) (=0-29 focm(prorf)= | folv(prod) | / 1 lv(sumandprod) |=0.29 mtfocm(sumandprod)=\r£olv(ov) | / 1 lv(sumandprod) |=0.041 ltfocm(sumandprod)=\lv ufolv(ov)| / 1 lv(sumandprod) |=0.46 atfocm(sumandprod)=\ufolv(ov) | / 1 lv(sumandprod) |=0.64 in this example, there are some common elements that are used to decide which function to be requested logically. the value of mtfocm of the logical cohesion is equal to 0.041. from the viewpoint of cohesion strength, this logical cohesion is just little stronger than coincidental cohesion. however, the logical cohesion is still a low-end cohesion. procedural cohesion is to evaluate whether the module that performs more than one function, and whether the module is related to a general procedural effected by the software [fenton, n. e. (1991)]. this means that if each function in the module needs to execute following a specific order, then it has the strong procedure cohesion. explicitly, both functions are elements of some iteration or decision operations [lakhotia, a. (1993)]. in the procedural cohesion example, the module performs a series of functions related by a sequence of steps. in other words, a module consists of functions related to the procedural processes in a repetition or selection constructs. however, a \ypica\procedural cohesion example and its abstract function diagram are given as follows: 01 procedure sumandprod(n: integer; var sum, prod: integer;arrl,arr2:int_array); 02 var i: integer; 03 begin 04 sum:=0; 05 prod:=l; 06 forl:=l tondo 07 sum:=sum+arrl[i]; 08 prod:=prod*arr2[i]; 09 end modik arl i n m2 insamereptionor condition construct, without same inputs by figure 16. a typical procedural cohesion implementation definition 1 and definition 2, we know \lv(sumandprod)\=\4. we express lv and ls of the procedural cohesion example in figure 17. and, we may have folv(sum)={sum<j6jf6jum7,arrl7j7} with size of 6, ?q\x(prod)={prod5j6fl6j>rods,arr2sjg} with size of 6, and nfolv(ov)={lv e folv(sum) and iv e folv(prod)\ iv e lv }={i6m with size of 2, and it's 23 ajis vol. 8 no. 2 may 2001 visualized fols is displayed in figure 18. in figure 19, the folv of sum, prod and their intersection are presented. foldgs of output functions and logical cohesion example are shown in figure 20. line 4 5 6 7 8 lv(sumandprod) sum4 sumsj3rod5 sumsjjrodfjf ^ sum 7,prod7,arrl 7/7 prods,arr28j8 count 1 2 4 4 3 14 ls r-sum, sum5 "-sum? r-prods prod? i7 arr!7 prodg ig 3rr2g figure 17. the lv and ls of z procedural cohesion example line 4 5 6 7 8 folv(sumandprod) suw4 prods i6^6 sum 7 cirri jj-j7) 7t 7 prods,arr28js count 1 1 2 3 3 10 fols -sum,, -sum-. i— pnods [ i* n« j, arr!7 ^i-_'^*b iq ax*r2g figure 18. the folv and fols of a procedural cohesion example variables sum i n arrl arr2 prod folv(sum) sum4,sum7 i6j6 n6 arr!7 6 folv(5ww) nfolv(praj) /6 nt 2 folv(pmd) 1<j* n6 arr28 prods,prods 6 figure 19. the folv of sum, prod and their intersection prod5 sunl?f^sum7 figure 20. the folvdg of output function and procedural cohesion example by the definition of function-oriented cohesion measures, we can derive the values of function-oriented cohesion measures of coincidental cohesion example as follows: )=| folv(sum) \ i \ lv(sumandprod) |=0.428 focm(prod)= | folv(prod) \ i \ lv(sumandprod ) |=0.428 mtfocm(sumandprod)=\rfolv(ov) \ i \ lv(sumandprod) |=0.142 ltfocm(sumandprod)=\lv ufolv(ov)| / 1 lv(sumandprod) |=0-288 atfocm(swm/lnrfpro<f)=|ufolv(ov)| / 1 lv(sumandprod) |=0.7 1 2 in this example, a number of elements are involved in different activities. but the activities are sequential. the value of mtfocm of the procedural cohesion is equal to 0.142. this value is larger than the value of logical cohesion. from the cohesion strength perspective, procedural cohesion is a little stronger than logical cohesion. communicational cohesion is to indicate whether the module performs more than one function, and whether the module is on the same data [fenton, n. e. (1991)]. this means that if every function in the module operates on 24 ajis vol. 8 no. 2 may 2001 the same data. in the communicational cohesion example, the module performs a series of functions related by the same data. in other words, the output functions are used to operate on the same data. however, a typical communicational cohesion example and its abstract function diagram are given as follows: 01 procedure sumandprod(n:integer; var sum, prod: integerjarrl :int_array); 02 var i:integer; 03 begin 04 sum:=0; 05 prod:=l; 06 for i:=l to n do begin 07 sum:=sum+arr 1 [i]; 08 prod:=prod*arrl[i]; 09 end end ••» :seqeuncepath modilc 10 function i narr in same repition or condition construct, and with same inputs abstract function diagram information flow diagram figure 21. a typical communicational cohesion implementation according to definition 1 and definition 2, we know \jj(sumandprod)= {sum^umsj}rod}^um6j)rod6j6j^6^um7j)rod7,arrl 7j7j>rods,arrlgjs} and \lm(sumandprod)^=l4, which are depicted in figure 22. again, folv(sum)={sum4j6^/6^um7,arrl7j7} with size of 6, ¥olv(prodj={prodsj6^!6j)roda,arrlgjg} with size of 6 and nfolv(ov)={lv e folv(sum) and iv € ¥olv(prod)\ iv € lv}={i6jf6} with size of 2, and it's visualized fols is sketched in figure 23. in figure 24, the folv of sum, prod and their intersection are illustrated. foldgs of output functions and logical cohesion example are shown in figure 25. line 4 5 6 7 8 lv(sumandprod) sum4 sum;jjrod} sum6j)rod6j6 j/6 sum 7j)rod7,arrl 7j7 prod8,atrlgjg count 1 2 4 4 3 14 ls -sum* sums sum^ *— sum? |-prods prode ,i6 ntf prod, i7 |-arrl7 -prodg *-ib larrie figure 22. the lv and ls of a communicational cohesion example line 4 5 6 7 8 ¥olv(sumandprod) sum 4 prods sum7,arrl7j7 prod8,arrlgjs count 0 1 2 0 3 6 fols -sum* i, r-arrl, -prod,, lia l_ arrl 3 figure 23. the folv and fols of communicational cohesion example variables sum i n arrl prod folv(swm) sum^um7 i 6^7 n6 arr!7 6 polv(sum)rfolv(prod) h n6 2 folv(prod) itjg n6 arrlk prod5j>rodg 6 figure 24. the folv of sum, prod and their intersection 25 ajis vol. 8 no. 2 may 2001 prod5 figure 25. the folvdgs of output functions and communicational cohesion example however, the values of function-oriented cohesion measures of coincidental cohesion example can be presented as follows: focm(s«m)=| folvcswm) | / 1 lv(sumandprod) |=0.428 ¥ocm(prod)=\ folv(prod) \ i \ lv(sumandprod) |=0.428 mtfocm(sumandprod)=\nfolv(ov)\ i \ lv(sumandprod) |=0.142 ltfocm(sumandprod)=\lv ufolv(ov)| / 1 lv(sumandprod) |=0.288 av?ocm(sumandprod)=\v?olv(ov)\ i \ lv(sumandprod) |=0.712 in this example, a number of elements are involved in different activities, but the activities are sequential. the value of mtfocm of communicational cohesion is equal to 0.142. the value seems to be the same as procedural cohesion. we have communicational cohesion, which is not significant stronger cohesion strength than procedural cohesion. sequential cohesion is to estimate whether the module performs more than one function. function dependency occurs in an order, which is described in the specification [fenton, n. e. (1991)]. generally, the output data from a function is the input for the next function in a module. in the sequential cohesion example, the module performs a series of functions related to i/o data. in other words, the functions of a module are related to perform different parts of a sequence of operations,_where the output of one function is the input to the next. however, a typical sequential cohesion example and its abstract function diagram are given as follows: 01 procedure sumandprod(n:integer; var sum, prod: integer;arr:int_array); 02 var i: integer; 03 begin 04 sum:=0; 05 prod:=l; module 06 forl:=l to n do begin 07 sum:=sum+arr[i]; 08 prod:=prod*sum; 09 end function / n abstract function diagram information flow diagram figure 26. a typical sequential cohesion implementation by the definition of live variables and function oriented live variables, we have uj(sumandprod)= {sum4,sumsj>rod5jsum6j>rod6j6.ff6jum 7j>rod7,arr7j7j)rods,sums} and $jf(sumandprod)\=\2i, we express lv and ls of the sequential cohesion example in figure 27. ¥oln(sum)={sum4j6fl6,sum7,arr7j7} with size of 6, ?g\jj(prod)={sum4j)rod5j6jj6,sum7j7j)rods£um8} with size of 9 and nfolv(ov)={lv e ¥oix(prod) and iv e folv(prod)| lv e lv }={sum4j6jf6,sum7,arr7j7} and | nfolv(ov)|=6, and their visualized fols are depicted in figure 28. in figure 29, the folv of sum, prod and their intersection are illustrated. foldgs of output functions and logical cohesion example are detailed in figure 30. 26 ajis vol. 8 no. 2 may 2001 line 4 5 6 7 8 lv(sumandprod) sum., sum^jjrodj sum6j>rod6j6jj6 sum7j)rod7,arr7j7 prodgjumg count 1 2 4 4 2 13 ls rsum, sum3 sums sum, -sumg rprod3 prods rl« ne prod, ll7 arr, uprodg figure 27. the lv and ls of a sequential cohesion example line 4 5 6 7 8 folv(sumandprod) sum4 prod) i6yn6 sum7,arr7j? prodgjumg count 1 1 2 3 2 9 fols sum* sum, sum. r-prod3 if ng li, arr, lprod. figure 28. the lv and ls of sequential cohesion example variables sum i n arr prod folv(sum) sum4^um7 i6j7 n6 arr7 6 folv(sum)rtfolv(prod) sum^um7 i6ji n6 arr7 6 folv(prod) sum4,sum7^umg itjj n6 arr7 prodsj)rods 9 figure 29. the folv of sum, prod and their intersection surn, sum, prodj sumg figure 30. the folvdg of output functions and sequential cohesion example focm(sum)=\ folv(sum) \ i \ lv(sumandprod) (=0.46 focm(prod)=\ folv(prod) \ i \ lv(sumandprod) (=0.69 mtfocm(sumandprod)=\nfolv(ov)\ i \ lv(sumandprod) |=0.46 ltfocm(sumandprod)=\lv ufolv(ov)| / 1 lv(sumandprod) (=0.31 avfocm(sumandprod)=\ufolv(ov)\ i \ lv(sumandprod) |=0.69 in this example, some elements involved in different activities. but the activities are sequential. the value of mtfocm of the sequential cohesion is equal to 0.46. the values show that sequential cohesion is a little stronger than communicational cohesion. functional cohesion is to check whether the module performs on a single function [fenton, n. e. (1991)]. the module is the one on which all of the elements contribute to exactly one function. in the functional cohesion example, the module achieves exactly one goal. however, a typical functional cohesion example and its abstract function diagram are given as follows: 27 ajis vol. 8 no. 2 may 2001 01 procedure sum(n:integer; var sum: integer; arr:int_array); 02 var i : integer; 03 begin 04 sum:=0; 05 for i:=l to n do begin 06 sum—sum + arr[i]; 07 end faction i n anfigure 31. a typical functional cohesion implementation rely on definition 1 and definition 2, it is easy to know that uv(sum)={sum4,sumsj5j^6^um6,arrgj6} with size of 7 and ~foln(sum)={sum4jij^5^um6,arr6j6} with size of 6. their visualized ls and fols are developed in figure 32 and figure 33 respectively. however, there is a single output function in the functional cohesion implementation. the live variables, live span, function-oriented live variables and function-oriented live span of the output function is the behalf of the functional cohesion. in figure 34, the folv of the single output function sum are depicted. foldgs of functional cohesion example are diagramed in figure 35. line 4 5 6 uj(sumandprod) sum6,arr6j6 count 1 3 3 7 ls t sum-5 r~is istj sum e ll^ arrc figure 32 the lv of a functional cohesion line 4 5 6 folv(sumandpmd) sum4 ism sum6,arr6j6 count 1 2 3 6 fols -sum,. ri, n, swne l i» arr6 figure 33. the folv of functional cohesion variables sum i n arr folv(sum) sum./, sum6 wt n5 arr6 6 figure 34. the folv of an output function sum 28 ajis vol. 8 no. 2 may 2001 sum4 figure 35. the folvdgs of output function and functional cohesion however, the values of function-oriented cohesion measures of functional cohesion example are given as follows: focm(s«ro)=| folv(sum) | / 1 lv(sum) |=0.86 mtfocm(sw/n)=|nfolv(ov)| / 1 lv(sum) |= 0.86 ltfocm(s«ro)=|lv(smm) ufolv(ov)) / 1 lv(sum) (=0.143 atfocm(smm)=|ufolv(ov)|/1 lv(sum) |=0.857 in this example, all elements involved in a single activity. the value of mtfocm of ihe functional cohesion is 0.86. functional cohesion is a stronger cohesion strength than sequential cohesion. the functional cohesion is the highest cohesion level. discussions and results cohesion of a module measures the strength of the relationship between the elements within the module: good modules should be highly bound [modularity (1999)]. cohesion can be evaluated on a one-dimensional scale [stevens, w.p. ( 1981)][ fenton, n. e. (1991)]. a scale of several points can identify different types of cohesion and giving their associated strength [modularity (1999)]. according to smc cohesion level and fenton's cohesion spectrum, we know that the cohesion strength spectrum can be illustrated as follows: in the previous section, we know the values of the cohesion of all experiments are in [0,1]. this also implies that the proposed cohesion measures are well-normalized. in particular, the values of mtfocm on the six typical cohesion examples from coincidental to functional are increasing between 0 and 1. in the other words, the cohesion strength of coincidental is the weakest type, but the functional cohesion strength is the strongest type. according to coincidental temporal cornrnuracational functional logical \ procedural bad good figure 36. cohesion strength spectrum the above experiments, it is not significant difference that the cohesion strengths of procedure cohesion and communication cohesion. more importantly, mtfocm does match the smc cohesion and fenton's cohesion strength spectrum. the quantitative values of the mtfocm experiments the levels of cohesion are sketched in figure 37. 29 ajis vol. 8 no. 2 may 2001 i07 .z! ~*1> h2 figure 37. the non-linear curve of cohesion levels strictly speaking, coincidental cohesion performs completely unrelated actions, which is the worst level of cohesion. in other words, a coincidental cohesion is an undesirable type of cohesion. logical cohesion performs function dynamically, which relies on other module calling by passing input arguments. the strength of logical cohesion is a little more than coincidental cohesion. so logical cohesion is still a lower level cohesion. procedural cohesion is also better than coincidental and logical cohesion because the procedural has more than one function being performed, which is related to one another in order to achieve the goal. but this type of cohesion is still on the lower level of cohesion. in our experiments, the cohesion strength of communication cohesion is not remarkable distinction to procedural cohesion. the cohesion strength of sequential cohesion is better than communicational cohesion because the output from one function is the input to the other functions, which is medium cohesion strength. functional cohesion performs exactly one action, which is a high-end cohesion. in other words, it is the most desirable type of cohesion. however, the curve of the quantitative values of mtfcom in figure 37 shows that low-end cohesiveness is more worsen than middle-range. the value of mtfocm of logical cohesion is just 0.041. from coincidental to logical cohesion, the cohesion strength is increasing very slowly. as a result, the difference of cohesion strength between coincidental cohesion and logical cohesion is not too significant. this implies that both coincidental and logical cohesions may be undesirable cohesion levels. the value of mtfocm of procedural or communication cohesion is 0.142. and, the curve of cohesion strength rises slowly from logical type to procedural or communication type. this means that those procedural and communication cohesion are low-end cohesion types. the value of mtfocm of sequential cohesion is 27.9% (more than communication). the curve of cohesion strength climbs up sharply at a turn around point of procedural or communication cohesion. this means that sequential cohesion is a high-end cohesion type. the value of mtfocm of functional cohesion is 27.4% more than sequential. the cohesion strength soars from sequential to functional. therefore, the functional cohesion should be a desirable cohesion type. furthermore, the nonlinear scale from coincidental cohesion to functional cohesion consists with the assertions of pressman and somerville in their software engineering articles. according to the empirical experiments, the proposed cohesion measures have some paradigmatic characteristics which are: (l)mtfocm(sp) <focm(ov,) <atfocm(.<p) for each ov, (2)ltfocm(s;?)+ atfocm(sp)=1.0. from the viewpoint of software quality, reusability, and maintainability, coincidental and logical cohesion are hard to describe the module purpose. but, on reviewing software in test maintenance stage coincidental and logical cohesion may be easy to break into some stronger cohesion modules because it performs multiple unrelated actions. logical cohesion has more than one action to be intertwined logically, which is not only difficult to understand the interface but also difficult to maintain and reuse. procedural cohesion performs a series of sequential actions with weak connection. communicational cohesion is alike procedural cohesion with performing on the same data, which may be little better than procedural cohesion for software quality. the cohesion strength of sequential cohesion is much stronger than communicational or procedural cohesion. therefore, a sequential cohesion module has a higher software quality better than procedural and communicational cohesion. however, functional cohesion accomplishes a single specific action, which can more readily be reused in a variety of situations. this type of cohesion is easier to understand and maintain. in practice, software engineer may not concern the cohesion categories in a specific procedure. rather, the cohesion concepts should be realized and low-end cohesion should be avoided when software is designed. 30 ajis vol. 8 no. 2 may 2001 conclusions in this paper, we proposed function-oriented cohesion measures based on the visualized analysis model of function-oriented live variables and live span. we gave a well-designed experiment to examine our proposed function-oriented cohesion measures. the experimental result remarkably meets smc cohesion and the fenton's cohesion spectrum, and matches a nonlinear scale of cohesion. the major contributions of this paper are that we proposed a formal definitions, scientific basis and well-normalized, well-experimented and easy measure algorithmically function-oriented cohesion metrics to improve software quality. furthermore, it is note worthy that this visualized function-oriented cohesion metrics can be easily incorporated with software case tool to help software engineers to enhance software development and quality. however, it may be inevitable to check theoretically the proposed metrics against cohesion properties to make them more precise. for the sake of space limitation, we leave it to future discussion and research. references bieman, j. e. and kang, b.k. (1998), "measureing design-level cohesion", ieee transactions on software engineering, feb, vol. 24. no. 2. bieman, j. m. and ott, l. m. (1994), "measuring functional cohesion", ieee transactions on software engineering, august, vol. 20. no. 8. conte, s. d., dunsmore h. e. and shen, v.y. (1986), software engineering metrics and models, thebenjamin/cummings publishing company, inc. canfora g and lakhotia a.(1999), "program comprehension", journal of systems and software, 44, pp.7980. curtis b. ( 1981), human factors in software development, pp. 170-179, silver spring, md: computer society press. cohesion, 1999. www http://www.cs.albanv.edu/~mhc/csi518/design/design.html (accessed aug. 1, 1999) demarco, t. ( 1982), controlling software projects, prentice hall, new york. fenton, n. e. (1991), software metrics, a rigorous approach, chapman& hall, london. fifty_percent, 1998, www, http//osiris.sunderland.ac.uk/~csohed/campbell/chap~7.html, (accessed may. 10, 1998) henry, s. and kafura, d. (1981), "software structure metrics based on information flow", ieee transactions on software engineering, feb, se-7(5),pp. 510-518. intramodule cohesion, 1999, www http://www.cs.unc.edu/~stotts/145/cohesion.html (accessed sep. 7, 1999) lucia, a.d and fasolino, a.r. (1996), "understanding function behaviors through program slicing", proceedings of 4th ieee workshop on program comprehension, berlin, germany, march, ieee comp. soc. press, pp. 9-18. lakhotia, a. (1993), "rule-based approach to computing module cohesion", in proceedings of 15lb international conference software engineering (icse-15). martin, j. mcclure, c. (1983), software maintenance the problem and its solutions, prentice-hall, inc., englewood cliffs, new jersey 07632. metr_dis, 1998. www http://www.hatteras.com/metr dis.html (accessed aug. 10,1999) modularity, (1999), www http://www.dis.unimelb.edu.au/staft/iacob/lectures/csdl/modularitv/tsld013.html (accessed sep. 7, 1999) ott, l. m. and thuss, j. j. (1989), "the relationship between slices and modules ohesion", in processdings of the 12ih international conference on software engineering (icse-12). pressman, r.s. (1988), software engineering: a practitioners approach, mcgraw-hill international editions. sommerville, i. (1996), software engineering, 5"1 ed. addison-wesley, 1996. stevens, w., myers, g and constantino, l. (1974), "structured design", ibm systems journal, vol. 13, no 2, pp. 115-139. yourdon e. and constantine l.l. (1979), structured design, prentice hall. 31 the australasian journal of information systems (ajis) began in 1993 and was based on the simple realisation that there was a need for an academic ‘outlet’ for is research in australia australasian journal of information systems volume 14 number 1 nov 2006 characterising information systems in australia: a theoretical framework dr gail ridley it control research group school of accounting & corporate governance university of tasmania. gail.ridley@utas.edu.au abstract the study reported in this volume aims to investigate the state of the information systems academic discipline in australia from a historical and current perspective, collecting evidence across a range of dimensions. to maximise the strategic potential of the study, the results need to be capable of integration, so that the relationships within and across the dimensions and geographical units are understood. a meaningful theoretical framework will help relate the results of the different dimensions of the study to characterise the discipline in the region, and assist in empowering the australian is research community. this paper reviewed literature on the development of disciplines, before deriving a theoretical framework for the broader study reported in this volume. the framework considered the current and past state of is in australian universities from the perspective of the development of a discipline. the components of the framework were derived and validated through a thematic analysis of both the is and non-is literature. this paper also presents brief vignettes of the development of two other related disciplines. the framework developed in this paper, which has been partly guided by whitley’s theory of scientific change, has been used to analyse data collated from the australian states and the australian capital territory. the degree of variation in australian is as an indication of its “professionalisation”, the nature of its body of knowledge and its mechanisms of control, will be used to frame the analysis. research reported in several of the papers that follow in this volume has drawn upon the theoretical framework presented below. introduction information systems (is) is a relatively new discipline in the australian context, as is discussed in clarke (2006) elsewhere in this volume. its contribution to australia has increased with the growing understanding of the importance of computer systems in assisting organisations and individuals to achieve their goals. given the growing contribution of the is academic discipline to 141 australasian journal of information systems volume 14 number 1 nov 2006 australia, the study reported in this volume is timely, as it aims to investigate the state of is in the universities of the region. the strategic benefits of gathering data in order to access increased power, status and resources for the is discipline in australia in the future are obvious. however, to maximise the future strategic benefits of doing so, particularly as this australian study is a pilot for future association for information systems (ais) studies in the asian-pacific region, the investigation needs to consider common questions in common ways. this paper develops a theoretical framework to provide a common way of looking at data collected over a range of dimensions from different geographical areas in australia. it is tempting to view the development of information systems (is) in australian universities as a unique case. however, there is a body of knowledge that attempts to explain changes in fields of knowledge, including the emergence of new disciplines. an understanding of the past development of is will help those in the discipline to better position the future of is in australia. much of the literature on the development of disciplines comes from the sociology and philosophy of science and dates from between the 1950s to the early 1980s. since that period the philosophy of technology has emerged as another branch of philosophy, which includes study of the role of technology within the development of society (gorokhov 1998). some literature from both sources is relevant to a consideration of is, as it is possible that many characteristics of its development arose because it was a new discipline, with involvement in technology. therefore, the features and milestones of the development of is may be typical of the early development of all or many disciplines and were not unique. this reasoning was supported in the is literature by farhoomand (1992) who contended that the nature of progress in a discipline needed to be examined within a framework of the philosophy of science. the concept accords with popper’s (1959) argument that discovery needs to be directed by theory, instead of theory being derived from empirical observation. the development of is as a discipline has been considered in several waves (fitzgerald 2003) since its first emergence, most recently by european and other researchers in 2002 and 2003. this work has examined both the origins and future of is. bauman (1992), however, believed that “only a [flawed] discipline …feels the need to justify its… exist[ence]” (p. 76). however, neither position has been influential in australia, where there has been little examination of the nature and development of is. at least two different views could be taken on the state of is, an external view from outside the academic field, and an internal one, as seen by is academics (hirschheim & klein 2003). this paper, like much of the literature, largely focuses on the internal view of is, leaving the external view to another time after an initial examination of the is discipline in australian universities has been undertaken. when taking an internal perspective, both research and teaching perspectives could be considered in a discussion of the state of is in australian universities. most of the literature on the state and development of is concentrates on research issues, rather than teaching. consequently, the review of the literature presented below regarding the state and development of is, places emphasis on research issues rather than on teaching issues. many view is as an applied science, as evidenced by the accreditation of is programs in us business schools by abet (formerly referred to as the accreditation board for engineering and technology) (challa, kasper & redmont 2005). science has been described as a convention, where the norms, expectations and values of the group while searching for understanding are relevant (klein, hirschheim & nissen 1991). as such, social characteristics are important to the development of science. bunge differentiated between culture-free pure science and scientific technology, where the latter was applied in nature and involved ethics (1979), while later writers saw science and technology as being interdependent or hybridised (pitt 2000; latour 2003). there has been much debate about how scientific progress comes about (lee 1989) with many 142 australasian journal of information systems volume 14 number 1 nov 2006 explanations having been put forward. the methods proposed for scientific progress include, for example, incremental verifications (the logical positivists), the increasing consensus of researchers (polanyi 1958), the use of falsifications (popper 1959), revolutions that overturn previous paradigms (kuhn 1970), progressive or degenerative research programmes conducted over extended periods (lakatos 1970), political practices (foucault 1977) or through research trails versus tinkering (chubin & connolly 1982). an overview of some of the literature related to scientific progress and the development of disciplines is presented in this paper, particularly where it has been linked in the past to is. it is believed that the literature provides a theoretical context for a study designed to characterise the state of the is academic discipline in australia. after an examination of the literature, a framework will be developed to guide the collection and analysis of data for the study reported elsewhere in this volume. is as a discipline or field there are different ways of defining a group of researchers undertaking related research. at least five definitions have been applied to is. keen (1991) saw nothing unique in is research in its topics, theory or methodology, and referred to is as a “self-defined community” as researchers “declare[d] themselves as members”. king (1993) viewed is as “not even a field”, but as “an intellectual convocation that arose from the confluence of interests among individuals from many fields”. a “field” has been defined as, ... an area of knowledge and learning which is not yet accepted as a discipline. fields of study tend to be more recent areas of scholarship with somewhat fuzzy boundaries; significant numbers of concepts within them are open to debate; and researchers and scholars in the area tend to draw heavily on old-established disciplines for their methodologies and conceptualisations (tardif 1989). a discipline has sharper boundaries. tardif (1989) saw a discipline as, ... a body of knowledge, definitions, and concepts built up over a long period and receiving consensus recognition by scholars; theories which interrelate the concepts and provide explanations of observed phenomena and permit predictions from them; and well established research methodologies. keen (1991 and king (1993) both saw is as a sub-field. hirschheim, klein and lyytinen (1996) referred to is as a field. even more recently, little consensus has been reached on whether is is a discipline or some other grouping. in 2002, paul viewed is as a “subject seeking a body of knowledge” (p. 175). in the following year fitzgerald (2003) saw is as not “even close to being a discipline” (p.225), but as a perspective placed between technology and some other subject areas such as management. a review of the literature suggests that researchers are still unsure about how to label is. many is researchers have used the terms field or discipline interchangeably, avoiding the issue. whether to label is as a discipline, a field or as something else is likely to become clearer with greater awareness and understanding of its nature and development. consequently, no attempt will be made in this paper to label is in the australian context as a discipline or otherwise. it is more appropriate to leave this analysis until after review has been undertaken of the study findings. 143 australasian journal of information systems volume 14 number 1 nov 2006 approaches to the development of disciplines a number of different approaches have been taken to account for the nature of different disciplines and their development. a review of the literature identified that three of the approaches were largely considered independently of other approaches. these were theory, social processes and research methods and standards. however, more holistic approaches have also been used to explain disciplinary nature and progress, by considering two or more of theory, social processes, research methods and standards, topics of knowledge, symbols sets for communication, the impact of local factors and the degree of professionalism. literature that deals with theory, social processes and research methods and standards to explain the nature and development of disciplines will each be examined in turn below, followed by literature that takes a more combined approach. largely independent approaches to explain disciplinary development interestingly, some of the approaches to explain disciplinary development have waxed and waned in popularity at particular times over the last fifty years. in 1959 popper argued that it was only through the generation of theories that scientific progress could occur. the importance of theory to a discipline has been recognised up to the present time. theory kuhn used the word “paradigm” in different ways including “universally recognised scientific achievements that for a time provide model problems and solutions to a community of practitioners” (1970, p.182). he argued that scientific progress arose as a result of new observation or experience that necessitated a “reconstruction of prior theory” and resulted in a paradigm shift. “normal science” represented the body of theory, practice, and methods of enquiry that were accepted by a group of researchers, typically expounded in textbooks of the discipline. wernick and hall (2004) analysed the textbooks of a discipline allied to is, software engineering, to examine the underlying belief system of authors from that discipline, to find that it was pre-paradigmatic with a common core of knowledge supplemented by competing sets of beliefs. it has been claimed by many researchers that there is limited theory in is (for example, grant 1991; keen 1991; paul 2002), while few textbooks have been published that provide an overview of the discipline. however, hirschheim and klein (2003) saw a “generalisation deficit” (p. 257) in is, rather than a lack of “theoretical knowledge” (p. 268). kuhn saw the presence of paradigms as a sign of maturity in a discipline, as they gave researchers a basis for choosing problems as well as guiding them in their investigation. despite frequent discussion of kuhn’s work in articles about its development, is has also been classified as pre-paradigmatic (culnan 1987; seddon 1991). consequently, from this perspective it appears that is may not have achieved the state of normal science, at least not by 1991. kuhn’s (1970) analysis of physical optics before the time of newton is illuminative. as that discipline had “no common body of belief ... each writer ... [built] his field anew from its foundations ... [and] ... there was no standard set of methods or of phenomena”. kuhn saw early fact gathering that was not guided by some “theoretical and methodological belief that permits selection, evaluation and criticism” as a “nearly random activity”. he saw the result of undirected research as a morass of “mere facts” that was too complex to be integrated with theory. senior is researchers have recognised the problem in is for decades. for example, “[w]e seem to randomly generate research projects with the outcome that we have a scattering of results which presents a 144 australasian journal of information systems volume 14 number 1 nov 2006 severe problem of pattern recognition” (dickson, benbasat & king 1982). more recently it has been claimed that is is characterised by the problems it studies more than a body of knowledge, or theories (paul 2002). it may be that some is research is guided more by the ease with which data may be gathered rather than by other criteria. it appears that, from kuhn’s perspective, is may be at a very early stage in the development of a discipline. elias (1982) referred to the nature of theory in traditional physics as “law-like theories” rather than “process theories” which may be more appropriate in other sciences. the ideas regarding theory types may be interesting ones to apply to is, even though not all researchers in the area, including fitzgerald (2003), would refer to the discipline as a science. the development of theory in is has been acknowledged as difficult (fitzgerald 1993; king 1994a; paul 2002; fitzgerald 2003), despite a well-known proponent of the philosophy of technology, bunge, seeing technology as “philosophically productive” (ihde 2004, p.120), with technological systems putting “forth … philosophically significant theories” (bunge 1979, p.172). just one part of the difficulty may be that the origin of is in technology and its past link with computer science has given those within and outside the discipline an expectation that law-like theories are appropriate for it. however, while the nature of is stresses organisational issues rather than technical ones (avison & fitzgerald 1991; galliers, 1992; hirschheim 1992; fitzgerald 2003), the more recent emphasis on interpretivism has not produced more general theory than the positivist approach (hirschheim & klein 2003). another researcher has proposed a list of theories for is, but argued for their greater consolidation (zahedi 2004). fitzgerald (2003) when discussing theory in is, distinguished between rules (or laws), evidenced guidelines and normative guidelines, where the latter is “…an interpreted view of something a practitioner developer might consider doing, under appropriate circumstances, but…would not necessarily lead to success” (p. 226). fitzgerald held that only the latter kind of “theory” was possible in is. furthermore, the location of management information systems (mis) in business schools in both the usa and, to a degree, in australia, also suggests that law-like theories and scientific method may not be the only, or even the most appropriate, approach for is. so a mismatch between expectation and achievement as well as the complexity of process theories may account in part for the limited production of theory in is. regardless of what kind of theory is produced in is, or what it is called, there is support for its development (paul 2002; fitzgerald 2003). a decade after popper referred to the role of theory in scientific progress, a very different view on the development of disciplines emerged. at that time the role of social conditions upon the production and assessment of scientific knowledge was recognised (whitley 1984a). social processes even though kuhn’s views have been referred to in previous discussion on the contribution of theory to the development of a discipline, he is associated more with a different approach. in 1970, at a time of burgeoning science and higher education sectors in many western nations, kuhn published a seminal analysis of the social process of science, which is still referred to in the is literature and elsewhere decades later (for example, wernick & hall, 2004). kuhn’s treatise, the structure of scientific revolutions, influenced the change in attitude to science and the nature of the development of disciplines. kuhn emphasised the social mechanisms that created a scientific discipline (ariav, desanctis & moore 1987), such as conferences, journals and academic departments, which have also been referred to as “mechanisms of control”. hirschheim & klein (2003) saw the control of rewards and punishments by academics from other disciplines as driving is research to become more theoretical and less applied. although such 145 australasian journal of information systems volume 14 number 1 nov 2006 pressure may increase the acceptance of is as a discipline, hirschheim and klein considered that a less applied orientation reduced is’s relevance to practitioners, and therefore its viability. a link between knowledge and power was proposed by foucault (1977), who also recognised the significance of social issues on the nature and development of a discipline. foucault would see the status of is as a political issue, rather than the achievement of ontological or epistemological positions. the supporters of this view see is as becoming a discipline only once sufficient “status has been conferred by institutional practices …(including) the ability to form departments, appoint chairs, organise conferences (and) edit journals…” (introna 2003), or the achievement of mechanisms of control. the nature of academic leadership is another way that social issues may impact the direction of a discipline and its perceived status (f.land, personal communication, january 23, 2006). it can be seen that one way to evaluate is’s status as a discipline would be to examine whether it had the mechanisms of control normally associated with a discipline. research methods and standards other researchers have considered research methods and standards in the development of a discipline. this work has led some researchers to examine the relationship between disciplines, particularly reference disciplines. elias (1982) argued that “high status sciences” retained their position by imposing their methods upon other sciences. he considered it inappropriate for scientific method to be imposed upon newly emerging sciences, particularly as it was developed for, and by, other disciplines. as the emphasis of one discipline may be on physical objects while in another discipline it may be on organisational issues, the scientific methods of some disciplines may be irrelevant to other disciplines. different disciplines rank more highly than others in the public and academic mind. as each has its own ideology and values that colour the knowledge they produce, elias (1982) saw interdisciplinary collaboration as “exceedingly difficult and almost impossible in many cases”. he argued that only low status disciplines would take heed of interdisciplinary criticism. moreover, elias contended that modelling a low-status discipline on a high-status discipline or its characteristics, in an attempt to gain kudos for the field or researcher, usually fails. if this last proposition is true it may have ramifications for is because of its close relationship to its reference disciplines. the use of theory and research approaches from reference disciplines may reduce the viability of is as a discipline while it attempts to improve the rigour of its research. furthermore, although it has been lamented that the is literature is not read by those in its reference disciplines (keen 1991), this characteristic may be a typical of any discipline. the concept of “restricted and unrestricted science” (rip 1982) is relevant to an analysis of the nature of is. in restricted sciences there is considerable control over the “knowledge object”, which allows a researcher to tightly restrict the behaviour of the object being studied, whereas in unrestricted science the reverse is true (rip 1982). rip argued that the high status of restricted sciences encouraged researchers from unrestricted sciences to become more like a restricted science by importing restrictedness. signs of a restricted science include use of sophisticated instruments, standardised procedures and empirical generalisations that give increased credibility and allow research assistants or research students to undertake routinised work. there is evidence that points to is as being unrestricted, despite technical is research appearing more restricted. criticism of is research approaches indicate that many of the instruments that have been used were not sophisticated, research procedures have been far from standardised (straub & 146 australasian journal of information systems volume 14 number 1 nov 2006 carlson 1989; boudreau, gefen & straub 2001) and a wide range of approaches from reference disciplines have been considered appropriate (ahituv & neumann 1986; culnan & swanson 1986). although greater standardisation of some research procedures has been seen more recently, at a time when it has been suggested that is is now itself a reference discipline (baskerville & myers 2002), other researchers have pointed to the difficulty in reaching consensus on the most appropriate methods for is (for example, hirschheim & klein 2003). even though the view of is’s development by practitioners is an external perspective, and this paper has restricted itself to the internal view from is academics, there will also be an internal perspective on the relevance and quality of is teaching to future and current practitioners and other students. research education itself may be seen as a standardised procedure where it is specialised to a particular discipline. relatively recent figures indicated that is researchers in the usa had gained their highest degree across a broad range of disciplines (walstrom, hardgrave & wilson 1995), revealing that the research training process for is academics trained prior to 1995 in that nation was far from standardised. it appears that a similar diversity was found in the education of australian is researchers to 1996, but that this diversity has narrowed since then. an anonymous reviewer of this paper contended that the diverse educational backgrounds of is’s “fathers of the field” are significant when examining the current nature of is. in addition to theory, social issues and research methods and standards, other approaches were identified in a review of the literature as having been proposed as contributing to an understanding of the nature and development of a discipline. these were more joint approaches, which combined two or more other approaches. combined approaches to explain disciplinary development there is majority support for examining both the body of knowledge along with the social processes, when considering the development of a discipline (becher 1994), as “we cannot…artificially separate the …substantive content from … social behaviour” (de solla price 1970). becher (1987) examined the nature of three different disciplines by examining their tacit knowledge (which derives from the body of knowledge) along with their linguistic behaviour (a social process). like tardif (1989), paul (2002) and hirschheim and klein (2003), fitzgerald (2003) considered that a discipline required a core body of knowledge. however, fitzgerald and paul saw a body of knowledge as being more than an agreed set of topics, but to include also the set of laws, rules or evidenced guidelines, that is, theory. fitzgerald postulated that is had “the trappings of a discipline...[such as] mechanisms of control [which are social processes], but without the core body of knowledge or agreed theory” (2003, p.226). consequently fitzgerald viewed is as a “perspective” rather than a discipline. hirschheim and klein (2003) saw the is body of knowledge as incorporating some social processes and theory, when they proposed that its four components were technical, theoretical, ethical and applicative knowledge. hirschheim and klein saw the development of applicative knowledge, that “required …[the application of] theoretical knowledge to specific circumstances” (p. 266), as being necessary to reach understanding and consensus in is. they considered that the limited extent of applicative knowledge in is threatened the viability of the discipline. the discourse needed to develop the body of knowledge for is, particularly applicative knowledge, would increase communication and is’s relevance, both internally to the academic discipline and externally. in later work, klein and hirschheim (2006) saw is at risk both because it was made up internally of several “communities of practice and knowing” (cop&k), each with different values and 147 australasian journal of information systems volume 14 number 1 nov 2006 legitimacy criteria, and because there was limited connection from academic is’s internal cop&ks to the outside. however, discourse would benefit the development of the discipline by helping to derive a common language across groups impacted by is, reducing the state of fragmentation in is and overcoming its significant communication gaps (hirschheim & klein 2003). shinn (1982) considered concepts that related social processes, research procedures and theory development when he examined the intellectual and social structure of a range of disciplines, in particular looking at the intellectual division of labour. he found some disciplines to be highly formalised, with a dichotomy between the gathering and collation of findings on the one hand, and experimentation, theory and hypothesis on the other. one would expect that the more restricted a science (rip 1982) then the more formalised its intellectual and social structure. as expected from its degree of restrictedness, is is not as formalised in this way as are some other disciplines. where interpretivist research is undertaken in is, the data gathering and the theory building may be interleaved. even in is positivist investigations it is likely that the chief investigator/s will be involved in all stages of the process. the latter characteristic is partly dependent on the limited success of is in attracting research funds (ridley et al. 1998), which relates in turn to the perceived status of the discipline. consequently, relatively few academic is researchers have funding to employ assistants to carry out some of the research tasks. the work of chubin and connolly (1982) allows further understanding of the combined pressures that have acted on the is discipline. the authors argued that “research trails” become institutionalised by offering potential rewards such as legitimacy and access to resources. on the other hand, “tinkering” with new ideas or novel developments is normally opposed. research trails are likely to use the epistemologies, research strategies, theory and perhaps even the topics of existing established research of reference disciplines. in is, those who follow the existing research trails that were established by the reference disciplines may be more likely to be rewarded with tenure, promotion and access to research grants, as rigour is easier to demonstrate. efforts to establish appropriate independent research approaches and traditions for is may be seen as tinkering, as it is more difficult to claim that work is rigorous if it does not follow established traditions. yet elias has argued (1982) that greater independence of a discipline leads to its development, and, it is assumed, eventual research rewards. it is possible that the path to the development of is may involve breaking with some traditions established by the reference disciplines. whitley’s theory of scientific change whitley’s theory of scientific change (1984b), which viewed disciplinary development as a social process in combination with other approaches, has been applied to many disciplines, including is (banville & landry 1992; checkland & holwell 1998). whitley categorised some sciences as highly professionalised, with high task certainty, routinisation of activities and division of labour, a categorisation that echoed the work of biglan (1973), kolb (1981), rip (1982) and shinn (1982). kuhn’s (1970) “normal science” fell into this category. however, other sciences were not highly professionalised, with high task uncertainty, decentralised control of work process and limited routinisation of tasks. where a discipline was not highly professionalised, local contingencies had high impact, such as the influence of local political pressure. ruscio (1987) also found that local factors resulted in substantial variations among universities for the same discipline. nonprofessionalised disciplines may account for ruscio’s finding. in disciplines that are not highly professionalised, researchers investigate disparate problems that are likely to vary in nature and approach to those of concern to practitioners. researchers work in 148 australasian journal of information systems volume 14 number 1 nov 2006 flat non-hierarchical groups, or independently, rather than in highly structured teams with a clear division of tasks. is appears to fit the mould of a discipline that is not highly professionalised. furthermore, if local contingencies are likely to have high impact on is, it would be expected that considerable variation would exist in the nature of is research between different universities and regions. there is evidence of considerable variation in the nature of is research between nations, is curricula and is research education. whitley (1984b) suggested that three conditions needed to exist for the establishment of distinct scientific fields. these were the need for: • scientific reputations to become socially prestigious and to “control critical rewards”; • establishing standards of research competence and skills, and • a unique symbol system to allow exclusion of outsiders and unambiguous communication between initiates within the field. whitley’s first condition is a social process and relates to mechanisms of control. scientific reputations are established, and critical rewards are obtained, through publication records and success at attracting research funding (mingers & stowell 1999). there were more is publication outlets available in 1995 than in 1980 (cule & senn 1995), and many more in recent years (hirschheim & klein 2003). in general, these publication outlets are now administered by fellow is researchers. however, access to funding remains tenuous while external funding decisions are made by individuals outside the discipline, as has happened with the allocation of australian research council grants (ridley 1997) during much of the development of the is discipline in australia. whitley’s second condition, the establishment of research skills (and standards), appears to be one component of a core body of knowledge, just as in the preceding discussion it has been seen that theory is also a component of the body of knowledge. ongoing debates regarding the quality of is research and appropriate epistemologies and methodologies (benbasat & weber 1996; boudreau, gefen & straub 2001) are signs that activity is taking place regarding whitley’s second criterion for the establishment of a field, but that it has not been resolved. however, more recently there have been some signs of increasing consensus. whitley’s third condition, the existence of a unique symbol set, appears to be another component of a core body of knowledge. whitley’s third criterion for the establishment of a field is hard to meet as long as reference disciplines remain important to is. because so many reference disciplines inform is research (walstrom, hardgrave & wilson 1995; baskerville & myers 2002) and the symbolic systems of each vary and compete, a dedicated and accepted is symbol system has yet to emerge. the frisco (framework of information system concepts) report that was produced in 1996 to clarify important is definitions (verrijn-stuart 2001) is one demonstration of attempts to satisfy the third criterion. interestingly, however, two other components that were identified in earlier discussion of the literature as contributing to a discipline’s body of knowledge fall outside whitley’s three conditions for the establishment of a distinct scientific field. theory (or laws, rules and evidenced guidelines) is not included within whitley’s conditions for the development of a distinct field; nor is an agreed set of topics. in earlier work, whitley (1984a) conceptualised seven stable categories to classify variations in the degree of mutual dependence between researchers of a field as against variations in the degree of 149 australasian journal of information systems volume 14 number 1 nov 2006 task uncertainty. these seven categories can be used to differentiate the nature of one discipline from another. the categories have been applied to is by researchers for two decades. researcher mutual dependency was defined as “…dependence upon particular groups of colleagues to make competent contributions to collective intellectual goals…”, while task uncertainty referred to “…the extent to which work techniques are well understood and produce reliable results…” (whitley 1984a, p.781). where task uncertainty is low, there is an “established set of research techniques…” which “…can be acquired through formal training programmes…” where “ …success is easy to determine” (whitley 1984a, p.781). of the seven categories, is has been classified as a fragmented adhocracy (culnan & swanson 1986; banville & landry 1992; culnan, swanson & keller 1993; swanson & ramiller 1993; checkland & holwell 1998; hirschheim, klein & lyytinen (1996); hirschheim & klein 2003; kanungo 2004). fragmented adhocracies display high task uncertainty with low researcher mutual dependence, so researchers from these disciplines make diffuse contributions to fluid goals that are contingent on local pressures (whitley 1984a). another characteristic of fragmented adhocracies is their openness to the general public (whitley 1984a), as they tend not to have unique symbol sets that exclude the uninitiated. it is suggested that the characteristics of is as a fragmented adhocracy work against it becoming a distinct scientific field. it has been seen in a review of the literature that the establishment of theory, social processes, research methods and standards, a unique symbol set and a set of key topics, have been used to explain the nature and development of disciplines in the past. a core body of knowledge appears to subsume theories, research methods and standards, the existence of a unique symbol set and a set of key topics. an examination of the relationship between the impact of local pressures and the degree of professionalism has also been used to help account for the nature of disciplines. although little literature was found that related teaching issues to the state of is in universities, as an internal academic perspective must include teaching issues, any framework developed will need to be capable of encompassing this area. it is argued that for completeness, both the relevance and quality of teaching need to be considered. the set of key topics must denote then, not only research topics but also relevant teaching topics, and will be one means of achieving interaction between the internal and external perspective of is. like research, teaching quality is also concerned with methods and standards. motivation to understand the nature and development of a discipline the author de solla price (1961, 1963) believed it is possible to trace the history of a discipline through its artefacts, which include the number of researchers as well as the number of papers, journals and scientific societies it engenders. he referred to the difficulty of a new field in making progress, and the characteristic that large disciplines grow faster than small (1963). this may be because large disciplines are more able to control critical rewards and exert political power, and so meet the first criterion for whitley’s establishment of scientific fields, controlling critical rewards through a range of mechanisms of control. taking a disciplinary perspective results in cross-fertilisation and an increased sense of unity (becher 1994), which brings other benefits in turn. disciplinary cultures frequently span institutional and national boundaries (becher 1994). a social mechanism, the “invisible college”, was described by de solla price (1961; 1963). culnan (1987) defined invisible colleges as the clustering of researchers into informal networks “which tend to concentrate on examining common questions in common ways”. the author, de solla price, saw membership of an invisible college as 150 australasian journal of information systems volume 14 number 1 nov 2006 conferring power and status on an individual and the network (1963). as these networks of researchers are best placed to lobby for better access to resources and funds (ridley 1997) than individuals, both individuals and researcher networks are likely to be advantaged by an increased awareness and understanding of their discipline. the following brief examinations of two other disciplines are presented to demonstrate both that it is possible to analyse disciplines from the perspective of their development, and that it is advantageous to do so. the reader is asked to note reference to approaches to clarify the nature and development of disciplines in the following vignettes, as identified from the review of the literature presented earlier. vignettes of disciplinary development an examination of the early development of two other related disciplines may help clarify the development of is. management whitley (1984a) tracked the development of management as a discipline. like is, management has been categorised as a fragmented adhocracy, but it has had a longer history. management originated as a distinct discipline in around 1960. until the late 1950s, united states (us) business schools taught material from economics, mathematics and psychology. the distinct labour market that emerged after a critical mass of management doctorates graduated, allowed the specification of a management doctorate and scholarly repute as criteria for appointment to academic positions and senior posts in management. these developments allowed management researchers to distance themselves from lay criteria and standards and increased their degree of mutual dependence. consequentially these changes limited their need to seek approval from non-management audiences for reputations and rewards (whitley 1984a). it can be seen in this brief vignette that a specific doctoral qualification in management, and the achievement of a critical mass of doctoral graduates, were keys that led to the development of the management discipline. many management sub-disciplines experienced a need to debate the most appropriate traditions to direct their research and choice of research methods. this has been true of organisational behaviour, accounting, marketing, strategic management and policy, operations management and operations research (klein, hirschheim & nissen 1991). it is little surprise then that is has not escaped similar debate. computer science computer science is another discipline that can be used to demonstrate that the concerns of the is community for the future of its discipline are not unique. computer science experienced problems associated with the youth of its academics (gries & marsh 1988), the diverse backgrounds of its researchers (hopcroft 1987), and a need to “cease its largely inward-looking activities and branch outward” (gries, walker & young 1989). doubts were expressed later about the future of the discipline and fears that computer science may become irrelevant (freeman 1995). there were calls for computer science to develop its own disciplinary characteristics and to avoid emulating high status disciplines like physics (hartmanis 1995b). other researchers in the same discipline debated the nature of computer science and tried to determine if it was a subset of engineering, science, mathematics or something more (denning et al. 1989; hartmanis 1995a). king (1994a) 151 australasian journal of information systems volume 14 number 1 nov 2006 observed that the majority of computer science departments were found within engineering schools, and that very few were completely independent with the same status as other schools. an analysis of literature to develop a framework after having examined is and non-is literature that reviewed approaches to the development of disciplines, and considered two examples from related disciplines, a method was sought to develop a framework that could be used to guide the examination, to be conducted elsewhere, of the nature and development of is in australian universities. not only did the components of the framework need to be identified, along with their relationship, but an evaluation was sought of whether a framework of the development of a discipline from the general literature was consistent with that developed from the is literature. in other words, would is and non-is researchers share a common view of the components that contributed to the nature and development of a discipline? it was reasoned that if the perspective of each group of researchers on the nature and development of a discipline was consistent, then this would act to validate the framework derived from the literature. methodology a thematic analysis process for a structured review was followed (dixon-woods, agarwal, jones, young & sutton 2005), where relevant literature previously identified was classified as having discussed one or more approaches that contributed to an understanding of disciplinary development. a data-driven approach was adopted, where the themes emerged from the data. the analysis did not reflect the frequency of the themes, but instead accepted themes that offered a “high level of explanatory value” (dixon-woods et al. 2005, p. 47). the relationship between the identified components was also examined. as it was considered necessary to examine the views of both researchers from is and other disciplines, it is acknowledged that any classification of approaches to disciplinary development derived from the literature is unlikely to be complete, due to the quantity of publications available on the topic. however, it is argued that only sufficient analysis is needed to identify the main issues when reviewing disciplinary development, until theoretical saturation is achieved, as is done when working towards concept development in primary qualitative research (dixon-woods et al. 2005). therefore any omission of literature in the area is unlikely to weaken the analysis and classification process. the purpose of distinguishing the themes identified from the is literature from those of the wider literature was to allow an evaluation of the degree of consistency between is researchers and those from other disciplines. if the framework developed matched the components identified from both groups of literature, and the relationship among the components, then the robustness of the framework would be strengthened. as classification is a largely subjective process, two trained is researchers categorised the literature independently. discussion took place where the classification differed, until agreement was reached. the major themes derived from the literature, social processes and core body of knowledge, were used for analysis. as foreshadowed, the latter category was broken down into four subgroups, research and teaching methods and standards, unique symbol set, key research and teaching topics and theoretical issues. to acknowledge the importance to is of providing professional training, any literature that considered teaching methods and quality in the development of the discipline, was grouped with the research and teaching methods and standards category, while literature on teaching relevance was grouped with key research and teaching topics. the impact of local influences was also sought. a record was made for each reference to these themes, by author. the results were further delineated by whether the author came from the is 152 australasian journal of information systems volume 14 number 1 nov 2006 discipline or elsewhere, as determined by the publication outlet. the relationship between the themes was captured by recording the combination of issues discussed with reference to disciplinary development, for each author. results table 1 sets out the results of the analysis of the literature, where the components explicitly examined in discussion of the nature and development of disciplines were identified and then classified. themes taken from a review of the is literature have been distinguished from those that were derived from the more general literature. the order of listing in the table matches that followed in the earlier discussion. core body of knowledge published research social processes research & teaching methods & standards unique symbol set for communication research & teaching key topics theoretical issues local influences popper 1959 klein, hirschheim & nissen 1991 ● bunge 1971 foucault 1977 keen 1991 ● paul 2002 ● ● wernick & hall 2004 o kuhn 1970 o o elias 1982 o o o becher 1994 o o hirschheim & klein 2003 ● ● ● de sola price (1961, 1963, 1970) o o tardif (1989) o fitzgerald (2003) ● ● ● rip (1982) o o shinn (1982) o o o ridley et al. (1998) ● ● chubin & connolly (1982) o o o o banville & landry (1992) ● ● 153 australasian journal of information systems volume 14 number 1 nov 2006 checkland & holwell (1998) ● ● biglan (1973) o kolb (1981) o whitley (1984b) o o o o ruscio (1987) o cule & senn (1995) ● mingers & stowell (1999) ● verrijn-stuart (2001) ● culnan & swanson (1986) ● ● culnan, swanson & keller (1993) ● ● swanson & ramiller (1993) ● ● kanungo (2004) ● ● culnan (1987) ● ● ● derived from is literature o derived from non-is literature table 1 identification from literature of framework components for disciplinary development discussion and findings as all the categories were found in both the is and the general literature, this finding was interpreted to mean that both groups shared a common view of the components used for explaining the nature and development of a discipline. consequently, it was assumed that the components identified were robust, and appropriate for inclusion in a framework of the development of a discipline. from the number of studies examined, and the results, it was assumed that saturation of topics had been reached. as many authors identified more than one component, a combination of components was considered most appropriate to account for the nature and development of the is discipline. framework development a framework to account for the nature and development of a discipline was prepared, using the components and their relationships identified in the review and analysis of the literature. whitley’s theory of scientific change and related concepts influenced the development of the theoretical framework. the framework has been used to consider the historical and present position of is in australian universities, in combination with a “body of knowledge”, using fitzgerald’s understanding of the term. the two constructs discussed below from whitley were utilised in the framework. 154 australasian journal of information systems volume 14 number 1 nov 2006 the first construct that sets out whitley’s three conditions for the establishment of a distinct scientific field has been used in part for the theoretical framework. as set out earlier, these are a. scientific reputations to become socially prestigious and to “control critical rewards”; b. establishing standards of research competence and skills, and c. a unique symbol system to allow exclusion of outsiders and unambiguous communication between initiates within the field. as outlined earlier, the first condition from whitley is a collection of mechanisms of control, or social processes, while the second and third conditions relate in part to a core body of knowledge. however, whitley’s second and third conditions were found insufficient to cover all aspects of a discipline’s body of knowledge. for example, the second condition excluded teaching issues, even though is academic teaching is largely the means by which an understanding of the discipline is imparted to future academics and practitioners. as demonstrated in the analysis of the literature, the “laws, rules or evidenced guidelines” component of “body of knowledge” (fitzgerald 2003; paul 2002) which has not been incorporated into the first construct above, also needs to be considered when examining progress towards the development of is in australia. furthermore, key topics were also shown by analysis to be an essential component of the body of knowledge, and included both relevant research and teaching topics. the second construct concerns the degree of “professionalisation” of the discipline, which is expected to decrease as the impact of local contingencies increases. consequently, a two-part framework was developed as a result of the literature analysis that includes mechanisms of control for the discipline and the core body of knowledge, both considered against time. the framework has been used to guide some of the regional data collection and analysis for the australian study. the second condition from whitley was adapted and incorporated into the core body of knowledge, as research and teaching methods and standards, while the third condition was included as unique symbol set. two additional components, laws, rules and evidenced guidelines, and research and teaching key topics have also been included in the framework. figure 1a illustrates the first part of the framework used for the study, and sets out the components that characterise the nature and development of a discipline. the second part of the framework is set out in figure 1b, which shows the inverse relationship between the impact of local contingencies on the is discipline and the degree of professionalism. 155 australasian journal of information systems volume 14 number 1 nov 2006 mechanisms of control core body of knowledge(in 4 components) research & teaching methods & standards unique symbol set key research & teaching is topics laws, rules & evidenced guidelines figure 1a framework for study: components of academic discipline degree of professionalism impact of local contingencies figure 1b framework for study: inverse relationship between impact of local contingencies on is discipline and degree of professionalism conclusions a body of literature exists that examines the theory of the development of disciplines. this literature suggests that the development of is in australian universities should not be viewed as a unique case, but instead a range of pressures need to be considered, that act on the discipline. a brief overview of the early development of the management and computer science disciplines was presented, to illustrate the development of two disciplines related to is. although some reference has been made in is to theory on the nature and development of disciplines, very little use of the material has been made to provide a context within which to view the past and present nature and position of is in australian universities. the project reported on elsewhere in this volume gathered data across a range of dimensions for universities in each large region in australia. it has been argued that if such data collection is to be utilised in a strategic way to increase access to resources for the is discipline, then it will be necessary to integrate the data and findings from regions across the dimensions, so that the 156 australasian journal of information systems volume 14 number 1 nov 2006 relationships within, between and among the regions and dimensions can be analysed, and the contributing factors better understood. a theoretical framework to guide data collection and analysis was developed from an analysis of the is and non-is literature on the development of disciplines, and presented in this paper. as the framework’s components could be derived independently from both the is and non-is literature, this characteristic strengthened the validity of the framework. the framework will integrate different aspects of the broader study, and provide a common way of looking at data collected over a range of dimensions from a range of regions in australian universities. data gathered from universities in the regions around australia can be used to assess the extent to which mechanisms of control have been established in australian is, along with the four components of the body of knowledge (research and teaching methods and standards, the existence of a unique symbol set, key research and teaching topics and laws, rules and evidenced guidelines). in addition, the relationship between the impact of local contingencies and the degree of professionalism can be evaluated through an analysis of the extent of variation in the nature of is in universities among the states of australia over time and at present. together these two constructs will provide a means of tracking progress in the state and development of the is discipline in australian universities in the past and future, and identify those issues that hinder progress. future work will analyse the utility of the framework, based on its effectiveness to characterise the development of is in australian universities. the external view of the development of is from those who are not is academics will also need to be evaluated at a future time. before applying the framework to examine the development of is in other regions of the world, it may be necessary for further refinement, based on the outcomes from the framework’s application to the australian study. acknowledgements i would like to thank a range of people for their valuable input on one or more aspects of this discussion and framework, including rudi hirschheim, frank land, bob smyth, guy gable, shirley gregor, craig mcdonald and the anonymous reviewers. references ahituv, n. & neumann, s. (1986) principles of information systems for management, 2nd ed., w.c. brown, dubuque, usa. ariav, g., desanctis, g. & moore, j. (1987) “competing reference disciplines for mis research”, proceedings of the eighth international conference on information systems, pittsburgh, pennsylvania, pp 455–458. avison, d. & fitzgerald, g. (1991a) “editorial”, journal of information systems, vol 1 no 1, pp 1– 3. banville, c. & landry, m. (1992) “can the field of mis be disciplined?”, in galliers, r.d. (ed.), information systems research: issues, methods and practical guidelines, blackwell scientific publications, oxford, uk. 157 australasian journal of information systems volume 14 number 1 nov 2006 baskerville, r. & myers, m. (2002) “information systems as a reference discipline”, mis quarterly, vol 26 no. 1, pp 1–14. bauman, z. (1992) intimations of postmodernity, routledge, new york. becher, t. (1987) “disciplinary discourse”, studies in higher education, vol 12 no 3, pp 261–274. becher, t. (1994) “the significance of disciplinary differences”, studies in higher education, vol 19 no 2, pp 151–161. benbasat, i. & weber, r. (1996) “research commentary: rethinking ‘diversity’ in information systems research”, information systems research, vol 7 no 4, pp 389–399. biglan, a. (1973) “the characteristics of subject matter in different scientific areas”, journal of applied psychology, vol 57 no 3, pp 195–203. boudreau, m., gefen, d. & straub, d. (2001) “validation in information systems research”, mis quarterly, vol 25 no 1, pp 1–16. bunge, m. (1979) “philosophical inputs and outputs of technology”, in the history of philosophy and technology, urbana, university of illinois press. challa, c., kasper, g. & redmont, r. (2005) “the accreditation process for is programs in business schools”, journal of information systems education, vol 16 no 2, pp 207–216. checkland, p. & holwell, s. (1998) information, systems and information systems -making sense of the field, john wiley & sons, chichester. chubin, d. & connolly, t. (1982) “research trails and science policies: local and extra-local negotiation of scientific work”, in elias, n., martins, h. & whitley, r., (eds.) scientific establishments and hierarchies, d. reidel publishing company, dordrecht, holland, pp 293–312. clarke, r. (2006) “key aspects of the history of the information systems discipline in australia”, australasian journal of information systems, ?, ?, ???-???. cule, p.e. & senn, j.a. (1995) “the evolution from icis 1980 to ais 1995: have the issues been addressed”, proceedings of the inaugural americas conference on information systems, aug. 25–27, pittsburgh, session mp-14. culnan, m.j. (1987) “mapping the intellectual structure of mis, 1980–1985: a co-citation analysis”, mis quarterly, vol 11 no 3, pp 341–353. culnan, m.j. & swanson, e.b. (1986) “research in management information systems, 1980–1984: points of work and reference”, mis quarterly, vol 10 no 3, pp 289–301. culnan, m.j., swanson, e.b. & keller, m.t. (1993) “mis research in the 1980s: shifting points of work and reference”, proceedings of the twenty-sixth hawaii international conference on systems sciences, vol. 3, ieee computer society press, los alamitos, california, pp 597– 606. denning, p., comer, d., gries, d., mulder, m., tucker, a., turner, j. a& young, p. (1989) “computing as a discipline”, communications of the acm, vol 32 no 1, pp 9–23. de solla price, d. (1961) science since babylon, yale university press, new haven. de solla price, d. (1963) little science, big science, columbia university press, new york. 158 australasian journal of information systems volume 14 number 1 nov 2006 de solla price, d. (1970) “citation measures of hard science, soft science, technology and nonscience”, in c. nelson and d. pollock (eds), communication among scientists and engineers, lexington ma, health & co., pp 3–22. dickson, g.w., benbasat, i. & king, w.r. (1982) “the mis area: problems, challenges, and opportunities”, data base, vo. 14 no 1, pp 7–12. dixon-woods, m., agarwal, s., jones, d., young, b. & sutton, a. (2005) “synthesising qualitative and quantitative evidence: a review of possible methods”, journal of health services research & policy, vol 10 no 1, pp 45–53. elias, n. (1982) “scientific establishments”, in elias, n., martins, h. and whitley, r., (eds.) scientific establishments and hierarchies, d. reidel publishing company, dordrecht, holland, pp 3–70. falkenberg, e.d., hesse, w., lindgreen, p., nilsson, b.e., oei, j.l.h., rolland, c., stamper, r.k., van assche, f.j., verrijn-stuart, a.a. & voss, k. (1998) a framework of information system concepts, the frisco report (web edition), 13 october 1998, available by anonymous ftp://leidenuniv.nl/pub/rul/fri-full.zip, accessed 28 october 1998. farhoomand, a.f. (1992) “scientific progress of management information systems”, in galliers, r.d. (ed.), information systems research: issues, methods and practical guidelines, blackwell scientific publications, oxford, uk. fitzgerald, e.p. (1993) “success measures for information systems strategic planning”, journal of strategic information systems, vol 2 no 3, pp 335–350. fitzgerald, g. (2003) “information systems: a subject with a particular perspective, no more, no less”, european journal of information systems, vol 12 no 3, pp 225–228. foucault, m. (1977) “truth and power”, in gordon, c. (ed.) power/knowledge: selected interviews & other writings 1972–1977, pantheon books, pp. 109–133. freeman, p.a. (1995) “effective computer science”, acm computing surveys, vol 27 no 1, pp 27–29. galliers, r.d. (1992) “choosing information systems research approaches”, in galliers, r.d. (ed.), information systems research: issues, methods and practical guidelines, blackwell scientific publications, oxford, uk. gorokhov, v. (1998) “a new interpretation of technological progress”, society of technology and philosophy, vol. 4 no 1. grant, r.a. (1991) “issues in conducting a field study of computerized work monitoring”, proceedings of the 1991 acm sigcpr conference, april 8–9, athens, georgia, usa, pp 68–79. gries, d., & marsh, d. (1988) “the 1986–1987 taulbee survey”, communications of the acm, vol 31 no 8, pp 984–991. gries, d., walker, t. & young, p. (1989) “the 1988 snowbird report: a discipline matures”, communications of the acm, vol 32 no 3, pp 294–297. hartmanis, j. (1995a) “the turing lecture, computing surveys symposium on computational complexity and the nature of computer science”, acm computing surveys, vol 27 no 1, pp 5–15. 159 australasian journal of information systems volume 14 number 1 nov 2006 hartmanis, j. (1995b) “response to the essays ‘on computational complexity and the nature of computer science’”, acm computing surveys, vol 27 no 1, pp 59–61. hirschheim, r. (1992) “information systems epistemology: an historical perspective”, in galliers, r.d. (ed.), information systems research: issues, methods and practical guidelines, blackwell scientific publications, oxford, uk. hirschheim, r. & klein, h. (2003) “crisis in the is field? a critical reflection on the state of the discipline”, journal of the association for information systems, vol 4 no 5, pp 237–293. hirschheim, r., klein, h. & lyytinen, k. (1996) “exploring the intellectual structures of information systems development: a social action theoretic analysis”, accounting, management and information technologies, vol 6 no 1/2, pp 1–64. hopcroft, j. e. (1987) “computer science: the emergence of a discipline”, communications of the acm, vol 30 no 3, pp 198–202. ihde, d. (2004) “has the philosophy of technology arrived? a state of-the-art review”, philosophy of science, vol 71, pp 117–131. introna, l. (2003) “disciplining information systems: truth and regimes”, european journal of information systems, vol 12 no 3, pp 235–240. kanungo, s. (1993) “information systems: theoretical development and research approaches”, information systems, vol 18 no 8, pp 609–619. kanungo, s. (2004) “on the emancipatory role of rural information systems”, information & people, vol 17 no 4, pp 407–422. keen, p.g. (1991) “relevance and rigor in information systems research: improving quality, confidence cohesion and impact”, in nissen, h.e., klein, h.k. & hirschheim, r. (eds) information systems research: contemporary approaches and emergent traditions, elsevier science publishers b.v., north-holland, amsterdam, pp 27–49. king, j.l. (1993) “editorial notes”, information systems research, vol 4 no 4, pp 291–298. king, j.l. (1994a) “rock and roll will never die”, sigois bulletin vol 15 no 1, pp 25–28. klein, h.k. & hirscheim, r. (2006) “further reflections on the is discipline: climbing the tower of babel”, forthcoming in king, j. & lyytinen, k. (eds) information systems: the state of the field, john wiley & sons. klein, h.k., hirschheim, r. & nissen, h. (1991) “a pluralist perspective of the information systems research arena”, in nissen, h.e., klein, h.k. &hirschheim, r. (eds) information systems research: contemporary approaches and emergent traditions, elsevier science publishers b.v., north-holland, amsterdam. kolb, d. (1981) “learning styles and disciplinary differences”, in a. chickering (ed.) the modern american college, san francisco, ca, jossey bass, pp 232–255. kuhn, t.s. (1970) the structure of scientific revolutions (2nd ed.), university of chicago press, chicago. lakatos, i (1970) “falsification and the methodology of scientific research programs”, in lakatos, i. and musgrove, a. (eds.) criticism and the growth of knowledge, cambridge university press, cambridge, uk, pp 91–196. 160 australasian journal of information systems volume 14 number 1 nov 2006 latour, b (2003) “do you believe in reality: news from the trenches of the science wars”, in r. scharff and v. dusek (eds.) philosophy of technology: the technological condition, blackwell publishing ltd., pp 126–137. lee, a.s. (1989) “a scientific methodology for mis case studies”, mis quarterly, vol 13 no 1, pp 33–50. mingers, j. & stowell, f. (eds.) (1997) information systems: an emerging discipline? mcgrawhill, maidenhead, berkshire, uk. paul, r. (2002) “is information systems an intellectual subject?” european journal of information systems, vol 11 no 2, pp174–7. pitt, j. (2000) thinking about technology: foundations of the philosophy of technology, seven bridges press, new york. polanyi, m. (1958) personal knowledge: towards a post-critical philosophy, university of chicago press, chicago. popper, k.r. (1959) the logic of scientific discovery, harper torchbook, london, england. ridley, g. (1997) “the role of conferences and refereed journals in australian information systems research”, the australian journal of information systems, vol 5 no 1, pp 69–82. ridley, g., goulding, p., lowry, g. & pervan, g. (1998) “the australian information systems research community: an analysis of mainstream publication outlets”, the australian journal of information systems, vol 5 no 2, pp 69–80. rip, a. (1982) “the development of restrictedness in the sciences”, in elias, n., martins, h. and whitley, r., (eds.) scientific establishments and hierarchies, d. reidel publishing company, dordrecht, holland, pp 219–238. ruscio, k. (1987) “many sectors, many professionals”, in b. clarke (ed.) the academic profession, berkeley ca, university of california press. seddon, p. (1991) “information systems: towards a definition for the 1990s”, proceedings of the second annual conference on information systems and database special interest group, sydney, pp 372–382. shinn, t. (1982) “scientific disciplines and organizational specificity: the social and cognitive configuration of laboratory activities”, in elias, n., martins, h. & whitley, r., (eds.) scientific establishments and hierarchies, d. reidel publishing company, dordrecht, holland, pp 239–266. straub, d.w. & carlson, c.l. (1989) “validating instruments in mis research”, mis quarterly, vol 13 no 2, pp 147–164. swanson, e.b. & ramiller, n.c. (1993) “information systems research thematics: submission to a new journal, 1987–1992”, information systems research, vol 4 no 4, p 299–330. tardif, r. (ed.) (1989) the penguin macquarie dictionary of australian education, a penguin australia reference book, penguin, ringwood, victoria, australia. verrijn-stuart, a. (ed.) (2001). a framework of information system concepts the revised frisco report, web document (draft). walstrom, k.a., hardgrave, b.c. & wilson, r.l. (1995) forums for management information systems scholars, communications of the acm, vol 38 no 3, pp 93–107. 161 australasian journal of information systems volume 14 number 1 nov 2006 wernick, p. & hall, t. (2004) “can thomas kuhn’s paradigms help us understand software engineering?”, european journal of information systems, vol 13 no 3, pp 235–243. whitley, r. (1984a) “the development of management studies as a fragmented adhocracy”, social science information, vol 23 nos 4/5, pp 775–818. whitley, r. (1984b) the intellectual and social organization of the sciences, clarendon press, oxford, uk. zahedi, f. (2004) “consolidation and learning in is: managing the research knowledge base”, information resources management journal, vol 17 no 2, pp 1–4. the author dr gail ridley is employed as a lecturer in the school of accounting and corporate governance within the faculty of business at the university of tasmania in australia. she is based at the hobart campus. her teaching interests include strategic information systems, information systems research methods, systems development methodologies and accounting information systems. her research interests include information systems research methods, it governance and it control frameworks. gail has published in both international and australian journals and conferences, and has worked as a consultant to government in the strategic information systems domain. 162 largely independent approaches to explain disciplinary development theory social processes research methods and standards combined approaches to explain disciplinary development whitley’s theory of scientific change motivation to understand the nature and development of a discipline vignettes of disciplinary development management computer science methodology results discussion and findings framework development acknowledgements the author ajis special edition knowledge management december 2001 an approach to managing repurposing of digitised knowledge assets diarmuid pigott school of information technology division of business, information technology and law murdoch university murdoch, wa 6150 valerie hobbs school of information technology division of business, information technology and law murdoch university murdoch, wa 6150 john gammack school of management griffith university 10 nathan, qld 4111 "the question is,' said alice, "whether you can make words mean so many different things." "the question is,' said humpty dumpty, "which is to be master that's all." abstract managing digital assets not only involves internal distribution, accessibilities and efficiencies, but also wider protections involved when derivative works or repurposings result from external distributions. the inherent polysemy of media artefacts has consequences for the reuse of such artefacts in multimedia systems. the multiplicity of possible meanings of a single media artefact depends on context and intention: reuse of a media artefact thus almost inevitably leads to the artefact being altered to suit the new context, and potentially losing its original meaning in the process. a system of rules is required to identify the circumstances under which a media artefact may or may not be reused throughout its life. within this context of practice we detail a protocol of usage status and reuse request. in this protocol, usage status is assigned to a media artefact by its initial custodian and the outcome of any reuse request is determined by the intention of the reuse, the current usage status and the implied new usage status. possible outcomes are forbidding change, permitting change, artefact forking, and changing artefact custodianship. we present a series of examples to illustrate the range of possible scenarios and their outcomes. introduction all media artefacts in use represent points of information. any use represents an organisational context, regardless of whether the media artefacts are shared or not, and any shift in this context represents a shift in meaning: when an artefact is reused or repurposed it acquires a diffuse or altered informational state which may work against the original intention that existed at its creation or adoption. although this problem can logically occur when there is only one user, the owner, it is manifest mostly when artefacts are shared. this problem becomes more than academic in the context of the development of organisational media capital, as the considerable investment in the creation of media artefacts can be a recoverable resource through planned reuse. organisations large and small are increasingly critically dependent on their information base and the knowledge resources that deploy information intelligently. their customers and markets are being driven by internet enabled technologies, and their products and services mediated electronically. for organisations of even moderate size, digitised multimedia artefacts, customer records, product designs, transaction histories, business processes and methods comprise valuable intellectual property impacting on operational and strategic decisions, and conferring comparative advantage internally or competitively. storing, accessing and securing, querying, and analysing this resource are already essential activities, with increased potential for effective operation enabled through useable web-based interfaces and architectures, and accessible multimedia repositories. beyond these proven systems for information management, however, are advanced technologies required for managing the knowledge that can identify and apply information in current contexts, and that can flexibly recombine components of information and other resources to provide new products and services, customised intelligently and market responsive. the ability to add value to an information repository through intelligent management and reuse of the knowledge involved in so doing gives an organisation in any sector a unique edge when the base technologies themselves are not a differentiator of competition. 10 from january 2002. the work was done when the third author was at murdoch university. 92 ajts special edition knowledge management december 2001 the inherent problem of media artefacts as information points is that they are polysemous and not at all selfdescriptive, so that some regularised form of data must be stored with them to permit conveying of meaning, storing, sorting and contextualising. by convention this is called metadata, but giving the practice a label only defers the problem, as the formalisation of rules of storage merely imports all of the problems that have been the subject of study of information organisation for centuries. much tacit knowledge, providing the interpretive context within the minds of knowers processing information, can be elicited and made explicit through various techniques (gammack, 1987), including unobtrusive modelling of usage patterns (gammack, 1991). knowledge is also shared, and it is this collective knowledge that typically allows an organisation to identify relevant data, to learn and to leverage its information resources effectively. its independent representation has however proved an enduring research problem. the issue of including context in knowledge codification and storage is a great challenge in this field and one of the important research questions identified by alavi and leidner (2001 pi28). producers of, and investors in, recorded digitised content typically have the right to determine the legitimate contexts of use, and the moral indenture associated with reuse or repurposing. internet technologies enable the reproduction, reformatting, selection and derivative product development from internal repositories of material, but also potentially by third parties in ways which may compromise the integrity of the source material. already, amateur publications and samizdatll academic materials of various kinds are spreading through peer-to-peer distribution arrangements, adapted, recontextualised and reinterpreted. the potential for "chinese whispers" (e.g. livemann, 2000) to occur is obvious, but even within corporate intranets, restructurings, mergers and staff turnover means that recorded corporate knowledge can mislead, both internally and externally. for effective reuse to occur, relevant contextual information accompanying the creation of digitised artefacts must be recorded. such information qualifies and constrains the possibilities of misrepresentation and misinterpretation, whilst admitting a variety of legitimate uses. issues include identifying provenance and legitimate ownership of digitised artefacts, allowed uses under different representational and contextualisation protocols, and the potential for reformatting and repurposing of material in knowledge repositories, using standard metadata storage mechanisms. apart from emerging delivery platforms and media formats, the distributed and global nature of the internet, encompassing numerous communities, jurisdictions and accepted standards further complicates this situation. a current case reported in the sydney morning herald typifies the issue (aap, 2001). here a text originally produced for consumption within one physical community (the usa) is considered (post-internet publication) to defame its subject in the community where he is best known (victoria, australia). whilst legal rulings and precedents will be set and established in due course, the principle of community context for information content is the point of law of relevance here. we recognise that media artefacts, and symbolic materials generally, are open to selective and contextual interpretation by others upon publication. this however does not exclude the intentions of the original author, nor the community context within which the work was produced and to which it references. without appeal to philosophical conceptions of knowledge admitting the transcendental, the universal or the transcultural, the frame of reference for any symbolised artefact must ultimately be embodied in community and cultural norms, at the level at which laws are shaped and socially agreed. this is important, since it conditions the status of an artefact and its potential for legitimate use. to manage the use of digital assets involves addressing issues of context, ownership and provenance, going beyond mere access controls and document management. instead the deep semantics associated with usage of particular digitised artefacts must be represented to a degree sufficient to avoid intentional or unintentional misrepresentation and misuse. the legitimate categories of use may be revised or redetermined by the asset owners or relevant communities of practice, but effective mechanisms to allow this are required if these artefacts are to retain their full authority as knowledge sources, and their provenance to be established. "samizdat (from the russian for self-publishing) refers to the clandestine network of reproducing and distributing unauthorised literature in soviet russia, and now by extension a generic term for self-publication which bypasses conventional production and distribution channels (such as internet sites and usenet) 93 ajis special edition knowledge management december 2001 recontextualisation, semantic drift and the problem of monodirectionality the potential for recontextualisation and either accidental or deliberate out-of-context repurposing is a particular instance of the generalised phenomenon of semantic drift: the gradual change in meaning that can happen to an artefact during its lifetime. consider an old photograph album found in a second hand shop. inside is an ordered sequence of brown photographs held on the pages by white paper corners. if the owner of the album has been diligent, then under each photograph in white ink will be a description of the occasion, time, date, place and subject matter. the point of keeping photographs in an album is partly convenience of storage, but mainly preservation of context. this is such an efficient mechanism of preservation that 60 years on intimate details about the lives of people recorded in photographs are easily retrievable and comprehensible, even though the people are unknown to us. we would know, for example, the names of a family group on the beach, and the identities of each one of them from the inscription underneath. the same photograph in serialised stock of a seaside photographer would lose its personalised context but acquire a new context in the record of the photographer's professional life. within this context it is almost impossible to recontextualise the identity of (e.g.) the photograph of the family on the beach. with further reuse the photograph acquires a new context of display and retrieval: if the album or the stack becomes part of a museum collection, the photograph may become something that stands for "seaside holidays of the forties" when it is part of a historical display the sense of personal identity in the photograph is lost. artefacts in use are in use precisely because one can ascribe intentionality to them. this is the difference between finding a photograph in a junk shop and being given one by someone who describes it to you. with the former, while the observer can infer a potential meaning and significance, there can be no certainty to the inference. with the latter (barring deceit and inaccuracy) he or she is provided with the photograph's meaning and significance. in any information system that uses media artefacts, this meaning and significance must be present before the artefacts can count as points of information. a useful analogy here compares the archaeological analysis of cultural artefacts with the sociological: the sociologist can proceed from the known to the unknown, asking questions of usage from the current owners, and building up a meaning-and-significance picture which can be verified with those owners. the archaeologist, on the other hand, is in the invidious position of proceeding in the opposite direction from shards, broken cookery vessels, hearths and inscriptions back to a living culture: trying to infer the context from the object normally found within that context. this type of reverse engineering works with a footprint, but not with a cultural artefact. the fact that we bother trying to do this in archaeology is because there is no other means at our disposal of working out the context. and we bother precisely because they are artefacts they must fit within a context somehow, because they were made within a context. media artefacts in an information system, then, can only be given their meaning and significance by accompanying metadata. by storing meaning-and-significance in metadata, we can always get back to a knowledge of the original context, even if all we have left of that context is the metadata. but where metadata must be created is at the point of the artefact's creation (or accession), and this leads to two separate problems. firstly, there is a distinction to be made between the metadata that is actually stored, and the potential set of all metadata that could possibly be stored for any artefact or class of artefacts. this potentiality can exist either as categories of metadata or as choices to be made within a category of metadata. secondly, and more importantly for our purposes here, the subsequent reuse or repurposing must involve either ignoring the new metadata, or overwriting or augmenting the original set. overwriting or ignoring metadata leads to a loss of information, while augmenting metadata may render it meaningless (e.g. a picture being both "attractive" and "ugly" from different points of view). the solution logically would lie in a system that permits the storage of usage and context in a continuing historical metadata narrative. this could be an mpeg-7 compliant description (iso, 2000) which also features versioning information, or in a set of xml rddl tags (rddl, 2001), or simply as transactions within a regular dbms. the pre-eminent means of displaying and retrieving information has become the internet web-page browser, as the lingua franca of html markup and inline image display has swept other presentation media before it. the resource sharing systems based on the http standards wherein images and metadata are served from a daemon via tcp have overcome many of the problems that beset universal presentation formats before now. however, this greatest common multiple solution has brought with it some profound problems for the implementation of an artefact-sharing system that involves knowledge of how each artefact is being used. the html linking system is ultimately derived from the practice of scholarly citation, where one paper or monograph refers to another by means of attribution, according to certain bibliographic standards. every paper contains a list of all (admitted) influences and sources, and so a backwards chain of referencing is possible that can extend for94 ajis special edition knowledge management december 2001 ever. however, the reverse is not true there is no means of automatically proceeding from an article or paper to those that cite it. in academic research this service is provided by the citation indexes, which use brute-force methods by indexing all references in the main academic journals. but there can be no guarantee of completeness in this direction. even if the citation indexes were sufficient for making a forward chain of referencing, they would still be insufficient for retrieving the context of citation. this can be clarified by using the terminology of polanyi (1966): when one goes from the citing to the cited, one proceeds from a context, one is attending from the citing article and is attending to the cited. when one goes from cited to citer, the reverse will never be true. the following axes of information and context are missing: generality/specificity: there is no way of knowing whether the referend is to a part or the whole of the referred object. essentiality/substitutability: there is no way of knowing whether only the referred object will suffice for the reference, or whether any similar referend will do. agreement/disagreement (or dismissal): there is no way of knowing whether or not the referring item is positive or negative about the referend. research/contingency: there is no way of knowing whether the referend was discovered after much research, and so has a logical trail leading to it, or whether it was discovered by chance and has less of a logical justification. fair-dealing and appropriateness/unfair dealing and out-of-context quoting: there is no way of knowing from the referend the extent to which the referring item is dealing fairly with the referend quoting in context or out-of-context, using a portion that is an indirect quote of a third item, or even using ellipsis to misquote completely. recontextualisation in repurposing: there is only a pious hope that the recontextualising in all of its considerations is such that the original meaning is not violated. secondary citation: the problem of the cascading referentiality that leads to semantic drift. here is the most significant for our current consideration: there is actually no way of determining whether or not the continuous referring and re-referring to the referend will lead to any or all of these problems to arise. thus we see that if the citation resides in the citer, with no reverse linking in place, there can be no possible way of reverse creation of the context of citation. these problems do not arise accidentally from careless usage: they are simply not determinable from the circumstances, as \hefact of use does not indicate anything about the nature of use. fundamental with the problem of href, and its replacement, the xlink instruction, is the problem of monodirectionality. this is in stark contrast with the original conception of hypertextuality, wherein the mechanism (and thus the process) of hyperlinking was duplex. this is what ted nelson envisaged in the xanadu project (nelson, 1990). "we long ago foresaw the problems of one-way links, links that break (no guaranteed long-term publishing), no way to publish comments, no version management, no rights management. all these were built into the xanadu design" (nelson, 1998). also missing from the property of linking as implemented as in http is the point of multiple departure, inherent in all of the semantic usages of linking. this was also an area of criticism in early comments on the http specification, from the hypertextuality development community (see berners-lee, 2000). this in many ways shows the origin of the http standard in the academic bibliographic tradition (burnard, 1991), wherein reference is a one-way mechanism, reinforced by the citation bibliographic tools that are the standard mechanism for academic research. the difference between true hypertextual structures and convenient computational structures for web maintenance is outlined by bernstein (2001). links in true hypertext create structure for the reader, using recurrence to reinforce meaning. "recurrence is the main way that people perceive a hypertext structure, the way they learn what contours they may follow and how those contours may change as the document evolves" (bernstein, 2001). these formalisms give semantic limitations in usage and distribution of digital assets since there is no inherent provenance or context represented in the resource, and, as nelson indicates, neither version nor rights management is inherent. even with a process of reverse linking in place, however, we are still moving from the contextualised towards the context, which is impossible in terms of tacit knowledge management theory (gammack, 1987). the existence of a formalised but unqualified system of reverse citation would still not permit the context to be recreated as the action of reverse-linking is still merely indexical, not actually the context itself. the problem of secondary citation or repurposing in particular cannot be guaranteed to avoid the problems described above: the context of citation must be stored in metadata with the link itself. 95 ajis special edition knowledge management december 2001 as we have seen, with the repurposing of a media artefact comes responsibility to not misuse it, not to breach copyright, or to take an image and use it in an ironic or offensive way. what is need is a statement of purpose by the new user, and a guarantee that the image will be used in an appropriate new manner. even then, the owner and original user must explicitly accept the new use, to ensure that no semantic drift occurs. current initiatives for reference and usage within the international information community there has been an intense period of research and development in both the systematisation of metadata, and the establishment of conventions for content-in-creation standards for assigning metadata to media artefacts. a large amount of the relevant work comes out of the digital libraries and digital media preservation communities, as it is here that change in the long term is seen as inevitable (largely because it has been met with already in the brief life of the electronic artefact), and therefore systems must be set in place now to ensure the long term survival of the identity and content of the media artefact. the library of congress has identified a set of relevant challenges to constructing digital libraries (library of congress, 1998), but without addressing the issue of knowledge based recontextualisation of categories. their list includes effective access to information within copyright and other legal concerns, retrieval of information across heterogeneous database sources and data formats and metadata schemes, and the construction, maintenance and enhancement of catalogued resources. several relevant major current projects internationally are sponsored through the nsfs digital libraries initiative phase 2 (digital libraries initiative, 2001). information-retrieval oriented work (e.g. chen, 2000) shows one typical approach, using computationally intensive fuzzy classification algorithms applied to large collections to replace knowledge structures derived from human intermediaries. these approaches are essentially brute force and historical, and have no basis in the deeper semantic codes of human usage. standardised protocols are already in use in many projects, but there are ongoing efforts at improving them (mccallum, 2000). for example, the iso z39.50 standard specifies a protocol for searching and retrieving information from remote databases, but has a number of perceived limitations (lynch, 1996). some multimedia formats may prove unsuitable for long term indexing as new standards replace existing ones for representation of content. in addition some resources may be in physical or non-digitised formats. the makers system of chen (1997) proposes an object-oriented knowledge representation, which builds on the dublin core (dcmi, 2001), and has the advantage of user centred extensibility, where additional fields relevant to user communities are represented without interference with permanent library semantics. chen's proposal does not appear to have been implemented to date, and does not specifically address the problem of concurrent mutuallyantagonistic metadata needs, nor the problem of potential metadata overwriting. also of interest is the concept of data provenance, as described by buneman (1999) in a digital libraries initiative project, since media artefacts may be extracted from one context and used in another, losing meaning or misrepresenting intention. usage of media as a meaningful action is irreducibly based in human semantic spaces and is not considered simply a property derivable from disembodied associations (clancey, 1997), and invokes notions of classificatory knowledge being grounded in processes of discourse (star & ruhleder, 1996). in practice, the organisation of information in digital repositories does not need to be bound by the metaphors guiding the arrangement of physical libraries, enabling new design metaphors to guide the dynamic structuring of resources. svenonius (2001) has identified key principles and functional objectives for organising intellectual resources, which point to a new type of cataloguing solution: one enabled by current technologies that addresses the traditional difficulty of constructing appropriate classifications of materials from limited category labels. besser (2000) cites the 'custodial problem' for digital longevity as yet there are no traditions for preserving digital materials as there are for non-digital, and a major problem is determining who should be responsible for maintaining material in an electronic form. besser also highlights the inevitability of mutation over time when files are repeatedly copied to new strata. he also cites the 'translation problem' wherein the change of the form of digital content often serves to change part of the meaning. this process is part of the general problem of changing meaning over time that we have termed semantic drift above. in a recent white paper, the oclc/rlg preservation metadata working group (2001) reviewed the state of the art in preservation metadata, using the oasis information model as framework for metadata requirements for digital preservation. this model includes both the data object itself (physical or digital), and a knowledge base that is associated with the users and necessary to interpret the data object. the knowledge base itself is considered to be external to the archive, and is not maintained as part of the archival function. 96 aj1s special edition knowledge management december 2001 another source of metadata usage is the metadata-in-creation community, where systems such as the digital object identifier (which continues the traditional cataloguing-in-publication into the digital domain) (international doi foundation, 2001) and the resource-based view work (habann, 2000) focus on the maximisation of the information potential (including recording of context) of a media artefact by imposing metadata standards at the moment of its creation. this work does not specify the usage system, but prefers to have an ecumenical approach to the systems of delivery. however, it does not permit the management of change in the semantic nature of the artefact, as the metadata is carried with the artefact while referencing standards, and is static. each usage is therefore independent of others, linked only by the common usage. the conceptual schema of a hybrid emergent/structure-based system for establishing semantic links between media artefacts, and amongst information repositories generally, has been identified as a key component of rational knowledge representations by sowa (2000). drawing heavily on sowa and the earlier cyc work by lenat (guha, lenat, pirtman, pratt, & shepherd, 1990) is the internet topic-mapping initiative (iso/iec, 1999), which aims to provide a unified representation of knowledge in a manner that relates to its embodiment in information resources. the proposed standard has xml-encoded structures that permit the creation of loosely established late-binding semantic maps, which in turn make use of the existing hypermedia protocols and formats of the web. gardenfors (2000) has elaborated the case for a framework for representing information at the conceptual level, which addresses some of the difficulties with purely symbolic approaches to concept representation. lagoze (2000) points out that while traditional library materials have always had to deal with a complex web of relationships among information entities such as translations, editions and transcriptions, networked digital information has greatly increased the complexity of those relationships. he proposes the role of catalogue as mediator of a distributed system: "rather than absorbing semantics (and descriptions) from distributed communities, libraries should promote the catalogue as a mapping, or interoperability mechanism, amongst distributed descriptions. technologies such as rdf [w3c, 2000] and its schema language... make it possible to undertake such a mapping role amongst individual descriptions that are distributed across the web". such a system would require protocols in place to ensure against semantic or physical alteration, and the problem would increase, rather than decrease, with the scale of usage. bearman and sochats (1996) identify situation variables "inflation, changes in permissions based on elapse of time since the event, re-engineered business processes, etc." that in alteration can create a change in the meaning of a digital asset. semantic consistency could be created by a stored object that is "encapsulated by metadata necessary to ensure its integrity and longevity", and he proposes a record keeping system or warehouse where it would be kept intact. while this system is a useful model, it is geared more towards maintaining accuracy in a single meaning, rather than permitting the maintenance of concurrent variance in meaning and ensuring the consistency of those multiple meanings through time. finally, the lockss model proposed by reich & rosenthal (2000) to preserve access to scientific journals published on the web, works by using redundancy by permitting multiple independent copies to be maintained in different locations, with integrity ensured by a system of digital signatures. however while this method preserves the content of the article, it is at the expense of a central reference scheme. we can see that there are two components missing from these current initiatives for meaningful preservation of media artefacts: the cumulative recording of the context of usage (which enables semantic drift to be recorded) and the means of censure/restriction to permit/restrict untoward usage. we now turn to the consideration of what procedures could be put into place in a media artefact repository to enable these requirements to be met, and present a proposal for a protocol to manage the reuse of digital assets. a protocol for reuse let us consider the acquisition and subsequent reuse of digitised photographic images in a museum archive. as we have discussed, when an image is added to the archive, it is essential that metadata describing semantic purposing is included in order that the base requirements of preservation of context be met, to manage any potential semantic drift that could occur for the images. we recall that there is an original intention for each image, and that additional meaning accumulates with subsequent reuse/repurposing. an analysis of the situations that can arise within the archive shows that the original or current owner (the custodian) of an image can have one of four possible attitudes towards an image in their hegemony: they could forbid reuse outright, for reasons of copyright or sensitivity. they could forbid reuse of the image per se, but permit a copy to be made with a new life of its own. they could be willing to share the image. they could be willing to share the image and lose hegemony over it if someone else required it. 97 ajis special edition knowledge management december 2001 these attitudes would be expressed by a usage status indicator assigned by the custodian when the imaged was catalogued. there would also be a concomitant set of attitudes for the proposed reuse, depending of the intention of the applicant user. this means that in any occasion of intended reuse, there would have to be a negotiation between the existing and intended usages of the image, leading to a protocol of reuse, using a hybrid system of versioning, messaging and automatic decision-making. we shall describe these next, using typical scenarios of use. all reuse forbidden: usage status forbid when reuse of an image is forbidden for social/legal reasons, there is no option. we call this image usage status forbid (figure 1). applicant requests^ custodian forbids 2. ^ existing usage is forbid figure 1. applicant requests reuse of image with forbid status; request is rejected. this scenario has an automatic rejection of proposed new use: all applications are automatically rejected by the system regardless of their proposed reuse policy, with a pre-arranged message of explanation from the custodian to all potential applicants for reuse. as long as the custodian keeps up the forbid usage status, this process is fully automated an example of this scenario would be where images have been collected by an ethnographer of culturally sensitive materials of a ritualistic nature; these are permitted to be used in the strict understanding that they won't be misinterpreted. any reuse whatsoever, any re-situating through additional keywording or description will violate that condition. therefore all reuse will be forbidden. note that this does not apply to secret images or images that are not available to the public but to images that are available to the public but whose meaning cannot be altered: e.g. clustering of shamanistic spirits through aggregation of the totemic animals on zoological lines could be enormously offensive to the traditional owners. on another level, family material in film and photograph could be given to a public archive, but only on the strict assurance that they were not used for public display, and never used in compilations. in each of these examples, the custodian of the image is the archivist with responsibility for cataloguing the image in the museum, according to the wishes of the donor. reuse is only permitted of a copy of the artefact: usage status restrict when an image serves an important role in an image database, and absolute control of that image is necessary, then reusing that image may be not desirable to its owner. however, if there is no other reason for this image not to be reused, then the image could be cloned, the original image keep its meaning, and entirely new image begin a new meaningful life. we call this image usage status restrict. in this case, there are two possible scenarios. in each, the applicant makes a request, and the custodian offers the possibility of a fork and the creation of a new artefact with new metadata to pass into the custodianship of the applicant. the two scenarios diverge with the decision of the applicant they can either reject the offer (because they do not want to be custodians) (figure 2) or they can accept it and repurpose the image with modified metadata (figure 3). 98 ajis special edition knowledge management december 2001 1. 2. 3. applicant requests custodian offers fork applicant replies no, end existing usage is restrict figure 2. applicant requests reuse of image with restrict status and rejects offer of fork. 1. 2. 3. 4. applicant requests . custodian offers fork applicant replies yes confirm metadata existing usage is restrict figure 3. applicant requests reuse of image with restrict status and accepts offer of fork. these two scenarios can no longer be automated completely, but only as far as the offer of forking: at this point the applicant has to make a decision based on his or her understanding of the role of custodianship of the media artefact, and then either accept or reject the automated offer. an example of this could again be family photographs donated to the archive. a photograph of the family homestead could include factual details of the family and their history, but would be unlikely to include information of a symbolic or interpretive nature. subsequent reuse and description of the image might lead to an unwanted associations: for example, the homestead in the photograph could be used to stand for a type of land use or farming style that was inimical to the actual owners of the homestead. thus it would be desirable to ban reuse of the original image (so that, for instance, family details were not retrieved with the new usage), but repurposing with new metadata could be permitted. shared reuse may be negotiable: usage status vote the next usage status concerns the case where there is no need for restriction on the use of the image, but where the nature of any repurposing of the image can potentially lead to problems. once the image is shared, the original custodian (by definition) no longer has total control over it, so that when any one of the users wishes to modify the image (crop, enhance, enlarge, shrink), modify its metadata, or completely replace it with another that can be substituted in terms of meaning, then that modification must be approved by all of the users for it to proceed. we call this usage status vote, since the mechanism of resolution employed involves voting on whether a change should happen or not. in this situation, the custodian serves as mediator. when a modification/substitution is mooted (either by an applicant user, or else by one of the existing ones), there are two possible outcomes. firstly, all existing users might consent which results in the proposed change (semantic or physical) to the image being carried out (figure 4). 99 ajis special edition knowledge management december 2001 applicant requests ^ custodian asks for new usage 3. applicant replies with new usage custodian calls for vote 6. permit new usage existing usage is vote figure 4. applicant (new or existing user) requests reuse of image with vote status. all users reply yes, so new usage is permitted. secondly, some or all of the existing users may dissent which means that the existing image must be cloned (forked) and the copy modified. when this happens, the applicant/user may reject (figure 5) or accept the forking, and in the latter case (figure 6), some of the users of the old image may elect to go with the newer (modified) image, including the semantic implications that it carries. applicant requests ̂ 2. 3. 4. 5. 6. 7. custodian asks for new usage applicant replies with new usage custodian asks for vote some users reply no custodian offers fork applicant replies no, end existing usage is vote figure 5. applicant (new or existing user) requests reuse of image with vote status. some users reply no, so custodian offers fork. applicant rejects fork. 100 aj1s special edition knowledge management december 2001 applicant requests cusodian asks for new usage applicant replies with new usage custodian asks for vote custodian offers fork applicant replies yes who wants to stay go custodian asks who wants which confirm metadata etc existing usage is vote figure 6. applicant (new or existing user) requests reuse of image with vote status. some users reply no, so fork is offered. applicant accepts fork and users are polled to select which usage they wish to continue with. an example of the replacement case might be where an image of a mountain was in use, and had the over-loaded semantic use of weather, tourism spot and geological formation. a better (or perhaps a clearer) image of "weather" may become available, and be mooted for replacement. however, the proposed replacement may not be good for illustrating one or more of the other existing uses. in that case, the forking would lead to a new image accession. a more complex case of the vote usage status arises when an image has both representative and symbolic usages concurrently the senses of showing that (an illustration of mount everest) and standing for (an illustration that mountains can arise from tectonics mount everest rather than kilimanjaro) lead to differing ways in which a new image might be substitutable for the image in current use. however, the protocol remains the same. we note a special case of vote where the new usage proposed status is restrict: here an automatic forking would occur, the processing of voting being unnecessary since the restrict status denies future sharing. the image can be shared, and custodianship transferred: usage status surrender. when the custodian is the sole user of an image, but the usage is not one that can only be served by a particular image, the custodian may not only be willing to let the image be shared, but also to have another user take custodianship of it. this usage status is called surrender (figure 7). 101 ajis special edition knowledge management december 2001 applicant requests ^ ^custodian surrenders confirm metadata o. existing usage is surrender figure 7. applicant requests reuse of image with surrender status and receives custodianship of image. this case would most likely arise when the image in use is standing for a type, and the specifics of either image or metadata would not necessarily concern the custodian. a custodian who has an image of kilimanjaro standing for "volcanic mountains" may be equally happy with a proposed replacement by an image of mount etna or fujiyama. in effect, this case arises when a custodian does not want to take on the role of mediator in the community arising from subsequent image reuse. the original custodian remains part of the usage community for that image, wherein the new custodian mediates any vote decisions. a proposed replacement of the image by one of mount everest to illustrate "snow on mountain" would lead to the original custodian's requesting a fork, so that their usage context would not be violated. discussion the protocol proposed here would, when implemented in an enterprise media artefact repository, enable its semantically consistent reuse and repurposing. the hybrid social/machine rule-set described can effectively meet the situations that arise in the lifetime of such a repository, and could be implemented using a version control system in conjunction with a standard internet rfc call-for-votes system. we have described our proposed protocol using the simple example of an image archive, but the principles hold equally for digital assets of any media type. there are several issues that remain the subject of further research. these issues include the potential trade-off between the lag-time in decision making and the possible urgency of a particular reuse, and the added complications that could arise from situations such as public archives where one part maintains use control, while another keeps copyright permissions. the preservation of media capital in an enterprise might require a derelict usage status to be assigned to artefacts that are not currently being used, so that their "usage pedigrees" are still available for subsequent mining of the metadata associated with them. in particular, we intend to investigate specific communities of usage practices and their metadata requirements, including metadata to provide some indication of the reasoning behind the assigned usage status. it is anticipated that the in-community usage behaviour will lead to network effects arising from the interconnectivity of reuse links, and the community of knowledge they represent, while the existence of community standards may find that additional requirements on the reuse/repurposing (similar to those mentioned in the forbid case) need to find expression in an additional set of rules. references aap. (wednesday, june 6, 2001). landmark case of internet publishing. sydney morning herald. retrieved from: http://www.smh.com.au/news/0106/06/pageone/ pageone6.html alavi, m., & leidner, d. e. (2001). knowledge management and knowledge management systems: conceptual foundations and research issues. mis quarterly, 25(1), 107-136. bearman, d., & sochats, k. (1996). metadata requirements for evidence. retrieved june 24 2001, from university of pittsburgh, school of information science: http://www.lis.pitt.edu/~nhprc/bacartic.html. 102 aj1s special edition knowledge management december 2001 berners-lee, t. (2000). weaving the web: the original design and ultimate destiny of the world wide web: harperbusiness. bernstein, m. (2001). hypertextnow. retrieved from: http://www.eastgate.com/ hvpertextnow/archives/c vcles.html. besser, h. (2000). digital longevity. in m. k. sitts (ed.), handbook for digital projects: a management tool for preservation and access (pp. 155-166). andover ma: northeast document conservation center. buneman, p. (1999). data provenance. retrieved from: http://db.cis.upenn.edu/research/ provenance.html. burnard, l. (1991). what is sgml and how does it help? tei edw25. retrieved from: http://www.oasisopen.org/cover/edw2 5. html. chen, h. (2000). digital libraries. journal of the american society for information science special issue on digital libraries, 57(3). chen, s.-s. (1997). digital libraries: the lifecycle of information. columbia: better earth publisher. clancey, w. j. (1997). situated cognition: on human knowledge and computer representations (learning in doing): cambridge university press. dcmi. (2001). dublin core metadata initiative. retrieved from: http://www.dublincore.org/. digital libraries initiative. (2001). digital libraries initiative phase 2. retrieved from: http://www.dli2.nsf.gov/. gammack, j. g. (1987). different techniques and different aspects on declarative knowledge. in a. l. kidd (ed.), knowledge acquisition for expert systems: a practical handbook . new york: plenum. gammack, j. g. (1991). methodological issues in validating knowledge based systems. paper presented at the amse international conference on signals and systems, warsaw. gardenfors, p. (2000). conceptual spaces: the geometry of thought. cambridge ma: mit press. guha, r. v., lenat, d. b., pittman, k., pratt, d., & shepherd, m. (1990). cyc: a midterm report. communications of the acm, 33(8). habann, f. (2000). management of core resources: the case of media enterprises. the international journal on media management, 2(1), 14-24. international doi foundation. (2001). the digital object identifier (doi) system. retrieved from: http://www.doi.org/index.html. iso. (2000). overview of the mpeg-7 standard (iso/iec jtc1/sc29/wg11 n3445). geneva: international organisation for standardisation. retrieved from: http://www.darmstadt.gmd.de/mobile/mpeg7/documents/w3445.htm iso/iec. (1999). iso/iec 13250 topic maps: information technology document description and processing languages (international standard). retrieved from: http://www.vl2.doe.gov/sgml/sc34/document/0129.pdf lagoze, c. (2000, november 2000). business unusual: how "event-awareness" may breathe life into the catalog? paper presented at the bicentennial conference on bibliographic control for the new millennium, library of congress. library of congress. (1998). challenges to building an effective digital library. library of congress national digital library program. . retrieved from: http://memory.loc.gov/ammem/dli2/html/cbedl.html. livemann, d. (2000). bacher caught in game of chinese whispers. criclnfo archive, 21 april 2000. retrieved from: http://www-aus3.cricket.org/link_to_database/archive/ cricket_news/2000/apr/030523_ci_2 lapr2000.html. lynch, c. (1996, september 3, 1996). defining and maintaining attribute sets for use with the z39.50 protocol: a discussion paper. retrieved from: http://lcweb.loc.gov/z3950/agency/attrarch/archive/cliff3.txt. mccallum, s. (2000). extending marc for bibliographic control in the web environment: challenges and alternatives. paper presented at the bicentennial conference on bibliographic control for the new millennium, library of congress. nelson, t. h. (1990). on the xanadu project. byte (september), 298-299. nelson, t. h. (1998). project xanadu. retrieved from: http://www.sfc.keio.ac.jp/~ted/xu/ xupagekeio.html. oclc/rlg. (2001). preservation metadata for digital objects: a review of the state of the art (white paper): oclc/rlg preservation metadata working group. retrieved from: http://www.oclc.org/digitalpreservation/ polanyi, m. (1966). the tacit dimension. garden city, n.y.: doubleday. rddl. (2001). resource directory description language (rddl). retrieved from: http://www.openhealth.org/rddl/. 103 ajis special edition knowledge management december 2001 reich, v., & rosenthal, d. s. h. (2000). lockss (lots of copies keep stuff safe/ paper presented at the preservation 2000: an international conference on the preservation and long term accessibility of digital materials, york, england. sowa, j. f. (2000). knowledge representation: logical, philosophical, and computational foundations. pacific grove, ca.: brooks cole publishing co. star, s. l., & ruhleder, k. (1996). the ecology of infrastructure: problems in the implementation of large-scale information systems. information systems research, 7(1), 111-134. svenonius, e. (2001). the intellectual foundation of information organization: mit press. w3c. (2000). resource description framework (rdf) schema specification 1.0. w3c candidate recommendation 27 march 2000. retrieved from: http://www.w3.org/ tr/rdf-schema/. 104 aj1s special edition-electronic commerce november 1998 the long term effects of electronic commerce and data interchange on firms professor francois bergeron d6partement systemes d'inforrnation organisationnels facult6 des sciences de 1'administration university laval quebec, quebec, canada g1k7p4 phone:(418)656-7940 email: francois.bergeron@fsa.ulaval.ca fax: (418) 656-2624 and professor louis raymond d£partement des sciences de la gestion university du quebec £ trois rivieres c.p. 500, trois-rivieres quebec, canada g9a5h7 e-mail: louis_raymond@uqtr.uquebec.ca abstract this research aims to identify the determinants of electronic commerce effectiveness, as well as the variation over time of their effect. a two-phased study was undertaken for this purpose, using edi as a specific form of electronic commerce. sixty-five organisations were surveyed twice over a three-year period. in phase 1, determining factors were found to be the level of organisational support, the rigour of the implementation process, the presence of control procedures, and the level of internal integration, external integration and imposition of edi. imposition also has a negative effect on the organisational context of edi. in phase 2, the implementation process, internal integration and imposition lost their importance, whereas the effect of imposition on the organisational context became positive. in the future, we expect the security and external integration to be the most important success factors of electronic commerce. keywords : electronic commerce, electronic data interchange, inter-organisational systems, longitudinal study introduction electronic data interchange (edi) is an application of electronic commerce (ec) that enables commercial partners to transact by exchanging and processing standardised business documents electronically from computer to computer. the number and types of organisations using edi has grown steadily in recent years. whole industries are now being integrated through this technology; witness for example, the automotive and air travel industries, food retailers, drugstores and hospitals. while edi was mostly based on private networks before 1985, value-added networks operated by third parties have been the main providers of this technology up to the mid-nineties. since then, edi has also become an internet-based technology (wheatman, 1996). while electronic commerce (ec) embraces three basic aspects, namely transactions, information and strategy, edi is mostly concerned with the transaction aspect of inter-firm relationships. as such, it is considered to be an enduring form of ec (kalakota and whinston, 1996) and a key technology for the implementation of interorganisational systems (sokol, 1995). edi has been shown effective by providing numerous advantages to organisations. among these are improvements in administrative costs and transaction costs, in information quality, in operations management, and in the attainment of a strategic advantage (bergeron and raymond, 1992b; mukhopadhyay, kekre and kalathur, 1995). however, there is no consensus on how effective edi truly is in providing these advantages. some researchers and consultants doubt that companies and other organisations have realised any significant cost savings or other durable benefits from implementing edi (venkatraman and zaheer, 1990). others have questioned the possibility of gaining any advantage at all from using this technology, particularly when the use of edi is imposed by a business partner (bergeron and raymond, 1992a). various conditions should also be met in order to realise these benefits. among the most frequently mentioned are the need for an adequate level of organisational support, the rigour of the implementation process, the presence of control procedures, and the integration of edi with the organisation's internal and external processes (swatman and swatman, 1991; cavaye and cragg, 1995). questions remain, however, about the validity of these research results in the long run. over the last few years, a number of empirical studies dealing with electronic data interchange have been published. among those, most have employed a cross-sectional research design. hence, there is insufficient knowledge about the evolution of ajis special edition-electronic commerce november 1998 edi advantages over time. a number of authors have deplored the lack of longitudinal designs in information systems research. as noted by vitalari (1985), is researchers are confronted by certain difficulties, one of which being the inability to make statements about the nature and characteristics of longer-term effects of information systems. this is presently true in the case of edi. in order to secure the diffusion and effective use of edi by a larger number of organisations, it is important to have a deeper understanding of the various aspects of this technology. the advantages obtained from the use of edi by organisations and the conditions under which these advantages can be obtained are the most critical of these aspects. this research aims at identifying the of edi, the conditions under which this technology can be effective, as well as the variation of these conditions over time. an empirical study on edi was previously conducted (bergeron and raymond, 1992b), aiming to investigate the advantages achieved through the implementation of edi, and the determinants of success. that study provided initial insights into benefits organisations can obtain from edi as well as under what conditions these benefits could be expected. given its cross-sectional nature, however, it did not provide information on the stability of the observations over time. in order to shed some light on this issue, the purpose of the present study is to answer the two following research questions: what are the determinants of edi effectiveness? do these remain the same over time? causality and edi effectiveness: a causal model of edi effectiveness a literature review was conducted to elaborate the research design of the initial study (phase 1) given the newness of the edi field at that time, an inductive approach was used to infer the research model. as part of an on-going effort, the present study can be seen as a validation step. in terms of methodology, it is our contention that a time-based research would be most beneficial at understanding critical edi issues while paving the way for future is research and theory development. the research model for this study includes four constructs: the imposition of edi by business partners, the organisational context (comprised of organisational support, implementation process and control procedures), the level of integration, and the advantages of edi. it was hypothesised that the imposition of edi upon business partners has a direct effect on the organisational context and on edi advantages. the organisational context and the level of integration were also hypothesised to have a direct effect on edi advantages. organisational context organisational context consists of three basic factors. the first one is organisational support. it refers to the various elements and activities conducted by an organisation to assure the effective introduction of edi. it addresses personnel training, top-management support, and the presence of an organisational structure specific to edi (banerjee and golhar, 1994). the second factor is the implementation process. it includes aspects such as the planning approach to edi adoption (eg. prototyping or pilot-project) and the evaluation of initial transactions (eg. in-house tests, error rate, transaction costs) (skagen, 1989). the last contextual factor is the control procedures. data integrity, data security and standards are the main elements included here (adolphson and hansen, 1991). integration the second construct is the integration of edi functions both internally and externally. internal integration of edi reflects the variety of the value chain functions (orders, shipments, inventory, etc.) interconnected through edi within the organisation (sanders, 1992). external integration refers to the variety of the trading partners (eg., clients, suppliers, distributors, etc.) with which the organisation interacts through edi ( zaheer and venkatraman, 1994). imposition by partners imposition by partners refers to a firm's obligation to implement edi to satisfy a major client or supplier. firms resist implementing edi under pressure, because of the difficulty of controlling critical aspects of the project such as the timing of the implementation, the edi function, and the technology involved. organisations of all sizes may have to adopt edi only because they have been obligated to do so by a business partner. expected benefits may vary depending upon the firm's bargaining power (riggins and mukhopadhyay, 1994). edi advantages the advantages of edi, the dependent construct in the research model, includes various types of operational, managerial and strategic benefits that can be derived from edi use. these advantages relate to improvements in ajis special edition-electronic commerce november 1998 the firm's administrative costs, transaction cycle, operations management, information quality and competitive position (o'callaghan, kaufmann and konsynski, 1992; raymond and bergeron, 1996). research hypotheses six variables (organisational support, implementation process, control procedures, internal and external integration, and imposition) were expected to explain the dependent variable (advantages of edi). additionally, imposition is expected to affect the organisational context. the research hypotheses are as follows: hypothesis #1: the quality of the organisational support is positively related to the advantages obtained from edi a favourable organisational environment is a necessary condition to obtain advantages from edi. the support of top-management, the training of personnel and the presence of a dedicated organisational structure have been reported as important dimensions supporting the introduction of edi (klein, 1992) and business process reengineering (champy, 1995). hypothesis #2 the quality of the implementation process is positively related to the advantages obtained from edi as with other types of technology, implementing edi requires a series of activities. these include analysis, testing and implementation, with approaches such as prototyping (senn, 1992). moreover, these activities become critical when introducing inter-organisational applications, such as computerised reservation systems (raymond and bergeron, 1997). it is expected that a similar relationship will be observed in the case of edi. hypothesis #3: the quality of the control procedures is positively related to the advantages obtained from edi. control procedures are meant to assure data security and integrity. making sure that data is valid and meets acceptable standards of quality, will prevent costly errors, allow quick corrections when needed and contribute to a stable organisational climate as well as positive relationships with business partners. adequate control procedures for edi are therefore needed and they should help produce the intended advantages (sokol, 1989, 1995). hypothesis #4: the level of internal integration is positively related to the advantages obtained from edi. economies of scale, improved production cycle and decreased transaction time are among the possible advantages of integrating a greater number of activities within the firm's value chain. this can also be true for quality improvements and inventory reduction (kekre and mukhopadhay, 1992). hypothesis #5: the level of external integration is positively related to the advantages obtained from edl the integration of edi with all types of upstream and downstream partners is also considered important (mackay, 1992). increased external links allow the firm to capitalise on the investment made in edi. higher potential rewards can be obtained by firms implementing strategic applications (such as edi) with an outward rather that an inward orientation (bergeron, buteau and raymond, 1991). hypothesis #6: the level of imposition by business partners is negatively related to the advantages obtained from edl when implementing edi is not a deliberate choice, crucial aspects of this implementation, such as the targeted business function, the timing and the choice of the value added network are not all under the firm's control. benefits can thus be more slanted toward the firm that initiates edi or imposes edi on others. for instance, the implementation of edi by a large manufacturer can significantly affect the bargaining power of distributors and thus lessen their advantages (raymond and blili, 1997). hypothesis #7: there is a negative relationship between the level of imposition by business partners and the quality of the organisational context earlier research (bergeron and raymond, 1992b) has suggested the existence of a negative link between the level of imposition and the organisational context of edi. indeed, the organisations upon whom edi is imposed ajis special edition electronic commerce november 1998 usually have less motivation and time to prepare for implementation of this technology. level of imposition is thus expected to have both a direct and an indirect effect upon the attainment of edi advantages. methodology a two-phased design was used to investigate the research questions and to test the hypotheses. this design consisted of two surveys over a three-year period. in phase 1, after pre-test interviews of ten edi managers, questionnaires were mailed to 560 canadian organisations. following a recall letter, 140 useable responses were obtained for a response rate of 25%. three years later (phase 2), the same questionnaire was sent to 119 of the 140 individuals who participated in phase 1. in phase 1, 21 questionnaires had initially been re-directed to respondents that were not clearly identified in terms of name or mailing address, thus the impossibility of reaching them for the second phase of this study. the cover letter explained the longitudinal aspect of the study, recalled their initial participation and asked again for their collaboration. as in phase 1, a reminder was sent one week after the mailing, followed by phone calls. sixty-five valid questionnaires were returned for a response rate of 55%. measurement of the research variables a number of scales were used to measure the variables hypothesised to influence the attainment of advantages from edi (appendix 1). the organisational support, implementation process, and control procedures were measured using the following statement: " please indicate for each of the following statements the extent to which they are representative of the edi implementation process in your organisation". each scale was anchored on three points (no: i, more or less: 2, yes: 3) and a total score was obtained by averaging on all scales. the level of internal integration was determined by numbering the different types of edi applications internal to the organisation, whereas the level of external integration numbered the different types of trading partners. for that purpose, the respondents were asked to indicate the exact types of edi applications presently implemented (eg., purchasing, billing) as well as the types of business partners presently communicating with the organisation through edi (eg. retailers, suppliers, clients). the imposition scale contained one item asking the respondents to evaluate the truthfulness of the following statement: " the initial decision to implement edi was imposed by your trading partner(s) " (no: 1, more or less: 2, yes: 3). this question was asked only in phase 1 since the first contact of the organisations with edi could not have changed over time. the dependent variable included various benefits thought to be provided by edi, in terms of administrative costs, transaction speed, information quality, operations management and strategic advantage (appendix 2). the individual responsible for edi in each enterprise was asked to rate the scales by the following statement: "please evaluate the impact of the edi implementation relative to the functions presently supported in your organisation", by indicating on a 5-point likert scale to what extent each of these advantages had been obtained by the organisation (strong decrease: 1, slight decrease: 2, no change: 3, slight increase: 4, strong increase: 5). at both points in time (phases 1 and 2), all alpha values of the independent and dependent variables were at or above an acceptable level (.7), except for the strategic advantage measure which had an alpha value of .5 in phase 1, increasing to .7 in phase 2. data analysis following kenny's (1979) approach to correlation and causality in longitudinal research, the data analysis consists of five steps, as follows. the first step tests the stability of the independent variables of the model; it aims at determining if the mean level of factors remained the same over time. the second step concerns the stationarity of the intercorrelations among the independent variables; ideally, the strength of these relationships should remain the same over time. the third step addresses the stationarity of the correlations; its purpose is to investigate the causal structure of the hypothesised relationships over time, hoping that this structure does not change. the research hypotheses are tested at this step using bivariate analysis. the fourth step examines the spuriousness of the correlations; it consists of an exploration of cross-lagged correlations (ie., independent variable at time t] with dependent variable at time t2, and independent variable at time 2 with dependent variable at ti), with a critical look at the direction of any significant differences. the fifth and last step, is the test of the research model as a whole at both points in time using the partial least square (pls) multivariate analysis technique. ajis special edition electronic commerce november 1998 results research results the participating organisations come from various sectors as follows: 43.7% manufacturing, 15.6% distribution, 7.8% transportation, 6.3% wholesale, 4.7% financial institutions and 21.9% other. the respondents hold various management positions: 3.1% are ceo, 26.5% are is managers, 20.3% are managers, 14.1% are edi managers, 9.4% are computer managers and 26.6% others. the organisations had median annual sales of $200,000,000, a median number of employees of 700 and a median annual edi budget of $17,500. their edi relationships were as follows: 76.2% with clients, 42,9% with suppliers, 23.8% with financial institutions, 22.2% with distributors, 18.8% with government agencies, 14.3% with shippers and 9.5% with others. on average, they had been using edi for 6.5 years (whereas it was 3.5 years in phase 1). stability of variables the descriptive statistics for the independent variables of the final research model are presented in table 1. paired t-tests were used to test for the "stability" of the edi success factors, ie. to determine if their mean level had changed over time (kenny, 1979, p. 231). matched sets of data were obtained by keeping only those 65 firms (out of the initial 140) who had answered the survey at both points in time. the results first indicate that on average, the organisational support for edi is perceived to be somewhat less favourable than it was three years ago. this can be attributed to the fact that the organisation is more familiar with edi and the main support activities such as training were mostly realised during the initial implementation. the required level of support thus lessens over time. edi is also presently more integrated internally than it was previously, as more functions other than ordering and billing were added gradually to increase the processing capabilities of the edi system. table 2 presents the descriptive statistics showing the stability of the advantages obtained from edi (dependent variables) over time. the one advantage to have shown a significant increase is the quality of information (in terms of accessibility, accuracy and service to clients). as firms become more experienced and integrate more internal functions through edi, it is plausible that they find new ways and become more adept at extracting valuable information from the transaction data provided by this technology. table 1 descriptive statistics of the independent variables (n=65) organizational context* organisational support implementation process control procedures level of integration internal integration0 external integration"1 imposition by partners6 mean 2.50 2.35 2.37 1.88 1.78 0.65 s.d. 0.3 0.5 0.5 0.9 1.2 mean 2.40 2.39 2.44 2.29 2.06 0.65f s.d. 0.3 0.5 0.4 1.0 1.3 -2.06* 0.43 0.93 2.60** 1.31 *p<.05 **p<.01 ***p<.001 "testing for stability, ie. equality of means (paired t-test) bl=no 2=moreorless 3=yes ctypes of edi transactions made by the organisation dtypes of edi partners linked to the organisation eo=edi implementation is voluntary 1=edi is imposed by one or more business partners ^impositions = imposition,! ajis special edition electronic commerce november 1998 stationarity of intercorrelaiions another comparison can be made by calculating the intercorrelations of the independent variables at ti and t2, as presented in table 3. the "stationarity" of these relationships, ie. determining if their strength has changed over time, can be ascertained by testing for the significance of the differences between the two sets of synchronous correlations (kenny, 1979, p. 231). as suggested by kenny (1979, p. 239), the pearson and filon test for stationarity was used since the standard fischer's z transformation is inappropriate when one is comparing correlated correlations. the results show that, among the organisational context factors, the relationship between the implementation process and the other two variables, namely organisational support and control procedures, has lessened significantly as indicated by the z value. this could be attributed to the former variable's reflection of a prior organisational situation that has since evolved in terms of activities. given that only 2 out of 10 intercorrelations differed significantly, a satisfactory level of stationarity is shown in that regard. table 2 descriptive statistics of the dependent variables (n=65) edi mean s.d. mean s.d. t6 advantages administrative costs transaction speed information quality operations management strategic advantage * p < .05 ** p < .01 3.32 3.43 3.66 3.25 3.24 ***p<.001 0.5 1.2 0.8 0.4 0.4 3.42 3.57 3.99 3.28 3.16 0.5 1.2 0.6 0.4 0.5 1.03 0.74 2.70** 0.38 -1.08 °l=strong deterioration 2=slight deterioration 3=no change 4=slight improvement 5=strong improvement ""testing for stability, ie. equality of mean (paired t-test) stationarity of correlations the research hypotheses can first be studied through bivariate analysis ie., by calculating the correlations between the independent and dependent variables in the final model. this being a longitudinal study, one is interested in ascertaining if the causal structure of the hypothesised relationships does not change over time. again, the key assumption is that of stationarity, ie. that the strength of each relationship remains the same at both points in time. table 4 presents results pertaining to hypotheses 1 to 6 at both points in time, linking six edi success factors to the attainment of five types of advantages. the initial study, based on 140 firms, had confirmed the positive effect of the organisational context factors and the negative effect of imposition on attaining certain advantages from edi, and in particular on the reduction of administrative costs. integration was also shown to be globally determinant, and especially in attaining a strategic advantage from this technology. comparing the two matched sets of synchronous correlation coefficients, the z values obtained from the pearsonfilon statistic allow one to identify the significant changes in the hypothesised bivariate relationships between t, and t2. stationarity can be ascertained overall by observing that only eight correlations out of thirty changed significantly between ti and t2. a first change concerns the lessened impact over time of the implementation process on the managerial benefits of edi (ie. information quality), as opposed to operational benefits (ie. transaction speed). given that global information needs have been analysed initially, that organisational learning has occurred in the determination of more detailed needs and that operations managers have thus integrated edi within their decision-making, the rigour of implementation of additional or complementary edi functions seems less determinant in this regard. ajis special edition electronic commerce november 1998 the second change of note regards the now significant impact of edi control procedures in terms of information quality, operations management and strategic advantage, whereas this effect was absent at ti. as edi integrates more elements of the firm's value chain and more of its partners over time, risks increase and data integrity and security becomes more important, and thus the need to effectively control edi operations. the increased positive impact of internal integration on information quality is another significant change that has occurred in the sampled firms. as more functions such as procurement, inventory, shipping, billing and payments are linked through edi, more data is rendered available, this data is more integrated and can be easily transformed into more comprehensive, timely and relevant information for decision-making. an analogous reasoning can be employed to explain the increased effect of external edi integration on operations management. as edi links the firm's value chain to the value chains of both its upstream and downstream partners, just-in-time (in manufacturing) or quick-response (in retailing) approaches can be implemented to reduce cycle times and improve service. looking at the bottom line of table 3 in regard to hypothesis 7, one sees that the initial negative influence of imposition on the organisational context has now become positive. at tj, firms who implemented edi under duress from their business partners provided a less favourable context. at tj, these same firms now show a significantly higher level of organisational support for edi. from a network perspective, edi advantages can be unequal but also interdependent among business partners of unequal size and power (raymond and bergeron, 1996). organisations upon whom edi was imposed were found here to be significantly smaller than those who did so voluntarily. this change in the effect of imposition could reflect the realisation by larger firms who initiate edi systems that they must eventually provide increased assistance (technical, financial, training) to their smaller partners, if this technology is to function efficiently and effectively to the benefit of all involved. regarding the effect of imposition on edi advantages, partial rather than zero-order correlations were calculated in order to control for the intervening effect of the organisational context variables, assumed in the research model. over time, imposition is shown here to lose its negative impact on administrative costs and operations management. this could be explained by the compensating effect of the increased assistance provided by large pivot firms in edi networks to their partners. the latter could thus learn better and faster in regard to attaining both operational and managerial advantages from edi. another explanation is the possibility for the firm upon whom edi was imposed to impose it in turn to its other business partners. the few benefits they obtained at the outset would thus be compensated later on. spuriousness of correlations while the preceding correlational analysis tests the key stationarity assumption, ie. an unchanging causal structure, it is insufficient to rule out "spuriousness" as an alternative explanation of causal effects (kenny, 1979, p.229). non-spuriousness can be ascertained by testing for the equality of the cross-lagged(as opposed to synchronous) correlations, using the same pearson-filon statistic. rozelle and campbell (1969) have pointed out the difficulty in interpreting significant cross-lagged differences. the results presented in table 5 show that only four out of twenty-five possible differentials are non-zero. the level of spuriousness of the model was thus considered acceptable. note that the imposition variable was not included in the cross-lagged analysis because it was assumed not to change over time. given the equality of synchronous correlations in the corresponding four cases, the interpretation problem would center on the validity and stability of the variables involved (kenny, 1973, p. 234). one could eventually interpret the four relationships in the reverse causal order or even as a positive feedback loop. in this case, for instance, it would be plausible that attaining operational and strategic advantages in phase 1 would "cause" a firm to add new edi functions and partners, and thus lead to an increased level of integration in phase 2. 10 ajis special edition electronic commerce november 1998 table 3 inter-correlation of the independent variables (n=65) organizational context org. context org. context impl. process control proc. org. support ti-n 12-12 za il-il 12-12 z tl-ll 12-12 z impl. process .63*** .18 3.25*** . control proced. .28 .15 .79 .56*** .07 3.34*** . integration internal integr. .02 .08 -.35 .01 -.18 1.12 .22* .40*** -1.16 external integr. -.07 .01 -.46 -.06 -.10 .23 .17 .19 -.12 level of integration internal integr. il-il 12-12 z .55*** .48*** .54 impos. by partners"tl-ll _ 1 9 ii-iz 24* z -2.91** -.21* .14 -2.30* -.21* .07 -1.78* -.17 -.15 -.13 -.13 -.16 .19 *:p<. 05 **:p<.01 ***:p<.001 testing for stationarity, ie. equality of synchronous correlations (kenny, 1979) with: z = (n)1/2(r,4-r23)/((l-ri42)2+(l-r232)2-k)1/2 k = i where: variable 1: indep. var.,i variable 4: other indep. var.,i variable 2: indep. varg variable 3: other indep. var.^ "imposition^ = imposition,! 11 aj1s special edition electronic commerce november 1998 table 4 synchronous correlation of independent with dependent variables (n=65) edi advantages org. support implement. process control procedure internal integr. external integr. impos. by partners'" *: p < .05 ti-ti c2-t2 za tl-tl 12-12 z tl-tl 12-12 z tl-tl 12-12 z tl-tl (2-t2 z tl-tl tl-t2 z **:p<.01 admin. costs .34** .24* .61 .31** .23 .49 .41*** .38*** .20 .16 .25* -.53 .03 .29* -1.54 -.40** .17 -3.39*** ***:p<.001 trans. speed .19 .09 .60 .15 .39** -1.51 .19 .14 .30 .09 .06 .17 .20 .12 .46 -.13 -.03 -.62 info. qual. .26* .02 1.42 .19 -.15 2.01* .09 .34** -1.52 -.04 .26* 1.77* -.15 .18 -1.92* -.14 -.22 .48 oper. manag. .26* .06 1.20 .06 -.24 1.81* .12 .27* -.90 .07 .16 -.52 .12 2q*#* -1.68* -.26* .02 -1.65* strat. advant. -.01 .15 -.92 -.29* -.16 -.78 -.07 .25* -1.89* .19 .21 -.12 .30** .18 .73 -.05 -.14 .54 testing for stationarity, ie. equality of synchronous correlations (kenny, 1979) with: z = (n)1/2(r14-r23)/((l-r14 2)2+(l-r23 2)2-k)1/2 k = (r where: variable 1 : edi success factor^ variable 4: edi advantage,! variable 2: edi success factor^ variable 3: edi advantage^ 'imposition^ = imposition,! (partial correlation, controlling for organisational context variables) 12 ajis special edition electronic commerce november 1998 table 5 cross-lagged correlation of independent with dependent variables(n=65) edi advantages org. support implement. process control procedure internal integr. external integr. impos. by partners1" *:p<.05 **: tl-t2 t2-tl za tl-12 12-11 z tl-t2 i2-tl z tl-12 12-tl z u-t2 12-11 z tl-t2 ll-tl z p<.01 admin. costs .05 -.01 .35 .03 .13 -.58 .18 .00 1.08 -.08 .26* -2.01* .03 .05 -.11 _ ***:p<.001 trans. speed .24* .09 .89 .05 .16 -.65 .01 .09 -.46 -.07 .18 -1.46 -.09 .26* -2.09* _ info. qual. .03 -.10 .75 .04 .17 -.75 .07 -.06 .75 .04 .12 -.46 -.10 .12 -1.26 . oper. manag. -.11 -.16 .29 -.28* -.19 -.54 .02 -.20 1.30 .04 .08 -.23 .19 .06 .76 _ strat. advant. .09 -.06 .86 .00 -.28* -1.68* -.05 .12 -.97 -.05 .27* -1.91* .02 .08 -.35 _ "testing for spuriousness, ie. non-equality of cross-lagged correlations (kenny, 1979) with: z = (n)"2(ri4-r23)/((l-r,42)2+(l-r23 2)2-k)1/2 k = (r where: variable 1: edi success factor,] variable 4: edi advantage^ variable 2: edi success factor^ variable 3: edi advantage,] 'imposition^ = imposition,] (cross-lagged analysis is inappropriate for variables that do not change over time) multivariate analysis a more comprehensive and powerful validation of the research model involves the multivariate analysis of the structural equations. the partial least squares (pls) technique was chosen for its emphasis on theory development and its robustness, as it allows for small samples, non-interval scaled and non-multivariate normal data (higgins, barclay and duxbury, 1991). two sets of correlations and path coefficients corresponding to the research hypotheses were thus obtained. the notion of "edi advantages" is defined as a global construct (latent variable) whose meaning is reflected in the five advantages types measured in both the initial and the present study. hence, the results of the two pls analyses also provide the loading for each type of advantage, ie. its correlation with the global construct. the validity and explanatory power of the research model can be assessed by comparing the path coefficients and the loadings obtained from the tj and t2 data. as opposed to the bivariate analysis, all 140 firms in the initial sample were kept for the pls analysis of the time 1 data because no statistical tests requiring paired data were involved. these results are presented in table 6. the first research hypothesis of the initial study remains confirmed at the second point in time, as indicated by significant path coefficients. there is thus strong evidence for a causal link between a firm's level of organisational support for edi and its attainment of advantages from this technology. this is not so however, in regard to hypothesis 2. the significant effect of the implementation process at t] is attenuated three years later. 13 ajis special edition. electronic commerce november 1998 the rigour of the edi implementation process is determinant for the initial applications but not for later ones, probably because of the experience previously gained by the firm. conversely, edi control procedures take on added importance over time, hypothesis 3 being more strongly confirmed at t2 than earlier. as edi affects more of the firm's core operations and strategy over time, questions of data security and integrity should take precedence in determining the successful use of this technology. overall, a causal link between the organisational context and the benefits obtained from edi seems present, with a relatively stable effect of organisational support, a decreasing effect of the implementation process and an increasing effect of control procedures over time. as for hypothesis 4, the path coefficient initially linking internal integration to edi advantages loses its significance over time. note however that the corresponding correlation coefficient between these two variables remains significant. to interpret this result, one should remember that the mean level of internal integration has significantly increased over the observed time period (table 1) and that it is strongly correlated to external integration (table 3). as predicted in hypothesis 5, greater upstream and downstream integration with multiple business partners would lead to greater benefits from edi. adding a new type of edi partner (eg. transporters) might also mean that new organisational functions (eg. shipping, receiving) are integrated through edi technology. initially, both variables are equally determinant, whereas internal integration is preempted by external integration at the second point in time. contrary to the sixth research hypothesis confirmed in the initial study, imposition has after a while no significant impact on the attainment of edi advantages. with the assistance of their partners, firms that envisioned edi only as an added cost to satisfy important clients or suppliers possibly learn to better manage and seize the opportunities provided by this technology. the same can be said regarding the effect of this variable upon the organisational context of edi (hypothesis 7). initially, firms who did not implement edi voluntarily presented a less favourable context, a less rigorous implementation process and less control procedures; this is not the case now. in fact, as discussed in the bivariate analysis (table 3), imposition seems to have a delayed positive impact on the level of organisational support for edi that can be attributable, for instance, to outside assistance obtained from a business partner. the results of pls analyses also provide reliability and validity assessments. looking at the initial loadings of the edi advantage measures, only one did not reach the .5 level for acceptable reliability, namely the strategic advantage. this last measure was more laden with random error, with a cronbach's alpha of .50, at t], compared to .70 at t2. however, it could not be removed or revised due to the longitudinal nature of the study. while the reliability of the strategic advantage measure did in fact increase sufficiently in the second phase of the study, the loading of the transaction speed measure did the opposite by falling below the .5 level. the changes in the loadings could be interpreted more substantively by the relative importance of certain types of edi advantages at different points in time. initially, the operational aspect would be most important, ie. the firm wants to reduce costs and time delays. later, managerial and strategic aspects come into play, ie. the firm realises it can manage its operations better and gain a competitive advantage through edi. the overall validity of the measurement model is confirmed at both points in time by the significant and comparable percentages of variance in edi advantages that are explained by the independent variables (27% at ti and 30% at 12). as evidence for discriminant validity, the average variance extracted by the measures of edi advantages is greater than the variance shared between this construct and all other constructs in the research model (obtained by squaring the correlations in table 6). finally, the edi advantages construct shows an acceptable level of internal consistency, using a reliability coefficient (rho) developed by fornell and larcker (1981). in summary, the proposed causal model shows a satisfactory level of stationarity over time, both with respect to intercorrelations and correlations among the independent and the dependent variables. in addition, the low number of cross-lagged correlations between the two phases (ie., the non-spuriousness) supported the causal direction of the model. 14 ajis special edition electronic commerce november 1998 table 6 path analyses (pls) of the research model (n=140) (n=65) t, (n=140) t2 (n=65) organisational support (h1) with edi advantages implementation process (h2) with edi advantages control procedures (h3) with edi advantages internal integration (h4) with edi advantages external integration (h5) with edi advantages imposition by partners (h6) with edi advantages imposition by partners (h7) with org. support with impl. process with control proc. correlations .290*** .164 path coefficients .129*** .115* .283*** .272*** .301*** .309*** -.317*** .181* .192** .159* -.021 .421*** .293** .378*** .067 .245* .065 .065 .145*** .081*** .167** .152*** -.219*** -.181*** -.192*** -.159*** -.106 .352*** .036 .312*** .073 .245*** .065 .065 loadings edi advantages administrative costs transaction speed information quality operations management strategic advantage r square aver, variance extracted" reliability coefficient1* .909 .641 .553 .462 .195 .267 .449 .704 .759 .375 .666 .786 .629 .296 .435 .785 *:p<.05 **:p<.01 ***:p<.001 ji2/n bzi=1.nloadingi) 2/((li=1.nloadingi) 2+ii=1.n(l-loadingi 2)) conclusion longitudinal results on edi implementation were obtained by twice surveying 65 organisations over a three-year period. in phase 1, all factors are positive determinants of edi except for imposition which shows a negative effect on both edi advantages and organisational context variables (organisation support, implementation process and control procedures). in phase 2, the implementation process, the level of internal integration and imposition by business partners lose their importance. in this second phase, organisational support, control procedures and external integration are the only significant direct determinants, with imposition having a unique positive effect on organisational support. in a way, security and external integration could become the major stumbling blocks of electronic commerce and therefore much care should be taken to improve these crucial aspects of electronic commerce. having observed edi in organisations over a three-year period, this study has implications for both researchers and practitioners. the basic validity of a research model that includes the organisational context, integration and 15 aj1s special edition-electronic commerce november 1998 imposition of edi as essential determinants of success has been confirmed, with evidence of causality provided by the longitudinal nature of the study. increased advantages can be attained through edi, if organisations can be persuaded to implement, use and manage this technology as an opportunity to enhance business processes and partnerships. using this model as a foundation, we see that further research is needed. the model could be enriched to include other potential factors emanating from a network point of view, or from a broader theoretical perspective. added work should also be done on the measurement of edi advantages, especially those of a strategic nature. from a practical point of view, there are still too few organisations that have truly profited from electronic commerce. too many firms, especially smaller ones, still consider edi simply as an added cost of doing business. as edi migrates into the internet and on to extranets, future extensions of ec networks initialised by large organisations and normalisation efforts should include ec planning, implementation and training frameworks. these frameworks would guide firms on how to provide a favourable context for ec, and how to better integrate their intra and inter-organisational systems to fully benefit from this new form of commerce. references adolphson, d. and hansen, j.v. "evaluating software controls for edi". european journal of operational research, 51, 1, march 1991,42-46. banerjee, s., and golhar, d.y., electronic data interchange: characteristics of user and nonusers, information & management, 26 (1994) pp. 65-74 bergeron, r, buteau, c., raymond, l. "identification of strategic information systems opportunities: applying and comparing two methodologies,".mis quarterly, vol. 15, no. 1, march 1991. bergeron, f. and raymond, l. "planning of information systems to gain a competitive edge". journal of small business management, 30,1, 1992a, 21-26. bergeron, f. and raymond, l. "the advantages of electronic data interchange". data base, 23, 4, 1992b, 1931. cavaye, a.m. and cragg, p.b., « factors contributing to the success of customer oriented interorganizational systems », journal of strategic information systems, (4,1) march 1995, pp. 13-30. champy, j., reengineering management, harper business, new york,. fornell, c. and larcker, d. "evaluating structural equation models with unobservable variables and measurement error". journal of marketing research, 18, 1981, 39-50. gebauer, j., «internet-based edi», briefings paper 96-bp-001, fisher center for information technology and management, december 1996. higgins, c.a., barclay, d.w. and duxbury, l.e. "an introduction to the partial least squares approach to causal modeling". working paper, school of business administration, university of western ontario, london, canada, 1991. kalakota, r. and whinston, a.b. frontiers of electronic commerce .addison-wesley publishing company, new york, 1996. kekre, s. and mukhopadhyay, t. "impact of edi technology on quality improvement and inventory reduction programs: a field study". international journal of production economics, 28, 3, december 1992, 265282. kenny, d.a., correlation and causality, john wiley and sons, n.y., 1979. klein, s. "a conceptual framework for the assessment of edi". proceedings of the 25th hawaii international conference on system sciences, vol. iv, kauai, hawaii, 1992, 369-379. mackay, d.r. "the contribution of edi to the structural change of the australian automotive industry". proceedings of the third australian conference on information systems, university of wollongong, australia, 5-8 october 1992,633-652. mukhopadhyay, t., kekre, s., kalathur, s., « business value of information technology : a study of electronic data interchange, mis quarterly, (19 :2), june 1995, pp. 137-156. o'callaghan, ramon, kaufmann, patrick j., konsynski, benn r., "adoption correlates and share effects of electronic data interchange systems in marketing channels", journal of marketing, vol. 26, no. 2, april 1992, pp. 45 56. raymond, f. and bergeron, f. "edi success in small and medium-sized enterprises: a field study", journal of organisational computing and electronic commerce, 6, 2, 1996, pp. 161-172. raymond, l. and blili, s. "adopting edi in a network organisation: the case of subcontracting smes". european journal of purchasing and supply management, 3, 3,1997, pp. 165-175. raymond, l. and bergeron, f. "global distribution systems: a field study of their use and advantages in travel agencies". journal of global information management, 5,4, 1997, 23-32. 16 ajis special edition • electronic commerce november 1998 riggins, f.j. and mukhopadhyay, t. "the impact of unequal interdependent benefits from intel-organizational systems". proceedings of the 27th hawaii international conference on system sciences, vol. iv, wailea, hawaii, 1994, 875-884. rozelle, r.m. and campbell, d.t. "more plausible rival hypotheses in the cross-lagged panel correlation technique". psychological bulletin, 71, 1969, 74-80. sanders, n.r. "merging edi with jit: the impact on u.s. manufacturing". journal of applied business research, 8, 2, spring 1992, 133-137. skagen, a., "nurturing relationships, enhancing quality with electronic data interchange", management review, vol:78, no. 2, february 1989, pp. 28-32. senn, j.a. "electronic data interchange: the elements of implementation", information systems management. winter 1992,45-53. sokol, p.k., edi: the competitive edge, mcgraw-hill, new-york, 1989. sokol, p.k. from edi to electronic commerce a business initiative, mcgraw hill, new york, 1995. swatman, paula m. c, swatman paul a., "integrating edi into the organisation's systems: a model of the stages of integration", proceedings of the 12th international conference on information systems, new york, 1991, pp. 141-153. venkatraman, n., zaheer, a., "electronic integration and strategic advantage: a quasi-experimental study in the insurance industry, information systems research, vol. 1, no. 4, december 1990, pp. 377-393. vitalari, nicholas p., "the need for longitudinal designs in the study of computing environments", research methods in information systems, opus citus, 1985, pp. 243 265. wheatman, v., « edi over the internet: mckesson case study », the gartner group electronic commerce strategies research note , june, 1996. zaheer, a. and venkatraman, n., "determinants of electronic integration in the insurance industry: an empirical test, management science, (40:5), may 1994, pp. 549-566. acknowledgments the authors wish to thank the government of quebec (fcar), and the edi council of canada for their collaboration and support. an earlier, abridged version of this paper was presented at 9th international conference of the information resources management association, boston, massachusetts, 1998. 17 ajis special edition-electronic commerce november 1998 appendix 1 organisational context of edi organisational support training of top-management training of department heads training of data processing personnel training of users top-management is favourable top-management supports the implementation project working group responsible for implementation edi managed by a specific organisational unit individual(s) specifically assigned to manage edi cooperation with commercial partners during implementation cooperation among departments during implementation implementation process analysis of strategic and operational impacts of edi use of pre-defined transmission standards and protocols availability of an edi implementation guide development of a prototype before implementation pilot project conducted before definitive implementation in-house tests, i.e. only within the organisation external tests, i.e. with trading partners analysis of initial transactions appreciation of system quality by users measure of response time measure of error rate measure of transaction costs control procedures transaction entry and control tasks assigned to different individuals transactions recorded before transmission transaction acknowledgments by partners are recorded backup of transactions backups stored outside the organisation transaction entry and safekeeping tasks assigned to different individuals only formally authorised personnel may use edi access code necessary to use edi authorisation required for special transactions, e.g. exceeding a certain limit user authorised for limited types of transactions transactions bear a user stamp transactions bear a date stamp transactions bear a time stamp transactions made only within pre-determined time period 18 aj1s special edition electronic commerce november 1998 appendix 2 advantages of edi administrative costs transaction costs forms costs forms manipulation costs forms storage costs postal costs telephone costs office personnel costs inventory level transaction speed transaction transmission time transaction processing time information quality transaction errors quality of service to clients timeliness of transaction information accessibility of transaction information accuracy of transaction information operations management operation cycle time decision cycle time price of products or services offered strategic advantage entry barriers for newcomers to the industry exit barriers for trading partners difficulty for competitors to offer similar edi services 19 30 assessing software cost estimation models: criteria for accuracy, consistency and regression bruce w.n. lo and xiangzhu gao school of multimedia and information technology southern cross university lismore, nsw australia 2480 email: blo@scu.edu.au abstract one of the problems in software cost estimation is how to evaluate estimation models. estimation models are usually evaluated against two attributes: estimation accuracy and estimation consistency. a number of measures are reported in the literature, but they have shortcomings. there is no generally accepted standard to evaluate estimation models and the existing measures sometimes are not consistent among themselves. this paper examines existing measures of estimation accuracy and consistency and proposes two new ones: the weighted mean of quartiles of relative errors (wmq) as a measure of accuracy and the standard deviation of the ratios of the estimate to actual observation (sdr) as a measure of consistency. besides, a new regression criterion is proposed to determine model parameters. the proposed measures and criterion were tested with a data set from real world software projects. results obtained show that these new measures and criterion overcome many of the difficulties of the existing ones. introduction since the early 1950s, software development practitioners and researchers have been trying to develop methods to estimate software costs and schedules (abdel-hamid, 1990). software cost estimation models have appeared in the literature over the past two decades (wrigley et al, 1991). however, the field of software cost estimation is still in its infancy (kitchenham et al, 1990). the existing cost estimation methods are far from standardised and reliable (rowlands, 1989). there is a need to evaluate estimation models and improve modeling processes. this paper will focus on how to quantitatively evaluate software cost estimation models. a new approach to determine model parameters is also proposed. in the field of software cost estimation, estimation models are usually evaluated against two attributes: estimation accuracy and estimation consistency. the rules or measures needed to describe these two attributes will be discussed in this paper. measurement may be used for two purposes: assessment and prediction (fenton, 1994). for assessment, the results of comparing different models may be used to judge which theory is more successful at explaining the behaviour of cost factors (pfleeger, 1991). successful models would provide further insight into software development processes. to assess models, we need to have common standards. although some measures for estimation accuracy and consistency have been introduced in the literature, they are not generally accepted for model assessment (verner et al, 1992). sometimes these measures are not consistent among themselves. for example, a model may be better than another one with respect to one measure but poorer with respect to a different measure. for prediction, the measurement may provide feedback to improve the modeling process so that the model can satisfy the rules of measurement as far as possible. modeling is associated with measurement (kan et al, 1994). in this context, it is necessary to define the procedures for determining model parameters and interpreting the results (fenton, 1994). this leads to the problem of how to determine the model parameters. unfortunately there is also no generally accepted criterion for researchers to follow in the modelling process. this paper, therefore, will also discuss the criteria for determining model parameters. the paper is organised into three parts. the first part examines the measures used to evaluate estimation models. to overcome the shortcomings of existing practice, two new measures are proposed: the weighted mean of quartiles of relative errors (wmq) which provides a measurement of accuracy and the standard deviation of the ratios of the estimate to actual value (sdr) which provides a measurement of consistency. the second part examines traditional mathematical procedures for formulating costing models and proposes a new regression criterion called least sum of logarithmic ratios of estimate to actual value. this is an unbiased method of finding parameters in a cost estimation model when the functional form of the model is known. the third part assesses the proposed measures and criterion. a data set from real world software projects was used to examine the proposed measures and the regression criterion so as to demonstrate their advantages over the other measures and criteria currently reported in the literature. ajis vol s no 1 september 1997 31 measures of accuracy accuracy is defined as the measure of how close a result is to its correct value (deeson, 1991). there are two ways to compare a result and its correct value: their difference and their ratio. let n be the number of projects in a data set, acti be the /'* (i= 1,2,3,...,") actual observed value and estj be the corresponding estimated value. the difference measure of estimation accuracy is based on the difference between estimated value and actual value esti act{ (i=l,2,3,...,n) the ratio measure of accuracy is based on the ratio of estimated value to actual value est{ '(j=l,2,3,...,n) actf in evaluating the accuracy of software cost estimation models, both difference and ratio measures have been used. these are discussed below. difference measures of accuracy (1) mean of absolute errors (mae) i n mae=— \\act. -esti " w (2) root mean of squares of error (rmse) n 1=1 (3) coefficient of determinant (r2 ) _ 1 » /• _ where act = — y acti is the mean of n actual observed values. for a given set of data \ (act, -act)2 is a n 1=1 1=1 constant. therefore r is a distance measure. (4) mean of residues (mr) 1 " mr = ̂ (estf ~ acti ) « 1=1 ratio measures of accuracy (1) mean (or average) of relative errors (are) n ~ act, ajis vol 5 no 1 september 1997 32 (2) mean of magnitude of relative errors (mre) act: est: act 1est. act. (3) root mean of squared relative errors (rmsre) rmsre = \many measures are based on magnitude of relative errors, mrei mre, = act. est: act: mre is the most widely used measure in the literature. however, it is influenced by extreme values or outliers. a smaller mre is not always better than a larger one (j0rgensen, 1994). to address this problem, some single values of mrei (i=\,2,3,...,ri) have been used to measure the estimation accuracy. they are described below. (4) prediction at level / (pred(/)) conte el al (1986) put forward a single value measure, prediction at level / pred(l) = — n where k is the number of projects in a set of n projects whose mre < i. they suggested that an acceptable accuracy for a model is pred(0.25) > 0.75, which is seldom reached in reality. (5) third quartile of mre (q3) conte et al's standard can be expressed in another way: 75% of the mre's are less than or equal to 0.25. in terms of quartiles, the third quartile, q3, of the mre's is less than or equal to 0.25, i.e. q3 < 0.25. as mentioned above, this is too high a standard at present, but it should be a goal to pursue. the smaller the <23, the more accurate the estimation. both qj anc' pred(0.25) avoid considering some extremely poor predicted values. so they eliminate the influence of extreme values. (6) other measures the median of mre's (q2 , second quartile) has also been used as one of the measures of accuracy (j0rgensen, 1994). it is only a middle number among the ordered mre's, so it hardly presents an average of all the mre's. miyazaki era/ (1991) believed that mre based measures favour underestimation. (this problem will be discussed in detail later.) to address this problem, they defined the relative error in another way: est — act r. =• r =• esti act( est: { acti > 0) (est. acti < 0) ajis vol 5 no 1 september 1997 33 they argued that by using /",, a balanced evaluation is obtained for both underestimated and overestimated cases. with the above definition of relative error, mre, rmsre or pred(0.25) can be used for accuracy evaluation (miyazaki et al, 1991). fenton (1994) argued that good measurement should be meaningful. in practice, it is generally accepted to express relative error by comparing error with actual observed value. with this commonly accepted idea of relative error, if the relative errors are 0.40, 0.50, 0.60 and 0.70 in the case of underestimation, the corresponding r values are 0.67, 1.00, 1.50 and 2.33. in an extreme case, if the relative error is 0.95, which is not unusual, the r value can be as high as 19.00. the variance of r( must be large and the outliers take an important role in mre. the large variance will also make rmsre and pred(0.25) values quite different from those commonly accepted. moreover this measure penalises underestimation too much. jenson and hartley (1991) proposed the weighted mre (wmre). they believe that one weakness of mre is that the relative size of each individual observation is not recognised. wmre is calculated such that each observation's relative error is weighted by the ratio of the observed value to the average of the observed values. they argued that by weighting larger projects more heavily than smaller ones, the wmre recognises the greater cost associated with estimation errors on larger projects. however, on closer examination, we can prove that their argument is untenable since act w*rr act act,-est: act; 1 nact^ act, est: in this expression, it does not appear that larger projects are weighted more heavily than smaller ones. on the contrary, the weight of project size included in relative error disappears from the formula. so it falls into the category of difference measure. occasionally, a combination of difference measure and ratio measure is used for accuracy. j0rgensen (1994) introduced pred2(0.25,0.5), the percentage of projects with mre < 0.25 or estimation error < 0.5 days. pred2(0.25,0.5) is actually pred(0.25), because "estimation error < 0.5 days" only contributes to the percentage when the estimation is performed for a project of less than two days. discussion on measures of accuracy kitchenham et al (1990) indicated that we should distinguish between ̂ z/ accuracy and prediction accuracy, i.e. how well a model fits the data from which it was generated and how good a prediction from the model will be. to assess fit accuracy, r is a good measure. it can be seen from the definition of r that it is related to the variance of actual observed values. the larger this variance is, the easier it is to obtain a large r . r measures the proportion of the variation in the dependent variable that is explained by the regression equation (kenkel, 1989). the higher the value of r , the greater the explanatory power of the regression equation. when r is close to 0, either the functional form of the model does not fit the data set from which it was generated or more independent variables are needed to further explain the variation in the dependent variable. to assess prediction accuracy, difference measures are not suitable. with respect to software cost estimation, the prediction error increases with the magnitude of the observed value. the larger the project is, the harder it is to estimate the effort. difference measures are not adequate if they are used for both large projects and small ones, because they do not take into consideration the size of projects. therefore difference measures should not be used to assess the prediction accuracy for they penalise the prediction for large projects. on the other hand, relative error is an average of prediction error in every unit of effort, which reflects "error rate". it takes into account the project size and allows larger absolute error for larger projects. therefore, ratio measure is more suitable for the assessment of the accuracy in software cost estimation. among the ratio measures, all mean measures are influenced by very large extreme mre values, which usually occur in software estimation. in the context of accuracy evaluation for software cost estimation, they cannot play the role of the measure for central tendency. some of them favour underestimation. for the single value measures, although they eliminate the influence of outliers, they can hardly provide an overall accuracy. for example, pred(0.25) takes into account less than 75% of all the wire's, because conte's acceptable level can seldom be achieved. although q3 indicates that 75% of the projects are estimated with mre less than or equal to <23, it does not provide any detailed information on these mre values. for the purpose of comparison among different models, the single value measures are rough and stochastic. moreover, these single value measures and mean measures are not always consistent when they are used to compare different models. ajis vol 5 no 1 september 1997 34 the above discussion may be summarized in three points: • the difference measure is not suitable for the evaluation of software cost models. the ratio measure is preferable. • the mean measures take into account every single relative error, but they are significantly influenced by outliers, which frequently occur in software cost estimation. some of them favour underestimation. • single value measures are not influenced by outliers, but they are stochastic and lose too much information. therefore the evaluation by single value measures is not always reliable when they are used to compare different models. because of the above drawbacks of the existing accuracy measures, there is a need for a better measure for accuracy evaluation. a new measure of accuracy weighted mean of quartiles of mre's (wmq) to resolve the above problems, it is proposed to use the weighted mean of quartiles (wmq) of mre to measure the prediction accuracy. the third quartile (q3) is the most important as 75% of mre's values are less than it. so it is weighted with 75. the second quartile (q2) is weighted with 50, and the first quartile (q,) is weighted with 25. the wmq is defined as: wmq 150 6 there are two assumptions underlying the use of wmq: 1) the number of outliers is less than 25% of all the mre values. this assumption is generally true. 2) if the estimation of 75% of the projects is acceptable, the model is desirable. because of the present low level of poor estimation accuracy, this assumption is reasonable. unlike the mean measures, the wmq is not influenced by extreme mre values, and at the same time it provides more general information about the distribution of mre than the single value measures. it will be shown later that the wmq presents a good average of mre in evaluating accuracy of software cost estimation. the wmq is consistent with the mre provided that the estimation is not obviously biased (tending to under/overestimate) and there are no outliers (extreme large values). in a sample of n projects an estimation model results in n relative errors, the magnitude of which is expressed as a variable mre. these mre's ait arranged in ascending order. the mre can be expressed as a function of the order number n, i.e. mre =f(n) with this equation a smoothed curve ( oq^q2q^q4 ) can be drawn. this curve is illustrated in figure 1, where (2i is the first quartile of mre and also expresses the intersection of the curve and the vertical line n = nq , where nq (= 0.25/0 >s me ordinal number of q\ , and so on. through point o and point q3 a straight line is drawn to intersect with vertical line n = nal point q4 . the vertical line n = nqi meets with straight line o q4 at point q} . point q2 is obtained in the same way. ajis vol 5 no 1 september 1997 35 mre o figure 1 relationship between wmq and mre n. n n if there are no outliers with respect to mre we assume that the area oqtq2q3q2qt is approximately equal to the area q^q4q4. therefore the area al under the smoothed curve is approximately equal to the area a2 of triangle oq4n. let k be the slope of straight line o q4 . 4mre = — n n n if we assume ql ~ q} and q2 ~ q2 , then wmq = 2q2 + 3q3 * 2q2 6 025kn + 2(0.50kn) + 3(0.75*n) = 6 in this particular case, as ql < q} and q2 < q2 in figure 1, wmq < 0.58/72. in the situation where there is a tendency to underestimate, the front part of the curve goes up and the rear part comes down. hence, ql is close to £?i and q2 is close to q2. wmq may be greater than o.sfcn. that is wmq > mre (the mre favours underestimation). for an unbiased estimation, ql and q2 are relatively further from ql and q2 respectively. wmq may be closer to o.skn. that is wmq = mre. this explains why wmq and mre are consistent when there are no outliers and the estimation is unbiased. measures of consistency a model that is sensitive to the influence of various productivity factors may nonetheless consistently overestimate or underestimate development, if the standard productivity rate assumed by the model is significantly different from that of the environment in which the software was developed (mukhopadhyay et al, 1992). models developed in different environments do not work very well without calibration. a consistently overestimating or underestimating model is easier to calibrate than an inconsistent one. therefore, besides accuracy, consistency is another important feature for an estimation model. correlation coefficient of estimates and actual values (sdr) to measure the level of consistency, some researchers have used the correlation coefficient, sdr, between observed and estimated values (mukhopadhyay et al, 1992). this measure tests the linear association between the actual values and estimates. for a highly consistent model, r should be close to 1 (-!</?< 1), otherwise it is ajis vol 5 no 1 september 1997 36 close to 0. if r is negative, it indicates that larger actual values are associated with smaller estimates. r is the square root of r , the coefficient of determination introduced earlier. so sdr is not consistent with the ratio measure which has been illustrated to be more suitable for software cost estimation. moreover, as r2 , sdr is influenced by the variance of data. the greater the variance of actual values, the larger the denominator in the expression. r varies according to not only the estimation accuracy but also the variance of data. we need a consistency measure, which assesses the estimation on the basis of ratio measure and is independent to the distribution of the actual observations. a new measure of consistency suppose that a model was developed in environment a and a set of data, which was collected from software projects developed in environment b, is used to test the estimation consistency of the model. for each estimation there is a ratio est. (i = 1.2,3 ..... n) acti in the case of consistent estimation, the values of r. (i = l,2,3,...,/i) are close to one another. on the other hand, if the values of r; spread over a wide range, the estimation is not consistent. the closer to one another the values of rf are, the more consistent the estimation is. statistically, standard deviation is a measure of variation or spread of the ri 's. so it is proposed to use the standard deviation of r; (sdr) as a measure of estimation consistency, sdr = n-\ where r is the mean of r. 's (;=l,2,3,...,/i) the smaller the sdr, the more consistent the estimation. it can be shown that standard deviation of relative error is equal to sdr. because the assessment of estimation accuracy is based on relative error, sdr is related to estimation accuracy. therefore sdr can be used to calibrate a model in order to improve the estimation accuracy in different environments. regression criteria in a previous section, it is argued that ratio measures are more suitable for accuracy assessment. we now consider how to determine model parameters so as to satisfy the ratio measures of accuracy. in modeling costing formulas, regression is the basic technique to determine model parameters. the traditional method employed by most statistical software is least squares (ls) (khoshgoftaar et al, 1992), which can be expressed as min[ this is a criterion of difference measure. this criterion cannot lead to an optimised functional expression of the model as ratio measure should be used to assess the prediction accuracy. in modelling formulas for software cost estimation, the criterion of minimising the sum of squared relative errors has been used (conte et al, 1986). this criterion is not suitable either. in the expression of mre ajis vol 5 no 1 september 1997 37 mrei = act, est. act: 1est: act. actf is positive for all i, and est( should be positive for all i. in the case of overestimation, that is est > act, an mre can be greater than 1, but in the case of underestimation, that is estf < act( , an mre can never be greater than 1. this explains the problem discussed in a previous section that mre favours underestimation if it is used as a measure of accuracy. to achieve the least sum of squared relative errors, the regression technique makes as many negative relative errors as possible. therefore, if least sum of squared relative errors is used as the regression criterion, the obtained formula will be one that systematically underestimates the effort. it has been found in practice that in most cases of software cost estimation, the errors came from underestimation (lederer et al, 1993). using least sum of relative errors as the regression criterion would make the situation even more serious. a new regression criterion from the above discussion we can argue that there are two special requirements for regression criterion in formulating software costing models. • it should be in the form of the ratio of est{ to act.t . • it should be an unbiased one, which results in neither systematic underestimation nor systematic overestimation. to meet these two special demands, a new regression criterion in formulating software costing models is proposed: est it is obvious that this criterion takes the ratio form of estf to actf. the closer (in '-) is to 0, the closer act( estf is to 1, and the more accurate the estimation. as shown in the above expression, est; and act-t are act, symmetric with respect to their positions in the expression. therefore, this criterion does not produce an estimation formula of either systematic underestimation or systematic overestimation. computational procedure with the new regression criterion we can make use of existing statistics software in performing non-linear regression with the proposed criterion. in statistics software, the criterion for finding the optimal parameters is ls min[ before the non-linear regression is undertaken, the observed values and the functional form of the model can be transferred into their logarithmic counterparts, actf and est{ .the ls can be expressed as n n n art v-i • 1 7 v"1 n \ « \-i2 x^ 11 eal: 2 min [ > (actf estf) ] = min { > [m(act.) m(esti.)] } = min [ > (in -) ] for example, we have a set of n pairs of data (size t, effort.) (i = 1,2,3,...,n). if the functional form of a model is effort = a(size) + b ajis vol 5 no 1 september 1997 38 we can take the logarithmic form for the both sides of the equation to obtain in(effort) = \n[a(size) + b] . the regression is to be performed according to this equation. in this case, the /'* estimate is esti = a(sizej ) + b and the actual value is acti = effort , . before the regression, we first calculate ln(effort: ) (i= l,2,3,...,n), which will be used as the values for the dependent variable for the regression. in the regression, the functional form is ln[fl(s/ze) 4b] . therefore we have the criterion mm i2.] } effort, assessment of the proposed measures and criterion a set of data quoted by desharnais (1988) from 81 software projects was used to assess these measures of estimation accuracy and consistency, and regression criteria. the main features of the data set are summarised below. effort range: 546-19,894 person-hours function point range: 621 , 1 1 6 fp team average experience: 0-4 years manager experience: 0-7 years language levels: level 1: 46 projects were developed in 3gl (cobol with ims or idms type databases). level 2: 25 projects were developed using a combination of the traditional 3gl approach together with screen and report generators. level 3: 10 projects were implemented in 4gl. integrated software cost model an integrated software cost model was proposed by gao and lo (1995) on the basis of cocomo (boehm, 1981) and fpa. the integrated model attempts to combine the advantages of these two approaches into a single model. to address the problem of language dependency, language-weighted function point is introduced. in addition, continuous, rather than discrete, "cost drivers" are used. the general form of the model is effort = a(wf/))*fjc,u'"'/i) where a and b ant constants, xi is the magnitude of the i' cost driver, c, and di are constants corresponding to the i cost driver, n is the number of cost drivers and wfp is language-weighted fp. the equation appropriate to desharnairs' data set is effort = a(lkfp)b. where /, = 1.00, l^ = 0.95 and l± = 0.20 are the weights for language level 1, 2, and 3 respectively (gao et al, 1995). the exponent b is the responsiveness of effort to fp. it indicates that effort increases by b% when fp increases by 1%. ajis vol 5 no 1 september 1997 39 assessment on regression criteria three criteria for finding optimal parameters are used in regression: • criterion 1: min[^ (act( — esti ) 2 ] 1=1 ^ .act.-est,^ • cnterion 2: min[ > ( ) ] and ti acti v^,, est: . • criterion 3: min[ 2^ (in ) ] in figure 2 and figure 3, the relative errors re, = • est{ acti act. (i= 1.2,3 81) are shown in an ascending order. figure 2 shows the relative errors with respect to criterion 2. it can be seen that more than 70% of the relative errors are negative. this indicates that the formula obtained from criterion 2 tends towards underestimation. figure 3 shows the relative errors with respect to criterion 3. the number of negative relative errors is approximately the same as that of positive relative errors. the formula obtained from criterion 3 does not result in systematic bias. (criterion 1 is similar to criterion 3 with respect to estimation bias, because the actt and estt can interchange in the expression of criterion 1.) this shows that the regression with criterion 3 overcomes the shortcoming of the tendency to underestimate which occurred with criterion 2. figure 2 relative errors (criterion 2) figure 3 relative errors (criterion 3) assessment on the measures of accuracy and consistency for comparison, q3, pred(0.25), mre as well as wmq are computed as measures of prediction accuracy, while sdr and sdr are computed as measures of estimation consistency. we also use ajis vol 5 no 1 september 1997 40 effort = a(fp)b for regression in order to fully assess the measures of accuracy. accuracy assessment table 1 uses fp as size measure while table 2 uses wfp. table 2 also includes consistency results, which are to be compared with table 3 in the next section. table 1 accuracy of fp as size measure criterion 1 2 3 ft 0.66 0.70 0.56 pred(0.25) 0.42 0.11 0.37 wmq 0.45 0.62 0.42 mre 0.67 0.58 0.58 table 2 accuracy and consistency of wfp as size measure criterion 1 2 3 ft 0.43 0.45 0.44 pred(0.25) 0.49 0.48 0.56 wmq 0.32 0.34 0.31 mre 0.33 0.30 0.32 r 0.8428 0.8389 0 8406 sdr 0.439 0.344 0.423 the first observation is that the compared measures of accuracy, q3, pred(0.25), wmq and mre, are not consistent in both tables. this observation is not unexpected because the q^ and pred(0.25) are stochastic. the difference between mre and wmq will be discussed later. the second observation is that criterion 2 leads to the best mre and criterion 1 leads to the worst mre. as discussed in the previous section, criterion 1 is based on difference measure while mre is a ratio measure. therefore the mre of criterion 1 is larger than that of criterion 2 and criterion 3, which are based on relative errors. criterion 2 leads to underestimation tendency and mre favours underestimation. therefore criterion 2 has the best mre. criterion 3 does not have overor underestimation tendency. therefore the mre of criterion 3 is located between those of criterion 2 and criterion 1. the third observation is that if there are no outliers and no underestimation tendency (criterions 1 and 3 in table 2), wmq = mre. if there are outliers (criterion 1 and criterion 3 in table 1), wmq < mre, and if underestimation tendency exists (criterion 2 in table 1), wmq > mre. as mentioned before, mre as a measure of accuracy has two drawbacks. it is influenced by outliers and it favours underestimation. wmq as proposed targets the solution of these two difficulties, and this third observation indicates that wmq has succeeded in its objective. the fourth observation is that the criterion 3 leads to the best wmq for all data conditions. as assessed above, wmq is a better measure of accuracy, so criterion 3 is superior to the other two criteria. consistency assessment in this section, it will be first shown that sdr is better than sdr, because r is influenced by not only the estimation result but also the variance of the actual effort, whereas sdr is not influenced by the later. for this purpose, the data of one project, which involves the largest development effort, is deleted from the data set. with aj1s vol5 no 1 september 1997 41 this change the variance of the actual effort changes. table 3 shows the accuracy and consistency results from the reduced data set. table 3 accuracy and consistency results from reduced data set criterion q3 pred(0.25) wmq 1 2 3 0.44 0.45 0.44 0.49 0.49 0.56 0.33 0.33 0.31 mre 0.34 0.30 0.32 r 0.7900 0.7885 0.7891 sdr 0.441 0.345 0.425 comparing table 2 and table 3, we find that the q values, wmq and mre of each criterion are virtually the same in the two corresponding situations. the consistency should not be different because it measures the quality of a model in a specific environment and it should be independent of the variance of the actual effort. however, the change of variance makes the r values decrease by more than 0.0500. on the other hand, the sdr values only change by less than 0.002, which is comparatively small. therefore, the sdr is superior over r in the aspect that the former is not influenced by the variance in effort. next, it will be demonstrated that a model producing consistent estimation errors is more easily calibrated to improve accuracy than is a model producing less consistent errors, if the consistency is measured by sdr. the calibration method is described as follows. with desharnais' data set, three formulas are obtained by regression with the three criteria described earlier. these formulas are to be calibrated to the environment of albrecht's data set (1983). in albrecht's data set, 15 projects used level 1 language and their fps fell in the size range of desharnais' data set. for the other projects, either the languages cannot be categorised into the 3 levels or the fps are beyond the size range of desharnais' data set. therefore only these 15 projects were used for the calibration. it is assumed that the responsiveness of effort to size (exponents in the formulas, refer to section of "integrated software cost model") does not change, but the environments of the two sets of data are different. in desharnais' environment, the formulas obtained with the three regression criteria are effort = a j(wfp)bi a =1,2,3) the environment is the overall level of cost drivers (gao et al, 1995), which is expressed by the coefficient a} (j = 1,2,3). for albrecht's data set, an environment adjustment parameter, kj (j = 1,2,3), is used in the above formulas to account for the environment change effort = kjaj (j= 1,2,3) where the k (/ = 1 »2,3) are determined by the following procedure. firstly, the estimated efforts est(i = 1,2,3,...,15, j = 1,2,3) are obtained with the formulas developed in desharnais' environment. secondly, it is assumed that the k • (j = 1,2,3) satisfy the following equations est.. act: = k, est;s orl = k.1 act, =1,2,3,...,15,7= 1,2,3) ajis vol 5 no 1 september 1997 42 where act( (i = 1,2,3,...,15) are the actual values of effort in albrecht's data set. thirdly, these 15 equations are summed to give is act (7=1,2,3) and the above expression is rearranged so that the kj (j = 1,2,3) are obtained 15 est 0'= u2,3) table 4 and table 5 show the results of accuracy and consistency before and after the calibration. table 4 results before calibration criterion 1 2 3 g, 0.10 0.21 0.11 g2 0.18 0.35 0.26 a 0.49 0.57 0.47 pred(0.25) 0.60 0.27 0.47 wmq 0.32 0.44 0.34 mre 0.36 0.42 0.36 r 0.5328 0.5361 0.5352 sdr 0 0 0 57 42 53 table 5 results after calibration criterion q} q2 1 2 3 0.10 0.09 0.10 0.17 0.23 0.20 g, 0.50 0.46 0.47 pred(0.25) 0.60 0.60 0.60 0.32 0.32 0 32 wmq 0 0 36 36 0.36 mre 0.5328 0.5361 0.5352 r sdr 0.58 0.57 0.57 for criterion 1 in table 4, the sdr is 0.57 (the largest sdr in this table), so the estimation is the least consistent. criterion 1 in table 5 shows that the calibration does not improve the accuracy. for criterion 3 in table 4, the sdr is 0.53. this consistency is better than that of criterion 1. the calibration improves the wmq from 0.34 in table 4 to 0.32 in table 5, while mre remains unchanged. for criterion 2 in table 4, the sdr is 0.42. this consistency is much better than that of criterion 1 or criterion 3. the calibration improves the wmq from 0.44 in table 4 to 0.32 in table 5, and the mre from 0.42 to 0.36. the results indicate that the smaller the sdr is, the easier it is to improve accuracy by calibration. the sdr values remain the same before and after the calibration. the linear relationship between estimated effort and actual effort does not change after calibration, because the calibration multiplies the uncalibrated estimated effort by a constant,. on the other hand, all the sdr values after calibration differ from those before calibration. this indicates that sdr and sdr are not the same. the sdr measures the linear relationship between estimated value and actual value while sdr measures the variation of relative errors. the accuracy results after calibration are comparable with those obtained when the original formulas are used with desharnais' data set. this observation further emphasises that a model needs to be calibrated if it is to be used in a different environment. ajis vol 5 no 1 september 1997 43 from table 4 and table 5 it can also be observed that mres are greater than wmqs in all the cases except in criterion 2 of table 4. as expected, there are extreme values in cases where mre > wmq, and the formula tends to underestimate the effort in criterion 2 of table 4, where mre < wmq. summary in order to evaluate estimation models and improve the modelling process, we need appropriate methods to measure these models and determine model parameters. this paper reviewed existing methods and proposed new methods to measure estimation accuracy and consistency, and to determine model parameters. the difference measures of accuracy favour the estimation for small projects. therefore it is argued in this paper that measures of accuracy should be based on relative error of estimation. as the mre (the most widely used measure of accuracy) is influenced by outliers and favours underestimation, and single value measures are stochastic, the wmq is proposed for accuracy evaluation. the wmq includes more information on the estimation than single value measures, so it is less stochastic. it is also consistent with mre when there are no outliers and estimation is unbiased. consistency examines the model's degree of ease of calibration. a consistently overestimating or underestimating model is more easily calibrated than an inconsistent one. the correlation coefficient r between observed and actual values has been used to evaluate consistency. the r favours a data set with large variance. it is determined not only by estimation but also by the distribution of the actual values. in this paper, the standard deviation of the ratios of the estimate to actual effort (sdr) is proposed as a measure of consistency. the sdr is a measure of the variation or spread of the relative error. the method of least squares is the conventional regression criterion, but it is based on a difference measure, which is not suitable for software cost estimation. the criterion of minimising the mre will produce a formula which tends to underestimate the effort. to overcome the shortcomings of these criteria, this paper proposes to use the criterion of least squares of the logarithmic ratio of estimate to actual value for regression. the proposed measures and criterion share the characteristic that they are based on relative errors of the estimation. applied to real-world data, the proposed measures and criterion appear superior to the measures and criteria that are currently used in the field and reported in the literature. it is therefore recommended that the proposed measures and criterion be used for further assessment references abdel-hamid, t.k. (1990) "on the utility of historical project statistics for cost and schedule estimation: results from a simulation-based case study", journal of systems & software, sep., vol.13, no.l, pp.71-82. albrecht, aj. & gaffney, j. (1983) "software function, source lines of code, and development effort prediction: a software science validation", ieee transactions on software engineering, nov., vol.9, no.6, pp.639-648. boehm, b.w.(1981) software engineering economics, prentice-hall, inc., englewood cliffs, new jersey,. conte, s.d., dunsmore, h.e. & shen, v.y. (1986) software engineering metrics and models, benjamin/cummings publishing company, inc., menlo park, calif.. deeson, e. (1991) collins dictionary of information technology, harper collins publishers, glasgow. desharnais, j-m. (1988) "programme de maitrise en informatique de gestion", university of montreal, canada. fenton, n. (1994) "software measurement: a necessary scientific basis", ieee transactions on software engineering, mar., vol.20, no.3, pp. 199-206. gao, x. & lo, b. (1995) "an integrated software cost model based on cocomo and function point analysis", proceedings of the software education conference (srig-et94), pp.86-93, ieee computer society, los alamitos, ca. jenson, r.l. & bartley, j.w. (1991) "parametric estimation of programming effort: an object-oriented model", journal of systems & software, may, vol.15, no.2, pp.107-114. j0rgensen, m. (1994) "a comparative study of software maintenance effort prediction models", proceedings of 5th australian conference on information systems, 27-29 september 1994, melbourne, australia, pp.699-709. kan, s.h., basili, v.r. & shapiro, l.n. (1994) "software quality: an overview from the perspective of total quality management", ibm systems journal, mar, vol.33, no.l, pp.4-19. kenkel, j.l. (1989) introductory statistics for management and economics (third edition), duxbury press, usa. khoshgoftaar, t.m., j.c. munson, et al (1992) "predictive modeling techniques of software quality from software measures", ieee transactions on software engineering, nov., vol.18, no.l 1, pp.979-987. ajis vol 5 no 1 september 1997 44 kitchenham, b.a., kok, p.a.m. & kirakowski, j. (1990) "the mermaid approach to software cost estimation", proceedings of the annual esprit conference, brussels, november 12-15, 1990, pp.296-314. lederer, a.l. & prasad, j. (1993) "information systems software cost estimating: a current assessment", journal of information technology, no.8, pp.22-33. miyazaki, y. takanou, a. & nozaki, h. (1991) "method to estimate parameter values in software prediction models", information & software technology, apr., vol.33, no.3, pp.239-243. mukhopadhyay, t., vicinanza, s.s. & prietula, m.j. (1992) "examining the feasibility of a case-based reasoning model for software effort estimation", mis quarterly, jun., pp. 155-170. pfleeger, s.l. (1991) "model of software effort and productivity", information & software technology, apr., vol.33, no.3, pp.224-231. rowlands, b.h. (1989) "estimating instructional systems development effort and costs", proceedings of the 7th annual conf. of the australian society for computers in learning in tertiary education, gold coast, 1989, pp.340-348. wrigley, c.d. & dexter, a.s. (1991) "a model for measuring information system size", mis quarterly, jun., pp.245-257. verner, j. & tate, g. (1992) "a software size model", ieee transactions on software engineering, apr., vol.l8,no.4,pp.265-278. ajis vol 5 no 1 september 1997 » oi. •« i\o. l may 1997 editorial this is the second issue of volume 4 of the australian journal of information systems. it is a little larger than the normal issues as we have included a special issue section dealing with the question of ethics. as editor, i am pleased to report the on-going interest in the journal, both from authors and more recently from a number of international conferences who have kindly agreed to advertise the journal to their participants. in this issue we have 15 articles, 8 from australia, 1 from canada, 2 from the united kingdom, and one each from france, israel, finland and new zealand. our first paper the effects of normalisation on the satisfaction of novice end-users querying database' by conrad benedict. paul bowen, colin ferguson and fiona rhode, department of commerce, university of queensland reports on the results of an experiment which investigated the effects of different structural characteristics on satisfaction by end-users. the results showed that unnormalised tables adversely affect enduser satisfaction. the second paper 'standardising the information technology environment at telstra corporation: a case study by carey butler & peter weill, key centre for technology management, melbourne university traces the development of telstra's data network and desktop soes. the paper examines the strategic drivers for the soes, the business benefits, and issues related to telstra's computing environment prior to, and after the introduction of network and desktop soes. our third paper 'reassessing function points' by g.r. finnic, g.e. wittig, school of information technology, bond university & j.m. dcsharnais, software engineering laboratory in applied metrics, anjou, canada reviews the validity of the weighting scheme and the value of adjusting for system characteristics by studying their effect in a sample of 299 software developments. the paper suggests that weighting scheme used to adjust system components in terms of being simple, average or complex appear suspect and should be redesigned to provide a more realistic estimate of system functionality. dan hawthorn, department of information systems and computing, unitec institute of technology, new zealand & ananth srinivasan, department of management science and information systems, university of auckland, new zealand contribute a paper entitled 'revisiting the usability of graphical displays: an experimental approach the paper reports on the results of an experimental study of the usability of graphical displays for decision making. the authors found that assisting users to extract information from graphs with design features such as a mouse driven display has a significantly positive effect on performance. our next paper 'evolution of 00 methods: the unified case' matti rossi, department of computer science and information systems, university of jyvaskyla takes an evaluative look into 00 methods and especially the evolution of the new unified method from its ancestors, omt and ooda. the paper tries to classify the components of earlier methods and shows how the metrics can be used to analyse the changes in the techniques. the next paper 'conceptual data modelling: an empirical study of expert and novice data modellers' by graeme shanks, department of information systems, monash university explores the differences between conceptual data models designed by expert and novice data modelling practitioners. the data models are evaluated using a number of quality factors synthesised from previous empirical studies and frameworks for quality in conceptual modelling. the study finds that data models produced by expert data modellers are more correct, complete, innovative and flexible than those produced by novices. peretz shoval, information systems program, ben-gurion university of the negev, israel contributes a paper entitled 'experimental comparisons of entity-relationship and object-oriented data models'. this paper addresses the issue of which of the two data models is better for data modeling. the results reveal that, amongst other things: a) schema comprehension: ternary relationships are significantly easier to comprehend in the eer model than in the oo model & b) the eer model surpasses the oo model for designing unary and ternary relationships. the next paper 'an investigation of information technology-enabled remote management and remote work issues by d. sandy staples, department of information systems, university of melbourne investigates remote work and remote management issues. the findings supported that higher trust leads to higher job satisfaction and lower job stress, and that more communication between the manager and the remote employee develops higher levels of employee organizational commitment. mohan thite & pamela simmons, swinburne university of technology contribute a paper entitled 'an empirical examination of project leadership in an information systems environment'. this study examines the most appropriate leadership style at the project management level for the successful execution of is projects. the results support the proposition that there is a difference in the leadership styles of more and less successful project managers as perceived by their subordinates and that the successful managers exhibit significantly more of transformational leadership characteristics. the next paper the tasks of information systems professionals in philippines, thailand, indonesia and malaysia' by graham winley, ang ang & sim lau, department of business systems university of aj1s vol. 4 no. 2 may 1997 wollongong presents the results of an empirical study into the present and future tasks expected of information systems professionals in a range of organisations in south-east asian nations. the results are related to the present and expected future profiles of these organisations and are of relevance to those responsible for developing is curricula in academic institutions as well as commercial providers of education and training courses in these important south-east asian nations. the next paper 'structure and extensions of the user information satisfaction construct'by clive wrigley, faculty of business, simon eraser university, don drury, faculty of management, mcgill university & ali farhoomand, school of business, university of hong kong investigates the application of the uis instrument across a number of key variables including respondent characteristics. several findings emerge from a survey of 379 is and non-is managers: first, the uis factors are stable and generalizable. second, the level of satisfaction varies quite substantially when compared to prior uis reports, although the is product satisfaction dominates. third, is managers, not surprisingly, evaluate their systems significantly higher than non-is managers, indicating the importance of stakeholders in evaluating information systems success. simon rogerson, centre for computing and social responsibility, de montfort university, provides the introduction to the section involved with the question of ethics. his short introduction entitled 'ethics and information technology' provides an introduction to the topic. the first of the papers concerned with ethics comes from duncan langford, computing laboratory, the university, canterbury, uk. his paper is entitled 'ethical issues in network system design'. the paper discusses some ways in which the technical design of computer systems might appropriately be influenced by ethical issues, and examine pressures on computer scientists and others to technically control network-related actions perceived as 'unethical'. emma rooksby & diarmuid pigott, murdoch university provide the next paper entitled 'living with the internet: ethics in the nob'sphere' which gives a holistic alternative to the conventional ground of existent ethical discourse on the internet. the paper suggests that the model of the net is unsatisfactory and present a discourse of co-operative ethics, based on a model of the internet as a noetic hyper entity. daniel salber, university of grenoble presents the final paper entitled 'the need for an applied computer ethics handbook'. this paper contends that new advances in computer science disciplines such as communications networks and mobile devices are suggesting radically new uses of information technology. through the use of existing ethical frameworks such as mason's papa or huffs impactcs he suggests that information technology practitioners need more explicit tools such as handbooks to help them understand and deal with ethical issues. once again i would like to pass on my thanks to the authors and consulting editors for the high quality of material in this issue. i would also like to thank david dodds for his assistance in producing final copy of the journal and the maintenance of the home page. r. macgregor editor home page: http://www.uow.edu.au/ajis/ajis.html ajis vol. 6 no. 2 may 1999 preliminary insights into the implementation of standardised it in franchise systems angele l m cavaye department of information systems colin mccosker department of marketing chad perry department of marketing faculty of business university of southern queensland toowoomba, qld 4350, australia abstract the franchise context provides a rich background for studying the implementation of standardised, interlinked it systems. benefits of it implementation in franchises appear obvious, but the small business context and potential tension between the two main stakeholders (franchisor and franchisees) tend to create difficulties in the implementation process. case study methods were used to investigate it implementation in franchise systems. franchisee-related factors that clearly impact on implementation include perceived need for the it system, cost of the system and the level of it literacy among franchisees. franchisor-related factors like coercion of franchisees and education of franchisees strongly facilitate implementation. introduction this paper reports on a study investigating the implementation of standardised it systems across franchise systems. in particular, the paper focuses on the implementation phase of the development process: on the phase when standardised it systems are introduced to franchisees and actually installed in franchisee outlets. the franchise context provides a rich background for studying the implementation of standardised, interlinked it systems. although the benefits of standardised it for franchise systems might seem obvious, actual implementation of such systems across a franchise is not a simple matter. difficulties are created through the potential tension between franchisors and franchisees, the two stakeholders, and because of the problem of balancing and sharing the costs and benefits. firstly, it implementation can involve friction between franchisor and franchisee. the franchisor may want to implement standardised it systems with links between franchisor and franchisees in order to reap franchise-wide benefits. however, the franchisee may not recognise such benefits and, in any case, may resent the franchisor gaining easy access to data about operations at the franchisee outlet. the second difficulty concerns the trade-off between costs and benefits of it implementation for the franchisee. all changes to business processes and all plans for it implementation carry both benefits and costs. the need for and potential benefits of franchise-wide ft may be recognised by franchisees. however, costs of implementation may be perceived to be too high; alternatively, the introduction of new processes and new ways of working as a result of it implementation may be perceived to pose too much difficulty. the paper proceeds as follows. first, the research issue is explored in more detail by reporting on previous research and on preliminary empirical work for this study. next, the research questions and case study method are described. then, findings of the multiple case comparison are presented and discussed. implementation of standardised, interlinked it systems in franchises this section briefly explains the franchise context, outlines the major literature relevant to the study of it systems in franchises, and describes the findings of exploratory case studies. this provides the background for the study and leads to the formulation of the research questions for this research. franchise systems the main parties in a franchise system are the 'franchisor' and the 'franchisees'. the 'franchisor' is the company, person or firm granting a contractual license to another party; the 'franchisee' is the company, person or firm obtaining a license from a franchisor (mendelsohn 1992). in this paper, the franchisor, company-owned outlets, and franchisees together comprise the 'franchise system'. ajis vol. 6 no. 2 may 1999 there is both a need and a requirement for regular and continual information exchange within the franchise system. the franchisor provides initial and on-going training, informs franchisees about new and updated procedures, provides up-to-date pricing and product information, and supplies marketing information to the franchisees (mendelsohn 1992). the franchisees are required to provide performance data to the franchisor on a regular basis. these information flows are depicted in figure 1 below. figure 1 communication within a franchise system updated and new procedures price and product information franchisor w * performance data (sales, profit/loss) franchisees regular information exchange among business partners implies an opportunity to use it. effective use of it facilitates and speeds up information exchange; it can also lead to improved business processes on the various premises. the franchise context provides a textbook setting for use of it between organisations. previous research two areas of is literature are relevant to the study of it implementation in franchise systems: the literature about inter-organisational systems and the use of it between and among business partners and, secondly, the literature about the use of it in small firms. the use of it to facilitate communication and to automate transactions between business partners has been studied for nearly 20 years. initial papers focused on opportunities and benefits of implementing it which links business partners (e.g. holland and lockett, 1994; venkatraman, 1991). other work highlighted factors facilitating and inhibiting implementation of electronic links between business partners (e.g. cavaye and cragg, 1995). recently, interest has focused on the business opportunities provided by the internet (mckeown and watson, 1996) and on the adoption of electronic commerce (kalakota and whinston, 1996; lawrence et al, 1998). another strand of the literature in this area has drawn attention to the effects of power differences among business partners and to the enforced use of it with partners who dominate the business relationship (e.g. webster, 1995). since franchise systems are collections of small firms, the literature about it adoption in small firms is relevant to this study. cragg and king (1993), lavocau et al (1995) and thong and yap (1995) identify many different factors which potentially influence adoption of it by small firms. the factors include perceived benefit and impact of technology, pressure to adopt (from outsiders), financial and technological resources of the firm, computer literacy of the owner/manager, size and industry sector of the firm. harrison et al (1997) focus on the executive or owner of the small firm as decisions made in small firms tend to be made unilaterally by the owner. all these issues are also likely to affect it adoption by franchisees. although franchise systems provide obvious opportunities for the use of it to link franchisor and franchisees, little has been published about such use of it by franchise systems. only a few papers have been identified which mention it in franchising. sen and lee (1994), simeonson and anstadt (1993) and nault (1997) contribute theoretical papers on the potential impact of it on the development and efficient operation of franchise systems. one paper addresses use of it to link franchise systems with suppliers (poorani 1996). it highlights the theoretical benefits of inter-organisational it for franchise systems and reports on a small scale, descriptive survey carried out in the united states. there has been no previous, structured investigation into the use of it between franchisors and franchisees; there is no previous research about such use of it in the australian context. preliminary case evidence because of the lack of published research about it in franchises, several exploratory case studies were conducted. discussions with management of three franchises provided preliminary descriptive information about it development and implementation processes in franchise systems. findings from the three cases strengthened the researchers' view that the implementation of franchise-wide it was worthy of further ajis vol. 6 no. 2 may 1999 investigation. full findings derived from the exploratory cases have been reported elsewhere (cavaye, 1998). the main findings relating to the subject of this paper are briefly outlined below. implementation an important phase of development. the focus of this paper is the implementation phase of the development process for standardised, franchise-wide it. case interviews showed that the it development process in the context of franchise systems is very similar to the process relating to any information system that is meant to link organisations. the process consists of three clearly identifiable phases: first, a system must be acquired or developed; then, the system is implemented in individual sites; next, the systems in individual outlets are linked to the franchisor, thus forming a franchise-wide network. once an appropriate system has been developed, the system is implemented in franchise outlets. this implementation takes place in two stages: first, implementation of standardised systems in individual outlets; then the linking of outlets to the franchisor. the usual course of action is to implement in franchisor-owned outlets first. when the system has been shown to work well in company-owned stores, the system is implemented in franchisee outlets. once the systems are accepted and used effectively by the franchisees, it is possible to commence linking of systems in outlets with the computer system of the franchisor at the franchise headquarters. the implementation phase is crucial to the success of the franchise-wide it system. preliminary case evidence showed that benefits for the franchise system increase directly with the number of outlets using the system and linked to the franchisor. hence, it is of utmost importance for the franchisor to ensure fast and full implementation of standardised it systems in individual outlets. factors affecting the implementation phase. the franchisor has to facilitate and enable implementation of standardised, interlinked it systems in the franchise. franchisors need to explain the benefits of franchise-wide it, to overcome franchisee resistance, to teach computing skills, and to train users on the specific systems to be introduced. franchisors try to convince and educate franchisees about the value of embracing interlinked it systems using a mix of seminars, training sessions, and face-to-face discussions with individual franchisees. there are factors that may withhold franchisees from accepting implementation of standardised, interlinked it. such factors include the perception of benefits, cost of the system, level of computer literacy among franchisees, and mistrust of the franchisor. it is vital for franchisees to understand the benefits that it can bring to their outlet and to the franchise as a whole, but not all franchisees understand these benefits. many franchisees are interested in ways to improve the efficiency of their operations and are willing to incur some costs if it means they receive obvious, subsequent benefits. however, financial costs for full it implementation may be significant; the cost of a sophisticated point-of-sale system can amount to a$40,000 per franchisee outlet. not all franchisees are easily persuaded to invest that much money in it systems. implementation of franchise-wide it may also be inhibited by low levels of computer literacy among franchisees. those who are not familiar with computers fear the introduction of franchise-wide it. in addition, there usually is a small proportion of franchisees that opposed the idea of the franchisor having full access to their operational data. often the distrusting franchisees are those who are not operating very well. research questions early case work confirmed the importance of the implementation phase in the complete development process for franchise-wide it and identified several factors affecting the implementation of it in franchisee outlets. this then formed the basis for the following research questions: 'what factors have most impact on implementation?', and 'how does the franchisor overcome franchisee resistance to implementation?". gathering data from multiple case studies the aim of this study was to investigate use and implementation of it as it occurs in franchise systems. there is no strong theoretical base explaining the use of it in franchise systems. inductive methods were needed in order to obtain information from practical settings to address the research questions. there was no intention of controlling or influencing events in individual organisations. because there is no theoretical base and because of the wish to study contemporary events in their natural setting, the case study approach was appropriate for this study (eisenhardt, 1989; perry, 1998; yin, 1994). ajis vol. 6 no. 2 may 1999 criteria for case selection individual cases had to be franchise systems listed in the australian franchise directory. it was important to select cases where we were likely to be able to gain access. for this reason case organisations were chosen which were known to at least one of the researchers. apart from these general criteria, individual sites were selected on the basis of dissimilarity. in this study an attempt was made to investigate a variety of firms by selecting cases which varied in terms of age, size, and industry. cases were selected from three different states in australia in order to avoid a possible regional bias to the findings. table 1 summarises the case selection criteria. table 1 criteria for selecting cases in this study general criteria criteria concerning franchise characteristics listed in australian franchise directory known to at least one of the researchers age: mix of recent and more established franchises; young (0-5 years old), medium (5-10 years old), established (over 10 years old) size: small (less than 20 outlets), medium (20-100 outlets), large (over 100 outlets) industry types: retail and service sectors geographical location: three different states in australia variety in age, size, and industry type of the cases was sought because this might provide information about relevant contextual factors. it is conceivable that older franchise systems may have more difficulty implementing it than more recent franchises, because older franchises may have more established practices and ways of working and may have franchisees who are older and, perhaps, less computer literate. with regard to size of the franchise, large franchises with a wide variety of individual franchisees may have more difficulty implementing it in outlets than small franchises with a more homogeneous group of outlets. relating to industry type, service franchises with few products and small outlets are likely to have more difficulty convincing franchisees to implement it than retail franchises with a large product line since potential benefits may be more clearly recognised by retail franchisees. data collection and analysis procedures an interview guide was constructed to investigate the research questions. individual questions in the interview guide related to the benefits and costs of implementation franchise-wide it and to the factors potentially influencing it implementation in franchise systems. questions were open-ended and were deliberately designed to provoke rich and lengthy answers. all data was collected within a 2 month period by visiting the headquarters of each franchise system in the case sample. all data was collected from face-to-face interviews with individuals at each case site. individual interviews lasted between 1 and 2 hours. informants were members of the senior management team in each franchise system. wherever possible relevant documentation (such as company information, it reports, system training schedules) was collected. in several of the cases the informant provided the opportunity to view screens and links of the it system that was the subject of the interview. senior management of a franchise system refers by definition to the franchisor. the franchisor informant was asked by franchisor strategy and policy; the informant was also asked about views and attitudes of franchisees. data about franchisees was collected from the franchisors rather than from franchisees themselves. it would have been very difficult to collect information from a range of franchisees for every franchise case that we studied. for that reason we used the second-best source of information: perceptions of franchisors about their franchisees. since this was a first investigation into the use of it by franchise systems, the use of franchisors as the only informants was considered acceptable. all interviews were tape-recorded and transcribed. various authors have outlined techniques for analysing qualitative data (miles and huberman, 1984; yin, 1994). these techniques include coding, tabulation, ranking and all of these have been used to analyse and present the qualitative data in this study. analysis was concerned with highlighting findings about each of the research questions. firstly, data from each individual case was described and categorised; next, the data of the individual cases was combined and compared in the multiple case analysis phase. ajis vol. 6 no. 2 may 1999 characteristics of the cases nine franchise systems provided the case data for this study. the franchise systems were promised confidentiality and anonymity in any publications; hence they are referred to as case a, b, c, etc. descriptive characteristics of the cases in the study are presented in table 2. table 2 identifies 'age of the franchise' by noting the year in which each franchise was started. rating of 'age' was achieved by classifying all franchises less than 5 years old as 'young', those between 5 and 10 years old as 'medium', and those older than 10 years as 'old'. the table identifies the size of each franchise case by noting the number of outlets. rating of 'size' was obtained by categorising a franchise with less than 20 outlets as 'small', a franchise with 20 to 100 outlets as 'medium-sized', and a franchise with more than 100 outlets as 'large'. the final columns in table 2 identify the franchises by industry and specific product supplied. the franchise sector itself identifies individual franchises as primarily retail or service based; their classification of 'retail' or 'service1 franchise was used to categorise the franchises participating in this research. table 2 characteristics of franchise systems in the case sample case a caseb casec cased casee casef caseg caseh case i age year started 1985 1997 1992 1995 1996 1986 1982 1994 1986 rating old young medium young young old old young old size number of outlets 41 16 180 23 10 95 230 14 102 rating medium small large medium small medium large small large industry type retail retail retail retail service retail service service retail specific product floorcovering car products cakes household products home delivered prod. books real estate transport sports clothing •fequipment during analysis of the case data, the three contextual factors (age, size and industry) were briefly looked at. the data did not provide sufficient information to identify clear associations between contextual factors and it implementation. it implementation affected by franchisee and franchisor related factors the first research question addresses factors which may make a franchisee accept or resist standardised it implementation in the outlet. the second research question relates to franchisor action to encourage it acceptance by franchisees. case findings for each of these questions are presented and discussed in this section. franchisee-related factors influencing franchise-wide it implementation discussion with informants identified six main factors affecting franchisee acceptance of it implementation in the outlets. ajis vol. 6 no. 2 may 1999 table 3 summary of factors affecting franchise-wide it implementation category business-related factors it-related factors individual factor perceived need for and business benefit of franchise wide it cost of it system to be implemented time available for management/it work level of it literacy of franchisee existence of current it systems in outlet attitude towards earlier franchise-wide it number of cases (n=9) 9 8 3 8 5 2 prevalence of factor v. high high low high medium low three of the factors identified from the case data are business-related: business benefits of it, financial costs of it, time available for it work; the three other factors concern the technology itself: lack of computer knowledge, conversion difficulties when there are existing it systems in the outlet, franchisee attitudes towards it due to earlier experiences with it. the business-related factors played a role in all cases; the it-related factors were mentioned less often. table 3 summarises the factors affecting franchise-wide it implementation; the factors are discussed below. perceived need for franchise wide it. all nine informants mentioned the importance of franchisees understanding the benefits that it can bring to their outlet and to the franchise as a whole. in all of the 9 franchise systems the need for and understanding of business benefits played a central role in gaining franchisee acceptance of implementation of standardised it in outlets. franchisees may need to be convinced that the operation of the franchise as a whole is improved by use of it in individual outlets and by linking of systems in outlets to the franchisor. there are two stages of implementation: first, standardised it is implemented in the outlets; then, outlets are linked to the franchisor and, sometimes, to each other. franchisees can usually perceive the benefits of introduction of appropriate it for the outlet; such it implementation has a direct impact on the efficiency and effectiveness of business operations. franchisees are less likely to acknowledge the benefits of linking with the franchisor; there may even be mistrust as to the 'secret purpose' of the franchisor in insisting on electronic linking between franchisor and franchisees. cost of it system. the cost of the system varies according to the type of it system that is proposed. one of the franchise cases (case h) had implemented an intranet which requires the franchisee to have a pc, modem and intranet software; this type of system is not very expensive. on the other hand, retail franchises with many products and high turn-overs (like case d, household products, and case i, sports products) require that their outlets obtain one or more terminals linked to a point-of-sale (pos) system with multiple functionality; this type of system can cost up to $40,000. apart from actual cost of the system, franchisees are particularly interested in the cost in relation to perceived benefits of using the system. one franchisor was adamant that there is no point in making franchisees pay for a system that is of no apparent benefit: "you know, it has to be a real benefit to them for them to spend the money. we are doing the wrong thing if we talk them into spending money unnecessarily. that is going to damage the relationship down the track." (case a) large outlets with high turn-overs or outlets in franchises with large product-lines readily understand the need for it to help manage the inventory and sales functions. the immediate benefits may be less obvious to smaller outlets and to outlets with few products. time available for it or other management tasks. several of the informants mentioned that franchisees feel they do not have the time to learn a new system of working or to operate and maintain an it system. many franchisees are interested in being involved in the day-to-day operation of the business. in fact, a recent survey shows that 95.7% of franchisors require the franchisee to be substantially involved in the hands-on operation of the outlet; very few franchise systems allow the franchisee to be a silent partner (franchising australia, 1998). in some franchises time spent in the office means time lost in making money. the informant from the real estate franchise (case g) summed it up this way: "they are so caught up in selling, that they don't want to take time to sit down and learn to understand the it system... many real estate offices are not sufficiently large enough for them to be administered correctly. real estate managers feel they don't have time to do that; to sit down and study and look at new ideas to support their work." (case g) ajis vol. 6 no. 2 may 1999 level of computer literacy. nearly all cases (8 out of the 9) made mention of a lack of computer literacy affecting the ability to implement it in franchisee outlets. informants discussed the fact that older franchisees in particular resisted it implementation because they were not familiar with day-to-day computer use. franchising australia (1998) reports that 54.4% of franchisees are over 40 years of age with 13.4% over the age of 50. the people in this age group have not grown up with computers and, although a proportion of these franchisees may have learnt to use computers in their own time, this group does provide the greatest difficulty for franchisors when trying to implement it in outlets. existence of current it systems in the outlets. a proportion of franchisees usually has already invested in it systems for their outlet even if the franchisor has not in the past suggested the use of computers to enhance and facilitate operations. in retail outlets use of it is particularly widespread. when franchisees have a current it system in place, they are not eager to change over to a new system. this resistance is more pronounced when the cost of the system is high. five of the cases mentioned difficulties with franchisees attributable to the existence of current it systems in the outlets. attitude towards former franchise-wide it implementation. two of the 9 informants mentioned the effect of previous attempts to introduce standardised it. franchisees that have been exposed to previous, unsuccessful attempts at it implementation have an on-going negative feeling towards franchise-wide it. it is not easy to convince these franchisees that 'this time it is going to be different'. apart from identifying factors that can inhibit or facilitate it implementation in franchises, it is important to understand their relative importance. some factors may be more important than others; some factors may have a stronger link with implementation than other factors. table 4 summarises the factors and the apparent strength of their association with ease of implementation. the factors are presented in descending order of importance (as identified in the final column). table 4 franchisee-related implementation factors and link with ease of implementation individual factors perceived need for and business benefit of franchise wide it high cost of it system it illiteracy among franchisee time available for management/it work positive attitude towards earlier franchisewide it existence of current it systems in outlet effect on ease of implementation positive negative negative positive positive positive prevalence v. high high high medium low low link with ease of implementation v. strong strong strong medium medium low analysis of case findings highlights the importance of franchisees understanding the benefits of implementing standardised it systems in their outlets and the value of linking their it system with the franchisor. the importance of perceived need for it and understanding of benefits is paramount: it is a factor mentioned by all case informants and it was also identified by case informants as the factor with the most impact on implementation. if franchisees really understand the benefits and perceive a need for it, then the franchisees are more likely to be willing to accept the financial cost of the system and are willing to make time available for learning and using the system. franchisor-related factors enhancing it implementation case findings show that franchisors use two main approaches to encourage implementation: education and coercion. all of the cases used a variety of ways to inform and educate franchisees; some of the cases also used coercion to ensure that franchisees implemented the appropriate it systems. coercion. franchisors can force franchisees to accept new procedures and new ways of working by writing such procedures into the franchise agreement. in this way the franchise agreement is a powerful tool to ensure that it implementation is incorporated in outlets. when written into the franchise agreement, there is no difficulty with implementation of it in the outlets of new franchisees. seven of the 9 case organisations had already implemented it in outlets. all 7 had included the purchase and use of the it system as one of the ajis vol. 6 no. 2 may 1999 conditions in the franchise agreement. hence, new franchisees joining the franchise systems comply with the it standards and links of the franchise. it is more tricky to coerce existing franchisees into accepting it. again, franchise agreements can be used: 92 percent of franchise systems have renewable contracts for their franchisees (franchising australia, 1998). at the time of renewal existing franchisees would be forced into accepting implementation of standardised, interlinked it. however, not all franchisors agree that this is an acceptable approach: "we have found from experience that if you offer a system and they accept it, they will use it. on the other hand, if you push the system, they will not use it. or, if they do use it, they will complain about it more than anything else." (case c) it is important to the franchisor to maintain good relationships with franchisees. coercing reluctant franchisees into implementation of the standardised it system can potentially harm the relationship. franchisors are very aware of this and may choose to avoid confrontation with existing franchisees. education. all informants mentioned the importance of education and training of the franchisees. support and general training is provided on a regular basis in order to maintain franchisee interest in and knowledge about the franchise business. franchise australia (1998) shows that australian franchisors provide an average of 4 days of official franchise training to franchisees every year. as part of that general training the use and support of it systems in discussed, thus placing standardised it within the context of improving the franchise and enhancing business operations. once a specific system has been developed and is ready for implementation, franchisors provide specific training relating to the hardware and software to be introduced to franchise outlets. the training itself may be carried out by the franchisor, but it is more likely that this is undertaken by the consultant or software provider that will support the system. framework of factors affecting ease of franchise-wide it implementation this research identified several factors affecting standardised it implementation in franchise systems. these factors may be categorised as factors primarily relating to the franchisee and the franchise outlet and, secondly, as factors relating to franchisor actions. both types of factors have been discussed above. the franchiseerelated factors included perceived need for franchise-wide it, cost of it implementation, it illiteracy among franchisees, time available for it work, positive attitude towards earlier franchise-wide it, and existence of current it systems in franchisee outlets. the factors relating to franchisor actions included coercing franchisees into it use and educating franchisees. based on the findings of this case research figure 2 presents a research framework showing the major factors influencing ease of standardised it implementation in franchises. generalisability of this framework can be tested by a survey to gather data from a large number of franchise systems in australia. the factors should also be validated in countries other than australia. figure 2 research framework based on case findings coercion by franchisor franchisee-related factors perceived need cost of it system level of it literacy time available for it work attitude towards earlier it existence of it systems in outlets ease of standardised it implementation education of franchisee concluding comments the objective of this study was to identify the main factors associated with ease of implementation of standardised, interlinked it in franchise systems. lack of existing research about it in franchise systems implied that an in-depth study was warranted. a multiple case approach was used to collect data. 10 ajis vol. 6 no. 2 may 1999 franchisee and franchisor related factors affecting standardised it implementation were derived from the interview data. a major factor influencing the attitude of franchisees is the perceived need for and business benefit of franchise wide it. this factor played a role in the implementation process of all cases and its link with ease of implementation was very strong. two other important franchisee-related factors are cost of the it system (a high cost is associated with difficult implementation processes) and the level of it literacy among franchisees (lack of it knowledge prohibits easy implementation). coercing franchisees into it use and education franchisees about it were the two actions used by franchisors to enhance it use and ease implementation of standardised it systems. this study is the first in-depth investigation into it implementation to enhance communication within franchise systems. the study has several weaknesses which may be addressed in further research. data about franchiseerelated factors was collected from franchisors; we did not discuss the findings with franchisees. a further indepth study could seek information from both franchisors and franchisees. current findings and the suggested research framework are based on case study data. further research could use a quantitative survey method to test the current findings in a wider setting. in addition, data could be gathered in other countries and compared with the findings in the australian context. references cavaye a l m, (1998), "ios in the franchising context: computerised communication between franchisor and franchisees", proceedings of australasian conference of information systems, september, sydney, pp 88-97 cavaye, a.l.m. & cragg, p.b. (1995). factors contributing to success of customer oriented inter organisational systems. journal of strategic information systems, 4(1), pp. 13-30. cragg, p.b. and king, m.k. (1993). small firm computing: motivators and inhibitors. mis quarterly, 17(1 ),pp. 47-59 eisenhardt k. (1989). building theories from case research. the academy of management review, 14, pp.532-550 franchising australia (1998). a survey of franchising practices and performance, prep, by c. mccosker and l. frazer, franchise council of australia and university of southern queensland, australia harrison d.a., mykytyn p.p. and rienenschneider c.k. (1997). executive decisions about it adoption in small business. information systems research, june, 8(2), pp!71-195 holland, c.p. and lockett, g. (1994). strategic choice and interorganisational information systems. proceedings of the 27th hawaii international conference of systems sciences, vol. iv, pp. 405-413 lavocau c.l., benbasat l. and dexter a.a. (1995). electronic data interchange and small organisations: adoption and impact of technology. mis quarterly, dec, 19(4), pp. 465-486 kalakota r. and whinston, a. (1996). frontiers of electronic commerce. addison-wesley, united states lawrence e., corbitt b., tidwell a., fisher j. and lawrence j.r. (1998). internet commerce. john wiley and sons, brisbane, australia mckeown p. and watson r. (1996). metamorphosis a guide to the world wide web and electronic commerce. john wiley and sons, new york mendelsohn, m. (1992). the guide to franchising. casell, london. miles, m. b. and huberman, a.m. (1984). qualitative data analysis: sourcebook of new methods. sage publications, beverly hills nault, b.r. (1997). mitigating underinvestment through an it-enabled organisation form. organisation science, 8(3), may-june, pp. 223-234 perry, c. (1998). processes of a case study methodology for postgraduate research in marketing. special joint issue of european journal of marketing and new zealand journal of business, (forthcoming) poorani, a.a. (1996). electronic data interchange: a strategic tool to increase franchise productivity and efficiency. proceedings of the 10th annual conference of the society of franchising, pp. 2-16 sen, k.c. and lee h.l. (1994). the impact of information technology on the franchise decision, proceedings of the 8th annual conference of the society of franchising, las vegas, nevada. simeonson, r.b. and anstadt, n. (1993). the role of computers and automated systems. in the franchising handbook, ed. a.j. sherman, international franchise association, new york. thong j. and yap c.s. (1995). ceo characteristics, organisational characteristics, and information technology adoption in small business. omega, aug, 23(4), pp. 429-442 venkatraman (1991). it induced business configuration. in the corporation of the 1990s, ed. m. scott morton , chapter 5, oxford university press, new york. 11 ajis vol. 6 no. 2 may 1999 webster, j. (1995). networks of collaboration or conflict? electronic data interchange and power in the supply chain. journal of strategic information systems, 4(1), pp. 31-43 yin r.k. (1994). case study research: design and method, 2nd edition. sage publications, london 12 aj1s vol 4 no. 1 a pattern-oriented approach to a methodical evaluation of modeling methods michael amberg university of bamberg, business information systems d-96045 bamberg, tel. (+49 951) 863-2578, fax (449 951) 9370412, e-mail: michael.amberg@sowi.uni-bamberg.de abstract the paper describes a pattern-oriented approach to evaluate modeling methods and to compare various methods with each other from a methodical viewpoint a specific set of principles (the patterns) is defined by investigating the notations and the documentation of comparable modeling methods. each principle helps to examine some parts of the methods from a specific point of view. all principles together lead to an overall picture of the method under examination. first the core ("method neutral") meaning of each principle is described. then the methods are examined regarding the principle. afterwards the method specific interpretations are compared with each other and with the core meaning of the principle. by this procedure, the strengths and weaknesses of modeling methods regarding methodical aspects are identified. the principles are described uniformly using a principle description template according to descriptions of object-oriented design patterns. the approach is demonstrated by evaluating a business process modeling method. introduction it is commonly accepted that modeling methods assist in system analysis and design. these methods help to develop well-understood high-quality models in an efficient and effective way. the method based system analysis and design is embedded in the following overall context (figure 1). for complex system engineering tasks one uses a formalized interim representation of a real system that is called model. the similarities between the model and the real system help to develop a well-engineered solution for the task (e.g. an application system or a reengineering recommendation). the model serves as a communication medium within the development process. the system engineering task is typically described by one or more objectives (such as to develop an enterprisewide application system) and an object of investigation (the real system such as the respective enterprise). then you select an appropriate modeling method. the modeling method is used to construct a model as an abstraction of the real world (object of investigation) considering all relevant views in order to fulfill the prescribed objectives. to meet these requirements a method consists of a set of (semi-formal) notations and guidelines about how and when to use the notations. the example in figure 1 illustrates the overall context. for the development of an application system (objective) you might select the object modeling technique omtby rumbaugh et al. (method). afterwards you specify an object model (the application model) of the enterprise (the object of investigation) and use the model to implement the application system. looking at the available system engineering methods the situation can be described as follows: numerous methods and various improvements of methods (variants) exist. additional methods are under development. the variety and the growing number of methods can be explained historically through the expansion and modification • of the object of investigations (e.g. software, application systems, information systems or business systems), • of the underlying paradigm (e.g. data-oriented, functional-oriented, object-oriented or business process oriented), • of the objectives that are pursued (e.g. to develop an application system or to reengineer an enterprise) and • of the methodical point of views as well as the modeling procedures that are regarded to be suitable. a.iis vol 4 no. 1 research layer modeling method research provides methods for reuse analysis and evaluation specification layer modeling method interpretation guidelines object of investigation (universe of discourse) model applicationapply e.g. mentation e.g. application system figure 1: the overall context of method based system analysis and design examples of functional-oriented methods are structured analysis (sa/de marco (1979)) and structured analysis and design technique (sadt/softech ross and schoman (1977)); of data-oriented methods are entity relationship model (erm/chen (1976)) and structured entity relationship model (serm/sinz (1988)); of object-oriented methods are object oriented design (ood/booch (1994)) and object modeling technique (omt/rumbaugh et al. (1991)); of business process modeling methods are line of visibility engineering methodology enhanced (lovem-e/ibm (1995)) and semantic object model (som/ferstl and sinz(1995)). a pattern-oriented approach to a methodical evaluation there are two main approaches apparent for an evaluation of methods: a methodical evaluation and an empirical evaluation. a methodical evaluation analyzes which aspects of real systems are considered by a method and what axioms the method is based on. whereas an empirical evaluation is inductively based on the experiences made when applying methods (e.g. best practice). this paper focuses on a methodical evaluation approach (see for instance stein (1994)). looking at modeling methods the degree of methodical support varies distinctly. for the use of existing methods and for the development of new methods it is important to understand: • which methodical viewpoints does a method provide? which characteristics of the object of investigation are regarded to be relevant? • which objectives does a method support? which guidelines does it recommend? • which methodical viewpoints are appropriate for which context? a methodical evaluation of a method is a difficult task: how to approach the evaluation? evaluate to which criteria? is a comparison with other methods appropriate? how should differences be assessed? what aspects are considered in too much detail or not detailed enough? are relevant aspects missing at all? to ease the evaluation the paper proposes a pattern-oriented approach. a specific set of principles (the patterns) is defined by investigating the notations and the documentation of the modeling methods taken into ajis vol 4 no. 1 consideration. each principle helps to examine some parts of the methods by focusing onto a specific aspect. thus, the principles guide the evaluation. all principles together lead to an overall picture of the method. evaluation procedures this approach can be used for two main, partly overlapping intentions: a. to become acquainted with a new method and to compare it with other methods: this is relevant for a method engineer (i.e. a method developer or a user who applies methods) who wants to learn more about an unfamiliar method. comparing the method with known methods can simplify the understanding. by this, the similarities and the differences are identified regarding methodical aspects. the comparison can be used to rate methods concerning given goals and measurements. it can also be used to rank methods concerning their methodical support. b. to select a method for a given task: this is relevant for a user who wants to select a method out of a set of methods and wants to apply the method for a given task. to do this, the user has to determine the principles that are relevant for the task. afterwards the user selects the method that fits the required principles best. thus the pattern-oriented approach helps to detect the methodical aspects that are ad dressed or are not addressed by a method. when combining several methods the approach helps to dis cover the overlapping methodical aspects. these intentions affect the evaluation procedure. for the first intention an evaluation procedure that produces satisfying results comprises the following steps: al. determine the method to evaluate and the set of methods to compare it with. a2. detect the specific set of principles by investigating the notations and the documentation of the meth ods. each principle will be used to examine some part of the methods from a specific point of view. a3. describe the core meaning of each principle by filling out the appropriate slots in the principle descrip lion template (see 2.2). a4. fill out the slots in the principle description template that are related to the method under evaluation. a5. fill out the slots in the principle description template that are related to the comparable methods in the sum. taking all comparable methods into consideration may unveil additional aspects. a6. all principles together lead to an overall picture of the method under evaluation. the similarities and differences to the core meaning and to the other methods help to find their strength and weaknesses (problem areas as well as opportunity areas). all principle description templates together are regarded to be an evaluation schema. the evaluation schema can directly be visualized by a table (such as in figure 2). the slots of the description template are represented by columns. the context of the principle templates are depicted as rows. slots of the ^ principle description template descriptions r—>. of the principles name principle 1 principle 2 principle 3 ,» description (core meaning) correlation to method correlation to other methods ... figure 2: an evaluation schema to become acquainted with a new method and to compare it with other methods (case a) the filled out principle description templates can be reused for the repeated evaluation of modeling methods. for instance if you want to become acquainted with several methods you have to repeat step a4 for every method. to select a method (case b), the evaluation procedure can comprise the following steps: ajis vol 4 no. 1 b1. determine the task to be performed and the set of methods to be compared. b2. detect the specific set of principles by investigating the task and the methods. b3. describe the core meaning of each principle by filling out the appropriate slots in the principle descrip tion template (see 2.2). b4. for each method: fill out the slots in the principle description template that are related to the method under evaluation. b5. rate each principle regarding its suitability for the task. rate each method regarding its support of each principle. b6. compare the required rating of a principle with the provided ratings of the methods for the respective principle. the result of the comparison might be that no, one or several methods provide the methodical support needed. an evaluation schema can assist the comparison, visualized by a table (such as in figure 3). a column is specified for the principle's name, for the task and for each method that is examined. the ratings of the principles are depicted as rows. name of the principle principle 1 principle 2 principle 3 other selection enter/as rating concerning the task e.g. optional e.g.mandatory e.g. not necas. rating concerning method 1 e.g.++ e.g. e.g. 0 rating concerning method 2 e.g. + e.g. ++ e.g. 0 ... figure 3: an evaluation schema to select one or more methods for a given task (case b) additionally, the evaluation schema can be extended to consider non-methodical selection criteria. such nonmethodical selection criteria are for instance tool support, documentation quality, ease of handling and so on. the problem is to select appropriate measurements and to make the measurements comparable to each other. a formalism to describe principles in a uniform way all principles within a system of principles must be described in a uniform way. such a description should support the evaluation of a method and simplify the comparison of methods. a description template is used to describe the principles in a uniform way. description templates are frequently used to describe object-oriented design patterns. a description template provides a set of slots that are regarded to be useful in the specific context. the principle description template used here is based upon the conceptual ideas introduced by alexander (1979) and is closely related to the one developed for design patterns in gamma et al. (1994). similar to the two description templates it comprises a principle's name, rationale, description and discussion. all other slots are regarded to be specific for the context described here. the principle description template comprises the following slots: • name. a name is used to address the principle. • rationale. the motivation (what and why) for the principle is summarized. • description. the core meaning is described in a method neutral way. the constraints of applying the principle are discussed. • methodology. what methodical steps are possible and preferable? • correlation to the method under examination. how does the method address this principle? what is intentionally not implied? ajis vol 4 no. 1 • correlation to other methods. how do comparable methods address this principle? what additional aspects do they imply? what is intentionally not addressed there? methods are regarded to be comparable, when they support similar objectives, analyze similar objects of investigation and follow the similar paradigm. • discussion. the relationship to other principles are discussed. the given set of slots is regarded to be essential. additional slots may be added if appropriate, e.g. slots for examples, variants, to describe the applicability, to propose improvements or to guide a tool implementation. figure 4: the object model for the description of the methodical coherence regarding a principle to describe the methodical coherence regarding a principle a graphical notation (an object model, figure 4) similar to a meta-model is used. the object model consists of the elementary components object, relationship, attribute and service. therefore the methodical coherence regarding a principle is made out of a set of objects having relationships to each other. attributes and services are assigned to objects if they are necessary to clarify the semantics. the graphical notation is used to describe the core meaning and the method's viewpoints in detail. evaluation of a business process modeling method the approach was used to evaluate the method lovem-e (line of visibility engineering methodology enhanced; ibm (1995)) and to compare it with som ferstl and sinz (1994, 1995). lovem-e and som are two alternative methods for business process modeling. they assist business and system professionals in the specification and reengineering of business processes. the following specific set of principles were regarded to be relevant for the evaluation. they were found by investigating the notation (e.g. helping to categorize the terms of the notation; see 3.2) and documentation (e.g. claimed applicability) of lovem-e and som: • the model/view principle distinguishes between the actual specification results concerning the object of investigation and the presentation of the specification results to users. a model contains the specification results. views concentrate on the different textual and graphical representations of models. the separation of the functional core from its user representation helps to focus on these aspects separate from each other. such a separation allows a model to have different visual representations for different users. • the asls model/tobe model principle is used to explicitly specify different models of the object of investigation. business reengineering is assisted when you specify tobe models and compare different tobe models with each other and with an asls model. the model/requirement principle is used to explicitly specify the requirements a model has to meet and to separate the requirements from the model specification. the separation helps to specify consistent requirements and helps to verify the consistency of a (core) model concerning the requirements. measurements that are based on the requirements allow the evaluation of the model. the principle also allows the flexible combination of the requirements with different models. the models are then regarded to be alternative solutions to a given set of requirements. • the task/resource principle distinguishes between what is or what will be done (business task) from how it is or how it will be performed (resource). this differentiation helps to handle complexity by concentrating on either the tasks or the resources of a business as well as on the link between them. • the logical model/physical model principle focuses on abstraction. abstraction is one of the fundamental principles to cope with complexity. the principle provides explicit barriers between abstractions. in comparison the logical model represents the abstract layer whereas the physical model represents the detail layer. a.1is vol 4 no. 1 example of a filled out principle description template to demonstrate a filled out principle description template the asls model/tobe model principle is represented (figure 5): rationale, (already mentioned above) description. an asls model reflects the business at its current architectural level. any inconsistencies within a model are accepted as long as the inconsistencies also exist in the business. a tobe model is used to show what you would like your business to be. a tobe model must not be consistent with the real business. but it has to meet the prescribed requirements (see the model/requirement principle). v °.'l tobe model odel j°'* 0,' asls model you normally have a single asls model and none, one or more tobe models concerning a single object of investigation. different tobe models can be linked to each other. you can also specify tobe models without explicitly specifying an asls model. methodology. you can either specify asls models, tobe models or both. the standalone specification of an asls model can be used to document your business. the standalone specification of a tobe model can be used to present a vision or to improve your business. you can for instance use a given reference model to customize a tobe model for your enterprise. it is generally acceptable and easier to start with the explicit specification of an asls model. you have to overcome a smaller abstraction level and you learn to understand the actual business as well as the naming conventions. afterwards one or more tobe models can be specified. you can then relate a tobe model to the asls model (either explicitly or implicitly) or to other tobe models (e.g. comparing alternative solutions). correlation to lovem-e. lovem-e does only rudimentary support this principle by now. different models can be named as asls models or tobe models. some elementary design guidelines are given. the comparison of models is not further addressed. by now no special transformation functions are offered to support a step-by-step transformation of an asls model into a tobe model. also the comparison of two models is not supported, for instance to identify the location of differences and similarities within two models or to evaluate different models. correlation to som. som has started to support this principle (see ferstl and sinz (1995)). the strongly hierarchical approach using different layers of abstraction and refinement simplifies the identification of relevant asls and tobe differences. discussion. there are different implementations of the principle possible using the model/view principle. either an overall model consists of a set of sub-models for each asls and tobe model. different sub-model can then be closely interrelated. or the overall model reflects exactly one asls or tobe model. then the models are far more independent. the first solution allows to define separate views for each sub-model as well as views on the relationship of different sub-models. the second solution allows primarily the definition of separate views for each model. to provide a relationship among different models naming conventions must be used (e.g. instances of different models with the same name are identical instances). this principle may be expanded in the long run to cope with the evolution of a business. in course of time you will have a set of asls models and sets of alternative tobe models. the analysis of historical asls models in comparison to the performed business may then give you hints for improving your business. figure 5: an example for an tilled out principle description template (asls model/tobe model principle) categorization of the notation according to the principles the notation of lovem-e is very extensive. it comprises a rich set of terms (more than a hundred terms). as a result of the evaluation, the principles assisted in categorizing the terms of lovem-e according to the ajis vol 4 no. 1 principles. a term was assigned to a categorization if it contributes to the respective principle. figure 6 illustrates such a categorization for some essential terms of lovem-e. principle i categorization model/view principle model: (see model/requirement principle) view: (charts:) alovc, hsd, llovc, plovc, ltl (chart elements:) customer band, logical band, manual/automation line, manual band asls model/tobe model principle asls model: (no specific terms) tobe model: (no specific terms) model/requirement principle model: (see logical model/physical model principle) requirement: assumption/issue/recommendatio n; critical measurement point; critical success factor; goal/strategy/policy; opportunity area; problem area task/resource principle task: activity customer activity customer process data bus process system task data flow resource: business branch business function customer band external organization int/ext organization unit logical band organization unit logical model/physical model principle logical model: business branch business function customer band customer process data flow external organization llovc logical band organization unit process process bus physical model: activity customer activity int/ext organization unit jlovc plovc system task figure 6: simplified categorization of the notation of lovem-e according to the principles summary this pattern-oriented approach assists people (from a methodical point of view) in becoming acquainted with a new method, in comparing methods with each other and in selecting one or more methods out of a given set of methods for a prescribed task. a flexible evaluation schema is defined for the methodical evaluation of a modeling method by a specific set of principles (the patterns). each principle helps to examine and evaluate some parts of the method by focusing onto a specific aspect. all principles together lead to an overall picture of the method. the principles help to guide an evaluation. they point out the context where methods provide good, poor or no support. by this, the areas are discovered for which a modeling method is appropriate. ajis vol4 no. l acknowledgment this document is a result of a research visit taken place during 8/7/95 and 9/15/95 at the gsdl (german software development laboratory) of the ibm deutschland entwicklung gmbh in boeblingen. in this time the outlined approach was used to evaluate a business process modeling method under development and to compare it with another process modeling method. references alexander c. (1979) the timeless way of building, oxford: university press booch g. (1994) object oriented analysis and design with applications, 2nd edition, massachusetts: addi son wesley chen p. (1976) the entity relationship model towards a unified view of data, acm transaction on database systems, vol. 1, no. 1, pp 9-36 demarco t. (1979) structured analysis and system specification, englewood cliffs, new jersey: prentice hall ferstl o.k., sinz e.j., amberg m., hagemann u., malischewski c. (1994) tool-based business process modeling using the som approach, bamberger beitrage der wirtschaftsinformatik nr. 19 ferstl o.k., sinz e.j. (1994) multi-layerd development of business process models and distributed bus! ness application systems, bamberger beitrage der wirtschaftsinformatik nr. 20 ferstl o.k., sinz e.j. (1995) re-engineering von geschaftsprozessen auf der grundlage des somansatzes, bamberger beitrage der wirtschaftsinformatik nr. 26 gamma e., helm r., johnson r., vlissides j. (1994) design patterns elements of reusable objectoriented software, massachusetts: addi son-wesley international business machines corp. ibm (1995) business modeling tool user's guide, revised first draft, release 1 international business machines corp. ibm (1995) ibm lovem-e interim user's guide, draft, version 3 ross d.t., schoman k.e. (1977) structured analysis for requirements definition, trans. on software engineering, vol. se-3, no. 1, pp 6-15 rumbaugh j., blaha m., premerlani w., eddy f., lorensen w. (1991) object-oriented modeling and de sign, englewood cliffs, new jersey: prentice hall sinz e.j. (1988) das strukturierte entity-relationship-modell (ser-modell), angewandte informatik 5, pp 191-202 stein w. (1994) objektorientierte analysemethoden vergleich, bewertung, auswahl, mannheim: bi wis senschaftsverlag 10 microsoft word ajisfinal 12 2 marina.doc ajis volume 12 no. 2 may 2005 178 a descriptive analysis of decision support systems research between 1990 and 2003 david arnott*+ graham pervan# gemma dodson* *faculty of information technology monash university victoria 3800, australia email: david.arnott@infotech.monash.edu.au email: gemma.dodson@infotech.monash.edu.au # school of information systems curtin university of technology gpo box 1987, perth 6845, australia email: pervang@cbs.curtin.edu.au + corresponding author abstract this paper is the first major report of a project that is investigating the theoretic foundations of decision support systems (dss). the project was principally motivated by a concern for the direction and relevance of dss research. the main areas of research focus are the decision and judgement theoretic base of the discipline, the research strategies used in published articles, and the professional relevance of dss research. the project has analysed 926 dss articles published in 14 major journals from 1990 to 2003. the findings indicate that dss research is more dominated by positivist research than general information systems (in particular experiments, surveys, and descriptions of specific applications and systems), is heavily influenced by the work of herbert simon, is poorly grounded in contemporary judgement and decision-making research, and falls down in the identification of the nature of clients and users. of great concern is the finding that dss research has relatively low professional relevance. an overview of the direction of further analysis is presented. keywords decision support systems, group support systems, executive information systems, data warehousing, business intelligence, research, theory. introduction decision support systems (dss) is the area of the information systems discipline that is focused on supporting and improving managerial decision-making. in terms of contemporary professional practice, dss includes personal decision support systems, group support systems, executive information systems, online analytical processing systems, data warehousing, and business intelligence. this project was principally motivated by a concern for the direction and relevance of dss research. we suspected that research in decision support was increasingly being distanced from professional practice. we also felt that dss research was addressing an overly narrow range of concepts and issues and in particular we were concerned about the decision theoretic foundation of the area. further, it seemed that unlike the general trend in information systems research, dss was strongly dominated by a positivist, quantitative research orthodoxy. to explore these concerns we initiated the project described here. arnott, pervan, o’donnell and dodson (2004) provided some preliminary results for the project based on the analysis of 380 papers. since that report we have changed the sample to remove non-academic industry publications and have added articles from 2003 to the sample. as a result this paper presents the first major report of the project. it presents descriptive results based on the analysis of 926 papers. ajis volume 12 no. 2 may 2005 179 the paper is structured as follows: first, the background and rationale of the project is presented. the research methodology and design is then defined. the following sections discuss the results in terms of general research approaches, dss specific factors, and judgement and decision-making. finally, some concluding comments are made and the future directions of the project are described. background and rationale a number of information systems researchers are concerned that there is a widening gap between research and practice, particularly in the systems development area (galliers, 1994; saunders, 1998). fitzgerald (2000) argues that most current systems development methodologies are based on concepts developed in the period 1967 to 1977. he also argues that changes in the organisational and technical environment since that period have been so great that these methodologies need fundamental review and believes that professional practice is currently leading theory in the development methodology area. this has usually been the case but the divergence is probably greater now than at any other time. benbasat and zmud (1999) identified five reasons why information systems research lacks relevance. the first is an emphasis of rigor over relevance in order to gain the respect of other academic disciplines; the second is the lack of a cumulative tradition that yields strong theoretical models that act as a foundation for practical prescription; the third is the dynamism of information technology, which means that practice inevitably leads theory; the fourth is a lack of exposure of is academics to professional practice; and the fifth is the institutional and political structure of universities which limits the scope of action of is academics. dss research, as part of is research, is likely to be subject to all five forces. as mentioned in the introduction, one of the triggers of this project was the perception that the judgement and decision-making foundations of dss research are relatively narrow. in particular, where judgement is addressed explicitly in dss research, simon’s process model seems ubiquitous. simon’s model of decision-making (simon, 1956; 1977) has been used in dss research since the field’s inception and was an integral component of gorry and scott morton’s seminal mis/dss framework (gorry & scott morton, 1971). simon won the nobel prize for economics in 1978 for his theory and as a result it is extremely influential in many social sciences. his model remains the most cited instance of the phase theorem of decision-making. however, in psychological research grave doubts have been expressed about both the descriptive and prescriptive validity of the phase theorem (lipshitz & bar-ilan, 1996). in management research the descriptive and prescriptive validity of simon’s theory has been repeatedly disconfirmed (witte, 1972; alexander, 1979). there is no convincing empirical evidence for the prescriptive validity of any form of the phase theorem, including simon’s. these fundamental, and seemingly fatal, problems with a major foundation of dss theory have not been widely acknowledged, although a small number of researchers have raised concerns. angehrn and jelassi (1994, p. 269) went as far as to claim: “simon’s theory has become a serious obstacle for the evolution of dss theory and practice”. elam et al. (1992) argued for a broadening of the theoretical base of dss through the incorporation of contemporary behavioural decision-making research via collaborative projects with psychologists, while alter (1992) argued that research on dss as a technical object had biased the field and called for a greater attention on managerial work and decision-making processes in dss research. alter’s call for change was particularly important as he was an influential early researcher in the area. methodology and design the general research questions that guide this project are: ajis volume 12 no. 2 may 2005 180 1. what strategies and methods are used in dss research? 2. what is the decision support focus and professional relevance of dss research? 3. what are the judgement theoretic foundations of dss research? to answer these questions this project involves the analysis of relevant published research. this style of research has appeared under a number of descriptions in the information systems literature including ‘review and assessment of research’ (robey, boudreau & rose, 2000), ‘literature review and analysis’ (alavi & leidner, 2001), ‘survey’ (malone & crowston, 1994), and ‘literature analysis,’ (pervan, 1998). time frame the time period of published research chosen for this project is 1990 to 2003. the start of this analysis period is marked by two much cited reviews: eom & lee (1990) and benbasat & nault (1990). both of these reviews cover the dss field from its inception to the late 1980’s. a third review paper focusing on dss implementation, alavi and joachimsthaler (1992), provides a further anchor for the 1990 starting date of our analysis, as does the tims/orsa and national science foundation sponsored discipline assessment (stohr & konsynski, 1992). the period 1990 to 2003 also marks an interesting period in the development of the information systems discipline as it seemingly witnessed a significant growth in the use of non-positivist research methods. in industry, the analysis period saw the deployment of several new generations of dss, especially the large-scale approaches of eis, data warehousing, and business intelligence. to reflect these generations the sample has been divided into three time periods: 1990 to 1994, 1995 to 1999, and 2000 to 2003. the article sample the sample of articles analysed in this project is dss research published between 1990 and 2003 in 14 journals: accounting, management & information technologies/information & organization (i&o); decision sciences (ds); decision support systems (dss); european journal of information systems (ejis); information & management (i&m); information systems journal (isj); information systems research (isr); journal of information technology (jit); journal of management information systems (jmis); journal of organisational computing & electronic commerce (joc&ec); journal of strategic information systems (jsis); group decision & negotiation (gd&n); management science (ms); and mis quarterly (misq). previous meta-analyses of information systems research have used a similar sampling approach (benbasat & nault, 1990; alavi & carlson, 1992; pervan, 1998). alavi and carlson (1992) used eight north american journals for their sample. however, webster and watson (2002) have criticised the over emphasis on north american journals in review papers. in response we included three top-tier european information systems journals (isj, ejis, jit) and another (jsis) which has a strong european connection. an alternative approach is to focus on a small number of influential papers (alavi & joachimsthaler, 1992) or to aim for a comprehensive sample of all published research in the area including journal papers, book chapters, and quality conference papers (webster & watson, 2002). we adopted a large set of journals as a basis of the sample because we believe that this best represents the invisible college of dss research. the articles were selected electronically by examining key words and titles. the first and second authors performed a manual check of the table of contents of each issue of each journal. in addition, the text of each potential article for analysis was examined to verify its decision support content. the distribution of articles is shown in table 1. dss dominates the sample with 35.7% of articles. the discipline share of ds, dss, and joc&ec has fallen over time. the latter is probably due to that particular journal’s ajis volume 12 no. 2 may 2005 181 declining interest in collaboration technology. the european journals have a surprisingly low dss publication rate. table 1: sample by journal journal 1990 1994 1995 1999 2000 2003 total no of articles % of period no of articles % of period no of articles % of period no of articles % of sample i&o 5 1.4 8 2.0 2 1.2 15 1.6 ds 32 8.9 20 5.0 10 5.8 62 6.7 dss 118 33.0 166 41.8 47 27.5 331 35.7 ejis 9 2.5 9 2.3 3 1.8 21 2.3 i&m 40 11.2 31 7.8 22 12.9 93 10.0 isj 7 2.0 4 1.0 3 1.8 14 1.5 isr 16 4.5 11 2.8 5 2.9 32 3.5 jit 14 3.9 6 1.5 2 1.2 22 2.4 jmis 30 8.4 34 8.6 13 7.6 77 8.3 joc&ec 36 10.1 25 6.3 8 4.7 69 7.5 jsis 2 0.6 2 0.5 4 2.3 8 0.9 gd&n 13 3.6 59 14.9 39 22.8 111 12.0 ms 18 5.0 13 3.3 7 4.1 38 4.1 misq 18 5.0 9 2.3 6 3.5 33 3.6 total 358 100.0 397 100.0 171 100.0 926 100.0 procedure a protocol was used to code each paper. some papers, termed ‘example articles’, were selected as being representative of the various article types. to calibrate the coding process the example articles were coded independently by two researchers. the third researcher reviewed all responses. the articles were then coded by the three researchers working independently. in coding each paper the emphasis was on the dominant attribute of each factor for each paper. any uncertainty in coding was referred to one researcher for adjudication. the coding of citations of judgement and decisionmaking research was also reviewed by one researcher. the coded protocols were entered into an spss database for analysis by another researcher. this researcher also performed consistency checks on the coding. analysis by general research factors in addressing the first research question (what strategies and methods are used in dss research?) the general research factors considered were research paradigm, research stage, and article type. these factors are not independent but each is a useful lens for analysis in itself. the period of analysis 1990 to 2003 saw a significant move in general information systems research towards interpretivism (orlikowski & baroudi, 1991; walsham, 1995b; cavaye, 1996) and to a lesser extent, critical theory (hirschheim, 1992). a major consequence of this paradigmatic trend was the rise of the case study as a major research strategy in information systems (walsham, 1995a). the movement to a more complex and sophisticated disciplinary structure also occurred in social science in general (guba & ajis volume 12 no. 2 may 2005 182 lincoln, 1994). table 2 shows the empirical papers in the sample coded for paradigm. dss research is overwhelmingly dominated by the positivist paradigm with 91% of empirical studies following that approach. chin and hirschheim’s (2004) study of is research from 1991 to 2001 reported that 81% of papers had a positivist orientation with 19% using an interpretivist approach. this means that dss research is more dominated by positivism than general is research. examination of the temporal trends in table 2 shows that interpretivism in dss research is gradually expanding from its low base. table 2: sample by research paradigm paradigm 1990 1994 1995 1999 2000 2003 total no of articles % of period no of articles % of period no of articles % of period no of articles % of sample positivist 204 93.2 241 92.0 113 89.0 558 91.8 interpretivist 15 6.8 20 7.6 14 11.0 49 8.1 mixed 0 0.0 1 0.4 0 0.0 1 0.2 total 219 100.0 262 100.0 127 100.0 608 100.0 galliers (1992) proposed a framework for understanding research and its interaction with theory by conceptualising the research process as a cycle of theory building, theory testing, and theory refinement. table 3 shows the sample by the dominant stage in the research cycle. it shows that dss research is dominated by theory building. on the one hand this is surprising given the life of the area relative to it in general. given this longevity it could be expected that theory testing and refinement would now have a much greater focus. in the sample, theory testing has significantly expanded, albeit from a low base. an explanation for the statistics could come from the development of new dss movements, especially eis, data warehousing, and business intelligence. each new movement has required significant theorising and this may keep the theory building percentage of research high in the sample. table 3: sample by dominant research stage research stage 1990 1994 1995 1999 2000 2003 total no of articles % of period no of articles % of period no of articles % of period no of articles % of sample theory building 251 70.1 255 64.2 101 59.1 607 65.6 theory testing 72 20.1 113 28.5 58 33.9 243 26.2 theory refinement 13 3.6 12 3.0 4 2.3 29 3.1 unclear 22 6.1 17 4.3 8 4.7 47 5.1 total 358 100.0 397 100.0 171 100.0 926 100.0 there are a number of different approaches to classifying the type of research in addition to paradigm and stage of research. the approach used in this project is that used by pervan (1998) in his analysis of published group support systems research. pervan’s taxonomy was based on alavi and carlson (1992). the only modification has been to substitute “dss” for “gss”. the article type taxonomy and the distribution of papers are shown in table 4. also provided in the table is an example of each article type. table 4 shows that around one-third (32.9%) of dss research is non-empirical, with two-thirds ajis volume 12 no. 2 may 2005 183 (67.1%) empirical. chin & hirschheim’s (2004) analysis of overall is research reported a significantly different split between non-empirical (40%) and empirical (60%). dss research has significantly more empirical research than general is. the high 17.4% figure for the category “description of specific application, system etc” and the low combined case study score of 8.4% are particularly noteworthy. table 4. sample by article type article type number % nonconceptual dss frameworks 41 4.4 conceptual models 23 2.5 conceptual overview 45 4.9 theory 20 2.2 illustrative opinion & example 19 2.1 opinion & personal experience 4 0.4 tools, techniques, methods, model applications 91 9.8 applied concepts conceptual frameworks & their application 62 6.7 empirica l objects description of type or class of product, technology, systems etc. 29 3.1 description of specific application, system 161 17.4 events/processes lab experiment 176 19.0 field experiment 15 1.6 field study 33 3.6 positivist case study 48 5.2 interpretivist case study 30 3.2 action research 7 0.6 survey 68 7.3 development of dss instrument 4 0.4 secondary data 23 2.5 simulation 27 2.9 analysis by dss factors in answering the second research question (what is the decision support focus and professional relevance of dss research?) the dss factors addressed were dss type, organisational level of support, decision support focus, and practical relevance. decision support systems, while addressing the computer-based support of management decision-making, is not a homogenous field in terms of applications. there are a number of different approaches to dss and each has had a period of popularity in both research and practice (arnott & o’donnell, 1994). one way of classifying a dss is by the nature of the information systems development. each of these “dss types” represents a different philosophy of support, system scale, level of investment, and potential organisational impact. personal dss (pdss) are small-scale systems that are normally developed for one manager (or a small number of independent managers) for one decision task. pdss are the oldest form of decision support system (keen & scott morton, 1978) and include modelling systems and what industry currently terms “analytics”. in a pdss an individual manager has power or responsibility for the decision but in a group support system (gss) decision responsibility is shared by a number of managers and a number of managers need to be involved in the decision process. gss are typically implemented as electronic meeting systems (dennis et al., 1988) or group decision systems (pervan & atkinson, 1995). negotiation support systems (nss) also operate in a group context but as the ajis volume 12 no. 2 may 2005 184 name suggests they involve the application of computer technologies to facilitate negotiations (rangaswamy & shell, 1997). executive information systems were originally systems that aimed to support senior executives (rockart & delong, 1988) but quickly spread through all management levels. they are oriented towards reporting aspects of organisational performance using multidimensional databases or olap (online analytical processing) technology (codd, codd & salley, 1993). a data warehouse is a set of databases created to provide information to decision makers (cooper et al., 2000). there are two fundamental approaches to data warehouses: enterprise level data warehouses (inmon & hackathorn, 1994) and division or department level data marts (kimball et al. 1998). data warehouses can also be viewed as an attempt to provide a large-scale infrastructure for decision support in that pdss and eis can use data from the data warehouse and data marts. artificial intelligence techniques have been applied to decision support and these systems are normally called intelligent dss or idss (bidgoli, 1998) although the term knowledge-based dss has also been used (doukidis, land, & miller,1989). knowledge management as an information systems movement has also had an impact on dss research with a major conference on the topic being held in 2000 (carlsson et al., 2000). table 5 shows that the research is mainly focused in three areas: personal dss, group systems, and large data driven systems (eis and data warehouses). personal dss and intelligent dss are declining in attention while data warehousing, knowledge management-based dss, and negotiation support systems are increasing significantly, although data warehousing and knowledge management-based dss have a very low of exposure in major journals. this may be a factor in the professional relevance findings discussed later. table 5: sample by dss type dss type 1990 1994 1995 1999 2000 2003 total no of articles % of period no of articles % of period no of articles % of period no of articles % of sample personal dss (incl. analytics) 134 37.4 135 34.0 44 25.7 313 33.8 group support systems 104 29.1 125 31.5 55 33.9 287 31.0 eis (includes bi & olap) 27 7.5 30 7.6 12 7.0 69 7.5 data warehouse 0 0.0 2 0.5 9 5.3 11 1.2 intelligent dss 58 16.2 55 13.9 14 8.2 127 13.7 knowledge mgt based dss 3 0.8 6 1.5 8 4.7 17 1.8 negotiation support systems 6 1.7 18 4.5 17 9.9 41 4.4 many 26 7.3 26 6.5 9 5.3 61 6.6 total 358 100.0 397 100.0 171 100.0 926 100.0 another way of classifying a dss is by the unit of analysis of the research. the unit of analysis specifies the focus of the research project and is usually guided by the reference theories and previous domain research used by the researchers. table 6 shows that the decision support focus of the papers was reasonably spread across system development, information technology, the impact of the systems on the organization, and the decision-making process. over time researcher focus on ajis volume 12 no. 2 may 2005 185 development and technology has declined and research with a focus on decision outcome and organizational impact has doubled. intuitively, this mirrors the increasing organizational and social focus of is research in general. table 6: sample by decision support focus decision support focus 1990 1994 1995 1999 2000 2003 total no of articles % of period no of articles % of period no of articles % of period no of articles % of sample systems development 83 23.2 87 21.9 30 17.5 200 21.6 information technology 95 26.5 96 24.2 35 20.5 226 24.4 decision outcome/org impact 40 11.2 68 17.1 39 22.8 147 15.9 decision-making process 75 20.9 71 17.9 35 20.5 181 19.5 many 56 15.6 69 17.4 28 16.4 153 16.5 unclear 9 2.5 6 1.5 4 2.3 19 2.1 total 358 100.0 397 100.0 171 100.0 926 100.0 the final dss factor that was analysed was the practical relevance of the research in each article. any professionally focused academic area (like dss) needs a reasonable balance between theory development and application since research and practice inform each other (galliers, 1994). the assessment of practical relevance is a subjective judgement that was informed by the aims and objectives of each paper, the nature of the discussion, and in particular the content of the concluding comments of each paper. the researchers spent considerable time in discussing and reviewing their coding of this factor to assist in calibrating the independent coding processes. table 7 shows that overall, only 9.5% of research is regarded as having high or very high practical relevance. on the other hand, 53.2% of research was regarded as having no or low practical relevance. even though the high and very high practical relevance statistics vary over time periods the figures are so low as to constitute a potential crisis in the dss discipline. while the project was initiated with a concern for the relevance of dss research we were surprised by the strength of this adverse finding. we believe that all of the factors identified by benbasat and zmud (1999) are in play in dss research. the relative lack of exposure of academics to contemporary professional practice is a particular problem for dss. table 7: sample by practical relevance practical relevance 1990 1994 1995 1999 2000 2003 total no of articles % of period no of articles % of period no of articles % of period no of articles % of sample very high 3 0.8 2 0.5 4 2.3 9 1.0 high 31 8.7 22 5.5 26 15.2 79 8.5 medium 121 33.8 164 41.3 80 35.1 345 37.3 low 178 49.7 173 43.6 66 38.6 417 45.0 none 25 7.0 36 9.1 15 8.8 76 8.2 total 358 100.0 397 100.0 171 100.0 926 100.0 ajis volume 12 no. 2 may 2005 186 analysis by judgement & decision-making foundations the third focusing research question was: what are the judgement theoretic foundations of dss research? the first sentence of this paper defined dss as “the area of the information systems discipline that is focused on supporting and improving managerial decision-making”. the managerial nature of dss seems axiomatic and even one of the first dss books was titled “management support systems” (mccosh & scott morton, 1978). this project identified the primary clients and users in dss research by evaluating what organisational role was played, or was assumed to be played, by the primary client and user in each paper. table 8 shows the results of the application of this classification to the sample. of note are the very high figures in the unclear category: 88.8% for the primary client and 57.3% for the primary user. this lack of identification of the client or sponsor is particularly noteworthy as research has repeatedly found that executive and operational sponsorship are critical success factors for information systems that support managers (poon & wagner, 2001). this lack of identification of primary clients and users is a major shortcoming in dss scholarship. table 8: sample by primary client and primary user primary client primary user frequency percentage frequency percentage executive 52 5.6 67 7.2 non-executive manager 15 1.6 83 9.0 professional 24 2.6 118 12.7 other knowledge worker 13 1.4 34 3.7 many 93 10.0 unclear 822 88.8 531 57.3 total 926 872 each article was examined to see if any reference theory in judgement and decision-making was explicitly used. surprisingly, 45.8% of papers did not cite any reference research in judgement and decision-making. table 9 shows the number of citations to judgement and decision-making reference research for each type of dss. group and negotiation support have the most reference citations, with the current professional mainstream of data warehousing having the poorest grounding. as predicted in the rationale section, of those who cited judgement and decision-making references, the work of simon was by far the most popular. another surprising finding was that 79.8% of dss research did not use a form of the phase theorem of decision-making in their theoretical foundation. table 9: number of cited judgement and decision-making references by dss type type of dss no of articles mean standard deviation median personal dss 313 2.28 3.87 1.00 group support systems 287 2.69 3.22 2.00 eis 69 1.67 2.95 0.00 data warehouse 11 0.00 0.00 0.00 intelligent dss 127 0.81 1.73 0.00 knowledge management based 17 1.24 1.86 0.00 negotiation support systems 41 2.37 2.66 1.00 many 61 2.92 4.88 1.00 total 926 2.16 3.42 1.00 ajis volume 12 no. 2 may 2005 187 the general theoretical approach to decision-making can be classified in many ways. two of the most common classifications are used in this project, with the first being the difference between descriptive and prescriptive approaches a descriptive approach aims to describe how decisions are made in reality and these theories can be useful for understanding the context of decision support. prescriptive theories, which are often called normative theories, aim to recommend the best or most appropriate way to make a decision. some authors use the terms differently and use “prescriptive” for the theory space between purely descriptive and purely normative (bell, raiffa, & tversky, 1988). we use descriptive and prescriptive as descriptors as they are the most commonly used in dss research. both descriptive and prescriptive theories have been important for dss since the early days of the field (keen & scott morton, 1978). table 10 shows that a prescriptive approach dominates dss research. table 10: sample by decision-making approach 1 1990 1994 1995 1999 2000 2003 total no of articles % of period no of articles % of period no of articles % of period no of articles % of sample descriptive 90 25.1 86 21.7 49 28.7 225 24.3 prescriptive 183 51.1 202 50.9 79 46.2 464 50.1 both 0 0.0 0 0.0 1 0.6 1 0.1 unclear 85 23.7 109 27.5 42 24.6 236 25.6 total 358 100.0 397 100.0 171 100.0 926 100.0 the second classification of decision-making approach as being economic or behavioural overlaps with the first. economic approaches are usually aimed at maximising some objective subject to constraints and tend to be prescriptive (goodwin & wright, 1991) while behavioural decision approaches, which come largely from psychology, are usually based on an understanding of actual behaviour (for example, gigerenzer, 2000). nevertheless, behavioural approaches can be prescriptive and some economic approaches have descriptive aspects. table 11 shows that a behavioural approach dominates dss research. ajis volume 12 no. 2 may 2005 188 table 11: sample by decision-making approach 2 1990 1994 1995 1999 2000 2003 total no of articles % of period no of articles % of period no of articles % of period no of articles % of sample economic 90 25.1 73 18.4 33 19.3 196 21.2 behavioural 114 31.8 153 38.5 76 44.4 343 37.0 both 34 9.5 28 7.1 9 5.3 7.1 7.7 unclear 120 33.5 143 36.0 53 31.0 316 34.1 total 358 100.0 397 100.0 171 100.0 926 100.0 concluding comments this paper has reported the first results of a project that aims to critically examine the nature and theoretical foundations of dss research. although the reported analysis is only descriptive it does throw some light on the issues and concerns that motivated the study. amongst other findings, the analysis suggests that: 1. dss research is focussed on three main application areas: personal dss, group support systems, and large-scale data-driven systems. personal dss research is declining in influence while large-scale data-driven systems research is increasing. 2. dss research is strongly dominated by empirical studies that adopt a positivist ontology and epistemology. the most popular research methods used in this group of papers are experiments, surveys, and descriptions of specific applications and systems. dss research is more dominated by positivism than general is research. 3. the assessment of the practical relevance of dss research shows a discipline that is significantly distanced from professional practice. 4. the lack of identification of the nature of the primary clients/sponsors and the primary users of dss is a major shortcoming of dss scholarship. 5. almost half of published dss research is not grounded in judgement and decision-making research. 6. prescriptive and behavioural approaches to decision-making are the most cited in dss research. 7. the work of herbert simon is the most influential judgement and decision-making reference theory in dss research. these findings provide dss researchers with a call for reflexion and reassessment of their discipline. it provides signposts for redefining research agendas to ensure that the discipline prospers. without this reflexion and redirection we believe that dss will be increasingly distanced from professional practice, contemporary reference research, and other sub-specializations of is. the next stage of the project will involve more sophisticated and complex data analyses, in particular, cross tabulations and correlation analysis. in addition to the descriptive statistics reported in this paper, the questions that we are interested in pursuing include: • what research paradigms are dominant in the various types of dss? • what judgement and decision-making theories underlie the various dss types? • what are the organizational and development focuses of the different types of dss? • what types of dss have the highest practical relevance? ajis volume 12 no. 2 may 2005 189 • has the nature and amount of judgement & decision-making research cited changed over time? • what is the nature of dss research published in the different journals? • how is dss research different to general is research? further, we intend to investigate the nature of the financial support of high quality published dss research. in particular we are interested in which styles of research and which types of dss are supported by major competitive grants. it is hoped that this program of research can help dss researchers in understanding the trends in dss research, suggest future research opportunities and improve the quality and relevance of their research. in particular, it is important for australian researchers to focus more on data warehousing and business intelligence as these dss types have had relatively little research attention but are central to current australian it investment. references alavi, m., & carlson, p. (1992). a review of mis research and disciplinary development. journal of management information systems, vol 8 (4), 45-62. alavi, m., & leidner, d.e. (2001). review: knowledge management and knowledge management systems: conceptual foundations and research issues. mis quarterly, vol 25 (1), 107-136. alexander, e.r. (1979). the design of alternatives in organizational contexts: a pilot study. administrative science quarterly, vol 24, 382-404. alter, s. (1992). why persist with dss when the real issue is improving decision making? in t. jelassi, h. klein & w.m. mayon-white (eds.), decision support systems: experiences and expectations (pp. 1-11). amsterdam: elsevier science (north-holland). angehrn, a.a., & jelassi, t. (1994). dss research and practice in perspective. decision support systems, vol 12, 257-275. arnott, d.r., & o’donnell, p.a. (1994). business process re-engineering and decision support systems: a cautionary tale. in b.c glasson, i.t. hawryszkiewycz, b.a. underwood & r. weber (eds.). business process re-engineering: information systems opportunities and challenges (pp. 127-135). amsterdam: elsevier north-holland. arnott, d., pervan, g., o’donnell, p., & dodson, g. (2004). an analysis of decision support systems research: preliminary results. in r. meredith, g. shanks, d. arnott & s. carlsson (eds.). proceedings of the 2004 ifip international conference on decision support systems (pp. 25-37). prato, italy:ifip. bell, d.e., raiffa, h., & tversky, a. (1988). descriptive, normative and prescriptive interactions in decision making. in d.e. bell, h. raiffa & a. tversky (eds.), decision making: descriptive, normative and prescriptive interactions (pp. 9-30). cambridge, uk: cambridge university press. benbasat, i., & nault, b. (1990). an evaluation of empirical research in managerial support systems. decision support systems, vol 6, 203-226. benbasat, i., & zmud, r.w. (1999). empirical research in information systems: the question of relevance. mis quarterly, vol 23 (1), 3-16. bidgoli, h. (1998). intelligent management support systems. westport ct: greenwood. carlsson, s., brezillon, p., humphreys, p., lunberg, b.g. mccosh, a., & rajkovic, v. (eds.) (2000). proceedings of ifip tc8/wg8.3 international conference on decision support through knowledge management. stockholm, sweden: stockholm university/royal institute of technology, sweden. cavaye, a.l.m. (1996). case study research: a multi-faceted research approach for is. information ajis volume 12 no. 2 may 2005 190 systems journal, vol 6, 227-242. codd, e.f., codd, s.b., & salley, c.t. (1993). providing on-line analytical processing (olap) to user-analysts: an it mandate (unpublished manuscript). e.f. codd and associates. dennis, a.r., george, j.f., jessup, l.m., nunamaker, j.f. jr., & vogel, d.r. (1988). information technology to support group meetings. mis quarterly, vol 12 (4), 591-624. doukidis, g.i., land, f., & miller, g. (1989). knowledge based management support systems. chichester: eliis horwood. elam, j.j., javenpaa, s.l., & schade, d.a. (1992). behavioural decision theory and dss: new opportunities for collaborative research. in e.a. stohr & b.r. konsynski (eds.). information systems and decision processes (pp. 51-74). los alamitos, ca: ieee computer society press. eom, h.b., & lee, s.m. (1990). a survey of decision support system applications (1971-1988). interfaces, vol 20, 65-79. fitzgerald, b. (2000). system development methodologies: the problem of tenses. information technology and people, vol 13 (3), 13-22. galliers, r.d. (1992). choosing information systems research approaches. in r.d. galliers (ed.), information systems research: issues, methods and practical guidelines (pp. 144-162). london: blackwell scientific. galliers, r.d. (1994). relevance and rigour in information systems research: some personal reflections on issues facing the information systems research community. in b.c glasson, i.t. hawryszkiewycz, b.a. underwood & r. weber (eds.). business process reengineering: information systems opportunities and challenges (pp. 93-101). amsterdam: elsevier north-holland. gigerenzer, g. (2000). adaptive thinking: rationality in the real world. new york: oxford university press. goodwin, p., & wright, g. (1991). decision analysis for management judgement. chichester, uk: wiley. gorry, g.a., & scott morton, m.s. (1971). a framework for management information systems. sloan management review, vol 13 (1), 1-22. guba, e.g., & lincoln, y.s. (1994). competing paradigms in qualitative research. in n.k. denzin & y.s. lincoln (eds.). handbook of qualitative research (pp. 105-117). thousand oaks, ca: sage publications. hirschheim, r. (1992). information systems epistemology: an historical perspective. in r. galliers (ed.), information systems research: issues, methods and practical guidelines (pp. 2860). oxford: blackwell scientific publications. inmon, w., & hackathorn, r. (1994). using the data warehouse. new york: john wiley & sons. keen, p.g.w., & scott morton, m.s. (1978). decision support systems: an organisational perspective. reading, ma: addison-wesley. kendall, k.e. (1997). the significance of information systems research on emerging technologies: seven information technologies that promise to improve managerial effectiveness. decision sciences, vol 28 (4), 775-792. kimball, r., reeves, l., ross, m., & thornwaite, w. (1998). the data warehousing lifecycle toolkit. new york: john wiley & sons. lipshitz, r., & bar-ilan, o. (1996). how problems are solved: reconsidering the phase theorem. organizational behavior and human decision processes, vol 65 (1), 48-60. malone, t.w., & crowston, k. (1994). the interdisciplinary study of coordination. acm computing surveys, vol 26 (1), 87-119. mccosh, a.m., & scott morton, m.s. (1978). management decision support systems. london: ajis volume 12 no. 2 may 2005 191 macmillan. orlikowski, w.j., & baroudi, j.j. (1991). studying information technology in organizations: research approaches and assumptions. information systems research, vol 2, 1-28. pervan, g.p. (1998). a review of research in group support systems: leaders, approaches and directions. decision support systems, vol 23 (2), 149-159. pervan, g.p., & atkinson, d.j. (1995). gdss research: an overview and historical analysis. journal of group decision and negotiation, vol 4 (6), 475-485. poon, p., & wagner, c. (2001). critical success factors revisited: success and failure cases of information systems for senior executives. decision support systems, vol 30, 393-418. rangaswamy, a., & shell, g.r. (1997). using computers to realize joint gains in negotiations: toward an ‘electronic bargaining table’. management science, vol 43 (8), 1147-1163. rockart j.f. and delong d.w., 1988. executive support systems: the emergence of top management computer use, illinois: dow jones-irwin. robey, d., boudrea, m-c., & rose, g.m. (2000). information technology and organizational learning: a review and assessment of research. accounting, management & information technology, vol 10, 125-155. saunders, c. (1998). the role of business in is research. information resource management journal, winter, 4-6. simon, h.a. (1956). rational choice and the structure of the environment. psychological review, vol 63, 129-138. simon, h.a. (1977). the new science of management decision (rev. ed.). englewood cliffs, nj: prentice-hall. stohr, e.a., & konsynski, b.r. (eds.). (1992). information systems and decision processes. los alamitos, ca: ieee computer society press. walsham, g. (1995a). interpretive case studies in is research: nature and method. european journal of information systems, vol 4, 74-81. walsham, g. (1995b). the emergence of interpretivism in is research. information systems research, vol 6 (4), 376-394. webster, j., & watson, r.t. (2002). analyzing the past to prepare for the future: writing a literature review. mis quarterly, vol 26 (2), xiii-xxiii. witte, e. (1972). field research on complex decision making processes: the phase theorem. management and organization, vol 2, 156-182. ajis vol. 3, no. 2 computerised support for requirements modelling in foom terence c. h. tan paul a. swatman centre for information systems research swinburne university of technology p. o. box 218 hawthorn victoria 3122 email: cisr@swin.edu.au abstract this paper briefly introduces an information systems development/acquisition method, foom swatman & swatman (1992a) and justifies the need for computerised support for the method prior to further field trials. the paper then describes the requirements of a workbench which may be used to support foom and reports the experience gained in using foom itself in the requirements analysis and modelling processes. introduction the wide and continuing acceptance of the results of boehm's ibm and trw studies (boehm, 1976) which indicated that approximately 60% of system faults can be traced to specification errors, ambiguities and omissions suggests that the most promising of the approaches to alleviating the 'software crisis' may be those which focus on the problem of specifying software. swatman & swatman (1992a) describe a framework, illustrated in figure 1, for an information systems development methodology, widely applicable within the conventional information systems domain, which draws upon a number of established areas of research: • socio-organisational contextual analysis following the work of checkland (1981, 1989) and checkland & scholes (1990) which, in the general case, denies the existence of a single, objective requirements specification waiting to be discovered by the systems analyst • the object oriented approach in which situations are modelled as systems of interacting, encapsulated objects, each object belonging to some classes • mathematically formal specification languages in particular, the object oriented specification language object-z (duke et al., 1991) by means of which the abstract characteristics of classes may be described precisely and unambiguously all three areas make a contribution to the systems analysis and requirements modelling process. the socioorganisational approach is relevance-centred. it is concerned with increasing the likelihood that the impact of intervention within the organisational context would be understood—and, thus, that any intervention would be appropriate. although the major contributions promoted by adherents to the object oriented approach are concerned with engineering aspects of the systems development process (such as modularity, information hiding, robustness, reuse, traceability), the approach also offers a contribution to communication between specifier and client, in that object orientation is a natural modelling paradigm (see, for example, henderson-sellers & edwards (1994)). formal specifications were, initially, considered to be a conformance-centred approach to software development. they were presented as a basis for a formal 'concretisation' process^" leading to provably correct code. more recently, formal specifications have come to be acknowledged for their contribution to problem understanding (hall 1990a; sommerville 1989; swatman & swatman 1992; wing 1990), their precision offering potential for enhanced communication and evaluation of understanding. 10 this process is normally referred to as refinement within the specialist literature (see, for example, morgan (1994)). in this paper we use refinement in its more common sense of clarification or increasing relevance. 66 ajis vol. 3, no. 2 (existing) organisational context (new) ftrgani concent "i absorption i infoflnattoii analysis tionj fcl conflict statement validation problem kalement formal modeling infbnqition system formal problem sqicnicnt requirement] definition i implementation i solution definition j solution j " i design i figure 1: a framework for information systems development our preliminary research (fowler et al., 1993; swatman 1993; swatman et al., 1992; swatman & swatman, 1992a ) suggested that a socio-organisational approach could be used beneficially in concert with object oriented formal specification techniques within the information systems domain. the foom framework was developed by synthesis and logical argument, drawing on research in a number of largely independent areas across the breadth of the information systems and software engineering domains (swatman & swatman, 1992a, 1992b). preliminary evaluation of feasibility and potential benefit were undertaken by means of simulated (swatman et al., 1991) and small commercial (swatman et al., 1992) system specifications; and by means of educational case studies and pseudo-laboratory experiments (swatman, 1993) the need for case support the deliverables from the foom methodology contain textual, symbolic and graphical material. although there are tools which support the preparation of conventional text, object-z specifications, extended moses (henderson-sellers & edwards, 1994; wafula & swatman, 1995a, 1995b) and event chains diagrams (fowler et al., 1995), such tools are not, in any useful sense, integrated. importantly, they fail to take advantage of the strong semantic connection between the different parts of the specification—although it is possible to generate a set of extended moses o/c icons relatively easily, from a given object-z specification (wafula, 1995) (see figure 2). the preparation of a presentation quality object-z specification is currently a tedious process, involving a number of different tools, none of which is designed for ease of use. preparing an object-z specification typically requires a conventional editor such as vi, a graphical editor (we have so far used corel draw), a typesetter such as latex or troff, a viewer for the processed document (xdvi, ghostview), possibly a syntax or type checker (fuzz), and finally a program (dvips) to translate the 'device independent' output file to a postscript file suitable for printing. 67 a.iis vol. 3, no. 2 classa t:x note that class a, x. y attributes, operation, eto are not shown for brevity ske _0pl. ^op2. ji? : >' ill • = (a) class specification (b) o/c diagram figure 2: object-z class specification to moses o/c compounding the problem, the tools are usually command-line driven, each offering unique command switches and user-interface, making them hard to learn and unappealing to most is professionals. the project reported in this paper is dually focused: • the elicitation and specification of the requirements of a foom workbench which is intended to alleviate the problems described above by providing an integrated environment, designed specifically for the task of preparing the required documents (a conventional research and development project) • the use and evaluation of foom in this context (an action research project) the foom workbench the workbench is made up of the following components (the high level structure of the workbench is illustrated in figure 3): diagram editors components of the foom workbench figure 3: components of the foom workbench 68 a.i is vol. 3, no. 2 • the object-z specification editor is responsible for preparing the object-z specification of the system under investigation. besides providing all the required support for editing the object-z specification, it handles the decomposition of the object-z specification into its subcomponents (object-z paragraph). • the o/c diagram editor is used for the preparation of the extended o/c model11 diagrams, providing all the necessary editing support, such as the ability to connect one class icon to another. the key for retrieval of this information is based on the class's name associated with each icon. • the event model diagram editor is used to prepare the diagrams which show the dynamic interaction (the sequence of the communicated messages) of objects1 ̂ . this editor is conceptually similar to the o/c diagram editor, some of the notable differences being that an event model diagram does not show class inheritance relationships; and an object icon contains additional attributes to represent names and aliases of objects. • the event chain diagram editor provides support for the generation of diagrams which illustrate system dynamics. the basic shape of the icons used in an event chain diagram is based on the diagrammatic notation of moses. • the repository has an important role in keeping track of the various information provided by the tools. the object-z specification and moses icons of each system are kept together in a project folder. • the foom editor itself, which is the tool which combines the different types of information used in the preparation of a foom document. its basic functions are similar to those of a sophisticated word processor or typesetter description a schematic representation of the class structure of the workbench is set out in figure 4 and a complete formal specification in object-z may be found in appendix a of (tan, 1994). in this formal specification, an essentially arbitrary and pragmatic approach was taken to determining the level of granularity. for instance, although the class schema was specified, the classes which formed its subcomponents were not. further decomposition leading to a specification of finer granularity is straightforward but would, of course, result in a much longer specification. granularity decisions were made to maximise the communication of concepts while retaining simplicity—characteristics which are sometimes in conflict. foom workbench figure 4: the workbench class composition 11 the o/c model is primarily used to show class relationships such as the inheritance hierarchy. 12 in moses, the class icon can also be used to show dynamic interaction (see henderson-sellers & edwards, 1994). by contrast, in the object diagram notation of booch (1994), only object icons are used to illustrate the sequencing of messages. 69 ajis vol. 3, no. 2 there is an additional commonality between the various tools which is not apparent from figure 4: the use of a set of standard user-interfaces so that, for instance, the tool-bar and menu-bar look-and-feel would be similar across the system. the inheritance hierarchy relating to the tools is shown in figure 5. ~"~n gui \ provide common skeleton | windows program support: editor ! tool bar, status bar. menu, etc. case tool class hierarchy qenericlty figure 5: tools hierarchy tools while it is inappropriate to specify the requirements of the workbench in detail here, it is useful to discuss the structure of a general foom specification and the way in which the foom editor manages to assemble the resulting document from the fragments stored in the repository. the functioning of the foom editor depends heavily on the functions provided by its frame objects. most of the functions provided by foom editor are not native operations, but promoted operations of the frame objects which in turn rely on the services of their component objects. the frame class encapsulates the complexity of different types of information from the viewpoint of the foom editor and, to the foom editor, its operations have been reduced to simply keeping a list of frames—all the hard work specific to a particular type of information is handled by the correct type of frame. five types of frame are required in the preparation of the foom documents: • textframe which allows text to be typed into the document • ozframe which is used to contain a series object-z paragraphs • classdiag which holds a group of 0/c icons. • objectdiag which holds a group of object icons. • eventchaindiag which holds a group of event chain icons. the inheritance hierarchy rooted at the frame class is shown in figure 6 and an illustration of a foom document labelled to indicate the classes of objects it contains is shown in figure 7. 70 ajis vol. 3, no. 2 !scr*«nlt»m fr«n. frame class hierarchy figure 6: frame class hierarchy frame discussion and conclusions the problem domain the initial research plan was: • to investigate the case support required to facilitate further field trials of foom, then • prepare a detailed specification of the foom editor and the repository for prototyping. the first phase progressed smoothly, but as work on the second phase progressed, it quickly became apparent that many of the components of the foom editor also form components of other editors within the system (object-z editor, o/c editor, etc). the clear potential for reuse, while obviously beneficial from a systems developmental point of view, requires the specifier (and the formal specification) to confront three levels of abstraction simultaneously: description. • system level the interactions between the editors and the repository. • subsystem level the composition of the components which gives the editors their functionality. • component level the organisation of the components (i.e. class hierarchy and composition) understanding, then specification of, the components of all other editors within the systems was required earlier within the process than was expected. using soft systems methodology the use of ssm was rather limited due to the nature of the project. in a user-developer situation like this, where time and the project scope did not permit significant consultation with users, conflicts simply did not arise or could usually be solved in a very simple way1^. an approach consistent with ssm was used however, in the validation of the specification which, was achieved through discussion with members of the centre for information systems research at swinburne. 13 it is easy to establish a consensus with oneself. 71 aj1s vol. 3, no. 2 components of a foom document a textframe char the following invariant suggest that no member of the set should be an instance of class x, as x is an abstract class. an ozframe schema classschema a classdiag classlcon associationconn figure 7: the components (objects) of a foom's specification the requirements specification (architectural design) of the repository is essentially a hard problem. the term 'hard problem' is used to referred to problems which are essentially how questions (wilson, 1990). the design of the repository is such a how question. although the use of the soft system aspect of foom was limited, it was helpful. we constructed a rich picture14 and generated a number of root definitions (tan, 1994, appendix b). this is an unusually liberal approach to looking at the problem situation. ssm does not dictate what information should be represented, or how the picture will be used-'a the rich picture focuses on the interactions between users and the human activity system (not necessarily a computer system) and between users and users, which may be difficult to express in a more formal notation (moses for instance). its less constricting and non-prescriptive approach to looking at the problem situation seems to offer the potential of uncovering the needs of the users more completely. using formal methods the properties exhibited by each of the tools is complex, and deserves to be analysed in great detail. in this project we focussed on two tools in particular: the repository which forms the basis of the workbench implementation; and the foom editor which is used to assemble the information from the repository together with new information (text), entered directly into the foom specification. we considered that the detailed analysis of these two tools would be likely to result in a better design of the system and consequently, a formal specification detailing their relationships was prepared. 14 this technique is often used to generate a set of views of the system under consideration. 15 in fact, a rich picture does not even need to be a "picture1 (kreher, 1993). 72 ajis vol. 3, no. 2 although, the focus of the specification was on the two tools, the specification also included the rest of the tools, as they share common components. for instance, the object-z editor and the foom editor both use the services provided by the ozparagh class. nonetheless, to constrain the amount of specification to a more manageable level (given the time constraints of the project), the following measures were taken: • the specification focuses mainly on the exploration aspect of the system as a whole, investigating primarily the hierarchy of classes and the structure of objects in order to determine a good architectural design for the system. • concurrency issues are not considered in this specification (although it is accepted that support for multiple users is an important issue for an vxindustrial-strength" tool). • operations and attributes are defined in the classes on an vvas needed" basis16. this measure eliminated many operations in the class specification. the eliminated operations, however, did not affect our insight into the system as similar operations are defined elsewhere. • some classes were not defined but their basic properties can be observed from the definition of similar classes. the specification of the repository not only expresses the requirements of the system, it also conveys a substantial amount of design information. for instance, the classes frame and ozparagh would certainly be used throughout the development life cycle, and the operations specified would have their counterparts in the implementation phase. the specification of the workbench was not verified formally (by the proof of theorems) but was validated through a series of walkthroughs. the walkthroughs focused on the structure of the workbench and, in a minor way, on the mathematical correctness of the expressions. as expected, it was found to be difficult to rely just on the formal specification to explain the requirements and the design of the system. the class (o/c model) diagrams were most valuable in communicating the overall design rationale. one clear advantage of the presence of a formal specification was that it allowed us to simplify the diagrams as the details could be found readily in the specification when required. we did not find the event model of moses to be particularly useful in the modelling process because the system did not exhibit the type of complex object interactions which are shown on object interaction diagrams. the reason may be that the current design in object-z is expressing primarily the what aspect of the system, whereas the event model is intended to show the sequencing of events. although our experiences using object-z in this context were positive overall, there are some areas where we experienced difficulty: • abstract class the idea of a virtual class (no instance of this class should be allowed) should be made more explicit using a keyword. the advantage of this is that the virtualness of the class can be observed directly from the declaration rather from the invariants of the class. • the equality of two objects the new reference (which has superseded value-based) semantics of objectmakes the definition of equality (as opposed to identity) difficult. by equality we mean that two objects from the same class have the same values in their attributes and constants. the difficulty arises if we test the equality of two objects (say x and y ) which have an attribute (say a ) which is of class type. for x to be said to be equal to v, do we require x.a=y.a (i.e. that the references are equal and thus x.a is the same object as y.a—or merely that the object referred to by x.a is equal to the object referred to y.a. of course, x.a may also have one or more attributes of class type an operator could (we think) be formally defined for value comparison rather than identity comparison—such an operator would have been useful (a informally defined symbol was used to stand in for such an operator within the specification). using formal methods and object-orientation the architecture of the workbench was designed using a combination of the top-down and bottom-up approaches. identification of the subsystems was relatively straight forward as the subsystems are the tools themselves. the identification of the components of tools was, however, considerably more difficult as component behaviour was dictated by more than one subsystem (editor). for instance, ozparagh is used by the 16 usually, we tried not to apply this rule unless the addition of the attribute or operation made the system seem unduly complex. 73 ajis vol. 3, no. 2 repository, the object-z editor and the foom editor indirectly; and its design has a significant effect on the structure of the system. as another example, the idea of frame was not obvious initially, and the original design of the foom editor manipulated the objects created by other editors directly. this approach was not very successful as the foom editor was handling too much detail-^. the use of frame objects makes the management of objects from other editors more straightforward. future work although the current specification has established an architecture for the workbench—the partition of subsystems, the components of the editors, the class hierarchy of the components, the communications between objects, the delegation (promote operation) of responsibility, etc—more detail will need to be specified before the implementation. the following list is a sample of critical issues which could benefit from further formal specification: • concurrency issues policy of multiple access and update of records. • validation rules rules which will validate the consistency of the data should be specified. • the library folder should be generalised so that each project can have its own library folder. conclusions of the three components of foom the first component, the ssm approach, was involved least because the project was essentially a hard problem. the other two components—formal methods and object-orientation— played a much larger part, and were found to have complementary roles. for instance, the diagrams provided by the moses notation helped to explain and illustrate the behaviour of the system. in general, foom was found to be a useful approach to modelling the requirements of this system. the specification of the workbench reported here is currently being extended and refined into a prototype implementation as part of an arc funded project at swinburne university of technology. references boehm, b. w. (1976). software engineering. ieee transactions on computers, c-25 (12), 1226-1241. booch, g. (1994). object-oriented analysis and design with applications. benjamin cummings, redwood city, california, 2nd edition. checkland, p.b. (1981) systems thinking, systems practice. wiley, chickester checkland, p. b. (1989) soft systems methodology. human systems management, 8(4), 237-289. duke, r., king, p., rose, g., & smith, g. (1991). the object-z specification language: version 1. technical report 91-1, software verification research centre, dept. of computer science, university of qld, australia. fowler, d.c., swatman, p.a. & swatman, p.m.c. (1993). implementing edi in the public sector: including formality for enhanced control. in j.gricar & j.novak, editors, strategic systems in the global economy of the 90's, the 6th international conference on electronic data interchange and interorganisational systems, pages 244-256, kranj, slovenia. moderna organizacija fowler, d.c., swatman, p.a. & wafula, e.n. (1995). formal methods in the is domain: introducing a notation for presenting object-z specifications. object oriented systems, 2(2) hall, a. (1990). seven myths of formal methods. ieee software, 7 (5), 11-19. henderson-sellers, b. & edwards, j.m. (1994) booktwo of object-oriented knowledge: the working object prentice hall. kreher, h. (1993) critique of two contributions to soft systems methodology. eur. j. information systems., 2 (4), 304-308. morgan, c. c. (1994) programming for specifications. prentice hall international series in computer science, 2nd edition. sommerville, i. (1989) software engineering. addison wesley, wokingham, 3rd edition 17 the specification was, in particular, much less elegant and more difficult to understand. 74 ajis vol. 3, no. 2 swatman, p. a. (1993) using formal specifications in the acquisition of information systems: educating information systems professionals. in j.p. bowen & j.e. nicholls, editors, z user workshop: london 1992, workshops in computing, pages 205-239. springer verlag, london. swatman, p. a. & swatman, p. m. c. (1992a). formal specification: an analytical tool for (management) information systems, journal for information systems 2 (2), 121-160. swatman, p. a. & swatman, p. m. c. (1992b) managing the formal specification of information systems. in proceeding of the international conference on organisation and information systems, bled, slovenia swatman, p. a., swatman, p. m. c & duke, r. (1991) electronic data interchange: a high-level formal specification in object-z. in p.a. bailes, editor, 6th australian software engineering conference (aswec'91): engineering safe software, sydney, nsw. swatman, p. a., fowler, d. c. & gan, c. y. m. (1992). extending the useful application domain for formal methods. in j.e. nicholls, editor, 2 user workshop: york 1991, workshops in computing. springer verlag, london. wafula, e. n. & swatman, p. a. (1995a). the foom specification structure: two complete examples. centre for information systems research, working paper, swinburne university of technology, po box 218, hawthorn, vic, 3122. wafula, e. n. & swatman, p. a. (1995b) merging foom and moses: a semantic mapping from object-z to object communication diagrams. centre for information systems research, working paper, swinburne university of technology. wing, j. m. (1990) a specifier's introduction to formal methods. ieee computer, 23 (9), 8-24. 75 microsoft word ajisvol10 1_final_edit.doc ajis vol. 10 no. 1 september 2002 41 database security and the problem of polyinstantiation: a moral scrutiny mikko t. siponen university of oulu, department of information processing science, linnanmaa, p.o.box 3000, fin-90014 oulu university, finland. e-mail:mikko.t.siponen@oulu.fi abstract in the area of computer security the problem of polyinstantiation is widely recognized. the research on polyinstantiation can be considered morally questionable, since it involves lying. this being the case, a moral scrutiny on the problem of polyinstantiation is warranted. the morality of polyinstantiation shall be critically analysed from the viewpoint of a moral philosophical framework. the moral philosophical framework used includes 1) kantian ethics, 2) the universality theses advocated by hare, rawls, gewirth, jewish-christian ethics, and confucian ethics, 3) utilitarianism, and 4) theory of information ethics (ie) by floridi. the result of this analysis suggests that research and practice on polyinstantiation is morally questionable, at least in the light of the chosen moral philosophical theories. the aim of the paper is not, however, to deem polyinstantiation as morally wrong altogether, but to provide researchers and practitioners with tools and insights for analyzing the morality of polyinstantiation in different cases. moreover, the results suggest that, as far as polyinstantiation is concerned, traditional ethics theories seem to be at least as adequate as ie. introduction information security solutions have an increasing role in the information age, given that security solutions technically ensure or deny access to information. this being the case, the moral scrutiny of security actions cannot be overlooked particularly when security techniques are developed and used to provide false information. polyinstantiation – which is a central research issue in database security (e.g., lunt, 1991; pernul, 1992; jajodia & sandhu, 1995; ellmer et al., 1995; gollman, 1999) is such a solution (developed to provide false information). it is developed for maintaining different "realities" or cover stories to different user groups, and therefore it involves lying to users. in other words, users who have, say top secret clearences4, are allowed to see the "true" (top secret) information within databases, while people who have lower security clearences (e.g., secret, classified, etc) are only allowed to see false information. even though the issue of polyinstantiation is a well-known research issue in database/computer security and is morally questionable due to the lying involved, it has not yet been analyzed from a moral point of view. this might be partly due to the conventional moral notion5 of those working in the field, that security activities are morally good per se (i.e., security people are the good guys). also, the blind technocratic view that "the scientist accepts uncritically the goal a, without questioning it or without understanding his or her moral responsibility in producing tools for reaching a" (niiniluoto, 1993 p. 17) seems to prevail in the field. the aim of this paper is to analyze the morality of polyinstantiation. the issues of polyinstantiation will be analyzed through ethics theories including kantian ethics and the impartial universality thesis advocated by hare (1963; 1981) and rawls (1972). as the universality thesis plays a role in many other ethics theories (from philosophy to religion-based), an analysis in the light of it may satisfy, at least in that respect, the requirements of the advocates of such theories. moreover, the issues will also be reflected through the theory of information ethics of floridi (1999), which has not been critically analyzed, and utilitarianism. conceptual analysis in terms of järvinen (1997) is used as the research approach to yield these results. an early version of this paper is presented at the amcis’2000, long beach, ca, usa. the paper is organized as follows. in the second section, the framework for the analysis is presented. in section 2.1, the ethics framework is succinctly considered and reasoned, and in section 2.2 polyinstantiation will be briefly presented. in the third section, the morality of polyinstantiation will be analysed. in the fourth section, certain implications for research and practice will be discussed. in the fifth section, the key issues of the paper will be summarised. the framework for the analysis 4 herein the security level (e.g., top secret, secret, confidential) of the users. 5 the conventional (moral) notion refers to a situation where a person unquestionably conforms to the standards of a group or society (taylor, 1975; hare 1981). ajis vol. 10 no. 1 september 2002 42 the ethics framework the selected ethics theories are described below (table 1). table 1. selected ethics doctrines theory/ thesis objectives weaknesses and objections kant 1) sense of duty; 2) "impartial" universality principle; 3) the rule of human dignity; partial universality thesis utilitarianism maximize happiness why is happiness the key issue? utilitarianism does not count on the preferences of individuals. leads to negative utilitarianism (which leads to absurd conclusions) rawls' "veil of ignorance" and hare's universalisability of moral judgment universality thesis: “what if everybody were to do that?” our decisions are always subjective. agape one should perform loving or caring actions towards fellow people love can be blind and irrational the theory of information ethics four moral laws; principle of ontological equality the claim that one needs to respect all entities leads to difficulties. kant's r1 and r3 kantian ethics is a traditional theory, three points from which will be presented (e.g., raphael, 1994; warburton, 1996). the first is the sense of duty, the second the thesis of universality (i.e., act only on maxims that you want to be universal laws), and the third is the rule of human dignity (treat other people always as an end, never only as a means). these will be referred to as kantian requirements r1, r2 and r3. variations of r2 have been widely used in jewish-christian ethics (e.g., the golden rule in the case of christian ethics), confucian ethics, universal prescriptivism (hare, 1981), and rawls' theory of justice (1972), etc. agape and kant’s r1 kantian r1 has raised a debate on whether a sense of duty is relevant as a qualifier of the rightness of an action. actions stimulated by a feeling of love are not acceptable, for example, as love can be blind, and therefore, according to kant, confuse rational thinking. in this respect, kant's doctrine contradicts some different doctrines and interpretations of christian ethics, where love (agape) plays an important role in determining if an action qualifies as being morally acceptable (e.g., hare, 1992a, 1992b; jackson, 1999; macquarrie and childress, 1986 p. 354-356; thomas, 1955). agape means, for example, that “you shall love you neighbor as yourself” where neighbor means everybody (jackson, 1995 p. 3) and can be regarded as a kind of universality thesis. however, by looking at an objection to kant's ethics by floridi (1999), we may see that kantian r1 has its relevance. floridi argues that different sorts of harmless vandalism such as a boy stoning abandoned cars (floridi, 1999 p. 53-54) cannot be deemed morally blameworthy by using the kantian argument. first, floridi argues that "its ends/means maxim is inapplicable" (floridi, 1999 p. 54). secondly, the possible problem of the universality thesis, if applied to this case, is its possible bias towards subjective decisions (see below), resulting in that the mentioned vandalism would be accepted. however, floridi seems to have forgotten kantian r1. namely, it seems to be clear that kantian ethics regards this action (the boy stoning the abandoned cars) as morally wrong due to the requirement of a sense of duty (r1): its is difficult to imagine that a boy's sense of duty (in the kantian sense), given that the boy is interested in behaving morally right, includes such stoning of the abandoned cars. kant’s r2 and other universality theses by hare, rawls one weakness of kantian r2 (act only on maxims that you want to be universal laws) is its lack of impartiality. it seems to be possible to formulate maxims that are not impartial, when impartiality would bring more equality. ajis vol. 10 no. 1 september 2002 43 for example, the adding of different "epithets" such as age and status to the maxims to be universalised (in a kantian sense) can result in a partiality that can be difficult to justify. one might form a maxim, say, putting his/her name or age in it (also presuming that this also satisfies the first and the last kantian requirements), thereby allowing some privileges to people whose age and name are different from these, for instance. therefore, to avoid a weakness of this kind, the universality thesis used hereafter will be formulated in an “impartial” manner (although no universality thesis is really impartial) without the aforementioned qualifiers, which are likely to be irrelevant with respect to the morality of an action (see hare, 1989; kukathas & pettit, 1990; siponen & vartiainen, 2002 for more). hence, if we consider whether action x is allowed or not in the light of the "impartial" universality thesis, we need to ask whether we would accept that everybody were allowed to do x (in similar situations). an interesting objection which seems to apply to both partial and impartial forms of the universality thesis has been outlined by macintyre (1986). he sees that the universality thesis involves a kind of hubris, given that an individual universalising a judgment functions also as a 'universal legislator', and that judgment applies to all (thus, it is not supererogative, but the action in question is required). hence, macintyre ponders who has a right to be such a universal legislator? the idealistic idea of macintyre (1986) has a weakness. given that all actions are supererogative, i.e., not necessary, we are in trouble in practical life. to claim that the avoidance of killing an innocent person is also supererogative, i.e., the avoidance of the killing is not compulsory, but "virtuous", is not a very persuasive claim. the theory of information ethics the theory of information ethics (ie) is a novel approach by floridi (1999) that has not been applied much as of yet, and therefore, the reader should consider the author’s interpretation of it with a certain amount of circumspection. a key element of ie is the concept of information entity. every existing entity is a consistent packet of information and does not contain a contradiction in itself in terms of ie, since they are entities in the infosphere, needs to be respected in a certain sense; one should let the entities develop in their natural ways. the latter view is called the ontological equality principle (floridi, 1999 p. 44). we see this principle of ontological equality as an extended version of thesis of “equal regard” in christian ethics. the thesis of “equal regard” means that we should express same respect to all people irrespective who they are (e.g., outga, 1972) this ontological equality principle of ie becomes interesting from the point of view of anti-virus activity. it seems that a computer virus is regarded as an information entity in terms of ie: it is a consistent packet of information. being so in the light of the ontological equality principle, computer viruses (being information entities) should be let ”develop in a way which is appropriate to their nature” (floridi, 1999 p. 44). thus, if spreading viruses, for example, is an action which is appropriate to their nature, anti-virus activity may be wrong, at least from this point of view. additionally, ie has been criticized on pragmatic grounds, namely the process it uses for measuring the amount of entropy is a “cold” mechanistic calculation (siponen & vartiainen, 2002). they view that ordinary people may not easily associate entropy with wrongdoing. for that reason, ie may be less suitable for dealing with problems of moral motivation and moral distance than universality theses (e.g., the golden rule). to solve ethics dilemmas, for example, ie provides four principles (in order of increasing moral value). three of them are as follows) i) an action ought not cause entropy; ii) entropy ought be prevented and iii) entropy ought to be removed (the fourth principle will not be applied here). let consider these principles of ie from the viewpoint of anti-virus activity. anti-virus activity seems to violate the null law, given that the action of deleting viruses increases entropy (also creation and distribution of viruses may satisfy the third law). of course, a malicious computer virus may also violate the null law. in that case, one may like to interpret ie in such a way that the amount of entropy can be calculated. the final moral judgment would be the action that constitutes the least amount of entropy: it is likely that the anti-virus activity is the less harmful action. utilitarianism utilitarianism holds that under any given circumstances the action which produces the greatest amount of happiness on the whole is the right one. this involves taking into account everyone whose happiness is affected by the action in question. according to utilitarianism, we may 1) calculate the happiness of all beings who are capable of pleasure and pain and whose feelings are affected by our action; 2) or confine our concern to human happiness. the weakness of utilitarianism relates to its incapability of taking into account individual preferences. moreover, it does not take into account the "quality" of preferences, e.g., should we take the preferences equally into account. are there preferences that are "sick" or inherently bad, and therefore should not be counted? ajis vol. 10 no. 1 september 2002 44 additionally, utilitarianism may lead to negative utilitarianism, which in turn leads to absurd conclusions (e.g., warburton, 1996). finally, the use of utilitarianism to solve one's ethics conflicts is difficult. how can one person take into account everyone’s preferences? it would require a popular election. the reader may wonder why certain theories such as intuitionism (we have an intuition that is capable of discerning what is right and wrong), cultural relativism (e.g., moral values are culturally dependent: what is considered wrong in one culture may be acceptable in another culture) and emotivism (moral views are purely an expression of our emotions) are not applied. the reason is that we consider these theories as unfruitful with respect to moral thinking. they do not help us in conflicting situations and involve inconsistencies. say (case 1) that according to the intuitions of person x polyinstantiation is acceptable, and according to the intuitions of person y it is not acceptable. similarly (case 2), presume that in the culture of person x polyinstantiation is acceptable, and in the culture of person y it is not acceptable. what can be concluded from cases 1 and 2? is polyinstantiation right or wrong? which one, x or y is right? the possible reply that both x and y are equally right is not a persuasive answer for two reasons. first, how can the same action in the same situation be at the same time right and wrong? secondly, the answer that both x and y are right does not give us much help when we are considering whether polyinstantiation is morally right or wrong, or whether we should apply it. thirdly, theories of intuitionism, emotivism and relativism do not give us a rational account of why this action (or any action) is right or wrong, given that we do not recognize arguments such as "because this is my intuition/emotions/cultural view" as adequate reasons to act (or avoid an action). it is a matter for sociology to study what the personal intuitions (preferences), emotions and cultural preferences are. but such sociological facts (e.g., what people consider as right/wrong) do not indicate what is right and wrong, i.e., how we ought to act. emotivism, as advocated by stevenson (1944), i.e., stating that our moral concerns are nothing but expressions of our emotions, is confronted with similar problems. for example, ethics cannot be purely an expression of emotions, otherwise ethics discussion would consist of different sounds, such as grunts, and all moral argumentation would become difficult (e.g., warburton, 1995). moral values cannot be discussed or contemplated without reference to emotions, although emotions per se are inapt justifications for actions. polyinstantiation such conventional requirements of information security as confidentiality, integrity, and availability in the area of databases have raised rather interesting concerns. one of the concerns that will be explored here is the need for maintaining different concepts of reality between different classes of users. this requirement relates to databases which are referred to as being multilevel secure (here the aim is not to consider whether they are secure in this way or how successfully they meet the security requirements). a database system supporting many levels of clearances such as top secret (ts), secret (s) and confidential (c) is called a multilevel database system. one of the simplest reasons for having such a system is to ensure that users who have a clearance for seeing secret level information are able to view only that level, but not information of the top secret level, for instance. such a system, if it satisfies the requirement just mentioned, can be said to be multilevel secure (mls). this requirement has been further tightened to maintain different realities, in other words to support lying. this problem is technically termed as the problem of polyinstantiation. a relation can be said to be polyinstantiated when it has two or more tuples with the same primary key. the approaches to solving this problem technically can be categorised as entity or attribute polyinstantiations (jajodia & sandhu, 1995) and as they both similarly encompass lying, their technical differences will not be discussed here. table 1 models an imagined mls relational database system where the earlier mentioned classification of different user level clearances has been implemented. let us assume that this example describes a relation consisting of information about a space shuttle going towards satellite x for military purposes, and this real objective can only be viewed by the people who have top secret (ts) level clearances (table 2). table 2. an example of polyinstantiation. name destination aim space shuttle ts space shuttle s space shuttle c satellite x satellite x satellite y military scientific scientific this example (table 2) shows the lying involved, as the user having a confidential level clearance (c) is mislead to believe certain false information. that is to say, the confidentiality level users (c) are being convinced by the owners of the system/ts people that the destination of the journey is satellite y and the objective is purely scientific. ajis vol. 10 no. 1 september 2002 45 the moral status of polyinstantiation the question whether polyinstantiation is morally wrong in general cases and in three particular cases (a, b, c) will be considered in the light of selected ethics theories (kantian ethics, universality theses by hare, rawls, etc, utilitarianism, agape and the theory of information ethics). polyinstantiation in general cases is polyinstantiation acceptable in general cases? case a: lying to maintain stability let us assume (as possible argument a) that lying with respect to the security of security solutions is done in order to maintain a human life that avoids disturbance (whatever its exact amount is), since the expression of truth would (momentarily) increase the amount of disturbance or even chaos. this argument is chosen since it is interesting from the point of view of ie. case b: an appeal to private information one may try to justify polyinstantiation by appealing to the presumed right for informational privacy or ownership of information. for example, one may first argue that databases consist of “private” information, since the information stored in databases is input by the owner of the system. hence, the information is owned by the owner of the systems. therefore, one may argue that this “private” information or owned information should be under the control of the owner. moreover, this control includes that one should have “the right” to hide (parts of) the owned/private information (that is “true information, for instance), and furthermore the right to decide what is “the public side” (herein “false information”). the weakness of this argument is that it is not obvious that the owners of the systems (e.g., an organization) really own all the information – and even if they do own it, it does not follow that the owners have so full a control over it that it includes manipulation of the information. even though we might appreciate private information, even privacy has its limits. if this were not the case, ethics dilemmas with respect to privacy would not exist. for this reason, e.g., to see whether one’s privacy claims are justified, we need ethics theories. case c: it can be used for good purposes an often used justification for basic scientific research is to claim that the techniques being developed, even those associated with weapons research, can also be used for morally good purposes. therefore, the development per se, one can argue, is an amoral matter, but its application or further use is not. the aim of this paper is not to consider whether bombs, for instance, can be reasonably justified in that manner, but rather whether the techniques of polyinstantiation providing lying can be used for morally good purposes? that would perhaps be the case when such solutions would be implemented to prevent a murder of innocent people, for example if a murderer insinuates him/herself into a system having c-level clearances and is lured into a trap due to false information. it is, however, rather difficult to believe that most of the solutions would be used for those kinds of purposes. it is at least equally presumable that such a solution is provided to cover activities that are morally questionable. kantian ethics it is often claimed that kant holds a rather absolute view towards lying, as indicated in his doctrine “on a supposed right to lie from altruistic motives” (kant, 1997). this raises objections about how a society can really work if lying is unacceptable in the situations he describes. however, his view on lying can also be considered in the light of the division of perfect and imperfect duties, in which the latter are required only to some extent, and therefore this latter view, if accepted, is similar to a prima facie duty (hill, 1971). polyinstantiation also seems to violate kantian r3, since such feeding of false information may not recognize the autonomy of other people. for instance, a person's autonomy may be violated given that his/her decision is made on the basis of false information and the decision would be totally different if the individual were provided with true information. the result of this is that polyinstantion would be wrong according to kant. all the cases violate kantian principle of human dignity. ajis vol. 10 no. 1 september 2002 46 the universality thesis according to kant, a society cannot be based on a situation where lying is accepted (kant, 1974 p. 444), and commonly speaking, perhaps many of us would end up with a similar belief after application of the universality thesis: “but what if everyone lied?” thus, the aforementioned view (the kantian universality thesis) would likely regard lying as wrong in a general sense. this would perhaps also apply to lying related to polyinstantiation in a general sense. argument a is not likely to survive from the point of view of the (impartial) universality thesis: "do you prefer a society where such a activity would be accepted?"). even though in case a, if telling the truth would instantly increase disorder (let us assume so), we assume that many of us would still choose to tell the ‘truth’ (as it would be a better option even with small momentary increases of disorder than a situation based on lying). argument b is also questionable from the viewpoint of the universality thesis. that is to say, the impartial universality thesis may not support such an activity ("what if everybody were acting in that way?") the third case (c) may be acceptable in the light of the universality thesis provided that we would consider polyinstantiation as a necessary activity in order to avoid some activity that we consider more unwanted (e.g., killing in our example). agape in addition to the golden rule, christian thinkers may avoid polyinstantiation in all cases since it is difficult to see it as an expression of love (agape). it is difficult to see that lying in a general sense could be carried out to express love. one may argue that the third case (c) makes an exception. given that polyinstantiation is necessary for stopping more unwanted activities (that are very anti-loving activities), it can be claimed that polyinstantiation would be acceptable in such a situation. utilitarianism whether polyinstantiation is wrong from the utilitarian point of view, can be ascertained by counting the preferences of all people. lying in a general sense may also not increase our happiness, but if acting honestly maximizes happiness, lying would be morally wrong from the utilitarian viewpoint. for example, if a were the prevailing state of affairs, and further, assuming that disorder due to 'truth' implies unhappiness, the utilitarian view would allow lying. if, on the other hand, truth increased pleasure, the truth should be revealed. the theory of information ethics if, in that case (example a), one applied ie, then lying might be at first sight morally more favorable than telling the truth, given that lying does not increase entropy (consider the null law of ie), but it may also prevent entropy and telling the truth, considering example a, increases entropy. above it was presumed that lying does not increase entropy at all. if the lying does cause some amount of entropy, however, it may not change the result under consideration, given that telling the truth would increase entropy more than lying. although in that case both actions may not be morally good (given that they both increase entropy), one should perhaps perform the action (provided that these are the only possibilities) which is perceived better based on the level of entropy. of course, it is not clear that telling the truth (in situations such as a) would increase entropy at all. the applier of ie, however, might consider lying as morally wrong in case a in the respect that correctability (which is one of the constructionist information properties of the infosphere) is a feature of the infosphere that should be respected. it therefore seems that according to ie we should respect the correctness of information, which lying is most likely not to do. when it comes to the other cases (b, c), the amount of entropy would be the final factor deciding whether polyinstantiation would be morally acceptable in these situations. discussion, limitations and implications of the study this study has analyzed the morality of polyinstantiation in the light of ethics doctrines. there may be a few objections to the moral scrutiny of polyinstantiation that are worth discussing. firstly (a), one may claim that the lying is intuitively seen as morally blameworthy and the issue is therefore obvious. secondly (b), it may be claimed that research is value-free, as is often viewed among the positivism tradition. thirdly (c), one may claim that the issue of lying (irrespective of whether research is perceived as value-free or not) can be traced back to the dispute between positivism and anti-positivism and is therefore more a philosophical issue about the relevance of positivism and anti-positivism, which 1) cannot be settled (making discussion needless), and 2) is not a research issue of the is community, but philosophers. fourthly (d), one may argue that polyinstantiation ajis vol. 10 no. 1 september 2002 47 should be accepted due to technological determinism, i.e., "technological change is completely and unique determined by internal laws independently of human will" – and all technological possibilities should be realized (niiniluoto, 1990 p. 181). fifthly (e), one may claim that many security managers do not care at all whether polyinstantiation is morally wrong or right, and therefore the morality discussion is useless. we shall next consider these five objections. a) it may be the case that the notion that lying is wrong has been inculcated in many of us through educational systems (as a conventional moral notion), and/or we may consider the imperative “do not lie” as a prima facie duty/principle (following the terminology of ross (1930) and hare (1981), respectively) . the crucial question is, however, how do we know whether our beliefs or conventional moral notions are right? and what are the reasons why such lying may be wrong (or right)? therefore, an analysis of such lying from the viewpoint of theories of ethics will hopefully clarify the moral status of these issues. b) for the same reasons given that we want to be sure of whether research, and particularly research with respect to polyinstantiation, is value-free – we cannot avoid the moral scrutiny. c) we share the view of hare (1985) that the entry of the disputes between positivism and anti-positivism in moral matters is misleading. it is misleading, since the issue of whether research in polyinstantiation is morally praiseworthy or blameworthy is not epistemological (how we obtain knowledge) nor ontological (what exists), but conceptual analytical (see hare, 1985). to include the issue of positivism/anti-positivism into a moral inquiry does not clarify, but rather confuses the most important question, namely what are ultimately the right actions. we do not see that the dispute of positivism/anti-postivism is capable of providing any explaining justification one way or another, but at worst it can be used as a pretence to justify actions dogmatically. when it comes to objection c2 about the relevance of a moral scrutiny of polyinstantiation, let us recall a widely agreed view that is can be divided into three levels: organizational, conceptual and technical (e.g., iivari & koskela, 1987; iivari, 1989; lyytinen, 1987). the moral scrutiny of polyinstantiation is relevant to the is community since it encompasses the technical, conceptual and organizational levels. technical solutions are cases in point with respect to the technical level (e.g., see jajodia & sandhu, 1995; gollman, 1999), and the modeling of security aspects including polyinstantiation (e.g., pernul, 1992; ellmer et al., 1995; pernul et al., 1998) are examples of a conceptual level issue. finally, the issue of whether polyinstantiation is morally acceptable can be seen at the organizational level (the guidelines/principles of right/wrong conduct are examples of organizational level research). d) technological determinism is a naive viewpoint: if something can be done it does not mean that this thing should be done (e.g., niiniluoto, 1990). we can image several malicious actions that we could do in theory, but it does not follow that we should perform such actions just because they can be done. e) the issue should be important for managers, as well. firstly, it is widely agreed that humans are moral beings: most of us want to do the right thing (taylor, 1975; hare, 1981; warburton, 1996). secondly, employees, public and third parties are likely to hold a more positive view when organizational activities can withstand moral scrutiny. and even if the fifth argument is interpreted as a pragmatic argument claiming that some managers may see the discussion on the moral status of polyinstantiation as vain it does not follow that the issue should not therefore be discussed from a moral point of view. implications for practice the results of this analysis suggest that polyinstantiation is, in general sense, a morally questionable activity. based on the moral scrutiny of polyinstantiation, we shall formulate a practical guideline for practitioners and researchers. consider whether polyinstantiation/lying can be avoided? is such lying necessary? consider in which situations polyinstantiation is used. consider whether it is acceptable in these situations with the help of theories of ethics. for example, when applying the universality thesis, you need to consider the situation and ask: would you like it if people acted towards you in a similar manner, i.e., would you prefer that people in similar situations lied/provided false information to you? conclusions technical research and "design research" is generally quite applied by nature, resulting in that moral scrutiny cannot be avoided (e.g., niiniluoto, 1993). polyinstantiation, a product of technical or design science, is an action that is indeed relevant to consider from the moral point of view: polyinstantiation entails lying. lying with respect to polyinstantiation was analyzed in the light of different theories of ethics, namely kantian ethics, the universality thesis (expounded by many ethicist such as hare and rawls), the theory of information ethics and utilitarism. it is argued that the activity is morally questionable, at least in light of the used moral philosophical framework. none of the theories gave support for maintaining polyinstantiation in a general sense. lying violates the kantian rules of human dignity. ie advocates that information should be true, and most of us ajis vol. 10 no. 1 september 2002 48 would not want that everyone in a similar situation would provide us false information (universality thesis), and it is difficult to see it as an expression of love (cf., agape in christian ethics). of our cases, the third case (c) was the only one that may be acceptable for polyinstantiation. assuming that polyinstantiation is necessary for avoiding more negative activities than polyinstantiation, it could be considered acceptable. it is concluded that the traditional theories of ethics (other theories used excluding ie) seem to be as adequate as ie as far as polyinstantiation is concerned. we hope that this analysis has given new insights to practitioners as well as researchers, and therefore provides valuable information on the value of the research on polyinstantiation. references ellmer, e., pernul, g. and kappel, g. "object-oriented modelling of security semantics," proceedings of the 11th annual computer society applications conference (acsac'95), 1995, ieee computer society press. floridi, l. "information ethics: on the philosophical foundation of computer ethics," ethics and information technology, (11), 1999, pp. 37-56. confucious, (1993), analects. translated by r. dawson. oxford university press. gollman, d." computer security," wiley & sons, uk, 1999. hare, r.m. "freedom and reason," oxford university press, uk, 1963. hare, r.m. "moral thinking: its levels, methods and point," oxford university press, oxford, uk, 1981. hare, r.m. "ontology in ethics," in morality and objectivity: essays in memory of john mackie (eds): t. hondrich. routledge, 1985. hare, r.m. "principles, " in r. m. hare (eds.): essays in ethical theory, pp. 48-65, oxford university press, uk, 1989. hare, r.m. "are there moral authorities?" in r.m. hare (eds): essays on religion and education, oxford university press, oxford, uk, 1992a. hare, r.m. "euthanasia: a christian view, " in r.m. hare (eds): essays on religion and education, oxford university press, oxford, uk, 1992b. hill, t. e. "kant on imperfect duty and superegoration," kant-studien (62), 1971. reprinted in immanuel kant: critical assessments (eds): r.f. chadwick. volume iii, kant’s moral and political philosophy, pp. 347369, routledge, 1992. iivari, j and koskela, e. "the pioco model for is design," mis quarterly, (11:3), pp. 401-419, 1987. iivari, j. "levels of abstraction as a conceptual framework for an information systems, " in e. d. falkenberg and p. lindgreen (eds): information system concepts: an in-depth analysis, north-holland, amsterdam, 1989. jajodia, s. and sandhu, r. "solutions to the polyinstantiation problem," in: information security: an integrated collection of essays, (eds.): m. d. abrahams, s. jajodia and h. j. podell. ieee computer society press, los alamitos, california, usa, 1995. jackson, t. p. “love discosoled: meditations on christian charity,“ cambridge university press, uk, 1999. järvinen, p. "the new classification of research approaches," the ifip pink summary – 36 years of ifip, edited by h, zemanek, laxenburg, ifip, 1997. kant, i. "kant's gemmelte schriften: 4. abt. vorlesungen über moralphilosophie," walter de gruyter & co., berlin, 1974. kant, i. "critique of practical reason,” cambridge university press, uk, 1997. kukathas, c. and pettit, p. "rawls a theory of justice and its critics," stanford university press, california, usa, 1990. lunt, t.f. "polyinstantion: an inevitable part of a multilevel world," proceedings of the iv computer security foundations workshop, ieee computer security press, 1991. lyytinen, k., (1987), a taxonomic perspective of information systems development: theoretical constructs and recommendations. in r.j. boland & r.a. hirscheim (ed): critical issues in information systems research, john wiley & sons ltd, chichester, uk. macintyre, a. "after virtue: a study in moral theory," second edition, london, duckworth, uk, 1986. macquarrie, j and childress, j. (eds): "a new dictionary of christian ethics, " scm press ltd, 1986. niiniluoto, i. "should technological imperatives be obeyed?" international studies in the philosophy of science, (4:2), pp. 181-189, 1990. niiniluoto, i. " the aim and structure of applied research," erkenntnis (38), pp.1-21, 1993, kluwer academic publishers. ajis vol. 10 no. 1 september 2002 49 outga, g., “agape: an ethical analysis,” yale university press, uk, 1972. pernul, g. " security constraint processing during multilevel secure database design," proceedings of the 8th annual computer security applications conference, ieee society press, 1992. pernul, g., tjoa a. m. and winiwarter, w. "modelling data secrecy and integrity," data & knowledge engineering, (26), pp. 291-308, north holland, 1998. raphael, d.d. "moral philosophy," oxford university press, uk, 1994. rawls, j. "theory of justice," oxford university press, 1972. ross, d. "the right and the good," oxford university press, 1930. siponen, m.t. "the relevance of software rights: an anthology of the divergence of sociopolitical doctrines," ai & society, 2001, vol. 15, no. 1&2, pp. 128-148. siponen, m.t. & vartiainen, t. ”teaching end-user ethics: issues and a solution based on universalizability” communications of the association for information systems, 2002, vol. 8, article 29. stevenson, c. l. "ethics and language," new haven, usa, 1944. taylor, p.w. "principles of ethics an introduction," dickenson publishing company, encino, california, usa, 1975. thomas, g., “christian ethics and moral philosophy,” charles scribner's sons, new york, usa, 1955. warburton, n. "philosophy: the basics, " second edition. t j press ltd padstow, cornwall, uk, 1996. assessing the graphical and algorithmic structure of hierarchical coloured petri net models dr george l. benwell and dr stephen g. macdonell department of information science university of otago dunedin, new zealand abstract petri nets, as a modelling formalism, are utilised for the analysis of processes, whether for explicit understanding, database design or business process re-engineering. the formalism, however, can be represented on a virtual continuum from highly graphical to largely algorithmic. the use and understanding of the formalism will, in part, therefore depend on the resultant complexity and power of the representation and, on the graphical or algorithmic preference of the user. this paper develops a metric which will indicate the graphical or algorithmic tendency of hierarchical coloured pctri nets. introduction to petri nets since the 1960s when petri nets were first devised (petri, 1962) they have been modified and advanced. as a result their power and useability have increased. the application domain has also increased to reflect these improvements. originally a petri net was defined as a bipartite graph consisting of places, transitions, arcs and tokens, such that; p = {p], p2 pn) is the set of n places (graphically represented as circles) t= (t|, t2 tm} is the set of m transitions (graphically represented by rectangles) a = (the set of directed arcs connecting places and transitions}, and m = (the set of tokens resident in places at a given instant} (tokens are represented by dots). in addition to these constructs a net has associated sets of enabling and firing rules. these rules control the actions relating to transitions. the enabling rules determine under what conditions (or particular marking) a transition is enabled and therefore may fire. the firing rules determine what action will occur as a transition is fired. originally, these rules were simplistic, but nonetheless appropriate. they were; a transition, tj, is enabled iff all its input places have a token and none of its output places have a token (symons 1982), that is; tj is enabled when pj = 1 and pq = 0 where p| is the set of input places for tj, and pq is the set of output places for tj. the result of this early theory was that the nets were complex even when modelling relatively simple processes. this was considered to be a disadvantage, and in part, was responsible for a reluctance to use the nets. there was little point in producing a model that was potentially more complex than the phenomenon being modelled. it is important to emphasise here that complexity refers to structure of the graphical formalism. as the rules were simplistic a process had to be decomposed almost to an atomic level. simple rules led to the creation of complex graphics. again, complexity needs to be clarified. the formalism has only four constructs so complexity arises not from that number, but rather from their excessive and repetitive use. these deficiencies were addressed with the introduction of coloured petri nets and also hierarchical coloured petri nets (reisig 1991). the extension of the theory allowed for coloured tokens and complex algorithmic rules (op. cit.) to control the flow structure of the net. the rules supplemented and extended the notion of enabling and firing rules. rules could now be far more complex. the result was that nets were no longer rampantly complex in terms of graphics. it is now possible to represent a complex process with minimal graphics, where the graphical structure is transformed into coloured tokens and complex rules. so petri nets can be now described in the following manner (purvis and purvis, 1993); • a net structure, which is just like that of ordinary petri nets, september 1994 17 • a set of data declarations, and • a set of net inscriptions. the net structure is like that of place/transition nets: nodes consisting of places and transitions, and directed arcs that always connect nodes of different types (either places or transitions). the declarations component, defines the colours, number and types of variables in the net. each place, instead of having a capacity attached to it, now will have a colour set attached to it. colour sets are analogous to abstract data types in programming languages, and just as with abstract data types, each colour set can have an associated set of operations and functions that can be applied to tokens of that colour. because of this analogy with abstract data types, the declarations component could be expressed in terms of any of a number of programming notations. for example, jensen's (jensen, 1990) original notation of standard ml could be used as it is in a commercial implementation of coloured petri nets (metasoft corp., 1992). the net inscriptions, are expressions which can be attached to a place, a transition, or an arc; • places can have colour sets and initialisation expressions (analogous to the capacities and initial markings of place/transition nets). • transitions can have guards, which are boolean expressions that may contain operations on objects, such as constants, variables, and functions that have been defined in the declarations component. guards must evaluate to true in order for the transition to fire. if a guard always evaluates to true, then it is not shown. • arcs may contain arc expressions, which also contain constants and items that have been defined in the declarations component. when the arc expressions are evaluated, their variables are bound to the appropriate colours. the value of the expression must be equal to a multi-set (bag) of the colour that is attached to the place at one end of the arc. the arcs associated with a single transition (whether incoming or outgoing) have a common scope: any variable that appears more than once must be bound to the same colour. the end result is that there is now a continuum of combinations. a net could be highly graphical with little more than simple rules or at the other end of the spectrum, it could be hardly graphical and highly algorithmic. figure 1 presents this concept. it is contended that it is appropriate to derive a measurement to assess the relative percentages of graphics and algorithmic rules of a net. in the short term, this metric could be simply to determine if there are classes of nets. in the long term, it could be possible to match 'net type' with 'end user type' to maximise the outcome of the application of the petri net formalism. this paper uses the term 'structure' as a measure of the relative percentages shown in figure 1. it may be argued that it would have been better to use 'complexity'. this latter term is related here more to the number of constructs in use rather than their alternate representations (graphical or algorithmic). this definition is supported by symons (1982) where he states; the structure of a net is defined by the interconnection pattern of the places and transitions' symons (1982,p3) to be absolutely sure then, this paper deals with measuring the two forms of representation graphical and algorithmic and at this moment is not concerned with comparing two different nets for complexity (number constructs). the latter would be important when measurements of efficiency and clarity are required. petri nets and information systems petri nets have been utilised in information science since they were devised in the 1960s. it is true to say, however, that the use has not been wide spread. it is claimed that this is due to their complexity, though this objection is surely diminishing with the advent of coloured and hiearachical nets. in fact petri nets have been seen as a rigorous formalism and as such an appropriate substitute for data flow diagrams (tse et al, 1989, pi). in that vision the complexity was not denied and it should not be regarded as totally negative nor insurmountable. a synergy is required between the 'simplicity and informality' of data flow diagrams and the 'complexity and rigour' of petri nets. benwell et al. (1991) have proposed an extended use of petri nets in the system development life cycle. 18 ajis 100% 8 "3 'c s, net with high relative % of algorithmic rules net with high relative % of graphics graphics (g) figure 1 100% when an information system is to be created it is necessary to understand the phenomena to be modelled. any representation of the reality depends upon some method for categorising and defining what is being observed in a given situation. for example, for any situation being observed it may be desirable to categorise items as entity types or processes. during the observation, all phenomena and their most obvious situation-related processes are then classified. for instance, in a vehicle rental situation (such as figure 5a), a customer and a vehicle would be classified as entity types. this method, with some variations, has been described as an object-oriented approach to systems analysis (bailin, 1989). in entity relationship models entity types may be people, objects or events. in structured process modelling the emphasis is more on processes which relate to the dynamics of information in reality. using this method, the reality of the situation being observed can be described in terms of its data and processes. the next step is to produce an entity relationship model and data flow diagrams. so a variance on existing sytsem delevolpment methods is one that is seen to model dynamic processes rather than static models of data. introduction to metrics to assess the structure of a petri net model it may be appropriate substantially to adopt techniques from the field of software metrics, particularly those investigations that have attempted to assess and analyse graphical product representations. software metrics is the common name for the branch of software engineering that is concerned with the (normally quantitative) assessment of software project effort, quality assurance and the like. underlying all of these major goals is the fundamental attribute measurement of products, resources and processes (fenton 1991). as a petri net is a tangible system representation, this paper is most interested in methods of product assessment. some previously proposed product measures are very simple, for example, the number of lines of code in a program, or the number of modules on a structure chart. others, such as basili and hutchens' sync family of measures (1983) and the macro/micro measures proposed by harrison and cook (1987), are more complicated combinations of several product attributes. software product measures such as these have been used in a wide variety of functions. some have been used, for example, in the estimation of software development costs, or to assist in the prediction of maintenance requirements, or in the evaluation of particular development strategies (macdonell 1992). in some cases, however, the validity of these investigations has been subsequently challenged, as the measures used have been chosen inappropriately and have therefore failed to actually assess the characteristic(s) of interest (fenton 1991). this has occurred most especially when assessments of poorly-defined attributes such as 'understandability' or 'useability' have been the goal. the concern in this paper, however, is only with the direct assessment of petri net structure. it is therefore likely that problems of validity can be largely avoided through the use of relatively rigorous measurement definitions. once this assessment has been shown to be valid, it may then be possible to determine useful relationships between the structure metrics and other aspects of the software process. in addition, discussion here is not concerned with functionality as this should appear in both (or any) representations september 1994 19 of the same system. the core theme is to only consider the structure of the chosen representation in terms of path/decision metrics. figures 2a and b provide a simple example. while being simple it must be said that the example also displays some relaxed theoretical rules. that is, the sink or counter at p4 in (a) and p3 in (b), once marked by a token would bar the transition tl from being enabled again. this relaxed rule matters little in terms of the measurement of structure being discussed here. figures 2a and 2b are equivalent. the functionality represented graphically in figure 2a is distributed between graphics and explicit algorithmic rules in figure 2b. p3 p2 ifpl = r, b then rtop2 and btops else do nothing endif p3 (a) (b) figure 2 as described earlier, a petri net is a system representation formalism that incorporates both graphical and algorithmic components. structural assessment of the whole would intuitively require separate assessment of each aspect, as both contribute to overall net size and structure. given that the algorithmic component is concerned with processing control (jensen, 1990), which can also be depicted graphically, it would seem appropriate to use graphical analysis methods for both petri net components. metrics that are derivable from graphic product representations are widespread in the software engineering literature; for example, the methods described by chen (1978), benyon-tinker (1979) and szulewski et al. (1981). some are based on conceptual or functional system descriptions, such as demarco's bang method (1982). others are derived from design documents, and are therefore generally based on a product's calling structure. of most interest here, however, are those assessment techniques that may be applied to lower level product abstractions as represented in graph form, as these will be directly applicable to the graphical structure of petri nets. although many of these methods were initially proposed as measures of software product complexity (for example, see mccabe (1976), woodward et al. (1979) and chen (1978)), the underlying rationale of the methods is appropriate for the less ambitious assessment of petri net structure. measurement of petri net structure a graphical structure assessment method adapted directly from software metrics (but which, interestingly was originally derived from graph theory) is that which determines the 'cyclomatic complexity' of a program flowgraph (mccabe 1976). as a petri net can be considered in terms of a directed graph c (peterson 1977) with nodes p and t (places and transitions), edges a (arcs), and connected components c, the formulation of the graph cyclomatic number v is as follows (berge 1973): v(c) = a p -1 + c (1) equation (1) applies when the petri net graph is strongly connected (for strong connections see, symons, 1982, p7); that is, a directed path may be traced between any pair of nodes in the graph. 20 ajis some graphs, however, are not strongly connected. for example, a given model may have logical starting or ending points, or both. this can be observed in the form of 'start' nodes s or 'finish' nodes f, places that only have inputs or outputs respectively. for petri net graphs of this type the above formulation must be adjusted accordingly (henderson-sellers 1992). this is important as, at least intuitively, they are two basic net forms. the first may be called cyclic, having no end or beginning, and is a net that models the ongoing logic of a system. the other may be called episodic, which clearly have start and finish nodes. the fact that the difference may only be the level of abstraction or the 'view' of the system or process does not remove the need for the classification. thus for nets that include a start place and/or a finish place the equation for the cyclomatic number becomes: v(c) = ( a + i ) p t + c (2) finally for nets that have two or more start places or two or more finish places the calculation of the cyclomatic number is of the form: v(c) = ( ( a + l ) + (s + f ) ) (p + t + 2) + c (3) figure 3 is an example of how equations (1) to (3) may be applied to small nets. values of v (as calculated from equations (1) and (2)) were used by mccabe (1976) as measures of graph 'complexity', and subsequently as a basis for determining software testing strategies. this approach was based on the underlying assumption that a greater number of decision constructs meant a greater number of paths, and that this in turn contributed significantly to higher product complexity (also see (peterson, 1981, pi 18)). since it was first proposed, mccabe's method has been the subject of extensive investigation, with somewhat mixed results. some empirical support has appeared for example, see the studies reported by curtis et al. (1979) and hartman (1982) and, on the whole, the measure is viewed as one of the more effective software metrics (boehm 1981; arthur 1985; li and cheung 1987). in contrast, however, it has also been the subject of some criticism in relation to its treatment of path control constructs. this has led in some cases to metric refinements being proposed based on notions of intuitive complexity (myers 1977; hansen 1978; ramamurthy and melton 1988). in the current study, however, the issue of varying construct complexity is of minimal concern, as the interest lies only in the underlying path structure of a petri net (which is unaltered by construct variations), not how this relates to complexity or understanding (shepperd 1988). thus the measures are used here simply to provide standardised indicators of the graphical and algorithmic path structure of petri nets. the c term in equations (1) to (3) relates to the number of connected components. mccabe (1976) adopted this term as an adjustment factor for the use of sub-programs, so that the number of components equalled the main module plus all called modules. thus when determining the value of v for a single module the value of c was equal to 1. in the current context it seems natural to take the same approach the value of c for each distinct selfcontained petri net will therefore also be set equal to 1. in addition there may be some need to consider the hierarchical nature of coloured petri nets as a contributing factor to the structure of a net. this is believed to have more to do with functionality than structure (as already defined). so those places and transitions which link layers in the hierarchy will not be given any higher weight than they do as ordinary nodes. it has also been shown that transitions and places can be transposed into places and transitions to produce an equivalent net (peterson, 1981, pi 3). it would therefore seem logical that places and transitions are all nodes of equal significance. given these conditions the three equations above may be reduced (stetter, 1984; henderson-sellers, 1992): equation 1 becomes, v(c) = a p -1 + 1 (4) equation 2 becomes, v(c) = a p -1 + 2 (5) equation 3 becomes, v(c) = a p -1 + s + f (6) algorithmic structure assessment is likely to follow a very similar approach, as the coded rules in a petri net perform essentially the same function as the graphical form; that is, they control the sequence of functions through the use of conditional tests (purvis and benwell, 1993). thus the graphical assessment methods september 1994 21 discussed above will also be adapted for use with the algorithmic net component. this clearly requires the development and measurement of a flowgraph for each place, transition or arc in which rules are used. the total structure measure for the algorithmic component would be the sum of the individual block measures adjusted for the number of blocks (henderson-sellers, 1992). the petri net shown in figure 2b will serve as an initial example. figure 4 shows the concept of forming a graphic equivalent of the algorithmic component for assessment purposes. the net originally shown in figure 2(b) is therefore now represented as two graphs, with the numbers on each indicating the counts of places, arcs and transitions. the derivation of the algorithm-based graph (on the right of figure 4) is intuitive place 1 (pi) represents the if statement, arc 1 is equivalent to the then branch, arc 2 the else branch, the transitions represent the two "actions", and arcs 3 and 4 represent the implicit return of control to the next sequential statement, the endif in p2. for equation (1) strongly connected: jv(q = a p -1 + c = 8 2 4 + 1 = 3 (a) for equation (2) one start place, one finish place for equation (3) two or more start places, two or more finish places v(c) = ((a +1) + (s + 0) (p + 1 + 2) + c = 14-11 + 1 cfo (c) figure 3 22 ajis / n 6 i fp l=r,b then rtop2 and btop3 else do nothing endif graphical component algorithmic component figure 4 net equivalent to the algorithmic component using equations (5) and (6) the metrics for graphical structure (sg) and algorithmic structure (sa) for figure 4 can be expressed as; = a p 1 + s + f sa = a p 1 + 2 so, from figure 4 it is determined that; (7) (8) and sg = (3 3 1 + 1 + 2) = 2 sa = (4 2 2 + 2) = 2 it may be held that it is necessary to determine the number of tokens in the net as a contributing factor toward structure. while this may be so, tokens are not considered separately as; 1. the number of types of tokens is the significant factor not the number of tokens, 2. the types of tokens are considered indirectly in the assessment of the algorithmics, 3. the number of tokens relates more to net complexity than net structure. safeness and conservational aspects of a net, directly concern tokens but relate to complexity and are not assessed here. as an example consider a petri net modelling a car hire process (benwell, firns and sallis, 1991) (see figure 5). in a part of the net, cars are represented by tokens; the number of cars will dictate the number of coloured tokens (but not necessarily the number of colours). this will not inherently change the structure of the net nor in fact complexity until such time as the processes to handle the number of cars change. then a new process is being modelled so it should be expected that structure (and maybe complexity too) will be different. therefore a net to model 10 cars or a 100 cars will have the same structure, provided all cars are treated equally and the process to handle the number of cars is the same. the measure of overall petri net structure sp, as given by equation (9), could be solely used as a tool for net evaluation. notwithstanding that possibility, the thesis of this paper is to derive a metric that will classify nets as predominantly graphical or algorithmic and then, and only then, to draw some conclusions as to any differences between the two types. structural dominance can be expressed in terms of graphics or algorithmics according to the following; sg/spdpg = but, in cases in which sg and sa contribute to the structure: (10) (11) (12) september 1994 23 live contracts customer signs contract (yellow) customer customers using vehicles customers finish with vehicles able vehicles (say 100 black) customer decide to rent customers settle contracts completed contrac prospective customers (say 100 red) car hire net (adapted from (benwell et. al. 1991)) figure 5 so, it is not possible to directly compare the influences and sum them to 100%. it is desirable to have the sum of equation (12) equal to one. this can be achieved by adjusting sp. therefore, and dpg = sg/sp. and, therefore, dp a=l (13) (14) (15) (16) now the two components may be compared directly as complementary percentages (equation (16)). the outcome is a direct comparison, but sp' no longer represents the total count of all paths through the net. equations (14) and (15) enable the assessment of the graphical or algorithmic dominance in a given petri net. some 'classes' of petri net user may prefer graphically or algorithmically dominated representations. the consequential matching of a graphical (/algorithmic) net with a graphical (/algorithmic) user has not escaped attention; though it is not the subject of this paper. for example, project managers as a group, may prefer a higher graphic component, as this may enable more effective communication. conversely, programmers might be better served by having algorithmic dominated nets, so that the implementation of the functions described by the net are made more straightforward. in addition, there may be classes of problems that are better portrayed in either representation. at this stage, however, the paper is not concerned with why different preferences exist. given the ratio form of the petri net dominance indicators, the value of dp/j) will fall somewhere along the continuum from graphically dominated to algorithmic dominated. when dpg equals dpa the structure is evenly distributed between the graphical and algorithmic components. a different rendition of figure 1 conveys the concept in figure 6. 24 ajis <sg increases <dpg = dpa -> increases -> «<«<«<«««<««< «<«<«<«<«<«<«<«<«<«<«<«<««<«c«<«<«<|»»»»»>»»»»»»>»»» »»»»»>*»»>*»»» graphical not dominated algorithmic domination by either domination figure 6 the dining philosophers problem the dining philosophers problems is now universally established (peterson, 1981, p65 and krishnamurthy, 1989, p54) and is a good practical example for comparing the structural dominance of the graphics and algorithmics. the petri net for the philosophers (adopted from peterson (op. cit.)) in given in figure 7. the problem is that there are five diners and five eating utensils, the latter can be assumed to be forks (a, b, c, d, e in figure 7). a diner needs two forks to eat. if two forks are not available the diner must be a true philosopher and forego eating for thinking. it is only possible for two opposite diners (say t and '3' or t and '4' for example) to eat at the same time. figure 7 represents this situation where diner t is eating, the immediate left ('5') and right ('2') dinner partners must think, while either of the upper two ('3' or '4') may choose to eat. if one chooses to eat, the other cannot. this situation is cyclic and continues ad nauseam. the net configuration, as presented, makes a deadlock impossible. in the more general case it would be possible where any diner can select a fork (as distinct from a pair) at a given time. the graphical measure of this net is given by; sg = a p t + c = 40-1510+ 1 = 16 and sa = 0, so sp = (sa + sp) ( c l) = 0+ 1 6 ( 1 l)= 16 sp' = sp + (c1)= 16 + (1 1)= 16 therefore, dno = sjsn= 16/16 = 1 » 100% dpa = sa/sp' = 0 » 0% which leads to the obvious conclusion that the net is graphically dominated. now a new net can be created where some or all of the graphical logic is transposed to algorithmics. this is presented in figure 8. calculation of the graphical structure of the figure 8 representation is relatively straight forward; sg = a p -1 +c = 4 2 2 + 1 = 1 the assessment of the algorithmic structure, sa, however, requires some preliminary calculations. given that there are two blocks of algorithmic code in the representation, the value for each block must be determined first; sal = ( a + l ) p + c = (30+1)17+ i = 15 sa2 = (a + 1) p + c = (4+ l ) 4 + 1 = 2 september 1994 25 the dining philosophers (adapted from, peterson, 1981, p65) figure 7 case of 5 philosophers if fork left and fork right then do function a end case case of 2 diners then do function p end case collapsed dining philosophers' net figure 8 26 ajis as stated previously, the total algorithmic structure value is the sum of the individual block measures adjusted for the number of blocks. hence; sa = ( s a l + s a 2 ) ( c l ) = (15 i -2 ) (2 1) = 16 now that values for sg and sa have been obtained the remaining indicators of overall structure, sp, sp', dpg and dpa can be evaluated; sp = (sg + s a ) ( c l ) = (1 + 16) (2 -1 ) = '6 (as can be expected, the overall structure value for this representation is equal to that of the representation in figure 7). sp. =sp + ( c l ) = 16 + (21) = 17 therefore, dpg = sg/sp' = 1/17 = 0.06 » 6% dpa = sa/sp. = 16/17 = 0.94 » 94% these values now show that the net, as represented in figure 8, is dominated heavily by algorithmic structure. an intuitive assessment would no doubt have generated the same conclusion; the calculations above, however, provide quantitative relative indicators of this fact. conclusion this paper has adopted and derived a numerical technique to measure the relative percentages of graphical and algorithmic structure of a petri net. the basis for the calculation has been the research and development undertaken in the area of software metrics. the metric so derived can be utilised to indicate the influence each component has on the structure of a petri net. this has fulfilled the initial goal of this research. in the longer term it may be possible to match the attributes of a net to the characteristics of a user and thereby enhance the functionality of petri nets as a formalism. references arthur, l.j. (1985) measuring programmer productivity and software quality, john wiley & sons, new york. bailin, s.c. (1989) "an object-orientated requirements specification method", communications of the acm, 32, 5. basili, v.r. and hutchens, d.h. (1983) "an empirical study of a syntactic complexity family", ieee transactions on software engineering 9 (6), pp 664-672. benwell, g.l., firns, p.o. and sallis, p.j. (1991) "deriving semantic data models from structured process descriptions of reality", journal of information technology, no. 6, pp 15-25. benyon-tinker, g. (1979) complexity measures in an evolving large system, conf. proc., workshop on quantitative software models for reliability, complexity and cost, new york ny, usa (1979, pp 117-127. berge, c. (1973) graphs and hypergraphs, north-holland, amsterdam. boehm, b.w. (1981) software engineering economics, prentice-hall, englewood cliffs nj. chen, e.t. (1978) "program complexity and programmer productivity", ieee transactions on software engineering 4 (3), pp 187-194. september 1994 27 curtis, b., sheppard, s.b., milliman, p., borst, m.a. and love, t. (1979) "measuring the psychological complexity of software maintenance tasks with the halstead and mccabe metrics", ieee transactions on software engineering 5 (2), pp 96-104. demarco, t. (1982) "controlling software projects", yourdon, new york. fenton, n.e. (1991) software metrics, chapman & hall, london. hansen, w.j. (1978) "measurement of program complexity by the pair (cyclomatic number, operator count)", acm sigplan notices 13 (3), pp 29-33. harrison, w.a. and cook, c. (1987) "a micro/macro measure of software complexity", journal of systems and software 7, pp 213-219. hartman, s.d. (1982) "a counting tool for rpg", acm sigmetrics per 11 (3), fall, pp 86-100. henderson-sellers, b.h. (1992) "modularization and mccabe's cyclomatic complexity", communications of the acm 35 (12), pp 17-19. jensen (1990.) coloured petri nets: high level language for system design and analysis, in advances in petri nets 1990, springer-verlag, berlin. krishnamurthy, e.v. (1989) parallel processing, principles and practice, addison-wesley, sydney, 332pp li, h.f. and cheung, w.k. (1987) "an empirical study of software metrics", ieee transactions on software engineering 13 (6), pp 697-708. macdonell, s g. (1992) quantitative functional complexity analysis of commercial software systems, unpublished phd thesis, university of cambridge. mccabe, t.j. (1976) "a complexity measure", ieee transactions on software engineering 2 (4), pp 308320. metasoft corp. (1992) metasoft corporation manual, design/cpn version 9, metasoft corporation, cambridge, mass. myers, g.j. (1977) "an extension to the cyclomatic measure of program complexity", acm sigplan notices 12 (10), pp 61-64. peterson, j.l. (1977) "petri nets", computing surveys, vol. 9, no. 3, pp 223-252. peterson, j.l. (1981) petri net theory and the modelling of systems, prentice-hall, eaglewood cliffs, new jersey, 290 pp petri, c. a. (1962) communication with automata, supplement 1 to radc-tr-65-377 vol. 1, griffiss air force base, new york, originally published in german 'kommunikation mil automaton', university of bonn, cited in (peterson, 1981, p 3). purvis, m.k. and benwell, g.l. (1993) "a causal agent approach for modelling dynamic systems", new zealand computer society, conference proc., 35th nzcs conference, hamilton, new zealand, pp 592-604. purvis m.a. and purvis, m.k. (1993) "dynamic modelling of the resource management act", conf. proc., 5th annual colloquium, spatial information research centre, university of otago, new zealand, pp 225-240. ramamurthy, b. and melton, a. (1988) "a synthesis of software science measures and the cyclomatic number", ieee transactions on software engineering 14 (8), pp 1116-1121. reisig, w. (1991) a primer in petri net design, springer-verlag, berlin. shepperd, m. (1988) "a critique of cyclomatic complexity as a software metric", software engineering journal, pp 30-36. stetter, f. (1984) "a measure of program complexity", computer language (uk) 9 (3/4), pp203-208. symons, f.j.w. (1982) "the application of petri nets and numerical petri nets", research laboratories report 7520, telecom, australia. szulewski, p.a., whitworth, m.h., buchan, p., and dewolf, j.b. (1981) "the measurement of software science parameters in software designs", acm sigmetrics per 10 (1), pp89-94. woodward, m.r., hennell, m.a., and hedley, d. (1979) "a measure of control flow complexity in program text", ieee transactions on software engineering 5(1), pp45-50. 28 ajis ajis vol. 8 no. 2 may 2001 cops, computers, and the right to privacy in the information age: unauthorised access and inappropriate disclosure of information complaints in new south wales 'mike and 2lily enders 'school of policing studies, charles sturt university, 2new south wales ombudsman's, sydney new south wales menders@csu.edu.au keywords police, computer, access, privacy, information, technology, email, ethics introduction the term the 'information age' is particularly applicable to australia. in a recent email, the australian institute of criminology's chief librarian, john myrtle, passed on statistics which showed that internet use and access in australia has increased about 50% during the last year (pers. comm. 14 july 1999). of greater interest is the fact that almost 20% of australian households, 1.3 million, have internet access and over one third of the adult population has accessed the internet at some time during the year ending february 1999. to further back these figures, the sydney morning herald of 12 february, 2000, carried statistics from the australian bureau of statistics which showed that 22.6% of australian families had home internet access (anon., 2000a, p. 105). these figures firmly place australians among the world's most computer literate societies. of course computers weren't always that popular. the authors of this paper entered law enforcement at a time when computers were owned by universities and major corporations not individuals and a decent calculator cost about a week's wages. however, things changed quickly and by the 1980s computers were an established part of policing. today, all major police services are committed to, and reliant on, some form of computerised information system. the two systems which the authors have had contact with are the crime reporting and information system for police (crisp) (queensland police service) and the computerised operational policing system (cops) (new south wales police service). while many aspects of these two systems are different, they, and all the other police information systems in existence, share one major similarity: they store and provide access to personal and confidential information on every individual with whom police come into contact during their duties. modern police investigation techniques rely on officers being able to access this information routinely to carry out their duties and fight crime. likewise, the individuals whose personal information is held on law enforcement computer systems are entitled to a reasonable expectation of privacy. this presents the single greatest dilemma to law enforcement officers using computers: when does investigation and intelligence gathering become common curiosity and therefore unlawful? is there a problem? as early as 1979 it was acknowledged by the new south wales police service (nswps) that computer terminals could be manipulated to produce false records such as traffic and fingerprint records. it was also identified that security measures concerning the provisions of documents for court, e.g. a person's criminal record, were inadequate. as recently as 1988, there was a complaint that an officer was paid $1500 to ensure that when a particular person with prior convictions for drink driving appeared in court, their previous record was not mentioned so that they would be treated as a first offender (nsw ombudsman, 1988). unlawfully accessing computer information is an offence in nsw under s309 (1) of the crimes act 1900 no. 40 (nsw) (see annexure 1). the offence is complete upon access to data being gained regardless of the intention of the person who accesses the data. if the data is accessed with some intent or if the data is of a specifically confidential nature, a circumstance of aggravation applies and the penalty increases markedly from 50 to 500 penalty units and six months to two years imprisonment. the ombudsman's annual reports reveal that unlawfully accessing computer information is a major issue in the nswps. as annexure 2 shows, complaints to the nsw ombudsman have increased at a fairly consistent rate over the last few years. it could be argued that the more familiar officers have become with computers, the more prevalent unlawful access and related complaints have become. it could also be argued that the number of these complaints is small, an average of 350 each year, compared to the number of employees, about 16,000. several things need to be considered in this context: 40 ajis vol. 8 no. 2 may 2001 • assuming that each person was only complained about on one occasion, about 15% of employees have been the subject of complaints over the last seven years; • at worst the complaint trend is increasing, at best it is not decreasing (brammer claims that the nswps figures for 1995-99 represent a downward trend, however, apart from a reduction in the 98-99 year, the figures appear to be stable or increasing (brammer, 2000, pp. 10-11). the 98-99 results are encouraging.); • in the 1997-98 year, 99 complaints of inappropriate disclosure were fully investigated, 42% of these investigated complaints resulted in an adverse finding or were conciliated (nsw ombudsman, 1998, p. 61); • in the 1997-98 year, 127 access complaints were fully investigated, 57% of these complaints resulted in an adverse finding or were conciliated (nsw ombudsman, 1998, p. 61); • there is no guarantee that the figures reflect more than a small proportion of the problem since most access complaints come from random checks conducted by the service; and • a complaint can involve an enormous number of potential offences (in one case, an officer was found to have committed 1,592 accesses, which even after the impact of statute barring, translated to 71 criminal charges and one departmental charge.). it is fairly clear that information technology is creating a problem within law enforcement (brammer, 2000; nsw ombudsman, 2000). (the recent hearings conducted by the qld criminal justice commission suggest that the problem is not restricted to nsw.) these complaint figures are particularly disturbing when one considers that after the introduction of cops and following the release of an independent commission against corruption (icac) report on unauthorised release of government information (icac, 1992), the nswps instituted a training program for officers to eradicate instances of unlawful access. in fact in the year following the commencement of training, complaints rose from 173 in 93/94 (nsw ombudsman, 1994) to 217 in 94/95 (nsw ombudsman, 1995a). the seriousness of these issues led to two reports from the ombudsman. improper access to and use of confidential information by police (nsw ombudsman, 1994b) was followed by a further report to parliament in 1995, confidential information and police (nsw ombudsman, 1995b). this report set out the 'positive police action' taken by the service to deal with the problem. the service's initiatives included a statement of responsibility (see annexure 3) for information and information systems. officers were required to sign this statement after completing the training referred to earlier. while not really amounting to a code of ethics for information access, the statement was based on ethical principles and the legal constraints which apply to computer access. unfortunately, it did not work. is the public entitled to an expectation of privacy a recent canadian case, r v weir (1996) involving child pornography provides an excellent example of the degree of privacy the law ascribes to information on a computer network (geddie, 1999). ab was sent an email message from another person. the message contained an extremely large graphic/video file which contained pornographic images of young children. the file was so large that it jammed the internet service provider to which ab subscribed. while fixing the problem, a technician discovered the content of the file and the police were contacted. the file was copied and given to the police and then delivered to ab's email box. the police obtained a warrant using the copied file and searched ab's home where they found the original file and seized his computer's contents and a disk. ab was charged with possessing child pornography. the court found that the act of copying the file for the police did not amount to an unwarranted search. more importantly, the court examined the nature of email compared to regular mail and decided that email carried with it a reasonable expectation of privacy. therefore, a warrant is required to seize email stored on a computer network. by extension this case confirms that the owner or intended recipient of even a seemingly innocuous email message stored on a computer is entitled to a reasonable expectation of privacy with regard to the data. a recent nsw case involving two women who were awarded $150,000.00 compensation after being sacked over sending emails to each other which defamed a supervisor seems to confirm this finding (harris, 2000a). most of the information held on police computer systems is much more confidential in nature than the average email and therefore a higher level of privacy should be expected. recent privacy legislation passed in nsw should clarify the situation here. further, in nsw, the provisions of section 309 (and section 310) of the crimes act apply to all computer systems, not just police or government computers. this effectively makes 'computer hacking' in any form an offence. 41 ajis vol. 8 no. 2 may 2001 what harm is done by access? the nature of computer access offences investigated varies from officers accessing information about themselves through to accessing information to pass on to a third party for payment corruption. it is obvious from this broad spectrum of breaches that an ethical solution needs to cover a lot of territory. it might be useful at this point to consider a number of case studies to provide a better understanding of what unlawful access means. the cases selected are representative of the types of information access complaints overviewed by the nsw ombudsman. case 1 curiosity (ombudsman, 1995b, pp. 6-7) a random audit of cops revealed that an officer had accessed cops on 1591 separate occasions to check on personal details of family, friends, former school class-mates (for a reunion), workmates and various motor vehicles in which he was interested. the police investigation was not completed until after six months had passed. eventually the officer was charged with 71 offences under section 309(3) and one departmental charge of misconduct. (at this time a statute of limitations of six months existed for offences under s309 (1). it is now two years.) case 2 noble cause curiosity (ombudsman, 1995b, pp. 11-12) a member of the public complained that his former brother-in-law, a police officer, had accessed his criminal history. the police officer claimed that he had information connecting the complainant with an armed hold-up. the explanation provided was found to be flawed in several respects. further, the officer also breached a commissioner's instruction that police officers not involve themselves in matters concerning friends or relatives. there was a clear conflict of interest in this case which was not recognised by the officer. remarkably, this also went unnoticed by the service's investigating officer and the acting region commander. the service found that the complaint was 'unsustained'. the ombudsman disagreed. case 3 amnesia and the 'good samaritan 'gone wrong (ombudsman, 1995b, pp. 12-15) a woman complained that a man had harassed her on two occasions demanding compensation for damage to his motor vehicle. a check of the cops indicated that a particular officer had accessed the woman's details and might have passed the information to the man whose car was damaged. the officer claimed that he 'could not remember' and that someone else must have used his password or his computer terminal whilst it was unattended. the officer was attached to the cops performance support unit and should have taken steps, in accordance with the statement of responsibility to avoid others using his password or active terminal. the matter of officers forgetting to log out of computer terminals received press coverage as recently as july, 1999 (keogh, 1999) the complaint was sustained. this particular case is remarkable for the poor quality of advice received by the investigating police from region legal services. the advice seems to exempt police from the provisions of section 309 by virtue of their office. the director of public prosecutions clarified the matter and established that a charge could indeed be supported in this case. case 4 corruption (ombudsman, 1995b, p. 7) as a result of information from 1c ac that an officer was passing information to a firm of private inquiry agents, a particular police officer's access of cops was examined. the officer was found to have accessed confidential details of a large number of people who were the subject of insurance or small debt claims or family court action. the officer subsequently pleaded guilty to twenty-one charges under section 309 after being allowed to resign from the service. he was fined a total of $2,700. 42 ajis vol. 8 no. 2 may 2001 case 5 cyberstalking there are many cases of male officers accessing the police data base to satisfy their curiosity about people, particularly young women, in whom they have a personal or sexual interest and further indiscreetly disclosing information about those people, e.g. one officer accessed personal details about his dentist's receptionist as he 'liked the look of her' and wanted to find out her age which he then passed on to his male colleagues (nsw ombudsman, 1992). in another case, a detective senior constable accessed personal information about his exwife to ascertain her financial situation and the people she was associating with (nsw ombudsman, 1993). another detective, an inspector, accessed the criminal history details of his former daughter-in-law's new husband, while another officer charged with assault accessed details of witnesses due to appear in his own court matter (nsw ombudsman, 1993). while traditional stalking requires that the activity create fear in the victim, these cases differ in that often the victim is unaware of the stalking until either, the stalker becomes a traditional type of offender, or, often in the case of nsw complaints, an investigator interviews them regarding contact with an officer as the result of internal random audits of information use. the fact that the fear is generated about past events of which they had no knowledge at the time should not diminish the victim's rights to see justice done. there can be little doubt that accessing personal information about individuals constitutes, at worst an attempt or at least preparation for, a stalking type offence. at best it is an unwarranted and unlawful invasion of a person's privacy. the ease of access to databases afforded by the internet is increasingly putting a strain on people's right to privacy (harris, 2000b). in the case of police officers, these matters are aggravated by the extent and privileged nature of the information provided to, stored on, and accessible to police officers on the service computer system (nsw ombudsman, 2000, pp. 13-15). the service has a genuine need for information from members of the public and other agencies to investigate crime. the public has a right, supported by legislation to expect that information they give to any agency will be used lawfully. the misuse of computer access by a substantial number of nsw police officers can only reduce public confidence in its police service (brammer, 2000, p. 3). case 6 misuse of email and internet access as stated previously, the nswps is experiencing the same problems as other corporations regarding the distribution of offensive material on in-house email networks and the use of the internet to harass others (harris, 2000a & 2000b; miranda, 2000). the nsw ombudsman is currently conducting investigations into this area. one issue which must be a major concern is the amount of time spent by officers gathering and distributing this material, especially in the current climate of claims for more staff and higher salaries (walsh, 2000). these cases draw attention to a number of issues which lie at the bottom of computer related complaints: • curiosity needs to be curbed at worst curiosity, noble or not, has the potential to do irreparable damage to people's character and reputation; • police at all levels have great difficulty recognising conflict of interest issues (the nsw ombudsman has reported to parliament on this specific issue recently.); • computer related complaints need to be taken very seriously as a corporate issue due to the potential for minor accesses to escalate to corruption; • the existing training programs and disciplinary system does not seem to be reducing the incidence of breaches; • officers do not seem to realise the gravity of committing breaches; • some officers do not seem to receive sufficient supervision regarding time spent using the internet and email; and • some police service senior staff and internal investigators are not aware of relevant legislation or the seriousness with which computer related breaches should be viewed. recent developments recently other aspects of police misuse of computer technology have received media attention and are being actively investigated the use of email to transmit sexually explicit material, including pornography, inappropriate jokes and personal comments about staff and clients, and revenge/harassment. (it should be remembered that these issues are not restricted to police officers. it simply shows that the adoption of technology by the nswps makes them vulnerable to the same technology-related problems faced by any corporation 43 ajis vol. 8 no. 2 may 2001 (omond, 2000; anon., 2000b). it is encouraging to note that the nswps is addressing these problems (brammer, 2000; nsw ombudsman, 2000). does ethics provide the answer? the authors believe that it would be extremely easy to blame a lack of ethics training for problems surfacing due to modem advances in information technology. ethics is a universal construct and deserves more respect than to be 'trotted' out whenever a problem becomes too hard. if a law enforcement officer does not know the difference between right and wrong before they sit down at a service computer terminal, giving them a dose of ethics training is not necessarily going to improve the situation. to be effective training should be conducted before officers are faced with the responsibility of accessing any information including computer data. the problems emerging with advances in technology are simply blinding policy-makers to the real issue officers have not been given the training they require. it is easy to train police to use technology. it is much harder to teach them how to use that technology with proper discretion. first they must be taught to respect the trust placed in them when computer access is granted. trust is a quality which is essential to public office, law enforcement and good governance (delattre, 1995, pp. 60-61). if an officer was asked, 'do you have the right to go to a colleague's desk, open their drawer and examine their reports, notes and diaries without permission?' the answer would be a resounding 'no'. likewise, if they were asked about whether they should pass this information on to another person, the answer would be 'no'. further, they would not appreciate a colleague searching through their own reports, notes and diaries. this is why locked filing cabinets and paper-based record areas are subject to stringent security. unfortunately, computer files don't seem to be viewed in the same way. ultimately, this is all that computer information systems are: the distillation of every police officer's confidential reports, notes and diaries. only the ease of access has changed. it is possible that this is due to a mindset which views paper as concrete and real and therefore valuable, while electronic data is viewed as abstract and virtual or 'unreal' and therefore worth less. some might say that if the information is taken away, the problem will disappear. this is very short-sighted. the information has always been there, it's just that it was difficult to access in the past. officers need to consider whether their need for information is great enough to justify a phone call to the colleague who wrote the original report before pushing the computer's key. they need to realise that every time they access a computer database they are making a conscious decision which might ultimately be tested by a court of law. this is a point recently reinforced in a police service notice (nswps, 1999, p. 21). when is access and disclosure ok? officers should feel free to access computer data whenever they need to in the course of their duties. all that is required is a simple entry in their notebook or existing file explaining the access, the reason and the result. if a police officer is asked by a third party, even another police officer, to access information on their behalf, the officer concerned should consider the matter carefully and either politely refuse or, if sure of the appropriateness, carry out the access and record it along with the result, the name of the person requesting the access and the reason for it. accesses should never be carried out for a third party if their identity has not been confirmed. if the access relates to a person whom the officer knows, including friends or relatives, there is always a possibility that their involvement could be viewed as a conflict of interest. in this case it is best for them to discuss the matter with a supervisor and request that another officer take over the matter. since access offences are complete upon access being achieved, any doubts about an inquiry should be referred to a supervisor before the information is accessed. summary there is little doubt that police services will continue to use technology as it becomes available, including a move to interview people over the internet (kearney, 2000). it should be remembered that human errors can still occur as darren carver can attest (o'shea, 2000). darren was falsely charged with two offences after his fingerprints, supplied for elimination when he reported an offence himself, were incorrectly linked to two house breakings. he was only released after successfully convincing police to double-check the fingerprints, something which should, perhaps, have been routine. often, one needs to step back and look at the 'big picture' when examining an issue. computer access and the technology it involves has the capacity to confuse and distort our view of the world when really only the medium for storing and accessing the information has changed. ethical access to, and use of, information is a constant in an ever changing world. the same issues facing society today were considered when people first learnt to store and spread information, initially using cave walls and rock carvings. the move from an oral tradition to writing and ultimately to the invention of the printing press presented the same concerns to professions about improving 44 ajis vol. 8 no. 2 may 2001 people's access to information that law enforcement agencies face today. one needs to have an increased respect for the right to access information in line with the streamlined access to the information rather than simply restricting the terms of that access. police work today, particularly intelligence driven policing, requires extensive access to information but the community is still entitled to an assurance that this information will be used in strict accordance with the legislation after all the community provides most of that information in the first place. 45 ajis vol. 8 no. 2 may 2001 annexure 1 offences relating to computers ( crimes act 1990 (nsw)) 309(1) a person who, without authority or lawful excuse, intentionally obtains access to a program or data stored in a computer is liable, on conviction before two justices, to imprisonment for 6 months or to a fine of 50 penalty units, or both. (2) a person who, with intent (a) to defraud any person; or (b) to obtain for himself or herself or another person any financial advantage of any kind; or (c) to cause loss or injury to any person, obtains access to a program or data stored in a computer is liable to imprisonment for 2 years, or to a fine of 500 penalty units, or both. (3) a person who, without authority or lawful excuse, intentionally obtains access to a program or data stored in a computer, being a program or data that the person knows or ought reasonably to know relates to: (a) confidential government information in relation to security, defence or intergovernmental relations; or... (c) the enforcement or administration of the criminal law; or... (e) the personal affairs of any person (whether living or deceased); or... is liable to imprisonment for 2 years, or to a fine of 500 penalty units, or both. (4) a person who: (a) without authority or lawful excuse, has intentionally obtained access to a program or data stored in a computer; and (b) after examining part of that program or data, knows or ought reasonably to know that the part of the program or data examined relates wholly or partly to any of the matters referred to in subsection 3; and (c) continues to examine that program or data, is liable to imprisonment for 2 years, or to a fine of 500 penalty units, or both. 46 ajis vol. 8 no. 2 may 2001 annexure 2 computer-related complaints about police year 1992-93 1993-94 1994-95 1995-96 1996-97 1997-98 1998-99 number of complaints 186 173 217 255 (ac) 608 (370 ac) (238 id) 5 16 (260 ac) (256 id) 495 (264 ac) (231 id) ac access complaint id inappropriate disclosure of information (source nsw ombudsman's annual reports) 47 ajis vol. 8 no. 2 may 2001 annexure3 statement of responsibility nsw police service information and information systems all employees of the nsw police service are to abide by the service's "code of best practice and guidelines for management of information and information systems" and are accountable to the commissioner to: (a) comply with all legal prescriptions regarding the handling of information; (b) protect information stored in both documentary and computer systems; (c) treat all information coming to their attention as strictly confidential; (d) not communicate in any way information regarding police business without proper authority; (e) report any breaches of security to their commander or the commander, office of professional responsibility. all employees of the nsw police service are responsible for the security of information on the service's computer system. users must understand and comply with the following protection requirements: (1) access [to] computers and computer systems is authorised for the performance of duties only. information is not to be put to any personal use. (2) access to sensitive information is logged. any users attempting unauthorised access will be called to account by their commanders and the assistant commissioner, professional responsibility. (3) unauthorised access, including computer games, must not be loaded onto police service microcomputer terminals or personal computers. (4) passwords must be memorised and kept confidential. do not display them on or near a terminal. never disclose passwords even to a supervisor or fellow worker who may seem to have a reasonable request. (5) users are required to change their password once a month. choose a password which is easy to remember but not easily guessed. passwords must be at least six characters in length. (6) users must not leave a logged on terminal unattended. this will prevent another person from accessing information which would in turn be logged against you as a user. (7) users will be held accountable for activities which take place under their passwords. if you suspect another person has knowledge of your password you must change it. report any suspected violations to commanders. (8) appropriate sanctions under the police service act, public sector management act and crimes act shall be applied for misuse and breaches of security. i have read the statement of responsibility and as a user understand my obligations. name/number/command signature date 48 aj1s vol. 8 no. 2 may 2001 references anon., (2000a, february 12), trapped behind the new digital divide, sydney morning herald, 101 & 105. anon., (2000b, august, 9), net porn staff caught the daily telegraph, 9. brammer, m. (2000) new south wales police service response to matters of interest cjc inquiry into release of confidential information, sydney: ns wps internal affairs. delattre, e. (1995) keynote address. in nswps (1995) ethics/professional standards seminar 24, 25 july 1995, ns wps: sydney. geddie, g (1999) email, the police, and the canadian charter of rights and freedoms: retooling our understanding of a reasonable expectation [of] privacy in the cyberage. 14th british and irish legal technology association annual conference, york. harris, sarah (2000a, may 21) women sacked over e-mails get $150,000 payout, the daily telegraph, 23. (2000b, june 11) goodbye to privacy, the sunday telegraph, 113-5. icac (1992) report on unauthorised release of government information, icac: sydney. jordan, j. (1999) hierarchies: 0 networks 1, management today, june, 10-11. kearney, simon (2000, july 23) police keen to interview on internet, the daily telegraph, 2. keogh, kylie (1999, july 1) police told to log out, daily telegraph. miranda, charles (2000, july 26) police on e-mail alert, the daily telegraph, 23. nsw ombudsman (1988) nsw ombudsman 1987-88 annual report, nsw ombudsman: sydney. nsw ombudsman (1993) nsw ombudsman 1992-93 annual report, nsw ombudsman: sydney. nsw ombudsman (1994a) nsw ombudsman 1993-94 annual report, nsw ombudsman: sydney. nsw ombudsman (1994b) improper access to and use of confidential information by police, nsw ombudsman: sydney. nsw ombudsman (1995a) nsw ombudsman 1994-95 annual report, nsw ombudsman: sydney. nsw ombudsman (1995b) confidential information and police, nsw ombudsman: sydney. nsw ombudsman (1996) nsw ombudsman 1995-96 annual report, nsw ombudsman: sydney. nsw ombudsman (1997) nsw ombudsman 1996-97 annual report, nsw ombudsman: sydney. nsw ombudsman (1998) nsw ombudsman 1997-98 annual report, nsw ombudsman: sydney. nsw ombudsman (1999) nsw ombudsman 1998-99 annual report, nsw ombudsman: sydney. nsw ombudsman (2000) submission to the criminal justice commission (qld) on the misuse of the queensland police service computer system and the unauthorised release of confidential police information, nsw ombudsman: sydney. nswps (1999, june 7) accessing cops, police service weekly, 11 (22), 21. omond, j. (2000) do not disturb, management today, august, 26-7. o'shea, frances (2000, august 2) the fingerprint that nearly ruined darren carver, the daily telegraph, 9. walsh, philippa (2000, july 18) cheating the boss, the daily telegraph, 1 1 . 49 ajis special edition-electronic commerce november 1998 planning and designing web-based electronic commerce: a case study in the insurance industry rolf alexander teubner, stefan klein institute for information systems, university of muenster, muenster, germany wialte@wi.uni-muenster.de abstract electronic commerce (ec) comprises new ways of doing business with customers, suppliers and other business partners. ec is enabled by information and communication technology, in particular the world wide we (in short, the web). while numerous organisations have started to use the web as an interface to clients and business partners, many have been disappointed by lower than expected response rates and higher than expected costs. one reason for non profitable web based ec applications is that they do not reflect the strategic preconditions of ec. this lack may be due to deficits in the process of developing an ec-strategy for their web application. furthermore, there is a void of instruments and tools to support this process. in order to fill this void, this paper proposes two basic frameworks for planning a web based ec application. the application of these frameworks is illustrated by the example of a medium seized insurance company. based on experiences drawn from the case-study, the proposed frameworks will be evaluated. introduction scarcely a platform has been tackled as intensively as the world wide web in discussions about the value of new media. euphoric statements have motivated companies to explore this medium for the purpose electronic commerce. because getting onto the web is not expensive, many of these companies have decided to offer a simple presentation but results are often not satisfactory. one reason for non profitable web based ec applications (shorter: web applications) is that they do not reflect a clear strategic rationale and that their position in relation to other media and functional areas is not clear. on the one hand the potential of the web as an instrument for marketing and sales is not exploited, on the other hand organisational and technical conditions of the enterprise are not taken into account. it is the intention of this paper to point out the most relevant aspects that have to be considered and provide a guideline for the process of planning a web site and designing the corresponding web application. fundamental frameworks for planning web-based ec on an operational level four basic parameters have to be considered when designing a web application. these parameters are the content, features, layout and the technologies used to implement the application. content: the content of a web application comprises the information presented to the visitor of a web site. the web offers hypermedia representation of information with few restrictions in the amount of content that is provided. these qualities are reflected in markedly different profiles for web advertising in contrast to traditional advertisements, which would gain little or no attention on the web. features: web applications in most cases offer more than store front representation of information. often applications offer opportunities to communicate by e-mail, to send information in predefined forms, to do some database query or transaction processing. all these functions besides information representation are covered by the parameter " features ". layout: the layout defines the interface which gives the user access to information content and functionality. layout design determines how user-friendly an application is and therefore has an impact on the level of user acceptance. the web as a new and emerging medium provides a plethora of design opportunities that are only gradually understood. experts in previous media, such as print, often show difficulties to capture the richness and specific features of this new medium and to develop visually attractive and operationally efficient solutions. technology: while the design parameters above define the application conceptually, all (software-) technical choices are comprised by "technology". first, the software architecture has to be defined. a typical architecture consists of a database server, a web server and dedicated software for electronic commerce. dedicated ec functions such as electronic catalogues, shopping baskets, customer profile management, payment and statistics are integrated in software packages called commerce or merchant servers. based on the architectural decisions, the technologies for implementation can be chosen. typical technologies for web applications are html and vrml for information presentation, java, java script, and cgi for computing and database access, and dedicated protocols for application and secure transaction services. 86 ajis special edition electronic commerce november 1998 although there are some general guidelines for web application design, not only the conceptual design of the application determines its success. rather the way the application is strategically positioned, organisationally anchored and technically integrated has an even stronger impact on success. these factors can be seen as context variables that have to be taken into account when planning the application. the model in figure 1 tries to point out the relationship between these context factors influencing web design and the design parameters (adapted from klein 1997). figure 1: factors influencing web design the model in figure 1 can be seen as a contingency model. it suggests interdependencies between context and design that have to be considered when planning a web strategy, but it does not imply deterministic relationships between context variables and the design parameters. the latter are enclosed in the inner ellipse in figure 1 while the former are outside this grey area. the context variables can be distinguished in external context and internal context variables. the external variables are not inside the borders of the enterprise (indicated by the large circle), while the internal variables are. the external variables are competition, target group and medium. competition does not only cover competing enterprises but also their web approaches. because web applications are highly visible and can be compared easily, the competition of imitation has intensified. it is a main challenge for an enterprise to find a web approach that differs from those of other companies, supports the strategic profile of the enterprise and facilitates a sustainable advantage. target group: a fundamental distinction among target groups is reflected in the terms business-to-consumer electronic commerce (b2c-ec) and business-to-business electronic commerce (b2b-ec). while the first term refers to the processes carried out between enterprise and consumer, the second one refers to transactions with suppliers and other institutional business partners. it is obvious that conditions and requirements are different in both cases. for example, b2b-relationsships are in general less anonymous and more stable than b2crelationships, integration with client information systems is possible etc. beyond the crucial decision whether existing or new business partners or customers shall be addressed, the known segmentation approaches from marketing can be applied accordingly. medium: in general, each web application is restricted or enabled by the characteristics of the medium "web". the web technology determines what is technically feasible. new technologies such as java applets, script languages and dynamic generation of html have expanded the possible range of features of web applications. the new transport protocol version, tcp/ipv6, is going to resolve some of the shortcomings of the current protocol, namely facilitating bandwidth guarantees. in contrast to the external context, the inner context variables are in the decision scope of the enterprise. the strategy, the organisational structure, the information systems and the area of the application are results of management decisions. but as these decisions are made before web application development, they are preconditions and have to be taken into consideration when developing a web strategy. 87 ajis special edition electronic commerce november 1998 strategy: the web strategy is part of the information strategy of the enterprise, which itself is closely linked to its business strategy. the information strategy is a complex set of explicit visions, goals and guidelines with respect to the demand and supply of information necessary to conduct the business. the web strategy is that part of the information strategy which refers to the usage of the web in delivering and accessing information in order to leverage business opportunities (lederer et al. 1997). organisational structure / information systems: as far as a web application exceeds store front presentation of information and offers dynamical information content, communication or transaction functionality, there is a great need to integrate it organisationally and technically. from an organisational point of view, the application has to be integrated into business processes, in case of a customer interface, e.g., order fulfilment or invoice management. furthermore, rules for maintenance and updating have to be established. from a technical point of view, interfaces for access to internal databases, to communication systems or to workflow management systems have to be defined. areas of application: the areas of application define the purpose and scope of the application. examples are advertising, providing value added services, delivery of (digital) products or simplifying administration tasks. though the framework in figure 1 is descriptive in its nature, it summarises and structures success factors for a web application. those success factors can be transformed into a development cycle model of web-applications as depicted in figure 2 (see klein 1998). • external: target groups, competition, technology plan * 'n(erna': application domain, (marketing) strategy, organisation, information systems • content, features, design design (layout) * technical design integrate organisational integration technical integration evaluate/ monitor • acceptance • benchmarks • new players/target groups • technological development figure 2: web engineering development cycle model this paper gives a detailed description of a concrete application of the proposed development cycle model (figure 2) and the underlying framework (figure 1). for this reason, the model will only be outlined at this point: • the planning phase covers the analysis of the external situation, the strategic positioning and the marking out of application areas. the outcome of the planning phase is a concrete definition of what shall be achieved with the web site. this definition can be expressed in terms of a web strategy, that commits the objectives of the application as well as its scope. • the design phase covers all decisions that aim at defining how the aims planned shall be accomplished. these decisions refer to the design parameters content, layout, features and technologies. the result of the design phase is a conceptual and technical specification of the web application. • the implementation of the application is done in the integration phase. but the challenge of this phase is less to derive code from the design specification but to embed the application into the enterprise infrastructure. besides coding, the technical and organisational interfaces to the internal infrastructure have to be built. in numerous cases, the adaptation of the infrastructures themselves will be necessary, too. 88 ajis special edition electronic commerce november 1998 • due to the highly dynamical development of the web and its usage, web applications require continuous improvement efforts as soon as they are established. this is done during the evaluation phase. observing competitors' activities, customer reactions and technology development aims at identifying the need for actualisation, adaptation, or enlargement of the web site. the regular analysis of competitors' web sites usually gives a good, albeit not sufficient, insight into competitors actions. developing a web-application: the case of an insurance company the development project that is described here was conducted in co-operation between a project group , consisting of seven students and one research assistant from the university of miinster, institute of information systems, and an insurance company called landwirtschaftlicher versicherungsverein minister (lvm). the lvm is a medium-seized company, well established in the german insurance market. it employs about 2000 employees, and has about 2.8 million customers. the annual revenue is dem 3.4 billion. the project started in the end of 1997. at this time, lvm had already a web presence. the design and concept, however, had mainly been done by an external advertising company that was also in charge of the maintenance of the application. these first web activities were mainly motivated by the interest for the new medium "world wide web" and the concern not to be left behind by technology leaders. this first initiative reflected more a "me-too"-approach than a convincing strategic rationale. as a consequence, success fell far from expectations. besides the problem, that lvm did not pursue a convincing web-strategy, there were problems in making full use of the web. the service provider for the lvm was experienced in the use of traditional marketing media but had little experience with the web and were not fully aware of the profound differences between traditional media and the web (kassaye 1997). the most striking difference is, that the web is an interactive medium in contrast to print media, broadcasting or television that enable only one-way distribution of information. other differences concern the integration of the web strategy into the overall marketing and communication strategy. while, for example, a corporate design is a necessity from a marketing communications point of view, it has been discussed controversially, whether the web presence, which often pursues a slightly different strategy than the traditional media, should follow the same principles and regulations. as a response to these problems, the lvm decided to develop a new, more convincing web strategy. furthermore, lvm put the provision of the web services into the hands of another provider. the new web strategy was developed in co-operation with the university of muenster. the common project was conducted according to the process model shown in figure 2. planning phase during the planning phase, the external context, i. e. the industry, was examined. business strategies, web usage in the industry and web strategies of relevant competitors were taken into account, when positioning the lvm's web approach. then, the areas of application were defined with respect to lvm's business, customers and infrastructures. external context analysis customers and competitors: industry situation and trends the insurance industry is just going through a process of fundamental change. a first trend is market saturation: while in the fifties market growth was nearly 15% a year, it was only just 3 % in 1996 (hackenberg; reiter 1997). with diminishing demand-pull, the traditional markets have become more and more customer dominated. in the past, strong relationships bound customers to their insurance companies. but customers have become more sensitive to price and quality. many of them do not mind to change their insurer for better products or conditions any longer. a second trend is fierce competition. this trend is partly stimulated from market saturation and customer power. the insurers natural reaction to sinking profits in a shrinking market is increased competition within the industry. another important factors stimulating competition is, that suppliers from other lines of business enter the insurance market. first, there are banks and companies offering financial services. these new players intend to offer customers full service packages including insurance and financial services, a strategy called "allfinanz". furthermore there is new competition by large independent insurance agents that reverse integrate their business. other new competitors are, for example, car manufacturers that offer car insurance services together with their products. globalisation, the third major trend, is in a pull and push relation to competition. globalisation is mainly enabled by the deregulation of the local insurance markets and the establishment of a common european market with a common currency, the euro. through globalisation, some great foreign insurance companies try to establish business on the german market, which is an attractive market with a total volume of dem 250 billions 89 ajis special edition-electronic commerce november 1998 (hackenberg, reiter 1997). on the other hand, globalisation gives large german companies access to foreign markets. a great potential is seen especially in emerging markets in eastern europe, asia and latin america. while international markets are only a potential for the large companies, there are also some opportunities for smaller companies that only have access to the national market. new potential is expected from new risks that need to be insured such as caused by sinking trust in pension insurance or risks of software. these new opportunities will be exploited by those companies, that quickly and flexibly respond to the emerging needs and offer innovative products fitting them. the necessary reactions to the trends stated above that can be seen as success factors in the actual situation of the insurance industry, are: • a shift from product orientation to customer orientation, • market segmentation (esp. household vs. industry), • differentiation of distribution channels according to market segments, • development of new, individualised products, • bundling and tailoring products, • more emphasis on financial services and financial market functions, • drawing to foreign markets, • cost reduction. the web can help address most of these success factors. information can be tailored to customers needs, if necessary even a segment-of-one-marketing is possible. the web is a new distribution channel. insurance companies with direct sales activities, for example, could use the web as a distribution channel by supporting all phases of ec from contacting to contracting. furthermore, multimedia presentation offers rich opportunities to describe complex products. web applications can offer powerful software that helps to tailor products or product bundles (customer decision support). next, the web is a global medium, that enables insurers to enter new markets all over the world in a simple but efficient way. and finally, as every information system, web systems can reduce costs, too, for example by facilitating administrative processes. web strategy is introduced. before these potentials will be discussed with respect to the lvm's situation, the current web usage in the german insurance industry shall be investigated. state of web usage in the industry in a web survey conducted during the project at the end of 1997, about 75 web sites of german insurance companies were identified. the comparison with the numbers of 1996, when 50 companies were present on the web (kortzfleisch, winand 1997), shows, that web presence in the insurance industry is increasing rapidly. the 75 web sites identified have been investigated for their information content and features. the results can be summed up like this: • most companies use the web as electronic store front. all companies offer enterprise presentations, a lot of companies offer product information in some way. but only few companies give in depth information on products, including examples of risks, legal obligations and so on. in most cases, the web sites appear to be nothing else but an "electronic pamphlet". • only a few companies make use of the interactive features of the web. most companies offer simple email services, of course. but response to e-mails is typically paper based. this media break underscores that the web is not fully integrated into information and organisational infrastructure. direct transaction processing and dynamic generation of information are exceptions. • less than three companies allow electronic contracting. "less than three" means that even though electronic contracting was offered, the web form was just a trigger to the traditional process. the contracting itself was paper based in part or at least accompanied by paperwork. another striking result of the web survey was no perceivable distinction of web sites according to the business strategies of the companies. for example, the great players in the market would have been expected to use the web as an interface to international markets. consequently, web sites should have been multilingual at least. or insurers with major direct sales activities would have been expected to use the web as a medium for market transactions. web sites should then offer extensive customer information, decision support and electronic contracting. but none of these expectations was confirmed convincingly by the web survey. the findings can be summed up by the hypothesis that most german insurance companies had not defined or at least implemented a web strategy clearly supporting their business strategy at that time. 90 aj1s special edition electronic commerce november 1998 web strategy development the success of a web site is not primarily determined by technical or layout qualities. because of the high visibility and comparability of a web site, technical and layout qualities can be adapted very soon by competitors. thus, a sustainable competitive advantage can hardly be achieved through those qualities. in contrast, a decisive success factor of a web presence is seen in the link between business and marketing strategy on the one hand and web strategy on the other (lederer et al. 1997). a web strategy embedded in and aligned with an overall marketing strategy can hardly be imitated on the short run. the link between marketing strategy and web strategy lvm is a medium seized player in the german insurance market. the company is well established in numerous regions and covers various market segments. the company has a good reputation, especially in car insurance. lvm is the third supplier on the german market in terms of volume. nevertheless, the enterprise does not have the size and capacity to compete successfully on foreign markets. because of the fierce competition in the national market, lvm intends to bind their customers closer to the company through a customer focused strategy. lvm is neither willing to compete solely on price (cost leadership strategy) nor on very specific market segments (niche strategy). the customer focused strategy instead combines attractive prices, a full set of services all around the customer's needs and intensive customer care in a broader set of market segments. the customer focused strategy carried out perpetuates a central strengths of lvm. the lvm has at its disposal a large network of (legally) autonomous sales agents distributed over most parts of germany. those agents are called "vertrauensleute", trusted agents, indicating that there is a strong personal relationship between agent and customer. through this network of agents customers are addressed and looked after personally. agents are able to give help immediately without much bureaucracy if needed. to expand customer orientation to a full customer focused strategy, requires customer specific products or product bundles that can be tailored to the customers' needs, increased personal customer care, empowering agents, keep costs and prices low. direct implications of the customer focused strategy on the web strategy are: • the web site must increase service and provide a clear value for the customer, • it should enhance customer care, • it should make use of the new sales channel "web" without excluding the agents, • this means that the web site must not offer electronic contracting in order not to corrupt the existing sales structure, • instead it should empower the agents. these implications guided the process of planning the objectives, purpose and scope of the web site. objectives of the web site after having identified a list of potential benefits of the web usage, the following objectives for the lvm web site were defined: • improve reputation through technological leadership, • improve advertising, • reduce response and delivery times, • enhance services, esp. access in place and time, response speed, value added services, • intensify customer relationships and loyalty, • facilitate co-operation with sales agencies, • improve individuality, • reduce transaction costs by removing or simplifying routine administration work, • give access to (new) markets. some of the objectives are very general like those aiming at image, advertising and transaction costs. advertising was identified as the most important objective. the web should offer a comprehensive presentation of the complex product "insurance" through it's multimedia qualities and "infotainment" to attract customers. other objectives, such as those concerning response and delivery times, service, customer loyalty, individuality and co-operation with sales agents were directly derived from the customer focused strategy. these objectives are not independent, since the provision of individualised information and of services with a clear value proposition (kambil et al. 1996) bind the customers to the enterprise and thus improve customer loyalty. 91 ajis special edition electronic commerce november 1998 there is still another objective that has to be explained: "access to markets" denotes the idea that the web enables sales agents to improve their care for a regionally dispersed group customers. this is especially important in regions with only a few sales agents, so that (remote) customers can only seldom be looked after personally. scope of the web site the following list of information contents and functions defines the scope of the web site: • introduce the enterprise, • present products with detailed information on all kinds of risks/ insurance policies, • entertain, • distribute individualised customer information, • give customer access to his insurance data, • respond to online customer requests, • deliver forms, offers, and policies, • accept claims statements, • support direct communication between sales agent and customer, • exchange information between sales agent and head office. one important underlying idea was to improve services by providing customer specific information. since information resources are collected and stored at the head office, and the knowledge about customers, their situation, and their needs is located at the agencies, intense co-operation is required. this co-operation should be facilitated by the web application again. interfaces to organisation and information infrastructure integration issues have not been investigated so far, since the proposed web site is still available only as a prototype. but the main challenge of integration is obvious: the web site has to be integrated with databases and processes at the head office as well as with the sales agent's infrastructure. web application design during web application design the parameters content, features, layout, and technology have to be defined aligned with the web strategy. table 2 gives a summary of possible implementations of the conceptual design parameters (also see kreileder, abmann 1997). design parameter examples content enterprise information, product descriptions and price, background information, policy data, claims examples. graphical design graphical presentation, user interface (orientation, navigation, use), animation. features customer inquiry / problems management, pamphlet and offer delivery, product calculation and configuration, electronic contracting, claims declaration, changes in personal and policy data, individualised counselling (customising, filtering), entertainment. table 2: design parameters of web sites 92 ajis special edition • electronic commerce november 1998 the lvm web application incorporates all of the examples above to a certain extent. most important are those parameter choices, that directly support the customer focused strategy. content the lvm application offers static and dynamic information. information on the enterprise and it's products is only static. the user can browse the information according to his preferences. choices to pre-select a view to the information are not offered. other information is customised to the user. it is generated dynamically out of a data base according to the user profile. such information are the customer's personal data, contracts and policies. other individualised information is actively (push) distributed to the customers, for example, if the customer has an insurance policy for a car manufactured by bmw, information about this car, technical problems, test reports, etc. dynamic information is only offered to registered users, i.e. the owners of policies. features the lvm application offers a lot of standard facilities such as delivery of pamphlets and, the calculation of products, counselling, and entertainment. counselling and entertainment is close related in the so called "sorglos-test" ("care-free-test"), which considers the customer's personal risk situation and a corresponding bundle of policies optimised in price and service. another entertainment features to attract new customers is a quiz. the questions asked by the system refer to the information on the web site, so that users taking part in the quiz have to investigate the site in detail. probably the most important feature is only offered to registered customers. for them, a direct communication channel to their sales agent is established. this channel includes mailing facilities as well as predefined forms that trigger standard business incidents, such as claims declaration and policy changes. the forms comprise all the data necessary to fulfil the requested transaction. the sales agent only has to trigger the procedure, that is executed at the head office. the agent is then continually informed about status and progress of the procedure. he is the interface and the person to turn to for the customer. only changes of personal data, the customer's address for example, can be made without explicit involvement of the sales agent. layout with respect to the layout, there is a distinction between features and content that are public, and those customised to specific users/customers that are pre-registered. the public parts of the application are designed according to more general layout guidelines for a professional and innovative presentation. there is no strict corporate design but a link to it, as the basic enterprise colours and the central logo are used. more interesting than the public areas of the application are the individualised content and features offered to the registered customers. the entire information contents and all features are organised around a central metaphor, the "virtual desktop". the desktop metaphor implies, that the customer is offered a full working environment supporting him in arranging his insurance affairs. the function of this metaphor is on the one hand to improve orientation and on the other hand to facilitate navigation and access to the application features. figure 3 shows a sketch of the graphical representation of the "virtual desktop". when the customers opens his desktop, a photo of his sales agent is displayed in the sales agent-area. he can access all information about his agent by a mouse-click onto this photo. when clicking onto the mail buttons a list of incoming or outgoing mail, respectively, is displayed like in a typical mail system. the folders at the left bottom of the desktop are generated according to the customers policies. each folder contains all information about a current policy. if a policy is opened, the work aids area offers tools and forms to treat a policy. for example, a claims declaration form can be used and sent to the sale agent (logically) and the head office (physically), respectively. the pinboard is a kind of individualised bulletin board, that contains information of general relevance to the customer. all bits of information are represented like notes pinned to a wall. they are displayed in the display area on a mouse-click. figure 5 shows the "virtual desktop" layout used for the prototype. displayed is the pinboard with (links to) bit of information for a customer assumed to be the owner of a bmw car. 93 ajis special edition electronic commerce november 1998 display area (workplace) news mail-lists forms policies pinboard work aids area figure 3: concept of the virtual desktop technology the technology for the application has not yet been decided, since lvm is just changing their technical infrastructure. the web application shall be integrated into the new lvm agency system, that will be based upon an object-oriented architecture. but as the new infrastructure is still in development, the technical parameters of the web application have not been decided definitely. the prototype that has been built to prove the concept is based on a microsoft platform. the microsoft information server has been used as a web server and microsoft access as the database management system. for prototyping purposes these technologies have proved to be valuable. the active server pages offered by the microsoft web server and easy integration of web and database server, facilitated development. nevertheless, this technological configuration does not have strong implications for the running system. for example, microsoft access does not appear to be the right data base management system to handle the transaction volumes for only a part of the 6.5 billion policies of the 2.8 million customer. figure 2 describes the development cycle of web applications. the preceding description of the planning and design phase refers to the joint development of a prototype which is not yet fully implemented. the prototype actually represents a redesign of the initial web site. the following description of the integration and evaluation phases refer therefore to the initial design. integration development and maintenance of the initial web presence had been outsourced to a service provider. the web application ran fully independent of lvm's it infrastructure. while this was advantageous from a security point of view, it not only restricted the functionality of the web application to a static information site, it also made any change of the content a cumbersome process. change requests had to be send to the service provider and checked afterwards. the organisational and technical separation between service provider and lvm was quickly perceived as a major disadvantage and totally inadequate for a dynamic web presence. the redesigned application will be controlled and maintained internally, although parts of the development may again be outsourced. a linkage to the operational database management systems is envisioned but not yet realised. evaluation and monitoring the critical assessment of the initial web presence and the mentioned analysis of the state of the web usage in the industry, both give evidence to evaluation and monitoring activities which were the basis for the redesign. 94 ajis special edition electronic commerce november 1998 .-dei virltiefle 5chi«4riiich netscape !£fc £* vim . bock sragd. bdnad. home saareh gufa pihsoortv aktnedei nm bmw znm roadster aus der antobfld bmw-sondertarife vomada neqestc infimnationen direlct von bmw dvr road tter in "golden eve was ist die pinwand? die pinwand at fitt sic tmc msgjiclikeit, die btr sie vom ver&raucnsi&mi zur vexfilgunfi mstcotcfi tnfnrmnttiiiiftii und bite acfaten sie daranc dafl sie durcb wtteres venweigen in den doft zur vejfikffuns gestelhen seiten uoter umstftndeo biren persdnhchen schreibtuch veriassen und thn nur *̂ * der benotsung des "back-buttons' des browsers wilder ermchtn kanoea figure 4: layout of the virtual desktop conclusions the analysis of the lvm case underscores the assumption that a successful web presence requires much more than a me-too strategy or a pure informational web site. the example shows how properties of the web as a new medium and the strategy can be aligned in order to develop specific value propositions for respective target groups. these value propositions clearly go beyond the conventional wisdom of product fit for the web: the functional areas that have been selected are neither easy to describe products nor direct sales activities. they reflect, however, the company's strategy and build on functional properties of the technology. if these value propositions are related to the core competencies, it might even be possible to achieve sustainable competitive advantages in a medium which facilitates imitation more than any medium before. however careful and innovative a solution might be, given the dynamics and volatility of the medium web, a continuous process of monitoring and learning has to be initiated in order to gain and/ or keep a competitive edge. references kassaye, w. (1997) "global advertising and the world wide web", business horizons, vol. 40, no. 3, pp 33-42. klein, s. (1998) "konzepte und vorgehensmodelle fur die web-evaluation", in: korte, werner b.; reinhard, ulrike (eds.): who is who in electronic commerce, whois verlagsund vertriebsgesellschaft, heidelberg, pp 200-209. kambil, a.; ginsberg, a. & bloch, m. (1996) "reinventing value propositions", working paper is-96-21, new york university, new york. kortzfeisch, h. f. o. & winand, u. (1997) "tele-insuring: der beitrag neuer kommunikationsund infromationstechniken (kit) zur verbesserung der kundenorientierung im versicherungsbereich", die betriebswirtschaft, vol. 57, no. 3, pp 338-356. 95 ajis special edition electronic commerce november 1998 kreileder, m.; abmann, k. (1997) "insurance online: die meisten kamen aus neugier". in: versicherungswirtschaft, vol. 16. lederer, a. l.; mirchandani, d. a. & sims, k. (1997) "the link between information strategy and electronic commerce", journal of organisational computing and electronic commerce, vol. 7, no. 1, pp 17-34. schoder, d.; straub, r. e. & welchering, p. (1998) "electronic commerce enquete 97/98 executive research report: empirische untersuchung zum betriebswirtschaftlichen nutzen von electronic commerce fur unternehmen im deutschsprachigen raum". 96 scan2022-03-31_164632 390-article text-rescan ajis vol. 6 no. 2 may 1999 business students' perceptions of electronic commerce will they join the revolution? dieter fink school of management information systems faculty of business edith cowan university pearson street churchlandswa60i8 australia fax 61-8-92738754, telephone 61-8-92738726 email d.fink@cowan.edu.au abstract the dramatic impact of electronic commerce (ec) is attracting much attention, most of which is directed towards emerging internet technology. however, it is generally acknowledged that it will take a few more years before ec becomes mainstream. the perceptions of business students, the future business leaders, on the promises and perils of ec are therefore vital. our research indicated that they believe that the capabilities of the internet for marketing and distribution are achieved to a greater extent than those for accounting and finance. the use of digital money appears to be still unrealistic in their opinion. their main concern is for the insecurity of ec in such areas as computer viruses, theft of information, and unauthorised access to data by intruders. as a consequence, the trust the students currently have in ec is relatively low. however, they are optimistic about future prospects of ec, which they perceive will significantly improve over the next few years. introduction the internet has become an important medium for organisations desiring to interact with a wide range of stakeholders. it has the potential to market products and services to prospective customers, to communicate information to a global community, to provide an electronic forum for communications, and to process business transactions such as orders and payments (nash et al., 1998). these activities have come to be known collectively as internet or electronic commerce (ec) the process of conducting commerce electronically over a variety of networks that constitute the internet. through the use of digitisation, business transactions can be transmitted virtually instantaneously around the world. this has significant implications for international trade and the management of information as we move towards 2000. in the words of buckeridge (1996, p 6), "the introduction of the new digital technologies elevates 'information' to its status as the predominant currency of postindustrial societies. this is eroding fixed notions of sovereign nation space, organisational structure, zones of production and marketing, and work processes." tapscott (1996) describes the 1990s as 'the age of networked intelligence'. he suggests that the decade be not simply about the networking of technology but about the networking of humans through technology. "it is not just an age of linking computers but of internetworking human ingenuity. it is an age of vast new promise and unimaginable opportunity." (p xv) however, he also expresses a warning. "for individuals, organizations, and societies that fall behind, punishment is swift." (p xv) research objective the effective exploitation of the internet for ec is dependent on our ability to effectively manage a range of factors. these appear to be well recognised in literature, and include information technology (it) and communications infrastructure, information security, application development tools, and ec management (society for information management, 1997); reliability and privacy (camp and sirbu, 1997); and network bandwidth and security (sim and rudkin, 1997). while much attention is currently being devoted to solving technological challenges, for example increasing the bandwidth on existing narrowband network platforms to overcome bottlenecks (buckeridge, 1996), no or little attention appears to be given to the human aspects of ec. this has been a mistake in the past as human resistance to, or incompetence during, the introduction of new it often caused information systems (is) to fail (sauer, 1993; beynon-davies, 1995). by focusing on the human elements early in the adoption of ec technology, we have the opportunity to avoid the mistakes made in the past. in this paper we examine the attitudes of business students to electronic commerce in order to establish their perceptions of the promises and perils of this new technology. the reason for choosing young persons was that it will take a few more years, "... certainly not before the year 2000before internet electronic commerce is in the mainstream of the global economy. and it will take at least another decade after that before it becomes a truly significant factor in terms of total commerce" stuck (1996, p 54). by then the current crop of business students 36 ajis vol. 6 no. 2 may 1999 will play a major role in the new ways of doing business. they will have been associated with ec from the ground level up. research variables and hypotheses in this research we examined the perceptions of business students on the promises and perils of ec. the variables comprised ec capabilities, in particular those of marketing/distribution and accounting/finance, and associated concerns and risks. they are discussed below together with our hypotheses. electronic commerce capabilities the use of the internet has the potential to cover most, if not all, phases of marketing and distribution and accounting and finance. for example, with the former, product discovery is facilitated through product information (e.g. catalogues) being dispersed on web pages. with the latter, funds can be transferred electronically to anywhere in the world. these activities are explored in more detail in the following sections. ec also has the capability of providing continuous service by offering access to information around the clock and globe in multiple languages. administration processes can be automated to a larger extent than before; for example customers can submit their orders online with no clerical intervention and receive payments by a direct transfer of funds to their bank account. at present, the literature deals mainly with marketing and distribution activities occurring on the internet. we sought to confirm this impression by framing our hypothesis as follows: hypothesis 1: in the use of the internet for electronic commerce, its marketing and distribution capabilities are achieved to a greater extent than its other capabilities. marketing and distribution the marketing and distribution capabilities of ec broadly fall into 2 major areas (stuck, 1996). first, ec enables buyers to enquire about products and services, place orders and receive goods and services online. second, ec enables sellers to advertise products and services, and deliver goods and services electronically and provide ongoing customer support. product discovery is facilitated through product information being dispersed on web pages and by targeting a special interest group, such as wine buffs. this has benefited smaller firms in particular because they can now provide their products to niche markets anywhere in the world, unconstrained by economies of scale. however, the way in which products can be delivered on the internet is determined by the degree to which the physical product can be transformed into a digital product. information products, including computer software, are the most obvious commodities for ec. the internet is not well suited for the distribution of those goods that typically require physical or tactical sampling, such as furniture and fresh food (woods, 1996). at present the communications capabilities of the internet appears to attract the most attention and we are witnessing an explosion in web sites. we, therefore, hypothesised as follows: hypothesis 2: in the use of the internet for marketing and distribution, its communications capabilities are achieved to a greater extent than its other capabilities. accounting and finance while banks and organisations have used electronic fund transfer (eft) to settle business transactions for many years, this has been done in a highly controlled manner. the internet has the potential to allow the free flow of funds between millions of internet subscribers. however, developments towards secure electronic cash (digital cash) have not reached a sufficient level of maturity (forcht and wex, 1996; liddy, 1996) and most internet payments continue to rely on the use of credit cards. the internet has provided the capability to substantially reduce accounting and financial transaction costs as illustrated by the example of electronic banking. the introduction of new technology has the potential to reduce the cost of banking significantly. to serve a customer over the counter is far more expensive than to serve him or her through automatic teller machines (atms) which in turn is more costly than conducting customer transactions electronically on the internet. at present, electronic payments systems are constraint because of the reliance on credit cards rather than the use of digital cash. we, therefore, framed the following hypothesis: 37 ajis vol. 6 no. 2 may 1999 hypothesis 3: in the use of the internet for accounting and finance, its data processing cost reductions capabilities are achieved to a greater extent than its other capabilities. concerns most literature on ec acknowledges that the promises of the technology are currently being constrained by their perils, be they real or perceived. as with the introduction of all new technologies, users have a variety of concerns that they need to weigh up when deciding whether or not to adopt internet technology (arnum, 1995). concerns relate to the apparent insecurity of the internet, potential disruption to business because of network failure, the lack of businesses partaking in ec, possibly caused by high setup and running costs, and slow performance (e.g. data transfer) due to inadequate bandwidth (nath et al., 1998). furthermore, there appears little overall control because universal standards and legislature have not developed sufficiently (pattison, 1997). it is generally acknowledged that security is holding back the full exploitation of the internet's capabilities. we, therefore, hypothesised as follows: hypothesis 4: in the use of the internet for electronic commerce, the concern for security is perceived to be greater than other concerns. risks essentially two types of risk factors can be identified for the internet, namely business risks and technology risks (matinez, 1996). the former relate to products and services ordered on the internet not being of the quality promised, not being delivered even though they have been paid for, or the buyer denying having placed the order. legal uncertainty provides further risk in relation to what constitutes an offer and acceptance on the internet, how do the copyright provisions work, and where does the jurisdiction exist when trade spans the globe (pattison, 1997). technological risks include computer viruses, introduced through data obtained from the internet, and hacker attacks penetrating computer system in an unauthorised manner for the purpose of causing damage (e.g. introducing a virus) and stealing valuable information. furthermore, all data and messages being forwarded to and from the internet are potentially subject to interception and modification while being transmitted (forcht and wex, 1996; liddy, 1996). adverse publicity relating to security breaches on the internet (e.g. the disclosure of credit card information the weekend australian, 23-24 november 1996), caused us to believe that users of the internet are mostly concerned with the security of information. hence, we framed the following hypothesis: hypothesis 5: in the use of the internet for electronic commerce, the risk of theft of valuable information is perceived to be greater than other risks. research conduct in our study, we used final-year university student as surrogates for young business persons. this approach has been used successfully in past studies, for example in advertising and marketing research (yavas, 1994) and technology evaluation (briggs et al., 1996). the students were completing the final year (1997) of a three-year bachelor of business degree in the faculty of business at edith cowan university (ecu) in perth, western australia. seventy-five students voluntarily completed a questionnaire that captured their opinions on various issues related to the current and future use of the internet for ec. the primary research question was: do business students perceive the internet as a suitable vehicle for conducting commerce? this was reflected in the preceding hypotheses that aimed to establish to what extent are ec capabilities being met and what are the concerns and risks associated with ec. when phrasing the variables we endeavoured to use terminology familiar to the participating students. for example, the capability of the internet to automate administration was illustrated by referencing the system allowing students to enroll themselves over the internet at ecu. the students belonged to 3 equal groupings of 25 each reflecting their major study, namely accounting, marketing, and information systems. the questionnaire was relatively short and captured the participants' opinions on a 7-item rating scale. the experiences of the participating students with the use of the internet are shown in table 1. 38 ajis vol. 6 no. 2 may 1999 table 1 :experience in using the internet ( "1 extremely low 7 extremely high") discipline accounting marketing information systems overall at this moment 3.75 3.60 4.84 4.06 in 1-2 years time 5.33 5.36 6.04 5.57 information systems students, as could be expected, indicated the highest levels of experiences, currently and in the near future. the experiences of those belonging to the other two disciplines were lower and close to each other. a definite upward trend over the next 1-2 years is apparent for all three disciplines. research findings the study began by establishing the perceptions of business students on the broad capabilities of ec; they are shown in table 2. table 2: broad capabilities of ec ( "1 not achieved 7 fully achieved") mean marketing and distribution (e.g. product information is provided on 4.72 the net) providing continuous service (e.g. information is provided around 4.66 the clock) finance and accounting (e.g. transferring funds electronically 3.91 around the world) automating administration (e.g. students enrolling themselves over 2.56 the net) as seen from table 2, business students supported our hypothesis that the internet currently achieves marketing and distribution activities to a greater extent than other activities. the t-test indicated that 'marketing and distribution' capabilities were, however, not significantly rated higher than 'providing continuous service' (t = .34, p>.05). they were rated significantly higher than 'finance and accounting' (t = 4.16) and 'automating administration' (t = -9.58) at p<.05. our hypothesis 1 can, therefore, be partially accepted. the study proceeded by requesting participants to rate the marketing and distribution capabilities of ec: they are shown in table 3. table 3: marketing and distribution capabilities of ec ( "1 not achieved 7 fully achieved") mean distribution of samples (e.g. enabling software to be downloaded 5.25 for a free trial) provision of online feedback from customers and prospects (e.g. via 4.97 email) as another communications media (e.g. like press, tv, radio) 4.81 to target specific interest groups (e.g. potential home buyers, wine 4.53 buffs) to sell well-known brand products (e.g. golf balls) 4.46 allows smaller firms access to niche markets (e.g. enabling product 4.45 discovery) as a shopfront for goods requiring physical sampling (e.g. furniture, 3.71 fresh foods) as seen from table 3, the capabilities of the internet as a 'communications media' was not rated the highest as predicted in the hypothesis. rather this capability rated third highest behind the capabilities of 'distribution of 39 ajis vol. 6 no. 2 may 1999 samples' and 'provision of online feedback from customers and prospects'. the t-test indicated that the rating difference with the highest rated item (t = -2.02) was statistically significant at p<.05 (this was not true with the second highest rated item (t = -1.02)). our hypothesis 2, therefore, had to be rejected. table 4 reflects the perceptions of the participants of how well accounting and finance capabilities are currently being met by ec. table 4: accounting and finance capabilities of ec ("1 not achieved 7 fully achieved") mean reducing data processing costs through greater automation 4.21 transferring funds electronically to anywhere in the world 4.09 being able to use 'digital money' 3.57 as predicted, 'data processing cost reduction' capabilities were rated the highest. however, the t-test revealed that this was only significant compared to the 'use of digital money (t = -3.84) but not compared with 'transferring funds electronically' (t = -.80) at p<.05. our hypothesis 3 can, therefore, be partially accepted. the major concerns associated with ec are shown in table 5. table 5: concerns with ec ( "1 extremely unconcerned 7 extremely concerned") mean security (e.g. potential for data interception) 5.68 legislature (e.g. lack of privacy protection) 5.13 confidence (e.g. lack of universal standards) 5.05 reliability (e.g. breakdown of the net) 4.89 performance (e.g. data transferred to slowly) 4.86 connections (e.g. lack of businesses connected) 4.59 setup and running costs 4.54 security concerns were rated highest as hypothesised. furthermore, t-tests revealed mean differences between this concern and all others to be significant at p<.05. our hypothesis 4 can, therefore, be accepted. table 6 provides the perceptions of specific ec risk factors. table 6: risks of ec ("1 no risk 7 extreme risk") mean computer viruses being introduced through data obtained from the 5.58 net theft of valuable information (e.g. credit card details) 5.43 unauthorised access to data by intruders such as hackers using the 5.20 net goods/services ordered on the net not being of the quality 4.93 promised uncertainty of using digital money (e.g. money 'disappears', is 4.93 'duplicated') interception and modification of data while being transmitted 4.91 inadequate legal provisions (e.g. concern over eligibility of 4.91 electronic medium as evidence) goods/services ordered and paid for on the net not being delivered 4.64 our prediction that the risk of 'theft of valuable information' would be rated highest was not realised; it followed the risk of 'computer viruses being introduced' in the ranking of risk factors. however, through t-tests it was established that the ratings of these two items were statistically not significantly different (t = -1.15) at p < .05. compared with other risk factors, 'theft of valuable information' was also not statistically significantly different 40 ajis vol. 6 no. 2 may 1999 from 'unauthorised access to data'; however it differed significantly from all other factors at p < .05. our hypothesis 5 can, therefore, be partially accepted. the students came from 3 backgrounds determined by the studies they were pursuing, namely accounting, marketing and information systems. according to the analysis of variance (anova) statistical procedure, few differences existed between the groups as to their perceptions and experiences of ec. those of statistical significance are reported in table 7 below. table 7: inter-disciplinary difference in perceptions of ec marketing & distribution: to provide online feedback between groups within groups total concerns: security between groups within groups total concerns: reliability between groups within groups total experience: at this moment in time between groups within groups total sum of squares 10.788 99.158 109.946 12.253 115.598 127.851 12.510 122.625 135.135 22.802 195.860 218.662 df 2 71 73 2 71 73 2 71 73 2 71 73 mean square 5.394 1.397 6.127 1.628 6.255 1.727 11.401 2.759 f sig. 3.862 .026 * 3.763 .028 * 3.622 .032 * 4.133 .020* *p<.05 **p<.001 disciplinary differences existed in respect of perceptions as to capability of the internet to provide online feedback, concern over the security and reliability of ec, and current internet experience levels. the findings are discussed further on in the paper. as indicated in the paper's introduction, it is expected that ec activities will evolve over the next few years. this prediction appears to be reflected in the students' perceptions of ec prospects as shown in table 8. table 8: prospects for ec ( "1 extremely poor 7 extremely good") mean in near-term (1 -2 years) 4.72 in medium term (3-5 years) 5.48 in long-term (> 5 years) 6.13 the prospects for ec are perceived as getting increasingly better. t-tests show that ratings improve significantly from the near term to the medium term (t = -6.39) and to long term (t = -7.23), and from medium term to long term (t = -6.01) at p < .05. discussion our research indicated that business students perceived greater benefits in the current use of the internet for marketing and distribution than for accounting and finance purposes. they indicated through relatively high ratings that various marketing and distribution capabilities of ec are reasonably well met at present, particularly in the distribution of digital goods, such as software, and enabling online feedback from customers and prospect. other activities highly rated were the use of the internet as a communications media and the ability to target specific interest groups. 41 ajis vol. 6 no. 2 may 1999 accounting and finance capabilities are currently not as highly rated than marketing and distribution. while acknowledging that it costs are being reduced and funds are successfully being transferred electronically, the study participants did not perceive that the use of digital money is currently being achieved. for digital money to become the norm it has to achieve true interoperability, i.e. money must be in a specific form and widely used as a standard or readily convertible to a standard form (camp and sirbu, 1997). the lack of assurance that the above requirements can presently be met appears to be holding back the adoption of digital money. the greatest concern is the security of ec. this is understandable when one considers the workings of ec. when information is sent over the internet, it is split into small data packets that are transmitted over shared communication lines to their destination. the sender has no control over the route that data takes or that messages are not being tampered with. even though security measures exists (e.g. encryption), they are not universally accepted as providing a failsafe environment. for example, "mass retail applications that rely on credit cards for payments are not amenable to encryption, even though they are among the transactions that most need protection from hackers." (arnum, 1995, p 36) furthermore, "message tracing, tracking, delivery receipts and acknowledgmentsstandard features of a private edi network service are still unknown to the internet" (arnum, 1995, p 37). risk factors were generally rated high. those of a technological nature were rated higher than those of a business nature. for the person receiving data there is the risk that the files downloaded contain damaging computer viruses. furthermore, with the use of an extended network, the potential for unauthorised persons (hackers) to access and cause damage to data increases substantially. it is possible that private or confidential information, such as credit card information, is intercepted during transmission. among business risk rated highly is the risk that goods and services ordered on the internet are not of the quality promised. an examination of the mean ratings to establish inter-disciplinary differences showed that marketing students rated the ability of the internet to provide online feedback from customers and prospects far lower than the accountants and information systems students. this is of concern since the capability of the internet to establish a direct link to customers and others is regarded as a very significant marketing advantage (ghosh, 1998). furthermore, the technology used to maintain electronic contact (e.g. email) is mature and reliable. concern over the security of ec was significantly lower among accounting compared to marketing and information systems students. this is surprising since accountants are generally trained to be security aware and adopt a more conservative attitude to business than other disciplines. however, their concern is at a high level and it can reasonably be expected that the prospects of conducting financial transactions over the internet will remain relatively low unless their security concerns can be overcome. information systems students expressed greater concern over the reliability of ec than accounting and marketing students. their concerns are important because they appeared to possess significantly higher current experiences with ec than their fellow students. in business organisations, it is common that information systems professionals are asked to provide assurance on the reliability of systems; if they are concerned it is less likely that other disciplines will be inclined to take-up the new technology. implications the research indicated that the current experience with ec among final year business students was not very high. on a scale of 1 (extremely low) and 7 (extremely high), information systems students rated their experience as 4.84, those in accounting as 3.75, and for marketing it was 3.60. the new generation of business persons as well as educators need to recognise that greater emphasis should be placed on obtaining and providing training and experience in the areas of ec. "for better or worse, electronic commerce will not be an elective course for most senior managersit has become part of the core curriculum." (frank, 1997, p 37) to be able to exploit and manage ec effectively, business persons have to be taught that ec applications are business systems and not just computer systems. whilst ec activities include the traditional elements of computer processing and technology, their impact is also felt in the areas of business strategy, customer-supplier relations, business processes and people. all these elements have to be managed in an integrated and holistic manner. according to daniels (1994), future business will act as if no boundaries exist, cross-functional networks will replace functional organisation structures, and the organisation will develop cultures that fit into any country. the ability of business executives to manage change will be more important than ever before. furthermore, ec is an inter-disciplinary activity which should not be delegate to the information systems (is) or marketing groups. while is professionals have become more business oriented through their involvement in inter-organisational systems and business process re-engineering they cannot replace the business executive's strategic expertise. they most likely will continue to place emphasis on technology issues. the marketing group, on the other hand, has a strong business orientation but generally lacks technological expertise. ec offers marketing the opportunity to foray into the is domain and is to interact more significantly with other business disciplines. 42 ajis vol. 6 no. 2 may 1999 limitations and conclusion one limitation of the study is that not much information was captured about the characteristics of the respondents, in particular their knowledge and types of experiences with the internet and ec. for example, the extent of their knowledge was not assessed by the nature of the units they had completed on this topic during the course of their study and the academic results they had obtained. this made it difficult to provide detailed explanations and recommendations on how their perceptions about ec were formed and could be changed. another limitation concerns the few differences found between the business disciplines being studied by the respondents as reported in table 7. these may have been due to chance and their interpretation contained in the discussion section is therefore of a tentative nature only. furthermore, it is not possible to discover more definite reasons for the differences in perceptions between the three student groupings without additional information background information of the nature highlighted in the preceding paragraph. in conclusion, the success or failure of ec will be determined by the trust that people show towards the use of internet technology. in other words, they will weigh up their perceptions of the capabilities of ec against perceived concerns and risks. our study indicated that business students believe that the capabilities of ec are not currently fully achieved and that high levels of concerns and risks exist. however, they hold high hopes for its future prospects which appear to be very good in both the medium (3-5 years) and long-term (over 5 years). this means that ec is expected to dramatically increase, thereby bringing about a paradigm shift in the way business will operate. business leaders of the future are advised to prepare themselves for the changes ahead. references arnum e. (1995) "doing business on the internet a question of balance", business communications review, august, pp 35-37. briggs r.o., balthazard p.a., & dennis a.r. (1996) "graduate business students as surrogates for executives in evaluating of technology", journal of end user computing, 8(4), pp 11-17. buckeridge r. (1996) "winning the cyberspace race", it chapter, spring, pp 6-7. camp l.j. and sirbu m. (1997) "critical issues in internet commerce", ieee communications magazine, may, pp 58-62. daniels n.c. (1994) information technology: the management challenge, addison-wesley publishing company, wokingham. forcht k.a. & wex r. (1996) "doing business on the internet: marketing and security aspects", information management & computer security, 4(4), pp 3-9. frank m. (1997) "the realities of web-based electronic commerce", strategy leadership, may/june, pp 3137. ghosh s. (1998) "making business sense of the internet", harvard business review, march-april, pp 126135. liddy c. (1996) "commercial security on the internet", information management & computer security, 4(1), pp 47-49. martinez a. (1996) "what risk managers should know about the internet", risk management, 43(11), pp 4346. nath r., akmanligil m., hjelm k., sakaguchi t., & schultz m. (1998) "electronic commerce and the internet: issues, problems, and perspectives", international journal of information management, 18(2), pp 91101. pattison m. (1997) legal implications of doing business on the internet, information management & computer security, 5(1), pp 29-34. sim s.p. & rudkin s. (1997) "the internet past, present and future", bt technology journal, 15(2), pp 1123. society for information management (1997) "electronic commerce: the work is just beginning", software magazine, 17(1), pp 102-106. stuck b.w. (1996) "internet transactions still yield small change", business communications review, july, pp51-54. tapscott d. (1996) the digital economy, mcgraw-hill, new york. wood r. (1996) "doing business on the internet", it chapter, spring, pp 4-5. yavas u. (1994) "research note: students as subjects in advertising and marketing research', international marketing review, 11(4), pp 35-43. 43 a.iis vol. 3, no. 2 groupware and business process improvement: technology enabled organisational learning nereu f. kock jr. robert j. mcqueen department of management systems, university of waikato private bag 3105, hamilton new zealand e-mails: nfk@waikato.ac.nz, bmcqueen@waikato.ac.nz abstract much of the evaluative research on groupware in organisations so far has been preoccupied with the role of groupware as a new interaction media to replace or extend face-to-face communication in groups. it has focused on gains and losses from a group interaction point of view, typically disregarding the impact of other functions such as allowing public access to relevant historical and business process information concerning the organisation. this paper tries to bridge this gap with a focus on the support provided by groupware to business process improvement groups. this paper is based on a qualitative analysis of the effects of the introduction of an asynchronous groupware system into a service company based in brazil. that analysis suggests that improvements on business redesign efficiency and effectiveness can be attained not only from asynchronous groupware support to group communication, but also from: l)its support for public sharing of historical information about former business process improvement; and 2)lts support for providing a repository of information about business processes that could be targets for improvement. the paper is finalised with the proposal of an explanatory model, describing the relationship between the introduction of technology, its integration with a business process improvement meta-process, and its effects on the efficiency and effectiveness of that meta-process. introduction organisations can be defined as sets of interrelated business processes, each composed of interacting agents, artifacts, and suppliers whose main goal is to modify inputs in order to produce valuable outputs for customers. as an exchange for delivering outputs to its external customers, the organisation will get an inflow of capital. this capital will typically be invested to supply further demand from customers, in a cyclic and repeatable process (goldratt and fox, 1986). the most successful organisations nowadays are those whose business processes are efficient, i.e. have an optimum balance between the capacity of generating outputs and costs, and effective, i.e. generate outputs that match or exceed the expectations of their customers. this enables those organisations to sustain a balance between production capacity and actual throughput. this balance, however, is hindered by an element being observed since middle 1980s: a continuous and fast changing marketplace (goldratt and cox, 1986; hammer and champy, 1993), which calls for changes of both radical and continuous proportions (deming, 1986; davenport, 1993). in order to appropriately and quickly respond to external changes, organisations must create local coping mechanisms, at the team level (senge, 1990). one instance of such decentralised mechanisms is referred to in this paper as business process improvement group (bpig). in the following sections business process improvement (bpi) is defined and the importance of bpigs in a decentralised learning context is discussed. the role of groupware in this context is described and illustrated based on a service organisation's experience. an explanatory model is presented to explain the effects of an instance of groupware asynchronous groupware on bpi in organisations. this model is discussed, in the last two sections, in terms of its limitations and implications for future research. business process improvement business process improvement (bpi) has been regarded as the basis for some world class organisational development approaches (french and bell, 1990). two representative examples of succesful bpi endeavours can be found within the management literature. first, the "economic japanese revolution", strongly based on the concept of gradual process improvement, also referred to as kaizen (deming, 1986; walton, 1989). second, and more recent, the business process re-engineering (bpr) movement, where working groups are expected to propose radical changes for business processes that will, in turn, generate revolutionary quality and productivity improvements (hammer and champy, 1993; davenport, 1993). bpi is here used as a general term to refer to improvement schemes based on the concept of business process, whether they are radical, incremental, or somewhere in between in terms of degree of improvement sought and realised. bpi is assumed to be carried out by small groups which analyse and propose improvement on business 36 a.iis vol. 3, no. 2 processes. from this assumption, two main components are identified as forming the basis on which bpi lies: 1 )business process; and 2)bpi groups. business process a business process can be understood as both a value-adding unit and as a set of interrelated activities. as a value-adding unit it is analogous to the classical view of system (checkland, 1981) for it can be defined as: an abstract entity which represents the transformation of inputs into outputs. a business process has its suppliers and customers. inputs are provide by suppliers and the outputs are consumed by customers. the transformation aims at adding value to the inputs by changing them in a way that turns them into desirable outputs from the customers' point of view. a business process can also be split into other sub-processes. the set of lower level processes into which a business process can be broken are often called "tasks" or "activities" which, in turn, bear some kind of interdependence among themselves (dennis, hayes and daniels, 1994). the main goal for modeling a business process by splitting it into component activities is to understand how the interrelations between activities affect the overall performance of the business process, in terms of efficiency and effectiveness. the combined study of a business process from an activity-based perspective and as a value-adding unit, enables a business process improvement group to propose changes in order to improve some of its attributes such as customer satisfaction, cost, lead time, and simplicity. business process improvement groups continuous and radical bpi have often been considered as extremes. this fact has led in the past to the definition of a dual taxonomy (davenport, 1993), and a continuum of bpi approaches where the main difference is the degree of improvement sought and realised (dewar and dutton, 1986; damanpour, 1988). however, all instances of bpi share some common characteristics, such as their dependence on working groups. these working groups, called here business process improvement groups (bpigs), have the following properties: l)they are generally small groups, i.e. from 3 to 12 members (soles, 1994); 2)their "modus operandi" is characterised by a process (king, 1990); and 3)they have defined roles (rosenfeld and servo, 1990; hammer and champy, 1993). bpigs are effective tools to enable organisation learning for they foster the thinking on and reshaping of the basic units of organisations, their business processes. in addition, bpigs have the potential to reduce barriers and foster information sharing among different work teams^. this is essential to generate team learning, which has its importance highlighted by senge (1990) who states: "team learning is vital because teams, not individuals, are the fundamental learning unit in modern organizations. this is where 'the rubber meets the road'; unless teams can learn, the organisation cannot learn" (p. 10). groupware and business process improvement groups the term "groupware" is new. in fact it has only become popular since 1991, when pc week issued the first special supplement^ on this subject (mcqueen, 1993). later, in 1992, a conference focusing on commercial cscw products and their impact on organisations was held in us. that conference was called groupware '92 (coleman, 1992). there is no generally accepted definition for groupware. one definition which is acceptable within the scope of this paper is provided as follows. this definition is a convergence of three other definitions (johansen, 1988; ellis, gibbs and rein, 1991; mcqueen, 1993). groupware is a generic term for computer-based systems which are particularly used to support groups of people engaged in a common task. typically, these groups are small and have relevant tasks with definite deadlines. 5a team is a permanent group, who perform activities in a business process. a bpig is typically a temporary group whose main goal is to improve a business process, whether its members are daily involved with it or not. 6pc week, special report on groupware, october 14, 1991. 37 aj1s vol. 3, no. 2 bpigs can have both their efficiency and effectiveness augmented by the use of groupware systems. this assumption is supported by related achievements that have been reported about several aspects, such as: • better support to group activities, such as making communication faster and cheaper (sproull and kiesler, 1991), reducing paper flow (wilson, 1991), making filing easier (brothers et al., 1992) and increasing cross-departmental communication (clement, 1994). • positive effects on individual behaviour, such as reducing stress (pietro, 1992) and making individuals communicate more openly (sproull and kiesler, 1991). • positive effects on group behaviour, such as distributing individual contributions more evenly (nunamaker et al., 1991), separating ideas from individuals (chidambaram and kautz, 1993), reducing repetition of old ideas and increasing commitment with group decisions (sheffield and gallupe, 1993). despite the broad range of possible applications of groupware systems to support bpigs, most of the research has been focused on group decision support systems. these systems are considered a category of groupware. they usually support decision meetings, where the final decision is the main outcome. brainstorming, voting, ranking and classification of ideas are examples of tasks supported by a typical group decision support system (dennis, hayes and daniels, 1994). group decision support systems support synchronous communication, where users must be connected to the system at the same time to be able to interact. it is like the telephone, in which one cannot communicate to another person unless s/he is able to answer the call. this category of groupware systems is opposed to asynchronous groupware, where individuals can interact at different times. examples of asynchronous groupware are electronic-mail, computer conferencing and workflow control systems (kock, mcqueen and fernandes, 1994). groupware and bpi: a service company's experience the experience of a service company based in brazil is investigated in this section. this investigation aims at illustrating the concept of bpi groups and related effects of groupware technology. the service company studied is an international events organiser, referred to as equipe7. equipe was started by early 1964 as a consulting firm. its main goal was to deliver consulting services to the manufacturing industry. equipe's main activity in the early years after establishment was facilities layout planning. after approximately five years struggling to survive amongst a fierce competition, they glimpsed a new market niche, to organise the participation of brazilian companies in international fairs in order to promote their products in new markets. it became and remained equipe's main business. soon after having started in the new market niche of international events operation, equipe had built a strong reputation of service quality and reliability. equipe sustained the position of main supplier in brazil for international events held in europe, united states and asian countries for 20 years, led by a couple of aggressive executives. some dark clouds, however, showed up on the horizon by 1989. inability to learn and adapt to change from early 1989 on, the events industry client basis started to shrink in brazil. new competitors entered the market grounded on either vertical integration of stands manufacturing and rental, or strong partnerships with stand suppliers. as most of the costs of an exposition come from the design, rent, transport and assembly of stands, it seemed to be impossible for equipe to survive in this new competitive arena without good partnerships. prices would increase and quality deteriorate without it. however, equipe's main executives failed to grasp the competitive importance of partnerships, mainly because they were too busy trying to find new clients and cope with an increasingly changing marketplace. as one of them mentioned with a frown of concern on his face: "we're passing through a period of turbulence. i hope things will settle down after a while". equipe's executives centralised most of the decisions. this approach for a growing company in a fierce competing environment proved to be a disaster in several occasions. one example was the insistence of equipe's executives on developing software systems at home to support their operations. this position was maintained against the advice of consultants and internal computer support staff, mainly based on fear that the company's know-how on events operation would go public if equipe hired an external software development 7it is a fictitious name. the real name is concealed to protect confidentiality. 38 ajis vol. 3, no. 2 company. this drove the company into a project that took three years to develop a computer system, which turned out to present several limitations. the system was designed to support the integrated work carried out by the marketing and operations divisions. internal users of the system would basically be able to store updated information regarding customers and keep track of the evolution of respective services provided by equipe through stages, ranging from prospecting new customers to post-evaluation of events. after two years the system finally achieved an acceptable level of efficiency, but now new technologies would enable radical changes in the core processes. these changes called for deep modifications in the system, which was eventually abandoned. several interviews revealed symptoms of an organisation in which the learning process was centered on its top management. both senior executives, who shared the ceo position, showed clear signs of stress. they were frustrated with the new market situation, accusing some stand suppliers of lack of loyalty. the failure of the software system described above was a source of cynicism and lack of trust on information technology. employees were unmotivated, reporting as the most important reason for their lack of motivation the excessive centralisation of decisions by the two executives. the employees showed no interest in improving business processes, and a strong lack of interest on learning new methods and techniques. as one of the managers reported:" what keeps them working are their wages and the prospect of international trips". starting the business process improvement croups the inability to motivate workers at equipe called for changes in the management approach. some techniques for enhancing the level of participation in decisions were tested such as brainstorming sessions, suggestion boxes, and campaigns for new ideas. none of them generated the expected gains, basically due to two reasons. first, they were strictly based on the assumption that front-line workers should have an active role on management and solution of problems, which is one of the several facets of process improvement. employees were, among other things, called to participate in routine strategic decisions irrespective of the type of decisions, following an approach suggested by semler (1989; 1993). however, it proved to be a counterproductive strategy, supporting the assumption that group decisions are not necessarily better than individual decisions. in fact, they can cause decisions to take more time to be reached and also be of less quality (senge, 1990). strategic decisions to form new partnerships or take a large government contract, for example, were found, after some trials, better be made only by managers as usual. sharing the responsibility of taking decisions with large groups of employees not directly involved with decision making in a daily basis simply delayed what could not be delayed, undermining both employees and management's confidence on participatory management. second, business processes received low priority. for example, the whole set of interrelated activities involved in signing a new contract, from gathering relevant customer and event information to the analysis of contribution margins, was never discussed. the emphasis was on having employees participate in management activities, rather than analysing how activities were executed and improvements could be attained. new ideas coming from employees covered a broad range of subjects, from new designs for fliers to better wages. the broad range of subjects and the repetition of ideas progressively undermined the interest of managers, and consequently employee motivation to generate new ideas. one of the authors (kock) worked as a organisational consultant at equipe for a little over one year, which enabled him to perform a study with action research characteristics (mckernan, 1991; elden and chisholm, 1993). the work was started with an analysis of the organisation and proposal of a set of changes at the management and operational levels. based on former experience and on a methodology developed to implement group-based quality and productivity improvement (kock and tomelin, 1994), a project to implement bpigs was planned and put into practice. it was predetermined that all bpigs at equipe should complete a process improvement proposal within a limited time. as opposed to business process re-engineering groups (hammer and champy. 1993; soles, 1994), they were neither expected to be composed mostly by management staff and external consultants, nor to come up with genuinely radical improvements i.e. the improvements proposed could be modest and gradual. each bpig was expected to complete the analysis and redesign of a business process within one to eight weeks. after that a proposal should be generated and handed to an improvement committee (which included the researcher) to be evaluated. should the proposal be considered attainable and prone to generate return on investment, the improvement committee would coordinate with the appropriate staff to implement the proposed redesign. introduction of an asynchronous groupware system bpigs started in a very modest fashion. after three months eight bpigs have been completed and five were ongoing. as the company had nearly 70 employees, its full capacity would be approximately 10 to 23 bpigs at 39 a.i1s vol. 3, no. 2 a time, what would give an average of 33 groups per quarter^. five, out of the first eight bpigs, proposed business process improvements. the remaining three proposed minor problem oriented solutions which affected business processes but clearly were not based on business process analysis, i.e. they were based on problem analysis rather than on business process analysis^, which is consistent with the results of previous research (dennis et al., 1993). this happened despite the eight hours training session in which the concept of business process was discussed along with a structured approach to perform analysis and design. these training sessions were performed with groups of 10 to 20 workers and involved all company members, including top management. some employees who have taken part in no improvement group reported, as one of the causes for such a behaviour, that they could not attend to bpig meetings because they would likely to be engaged in an external activity (e.g. discussing a contract at a client) during group meetings. this problem called for an asynchronous groupware system through which group members could interact from their own computers at different times. as nearly 95% of the workers had access to a networked computer, an electronic-mail system was installed and adapted to incorporate simple computer conferencing features. each bpig was assigned a group mailbox upon its creation. bpig mailbox titles were suggestive of the business process under consideration, and held postings from group members related to different stages of the bpi meta-process (i.e. business process definition, analysis, and redesign). some bpig mailboxes allowed public access within the organisation, so other employees could participate on ongoing discussions. other bpig mailboxes restricted access to their respective bpig members only, when the bpi work was perceived as addressing delicate matters. some public mailboxes were created and gradually fed with information regarding the main business processes of the organisation. this information embodied descriptions of external suppliers for the process, main activities, people and resources involved, and main customers of the process. it also contained some regularly updated postings with information about efficiency, costs, lead time, and customer complaints for each business process. employees were given some training on how to use the groupware system to carry out bpig activities and were encouraged to make as much use of the system as possible. qualitative results from the use of the system during the first three months after the introduction of the groupware tool the number of bpi groups nearly doubled. the percentage of groups proposing process focused improvements remained the same, relatively to the total numbers of groups, but a help document attached as a message in a public mailbox was reported as a useful support for the members to avoid lack of structure when carrying out bpig activities. from a communication media perspective, most of the groups reported using the groupware system for supporting approximately two thirds of the interactions between group members. some problems arose at the earlier stages of utilisation of the groupware system. these problems were caused by the lack of features presented by asynchronous groupware to replace face-to-face meetings. each bpig had a leader and a facilitator, chosen from within the group by the group members themselves. the leader basically coached the group, while the facilitator provided support on business process modeling and analysis. in a typical face-to-face meeting all tasks are accomplished by all members at the same time. through the use of asynchronous groupware, though, it was not possible or would take too long for them to communicate, since members are separated by time and space. it called for a change in the structure of the traditional bpig. the new structure devised pushed the responsibility for accomplishing each one of the stages of a bpig to a certain member of the group, which would post in the mailbox the results of her/his work and wait for comments. for example, the analysis stage, in which the business process is modeled and analysed, is carried out by one of the members supported electronically and face-to-face by the facilitator. the function of the leader is still to coordinate the activities and resolve conflicts. both the leader and the facilitator functions can be overlapped with other group activities, e.g. process analysis. members reported that this new structure improved productivity. reasons presented were the ability to carry out parallel activities and the reduction of the time spent, in comparison to ordinary meetings. other results reported during interviews or identified via participant observation were: l)the reduction of redundant improvement proposals after the adoption of the new bpi approach, when compared with previous improvement approaches tried at equipe. 2)the perceived higher efficiency in the analysis of business processes, reported as the most time consuming bpi stage, and in the redesign stage. 3)the perceived high importance of the bp analysis stage to the next stage, bp redesign. 4)the perceived usefulness of the public considering that a group would take in average 4 weeks to complete its work. 9for example, the response to customer complaints on quotations for events was a change in the format of the quotation document, as opposed to a change in the way the quotation process was carried out. 40 ajis vol. 3, no. 2 access to historical information on former bpi, both as a source of examples to perform current bpi and as a source of solutions whose application do not demand duplicate bpi effort. 5)the perceived value of public access to business process information regarding customer satisfaction, throughput, and lead time. it was reported that the easy access to this information allowed better identification of bpi opportunities and performance in the bp analysis. as reported by one events operator "before we stoned using it [the group-ware system] we had to spend hours analysing a pile of paper to find out some information about our main operations. even so i couldn't tell you whether there was something wrong with them or not". 6)the perceived value of faster and easier communication supported by the system, in the bp analysis and design stages, for the identification and correction of mistakes and wrong assumptions. 7)the perceived importance of the role of the groupware system to support the spread of historical information about former bpi and bp-related information. both senior executives reported great satisfaction with the results of the bpigs. one of them reported that "we have never been through a so successful motivational endeavour since the firm was founded". it was also reported that bpigs improved the relationship between high management staff, middle managers, and workers. some managers reported being impressed, when browsing through bpig mailboxes held by the groupware system, by the increased awareness of workers about the problems of the firm and their willingness to find solutions. after one international event where some problems were detected at the check-in and stand assembly processes, groups were immediately formed and came up with effective business process redesigns. this was a hint that the ability to learn and adapt to change was spreading over the company's body, instead of concentrating only in its head. an explanatory model the analysis of equipe's experience, complemented by a review of related literature, provides the necessary background to build an explanatory model regarding the use of asynchronous groupware to support business process improvement. this model is illustrated in figure 1. in figure 1, each rectangle represents one variable. each arrow represents an interdependence between variables. at the destination of each arrow is a variable which depends on the variable at the source of the arrow. in this sense, for example, the variable <bpi framework> is affected by the variables <bpi metaprocess> and asynchronous groupware methodologyx these two variables are also the independent variables of the model, since they are not influenced by any variable. the dependent variables of the model, seen as a whole, are the two rightmost ones <bpi efficiency> and <bpi effectivenessx the variables in between have the function of providing an explanation on how the model operates. the model depicted in figure 1 places emphasis on the interdependence between variables. a description for each one of the variables is provided as follows, starting from the right part of the figure and moving leftwards. the meaning of the interdependencies is supplied along with supporting rationale based on equipe's experience and related literature. bpi history information access / bpi oportunily identification bpig communication bp redesign figure 1: asynchronous groupware and business process improvement (an explanatory model) • bpi efficiency. this variable represents the efficiency of the bpi meta-process. it is a combination of the measures of the average time spent by bpigs to redesign a business process, the number of bpigs at a specific time, and the costs involved in the bpi meta-process. 41 ajis vol. 3, no. 2 • bpi effectiveness. it represents the average effectiveness of the business process redesigns carried out by bpi groups. this effectiveness can be understood as the effect that the redesign has on the quality and productivity of the redesigned business process. bpi effectiveness can be evaluated by experts, who forecast quality and productivity improvements in the business process after the redesign. however, an actual improvement measure can only be obtained after the redesign is consolidated through implementation and routinisation. • bpi opportunity identification. this variable represents both the efficiency and effectiveness in the identification of bpi opportunities. by making it easier, less expensive, or more efficient e.g. by providing information about similar bpi that can be replicated with little effort the efficiency of the bpi metaprocess itself is increased, which is illustrated by the connection between this variable and <bpi efficiencyx this assumption is suggested by the reduction in the number of redundant improvement ideas, reported from previous improvement approaches tried at equipe, after the new bpi framework was implemented. • bp analysis. this variable represents both the efficiency and effectiveness of the analysis stage of bpi, performed by a group. measurements for this variable can be done based on the time taken to carry out the bp analysis, as well as on the clarity and completeness of the information generated for the next stage in the bpi meta-process, bp redesign. variations in the <bp analysis> variable affect the efficiency of the whole bpi meta-process as well as the next stage in the bpi meta-process, bp redesign. the effect on the efficiency of the bpi meta-process is suggested by the fact that it is the longest stage of the meta-process. the effect on the next stage, bp redesign, was indicated in interviews with several bpigs members, where they showed concern about the negative effects of lack of clarity and completeness in the information coming from the bp analysis stage. • bp redesign. this variable represents both the efficiency and effectiveness of the redesign stage of bpi, performed by a group. this variable is likely to affect both the efficiency and effectiveness of the bpi meta-process as a whole. as it is a substantial part of the bpi meta-process, a loss in efficiency in the bp redesign stage e.g. extended time and high costs has direct impact on bpi's efficiency. on the other hand, a poor redesign in terms of quality is likely to impair the effectiveness of the bpi meta-process, as the actual benefits sought may not be attained. the connection between efficiency and effectiveness of the bp redesign and the bpi meta-process is supported by a number of previous studies (hammer and champy, 1993; davenport, 1993; dennis, hayes and daniels, 1994). yet metrics to predict the effectiveness of bpi meta-processes, based on bp redesign quality are still to be developed and validated. • bpi history information access. this variable represents the degree of access prospective bpig members have to historical information about former bpi. this historical information consists of documentation generated by former business process improvement groups. a high degree of access to information regarding other bpigs' work is likely to positively affect the variable <bpi opportunity identificationx it provides examples to be followed, and results that can be replicated without duplicated effort, as reported by bpig members at equipe. • bp-related information access. this variable represents the degree of access prospective and actual bpig members have to information regarding business processes. it affects two variables: <bpi opportunity identification> and <bp analysisx the access to bp-related information, such as customer satisfaction and throughput, allows prospective bpig members to identify the best opportunities for bpi. it also improves the productivity of business process analysis, which relies heavily on information gathering and analysis. these effects were reported by bpig members at equipe. • bpig communication. this variable represents the efficiency and effectiveness of communication between the members of a bpig and with outside. it is likely to affect both the efficiency and effectiveness of the bpi analysis and design stages, i.e. the variables <bp analysis> and <bp designx this relationship is explained by the fact that the analysis and redesign phases rely strongly on information exchange and discussion between bpig members and with others who, despite being outside the group, can provide relevant information about the business process being improved and possible redesign alternatives. this point is supported by the literature on group work (napier and gershenfeld, 1993) and also by equipe's experience. the experience at equipe shows that fast and easy communication between bpig members 42 ajis vol. 3, no. 2 and with others in the organisation, makes it easier for mistakes in the bp analysis and redesign stages to be identified, which affects both the efficiency and effectiveness of those stages. • bpi framework. this variable represents the existence of a bpi framework. this framework is the result of the combination between a bpi meta-process, i.e. a structured process to perform business process improvement, and asynchronous groupware technology, i.e. a software system with asynchronous groupware features. the bpi framework will allow business process improvement to be carried out with technology, which directly affects three variables: <bpi history information access>, <bp-related information access>, and <bpig communicationx from the analysis of equipe's experience it can be seen that the introduction of an asynchronous groupware system is unlikely, per se, to impact business process improvement in a positive way. it must be accompanied by a redesign of the meta-process through which bpi is carried out. however, equipe's experience also suggests the potential of asynchronous groupware systems to improve organisational access to bpi historical information and bp-related information access. the effect of asynchronous groupware on bpig communication is extensively supported by the literature, as discussed at the end of this section. • asynchronous groupware technology. this variable represents the presence of a software system with asynchronous groupware features, which is integrated with the bpi meta-process to form the bpi framework. • bpi meta-process. this variable represents the existence of a bpi meta-process to perform business process improvement. it is integrated with the asynchronous groupware system to generate the bpi framework. the literature of evaluative studies of groupware systems to support communication in groups generally reports gains in productivity and quality (to a lower extent), concerning the interaction process (sproull and kiesler, 1991; nunamakeret al., 1991; brothers et al., 1992; chidambaram and kautz, 1993; clement, 1994). support to those results comes from equipe's experience, specifically regarding asynchronous groupware, by the fact that some group members could not participate on bpigs, before the introduction of the groupware system, due to external meetings and time constraints. after the introduction of the system they could become fully engaged members of bpigs, and reported enjoying a more active participation in the improvement of the company. the proposed explanatory model, though, presents a contrast with much of the research performed so far on groupware evaluation. former research has been preoccupied with the role of groupware as a tool to replace or expand face-to-face communication in groups (serida-nishimura, 1994). it has focused on gains and losses from a group interaction point of view and, with some exceptions (orlikowski, 1992; rein, holsapple and whinston, 1993), typically disregarded the impact of other groupware functions such as allowing public access to relevant historical and business process information regarding the organisation. these latter are the main aspects stressed in the model described in this paper. limitations of the current research the results outlined in the explanatory model might have been distorted by some sources of bias. the two main sources of bias in this research were: l)the fact that the research was based on only one site, which may affect the generality of the research from an organisational perspective, and 2)the high involvement of the researcher with the organisation, conducting the research concurrently with a consulting project. in fact, both of the sources of bias are characteristic of action research (galliers, 1992; sommer, 1994). most action research in organisations have been performed in only one or a few sites in order to allow the researcher to have a deeper understanding of the context being studied within a relatively short period of time (candlin and wright, 1991). several sites would either delay the reporting of results, or disperse the focus of the research. both effects are undesirable. the second source of bias, the high involvement of the researcher, was inevitable since it is one of the attributes that define action research as a distinctive research approach (francis, 1991). the first source of bias could be reduced in the current study by following one of two different options. the first option would be to produce as the final result of the research a specific model, explaining the relationship between asynchronous groupware and bpi within a specific context (e.g. a type of organisation or activity). the second option, which was the one adopted, was to base the investigation on units of analysis that are common to all organisations, i.e. business process, business process agents, business process improvement, and business process improvement groups. it also allowed the access to a considerable number of instances for each unit of analysis, which is likely to improve the significance of the results. 43 ajis vol. 3, no. 2 the second source of bias could also be reduced by the researcher following two complementary protocols: 1)analyse data in a rational way and from a disengaged perspective. this is highly dependent on the researcher's own ability to detach from the context being studied, and on his/her lack of commitment toward positive results for the organisation. 2)involve a group of external researchers in the analysis of the data collected. in this research none of the protocols were fully implemented. the breach in the first protocol was caused by the position hold by the researcher as a coach for some bpigs and champion in the introduction of the groupware system. this position was taken in order to enhance the likelihood of success of the endeavour. it could have affected the results of the opinions expressed in the interviews, as the interviewees could feel compelled to express over-optimistic opinions about the effect of the system in order to please the researcher. an alternative to reduce this source of bias would be the search of coaches and champions for the research project within the organisation. this alternative, though, could hinder the research development due to lack of suitable individuals with enough time to commit to the research project. the breach in the second protocol is due to the fact that the study was carried out by only one researcher, in cooperation with some of the organisation members. since there were potential sources of bias from both the researcher and organisation members, an alternative would be to involve researchers from outside the organisation. recommendation for further research the explanatory model proposed is preliminary. it is the result of a qualitative analysis based on a study with action research characteristics. further research is suggested to improve the model proposed by concurrently expanding and refining it. possible research approaches suggested are case research (yin, 1989) and action research (checkland, 1991). other major research approaches such as survey research and experimental research are not recommended due to some constraints, such as: l)the longitudinal property of the research that originated the model, and its disparity with cross-sectional studies; 2)the limited number of organisations currently using asynchronous groupware systems to support bpi; and 3)the orientation of the model towards the description of real organisational settings, as opposed to controlled environments. acknowledgments the authors would like to thank the anonymous reviewers of this paper for their helpful comments. the authors also reported on this study in "integrating groupware into a business process improvement framework" in information technology & people, vol. 8, no. 4, 1995. references brothers, l., j. hollan, j. nielsen, s. stornetta, s. abney, g. furnas and m. littman (1992), supporting informal communication via ephemeral interest groups, proceedings of cscw'92 conference, j. turner and r. kraut (eds), new york: acm press, pp. 84 to 90. candlin, d.b. and s. wright (1991), managing the introduction of expert systems, international journal of operations & production management, vol.12, no.l, pp. 46-59. checkland, p. (1981), systems thinking, systems practice, new york: john wiley & sons. checkland, p. (1991), from framework through experience to learning: the essential nature of action research, information systems research: contemporary approaches and emergent traditions, h. nissen, h.k. klein and r. hirschheim (eds), new york: north-holland, pp. 397-403. chidambaram, l. and j.a. kautz (1993), defining common ground: managing diversity through electronic meeting systems, proceedings of the 14th international conference on information systems, j.i. degross, r.p. bostrom and d. robey (eds), new york: the association for computing machinery, pp. 1-11. clement, a. (1994), computing at work: empowering action by low-level users, communications of acm, vol.37, no.l, pp. 53-63. coleman, d. (ed) (1992), proceedings of groupware '92, san mateo, ca: morgan kaufmann, pp. 84-90. damanpour, f. (1988), innovation type, radicalness, and the adoption process, communications research, vol.15, pp. 545-567. davenport, t.h. (1993), process innovation, boston: harvard business press. deming, w.e. (1986), out of the crisis, mit, cambridge, ma: center for advanced engineering study. dennis, a.r., r.m. daniels jr, g. hayes, j.f. nunamaker jr. (1993), methodology-driven use of automated support in business process re-engineering, journal of management information systems, vol.10, no.3,pp. 117-138. 44 a.iis vol. 3, no. 2 dennis, a.r., g.s. hayes and r.m. daniels jr. (1994), re-engineering business process modeling, proceedings of 27th hawaii international conference on system sciences, los alamitos, ca: ieee computer society press, pp. 244-253. dewar, r.d. and j.e. dutton (1986), the adoption of radical and incremental innovations: an empirical analysis, management science, vol.32, pp. 1422-1433. elden, m. and r.f. chisholm (1993), emerging varieties of action research, human relations, vol. 46, no. 2, pp. 121-141. ellis, c.a., s.j. gibbs and g.l. rein (1991), groupware: some issues and experiences, communications of acm, vol. 34, no. 1, pp. 38-58. francis, d. (1991), moving from non-interventionist research to participatory action, proceedings of the first world congress on action research, vol.2, c. collins & p. chippendale (eds), sunnybank hills, queensland: acorn, pp. 31-42. french, w.l. and c.h. bell jr. (1990), organization development, englewood cliffs, nj: prentice hall. galliers, r. (1992), choosing information systems research approaches, information systems research, r. galliers (ed), boston: blackwell scientific publications, pp. 144-162. goldratt, e.m. and j. cox (1986), the goal: a process of ongoing improvement, new york: north river press. goldratt, e.m. and r.e. fox (1986), the race, new york: north river press. hammer, m. and champy, j. (1993), reengineering the corporation, new york: harper business. johansen, r. (1988), groupware: computer support for business teams, new york: the free press. king, n (1990), innovation at work: the research literature, innovation and creativity at work, m.a. west and j.l. farr (eds), new york: john wiley & sons, pp. 15-59. kock jr., n.f., r.j. mcqueen and c.t. fernandes (1994), quality management and groupware, proceedings of the 18th annual conference of the national association of higher education programmes in management, information management section, c.m. da silva, j.e. fensterseifer and c.g. neto (eds), rio de janeiro: anpad, pp. 98-113. kock jr., n.f. and c.a. tomelin (1994), pmqp: total quality management in practice, curitiba, brazil: business development centre-senac/pr (in portuguese). mckernan, j. (1991), action research, proceedings of the first world congress on action research, vol.2, c. collins & p. chippendale (eds), sunnybank hills, queensland: acom, pp.65-86. mcqueen, r.j. (1993), groupware: experience in new zealand, proceedings of the 13th new zealand computer society conference, vol. 1, auckland, new zealand: the new zealand computer society, pp. 10-20. napier, r.w. and m.k. gershenfeld (1993), groups theory and experience, boston: houghton mifflin. nunamaker, j.f., a.r. dennis, j.s. valacich, d.r. vogel, and j.f. george (1991), electronic meeting systems to support group work, communications of acm, vol.34, no.7, pp. 40-61. orlikowski, w.j. (1992), learning from notes: organizational issues in groupware implementation, proceedings of cscw'92 conference, j. turner and r. kraut (eds), new york: the association for computing machinery, pp. 362-369. pietro, c. (1992), groupware meetings that work, proceedings of groupware'92 conference, san mateo, ca: morgan kaufmann, pp. 50-58. rein, g.l., c.l. holsapple and a.b. whinston (1993), computer support of organisation design and learning, journal of organizational computing, vol.3, no. 1, pp. 87-120. rosenfeld, r. and j.c. servo (1990), facilitating innovation in large organizations, innovation and creativity at work, m. a. west and j. l. farr (eds), new york: john wiley & sons, pp. 251-263. semler, r. (1989), managing without managers, harvard business review, september-october, pp. 76-84. semler, r. (1993), maverick, london: arrow. senge. p.m. (1990). the fifth discipline, new york: doubleday. serida-nishimura, j.f. (1994), an organizational culture pespective for the study of group support systems, proceedings of the 15th international conference on information systems, j.i. degress, s.l. huff and m.c. munro (eds), new york: the association for computing machinery, pp. 201-211. sheffield, j. and b. gallupe (1993), using electronic meeting technology to support economic policy development in new zealand: short term results, journal of management information systems, vol.10, no.3, pp. 97-116. soles, s. (1994), work reengineering and workflows: comparative methods, the workflow paradigm, e. white and l. fischer (eds.), alameda, ca: future strategies. sommer, r. (1994), serving two masters, the journal of consumer affairs, vol.28, no.l, pp. 170-187. 45 ajis vol. 3, no. 2 sproull, l. and s. kiesler (1991), computers, networks and work, scientific american, september, pp. 8491. tapscott, d. and a. carston (1993), paradigm shift, new york: mcgraw-hill. walton, m. (1989), the deming management method, london: mercury. wilson, p. (1991), computer supported cooperative work, oxford, england: intellect. yin, r.k. (1989), research design issues in using the case study method to study management information systems, the information systems research challenge: qualitative research methods, j.i. cash and p.r. lawrence (eds), boston: harvard business school, pp. 1-6. 46 ajis vol. 4 no. 2 may 1997 an investigation of information technology-enabled remote management and remote work issues d. sandy staples department of information systems the university of melbourne parkville, victoria 3052 australia phone: +61 3 9344 9260 e-mail: s.staples@dis.unimelb.edu.au abstract a two phase research study was done to investigate remote work and remote management issues. in phase 1, focus groups were carried out with remote managers and remote employees to identify key issues. the most common key issues dealt with communications, information technology, leadership and coaching, teamwork, building trust, and performance management. in the second phase, a questionnaire was used to test hypotheses developed from phase 1. the findings supported that higher trust leads to higher job satisfaction and lower job stress, and that more communication between the manager and the remote employee develops higher levels of employee organizational commitment. introduction terms such as downsizing, restructuring, delayering, telecommuting, and virtual organizations are common in today's business literature. all these phenomena have at least one thing potentially in common. they can move the manager farther from the worker, and vice versa. this can occur because the span of control is larger (i.e., have more people to manage) or simply because the organization is structured to be a virtual one, where few people are physically located near each other. therefore, it is becoming common for managers to have to manage some or all of their employees remotely. the purpose of this research study was to investigate the practice of managing people remotely and the potential enabling role of information technology in creating the ability to work remotely. the concept of remote management is defined as managing employees who work in a physically separate location from their managers. the employee's location can vary considerably from working at another company office or in their home, to working at a customer's location or out of their car. employees working at home are by definition telecommuting; however, telecommuting is just one work arrangement that results in remote management. as suggested by jenner (1994), telecommuting is only a small part of the virtual workplace, in which people work together while being physically distant from each other. the concept of virtual organizations and the resulting remote management is not new. for example, sales organizations have been working remotely for decades or more. handy (1995) stated that what is new is the technology that allows the concept to be applied much more broadly than in the past. the basic assumptions of modern organization theory and practice, such as the need to physically group people and units together to provide for coordination and supervision, are being increasingly invalidated by information technology (it). coordination of tasks, functions and people no longer require physical proximity (lucas and baroudi, 1994; snell, 1994). information technology plays a key enabling role for the new forms of organization (handy, 1995; lucas & baroudi, 1994; freedman, 1993; greengard, 1994; o'hara-devereaux & johansen, 1994). the technology allows tasks to be distributed in different places and executed at different times while integrating and effectively controlling the whole process (mowshowitz, 1994). the virtual workplace provides access to information needed to do a job anywhere, anytime, anyplace and the latest in communication technology is used to accomplish this (jenner, 1994). bleecker (1994) predicts that integrated computer and communication technology will transform corporations into collaborative networks linking people together. remote management and remote work also appears to be a growing phenomena and there is a high level of interest in the topic at the practitioner level. illingworth (1994) reports that companies are asking about the impact the virtual office concept is having on worker satisfaction, individual identity, and corporate community. experience with this study supports illingworth's claim since support for this study has been strong from industry. the research program was broken into two phases. exploratory research was first undertaken to investigate the suggestions in the literature and develop hypotheses for testing, via a questionnaire, in phase 2. therefore, this paper is organized as follows. section 2 reviews literature relevant to remote management. section 3 presents the methodology and results of phase 1 of the study. section 4 presents the hypotheses to be tested in phase 2. section 5 discusses the methodology used for the second phase 82 i ajis vol. 4 no. 2 may 1997 of the study and reviews the findings. the last section comments on the findings, their contributions and limitations. current state of the literature although the literature that addresses remote management is sparse and based on anecdotal evidence, it does contain some suggestions for managing people that are physically separate. developing trust and minimal supervision expectations are important since it is very difficult to supervise and control remote employees (handy, 1995; lucas & baroudi, 1994; snell, 1994). however, trusting employees often goes against a managerial tradition that believes control and efficiency are closely linked and that you can't have one without the other (handy, 1995). gerber (1995) suggests that trust can be generated in two ways. it comes from developing a relationship through communications and informal activities, as well as from performance (i.e., deliver on promises and be competent). another remote management issue from the literature is the ability of remote managers to manage the subtle processes of cultural and political change in organizations (walsham, 1994). morgan (1988) suggests that the values and beliefs (i.e., corporate culture) of an organization serves as a guide to help parts of the organization move in the same direction in a highly decentralized style of management. maintaining the corporate culture in a remote environment (illingworth, 1994; greengard, 1994) and changing the culture to accommodate the new way of working that it enables (lucas & baroudi, 1994; beyers, 1995; bleecker, 1994; morgan, 1988) are challenges that remote managers and workers face. illingworth (1994) and others (e.g., gerber, 1995; mowshowitz, 1994; greengard, 1994) also suggest that remote managers need to move towards result-oriented assessments (i.e., a focus on what did you contribute today, not whether or not you were at your desk), rather than process-orientated assessments. mowshowitz (1994) suggests that "the essence of virtual organization is the management of goaloriented activity in a way that is independent of the means for its realization" (p. 270). the telecommuting literature also indicates that the management of people who are remote from their managers (i.e., in this case, working at home) is a major issue. managers are concerned that telecommuting will hamper their ability to manage effectively. the fear of lost managerial control is reported to be a significant factor preventing widespread adoption of telecommuting (e.g., desanctis, 1984; duxbury, higgins & irving, 1987; roderick & jelley, 1991; phelps, 1985; savage, 1988; & nilles, 1985). managers fear that they will have to change their management style since they cannot rely on visual contact for monitoring and control. several researchers suggest that a more result-oriented management style is needed (olson, 1988; judkins et al., 1985; hamilton, 1987; metzger & von glinow, 1988; duxbury et al., 1987). trust is also suggested as being a key ingredient to manage telecommuters as well as a shift in management from being a passer of information to being a leader or coach (duxbury et al., 1987; savage, 1988;). beyers (1995) suggests that virtual organizations need virtual management which implies fewer boundaries, more interaction supported electronically, and increasingly direct communication from one to another without the traditional organizational structures (i.e., moving through a structure to do what must be done). managers' roles will change as traditional, hierarchical methods are no longer appropriate (snell, 1994; jenner, 1994). lucas & baroudi (1994) suggest that this creates misgivings about supervision in the typical manager. in short, the literature suggests that operating in a remote environment will bring different management challenges and perhaps require a different management style. operating on a basis of trust between the manager and employee will become more important, as will the need for focusing more on the results achieved rather than the process used. managers will also face challenges managing the corporate culture and corporate politics which results from remote working arrangements. phase 1 exploratory research research methodology focus group interviews were chosen as the method to gather data. focus groups were conducted with both remote managers and remote employees. in order to obtain homogeneous participants (krueger, 1989), where possible, separate focus groups were conducted with manager participants and manager employees. focus groups were conducted in three modes due to logistical needs: face-to-face, teleconference, and combined (some people were face-to-face while others phoned into the meeting room). conducting focus groups via teleconference made it possible to reach participants over a much larger geographic area than would have been possible if only face-to-face focus groups were used. five organizations participated in this study, three of which were private sector firms (i.e., hewlett packard (hp), northern telecom (nt), imperial oil (imperial)), and two of which were public sector organizations (i.e., ministries of provincial governments in canada). these five organizations had a 83 ajis vol. 4 no. 2 may 1997 large variance in the information technology (i.e., ranging from the telephone to groupware and videoconferencing) used to manage remotely and work remotely which was advantageous for the study. managers were selected who had employees who reported directly to them and who worked from a different location than the employee. employees were selected who worked in a different location than their manager. almost all of the employees worked in different cities than the city where their manager had his/her office. in addition, when it was possible, remote managers and remote employees who had a reputation for either managing or working effectively in a remote environment were specifically targeted. this was done since data were being collected on best practices. table 1 summarizes the number and type of participants. a total of 104 people participated in nineteen focus groups, split fairly evenly between remote managers (58 participants; 56%) and remote employees (46 participants; 44%). sixty percent (n=63) of the participants worked in canada, thirtyseven percent (n=38) worked in the united states, and three percent (n=3) of the participants worked in england. the managers and employees were not paired. the participants carried out a wide range of job functions. table 2: summary of focus group participation organization hp nt imperial ministry 1 ministry 2 totals focus group participants managers 19 10 5 13 11 58 employees 17 l 11 9 9 0 46 total 36 21 14 22 11 104 number and type of focus group face-toface 3 0 0 2 1 6 teleconference 5 4 1 1 0 11 combined 0 0 2 0 0 2 total 8 4 3 3 1 19 phase 1 findings • key issues regarding working remotely and managing remotely each participant in the focus groups identified the top three issues from their perspective and ranked them in descending order. the key issues were examined to eliminate any duplication and then grouped into categories. this analysis identified 13 relatively distinct categories of issues which are presented in table 2. many of the issues within the various categories are interrelated and table 2 does not indicate any ranking of the issue by the focus group participants, just that it was identified as a key one. the category that had the most key issues identified in it was the communications category. fourteen of the nineteen focus groups identified some communication aspect as a key remote management issue. typical communication issues included: keeping remote sites in the communication loop and having an equal information transfer for remote and non-remote sites; how can you be confident that you will get the required information about critical issues; the communication skills required; the loss/lack of informal contact; how to replace the non-verbal signals lost by less face-to-face communications; how to get timely access, both for managers to employees and vice versa, and timely response to messages; communications between manager and employees and between peers and management for sharing ideas and keeping in touch; how to deliver negative feedback remotely; and, communications to ensure that the roles of everyone in a work group/team are clear. the second most frequently mentioned category of key remote management issues (68% of the focus groups) pertained to information technology (it) equipment, support and training. it was clearly found to be a key enabler of effective remote management and many focus groups had suggestions for improvements in their organizations. participants from the organizations that were less technologically advanced in their it infrastructure were more likely to identify it key issues than were those from the it advanced organizations (i.e., nt and hp). information technology issues identified by focus groups included: the lack of information technology equipment and support; that more effective information technology was needed to support communication efforts; and, the need for an infrastructure to support technical and logistical support needs. teamwork issues and leadership/coaching issues were the next most frequently mentioned key issues. this highlights the difficulty of working with and communicating with people who are located in different locations. some of the issues identified dealt with: how to coach and lead remotely; how to develop employees remotely; and, how to building relationships remotely and a feeling of being part of a remote team. 84 ajis vol. 4 no. 2 may 1997 table 3: summary of key remote management issues in the five organizations issue categories communications information technology leadership/coaching teamwork performance management travel time/family balance differences across sites isolation career development/ recognition new employees remote environment brings different mg't challenges administrative support training number of focus groups that identified key issues in the category* hp 50% 63% 75% 38% 63% 63% 38% 13% 25% 38% 0 25% 0 nt 75% 50% 100% 100% 75% 100% 75% 50% 0 25% 0 0 0 imperial 100% 100% 33% 67% 33% 33% 33% 67% 33% 0 0 33% 67% ministry 1 100% 67% 33% 67% 67% 33% 0 33% 33% 0 100% 0 33% ministry 2 100% 100% 0 100% 100% 0 100% 0 0 0 0 0 0 overall % 74% 68% 63% 63% 63% 58% 42% 32% 21% 21% 16% 16% 16% * the numbers in table 2 show the percentage of times the issue was identified as being a key one in a focus group conducted in that organization. for example, in 50% (i.e., 4 out of 8) of the hp focus groups, someone identified a communication issue as being one of their top three issues (i.e., a key issue). over half the focus groups mentioned issues of performance management, including how to build trust and a relationship between managers and employees. performance management was largely a management issue as it was mentioned in most management participant focus groups. some of the key issues identified by the focus group participants were: how to gather information remotely to carry out employee appraisals, especially softer information; how to build trust between managers and employees; accountability issues including how to measure productivity; how to identify employees who are struggling and help them; and, that managers have to change their expectations regarding accountability. trust is needed that the job is being done. about half the focus groups also mentioned travel time/family balance issues. the distances between manager and employees were very large for many of the participants (e.g., one manager had several employees across the country canada or the us), which created large travel times for face-to-face meetings between the manager and employee or for the work group/team. the time burden this travel imposes on a schedule can be severe. also, for work-at-home employees, balancing family and work life can be an issue since it can be difficult to separate the work environment. slightly under half of the focus groups identified key issues caused by differences among sites. these ranged from problems caused by working across different countries or regions which had different languages, cultures and laws, to different standards being adopted at different sites, to the perception that different organizational cultures likely develop at different sites. a key concern in this category among hp employees was how to maintain their strong corporate culture so that it was consistent across the organization. issues dealing with employee career development and lack of recognition, hiring and training new employees for remote postings, recognizing that remote management is different and it brings different management challenges, training, and administrative support were all identified by a few focus groups as key remote management issues. lack of administrative support was a key issue for work-at-home employees. table 2 presented the key issues for both the managers and employees combined. further analysis was done to determine if there were differences between manager's and employees' key issues. a number of differences were found. managers were more likely to identify key issues dealing with 85 ajis vol. 4 no. 2 may 1997 leadership/coaching, performance management, and new employees. this is not surprising given that the issues are associated with tasks that would normally be thought of as management responsibilities. however, employees were not oblivious to these issues. half of the employee focus groups identified key issues dealing with leadership/coaching, and performance management. employees were much more likely to identify key issues dealing with information technology, travel time/family balance, isolation, and administrative support. this is not surprising given that these issues more strongly effect the employee than the manager. in most cases, it is the employee that is isolated from others in the organization and who suffers from the lack of administrative support. the employee depends heavily on it and feels the lack of support for it more severely than a manager who has a corporate office, although 45% of the manager focus groups identified it issues. while both managers and employees brought up travel time/family balance issues, employees did so more often. this likely reflects that either the employee is doing the majority of the traveling (i.e., going to headquarters) or the employee is working at home which may create family/work tensions. the frequency of identifying issues dealing with communications, teamwork, differences across sites, career development/recognition, different management challenges caused by the remote environment, and training issues were fairly similar for both managers and employees. phase 1 findings best practice suggestions after identifying the key issues in each focus group, discussion centered on practices that were or could be used to address the issues. a lengthy list of the best practices and ideas were generated from the focus groups. the major items are summarized below. it was suggested that a high degree of trust between manager and employee facilitates remote work. trust can be enhanced through communications and developing a relationship. therefore, the stronger the personal relationship, the higher the trust and the more communications, the higher the trust. frequent communications are necessary to maintain a feeling of being informed and part of the organization for an isolated worker. therefore, the more frequent the communications, the higher the employee commitment and involvement. due to the remote environment, communications often have to be electronic. therefore, the more information technology (it) available, the higher the employee effectiveness. new remote workers need contact with experienced personnel in order to understand and adopt the organization's policies, expectations, and culture. therefore, the more time spent in training with others, or in a buddy/mentor relationship, the stronger the match between the remote worker's perception of corporate climate/culture and practices and the non-remote worker's perceptions. designating champions and resources can help the new person identify more quickly experienced people who can help in certain areas. the remote work environment adds to the standard work burden, due largely to travel since some face to face communication is needed. this creates work/family conflict and work stress, due to working longer hours. focus group participants suggested that travel should be seen as a last resort. use all other forms of communication technology first and make sure that the travel is absolutely necessary. establishing expectations regarding the amount of face-to-face time that will be available and for what, is very valuable. the amount of face to face communication required decreases as the manager and remote worker build experience and build a relationship. therefore, since the reduced need for face to face communications reduces travel demands, work/family conflict and work stress decreases as experience increases and trust builds. other ideas and best practices suggested in the focus groups included: • it is hard to monitor and/or measure productivity in a remote work environment. expectations should shift toward a discipline/results oriented focus. • responsibility should be delegated effectively (i.e., set parameters, objectives, expectations and priorities and give the resources needed). • trust is needed in coaching so that workers feel safe in asking for help. the manager must also be accessible and available. • face to face contact is needed for developing relationships and dealing with negative situations. • managers should value and support social activities and team building activities which help build relationships. • administrative support helps, especially for work at home (wah) or isolated remote workers (i.e., working on their own). • the remote worker needs to be organized and needs certain skills to be effective. these include: time management skills, ability to set objectives, adequate typing skills, office 86 ajis vol. 4 no. 2 may 1997 management skills, and the skills to use whatever it tools they have available. initiative and self-motivation are also critical traits for effective remote workers. • standardization of policies/practices across sites will reduce the chance for confusion and perceptions of unequal treatment. • career development and recognition: an employee's visibility is less when they are out of the head office so they have to work at being seen. managers should recognize that recognition can be a problem and give recognition where it is deserved. • managers should support training and make it part of the performance evaluation process so that the employee also takes responsibility for it. development of hypotheses from the findings of phase 1 and suggestions in the literature the role of information technology was identified as a strong enabler by most of the focus groups and by various authors in the literature. the level of connectivity provided to remote people was shown to be very important. people that were not well connected wanted more capabilities including voice-mail, electronic-mail, groupware, and the perceived ultimate capability, videoconferencing. participants that were well connected realized the value of it. as one participant put it, it was "their lifeline" to the rest of their work group and the organization. therefore, examining the level of connectivity and its impact on the employee's perception of remote work and productivity is warranted. hi: higher levels of connectivity will positively impact the remote worker's attitude toward working remotely. the focus group findings also suggested that more frequent communications reduced feelings of isolation in a remote worker which increases organizational commitment. since communications often have to be electronic in a remote setting, this suggests that high levels of connectivity, which enables electronic communications, may lead to higher levels of employee commitment. h2: frequent communications will increase the remote employee's organizational commitment. h3: higher connectivity will increase the remote employee's organizational commitment. both the literature and phase 1 research suggested that trust between the manager and employee is an important ingredient since face-to-face supervision is limited. trust is developed through effective communications, both formal and informal, as well as through mutual performance. therefore: h4: higher levels of trust between the manager and employee will lead to a more effective remote work environment. consistent with the literature, maintaining or developing an appropriate corporate culture in a dispersed work environment was found to be a key issue with many of the participants. this was especially true in companies that viewed their corporate culture as a valuable asset. replacing the informal sharing of values and stories that occurs naturally when people are physically together has to be replaced in a virtual setting by explicit efforts that will typically fall upon the manager's shoulders. therefore, it may be that remote employees develop different perceptions of the organization's culture. h5: remote employees will have a different perception than non-remote employees of the organization's corporate culture. the extra burdens imposed by remotely working suggests that remote workers will have higher job stress than non-remote workers. the focus group research also suggested that this job stress declines as experience increases and trust between the manager and employee increases. therefore: h6: remote employees will have higher job stress than non-remote employees. h7: the job stress of remote employees will be less for more experienced employees. h8: higher levels of trust between the manager and employee will lead to lower levels of job stress for remote employees. phase 2 testing the hypotheses research methodology data were gathered from both remotely-managed employees and non-remotely-managed employees in a large canadian insurance company via a questionnaire. both remotely-managed and locally-managed employees received the same questionnaire. all recipients of the questionnaire received a reminder letter two weeks after the initial mailing of the questionnaire. after six weeks, 123 out of 184 questionnaires had been returned for a response rate of 66.8%. trust, organizational climate, job satisfaction, organizational commitment, and job stress were measured in the questionnaire with existing scales taken from the literature that had demonstrated 87 ajis vol. 4 no. 2 may 1997 acceptable psychometric properties in previous studies. trust was measured using an 11 item scale developed by mcallister (1995) which captures two dimensions of trust: affect-based trust and cognition-based trust. fineman's (1975) job climate questionnaire (25 items long) was used to measure organizational climate. a measure of organization climate was chosen over a culture measure since organizational climate was seen to fit better with this study. organizational climate has a somewhat shorter time frame (relatively enduring) than organizational culture (highly enduring) and climate is more practice oriented, operating at the level of attitudes and values (moran & volkwein, 1992). warr, cook and wall's (1979) 15 item scale was used to assess job satisfaction. the nine-item short form of mowday, steers and porter's (1979) organizational commitment questionnaire scale was used to measure organizational commitment. new scales were developed to measure the rest of the required measures. degree of connectivity was measured by asking the respondents which of the following information technologies they had: voicemail, e-mail, groupware collaborative system(s), videoconferencing. six items were used to collect information on remotely-managed employees perceptions of the effectiveness of working remotely. information on the frequency of communication was gathered by asking respondents to indicate how frequently, on a five point scale ranging from "never or almost never" to "daily", they used six different media (face-to-face meetings, written correspondence, telephone, e-mail, groupware, and videoconferencing) for seven different activities (i.e., asking for and receiving coaching feedback, performance feedback, other information, and staying in touch with the manager). scale items were checked by carrying out structured interviews with remote employees. a pre-test of the questionnaire was also carried out with experts in questionnaire design and with remote employees. analysis of variance (anova and manova) techniques were used to test the hypotheses. anova was used where there was only one dependent variable. in the other cases, manova, which is a technique for analyzing differences between group means in situations where there is more than one dependent variable, was used. manova results are assessed in two steps. first, the overall test of significance is examined which takes into account the intercorrelations of the dependent variables. if the overall test is significant, the dependent variables can then be examined individually. if an individual item's f-test is statistically significant, then there are differences between the groups for that dependent variable. results of phase 2 a summary of the characteristics of the respondents follows. about two-thirds of all of the respondents were males. the average age was 39 years and over four-fifths of the respondents were living with a spouse or significant other. two thirds of the respondents lived with dependents. about one-third of the respondents had an undergraduate degree or graduate degree, while about one quarter had only finished secondary school. about 45 per cent of the respondents had been with the company 11 or more years. three-quarters of the respondents had been in their present position three or more years. the respondents had worked for their present manager an average of 2.5 years. seventy-five per cent of the respondents were non-managers (i.e., had no people reporting to them). only twenty percent of the respondents worked at home with the vast majority of these indicated that it was easy for them to so. eighty-seven percent of the respondents felt that remote management was a growing practice in their organization. seventy-three percent of the respondents had voicemail, e-mail, and groupware capabilities. twenty-two percent only had voicemail and e-mail, while four percent had only voicemail. slightly over half of the respondents were remotely managed (n=64) with the average distance between the respondent's office and their manager's being 309 kilometers. for the most part, the demographic characteristics of the remotely-managed respondents were similar to those of the locally-managed respondents. as would be expected, more remotely-managed respondents worked from their home which meant that on average, the remotely-managed respondents had a shorter commute time. the remotely-managed respondents had, on average, longer tenure in the organization while the locallymanaged respondents had worked for their present manager longer. the positions held by remotelymanaged employees appeared to be similar to those held by locally-managed employees. as shown in table 3, only three of the hypotheses had significant results. hypothesis 2 was supported. respondents in the top half of the responses regarding frequency of communications with their manager had significantly higher levels of overall organizational commitment. hypothesis 4 was also partially supported. while trust had no significant impact on remote employees' perceptions of the effectiveness of working remotely, trust did significantly affect the remote employees' levels of job satisfaction. individuals with high levels of trust had significantly higher job satisfaction. the individual items in the job satisfaction scale that were significantly different are listed in table 4. in all cases, higher satisfaction lead to higher organizational commitment. all four items are satisfaction with issues that deal with management. 88 ajis vol. 4 no. 2 may 1997 hypothesis 8 was also supported. employees that had higher trust between themselves and their managers had significantly lower job stress. the other hypotheses were not supported. neither high trust nor the level of connectivity significantly impacted remote workers attitudes toward remote work or their job satisfaction. connectivity also did not impact organizational commitment. remote workers did not have significantly different levels of stress nor different perceptions of organizational climate than did the non-remotcly managed employees. greater experience, either with the organization, with the current position, or with working remotely, did not significantly reduce the job stress perceived by remotely-managed employees. table 4: a summary of the results of the analysis of variance hla h l b h2 h3 h4a h4b h5 h6 h7a h7b h7c h8 hypothesis description impact of connectivity on remote worker's attitudes toward remote work impact of connectivity on remote worker's job satisfaction impact of frequency of communications on organizational commitment (remote workers only) connectivity impacts organizational commitment (whole sample) impact of trust on perceptions of remote work impact of trust on perceptions of remote workers' job satisfaction remote workers have a different perception of organizational climate remote workers have higher job stress greater remote work experience reduces job stress greater tenure in organization reduces job stress greater experience reduces job stress trust reduces job stress f-statistic wilks lambda f = 0.65 f(2, 60) = 2.47 f(l ,61) = 6.56 f(2, 119)= 1.32 wilks lambda f = 1.49 wilks lambda f = 2.90 f(l, 119) = 0.31 f(l, 121)= 1.52 f( 1,60) = 0.33 f(4,58)= 1.75 f(3,59)= 1.18 f( 1,62) = 0.33 significance level d = .793 e = .093 e = .013 e=.271 e=.198 e = .003 e=.578 e = .220 e = .570 e=.151 e = -325 e = .048 table 5: the significant dependent variables from the manova analysis individual test results * (f(1,57) = 35.29,e<.001) (f(l,57)= 1 8.45, £ < . 001) (f(l,57) = 23.13, e < 001) (f(l,57) = 17.13, e < 001) item description satisfaction with your immediate boss satisfaction with relations between management and workers in organization the satisfaction with the way you are managed satisfaction with the attention paid to the suggestions you make value after bonferroni adjustment discussion the importance of developing trust between the remote employee and his/her manager was supported by the findings. not only did higher trust in the relationship lead to higher job satisfaction, it also lead to lower job stress. this implies that managers and employees should work hard at developing a relationship based on trust; some of the suggestions from the first phase of this study may help to achieve this. after the strong suggestions from phase 1 regarding the criticality of it in a remote work setting, it was disappointing that none of the hypotheses associated with the level of connectivity were supported in phase 2; however, there was not a great deal of variance in the level of connectivity among the respondents which meant that the sample sizes for the groups with low connectivity was small. repeating the study with a larger sample that has greater variance in the it used would be necessary before drawing any conclusions regarding the impact of it on remote worker's perceptions of remote work effectiveness and organizational commitment. also, the support for the hypothesis that frequency of communications leads to organizational commitment does indirectly support the suggestions that it is important in a remote setting since communications are enabled by it in this environment. the small sample size of the second phase of this study limited the power of the statistics and suggests that the findings be interpreted as preliminary until the study can be replicated with a larger sample. another limitation of this study was that the participants involved in the first phase of the study were 89 ajis vol. 4 no. 2 may 1997 not a randomly selected group. instead, they represent early innovators and individuals who were interested enough in the topic to commit their time to it. therefore, the ability to generalize the findings may be limited. again, further studies will help to determine the generalizability of the findings. in addition to future questionnaire-based studies with larger sample sizes, more qualitative research, such as longitudinal case studies, would be valuable. while questionnaires lend themselves to quantitative analysis, case studies would gather richer, deeper information. this information would be valuable in examining issues such as the role of informal communications in a remote management setting. the results from this research study makes a contribution, from a practitioner point of view, by identifying key issues regarding remote management and offering suggestions to deal with the issues. the findings will also help guide future research by identifying important aspects of remote management to focus on and suggesting several hypotheses for future research. for example, issues such as the use of information technology, communications, and the role of trust, are all important remote management and remote work issues that warrant further study. references beyers, m. (1995) is there a future for management? nursing management, vol 26, issue 1, pp 2425. bleecker, s. e. (1994) the virtual organization. futurist, vol 28, issue 2, pp 9-14. desanctis, g. (1984) attitudes toward telecommuting: implications for work-at-home programs. information & management, vol 7, issue 3, pp 133-139. duxbury, l. e., higgins, c. a. & irving, r. h. (1987) attitudes of managers and employees to telecommuting, infor, vol 25, issue 3, pp 273-285. fineman, s. (1975) the influence of perceived job climate on the relationship between managerial achievement motivation and performance, journal of occupational psychology, vol 48, pp 113-124. freedman, d. h. (1993) quick change artists, cio, vol 6, issue 18, pp 32-28. gerber, b. (1995) virtual teams, training, april, pp 36-40. greengard, s. (1994) making the virtual office a reality, personnel journal, september, pp 66-79. hamilton, c. (1987) telecommuting, personnel journal, vol 66, issue 4, pp 90-101. handy, c. (1995) trust and the virtual organization, harvard business review, june, pp 40-50. illingworth, m. m. (1994) virtual managers, informationweek, june 13, pp 42-58. jenner, l. (1994) are you ready for the virtual workplace? hr focus, vol 71, issue 7, pp 15-16. judd, c. m., smith, e. r., & kidder, l. h. (1991) research methods in social relations, toronto: holt, rinehart and winston, inc. judkins, p., west, d. & drew, j. (1985) networking in organizations: the rank xerox experiment, gower. krueger, r. a. (1989) focus groups: a practical guide for applied research, newbury park, california: sage publications. lucas, h. c., jr. & baroudi, j. (1994) the role of information technology in organization design, journal of management information systems, vol 10, issue 4, pp 9-23. mcallister, d. j. (1995) affectand cognition-based trust as foundations for interpersonal cooperation in organizations, academy of management journal, vol 38, pp 24-59. metzger, r. o. & von glinow, m. a. (1988) off-site workers: at home and abroad, california management review, vol 30, issue 3, pp 101-111. moran, e. t. & volkwein, j. f. (1992) the cultural approach to the formation of organizational climate, human relations, vol 45, pp 19-47. morgan, g. (1988), chapter 7: developing skills in remote management. riding the waves of change: developing managerial competencies for a turbulent world. san francisco: jossey-bass publishers. mowday, r. t., steers, r. m. & porter l. w. (1979) the measurement of organizational commitment, journal of vocational behavior, vol 14, pp 224-247. mowshowitz, a. (1994) virtual organizations: a vision of management in the information age, the information society, vol 10, issue 4, pp 267-288. nilles, j. m. (1985) teleworking from home, the information technology revolution, forester, t. & blackwell, b. (eds.). o'hara-devereaux, m, & johansen, r. (1994) gobalwork: bridging distance, culture, and time, san francisco: jossey-bass publishers. olson, m. h. (1988) organizational barriers to telework, in w. korte, & w.j. robinson, (eds.), telework: present situation and future development of a new form of work, north-holland. 90 ajis vol. 4 no. 2 may 1997 phelps, n. (1985) mountain bell: program for managers, office workstations in the home, washington, dc: national research council, national academy press. roderick, j. c. & jelley, h. m. (1991) managerial perceptions of telecommuting in two large metropolitan cities, southwest journal of business & economics, vol 8, issue 1, pp 35-41. savage, j. a. (1988) california smog fuels telecommuting plans, computerworld, vol 22, issue 18, pp 65-66. snell, n. w. (1994) virtual hr: meeting new world realities, compensation & benefits review, vol 26, issue 6, pp 35-43. walsham, g. (1994) virtual organization: an alternative view, the information society, vol 10, issue 4, pp 289-292. warr. p., cook, j. & wall, t. (1979) scales for the measurement of some work attitudes and aspects of psychological well-being, journal of occupational behavior, vol 52, pp 129-148. 91 ajis special issue december 2002 ajis editorial the information systems discipline is still relatively new and, unlike many more traditional areas of study, concerns aspects of the world that exhibit rapid and continuous change. there is continuing debate with respect to the identity of the discipline, its relationships with other disciplines and its research paradigms. the papers in this special issue focus on an area that has perhaps received less critical attention that it warrants the question of the fundamental knowledge that underlies the information systems discipline. the aim of the issue is to examine some of the theoretical foundations of information systems and the implications of candidate bodies of theory for practice. authors were encouraged to prepare papers that were stimulating, speculative and would encourage debate. in asking questions about the theoretical foundations of our discipline we are asking questions about its essence the form and content of the knowledge we contribute to the world. it is in attempting to answer these primary questions that we have the potential to place our discipline on a more solid base than is currently the case. as speculation was encouraged, we have some papers that may raise more questions than they answer. nevertheless, in each paper there are valuable insights into elemental problems. we have papers that are metatheoretical they examine the nature and types of information systems theory, the form and justification for knowledge statements, and definitional aspects of theory. in addition, authors have nominated specific candidate theories as important for the information systems discipline: including karl popper's ontology, the bungewand-weber ontology, actor-network theory, cultural-historical activity theory, philosophical theories of ownership and property, and theories of agency. the papers selected to appear in this issue originated in contributions to the workshop "information systems foundations: building the theoretical base" at the australian national university, 1-2 october, 2002. we thank the participants in this workshop, the reviewers, and the sponsors of the workshop: the school of business and information management and the national institute of economics and business. we also thank the members of the plenary panel at the workshop, who gave their views on the critical issues facing the information systems discipline. these panel members were professor graham pervan, (curtin university), ann steward, (principal advisor e-strategy, national office of the information economy), and roxanne missingham (assistant director general, resources sharing division national library of australia). shirley gregor dennis hart school of business and information management australian national university whistleblowing: an ethical dilemma joan k. pierson, karen a. forcht, ben m. bauman information and decision sciences department james madison university harrisonburg, virginia, usa abstract because most organizations depend on computer systems that electronically store important data to perform crucial business functions, the integrity of these information systems is paramount. securing company systems, however, is not always an easy task. more sophisticated systems often provide widespread access to computer resources and increased user knowledge, which may lead to added difficulties in maintaining security. this paper explores whistleblowing-employees' exposing illegal or unethical computer practices taking place in the organization~as a method of computer security and the support for whistleblowing found in codes of ethical conduct formulated by professional societies. introduction whistleblowing is the term applied to the reporting by employees of illegal, immoral, or illegitimate practices under the control of their employers to parties who can take corrective action (elliston 1985). whistleblowing is a controversial organizational issue. on the positive side, whistleblowers can help organizations correct unsafe products or working conditions and curb fraudulent or wasteful practices. whistleblowers may provide a previously underutilized source of information critical in maintaining the performance of large complex organizations (ewing 1983, miceli & near 1985). conversely, whistleblowers may threaten an organization's authority structure, cohesiveness, and public image (weinstein 1979). despite the problems, there is an increased interest on the part of managers in the issue of whistleblowing and how to handle such incidents (bamett 1993, ewing 1983, keenan 1988a, rowe & baker 1984). codes of ethics provide guidance for professionals in such fields as accounting, law, engineering, medicine, and education, as well as in information systems. this paper examines support for whistleblowing found in codes of ethics formulated by computer-related professional societies. attitudes toward whistleblowing most studies on whistleblowing address the topic in a general manner-there are few published reports of research activities that concentrate on computer-related incidents. research in the area of whistleblowing is difficult because the presence of organizational blocks aimed at thwarting whistleblowing also serve to block inquiry from outsiders (parmerlee, near, & jensen 1982). most managers and employees personally approve of the practice of whistleblowing, although managers appear to be slightly less empathetic on the issue than do employees. in a study conducted by keenan, 96% of the employees and 87% of the managers indicated their personal approval of whistleblowing (keenan 1988b). those results are similar to earlier studies (united states merit systems protection board 1981). most managers and employees also believe that whistleblowing is in the best interest of the company. employees were again slightly more positive than the managers about the benefits to a company of whistleblowing in keenan's study. deciding to report wrongdoing it appears that the majority of personnel, regardless of whether they are managers or not, have witnessed some sort of wrongdoing in the workplace (keenan 1988b). most such incidents are not reported. a few research studies have examined factors affecting the decision of an observer to report wrongdoing. ignoring wrongdoing is an expected response since ignoring observed organizational problems is the standard or norm. 58 ajis miceli and near note that the observer's sense that he or she can be manipulated is another factor that reduces the likelihood that the incident will be reported-employees with a sense of powerlessness are not likely to jeopardize their careers. inactive observers, those who choose not to blow the whistle, tend to be low paid, highly educated, supervisory "fast-trackers." they may be more likely than whistleblowers to have "lofty executive ambitions." observers who have impressive evidence of serious wrongdoing are more likely to blow the whistle than those who do not. observers are also more likely to blow the whistle if the observed wrongdoing has a direct effect on them personally than if it does not. miceli and near found no support for the hypothesis that whistleblowers who use internal communications channels for whistleblowing (rather than external channels) are members of organizations that use more extensive methods to communicate information about wrongdoing and whistleblowing than other organizations. it appears that employee decisions on whether to blow the whistle are not affected by their organizations' successful attempts at communication. the study did find, however, that whistleblowers were more knowledgeable of the presence of internal whistleblowing channels than were inactive observers (miceli & near 1985). barnett found a relationship between the existence of formal disclosure policies or procedures and the number of whistleblowing actions (bamett, cochran, & taylor 1993). observers are more likely to blow the whistle if they believe that whistleblowing in general is ethical (hauserman 1986). reactions affecting decisions on whether to report problems in an organization include fear of reprisal, loyalty to the company, privacy and personal control, lack of skills in effective disputing, and a belief that it is pointless (rowe & baker 1984). often, whistleblowing is viewed as a threat to hierarchical authority. policymakers may want to discourage frivolous whistleblowing but not legitimate complaints (parmerlee, near, & jensen 1982). in some organizations interested in exploring legitimate complaints, ombudsman systems have been set up in which a senior executive operating outside the normal chain of command is available to deal with employee grievances and concerns on a confidential basis (brody 1986). the fear of personal reprisal is a common thread running through the published reports of whistleblowing research. whistleblowing statutes with the dual purpose of protecting whistleblowers and encouraging whistleblowing have been passed in many states. they do not appear to be having the desired effects; but they may be having an unanticipated, positive influence on some companies to change their policies, thereby reducing the incidence of cases brought by whistleblowers who make retaliation charges against their employers (dworkin, morehead, and near 1987). whistleblowing in the computer environment in one of the few research studies of whistleblowing in the computer environment, information systems professionals' attitudes toward whistleblowing were found to be similar to those referenced in research on whistleblowing that was not limited to computer environments. ninety-six percent approved of whistleblowing and 92% agreed with the statement that whistleblowing is in the best interest of the organization. other findings indicate that only 58% felt it is possible to protect a whistleblower from reprisal. thirty-three percent responded that protection is probably or definitely not possible. yet the majority of the information systems professionals believed that employees should be encouraged to act as whistleblowers (75%) but that monetary rewards should not be given (73%) (pierson and forcht 1990). nearly 40% of the information systems professionals in the study believe that their organizations do not provide as much encouragement for whistleblowing as is needed; 39% think that the level of encouragement is adequate; 21% stated that they are not sure whether the level of encouragement is appropriate. the respondents were much more confident of their own knowledge about when to blow the whistle on unethical or illegal computer use than they were of the abilities of other employees in their organization. about 66% of the respondents felt confident of their own knowledge about when to blow the whistle, whereas only 20% felt confident about other employees knowing when to blow the whistle. in addition, approximately 67% believe that their organizations do not disseminate enough information about when to blow the whistle on computer misuse (pierson and forcht 1990). ethical standards of conduct one of the disturbing facts brought out in the study on whistleblowing in the computer environment is the evidence of lack of organizational policies or procedures outlining ethical and legal use of computers or the lack of awareness of such policies and procedures. only 38% of the respondents in september 1993 59 the study were positive that their organizations had such guidelines; 42.6% were sure that there were none; the remainder were unsure (pierson and forcht 1990). the lack of guidance evidenced in the study leads to the question of whether the level of computer misuse might be decreased if employees knew what constitutes appropriate and inappropriate computer usage. the need for statements outlining ethical and legal use of computers is two dimensional. first, there is a need for guidance for end users of computer technology. information systems are organizational assets and must be protected in the same way as any other asset. computer users have responsibilities to society and their employers to ensure that information system assets are safeguarded. second, there is a need for guidance for information systems professionals that far surpasses that needed for end users. the responsibilities of information systems professionals include responsibilities to society, their employers, clients, colleagues, and profession. most people consider whistleblowing a last resort, an action to be taken only when all else fails. although the majority of employees and managers approve of whistleblowing in general, they are hesitant to act as whistleblowers. they need guidance in determining for themselves whether or not unethical, illegal, or fraudulent actions have taken place. codes of ethics often provide the needed guidance; some ethics codes contain support for whistleblowing actions. ethical codes for end users employees' decisions are governed by at least five sets of standards: general cultural, company, personal, situational, and industry. each set of standards has an official form, such as that espoused in written documents, and an unofficial form that develops as people use the standards. it is little wonder that employees need guidance to determine what is ethical and unethical behavior with regard to computer usage. but no matter how many written policies and procedures are available for guidance, they will be worthless if the organization tolerates violations of stated ethical behavior (fimbel and burstein 1990). many businesses have formal policies that prohibit unethical conduct and prescribe punishment for it. these policies are typically found in operating and policy manuals and in supervisors' workplace statements (bommer, gratto, gravander, tuttle 1987). the relationship between the existence of company codes of ethics and reduced unethical practices is noted by several researchers (bommer, gratto, gravander, tuttle 1987, fimbel & burstein 1990, vital & davis 1990). lack of awareness of company standards of ethical conduct for computer usage may diminish the effectiveness of the standards. in one study of chief executive officers, over 80% responded that their company did have codes (forcht 1991). yet in another study of mostly lower-level managerial employees, 42% of the respondents were positive that their organization had not formulated conduct standards for employees for ethical computer behavior (pierson and forcht 1990). it may be that such codes do exist but that they are not advertised within the organization. widespread computer usage is a comparatively new phenomenon in private and public organizations. it is to be expected that without appropriate guidelines for and training in computer ethics some employees will unknowingly violate rules of ethics. in one large organization, an ethics program was developed that included both establishment of conduct standards and training. the results were favorable. after several years, employees were generally clear about the standards of conduct and their own responsibility in upholding the standards. corrective action systems were developed and implemented to enforce compliance with the standards. disciplinary and other corrective actions were taken as a result of the exposure of wrongdoing (barker 1993). there does not appear to be any standard code of conduct for employee usage of computers that specifies the employee's obligations for ethical behavior. it may be necessary for individual organizations to look to information systems professional societies for guidance in establishing reasonable usage standards and the obligations of the employee for proper use of computers. ethical codes for information systems professionals currently there are several professional information systems societies that have adopted codes of ethics for their members. it has been proposed that the differences among the codes be resolved and a single, coherent, international code of ethics for the information systems community be adopted. oz (1993) points out that "physicians, lawyers, and engineers have moral responsibilities and know to whom they are responsible. professionals in the information systems field need similar guidance." 60 ajis the three largest information systems professional organizations have ethical codes for their members that provide support for a potential whistleblower. the three organizations are the data processing management association, the association for computing machinery, and the institute for certification of computer professionals. the pertinent guidelines in the three codes are noted below. the data processing management association (dpma) is a worldwide organization. its mission is "to advocate effective, responsible management of information to the benefit of its members, employers, and the business community." two guidelines in its code of ethics and standards of conduct are of particular interest. one refers to the obligation of information systems professionals to their employers to "protect employer's interests at all times." a second cites a responsibility to the professional organization or its members to "take action against others' unethical conduct." the association for computing machinery is the largest professional organization in the information systems industry. the standards of the organization are set forth in the acm code of ethics and professional conduct in which is specified an obligation to the professional organization to "uphold and promote the code" and to "agree to take action to remedy if the code is violated." the institute for certification of computer professionals (iccp) offers certification for information systems professionals. the certificates offered are associate computer professional, certified computer programmer, certified systems professional, and certified in data processing. as an obligation to the profession, the iccp's code of conduct holds members responsible to "report violations of the code; testify in ethical proceedings; and serve on panels to judge." the responsibilities noted in the three ethical codes are clear in their intent: unethical and illegal actions should be reported. conclusion information systems are of increasing importance to organizations. protection of these important assets becomes more difficult as the percentage of employees using information systems increases. in a perfect world, there would be no need for whistleblowers. for now, managers of information systems must utilize all methods available to control risks. whistleblowing is one method. it is unfortunate that the term "whistleblowing" is the one chosen to describe an action taken in good faith by an employee and in accordance with personal and professional codes of proper conduct. however, organizations should be aware of the circumstances in which it is appropriate to report wrongdoing and the long-term benefits of these actions. standards of ethical conduct established by professional computer-related societies can help employees in decisions of whether or not to blow the whistle on improper conduct public and private organizations should ensure that standards clearly setting forth employee obligations for computer usage are adopted and widely distributed. not only do such ethical standards render guidance for determining ethical, moral, and legal behavior, they provide support in case whistleblowing is the only alternative. references barker, richard a. (1993), "an evaluation of the ethics program at general dynamics," journal of business ethics, vol 12, no 3, pp 165-177. bamett, tim, cochran, daniel s., and taylor, g. stephen (1993) "the internal disclosure policies of private sector employees: an initial look at their relationship to employee whistleblowing," journal of business ethics, vol 12, no 2, pp 127-136. september 1993 61 bommer, michael, gratto, clarence, gravander, jerry, and tuttle, mark (1987) "a behavior model of ethical and unethical decision making," journal of business ethics, vol 6, pp 265-280. brody, michael (1986) "listen to your whistleblower," fortune, vol 114, pp 77-78. charney, scott (1992) "the justice department responds to the growing threat of computer crime," computer security journal, vol viii, no 2, pp 1-12. dworkin, terry morehead, and near, janet p. (1987) "whistleblowing statutes: are they working," american business law journal, vol 25, pp 241-264. elliston, f. a., (1982) "civil disobedience and whistleblowing: a comparative appraisal of two forms of dissent," journal of business ethics, vol 1, pp 167-177. ewing, d. w. (1983) do it my way--0r you're fired, new york: john wiley and sons. fimbel, nancie and burstein, jerome s. (1990) "defining the ethical standards of the hightechnology industry," journal of business ethics, vol 9, pp 929-948. gordon, richard (1992) "the long road to responsible computing," computer security journal, vol viii, no l.pp 69-80. hauserman, john l. (1986) "whistleblowing: individual morality in a corporate society," business horizons, vol 29, no 4, pp 28-38. keenan, john p. (1988a) "communication climate, whistleblowing, and the first-level manager: a preliminary study," academy of management best papers proceedings, pp 247-251. keenan, john p., (1988b) "comparing employee and managerial whistleblowing: a preliminary study and evaluation," proceedings, annual national conference of the council on employee responsibilities and rights, october, virginia beach, virginia. malone, david (1993) "the ethical issues of automated information processing," journal of computer information systems, vol xxxiii, no 3, pp 82-84. miceli, m. p. and near, j. p. (1985) "characteristics of organizational climate and perceived wrongdoing associated with whistleblowing decisions," personnel psychology, vol 38, no 3, pp 525-544. oz, effy (1992), "ethical standards for information systems professionals: a case for a unified code," mis quarterly, vol 16, no 4, pp 423-433. parmerlee, marcia a., near, janet p., and jensen, tamila c. (1982), "correlates of whistleblowers' perceptions of organizational retaliation," administrative science quarterly, vol 27, pp 17-34. pierson, j. k., and forcht, karen (1990) "whistleblowing as a computer abuse countermeasure," data security management, new york: auerbach publishers. rowe, m. p. and baker, m. (1984) "are you hearing enough employee concerns?" harvard business review, may-june, pp 27-35. united states merit systems protection board (1981) whistleblowing and the federal employee, washington: government printing office. vitell, scott j. and davis, donald l. (1990) "ethical beliefs of mis professionals: the frequency and opportunity for unethical behavior," journal of business ethics, vol 9, pp 63-70. weinstein, d. (1979) bureaucratic opposition, new york: pergamon press. 62 ajis ajis vol. 6 no. 1 september 1998 reengineering a software review process with the function-based process analysis (fpa) method claude strieker* jintae lee** department of decision sciences college of business administration university of hawai'i abstract with the current popularity and success of the world wide web, the question. how can the www technology improve existing processes, is in the forefront of many managers' mind. this paper examines this question in the context of the software review process and proposes a method that helps us systematically answer it. the method, function-based process analysis (fpa °), represents a process as a function lattice, in which the functions that the process is to serve together with the subfunctions that implement these functions form a lattice. in fpa a technology like the www is also represented as a function lattice. fpa then exploits the uniform representation of both a process and a technology to help us systematically identify and examine the aspects of the process that can benefit most from the given technology. the paper illustrates these claims by applying the method and generating www-based alternatives to a software review process currently employed at a major company. more valuable than the specific alternatives generated, however, is the systematic way that the method provides for examining the relation between a process and a technology as well as the framework in which the relevant issues can be placed. introduction with the current popularity and success of the web technology, any manager responsible for running a given process has probably wondered about how this technology can improve the process. some managers have explored this question and reengineered their processes. for example, seafood credit implemented a web-based system that maintains financial information on more than 50,000 companies across north america and allows its analysts and customers to instantly and remotely access all the credit information (cf. baum (1997)). patients by kaiser can request a non-urgent appointment for a procedure like a cholesterol check, ask confidential questions of a nurse or pharmacist, and search a health-care encyclopedia for anything, directly by connecting to kaiser permanente online web site (cf. kalin (1998)). this reengineering phenomenon is nothing new nor peculiar to the web technology. it happens with the emergence of any new powerful technology like lotus notes, case, or tqm. with new technologies to come in the future, researchers as well as managers will continue to ask the question of how these technologies can help improve their processes. few methods exist, however, that help us systematically answer this question, and one has to rely on inspiration or serendipity for their answers. to be sure, exploring how a given technology can change and improve the existing process will always require good dosage of insights. however, we claim that it is possible to have a systematic method that, given descriptions of an existing process and of a technology, helps identify the list of areas where the technology can be used to improve the process. we claim further that such a method can help generate insights about how they can be improved, provide a checklist for completeness, and support reuse of the analysis process. * email: claude.stricker@epfl.ch ** email: jl@hawaii.edu 26 \\re are aware that sometimes fpa stands for function points analysis (international function point users group '94). however, the acronym fpa will be still used for the method proposed here in order to preserve continuity with our earlier work. 117 ajis vol. 6 no. 1 september 1998 this paper describes such a method called function-based process analysis (fpa) and illustrates how it has helped us think about using the web technology to improve a software review process. fpa represents a process as a function lattice, an extended function decomposition tree with multiple inheritance. fpa also represents a technology like the www as a function lattice. fpa then exploits the uniform representation of both a process and a technology to enable us to systematically identify and examine the aspects of the process that can benefit most or is easier to implement with the given technology. the paper illustrates these features by applying the method to a software review process currently employed at a major company and examine which aspects of the process can be improved by the web technology (in particular, the web document annotator technology). the rest of the paper is organised as follows. the fpa method is described in the next section. section 3 then briefly describes the software review process that has been examined. section 4 illustrates the process of constructing the function lattice representation with the software review process as an example. section 5 describes the www document annotator technology and its function lattice. section 6 describes how the fpa method has been used on the two lattices, of the software review process and of the www document annotator technology, to explore the different ways in which the software review process can be improved with the technology. section 7 concludes the paper with a few caveats in the use of this methodology as well as the current status and future directions of the fpa research. function-based process analysis definition function-based process analysis extends the function decomposition method27 for the purpose of process modeling and redesign. in fpa, no distinction is made among processes, organisations, or technologies. they are all viewed and represented uniformly as a lattice of functions. in the systems analysis context, gibson & hughes (1994) define a function as an "ongoing activity a business performs to further its mission" (e.g. manufacturing). note that "function" is used here not in the mathematical term (input/output) sense, but in the sense of objective or feature. a function can be decomposed into a set of subfunctions (e.g. product research and development, product production, and product distribution), i.e. the functions that help achieve the original function (i.e. manufacturing in our example). each of these subfunctions can then be decomposed into its own subfunctions. a tree that results from this repeated decomposition is called a function decomposition tree (cf. table 1). manufacture product maintain operation maintain parts production operations research and development distribute product market product market analysis advertising research and development table 1. example of function decomposition tree a function decomposition tree provides a good conceptual overview of the organisational activities in support of the overall objectives. creating a function decomposition tree also forces one to be explicit about how a function is currently implemented (descriptive) or to be implemented (prescriptive). 27description of function decomposition method are given in many software engineering methods, e.g. martin, (1990) 118 ajis vol. 6 no. 1 september 1998 fpa extends the function decomposition method in several ways. the extensions include: generalized notion of function, function sharing via multiple parents, introduction of and and or constructs that resolve expressive semantic ambiguities, and additional relations such as specialisation and conflicts that complement the traditional decomposition relation. this paper will describe only the first two extensions, which were used extensively in the project being reported. in fpa, "function" is generalized to include not only activities but also any feature whether it is a desirable feature or an existing feature. in particular, not only market product or advertising is a function, but also is make profit (desired feature) as well as electronic mail (existing feature). in other words, fpa views organisations, processes, technologies indiscriminately as collections of functions because at some level of abstraction they all exist to serve functions even though some of them may be intangible (e.g. promote the feeling of loyality among the employees) or outdated (e.g. circulate physical copies of the monthly reports.). a consequence of this uniform representation is that one can create a function lattice which consists of goals, which are implemented by activities, which in turn are implemented by technological features without worrying about where their boundaries are. also it means that this function lattice could be generated by synthesizing the lattices that represent objectives, processes, and technologies. for example, the activity advertise can be decomposed into internet ad, magazine ad, tv ad that implement this activity. internet ad can in turn be decomposed of the technological features that make it up. table 2 shows an example of a function tree with extended notion of function. market product market analysis advertising internet ad www www server www client www protocol electronic mail internet magazine ad tv ad research & development table 2. an example of function tree with extended notion of function fpa also allows multiple functions to be implemented by a given function. that is, the same function can be a subfunction of more than one function. this multiple "parents" allows us to express the fact that one function can help achieve multiple objectives. for example, if market analysis helps not only market product but also distribute product, then market analysis will be a subfunction of both. this is called "function sharing" in design literature. this function sharing is represented in the lattice by a function linked to multiple parent functions, (cf. figure 1). the lattice representation makes clear any occurrence of function sharing by having multiple links coming out of the node being shared, whereas in the outline representation (cf. tables 1 and 2) function sharing is more difficult to detect because a node being shared is multiply copied where appropriate in the outline. 119 ajis vol. 6 no. 1 september 1998 market product distribute product i advertizing research and development f,market analysis figure 5. function sharing with this extension, fpa helps us: • model a process as a function lattice • model a technology as a function lattice • identify the areas of the process that can be redesigned and improved with the technology. modeling a process fpa models a process by identifying the functions of the process, the subfunctions that implement these functions, and the subfunctions that implement these subfunctions, and so on recursively. this generates a function tree or lattice, whose root consists of high level functions that the process is designed to serve (e.g. producing good quality software for software review process) and whose leaves consists of the most specific subprocesses such as the individual speech of a meeting's participant. a function lattice can be built from top-down and from bottom-up. the top-down construction proceeds by identifying the high level objectives (e.g. detect defects) that the process is to serve and examining how they are implemented with more and more concrete activities. the bottom-up construction proceeds by examining the observable, individual activities (e.g. moderator asks opinion of author) and articulating the higher level functions that they serve. the concurrent use of the top-down and the bottom-up approaches help us not only build a model of the process but also help us to identify the areas in the current process that can be improved, as discussed further in lee (1993). modeling technology a function lattice can also be used to represent a technology because a technology such as www can be viewed as a collection of functions, implemented in terms of subfunctions, which in turn are implemented as subfunctions, and so on. as in the case of process modeling, a technology can be modelled top-down by identifying the high level functions that the technology provides (e.g. access documents anywhere anytime) or bottom-up by identifying the purpose of an existing feature (e.g. http protocol). identifying the areas of the process for improvements in order to identify the areas that can be redesigned to benefit from the given technology, the two function lattices representing the process and the technology can be matched systematically. that is, for each function in one lattice, one asks whether one can find a similar function in the other lattice. if so, then one can examine whether the 120 ajis vol. 6 no. 1 september 1998 subfunctions that implement this similar function can be used to implement the function in the first lattice. if the alternative implementation offers any advantage, then one marks the current subprocess as a potential area for redesign. section 6 illustrates this step with examples. although a typical procedure is to start with the subprocesses (i.e. subfunctions) that are known to be problematic or inefficient (e.g. too much searching time of relevant documents), this step can be in principle performed for each of the functions in the lattices. software formal review process in the organisation we have studied, the quality assurance standard is implemented through a software formal review process adapted from pagan (1976) and freedman & weinberg (1990). the goal of the review process is to prevent models, concepts, design and codes from major problems by detecting and correcting defects before the product tests. it consists of a methodical inspection by reviewers of different types of artifacts (code sources, specifications, data models, etc.). the reviewers are usually experts in the domain of the artifact and are selected outside the project team. in the first stage, the reviewers try to detect potential defects by reviewing privately the artifact usually with a checklist. then they review the artifacts again, but this time in a group meeting. the review process organizer plays the role of a moderator, guiding the discussion between the reviewers. the goal of the meeting is to discuss the potential defects, agree on their seriousness, and produce an official report stating which defects should be corrected. the process terminates with the corrections produced by the author and approved by one of the reviewers. other important goals of this formal review process are to develop a better understanding of the work products and to improve the skill of all the review participants. the main steps of the software review process are shown in figure 2. the first step is preparing the resources for the next steps. the main input resources to the reviewer's preparation, besides the reviewers, are the artifacts to be reviewed and the list of checkpoints. an essential input resource to the group meeting is the potential defects, detected in the preceding step by the reviewers. these potential defects are discussed during the review meeting. review 1 organization 1 reviewer's preparation review group meeting findings figure 6. main functions or chronological steps of the formal review process and their input/ouptut. software review process as a function lattice before we began building the function lattice, information about the goals, steps and activities of the process were gathered by looking at the document standards, by interviewing the process participants, by collecting questionnaires, and by observing a meeting recorded on video tape. the lattice has been built by using concurrently the following top-down and bottom-up strategies. 121 ajis vol. 6 no. 1 september 1998 top-down strategy the most general goals of the process are placed at the top levels of the lattice. in figure 3, the most general goal improve quality of software products is decomposed into prevent models, concepts, design and codes from major problem, develop better understanding of the work products and improve developers' skill. these goals have been directly extracted from the gathered information or from discussion with the people. each of them is then decomposed down further to the level of activities implementing them. in some cases, a subgoal or a subtraction serves more than one function. for instance review artifacts formally shown in the diagram of figure 3 serves the three goals detect defects before testing, develop better understanding of the work products and improve developers' skill. as discussed above, the lattice allows the representation of the function sharing as multiple parents. curved lines indicate that the subfunction is an alternative way to implement its parent. dots indicate that the decomposition includes more subfunctions, but they are not represented in the diagram for space reason. improve quality of software products prevent models, concepts, design and codes from major problem develop better understanding of the work products detect defects before testing i i re view artifacts i • ' formally organize process detect defects c determine store defects j seriousness of . _ ' i defects . report resutls figure 7. top goals and major functions of the software review process at the lowest level, activities directly observed during the review meeting are placed. these activities can be as detailed as individual speeches of the participants. figure 4 shows how the function determine seriousness of defects is decomposed down to the level of activities performed during a review meeting 122 ajis vol. 6 no. 1 september 1998 nnotatlonjn ; -voluniirvous' 'srutact: figure 8. detailed activities' decomposition of the meeting bottom-up strategy in conjunction with the top down strategy, a bottom up strategy has been applied in the construction of the lattice. in some cases, observed activities could not be attached anywhere in the function lattice created during the top-down strategy. for example, one of the activities observed is that the moderator asks opinion of author. however this new activity can be placed in the existing lattice only if two intermediary nodes, moderator checks author's understanding and check validity of findings, are introduced to make clear the function of this activity (cf. figure 5). 123 ajis vol. 6 no. 1 september 1998 explain potential delect to the articipant moderator asks, opinion i of author • i figure 9. bottom-up construction of the lattice the final lattice of functions contains more than 300 nodes, which have been validated and refined based on the feedback provided by the people involved in the process itself. in the course of building the function lattice, fpa helped us identify potential problem areas by forcing us to articulate functions at appropriate levels of abstraction. for instance, it was noticed in the recorded meeting that reviewers had difficulty retrieving findings. they spent much time browsing through their voluminous artifacts in order to find the location of written notes and then trying to remember what their notes said. this observation was captured in the lattice as the function reviewers retrieve annotation in voluminous artifact as shown in figure 6. at first, no parent function seemed to be served by this function. however, because we knew that this function exists to serve the reviewers remembering the defects found earlier, the parent function reviewers remember findings was added under the pre-existing function announce potential defects. this newly identified function then served later as a starting point for exploring alternative ways of remembering, (cf. figure 6). 124 ajis vol. 6 no. 1 september 1998 / \ store delects \ / f explain ̂ potential delect to tke larticipant, moderator asks opinion ol author figure 10. placement of function at the appropriate level of abstraction fpa also helped us to define the desired level at which metrics could be computed. for example, a metric for the meeting management overhead could be computed by summing over the time duration associated with the following functions: articulate defects in group discussion, check validity of finding and classify defects (cf. figure 7.). the total time associated with the time function, it turns out, was 75 % of the total duration of the 2 hours meeting. it means that a quarter of the time of the meeting was devoted to the function conduct meeting, which contained the activities of meeting management and moderation. 125 ajis vol. 6 no. 1 september 1998 meeting 25% of total duration of the meeting determine in group the seriousness of defects •— n duct figure 11. a sublattice can help define a process metric by revealing the appropriate level at which individual submetrics could be aggregated. w3 document annotator (wda) technology as a function lattice in order to illustrate how the fpa method can be used to model a technology, w3 document annotator groupware system built by schenk (1995) has been modelled, it also provides the annotation technology that is necessary for the web-based software review process. however, other group annotation technologies28 could have been chosen for illustration here equally as well. wda is a system that supports group annotations on the world wide web documents. annotations may be linked to any specific part of a document and may themselves be further annotated. based on the "argumentation model" gibis of conklin and yakemovic (1988) and sybil of lee (1990), it allows a group of people to enter and retrieve annotations linked to any pre-defined part of a www document. in particular, a user can propose a topic of discussion by issuing proposals. the participants can respond to each proposal with arguments that either support or refute the proposal. arguments can be supported or countered with another argument and so on. the user can browse through the list of annotations and their corresponding arguments, and access their content displayed in a separate www document. figure 8 portrays a part of the lattice of functions constructed for wda, following the strategies described earlier in section 2 and 4. 28 see for instance: wit (w3 interactive talk) of ari luotonen at http://www.cern.ch/wit 126 ajis vol. 6 no. 1 september 1998 *„.... ~\ r ~\ t \ ( \ /.̂... n arguments of others j select argument in the list ol arguments select annotations in the list ol annotations fill argument w3 form select an argument position in the w3 form communicate by e mail directly trom within the w3 document store html document on a w3 server select e.mail address displayed in thew3 document figure 12. the function lattice of the wda technology www-based formal review process solution when the function lattices of both the process and the technology have been constructed, the next step is to find if certain functions in the process lattice can be replaced or implemented by the functions in the technology lattice by systematically comparing the two lattices. first the matching process is described, followed by the description of the matching results. matching process first, for each function of the review lattice, the wda lattice is searched29 for a similar function, or for a function which could implement it. a similar function is an action that is described by the same verb or by a synonym, a hyponym or a hyponym of the action's verb used in the review lattice. for example, the software review process function reviewers argue is similar to the wda function argue about an annotation (cf. figure 9). f 29 the wda lattice is searched systematically by following a tree traversal method, i.e. by following each branch of the lattice from the top nodes, down to the detailed subfunctions. 127 ajis vol. 6 no. 1 september 1998 ..software review process classify the seriousness of defects define precisely the defects announcers potential; •.delect* j explain potential defect to the participant reviewers remember findings review the\ v document in < group with \ wda j \ argue .about an annotation view annotations of others wda technology figure 13. matching software review functions with wda functions another example is the function announce potential defects which is currently implemented by the functions reviewers remember findings and explain potential defect to the participant. it could be implemented by the wda function view annotations of others, if the reviewers would annotate the potential defects in the html document, in order that it can be retrieved and displayed to the other reviewers. alternatively, for each function describing the wda system, the software review lattice is searched for similar functions or for functions that could be potentially implemented by the wda function. for example the wda function annotate the w3 document could potentially implement the software review function annotate artifact, as shown in figure 10. 128 ajis vol. 6 no. 1 september 1998 software review process ' f n( ( organize process j \ x wda technology figure 14. matching wda functions with software review functions in many cases, one need to fully understand the context of the current function, before marking it a potential alternative. for instance, before deciding that view annotations in the wda lattice can be used as an alternative implementation of the software review function announce potential defect, one needs to know that reviewers propose their finding based on their previous annotations and that announcing a finding is made in the context of reviewing a work product or "document". this context information can be obtained by examining the parent nodes of the process. for example, one can examine the parent nodes of review artifacts formally, shown in figure 3, to make sure that the objects of the review are those that can be automated. however a large part of the comparison process still has to rely on the analyst's knowledge of both domain. successful use of the fpa method usually requires that the analyst be the same one who build the function lattices to ensure that the analyst is familiar with the domains as well as the ways in which they have been represented. in order for the revised lattice to work, however, it needs to make sure to incorporate not only the nodes implementing the present function but also all the other nodes required by any of the nodes being imported from the technology lattice. for example, the function annotate the w3 document is a potential alternative to the software review function annotate artifact for implementing the parent function store defects. however, simply replacing this function (along with all of its subfunctions) for the annotate artifact would not work because it also requires the function, edit html annotation anchor in the document, which appears in another branch of the wda lattice, i.e. under the function prepare html document (cf. figure 11 and figure 8). 129 ajis vol. 6 no. 1 september 1998 consequently one must also place the latter function in the revised lattice. in selecting an adequate place under which to place this function, one must first look for a function which could be implemented by prepare the html document. in the present example, the function organize process is a good candidate because it contains the activities of collecting and distributing the material to be reviewed and can be implemented by prepare the html document. software review process r a cm» o| 1 ktnra html document on a w3 server wda technology figure 15. placement of prerequisite functions in some cases, one has to create intermediary functions between an existing parent function and the function to be placed, in order to better represent the rationale of the new function or to better represent alternatives composed of a group of functions. in figure 12, determine seriousness of defects asynchronously is created between determine in group the seriousness of defects and review document with wda. 130 ajis vol. 6 no. 1 september 1998 software review process review artifacts formally ( organize process determine seriousness of defects determine in group the seriousnes of delects determine seriousness of defects in lace to-face group meeting determine •; seriousness of '"defects }". asyhchronously wda technology review the document in group with wda artoculate defects in group discussion argue about an annotation annotate a w3 document view annotations of others view arguments of others figure 16. adding functions for better understanding alternative www-based solution a number of possible redesign alternatives have been already mentioned briefly while describing the matching process. two of them are described below in detail: select artifact and detect defects processes. 131 ajis vol. 6 no. 1 september 1998 one of the software review functions that can benefit from the www technology is select artifacts to be reviewed (cf. figure 13). the subfunctions of select artifact to be reviewed show that the coordinator must find the information about the risks and the potential defects associated with the artifacts. currently this is usually done by contacting the people such as project managers who would have such information. this need for human contact often cause delays the whole review process. in examining the wda lattice for an alternative, one finds the function access documents from anywhere at any time, suggesting the possibility of eliminating this source of delay. the relevant information can be listed in a web page and updated by the project managers. other web pages could contain the status of the monitored risks and the task assignments. the review coordinator can then access these documents and get the needed information asynchronously. another feature of the www-based technology appears in the www lattice as the subfunction email address embedded in www document serving the function communicate by email directly from inside a www page (cf. figure 13). many of the activities of the review coordinator appear in the review lattice as communication between him and the process participants (reviewers and author). this function of communication could be potentially served efficiently by the www communication feature, where a shortcut between the documents and information exchange with the participants is easily implementable. another software review function that can be improved by the wda technology is detect defects. in the software review lattice, the function detect defects is implemented in two ways (cf. figure 13). on the one hand, possible defects are found during a private examination by the reviewers before the group meeting. on the other hand, potentially remaining defects are discovered during the group meeting by discussing the previously found defects. these two subfunctions are very different from a logistic perspective, but both contribute to the same goal detect defects. detecting defects during the group discussion, in turn, is implemented by the functions announce potential defects and store defects. announce potential defects is implemented by the reviewers explaining during the meeting the findings that they have previously annotated in the artifacts. store defects, in turn, is implemented in two different ways. firstly the reviewers annotate the artifacts during their private examination before the group meeting. secondly the classified defects are recorded during the meeting. in the wda-based implementation of these functions, the entire process becomes asynchronous and simpler due to the ubiquity that the www technology provides. moreover the functions appearing in the wda lattice suggest that the storing of the findings as well as their announcements and discussions (argumentation and counter-argumentation in favor of or against the official classification of a defect) could be alternatively implemented by the wda functions annotate the w3 document and argue about a selected annotation as shown in figure 13. each reviewer would go through the complete set of annotations created by all the reviewers and react by entering arguments and counterarguments. 132 ajis vol. 6 no. 1 september 1998 software review process review artifacts formally determine seriousness of defects detect defects i determine in ] group the •, senousnes of i v "•'•«« )detect findings> during the group discussion j reviewers delect findings i privately i determine seriousness of defects asynchronously i choose artifacts ] with high potential of defects get list of artifacts^ and authors ' names get project risk status verity artifacts against i checkpoints ask project manager commun/cats '<hnctlytrom classify seriousness of i defects ) define precisely the defects announce potential delectselect e.mail address displayed in the w3 document 'about an. annotation store html document on a w3 server wda technology figure 17. partial view of the redesigned software review process with wda technology 133 ajis vol. 6 no. 1 september 1998 conclusion this paper describes a method for systematically exploring the ways in which a given technology can improve an existing process. it shows how a process can be modelled as a function lattice and how the construction of the lattice helps identification of potential problem areas. it also shows how a technology such as the www document editor can be represented as a function lattice. with both a process and a technology represented in the common representation, i.e. function lattice, it shows how a systematic matching between them can help us identify the features of the technology that can implement some parts of the process. whether the actual implementation of the technology will in fact improve the process is an empirical question that depends hot only on the technology alone but also on many other factors such as how it is deployed and integrated in the overall workplace setting. however, this preliminary work has demonstrated the ways in which the method can help generate potentially promising alternatives based on the technology. a few caveats about the use of the method are in order. as mentioned earlier, the method still requires an analyst with good understanding of the process and the technology because constructing and matching of function lattices requires detailed knowledge of the ways in which the functions have been represented as well as the context information that may not be explicit in the lattices. secondly, one has to be careful with the dependencies that hold among the process steps. a function lattice representation is not expressive enough to capture all the dependencies that may hold among the subfunctions or subprocesses. as mentioned briefly in section 6, when considering whether a set of functions can implement a given function, currently one needs to check manually to make sure whether there are other functions on which this set depends. if so, one needs to add the additional functions and consider the implications. in malone et al. (1993), we explore a representation that makes explicit these dependency relations so that a system can help explore the implications. the third caveat is that so far in the discussion of fpa, it is assumed that a technology such as the web comes in individual features. but sometimes a given technology has to be accepted as a whole or not at all. it implies that even though some part of a technology may help improve the current process, adopting it as a whole may introduce more problems, apart from the usual training problems, because of the other features that bring with it (e.g. security, incompatibilities). the fpa work is continuing in several directions. first additional ways to support the process of comparing function lattices, are explored. a generic taxonomy of functions is articulated, in order to facilitate the search, creation, navigation, and matching of function lattices. in this taxonomy, a new function is created as a specialisation of an existing function and inherits the attributes of the parent function (such as the agent, objects, and instruments, associated with the function). these attributes, in turn, can be used to acquire relevant information from the user in characterizing a new function or searching in existing function. although the use of fpa has been described in exploring how a given technology can improve an existing process, its use in evaluating which of the technologies should be most appropriate for a given process is also being explored. for this purpose, a library of function lattices representing different technologies such as the www, the lotus notes, and other groupware tools, is created. one may then use the same matching procedure as described in this paper not against one technology but against all the technologies in the library and see which of them produce the best result. the fpa can also be used as a framework for comparing and contrasting a given group of processes or technologies that reveals what is generic and different about them. this use of fpa, however, requires, in addition to the decomposition relation, the specialisation relation which was not described in this paper and is a topic of another paper. acknowledgement: the authors would like to thank the anonymous reviewers for their useful comments and suggestions. the authors would like to also thank cecile bruellhart, thomas mueller, juerg braun and franz brunner for their important participation in the gathering of the data. the work and the people at the mit center for coordination science have been also influential in the shaping of the ideas underlying this paper. 134 ajis vol. 6 no. 1 september 1998 this work would not have been possible without the funding of the swiss national fund for scientific research (grant no 1214-039627.93), bank of vaud, centre informatique de 1'etat de vaud, credit suisse, unicible, and by the support of the department of decision sciences at the university of hawaii (usa) and elca informatique in lausanne (switzerland). references baum, d. (1997) "intranet politics and technologies" in byte, north-america issue. may, 1997, vol. 22, number 5. page 88f. conklin, e., j. & yakemovic k. c. b. (1991) "a process-oriented approach to design rationale" in humancomputer interaction, the special issue on design rationale. 6 (3-4), pp. 357-392. pagan, m. e. (1976). "design & code inspection to reduce errors in program development". ibm system journal, vol. 15, no 3 freedman, d. p. & weinberg d. (1990). handbook of walkthroughs, inspections & technical reviews evaluating programs, projects & products. dorset house publishing. new york gibson, m. l.. & hughes, c. t. (1994) systems analysis & design. boyd & fraser publishing company. one corporate place. danvers, mass. international function point users group (1994). guidelines to software measurement, release 1.0, international function point users group (efpug) standards, 1994. westerville, oh kalin, s. (1998) "kaiser permanente online" in cio webbusiness magazine. cio communications, inc. <http://www.cio.com/archive/webbusiness/050198_kaiser.html> 1.5.1998 lee, j. (1990) "what's in design rationale?" in human-computer interaction, the special issue on design rationale. 6 (3-4), pp. 251-280 lee, j. (1993) "goal-based process analysis". proc. acm conference on organisational computing 11-93 milpitas, ca malone, t. w., crowston, k., lee, j. & pentland, b. (1993) "tools for inventing organisations: toward a handbook of organisational processes". proceedings of the 2nd ieee workshop on enabling technologies infrastructure for collaborative, ieee computer society press martin, j. (1990) information engineering • planning & analysis, prentice hall, englewood cliffs, new jersey schenk, m.-a. (1995) "w3 document annotator". inforge institute, business school hec, university of lausanne. technical report 5-1995. lausanne 135 ajis vol. 6 no. 2 may 1999 practical and tacit knowing as a foundation of information systems john d. haynes chair of information systems, faculty of business ucol, palmerston north, new zealand i .havnes @ ucol .ac.nz abstract the importance of the human subject (and subjectivity) with its inherent capacity for intuition as a foundational concept for information systems is explored in this paper. the paper considers two philosophical contributions: (1) michael polanyi's distinction between practical and tacit knowing, and (2) rudolf steiner's distinction between intellectual thinking and intuitive thinking. intuition as it arises from subjectivity for an information system is the key to determining the appropriate balance between practical and intuitive considerations necessary for the continued evolution of an information system. given this balance the theme of an information system can emerge. since an information system is a social system bound to technology, it is the tacit capacities of an individual's subjectivity within an information system that allow us to connect with the culture and the wider perspective of decisionmaking in an information system. introduction this paper is concerned with the relationship between human subjectivity and its capacity for intuition in relation to subjectivity and intuition within an information system. it is contended that given that an information system is a social system connected to technology, a fuller understanding of the information system's social aspects, in particular the attributes of the humans that comprise the social system, will generate a better understanding of the information system per se. in particular the work of two philosophers, polanyi and steiner, will be examined in the context of subjectivity and intuition. early in the paper some aspects of heidegger's work are cited in support of polanyi's phenomenology and later in the paper brief aspects of the work of habermas and goethe are enlisted as support for steiner's (and polanyi's) perspective. practical and tacit knowing it is helpful to begin an explication of polanyi's distinction between practical and tacit knowing (also referred to as, respectively, practical and theoretical knowledge2) with a paradox that polanyi3 himself discovered in plato's work the meno. prosch (prosch, 1986, p 96) mentions that: to search for the solution to a problem, plato told us there (the meno), would seem to be absurd, since, if you know what you are looking for, then there is no problem. if you do not know what you are looking for, then you cannot expect to find anything. polanyi maintained that this was a genuine paradox. because, "to see a problem is to see something that is hidden. it is to have the intimation of the coherence of hitherto not comprehended particulars". yet in spite of the apparent contradiction, involved in claiming to be able to see a problem, pointed out by plato long ago, polanyi noted that people have continued for two thousand years to see and solve many problems. what the meno really shows, therefore, said polanyi, is not that knowing is impossible (as indeed plato also thought it did not show) but "that if all knowledge is explicit, i.e. capable of being clearly stated, then we cannot know a problem or look for a solution"4. since we apparently do know good problems that can be solved, knowing a problem must be, he held, a kind of tacit knowledge, like the knowledge we have of a face or a class, a knowledge of which we cannot give a fully explicit account, but which nonetheless does exist. contained in the passage above is a key to unlocking polanyi's distinction between practical and tacit knowing. in the above passage polanyi is giving an example of tacit knowing. for more abundant proof, polanyi says precisely this in his tacit dimension (polanyi, 1967, pp 22-23) "the kind of tacit knowledge that solves the paradox of the meno consists in the intimation of something hidden, which we may yet discover". in my opinion, polanyi clearly maintains a phenomenological stance for tacit knowing, because for him, knowledge is implicit or hidden together with the recognition that it is the subject's intuition that at some level recognises the tacitness of the knowledge. 2 see (polanyi 1967, pp 5,6). 3 i thank frank land for introducing me to the work of michael polanyi. 4 tacit dimension, (polanyi, 1967, p22) 56 ajis vol. 6 no. 2 may 1999 this recognition of the subject's intuition relates to both what the question touches upon and the knowledge contained in its subsequent answer. but it is the knowledge that is touched upon in the question of a problem that is hidden and therefore recognised as tacit, whereas the answer merely makes what is touched upon explicit. in other words, the subject recognises that knowledge or the path or the way to take to solve a problem is, in a very important sense, hidden already in the question that articulates the problem in the first place. for example, as is well known in heidegger's work, heidegger maintained that a question was far more important than its consequent answer. heidegger considered this point from the perspective of phenomenology, that to ask a penetrating question is to already have come to an opinion on its answer. often the very asking of a substantial question is like taking the lid off an almost pre-specified pandora's box since a penetrating question also lays the paradigmatic groundwork, however complex, for its subsequent uncovering (or answering). but polanyi saw the importance of a properly framed question for thinkers in general, or more precisely the values, assumptions and belief systems of individuals (in relation to their subjectivity) taken as human existence in general. we see this emphasis on individuals and their subjectivity again and again in polanyi's work. and we see polanyi's individual is in a paradigm way always grounded in an ethic of goodness or what benefits all individuals, as if the concern for all individuals allows each separate individual intuition to "flow" better. for example webb notes (webb, 1988, p 28) : " [in relation to] human existence for michael polanyi ... the individual is grounded in values and ethic, rather than in logic and reason". why is it that ethics provides a different ground which is more fertile for intuition than to that of logic and reason for polanyi? to uncover the hiddenness within this question we need to turn to polanyi himself. in personal knowledge, (polanyi, 1962, p 267) the answer, already implicit in the previous question, is revealed: our mind lives in action, and any attempt to specify its presuppositions produces a set of axioms which cannot tell us why we should accept them. ... our basic beliefs are indubitable only in the sense that we believe them to be so. otherwise they are not beliefs, but merely somebody's [some one else's] states of mind. this then is our liberation from objectivism: to realise that we can voice our ultimate convictions only from within our convictions from within the whole system of acceptances that are logically prior to any particular assertion of our own, prior to the holding of any particular piece of knowledge. ethics, we deduce from the above passages, that is, our own systems of accepted convictions, from within which we speak with conviction, or what i take to be an instance of a belief in our own beliefs, arise out of the our whole system of acceptances. not from one piece of knowledge; not one reasoned bit; not one logical element or group of logical elements, but the whole system. so what polanyi argues is the ground itself for the development of an individual's system of ethics comes from the whole system, not from one individual element. the power of an individual ethic arises because it is respectful of the whole system of acceptances. the emergence of an ethic for an individual ethic being a belief in the good of things: action that both benefits the individual self and others in that one action is not sidetracked by any individual desire. nor is it sidetracked by any set of instances of selfgratification, but subsumes all of these desires and groups into the whole system of beliefs and takes its grounding from that synthesis. in this way an ethic develops independently of any logical or reasoned process. we do not condition our ethic by analysing it, rather our ethic arises out of the integration of all of our beliefs and desires. so our ethic is distilled and distinctly non-logical; non-reasoned and independent of "intellectual" processes. our ethic retains its capacity for intuition because it is independent of reasoning and therefore emerges out of a deep sense of care. we are all born with this deep sense of care5. the quicker the emergence, i would suggest, the purer is the process of its being grounded and continuation for being grounded. one can imagine a paradigm case of an individual not being able to produce an ethic from which to base intuitions. such a case would arise where certain obsessions and biases of self-gratification precluded a synthesis of the whole of the individual beliefs. a literal view of capitalism would be an example of such a bias. consider now why the question "why does ethics ground intuition and logic and reasoning does not" already contains the seeds of its own answer, or already provides a viewing of the hiddenness of the answer. nor does it depend on the arrangement of words, i argue in polanyian terms that a recognition of the hiddenness is still possible given further re-arrangements, such as: does intuition come from ethics or logic and reasoning? i have indicated that for polanyi ethics arises or emerges from a distillation of beliefs, and that such a distillation must have at some stage brought together both opposing and consistent beliefs. it is the critical nature of emergence that provides the clues to the recognition of hiddenness. polanyi, as webb notes (webb, 1988, p 48) does "discuss a further major concept .. that of 'emergence'." but webb further comments ( .. p 48) that lonergan "makes better use of it than polanyi succeeded in doing". in philosophers of consciousness, webb (webb, 1988, p 78) indicates 51 refer the reader to the work of gary zukav, in particular the seat of the soul (rider books, 1990) and the work of david whyte in the mean aroused poetry and the preservation of the soul in corporate america (currency doubleday, new york, 1994) for an explication of a human's deep sense of care in the context of organisations which is outside the scope of this paper. 57 ajis vol. 6 no. 2 may 1999 lonergan's extension to polanyi's concept of emergence was the recognition that things that are viewed as recognised as having hidden qualities have "proportionate beings ... ( and have) .. 'parallel structures' to our knowing". webb further comments of lonergan's contribution (.. p 79) that by "proportionate being, he says, is intrinsically intelligible because it is precisely that which we are able to inquire into". in other words there is something in the question itself that triggers a response in the system of beliefs. what does this mean? it means that in asking a question for which we can recognise a hidden answer, we are putting forward a question that has already been asked6 at an unconscious level. at this unconscious level the asking has been "sorted out"7 and we, as an individual, are now putting that question forward for the conscious level of our being to apprehend. this is the way tacit knowing is both recognised as hidden and how it makes itself manifest as a process of uncovering what is hidden. a practical way of seeing the point that i am making is in relation to the example that i have explicated as a resolution to polanyi's paradox of the meno, which is both an example of tacit knowing and by the resolution of the paradox a recognition8 of it in action. perhaps another way of seeing the concept of emergence is to consider a situation where rational thinking is argued not to apply. flemons in completing distinctions notes a situation based on taoist principles where rational knowledge is argued to be of no use. consider the following passage (flemons, 1991, p 70): if we are up against a mystery, then we dare act only on the most modest assumptions. the modern scientific program has held that we must act on the basis of knowledge .. but if we are up against a mystery, then knowledge is relatively small, and the ancient program is the right one: act on the basis of ignorance9. in other words, from the above passage the meaning is that in acting from ignorance we are not contaminating our intuition10 with our rational knowledge11. and in this strict sense ignorance can ultimately know more than knowledge. on the other hand, practical knowledge for polanyi does not have any hidden aspects to it. in the tacit dimension (polanyi, 1967, p 32, 33), polanyi states that things for which practical knowledge is appropriate are "less real" than things for which tacit knowing applies. he gives the example of cobblestones as being more real (than tacitly known things) in the sense of being more tangible, but less real in the sense that they have no hiddenness to them. polanyi's view of the objects of practical knowledge align fairly closely with heidegger's concept of a thing. joseph kocklemans in his book on the truth of being reflections on heidegger's later philosophy, had this to say about a third conception of a heideggerian thing that i conclude is close to polanyi's object of practical knowledge (kockelmans, 1984, pp 175,176): the thing is nothing but formed matter; and this conception also holds good for both natural and man made things. this conception accounts for the thingly element we find in every work of art. if an object then is "thingly" it exhibits an absence of hiddenness if and only if it is the pure "thingliness" of the object that we are considering. under such circumstances it is a candidate for being an object of practical knowledge. examples of such practical knowledge would be diagrams entailing information as distinct from the mathematics behind that information (the mathematical process of producing that information). practical and tacit information systems an information system is a social system bound to technology. the practical knowledge of an information system includes its technological elements and all other considerations that are conducive to logical analysis. but insofar as it is a social system it carries the power inherent in the humanity that comprises that social system. the upshot of polanyi's position in relation to human subjectivity is that its capacity for intuition based on tacit knowing allows it to recognise the difference between practical and tacit considerations. without the tacit capacities of subjectivity we would, as a human component of an information system, be nothing but practical (or machine-like) in our decisionmaking. by providing a thorough-going conception of the features and power of tacit knowing polanyi allows an information system to see itself in terms of its own evolvement. the polanyian concepts of hiddenness, emergence and "tacitness grounded in ethics rather than logic" allows us to see a wider perspective for an information system, that like a human being an information system is founded on more than the components that make it up. 6 to ask a question at the unconscious level is to virtually answer it, hence my expansion in the next sentence of the need for the asking to be "sorted out". 7 or more precisely placed within the context of the synthesis of the whole system of beliefs. 8 that is, the "tacit knowing" of the reader. 9 quoted from wendel berry, in home economics, north point press, san francisco, 1987, pp 4-5. 10 which, incidentally, literally means inner-teaching. 11 hence, from footnote 9, we may construe this as outer-teaching. 58 ajis vol. 6 no. 2 may 1999 intellectual thinking and intuitive thinking it seems apt to cross from polanyi to steiner with a quotation from polanyi on goethe, if only because steiner was greatly preoccupied with the work of goethe and polanyi was certainly sympathetic with goethe's general views. polanyi made the point that (polanyi, 1967, pp 79-80): the degree of originality any particular scientist trusts himself to possess should thus determine the range which he will venture to tackle and hence also the range of information which he will unquestioningly accept. goethe wrote that the master proves himself by his restraint. for polanyi, originality then, inter alia, comes from an ability to be able to discriminate "within" information. i suggest that such discrimination is evidence of selecting only that which is appropriate to the theme of the inquiry. i suggest that if a theme can be recognised then meaning can be perceived. the theme of the inquiry is a key to goethe's (by bortoft12) and steiner's distinction between intellectual and intuitive thinking. such a key can be clearly discerned as be-fitting of bortoft's goethe in the following passage (bortoft, 1996, p 57): discovery in science is always a perception of meaning, and it could not be otherwise. the essence of a discovery is therefore in the nonempirical factor in cognition. the recognition that meaning is a primary datum of cognitive experience brings a considerable simplification to the philosophy of science. of course, the meaning in question may be several stages removed from the meaning in everyday cognition, and at a much more comprehensive level. and in steiner too, we see the following passage echoes the nature of meaning and theme in relation to cognitive experiences of the "higher states"13 and the "everyday states"14 (steiner, 1995, pp 182-183): moral efficacy depends on knowledge of the phenomenal world with which one is dealing. this knowledge must therefore be sought in a branch of general scientific knowledge. hence, along with the faculty for moral ideas and imagination, moral action presupposes the capacity to transform the world of percepts without interrupting its coherence in natural law. by "moral efficacy" steiner means a realisation (making real; bringing into being) of meaning, or as he says himself (.. p 182): free spirits need moral imagination to realise their ideas and make them effective ... to turn a mental picture into a reality, moral imagination must set to work in a specific field of percepts. human action does not create percepts, it recasts already existing percepts and gives them a new form. to be able to transform a specific perceptual object or group of objects in accordance with a moral mental picture, one must have understood the laws of the perceptual picture to which one wants to give new form or new direction. a free spirit to steiner is one who is not limited by what others may think nor by what his or her ego dictates. in this way the free spirit is unencumbered, and, given the opportunity to develop an imagination (as the consequence of a special form of action), can, theoretically, begin to draw any contradictory idea or underpinning together to consider. the results of such considerations are then filtered through the "moral" imagination, which for steiner, is always mindful of the good of the individual and the good of all individuals simultaneously. the moral imaginer is therefore able to make real (real-ise) his or her ideas more effectively because they have been given flight by the steinerian "moral" dimension. the upshot for the steinerian system is that a person living according to the practices of a moral imagination, draws to him/herself a community of imagination (although he would not use these terms). it is the very comfort and support of this community that provides the ground for making the fruits of moral imagination more effective. the notion of "percepts" (in the next section) sheds more light on the inner process of the thinking of a moral imaginer and indeed tacit knowing itself. the notion of percepts the bortoftian comment that "the essence of discovery is in the nonempirical factor of cognition" and the steinerian remark that "human action does not create percepts, it recasts already existing percepts and gives them a new form" are a conceptual collage of each other. both use the same concepts but they are juggled in different ways and the effect is that each conception explains the other. when we consider that a percept is the mental result or product of the act of perception, i argue that we are very much reminded of the unconscious side of tacit knowing (as mentioned above) and how it makes itself concretely manifest and thereby uncovers what is hidden. so, in this sense, human action does not create the unconscious side of tacit knowing that is already given "it recasts" what is hidden as existing percepts and gives those percepts a new form. the hidden percept is what bortoft means by the nonempirical factor of cognition. but it is precisely the essence of discovery that resides in the hidden side of 12 who, himself, was a steinerian. 13 "higher states" refer to holistic modes of thinking, for example, composing something original. 14 "everyday" states refer to practical considerations, such as survival needs. 59 ajis vol. 6 no. 2 may 1999 cognition and the concretisation of that particular discovery (or any particular discovery hidden in cognition) is the new form that becomes the empirical result of cognition. furthermore, it is the empirical result of cognition that is then perceived as having meaning, because what perceives it is the nonempirical factor in the observer's cognition and it is this realisation that, for example, "brings a considerable simplification to the philosophy of science". to conceptually reconfigure bortoft's and steiner's remarks we have: * human action does not create the essence of discovery. * why? because the essence of discovery already lies in the creation of percepts. * the creation of percepts is a constantly occurring nonempirical factor of cognition. * human action recasts the essence of discovery and gives it a new visible form. the result appears to be innocent enough: in order to realise a certain discovery that one has made, one needs to perform a certain action that makes what is implicit as a percept explicit as a new concrete appearance of that form. but bortoft and steiner have a more powerful and extensive vista in mind. this seems to be that a special form of action is more powerful than thought itself. the implication is that thought is somehow always there; always in the process of coming to percepts; always coming to the essences of various discoveries. whereas the differences in mankind are revealed in the kinds of actions that they perform that consequently enable them to uncover those percepts; that express an ability to give them a "new form" as a real appearance. so, differences in the abilities of (at least normal) humans are, therefore, not intellectual per se, but are a direct consequence of the active physical process by which humans access the percepts of their cognition; access the mental result or product of their intuitions. for steiner and bortoft, i submit, thought is subservient to a special form of action. in suggesting a distinction between intellectual thinking and intuitive thinking, it is all forms of intellectual thinking that fall short of15 the pervasive creative power of intuitive thinking. eastern thinking this is also born out by a long tradition in eastern cultures. as joseph bogen notes (bogen, 1973, p 124), hindu and buddhist teaching distinguishes intellect (buddhi) from mind (manas) as "rational thought" (intellectual) and "integral thought" (intuitive) respectively. "rational thought" is considered to be far inferior to "integral thought". habermas it seems habermas, for example, would support intuitive thinking over rational thinking, even though habermas is taken to be one of the fathers of rational discourse in the more recent information systems literature (see (hirschheim & klein, 1996)). we see habermas extending rationality to a concept (i.e. beyond rationality) and transcending it even to consciousness, in the following points made by thomas mccarthy (mccarthy, 1978, pp 272, 273): [habermas's] theory of communicative competence is a new approach to a familiar task: to articulate and expound an expanded conception of rationality ... the transcendental critiques of language transcends that of consciousness. from mccarthy we can see that habermas's project is distinctly not a philosophy of science approach. it is clear that habermas's interests are far more towards phenomenology and hermeneutics because habermas, like polanyi and steiner (and goethe), is interested in an "expanded conception of rationality". it is often the case that phenomenology is not associated with habermas. however, as further evidence, i think the following passage from martin jay referring first to benjamin and then to a quotation from habermas, places habermas at the very least in the phenomenological tradition (jay, 1985, p 129): what makes this type of expression and mimesis so attractive for benjamin is its priority to the break between subject and object; in other words, that schillerian motif of reconciliation ... "as odd as this mimetic [benjamin's] theory of language sounds", he [habermas] contended, benjamin is right in assuming that the oldest semantic stratum is that of expression ... with the meanings that comprise this potential, human beings interpret the world in terms of their own needs and create thereby a net of correspondences."16 one of the interesting consequences of habermas wanting to extend the concept of western rationality, is the implication that existing western rationality cannot maintain itself as it is. or further, that it needs to be replaced altogether. personally i do not think that western rationality is finished. i agree with yount (following marsh) that it can be redeemed dialectically. "is the tradition of modern western rationality simply to be overcome or is it to be redeemed dialectically"; "could it be, pythagoreans, that this irrationality belongs to the hypotenuse, and not the 15 but are nevertheless critical to the functioning of human beings. 16 habermas, "consciousness-raising or redemptive criticism", pp 48-49 (no reference provided by jay). 60 ajis vol. 6 no. 2 may 1999 geometer?" (yount, 1992, p 29). in other words, western rationality, which entails the inclusion of western conceptions of information systems, needs to be redeemed dialectically. goethe now consider the implications of the role of intellectual thinking in the following quotation from bortoft (bortoft, 1996, p 58): the recognition of the active role of the intellectual mind gave rise to a philosophy of science which maintained the view that a scientific theory is only a framework which we construct for holding the facts together for our own convenience where in this case it is believed that the facts themselves are perceived entirely by the senses, independently of the theory. the above quote shows why bortoft sees obvious deficiencies in elevating intellectual thinking over intuitive thinking. the quotation also illustrates how an intellectual fact-based approach could not even conceive of the possibility of another form of thinking, such as the intuitive, because there is no ground from which such a possibility could spring. in the following passage bortoft considers the implications of the intuitive approach as a practical alternative, while also establishing some of the implications of the deficiencies in an intellectual approach (bortoft, 1996, pp 66-67): [the] psychological interpretation of goethe's way of access to nature in terms of deautomatization is reinforced by considering the subjective experience of the procedure. through trying to direct attention to the act of looking, we can experience for ourselves the gap which there is between our habitual awareness and the direct experience of what is there in front of us. it is only when the hiatus in experience is overcome that we realise how little we usually perceive directly of the concrete detail of the particular. we usually classify verbally and experience just a vague generality. a striking feature of this attempt to give attention to active looking and exact sensorial imagination is how much subjective resistance it can set up in a person. this in itself is an indication that the orientation which it calls for is towards deautomatization or dishabituation. this subjective effect is an instance of the psychological inertia which has to be matched by a person's own activity if the state of his or her awareness is to change, just as the inertia of a material body has to be matched by a force if its state of motion is to be changed. when consciousness is thus restructured into an organ of holistic perception, the mind functions intuitively instead of intellectually. there is a lot of confusion and misunderstanding about intuition, as if it were something intangible and mysterious. but in fact it is a very clear and precise notion. omstein defines intuition as "knowledge without recourse to interference"17. he links it with a simultaneous perception of the whole, whereas the logical or rational mode of knowledge "involves an analysis into discrete elements sequentially (inferentially) linked"18. he connects the intuitive mind with the holistic mode of consciousness as the intellectual mind is linked with the analytical mode. thus intuition is connected with a change of consciousness, and moreover in a way which can be made quite precise and not just left vague. it now follows that goethe's procedures are practical exercises for educating the mind to function intuitively instead of intellectually, leading to a science which is intuitive instead of organised intellectually. what are the practical exercises that goethe speaks of above? bortoft tells us (bortoft, 1996, p 75) that goethe: learned to read [a] plant in terms of itself, so that the plant becomes its own language, and secondly how it is possible to read the language of animal form so that the animal becomes its own explanation. in view of this, goethe's intuitive way of science can be recognised as a concrete illustration of gadamer's principle of universal hermeneutics that "being that can be understood is language"19. the philosophy of goethe's science can therefore be identified more precisely as the hermeneutic phenomenology of nature. the difficulty between goethe's phenomenological way and the mainstream of mathematical physics from newton onwards was summarized memorably and concisely by cassirer: "the mathematical formula strives to make phenomena calculable, that of goethe to make them visible"20. it is now possible to discuss the special form of action that bortoft and steiner indicate is essential for engaging intuitive thinking. in phenomenology we move with the concepts that are pointed to in a written piece of phenomenological literature. the same sense of movement applies as a project for bortoft and steiner. except for bortoft and steiner it is a theme that is being actively looked for, without any interference of thinking about what 17 robert e. ornstein, the mind field, octagon press, london, 1983, p 24. 18 ibid, p 26. 19(gadamer, 1975, p 474) 20 a.g.f. gode von aesch, natural science in german romanticism, columbia university german studies, 1941,p74. 61 ajis vol. 6 no. 2 may 1999 we are looking for, we simply let the theme, as it were, speak to us and we engage with that "speaking". in the speaking we hear the "language" that gadamer speaks of as the "being" (gadamer, 1975, p 474)! for bortoft and steiner it is the theme of a phenomenon that speaks to us in the language of thought; a language of concepts. the special action is subjective but in order to remain special it is not interfered with by any kind of analysis; by any kind of intellectual thinking about what is being performed. we simply move with whatever is presented to us. goethe gives examples of plants and animals forms becoming their own speaking or language, but any living matter or manner of a work by man or woman potentially holds conceptual themes that can be comprehended in a like manner. perhaps an example from architecture will help. a doorway for any type of building can be appreciated for itself as announcing a forthcoming interior. that announcement is implicit until someone moves through the doorway. the more important the building, the more effective should be the announcement (note the case of certain religious buildings where one is forced by the restriction in the height of the doorway to stoop or bow one's head to enter). but if a person was preoccupied with some matter (other than being with the experience of the doorway) while walking through the doorway, the effect of the announcement of the forthcoming interior would be lost or at the very least reduced. if the person was engaging in moving through the doorway; experiencing it as a shift in space and experiencing the design of the architecture, then the implicit announcement "hidden" in that design would announce itself in an explicit conceptual way as an experience to that person. consider another example from the area of interior design. as a person gets older often the memories of when a familiar article of clothing was last worn begins to fade or even the memory becomes unimportant. imagine then a walk-in wardrobe that contained pictures of that person in happy times wearing articles of clothing that were still available for wear in that wardrobe. the possibility that i am suggesting is that with prompts or cues even dressing in familiar clothes can be an experience of uninterrupted subjectivity: the clothing can potentially "speak" to the wearer of past experiences that the wearer can experience via those clothes. perhaps the photos can be placed outside the walk-in wardrobe. in this case the person in question might go into the wardrobe mindful of some memory and it could be the memory that speaks to the person of the choice of clothing to wear. implications for information systems in all three cases above, i try to convey (1) a sense of being at ease with what is being done or experienced, and (2) exploring a set of circumstances which allow for a heightening of awareness of what is being experienced as it is being experienced (and thereby staying in that experience). it is circumstances like these that bortoft and steiner argue although they do not use these examples set in train intuitive thinking. the more often we stay in an experience of an outside phenomenon and let that experience "speak" to us, the better we are able to access our primary internal phenomenon, namely our percepts, thereby letting the fruits of our intuitive abilities, also "speak to us". the same phenomenon occurs in an information system, if we allow logical analyses of an information system to dominate our thinking then we are not free to explore other avenues of expanding the information system. one way of appreciating a sense of expanding our consciousness in relation to an information system is to consider the concept of the culture of an information system. one clear way in which employees fail to contribute their full potential as a cohesive social unit to the information system is where individuals lack a sense of belonging to the information system. this feeling of an absence of belonging leads to a fragmentation of the social unit and consequently diminishes the potential contribution each employee can make. it needs no reminding that corporations lose large portions of their potential net profits each year as a result of employees who do not understand how they "fit" into the "structure" of a corporation. this alienation is passed on to customers of the corporation through the inept actions of the alienated employee. by understanding intuitive thinking a culture can be established and allowed to emerge in an information system. the culture of an information system is the way in which the theme of an information system (the particular products which are developed in relation to its inherent social system) makes itself manifest. conclusion for both polanyi and steiner the implications of the above distinctions for an information system point to the view that an information system is potentially thematic. that is, each information system, if it is allowed to develop as a culture, has its own theme. for polanyi practical knowing can never be any more than collections of facts, whereas tacit knowing is able to comprehend both itself and practical knowing, which means that tacit knowing comprehends the essence or themes of things. for steiner intuitive thinking is the language of thought or concepts and as such is distinctly thematically based. the decision-makers of an information system will not discover the theme of an information system in a logical way; no amount of reporting on profits and losses or efficiencies and failures will discover that theme. the theme is only apprehended by intuition. the absence of the discovery of the theme of a particular information system also entails the absence of an effective capacity to determine how that 62 ajis vol. 6 no. 2 may 1999 information system will react to powerful changes. in short, without a comprehension of its own theme an information system cannot effectively evolve. references bogen, joseph e. (1973) the other side of the brain: an appositional mind, in the nature of human consciousness a book of readings edited by ornstein, robert e , w.h. freeman and co. san francisco, pp 101-125. bortoft, henri (1996) the wholeness of nature goethe's way toward a science of conscious participation in nature, lindisfarne press, hudson ny, usa. flemons, d.g. (1991) completing distinctions, shambhala books, boston, usa. gadamer, h-g. (1975) truth and method, translated and edited by barden, g and cumming, j, seabury press, new york. hirschheim, rudy and the rationality of value choices in information systems klein, heinz k. (1996) development, foundations of information systems, electronic journal, http://www.cba.uh.edu/~parks/fis/fis.htm jay, martin (1985) habermas and modernism, in habermas and modernity. bernstein, r.j. (ed), the mit press, cambridge, mass, usa, pp 126139. kockelmans, joseph, j(1984) on the truth of being reflections on heidegger's later philosophy. indiana university press bloomington, usa. mccarthy, thomas (1978) the critical theory of gurgen habermas. mit press cambridge, mass, usa. polanyi, michael (1962) personal knowledge towards a post critical philosophy. the university of chicago press, usa. polanyi, michael (1967) the tacit dimension. anchor books, double day and company, garden city, new york. prosch, harry (1986) michael polanyi a critical exposition. state university of new york press, usa. steiner, rudolf (1995) intuitive thinking as a spiritual path: a philosophy of freedom, anthroposophic press, hudson, ny, usa. webb, eugene (1988) philosophers of consciousness polanyi, et al. university of washington press. usa. yount, mark (1992)uncapitalising on radical hermeneutics, modernity and its discontents, fordham university press, usa, pp 23-44. 63 the australasian journal of information systems (ajis) began in 1993 and was based on the simple realisation that there was a need for an academic ‘outlet’ for is research in australia australasian journal of information systems volume 14 number 2 june 2007 43 a socio-pragmatic constructionist framework for understanding quality in process modelling jan recker school of information systems queensland university of technology brisbane, australia email: j.recker@qut.edu.au abstract quality is one of the main topics in current conceptual modelling research, as is the field of business process modelling. yet, widely acknowledged academic contributions towards an understanding or measurement of business process model quality are limited at best. in this paper i argue that the development of methodical theories concerning the measurement or establishment of process model quality must be preceded by methodological elaborations on business process modelling. i further argue that existing epistemological foundations of process modelling are insufficient for describing all extrinsic and intrinsic traits of model quality. this in turn has led to a lack of holistic understanding of process modelling. taking into account the inherent social and purpose-oriented character of process modelling in contemporary organizations i present a socio-pragmatic constructionist methodological framework for business process modelling and sketch out implications of this perspective towards an understanding of process model quality. i anticipate that, based on this research, theories can be developed that facilitate a more comprehensive and adequate evaluation of the ’goodness’ of a business process model. keywords: process modelling, epistemology, pragmatism, constructionism, model theory introduction recent years have seen an increasing popularity of methodologies, techniques and tools within the context of business process management (bpm). in fact, improving business processes continues to receive attention amongst chief information officers throughout industry (gartner group, 2007). the proliferation of bpm as an organizational paradigm has triggered substantial academic contributions aiming at advanced business process management solutions. one prominent example mailto:j.recker@qut.edu.au� australasian journal of information systems volume 14 number 2 june 2007 44 in this context is the increased popularity of business process modelling (davies et al., 2006). due to a strengthened interest in a more disciplined approach for business process management, many organizations have been motivated to make significant investments in process modelling initiatives, which in turn has triggered substantial related research. many studies have shown the relevance of process modelling to bpm initiatives, e.g., (davenport, 1993). process modelling denotes a requirement for a number of iso 9000 quality programs (ould, 1995) and is the basis of processrelated it implementations, such as enterprise resource planning systems (robinson and dilts, 1999) or process-aware information systems (dumas et al., 2005). the recent introduction of legislative frameworks such as the sarbanes-oxley act (nielsen and main, 2004) further contributed to the increasing interest in business process modelling. from a practitioner perspective, business process modelling is a way of capturing and graphically documenting how businesses conduct their operations (curtis et al., 1992). it typically includes depictions of at least the activities, events/states, and control flow logic that constitute a business process (scheer, 2000). process models are used, amongst others, for the purposes of business improvement, documentation, automation and simulation. from an academic perspective, process modelling resides in the research field of conceptual modelling. conceptual modelling is the process of building a representation of selected semantics about a domain of interest for the purpose of understanding and communication among stakeholders (mylopoulos, 1992; wand and weber, 2002; siau, 2004). it is arguably inevitable for the tasks associated with information systems analysis and design (karimi, 1988; garda et al., 2004) and has, correspondingly, repeatedly been proposed to reside at the core of the is discipline (weber, 2003; nelson et al., 2005). traditional forms of conceptual modelling accounted only for an organization’s data and, if at all, that portion of its processes that interacted with data. newer uses of information systems, however, extend deployment beyond transaction processing into communication and coordination, viz., a process-aware perspective on information systems (dumas et al., 2005), which gave rise to a heightened interested in the conceptual modelling of business processes. the ongoing and strengthened interest in process modelling has, over time, led to a wide range of process modelling techniques (also called languages) since carl petri first published his initial ideas on petri nets in 1962 (petri, 1962). available approaches range from simple flowcharts and typical business modelling techniques to advanced variants of petri nets with high expressive power. consequently, a competitive market is providing a plethora of complementary tools and methods (sinur, 2004). the juxtaposition of methodologies, methods, and modelling techniques leads to confusion: are there really so many different ways to analyze and design information systems let alone business processes? where are similarities in the approaches? which one serves a particular purpose the best? these and similar questions reveal relevance not only to academics but also to practitioners (olle et al., 1991). but why bother? is it not acceptable to leave evaluation to evolution? contemplating about the history of the is discipline, it is advised that a darwinist approach (“survival of the fittest”) does not seem to be a satisfactory option. first, it does not allow for an exante selection of competing alternatives since it leaves evaluation to the final stages of development or deployment. second, it does not allow for individual appropriateness criteria since it focuses on general “best practice” solutions that seem suitable in principle rather than in context. clearly, there is a need for rigorous theory to assist the development, usage, and evaluation of process modelling activities in order to enhance the quality of the approaches. simply speaking, the question can be reduced to: what constitutes a ‘good’ business process model? surprisingly, this question remains not only unanswered yet but is, in the form “what constitutes a ‘good’ conceptual australasian journal of information systems volume 14 number 2 june 2007 45 model?”, a research field that has only recently begun to emerge and that is slowly gaining momentum. to date, research on quality in conceptual modelling is still believed to be in its infancy (moody, 2005; nelson et al., 2005). what holds true for conceptual modelling in general (buhl and heinrich, 2005), must be stressed even more for the field of business process modelling: several researchers explicitly state the need for research aiming at understanding and developing a common notion of business process model quality (brito e abreau et al., 2002; wand and weber, 2002; poels et al., 2003; moody, 2005; krogstie et al., 2006). before this background, the research discussed in this article aims at arriving at an understanding of business process model quality. more precisely, the research presented in this paper seeks to countervail the lack of a theoretically and methodologically sound and comprehensive appreciation of process model quality. the basic assumption of this paper is that a better understanding of process model quality can only be developed if business process modelling firstly is appreciated from a methodological perspective. in short, this paper argues that a methodology of process modelling is needed that exhibits sufficient explanatory power to address all those traits of quality that are relevant to contemporary process modelling activities in both practitioner and scholar communities. in this context, i make the same distinction as wilson (2002) between methodology and method. that is, methodology precedes method and is more fundamental in that it provides the philosophical groundwork for methods. sadly however, evidence shows that there has been comparatively little research done on the nature of methods and methodology (rescher, 1973). accordingly, in this paper, i will outline a fundamental framework in terms of a methodology and epistemology of business process modelling. this will open the stage for some fundamental criticism of our scientific understanding of concepts in the nexus of bpm research in general and conceptual process modelling in particular. the contention of methodological and epistemological aspects in the framework will set the stage for the design of compliant methodical technical theories for investigating quality on the quality of business process modelling. accordingly, in this paper i seek to discuss considerations towards an understanding of process model quality by shedding light into methodological and epistemological aspects of process modelling. in the remainder of this paper i proceed as follows. in the next section, i briefly recapitulate previous research in the field of process model quality. then, i introduce an alternative perspective upon organizational reality, socio-pragmatic constructionism, which i deem promising as a starting point for this endeavour. in the fourth section i apply this perspective to the conception of process modelling to discuss implications of this theory towards an understanding of models. next, i sketch some implications of these elaborations towards an appreciation and understanding of process model quality in a basic framework. this paper then closes with a discussion of the conclusions drawn from this work and directions for future research. the research approach chosen in this paper is that of conceptual/philosophical research, in particular that of critique (in the kantian understanding). this research method is dedicated to identifying, scrutinizing and questioning the presuppositions of research approaches in order to determine their scope, applicability, possibilities and limits towards a given research objective (kant, 1929). i will hence provide philosophical-logical arguments rather than empirical ones. however, the arguments will (where applicable) also refer to empirical research results, for instance (chen and hirschheim, 2004) and others. furthermore, i will present additional evidence by giving examples from is research practice. australasian journal of information systems volume 14 number 2 june 2007 46 related work little research has comprehensively investigated the notion of business process model quality (moody, 2005). work related to this field mainly stems from investigations upon conceptual model quality in general. noticeable is the work of lindland et al. (1994) who developed an understanding of conceptual model quality based on semiotic theory (morris, 1971), defining a syntactic, semantic and pragmatic level of model quality. more recently, krogstie et al. (2006) revised this framework to address further aspects pertaining to the modelling of processes. their approach will serve as a reference for my subsequent elaborations on the implications of a socio-pragmatic constructionist process modelling methodology towards an understanding of quality. early attempts that investigated process model quality include the guidelines of process modelling (becker et al., 2000), a framework that defines six general guidelines, correctness, relevance, economic efficiency, clarity, comparability and systematic design. this approach proposes the differentiation of different abstraction layers of quality assessment. on a first layer, for instance, generic general modelling guidelines are suggested. these guidelines are then refined for certain views, e.g., models for business processes, and finally broken down to fully specified guidelines for certain modelling techniques (e.g., event-driven process chains). the approach, however, lacks a sound theoretical methodology, and provides only limited empirical proof as to its feasibility as a quality framework (rosemann et al., 2001). evaluation of process models has, to a certain extent, gained popularity since the emergence of quality frameworks with a focus on the representational capabilities and expressive power of process modelling techniques. such frameworks have been developed either inductively from observable practice or deductively from applicable theories. an example for inductively derived frameworks is the set of workflow patterns developed by van der aalst et al. (2003). the development of this framework was triggered by a bottom-up analysis and comparison of fifteen workflow management systems, with focus on the expressive power of the underlying process modelling techniques, to outline similarities and differences between the analyzed systems. the evaluation of process modelling techniques, e.g., (wohed et al., 2003; russell et al., 2006; wohed et al., 2006) is based on the assumption that a more complete coverage of the workflow patterns leads to techniques and systems with advanced expressive power. other aspects or purposes of process model quality are neglected. an example for an evaluation framework that has been derived through deductive research methods is representational analysis of process modelling techniques based on foundational ontologies, in particular the bunge-wand-weber representation model for conceptual modelling (wand and weber, 1990, 1993). this theory of representation has been used in over thirty-five research projects for the evaluation of different modelling techniques, including data models, object-oriented models and reference models. it also has a strong track record in the area of process modelling, with contributions coming from various researchers, e.g., (keen and lakos, 1996; green and rosemann, 2000; green et al., 2005; recker and indulska, 2007). rosemann et al. (2006) provide an overview of these studies. essentially, the bunge-wand-weber model proposes a set of semantics representation constructs for models of information systems that modelling techniques should be able to express. as such, these types of evaluations emphasize a semantic level of model quality neglecting pragmatic or syntactic facets. other work in the area of process model quality has suggested more or less comprehensive lists of quality criteria such as soundness (verbeek et al., 2007), structuredness (dehnert and zimmermann, 2005) and others, e.g., (bajaj and ram, 1999; soffer and wand, 2007). these lists denote examples australasian journal of information systems volume 14 number 2 june 2007 47 of the type of research in the bpm that has traditionally focussed syntactic and semantic investigations but lacked an appreciation of further traits of model quality such as pragmatic aspects. in summation, most of the work presented defines methodical approaches towards the investigation of certain aspects of quality in process models. only the bww model approach is based on a sound methodological foundation, which however has at times been subjected to criticism, e.g., (lyytinen, 2006; wyssusek, 2006), especially with regards to its negligence of social and pragmatic traits of modelling quality. as i concur with wilson (2002) that methodical solutions only reveal pertinence to certain methodological presuppositions, there is henceforth a need to investigate underlying methodological foundations of process modelling to arrive at a better understanding which methodical solution is applicable to appreciating process model quality. i will subsequently present a methodological perspective on process modelling that i deem very explanatory and explorative. it will serve as a starting point for an investigation of quality aspects in process models. during this investigation i will refer to aspects of related work whenever appropriate. in particular, i draw on the work of stachowiak (1973) to discuss characteristics of process models, and the work of morris (1971) on a general theory of signs, which has for instance been used in the semiotic quality framework (lindland et al., 1994; krogstie et al., 2006), to sketch implications of socio-pragmatic constructionism on the understanding of process model quality. socio-pragmatic constructionism the following elaborations proceed on the basis of the paradigm debate that has evolved in the is research discipline lately, refer, for instance, to (benbasat and weber, 1996; weber, 2004; gregor, 2006). in particular, traditional positivist research approaches have been subjected to a powerful base of criticism originating from the post-modern turn in the human and social sciences, see (eden et al., 1981; mir and watson, 2000). yet, evidence show that positivism still dominates the area of is research (orlikowski and baroudi, 1991; chen and hirschheim, 2004; mingers, 2004). i will neither re-cite the well-know criticisms nor repeat the debate of positivism versus anti-positivism here. instead i seek to outline an alternative way based on perspectives of social constructionism and pragmatism, which, in my belief, offers powerful perspectives on is research in general and process modelling in particular. information systems research positions itself at the intersection of historically well-established research fields such as management sciences, technology sciences, social sciences, human sciences etc (vessey et al., 2002). yet, the diversity of influential fields is hardly reflected in is studies. only few studies present multiple perspectives upon the phenomenon being studied. despite the social nature of information systems and the fact that contemporary organizations are exposed to continuous and rapid internal and environmental changes, many is research papers still present models and theories as universal panaceas holding objective truth (chen and hirschheim, 2004). an example of this is the use of representational theories based on philosophical ontology as axiomatic reference frameworks for the evaluation of the “goodness” of a conceptual model, e.g., (wand and weber, 1990, 1993), a theory that is coined by the underlying assumption that the underlying model of representation itself contains necessary and sufficient semantic categories to appropriately represent relevant real-world domains. i argue that two particular developments, namely social constructionism and a recently re-emerging pragmatism, have significant explanatory power as a modelling methodology and hence may pose significant implications to modelling-related research. these developments emerge from postmodern ideas that have for a significant amount of time influenced and guided research in related disciplines such as philosophy (lyotard, 1984), management science (calás and smircich, 1997), organizational studies (chia, 1996) or knowledge management (styhre, 2003). in the following, the australasian journal of information systems volume 14 number 2 june 2007 48 basic principles of social constructionism and pragmatism will be outlined and i will then reconcile these perspectives in the paradigm of socio-pragmatic constructionism. social constructionism to start with the introduction of (social) constructionism, the term first needs clarification. constructionism is closely related to the notion of constructivism. both rely on an anti-positivist epistemology questioning the direct relationship between knowledge (expressed in symbols or models) and reality. constructivism, often referred to as radical constructivism (von glasersfeld, 2001), focuses an anthropocentric approach by claiming that reality, and also knowledge, is constituted as a solipsistic mental construction of an individual. correspondingly, constructivism is concerned with how individuals construct and interpret their world. it insists on the mental states of singe individuals as the sole instance of knowledge creation. consequently, as realities are subjective, so are knowledge, symbols and models. social constructionism, on the other hand, has a social focus and proposes “the redefinition of social realities as constituted through discourse” (neimeyer, 1998, p. 135). it emphasizes the social aspect of cognition. it states in particular that the acquisition of conceptual knowledge takes place before the background of a social community that defines language and action amongst its members. consequently, realities are shaped through social discourse and are not the sole achievement of an individual. berger and luckmann (1966, p. 95) call such a social reality a “symbolic universe”. this notion, even more so than constructivism, appears naturally conducive to an understanding of process modelling. process modelling teams exist in every bpm-affiliated organization. each of these has defined and is and using their own approaches, languages and techniques to conduct process modelling tasks. it would appear sound, then, to assume that the social interaction between the individuals would have an impact on how each one of the individuals perceives organizational reality and describes the knowledge about it in the form of process models. if we assume that reality is constructed socially through investigation, deliberation and discourse, consequently multiple perspectives upon reality may emerge. the role of language (such as the languages we use to create models of reality) in this process must be given special emphasis. perception and interpretation of whatever reality domain is shaped and mediated through language, which defines and restricts our perception of the world and our knowledge about it (quine, 1960). yet, given that in a language community it is still the individuals that construct realities, we have to admit that multiple realities may emerge. the admittance of multiple realities to be constructed in a community through language discourse gives social constructionism a relativist tenet. none of the potential constructions of reality is preferred. multiple pictures of reality are thus equally valid or acceptable. yet, this relativism considerably limits the comparison or evaluation of such pictures, viz., such models, of the domain of interest. consider the case of business process modelling. a team of business analysts and technical systems designers might discuss the way business processes are executed in an organization and seek to develop a model of these processes to present to the ceo or cio. the technical analysts construct a picture of these processes as they are supported through it, while the business analysts conceptualize these processes in terms of inputs, outputs, involved stakeholders and value-adding revenue streams. clearly both pictures denote valid conceptualizations of the universe of discourse in light of the individuals who constructed the models – but which one fits better the internal model of thought that the ceo/cio might have in mind as a pre-conception of the very same domain? the related question is whether both pictures are indeed equal in terms of providing sufficient understanding of the universe of discourse to the target audience. if all pictures or models are partial truths, how can the validity or superiority of a model compared to other models be measured? such are questions social constructionists may be aware of but which have not been addressed with a real answer (morgan, 1983, p. 407). i consequently concur with fellow researchers, e.g., (wyssusek et al., 2001; marshall et al., 2005), australasian journal of information systems volume 14 number 2 june 2007 49 that social constructionism needs to be widened to embrace a means for differentiating, and establishing differences between, the multiple models of reality that may be constructed within one social community. one means of doing so would be to include pragmatic aspects into the discussion. the embracement of pragmatic values within social constructionism opens the stage for incorporating different values and norms as relevant aspects of concerns in knowledge and quality studies. a good reference as to such an approach is the notion of fitness-for-use that builds the cornerstone of total quality management approaches towards quality improvement of products and processes (hradesky, 1994). pragmatism in order to overcome the fallacy of social constructionism to be unable to determine what pictures of reality constitute ‘better’ forms of meaning creation, pragmatism offers a way to deal with the dilemma of relativism. pragmatism was developed in the usa in the late 19th century (wicks and freeman, 1998). its main claim is that the worth of a proposition, theory or model is to be judged by the consequences of accepting it. basically, the tenet of pragmatism is that any picture, theory or model is good or true if and only if it is useful – in the sense of helping people to fulfil a given need. the pragmatic notion of knowledge acquisition, then, is “to gain an understanding which is necessary to deal with problems as they arise” (dewey, 1988, vol. 4, p. 14). pragmatists consequently do not search for universal truths. instead, they agree with social constructionists that all construction of knowledge, i.e., the association of perceptual input to cognitive concepts, occurs before the background of our historically and socially situated pre-understanding of the context. thus, interpretation is contextual, depending on the social environment and the horizon brought to it by the interpreter (gadamer, 1989). in particular, pragmatism offers a criterion of usefulness spread across an epistemological (is this picture credible and reliable?) and a normative (does this picture help us in our actions?) dimension. this enables the researcher to advance the boundaries of positivist and anti-positivist philosophies (wicks and freeman, 1998). the tenet of pragmatism has been reflected in some previous work on conceptual model quality, for instance in the notion of feasibility in the quality framework of lindland et al. (1994), which defines a ‘satisfactory’ threshold for quality aspects. socio-pragmatic constructionism social constructionism as a paradigm for is research in general and business process modelling specifically can in my belief be considerably enhanced in accepting a pragmatic viewpoint upon “truth”. it advances the perspective upon research by providing means of evaluating pictures, theories, propositions, and models in terms of their relevance and usefulness to given values, norms and objectives. hence, it opens the stage for incorporating viewpoints and needs of various stakeholders in a given situation. socio-pragmatic constructionism emphasizes the social aspect of knowledge acquisition and seeks to explain human action and their consequences. this claim roots the creation of conceptual models in a pre-existent social practice of communicating and acting, with the emphasis that the practices of model (or knowledge) creation are never solipsistic. instead, these practices are coined by the primate of the social over the individual and always bound to a certain purpose. the implication of this tenet is that models are created within a community for that community. and this is actually what we see in modelling initiatives. the usage of a shared modelling language and a set of common modelling practices enable its members to build and interpret models for and from each other. imagine the case of a process modelling team within an organization. such team is being trained by an experienced modelling champion that establishes not only common modelling australasian journal of information systems volume 14 number 2 june 2007 50 practices across the newly formed team but also introduces a shared language that leverages a certain understanding of modelling concepts and symbols. the models that are created by this team will be easily interpreted by members of the team. as practical experiences show, however, problems arise once such models are presented to “outsiders”, for instance senior management staff or ceos, who are unlikely to be members of the modelling community and most probably have not participated in the establishment of common language and practice. the extrinsic form of representation chosen in the model may actually be easily understandable. yet, an outsider may have significant problems grasping what is represented. the intrinsic model content has been created within the social community of the modelling team, which established a shared understanding of the content, for instance a set of business processes, through continuous investigation and discourse. a socio-pragmatic constructionist perspective on process modelling the implications of a socio-pragmatic constructionist perspective upon conceptual modelling in general and business process modelling in particular are significant. in the following, i seek to explore the implications in more detail. in order to structure this discussion, i draw on the general model theory (stachowiak, 1972, 1973) as a reference for my elaborations. the general model theory is a well-known reference theory for conceptual modelling that distinguishes three constituent and common characteristics of models independent from any epistemological or ontological commitment, namely representation, simplification and pragmatic orientation. due to its dissociation from any philosophical commitment, it serves as a well-structured framework to discuss common traits of any model. the basic idea is that modelling is essentially just a mapping from a set of original attributes from a phenomenon of interest to another set of modelled attributes. the mapping process can be described as follows (see figure 1). omitted attributes relevant original attributes original phenomenon mapping of attributes superfluous attributes model of phenomenon relevant modelled attributes representation pragmatic orientation simplification figure 1: general model theory (stachowiak, 1972, 1973) australasian journal of information systems volume 14 number 2 june 2007 51 using these three model characteristics, the implications of a socio-pragmatic constructionist perspective upon process modelling can be explored, thereby allowing for the derivation of basic traits of process model quality as will be discussed later in this paper. representation models are models of something (stachowiak, 1973, p. 131). social constructionism states that “something”, i.e., the phenomenon to be modelled, itself is being constituted within the social context in which the modeller operates. to be more concise: the universe of discourse that is subject to being modelled is a social phenomenon embedded into shared social practices and language use. the social community (let’s say, a process modelling team within an organization), contextualizes through established language and common practice the way that the phenomenon of investigation (let’s say a business process within the organization) is conceptualized by the modeller. this internal conceptualization of the original phenomenon is then externalized by the modeller within a model. transferring these insights to the field of business process modelling, the phenomenon being modelled is a set of business processes within an organization. now, obviously, both organizations and business processes are for themselves linguistic conceptualizations – a factual existence of either of them cannot be presumed. rather, they are metaphors for certain perspectives or views upon the universe of discourse, and are thus conceptualized before the background of the established social context. the process model then is subjected to the conceptualization of the phenomenon (a business process) as it is socially contextualized within the community (a business organization). in simple terms, this means that processes are merely metaphors we use within an organizational community to describe the actions and events that happen within the organizational context. and this is what process modellers seek to represent in a process model. the models of the processes then can only symbolizations of what the model creators understands under the concept of a business process, i.e., how (s)he perceives the series of actions and events that take place within the organization. simplification models exhibit a reductive trait in that they map a non-abundant subset of attributes of the phenomenon being modelled rather than the complete set of attributes (stachowiak, 1973, p. 132). from a pragmatic viewpoint, the preference for a subset of attributes being modelled is determined by its relevance for the model designers and/or users. process models may be designed for multiple reasons; yet, in a simplistic fashion there are two main kinds: first, intuitive business process models are used for scoping the project, and capturing and discussing business requirements and objectives with subject matter experts. second, business process models are used for process automation, which requires their conversion into executable languages. these automatable models have higher requirements in terms of expressive power and also a different focus that emphasizes formal rigor rather than intuitiveness (dehnert and van der aalst, 2004). from a social constructionist perspective, the preferred set of attributes is determined by the socially contextualized background of the model designers and users. this implies that their existing ways of pre-understanding process models constitute their model interpretation practices. hence, every single individual looking at a process model will deem it good or bad in respect to how well it fits the interpretation horizon (s)he is able to bring to bear. an it analyst is more likely to judge a model based on its extent of deadlocks than a business analyst, simply because the it analyst traditionally has had this focus when scrutinizing models. put together, socio-pragmatic constructionism stresses the need for the creation of a common horizon of meaning. a shared understanding should be created of what is to australasian journal of information systems volume 14 number 2 june 2007 52 be included and neglected in a model. the ultimate goal of this endeavour remains to be that the model should effectuate shared understanding and successful practice. pragmatic orientation models have a substitutive function in that they substitute a certain phenomenon as being conceptualized by a certain subject in a given temporal space with a certain incentive or operation in mind (stachowiak, 1973, p. 132). the process of creating a model is never for the pure sake of modelling, instead, they are designed and interpreted having a certain purpose in mind – in short, to fulfil a certain need. this statement may appear obvious. yet, recent reflections on current process modelling practice (e.g., recker, 2006; rosemann, 2006a, b) do not get tired of stressing this message to practitioners over and over again. the need for doing so simply stems from the observation that many organizations still invest considerable amounts of time and resources into the creation of process models without being clear about why they are doing it in the first place. for example, a process model can be designed for senior management staff to intuitively conceptualize the operational details of some business operations. or, process models can be designed for systems analysts to determine the extent of compliance of existing or future it infrastructure to the operational business activities. the socio-pragmatic constructionist perspective here again stresses the necessity of forming a shared horizon of meaning (a mutual acknowledgement), in order to arrive at meaningful interpretations of the models amongst various stakeholders. in other words, it is required to establish upfront why the organizations engages in a modelling exercise and for what task the resulting process models will be used as input. implications towards model quality forthcoming from the preceding elaborations i will sketch out some implications towards an understanding of process model quality in a basic framework. i base my elaborations on the understanding that the concept of model quality is inherently complex and cannot completely be appreciated via a correspondence theory in a factual or objectivist sense, let alone in a list of syntactical or semantic correctness criteria. rather, these discussion should incorporate social, contextual and pragmatic variables. models are in their essence merely statements created in some (artificial) language and can hence be related to the study of language and signs. i refer to the semiotic theory of signs (morris, 1971) as a theoretical reference for the subsequent discussion. in is research, several researchers turned to this theory, for instance in theories on data model quality (lindland et al., 1994) or information quality (price and shanks, 2005). correspondingly, it would appear that morris’ theory of signs serves as a suitable starting point for deriving dimensions of a framework to structure my investigation. basically, semiotic theory distinguishes three components within the study of linguistics signs, namely object, representamen and interpretant. the relationships between these three components is in contemporary studies expressed in the so-called semiotic triangle (see figure 2), which originates from the work of ogden and richards (1923). the object (o) is the phenomenon that is to be represented. yet, such object can never be entirely represented but merely an idea of it, the concept of that object. the representamen (r) is some type of language sign, or symbol, which substitutes (stands for) the object being studied in the absence of it. hence, the symbol posits a conventional relation between the represented concept and the phonic or graphical form that represents it. the interpretant (i) establishes the object-representamen relationship by australasian journal of information systems volume 14 number 2 june 2007 53 imposing onto this relationship a perceived meaning. the act of interpretation is necessarily dependant on the use of the sign (θ) by the interpreter, thus on his social context (shaped for instance via linguistic norms and common practice) and on his individual background, viz., his preunderstanding and conceptualization (φ) of the context. hence, there is no direct relationship between a linguistic symbol and some object in the “real world”. the relationship can only be established by the interpretant. in this sense, any linguistic symbol (such as the elements that comprise a process model) can only be interpreted; it cannot signify a priori. based on these relationships morris (1971) defines three semiotic levels that can be used to measure quality: • syntax describes the formal relations between signs. syntactic quality hence assesses the extent to which a representamen is created in accordance to formal laws of the symbolic language. • semantics studies the relation between the representamen and its designatum, i.e., the object it represents. hence, semantic quality describes the extent to which the sign corresponds sufficiently to the concept it seeks to model. • pragmatics studies the extent to which interpretants of a sign are able to make use of the interpreted meaning of the sign for a given use. pragmatic quality then is the sufficiency of a sign to act as a designatum for a certain object or concept before the background of a certain purpose. figure 2: semiotic theory of signs (morris, 1971) taking together the findings from the semiotic theory of signs and the methodological considerations of process modelling based on a socio-pragmatic constructionist perspective, aspects and dimensions of quality can be sketched that need to be considered in the domain of process modelling. the following discussion is structured by the before-mentioned three semiotic levels of quality. syntax the syntactic criterion describes how a representamen is meaningfully formed in accordance to the laws that govern the use of one or several types of representamen. in other words, it is concerned with the grammatical rules that describe how to use certain symbols or elements of a language to australasian journal of information systems volume 14 number 2 june 2007 54 form meaningful statements. translated to process modelling, the syntactic criterion demands that the creation of a process model based on the symbols provided by a process modelling language is accomplished in conformance to the formal laws of the modelling language, viz., its grammar. now, while formalization of conceptual modelling languages is still a rather under-represented concept in is research (ter hofstede and van der weide, 1992), one can generally observe two notions of formalization that facilitate the assessment of syntactic quality. meta model-based approaches rest on the depiction of the grammar of a certain language l1 using another modelling language l2 (note that l2 may equal l1) (holten, 2003). for instance, the grammar of the ebxml process specification language is expressed in uml (oasis, 2001). on the other hand, the syntax of a modelling language (and thus a model) can be formally specified using logical expressions such as first-order predicate calculus or set theory. an example for this is the formal specification of epc models provided by van der aalst (1999). another example is the use of formal petri-net specifications for describing the syntax of process models (van der aalst, 2005). based on the grammar specification, a model can be syntactically verified in the sense that it is assessed whether it denotes a well-formed model in accordance to the formal laws of the symbolic language in which it was created. while in general it would appear safe to assume that syntactically lawful models are desirable, the relevance of this criterion may diverge with respect to pragmatic considerations. for example, a process model that serves as an input to a workflow execution engine has higher requirements in terms of syntactical rigor, for example a verification of the model concerning the possibility of deadlocks or starvation areas. process models that are used to communicate business requirements amongst senior management staff, however, may have less rigid requirements in terms of syntax and may even posit a superiority of clarity and visualization concerns over syntactic rigor. for instance, dehnert and van der aalst (2004) concede that process models used for business purposes demand intuitiveness rather than unambiguity and understandability rather than machinereadability. i hence propose to relate seemingly factual aspects of quality (such as the consistency of language meta model) to social and pragmatic aspects such as, for instance, who is the intended audience and what is the purpose of the model created. this way, it can be established whether certain syntactic quality criteria, e.g., (kiepuszewski et al., 2003; verbeek et al., 2007) should be taken into account or not. semantics the semantic criterion is concerned with the relation of representamen to designatum (the object it intends to represent). in the field of process modelling, this relates to the assessment of the ‘goodness’ of the process model in terms of how well it captures a business process. now, taking the preceding methodological elaborations in consideration, admittedly it is impossible to assess the model against an objectively existent business process. business processes themselves are mere conceptualizations of ‘the way an organization conducts business’. as such, a business process is no more than a metaphor (hirschheim and newman, 1991) that stands for some organizational phenomenon. the social context, in which a model is created and used, determines the way a process is conceptualized and ultimately described in a model. in the theory of signs, this is acknowledged by the fact that there is no direct relationship between representamen and object – instead this relationship is established through the interpretant who conceptualizes the object and symbolizes it in a representamen. established language and modelling practices influence this conceptualization. for instance, the chosen representation language for process models influences the way we perceive, conceptualize and ultimately depict business processes. as an example, the epc notation (scheer, 2000) is based on an event-driven paradigm and conceptualizes business australasian journal of information systems volume 14 number 2 june 2007 55 processes as an interleaved series of events that trigger certain business activities that in turn lead to series of events. quite contrary, idef3 diagrams (plaia and carrie, 1995) conceptualize processes based on an information-flow paradigm. they contain sets of so-called ‘units of behaviour’ that process and exchange information objects amongst themselves. the notion of events as triggers for business activities, is more or less absent in this language. hence, the notion of events is likely to be absent in the way modellers, whose pre-understanding of process models has been coined by their idef3 modelling practice, conceptualize business processes. what holds true for the development of process models holds true for their evaluation as well. as an example, an academically acknowledged theory for the evaluation of business process models is based on a series of control flow patterns (van der aalst et al., 2003). a process modelling language is evaluated with respect of how many and how well it allows for the depiction of these workflow patterns. the underlying conception, obviously, is that a business process domain can be described as a composition of these control flow patterns. another example for domain preconceptions in this field is the bunge-wand-weber model (wand and weber, 1990, 1993), which prescribes a set of domain semantics that should be covered by any process modelling technique. again, such a preconceived model of domain representation has to be continually assessed as to its relevance as a model of domain understanding within every social modelling context. in other words, the question is whether these reference frameworks for model quality evaluation match the existing preconceptions within process modelling domains that are brought to the evaluation task by the members of the community, viz., the model creators and model users within an organization. a social constructionist perspective postulates that in order to assess the relation between model and phenomenon being modelled, the social context that determines the perception and cognition of the universe of discourse must be scrutinized. this includes the assessment of social practices. they determine the individual practices and the common language by which the modellers within the organization conceptualize and model the relevant phenomena. the pragmatic claim of a sociopragmatic constructionist perspective furthermore demands an investigation of values, norms, needs and interests of the stakeholders involved in the modelling practice. this means that the perception of the universe of discourse (the set of processes to be modelled) is selected by the stakeholders interests (vickers, 1970) – it is driven by their needs. hence, the semantic criterion is concerned with the relation of the process model and the socially contextualized and purpose-driven conceptualization of the phenomenon being studied (here: a business process). in simple terms, a good process model would be one that matches the way the model reader thought a process model should look like – based on their needs and historically established understanding of the organization. pragmatics the pragmatic criterion is concerned with the compliance of the model to the aims and purposes for which the model was created. in other words, pragmatism is concerned with establishing what the end user of the model actually seeks to do with it. accordingly, the pragmatic dimension is not solely concerned with whether different stakeholders sufficiently understand the model (lindland et al., 1994). rather, it is concerned with whether the model, as a sort of statement or sentence, enables its interpretants to make use of it for fulfilling their need. that is, the pragmatic dimension transcends pure information delivery concerns such as the ease of retrieving desired information about the process from the process model or the suitability of the presentation form to the comprehension capabilities of different stakeholders. the pragmatic dimension instead is concerned with assessing the value of the process model for helping its interpretants to better cope with their problems. and these problems are typically manifold. they may include, amongst others, the tasks of introducing process-aligned organizational structures, meeting process improvement objectives australasian journal of information systems volume 14 number 2 june 2007 56 or simulating and automating office procedures. in order to arrive at an evaluation of how well a model helps achieving these goals, it is necessary to form mutual agreements about the horizon of meaning of the process model amongst model designers and users. this means, it is vital to establish upfront within the community of model creators and users for what task exactly the model will be needed. for instance, imagine an organization that recently spent considerable time and money to upgrade their it infrastructure. yet, the upgraded infrastructure still induces problems in the day-to-day business operations. given the extent of resources already spent, the ongoing process improvement exercise is imposed the restriction that the it infrastructure in place for the business process under scrutiny is not to be changed and hence should not be included in the process models. this restriction should then be a valid criterion in the judgment of the process models. hence, the pragmatic judgment of process model quality is subject to individual norms, ethics, values and needs. appropriate means for evaluation accordingly call for empirical research strategies rather than theoretical ones. interestingly, other domains of conceptual modelling have taken into account some aspects of pragmatism in their studies. taking the example of conceptual data modelling research, for instance, several researchers have turned to the exploration of different modelling forms and styles and the impact on problem solving tasks, such as, for instance, devising data structures (khatri et al., 2006) or formulating database queries (bowen et al., 2006). yet, the domain of process modelling has not yet been thoroughly approached from this perspective and thus, examples for investigations into pragmatic aspects of process modelling quality are scarce at best. one of the few examples includes the work by danesh and kock (2005) who examined how two different process model styles impact redesign success. in summation, i see potential and first evidence that some of the successful research streams from related conceptual modelling domains could be adopted to the area of process modelling o extend the current body of knowledge. contributions & conclusions this paper presented a socio-pragmatic constructionist perspective on the methodology of business process modelling. driven by the objective of better understanding the quality of a process model i outlined a framework that places emphasis on both the social context in which modelling occurs and the pragmatic aspects of process modelling. i sketched implications of this methodology towards an understanding of process model quality in a basic discussion framework. the main tenet of the work presented in this paper it to place emphasis on the “why” and “where” questions of process modelling initiatives. the paper sought to elicit why it is of importance to establish the purposes and tasks for which process models are to be used in order to be able to ascertain whether the models are ‘good’. also, this paper sought to stress that process modelling, similar to any other form of communication through language discourse, is bound to the social context in which it is exercised. especially in the case of business process management, where an organization is perceived and discussed as a set of intangible business processes, it is essential to establish that everyone involved brings to bear the same understanding of the subject matter in order to facilitate meaningful discussion. given the conceptual/philosophical nature of this paper, the research presented exhibits some obvious limitations. most arguably, the paper does not provide empirical proof for its arguments. as such, it fails some of the traditional assessment criteria nominally ascribed to scientific theory, e.g., (carnap, 1950; hempel, 1965). however, similar to other post-positivist research approaches, think, for instance of action research (susman and evered, 1978; baskerville and wood-harper, 1996), the ultimate sanction of the approach towards process model quality presented should be in the australasian journal of information systems volume 14 number 2 june 2007 57 perceived functionality of the methodology to produce desirable and accurate consequences for research and practice, for instance, by means of providing comprehensive explanatory power. the framework presented aims at providing an epistemology that enables the production of knowledge contingent to the particular context in which it is being applied. implications for practice although methodological and/or philosophical argumentations often appear ‘far-stretched’ rather than directly applicable to is practice, there are arguably practical merits of the work presented. first and foremost, the incorporation of social and pragmatic considerations into the discussion of process modelling and the model(s) produced facilitates an appreciation of the organizational and situational context in which modelling activities occur. by focusing aspects of model quality that transcend traditional syntactic and/or semantic concerns it is made possible to ultimately produce modelling outcomes that are not only of interest to relevant stakeholders but moreover are helpful in solving concrete modelling problems. this way, the ultimate goal of establishing and maintaining business success through principles and methods associated with business process management can be assisted through process modelling. and this should indeed be the goal of every process modelling exercise rather than building ‘correct’ models that are more or less useless for solving organizational problems. a second practical benefit stems from the implications that the framework described presents for education domains. teaching domains can draw upon the discussion in this paper and derive conclusions as to the design of modelling-related curricula. instead of focusing traditional aspects of modelling technique and/or tool features, more consideration can be placed upon situational and motivational aspects. by sharpening an understanding for the roles of actors, stakeholders, pragmatic motivations and the role of knowledge historicity, a better appreciation of the possibilities as well as limitations of process modelling can be taught. existing curricula should be scrutinized in respect to the emphasis they place on syntactical, semantic and/or pragmatic aspects of process modelling. it would not surprise me to learn that to date, most process modelling courses are wellequipped to teach grammars and syntactical aspects (e.g., how do i model a feedback loop and what is an or-join?) but falter when it comes to educating about the role of social discourse, organizational conventions and the prevalent question of the “why” of process modelling. implications for theory this work may contribute as a reference for further research on at least two premises: first, it presents a methodology of business process modelling that provides strong explanatory and explorative power for understanding modelling activities in contemporary organizations. given that the research field of bpm is only slowly “catching up” on its practitioner’s counterpart, i argue that the existence of a rich and multi-perspective theory may serve as a fruitful starting point for further investigations in an under-represented research field, which in turn posits major practical relevance and impact. second, the methodological considerations presented in this paper provide an initial framework for understanding process model quality and can serve as a basis for deriving adequate and theoretically sound dimensions and measures of quality. as such, the current criticisms of model evaluation approaches that claim a lack of theory in the evaluation research field can be counteracted. furthermore, as the methodology presented in this paper stresses the pragmatic, social and contextual aspects of modelling it naturally calls for a well-developed interplay between theoretical and empirical research strategies. this will not only allow to develop an academically stimulating approach but also to provide applicable and relevant results to the business community. australasian journal of information systems volume 14 number 2 june 2007 58 acknowledgements i am indebted to peter marshall from university of tasmania for engaging in a series of fruitful discussions on research epistemology and also for introducing to me the philosophy of pragmatism. i also would like to thank michael rosemann from queensland university of technology for his stimulating comments when preparing an earlier version of this paper. also, i would like to thank the reviewers and editor for their help in shaping this paper. references bajaj, a. & ram, s. (1999) "evaluating completeness of conceptual business process models: a metric based on case studies", journal of information technology cases and applications, vol 1, no 4. baskerville, r. & wood-harper, a.t. (1996) "a critical perspective on action research as a method for information systems research", journal of information technology, vol 11, no 3, pp 235-246. becker, j., rosemann, m. & von uthmann, c. (2000) "guidelines of business process modeling" in van der aalst, w.m.p., desel, j. & oberweis, a. (eds.) business process management. models, techniques, and empirical studies, springer, berlin, germany, pp 30-49. benbasat, i. & weber, r. (1996) "research commentary: rethinking "diversity" in information systems research", information systems research, vol 7, no 4, pp 389-399. berger, p.l. & luckmann, t. (1966) the social construction of reality: a treatise in the sociology of knowledge, doubleday, garden city, kansas. bowen, p.l., o'farrell, r.a. & rohde, f. (2006) "analysis of competing data structures: does ontological clarity produce better end user query performance", journal of the association for information systems, vol 7, no 8, pp 514-544. brito e abreau, f., henderson-sellers, b., piattini, m., poels, g. & sahraoui, h.a. (2002) "quantitative approaches in object-oriented software engineering" in frohner, a. (ed.) object-oriented technology. ecoop 2001 workshop reader, springer, berlin, germany, pp 174-183. buhl, h.u. & heinrich, b. (2005) "empirical research strategies in conceptual modeling silver bullet or academic toys?" wirtschaftsinformatik, vol 47, no 2, pp 152-162. calás, m.b. & smircich, l. (eds.) (1997) postmodern management theory, ashgate, aldershot, england. carnap, r. (1950) logical foundations of probability, the university of chicago press, chicago, illinois. chen, w.s. & hirschheim, r. (2004) "a paradigmatic and methodological examination of information systems research from 1991 to 2001", information systems journal, vol 14, no 3, pp 197-235. chia, r. (1996) organizational analysis as deconstructive practice, gruyter, berlin, germany. australasian journal of information systems volume 14 number 2 june 2007 59 curtis, b., kellner, m.i. & over, j. (1992) "process modeling", communications of the acm, vol 35, no 9, pp 75-90. danesh, a. & kock, n. (2005) "an experimental study of two process representation approaches and their impact on perceived modeling quality and redesign success", business process management journal, vol 11, no 6, pp 724-735. davenport, t.h. (1993) process innovation: reengineering work through information technology, harvard business school press, boston, massachusetts. davies, i., green, p., rosemann, m., indulska, m. & gallo, s. (2006) "how do practitioners use conceptual modeling in practice?" data & knowledge engineering, vol 58, no 3, pp 358380. dehnert, j. & van der aalst, w.m.p. (2004) "bridging the gap between business models and workflow specifications", international journal of cooperative information systems, vol 13, no 3, pp 289-332. dehnert, j. & zimmermann, a. (2005) "on the suitability of correctness criteria for business process models" in van der aalst, w.m.p., benatallah, b., casati, f. & curbera, f. (eds.) proceedings of the 3rd international conference on business process management, springer, nancy, france, pp 386-391. dewey, j. (1988) the middle works of john dewey, southern illinois press, carbondale, illinois. dumas, m., van der aalst, w.m.p. & ter hofstede, a.h.m. (eds.) (2005) process aware information systems: bridging people and software through process technology, john wiley & sons, hoboken, new jersey. eden, c., jones, s., sims, d. & smithin, t. (1981) "the intersubjectivity of issues and issues of subjectivity", journal of management studies, vol 18, no 1, pp 37-47. gadamer, h.-g. (1989) truth and method (2nd edition), continuum, new york, new york. garda, j.a., casal, j.a., vázquez, r.g., pazos, j., yánez, s.r. & silva, a. (2004) "a methodological framework for generic conceptualisation: problem-sensitivity in software engineering", information and software technology, vol 46, no 10, pp 635-649. gartner group (2007) "creating enterprise leverage: the 2007 cio agenda", exp premier report no january2007, gartner, inc, stamford, connecticut. green, p. & rosemann, m. (2000) "integrated process modeling. an ontological evaluation", information systems, vol 25, no 2, pp 73-87. green, p., rosemann, m. & indulska, m. (2005) "ontological evaluation of enterprise systems interoperability using ebxml", ieee transactions on knowledge and data engineering, vol 17, no 5, pp 713-725. gregor, s. (2006) "the nature of theory in information systems", mis quarterly, vol 30, no 3, pp 611-642. hempel, c.g. (1965) aspects of scientific explanation and other essays in the philosophy of science, the free press, new york, new york. hirschheim, r. & newman, m. (1991) "symbolism and information systems development: myth, metaphore and magic", information systems research, vol 2, no 1, pp 29-62. australasian journal of information systems volume 14 number 2 june 2007 60 holten, r. (2003) "specification of management views in information warehouse projects", information systems, vol 28, no 7, pp 709-751. hradesky, j. (1994) total quality management handbook, mcgraw-hill, new york, new york. kant, i. (1929) the critique of pure reason [translated by norman kemp smith], st. martin's press, new york, new york. karimi, j. (1988) "strategic planning for information systems: requirements and information engineering methods", journal of management information systems, vol 4, no 4, pp 5-24. keen, c.d. & lakos, c. (1996) "analysis of the design constructs required in process modelling" in purvis, m. (ed.) proceedings of the international conference on software engineering: education and practice, ieee computer society, dunedin, ireland, pp 434-441. khatri, v., vessey, i., v. ramesh, p.c. & sung-jin, p. (2006) "understanding conceptual schemas: exploring the role of application and is domain knowledge", information systems research, vol 17, no 1, pp 81-99. kiepuszewski, b., ter hofstede, a.h.m. & van der aalst, w.m.p. (2003) "fundamentals of control flow in workflows", acta informatica, vol 39, no 3, pp 143-209. krogstie, j., sindre, g. & jørgensen, h.d. (2006) "process models representing knowledge for action: a revised quality framework", european journal of information systems, vol 15, no 1, pp 91-102. lindland, o.i., sindre, g. & solvberg, a. (1994) "understanding quality in conceptual modeling", ieee software, vol 11, no 2, pp 42-49. lyotard, j.-f. (1984) the postmodern condition: a report on knowledge, university of minnesota press, minneapolis, minnesota. lyytinen, k. (2006) ""ontological foundations of conceptual modeling" by boris wyssysek a critical response", scandinavian journal of information systems, vol 18, no 1, pp 81-84. marshall, p., kelder, j.-a. & perry, a. (2005) "social constructionism with a twist of pragmatism: a suitable cocktail for information systems research" in campbell, b., underwood, j. & bunker, d. (eds.) proceedings of the 16th australasian conference on information systems, australasian chapter of the association for information systems, sydney, australia. mingers, j. (2004) "real-izing information systems: critical realism as an underpinning philosophy for information systems", information and organization, vol 14, no 2, pp 87103. mir, r. & watson, a. (2000) "strategic management and the philosophy of science: the case for a constructivist methodology", strategic management journal, vol 21, no 9, pp 941-953. moody, d.l. (2005) "theoretical and practical issues in evaluating the quality of conceptual models: current state and future directions", data & knowledge engineering, vol 15, no 3, pp 243-276. morgan, g. (ed.) (1983) beyond method: strategies for social research, sage, beverly hills, california. morris, c.w. (1971) writings on the general theory of signs, mouton de gruyter, the hague, the netherlands. australasian journal of information systems volume 14 number 2 june 2007 61 mylopoulos, j. (1992) "conceptual modelling and telos" in loucopoulos, p. & zicari, r. (eds.) conceptual modelling, databases, and case: an integrated view of information system development, john wiley & sons, new york, new york, pp 49-68. neimeyer, r.a. (1998) "social constructionism in the counselling context", counselling psychology quarterly, vol 11, no 2, pp 135-150. nelson, h.j., poels, g., genero, m. & piattini, m. (2005) "quality in conceptual modeling: five examples of the state of the art", data & knowledge engineering, vol 55, no 3, pp 237242. nielsen, p. & main, c. (2004) "company liability after the sarbanes-oxley act", insights, vol 18, no 10, pp 2-12. oasis (2001) "ebxml business process specification schema version 1.01". http://www.ebxml.org/specs/ebbpss.pdf accessed march 12. ogden, c.k. & richards, i.a. (1923) the meaning of meaning: a study of the influence of language upon thought and of the science of symbolism, harcourt, brace & company, new york, new york. olle, t.w., hagelstein, j., macdonald, i.g., rolland, c., sol, h.g., van assche, f.j.m. & verrijnstuart, a.a. (1991) information systems methodologies: a framework for understanding, addison-wesley, wokingham, england. orlikowski, w.j. & baroudi, j.j. (1991) "studying information technology in organizations: research approaches and assumptions", information systems research, vol 2, no 1, pp 128. ould, m.a. (1995) business processes: modelling and analysis for re-engineering and improvement, john wiley & sons, chichester, england. petri, c.a. (1962) "fundamentals of a theory of asynchronous information flow" in popplewell, c.m. (ed.) ifip congress 62: information processing, north-holland, munich, germany, pp 386-390. plaia, a. & carrie, a. (1995) "application and assessment of idef3-process flow description capture method", international journal of operations & production management, vol 15, no 1, pp 63-73. poels, g., nelson, h.j., genero, m. & piattini, m. (2003) "quality in conceptual modeling new research directions" in olivé, a. (ed.) advanced conceptual modeling techniques, springer, berlin, germany, pp 243-250. price, r. & shanks, g. (2005) "a semiotic information quality framework: development and comparative analysis", journal of information technology, vol 20, no 2, pp 88-102. quine, w.v.o. (1960) word and object, the mit press, cambridge, massachusetts. recker, j. (2006) "process modelling in the 21st century", bptrends, vol 3, no 5, pp 1-6. recker, j. & indulska, m. (2007) "an ontology-based evaluation of process modeling with petri nets", journal of interoperability in business information systems, vol 2, no 1, pp 45-64. rescher, n. (1973) the primacy of practice, basil blackwell, oxford, england. robinson, a.g. & dilts, d.m. (1999) "or & erp: a match for the new millenium?" or/ms today, vol 26, no 3, pp 30-35. http://www.ebxml.org/specs/ebbpss.pdf� australasian journal of information systems volume 14 number 2 june 2007 62 rosemann, m. (2006a) "potential pitfalls of process modeling: part a", business process management journal, vol 12, no 2, pp 249-254. rosemann, m. (2006b) "potential pitfalls of process modeling: part b", business process management journal, vol 12, no 3, pp 377-384. rosemann, m., recker, j., indulska, m. & green, p. (2006) "a study of the evolution of the representational capabilities of process modeling grammars" in dubois, e. & pohl, k. (eds.) advanced information systems engineering caise 2006, springer, luxembourg, grand-duchy of luxembourg, pp 447-461. rosemann, m., sedera, w. & sedera, d. (2001) "testing a framework for the quality of process models – a case study" in kim, j.s., lee, j.k., suh, k., wei, k.k. & lee, h. (eds.) proceedings of the 5th pacific asia conference on information systems, the korea society of management information systems, seoul, korea, pp 978-991. russell, n., van der aalst, w.m.p., ter hofstede, a.h.m. & wohed, p. (2006) "on the suitability of uml 2.0 activity diagrams for business process modelling" in stumptner, m., hartmann, s. & kiyoki, y. (eds.) conceptual modelling 2006: 3rd asia-pacific conference on conceptual modelling, australian computer society, hobart, australia, pp 95-104. scheer, a.-w. (2000) aris business process modeling (3rd edition), springer, berlin, germany. siau, k. (2004) "informational and computational equivalence in comparing information modeling methods", journal of database management, vol 15, no 1, pp 73-86. sinur, j. (2004) "magic quadrant for business process analysis", gartner research note no m-220651 march, gartner, inc, stamford, connecticut. soffer, p. & wand, y. (2007) "goal-driven multi-process analysis", journal of the association for information systems, vol 8, no 3, pp 175-202. stachowiak, h. (1972) "models" in unesco (ed.) scientific thought. some underlying concepts, methods, and procedures, mouton/unesco, paris, france, pp 145-166. stachowiak, h. (1973) allgemeine modelltheorie [in german], springer, wien, austria. styhre, a. (2003) understanding knowledge management. critical and postmodern perspectives, copenhagen business school press, herndon, virginia. susman, g.i. & evered, r.d. (1978) "an assessment of the science merits of action research", administrative science quarterly, vol 23, no 4, pp 582-603. ter hofstede, a.h.m. & van der weide, t.p. (1992) "formalisation of techniques: chopping down the methodology jungle", information and software technology, vol 34, no 1, pp 57-65. van der aalst, w.m.p. (1999) "formalization and verification of event-driven process chains", information and software technology, vol 41, no 10, pp 639-650. van der aalst, w.m.p. (2005) "pi calculus versus petri nets: let us eat humble pie rather than further inflate the pi hype", bptrends, vol 3, no 5, pp 1-11. van der aalst, w.m.p., ter hofstede, a.h.m., kiepuszewski, b. & barros, a.p. (2003) "workflow patterns", distributed and parallel databases, vol 14, no 1, pp 5-51. verbeek, h.m.v., van der aalst, w.m.p. & ter hofstede, a.h.m. (2007) "verifying workflows with cancellation regions and or-joins: an approach based on relaxed soundness and invariants", the computer journal, vol 50, no 3, pp 294-314. australasian journal of information systems volume 14 number 2 june 2007 63 vessey, i., ramesh, v. & glass, r.l. (2002) "research in information systems: an empirical study of diversity in the discipline and its journals", journal of management information systems, vol 19, no 2, pp 129-174. vickers, g. (1970) freedom in a rocking boat, penguin books, london, england. von glasersfeld, e. (2001) "the radical constructivist view of science", foundations of science, vol 6, no 1-3, pp 31-43. wand, y. & weber, r. (1990) "an ontological model of an information system", ieee transactions on software engineering, vol 16, no 11, pp 1282-1292. wand, y. & weber, r. (1993) "on the ontological expressiveness of information systems analysis and design grammars", journal of information systems, vol 3, no 4, pp 217-237. wand, y. & weber, r. (2002) "research commentary: information systems and conceptual modeling a research agenda", information systems research, vol 13, no 4, pp 363-376. weber, r. (2003) "editor's comments: still desperately seeking the it artifact", mis quarterly, vol 27, no 2, pp iii-xi. weber, r. (2004) "editor's comments: the rhetoric of positivism versus interpretivism: a personal view", mis quarterly, vol 28, no 1, pp iii-xii. wicks, a.c. & freeman, r.e. (1998) "organization studies and the new pragmatism: positivism, anti-positivism, and the search for ethics", organization science, vol 9, no 2, pp 123-140. wilson, t.d. (2002) "alfred schutz, phenomenology and research methodology for information behaviour research" in proceedings of the fourth international conference on information seeking in context, universidade lusiada, lisbon, portugal. wohed, p., van der aalst, w.m.p., dumas, m. & ter hofstede, a.h.m. (2003) "analysis of web services composition languages: the case of bpel4ws" in song, i.-y., liddle, s.w., ling, t.w. & scheuermann, p. (eds.) conceptual modeling er 2003, springer, chicago, illinois, pp 200-215. wohed, p., van der aalst, w.m.p., dumas, m., ter hofstede, a.h.m. & russell, n. (2006) "on the suitability of bpmn for business process modelling" in dustdar, s., fiadeiro, j.l. & sheth, a. (eds.) proceedings of the business process management bpm 2006, springer, vienna, austria, pp 161-176. wyssusek, b. (2006) "on ontological foundations of conceptual modelling", scandinavian journal of information systems, vol 18, no 1, pp 63-80. wyssusek, b., schwartz, m., kremberg, b., baier, f. & krallmann, h. (2001) "business process modelling as an element of knowledge management – a model theory approach" in proceedings of the 2nd international conference on managing knowledge: conversations and critiques, leicester university, leicester, england. editorial board thomson scientific (isi) impact factor. using existing ranking lists polling eminent scholars stage one: identification of existing rankings stage two: initial ranking list drawn up stage three: feedback on the initial ranking list stage four: information and finalisation of ranking list level a journals level b journals premier professional journals abstract introduction the pending australian research quality framework (rqf) methodology sourcing and procuring the data examining the data for correctness results rq1 what journal outlets are used by australian academics to publish is-related research? rq2 which journals have been most preferred by australian academics for article publication as measured by the total number of articles published in each journal? conclusion limitations and future research socio-pragmatic constructionism social constructionism pragmatism socio-pragmatic constructionism representation simplification pragmatic orientation syntax semantics pragmatics implications for practice implications for theory passive dimension (perceptions); active dimension (conceptions) the aesthetic dimension is not characterized as “plug and play”. the aesthetic dimension is not described by graphical user interfaces. aesthetics occurs when the system elevates the work to a new level of effective vision. aesthetics may occur on different levels or from different perspectives. 1. rational goal seeking (intelligence(design(choice) 2. happenstance rationalization (design(choice(intelligence) 3. existential introversion (choice(intelligence(design) 4. deferred design (intelligence(choice(design) 5. situational sensemaking (design(intelligence(choice) 6. existential heroism (choice(design(intelligence) constructs output performance outsourcing experience contract establishment the feedback effect business strategy it strategy firm-wide active involvement long term focus meeting of the minds clarity and consistency management skill and capability alignment facilitation processes organisational structure organisational culture communication it as an organisational tool organisation background information factors that promote alignment people factors process factors organisational factors high level factor comparisons alignment of business & information technology strategy a cross industry australian study part 1: background part 2: business strategy part 3: information technology strategy part 4: factors that promote the alignment between business and information technology strategy documentation interviews researcher bias ajis vol. 5 no. 2 may 1998 the australian information systems research community: an analysis of mainstream publication outlets10 gail ridley1, paula goulding2, glenn lowry3, graham pervan4 'school of information systems, university of tasmania hobart australia email: gail.ridley@utas.edu.au 2department of information technology, murdoch university perth australia 'college of business administration, university of bahrain state of bahrain 'school of information systems, curtin university of technology perth australia abstract this paper reports on an investigation of the nature of and interactions among the information systems academic research community in australia. the 382 refereed papers from the two mainstream is refereed publication outlets in australia were analysed for the period 1990-1996 to determine the contributions of individual universities, departments and authors to the pattern of collaboration, and to examine the nature of the contributors. the study revealed that several institutions, departments and authors dominated the publication outlets and that the majority of the collaboration that occurred took place within university departments. collaboration between and among different disciplines and nations, and between practitioners and academics was limited. it was found that although the australian academic is research community had only recently emerged, it was growing strongly. introduction information systems (is) is an emerging field with a short history that has been subject to the influence of many reference disciplines. north american calls for guidelines to recognise quality is research (king 1993) must be even more pertinent in australia where is researchers have additional pressures of diverse work histories and educational backgrounds (shanks, rouse & arnott 1993). partly as a consequence, australian is research traditions have yet to be established and gain recognition (ridley 1996). culnan (1987) urged the monitoring of the 'state of development [of is] on an ongoing basis'. this paper is one attempt to monitor those aspects of the development of mainstream is research traditions in australia that relate to the contribution of institutions, departments, authors and collaboration to the is research profile. the research reported below is part of a larger study that investigates the nature and development of is research in australian universities. one phase is an analysis of the publication patterns of australian is researchers. if publication is a 'legitimator of academic performance' (king 1993), then it is important for the greater status of is research and consequent improved access to resources that australian is publication patterns be known (ridley 1996). as resources are usually allocated on a national or more local level, for example australian research council grants and general university funding, traditions of is research need to be established at a national level, not only at an international level. despite a body of literature emanating from north america that has investigated the contribution of institutions and authors to the is research profile, little work of this kind has been conducted in australia. however, research of such a nature is timely now that an awareness of a need for a political presence is developing in australian is, as demonstrated by the formation of potential lobby groups such as the is heads of discipline meetings (ridley 1996). one example of such research conducted at the individual department level is reported in a paper by cue and urquhart (unpub. 1997) that benchmarks the research output of one australian is department against that of others. however, as far as the authors are aware, a specific analysis of australian is academic research has not been published. such an analysis will provide useful information for staff and postgraduate students seeking a place in australian is, for benchmarking within a department or school (hereafter stated only as department), institution or the discipline and for the allocation of resources. changing patterns of research collaboration will track growth in a shared understanding of the nature of australian is research, and over time will have the potential to trace progress in the development of the is scientific community. such data must also provide fundamental 10 a previous version of this paper was presented at the 8th australasian conference on information systems, 1997, and appeared in the proceedings of that conference. 69 ajis vol. 5 no. 2 may 1998 background information for people striving to improve the status and power of australian is research (ridley 1996). the aim of this paper is to report on an investigation of the contribution and collaboration of institutions, departments and authors to mainstream is research published in australia over the period 1990 to 1996, and to study some characteristics of the authors. it seems reasonable to examine australian is mainstream publication outlets in order to investigate australian is mainstream research. it is acknowledged however that some is researchers will have submitted papers to specialist publication outlets in addition to mainstream is outlets. other australian is researchers will have targeted mainly international publication outlets. there are only two australian refereed outlets that publish papers at least annually and cover a broad range of is themes and methodologies. the first is the australasian conference on information systems (acis) and the second is the australian journal of information systems (ajis). moreover, ajis and acis are the two leading australian publication outlets considered to be appropriate for the dissemination of australian is research (ridley 1996). acis proceedings and ajis have been analysed from the first issue until the end of 1996. the analysis period commences in 1990 as it was the first year that acis was held. background journals as machines of control many authors have argued a need for is researchers to work less independently, develop underlying theories for the discipline (farhoomand 1987) and for individual researchers to build on the work of others (keen 1980). 'fact construction' is a collective process needing the support of a network or scientific community of researchers, a process that cannot be achieved by isolated researchers (latour 1987). latour stated that with the assistance of a 'machine', which can be either a physical mechanism or a less tangible 'stratagem' such as an organisational system, it is possible to unite researchers to form a scientific community. walsham (1995) built on latour's ideas in his application of them to the discipline of is, and considered is journals to be 'machines or stratagems for enrollment [sic] and control'. he considered that journal editors enlisted both the editorial board and referees to control the scope, epistemology and research strategies of papers from prospective contributors to the journal. if walsham is correct, over time in the history of a publication outlet, there will arise certain institutions, departments, authors and employment sectors that are favoured through repeated publication because the scope and nature of their research represents quality research from the perspective of the editorial board and referees, which will encourage further submissions. ajis and acis proceedings are the only two at least annual general is academic publication outlets in australia. therefore it is argued that institutions, departments and authors that repeatedly publish in the two outlets will be mainstream members of the australian is scientific community and growth in their numbers will represent growth in that community. furthermore, the nature of the papers will constitute many of the 'research traditions' of that community. patterns of collaboration will trace growth and links of the australian is scientific community between and among universities, departments, researchers, is practitioners and academia and between australia and other nations. developments in these links will reveal changes to the degree of isolation of individual researchers and the development of the australian is scientific community itself from a number of individuals to a network of researchers, able to undertake 'fact construction'. perhaps it is in the early stages of the development of a discipline that journals (and refereed conferences) are most useful as a means of communicating and instructing neophyte and potential members of a scientific community in the group research traditions. once the discipline is established, it appears that other journals emerge to represent non-mainstream groups. for example, in the international is community, positivist papers such as those commonly found in mis quarterly (first published in 1977) typified mainstream research (walsham 1995), but more recently other, often european 'nontraditional' is journals were introduced that published interpretivist papers (orlikowski and baroudi 1991; walsham 1995). examples of 'nontraditional' is journals include the european journal of information systems and accounting, management and information technologies (both first published in 1991). after a discipline has been established for some time the strong unifying influence of a traditional journal is challenged by 'stratagems' other than non-mainstream journals. these include a uniform discipline-specific educational programme for prospective researchers, national electronic mailing lists, the strong bonding influences of research groups within a department and networks of 'invisible colleges' that link researchers in different departments, universities and nations to common topics, epistemologies, research methods and approaches. is has been established in north america for a considerable time. as stated earlier mis quarterly was first published in the united states of america in 1977 where the first icis conference was also held in 1980. the influence of north america is powerful and prevails upon australian is researchers. the prestigious north american journals and conferences are regarded as mainstream publication outlets in australia, and are 70 ajis vol. 5 no. 2 may 1998 considered appropriate for the publication of australian is research results (ridley 1996). however, despite this influence the authors suspect that australian is research has a distinctive style which is different to that carried out in north america. if there is an australian flavoured is research, then at least some is research traditions in australia are likely to differ in nature to those held elsewhere. in australia, the local perspective of appropriate and quality is research traditions have been represented and engendered by acis since 1990 and ajis since 1993. it is only in 1997 that the influence of acis and ajis to 'control' the research traditions of australian is researchers has been challenged with the planned introduction of the journal of holistic research in information systems. the aims of the journal are stated as fostering '...primarily interpretive or qualitative research methods...' (standing 1996). invisible colleges de solla price (1961, 1963) first described the 'invisible college'. culnan (1987) defined 'invisible colleges' as the clustering of researchers into informal networks 'which tend to concentrate on examining common questions in common ways'. de solla price (1963) described how invisible colleges diffuse knowledge through collaboration. therefore, patterns of collaboration will help identify invisible colleges. de solla price saw some practices of invisible colleges as threatening the role of journals, and gave as an example the distribution of prepublication papers to members of an invisible college (1961; 1963). the use of electronic mailing lists for this purpose exemplifies the threat de solla price foresaw 25 years ago. de solla price's concern suggests there is a time sequence in the development of a scientific community, with the setting up of mainstream journals belonging to an earlier phase than the operation of resilient invisible colleges. he saw membership of an invisible college as conferring power and status on an individual and the network (1963). it is these networks of researchers that are best placed to lobby for better access to resources and funds (ridley 1996). for collaboration to be possible, researchers must have a common understanding that their research scope is valued and agree on the philosophical approach and research strategies to be employed. therefore it is through collaboration that traditions of research are established and disseminated, to make invisible colleges a powerful 'stratagem' to initiate and nurture researchers into the norms and mores of a research community. although several north american papers have investigated the research profile of is institutions, departments and authors, few, if any, considered the contribution of australian research. as a result there is no means of comparing australian researchers. an overview of the relevant north american literature is now presented. emphasis is placed on the methodology of the studies, in addition to the results. literature on research profiles of institutions, departments and authors joint authorship of papers is commonly found in is journals and conference papers. lindsey (1980) examined three different approaches to calculate the contribution of an individual author where joint authorship exists. the first is the 'normal count', where an equal unit count is ascribed to each joint author. in the 'adjusted count' one joint author is attributed with a fraction of a unit count which will decrease as the number of authors increases. the 'straight count' attributes the paper solely to the first listed author so that co-authors receive no credit. however the straight count is seen as inequitable to co-authors other than the first listed. the adjusted count is viewed by lindsey as more just, but it is hard to maintain. as the normal count is the most usual method used to attribute credit to multiple authors (lindsey 1980) it has been used in this study. jackson and nath (1989) examined the publication patterns of management information systems (mis or is) researchers over the period 1975-1987. they analysed 899 papers from ten well known north american journals to find that the leading 32 researchers contributed 18 percent of the total research examined and that a majority of the leading is researchers were located at a few schools. the study found that 47 percent of the papers had one author, 41 percent had two authors, ten percent had three authors and two percent had four or more authors and that there had been an increase in the number of institutions with leading researchers. jackson and nath (1989) calculated credit for joint authorship using all three methods in turn, but referred to the normal count findings in their conclusion. grover, segars and simon (1992) weighted is article contributions over the period 1982-1991 in three timeslices from five leading journals using 'standardized pages' and an adjusted count. the papers were credited to over 190 institutions which were then ranked. following the recommendation of niemi (1987) the authors made no adjustment for department size. they found that the top fifteen institutions contributed over half the research examined. between 1982-86 and 1987-91 the is research output of the journals increased by 97 percent. grover, segars and simon allotted credit to institutions based on the affiliations listed on the papers even though they may have changed since publication. lending and wetherbe (1992) and vogel and wetherbe (1984) used an adjusted count to produce a profile of leading universities by identifying 389 and 744 is papers in 15 and 13 leading journals respectively. they found 71 ajis vol. 5 no. 2 may 1998 that both the quantity of papers and the number of universities had almost doubled between 1977-1983 and 1984-1990. in the 1992 study the authors concluded that even though ten percent of the mis schools produced 44 percent of the total published research there was less domination by the most productive schools. shim, english and yoon (1991) determined the 60 most frequently published is authors from 817 papers in a weighted group of eight leading is journals for the years 1980-1988 using an adjusted count. they found that each of the sixty authors had a minimum of four papers. methodology epistemology, research question and methods the study was carried out within a descriptive, positivist framework (orlikowski & baroudi 1991). the aim of the paper was to report on an investigation of the contribution and collaboration of institutions, departments and authors to non-specialist is research published in australia during 1990-1996 inclusive, and to study some characteristics of the authors. space limitations prevent the reporting of an analysis of is papers by australian authors published elsewhere. descriptive statistical analysis was employed. sources of data other than acis, general is conferences are only occasionally held in australia, for example the pacific asia conference on information systems and the tc8aus 1f1p information systems international working conference. even though the australian computer journal publishes a mix of is and computer science papers, the number of is papers is low, whereas in ajis all papers are from is. although eminent australian is researchers do publish in general refereed is international journals and conference proceedings, the total number of such papers is very small. specialised publication outlets were excluded for the reasons previously stated. apart from acis and ajis there is no other general publication outlet which publishes australian is research papers both frequently and regularly. the acis proceedings derive from a conference while ajis is a journal. as the combined number of papers is sizeable, an analysis of the papers should be representative of generalist refereed is research published by australians. both acis and ajis were perceived by influential australian is researchers as belonging to the third of three tiers of is publication outlets considered appropriate for the publication of australian is research when ranked by quality (ridley 1996). however, very few australian published papers are found within publication outlets from the first and second tiers. acis has been held annually since its inception in 1990. it is regarded as the 'focal meeting point for is researchers in australasia' (sutton 1997). the first two conferences were run in association with the australian conference on computer science (arnott unpub. 1995), but from 1992 the conferences were held independently. in 1990 the acis proceedings contained fifteen papers while in 1996 there were 72 refereed papers in 846 pages. ajis is a biannual journal first published by the department of business systems at the university of wollongong in 1993. it publishes papers that relate to the theory and practice of information systems. it has published from six to nine papers per issue and regularly publishes papers from both australian is researchers and those from outside australia. procedure and analysis all refereed papers from all editions of ajis and acis for the inclusive period 1990-1996 were analysed. although the institution and department names were recorded from affiliations stated on the papers, some institutions and departments have restructured while some authors have changed their institution or organisation since their paper was published. where the department of one of joint authors was given but not the institution it was assumed that the institution was the same as for the other author/s. where two affiliations were given for an author, only the first was recorded on the assumption it was the major one. when only a private address was given for an author it was assumed that he or she was a practitioner (or ex-student). post-graduate students were recorded in the academic count. as ajis volumes run across calendar years, to be consistent with the acis counts calendar years were used for the ajis figures rather than volumes. the paper was used as the unit of measurement as percentages were rounded to the nearest whole number, totals may exceed one hundred percent. a frequency count of australian academic institutional and departmental affiliations of authors was made. if a research centre was associated with a department, the department name was used. then a frequency count of academic and practitioner australian authors was performed. 72 ajis vol. 5 no. 2 may 1998 this study regarded collaboration as the publishing of joint research resulting from the co-operation of researchers in different institutions. to investigate the nature of author collaboration, firstly the number of authors was recorded for those papers authored by australian practitioner and academic authors. where papers were jointly authored by at least one author in australia and one outside australia, those papers were included. secondly, analysis was conducted of all jointly authored papers with at least one australian author by the employment status and location of the authors, that is whether collaboration took place within the same department of an australian university or organisation, within different departments of the same australian university, different universities within australia, between universities within and outside australia, between australian academics and practitioners or between academics and practitioners where at least one came from australia and at least another from outside australia. then an analysis was undertaken to investigate some characteristics of authors. frequencies of the employment status and geographic location of australian university author/s were recorded. results and discussion a total of 382 papers were examined, of which 50 were from ajis and the rest from acis. most published universities table 1 lists the seven most published universities with 24 or more is refereed papers in acis and ajis during 1990-1996. the cut-off point was chosen at the 81st percentile (of universities) after the seventh where a break from 24 to 16 papers occurred. the group of seven made up only 19 percent of the 36 universities that published papers, yet accounted for 61 percent of the 496 total author credits for australian academic papers. as the normal count was used, each entry may record the sole contribution of an author to a paper, or a co-author contribution, which may have been in conjunction with another university. the figure for the university of new south wales included the papers for the australian defence force academy. university curtin monash swinburne u of melbourne u of nsw u of tasmania uof wollongong totals '90 aci s 3 ' , 2 5 *• < 10 ajis |t|£; '•$» \&f f ', i »'• * ,? «. ' ~ f d '91 aci s 1 v -2 ~ '!' i '3, ; • c& \ i; <• <io ajis i • , "».t'-'. 4c * tv* r ^j 'i* * •r.™ *i' ^ ' -0 '92 aci s 6 ^4 • <>' ^2 ,3 1 m \ < 26 ans c'1't * *' , •. '.' ' , ', : "° '93 aci s 9 9 -8 1 5 , 1 2 35 ajis 6 1 7 '94 aci s 4 ,38 !6 2 ',7 6 6 69 ajis 1 3 4 '95 aci s 8 , 22 15 ,6 ' 3 6 6 66 ajis 2 3 2 2 9 '96 aci s 4 11 ;5 6 3 12 3 > 44 ahs 2 4 1 1 1 9 total s 36 90 45 24 35 26 33 289 table 1 most published universities a correlation between the university hosting acis and the number of papers published in that year was noted, a likely consequence of the increased consciousness and enthusiasm that hosting the conference generated in the host university. there were 36 australian universities that participated in the is research community in australia and were willing to conform to the research scope, methodology and methods considered relevant and rigorous by ajis and acis. a core group of seven universities assumed a leading role in the establishment of australian research traditions. most published departments there were 96 departments from which an author had published a paper in ajis or acis. table 2 lists the four most published australian university departments with 22 or more is refereed papers published in acis and ajis during 1990-1996. the cutoff point was chosen after the four most published departments despite the belief by the authors that a larger listing would have been of interest to the australian is community. however, a large break from 22 to 14 papers was apparent after these departments, the fourth of which was on the 96th percentile (of departments), and no break with more than a difference of one occurred at any other place in the 73 ajis vol. 5 no. 2 may 1998 table. to have made the cutoff point at any other point in the table would have resulted in choosing the most published departments in an arbitrary manner. the four departments authored around 29 percent of the 496 total author credits for papers from australian departments. any entry in table 2 may record the sole contribution of an author to a paper, or a shared contribution with another department where there were multiple authors. the percentage of the total papers published by each of the four most published departments was considerably less than the percentage of the total papers published by their universities as many universities had more than one department where is research was undertaken. the departments that were not the most published and yet belonged to one of the 'most published universities' may have included authors who published in ajis and acis and identified with the is community. however, the departments may also have included other authors who did not identify with general is but who found it expedient to publish in is publication outlets, or who identified with is as well as some other or more specialised discipline or field. department curtin school of is monash deptofis swinburne school of is u of nsw school of is totals '90^. '•. v aci s ' . :\ :,.; •s\ ;'' . .. :{ ^f> ^/ /:".;* r~:$: .;': v~; >< -c^; ?•*£•' : ;:4 w& ajis . -•-,-,.. •'•"!-. -.' ••',-'vi': ;ht: '•"..•'; • ':'-'':/ ''•• 7:^' vv* •?!.•• aci s ;!.' '.'v'.';; ; \ ...*s •''~'')"-~•'•'<-.; -x-2 :;i! >t jjss' r'i::r'3. -a^ ^5to ajis tc^? ;v^• -.t.vil: ^^|̂ | ^•fe?k 3k?' =lfp '92; ' ' . " ' ;i aci s ' .1 ; k ; ^i'6 ;?' ' ' >!/ ; l:ff:2 i"-^ :-,j'm! :" v '. ' r* "i;3 •^ $®$ <$$£• ajis li^. ir$i 4 > : a*-'" -!' '•"ilti-n?-';*1! •sift;? -^fip '.hf^ff •'••"-'•si: y~& ,'93 aci s {'• '•:•] -6 •;;:!/ > ^.-' : ' •• :k-'-'2 ?v: "»-;.. '" •;-ti-3 ahs 4 4 '94 aci s 4 28 „,; '3 ^ «> 5 40 ajis 1 1 ?95 aci s •7 ' n ~ 1 2 2 38 ajis 2 1 3 '96 aci s 3 4 2 • . •• 1 10 ajis 1 4 5 total s 0 27 0 61 0 26 0 22 136 table 2 most published departments the findings indicated that there were 96 australian departments that participated to varying degrees in the general australian is research community and had been willing to comply with the scope, methodology and research methods viewed as acceptable by ajis and acis. of this large group there was a primary group of four departments that had taken a leading role in the establishment of research traditions in australia. these four were considered to be leading departments because of their publication activity. in the previous two tables, it is interesting to consider the period over which the individual universities and departments published their papers. in some instances the publications were achieved over the full seven year period, while in others, in fewer years. in general, however, it was from 1993 that the number of publications increased. most published authors a total of 314 australian authors had published in one, either or both of the two outlets. table 3 gives the number of authors with one, two, three and four or more papers published in ajis and acis. it can be seen that there was a cohort of 20 australian academic or practitioner authors with four or more refereed papers published during 1990-1996. the cohort cutoff point at the 94th percentile (of authors) excluded those authors who had published three papers. members of the most published author cohort were listed as authors in 105 of the 496 author credits from the 382 papers, and made up more than 21 percent of the total author credits. each paper may represent either a solely or jointly authored paper. no. of papers 4+ 3 2 1 no. of authors 20 22 53 219 table 3 number of authors with four or more, three, two or one published papers it is interesting to note that the seven most published universities employed fifteen of the most published authors, while the four most published departments employed twelve. the calculations were based on the affiliation given in the authors' latest publication. the figures demonstrate that the centres of influence in australian is research had a narrow base. 74 ajis vol. 5 no. 2 may 1998 if publishing a refereed paper in one of the two publication outlets during the period is regarded to be the criterion for membership, it can be seen that at the end of 1996 there were 314 people in australia who could be considered to belong to the general is research community, in addition to the handful of eminent researchers who published internationally. however, some of the 314 researchers were likely to identify more strongly with another research community. is intersects with many disciplines, and it may be expedient for researchers from an allied discipline to publish in an is outlet. members of this group were most likely to be found in the group who published just once. increase in number of australian authors 400 t 300 • * 100 • • 1990 1991 1992 1993 year 1994 1995 1996 figure 1 cumulative total of published authors, acis and ajis combined as can be seen in figure 1 above, there has been a steady growth in the combined acis and ajis cumulative total of australian authors over the period 1990-1996, which indicates in turn growth in the australian is research community. collaboration figure 2 and table 4 investigate the nature of collaboration during the period 1990-1996. figure 2 presents the combined yearly totals of acis and ajis australian practitioner and academic papers by the number of authors. non-australian authors were included where a paper was written in collaboration with an australian author. it was found that 143 of the 307 total papers under consideration, or 47 percent were written by one author, 119 or 39 percent were written by two authors, 34 or eleven percent were written by three authors while eleven or four percent of the papers were written by four or more authors. over the period more than half of the papers were written collaboratively which suggests that in general, australian is researchers did not research independently but undertook research with others, most commonly with one other. after 1993 there was a decline in die number of papers written by one author and an increase in the number of collaborative papers. the data suggests that invisible colleges were established in australian is research from around 1993. note that non-australian authors were included where they co-authored with an australian. collaboration leads to the articulation and sharing of research traditions and the eventual development of invisible colleges. the most obvious way that collaboration encourages shared research traditions is through the supervisor/research student relationship, and it is likely that many of the collaborative links observed have arisen in this manner. when a research student graduated and found work elsewhere in academia the invisible college was extended. however where a university's policy towards the allocation of department of employment, education, training and youth affairs publication points is to use the 'straight' or 'adjusted count' and so not give full credit to all authors, this will act to limit cross-departmental and intervarsity collaboration. having established that collaboration is common and invisible colleges are likely, it is interesting to contemplate where the collaboration has taken place. 75 ajis vol. 5 no. 2 may 1998 number of authors, acis and ajis 1990 1991 1992 1993 year 1994 1995 1996 2 3 no. of authors 4+ figure 2 graph of number of authors in acis and ajis australian papers table 4 analyses acis and ajis australian papers by author collaboration. in this table one paper may be recorded within several categories where there were greater than two authors. only jointly authored papers with at least one australian author were recorded. it can be seen that 101 of a total of 174 papers recorded in this way, or 58 percent were written collaboratively where the authors came from within the same department of the same university. although departments differ in their policy regarding the addition of a supervisor's name to papers written by research students, it is likely that some of these papers were co-authored by research student/supervisor groups. it is possible that some papers jointly authored by a supervisor and research student where the student was employed as an academic at another department or institution may have stated the affiliation of the student as that of the supervisor. however, such an under-estimation of intervarsity collaboration may have been balanced by other papers. for example, some papers written under the guidance of a supervisor may have given only the student's name and affiliation. author nos within uni. dept within practit. orgai interdepartmental, same university different aust. uni. different uni. ausu aust. acadvpractit. acad7practit. 1 aus totals '90 aci s isn o'sea. t., 1 o v-;:* ajis '. ':'•'/". -r?''j. ' • -jr.-'• 'seas :;.t'0 '91 aci s '•••-il "14 ajis p ii 'pr .j!3 .-' ps '92 aci s ±:k ajis ;-:lif• i t,,g~. •'it'^'v '-.-. . •'". a ':m '93 aci s 14 1 1 2 18 ajis 2 1 3 '94 aci s 3 i 1 > 4 \32 ajis 1 2 3 '95 aci s ' • " • •4 ;'s .:>'-3 ;-:38 ajis 4 1 5 '96 aci s ;::"-l:8 • •••• i 3 im ajis 2 1 1 1 5 total s 101 8 15 24 11 11 4 174 table 4 analysis of acis, ajis joint australian papers by author collaboration only eight of the 174 papers, or five percent were collaborative practitioner papers and in fifteen of the 174 papers, or nine percent, authors collaborated within different departments of the same university. just 24 of the collaborative papers or fourteen percent were written by authors from different universities within australia. collaboration between different universities in australia and outside australia, and joint academic/practitioner papers where both parties came from within australia each accounted for eleven of the papers, or six percent finally, four of the 174 papers or two percent were written jointly by academics and practitioners where one came from australia and the other from outside australia. 76 ajis vol. 5 no. 2 may 1998 the data demonstrated that collaboration between colleagues in the same department of a university was more than four times as common as any other form of collaboration. this finding suggests that in addition to researchers working independently, they also worked in independent departmental units. moreover, over twice as much collaboration occurred between researchers in different universities than between different departments of the same university. this is remarkable given that, (a) the broad scope of is and the influence of many different reference disciplines offer considerable potential for interdisciplinary research, and (b) the finding discussed earlier that many universities had several departments where is research took place, yet little shared research occurred between them. if is research methods borrowed heavily on those from other disciplines such as computer science, management and sociology, they did so in australia without a great deal of interaction taking place with those disciplines. this finding has implications for both the quality of the research produced and the likelihood of researchers gaining access to grant funds. however, it is likely that the sizeable collaboration that has occurred between researchers in different universities within australia and outside australia has contributed to the development of a shared understanding of research traditions, and is evidence of invisible colleges. the data displayed patterns of collaboration between academics and practitioners. in general little collaboration took place between academics and practitioners despite the applied nature of is. it is interesting to note that there was more research published that resulted from collaboration between academics and practitioners than was published by practitioners. practitioners may have felt that the publication outlets were not appropriate forums. whatever the cause, it indicates a divide between the practitioner and academic is world in australia. comparison was made between the findings of this study and that of north american research. it was remarkable to observe that the proportions of australian papers with one, two, three or four or more authors were almost identical to those reported in a north american study (jackson and nath 1989). in the same study the 32 most prolific researchers authored 18 percent of the total research whereas in the australian study twenty researchers accounted for 21 percent of the total number of author credits. following a similar pattern, grover, segars and simon (1992) found that the fifteen most prolific universities authored over half the research examined, while lending and wetherbe (1992) reported that the most prolific five mis institutions accounted for eighteen percent of the published research. in comparison, in the australian study, the most prolific seven universities were responsible for 61 percent of the total author credits. it is worth noting that in general the north american studies made no differentiation between leading institutions and schools as mis academic research is commonly based in one school within each institution whereas in australia the current study found that is research is often undertaken within several departments of the one university. as a result although it was found that in australia the most prolific four percent of departments accounted for 29 percent of the total author credits there is no meaningful north american comparison. even though measurement has been made in different ways, in the comparisons made the most prolific universities and authors in australia accounted for either an absolute or suggested greater proportion of the is research output than was reported in north america. this finding and the finding that is research in north american universities tended to be based in one centre are indicative that at least some characteristics of australian is research are different to those in north america. it was reported in two comparative studies (lending & wetherbe 1992; vogel & wetherbe 1984) that there was less domination by the most prolific schools in the later study. from this finding it is surmised that the earlier in the establishment of an is scientific community, the greater the domination of the research output by a few universities, departments and authors. the australian findings are consistent with this, as australia clearly is at an earlier stage of development of its is scientific community. it is expected that over time, the leading universities, departments and authors will reduce their domination of the published research output in australia as the scientific community grows. all refereed authors published by acis and ajis during 1990-1996 were analysed by their employment category and whether they were from within australia. table 5 gives a breakdown of authors of refereed papers from ajis and acis by their employment group and geographic location. each paper appears only once. it can be seen from table 5 that 261 of the total of 382 papers analysed came from australian academic authors. this was over four times as large as any other group and represents 68 percent of the total. there was only one other sizeable group. a total of 61 or sixteen percent of the papers were written by academics with affiliations from outside australia. this means that academic authors dominated the publication outlets. only six percent of papers were written by practitioners and seven percent of the papers were written by practitioners in collaboration with academics. the analysis of the characteristics of all the practitioner papers in the two publication outlets corroborates the findings of the prior analysis that focussed on the pattern of collaboration in joint papers. there was only limited evidence of interaction between academic and practitioner researchers in the australian publication outlets, either from the exposure of academics to practitioner papers or in joint papers. this was particularly so when the practitioner or joint practitioner/academic papers from outside australia were excluded. 77 ajis vol. 5 no. 2 may 1998 author characteristics author characteristics aust academics 13 aust practitioners aust acad/practit o'seas academics o'seas practitioners o'seas acad/practit aust/o'seas academics aust/o'seas practitioners aust/o'seas acad/practitioners '90 aci ajis 91 aci s 22 :>.' 2 ajis 92 aci s si ajis aci s 93 -41 ajis 5 '94 aci s 39 2 4 7 3 ajis 6 '95 aci s 41 2 3 7 2 •0 1 ajis 9 1 '96 total s aci s 52 6 8 4 ajis 5 261 14 19 61 8 6 10 0 3 totals :,:28hbi 8 56 12 "f 13 78 17 382 table 5 analysis of acis, ajis papers by author characteristics papers jointly authored by academics from australia and outside australia accounted for ten papers, or only three percent of the total published papers. this result indicates that little general is collaborative research between australian academics and academics from outside australia was published in australia. this may be a result of little research of this nature being undertaken, or it could be that the collaborative authors preferred to publish their work in international journals as those journals have more prestige. as beguiling as the second possibility seems, it is considered unlikely as very few collaborative papers of this nature were found in leading international journals by one of the authors in work carried out for another purpose. even when the small group of eminent authors who publish in international rather than national outlets is remembered, there are few general is research invisible colleges that reach out from australia to researchers elsewhere. limitations; recommendations for future research an obvious limitation of the study is the restricted number of publication outlets used for the source of the data. a related limitation is the exclusion of specialised conferences and journals. as has been noted, a very small number of eminent australian is researchers identified most closely with the international is scientific community, published in international journals and participated in international conferences. these authors have been missed by the current study which examined australian publication outlets only. it can also be seen that use of the normal count method will give rise to different results than if use was made of the adjusted or straight count. further investigation will need to determine whether australian specialist is publication outlets are similar to the two mainstream outlets for the characteristics examined in the current study. the next step will be to identify the research traditions in use by the is academic community. it will be possible to specify the research scope, methodology and methods used by the australian mainstream is research community by analysing acis and ajis papers for these characteristics. this task is currently being undertaken by the authors. conclusions the authors who published in ajis and acis were willing to comply with the research traditions considered acceptable by ajis and acis, and these publication outlets are largely shaping the research traditions of the australian mainstream is scientific community. if at least one publication of a paper by an australian academic or practitioner in one of the two publication outlets is taken to be the criterion of membership of the australian mainstream is research community, it was found that the community comprised of around 36 universities, 96 departments and 314 authors. it needs to be remembered however, that some of these universities and departments do not currently exist in the same form. seven universities, four departments and twenty authors were identified as the most influential by being both the most prolific and writing a big proportion of the total published papers. the study demonstrated that the australian mainstream is research output was dominated by a very few universities, departments and authors, considerably more so than in north america. however, as the research community grows, this domination is expected to diminish. the australian research community has had steady growth and has more than doubled researcher numbers between 1993 and 1996. evidence from collaboration patterns indicated that invisible 78 ajis vol. 5 no. 2 may 1998 colleges have been present in australian is research at least from 1993. however, the is presence in many universities is fragmented and it seems likely that benefits will arise from rationalising this arrangement. australian researchers tended to work collaboratively, and the most common form of collaboration was with just one other researcher from within the same department. it seems likely that many of these collaborations were between supervisor and research student. it was found that australian is researchers worked within isolated departmental units with little contact with other disciplines. where researchers work outside their immediate area of expertise, such a research style is likely to reduce the opportunity for quality research. as is has an interdisciplinary nature, this is likely to be an obstacle for the future development of the australian mainstream is scientific community. the majority of the papers analysed were authored by australian academics. there was little evidence of collaboration between the australian is academic community and academics or others from outside australia, an indication that the mainstream is research invisible colleges that exist are largely limited to australia. a sizeable degree of collaboration has occurred between and among universities from within australia which has contributed to the establishment of an is research tradition, and allows invisible college networks to be traced. acis and ajis were found to be very largely academic publication outlets, with 84 percent of the papers published being written solely by academics. little practitioner research was published in the outlets examined. furthermore, little research collaboration took place between academics and practitioners. if is is an applied science, academics are going into industry as external observers rather than working with is personnel already placed in the organisations. this finding may have ramifications for both the quality and methodology of the research conducted, particularly for interpretivist studies. it was found that the invisible colleges of the australian non-specialist is research community were narrow both in their definition and distribution, but were growing. a major implication of the study findings is that australia is at an early stage in the development of a mainstream is scientific community. this empirical evidence supports a frequently expressed view. references arnott, d. (1995) australian conference on information systems: an evolving charter, version 1.0, in possession of the author. cue, d. & urquhart, c. (1997) research benchmarking report 1997 (draft), department of information systems, university of melbourne. culnan, m. (1987) "mapping the intellectual structure of mis, 1980-1985: a co-citation analysis", mis quarterly, vol 11, no 3, pp 341-353. de solla price, d. (1961) science since babylon, new haven: yale university press, de solla price, d. (1963) little science, big science, new york: columbia university press. farhoomand, a. (1987) "scientific progress of management information systems", in r. galliers (ed.) information systems research: issues, methods and practical guidelines, oxford : blackwell scientific publications. grover, v., segars, a. h. & simon, s. j. (1992) "an assessment of institutional research productivity in mis", data base, vol 11, no 2, pp. 5-8. jackson, w. m. & nath, r. (1989) "publication patterns of mis researcher", interface, vol 11, no 2, pp 1520. keen, p. (1980) "mis research: reference disciplines and a cumulative tradition", proceedings of the first international conference on information systems, philadelphia, research and education in information systems, the society for mis, pp 9-18. king, j. l. (1993) "editorial notes", information systems research, vol 4, no 4, pp 291-298. latour, b. (1987) science in action: how to follow scientists and engineers through society, milton keynes: open university press. lending, d. & wetherbe, j. (1992) "update on mis research a profile of leading journals and us universities", data base, vol 23, no 3, pp 5-11. lindsey, d. (1980) "production and citation measures in the sociology of science: the problems of multiple authorship", social studies of science, vol 10, pp 145-162. niemi, a. (1987) "institutional contributions to the leading finance journals, 1975 through 1986: a note", journal of finance, vol 42, no 5, pp 1389-1397. orlikowski, w. & baroudi, j. (1991) "studying information technology in organizations: research approaches and assumptions", information systems research, vol 2, no 1, pp 1-28. ridley, g. (1996) "establishing australian is research traditions: identifying appropriate quality publication outlets and the significance of conference publications", proceedings of the australasian conference on information systems, vol. 2, pp 567-578. 79 ajis vol. 5 no. 2 may 1998 shanks, g., rouse, a. & arnott, d. (1993) "a review of approaches to research and scholarship in information systems", working papers series 3/93, department of information systems, monash university. shim, j. p., english, j. b. & yoon, j. (1991) "an examination of articles in the eight leading management information systems journals: 1980-1988", socio-economic planning science, vol 25, no 3, pp 211219. standing, c. (1996) "the journal of holistic research in information systems; call for papers", url http://www.bs.ac.cowan.edu.au/publications/jonris.html. sutton, d. (1997) "the eighth australasian conference on information systems, 1997", url http://business.city .unisa.edu.au/acis97/. vogel, d. & wetherbe, j. (1984) "mis research: a profile of leading journals and universities", data base, vol 16, nol,pp3-14. walsham, g. (1995) "the emergence of interpretivism in is research", information systems research, vol 6, no 4, pp 376-394. acis proceedings monash university (1990) proceedings of the first annual conference on information systems, february, melbourne. university of new south wales (1991) proceedings of the second annual conference on information systems and database special interest group, february, sydney. macgregor, r., clarke, r., little, s., gould, t. & ang, a. (eds) (1992) proceedings of the third australian conference on information systems, october, wollongong: dept. of business systems, university of wollongong. department of commerce, university of queensland (1993) proceedings of the 4th australian conference on information systems, september, brisbane. shanks, g. & arnott, d. (eds) (1994) proceedings of the 5th australasian conference on information systems, september, melbourne: department of information systems, monash university. pervan, g. & newby, m. (eds) (1995) proceedings of the 6th australasian conference on information systems, september, perth: school of information systems, curtin university. keen, c., urquhart, c. & lamp, j. (eds) (1996) proceedings of the 7th australasian conference on information systems, december, hobart: department computer science, university of tasmania. 80 microsoft word vol 15 2 final.doc australasian journal of information systems volume 15 number 2, 2009 149 online trust mechanisms: a domain study of online auctions glenn bewsell school of information systems and technology university of wollongong email: gbewsell@uow.edu.au abstract this research investigates mechanisms that impact trust at online auctions. a qualitative study is discussed within this paper to identify mechanisms that impact trust at online auctions. the mechanisms identified are discussed within normative and grounded theoretical approaches to trust. a result of this discussion i seek to answer the question: what factors do people perceive as important to increase online trust prior to an online transaction? this discussion will help professionals understand the factors that emerge from normative and grounded theoretical approaches to online trust research. a better understanding of online trust adds to the body of theory and can be used to improve the design of online auctions. introduction online auction users are reliant on technology. technology allows items to be offered for auction and transactions to be formed between online auction users for the exchange of goods and consideration. technology is able to replace people, and users may not know when they are dealing with another person or a piece of technology such as a computer program. at online auctions, users are represented by an online identity and users can be virtually anonymous to each other (zhao et al., 2006). these users may be real, misrepresented, honest or opportunistic. buyers and sellers at online auctions are vulnerable and trust is required when entering into a transaction. a better understanding of trust is required, especially in relation to online auction transactions. the motivation for this research is to identify mechanisms that impact online auction trust. online auction trust is considered as a broad construct as it “spans the end-to-end aspects of e-business …. it extends beyond the web site and includes all electronic networks used by the firm” (shankar et al., 2002 p.326). this paper presents an analysis of qualitative data collected as part of an experimental research study that investigated trust in online auctions. this research seeks to answer the question: what mechanisms do people perceive as important to increase online trust prior to a consumer-toconsumer online auction transaction? this research is important for practitioners and researchers to gain a better understanding of trust. trust is an important, broad and little understood construct in e-commerce which needs to be more australasian journal of information systems volume 15 number 2, 2009 150 effectively understood in order to build better e-commerce models and to attract people to transact rather than browse online. theoretical perspective this section provides the theoretical perspective relevant to this research. trust and power are believed to be two key constructs in understanding what drives and inhibits people to transact online. these constructs are discussed in turn. trust at online auctions, buyers and sellers can be virtually anonymous (zhao et al., 2006) to each other and trust is required due to the possibility of exploitation and exclusion (lind, 1995; van den bos et al., 2002). trust is the “consumer’s willingness to rely on the seller and take action in circumstances where such action makes the consumer vulnerable to the seller” (jarvenpaa et al., 1999). a buyer at an online auction is faced with the choice to “cede authority and subsume to the power” (van den bos et al., 2002 p.53) of a seller and bid with the purpose of entering into a transaction. the objects of trust that are of interest to this research are “the things that can be trusted” (nooteboom, 2002 p.10) and those things that people perceive will increase online trust. the normative model of trust used in this research is presented in figure 1. this normative model of trust includes: i) trusting beliefs; ii) attitude towards trust; iii) intention to trust; and iv) behavioural trust. this model of trust appears consistent with the theory of planned behaviour (fishbein & ajzen 1975; ajzen 2005) where a relationship exists between attitudes and behaviour. the five bases of trust identified within the model of trust are: personality-based trust; institution-based trust; cognitive-based trust; calculative-based trust; and knowledge-based trust. each of these bases of trust will be discussed. figure 8: normative model for trust (based on mcknight et al., 1998) australasian journal of information systems volume 15 number 2, 2009 151 personality-based trust is a “general tendency to trust others (rotter, 1967)” (mcknight et al., 1998 p.475), a “trust propensity” (koufaris & hampton-sosa 2002 p.379) or a ‘‘general willingness based on extended socialization to depend on others’’ (ridings et al., 2002 p.278). this propensity or tendency to trust others indicates that people will have a tendency to trust an online auction site and an online auction transaction. institution-based trust “reflects the security one feels about a situation because of guarantees, safety nets, or other structures (shapiro, 1987; zucker, 1986)” (mcknight et al., 1998 p.475). institutionbased trust or institutional trust contains three dimensions: i) situational normality or the situation appears normal such as websites that appear normal (bhattacherjee 2002; koufaris & hamptonsosa 2002; mcknight et al., 1998; mcknight et al., 2002; pavlou & gefen 2004; pavlou et al., 2003); ii) structural assurances such as third-party structures and contracts (mcknight et al., 1998; mcknight et al., 2002); and iii) facilitating conditions such as conditions that match market expectations including shared standards, beliefs, and values (mcknight et al., 1998). institutionbased trust will help to assure online auction users that the online auction looks normal and appropriate structures are in place to support online auction transactions. the major dimensions of institution-based trust of interest to this research are situational normality and structural assurances, refer to table 1. structural assurances situation normality guarantees safety nets recourse regulations other procedures normal appearance proper order credit card guarantees escrow services such as paypal legal system, contracts, promises, plans for contingenc ies and sanctions specified (eg. site expulsion) technical and professional standards, codes of ethics, standards of practice, precedents, judicial decisions and international law buyer driven certification, feedback mechanisms for buyers and sellers, relationships in the same markets and community, certifications and accreditations web site appearance, presentation and specification of the item being auctioned (images, description and specifications) privacy, security, conditions of use, user agreement s, identifiers and socially constructe d roles table 2: institution-based trust (based on bhattacherjee 2002; koufaris & hampton-sosa, 2002; luhmann, 1979; mcknight et al., 1998; mcknight et al., 2002; nooteboom, 2002; pavlou & gefen, 2004; pavlou et al., 2003; shapiro, 1987) situation normality is a belief based on a normal appearance and proper order of things (bhattacherjee 2002; koufaris & hampton-sosa, 2002; mcknight et al., 1998; mcknight et al., 2002; pavlou & gefen, 2004; pavlou et al., 2003). at online auctions things such as website appearance, the presentation of items, user agreements and socially constructed roles should provide indicators for people to assess whether things appear normal and orderly. within situation normality, this research considers the distinction between the normal appearance of the online auction site and the proper order of the site that would facilitate a successful transaction. australasian journal of information systems volume 15 number 2, 2009 152 structural assurances are the safeguards in place such as guarantees, safety nets, regulations, promises, legal recourse and other procedures (mcknight et al., 1998; mcknight et al., 2002; nooteboom 2002; pavlou & gefen, 2004; pavlou et al., 2003; shapiro, 1987). structural assurances at online auctions may include: credit card guarantees provided by a financial institution when payment is made by credit card; safety nets provided by escrow services; and transactional feedback from other users. cognitive-based trust is based on “rapid, cognitive cues or first impressions” (mcknight et al., 1998 p.475) or “shared cognitive structures, which yield empathy” (nooteboom, 2002 pp.12-13). in figure 1, cognitive-based trust is represented as an oval to indicate cognitive processes (mcknight et al., 1998). two types of cognitive-based trust are categorisation processes and illusions of control, refer to table 2. categorisation processes can provide high levels of trust in an online auction where the other user shares common values, shares common goals, have a good reputation and possess certain stereotypes that are appealing (kramer, 1999; mcknight et al., 1998). furthermore illusions of control may also provide high levels of trust at an online auction such as some small action to confirm ones initial trusting belief (mcknight et al., 1998). categorisation processes illusions of control shares common values, shares common goals, certain stereotypes that are appealing and good reputation. similar interests (e.g. product, types, auctions, email hosts, type of replies) questions to the seller (e.g. responsiveness to questions, quality of reply), and ability to take transaction off the auction site. table 3: cognitive-based trust knowledge-based trust is based on an interaction history and the first hand knowledge that parties have of each other (mcknight et al., 1998) or observation of the trustee (nooteboom, 2002). buyers and sellers that have previous interactions with each other are able to base their perceptions of trust on this knowledge. at an online auction, the buyer and seller are virtually anonymous and are not expected to have firsthand knowledge of each other; however, they may have firsthand knowledge of the online auction site. calculative-based trust is a “trusting stance” (mcknight et al., 1998 p.475) or a “calculative selfinterest” (nooteboom, 2002 p.13). a buyer or a seller may consider that things will probably “turn out best when they are willing to depend on” (mcknight et al., 1998 p.478) the transaction. a buyer or a seller may consider or calculate the potential benefits and losses of the other party’s cooperation or non-cooperation. benefits can encourage good behaviour and penalties can discourage bad behaviour. at an online auction, a benefit is good transaction feedback and a penalty is a fine or a suspended trading account. power power is a broad construct that includes various types of power. french & raven (1959) identified various types of social power as: coercion; reward; legitimate (position); expert; referent; and informational power. at an online auction the buyer, the seller, and potentially the mediator may use power to influence other parties and try to resolve a dispute. power may be more balanced or less balanced. power will be more balanced if the resources are more equally shared between the parties, and power will be less balanced if one party has access to more of the resources. the resources can include the goods, the consideration, or special skills and australasian journal of information systems volume 15 number 2, 2009 153 knowledge. the balance of power related to an online auction transaction changes as goods and consideration are transferred between the parties. power at an online auction is “the ability to influence the decisions or actions of others” (thorelli, 1986 p.38). the buyer and the seller may try to communicate to influence each other or they may not. at an online auction the buyer and seller may trust a powerful third party such as the auction site or an independent third party. a relationship between power and trust has been supported by previous researchers and these researchers support that power is antecedent to trust (duarte & davis, 2000; geyskens et al., 1998; thorelli, 1986). this section has provided the theoretical perspective for this research. trust, the bases of trust and power have been discussed in relation to online auctions, and a relationship between power and trust has been established. research methodology to investigate what mechanisms people perceive as important to increase online trust in an online auction situation, qualitative feedback was requested as part of an experimental study. this paper discusses this qualitative feedback in order to shed some contextual ‘light’ on the motivation of the research subjects. the qualitative feedback includes items that may increase the participant’s level of trust in dealing with sellers at an online auction. the experimental scenario undergraduate and postgraduate participants from a leading university in sydney were provided a booklet for use in the judgmental experimental design study. each booklet contained an introduction, instructions, scenario, a set of screenshots of the online auction website and a questionnaire. each participant was randomly allocated one of four levels of power. the introduction, instructions and basic scenario were the same for each booklet. these sections introduced the participants to the study, familiarised participants with the setting, provided any information required for the study and advised each participant how to proceed. the introduction presented an online auction as the domain of interest and allowed information to be provided to participants to control for their level of knowledge. the type of online auction presented to participants was the normal or english auction where a seller placed an item for sale and buyers submitted bids on the item. for a bid to be successful, it must be the highest bid at the close of the auction and exceed the minimum acceptable price (that is the reserve). an example of this type of auction which participants may be familiar with was ebaytm (ebay 2006). in the introduction, participants were introduced to the u.s. federal trade commission’s (ftc’s) report about online auction fraud. reports such as the ftc’s report were in the public domain and online auctions were becoming more newsworthy. prior to the experiment participants may or may not be aware of this report; however, the general level of awareness of problems related to online auction fraud was expected to increase over time. the risk of not providing public domain information from the report was that participants would have unequal levels of knowledge and participants would become aware of this report or similar reports some time later. the provision of pertinent public information from the report provided some control over the participant’s level of knowledge and this may avoid the need to rerun multiple experiments as the level of knowledge in the public domain increased. the participant’s level of knowledge was a variable in the outer core of the research model and control of this variable was reasonable. one question in the questionnaire tested for the participant’s level of knowledge. australasian journal of information systems volume 15 number 2, 2009 154 the instructions section advised participants a research purpose, the level of analysis that would be performed, information in response to participant’s potential privacy concerns and how to complete the booklet. the online auction site was carefully presented within the booklet to avoid constructs of disinterest conflicting with constructs of interest. for example information about the seller was not disclosed on purpose. the online auction site had a generic domain name ‘auction.com.au’ that did not exist on the internet. the online auction site was described within the booklet as relatively new and had been operating for a short period of time. participants had not used the site previously and would not know anyone who had used the auction site. site reputation and word-of-mouth (wom) referrals were non-existent and therefore controlled. the scenario described the setting, advised the participant the layout of the balance of booklet, asked the participant to familiarise themselves with the booklet and website printouts, asked the participant to decide whether to place a bid at the online auction, and then asked the participant to complete the questionnaire. every booklet included screenshots of the online auction site’s home, conditions of use, privacy, security and registration web pages. each web page included navigation tabs for clarity. every booklet included one of four different variations of dispute resolution procedures. the four variations of dispute resolution procedures were used to support a factorial research design, to manipulate power and to affect trust levels within the experiment. one variation of the booklet was used for control and dispute resolution was not mentioned within this booklet. the other three variations of the booklet contained one of three different dispute resolution web pages. the four variations, treatments or manipulations of dispute resolution procedures and power are presented in table 3. each treatment contains different dispute resolution functionality and ‘power to resolve’ a dispute. this table acknowledges that power may reside in different places, the relationships between the parties may change and the bases of power may be different. a higher ‘power to resolve’ was expected to resolve more disputes, and a lower ‘power to resolve’ was expected to resolve less disputes. some manipulations included the availability of a third-party role, as a third-party was expected to increase ‘situation normality’ by providing a proper order to an online auction and mastery over ambiguous circumstances. mediation was also expected to increase structural assurances by providing parties access to a form of recourse. ‘situation normality’ and structural assurances were expected to reduce complexity and increase trust. treatment dispute resolution functionality ‘power to resolve’ m1 process is implied (participants perceptions) m2 buyer and seller in a dyadic relationship low m3 mediator available for a triadic relationship (mediator can recommend a resolution) medium m4 mediator available for a triadic relationship (mediator can make and enforce their final decision to solve conflict) high table 4: experimental manipulations of power with dispute resolution mechanisms every participant was assigned the role of the buyer. participants were asked to assume that they needed to purchase an item (a motor vehicle) of the type offered at an online consumer-to-consumer auction, so that the participant’s needs and wants were established. participants understood the australasian journal of information systems volume 15 number 2, 2009 155 seller to be a private individual and that the current bid reflected a fair price. as the auction was closing soon, participants needed to decide whether to place a bid. a motor vehicle was expected to have a high potential for information uncertainty compared with that of a commodity item (akerlof, 1970; ba & pavlou, 2002; mishra et al., 1998). information related to a motor vehicle could include condition, colour, accessories, type, make, model, year, and the expertise of the seller. once a participant was familiar with the website and scenario they completed the questionnaire. the questionnaire consisted of three main parts: i) a general section to capture respondent demographics such as gender, previous participation in online auctions, and knowledge of online auctions; ii) a quantitative section that captured empirical data; and iii) a qualitative section that captured additional insights into a participant’s beliefs. this paper focuses on the qualitative section of the study. in the qualitative section of the questionnaire, participants were able to identify any items that they felt would help increase or decrease trust in dealing with sellers at the online auction. mechanisms identified by participants to increase or decrease online trust were abstracted or grouped into key themes. these themes emerged from the data as in a grounded theoretical approach or were derived from the literature as in a normative approach. to allow themes to emerge from the data, the qualitative data was codified using ihmc cmaptools version 4.09 software (ihmc, 2006). the responses were codified as concepts, the concepts were arranged to highlight commonality and abstracted to form themes. the concepts and abstractions were linked and verbs used to help explain these linkages. the abstractions were then arranged around the main concept of online trust. this technique allowed the concepts and themes to emerge from the data in a ‘bottom up’ approach. the normative view of online trust was considered by placing participant’s qualitative responses into the main normative themes of online trust derived from the theory; refer to tables 1 and 2. the qualitative responses were also analysed using participant demographics to determine if differences existed between the types of participants. qualitative data analysis and results no discernable differences were found in the demographics between the thirty-seven participants that responded to the qualitative section of the research and the one hundred and one participants that participated in the overall research design. the qualitative results presented here appear to be representative of all of the study’s participants. results will now be discussed based on the thirtyseven qualitative responses received. 27.0% of all participants had participated in one or more online auction transactions. participants had purchased or sold items in 92 online auction transactions and 5 of these transactions had resulted in a dispute. in other words a dispute had been experienced once in every 18.4 online auction transactions. participant demographics for this study are presented in table 4. on average participants (p) rated their knowledge of online auctions slightly lower than ‘average’ as shown in figure 2. participants that had no previous transactions (npt) at an online auction rated themselves on average as having poor to average knowledge of online auctions whereas those that had previous transactions (pt) rated themselves on average as having average to good knowledge of online auctions and closer to good knowledge. participants that had purchased and sold (ps) items at an online auction on average rated themselves as having more knowledge (good to very good) than respondents that had only purchased (pu) at an online auction (average to good). participants that had completed more than five transactions (>5) on average rated themselves as having more australasian journal of information systems volume 15 number 2, 2009 156 knowledge (good) than people that have less than five transactions (<5) as average to good. the averages support that a person that had completed more online auction activities perceived themselves as having more knowledge of online auctions than a person that had completed less online auction activities. a person that had previously visited, interacted with sellers, bid, purchased, and sold at an online auctions had more experience and greater perceived knowledge than a person that had not participated in all of these activities. in general a participant’s reported knowledge of online auctions appeared to increase as the types and number of their activities increased. demographic variable participants (n=37) frequency % gender male 24 64.9 female 11 29.7 missing data 2 5.4 online auction activity participants that purchased 10 27.0 participants that sold 4 10.8 participants with no experience 27 73.0 missing data 0 0.0 table 5: study demographics figure 9: knowledge rating legend: p=participants, npt = no previous transactions, pt = previous transactions, pu = purchased, ps = purchased and sold, >5 = greater than 5 transactions, <5 = less than 5 transactions. concept mapping tools (ihmc 2006) were used to analyse the qualitative data. eight major themes emerged from the concept mapping techniques as: sellers (35.1%); processes (32.4%); assurances/guarantees (32.4%); references/testimonials (18.9%); website reputation and associations (21.6%); security (10.8%); services (8.1%); and site presentation (5.4%). these themes and associated concepts are presented in the concept map of online trust, refer to figure 3. each theme will now be discussed in turn. australasian journal of information systems volume 15 number 2, 2009 157 australasian journal of information systems volume 15 number 2, 2009 158 the theme of sellers included history details, identification of the seller, contact with the seller, sellers’ ratings, and transacting with sellers offline. to increase online trust, participants identified the need for: better access to seller histories of previous sales, detailed sales information, feedback related to those transactions and seller’s ratings; better identification of sellers with linkages to bank account details, business locations and credit card details; the ability to contact the seller offline to decide if the seller is trustworthy; and physical contact with the seller and display of the item at the seller’s premises. the processes theme related to terms and conditions, the seller’s agreement, dispute resolution processes, recourse, binding processes and more detailed processes. to increase online trust, participants identified the need for: more detailed terms and conditions; the seller’s agreement with the online auction site; clear dispute resolution processes; binding arbitrary mediation; and processes that provide an ability to sue and penalise the seller. assurances/guarantees theme was the auction site taking responsibility for transactions, positive declarations, assurances similar to credit card guarantees, escrow and insurance. to increase online trust, participants identified the need for: the online auction site to provide binding guarantees and assurances; access to insurance in case of a dispute; and safe transaction assurances such as credit cards, third party providers and escrow services. references/testimonials theme was references and testimonials from friends, acquaintances, relatives and other users. to increase online trust, participants mainly needed advice from users, friends, relatives and acquaintances who had used this auction site before. other items identified were a past user forum and trustworthy testimonials. the website reputation/associations theme related to the size of the site, how long the online auction had been operating and other organisations that were associated with the site. to increase online trust, participants identified the need for the online auction site: to have been in business longer; to be more well-known; to be a larger organisation; and to have associations with wellknown and trustworthy companies. the security theme was secure systems and data encryption for information transfer. to increase online trust, participants identified the need to clearly identify details related to security such as encryption and security certifications. the services theme was required for mediation services, services to verify goods, and trusted delivery services. to increase online trust, participants identified the need for services that supported the transaction such as mediation services, verification services and trustworthy delivery services. finally, the presentation theme related to the presentation and layout of the website. two participants identified that the layout and presentation of the site and web pages could be improved to increase online trust. the qualitative data and results were analysed based on participant demographics collected as part of the questionnaire. the participant demographics considered were: transaction history such as transacted or not transacted; gender; and other such as previous disputes. transaction history participants that had not previously transacted online were concerned with: site reputation/associations (25.9%); the seller (22.2%); references/testimonials (22.2%); processes (18.5%); and assurances/guarantees (18.5%). participants that had previously transacted online were australasian journal of information systems volume 15 number 2, 2009 159 mainly concerned with the seller (70.0%), assurances/guarantees (40.0%) and penalties (20.0%). penalties was a concept and related to the processes theme. common items between the groups were transaction specific items related to the seller, assurances/guarantees and processes themes. although these items were common between the groups the percentages differ greatly. seventy percent (70.0%) of participants that had previously transacted at an online auction were concerned with the seller and forty percent (40.0%) concerned with assurances/guarantees, whereas the percentages in the other group were much smaller. the participants that had not previously transacted at an online auction were concerned with more generic themes related to the auction site and these were: site reputation; site associations; and references/testimonials. gender feedback received was checked for similarities and differences between the genders. major items to improve trust for both genders were assurances/guarantees and seller themes. males most frequently suggested that items associated with the seller (37.5%) were required to increase trust. the seller’s history, identification of the seller, contact with the seller, other buyer’s ratings of the seller, information related to the seller’s previous sales and other seller information may help to increase trust. assurances/guarantees theme (29.2%) was the second most frequently suggested theme to increase trust. the assurances/guarantees theme suggested by males included auction site guarantees, safe transaction methods such as visa, escrow and insurance services. other ways to increase trust were associations/reputation (12.5%), processes (12.5%), and security (12.5%) related items. females most frequently suggested that references or testimonials (36.4%) from family, friends and others were necessary to increase trust. auction site reputation and associations (36.4%) was the next most frequently suggested theme to increase trust. reputation and associations related to how long the auction site had been trading, becoming a world-wide known site, and becoming bigger like ebay. seller (27.3%) related items were also identified by females as a way to increase trust. the seller related items included: other buyers providing an indication of the seller’s trustworthiness; seller history; and contacting the seller to buy offline after finding the price is good. processes (18.2%) were mentioned by females as a theme to increase trust. sellers, processes and association/reputation were common to both groups but the percentages were quite different. whereas males were mainly concerned with transaction specific items, females were more concerned with the auction site reputation and associations. other other groupings of participants were considered based on previous disputes and research manipulations. participants that had previously experienced a dispute thought that the seller (100.0%) was the major item to increase trust. seller related items included: the ability to make contact with the seller; the ability to identify the seller; and the physical display of the item for auction at the owner’s premises. the auction site should be able to sue the seller, if necessary. other analysis considered groups based on the research manipulations or experimental treatments assigned. the results based on these groups were very similar. it appeared that the manipulation presented to participants had not impacted the items considered to increase trust. australasian journal of information systems volume 15 number 2, 2009 160 normative approach the normative approach was considered and the qualitative responses were placed within theoretical groupings (tables 1 and 2), refer to table 5. the highest rating categories were: other procedures (37.8%); categorisation (32.4%); recourse (32.4%); and proper order (29.7%). the two highest rated response items were: references from friends and acquaintances with seven responses; and procedures to capture and access detailed information related to the seller with six responses. items that received three responses were: seller identifiers; improved registration procedures; trusted third-parties; associations and affiliation with known brand names; site reputation; auction site liability; encryption and security certification; and enhanced credibility or verification of goods and services. no participants identified regulations to increase trust at an online auction. when participant demographics were considered the four highest rating categories were evident throughout, except for experienced online auction users. males identified: other procedures (50.0%); proper order (37.5%); and recourse (33.3%). in the highest rated category males identified the need to: improve user registration by linking registered users to a credit card or bank account; enhance credibility or verification of goods and services; and capture and access detailed seller information. in addition, males identified that online auction sites needed to accept liability for transactions formed by them. females identified: categorisation (54.5%); and recourse (27.3%). in the highest rated category females identified the need for: references from friends and acquaintances; association or affiliation such as brand names and trusted companies; and site reputation. experienced online auction users identified that other procedures (70.0%) and recourse (30.0%) were major items to increase online trust. these experienced users identified that credit card and bank account details were important for user registration and procedures were required to capture and access detailed seller information. experienced online auction users had not identified any items that would be categorised as proper order. proper order was notable for its absence. inexperienced online auction users identified: proper order (40.7%); categorisation (37.0%); recourse (33.3%); and other procedures (25.9%). discussion this section will discuss potential users of online auctions; the grounded theoretical approach to online auction trust; the normative approach to online auction trust; compare the two approaches to online auction trust; the potential impact of the manipulation; and demographic differences. one non-user of online auctions indicated that they would not trust an online auction because they did not trust the online environment. this finding supported the claim of tan & thoen (2002) that people needed to trust the virtual environment to engage in an online transaction. this participant was not likely to become a user of online auctions; the other twenty-six (26) non-users were considered potential users of online auctions. in other words 96.3% of the non-users were potential users of online auctions if sufficient improvements were incorporated to increase online auction trust. the grounded theoretical approach used concept mapping and this approach allowed the data to be organised in new ways that provided new insights into online auction trust. three of the eight australasian journal of information systems volume 15 number 2, 2009 161 themes were identified for discussion: assurances/guarantees; the importance of building trust through sellers; and processes. category no. of respondents (% of participants) sub-category (no. of responses per sub-category) other procedures 14 (37.8%) procedures to capture & access detailed seller information (6) improved registration procedures: use credit card/bank account (3) encryption & security information certification (3) enhance credibility or verification of goods & services (3) procedure for fines (1) safe transaction procedures (1) delivery methods (1) categorisation 12 (32.4%) references from friends & acquaintances (7), association or affiliation eg brand names, trusted companies (3), site reputation (time trading, size, trustworthiness) (3) feedback related to sellers (2), location of seller (1) recourse 12 (32.4%) auction site liable and no immunity (3), clear procedures for recourse (2) mediation service (1) dispute resolution on seek court action (1) precise terms & conditions (1) actions taken against those that do not follow instructions or guides (1) be able to sue the seller (1), assurances for both sides (1), dispute resolution statistics (1) proper order 11 (29.7%) trusted 3rd party (3), seller id./seller information (3), security, encryption, safe transaction (2), terms and conditions, sellers agreement (2), mediation service (1) normal appearance 4 (10.4%) site layout (1), case history (1), more attractive web pages (1), provision of statistics, examples, information on dispute resolution (1) calculative 4 (10.4%) lower value items (3) seller fines (1) illusions of control 3 (8.1%) contact with seller (2), user forum (1), guarantees 2 (5.4%) more guarantees – binding (1), credit card guarantee (1) safety nets 2 (5.4%) escrow (1), insurance (1) regulations 0 (0.0%) table 6: qualitative responses a priori australasian journal of information systems volume 15 number 2, 2009 162 the assurance/guarantees theme included: transaction assurances; web site guarantees; and safety nets such as insurance and escrow services. users wanted improved identification of users and online auction site assurances to increase online auction trust. users appeared to want an online auction site to demonstrate their respect for online auction communities, to be more responsible for transactions formed by their site and to provide adequate assurances that demonstrated the auction site’s trust for their communities. the assurance theme included elements of recourse. zhao et al., (2006) expressed the concern that “[sellers] can take the money and run” (p.72), and these assurances should help to reduce this concern and increase trust. this concern may be true for items that have a high information asymmetry or all online auction items. the importance of building trust through sellers was supported with things that were linked to sellers, rather than the more personal qualities and personal assurances of a seller that could be assessed in an offline or face-to-face transactional context. this finding supported tan & thoen (2002) that electronic commerce trust is ‘trust in the transaction’. the processes themes were: the online auction site taking more responsibility in ensuring that any transactions formed were completed to the satisfaction of the parties involved; providing suitable dispute resolution procedures; assisting customers with dispute resolution; providing access to thirdparty mediation; and ensuring that sellers accepted their responsibilities. some online auction sites do provide some level of access to third party mediation services; however, this research indicated that well integrated operational processes were required to support dispute resolution mechanisms and recourse. this finding may support that processes need to be fair (van den bos et al., 2002). further theme abstraction supported that power was a higher level theme. power included the themes of assurances/guarantees and processes. this finding supported: that people transacting at online auctions would like a powerful party that was not part of the immediate transaction to ensure that sellers were able to be identified; that processes were in place and operational; and that powerful parties provided assurances and guarantees. powers that were provided at online auctions may need to be balanced so that buyers do not “engage in strategic behaviour” (zhao et al., 2006 p.72). this research supported that better assurances and guarantees would improve users’ perceptions of trustworthiness and impact their trust beliefs. governmental regulations were not identified by this research as an item to increase online auction trust. the lack of identified regulation improvements may indicate: that existing regulations did not impact online auction sites because sites limited their responsibility to the formation of transactions; that existing regulations were adequate to support online auction transactions; or that online auction sites had the best ability to increase trust at online auctions. this research supported the concept of guardians of trust and the need for guardians to deter or limit opportunism (shapiro, 1987) and that auction sites were perceived to be the primary guardian of trust. many of the processes identified could be considered as control procedures and perceived behavioural controls. for example, better seller identification can make it easier for a buyer to physically identify a seller. this finding is supported by tan & thoen (2002) where sufficient controls need to be in place and these control procedures need to be understood and shapiro (1985) acknowledged the need for control of impersonal trust. perceived behavioural controls are supported by the theory of planned behaviour (fishbein & ajzen 1975; ajzen 2005). tpb supported relationships between perceived behavioural controls and trust as an attitude and perceived behavioural controls and intention to trust (fishbein & ajzen 1975; ajzen 2005). australasian journal of information systems volume 15 number 2, 2009 163 the normative approach identified ‘other procedures’, categorisation, recourse and proper order as the major items to increase trust. ‘other procedures’, recourse and proper order supported structural assurances within institution-based trust, and categorisation supported cognitive-based trust. in other words, most items identified by this research supported that institution-based trust and cognitive-based trust impacted online auction trust. the allocation of things identified by participants to categories and sub-categories pertaining to the normalised view of online trust was not intuitive. for example security could have indicated: i) a proper order within situation normality; ii) a technical standard or standard of practice and hence a regulation within structural assurances; or iii) an accreditation or a certification and hence another procedure within in structural assurances. this research indicated that the integrated model of online trust proposed by zhang & zhang (2005) could be investigated as a potentially better model of trust. this integrated model of trust included institution-based trust factors consistent with the normative model of trust used within this research and the trustor, trustee and trustee website as trust factors. themes were compared between the grounded theoretical and normative approaches to online auction trust. the comparison indicated similarities and differences between these two approaches to trust. items to increase online auction trust were able to be categorised using each approach to trust. both approaches identified recourse as an item to increase online auction trust. the assurance/guarantees theme from the grounded theoretical approach included the guarantees, safety nets and elements of recourse themes from the normative approach. clear procedures for recourse and the ability to hold the auction site liable with no immunity were identified as major items within recourse. recourse can be considered as a control procedure and a service within the grounded theoretical mapping approach. the need for clearer and better understood control procedures is supported by tan & thoen (2002). the grounded theoretical approach highlighted the seller, power and assurances as items to increase online trust. the normative model of trust highlighted other procedures, categorisation, recourse and proper order as items to increase online trust. the normative approach tended to hide the importance of sellers across the normative themes, appeared less intuitive and appeared more limited in explaining more anonymous consumer-toconsumer online auction transactions. this research did manipulate dispute resolution functionality and the manipulation may have drawn each participant’s attention to the potential for a dispute and the potential need to resolve a dispute; however, participants assigned to higher levels of dispute resolution functionality identified just as many recourse items as did other participants. the results did not support that the manipulation resulted in participants overestimating recourse as an item to increase online auction trust. this research supported that the potential for a dispute was salient prior to an online auction transaction. demographic differences found were gender and transaction history. whereas, males appeared to rely on their own assessments of transaction specifics, females tended to look to opinions of others. females relied on references from acquaintances, relatives, friends, and users in their decision to participate in online auctions. this gender difference may be characteristic of all motor vehicle sales rather than for online auction transactions. bell (1967) suggests that one answer may lie with “variations in exposure to information about automobiles” (p.48). auction categories may have a greater appeal to particular genders, groups of users may be more knowledgeable about items within australasian journal of information systems volume 15 number 2, 2009 164 certain auction categories and designs that catered for their target audience would be expected to perform better than designs that did not. online auction sites that were flexible and able to cater for these user differences were expected to provide higher levels of trust and do better than sites that do not. user group differences were supported as an important consideration in the usability and design of online auctions. the findings of this research are important because they represent both non-users and users of online auctions. one explanation for the research findings based on shankar et al., (2002) was that potential and existing users of online auctions would like proper systems in place that extended beyond the forming of transactions to the setting, monitoring and maintenance of standards of practice within the online auction community. this research supported that a better model of trust is required to better understand online auction transactions. a model of online auction trust grounded within tpb may be better than existing models of online auction trust adapted from other domains. limitations and future research the data collected was qualitative information received in conjunction with an experimental research design. all care was taken to make the research design as real as possible. the site and information presented was based on actual online auction sites. only 5.4% of the participants identified that improving site presentation and layout would lead to an increase in online trust. no participants that had previously used an online auction site identified site presentation or layout as a possible factor to increase online trust. this research was exploratory, reduced data into themes and was intended to provide new insights into online auction trust. this research does not represent the population of potential and existing users of online auctions. future research is required: to include a wider population of online auction users and to include data that represents all stages of online auction transactions; to confirm and further explore usability requirements at online auctions; and to develop a better and more specific model of online auction trust grounded within tpb. conclusions this qualitative research identified mechanisms that people perceived as important to increase online auction trust prior to a transaction using normative and grounded theoretical approaches to trust. key trust mechanisms identified were assurances/guarantees, sellers and processes. these mechanisms supported that trust in the overall transaction was required rather than trust in any one person or thing. user differences were identified as an important consideration for the usability and design of online auction sites. a disconnection existed between items currently provided at online auctions and items that users and potential users and males and females would like provided at online auctions. whereas existing users were focused on the transactional level characteristics, potential users were concerned with both the auction site and transactional level characteristics; and whereas males placed a high focus on seller themes and assurances, the most frequently suggested theme by females was references and testimonials. this research compared and contrasted normative and grounded theoretical approaches to online auction trust. at an online auction, the normative approach appeared less intuitive, tended to hide the importance of sellers and have limitations in helping to understand trust. this research supported australasian journal of information systems volume 15 number 2, 2009 165 that the grounded theoretical approach to trust can help theorists and practitioners to better understand online auction trust. acknowledgments the author gratefully acknowledges the helpful comments of dr. deborah bunker and associate professor rodger jamieson on this article. references ajzen, i. (2005) attitudes, personality and behavior (2nd ed.), open university press, maidenhead. akerlof, g. a. (1970) “the market for ‘lemons’: quality uncertainty and the market mechanism”, the quarterly journal of economics, vol 84 no 3, pp 488-500. ba, s., and pavlou, p.a. (2002) “evidence of the effect of trust building technology in electronic markets: price premiums and buyer behavior”, mis quarterly, vol 26 no 3, pp 243-268. bell, g.d. (1967) “self-confidence and persuasion in car buying”, journal of marketing research, vol 4 no 1, pp 46-52. bewsell, g.r. (2008) “an investigation of mechanisms that impact trust: a domain study of online auctions”, unpublished doctoral dissertation, the university of new south wales. bhattacherjee, a. (2002) “individual trust in online firms: scale development and initial test”, journal of management information systems, vol 19 no 1, pp 211-241. duarte, m., and davies, g. (2000) “trust as a mediating variable in a complex model of channel member behaviour”, imp group, url http://www.bath.ac.uk/imp/pdf/139_duartedavies.pdf, (accessed date 23 june 2003). ebay (2006) ebay, retrieved 18 april 2006, 2006, from the world wide web: http://www.ebay.com/ fishbein, m., and ajzen. i. (1975) belief, attitude, intention, and behavior: an introduction to theory and research, addison-wesley, massachusetts. french, j. r. p., jr., and raven, b.h. (1959) the bases of social power, in d cartwright (eds) studies in social power, university of michigan press, michigan. ftc (2006) top 10 consumer fraud complaint categories, federal trade commission release, url http://www.ftc.gov/opa/2006/01/topten.htm, (dated 25 january, 2006, accessed date 20 february 2007). geyskens, i., steenkamp, j-b.e.m., and kumar, n. (1998) “generalizations about trust in marketing channel relationships using meta-analysis”, international journal of research in marketing, vol 15 no 3, pp 223-248. ifcc (2001) internet auction fraud, fbi & national white collar crime centre, auction fraud report, url http://www1.ifccfbi.gov/strategy/auctionfraudreport.pdf, (accessed date 14 june 2003). ihmc (2006) cmaptools, institute for human and machine cognition (ihmc), url http://cmap.ihmc.us, (accessed date 22 february 2007). australasian journal of information systems volume 15 number 2, 2009 166 jarvenpaa, s. l., tractinsky, n., and vitale, m. (1999) “consumer trust in an internet store: a cross-cultural validation”, journal of computer mediated communication, december, vol 5 no 2. katsh, e., rifkin, j., and gaitenby, a. (2000) “e-commerce, e-disputes, and e-dispute resolution: in the shadow of ebay law”, ohio state journal on dispute resolution, vol 15 no 3, pp 705-734. koufaris, m., and hampton-sosa, w. (2002) initial perceptions of company trustworthiness online: a comprehensive model and empirical test, cis working paper series zicklin school of business, url http://cisnet.baruch.cuny.edu/papers/cis200211.pdf, (accessed date 30 april 2003). kramer, r. m. (1999). trust and distrust in organizations: emerging perspectives, enduring questions. annual review of psychology, vol 50, pp 569-598. lind, e. a. (1995) social conflict and social justice: some lessons from the social psychology of justice. leiden university press, leiden. luhmann, n. (1979) trust and power, john wiley and sons, new york. mcknight, h., cummings, l.l. and chervany, n.l. (1998) “initial trust formation in new organizational relationships”, the academy of management review, vol 23 no 3, pp 473490. mcknight, d. h., and choudhury, v. and kacmar, c. (2002) “developing and validating trust measures for e-commerce: an integrative typology”, information systems research, vol 13 no 3, pp 334-361. mishra, d. p., heide, j.b., and cort, s.g. (1998) “information asymmetry and levels of agency relationships”, journal of marketing research, vol 35 no 3, pp 277-295. nooteboom, b. (2002) trust: forms, foundations, functions, failures and figures. edward elgar, cheltenham. pavlou, p. a., and gefen, d. (2004) “building effective online marketplaces with institution-based trust”, information systems research, march, vol 15 no 1, pp 37-59. pavlou, p. a., tan, y-h., and gefen, d. (2003) “the transitional role of institutional trust in online interorganizational relationships”, proceedings of the 36th hawaii international conference on system sciences. ridings, c. m., gefen, d., and arinze, b. (2002). some antecedents and effects of trust in virtual communities. journal of strategic information systems, vol 11 no 3-4, pp 271-295. shankar, v., urban, g.l., and sultan, f. (2002) “online trust: a stakeholder perspective, concepts, implications, and future directions”, journal of strategic information systems, december, vol 11 no 3-4, pp 325-344. shapiro, s. p. (1987) “the social control of impersonal trust”, american journal of sociology, november, vol 93 no 3, pp 623-658. tan, y.-h., and thoen, w. (2002) “formal aspects of a generic model of trust for electronic commerce”, decision support systems, vol 33 no 3, pp 233-246. thorelli, h. b. (1986) “networks: between markets and hierarchies”, strategic management journal, vol 7 no 1, pp 37-51. australasian journal of information systems volume 15 number 2, 2009 167 today, c. (2003) “internet fraud on rise, ftc says”, communications today, january 24, vol 9 no 15, pp 0-1. van den bos, k., van schie, e.c.m., and colenberg, s.e. (2002) “parents' reactions to child day care organizations: the influence of perceptions of procedures and the role of organizations' trustworthiness”, social justice research, vol 15 no 1, pp 53-62. zhang, x., and zhang, q. (2005) “online trust forming mechanism: approaches and an integrated model”, 7th international conference on electronic commerce (icec'05), 15-17 august 2005, xi’an, china. zhao, x., fang, f., and whinston, a.b. (2006) “designing on-line mediation services for c2c markets”, international journal of electronic commerce, vol 10 no 3, pp 71-93. zucker, l. g. (1986) “production of trust: institutional sources of economic structure”, research in organisational behaviour, vol 8, pp 53-111. ajis vol. 8 no. 1 september 2000 transitioning existing content: inferring organization-specific documents arijit sengupla', sandecp purao1'2 1: department of cis, robinson college of business, georgia state university, atlanta, ga 30302. tel: +1 (404)651-3880 fax: +1 (404)651-3842 e-mail: {asengupt,spurao}@gsu.edu, url: http://www.cis.gsu.edu 2: institutt for informasjonvitenskap, agder university college, tordenskj 65, kristiansand, norway abstract a definition for a document type within an organization represents an organizational norm about the way the organizational actors represent products and supporting evidence of organizational processes. generating a good organization-specific document structure is, therefore, important since it can capture a shared understanding among the organizational actors about how certain business processes should be performed. current tools that generate document type definitions focus on the underlying technology, emphasizing tags created in a single instance document. the tools, thus, fall short of capturing the shared understanding between organizational actors about how a given document type should be represented. we propose a method for inferring organization-specific document structures using multiple instance documents as inputs. the method consists of heuristics that combine individual document definitions, which may have been compiled using standard algorithms. we propose a number of heuristics utilizing artificial intelligence and natural language processing techniques. as the research progresses, the heuristics will be tested on a suite of test cases representing multiple instance documents for different document types. the complete methodology will be implemented as a research prototype keywords: document management, document models, reverse engineering, heuristics, document type definition introduction documents are a central part of every organizational process. they are used to capture information for archival, publication or as intermediate records of organizational processes. with the advent of the world-wide-web, a distinct shift is being felt toward the use of documents as the source of corporate data. instead of proprietary formats, more and more businesses are exploiting the web for their data creation, storage and distribution requirements. the introduction of xml and with it, higher control over the document structures and meta-data has clearly increased the potential of document data management on the web (goldfarb and prescod 2000). along with the power of representation, however, xml has introduced newer problems of structural inconsistencies. xml was designed for evolutionary data management, where authors did not need to conform to a fixed structure to create valid documents. in xml, a document can be easily parsed so long as it is well-formed, a constraint that allows parsers to build the document hierarchy without requiring a pre-defined document type. the property of well-formedness ensures that an electronic representation of the hierarchical structure of the document can be constructed from source documents without ambiguity. however, in many cases, it is necessary to capture and store the metadata as a document type definition (dtd) or a schema. a dtd can be constructed, using standard algorithms, from the structure evident in a marked-up instance document (shafer, 1995; garofalakis, 2000; herman and diaz, 1999; kay, 2000). such reverseengineered dtds, however, simply end up capturing decisions about markup by the author of the xml instance document. the markups may not be sufficiently generic, may enforce incorrect cardinality, may even be too generic, or may use inappropriate nesting. all these can significantly reduce the usefulness of the generated dtd, rendering them less usable. if xml is to succeed, we must bring into fold existing documents and reconcile, where necessary, implicit assumptions underlying their content and structure. the well-documented benefits of xml (goldfarb and prescod 2000) cannot be realized unless the large base of documents that do not yet conform to the xml infrastructure are provided the required infrastructure. ensuring that a class of documents conforms to a common, shared intent and structure, therefore, will require a reconciliation of the various, probably different, declarations of tags, metadata and structure that different organizational actors may assign to comparable parts of different instance documents. for example, consider a memo that contains information about a customer complaint and its resolution. the parts of such a memo, disregarding the headers, may be variously declared as <text (complaint, resolution)> or <body (complaint, complainttype, resolved, comments)> or <response (complaint, action)> or something else. inferring organization-wide document structures that may be shared by these actors should, then, improve and streamline the related processes. further, the benefits of aggregating the information 91 ajis vol. 8 no. 1 september 2000 available in these instance documents should be clear. resolution of different structures and tags that different actors may impose on individual instance documents is, therefore, a prerequisite for this reconciliation. our goals in this research are to create techniques and tools that can facilitate this process of reconciliation. the objective of this research, therefore, is to: develop a methodology to semiautomatically generate high-quality, organization-specific document type definitions from the instancespecific dtds compiled using standard algorithms. a dtd reconstruction tool such as ddbe will generate a dtd that is guaranteed to parse the source document(s). our intent, instead, is to capture the core document model, and generate a dtd that will parse a large proportion of the source documents, and could be used for conversion, exchange, and creation of standardized and organization-specific documents. related research document management documents can capture meanings and purposes utilized and produced by organizational actors, by exploiting technological features necessary for the storage and processing of that information (tyrvainene and paivarinta, 1999). electronic document management is the capture, management and manipulation of documents in an organized manner. a rich stream of research exists on document management (sprague, 1995). a document, among other things, can be considered as recorder of contents, evidence of action or proof of authenticity. (tyrvainene and paivarinta, 1999), in fact, describe as many as eleven different facets of documents, which they identify based on an analysis of writings in is research. markup languages such as sgml and xml are technologies, which, along with others, can make electronic document management easier and more effective. standardization of documents, whether using these technologies or otherwise, is an important aspect of document management. several ongoing research efforts (see, e.g. jyvaskyla, 2000) emphasize the importance of document standardization and report on the high costs in terms of time, money and effort that organizations incur when they embark on document standardization projects. standardization of documents may involve the standardization of usage or content of documents. the first requires an understanding of and conformance to organizational processes as well as buy-in from organizational actors. in contrast, standardization of the second involves analysis and reconciliation of varying perspectives that are made explicit by the organizational actors. our focus in this research is on the latter. current research suggests that when performed manually, this kind of standardization can be a long and arduous process, taking much effort and involving much expense. in the context of xml technology, this translates to standardization of the document type definition (dtd) or schema. since the schema standard (xschema, 2000) is still fairly new, our focus in this paper is on the standardization of organizational dtds. automatic dtd generation a number of approaches, research prototypes and state-or-the-art commercial tools can extract (reverseengineer) the document type definition from xml instance documents. ddbe (data descriptors by example) (berman/diaz, 1999) is a java library from ibm alphaworks that can generate a dtd from an xml document. a recently developed system, xtract (garofalakis et al, 2000) is under patenting process from bell labs which also has similar functionality, but can generate more optimized dtds for single productions. a somewhat more dated, but quite well known tool, fred (shafer, 1995) is also capable of generating dtds from arbitrary sgml documents (and hence can also work with most xml documents). a web-based tool called dtd generator for this purpose using the saxon xml parser is also available (kay, 2000). all of these tools aim at generating dtds for the purpose of validating. the techniques they use include different ways of 'guessing' the intended structure from different combinations of the same set of tags (berman/diaz, 1999; garofalakis et al, 2000). some of these tools also try to distinguish the special attributes such as links, ids, idrefs, and so on, using heuristics based on the number of times an attribute value is used (kay, 2000). the tools described above excel at constructing a dtd for a given instance document. they can infer cardinalities, nesting of elements, default values of attributes, as well as references (links) between elements. the tools, however, clearly fall short of generating a standardized dtd that may be appropriate for that class of documents. their focus, instead, is on the compilation of a dtd that can make explicit the content and structure of the xml instance documents. our research, then, involves 92 aj1s vol. 8 no. 1 september 2000 use of these dtds as inputs to infer a standardized dtd for the organization. the existence of multiple instance documents sets apart our document structure reverse-engineering effort as opposed to those proposed by others (fred, xtract, ddbe, dtd generator) which have focused on a single xml instance document. in one sense, our approach mirrors that taken for database reverse engineering, where a number of instances are considered to identify entities and attributes of interest in the relational or conceptual schema. we can, therefore, use and extend lessons from database reverse engineering for our problem but should be careful to account for any differences. database reverse engineering a rich stream of research on database reverse engineering has focused on creating logical or conceptual data models from database instances (wcre 1999). various problems have been addressed and solved to varying degrees such as reverse engineering of entities, attributes, binary relationships and ternary relationships (chiang et al., 1994, permerlani and blaha 1994). the core technological model for these efforts has been the relational data model. the research has resulted in a number of practical and innovative results (fahrner and vossen, 1995, chiang 1995). though it is possible to extend and augment these approaches for our context -organizational document documents there are sufficient differences between 'database reverse engineering' and 'organizational document standards inference' to warrant a different approach. first, documents can contain highly unstructured data unlike that seen in most relational databases, making inferences based on data instances difficult. second, unlike relational database schemas, which have a well-established foundation, the xml schema standard (xschema 2000) is still fairly new (may 2000), and the functionality of extended links among documents is also poorly implemented. the focus of our efforts, therefore, is on individual document types without the benefit of exploiting information that may be available in links across documents. third, perhaps most important, is the nature of data instances used as inputs. for reverse engineering of relational database schema, the inputs contain multiple database designs, which are generated from the data by one source, say, a program written with a second generation language. they share a common layout but do not have any metadata associated with them. the reverse-engineering process is, then, expected to discover this metadata and its structure. on the other hand, the document instances are marked up by different sources (e.g., different authors) and can contain different tags indicating the same or different metadata, arranged in different layouts. the problem of inferring a standardized document type, thus, involves more than a mere extension of database reverse engineering approaches. our research question and focus inferring organizational documents is, therefore, shaped by three research streams: document management, automatic document type generation and database reverse engineering. we motivate it from a document management perspective. the automatic dtd generation provides appropriate inputs to our process. finally, the database reverse engineering research suggests approaches that we adapt and extend for document type generation. inferring organizational document structures a simple dtd can be directly compiled from an xml document using a tool such as the ones described above (e.g. ddbe). for xml (as opposed to sgml), the task of generating a dtd from a document is not very hard, since well-formedness guarantees that the document structure (the tag hierarchy) can always be inferred from the documents. our problem is, therefore, defined as: given a set of documents, possibly clustered by different authors or sites, but all having the same intent, and their associated dtds, compiled with standard algorithms, to develop a strategy to generate a superdtd that captures the core model covering the document set. one fairly simple approach to this would create a simple union of all the dtds. however, a dtd representing such a union will be mostly redundant, and even a union would require choices to reconcile mismatches in sequencing and nesting. our problem, thus, is to find a suitable dtd and its variants given a set of xml instance documents that share the same intent but have different tagsets (due to different authors). the generated dtd should capture the core structure of all the documents and its variants. figure 1 below shows the focus of our research. 93 ajis vol. 8 no. 1 september 2000 marked-up instance document: xml marked-up instance document: xml marked-up instance document: xml figure 1: research focus we use as inputs the dtds compiled from instance documents by existing tools. these dtds contain elements, attributes, their sequences, cardinalities and structures. our approach, then, is focused on creating a core dtd plus its variants, which, taken together will cover a substantial proportion of document instances of this document type. the core dtd and the variants may be combined with use of entity's, that is, with the help of mechanisms resembling aggregations or inheritance, ensuring coverage of a very large proportion of source instance documents. document structure generation heuristics a number of issues need to be considered for the generation of a normative document type definition from a set of possibly inconsistent documents with the same general structure. although in some cases a definitive solution can be reached, most cases require the use of some intelligence and experience to decide on a resulting structure. in this section, we first describe the issues in generating a single document type from possibly inconsistent document structures, propose heuristics that will result in a potential solution, and show an application. elements versus attributes: elements contain and envelop parts of a document. on the other hand, attributes provide slots for further information about these parts. in a choice between elements and attributes, therefore, it may be argued that the former should have primacy. there may be exceptions to this rule, for example, if rendering of values depends upon whether a certain piece of information is an element or an attribute. order of elements: unlike a relation in a relational database, ordering between elements is important in a document. when document instances contain different sequences, but similar sets of elements, heuristics may generate a compromise, but not a perfect solution. on the other hand, ordering between attributes is unimportant (w3c 1998). cardinality of elements: two issues are important here. the first concerns whether the cardinality of an element is optional or mandatory. if an element is marked as mandatory in one instance document and optional in another, there are clearly two approaches to resolve this mismatch. one is to enforce the mandatory cardinality, forcing instance documents conform to this requirement. this can mean a strict interpretation, giving primacy to a more rigid set of definitions. the other is to allow an optional cardinality, covering a larger proportion of instance documents with the definition. additional information may be necessary to devise a heuristic that resolves this anomaly. another aspect of cardinality is the choice between one or multiple occurrences if a mismatch is found between instance documents. this can represent a choice similar to optional versus mandatory, though single occurrence may be considered a special case of multiple elements, giving multiple occurrences higher priority during reconciliation. required vs. optional attributes: a parallel to the choices above can also be seen in the choice for required or optional attributes. this decision too, may be dictated by concerns similar to the ones 94 ajis vol. 8 no. 1 september 2000 described above. in addition, manipulations and renderings applied to the attributes may be examined to decide the appropriate reconciliation. nesting of elements: this is perhaps the most problematic issue in intelligent dtd creation. nesting between elements indicates the presence of some form of relationship (usually "contains") between the elements. however, different document instances may nest the same pair of elements differently. it is conceivable that the nesting may be reversed or may contain different levels. a set of heuristics that combines parsing as well as tree traversal may be needed for meaningful reconciliation of any mismatched nesting structures between document instances. links: given the focus of the research a document type, instead of schemas containing different document types we treat links as standard attributes. since the linking standard is still evolving (xlink 2000), this aspect of reconciliation can be retained for the next phase of our research project. to address the above issues, we now define a set of heuristics that can produce a resulting document structure from a set of dtds. intelligent heuristics the heuristics are grouped in three categories: element reconciliation, attribute reconciliation and structure reconciliation. we describe these below with the help of simple examples. element reconciliation heuristics 1. partial or complete overlap between element names: an element name in one dtd may be fully contained in an element name from another. this suggests that, in the original designs of the document structures, the same concept was being modeled. therefore, the element names are parsed to find the stems and from this suggestions made to the designer about the possibility of additional elements(s). e.g., an application may have a "heading" element under a chapter and another may explicitly call it "chapterheading". in such cases, designer is suggested to choose between the two or retain both. 2. optional versus required elements: if it is indicated as required in at least one of the instance documents, include it as required, with a default value. e.g., a document instance may have a required section_no element in a section, where in another instance it may be optional. result: make the section_no required, with a default value. 3. single versus multiple occurrences of an element: if an element appears with more than one cardinality in at least one document instance, include it with higher cardinality in the dtd. e.g., a particular book structure may have "editor" as a multivalued element, where another structure may only allow one editor. result: make editor a multivalued element. 4. elements that appear more than once: if an element appears in more than one input dtd, then it is added to the output dtd. it is assumed that an element that appears in more than one dtd, it is not dependent upon naming idiosyncrasies and, thus represents an important domain concept. e.g., if the book element appears in at least two documents, include it as an element. attribute reconciliation heuristics 5. elevating attributes to elements: an attribute is promoted to an element when it appears as an element in one or more input dtds and as an attribute in at least one other. since we are interested in creating a normative document model, if at least one document includes this as an element, we reason that it was judged important enough to be an element and include it. a variation of this heuristic could involve searching for attribute names that have common stems to element names, and promoting these attributes to elements. designer interaction may be necessary here because of a higher possibility of error. e.g., a document may have publication year as an element and another as an attribute. result: make publication year an element and retain the structure of the former document. 6. derived attributes: remove derived, that is, computed attributes from the dtd. this functionality can belong in transformations, that is, xsl. e.g., if an element 'weight' is specified in grams (an attribute), and another element 'calories-pergram' specifies the calories, the computed element, 'totalcalories' need not be included in the dtd. 95 ajis vol. 8 no. 1 september 2000 <!element book (title, author+, editor, publisher, price, body)> <!attlist book pubyr cdata #implied> <!element title (#pcdata | subtitle)*> <!element body (chapter+, appendix*, references)> <!element chapter (chno, heading, chbody)> <!element chbody (#pcdata | section)*> <!element section (secno, heading, secbody)> dtd 1. a partial book dtd instance <!element book (title, author+, editor+, publisher, pubyr, price, priceindollars, body)> <!element body (chapter+, appendix+, references)> <!element chapter (chno?, chheading, chbody)> <!element chbody (#pcdata | section)*> <!element section (secno?, secheading, secbody)> dtd 2. another partial book dtd instance figure 2 input dtds structure reconciliation heuristics 7. restructuring elements: ambiguous structuring of elements may cause additional problems in producing a satisfactory result. it may be possible that an element is the sub-element of one element in one structure, and of a different element in another. for example, a book may have a title, and a chapter may also have a tide. a reasonable solution would be to include both the relationships. in contrast, if a structure includes subtitles in titles, and another includes only character data, the result may be an optional subtitle in title. the heuristics we have outlined above represent a start towards an approach for reconciling the input dtds to create a super-dtd. additional heuristics that are in the process of development include the identification of core dtd, and the use of entities to combine modules to create variants. as the xschema standard evolves (xschema 2000), we expect that the parsing techniques in our heuristics will also exploit information about data types in the source documents, generating a schema from a group of source dtds. the above set of heuristics, however, can be used to generate a plausible superdtd. we demonstrate below how these heuristics may be applied with an example. <!element book (title, author+, editor+, publisher, pubyr, price, body)> <!element title (#pcdata | subtitle)*> <!element body (chapter+, appendix+, references)> <!element chapter (chno, chheading, chbody)> <!element chbody (#pcdata | section)*> <!element section (secno, secheading, secbody)> dtd 3. dtd generated from application of heuristics. figure 3: reconciled super dtd application a simple instance of the above heuristics is shown in figures 2 and 3. figure 2 shows the input dtds, compiled from source documents, marked up by different authors, in our case, different publishers. these dtds may have been compiled using any of the tools mentioned earlier. they contain specification of elements, attributes, cardinalities, sequencing and nesting. the dtds shown are partial, that is, they show only one of the branches of the tree fully. we describe below how the heuristics may be applied to reconcile these source dtds. in this example, the attribute 'pubyear' appearing in dtd 1 can be promoted to an element using heuristic 5. the element 'heading' under 'section', is found to be similar to the 'secheading' element, and the two have been combined using heuristic 1. using the same heuristic, 'chheading' has been retained in favor of 'heading' under 'chapter.' using heuristic 2, 'chapno' and 'secno' have been made 96 ajis vol. 8 no. 1 september 2000 required since one of the dtds required them. the 'editor' element has been elevated to a multipleoccurrence element using heuristic 3. elements that have been included in the output dtd are elements that were included in both the dtds (heuristic 4). heuristics 6 and 7 were not applied in this example. notice that since these dtds were generated by reverse-engineering sample xml documents from one organization, most element names were same or similar. if the element names are different but with the same meaning, various linguistic analyses need to be performed in order to determine their similarity. dtd graphs (from input) secondary index external thesaurus/ dictionary dtd graph (output) figure 4. architecture of dtd extractor implementation proposed implementation and testing we are developing a proof-of-concept prototype for the approach. the implementation is primarily in java with the xp xml parser library from james clark. xp allows parsing of xml 1.0 compliant documents including xml dtds, and includes a class library that can be instantiated to process different parts of the dtd. the input to this program consists of two or more dtds (possibly with sample xml content, although the xml content is currently ignored). the parsed dtds are used to create memory-based graph structures representing the dtds. the heuristics in this paper are then applied to the memory structures. the approach is similar in spirit to other ongoing research on relational databases (purao et al, 2000), which proposes to develop domain models from schemas reverse-engineered from multiple relational database models. like entities and attributes in relational model, the elements and attributes in xml can be identified in a generic manner. the proposed architecture of the structures and processing is shown in figure 4 below. figure 4 shows the basic architecture of the system under implementation. initially, ddbe is used to generate dtds from input documents (although xtract is the most optimized dtd generator known, a public implementation is not yet available). these dtds are read in memory-based graph-like structures, and secondary indices (currently a variant of b-tree) are created on top of the graph elements. these structures are then processed using the proposed heuristics to generate another graph structure for the output dtd, which is then used to generate the final dtd for output. planned extensions to the architecture include hooks for complex processing such as linguistic analysis and the use of external dictionaries. we will test the tool to reconcile content management dtds created for managing lecture and course management content. the reconciliation will be done manually as well as with our tool in order to understand the behavior of the heuristics and how they can be improved. 97 ajis vol. 8 no. 1 september 2000 conclusion good document type definitions are invaluable for maintaining consistency between organizational documents. documents created in xml may not have a dtd, in which case one can be generated from document instances using dtd generation tools. however, these tools fall short in capturing the organizational document design, but simply capture the idiosyncratic naming decisions of individual authors. this paper describes a method for generating organizational document type definitions from multiple xml instances, possibly created from different sources. our contribution is in using the current methods to assist in the generation of organization-specific document models. further research, including development of a research prototype as well as experimentation is under way to assess the viability of the approach, its utility, and future applications.17 references berman, l., a. diaz (1999) data descriptors by example (ddbe). ibm alphaworks research project documentation (at http://www.alphaworks.ibm.com) june, 1999. chiang, r., (1995) a knowledge-based system for performing reverse engineering of relational databases, decision support systems, 13, pp. 295-312. chiang, r., t. m. barren, v. c. storey. (1993) reverse engineering of relational databases: extraction of an eer model from a relational database, data & knowledge engineering 12, pp. 107-142. fahrner, c., g. vossen (1995) a survey of database design transformations based on the entityrelationship model, data & knowledge engineering. 15, pp. 213-250. garofalakis, m., a. gionis, r. rastogi, s. seshadri, k. shim, (2000) xtract: a system for extracting document type descriptors from xml documents. proceedings of acm sigmod'2000, dallas, june 2000. goldfarb, c., p. prescod (2000) the xml handbook. upper saddle river, nj: prentice hall ptr, 2000. jyvaskyla (2000) document management research program at university of jyvaskyla. at http://www.cs.jyu.fi/~airi/docman.html kay, m. (2000) saxon dtdgenerator a tool to generate xml dtds. technical software documentation, at http://users.iclway.co.uk/mhkay/saxon/dtdgen.html. february 2000. premerlani, w. j., m. r. blaha. (1994) an approach for reverse engineering of relational databases, communications of the acm. 37, 5, pp. 42-49. purao, s., v. c. storey, m. moore, a. sengupta (2000) reconciling and cleansing an approach to domain models. working paper, georgia state university. february 2000. shaffer, k. (1995) creating dtds via the gb-engine and fred. proceedings: sgml'95 conference. boston, ma, december 1995. sprague, r. h. (1995) electronic document management: challenges and opportunities for information systems managers. mis quarterly. 19, 1. pp. 29-50. tyrvainene, p., and t. paivarinta (1999) on rethinking organizational document genres for electronic document management. proceedings: hicss-32, 1999. hawaii. w3c (1998) xml specification 1.0. february 1998.http://www.w3.org/tr/1998/rec-xrnl19980210. wcre (1999) 6th working conference on reverse engineering. 6 8 october 1999 at atlanta, ga. xlink (2000) the linking specification for xml. http://www.w3.org/xmi7linking.html xschema (2000) xml schema specification. at http://www.w3.org/tr/xmlschema-0/ draft 7 april 2000. acknowledgements we wish to acknowledge comments from anonymous reviewers, which have helped to considerably improve the content and the presentation. the paper has also benefited from discussions with airi salminen and pasi tyrvainene. 17 a preliminary version of this paper, under a different title, was presented at the conference xml meets business in heidelberg, germany in may 2000. 98 microsoft word ajis vol 15-1 m.doc australasian journal of information systems volume 15 number 1 december 2007 153 project management effectiveness: the choice formal or informal controls andy susilo university of queensland, qld, australia jon heales university of queensland, qld, australia email: j.heales@business.uq.edu.au fiona rohde university of queensland, qld, australia email: f.rohde@business.uq.edu.au abstract development failures and runaway projects in the information systems (is) area can result in substantial losses to organizations, either financially or to a company’s reputation. one important strategy in mitigating risk is the use of effective controls over is projects. this research investigates the effectiveness of control mechanisms in is projects, i.e., how they have been established, applied, and how they have evolved throughout the project. we conducted an in-depth study of six information systems projects with six different project managers. we found that formal controls were adopted on project elements with clearly defined project outcomes and informal controls were adopted mainly on project elements that were unclear and often related to people. furthermore, formal controls were dominant at project initiation and in most cases these controls became less dominant over the project duration. these formal controls were more rigorously applied in external is projects while informal controls increased in importance throughout the project duration in internally developed projects. we found the existence of informal controls to be very important in helping ensure successful project outcomes. australasian journal of information systems volume 15 number 1 december 2007 154 introduction failing to successfully deliver an is project can result in substantial losses to organizations, either financially or to a company’s reputation (montealegre & keil 2000, p.5; wallace & keil 2004). moreover, the causes and outcomes of many runaway it projects are covered up or ignored to save embarrassment, but unfortunately, the same mistakes occur repeatedly (hartman & ashrafi 2002; johnston 2003). research findings to date show that the average percentage rate of successful is projects over a number of years is still considered to be low (figure 1). figure 1 shows the failed project rate decreased substantially between 1996 2004, however, the successful project rate has only marginally increased during the same period the standish group (2004) identified that the increasing rate of successful projects were occurring in small projects. these projects were less complex, had fewer potential problems and management had better control over the budget. in their latest report, the standish group concluded that smaller teams and projects with a reduced scope of work delivered better is project outcomes. interestingly, the challenged project rate rises quite dramatically for the same period. thus while it appears the project failure rate has reduced, the failures have not been turned into project successes, merely challenged projects. these examples show that is projects continue to be risky and warrant serious consideration before commencement. their large budgets and low rates of success mean organizations invoke risk and, to some extent, risk cannot be avoided where project functionality is imperative (see broadbent 2004; turban, mclean & wetherbe 2004; weber 1999). 0% 10% 20% 30% 40% 50% 60% 1994 1996 1998 2000 2004 year % successful project rate challenged project rate failed project rate figure 1: trend of is project history (adapted from chaos report -standish group 1999; standish group 2001, 2004) one important strategy in mitigating risk is the establishment of effective controls over is projects. several theories have been adapted from general project management theory and applied to the establishment of control over is projects. the project management institute (pmi) (2004) details nine elements of control known as the project management body of knowledge (pmbok) (pmbok 2004; schwalbe 2004). using the pmbok as a framework, we investigate the establishment, application, and evolution of control mechanisms in is projects and how they deliver better project outcomes. australasian journal of information systems volume 15 number 1 december 2007 155 we distinguish two different types of is projects, internal is projects and external is projects. turban et al. (2004, p613) define external is projects or outsourcing is projects as ‘the strategy to obtain it services from outside vendors rather than from internal is units within the organization’. therefore, internal is projects can be defined as obtaining it services by using an organization’s internal resources. they noted an increasing trend towards outsourcing is projects. the border between internal and external is projects, however, has become blurred due to increasingly complex business processes and external parties taking on broader roles, e.g., an active role in decision making processes within the organization (cio 2002). to capture best practice in is project management, sauer et al.,(2001) compared is project management with construction project management. they found that construction projects had a better chance of success due to key factors inherent in the project manager such as experience, knowledge, and leadership. by having a good project manager, the project would have a better chance of implementing effective control mechanisms over the project, i.e., the better the set up of the control framework, the better the chance of a more successful project. therefore effective controls should be set up during project initiation and exercised during project development. we aim to investigate the effectiveness of control adoption in various types of is project types, from purely internal to purely external is projects, from the initiation stage to the project implementation stage. we assume the existence of control elements is important to ensure the quality of the project outcome and that their existence and adoption is vital to keep the project on track. we use control theory that has been refined over many years in combination with the pmbok to frame the adoption of the control modes in is projects for both external and internal is projects. the theory examines the control framework in both formal and informal modes (choudhury & sabherwal 2003; eisenhardt 1985; kirsch 1997; kirsch et al. 2002; ouchi 1979, 1980). we propose two research questions, (1) what controls are in place at the project initiation stage? and (2) how do the controls (formal and informal) evolve during an is project? the format of this paper is as follows. section 2 the conceptual framework underlying this research and develops initial propositions based on the research questions and theory development. section 3 describes the research method and data collection process. section 4 presents the research results. section 5 summarizes this research, lists the limitations, and outlines possible future research. theoretical framework and proposition development project management theory we have adopted the pmbok from the project management institute (pmi) (2004) as a framework for the data collection and analysis because it is derived from theory in the project management area. it is accepted as generic theory that can be used in many projects in many fields, not only in is (wideman 1995). the project knowledge management theory covers crucial issues in project management and divides those issues into nine consecutive elements. they are derived from best practice in project study cases and can be considered as the most important elements in project management (for example, see leach 2003; morris 2001; zobel & wearne 2000). although each element consists of a number of processes, some differences in the detail of the project knowledge elements across various project types can be found, e.g., success criteria, requirements management, australasian journal of information systems volume 15 number 1 december 2007 156 information management, performance measurement (morris 2001). see table 1 for the nine elements and their processes. element process within those elements 1. project integration management project plan development project plan execution and integrated change control 2. project scope management scope planning, scope definition, create work breakdown structure (wbs), scope verification and scope control 3. project time management activity definition, activity sequencing, activity resource estimating, activity duration estimating, schedule development, and schedule control 4. project cost management cost estimating, cost budgeting, and cost control. 5. project quality management quality planning, quality assurance, and quality control 6. project human resources management human resource plan, human resource acquisition, human resource development, and human resource management. 7. project communications management communications planning, information distribution, performance reporting, and stakeholder management 8. project risk management risk management planning, risk identification, qualitative risk analysis, quantitative risk analysis, risk response planning, and risk monitoring and control. 9. project procurement management plan purchases and acquisitions, plan contracting, request seller responses, select sellers, contract administration, and contract closure table 1: project management body of knowledge (adapted from pmbok 2004, p.11) australasian journal of information systems volume 15 number 1 december 2007 157 control theory formal controls, i.e., behaviour controls and outcome controls, are defined as tools to measure performance (see eisenhardt 1985; kirsch 1996). behavior controls articulate specific rules and procedures to be followed, while outcome controls articulate specific outcomes (eisenhardt 1985; kirsch 1996). a formal control would be a formal meeting with an agenda and formal minutes taken for distribution. informal controls, i.e., clan control and self control, rely more on ‘unwritten practice code’ such as social values, common beliefs, or traditions among people or individuals (eisenhardt 1985; ouchi 1980). informal controls might include informal meetings where there is no agenda or minutes taken. clan controls are controls that exists in a group or in individuals who share common beliefs or common goals (ouchi 1980). self controls are controls that are established by an individual to undertake a particular task by controlling their own actions (kirsch 1997). control determinants such as task programmability, information systems, and uncertainty factors, influence the control model. for example, behavior controls are specific rules and procedures and as such ideal in controlling routine and simple jobs. outcome controls could be used when the level of the risk became higher and where behaviors were difficult to measure (eisenhardt 1985). govindarajan and fisher (1990) incorporated two additional variables, “behavior observability construct” and “retained task programmability” into the original control model matrix of eisendhardt (1985) and ouchi (1979). table 2 shows the adjusted matrix. task programmability high low high behavior observability behavior outcome high outcome observability low behavior observability outcome outcome high behavior observability behavior clan low outcome observability low behavior observability self self table 2: control matrix related to behavior andcontrol (kirsch 1996) kirsch (kirsch 1996)found that the more the project manager understood the project content, the more the processes became clearer resulting in behavior control domination. outcome measurability affected both outcome control and self control. she pointed out that when the project outcomes became clearer and more visible, the objectives became easier to specify. as a result, the goals are easily established (outcome control) and this in turn motivates personnel to establish individual goals (self control). although a portfolio of control is established at project initation, the controls vary in response to problems encountered during the project. their data showed that formal controls dominated the model over time (choudury and sabherwal, (2003). when appropriate mechanisms exist in the project, project managers tended to utilize formal control. in situations when the appropriate mechanisms were unavailable, the project managers resorted to enforcing inappropriate informal controls or defining new mechanisms including new informal controls (kirsch 1997). kirsch (kirsch 2004) noted that informal controls function as complementary mechanisms to support formal controls. both control modes evolve as the project progresses and both work side by side to achieve better project outcomes. because of their increasing importance in the is portfolio kirsch et al. (2002) undertook further refinement of project controls in the external is project environment. kirsch et al. (2002) reviewed the application of the control mechanisms in outsourced is projects and found it was important to australasian journal of information systems volume 15 number 1 december 2007 158 have a client that understood the system development process so that the client could apply the appropriate control. a solid understanding could lead to behavior control whereas an incomplete understanding could result in clan control. one important observation revealed that the client’s satisfaction was improved by introducing the formal and informal controls to the client so that they could incorporate these controls in their own projects. research scope and propositions we contribute to existing theory by extending the investigation of formal and informal controls over both internal and an external is projects. this study compares and contrasts the similarities and differences between the two types of is projects, and uses project management knowledge from the pmbok as a framework to evaluate the application of controls in these projects. we distinguish between internal and external projects because external projects are often outsourced and performed by non-organizational employees. there is a perception that external projects are better controlled and more successful. the scope of this research is as follows. first, the control modes in this study are investigated from the client perspective as in the previous studies such as, kirsch (1997), kirsch (2002) and choudhury and sabherwal (2003). we build on the way the control modes are setup and applied throughout the project life cycle. second, due to the size and dimension of the data structure, and the timeframe of this study, the analysis and discussion are performed at the level of formal and informal control definitions. finally, we do not elaborate on the detailed processes within each element from the pmbok but focus on the adoption of formal and informal control elements at the nine project element layers and discuss the research results at each level. the first research question “what controls are in place at the project initiation stage?” aims to determine the existence of controls at project initiation. most elements in the pmbok recommend control methods. moreover, kirsch (1997) and choudhury and sabherwal (2003) also noted the dominance of formal controls at the beginning of the external is project. therefore: proposition #1: controls are in place for each pmbok element. each pmbok element listed in table 1 requires a series of processes to be undertaken. many of the processes will be required to produce specific outcomes that can result in a series of rules and procedures that are to be followed or result in specific outcomes. these types of situations lend themselves to the adoption of either behavior controls or outcome controls, i.e., formal controls. thus: proposition #2: formal controls are applied to pmbok elements that have clear requirements and can support the formal controls. while the preference is to establish formal control, there will be times when executing the processes within the pmbok deals with uncertainty, especially when the project is innovative. innovative projects are associated with increased risks (for example, see argument from ferguson, finn & hall 2005). uncertainty leads to a lack of clear requirement specifications and as such the establishment of clear rules or procedures to ensure the appropriate outcomes are difficult, i.e., formal controls are difficult to establish. when formal controls are difficult to establish there is the tendency to adopt informal controls as one mechanism to mitigate the risk. hence: proposition #3: in the absence of clear formal control requirements, informal controls will be adopted. this second question is “how do the controls (formal and informal) evolve during an is project?” external projects that had ‘tight’ formal controls at the project initiation stage usually ended with fewer problems. however, external projects were also subject to ‘unstructured tasks’ emerging as australasian journal of information systems volume 15 number 1 december 2007 159 the project progresses (choudhury & sabherwal 2003). at the project planning stage, the pmbok elements provide clear requirements for the identification of controls that should be implemented. as the project evolves there should be less and less uncertainty surrounding the project and thus its management. internal is projects staffed by internal resources incur stable (salaried) costs on an ongoing basis. these projects invite few non routine events and are therefore unlikely to attract attention. agreements are made internally in terms of cost, schedule and resources (but usually lack contractual agreement). because internal projects do not have to consider the formality of contractual obligations, project overruns are not subject to the same scrutiny. factors such as, time, budget, people and other factors could be discussed in a more ‘friendly’ environment that was less rigid, more negotiable, and allows for ex post changes. in contrast, external projects incur costs and are subject to schedules that have been contractually agreed. invoices are scrutinized for compliance with the contract, and any deviations such as schedule slippages or added functionality are highlighted and questioned. thus, external projects have greater visibility because of the formal processes associated with them. with the importance of maintaining control over external providers it is anticipated that as the level of uncertainty surrounding a project decreases any established informal controls may be replaced by more formal mechanisms. thus as the project evolves formal controls are still the more extensive control mechanisms in place. therefore: proposition #4: informal controls evolve to be more extensive than formal controls during an internal is project’s lifespan. proposition #5: external is projects evolve more extensive formal controls than internal is projects during a project’s lifespan. research method six projects were chosen from three organizations, a portfolio management company (four projects), and insurance company (1 project) and a large municipality (1 project). the projects were chosen by the senior management of each the organizations, and all were large scale multi-person projects. project managers across six different is projects were subjected to indepth interviews to determine how controls are adopted in is projects. to enrich the interview data, some of the interview sessions also involved a project team member, such as a business or system analyst. the initial interview questions were framed around the pmbok components, and the control framework examined to determine the existence of controls for each component of the pmbok. the questions were expanded according to the interviewee’s answer. the aims of the interview were to explore the interviewee’s experience and frame the experience in terms of the research questions and propositions. the interview results were recorded and scripts summarized in a more concise and ordered report (according to the pmbok sequence). keywords were used to identify the existence of control modes for each particular pmbok element. the summarizing process used a content analysis technique to paraphrase and remove unnecessary words (see bouma & ling 2004; czarniawska 2004). the summary scripts were analyzed using pattern matching (yin 1994) and processed through several iteration processes using the case ordered meta matrix method (miles & huberman 1994). these iterations categorized the projects and their project management elements into the appropriate control categories that have been established for each iteration. each iteration had its own category or ‘key words.’ the initial data were the conclusion script from the previous section. the first iteration identified the dominance of the control mode within each pmbok element. the australasian journal of information systems volume 15 number 1 december 2007 160 second iteration interpreted the ‘extensive’ use of the controls over the project duration and analyzed the trend throughout the duration. results the six in-depth studies took place across three different organizations: an investment corporation, a municipality, and an insurance company. table 3 details the existence of controls within the elements of pmbok for each project. australasian journal of information systems volume 15 number 1 december 2007 161 table 3 australasian journal of information systems volume 15 number 1 december 2007 162 initially the data was summarized according to the four different is project classifications, (a) purely internal, (b) a combination of internal and external where internal is more dominant signed as ‘more internal’, (c) combination of internal and external where external is more dominant signed as ‘more external’ and (d) purely external (table 4). the degree of dominance was determined by the vendor’s involvement in the project. there were four different classifications in the measurement criteria as described in the notes for table 4. proposition 1 predicted controls would be in place for each pmbok element. table 3 shows clearly the dominance of formal controls in most elements of pmbok providing strong support for this proposition. there are some exceptions, mainly in human resources and communications where control is shared between formal and informal modes. proposition 2 predicted that formal controls are applied to elements that provide clear requirements and can support the formal controls. table 4 shows that elements such as budget (cost), quality, and risk assessment made more use of formal controls. this was especially true for the procurement element in the purely external is project. proposition 3 predicted that for each pmbok element, in the absence of clear formal control requirements, informal controls will be adopted. the results of iteration 1 (table 4) show that primarily formal controls were dominant, extensively used, and useful at the project initiaton stage. however, some elements in the purely internal and more internal is project types show that informal controls were equally present at the project initiation stage especially where they related to people management. thus, there is moderate support for proposition 3. purely internal more internal more external purely external setup formal equal dominant dominant dominant informal equal not dominant na na scope formal dominant dominant dominant (?) dominant informal na na na not dominant time formal dominant dominant dominant (?) dominant informal not dominant not dominant na not dominant cost formal dominant dominant dominant dominant informal na na na na quality formal dominant dominant dominant (?) dominant informal na na less dominant (?) na hr formal na less dominant less dominant (?) dominant informal dominant dominant dominant (?) less dominant communications formal equal equal na dominant informal equal equal dominant (?) na risk formal dominant dominant na dominant informal less dominant less dominant less dominant (?) not dominant procurement formal dominant dominant dominant dominant informal na na na less dominant pmbok elements control modes is project types table 3: iteration 1, control dominance interpretation (notes: there are three measurement levels plus one dummy situation as follows: dominant clearly articulated; less dominant not clearly articulated, some impression exists; equal exist and share the same position equally; na not exist, unwritten, undetected) australasian journal of information systems volume 15 number 1 december 2007 163 the data were then analysed to determine how the formal and informal controls were exercised in accordance with the project phases of initiation, development and implementation (kirsch 2004, p. 390). the given value for each cell was the extended analysis from the framework table and interpretation from the original transcripts. the result of this process is summarized and presented in table 5. initiation development implementation formal extensive extensive less extensive decreasing informal less extensive extensive extensive increasing formal extensive extensive less extensive decreasing informal less extensive extensive extensive increasing formal extensive extensive (?) extensive (?) steady informal less extensive less extensive (?) extensive (?) increasing formal extensive extensive extensive steady informal less extensive less extensive less extensive steady more internal purely internal control modes project stages overall trendproject types more external purely external table 4: iteration 2, extent of use of controls interpretation the discussion above indicates that in the absence of clear formal control requirements, informal controls will be adopted prior to formal controls. all interview data indicated that the project managers applied strong formal controls at the project initiation stage (unfortunately, this study could not find example of new project without formal controls at the initiation stage). informal controls were less extensive but the trend increased as the project progressed. from the interview data, project managers used informal controls both to keep track of the project progress within their project team, such as informal meetings with the team to determine progress and to train new people using informal training or mentoring. the project managers increased the adoption of informal controls when facing unstructured problems and increased the formal controls as soon as the problem became clearer. for example, the project manager in project #3 used informal meetings with the stakeholders to anticipate changes during the development and reformulate the formal requirements after the changes had been clarified. thus, when facing a new challenge, project managers approach the challenge informally. this reasoning suggests moderate support for proposition 4. proposition 4 predicted that internal is projects have less rigid formal controls than external is projects whereas proposition 5 predicted that informal controls are more extensive than formal controls in internal is projects. table 5 and figure 2(b) show an increasing trend in the use of informal controls in the purely internal, more internal and more external project types. australasian journal of information systems volume 15 number 1 december 2007 164 figure 2(a): trend analysis formal controls (a block with crossed lines ( ) identifies a problem occurrence) figure 2(b) : trend analysis informal controls (a block with crossed lines ( ) identifies a problem occurrence)) project managers of these project types began to adopt informal controls, specifically during the project development stage. on the other hand, the use of informal controls in the purely external project type remained the same. interview data suggested that the usefulness of formal controls in several elements in the purely internal project type decreased over the project duration. hence, the adoption of formal controls became less rigid in the elements that showed a decreasing trend because project managers relied on more informal control elements as the project progressed. however, some of the control elements in the more internal and more external projects show a steady trend (extensive at the project initiation). consequently, the adoption of formal controls in these project types was still high as the project progressed. this implied that in the purely internal project, proposition 4 is confirmed but in the combination of project types, this proposition is only moderately supported. australasian journal of information systems volume 15 number 1 december 2007 165 proposition 5 predicted that external is projects have more extensive formal controls than internal is projects. table 5 and figure 2a show that the overall trend of using formal controls decreased over the project duration. the more external and purely external is projects showed that formal controls had been used extensively during project initiation, development and implementation. one important note is that the use of formal controls in the development stage of semi-more external projects caused problems. while most of the interview data supports this proposition, there was one exception from the iteration results. the more-external project type (project #4) showed that although there were strong formal controls at the beginning of the project, it suffered from inadequate infrastructure that would normally have detected the problems in the early stages. although the project manager stated that they adopted formal control mechanisms such as a project schedule, formal meetings, and written documentation, those mechanisms lacked the monitoring required from the project personnel, i.e., project staff lacked the management skills necessary to ensure the success of the formal controls. thus, there is moderate support for proposition 5. conclusions through the use of a different approach to data analysis, this study reveals some interesting findings and extends the results from previous research. first, this research analyzed the control adoption of the pmbok elements across all project types studied. this research found that formal controls were adopted for pmbok elements that could deliver clear outcomes, such as budget, quality, and procurement (especially when dealing with external parties). informal controls, on the other hand, were adopted mainly on pmbok elements that related to people, such as human resources and communications. this research also confirmed that some of the elements used both control modes such as in project set up (or project initiation) management, project time management and project risk assessment, and the degree to which both control modes were employed varied across the project types. second, from the control point of view, this study supported the results of choudury and sabherwal (2003), i.e., that formal controls were dominant at project initiation and in most cases these controls became less dominant as the project progresses. for example, as the project team started to get to know each other and adapt to the project environment, there is less reliance on outcome/behavior controls and the informal control adoption process becomes more important and extensive. moreover, this research extends choudhury and sabherwal’s (2003) work by considering internal is projects. we found that formal controls were more rigorously applied in a purely external is project. informal controls, on the other hand, increased in importance throughout the project duration in purely internal and more internal project types. in most cases, the purely internal and more internal project types shared a similar trend. the last contribution relates to the existing project management knowledge theory. this study found that the existing pmbok theory emphasizes ‘physical’ methods or, in this study, formal controls. the theory does not address the ‘unwritten method’ or informal approach. indeed, the data showed that the existence of informal controls is also important to ensure that the project delivers its outcomes successfully and this agrees with previous research findings (choudhury & sabherwal 2003; kirsch 2004). therefore, pmbok theory needs to be extended to incorporate informal methods of control into the existing body of theory. the results of this research should be interpreted taking into account the following limitations. first, the usual caveats associated with interview and qualitative research are acknowledged. second, only six projects were used during the research. third, the organizations selected were well australasian journal of information systems volume 15 number 1 december 2007 166 established but from only one geographical location. as such there is a potential lack of generalisability. there are a number of opportunities for future research with four mentioned here. first, the controls could be further segmented into outcome, behavior, clan, and individual controls. second, the study could be replicated using a more quantitative approach. third, the research could be replicated and also include “troubled” projects from different firms. finally, a longitudinal study could be performed to fully appreciate the controls as they evolve through the project lifecycle. references bouma, gd & ling, r 2004, the research process, 5th edn, oxford university press, melbournce. broadbent, m 2004, 'playing by new rules', cio, vol. tuesday, 13 january 2004, no. tuesday, 13 january 2004, tuesday, 13 january 2004. choudhury, v & sabherwal, r 2003, 'portfolios of control in outsourced software development projects', information systems research, vol. 14, no. 4, pp. 291-314. cio 2002, adventures in outsourcing, cio, viewed 19 august 2005, <http://www2.cio.com/research/surveyreport.cfm?id=10>. czarniawska, b 2004, narratives in social science research, vol. 1st, sage publications ltd, london. eisenhardt, km 1985, 'control: organizational and economic approaches', management science, vol. 31, no. 2, pp. 134-49. ferguson, c, finn, f & hall, j 2005, 'electronic commerce investments, the resource-based view of the firm, and firm market value', international journal of accounting information systems, vol. 6, no. 1, pp. 5-29. govindarajan, v & fisher, j 1990, 'strategy, control systems, and resource sharing: effects on business-unit performance', academy of management journal, vol. 33, no. 2, pp. 259-85. hartman, f & ashrafi, ra 2002, 'project management in the information systems and information technologies industries', project management journal, vol. 33, no. 3, pp. 5-15. johnston, ak 2003, a hacker's guide to project management, butterworth-heinemann, oxford. kirsch, lj 1996, 'the management of complex tasks in organizations: controlling the systems development process', organization science, vol. 7, no. 1, pp. 1-21. ---1997, 'portfolios of control modes and is project management', information systems research, vol. 8, no. 3, pp. 215-39. ---2004, 'deploying common systems globally: the dynamics of control', information systems research, vol. 15, no. 4, pp. 374-95. kirsch, lj, sambamurthy, v, ko, d-g & purvis, rl 2002, 'controlling information systems development projects: the view from the client', management science, vol. 48, no. apr, pp. 484-98. leach, l 2003, 'schedule and cost buffer sizing: how to account for the bias between project performance and your model', project management journal, vol. 34, no. 2, pp. 34-47. australasian journal of information systems volume 15 number 1 december 2007 167 miles, mb & huberman, am 1994, qualitative data analysis: an expanded sourcebook, sage publications, thousand oaks. montealegre, r & keil, m 2000, 'de-escalating information technology projects: lessons from the denver international airport', mis quarterly, vol. 24, no. 3, pp. 417-22. morris, pwg 2001, 'updating the project management bodies of knowledge', project management journal, vol. 32, no. 3, pp. 21-30. ouchi, wg 1979, 'a conceptual framework for the design of organizational control mechanisms', management science, vol. 25, no. 9, pp. 833-48. ---1980, 'markets, bureaucracies, and clans', administrative science quarterly, vol. 25, pp. 12941. pmbok 2004, a guide to the project management body of knowledge, 3rd edn, pmi publications. sauer, c, liu, l & johnston, k 2001, 'where project managers are kings', project management journal, vol. 32, no. 4, pp. 39-49. schwalbe, k 2004, information technology project management, 3rd edn, thomson. standish group 1999, chaos report, project overview. ---2001, extreme chaos, http://www.standishgroup.com/sample_research, accessed on 5 march 2005. ---2004, chaos report, 3rd quarter. turban, e, mclean, e & wetherbe, j 2004, information technology for management: transforming organizations in the digital economy, 4 edn, john wiley. wallace, l & keil, m 2004, 'software project risk and their effect on outcomes', communications of the acm, vol. 47, no. 4, pp. 68-73. weber, r 1999, information systems control and audit, prentice hall, upper saddle river. wideman, rm 1995, 'criteria for a project-managemet body of knowledge', international journal of project management, vol. 13, no. 2, pp. 71-5. yin, rk 1994, case study research: design and methods, 2nd edn, vol. 5, applied social research methods, sage publications, london. zobel, am & wearne, sh 2000, 'project management topic coverage in recent conferences', project management journal, vol. 31, no. 2, pp. 32-7. microsoft word ajis vol 15-1 m.doc australasian journal of information systems volume 15 number 1 december 2007 41 the value of mentoring in facilitating the retention and upward mobility of women in ict keri logan information systems department massey university, wellington, new zealand email: k.a.logan@massey.ac.nz barbara crump information systems department massey university, wellington, new zealand email: b.j.crump@massey.ac.nz abstract the low employment and poor retention of women in the information and communication technology (ict) work force remains a serious issue at a time when there is a shortage of skilled ict workers. effective intervention strategies such as mentoring have been found to contribute to the retention and promotion of ict women to senior decision making positions. using a family of concepts inherent in organisational and the ict occupational cultures as a framework of analysis this paper presents the results from interviews with 90 professional women in the new zealand ict workforce regarding their mentoring experiences. only the large government and international organisations provided formal mentoring programs, in which 12 of the women had participated. forty of the women had developed mentoring relationships serendipitously, usually with senior male colleagues. these relationships definitely contributed to their career advancement. a number of women wanted mentors but were unable to find them. if the industry is concerned about the lack of women in ict then it is recommended that where formal programs are not available access to external mentoring programs should be a part of the human resources policies for ict workers. introduction australasian journal of information systems volume 15 number 1 december 2007 42 with ict being acknowledged by many governments and organisations as the key driver to economic prosperity and with the need for skilled people in large ict projects such as those required for e-government, it is vital that all prospective talent is used. the demand for ict professionals in new zealand is predicted to increase steadily as government and the industry push innovative ict as the major pathway to future prosperity (hendery, 2006). the government’s ict taskforce aims to have the ict sector contribution at 10% of gross domestic product by 2012 (swain, 2003). yet in 2004 the ‘fill rate’ for ict professionals dropped from 89% in 2003 to 53% (department of labour, 2005). meanwhile the number of ict vacancies continues to increase and in may 2007 measured 1,607 (see figure 1); a substantial number considering new zealand’s small population (department of labour, 2007). figure 1: department of labour ict job vacancy programme reports recognising the need for a highly skilled workforce the taskforce identified the lack of women as an area which needs building and sustaining (ict taskforce, 2003). low employment and poor retention of women in the ict workforce has been well documented and is of concern both in new zealand and throughout the developed world. yet women represent a substantial pool of talent and a huge resource into which the industry could tap. the arguments for achieving a more equitable balance of gender within it are similar to those put forward for diversity of individuals in other male dominated fields – not just for fairness and equity, but because an opportunity cost is paid in that certain designs are not considered and therefore products are not built and processes not invented (wulf, 1998). the consequences of not having input from women are serious as without them male perspectives will continue to dominate the way in which the technologies are developed and used and, as wajcman (2004) writes, “unless women are in the engine rooms of technological production, we cannot get our hands on the levers of power” (p111). thus, many rewarding careers will not be available to women and they will not be part of developing the new technologies that will impact on their lives. they will also be left out of the decision-making processes when it comes to development of, and policy-making about, technology (aauw, 1999; unctad, 2002). a further argument is that diversity in ict teams increases creativity and innovation, both essential elements in the development of good ict projects (barkema, et al, 2002). the large retrenchment suffered by the ict industry in the early 21st century impacted on women’s retention in the industry where the lack of female mentors and career support made it difficult for women to find jobs in the industry (cone, 2007; bartol & aspray, 2006). a recent report on the ict workforce in the united states shows a trend towards organisations increasing their ict staffing levels (zweig et al., 2006). but the falling working population in western countries, attributed to the australasian journal of information systems volume 15 number 1 december 2007 43 lower birth rate and retirement of the bubble of “baby boomers”, means the numbers available for the workforce over the next few decades will substantially decrease (feyerherm & vick, 2005; schwarz, 2005). as a result many human resource departments are developing intervention programmes in an effort to encourage and retain valuable staff. despite such proactive efforts to redress the lack of women, the percentage of women compared to men at the higher levels of ict remains disparate. this is of concern when studies have revealed a correlation between organisational profitability and a higher level of top female representation (greene, 2004; feyerherm & vick, 2005). intervention strategies such as mentoring (which has long been recognised as a significant factor in retention and progression in all professional careers for both male and female employees) are one way of attracting and retaining suitable staff (ahuja, 2002; friday, et al., 2004; kram, 1985; pfleeger & mertz, 1995). mentoring is said to give the protégé a better understanding of organisational politics, the gaining of reflected power and visibility, acceptance within the culture, and therefore the achievement of promotion up the managerial ranks (headlam-wells, 2004). this paper draws on the results of two recent new zealand studies that explored the nature of the ict environment and culture, and the perceptions of the women who worked in it. the aim of the research was to increase our understanding of new zealand’s ict work environment in which women are the minority gender. the results that this paper presents refer to the views of the women and their mentoring experience only and has the following objectives: to investigate the nature and extent of mentoring within the ict workplace; to identify whether successful mentoring led to greater career opportunities. mentoring mentoring relationships are personal, reciprocal and dynamic associations whereby a “high-ranking, influential member in your environment who has advanced experience and knowledge … is committed to providing upward mobility and support to your career” (ragins & cotton, 1999, p535). the term itself is rooted in ancient greek mythology where odysseus committed his son telemachus to the teachings of his friend, mentor. since then trusted advisers have been referred to as mentors. mentors can come from within or outside the protégé’s organisation, or through formal or informal arrangements. very often those that begin informally are more effective than those that are formally arranged. ahuja (2002) theorises that this is because interpersonal attraction may be an essential factor in successful mentoring relationships. some research also suggests that women are more comfortable with informal relationships that have developed through friendship (linney, 2000). nevertheless, formal mentoring programmes are becoming more common in the corporate world, as well as in government and non-profit sectors (marquardt & loan, 2006). this is where an organisation authorises a partnership between a mentor and mentee and where the intent is understood and the objectives are clear (linney, 2000). for example, prompted by the small number of women throughout the ict industry hewlett packard introduced a formal mentoring programme within their organisation (thorp, 2004). this programme used external rather than internal mentors because of the difficulty they foresaw in maintaining relationships due to the high international movement of their own senior staff. it has proved so successful in australia that it is being extended throughout the asia-pacific region. ibm has also been proactive in supporting mentoring relationships, on a formal and informal basis, both within and outside the company. at least one other programme in new zealand has been established specifically to encourage women in it. australasian journal of information systems volume 15 number 1 december 2007 44 women in technology (wit), an organisation set up to support and promote women at all levels of the ict industry, trains and provides external mentors on a formal basis for women interested in advancing their ict careers (nz trade and enterprise, 2004). according to johnson and ridley (2004) outstanding mentors are selective in their choice of protégé and are careful to ensure that the “match” is suitable as they invest considerable time and energy, and emotional and professional resources encouraging, teaching and supporting them in their career. therefore they need to ensure that the protégé has the appropriate talent, ambition and interests to make their investment worthwhile. this can, of course, lead to resentment by those who have not been chosen (linney, 2000). it is generally agreed that benefits accrue to both mentors and mentees. mentors gain from being able to enhance their management skills and gain recognition from their peers. they also benefit through opportunities to network with different people and gain personal satisfaction from knowing they have helped someone else. it provides an opportunity for self-reflection and self-renewal (marquardt & loan, 2006). for the protégé, the benefits of having a powerful mentor are manifold. they can provide their protégé with visibility, an advantage acknowledged as being a significant factor in progression within an organisation. this is important for women as they often do not have the same access as males to informal networks. mentors can be powerful sponsors for their protégés, helping them to fast track through the ranks by giving them the opportunity to share, observe, discuss and learn from an experienced professional (alboim, 2002). they can be particularly helpful through the sharing of tacit information, which leads to an understanding of the values and culture of the organisation (marquardt & loan, 2006). a further important characteristic is their ability to advise them on political factors, such as who is influential within the organisation (ragins & cotton, 1991). female mentors for female protégés are sometimes considered more suitable because of an essentialist belief they are better able to empathise and understand those difficulties pertaining specifically to women. as well, having a mentor of the same gender can make it more realistic for women to see themselves attaining similar goals. yet the skewed gender ratios in the ict industry means that if senior women commit to mentoring roles this places yet more pressure on them, many of whom are already precariously balancing home and work responsibilities (lyness & thompson, 2000; pfleeger & mertz, 1995). generally males are considered to be more effective because of their access to formal and informal networks to which they can introduce their protégés and because they are more likely to have legitimate status and power within the organisation. this is a direct result of the well documented shortage of females in senior ict positions and their traditional lack of access to networks and associates essential for gaining visibility and learning about the largely foreign male culture (sernak & may, 2003; wajcman, 1998). for example, wajcman (1998) believes that the “support and sponsorship from a male boss is crucial for the advancement and promotion of both men and women in organizations” (p125). on the other hand she also considers that a major “problem is that a mentoring relationship is essentially about a powerful older man identifying with a younger version of himself” and that version is usually a young male, thus females are less likely to be selected as proteges. on a different dimension it is suggested that senior men are sometimes reluctant to enter into crossgender mentoring relationships with junior women as others in the organisation may misconstrue such an approach as a sexual advance and the innuendo accompanying this tends to demean the woman rather than the man (ragins & cotton, 1999). similarly, some women are reluctant to establish a mentoring relationship with a male, particular a powerful one (ragins & cotton, 1991). concern has also been expressed that males do not care whether or not women are gaining australasian journal of information systems volume 15 number 1 december 2007 45 promotion to senior positions. when pfleeger and mertz (1995) developed an executive mentoring strategy for women and minorities in ict they discovered that senior men “were not interested in encouraging women and minorities to move up in their organizations” (p64). organisational policies, and the guidance and sharing involved in mentoring are situated within a socio-cultural environment that is influenced by the occupational it culture. there is evidence from the literature that cultural artefacts, beliefs, image and values, for example, language and esoteric knowledge requiring continual updating and learning, shape this environment. the following section discusses the multi-dimensional nature of the ict environment. the ict environment and culture one of the most widely read and replicated researchers into culture, geertz hofstede, collected and analysed information from people working in multi-national corporations across 66 countries. hofstede (1980) argues that culture is historically derived and is a collection of selected ideas and their attached values which have proven to work for a particular group. it is a function of the individual and their interaction with their social environment. every group or sub group has a common pattern of beliefs that makes up its culture and these values and beliefs affect group members’ interactions according to attributes of the specific occupational subculture (trice, 1993). the relationship of the multi-dimensions of culture are discussed by kumar and bjorn-anderson (1990) who suggest that design and management choices in information systems are the result of the designers’ individual values and these values are the product of the socio-cultural environment of their professional (or occupational), social and organisational context. the notion of an occupational culture that exists within an organisational culture has only been explored by scholars in recent years. an occupational culture is independent of the particular organization and is distinguished by their members tending to emphasise what makes them similar to each other and different from others (ogbor, 2001). historically such cultures were often associated with blueand white-collar workers but new occupational groups such as computer technicians or financial traders are evidence of emergent occupational cultures (heery & noon, 2001). as with national and organisational cultures, occupational subcultures are not static but as hofstede (1980) emphasises change is more likely to be at the superficial level rather than affecting the fundamental values of the group. change is slow because these values are deeply embedded in the group psyche. nevertheless, as different individuals move in and out of the group and the external environment changes so will the culture (newman, 1995). shultz (1995) states that subcultures contain shared or non-shared webs of meaning where reality is defined as subjective and multi-dimensional, with the possibility of different meanings attached to the same phenomenon and, conversely, the same meaning may be conveyed by different phenomena. thus there is a local creation of meaning that reflect the symbols, artefacts, social image of the ict occupation, cultural forms, pervasiveness, consciousness of kind, extreme or unusual demands and esoteric knowledge. further, the creation of meaning is influenced and related to gender, generation and social class and may be at odds with organisational values (hofstede & hofstede, 2005). the ict workplace literature reports the consequences of cultural practices that reflect the values, beliefs, image, language and behaviour of members of the ict subculture. examples include: the poor retention of skilled personnel (moore, 2000; tapia, et al., 2004), the management of the sometimes extreme and unusual demands of the profession that lead to “burn out” and exhaustion (moore, 2000; sethi, et al., 1999) and, consequently work-life balance (ahuja, et al., 2006; moore, australasian journal of information systems volume 15 number 1 december 2007 46 et al., 2006), the image of the ict professional (roldan, et al., 2004), the “chilly culture for women in the ict workforce” (faulkner et al., 2004), an examination of gender effects on the career success of information systems employees (baroudi & igbaria (1994/95) and the low participation rate of women in the industry (baroudi & igbaria, 1994/95; british computer society, 2001; faulkner et al., 2004; griffiths, et al., 2007; ramsey & mccorduck, 2005). in most developed countries the ict industry is disproportionately dominated by white males. in all but the lower-paid data entry levels of computing, females fail to reach a critical mass of 30 percent. the ensuing culture has evolved around masculine values (grundy, 1996; kiesler, et al., 1985; spertus, 1991; margolis, et al., 2002) and the environment is said to be one where members are insular, individualistic, competitive and anti-social (hendery, 2006; sproull, et al., 1984; tapia, et al., 2004; woodfield, 2002). working in this type of setting is often not appealing to women whose patterns of values are said to include a preference to be more collaborative, inclusive, sharing and non-confrontational (brush, et al., 2004; debare, 2003). one effect of this male dominated workplace is that women often find their interests and values trivialised leaving them the options of conforming to the widespread male culture, working in an alien environment or searching for other work opportunities. the introduction of intervention and diversity strategies, such as mentoring, can help to alleviate tensions caused through working in the male dominated ict occupational subculture. the mentoring results from two recent studies of new zealand ict female professionals are presented in the next section. we begin with a description of the research design of the two studies, followed by the results. analysis was guided by a socio-cultural framework that references organisational and occupational sub cultures that are, in turn, interpreted and informed by the mentoring literature. by using this “family of concepts” of culture (pettigrew, 1990) we are better able to provide a frame of reference for viewing the impact of mentoring. studies and method the mentoring data on which this paper based is derived from two studies in which 90 professional women working in the ict industry were interviewed. the first exploratory study consisted of indepth, unstructured interviews in the city of wellington, new zealand with 20 women aged between 24 and 45, all of whom held responsible positions in their work place. a life story method was used whereby the women were encouraged to reflect on their working life in the industry and to voice their perceptions and experiences (cox, et al., 1994). it was felt necessary for the participants to have the freedom to shape their own accounts and to reflect on their relationships and histories that had influenced their careers in ict. these unstructured interviews allowed the women to express themselves in their own way, at their own pace and emphasised the importance of understanding the context in which events take place. a snow-ball sampling system was used (trochim, 1999). this is a technique whereby interviewees are asked to recommend further participants from their acquaintances, thus the sample appears to grow like a snowball. although participants may recommend friends and associates with similar views to their own, thus potentially creating a biased sample, it was considered an appropriate method for this study. bias could be offset by giving participants freedom to express their views through the types of interview adopted. therefore, interviews were initially undertaken with acquaintances and friends in the industry who then referred the researchers to other women working in similar areas. seventy women, aged between 23 to 61, from auckland, wellington, christchurch and dunedin, new zealand’s four major cities, were interviewed. again, all the women worked in skilled to australasian journal of information systems volume 15 number 1 december 2007 47 highly skilled areas of the ict industry, in a variety of occupational sectors. funded by a grant from the society for research on women, the new zealand computer society (nzcs) and the women in technology (wit) group advertised the details on their web sites. most of the women contacted the researchers via email after visiting these sites. prior to the interviews the women were asked to fill out a form giving demographic information regarding age, salary range, and type and size of organisation. a checklist of questions that reflected themes from the it professionals’ literature and results from the first study was devised by the interviewers. the themes included entry into it, employment practices, the cultural image of computing, mentoring and the future. these served as a focus for the interviews yet did not constrain the women from the freedom to discuss issues they felt were important and relevant. the intent was to allow the women to speak for themselves, giving their own perspectives from their own experiences. therefore the checklist was used as a guideline to ensure the required topics were covered with all interviewees. the first question all the women were asked was how they began their ict careers. in answering this question the women invariably brought up other associated issues. for example, after explaining that the high salaries was the main driver for choosing a career in ict, one woman then began discussing the difficulties of women working in a male dominated environment. she felt men and women needed different support networks around them, and that “the talk is quite different when groups of males get together from when groups of females get together – they will slip into male ways of talking with lots of swearing, every second word is a swear word”. for another woman the same question led into a discussion about ict recruitment policy and equity at her present work place. in order to minimise the potential for bias a minimum of two interviewers attended all interviews in both studies. these were audio-taped (after ensuring that the interviewees were agreeable) and transcribed within a week of each interview. patton's (1990) suggestions for content analysis guided the identifying, coding and categorising of the primary patterns in the verbatim transcripts. analysis was informed by the “family of concepts” (pettigrew, 1990) inherent in organisational and ict occupational cultures and framed by seeking answers to the questions: 1. how did mentoring facilitate enculturation into the ict occupational subculture? 2. are formal mentoring programmes a positive strategy for challenging the ict occupational subculture? 3. did mentoring progress careers and provide confidence? 4. does a mentor’s gender affect the mentoring outcomes for women? 5. what are the difficulties in finding a mentor? the headings for the following results section relate to these questions which are derived from the content analysis that reflects the major concerns and interests that the women expressed about mentoring. results fifty two of the 90 interviewees claimed to have had a mentor during the course of their working life. they variously described their mentor as ‘friend’, ‘confidant’, ‘guide’, ‘a wise man’. their descriptions fit ragins and cotton’s (1999) definition of a mentor as a senior and experienced member of an industry who supports, guides, offers practical help, and brings wisdom and constructive ideas to a younger, inexperienced member. the women were asked if they believed mentoring helped their enculturation into the ict occupational subculture and to describe the outcome. australasian journal of information systems volume 15 number 1 december 2007 48 enculturation into the ict occupational subculture understanding the common patterns of values and beliefs held by the members of the ict subculture and the wider organisation was recognised by many women as critical to negotiating office politics. jane appreciated the role her mentor played in helping her understand and be accepted into the culture of both the organisation as a whole and the ict department in particular and her comments corresponded with many of the other women in that: it was all about handling the politics, which is the key, key thing. who are the important people, who are the people whose opinion really matters and how do you nurture and make, build and nurture relationships with those people. what to do with the tricky hr type issues or non-performers. career decisions. he helped me a lot with organisational culture and introduced me to a lot of people which was excellent. jane’s mentor provided a gateway into understanding the patterns or systems of values that hofstede and hofstede (2005) term “mental programmes” and which they note as being shared with some but not with other people. his ability to advise her on political factors was recognised by jane as a critical and positive feature of her mentor’s guidance; identified by ragins and cotton (1991) as a very important contribution. gaining visibility through having a high profile mentor was another important factor in the enculturation process. wajcman (1998) highlights this when she states that visibility, acceptability and networking go hand in hand. the advantage of being mentored by a senior, experienced sponsor who shares tacit information that leads to an understanding of the values and culture of the organisation and group (marquardt & loan, 2006) proved to be a valuable and supportive experience for wilma. wilma purposely chose a man in his 50’s to be her mentor to help her deal with male staff whom she felt were circumventing her because she was female and therefore not acceptable as their manager. he is an older experienced gentleman because i can get into quite aggressive relationships when staff are questioning me as the decision maker. i chose him specifically to guide me through some of those, the politics and culture and his experience in all of that. because i see that quite strongly as in when they [staff] first meet you and they are sort of questioning as if “are you really the manager” while they reported directly to me they were always trying to get around to the big kahuna [the cio] to whom only i reported. he has been invaluable guiding me through that. formal programmes increasing numbers of organisations are implementing diversity programmes that include policies on non-discrimination, inclusion of people with a disability, work-life balance, cultural diversity and mentoring. the women were asked if their organisation offered a formal mentoring programme and their responses revealed that 7 out of the 68 companies in which these women worked had a formal system in place. the companies were all large multi-national or government organisations and the schemes were mainly for new employees to ease them into the job and facilitate their productivity as quickly as possible. nine of the 12 women who worked for these organisations had found the mentoring useful. typical of their responses is the following from anita: when they asked me to come into the is structure, yes i had a mentor assigned formally then. i think it is a very important part, coaching and mentoring and providing that for people if you are expecting them to shift or change what they are doing or wanting to accelerate their progress. i have got three people assigned to me now. so no i believe in it very strongly. australasian journal of information systems volume 15 number 1 december 2007 49 anita acknowledged the value the mentoring programme had in the enculturation process and as a support when she joined “the is structure”. she recognised that the way a person behaves in a group has more to do with the group than the individual. anita’s mentor eased the “fit” and the interaction between the person and their socio-cultural environment thus causing the adaptation of the behaviour of the individual to fit those of the collective . for some of the women such formal programs were not practicable or flexible enough to accommodate some individual circumstances. for example, jaquie explained why she had not become involved in the program: we do have a mentoring policy in this organisation, but i think because, well first i worked from home for such a long time and then i worked for a client at their site and i have missed out on a lot of the things that probably i shouldn’t have in that way. i’d say no, i haven’t really been mentored, and yes it would have been good. this highlights two points. first, working from home or teleworking is often recommended as a way of addressing the difficulties women in ict face in balancing home and work responsibilities, yet in such situations it is difficult to develop a mentoring relationship. an alternative is to conduct the relationship via the internet which requires commitment by both parties. second, it is common for ict workers to be sent to client sites for lengthy periods and this also tends to act negatively on the formal mentoring programmes. mentors are usually senior members of the organisation and therefore ‘time scarce’, and unless the protégé is proactive in such situations the contract is almost certain to break down. joan works in an organisation where it is the policy for all senior members to mentor graduates as part of leadership development. she believed the formal mentoring program had never worked for her, saying “i did have a mentor in senior management but i could never meet up with him because he was either too busy or not available”. her comments about mentoring are mixed as she did “not really like mentoring people much”. she stated: what do you tell them? when i mentor people i usually try and help sort out their personal problems rather than their work things, things that are affecting their lives … actually i have a graduate that i am supposed to be mentoring this year and i haven’t done anything with her, which is really poor isn’t it?” joan’s comment on sorting “out their personal problems” reflects her psychosocial approach by providing counselling but her ambivalence about the usefulness of mentoring obviously impacts negatively on her own mentoring duties. her belief that the mentee “should drive the process – if they want to put some time into it and push you to meet with them then i will always meet with them” may not be suitable for all mentees, especially a new graduate. while this attitude may be satisfactory for those who have the confidence to drive the process, as a young woman working in a male dominated environment it is not always easy to be forceful with a senior member of staff. no doubt there are other mentors with similar perceptions as to the value of mentoring and without total buy in of the staff involved the objectives of the scheme would be somewhat diminished. kram and isabella (1985) state that it is necessary for a mentoring relationship to be vital and significant if both individuals are to benefit from the experience. to ensure positive outcomes they note that there needs to be complementarity of needs that will then propel it forward to the cultivation phase where effectiveness is maximised. an organisation that implements a mentoring programme must therefore ensure that mentors and mentees perceive benefits from involvement in the scheme. career development and confidence reflecting on their experiences many of the women believed that their mentor had been critical in helping them advance their careers. most of the mentoring relationships were formed australasian journal of information systems volume 15 number 1 december 2007 50 serendipitously, usually because a senior male identified the woman as having potential and took a greater than usual interest in her. for example, julie a very successful and senior ict professional, was mentored by an internationally renowned programmer when she joined a specialised ict company as a young, and the sole, female in her division. her mentor singled her out as his protégé and julie attributes her success in the ict profession to the time and energy that he invested in her. johnson and ridley (2004) point out that outstanding mentors carefully select their protégé and julie’s experience is an example of this. his belief in her abilities gave her confidence, she said, to attempt many other things. they also developed a special friendship which continued until he died. friendship and respect were two of the major components many of the women noted of their successful relationships. elements of the following quote from julie were evident in the accounts of other interviewees: he took me under his wing and gave me games to play and things to figure out and stretched me and really wanted to make me into something that was a singing, dancing version of consulting. he saw some of the same sparks as were in himself i think – he was very dry, very internal sort of focus, very nice. then they sent me to america when i was 21 and because he assumed i could do it, so did i. wajcman’s (1998) observation that “a powerful older man” invariably selects a younger, male version of himself is confirmed in one respect. julie believed that he “saw some of the same sparks as were in himself” and, notwithstanding her gender, was prepared to guide, mentor and progress her career. julie’s close relationship with her mentor nurtured her development and career by guiding her growth within the ict socio-cultural environment and contributed to sense-making of her participation within that world and various aspects of herself such as confidence (jenks, 1983). similarly mary, now a high profile woman in the ict industry explains: yes numbers of mentors and all men, which is interesting isn’t it. a couple of them have died which has been a bit sad, but people like [name not given] he sort of gave me, like a chance. i had this wonderful project proposal and he said to me if you have the energy to make it last, make it fly. you never forget that do you. he just took a punt. situations arose where clients assumed that male colleagues were the manager and ignored the woman who was the manager or male colleagues ignored the women’s ideas at meetings, yet when a male repeated them they were considered his ideas. such invisibility was a recurring problem for many. the ability to talk through these problems with a mentor consistently led to successful solutions. problems such as invisibility are inherently socio-cultural and are connected to underlying beliefs and images of the ict professional as male. gaining visibility and support through having a high profile mentor was an important factor in negating the norms of the ict occupational culture within which these women worked. female versus male mentors understandably, given the skewed gender balance in it, most of the mentors were male. margaret, who had unsuccessfully sought a female mentor, believed that apart from the small number of senior women in the industry, many were “not interested [in becoming mentors] as they don’t have the time with all their other responsibilities”. the women did not consider the predominance of male mentors a problem and emphasised that the most important factor was having someone they could respect, who could give appropriate advice and who had good attitudes and approaches. they also appreciated the fact that because their mentors were experienced ict professionals and had large networks of contacts they were able to provide visibility, as well as help them understand the australasian journal of information systems volume 15 number 1 december 2007 51 culture and deal with the different ways in which males and females tended to operate in the ict workplace. nevertheless, some spoke about the differences in relationships between male and female mentors and how sometimes it was better to have a female who could understand their uniquely female issues and problems. merle, for example, observed that: it is nice to sit down and talk with them [females]. you can relax a bit more and bring in other issues. with a male you’d probably just sit there and talk about work and that’s it. mary also spoke about some of her “very strong minded” friends losing confidence after being absent from the work force through parental leave and believed that for women returning to the ict workforce, after even a short absence, a female mentor who has been in a similar situation and survived, could have a huge impact on their retention. this was confirmed by judith who discovered that her female mentor: … could understand my problems with working, pregnancy and having a small child. she reassured me that the awful time would go and that i just had to hang in there – it was ok, possible to work and have my family. although it is generally accepted these days that women have careers, the assumptions that maintain the gender specific division of labour around family and home responsibilities remain largely unchallenged leaving women the predominant caregivers. striking a balance between work and home responsibilities is a worry to many women (bartol & aspray, 2006) because of the need to work difficult hours and maintain currency in a rapidly changing technological world. a number of the younger interviewees (30-40) expressed concern about their ability to continue working in ict if they had children. while not suggesting that all females have identical problems, or are even more empathetic towards domestic and family situations, in some circumstances female mentors may be more appropriate, even though they may not have the same status and power of their more senior male colleagues. the difficulties of finding a mentor at least a dozen women were desperate to find a mentor but had found it difficult. one interviewee had approached seven people before finding a person who agreed to be her mentor. reasons for desiring such an alliance were loneliness, a need for a support structure, receiving good advice and “just working out a way [into the group]”. lianne’s account of her feelings demonstrates one effect of being a minority. she is not concerned about sexual harassment but suggests that it is this fear from her male colleagues which creates a barrier to more collegial and supportive working relationships. because particularly now, i guess, i feel quite left to my own devices really at work. i’m not isolated because there are a lot of other people within the industry. but i have no-one who understands what i am doing. i haven’t ever had a mentor. and you know it is a very dry business relationship. i think they are scared to know me closer because you know they might get sexual harassment or something. and we never meet outside work. i think if there were more women in my area that work with me quite possibly we would have a slightly warmer relationship. commenting on what she would do differently if she had her time again, barbara, a mature woman running her own consultancy, explains: one of them is finding mentors. i never really had a mentor and i think that is something i’d benefit from, and i don’t quite know how you do that. and it’s ridiculous really because i am a good networker. and so getting some good advice, australasian journal of information systems volume 15 number 1 december 2007 52 good support and just working out a way in, i never really worked out a way in at [last place of work]. and maybe some of that is a girly thing as well, about the confidence to actually ask for it. the comments of these women reflect the isolation some women experience working in a sociocultural environment where males are the dominant gender and consequently the masculine norms and values of the occupational ict culture is gendered male (moore et al., 2006; tapia & kvasny, 2004). one woman said “if i could get one now [mentor] i would be more inclined to stay” indicating that having a mentor may be a significant factor in female retention. in an attempt to overcome this isolation a number of the women belonged to wit and were participating in their formal mentoring scheme, both as mentors and mentees. conclusions ict remains one of the few sectors where men significantly dominate the work force (hendery, 2006). consequently, outdated patriarchal attitudes continue to influence and define the social system (hofstede, 1980) as indicated by the women’s complaints about being invisible. therefore it has been useful to view the data through a “family of concepts of culture” (pettigrew, 1990) that includes organisational culture and the ict occupational culture with its values and behaviours dictated by the majority, male group. the retention and promotion of women in the ict environment is a complex issue. obviously mentoring is not the only means of leveraging their opportunities, but for those who have the good fortune to become involved in mentoring relationships the consequences are highly beneficial. it is clear that good mentoring facilitates upward mobility yet most of the mentoring relationships were informally developed. these results reinforce previous research (ie that having an experienced and senior person as a mentor gives the employee a number of benefits). they have helped their protégés gain the confidence to take on difficult projects; they have extended their knowledge, given them visibility, supported them in difficult situations, guided them through tricky management situations and taught them how to deal with the more covert political aspects of their workplace. they have also provided them with opportunities to network and make the contacts which are so important to gaining promotion. it is this interaction between the protégé and their social environment that gains them acceptance in the ict work place. why then, do so few new zealand companies offer formal mentoring schemes for their ict employees? is it because the successful relationships are those that are formed on a casual basis through recognition of ability and mutual respect? is it because mentoring is not considered necessary by the organisation? is it because top management is generally male and therefore they do not consider a need to focus on developing female talents. or is it, as ahuja (2002) posits, that formal mentoring schemes do not match personalities satisfactorily and therefore are not successful? in the end it may be that interpersonal attraction is the critical success factor in such relationships. interpersonal attraction certainly was a major factor in the development of many of these women’s mentoring relationships. the results and analysis of the 90 interviews have identified both positive and negative aspects of mentoring from which the following recommendations are made: 1. ensure formal schemes involve commitment by both mentor and mentee. australasian journal of information systems volume 15 number 1 december 2007 53 when providing a formal mentoring scheme it must be seen to be supported by senior management, and implemented in a positive manner. while there may be an expectation for senior employees to act as mentors, participation should be voluntary. feed-back mechanisms should be put in place to ensure that both parties are suitably matched and result in constructive outcomes. there should be a guarantee that the mentor will be willingly available on a regular basis and a clear outline of the intent and objectives. however, the nature of the relationship must be defined by the participants and the mentor must take into account the needs of the mentee. 2. encourage and facilitate informal mentoring relationships if a formal scheme is not considered viable, then the ict department should encourage female staff to form such relationships outside of the workplace, within organisations such as wit and the computer society. such organisations offer mentoring opportunities for their members for a relatively small payment. they spend considerable time in matching mentors with mentees so that satisfactory outcomes are produced. as well, they provide their members with opportunities to network and meet with influential and experienced ict professionals. they have regular breakfast, lunch and evening meetings with guest speakers and provide time for members to interact and get to know each other. such informal activities encourage the development of networks and contacts that may be available to act as informal mentors. this would be particularly useful for those women who found it difficult to strike a mentoring relationship yet felt an overwhelming need for some sort of guidance and support, particularly when a feeling of aloneness was the reason. these feelings originate from standing apart from the dominant group. 3. provide an inclusive and responsive mentoring scheme for part-time and teleworkers who encounter administrative difficulties in developing a mentoring relationship, encourage participation in e-mentoring schemes. women in ict are already fluent in their use of computers and comfortable with using the internet. many use chat rooms and women’s lists for other purposes. there are now a variety of excellent ementoring schemes and ict departments should encourage their female staff to participate in such schemes as a way of managing their careers and promotion. e-mentoring schemes do not make the women more visible within their organisation or local community, or raise their profile. nevertheless they do provide a degree of support and advice and thus contributes to alleviating some of the sense of “aloneness” that the women who wanted a mentor expressed. finally, regarding those women who unsuccessfully tried to attract a mentor, there are several questions that need to be asked. was their lack of success because the men did not see them as a good “investment”? and if so, was this because they were female and not in their own male image? references american association of university women (1999) gender gaps. where schools still fail our children, marlowe & company, new york. ahuja, m.k. (2002) “women in the information technology profession: a literature review, synthesis and research agenda”, european journal of information systems, vol 11 no 1, pp 20-34. ahuja, m., k., chudoba, k., m., kacmar, c., j. & mcknight, d. h. (2006) “it road warriors: balancing work-family conflict, job autonomy, and work overload to mitigate turnover intentions”, mis quarterly, vol 30 no 3, pp 1-18. australasian journal of information systems volume 15 number 1 december 2007 54 alboim, n. (2002) fulfilling the promise: integrating immigrant skills into the canadian economy, caledon institute of social policy, ottawa. barkema, h.g., baum, j.a. & mannix, e.a. (2002) “management challenges in a new time”, academy of management journal, vol 45 pp 916-930. baroudi, j.j., & igbaria, m. (1994/95) “an examination of gender effects on career success of information systems employees”, journal of management information systems, vol 11 no 3, pp 181-102. bartol, k.m. & aspray, w. (2006) “the transition from the academic world to the it workplace” in j. mcgrath-cohoon & w. aspray (eds.), women and information technology. research on underrepresentation (pp. 353-377, the mit press, london. british computer society. (2001) “it employment: gender bias in it the insoluble problem?”. http://www.bcs.org.uk/review/2001/html/p237.htm accessed 9 october 2001. brush, c.g., carter, n.m., gatewood, e., greene, p.g., & hart, m.m. (2004) clearing the hurdles: women building high-growth businesses. nj: pearson education, upper saddle river. cone, e. (2007) “why do women leave it” cio insight”. http://www.sioinsight.com/print_article2/0,1217,a=209724,00.asp accessed 27 june 2007 cox, a., dickinson, h. & parsons, v. (1994) “gender and computing in further education: a life story approach” the vocational aspect of education, vol 46 no 3 pp 297-305. debare i. (2003) “high-tech industry zipping along, but women often are left behind. women in computing, part 1”. http://www.sacbee.com/static/archive/news/projects/women/wcm accessed 12 november 2003. department of labour (2005) “information technology professional: occupational skill shortage assessment”. http://www.dol.govt.nz/publications/jvm/professionals/2005/ict.asp accessed 27 june 2007. department of labour (2007) “findings from the information technology recruiters survey 2006” http://www.dol.govt.nz/publications/jvm/recruit/2006/summary.asp accessed 27 june 2007. faulkner, w., sørensen, k., gansmø, h., rommes, e., l. pitt, berg, v. l., et al. (2004) “gender inclusion within the ict workforce” http://www.sigis-ist.org: accessed 29 august 2007. feyerherm, a. & vick, y. (2005) “generation x women in high technology. overcoming gender and generational challenges to succeed in the corporate environment”, career development international, vol 10 no 3, pp 216-227. friday, e., friday, s.s. & green, a.l. (2004) “a reconceptualization of mentoring and sponsoring”, management decision, vol 42 no 5, pp 628-644. greene, k. (8 november 2004) “women at the top – a smart move”, the dominion post, c4. griffiths, m., moore, k., & richardson, h. (2007) “celebrating heterogentity? a survey of female professionals in england”, information, communication & society, vol 10 no 3, pp 338357. grundy, f. (1996) women and computers. whittshire: cromwell press. headlam-wells, j. (2004) “e-mentoring for aspiring women managers”, women in management review, vol 19 no 4, pp 212-218. australasian journal of information systems volume 15 number 1 december 2007 55 heery, e., & noon, m. (2001). a dictionary of human resource management. oxford: oxford university press. hendery, s. (2006) “it grappling with image problem”, the new zealand herald, http://www.nzherald.co.nz/section/story.cfm?c_id=5&objectid=10390657 accessed 13 july 2006. hofstede, g. (1980) culture's consequences. international differences in work-related values, sage publications, new delhi. hofstede, g. & hofstede, g.j. (2005) culture and organizations. software of the mind, mcgrawhill, ny. ict taskforce (2003) breaking through the barriers, http://www.nzte.govt.nz/common/files/ictfinal-report.pdf accessed 15 july 2003. jenks, j. a. (1983). the mystery of women's early adulthood. clinical social work journal, vol 11 no 1, pp 52-63. johnson, w. & ridley, c. (2004) “the elements of mentoring”, palgrave macmillan, ny. kiesler, s., l. sproull and j. eccles (1985) "pool halls, chips and war games: women in the culture of computing." psychology of women quarterly, vol 9 no 4, pp 451-462. kram, k.e. (1985) mentoring at work: developmental relationships in organizational life, scott foresman, glenview, ill. kram, k.e., & isabella, l.a. (1985) “mentoring alternatives: the role of peer relationships in career development”, the academy of management journal, vol 28 no 1, pp 110-132. kumar, k., & bjorn-anderson, n. (1990) “a cross-cultural comparison of is designer values”, commuinications of the acm, vol 33, no 5, pp 528-538. linney, b.j. (2000) “mentoring: women learning from others”, physician exec, vol 26 no 1, pp 7275 lyness, k.s. & thomson, d.e. (2000) “climbing the corporate ladder: do female and male executives follow the same route?” journal of applied psychology, vol 85 no 1, pp 86-101. margolis, j., & fisher, a. (2002) unlocking the clubhouse: women in computing, mit press, cambridge, ma. marquart, m.j. & loan, p. (2006) the manager as mentor, praeger publishers, westport, ct. moore, j. e. (2000) “one road to turnover: an examination of work exhaustion in technology professionals”, mis quarterly, vol 24 no 1, pp 141-168. moore, k., grifiths, m., & richardson, h. (2006) “moving in, moving out? a survey of women in ict”, paper presented at the symposium on gender and ict: women for change. nz trade and enterprise (2004) new programme to ensure women get ict opportunities, http://www.nzte.govt.nz/display404.aspx accessed 11 november 2004. newman, j. (1995) “gender and cultural change”, in c. itzin & j. newman (eds.), gender, culture and organizational change: putting theory into practice, pp. 11-29, routledge, london. ogbor, j.o. (2001) “critical theory and the hegemony of corporate culture”, journal of organizational change management, vol 14, no 6, pp 590-608. australasian journal of information systems volume 15 number 1 december 2007 56 pettigrew, a. (1990) conclusion: organizational climate and culture. in b. schneider (ed.), organizational climate and culture (pp. 413-434). jossey-bass, san francisco. patton, m.q. (1990) qualitative evaluation and research methods, sage publications, california. pfleeger, s.l. & mertz, n. (1995) “executive mentoring: what makes it work?” communications of the acm, vol 38 no 1, pp 63-74. ragins, b. r. & cotton, j. l. (1991) “easier said than done: gender differences in perceived barriers to gaining a mentor”, academy of management journal, vol 34 no 4, pp 939-951. ragins, b.r. & cotton, j.l. (1999) “mentor functions and outcomes: a comparison of men and women in formal and informal mentoring relationships”, journal of applied psychology, vol 84 no 4, pp 529-550. roldan, m., soe, l., & yakura, e. k. (2004) “perceptions of chilly it organizational contexts and their effect on the retention and promotion of women in it”, paper presented at the proceedings of the 2004 sigmis conference on computer personnel research careers, culture and ethics in a networked environment, tucson, az. schwarz, r. (august 1, 2005) “harvesting years of experience”, the dominion post, c4. sernak, k.s. & may, j.p. (2003) conversation and voice; feminist mentoring for social and political change, advancing women, http://www.advancingwomen.com/awl/spring2003/ accessed 24 july 2004. sethi, v., barrier, t., & king, r. c. (1999) “an examination of the correlates of burnout in information systems professionals”, information resources management journal, vol 12, no 3, pp 5-13. shultz, m. (1994) on studying organizational cultures: diagnosis and understanding, de gruyter, berlin. spertus, e. (1991) “why are there so few female computer scientists?” mit, available by public ftp at: ftp.ai.mit.edu/pub/ellens/wp-womcs.zip accessed 20 july 2004. sproull, l.s. kiesler, s.b. & zubrow, d. (1987) “encountering an alien culture”, in s.b. kiesler and l.s. sproull, (eds.), computing and change on campus, pp 173-194, cambridge university press, cambridge [cambridgeshire], ny. stapley, l.f. (2006) individuals, groups, and organizations beneath the surface, h. karnac (books) ltd, london. swain, p. (2003) “government responds to ict taskforce” http://www.beehive.govt.nz/viewdocument.aspx?documentid=18249 accessed 15 may 2004. tapia, a.h., & kvasny, l. (2004) “recruitment is never enough: retention of women and minorities in the it workplace”, paper presented at the proceedings of the 2004 sigmis conference on computer personnel research: careers, culture, and ethics in a networked environment, tucson, az. tapia, h. kvasny, l. & trauth, e. (2004) “is there a retention gap for women and minorities? the case for moving in versus moving up”, in m. igbaria and c. shayo (eds.), strategies for managing is/it personnel, idea group publishing, london. thorp d. (2004) hp big on mentoring women, australian it today media monitors, vol 33. australasian journal of information systems volume 15 number 1 december 2007 57 trice, h.m. (1993) occupational subcultures in the workplace,ilr press, ithaca, ny. trochim, w.m. (1999) research methods knowledge base, proquest database on the world wide web, http://trochim.human.cornell.edu/kb.sampnon.htm accessed 13 december 1999. united nations conference on trade and development (2002). e-commerce and development report 2002, united nations, new york. wajcman, j. (1998) managing like a man: women and men in corporate management, polity press, cambridge. wajcman, j. (2004) technofeminism, polity press, cambridge. woodfield r. (2002) “woman and information systems development: not just a pretty (inter)face?” information technology & people, vol 15 no 2, pp 119-138. wulf w.a. (1998) “diversity in engineering”, the bridge, vol 28 no 4, pp 1-7. zweig, p. kaiser, k. beath, c. bullen, c. gallagher, k. goles, t, howland, j. & simon, j. (2006) “the information technology workforce: trends and implications 2005-2008” mis quarterly executive, vol 5 no 2, pp 101-108 australasian journal of information systems volume 15 number 1 december 2007 58 the australasian journal of information systems (ajis) began in 1993 and was based on the simple realisation that there was a need for an academic ‘outlet’ for is research in australia australasian journal of information systems volume 16 number 2 2010 1 exploring the factors affecting employees’ adoption and use of innovation majharul talukder faculty of business and government university of canberra, canberra, act 2601, australia email: majharul.talukder@canberra.edu.au ali quazi faculty of business and government university of canberra, canberra, act 2601, australia email: ali.quazi@canberra.edu.au abstract this paper explores the factors affecting australian organizations‟ employees in their adoption and use of innovative technologies. using a qualitative research strategy, this paper uses a series of in-depth interviews and focus group discussions with academic and administrative employees at a university in australia. the qualitative approach is considered to have a deeper insight into the complexities and dynamic factors that influence the adoption of innovative processes in organizational settings. the findings of this research highlight the need for a broader understanding of the key factors affecting employees‟ decisions to adopt innovation. the results provide important clues for comprehending the factors influencing and determining employees‟ adoption and continued use of innovation in the workplace. with the trend towards end-user applications of technological innovation increasing, the results suggest some guidelines in effectively and efficiently managing the adoption process of innovation in organizational settings. this paper highlights the implications for management practices relating to innovation management in australia‟s tertiary sector. finally, this paper identifies possible limitations and areas of future research in this field. introduction technological innovation leads to improved systems and potentially improves organizational performance. for the purpose of this paper the term „innovation‟ is used to mean a technologically advanced system. we define an „innovation‟ as a system possessing some elements of newness in it which is instrumental in improving work performance leading to enhanced organizational efficiency. performance gains are often obstructed by users‟ unwillingness to accept and use the available systems. thus, the issue of users‟ acceptance of technology has generated considerable interest amongst academics australasian journal of information systems volume 16 number 2 2010 2 and practitioners, prompting numerous studies to be undertaken (davis, 1989). understanding information technology (it) acceptance is considered important because the expected benefits of it usage such as improved efficiency, effectiveness, or productivity, cannot be realized if individual users do not accept these systems in the first place (bhattacherjee & sanford, 2006, p.805). the past three decades have witnessed rising organizational investment in information technologies (venkatesh, morris & ackerman, 2000). such investment is aimed specifically to increase individual productivity and contribute to organizational productivity. while advances in technology continue at an astronomical pace, the use of these emerging information technologies has fallen well below expectations. this represents one plausible explanation for the less than expected productivity gains from investment in it. thus, understanding the factors influencing user acceptance, adoption, and use of emerging information technologies in the workplace is a critical issue for researchers and practioners (venkatesh, morris, & ackerman, 2000, p.34). according to bhattacherjee (1998, p.139) the “shelfware syndrome‟ has been coined to describe software productivity packages sitting idle on book-shelves without being used by the persons for whom they are intended…the issue of motivating the potential users to utilize technology remains a significant problem for organizations”. over the years, more and more organizations have invested heavily in new technologies in order to stay competitive within their respective industries. this increasing investment calls for an examination of factors affecting the adoption of innovation by organizations‟ employees and the relationship of this adoption to performance. organizational innovations that are incorporated into the work processes of an organization have little value if they are not used or complied with by employees. it is argued that employees must adopt innovation to reap its intended benefits, yet it is evident that there are employees who are not accepting technological innovations nor the supposed benefits. this leads to the abandonment of such innovation. therefore, an organization should understand the acceptance process and factors that are crucial in making this process effective (lee, kim, rhee & trimi, 2006, p.470). although innovation adoption has been studied extensively, drivers of adoption and research on innovation acceptance at the individual level remain limited (frambach & schillewaert, 2002). designing an effective approach that increases end-user acceptance and subsequent use of innovation continues to be a fundamental challenge that has not been addressed in the literature. in this sense prior research is limited in terms of individual employees‟ adoption of innovation at the tertiary educational level. the current literature indicates that we know relatively little about the ways in which individuals adopt and the factors that influence individual adoption of innovation (bhattacherjee, 1998; frambach & schillewaert, 2002; van everdingen & wierenga, 2002; venkatesh & davis, 2000). therefore, innovation implementation issues remained unsolved. for this reason further research is required to fill this gap in the literature (frambach & schillewaert, 2002; schepers & wetzels, 2007; yi, jackson, park, & probst, 2006). this paper is, therefore, devoted to identifying the relevant factors that contribute to the development of a work environment conducive to individual adoption of innovation and thereby gaining expected benefits from the innovation. australasian journal of information systems volume 16 number 2 2010 3 objectives and motivation for the research the main objectives of this study are to:  explore the factors influencing individual employees‟ adoption of an innovative technology in their workplaces in the australian tertiary education sector.  identify and discuss the academic and managerial implications of the findings of this research.  identify the limits and future research potential in the field. motivation for the study numerous prior studies examined the possible drivers of innovation adoption using quantitative data (ahuja & thatcher, 2005; al-gahtani & king, 1999; chang & cheung, 2001; cheung, chang, & lai, 2000; igbaria, guimaraes, & davis, 1995; igbaria, zinatelli, cragg, & cavaye, 1997; jasperson, carter, & zmud, 2005; lewis, agarwal, & sambamurthy, 2003; schepers & wetzels, 2007; van der heijden, 2004; venkatesh & davis, 2000). virtually all the above studies examined the issues that quantitatively lacked deeper insights into causes or themes underpinning adoption of innovation. therefore, the motivation for this research principally stems from the lacuna in the extant literature in terms of rich qualitative information relating to factors affecting employees‟ adoption and use of innovation. identification and exploration of factors affecting such adoption is a complex research task as adoption is a dynamic and complex process. therefore, studies using quantitative data often lack the required depth in terms of uncovering information hidden deep in respondents‟ minds. this particular issue was the main rationale for undertaking this qualitatively based research to deepen our understanding of the issues. it does so by highlighting the underlying driving forces shaping workplace usage of an innovation in general and university employees‟ usage behavior of an advanced technology in particular. therefore, the outcome of the study is likely to contribute to broadening our understanding of the acceptance behavior and practice of technological innovation in australian workplaces. literature review and the theoretical framework the literature review presented in this paper includes the general literature on innovation adoption as well as the specific literature relating to the issues associated with the practice of innovation. furthermore, the literature includes both qualitative and quantitative research with special emphasis on the factors affecting organizations‟ employees‟ adoption and practice of innovation. the literature review is presented under the following main headings that will eventually be consistent with the factors included in the theoretical model: individual factors according to lewis, agarwal and sambamurthy (2003), individual factors are one of the most important determinants of adopting innovation. it refers to individuals‟ cognitive interpretations of innovation and themselves. several studies found that individual factors such as perceived usefulness, personal australasian journal of information systems volume 16 number 2 2010 4 innovativeness, prior experience, image and enjoyment of innovation have a stronger influence on an individual‟s attitude towards innovation (al-gahtani & king, 1999; davis, 1989; igbaria, parasuraman, & baroudi, 1996; lewis, agarwal, & sambamurthy, 2003; venkatesh & davis, 2000). it has been noted that “individual characteristics have been reported to play a key role in management information systems success” (lee, kim, rhee, & trimi, 2006, p.472). some authors found that innovativeness tends to affect people‟s perceptions regarding innovation (yi, wu, & tung, 2006). prior experience and familiarity with technological innovation have also been found to reduce anxiety and build confidence in innovative ideas or practices (fuller, vician, & brown, 2006). similarly, yi, jackson, park and probst (2006) noted that a system is believed to be useful provided that system helps increase one‟s social status and image. furthermore, the entertainment potential of an innovation may strongly influence whether to accept that innovation (van der heijden, 2004). organizational factors several studies have indicated that an individual‟s adoption of innovation not only depends on individual attitudes but also on organizational policies, approaches and actions (peansupap & walker, 2005). organizational factors are external to an individual who is considering adopting the innovation. macgregor et al. (2009) found that organizational characteristics are linked to the perceived importance of the drivers of adopting technological innovation. these form the basis of organizational factors where individuals work. organizations need to provide facilitating conditions, which include the extent and type of support provided to individuals that would influence their use of innovation. facilitating conditions are believed to include the availability of training and provision of support. facilitating conditions have been identified as influencing the adoption of a number of new information system innovations (lu, yu, & liu, 2005). these factors include training (al-gahtani & king, 1999; clegg, carey, dean, hornby, & bolden, 1997), managerial support (ahuja & thatcher, 2005; igbaria, parasuraman, & baroudi, 1996) and incentives (bhattacherjee, 1998). these influences have an impact on an individual‟s awareness of the functioning and application of an innovation, its usefulness and fit with the job, which ultimately dictate whether something new can be adopted (frambach & schillewaert, 2002). organizational influences can motivate employees to adopt an innovation. training increases level of confidence in the ability of an individual to learn and practice an innovation because training enhances individual skills and abilities in performing similar jobs (lee, kim, rhee, & trimi, 2006). managerial support is associated with greater level of adoption and acceptance of an innovation, while conversely lack of organizational support is considered to create problems when innovations are being considered (lee, lee, & kwon, 2005). factors such as the provision of discretionary time to individuals working with the innovation or recommendation of bonuses for working with an innovation impact on people‟s decision to adopt it (bhattacherjee, 1998; chen & ching, 2002). social factors employees‟ adoption of innovation is driven by their social environment. innovation used by others in employees‟ social environment is also likely to play an important role. social influence is the extent to which members of a social group influence one another‟s behavior in the context of adoption (konana & balasubramanian, 2005; venkatesh & brown, 2001). it is perceived pressure and influence that peers feel when adopting an innovation and this influence is exerted through messages and signals that help to form perceptions of the value of innovation or activity (fulk & boyd, 1991). according to parker and castleman (2009, p.175), “[a]n actor‟s ties with another actor can vary on a continuum from strong (e.g. friends and/or those with frequent interactions) to weak (e.g. acquaintances)”. ajzen and fishbein (1980) australasian journal of information systems volume 16 number 2 2010 5 refer to such influence as normative beliefs about the appropriateness of adoption of innovation. according to this perspective, employees may adopt an innovation not because of its usefulness but due to perceived social pressure. such pressure may come from individuals whose beliefs and opinions are important, including peers and people who are in social networks (igbaria, parasuraman, & baroudi, 1996). according to abrahamson and rosenkopf (1997), it is a largely internal influence that potential adopters exert on each other that persuades them to adopt. research also revealed that social factors were more significant than economic factors in driving adoption of innovation by individuals in an organization (peansupap & walker, 2005; westphal, gulati, & shortell, 1997). research also found that, moral support and encouragement from peers are instrumental in employees considering the merits of innovation (farr & ford, 1990; lewis, agarwal, & sambamurthy, 2003; yuan et al., 2005). furthermore, a person‟s positive feelings about the usefulness and performance of an innovation can be influenced by favorable references about that innovation (yi, jackson, park, & probst, 2006, p.355). it should be noted that most of the studies included in table 1 below are quantitatively based because not many qualitative studies on this area of research have been published. few studies have used qualitative research methodologies in contexts that differ from the present study (e.g. kartiwi, 2006; drew, 2002). furthermore the literature review points to a number of issues that may affect the adoption and implementation of innovative processes by organizations‟ workforce. a brief summary of the literature review on the above three broad factors is presented in table 1. the table organizes the three broad factors into specific factors within each broad category. factors authors (year) scope and findings organizational factors peansupap & walker (2005); frambach & schillewaert (2002) these two studies focused on the innovation adoption process in organizational settings. both papers specifically looked at innovation implementation issues. these studies developed useful conceptual frameworks in specifying the innovation adoption process by proposing some useful constructs. training needs lee, kim, rhee, & trimi (2006) this paper identified some external factors affecting technology acceptance behavior. this quantitative paper tested a model of innovation acceptance by introducing a number of factors including training and found a direct influence of training on actual usage without any mediating effect. managerial support lee et al. (2005); igbaria, zinatelli, cragg, & cavaye (1997). these papers specifically examine the effect of management support on intention to use a technological innovation and found a significant moderating effect on adoption behavior. incentives for the usages bhattacherjee (1998) this paper identified incentives as a major driving force of it usage with strong empirical data. australasian journal of information systems volume 16 number 2 2010 6 individual factors lewis et al. (2003) lee, kim, rhee, & trimi (2006) the studies investigated the impact of individual and managerial factors on technology use in work place. both studies used quantitative methods and were based on established theory such as tam. usefulness of an innovation kim et al. (2007); algahtani, & king (1999); davies (1989) this paper inter alia examined the impact of ease of use on enjoyment and then the mediating effect of enjoyment on the relationship between ease of use and relative advantage. personal initiative yi et al. (2006); lewis et al (2003) yi et al. (2006) used an integrative approach and investigated if innovativeness in it along with other variables has any effect on behavioral intention of healthcare professionals in relation to using an innovative technology. the results supported the proposition. lewis, et al (2003) found empirical evidence in favor of significant effect of personal innovativeness on it use. prior experience venkatesh & davis (2000) venkatesh & davis (2000) utilized longitudinal data to examine the moderating effect of experience on: firstly, the impact of social factors on usefulness; and secondly, on impact of social factors on usage behavior via intention to use an innovation. results supported these moderating effects of experience on innovation adoption self image moore & benbasat (1991); venkatesh & davis (2000) moore and benbasat (1991) developed a number of constructs to measure users‟ perception of an innovation following a series of relevant statistical procedures. image was one of those constructs. the results of the pilot study revealed image is a significant factor affecting individual adoption of innovation. venkatesh & davis (2000) used image and social influence as a predictor of usefulness of an innovation which affect the intention leading to use an innovation. the results confirmed the hypothesized relationships. enjoying the innovation van der heijden (2004); venkatesh & brown (2001) the author empirically tested a structural model to explore the impact of perceived enjoyment (hedonic aspect) along with perceived usefulness of using an innovation on the intention to use that innovation. the results indicated a strong australasian journal of information systems volume 16 number 2 2010 7 effect of enjoying an innovation on the intention to use. social factors konana and balasubramanian (2005); venkatesh & brown (2001) the authors generally investigated the impact of social factors on adoption of an innovation. peer assistance yuan et al. (2005); van everdingen & wierenga (2002) these papers used a variety of theoretical models to examine the impact of peer influence on usage of technological innovation. yuan et al. (2005) examined team level peer influence on internet usage across five organizations and found support for a „socialized model „of innovation adoption‟. van everdingen and wierenga (2002) looked at intergenerational peer influence on inter-firm adoption decision and found that peer and top management influence are instrumental in explaining adoption decision. social net work yi et al. (2006); venkatesh & brown (2001) yi et al. (2006) tested a complex model of it adoption and found that subjective norms (social network) have a significant impact on behavioral intention to adopt technology. brown & venkatesh (2005) conducted a longitudinal examination of factors including social influence. the results confirmed the influence of family and friends have positive effects on usage behavior of an innovation. table 1: a summary of the literature theory and conceptual model this paper uses a number of theoretical frameworks that are well established in the literature and are considered relevant to the philosophical foundation on which the research is based. these are the theory of reasoned action (ajzen & fishbein, 1980; ajzen, 1991), the technology acceptance model (davis, 1989) and the conceptual framework provided by frambach and schillewaert (2002). this paper also considers the theoretical advancement of technology acceptance model (tam) and tam2 (karahanna, agrawal & angst, 2006; venkatesh & davis, 2000). the theory of reasoned action the theory of reasoned action (tra) is a widely established and applied model concerned with the determinants of conscious intended behavior. tra lays the conceptual framework in terms of the distinction between beliefs, attitudes, intentions and behaviors (al-gahtani & king, 1999). the theory is australasian journal of information systems volume 16 number 2 2010 8 designed to predict and understand the behavior of an individual. according to tra, an individual‟s specific behavior is determined by his or her intention to carry out the behavior. behavioral intention is jointly determined by two basic determinants: firstly, the person‟s attitude toward the specific behavior; and secondly, subjective norms concerning the behavior in question (ajzen & fishbein, 1980, p.6). tra theorizes that a person‟s attitude toward a behavior is determined by his or her salient beliefs about the consequences of performing it and an evaluation of that behavior‟s outcome (davis, bagozzi & warshaw, 1989, p.984). behavioral beliefs are defined as the individual‟s subjective probability that performing the target behavior will have consequences and evaluation refers to an implicit evaluative response to the consequence (ajzen & fishbein, 1980). the technology acceptance model the technology acceptance model (tam) first introduced in 1986 is an adoption of the theory of reasoned action specifically tailored to explain user acceptance of information technology (davis, bagozzi & warshaw, 1989). davis, bagozzi and warshaw, 1989 have postulated that “the goal of tam is to provide an explanation of the determinants of computer acceptance that is general, capable of explaining user behavior across a broad range of end-user computing technologies and user populations, while at the same time being both parsimonious and theoretically justified” (p.985). tam emphasizes that two particular beliefs perceived usefulness and perceived ease of use are of primary relevance for computer acceptance behaviors (bruner & kumar, 2005; davis, 1989). theoretically tam and tam 2 differ in that tam2 includes the social and cognitive instrumental process (vankatesh & davies, 2000). building on this theme, this paper integrates some of these additional variables which include subjective norms such as influence of peers and social networks as well as experiences and images. furthermore, tam2 hypothesizes that beliefs are directly linked to intention and usage behavior (karahanna, agrawal & angst, 2006). individual technology acceptance model frambach and schillewaert (2002) presented a conceptual framework based on the argument that individual acceptance of innovation is based on perceived beliefs concerning the innovation that are being focused on. frambach and schillewaert‟s model indicates that individual usage of innovation relies not only on attitudes but also on management strategies, policies and actions; and these factors include training, social persuasion and organizational support. “organizations will try to influence subordinates‟ attitudes toward adoption of an innovation and some individuals more readily accept certain innovations while others do not” (frambach & schillewaert, 2002, p.171). the model further proposes that personal innovativeness is determined by various personal characteristics such as demographics, tenure, product experience and personal values (frambach & schillewaert, 2002, p.171). according to the model, individual acceptance of innovations is also driven by the use of a focal innovation in their social environment. such social influences may stem from two sources network externalities and peers (frambach & schillewaert, 2002, p.171). in this study we have used the conceptual model presented in figure 1 and tested it with qualitative data. the relationships between the factors and innovation adoption as identified by the literature review (see table 1) have been used as the basis of the proposed association between various factors and innovation adoption in the conceptual model (figure 1) for testing purpose. the qualitative data provided support for the relationship between individual, organizational and social factors on one hand, and individual adoption of innovation on the other. as the model shows, we have decided to explore the linkages between the above mentioned factors and the adoption of innovation at the individual level. this involves australasian journal of information systems volume 16 number 2 2010 9 passing the mediating influence of attitude toward innovation following the recent theoretical advancement of tam (known as the tam2 framework). importantly, attitudes had been excluded from this model (karahanna, agrawal & angst, 2006; venkatesh & davis, 2000). figure 1: a conceptual model of factors affecting innovation adoption methodology the method the study examines the use of selected new features of microsoft outlook. we have chosen these features because they are regarded as an advanced function of calendar. furthermore, most employees are not supposed to use this sophisticated feature of this particular application in their day to day work. some employees may not be aware of the existence of such an advanced feature at all. the categorization of the advanced feature of microsoft outlook is based on the fact that this feature is beyond the core (email) function of microsoft outlook. to evaluate the specific technological advances relating to the calendar application in microsoft outlook, this paper uses a series of qualitative interviews with academic and professional employees at the university of south australia (unisa). individual factors (behavioral beliefs) -usefulness of an innovation -personal initiatives -prior usage experience -unique individual ability -enjoying the innovation social influence (normative beliefs) -peers assistance -social network organizational factors (external factors) -training needs -managerial support -incentives for the usage attitude towards innovation individual adoption of innovation australasian journal of information systems volume 16 number 2 2010 10 the study instrument was validated following six steps involving opinion and feedback of academic experts, it technologists, personnel managers and prospective respondents (university employees). the ten specific questions on various factors affecting workplace adoption of technological innovation were developed following a six-step confirmation process, described below: step 1: developing a preliminary list of questions based on a review of prior studies that either directly used or mentioned these questions or provided clues as to this question in their observation. step 2: a formal list of questions was developed from the preliminary list with meaningful wordings for each question. step 3: these formal questions were asked of experts in the relevant field including academics, personnel managers and it specialists at unisa. furthermore their comments and suggestions were recorded. step 4: a list of final and appropriately worded questions (10) was then developed by incorporating the personnel‟s comments and suggestions. step 5: a pilot study was then conducted involving 15 randomly selected employees from unisa to confirm the validity of this instrument consisting of ten questions. step 6: the pilot tested questions were further modified based on feedback received from the pilot study. after the validity of the questions were confirmed through the above process, the ten refined questions were then finalized for interviewing respondents. rationale for using qualitative method as stated earlier, the rationale for using the qualitative research approach stems from the fact that most prior studies on this topic used quantitative methods. the qualitative method was chosen because it has the capacity to record people‟s deeper insights or perceptions into phenomena (gummesson, 2000) and is suitable for investigating the acceptance of innovation in the australian tertiary education sector. qualitative research strategies have been used in other business and management-related areas (see woodruff & schumann, 1993; masberg & silverman, 1996; hirschman & thompson, 1997; price, arnould & curasi, 2000). furthermore, interviews are integral to the qualitative method and for this reason have been selected so that the researcher can interact directly with respondents with the specific aim of obtaining information from them. the open response format provided an opportunity to obtain a rich amount of information in the respondents‟ own words, thus facilitating a deeper level of understanding. sampling method and data collection approach the population sample for this study consisted of academic and administrative staff at unisa. data were collected using an unstructured questionnaire containing open-ended questions. fifty-three employees were selected following random sampling technique using a random table. the selected employees were then contacted by telephone so that their consent and availability to participate could be confirmed. they were then sent emails with attached an information sheet explaining the background and purpose of the research. a total of forty-one participants confirmed their willingness to participate. hence, these 41 employees constituted the final sample for this study. representativeness of the sample samples were drawn from the whole university population, i.e. employees. therefore, care was taken to ensure that all the four faculties (division of business; education, arts and social sciences; health sciences; and information technology, engineering and environment) and the three administrative units (chancellery, flexible learning center, campus central) are represented in the sample. furthermore, to ensure representativeness of the sample the gender, job classification and academic divisions of the australasian journal of information systems volume 16 number 2 2010 11 employees were considered prior to the invitation to participate. table 2 presents a breakdown of the respondents according to the divisions in which they work. the table shows that there is a balance in terms of numbers and locations from which respondents were chosen for interviews. work location number of respondents percentage of respondents division of business 8 19.6 arts and social sciences 6 14.6 education 6 14.6 health sciences 7 17.1 information technology, engineering and environment 9 21.9 others 5 12.2 total 41 100% table 2: a breakdown of respondents according to their work locations data analysis technique there are several ways in which data can be analyzed (creswell, 1994). for the purpose of this paper we have followed the format suggested by creswell (1994) and developed categories of data and made comparisons and contrasts between the types of information gathered. we also looked at alternative explanations for the findings. in conducting data analysis we have adopted the approach suggested by miles and huberman (1994) in the management of the analysis of the qualitative data. additional care was also taken to ensure that the data analysis was relevant to the theme of this study. the actual data entry included using a digital recorder to record interviews which was then transferred into a computer file. a brief report of each interview was also generated following completion of each interview. transcription of the interviews was recorded to facilitate data analysis. discussion of findings the results are presented in terms of raising, addressing, reporting and analyzing answers to key questions examined. these questions have been formulated based on the extant literature exploring the innovation adoption process generally and the possible issues impacting on such adoption and its usage. the answers are organized and interpreted on the basis of employees‟ perceptions concerning the possible factors that influenced their adoption of new technological systems introduced into their workplaces. the responses and comments on each issue are summarized. the relevant quote(s) from respondents in response to a particular question can be seen in appendix 1. the findings have been compared to those elicited in prior studies. furthermore the findings relating to each question are linked to the adoption of an innovation in the context of a university. the results are presented under three broad factors by further breaking them down into a number of issues as follows: individual factors individual factors represent an individual‟s cognitive interpretations of themselves as well as the technological innovation that an individual is considering adoption. individual factors comprise usefulness of an innovation, personal initiative, prior experience, unique individual ability australasian journal of information systems volume 16 number 2 2010 12 and enjoyment issues. literature provides strong support in favour of the argument that individual factors drive innovation in a significant way (al-gahtani & king, 1999; davis, 1989; igbaria, parasuraman, & baroudi, 1996; lewis, agarwal, & sambamurthy, 2003; venkatesh & davis, 2000; lee, kim, rhee, & trimi, 2006). the specific results are presented under the following specific issues: usefulness of an innovation the responses relating to the above issue were obtained through the question, “how do you find out the usefulness of a new system or technology before adopting it?” the respondents were unanimous on this issue in that they discover the potential benefits of using a new technology before adopting it. many respondents mentioned about checking the usefulness of the new technology in terms of its specific purpose and merits over the previous one before making any decision to use the innovation. respondents also mentioned about experimenting with the new feature of the concerned technology by consulting fellow employees, friends both within and outside the organization who have prior experience with the technology. these comments confirm the findings of kim, chan and gupta (2007) that individuals evaluate the consequences of their behavior in terms of perceived usefulness and base their choice of behavior on the desirability of the usefulness. it has been observed that if an individual believes that using the new system will lead to more efficiency and effectiveness in their work, the innovation is more likely to be considered for adoption (lee, 2004). therefore, it follows from kim, chan and gupta (2007) and lee (2004) that technological innovation needs to be measured to assess its effectiveness before it is accepted in the workplace. the above findings suggest a pilot study of the potential user of the technology to discover the actual usefulness of the innovation. this pilot study is likely to reveal the perceived usefulness of an innovation in terms of some specific benefits. the university‟s management can then use the findings of the study as the basis of a training guideline for prospective adaptors and users of an innovation. personal initiative information as to whether personal initiative can play any role in adopting new technology has been gathered using the question, “how does personal innovativeness (initiative) influence individuals to explore a new system or a technology?” respondents were of the opinion that personal initiative and innovativeness are instrumental in trying and practicing an innovation at the workplace. employees mentioned that since exploring a new technology is a sort of fun for them, they enjoy playing with it. some employees explore new technologies because they have potential personal benefits in terms of improvement of their skills leading to better job prospect. these findings are consistent with frambach and schillewaert‟s (2002) argument that personal innovativeness drives a person‟s tendency to adopt a new technology. furthermore, prior research also found that individuals with higher personal innovativeness are expected to develop a more positive attitude toward the adoption of technology (agarwal & prasad, 1998; lewis, agarwal, & sambamurthy, 2003). this finding is relevant to the introduction of new technologies in australia‟s education sector. in particular the personal initiatives of university employees to try a new technology stems from the intellectually driven research environment of the university, something that enthusiastic employees embrace. in fact, this may become the basis of survival for many employees in terms of securing their australasian journal of information systems volume 16 number 2 2010 13 jobs through positive and innovative attitudes and actions concerning advanced software applications in their workplace. prior experience of employees as a drive this issue has been examined using the question, “how does prior experience with similar technology can help adopt a new system?”. responses to this issue seemed to be polarized. some respondents endorsed the view that prior knowledge and experience with similar technology is very helpful in making a decision regarding the adoption of new technological devices. more specifically the respondents mentioned that previous experience with an older version of the system helps in the transition to an advanced system where familiarity makes the transition easier and smoother. these findings corroborate the finding of fuller, vician and brown (2006) that prior experience and familiarity with technology reduces anxiety and provides users with the confidence to try a new technology. prior experience was also found to have a positive effect on the use of systems (lee, kim, rhee & trimi, 2006). however, some respondents expressed contrary viewpoints on this issue as they believe that prior experience might obstruct the adoption of technological innovation. the argument was that people often liked to compare a system with its previous version and if the new system did not fit to their mindset, they would tend to react negatively to the proposed innovation resulting in resisting the implementation of the innovation at their workplaces. while the above polarized positions of respondents are interesting, the situation is understandable from the point of view of the current trends in the tertiary education sector in australia. computer application software is regularly upgraded at universities to remain competitive at the industry level. therefore tertiary institutions prefer to recruit employees with prior knowledge in computer software applications. resistance to introduction of new technologies in the workplace is also not uncommon in the tertiary sector. therefore, employees with prior experience in regular adoption of computer technologies would have a positive mindset in embracing innovation. consequently experienced employees are less likely to stand on the way of implementation of an innovation at their workplaces. therefore, these findings have implications for recruitment of experienced personnel that would result in cost savings for universities in terms of minimizing or avoiding wasteful resistance as well as in terms of saving training costs. employees’ self image as a driver the above issue has been investigated using the question, how do you feel when you find out that you can do something with technology that your co-workers cannot do?. respondents agreed that an employee having some skills in using a technology is seen by many as a capable and knowledgeable person that provides that person with a certain amount of status and prestige in the organization. it gives a sense of achievement and a feeling of power as well. they feel good in that they have some skills that others don‟t have. however, those employees believe that possessing such skills implies a responsibility on them to teach and show others how to use the system requiring additional workload for them. some respondents believe that they cannot afford any extra time as they could use that time otherwise to increase their productivity at work. respondents also opined that in a typical work environment where there is a high degree of interdependence with other employees in carrying out one‟s duties, improved image and status become important to many individuals. research shows that using a technological system would result in improving an individual‟s image in terms of job performance (venkatesh & davis, 2000) because a system is instrumental in enhancing the social status of an individual (yi, jackson, park, & probst, 2006, p.355). employees with expertise in a particular technology are well recognized as useful resources in a university setting. recruitment of such employees with a technology background would not only promote australasian journal of information systems volume 16 number 2 2010 14 the image of an individual but also that of the university. furthermore, employees with knowledge of a technological system as well as having interpersonal skills can be instrumental in converting a fellow „technophobic‟ into a „techno-lover‟ in a learning environment where technophobia is quite common. in this sense, self image of an individual employee would be instrumental in popularizing a new technology in the workplace. the employers can also develop a workload concession program for those who are actively engaged in mentoring their colleagues. enjoyment with innovation as a driver the above issue has been explored with reference to the question, “what is your opinion of the argument that people enjoying working with computer technology are the early adopters of a technological innovation?” many respondents believe that people, who enjoy working with technology, tend to be the early adopters of technology. people who enjoy working with computers are the first in line to adopt the system. they indicated that playfulness and having fun are important in adopting a new thing. few respondents mentioned that if the technology seems familiar, then they enjoy it and become more confident and are inclined to use it. however, if the system is totally new, they do not feel much enjoyment with the technology rather they may be afraid of it. there are individuals who just like to discover things. they like to explore the technology and there is no pressure from management at all. in a similar vein van der heijden (2004) stressed that entertainment potential is expected to have a strong influence on innovation adoption decisions. it also provides an opportunity to escape reality and become absorbed in a new world. this finding makes sense in that an individual employee‟s strong inclination towards an innovative technology is likely to impact on that person‟s propensity to adopt that technology. at the university level employees who really enjoy working with an advanced technology in many instances would naturally drive the introduction of that technology and would possibly lobby their managers for its introduction. furthermore, employees with prior knowledge would also encourage and assist peers in embracing a technological innovation at the university. social factors innovation used by others in employees‟ social environment is likely to play an important role in adoption of innovation in the workplace. social influence is the extent to which members of a social group influence one another‟s behavior in innovation adoption (konana & balasubramanian, 2005; venkatesh & brown, 2001). it is perceived pressure and influence that peers feel when adopting an innovation and this influence is exerted through messages and signals that help to form perceptions of the value of innovation or activity (fulk & boyd, 1991). ajzen and fishbein (1980) refer to such influence as normative beliefs about the appropriateness of adoption of innovation. according to this perspective, employees may adopt an innovation not because of its usefulness but because of perceived social pressure. such pressure may be perceived as coming from individuals whose beliefs and opinions are important, including peers and people who are in social networks (igbaria, parasuraman, & baroudi, 1996). according to abrahamson and rosenkopf (1997), it is a largely internal influence that potential adopters exert on each other that persuades them to adopt. several studies found that social factors were more significant than economic factors in driving adoption of innovation by individuals within an organization (peansupap & walker, 2005; westphal, gulati, & shortell, 1997). the following two issues are explored under social factor. australasian journal of information systems volume 16 number 2 2010 15 peer influence as a driver whether peers can influence an employee to adopt an innovation has been examined by the question, do you believe that peers can help you understand and use a new system or technology?. results suggest that peer influence is crucial in understanding and using an innovative technology. peers are the first people that employees approach for help and reference as to whether to use an innovation. or not. peers can also show an employee as to how to use and manage certain applications of an innovation. the encouragement from peers is also identified as a motivational issue for possible adoption of an innovation. employees prefer the technological system that their peers use to be able to maintain social bond by interacting with them. good and helpful peers are essential and they always assist others without putting someone down and do not undermine co-workers for their lack of knowledge in the system. a healthy and friendly environment is, therefore, essential in order for people to receive cooperation and assistance from peers. respondents also agreed that active and constructive peer assistance is instrumental in learning the innovation quickly and easily. peer assistance makes the adoption efficient and effective which is considered more useful than guidelines/ handbooks, online material or even participating in training sessions. however, many respondents mentioned that peers should be willing and able to help others. respondents further agreed that help from peers saves time for adoption of an innovation. the above findings are consistent with those of prior research that found that support, motivation and encouragement from peers can greatly influence individual employee to adopt an innovation (lewis, agarwal, & sambamurthy, 2003; yuan et al., 2005). research also found that adoption of an innovation by peers may signal its importance and advantages, and thus eventually motivate the individual employees to adopt. like employees at any other institutions, university staff members are also likely to be highly influenced by peers in developing a positive attitude towards adopting and practicing a new technology. often, employees are under pressure from management to learn a new technology that is considered to be appropriate and may be competitive with other universities‟ systems. in such a situation peer support becomes crucial in adopting and practicing an innovation at the tertiary level of education. in most cases universities formalize the mentoring roles of experienced employees in educating and training fellow employees in learning new technologies and systems. however, seeking peer support in learning a new technology is a very common practice amongst employees in many educational institutions in australia and internationally. social network as a driver the possible impact of social network on adoption has been assessed in terms of the question, what is your opinion about the argument that social networks can influence an employee to adopt a new system or technology?‟ employees stressed the fact that network‟s advice and recommendations influence their decisions to use the new system to keep up with their network. respondents use internet as the possible source of information and communicate online to exchange important information about new technologies and their usage. furthermore virtual communities are also crucial in generating ideas about the technology around the world. respondents indicated that social networks generate updated information about new technologies that boosts employees‟ motivation and enhance their confidence level resulting in active trial of a new technology. some employees opined that while a social network can positively influence the adoption of a new system or technology, this can also have a negative effect on adoption of an innovation. respondents have pointed out that social network‟s influence has its most impact when individuals are convinced of the real benefits of the innovation in terms of improvement of their workplace performance. it is interesting to note that some respondents are often under undue pressure from network to try a new technology that employees would not have normally tried. australasian journal of information systems volume 16 number 2 2010 16 the influence of social network on adoption of innovation is understandable given the very strong networking culture existing in many australian educational institutions. a university has a large population of students and employees who are most likely to be connected via a wide range of social networks internally and externally. since social networks can play a major role in diffusion of computer technologies, the influence of these networks on university employees in adopting new technology is inevitable. as australia‟s education system has become market-oriented and internationalized, social networks have now become internationalized as well. thus the globalized nature of social networks would be even more effective in driving prompt and efficient adoption of innovative technology in the australian education sector. organizational factors the results relating to organizational factors are discussed with reference to three issues, these being training needs, managerial support and incentives for the usage of an innovation. these three issues together capture the role of an organization in fostering innovation in the workplace. the results are presented in terms of responses to relevant questions linked to the above issues. organizational factors in general have been found to be crucial in driving adoption of an innovation as organizations need to provide facilitating conditions for adoption of technology in workplaces. these conditions include active support to individuals that would influence their use of innovation. facilitating conditions are believed to include the availability of training and provision of support. facilitating conditions have been identified as having an effect on infusion or adoption of a number of new information system innovations (lu, yu, & liu, 2005). these influences affect an individual‟s awareness of the functioning and application of an innovation, its usefulness and fit with the job which leads to its adoption (frambach & schillewaert, 2002). the results relating to organizational impetus for adoption are discussed below under three issues. training as a driver “whether training plays any role in the adoption process of an innovation has been assessed through the question, how does training help employees learn and understand a new system or a new technology?” the respondents generally agreed that training is a critical issue and it helps employees to better understand a new system and learn about the new technology. they strongly feel that training is essential and need to be more specific and effective in the organization; especially when an unknown system or new technology is introduced. in the absence of proper training, employees will not be able to use the new system optimally towards achieving its set purposes. respondents further believe that training improves employees‟ efficiency and effectiveness, productivity and knowledge of the new technology and its application. the respondents specifically indicate that training does make them more comfortable and confident when using new technology and helps them to learn different features and how to use them in practice. therefore, training becomes essential because it equips employees with the required knowledge and skills to use the new technology in the most effective and efficient way. however, the respondents differed as to whether individual or group training would be useful in learning a new technology. most respondents mentioned that individual training works much better than group training. in individual training participants can learn and understand quickly. in a group training context participants may be introduced to ideas about the basic functions of the technology but frustration occurs when they try to use it by themselves. some respondents opined that individual training is very effective but group training is also useful when most time is spent with individuals showing them how to do it. for individual needs, however, personal training becomes vital. australasian journal of information systems volume 16 number 2 2010 17 it is to be noted that finding extra time required to learn a new technology is a major source of stress for most employees in australian tertiary institutions. academics, for example, may believe that adoption of a new technology takes away their time which they could use productively in research related activities that are required for their survival. since they are rewarded for their research efforts in terms of promotion, for example, they are most likely to be reluctant to spend extra time trying a new technology. stressful situations may also arise from the uncertainty associated with how much time is needed to become familiar with the new technology. the above finding implies that management should provide extra support in terms of estimating a set period of time and then freeing-up that amount of time so that employees can learn how to use new technology. alternatively, management could set aside some time for all employees for the required training each year so they do not feel additional pressure in accommodating extra responsibilities. furthermore, management can offer reduced teaching and administration to accommodate the extra time needed to learn and manage a new technology. given the importance of time required to learn about and implement an innovation as revealed in this study, we propose that future studies consider the time factor as a separate driver of innovation uptake in tertiary institutions. managerial support as a driver impact of above issue on employees‟ technology adoption behavior has been explored via the question, how does managerial support (providing resources) affect you in accepting a new system or technology?‟ most respondents interviewed believed that managerial support is important in accepting new technology or systems because it will be difficult to learn and use a new system or technology without active management support. this support includes financial and technical support as well as providing time to learn and understand a new technology. managerial support is vital as it provides employees with the opportunity to learn about a new technology and boost their confidence and skills. as indicated by many respondents, time is one of the crucial issues for them when trying a new technology or system by developing an adequate knowledge base in that system. management must provide enough time so that staff can manage their own work and have the time to get acquainted with new technology. some respondents argued that employees should not be forced to adopt a new technology or system because applying force may result in employees merely accepting the technology but they would not explore the technology to its full potential. management needs to provide enough training and other necessary support to employees so that they understand how to use it as well as the benefits of the new technology in their daily job-related activities. lee, lee and kwon (2005) found that managerial support is associated with more adoption and usage while lack of organizational support is deemed to discourage the adoption and effective utilization of a new innovation. senior management support, in particular is considered vital for innovation to get through. research by lee, kim, rhee & trimi, (2006) provides support for this proposition. this finding makes sense for an education institution that is under constant pressure from students and the government to upgrade its existing technologies to remain competitive in the educational markets at the national and international levels. therefore, management needs to provide continuous support to its employees in terms of hands-on training and allocating enough time to attend training. the outcome must be the ability to provide better and quicker service to students and staff. an evidence for this argument can be found in the australian federal government‟s initiative to provide additional computers to schools so students can enhance their computing skills toward improving their academic performance in the long term. australasian journal of information systems volume 16 number 2 2010 18 incentives as a driver the possible role of proper incentives on adoption of new technology at work has been investigated with reference to the question, how can incentives motivate employees to adopt a new technology? many respondents equate time with incentives. they feel that time is another important factor and providing the required time may represent a good incentive for adopting a new technology or system. however, a few respondents remarked that financial incentives may not be effective in attracting employees to try a new system. some employees, however, equated the issue with the retention of their jobs as many employers would like to see their employees update themselves with the latest technologies that are tied to employees‟ productivity at their workplace. furthermore, many respondents believe that if the adoption of new technology benefits the organization then the organization should reward employees providing mentoring leadership in implementing innovations at work. according to bhattacherjee (1998) employees value individual benefits such as career advancement, personal skills, formal and informal recognition, increased autonomy, beliefs about self worth and achievement and greater job security. it therefore, follows from the above findings that management support in the form of incentives is crucial for engaging employees in the adoption and usage of an innovation. incentives are likely to provide extra encouragement to employees in adopting and participating in the continuous use of an innovation in australian universities, which are currently experiencing major restructuring in terms of providing better quality services to students and clients. proper incentives would, therefore be most crucial in getting employees involved in the effective use of new technologies. however, management needs to determine the most realistic incentives that management can afford that employees would also accept. some suggested incentives that employees may prefer include time off, additional points for promotion and certificate of appreciation which would be deemed acceptable in the university environment. however, determining the exact form and extent of incentives that would be acceptable to employees in a university setting, is a subject of further research. comments on overall results interview outcomes presented above demonstrate that many of the specific drivers under the three broad categories of factors have a nexus with employees‟ adoption of technological innovation at their workplaces in tertiary educational institutions in australia. the level of influence of a particular factor depends, however, on the degree of relevance of those factors to the use and practice of an innovation as indicated by the respondents. the influence of social factors comprising two specific drivers such as peer influence and social network is worth mentioning because these two drivers have been rarely explored in the extant literature using qualitative research approaches. prior empirical evidence confirms that external influence is an important factor when technological innovations are being considered for implementation at the organizational level (yi, jackson, park, & probst, 2006). therefore, significant impact of social factors on innovation adoption and implementation is consistent with the conventional wisdom. in terms of adoption of an innovation both peer assistance and social network imply assistance and encouragement from others in internal and external terms. hence, these two factors together contribute to acceptance and practice of an innovation in an educational organization. respondents have provided their strongest support in favour of influence of australasian journal of information systems volume 16 number 2 2010 19 these two factors in their considerations for trying and eventually using the technological innovation. the above findings makes sense as university employees are likely to be motivated to adopt an innovation if they discover that their peers both within and outside their organizations use a similar technology and recommend that colleagues in another institutions should consider using that particular feature of a new technology (brown & venkatesh, 2005). this finding corroborates the findings of prior research that found social networks wield a positive impact (lu, yao & yu, 2005; sykes, venkatesh & gosain, 2009; yi, jackson, park & probst, 2006; brown & venkatesh, 2005) and peer influence (yuan et al., 2005; farr & ford, 1990) on adoption of an innovation effective and efficient application and practice of innovative technology leads to enhancement of organizational efficiency and cost efficiency in the long-term. these principles apply to tertiary institutions as computer technology and especially various application softwares are most commonly used in australian universities. since computer software is an ever-changing type of technology, universities must continuously adopt and apply the advanced software technologies in its various operations. in this vein, the current study has documented some important issues associated with the introduction and use of advanced technology. in particular, the study revealed that training plays an important role in developing employees‟ required knowledge base and interest in using new computer software. training may be financially expensive and time consuming but in practice training is likely to save money in terms of minimizing the time and energy required for self learning exercises. another crucial issue that needs discussion is managerial support which is vital for the successful adoption and practice of software in the tertiary education sector. for example, if management is reluctant to provide necessary training and support regarding new software, the adoption process would be lengthy and this may compromise the efficiency and effectiveness in using new technological systems. therefore, the factors „training‟ and „managerial support‟ do complement each other. similarly, prior positive experience and enjoyment associated with an innovation are closely related factors because an employee is likely to adopt a technological innovation if he or she is already familiar with it. furthermore, an active peer assistance program can also add to saving time in the adoption process, and management support is also instrumental in achieving this objective. these interpretations of the findings further reflect on the support that this qualitative research has provided to the conceptual model. as noted earlier this research revealed some critical issues concerning use and adoption of innovative technologies in tertiary institutions in australia. tertiary institutions constitute a vital sector of the australian economy which is currently growing rapidly in terms of more local and international students (karmel & nguyen, 2003), and greater integration with the global economy. regarding the provision of better services to students and the wider society, the university sector is the focal point for government and policy makers. it is presently experiencing reforms in all spheres of its operation and management particularly in the area of the latest computer technology. the main purpose of this reform is to increase managerial and operational efficiency in terms of employee productivity and quality improvement in the delivery of services. australia‟s federal government is committed to improving ict infrastructure in tertiary and other education settings by spending millions of dollars in recent years (broadband use in the higher education sector, 2007). the aim is to make students more computer literate by providing additional computers to schools. this initiative of the government is consistent with the findings of this research that suggested more strategic importance of adoption and active use of technologically advanced computer systems in australia‟s education sector. another important finding is that active support from senior management in terms of relevant training and materialistic incentives would make the adoption process of technology attractive to employees in australasian journal of information systems volume 16 number 2 2010 20 australia‟s higher education sector. the findings further imply that university management groups need to develop plans for employees so that the new technologies are accepted more easily in tertiary institutions. specific attention needs to be paid to freeing up time for employees so they can familiarize themselves with new technologies in terms of learning how to use them. the real and measurable benefits of using the innovation can be clearly delineated through pilot surveys of users of technological innovation. this is an important issue that needs to be addressed so employees can use an innovation as informed users. furthermore, active support from internal peers and external social networks will enhance employees‟ ability to adopt innovative practices in the workplace. besides this, resistance of university employees to the introduction of new technology adds to the complexity surrounding uptake of an innovation in the workplace. although resistance to change is not a new issue in management literature, resistance of employees to technological change in an educational institution setting is of particular relevance to this paper. employees‟ resistance to new technology or in other words „technophobia‟ may stem from the perceived fear resulting from uncertainty associated with the complexity of use, learning problems, extra workloads and additional time needed to familiarize oneself with new technology. management needs to develop a well thought out strategy on managing resistance to adopting new technology at the organizational level. the strategy may include providing solid management support that directly reduces user resistance to new technology adoption (kim & kankanhalli, 2009) and also tend to ease the use of an innovation (lewis et al., 2003). furthermore, senior management commitment to innovation implementation has been found to be instrumental in increasing the possibility of organizational support for the technological change (kim & kankanhalli, 2009). since favorable opinion of peers about an innovation can contribute to countering resistance to a new technology, massey et al. (2001) suggested that management can first influence opinion leaders in favor of the innovation to enhance acceptance by others. the major findings of the study can be seen at a glance in table 3. australasian journal of information systems volume 16 number 2 2010 21 factors examined brief findings perceived importance relevant studies supporting the findings training needs formal training positively influences individuals‟ ability to adopt a new technology as training equips employees with the knowledge and skills required to use a new technology in the most effective and efficient way. very critical ligon, abdullah, & talukder (2007) managerial support managerial support is vital for employees using a new technology. it includes providing financial and technical assistance as well as allocating the time required to learn and understand a new technology. very crucial lee, lee and kwon (2005) incentives for the usages incentive is a great motivating factor for employees to adopt and practice an innovation. the specific incentives mentioned include promotion, financial benefits and time allocation. moderately important bhattacherjee (1998) usefulness of an innovation the potential benefit of the innovation is an important factor for acceptance and practice of an innovation. employees discover the usefulness of an innovation through individual assessment of the benefits via trial, consultation with peers and colleagues having experience in the same technology, exploration of word of mouth and seeking comments and recommendations from others, use of advanced product quality planning (apqp) devised to ascertain the risk and benefits of the new system, assess if the time and money spent on the technology is worth the values it creates. very important kim, chan and gupta (2007) personal initiative personal interests and initiatives of employees play a pivotal role in accepting and using a new technology. individuals who are innovative in nature and younger generally accept a new system. personal interests in technology have been associated with improvements in job satisfaction, creation of better scope to switch between jobs and making life easier in terms of doing things quicker and better. moderately important frambach and schillewaert (2002) prior experience prior experience of potential employees in a comparable system of technology has been found to increase the likelihood of trying a new technology. prior experience makes it comparatively easier for employees to use a new system as they feel comfortable and confident in doing so. there is also a view that while prior experience is deemed necessary in adopting a new technology, some people love to explore an unknown technology for the thrill of it. fairly important fuller, vician and brown (2006) unique individual ability people able to do their job efficiently and have the skill to share their knowledge with others have been found to be instrumental in making other people try a new technology. these people feel proud in being recognized for their unique skills by others and this in turn motivates them to help those needing assistance and mentoring. important venkatesh and davis (2000). enjoying the innovation enjoyment has been singled out as an important issue in accepting and practicing an innovation. people who love to discover new things tend to first find about a new technology and enjoy the time and excitement in working with it and are most likely to be the innovators of that technology. fairly important van der heijden (2004); yuan et al. (2005) peer assistance while information technologists play a role in assisting employees in learning and adopting an innovation, peers play a different role by going beyond the formal path to assist their peers. peer direction, guidance, and support are provided in a more informal, close and congenial manner. this encourages and motivates employees further to understand and adopt an innovation. extremely important lewis, agarwal, & sambamurthy, 2003; yuan et al. (2005) social net work social networks do greatly influence the adoption of a technological innovation. employees are influenced by both positive and negative recommendations received from their social network as to whether a particular innovation should be accepted or not. a social network enables members to be aware of the systems that are widely used in their designated industry, which indirectly puts pressure on network members to use that technology in order to remain competitive in the market. extremely important brown & venkatesh, (2005); yi, jackson, park, & probst (2006) table 3. major findings (note: the level of importance attached to a particular factor has been assigned based on the qualitative analysis of the information provided by the subjects.) australasian journal of information systems volume 16 number 2 2010 22 concluding remarks, managerial implications and contribution this qualitative study has revealed some interesting findings that have refined, deepened and advanced our knowledge of the driving forces shaping adoption and practice of an advanced technological innovation in an australasian organizational setting. this study is one of the first to pull together all the social, personal and organizational factors and used these factors as drivers of technology adoption and endeavored to present a fuller picture of the forces that drive adoption of technological innovation in an organizational setting in the context of australian higher education sector. the discussion of results in relation to each factor presented and analyzed vis-à-vis prior knowledge in the extant literature has provided a comprehensive view of the findings. the study revealed that social factor comprising peer support and social network are the two emerging issues driving adoption and practice of technological innovation in an organizational setting. these findings are understandable and have particular relevance to the growing virtual community driving diffusion of advanced technology in an information age. furthermore, university employees in australia are an integral part of social network both within and outside their own institutions that assists and influence their decision to use and practice an innovation in their workplaces. these socially driven issues seem to gain increasing recognition in the technology acceptance and diffusion process and are likely to generate considerable attention of academics, practitioners, information technologists and employers. amongst the individual factors the most important ones that frequently came up in the analysis are the prior experience with perceived usefulness of and enjoyment with the innovation. these findings can be linked to the social factors as social network and peers can generate enhanced usefulness of an innovation in the minds of innovation users who perceive the usefulness of an innovation positively. the social factor would likely to exert even stronger influence where employees have prior experience with a similar or an earlier version of an innovation. amongst the organizational factors, management support and incentive are the two most important drivers as revealed in the study. this finding suggests that management has to play an active role in providing necessary technical, administrative and other necessary support including training and dissemination of information to employees which would be instrumental in getting an innovation acceptable and practicable to employees. the findings further suggest that proper incentives can also be influential in motivating employees to try and eventually practice a technological innovation. incentives that have been frequently referred to include allocation of additional points for promotion, workload concession for learning a new system and certificate of appreciation. based on the above discussion, it can be safely said that the conceptual model presented in figure 1 is supported by the findings of this study. overall, some of the findings of this paper lead us a step further in broadening our understanding of the complexities of adopting and using technological innovations in the workplace in australian universities. the newness of the study the findings of this research add new knowledge to the topic. firstly, this research can be considered an advancement over previous ones in that it has provided some new directions in deepening and broadening our understanding of the drivers of innovation in a workplace context. for example, no previous research has used such a comprehensive model as this study did in terms of examining numerous issues under the broad categories of individual, social and organizational factors. more specifically, this study has combined these factors together to ascertain the collective impact of all the possible factors on the usage behavior of innovation in workplace settings in the australian tertiary australasian journal of information systems volume 16 number 2 2010 23 educational context. secondly, prior research mostly used the quantitative approach where there was no considerable depth in the understanding and exploration of the driving forces. thirdly, no prior qualitative research addressed the lacuna that existed in the literature concerning the impact of social factors on innovation adoption in the workplace. this research has produced interesting findings suggesting the growing importance of the socialization aspect innovation adoption in educational sectors. because of this research the specific influences of social factors have now been revealed. this newness is expected to prompt further qualitatively based studies with broader and deeper insights into additional factors in a multiple organizational setting in australasia and beyond. implications the findings of the research have implications for various types of managers and practitioners who are responsible for introducing and advancing new technologies in the education sector. given the trend towards end-users applications, the results of this study will guide management to consider more carefully the effective adoption and usage of technological innovations in a tertiary education organization. the research findings will inform management of the importance of the relevant factors (such as training, and peer support) in driving technology adoption and help reduce new technology implementation-related costs and the ability of tertiary institutions to react faster and act more efficiently in the individual uptake of technological innovation. furthermore the findings will also assist management to identify individual employees‟ problem areas and improve their level of technological expertise. for example, university management groups in australia can emphasize the leading factors impacting on the planned technology adoption process and design appropriate implementation strategies. the focus should be on the proper training of employees for which necessary funding should be made available. management should also create a congenial atmosphere where there is a greater level of cooperation amongst employees and facilitates a more productive use of new technology. some form of incentives can also be introduced so that potential users of innovative technology are motivated to use them. furthermore, incentives for mentors may be introduced to support their fellow employees who are keen to learn about an innovation. theoretical contributions this paper makes a contribution to the following theoretical areas. firstly, this paper has clearly identified the role of social factors in adoption and implementation of an innovation which contributes to refinement of the existing theories in terms of inclusion of social factors as one of the predictors of innovation implementation in an organizational perspective. this variable was either ignored or under-emphasized in prior research. venkatesh and davis‟s (2000) proposed inclusion of social factor into any future modeling is now confirmed, which opens the scope for social factors as an emerging area of research. secondly, this paper made some unique contribution towards theorizing the impact of factors on adoption of technological innovation in the tertiary education sector. thirdly, this paper has revealed clear evidence of combining the three broad factors to capture the fuller picture of the influence of drivers on adoption. this should now be the basis of any comprehensive model development that seeks to deepen our understanding of the complexities of innovation adoption. limitations and further research this paper has a number of limitations which may limit the generalization of the findings. firstly, the factors explored in this paper are based on qualitative information provided by a small number of employees in a single tertiary institution in australia. for future studies‟ findings to be applicable and gain wider acceptability, they should include several tertiary institutions in a sample chosen either from a number of regions of one country or several countries. secondly, the factors do need o be australasian journal of information systems volume 16 number 2 2010 24 empirically verified by quantitative data. it is contended that future research on this topic should develop a mixed methodology approach in which quantitative data and the application of principal component analysis will uncover the underlying determinants of innovation adoption and use. furthermore, future research studies may be able to use the perceived characteristics of innovation model (pcim) as an alternative to the tam framework (see jebeile & reeve, 2007). this paper provides a useful understanding of individuals‟ innovation adoption in the context of a tertiary educational institution. the results reported here would seem to be generalizable to other organizations that have similar characteristics, yet the scope and the context of such generalization of results are somewhat limited. the interpretation and implications of the findings in this research paper depend on the nature of an organization and its innovation ethos. therefore, further research is needed to broaden and confirm the results. finally, this paper noted that employees do resist new technologies in tertiary institutions in australia. therefore, future research should consider „resistance to change‟ as a new factor affecting the adoption of innovative technologies. acknowledgement the authors would like to thank dr. jon heales, editor, ajis, professor john campbell and anonymous reviewers for their valuable comments on this paper. references abrahamson, e., & rosenkopf, l. (1997). social network effects on the extent of innovation diffusion: a computer simulation. organization science, 8(3), 289-309. agarwal, r., & prasad, j. (1998). the antecedents and consequents of user perceptions in information technology adoption. decision support systems, 22(1), 15-29. ahuja, m., & thatcher, j. (2005). moving beyond intentions and toward the theory of trying: effects of work environment and gender on post-adoption information technology use. mis quarterly, 29(3), 427-459. ajzen, i. (1991). the theory of planned behavior. organizational behavior and human decision processes, 50(2), 179-211. ajzen, i., & fishbein, m. (1980). understanding attitudes and predicting social behavior. englewood cliffs, new jersey: prentice-hall. al-gahtani, s., & king, m. (1999). attitudes, satisfaction and usage: factors contributing to each in the acceptance of information technology. behaviour & information technology, 18(4), 277297. bhattacherjee, a. (1998). managerial influences on intra-organizational information technology use: a principal-agent model. decision sciences, 29(1), 139-162. bhattacherjee, a., & sanford, c. (2006). influence processes for information technology acceptance: an elaboration likelihood model. mis quarterly, 30(4), 805-825. broadband use in the higher education sector (2007). http://www.archive.dcita.gov.au, department of communication, information technology and the arts, accessed on 20 july 2010. brown, s., & venkatesh, v. (2005). model of adoption of technology in households: a baseline model test and extension incorporating household life cycle. mis quarterly, 29(3), 399-426. http://www.archive.dcita.gov.au/ australasian journal of information systems volume 16 number 2 2010 25 bruner, g., & kumar, a. (2005). explaining consumer acceptance of handheld internet devices. journal of business research, 58(5), 553-558. chang, m., & cheung, w. (2001). determinants of the intention to use internet/www at work: a confirmatory study. information & management, 39(1), 1-14. chen, j., & ching, r. (2002). a proposed framework for transitioning to an e-business model. quarterly journal of electronic commerce, 3(4), 375-389. cheung, w., chang, m., & lai, v. (2000). prediction of internet and world wide web usage at work: a test of an extended triandis model. decision support systems, 30(1), 83-100. clegg, c., carey, n., dean, g., hornby, p., & bolden, r. (1997). user‟s reactions to information technology: some multivariate models and their implications journal of information technology, 12(1), 15-32. creswell, j. (1994). research design: qualitative and quantitative approaches. thousand oaks, california: sage publications. davis, f. (1989). perceived usefulness, perceived ease of use, and user acceptance of information technology. mis quarterly, 13(3), 319-340. davis, f., bagozzi, r., & warshaw, p. (1989). user acceptance of computer technology: a comparison of two theoretical models. management science, 35(8), 982-1003. drew, s.a.w. (2002), e-business research practice: towards an agenda, electronic journal on business research methods, 1(1), 18-25. farr, j., & ford, c. (1990). individual innovation. in m. west & j. farr (eds.), innovation and creativity at work (pp. 63-80). new york: john wiley and sons. frambach, r., & schillewaert, n. (2002). organizational innovation adoption: a multi-level framework of antecedents and opportunities for future research. journal of business research, 55(2), 163-176. fuller, r., vician, c., & brown, s. (2006). e-learning and individual characteristics: the role of computer anxiety and communication apprehension. journal of computer information systems, 46(4), 103-115. fulk, j., & boyd, b. (1991). emerging theories of communication in organizations. journal of management, 17(2), 407-446. gummesson, e. (2000), qualitative methods in manasgement research, sage publication inc, california, usa. hirschman, e. c. & thompson, c. j. (1997). why media matter: toward a richer understanding of consumers‟ relationship. journal of advertising 26(1), 43.-60. igbaria, m., guimaraes, t., & davis, g. (1995). testing the antecedents of microcomputer usage via a structural equation model. journal of management information systems, 11(4), 87-114. igbaria, m., parasuraman, s., & baroudi, j. (1996). a motivational model of microcomputer usage. journal of management information systems, 13(1), 127-143. igbaria, m., zinatelli, n., cragg, p., & cavaye, a. (1997). personal computing acceptance factors in small firms: a structural equation model. mis quarterly, 21(3), 279-305. australasian journal of information systems volume 16 number 2 2010 26 jasperson, j., carter, p., & zmud, r. (2005). a comprehensive conceptualization of the post adoptive behaviors associated with it-enabled work systems. mis quarterly, 29(3), 525-557. jebelie, s. & reeve, r. (2007). explaining intention to use an information technology innovation : an empirical comparison of the perceived characteristics of innovating and technology acceptance model. australasian journal of information systems, 15(1), 137-151. karahanna, e., agrawal, r., & angst, c. (2006). reconceptualising compatibility beliefs in technology acceptance research. mis quarterly, 30(3), 781-804. kartiwi, m. (2006). case studies of e-commerce adoption in indonesian smes: the evaluation of strategic age. australasian journal of information systems, 14(1), 69-79. kermel, t. and nguyen, n.(2003). australia‟s tertiary education sector, paper presented to the 7 th national conference of the centre of economics of education and training, melbourne, september 1-13. kim, h., chan, h., & gupta, s. (2007). value-based adoption of mobile internet: an empirical investigation. decision support systems, 43(1), 111-126. kim, h. w., & kankanhalli, a. (2009). investigating user resistance to information systems implementation: a status quo bias. mis quarterly, 33(3), 567-582. konana, p., & balasubramanian, s. (2005). the social-economic-psychological (sep) model of technology adoption and usage: an application to online investing. decision support systems, 39(3), 505-524. lee, h., lee, y., & kwon, d. (2005). the intention to use computerized reservation systems: the moderating effects of organizational support and supplier incentive. journal of business research, 58(11), 1552-1561. lee, j. (2004). discriminant analysis of technology adoption behavior: a case of internet technologies in small business. journal of computer information systems, 44(4), 57-66. lee, s., kim, i., rhee, s., & trimi, s. (2006). the role of exogenous factors in technology acceptance: the case of object-oriented technology. information & management, 43(4), 469480. lewis, w., agarwal, r., & sambamurthy, v. (2003). sources of influence on beliefs about information technology use: an empirical study of knowledge workers. mis quarterly, 27(4), 657-678. ligon, j., abdullah, a. and talukder, m. (2007). the effect of formal education, technical and management training on information systems (is) managers‟ managerial effectiveness as perceived by their subordinates. performance improvement quarterly, 20(1), 23-37. lu, j., yu, c., & liu, c. (2005). facilitating conditions, wireless trust and adoption intention. journal of computer information systems, 46(1), 17-24. massey, a. p., montoya-weiss, m. m., and brown, s. a. (2001). reaping the benefits of innovative it: the long and winding road, ieee transactions on engineering management, 48(3), 348357. australasian journal of information systems volume 16 number 2 2010 27 masberg, b. a. & silverman, l. h. (1996). visitor experiences at heritage sites: a phenomenological approach. journal of travel research,, 34(4), 20-25. mcgregor, r.c., hyland, p.n. and harvie, c (2009), do organisational characteristics explain the differences between drivers of ici adoption in rural and urban general practices australia? australasian journal of information systems, 16(1), 77-97. miles, m., & huberman, a. (1994). qualitative data analysis. thousand oaks, california: sage publications. moore, g. & benbasat, i. (1991). development of an instrument to measure the perceptions of adopting an information technology innovation. information systems research, 2(3), 192-222. parker, c.m., castleman, t. (2009). small firm ebusiness adoption: a critical analysis of theory. journal of enterprise information management, 22(1/2), 167-182. peansupap, v. & walker, d. (2005). factors enabling information and communication technology diffusion and actual implementation in construction organizations. itcon, 10, 193-218. price, l., amould, e. & curasi, c. (2000). older consumers„ disposition of special possessions. journal of consumer research, 27, 179-201. schepers, j., & wetzels, m. (2007). a meta-analysis of the technology acceptance model: investigating subjective norm and moderation effects. information & management, 44(1), 90103. sykes, venkatesh & gosain, [initials?] (2009). model of acceptance with peer support: a social network perspective to understand employees' system use. mis quarterly, 33(2), 371-393. van der heijden, h. (2004). user acceptance of hedonic information systems. mis quarterly, 28(4), 695-704. van everdingen, y., & wierenga, b. (2002). intra-firm adoption decisions: role of inter-firm and intra-firm variables. european management journal, 20(6), 649-663. venkatesh, v., & brown, s. (2001). a longitudinal investigation of personal computers in homes: adoption antecedents and emerging challenges. mis quarterly, 25(1), 71-102. venkatesh, v., & davis, f. (2000). a theoretical extension of the technology acceptance model: four longitudinal field studies. management science, 46(2), 186-204. venkatesh, v., morris, m., & ackerman, p. (2000). a longitudinal field investigation of gender differences in individual technology adoption decision-making processes. organizational behavior and human decision processes, 83(1), 33-60. westphal, j., gulati, r. & shortell, s. (1997). customization or conformity? an institutional and network perspective on the content and consequences of tqm adoption. administrative science quarterly, 42 (2), 366-396 woodruff, r. b., schumann, s. f. g. (1993). understanding value and satisfaction from the customer‟s point of view. survey of business, 29(1), 33-40. australasian journal of information systems volume 16 number 2 2010 28 yi, m., jackson, j., park, j., & probst, j. (2006). understanding information technology acceptance by individual professionals: toward an integrative view. information & management, 43(3), 350-363. yi, y., wu, z., & tung, l. (2006). how individual differences influence technology usage behaviour? toward an integrated framework. journal of computer information systems, 46(2), 52-63. yuan, y., fulk, j., shumate, m., monge, p., bryant, j., & matsaganis, m. (2005). individual participation in organizational information commons. human communication research, 31(2), 212-240. appendix 1: qualitative comments of respondents on the factors affecting individual adoption of innovation factors comments of respondents training needs “training demystifies the fear that people have with new technology. no one in this office embraced the new software we bought here recently (although fairly easy to us) because there was no training about it.” “when i have any new software i try it myself but it makes it very difficult for me and consumes a lot of my time and i am not comfortable but when i am imparted training to do all those things it becomes easier for me to use it. so, i believe that training is extremely important.” “in my previous office the senior-most designer could not use a computer. then we gave him training and then he became one of the best designers using computers.” “recently i had training for a new system. initially, i was scared to use it without training .now i am more confident. the training was really helpful.” “one of the main worries is time. to find enough time to go for training is quite difficult. the main problem is finding the extra time for training.” managerial support “if you have strong managerial support and resources and you have the required training, then your level of acceptance will be greater and you will be ready to take on changes.” “managerial support makes you much more confident as the support ensures training, allocation of time and other key resources to try a new technology.” “with our previous software it used to take up to three days to process a job and now with our recent technology it takes about one and half hours to complete the same job. i looked for the software and management bought it. now we need to go for some training. managerial support was essential to buy the software in the first place.” australasian journal of information systems volume 16 number 2 2010 29 “it is time that i need. time must be built into my workload to enable me to learn a new system or technology and i feel frustrated because i do not have enough time to do so. i am too busy with my teaching or projects. so, providing time would be crucial for the adoption of new technology.” incentives for the usages “i think such incentives are a very important factor. if management wants employees to use new technology then they need to support it and incentives can be a very useful way of gaining motivation from employees.” “if i am promised new advantages, promotion or financial incentives then it would motivate me to accept new technology.” “incentives do not have to be money. time would be a good incentive. if you are given a new technology and you do not have time then you will be reluctant to use it. when management provides you with the required time then you will have the motivation to engage in new technology and explore it.” usefulness of an innovation “before using any technology i always check the purpose and the advantages of that technology and find out what i can do with it.” “for any new technology that i am considering using, i spend about a week finding out an adequate amount of information about it and then i have an initial trial to check the technology and then i decide whether to use it or not.” “i use the technology in the testing process and find out its benefits upon using it. then i form an opinion about its usefulness and make a decision as to whether i would accept it or not.” “i use apqp which is an advanced product quality planning. this is the best practice used by automotive weapon manufacturers. i use this analysis to predict the entire thing going wrong, how likely it is that it could go wrong, what controls are, what the risk is and what the benefits are. from these analyses i can make plans and decisions.” personal initiative “individuals who are innovative in nature are usually keen to adopt new technology. i think you have to be really innovative to explore new systems or technology.” “almost everything i do is based on my own interest. individual interest is a big factor for me. i find that i am comfortable with new technology and it is a lot of fun for me.” “age has a huge impact. some people just do not try new things. the younger people aged below 30 who grew up with computers and technologies are most excited if you show them something new. they would be quickly interested in engaging themselves in new things. it is a lot to do with age.” “people will have an interest in technology which relates to their work and helps with their job. i do not like endnotes software but i know it would be good for my research. i have an interest in engaging with technology that will facilitate my work.” prior experience “experience with similar technology makes a reasonable impact on how to begin with something new. to try out a new technology, it is not necessary to have prior experience but it would be better if you had experience.” “absolutely, experience with similar technology shows you the way ahead. you will know the style of the system, working processes, and similarities which make a world of difference compared to the person who does not know. it makes a big difference.” australasian journal of information systems volume 16 number 2 2010 30 “i feel that it is very important. if someone has prior experience with similar technology and similar use then they will adopt it quite easily. they will feel comfortable with the product as they are familiar with the system.” unique individual ability “it makes me feel quite special when someone has a problem and my peers ask the person to see me for the solution of that problem. i really feel very proud of myself.” “i feel proud of myself as a human being. you know you have the knowledge of mastering something.” “i like to have that knowledge. if i know something that others do not know, i feel that i have achieved something special. it gives me definite pride.” enjoying the innovation “people who enjoy the technology are most likely to be the early adopters. most of the people i know working with computer technology do enjoy it and all of them become early adopters. enjoyment plays a major role in the early adoption process of an innovation.” “that is very true. people who enjoy it, they live around the technology, and they are the first to find out about it and use it. it is almost like a hobby.” “there are people who get excited to do something with computers and enjoy doing so. they enjoy the time using it and they love to discover new things.” peer assistance “it does not matter how much you know or what you know, peers can help you to a great extent. peers can show you very important applications which even it experts did not show you. i have received a great deal of help from peers.” “in a friendly environment, peers are quite important. they normally help each other to understand new things. i have a lot of experience in this aspect because when i first came here i found new technologies which i had not used before. i got help from peers and they were very helpful.” “the help from peers is crucial and very important and encouragement from peers is also very critical. if i did not have peers to support me it would be very difficult to manage. i not only received direct help but also received encouragement from them. they provided me with direction and guidance.” social net work “social networks have a huge influence on me. i found out a variety of new technologies from the people i worked for before and from the people i know. often i hear about new technology from outside my work place. i adopt the technology to remain competitive as other competitors (universities) are using those technologies.” “i always follow social networks. i consult people and take into account what they suggest to me about new technology.” “in an era of emerging technology, you need to have a network of people who are using similar technology globally because it is easier to communicate.” ajis vol. 7, no 2 may 2000 the integrated decision model in emergency dispatch management and its implications for design b.l. william wong department of information science, university of otago dunedin, new zealand. email: william.wong@stonebow.otago.ac.nz http://divcom.otago.ac.nz/infosci/msrl abstract this paper reports on a review of a number of well-known naturalistic decision making. ndm, models (zsambok & klein, 1997). key features of these models were identified and were found to represent different views of the same naturalistic decision making process. the key features of these different views were then integrated into a single model called the integrated decision model. to validate the model, a study of emergency dispatch co-ordinators was conducted to determine the 'goodness of fit' of the model in a naturalistic domain. a critical decision method-based study (klein et al., 1989) was conducted and found enough evidence in the emergency dispatch management process to support the proposal of the integrated decision model. key words naturalistic decision making, cognitive engineering, emergency dispatch management, integrated decision model. introduction one of the primary concerns in cognitive engineering is the understanding of how operators make decisions about their tasks in various dynamic human-machine work domains such as nuclear power plants, and manufacturing (rasmussen et al., 1994; vicente, 1992). such domains have generally been classified as causal domains (rasmussen, 1986), i.e. domains in which the outcomes are predictable by the laws of nature. naturalistic decision making (ndm) research has focused on how operators make decisions in dynamic human activity systems such as military and emergency command and control (drillings & serfaty, 1997; kaempf et al., 1996; wong, 1999). such domains have generally been classified as intentional domains (rasmussen, 1986), i.e. domains where the outcomes are not predictable by the laws of nature, but are influenced by the motivations of individuals in the process. despite differences in their areas of focus, both fields of research have studied decision making in a number of actual situations and have explored how this might affect the manner in which information about the work domain should be portrayed (klein, 1993; vicente et al., 1995; wong et al., 1998). this has lead to the development of a number of decision models which differ in certain key characteristics depending upon the perspective of the various researchers. these models thus represent different views of decision making in such dynamic work domains. this paper will summarise the key features of a number of such decision models and attempt to integrate them into a single model that incorporate all the key features of decision making in dynamic decision making domains. this model has been referred to as the integrated decision model, or idm (wong, 1999). based on a grounded theory approach, a study was undertaken to identify the decision goals and strategies of emergency ambulance dispatchers at the st john's ambulance southern regional communications centre in dunedin, new zealand, and at the ambulance co-ordination centre of the ambulance service of new south wales in sydney (wong et al., 1996; wong et al., 1995). the strategies were evaluated in the light of the key features of the ndm models and enough evidence was found to support the proposition that the integrated decision model can usefully bring together and collectively demonstrate the different perspectives in a single decision model. the usefulness of the idm to interface design is that it reinforces the need for displays to support situation awareness and situation assessment through pattern matching and recognising, rather than by analytical strategies that require option generation and simultaneous evaluation of multiple options to select an optimal solution. the next section will summarise the key features of a number of decision models and then describe the integrated decision model. the following section will briefly report on the key findings from a study that elicited evidence from dispatchers that provide support for the proposed idm. then the final section will discuss some implications that the idm may have for the design interfaces to dynamic processes. key features of naturalistic decision models ndm environments have been found to exhibit the following characteristics (orasanu & connolly, 1993): a. ill-structured problems where the causes are not clear or obvious. b. uncertain, dynamic environments that are constantly changing. c. shirting, ill-defined, or competing goals due to the changing situation and competing needs. d. action/feedback loops where outcomes of an earlier decision have an impact on downstream decisions. 95 ajis vol. 7, no 2 may 2000 e. time stress. f. high stakes, e.g. possible loss of life. g. multiple players that need to co-ordinate actions and decisions, h. organisational goals and norms that constrain actions. several decision models have been developed to describe the nature of decision making in these ndm environments. a number of these decision models were compared to identify their key features: the decision ladder (rasmussen, 1986) the recognition-primed decision model (klein, 1997) the simplified decision model (orasanu & fischer, 1997) the situation awareness model (endsley, 1995) and the information processing model of decision making (wickens, 1992). these key features are summarised below. situation assessment an important part of decision making a situation assessment is a conceptualisation or mental model of what is happening and is likely to happen in an environment. decision makers are said to be situationally aware if they have a mental model that is an up-to-date representation of the situation. the models reviewed described how decision makers in dynamic environments place a great deal of effort on understanding the situation and what is likely to occur next, rather than on generating and evaluating options. incomplete or inaccurate situation assessments will lead to actions that may fail to address key aspects of the situation. however, once the problem is understood, courses of action usually become obvious. feature matching and story building key to situation assessment situation assessment is said to develop through either a feature matching or a story building strategy. the feature matching strategy is based on the pattern-recognition of cues in the current situation. when the situation is recognised to be sufficiently similar to previous experiences, appropriate actions that satisfy the condition-action rules are selected and implemented. however, when the patterns of activity in the situation are not sufficiently similar to past experiences, and therefore fail to activate a relevant schema, the story building strategy is invoked to construct a causal explanation of the current circumstances. story-building techniques are also used by decision makers to cope with incomplete and uncertain data. by drawing on previous experiences, knowledge and expertise (schemata), decision makers make inferences about the data by filling in gaps created by the incomplete data. the resulting understanding of the situation leads to the selection of an appropriate set of actions. situations are not presented in an optimal manner the cues of a situation are often presented through different sources and over a period of time. how the cues are pieced together will influence how the situation is understood, the conclusions that can be drawn, and hence the choices that are made. human biases, e.g. anchoring, are to be expected. designs need to anticipate and avoid the likelihood of such biases. option generation: multiple vs. single decision makers in dynamic environments do not have the time to analytically generate multiple options. they do not ask the question, "what are the likely ways to solve the problem?" instead, they seek to identify the actions that best match the pattern of activities recognised in the situation assessment, one option at a time. serial simulation, not concurrent analytical evaluation in contrast with analytical decision theory, the ndm models suggests that decision makers serially evaluate courses of actions by pattern-recognition rather than by comparing the strengths and weaknesses of multiple options in parallel. in terms of the very short time horizon (seconds and minutes), decision makers simply do not have the time to generate multiple options and then to analytically evaluate them. if the decision maker has some extra seconds available, he or she may choose to re-evaluate the situation to see if there are late changes in the situation, or to fine-tune the course of action. courses of action are directly implemented because they satisfy the required condition-action rules. if there is any doubt that the course of action will satisfy the situation, it can be mentally rehearsed to determine if it will work. modifications are made to the plan and the actions are implemented. if, however time is not available, the closest match is implemented and the situation monitored and the actions adjusted as the situation develops. this kind of behaviour is very apparent in fast-paced environments such as aviation and emergency dispatch management. 96 ajis vol. 7, no 2 may 2000 the integrated decision model based on the above review, the literature suggests that people in real-world, time pressured environments employ decision making strategies that are different from what have been prescribed by traditional rationality-based decision models, e.g. model of managerial decision making (simon, 1977). they do not generate multiple options which are then concurrently evaluated on the strengths and weaknesses in order to select an action. instead, ndm researchers have observed that these people first make a situation assessment by recognising critical cues in the situation, and then using a set of rules, select a course of action by matching the pattern of cues to the first most appropriate action, and then if time permits, serially evaluate and modify the course of action before implementing it. these key features have been brought together in the integrated decision model (figure 1). the idm highlights two major processes: a situation assessment process, and a choice process. in assessing what is going on in a situation, the decision maker is presented with various information cues about the situation. not all cues are attended to due to various factors such as their existing schemata, biases, salience of the cues, and time constraints. the decision maker's schemata of operations feature recognition and pattern matching process is applied to the cues that are perceived. the literature suggests that a story building strategy is invoked to help explain the situation, especially when there is insufficient or incomplete data from which to assess the situation. decision makers in this manner build up a mental model of the situation. they will use this mental model in subsequent mental rehearsals to verify the appropriateness of action strategies. the selection of these actions is discussed next. vhat is the pro "/me available? vhat are the ris cues c1 c2 c3 1 i 1 llem'f recognise key features. match or develop mental model of «? situation time is available (seconds & minutes) time is not available situation is familiar implement aclion (rbb) or serially simulate / modify (kbb) implement action (rbb) '"' ^ selective attention pattern jealure n 'storybuik to explaii situation is not familiar devise new / modify closest coa (kbb) and serially simulate / modify (kbb) implement best guess / closest match (rbb) biases matching schemata latching i situation i r"= *apply * h8pe ' appropriate rules to select actions io n a ss es sm en t i co 1 •. o f a ct io n i 1 rtoire j £ o 6 figure 1: the integrated decision model showing a distinct situation assessment phase and choice phase having assessed the situation and updated their mental models of the situation, the decision maker progresses on to the action selection stage of the idm, 'choosing course of action'. the literature suggests two factors determine how actions are selected: (i) how much time is available, and (ii) how familiar is the situation. these two factors are integrated into a two dimensional action space in the idm. the four segments of this action space represent: a. situations that are familiar and when time (in terms of seconds and minutes) is available. if the situation is one which the decision maker has had experienced with, he or she will apply a rule-based behaviour (rbb) strategy and implement an action that satisfies a given rule set. if however the situation is familiar but the same as previously experienced ones, and hence there is only a partial match between the cues of the situation and the rule set, the decision maker will then embark on a knowledge-based behaviour (kbb) mental simulation strategy to serially test and modify existing actions for which the situational cues and rules do not fully match (see sanderson & harwood, 1988) for a good discussion of skill-, rule-, and knowledge-based behaviours.) b. situations that are familiar but time is not available. the decision maker will apply the action most closely reflect actions that have worked in the past with the view of correcting and compensating once the outcomes of those actions are known. c. situations that are not familiar and time is available. in such situations, the decision maker will have the time to devise new courses of action and to then modify them. such an action development strategy is 97 ajis vol. 7, no 2 may 2000 typical of knowledge-based behaviour patterns. a serial evaluation of other actions for possible fit in the current situation is also undertaken, d. situations that are not familiar and time is also not available. the selected action is usually to implement the best guess based on the closest match between the situational cues and the rule set for similar types of situations. evidence was then sought to verify whether such a model could describe decision making behaviour in emergency dispatch management. the next section presents some findings from such a study. evidence for supporting the integrated decision model a retrospective protocol analysis technique called the critical decision method (klein et al., 1989; wong et al., 1997) was used to elicit decision strategies of emergency ambulance dispatchers at ambulance control centres in dunedin, new zealand, and sydney, australia. the protocol analysis revealed evidence that supported the idm. key findings from this study are summarised below. transcripts that support the key findings have not, however, been included here due to space constraints, but have been reported in detail elsewhere (wong, 1999). intense and time constrained environment one situation that is representative of the intense and time constrained nature of work in the rcc was selected for further investigation. this part of the study compared audio tapes of operations with computer print-outs of events that occurred during that period. the workload in the first nine minutes of the situation was so intense that many of the events that occurred were not documented in the computer until later. during that short span of 9 minutes, a total of 10 emergency calls were received by two duty officers. the calls related to three major road traffic accidents and one attempted suicide. during this initial nine minutes, the dispatchers had to answer the calls, diagnose the problems, type in the details of the call, distinguish between multiple calls about the same incident, plan the response, and then co-ordinate with the six emergency ambulances. ambiguous, ill-defined and unstructured problems analysis of the transcripts show that the problems faced by the dispatchers are largely ill-defined and unstructured. callers are frequently ambiguous in describing the problem and even the location. the cause of ambiguity can be attributed to caller incoherence because "... he was so hyped up." (#4/226), or it can be due to information arriving through secondary sources at different times. the situation is often further complicated by unclear symptoms reported by the caller. dispatchers have found it necessary to control the caller's conversation in order to reduce this ambiguity. failing to clearly diagnose the problem or to determine where it occurs, leads to two major difficulties: dispatchers cannot devise an appropriate plan, and they cannot dispatch any ambulances. failing to correctly ascertain the location will lead to major errors in dispatch, delaying the delivery of medical assistance, and wasting a resource. the actual dispatch task is often straightforward, send an ambulance to a particular address. recognition-primed situation assessment through their training and experience, dispatchers build up a schemata of problem situations. these schemata provides the basis for recognising the type and severity of situations (lipshitz & ben shaul, 1997). it is with these schemata that dispatchers compare presented cues of a new situation to diagnose what is wrong. familiarity with or a recognition of the situation allows a quick translation of the severity assessment into the number of ambulances or type of resources needed. this is usually a straightforward calculation based on the carrying capacity of each type of ambulance. identifying which ambulances to send is the problem. this is discussed next. looking for the nearest available ambulances once it is clear what the requirement for ambulances are and where the ambulances should go, the dispatcher next need to identify and send the nearest available ambulances. several strategies for identifying the nearest available ambulances have been identified through the cdm interviews. these strategies are (a) ask the caller, (b) areas of responsibilities, (c) diversion of non-critical job ambulances, and (d) focus and compare. while the first three strategies are somewhat self-explanatory, the focus and compare strategy requires some explanation. in deciding which ambulances to send, the dispatcher mentally combines two types of information: location of the incident, and location of the stations to determine the nearest ambulances. having determined which stations are closest to the incident, the dispatcher focuses and eliminates from consideration all other possible stations that are further away from the accident. he focuses attention only on the ambulance stations that are relevant to the situation. the dispatcher then compares between the candidate stations and dispatches the most appropriate ambulances based on availability and crew qualification. 98 ajis vol. 7, no 2 sequentially develop and evaluate courses of action. may 2000 dispatchers do not appear to generate multiple options which are then evaluated and from which the most suitable option is selected. based on the information presented to him, the dispatcher generates one single option, i.e. send the closest ambulance. if necessary, this strategy is modified to work within the circumstances of the situation. if the first course of action was unworkable, based on her situation awareness and where the incident is, the dispatcher mentally rehearses and modifies the plan in her head. balancing ambulance coverage. whenever ambulance coverage is depleted in one region due to a high level of activity or due to a major incident, dispatchers still need to provide ambulance cover for those communities. they do this by sharing the coverage between depleted regions and regions that still have ambulances. coverage between those regions that have ambulances and those that do not is balanced by re-deploying ambulances in locations that will allow them to respond to emergencies in either region, although with a degraded response time. this is described as the 'snowball effect'. they move the closest available ambulance nearer to the depleted zones, and then move the next nearest available ambulances closer again to cover the gaps created, and so on. situation awareness another important aspect of the decision process that is not usually identified through a traditional systems analysis (e.g. data flow diagram) is that of situation awareness. endsley has described situation awareness as the state of knowing what is going on in one's area of interest (endsley, 1995). although not reported throughout the interviews it was observed that dispatchers found it necessary to build and maintain a good mental picture of where each ambulance generally is and what each is doing. the status screen provides a quick reference to confirm their mental picture of what is going on in the region. the decision process in terms of the idm the various aspects of the decision process identified through the cdm interview may be viewed in the context of the integrated decision model. this is presented in figure 2. the caller provides the problem cues from which the dispatcher assesses in terms of his schemata of problem situations. he assesses the seriousness of the incident. understanding what the problem is and where it is, leads the dispatcher to know what ambulances need to be sent and where it is to be sent. this is the assessment of the situation. if the situation is familiar, the dispatcher simply responds by selecting an action based on a given set of rules, e.g. look for the nearest available ambulance and send it if it is not familiar or that the requirement cannot be satisfied easily, alternative courses of action to balance the region or to organise a multi-ambulance response, are generated and evaluated sequentially. the characteristics of dispatch management as described above compares favourably with the proposed integrated decision model. situation (caller & ambulance status) cues selective attention number injured? pattern matching to diagnose what is wrong. ^ what is going on (is the situation familiar?) 1 v, m 01 v) m •* *s itu u io n 1 schemata assessment /m ot the situati requirement = what ambulances are needed, where to send them actions are selected on: rules new actions devised based on i*9 « fc^ -r-» g 1 — find the nearest ambulance $ organise muni-ambulance response balance the region sequential generation and evaluation action repertoire figure 2 emergency ambulance dispatch management in terms of the idm 99 ajis vol. 7, no 2 may 2000 implications for information portrayal design support situation assessment since situation assessment plays such an important role in decision making, displays should portray information about the situation, and where possible show relationships that help explain the situation. accessibility of information task-critical information should be easily accessed, at the fingertips or displayed on system. the dispatcher should not have to spend more time than is necessary, or be engaged in protracted procedures to retrieve information required by the decision. for example, in the current system, the dispatcher has to work through five different computer menu changes in order to access the screen that gives directions on how an ambulance might travel to a particular address. focus and compare strategy dispatchers were observed to practise the focus and compare strategy in a number of situations to locate or confirm the nearest available ambulances. these situations include planning for multi-vehicle responses, balancing the regions, and for maintaining situation awareness. in the focus and compare strategy the dispatcher focuses on a subset of related stations to compare the availability of ambulances for an incident that has occurred in or around the vicinity of the stations. the proximity-compatibility principle (wickens & carswell, 1995) predicts that if information is used together in the making of a decision (task proximity), then especially in time-constrained environments, it will benefit the decision maker to have task relevant information displayed together (display proximity). the information used to determine which ambulances to dispatch to an incident are, according to the pcp, non-integrative in nature and are functionally and temporally similar. this means that the information serves a common goal, is used together or combined at the same time for the same task, but is not integrated into a common variable. this represents a high degree of task proximity. therefore, in order that the display is compatible with the focus and compare strategy, it is necessary to portray these pieces of information in a manner that exhibits high display proximity, i.e. close together and in a manner compatible with the mental strategy of the task. conclusion this paper has proposed the integrated decision model of naturalistic decision making in an attempt to streamline what appears as parallel description of the same phenomena. a study was done to determine if the proposed idm can be used to explain behaviour in dynamic decision making environments, in particular, emergency ambulance dispatch management. while there appears to be adequate evidence to suggest that the idm can be used to explain decision making behaviour in ndm environments, further work is being planned to verify the model and its usefulness for design. references drillings, michael, & serfaty, daniel. (1997). naturalistic decision making in command and control. in g. k. caroline e. zsambok (ed.), naturalistic decision making (71-80). mahwah, nj: lawrence erlbaum associates, publishers. endsley, mica r. (1995). toward a theory of situation awareness in dynamic systems. human factors, 37(1), 3264. kaempf, george l., klein, gary, thordsen, marvin, & wolf, steve. (1996). decision making in complex naval command and control environments. human factor (special issue),. klein, gary. (1993). naturalistic decision making: implications for design (soar 93-1): crew systems ergonomics information analysis centre, wright-patterson afb, usaf, oh. klein, gary. (1997). the recognition-primed decision (rpd) model: looking back, looking forward. in g. k. caroline e. zsambok (ed.), naturalistic decision making (285-303). mahwah, nj: lawrence erlbaum associates, publishers. klein, gary a., calderwood, roberta, & macgregor, donald. (1989). critical decision method for eliciting knowledge. ieee transactions on systems, man and cybernetics, 79(3), 462-472. lipshitz, raanan, & ben shaul, orit. (1997). schemata and mental models in recognition-primed decision making. in g. k. caroline e. zsambok (ed.), naturalistic decision making (293-303). mahwah, nj: lawrence erlbaum associates, publishers. orasanu, judith, & connolly, terry. (1993). the re-invention of decision making. in g. a. klein, j. orasanu, r. calderwood, & c. e. zsambok (eds.), decision making in action: models and methods (3-20). norwood, nj: ablex publishing corp. 100 ajis vol. 7, no 2 may 2000 orasanu, judith, & fischer, ute. (1997). finding decisions in natural environments: the view from the cockpit. in g. k. caroline e. zsambok (ed.), naturalistic decision making (343-357). mahwah, nj: lawrence erlbaum associates, publishers. rasmussen, jens. (1986). information processing and human-machine interaction: an approach to cognitive engineering. (vol. 12). new york: elsevier science publilshing co., inc. rasmussen, j., pejtersen, a.m., & goodstein, l.p. (1994). cognitive systems engineering. new york: wiley. simon, h.a. (1977). the new science of management decision. englewood cliffs, nj: prentice hall. vicente, kim j. (1992). multilevel interfaces for power plant control rooms i: an integrative review. nuclear safety, 33(3), 381-387. vicente, kim j., christoffersen, klaus, & pereklita, alex. (1995). supporting operator problem solving through ecological interface design. ieee transactions on systems, man, and cybernetics, 25(4), 529-545. wickens, christopher d. (1992). engineering psychology and human performance. (2nd ed.). new york, ny: harper collins publisher. wickens, christopher d., & carswell, c. melody. (1995). the proximity compatibility principle: its psychological foundation and relevance to display design. human factors, 37(3), 473-479. wong, b.l. william. (1999). information portrayal in dynamic decision making environments. unpublished phd thesis, department of information science, university of otago, dunedin, nz. wong, william b.l., o'hare, david, & sallis, philip j. (1996). a goal-oriented approach for designing decision support displays in dynamic environments. in j. grundy & m. apperley (eds.), proceedings of ozchi '96, the sixth australian computer human interaction conference, november 24-27, 1996, hamilton, new zealand, 78-85. wong, william b.l., o'hare, david, & sallis, philip j. (1998). the effect of display location on decision making in dispatch management. in h. johnson, l. nigay, & c. roast (eds.), people and computers xiii, hci '98 conference (221-238). sheffield, uk: springer, in collaboration with the british computer society. wong, william b.l. , sallis, philip j. , & o'hare, david. (1995). information portrayal for decision support in dynamic intentional process environments. in h. hasan & c. nicastri (eds.), ozchi '95, the fifth australian conference on computer human interaction (43-48). university of wollongong, wollongong, australia: computer human interaction special interest group of the ergonomics society of australia. wong, william b.l., sallis, philip j., & o'hare, david. (1997). eliciting information portrayal requirements: experiences with the critical decision method. in h. thimbleby, b. o'conaill, & p. thomas (eds.), people and computers xii, hci '97 conference of the british computer society special interest group on humancomputer interaction (397-415). university of west england, bristol, uk: springer. zsambok, caroline e., & klein, gary (eds.). (1997). naturalistic decision making. mahwah, nj: lawrence erlbaum associates, publishers. 101 ajis vol. 7 no. 1 sept 1999 confusion with diffusion? unravelling is diffusion and innovation literature with a focus on smes for correspondence: jeanette k van akkeren lecturer in information systems university of the sunshine coast locked bag no. 4 maroochydored.c. qld 4558 dr. angele l m cavaye senior lecturer in information systems university of southern queensland toowoomba qld 4530 abstract the aim of this paper is to contribute to understanding of the various models used in research for the adoption and diffusion of information technology in small and medium-sized enterprises (smes). starting with rogers' diffusion theory and behavioural models, technology adoption models used in is research are discussed. empirical research has shown that the reasons why firms choose to adopt or not adopt technology is dependent on a number of factors. these factors can be categorised as owner/manager characteristics, firm characteristics, and other characteristics. the existing models explaining is diffusion and adoption by smes overlap and complement each other. this paper reviews the existing literature and proposes a comprehensive model which includes the whole array of variables from earlier models. introduction there is a large body of literature about the adoption of innovations in general; there is also a significant amount written about the adoption of technology and information technology (it) in particular. within the area of literature discussing diffusion of technology and adoption of it, there are papers focusing on adoption by organisations and articles focusing on adoption by individuals. these two types of adoption both play a role when investigating the diffusion and adoption of it by small and medium-sized enterprises (smes). after all, smes are small organisations where many of the decisions are made by the owner/manager. the organisational decision to adopt technology becomes intertwined with the personal perceptions and attitudes of the owner/manager towards that technology. this paper reviews the existing literature concerning the adoption of innovation and adoption of it by discussing the different approaches that have been taken and the various models that have emerged. focusing on adoption of it by smes, many variables and explanations of factors said to inhibit or facilitate the adoption of it can be identified. the purpose of the paper is to build a comprehensive framework of the factors associated with the adoption of ft by smes. such a framework can be used as a summary of existing literature and can form the basis of subsequent empirical enquiry. before embarking on the discussion of adoption models, it is useful to define the main concepts relating to this area of literature: technology, technological change, adoption, innovation, invention, and diffusion. technology can be defined as the use of pieces of knowledge, both practical and theoretical, in a productive manner producing given goods and services (bannock et al. 1984). it includes the use of know-how, methods, procedures, experience of success and failures, and also physical devices and equipment technological change is viewed as an innovative process where firms build upon their existing knowledge base and on other assets when they search for new opportunities (alange et al. 1998). lundvall (1992) adds that technological change is a learning process, which is gradual and cumulative in nature. adoption can be defined as the taking up of an idea through the approval or acceptance of that idea. bannock et al (1984) describes innovation as the introduction of new products or production processes in the last stage of invention (invention being the discovery or devising of new products and processes). coombs et al (1987) suggests that the term diffusion relates to the level of adoption of an innovation as a ratio of adopters to non-adopters in an industry. this paper reviews rogers' original diffusion of innovation theory highlighting the characteristics identified as most pertinent to technology adoption and diffusion. following this, models using theories of behaviour applied by other researchers are analysed. then, research highlighting adoption factors is discussed. finally, a research model is presented; the model incorporates the factors identified in existing literature as impacting on technology adoption by smes. 60 ajis vol. 7 no. 1 sept 1999 adoption and diffusion models this section outlines rogers' diffusion of innovations theory including the identification of the characteristics of innovation and the logistic diffusion curve. following this, models based on theories of behaviour are presented and briefly reviewed. rogers' diffusion of innovations theory rogers posited diffusion of innovation (doi) theory in 1962 and he and other researchers have since completed work resulting in the modification of his original theory (rogers 1962, 1971, 1983, 1995). rogers (1995) suggests that characteristics of innovations help persuade potential adopters to embrace or reject an innovation. the doi model has its primary focus on how potential adopters perceive a technology in terms of relative advantage/disadvantage of an innovation (rogers 1995). the five characteristics of an innovation identified by rogers (1995, pp.15-16) are: • relative advantage the degree to which an innovation is perceived as better than the idea it supersedes; • compatibility the degree to which an innovation is perceived as being consistent with the existing values, past experiences, and the needs of potential adopters; • complexity the degree to which an innovation is perceived as difficult to understand and use; • trialability the degree to which an innovation may be experimented with on a limited basis; • observability the degree to which the results of an innovation are visible to others. the chance of a firm adopting an innovation and the extent of that adoption is largely dependent on the characteristics of the innovation as perceived by the adopting firm (premkumar et al. 1994). the diffusion of innovation over time follows an s-curve shape. coombs et al. (1987) suggest that the risk associated with the innovation decreases as more firms adopt and explains the move upwards in the s-curve. this diffusion process can be mathematically represented as d[x(t)]/dt, where t is time and d[xt] is the rate of diffusion. adoption can then be graphically represented in the form of a logistic diffusion curve as shown in figure 1. in the logistics curve the rate of diffusion first increases up to an inflection point, and then decreases (coombs etal. 1987). different innovations will have different rates of adoption which means that the slope of the s-curve can be flat or have a sharp incline. different firms may be interested in different types of technology. firms tend to improve and to diversify their technology by searching in zones that enable them to use and build on their existing technological base (dosi 1988). there are very few studies in information systems (is) that use rogers' doi theory directly. models focusing on the adoption of it are based on rogers original work and also take account of other original work outside the area of is. it is this other work that is discussed next. /in cummulative number of firms, or consumers, that have adopted. / / years figure 1: logistic diffusion curve behavioural theory and adoption models another area of literature that has influenced is adoption researchers consists of publications highlighting theories of behaviour. the main constructs in these theories are individual behavioural characteristics like perception and attitude. 61 ajis vol. 7 no. 1 sept 1999 ajzen (1991) discusses the theory of planned behaviour (tpb) which extends his original work on the theory of reasoned action (tra). in the theory of planned behaviour model, behaviour is a direct function of behavioural intention and perceived behavioural control. he further suggests that behavioural intention is formed by one's attitude, subjective norm (that is, social influences) and perceived behavioural control. ajzen (1991) acknowledges limitations with the theory of planned behaviour model, in particular, that the relationship between the belief structures and the determinants of intention are not particularly well understood causing some inconsistency in results. taylor and todd (1995) suggest an alternative version to the theory of planned behaviour model called the decomposed theory of planned behaviour. this model attempts to overcome limitations of the ajzen (1991) version particularly in relation to attitudinal, normative and control beliefs that are decomposed into multidimensional belief constructs (taylor and todd 1995). problems with operationalising the earlier model are overcome as relationships become clearer and more readily understood, pointing to specific factors that may influence technology adoption and usage. these factors include the perceived characteristics of an innovation, relative advantage, and compatibility of the innovation to the adopter's existing values, previous experiences and current needs (taylor and todd 1995). davis (1989) puts forward the technology acceptance model (tam) which addresses it adoption, implementation and diffusion in terms of perceived ease of use and perceived usefulness. the causal model hypothesises that actual systems use is affected by behavioural intentions that are themselves affected by attitudes toward use. gefen and straub (1997) suggest that belief about the system, perceived use and perceived ease of use, directly affects attitudes toward use. one limitation of the tam is that the model excludes the influence and personal control factors on behaviour. in addition, economic factors or outside influences (suppliers, customers, competitors) are not specifically addressed in this model. models focusing on it adoption by smes there is a relatively small but significant body of literature that focuses on the adoption and use of it by smes. small and medium-sized enterprises are significantly different from large organisations and sme characteristics affect their ability and willingness to adopt it. large organisations generally have the funding, resources and expertise to pursue technology innovation and adoption, or if not, can outsource the needed expertise. in comparison, small businesses have to rely on a small number of employees, and are further constrained in terms of time and capital availability. the problem of resource poverty has been recognised for some time, but recent research shows that this remains a major difficulty when it comes to smes adopting it. for instance, delone (1988) shows that inadequate financial resources, inadequate managerial time, and lack of internal expertise discouraged it growth in smes. similarly, lawrence (1997), in an exploratory study of smes in tasmania, found that many smes were not readily incorporating new technologies into their business. the level of sophistication of a firm in terms of it usage and knowledge appears to impact heavily on whether technology will or will not be adopted. recent research indicates that reluctance to adopt innovations can be attributed to factors such as resource poverty, lack of ft knowledge, and a perception of a lack of business benefit (mien and raymond 1994, kirby and turner 1993, yellow pages australia 1997). a number of it adoption models based on empirical data bout smes have been put forward (mien and raymond 1994, brooksbank et al 1992, kirby and turner 1993, lacovou et al. 1995, thong and yap 1995, harrison et al. 1997, gefen and straub 1997). following is a review of the recent literature, lacovau et al (1995) based their model on roger's work, particularly in relation to relative advantage and trialability, with perceived benefits and impact of technology as adoption factors. in addition, they identified the following factors as affecting technology adoption: • organisational readiness (financial and technological resources of the firm), and • external pressure to adopt (from competitors and trading partners). behavioural and attitudinal factors, and firm characteristics in terms of size, sector, status and complexity, are not clearly addressed in this study as determinants of technology adoption in small business. a model developed by kirby and turner (1993) suggests the following factors as impacting technology adoption: • the computer literacy of the small business owner, • lack of knowledge of benefits derived from it, and • dependency of the small customer on the supplier. however, constructs measuring technology adoption and acceptance are limited given the models previously developed and tested in the it literature. using similar constructs to kirby and turner (1993) is the model developed by mien and raymond (1994). this model includes another three factors thought to influence new technology adoption in small business: • structural sophistication of the firm in terms of centralisation and complexity; 62 ajis vol. 7 no. 1 sept 1999 • the level of assertiveness, rationality and interaction in business decision processes; • size, sector, and status of the firm (independent or affiliated). again this model is limited in terms of the number of constructs previously shown in it literature to influence technology adoption, with many factors not included in the research design, for example, perception, attitude, and economic factors. another model developed by thong and yap (1995) uses two main categories of factors influencing it adoption in smes. they are: • individual characteristics (ceo innovativeness, attitude towards adoption of it, and it knowledge); • organisational characteristics (business size, competitiveness of environment, information intensity). however, perceived control of it adoption (an important factor for the owner/manager), and structural sophistication of the firm (its ability to incorporate new technologies into its work practices) are ignored in this study. finally, harrison et al (1997) use an extension of the theory of planned behaviour model, similar in context to factors used by the thong and yap (1995) study, to explain and predict a small business executive's decision to adopt information technology. factors in their model include both owner/manager characteristics and firm characteristics: • decision processes based on attitude; • subjective norm; • perceived control regarding it adoption; • firm size, sector and status; and • individual characteristics. a limitation of many studies is that they do not include return on investment. financial issues are vitally important to owner/managers and often drive technology adoption in small firms. an innovative sme owner may recognise all the benefits to his firm in adopting it in terms of both a shortor long-term investment. however, without sufficient funding the owner cannot adopt. return on investment is addressed by economics of technology (eot) theory which is based on the returns on investing in an innovation for a firm. fichman and kemerer (1993) discuss the effects of it investment in terms of four determining characteristics: prior technology drag, irreversibility of investments, sponsorship, and expectations. however, eot theory does not capture firm characteristics (external pressures to adopt, the sector and status of the organisation) or the doi perceptions of the owner/manager. firm characteristics are generally of an economic nature, and not addressed in the doi model or eot theory. since empirical studies have determined these factors as influencing technology adoption in smes, it would be useful to include them in a general it adoption model. factors affecting it adoption in smes many different factors have been identified in the literature as affecting it adoption by smes. the factors identified in the studies reviewed above can be separated into two main categories: owner/manager characteristics and firm characteristics, with the addition of return on investment as an influencing factor. the owner/manager category includes views and perceptions of the owner/manager about the use and usefulness of it for the sme; firm characteristics include factors relating to the size of the firm, the position of the sme in its industry, and other characteristics of individual organisations. table 1 (following page) identifies individual factors for each category and their source. 63 ajis vol. 7 no. 1 sept 1999 category ^ owner/manager characteristics firm characteristics other factors i. individual factor ' • perceived benefits (ease of use, usefulness) • computer literacy of business owner • level of assertiveness, rationality and interaction of business decision processes • perceived control regarding it adoption • subjective norm • organisational readiness • external pressure to adopt • dependency of the small business customer on supplier(s) • structural sophistication of the firm in terms of centralisation and complexity • sector • status • information intensity • return on investment v,; . ',,'1 v * source •-: '. '• lacovau et al. (1995), kirby and turner (1993), thong and yap (1995) kirby and turner (1993), thong and yap (1995) julien and raymond (1994), harrison et al. (1997) harrison et al. (1997) harrison et al. (1997) lacovou et al. (1995) thong and yap (1995), lacovau et al (1995) kirby and turner (1993) julien and raymond (1994), harrison et al. (1997) julien and raymond (1994), harrison et al. (1997) thong and yap (1995), julien and raymond (1994), harrison et al. (1997) thong and yap (1995) fichman and kemerer (1993) table 1: summary of factors influencing it adoption in smes the effects of each characteristic on sme it adoption or non-adoption are discussed below; sources will not be repeated as they can be found in table 1. owner/manager characteristics • perceived benefits affect technology adoption in terms of the perceived ease of use and/or usefulness of the technology. if an owner/manager does not perceive the technology in a positive way, he/she will be reluctant to adopt. • the computer literacy of the business owner can also influence technology adoption. if the owners are unaware or do not understand the technologies available, they are unlikely to adopt them into their own business. • the level of assertiveness, rationality and interaction of business decision processes can also impact on it adoption. if owners of the firm are assertive in business decision processes, understands the benefits and uses of the technology to their organisation, and are able to rationalise that information, they will be more likely to adopt it. • perceived control relates to the amount of requisite opportunities and resources (time, money, skills, co-operation of others) someone possesses to be able to carry out the course of action (technology adoption). for example, a small business owner may decide that connection to the internet is an important competitive use of it. yet if there is a possible budget shortfall, their decision to adopt will be influenced. • subjective norm is thought to affect technology adoption in terms of the strength of the person's normative beliefs that 'groups' think the behaviour of interest (technology adoption) should or should not be performed, multiplied by a person's motivation to comply with the group. 64 ajis vol. 7 no. 1 sept 1999 firm characteristics • organisational readiness refers to the level of technology currently incorporated into business processes. if there is little technology incorporated, or outdated/inefficient technology being utilised, a firm is less likely to adopt new technologies. • a sme will be reluctant to adopt innovative it unless there is a specific request for it by their trading partners and/or customers. if this external pressure to adopt it is not present in the industry sector, then the business owner may perceive the technology as a waste of resources. • the dependency of the small business customer on the supplier is linked to the previous factor. not only would the supplier need to have adopted the technology to make it viable, the sme owner would need to recognise and understand the benefits to his or her firm in adopting the technology. in addition, an organisation may perceive that their clientele was of a certain socio-economic level that would not readily benefit from the introduction of new technologies. • the structural sophistication of the firm in terms of centralisation and complexity will also influence technology adoption in its ability to incorporate new technologies into its work practices. a particularly complex structure could either inhibit or facilitate technology adoption and would be dependent on whether the owner believed that it could be easily incorporated and enhance operations, or excessively disrupt operations. • the sector that an organisation belongs to affects it adoption some industry groups embracing technological innovation and others reluctant to adopt. if competitors and trading partners within the sector have adopted it, an owner may be more inclined to adopt as well. the yellow pages australia (1997) findings indicated that the business services sector, manufacturing sector, and transport/storage sectors were the most innovative in terms of technology adoption. in contrast, building/construction and personal service sectors ranked low in their adoption of ft with adoption inhibitors identified as including a lack of perceived benefits, a fear that technologies would waste staff time, lack of it knowledge/skills, and the cost of the technology. • another factor that has been shown to impact on the rate of sme technology adoption is the status of the organisation: whether the organisation is franchised or otherwise connected to partner organisations, or completely independent. treadgold (1990) found that small retail businesses with high it adoption rates had been influenced by trade associations, wholesalers, and franchisors that encouraged ft adoption. • finally, the level of information intensity within the organisation may influence the owner to adopt or not adopt a technology. for example, if large amounts of data and information are part of the business processes, an owner may be more likely to adopt technologies that could streamline operations and lead to process efficiencies within the organisation. other factors • the need by sme owners for an immediate return on investment is due to the necessity to be concerned with medium-term survival rather than the long-term attainment of market share. to make a substantial outlay of capital resources, the owner would need to see exactly where the return was going to be in the short-term. a research model for investigating it adoption by smes doi theory, the technology acceptance model, theory of planned behaviour, and decomposed theory of planned behaviour, generally support factors in the 'owner/manager characteristic' category. the focus of these early adoption models is perceptions, attitudes and beliefs. more recent literature adds enhancements to these earlier models including additional constructs thought to impact on technology adoption and diffusion such as economic factors and firm characteristics (e.g. thong and yap 1995, julien and raymond 1994, harrison et al 1997). the various it adoption models identify many factors affecting it adoption in smes. the models overlap and complement one another. determining constructs that are most pertinent to technology adoption by the sme owner/manager requires the integration of a combination of factors identified in prior studies. a comprehensive model of factors affecting the adoption of it by sme owner/managers is presented in figure 2. 65 ajis vol. 7 no. 1 sept 1999 owner/manager characteristics perceived benefits computer literacy assertiveness perceived control subjective norm teturn on investment firm characteristics organisational readiness external pressure to adopt customer/supplier dependency structural sophistication of the firm sector status it adoption in sme figure 2: factors affecting it adoption in smes this model attempts to capture and summarise all factors identified from the literature as affecting technology adoption by smes. in addition to these there may be other factors not previously identified that affect technology adoption in smes. further research may uncover additional constructs not yet listed in the model in figure 2. conclusion this paper has reviewed and summarised those factors thought to affect adoption of information technology in smes. since rogers posited his diffusion of innovation theory more than 25 years ago, there have been a number of models used to identify characteristics of technology adoption. some of these models are based closely on rogers work; others have used constructs identified after empirical research. none of the previous models attempt to capture all factors shown in the literature to impact on whether a firm will adopt or will not adopt ft. figure 2 presents a research model comprising all the factors identified in existing literature that have been shown to affect it adoption by smes. the model is intended as a summary of current findings and as a framework that may be used by future researchers. there are opportunities for further research into factors that facilitate or inhibit an sme owner/managers decision on whether to adopt it. one avenue for further research is the identification of other adoption factors. the proposed model may not be complete and there may be additional factors potentially affecting it adoption. further research could also use the model as a basis for investigating the adoption of new technologies by smes. the individual factors in figure 2 could then act as a guide when investigating inhibitors to the adoption of a particular technology by smes. for instance, internet technologies have the potential to 'level the playing field' for smes, but recent surveys show that australian smes have been slow to adopt them (yellow pages australia 1997, forrester research 1997). studies could investigate the reasons for slow adoption and look at ways to overcome inhibitors to adoption. 66 ajis vol. 7 no. 1 sept 1999 references ajzen, i. 1991, "the theory of planned behaviour", organisational behaviour and human decision processes, pp. 179-211 in taylor, s. and todd, p.a. 1995, 'understanding information technology usage: a test of competing models', information systems research, june, vol. 6, no. 2, pp.144 -176 alange, s., jacobsson, s., and jarnehammer, a. 1998, "some aspects of an analytical framework for studying the diffusion of organisational innovations", technology, analysis and strategy, march, vol.10, no.l, pp. 3-21 bannock, g., baxter, r.e., and rees, r. 1984, the penguin dictionary of economics, 3rd edn, penguin books, england brooksbank, r., kirby, d., and kane, s. 1992, "it adoption and the independent retail business: the retail newsagency", international small business journal, apr-jun, vol. 10, no. 3, pp. 53-61 coombs, r., saviotti, p., and walsh, v. 1987, economics and technological change, macmillan education limited, london davis, f., 1989, "perceived usefulness, perceived ease of use and user acceptance of information technology", mis quarterly, september, vol. 13, no. 3, pp. 319-340. delone, w.h. 1988, "determinants of success for computer usage in small business", mis quarterly, march, vol. 12, no. 1, pp. 51-61 dosi, g. 1988, "sources, procedures and microeconomic effects of innovation", journal of economic literature, vol. 26, pp. 1120-1171 fichman, r. g. and kemerer, c.f. 1993, "adoption of software engineering process innovations: the case of object-orientation", sloan management review, winter, pp. 7-22 forrester research incorporated, forrester ranks world economics for ecommerce, 30 april 1997, cambridge, ma, [online] <http://www.forrester.com.>. accessed august 1998. gefen, d., and straub, d.w. 1997, "gender differences in the perception and use of e-mail: an extension to the technology acceptance model", mis quarterly, december, vol.21, no. 4, pp. 389-400. harrison, d.a., mykytyn, p.p. jr, and rienenschneider, c.k. 1997, "executive decisions about it adoption in small business: theory and empirical tests", information systems research, a journal of the institute of management sciences, june, vol. 8, no. 2, pp. 171-195 lacovou, c.l., benbasat, i., and dexter, a.a. 1995, "electronic data interchange and small organisations: adoption and impact of technology", mis quarterly, december, vol. 19, no. 4, pp. 465-485 julien, p.a., and raymond, l. 1994, "factors of new technology adoption in the retail sector", entrepreneurship: theory and practice, summer, vol. 18, no. 5, pp. 79-90 lawrence, k.l. 1997, "factors inhibiting the utilisation of electronic commerce facilities in tasmanian smallto medium-sized enterprises", 8* australasian conference on information systems, october lundvall, b.a. 1992, national systems of innovation. towards a theory of innovation and interactive learning, francis pinter, london. kirby, d., and turner, m. 1993, "it and the small retail business", international journal of retail and distribution management, vol. 21, no. 7, pp. 20-27 premkumar, g., ramamurthy, k., and nilakanta, s. 1994, "implementation of electronic data interchange: an innovation diffusion perspective", journal of management information systems, fall, vol. 11, no. 2, pp. 157-186 rogers, e.m. 1995, diffusion of innovations, 4th edn, the free press, new york. taylor, s. and todd, p.a. 1995, "understanding information technology usage: a test of competing models", information systems research, june, vol. 6, no. 2, pp.144 -176 thong, j. and yap, c.s. 1995, "ceo characteristics, organisational characteristics, and information technology adoption in small business", omega, august, vol. 23, no. 4, pp. 429-442 treadgold, a., 1990, "the role of distribution channel agents in promoting innovative activity in small retail businesses: a case study of it adoption", service industries journal, vol. 10, no. 4, pp.651-663 yellow pages australia 1997, small business index: a special report on technology in the small business sector, august. yin, r.k. 1994, case study research: design and methods, 2nd edn, sage publications inc, thousand oaks, ca, united states of america 67 ajis special edition-electronic commerce november 1998 electronic commerce: a battle on regulation, standards, transportation media and business integration kim viborg andersen, associate professor niels bj0m-andersen, professor niels christian juul, associate professor center for electronic commerce (cec) department of informatics, copenhagen business school howitzvej 60 dk 2000 frederiksberg (denmark) phone: 445-3815-2400 fax: +45-3815-2401 e-mail kva.inf, nba.inf, ncjuul.inf//@cbs.dk url http://www.inf.cbs.dk please forward all correspondence regarding this submission to associate professor, p.o. kim viborg andersen at e-mail address andersen@cbs.dk abstract the drive towards the electronic commerce organisation is both a troublesome and a strifted path for government, enterprises and consumers worldwide. in this article we analyse the evolution of electronic commerce in denmark during the period 1995-1998 from four perspectives: regulation, standards, transportation media and business integration. the danish government is stimulating the use of electronic commerce using direct and indirect policy initiatives on central, governmental regulation and establishment of greens enabling self-regulation. also, the governments own, organisational management is in denmark seen as a mean to exalt electronic commerce. onwards, the fist on proprietory standards and the un/edifact is an ongoing source of delaying, stimulating or preventing electronic commerce depending on the business sector and the size of market actor addressed. in denmark, the direct access and value added network supporters have so far been the dominating mean to transport the electronic documents in the business transaction. the internet and xml technology is at the turn of the century challenging the way of doing business within this field. some view this as the big blue to speed the diffusion of electronic commerce; others are worried that the incentive to investment and pull the partners in the value chain to use edi might be lost. finally, our study conclude that the debate on business integration issues is ambiguous and characterised by substantial uncertainty on for example the role of intermediates, direct sale, hyper-shift in business partners and the pull/push of global enterprises at the local markets. introduction major software houses like oracle and ibm are setting up separate electronic commerce units. the major is conference as the pacific asia conference on information systems (pacis), european conference on information systems (ecis), american conference on information systems (acis) and the icis all are running tracks on electronic commerce. in addition, a variety of specially designated conferences (e.g., bled international conference on electronic commerce), workshop and job fairs are popping up on electronic commerce. it is not all just talk and conferences. approximately 50 per cent of danish enterprises with more than five employees have access to the internet. almost 80 per cent of the enterprises communicate electronically via call connections or leased lines. within the public sector, about 50 percent of the counties used the internet to access information about products and services: 42% of the counties used the internet to perform payment. in the municipalities, only about 5% used it for payment purposes (danish ministry of research and information technology, 1998). looking at the emerging bulk of electronic commerce literature there is major room for clarification on what is analysed: we need to address the how and why questions surrounding the ongoing development of electronic commerce path. in this article we point to the need for awareness of the business-to-business (b2b) electronic commerce in particular on standards, the transportation media and the business integration issues, we are not yet at the stage where our research models enable us to move onwards. relying on an analysis of the b2b electronic commerce in denmark, this article provides insight in the nature of each of these areas and suggestions for important research to be taken in the coming years entering the new millennium. first of all, we outline our research model and the collection of data. next, we address the ways the government in denmark is stimulating the use of electronic commerce using direct and indirect policy initiatives on central, governmental regulation and establishment of greens enabling self-regulation. also, the governments own, organisational management is in denmark seen as a mean to exalt electronic commerce. we point to similarities both with respect to means and motives to the ones operating the business associations. we then provide b2b examples which use electronic commerce in denmark. an examination is then provided on proprietary standards and the un/edifact is an ongoing source of delaying, stimulating or preventing electronic commerce depending on the business sector and the size of market actor addressed. also, we look at the transportation media. in denmark, the direct access and value added network supporters have so far been the dominating mean to transport the electronic documents in the business 130 aj1s special edition electronic commerce november 1998 transaction. the internet and xml technology is at the turn of the century challenging the way of doing business within this field. some view this as the big blue to speed the diffusion of electronic commerce; others are worried that the incentive to investment and pull the partners in the value chain to use edi might be lost. finally, we look at the contention on business integration issues is ambiguous and characterised by substantial uncertainty on for example the role of intermediates, direct sale, hyper-shift in business partners and the pull/push of global enterprises at the local markets. our research model and collection of data we view electronic commerce as a tripod of actors, work processes and products each being either physical by nature or digitised. in turn, we have eight cells of which seven qualify for our concept of electronic commerce. our definition of electronic commerce is in line with the one proposed by choi, stahl and whinton (1997) although we find the bulk of electronic commerce both at present and in the coming years to be driven by the business-to-business commerce rather than the business-to-consumer commerce. also, we do view the public sector as a major market player in both the initiators and actual electronic commerce action. this active role is attributed to the large size of government in the scandinavia countries and to the ongoing trend with publicprivate sector partnerships. virtual products *ec core virtual processes digital products physical products physical agents digital agents virtual players figure i. the electronic commerce tripod source. modified after choi, stahl and whinton (1997) to obtain data on the business-to-business electronic commerce, data on number of messages and size of messages distributed on business sectors were obtained from the seven major vans operators in denmark. the data covered the period january 1 till june 30 1998. also, we interviewed 14 business associations, eight major enterprises and eight public sector units directly involved in edi-business (procurement, document exchange with the private sector on legal basis, or document exchange with international organisations within trade and statistics) (andersen, bj0rn-andersen, & juul, 1998). to obtain data on the business-to-consumer electronic commerce, we interviewed a total of 30 enterprises as well as analysing recent statistical surveys conducted by the danish ministry of research and information technology, statistics denmark, oecd and idc. 131 aj1s special edition electronic commerce november 1998 regulation: the old fight between markets and governments the edi-strategy is by and large a consensus strategy among the corporate sector and the government. in denmark the edi council is the administrative unit responsible for the electronic commerce strategy. the ministry of research and information technology and the ministry of business and industry grant about dkk 18m over a 3-year period to finance the increased activities of the danish edi council. moreover, these ministries provide a pool of dkk 6.6m over a 3-year period to subsidise standardisation work, in particular, across industrial boundaries. these subsidies are distributed under the auspices of the danish agency for development of trade and industry, as recommended by the danish edi council. the national electronic commerce strategy formulated in 1996 has seven key areas: 1. establishment of edi standards in all sectors 2. edi to be used for public procurement contracts under eu tendering by 1998 3. ah public-sector financial systems should handle all commercial documents in edi by the end of 1998 4. edifact-based interchange of administrative information with the public sector in fax, statistics etc. 5. development of edi software to facilitate the above 6. legislation on digital signatures and electronic documents by early 1998 7. danish edi council to act as initiator and coordinator the headquarter chief of defense denmark, the danish state railways, the copenhagen hospital cooperation, the danish palaces and properties agency, as well as the municipality of copenhagen formulated an edistrategy to include their suppliers' ability to partake in edi as an integral condition for inviting tenders as of 1998. finally, the danish government has made it official policy that all public procurement in year 2000 should be done using edi. it is our view that too much of the regulation debate concerns actions solely by governments ignoring the self regulation patterns in most business area where some governmental but more business associations corporate at national and international level to regulate their activity. although government and business do hold conflicting motives, we find similarities in the means applied and the problems encountered for the institutional actions by business associations and government. governments and the actions taken to drive society towards the digital economy is in the long run at the mercy of the voters. similarly the managers/ consultants of programs on electronic commerce within the business associations can be portrayed as balancing at the mercy of their members. although we do not subscribe to this rational and non-conflictual view on decision-making in organisations, we do acknowledge these motives. indeed, business associations do hold their own motives that might in the short term differ from the preferences of the members. encountering the actions taken by business associations one also find the endless line of seminars and information campaigns, videos, go-home-meetings, demonstration cases and financial support for development of prototype software. to comprehend the structural features, the initiatives at the local level of government, and the operation management choices within the government, we propose to distinguish between whether the actions by institutions are direct or indirect in scope. direct actions are targeting edi usage by subsidising the direct investment for ongoing edi use, for example, by defining protocols for edi use by customs and port authorities. indirect actions capture initiatives which aim at increasing edi "awareness" or acceptance, accomplished through public procurement as well as participation in edi councils, international committees and special interest groups. we view both direct and indirect actions as having four modes: 1) pedagogical, 2) economic, 3) normative initiatives for the use of edi in general, and 4) organisational management for the public sectors' own organisations. these four categories encompass the categories outlined by damsgaard and lyytinen (1997) on "macro level factors that can affect edi diffusion and how these factors are being modified through government intervention." we encompass both national governmental intervention modes, other overall macro-level factors (cultural, economic), supra national actions, and local governmental actions. thus, our four modes of action cover all these levels. pedagogical initiatives involve knowledge dissemination with edi seminars, materials aiming for improving edi knowledge, and support for general edi related research at the universities and international research committees. economic initiatives involve the provision of favourable, often subsidised, pricing for network services, as well as direct subsidies of edi activities, either as "pilot-projects" or on a more permanent basis. 132 ajis special edition electronic commerce november 1998 our third mode, normative initiatives, captures directives and legislation on syntax rules, data dictionaries, and document standards utilised in business transactions with the public sector. finally, our forth mode, organisational management, focuses on the work processes in the public sector's own organisations and level of government. due to a broader range of special interests, the politics of edi management can be significantly more complex in the public sector. for example, the tedis project noted that in the public administrations' use of edi, "...links between public administration and commercial organisations, for example, taxation, state insurance and statutory company declarations, form a significant special case. on the one hand the government departments responsible represent a monopoly, so at the local level they do not have to co-ordinate the introduction of electronic trading with any other equivalent body. however, public administrations lack the commercial pressures to seek efficiency improvements through it innovations faced in the private sector. also political considerations inhibit public bodies from insisting on trading by edi, so they face the prospect of operating parallel systems for the foreseeable future (graham & lobet-maris, 1994). table 1. classification of government actions in the electronic commerce area mode of intervention direct actions indirect actions pedagogical • knowledge deployment • innovation directives • transfer of ec technology • mobilisation • knowledge building ec-councils co-ordination of global electronic commerce economic direct subsidy of ec projects • subsidy for educational ec-projects and general it learning • subsidy for information infrastructure (for global electronic commerce) • provision of favourable charging modes for network services normative innovation directive standard setting legislation on signature and ec electronic innovation directive standard setting public procurement organisational management public procurement legislation on the use of electronic signature and ec co-ordination of electronic commerce with governments abroad adoption of private management techniques services reengineered to be more consumer oriented collaborative working relationship balance of power in government 133 ajis special edition-electronic commerce november 1998 in table 1 the first three modes encompass the traditional government actions. but organisational management is equally important to each of these three modes, since, in many countries, such as denmark, the public sector may be a more avid user of edi than the private sector. consequently, government actions may affect both the demand-pull as well as the supply-push for edi in the private sector. accordingly, when we address the question "how does government intervention affect the edi-diffusion process", both the public sector's diffusion and the private sector's adaptation are of equal interest. in addition, the plethora of instruments outlined in table 1 reflects how complicated it is to stimulate the diffusion of electronic commerce and to estimate how government intervention affects the process. for example, the tradenet in singapore did not achieve success status merely by bottom-line analysis and top-down steering. instead, this example showed the need to stimulate and evaluate edi diffusion in its organisational context, public or private regardless. also, local government and quasi-governmental organisations might be just as successful, or more so, in initiating low-cost edi solutions compared with central government. we believe this is a very important observation given that governments have multiple forms and the distinction between private and public is no longer as clear as it was a century ago. moreover, the spread of semi-public companies is changing the roles of politicians, managers and the staff in the public sector and influencing the mien of the private sector. in addressing edi and data networking these changes in the environment, both the political and the commercial elements, pose challenges for the successful diffusion of edi. all too often this notion is ignored (saxena & wagenaar, 1997; scala & macgrath, 1993). accordingly, governments might not see it as their primary role to pursue top-down steering or legislation of the edi diffusion process. using such crude strategy might stifle innovation, discourage competition, and eventually leave the national economy worse off. instead, central government should move onward on a large variety of fronts including fostering conditions that tear down obstacles for effective edi use. more specific examples could be encouraging legislation for digital signatures, investing and building an information technology infrastructure, as well as organising a watchdog against monopoly practices. within the public and the private sector, we have found some very interesting b2b examples. examples on b2b electronic commerce the danish national board of industrial injuries the danish national board of industrial injuries is a financially autonomous agency under the ministry of social affairs handling individual work related injuries. the work process involves retrieval of data, checking of legal data/ reference materials, and co-working with other colleagues in the office. most contact outside the organisation is done through traditional mail (incoming mail is scanned) and phone. within the organisation communication is either face-to-face meetings, phone meetings, memos, and exchange of electronic documents. in 1995, the board received 327,000 letters, while they sent off 375,000 letters. they received 46,000 notifications of injuries and made 90,000 decisions. in 1997, the insurance companies and the board started using edi. the insurance companies are required by law to use the board in cases of worker compensation related to injuries. the board needs to check insurance numbers, the insurance companies need to check social security numbers. by using edi, the two partners have bypassed a legal barrier, which prohibits insurance companies from obtaining direct access to centrally stored personal data. public procurement the most demanding challenge for the near future is to implement edi based electronic trade into public purchasing. while the need to obtain cost reductions in administrative processes is evident, the barriers of technique, tradition and attitude are still pervasive. national procurement has assumed a major role in preparing the public sector in denmark for the electronic purchase of the goods and services. in 1997, national procurement ltd. introduced an edifact based database which encompassed all goods and services in the current paper based purchasing system. in addition, an electronic public purchasing system has been designed in cooperation with the national and local public network operators. these two systems are completed with ean location numbers for all subscribers as well as a set of edi documents and standards which will form the backbone of a thorough public trade environment. this will be combined with an open interface to other 3rd party goods and services databases and administrative systems used in the public sector in the course of 1998 all public authorities will be able to commence transacting via edi. together with national procurement, ltd., the agency for financial management and administrative affairs is developing a basic procurement system for public economic management systems. the procurement system became operative for municipal and central government users as of february 15 1998. 134 ajis special edition • electronic commerce november 1998 taxation the danish central customs and tax administration (customs*tax) aims to receive all documents electronically. their edi strategy consists of two major elements: 1) a strategy for the handling of all incoming data from companies electronically through an alliance with the statistics denmark and the danish commerce and companies agency, ministry of business and industry; 2) enabling citizens to deliver their advance tax assessments and income tax statements via the internet and voice-response. the requirements for edi applications used by customstax customers are: 1. right amount on time. 2. better servicing of companies, including a reduced effort on their behalf. 3. effective use of resources. 4. correct and lawful administration. 5. development oriented. 6. political satisfaction. the new edi interface developed by statistics denmark, danish commerce and companies agency, the ministry of business and industry, and the central customs and tax administration enables companies to deliver their declaration regarding vat and individual article tax electronically. in 1997, more than 20% of all danes made use of the service to enter and transmit information for the advance tax assessment and the income tax return by telephone. less than 1% used the internet for this task in 1997. table 2. central customs and tax administration: documents handled manual and electronically, 1997 (1,000) document: manual data electronic data transfer entry employers' income statement interest (financial institutions) trade union dues income tax statement vat intrastat import and export, total a) via internet health sector 1,000 150 100 780 1,600 900 2,800 edi 10,000 31,000 5,300 30" 0 4,000 1,100 phone 0 0 0 600 0 0 0 the danish health care sector is considered to be one of the most automated in europe. currently, 15% of discharge letters, 7% of laboratory results, and 10% of pharmacy prescriptions are handled by edi on average. in the copenhagen municipality, edi is also used in the communication with the dentists and eye specialists. within fuen county, the general practitioner (gp) sends a referral to the hospital department, and the hospital sends discharge letters to the gp and the municipal social administration and the health insurance using edi (andersen, 1998; fuen county, 1995, 1996). also, when the pharmacy, hospital and gp order medicine, edi is replacing paper based order and invoice forms. currently, the fuencom project is being expanded to cover the entire country (medcom). after establishing a computerized health data network within the organisation, the county estimated that substantial revenue had been achieved (fuen county, 1996, p. 46). there is, however, major variance in how much of the communication is wired. in figure 2, we compare the communication in fuen and the average communication in denmark based on the number of letters of discharge, prescriptions, pharmacies, laboratory test results, and doctor's practices. twenty-seven percent of the letters of discharge are computerized and are part of a network in fuen county. yet, at the national level, only 10 percent of the letters of discharge are part of a network. most remarkable, however, is that almost 80 percent of the laboratories send and receive their messages through electronic communication in fuen county, whereas only 20 percent of the communication in the labs are processed electronically at the national level. furthermore, more than 4 out of 5 pharmacies are wired. this is true for fuen county as well as the national average. 135 ajis special edition electronic commerce november 1998 figure 2 example of edi in the danish health sector source. constructed after fuen county (1995, 1996) source. andersen, ed. (1997), andersen (1998) the large companies apply edi both internally between a company's departments at home and abroad and in relation to suppliers and the distribution link. there is hardly any application of edi in relation to small buyers of products. owing to the dissemination of efficient consumer response (ecr), especially in the provisions trade, edi application has increased as electronic point of sales (epos) systems use the hancom standard. in january 1997, dansk supermarket started to use edi invoices, and at present the company runs a production with 29 suppliers. from 1996 to 1997 the number of suppliers with edi application has increased by 61 per cent. measured in terms of order items the number has increased by 27 per cent in the same period. approx. 80 *-/-.r cent of the deliveries to fdb's stores are made via the central warehouses. all communication to the stores i • ^de via an internal computer system that does not apply edi format. there are no plans to change the systerr 'i is not applied in connection with domestic transports, however, it is extensively applied in connectio.h transport abroad. shipping via dantransport enables fdb to get edi information about e.g. cargo froii. ihe far east. together with dsb gods the company also participates in a development project on edi application in connection with full-load transport of goods. fdb has been a pioneer in the application of edi in danish provisions trade, and they made the first edi messages (orders and invoices) in 1991 after hancom was released the first time. already in the 1970's, fdb made their own proprietary standards with meat packers, the dairy industry, breweries and newspaper headquarters. at present (1997), fdb receives 96 per cent of all invoices in edi format. in 1995, the figure was 85 per cent. the remaining percentages are made up of small companies or suppliers that only perform a small amount of trade with fdb. from this category fdb may still receive hand written invoices. placing orders is done via edi, letter or fax, where roughly half the orders are placed via edi. fdb has approx. 1,200 suppliers, and roughly 400 have agreed to apply edi. the major part of the transfers are batch-transfers. not only don't the smallest suppliers use edi, but some of the large suppliers have been slow to implement the technology. according to fdb it is only a minority of suppliers that will take the lead. 136 ajis special edition electronic commerce november 1998 figure 3. network distributed messages in fuen and in denmark on average 90 n 80 70 60 50 40 letter of descriptions pharmacies laboratory doctors' discharge responses practice note. data provided by the danish center for health informatics the retail sector: dansk supermarket and fob fdd's deliveries to local authorities are not part of the national procurement ltd. agreement. local authorities' procurement to institutions comprises e.g. small, daily purchases of food, etc. at present, efforts are being made to introduce a card solution to fob, where purchases automatically will be forwarded to the basis procurement system and thus reduce the high costs of processing and forwarding paper invoices. the project is carried out in cooperation with kommunedata (computer center of local authorities). fob does not use the internet in connection with orders and invoices. in 1996, fob made an experiment with placing orders via the internet in vellev in jutland, where the local fdb store had been closed down. fob evaluates that the internet may become a competitor to vans. toy manufacturing: lego according to lego, edi application is only an advantage provided bar coding and tracking of goods have been introduced as well. most of lego's activity within the edi area is focused on sales. partly, lego applies edi at an advantage, and partly edi is a necessary condition for implementing efficient consumer response in the company. in denmark, orders and invoices are run in edifact. there is no venture management in the danish part of lego. a fairly small number of stores use computerized cash registers in connection with reordering goods due to lack of precision in data registration. edi application is considerably more popular in the american part of lego than in the european departments. it is not a problem that ansi standards are used in the usa and edifact in europe as the two systems are not integrated. as 60 per cent of the turnover is made in the christmas month, the stock must be built up to reach its peak in september/october. lego applies the just-in-time principle in connection with all large business transactions. the principles from the provisions trade have spilled over into the toy business. edi is not used in relation to toy chains, etc. they place their orders by fax or via a sales representative. in a large number of areas edi application in europe is way behind edi in the usa, where lego has almost 100 per cent of its customers on edi. at the same time, lego has achieved a drastic increase in sales. small distribution costs, just-in-time and efficient order and delivery control are some of the principal factors. chains, such as walmart, require that suppliers apply edi. in the usa, the concept is larger chains, operated especially by companies such as walmart on the edi area. as a consequence, there is a great pressure on the retail area. in addition to the provisions trade, it applies to the branded goods area, especially proctor & gamble. previously, a chain, such as walmart, fought with brand mark producers about discounts in connection with large orders. now the trend is to secure advantages for both the company placing the order and the producer by manufacturing directly for delivery instead of for building up expensive stocks. in the usa, the smallest sales unit is a carton, in denmark it is a single package. it goes without saying that the logistic needs are many times bigger. within the transport department area all booking, tracking, invoicing, including statistics are made via edi. 137 ajis special edition-electronic commerce november 1998 production plants: grundfos and danfoss already before the edi agenda was introduced, grundfos and danfoss engaged in a complete application of edifact. grundfos has an extensive use of edi in connection with suppliers, shipping and between the company's departments (17) in denmark and abroad. grundfos has been a front runner in relation to edi application and had already in 1990 started on electronic document interchange, where a substantial part was in edifact format. they apply edifact in connection with orders, invoices and delivery status. 20 per cent of the communication with suppliers is via edifact. all communication with shipping in denmark is in edifact format. between the company departments 80-90 per cent of communication on orders, invoices and delivery is via edifact. these figures have not changed since 1995. at present, grundfos is implementing sap-r3 in production and management accounting systems. this may imply a larger number of electronic documents in addition to a more extensive integration of subsidiaries. naturally, it becomes a question whether it is possible to measure the edifact application merely by the vans traffic, as it will depend on the form of communication chosen for sending data between the different departments. danfoss applies edi between the individual (financially separated) units. primarily, danfoss markets its products through the subsidiaries, which involves a range of business transactions where edi can be used due to the great volume in communication. orders and invoices are sent between all business units in danfoss. in the last 10-15 years, the company has engaged in edi, i.e. before the industry associations and the government seriously began to focus on edifact. therefore, the group was forced to develop their own applications and document layout. in 1998, this is still apparent as roughly 70 per cent of all electronic communication is made via their own proprietary standards and only 30 per cent in edifact format. however, it has not been possible to make an estimate of the marketing subsidiaries' edi communication with their customers. in the electronic post office in danfoss (gate) there is an interchange of data between the different product divisions (10-15), marketing subsidiaries (30-40) and production facilities (10-20) and with external relations (banks, suppliers, shipping agents, central customs and tax administration, etc.). they are spread all over the world. the communication goes via infonet, dan net and ibm global network. danfoss has 15-20 trading partners with whom they have exchanged invoices, orders and order confirmations over the past 5-7 years. a new element is transport companies. on the danish market danfoss cooperates with a few large transport companies where edifact is applied. another new element in danfoss (since october 1997) is web-business applications, where a customer can place an order in different parts of danfoss via internet. up till now two applications have been developed. they handle placing of orders (customer side) and order confirmation (supplier side). invoices, etc. cannot be sent. orders placed via form-based edi are converted in gate to normal edi format and forwarded to the production unit. shipping: mcersk operationally, maersk shipping line is covered 100 per cent by edifact in terms of bayplan and the large ports such as rotterdam and felixstove. roughly, maersk shipping line receives 1 million edifact messages a month and sends 500,000. in terms of customer relations the company receives approx. 200 consignment notes and bookings daily via edifact application. settlement of payments is not run via edifact. edifact messages are received via vans operators and as attached files to e-mail. the option to send orders by e-mail is especially employed by small companies. the battle of standards throughout the 1990s journals and conferences proceedings were filled with questions on whether edi and in particular edifact/ ansi x.12 would tighten business partners so much that the market clearing mechanisms were threaten. economists were speculating on whether the costs related to a potential decreased search for market alternatives would exceed the benefits achieved on reducing transaction costs and lowering the error ratio of re-keying the data in the value chain. turning to the vans based traffic, our data show that there has been sent about 45 percent more messages annually and 33% larger volume of messages (bytes). in particular, the health area, finance and distribution use vans operators 138 ajis special edition electronic commerce november 1998 table 3. edifact traffic in denmark, 1995-1997. year bytes messages n annual n annual growth growth (%) (%) 1997 57.110.724 33% 1996 43.007.936 33% 1995 32.375.495 26.593.813 45% 18.401.306 46% 12.630.458 in figure 4 and 5 we have analysed the vans based traffic for all vans operators in denmark. the adjusted line (dotted) in figure 1 where y equals 47.996x + 712.793, with y representing the number of messages and x is the time (months). r-square for the simple regression line is 0,96. also, the data can be analysed applying an exp function with y equal to 850.538e°-03i3]t. the data show a rapid growth in the last part of the period included in our data material. figure 4. edifact via vans 1995-1997 (months and messages) 3200000 2700000 2200000 1700000 1200000 700000 y = ssosaae0'0313" rp = 0,959 y=47996x+712793 r>= 0,9597 1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 similarly, in figure 5 we have analysed the vans traffic measured by bytes. the adjusted curve is y equals 84.572x + 2.200.384, with r-square = 0,9. applying an exponential-function provide, however, a better r-square on 0,91. 139 ajis special edition electronic commerce november 1998 figure 5. edifactvia vans 1995-1997 (months and bytes) 6000000 5500000 y = 84572x + 2e+06 f¥= 0,8971 2000000 11 13 15 17 19 21 23 25 27 29 31 33 35 rather than using vans operators, one might use the internet for edi. approximately 50 per cent of danish enterprises with more than 5 employees have access to the internet. almost 80 per cent of the enterprises communicate electronically from the enterprise via call connections or leased lines. when considering the variety of tasks, in which edi is used, approximately 50 percent of the counties used it to obtain information about products and services, whereas roughly 42 percent use it for payment. by contrast, only 5% of the municipalities used it for payment purposes. thus counties appear to have a significant lead in utilising electronic commerce. this is most likely attributable to the business-to-business edi predominance in the health sector, which are primarily administrated by the counties in denmark. carriage media transformation using internet technology one has the potential to substitute the use of vans operators or direct access in favour of internet edi solutions, labelled browser-edi in this article. this triangle of transportation media might not be an either or, rather it is often complementary modes of transportation such that some require and use direct access, others, vans-based communication while others settle the internet mediated edi solution. as we have seen in the previous section, the vans operators were not out of business or showing any indication of less market growth. however, in the coming years there might be an expansion in the edi market using other techniques for transportation. rather than using vans operators, one might use the internet for edi carriage. approximately 50 per cent of danish enterprises with more than 5 employees have access to the internet. almost 80 per cent of the enterprises communicate electronically from the enterprise via call connections or leased lines. interviewing business organisations and key informants in the public sector, we found that overall they estimate an immense potential for the edi usage within the next two years. nine out of ten indicated that 90% of the current paper documents can be transformed to electronic forms transported either through the internet, direct access or vans-operators. the data is shown in the first column in table 4. we also asked the respondents to give their estimate on where the potential for transforming paper based documents to electronic forms is most likely to be completed using vans, internet or direct access line. the opinions on these future trends are shown in columns two, three and four, each distributed on business sector. 140 aj1s special edition electronic commerce november 1998 figure 6. the transportation media, integration, format triangle trans port media vans internet leased lines (intranet, extranet, a.o.) message format edifact propietary lite raction /automation full integration (both ends) semi-integration, edi-lite table 4. edi transportation media business sector the public sector insurance companies and banks industry and commerce transportatio n national counties local government insurance and pension banks real estate industry wholesale carpenters etc. (smes) construction consumer commerce agriculture heavy freight letters and packaging shipping potential for using edi transportation mode/ standard proprieto ry edifact browseredi percent — 50 60 50-60 30-40 20-30 20-30 80-85 95 95 95 70-80 30-40 30 40 20-30 10 10 10 10 75 75 n.a n.a n.a n.a n.a n.a 30 20 30 15 20 25 50 2 n.a 15 5 5 5 25 30 30 25 30 15 10 1 1 0.1 1 1 1 1 1 1 10 10 10 15 141 ajis special edition electronic commerce november 1998 the edi technology might be fully integrated or semi-integrated and as such can is used to reach not only smes but also end-consumers. the spread of xml-technology can spiral this. in our interviews with business leaders in business associations, enterprises and public sector institutions most indicated a major shift in the use of transportation media for edi. the winner might be the stakeholder in the internet. the dream wave keeps coming business consultants worldwide sell the dreams of the ultimate integration of dataflow within the department and with the business partners. enterprise systems such as sap and oracle are the latest elements in the drive towards the fully integrated organisation. yet, the nightmare of multiple data-reentry, scanning invoices, mailing letters in hard cover form, ordering computer manual by fax continues. most organisations today run multiple formats of data, among them the digital format appears in yet more instances. the danish government made it a policy goal that by year 2000 all public procurement should be done electronically. at the time (1996) of the policy formulation, this implied the use of edi-systems through other channels than the internet as this was not advanced to a stage where issues as technology, security and trust were advanced to an implementation stage. the internet technologies were at that time in the making. the policy commitment to edi-use spiralled a policy debate quite similar to the one on internet use: would it benefit the larger, international owned countries, would the local shops loose business, would the local community suffer as a result of the business transformation? this classical dilemma faced in the municipality of skive, the design of new edi system labelled "basisshopping" developed by the partners (local government computing association, the 0konomistyrelsen and the statens and kommunernes indk0bsservice) in the vibe-project. in skive municipalities there are about 1,000 suppliers ranging from the bakery with fresh bread supply to the kindergarten, to the computer suppliers, to the accounting department and the car supplier, to the road construction department. the accounting department handling all the bills and accounting processing data, signed up with suppliers to use edi systems to decrease the costs of keying the incoming bills, transferring the payment and in processing the data on to the accounting and financial systems. along with the installation of the edi-converters and the offer to the business-partners to join in, a scanner was installed in the accounting department. by year 1 after the edi-systems was in use, only 18 of the suppliers signed up for the edifact-system. however, the accounting benefited immensely from the investment in a scanner enabling electronic bookkeeping. when we look at the data for danish local government in general for 1997, only 8% of the municipalities used edi, whereas 75% of the counties and 15% of the private enterprises used electronic commerce in the procurement function. thus while edi adoption has been quite successful at the county level, substantial opportunities exist for increasing electronic transacting at the lower governmental levels. 60 per cent of the ministries and 36 per cent of agencies and directorates have a home page. 90 per cent of the ministries and 53 per cent of agencies and directorates may be contacted via an official e-mail address. 36 per cent of the counties have a home page and 64 per cent have an e-mail address. 16 per cent of the municipalities have a home page and 30 per cent have an e-mail address (danish ministry of research and information technology, 1997). accessing the overall use of it to reconfigure organisations, most of the applications are localised or internal integration. however, there has been a high degree of informatization during the period 1993-1996. in 1995, for example, 31 percent of the national government units had no use of electronic archives, management systems or mail systems, while in 1996, only 4 percent had no use of these applications. similarly, in 1995, 6 percent of the municipalities used electronic incoming mail and electronic case handling, while in 1996, 51 percent of the municipalities used these applications. when considering the variety of tasks in which edi is used, approximately 50 percent of the counties used it to obtain information about products and services, whereas roughly 42 percent use it for payment. by contrast, only 5% of the municipalities used it for payment purposes. thus counties appear to have a significant lead in utilising electronic commerce. this most likely attributable to the business-to-business edi predominant in the health sector, which is primarily administrated by the counties in denmark. 142 ajis special edition electronic commerce november 1998 table 5. it in the public sector, 1995-1996. (percentages) level of no use of electronic electronic electronic governme electronic mail used for archiving or handling of nt/ year archives, internal or electronic case incoming mail management external use handling and electronic or mail systems case handling systems introduced national 1993 31 1995a 15 1996a 4 municipal 1995b 11 ties 1996c 2 counties 1995b 7 1996c 0 note. a= spring, b= primo, c= ultimo source. danish ministry of research and 49 34 14 45 19 36 21 information 20 49 46 38 28 57 14 technology (1997). 0 2 35 6 51 0 64 table 6. electronic commerce: the content (percentages)22 level of gov'ment municipalities counties enterprises information on products/ services 6 50 15 prices and discounts 4 34 7 delivery conditions 3 25 5 ordering of products/ services 5 34 6 payment for products/ services 6 42 2 in the private sector we find that the choices faced by managers in the public and private sector on business integration issues are ambiguous and characterised by substantial uncertainty on for example the role of intermediates, direct sale, hyper-shift in business partners and the pull/push of global enterprises at the local markets. at the private market, the role of intermediaries is changing but seems not to eliminate either the intermediaries or the retail stores. we do see showrooms popping up and market share taken by virtual intermediaries such as amazon and in scandinavia boxman for cd shopping. the entertainment industry and software are in denmark among the economically favoured by the internet. the public sector is confronted with this development not only from a regulation point of view, but also in daily operations in for example procurement function. conclusions the drive towards electronic commerce organisation is a troublesome path for government, enterprises and consumers worldwide. the danish government is stimulating the use of electronic commerce using direct and indirect policy initiatives on central, governmental regulation and establishment of greens enabling selfregulation. also, the governments own, organisational management is in denmark seen as a mean to exalt electronic commerce. onwards, the fist on proprietory standards and the un/edifact is an ongoing source of delaying, stimulating or preventing electronic commerce depending on the business sector and the size of market actor addressed. in denmark, the direct access and value added network supporters have so far been the dominating mean to 22 205 of the 275 danish municipalities and 12 of the 14 danish counties participated in the survey source. danish ministry of information technology and research (1997). 143 ajis special edition electronic commerce november 1998 transport electronic documents in business transactions. the internet and xml technology is at the turn of the century challenging the way of doing business within this field. some view this as the big blue to speed the diffusion of electronic commerce; others are worried that the incentive to investment and pull the partners in the value chain to use edi might be lost. finally, our study concludes that the debate on business integration issues is ambiguous and characterised by substantial uncertainty on, for example, the role of intermediates, direct sale, hyper-shift in business partners and the pull/push of global enterprises at the local markets. the future areas of research to be undertaken within this field includes development on the technical elements, the institutional and organisational features, the suppliers and the demand pull patterns. on the technical elements the battle between edifactand ansi x.12 versus proprietory will be supplemented with technical issues on transportationmedia and dataintegration issues. the transportation media include the dilemma on direct access, vans-operated edi-transport and internettechnology. the heavily consumer orientation in the current design of internet application enabling the marketing, sale and service of either digital or physical products are in our view holding the potential to define businesses and challenge the role of intermediaries. in turn, the choices faced by managers in the public and private sector on business integration issues is ambiguous and characterised by substantial uncertainty on for example the role of intermediates, direct sale, hyper-shift in business partners and the pull/push of global enterprises at the local markets. also, there is a major need for studies on methodological issues such as the definition of use and the business impacts of the use. at the danish center for electronic commerce (cec) at the center for research on it in the public sector (cips) these issues are addressed in our ongoing research on it. references andersen, k. v. (1998). health data networking: capability, interaction, orientation, and political values. hawaii international conference on systems sciences (hiccs). andersen, k. v. (ed.). (1997). edi and data networking in the public sector: governmental action, diffusion, and impacts. amsterdam: kluwer. andersen, k. v., bj0rn-andersen, n., & juul, n. c. (1998). elektronisk handel og dokumentudveksling: edi-anvendelsen i danmark. copenhagen: samfundslitteratur. choi, soon-yong, stahl, dale 0., & whinston, andrew (1997). the economics of electronic commerce. indianapolis, in: macmillan. damsgaard, j.,& lyytinen, k. (1997). government intervention in the diffusion of edi: goals and conflict. in k. v. andersen, k. v. (ed.), edi and data networking in the public sector: governmental action, diffusion, and impacts. amsterdam: kluwer. danish ministry of information technology and research. (1996). information technology in danish government. copenhagen: author. danish ministry of information technology and research. (1997). authorities heading for a fall: it policy white paper presented to the folketing 1997 danish ministry of research and information technology (1998). information technology in denmark. copenhagen: author. fuen county. (1995). fyncom: det fynske sundhedsdatanet [the health data network in fuen]. odense: danish center for health informatics. fuen county. (1996). medcom: det danske sundhedsdatanet [medcom: the danish health data network]. odense: danish center for health informatics. graham, i., & lobet-maris, c. (1994). edi impact: social and economic impact of electronic data interchange. tedis project c9. brussels: european commission. saxena, k. b. c., & wagenaar, r. w. (1997). critical success factors of edi technology transfer: a conceptual framework. proceedings of the 3rd european conference on information systems (ecis) (pp. 57-74). june. athens, greece. scala, s., & macgrath, r., jr. (1993). advantages and disadvantages of electronic data interchange. information & management, 25(2), 85-91. statistics denmark (1998). danmark i tal. copenhagen: statistics denmark. 144 the australasian journal of information systems (ajis) began in 1993 and was based on the simple realisation that there was a need for an academic ‘outlet’ for is research in australia australasian journal of information systems volume 18 number 2 2014 21 an empirical evaluation of the potential of public e-procurement to reduce corruption arjun neupane university of southern queensland email: arjun.neupane@usq.edu.au jeffrey soar university of southern queensland email: jeffrey.soar@usq.edu.au kishor vaidya university of southern queensland and university of canberra email: kishor.vaidya@usq.edu.au kishor.vaidya@canberra.edu.au abstract one of the significant potential benefits of e-procurement technology is reducing opportunities for corruption in public procurement processes. the authors identified anticorruption capabilities of e-procurement through an extensive literature review and a theoretical model representing the impact of three latent variables: monopoly of power, information asymmetry, and transparency and accountability upon the dependent variable, the intent-to-adopt e-procurement. this research was guided by the principalagent theory and collected the perceptions of 46 government officers of the potential of public e-procurement to reduce corruption in public procurement processes. results were analysed using the partial least squares structural equation modeling (pls-sem) approach. the findings suggest that the intent-to-adopt e-procurement has a positive and significant relationship with the independent variables that might inform developing countries in strategies to combat corruption in public procurement. keywords: public procurement, public e-procurement, anti-corruption, monopoly power, information asymmetry, transparency, accountability introduction there is interest amongst governments across the world in the adoption of information and communication technology (ict) tools for the promotion of the better service delivery (ndou, 2004), greater transparency, and improved accountability (bertot,jaeger & grimes, 2010; kim,kim & lee, 2009). there is recognition that the implementation of public e-procurement technology is a key tool for reforming government public procurement processes. thai (2001) pointed out that public procurement is a major function of government and thai argued that government agencies, policy makers, and public procurement professionals have a great interest in public procurement reforms. the potential benefits of e-procurement include reducing corruption risks (iqbal & seo, 2008; mistry & jalal, 2012). ict-enabled technology tools that have been adopted by government include egovernment, e-governance, e-procurement, e-tendering, web-based erp, e-services, e-democracy and other electronic services. each system is developed for a purpose with its own specific function and australasian journal of information systems volume 18 number 2 2014 22 characteristics. the main question is how these integrated systems link to reduce corruption. this study evaluates the perceived potential of perceived benefits of public e-procurement technologies to help to curb corruption. this study discusses three concepts: public procurement, corruption in public procurement, and how public e-procurement may help to reduce the risk of corruption. public procurement refers to the processes whereby public sector organizations acquire goods and services, works, and other activities from third parties. governments spend much of their budget on goods and services. government procurement of goods and services typically accounts for 10-15 % of gross domestic product (gdp) for developed countries (kashap, 2004) and 20 % to 70% gdp in developing countries (gtn, 2003; undp, 2006). vaidya, callender and sanjib, (2009, p 474) found that “governments aspire to use public procurement as a lever of economic, technological or social reform”. burton (2005) and thai (2001) argued that public procurement is the central instrument and major function of the government to assist in the efficient management of public resources. shakya (2012) research concluded that success factors of public procurement are value-for-money, transparency, non-discrimination, competition, and ethical behaviour. the term corruption is defined as the exercise of public power for private gain (cuervo-cazurra & genc, 2008). corruption can take different forms including bribery, theft, embezzlement, abuse of discretion, favouritism, exploiting conflict, putting private interests above public interests, and improper political contributions (unodc, 2004). corruption in public procurement is increasing especially in developing countries as evidenced by reports of the world bank, the united nations (un), transparency international, and the asian development bank (adb) reports. corruption can hinder the development of country by weakening national institutions, increasing business costs, eroding trust, and discouraging external investment (lio, liu and ou (2011). raymond (2008) argued that corruption is a serious problem for developing countries that actively hinders their progress. the annual value of bribes paid worldwide is estimated at us $1 trillion (wb, 2004). corruption threatens the economic and human development of many countries. corruption in public procurement processes is linked with unjustified and hidden procurement planning, lack of monitoring and controlling mechanism, lack of transparency and accountability, and weak professionalization of the bureaucracy of the countries (del monte & papagni, 2007; kolstad & wiig, 2009; neupane, soar, et al., 2012; pellegrini & gerlagh, 2008; subedi, 2006; ware et al., 2012). electronic procurement is the use of information and communication technology (ict) such as the internet or web-based systems designed to facilitate the acquisition of goods and services by the government or by private institutions (davila,gupta & palmer, 2003). it has played a significant role in the successful management of public resources (mahmood, 2010) and most importantly to better enhance the transparency and accountability of government procurement (croom & brandon-jones, 2005; panda,sahu & gupta, 2010; vaidya,sajeev & callender, 2006). for the purpose of this study public e-procurement has been defined as “the use of any internet based inter-organizational information systems which automates and integrate any parts of procurement process in order to improve efficiency, transparency, and accountability in the wider public sector” (vaidya, 2007). the internet is typically used for providing comprehensive information about bidding at a single web portal that can be accessed by the government and all the register potential bidders. reducing corruption in government procurement is a key component of the agenda of governments around the world (henriksen & andersen, 2003). countries of asia and the pacific have increasingly adopted ict systems in order to enhance government services and business transaction (wescott, 2001). public e-procurement can be an important tool to help reduce the risk of corruption within public procurement systems. asian development bank /oecd (2008, p. 129) stated that “public eprocurement plays an important role for minimising corruption in public procurement”. e-procurement also has other numerous potential benefits including standardising and monitoring procurement, australasian journal of information systems volume 18 number 2 2014 23 increasing transparency, reducing personal discretion in purchasing decisions, enhancing fair competition amongst bidders, avoiding human interference, and maximising value for money. this study explored the perceived potential of public e-procurement to reduce corruption in the public sector by addressing the following research questions: which anti-corruption factors are associated with the intent to adopt public e-procurement technology? in particular: are lower levels of perceived monopoly of power related to the willingness to adopt eprocurement as a form of anti-corruption technology? are lower levels of perceived information asymmetry related to the willingness to adopt eprocurement as a form of anti-corruption technology? are higher levels of perceived transparency and accountability related to the user’s willingness to adopt e-procurement as a form of anti-corruption technology? the paper is organised into eight sections. in section two, there is a discussion of the literature on anticorruption capabilities of public e-procurement. in section three, there is a discussion of the theoretical foundation of the study. the development of the research hypothesis and research model based on principal-agent theory presented in section four. the research methodology has been discussed in section five. section six presents the empirical data analysis. section seven discusses the finding of the research and recommendation. in the final section, the conclusion is presented, including academic contributions to literature and practitioner implications of the research. literature review it is clear that the recent boom of icts brought an extensive role for making transparent and accountable government as well as improving the living standards of global citizens. on the other hand, ict can been seen as the leader for socio-economic development of many countries (braund et al., 2007) and it is a crucial for sustainable development in developing countries (credé,mansell & mansell, 1998). many scholars have identified that icts have the potential to reform the government work and services and to improve public sector service delivery (singh et al., 2010), to support good governance (basu, 2004), to promote transparency and anti-corruption (bertot et al., 2010), to develop social capital (shim & eom, 2009) and many more other benefits. pathak et al. (2009) research highlighted based on the community perceptions on the service delivery that it enabled technology such as electronic government initiatives can cut the corruption as well as reshape public sector activities and processes, increases overall transparency, and strength relations between government and citizens. kim et al (2009) reported on an anti-corruption system called open (online procedures enhancement for civil application) which made a positive role in reduction of corruption. citizens could access acquire real time information system on their applications, their status, estimated date of approval, and the reasons for decisions from open system. likewise, krishnan, teo and lim (2013) study reveals that the maturity of e-government is seen to be linked to the effectiveness in reducing the level of corruption. oye (2013) pointed out e-government has been seen to reduce the administrative corruption and transparency in service delivery in african developing countries. in developing countries, major fraud and corruption happens in public procurement processes including procurement planning, project documentation, tender processes, contract awards and implementation, and accounting and auditing. report published by independent commission against corruption (icac) australia, clearly indicated that procurement processes are vulnerable to corruption (icac, 2011) and there is a risk of corruption in every stage of tendering process such as identification of needs and design of tenders, selecting a business, the bidding procedure, contract awards, and contract australasian journal of information systems volume 18 number 2 2014 24 execution (oecd, 2007). table 1 provides a link of public e-procurement anti-corruption factors to research papers. anti-corruption factors some key references avoid unnecessary purchase / project (achterstraat, 2011) real time access information or real time bidding (ndou, 2004) automation of procurement process (henriksen & mahnke, 2005) increase competition among the bidders or suppliers (hanna, 2010; imf, 2010; mahmood, 2010; thai 2001) reduce human intervention in bidding process (khanapuri et al., 2011; magrini, 2006) standardization enactment (more consistency in procurement phase) (un, 2006; zhang & yang, 2011) monitoring and tracking application (achterstraat, 2011; adb, 2010; oecd, 2011) efficient and secure document transmission (ag, 2005; chang, 2011; hanna, 2010; zhang & yang, 2011) managerial control and collaboration (henriksen & mahnke, 2005) transparency and accountability (croom & brandon-jones, 2005; panda et al., 2010; pathak et al., 2009; vaidya et al., 2006) make a procurement process faster and easier (hanna, 2010) obtain the best quality / price ratio (kaliannan,raman & dorasamy, 2009; lee,oh & kwon, 2008) table 1: public e-procurement anti-corruption factors source: (neupane, soar, et al., 2012) there is limited research on ways to reduce corruption within public procurement. vaidya et.al (2004) discussed twelve case studies and identified key benefits of public e-procurement in the australian public sector based on the use of the balanced scorecard approach and critical success factors. other literature discussed the key benefits of e-procurement including perceived direct benefits (brun et al., 2010; chang, wang & chiu, 2008; croom & brandon-jones, 2007; gunasekaran & ngai, 2008; ho et al., 2008; vaidyanathan & devaraj, 2008) that help to identify different key factors of better performance of public or private organizations including perceived direct benefits of e-procurement (brandon-jones & carey, 2011; mitchell, 2000; panayiotou, gayialis & tatsiopoulos, 2004), perceived indirect benefits of e-procurement (gunasekaran et al., 2009), perceived cost, firm size, top management support, information sharing culture, business partner influences (teo, lin & lai, 2009). eei, husain & mustaffa (2012) identified e-procurement’s potential tangible and intangible benefits including enhanced efficiency and effectiveness in an organisation through reductions in costs and procurement process cycle times, increased transparency in contracts and overall competitiveness e-procurement has been recognised as a core application in e-commerce / e-government (ronchi et al., 2010). research on combating corruption or anti-corruption includes: e-government as an anticorruption tool (andersen, 2008), e-government for improved public sector service delivery in india, ethiopia and fiji (singh et al., 2010), applying e-government information system for anti-corruption strategy (zhang & zhang, 2009), e-government, transparency and anti-corruption (fan,zhang & yue, australasian journal of information systems volume 18 number 2 2014 25 2009), anticorruption effects of information communication and technology and social capital (shim & eom, 2009), using ict to combat corruptiontools, methods and results (grönlund et al., 2010), and an institutional analysis of an e-government system for anti-corruption: the case of online procedures enhancement for civil application (open) (kim et al., 2009). kin et al. (2009) study results indicate that the open system made a positive impact in terms of a reduction of corruption within the seoul metropolitan government. studies regarding public procurement and corruption (auriol, 2006; lindskog,brege & brehmer, 2010; mahmood, 2010) indicate that the quality of public administration must be improved and made accountable. e-procurement can improve the transparency and integrity of public service processes such as tendering, sourcing, ordering, and auctioning. e-procurement has been recognised internationally as an important instrument for checking corruption and misuse of power (sohail & cavill, 2008). e-procurement systems implemented in korea and mexico demonstrate how the innovative use of information technology to prevent and control corruption in public procurement (oecd, 2005). the nepalese e-government master plan was developed in 2006 with the main objective of a more efficient, productive, transparent, and responsible government that provides value-added quality service through ict. the nepalese government has progressively developed and implement it-related policy, strategy, and legal framework including information technology policy 2000, telecommunication policy 2004 and electronic transaction act 2007 (pariyar, 2007). the nepalese government’s public procurement act 2007 is one of the prime pieces of legislation designed to reform government procurement systems to make them more transparent, allowing for accountable government, fair, competitive, and efficient public procurement processes by ensuring equality of opportunity for the bidders to participate in public procurement processes without any discrimination (nepal public procurement strategic framework, 2010). corruption in public procurement has been widespread around the world and is more serious and at a more alarming stage in developing countries (iqbal & seo, 2008; kumar et al.,2007). poverty and low civil service salaries can motivate public officials to earn extra income via corruption (bannon,1999). other important factors that have driven corruption in public procurement include political instability, lack of transparency and accountability, monitoring and auditing of government works and services, low level of professionalism of the bureaucracy, and a weak separation between the civil society and politics (del monte & papagni, 2007; kolstad & wiig, 2009; subedi, 2006). corruption in public procurement is a serious problem in nepal, and due to the use of traditional paper-based purchasing systems, there is a potential to avoid the rule of law for securing contracts (bhattarai, 2011). there are instances in nepal where contracts and tenders are awarded because of an abuse of power and political interference. , government officers are also involved indirectly and use their official power for their own private benefit. members of political parties are also involved and have manipulated public procurement process for their own ends. in the “sudan darfur scam” the special court (sc) convicted three former police chiefs and two suppliers of embezzlement involving the purchase of an armoured personnel carrier (apc) for the nepalese peacekeeping mission in the darfur region of sudan (sharma, 2012). ict tools have the potential to assist countries like nepal to combat the problem presented by systemic corruption. theoretical approach theories provide guidelines that govern research and provide a structure to the concepts and relationships between constructs that collectively present logical, systematic, and coherent explanation of phenomenon of the interest with some assumption (bacharach, 1989). this study has used agency theory that is also known as principal-agent model that refers to the relationship between two parties: the principal and the agent who makes the decision or takes any action on the behalf of the principal australasian journal of information systems volume 18 number 2 2014 26 (eisenhardt, 1989; jensen & meckling, 1976). the agency theory is considered one of the most important contributions in the fields of organisational economics, business management, and information systems research (basu & lederer, 2011; gefen & carmel, 2008; oh,gallivan & kim, 2006). dawson, watson and boudreau (2011, p 1) suggest that “agency theory has long been a stalwart of is research and is one of the most commonly used lenses to study the relationship between a principal and an agent”. agency theory can be useful in the contractual and social relationship between consultant and clients. it refers to the relationship between two parties: a principal and the agent where the one party (principal) delegates work to the other party (agent) and the agent takes any action on the behalf of the principal (eisenhardt, 1989; jensen & meckling, 1976). originally, this concept was developed to assist the risk sharing between individuals and groups (jensen & meckling, 1976). this theory was applied in different agency relationships including buyers and sellers (waterman & meier, 1998), government and citizens (singh et al., 2010), public institutions and bureaucracy (lambsdorff, 2001). in this study the government is the principal body that provides public services to the people and agents are the bidders (contractor, supplier) who work for the government and are providers of goods and services. the main role of principal or government authority is its responsibility for formulating policy options to help in the regulation and development of the public procurement, the monitoring of the various procurement activities, the regulation and maintenance of standards of procurement, capacitybuilding and professional development, information management and dissemination. agents’ roles are to bid for the government work and services, comply with all statutory, legal and award requirements relative to the work and services, complete all task within agreed cost structures, maintain quality and also complete all tasks within the designated time frame. in this regard, the main contribution of principal-agent theory is to explain the risk of corruption in public procurement processes between two parties principal and agent and examine contracting problems to determine the most efficient contract type that will satisfactorily govern the agency relationship (whipple & roh, 2010). principalagent theory helps explain how the major issue in managing agency relationships resides in ensuring that the agent acts in the interests of the principal. the theory helps understand the risk of corruption in government procurement through considering: reduction of monopoly power, information asymmetry, and transparency and accountability within the public procurement processes as they affect the relationship between government officials and bidders (neupane, soar & vaidya, 2014). all of these factors are expected to have a relationship (either positive or negative) with a dependent variable, which is the intent-to-adopt e-procurement. development of hypothesis and research model in the context of the public procurement process, a government official plays an important role in the provision of goods and services and is the key person to provide a goods and services to the public in a transparent way. they may assign a public contract or tenders to favoured contractors or bidders. in some situations, “government officers have monopoly power over the provision of goods and services that are crucial for explaining the incidence of corruption without theft”(neupane, soar & vaidya, 2012b). klitgaard (1988) pointed out that “corruption tends to emerge when an organization or a public official has monopoly power over a good or service that generates rent, has the discretionary power to decide who will receive it, and is not accountable”. this is more common in developing countries in relation to the awarding of public contracts where many tenders are assigned to favoured contractors or bidders for corrupt reasons. to avoid this issue, public e-procurement can be a great tool to reduce the monopoly on power held by government officials. e-procurement can help to reduce the risk of corruption by providing benefits such as real time information, fixed price contracts, automation and auditing controls, efficiency, and reduced accounting errors. all bidders can view tender documents australasian journal of information systems volume 18 number 2 2014 27 through a public portal that provides greater transparency. they may be able to access real project information such project cost, and time. the following hypothesis summarises this relationship. h1: higher level of perception of the potential of public e-procurement to reduce monopoly power is positively related to the willingness to adopt. information asymmetry (ia) is a core component of the principal-agent theory (amagoh, 2009, p 6) and reducing information asymmetry problems is another key factor of public electronic government (song, 2002). the relationship of principal to agent exists during the procurement processes between government tender administrators and bidders (finkle, 2005; larbi, 2006). information asymmetry arises when the agent has more information than the principal, or vice versa, in the contract between government and bidders (amagoh, 2009). these kinds of information gaps take place due to incomplete information, incompleteness of contract, problems with monitoring mechanisms, and the cost of configuration of the projects in contracting process (finkle ,2005; gauld, 2007; taylor, 2005). in this regard, the perceived benefits of public e-procurement (see appendix a) can be considered to reduce information asymmetry problems as well as to reduce the chances of corruption in public procurement. the present study examines this relationship: h2: higher level of perception of the potential of public e-procurement technology to reduce information asymmetry is positively related to the willingness to adopt. public e-procurement has a range of potential benefits for both process and transparency in public procurement, schapper (2008, p 115) stated that “procurement of goods, works, and services through internet-based information technologies (e-procurement) is emerging worldwide with the potential to reform processes, improve market access, and promote integrity in public procurement”. transparency and openness are core principles of public procurement and transparency is widely recognised as a foundation of good governance (parigi et al., 2004). lack of transparency creates opportunities for the public officials such as government officers and politicians to abuse their positions of authority for private gain. this indicates a weak accountability mechanism that tends to facilitate corruption. corruption flourishes due to a lack of transparency and accountability in public goods and services. as a result, corrupt bureaucrats realise that they can take advantage of regulation, so they generate layers of government regulation governing which in turn then runs the risk of making processes over-complex and vague and avoid detection. accountability refers to the “service guarantee” of a govern¬ment; the extent to which its actions are accounted for and corrected if not carried out correctly in the first instance. technically, accountability can be improved fairly and easily. information can be published online, processes and decisions can be traced for audit and analysis, and there can be rules for compensation where accountability is not delivered. the following hypothesis summarises this relationship. australasian journal of information systems volume 18 number 2 2014 28 h3: higher level of perception of the potential of public e-procurement technology to increase transparency and accountability is positively related to the willingness to adopt. figure 1: research model of this study the focus of this research is on the management of transparency and accountability for which eprocurement technology can be an enabler. figure 1 presents the conceptual model of this study consisting of the hypothesis defined above. this model shows three independent variables as they relate to intent-to-adopt e-procurement technology as the dependent variable. each dependent variable has different items that are shown in the appendix. the most important perceived anti-corruption factors of public e-procurement technology include: real time access to information, automation of procurement processes, tracking and monitoring, consistency in procurement, competition among bidders, transparency, efficiency, quality, and accountability in public procurement. similar kinds of research models were presented by neupane et al. (2012a, p 6) as part of an evaluation of the anticorruption capabilities of public e-procurement in developing country. yusoff & islam’s (2011) study into electronic government procurement adoption behaviour amongst malaysian smes supported this finding. van der heijden,verhagen and creemers (2003, p 44) presented a research model that furthers our understanding of online purchase intentions: contributions from technology and trust perspective. in the context of developing countries, the implementation of public e-procurement at the government level is in its infancy and its success and future prospects are not yet assured. some developed countries such as singapore, denmark, japan, new zealand, and australia have experienced success with public procurement resulting in greater levels of transparency. other examples include the republic of korea and andhra pradesh (india) that have generated significant benefits to the government through eprocurement (bhatnagar & singh, 2010; chang, 2011). research methodology this study employed the positivist paradigm to examine the intent-to-adopt public e-procurement technology in government, and to explore perceptions of the potential of public e-procurement technology to reduce corruption in government procurement. in research context, positivist philosophy involves the hypothesis testing to obtain the objective truth and predict what may happen at the future date as well (greener, 2008). the positivist philosophy has number of implication for social science research including methodological, value-freedom, causality, operationalization, independence, and reductionism (bond, 1993; easterby-smith,thorpe & lowe, 1997; hughes, 1994). the population for this study was drawn from the government departments of nepal and was conducted in the capital city of kathmandu. most of the government departments’ head offices are located in this region. the participants were invited from seven nepalese government departments including the department of roads (dor), nepal electric authority, department of urban development and monopoly power information asymmetry transparency and accountability intent to adopt eprocurement technology h1 h2 h3 australasian journal of information systems volume 18 number 2 2014 29 building construction, department of local infrastructure development of agricultural roads, roads board of nepal, public procurement monitoring office, and ministry of irrigation, department of water induced disaster prevention. the main reason for selecting these organisations for this study was that they have already implemented, and are currently using public e-procurement systems. the participants’ contact details were gathered from the government of nepal directory of government officials (http://nepalgov.gov.np). the researcher then took steps to confirm that the participants were indeed implementing and using public e-procurement systems. the public procurement monitoring office (ppmo) is a guiding body for procurement policy formulation, implementation, and monitoring procurement system in nepal. the main mission of ppmo is to maintain transparency, accountability, effectiveness, efficiency, and as well as nondiscrimination and equality in public procurement processes in nepal. the ppmo installed (http://gepson.gov.np) as a single web portal system for all public entities of government procurement of nepal. the ppmo has begun the process of creating a centralised web portal. some of the government departments are also using the dor web portal (http://eproc.dor.gov.np/) for procurement. dor was the first organisation to introduce public e-procurement for submission and opening of bids in nepal in 2008. research participants included government officers who were involved in procurement and administration sections including joint secretary, director, senior divisional engineer, computer officer, it manager, and training officer. the study used a convenience sampling approach as determined by the level of interest shown by the participants to respond to the survey and finally 46 government officers were approached to complete questionnaire survey. the items used in the field survey are listed in appendix a. survey research has several inherent strengths compared to other research methods because it measures a wide range of unobservable data such as peoples’ attitudes, beliefs, factual information and behaviours (bhattacherjee, 2012, p. 73). in this research, data was collected through face-to-face field surveys. firstly, the researcher discussed and asked questions of the participant about public e-procurement. secondly, a questionnaire was conducted to collect the data related to the measuring of the perceived benefits of public e-procurement in the context of reducing corruption. the study explored three independent variables: monopoly on power, information asymmetry, transparency and accountability. each construct has different scale measurement item, and those items were measured by asking the respondents perceptions on e-procurement benefits and its application. all the variables are coded on a five-point likert scale ranging from “strongly disagree” to “strongly agree”. data analysis the collected data was analysed quantitatively using partial least squares (pls) smartpls 2.0. pls path modelling (pls-pm) or pls structural equation modeling (pls-sem) is a method for data analysis and has been used in various fields of computer science, marketing, accounting, management and psychology (ismail,hamid & idris, 2012). pls-pm has been considered as a very flexible and soft modelling approach to multi block analysis by the means of both hierarchical pls path model and more as exploratory approach than a confirmatory one (vinzi,trinchera & amato, 2010). hanafi (2007) pointed out pls-pm is a statically different approach from covariance structure analysis for path analysis with latent variables, interpretation of the basic entities of model, type, and parameters. pls can be used when the sample size is relatively small (chin, 1998a; henseler,ringle & sinkovics, 2009; lehner & haas, 2010; wetzels,odekerken-schroder & van oppen, 2009). likewise, pls is used to predict complex relationships in a large model with multiple independent variables and multiple dependent variables (abdi, 2003). pls aids the prediction of dependent endogenous variables through australasian journal of information systems volume 18 number 2 2014 30 latent variable scores that can be used to predict models (chin 1998a; henseler et al., 2009), and supports continuous moderation without losing information (chin, 1998a). the reliability of cross loading value of the measurement items is illustrated in appendix b. loading should exceed over 0.70 to be considered (carmines & zeller, 1979). other authors suggest that the cross loading value above 0.40 or 0.50 is appropriate (hulland, 1999). in this study, based on the plspm results (appendix b), all the items had a range from 0.481 to 0.924. lew and sinkovics (2012, p 23) suggested that a score of outer loading over 0.5 can be acceptable and chin (1998a) stated 0.40 can be acceptable. as a result, the factor loading met the recommended threshold value 0.4 (chin, 1998a; henseler et al., 2009). table 2 presents the results of reliability of the construct, including average variance extracted (ave), composite reliability, and cronbach’s alpha. all the constructs demonstrate the composite reliability (cr) and cronbach’s alpha value is higher than 0.70 that is greater than the threshold value 0.7. george & mallery (2003, p. 231) provided the range of cronbach’s alpha values as “>.9-excellent, >.8-good, >.7-acceptable, >.6-questionnable, >.5-poor”. the cr values of construct ranged from 0.81 to 0.90, all greater than the threshold value of 0.7 (bagozzi & yi 1988). henseler et al. (2009) suggested that the ave value of above 0.50 indicates the sufficient level of convergent validity. our results also reveal ave value is greater than 5.0 except information asymmetry that is also close to 0.5. variables ave composite reliability cronbach’s alpha information asymmetry (ia) 0.477 0.810 0.713 monopoly of power (mp) 0.557 0.880 0.833 transparency and accountability (ta) 0.624 0.890 0.849 intent to adopt e-procurement (ita) 0.544 0.903 0.881 table 2: reliability and validity of constructs discriminant validity is another type of construct validity used in pls path modelling and table 3 demonstrates the square root of ave for each construct were greater than the highest correlation between that the construct and the other constructs (fornell & larcker, 1981; gefen & straub, 2005). the diagonal elements highlighted in bold (square root of ave), are greater than the off-diagonal elements. variables ia mp ta ita ia 1.000 mp 0.864 1.000 ta 0.790 0.794 1.000 ita 0.633 0.669 0.448 1.000 table 3: latent construct correlation table with squared of ave values the exploratory power of the pls structural model can be evaluated by examining the amount of variation in the dependent variables that can be explained by the model figure 2. information asymmetry was found to be a most important driver for intent-to-adopt public e-procurement, with a australasian journal of information systems volume 18 number 2 2014 31 path coefficient equal to 0.459. other important factors included monopoly power and transparency and accountability path with coefficient values of 0.328, 0.231 respectively. figure 2: pls structural equation model the r2 value (figure 2) is 0.812 that is interpreted as 81.20 percent of the variation in the level of intent-to-adopt e-procurement significantly explained by the pls model. pls-pm is a component based estimation method (chin, 1998a; vinzi et al., 2010) and it uses an iterative algorithm to obtain latent variables estimates through the system of multiple regression or simple linear regression (aibinu & allawati, 2010). this study uses partial least square (pls) regression for obtaining cross loading, path coefficient, discriminant validity, r2 value, and weight value for the latent variables of the measurement items. the below equation presents the pls regression model of the equation: intent-toadopt e-procurement (ita) = 0.328*mp+0.459*ia +0.231*ta. in pls-pm structural equation modelling, the bootstrapping method was used to test the significant of paths in the study measurement model. the bootstrap is an alternative way to produce better approximations for the true small sample properties (schmidheiny & basel, 2012). bootstrap techniques are used for testing hypothesis (beaumont & bocci, 2009) and chin (1998a)undertook bootstrapping with 500 samples. hair, ringle & sarstedt (2011) considered that critical t-values for two-tailed test need to be 1.65 (significance level =10 percent), 1.96 (significance level =5 percent), and 2.58 (significance level=1 percent). table 5 demonstrated the standardised path coefficient results and hypothesis test. the result found that all the independent variables are positively significant to the dependent variables intent-to-adopt e-procurement (mp β=0.328 t-value=2.767 p 0.01; ia β=0.459 tvalue=3.374 p 0.01; ta β=0.231 t-value=2.492 p 0.05). table 4 shows the standardised path coefficient results and hypothesis test. all the independents variables were found to be positive and significantly strong correlated with the dependent variables intent-to-adopt public e-procurement. the path coefficient value needs to be least 0.20 and ideally above 0.30 in order to considered as a strong relationship and meaningful for discussion (chin, 1998b). australasian journal of information systems volume 18 number 2 2014 32 hypothesis path coefficient value t-value sig. (twotailed supported? h1: monopoly power--->intent to adopt 0.328 2.767 sig. at 1% yes h2: information asymmetry--->intent to adopt 0.459 3.374 sig. at 1% yes h3: transparency and accountability-->intent to adopt 0.231 2.492 sig. at 5% yes table 4: path coefficient (intent to adopt e-procurement) this relationship suggests that higher levels of adopting public e-procurement technology in government levels lowers the risk of the exercise of monopoly power by government officers in government procurement, minimising the information gap between government and its bidders in contracting process, and increasing higher level of transparency and accountability in public procurement. these outcomes ultimately will help to reduce the chances of corruption. discussion this study set out to research the perceptions of government officers on the potential of e-procurement for reducing corruption. the study contributes to the existing literature on the perceived benefits of public e-procurement by combining the different constructs into a single research model and by direct link to intent-to-adopt e-procurement. the principal-agent theory applied in this research helps to evaluate the technology benefits including transparent and real time information, automation of transaction process, fixed price contract, track, and monitor, fairness of competition, and in reducing the chances of corruption in public procurement. the results suggest that monopoly power, information asymmetry, and transparency and accountability are antecedent attitudes towards intent-to-adopt public e-procurement. all the latent variables were positively significant to the intent-to-adopt public e-procurement. our results indicate that a positive and strong relationship (β=0.328) between reduction of monopoly power and intent to adopt public eprocurement which is similar to the research electronic government production behaviour observed amongst malaysian smes conducted by yusoff & islam (2011). monopoly power of officials is one of the critical factors bearing upon opportunities for corruption (klitgaard, 1988). public e-procurement can reduce the power of government officers in procurement processes. information asymmetry and intent to adopt e-procurement were found to be correlated with each other (β=0.459). information asymmetry is a critical factor in principal-agent relationship that leads to create an information gap and a problem for monitoring between two parties. singh & sirdeshmukh (2000) argued that asymmetrical information increases the probability of opportunistic behaviour. wathne and heide (2000) found that information asymmetry is a facilitating condition for opportunism. xinzhang & yonggang (2011) suggested that public procurement application could enhance transparency, accountability, economic performance, and efficiency and mitigate the asymmetric information by increasing accessing information. this study believed found that public e-procurement is seen to reduce the information gap (lowering information asymmetry problem) between government and bidders. a lower level of information asymmetry directly decreases opportunities of corruption in public procurement. australasian journal of information systems volume 18 number 2 2014 33 our results indicate that transparency and accountability was positively significant (β=0.231) to the intent to adopt e-procurement. both transparency and accountability are two important pillars for the government and bidders to reduce corruption. kolstad and wiig (2009) claimed that transparency can reduce the political and bureaucratic corruption as well as facilitate cooperation over opportunistic rentseeking, maintain norms of integrity, and trust. , public e-procurement can facilitate real time access to information, automated procurement procedures, more consistency in bidding procedures, and importantly can reduce the human intervention in bidding processes. our research found that these anticorruption factors lead to reducing the chances of corruption. filho & mota (2012) and found that the main contribution of e-procurement to reforming government procurement system and enhanced efficiency and quality of government services, as well as enhance transparency and accountability (brun et al., 2010; wen & wei, 2007). the finding also supports previous research by neupane et al. (2012a) that highlighted the anticorruption capabilities of public e-procurement in developing countries. pathak et al. (2008) suggests that electronic government initiatives improved government-citizen relationship and reduce corruption. carayannis and popescu (2005) found that information technology provides an opportunity to make procurement activities more transparent and accountable. kim, kim and lee (2009) demonstrated that electronic government has made a positive impact upon improving transparency in the government sector as well as helping to combat corruption. our findings are in line with recent research conducted by vaidya & hyde (2011) who evaluated the benefits of public e-procurement in light of the diffusion of innovation theory. the limitations of our research include that it was conducted in only one country and only with government officials. future research will need to work on cross-sectional country analysis, adding more variables, and employing rigorous research methodologies interview, focus group discussion about the better understanding of perceived benefits of e-procurement to reduce corruption in public procurement. more research is need to identify the anti-corruption factors of perceived benefits of technologies in reducing corruption in government procurement and it also need to be strong empirical test to valid the factors that influence the adoption of e-procurement systems. the results of this study should be considered in the context of governments’ public e-procurement and anti-corruption policies. this has been supported by concha et al. (2012) who observe that e-procurement has profoundly transformed the governments’ procurement process exceeding the policy makers’ expectations. the result of this study has several interesting contributions for both procurement practitioners and government agency focus on performance of public e-procurement for reducing corruption in public procurement. based on the above discussion, the study has suggested some practical recommendations: government can encourage e-procurement as an anti-corruption reform in public procurement this study results suggest that the emerging countries’ governments can promote public e-procurement as an anti-corruption tool to fight against corruption. countries can encourage the awareness of transparency and accountability as key instruments tools for promoting integrity and preventing fraud and corruption in public procurement. government can do the excellent job through the e-procurement web portal of creating transparent and accountable government works and services. it can help to provide publically available and open procurement related information in a web e.g. forms, contracts, bids, and awards. however, political will and efforts are necessary to implement the public eprocurement in the country. governments should give priority for e-procurement in budgeting and ministerial decisions. prioritise and promote public e-procurement as an anti-corruption agenda governments should promote public e-procurement as an anti-corruption tool and promote awareness of transparency and accountability as key instruments for reducing fraud and corruption in public australasian journal of information systems volume 18 number 2 2014 34 procurement. governments should give priority for e-procurement in budgeting and ministerial decisions. use public e-procurement initiative to drive administrative reforms a review and consolidation of the existing legislative framework is necessary for government administrative reforms so that it can give a clear transformation and wider support procurement issues. governments need to develop policies to support adopting public e-procurement and launch awareness raising education and training, seminar to the companies including suppliers, contractor, and government agencies. having recognised that procurement policies help governments to work, services to be more transparent, and reduce opportunities for corruption in public procurement. incorporate the anti-corruption capabilities of public e-procurement in the technical/design phase governments should identify the key measures of anti-corruption in public procurement and incorporate the anti-corruption capabilities in the technical design of pubic e-procurement. there is a need to develop an evaluation strategy to measure progress that defines what to measure and how to measure. develop and implementation of public e-procurement strategy in planning level: governments should make action plans for implementing public e-procurement and establish a steering committee of executives whose credentials are impeccable to develop a public e-procurement implementation strategy. the main mission of the strategy should be designed to centralise eprocurement system, which makes to increase monitoring capabilities, transparency and accountability, easy access to information, and make consistency to all bidders and government agencies. as a result, it helps to combat corruption. conclusion this study has some either academic or theoretical and practitioner implications for government and international agency, and procurement practitioner of developing and transitioning countries to make decisions about e-procurement system. academically, this paper has developed a research model based on the existing principal-agent theory as well as public procurement research. this model can help to explain the relationship among the anti-corruption factors with intent-to-adopt public e-procurement. at a glance, the model provided reveals the opportunities to access the impacts of public e-procurement perceived benefits likely to reduce the chances of corruption in public procurement. from the point of view of the practitioner, it provides for better understanding of the potential of public e-procurement; as well as impetus to promote and prioritise public e-procurement as an anti-corruption agenda and the strength and relevance of use of technologies to procurement reform. this paper identifies the three significant latent variables that could influence the willingness to adopt e-procurement at all levels of government on the perception of the government officers. it demonstrates the interesting feature of the potential of public e-procurement in reducing corruption in public procurement processes. it also reveals the higher level of intent-to-adopt public e-procurement has a positive relationship with the concept of reduction of monopoly power and information asymmetry, and transparency and accountability that contributes to the reduced chances of corruption in public procurement. the research objective was to identify the most important anti-corruption factors of public e-procurement on the perception of the government officers in relation to intent-to-adopt eprocurement. australasian journal of information systems volume 18 number 2 2014 35 references abdi, h. (2003) “partial least squares regression (pls-regression)”, in m lewis-beck, bryman,a., & futing, t. (ed.), encyclopedia of social science research methods, thousand oaks, ca: sage, 792-5. achterstraat, p. (2011) compliance review report procurement reform of e-procurement and etendering, the legislative council parliament house, sydney nsw 2000, http://www.audit.nsw.gov.au/articledocuments/190/03_vol_1_2011_compliance_review.pd f.aspx?embed=y accessed 5 january 2012 adb (2010) (asian development bank) technical assistance report, republic of indonesia: streangthing national public procurement process, manila. ag (2005) (australian government) case studies on e-procurement implementations italy, new south wales, new zealand scotland, western australia, attorney-general’s department,canberra, australia, http://www.finance.gov.au/publications/e-procurement-researchreports/docs/case_studies_on_e-procurement_implementations.pdf accessed 10 october 2011. aibinu, a.a. & al-lawati, am (2010) “using pls-sem technique to model construction organizations' willingness to participate in e-bidding”, automation in construction, 19(6), 71424. amagoh, f. (2009) “information asymmetry and the contracting out process”, the innovation journal: the public sector innovation journal, 14(2), 1-14. andersen, t.b. (2008) e'government as an anti'corruption tool, department of economics, university of copenhagen, studiestrêde 6, dk-1455, copenhagen k, denmark. auriol, e. (2006) “corruption in procurement and public purchase”, international journal of industrial organization, 24(5), 867-85. bacharach, s.b. (1989) “organizational theories: some criteria for evaluation”, academy of management review, 14(4), 496-515. bagozzi, r.p. & yi, y. (1988) “on the evaluation of structural equation models”, journal of the academy of marketing science, 16(1), 74-94. bannon, l. (1999) the fight against corruption : a world bank perspective, http://www.iadb.org/regions/re2/consultative_group/groups/transparency_workshop6.htm accessed 16 july 2013. basu, s. (2004) “e government and developing countries: an overview”, international review of law, computers & technology, 18(1), 109-32. basu, v. & lederer, a. (2011) “agency theory and consultant management in enterprise resource planning systems implementation”, sigmis database, 42(3), 10-33. beaumont, j-f. & bocci, c. (2009) “a practical bootstrap method for testing hypotheses from survey data”, survey methodology, 35(1) 25-35. bertot, j., jaeger, p. & grimes, j. (2010) “using icts to create a culture of transparency: e-government and social media as openness and anti-corruption tools for societies”, government information quarterly, 27(3) 264-71. bhatnagar, s.c. & singh, n. (2010) “assessing the impact of e-government: a study of projects in india”, information technologies & international development, 6(2) 109-27. bhattacherjee, a. (2012), social science research: principles, methods, and practices, 2nd edition edn, open access textbooks, book 3, http://scholarcommons.usf.edu/oa_textbooks/3/. australasian journal of information systems volume 18 number 2 2014 36 bhattarai, p., (2011) curbing procurement corruption', voices against corruption, http://voicesagainst-corruption.ning.com/profiles/blogs/curbing-procurement-corruption accessed 10 june 2013. bond, s. (1993) “experimental research nursing: necessary but not sufficient”, in a kitson (ed.), nursing, art and science, chapman and hall, london. brandon-jones, a. & carey, s. (2011) “the impact of user-perceived e-procurement quality on system and contract compliance”, international journal of operations & production management, 31(3) 274-96. braund, p., frauscher, k., schwittay, a. & petkoski, d. (2007) information and communications technology for economic development exploring possibilities for multi-sector technology collaborations, global e-discussion, the world bank. brun, a., ronchi, s., fan, x. & golini, r. (2010) “what is the value of an it e-procurement system?”, journal of purchasing and supply management, 16 131-40. burton, r. (2005) “improving integrity in public procurement: the role of transparency and accountability”, in fighting corruption and promoting integrity in public procurement, oecd publishing, 23–8. carayannis, e.g. & popescu, d. (2005) “profiling a methodology for economic growth and convergence: learning from the eu e-procurement experience for central and eastern european countries”, technovation, 25(1) 1-14. carmines, e.g. & zeller, r.a. (1979) “reliability and validity assessment”, sage university paper series on quantitative applications in the social sciences, 17, sage publications, sage beverly hills, ca. chang, h.l., wang, k. & chiu, i. (2008) “business-it fit in e-procurement systems: evidence from high-technology firms in china”, information systems journal, 18(4) 381-404. chang, k-s. (2011) enhancing transparency through e-procurement, public procurement service (pps) the republic of korea, http://www.oecd.org/dataoecd/47/30/49311011.pdf accessed 12 february 2012. chin, w.w. (1998a) “the partial least squares approach for structural equation modeling”, in ga marcoulides (ed.), modern methods for business research lawrence erlbaum associates publishers, new jersey, pp. 295-336. chin, w.w. (1998b) “issues and opinion on structural equation modeling”, misq., 22(1) 7-16. concha, g., astudillo, h., porrúa, m. & pimenta, c. (2012) “e-government procurement observatory, maturity model and early measurements”, government information quarterly, 29, s43-s50. credé, a., mansell, r. & mansell, r.e. (1998) knowledge societies--in a nutshell: information technology for sustainable development, idrc, ottawa, canada. croom, s. & brandon-jones, a. (2007) “impact of e-procurement: experiences from implementation in the uk public sector”, journal of purchasing and supply management, 13(4) 294-303. croom, s.r. & brandon-jones, a. (2005) “key issues in e-procurement : procurement implementation and operation in the public sector”, journal of public procurement, 5(3) 367-87. cuervo-cazurra, a. & genc, m. (2008) “transforming disadvantages into advantages: developingcountry mnes in the least developed countries”, journal of international business studies, 39(6) 957-79. davila, a., gupta, m. & palmer, r. (2003) “moving procurement systems to the internet::: the adoption and use of e-procurement technology models”, european management journal, 21(1) 11-23. australasian journal of information systems volume 18 number 2 2014 37 dawson, g, watson, r. & boudreau, m-c. (2011) “an empirical test of the theory of relationship constraints”, paper presented to thirty second international conference on infromation systems, shanghai del monte, a. & papagni, e. (2007) “the determinants of corruption in italy: regional panel data analysis”, european journal of political economy, 23(2) 379-96. easterby-smith, m., thorpe, r. & lowe, a. (1997) management research: an introduction, second edn, sage, london. eei, k.s., husain, w. & mustaffa, n. (2012) “survey on benefits and barriers of e-procurement: malaysian smes perspective”, international journal on advanced science, engineering and information technology, 2(6) 14-9. eisenhardt, k. (1989) “agency theory: an assessment and review”, academy of management review, 14(1) 57-74. fan, y., zhang, z. & yue, q. (2009) “e-government, transparency and anti-corruption”, paper presented to 2009 international conference on management of e-commerce and e-government, 16-19 september, nanchang, china. filho, j.r. & mota, f.p.b. (2012) “public e-procurement implementation: insights from the structuration theory”, in k vaidya (ed.), inter-organizational infromation systems and business management : theories for researcher, igi global, hershey, pa, usa, p. 259. finkle, a. (2005) “relying on information acquired by a principal”, international journal of industrial organization, 23(3-4) 263-78. fornell, c. & larcker, d.f. (1981) “evaluating structural equation models with unobservable variables and measurement error”, journal of marketing research, 18(1) 39-50. gauld, r. (2007) “principal-agent theory and organizational change”, policy studies, 28(1) 17-18. gefen, d. & straub, d. (2005) “a practical guide to factorial validity using pls-graph: tutorial and annotated example”, communications of the association for information systems, 16(1) 109. gefen, d. & carmel, e. (2008) “is the world really flat? a look at offshoring in an online programming marketplace”, misq, 32(2) 367. george, d. & mallery, p. (2003) spss for windows step by step: a simple guide and reference, 11.0 update, boston, ma: allyn & bacon. greener, s. (2008) business research methods, ventus publising aps, london, united kingdom. grönlund, å. (2010) “using ict to combat corruption-tools result and methods”, increasing transparency & fighting corruption through ict,stockholm university,stockholm sweeden. gtn (2003) global trade negotitation,s centre for international development at harvard university: government procurement”, http://www.cid.harvard.edu/cidtrade/issues/govpro.html accessed 14 october 2011 gunasekaran, a. & ngai, e.w.t. (2008) “adoption of e-procurement in hong kong: an empirical research”, international journal of production economics, 113(1) 159-75. gunasekaran, a., mcgaughey, r.e., ngai, e.w.t. & rai, b.k. (2009) “e-procurement adoption in the southcoast smes”, international journal of production economics, 122(1) 161-75. hair, j.f., ringle, c.m. & sarstedt, m. (2011) “pls-sem: indeed a silver bullet”, the journal of marketing theory and practice, 19(2) 139-52. hanafi, m. (2007) “pls path modelling: computation of latent variables with the estimation mode b”, computational statistics, 22(2, ) 275-92. hanna, n.k. (2010) transforming government and building the information society: challenges and opportunities for the developing world, springer verlag. australasian journal of information systems volume 18 number 2 2014 38 henriksen, h. & andersen, k. (2003) “e-procurement adoption: theory and practice”, in jh g. goos, and j. van leeuwen (ed.), electronic government, springer berlin 2739, 121-4. henriksen, h.z. & mahnke, v. (2005) “e-procurement adoption in the danish public sector”, scandinavian journal of information systems, 17(2) 5-26. henseler, j., ringle, c.m. & sinkovics, r.r. (2009) “the use of partial least squares path modeling in international marketing”, advances in international marketing, 20(1) 277-319. ho, cf, tai, y.m., wu, w.h. & jou, j.j. (2008) “exploring the impacts of web-based e-procurement on organizational performance”, pacis 2008 proceedings, p. 106. hughes, j. (1994) the philosophy of social research, essex, logman. hulland, j. (1999) “use of partial least squares (pls) in strategic management research: a review of four recent studies”, strategic management journal, 20(2) 195-204. icac (2011) corruption risks in nsw government procurement: suppliers perceptions of corruption, independent commission against corruption, sydney, nsw, australia. imf (2010) maldives: public financial management—performance report, international monetary fund 10/137 washington, d.c. usa. iqbal, m. & seo, j. (2008) “e-governance as an anti corruption tool: korean cases”, korean local information, 11(2) 51-78. ismail, irb, hamid, rba & idris, f. (2012) “pls application in journal of operations management: a review”, paper presented to global conference on operations and supply management (gcom 2012), bandung, indonesia, 12-13 march 2012. jensen, m.c. & meckling, w.h. (1976) “theory of the firm: managerial behavior, agency costs and ownership structure”, journal of financial economics, 3(4) 305-60. kaliannan, m., raman, m. & dorasamy, m. (2009) “e-procurement adoption in the malaysian public sector: organizational perspectives”, paper presented to enterprise distributed object computing conference workshops, auckland, 1-4 september 2009. kashap, s. (2004) “public procurement as a social, economic and political policy”, paper presented to international public procurement proceedings, florida usa, october 21-24, 2004. khanapuri, v.b., nayak, s., soni, p., sharma, s. & soni, m. (2011) “framework to overcome challenges of implementation of e-procurement in indian context”, paper presented to international conference on technology and business management, dubai, march 28-30, 2011. kim, s., kim, h.j. & lee, h. (2009) “an institutional analysis of an e-government system for anticorruption: the case of open”, government information quarterly, 26(1) 42-50. klitgaard, r. (1988) controlling corruption, university of california press, usa, berkeley. kolstad, i. & wiig, a. (2009) “is transparency the key to reducing corruption in resource-rich countries?”, world development, 37(3) 521-32. krishnan, s., teo, t.s. & lim, v.k. (2013) “examining the relationships among e-government maturity, corruption, economic prosperity and environmental degradation: a cross-country analysis”, information & management, 50(8) 638-49. kumar, v., mukerji, b., butt, i. & persaud, a. (2007) “factors for successful e-government adoption: a conceptual framework”, the electronic journal of e-government, 5(1) 63–76. lambsdorff, j. (2001) how corruption in government affects public welfare: a review of theories, zentrum für globalisierung und europäisierung der wirtschaft, georg-august-univ. larbi, g. (2006) “new public management as a template for reforms in low-income countries: issues and lessons from ghana”, international journal of organization theory and behavior, 9(3, p. 378. australasian journal of information systems volume 18 number 2 2014 39 lee, j., oh, k-t. & kwon, h.y. (2008) “striving for transparency and efficiency in e-government: procurement reform through e-procurement”, acm, cairo, egypt, 978-1-60558-386-0. lehner, f. & haas, n. (2010) “knowledge management success factors–proposal of an empirical research”, electronic journal of knowledge management, 8(1) 79-90. lew, y.k. & sinkovics, r.r. (2012) “crossing borders and industry sectors: behavioral governance in strategic alliances and product innovation for competitive advantage”, long range planning, 46(1-2) 13-38. lindskog, h., brege, s. & brehmer, p.o. (2010) “corruption in public procurement and private sector purchasing”, organisational transformation and social change, 7(2) 167-88. lio, m-c, liu, m-c & ou, y-p (2011) “can the internet reduce corruption? a cross-country study based on dynamic panel data models”, government information quarterly, 28(1) 47-53. magrini, p. (2006) “transparency in e-procurement: the italian perspective”, 1st high level seminar on e-procurement 30 –31 january 2006 naples, italy, http://www.oecd.org/dataoecd/57/31/36238443.pdf accessed 10 february 2012 mahmood, sai (2010) “public procurement and corruption in bangladesh confronting the challenges and opportunities”, journal of public administration and policy research, 2(6) 103-11. mistry, j.j. & jalal, a. (2012) “an empirical analysis of the relationship between e-government and corruption”, the international journal of digital accounting research, 12(18) 145-76. mitchell, k. (2000) “instituting e-procurement in the public sector”, public management (pm), 82(11) 21-5. ndou, v. (2004) “e-government for developing countries: opportunities and challenges”, the electronic journal of information systems in developing countries, 18(1) 1-24. nepal, (2010) nepal public procurement strategic framework, ppmo government of nepal office of the prime minister and council of ministers, kathamndu. neupane, a., soar, j. & vaidya, k. (2012a) “evaluating the anti-corruption capabilities of public eprocurement in a developing country”, the electronic journal on information system in developing countries, 55(2) 1-17. neupane, a., soar, j. & vaidya, k. (2012b) “the potential of e-procurement technology for reducing corruption”, international journal of information technology and management, 11(4) 273-87. neupane, a., soar, j. & vaidya, k. (2014) “anti-corruption capabilities of public e-procurement technologies: principal-agent theory”, in kj bwalya (ed.), technology development and platform enhancements for successful global e-government design, information science reference, hershey, pa. united states) 185-203. neupane, a., vaidya, k. soar, j. & young, j. (2012) “role of public e-procurement technology to reduce corruption in government procurement”, paper presented to 5th international public procurement conference (ippc5), seattle, united states, 17-19 august 2012. oecd (2005) fighting corruption and promoting integrity in public procurement, organisation for economic co-operation and development,paris. oecd (2007) integrity in public procurement: good practice from a to z, organisation for economic co-operation and development, paris. oecd (2011) government at a glance 2011 country note: finland, oecd, http://www.oecd.org/dataoecd/60/25/47876433.pdf accessed 24 january 2012. oecd, a. (2008) “fighting bribery in public procurement in asia-pacific”. http://www.etransform.org/gti/sites/etransform.org/files/documents/bribery.pdf accessed 21 july 2011. australasian journal of information systems volume 18 number 2 2014 40 oh, w., gallivan, m.j. & kim, j.w. (2006) “the market's perception of the transactional risks of information technology outsourcing announcements”, journal of management information systems, 22(4) 271-303. oye, n. (2013) “reducing corruption in african developing countries: the relevance of e governance”, greener journal of social sciences, 3(1) 006-13. panayiotou, n.a., gayialis, s.p. & tatsiopoulos, i.p. (2004) “an e-procurement system for governmental purchasing”, international journal of production economics, 90(1) 79-102. panda, p., sahu, g. & gupta, p. (2010) “promoting transparency and efficiency in public procurement: e-procurement initiatives by government of india”, paper presented to 7th international conference on e-procurement (iceg) 2010, 20-24 apr 2010, iim banglore, india. parigi, v, leader, w, geeta, p & kailasam, r 2004, ushering in transparency for good governance, centre for goodgovernance, hyderabad, 30 december 2012, <http://www.cgg.gov.in/workingpapers/ushering_in_transparency.pdf>. pariyar, m.p. (2007) “e-government initiatives in nepal: challenges and opportunities”, paper presented to proceedings of the 1st international conference on theory and practice of electronic governance, macao, china, 10-13 december 2007. pathak, r., singh, g., belwal, r., naz, r. & smith, r. (2008) “'e-governance, corruption and public service delivery: a comparative study of fiji and ethiopia”, joaag, 3(1) 65-79. pathak, r.d., naz, r., rahman, m.h., smith, r.f.i. & agarwal, k.n. (2009) “'e-governance to cut corruption in public service delivery: a case study of fiji”, intl journal of public administration, 32(5) 415-37. pellegrini, l. & gerlagh, r. (2008) “causes of corruption: a survey of cross-country analyses and extended results”, economics of governance, 9(3) 245-63. raymond, j. (2008) “benchmarking in public procurement”, benchmarking: an international journal, 15(6) 782-93. ronchi, s., brun, a., golini, r. & fan, x. (2010) “what is the value of an it e-procurement system?”, journal of purchasing & supply management, 16(2) 131-40. schapper, p. (2008) “the impact of e-procurement on corruption: the potential of e-procurement for curbing corruption risks”, paper presented to 7th regional seminar on making international anti-corruption standards operational, bali, indonesia, november 5-7 2007. schmidheiny, k. & basel, u. (2012) “the bootstrap”, short guides to microeconometrics. universität basel., 26, p. 2012. shakya, r.k. (2012) “procurement governance framework: success to e-government procurement (e-gp) system implementation”, paper presented to 5th international public procurement conference (ippc5), seattle, united states, 17-19 august 2012. sharma, b. (2012) “sudan scam: 5 convicted, 31 get clean chit”, the kathmandu post, 13 february 2012. shim, d.c. & eom, t.h. (2009) “anticorruption effects of information communication and technology (ict) and social capital”, international review of administrative sciences, 75(1, p. 99. singh, g., pathak, r., naz, r. & belwal, r. (2010) “e-governance for improved public sector service delivery in india, ethiopia and fiji”, international journal of public sector management, 23(3) 254-75. singh, j. & sirdeshmukh, d. (2000) “agency and trust mechanisms in consumer satisfaction and loyalty judgments”, journal of the academy of marketing science, 28(1) 150-67. australasian journal of information systems volume 18 number 2 2014 41 sohail, m. & cavill, s. (2008) “accountability to prevent corruption in construction projects”, journal of construction engineering and management, 134(9) 729-38. song, h.j. (2002) “prospects and limitations of the e-government initiative in korea”, international review of public administration, 7(2) 45-53. subedi, m.s. (2006) “corruption in nepal: an anthropological inquiry”, dhaulagiri journal of sociology and anthropology, 1(0) 110-28. taylor, p. (2005) “do public sector contract catering tender procedures result in an auction for “lemons”?”, international journal of public sector management, 18(6) 484-97. teo, t.s.h., lin, s. & lai, k-h. (2009) “adopters and non-adopters of e-procurement in singapore: an empirical study”, omega, 37(5) 972-87. thai, k.v. (2001) “public procurement re-examined”, journal of public procurement, 1(1) 9-50. un (2006) e-procurement: economic and social commission for asia and the pacific asian development bank institute public procurement service of the republic of korea, united nations publication, thailand. undp (2006) capacity development practice note, <http://capacity.undp.org/index.cfm?modulelibrary&page=document&dovumentid=5599 accessed 15 july 2012. unodc (2004) the global programme against corruption: un anti-corruption toolkit, 3rd edn, unodc vienna. vaidya, k. (2007) electronic procurement in the australian public sector: the organizational assimilation process and its impact on public procurement performance, university of new england. vaidya, k. & hyde, m. (2011) “inter-organisational information systems assimilation: an empirical evaluation in light of the diffusion of innovation theory”, international journal of business information systems, 7(3) 247-68. vaidya, k., sajeev, a. & callender, g. (2006) “critical factors that influence e-procurement implementation success in the public sector”, journal of public procurement, 6(1/2, p. 70. vaidya, k, callender, g.c. & sajeev, a.s.m. (2009) “facilitators of public e-procurement: lessons learned from the u.k., u.s., and australian initiatives”, in kv thai (ed.), international handbook of public procurement, auerbach publications taylor & francis group, 473-90. vaidya, k., callender, g, sajeev, a & gao, j 2004, 'towards a model for measuring the performance of e-procurement initiatives in the australian public sector: a balanced scorecard approach', paper presented to a paper prepared for the australian electronic governance conference. centre for public policy, university of melbourne, melbourne victoria, 14th and 15thapril, 2004. vaidyanathan, g. & devaraj, s. (2008) “the role of quality in e-procurement performance: an empirical analysis”, journal of operations management, 26(3) 407-25. van der heijden, h., verhagen, t. & creemers, m. (2003) “understanding online purchase intentions: contributions from technology and trust perspectives”, european journal of information systems, 12(1) 41-8. vinzi, v.e., trinchera, l & amato, s. (2010) “pls path modeling: from foundations to recent developments and open issues for model assessment and improvement”, handbook of partial least squares concepts, methods and applications, springer berlin heidelberg. ware, g.t., moss, s., campos, j.e. & noone, g.p. (2012) “'corruption in procurement”, in a graycar & r smith (eds), handbook of global research and practice in corruption, illustrated edn, edward elgar publishing, uk. australasian journal of information systems volume 18 number 2 2014 42 waterman, r.w. & meier, k.j. (1998) “principal-agent models: an expansion?”, journal of public administration research and theory, 8(2, p. 173. wathne, k.h. & heide, j.b. (2000) “opportunism in interfirm relationships: forms, outcomes, and solutions”, the journal of marketing, 64(4) 36-51. wb (2004) the cost of corruption, news & broadcast, the world bank ,washington, dc, usa, http://web.worldbank.org/wbsite/external/news/0,,contentmdk:20035595~menupk: 36691~pagepk:116743~pipk:36693~thesitepk:4607,00.html accessed 5 october 2012. wen, w. & wei, l. (2007) “decision-making analysis of e-procurement with the rough set theory”, paper presented to international conference on wireless communications, networking and mobile computing, shanghai, 21-25 september 2007. wescott, c.g. (2001) “e government in the asia pacific region”, asian journal of political science, 9(2) 1-24. wetzels, m., odekerken-schroder, g. & van oppen, c. (2009) “using pls path modeling for assessing hierarchical construct models: guidelines and empirical illustration”, misq, 33(1, p. 177. whipple, j.m. & roh, j. (2010) “agency theory and quality fade in buyer-supplier relationships”, international journal of logistics management, 21(3) 338-52. xinzhang, c. & yonggang, w. (2011) “e-government, government procurement and the development of e-commerce: korean experience and its implications”, paper presented to international conference on e -business and e -government (icee), shanghai, china, 6-8 may 2011. yusoff, w.s. & islam, m. (2011) “electronic government procurement adoption behavior amongst malaysian smes”, international business research, 4(1) 100. zhang, h. & yang, j. (2011) “'research on application of e-tender in china”, paper presented to internet technology and applications (itap), 2011 international conference on, 16-18 aug. 2011. zhang, j. & zhang, z. (2009) “applying e-government information system for anti-corruption strategy”, paper presented to management of e-commerce and e-government, 16-19 september, nanchang, china. appendix a questionnaire: a five point likert scales was used to measure the perception of the government officers about the evaluation of public e-procurement technology to reduce corruption in public procurement. reducing monopoly power through e-procurement technology mp1 e-procurement technology provides our department auditing capabilities with real time procurement information. mp2 e-procurement technology reduces monopolies or selection of favoured bidders (e.g. suppliers / vendors) in contracting process. mp3: the potential of e-procurement technology improves internal efficiency across procurement process (e.g. tender notice time, reduce paper files, advertisement time), which reduces the monopoly power of government officers in tendering process. mp4: e-procurement technology facilitate accounting control it reduces the likelihood of fraud or accounting errors by enabling the electronic matching of requisition, purchase orders, invoice, and receipts. australasian journal of information systems volume 18 number 2 2014 43 mp5: e-procurement technology helps to provide fix price contract to all buyers and sellers (e.g. equal tendering price for all bidders, price shall fixed throughout the contract). reducing information asymmetry through e-procurement technology ia1: e-procurement technology helps to more competition in terms of quantity (participation) and quality (openness and fairness) among our bidders in tendering process. ia2: our department has frequently up-to-date procurement information (e.g. tender notice) posted in our procurement web portal. ia3: e-procurement helps to track and monitor all the procurement related activities such as tracking bidders’ information, monitoring all procurement processes etc. ia4: e-procurement technology provides consistency in procurement process (same rule and regulation to all bidders in tendering process) to all registered bidders. ia5: e-procurement technology reduces discretion (e.g. limit financial delegation) in contracting process. increasing transparency and accountability through e-procurement technology ta1: e-procurement technology contributes us to enhance more transparent and accountable in procurement process. ta2: e-procurement technology helps to reduce human errors (e.g. tender checklists) as well as increase accuracy of orders in procurement process. ta3: e-procurement makes it more convenient to acquire information. ta4: all government procurement policies, processes, and guidelines are available online (eprocurement portal). ta5: e-procurement provides the better relationship between government and suppliers. ta6: e-procurement brings an increase in transmission of timely public information on contract awards on price, volume, and execution of time. ta7: e-procurement increases the availability of public information on bids/offers and other processes. ta8: e-procurement can increase accuracy of orders. intent to adopt anti-corruption technology (e-procurement) ita1: our organization would like to intent-to-adopt e-procurement system for our organization needs. ita2: our organization is using e-procurement technology for handling our procurement-processing task (e.g. tendering, contract awards etc.) ita3: e-procurement helps our organization to coordinate with other bidders partners for government work and services. ita4: our organization adopts e-procurement system to enhance procurement efficiency and for better decision making in government work and services. ita5: our organizations adopt e-procurement system to avoid the human interference in public tendering process. ita6: overall, our organization intent to adopt e-procurement system to increase transparency, for non-discrimination, equality of access, and open competition in government work and services. australasian journal of information systems volume 18 number 2 2014 44 appendix b cross loading table variables monopoly of power (mp) information asymmetry (ia) transparency and accountability (ta) intent to adopt e-procurement (ita) mp1 0.773 mp2 0.869 mp3 0.597 mp4 0.944 mp5 0.764 ia1 0.578 ia2 0.924 ia3 0.481 ia4 0.821 ia5 0.543 ta1 0.718 ta2 0.849 ta3 0.847 ta4 0.527 ta5 0.789 ta6 0.859 ta7 0.605 ta8 0.634 ita1 0.486 ita2 0.703 ita3 0.891 ita4 0.813 ita5 0.823 ita6 0.697 ajis vol. 6 no. 2 may 1999 the origin of the year 2000 date problem: an alternative hypothesis richard kingsford22 parliamentary information systems office parliament of a ustraua leone dunn1 school of information technology and computer science university of wollongong, australia abstract this paper aims to identify the primary origin of the year 2000 date problem, in which many computer-based systems may fail because they cannot correctly interpret two-digit year dates which lie in and after the year 2000. the paper proposes a hypothesis within a cultural anthropological framework that the fundamental origin of the problem lay in a pre-existing, commonplace norm of western culture the pervasive use of two-digit year dates. this cultural practice set western society on a collision course with the technology of the future. the cultural norm underpinned and influenced the actions taken within the it industry from the 1960s onwards, which laid the physical foundation for the problem. keywords year 2000, y2k, century, millennium, information technology, information systems, human-computer interaction, culture, society, anthropology introduction the impact which technology can have on society has been widely recognised (drucker 1970; kranzberg and davenport 1972). however it has also been demonstrated that social and cultural factors may have a formative or shaping influence on technology (bijker et al 1987; bijker 1995; mackenzie and wajcman 1987). this latter insight underpins the hypothesis proposed in this paper. the social factors may be ones which pervade a whole society, or more localised aspects. in this context "technology" is taken to include associated human actions and knowledge, and does not simply refer to "hardware". the year 2000 date problem (or "y2k bug", century problem or "millennium bug") refers to a serious issue which many nations of the world have faced in recent years. many computer-based information systems, and devices containing embedded computing systems, may fail because they are unable to correctly interpret twodigit year dates which lie in and beyond the year 2000. the immediate cause of the problem is that the developers of such systems used two-digit dates in their systems, instead of full four-digit year dates. this means that if in a system "10/05/01" is intended to mean "10 may 2001", the system is likely to interpret it as something different, often "10 may 1901". this is likely to cause the system to fail, often with uncertain consequences (e.g. de jager 1993, 1997, 1999; de jager and bergeon 1997; gartner 1997; jones 1998a; schick 1995; ulrich and hayes 1997; yourdon and yourdon 1998). the problem was probably first encountered by the banking and insurance industries from the 1960s onwards, when some systems were required to handle dates 30 years or more into the future (de jager 1998b; ibm 1996; keogh 1997). many such companies therefore ensured that such systems could handle four-digit dates. however it has been estimated that year 2000 compliant systems amounted to around only one in twenty of all systems (jones 1998b). embedded systems refer to devices which contain microprocessors or microcontrollers which contain hard-wired instructions. these devices are installed in diverse equipment including vehicles, aircraft, alarm systems, factory machines and robots. many such devices utilise two-digit dates, and there are estimated to be many millions world-wide which may experience problems (bylinsky 1998; gartner 1997; greif 1997). it has been estimated that the australian government will need to spend $600 million to correct "missioncritical" systems alone, while the total bill may be $3 billion (davis 1998). the full cost of the problem in australia has been estimated at $5 to 10 billion (australian 1998; financial review 1998; binning 1998). the us government cost is likely to be $6.4 billion however some estimates place it at $10 to 12 billion (luening 1999). the us defense department alone may spend up to $4 billion (burnett 1998). large us businesses may 22 this paper is a revised version of a technical report written by the authors (kingsford and dunn 1998). it is not based in any way on the authors' current places of employment. 88 ajis vol. 6 no. 2 may 1999 spend up to $50 billion in all. it has been variously estimated that the world-wide cost will be sus200 to 800 billion (dci 1999; luening 1999; strassman 1998). various possible outcomes have been forecast, ranging from inconvenience to more serious aftermaths (ulrich and hayes 1997; yourdon and yourdon 1998). in one recent us survey, 93 per cent of responding organisations said that if they did nothing, they would be damaged (luening 1998a). businesses and governments are interdependent, and it has been suggested that year 2000 problems may have adverse repercussions. the legal ramifications have also been canvassed (australian 1998; dunn 1998). many of the systems developed since the 1960s use two-digit dates, but some very recent it work does also. some it schools continued using two-digit dates in teaching into the 1990s (morgan stanley 1997). recent client/server systems are generally better than earlier systems, but are not immune (ghosh 1997). in late 1998 it was estimated that 64 per cent of some 5000 commercial software packages had year 2000 problems (gifford 1998a). one practitioner found that much java code on the internet was not compliant (amon 1998a). of around 250 million personal computers (pcs) in use, many are not year 2000 compliant, and even many new ones have problems (australian 1998; feilder 1997; ulrich and hayes 1997). in late 1998 it was estimated by a consultant that 11 per cent of new pcs would fail on year 2000 hardware tests, while all would fail the british standard test for operating systems (gifford 1998a). in 1991 a large consulting firm installed a non-compliant system for a us retail chain. the design "explicitly set out a two-digit year dating format", but both parties were satisfied at the time (luening 1998c; overby 1998). the authors examined systems developed in 1997 which were not compliant. when asked, the developer of one explained the use of two-digit dates by saying, "they just looked alright, and i didn't really think about them." some programmers have even re-coded two-digit dates back into systems which were already compliant (de jager 1998a). a pertinent question is why the managers in such cases did not enforce the use of four digits. in summary, the issue has been seen as "a dust-like problem in which two-digit dates have been blown into every nook and cranny of technological societies" (dunn 1998). to achieve accuracy and quality, system development and it work in general requires close attention to detail. how should we explain the widespread neglect of dates? as only around one in twenty developed systems were year 2000 compliant, were about 95 per cent of it practitioners incompetent or negligent? this has been suggested by some, and some negligence may of course have occurred (keogh 1997). some have gone further and implied that the it profession may, or should be held responsible in some way (hayes 1997; ratcliffe 1998). for example: "while the all-pervasive nature of the problem has become apparent, there is still a widespread belief that the blame for the whole mess should be sheeted back to the it industry" (connors 1998). it has been suggested that it professionals may ultimately pay a price: "but once the crisis is over, all the anger now building up will be released" (m. vitale, quoted in connors 1998). the hypothesis presented in this paper suggests that such inferences are unwarranted. cultural hypothesis this paper proposes the hypothesis that the primary origin of the year 2000 problem lay in a commonplace cultural norm which has existed in many western nations since the late nineteenth century or earlier, namely the widespread use of two-digit dates, especially in written form. this norm has underlain all subsequent humancomputer interaction, in its broadest sense. this hypothesis draws on anthropology in terms of identifying culture as a set of widely shared values, behaviours, beliefs and norms (e.g. harris 1975; spradley and mccurdy 1975). cultural traits are deep-seated and usually long-established, and culture has a powerful influence on the actions of its members. the elements of a culture are often strongly influenced by its history and the shared experiences of its members. conversely, the culture of a group is often a formative influence on events which contribute to its history. this paper refers to the aggregated societies of western nations as "western society", and the set of common elements of the cultures of western nations as "western culture". some may think of western culture in terms of great art, architecture, classical music, opera and so on. however to anthropologists culture is just as much about unobtrusive, widely shared elements, which can strongly influence people's everyday behaviour because they are commonplace. such aspects may be undocumented and even rarely articulated, but they nevertheless seem natural and familiar to members of the culture. every time actions are influenced by such a cultural norm, this process also reinforces the legitimacy of the norm. this paper identifies the widespread use of two-digit dates as a common cultural practice in many countries mainly western which use the gregorian calendar. the origin of the practice is beyond the scope of this paper. the convention influenced all people concerned, including analysts, programmers, engineers and users. common european and australian practice is to abbreviate dates in 89 ajis vol. 6 no. 2 may 1999 the form "dd/mm/yy" or similar. a common us practice is to abbreviate them as "mm/dd/yy", while canada appears to use both forms. in each case members of the respective societies readily make the inferential leap to the full date with unconscious ease. two-digit dates are as much a part of human behaviour as of technology. there are few direct studies of behaviour in the it workplaces of the 1960s and 1970s, and few if any studies in any period of past behaviour and attitudes regarding two-digit dates. this implies that any real understanding of such behaviour must now rely largely on the recollections of people who were there. the cultural hypothesis is based on published evidence, on the authors' own experience in the it industry, and on discussions with veterans of it from the 1960s onwards. qualitative and anecdotal evidence are relevant and valuable in this case. however the year 2000 problem received extensive publicity in the late 1990s, often implying that past actions were at least short-sighted (hill 1998; ibm 1996). this has led to some tendency to rationalise and justify what are now widely seen as past mistakes. it work in general is analytical, logical and precise, and many find it hard to understand how something now seen as misguided could have occurred so widely hence there is a strong emphasis on explanations within a "rational" genre such as storage limitations. this issue is examined further below. outside the realm of computing, two-digit dates have been used extensively since at least the late nineteenth century. some suggest even earlier: "since at least the early 1800's the culture had standardized on the two digit year" (carmichael 1998). the prospectors in the california gold rush of 1849 were commonly referred to as "forty-niners" (van steenwyk 1991). an australian magazine commented in 1898, "let's see, this year's '98; and next year's '99; and the year after '00?" (bulletin 1898). the account records of a small australian business from the late 1890s were viewed by the authors. the records were written by hand with two-digit dates, and they progressed smoothly from 98, 99 to 00 obviously the owner had no problem in making the inferential leap to 1898, 1899 and 1900. hollerith punched cards were also in common use from late last century until the 1970s, and often recorded two-digit dates (austrian 1982; morgan stanley 1997). this helped minimise storage to some extent, but it also reflected common practice. later collections of business and personal letters, records of meetings, cheques, paper forms and manual date stamps were viewed, and again there was a definite tendency over many decades to use two-digit dates (see also ulrich and hayes 1997). more recent documents sometimes reflect the fact that word processors can generate four-digit dates although most still offer two-digit dates as options even now. as one example, a prisoner in world war two in java received a signed handkerchief from fellow prisoners, bearing the date they were taken away by the japanese this is now in the australian war memorial (begbie 1998) (see figure 1). jan ruff-0'herne's handkerchief. awm rel26396 figure 1: handkerchief, world war two (begbie 1998) 90 ajis vol. 6 no. 2 may 1999 some significant but brief suggestions have been made which imply support for the cultural hypothesis. for example: "computer software generally stores dates as two-digit numbers (for no better reason than social convention)" (p. bailes, quoted in mathieson 1998). there are similar comments, although these often go on to nominate other principal causes of the year 2000 problem. for example, it has been noted that two-digit dates were already a shorthand habit for our society in the 1950s (ulrich and hayes 1997). one suggestion is that "people think and work in two digit dates. computers think the way people work" (microsoft spokesperson, quoted in luening 1998b). another comment is that "in our society, people automatically represent years in two digits" (microsoft 1998). one source suggests that "human nature seems to dictate that writing and keyboard entry of two digits is more practical than four digits" (ibm 1996). the reference to human nature serves to underline how deep-seated the practice is but it is a component of western culture, not human nature. it is human nature for people to communicate. it is also commonplace to use shared abbreviations, but just how this is done is an aspect of culture. for example, abbreviations such as "dr" and "mr" are common, and are often also embedded in computer systems. it is sometimes implied that the convention of two-digit dates actually stemmed from it decisions to use them in earlier years (keogh 1997; microsoft 1998; n. weiss-dolev, in radio national 1997). in reality it was surely the other way around. main explanations that have been suggested various suggestions have been put forward to explain the use and continued existence of two-digit dates in computer systems (e.g. gerner 1997). the main ones focus on the demands of users, the need to conserve storage, the lack of an official date standard, the focus on core priorities, short-term planning, the demands of system maintenance, the high mobility of it staff, the impact of end user computing, the force of precedent and denial by managers. these are discussed further below. some suggestions acknowledge that over the last 40 years various inter-related technological and business decisions and actions were taken which combined to physically create the problem. it can be demonstrated that these actions were not inevitable, and that in many cases there were feasible alternatives entailing the use of four-digit dates. in most instances these options were not taken up. in all aspects of human-computer interaction the cultural norm of two-digit dates was a common underlying thread, which essentially implied, "if it seems alright, it is alright" and it influenced behaviour. user demand user demand has been suggested as a key causative factor in the year 2000 problem (gerner 1997; swaine 1998). on this argument, two-digit dates were specified by customers to reflect their everyday practice. while there is some validity in this, the organisation of work in the early years of business computing is also relevant (see also haselkom 1998). this was the period of development of many large systems which survived for decades as "legacy" systems, and which used two-digit dates. in the 1960s and early 1970s it was a different world in many ways. mainframe computers dominated, and it was primarily larger organisations which developed software. in the experience of the authors and their informants, in many such organisations there was typically a marked role division between systems analysts and programmers. the analysts interacted with business users and, acting on behalf of the users, wrote specifications for programmers to follow. the analysts specified the two-digit dates, often in explicit detail in file and report layouts. programming was then a specialised technical craft, and analysts quite often did not fully understand the internal working of the computer. the analysts' skills were typically oriented mainly to the business of the organisation, while technical it knowledge was left to the programmers. it has been suggested that the it industry had "no adult supervision", and that this contributed to the year 2000 problem (l. kappelman, quoted in hayes and bullers 1998). however systems were not developed by programmers alone. while it is true that programmers were often aged around 18 to 25 years, the analysts and user managers were more usually aged 25 to 50 years. analysts generally had higher relative status than programmers, who often simply did as instructed (see also c. jones, in foremski 1998). the it work environment of the early period was typically quite hierarchically structured often more so than in later years. there are indications that some it practitioners did raise early concerns about the century risk, but were generally treated with disbelief (bemer 1979; morgan stanley 1997). being removed from users and management, programmers had little influence. one college it student brought up the problem in class in 1974, but was greeted with "stunned silence then... incredulous laughter" by colleagues, even though they understood it (guy 1998). anthropologists recognise this as a common reaction of people confronted by a concept outside their normal frame of reference. an it practitioner developed and attempted to market a windowing tool to address the problem in 1983, but was "repeatedly shown the door, or laughed at" (foremski 1998). the year 91 ajis vol. 6 no. 2 may 1999 2000 issue at the time was "acknowledged occasionally in trade conference jokes and dp shop banter butattracted little serious attention" (gillin 1984). on this interpretation, the two-digit dates were specified mainly by those who did not fully understand their implications. those who could have better understood the issue generally lacked decision-making power. some members of the latter group did occasionally voice concerns, but this was relatively rare implying that two-digit dates generally appeared acceptable or at least innocuous to most of these people also. from the 1970s the separate roles of analysts and programmers began to merge in many it shops into the profession of analyst/programmer, narrowing the gap between user and developer. however the use of two-digit dates continued. a significant factor was the underlying cultural norm which influenced both perceptions and behaviour. storage limitations the need to save on storage has been widely publicised as an explanation and is an oft-repeated mantra, amounting almost to "received wisdom". this proposition assumes that from the 1960s business computing was driven by the storage limitations of punched cards, paper tape, memory and magnetic tape and disk (e.g. de jager 1993, 1997; gerner 1997; ibm 1996; jones 1998a; keogh 1997; microsoft 1998; ulrich and hayes 1997). the need to conserve storage steadily declined with hardware advances from the 1970s onwards (e. yourdon, in hayes and bullers 1998). however the use of two-digit dates did not abate. there was no official date format standard in the 1960s and 1970s (gerner 1997), and no overall coordinated strategy for date storage. given the extensive spread of it in western countries, this implies that a very large number of independent decisions were taken to store two-digit dates. in fact a wide variety of date formats have been observed, but many share the common factor of two-digit years. it is now not possible to directly establish the reasons underlying most decisions to use such dates. it is probable that most were not taken with a specific aim of conserving storage (see also haselkorn 1998). the it veterans consulted invariably said they did not think much about the issue at the time, sometimes adding that the dates were specified by others anyway. none said that they had been aware of any conscious decision to use two-digit dates for the explicit purpose of conserving storage. but most it practitioners also failed to clearly draw attention to potential problems. there are factors indicating that this situation was widespread, and these are discussed below. punched cards have been in use since the late nineteenth century (heide 1997). it has been suggested that as cards are of fixed size, there was little option but to use two-digit dates, and that this practice was carried over into computer systems (chorafas 1998; de jager 1997). was this the full story? a typical business application would require the recording of a reference number, one or two dates, a name and address of 60 characters or more, and further details. the non-date information typically overwhelmed the dates in space occupied. economising on dates occasionally prevented a record overflowing a card, but rarely saved much overall and in any case was not done when there was a business need for four-digits. however there was a cultural norm which favoured two-digit dates, and this was an underlying driver. even when cards carrying pre-set two-digit dates were used as input to a computer system, programmers still had the option to implement simple windowing logic to interpret them in context as four-digit dates, replicating human reasoning (moore and foley 1998). (the term "windowing" is used here to mean the process of interpreting a two-digit date as a four-digit date within a window of 100 years which may span two centuries regardless of how the date is subsequently stored.) for example, in 1965 a birth date of "10/05/90" would be interpreted as 10 may 1890, while "10/05/40" would be 10 may 1940. any ambiguities such as "10/05/60" would generate error reports to be clarified later (this was common practice for other purposes). but programmers only rarely coded such logic as they, like everyone else, were influenced by the prevailing cultural norm. in the early period of it, card records were sometimes copied intact to tape or disk as an initial step. the systems would often then process these "card images" instead of actual cards. this was done mainly to reduce the time taken for a deck to pass through the card reader. however it may also have tended to reinforce the apparent legitimacy of using two-digit dates in magnetic storage. from the 1960s direct key-to-tape and key-todisk systems began to displace punched cards for data entry, and became the precursors of later on-line systems. this tends to diminish any argument that there was no alternative to two-digit dates because cards were "unstretchable". even when two-digit dates were keyed in, programmers had the same option to use windowing, but rarely did. it has been suggested that two-digit dates were used to save keying effort by operators (de jager 1999; keogh 1997). however dates typically comprised relatively little of the keying workload, and this is also a tenuous argument when applied retrospectively to a period prior to widespread awareness of repetitive strain injury. in fact the operators usually entered two-digit dates directly from forms designed by analysts or users, originating in the business realm. the cultural norm underlay such decisions. 92 ajis vol. 6 no. 2 may 1999 the use of two-digit dates in magnetic storage was not inevitable. again, the non-date information typically overwhelmed the dates in terms of space. memory was very limited, but generally only relatively few records were held in memory at any one time. most systems afflicted by the year 2000 problem stored two-digit dates on tape, disk or drum as six text characters, such as "100568" for 10 may 1968. a preliminary ieee statement commented "it is wrong to believe that the year 2000 date problem [resulted] simply from a rational but shortsighted decision to save memory and storage space by using two digit years rather than four. the problem is far more complex... much more could have been saved by boiling these dates down to binary" (haselkorn 1998). there are various ways of compressing four-digit dates in magnetic storage (ibm 1996; moore and foley 1998). in the 1960s and 1970s, most it shops employed programmers skilled in assembly language, even when highlevel languages were in use such as cobol (common business-oriented language). assembler and similar low-level languages were widely used to manipulate data at the binary digit (or bit) level. if saving on date storage had been a conscious aim, it would have been a trivial exercise to write an assembler routine to compress four-digit dates into bits occupying the same or less space than two-digit dates expressed in characters. this was confirmed by former assembler programmers consulted, who frequently faced far more difficult problems. a typical example (only) of a simple compressed date format is: range bits required stored in day i t o s l 5 bits 8 bits month 1 to 12 4 bits 8 bits year 1 to 9999 14 bits 16 bits 32 bits total that is, in a byte-addressable computer similar to an ibm machine, a four-digit date could be compressed into four bytes (32 bits) or less. (the example above would fail within 8000 years.) similar binary compression schemes could be implemented in other computers, depending on their basic architecture. various other techniques were also possible, such as the use of julian dates, that is, a count of days forward from a fixed starting point, with the result stored as a binary integer (moore and foley 1998). such techniques could maintain human readability by expanding all output dates. it has been suggested that the computation time needed to compress and expand dates would have significantly extended batch processing times (ted 1998). however it is unlikely this would have been excessive, and it would be about the same as the inbuilt date compression routines which some cobol compilers offered, but which were also rarely used in practice. the simple answer is, if binary date compression had been recognised as a business need, the batch runs would have been adapted to it just as they accommodated the much slower overall processing rate of the period. a date compression routine could be used both in programs written in assembler, and also in high-level languages. cobol had the ability to invoke assembler routines from within a program and this was commonly done for other purposes, such as calculating a "checksum" used to help confirm that records were written and read correctly. some cobol compilers had only limited date formats, but "date" formats as such were never essential for date processing many programmers created their own (see also haselkorn 1998). it can also be noted that whenever date calculations were needed, a date in characters had to be converted to a numeric form in any case, and this took time again diminishing any argument that date conversion would have been too timeconsuming. within the "rational" school, storage limitations are seen as a key causative factor: "a problem of rational origin that requires a rational solution" (kappelman and cappel 1996). it has been suggested that the problem stemmed from it professionals applying "sound business judgement" in a period of limited and expensive storage: "it is estimated that conserving precious disk space with the two-digit year format saved the typical organization over $1,000,000 per gigabyte of total storage in the 30 year period from 1963-1992 (in 1992 dollars)... to mitigate the potential response of anger from top management, is managers need to emphasize the rational origin of the problem" (kappelman and cappel 1996). however studies within this genre beg the question of motive and intent. if such savings were achieved deliberately, why were equal or greater economies not achieved through date compression which would have simultaneously prevented the year 2000 problem? there is evidence that conscious decisions to use two-digit dates with the aim of conserving storage were relatively rare, and if so, calculations of imputed monetary savings have less relevance. the occasional cases of the use of one-digit year dates underline the fact that storage was limited. some us defence organisations and car manufacturers used such systems in the 1970s, mostly redeveloped from systems using two-digit dates first written in the 1960s (gates 1998; gerner 1997). such decisions were therefore taken against a baseline of the use of two-digit dates as a "given", and this influenced thinking. these systems no doubt included forward planning applications which stored dates some years ahead. problems inevitably soon emerged when dates in the 1980s were encountered. at that time, systems were not as widely interconnected as they were later, and correction was simpler. however these instances could be seen as demonstrating negligence, 93 ajis vol. 6 no. 2 may 1999 in that conscious decisions were made to use one-digit dates while knowing that problems would shortly arise. if the use of one-digit dates had in fact been widespread, such negligence would also have been widespread and the accusations of incompetence directed at the it industry would be justified. however on the basis of published reports, actual instances of the use of one-digit dates were relatively limited. the techniques and skills were available to use binary date compression from the earliest years of it. however such techniques were rarely used, and mostly only when there was a known business need for four-digit dates, as in some banks. the unix operating system was first developed in 1969, and it was also designed to use compressed four-digit julian dates (bach 1986). however some unix system utilities and services have date problems. application systems developed to run on unix, or migrated to unix, are also often not immune (inman 1998). the basic question is not so much why storage needed to be conserved this is unarguable. the more pertinent question is why two-digit dates were so widely and thoroughly accepted without question as a valid abbreviation, which tended to ensure that binary date compression was rarely considered. this argument does not deny the possibility that in the early decades of it some practitioners perhaps many noticed the use of two-digit dates and, in a prevailing climate of storage minimisation, assumed that the dates were an aspect of this practice. however this does not logically imply that deliberate decisions were made to use two-digit dates for the explicit purpose of economising on storage. such decisions were rarely made. it was primarily the underlying cultural norm which informed the decisions to use two digits, and this also helped legitimise the actions taken by the parties involved. it is acknowledged that the informants for this paper represent only a very small minority of the millions of it practitioners whose work entailed date storage. however if it could be demonstrated that many of these people consciously used two-digit dates with the explicit aim of saving storage, the it profession could then legitimately be asked to explain why it did not widely adopt binary date compression, which would have both saved on storage and "future proofed" systems against the year 2000 problem. lack of official date standard the lack of any widespread official date standard has been suggested as a factor in the year 2000 problem. there has been little consistency among date formats used, apart from the common factor of two-digit years. some localised standards have existed, such as a us government standard which in 1968 recommended the use of twodigit dates. in 1971 this standard advised, but did not mandate the use of four-digit dates (landers 1998). by 1988 the standard still only "highly recommended" them (peckinpaugh 1999). there was no international fourdigit date standard until 1988 (iso 8601) and this was rarely mandated or widely accepted anyway (gerner 1997; haselkorn 1998; jones 1998a). this was partly related to the fact that there were differences in informal date formats among countries, for example common practice in europe and australia is day/month/year, whereas one common north american practice is month/day/year. neither of these match the "official" standard, which is year/month/day. this reflects the difficulty of attempting to impose a standard which conflicts with pre-existing conventions. the limited adoption of the standard was also compounded by the fact that the it industry has historically had difficulty in agreeing on any standards (landers 1998). this largely stemmed from a widespread adoption of minimum standardisation wherever possible by it vendors, to gain commercial advantage. this entailed the deliberate use of proprietary standards, aimed at "locking in" customers to a vendor. for example, ibm, a major player in the period 1960 to 1980, wrote "basic principles" for itself in 1968 which included: "implementation of industry standards is a business decision based upon market considerations. standards are "minimums" do not preclude added features nor product different from the standards" (delamarter 1986). this led to a general acceptance among it practitioners of the prevailing lack of uniformity. in summary, while it is true that if there had been a mandatory four-digit standard the year 2000 problem might not have emerged, this in itself does not explain the widespread adoption of two-digit dates. in the absence of official standards, everyday conventions took over. focus on core priorities a focus on "core priorities", to the relative exclusion of other issues, has been suggested as a factor (gartner 1997; gerner 1997). this is related to a focus on productivity and profit (or in the public sector, cost saving). in fact this general approach was applied in all professions over the history of it. thus analysts, designers and programmers concentrated on the issues they saw as most important to achieve their respective goals. these usually related to getting the accounts correct, or the refinery operating and so on and not to how the dates were stored. for programmers, the main focus was often simply on getting the program to run. it is sometimes suggested that the use of two-digit dates saved effort for programmers. one it consultant wrote, "however... i 94 ajis vol. 6 no. 2 may 1999 have found, compliant code would have been just as easy, if not easier, to produce. this confirms to me that most programmers, like most of the population, haven't given this matter any thought at all" (amon 1998a). another consultant said, "people in the software industry have not paid attention to this problem. it's an issue that people have completely overlooked" (quoted in overby 1998). the authors of it textbooks also did not mention the issue until very recent years (foremski 1998). the underlying cultural norm shifted the focus away from two-digit dates. in only relatively few cases a focus on core business did in fact entail attention to dates and the use of four-digit dates. short-term planning short-term planning is a related issue, and this has been common practice in both business and it (gartner 1997). it has been suggested that in the 1960s and 1970s there was a view that the use of two-digit dates would not be a problem for 20 years or more (de jager 1997; keogh 1997). however there have always been some systems which stored future dates sometimes far distant. it has also been suggested that systems were then not expected to last for 20 years or more, and therefore their use of two-digit dates did not matter (de jager 1999; gerner 1997; ulrich and hayes 1997). this may have been so in some cases, but often no such assumptions were made. one programmer wrote, "the year 2000 was 20 years away, and we programmers were in unspoken agreement that our programs would be well and truly replaced..." (armstrong 1999). but this suggests a post hoc rationalisation. even if any "unspoken agreement" did in fact exist, the writer could not know how widely it was shared, if at all. what did "replacement" really mean? it was expected that the hardware would be replaced, but not necessarily the application software and especially the data (ulrich and hayes 1997). from the earliest years many data holdings often containing two-digit dates were retained relatively intact while both hardware and application code were updated. for example, in the 1960s many assembler programs were converted to cobol, leaving the master data files with their dates intact and these were clearly visible to analysts and programmers. such arguments are also countered by the fact that it became apparent quite early that many systems were surviving for long periods. in fact by the late 1970s there were clear signs that legacy systems were emerging (keogh 1997). for example, ibm's mvs operating system was then approaching 20 years old and is still running (jones 1998b). by the 1980s, the australian government was still using systems developed in the 1960s. one pertinent comment is, "there's a myth in our industry that we change software every five or seven years. but the real systems, the ones that really drive our business, do not get changed every year. they don't get changed every ten years. the air traffic control system [in the us] is more than 20 years old" (p. de jager in radio national 1997). some accounting systems have lasted 30 to 40 years, and 10 to 20 years is common in manufacturing (mckenney et al 1995). in the early decades of it there was usually no way of forecasting how long any particular system would last, and to suggest otherwise is simply rationalising after the event. the influx of end user computing in the 1980s led to rising pressures to replace legacy systems, driven largely by the availability of improved user interfaces and better access to and control over information. in some cases legacy systems were "encapsulated" in new interfaces, leaving their two-digit dates intact. in other cases a system was redeveloped in relational database or client/server form, and the old data or data model migrated to the new system (gartner 1997; johnston 1998). in both cases either dates or date formats were visible to developers or testers, but were often ignored. sometimes the survival of an older system was necessitated by the failure of a redevelopment project, but there was often also a reluctance to disturb systems which met their core business needs. the cultural norm minimised the perceived importance of two-digit dates, and prolonged their use. system maintenance system maintenance could have rectified the problem, once a system using two-digit dates survived. however maintenance usually only addressed the problem if there was a recognised business need for four-digit dates. but this was rare, and there was often a large backlog of other more urgent corrections needed the it industry has rarely got it right first time (de jager 1999; gibbs 1994). as a result the focus of maintenance was on immediate priorities, not longer-term issues. maintenance has also typically had lower status and prestige than system development, and been allowed only limited funding (haselkorn 1998). as the years advanced, two-digit dates became progressively more deeply embedded in systems. this made it increasingly difficult to address the problem through routine maintenance, as opposed to special projects but such projects were rare. more recently, downsizing, budget cuts and outsourcing have often compounded the problem (gerner 1997). a related suggestion is that some it shops may have waited until a "silver bullet" emerged to solve the year 2000 problem, despite that fact that this was always highly unlikely (keogh 1997; ulrich and hayes 1997). all of these factors exacerbated the year 2000 problem, but the underlying convention of two-digit dates was also a significant influence. 95 ajis vol. 6 no. 2 may 1999 high mobility of it professionals the high mobility of it professionals among jobs has been seen as implying that it staff did not care about date problems that might emerge after they left an organisation (de jager 1997). the same observed mobility could equally imply that year 2000 awareness should have spread more than it did. that is, it professionals who had experienced the date issue in one organisation should have taken that awareness to the next. however a curious but genuine "amnesia" seemed to afflict many people after they moved to another job, if their new role did not specifically target four-digit dates. for example, in the mid-1990s an it quality inspector worked at a site employing 200 it staff. she held regular meetings of analysts and programmers to "work through all... specifications to detect any problems and oversights. in the six months i held this position i worked with 90% of all programming staff, reviewed 90% of all work to be done and regularly attended meetings with management on quality issues. in that time the year 2000 issue was never raised not by myself, not by management and not by any of the programmers or analysts" (amon 1998a). it is improbable that none of these it professionals had ever encountered date issues before. the authors have experienced similar effects, and believe that the cultural norm shifted the focus away from two-digit dates. end user computing the end user computing wave which followed the microcomputer revolution of the 1980s introduced widespread systems development by non-it professionals. such developers were relatively shielded from the internal complexity of computers. nevertheless they certainly saw and used two-digit dates, which were often offered as options in end user support products such as system generators and this reinforced their legitimacy. some such products used windowing in their system software. not all applications so generated were fully year 2000 compliant, unless the business context was accurately reflected in the application. the client/server computing trend of the 1990s again emphasised the role of users in system development (adhikari 1994), and it is known that many such systems also have year 2000 problems (johnston 1998). the underlying cultural norm served to promote and legitimise the use of two-digit dates. force of precedent the force of precedent exacerbated the year 2000 problem, once it had become entrenched. this was related to the development of new systems by integrating into them pre-existing components including code modules, programs, designs, whole systems and data holdings, and this occurred at a steadily increasing rate from the 1970s onwards (freeman 1987; gerner 1997). even in the 1960s, established business procedures such as the use of punched cards had helped shape later systems. if a pre-existing component used two-digit dates, the new system often did so as well. related to the integration process were the need for the compatibility of new versions of a product widi previous versions, the encapsulation of legacy systems and legacy data within new user interfaces, the integration of entire systems into larger networks, and the development of embedded systems. in general these processes all entailed the integration of existing components into new systems. in fact the history of computing largely entails a progression of ever more complex integration of components. as just one example, remnants of a basic interpreter some 16 years old were found at the heart of a modern pc: "the computer was suddenly revealed as... a series of skins, layer on layer... we build our computers the way we build our cities over time, without a plan, on top of ruins" (ullman 1998). the pc itself is a legacy system little wonder that manufacturers have found it difficult to achieve full year 2000 compliance. designers and developers often treated pre-existing components as "black boxes", and did not look inside them. however they necessarily had to attend to the interfaces among system components. if two-digit dates were used in a "black box" they usually crossed its interfaces in some way. either dates or date formats were visible to developers or testers but were often seen as insignificant and not relevant to the core priorities, and overlooked. the cultural norm influenced such perceptions. there was also pressure in many cases to simply integrate the "black box" without any changes to it. if the developer had actually perceived a serious problem in a component, it may have been either corrected or rejected. this did sometimes happen, but only rarely because of the use of two-digit dates. the it revolution has brought dramatic changes to the world economy over the last fifteen years, and this has exacerbated the year 2000 problem. computing was originally mainly a western phenomenon, but it later spread to many non-western nations. there are now many large, diverse and interdependent systems connected together 96 ajis vol. 6 no. 2 may 1999 widely on almost a global scale, often using and exchanging two-digit dates (de jager 1999; ulrich and hayes 1997). the cultural norm has contributed to a neglect of two-digit dates in these systems and their interconnections. many asian economies are vulnerable to the year 2000 problem (economist 1998). however historically many non-western nations did not use the western calendar (bade 1995; o'neil 1975). the fact that many such nations now have year 2000 problems underlines the extent of their adoption of the western calendar, and also the extent to which they have derived their systems and business practices from the west. denial denial that a serious problem exists has been common in recent years, especially among business managers and some it managers (amon 1998b; furman and marotta 1994; ulrich and hayes 1997). such denial can result in an organisation doing little or nothing. for example, in 1997 few australian government agencies had fully gauged their "year 2000 exposure" and estimated the costs (anao 1997). in late 1997 it was estimated that twothirds of us companies did not have a year 2000 plan (american banker 1998). a 1998 survey of us small businesses found that 82 per cent were at risk, but only 23 per cent had taken action (fysh 1998). the australian bureau of statistics found that by late 1998 about 43 per cent of 6500 businesses surveyed had not started work (business today 1998). a new zealand survey in late 1998 found that fewer than a third of responding organisations were "well placed" to manage their risks effectively. responses were "along the lines of 'she'll be right'... they 'are under-informed and over-confident'" (gifford 1998b). the underlying cultural norm acted to diminish the perceived importance of two-digit dates and promote denial if the dates looked alright, how could there be a major problem? denial may in some cases be based on a view that the year 2000 issue is a "racket" or "ruse" driven by the it industry (zvegintzov 1996). such arguments are countered by the large budgets allocated by many technologically astute organisations evidently for sound business reasons, and with much of it already spent. for example, the australian telecommunications company telstra has allocated up to $500 million, while the national australia bank has budgeted about $250 million (financial review 1998; gottliebsen 1998). conclusion the hypothesis proposed in this paper is based on the fact that a seemingly insignificant, everyday component of culture can nevertheless strongly influence behaviour. one such aspect is the western cultural norm relating to two-digit dates, which constitutes a common thread underlying the various actions taken within the it industry over the past four decades. these actions in themselves reinforced the legitimacy of the cultural norm the dates seemed right, and therefore were right. the year 2000 problem resulted from the collision of this long-standing cultural norm with twentieth century technology, a collision which drove two-digit dates deeply into our systems. the agents of this process were ordinary people working in various professions, whose actions were simply influenced by their culture. in all cases it is people who are the bearers and agents of culture. for these same reasons, the convention of two-digit dates is also likely to persist into the next century. it has been demonstrated that social and cultural factors can shape technology (bijker et al 1987). technologies, tools and material artefacts reflect the societies and cultures which create them (harris 1975; schusky 1975). one pertinent comment is that "values pervade the invention and production of a technology, and these values are put into technology in such a way that they cannot be separated from it... the technology may therefore be tainted because of the practices, institutions, and people involved in its creation..." (johnson 1997). it is evident that cultural norms may also pervade technologies and artefacts. a computer-based information system is a prime example of a modem artefact which reflects the culture of both the organisation and the people involved in its creation and use. the year 2000 problem highlights the importance of bringing a wider perspective to bear on human-computer interaction an approach which is often overlooked or seen as less "serious" in information system studies. virtually all information systems are conceived, designed, built and used by people all of whom invariably bring their own distinct understandings and expectations to their roles. the technology is clearly important, but the human factors also can have a substantial and all too often unforeseen impact on information systems, and on their success or failure. the year 2000 problem has been seen as "part of the flawed integration of technology with society and with... real human beings" (carmichael 1998). with regard to dates, the full human requirement has always been to: a) accept and display all dates in forms natural and familiar to users, and b) interpret and process dates accurately at all times and in all contexts. for many computer-based information systems, only part (a) of this requirement was met. the full requirement was rarely even made explicit. this in itself was influenced by the cultural norm, which evidently implied (falsely) that the requirement had been met ironically, the manual information systems of the late nineteenth century satisfied the requirement in full. these records often used familiar two-digit dates, which were correctly and 97 ajis vol. 6 no. 2 may 1999 effortlessly interpreted by users. the challenge for it is to better understand the full range of human needs, and to reflect them more accurately and carefully in systems which are more "human-centred" (kling and star 1998). the development of such systems would require significantly improved analysis of social and cultural factors, including values, behaviours and norms. will modern civilisation be destroyed by the year 2000 problem? such an outcome is unlikely. will society pay a high price for its failure to properly understand and fulfil a basic human requirement of numerous information systems? this is now beyond doubt. references adhikari, r. (1994) implementing client/server technology, charleston: computer technology research corporation american banker (1998) year 2000 bankers conference, american banker and strategic research institute amon, j. (1998a) "the year 2000 problem: a programmer's perspective on the dilemma of the century", computerworld nz, 20 july amon, j. (1998b) "programmer denial threatens y2k projects", computerworld nz, 23 november anao (1997) managing the year 2000 problem risk assessment and management in commonwealth agencies, australian national audit office, audit report no. 27 armstrong, d. (1999) "not guilty confessions of a programmer", the press (nz), 20 january australian (1998) "y2k: the bug that ate the future", the australian, april-december austrian, g. (1982) herman hollerith: forgotten giant of information processing, new york: columbia university press bach, m. (1986) the design of the unix operating system, englewood cliffs: prentice-hall begbie, r. (1998) "they endured the unimaginable", canberra times, panorama, 27 june bemer, b. (1979) "time and the computer", interface age, february bijker, w. (1995) "sociohistorical technology studies", in handbook of science and technology studies, jasanoff s., markle, g., petersen, j. and pinch, t. (eds.), sage publications bijker, w., hughes, t. and pinch t. (eds.) (1987) the social construction of technological systems, cambridge mass.: mit press binning, d. (1998) "cost of y2k blows out by more than 100 per cent", sydney morning herald, 25 september bulletin (1898) untitled item, bulletin (sydney), 24 december, p. 28 burnett, r. (1998) "defense funds likely to chase bug", orlando sentinel, 31 october business today (1998) "australian businesses slow to start work on millennium bug", business today, 8 december bylinsky, g. (1998) "industry wakes up to the year 2000 menace", fortune, 27 april carmichael, d. (1998) year 2000: who will do what and when will they do it? towards actions, 23 july, http://www.tmn.com/y2k/y2kwho.htm chorafas, d. (1998) agent technology handbook, new york: mcgraw-hill connors, e. (1998) "it's a $1 trillion jigsaw puzzle without borders", australian financial review, 20 april davis, i. (1998) "canberra's y2k outlay on the rise", australian financial review, 28 april dci (1999) "y2k's worldwide financial impact", dci, 13 january de jager, p. (1993) "doomsday 2000", computer world, 6 september de jager, p. (1997) "computers on strike! you've got to be kidding", year/2000 journal, january-february de jager, p. (1998a) "keynote address", year 2000 bankers conference, american banker and strategic research institute de jager, p. (1998b) "financial community leads the y2k race", wall street journal, 24 september de jager, p. (1999) "y2k: so many bugs, so little time", scientific american, january de jager, p. and bergeon, r. (1997) managing 00 surviving the year 2000 computing crisis, new york: wiley delamarter, r. (1986) big blue: ibm's use and abuse of power, london: macmillan drucker, p (1970) technology, management, and society, london: heinemann dunn, a. (1998) "year 2000 bug likely to infest courtrooms", los angeles times, 8 september eade j. (1995) the calendrical systems of mainland south-east asia, london: brill economist (1998) "survey: the millennium bug", economist, 19 september feilder, k. (1997) "unloved forgotten: year/2000 issues affecting personal computers", year/2000 journal, july-august financial review (1998) "special report: the millennium bug", australian financial review, 20 april foremski, t. (1998) "date change dilemma", computer weekly, 17 december 98 ajis vol. 6 no. 2 may 1999 freeman, p. (ed.) (1987) tutorial: software reusability, ieee computer society pressfurman, j. and marotta, a. (1994) "year 2000 denial". computer-world, 28 (43), 24 october fysh, g. (1998) "wising up to 'y2k'", news tribune, 27 september gartner (1997) the year 2000 crisis: an enormous challenge that must be addressed, gartner group, strategic analysis report, march gates, d. (1998) "before y2k: the 1980 problem", pretext magazine, may gerner, m. (1997) "why has the year/2000 problem happened?" year/2000 journal, march-april ghosh, d. (1997) "surprise! client/server has the year/2000 problem, too", year/2000 journal, november/december gibbs w. (1994) "software's chronic crisis", scientific american, 271(3) gifford, a. (1998a) "bug carries a nasty bite", nz herald, 6 october gifford, a. (1998b) "y2k why we should start to worry", business herald, 18 september gillin, p. (1984) "the problem you may not know you have", computerworld, 13 february gottliebsen, r. (1998) 'time runs out for canberra on millennium bug", the age, 18 april greif, s. (1997) "embedded systems: surprising exposures to the year/2000 problem", year/2000 journal, may-june guy, j. (1998) y2k fun in 1974, http://webhome.idirect.com/~joeguy/y2000.htm harris, m. (1975) culture, people, nature: an introduction to general anthropology, new york, crowell haselkorn. m. (1998) the year 2000 problem. preliminary draft technical information statement, version 1.2. ieee technical activities board. new technologies development committee (www.mindspring.com/-pci-inc/year2000 in 1998) hayes, d. and bullers, f. (1998) "chances to fix year 2000 problem squandered for decades", kansas city star, 26 september hayes, f. (1997) "year 2000: a regular laff riot", computerworld, 31(31) p.101 heide, l. (1997) "shaping a technology: american punched card systems 1880-1914", ieee annals of the history of computing 19(4) hill, d. (1998) "if business is late on y2k, govt has some blame", computerworld nz, 28 september ibm (1996) the year 2000 and 2 digit dates: a guide for planning and implementation, fifth edition, ibm corporation inman, g. (1998) "the year/2000 problem and its importance to unix users", year/2000 journal, september/october johnson d. (1997) "is the global information infrastructure a democratic technology?" computers and society, 27(3), september johnston, l. (1998) "getting a grip on client/server applications year/2000 isn't only a mainframe problem", year/2000 journal, september/october jones, c. (1998a) the year 2000 software problem quantifying the costs and assessing the consequences, reading, ma: addison wesley jones, c. (1998b) dangerous dates for software applications, version 2, 24 march, http://www.year2000.com/y2karchive.html kappelman, l. and cappel, j. (1996) 'taring the year 2000 problem", http://www.coba.unt.edu/bcis/faculty/kappelma/criucal.htrn (earlier version in journal of systems management, july/august 1996) keogh, j. (1997) solving the year 2000 problem, boston: ap professional kingsford, r. and dunn, l. (1998) the origin of the year 2000 date problem: an alternative hypothesis, technical report 1998/2, school of information technology and computer science, university of wollongong kling, r. and star, s. (1998) "human centred systems in the perspective of organizational and social informatics", computers and society, 28(1), march kranzberg m. and davenport, w. (eds.) (1972) technology and culture, an anthology, new york: new american library landers, j. (1998) "lost chances litter year 2000 bug's path", dallas morning news, 4 october luening, e. (1998a) "study: y2k already a problem", cnet news, 25 march luening, e. (1998b) "microsoft debuts y2k strategy", cnet news, 15 april luening, e. (1998c) "andersen faces y2k suit", cnet news, 31 august luening, e. (1999) "who will pay the bills?" cnet news, 20 january mackenzie, d. and wajcman, j. (1987) the social shaping of technology, milton keynes: open university mathieson, n. (1998) "beware the bug!" graduate contact, university of queensland, 17 99 ajis vol. 6 no. 2 may 1999 mckenney, j., copeland, d. and mason, r. (1995) waves of change: business evolution through information technology, boston: harvard business school press microsoft (1998) microsoft year 2000 resource centre, april 15, http://www.microsoft.com moore, r and foley, d. (1998) "date compression and year 2000 challenges", dr. dobb's journal, may morgan stanley (1997) technology: enterprise software, morgan stanley us investment research, 25 april, http://www.year2000.com/y2karchive.html o'neil, w. (1975) time and the calendars, sydney: sydney university press overby, s. (1998) "andersen consulting sues to block j. baker's request it pay for y2k flaw in systems", business today, 3 september peckinpaugh, c. (1999) "who to blame for the y2k problem?" federal computer week, 11 january radio national (1997) "the millennium "bug": the year 2000 problem", background briefing, radio national transcripts, 23 march ratcliffe, m. (1998) "responsibility: business and tech pros will bear the brunt of blame", zdnet news, 7 oct schick, k. (1995) three certainties: death, taxes and the year 2000, gartner group, application development and management strategies, research note, strategic planning spa-215-1116, 25 january schusky, e. (1975) the study of cultural anthropology, new york: holt, rinehart and winston spradley j. and mccurdy, d. (1975) anthropology: the cultural perspective, new york: wiley strassman, p. (1998) "hard data (at last) on year 2000 costs", computer world, 15 may swaine, m. (1998) "a chat with bob bemer", dr. dobb's journal, 23(5) p. 115 ted, b. (1998) "the mailbox is full of ideas about the causes and effects of the year-2000 problem", infoworid,20(10)p.l08 ullman, e. (1998) "the dumbing-down of programming", salon, 13 may ulrich, w. and hayes, i. (1997) the year 2000 software crisis challenge of the century, upper saddle river: prentice hall van steenwyk, e. (1991) the california gold rush: west with the forty-niners, new york: watts yourdon, e. and yourdon, j. (1998) time bomb 2000: what the year 2000 computer crisis means to you, upper saddle river, nj: prentice hall zvegintzov, n. (1996) "the year 2000 as racket and ruse", american programmer, february 100 ajis vol. 7 no. 1 sept 1999 advances and research directions in data-warehousing technology mukesh mohania school of computer and information science university of south australia mawson lakes, adelaide. south australia 5095 mohania@cis.unisa.edu.au sunil samtani department of computer science telecommunications, university of missouri-kansas city, kansas city, mo 64110, u.s.a. ssamtani @cstp.umkc.edu johnf. roddick school of computer and information science university of south australia mawson lakes, adelaide. south australia 5095 roddick@cis.unisa.edu.au yahiko kambayashi department of social informatics, graduate school of informatics, kyoto university, kyoto 606-8501, japan. vahiko@kuis.kvoto-u.ac.ip abstract information is one of the most valuable assets of an organisation and when used properly can assist in intelligent decision making that can significantly improve the functioning of an organisation. data warehousing is a recent technology that allows information to be easily and efficiently accessed for decision-making activities by collecting data from many operational, legacy and possibly heterogeneous data sources. on-line analytical processing (olap) tools are well-suited for complex data analysis, such as multi-dimensional data analysis, and to assist in decision support activities while data mining tools take the process one step further and actively search the data for patterns and hidden knowledge in the data held in the warehouse. many organisations are building, or are planning to develop, a data warehouse for their operational and decision support needs. in this paper, we present an overview of data warehousing, multi-dimensional databases, olap and data mining technology and discuss the directions of current research in the area. we also discuss recent developments in data warehouse modelling, view selection and maintenance, indexing schemes, parallel query processing and data mining issues. a number of technical issues for exploratory research are presented and possible solutions are also discussed. introduction relational database systems are designed to record, retrieve and manage large volumes of real-time transaction data and to keep an organisation running by supporting day-to-day business transaction processing. these systems are generally tuned for a large community of users and the user typically knows what is needed and generally accesses a small number of rows in a single transaction. indeed, relational database systems are suited for robust and efficient on-line transaction processing (oltp) on operational data. one of the main objectives of relational systems is to maximize transaction throughput and minimize concurrency conflicts. however, these systems have generally limited decision support functions and do not possess or enable the extraction of all the necessary information required for faster and more intelligent decision-making essential for the growth of an organisation. for example, it is hard for a relational database system to answer queries such as what were the supply patterns of product x in south australia in 1997 and how were they different from the previous year? it has therefore become important to support analytical processing capabilities in organisations for (1) the efficient management of organisations, (2) effective marketing strategies and (3) efficient and intelligent decision-making. on-line analytical processing (olap) tools are well-suited for complex data analysis, such as multidimensional data analysis, and to assist in decision support activities which access data from a separate data warehouse, which may consist of data collected from many, possibly heterogeneous, operational and legacy data sources. the data warehouse system is typically tuned for a smaller community of users including business and technical managers, directors and/or decision-makers. a data warehouse is a set of subject-oriented, integrated, time-varying and non-volatile databases used to support the decision-making activities (inmon 1992). the term subject-oriented indicates that information is arranged according to the business areas of an organisation, such as manufacturing, sales, marketing, finance, etc. the term integrated refers to the fact that a data warehouse integrates data derived from various functional 41 ajis vol. 7 no. 1 sept 1999 systems in the organisation and provides a unified and consistent view of the overall organisational situation. time-variant means that warehouse data is able to represent the flow of data through time, and queries may be based on time ranges. the data is non-volatile in that it is always appended to (and rarely deleted from) the warehouse thus maintaining the company's entire history. thus, a data warehouse system is a repository system that contains highly aggregated and summarised data used for complex data analysis and decision support activities, such as data mining, comparisons of historical data and trend analysis. for efficient and correct analysis, the quality of data is important, that is, • the warehouse data should be accurate, complete, consistent, integrated, well-defined and time-stamped for informational purposes, • the warehouse data should conform to appropriate business rules and satisfy integrity constraints, including primary/foreign keys, referential integrity and data dependencies, • users, including the auditors, should be satisfied with the quality of data and information derived from warehouse. typical warehouse queries are complex and ad-hoc in nature and generally these queries access huge volumes of warehouse data and perform many joins and aggregations. query response time and throughput are therefore more important than transaction throughput. in summary, the data warehousing environment provides a computerised interface that enables business decision-makers to creatively approach, analyse and understand business problems. the aim of the data warehouse system is to turn data into strategic decision making information and to bring solutions to users. this process is done by tuning the data at many steps. ( ) observe s "\*( information 1̂ verify and assimilate ( knowledge j^ synthesize and analyse [ decision t_ evaluate and apply c direction jf execute and determine figure 1. data analysis chart data warehousing system the data warehouse creation and management component includes software tools for selecting data from appropriate information sources (which could be operational, legacy, external, etc. and which may be distributed, autonomous and heterogeneous), and cleaning, transforming, integrating and propagating data into the data warehouse. it also refreshes the warehouse data and metadata when source data are updated. this component is also responsible for managing the warehouse data, creating indices on data tables and data partitioning. the warehouse data contains the detail data, summary data, consolidated data and/or multidimensional data. the metadata is generally held in a separate repository. the metadata contains the informational data about the creation, management and usage of the data warehouse. it serves as a bridge between the users of the warehouse and the data contained in it. the warehouse data is also accessed by the olap server to present the data in a multi-dimensional way to the front-end tools (such as analytical tools, report writers, spreadsheets and data mining tools) for analysis and informational purposes. essentially, the olap server interprets client queries (the client interacts with front-end tools and passes these queries to the olap server) and converts them into complex sql (indeed extended sql) queries required to access the warehouse data. it might also access the data from the primary sources if the client's queries need operational data. finally, the olap server passes the multi-dimensional views of data to the front-end tools and these tools format the data according to the client's requirements. the conceptual architecture of a data warehousing system is shown in figure 2. there are two approaches to creating the warehouse data — bottom-up and top-down. in a bottom-up approach, the data is obtained from the primary sources based on the data warehouse applications and a profile of the likely queries, which is typically known in advance. the data is then selected, transformed and integrated by 42 ajis vol. 7 no. 1 sept 1999 data acquisition tools. in a top-down approach, the data is obtained from the primary sources whenever a query is posed. in this case, the warehouse system determines the primary data sources in order to answer the query. these two approaches are similar to eager and lazy approaches discussed by widom (1995). the bottom-up approach is used in data warehousing because user queries can be answered immediately and data analysis can be done efficiently since data will always be available in the warehouse. hence, this approach is feasible and improves the performance of the system. another approach is a hybrid approach, which combines aspects of the bottom-up and top-down approaches. in this approach, some data is stored in a warehouse, and other data can be obtained from the primary sources on demand (hull and zhou 1996). f "n data network ' others v j r ~ > /• n select transform dean integrate refresh v j (d7 . ^~2\ 1. 'warehouse 1 j^^t\ ;.pata '-. |̂ > if \^__ . jl^ ^^ id iz!> ^s^ • • • ••' '~\\je? meta data f '-' " 1 .rolap/ , molap; f "\analysis data mining report ' generation other v j • clients information sources motivation warehouse creation and management component olap server front-end tools figure 2. a conceptual data warehousing architecture widom (1995) outlined a number of technical issues in data warehousing which warranted the attention of the research community. a lot of work has been done in many areas since then and new issues have come to the fore. different aspects of data warehousing present different related and unrelated challenges with the common goal of presenting integrated information to the user quickly. different methodologies have been used to achieve this goal. thus, data warehousing has evolved into a number of different research streams. these streams deal with different phases of data warehousing design, implementation, operation and maintenance. in this paper, we explore the different aspects of this emerging technology and highlight new avenues still unexplored. chaudhuri and dayal (1997) present the state of art in data warehousing and olap with an emphasis on new requirements. the work presented serves as a good overview and is an excellent survey in this area. however, there is a need to initiate a discussion on the impending research problems in the field. in this paper, we suggest new directions to some of these problems by providing insight for developing efficient solutions and by discussing some of the well known solutions. structure of this paper the warehouse data is typically modelled multi-dimensionally. new data models and new data structures that support the multi-dimensional data model effectively needs to be developed. the multi-dimensional data model (agrawal, gupta and sarawagi 1997; lehner, ruf and teschke 1996) has been proved to be the most suitable for olap (on-line analytical processing) applications and olap tools provide an environment for decision making and business modelling activities by supporting ad-hoc queries. there are two ways to implement multi-dimensional data model: • by using the underlying relational architecture to project a pseudo-multi-dimensional model and • by using true multi-dimensional data structures such as arrays. we discuss the multi-dimensional model and the implementation schemes in the next section. although the idea of a true multi-dimensional implementation is lucrative theoretically, it has not been widely investigated by either the research or commercial communities. however, some work has been done in defining a new algebra for multi-dimensional data model and new algebraic operations and extensions to existing sql have been proposed in the literature. nevertheless, the use of this multi-dimensional data model has not been exploited for all aspects of data warehouse design and maintenance. the basic feature of this model is that it allows the user to visualise data from different perspectives. there is a need to extend its capabilities by defining a set of constraints on this model so that they can be exploited in other areas of warehouse implementation. this paper contributes in a useful way by setting the platform for extending the multi-dimensional data model by 43 ajis vol. 7 no. 1 sept 1999 introducing the concept of constraints on the data cube. we also highlight the need for constraints in the multidimensional environment and its proposed uses. the multi-dimensional data model is stored in the data warehouse as a set of materialized views over source data. there are limitations to the concept of materializing views at the data warehouse. precomputation of queries in materialized views can give answers quickly but the number of views that should be materialized at the warehouse needs to be controlled, otherwise this can result in data explosion. we investigate the issue of the selection of views to be materialized at the data warehouse and describe well-known heuristics in detail. one challenge in data warehousing is how to maintain the materialized views. when the data at any source changes, the materialized views at the data warehouse need to be updated accordingly. the process of keeping the views up-to-date in response to the changes in the source data is referred to as view maintenance. refreshing the warehouse data is often done, for reasons of efficiency, using incremental techniques rather than recomputing the view from scratch. there has been significant research on view maintenance in traditional databases (bailey, et al. 1998; blakeley, larson and tompa 1986; dong and mohania 1996) (griffin and libkin 1995; gupta and mumick 1995; gupta, mumick and subrahmanian 1993; ramakrishnan, etal. 1994; segev and fang 1990; segev and park 1989) and in data warehouses (agrawal, et al. 1997; hull and zhou 1996; huyn 1997; quass, et al. 1996; yang, karlapalem and li 1996) (zhuge, et al. 1995). in data warehousing, the view maintenance has branched into a number of sub-problems such as self-maintenance, consistency maintenance, update filtering and on-line view maintenance. we investigate these issues and propose new ways of looking at some of these problems. the technique of view materialization is hampered by the fact that one needs to anticipate the queries to materialize at the warehouse. the queries issued at the data warehouse are mostly ad-hoc and cannot be effectively anticipated at all times. users might query on dimensions that are not materialized in views. thus, the precomputation strategy is limited in its ways. effective translation of data cubes in the relational model results in star/snowflake schema. these require effective indexing methods since they involve joins on multiple tables for answering queries. the query is not precomputed and needs to be computed on the fly. the data is integrated from various sources and stored in the data warehouse in advance to answer decision support queries from the user. this needs faster access structures to keep the processing on-line. the traditional indexing methods in relational databases do not work well in the data warehousing environment and new access structures have been proposed for data warehousing environments. we investigate different types of indexing schemes in section 4. a relatively new approach to compute expensive operations such as join and aggregation is through the use of parallel processing techniques. we discuss the issues of parallelism in data warehousing in section 5. the accumulation of data in a data warehouse provides opportunities for data mining and issues relating to mining from data warehousing are discussed. finally, we comment on other areas of active research and conclude the paper with a discussion on possible future work. data models for data warehousing the data models for designing traditional oltp systems are not well suited for modelling complex queries in data warehousing environment. the transactions in oltp systems are made up of simple, pre-defined queries. in the data warehousing environments, the queries tend to use joins on more tables, have a larger computation time and are ad-hoc in nature. this kind of processing environment warrants a new perspective to data modelling. the multi-dimensional data model, sometimes referred to as the data cube has turned out to be a satisfactory model that provides a way to aggregate facts along multiple attributes, called dimensions. data is stored as facts and dimensions instead of rows and columns as in relational data model. facts are numeric or factual data that represent a specific business activity and a dimension represent a single perspective on the data. each dimension is described by a set of attributes. 44 ajis vol. 7 no. 1 sept 1999 1997.x 1996̂ "̂ 1999.xdb is os se hw t major_r-area dimensions: location, time, research_area attribute: research_area (researchjd, minor_r_area, major_r_area, broad_r_area) attribute hierarchy: minor_r_area" a b c d university major_r_area i broad_r_area, figure3. data cube a multi-dimensional data model (mddm) supports complex decision queries on huge amounts of enterprise and temporal data. it provides an integrated environment for summarising (using aggregate functions or by applying some formulae) information across multiple dimensions. mddm has now become the preferred choice of many vendors as the platform for building new on-line analytical processing (olap) tools. the user has the leverage to j/ice and dice the dimensions, allowing users to employ different dimensions during an interactive query session. the data cube allows the user to visualise aggregated facts multi-dimensionally. the level of detail retrieved depends on the number of dimensions used in the data cube. when the data cube has more than three dimensions, then it is called a hypercube. the dimensions form the axes of the hypercube and the solution space represents the facts as aggregates on measure attributes. the cube for publications data is shown in the figure below. implementation schemes the conceptual multi-dimensional data model can be physically realised in two ways, firstly by using trusted relational databases and secondly by making use of specialised multi-dimensional databases. each approach is briefly described below. relational scheme this scheme stores the data in specialised relational tables, called fact and dimension tables. it provides a multidimensional view of the data by using relational technology as an underlying data model. facts are stored in the fact table and dimensions are stored in the dimension table. for example, the research_area dimension consists of four attributes, namely, researchjd, minor_r_area, major_r_area and broad_r_area. examples of fact tables include sales data, publication data, etc, and the examples of dimension tables include products, stores, documents, research areas, etc. facts in the fact table are linked through their dimensions. the attributes that are stored in the dimension table may exhibit attribute hierarchy. 45 ajis vol. 7 no. 1 sept 1999 dimension table: document doc id type_of_doc title_of_doc fact table: publications -100 tuples dimension table: authors author id name email_address affiliation doc id location id author id research id time id total_journal_count total_conf_count -3,000,000 tuples dimension table: time -5000 tuples time id year d x* imension table: locations location id university_name city state country -1000 tuples dimension table: research areas research id minor_r_area major_r_area broad_r_area -50 tuples -20 tuples figure 4. star schema example star schema/snowflake schema are used to support multi-dimensional data representation. they offer flexibility, but often at the cost of performance because of more joins for each query required. a star/snowflake schema models a consistent set of facts (aggregated) in a fact table and the descriptive attributes about the facts are stored in multiple dimension tables. this schema makes heavy use of denormalization to optimise complex aggregate query processing. in a star schema, a single fact table is related to each dimension table in a many-toone (m:l) relationship. each dimension tuple is pointed to many fact tuples. dimension tables are joined to fact table through foreign key reference; there is a referential integrity constraints between fact table and dimension table. the primary key of the fact table is a combination of the primary keys of dimension tables. note that multiple fact tables can be related to the same dimension table and the size of dimension table is very small as compared to the fact table. the figure below shows an example of star schema. here, publication data is stored in fact table and location, document, authors, research area and time are dimension tables. as can be seen above, the dimension tables are denormalized and therefore, the star schema does not capture hierarchies (ie. dependencies among attributes) directly. this is captured in snowflake schema. here, the dimension tables are normalised for simplifying the data selecting operations related to the dimensions, and thereby, capture attribute hierarchies. in this schema, the multiple fact tables are created for different aggregate levels by pre-computing aggregate values. this schema projects better semantic representation of business dimensions. 46 ajis vol. 7 no. 1 sept 1999 multiple inheritance graph method: traffic-violation; methodi/craffic-violajtion (a) repeated inheritance graph string: driver-name; method: birthday-died; (b) redundant inheritance graph string: driver-name; method: birthday-died; (0 figure 5. snowflake schema example multi-dimensional data model which translates the data cube into a star/snowflake schema is defined as follows (baralis, paraboschi and teniente 1997): a multi-dimensional database is a collection of n hypercubes (h,, h2,..., hn}, where each hypercube h; is a set of relations {di, d2,.... dm, f) such that • each dj is a dimension table, ie. a relation characterised by its unique identifier dj (dimension) that uniquely identifies each tuple in dj and provides the granularity for representing facts along the axis of the hypercube. • f is a fact table, ie., a relation connecting all dimension tables d,, d2, ..., dm on the dimension attribute dj in each table; the identifier of f is given by the foreign keys db ..., dm of all the dimension tables it connects; the schema of f contains a set of additional attributes m called measure attributes on which the aggregate functions are computed; the aggregate functions on m form the solution space of the hypercube. each dimension table may contain either a simple dimension or a composite dimension.. a composite dimension is a collection of attributes that exhibit certain functional dependencies that gives rise to dimension attribute hierarchies. let d be a composite dimension in dimension table d such that schema(d) = (a,, ..., a0}. the attribute hierarchy on d is defined as set of functional dependencies fdd = (fdi, fd2,..., fdn) where each fdj represents a functional dependency from a; to aj; the dependency is represented by fdj: a, —»aj. the hypercubes are materialized as views at the data warehouse. the materialized views that store the preprocessed queries at the data warehouse along different dimensions are selected to minimize the query response time and the view update costs. another criterion for selecting a view in a data warehouse is its frequency of access. a query might not be accessed frequently but it should be materialized in a view at the data warehouse (dw) if other queries that are accessed frequently can be answered quickly using this view. this is an area of active research and several heuristics are proposed (baralis, ceri and paraboschi 1996; baralis, paraboschi and teniente 1997; labio, quass and adelberg 1997; ross, srivastava and sudarshan 1996). the problem of view maintenance and the problem of view selection is discussed further later. 47 ajis vol. 7 no. 1 sept 1999 multi-dimensional scheme this scheme stores data in a matrix using array-based storage structure. each cell in the array is formed by the intersection of all the dimensions; therefore, not all cells have a value. for example, not all authors publish in all journals. the multi-dimensional data set requires smaller data storage since the data is clustered compactly in the multi-dimensional array. the values of the dimensions need not be explicitly stored. the n-dimensional table schema is used to support multi-dimensional data representation, which is now described. an n-dimensional table schema is the fundamental structure of a multi-dimensional database that draws on terminology of the statistical databases. the attribute set associated with this schema is of two kinds: parameters and measures. an n-dimensional table has a set of attributes r and a set of dimensions d associated with it. each dimension is characterised by a distinct subset of attributes from r, called the parameters of that dimension. the attributes in r, which are not parameters of any dimension, are called the measure attributes. this approach is a unique way of flattening the data cube since the table structure is inherently multidimensional. the actual contents of the table are essentially orthogonal to the associated structure. each cell of the data cube can be represented in an n-dimensional table as table entries. these table entries have to be extended by dimensions to interpret their meaning. the current literature on n-dimensional table however does not give an implementation of the mddb that is different from the implementation suggested by the already existing schemas. this implementation breaks up the n-dimensional table into dimension tables and fact tables, which snowballs into snowflake schema and traditional rolap. the challenge with the research community is to find mechanisms that translate this multi-dimensional table into a true multi-dimensional implementation. this would require new data structures for the implementation of multiple dimensions in one table. the relation in the relational data model is an example of 0-dimensional table. constraints on the cube model in a relational schema, it is possible to define a number of integrity constraints in the conceptual design. these constraints can be broadly classified as key constraints, referential integrity constraints, not null constraint, relation-based check constraints, attribute-based check constraints and general assertions (business rules). these constraints can be easily translated into triggers that keep the relational database consistent at all times. this concept of defining constraints based on dependencies can be mapped to a multi-dimensional scenario. the current literature on modelling multi-dimensional databases has not discussed the constraints on the data cube. in a relational model, the integrity and business constraints that are defined in the conceptual schema provide for efficient design, implementation and maintenance of the database. taking a cue from the relational model, it is necessary to identify and enumerate the constraints that exist in the multi-dimensional model. an exploratory research area would be to categorise the cube constraints into classes and compare them with the relational constraints. the constraints can be broadly classified into two categories: intra-cube constraints and inter-cube constraints. the intra-cube constraints define constraints within a cube by exploiting the relationships that exist between the various attributes of a cube. the relationships between the various dimensions in a cube, the relationships between the dimensions and measure attributes in a cube, dimension attribute hierarchy and other cell characteristics are some of the key cube features that need to formalised as a set of intra-cube constraints. the inter-cube constraints define relationships between two or more cubes. there are various considerations in defining inter-cube constraints. such constraints can be defined by considering the relationships between dimensions in different cubes, the relationships between measures in different cubes, the relationships between measures in one cube and dimensions in the other cube and the overall relationship between two cubes, ie. two cubes might merge into one, one cube might be a subset of the other cube, etc. using constraints the cube constraints facilitate the conceptual schema design of a data warehouse and allow the provision of triggers in a multi-dimensional scenario. in a relational database system, triggers play an important role by enforcing the constraints and business rules in an effective manner. the update, insert and delete triggers in the relational model have provided the robustness by allowing least manual intervention. the current data warehouse maintenance algorithms have neglected the role of triggers in designing effective view maintenance paradigms. the research community needs to consider the new types of triggers that could be used for effective data warehousing. the constraint set will be of vital importance to this faculty. the constraint set can also be used to solve existing problems such as view maintenance. in (mohania, konomi and kambayashi 1997; ross, srivastava and sudarshan 1996), the authors have proposed view maintenance techniques based on the view expression tree that is used for evaluating the view as a query. the auxiliary 48 ajis vol. 7 no. 1 sept 1999 relations are materialized for each node in the tree in addition to the materialized view. the updates to nodes in the expression tree are calculated and propagated in a bottom-up fashion. the update to each node in the tree is derived from the updates to its children nodes and the auxiliary relations materialized for the children and the node itself. if the constraints are defined on the view, then they can be broken down into smaller components (ie. sub-constraints) and these sub-constraints can be pushed down into the expression tree to provide new efficient mechanisms for view maintenance and aggregate query optimisation. that is, the constraints can be defined at each node in the tree. in this case, the updates to each node can be checked using these constraints before propagating them to the parent node. if some of the updates violate constraints, then there is no need to propagate these updates to the parent node. this method of decomposing the constraints and pushing them down along the edges of the expression tree can effectively reduce the communication and computation costs. the constraint set acts as a sieve that effectively filters the updates that do not affect the view. furthermore, the constraint set will be an effective tool that can be used in other aspects of data warehousing. for example, the constraint set would enable the definition of a complete algebra for the multi-dimensional data model that is independent of the underlying schema definitions. the constraint set would also allow evolution of new attributes in the multi-dimensional scenario. some of these constraints can be used to map the multidimensional world into the relational world. the data model can be viewed as a ball and the constraints can help visualise that ball from various perspectives. the constraint set would enable new implementation mechanisms that are better able to conserve the multi-dimensionality of data. the first task is to define a constraint set on the multi-dimensional data model that is complete in any implementation scheme. the constraints need to be formally defined and the precedence order for the constraints needs to be decided. operations in multi-dimensional data model data warehousing query operations include standard sql operations, such as selection, projection and join. in addition, it supports various extensions to aggregate functions, for example, percentile functions (eg. top 20 percentile of all products), rank functions (eg. top 10 products), mean, mode and median. one of the important extension to the existing query language is to support multiple group by by defining rollup, drill-down and cube operators. rollup corresponds to doing further group-bys on the same data object. for example, let the publications be grouped by minor-research area. it is possible to further group-by the same publications data by major-research area and broad-research area by using rollup operation. note that rollup operator is order sensitive, that is, when it is defined in the extended sql, the order of columns (attributes) matters. the function of drill-down operation is the opposite of rollup. the hypercube which involves joining of multiple tables to represent facts needs a new set of algebraic operations. a new algebra needs to be proposed for the multidimensional environment. the idea of faster query processing will require extensions to existing sql in the existing environment. new operators such as cube, push, pull, restrict, star, join and merge have been proposed in literature but all these operators are specific to the schema for which they are designed (agrawal, gupta and sarawagi 1997; bauer and lehner 1997; lehner, ruf and teschke 1996; li and wang 1996). view selection and maintenance view selection we discussed earlier which views should be materialized at the data warehouse. the problem of view selection was first studied by roussopoulos (roussopoulos 1982). recently, several heuristics have been proposed for the same problem (baralis, ceri and paraboschi 1996; baralis, paraboschi and teniente 1997; labio, quass and adelberg 1997; ross, srivastava and sudarshan 1996)}. in (roussopoulos 1982), an algorithm based on a* and the approximate knapsack problem has been proposed. this algorithm does not consider the cost of subviews to materialise and index selection. (ross, srivastava and sudarshan 1996) describes an exhaustive enumerative search algorithm where they consider the cost of subviews to materialise, but without considering indexes. (labio, quass and adelberg 1997) has considered both the cost of subviews and indexing in their optimal view selection algorithm. we now discuss a heuristic (baralis, paraboschi and teniente 1997) in detail which uses the data cube technology and the lattice model. the lattice model feeds on the attribute hierarchy defined earlier. the nodes in the lattice diagram represent views (aggregated on certain dimensions) to be materialized at the data warehouse. if the dimensions of two views a and b exhibit attribute hierarchy such that dim(a) -» dim(b) then there is an edge from node a to node b. node a is called the ancestor node and node b is called the dependent or descendant node. the lattice diagram allows us to establish relationships between views that need to be materialized. some queries can be answered by using the already materialized views at the data warehouse. for answering such queries, it is not necessary to use the raw data. the lattice diagram depicts dependencies between the views and a good view selection heuristic exploits this dependency. the view selection algorithm tries to minimize the average time required to evaluate a view and also keeps a constraint on 49 ajis vol. 7 no. 1 sept 1999 the space requirements. the space requirements that can be expressed as the number of views to be materialized translates this into an optimisation problem that is np-complete. an approximate and acceptable solution for this problem is the greedy heuristic. the greedy algorithm selects a view from a set of views depending upon the benefit yielded on selecting that view. a view a that is materialized from view b incurs a materializing cost that is equal to the number of rows in view b. if there is a view c (materialized on b; number of rows in c • number of rows in b) such that view a can be derived from c, the cost of materializing a reduces to the number of rows in c. thus the benefit of materializing view c includes the benefit incurred by view a in the form of reduction in its materializing cost (number of rows in b number of rows in c). the greedy heuristic selects a view that maximizes the benefit that will be yielded on materializing that view. the benefit of materializing each dependent view (a node in the lattice diagram) will change with the selection of an ancestor view in the data warehouse. after each selection is made, the benefit at each dependent node in the lattice is recalculated and a view with maximum benefit is selected. it has been shown that the greedy algorithm is at least 3/4 of optimal. the greedy algorithm can be extended to restrict on actual space requirements rather than the number of views to be materialized. the frequency with which the views are accessed can be incorporated in this algorithm. other criteria for selecting views may also be based on the way the views are implemented at the warehouse. different schema design methods will give different set of tables at the data warehouse. view maintenance a data warehouse (dw) stores integrated information from multiple data sources in materialized views (mv) over the source data. the data sources (ds) may be heterogeneous, distributed and autonomous. when the data in any source (base data) changes, the mvs at the dw need to be updated accordingly. the process of updating a materialized view in response to the changes in the underlying source data is called view maintenance. the view maintenance problem has evoked great interest in the past few years. this view maintenance in such a distributed environment gives rise to inconsistencies since there is a finite unpredictable amount of time required for (a) propagating changes from the ds to the dw and (b) computing view updates in response to these changes. data consistency can be maintained at the data warehouse by performing the following steps: • propagate changes from the data sources (st] current state of the data sources at the time of propagation of these changes) to the data warehouse to ensure that each view reflects a consistent state of the base data. • compute view updates in response to these changes using the state stt of the data sources. • install the view updates at the data warehouse in the same order as the changes have occurred at the data sources. the inconsistencies at the data warehouse occur since the changes that take place at the data sources are random and dynamic. before the data warehouse is able to compute the view update for the old changes, the new changes change the state of the data sources from st] to st2. this violates the consistency criterion that were listed. making the mvs at the data warehouse self-maintainable decimates the problem of inconsistencies by eliminating the finite unpredictable time required to query the data source for computing the view updates. in the next subsection, the self-maintenance of materialized views at the data warehouse is described. self-maintenance consider a materialized view mv at the data warehouse defined over a set of base relations r = [r], r2,.... /?„}. mv stores a preprocessed query at the data warehouse. the set of base relations r may reside in one data source or in multiple, heterogenous data sources. a change a/?j made to the relation r, might affect mv. mv is defined to be self-maintainable if a change amv in mv, in response to the change ar, can be computed using only the mv and the update arj. but the data warehouse might need some additional information from other relations in the set r residing in one or more data sources to compute the view update amv. since the underlying data sources are decoupled from the data warehouse, this requires a finite computation time. also the random changes at the data sources can give rise to inconsistencies at the data warehouse. some data sources may not support full database functionalities and querying such sources to compute the view updates might be a cumbersome, even an impossible task. because of these problems, the preprocessed query that is materialized at the warehouse needs to be maintained without access to the base relations. one of the approaches is to replicate all base data in its entirety at the data warehouse so that maintenance of the mv becomes local to the data warehouse (gupta and murnick 1995; gupta, mumick and subrahmanian 1993; kuchenhoff 1991). although this approach guarantees self-maintainability at the warehouse, it creates new problems. as more and more data is added to the warehouse, it increases the space complexity and gives rise to information redundancy which might lead to inconsistencies. this approach also overlooks the point that the base tuples might be present in the view itself, so the view instance, the base update and a subset of the base relations might be sufficient to achieve self-maintainability in the case of spj (select-project-join) views (huyn 1997). but how can the subset 50 ajis vol. 7 no. 1 sept 1999 of the base relations that is needed to compute the view updates be stored at dw1 this question was addressed in (quass, et al. 1996), which defines a set of minimal auxiliary views (avs) to materialize that are sufficient to make a view self-maintainable. although materializing auxiliary views at the dw was a novel concept, the minimality of auxiliary views defined was still questionable since the mv instance was never exploited for selfmaintenance. most of the current approaches maintain the mvs separately from each other using a separate view manager for each view and such approaches fail to recognize that these views can be maintained together by identifying the set of related materialized views. this issue of multiple-view self-maintenance was addressed for the first time in (huyn 1997). in some approaches to multiple-view self-maintenance, a set of auxiliary views (av) are stored at the data warehouse along with the set of materialized views (mv) such that together mv u a v is self-maintainable. the research challenge lies in finding the most economical avs in terms of space complexity and computational costs. the view self maintenance is still an active research problem. it is not always feasible to provide selfmaintainability of the views at the data warehouse. when the cost of providing self-maintainability exceeds the cost of querying data sources for computing view updates, it is profitable to allow querying of data sources instead. consistency maintenance current research has also concentrated on ensuring consistency of the data warehouse when the mvs are not self-maintainable since it is not always possible to provide for complete self-maintainability at the data warehouse. the eca family of algorithms (zhuge, et al. 1995) introduces the problem and solves it partially. the strobe algorithm (zhuge, garcia-molina and widom 1996) introduces the concept of queuing the view updates in the action-list at the data warehouse and installing the updates only when the unanswered query set (uqs) is empty. the algorithm solves the consistency problem but is subject to the potential threat of infinite waiting. there are other mechanisms that are based on time stamping the view updates (baralis, ceri and paraboschi 1996). these methods do not address the consistency problems in their entirety and also assume the notion of global time. we propose that the self-maintainability of views at the data warehouse should be a dynamic property. the cost of providing self-maintainability should be continuously monitored and when this cost increases beyond a certain threshold, the maintenance mechanism should be shifted to querying data sources to compute the view updates. this threshold can be computed depending on the cost of querying data sources. an effective algorithm that provides this dynamism and efficient garbage collection is the need of the hour. update filtering the changes that take place in the source data need to be reflected at the data warehouse. some changes may create view updates that need to installed at the data warehouse; some changes leave the views at the data warehouse unchanged. if it is possible to detect at the data source that certain changes are guaranteed to leave the views unchanged, we need not propagate these changes to the data warehouse. this would require checking of distributed integrity constraints at a single site. as many changes as possible can be filtered at the sources and only the changes that result in view updates may be propagated to the warehouse. the update filtering will reduce the size of the maintenance transactions at the data warehouse, thus minimizing the time required to make the data warehouse consistent with the data sources. the side effect of update filtering is that we need to make our data sources (and the wrapper/monitor) components more intelligent. they need to know about their participation in the data warehouse and the data warehouse configuration so that the updates can be checked against the constraint set before propagating them. to be able to realise this, the data sources cannot be decoupled from the data warehouse. this would give rise to new problems such as configuration management ie., if there is a change in the schema at any data source or at the data warehouse, all the participating entities need to be informed of this change so that they can modify the constraint set to reflect this change. the view maintenance strategies would now be based on the constraint set and any change to the constraint set would warrant a change in the existing view maintenance transaction. on-line view maintenance warehouse view maintenance can be performed either incrementally or by queuing a large number of updates at the data sources to be propagated as a batch update from the data sources to the data warehouse. in current commercial systems, a batch update is periodically sent to the data warehouse and view updates are computed and installed. this transaction is called the maintenance transaction. a user typically issues read-only queries at the data warehouse and a long-running sequence of user queries is called a reader session. the batch maintenance transaction is typically large and blocks the reader sessions. this renders the data warehouse 51 ajis vol. 7 no. 1 sept 1999 unavailable for the duration of the maintenance transaction. with the advent of the 24-hour, global internet market, the use of overnight maintenance runs will have to change and online processes developed. incremental view maintenance which updates the data warehouse instantaneously in response to every change at the data source is expensive and gives rise to inconsistent results during the same reader session. an update from the data source will change the results a user might see over a sequence of queries. quass and widom (1997) discuss a possible approach to this problem by maintaining two versions of each tuple at the data warehouse simultaneously so that the reader sessions and the maintenance transactions do not block each other. a possible solution may need the integration with self maintenance techniques, where auxiliary views can be used to answer queries during maintenance transactions. indexing schemes indexing is the creation of access structures that provide faster access to the base data relevant to the restriction criteria of queries (viguier and datta 1997). the size of the index structure should be manageable so that benefits can be accrued by traversing such a structure. the traditional indexing strategies used in database systems do not work well in data warehousing environments. most oltp transactions typically access a small number of rows; most oltp queries are point queries. b trees which are used in most common relational database systems are geared towards such point queries. they are well suited for accessing a small number of rows. an olap query typically accesses a large number of records for summarising information. for example, an oltp transaction would typically query for a customer who booked on flight qfan1234 on say april 25th; on the other hand an olap query would resemble a query such as return the number of customers who booked flights on qfan1234 in april. the second query would access more records and is typically a range query. b tree indexing scheme which is so apt for oltp transactions is not the best suited to answer olap queries. there is a need to look at new indexing schemes and variants of existing schemes that will work well with olap queries. a number of indexing strategies have been suggested for data warehouses (flanagan and safdie 1997; informix 1997; o'neil and quass 1997) in the literature. we review a few important strategies. • value-list index the value-list index consists of two parts. the first part is a balanced tree structure and the second part is a mapping scheme. the mapping scheme is attached to the leaf nodes of the tree structure and points to the tuples in the table being indexed. the tree is generally a b tree with varying percentages of utilisation. oracle provides a b* tree with 100% utilisation (oracle 1992). two different types of mapping schemes are in use. first one consists of a rowid list which is associated with each unique search-key value. this list is partitioned into a number of disk blocks chained together. the second scheme uses bitmaps. a bitmap is a vector of bits that store either a 0 or a 1 depending upon the value of a predicate. a bitmap b lists all rows with a given predicate p such that for each row r with ordinal number j that satisfies the predicate p, the jth bit in b is set. bitmaps efficiently represent low-cardinality data, however to make this indexing scheme practical for highcardinality data, compression techniques must be used. value-list indexes have been shown in (o'neil and quass 1997) to outperform other access method in queries involving the min or max aggregate functions, as well as queries that compute percentile values of a given column. • projection index a projection index is equivalent to the column being indexed. if c is the column being indexed, then the projection index on c consists of a stored sequence of column values from c in the same order as the ordinal row number in the table from where the values are extracted. it has been shown in (o'neil and quass 1997) that projection indexes outperform other indexing schemes for performing queries that involve computation on two or more column values and appear to perform acceptably well in group-by like queries. • bit-sliced index a bit sliced index represents the key values of the column to be indexed as binary numbers and projects a set of bitmap slices which are orthogonal to the data held in the projection index. this index has been shown (o'neil and quass 1997) to particularly perform well for computing sums and averages. also, it outperforms other indexing approaches for percentile queries if the underlying data is clustered, and for range queries whose range is large. • data index a datalndex, like the projection index, exploits the positional indexing strategy (viguier, datta and ramamritham 1997). the datalndex avoids duplication of data by storing only the index and not the column being indexed. the dataindex can be of two specific types: basic datalndex and join datalndex (for more information, see (viguier, datta and ramamritham 1997)). parallelism in data warehousing in this paper, we have discussed possible techniques to evaluate olap queries faster. all these techniques are restricted in their ways. the precomputation strategy needs to anticipate queries so that it can materialize them in the data warehouse. olap queries are of ad-hoc nature and restricts precomputing to the imagination of the 52 ajis vol. 7 no. 1 sept 1999 designer. the indexing schemes we have discussed provide faster access to the data stored in the warehouse, but does not reduce the size of tables stored at the data warehouse. one can say that the strategies presented provide faster query processing, but they need not be fast enough as perceived by the user. one mechanism that is recently being exploited by vendors to compute queries quickly is to execute the query in parallel by partitioning data among a set of processors. this can potentially achieve linear speedup and can significantly improve query response times. however to exploit this technology we need to address a few key issues such as parallel data placement and parallel join. the existing data warehouse schema design strategies is not catered to parallel data placement, although the star schema suggests implicitly the need of partitioning the data into a set of dimension tables and fact tables. many dbms vendors claim to support parallel data warehousing to various degrees. most of these products, however, do not use the dimensionality of data that exists in a data warehouse. the effective use of parallel processing in this environment can be achieved only if we are able to find innovative techniques for parallel data placement using the underlying properties of data in the warehouse. (datta, moon and thomas 1998) use the data indexing strategy to provide efficient data partitioning and parallel resource utilisation. effective algorithms that split the data among n parallel processors and perform parallel join operation have been illustrated. data mining issues for data warehouses recent years have seen a continued growth in the size and number of databases and, more recently, in the development of data warehouses. whilst much useful knowledge is contained implicitly in them, the large volumes of data involved have prevented their full exploitation. this has led to research into the automatic extraction or synthesis of knowledge from databases commonly termed data mining or knowledge discovery from databases. in the late 1980s and early 1990s, data mining, and the wider field of knowledge discovery from databases, became a research area with roots in active and deductive databases, artificial intelligence research (specifically that research looking into machine learning), statistical analysis and information retrieval. more recently, the field has divided into two sub-areas: • autonomous and semi-autonomous knowledge discovery, which is looking at applying previous methods for • the maintenance of summary databases, including a strong emphasis on the development of techniques that can incrementally keep summary databases and data warehouses up to date; • the extraction of knowledge for the more highly structured data warehouses; • the extraction of data from multi-dimensional databases including temporal and spatio-temporal databases. • the development of databases that accommodate induction in databases and data warehouses. from the standpoint of data warehouses, data mining has much to offer although to date, with a few notable exceptions, the 'mining' of data in commercial or industrial databases has largely been under the umbrella of olap technologies. notable exceptions include (fayyad, weir and djorgovski 1993; keats and loo 1997; shortland and scarfe 1994; viveros, nearhos and rothman 1996; ziarko, golan and edwards 1993) which show the utility of autonomous data mining. the predefined schemata (which may be in various formats as described earlier) provide a template from which mining can then proceed. as discussed in (rainsford and roddick 1999), learning from data warehouses has a number of advantages: • the data is stored in a more or less structured manner. for example data is typically structured into (normalised or denormalised) relations that eliminate redundancy and can be joined in various ways to retrieve the required data sets from a database. in other database paradigms, either more or less structure is available. nevertheless a certain amount of apriori known structure is available and can, and arguably should be, utilised in the mining process. • in addition to the structure above, some domain knowledge may already be encoded implicitly or explicitly within the database. for example, the existence of a participation constraint may be flagged by a not null constraint. similarly, the cardinality of relationships is also often explicit within database structure and constraints. • high performance query, data manipulation and transaction tools are already available. this would include the database management system, associated query language, specialised hardware and other database tools. it therefore makes some sense to use these tools to interrogate the database where appropriate. similarly, the use of data resident in databases and data warehouses imposes a number of characteristics and constraints not necessarily encountered in smaller volume machine learning applications: • the volume of data is typically large. for example, the skicat system has been developed to process three terabytes of graphic images resulting from a sky survey (fayyad, weir and djorgovski 1993). 53 ajis vol. 7 no. 1 sept 1999 while this may be at the high end, data warehouse volumes are typically in or close to the gigabyte range. any data mining tool must therefore perform satisfactorily on large volumes of data. • the data may contain noise. data mining tools must provide adequate mechanisms for finding sufficiently accurate results from noisy data. • the database may contain incomplete information. not all information useful for the discovery of knowledge may actually be stored within the database. likewise, much redundant and useless data may also be present. data mining tools must therefore facilitate both the selection of relevant data, and learning with incomplete knowledge. • in general, data is not collected for the purpose of knowledge discovery. as well as leading to some of the above problems this means that data may be formatted or interpreted inappropriately. knowledge discovery tools must therefore be able to access data stored in various formats. a number of models of the data mining process have been suggested. all split the mining process into a number of phases that are iterated as appropriate. the models suggested can be adapted to use with data warehouses as shown in the figure. applications .j v'» .%-••** -• <f ?y! ••-•• s« • » « , * • ' " -« r; • ,•, • • • -t,• • expert systems semantic query optimisation '>'", intelligent interfaces decision support systems ? integrity enforcement database design • : / . > ~ . . . ' . . . . . ' ..;v **. «*.' • : .>• •.#< tx „. • , ;-•. ' j' '>' ^' '* ''! " a a a 4— _ .i _ . i templates, visualisation and selection tools > rule type specification and templates i > parameters and thresholds i > browsers and filters k > k • user input figure 6. a model of the data mining process (from (rainsford and roddick 1999)) for any database, the number of possible rules that can be extracted is far greater than the number of facts in a data warehouse. knowledge discovery can be viewed as a multi-stage process of selecting interesting rules from the total rule-space that exists within a database. this is therefore a process of progressively reducing the initial infinite rule space down to a small subset of useful rules. this reduction process is performed using a selection of filters that reduce the target rule space on the basis of source data, rule pattern, statistics and semantics. special cases may exist where a phase of the filtering process does not exist within a system, and 54 ajis vol. 7 no. 1 sept 1999 therefore the filter will allow the rule space to pass unreduced. for example, semantic filtering may not be used if rules are to be used for query optimisation. each of the filtering stages may therefore consist of zero or more filters, specified by the user or discovery system. the target rule set may be passed back and forth between the filters for reprocessing. a central controller coordinates the operation of the filters. as noted on the diagram the final rule set can be integrated into the existing knowledge base and both the knowledge base and the user may interact with each step of the rule-space reduction process. note that the filtering processes may have a profound effect on the outcome of the data mining process. thus the outcome of the data mining process is quasi-nondeterministic. the rules derived from a data warehouse are clearly dependent on both the data held and the algorithm applied. classification rules, for example, can be used to categorise examples into classes on the basis of known properties (agrawal, et al. 1992), while a characteristic rule can be defined as an assertion that characterises the concept satisfied by all of the relevant data in the database (han, cai and cercone 1993). association rules provide a measure of correlation between the occurrence of items in set of transaction (agrawal, imielinski and swami 1993). other forms of rules, such as functional, causal and temporal can also be mined if the data warehouse data allows it; some of these are outlined in (rainsford and roddick 1999). a number of surveys have been published that provide different perspectives on the development and use of data mining tools in various domains (chen, han and yu 1996; frawley, piatetsky-shapiro and matheus 1992; holsheimer and siebes 1994) (fayyad, piatetsky-shapiro and smyth 1996; fayyad, et al. 1996; koperski, adhikary and han 1996; rainsford and roddick 1999). these and other publications indicate that a number of advanced utilities and approaches may be required in the future as follows: embedded summarisation research into the construction of data cubes in databases (gray, et al. 1997) suggests that olap applications would benefit from a number of advanced summarisation functions. unfortunately, it is currently unclear how efficient query optimisation can be maintained if data cube operations are performed at the level of middleware. there is already some concern about whether current query optimisation techniques are able to accommodate data warehousing (chaudhuri 1997). concept hierarchies/lattices many algorithms proposed for data mining rely on the idea of concept ascension. in addition, concept hierarchies would be a useful addition to many other application domains, including arguably, traditional transaction processing. the lack of accommodation of concept hierarchies has therefore necessitated more or less extensive, but always fairly similar, middleware layers to add concept hierarchies and/or lattices. sampling, clustering and approximation while there may be specified bounds, many multimedia databases and real-time systems are commonly willing to accept close answers now in preference to exact answers eventually. for different reasons, data mining applications are also commonly content to accept approximate answers, perhaps based on the random sampling of data or of a given cluster or sub-set of the data. inductive queries data mining researchers and, to a lesser extent those involved with data warehousing, sometimes find that it is useful to relax the closed world assumption. that is, it is occasionally useful to ask questions of the database without necessarily assuming that either all the data is held or that anything not held in the database is untrue. moreover, al and deductive database research has long investigated, for example, the explicit holding of known falsities so that a tri-state logic of true, false and unknown can be returned. the development of other database paradigms to accommodate this may also be useful. incremental knowledge discovery the volume of data currently being mined is outstripping the normal capacity of data mining systems to mine it, primarily due to the i/o-boundness of data mining routines and the fact that i/o channel speed improvements are not keeping pace with improvements in either storage capacity of cpu speed. it is thus important that both incremental discovery routines, which can amend data warehouse data in bursts as changes are made to the transaction database, and that restartable routines, which can be run whenever resources permit, such as overnight or during other periods of low activity, be developed. it would also be useful if cooperating data mining algorithms, each tailored to mine different aspects of the same data store, could be used. to enable both to happen, a common mechanism for storing interim results would be useful. other research challenges and conclusions the research challenges discussed so far are directly concerned with the problem of faster data integration. there are other challenges which are a result of the data warehousing concept. here we mention a few such challenges which are currently active. one such problem arises due to the changing user requirements. as the 55 ajis vol. 7 no. 1 sept 1999 requirements of the user changes, the view definitions at the data warehouse change dynamically. there is a need to develop view adaptation techniques that do not need the entire recomputation of the materialized view at every change. in (gupta, mumick and ross 1995; mohania 1997; mohania and dong 1996), the authors have presented the view adaptation techniques for a data warehouse that recompute only parts of the view (if any) which cannot be derived from the existing materialized views. these techniques are applicable for spj queries. however, there is a need to devise adaptation algorithms for aggregated queries. as discussed in (munneke, wahlstrom and mohania 1999), the fragmentation of a multi-dimensional databases has not been widely researched. the fragmentation mechanisms employed to partition a database over multiple sites play an important role in the design of a distributed database system as proper fragmentation enhances query performance by defining appropriate units of distribution which allow for concurrent, parallel execution of queries. the parallel processing of olap operations to achieve quick response times is similarly important (datta, moon and thomas 1998). this can be achieved by fragmenting multi-dimensional database into number of fragments. we identify two types of possible fragmentation. firstly, slice and dice that selects subsets of data by 'cutting up' the multi-dimensional hypercube representation of the global data into sub-cubes. slice and dice is currently employed by some multi-dimensional databases that support fragmentation; for example the essbase olap server partitioning option. secondly, severing that divides the global data by removing dimensions from a hypercube. this method may be used to divide a cube into two cubes with different dimensions from each other. operations performed during severing ensure that the original cube may be reconstructed from the two severed cubes (munneke, wahlstrom and mohania 1999). another interesting problem is the problem of data expiry in the warehouse. data that is materialized in the warehouse may not be needed after a point. there is a need to devise efficient schemes that determine the data expiry point and do effective garbage collection by removing the unnecessary tuples. the challenge that awaits the data warehousing community is the integration of web data in the warehouse. the internet has become a repository of unstructured information and the need to incorporate such external information will require the development of effective schemes for extracting semi-structured and unstructured data for storing in the data warehouse. conclusions a data warehouse is a central repository that contains highly aggregated, summarised and consolidated data, and is used for data analysis and decision support functions. to do effective data analysis, the quality of data is important which can be achieved by cleaning the data. this process involves data filtering, transformation and integration in order to get rid of data inconsistencies in the warehouse. in this paper, we have discussed several research issues in the areas of data warehouse modelling, view selection and maintenance, indexing and parallel query processing. further, we have exploited the multi-dimensionality of data in the warehouse to introduce the concept of constraints on the data cube. we highlight the usefulness of the constraint set in every aspect of data warehousing, that is, design, implementation and maintenance. this paper also underlines the need for dynamic self-maintainability of materialized views and discusses the issues involved. in this paper we have also discussed the advantages and differences in mining from data repositories such as data warehouses. it is our view that while the majority of commercially implemented mining systems are currently based around the interactive olap mode of knowledge discovery, the use of semi-autonomous mining search engines will become more commonplace once some research problems, such as rule useability, are resolved. the area discussed in the paper also contains a number of research issues which will need to be dealt with before commercial information systems can leverage the maximum advantage from data warehouses and the associated technology. apart from the technical issues discussed at various points in the paper, of particular immediate concern is the manner by which data warehouse design and implementation processes are seamlessly integrated with conventional information systems design methodologies. acknowledgments we are grateful to kirsten wahlstrom for bringing an important omission to our attention. references agrawal, d., et al. (1997). "efficient view maintenance in data warehouses". acm sigmod international conference on management of data, tucson, arizona, usa, pp. 417-427. agrawal, r., et al. (1992). "an interval classifier for database mining applications". eighteenth international conference on very large data bases, vancouver, canada, pp. 560-573. agrawal, r., et al. (1997). modeling multidimensional databases. research report report ibm. 56 ajis vol. 7 no. 1 sept 1999 agrawal, r., et al. (1993). "mining association rules between sets of items in large databases". acm sigmod international conference on management of data, washington dc, usa, acm press. vol. 22. pp. 207-216. bailey, j., et al. (1998). "distributed view maintenance by incremental semijoin and tagging". distributed and parallel databases. vol. 6, no. 3, pp. 287-309. baralis, e., et al. (1996). "conservative timestamp revised for materialized view maintenance in a data warehouse". workshop on materialized views, montreal, pp. 1-9. baralis, e., et al. (1997). "materialized view selection in a multidimensional database". internationa] conference on very large databases, athens, greece, morgan kaufmann. pp. 156-165. bauer, a. and lehner, w. (1997). "the cube-query-language (cql) for multidimensional statistical and scientific database systems". fifth international conference on database systems for advanced applications (dasfaa), melbourne, pp. 263-272. blakeley, j. a., et al. (1986). "efficiently updating materialized views". acm sigmod international conference on the management of data, pp. 61 -71. chaudhuri, s. (1997). "panel on "query optimisation at the crossroads"". acm sigmod international conference on the management of data, tucson, arizona, usa, pp. 509. chaudhuri, s. and dayal, u. (1997). "an overview of data warehousing and olap technology". sigmod record. vol. 26, no. 1, pp. 65-74. chen, m.-s., et al. (1996). "data mining: an overview from database perspective". ieee transactions on knowledge and data engineering. vol. 8, no. 6, pp. 866-883. datta, a., el al. (1998). a case for parallelism in data warehousing and olap. department of mis, university of arizona, tucson, az. dong, g. and mohania, m. (1996). "algorithms for view maintenance in mobile databases". first australian workshop on mobile computing and databases, monash university, australia, fayyad, u., m., et al. (1996). "from data mining to knowledge discovery: an overview", in advances in knowledge discovery and data mining. aaai press/ mit press, pp. 1-34. fayyad, u., m., et al, ed. (1996). advances in knowledge discovery and data mining. menlo park, california, aaai press. fayyad, u. m., et al. (1993). "automated analysis of a large-scale sky survey: the skicat system". aaai93 workshop on knowledge discovery in databases (kdd '93), washington d.c., aaai press, menlo park, california, pp. 1-13. flanagan, t. and safdie, e. (1997). data warehouse technical guide. white paper report sybase. frawley, w. j., etal. (1992). "knowledge discovery in databases: an overview". al magazine. vol. 13, no. 3, pp. 57-70. also in piatetsky-shapiro, g. and frawley, w.j., (eds.) 1991. knowledge discovery in databases. aaai press/mit press, menlo park, ca. 1-30. gray, j., et al. (1997). "data cube: a relational aggregation operator generalising group-by, cross-tab and sub-totals". data mining and knowledge discovery. vol. 1, no. 1, pp. 29-54. griffin, t. and libkin, l. (1995). "incremental maintenance of views with duplicates". acm sigmod international conference on the management of data, san jose, usa, pp. 328-339. gupta, a. and mumick, i. s. (1995). "maintenance of materialized views: problems, techniques and applications". ieee data engineering bulletin, special issue on materialized views and warehousing. vol. 18, no. 2, pp. 3-18. gupta, a., et al. (1995). "adapting materialized views after redefinitions". acm sigmod international conference on management of data, san jose, usa, pp. 211 -222. gupta, a., et al. (1993). "maintaining views incrementally". acm sigmod international conference on management of data, pp. 157-166. han, j.( et al. (1993). "data-driven discovery of quantitative rules in relational databases". ieee transactions on knowledge and data engineering. vol. 5, no. 1, pp. 29-40. holsheimer, m. and siebes, a. p. j. m. (1994). data mining: the search for knowledge in databases. technical report report cs-r9406. cwi, the netherlands. hull, r. and zhou, g. (1996). "a framework for supporting data integration using the materialized and virtual approaches". acm sigmod conf. on management of data, montreal, canada, pp. 481-492. huyn, n. (1997). "multiple view self-maintenance in data warehousing environments". international conference on very large databases, athens, greece, pp. 26-35. informix (1997). a new generation of decision-support indexing for enterprise-wide data warehouses. white paper report informix. inmon, w. h. (1992). building the data warehouse. qed technical publishing group. keats, g. and loo, s. (1997). "an intelligent business workbench for the insurance industry: using data mining to improve decision making and performance.". eighth international database workshop, data mining, data warehousing and client/server databases., hong kong, springer-verlag singapore, pp. 256-274. 57 ajis vol. 7 no. 1 sept 1999 koperski, k., et al. (1996). "knowledge discovery in spatial databases: progress and challenges". acm sigmod workshop on research issues on data mining and knowledge discovery, montreal, canada, pp. 55-70. kuchenhoff, v. (1991). "on the efficient computation of the difference between consecutive database states", in second international conference on deductive object-oriented databases. lecture notes in computer science. vol. 566. springer-verlag. pp. 478-502. labio, w. j., et al. (1997). "physical database design for data warehouses". thirteenth international conference on data engineering, birmingham, u.k., ieee computer society, pp. 277-288. lehner, w., et al. (1996). "cross-db: a feature-extended multidimensional data model for statistical and scientific databases". international conference on information and knowledge management, rockville, maryland, usa, pp. 253-260. li, c. and wang, x. s. (1996). "a data model for supporting on-line analytical processing". international conference on information and knowledge management, rockville, maryland, usa, pp. 81-88. mohania, m. (1997). "avoiding re-computation: view adaptation in data warehouses". eighth international database workshop, hong kong, pp. 151-165. mohania, m. and dong, g. (1996). "algorithms for adapting materialized views in data warehouses". international symposium on cooperative database systems for advanced applications, kyoto, japan, pp. 62-69. mohania, m., et al. (1997). "incremental maintenance of materialized views". eighth international conference on database and expert systems applications (dexa '97), toulouse, france, springerverlag. pp. 551-560. munneke, d., et al. (1999). "fragmentation of multidimensional databases". tenth australasian database conference, auckland, new zealand, springer, pp. 153-164. o*neil, p. and quass, d. (1997). "improved query performance with variant indexes". acm sigmod international conference on management of data, tucson, arizona, usa, pp. 38-49. oracle (1992). oracle 7 server concepts manual. redwood city, ca, oracle corporation. quass, d., et al. (1996). "making views self-maintainable for data warehousing". international conference on parallel and database information systems, miami beach, florida, usa, quass, d. and widom, j. (1997). "on-line warehouse view maintenance for batch updates". acm sigmod international conference on management of data, tucson, arizona, usa, pp. 393-404. rainsford, c. p. and roddick, j. f. (1999). "database issues in knowledge discovery and data mining". australian journal of information systems. vol. 6, no. 2, pp. 101-128. ramakrishnan, r., et al. (1994). "efficient incremental evaluation of queries with aggregation". international logic programming symposium, ross, k. a., et al. (1996). "materialized view maintenance and integrity constraint checking: trading space for time". acm sigmod international conference on management of data, montreal, canada, roussopoulos, n. (1982). "view indexing in relational databases". acm transactions on database systems. vol. 7, no. 2, pp. 258-290. segev, a. and fang, w. (1990). "currency based updates to distributed materialised views". ieee international conference on data engineering, los angeles, california, pp. 512-520. segev, a. and park, j. (1989). "maintaining materialised views in distributed databases". ieee international conference on data engineering, los angeles, california, pp. 262-270. shortland, r. and scarfe, r. (1994). "data mining applications in bt". bt technology journal. vol. 12, no. 4, pp. 17-22. viguier, i. r. and datta, a. (1997). sizing access structures for data warehouses. technical report report dept. of mis, university of arizona, tucson, az. viguier, i. r., et al. (1997). 'have your data and index it, too', efficient storage and indexing for data warehouses. technical report report good-tr-9702. department of mis, university of arizona. viveros, m. s., et al. (1996). "applying data mining techniques to a health insurance information system". twenty-second international conference on very large data bases (vldb '96), mumbai, india, morgan kaufmann publishers, inc. san francisco, usa. pp. 286-293. widom, j. (1995). "research problems in data warehousing". fourth international conference on information and knowledge management (cikm), baltimore, maryland, pp. 25-30. yang, j., et al. (1996). a framework for designing materialized views in data warehousing environment. techn ical report report hkust-cs96-35. department of computer science, the hong kong university of science and technology. zhuge, y., et al. (1995). "view maintenance in a warehousing environment". acm sigmod international conference on management of data, san jose, usa, pp. 316-327. zhuge, y., et al. (1996). "the strobe algorithms for multi-source warehouse consistency". international conference on parallel and distributed information systems, miami beach, florida, 58 ajis vol. 7 no. 1 sept 1999 ziarko, w., el al. (1993). "an application of datalogic/r knowledge discovery tool to identify strong predictive rules in stock market data". aaai-93 workshop on knowledge discovery in databases (kdd '93), washington d.c., aaai press, menlo park, california, pp. 89-101. 59 ajis vol. 7, no 2 may 2000 efficient mobile client caching supporting transaction semantics ilyoung chung and chong-sun hwang dept. of computer science and engineering korea university 5-1 anam-dong, seongbuk-ku seoul 136-701, korea abstract in mobile client-server database systems, caching of frequently accessed data is an important technique that will reduce the contention on the narrow bandwidth wireless channel. as the server in mobile environments may not have any information about the state of its clients' cache(stateless server), using broadcasting approach to transmit the updated data lists to numerous concurrent mobile clients is an attractive approach. in this paper, a caching policy is proposed to maintain cache consistency for mobile computers. the proposed protocol adopts asynchronous(non-periodic) broadcasting as the cache invalidation scheme, and supports transaction semantics in mobile environments. with the asynchronous broadcasting approach, the proposed protocol can improve the throughput by reducing the abortion of transactions with low communication costs. we study the performance of the protocol by means of simulation experiments. introduction mobile computing enables people with unrestricted mobility. it can satisfy people's information needs at any time and in any place. due to the recent development of the hardware such as small portable computers and wireless communication network, data management in mobile computing environments has become an area of increased interest to the database community[elmagarmid(1997), badrinath(1992), pitoura(1995a), alonso(1993)]. in general, the bandwidth of the wireless channel is rather limited. thus, caching of frequently accessed data item in a mobile computer can be an effective approach to reducing contention on the narrow bandwidth wireless channel [pitoura(1995b), dunham(1995), imielinsky(1993)]. however, once caching is used, a cache invalidation strategy is required to ensure the cached data in mobile computers are consistent with those stored in the server[wilkinson(1990), franklin(1993a), gottemukkala(1996)]. several proposals have appeared in the literature regarding the support of transactions in mobile systems[elmagarmid(1995), jing(1995), wong(1995), narasayya(1993)]. however, most of these approaches didn't attempt to make use of a common feature in wireless systems: the ability that the server has to broadcast information to the mobile clients. because the server in mobile environments may not have any information about the state of its clients' cache(stateless server), using the broadcasting approach to transmit the updated data lists to numerous concurrent mobile clients is an attractive approach[barbara(1994)]considering these limitations of the mobile environments, barbara and imielinski proposed an approach that a server periodically broadcasts an invalidation report that reports the data item which have been updated[barbara(1997), barbara(1994)]. this approach is attractive in mobile environment because a server need not know the location and the connection status of its clients, and because the clients need not establish an uplink connection to invalidate their cache. in this paper, we present a protocol that adopts asynchronous(non-periodic) broadcasting, to reduce the wait time of a transaction that has requested commit, and to reduce the abortion of transactions that may show conflicts in the periodic broadcasting strategy. with asynchronous broadcasting approach, the protocol can reduce the number of broadcasting occurrence under high ratio of updates, and can reduce the abortion of transactions under low ratio of updates. the remainder of this paper is organized as follows. section 2 introduces the mobile transaction model. in section 3, we describe and discuss our caching protocol. section 4 presents experiments and results, and finally we state our concluding remarks in section 5. the mobile transaction model figure 1 presents a general mobile database system model. in this model, both a database server and a database are attached to each fixed host[wu(1996), acharya(1995)]. users of the mobile computers may frequently query databases by invoking a series of operations, generally referred to as a transaction. serializability is widely accepted as a correctness criterion for execution of transactions[bernstein(1993), tannenbaum(1995), franklin(1994), berstein(1987)]. this correctness criterion is also adopted in this paper. a database server is intended to support basic transaction operations and as resource allocation, commit, and abort. each mobile support station(mss) has a coordinator which receives transaction operations from mobile hosts and monitors their execution in database servers within the fixed networks. transaction operations are submitted by a mobile host to the coordinator in its mss, which in turn sends them to the distributed database servers within the fixed networks for execution[elmagarmid(1997), alonso(1993)]. ajis vol. 7, no 2 may 2000 mc cache ,/ mc cache . mss: mobile support station mc: mobile client figure 1: the mobile database system architecture in general, there are two ways to structure a mobile transaction processing system[narasayya(1993)]; the first situation is that the mobile host behaves like a remote i/o device, and the data must be placed in the static part of the network[pitoura(1995a), dunham(1995)]. this situation arises when the amount of resources on the mobile device is small. secondly, we can consider a mobile host as a full fledged server to manage transaction, and place data locally in mobile hosts[elmagarmid(1995), jing(1995), franklin(1992)]. in this paper, we adopt an way between these two situation to structure a mobile transaction processing system. that is we still keep the data locally, but we treat it as a cache rather than as a primary copy. thus, as shown in figure 1, each mobile host can have its own cache to maintain some portion of data for later reuse. the transaction operation that accesses the data item stored in the cache can be processed without the interaction with the database server. if the data item does not exist in the cache of the mobile host, it sends an message to the server to download the appropriate data item. our protocol in this section, we present our concurrency control protocol, which adopts asynchronous broadcasting approaches, supporting the stateless server. we assume that there is a central server that holds and manages all data. also, we assume that only one transaction may be initiated by a mobile client at any time. that is, a mobile client can initialize a transaction only after the previous transaction has finished. asynchronous broadcasting the proposed protocol adopts broadcasting approach to maintain cache consistency, and to control the concurrent transactions. with the broadcasting approach, the mobile client sends the commit request of the transaction after executing all operations, to install the updates in the central database. then the server decides commit or abort of the requested transactions, and notifies this information to all of the clients in its cell with the broadcasting invalidation reports. the broadcasting strategy does not require high communication costs, nor require the server to maintain additional information about the mobile clients in its cell, thus is attractive in mobile databases[barbara(1994), narasayya(1993)]. some proposals have appeared in the literature regarding the use of the broadcasting for the control of the concurrent transactions, and all of these approaches adopt synchronous(periodic) manner as the way of broadcasting the invalidation reports [barbara(1997), barbara(1994)]. in these schemes, the server broadcasts the list of invalidating data items and the list of transactions that will be committed among those which have requested commit during the last period. these schemes present some problems which arise with the synchronous manner of broadcasting approach. • if two or more conflicting transactions have requested commit during the same period, only one of them can commit and others have to abort. • the mobile client is blocked on the average for the half of the broadcasting period until it decides commit or abort of the transaction. ajis vol. 7, no 2 may 2000 in this paper, we adopt asynchronous broadcasting approach as the way of sending the invalidation report. unlike the schemes using periodic broadcasting, in our scheme, invalidation reports are broadcasted immediately after a commit request arrives. using the asynchronous broadcasting approach, most transactions that may show conflicts in the same period by synchronous broadcasting, can avoid them, thus the protocol can reduce the abortion rate of transaction processing. also, the blocking time of the transaction that have sent the commit request can be reduced, as the server immediately broadcasts the invalidation reports which notifies commit or abort. the protocol our protocol uses a modified version of optimistic control[liskov(1992), franklin(1994), franklin(1993b), wang(1991)], in order to reduce the communication costs on the wireless network. with optimistic approach, all operations of a transaction can be processed locally using cached data, and at the end of the transaction, the mobile client sends the commit request message to the server. then the server immediately broadcasts the invalidation report including the data items that should be invalidated, and the mobile client identifier. receiving invalidation report, the invalidating action preempts the ongoing transaction that shows conflicts with the now-committed transaction. with this approach, the mobile client can early detect the conflicts of a transaction that should be aborted later at the server. all data items in the system are tagged with a sequence number which uniquely identifies the state of the data. the sequence number of data item is increased by the server, when the data item is updated. the mobile client includes the sequence number of its cached copy of data(if the data item is cache resident) along with the commit request message. we now summarize our protocol for both the mobile client and the server. mobile client protocol: • whenever a transaction becomes active, the mobile client opens two sets, read-set and write-set. initially, the state of a transaction is marked as reading. data item is added to these sets with the sequence numbers, when the transaction requests read or write operation on that data item. the state of the transaction is changed in updating state, when the transaction requests write operation. • whenever the mobile client receives an invalidation report, it removes copies of the data items that are found in the invalidating list. and, if any of the following equations becomes true, the transaction of reading state is changed into read-only state, and the transaction of updating transaction is aborted. • when a transaction of read-only state requests write operation, the mobile client aborts the transaction. • when a transaction is ready to commit, the mobile client commits the transaction of reading state or read-only state locally. if the state of the transaction is updating, the mobile client sends a commit request message along with the read-set, write-set and the identification number of the mobile client. • after that, the mobile client listens to the broadcasting invalidation report which satisfies any of the following. if the invalidation report, satisfying above equations, is attached with the identification number of this mobile client, the transaction is committed, otherwise aborted. server protocol: • whenever the server receives a commit request from a mobile client, and if sequence numbers of all data items in read-set and write-set are identical with the server's, it broadcasts the invalidation report along with the invalidating list, which is the list of data item in the write-set of the commit request, and identification number of the mobile client. otherwise, the server just ignores the commit request. the protocol described above adopts asynchronous broadcasting, thus the server immediately broadcasts invalidation reports, when it receives a commit request from a mobile client. our protocol has some advantages with the asynchronous approach. at first, when write operations are infrequent at mobile clients, the protocol can reduce the communication costs by broadcasting invalidation reports only when updating transaction occurs. it is unnecessary to send invalidation reports periodically without data items that should be invalidated. on the other hand, when updating transactions occur frequently, the protocol can avoid many aborts, by reducing the conflicts between updating transactions. in synchronous broadcasting approach, when two or more updating transactions are conflicting in a period, only one of them can commit, as invalidation reports are broadcast once for a period. our protocol can avoid much of these aborts, because mobile clients are informed the list of updated data items immediately. in this case, the increased number of broadcasting is not an additional communication overhead that may degrades the throughput of transaction processing, as the server initiates only one broadcasting for a committing transaction. as shown in the above protocol, no notification is required for an aborted transaction. the server just ignores the commit request with sequence number which has fallen behind the server's number. ajis vol. 7, no 2 may 2000 mci server mc2 mci server mc2 commit .^(x) commit £\" "^••*. •ww data request commit request invalidation report commit "̂ (a) (b) mci server mc2 mci server mc2 r(jc) r(x) commit r'z) commit w(z) (c) (d) figure! : example execution example execution under the protocol is shown in figure 2. this example uses two mobile clients and a server. in this example, r(x) and w(at) denote a read and write operation performed by a transaction on data item x. in (a), the transaction initiated in mobile client 2 can be committed by the server, as it can read the data item x that has been updated by the transaction in mobile client 1, by the immediate invalidation report. with the synchronous broadcasting approach, if the commit requests of the two transactions arrive in the identical period, both transactions cannot be committed. in (b) and (c), the state of the transaction in mobile client 2 is changed from reading state to read-only state, when the mobile client receives the broadcasting invalidation report. in case of (b), the transaction can commit locally, as the state of the transaction remains read-only state, when the transaction is ready to commit. however, in (c), the transaction will be aborted, because it requests write operation in read-only state. in example (d), the mobile client 2 sends the commit request slightly before it receives the invalidation report that indicates the updates of the data item x. the server ignores the commit request, as the sequence number of data item x in the message is lower than the server's. no broadcasting message is required for the aborted transaction, and the transaction in mobile client 2 is aborted, receiving the invalidation report of data item x, with the identification of mobile client 1. thus, in our protocol, the server initiates one broadcasting for each commit of transactions. performance in this section, we develop the simulation model and present the results of experiments, in order to evaluate the performance of the proposed algorithm. we can divide our simulation model into three parts: database model, transaction model, and system model. ajis database model vol. 7, no 2 may 2000 database is composed of multiple data objects which have the same attributes. there are two attributes for each data object: identifier and sequence number. the database model parameters are summarized in table 1. parameter nobject cachepercent meaning number of objects in the database percentage of cache size table 1: database parameters transaction model the transaction model supports the following operations. — readobject: reads an object from its cache. if the object does not exist in the cache, reads it from the server database. — writeobject: updates an object in the client cache and increase the sequence number. — committr: commit a transaction. — aborttr: abort a transaction. a transaction is modeled by a finite loop of readobject and writeobject operations, which are followed by committr or aborttr operation. table 2 summarizes the parameters that characterize a transaction type. the number of readobject and writeobject operations in a transaction is called trsize, which is uniformly distributed between mintrsize and maxtrsize. the parameter probwrite indicates the probability that writeobject operations occur in a transaction. the delay parameters are exponentially distributed delay times used to model interactive system. parameter trsizet probwrite readdelay writedelay trstate meaning number of operations in a transaction probability that write operation occurs average delay of a read operation average delay of a write operation state of transactions that are processed table!: transaction parameters system model the system model consists of a network manager and clients and server modules. the parameter for all the modules are summarized in table 3. parameter nclient netdelay dbaccessdelay cacheaccessdelay readhitprob meaning number of mobile clients average communication delay on the wireless network average delay to access database average delay to access cache probability of read hit at mobile client table3: system parameters experiments in this section, we present results from several performance experiments involving the protocol described in section 3. the main performance metric presented is system throughput, measured in committed transactions per second. the throughput results are, of course, dependent on the particular settings chosen for the various physical system resource parameters. thus while the throughput results show performance characteristics in what consider to be a reasonable environment, we also present other performance measures, the number of messages and the percentage of aborts, to provide additional insights into the fundamental tradeoffs between protocols. table 4 shows the values of the simulation parameters used for these experiments. ajis vol. 7, no 2 may 2000 parameter nobject cachepercent trsize(max) trsize(min) probwrite readdelay writedelay nclient netdelay dbaccessdelay cachea ccessdelay readhitprob value 1,000 5% 15 3 0, 2.5, 5, 7.5, 10, 12.5, 15, 17.5, 20, 22.5, 25 % 0.01 sec 0.04 sec 20 0.2 sec 0.05 sec 0.01 sec 0.5 sec table4: simulation parameter settings results in this section, we present the results from several performance experiments. we ran a number of simulations to compare the behavior of the proposed protocol and the protocol with periodic broadcasting. figure 3 shows the required number of messages to commit a transaction with the proposed protocol and the synchronous one. the message counts of each protocol increases in a sublinear fashion in whole range of update ratio. when the update ratio is lower than about 13%, our protocol using asynchronous broadcasting approach requires less messages than the synchronous protocol, because broadcasting is rare with low ratio of updating transactions. if the period is longer than 13%, the frequently broadcasted invalidation reports are the main factor that cause our protocol to require more messages. in the synchronous protocol, the message costs does not increase so rapidly as our protocol, because the invalidation reports are broadcasted periodically. figure3: impact of updating ratio on message counts 7.5 10 125 15 175 20 225 25 figures: impact of updating ratio on message counts ajis vol. 7, no 2 may 2000 70 60 50 o 40 c 30 20 10 -ourprotocol • synchronous protocol x' x 2.5 7.5 10 12.5 15 17.5 20 225 25 updathg ratb figure4: impact of updating ratio on aborts 12 10 •d s •our protocol • synchronous protocol 2.5 7.5 10 125 15 updathg ratb 175 20 225 25 figures: impact of updating ratio on throughput figure 4 presents the percentage of aborts of transaction that take place both at the client side and at the server. as can be seen, when updating ratio increases, the percentage gets to be significant. when the updating ratio is lower, most aborts are due to the transaction's read operation on the stale cached data. it occurs when an invalidation report is not delivered by disconnection, or when a mobile client requests commit of a transaction before it receives the invalidation report which include the data item referenced by the transaction. as the updating ratio gets higher, aborts increase with both protocols, mainly because of the increased conflicts between transactions. in case of the synchronous protocol, the percentage increases more rapidly, as the number of transactions that show conflicts in the same broadcasting period increases. figure 5 shows the throughput results for two protocols. as shown in this figure, the throughput degrades with increasing write operations, because of the message costs for updating transactions. when the updating ratio is very low, two protocols show almost the same performance, because there are few transactions that are aborted by the server, in both cases. however, the throughput of the synchronous protocol degrades more rapidly as the updating ratio increases, because of increasing aborts of transactions. with synchronous approach, lots of aborts caused by conflicts may happen, as the server checks the conflicts between transactions once in a period. in our protocol, most of such aborts can be avoided by broadcasting invalidation reports immediately after receiving a commit request. 10 ajis vol. 7, no 2 may 2000 conclusions we have presented a new protocol to support transactions in mobile client-server environments. the proposed protocol adopts broadcasting cache invalidation strategy to support the stateless server scheme, a common feature in mobile environments. in this paper, we adopt asynchronous approach as the way of broadcasting invalidation reports. with this approach, our protocol can reduce communication costs when updating is rare, and can avoid most aborts of adjacent conflicting transactions without additional communications on the wireless network. simulations were conducted to evaluate the performance of the protocol. our performance experiments show that relative merit of the proposed protocol under various updating ratio. references acharya s.and r. alonso (1995) "the computational requirements of mobile machines," proc. of the international conference of engineering of complex computer systems alonso r. and h. korth (1993) "database system issues in nomadic computing," proc. of the acm sigmod conference on management of data badrinath b. and t. imielinsky (1992) "replication and mobility," workshop on the management of replicated data barbara d. (1997) "certification reports: supporting transactions in wireless systems," proc. of ieee international conference on distributed computing barbara d. and t. imielinsky (1994) "sleepers and workaholics: caching in mobile environments," proc. of acm sigmod conference on management of data bernstein a.j. and p/ m. lewis (1993) concurrency in programming and database systems, jones and bartlett publishers berstein p.a. v. hadzilacos and n. goodman (1987) concurrency control and recovery in database systems addison-wesley chu s.i. and m. winslett (1994) "minipage locking support for object-oriented page-server dbms," proc. international conference on information and knowledge management dunham m.h. and a. s. helal (1995) "mobile computing and databases: anything new?," acm sigmod record elmagarmid, a., j. jing and o. bukhres (1995) "an efficient and reliable reservation algorithm for mobile transactions," proc. of international conference on information and knowledge management elmagarmid, a., j. jing and t. furukawa (1997) "wireless client/server computing for personal information services and applications," acm sigmod record franklin m.j.(1993a) caching and memory management in client-server database systems ph.d. thesis, dept of computer science, university of wisconsin franklin m.j.and m. j. carey (1992) "client-server caching revisited," proc. of the international workshop on distributed object management (published as distributed object management, ozsu, dayal, valduriez, morgan kaufmann, 1994) franklin m.j., m. j. carey and m. livny (1992) "global memory management in client-server dbms architecture," proc. of the international conference on the very large data bases franklin m.j., m. j. carey and m. livny (1993b) "local disk caching for client-server database systems," proc. of the international conference on very large data bases gottemukkala v., e. omiecinski and u. ramachandran (1996) "relaxed consistency for a client-server database," proc. of international conference on data engineering imielinsky t. and b. r. badrinath (1993) "data management for mobile computing," acm sigmod record v. j. jing o. bukhres and a. elmagarmid (1995) "distributed lock management for mobile transactions," proc. of international conference on distributed computing systems liskov b., m. day and l. shrira (1992) "distributed object management in thor," proc. of the international workshop on distributed object management, (published as distributed object management, ozsu, dayal, valduriez, morgan kaufmann, 1994) narasayya r. (1993) "distributed transactions in a mobile computing system," proc. of interbational conference on parallel and distributed systems pitoura e. and b. bhargava (1994) "building information systems for mobile environments," proc. of international conference on information and knowledge management pitoura e. and b. bhargava (1995) "maintaining consistency of data in mobile distributed environments," proc. of international conference on distributed computing systems tannenbaum a.s. (1995) distributed operating systems, prentice hall wang y. and l. rowe (1991) "cache consistency and concurrency control in a client/server dbms architecture," proc. of the acm sigmod international conference on the management of data wilkinson k. and m. neimat (1990) "maintaining consistency of client cached data," proc. of the international conference on very large data bases 11 ajis vol. 7, no 2 may 2000 wong m.h. and w. m. leung (1995) "a caching policy to support read-only transactions in a mobile computing environment," technical report, the chinese univ. of hong kong, dept. of computer science wu k.l., p. s. yu and m. s. chen (1996) "energy-efficient caching for wireless mobile computing," proc. of the international conference on data engineering 12 ajis vol 4 no. 1 a partial test and development of delone and mclean's model of is success3 peter b. seddon department of information systems, the university of melbourne min-yen kiew rating agency malaysia berhad, kuala lumpur. abstract delone and mclean's (1992) comprehensive review of different information system success measures concludes with a model of interrelationships between six is success constructs. this paper critically examines the meaning of four of these constructs and the evidence of relationships between them. it then provides results from empirical tests of these relationships. tests are conducted using both conventional ordinary least squares regression path analysis and structural equation modeling with substantially similar results. the empirical results provide substantial support for the "up stream" two thirds of delone and mclean's model. three factors. system quality, information quality, and usefulness, are found to explain 75% of the variance in the overall user satisfaction measure. the empirical results also provide substantial support for the use of usefulness as an is success measure, and of the hitherto-unreported importance of "importance of the task" in user perceptions of is usefulness. key words and phrases: information system success, user satisfaction, perceived usefulness, importance of task, information quality, ease of use. introduction delone and mclean's (1992) comprehensive review of different information system success measures makes two important contributions to our understanding of information system (is) success. first, it provides a scheme for classifying the multitude of is success measures that have been used in the literature into six categories. second, it suggests a model of interdependencies between these categories. commenting on their figure 2, reproduced here as figure la, delone and mclean say (p.88): the l/s success model proposed in figure 2 is an attempt to reflect the interdependent, process nature of i/s success. rather than six independent success categories, there are six interdependent dimensions to us success. this success model clearly needs further development and validation before it could serve as a basis for the selection of appropriate i/s measures. the last sentence in the above quotation provides the motivation for this paper. we do not address the two right-hand constructs4' but we do examine the meaning of, and relationships between, the other four. 3 acknowledgments: the authors with to thank participants at workshops at the university of melbourne, curtin university, university of british columbia, michigan state university, and the international conference on information systems, vancouver, december, 1994 for the suggestions regarding earlier versions of this paper. comments from the four reviewers for the icis 94 conference also improved this paper considerably. finally, thanks to all those who responded to the survey, and so made this study possible. 4melone (1990) provides a detailed discussion of the relationships between use, satisfaction, and is effectiveness that considers the right-hand constructs in figure la in more depth. at the empirical level, goodhue and thompson (1995) explore links between it use and job performance. 90 ajis vol 4 no. 1 system quality information quality individual impact organizational impact figure la: delone and mclean's model of is success (delone and mclean (1992), figure 2, p.87) system quality information quality use user satisfaction individual impact organizational impact figure ib: path model that seems to be implied by delone and mclean's model if the six categories of is success measure in figure la are treated as variables, the causal model implied by delone and mclean's paper is as shown in figure ib. each line in figure ib is, in effect, an hypothesis about an independent causal relationship between the constructs in the model. thus the dotted-line box in figure ib contains six hypotheses (six arrows on the path diagram) linking the four success measures system quality, information quality, use, and user satisfaction. this paper provides an empirical test of relationships between these four variables as perceived by over one hundred users of one specific information system. for reasons given below, the model actually tested is as shown in figure 2. the three differences between delone and mclean's model (figure ib) and the model tested (figure 2) are as follows: (a) use in delone and mclean's model has been replaced by usefulness; (b) a new variable, system importance, has been added to delone and mclean's model to help explain variations in users' perceptions of usefulness and user satisfaction; (c) the simultaneous causality between use and user satisfaction in delone and mclean's model has been replaced by one-way causality, i.e., we argue that usefulness causes user satisfaction, not vice versa. 91 ajis vol 4 no. 1 h6 importance of the system h7 system quality h3 information quality h2 h4 hi usefulness h5 user satisfaction figure 2: the model of user satisfaction tested in this study (this is a slightly modified version of the variables within the dotted box in fig. ib. use has become usefulness, and importance of the system has been added.) the meaning of the five constructs used in figure 2, and the theory supporting the relationships between them, are discussed in the second section of this paper (the next section). to test the model, data were collected from 104 users of a university's departmental accounting system. discussion of methodology used for testing the model, measurement of variables, and test results are presented in the third section of this paper. the model is analyzed using both conventional ordinary least squares (ols) linear regression and using a lisrel-like structural equation modeling (bollen 1989; joreskog and sorbom 1988) package called amos5 (arbuckle, 1994a). finally, the paper concludes with a brief discussion of the implications of these results for the measurement of is success in future research. our conclusions are that (a) there is substantial support for the up-stream side of delone and mclean's model, (b) of the four measures examined, user satisfaction is the most general individual-user perceptual measure of information system success, and (c) future researchers will need to control for task importance whenever they measure the usefulness of an information system (because systems that perform more important tasks are perceived as more useful, irrespective of the quality of the actual system). defining constructs and theorizing about their relationships replacing use in figure 1 by usefulness in figure 2 as reported by delone and mclean (1992), many researchers have used use as an objective measure of system success. the implication is that if a system is used, it must be useful, and therefore successful. however, non-use does not necessarily mean a system is not useful, it may simply mean that the potential user has other more pressing things to be done. in addition, as delone and mclean point out, "usage, either amos stands for "analysis of moment structures". 92 ajis vol 4 no. 1 perceived or actual, is only pertinent when such use is voluntary" (p.68). when usage is compulsory, the number of hours a system is used conveys little information about system usefulness, and so success. reflecting on the relevance of use as an indicator of system success in some situations and its irrelevance in others, we conjectured that the underlying success construct that researchers have been trying to tap is usefulness, not use. use is a good proxy for usefulness in situations where a tool is used, and use is not mandatory. it then provides a simple objective measure of success. however, in cases where a system is not used during the period of study, or where usage is mandatory (as in the accounting system examined in this study), we argue that usefulness continues to be a meaningful measure of success, even though use does not. we therefore decided to measure usefulness, not use, in our test of delone and mclean's model (figure 2)6. the first four hypotheses in figure 2 (h1-h4) the first four (independent) hypotheses to be tested in this paper follow directly from delone and mclean's model: hi: increases in information quality will cause increases in user satisfaction h2: increases in system quality will cause increases in user satisfaction h3: increases in system quality will cause increases in usefulness h4: increases in information quality will cause increases in usefulness there are four constructs involved. information quality is concerned with such issues as the timeliness, accuracy, relevance, and format of information generated by an information system. system quality is concerned with whether or not there are "bugs" in the system, the consistency of the user interface, ease of use, response rates in interactive systems, documentation, and sometimes, quality and maintainability of the program code. usefulness of an is is "the degree to which a person believes that using a particular system would enhance his or her job performance" (davis 1989). satisfaction is "the result of the individual taking outcomes that have been received and evaluating them on a pleasant-unpleasant continuum" (naylor et al. 1980). there is quite strong support in the literature, both theoretical and empirical, for hypotheses h1-h4. a brief summary of seven relevant studies follows. first, based on the work of bailey and pearson (1983), ives et al. (1983) discuss the development of two instruments for measuring user satisfaction. the longer of these instruments was factor analyzed into five factors: edp staff and services, information product 1, vendor support, information product 2, and knowledge or participation1. the shorter instrument later validated by baroudi and orlikowski (1988) factor analyzed into three factors: edp staff and services, information product, and user knowledge or participation. a number of researchers have been critical of the use of these multi-factor measures of user satisfaction. treacy (1985) described the factors as "imprecise and ambiguous". galletta and lederer (1989) argued that because of the heterogeneity of the items, results from the ives et al. (1983) instruments should be "interpreted with caution". our explanation of the factors that emerge from the ives et al. instruments is that they measure the independent variables that bailey and pearson (1983) and ives et al. (1983) thought were likely to cause satisfaction, not user satisfaction itself. in particular, we view the presence of so many questions about information quality in the ives et al. user satisfaction instruments as providing theoretical support for hypothesis 1 in figure 2. second, in a recent empirical study, seddon and yip (1992) constructed a four-item instrument that attempts to measure user satisfaction directly8. this they compared with the factors from ives et al.'s short form instrument. seddon and yip found that for users of computer-based accounting systems, factors such information quality, usefulness, and user knowledge of various system features explained over 70% of the variance in their user satisfaction measure9. the t-statistics in their regression 7.3 (p.89) indicate that 6 correlations between use and usefulness are reported later in the paper, in table 2. 7 ives et al. actually used the term "involvement" not "participation". to avoid confusion with barki and hartwick's (1989) user involvement, a construct that is closely related to the system importance construct used in hypotheses 6 and 7, we have renamed the ives et al. term "user participation" in this paper. 8these four questions (as they apply to das, the system for which data were collected for this study) are reproduced in appendix 2 to this paper, section c. 9such high correlations of multi-factor measures and overall satisfaction measures are not uncommon. bailey and pearson (1983, p.536) report a correlation of 0.79 between their normalized importance-weighted measure of user satisfaction (based on up to 39 questions) and their single-scale measure of overall "self-assessed" satisfaction. 93 ajis vol 4 no. 1 information quality (t-statistic=7.48) is an important determinant of satisfaction. this provides strong empirical support for hypothesis 1 in figure 2. third, doll and torkzadeh (1988) developed a measure of end user computing satisfaction (eucs) that asked ten questions about information quality and two about ease of use. these questions factor analyzed into five factors. the first four factors relate to the construct that we call information quality: information content, accuracy, format, timeliness. the fifth factor is ease of use, a component of system quality. as with the ives et al. instruments, we contend that doll and torkzadeh's instrument is actually measuring two variables that are causes of user satisfaction. thus as with the ives et al. instrument, we view doll and torkzadeh's decision to include questions concerning information quality and system quality in their measure of end user computing satisfaction as providing theoretical support for hypotheses 1 and 2 in figure 2. fourth, in their empirical study, seddon and yip (1992) report that doll and torkzadeh's eucs explained over 70% of the variance in their four-item user satisfaction measure, but that after information quality was included in the regression, ease of use was not significant (regression 7.2). this provides further empirical support for hypothesis 1, but not for hypothesis 2. fifth, davis (1989) has provided is researchers with two highly reliable measures of perceived usefulness and ease of use. davis was interested in explaining peoples' ex ante decisions to use information technology, but the measures seem equally applicable to ex post evaluations of information systems. concerning ease of use (an important component of system quality), davis found significant correlations (p<0.001) between ease of use and usefulness for three of four systems studied (table 8, p.332), and suggests that "ease of use influences usage indirectly through its effect on usefulness" (p.330). davis's work provides both theoretical and empirical support for hypothesis 3 in figure 2 (i.e., that increased system quality is associated with increased usefulness). sixth, in support of hypothesis 4, larcker and lessig (1980, p.123), franz and robey (1986), and kraemer, danzinger, dunkle, and king (1993, q.2a, p. 133) have all argued that increased information quality will lead to increased usefulness. franz and robey (1986) include two questions in their perceived usefulness instrument (p.353) that suggest they think higher information quality implies higher usefulness. the first question asks: "to what extent does this system overload you with more data than it seems you can possibly use?". the second asks: "to what extent does this system provide report(s) to you that seem to be just about exactly what you needt. the comments from larcker and lessig and kraemer et al., and the inclusion of the two questions just cited in franz and robey's questionnaire all provide theoretical support for hypothesis 4 in figure 2. seventh, kraemer et al. (1993) studied factors that influence the perceived usefulness of computer-based information (cbi). they used regression analysis to assess the relative importance of factors that affect usefulness. for data from 211 operations managers in public organizations, they report: "finally, from the foregoing factors, cbi accessibility, cbi quality, and reliance on experts were found to have the most significant influence on the perceived usefulness of cbi" (p. 139). this provides limited empirical support for hypothesis 410. usefulness causes user satisfaction, not vice versa (h5) delone and mclean's model (figure la) suggests that there is a two-way causal relationship between use and satisfaction. a simple duplication of this two-way relationship in figure 2 would imply a two-way causal relationship between usefulness and satisfaction. is this valid, or does the change from the use construct to usefulness introduce a subtle change in the causality relationships? as we could think of no statistical test to determine whether usefulness causes satisfaction, or vice versa, we were forced to resort to semantics to try to answer this question. the conclusions from this back-to-first-principles analysis are presented in the next two paragraphs. something is useful if it provides future benefits". a car is useful for getting to work. a bicycle is less useful for the same task because it goes slower and you may get wet if it rains. a car that won't start is not useful for getting to work today, but it will be useful again in the future once someone has got it working. a bicycle is not '"support is "limited" because in addition to asking questions similar to our questions q.6, 7, 8, and 9 on information quality (see appendix 2) under the heading "information quality", kraemer et al. also asked whether "the computer makes new information available to me that was not previously available". this idea, which was absent from our study, appears to have been the most significant factor in kraemer's regression (their table 7, p. 141). "davis (1989, p.320) expresses the same idea when he says something is useful if it is "capable of being used advantageously". 94 ajis vol 4 no. 1 useful for getting to work today if you don't know how to ride. consistently, across many examples, we found that usefulness is concerned only with the future benefits of performing some task. costs are much less important. for instance, your new $30,000 automobile may be judged to be marginally more useful than the reliable old $6,000 automobile it replaced, but not by five times. the benchmark for judging usefulness of a tool is that the value of the benefits flowing from its use in some specific task must exceed zero. the benchmark for judging satisfaction is different. user satisfaction is the net feeling of pleasure or displeasure that results from aggregating all the benefits that a person hopes to receive from interaction with the information system. each user has a set of expected benefits or aspirations for the information system. to the extent that the system meets or fails to meet each of these aspirations, the user is more or less satisfied. at a minimum, a tool is expected to be useful. beyond that, the more useful the tool, the more likely the user is to be satisfied with it. however, satisfaction reflects a wider set of expected benefits or aspirations than mere usefulness. for instance, we are more likely to be satisfied with our new $30,000 automobile than with the old $6,000 one, even though the annual cost of ownership is much higher. assuming the new car is only marginally more useful than the old, this example implies that we must be valuing other non-usefulness benefits (such as comfort and status) in determining satisfaction. considerations along these lines lead us to believe that increases or decreases in usefulness will lead to increases or decreases in user satisfaction with information systems, but not vice versa (because some increases in satisfaction are unrelated to usefulness). this is the basis for hypothesis 5 in figure 2: h5: increases in usefulness will cause increases in user satisfaction in effect, we regard delone and mclean's model as saying that user satisfaction responds primarily to three types of aspirations that people have for information systems: people want their information systems to provide high quality information (hi), to be of high quality (h2), and to be useful in their jobs (h5). what support is there in the literature for the hypothesis that usefulness causes satisfaction, but not vice versa? whilst many researchers have studied the relationship between use (a behaviour) and satisfaction (an attitude), delone and mclean's model is concerned with use as a measure of success, i.e., with usefulness (a belief)12. surprisingly few researchers have considered the relationship between usefulness and satisfaction. apart from bailey and pearson (1983), who used a question about utility (the relative balance between cost and usefulness) in their measure of satisfaction, and goodhue (1986), who speculated that there might be weak links in both directions between satisfaction and usefulness, the most helpful study we could find was by franz and robey (1986). franz and robey used perceived usefulness as the dependent variable in their study of user participation and organizational context. in discussing the choice of independent variable, they say: our questions were designed to assess perceptions of usefulness rather than more-general attitudes [robey 1979; schewe 1976] or satisfaction [bailey and pearson 1983; ives et al. 1983] (p.338). it is clear from this sentence that they believe that satisfaction is a more general concept than usefulness. however, it is not clear how they think the two concepts are related. since franz and robey's was the only study we could find that offered clear insights into the relationship between usefulness and user satisfaction, we were forced to rely on the "first-principles" analysis above to justify our interpretation of the arrows in delone and mclean's model, i.e., to justify hypothesis 5. system importance used to explain variance in usefulness and user satisfaction (h6 and h7) because delone and mclean's model was proposed as a way of interrelating various measures of is success, it does not consider factors that might influence peoples' evaluations of success. however, as all four constructs in the dotted line box in figure 2 are perceptual, it seems highly likely that users' opinions about the relevance of the system to their own goals and aspirations will influence their opinions about the value and so success of the system. for example, if what the system does is unimportant to the user, there seems little chance that the user will perceive the system as useful, no matter how well designed it is, or how easy it is to use. conversely, if the task the system supports is perceived as very important, a poor system may be perceived as useful, even if it is quite user unfriendly. in this study of success measures, it therefore seems essential to consider the interests of the individuals being asked to evaluate the information system. how should users' interests be incorporated into figure ib? for help in this regard, we turned to barki and hartwick's notion of user involvement. barki and hartwick's (1989) revolutionary paper presents a strong 12goodhue (1986) makes the point that usefulness (which he calls is satisfactoriness) is a belief (a relatively objective judgment about whether a tool can assist with the job), whereas satisfaction is an attitude (a "predisposition to respond favorably or unfavorably to" the system (melone 1990)). 95 ajis vol4 no. 1 case for distinguishing between user participation, which they define as "participation in the system development process" (p.53), and user involvement, which they define as "the subjective psychological state" of the user "when he or she considers a system to be both important and personally relevant" (p.53). this user involvement concept is very similar to task-relevance concept we had in mind, so we adopted the importance component of user involvement for our study. intuitively, we felt that people who believed that the task performed by the system was important would also perceive the system that performed it to be useful. the more important the task, the more useful the system. thus h6 proposes that the greater the user's perception of the importance of the task, the more useful the system will be perceived to be. additional support for h6 comes from larker and lessig (1980, p.123), in the context of information usefulness. larker and lessig used the term "perceived importance" to refer to "the quality that causes a particular information set to acquire relevance to a decision maker". consistent with our h6, they go on to argue that perceived importance will "tend to increase the perceived usefulness of the set". the link from importance of the task to satisfaction is less clear. provided the system works, the more important the task to the user, the more satisfied he or she is likely to be (importance up, satisfaction up). on the other hand, if the system does not work and the task is important, the user may be very dissatisfied (importance up, satisfaction down). finally, if the task is unimportant, the user's threshold for satisfaction may be so low (i.e., the user would be indifferent to the system) that satisfaction scores might be moderate (importance down, satisfaction indeterminate). these last two scenarios would reduce correlations between importance and user satisfaction. on balance, since the system we proposed to test did work (though not very well) we expected the correlation to be positive. this is the basis for h7 in this study. perceptions of system quality and information quality are less likely to be coloured by perceptions of system importance than perceptions of usefulness and satisfaction. for instance, twenty years ago, card-punch machines were the normal everyday way of communicating with computers. they were easy to use, and they were useful. today, they are no less easy to use, but now that card decks are no longer used for communicating with computers, card-punch machines are considered useless. based on this analogy (in which usefulness changes as importance changes but ease of use does not), we argue that system quality (of which ease of use is a central component) is unlikely to be influenced by the importance of the system to the user, i.e., system importance. similarly, we argue that information quality is unlikely to be influenced by system importance. information quality is concerned with the timeliness, accuracy, relevance, and format of information generated by an information system. these are relatively objective qualities of information, and it seems unlikely that a user will judge information quality to be high simply because he or she thinks the task performed by the system is important. we therefore expect system importance will be positively associated with both usefulness and satisfaction but not with system quality and information quality. these expectations are shown as hypotheses 6 and 7 in figure 2. h6: increases in system importance will cause increases in usefulness h7: increases in system importance will cause increases in user satisfaction surveying the literature for support for these hypotheses, we found two papers of particular interest. first, in the empirical study, jackson, chow, and leitch (1993) studied factors affecting behavioural intention to use an information system. they used zaichkowski's (1985) involvement instrument to measure user involvement, and found a highly significant relationship between user involvement and perceived usefulness (t-statistic = 6.52). there are some difficulties with the use of all items in zaichkowski's involvement instrument in testing the relationship between user involvement and usefulness because some items in zaichkowski's instrument actually measure usefulness directly. nonetheless, the large t-statistic in the jackson et al. (1993) study provides considerable empirical support for the use of user involvement, hence importance™ to explain variance in perceived usefulness (h6). 13 recall that importance and personal relevance are the two major components of barki and hartwick's user involvement. 96 ajis vol 4 no. 1 second, kappelman and mclean (1991), used the same zaichkowski instrument to investigate the relationship between user involvement and user satisfaction. they report a highly significant association (p<0.001). this provides empirical support for our hypothesis 7. testing the model of user satisfaction shown in figure 2 the questionnaire to test the model in figure 2, we prepared a questionnaire based on a number of standard instruments. the questionnaire was to be administered to all users who had completed the training course for our university's recently-implemented departmental accounting system, das. details of all questions used in measuring variables for this study are presented in appendix 2. there are six groups of questions that measure seven variables as follows. first, the eight questions on system quality are based on doll and torkzadeh's (1988) two questions on ease of use, four of davis's (1989) questions on perceived ease of use14, plus three additional questions -all phrased in present (not future) tense15. second, the ten questions on information quality are all from doll and torkzadeh (1988). third, the four questions on overall satisfaction are from seddon and yip (1992). fourth, the six questions on perceived usefulness are from davis (1989). fifth, the five questions on system importance are from zaichkowski (1985) via kappelman and mclean (1991). data for this study were collected before barki and hartwick (1994) published their instruments for measuring user involvement, so we did not use their instrument for measuring importance. however, because respondents answered all 14 questions from kappelman and mclean (1991), it was possible to construct a measure very similar to barki and hartwick's. in fact, three of their five importance questions and one of their four personal relevance questions (1994, table 2a, p.70) are included in our five-item importance measure. finally, the last two variables used in this study are use if not mandatory and 75 use %16. single questions to measure these two variables were included to test the relationships between use (the variable required in delone and mclean's model) and usefulness (the variable that we argued earlier would be more relevant in situations where usage was mandatory). use if not mandatory is a hypothetical intention-to-use measure because it asked users to pretend that usage of das was not mandatory. based on tests of davis's technology acceptance model (tam) (davis (1989, 1993), davis et al. (1989), adams et al. (1992), segars and grover (1993), and igbaria et al. (1995)) one would expect quite a high correlation between usefulness and use if not mandatory. is use % measures self-reported actual use of das. because some respondents were part time, is use % is measured as a percentage of the respondent's average working week. details of the system and its users as noted above, the system we chose for testing the model in figure 2 was the university's recentlyimplemented departmental accounting system, das. we chose this particular system because we knew there had been some difficulties with its implementation, and a wide range of usefulness and satisfaction scores was likely17. a second attraction was that the same system was in use in all departments, and all users had been trained by the same trainers. there were therefore a smaller number of extraneous factors that could cause variance in the usefulness and satisfaction scores. the goal of das is to keep track of all of each department's diverse range of income and expenditure, so that those who are managing various projects, research grants, etc., can know, at any time, how much money they have left to spend. built in ingres at a fully-implemented cost of over $500,000, das runs on a central computer and acts as a front end to the university's central accounting system. about half the implementation cost was for user training and support during the twoyear phased implementation of the system. the typical das user is a relatively senior clerical officer in each department or faculty office who uses the system for about 4-5 hours per week for maintaining the department's accounting records. at the time of the survey, 169 14 the question on das being "easy to use" is common to both doll and torkzadeh (1988) and davis (1989). 15 note that these questions about system quality are all based on perceived ease of use. since the same system is being used by all users, the objective technical characteristics of the system are the same for all users, so there was no point asking questions about technical aspects of system quality. 16 these variables are based on questions 1 and 3 in section f of appendix 2. 17 this opinion proved to be correct. mean user satisfaction scores for data from this study were 4.68 (s.d. 1.57) on a scale from 1 to 7. mean scores when seddon and yip (1992) used the same instrument to measure user satisfaction with commercial general ledger systems was 5.56 (s.d. 1.16). our users were significantly less satisfied. 97 ajis vol 4 no. 1 people had completed the das training course: 152 were women, 17 were men; 118 were accessing das through apple macintosh computers in their departmental offices, and 51 from ibm pcs. data collection after ten face-to-face trials, the questionnaire was mailed to the remaining 159 trained das users. 134 questionnaires were returned, but only 94 were useful for data analysis. non-useable responses were for the following reasons: some departments were not yet using das, some trained users were on leave, some were no longer responsible for das, and some had resigned. combining the 10 initial responses with the 94 returned by mail gave 104 responses for statistical analysis18. for each of the 35 questions asked, responses from early and late respondents (83 and 21 respondents, respectively) were compared using the mann-whitney u statistic (siegel and castellan 1988). none of the u statistics from these 35 questions indicated a significant difference, at the 5% level, between early and late respondents. because of the high response rate and the lack of significant differences between early and late respondents, non-response bias is not considered a problem for this study. descriptive statistics for distributions of responses for each variable are given in table 1. principal components factor analysis (no further details in this paper) of the eight questions on system quality showed that one question (question 7 in appendix 2) had a loading of only 0.25 on the main factor. it was therefore deleted, and all statistics that follow are based on the remaining seven-item measure of system quality. reliabilities (cronbach alpha) and the pearson correlation matrix for the variables used in the study are shown in table 2. was it incorrect to argue (as we did in the previous section) that usefulness should be used in place of use in delone and mclean's model? the answer to this question proceeds in two steps. first, table 2 shows that the correlation between usefulness and hypothetical use if not mandatory was 0.699. in other words, usefulness explained approximately 50% of the variance in use if not mandatory. this is entirely consistent with results from the tam studies, and suggests that, in this mandatory-use setting, usefulness is tapping the same construct that delone and mclean sought to measure with use. second, the correlation between usefulness and self-reported /s use % was only 0.29519. in addition, it is obvious from respondents' written comments (at the end of each questionnaire) that usage was determined by the needs of different departments, not by the desires of the users. 18 one question was rephrased after the ten trials. this is why there are only 90 responses to the question concerning use if not mandatory in table 2. 19 the reason these two variables are significantly correlated (r=0.295, p=0.003) is probably because those who worked more hours per week with das viewed the system as more important (r=0.415, rxo.ooo) and as will be shown shortly, more important systems are viewed as more useful. correlations between is use % and the other success measures in table 2 are low. 98 ajis vol4 no. 1 table 1: descriptive statistics variable ;; ; 1. system quality 2. information quality 3. system importance 4. usefulness 5. satisfaction 6. use if not mandatory 7. is use % meab 4.21 4.66 4.76 3.92 4.67 4.14 20.1 'sd. 1.46 1.29 1.42 1.87 1.58 2.19 21.4 histogram i [tffftc "*• ^llttfk [v -^tunj! rbr lll-rrfhilh1 .^tnftflr ;]tjd!]h,.p jl|pph~-^_^ mia. 1.29 1.60 1.00 1.00 1.00 1.00 0% max. 7.00 7.00 7.00 7.00 7.00 7.00 95% kurtosis -.89 -.60 -.26 -1.02 -.34 -1.39 2.50 se kurt 0.48 0.48 0.48 0.48 0.48 0.50 0.49 skewness .05 -.34 -.40 .04 -.58 -.22 1.72 s.e. skew 0.24 0.24 0.24 0.24 0.24 0.25 0.24 table2: reliabilities (cronbach alpha) and pearson correlation matrix (two-tailed test, significance in brackets, spearman correlations very similar) variable ; 1. system quality 2. information quality. 3. system importance 4. usefulness 5. satisfaction 6. use if not mandatory 7. is use % valid obs. 101 100 101 100 101 90 97 no. of items 7 10 5 6 4 1 1 crobbacb 0.935 0.951 0.885 0.986 0.921 1 .549 (.000) .409 (.000) .644 (.000) .708 (.000) .603 (.000) .260 (.010) 2 .315 (.001) .546 (.000) .754 (.000) .478 (.000) .105 (.309) 3 .623 (.000) .438 (.000) .631 (.000) .415 (.000) 4 5 .713 (.000) 6 .699 .572 (.000) (.000) .295 .169 .283 (.003) (.098) (.008) results: use versus usefulness in our opinion, the high correlation of usefulness with use if not mandatory, and the low correlations between the various perceptual success measures of is success and actual mandatory is use % provide quite strong empirical support for delone and mclean's argument that when measuring is success, "usage, either perceived or actual, is only pertinent when such use is voluntary" (1992, p.68). the remainder of this analysis therefore proceeds on the understanding that when usage is mandatory, usefulness is the most appropriate usage-related measure of is success. 99 ajis vol 4 no. 1 results: path analysis results from path analysis of the model in figure 2 are reported in table 3. path coefficients in column 4 were computed using two ordinary least squares (ols) linear regressions, the first with usefulness as the dependent variable (hypotheses 6, 3, and 4), and the second with user satisfaction as the dependent variable (hypotheses 7, 2, 1, and 5). path coefficients in column 5 were calculated using maximum likelihood estimation in the structural equation modelling (sem) package called amos20. to avoid reliance on any assumptions of multivariate normality, standard errors in column 5 were calculated using bootstrap simulation with 1,000 replications21. the path coefficients from the two methods of analysis are very similar, although the t-statistics for the sem in column 5 are considerably lower than those for ols regression. the reason for including two sets of path coefficients (ols and sem) in table 3 is that structural equation modeling is becoming much more common in the information systems literature (e.g., lisrel: adams et al. 1992, segars and grover 1993, chin and todd 1995; pls: rivard and huff 1988, thompson et al. 1991 & 1994, igbaria et al. 1995), and many researchers probably wonder how the two/three methods of analysis compare. most studies opt for just one method of analysis, a decision that leaves the reader without any basis for gaining an intuitive understanding of the relative merits of one method over another. hence the two columns in table 3. a detailed comparison of the techniques used for calculating the ols and sem path coefficients is presented in appendix 1. despite the differences in analysis techniques, it is apparent from table 3 (and figure 3, which presents the results graphically) that for this particular data set, the conclusions one would draw from either analysis technique are the same. the results in table 3 provide quite strong support for five of the hypotheses in figure 2. the t-statistics in columns 4 and 5 are so large that we have confidence in rejecting the null hypotheses (of no association between the constructs) for hypotheses 1, 2, 3, 5, and 6. support for hypothesis 4 was weaker, however, and after controlling for the effect of importance on usefulness there is no support for hypothesis 7. with the benefit of hindsight, one can see that the lack of significance in hypothesis 7 is consistent with the difficulties with our theoretical analysis (see end of the hypothesis development section). perhaps kappelman and mclean's correlation was so highly significant (p<0.001) because the systems they studied were all working well — the "importance up, satisfaction up" case discussed in the development of hypothesis 7. conclusions for the individual user of a specific information system, our results provide considerable support for the upstream side of delone and mclean's model of is success. in effect, delone and mclean's model may be interpreted as saying that user satisfaction is a response to three types of user aspirations for an information system: information quality (hi), system quality (h2), and usefulness (h5). as delone and mclean predicted, these three factors explained a large proportion of variance in user satisfaction (over 70% in this study). 20 amos' coefficients are claimed to be almost identical ~ equal to three decimal places ~ to those from joreskog and sorbom's (1988) lisrel. 21 "bootstrap simulation" means randomly selecting different combinations of 100 records from the 100 observations. sampling is with replacement, so in a given bootstrap sample of 100 records, some records will be selected more than once and other records will not be selected at all. typically, about 60 distinct records appear in each sample. in some samples, some records may appear as many as five times. 100 ajis vol 4 no. 1 table 3: path coefficients for the model in figure 2 (1) (2) (3) (4) (5) h6 h3 h4 path , from system importance system quality information quality to usefulness usefulness usefulness r2 for regression with usefulness as dependent variable ols regression (using spss) (n=98) 0.427 (6.028) *** 0.336 (4.169) *** 0.225 (2.927) ** 0.612 (adj.=0.600) structural equation modeling (in amos) (n=100) 0.449 (4.777) *** 0.332 (3.648) *** 0.214 (2.326) * 0.628 h7 h2 hi h5 system importance system quality information quality usefulness user satisfaction user satisfaction user satisfaction user satisfaction r2 for regression with user satisfaction as dependent variable 0.010 (0.149)ns 0.266 (3.674) *** 0.451 (6.798) *** 0.287 (3.396) *** 0.739 (adj. =0.727) chi-square per degree of freedom adjusted goodness of fit index root mean square residual 0.013 (0.383) ns 0.285 (3.132)** 0.463 (5.861)*** 0.263 (2.267) ** 0.781 880/454=1.94 0..63 0.19 rows in the above table are sequenced to correspond with figures 2 and 3. path coefficients and critical values for the structural equation modeling, column 5, were estimated using 1,000 "bootstrap" samples (see text). tstatistics/critical values are in brackets. significance levels are based on one-tailed tests because all path coefficients are expected to be positive. *** p<0.001, ** p<0.01,* p<0.05, ns = not significant 101 ajis vol 4 no. 1 (a) 0.427*** the system system quality information quality \ (a) 0.015ns \ \ (.b) 0.010ns \ \ (a) 0.336*** \ \ (b) 0.332 *** \ \ (a) 0 266*** \ \ / (b) 0.285** \ v (a) 0.225** /\\ (b) 0.214* / \ (a) 0.451*** (b) 0.463*** \ r2: (a) 0.6 12 . usefulness (b) 0.628 (a) 0.288*** (b) 0.263*^ (a) 0.739 (b) 0.781 user satisfaction / figure 3: results of path analysis case (a): analysis using ols regression in spss, n=98 (listwise deletion of missing) case (b): analysis using structural equation modelling in amos (like lisrel), n=100 significance levels: *** p<0.001,** p<0.01,* p<0.05, ns = not significant to the best of our knowledge, no prior research has identified the importance of the importance of. the task construct to a system's perceived usefulness and use. importance of the task is independent of the system, yet as shown in figure 3, it has a major influence on perceived usefulness. future research, both in the prediction of information system use (via usefulness), and in the investigation of links between user participation, user involvement*2 and system success (barki and hartwick 1994, hartwick and barki 1994), will need to control for the importance of task performed by the system. for instance, it is not level of user involvement that matters in user participation research. it is the change in involvement (the change in the perceived importance of a system as a result of participation) that matters. if researchers are looking for a simple, short measure of is success, which one should they choose? delone and mclean argued that multiple indicators of success are required. to some extent, we agree. is success can mean many things to many people, and researchers must measure the is success outcomes that are relevant to their own research goals. but as satisfaction appears to be the most inclusive of the four perceptual measures investigated in this study, if a single measure is required, we recommend using user satisfaction as the most general-purpose perceptual measure of system success. the four questions from seddon and yip (1992), which returned reliability coefficients (alpha) of 0.95 in their survey, and 0.92 in this study, may be all that is needed23. researchers wanting to understand why users are dissatisfied with their systems will probably find it helpful to measure at least the three causal constructs suggested by our slightly-modified delone and mclean model, namely, information quality, system quality/ease of use, and usefulness. when measuring usefulness, it will of course be necessary to control for importance of the task. references adams, d.a., r.r. nelson, & p.a. todd (1992) "perceived usefulness, ease of use, and usage of information technology: a replication", mis quarterly june, pp.227-247. 22 recall, barki and hartwick (1989, 1994) define user involvement as the importance and personal relevance of the system. 23 one of the present authors is presently attempting to develop an even more general measure. 102 ajis vol 4 no. 1 arbuckle, j.l (1994a). amos 3.51, chicago, smallwaters corp (smallwaters@acm.org). (1994b). "advantages of model-based analysis of missing data over pairwise deletion", presented at the rmd conference on causal modeling, west lafayette, indiana. bailey, j.e. & s.w. pearson (1983) "development of a tool for measuring and analyzing computer user satisfaction", management science 29, may, pp.519-529. barki, h. & j. harrwick (1989) "rethinking the concept of user involvement", mis quarterly march, pp.5263. (1994) "measuring user participation, user involvement, and user attitude", mis quarterly march, pp.59-79. baroudi, j.j., m.h. olson, & b. ives (1986) "an empirical study of the impact of user involvement on system usage and information satisfaction", communications of the acm 29, march, pp.232-238. baroudi, j.j. & w.j. orlikowski (1988) "a short-form measure of user information satisfaction: a psychometric evaluation and notes on use", journal of mis 4, spring, pp.44-59. bollen, k.a (1989) structural equations with latent variables new york, wiley. chin, w.w. & p.a. todd (1995) "on the use, usefulness, and ease of use of structural equation modeling in mis research: a note of caution", mis quarterly 19, 2, june, pp.237-246. davis, f.d. (1989) "perceived usefulness, perceived ease of use, and user acceptance of information technology", mis quarterly 13 september, pp.319-340. (1993) "user acceptance of information technology: system characteristics, user perceptions, and behavioral impacts", int. j. man-machine studies 38, march, pp.475-487. , r.p. bagozzi, & p.r. warsaw (1989) "user acceptance of computer technology: a comparison of two theoretical models", management science 35, august, pp.982-1003. delone, w.h. & e.r. mclean (1992) "information systems success: the quest for the dependent variable", information systems research 3, march, pp.60-95. doll, w.j. & g. torkzadeh (1988) "the measurement of end-user computing satisfaction", mis quarterly 12, june, pp.259-274. franz, c.r. & d. robey (1986) "organizational contexts, user involvement and the usefulness of information systems", decision sciences 17, pp.329-356. galletta, d.f (1986) "a longitudinal view of an office system failure", sigoa bulletin 7, pp.7-11. & a.l. lederer (1989) "some cautions of the measurement of user information satisfaction", decision sciences summer, pp.419-438. goodhue, d.l (1986) "is attitudes: toward theoretical and definitional clarity", proceedings of the seventh international conference on information systems, 181-194. & r.l. thompson (1995) "task-technology fit and individual performance", mis quarterly 19, june, pp.213-236. hartwick, j. & h. barki (1994) "explaining the role of user participation in information system use", management science 40, april, pp.440-465. igbaria, m., t. guimaraes, & g.b. davis (1995) 'testing the determinants of microcomputer usage via a structural equation model", journal of mis, 11,4, spring, pp.87-114. ives, b. & m.h. olson (1984) "user involvement and mis success: a review of research", management science 30, may, pp.586-603. ives, b., m.h. olson, & j.j. baroudi (1983) "the measurement of user information satisfaction", communications of the acm 26 october, pp.785-793. jackson, c.m., s. chow, & r.a. leitch (1993) 'toward an understanding of the behavioral intention to use an accounting information system", working paper, boston university school of management, may. jenkins, a.m. & j.a. ricketts (1979) "development of an instrument to measure user information satisfaction with management information systems", working paper, indiana university, bloomington november. joreskog, k.g. & d. sorbom (1988) lisrel 7: a guide to the program and applications. chicago, spss inc. kappelman, l.a. & e.r. mclean (1991) "the respective roles of user participation and user involvement in information system implementation success", proceedings of the international conference on information systems, pp.339-349. kraemer, k.l., j.n. danzinger, d.e. dunkle, & j.l. king (1993) "the usefulness of computer-based information to public managers", mis quarterly june, pp. 129-148. larcker, d.f. & v.p. lessig (1980) "perceived usefulness of information: a psychometric examination", decision sciences 11, pp.121-134. 103 ajis vol 4 no. 1 lawler, e.e. & t.h. douglas (1970) "relationship of job characteristics to job involvement, satisfaction, and intrinsic motivation", journal of applied psychology, 54, august, pp.305-312. melone, n.p (1990) "a theoretical assessment of the user satisfaction construct in information systems research", management science 36, pp. 1-16. miller, j. & b.a. doyle (1987) "measuring the effectiveness of computer-based information systems in the financial services sector", mis quarterly 11, pp. 107-124. naylor, j.c., r.d. prichard, & d.r. ilgen (1980) a theory of behavior in organizations. academic press. rivard. s. & s.l. huff (1988) "factors of success for end user computing". communications of the acm, 29,5, may, pp.486-501. schewe c.d. (1976) "the management information system user: an exploratory behavioral analysis", academy of management journal 19, pp.577-590. seddon, p.b. & s.k. yip (1992) "an empirical evaluation of user information satisfaction uis, measures for use with general ledger accounting software", journal of information systems, 75-92. segars, a.m. & v. grover (1993) "re-examining perceived ease of use and usefulness", mis quarterly 17, 4, december, pp.517-525. siegel, s. & n.j. castellan (1988) nonparametric statistics. 2d ed. mcgraw-hill. thompson, r.l., c. higgins, & j.m. howell (1991) "personal computing: towards a conceptual model of utilization", mis quarterly, 15,1 march, pp.125-143. (1994) "influence of experience on personal computer utilization: testing a conceptual model", journal of management information systems 11,1 summer, pp.167-187. treacy, m.e (1985) "an empirical examination of a causal model of user information satisfaction", center for information systems research, sloan school of management, mit april. zaichkowski, j.l (1985) "measuring the involvement construct", journal of consumer research 12, pp.341352. 104 ajis vol 4 no. 1 appendix 1: comparison of ordinary least squares (ols) regression and structural equation modeling (sem) to the non-statistician, the advantages of structural-equation models (sem) over conventional factor analysis and ordinary-least-squares (ols) regression techniques are unclear. bollen (1989, page v) describes the advantages of lisrel-style sem in the following rather vague terms: (structural equation models)... are regression equations with less restrictive assumptions that allow measurement error in the explanatory as well as the dependent variables. they consist of factor analyses that permit direct and indirect effects between factors. they routinely include multiple indicators and latent variables. in brief, these models encompass and extend regression, econometric, and factor analysis procedures. so, how much difference is there between ols and lisrel-style sem? do these differences matter? in the case of the present data set, the principal differences between the two sets of path coefficients in columns 4 and 5 in table 3 (in the main paper) are as follows: first, for the ols regression, the score for each latent variable (theoretical construct) is the mean of responses to the questions used to measure the variable24. for example, for any one respondent, the score for the latent variable called user satisfaction is calculated as the mean of that respondent's responses to the four questions shown in the appendix 2. the cronbach alpha of 0.92 in table 1 indicates that for the sample as a whole (100 observations), scores for these four questions tended to vary together very consistently. ols then assumes these average latent variable scores for the independent variables are measured without error, and minimizes the squared difference between the dependent variable's average latent variable scores and the scores predicted by the regression equation. by contrast, in lisrel-like sem, no scores are calculated for any latent variables, dependent or independent. instead, scores on individual questions are used to calculate the covariance between each question and every other question. amos then uses a combination of the researcher's path diagram and maximum likelihood estimation to minimize the difference between the sample covariance matrix and that implied by the path diagram. in figure 4, the rectangles represent responses to individual questions, and the ovals represent the latent variables. "loadings" of each question (rectangle) on its underlying latent variable (oval) are the result of this covariance-matrix-fitting process25. the chi-square coefficient reported by amos (chi-sq.=880, df=454, chi-sq/df=1.94) is the extent of misfit between the sample and estimated covariance matrices. there is a large misfit. the adjusted goodness of fit index for this model is only 0.63 (root mean square residual = 0.19), but as we have no strong theoretical grounds for re-specifying the model, we have not attempted to "tune" the model to increase the goodness of fit. second, for the ols regression, missing values are handled either by the averaging process or else the entire response is deleted (listwise deletion). in this study, if one respondent only answers three of four questions, the score for the latent variable for that respondent is the average of the three (not four as with the other respondents). if the researcher judges that too few questions have been answered for the mean of responses to be meaningful, the score for that respondent is left undefined (missing), and the entire response from that respondent is deleted by spss during listwise deletion of missing variables during estimation of the regression equation. by contrast, in structural equation modeling, covariances from each question to every other question are required before model fitting can begin. thus in sem, the treatment of missing values is handled in during calculation of the sample covariance matrix. amos provides a technique for such estimation (described in arbuckle (1994b)), but when bootstrapping is used to obtain what arbuckle describes as "more robust" estimates of standard errors (as in the data reported in table 3, column 5) this missing-value treatment is not available. thus for the data in table 3, missing values for particular questions were replaced by means for the respective questions. such mean replacement of missing values occurred for data from only 2 of the 100 respondents. (these two responses were also those deleted in spss during its listwise deletion phase.) a total of 17 missing values from 32*100 = 3,200 respondent answers (0.53%) were replaced. the effect on the covariance of replacing missing 24 an indication of the degree to which these questions tap the one underlying construct is given by cronbach's alpha coefficient. 25 the smallest loading is 0.69 between the latent variable infoqual and question iq6 in the bottom left of figure 4. the 0.48 to the left of the iq6 rectangle is the variance in iq6 explained by infoqual, i.e., 0.692. 105 ajis vol 4 no. 1 values by the mean of the remaining observations for a given question is entirely neutral. to see this, consider the formula for calculating covariance between responses to two questions xi and x2: tv v , 2rf,-=i(*l« ~*lx*2,"^2) , v i e r\ • • i icov(x,,x2) = —— , where xj = sample mean for q. 7,7 = 1,2 n — i in this case, because 100 people responded, calculation of the covariance involves a summation of 100 terms of the form (xu — x,)(x2l — x2). suppose the person completing the 88th questionnaire did not answer question x2, and that the missing score for this question was replaced by the mean of all other respondents' scores for x2, i.e., by x2 . this means the 88th of the 100 terms being summed to calculate the covariance would be zero (because (x288 — x2) = ( x2 — x2 ) = 0). thus the covariance is determined from the other 99 observations, and the mean-substituted 88lh observation has no effect on the calculation of the numerator of the covariance 106 ro cd co o) .52 'of ajis vol 4 no. 1 appendix 2: departmental accounting system (das) evaluation questionnaire please circle the appropriate number part a: system quality. strongly agree disagree 1. das is easy to use. 2. das is user friendly. 3. compared to other computer software, das is easy to learn. 4. i find it easy to get das to do what i want it to do. 5. it is easy for me to become skilful at using das. 6. i believe that das is cumbersome to use. 7. my using das require a lot of mental effort. strongly 1 1 1 1 1 1 1 2 2 2 2 2 2 2 3 3 3 3 3 3 3 4 4 4 4 4 4 4 5 5 5 5 5 5 5 6 6 6 6 6 6 6 7 7 7 7 7 7 7 8. using das is often frustrating. part b: information quality. for the system overall, 1. do you think the output is presented in a useful format ? 2. are you satisfied with the accuracy of the system ? 3. is the information clear ? 4. is the system accurate ? 5. does the system provide sufficient information ? 6. does the system provide up-to-date information ? 7. do you get the information you need in time ? 8. does the system provide reports that seem to be just about exactly what you need ? 9. does the system provide the precise information you need ? 10. does the information content meet your needs ? 1 2 3 4 5 6 7 never always 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 2 3 4 5 6 7 2 3 4 5 6 2 3 4 5 2 3 4 5 6 6 7 7 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 part c: overall satisfaction. on the following scales, please circle the number which best reflects vour overall satisfaction with das. 1. how adequately do you feel das meets the information processing needs of your area of responsibility ? adequate 1 2 3 4 5 6 7 inadequate 2 . how efficient i s das ? efficient 1 2 3 4 5 6 7 inefficient 3. how effective i s das ? effective 1 2 3 4 5 6 7 ineffective 4. overall, are you satisfied with das ? dissatisfied 1 2 3 4 5 6 7 satisfied part d: perceived usefulness. on the following scales, please circle the number that best reflects how useful you perceive das to be. strongly agree disagree 1. using das in my job enables me to accomplish my tasks more quickly. 6 7 2. using das improves my job performance. 1 2 3. using das in my job increases my productivity. 7 2 4. using das enhances my effectiveness in the job. 7 2 5. using das makes it easier to do my job. 1 2 6. overall, i find das useful to my job. 1 2 strongly 1 2 3 4 5 3 3 3 3 3 4 4 4 4 4 5 5 5 5 5 6 6 6 6 6 7 7 7 7 7 part e: importance of the system (presented to respondents as questions about their involvement with das). 4. for me personally, in my job, das is unimportant relevant trivial essential needed 1 1 1 1 1 2 2 2 2 2 3 3 3 3 3 4 4 4 4 4 5 5 5 5 5 6 6 6 6 6 7 7 7 7 7 important irrelevant fundamental non-essential not needed 108 ajis vol 4 no. 1 part f: usage of the system strongly agree strongly disagree 1. if das were not mandatory, i would still use it 1 2 3 4 5 6 7 2. on average, i spend approximately hours/week working with das. 3. this represents % of my work. 109 the australasian journal of information systems (ajis) began in 1993 and was based on the simple realisation that there was a need for an academic ‘outlet’ for is research in australia australasian journal of information systems volume 17 number 2 2012 79 generalization in qualitative is research approaches and their application to a case study on soa development matthias goeken frankfurt school of finance & management m.goeken@fs.de rené börner frankfurt school of finance & management r.boerner@fs.de abstract in qualitative information systems research, little methodological support has been provided so far for the generalization from empirical data. generalizability, however, is a major concern in this field. it has been subject to a number of publications in recent years, but commonly accepted conceptualizations of generalizability and methodological guidance for the process of generalization are still missing. in order to address this problem, this paper investigates if and how generalization approaches from the literature and abstraction mechanisms from the field of conceptual modeling can be utilized to generalize from case study data. an explorative single case study on the development of service-oriented architectures provides the raw data for an exemplary application. the paper shows that abstraction mechanisms from the field of conceptual modeling can be used – in conjunction with generalization approaches from the literature – for the generalization of case study data and provides guidance on how to use these mechanisms. this increases transparency, traceability, and reliability of the generalization and might help to improve other qualitative research endeavors as well. keywords: generalization, case study, qualitative research, abstraction mechanisms, service-oriented architectures, service identification introduction generalizability is widely discussed in different kinds of research. therefore, it also constitutes a major challenge in qualitative information systems research, and quite a few frameworks and definitions from this and other fields of research exist (smaling, 2003; walsham, 1995; lee and baskerville, 2003). so far, a widely accepted conceptualization of what generalizability means is missing. as lee and baskerville (2003) point out, “qualitative is researchers (…) have not yet broached conceptions of generalizability appropriate to their own research” (p. 221). however, generalizability of research results is extremely important. usually, scientific readers do not want to learn about specific subjects that researchers observed in one case (babbie, 2010). the interest is on what one can learn from a case and how to generalize the observations of a specific mailto:m.goeken@fs.de mailto:r.boerner@fs.de australasian journal of information systems volume 17 number 2 2012 80 subject in order to explain the findings from a theoretical point of view, to predict future occurrences, or to give guidance in terms of recommendations for similar situations that might occur in the future. due to accepted sampling procedures and data analysis methods, quantitative research is often considered more conducive to producing generalizable results (stoddart, 2004). babbie (2010) outlines three major problems qualitative researchers are faced with, i.e., the subjectivity of researchers, the small number of researched cases, and doubts about the representativity, and argues that they lead to an endless potential for biased sampling and abstractions. however, prominent authors (e.g., walsham, 1995; lee and baskerville, 2003; yin, 2003) contend that generalization is desirable and possible, but that qualitative research must employ special kinds of generalization. in this respect, williams points out that papers reporting on results of research using qualitative and interpretive methods frequently “will make generalizing statements about findings whilst not commenting upon the basis upon which such generalisations might be justified” (williams, 2000). hence, to achieve status for their work, qualitative researchers should be more explicit about the way of generalization and adopt more formal methods. in order to better argue for the generalization of case study data and to make traceable the basis upon which generalizations might be justified, this paper investigates various conceptualizations and approaches of generalization in qualitative research and their application in a research project. furthermore, we propose and apply abstraction mechanisms from the research area of conceptual modeling to provide methodological support for generalization. this paper will utilize a case study on the development of a service-oriented architecture (soa) to demonstrate how generalization applying different approaches can be performed to develop several hypotheses and finally three model fragments. the paper is organized as follows: section 2 elaborates on case study research as part of qualitative research. generalization is discussed regarding both qualitative research in general and our case study research in particular. the case analysis is presented in section 3. section 4 discusses findings and limitations. finally, section 5 presents the conclusions of this paper. case study research and generalization the appropriateness of case study research we suppose that qualitative case study research can make a useful contribution to our research endeavor. case studies are particularly relevant for research in its “early, formative stages” (benbasat et al., 1987; myers, 2009) which applies to the field of soa (luthria and rabhi, 2009; stebbins, 2001). furthermore, benbasat et al. (1987) state that case study research is especially appropriate for the study of information systems development, implementation, and use within organizations. our research goal is to understand and to explore how soa development and service identification in particular are performed in real-life projects and to examine this in a real-life setting. as a case study can be descriptive and explorative in nature, it is supposed to give insights into how soa development is performed. descriptions and explanations of why a phenomenon occurs are provided by giving insight into the “generative mechanisms at work” (walsham, 1995) (p. 79) observed within the case data. with respect to our research goal, we derive hypotheses and models (model fragments) from the observations that have been made. thus, this case study takes the first steps in developing theory. australasian journal of information systems volume 17 number 2 2012 81 darke et al. (1998) distinguish single-case and multiple-case design (p. 277). a single case study might be appropriate for our purpose, but there are limitations to be considered. according to darke et al. (1998), “where explanatory research is undertaken, a single case may provide the basis for developing explanations of why a phenomenon occurs, and these may then be further investigated by applying them to additional cases in other settings” (p. 281). hence, in single case study research theoretical or analytical generalization is suitable, where case study results are used to develop theory (p. 278). in our study the intention was not to generalize to another population but to a theoretical understanding and explanation. this resulted in models representing generic socio-technical processes. since our case study is explorative in nature, it is well suited to show starting points for further research in this field. it is intended to produce generalizable results. hence, we will first outline different notions of generalization in qualitative research from literature (2.2). thereafter, the approach to and conception of generalization applied in this case study is discussed (2.3) and the research process is described (2.4). generalization in qualitative research in the discussion of generalization, we will first refer to the meaning of generalization in a colloquial sense. thereafter, we are going to develop a framework describing the outcomes of generalization and different conceptualizations and ways to generalize that we identified in the literature. in subsequent sections, the discussion will refer to this framework. according to merriam-webster’s dictionary, “to generalize” means “to derive or induce (a general conception or principle) from particulars.” lee and baskerville (2003) quote the oxford english dictionary which defines “to generalize” as “to form general notions by abstraction from particular instances.” hence, abstraction, derivation, and induction are closely linked to the phenomenon of generalization in a colloquial sense. they are typical activities or mechanisms within the generalization process, aiming at the development of general propositions which are “of a different kind from those developed on the first level of common-sense thinking which they have to supersede” (schutz, 1954) (p. 270). however, in science, there are different, partially conflicting notions and conceptualizations regarding the questions what generalization means and how to generalize (byrne and sahay, 2007). the most significant differences occur when qualitative and quantitative research is compared. whereas the latter predominantly concentrates on statistical generalization, many authors deem this type of generalization inadequate for interpretative, qualitative research (yin, 2003; lee and baskerville, 2003; smaling, 2003). even though lee and baskerville criticized researchers in the field of qualitative research (see introduction), sporadic attempts and a handful of approaches to conceptualize and apply generalization in research projects can be found in literature. in the following, this section will provide a discussion of existing literature that deals with the outcomes and types of generalization (table 1). throughout the paper, the authors will refer to table 1 where appropriate in order to show how different aspects of generalization are addressed and why some approaches have been chosen for our analysis. geertz (1973) promotes “thick descriptions” as the outcome of the research process. they provide details for phenomena in their specific context, are ‘thick’ because they embrace the meaning behind the mere observation, and thus might enable an assessment of similarities and differences between two or more cases. yet, first and foremost, they do “not ... generalize across cases but … within them” (p. 26). this notion is akin to the “working hypotheses” introduced by lincoln and guba australasian journal of information systems volume 17 number 2 2012 82 (1985). they argue that transferability depends on the suitability of working hypotheses that represent tentative propositions of situations and their similarities. source major concept outcomes of generalization geertz (1973) “thick descriptions” lincoln & guba (1985) “working hypotheses” popay et al. (1998) (logical generalization to a) “theoretical understanding” walsham (1995) concepts specific implications theory rich insight types of generalization lee & baskerville (2003) type ee: generalizing from data to description type et: generalizing from description to theory type te: generalizing from theory to description type tt: generalizing from concepts to theory klein & myers (1999) contextualization hermeneutics cycles stake (1995) “naturalistic generalization” smaling (2003) “analogical generalization” eisenhardt (1989) grounded theory techniques for data analysis; enfolding literature yin (2003) “analytic generalization” / “generalizing to theory” stoddart (2004) generalization about “generic social processes” hedström & ylikoski (2010) and woodward (2002) generalization by detecting “causal mechanisms” / “causal generalizations” table 1: approaches to generalization whereas geertz as well as lincoln and guba are rather critical towards the possibility of generalization from a specific situation to others, williams (2000) presumes that working hypotheses and thick descriptions at least “take the form of speculative generalisations” (p. 212) and are – what he calls – “moderatum generalisations”. in this type of generalization, aspects of a situation “can be seen to be instances of a broader recognisable set of features” (p. 215). he distinguishes moderatum generalizations from statistical and total generalizations (total generalization means that one situation is an instance of a general deterministic law that governs another situation as well). popay et al. (1998) refer to the different outcomes of qualitative and quantitative generalization. in qualitative research, “the aim is to make logical generalizations to a theoretical understanding of a similar class of phenomena rather than probabilistic generalizations to a population” (p. 348). they australasian journal of information systems volume 17 number 2 2012 83 point out that in order to reach a theoretical understanding through logical generalization, one needs to apply methods different from the ones in quantitative research. along these lines, tsoukas (2009) refers to znaniecki and points out that while both forms tend to reach general and abstract truths, quantitative/statistical induction abstracts by generalization, whereas qualitative/logical induction generalizes by abstracting. walsham (1995) presents four types of generalization from case studies which are also outcomes in the aforementioned sense: development of concepts, generation of theory, drawing of specific implications, and contribution of rich insight. they encompass fairly different results that can be obtained from case study data. on the one hand, he is explicit about the characteristics of these outcomes when he points out that they “should be viewed as 'tendencies', which are valuable in explanations of past data but are not wholly predictive for future situations,” and they are “explanations of particular phenomena (…) which may be valuable in the future in other organizations and contexts” (p. 79). on the other hand, he does not elaborate on the process of generalization in his paper and an extension (walsham, 2006). whereas literature provides different descriptions of different outcomes of the generalization process, it does not offer a clear picture of how to generalize. a methodological guidance for the way the mentioned outcomes can be obtained or how to abstract, derive or induce in different situations is missing. however, we can identify hints, and orientation in the related literature that helps to develop a frame of understanding. in order to structure the ways generalization can be performed, lee and baskerville (2003) develop a framework of four different generic types of generalization. on the one hand, empirical statements are generalized to either other empirical statements (type ee: generalizing from data to description) or to theoretical statements (type et: generalizing from description to theory). on the other hand, theoretical statements are generalized to either empirical statements (type te: generalizing from theory to description) or to another theoretical statement (type tt: generalizing from concepts to theories), respectively. klein and myers (1999) offer a generic description in their principle #4 “the principle of abstraction and generalization:” they describe the way of generalizing as “relating the idiographic details (…) to theoretical, general concepts” (p. 72). to do so, they recommend “contextualization” and “hermeneutic cycles” (their principles #1 and #2) as two important means. since klein and myers’s aim is to propose a useful set of principles, along with their philosophical rationale, there is no detailed description provided. even though the principles and generic types are described in some detail and various illustrative examples are given, lee and baskerville as well as klein and myers do not intend to give guidance in terms of an applicable research method. based on the relationship between the reader’s experience and the case study itself, stake (1995) argues for an empirically-grounded generalization that he calls “naturalistic generalization.” accordingly, the case data can be understood and interpreted by readers more comprehensively if it matches their experience. in this case, the generalization emerges when the reader recognizes similarities in the case study details and finds descriptions that resonate with his own experience. smaling (2003) deals with the problem if and how findings from one case study can be transferred to another. he argues that “analogical generalization” – in contrast to inductive generalization – is “plausible when there are solid arguments that, when a particular researched case has characteristics which are relevant for the research conclusions, another case that has not been researched also has these relevant characteristics” (p. 57). this includes case-to-case generalization as well as exemplary generalization. smaling makes one important assumption, namely that generalization is more firmely australasian journal of information systems volume 17 number 2 2012 84 based “for the more one knows about similarities and differences between a case that has been researched and one that has not” (p. 55). his approach is partially in line with the notion of lincoln and guba (1985) and their idea of transferability and fittingness. eisenhardt (1989) provides useful procedural knowledge (a “roadmap”). she describes a process of building theory from case study research. however, regarding the most important step analyzing data she points out: “analyzing data is the heart of building theory from case studies, but it is both the most difficult and the least codified part of the process” (p. 539). she advocates the application of grounded theory techniques and approaches to qualitative data analysis in order to support the emergence of theoretical categories and concepts. in addition, she recommends “enfolding literature” to improve the generalizability of the research findings by “tying the emergent theory to existing literature” (p. 545). accordingly, yin (2003) suggests an “analytic generalization” with the goal “to expand and generalize theories” (p. 10). he argues that single case studies “are generalizable to theoretical propositions and not to populations or universes” (p. 10). this corresponds with the “type et generalizability” in lee and baskerville (2003) and the “logical generalization to a theoretical understanding” of popay et al. (1998). following his approach, theory also becomes a vehicle for generalization. this, of course, is only applicable where appropriate theories exist. stoddart (2004) criticizes “folk notions of science (…) entwined with the positivist tradition” and advocates the idea of “generic social processes.” based on previous work by becker (1990) and prus (1994), he abandons claims to generalizability about populations. instead, he focuses on generalizing about “generic social processes” in order to “see how they play out in potentially diverse social settings” (p. 308). in newer social science, some authors pursue a similar idea (hedström and ylikoski, 2010; tsoukas, 2009). this attempt to generalize is the delineation of causal mechanisms. generalization occours by “opening up black boxes and making explicit the causal cogs and wheels through which effects are brought about” (hedström and ylikoski, 2010) (p. 54). in this approach, researchers attempt to identify possible mechanisms and to turn them into a plausible mechanism through the collection of empirical evidence about the assumed entities, activities, relationships, etc. “what separates proper mechanism-based explanations from mere mechanism-based storytelling is this kind of rigorous checking of the assumptions upon which the mechanism schemes rest” (hedström and ylikoski, 2010) (p. 53). in this domain, woodward (2002) puts forward the idea of “causal generalizations” by providing an account of explanatory relevance. these ideas seem to be very similar to bhaskar’s “generative mechanisms” which have been adopted by walsham (1995) but, interestingly enough, attract considerable interest in modern sociology. in the following sections, we apply different approaches to generalization presented in table 1. in addition, we introduce techniques from conceptual modeling for methodological guidance in qualitative research. research paradigm and generalization in our case study there is a commonly drawn distinction in case study methodology between positivist and interpretivist research (doolin, 1996; darke et al., 1998). even though this distinction is commonly used, there seems to be a loss of clarity in what makes the difference. weber (2004), for example, argues, that “many, if not all, of the alleged metatheoretical differences between positivism and interpretivism are spurious” and that the real differences “lie more in the choice of research methods” (p. x). others contradict this claim by explicating the “functional outcomes of research,” which is basically the distinction between “understanding” and “explanation” (hovorka and lee, 2010). australasian journal of information systems volume 17 number 2 2012 85 understanding, which is associated with interpretivism, is conceptualized in a subjective fashion. researchers should develop an understanding of the subjective understanding of the participants’ own understanding (lee, 1991). in doing so, he or she is closely connected to the thoughts and motivations of the human objects under study and, in addition, offers an interpretation of and for human conduct (doolin, 1996), which is also called “interpretive understanding of the subjective understanding.” in contrast, explanation in positivism refers to an observing researcher’s formal position and is guided by the criteria of the natural science model (darke et al., 1998; lee, 1989). accordingly, the constructs and variables used belong to the researcher and are not part of the subjects’ experience (hovorka and lee, 2010). this leads to a positivist understanding, providing explanation of the empirical reality. despite these basic differences, scholars argue that the two perspectives can be mutually supportive, rather than mutually exclusive (lee, 1991) and that the choice of techniques and methods to improve scientific knowledge should be based on their specific strengths and weaknesses (weber, 2004). for that reason, hovorka and lee (2010) and lee (1991) demonstrate how the two approaches can be integrated and how the linkage between explanation and understanding can be used to improve knowledge creation. basically, this integration is viable because the interpretative understanding may provide the basis on which to develop the positivist understanding. thus, the positivist interpretation of the interpretive understanding will lead to a different theoretical explanation. however, it also has to be mentioned that previously interpretative researchers have already argued in a similar manner: insofar as research begins with statements of particulars and ends with a general statement (a theoretical explanation), this reasoning process is a form of generalization. “hence, generalizability is an essential feature of interpretive research that endeavors to provide theory and not just description.” (lee and baskerville (2003); with reference to geertz (1973) and yin (2003)). in our case study, we make use of the two perspectives, as they are mutually supportive, to analyze data and elaborate on patterns and mechanisms, and hereby to generalize by formulating hypotheses and designing model fragments. according to lee (1991), “understanding has at least two specific meanings. in its first sense, understanding refers to “understanding” as the process by which people in everyday life come to interpret and, therefore, to understand and guide themselves in their world. however, the observing social scientist is also one such person, albeit with different cognitive motives” (p. 348). thus, the subjective meanings that give rise to the behavior of the people that are studied play a significant role in our research process. they are reflected throughout the whole research process and can be traced in section 3. however, with every generalization, the researchers more or less move away from subjective meanings in order to structure the self-perception of the project members. our conceptualization of generalization for the case study at hand is as follows: our goal is to reach a higher level of abstraction by identifying theoretical, general concepts in a first step. this is done by relating the idiographic details and the subjective understanding to concepts in order to get from data to observations and – in a second step – from observations to theory which is in line with lee and baskerville’s (2003) approach (table 1). our notion of theory is adopted from gregor (2006), who defines theory as “statements providing a lens for viewing or explaining the world” (p. 4). a critical issue remains how data and concepts can be related. the approaches presented in table 1 propose that these relationships can be drawn by “naturalistic generalization” (stake 1995) or “analogical generalization” (smaling, 2003). the former is not used because we do not agree that the interpretation and generalization should be made by the reader alone. the latter approach refers to case-to-case generalization, which is not our primary goal. an “analytic generalization” as proposed by (yin, 2003) is not chosen because there are no commonly accepted theories in this field of research. australasian journal of information systems volume 17 number 2 2012 86 instead, we apply techniques of the grounded theory methodology and interpretative techniques; in later phases, we also compare our findings with conflicting and similar literature (see ‎2.4). this corresponds to logical generalization aiming at the development of a theoretical understanding and a mode of induction described by znaniecki as “generalization by abstraction.” additionally, in search for guidance and methodological support, we identified abstraction mechanisms of conceptual modeling as a helpful means because they aim at identifying “useful abstractions of the similarities of classified phenomena” (parsons and wand, 2008), an aim very similar to the one of qualitative research. abstraction mechanisms are generic relationship types with a defined semantic. this helps with linking data and concepts residing on different levels of abstraction. we argue that in qualitative research these mechanisms will support the grouping and structuring of details and concepts. hence, in terms of eisenhardt (1989), abstraction mechanisms are “structured and diverse lenses” which help to identify concepts and categories and transparently structure “similarities and differences” (geertz, 1973; smaling, 2003). literature offers a range of abstraction mechanisms (goeken, 2006; olivé, 2007; mattos, 1989; analyti et al., 2007). two of them will be applied to the case study data in the following sections:  classification relates instances with a type, or to be more precise, it consists of determining the types which an object is an instance of (semantic: “has type”/”type of”). instances have common properties and are assembled into a new entity type for which uniform conditions hold. hence, a type is an abstraction representing instances on a higher abstraction level.  aggregation (composition) defines part-whole structures by describing that the whole is a composite formed by parts. hence, aggregation is an alternative way of forming an abstraction (on a higher abstraction level) with the semantic “part of” (also called meronymic relationship/holonymic relationship). the application of other abstraction mechanisms that we do not utilize in this paper (e.g., specialization (semantic: “kind of”/“is a”); grouping (semantic: “member of”); roles (semantic: “role of”); materialization (semantic: “materializes”) will be subject of future research. in the following, we will use aggregation and classification as a means to abstract from idiographic details and concepts to categories. furthermore, the induction of categories (“types” and “aggregations”) supports inferences about non-observed properties. as they are abstract placeholders, they enable and guide the derivation of alternative concepts that could not be observed in the original data. in doing so, researchers leave the firm empirical foundation. however, by making this procedure as transparent as possible, the creation of abstract types, and thus the generalization by abstraction, becomes traceable and can be justified. one further goal is to identify generic socio-technical processes comparable to “generic social processes” and “causal generalizations” (see table 1) which are abstracted formulations of social behavior and the interplay of social and technical aspects. these can be interpreted as “middle range theories” because they are sufficiently abstract to be applied to different contexts but do not offer a set of general laws (hedström and ylikoski, 2010) (p. 61). middle range theories allow contextual explanations and “lie between the minor but necessary working hypotheses that evolve in abundance during day-to-day research and the all-inclusive systematic efforts to develop a unified theory that will explain all the observed uniformities of social behaviour, social organization and social change” (merton, 1967) (p. 39). in order to prevent excessive overestimation of generalizability, the main findings of our research (e. g., hypothesis and model fragments which form the “generic social processes” and “causal australasian journal of information systems volume 17 number 2 2012 87 generalizations” ) “should be viewed as 'tendencies', which are valuable in explanations of past data but are not wholly predictive for future situations” (walsham, 1995) (p. 79). subsequently, developed theories should not be seen as “proven statements” but rather as “well founded but as yet untested hypotheses” (lee and baskerville, 2003). this parallels the notion of “moderatum generalization” (williams, 2000 and section ‎2.2). research process of our case study following eisenhardt (1989), we designed a research process for the study at hand which is depicted in figure 1. the process commences with the explication of “a priori constructs.” furthermore, the first activity includes creating an initial research question and more specific questions of investigation (see section ‎3.2). this is an important first step to guide the case analysis and focus efforts. in a second activity, an appropriate case has to be selected. since the objective of this case study is to enhance understanding and develop theory rather than testing it, the chosen case does not have to be representative. one goal of this study is thus the generation of theory and not its justification through testing. a theoretical sampling though is “particularly suitable for illuminating and extending relationships and logic among constructs” (eisenhardt and graebner, 2007). hence, we intentionally choose an extreme case that will be described in section 3.1. as outlined in table 1, a detailed description and comprehensive background information is important to support the “empiricallygrounded” and “naturalistic generalization” promoted by stake (1995) and to support the principle of contextualization by klein and myers (1999). multiple data collection methods such as interviews and analysis of documentation are used in the third activity (see section 3.3). in order to have a solid basis to build upon, collected data has to be gathered from multiple sources of evidence to underpin the completeness and correctness of data. this triangulation of data is important for the reliability of the case study’s outcomes (yin, 2003). flexible and opportunistic data collection methods allow for reacting to emergent themes by adjusting data collection when necessary (eisenhardt, 1989). the fourth activity – analyzing data – begins with the grounding of our observations, as described in section 3.4. this is followed by the shaping of hypotheses through generalization, which is the subject of section 3.5. in the presentation of our research, we distinguish between observations and hypotheses (analyzing data and generalization, respectively) in order to enhance transparency and traceability of the procedure, even though they are closely interwoven. activity four is dedicated to properly grounding concepts and relationships. their identification is conducted by employing techniques from grounded theory (see eisenhardt (1989) in table 1) and interpretative techniques (see walsham (1995) and boland et al. (2010) for hermeneutical exegesis in is). even though we do not use the coding techniques to their fullest extent, the general approach and respective tools support the assignment of statements from the interviews and documents to concepts. the goal is to detect relevant particulars within the idiographic details and to discover underlying reasons for why concepts, relationships or patterns exist. this activity results in observations which are judgments or inferences from what we observed (adapted from merriam-webster). these judgments are a first abstraction from the raw data, labelled “first-order concepts” by van maanen (1979). second-order concepts (the concepts forming our observations) are “notions used by the fieldworker to explain the patterning of the first-order data” (van maanen, 1979). hence, they move away from the subjective understanding (hovorka and lee, 2010) and ‎2.3). australasian journal of information systems volume 17 number 2 2012 88 while concepts and observations are still closely linked to the idiographic details, in activity five (generalization), we generate hypotheses consisting of abstract categories. extending van maanen’s terminology (1979), we might label these categories, which form hypotheses, third-order concepts, which thus represent abstractions of abstractions of first-order data. we generate these hypotheses by relating concepts described in the observations to categories applying abstraction mechanisms adopted from conceptual modeling. the categories and relationships between them should apply to multiple situations and hereby reach a higher level of generality. the sixth activity consists of a comparison with related work, so called “enfolding literature” (eisenhardt, 1989).1 it aims at comparing opinions and positions found in related literature with the observations we made and improving the hypotheses. this results in hypotheses “with stronger internal validity, wider generalizability, and higher conceptual level” (eisenhardt, 1989). in doing so, we try to turn possible mechanisms into plausible mechanisms (hedström and ylikoski, 2010) (p. 52). analyzing data (4) and generalization (5) are closely interwoven and both supported by the analysis of literature. according to gadamer (1976), “the harmony of all the details with the whole is the criterion of correct understanding” (p.117), and in a number of iterations “a complex whole of shared meaning emerges” (klein and myers, 1999). this is represented by the iterative layout of activities four to six which accords to the hermeneutic cycles promoted by klein and myers (1999). the focus is on constantly comparing hypotheses, data, and competing and similar literature iterating towards a set of hypotheses which closely fits the data (eisenhardt, 1989). finally, activity seven integrates several hypotheses to build model fragments (section 3.6). through another generalization, these fragments have an even higher level of abstraction and are thus more remote from the first-order data. figure 1 illustrates the increase of abstraction with each activity performed moving from left to right. considering semantic levels, model fragments encompass both the level of instances and the level of types. they incorporate categories used in our hypotheses and also concepts – including those we were not able to observe in our case study. generalization in an soa case case description in the context of the project “ad-hoc data grids environments” (adage), researchers at the university of new south wales (unsw) implemented a service-oriented architecture for sirca, a data provider. the authors of this paper did not participate in the actual soa implementation project but analyzed data about it ex post as described in the following sections. the project aimed at providing customers an easier retrieval and analysis of heterogeneous data from different sources (grid environment) spontaneously in an ad-hoc fashion. soas were not the focus of this software implementation project but service orientation was chosen as the preferred architectural paradigm. thus, services (and their identification) were used as a means to meet the company’s requirements rather than being the subject of analysis themselves. 1 due to the focus on generalization, the extensive discussion of literature related to every hypotheses and model is omitted in this paper. it can be found in börner et al. (2012). australasian journal of information systems volume 17 number 2 2012 89 sirca provides a huge data repository containing historical financial market data such as news and trading data. its aim is to supply this data to researchers especially at australian and new zealand universities. first-order concept fourth-order concept third-order concept second-order concept in st an ce s ty p es level of abstraction activity activity 1: getting started a priori constructs deliverable research questions & questions of investigation activity 2: selecting cases activity 3: data collection: selection of documents and crafting protocols case study database: transcripts, presentations, … observations activity 4: analyzing data activity 5: generalization (shaping hypotheses) hypotheses model fragments activity 7: integrating hypotheses activity 6: enfolding literature raw data e.g. interviews observations (concepts) hypotheses (categories) model fragments figure 1: research process and deliverables of the respective activities in adage, services were created based on the available data. this implied a technical understanding of services. the company’s employees were not thinking in terms of business processes, so that no model that could have been analyzed in the course of service identification was delivered. the company’s management, however, had some requirements in mind that should be fulfilled by services. unfortunately, these were not documented, which makes traceability difficult. requirements were communicated to the project team in scheduled weekly meetings and workshops. service candidates were identified on the basis of these meetings and prototyped. in an iterative and incremental approach, the functionality of these candidates was adjusted to finally meet the company’s requirements. a close collaboration between the sirca’s research and development department and the university’s project team was a key to ensure the successful identification of services. first and foremost, the search for services was driven by the idea to retrieve and integrate data from different sources. in a second step, project members identified which services could support researchers in analyzing data, e.g., building time series of financial data. clearly, this was a requirements-driven bottom up approach. goals included the provision of a graphical user interface australasian journal of information systems volume 17 number 2 2012 90 (gui) to customers, enabling them to directly invoke services in an ad-hoc fashion to analyze financial market data. certainly, sirca’s case is not a typical example of service identification projects. due to its rather extreme character, it helps to identify possible instances of situational factors that usually cannot be found in typical cases. since this paper focuses on methodological aspects, we partly omit more detailed descriptions of the case study data. a comprehensive discussion and description including interview guidelines, transcripts, the raw data leading to our observations, etc. can be found in börner et al. (2012). getting started as illustrated in figure 1, outlining the theoretical background in the sense of “a priori constructs” is the first step of our research process. since the research question is the foundation for the analysis of every case study (eisenhardt, 1989), our research process continues with the formulation of a research question. shortly after the beginning of the case study, it became obvious that a single focus on service identification might not deliver satisfactory results in the adage case. thus, we extended the scope of our analysis to the early stages of the soa lifecycle, namely the design and development of services and the service-oriented architecture in adage. some authors have already developed methods to support identification, design, and development of services (for an overview see börner and goeken (2009b), kohlborn et al. (2009b) and birkmeier et al. (2009)). however, little is known about service identification and the early stages of the soa lifecycle in real-life projects. thus, in order to guide us through the analysis, we formulate the following initial research question: how does soa development and service identification work in real-life projects? the rather generic nature of our research question necessitates the formulation of further, more specific questions, the questions of investigation. these help to focus our analysis and support, for example, the hermeneutic analysis of the interview transcripts. the following questions of investigation are reflected by the guidelines used for the conducted interviews.  which circumstances influence the proceeding of the service identification process?  did the understanding of services change/develop in the course of the project? was there a change of the skill level of the project participants?  how was the soa implemented technically? which software development techniques had been used? data collection written documentation, i.e. various sorts of electronic files, and interviews are major sources of our data collection process. 19 presentations that were held between december 2006 and november 2009 provide a good overview of the general proceeding, achieved objectives, and next steps required at the time of the presentation. furthermore, official progress reports give a structured overview of how the project was proceeding. due to their much more formal character, these reports offer fewer insights into actual work practices, such as the identification of services. a considerable number of published and unpublished papers have been authored by project team members in the course of the project. the variety of publications in which the papers have appeared shows the scientific and practical value of the project’s outcomes. additionally, various websites about the project or involved parties are used to retrieve background information. some newspaper articles about the project could be found as well and are added to the case study database. australasian journal of information systems volume 17 number 2 2012 91 in order to retrieve additional information, particularly about the identification of services in the adage project, four interviews are conducted. three of the interviewees are project team members at the university. all of them were involved in identifying and designing services in the project. another interviewee is a representative of the company that implemented the soa. the interview questions are open-ended. however, guidelines with questions are used in each case. for the three project team members at the university, the same question guideline applies, whereas a different one is used in the interview with the company’s representative. the interviews serve as a valuable means to capture subjective meanings that are essential to our interpretivist approach. all labels of concepts used in our observations are retrieved verbatim from the interviews. analyzing data based on the collected case study data, we first strive to investigate the subjective understanding of the project members. from this understanding, observations residing on a higher level of abstraction are induced (second-order concepts). due to space limitations, we cannot give detailed insight into the subjective understanding. this is done in a working paper (börner et al., 2012) incorporating a plethora of quotations from documents and interviews. the adage project members perceived themselves as rather innovative and archaic in a creative environment with little constraints. due to the project setting, i.e., a public private partnership, the pressure regarding time and budget was comparably low. the developers used their freedom to follow a flexible free-style approach based on their available skills. according to the members’ perception, they were able to compensate for the absence of any formal method in their development process through extensive communication. furthermore, based on the mainly technical skills of the project members, the proceeding in the project was seen as a creative technically-driven trial-and-error endeavor. our observations attempt to structure the subjective meanings that were found in the raw data and support the perception of the project situation. grounded in document studies and interviews, we arrive at several concepts that we consider as being of importance in the course of the adage project, e.g., we regard the subjective understanding mentioned above as the concept “absence of a formal method.” in doing so, and applying the contextualization principle (see table 1), we seek to present the subject matter in its context “so that the intended audience can see how the current situation under investigation emerged” (klein & myers, 1999) (p. 73). all 16 observations describing the concepts and their relationships are shown in table 2 and are the result of activity four. shaping hypotheses in this next activity we further abstract the observations by generating hypotheses consisting of categories which are based on the concepts observed. the concepts we identify in the observations can be regarded as instances, and thus we can use abstraction mechanisms to generalize from these observations. the resulting types or compositions form the components of our hypotheses. hence, by applying abstraction mechanisms, we create abstract categories on the type-level, the ‘third-order concepts’ (figure 1) we have labeled categories (e.g., we can classify the concept “fine-grained services” as an instance of the category “granularity”). in so doing, we utilize mechanisms from conceptual modeling that guide the generalization by abstraction which we regard as a logical generalization. as the categories are abstract placeholders, in other settings they will have different concepts than those we have observed in our case study. granularity, e.g., also has the instance “coarse-grained australasian journal of information systems volume 17 number 2 2012 92 services.” by incorporating alternative concepts, we clearly move beyond the details observed and further generalize the findings (see section ‎3.6). observation 1. the generous funding significantly influenced the way of service identification. 2. the company’s small size significantly influenced the way of service identification. 3. people’s skills significantly influenced the way of service identification. 4. the implemented soa satisfied users, although no formal method for the identification of services was used. 5. little soa experience led to a technical understanding of services. 6. personal communication was vital to meet user requirements. 7. a technical understanding of services led to very fine-grained services. 8. a technical understanding of services led to a bottom up direction of analysis. 9. the implemented fine-grained services provided flexibility for users and developers at the same time. 10. fine-grained services supported reusability. 11. orchestration of services significantly contributed to flexibility. 12. fulfilled user requirements led to user satisfaction. 13. identification and development of services were incremental and iterative processes that supported fulfillment of user requirements. 14. the flexibility provided by services led to user satisfaction. 15. user satisfaction was a success measure. 16. reusability was a success measure. table 2: observations from the case study in the following, we present our hypotheses and describe how the generalization and the merging of the observations is performed, applying the abstraction mechanisms described in section 2.3. table 3 illustrates our observations and the resulting hypotheses. due to space restrictions, only hypotheses 1, 3, 5 and 6 will be described in detail because they are further used in the models that will be developed in section 3.6. australasian journal of information systems volume 17 number 2 2012 93 hypothesis observation 1. context factors significantly influence the way of soa development and service identification. 1. the generous funding significantly influenced the way of service identification. 2. the small company size significantly influenced the way of service identification. 3. people’s skills significantly influenced the way of service identification. 2. not using a formal method for service identification does not necessarily lead to a failure of soa projects. 4. the implemented soa satisfied users, although no formal method for the identification of services was used. 3. soa project experience leads to a different understanding of services and thus affects success measures. 5. little soa experience led to a technical understanding of services. 4. personal communication can substitute utilization of formal methods. 6. personal communication was vital to meet user requirements. 5. the understanding of services affects both the granularity of services and the direction of the service identification approach. 7. a technical understanding of services led to very fine-grained services. 8. a technical understanding of services led to a bottom up direction of analysis. 6. the right granularity of services can affect multiple success measures and depends strongly on the project at hand. 9. the implemented fine-grained services provided flexibility for users and developers at the same time. 10. fine-grained services supported reusability. 7. the success of soa projects is expressed through technical and business-oriented success measures. 15. user satisfaction was a success measure. 16. reusability was a success measure. table 3: hypotheses related to observations figure 2 shows how the relation between concrete concepts (observations) is mapped to an abstract view that generalizes from the adage case. while the lower part links instances through observations, the upper part shows the relation between types and the resulting hypotheses. concepts (e.g., fine-grained services, reusability) that could be observed in our case study are linked through observations (table 2), e.g., o10 “fine-grained services supported reusability”. on the next abstraction level, categories such as granularity are related through hypotheses like number 6 “the right granularity of services can affect multiple success measures and depends strongly on the project at hand.” these categories reflect phenomena that can be found not only in the adage case but in soa implementation projects in general. in addition, they represent „causal mechanisms“ and „causal generalization“ (hedström and ylikoski, 2010). thus, the hypotheses are a generalization of our observations as shown in table 3. australasian journal of information systems volume 17 number 2 2012 94 adage success no method used generous funding personal communication user satisfaction high flexibility reusability orchestration technical understanding incremental/iterative development approach little soa experience observation o8 fine-grained services bottom up approach limited skills small company o7 o4 o1 o2 o3 o5 fulfilled user requirements o6 o9 o11 o10 hypothesis o14 o16 o15 method orientation communication flexibility coordination understanding of services development paradigm experience granularity direction of approach general context h2 h1 customer orientation h4substitute soa success h7 h6 h3 technical success measures business-oriented success measures h5 o13 o12 general way of si/soa development figure 2: interrelationships of observations and hypotheses hypothesis 1: the general context significantly influences the method of soa development and service identification. certainly, there are many factors that affect the approach to service identification in general and the choice of certain methods in particular. those factors that cannot be influenced by the project team are commonly called context factors (bucher et al., 2007). we identify generous funding, small company size, and limited skills in adage to be such context factors (observations 1 to 3). at first, we use the abstraction mechanism aggregation to subsume the identified concepts under the new concept adage context. thus, generous funding, small company size, and limited skills have an “is-part-of” relation to adage context as shown in figure 3. on the level of abstract categories these relations apply, respectively. australasian journal of information systems volume 17 number 2 2012 95 budget company size skills generous funding small company limited skills general context adage context is part of is part of is part of is part of is part of is part of has instance has instance has instance has instance categories concepts figure 3: applying abstraction mechanisms to hypothesis 1 applying the abstraction mechanism classification, we can generalize our concepts (instances) by determining abstract categories (types). for example, the category company size is the result of classification from the observed concept small company. thus, there is a has-instance(instance-of-) relationship between these two. budget and skills are the other two categories that were directly derived from our observed concepts described in observations 1 to 3. the concept adage context that is created by aggregation is transformed into the category general context by classification. all categories are related to respective concepts by “has-instance” relations. particularly, the aggregation that sums up several concepts to the adage context necessarily moves away from subjective meanings that served as the starting point for our analysis. our observations show that the combination of a small company and employees with limited skills (trumping the opportunities of a generous budget) leads to an absence of formal methods in service identification. this is in line with the subjective understanding of the project members (see 3.4) and certainly an extreme case. however, it seems to be plausible to assume that context factors significantly influence the way of service identification and, in the following, we will discuss the underlying mechanisms. thus, we can e.g., tentatively hypothesize that large companies with highly skilled employees will use different methods for soa development and the identification of services. hypothesis 3: soa project experience leads to a different understanding of services and thus affects success measures. in adage, we observed that little soa experience accompanied with the dominance of it specialists resulted in a technical understanding of services (observation 5). classifying the concepts observed, we assume that the level of experience influences the understanding of services. we therefore hypothesize that companies with a track record of service implementation tend to involve business departments early in the process of service identification. therefore, the understanding of services is much more process-oriented. through multiple cause-and-effect relationships shown in figure 2, success measures such as reusability are affected. the task of planning a service-oriented architecture should be tackled by both business departments and an organization’s it division (börner and goeken, 2009b). implementing an soa is primarily a technical challenge. we assume that – like in the adage case – many soa projects underestimate the importance of properly planning an implementation. hence, consideration of business processes is often poor, and employees of the it division take the lead in these projects. especially in companies that have little or no experience with the nature of services, the dominance of it specialists usually leads to a technical understanding of services. more experience in this field might result in different outcomes. a project team might decide to use a process-oriented approach, a user-oriented approach, australasian journal of information systems volume 17 number 2 2012 96 or even choose a technical approach if suitable. the most significant difference is that an experienced project team with a broader understanding of soa is free to choose an adequate approach from a range of options. unfortunately, this makes a prediction almost impossible. the only cause-and-effect relationship that can be established here is that more experience allows for different opportunities whereas little experience most certainly leads to a technical understanding. anderson et al. (2005) argue that the extent to which the enterprise architects, service developers, and operations project staff in the it department are skilled (i.e., their experience and subsequently their understanding of services) is a critical factor for web service implementation. looking at our observations 5, 7, and 10, we can further hypothesize that soa experience indirectly influences reusability of services. this is confirmed by baskerville et al. (2005), who show that “very few web services could be reused exactly as originally implemented” (p. 7) and thus, experience is essential for soa implementations. becker et al. (2009) state that “especially in the early phases of soa maturity [i.e., with little experience] there is no experience about [what] a reusable service would look like” (pp. 7-8). this is another hint that soa project experience tends to change the understanding of services and enhances reusability. hypothesis 5: the understanding of services affects both the granularity of services and the direction of the service identification approach. in the adage project, a technical understanding of services led to very fine-grained services and a bottom up approach stressing technical aspects of the underlying (data) infrastructure (see observations 7 and 8). in order to develop our hypotheses, a classification leads to the abstract categories understanding of services, direction of approach and granularity. the category granularity, e.g., is the abstraction of fine-grained service, the instance that we are able to observe. firstly, we can hypothesize that a different understanding of services leads to different granularities. moving on a scale from technical to business-oriented understanding, the identified services might become coarser-grained along this line. from a business perspective, services should support business processes or at least sub processes. hence, they encompass more functionalities than an elementary service that, for example, extracts an address from a database. the latter will more likely be the outcome if service identification is conducted from a technical point of view. secondly, the direction of the approach is influenced by this understanding. again, moving along the above scale will result in the application of bottom up approaches on one side and top down approaches on the other side. it is unlikely to end up with a pure instance of any of the approaches. even in the adage case, a very technical understanding did not result in a pure bottom up approach. however, the share of techniques that are typically assigned to top down approaches (such as strategic analyses) will increase with a business-oriented understanding of services. accordingly, in their comparison of service analysis approaches, kohlborn et al. (2009a) differentiate two types of soa concepts, i.e., understandings of soa. one of them is rather technical and, thus, delivers so-called software services while the other (business-oriented) one results in business services. hypothesis 6: the right granularity of services affects multiple success measures and depends strongly on the project at hand. in service-oriented architectures, granularity of services is a widely discussed issue among researchers and practitioners alike. our observations 9 and 10 indicate that fine-grained services positively contributed to both reusability and (indirectly) user satisfaction in the adage case. thus, we hypothesize that granularity indeed plays a major role for the success of soas in general since it potentially affects more than one success measure. the categories granularity, flexibility, and success australasian journal of information systems volume 17 number 2 2012 97 measures that constitute hypothesis 6 were developed by classification. they all abstract the instances and concepts observed and, thus, have “is-instance-of” relationships to the concepts fine-grained services, high flexibility, user satisfaction, and reusability. whereas the adage case shows clear advantages of fine-grained services, coarse-grained services might be the better choice in other settings. some customers might not be interested in the flexible composition of services every time they use them. if they simply want to outsource part of a business process, they might prefer a coarse-grained service that encompasses all necessary functionalities and delivers a comprehensive result. accordingly, elfatatry (2007) argues that “the appropriate level of granularity for a service and its methods is relatively coarse. a service generally supports a single distinct business concept or process.” (p. 38) in the adage case, fine-grained services enhanced both reusability and user satisfaction, i.e., technical and business-oriented success measures, respectively. in other cases, the granularity of services might be a trade off because effects on these two kinds of measures could be converse. we argue that, due to different preconditions, the right granularity of a service has to be elaborated depending on the situation at hand. there is no silver bullet for right-sizing a service without considering the context of service implementation. thus, a situation-specific approach to the choice of methods is important to provide for an adequate granularity. integrating hypotheses through generalization of our observations, we create hypotheses as illustrated in figure 2. this is done by applying classification and aggregation, two abstraction mechanisms with a clear semantic (instance-of/part-of). since they are applied upwards (e.g., a concept ‘is instance-of’ a category), the investigation is based on and refers to observed instances and respective concepts. hence, both the resulting observations and hypotheses are still closely linked to the primary data we retrieved from documentation and interviews. in the following, we will describe phenomena that move beyond this data and reach another level of abstraction. by performing another generalization and looking at causal mechanisms between concepts, we arrive at model fragments that were not necessarily obvious beforehand. due to the fact that the categories are abstract placeholders and the hypotheses show possible and plausible causal mechanisms, we can now reason about further conceivable concepts in terms of alternative instances. this is supported by applying the abstraction mechanisms downwards (a category ‘has instance’ nonobserved concept). since we assume that the hypotheses describe plausible mechanisms and the placeholders as well as the abstraction mechanisms guide the derivation, this is not arbitrary but another generalization by abstraction. because the observed concepts as well as the categories in the different hypotheses overlap, we are also able to combine the hypotheses in a model or in a small set of model fragments. the model fragments we present in this section can best be interpreted as middle range theories (see section ‎2.3) because they have limited scope and are applicable to limited conceptual ranges, as they only grasp phenomena from the systems development / soa domain. furthermore, they refer to selected aspects in the realm of soa development because we focus our analysis on the questions of investigation presented in ‎3.2. our model fragments allow for further detailing and the formulation and discussion of alternative theories. they correspond to the “theories” introduced by walsham (table 1), and are causal generalizations. in the following subsections, three model fragments which present an external, internal, and a success perspective are elaborated. australasian journal of information systems volume 17 number 2 2012 98 contingency model (cm) context factors such as budget, company size, and people’s skills are consolidated in hypothesis 1. all of them (“general context”) influence the application of methods, communication, and the general approach to implementing soa projects. the contingency model in figure 4 includes our observations from the adage case (underlined concepts) and alternative concepts which are derived by conceivable instantiation of the categories applying abstraction mechanisms. we assume that alternative instances of the mentioned categories will result in different ways of si/soa development, as depicted in figure 4. in our model, the list of categories constituting the contingency model is limited to those we were able to derive from observed concepts. we assume that the category “general context” can also be extended by non-observed concepts and resulting categories. for instance, in a more business-driven soa implementation project, the choice of a bpm software tool could influence the way of si/soa development. since there is no bpm tool used in our case, we are not able to observe such an impact. accordingly, the way of si/soa implementation will most likely include more categories than method orientation and communication which we are not able to observe. budget generous funding average funding low funding general way of si/soa development underlined concept = our observationlegend: = category company size big company medium-sized company small company skills high skills average skills limited skills way of si/ soa development in adage other way of si/soa development general context figure 4: influence of the general context on the way of si/soa implementation (contingency model – cm) the generalization in this contingency model follows stoddart’s (2004) notion. the general way of si/soa development is a socio-technical process and we generalize this process by abstracting from the observed instances. due to the single case study design, we are only able to observe one way of si/soa development, i.e., the one in the adage case. subsequently, it is difficult to define more instances (concepts) of the category general way of si/soa development whereas additional instances for skills, budget and company size are easier to derive. to the best of our knowledge, there are neither frameworks/models nor empirical studies that analyze situational factors in the realm of soa. a first step towards a comprehensive list using more case studies has been made by börner et al. (2011). further empirical as well as conceptual research should strive to extract and develop models and causal patterns with respect to this category. australasian journal of information systems volume 17 number 2 2012 99 model of soft factor transition (msft) the model fragment which ties together hypotheses 3, 5, and 6 is depicted in figure 5. our observations (underlined concepts) and the resulting cause-and-effect chain show that a team with only little soa experience produced a highly flexible service-oriented architecture and highly reusable services. in our hypotheses, experience leads to a distinct understanding of services (h3). this results in different granularities and directions of approaches (h5) and subsequently in a certain degree of flexibility and reusability (h6). hence, the experience of project teams influences flexibility and reusability as well as the direction of the identification approach through a chain of cause-andeffect relationships as depicted in figure 5. understanding of services granularity experience no experience little experience comprehensive experience process-oriented user-oriented technical more…? fine-grained middle-grained coarse-grained h3 h5 h6 flexibility reusability high low high low underlined concept = our observationlegend: = category direction of approach top-down meet-in-themiddle bottom-up figure 5: causal link of experience and flexibility/reusability (model of soft factor transition – msft) finding further plausible and alternative instances for this msft is easier than in the case of the contingency model. the classification that is used to abstract categories from concepts is now inverted to identify new instances, i.e., concepts that could not have been observed in the case study. again, as shown in figure 1, this step moves back from the type to the instance level but still increases the degree of abstraction from the raw data. outlining conceivable and alternative instances, we are able to describe project patterns as shown in figure 6. however, these patterns are not proven theories but rather hypotheses in terms of plausible tendencies. theoretically, the number of possible patterns equals the number of all combinations of concepts. however, an identification of plausible patterns will be a contribution itself. certain combinations of concepts might not be found in any pattern due to either conflicting goals (e.g., coarse-grained services and high flexibility) or incompatible preconditions and goals (technical understanding of services and top-down approach for their identification). all major concepts of every category should be included but at the same time restricted in number as far as possible. only those concepts that help to differentiate one project from another should be part of the framework. identifying pattern could be impeded if concepts are too granular (e.g., a scale from 1 to 7 for reusability instead of “high” and “low”) or not disjunct (e.g., “technical” or “functional” understanding of services). australasian journal of information systems volume 17 number 2 2012 100 category concept experience understanding of services granularity flexibility no experience technical fine-grained high little experience comprehensive experience process-oriented user-oriented more… middle-grained coarse-grained low our observation possible patternlegend: direction of approach top-down bottom-upmeet-in-the-middle reusability high low figure 6: project patterns teams with little soa experience and limited knowledge regarding business functions and processes are forced to rely on the experience they have, which is basically technical. we assume that developing an soa based on the knowledge stemming from experience made with traditional, more technically-driven software engineering paradigms, for example, object-oriented programming or modules, results in fine-grained services. the understanding of services can be seen as the generative mechanism in this model fragment. we could assume that a very experienced team might have a process-oriented understanding which leads to rather coarse-grained services that provide less flexibility. since flexibility is regarded to be a big advantage of soas, this causal link is counter-intuitive. we would expect an experienced team to be able to reap the benefits of soas, including an enhanced flexibility. figure 6 illustrates our observed pattern (dotted line) and the assumed possible pattern described above (dashed line). however, we have to keep in mind that more experience does not necessarily lead to one specific understanding of services but enables an adequate choice of approach and understanding, as argued in hypothesis 3. a comprehensive experience, for instance, is not closely knit to a process-oriented understanding and could as well produce middle-grained services. a large number of patterns are thus conceivable. we argue that – depending on situation-specific circumstances in projects – the resulting patterns represent certain types of projects. an identification of such project types could lead to a targeted application of methods supporting soa development and the identification of services. project types can define situations in the sense of situational method engineering and would subsequently contribute to a more sophisticated service identification through the use of situational methods (bucher et al., 2007). observations 7 and 8 show that the technical understanding of services in the adage project lead to fine-grained services and an almost pure bottom up approach for identifying services. in this regard, the lack of experience once again leads to a rather object-oriented thinking which reveals another generative mechanism at work in our case study. in hypothesis 5, these observations are generalized. undoubtedly, there is an effect of this understanding on granularity and the direction of service identification approaches. actually, the question of granularity continues to be an open issue among scientists and practitioners alike and constitutes an active research area (artus, 2006). australasian journal of information systems volume 17 number 2 2012 101 success model (sm) many observations and hypotheses are bound to the notion of success. naturally, the primary objective of every project is to be completed successfully. this is also true for soa implementation projects. but how can you evaluate the success of such a project? our observations and hypotheses utilize several success measures that are influenced by a number of different concepts in our model illustrated in figure 2. in our observations, we find success factors such as reusability and user satisfaction. they are classified to technical and business-oriented success measures, respectively. both success factors can be aggregated to the category soa success. however, they take different view points and usually represent interests of different stakeholders. on the one hand, reusability is a rather technical aspect. subsequently, in most cases, it is pursued by the it department that implements an soa. the number of business processes that invoke a service and the frequency of invocation can serve as a unit of measurement. if the it department can prove that a service is reused frequently in many processes, it is easy to argue that maintenance and development of this single service is more cost efficient than having the same functionality provided by multiple applications scattered across the organization. on the other hand, user satisfaction is clearly a business goal. users have little interest in cost savings or technical demands on the company’s side. instead, they appreciate flexibility and a timely satisfaction of their demands. hence, business processes must be flexible and agile to guarantee a certain degree of user satisfaction. as argued before, a successful soa can cater to these demands. a number of success measures can be employed (see, e.g., the framework by börner and goeken (2009a)). all of them would serve as further instances of our success categories. business departments can thus consider flexibility and time-to-market as important factors. from the technical side, autonomous services or a high degree of standardization might be important. in both cases, the measurement of success factors is far from trivial, and appropriate measures will have to be defined to operationalize the measurement of soa success. to which extent technical and business-oriented success factors are aligned or not is an interesting question that is left to further research. based on a literature review and expert interviews, lee et al. (2010) identify 20 critical success factors in soa implementation. however, their interrelationships and contribution to success are not clear. findings and implications in our analysis we identify factors and aspects (concepts and categories) relevant in the adage project. with respect to our research questions our findings for soa development projects are the following:  we identified circumstances (context factors) that influenced the proceedings of the project. we were able to observe how some of these factors (e.g., funding) affected an soa project and incorporated these observations into our hypotheses and model fragments.  we found that the understanding of services and related skills significantly influence the way of soa development. several hypotheses and our models show that concepts such as granularity or the direction of the identification approach are more or less directly affected by the understanding of services (which is usually subject to change with increasing experience).  due to the absence of a method for service identification, we were not able to observe a stringent application of software development techniques. software development was mostly intuitive and bound to few developers because of the limited scope of the project. an astonishing finding is that the “absence of a formal method“ apparently has had no negative impact. overall, we have made 16 observations and subsequently developed seven hypotheses and three model fragments. limitations stem from at least two sources: australasian journal of information systems volume 17 number 2 2012 102  due to the single case study design, our hypotheses and model fragments are limited and it has to be taken into account, that there are different concepts and categories to consider in more comprehensive models. to extend the models, further research is desirable.  additionally, the model fragments and hypotheses must be seen as tendencies, explaining plausible mechanisms. their strength is that they describe possible cause-effect-relationships and give insight into the generative mechanisms at work. they are limited insofar as they are not tested yet. hence, we neither can be sure that they are representative; nor do we know about the magnitude of relationships of concepts and/or categories. using multiple case studies is one possibility to find supporting or contradicting evidence for the generated hypotheses. this would enable a cross-case pattern search and underpin the validity of the results (eisenhardt, 1989). further interviews and document examination could be a basis for qualitative or (in large numbers) even quantitative cross-sectional analyses. in the following, we will discuss key findings and limitations of the models and their possible interplay. in our contingency model, we capture the influence of a number of context factors on the way of si/soa development. an extension of the list of factors, which form part of the general context, will be left to future research and can be achieved for instance through further case studies or expert interviews. accordingly, more categories (and relating concepts) such as a management commitment (with the possible instances ‘high’, ‘medium’, ‘low’) might be discovered and it should be analyzed, if this influences the way of si/soa development e.g. by inducing top down approaches incorporating strategic aspects. moreover, based on only one case study, it is impossible to determine the extent to which a single factor contributes to certain outcomes. after conducting more case studies, a factor analysis could help to understand more reliably how the combination of identified factors affects the use of methods. since there is already a large body of literature that provides a comprehensive list of context factors for domains different from soa, future research should concentrate on identifying those that have a significant impact on soa implementation projects and, in doing so, reason about the mechanisms more deeply. quantitative research should aim at validating the resulting hypotheses and models. the model of soft factor transition links experience with soa design and implementation aspects (flexibility and reusability) and provides a description of a generative mechanism which is likely to be at work. one question for future research that arises from this model is if a technical (resp. businessoriented) understanding of services necessarily leads to fine-grained (resp. coarse-grained) services and a bottom up (resp. top down) approach. most likely, more experience would enable the project team to flexibly adapt granularity, create services on different levels of granularity, or apply a top down (resp. a hybrid) approach. further case studies would help to shed light on these cause-andeffect relationships. another intriguing question is whether one of these mindsets yields more success, and if this is the case, by which measure? according to most literature, business orientation is crucial for successful soa implementations because only this focus ensures that services can support business processes. however, the adage case shows that an almost pure bottom up approach and very fine-grained services stemming from a technical understanding still can result in a successful soa implementation. of course, the adage architecture is far from being business process-oriented. nonetheless, it can be considered a success. since case studies can be used for generating hypotheses (like in this paper) and testing hypotheses (de vries, 2005), a multiple case study could be useful to investigate correlations of service understanding and success in soa projects. australasian journal of information systems volume 17 number 2 2012 103 since relationships and interdependencies in our model fragments are designed based on the evidence retrieved from the adage project, they might be incomplete. literature e.g. argues that experience is one possible source to improve skills (see, e.g., adelson and soloway (1985), dokko et al. (2009), guile (2002)). thus, further empirical evidence might support interdependencies found in literature and make an adjustment of our model necessary. our success model shows that multiple concepts more or less directly influence success of an soa. this success can be viewed from different perspectives, i.e., technical or business-oriented. a model illustrating how these success factors contribute to the overall success of an soa implementation could be a result of further empirical research. currently, the success measures in our model are not well operationalized. future research should improve the conceptualization of success measures as well as their measurement. it is necessary to describe how to measure these factors and which units to use. finally, an evaluation of the importance of a single concept (e.g., the direction of approach) and to what degree it influences the success of an soa would be desirable. such quantification could be achieved by a factor analysis of context factors. practitioners would be able to identify the setting of their specific project in advance and subsequently adjust or influence certain conditions (i.e., concepts) in order to improve the outcomes of their project. a framework giving advice on which measures could improve a project’s results based on its setting is another objective for future research. a targeted application of methods based on identified project patterns (figure 6) could thus improve the probability of success for soa projects. the model fragments cannot be treated as being independent. on the one hand, there are evident relationships that can be traced in figure 2. obviously, the notion of success which is operationalized by certain measures integrates the cm and the msft. on the other hand, there are most likely more links that have not been observed in our case. the concept of skills in the cm is probably related to what is called experience in the msft. moreover, the way of si/soa development in the cm might be influenced by the direction of the identification approach, which is currently not part of the contingency model but of the msft. hence, we assume that there are more causal links on several levels of our proposed model so that the latter will have to be adjusted after more elaborate work on this topic is performed. discussion, limitations and further research our research endeavor presented in this paper can be characterized against the background of the conceptualizations of generalization we presented in section 2.2. regarding the types of generalization presented by lee and baskerville (2003), we used the two types concerned with generalizing from empirical statements because there are no appropriate theories in this field of research yet. firstly, by formulating our observations in activity four, we generalized from data to description. secondly, in activities five and seven, we generalized from description to theory. this second type was applied twice, namely in generating hypotheses and in developing model fragments. the types of outcomes that are described by walsham (1995) provided us with guidance to distinguish the results of our research. concepts and categories on different levels of abstraction are the main elements of observations, hypotheses, and model fragments. these main elements are similar to the concepts that walsham considers to be an outcome of qualitative research. however, he does not draw the distinction between different levels of abstraction. the generation of theory is reflected in several steps of our research process. firstly, we developed hypotheses as an abstraction australasian journal of information systems volume 17 number 2 2012 104 from our observations. secondly, three models describing cause-and-effect relationships are presented as middle range theories. by following popay et al. (1998), our intention was to generalize logically to a theoretical understanding. in doing so, we omitted other approaches like statistical, naturalistic, analytical, and analogical generalization (see 2.3). the resulting limitations of this generalization approach are discussed in section 3.7. the generalization approach following “generic social processes” (stoddart 2004) or socio-technical processes and the idea of mechanism-based explanations (causal mechanisms and causal generalization (see hedström and ylikoski 2010 and woodward 2002)) fostered a more procedural perspective resulting in cause-and-effect relationships in order to structure the research process, we adopted the sequence of activities from eisenhardt (1989). furthermore, we utilized her idea of “enfolding literature” to underpin the validity and to reach a higher conceptual level in a field where theories do not exist. this supported the logical generalization. in addition, we extended this methodological basis stemming from the literature by the utilization of abstraction mechanisms from conceptual modeling. to the best of our knowledge, abstraction mechanisms have not been used in the realm of qualitative research yet. we believe that they provide a stable set of well described (and in some way formalized) ways to abstract from instances to types and, hence, to relate different levels of abstraction applying a clear semantic. thus, they might reduce biased abstractions. in this respect, it could be argued that these mechanisms reduce creativity and limit abduction in the qualitative data analysis. however, we consider the guidance an advantage rather than a limitation. in all likelihood, these mechanisms have been used implicitly by researchers before, but an explicit description of the abstraction mechanisms intends to add rigor to the abstraction process to better justify the findings and to better support logical generalization and generalization by abstraction. from our point of view, this backing in methodological work enhances reliability and traceability of our proceeding and of the findings in the case study analyses. furthermore, it adds transparency to the basis upon which generalizations can be made and justified. on the other hand, it is useful in revealing limitations. it makes clear that usually generalization about populations is not possible. in addition, it reveals that the findings are untested theories and tendencies, “moderatum generalizations” (p.215) as williams (2000) puts it. the application of different approaches and conceptualizations enabled us to develop a basis upon which we can identify and justify the possibilities and limitations of generalization. nevertheless, one can argue that this is rather an eclectic approach to enhance traceability and reliability. hence, further research in the realm of qualitative methodology should be directed toward developing comprehensive and integrated frameworks, synthesizing different viewpoints of generalization. sound frameworks may provide valuable guidance for endeavors in qualitative research. as described previously, there are more abstraction mechanisms in the literature of conceptual modeling. we assume that they might be useful for qualitative research approaches as well, but a more detailed elaboration, e.g., the development of an appropriate framework for the use of abstraction mechanisms in qualitative research requires further intensive methodological work. conclusion since a common understanding of generalization in qualitative information systems research is missing, this paper offers an overview of existing approaches and conceptualizations. different types and outcomes are described in table 1 which is referenced throughout the paper to classify the australasian journal of information systems volume 17 number 2 2012 105 methods used and the results generated. we extended the notion of generalization by introducing abstraction mechanisms of conceptual modeling. from a methodological perspective, we showed that abstraction mechanisms from the field of conceptual modeling can be successfully applied in conjunction with existing approaches in the realm of qualitative information systems research. in this respect, the paper represents an attempt to bring some clarity to the many different conceptualizations in the methodological discourse on generalizability and to discuss and demonstrate their applicability. from another point of view, the paper contributes to the empirical research in soa, especially in soa development and service identification. based on an explorative single case study, we developed hypotheses and model fragments that abstract from the subjective understanding of the project participants and observations made by the researchers. by referring to the aforementioned conceptualizations and mechanisms, we tried to make the research process transparent and traceable, so soa researchers could appraise the reliability of the results. further research is needed and desirable regarding both perspectives. the model fragments and hypotheses described are based on case data of a single case study and should be complemented by more cases in order to extend the models. further research should also aim at verifying and supporting the results applying qualitative and/or quantitative methods. the methodological discourse on generalizability in qualitative research should lead to comprehensive and integrated frameworks giving sound guidance to research projects in this area. references adelson, b. & soloway, e. (1985) "the role of domain experience in software design", ieee transactions on software engineering, vol 11 no 11, pp 1351-1360. analyti, a., theodorakis, m., spyratos, n. & constantopoulos, p. (2007) "contextualization as an independent abstraction mechanism for conceptual modeling", information systems, vol 32 no 1, pp 24-60. anderson, d., howell-barber, h., hill, j., javed, n., lawler, j. & li, z. (2005) "a study of web services projects in the financial services industry", information systems management, vol 22 no 1, pp 66-76. artus, d. j. n. "soa realization: service design principles". [http://www.ibm.com/developerworks/webservices/library/ws-soa-design/], accessed: 25-082010. babbie, e. (2010) the practice of social research, wadsworth, belmont. baskerville, r., cavallari, m., hjort-madsen, k., pries-heje, j., sorrentino, m. & virili, f. (2005) "extensible architectures: the strategic value of service-oriented architecture in banking", european conference on information systems, regensburg, germany. becker, a., buxmann, p. & widjaja, t. (2009) "value potential and challenges of service-oriented architectures a user and vendor perspective", 17th european conference on information systems (ecis), verona, italy. becker, h. s. (1990) "generalizing from case studies", in: e.w. eisner and a. pecking (eds.), qualitative inquiry in education: the continuing debate, teacher's college press, new york london, pp 233-242. benbasat, i., goldstein, d. k. & mead, m. (1987) "the case research strategy in studies of information systems", mis quarterly, no 11, pp 269-386. http://www.ibm.com/developerworks/webservices/library/ws-soa-design/%5d australasian journal of information systems volume 17 number 2 2012 106 birkmeier, d., glöckner, s. & overhage, s. (2009) "a survey of service identification approaches", enterprise modelling and information systems architectures, vol 4 no 2, pp 20-36. börner, r., goeken, m. & rabhi, f. (2012) "soa development and service identification a case study on method use, context and success factors", frankfurt school working paper, forthcoming. börner, r., goeken, m., kohlborn, t. & korthaus, a. (2011) "context factors for situational service identification methods", 6th international conference on internet and web applications and services (iciw), xpert publishing services, st. maarten, the netherlands antilles, pp 35-42. börner, r. & goeken, m. (2009a) "identification of business services literature review and lessons learned", 15th americas conference on information systems (amcis), san francisco, calif., paper 162. börner, r. & goeken, m. (2009b) "methods for service identification", in: daniel moldt, juan c. augusto and u. ultes-nitsche (eds.), 7th international workshop on modelling, simulation, verification and validation of enterprise information systems (msvveis), milan, italy, pp 76-84. boland, r. j., newman, m. & pentland, b. t. (2010) "hermeneutical exegesis in information systems design and use", information & organization, vol 20 no 1-20. bucher, t., klesse, m., kurpjuweit, s. & winter, r. (2007) "situational method engineering on the differentiation of "context" and "project type"", in: jolita ralyté, sjaak brinkkemper and brian henderson-sellers (eds.), situational method engineering: fundamentals and experiences, springer, boston, pp 33-48. byrne, e. & sahay, s. (2007) "generelizations from an interpretive study: the case of a south african community-based health information system", south african computer journal, no 38, pp 8-19. darke, p., shanks, g. & broadbent, m. (1998) "successfully completing case study research: combining rigour, relevance and pragmatism", information systems journal, vol 8 no 4, pp 273-289. de vries, e. j. (2005) "epistemology and methodology in case research: a comparison between european and american is journals", 13th european conference on information systems, regensburg, germany. dokko, g., wilk, s. l. & rothbard, n. p. (2009) "unpacking prior experience: how career history affects job performance", organization science, vol 20 no 1, pp 51-68. doolin, b. (1996) "alternative views of case research in information systems", australasian journal of information systems, vol 3 no 2, pp 21-29. eisenhardt, k. m. (1989) "building theory from case study research", the academy of management review, vol 14 no 4, pp 532-550. eisenhardt, k. m. & graebner, m. e. (2007) "theory building from cases opportunities and challenges", academy of management journal, vol 50 no 1, pp 25-32. elfatatry, a. (2007) "dealing with change: components versus services", communications of the acm, vol 50 no 8, pp 35-39. gadamer, h.-g. (1976) "the historicity of understanding", in: p. connerton (ed.), critical sociology, selected readings, penguin books, harmondsworth, uk, pp 117-133. geertz, c. (1973) the interpretation of cultures, basic books, new york. goeken, m. (2006) entwicklung von data-warehouse-systemen. anforderungsmanagement, modellierung, implementierung, deutscher universitäts-verlag, wiesbaden. australasian journal of information systems volume 17 number 2 2012 107 gregor, s. (2006) "the nature of theory in information systems", mis quarterly, vol 3 no 30, pp 611-642. guile, d. (2002) "skill and work experience in the european knowledge economy", journal of education & work, vol 15 no 3, pp 251-276. hedström, p. & ylikoski, p. (2010) "causal mechanisms in the social sciences", annual review of sociology, vol 36, pp 49-67. hovorka, d. s. & lee, a. s. (2010) "reframing interpretivism and positivism as understanding and explanation: consequences for information systems research", 31st international conference on information systems (icis), st. louis. klein, h. h. & myers, m. d. (1999) "a set of principles for conducting and evaluating interpretive field studies in information systems", mis quarterly, vol 23 no 1, pp 67-94. kohlborn, t., korthaus, a., chan, t. & rosemann, m. (2009a) "identification and analysis of business and software services a consolidated approach", ieee transactions on services computing, vol 2 no 1, pp 50-64. kohlborn, t., korthaus, a., chan, t. & rosemann, m. (2009b) "service analysis a critical assessment of the state of the art", 17th european conference on information systems (ecis), verona, italy. lee, a. (1991) "integrating positivist and interpretive approaches to organizational research", organization science, vol 2 no 4, pp 342-365. lee, a. s. (1989) "a scientific methodology for mis case studies", mis quarterly, vol 13 no 1, pp 33-50. lee, a. s. & baskerville, r. l. (2003) "generalizing generalizability in information systems research", information systems research, vol 14 no 3, pp 221-243. lee, j. h., shim, h.-j. & kim, k. k. (2010) "critical success factors in soa implementation: an exploratory study", information systems management, vol 27 no 2, pp 123-145. lincoln, y. s. & guba, e. g. (1985) naturalist inquiry, sage publications, newbury park london new delhi. luthria, h. & rabhi, f. a. (2009) "building the business case for soa: a study of the business drivers for technology infrastructure supporting financial service institutions", in: dennis kundisch, daniel j. veit, tim weitzel and christoph weinhardt (eds.), enterprise applications and services in the finance industry, springer, pp 94-107. mattos, n. m. (1989) "abstraction concepts: the basis for data knowledge modeling", in: carlo batini (ed.), 7th international conference on entity-relationship approach: a bridge to the user., north holland, pp 473-492. merton, r. (1967). on sociological theories of the middle-range. in r. merton (ed.), on theoretical sociology: five essays old and new. new york: free press. myers, m. d. (2009) qualitative research in business & management, sage publications, london. olivé, a. (2007) conceptual modeling of information systems, springer, berlin heidelberg. parsons, j. & wand, y. (2008) "using cognitive principles to guide classification in information systems modeling", mis quarterly, vol 32 no 4, pp 839-868. popay, j., rogers, a. & williams, g. (1998) "rationale and standards for the systematic review of qualitative literature in health services research", qualitative health research, vol 8 no 3, pp 341-351. australasian journal of information systems volume 17 number 2 2012 108 prus, r. c. (1994) "generic social processes: intersubjectivity and transcontextuality in the social sciences", in: mary lorenz dietz, robert c. prus and william shaffir (eds.), doing everyday life: ethnography as human lived experience, copp clark longman, toronto, ontario. schutz, a. (1954) "concept and theory formation in the social sciences", the journal of philosophy, vol 51 no 9, pp 257-273. smaling, a. (2003) "inductive, analogical, and communicative generalization", international journal of qualitative methods, vol 2 no 1, pp 52-67. stake, r. (1995) the art of case research, sage publications, newbury park. stebbins, r. a. (2001) exploratory research in the social sciences, sage publications, thousand oaks. stoddart, m. c. j. (2004) "generalizability and qualitative research in a postmodern world", graduate journal of social science, vol 1 no 2, pp 303-317. tsoukas, h. (2009) "craving for generality and small-n studies: a wittgensteinian approach towards the epistemology of the particular in organization and management studies", in: david a. buchanan and alan bryman (eds.), the sage handbook of organizational research methods, sage, london et al., pp 285-301. van maanen, j. (1979) "the fact of fiction in organizational ethnography", administrative science quarterly, vol 24 no 4, pp 539-550. walsham, g. (1995) "interpretive case studies in is research. nature and method", european journal of information systems, no 4, pp 74-81. walsham, g. (2006) "doing interpretive research", european journal of information systems, no 15, pp 320-330. weber, r. (2004) "the rhetoric of positivism versus interpretivism: a personal view", mis quarterly, vol 28 no 1, pp iii-xii. williams, m. (2000) "interpretivism and generalisation", sociology, vol 34 no 2, pp 209-224. woodward, j. (2002) "what is a mechanism? a counterfactual account", philosophical science, vol 69, pp 366-377. yin, r. k. (2003) case study research design and methods, sage publications, thousand oaks london new delhi. the australasian journal of information systems (ajis) began in 1993 and was based on the simple realisation that there was a need for an academic ‘outlet’ for is research in australia australasian journal of information systems volume 17 number 2 2012 5 a multi-dimensional framework to assist in the design of successful shared services centres mark borman the university of sydney email: mark.borman@sydney.edu.au abstract organisations are increasingly looking to realise the benefits of shared services yet there is limited guidance available as to the best way to proceed. a multi-dimensional framework is presented that considers the service provided, the design of the shared services centre and the organisational context it sits within. the specific dimensions identified are task, strategy, structure, management processes, individual skills, information technology, environmental conditions, history and organisational resources. case studies are then used to determine what specific attributes from each dimension are associated with success. it is concluded that there appears to be broadly standard patterns of attributes across the dimensions that differentiate between successful, moderately successful and limited success shared services centres (sscs). keywords: shared services, alignment, information technology, success, attributes introduction why look at shared services? primarily because while they have been promoted as an emerging area offering enormous potential, limited guidance is available to organisations. bergeron (2003) suggests that approximately 50% of fortune 500 companies have established some form of shared services operation. he also provides the example of bristol myers squib’s global business service unit realising annual savings of us$1.5billion. rayner (2006) similarly highlights the cost savings achieved by reuters while cecil (2000) gives the example of ford being able to reduce its finance department staffing from 14,000 to 3,000 through the introduction of shared services. longwood et al. (2007) however suggest that success is not guaranteed and that many organisations will fail to achieve their objectives. one recent example was the government of western australia which sought to merge finance and payroll processing services across multiple departments. the project, started in 2007, and was estimated to cost au$82 million and deliver annual savings of au$57 million. however by the time of termination au$401 million had been spent (kerr 2011). shared services are of specific interest from an information systems (is) perspective because it has increasingly been recognised by researchers in the field that there is merit in looking beyond the is itself to examine the functions and activities that are enabled by it – see for example willcocks et al. (2007), macgregor et al. (2009), roseman et al. (2009), leonardi et al. (2008) and kerr et al. (2010). shared services would appear to represent such an activity with authors such as hagel iii et al. (2001) arguing that is will underpin their delivery. indeed the shared services in western australia referred to above was founded upon the establishment of a common enterprise resource planning (erp) system (kerr 2011). yet while a significant body of research has been constructed regarding other modes of organising business activities that rely and build upon is – see for example borman (2006) with regard to business process outsourcing – relatively little attention has been paid to shared services. the majority of the available advice to organisations is currently mailto:m.borman@econ.usyd.edu.au australasian journal of information systems volume 17 number 2 2012 6 provided by consultants – for example schulman et al. (1999) or quinn et al. (2000) – and is often anecdotal in nature. while an academic literature is slowly emerging, to date it has often been unstructured with many of the claims or assertions unsupported by empirical evidence – see for example publications by shah (1998) and kreklow et al. (2007). there are notable exceptions – for example janssen et al. (2006) – but even here the research has often been restricted to a relatively narrow domain – in that instance to the public sector. the objective of the current paper is to develop a more comprehensive analysis of shared services. in particular the research examines a range of sectors through the lens of a multi-dimensional framework to determine what attributes might be commonly associated with successful shared services. as such it should provide enhanced guidance for organisations embarking on shared services initiatives. the remainder of the paper comprises two sections. the first outlines a set of dimensions that can be used to analyse shared services centres (sscs) . the second examines shared services in 11 organisations in australia through the lens of those dimensions seeking to determine if there is a prevalent pattern or alignment of attributes across them associated with success. before progressing however it is useful to define what shared services are. many definitions with slightly different nuances exist see schulz et al. (2010) for an extensive review. schulman et al. (1999) for example define them as “the concentration of company resources performing like activities, typically spread across the organization, in order to service multiple internal partners at lower cost and with higher service levels, with the common goal of delighting external customers and enhancing corporate value” (p9) while bergeron (2003) suggests “shared services is a collaborative strategy in which a subset of existing business functions are concentrated into a new semi-autonomous business unit that has a management structure designed to promote efficiency, value generation, costs savings and improved service for the internal customers of the parent corporation, like a business competing in the open market” (p3) the essence though is that a shared service is one where the provision of a back office service – such as payroll processing, accounts payable or foundational it services – is consolidated within a single area of an organization (longwood et al. 2007; quinn et al. 2000; ulbrich 2006). shared services: dimensions, alignment and performance the overarching objective of this paper is to determine if there are one or more alignments of attributes, across a series of dimensions, associated with successful sscs. the following section will first discuss prior research into alignment and its association with success before outlining the specific conceptualisation adopted here. it will then identify a set of dimensions across which an alignment of attributes associated with successful sscs can be sought. finally the view of how success will be assessed is presented. research examining alignment has often struggled because it has failed to clearly define what is meant by alignment (see chan and reich, 2007). alignment has been described in a variety of ways – for example as fit (white 1986), support (luftman et al. 1993) or configuration (mintzberg et al. 1999). perhaps the most comprehensive consideration of alignment is that of venkatraman (1989) who identified, and defined, six alternative definitions – moderation, mediation, matching, gestalts, profile deviation and covariation. mediation, moderation and matching look only for a fit between two variables – an approach which child (1975) suggests can lead to potential contradictions. of the remaining multivariate approaches, profile deviation has been criticised for its focus on ideal types which are not generally observable in reality (lee et al. 2004; weber 1978). the gestalt approach australasian journal of information systems volume 17 number 2 2012 7 however is well accepted in the is literature (drazin et al. 1985; lee et al. 2004; pollalis 2003) and has been applied previously to outsourcing (kern 1997). at its core, according to miller (1982), is a focus on the whole, describing relationships between a range of elements, to identify unique forms or gestalts rather than looking at simple associations between a restricted set of variables . while avoiding issues of causality (hrebiniak 1981), a focus on gestalts recognises that it is possible for a variety of organisational forms, or configurations, to emerge and flourish within a particular environment (hannan et al. 1977; katz et al. 1966) thus avoiding tendencies towards prescription and the best solution. as compared to the remaining definition of alignment – covariation – venkatraman (1989) suggested that the two are very similar differing “only in relation to the degree of specification of the functional form” (p436) with gestalts viewed “as products of cluster analysis .. whereas covariation is modeled as factor analysis” (p436). it is proposed here that aspects of the gestalt and covariation approaches can potentially be usefully combined. the principles of covariation can be used to identify relevant attributes for each proposed alignment dimension, with a gestalt approach then being used to determine whether organisations can be grouped based upon variations in those attributes. with regard to identifying the specific dimensions to be aligned, the focus is on the requirements for a particular function – shared services – to be successful. a review of the literature suggests that a number of authors have considered the alignment requirements between functions within an organisation – for example hooper et al. (2007) with regard to it and marketing – or between one function and the broader business operation – chan et al. (1997), for example, consider the relationship between it and business strategy. however there appears to be scant literature examining the alignment requirements for the effective operation of specific single function or activity. exceptions are hall (1962) and van de ven et al. (1974) but these have a narrow focus on the relationship between the nature of a task and one other factor – the degree of bureaucratisation in the first instance and the work-unit structure in the second. by far the most extensive and developed body of literature in the area relates to the examination of alignment at the organisation level. numerous authors have suggested that the complexity of the modern organization needs to be understood from the perspective of multiple dimensions with its success being determined by the congruence, fit or alignment amongst them (galbraith 2002; henderson et al. 1993; leavitt 1965; nadler et al. 1997; scott-morton 1991). leavitt (1965) for example emphasised that the dimensions “are highly interdependent .. so that change in any one usually results in compensatory (or retaliatory) change in others” (p1145). this literature was therefore used as the starting point for indentifying a set of functional alignment dimensions. the actual approach adopted is a synthesis of the frameworks developed by scott-morton (1991) and nadler et al. (1997). chan et al. (2007) in their comprehensive review of the organisational design literature trace many alignment models back to scott-morton (1991) which itself draws significantly from an earlier work by leavitt (1965). however scott-morton (1991) is not without its own limitations. while recognising that the organisation cannot be considered in isolation but needs to be positioned within an environment he did not seek to examine that environment in detail. nadler et al. (1997) suggested that the “givens or setting within which the organisation must operate” (p29) was made up of three components: environmental conditions, organisational resources and history. there are similarities between the nadler et al. (1997) categories and those of other, well respected, authors, such as porter (1991) and mintzberg (1998), who have sought to examine the influence of an organisation’s environment on its performance. however the former was chosen as it appears to more directly capture both the range of influences and their essential essence. for example, the components allow for a broader consideration of influences than the five forces approach of porter (1991) and resources and history more clearly encapsulate that it is past decisions and the ability to utilise an organisation’s resources that is of interest than do the more indirect age, size and power constructs of mintzberg (1998). australasian journal of information systems volume 17 number 2 2012 8 one further addition was made to the scott-morton (1991) framework. without explanation or justification, scott-morton (1991), and the authors who subsequently built upon and developed his work, omitted consideration of what leavitt (1965) described as the “industrial organizations’ raisons d’être” (p1144) – the specifics of the task it has been designed to undertake. as outlined in march (1965) there are many different types of tasks with varying characteristics that will influence what constitutes an appropriate organisational design. task was therefore added as a dimension. the complete framework used here thus includes the following dimensions:  task  strategy  structure  management processes  individual skills  information technology  environmental conditions  history  organisational resources a problem with many of these dimensions is that they have been left partially or wholly undefined. below the work of leading authors who have considered organisational design is discussed as it relates to each dimension. an initial outline of each dimension, mapped from the level of an organization to that of a function – shared services is then provided. clearly alternative perspectives are possible – there is considerable debate regarding many of the dimensions – see faulkner (2002) for a comprehensive review of the literature relating to strategy for example – but the authors cited are recognised as pre-eminent in their fields. the objective is not a complete review of the literature but to establish a starting point relevant to the objectives of the research (walsham 1993). task porter (1991) recognised that within an organisation there will be many different types of activity – often with fundamentally different characteristics. he classified activities though on the basis of what function they fulfill, for example inbound logistics, rather than on the basis of those characteristics. mintzberg (1998) however did look at the nature of the work itself categorising it on the basis of the proportion of the total activity or process included and the degree to which it is routine with pre-specified actions requiring limited discretion. application to shared services: the extent to which the activities provided by the ssc are routine and provided in their entirety. strategy chandler (1962) defined strategy “as the determination of the basic long term goals and objectives of an enterprise and the allocation of resources necessary for carrying out these goals” (p13). similar general definitions have been provided by other authors such as andrews (1971) and venkatraman (1985). porter (1985) was more specific suggesting three alternative strategies – differentiation, cost leadership and focus – that organisations should work towards. application to shared services: the long term goals of the ssc, with respect to differentiation, cost leadership and focus, and approaches to their realisation. australasian journal of information systems volume 17 number 2 2012 9 structure chandler (1962) defined structure as “the design of organization through which the enterprise is administered’ (p14). etzioni (1965) provided more detail suggesting it is the “means used by an organization to elicit the performances it needs and to check whether the quantities and qualities of such performances are in accord with organizational specifications” (p650). application to shared services: the mechanisms by which the performance of the ssc is enabled and monitored. management processes according to scott-morton (1991) management processes relate to the means through which the general employee body within an organization are directed and controlled (p12). the domain of influence of management processes therefore differs from that of structure – though the two ideally reinforce one another. structure focuses in broad terms on establishing hierarchies, rules policies and procedures. management processes relate to the operational requirements and mechanisms for directing specific individuals or groups. mintzberg (1998) suggested that consideration should also be given to how relationships – particularly boundary crossing ones – are developed and maintained. application to shared services: the means through which management direct and supervise ssc employees and establish and maintain relationships between the ssc and the broader organisation. individual skills rather than specific skills scott-morton (1991) and others, such as taylor (1911), davis et al. (1955) and dunphy et al. (1998), have focused on job design and the skills required of employees especially with regard to the potential for deskilling, up-skilling or multi-skilling. application to shared services: the current and required skill base of the ssc. information technology scott-morton (1991) defined information technology in terms of six components – hardware, software, networks, workstations, robotics and smart chips. perhaps more useful as a guide, than such a focus on specific artifacts, is the work of authors such as weill et al. (1996), who examine the contribution of technology to organisational success from the perspective of overarching design principles – particularly with regard to the significance attached to technology and the extent to which that technology has been designed with the purpose of serving as an underpinning infrastructure. application to shared services: the design and contribution of the it used by the ssc. environmental conditions nadler et al. (1997) proposed that consideration of an organisation’s environment should cover “people, other organizations, social and economic forces, and legal constraints.. markets (clients or customers), suppliers, governmental and regulatory bodies, technological and economic conditions, labour unions, competitors, financial institutions and special interest groups” (p29). mintzberg australasian journal of information systems volume 17 number 2 2012 10 (1998), and others such as porter (1991), though suggest that the focus should not be on describing the different elements of an environment but rather on understanding the forces that influence its attractiveness. porter (1991) primarily considered how threatening an environment was while mintzberg (1998) also focused on the variety present (more variety restricting the degree of standardisation that could be introduced into an organisation’s operations). application to shared services: the variety and hostility faced by the ssc. history porter (1991) recognised that “firms inherit positions that constrain and shape their choices” (p104) but did not provide any details. nadler et al. (1997) provided more guidance suggesting that the decisions an organisation took in its past with regard to factors such as focus, behaviour and culture shape the way it functions in the present. application to shared services: the organizational modus operandi inherited by the ssc. organisational resources porter (1991) suggested that a firm accumulates strengths in certain resources and skills – and potential weaknesses in others – which together shape and constrain what it can achieve. nadler et al. (1997) added further granularity by differentiating between tangible – employees, technology, capital – and intangible – reputation, morale – resources. application to shared services: the organisation’s tangible and intangible resources and skills whose presence or absence are seen as key influences on the effectiveness of the ssc. in addition to identifying and defining the dimensions of interest, the research, with its focus on the contribution of alignment to success, requires that what represents success be specified. there has been a long debate in the is field about appropriate measures of success and many alternatives suggested (delone et al. 1992). seddon et al. (1999) and stockdale et al. (2006) have questioned though whether the most commonly used deterministic approaches can effectively capture the concept of success – especially across multiple varied contexts. with the objective of investigating alignment for shared services in general a heterogeneous sample of cases was sought, as outlined in the methodology section below, making it difficult to select objective measures that would be equally applicable to all. consequently a broader, more subjective, approach – informed by the work of walsham (1993) and farbey et al. (1999) – was adopted which allowed interviewees to make, and explain, an assessment based upon the criteria that were important in their context. this has been termed perceived success. such an approach has previously been seen as appropriate for ongoing relationship based settings (dibbern et al. 2004; domberger et al. 2000; willcocks et al. 2007) – of which shared services represent a prime example. success was assessed from the perspective of the ssc not the users of its services. it is recognised that users might have a different viewpoint (walsham 1993) but the focus of the research was on the ssc and whether it attained the goals the organisation had set it. australasian journal of information systems volume 17 number 2 2012 11 methodology given that little research has been conducted on shared services, a case study based approach was deemed to be appropriate (benbasat et al. 1987; strauss et al. 1990). a multi-case approach was taken for the study with a primary emphasis on maximising the likely generalisability of the results since the ultimate objective was to increase understanding of the relationship between ssc design and performance (herriott et al. 1983; yin 1981). as such the focus was on the breadth, rather than the depth, of coverage and the number of interviews conducted within each organisation was restricted. given the research area of interest was the design of shared services the focus was on the ssc not their clients or customers. a total of 11 sscs were interviewed across australia – which is in line with the median number of sites included in multi-site studies as reported by herriott et al. (1983) – in a range of sectors including government, power, transport, building, mining and telecommunications. australia was chosen as it has been recognised as an early adopter with regard to shared services (longwood et al. 2007). interviews were between one and two hours in duration and a semi-structured interview protocol was followed to introduce a degree of commonality while minimising the potential for overlooking the unique aspects of each context (firestone et al. 1982). details of interviewees and their sscs are provided in table 1 and the foundation protocol in appendix 1. table 1: interviewee details shared service centre (ssc) sector age of ssc no. of employees – full time equivalent interviewees ssc1 government 6 years ≈ 700 managing director (md) ssc2 government 9 years ≈ 550 director corporate services (dcs), general manager corporate services (gmcs) ssc3 government 12 years ≈ 540 general manager (gm), finance director (fd) ssc4 construction ≈ 12 years ≈ 80 manager shared services (mss) ssc5 energy 6 years ≈ 120 manager financial services (mfs), group executive shared services (gess) ssc6 energy 4 years ≈ 500 general manager (gm), manager client services (mcs) ssc7 mining ≈ 7 years ≈ 500 general manager shared services (gmss) ssc8 government 6 years ≈ 700 general manager (gm), project director (pd) ssc9 logistics ≈ 8 years ≈ 270 general manager (gm), manager service delivery (msd) ssc10 telecommunications ≈ 20 years ≈ 700 general manager (gm), finance director (fd) ssc11 government ≈ 6 years ≈ 250 general manager (gm), executive manager (em) australasian journal of information systems volume 17 number 2 2012 12 with regard to analysis, data was first reviewed and coded in terms of its relationship to the dimensions identified – an approach in accord with the recommendations of miles et al. (1994) and yin (1981) who suggest organising data “around the substantial topics of the case study” (yin 1981, p60). at the same time a case comparison approach underlay the analysis (yin 1981). the resultant data display tables represent a crucial link in the chain of evidence between the raw data and the subsequent analysis and development of conclusions (yin 1979). given the research focus was on determining whether there are one or more alignments of attributes associated with success it was felt to be inappropriate to develop propositions relating to the contribution of the dimensions individually (hathaway 1995). consequently, at a holistic level the following propositions were developed: proposition 1: there exists at least one alignment of attributes across the dimensions identified that is associated with successful shared services proposition 2: deviations from that alignment will be associated with less successful shared services results as outlined in the methodology the results of the case study interviews were codified and collated into data display tables – see appendix 2 for a summary. table 2 maps the themes identified in the interviews to the definitions of the dimensions outlined previously while table 3 maps attributes to each ssc. the themes are elaborated upon below. success originally it was intended to have a simple successful/unsuccessful dichotomy. however the interviews made it clear that there were a number of sscs that classified themselves as partially successful or “on the way but not there yet”. as such a three point scale was introduced. three sscs assessed themselves as performing on the bottom point of the scale (limited success), three at the middle (moderate success) and five at the top (successful). the explanations behind the assessments can be summarized as: limited success very little if any benefits realised and significant questioning of the merits of shared services and whether to continue moderate success some benefits starting to be realised but a perception that there is still work to be done in convincing the organisation of the merits of shared services and the bulk of the benefits are further down the line successful significant benefits realized table 4 summarises the perception of each ssc regarding its success together with illustrations of the explanations given in support. there did not appear to be any obvious association between the age, sector or size of a ssc and success. those sscs perceived as being successful for example operated in five different sectors, ranged in age from 6 to 20 years and employed from 80 to 700 fte employees. australasian journal of information systems volume 17 number 2 2012 13 table 2: mapping of dimension definitions to interview theme dimension components of definition interviewee themes success perception successful moderate limited task routine transaction processing not expert services complete (entirety) provided/managed end to end multiple service lines strategy long term goal economies of scale approaches to realisation process improvement structure performance enablement compulsory use standardised processes user pays performance monitoring accountable for performance improvement external benchmarking of performance management processes direction leadership focus process & work level understanding large group management expertise transparency control measurement emphasis relationship establishment & maintenance good relationship with the business exit path for staff into business individual skills skill base in place not too low a skill base skill base required enable flexibility information technology contribution of it it as foundation design of it standard it environmental conditions variety standard operating model hostility no cost cutting backdrop not demanding immediate results history organisational ethos inherited strong centre core business focus organisational resources tangible resources available capital investment funds it in situ beforehand intangible resources senior management commitment australasian journal of information systems volume 17 number 2 2012 14 table 3: ssc attributes raised in interviews classific -ation dimension element/attribute limited success (3) moderate success (3) successful (5) s s c 3 s s c 6 s s c 11 s s c 1 s s c 8 s s c 9 s s c 2 s s c 4 s s c 5 s s c 7 s s c 1 0 fundame ntals strategy economies of scale 1 1 1 1 1 1 1 1 1 1 1 task transaction processing 1 1 1 1 1 1 1 1 1 1 1 technology it as foundation 1 1 1 1 1 1 1 1 1 1 1 task multiple service lines 1 1 1 1 1 1 1 1 1 1 1 technology standard it 1 1 1 1 1 1 1 1 1 resources available capital investment funds 1 1 1 1 1 1 1 1 1 history core business focus 1 1 1 1 1 1 1 1 different iators between successf ul or moderate success and limited success strategy process improvement 1 1 1 1 1 1 1 1 environment not demanding immediate results 1 1 1 1 1 1 1 1 structure compulsory use 1 1 1 1 1 1 1 1 1 structure standardised processes 1 1 1 1 1 1 1 1 management measurement emphasis 1 1 1 1 1 1 1 structure accountable for performance improvement 1 1 1 1 1 1 1 structure external benchmarking of performance 1 1 1 1 1 1 1 task provided/managed end to end 1 1 1 1 1 1 1 1 management good relationship with the business 1 1 1 1 1 1 1 1 management process & work level understanding 1 1 1 1 1 1 1 1 management large group management expertise 1 1 1 1 1 1 1 1 skills enable flexibility 1 1 1 1 1 1 1 management transparency 1 1 1 1 1 1 resources it in situ beforehand 1 1 1 1 1 1 skills not too low a skill base 1 1 1 1 1 1 1 different iators between successf ul and moderate or limited success history strong centre 1 1 1 1 environment standard operating model 1 1 1 1 resources senior management commitment 1 1 1 1 1 1 1 management leadership focus 1 1 1 1 management exit path for staff into business 1 1 1 1 structure user pays 1 1 1 1 1 task not expert services 1 1 1 1 1 environment no cost cutting backdrop 1 1 1 1 australasian journal of information systems volume 17 number 2 2012 15 table 4: assessment of success of sscs ssc perception of success sample supporting evidence ssc2 successful “we benchmark.. and hit or beat the benchmark in about 80% of cases.. have people from the private sector coming to learn from us” ssc2-dcs “in two and a half years we’ve saved $29 million in procurement contracts alone” ssc2-gmcs ssc4 “we’re at arms length.. charge on a per transaction basis largely.. we have our kpis.. once you move it out the relationship changes.. more finger pointing if you do not hit them.. but we do not get too much of that discussion now” ssc4mss ssc5 “very visible in terms of data.. users can see how we did and how they did.. see how much they are spending relative to others” ssc5-gess “we have a productivity index.. a basket of services.. and now based on this index we have upped productivity 44% since we started” ssc5-mfs ssc7 “we do a lot of benchmarking.. we’re doing a bit of internal surveying and stuff like that .. positive .. some of the organisations had 40 and 50 people doing what we have six people do now” ssc7-gmss “we’ve won awards” ssc7-gmss “gone from 300 people to 80” ssc7-gmss ssc10 “when i first took that group on there were over 400 people, 66 sites we were processing out. it is down to one site and you know, less than 30 people maximum, so that is a huge reduction” ssc10-gm ssc1 moderate success “moving in the right direction.. now put the basics in place.. a service delivery platform.. metrics.. but early days yet” ssc1-md “starting to get some runs on the board.. as the volume is going up our costs are not..maybe 30 cents in the dollar at the marginal end” ssc1-md ssc8 “on a scale of one to 10, we’re probably about three in terms of our level of sophistication. i think we’ve got a long way to go” ssc8-gm “we’ve got some success .. recognition of shared services . but we still have to translate that into significant cost savings .. some improvements but pretty marginal cuts [in costs]” ssc8-pd ssc9 “customer service wasn’t great, perception wasn’t great of shared services, and i think also the service levels were down and cost was high” ssc9-msd “we started off with over 400 staff in 1999 now we’ve got about 270 doing more work that we did originally and with less charge out back to the units.. still got a way to go.. we’ve got some trust but moving slowly” ssc9-gm ssc3 limited success a lot of systems where they are having difficultiesespecially the hr side of it, the payroll and rostering and all of those sorts of things” ssc3-gm “it has not been received well by the organisation.. the change was not done very well. what we’ve got now is a model that probably needs a lot of customer service focus and needs a restructure” ssc3-fd ssc6 “on the horns of a dilemma.. basic thrust is whether the shared services stays or we get folded back into the lines of business. and so maybe for us, the shared services model failed” ssc6-gm ssc11 “its been a slow road.. problematic.. going back 15 years .. there’s a lot of education” ssc11-gm “one of our foundational clients pulled out .. thought they could do better themselves” ssc11-gm australasian journal of information systems volume 17 number 2 2012 16 task table 5 illustrates the range of tasks undertaken by sscs. common task related items that emerged from interviews are:  all the sscs provide transaction processing services  all the sscs provide, or seek to provide, multiple service lines  successful/moderate success sscs are more likely to manage a process end-to-end but not necessarily to provide it all  expert services are less likely to be provided by successful sscs table 5: tasks undertaken by ssc shared service operations transaction processing payroll cash management billing accounts payable accounts receivable procurement recruitment asset management banking cards management records management logistics internal mail services outbound sales expert services office services workers compensation fleet management insurance process improvement accounting (financial & management) property management reporting employee relations (performance management etc) training advisory (hr, finance, process, project management etc) employee services debt recovery general administration credit management analysis and reporting it operations / infrastructure end user support systems development and management disaster recovery projects data management single point of contact (internal/external) inbound/outbound sales it helpdesk australasian journal of information systems volume 17 number 2 2012 17 transaction processing was seen as the “genetic” core of the ssc “we [often] know we can improve it because its transactional, its process-driven and we’re good on process.” ssc2-dcs the more successful sscs sought to manage or own a process end-to-end “end to end will still mean the process goes in and out of the shared service provider. so what we’ve got to do is be very active with [customers] to say, well, we’re going to change our bed, but you’ve got to be active in changing your bed” ssc8-gm in addition they saw the benefits of undertaking all of an organisation’s transaction processing activities. “in terms of transaction management, it doesn't matter if you’re doing the payroll or paying bills or moving assets, the concept is make sure you’ve got low decision making .. basically, it’s a formula, it’s getting accuracy the first time, and so what we did is actually combined all the transaction areas and just called it transactional services” ssc7-gmss the most successful sscs however also tended not to provide so-called expert services preferring to “stick to their knitting” “chances are that you’re going to do okay there if you keep the operation simple. if you don’t add services that make things more complex and difficult to provide, such as value adding [expert] type services” ssc7-gmss it was recognised though that the categorisation of activities as being transaction based – or not – was not absolute but depended upon definition and perspective. “often difficult to define whether transaction.. you might say that recruitment is transactional .. certainly it would appear to have many transactional elements but hr people would all say no that is core and a professional service”ssc4-mss strategy the common strategy related themes that emerged from interviews were:  all the sscs seek to realise economies of scale  successful/ moderate success sscs seek to improve existing processes economies of scale through centralisation, the removal of duplication and being better positioned to secure funds to invest in the latest technology were the most common reasons given for establishing sscs. associated with this was a focus by the more successful sscs on process improvement with effort concentrated on those areas where the greatest potential was felt to lie. “the main thrust is economy of scale” ssc4-mss “realisation that services [are] provided to the business from different areas and if put under one area of accountability.. avoid duplication” ssc6-gm “the keeping up to date of systems, state of art, being aggressive, because that, in itself can save money” ssc8-pd “if you look at that as an end to end process, looking at where the exceptions are occurring, rather than just looking at what the exceptions are saying in the billing area or the metering area, looking at it as an end to end process. allocating resources in preventative areas, you can actually start decreasing those costs and increasing the australasian journal of information systems volume 17 number 2 2012 18 operational effectiveness”ssc11-gm structure the common structure related themes that emerged from interviews were:  for successful/ moderate success sscs use is compulsory  successful/ moderate success sscs have standardised processes  successful/ moderate success sscs are more likely to be held accountable for the delivery of performance improvements and assess their performance relative to external parties  successful sscs are more likely to charge users for the services provided for the more successful sscs the use of shared services was mandated – optional use was seen as leading to potential problems and a dilution of the benefits achievable. “yeah, that’s mandatory. there are a number of reasons for that. in our model what we really see is we will put structures in place to make our shared services as efficient as anybody else. so we get the pricing right, the service qualities right then you don’t go anywhere else. if we can’t get the price right and we can’t get the service quality right then we’ll shut the bloody thing down and you can go anywhere. you don’t want to weaken your economies of scale and if you start picking and choosing” ssc2-dcs however it was also agreed that even when mandated work needs to be done to demonstrate the benefits and avoid the emergence of work-arounds such as obtaining exemptions. “absolutely mandated .. but having said that, that is [x] years on but people – the reality is back on day one if you didn’t want to come here you didn’t .. it was a transition that i had to manage and from day one .. really be able to demonstrate that you could not go anywhere and get a better process” ssc10-gm standardisation to make sure that all parts of the business were following the same process was also seen as key. “it’s about standardisation and it’s about being able to add value to a process. so unless you can standardise it, you can’t get cost savings.”ssc8-gm being made accountable for delivering performance improvements – for example cost reductions – was seen as focusing the ssc. “they will say … there’s a global savings target that all areas have to meet and this is… let’s just say it’s one per cent or whatever is the operating base and it starts from next year.” ssc10-gm external benchmarking in order to assess performance and progress was seen as critical. “[members of a] benchmarking association.. so you get a lot of different machinations of what’s going on, how they’re structured, where they’re developing, what systems they’ve got, how they operate their back office, what efficiencies they are looking for internally, what drives them as a business, how does their company see the benefits of their shared service operation, where are they in the maturity chain?” ssc2-dcs the most successful sscs also had a charging mechanism in place that formed the basis for regulating user demand, behaviour and expectations. “work with the business to get them to improve.. and rather than hitting them with a stick, we’ve got a recharge model” ssc7-gmss australasian journal of information systems volume 17 number 2 2012 19 “but it became a [free good], you keep asking and asking, and we keep delivering... give them some accountability for the volume and the cost to them” ssc1-md management processes the common management related themes that emerged from interviews were:  successful/ moderate success sscs are more likely to focus on developing good relationships with their customers in the business often accompanied by successful sscs encouraging their staff to seek future opportunities with those customers  successful/ moderate success sscs are more likely to have an emphasis on measurement  successful/ moderate success sscs are more likely to have a transparent management style  successful sscs are more likely to a focus on leadership  limited success sscs are more likely to not fully understand the processes they provide or the level of work  limited success sscs are more likely to not have expertise in managing large groups of employees good relationships with the business were generally seen as critical. “it’s important for all my managers and myself to actually put a lot of time into relationship managing.. we try to be very customer focused.. we try to treat them like a customer, as opposed to an internal part of the organisation”ssc7-gmss losing employees to customers was also considered beneficial by the most successful sscs. “ i‘m happy when one of our staff gets a job elsewhere in the business – it means we gain an ally and advocate” ssc4-mss it was suggested that a focus on measurement enabled prior performance levels, and progress from them, to be established and demonstrated. “our problem is twofold. a, we nor the [business] knew what it used to cost before they gave it to us. b, at this stage of the game, we are still unable to determine what the cost potentially would have been if the [areas] had continued to share work” ssc8-gm but the need to be selective in measurement – knowing why something was being measured and how the results would be acted on – was also widely commented on. “an industry in itself and you just don’t want to create that overhead again.” ssc1-md transparency was seen as key – with ssc customers and staff. “we’re very transparent with decisions that we make. all our staff know why – we can make tough decisions. we’re very clear about what their role is and what they need to do and how it fits into the corporate plan” ssc9-gm a consistent theme amongst the most successful sscs was the importance of leadership “his strength is leadership so he instils in the four or five people who report to him a real culture of getting your people ripe.” ssc5-mfs “a good percentage of the time is staff and coaching and mentoring and leading my guys. i spend a fair bit of time with my direct reports in a coaching role, in a mentoring role.” ssc10-gm australasian journal of information systems volume 17 number 2 2012 20 one of the most necessary things to have in place before moving to shared services was seen to be a good understanding of how the impacted processes work (and thus where improvements can be made) and the level of work that can be expected – otherwise the ssc will find itself in catch-up mode from the outset. “my initial focus was clearly on understanding the work then understanding the expectations of the customer and delivering to that.” ssc10-gm it could also be problematic if the ssc did not have experience or expertise in managing large groups of employees “do not have the capabilities in terms of managing large groups of people in support areas” ssc6-mcs individual skills the common individual skill related themes that emerged from interviews were:  successful/ moderate success sscs are more likely to look for ways to increase the flexibility of their workforce  limited success sscs are more likely to have a workforce with a skill base that is too low for their needs a number of interviewees in limited success sscs complained that the lack of flexibility in staffing arrangements within their organisations made it hard to cystallise efficiency improvements into staff and cash savings “practically need to get union dispensation from the unions [to employ contractors] .. we do not have much flexibility” ssc11-em the more successful sscs however suggested that rather than sit back and complain you needed to push to the limits of what you could actually do. “a rough review has shown that even within all the employee frameworks that we have, there’s a lot of flexibility that we’re not using .. it just simply hasn’t been practised. the tradition is not there. no one has put their hand out and said, i’ll take the risk and start it. but we have.” ssc8gm the ability – depending upon how the ssc was structured – to develop generalists was often seen as one way of managing the peaks and troughs of demand. “provide us with the ability to multi-skill a bit more and actually we tend to get obviously some bits of the shared services are busy at some times … and we can tend to drag some people from the other areas” ssc7-gmss a number of the sscs though appeared to be starting from a low capability base which was going to take time to overcome “what we end up with is having people placed in the square peg in the round hole syndrome .. what they’ve done is they’ve dumped them together, not trained them in relation to customer service and what they’re supposed to do. so i’ve got a whole lot of people that haven’t been trained” ssc3gm information technology the common it related themes that emerged from interviews were:  it is a key foundation of sscs australasian journal of information systems volume 17 number 2 2012 21  sscs use a single, standard it platform it, and especially an enterprise resource planning system (erp), was generally seen as underpinning shared services “[our erp] is fundamental, we believe, to our shared services and has made the transition to shared services much easier than it otherwise would have been if we didn’t have a very good [erp] platform to help us with that transition to a shared services.. because the modules are integrated.. process and applications feed off the same data” ssc5-gess “we automated – we targeted particular companies that we dealt with that were high transaction. and, we automated that transaction process, the backwards and forwards. the order, the receipt, the invoice, the payment right. so we spent a bit of money on edi over the years and we got four of five top customers on and that knocked half of our transactions over night” ssc10-gm as such one of the fundamentals was seen to be ensuring everyone was using the same core platform. “now a myriad of systems… so if you’re a poor shared service provider sort of operative trying to say do payroll or accounts payable .. you can’t easily move from servicing [a] to servicing [b], because the systems are different.” ssc8-gm environmental conditions the common environment related themes that emerged from interviews were:  successful/ moderate success sscs are more likely to operate in organisations that see shared services as an evolutionary initiative rather than one that will be fully formed and deliver results immediately  successful sscs are less likely to be within organisations where there is a major emphasis on cutting costs  successful sscs are more likely to have a standard operating model interviewees in successful/developing sscs widely commented that it was important for the business to realise that immediate benefits and success would not be forthcoming – indeed that the introduction of shared services could be costly initially. the introduction of shared services was seen as an evolutionary process with the focus and approach adopted likely to change over time as the business became more knowledgeable and experienced regarding what was possible and achievable. “often when you’re doing this there’s a significant upfront investment cost. you know, that won’t be paid back till about two or three years because you will have to change, radically change systems.”ssc3-fd in a number of organisations the scope of shared services is somewhat fluid with new activities coming in and some being passed back to the business – often if it is discovered that it can not effectively be decoupled. with the latter though instances were cited where even when activities were ultimately passed back the process improved capabilities of the ssc yielded benefits during the “in and out cycle” “we’ve given a couple of bits and pieces back, when actually something doesn't make sense.. we just believe that in fact it’s probably something that actually makes more sense to go back to the business. but some of that’s only been able to be done, because in fact we’ve improved or automated something australasian journal of information systems volume 17 number 2 2012 22 .. we’ve probably taken something that was a bit of a mess, tidied it up and then actually may be automated bits of it, so it didn’t require a lot of hands on, and then actually been hey, you can go back and do this” ssc7-gmss it was also suggested that while cost savings will likely be realised by shared services the actual focus should be on improving service delivery. “the idea of why you set up shared services and well some places say it is to reduce costs. but if you set up with that in mind you’ll fail; if it’s set up as a strategic decision to implement shared services one of the outcomes will be a reduction in costs and an improvement in service” ssc2dcs “it wasn’t received well by the organisation because what it meant was people were losing their jobs .. was looked on as a cost saving strategy for the organisation to downsize its administrative staff” ssc3-gm a differentiator of successful sscs was that they had a standard operating model across the business with little variation across products or trading partners “operate in multiple different markets .. work in different ways.. effectively separate businesses”ssc11-gm “we do not have the complexity or variety of some of the others .. makes life easier”ssc4-mss “we have a pretty standard business operating model across everything we do”ssc2-gmcs “out of the city it is a completely different ball game”ssc6-mcs “the different areas have very different needs and ways of working which we have to accommodate”ssc1-md history the common history related themes that emerged from interviews were:  the majority of sscs operate in organisations where there is an established emphasis on functions concentrating on their core business activities  successful sscs are more likely to operate in organisations where a strong centre has retained significant control over operations it was felt to be useful if the organisation had emphasised the importance of business units focusing on their core activities with managing a back office positioned as a distraction. “what there was over the last five years, there was a process by which they started focusing on what the core competencies were ..” ssc6-gm “in business the pressure is always on other areas to do what they’re meant to be doing, so if you’re meant to be selling, well, increase revenue, don’t worry about all your admin bits and pieces” ssc9-gm an established ability, and willingness, of the corporate centre to impose its desire on business areas – in this case to establish shared services – was also stressed. “historically the culture of the organisation is control, they like to have control, they’re very autonomous in decision making and so they like things not to succeed … they’ve always been told that if you own [an area] you’re autonomous, you make the decisions, you’ve got the budget, you’ve got the resources you can manage it.”ssc3-gm australasian journal of information systems volume 17 number 2 2012 23 “a lot of historic reasons that parts of the business never really were engaged with shared services .. independent units .. so it is a company wide philosophy that shared services is caught up in .. i guess you always have that sort of company history” ssc4-mss organisational resources the common resource related themes that emerged from interviews were:  capital investment in sscs is required and generally perceived to be available  successful sscs are more likely to have senior management commitment to shared services  successful/ moderate success sscs are more likely to have the required it to be in situ before the ssc is established the establishment of an ssc requires the availability and commitment of significant investment funds. “highly dominated by the it infrastructure, and we need a number between about six million to eight or nine million .. and we got that” ssc1-md “funding for the redundancies” ssc6-gm “got allocated $xm to start and had to go back to get more a couple of times” ssc8-gm support from the top of the organisation was also seen as necessary to the effective establishment and operation of a ssc due to the degree of organisational change required – but it then needed to be buttressed by performance. “process is very strongly supported by the ceo … there’s no points to be gained from messing it up” ssc2-dcs “you couldn't survive just on the fact that well the boss says i have to be here. you can rely on that for so long, but eventually the noise would be so loud it wouldn't survive”ssc7-gms a number of interviewees suggested that the greatest challenge was moving areas to a standardised erp platform – and that rolling out shared services was relatively simple if this was in place. “we didn’t change a whole lot .. we reshuffled some of the chairs and a few chairs could then go .. the introduction of [the erp] beforehand that was the big culture change” ssc5-mfs discussion the propositions are perhaps best considered together since they are in many ways the opposite sides of the same coin. proposition 1: there exists at least one alignment of attributes across the dimensions identified that is associated with successful shared services proposition 2: deviations from that alignment will be associated with less successful shared services the case studies suggest the answer is more complex than anticipated a priori. there appear to be three distinct blocks of attributes –those that all sscs exhibit, those that only successful and moderate success ones have and those that only the most successful sscs possess. it is also clear that for two of the blocks there are some attributes that all of the sscs in the relevant categories do or do not possess. there are others though that not all of the relevant sscs possess or that others australasian journal of information systems volume 17 number 2 2012 24 from a different category also possess. this can be interpreted in two possible ways. either not all of the attributes currently identified are necessary to determine which category of success a ssc sits in. alternatively within each category there are varying degrees of success such that the ssc that can “tick off” the most attributes will be the most successful. related to and leading on from this is the case of ssc9. in absolute terms ssc9 exhibits a comparable number of attributes to the successful sscs – and in the cases of ssc5 and ssc10 more. yet interviewees self-assessed the ssc to be a developing success. the data collected as part of the research does not allow for it to be determined why this is the case. based upon the above discussion though it could be that one or more of the attributes that the successful sscs display that ssc9 does not is critical to success. if this is the case it may relate to standardisation at the organisational level, for example with regard to policies and practices. all of the successful sscs either reported the presence of a strong centre and/or a standard operating model which could be interpreted as reflecting such standardisation. ssc9 possessed neither. alternatively – and somewhat related to the possibility of nuances in the degree of success existing – is the possibility that perceptions as to what constitutes success may vary between sscs. in this scenario ssc9 could have a more demanding definition of success than ssc10 for example. returning to the specifics of the propositions it is suggested that there is one alignment of attributes that is associated with successful sscs. with regard to deviations and their association with reductions in success it appears though that there are two distinct groupings – sscs with moderate success and those with limited success. below is an attempt to summarise the characteristics required for a successful ssc in narrative form: the ssc should be given responsibility for demonstrably reducing costs through economies of scale and process improvement of multiple transaction oriented tasks across multiple organisational functions. an organisation wide it platform is key to realising those savings and should be put in place before the transition to shared services. use of the ssc should be compulsory and charged for. the organisation should have a standard modus operandi, recognise the importance of focusing on core competencies and have a strong centre that is committed to shared services and willing to invest in the ssc for the long term. ssc management should understand the requirements and volume of each task, standardise them and develop a human resource base with the variety of skills and flexibility required to meet demand. conclusions and future research the research has identified a series of attributes across the dimensions of task, environment, history, resources, strategy, structure, management, technology and individual skills that differentiate between successful, moderately successful and limited success sscs. it is perhaps interesting that while the gestalt literature – on which the conceptualisation of alignment utilised here is largely based – suggests that multiple different, but internally coherent, combinations could be associated with success only one has been identified in this instance. if there is indeed only one combination that can deliver success this may mean that the appropriateness of shared services is much more limited than anticipated and organisations should conduct a thorough analysis before adopting them. the research contributes to the academic literature in three principal ways. first, it helps to establish that there is merit in extending the boundaries of is research to incorporate shared services given the central role played by technology. further while having appropriate technology is seen as crucial by all of the sscs studied it is also clearly not sufficient by itself for success. this supports the view that is research needs to take account of the broader milieu if it is to contribute fully to improving australasian journal of information systems volume 17 number 2 2012 25 business effectiveness. second, the research extends use of the alignment concept from the organisational to the functional level and demonstrates the usefulness of combining consideration of the task, organisation and environment. third, and most specifically, the research provides a framework that identifies the requirements for successful shared services. from a practical perspective the research, and in particular table 3, provides a checklist that can be used to determine whether a planned ssc is likely to be successful or not. the structure and detail of that checklist provides guidance beyond a basic consideration of whether an attribute is present or not. it can help organisations determine whether their current situation is likely to be readily changeable or not. some attributes, such as the existence of a strong centre, are likely to be difficult to alter. others though, such as having the supporting it in situ beforehand, may be easier to address. in this case it may simply mean that instead of implementing the necessary it in tandem with the introduction of shared services it is done first. furthermore the specific assessments provided for the 11 sscs studied – and differences within and between success categories provides additional insight regarding the relative importance of each dimensions and attribute. for example there is some evidence to suggest both that technology is fundamental to shared services and that deficiencies in management, structure and skills are the major shortcomings of limited success sscs. with regard to future research two major avenues are suggested. firstly the concept of success needs to be examined in more detail. here perceptions, generally by senior management directly associated with the ssc, were relied upon. it would be valuable to consider different perspectives – for example of users or customers of the ssc. it would also be useful to determine if more objective measures of success can be identified since ssc9 raises the possibility that what is seen as success can vary between organisations. secondly, given that the sscs within a success grouping do not always score the same on every attribute, research that examines the attributes in more detail would be useful to provide organisations with a greater understanding of which are the most important. references andrews, k. r. the concept of corporate strategy irwin, homewood, 1971. benbasat, i., goldstein, d. k., and mead, m. " the case study research strategy in studies of information systems," mis quarterly (11:3) 1987, pp 369-386. bergeron, b. essentials of shared services wiley, hoboken, 2003. borman, m. "applying multiple perspectives to the bpo decision: a case study of call centres in australia," journal of information technology (21:2) 2006, pp 99-115. cecil, r. "shared services: moving beyond success," strategic finance (81:10) 2000, pp 64-68. chan, y. e., huff, s. l., barclay, d. w., and copeland, d. g. "business strategic orientation, information systems strategic orientation, and strategic alignment," information systems research (8:2) 1997, pp 125-150. chan, y. e., and reich, b. h. "it alignment: what have we learned?," journal of information technology (22:4) 2007, pp 297-315. chandler, a. d. strategy and structure: chapters in the history of the american industrial enterprise mit press, cambridge, 1962. child, j. "managerial and organization factors associated with company performance – part ii, a contingency analysis," journal of management studies (12:1) 1975, pp 12-27. davis, l. e., canter, r. r., and hoffman, j. "current job design criteria," journal of industrial engineering (6:2) 1955, pp 5–8, 21–23. australasian journal of information systems volume 17 number 2 2012 26 delone, w., and mclean, e. "information systems success: the quest for the dependent variable," information systems research (3:1) 1992, pp 60-95. dibbern, j., goles, t., hirschheim, r., and jayatilaka, b. "information systems outsourcing: a survey and analysis of the literature," database (35:4) 2004, pp 6-102. domberger, s., fernandez, p., and fiebeg, d. "modelling the price, performance and contract characteristics of it outsourcing," journal of information technology (15:2) 2000, pp 107118. drazin, r., and van de ven, a. h. "alternative forms of fit in contingency theory," administrative science quarterly (30:4) 1985, pp 514-539. dunphy, d., and griffiths, a. the sustainable corporation: organizational renewal in australia allen and unwin, sydney, 1998. etzioni, a. "organizational control and structure," in: handbook of organizations, j.g. march (ed.), rand mcnally, chicago, 1965, pp. 650-677. farbey, b., land, f., and targett, d. "moving is evaluation forward: learning themes and research issues," journal of strategic information systems (8:2) 1999, pp 189-207. faulkner, d. strategy: critical perspectives on business and management routledge, oxford, 2002. firestone, w. a., and herriott, r. e. "the formalization of qualitative research: an adaptation of “soft” science to the policy world," in: multisite qualitative policy research in education: a study of recent federal experience, w.a. firestone and r.e. herriott (eds.), authors, report no. nie-400-80-0019 concord, 1982. galbraith, j. r. designing organizations: an executive guide to strategy, structure and process jossey-bass, san francisco, 2002. hagel iii, j., and brown, j. s. "your next it strategy," harvard business review (79:9) 2001, pp 105–113. hall, r. h. "intraorganizational structural variation: application of the bureaucratic model," administrative science quarterly (7:3) 1962, pp 295-308. hannan, m., and freeman, j. "the population ecology of organizations," american journal of sociology (82:5) 1977, pp 929-964. hathaway, r. s. "assumptions underlying quantitative and qualitative research: implications for institutional research," research in higher education (36:5) 1995, pp 535-562. henderson, j., and venkatraman, n. "strategic alignment: leveraging information technology for transforming organizations," ibm systems journal (32:1) 1993, pp 4-16. herriott, r. e., and firestone, w. a. "multisite qualitative policy research: optimizing description and generalizability," educational researcher (12:2) 1983, pp 14-19. hooper, v., huff, s., and thirkell, p. "is-marketing alignment: its impact on marketing performance and on business performance," european conference on information systems, st gallen, switzerland, 2007. hrebiniak, l. g. "the organization and environment research program: overview and critique," in: perspectives on organization and behavior, a.h. van de ven and w.f. joyce (eds.), wiley, new york, 1981, pp. 338-345. janssen, m., and joha, a. "motives for establishing shared service centres in public administrations," international journal of information management (26:2) 2006, pp 102115. katz, d., and kahn, r. the social psychology of organizations wiley, new york, 1966. australasian journal of information systems volume 17 number 2 2012 27 kern, t. "the gestalt of an information technology outsourcing relationship: an exploratory analysis," international conference on information systems, atlanta, georgia, 1997. kerr, d. v., and houghton, l. "just in time or just in case: a case study on the impact of context in erp implementations," australasian journal of information systems (16:2) 2010. kerr, p. "wa junks pay system, oracle," in: australian financial review, fairfax, sydney, 2011, p. 9. kreklow, s. r., and kinney, a. s. "the effective organization of administration functions through shared services," government finance review (23:3) 2007, pp 61-62. leavitt, h. j. "applied organizational change in industry: structural, technological and humanistic approaches," in: handbook of organizations, m. j.g. (ed.), rand mcnally, chicago, 1965, pp. 1144-1170. lee, j., miranda, s. m., and y, k. "it outsourcing strategies: universalistic, contingency and configurational explanations of success," information systems research (15:2) 2004, pp 110-131. leonardi, p. m., and bailey, d. e. "transformation technologies and the creation of new work practices: making implicit knowledge explicit in task-based offshoring," mis quarterly (32:2) 2008, pp 411-436. longwood, j., and harris, r. g. "leverage business process outsourcing lessons to build a successful shared business service organisation," g00144283, gartner, stamford. luftman, j., lewis, p. r., and s.h., o. "transforming the enterprise: the alignment of business and information technology strategies," ibm systems journal (32:1) 1993, pp 198-221. macgregor, r. c., hyland, p. n., and harvie, c. "do organisational characteristics explain the differences between drivers of ict adoption in rural and urban general practices in australia," australasian journal of information systems (16:1) 2009. march, j. g. (ed.) handbook of organizations rand mcnally, chicago, 1965. miles, m. b., and huberman, a. m. qualitative data analysis: an expanded sourcebook, (2nd ed.) sage, thousand oaks, 1994. miller, d. "evolution and revolution: a quantum view of structural change in organisations," journal of management studies (19:2) 1982, pp 131-151. mintzberg, h. the structuring of organizations prentice-hall, upper saddle river, 1998. mintzberg, h., and lampel, j. "reflecting on the strategy process," sloan management review (40:3) 1999. nadler, d. a., and tushman, m. l. competing by design oxford university press, new york, 1997. pollalis, y. a. "patterns of co-alignment in information intensive organizations: business performance through integration strategies," international journal of information management (23:6) 2003, pp 469-492. porter, m. competitive advantage: creating and sustaining superior performance free press, new york, 1985. porter, m. "towards a dynamic theory of strategy," strategic management journal (12:winter) 1991, pp 96-117. quinn, b., cooke, r., and kris, a. shared services: mining for corporate gold prentice-hall, harlow, 2000. rayner, r. "reuters’ finance and accounting shared services drive global efficiency," gartner, stamford. australasian journal of information systems volume 17 number 2 2012 28 roseman, m., recker, j., indulska, m., and green, p. "business process modelling – a comparative analysis," journal of the association for information systems (10:4) 2009, pp 333-363. schulman, d. s., dunleavy, j. r., harmer, m. j., and lusk, j. s. "shared services: adding value to the business units," wiley, new york, 1999. schulz, v., and brenner, w. "characteristics of shared service centers," transforming government: people, process and policy (4:3) 2010, pp 210–219. scott-morton, m. s. the corporation of the 1990s oxford university press, new york, 1991. seddon, p. b., staples, s., patnayakuni, r., and bowtell, m. "dimensions of information systems success," communications of the ais (2:20) 1999, pp 1-32. shah, b. "shared services: is it for you," industrial management (sept-oct:4-8) 1998. stockdale, r., and standing , c. "an interpretive approach to evaluating information systems: a content, context, process framework," european journal of operational research (173:3) 2006, pp 1090-1102. strauss, a., and corbin, j. basics of qualitative research: grounded theory procedures and techniques sage, newbury park, 1990. taylor, f. w. principles of scientific management harper and row, new york, 1911. ulbrich, f. "improving shared service implementation: adopting lessons from the bpr movement," business process management journal (12:2) 2006, pp 191-205. van de ven, a. h., and delbecq, a. l. "a task contingent model of work-unit structure," administrative science quarterly (19:2) 1974, pp 183-197. venkatraman, m. "the concept of fit in strategy research: toward verbal and statistical correspondence," academy of management review (14:3) 1989, pp 423-444. venkatraman, n. "strategic orientation of business enterprises: the construct and its measurement," university of pittsburgh, pittsburgh, 1985. walsham, g. interpreting information systems in organizations wiley, chichester, 1993. weber, m. economy and society university of california press, berkeley, 1978. weill, p., broadbent, m., and butler, c. "exploring how firms view it infrastructure," melbourne business school. white, r. e. "generic business strategies, organizational context and performance," strategic management journal (7:3) 1986, pp 217-231. willcocks, l. p., lacity, m., and cullen, s. "outsourcing: fifteen years of learning," in: the oxford handbook of information and communication technologies, r. mansell, c. averou, d. quah and r. silverstone (eds.), oxford university press, oxford, 2007. yin, r. changing urban bureaucracies lexington books, lexington, 1979. yin, r. "the case study crisis: some answers," administrative science quarterly (26) 1981, pp 58-65. australasian journal of information systems volume 17 number 2 2012 29 appendix 1: shared services case study protocol service follow ups regarding effectiveness what activities are provided by the ssc what characteristics make an activity appropriate for shared services shared services centre design follow ups regarding effectiveness strategy what are the principal goals of the ssc how are these goals to be realised – in principal enablers structure how will work be coordinated to achieve the desired results – specific practices what performance measures/checks are in place management processes what are the principal management foci (direction and control) of the ssc what are the critical skills managers require individual skills what types (and level) of skills are required of employees were such skilled employees already available within the organisation information technology [it] what it is used by the ssc how important to the ssc is it organisational context follow ups regarding effectiveness [and changes since establishment] organisational resources what assets (employees, technology, capital and information) were important to the establishment of the ssc were all these assets sufficiently available history how were activities organised before the ssc did this arrangement impact on the ssc environmental conditions was anything occurring in the broader organisation that impacted the ssc (eg initiatives, leadership changes, regulation, union activity, competition) shared services success how successful is the ssc on what do you base that assessment australasian journal of information systems volume 17 number 2 2012 30 appendix 2: summary data display table performance perceptions task strategy structure management processes ssc1  moderate  broad range of services  transaction and expert  partial process  scale  process improvement  effective use of capital  independent company structure  journey towards standard  compulsory  price charging  benchmarking in progress  leadership  service delivery framework  measurement/analytical focus  client education  manage suppliers  transparency ssc2  successful  broad range of services  transaction focus  end to end focus but not all within shared services  scale  process improvement  automate  standard  compulsory  corporate keeps savings  user pays (cost recovery)  benchmark  demonstrate value for funding  business benefits focused  leadership  business relationships  measurement emphasis  different skills at different times ssc3  limited  broad range of services  transaction and expert  partial process  orphan activities  scale – efficiencies with simple process focus  level of service (reduce)  custom  compulsory  policies and procedures focus  education ssc4  successful  broad range of services  transaction  end-to-end process  scale  process improvement (and consistency)  core business focus  automation  measurement  standard  compulsory  price charging  benchmark  put under pressure (cost)  business relationships  measurement (appropriate)  maintaining alignment across services  transparency  train staff for the business ssc5  successful  broad range of services  transaction  end-to-end process  expert focus  scale  process improvement  standard  compulsory  benchmark  performance assessed  leadership  manage transactions/performance  transparency  business relationships/partner  measurement emphasis  train staff for the business ssc6  limited  broad range of services  transaction and expert  partial process  scale  enable core business focus  custom  optional use – cherry pick services  price charging  education  lack large scale hr management expertise  lack understanding of process & work levels australasian journal of information systems volume 17 number 2 2012 31 ssc7  successful  broad range of services  transaction  own the process  scale  process improvement  standard template for expansion  control  risk management  standard  compulsory  corporate keeps savings – but user incentive  user pays (cost recovery)  benchmark  demonstrate value for funding  business relationships  transparency  measurement emphasis  train staff for the business ssc8  moderate  broad range of services  transaction  expert  end to end focus but not all within shared services  scale  process improvement  journey towards standard  compulsory  intro of accountability  performance assessed  business relationships  customer rather than administrative orientation ssc9  moderate  broad range of services  transaction  end to end focus but not all within shared services  scale  process improvement  risk management  standard – where appropriate  “optional” – but compulsory  benchmark  put under pressure (cost)  transparency  business relationships  measurement emphasis  key skill lacking is large scale hr management  train staff for the business ssc10  successful  broad range of services  transaction focus  end-to-end process  own the process  centralisation scale  process improvement  reduce labour costs  standard  compulsory (over time)  corporate overhead  benchmark  put under pressure (cost)  business relationships – sell contribution  measurement determines focus  leadership (mentoring)  transparency ssc11  limited  narrow range of services  transaction and expert  end-to-end process  operational changes increase complexity  scale  synergies – seamless /scope/accountability  automation  measurement – right first time (exception focus)  custom  “compulsory” – but optional  cost recovery central  business relationship (but mixed)  operational rather than forward thinking focus  lack large scale hr management expertise  reluctance to confront hr issues australasian journal of information systems volume 17 number 2 2012 32 part ii data display summary of the shared service centre dimensions individual skills information technology environmental conditions history organisational resources ssc1  low skill base  flexible – can offload excess staff  process mapping  workforce planning – multiskill / systems training  towards standard  enabler  cost cutting focus  structural change  varied client needs  mandated  seed funded  self fund beyond ssc2  low skill base  manage to enable flexibility  share ideas – communicate  self service  standard  enabler  cost cutting focus  impetus  standard operating model  project success  strong centre  core business focus  business case  largely in situ beforehand  management focus ssc3  low skill base  inflexible  process mapping  need to upskill / build expertise  standard  enabler  modify system to processes  cost cutting focus  expect immediate results  autonomy  big bang approach  short term focus  allocated limited  management support  nobody wants to work in ss ssc4  manage to enable flexibility  workforce planning and career paths (internal)  standard  enabler – but can restrict  primarily outsourced  growth – positive atmosphere  simple business (not complex)  strong centre  greenfield  core business focus  business case  put under pressure (cost)  largely in situ beforehand  management focus ssc5  self service  solution  standard  standard operating model  autonomy  allocated sufficient  largely in situ beforehand  management focus ssc6  low skill base  inflexible  varied  enabler  cost cutting focus  structural change  expect immediate results  outside the core  autonomy  core business focus  limited  allocated sufficient australasian journal of information systems volume 17 number 2 2012 33 ssc7  green field – select skills required  multiskill flexibility  solution – but not it for it’s sake  standard  expansion (takeover)  growth – positive atmosphere  lack of interest  one way of doing things  strong centre  core business focus  business case  generate working capital  management focus ssc8  shortage  innovation for flexibility  work group  solution  standard  mandated  structural change  cost cutting focus  autonomy  core business focus  allocated sufficient  largely in situ beforehand ssc9  process mapping and measurement  well defined roles  multiskill flexibility  enabler  standard  have to accommodate different ways of working  autonomy  core business focus  business case  largely in situ beforehand  management interest ssc10  manage to enable flexibility  standard  enabler/solution – workflow/automation  cost cutting focus  politics  strong centre  core business focus  limited capital expenditure  allocation but business case possible  management focus ssc11  inflexible  standard  solution  modify system to processes  expansion (organic)  cost cutting focus  structural change  expect immediate results  soft restructure  autonomy  core business focus  constrained – allocated  largely in situ beforehand editorial this is the second issue of the australian journal of information systems (ajis). as editor i am pleased to report that over the past six months there has been an ongoing interest in the journal, both from authors and subscribers in the ever widening area of information systems. in this issue we have contributions from six australian universities, together with articles from the usa and greece. our first paper 'using annotated conceptual models to derive information systems implementations' by a. berglas, university of queensland explores an important issue in the development of information systems, the time consuming process of producing quality information systems from conceptual models. it demonstrates that by annotating the conceptual model with additional parameters, it is possible to automate the implementation of quality applications from the conceptual model. the next paper 'what happens when designers don't play by the rules: towards a model of opportunistic behaviour in design' by a. khushalani, centre for systems methodologies, swinburne university of technology, r. smith, department of information systems, monash university, & s. howard, centre for systems methodologies, swinburne university of technology is concerned with designers needing to continually adapt to the specific problem and the environment in which the problem exists, termed 'opportunism'. it develops a model for opportunistic behaviour and discusses directions for further work and research. graham pervan, school of information systems, curtin university of technology contributes a paper entitled 'information systems management: an australian view of the key issues'. this describes a longitudinal study of information systems managers in australia's largest 300 organisations. the next paper is 'synthesising expert systems and operations research techniques: system classification and review' by h. rao, school of management suny, buffalo ny, b. lingaraj, school of business and management sciences, university of purdue, & m. rao, planning and development, fallen community health plan. this paper examines the synthesis of expert systems and operations research techniques and classifies various expert systems which utilise the two for management problem solving. the next paper is ' business process redesign using edi: the bhp steel experience' by paula swatman, department of information management and marketing, university of western australia. this paper examines the re-engineering of business processes by using edi as the standard infrastructure in bhp. a. tsalgatidou, d.gouscos, & c. halatsis, from the department of informatics, university of athens, greece contribute a paper entitled 'towards the development of effective and flexible information systems'. this paper presents a new approach to requirement analysis through an environment called venus. the final paper in this issue is 'using artificial neural networks and function points to estimate 4gl software development effort' by g. wittig & g finnic, school of information technology, bond university. this paper examines the use of back-propagation artificial neural networks in the improvement of accuracy in the forecasting of developmental effort once again i would like to pass on my thanks to the authors and consulting editors for the high quality of material in this issue of the journal. i would like to single out mr ayman el-ardenli, without whose help this journal would not have been produced. r.macgregor editor an ontological evaluation of jackson's system development model fiona rohde department of commerce the university of queensland, 4072. australia abstract within the discipline of information systems, numerous methodologies exist to aid the analyst and designer develop a well-designed system that meets the specifications put forward by the user. these methodologies tend to fall into two categories: (a) reality-driven, or (b) informationsystems driven. wand and weber (1989), have mapped a set of ontologically-based constructs onto two widely-used analysis and design techniques being data flow diagrams (an informationsystems driven methodology) and entity relationship modelling (a reality-driven methodology). in this paper, wand and weber's work is extended to investigate the systems analysis and design methodology created by michael jackson, namely jackson system development (jsd). jsd was selected as it combines the reality-driven and information-system driven approaches by modelling what is happening in reality through the use of constructs related to the process to be modelled (usually computerised processes). the oncological analysis of jsd undertaken during this paper is used to evaluate the methodology's strengths and weaknesses as a modelling technique and ultimately to investigate its usefulness as a systems analysis and design methodology. introduction an important goal of any system developer should be to produce a system that is well designed. according to bergland (1981), a well-designed system implies a good system structure. to produce a system that has good structure, methods need to be available to help the designer or analyst achieve their goal. over the years, a large number of methodologies have become available to assist during the process of analysis, design, and implementation of an information system, olle (1982). during the same period, numerous studies including olle (1982) and floyd (1986), have been conducted in an attempt to gain an understanding of and to aid in an evaluation of the methodologies. this paper will review a framework in which an evaluation can take place. it will then use this framework to evaluate the system analysis and design methodology, jackson system development (hereafter referred to as jsd), developed by jackson in 1983. to be able to determine the modelling power of jsd, some criteria are needed against which the derived system structure can be evaluated. wand and weber (1989, 1990) have proposed an ontological model for an information system. they "formally define and articulate a number of fundamental information systems concepts", wand and weber (1993, p 218). in another work, wand and weber (1989) used their ontological framework to examine the 77 modelling power of two systems analysis and design methodologies: data flow diagrams and entity-relationship modelling. to perform an evaluation of different methodologies, fundamental information systems concepts underlying the methodologies need to be clarified and defined precisely. by clarifying these underlying concepts, a theoretical base can be developed upon which the different methodologies can be evaluated. the model proposed by wand and weber, describes and defines some of the fundamental information concepts. it also develops a theoretical basis that allows the different features of the methodologies to be clarified and evaluated. for these two reasons the wand and weber model has been selected as the basis for evaluating jsd. the format of this paper is as follows. section two reviews the ontological model of an information system proposed by wand and weber, while section three contains a brief review of jsd. section four contains an ontological evaluation of jsd, together with an integration of the methodology and model. section five summarises the paper and discusses any limitations together with areas for further investigation. wand and weber model as stated previously, it would be advantageous to compare different systems analysis and design models. to make such a comparison, models that are both information systems and reality driven need to be compared. additionally, each methodology's capabilities to model both the static and dynamic aspects of the modelled domain. by utilising such a basis for comparison, the modelling power of very different systems analysis and design methodologies can be evaluated though the use of the same underlying constructs. ontology is one branch of philosophy that deals with both, modelling the existence of things, as well as the structure and behaviour of the world. the wand and weber model is based on the "ontological formalism proposed by bunge (1977, 1979) having been extended and modified for the purpose of modelling information systems" wand and weber (1989, p 84). this section briefly describes a number of ontological (real-world) constructs that form the foundation of the work carried out by wand and weber in this area. as jsd is to be evaluated according to this ontology, it is important to provide a brief description of the constructs the aforementioned authors have derived from bunge's ontology. the terms and concepts defined here are required in section four of this article to examine the ontological aspects of jsd. the following definitions are adapted from those found in wand and weber (1989, 1990, 1993). definition: state variable: the state variable, vj, of a thing y to be modelled is the system identity for the property used to describe that thing in the real world ("thing" is the atomic element within the model and "the world is made of things that have properties" wand and weber (1989 p 84)). definition: possible state space: the cartesian product of the sets of all possible values of each state variable is the possible state space. 78 definition: laws: restrictions (referred to an laws) may be placed on either the possible values of the properties of the thing to be modelled or on the relations existing between two or more properties. definition: lawful state space: the lawful state space of a thing to be modelled is the subset of the possible state space where the combinations of state variable values are deemed lawful. this state space, by definition, must be equal to or smaller than the possible state space. definition: event: an event is a change of state. the notion of an event is extended later to include a definition for both external and internal events. information systems are usually implemented using data (implementations of states and events) and processes (implementations of laws) definition: transformations: given the lawful state space for a thing to be modelled, the set of transformations from the lawful state space onto itself are the transformations that are given as lawful in the system. an event, as defined above, is considered to be a lawful event if the two states s and s' are both within the lawful state space and the transformation is lawful. definition: coupled: two things to be modelled, x and y, are coupled if and only if x acts on y or y acts on x; i.e., x changes y's history of values or y changes x's history of values. definition: system: a system is a set of things, where each thing is coupled to at least one other thing in the set. furthermore, the set cannot be partitioned into smaller sets without losing one or more of the couplings. that is, the set of things can be represented by a connected graph. definition: subsystem: to be able to arrive at this definition, three additional concepts need to be defined: the composition, the environment, and the structure of a system. to arrive at a definition for each of these concepts, assume we have a system s. the composition of s at time t is the set of things modelled in s at time t. the environment of s at time t is the set of things that are not components of s at time t but act upon or are acted upon by components of s at time t. that is, those things that are not components of s but are coupled to the components of s. the structure of s at time t is the set of couplings among the components of s and the set of couplings among the things in the environment and the components of s at time t. now we can arrive at a suitable definition of a subsystem, x is a subsystem of s (denoted x < s) if and only if 1. x is a system in its own right at time t, and 2. the components of x at time t are a subset of the components of s at time t, and 3. the structure of x at time t is a subset of the structure of s at time t, and 4. the environment of x at time t is a subset of the combination of the environment of s at time t and those components of s at time t that are not components of x at time t. 79 definition: stable state: given a state and a set of transformations, the state is said to be stable with respect to the set of transformations if and only if the state does not alter unless forced to by an external event. definition: unstable state: given a state and a set of transformations, the state is said to be unstable with respect to the set of transformations if and only if the state will be forced to change because of transformations within the system, i.e., internal events. definition: external event: an event is an external event if and only if a change of state occurs because of an interaction with something within the environment. definition: internal event: an event is an internal event if and only if the event arises because of lawful transformations. for a system to act as an information system there are four requirements that are necessary and sufficient for an information system to provide full knowledge of the states that the real system traverses. these four requirements are (a) states of the real system can be mapped to states of the information system, (b) there is a mirroring of external events in the real system to external events in the information system, (c) the information system must interact with its environment, therefore the things in the system must interact with things in the information system and (d) events in the real system will trigger their counterpart in the information system in the same order of occurrence. definition: decomposition: wand and weber (1990) define decomposition in terms of the elements a system, that is, the components, the environment, the structure, and the subsystems. a decomposition is a set of subsystems of a system where (a) every component of the system is included in at least one subsystem; (b) the difference between the union of the environments of the subsystems and the components of the system is the environment of the system; (c) each element in the structure of the system is included in at least one subsystem of the system. as stated previously, the ontological constructs defined above, will be used during section four of this paper. the constructs are used to examine to ontological aspects of jackson's system development methodology. jackson's design model there are many different systems analysis and design methodologies. this section describes the system development methodology (jsd) that evolved from work carried out by michael jackson, jackson (1983). jsd was selected for two reasons. first, the methodology, while not having widespread use in australia, does have numerous supporters throughout europe. second, jsd is an interesting methodology in that it employs constructs related to the computerised processes as a means of modelling objects in reality. thus, the methodology can be viewed as a combination of the reality-driven methodologies that focus on constructs related to reality and information-driven methodologies that tend to focus on data and processes. 80 the methodology comprises six steps or stages. the first four are concerned predominantly with deriving a specification of the required system. the remaining two are concerned with the implementation of the above specification. the major aim of jsd is to ensure the final system is a true reflection of both the developer's and user's perceptions of the real world. the methodology starts by taking the current state of the real world and describing it. by insisting the developer first makes an abstract description of the real world, the methodology is trying to ensure the way in which the user views the real world is incorporated into the specification and eventually part of the system itself. the real worlds modelled in jsd are always worlds in which the time dimension is of central importance, jackson (1983, p 16), i.e. dynamic real worlds. when referring to such a world, not only does the analyst take into consideration the items in the real world and the actions they perform or suffer but also the order of the events and the properties and relationships existing among them. although there appears to be six independent steps to this methodology as the developer learns more about the system itself, some iterations may be necessary. since the methodology is not commonly employed throughout australia, each of the six steps will be briefly described in turn. entity action step the developer describes, in abstract terms, part of the real world in terms of entities and the actions in which they participate. general criteria are provided for identifying entities and actions. jackson (1983, p 40) entity structure step the ordering of an entities actions needs to be expressed. to represent entity structure and the time ordering of actions, jsd has introduced the diagrammatic notation known as structure diagrams. actions are applied to an entity as a sequence, as part of an either-or selection, or repetitively (iteration). initial model step thus far the developer has derived an "abstract description of the real world in terms of sequential processes", jackson (1983, p 44). this step converts the abstract description to a specification of the system by stating how the processes existing in the real world are connected to the processes in the model. the process connection is portrayed using a system specification diagram. additionally, the internal details of the model processes can be specified using structure text. the function step the developer connects newly-defined functions to the model processes already existing in the system specification diagram. the following should be documented as part of this phase: restate the informal requirements for each function in terms of the initial model; expand the system specification diagram by illustrating how each function can be connected to a process or may be built into them; and show the detail of each function using structure text. system timing step once a specification for a system has been produced the developer can consider the question of timing and how different time lags within processes may affect the output, determining which are acceptable and which are not. the system timing, although usually only informally documented, is one of the major sources of input to the final phase of implementation. 81 implementation step as soon as the specification derived is considered complete the developer then draws upon the jackson program design techniques to eventually derive the remaining program text. jsd is an enhancement of jackson's structured programming, jackson (1975), by taking into account the needs of not only the programmers but also the system developers. jsd's one main advantage over many other methodologies is the introduction of a preliminary stage that concentrates on process modelling and control. jsd uses the preliminary stage of the methodology to investigate the real world and to actually model this world. once the model of the real world has been derived, specification of the system can be undertaken. this specification is the starting point of the next phase where the data structures are investigated and finally a program written. ontological discussion of jsd so far this article has been concerned with describing the wand and weber model of a system and the system analysis and design methodology proposed by jackson. this section seeks to integrate the two by describing jackson's methodology using the terminology employed in the wand and weber model. an information system is simply a model of some real-world system. two steps are involved in the construction of a system: first, create a model of reality; and second, transform this model of reality into the information system. wand and weber (1989) state the modelling method chosen can fall into one of two categories: reality driven or information system driven. jackson's methodology uses processes to model reality. a good methodology, when deriving the decomposition of a system, should be able to represent both the static aspects of a system (structure) and the dynamic aspects of a system (behaviour). certain ontological constructs present within the wand and weber model need to be examined to determine how these constructs are represented within the methodology. in the remainder of this section, the concepts and definitions given in section two of this article, will be used to evaluate the modelling power of jackson' methodology that is, jsd is examined in terms of how various ontological constructs are represented. the modelling capacity and thus the strengths and weaknesses of jsd can be assessed by mapping the ontological constructs from the wand and weber model onto the constructs developed during the use of jsd. the following ontological constructs will be investigated: things; properties; possible state space; lawful state space; integrity and stability laws; events; coupling; system; environment; and decomposition. the above constructs represent the set of basic constructs from which any others defined in section two can be constructed. for example, once "things" are represented, then "components" can be extracted from the system. similarly, the "structure of a system" can be determined using the "things," "couplings," and the "environment." thus the following ontological constructs form the minimal set that need to be discussed. things a thing is the atomic element within the model proposed by wand and weber. jackson's methodology identifies entities and actions during its description of the real world. using the criteria for identifying entities and actions, both must exist or take place in the real 82 world outside the system and not merely be a part or an action of the system itself. entities may be viewed as items that exist in the real world. as such, they can be considered as "things" in the wand and weber model. entities will have certain attributes associated with them. these attributes are represented by state variables. actions are behaviours that entities either suffer or perform. as such, they represent the dynamic aspects of the system. they will be considered later under the heading events. properties a property is something that is used to describe the "thing" in the real world. thus, in jsd how are the properties used to describe the jsd entities represented? properties are not explicitly represented on the structure diagrams or system specification diagrams. properties are included in the narrative versions of the processes (process structure text) and functions (structured english). state space and lawful state space properties are not explicitly represented within jsd, and thus the possible values of the state variables representing these properties are also not represented. the cartesian products of the set of all possible values of each state variable is the possible state space. since the possible values for each state variable are not represented, the cartesian product cannot be determined. thus the possible state space cannot be represented. the lawful state space is a subset of the possible state space after laws have been considered. some constraints on values may be reflected through the structured text. most cannot be represented, however, as there appears to be no appropriate construct. if few constraints on the values are explicitly shown, then the lawful state space cannot be represented. integrity and stability laws wand and weber (1989) state an integrity law is a constraint on the allowed states of a thing that enables system statics to be modelled. a stability law, on the other hand, is a function that maps a thing from an unstable state to a stable state, thus enabling the dynamics of a thing to be modelled. the processes are defined in detail in the structure diagrams. these diagrams, together with the structured text, describe the actions that the entity may perform or suffer. the actions for each entity tend to be expressed in a procedural manner that is, the procedures each of the state variables undergo while the action is performed or suffered are represented in the structure text and structured english sections. the actions are not explicitly listed as states and state changes. therefore the integrity and stability laws are not explicitly represented in jsd in terms of states. they tend to be shown as restrictions on the procedures the state variables may undertake. events an event is a change of state that arises because the value of at least one state variable alters. there are two types of events: internal and external. an external event is where a state variable representing a property of a thing alters because of some action of a thing in the environment. an internal event is where a state variable alters because some action is performed or suffered by a thing not in the environment. events are represented by the actions that entities may perform or suffer. these are represented by the actions on the structure diagrams. we need to determine whether both internal and external events are represented. jsd allows for processes to exist in either the real world or within the system. those within the system (postscripted -1 or greater) may perform actions that are internal events. those postscripted -0 represent processes within the real world or the environment of the system. these actions impact on a thing in the system and thus are external events. 83 after examining the manner in which the preliminary stage of jsd describes the specification of the real world by concentrating on process modelling and control the following requirements of information system should be properly considered: (a) states of the real system should be mapped to states of the information system, (b) there should be a mirroring of external events in the real system to external events in the information system, (c) the information system's interaction with the environment must be considered and (d) events in the real system should trigger their counterpart in the information system in the same order of occurrence. coupling coupling between two things is the ability of one thing to affect the history of another thing. within jsd, two types of diagrams are used. structure charts illustrate the actions that are performed or suffered by entities. system specification diagrams illustrate the relationships that exist between processes. these two types of diagrams are interrelated. structure charts, by illustrating entities and their associated actions are, in fact, a representation of a process. this process is the item depicted within the system specification diagram. the actions that an entity performs or suffers may alter the state variables of that entity. when an entity has state variables that alter, it may affect state variables contained within other entities. this alteration of state variables of other entities is due to the state vector and data flow connections. the state variable and data flow connections are the links between processes on the system specification diagram. these links between processes thereby represent the couplings. these couplings reflect the fact that some process needs the information contained within another process to work correctly. environment a jsd system specification diagram shows which processes are parts of the system and which belong to the environment. as stated previously, those processes postscripted -0 belong to the environment, and those postscripted -1 or greater belong to the system. system all processes are represented on a system specification diagram. so are all the connections between these processes. as stated under the heading "couplings," the links or connections between these processes represent the couplings between components. all links between processes are represented. the question that needs to be raised is "are these links on the software specification diagram all the couplings?" to answer this question, we need to consider the ways in which state variables can alter and determine how each is represented. first, by one process affecting another process, the coupling is represented by a state vector or data flow connection. second, the state variable may be altered by actions within process. these actions do not affect any entities outside this process's structure chart and therefore these changes do not indicate couplings. third, the links to the environment are depicted on the system specification diagram. therefore, the couplings between the environment and the system are depicted. there is no other way by which a state variable can alter and thereby all the couplings between things are represented. since all couplings have been represented, the system is represented in full by the system specification diagram. decomposition jsd tries to model reality directly. thus, hopefully the structure of the system is a good representation of the events in the real world. however, floyd (1986) states there is no mechanism for forming abstractions, subsystems, or layers, thereby making it difficult to identify or understand which functions belong together. jsd appears to be able to 84 capture things in the real system in a time-ordered manner. it allows the user to build up a picture of the system as a whole. however, it lacks a mechanism for saying exactly how the system should be broken up into subsystems. therefore, it can be viewed as showing an overall decomposition of the system. the analyst must then apply heuristics to decide which processes and functions to place in one subsystem and which to place in another. the above section has examined the modelling power of jsd in terms of the ontological constructs proposed by wand and weber. the manner in which the various ontological constructs are represented within jsd were explicitly examined. conclusions this article has investigated the ontological aspects of the systems analysis and design methodology proposed by jackson. the aim of the ontological evaluation was to evaluate jsd in terms of the wand and weber model so that the strengths and weaknesses of the methodology can be examined. the article commenced by examining the formal model of an information system proposed by wand and weber (1990). the system analysis and design methodology, jsd, was then described in detail before the methodology and model were integrated. jsd seems to be able to include through the use of its methodology "things" that exist in the real world. these "things" are identified as part of the first step of the methodology and thus the system can be validated against the user requirements at a very early stage in development. this is certainly an advantage as a system that does not comply with the user's perception of reality can be costly to correct. many of the other ontological constructs identified by the wand and weber model are also explicitly represented by the jsd approach through the use of the diagrammatic notation used. the use of structured text is also an aid to include other ontological constructs not explicitly included in the diagrams. a good methodology should be able to represent both the structure of the system and also the behaviour of the system. jsd through the use of structure diagrams appear to capture the things in the real world in an ordered manner. through the aid of these diagrams, the structure of the system as a whole is built up by the designer. the methodology, through the aid of these same diagrams used in conjunction with the structure text, describes the actions the things may perform or suffer. in this manner, some of the behaviour of the system, i.e., the dynamics, are also represented. during the ontological evaluation of jsd, it was discovered that jsd derives the overall structure of the system to be implemented. how to decompose this structure further into subsystems is not addressed within the methodology. there are no steps to say which entities/processes/functions should be joined to form a subsystem. the decomposition procedure is entirely up to the analyst and is based on heuristic methods. one major limitation of this study comes from its heavy reliance on the ontological model proposed by wand and weber. the model of an information system proposed by wand and weber has been taken as correct. from this model, the definitions were derived that were 85 thought to be necessary to allow jsd to be evaluated using ontological constructs. any extension of the underlying ontological constructs discussed during this paper may result in an extension to this piece of research. there are many opportunities for future research in this area. the issues raised during the mapping of the ontological constructs from the wand and weber model onto the constructs created during the use of jsd form the basis of the implications and future research directions of this study. the study could be replicated for other methodologies apart from jsd and those previously investigated by wand and weber. currently there is no other theoretical foundation upon which to systematically evaluate the various methodologies available to the analyst and designer. through this type of research, hopefully a better understanding of the strengths and weaknesses of each methodology can be evaluated and eventually different methodologies can be compared using the same theoretical base. another future research direction that could be pursued relates to extensions of the wand and weber model. the aforementioned researchers have extended their model to include different criteria that they consider necessary for a decomposition to be considered a "good" decomposition. the ontological evaluation of jsd could be extended to evaluate different decompositions derived through the use of jsd to determine whether each is or is not a good decomposition. by examining the different decompositions that are obtained by the system analyst and designer and by determining whether or not they are good decompositions, formal steps may be able to be incorporated into jsd to allow the decomposition phase to be achieved with the aid of fewer heuristics. this paper has contributed to the area of information systems by attempting to provide an ontological evaluation of jsd and thus evaluating the strengths and weaknesses of the underlying methodology. the results indicate that jsd gives the user an overall system structure comprising things described by the minimal number of state variables. however, no formal way of arriving at a decomposition of the system is provided. considering this limitation of jsd, the overall ability of the methodology as a systems analysis and design methodology was considered, and it was found that many ontological constructs appear to be included. the results of this study could be added to the previous work undertaken by wand and weber, thus providing an examination of an additional systems analysis and design methodology in terms of ontological constructs. as stated in the introduction of this paper, for a proper evaluation of all methodologies, they need to be examined on common ground. this study has now placed as additional methodology on this common ground. the work undertaken here could be extended by now comparing jsd with the other methodologies already examined in terms of their ontological constructs. 86 the australasian journal of information systems (ajis) began in 1993 and was based on the simple realisation that there was a need for an academic ‘outlet’ for is research in australia australasian journal of information systems volume 16 number 2 2010 5 green it readiness: a framework and preliminary proof of concept alemayehu molla vanessa cooper school of business information technology and logistics, rmit university, vic 3000, australia abstract businesses are under increasing pressure from customers, competitors, regulators and community groups to implement sustainable business practices. balancing economic and environmental performance to be green and competitive is therefore a key strategic issue. the information technology (it) sector is one of the pioneer sectors which started working on the sustainable development model. however, it is only lately that researchers and organisations have begun to consider the role of it, not only in contributing to a businesses environmental footprint but also in tackling climate change problems. usually coined as, “green information technology”, the role of it in causing and resolving ecological sustainability, in maintaining low cost it shops, in building green reputation capital and in supporting corporate green strategies has hardly been researched. this paper identifies five main areas of green it capability and describes the main pillars of a g-readiness framework to help organisations evaluate their maturity for green it. the utility of the framework is demonstrated through a desk-based research case study of four organisations. the paper argues that just as ereadiness has been, and continues to be, a critical capability in the digital economy, greadiness is an equally critical capability in the low carbon digital economy. without a clear understanding of g-readiness, organisations would approach green it initiatives on an ad hoc and somewhat reactive basis, which is undesirable. keywords green it, digital business, g-readiness index, it strategy, sustainable business. introduction the advance of information and communications technology (ict) based business and social practices in the last few decades have transformed many, if not most, economies into e-economy and businesses into e-business. for economies, icts are increasingly playing critical roles in transforming and generating economic opportunities through automotive, informative and transformation capabilities (chen et al. 2008; elliot and binney 2008). the expansion in the use of icts in the context of global warming, climate change and sustainability portends two major issues. on the one hand, estimates indicate that the ict industry accounts for 2% of global co2 emissions, which is equivalent to the amount generated by the aviation industry (goasduff and forsling 2007). each stage of the it resources lifecycle from manufacturing to usage and disposal can pose environmental damages (elliot and binney 2008; murugesan 2007). in addition, the use of icts might have had an australasian journal of information systems volume 16 number 2 2009 6 undesirable consequence of increasing the expansion of environmentally polluting business undertakings (chen et al. 2008). on the other hand, icts can be deployed to tackle the environmental footprint of a business. this role of icts can range from a carbon foot print analysis and reporting capability through supplanting traditionally carbon emitting and eco-unfriendly business practices, to deploying computerized models to reduce energy consumption. for example, in 2005 ibm usa‟s work-at-home program enabled by ict, involved over 20,000 employees, saved more than 5 million gallons of fuel and avoided more than 50,000 tons of co2 emissions. thus, telecommuting offers an important means by which traffic congestion and pollution can be addressed (chen et al. 2008). a second example is the increasing number of utility companies that are moving to electronic billing, which has a potential to reduce paper consumption. the above implies that as much as organisational it infrastructure can contribute to carbon emission, effective and smart use of it can help organisations to reduce their emissions. this paper refers to this dual role of it as green it. green it poses two challenges to organisational it strategy and governance which thus far has focused on the economical value of it and the ethical conduct of it professionals (marshall et al. 2005; ridgley 2009). first, organisations are likely to apply environmental and sustainability considerations in it governance decisions. for example, aronson (2008) recognised the increased environmental demands of ict and proposed power efficiency as the most significant way of reducing icts environmental impacts. second and perhaps more importantly, it managers are expected to provide new tools and capabilities to support a business‟s carbon reduction and sustainability strategy (chen et al. 2008). as companies come under increasing regulatory scrutiny to reduce their carbon intensity, it initiatives can deliver some of these reductions. green it, however, requires new organisational capabilities. strategies in greening it have to start with a clear assessment of where the organisation is, its intended state and how far along the green it path it has progressed. the speed with which environmental sustainability impacts it activities and economics requires a framework that not only accounts for factors enabling the spread and usage of green it but that also explicitly considers the roles played by key stakeholders, such as vendors, it professionals, business executives and the government. to date, few frameworks have emerged to serve as reliable guides for organisations and researchers alike on their green it practice and investigation respectively. as a result, there is a lack of understanding and tool to assess the green it maturity of organisations. this paper has produced a green it readiness framework and case examples that demonstrate the utility of the framework. the paper aims to answer the following question: how can businesses assess their green it readiness? the contribution of this paper to both it theory and practice are manyfold. first, the paper addresses a critical research area that has received little academic research attention. this paper is, therefore, one of the first attempts to provide a theoretical lens to research green it. second, the framework offers a common platform for practitioners to assess and benchmark their green-it initiatives and progress. third, researchers can use this framework to understand important issues such as the drivers, values and antecedents of green it. research methods the conceptual g-readiness framework described in this paper was developed by applying understanding from extant literature on e-readiness and corporate social responsibility (csr), along with extant literature and practitioner publications investigating issues surrounding green it, such as the adoption of green supply chain practices. extant academic information systems (is) literature is australasian journal of information systems volume 16 number 2 2009 7 used where available – although an extensive literature review of top is journals has revealed that little academic research has occurred on the topic over the last 30 years (elliot and binney 2008). in order to demonstrate the utility of the “g-readiness” framework, desk-based research utilising secondary data from case studies of anz, telstra, australia post and bhp are presented. these organisations were selected because they report to be proactive market leaders in the practice of csr, e-business and the green movement as summarised in table 1. company description key csr, e-business and environmental activities anz bank one of the largest australian companies and a major international financial services organisation. anz offers a comprehensive range of financial services to individuals and small and large organisations (anz 2009).  ranked as the number one bank globally on the dow jones sustainability index  employs strategies to reduce direct and indirect negative impacts of the organization on the environment  employs a strategy to improve the technology levels used by banks globally australia post australia post is a government business enterprise (gbe). australia post offers a range of traditional mail and courier services for individuals and corporations, along side a suite of electronic services such as bill payments (australia post 2009).  developed policies and strategies for csr including for the environment incorporating substantial commitments to minmising its environmental footprint  provides support for a range of community sustainability initiatives such as “green pc” bhp billiton a large resources company operating in aluminum; energy and metallurgical coal; copper; manganese; iron ore; uranium; nickel; silver and titanium minerals; diamonds; and oil, gas, liquefied natural gas (bhp 2009).  has extensive policies for health, safety, csr and the environment  sets specific targets for greenhouse gas emissions and has had greenhouse gas reporting processes in place for some time telstra australia‟s largest telecommunications company offering mobile and home phone, internet and tv-based services (telstra 2009).  a range of environmental and csr policies covering issues such as carbon emissions; resource efficiency and waste; and energy efficiency.  measurement of greenhouse gas emission in place for several years table 1 company backgrounds conceptual background although the term “green it” is becoming more common in discussion, there is still little common understanding of what green it actually means. as green it is a nascent field, there is very little academic research on the topic. in order to develop the conceptual foundation of the framework proposed here, we undertook review of practitioner oriented green it publications and literature on the adoption of green process technologies, green supply chain practices and e-readiness. australasian journal of information systems volume 16 number 2 2009 8 green it a review of practitioner publications as summarised in table 2 indicates that green it is usually associated with technologies and techniques for improving the energy efficiency of data centres (acs 2007; mitchell 2008; pettey 2006); for reducing the environmental impact of it operations (nunn 2007); and the use of it as an enabler of organisational-wide green initiatives (gartner 2008; mines and davis 2008). it encompasses not only hard technological solutions but also soft business practices and managerial actions to make it-decision making as eco-friendly as possible (info~tech 2007a). from a process technology perspective, green process technologies can be classified as end of pipe technologies and clean technologies (del rio gonzalez 2005). end of pipe technologies reduce the environmental impact of emissions without necessarily changing the production process. on the other hand, clean technologies cause significant changes in the production process and their adoption is intended to reduce the level of environmental impact along a product‟s life cycle from design to consumption. from a supply chain perspective, green supply chain refers to integrating environmental thinking into the product design, sourcing, manufacturing, warehousing, distributing and end of life product management aspects of a supply chain (rao and holt 2005; srivastava 2007). reference conceptualisation of green it acs (2007) concern “about energy consumption and subsequent carbon dioxide emissions from commercial ict equipment”. mines & davis (2008) “green it is part of a fundamental change in the economy and society. it is a subset of the larger green (sustainable) business trend, which reconciles sustainable business practices with profitable business operations”. info~tech (2007a) identified 11 technologies and initiatives as indicators of green it. these include “equipment recycling, server consolidation and virtualisation, optimizing data centre energy efficiency, print optimisation, data centre airflow management, rightsizing it equipment, green considerations in sourcing and rfps, hot aisle/cool aisle data centre layout, budget allocation for green it projects, liquid cooling for it equipment, dc powered it equipment, airside/waterside economizer, carbon offsetting”. gartner (2008) defines green it based on the role of cios. “for most cios greenhouse gas emissions and getting it’s own house in order are the main issues. their more important role, though is to help the enterprise address its enterprise wide environmental sustainability issues… [by playing] one or more of three roles: provide analytical tools, provide analytical/technical insight [and] lead change”. nunn (2007) for it hardware suppliers, green it is “running it assets and services in a more energy-efficient way… how to limit and hopefully reduce the organisation’s overall carbon foot print”. mitchell (2008) in the data centre, going green is about energy efficiency first and foremost. fortunately, efficiency is a natural by-product of solving the cooling, power and space challenges that today’s data centres face. table 2 examples of green it conceptualisation e-readiness one of the key challenges following the introduction of the internet has been how to successfully transform from traditional business models and practices to e-business models. a number of national and international initiatives have been undertaken to assess the e-readiness of countries (choucri et al. 2003; mia and dutta 2007). the aims of such initiatives at macro levels are to benchmark and australasian journal of information systems volume 16 number 2 2009 9 monitor the progress of countries to capture the opportunities enabled by the internet and other forms of icts. from organisational specific e-readiness frameworks, we understand that e-readiness can be a source of competitive advantage in the networked economy and the prerequisite for successful ebusiness (hartman et al. 2001; molla and licker 2005). likewise, e-readiness assessment helps an organisation to pinpoint some of the hurdles that it might face in its trajectory towards e-business. it facilitates determining an organisation‟s capacity for e-business and serves as a tool for guiding strategic planning processes in developing e-business. having resources such as skilled manpower, technology, appropriate organisational culture, organisational capabilities and learning, and overall organisational commitment in the form of management and administrative support, staff involvement and championship have been identified as constructs of e-readiness (mia and dutta 2007; molla and licker 2005). the insights from e-readiness studies highlight not only some of the variables but also the importance of e-readiness as a critical capability required to execute in the e-economy successfully. we draw a parallel from e-readiness and argue that green it readiness (g-readiness) could be an equally critical quality required to execute e-business or e-government successfully in the low carbon e-economy. the g-readiness framework the development of the g-readiness framework proceeds in two steps – defining green it and explaining the constructs that make up the g-readiness. based on the review in section 3.1, green it can be considered as a holistic and systematic approach to address the challenges surrounding the it infrastructure such as data centre energy efficiency; it’s contribution to reducing the environmental impacts of business it activities (such as through adopting green technologies), it’s support for environmentally sustainable business practices (such as in enabling green supply chain management through carbon foot print monitoring and building tools for energy management options) and it’s role (such as supplanting high co2 emitting business practices) in the low-carbon economy. thus conceptualised, green it covers four different but interrelated perspectives – sourcing, operations, services and end of it life management:  from a sourcing perspective, green it implies the practice of environmentally preferable it purchasing. this involves adoption of sourcing practices such as analysis of the environmental foot print of an it hardware supply chain, evaluation of the green track record of software and it services providers, incorporating green issues (such as recyclable design and packaging) in vendor evaluation, and inclusion of social concerns (such as the presence of harmful materials in it supply chain) in it procurement decisions.  from an operations perspective, green it implies improving energy efficiency in powering and cooling corporate it assets and reducing it induced greenhouse gas emissions. two types of energy consumption reduction can be identified – temporary and structural consumption avoidance. while temporary avoidance refers to optimisation of energy utilisation without reducing the installed power base, structural avoidance results in reduction in installed power capacity. a number of green technologies, information systems and practices related to the two categories can be used as indications of the adoption of green it operation.  from a systems perspective, green it refers to the role of it in supporting a business‟s overall sustainability initiatives – green is. green is therefore includes adoption of analytical information systems for green supply chain management, environmental management and carbon foot print analysis. it also includes ict based low carbon business solutions such as telecommuting, video-conferencing, thin client and web based business services, virtual collaboration and ip telephony.  from an end of it life management perspective, green it refers to practices in reusing, recycling and disposing it hardware. australasian journal of information systems volume 16 number 2 2009 10 one of the primary questions following such a broad understanding of green it is therefore: what does it take for organisations to succeed in greening their it? drawing from the background review presented above, this paper argues that as much as e-readiness has been and still continues to be a critical capability to succeed in the digital, web-centred and e-economy, g-readiness is a critical capability to succeed in the low carbon and green digital economy. g-readiness is conceptualised as an organisation‟s capability to implement holistic green it practices. it demonstrates the comparative levels of green it development among businesses and serves as a benchmark for measuring an enterprise‟s progress to participate in the global low-carbon e-economy. greening it naturally starts with it and business leaders‟ sentiments towards climate change and the role of business in environmental sustainability (rao and holt 2005). the literature review on climate change shows that there are two main positions, one dominant and mainstream and another minority and sceptical, about the causes and issues of climate change. most submit that the climate is changing at an alarming rate and human (including business) activities are the main causes of the change (ipcc 2007; stern 2008). on the other hand, there are a minority of scholars who are skeptical about the causes and impacts of climate change (royal society 2007). this group maintains that the climate is always changing and its impact is not as severe as the other side would have liked us to believe. they believe that human and business activities have nothing or very minimal to do with the changing environment. info~tech‟s (2008a) global green it attitude and action survey of 1260 it professionals indicates that only 50% of participants are concerned about climate change. the survey further found that attitudes towards green it vary from region to region where the rest of the world (africa, south america and oceania) leads europe, asia and north america in terms of positive opinion towards climate change. the attitude of managers and business leaders towards environmental sustainability is a key factor in understanding not only the challenges of green it but also the opportunities associated with it (gartner 2008). organisations that are concerned about their social and environmental responsibilities, business sustainability and green it can start by articulating policies to tackle those issues. green it policies can cover areas such as pc power management (info~tech 2007b; velte et al. 2008), environmentally preferable purchasing (info~tech 2007c; velte et al. 2008), it architecture and data centre design (rasmussen 2006) and end-of-life disposal (velte et al. 2008). attitude and policies demonstrate the necessary but not sufficient capability for greening it. although, open standards such as advanced configuration and power interface (acpi) provide pc power management capabilities, in practice, few organisations utilise these capabilities (info~tech 2007a; velte et al. 2008). there are also variations in the practice of how aggressive an organisation can be in its power management regime and in terms of user education and user compliance (info~tech 2007a; velte et al. 2008). in a similar fashion, despite showing concerns for the environment and having corporate social responsibility (csr) policy statements, only limited numbers of businesses actually consider green issues in data centre design, sourcing it hardware, adoption of green technologies, and end of life it management (info~tech 2007a, 2008a; mitchell 2008; velte et al. 2008). this implies two things – disconnect between policy and practice, and variations in the actions and practices of green it. there are also issues surrounding the organisational arrangement on who should lead green it initiatives. existing governance arrangements vary from it playing a role of tool provider to leading changes (gartner 2008). australasian journal of information systems volume 16 number 2 2009 11 based on the review, we argue that there are at least five important properties for success in greening it – attitude, policy, practice, technology and governance – which together create the critical capability we call “g-readiness” (figure 1). figure 1 the g-readiness framework g-readiness attitude policy technology governance it attitude business attitude sourcing business infrastructure it infrastructure it governance environmental governance practice operations end of it life operations end of it life sourcing australasian journal of information systems volume 16 number 2 2009 12 attitude attitude refers to the affective characteristics of both business and it leaders and professionals. it can define the level of ict professionalism (boughton 2009) and has been found to be one of the major factors in influencing the acceptance and use of technologies (handy et al. 2001). in this paper, attitude refers to the extent to which both it and business professionals are aware and interested about the environmental concerns related to the use of it and the role of it in resolving environmental problems. while investigating the relationship between environmental attitudes and behaviour, chan and yam (1995) found that to encourage people to act environmentally, emotional appeal has a stronger impact than logical reasoning or factual description of harmful effects from environmental pollution. this is because the knowledge of an individual is weakly related to self-reported actual environmental behaviour. on this basis, whether or not an organisation takes green it issues seriously will be (at least partially) dependent on it and business leaders‟ sentiment to environmental concerns. existing surveys (see table 3) indicate that not only could opinions for green it might vary from organisation to organisation, they might vary from region to region. table 4 provides a preliminary proof of concept of the relevance of assessing green it attitude. item europe north america asia others reference environmental concerns in planning it operations are: very important somewhat important not important 48% 45% 6% 33% 52% 15% na na mitchell (2008) i am concerned about climate change: (strongly agree percentage); n= 1260 59% 52% 55% 59% info~tech (2008a) the issues of green it is on my company’s radar (strongly agree percentage); n=1262 16% 16% 15% 18% info~tech (2008a) i am concerned about reducing it’s power consumption (strongly agree percentage); n= 1264 45% 35% 39% 38% info~tech (2008a) table 3 survey evidence on green it attitude policy policy readiness measures the extent to which green and sustainability policies are developed throughout an organisation and permeate the value chain. for example, while some companies have clear green it policy others do not. three value chain areas can assess the extent of policy readiness – it sourcing, it operations and services and it end-of-life management. it sourcing policy – refers to the extent to which an organisation has adopted an environmentally preferable purchasing policy (epp) (velte et al. 2008) and articulated clear green guidelines for buying it equipment and services. epp is a policy choice that encourages purchasing decisions with minimum environmental impacts. some of the negative environmental impacts of it equipment include energy-intensive production methods, wasteful packaging, poor recycling practice and heavy australasian journal of information systems volume 16 number 2 2009 13 use of hazardous materials and practices (elliot 2007; info~tech 2007c). epp enables organisations to adopt a more sustainable sourcing strategy. for example, a policy that favours acquiring laptops over desktops demonstrates a commitment to green it. estimates indicate that selecting an efficient laptop and operating it efficiently can reduce energy use by 98 99% (commonwealth government of australia 2001). the extent of green guidelines might cover both modern and traditional it equipment. for instance, the commonwealth government of australia published a green office guide in 2001 which covers the acquisition and use of office equipment – desktop, monitors, photocopiers, printers, fax machine, scanners, and multifunction devices. company preliminary evidence of concept anz anz‟s top management attitude towards green it is highlighted from statements in its carbon disclosure project (anz, 2007). the ceo opined “while we are on the earth today, our primary responsibility is to leave the planet in a better state than when we inherited it”. this sentiment to environmental concerns from the use of it is further strengthened by the involvement of top management through the initiatives of anz cio, who‟s intention was to remove a total of 400 servers from the bank's infrastructure by the end of 2008 both to reduce electricity cost and carbon emissions. the cio opined “we have an obligation and an opportunity to take the lead in our organisations about reducing at least the it impact on the environment that our organisations have”. anz has expressed a commitment to become carbon neutral in australia and new zealand by using renewable energy by 2009. the above offers an initial insight as to anz‟s top managements interest, commitment and involvement towards greening it. australia post in australia post‟s 2002 greenhouse challenge report, the organisation has identified that it has been a participant in the greenhouse challenge program since 1997/1998 and has since accumulated greenhouse gas emissions savings of some 200,373 tons of co2. bhp bhp‟s green it attitude can be gauged indirectly from its commitment in reducing its greenhouse gas emissions using advancements provided by technology and improving the recycling of its waste products. bhp is looking into energy efficiency improvements and alternative energy for its operations; setting greenhouse gas emission reduction plans and energy targets and measures; investing in research and development of low carbon emissions technologies; and has shown a commitment to the design of effective national and international climate change policies through active participation. telstra telstra has set an international precedent among the telecommunications industry by finding ways that the industry can contribute towards a national shift to a low-carbon economy. it has spearheaded the commissioning of a study called “a highbandwidth, low-carbon future: telecommunications-based opportunities to reduce greenhouse gas emissions.” this study proposes that telecommunications networks can help reduce australia's greenhouse gas emissions by almost five per cent by 2015 and deliver up to $6.6 billion a year in financial savings for australian businesses and households. as such, telstra is aware as well as concerned about environmental issues and the impact of green it to the environment. table 4 example indicators of green it attitude it operations and services policy – encompasses the extent to which the services provided by the it infrastructure support issues encapsulated in business sustainability. some of the policy considerations include pc power management (info~tech 2007b); policy on staff computer usage australasian journal of information systems volume 16 number 2 2009 14 (commonwealth government of australia 2001) and environmental policy (goasduff and forsling 2007). it end of life policy – refers to the policies and regulations related to the disposal and settlement of it equipment/machineries in organisations. in certain regions, end-of-life recycling is required by the provisions of law (elliot 2007; info~tech 2008b). for example, the japan ministry of trade and industry promotes recycling policy to create a sustainable society that strikes a balance between the environment and the economy (ministry of economy, trade and industry, environmental protection and recycling 2008). however, in general, there are still no mandatory policies to enforce the rules in most regions. the g-readiness of businesses in terms of policies can be assessed on the basis of the extent to which the three areas of green it polices are developed. this policy reflects an organisation‟s commitment to technology redundancy and to the roll-over of equipment in order to gain the benefits of each technological advance. table 5 offers a preliminary proof of concept of the relevance of assessing policy. company preliminary evidence of concept anz anz has a sustainable sourcing policy with its suppliers to reduce the amount of greenhouse gas emissions within its organisation. anz has clear guidelines in place with its it suppliers in the use of green technology. for example, in the area of it sourcing, anz has an obligation and a clear-cut policy with green guidelines to identify energy efficiency opportunities within its organisation and report progress towards implementation of this to the federal government under the energy efficiency opportunity act (2006). australia post australia post has participated in the federal government's co-operative greenhouse challenge program since 1998 and has internal initiatives for energy-saving; wastemanagement; vehicle fleet fuel efficiency and providing advice and assistance to other large fleet operators. bhp bhp intends to have greenhouse gas management and energy conservation programs at all of its sites that have annual emissions greater than the equivalent of 100,000 tons of co2. bhp also has in place an health, safety, environmental responsibility and sustainable development (hsec) policy. telstra telstra has undertaken a project to investigate how telecommunications networks and digital products can enable business enterprises, households and governments to reduce carbon emissions within australia. telstra has a green purchasing environmental policy that was launched in june 2002. some of the objectives of this policy are to “consider environmental issues in the purchasing of all products and services; purchase environmentally preferable products and services; contract with vendors that demonstrate responsible environmental performance; and establish a purchasing framework that encourages vendors to improve their environmental performance and/or the environmental performance of their product or service”. table 5 example indicators of green it policy practice the policy dimension of g-readiness captures the intellectual dimension of g-readiness. however, not all policies are implemented smoothly. indeed, organisations might vary in the actual australasian journal of information systems volume 16 number 2 2009 15 implementation of their policies. a business‟s green practice along its value chain from inbound to reverse logistics influences g-readiness (rao and holt 2005). green it sourcing practice captures the extent to which environmental considerations are factored in it and other purchasing decisions. this practice might vary from one where there is no environmental consideration to a case where environmental considerations are given higher weight (info~tech 2007a; snege et al. 2008). as noted earlier, consumer buying habits are increasingly driven by ethical concerns. green issues may not only have an impact on consumer buying power, but also affect how both the public and private sector award competitive tenders (whitby 2007). generally though, green sourcing revolves around evaluating the environmental behaviour of suppliers and partnering with suppliers to improve their performance (rao and holt 2005). green sourcing practices also include advocating the use of green technologies during request for proposal processes and shortening it equipment refresh periods to gain access to energy efficient equipment (info~tech 2007d). the involvement of suppliers is a critical element of green it sourcing practice (rao and holt 2005). green it operations and services practices – green it operation practices can involve people, clients, servers, and the network critical physical infrastructure (ncpi). at the client level, using advanced configuration and power interface (acpi), it managers not only can reduce power consumption by “slowing down processors, spinning down hard disks and shutting off monitors” but also reduce a firm‟s environmental footprint (info~tech 2007b). however, few businesses implement this. other operational actions to reduce power consumption include retiring systems, operating existing systems in an efficient manner and migrating to more energy efficient platforms (for example using blade servers) (rasmussen 2006). in addition to supporting green it by purchasing green hardware equipment and software to reduce energy cost and consumption, companies can also cultivate green it practices – the way employees use it (velte et al. 2008). at the ncpi level, techniques such as “right-sizing the ncpi system to load, using efficient ncpi devices and designing an energy-efficient system” can be used (rasmussen 2006). green it end of life management practices – this refers to the compliance of it equipment/machinery manufactures, users, and resellers in green it end of life management. in terms of it manufactures, issues concerning green it end of life management would be whether the it equipment and/or packaging is reusable because to crush and/or burn these it equipment/materials could harm the environment (alsever 2008). for example, when a customer purchases an apple computer or monitor, the company will provide free recycling services to handle the customer‟s old computer or monitor, regardless of the manufacturer (apple 2008). it users would refer to practices taken in handling the broken and unwanted it equipment. for example, users are encouraged to consider selling their unwanted equipment through the cellforcash.com for recycling, whereby unwanted items will be refurbished and resold (alsever 2008). many organisations have encouraged donation of unwanted it equipment/machinery to non profit organisations and schools. overall, the extent to which it has addressed the following issues can provide an indication of the practice dimension of g-readiness:  data collection on vendors‟ green rating  the extent of contacts awarded to suppliers that use green technology  auditing the power consumption of existing systems  it projects implemented to reduce power consumption requirements  it projects implemented to maximise power utilisation (pc power management)  the extent of enforcement of pc power management  the energy rating of implemented technologies  actions taken to reduce it‟s carbon footprint  projects implemented to monitor enterprise carbon footprint australasian journal of information systems volume 16 number 2 2009 16  the extent that equipment/machinery is recycled. based on both the secondary evidence (table 6) and our analysis, practice readiness indicates to what extent an organisation has translated its concerns and policies into actions. company preliminary evidence of concept anz the anz supplier sustainability assessment tools is a requirement for all tenders and includes specific questions relating to greenhouse gas emissions. anz‟s goal was to increase the number of suppliers (including it suppliers) undergoing detailed social and environmental screening by 50% in 2008. so far anz has conducted social and environmental screening of major 32 suppliers in australia and new zealand. it also uses life-cycle assessment to evaluate the greenhouse gas emissions associated with electronic transactions and to reduce its greenhouse gas emissions. australia post in disposing its old computers, australia post has partnered with greenpc and technology for schools to refurbish and recycle old computers and donate them to schools and charities that can't afford such technology. in recycling ink and toner cartridges, australia post has partnered with cartridges 4 planet ark, and this partnership enables customers to drop off their used ink and toner cartridges for recycling. it has also conducted a company wide staff environmental awareness survey covering everything from their recycling practices to switching off lights. bhp bhp has the world‟s first power plant fuelled by coalmine ventilation air that powers extensive business operations including it operations. this a$30 million plant generates some six megawatts of electricity per hour while reducing greenhouse gas emissions. at bhp billiton illawarra ventilation air containing dilute methane is being used to generate electricity commercially. telstra telstra has reduced vehicle fuel consumption by 5 per cent and increased productivity by 15 per cent by installing gps; used energy management programs that are it inclined to save the equivalent annual greenhouse gas emissions of 2,225 australian homes; reduced office paper consumption from 9 to 7.4 reams-per-staff-member; and has replaced its 36,000 colourful screensavers with black screens. table 6 example indicators of green it practice. technology green it is also about acquiring more environmentally effective (greener) technologies. a key driver of g-readiness success in the area of technology is to build a green technological infrastructure. this includes both ncpi, such as power supplies and it infrastructure (rasmussen 2006; valte et al. 2008). some of the commonly adopted green technologies include server virtualisation, data centre energy optimisation and rightsizing it equipment (info~tech 2007a; velte et al. 2008). further, rossi (2007) highlights that businesses and countries spend billions of dollars each year to power computers. this creates a bad image for it as being energy-consuming and bad for the environment. rmit university, which claims to be leading australian universities in green electric power consumption, is incrementally moving to buy more green power from 2% in 2007 to 15% in 2008 to 20% in 2010 (rmit 2007). to measure g-readiness along the technology dimension, organisations can look at the following indicators:  the extent to which an organisation has a green business infrastructure (such as green rated buildings) and green power sources  the development of green it standards across the enterprise australasian journal of information systems volume 16 number 2 2009 17  server consolidation and virtualisation  the extent that applications and technologies are retired for greener technologies  the extent of solutions development to support enterprise wide green initiatives. based on the secondary evidence (table 7) our analysis identifies that the four organisations investigated in this study have undertaken a range of green technology initiatives. company preliminary evidence of concept anz anz uses solar-powered atm's, new building to achieve a 6 star green star environmental rating and has decommissioned 192 servers to date with an estimated saving of au$120,000 in electricity and about 2,500 tons of co2 emissions. future plans include the decommissioning of 400 servers with estimated electricity savings of au$250,000. replacement of approximately 12,000 higher energy intensive crt monitors to more greenhouse friendly lcd screens is also planned, with an are expected reduction of more than 1000 tons of co2 emissions. anz has „pc snoozer branches‟ where pcs are automatically powered down overnight. australia post australia post has implemented a new building management system (bms) in its largest mail facility in sydney. this was done to control the air handling units and the lux levels of the dock lighting. the introduction of this facility is expected to result in a 4.3% reduction in energy consumption, with a greenhouse gas emission saving of 477 tons of co2 equivalent. australia post is currently working with the csiro and the queensland government to establish a trial of a fuel cell. bhp bhp uses an emissions inventory system throughout the group that meets evolving best practices. the company has a us$300 million financial commitment to invest in energy efficiency improvements and alternative energy for its operations. the company also supports research and development and demonstration of low emissions technologies with relevance to its businesses, customers and communities. telstra telstra makes use of new “in-person” video-conferencing in place of many face toface meetings. telstra has implemented successful systems in its directories business, sensis, to encourage recycling and has installed a renewable energy center in one of its melbourne buildings. telstra recommends the use of text messaging to public transportation using wireless broadband, of real-time allocation systems for better freight management. table 7 example indicators of green it technology governance governance is a key consideration of the dimension of g-readiness and refers to the management infrastructure to implement green it. governance offers a framework for defining the complex set of relationships and activities in organisational relationships (davies 2002). it is the operating model that defines the administration of green it initiatives. gartner‟s (2008) case study reveals that green it requires “sound management infrastructure to understand impacts, prioritise actions and manage the enterprise‟s responses”. roles, responsibilities, accountability and control for green it initiatives need to be clearly established. should a business assign the responsibility for green it initiatives to cio‟s, or should it come under environmental managers? should it organisations be held responsible for electricity costs and accountable for energy efficiency? answers to these and similar questions define the governance dimension of green it. australasian journal of information systems volume 16 number 2 2009 18 how a business as a whole manages its environmental and social responsibility influences the role of cios in green it initiatives. existing practices vary. in the anz, it leads green it initiatives (gartner 2008). in others, it‟s role is restricted to providing either tools or insights (gartner 2008). in some companies in asia, and in very few in europe and north america, electricity costs are the responsibilities of it (info~tech 2008a). governance also includes allocation of budget and other resources to green it initiatives and defining metrics for assessing the impacts of green it initiatives. for example, info~tech‟s (2008a) global survey reveals that out of 1257 respondents, about “65% to 70% in asia, europe, africa, south america and oceania agree that their companies will have green it budget”. further, the survey reveals that only one fifth of enterprises in europe and 15% in north america, have developed metrics for measuring it power consumption efficiency. businesses need to develop a standard administrative process to develop green it. currently, such frameworks come in the form of six-sigma, iso 14001; isoi 9000 and world business council for sustainable development (wbcsd) standards (gartner 2008; rao and holt 2005). in the us, data centres are beginning to get lead (leadership in energy and environmental design) certified, a certification from us‟s green building council (dunn 2008). overall, the governance dimension of g-readiness can be measured using the following indictors:  clearly defined roles, responsibilities, accountability and control for green it initiatives  existence of standard administrative processes for developing green it initiatives  establishment of metrics for assessing the impact of green it initiatives  allocation of budgetary and other resources for green it  the role of cios in enterprise wide green initiatives  the responsibility of it in electricity costs. based on the secondary evidence (table 8) the four organisations investigated in this study have approached green it governance in a variety of ways. company preliminary evidence of concept anz cio plays a key role in enterprise wide green it initiatives. operations, technology and shared services has overall responsibility for developing and implementing anz‟s energy efficiency program, including aspects pertaining to meeting anz‟s carbon neutral commitments. australia post australia post has an environment manager in charge of pc refurbishment and recycling. it partners with greenpc and infoxchange, with the assistance of infotech research, in leading the development of a supply chain program to help maximize recycling and minimize waste. bhp manages environmental sustainability as part of its health, safety, environmental responsibility and sustainable development portfolio. has a vice president of sustainable development and community relations but its reign over it issues is not clear. telstra telstra has a management infrastructure led by its cio in implementing and advocating for green it. table 8 example indicators of green it governance australasian journal of information systems volume 16 number 2 2009 19 discussion and conclusion the green movement has been identified as a significant social movement and a general reaction to the malfunctioning of the western social formation (galtung 1986; mathur and mathur 2000). it incorporates many aspects of everyday life such as politics, consumerism, technology, product purchases and consumption, marketing, manufacturing and resources (mathur and mathur 2000). the green movement differs from many other social movements in that it denies that basic social problems can be solved by addressing a single factor and argues that a holistic approach is needed (galtung 1986). as identified earlier, whether an individual joins the green movement or not, is said to be based on subjective motivation and subjective capability (galtung,1986). motivations for joining the green movement have will have important implications for organisations considering “green it” and sustainable business practice. from an organisational perspective, references to csr and “sustainability” have become commonplace (hendry and vesilind 2005). the literature on sustainable business practice and csr indicates that organisations‟ capability to comply with the mounting demands of different environmental groups and government regulations and practice socially oriented moral management is a major concern and an issue that might affect competitiveness (carroll 1991; gartner 2008; rao and holt 2005). businesses use sustainability and csr initiatives such as green supply chain management, environmentally preferable business practices; environmentally friendly technologies, and an aggressive stance towards co2 emissions, to demonstrate their commitment to the environment (mcwilliams et al. 2006). this paper presents a framework to understand green it. as green issues continue to entice global debate, it is expected to play a crucial role in both greening its operations and services and supporting a business‟s overall environmental sustainability objectives. most cios and it managers are facing two conflicting demands. on the one hand, the growth of digital business has led to increasing demands for data centres and related technology in general. on the other hand, the rising cost of energy, its cleanliness and its availability are limiting the supply of energy to those data centres and other technologies. this requires it management to turn to green it solutions. in this paper, four dimensions of greening it – sourcing, operations, systems and end-of-it life management have been identified. for a business to operate successful in one or all of these dimensions, it needs to demonstrate g-readiness. the framework can relate to implementing green business practice in general. however in this paper, g-readiness is conceptualised as a measure of a company‟s it preparedness to be environmentally responsible and competitive. the five dimensions that make up g-readiness can be combined in a variety of permutations to separate organisations that are successful in building green it from those that are less successful. separately, the low level development of the five attributes can represent barriers to green it success. in this paper, we have also explored secondary sources of four large well-known australian organisations to demonstrate the practical relevance of each of the dimensions of the g-readiness framework. the organisations explored in this paper are reportedly forward thinking, with respect to csr and environmental issues and we anticipate that future studies will reveal that the components that make up g-readiness are well developed at a handful of companies, in their infancy at a few more, practically nonexistent in most. further research investigating potential levels of sophistication within each dimension and potential relationships between the dimensions of the g-readiness framework would also be valuable. g-readiness doesn‟t come easily. it might come easily for “born-green” (backer 2007) companies. by virtue of being created green, these companies might avoid much of the digital age inertia with which their more-established counterparts must contend. for others, however, it requires a concerted effort australasian journal of information systems volume 16 number 2 2009 20 from attitude to technology from policy through practice to governance. disparities in the level of greadiness could translate to disparities in the sustainability of businesses and hence could influence a business‟s performance. understanding and leveraging green it is critical for businesses‟ continued progress. without a clear understanding of g-readiness, companies will approach green it initiatives on an ad hoc, somewhat reactive basis insufficiently supported by the structural requirements to execute competently. this may lead to sinking resources into green it initiatives without seeing results anywhere near expected returns. as green issues continue to impact strategy, business operations and it itself, lack of g-readiness may translate to missed opportunities for competitiveness and success. references acs 2007. “australian computer society policy statement on green ict”, retrieved on 19 december 2009 from www.acs.org.au. alsever, j. 2008. “the „green‟ way to dump electronic junk”, retrieved on 23 june 2008 from http://www.msnbc.msn.com/id/24163506/. anz 2007. “carbon disclosure project (cdp5) greenhouse gas emissions questionnaire”, retrieved on 19 december 2009 from http://www.anz.com/documents/au/aboutanz/anz_cdp5_response_final.pdf. anz 2009. “anz about us”, retrieved on 19 december 2009 from http://www.anz.com.au/ aboutus. apple 2008. “apple and the environment”, retrieved on 24 june 2008 from http://www.apple.com/environment/recycling/program/index.html accessed. aronson, j. 2008. “making it a positive force for environmental change”, it professional, ieee computer society, (january/february), pp. 43-45. australia post 2009. “australia post – about us”, retrieved on 19 december 2009 from http://www.auspost.com.au/cap/0,1469,ch2007%257emo19,00.html. backer, l. 2007. “when oil and wind turbine companies make green sense together”, business strategy and the environment, (18:2), pp. 43-52. bhp 2009. “bhp billiton resourcing the future”, retrieved on 19 december 2009 from http://www.bhpbilliton.com/bbcontentrepository/docs/sustainabilitysummaryreport2009.pdf boughton, c. 2009. “what is an ict professional anyway”, australasian journal of information systems, (16:1), pp. 149-163. carroll, a. 1991. “the pyramid of corporate social responsibility: toward the moral management of organisational stakeholder”, business horizon, (july-august), pp. 39-48. chan, r. y. and yam, e. 1995. “green movement in a newly industrializing area: a survey on the attitudes and behaviour of the hong kong citizens”, journal of community and applied social psychology, (5), pp. 273-284. chen, a. j. w., boudreau, m., and watson, r.t. 2008. “information systems and ecological sustainability”, journal of systems and information technology, (10:3), pp. 186-201. choucri, n., maugis, v., madnick, s. and siegel, m. 2003. “global e-readiness for what?”, retrieved on april 1 2004 from http://ebusiness.mit.edu. commonwealth government of australia. 2001.“the green office guide”, retrieved on 19 december 2009 from http://www.energyrating.gov.au/library/pubs/greenofficeguide.pdf. davies, j. 2002. “models of governance a viable systems perspective”, australasian journal of information systems, (9:2), pp. 57-67. http://www.acs.org.au/ http://www.bhpbilliton.com/bbcontentrepository/docs/sustainabilitysummaryreport2009.pdf australasian journal of information systems volume 16 number 2 2009 21 del rio gonzalez, p. 2005. “analysing the factors influencing clean technology adoption: a study of the spanish pulp and paper industry”, business strategy and environment, (14:1), pp. 2037. dunn, d. 2008. “building to be green all over”, computerworld, retrieved on 23 august 2008 from http://www.computerworld.com/pdfs/lfg_green_it_2008.pdf. elliot, s. 2007. “environmental sustainable ict: a critical topic for is research?”, in f. tan and j thong (eds) proceedings of pacis 2007, ais, 4-7 july, auckland new zealand. elliot, s., and binney, d. 2008. “environmentally sustainable ict: developing corporate capabilities and an industry relevant is research agenda”, proceedings of pacis 2008, 4-7 july suzhou, china. galtung, j. 1986. “the green movement: a socio-historical exploration”, international sociology, (1:1), pp 75-90. gartner 2008. “going green: the cio‟s role in enterprise-wide environmental sustainability”, gartner exp premier, may 2008. goasduff, l., and forsling, c. 2007. “gartner says 50 percent of mid and large sized western european it organisations will develop a green strategy by the end of 2008”, gartner, egham, 22 may. gorard, s. 2001. “quantitative methods in educational research: the role of numbers made easy”, continuum international publishing group, london. handy, j., whiddett, r., and hunter, i. 2001. “a technology acceptance model for interorganisational electronic medical records systems”, australasian journal of information systems, (9:1), pp. 39-50. hartman, a., sifonis, j., and kador, j. 2001. net ready: strategies for success in the economy. new york: mcgraw-hill. hendry j. r., and vesilind, a. p. 2005. “ethical motivations for green business and engineering”, clean technologies and environmental policy, (7:4), pp. 252-258. info~tech 2007a. “11 green initiatives your peers are cultivating”, info~tech research group, (july), pp. 1-14. info~tech 2007b. “pc power saving plans reduce costs and environmental impact”, info~tech research group, (december), pp. 1 -10. info~tech 2007c. “exercise environmentally preferable purchasing for a healthier plant and pocketbook”, info~tech research group, (october), pp. 1-9. info~tech 2007d. “top 10 energy-saving tips for a greener data centre”, info~tech research group, (april), pp. 1-11. info~tech 2008a. “north america underperforms in green it attitudes and actions”, info~tech research group, (january), pp. 1-15. info~tech 2008b. “use epeat to decrease equipment costs and help the environment”, info~tech research group, (february), pp. 1-3. ipcc 2007. “climate change 2007: climate change impacts, adaptation and vulnerability. summary for policymakers‟, retrieved on 18 december 2009 from http://www1.ipcc.ch/ipccreports/assessments-reports.htm. le matais, j. 1999. “approaches to comparology”, international electronic journal for leadership in learning, (3:15), retrieved on 18 march 2009 from http://www.ucalgary.ca/~iejll/volume3/lemetais.html. australasian journal of information systems volume 16 number 2 2009 22 marshall, p., mckay, j. and prananto, a. 2005. “business value creation from it investments: towards a process theory of it governance“, australasian journal of information systems, (12: 2), pp. 192-206. mathur, l.k., and mathur, i. 2000. “an analysis of the wealth effects of green marketing strategies”, journal of business research, (50:2), pp. 193-200. mcwilliams, a., siegel, d. s. and wright, p. m. 2006. “corporate social responsibility: strategic implications”, journal of management studies, (43:1), pp. 1-18. mia, i. and dutta, s. 2007. “the global information technology report 2006/2007”, world economic forum, geneva. mines, c. and davis, e. 2007. “topic overview: green it”, forrester research, retrieved on 23 june 2008 from http://www.forrester.com/research/document/excerpt/0,7211,43494,00.html. ministry of economy, trade and industry, environmental protection and recycling 2008. retrieved on 24 june 2008 from http://www.meti.go.jp/english/policy/index_environment. mitchell, r. l. 2008. “green by default”, computerworld, retrieved on 23 june 2008 from http://www.computerworld.com/pdfs/lfg_green_it_2008.pdf. molla, a. and licker, p. 2005. “perceived e-readiness factors in e-commerce adoption: an empirical investigation in a developing country”, international journal of electronic commerce, (10:1), pp. 83-110. murugesen, s. 2007. ”going green with it: your responsibility towards environmental sustainability”, cutter business – it strategies executive report, (10:8), pp. 24-33. nfer (n.d.) “desk based research”, retrieved on 18 march 2009 from http://www.nfer.ac.uk/whatwe-offer/nfer-research-methods/desk-based-research.cfm. nunn, s. 2007. “green it: beyond the data centre how it can contribute to the environmental agenda across and beyond the business”, accenture, pp. 17. pettey, c. 2006. “gartner says 50 percent of mid and large sized western european it organisations will develop a green strategy by the end of 2008”, gartner inc., stamford, 29 november. rasmussen, n. 2006. “electrical efficiency modeling for data centers”, retrieved on 23 june 2008 from http://www.apcmedia.com/salestools/nran-66ck3d_r1_en.pdf. rao, p. and holt, d. 2005. “do green supply chains lead to competitiveness and economic performance?”, international journal of operations & production management, (25: 9), pp. 898-916. ridgley, c. 2009. “the decision disconnect”, australasian journal of information systems, (16:1), pp. 185-208. rossi, s. 2007, “australia spends millions each year powering computers”, computerworld australia, 19 september, retrieved on 5 june 2008 from http://www.computerworld.com.au. rmit 2007. “rmit goes a deeper shade of green”, retrieved on 19 december 2009 from http://www.rmit.edu.au/browse;id=vpusc7o147se1. royal society 2007. “climate change controversies: a simple guide”, retrieved on november 2009 from http://royalsociety.org/climate-change-controversies/. snege, p., smith, b., kruschwitz, n., luar, j. and scheley, s. 2008. the necessary revolution: how individuals and organisations are working together to create a sustainable world, new york: doubleday. stern, n. 2008. “the economics of climate change”, american economic review, (98:2), pp. 1-37. australasian journal of information systems volume 16 number 2 2009 23 srivastava, s. k. 2007. “green supply-chain management: a state-of-the-art literature review”, international journal of management reviews, (9:1), pp. 53-80. telstra 2009. “telstra – about us”, retrieved on18 december 2009 from http://www.telstra.com.au/abouttelstra/. velte, t.j., velte, a.t., and elsenpeter, r. 2008. green it: reduce your information system’s environmental impact while adding to the bottom line, new york: mcgraw hill. whitby, p. 2007. “the benefits of green it: the reasons for adopting a greener it policy have become too good to ignore‟, retrieved on 5 june 2008 from http://www.computing.co.uk/computing-business/analysis/2199135/benefits-green-3420872. editorial this is the first issue of volume 2 of the australian journal of information systems (ajis). i am pleased to announce that, over the last 6 months, there has been ongoing interest in the journal, both from subscribers and authors in the area of information systems. i am particularly pleased to mention that as editor of ajis i have been asked to participate in a panel discussion at the upcoming icis conference in vancouver. this would indicate that ajis has become accepted in the international information systems community which is gratifying to all those involved with the journal. in this issue we have contributions from usa, new zealand, uk and australia. our first paper 'forecasting demand using survival modelling: an application to us prisons' by j.r. baker, james madison university, harrisonburg, virginia and p.k. lattimore, national institute of justice, washington , takes a systems approach to modelling population projections in the us prison system. the approach differs i from alternate approaches by capitalising on the impact of recursion in the prison population. this reduces the amount and complexity of the data while maintaining accurate forecasts of population. the next paper 'assessing the graphical and algorithmic structure of hierarchical coloured petri nets' by i g.l. benwell and s.g. macdonell, university of otago, new zealand is concerned with the use and ' understanding of hierarchical coloured petri nets. the authors suggest that the understanding of hierarchical coloured petri nets, in part, depends upon the algorithmic or graphical preference of the user and they develop a . metric which indicates the graphical or algorithmic tendency of a given petri net model. | the next paper 'information systems contribution to business performance: a study of information systems ^ executives' attitudes' by d. fink, the australian national university and f. tjarka, edith cowan university shows the results of a study involving is executives from 200 of australia's largest companies. among the results are findings that suggest that australian is executives are more technology-oriented than business-performancei oriented. m.s. madsen, i.s. fogg and c.l.n. ruggles, university of leicester, uk, contribute a paper entitled metainformation systems: a design study. this describes an approach which combines semantic metadata and i resource metadata to give a more powerful and flexible system than existing global directory systems. t the final paper in this issue is 'a novel interface for information exploration and visualisation' by m.m. north, s.m. north and n.a. warsi, human-computer interaction group, clark atlanta university, usa. this paper presents a graphical user interface prototype called infovis which has been designed to allow users to explore a database without the need for complex syntactical query statements. • once again i would like to pass on my thanks to the authors and the consulting editors for the high quality of i material in this issue of the journal. again i would like to give a special thanks to mr ayman el-ardenli for his help in finalising the journal presentation. r.macgregor editor i the australasian journal of information systems (ajis) began in 1993 and was based on the simple realisation that there was a need for an academic ‘outlet’ for is research in australia australasian journal of information systems volume 16 number 2 nov 2010 5 just in time or just in case: a case study on the impact of context in erp implementations d. v. kerr faculty of business university of the sunshine coast luke houghton department of international business and asian studies griffith university abstract this paper looks at a case study of an enterprise resource planning system (erp) implementation in a geographically dispersed utility company and the approach by middle management to several problem situations involving contextual issues and enduser developed (feral) systems. the paper contends that despite the extremely effective databases and sophisticated modules for business analytic functions within most erp‟s, middle managers are still in the “just in case of an incident” mode of inventory management and data entered in their own feral systems may be significantly different to the inventory levels recorded in the main erp. we contend that these problems point towards the failure of erp systems to be context sensitive to organisations. it highlights problems in contemporary research which appears to lack understanding of different organisational contexts and how they could impact on erp implementations. this paper argues that context and the ideals of erp systems are often mismatched which leads to the development of feral systems, poor inventory practices and erp systems implementation failure. the paper discusses the literature around erp systems implementation and argues that context is an overlooked factor in the analysis of such systems. the paper uses case study evidence to demonstrate mismatched context and highlights the problems with assuming erp systems are one size fits all. the paper concludes with a call to build more contextual research in the study of erp implementations. introduction johns (2006) discusses the problem of context in organisational behaviour by arguing that most studies have disparate findings that appear to be disconnected. that is, studies which should be linked showing how certain factors relate to others, don‟t link and often have completely different findings. he notes that by changing just one contextual variable the outcome of any study is likely to change, actors responses to phenomena shifts as well as the outcome of any theory testing because context can shift the meaning of all other variables in a study. in particular, a study done by two different research teams can have totally different outcomes using the same instrument or theory australasian journal of information systems volume 16 number 2 nov 2010 6 testing device. johns (2006) argues that context plays a very important role in determining how a theory emerges and it is often ignored in organisational behaviour research models. in enterprise resource planning system (erp) research and in particular the study of how large scale information systems effect organisations, there is a surprising lack of contextual analysis undertaken by researchers in the field (quattrone and hooper, 2001 highlight this). there are countless studies on “success factors” (soja 2006, remus 2007, dawson and owens 2008 for example) but very few if any studies on how context effects erp implementations. as the case in this paper (discussed later), context can vary widely depending on: organisational hierarchy, structure, type of organisation (government versus private) and organisational culture. yet, in most studies conducted in erp implementations it is assumed that management hierarchy and organisational fit (for example) are taken as given (tang 2007, shanks 2005, finney and corbett 2007). we argue that such contextual variables should not be ignored in erp implementations and in fact should be explained upfront to readers and practitioners alike. the main argument of this paper is that context should not be ignored in erp implementations because as johns (2006) notes, the slightest change of environmental factors influences the outcomes of any study (ma 2007). others (adler and rips, 2008, yang and terejsen 2007) also highlight problems with context in the broader arena of research settings suggesting that factors are often localised and contextualised. this presents a problem for generalisability for example may (2007) in sociology and medicine and dourish (2004) in computing science. it cannot be assumed that every organisation is the same (garside, 1998, adler and gundersen 2008) because contextual factors such as demography, organisational type, culture and many other aspects make them different. this paper presents one study in which the context of the system mismatched the context of the organisational type and resulted in a community of practice we have called feral systems (houghton and kerr, 2006 and kerr, houghton and burgess 2007). this paper continues with the concept of “feral systems” by placing it in the context of their effects on the efficiency and accuracy of the business intelligence (bi) or business analytics (ba) functions within the erp environment of a large, geographically dispersed utility organisation. kerr, houghton and burgess (2007) discussed the feral systems phenomena and provided some explanations for the reason for their development in terms of power imbalances within the organization. in particular, they have defined feral systems as “an information system [computerised] that is developed by individuals or groups of employees to help them with their work, but is not condoned by management nor is part of the corporation‟s accepted information technology infrastructure. its development is designed to circumvent existing organisational information systems” (kerr, houghton and burgess , 2007 page 142). feral systems are argued to be those mechanisms which circumvent regular systemic procedures to the extent that they create alternative means of accessing data. previous research by the authors has found that feral systems are developed (usually by the managers themselves) to provide information to middle managers about inventory levels in a “just in case” fashion rather than the “just in time” approach advocated by many value or supply chain software modules within the erp environment. this paper concentrates on feral systems and how their development and use with respect to reporting inventory levels could result in a different supply and demand analysis outcome. it is a contention in this paper that the inventory levels in feral systems could be vastly different to the centralized erp levels because of the mismatched context. it is suggested that acknowledgement of these feral systems by upper management may improve the accuracy and effectiveness of ba initiatives within utilcom, (a government owned corporation and the focal firm of the case study) and therefore create an understanding of context and how it impacts on inventory analysis. it is also contended that these (hidden inventory) levels are quite substantial and this is due to the need by middle managers of the assurance of ready accessibility to inventory items in times of an emergency. australasian journal of information systems volume 16 number 2 nov 2010 7 just in time versus just in case: mismatching contexts just in time management refers to the practice of keeping „just enough‟ stock in inventory as is required for the day-to-day management of the enterprise (stalk, 1998). on the other hand the just in case (jic) approach highlights the „just in case‟ practice of keeping an emergency supply „just in case‟ it is required. the two are valid responses to emergent phenomena and are two different contextual factors that are diametrically opposed to each other, just in case being an „at-hand‟ nonlinear emergency perspective versus just in time, being a traditional management concept about inventory practices. in particular, sap is noted for its ability to optimise processes (scheer and haberman, 2000; holstrom 2000) which could be at odds with the context of a just in case approach. soliman and youssef (1998) argue, sap is not just a tool for optimising inventory… it‟s a process driven „reengineering‟ tool, that is often used to create new change processes (martin and cheung, 2000) and move an organisation towards such optimisation on much bigger scale (larsen and myers, 1998). perreira (1999) also argues a similar point stating that sap reengineering process should be used to change organisational culture towards the process optimisation built into sap. this focus on „process efficiency‟ makes assumptions about organisational structure and context (hammer and champy, 1993). it appears that the process minded methodology inherent in the literature and sap also highlights an underpinning lean manufacturing ideal (al-mashari and zairi, 2000). huang et al. (2001) argue that the lean manufacturing ideal (optimising inventory) and the ideals of sap are linked around quality management principles that are underpinned by certain strategic assumptions about how to „do‟ inventory management. sap assumes that all organisations aspire to process efficiency and by doing this, the software naturally excludes the ideals of a just in case approach (noted in the case). managers are often concerned about the effective operations of their section and as such like to have controls (plan b‟s) in place in case there is an emergency. this often leads to tensions between centralized management of inventory and the “just in case” approach many middle managers feel they need in order to give them a comfort zone for emergency situations. such tension presents a mismatched context between what the system ideals are and what the organisational ideals are. due to the increasing numbers of erp implementations and with increasing emphasis on supply chain management and customer relationship management and subsequent data warehousing, there are increasing opportunities for the ba functions within erp systems to be used to produce forecasting and other models to help businesses to make better decisions. (shanks. seddon and willcocks, 2003). for example arnott and pervan (2005 page 71) report that “the business intelligence (bi) movement of the late 1990s changed the direction or emphasis of eis [executive information systems] by focusing on enterprise-wide reporting systems”. the research shown this paper indicates that, with respect to inventory management and forecasting, even if the ba functions within an erp system were used more effectively, their usefulness may not be as good as expected due, in part, to this “just in case” approach to inventory management that appears to be adopted by middle management in some organisations. in other words, the context between what we want from enterprise wide reporting systems, and what we need „just in case‟ an incident occurs can be very different. other research (see locke and lowe 2007, wagner and newell 2004, parr, shanks and darke, 1999, for example) into erp implementations has identified various impacts of erp use and the authors of this paper suggest that some of these factors could be indicators that the feral systems and hence mismatched systemic contexts phenomena may also be prevalent in other companies. for example, rikhardsson and kræmmergaard (2006, page 36) report that a study of six dutch companies revealed, “the impact of es [erp] implementation and use are seldom fully predictable by management. the australasian journal of information systems volume 16 number 2 nov 2010 8 es [erp] can be seen as an organizational actor in its own right; to a large extent, it influences values, culture, behaviour, processes and procedures of other actors in the organization”. this paper contends that the impact described above also affects the way employees try to cope with the changes and that it is very difficult for many employees to suddenly change their values, culture, behaviour, processes and especially their procedures for doing things. therefore, the context of how things operate in the organisation can become threatened by the implementation of a system wide enterprise resource planning system. it is during this difficult adjustment time that the development of feral systems or the continuation of existing feral systems proliferates. the context of one overpowers and attempts to shift the other. other research (see shin, chae, ho and yu 2007 for example) provides some indications of employee dissatisfaction with erp implementations and we contend that this situation of unrest, context mismatch and dissatisfaction is a breeding ground for the proliferation of unmaintained renegade feral systems. for example, although feral systems are not explicitly stated, the research reported by rikhardsson and kræmmergaard (2006, page 41) indicates a level of dissatisfaction amongst employees. statements such as ”all the companies mentioned that the time after going-live was characterized by user insecurity, changes, brush fires, frustration due to errors in system set-up as well as unfamiliarity with the new system” and “some mentioned that the actual challenge was not always to get the software and hardware to work properly but to change employee behaviour and attitudes” leads to the possibility of the development of feral systems and the resultant problems with reporting and data analysis. the research we report in this paper relates to operational management within a geographically dispersed utility company. this difference in perspectives (context) with respect to success is emphasized through the erp implementation in this organisation. the implementation was initiated by accountants within the company and changes in accounting procedures and reporting to activity based costing were a major consideration in the decision to implement the erp in the first place. in other research, gattiker and goodhue (2004) used organisational information processing theory to determine factors that affected the local-level costs and benefits of erp implementations. these researchers found that two organisational characteristic, namely interdependence and differentiation were related to activity coordination and information flows. gattiker and goodhue (2004) also determined that if differentiation between sub units was high then the organisation had to make compromises with respect to the erp implementation. when there are different contexts in existence as is the case here (just in time versus just in case procedures), this shifts the underlying context implicit to work practices. as in many cases these are not explicitly agreed on or known, confused employees are encouraged by circumstance to develop feral systems, as demonstrated in the following case study. the case study organisation has a high level of sub unit differentiation due to their large network infrastructure and hence provides an example of shifting context because of the decision to implement the erp with no modifications and their culture of “just in case” inventory. their use of inventory systems in both a just in time and just in case mode are offered as a method for understanding the different contexts (jit versus jic). this study demonstrates that informal reward structures are in place that encourages jic. the implications to organizations can be enormous from the perspective of inaccuracies in centralized data resulting in reductions in the effectiveness and accuracy of business analytic functions such as data mining and forecasting. the authors wish to highlight that the following case is not indicative of all inventory management systems and neither is it likely to be the only „feral‟ system. the case is australasian journal of information systems volume 16 number 2 nov 2010 9 supplied as an example of this phenomenon in light of recent developments in the sphere of business analysis. more to the point, other feral systems could exist (and indeed the authors have highlighted this in previous work – see houghton and kerr 2006, for example) in the context of this organisation. what follows is a description of inventory management practice as noted in one context. as the authors have reported elsewhere (see above) other feral systems are likely to exist in this particular organisation. the case study the research at utilcom (a pseudonym) was qualitative in nature with interviews with key stakeholders in the supply chain being conducted. the corporation has an annual turnover of over 2 billion dollars (aud) and employs over 13,000 people. utilcom implemented the erp, sap r/3 in 2005. the company was one of the last to change from r/2 to r/3 due in the main to the large investment they had made in terms of modifying the r/2 software to suit their own business processes. it wasn‟t until sap told utilcom that r/2 would no longer be supported that they finally decided to upgrade. utilcom also decided to not to make the same error with respect to r/2, rather they implemented r/3 with the aim of modifying their business processes to suit the software. the implementation was designed to improve reporting and other functions including providing business intelligence capabilities. the modules involved in the implementation were: financial; material management; logistics; forecasting and planning; materials resources planning (mrp); human resources; information systems including executive information systems; project management; and office integration case study methodology and framework this research uses an interpretive case study approach (klein and myers 1999, walsham, 1993). the aim of this research is not to determine factors or derive an explanatory reference model, it is to gain rich insights and perspectives (walsham, 2006). examples of recent case work in information systems based around this approach can be found in beverakis, dick, and cecez-kecmanovic (2009), and shaft, teresa, jordan l and jasperson (2008) for example. the main aim of this study is to gain insights into the social aspects, and specifically what role context plays, in erp systems development. to gain these insights, the authors followed a case study approach as demonstrated in yin (1994) and stake (1995). for evidential sources the researchers used suggestions from the klien and myers (1999) framework described in table 1: source sources used in this case study documentation intranet postings, company annual reports and reports by the information technology section of utilcom participant observation during the three month period at utilcom, one of the authors participated in training and conversed with personnel within utilcom interviews the semi-structured interview approach was used table 1 – evidence and sources for the case study. the research project was conducted as two phases. the first was designed to develop an understanding of the workings of the supply chain and how factors such as social, technical and political factors interacted with each other in order to gauge an understanding of the social setting or context of the implementation. for this reason, 31 semi-structured interviews were undertaken with australasian journal of information systems volume 16 number 2 nov 2010 10 key stakeholders in utilcom from a variety of levels and contexts within the organisation. phase 2 of the research, conducted 12 months later, has 16 semi-structured interviews with identified stakeholders and key experts along the length of utilcom‟s supply chain. these personnel were identified as „expert‟ by the general manager of the supply division of utilcom and by the researchers. results this section is divided into the results from the three data sources shown in table one. documentation the documentation at utilcom portrayed a positive slant to the erp implementation and none of the publicly available documentation mentioned the use of relevant modules of sap for ba. intranet postings were available to all employees at utilcom. there were no negative comments about the sap implementation and no information was given about the possible strategic use of the system. participant observation one of the authors spent three months at the headquarters of utilcom and this time enabled him to observe the implementation process of sap. this period of study allowed the research team to more fully understand the underlying history of the sap implementation. the relationships developed and observations made over the three month period at utilcom enabled the authors to gain an appreciation of the broader context of the erp implementation through the insights of this author‟s involvement. this enabled a degree of relationship building with the actors within utilcom and allowed for a better interpretation of the meanings within the interview transcripts. the participant in the observation is an experienced researcher with an appreciation of both qualitative and quantitative approaches to research within the information systems discipline. he was on academic study leave while undertaking the participant observation during this case study and had some knowledge of the problems of erp implementations at utilcom through his association with a general manager at utilcom. the participant was able to attended meetings and training sessions in the organisation. he was also able to have informal discussions with staff. the meetings covered may aspects of inventory control within the supply (shared services) division of utilcom as well as the upcoming sap r/3 implementation. informal discussions related to concerns about how the help desk could handle “go live” day and the fact that many operational staff were still not trained. other discussion points included workarounds for training and these included repeatedly hitting the return key in order to speed up the training process. this technique appeared to be adopted by many and the training sessions became a non-thinking mechanical process in order to speed up training and allow the worker to get back to work or go home as soon as possible. interviews interviews were semi-structured and open-ended in nature with questions expanding on emerging themes asked as required. the initial 31 interviews (the first stage of the research) were conducted before the implementation of the sap r/3 and the remaining 16 were conducted a year after implementation. coding of the relevant theme was completed by the open coding method (nueman 2003). this involved organizing transcript passages into concepts and emerging themes by looking for a consistent pattern. insights from this first series of 31 interviews led to the following research question: australasian journal of information systems volume 16 number 2 nov 2010 11 how effective is the current erp approach in providing a method for employees to obtain data relevant to their own job requirements? the stage one interviews indicated that many of the staff at utilcom was not fully convinced of the usefulness of the erp implementation. this was emphasised in many occasions during both the first and second stage interviews and led to many “work-a-rounds” by staff. this identification of “worka-rounds” led the authors to the concept of “feral systems” (kerr, houghton and burgess 2007). the second stage of interviews again used the semi-structured approach and expanded on the themes identified in the first stage of interviews. feral systems and business analytics initiatives as mentioned earlier, the feral systems phenomena has been established in earlier research, however the context with respect to inventory management and the possible implications to ba have not been discussed in much detail. the existence of feral systems in utilcom was first established when middle managers were talking about what they referred to as “grass stocks”. that is stocks of inventory parts that were kept in places other than warehouses for the “just in case” emergency situation. records of these grass stocks were usually recorded on spreadsheets or databases entirely separate from the erp system. the case study interview transcripts revealed many cases of these grass stocks or hidden inventory. for example in the second stage of the research in response to the question “so you think that sap has some doubtful figures?” a respondent answered “that’s what i've been led to believe from speaking to various people who are right into sap which is an avenue i don’t pursue but the figures seem to be different in a business warehouse to an sap type figure. queries get run and you get results, the level of confidence just isn't there.” (lower manager – analyst) other comments such as “patterns are not easy to find (no smart tool to find patterns in the data like missing inventory etc.)” and “missing inventory because of „under the counter‟ just in case and “a lot of grass stocks (hidden inventory items not shown in the central inventory)” indicate a misunderstanding or mistrust of the system and the reports it produces. other examples include statements by middle management (an engineer) “… i rely heavily on we’ve got a diary that tracks all material usage on a daily basis so it will have on there how many [of the product] we unloaded today so bruce will come in write in the diary in what section, how many lengths of [product] he does. that diary then goes into a database internally within here and onto a spreadsheet …i do the inventory audits probably once every 3 to 6 months … sort of thing.(construction engineer). this statement indicates a high level of internal information systems (feral systems) are developed and used to either supplement the sap system or replace it. in either case there is no certainty that the information will be recorded centrally in the erp the use of feral systems was further confirmed in stage 2 interviews when the researcher mentioned the feral systems concept, respondents recorded statements such as; “yes that’s a very good point and the people that make these feral systems will have no end of argument about why they are needed and yes there may be justifications but at the end of the day if everyone goes down their own little track with their own systems we are going to be in the same position as we were before we implemented r3” (upper level manager) australasian journal of information systems volume 16 number 2 nov 2010 12 other respondents indicated that feral systems may not be as much of a problem as indicated and that they could be confused with other reporting tools, for example; “certainly we've pulled data out of sap and put it into access databases or excel spreadsheets to do various manipulations so to that extent yes we are doing that but i don't see them as feral systems. i see those as different reporting tools” (upper level manager commodities) on the other hand the same respondent indicated that he could see problems with feral systems in certain cases where there was little or no visibility for the organisation, for example: “at the other end of the equation there are a number of customers who are doing their planning outside of sap so there is no visibility for the organization because their planning exists in excel spreadsheets or access databases and isn't rolling up into any corporate plan so that does create a problem” (upper level manager commodities) and: “(interviewer) the information from [supervisors], how does that information come? is it stored in [enterprise] system?(interviewee) [no] stored in email and excel. but can be referred back to. (interviewer) is that shared with others or not. (interviewee) no not really.[we use] emails and telephones. in the selected quotes above, none of the interviewees perceive a need to get information back into the main erp system for either forecasting or knowledge management purposes. discussion the development of an it strategic plan that includes the use of ba is an essential aspect in ensuring that the technology can deliver the promised strategic and competitive advantage that has been promised by much of the information systems literature. if australian businesses wish to remain competitive in world markets, they need to adopt world‟s best practice with respect to the use of ba approaches. this research may provide an insight into potential problems with ba approaches using centralized erp modules as it appears that the “real data” maybe different to the data reported in the erp system. if this is the case then forecasts and other ba outputs from data mining, on-line analytical processing and other explanatory or predictive models will also be inaccurate. in the case shown in this study, the „grass stocks‟ have been around for quite some time and this has been reflected in forecasts over a substantial period of time. it has been estimated by the general manager of supply division that a great deal of inventory has been hidden and that utilcom has excessive amounts of hidden stocks that are kept away from the central inventory database. the problem is that this has resulted in a continual over ordering of excessive stocks in a “just in case” mode of operation rather than the “just in time” approach that is proposed by ba as being best management practices. arguably, there are two failures of context in this case study that are worth noting. firstly, there is a contextual mismatch between what is expected of the system (just in time) versus what happens through the culture of the organisation (just in case). if employees wilfully hide material from the accepted system (erp), what hope is there for a ba system to provide accurate reports if it is based on the jit ideal? does this kind of phenomena happen in other organisations or is it just the case in this particular organisation? management were not aware of the feral systems that were proliferating under the erp as operators carefully hid the jic approach from them. two different contexts were operating under different strategic assumptions and the effects on the erp systems were marked. australasian journal of information systems volume 16 number 2 nov 2010 13 the lack of context arguably could be placed on the shoulders of the culture of the organisation itself and what erps are designed to achieve. that is, erp systems are designed for process maximisation and efficiency. there are massive differences in processes, procedures, routines and strategic goals. while it is possible that a large percentage of the phenomena discussed in the case study is more than likely the result of a context mismatch between management and operational management ideals, it is also possible that a large amount of the context discussed here is because the erp system ideals do not match those that are found in most public organisations. in essence, a public institution may seek process efficiency but may be unable to due political arrangements, governmental agendas and overwhelming bureaucracy. the question for further analysis and study is: how do government owned corporations fare with the highly structured process methodology of an erp system when in many cases the processes cannot be optimised due to multiple stakeholders and aforementioned government agendas? as johns (2006) argued, the slightest change can influence outcomes and in citing gladwell‟s (2002) tipping point he notes that a handful of influential people can change the way a system behaves. perhaps the underlying mismatch of context, jit versus jic, in this case is to blame for the massive project overruns and failures associated with some erp implementations. perhaps because of this issue, success factors may not apply in some particular contexts because they seek to generalise globally, where it might not even be possible to do so locally, within the context of a single organisation. these contextual problems are implicit, and the resulting systems development process may tease them out revealing a huge mismatch between what is being analysed and what is being developed. however, only more research in this area will be able to uncover the impact of context on erp implementations. limitations yin (1994) notes and eisenhardt and graebner (2007) confirms, data can inducted from a single case for the purposes of generalisation but it‟s better to have a long running sample of cases to see if patterns replicate and expand the general themes found in the single case. in this case study, if there is a failure of context, then which actors, groups and sections played a role in circumventing erp systems? who are the people in the group that are driving the jic paradigm? which management strategies create the context for confusion in erp implementations? is a process optimisations strategy and methodology relevant given the context? this is a limitation of this study because much more needs to be known about the context and how it relates to these methodologies. a lot more work needs to be done here to establish what particular contexts effects the system in which particular way. while this study highlights the important problem of context, it‟s hard to say with any degree of certainty if the results can be interpreted beyond the context shown in this particular case. in information systems research in this area the authors feel that much more work needs to be done in highlighting contextual variables in erp implementations and building models to test and evaluate through case studies before too many conclusions can be drawn. perhaps then, a discussion on the systemic effects could be worthwhile, but for now, this study only highlights several small areas where context effects the implementation of erp systems. conclusions for ba to be effective in an organisation, the fundamental information systems principles of accuracy and robust databases must be in place. many companies spend vast amounts of money in terms of erp implementations to ensure that their databases are accurate and robust enough to allow the australasian journal of information systems volume 16 number 2 nov 2010 14 development and use of strategic tools such as forecasting and other ba principles. however, just because the infrastructure is of a high standard, it does not ensure that the data contained in the database is accurate or contextual. this research indicates that many contextual issues in each erp implementation need to be considered also. issues such as rewards given to employees and mistrust of the centralized erp have resulted in development of systems independent of the erp. we have called these feral systems. an awareness of these systems, the social reason for their creation and the context in which they proliferate is essential for upper management to understand and consider when making strategic decisions. complete trust in the ba coming from the erp may lead to misleading results and errors in forecasting that may have wide ranging implications for the organisation. an appreciation of the contextual factors that lead to the development of feral systems and the resultant inaccuracies in recording true inventory levels can lead to improvements in job training as employees will be able to recognise the “object of the exercise” at more of a macro, whole of firm level. whilst this is not proof of the presence of feral systems in other organisations, there is certainly a great deal of anecdotal evidence to indicate the possibility in many other organisations. other research indicates continual employee dissatisfaction with erp implementations. this is often associated with the decision by companies not to make changes to the erp to fit their existing business processes but a preference to change company processes to fit the erp. these factors, as well as a reward system that still encourages the keeping of excess inventory (jic) lead to the development of feral systems at utilcom and we suspect the situation is similar in many other organisations. while it is accepted that many organisations overcome initial difficulties with erp implementations and, in time are able to take advantage of the full functionality of the system, the authors suggest that with the increasing number of mergers and acquisitions occurring in both the private and public sectors worldwide, the problem will continue well into the foreseeable future. references adler, n.j. & gundersen, a., (2008). international dimensions of organisational behavior, case western reserve university: thomson. adler, j. e. and rips, l. (eds) (2008) reasoning: studies of human inference and its foundations, cambridge university press, uk. al-mashari, m and zairi, m (2000) revisiting bpr: a holistic review of practice and development, international journal of physical distribution & logistics management, vol. 6, no.1, pp. 1032. arnott d and pervan g (2005) a critical analysis of decision support systems research, journal of information technology, vol. 20, pp. 67–87. beverakis g., dick g. and cecez-kecmanovic, d (2009) taking information systems business process outsourcing offshore: the conflict of competition and risk. journal of global information management, vol. 17, no. 1, pp.32-48. dawson, j., owens d. (2008) critical success factors in the chartering phase: a case study of an erp implementation, international journal of enterprise information systems, vol. 4, no. 3, pp. 9-24. davenport, t.h. and brooks, j. d. (2004) enterprise systems and the supply chain, journal of enterprise information management, vol. 17, no. 1, pp.8-19. dourish, p (2004) what we talk about when we talk about context, personal and ubiquitous computing, vol. 8, no.1, pp. 19-30. australasian journal of information systems volume 16 number 2 nov 2010 15 eisenhardt k and graebner m (2007) theory building from cases: opportunities and challenges, academy of management review, vol. 50, no. 1, pp. 25-32. finney, s. corbett, m. (2007) erp implementation: a compilation and analysis of critical success factors, business process management journal, vol. 13, no. 3, pp. 329-347. garside, p (1998) organisational context for quality: lessons from the fields of organisational development and change management, quality in health care, vol. 7, no. 1, pp. 8-15. gattiker t.f. and goodhue d.l. (2004) understanding the local-level costs and benefits of erp through organizational information processing theory, information & management, vol. 41 pages 431–443. gladwell, m. (2000) the tipping point: how little things can make a big difference, little brown, united states. hammer, m. and champy, j. (1993), reengineering the corporation: a manifesto for business revolution, harpercollins, new york, ny. holstrom, j (1998) business process innovation in the supply chain: a case study of implementing vendor managed inventory, european journal of purchasing & supply management, vol. 4, pp. 127-131. houghton, l and kerr d. (2006) a study into the creation of feral information systems as a response to an erp implementation within the supply chain of a large government owned corporation, international journal of internet and enterprise management, vol. 4, no. 2, pp. 135-147 huang, s, kwan, i and hung, y (2001) planning enterprise resources by the use of a reengineering approach to build global logistics management system, industrial management and data systems, vol. 101, no. 9, pp. 483-491. kerr d, houghton l. and burgess k. (2007) power relationships that lead to the development of feral systems, australasian journal of information systems, vol. 14, no.2, pp. 141-152. klein h. k. and myers m.d. (1999) a set of principles for conducting and evaluating interpretive field studies in information systems, mis quarterly, vol. 23, no.1, pp 67-94. larsen, m and myers, m (1999) when success turns into failure: a package-driven business process re-engineering project in the financial services industry, journal of strategic information systems, vol. 8, pp. 395-417. locke, j. and lowe, a. (2007). a biography: fabrications in the life of an erp package. organization, vol. 14, no.6, pp 793-814. ma, h, (2007) process-aware information systems: bridging people and software through process technology, journal of the american society for information science and technology, vol. 58, no.3, pp. 455-456. may c. (2007) the clinical encounter and the problem of context, sociology, vol. 41, no.1, pp. 2945. martin, i and cheung, y (2000) sap and business process re-engineering, business process management, vol. 6, no. 2, pp.113-121. parr, a. n., g. g. shanks, and p. darke (1999). identification of necessary factors for successful implementation of erp systems. in proceedings of the ifip tc8 wg8.2 international working conference on new information technologies in organizational processes, deventer, the netherlands, the netherlands, pp. 99-120. kluwer, b.v. perriera r (1999) resource view theory analysis of sap as a source of competitive advantage in firms, acm sigmis database, vol. 30, no. 4, pp.38-46. neuman, w. l. (2003) social research methods: qualitative and quantitative approaches, 5th edition, allen and bacon, boston, usa. australasian journal of information systems volume 16 number 2 nov 2010 16 quattrone p and hopper t (2001) what does organisational change mean? speculations of a taken for granted category, management accounting research, vol 12, no. 4, p.403-435. remus, u. (2007) critical success factors for implementing enterprise portals: a comparison with erp implementations, business process management journal, vol. 13, no. 4, pp. 538-552 rikhardsson p and kræmmergaard p (2006) identifying the impacts of enterprise system implementation and use: examples from denmark international journal of accounting information systems, vol. 7, pp 36– 49. sap (2008) components and tools of sap netweaver: sap netweaver business intelligence http://www.sap.com/platform/netweaver/components/bi/index.epx (accessed may 31 2008) scheer a and habermann f (2000) making erp a success, communications of the acm, vol. 43, no. 4, pp. 57-61. shaft, m. teresa, a., jordan l., jasperson, j (sean) (2008) managing change in an information systems development organization: understanding developer transitions from a structured to an object-oriented development environment, information systems journal, vol. 18, no. 3, pp.275-297. shanks, g., seddon, p.b and willcocks l (2003) second wave enterprise resource planning systems implementation and effectiveness, cambridge university press. shin, e.j, chae y.m., ho sh, and yu, k. (2007) erp system performance in a hospital, journal of korean society of medical informatics, vol. 13, no. 2, pp. 77-82. soja, p (2006) success factors in erp implementations, journal of enterprise information management, vol. 19, no. 4, pp. 418-433. soliman, f and youssef, m (1998) the role of sap software in business process re-engineering, international of operations and production management, vol. 18, no. 9/10, pp. 886-895. stake, r. (1995). the art of case research. thousand oaks, ca: sage publications. stalk, g. (1998) time: the next competitive advantage, harvard business review, august, 1998. tang, yinghua (2007) erp implementation and critical success factors: a study of shanks erp model on lenovo, nottingham university phd thesis. wagner, e. l. and newell, s. (2004). `best' for whom?: the tension between `best practice' erp packages and diverse epistemic cultures in a university context, the journal of strategic information systems, vol.13, no. 4, pp.305-328. walsham g., (1993) interpreting information systems in organizations, john wiley, chichester. walsham, g. (2006) doing interpretive research. european journal of information systems vol.15, no. 3, pp.320-330. yang, x and terejsen, s. (2007), in search of confidence: context, collaboration, and constraints, asia pacific journal of management, vol. 24, no.4, pp. 497-507. yin r. k. (1994) case study research: design and methods, sage publications london. microsoft word ajisfinal 12 2 marina.doc ajis volume 12 no. 2 may 2005 21 software development methods and tools: a new zealand study chris phillips, elizabeth a. kemp, duncan hedderley institute of information sciences and technology college of sciences massey university palmerston north email: c.phillips@massey.ac.nz, e.kemp@massey.ac.nz, hedderleyd@crop.cri.nz abstract this study is a more detailed follow-up to a preliminary investigation of the practices of software engineers in new zealand. the focus of this study is on the methods and tools used by software developers in their current organisation. the project involved detailed questionnaires being piloted and sent out to several hundred software developers. a central part of the research involved the identification of factors affecting the use and take-up of existing software development tools in the workplace. the full spectrum of tools from fully integrated i-case tools to individual software applications, such as drawing tools was investigated. this paper describes the project and presents the findings. keywords: software development methods, case tools, software engineers, new zealand introduction there are few studies that attempt to model the activities of software engineers. singer et al. (1997) commented on the paucity of studies in this area noting that little effort had been expended in understanding how software engineers work. in regard to new zealand, groves et al. (2000) carried out a survey of software requirement specification practices, but were primarily concerned with the requirements gathering activity. they concluded that there was a wide range of types and organisations engaged in software development. in addition many kinds of applications were being built. whilst this made it difficult to identify meaningful patterns from the data collected, some tentative conclusions were put forward. the most significant factor when determining the practices employed appeared to be the size of the software development group. such groups tended to have better defined development processes, spent more time on requirements capture and had more rigorous testing regimes. this study has a wider focus which includes the software development methods adopted, and the tools and notations used to support these methods. a central part of the research has involved an investigation to identify factors affecting the use and take-up of existing software development tools in the workplace. the full spectrum of tools from fully integrated i-case tools to individual software applications, such as drawing tools, has been investigated, with the purpose of determining what tools are currently being used to support software development activities. the research has been undertaken in two phases: a preliminary study of the practices of software engineers in new zealand. this involved a review of relevant literature on software engineering and case tools, the development and testing of an interview protocol, and structured interviews with five software engineers. kemp, phillips and alam (2003) describe this phase of the project, present an analysis of the data, and examine the results in the context of the theory described in the literature. the results of this study will be summarised later in this paper. a follow-up study involving detailed questionnaires being piloted and sent out to several hundred software developers in new zealand. the focus of this study, which is being reported in this paper, ajis volume 12 no. 2 may 2005 22 has been on the software development methods and tools used by each developer in their current organisation. specific information has been sought on the usefulness and take-up of tools, and on how well i-case tools meet claimed benefits for this type of tool. the goals of the research are improved software engineering processes and methods, better computer-aided software engineering (case) tool support, and improved training of software engineers. close links have been established with major software development companies in new zealand, and part of the research has been conducted at selected sites. this paper continues by briefly reviewing software development methods and case tools. the main results from the preliminary study are then outlined. the research method adopted in this study is next described, and the results presented in detail. following a discussion of the results some conclusions are drawn. the questionnaire used in the research is presented in full in the appendix. software development methods those who write about the discipline of software engineering describe the lifecycle activities that should be undertaken (lending and chervany, 1998; pfleeger, 1998; schach, 1999; sharma and rei, 2000; pressman, 2001; sommerville, 2001). life cycle activities depend upon the software development method followed such as the waterfall, data-centred, prototyping or evolutionary approaches. at a practical level, the activities undertaken may vary depending upon the size and nature of a given project. tool support may also vary, as discussed in the next section. these variables were explored through the questionnaire used in the research. the traditional lifecycle or waterfall model is typically described in a sequential fashion moving from stage to stage involving requirements specification, software design, coding and testing, each requiring a set of deliverables (sommerville, 2001). iterations between phases are required however, to enable revisions to be made whilst an application is under development. in the data centred approach the focus is on the data rather than function and the specific method of entity relationship modelling (sallis et al., 1995). prototyping an application begins with some requirements gathering and moves on to the development of some part of the system. it assumes that the customer can communicate their requirements more effectively once there is something to look at (yourdon, 1994). the developer may undertake prototyping when there are no clearly stated requirements. it is also appropriate when the developer has little experience of the application domain or of the tools and languages that will be used to develop the system (yourdon, 1994). the subsequent prototype can be discarded as in throw away prototyping or retained as the basis for further development as in evolutionary or incremental prototyping (schach, 1999; pressman, 2001). problems can arise if the prototype is taken to be a working version of the application. there are other iterative and evolutionary approaches. evolutionary in this context means that increasingly complete versions of the software are developed (pressman, 2001). one such is the spiral model (boehm, 1988), which uses prototyping to reduce risks as well as incorporating the phases of the classic life cycle (pressman, 2001). another evolutionary approach involves component-based development and is centred round the object-oriented approach. the unified process (jacobson et al., 1999) and rational unified process (kruchten, 1999) are examples of this. currently, there is also a move to using what are termed agile methods (beck, 1999; fowler, 2001). such methodologies “attempt a useful compromise between no process and too much process, providing just enough process to gain a reasonable payoff.” (fowler, 2001, p2). these methodologies emphasise the frequent delivery of software, regular (daily) contact with the customer, and minimising documentation. ajis volume 12 no. 2 may 2005 23 once a life cycle has been selected a software engineer has to make a decision about what notations to use to model and document the system. these make it easier to understand a problem and shape a solution (robillard et al., 2003). a large number of notations are available including entity relationship, data flow and data structure diagrams (jordan and machesky, 1990) and the nine notations specified in the unified modelling language: use case, class, sequence, collaboration, activity, state, component, deployment and package diagrams (booch et al., 1999). robillard et al. (2003) observe that gains in uniformity, reliability and productivity are made when notations are consistent across the various phases of the lifecycle. it is important though that staff have tools to support them and are trained in their use. a further decision relates to the programming language (or languages) used to implement the software. the choice of language will depend upon the nature of the system, and may be constrained by company standards and by the range of tools available to support the language. many systems now incorporate the use of graphical user interfaces and the web, which further extends the range of possibilities. a wide range of programming support environments is available, including integrated development environments (ides), 4gl workbenches, web development tools, and data base management tools. the level of training provided for development staff in methods and tools may impact their effectiveness. a further issue is whether it is better for staff to learn tool use before, during or after they learn the method. these issues will also be explored through the questionnaire. case tools case tools are available which can assist software engineers carry out activities at all stages of the lifecycle (pressman, somerville). at one end of the spectrum there are tools which support one activity and at the other complete environments which support many lifecycle activities. these are referred to as integrated case (icase) tools. a complete taxonomy is provided by pressman (table 1). the benefits of case tools are said to include: a smooth transfer of information, greater productivity, an increase of project control and improved co-ordination. there may though be an initial decrease in productivity when case tools are introduced. there are also tools known as metacase tools which allow you to specify your own case tool (aldersen, 1991) this is to enable a fit between the organisations’ development process and the tool. business process engineering analysis and design process modelling and management pro/sim project planning interface design and development risk analysis prototyping project management programming requirements tracing web development metrics and management integration and testing documentation static analysis system software dynamic analysis quality assurance test management database management client/server testing software configuration management re-engineering table 1: classification of case tools (based on pressman, 2001) in a study by sharma and rei, which had a response rate of 23%, only one seventh (that is 46) of ajis volume 12 no. 2 may 2005 24 those 350 who returned the questionnaire adopted case tools. overall 60% of activities performed by the departments surveyed were supported to some degree by case. there was a very high case tool adoption rate for activities such as the representation of objects, relationships and processes (95% of the respondents) with a much smaller proportion of adopters using case for testing and validation purposes (39%). they found that as the size of the information systems department grew larger, the proportion of adopters was almost twice as large as the non-adopters. in another study, gray et al. (2000) concluded that it was hard to provide evidence of productivity gains or increases in the quality of the product from the use of case tools, and that further research into the practices of software engineers was required in order that more usable tools might be provided. the research being reported here represents a step along this road. organisational issues cost jarzabeck & huang, 1998 vendor support mcchesney & glass, 1993 training mcchesney & glass, 1993 use of appropriate life cycle holt, 1997 compatability misra, 1990 standards iivari, 1996 interface ease of use finnigan et al., 2000 ease of learning lending & chervany, 1998 ease of navigation misra 1990 help and error messages finnigan et al., 2000 reporting and feedback finnigan et al., 2000 people creativity mcchesney & glass, 1993 experience orlikowski, 1993 high expectations aaen, 1994 autonomy lending & chervany, 1998 functionality smooth transition lending & chervany, 1998 complexity of features lending & chervany, 1998 supports multiple users gray et al., 2000 supports re gray et al., 2000 customisation misra, 1990 table 2: factors involved in case adoption there is a large literature on the adoption of case tools which focuses on the factors that play a role in whether they are accepted or not. table 2 shows the main issues involved. these factors are explored through the questionnaire. some key problems relating to the adoption of case tools are as follows: 1. developers with considerable investment of time and experience of traditional software development practices are more reluctant to use case tools than those with less experience (orlikowski, 1993). 2. people do not find case tools easy to use even when happy with the provision of help, navigation facilities etc (finnegan et al). 3. users find notations and editors inadequate for their purpose, seeing them as an obstacle (gray et al., 2000). 4. tools do not support problem solving and creativity (jarzabeck & huan, 1998). 5. intrinsic motivation (tools are fun) and extrinsic motivation (tool is useful) are low ajis volume 12 no. 2 may 2005 25 when using case tools (lending and chervany, 1998). 6. the complexity of case tools is seen as an obstacle to their use by developers (iivari, 1996, gray et al. 2000). lending and chervany (1998) also found the use of case tools to be time-consuming and difficult. 7. training is not often seen as a worthwhile activity by it managers (schwarzkopf et al 2004) who have limited budgets. training is also seen having a limited shelf life (arnold and niederman, 2001). the above difficulties suggest that current case tools may not be providing claimed benefits. the take-up of case tools, including icase, is one of the issues explored through the questionnaire used in the study being reported in this paper. preliminary study this earlier study (kemp, phillips and alam, 2003) was a preliminary investigation of the practices of software engineers within new zealand, with particular reference to their use of development tools. it was decided to employ a qualitative research approach using structured interviews, rather than a quantitative method such as a questionnaire, because of the exploratory nature of the study and the complex processes involved. once the research objectives were determined, the relevant literature on software engineering practice and the use of case tools was reviewed. this was used as the basis for developing the interview protocol which was subsequently revised after testing it out in a pilot interview. the focus was on determining the activities of the developer and other members of the team on a current project. five participants were selected, who it was anticipated, would provide rich data about working as software engineers. all of them were required to be currently developing software and using tools. it was also seen as important to cover different environments rather than interview several developers from an organisation. the people selected were working on quite different kinds of projects and for companies of different types and sizes. results the companies studied varied in size from large to small and worked on projects of various kinds. project planning and management was undertaken by all the developers studied using appropriate software. in 60% of cases this was microsoft project. structured analysis and design, object oriented analysis and design, prototyping and data-centred approaches were used by the developers when building systems. in three cases there was a mix of software development methods used, tailored to the requirements of the organization in all of the organisations well-accepted notations were used on the projects described by the developers to model the system: entity relationship, data flow diagrams, class diagrams, state charts etc. some but not all of these were used on any given project. generally tools were used to draw the diagrams and models but in one case some models were hand-drawn. visio was used as a drawing tool in three of the organisations. case tools were provided for all the developers although only in one organisation was the use of all tools made compulsory. all of the organisations used case tools for representation and analysis purposes but they played a limited role with regard to testing. there were problems though moving from one phase of the life cycle to another, that is integrating the relevant information. because of time pressures, training in the use of case/i-case tools was limited and they were not always used to their full potential. all the developers noted problems for staff in coming to speed with the functionality of the tools. tools could be rejected if they did not seem to meet the perceived need. if a suitable tool was not available then sometimes one was written in house that was tailored to the ajis volume 12 no. 2 may 2005 26 company’s requirements, such as performance tracking software and a defect management tool . with respect to case adoption only two of the firms had purchased an i-case tool (rational rose). in neither case was it used to integrate models but more as a drawing tool. cost, complexity and other organisational issues such as introducing i-case when a team was under pressure to develop systems were seen as obstacles to investment in this area. there was also a concern expressed by one developer that the introduction of such a tool would slow down the production of code. the functionality and usability of i-case tools were also seen as an obstacle. the preference seemed to be for flexible, light-weight tools. despite the various differences reported, all of the participants in this study followed appropriate procedures for developing software. there was no instance where the process was ad hoc. perhaps this was because no one was involved in small projects. many factors appeared to play a part in accounting for the differences reported: the project type, the kind of development, the financial situation of the company, the culture of the organisation, the size of the project, the experience of staff, and the developer’s preference. none of the organisations used an agile methodology (fowler, 2001). this study the findings reported in the above study were the result of only five case studies. exploratory studies of this type are required so that the software engineering community can start to find out how practitioners actually work. however more comprehensive research is seen as necessary. the study being reported in this paper is a more substantial one involving a questionnaire sent to several hundred developers. the focus has been on the software development methods and tools used by developers in their current organisation. specific information has been sought on the background and experience of the participants, on the software development methods and tools being used, on the usefulness and take-up of available tools, and on how well i-case tools meet claimed benefits for this type of tool. finally, through a number of open-ended questions, participants have been asked to present their views on a number of issues relating to current and future software tools. the questionnaire is discussed in detail in the next section. the broad goals of the research are a better understanding of software development methods and tools from the viewpoint of software developers. we also wish to investigate whether there is any relationship between the survey results and the size of organisation, the type of company, the length of project or the type of application developed. it is hoped that the research will lead to improved software engineering processes and methods, better computer-aided software engineering (case) tool support, and improved training of software engineers. research method the study involved detailed questionnaires being piloted and sent out to several hundred software developers in new zealand. the developers were randomly selected from addresses in a business directory. questionnaires are a less flexible medium than interviews, since questions must be fixed in advance. however they can be used to reach a much wider subject group, take less time to administer, and are open to more rigorous analysis (dix, finlay, abowde and beale, 1998). it is vital however that the questionnaire is well designed, and that it gathers relevant and unambiguous data. shneiderman (1998, p132) states that “the keys to successful surveys are clear goals in advance, and then development of focussed items that help to attain those goals”. it is also essential that a questionnaire is piloted prior to being sent out. ajis volume 12 no. 2 may 2005 27 the questionnaire the focus of the questionnaire was on the software development methods and tools used by each developer in their current organisation. the seven-page questionnaire, made up of 25 questions, appears in full in the appendix. it contains a mix of general, open-ended, scalar, multi-choice and ranked questions. prior to being sent out, it was piloted with three local developers, and modified accordingly. section a (questions 1-7) gathered background information on the respondent and their organisation, including job title, length of time with the organisation, the nature of the business, the size of the organisation, the size of the software development section and the types of applications developed. the intention here was to build a picture of the respondent and their working environment. the questions in the main body of the questionnaire (section b) fell into three broad groups. the first group (questions 8-16) asked the respondent to select a recent (or current) software project with which they were involved, and answer questions relating to the project, and the development methods and tools used. in regard to the project, information was gathered on the duration, the size of the project team, and the development methods, modelling notations, and programming languages used. the names of the operating systems used to support the project were also elicited. in regard to tool use, a total of 20 software development life cycle activities under four main categories, were listed on the questionnaire. specific information was sought on the tools used to support each of the life cycle activities, the usefulness of these tools, and whether their use was compulsory. the categories were requirements analysis, design, coding and testing, and project management. respondents could define additional activities in each category. with regard to i-case tools (questions 17-19), information was gathered on how well they met claimed benefits for this type of tool, and how well they met a range of functional and nonfunctional requirements, including those relating to usability. where i-case tools were not being used, respondents were asked to provide any insights as to why they had not been adopted. in a series of open-ended questions to complete the questionnaire (questions 20-25), respondents were asked to state what features and facilities they did not use in current tools, whether they had tried and rejected tools (and for what reasons), what new tools if any were being considered for adoption in their organisation, and what they would like to have in an ideal set of software development support tools. information was also elicited on the amount of formal training on software development tools being provided within their organisation. the results there were 147 responses to the questionnaire which was sent out to 561 people randomly selected from addresses in a business directory. organisations of all sizes were mailed. this gives a response rate of 26.2 % of the sample. 75 of the respondents were involved in the software development, process which is 13.4% of the original sample. overall the 75 respondents had been involved in software development for many years with 73% having had more than 10 years experience and 8% less than 3 years. this was a very experienced group who could provide a great deal of information about current practice. with regard to type of company, nearly 41% of respondents worked for software development firms and an additional 6.5% for consultancies. of the remainder, 12.5 % worked in manufacturing, another 10.5% were employed by organisations specialising in telecommunications and the others from areas such as wholesale, banking, research, education and unspecified organisations. ajis volume 12 no. 2 may 2005 28 background information (questions 4 to 7) over half of the respondents worked in small firms (20 or fewer employees), 9% in medium (21 to 50 employees) and 36% in large firms (greater than 50 employees). seventy six percent of the respondents worked in organisations with small software development/maintenance teams (up to 10 people). only 16% were working in organisations with large software development units (over 50 people). respondents were asked to state what types of applications were most commonly developed in their organisation (table 3). if we consider first and second rankings then management information systems (mis) and transaction processing (tp) systems are most likely to be developed, with embedded and systems software least common. if we consider all rankings for systems developed within an organisation then web based systems are almost as likely to be developed as mis and tp systems. nine companies were only involved in one type of development – 5 of them building mis. for each of real time, web, system and embedded software, there was only one specialist organisation. on the other hand, there were 12 companies involved in building at least 4 kinds of software. mis tp rt web system embedded first and second rankings 38 36 14 23 10 8 all rankings 45 42 24 38 14 14 table 3: types of applications commonly developed project, methods and tools (questions 8 to 16) the respondents were asked to specify the length of the project on which they worked before answering questions on methods and tools. sixty percent worked on a project that lasted between 3 months and 1 year. only a small number worked on a very small (<3 months) or very large project (>3 years). eighty six percent of those who replied to this question worked on projects where the number of participants was less than ten. only 7% worked on projects where more than 20 people were involved. respondents were asked which software development methods were used on the project. they could mention more than one. two people used no formal development approach whatsoever although they explained that some thought went into the process. the other responses indicated that structured, object-oriented analysis and design, and prototyping (more than 40% in each case) were widely used. there were also 6 people using methods not mentioned in the questionnaire such as xtreme programming, functional definitions and rep. table 4 shows the percentage of the 73 respondents using each software development method. structured prototyping objectoriented data-centred other number 33 33 31 16 10 percentage 45.0% 45.0% 42.5% 22.0% 14.0% table 4: use of each software development method ajis volume 12 no. 2 may 2005 29 of the 73 who used development methods, 33 people (45%) used only one approach with 28 choosing from the methods listed. the most favoured single method was object-oriented analysis and design used by 10 respondents. there were 7 people who used each of the structured and prototyping approaches. this left 40 people who used various combinations of methods, 35 of whom (48% of the respondents) used two or more of the listed development methods. the figures for these combinations in order of their popularity are shown in table 5. combination number structured analysis & design/prototyping 16 structured analysis & design/object-oriented 11 data-centred/prototyping 8 structured analysis & design/data-centred 7 object-oriented/prototyping 4 object-oriented/data centred 4 table 5: software development methods used in combination only 66 people answered the question concerning whether tools were introduced before, at the same time or after the software development method. if we ignore the 9 respondents who chose not to answer, then in 43% of cases a tool to support the method followed was introduced first. in another 43% of cases the tool and method were introduced at the same time. only in a small proportion of cases was the tool introduced after the method. respondents were asked about the value of notations used on a scale of 1 to 5 (1 represented not useful and 5 very useful). only erds had a mean above 3.5 which shows a reasonable though not high rating for usefulness (table 6). notation n mean erds 44 3.6 dfds 43 3.3 flowcharts 42 3.3 sequence diagrams 34 3.0 structure charts 38 2.9 detailed uc specs 34 2.8 activity diagrams 32 2.7 class diagrams 37 2.7 state diagrams 32 2.5 use case diagrams 30 2.4 table 6: perceived usefulness of notations overall, erds and dfds were seen as the most useful notations with state diagrams and use case diagrams the least. using tukey’s hsd, the differences between erds on the one hand and class, state and use case diagrams on the other were significant. it should be noted that 35 respondents (47%) used 3 notations or less. several languages were often used on a project but in over 30% of the cases, vb was one of those selected. only the c/c++ family approached the popularity of vb (table 7). ajis volume 12 no. 2 may 2005 30 language number percentage vb 23 30.7% c/c++ 16 21.3% delphi 8 10.7% java 8 10.7% assembler 4 5.3% perl 4 5.3% html 3 4.0% other 39 52.0% table 7: programming language used overwhelmingly the operating system of choice was windows (used by 84% of respondents) although nearly 19% of respondents used unix or an associated version. both of these obviously were used on some projects. with regard to life cycle activities, respondents were asked to check those they carried out and name the support tool used (if any). table 8 shows the number of respondents and percentage of the sample who carried out an activity. it also shows the number who used a tool to support them. percentage* shows the proportion of those performing an activity who used a tool. the most frequently carried out activities were requirements gathering, software coding and software testing. modelling activities were often omitted, many being undertaken by less than 40% of the respondents. in the case of object modelling this figure was as low as 16%. performed by using tool activity number percentage number percentage* requirements gathering 58 77.3% 28 48.3% software coding 55 73.3% 38 69.1% software testing 53 70.7% 29 54.7% software documentation 49 65.3% 32 65.3% user interface design 46 61.3% 26 56.5% project planning/scheduling/review 46 61.3% 33 71.7% data base design 43 57.3% 24 55.8% software design 39 52.0% 16 41.0% user interface prototyping 36 48.0% 23 63.9% design specification 35 46.7% 21 60.0% architecture 32 42.7% 17 53.1% data structure design 32 42.7% 15 46.9% data modelling 28 37.3% 15 53.6% test plan specification 28 37.3% 15 53.6% process modelling 23 30.7% 12 52.2% algorithm/component design 21 28.0% 13 61.9% ajis volume 12 no. 2 may 2005 31 task modelling 18 24.0% 10 55.6% workflow modelling 17 22.7% 9 52.9% object modelling 12 16.0% 8 66.7% capturing of design rationale 6 8.0% 5 83.3% table 8: activities performed and tool support whilst requirements gathering was the activity most frequently performed, less than half of those who carried it out were supported by a tool. some respondents mentioned their use of a whiteboard for such purposes. the only other activity where less than 50% of users were assisted by a tool was data structure design. some activities that were both frequently carried out and supported by a tool included software coding, software documentation and project management. the activity “capturing design rationale” was only carried out by 6 people but tool support was 83.3%. over 60 tools were used by the respondents but those mentioned most frequently are shown in table 9. tool number tool number microsoft word 33 microsoft excel 10 microsoft project 22 rational products 8 vb 14 access 7 visio 13 delphi 7 table 9: most commonly used tools an analysis of the results of this question by phase shows that visio was used most frequently for activities in the software requirements analysis phase, microsoft word for most activities during software design, coding and testing (e.g. for specifying test cases) and vb was the most commonly employed tool for prototyping and coding. ms project was the piece of software most widely used for project management. the respondents were asked to rate the usefulness of different kinds of tools on a scale from 1 to 5 where 1 represented not useful and 5 very useful. more respondents answered this question on the general usefulness of the different kinds of tools than specified their use on the project. table 10, which shows the usefulness of the different categories of tools is sorted by the mean obtained. programming environments were seen as most useful and metacase tools least. other types of tools that scored quite highly (> 3.5) were db management tools, word processors, drawing tools and spreadsheets. case tools (icase and metacase fared badly). the means for programming environments, dbm tools, word processors, drawing tools and spreadsheets were statistically significant (using tukey’s hsd) from those for icase tools, auto testing tools and metacase tools. the “percentage compulsory” figure was worked out by dividing the number who stated that they used a tool of a particular type by the number who stated its usage was compulsory. only in one case – for programming environments (71.4%) was the “use compulsory” figure greater than 50%. there were then only 2 cases where the figure was greater than 40% for db management tools (48.4%) and word processors (45.3%). spreadsheets, drawing tools and project management software were all used by reasonably large numbers but in none of these cases did the percentage compulsory figure reach 24%. of the 73 people who replied to this question, thirty seven people (49%) who worked on projects ajis volume 12 no. 2 may 2005 32 could, but did not have to, use any case tools. only three used none at all. ten people (13%) reported that usage of all the tools they rated for usefulness was compulsory. in no case when tool usage was compulsory did the number of tools used on the project exceed five. finally for the remaining 28 (37%), some tools had to be used whilst others were optional. numbe r mean percentage compulsory prog env 49 4.5 71.4% dbm tools 31 4.3 48.4% word proc's 53 4.1 45.3% drawing tools 40 3.9 20.0% spreadsheets 39 3.9 23.1% prototyping tools 26 3.5 11.5% proj mngmt tools 35 3.4 14.3% web dev tools 21 3.2 28.6% config mngmt tools 20 3.2 25.0% icase tools 20 2.5 25.0% auto testing tools 15 2.3 0.0% metacase tools 10 1.4 0.0% table 10: usefulness of tools overall, people who had to use tools rated the usefulness of tools on average 1 point higher than people who did not have to use tools. this applied across the board, not just for some tools. icase (questions 17 to 19) only ten people were using an icase tool. five of the respondents were using rational rose. no other tool had more than 1 user. these users were asked to rate how well their icase tool met a range of claimed benefits on a scale from 1 to 5 where 1 represents very poor and 5 represents very well (table 11). none of the means are above 3.5 so there was no benefit that received a high rating. the results indicated though that productivity was seen as the most likely benefit with reduction in development costs the least likely. the means for productivity, ease of maintenance, control of process and integration were statistically significant (using tukey’s hsd) from that for the item with the lowest mean – reduced cost. ajis volume 12 no. 2 may 2005 33 number mean productivity 10 3.4 ease of maintenance 8 3.4 control of process 9 3.3 integration 10 3.3 documentation 10 3.3 project communication 8 3.1 reliability 10 3.1 change control 8 3.0 complexity 8 3.0 flexibility 8 2.8 reduced costs 8 2.4 table 11: icase tools: perception of benefits icase users were also asked to rate how well the tool met 23 specified requirements (table 12). the same scale was used as that in the previous question. numb er mean consistency 10 3.7 documentation 10 3.7 support re 10 3.7 reliability 9 3.6 integration/langs 8 3.5 support re-use 10 3.5 transit models 9 3.4 effective editing 10 3.4 smooth navig 9 3.3 efficient working 9 3.3 transit activities 8 3.1 support methods 8 3.1 performance 9 3.1 interface 10 3.1 integration/tools 8 3.0 integration/sw 9 3.0 support group 9 3.0 reporting & feedback 9 2.9 help 9 2.9 vendor support 9 2.9 customisation 7 2.9 ajis volume 12 no. 2 may 2005 34 easy to learn 10 2.8 cost 9 2.6 table 12: icase tools: perception of how well they meet requirements there were six requirements where the mean was equal to or greater than 3.5 – consistency, documentation, support for reverse engineering, high reliability, support for re-use and transition between models. the conclusion can be drawn, therefore, that these requirements on the whole are met by icase tools. this type of tool though was seen as failing to meet the requirement for acceptable cost – with a mean of 2.6 and for ease of use (easy to learn, support, feedback). the means for consistency and documentation were statistically different from that for cost. fifty two people answered the question about why icase tools were not employed in their organisation. some of the respondents gave more than 1 reply. the principal reasons given for not employing icase are shown in table 13. almost half the respondents (46%) saw icase tools as unsuited to current projects, while high cost was seen as a significant factor by 27% of those who replied. nineteen percent saw icase tools as unfit for purpose, and a similar number considered the learning curve to be too great. category number percent of respondents cost (of product, time to make changes, extra lifecycle costs ) 14 27% not required (not needed yet, size or type of project ) 24 46% unsuitable for purpose i.e. not useful (inadequate features, too restrictive) 10 19% time – learning curve, pressure to deliver constraints 10 19% unaware of icase 6 11.5% not integrated with other tools 2 4% requirement for champion 2 4% table 13: icase tools: reasons for non-adoption open ended questions (20 to 25) only twenty two people answered the question about which features or facilities of tools available to them were not used. the name of the tool was not always given and no tool was mentioned more than twice. the respondents rarely specified the facility not used. what is of interest, however, are the principal reasons given by the respondents for not using features of a tool. forty percent of those who replied stated that they did not have the knowledge to use all the appropriate features. for almost 25% there was insufficient time for training. a study by huff et al (1995) confirmed that many users fail to apply the technology to anywhere near its potential. forty one people answered the question about what new software tools, if any, were being considered within the organisation. overall, more than forty different tools were mentioned. seven respondents, a sixth of those who answered this question, mentioned rational rose as being under consideration. visual studio was mentioned almost as many times (by 6 people, that is in a seventh of the cases). otherwise there was little commonality. there was a trend, though, for tools to be under consideration for web related development. some of the languages and tools mentioned in this regard were xml, html, and dreamweaver. thirty four people answered the question about a tool that had been rejected, sometimes ajis volume 12 no. 2 may 2005 35 giving more than one reason for this. a variety of tools were mentioned although the only one to be rejected by more than 2 organisations was rational rose which was no longer employed in four cases. icase tools, it should be noted, were rejected on at least 7 occasions (in about 20% of cases.) the principal reasons given for rejection were: cost (8), unsuitability for purpose (8) and practical issues such as performance and portability (6). twenty six people mentioned one or more features that they would like to see incorporated into tools. a wide range of suggestions was made and there was little overall agreement about what was required. the need for integration was mentioned by seven respondents in total (representing about a quarter of those who answered this question.) and not only in the context of version control. one person requested better integrated regression testing, whilst another was concerned with a feature that provided integrated project and task management. people see the need to integrate tools whilst still rejecting a single integrated tool on the ground of cost, complexity etc. seventy two people answered the question about the days of formal training provided per annum on development tools (table 14). forty percent of the respondents received less than 1 day’s training per year at most with 29.5% percent getting between 1 and 3 days training. the remainder received more than 4 days. large companies with large software development groups tended to provide more training than other company structures. <1 1-3 4-10 >10 number 29 19 16 7 percentage 40.0% 29.5% 11.5% 11.5% table 14: days of formal training per annum fifty nine people answered the question on whether the number of days training per year was adequate. it was clear from an analysis of the answers that they were discussing the effectiveness of the training they received whether informal or not. it was not an issue for five people. of the rest, twenty five replied that the training provided was adequate whilst twenty six thought that it was insufficient. three people stated that it was good in some circumstances but not in others eg “enough for the capable developers but insufficient for new developers.” sixteen people were happy with formal training. for two of the latter, training was seen as an ongoing process eg “this organisation promotes constant learning.” six people, though, were quite scathing about the value of formal training e.g. “i have had no training at all for ten years. the little i did have was a waste of time.” there were also a group of nine people who were satisfied with the informal training they received. “on-project training is more useful.” twenty five percent of those who replied that their training was inadequate cited time constraints as a reason for the lack of training. almost as large a percentage (23%) quoted budgetary constraints. for a little more than 10% the lack of local provider was a problem. this factor was closely associated with cost. three people mentioned that their organisation could not afford the cost involved in sending people overseas to train. for about 8%, their organisation did not realise the necessity for training. if we consider days training in association with the compulsory usage of tools, 13 out of the 30 people who did not have to use any tools had less than 1 days training per year. twenty nine people replied to the last question which asked for any other comments about software development, methods and tools. many factors that had been mentioned earlier, with regard to tools, were re-iterated: their cost, poor interface, time to learning, unsuitability etc. however some other issues were brought up, principally the philosophy of an organisation. forty five percent of those who answered this question, wrote about their underlying philosophy for success when developing ajis volume 12 no. 2 may 2005 36 software. they tended to focus on the issue of process rather than tools with two people specifically stating this was more important eg “i believe we should be concentrating more on processes and documentation than tools”. three others in this group referred to the importance of understanding the business and two to meeting end user requirements. for one respondent, successful communication with users was vital to effective user involvement. three people described the main features of their process with one person mentioning their quality management system. finally there were three respondents who advocated the use of agile processes eg the tool is only a small part of the software development process. a quality, agile sdlc is far more important. other issues mentioned by the respondents included the need for specialized tools, real world constraints on the use of tools and the lack of a design culture. some particularly thoughtful comments were made about real world constraints: “tools – in my experience you need to be very committed to using a case tool and put a lot of effort into using it properly, otherwise it’s just a specialised drawing tool”, and “methodologies are great, but you have to pick and choose and customise, which means many of the supporting tools over or under deliver.” discussion the response rate for this survey (26%) is similar to that in the sharma and rei study (23%). only 75 of the respondents were employed in the software industry and they worked for many different types of organisations both large and small. a large variety of application types were being developed by the companies for which the respondents worked. mis, transaction processing and web systems, though, were each being developed by more than 50% of the sample. given the importance of the web, this finding was not unexpected. overwhelmingly most respondents had been involved in software development for many years. this was a very experienced group who could provide a great deal of information about current practices. seventy three of the respondents compared to 46 in the sharma and rei (2000) study worked on projects where case tools were used. using the chi-square test, we found few associations between the results of the survey and either the size of the organisation, the type of company, the length of the project or the type of applications developed. the strong association (p<.0001) was between sector and size of company/software development team with manufacturing companies tending to be large but with small software development groups and telecoms companies tending to be large with large software development groups. software development companies, however, were usually on the small side. the more people were involved on the project, irrespective of project size, the more likely icase was to be used. generally this involved large companies with large software development groups (although the data set was too small to be statistically significant). such companies also tend to provide more training (p=.0026). groves et al. indicated that the size of the software development group seems of some importance when determining practices. here the evidence suggests that this was true with regard to investment in icase and training. otherwise methods on projects, project lengths etc tended to be similar.. structured systems analysis and design (usually associated with the waterfall model) and prototyping were the most commonly used development approaches followed closely by objectoriented analysis and design. given the greater longevity of the first two of these, it was interesting to note the wide use of the oo paradigm. it should also be noted that more than one method was employed on a large number of projects. the same mix of results was found in the preliminary study (kemp et al. 2003). as gray (2000) claimed, notations and conceptual modelling do not appear to be seen as especially useful, with none used by more than 60% of the sample. in fact merely 4 of the 10 notations were ajis volume 12 no. 2 may 2005 37 used by more than 50% of the respondents and in almost 50% of cases 3 notations or less were used on a project. only erds, a notation of long standing, had a mean above 3.5 (which can be seen to represent somewhat useful). surprisingly, given that 40% of the respondents stated that they used an object oriented analysis and design approach, class, state and use case diagrams all had low means with the difference being statistically significant. sequence diagrams though were seen as more useful than the other uml notations. with regard to activities and tools used to support them, it should be noted that there was no activity performed on 100 % of projects. one reason for this relates to the fact that not all organisations involved in the survey were in the business of developing the complete product. some companies for example were involved with requirements gathering but not coding. the activity that was most frequently performed was that of requirements gathering followed by software coding and testing both of which were carried out on more than 70% of the selected projects. modelling activities fared badly overall coming 13, 15, 17, 18 and 19 (out of 20) in the table. object modelling, carried out on only 16% of the projects, scored very poorly. this is consistent though with the poor opinion of class diagrams as a notation. there were only two activities that were widely performed where the use of a tool was compulsory, software coding and project planning activities. the lack of compulsion was something of a surprise. for instance, in only 20% of cases, was the use of drawing tools made compulsory. this fits in of course with the lack of interest in modelling activities. design activities fared somewhat better. with regard to the perceived usefulness of case tools, more respondents evaluated tools than used them on the project referenced. this could be for two reasons. firstly, they did not write down all the tools that they used on a project. secondly they are accustomed to using tools of this kind on other projects and felt that they could evaluate their usefulness. the discrepancy between the totals for usage is particularly marked for spreadsheet usage . at a detailed level, three kinds of tools (programming environments, db management tools and word processors) had a mean greater than 4 indicating that these were seen as really useful. icase and metacase, the most complex tools, performed particularly badly. it was interesting that in less than half of the cases tools to support new methodologies were introduced before the methodology was used. it was as likely for both tools and methodologies to be introduced together. it is not surprising given that staff might be struggling to learn a methodology as well as a tool that that many tools were not perceived as useful. when considering the tools overall, the percentage using each one was not very high with only 4 used by more than 50% of the sample. nonetheless, on all but 2 projects a case tool was used. the results cannot confirm or refute orlikowski’s contention that developers with considerable investment of time and experience of traditional software development practices are more reluctant to use case tools than those with less experience (orlikowski, 1993). the results do indicate though that a large group (83%) of predominantly experienced developers chose to use one or more case tools. those people who were made to use a tool rated them on average 1 point higher than people who did not have to. this applied to all tools and suggests that once people have to use the tool then they perceive its benefits. lending and chervany (1998) believe that extrinsic motivation (tool is useful) is low when using case tools, but perhaps when compulsion is taken into account the level of extrinsic motivation increases. this may be because people have to come to grips with features that they otherwise would not use and appreciate their value. it may also be because people who have to use tools are given some training. the data in the questionnaire on training did not relate to specific tools so could it was not possible to determine if there was any effect of this kind. it is interesting to consider the value of a tool to an organisation. some tools are clearly very specialised eg microsoft project whilst others such as rational rose and oracle (icase) tools are ajis volume 12 no. 2 may 2005 38 meant to be used at several stages of the life cycle. there were also two tools, visio and word, that were used for several activities. many readers might not consider word to be a typical case tool but it was included by several respondents. in particular it was seen as useful for specifying test cases. as in the preliminary study icase tools were not widely used. in this study they were only used by 10 people. most (but not all) respondents were aware of what they were. forty eight people were able to explain why icase was currently not used in their organisation or on the project (not required, unsuitable, time constraints). icase tools were also being considered by 7 more organisations. given that no benefit had a mean above 3.5 then clearly no-one perceived icase tools as having any outstanding benefits. productivity which is often mentioned (pressman, 2001) only had a mean of 3.4. the use of icase was not seen as reducing development costs the situation was somewhat improved with regards to icase tools meeting requirements. four requirements had a mean higher than 3.5: consistency, documentation, support for reverse engineering and reliability. an acceptable cost, however, scored very poorly with a mean of 2.6. if the requirements are divided into five groups – those related to artefacts, support, performance integration, and human computer interaction (hci), and the means of each group compared, it can be seen that there was no mean greater than 3.5. the mean for the hci group of requirements is the lowest, thus indicating usability of the system was not found to be satisfactory (finnegan et al, 2000) (figure 1 and table 15). 2.7 2.8 2.9 3 3.1 3.2 3.3 3.4 3.5 artefacts support perfomance integration hci m e a n figure 1: icase tools: analysis of requirements by category ajis volume 12 no. 2 may 2005 39 group name components artefacts consistency of work products effective drawing/editing of diagrams good documentation of work products smooth navigation of diagrams smooth transition between activities smooth transition between models hci customisation easy to learn efficient working interface help interface reporting & feedback integration integration with other software integration with programming languages integration with tools support support for multiple methods support for reverse engineering support for re-use support for group working performance issues performance reliability vendor support note cost was not included in any of these categories. table 15: icase tools: category components with regard to case tool acquisition and rejection overall, the picture which emerges is a dynamic one. despite criticisms of tools, many organisations (40) are investigating the purchase of further case tools. given the fact that more than half of the companies were developing web-based systems, it was no surprise that several of the tools under consideration were specifically for building web applications. on the other hand tools were also being discarded. in 34 organisations previously purchased tools were being rejected for reasons of cost or unsuitability. when people were asked to state why they did not use features or facilities in tools that were available to them, the principle reason given was lack of knowledge. ignorance of the capabilities of tools probably relates to their complexity (iivari, 1996 and gray et al, 2000). shortage of time was also reported as an obstacle to people improving their skills (lending and chervany, 1998). when considering features that could be incorporated into tools, the main request was for better integration. it appeared that people see the need to integrate tools whilst still rejecting a single integrated tool on the basis of cost. forty percent of the respondents received less than 1 day of formal training per year. with regard to training in tool use, cost (schwarzkopf, 2004) and time constraints were again mentioned as the main reason for the failure to deliver appropriate training. as a result, practitioners may not be using tools to their full potential (huff, 1995). the issues of cost, complexity and time (jarzabeck & huang, 1998) emerged as key complaints in ajis volume 12 no. 2 may 2005 40 the preliminary study. in this questionnaire, cost is clearly a key reason for not purchasing icase tool as well as for the failure to provide training in case tools of all kinds. given the small size of many of the software development companies this is not surprising. the lack of time and complexity were also important factors. the appropriateness of a tool though for its intended purpose also emerged as an issue in this questionnaire. this relates to a tool not meeting requirements i.e. being too cumbersome/restrictive. the last question, which was open ended, asked people to add any further comments. a surprisingly large percentage of the 29 respondents (45%) considered their philosophical approach to software development, the issue of process, to be more important for software development than the tools used. this attitude may well relate not only to the complexity and unsuitability of tools but also to their lack of support for problem solving( jarzabeck and huang, 1998 ). there could also be, overall, little intrinsic and extrinsic motivation as hypothesised by lending & chervany (1998). the value of tools may only be fully appreciated if people are fully trained in their use and have to use them. conclusions the work reported here is a follow-up study to an earlier investigation into the practices of software engineers in new zealand. the study involved a detailed questionnaire being piloted and sent out to several hundred software developers, and focussed on the software development methods and tools used by each developer in their current organisation. specific information was sought on the usefulness and take-up of tools, and on how well i-case tools meet claimed benefits for this type of tool. most of the respondents had been involved in software development for many years, and were an experienced group who could provide significant insights into current practices. few associations were found between the size of the organisation, the type of company, the length of the project or the type of applications developed. it appeared that developers see the need to integrate tools whilst still rejecting a single integrated tool on the basis of cost and complexity icase and metacase, the most complex tools, scored particularly badly for usability. notations and conceptual modelling do not appear to be seen as especially useful. despite criticisms of tools however, many organisations are investigating the purchase of further tools. a significant finding was that close to half the respondents who answered the final open-ended question, considered the issue of process to be more important for software development than the tools used. through the investigation, links have been established with software development companies in new zealand, and part of the research was conducted at selected sites. it is hoped that this research will make a contribution to improved software engineering processes and methods, better case tool support, and improved training of software engineers. through university teaching being conducted by the authors, the findings of this study are also being made available to software engineering students – tomorrow’s practitioners for debate and study. ajis volume 12 no. 2 may 2005 41 acknowledgement the authors gratefully acknowledge funding from the new zealand foundation for research, science and technology to carry out part of this research. references aaen, i. (1994) problems in case introduction: experiences from user organisations, information and software technology, 36, 11, pp 634-654. arnold, d. & niederman, f. ( 2001)the global it work force. communications of the acm 44, 7, pp 3133 beck, k. (1999) extreme programming explained: embrace change, reading, ma: addisonwesley. boehm, b (1988) a spiral model for software development and enhancement, computer, vol. 21, 5, pp 61-72. booch, g., rumbaugh, j. & jacobson, i. (1999) unified modeling language user guide, reading mass: addison-wesley. dix, a., finlay, j., abowd, g., & beale, r. (1998): human-computer interaction, prentice hall, hemel hempstead. finnigan, d., kemp, e. & mehandjiska, d. (2000) towards an ideal case tool, smt 2000, wollongong, ieee computer society press, pp 189-197. fowler, m. (2001) the new methodology, http://www.martinfowler.com/articles/newmethodology.html, updated nov 2001. gray, j.p., liu, a. & scott, l. (2000) issues in software engineering tool construction. information and software technology. 42, pp 73-77. groves, l., nickson, r., reeve, g., reeves, s. & utting, m. (2000) a survey of software requirements specification practices in the new zealand software industry, proceedings aswec 2000: australian software engineering conference 2000. (ed.) d. d. grant. ieee computer society. pp 189-201. holt, j. (1997) current practice in software engineering: a survey, computing and control engineering journal, 8 (4), pp 167– 172. huff, s.l., munro, m.c., marcolin, b.l. & compeau, d.r. (1995): understanding and measuring end user sophistication, proc. of 14 th new zealand computer conference, wellington, 23-25 august. iivari, j. (1996) why are case tools not used?, 39, 10, pp 94-103. jacobson, i., booch, g. & rumbaugh, j. (1999) the unified software development process, reading, mass : addison-wesley. jarzabek, s. & huang, r. (1998) the case for user-centered case tools, comms of the acm, 41, 8, pp 93-99. jordan, e.w. & machesky, j.j. (1990) systems development: requirements, evaluation, design, and implementation, boston: pws-kent publishing company. kemp, e.a., phillips, c.h.e. & alam, j. (2003): software engineering practices and tool support: an exploratory study in new zealand, australian journal of information systems, vol 11 no 1, 37-54. kruchten, p. (1999) the rational unified process, an introduction (2nd edition), reading, mass: addison-wesley. lending, d. & chervany, n.l. (1998) case tools: understanding the reasons for non-use, computer personnel, april 1998, pp 13-24. ajis volume 12 no. 2 may 2005 42 mcchesney, i.r. & glass, d. (1993) post-implementation management of case methodology, european journal of information systems, 2, 3, pp 201-209. misra, s.k. (1990) analyzing case system characteristics: evaluative framework, information & software technology, 32, 6, pp 415-422. orlikowski, w. j. (1993) case tools as organizational change: investigating incremental and radical changes in systems development, misq, vol. 17, no.3, pp 309-340. pressman, r. s (2001) software engineering: a practitioner’s approach, 5 th ed., singapore: mcgraw-hill. robillard, p., d’astous, p., & kruchten, p. ( 2003) software engineering process with the upedu, boston: pearson education. sallis p., tate, g., & macdonnell, s. (1995) software engineering practice, management, improvement, sydney: addison-wesley. schach, s. (1999) classical and object-oriented software engineering, new york: mcgrawhill. scott, c.a., clayton, j.e. and gibson, e.l. (1991) a practical guide to knowledge acquisition, reading, mass: addison-wesley. schwarzkopf, a., mejias, r., jasperson, j., saunders, c., & gruenwald h. (2004) effective practices for it skills staffing, communications of the acm, 47, 1, pp 83-88. sharma, s. & rei, a. (2000) case deployment in is organisations, communications of the acm, 43, 1, pp 80-88. shneiderman, b. (1998): designing the user interface: strategies for effective human computer interaction, (3rd ed.). addison-wesley, reading, ma. singer, j., lethbridge, t., vinson, n. & anquetil, n. (1997) an examination of software engineering work practices, proceedings of cascon '97, toronto, pp 209-223. sommerville, i. (2001) software engineering, reading, mass: pearson. yourdon, e (1994): object-orientated systems design, usa: prentice hall international editions. ajis volume 12 no. 2 may 2005 43 appendix questionnaire software development methods and tools note: the focus is on the software development methods and tools which are used in your present organisation. the questionnaire is in two parts, and should take about 20 minutes to complete. a. background information 1. what is your job title? _______________________________________ 2. how long have you been with your present organisation? <1 year 1-3 years 4-10 years >10 years 3. how long have you been involved with software development? <1 year 1-3 years 4-10 years >10 years 4. how is your organisation best described? software development telecommunications consulting banking/finance/insurance wholesale/retail trade manufacturing education public service/local government health central government research military other (please specify) __________________________________________ 5. how many people are employed in your organisation? <5 5-20 21-50 51-150 >150 6. how many people are engaged in software development/maintenance? <3 3-10 11-20 21-50 >50 7. what type(s) of applications are developed in your organisation? (please rank: 1 most common, 2 next….., leave blank if not applicable) management information systems (e.g. decision support) transaction processing systems (e.g. payroll, pos, accounting, inventory) real time applications (e.g. process control, manufacturing) e-commerce/web applications embedded systems (e.g. software running in consumer devices or vehicles) systems software (e.g. telecommunications software) other (please specify) __________________________________________ ajis volume 12 no. 2 may 2005 44 b. software development methods & tools choose a recent (or current) software project in your present organisation, with which you were involved, and answer questions 8-16 in relation to this project. 8. what was the length of the project (from inception to an operational system)? please estimate if the project is a current one. <3 months 3-6 months 7-12 months 1-3 years >3 years 9. how many people were involved with the project? <3 3-9 10-20 >20 10. which software development methods were used on the project? you may check more than one box. structured analysis and design object-oriented analysis and design  data-centred design  prototyping other (please specify) __________________________________________ ____________________________________________________________ 11. in relation to the principal method used on the project, were the tools which support it introduced: before the method?  at the same time as the method?  after the method?  12. for each modelling notation used on the project, indicate on the following scale how useful it was: not useful………………….very useful 1 2 3 4 5 entity-relationship diagrams (erds)      data flow diagrams (dfds)      structure charts      class diagrams      use case diagrams      detailed use case specifications      activity diagrams      state diagrams      flowcharts      sequence diagrams      other (please specify):____________________      ____________________________________      ajis volume 12 no. 2 may 2005 45 13. list the major programming languages used on the project: ______________________________________________________________ ______________________________________________________________ 14. list the major operating systems used to support the project: ______________________________________________________________ ______________________________________________________________ 15. indicate by checking the boxes which of the following software development life cycle activities were undertaken on the project. enter the name(s) of any software development tool(s) used to support each activity. activity development tool(s) used software requirements analysis requirements gathering   task modelling   process modelling   object modelling   data modelling   workflow modelling   user interface prototyping   test plan specification   other (specify) ________________   ___________________________   software design architecture   algorithm/component design   data structure design   data base design   user interface design   capturing of design rationale   design specification   other (specify) ________________   ___________________________   software coding & testing software coding   software testing   software documentation   other (specify) ________________   ___________________________   project management project planning/scheduling/review   other (specify) ________________   ajis volume 12 no. 2 may 2005 46 16. for each of the following tools used on the project, indicate on the scale how useful it was, and check the final box if its use was compulsory. not useful………..very useful compulsory 1 2 3 4 5 integrated case tools       metacase tools       web development tools       drawing tools       database management tools       prototyping tools       programming environments       automated testing tools       project management tools       configuration management tools       word processors       spreadsheets       other (please specify):_______________       _______________________________       _______________________________       _______________________________       questions 17-25 are more general questions relating to tool use in your present organisation if integrated case (icase) tools (e.g rational rose, oracle) are used, please answer questions 17 and 18. if not, please proceed to question 19. 17. indicate on the following scale how well the icase tool used in your organisation meets each of the following claimed benefits for this type of development tool. please name the tool. icase tool: ___________________ very poorly………………….very well 1 2 3 4 5 improved software reliability      improved productivity      greater control of development process      easier software maintenance      greater integration of system models      better system documentation      improved project communication      better change control      greater flexibility      reduced complexity      reduced development costs      ajis volume 12 no. 2 may 2005 47 18. indicate on the following scale, how well the above icase tool meets the following requirements: very poorly……….….……... very well 1 2 3 4 5 smooth transition between activities      smooth transition between models      effective drawing/editing of diagrams      smooth navigation of diagrams      consistency of work products*      good documentation of work products*      integration with other tools      integration with programming languages      integration with other software      support for group working      support for re-use      support for reverse engineering      support for multiple methods      ease of learning      good reporting and feedback      good help and error messages      good customisation facilities      efficient working      good user interface      good performance      high reliability      acceptable cost      good vendor support      *work products refers to any artefacts produced along the way (models, documents, code etc): 19. if your organisation does not employ icase tools, please briefly indicate why not: ______________________________________________________________ ______________________________________________________________ ______________________________________________________________ ______________________________________________________________ ______________________________________________________________ ______________________________________________________________ ______________________________________________________________ ajis volume 12 no. 2 may 2005 48 20. are there any major features and/or facilities in the current software development tools in your organisation which you do not use? briefly indicate why. ______________________________________________________________ ______________________________________________________________ ______________________________________________________________ ______________________________________________________________ ______________________________________________________________ ______________________________________________________________ 21. have any software development tools been tried and rejected within your organisation? please list with brief reasons for rejection: ______________________________________________________________ ______________________________________________________________ ______________________________________________________________ ______________________________________________________________ ______________________________________________________________ ______________________________________________________________ 22. what new software development tools, if any, are being considered within your organisation at present? ______________________________________________________________ ______________________________________________________________ ______________________________________________________________ ______________________________________________________________ ______________________________________________________________ ______________________________________________________________ ______________________________________________________________ 23. what features and/or facilities would you like to see in future software development tools which are not provided in current tools? ______________________________________________________________ ______________________________________________________________ ______________________________________________________________ ______________________________________________________________ ______________________________________________________________ ______________________________________________________________ ajis volume 12 no. 2 may 2005 49 24. approximately how many days of formal training per annum in software development tools are provided for each developer in your organisation? <1 day 1-3 days 4-10 days >10 days do you consider this training to be adequate? please comment. ______________________________________________________________ ______________________________________________________________ ______________________________________________________________ ______________________________________________________________ ______________________________________________________________ 25. please add any other comments relating to software development methods and tools (continue on reverse side if necessary): ______________________________________________________________ ______________________________________________________________ ______________________________________________________________ ______________________________________________________________ ______________________________________________________________ ______________________________________________________________ ______________________________________________________________ ______________________________________________________________ ______________________________________________________________ ______________________________________________________________ microsoft word ajis special 2002.doc ajis special issue december 2002 76 guidelines for conducting positivist case study research in information systems graeme shanks department of information systems the university of melbourne g.shanks@dis.unimelb.edu.au abstract the case study research approach is widely used in a number of different ways within the information systems community. this paper focuses on positivist, deductive case study research in information systems. it provides clear definitions of important concepts in positivist case study research and illustrates these with an example research study. a critical analysis of the conduct and outcomes of two recently published positivist case studies is reported. one is a multiple case study that validated concepts in a framework for viewpoint development in requirements definition. the other is a single case study that examined the role of social enablers in enterprise resource planning systems implementation. a number of guidelines for successfully undertaking positivist case study research are identified including developing a clear understanding of key concepts and assumptions within the positivist paradigm; providing clear and unambiguous definitions of the units and interactions when using any theory; carefully defining the boundary of the theory used in the case study; using hypotheses rather than propositions in the empirical testing of theory; using fuzzy or probabilistic propositions in recognising that reality can never be perfectly known; selecting case studies carefully, particularly single case studies; and recognising that generalisation from positivist, single case studies is inherently different from generalisation from single experiments. when properly undertaken, positivist, deductive case study research is a valuable research approach for information systems researchers, particularly when used within pluralist research programs that use a number of different research approaches from different paradigms. introduction the case study research approach is widely used within the information systems community. case studies can be undertaken within a positivist or interpretivist paradigm, may be deductive or inductive, may involve single or multiple cases using literal or theoretical replication and may use qualitative and quantitative data (cavaye, 1996). this paper focuses on positivist, deductive case study research in information systems. positivist, deductive case study research is well documented, with methodological guidelines published by benbasat, goldstein and mead (1987), lee (1989) and yin (1994). furthermore, key concepts and assumptions within the positivist paradigm are also well documented (for example see guba and lincoln (1994) and neuman (2000)). two recently published research studies based on a strictly positivist case study research approach provide useful exemplars of this type of research for the information systems community (darke, 1997; sarker and lee, 2000). the first is a multiple, positivist case study that validated concepts in a framework for viewpoint development in requirements definition (darke, 1997). the second is a single, positivist case study that examined the role of social enablers in enterprise resource planning (erp) systems implementation (sarker and lee, 2000). both studies are reported in a rigorous manner including proposition formulation, deductive testing of the propositions and consideration of validity and reliability. this paper first discusses the positivist research paradigm and presents precise definitions for key concepts including theory, proposition, hypothesis and hypothesis testing, and uses examples from darke (1997) to illustrate the concepts. the following section discusses positivist case studies and how they may be conducted. the third section presents a description of the process and outcomes of darke’s study. the following section presents a description of sarker and lee’s study, particularly focusing on their use of the outcomes of the study on critical success factors for erp systems implementation by parr et al, (1999) and how they were used in the development of propositions. the next section presents several guidelines for successfully undertaking positivist case study research based on a critical analysis of the two studies. the paper concludes with a discussion of the place for positivist case studies in information systems. the positivist paradigm: theories, propositions, hypotheses and hypothesis testing a paradigm is a set of beliefs about the nature of social reality, that is, the nature of the “world” and the individual’s place in it (guba and lincoln, 1994; shanks, rouse and arnott, 1993). guba and lincoln note that a paradigm has three dimensions: • what is the form and nature of reality (the ontological question)? • what is the relationship between the researcher and what can be known (the epistemological question)? • how can the researcher find out if whatever they believe can be known (the methodological question)? paradigms are human constructions that are neither right nor wrong. they should be understood as assumptions that are not subject to proof: proponents must argue for their utility (guba and lincoln, 1994). the positivist paradigm has the following positions with regard to the three dimensions (guba and lincoln, 1994; shanks, rouse and arnott, 1993): ajis special issue december 2002 77 • an objective reality is assumed which can be systematically and rationally investigated through empirical investigation, and is driven by general causal laws that apply, in particular, to social behaviour. this is sometimes called naïve realism (the ontological position). • the researcher and the phenomena being investigated are assumed to be independent, and the researcher remains detached, neutral and objective. any reduction in independence is a threat to the validity of the study, and should be reduced by following prescribed procedures (the epistemological position). • general theories are used to generate propositions that are operationalised as hypotheses and subjected to empirical testing that is replicable. hypotheses should be testable and provide the opportunity for confirmation and falsification. this is the essence of the scientific method (the methodological position). in the following discussion about theory, proposition, hypothesis and hypothesis testing we assume a positivist position. theories a theory is a system of ideas that abstracts and organises knowledge about the social world (neuman, 2000). there are many types of theory including implicit (preconceptions, biases and values etc.) and explicit theory (sets of organised concepts and their interrelationships) (miles and huberman, 1994). there are highly abstract theoretical frameworks, and focused mid-range theories more suited to empirical work (neuman, 2000). for empirical studies conducted using a positivist, deductive case approach mid-range, explicit theories are relevant. a theory of this type has three main elements (dubin, 1978; bacharach, 1989). 1) a set of well-defined units (or concepts); 2) laws of interaction (or interrelationships between the units); 3) a boundary within which the theory holds. dubin (1978) provides a full discussion of these elements, but for the purposes of this paper an example will be used to explain their meaning. darke (1997) developed a theoretical framework for understanding viewpoint development within requirements definition that comprises four units, seven laws of interaction and a clearly articulated boundary. i will use part of this theoretical framework to explain dubin’s elements of a theory. two units of the theoretical framework are “viewpoint development role” and “viewpoint representation”. each of these units is an enumerative unit, that is, a property characteristic of viewpoint development that must always be present. further, they are attributive enumerative units as they are properties characteristic of viewpoint development in all conditions. the values of both units are used to identify types of that unit. darke (1997) defines three values for viewpoint development roles: supporting requirements acquisition, supporting requirements modelling or supporting both phases of the requirements definition process, and three values for viewpoint representations: formal, semi-formal and informal. laws of interaction may be associational (the categorical interactions of dubin (1978)), sequential (the sequential interactions of dubin (1978)), and causal (the determinant interactions of dubin (1978)). these three forms of interaction build on each other, so the associational laws are the most basic form of interaction. a stronger form of interaction is sequential (which subsumes associational), and the strongest from is causal (which subsumes sequential). furthermore, causal interactions may be necessary or necessary and sufficient. darke (1997) identifies a categorical, bi-directional law of interaction between the units “viewpoint development role” and “viewpoint representation”. the law of interaction is “different viewpoint representations tend to be used during different viewpoint development roles”. the two units and the law of interaction may be represented using a diagram as shown in figure 1. figure 1: (part of) the viewpoint development theoretical framework (darke, 1997, p111) the boundary of the theoretical framework specifies the portion of the world within which the units and laws of interaction hold. darke (1997) specifies the boundary as viewpoint development within the domain of requirements definition. propositions predictions about the world are made using propositions, that is, conclusions that may be deduced logically from the theory. propositions link the values of units. propositions in the viewpoint development theoretical viewpoint development role viewpoint representation tend to be used during ajis special issue december 2002 78 framework will therefore link specific values of viewpoint representation with specific values of viewpoint development role. dubin (1978) notes that the most usual form of propositions is the “if … then …” format. one proposition identified by darke (1997) that relates to the units shown above in figure 1 is (adapted to the form suggested by dubin): • if representation techniques are informal or semi-formal then they tend to be used during the requirements acquisition viewpoint development role. hypotheses a hypothesis is an empirically testable statement that is generated from a proposition. terms in propositions belong to the abstract world of theory. each of the terms must be assigned an empirical indicator. these empirical indicators are then substituted into the proposition to form a corresponding hypothesis. once hypotheses have been generated they may be used in empirical studies. empirical indicators for informal representation techniques are text, rich pictures and animations. empirical indicators for semi-formal representation techniques include data flow diagrams (dfds), unified modelling language (uml) diagrams and entity relationship (er) diagrams. empirical indicators for requirements acquisition include elicitation, understanding and elaboration of information system requirements. hence, a hypothesis (h1) generated from the first proposition and expressed in a form suitable for refutation is: h1 text, rich pictures, animation, dfds, uml diagrams and er diagrams tend to be used during elicitation, understanding and elaboration of information system requirements. the hypothesis is clearly in a form that can be empirically tested. it is expressed in terms that support collection and analysis of empirical data in contrast to the proposition, which is expressed in terms of the theoretical framework. terms used in the hypothesis are “things observable” (dubin, 1978, p205). one proposition may generate many hypotheses; in fact h1 above could have been expressed as multiple hypotheses, each linking a pair of empirical indicators. hypothesis testing hypotheses are tested by comparing their predictions with observed data. observations that confirm a prediction do not establish the truth of a hypothesis. the deductive testing of hypotheses involves looking for disconfirming evidence to falsify hypotheses (lee, 1989). falsified hypotheses are then refined based on the reasons for falsification and subjected to further empirical testing. the positivist approach to case study research in information systems a case study is “an empirical enquiry that investigates a contemporary phenomenon within its real-life context, especially when the boundaries between phenomenon and context are not clearly evident” and it “relies on multiple sources of evidence” (yin, 1994, p. 13). case study research investigates pre-defined phenomena but does not involve explicit control or manipulation of variables: the focus is on in-depth understanding of a phenomenon and its context (cavaye, 1996). case studies typically combine data collection techniques such as interviews, observation, questionnaires, and document and text analysis. both qualitative data collection and analysis methods (which are concerned with words and meanings) and quantitative methods (concerned with numbers and measurement) may be used (yin, 1994, p. 14). case study research may involve inductive theory building or have clear a priori definitions of variables to be studied and the ways in which they can be operationalised (benbasat et al., 1987; yin, 1994, p. 34). case study research needs to be distinguished from the use of case studies as teaching devices, where the purpose is to illustrate particular situations and provide a framework for discussion amongst students (yin, 1994, p. 10). teaching cases do not necessarily include a complete or accurate description of actual events, as case study details and materials may be changed in order to better illustrate a specific point. the criteria for developing high quality teaching cases are very different from those for producing high quality case study research (yin, 1994, p. 2). case study research can be used to achieve various research aims: to provide descriptions of phenomena, develop theory, and test theory. case study research has often been associated with description and with theory development, where it is used to provide evidence for hypothesis generation and for exploration of areas where existing knowledge is limited (cavaye, 1996). approaches such as grounded theory (glaser & strauss, 1967), in which theoretical concepts and propositions emerge as the researcher gathers data and investigates phenomena, may be used to develop theory. the use of case study research to test theory requires the specification of theoretical propositions and related testable hypotheses derived from an existing theory. the results of case study data collection and analysis are used to compare the case study findings with the expected outcomes predicted by the hypothesis (cavaye, 1996). the theory is either validated or falsified and found to be inadequate in some way and may then be further refined on the basis of the case study findings. ajis special issue december 2002 79 case study research has been used within both the positivist and the interpretivist philosophical traditions (cavaye, 1996; doolin, 1996). case study research within the positivist tradition is designed and evaluated according to the criteria of the natural science model of research: controlled observations, controlled deductions, replicability, and generalisability (lee, 1989a). although manipulation of variables in the experimental sense is not possible in case study research, theoretical constructs can be defined, operationalised and empirically evaluated, and naturally occurring controls can be identified (lee, 1989; cavaye, 1996). literal and theoretical replication in multiple case study research provides for generalisability of case study research findings (lee, 1989; yin, 1994, pp. 46-51). rigour in positivist case study research validity and reliability in positivist case study research involves using clearly defined methodological guidelines for ensuring construct validity, internal validity, reliability and external validity (lee, 1989; yin, 1994). construct validity concerns the issue of whether empirical data in multiple situations leads to the same conclusions, and is improved by using multiple sources of evidence (to essentially provide multiple data points for the same phenomenon), having key informants review the case study report (to improve the accuracy of case study data) and establishing a chain of evidence so a reader can trace the chain of evidence (yin, 1994). internal validity concerns the issue of whether empirical data provides information about the theoretical unit, and is achieved by using pattern matching to ensure that case study data cannot be explained by rival theories with different independent variables in the hypotheses (yin, 1994). reliability concerns the stability and consistency of the study over time and is ensured by creating and maintaining a case study database and developing a clear case study protocol (yin, 1994). external validity concerns the generalisability of the findings of the study and is ensured by selecting “typical” cases (one or more cases that are representative of a large number of other cases) and selecting one or more cases that are likely to confirm the hypotheses, so that disconfirming evidence can be considered decisive (markus, 1989). the case study of darke (1997) the positivist case study reported by darke concerns viewpoint development in requirements definition. a theoretical framework was first synthesised from the literature, comprising four units and seven laws of interaction. elements of the theoretical framework were then subjected to empirical testing using a multiple case study approach. theory and propositions the theoretical framework in darke’s study is clearly defined and presented in considerable detail. the units and laws of interaction are well supported by an extensive synthesis of literature in information systems development, conceptual modelling and requirements engineering. each of the units in the model has several values that identify types of the unit and are used in generating propositions. for example, the unit viewpoint representation has three values, formal, semi-formal and informal. the theoretical framework is shown in figure 2 below. darke identifies seven bi-directional propositions that associate particular values of one unit with particular values of a second unit. note that not all values need participate in propositions. ajis special issue december 2002 80 figure 2: viewpoint development theoretical framework (darke, 1997, p111) two of the seven propositions identified by darke (1997) that will be used in this analysis are: “p1 informal and semi-formal viewpoint representation techniques tend to be used during viewpoint development supporting requirements acquisition” (darke, 1997, p114) “p2 semi-formal and formal viewpoint representation techniques tend to be used during viewpoint development supporting requirements modelling” (darke, 1997, p115) these propositions are based on associational laws of interaction and are carefully worded to include an element of fuzziness by using the phrase “tend to be used during”. an alternative and stronger wording would be “are used during”, however due to the tentative nature of the theoretical framework the fuzzier wording was deemed appropriate. theory testing using multiple case studies four case studies of the requirements definition phase of systems development projects were conducted in order to empirically test the theoretical framework and propositions. literal replication was used in selecting the case study sites to allow cross-case comparison and strengthen the research findings by increasing the external validity of the study. the four cases were conducted in three different australian organisations in the first half of 1997. each case involved a large systems development project in which the requirements definition phase was either in-progress or recently completed. data collection involved unstructured and semi-structured interviews with several people in different roles in the requirements definition process and access to documents about the project, requirements definition documents and annual reports and other published information about the organisations. data collection was carefully undertaken and a case study database maintained. units in the theoretical framework guided qualitative data analysis to determine their applicability in practice. units were considered applicable if case participants perceived them to be both relevant and meaningful or that units from the framework were present in the data. both within-case and between case analysis was conducted using pattern matching qualitative data analysis techniques (miles and huberman, 1994). deductive testing of the validity of units in the theoretical framework and the seven propositions was reported by listing evidence from detailed descriptions of each of the four cases. case study analysis confirmed the applicability of the four units in the theoretical framework and no additional units were identified. evidence was found to support all seven propositions and one additional law of interaction between the viewpoint representation and viewpoint development process units. the case study of sarker and lee (2000) the positivist case study reported by sarker and lee (2000) concerns the role of “social enablers” in erp systems implementation. three propositions that identify necessary conditions for erp systems implementation success are synthesised from relevant literature, and then subjected to empirical testing using a positivist, single case study approach. viewpoint development role viewpoint agent viewpoint representation viewpoint development process tend to be used during tend to carry out only limited tend to be used for expressing tend to focus on ajis special issue december 2002 81 theory and propositions sarker and lee do not provide an explicit theoretical framework in their study. however, an implicit theoretical framework may be inferred from the three propositions they synthesised: “p1 erp implementation can be successful only if there is a strong and committed leadership guiding the initiative” (sarker and lee, 2000, p416) “p2 erp implementation can be successful only if there is open and honest communication among the stakeholders” (sarker and lee. 2000, p416) “p3 erp implementation can be successful only if the implementation team is empowered and balanced” (sarker and lee, 2000, p416) the implicit theoretical framework has two units, social enablers and erp implementation outcome, and one causal law of interaction. the unit social enablers has three values, strong and committed leadership, open and honest communication and an empowered and balanced implementation team. the unit erp implementation outcome has two values, success and failure. the “only if” label on the law of interaction corresponds to a “necessary” or relatively weak causal law. the implicit framework is shown in figure 3 below. figure 3: implicit erp implementation theoretical framework one of the key references used in the synthesis of the three propositions was parr, shanks and darke (1999) in which critical success factors for erp system implementation projects were identified. in this paper the scope of the erp system implementation project was defined as the configuration and implementation of the erp software package, and success was defined as the project meeting budget and time constraints. critical success factors for successful erp system implementation were elicited from interviews with 10 consultants each of whom had extensive experience in erp system implementation. four of the critical success factors identified as necessary for erp system implementation success were used by sarker and lee (2000). management support (or strong and committed leadership) was clearly identified by all interviewees as a necessary condition for erp implementation success. hence, proposition one of sarker and lee is strongly supported by the parr et al. (1999) study. however, communication was identified as a necessary condition by only 50% of interviewees. hence, the second proposition of sarker and lee was only weakly supported by the parr et al. study. a balanced team was identified by 80% of interviewees as a necessary condition for successful erp implementation; the support for empowered decision makers was not as strong. the third proposition of sarker and lee combines these two critical success factors and therefore is supported by the parr et al. study but not nearly as strongly as management support (in the first proposition). sarker and lee divide their erp implementation process into three phases, of which only the second phase, “implementing … the chosen erp software”, corresponds to the scope of erp implementation used by parr et al. in their study. the first phase concerns changes to the organisation prior to implementing the erp software and the third phase concerns enhancements to the erp system by adding further modules. theory testing using a single case study the case study involved implementation of several modules of an erp system in a medium sized manufacturing company, with offices throughout the united states of america and subsidiaries in great britain, germany and australia. data collection included interviews with 17 people in various roles in the firm, attendance at several meetings associated with the erp implementation, and access to documents about the company background and the erp implementation project from 1996 to 1997. great care was taken to ensure rigour in the conduct of the case study. the guidelines listed previously in this paper were followed closely and reported clearly. for example, pattern matching in qualitative data analysis was used to ensure internal validity by matching predictions derived from propositions with patterns found in the data. success in erp implementation was defined as follows: “we consider implementation to be successful if different stakeholders state or indicate through actions that such was the case” (sarker and lee, 2000, p 419). erp implementation outcome social enablers only if ajis special issue december 2002 82 sarker and lee note that the concept of success is complex; no universally defined criteria exist and criteria vary with the phase of implementation. deductive testing of each proposition was reported by listing evidence from relevant empirical data and justifying a conclusion. proposition p1 (strong and committed leadership) was validated, proposition p2 (open and honest communication) was refuted, and proposition p3 (empowered and balanced team) was refuted. based on these results, sarker and lee note the contradiction between the literature and the empirical evidence from the case study. they conclude that their case study presents a challenge to future erp researchers to investigate the interactions between the three conditions used in the propositions and whether strong and committed leadership can compensate for the absence of the other social enablers. guidelines for conducting positivist case study research several guidelines for conducting positivist case study research emerge from the discussion of the positivist paradigm earlier in this paper and a critical analysis of the two positivist case studies described in the previous section. 1. develop a clear understanding of key concepts and assumptions within the positivist paradigm a deep understanding of the positivist paradigm and its underlying ontological, epistemological and methodological positions is crucial in conducting positivist case study research. the key concepts of theory (including units, laws of interaction and boundary), proposition, hypothesis and hypothesis testing need to be clearly defined and understood. the process of designing and conducting positivist case study research in information systems and guidelines for ensuring validity and reliability are well documented (lee, 1991; yin, 1994). both darke (1997) and sarker and lee (2000) have generally been very thorough in undertaking and reporting their studies, although a critical analysis reveals some important issues that lead to the remaining guidelines. 2. provide clear and unambiguous definitions of the units and interactions in the theory darke takes great care in rigorously and completely defining her theoretical framework. all units in the framework are clearly and unambiguously defined. the laws of interaction are also clearly defined and well supported by relevant literature. sarker and lee do not propose a “theory” per se; they synthesise three propositions from a selection of the erp system implementation literature. however, there is an implicit theory that includes two units and one interaction. while there is some discussion about each of the three social enablers and reference to relevant literature, sarker and lee do not provide precise definitions for these units. of greater concern is the definition provided for success in erp system implementation, relying on stakeholders indicating, “through [their] actions that such was the case”. parr et al. (1999) define success in erp implementation to be “the meeting of budgetary and time constraints”. this is clearly different to the definition of success offered by sarker and lee. the concept of success in erp system implementation is complex, relative and changes over time (markus & tanis, 1999, markus et al, 2000). clear definitions of the units and values are required to ensure the validity and reliability of case study research. 3. carefully define the boundary of the theory used in the case study darke provides a clear definition for the boundary of her theoretical framework by articulating the scope as viewpoint development within the domain of requirements definition. this defines the boundary within which the theoretical framework applies and the limits to which outcomes from the study can be generalised. sarker and lee define the boundary of their case study as the three phases of erp systems implementation: changes to the organisation prior to implementing the erp software, implementing the erp software, and enhancements to the erp system. this is inconsistent with the scope of the factors necessary for erp implementation success identified by parr et al. (1999). parr et al. limited their study to implementing the erp software, or phase two only of the sarker and lee study. interestingly, case study data from phase two of the sarker and lee study does not provide strong evidence to refute any of the three propositions. clearly, when synthesising propositions from the literature, care should be taken to ensure the scope of the studies are the same or very similar. 4. use hypotheses (not propositions) in empirical testing propositions are predictions about the world that may be deduced logically from theory. values of units that appear in propositions should be operationalised before empirical testing can be conducted. each of the values must be assigned an empirical indicator. these empirical indicators are then substituted into the proposition to form a corresponding hypothesis. once hypotheses have been generated they may be used in empirical studies. ajis special issue december 2002 83 darke uses propositions in her empirical study and does not explicitly define and test hypotheses. however, in the detailed description of her case studies, it is apparent that indicators for each of the values used in her propositions exist and could have been explicitly used. for example, empirical indicators for the unit semiformal representation techniques included in the case study description are data flow diagrams, unified modelling language diagrams and entity relationship diagrams. use of hypotheses and empirical indicators in the case study would have strengthened the rigour and validity of the study. in their case study, sarker and lee also directly test propositions rather than first converting them to hypotheses. they did not identify empirical indicators for strong and committed leadership, open and honest communication, an empowered and balanced team, and erp implementation success. furthermore, there is limited detail in the case study description other than a number of quotes from interviewees that generally relate to units in the propositions. two empirical indicators for “success” as defined by parr et al. (1999) that could have been used are that the project is completed on time and within budget. direct testing of propositions with imprecise definitions of values and empirical indicators reduces the rigour of the study and leads to imprecision in findings. 5. consider using fuzzy or probabilistic propositions and embrace post-positivism the two case studies described in this paper have very different types of propositions. darke (1997) develops an associational law of interaction with the label “tends to be used during”. this introduces an element of fuzziness into the proposition that strongly influences the subsequent data analysis in empirical testing. in contrast, sarker and lee (200) develop causal laws of interaction with the label “only if” (a necessary condition). compared to darke’s laws of interaction, this is a relatively strong law of interaction. hypotheses involving causal laws of interaction with the label “if and only if” (both necessary and sufficient) could be easily refuted by selecting specific case situations. in case study research where contextual circumstances are prevalent, strong laws of interaction do not take into account our imperfect knowledge of reality and our inability to be objective and detached, particularly in social situations. in response to these issues many researchers have embraced post-positivism, which relaxes the philosophical assumptions of positivism. guba and lincoln (1994) offer the following three amendments to the dimensions of positivism in defining post positivism: • an objective reality is imperfectly knowable (critical realist ontological position). • a subjective researcher can only know about reality to a degree of probability (modified dualist epistemological position). • a modified experimental method is used including hypothesis refutation using both quantitative and qualitative methods (modified experimental methodological position). in recognising the uncertainty about what can be known, propositions need to have a “degree of uncertainty” or fuzziness about them. 6. select case study sites carefully, particularly single case study sites it is crucial in designing case study research to carefully select case study sites. for multiple case studies, case study site selection should be justified using either literal or theoretical replication. it is important to be very clear about why particular sites are selected and how they contribute to the overall research design. darke used literal replication to select four case studies that were similar in size, nature and complexity. positivist, single case studies should be very carefully selected. yin (1994) argues that single case studies should be unique and revelatory. that is, they are best used in unusual circumstances. however, lee (1989) takes a different position and argues that a single, positivist case study can be used more generally. however, when testing hypotheses in positivist, single case studies, care must be taken to select a typical, representative case study site if the results are to be generalised. 7. generalisation from single case studies is inherently different to generalising from experiments lee (1989) argues that a single, positivist case study is similar to a single experiment. he further argues that the output of a single experiment is not generalisable beyond the set of empirical circumstances encountered in the experiment (lee, 1989). additional experiments with different empirical circumstances that confirm a theory strengthen and extend the generalisability of the theory. similarly, no theory would be generalisable on the basis of a single case study; generalisability is strengthened when the theory is tested on other empirical circumstances (lee, 1989). this argument ignores the fact that experiments can be carefully designed with controls and random assignment of participants to different treatment groups. these experimental designs and statistical analysis techniques are intended to strengthen the generalisability experimental outcomes. in an experiment each participant is subjected to a treatment and separately measured. a single case study is very different. participants are separately interviewed in data collection but their responses are consolidated at “unit of analysis” level. for ajis special issue december 2002 84 example, in sarker and lee’s (2000) case study, the concept of erp implementation success is determined for each phase of the process of erp implementation by consolidating evidence from several participants. they each contribute to a single view of “success”. conclusion the guidelines presented in the previous section are of great significance for information systems researchers and for reviewers assessing positivist, deductive case study research. they suggest that information systems researchers undertaking positivist, deductive case research should: 1. develop a clear understanding of key concepts and assumptions within the positivist paradigm including theory, proposition, hypothesis and hypothesis testing; 2. provide clear and unambiguous definitions of the units and interactions when using any theory; 3. carefully define the boundary of any theory used in the case study; 4. use hypotheses rather than propositions in empirical testing of theory; 5. consider using fuzzy or probabilistic propositions and move to post-positivism; 6. carefully select case study sites, particularly single case study sites; 7. recognise that generalisation from positivist, single case studies is inherently different from generalisation from single experiments. positivist, deductive case studies are an important research approach within information systems. they provide a very sound and systematic approach for conducting research and may be used in conjunction with other approaches to provide richer and more reliable research results. different approaches to research within information systems each provide value and should be seen as complementary to each other. mingers (2001) encourages the design of research programs that use a number of different research approaches from different paradigms. positivist, deductive case studies are an important component of such pluralist approaches to information systems research. references bacharach, s.b. (1989) organisational theories: some criteria for evaluation, academy of management review, 14(4), 496-515 benbasat, i., goldstein, d.k. & mead, m. (1987) “the case research strategy in studies of information systems”, mis quarterly, 11, 369-386. cavaye, a.l.m. (1996) “case study research: a multi-faceted research approach for is”, information systems journal, 6, 227-242. darke, p., (1997) viewpoint development in requirements definition, unpublished phd thesis, monash university darke, p., shanks, g. and broadbent, m. (1998) “successfully completing case study research: combining rigour, relevance and pragmatism”, information systems journal, 8, 273-289 doolin, b. (1996) “alternative views of case research in information systems”, australian journal of information systems, 3(2), pp 21-29 dubin, r. (1978) theory building, free press, new york glaser, b. g. & strauss, a. l. (1967) the discovery of grounded theory: strategies for qualitative research, aldine: chicago guba, e.g. and lincoln, y.s. (1994) “competing paradigms in qualitative research”, in n.k. denzin and y.s. lincoln (eds) handbook of qualitative research, sage publications, thousand island, 105-117 lee, a.s. (1989) “a scientific methodology for mis case studies”, mis quarterly, 13, 33-50. lee, a. s. (1991) “integrating positivist and interpretive approaches to organizational research”, organization science, 2(4), pp 342-365 markus, l. (1989) “case selection in disconfirmatory case studies”, in j. cash and m. lawrence (eds) the information systems research challenge, vol 1, harvard business school, 20-26 markus, m.l. and tanis, c (1999) the enterprise systems experience – from adoption to success. in zmud, r.w. (ed.) framing the domains of it research: glimpsing the future through the past, pinaflex educational resources, cincinatti oh: 173-207 markus, m. l., tanis, c. & van fenema, p. c. (2000) “multisite erp implementations” communications of the acm, 43(4), pp 42-46 miles, m.b. & huberman, a.m. (1994) qualitative data analysis: a sourcebook of new methods, sage publications, newbury park mingers, j. (2001) “combining is research methods: towards a pluralist methodology”, information systems research, 12(3), 240-259 neuman, w.l. (2000) social research methods – qualitative and quantitative approaches, 4th ed. allyn and bacon ajis special issue december 2002 85 parr, a., shanks, g. and darke, d. (1999) “identification of necessary factors for successful implementation of erp systems”, in o. ngwenyama, l.d. introna, m.d. myers and j.i. decross (eds.) new information technologies in organisational processes, boston: kluwer academic publishers, 99-119 sarker, s. and lee, a.s. (2000) “using a positivist case research methodology to test three competing theories-in-use of business process redesign”, proceedings of the international conference on information systems, brisbane, 414-425 shanks, g., rouse, a. and arnott, d. (1993) a review of approaches to research and scholarship in information systems, proc. 4th australian conference on information systems, brisbane, 29-44 yin, r.k. (1994) case study research: design and methods, 2nd edition, sage publications, thousand oaks ajis vol. 7 no. 1 sept 1999 strengthen and support the maintenance of object-oriented software ming-chi lee dept. of business administration, national ping tung institute of commerce taiwan, r.o.c. e-mail: lmc@sunl.npic.edu.tw timothy k. shih and teh-sheng huang dept. of computer science and information engineering, tamkang university taiwan, r.o.c. abstract inheritance is one of the most common features of object-oriented languages, and has been widely applied to develop large and complex software system. however, designing a suitable inheritance hierarchy, involving redundant inheritance, is a difficult task and easily suffers from name-confliction and repeated inheritance which are error-prone and difficult to test. in this paper, we explain how redundant inheritance makes object-oriented programs difficult to test and maintain, and we give a concrete example of the problems that arise. we show that the difficulty lies in the fact that we lack an effective detection tool suited for work with inheritance problems. therefore, a formal checking mechanism is proposed to detect and resolve redundant inheritance. furthermore, this checking mechanism can be easily incorporated with object-oriented case tool to enhance software quality. keywords: inheritance hierarchy, redundant inheritance, repeated inheritance, inference rule, objectoriented program introduction inheritance is a relationship among classes wherein one class shares the structure or behavior defined in one (single inheritance) or more (multiple inheritance) other classes. class hierarchy consists of a set of ordered inheritance relationships and is often denoted as a directed acyclic graph. it plays a vital role and a fabric in object-oriented design (ood). however, designing a suitable inheritance hierarchy, especially with multiple inheritance, is a difficult task (moises et al 1992). this is because several problems associated with multiple inheritance remain still in debate, such as name collisions and repeated inheritances (cardelli 1984, meyer 1988). booch (1991) showed that we have never been able to define a class hierarchy right the first time except for trivial small cases. in practice, this design of class hierarchy is an incremental and iterative process. if you allow multiple inheritance into a language, then sooner or later someone is going to write a redundant inheritance. for example, given a class hierarchy q={ o^-mx,, a,—»a3 , a2-»a4, oj-mx, }. if we add a new inheritance relationship, at—»<x4, then this inheritance ccj-^oc,, is called a redundant inheritance because oc4 could inherit indirectly from a,. after adding c^—mx4, <x4 inherits twice (or more) from o^. this situation suffers from name-confliction problem and the ambiguous method invokation. as a consequence, many implicit software faults very much difficult to test are generated (chung & lee 1997). although some object-oriented programming languages permits this writing, it is necessary to detect and refine them before coding (meyer 1988). we argue that an inheritance-based checking mechanism is essential for effective testing and maintenance of object-oriented programs. in addition, it is worthy to note that an inheritance hierarchy is dynamic rather than static during the lifetime of oo software development. it is therefore almost impossible to maintain an unambiguous and nonredundant inheritance hierarchy forever without a checking mechanism. in section 2, we introduce the redundant inheritance problem associated with repeated inheritance and name conflictions. in section 3, inference rules and the inheritance constraints are introduced. meanwhile, we show how to compute the closure set of an inheritance hierarchy by inference rule. in section 4, a concept of minimal class hierarchy is proposed to address the issue of redundant inheritance. also, a simple but delicate method to compute the closure of a set of classes with respect to a class hierarchy is introduced. in section 5, we derive an algorithm based on the concept of minimal class hierarchy to determine the redundant inheritances in an inheritance hierarchy. three approaches to resolving redundant inheritances are presented. finally in section 6 the conclusion and future research is presented and discussed. . redundant inheritance and basic works basically, inheritance relationships have two different structures. one is single inheritance which allows every class inherits from at most one superclass. in contrast, multiple inheritance allows every class inherits from more than one superclass. in ood, designing a suitable class hierarchy involving multiple inheritance is a difficult task. two problems present themselves when we have multiple inheritance: how to deal with name conflictions from different superclasses, and how do we handle redundant inheritance. name conflictions are possible when two or more different superclasses use the same name for some element of their interfaces, such as instance 32 ajis vol. 7 no. 1 sept 1999 variables and methods (carre &geib 1990). in part (a) of fig.l, class us-driver and class china-driver both have a method named traffic-violation, representing the number of traffic violations in us or china respectively. if someone has driver licenses both in us and china, then a class us-china-drive is declared to be inherited from both of these classes, what does it mean to inherit two traffic-violation with the same name? there are basically three approaches to resolving this clash. first, the language semantics might regard a name confliction as illegal, and reject the compilation of the class. this is the approach taken by smalltalk and eiffel (borning & ingalls m982). in eiffel, however, it is possible to rename items so that there is no ambiguity. second, the language semantics might regard the same name introduced by different classes as referring to the same trafficviolation, which is the approach taken by clos. third, the language semantics might permit the confliction, but require that all references to the name fully qualify the source of its declaration. this is the approach taken by c++. multiple inheritance graph method: traffic-violation; netnodj/fraf fieviolation (a) repeated inheritance graph string: driver-name; method: birthday-cfledc; (b) redundant inheritance graph string: driver-name; method: birthday-check; (c) figure 1: mutiple, repeated and redundant inheritance the second problem is redundant inheritance which is raised by the presence of repeated inheritance when a class is an ancestor of another in more than one way. consider the inheritance graph in fig.l-b. a base class driver is declared to be the parent class of both class us-driver and class china-driver. we find that class uschina-driver inherits twice from class driver. this situation is called a repeated inheritance. furthermore, the class driver, suppose, is declared to be the parent class of class us-china, then this inheritance is called a redundant inheritance because class us-china-driver could inherits all the value and behaviors from class driver through class us-driver or class china-driver (see fig.l-c). from the programming viewpoints, the redundant inheritance is not only unnecessary but also error-prone. in this redundant inheritance graph, class us-china-driver inherits twice (or more) from class driver in the redundant inheritance. redundant inheritance is often confused with repeated inheritance. to formally verify the difference, repeated and redundant inheritance are defined individually as follows: definition 1: given two classes denoted as a, and 0:̂ in some class hierarchy £1, if there exists at least two (or more) inheritance paths between a, and ar all the classes on these inheritance paths form a repeated inheritance. definition 2: given a class hierarchy q with n inheritances, for an inheritance relation o^-hxj e q, if there has been existed other k inheritances for 1 < k < n, such that a,— mx,., — mx,,, . . .— >a,.)[-hxj, then c^-mx, is called a redundant inheritance. basically, redundant inheritance is generated on the extension of repeated inheritance. in this paper, we call a repeated inheritance graph which contains a redundant inheritance as a redundant inheritance graph. consequently, a redundant inheritance graph must also be (contain) a repeated inheritance graph. apparently, fig.l-c is just a redundant inheritance graph. after the above analysis, we can derive a mathematical expression to verify the relationships among multiple inheritance graph, repeated inheritance graph and redundant inheritance graph. this verification is helpful to reduce the occurrence of controversial and error-prone inheritances. 33 ajis vol. 7 no. 1 sept 1999 multiple inheritance graph e repeated inheritance graph c redundant inheritance graph (1) for a large ood system, the detection of redundant inheritance is a time-consuming task. in next section, inference rule is used to address the redundant inheritance. this inference rule can be easily programmed and maintained in the oo software development. inheritance constraints and inference rules one of the key issues in object-oriented design is certainly the specification of inheritance constraints. in the object-oriented design (ood) phase, system designer declares inheritance constraints initially and modifies them later. since inheritance constraints must not be ambiguous and redundant, some external checking mechanisms must be provided. the constraints detected in the oo design should be embedded in some way so that unambiguity and nonredundancy are preserved. the classification of inheritance constraints are shown as follows: 1. single inheritance constraints (sic): given two classes, a, and p, a -» p means that 3 is a subclass of o, and that every instance in p is also an instance in a. 2. multiple inheritance constraints (mic): given three classes, a, p, and y, ocp — > y means that y is a subclass of a and also a subclass of p. 0$ -» y implies that a — »y *p — > y. we can characterize the constraint membership problem in an inference rule system. in what follows, we assume that we are given a set of classes denoted by *f and a class hierarchy denoted by q, involving only classes in *f. the inference rules are: axiom 1: the inference rules for single inheritance constraints are 1 . transitivity: given three classes, a, p and y e *f, if a -» p and p -> y hold, then a — » y. 2. union: if a — > p and a -» y.hold, then a — > py. 3. decomposition: if a — »py hold, then a -» p and a -»y. there are several other inference rules that follows from axiom 1. we state two of them in the next theorem. theorem 1: the inference rules for multiple inheritance constraints are 1. the composition rule: given classes p, <xi, oj, ..., a* € *f, if oq -> p for every i= 1, ..., n, then (xi 02 ... a.-» p. 2. the pseudotransitivity rule: given classes a, p, y and <|), if a — » p and <|>p — > y hold, then ou|> -> y holds. proof: 1): we are given <xi — » p, oj -» p. by composition rule, oti -» p, 02 -> p deduce <xi 02 -> p. by induction, p holds. 2): by decomposition rule, <|>p -> y tells us <|> -» y, p -» y holds. by union rule, a -» y, <|) -> y implies oa|> -» ybefore tackling the main issues in this paper, it is important to introduce transitivity rule and the closure computation with respect to a given class hierarchy. a. transitivity rule for inheritance hierarchy given a class hierarchy consists of a — » p and p — > y. then, we could claim that a— > y must also hold in si. this proof is easy by the property of transitivity. in general, let si be a set of inheritance relations. we say ^ — > t| is indirectly inherited from q, written £1 ̂ £ -> t|. we saw above that if £1 contains a — » p and p — » y, then cc» y is indirectly inherited from cl. that is, { a -» p , p -> y} =* a» y. let q+, the closure of q, be the set of inheritance relations that are indirectly inherited from q, i.e., fl+ = { ̂ -> f\\£l =>+ £ — » t| } . for example, let 4>={oc, p, y), and q={ a -> p , p -» y}. then q* consists of all those inheritances ^ -» t\ such that either 34 ajis vol. 7 no. 1 sept 1999 1. £ contains a, e.g., ap -> y, a -» p , or p -» y, 2. £ contains p but not ct, and t| does not contain a, e.g., p -» y, or p -> <|>, and 3. £ -^ t| is y -> <)). it turns out that computing the closure set for a class hierarchy £2 is a time-consuming task in general, simply because the set of relationships in £2* can be large even though £2 itself is small. consider the set £2 = {ai -> 02, .... on.i —» on }. then £2+ includes all the transitive inheritances ctj —» 4*, where 4* is a subset of {cti, oj, ...,«„ }. obviously, the number of all derived inheritances on the 4* by inference rules could be c(n,l) + c(n,2) + ... + c(n, n-1) + c(n, n). the number is equal to 2n. as there are 2" such sets of £2, we can not expect to list £2+ conveniently, even for a reasonably sized n. by contrast with £2*, computing £*, for a set of classes £, is not hard; it takes time proportional to the length of all inheritances in £2, written out. instead of computing the tedious £2*. £* is informative enough to tackle the cyclic and redundant inheritance issues. in the next section, an algorithm to compute £* will be presented. b. equivalence of two class hierarchies let £2 and 0 be two class hierarchies; £2 and 0 are said to be equivalent if and only if £2* = 0*. to test whether £2 and 0 are equivalent, we must verify whether both £2* e 0+ and 0+ £ £2+ hold. the verification of the equivalence of two class hierarchies by inference rules is relatively time-consuming. an illustration is shown as follows: example a: given a class hierarchy £2 consists of two multiple inheritance constraints (mic) and two single inheritance constraints (slc)as follows: q = 1 5ic :« 1 -» « 2« 3« 4 <« 6 -» « 5 [ m 1c -.a !« 2a 3 -» a 5 ,a 3a t -> a 6 we can find other equivalent class hierarchies whose closure is equal to £2. suppose 0 is an equivalent one which contains four single inheritance constraints and one multiple inheritance constraint as follows: — » z: «2a3 -> a5 by use of axiom 1 and theorem 1, we can verify that £i and 0 are equivalent by checking whether £2* is equal to 0*. however, we cannot expect to list q+ and 0* conveniently. a useful alternative equivalence is addressed in the next theorem. theorem 2: for each class hierarchy f l , there is an equivalent class hierarchy 0 in which no both sides of an inheritance have more than one class. proof: (1) let 0 be the set of inheritance relations a -> ty\ such that for some a — » y in £i, 4>( e vy where (fc is a single class. the a -> <t»j follows from y by the decomposition rule. thus, 0 e fi* holds. on the other hand, £2 £ 0* also hold, since if \y = ty\ .. .<t>n, then a • y follows from a -xj»i, .... a -*(>„ using the union rule. (2) let 0 be the set of inheritance relations fy, -» p such that for some \|/ -»p in £2, <h e y. the «j>j -» p follows from \|/ by the decomposition rule. thus, 0 c £2* holds. on the other hand, £2 c; 0+ also holds, since if \j/= 4>i . . .<(»„, then y -» p follows from $1 -> p, fo -> p ..... <j>n -> p using the union rule. example b consider class hierarchy £2 in example a again, by theorem 2, we can find an equivalent class hierarchy denoted 0 whose both sides has only one class and is shown as follows: *=>-l' **1 ~* u2°'3u4 'u6 u5 equiabtt . q _ ajis vol. 7 no. 1 sept 1999 it turns out to be useful, when we develop a class hierarchy design, to consider a stronger restriction on equivalence than that both sides have but one class. minimal class hierarchy in this section, we propose a useful concept called minimal class hierarchy to address redundant inheritance. this minimal class hierarchy plays a vital role in our checking mechanism for redundant inheritance. definition 3: a class hierarchy is said to be minimal if: 1. every both sides of an inheritance relation in ft is a single class. 2. no £ — » r\ e ft such that the set (ft { j; -» t| }) is equivalent to q. theorem 3: if a class hierarchy & contains redundant inheritance relationships, then there is a minimal class hierarchy 0 such that 0 c ft and ft* = 0+. proof: we have to show that 0 satisfy the two conditions mentioned above. 1. for each redundant inheritances £ — > t) € ft, (ft {£ -» t\})+ = ft+ must hold, and we can remove i; -» t| from ft. let 0 = ft (£ -> t|). because q contain redundant inheritance relationships, there exists at least one redundant relationships j; — > t| such that ft { ̂ -» t| } is equivalent to q. 2. by theorem 3, all the inheritance relationships in ft can be decomposed into a class hierarchy 0, in which no both sides have more than one class. therefore, 0 can be done in the same way. example c: consider the class hierarchy 0 in example b again. we apply theorem 3 to 0, then get a minimal class hierarchy denoted a equivalent to 0 shown as follows. by the comparison of example a and example c, we can conclude that £2, 0, and a are equivalent mutually.( i.e., £i+ = 0+ = a+.) theorem 3 derives a useful property that a minimal class hierarchy itself is a non-redundant class hierarchy. this non-redundancy is just what we are seeking for. we can derive a redundant inheritance detection algorithm straightforward from the theorem 3. the idea of this algorithm is that for each inheritance % • tj e £i, if q+ is equal to (ft { ̂ -> t| } )+, then ̂ -> t| is a redundant inheritance. algorithm 1: find out redundant inheritances with respect to a class hierarchy input: a class hierarchy q={ei, 62, ..., en} with n inheritances. output: a redundant inheritance set f 1. f = 0; initialize f with empty 2. for each cj e £2, i = 1,2,. ..,n if q+= {£i-ei}+ then add ei to f however, this algorithm is not efficient because the computing of ii+ is an exponential time. fortunately, there is an alternative approach to replace £i+. at the other extreme, computing %+, for a set of classes ^, is not hard; it takes time proportional to the length of all inheritances in q, written out. a more efficient algorithm based on e,+ will be proposed in the next section. now we define %? formally as follows: definition 4: let ft be a class hierarchy on a set of classes denoted by t, and let ^ be a subset of xf. then the closure of ^ with respect to ft, denoted by e,+, is defined as the set of classes t| such that ^ — » t\ can be deduced from ft by inference rule. the redundant inheritance detection could be achieved by checking whether %+ still contains t| after removing ^ — > t). if it does, then j; — » tj is a redundant inheritance relationship. a simple algorithm to compute £*" is shown in the following. 36 ajis vol. 7 no. 1 sept 1999 algorithm 2: computation of the closure of a set of classes with respect to a class hierarchy input: a finite set of classes t, a class hierarchy £1 on 4*, and a set ̂ c 4*. output: £*, the closure of £ with respect to £2 method: we compute a sequence of sets of classes ^(0), £(1),..., by the rules:u<°>^ 2. £(1+1) is £(l) plus the set of classes t| such that there is some inheritance y -> z in q, t| is in z, and y g £(i). since £ = £(0> c ... £ £(i) c, ... c 4*. and m* is finite, we must eventually reach i such that £(i) = £(i+l). it then follows that £(i) =£<i+1)= £(i+2)= .... there is no need to compute beyond £(0 once we discover £(i)= £(i+i). we can (and shall) prove that £* is £w for this value of i. in the following, we use an example to illustrate the algorithm. to illustrate the algorithm clearly, a complex class hierarchy is chosen and executed instead of a trivial one. although the class hierarchy may be very complex and contains cyclic inheritance which is controversial in semantics, the major goal is to demonstrate the process of algorithm execution. therefore, we do not request its programming feasibility in practice. example: let £1 consists of the following eight inheritances: a^jtt, -> a2 at -> «5a6 «2a5 -» «3 «3a6 -> a2at a3a5 -» «1a6 and let ^=0204. to apply algorithm 2, we let £(0)= c^ct). to compute £(1> we look for fi that have a left side oj, 04, or 0204. there is only one, o» -» 050$, so we adjoin 05 and 0$ to £<0) and make ^(1)= o^o^ocsog. for £(2), we look for left sides contained in £(1) and find 04 -4 osoe and 0205 -» o3. thus £(2>= o^ajo^oscif,. then, for £(3) we look for left sides contained in 020:3 otosoe and find, in addition to the two previously found, o3 -» oi, 0703 -» 04, a3o6 — » 02 ct», and a3(x5 -» ccic^. thus ^(3)= 010203040506. it therefore comes as no surprise that £(3)= £<4)=.... thus (a2o4)+= oi 02 a3 04 05 o6. time complexity and data structure analysis for algorithm 2 algorithm 2 can be implemented to run in proportional to the sum of the lengths of the inheritances in £1 if we keep, for each inheritance y — » z, a count of the number of classes in y that are not yet in £(l). we must also created a list, for each class t|, of the inheritances on whose left side t| appears. when t) is adjoined to some £(l>, we decrement by one that count for each inheritance on t\'s list. when the count for y — » z becomes 0, we know y £ £(l>. lastly, we must maintain £(l) as a boolean array, indexed by class numbers, so when we discover y £ £(l), where y — » z is an inheritance in £i, we can tell in time proportional to the size of z those classes in z that need to be adjoined to £(l)when computing £(m'1) from £(l) we have only to set to true the entries of the array corresponding to classes added to £(l); there is no need to copy £'". now the problem of proving that algorithm 2 is correct must be addressed. it is easy to prove that every class placed in some £(k) belongs in £*, but harder to show that every class in £* is placed in some ̂ theorem 4: algorithm 2 correctly computes £+. proof: first we show by induction on k that iff) is placed in £*', then t| is in £+. basis: k=0. then t| is in £, so by reflexivity, £ -» tj. induction: let k>0 and assume that jj*"0 consists only of class in £*. suppose t| is placed in ̂ because t| is in z, y -* z is in q, and y c £(l"'). since y £ £(tl), we know y c t? by the inductive hypothesis. thus £ -» y by lemma 1. by transitivity, £ -> y and y -> z imply £ -» z. by reflexivity, z -»£, so £ -» t| by transitivity. thus t| is in £+. redundant inheritance detection and resolution in this section, a detection algorithm for redundant inheritance will be presented. also, three approaches to resolving redundant inheritances are proposed. in section 3, we have shown that the computation of a closure set of a class hierarchy, for example f l , is very time consuming. in contrast, computing ̂ , for a set of classes ^, is not hard. previously, ^+ is defined as the closure of a set of classes ^ on some class hierarchy. because there may exist various different class hierarchies in an oo design, the definition of the closure ^ is not precise 37 ajis vol. 7 no. 1 sept 1999 enough to specify to which class hierarchy it belongs. for the sake of precision, the term 4+n is used to strengthen the previous definition. simply stated, the closure of £ on £2 is denoted as £,+n. the idea of the algorithm is that for each ^ -> tl e q, if tl e £+n-(£. n } then ̂ -» n, is a redundant inheritance. algorithm 3: find out redundant inheritances input: a class hierarchy q. output: a set of redundant inheritance f 1. apply theorem 2 to ft, then get an equivalent class hierarchy 0 2. for each ^ -» t| e 0 do (1) apply algorithm 2 to compute 4v{5ti ( (2) if ti e £+e-($. n )} t"611 add ^ -» tl to f example d: given a class hierarchy q, find out its redundant inheritances. let q consists of the following four inheritances: step 1. to apply theorem 2 to £i, we then get a class hierarchy 0 as follows: step 2. for each inheritance £ • t| in 0 we do: 1. ai —» a3 is chosen: we apply algorithm 2 to compute the closure of oi+ on (0-{(x| -» 0.3}), and we get ai +={ 05}. we find that oc3 e cti+ does not hold, then discard aj -* a3. 2. d] —»ots is chosen: we apply algorithm 2 to compute the closure of oti+ on (0-{cti —> ots}), and we get a\ +={a3, 04,05}. we find that 05 e oti+ holds, then we adjoin cii —> 05 to f. 3. a2 —> as is chosen: we apply algorithm 2 to compute the closure of 02* on (©-{02 —> 013}), and we get ct2*={0} (denoting empty set). we find that ct3 e 02* does not hold, then discard 02 -» a3. 4. a3 -> 04 is chosen: we apply algorithm 2 to compute the closure of ot3 + on (0-{ct3 -» 014}), and we get a3+={ots}. we find that 04 e a/ does not hold, then discard a$ —» o .̂ 5. ct3 -» 05 is chosen: we apply algorithm 2 to compute the closure of a3 + on (0-{a3-> 05}), and we get a3 +={ct4, ots}. we find that o^ e a^ holds, then we adjoin <x3 -» o^ to f. 6. 04 —» ots is chosen: we apply algorithm 2 to compute the closure of oj* on (0-{oi —» 0(5}), and we get o4+={0}. we find that 05 e a/ does not hold, then discard 04 -» 05. after the execution of algorithm 3, f will contain a set of redundant inheritances {oti -» 05, ot3 -> 05 }. for this redundant inheritance at —» 05 in f, it means that class 0$ could inherit directly or indirectly from oti . equation (1), shown in section 2, reveals that 05 would inherit twice (or more) from cti . to reduce the occurrence of implicit errors caused by this redundant inheritance, we had better refine all the ancestor classes of class 05. the depth-first or breadth-first traversal algorithm can be used to determine these ancestor classes. the refinement depends on the actual inheritance relations. we can combine related classes to remove these unnecessary duplicates. this behavior is also called class aggregation (hendler 1986). the redundant inheritance a3 —» 05 can be dealt by the same way. analysis for algorithm 3 algorithm 3 basically consists of two parts. first part is to apply theorem 3 to a given class hierarchy. as the proof of theorem 3 mentioned above, we use decomposition rule and union rule to get a class hierarchy whose both sides contain only one class. for an inheritance £ —» t), the decomposition rule can be implemented to run in time proportional to the length of r\, and union rule is in proportional to the length of %. second part is to apply algorithm 2 to compute the closure set of £ on 0 for each inheritance ^ -> ti e 0. the analysis of 38 ajis vol. 7 no. 1 sept 1999 algorithm 2 has been shown above. integration of the analysis of the two parts, the algorithm 3 runs in proportional to the sum of the lengths of inheritances in a given class hierarchy. resolutions for redundant inheritance after the redundant inheritance have been identified, it is important to resolve them before coding. basically, there are three approaches to dealing with the problem of redundant inheritance. first, we can treat occurrences of redundant inheritance as illegal. this is the approach taken by java, smalltalk and eiffel (with eiffel permitting renaming to disambiguate the duplicate references). second, we can permit duplication of superclasses, but require the use of fully qualified names to refer to members of a specific copy. this is one of the approaches taken by c++. third, we can treat multiple reference to the same class as denoting the same class. this is the approach taken by c++ when the repeated superclass is introduced as a virtual base class. a virtual base class exists when a subclass names another class as its superclass and marks that superclass as virtual, to indicate that it is a shared class. the redundant inheritance in fig.l-c can be resolved by declaring superclass driver as a virtual base class: class driver {....}; class us-driver, virtual public driver {...}; class china-driver: virtual public driver { ... }; class us-china-driver: public us-driver, public china-driver {...}; similarly, in clos repeated classes are shared, using a mechanism called the class precedence list. this list, calculated whenever a new class is introduced, includes the class itself and all of its superclasses, without duplication, and is based upon the following rules: 1. a class always has precedence over its superclass 2. each class sets the precedence order of its direct superclasses in this approach, the inheritance graph is flattened, duplicates are removed, and the resulting hierarchy is resolved using single inheritance (dussud 1993). this is akin to the computation of a topological sort of classes. if a total ordering of classes can be calculated, then the class that introduce the redundant inheritance is accepted. note that this total ordering may be unique, or there may be several possible orderings. if no ordering can be found (for example, when there exist cyclic inheritance), the class is rejected. in fig.l-c, the class us-chinadriver would be accepted, because there is a unique ordering of superclasses; the superclass hierarchy includes exactly one (shared) appearance of the class driver. invoking a method hi redundant inheirtance in traditional programming languages, invoking a subprogram is a completely static activity. in pascal for example, for a statement that calls the subprogram p, the compiler can generate code that creates a new stack frame, places the proper arguments on the stack, and then changes the flow of control to being executing the code associated with p. however, in oo languages that support polymorphism, invoking a method is dynamic because the class of the object being operated upon may not be known until runtime. matters are even more complicated when we add redundant inheritance to this situation. redundant inheritance with polymorphism requires a much more sophisticated technique. consider the class hierarchy shown in fig.l-c, which shows the base class driver along with two subclasses named us-driver, and china-driver. both of them has a subclass, named us-china-driver. in class driver, the method birthday-check is common to all subclasses, and therefore need not be redefined. however, the method traffic-violation must be redefined by each of the subclasses, since either in china or us, the number of violations must be mutually independent. thus, since the class driver is an abstract class', traffic violation has an empty implementation (it is a pure virtual function, in c++ terminology). in c++, the developer can decide if a particular method is to be bound late by declaring it to be virtual; all other methods are considered to be bound early, and thus the compiler can statically resolve the method call to a simple subprogram call. in this redundant inheritance, we might have declared traffic-violation as a virtual 1 a class with no instances is called abstract data class. 39 ajis vol. 7 no. 1 sept 1999 member function and the method birthday-check as nonvirtual because the birthday of anyone is unique and unexchangeable. therefore, birthday-check need not be redefined. conclusion in this paper, we propose a checking mechanism to determine redundant inheritance in a given class hierarchy. inference rule system is used to specify the inheritance constraints. a minimal class hierarchy concept is presented to address the redundant inheritance occurrence. an algorithm based on the minimal class hierarchy is derived to detect redundant inheritances. this work will contribute to object-oriented software testing and maintenance. it is note worthy that this checking mechanism can be easily incorporated with oo case tool to facilitate oo software development and quality. the occurrence of cyclic inheritance which causes selfinheritance and endless type-checking is another problem associated with multiple inheritance in the class hierarchy design. for most theoretical papers, it is strictly prohibited and is assumed to never happen. however, the occurrence of cyclic inheritance due to careless design or specific purpose is inevitable and inherent for software designers. for the sake of space limitation, we leave it to future discussion and research. references horning, a.h. and ingalls, d.h.(1982), "multiple inheritance insmalltalk-80," in proc. of the aaai-82, pittsburgh, 1982, pp. 234-237. cardelli, luca (1984)," a semantics of multiple inheritance, " in semantics of data types, lecture notes in computer science 173, springer-verlag new york, 1984, pp. 51-67. carre, b. and comyn, g. (1988)," on multiple classification, points of view and object evolution," in demongeot, herve, t., rialle, v., and roche, c. (eds.), artificial intelligence and cognitive science, manchester university press, 1988. carre, b. and geib, j.m.(1990), "the point of view notion formultiple inheritance, " oopsla, 1990, pp. 312-321. chung, c.m. and lee, m.c.(1997)," integration object-oriented software testing and metrics," international journal of software engineering and knowledge engineering, vol. 7, no. 1,1997, pp. 125-144. dussud, p. (1993)" ticlos: an implementation of clos for the explorer family." sigplan notices vol. 24(10), 1993. freeman, lee, ed.(1991), computer aided software engineering (case), james martin insight inc, naperville, 111., the martin reprt, vol. vi, updated quarterly, 1991. grady booch (1991), object-oriented design with application, pp. 105-113, 1991, the benjamin. hendler, j. (1986)"enhancement for multiple inheritance," sigplan notices, vol. 21 (10), oct. 1986, pp. 100. kim, won modern database systems (1995), the object model, nteroperability, and beyond, acm press. 1995. pp.211-222. lee, m.c. and chiang, d.a.(1992), "cyclic inheritance detection for object-oriented database," ieee region 10 conference, tencon'92 australia, november, 1992, pp. 633-637. meyer, bertrand, object-oriented software construction, new york, prentice-hall 1988, pp. 274. moises lejter, scott meyers, and steven p. reiss, (1992)"support for maintaining object-oriented programs," ieee trans, on software engineering, vol. 18, no. 12, dec. 1992, pp. 1045-1052. ullman, j.d., (1988) principles of database and knowledge-base systems, vol. 1, computer science press, maryland 1988, pp. 391-392. 40 ajis vol. 8 no. 1 september 2000 top-down enterprise application integration with reference models willem-jan van den heuvel & wilhelm hasselbring & mike papazoglou infolab, dept. information management and computer science, tilburg university, po box 90153, nl-5000 le tilburg, netherlands, email: { wjheuvel | hasselbring | mikep } @kub.nl abstract for enterprise resource planning (erp) systems such as sap r/3 or ibm sanfrancisco, the tailoring of reference models for customizing the erp systems to specific organizational contexts is an established approach. in this paper, we present a methodology that uses such reference models as a starting point for a top-down integration of enterprise applications. the re-engineered models of legacy systems are individually linked via cross-mapping specifications to the forward-engineered reference model's specification. the actual linking of reference and legacy models is done with a methodology for connecting (new) business objects with (old) legacy systems. introduction with the traditional bottom-up approach to the integration of existing (legacy) systems, the structure of the (merged) integrated information models is highly determined by the overlaps among the component system models. as discussed in hasselbring (1999), the maintenance of such integrated models may become a serious problem, because the merged models rapidly become very complex; usually more complex than required for the actual integration goals. this situation can lead to severe scalability problems with respect to execution performance, usability, and maintenance. for a discussion of the resulting problems refer to hasselbring (1999). another way to approach the integration of heterogeneous information systems is a top-down process. starting with common reference models, the individual component models are integrated into these common reference models hasselbring (1999). the resulting integration process is illustrated in figure 1. the local models of the legacy systems are not integrated into a common global model (which would be the 'federated schema' in federated database systems (sheth & larson 1990)) as it would be the case with the traditional bottom-up approach. instead, an integration of the given reference model with each individual local model is constructed via a unking mechanism (a form of type matching in our methodology). a cross-mapping specification defines the mapping from the (given) reference model to the (local) legacy models. the integration process starts top-down with the reference model. the linking process combines forward and reverse engineering techniques. both, the reference model and the legacy models are specified in our component definition language (cdl), before they are integrated. an important difference with integration in federated database systems is that with enterprise applications, we integrate business models, not database schemas. cdl has been proposed as the standard component specification language by the business object domain task force of the omg. it is a superset of the omg interface definition language (idl) and the odmg object definition language (odl). we introduce specific extensions for business modeling and, in particular, cross-mapping specification. 126 aj1s vol. 8 no. 1 september 2000 cross-mapping :;j-:~ grpssttnappjiig ',. re-engineered . ; legacy" system '.model ••• .-.. . < . . $ . .& f-<? re-enigineered legacy systern, model''-,''' forward engineering i q. reverse engineering figure 1: top-down integration of legacy systems with a reference model for enterprise applications as starting point for the integration. for the actual linking of reference and legacy models, we employ a methodology for business model integration, called business applications to legacy systems (bales) (van den heuvel 2001). this methodology allows reusing as much of the legacy data and functionality as needed for the development of applications that meet modem organization requirements and policies. in particular, the bales methodology allows to construct configurable business applications on the basis of business objects and activities that can be linked through parameterization to their legacy counterparts. the bales methodology for linking business models most of the approaches to integrate legacy systems with modern applications are designed around the idea that data residing in a variety of legacy database systems and applications represents a collection of entities that describe various elements of an enterprise. moreover, they assume that by combining these entities in a coherent manner with legacy functionality and objectifying (wrapping) them, legacy systems can be readily used in place. in this way it is expected that the complexities surrounding the modem usage of legacy data and applications can be effectively reduced. unfortunately, these approaches do not take into account the evolutionary nature of business and the continual changes of business activities and policies which need to be reflected in the legacy systems. although part of the functionality of a legacy system can be readily used, many of its business activities and policies may change with the passage of time. one important characteristic of business object technology, that also contributes to the critical challenge described above, is the explicit separation of interface and implementation of a class (eeles & sims 1997). business object technology takes this concept a step further by supporting interface evolution in a way that allows the interfaces of classes to evolve without necessarily affecting the clients of the modified class. this is enabled by minimi/ing the coupling between business components. client and server classes are not explicitly bound to each other, rather messages are trapped at run-time by a semantic data object that enforces the binding at the level of parameter passing semantics. the bales methodology, that is under development (van den heuvel 2001), has as its main objective to link business objects (bos) with legacy objects (los). legacy objects serve as conceptual repositories of extracted (wrapped) legacy data and functionality. these objects, just like business objects, are described by means of their interfaces rather then their implementation. a business object interface can be constructed from a legacy object interface partition comprising a set of selected attribute and method signatures. all remaining interface declarations are masked off from the business object interface specification. in this way, business objects in the bales methodology are configured so that part of their specification is supplied by data and services found in legacy objects. a business object within the reference model can thus have a part that is directly supplied from some legacy data and services which it integrates with data and services defined at its own level. this means that the business object interfaces are parameterizable to allow these objects to evolve by accommodating upgrades or adjustments in their structure and behavior. 127 ajis vol. 8 no. 1 september 2000 workflow business activities business object legacy activities legacy object •*£.; -x material-: \ requirements plann forward engineering (reference model) linking reverse engineering (legacy model) figure 2: methodology of linking reference and legacy models. workflows initiate business activities and business activities use business objects. the dashed lines are meant to illustrate (possible) mappings between reference and legacy elements. the core of the bales linking methodology comprises three phases, as illustrated on the right border of figure2: forward engineering, reverse engineering and linking. to illustrate this linking methodology, a simplified example is drawn from the domain of maintenance and overhaul of aircrafts. this example is inspired from building block definitions that are currently developed at the department of defense in the netherlands (dodn 1997). the upper part of figure illustrates the reference model for the business domain (which is based on the defense and aerospace reference models for sap r/3 (sap 2000)) in terms of workflows, business activities and business objects. as can be seen from this figure, the reference model defines a request_part workflow which comprises three business activities: request, prognosis and issue. the request_part workflow is initiated by a maintenance engineer who requests parts (for maintaining aircrafts) from a warehouse. a warehouse manager can react in two different ways to a request: 1. firstly, the manager can directly issue an invoice and charge/dispatch the requested products to the requester. in this case, the workflow will use information from the request activity to register the maintenance engineer's request in an order list. this list can be used to check availability and plan dispatch of a specific aircraft part from the warehouse. the request activity uses the business (entity) objects part and warehouse for this purpose. subsequently, the workflow initiates the issue activity (see figure). the issue activity registers administrative results regarding the dispatching of requested parts and updates the part inventory record by means of the part stock business object. the business object request part control is an auxiliary control object used during the execution of the workflow to store and control the state of the running business activities. if the requested part is not in stock, then an order part workflow is triggered (not shown in this figure). this workflow then orders the requested parts to fulfill the request of the request part workflow. 2. secondly, in case of an 'abnormal' request, for example if the customer informs the warehouse manager about a large future purchase, the manager may decide to run a prognosis. this activity first registers the request information provided by the request business activity and runs a prognosis on the basis of the availability and consumption history of the requested part. the prognosis activity uses information from the part and warehouse business objects for this purpose. after the prognosis finished successfully, the part can be reserved. if the results of the activity prognosis are negative with respect to the future availability of the requested aircraft part, another workflow for ordering parts is activated. the lower part of figure represents the result of the reverse engineering activity in the form of two activities (wrapped applications and related databases) material_requirements_planning and purchasejrequisition. these activities make use of five legacy objects to perform their operations. 128 ajis vol. 8 no. 1 september 2000 figure also indicates that the reference workflow draws not only on "modern" business objects and activities, but also on existing (legacy) data and functionality to accomplish its objectives. for example, business activities such as request and issue on the reference model level are linked to the legacy activities materiai_requirements_planning and purchase _requisition by means of solid lines. this signifies the fact that the activities on the reference model level reuse the functionality of the activities at the legacy model level. the same applies for business objects at the reference model level such as part, part_stock and stock_ location, which are parameterized with legacy objects. in this simplified example we assume that problems such as conflicting naming conventions and semantic mismatches between the reference and legacy models have already been resolved. /'i. reference modeling 2. cdl-specification 3. instantiate the meta-cdl reference model forward engineering phase ir— linking phase cross mappable constructs 7. query model to find potentially mappable constructs reverse engineering phase ± 8. define bos and business processes in terms of los & processes incdl ajjyt!fs!iate tne meta-cdl legacy model ,.'' figure 3: the bales linking methodology. figure illustrates the integration approach of the bales methodology and the individual steps applied during its three phases. the forward engineering phase transforms a conceptual reference model (e.g., sanfrancisco (abinavam et al. 1998) reference models specified in the uml notation (booch et al. 1999)) into cdl and maps this cdl definition to a meta-cdl model which serves as a basis for comparison between business and legacy enterprise models. this phase comprises the activities which correspond to steps 1, 2, and 3 in figure. in the second phase of the bales methodology, we represent the legacy objects and activities in terms of cdl and link them to a meta-cdl legacy model. the activities during the reverse engineering phase, which correspond to steps 4, 5, and 6 in figure 3, are similar to those performed during the forward engineering phase. the actual linking is then done in step 7, and the cross-mapping specification is constructed in the final step 8. below, we illustrate the bales methodology for constructing a cross-mapping specification by means of the aircraft maintenance and overhaul example following those steps: 1. reference modeling: the forward engineering activity starts with the given reference model. the reference model reveals the activities, structure, information, actors, goals, and constraints of the business in terms of business objects, activities, and workflows, and is illustrated by the reference workflow in the upper part of figure 2. 129 ajis vol. 8 no. 1 september 2000 2. cdl-specification of the reference model: the interface descriptions of the business objects and activities need to be constructed on the basis of the reference model. to formally describe the interfaces of business objects, we use a variant of the cdl that has been developed by the omg (dat 1997). cdl is a declarative specification language — a superset of the omg interface definition language (idl) and the odmg object definition language (odl) with specific extensions for business modeling and, in particular, cross-mapping specification. a specification in cdl defines business object interfaces, structural relationships between business objects, collective behavior of related business objects, and temporal dependencies among them (dat 1997). detailed descriptions of the cdl syntax can be found in dat (1997) and some practical experience with the use of cdl is discussed in hordijk (1998). the reference model represented in the upper part of figure 2, serves as a starting point to specify the business object/activity in cdl. figure 4 gives an extract of the cdl specification involving a business object with interesting dynamic behavior, namely the request_part_control object. this cdl specification describes the interface of the business control object request_part _control (see figure 2) and shows that this business object encapsulates three business activities: request, prognosis, and issue. the dynamic behavior of the encapsulated prognosis activity should be interpreted as follows: the prognosis activity is triggered by the incoming signal register_expected_stock. after this signal is received, the activity moves from the state initial to the state forecasting. this is expressed with the state transition rule (str) start_forecasting. while the prognosis activity resides in the state forecasting, it can perform the forecast operation. this operation stochastically estimates the required stock on the basis of past data (stock levels) in the warehouse (warehouseld) and required future demand of the part (parted) for a specific period (consumptionperiod). the manualreorderplanning business operations of the prognosis activity refers to the situation where the user has to define the reorder point and the safety stock him/herself. this approach is in contract to the automatic reorder planning (method where both parameters are automatically forecasted). reorder planning is a special category consumption-based planning that calculates the reorder point on the basis of past and future consumptions, delivery lead times, etc. in this planning strategy, the available stock is compared with the reorder point; if the actual stock gets below the reorder point, the system automatically creates an order form (keller, g. & teufel, t. 1998). 3. instantiating the meta-cdl reference model: the cdl descriptions of both the forwardand backward-engineered models have to be connected to each other in order to be able to ascertain which parts of the legacy object interfaces can be re-used with new applications. to achieve this, we represent both business and legacy cdl specifications in a repository system. for this purpose we utilize the conceptbase system (jeusfeld et al. 1998), which has an advanced query language for abstract models (like the cdl meta model) and it uniformly represents objects at any abstraction level (data objects, model components, modeling notations, etc.). the advantage of this repository approach is that the content of the repository, viz. meta-cdl models, is subject to automated analysis, mainly by means of queries. 130 entity request_part_control // this object comprises all three business activities and registers // the state of them (initial, processing or handled). activity prognosis relationship borpc ispartof request_part_control inverse bpp ; relationship wflr many 0..* request_art_workflow inverse bpp ; attribute string consumptionhistory; attribute date consumptionperiod; attribute integer expectedspecialsale; attribute integer totalexpectedconsqualityinperiod; void manualreorderplanning (in integer partid, in integer stockid, in integer warehouseld); signal register_expected_stock (); states prognosis (initial, manual_planning, forecasting, stopped); during (prognosis==forecasting) withforecast attribute integer partid; attribute integer stockid; void forecast (in integer partid, in integer stockid, in integer warehouseld, in date consumptionperiod, in string consumptionhistory); , apply statetransitionrule start_forecasting trigger = register_expected_stock(); source = initial; target = forecasting ; // end: prognosis activity issue_part .. ; activity request ,# end entity request_part_control figure 4: the cdl specification for the reference business (entity) object request_part_control. after the interfaces of the business objects and activities have been specified in cdl, the cdl specifications are instantiated according to a meta-cdl reference model. this meta model depicts the instantiations of the cdl model elements. it defines how the cdl constructs are related to each other, and provides information about their types. the cdl meta-modeling step is used as a basis to infer how the constructs found in a meta-cdl reference model can be connected to related constructs found in the legacy models (see below). in summary, the meta-cdl model serves as a shared description (could be compared to the 'canonical data model' in federated database systems (sheth & larson 1990)) to which the forward and the reverse engineered cdl models will be linked in order to ascertain which (portions of) legacy elements can be linked to the reference model level. in this way, it is possible to parameterize reference model elements with related legacy elements for linking them to each other in the cross-mapping specification. 4. reverse engineered legacy model: the reverse engineered model represents the wrapped legacy data and functionality. to construct the legacy objects, we rely on techniques that combine object wrapping and meta-modeling with semantic schema enrichment (papazoglou & russel 1995, papazoplou & van den heuvel 2000). the legacy object model comprises a distinct legacy object and activity layer in the bales methodology (see bottom part of figure 2). 131 ajis vol. 8 no. 1 september 2000 reverse engineered legacy activities such as material_requirements_planning and purchase_requisition and wrapped objects like part, plant, warehouse, etc., are represented in the reverse engineered model as illustrated in figure 2. the legacy activity material_requirements_planning is used to determine the requirements for parts at an aircraft maintenance location. figure 5: the cdl specification for the legacy (entity) object warehouse. legacy_entity warehouse relationship ordered_for many part inverse ordered_by; relationship has many plant inverse of; attribute integer plantid; [required] attribute string warehouse_name, warehouse_address, warehouse_place; state orderinginitial, planning, planned // definition of the material requirements planning legacy process legacy_activity material_requirements_planning relationship lop many 0..* part inverse mrplpp ; relationship low many 0..* warehouse inverse mrplpw ; relationship lopl many 0..* plant inverse mrplppl ; attribute string consumptionhistory; attribute date consumptionperiod; attribute integer totalexpectedconsqualityinperiod; signal register_expected () ; signal register_expected_stock (); void forecaststochmodel (in integer partid, in integer stockid, in integer warehouseld, in date consumptionperiod, in string consumptionhistory) ; states prognosis (initial, manual_planning, forecasting, stopped); apply statetransitionrule start_forecasting trigger = register_expected_stock(); source = initial; target = forecasting r ; // end warehouse entity 5. cdl specification of the legacy model: the interfaces of the legacy objects and activities are described in cdl in the same way as we explained for reference activities and objects. figure 5 presents an example interface of the legacy object warehouse. as can be seen from this example, the legacy object warehouse offers the legacy activity material_requirements_planning. this legacy activity can be used to plan all the part requirements in the warehouse. for this purpose it uses the legacy operation forecaststochmodel (where stochastic planning is a special form of consumption-based planning, see the business operation forecast). the definitions in the legacy object warehouse will subsequently be used as a basis to construct the interface of the reference business object warehouse. 6. instantiating the meta-cdl legacy model: after the cdl specifications of the legacy components are available, they are also instantiated into the meta-model repository. 7. link phase of the cdl meta models: when both the forward and reverse engineered cdl descriptions have been instantiated by means of the meta-cdl model in conceptbase, the actual linking of business objects and activities to legacy objects and activities can take place. at first, a measure of similarity between a given cdl description of the reference model iq and (a set of) cdl description(s) of the legacy system(s) l\— in is calculated. this measure is used to identify (part of) the cdl description of the legacy components in the set that 132 ajis vol. 8 no. 1 september 2000 fit(s) best to the given cdl definition of the reference (enterprise) model. in step 8 (see below), we actually define the (partial) mapping based on these measurements. the algorithm to calculate the similarity of two (or more) cdl specification deals with type conformance issues like covariance (e.g., of the method results) and contra-variance (e.g., of method arguments). however, it leaves semantic integration of both schemes outside the scope. semantic interoperability has been extensively investigated in the field of (federated) database systems, f.e., in the context of data scheme integration. most solutions comprise (a combination of) ontologies, meta-data or contexts (kashyap 1998). semantic integration of a reference model and a reverse engineered model of a legacy system is even harder as the terminology used in both specifications is inherently totally different: reverse engineered cdl code often includes obscure, "empty" names derived from code variables, whereas the reference model is mostly stated in (standardized) business terminology. for example, the legacy variable dsag_tas_ldd that has automatically been reverse engineered into a legacy attribute of a legacy object dsag_tas in a legacy model is semantically equivalent to the attribute product_name of the business object product of a corresponding reference model. therefor, we propose to firstly compare type skeletons, that we call type trees (see figure 6), of both legacy and reference objects. later on, we will resolve any remaining semantic mismatches. in a type tree, a leaf represents a type, and a node represents a typed entity. a type-tree is a directed acyclic graph g=(n,e,s,c,r), where n is the set of nodes, and e is the set of edges. the leaves l c n of g are nodes, with no outgoing edges. the leaves are marked by simple type, all other nodes by composite type. the inner nodes in e l are called constructor nodes. the predefined set simple types includes types as integer, string, etc. the predefined set composite types includes type constructors like operation and during. both sets may be enhanced by user defined types. one can assign a role to some edges, e.g., for a node with the constructor method, attribute,return type, input type, and output type. both sets may be extended by user defined types. the function s: l ^simpletypes maps a simple type to each leaf. the function c: e-l •> composite types assigns to each constructor node a composite type. the partial function r: e ̂ roles can be used to assign a role to an edge (heuvel, w.-j. van den & reussner, r. 2000). given two type trees t j , t2 the algorithm computes a number, which is a measurement for the similarity of t j and t2 . the higher this number, the greater the similarity. the algorithm is implemented as a set of active rules and queries to compare the (linked) legacy and reference models that are stored in the conceptbase system. figure 6: the type-tree matching process for the reference model and the legacy enterprise model. figure 6 illustrates the result of the type-tree matching process for the cdl examples. at the left hand side of this figure, the type tree of the reference (enterprise) model is depicted. the right hand side expresses the reverse engineered (sap r/2) legacy system model, and has been derived from figure 5. the algorithm now calculates the match between (parts of) both type trees. the result is graphically 133 ajis vol. 8 no. 1 september 2000 expressed as the grey area at the left hand side of figure 6; the arrows point to the constructs that we compared. the linking algorithm results in an equivalence rate of 139/180 (=77%) for the business activity prognosis and the legacy activity material_requirements_planning. the equivalence rate of the business operation forecast with the legacy operation forecaststochmodel is obviously 1 (thus a full match). for details on the algorithm, we refer the reader to van den heuvel (2001). 8. cdl specification of the cross mapping: the bales methodology results in a cdl specification of reference elements in terms of their related legacy counterparts. the interfaces that are most likely to match according to the linking algorithm, now need to be checked by the designer to resolve semantic conflicts. the syntactically and semantically matched constructs thereafter need to be specified in the resulting parameterized business (/task) object(s). for this purpose we use the initial cdl specification for reference objects from step 3, in which we connect reference element specifications with links to equivalent (mappable) legacy component specifications that we identified by means of the matching algorithm. an example of such a mapping is given in figure 7. this (simplified) example defines the reference object operation forecast in terms of the legacy operation forecaststochmodel (part of the material_requirements_planning legacy activity/program) which is embedded in the business object by means of the linking operator ->. this linking process is followed for each legacy system that shall be integrated into the common reference model. figure 7: the cdl specification for the reference business object's operation forecast in terms of the activity prognosis relationship borpc ispartof request_part_control inverse bpp ; relationship wflr many 0..* request_art_workflow inverse bpp ; attribute string consumptionhistory; attribute date consumptionperiod; attribute integer expectedspecialsale; attribute integer totalexpectedconsqualityinperiod; void manualreorderplanning (in integer partid, in integer stockid, in integer warehouseld); during (prognosis==forecasting) withforecast attribute integer partid; attribute integer stockid; // mapping of forecasting method to legacy process component mrp void this.forecast --> warehouse.material_requirements_planning. forecaststochmodel(in integer partid, in integer stockid, in integer warehouseld, in date consumptionperiod, in string consumptionhistory) / apply statetransitionrule start_forecasting trigger = register_expected_stock() ; source = initial; target = forecasting , ; // end: prognosis legacy operation forecaststochmodel. conclusions and future research in this paper, we present a top-down approach to enterprise application integration, whereby reference models are used as starting point for the integration process. the linking of reference and legacy models combines forward and reverse engineering techniques employing the bales methodology. a resulting cross-mapping specification defines the mapping from the reference model to the individual legacy models. the bales methodology has as its main objective to inter-link parameterizable business objects to legacy objects. legacy objects serve as conceptual repositories of extracted (wrapped) legacy data and functionality. these objects are, just like business objects, described by means of their interfaces rather than their implementation. business objects in the bales methodology are configured 134 ajis vol. 8 no. 1 september 2000 so that part of their implementation is supplied by legacy objects. future research includes considering similarity weights in the matching algorithm. the reference models serve as the starting point in the integration process with the top-down approach. the overall integrated system will be more scalable than with the bottom-up approach, because the integrated reference models do not grow linearly with the number of component systems. the decentralized responsibility for maintaining the cross-mapping specifications reduces the central coordination needs and distributes the maintenance cost. starting with the reference models should avoid changes to the fundamental structure of these models, making the integration more usable and scalable. for integrations with a small number of local/component systems, the top-down approach may not offer the optimal solutions, but for integrations with a large number of connected systems, we will obtain a more usable and maintainable overall systems architecture. in practice, we can also expect a yo-yo approach, as discussed in hasselbring (1999): the integration process alternates with bottom-up and top-down steps. for instance, the bottom-up process may provide input for extending the reference models. in the presented example, that was taken from a project with the department of defense in the netherlands, it turned out that the existing reference models (the so called defense and aerospace solution maps (sap 2000)) for sap r/3 did not cover all requirements of this dutch setting. these problems are currently addressed by sap through extending thenrespective reference models according to these additional requirements. the development process must take feedback, which is based on experience with actual applications, into consideration. anyway, it is important to start at the top (with the common models). to take the analogy of the yo-yo toy: when the game starts, the reel should be coiled up. references abinavam, s. et al. (1998) san francisco concepts & facilities. technical report sg24-2157-00, ibm international technical support organization, 1998. booch, g. et al. (1999) unified modeling language user guide. object technology series. addison-wesley, reading, ma, 1999. dat (1997) data access technologies, omg business object domain task force,. business object architecture (boa) proposal bom/97-11-09, 1997. dodn (1997) department of defense netherlands. methodiek voor bet inrichten van de informatievoorziening op basis van bouwstenen ten beboeve van bet ministerie van defensie, 1997. eeles, p. & sims, o. (1997) p building business objects. john wiley & sons, new york, 1998. hasselbring, w. (1999) top-down vs. bottom-up engineering of federated information systems. in engineering federated information systems (proc. efis'99), pages 131-138. infix-verlag, 1999. heuvel, w.-j. van den & reussner, r. (2000) matching component interfaces, technical report infolab, tilburg university, the netherlands and interner bericht at the department of informatics, university of karlsruhe, germany hordijk, w. et al. (1998) working with business objects: a case study. in oopsla'98 business object workshop iv. springer-verlag, 1998. jeusfeld, m. et al. (1998) conceptbase: managing conceptual models about information systems. in p. bermus, k. merlins, and g. schmidt, editors, handbook on architectures of information systems, pages 265-285. springer-verlag, 1998. v. kashyap and a. sheth (1998) semantic heterogenity in global information systems: the role of metadata, context and ontologies. in papazoglou and schlageter, editors, cooperatve information systems, academic press, 1998. keller, g. & teufel, t. (1998) sap r/3 process-oriented implementation, addison wesley longman 1998 papazoglou, m.p & russell, n.(1995) a semantic meta-modeling approach to schema transformation. in n. pissinou, a. silberschatz, e. k. park, and k. makki, editors, cikm-95: int'l. conf. on information and knowledge management, baltimore, maryland, 1995. acm press. papazoglou, m.p. & van den heuvel, w.-j. (2000) leveraging legacy assets. in m. papazoglou, s. spaccapietra, and z. tari, editors, advances in object-oriented modeling. mit-press, 2000. (in press). sap (2000) sap aerospace & defense, available at http://www.sap.com/products/uidustry/aero/. sheth, a. & larson, j. 1990) federated database systems for managing distributed, heterogeneous, and autonomous databases. acm computing surveys, 22(3): 183-236, 1990. 135 ajis vol. 8 no. 1 september 2000 van den heuvel, w.-j. (2001) integrating business applications with legacy systems. phd thesis, tilburg university, 2001. (in preparation). 136 a.iis vol. 3, no. 2 grammatical conversion of descriptive narrative an application of discourse analysis in conceptual modelling bruce a. calway & james a. sykes centre for information systems research school of information systems swinburne university of technology bcalway@gpo.swin.edu.au p.o. box 218, hawthorn vic. australia, 3122 phn +61-3-214-8451 fax+61-3-819-1240 abstract keyword codes: e.4, h.i.i keywords: discourse analysis. conceptual modelling, fact abstraction. fact-oriented conceptual modelling begins with the search for facts about a universe of discourse (uod). these facts may be obtained from many sources, including information systems reports, tables, manuals and descriptive narrative both verbal and written. this paper presents some initial findings that support the use of discourse analysis techniques as an approach to developing elementary fact based sentences for information systems conceptual schema development from written text. although this discussion paper only considers the niam (fact-oriented) conceptual schema modelling method, the is087 report from which the research case study is taken describes other conceptual methods for which the research contained in this paper could be applicable (e.g. entity relationship analysis). the case study could be modelled exactly in the form in which the text is initially found, but grammatical analysis focuses consideration on alternative, potentially better, expressions of a sentence, a theme which is described and demonstrated. as a result of having applied grammatical sentence simplification with co-ordinate clause splitting, each sentence could be expressed as a complete, finite, independent collection of declarative simple statements. the outcome from the application of the techniques described provides at a minimum a discourse analysis of descriptive narrative which will have retained its meaning and contextual integrity while at the same time providing a simplified and independent clause representation for input to the fact-oriented conceptual schema modelling procedure. problem description and research outline context fact-oriented conceptual modelling of information systems begins with the search for facts about the universe of discourse (uod), expressed as elementary declarative sentences in natural language. these facts may be obtained from many sources, including reports, tables, manuals and descriptive narrative both verbal and written. this paper presents some initial findings that support the use of discourse analysis techniques a way of extracting elementary facts (ie. simple relationships which exist between people, places and things) from written text. nijssen and halpin state (nijssen 1989:31) the fundamental approach to building a conceptual schema as taking "specific examples" then populating the resultant schema with instances for validation purposes. the specific examples may appear in many forms, or combination of forms including graphical, tabular, textual, etc. it is often a role of the systems analyst to take such disparate material and uod discourse as the input to the conceptual schema development process. one step which the analyst may undertake is to collect discourse to describe in detail the uod from the perspective of the various information systems agents (e.g experts, users, developers). discourse both written and verbalised (spoken) can provide a wealth of knowledge about the organisation's uod and the application of information systems within that uod falkenberg (nijssen 1989:31) is credited with proposing the idea to base conceptual schema development on elementary natural language sentences, a proposal which is said to have been influenced by the grammatical work of c.j. fillmore. despite such work there persists a general perception that natural language lacks structure or is too complex for analysis purposes. this is not necessarily supported if only on the basis that if this were true no individual could understand communicative discourse. sykes (sykes 1994), andersen (andersen 1991), stamper (stamper 1992b), halliday (halliday 1994), have shown that natural language, although very complex in structure, contains considerable detail available for analysis. the international organization for standardization released a technical report 9007 (iso87) which outlined the concepts and terminology for developing conceptual schema models for information systems specification. within that report was a detailed theoretical foundation for describing a universe of discourse (uod) which was defined as "the collection of all objects that ever have been, are, or ever will be in a selected portion of a real world or postulated world of interest that is being described" (is087:12). an integral part of that description 10 a.i is vol. .vnn. 2 was the formal system of logic to describe the uod (18087:47-48) expressed at two levels ie. syntactic and semantic. an analyst conducting a system analysis of an organisation is likely to seek out at least three forms of discourse, namely: • prescriptive discourse; which expresses an organisation's work practices formally as manuals, process specifications, etc., • descriptive discourse; where an individual or group provides a description, in prose form, of pan or all of an organisation's uod, • interview discourse; written records (transcripts, notes, etc.) of an interactive dialogue of questions and statements about an organisations uod, between some individual/s or group/s from within the uod and an individual or group investigating the uod in order to describe aspects of that organisation's uod. all would seem possible sources relevant to the application of discourse analysis techniques. a collective descriptive discourse can be developed for the uod by restatement as prose from prescriptive, interview and descriptive sources, for communicating structures and meanings through a written textual medium. given that natural language discourse, as written text, could be valuable in the analysis and design process when developing information systems using information technology, the question then arises as to an appropriate approach for investigating and restating, as simple facts for conceptual model development, the content and contextual expression of the discourse. problem description and research method the success of the analysis and development of conceptual schema specifications for a uod depends upon completeness and accuracy of the uod fact discovery. as stated earlier fact-oriented conceptual schema modelling begins with the search for facts about the universe of discourse (uod). when the analyst must search written text, fact-oriented methods like niam offer no guidelines for discovering these facts (other than that the analyst knows what they should look like, and how to verify them when discovered by getting agreement from the uod expert (nijssen 1989)). prior work by sykes (sykes 1994) has shown that the study of the grammar of natural language sentences has value for the early stages of schema development. it was also shown that a syntactic grammatical model of english sentences is applicable. the present work can be seen as seeking to extend that line of study, which only considered a single sentence/clause, to dealing with groups of complex sentences in a descriptive narrative expressed as written text. the larger research project, of which this paper forms a part, proposes: that techniques from the field of discourse analysis, heing the relevant aspect of linguistics, might offer a repeatable methodical way for a systems analyst to deal with uod descriptions when found as written textual discourse. for this paper it is proposed to survey the discipline of discourse analysis initially to find individual techniques that appear suitable, and not to be concerned at this point that a method necessarily be the best available. each method selected will individually be applied to a selected case study (below) and evaluated for results obtained, specifically for the ability to: • allow the separation of clauses within compound sentences, • identify and provide for inter-clausal referential identification without loss of meaning, • deliver independent clauses, while retaining the narrative sequence of the clauses in the text. discourse analysis discourse analysis is concerned with the understanding of bodies of written or spoken text. some of its techniques are aimed at simplifying complex sentences, the subject of this current study. the use of such techniques can yield simple declarative clauses (ie. simplest form of sentence) without loss of meaning. therefore discourse analysis can be a useful addition to fact-oriented conceptual schema development processes. subsequently, if the methods can be easily understood and applied, they can be taught to systems modelling analysts. 11 a.iis vol. 3, no. 2 a case study based approach is taken for this work with the discourse analysis being limited in scope to a single discourse case, ie. the is087 technical report example uod. the selected case study is well documented in the technical report and provides an established framework of definition and analysis for conceptual schema development using various conceptual schema methods. case study "a car is of a particular model and is given a serial number by its manufacturer that is unique among the cars made by that manufacturer. the manufacturer is registered as the owner of the car as soon as practicable. at this time it is given one registration number, unique for all cars and for all time. the year of production is also recorded. during the month of january only, a car may be declared to have been produced in the previous year. eventually a car is destroyed and the date of destruction is registered. the history of a car must be kept until the end of the second calendar year following its destruction." (18087:77) discourse analysis framework discourse analysis and simple terms discourse analysis is generally considered as "the analysis of language in use" and as such is dependent upon the description of language usage within the uod (e.g. work language, etc.). the methods for studying the diversity of discourse are extensive and therefore for this discussion have necessarily been constrained to specific domains of language analysis. the subsequent discussion is limited to written text as declarative narrative with the discourse analysis techniques arbitrarily drawn from the syntactic and semantic disciplines as described by quirk et al. (quirk 1985), halliday (halliday 1994), nesfield (nesfield 1936), eggins (eggins 1994), winter (winter 1982). choosing a path which allows the discourse analysis techniques to be selected is first clouded by the issues surrounding the definition of "sentence". the problem arises when groups of sentences are to be considered, because of the need to have a workable grammar of english besides the linguistic grammar of phonics, morphemes, words, phrases, etc. as halliday (halliday 1994) points out it is often assumed that discourse analysis can be undertaken without grammar and states that discourse analysis that is not based on grammar is not an analysis at all. a contextual grammar for describing sentences is necessary in order for real-world (day to day) communication which will use clauses and sentences in unfamiliar ways as part of written or spoken discourse. grammarians (nesfield 1936, winter 1982, quirk 1985, et al.) traditionally have considered sentences in terms of form or class labels ranked as word (ie. parts of speech as noun, article, etc.), phrase/group (ie prepositional phrase, adverbial phrase, etc.) and clause (finite, non-finite, dependent clause, etc.). the independent clause (single subject with single predicate) is often referred to as the minimum sentence (or simple sentence). this approach is adopted by sykes (sykes 1994) when considering the niam conceptual modelling method, displaying that the use of english grammar expressed by quirk et al. was of value. quirk et al. (quirk 1985), as an alternative grammatical classification, subdivide a clause into a series of functional parts (ie subject, verb, object, complement, adverbial (svoca)). nijssen and halpin (nijssen 1989) state, as an objective, the development of facts about a uod. these facts are expressed in a very structured format which reflects two known grammatical representations of a sentence (ie. subject/predicate, and subject/verb/object, ref. below). nijssen and halpin describe these facts as elementary sentences which therefore could reasonably be expected to respond to discourse analysis in terms of these grammatical representations. the two english sentence representations under consideration are: • subject predicate; where it is implied that a single subject is expressed or implied and that the predicate expresses something about the subject referred to in the sentence. one can regard the subject/predicate form as focussing on the logic of the sentence. • subject, verb, object, complement, adverbial (svoca); where the sentence is seen to operate as functioning parts, the simplest being subject and verb e.g. "dogs bark". the svoca analysis emphasises the grammatical structure. 12 a.i is vol. 3, nci. 2 during this study what is being sought from the application of discourse analysis are elementary facts expressed as independent descriptive or declarative clauses. a brief study of grammar basics will assist in clarification of the use of these terms and also point to the reasons for their use within this paper. this is due to the explicit limitations in meaning for each term used. the following definitions represent a synthesis of those offered by kerl (kerl 1985), kellogg (reed 1886), nesfield (nesfield 1936), quirk (quirk 1985)etal. • discourse any series of thoughts embodied in language which may be: • descriptive an account of persons, places and things, • narrative chiefly temporal as a series of events and states, • illustrative generally rhetorical and used to make the meaning more intelligible or impressive. (nb. any or all forms may exist in a single piece of discourse) discourse can be taken as a sentence or a series of sentences (e.g. paragraphs) (series here is used advisedly as the removal or restructuring of sentences within a series may alter the contextual meaning, a point discussed later in this paper when considering sentence analysis and conversion). the division of discourse can also be by volumes, books, pans, cantos, verse, chapters, sections, paragraphs, sentences, and is generally left to the writer's own tastes. of interest to this study are: • sentence being a thought expressed by at least one subject/predicate clause, or a union of clauses, followed by a full pause (e.g..!? etc.). • clause a sentence as a whole, or that makes part of a larger sentence. a sentence or clause expresses at least three forms: • declarative an expression about something, • interrogativeexpressing a question, • imperative expressing a command, permission, etc. kerl (kerl 1985:229) suggests a further option ie. exclamatory which could be seen as the cooperation of interrogative and imperative. also many other compounded forms may exist e.g. declarative and interrogative, etc. sentences may also be divided into classes: • simple having only one clause expressed or implied, • complex being a sentence containing one independent or principal clause with one or more dependent clauses, • compound being a sentence that has two or more independent clauses. it should be noted that the compound sentence may consist of two or more complex sentences or a mixture of all classes. fact-oriented sentence therefore defmitionally what would seem to be required for fact-oriented conceptual schema development using discourse as simple sentences is: single clause sentences to express a proposition which is declarative in form and descriptive and/or narrative by nature. the recommended practice for interpretation of source material into fact-oriented elementary sentences (simple sentences) (nijssen 1989:32-48) for example, is that elementary facts should emerge from the analysis of natural language utterances about a uod that can be or have been validated by a uod domain expert. the word "fact" indicates that the system is to treat the assertion contained in the sentence as being true of the uod whether or not this really is so. a "simple sentence" in english as with fact-orientation is one which has only one finite verb expressed or understood, although there may exist many verbs within the sentence. the sentence following is an example of a simple sentence in english with a single subject/predicate structure. it is the intent of niam conceptual modelling (nijssen 1989) to have a subject/predicate structure, but only for a single clause (ie. 13 a.jis vol. 3, no. 2 no subordinate or coordinate clauses either existing or implied in the sentence) of the svoca form as sykes points out (sykes 1994). • simple sentence with single subject/predicate e.g . subject the manufacturer, having a large volume of cars to sell predicate caused all of its stock to be conveyed by train, there being no market in that area. in this sentence there are five verbs; having, to sell, caused, to be conveyed, being. of these "caused" is to be taken as the finite verb, as it is the only verb out of the five which has or could have a subject attached to it, but there exist several phrases which could be converted to (dependent) clauses or even made independent. one objective in creating sentences for input to the fact-oriented conceptual modelling method would be to create sentences which as a worst case contain co-ordinate clauses with no subordinate clauses existent. a co-ordinate clause is one that can be separated from other clauses to make an independent sentence which gives an independent sense. it is possible to convert simple, compound and complex sentences into purely co-ordinate clause sentences (nesfield 1936, kerl 1985, et al.) as shown later in this paper. sentence analysis and conversion the discourse analysis approach which will be applied to the case study and provide the balance of the discussion in this paper is: • grammatical conversion where sentences are grammatically modified to remove subordinate clauses and provide either simple (one clause) or co-ordinate clauses (self contained but contextually connected). winter (winter 1982) has laid a foundation which has been followed within this discussion where he investigates and redefines "sentence" in terms of the clause. that definitional work is founded upon work by such significant grammarians as fries, halliday, kerl, kellogg, jespersen, quirk, et al. in attempts to bridge the dichotomy between clause and sentence structure within discourse. the definition of sentence offered by winter (winter 1982) and halliday (halliday 1994) argues that the term "clause complex", which refers to the clause relationship as a sentence, is taken as the starting point for discussion. therefore, the clause becomes the significant first level decomposition of a sentence and the svoca structure (ie. subject, verb, etc.) offered by quirk et al. becomes the relevant grammatical structure of a clause. the current study is limited to descriptive discourse because the discourse considerations for analysis of information systems are normally descriptive representation of the uod and declarative in nature. the structure of declarative sentences can be very complex with subordinate and coordinate components producing implied meaning through grammatical structure as is the scenario with the first sentence of the case study where the implication is that there is a relationship between a car, the model and the manufacturer. winter, eggins, kerl and nesfield (winter 1982, eggins 1994 kerl 1985, nesfield 1936) each offer similar principles for grammatical conversion of complex declarative sentences into simple sentences; approaches which may prove beneficial to sentence structuring for elementary sentence development and conversion. in the first sentence from the case study "a car manufacturer" there are four verbs or verb groups. of these only one has or could have a subject attached to it. therefore since there is only one finite verb the sentence could be considered simple but in reality is not because of the clause structure. a simple sentence (clause) could be "a car is of a particular model". in order to convert from simple sentences to multiple co-ordinate clauses nesfield and kerl among other authors suggest (nesfield 1936:143,152/3, kerl 1985) the following substitutions which can occur: • simple sentences can be converted into multiple co-ordinate clauses, by expanding words or phrases into clauses joined with co-ordination conjunctions or connectives (e.g. and, before, which). • cumulative connective (adding) simple besides making a promise, he kept it. 14 a.iis vol. 3. no. 2 multiple he not only made a promise, but also kept it. • alternative connective (offering a choice) simple he must confess his fault to escape being fined. multiple he must confess his fault, or he will be fined. • adversative connective (expressing a contrast or a difference) simple notwithstanding his sorrow, he is hopeful. multiple he is sorrowful, but yet hopeful. • illative connective (drawing an inference) simple owing to bad health, he could not work. multiple he was in bad health, and therefore he could not work. simple sentences as seen can be converted to that of double or multiple co-ordinate clauses (should more than one clause be observed in the sentence), by expanding words or phrases to form the co-ordinate clauses. similarly, to convert complex sentences to that of double or multiple clauses, the main clause must be placed last, and the subordinate clause/s must be placed first (nesfield 1936 pp 152-153. 162). • complex to multiple (main clause last). complex i am certain that he will not recover. multiple he will not recover, and of this i am certain. a further grammatical simplification (eggins 1994:134) which can be considered is to divide a clause into its constituent parts and decide when a particular component (phrase or group, as pan of a clause) is operating at clause level. three tests are available: • movability where the component is independently moveable wi thin a sentence, • substitution where a clause or phrase can be replaced with a single substitute verb or pronoun. this is achieved by asking what the substitution item is standing for. • probe questions which allow testing for verb groups, prepositional phrases, adverbial elements, etc. there are three kinds of subordinate relationships recognised (nesfield 1936:155) and therefore available for expansion as co-ordinate clauses: • noun clauses acting as a noun in relation to some word in some other clause, • adjective clauses acting as an adjective in relation to some word in some other clause, • adverb clauses acting as an adverb in relation to some word in some other clause. even after dealing with issues of grammatical conversion, problems of referential packing still exist (ie. packing being the avoidance of needless repetition of the words within a clause or sentence), bearing in mind the targeted output from any analysis is a set of sentences of no more than one clause expressed or implied, therefore: • all subordinate clauses within a sentence must be removed. • each clause within a sentence must be made independent of all other clauses. when analysing sentences in consideration of this latter point, for the purpose of creating independent clause sentences, the following referential unpacking substitutions suggested by nesfield et al. must be completed (nesfield 1936, kerl 1985): • when there are two finite verbs to the same nominative, the nominative is not usually mentioned more than once, but it must be repeated in the analysis. e.g. the sun rose and (the sun) filled the sky with light. 15 a.iis vol. 3, no. 2 • when there are two nominatives to the same finite verb, the finite verb is not usually mentioned more than once, but it must usually be repeated in the analysis. e.g. either this man sinned or his parents (sinned). however with some sentences the verb must not be repeated, because nominatives are seen as a collective. e.g. truth and honesty is the best policy. « the omission of the conjunction "and" can often be seen when the author is stating a single fact with a string of related objects. e.g. a steam engine can propel, elevate, pump, (and more thing than can be enumerated.) a steam engine can propel and a steam engine can elevate, etc. case study examined using grammatical conversion (a) here the original case study is reproduced with sentence sequencing, ie. numbering for ease of identification in subsequent analysis, as grammatical conversion can alter the structure of the sentence: 51 a car is of a particular model and is given a serial number by its manufacturer that is unique among the cars made by that manufacturer. 52 the manufacturer is registered as the owner of the car as soon as practicable. 53 at this time it is given one registration number, unique for all cars and for all time. 54 the year of production is also recorded. 55 during the month of january only, a car may be declared to have been produced in the previous year. 56 eventually a car is destroyed and the date of destruction is registered. 57 the history of a car must be kept until the end of the second calendar year following its destruction. (b) one possible solution using grammatical conversion of the first sentence si of the above (a) is therefore: 51 a car is of a particular model and (a car) is given a serial number by its manufacturer that (serial number) is unique among the cars made by that manufacturer. • this sentence has two co-ordinate clauses and one relative (dependent) clause, therefore this is a complex sentence. • there are a number of finite verbs to choose from, but if one were to consider one which was pivotal it is "is given". (c) one possible solution using grammatical conversion of sentences s2-s7 of the above (a) is therefore: 52 the manufacturer is registered as the owner of the car as soon as practicable. 53 one registration number is given (to a car) at this time and (the registration number) is unique for all cars and (the registration number) is for all time. 54 the year of production (of a car) is also recorded. 55 if the month of production (of a car) is january only, a car may be declared to have been produced in the previous year. 56 eventually a car is destroyed and the date of destruction (of a car) is registered. 57 the history of a car must be kept. [until the end of the second calendar year following (a cars) destruction.] 16 a.i is vol. 3, no. 2 by applying grammatical conversion, a complex and/or compound sentence structure can be changed to create a double or multiple co-ordinate clause structure, but co-ordinate clause sentences still do not comply with the structure of elementary facts where an elementary fact has a subject and single predicate as a single independent clause. second, clauses and phrases contained within a discourse have pronouns and inferential or thematic referential connections and while this allows ease of reading it hinders elementary fact development when sentences are grammatically converted and then considered independently, ie. outside of the discourse context, as would be the case for sentence s4. subordinate clauses and sentences are thematically and inferentially connected as is the case for sentences si s2, s2 s3 s4, s2 s5, etc. further, clausal items may refer to each other but with differing lexical items as in si "car is made" and s4, s5 where the reference is to "production". therefore, although clauses may be separated and made independent this does not provide the referential identification which is required between clauses within a sentence or descriptive narrative. • the first finding is that sentences which are analysed and converted from complex or compound into co-ordinate clauses require a degree of observer (analytical) intuition to derive independent elementary facts based upon the observable co-ordinate clauses structure contained in the sentence. • the second finding is that no clause or sentence can be made independent of the discourse in which it is found by the application of grammatical sentence conversion alone. this finding identifies the need for referential identification to occur prior to, or at a minimum in parallel with, grammatical clause analysis. the resulting list (above) of clauses is a declarative narrative expression of the independent clause/phrase structure to be found within the case study text. each clause contains an independent meaning (albeit intuitively derived for referential identification) in that a clause can now be taken out of sequence without loss of referential meaning and modelled as an elementary fact. if a sentence is individually considered it can be observed that contextual meaning is still required to be interpreted as referential constraints, e.g. (eventually a car is destroyed) where, as a sentence it syntactically correct and semantically plausible, but out of context is functionally meaningless. summary and conclusions the research project, of which this discussion paper forms a part, proposed: that techniques from the field of discourse analysis, being the relevant aspect of linguistics, might offer a repeatable methodical way for a systems analyst to deal with uod description when found as written textual discourse. it has been shown that the grammatical approaches taken do: • allow the separation of clauses within compound sentences to be delivered as a series of independent co-ordinate clauses for a single discourse as written text, • provide a methodical approach of discourse analysis for a system analyst to deal with uod descriptions, • highlight the information packing and shortcomings inherent within the complex sentences. grammatical reconstruction of discourse into elementary sentences could prove difficult on its own for analysts who are not familiar with the rules of english grammar, but it is believed that the methods when documented fully could be taught to systems analysts faced with using descriptive narrative for derivation of conceptual schemas or schema verification. in summary, it can be concluded that methods of analysis are available within the linguistics discipline which when applied to descriptive narrative allow the derivation of simple sentences of the svoca form. although sentences in the case study narrative could be modelled from the descriptive narrative form in which they were initially found, grammatical analysis methods focus consideration on alternative, potentially better, expressions of a sentence, a theme which has been described and to some extent demonstrated through the analysis processes described within this paper. 17 a.iis vol. 3, no. 2 references andersen p.b.(1990) a theory of computer semiotics. cambridge university press (0-521-39336-1) andersen p.b.;(1991) a semiotic approach to construction and assessment of computer systems. (in nissen h., klein h.k., hirscheim r. (ed);(1991) information systems research: contemporary approaches & emergent traditions. elsevier science pub nth-holland.(ifip tc8avg8.2 dec 1990):465-514) avison d.e., fitzgerald g.;(1988) information systems development methodologies, techniques and tools. blackwell scientific 1993 reprint (0-632-01645-0) chen m, nunamaker j.f., weber e.s.;(1989) the use of integrated organisation and information systems models in building and delivering business application systems. ieee transactions knowledge and data engineering vol. 1 no. 3 sep. 1989 clarke r.j.;(1992) some applications of social semiotics in information systems discipline and practice. isop proceedings of the third australian conference on information systems, wollongong nsw.oct. 1992: 67-80 deely j.(1990) basics of semiotics. indiana university press (0-253-20568-9) eggins s.(1994) an introduction to systemic functional linguistics pinter pub. (1-85567-209-x) filltnore c.j.(1968) the case for case in universal in linguistic theory, bach e. and harms r.t. eds. holt, reinhart and winston inc. pp2-88. goguen j.a.(1992) the dry and the wet. in falkenberg e.d., rolland c., el-sayed e.n. eds information systems concepts : improving the understanding. elsevier science pub nth-holland. (ifip tc8/wg8.1 apr 1992) pp 1-17 halliday m.a.k.;(1994) introduction to functional grammar. 2nd ed. edwards arnold. hatim b., mason i.;(1990) discourse and the translator longman (0-582-05925-9). hawkes t.;(1977) structuralism and semiotics. routledge uk 1991 edition iso/tr 9007; (1987) information processing systems concepts and terminology for the conceptual schema and the information base. international organization for standardization (ref.no.iso/tr 9007:1987). jackendorf r.;(1990) semantic structures. mit press (0-262-60020-x) kbmt project goodman k., nirenburg s. eds.;(1991) a case study in knowledge-based machine translation morgan kaufman pubs. (1-55860-129-5). kerl s; (1985) a common-school grammar of the english language (1878). scholars' facsimiles & reprints, delmar ny. lindgreen p. ed;(1990) a framework of information systems concepts. interim report ifip wg8.1 task group frisco the netherlands. nesfield j.c.;(1936) outline of english grammar macmillan and c/nijssen g.m., halpin t.a.;(1989) conceptual schema and relational database design a fact oriented approach. prentice hall (0-7248-0151-0) nijssen, g.m.,(1989) "an axiom and architecture for information systems", in falkenberg, e.d. and lindgreen, p. (eds.), information systems concepts: an in-depth analysis, elsevier,. nord c.;(1991) text analysis in translation rodopi amsterdam (90-5183-311-3) palmer f.r.;(1981) semantics 2nd ed. cambridge university press 1988 reprint (0-521-23966-4) palmer m.s.;(1990) semantic processing for finite domains cambridge university press (0-521-36226-1) quirk r., greenbaum s., leech g., svartvik j.;(1985) a comprehensive grammer of the english language longman (0-582517-346) reed a, kellogg b; (1886) higher lessons in english 1886. scholars' facsimiles & reprints, delmar ny. riet van de r.p., meersman r.a. eds;(1992) linguistic instruments in knowledge engineering. elsevier science pub nth-holland. stamper r.k.;(1992) on a framework of information systems concepts. (in falkenberg e.d., rolland c., eisayed e.n. eds;(1992) information systems concepts : improving the understanding. elsevier science pub nth-holland. (ifiptc8/wg8.1 apr 1992) pp 165-166) stamper r.k.;(1992a) signs, organisation, norms and information systems. (in isop (1992); proceedings of the third australian conference on information systems, wollongong nsw.oct. 1992:21-66) stamper r.k.;(1992b) language and computing in organised behaviour. (in riet92 pp!43-163) sykes j.a.;(1994) english grammer as a sentence model for conceptual modelling using niam (working paper) swinburne university of technology, centre for information systems research. 18 a.i is vol. 3. no. 2 van bommel, p., ter hofstede, a.h.m., van der weide. th. p.,( 1991) semantics and verification of objectrole models, information systems, vol. 16, no. 5:. 471-495 van dijk t.a.;(1986) text and context explorations in the semantics and pragmatics of discourse longman (0-582-29105-4) verheijen, g.m.a., van bekkum, j.,(1982) "niam: an information analysis method", in olle, t.w. et al. (eds), information systems design methodologies: a comparative review, north-holland wilson b.;(1990) systems: concepts, methodologies, and applications. 2nd ed. john wiley & sons(()-47192716-3) winter e.g.; (1982) towards a contextual crammer of english. allen & unwin (0-04-425027-4) wintraeken, j.j.v.r.,(1989) the niam information analysis method: theory and practice, kluwer. 19 editorial this is the first issue of volume 5 of the australian journal of information systems. as editor, i am pleased to report the on-going interest in the journal, both from authors and more recently from a number of international conferences who have kindly agreed to advertise the journal to their participants. our first paper a comparison of five alternative approaches to information systems development rudy hirschheim, university of houston, juhani livari, university of oulu and heinz klein, suny university binghampton compares and contrast the major developmental approaches in the light of emerging technologies. the authors provide both background as well as future trends in the approaches. our second paper assessing software cost estimation models: criteria for accuracy, consistency and regression bruce lo and xiangzhu gao provides two new models: the weighted mean of quartiles and die standard deviation of the ratios of estimates. the results suggest that these new measures may overcome some of the hitherto found difficulties. our third paper is entitled a pilot study of small business's perception of vendor provided services: are these associated with small business it educational requirements? rob macgregor & deborah bunker, university of wollongong, joan pierson and karen forcht, james madison university. this paper examines the type of it training required by small business managers and questions whether the type of required training is associated with satisfaction with vendor services. graham pervan, curtin university of technology provides our next paper entitled information systems management: an australasian view of key issues 1996. this paper is part of a longitudinal study. the key issues revealed include the need for a technology/business and people skill mix in is executives. our next paper the role of conferences and refereed journals in australian information systems research by gail ridley, university of tasmania reports on a survey conducted across australian academics as to their perceptions concerning conferences and journals. the paper also compares its results with north american data. our final paper is entided executive support of information technology and information systems in australian hospitals: an empirical study by graeme rose, charles sturt university and robert reeve, macquarie university. this paper examined the concepts of executive participation and executive involvement and their relationship on progressive use of it in australian hospitals. the results showed that there was a strong relationship between executive involvement and progressive use of it. once again i would like to pass on my thanks to the authors and consulting editors for the high quality of material in this issue. i would also like to thank david dodds for his assistance in producing final copy of the journal and the maintenance of the home page. r. macgregor editor ajis vol 5 no 1 september 1997 ajis vol. 4 no. 2 may 1997 ethics and information technology simon rogerson centre for computing and social responsibility de montfort university the gateway leicester lei 9bh uk email: srog@dmu.ac.uk web site: http://www.cms.dmu.ac.uk/ccsr/ introduction the versatility and adaptability of information technology (it) offer many potential benefits to society, its organisations and its citizens, but there are also associated risks. the social and ethical implications of this technology warrant special attention and have resulted in the creation of information ethics as a discrete area of study. the overall goal of information ethics is to integrate it and human values in such a way that it advances and protects human values rather than doing damage to them. information ethics activity work in information ethics has, until recently, been dominated by the usa. in the last two years there has been an increase in activity in europe due to a number of contributory factors including the european union's commitment to establishing a european information society, several national initiatives promoting information ethics work and the occurrence of several major information systems disasters. there has been similar activity in australia. there now exists an informal global network of information ethics researchers. several applied ethics centres include information ethics in their activities, and there are individuals in computer science, information systems, philosophy and other university departments undertaking work in this area. in the uk the formation of the centre for computing and social responsibility (ccsr), the only uk-based centre of its kind, has provided a focus for information ethics activity. two years ago ccsr launched the ethicomp conference series to provide a forum to debate the social and ethical implications of it application. ethicomp95 was held in march 1995 at de montfort university, england. in november 1996 ethicomp96 was held at the pontifical university of salamanca in madrid, spain in association with the complutense university of madrid, the research center on computing and society in the usa and ccsr. there were over 300 participants representing 20 countries from six continents including delegates and three speakers from australia. it is planned to hold the next conference in march 1998 at erasmus university in the netherlands. three papers have been selected from ethicomp96 which illustrate some of the issues within information ethics. papers in ethical issues in network system design duncan langford discusses how the technical design of computer systems might be influenced by ethical issues and how there is often pressure on practitioners to build systems in ways that might be perceived as unethical. he highlights several ethical issues associated with network design and use, and suggests that the technical design cycle should include explicit consideration of ethical issues. one of these networks, the internet is the largest and most dynamic information system yet devised and consequently has many ethical dilemmas associated with it. the second paper is living with the internet: ethics in the noosphere by emma rooksby and diarmuid pigott. they argue that it is a misconception to regard the internet as a network of individuals pursuing self-indulgence, self-expression or self-improvement. rather it should be thought of as a high level single entity where the focus is on co-operation and community. they suggest that the ethical focus regarding the internet should involve both value-based ethical norms and a set of explicit rules that delineate injustice and determine which new behaviours are virtuous or otherwise. it is the software systems of the internet that are used by daniel salber in the need for an applied computer ethics handbook to illustrate the ethical problems software designers encounter when developing systems. he argues that the internet is just the forerunner of more innovative applications which will have even more inherently difficult ethical issues associated with them. he suggests that there is an urgent need for better ethically educated practitioners and the provision of practical guidance perhaps in the form of an practitioner's ethics handbook in order to prepare for an even more ethically charged it future. 125 ajis vol.4 no. 2 may 1997 conclusion social and ethical consideration is paramount in realising a democratic and empowering technology such as it. there is a growing awareness that such consideration must take place and must be explicit in nature. both the issues to be considered and the manner in which they are considered must be addressed so that effective action occurs leading to ethical sensitivity in it development and application. it is only then that systems will be socially acceptable by design rather that by accident which is often the case currently. 126 defining system requirements: a critical assessment of the niam conceptual design procedure peta darke and graeme shanks department of information systems, monash university, p.o. box 197, caulfield east, melbourne, australia, 3145. email:pdarke@ponderosa.is.monash.edu.au abstract requirements definition is a fundamental activity within information systems development. social and organisational issues are at the centre of many of the problems experienced during the development and implementation of information systems, and these need to be explored during requirements definition. the niam conceptual schema design procedure (csdp) is a method for identifying and describing information requirements using fact types. this paper discusses some limitations of the information requirements definition step of the csdp which result from its lack of focus on the socio-organisational dimension of information systems development. four different approaches to exploring the socio-organisational contexts of systems are discussed. it is proposed that one of these, viewpoint development, be incorporated into the niam csdp to provide a means of exploring and understanding a system's socio-organisational context and to ensure that contextual information is a major input to the requirements definition process. this results in an enhanced design procedure. future and current research areas are identified. introduction requirements definition is a fundamental activity within information systems development. inadequate and incorrect requirements definitions are known to be a major source of difficulties for large systems projects (boehm 1981). it has been suggested that the so-called "hard" or "engineering" type approaches to systems development emphasise the technical aspects of information systems and the systems development process, ignoring the socioorganisational dimension of information systems as human activity systems (backhouse et al. 1991, doyle et al. 1993). thus technical-oriented approaches cannot adequately address the social and organisational issues which are at the centre of many of the difficulties experienced both during systems development itself and with the implementation of computer-based information systems. it is essential that such issues are explored and understood during the requirements definition phase (goguen 1992, lewis 1993a, doyle et al. 1993). requirements definition within the "hard" approaches concentrates on modelling users' requirements in terms of the data, process and behaviour perspectives (olle et al. 1991). data analysis and the design of conceptual data models can be viewed as requirements definition in terms of the data perspective. nijssen's information analysis method (niam) is an approach to information systems development which focuses on the data perspective (nijssen 50 and halpin 1989). the major component of niam is a fact-oriented modelling technique which forms the basis of the method's data analysis activities and which results in the specification of a conceptual schema (iso 1982). the modelling technique, known as the conceptual schema design procedure (csdp), consists of a sequence of nine steps. the niam method has been used successfully for the development of many large systems (halpin and orlowska 1992). this paper identifies some limitations of the requirements definition step of the csdp which result from its technical-oriented perspective of the requirements definition process. these limitations could be addressed by including techniques within the csdp which focus on exploring and understanding a system's socio-organisational context. four different approaches to exploring the socio-organisational contexts of information systems are discussed: the soft systems methodology (checkland and scholes 1990), the use of concepts from semiotics as in the measur approach (stamper 1992), the use of concepts and techniques from anthropology, sociology and sociolinguistics (goguen 1992), and the use of viewpoint development to identify, understand and represent different views and perceptions of a problem domain (darke and shanks 1995). it is proposed that viewpoint development be incorporated into the first step of the csdp to facilitate exploration and understanding of a system's socio-organisational context and to ensure that contextual information is a major input to the information requirements definition process. the paper consists of five sections. the next section describes the niam method and the limitations of the requirements definition step of the csdp. the third section discusses the four approaches identified above, and the fourth section outlines how viewpoint development can be incorporated into the csdp. the conclusion describes current research and future research directions. requirements definition within the niam csdp the niam conceptual schema design procedure (csdp) niam is a data-focused information systems development approach which includes a graphical notation to represent a conceptual schema and a design process to guide the modelling of the conceptual schema. the niam notation is based on the fact construct and is described as a fact-oriented approach (halpin and orlowska 1992). the current niam approach has evolved from earlier work by falkenberg (1976), verheijen and van bekkum (1982) and nijssen (1989) and is described in detail in nijssen and halpin (1989) and updated in halpin and orlowska (1992). the niam graphical notation provides a rich set of constructs to represent object types and the roles they play in fact types. fact types "connect" object types and form the basic building blocks of a niam conceptual schema design. a number of notations are available to specify constraints on and between fact types, including uniqueness, cardinality, equality, exclusion, and subset. information flow diagrams provide limited support for modelling the process perspective. niam does not currently provide adequate support for either the process or behaviour modelling perspectives, although research efforts in this area are in progress (halpin and orlowska 1992). an algorithm for transforming niam conceptual schema diagrams into optimum normal form relational database tables allows for ready implementation of niam designed schemas using relational database technology. 51 the csdp provides a method for the development of a niam conceptual schema and consists of nine steps (see fig 1). steps 1 and 2 of the csdp lead to the drawing of a first draft conceptual schema diagram. subsequent steps refine the model and add constraints. a number of steps are oriented towards quality checking of the model and particular emphasis is placed on checking the model against example fact types. 1. transform familiar information examples into elementary facts, and apply quality checks. 2. draw a first draft of the conceptual schema diagram, and apply a population check. 3. eliminate surplus entity types arid common roles, and check for derived fact types. 4. add uniqueness constraints for each fact type. 5. check arity and logical derivation of fact types. 6. add object type, mandatory role, subtype and occurrence frequency constraints. 7. check that each entity can be identified. 8. add equality, exclusion, subset and other constraints. 9. check that the conceptual schema is consistent with the original examples, has no redundancy, and is complete. figure 1 the niam conceptual schema design procedure (csdp) (from halpin and orlowska 1992, p 99) step 1 of the csdp is critical to the whole process: it is "the foundation of niam's design procedure" (halpin and orlowska 1992, p 97). in this step, familiar information examples are translated into elementary facts which define the information requirements of the system being developed. step 1 is therefore the requirements definition process in terms of the system's data perspective. step 1 is carried out by "natural verbalisation of examples in terms of elementary facts" (halpin and orlowska 1992, p 97). examples of the kinds of information required by the system are collected from input and output documents and reports used in the existing system. where these sources are not available, eg. in the case of a new system, then the "analyst should begin by getting the user to write down some examples and then work from these" (nijssen and halpin 1989, p 35). the "natural verbalisation" is carried out by the analyst, with the assistance of a user who is familiar with the application area, reading off information from output reports in the form of simple sentences or elementary facts, where an elementary fact "asserts that a particular object has a property, or that one or more objects participate in a relationship" and "a fact is elementary if it cannot be expressed as a conjunction of simpler facts" (halpin and orlowska 1992, p 99). it is suggested that the "telephone heuristic" be used to assist with verbalisation. this requires the analyst or user to "imagine you are on the phone and have to convey the information, in simple sentences, to the person at the other end of the line" (nijssen and halpin 1989, p 33). the information examples selected are required to be "familiar" either to the analyst or the uod expert, so that during the verbalisation process their background familiarity with the situation can be used to resolve any doubts concerning the information on the reports. however the issue of interpretation of data is considered only in terms of whether or not the examples collected are significant, in the sense of covering all possible types of information and all possible constraints on combinations of information required for the application area. two "quality checks" are then applied: check that the objects in the elementary facts are welldefined, and check that the elementary facts are unable to be split further. the remaining steps of the procedure are then carried out. 52 limitations of requirements definition within the niam csdp the ability of step 1 of the csdp to adequately support the identification and description of a system's information requirements is significantly reduced by its simplistic view of requirements definition which ignores the socio-organisational dimension of information systems. the limitations of step 1 are caused in particular by its underlying assumptions about three key aspects of requirements definition: the nature of the requirements definition process, the nature of system requirements, and the nature of representations of system requirements. the limitations of step 1 in terms of these three aspects are discussed in this section. the nature of the requirements definition process. requirements elicitation and definition is to a large extent a social process. the human interaction between user groups and between analysts and users contributes to the evolution of an understanding of the nature of the problems and issues which any new or amended system must address. interaction between users and analysts is necessary to combine knowledge about the business area, which the users have, and knowledge about technical possibilities, which the analysts have (goguen and linde 1993). the "natural verbalisation" process of step 1 precludes this type of interaction. there are no activities within step 1 which support exploration of alternative viewpoints or solutions. it is assumed that there is complete agreement as to the nature of the present situation, what problems exist, and what the information requirements of the new system are. the possibility of conflicting views is not considered, so no conflict resolution processes are included. emphasis on socio-organisational issues has increased as the role of data analysis has changed. originally, data analysis was seen as part of the data storage design activity, and occurred later in the development lifecycle. now data analysis is being used much earlier, in the planning and requirements definition stages, as one of the techniques for defining the problem domain itself. this means that technical issues are of less importance, and questions concerning the nature of organisations, how they can best be investigated, and how their user groups make sense of their organisational "world" become important (lewis 1993a). there are no activities within step 1 which support investigation of these issues. there are two quality checks applied as part of step 1, but these concentrate on the technical correctness of the elementary facts. there are no techniques for ensuring the completeness and validity of the elementary facts as a representation of the various user groups' perceptions and understanding of the application domain. the nature of system requirements. system requirements have been described as "emergent"; they do not exist already, rather they emerge from the interaction between user groups and analysts (goguen 1992). step 1 of the csdp assumes that information requirements are predefined, and can be obtained from the input and output documents of the existing system. these documents are also the major source of information about system requirements within step 1 of the csdp. they are examples of what goguen (1992) describes as "dry" information: the formal, context insensitive information associated with formal information systems. he argues that, in order to capture a more complete view of requirements, the informal, context sensitive information which occurs in human interaction, described as 53 "wet" information, must also be considered. this type of information is typical of the informal information system which surrounds any formal information system. it is essential to also incorporate "wet" information into the definition of requirements to ensure that they reflect the system's socio-organisational context. the informal information system determines the context of the formal system, and shapes its organisational reality (benyon-davies 1992). step 1 assumes that defining information requirements is about modelling an objective reality that exists independently of our perceptions of it. the set of elementary facts defined as the basis for the conceptual schema is considered to represent a neutral, objectively true description of a problem domain, expressed as a set of true propositions about reality. the analyst's role is primarily that of providing the technical expertise required to convert an objective reality into a data model. this represents an objectivist approach in terms of klein and hirschheim's (1987) data modelling paradigms. problems with the objectivist paradigm have been described (klein and hirschheim 1987, lyytinen 1987, lewis 1993a). a subjectivist approach considers that reality is subjectively constructed via our particular framework of social values and beliefs. thus a data model is an interpretation of a sociallyconstructed reality, reflecting the socio-organisational values of those involved in developing the model. there is no one "correct" model to be discovered and described. different viewpoints can exist, and these need to be explored and understood if the resulting system is to meet the needs of all involved user groups. the nature of representations of system requirements. much of the interaction between user groups and between analysts and user groups takes place using natural language, i.e. without the use of formal techniques and specification languages. natural language is considered in many ways to be inadequate for representing requirements because of its informality and ambiguity. however, these features can in fact be useful. they help to facilitate the gradual evolution of requirements by requiring clarification and interpretation of user statements and viewpoints, thus preventing "too early a resolution of conflicts and ambiguities" (goguen 1992, p 8). the representation of requirements within step 1 moves directly from the existing system's input and output documents to verbalisation of their contents in the form of fact types. this does not support an evolutionary requirements definition process. the use of informal representations of requirements as a precursor to their representation as fact types could facilitate exploration of different user groups' perceptions of the application domain and result in the definition of system requirements which are better able to meet the information needs of all involved user groups. approaches to exploring socio-organisational contexts the previous section has identified limitations of requirements definition within the niam csdp which are caused by a lack of focus on the embedding of a system in its socioorganisational environment: there are no activities to support development of an understanding of a system's context, and definitions of requirements and their representations do not draw on the socio-organisational dimension of systems as human activity systems. four approaches which emphasise understanding of a system's socioorganisational context during requirements definition are described in this section. each could provide a means of enriching step 1 of the csdp to facilitate exploration and representation of contextual information. 54 the soft systems approach the soft systems methodology (ssm) (checkland and scholes 1990) recognises that in many system development projects there may be complex problem situations where there is not a high level of agreement as to what the present situation is and what any new information system should do. ssm attempts to explore, understand and integrate different perceptions of problem situations and their socio-organisational contexts. however ssm does not yet have specific techniques for data analysis and modelling as part of its problem exploration activities. lewis (1993a) argues that embedding conventional, objectivist data modelling techniques within ssm is not desirable because the objectivist paradigm is fundamentally opposed to the soft systems approach. objectivist approaches view data modelling as revealing an objectively existing structure of data within a problem domain, whereas the soft systems view is that there is no neutral, objectively true description of a problem. according to ssm's "appreciative" model of human sense-making "no organisation or individual can ever 'know1 the 'real' situation directly but can only interpret the situation through a particular set of standards, norms and values" (lewis 1993a, p 181). lewis (1993b) proposes an "interpretative", subjectivist data analysis approach based on the definition of "cognitive categories" to build "system data models". he suggests that this could be incorporated into ssm to complement its conceptual models. the possibility also exists, though, of incorporating interpretative data analysis using the concept of cognitive categories, which reflects a subjectivist approach, into step 1 of the niam csdp which is an objectivist data modelling approach. this could provide a means of supporting exploration of different perceptions and contexts as part of information requirements definition within the csdp. however the usefulness of this approach cannot yet be determined as lewis' technique of interpretative data modelling based on cognitive categories is not yet sufficiently developed. it is not clear exactly how cognitive categories differ from the entity types or objects of objectivist data modelling techniques such as the csdp, except that they are derived from within the framework of ssm's root definitions. the application of concepts from semiotics semiotics (the theory of signs) is concerned with language, communication, signs and codes. information can be defined in terms of properties of signs at various semiotic levels: the syntactic (dealing with the structure of signs), the semantic (dealing with the meanings of signs), and the pragmatic (dealing with the usage of signs) (stamper 1992). semiotics has relevance for conceptual data modelling and database design because it is the various socioorganisational groups within an organisation which give meaning to the signs which exist within an information system (benyon-davies 1992). information systems are considered to be social systems and "only by correctly embedding the computer-based system in the social system can the data it contains have any meaning, express knowledge or support intelligent behaviour" (stamper 1992, p 32). semantics has, of course, been recognised as an important aspect of conceptual data modelling, but most attention has focused on representation notations (for example, see discussion of semantic data models in peckham and maryanski 1988 and hull and king 1987) and on prescriptive methods for developing a conceptual schema, such as niam's csdp. little has been done in the area of human interpretation of data models and the "social and psychological processes by which meaning is produced" (benyon-davies 1992, p 215). 55 stamper (1992) describes a methodology for information systems development, measur, which is based on the principles of semiotics because it "concentrates on the meanings of signs, the purposes for which they are used and the social consequences they produce" (stamper 1992, p 32). measur leads to the creation of a "normbase" rather than a conventional database. a normbase is a collection of "norms": specific patterns of organised behaviour which should be displayed by user groups within an organisation in specific situations. norms embody the users' business knowledge and their interpretation and perceptions of their organisational situation. the normbase is intended to contain not only business data, as in a typical database, but also knowledge of business policy which is usually retained within various application programs. definition of norms is an attempt to express more than just a system's technical data and function requirements, as norms reflect a system's socio-organisational context, including the users' expected behaviour patterns. the first three phases of measur focus on exploring and understanding a system's socioorganisational context. problem articulation (a "soft systems analysis") produces a set of problem statements defining the business problem in the users' own terminology. semantic analysis clarifies the meaning of each problem statement, producing a semantic schema for the normbase. norm analysis involves creating the normbase by populating the schema with particular instances. the use of problem articulation and semantic analysis techniques could be investigated as a means of enriching the csdp. the feasibility of this depends on whether the techniques can be applied outside the context of the measur methodology. the application of sociological and ethnomethodological concepts and techniques recent research in this area has identified two basic types of information: "dry" or formal, context insensitive information, and "wet" or informal, situated information (goguen 1992, goguen and linde 1993). both are relevant to defining system requirements. formal or "dry " information is exemplified by the syntactic representations found in computer systems, and by much of the formal documentation associated with information systems. informal or "wet" information typically occurs in the social interaction between people, eg. between user groups and between users and analysts. they are complementary aspects of information and therefore an understanding of both is important. both are relevant as sources of information about system requirements because requirements definition is not just about solving technical problems; the socio-organisational context of information systems is of major importance in this phase (goguen and linde 1993). requirements are often not pre-existing and clearly defined, but rather emerge from the interaction between the user groups and systems analysts. research is being conducted into the use of concepts and methods from the social sciences (sociology, anthropology and sociolinguistics in particular) to gain a better understanding of this interaction (goguen 1992, sommerville et al. 1993). discourse analysis and other interaction analysis techniques can be used to analyse written texts in their social context and to analyse spoken language and other interaction structures. these techniques are useful in situations where social interaction is significant, and can be used to explore the value systems of organisations. discourse analysis of user explanations of activities can also be used for situated task analysis (goguen and linde 1993). these techniques can assist in understanding the structure of the communication and interaction between people, and the ways in which these are determined by the social context in which they occur, and the ways in which they at the same time define and construct the social context (descriptions of these techniques and further references can be found in goguen 1992, goguen and linde 1993, and sommerville et al. 1993). 56 ethnomethodology and ethnographic studies are used to study the structure, organisation and practices of social groups. ethnographic studies require spending an extended period in an actual working environment, noting work practices and social processes. this aids in understanding the socially-constructed informal information system in which any formal, procedural information system is embedded (sommerville et al. 1993). the success of any formal information system is considered to be dependent upon how well it aligns with the informal information system. sommerville et al. (1993) describe a system development project in which ethnomethodological techniques are being used to explore and document the users' views of their socio-organisational situation. this can help to ensure that it is their views, rather than those of systems analysts, which are applied to the problem situation. these sociological and ethnomethodological techniques could also be used within step 1 of the csdp in problem situations where social and organisational issues appear to be critical. goguen and linde (1993) discuss the general suitability of these techniques for requirements elicitation. they suggest that as these techniques are very labour intensive they should be used in conjunction with traditional methods such as interviews, questionnaires and jad sessions, in situations where significant problems have been revealed which cannot be adequately understood using traditional methods. viewpoint development for requirements definition viewpoint development provides a means of understanding a system's socio-organisational context by identifying and exploring the different perceptions and viewpoints of the various stakeholder groups involved (darke and shanks 1995). representations of viewpoints can then become an important and useful input to requirements definition. the feature which distinguishes viewpoint development from other requirements definition activities is that multiple viewpoints are developed as separate, independent "objects", i.e. in no way are the viewpoints just distinct views of a common underlying representation. a conceptual framework for understanding viewpoint development has been proposed (darke and shanks 1995). this is depicted in figure 2. role of viewpoint development viewpoint integration figure 2 conceptual framework for viewpoint development (from darke and shanks 1995) the conceptual framework identifies six core elements as central to an understanding of viewpoint development. these are: the role of viewpoint development, the viewpoint type, the representation of viewpoints, the viewpoint development process model, management of 57 viewpoint conflicts and inconsistencies, and integration of viewpoints. a number of key features of each of the framework elements have also been defined and described. the framework elements are briefly outlined here (see darke and shanks 1995 for full details). the role of viewpoint development. viewpoint development may focus on either or both the requirements acquisition and requirements modelling phases of requirements definition. during requirements acquisition it supports the elicitation and understanding of system requirements. during requirements modelling it supports the representation and management of multiple perspectives and components of system requirements models and specifications. viewpoint types. either user or developer viewpoints may be identified. a user viewpoint captures the perceptions and domain knowledge of a particular user group. a user group may have several organisational roles, so it is possible to associate more than one viewpoint with a specific user group. a developer viewpoint captures the perceptions, domain knowledge and modelling perspective relevant to a systems analyst or other developer responsible for producing some component of a requirements specification. representation of viewpoints. viewpoints can be represented using informal, formal or semi-formal representation techniques. the suitability of a particular technique for viewpoint representation depends on both the role of viewpoint development and the viewpoint type. no single representation technique is adequate for representing all types of viewpoints or for supporting viewpoint development during both requirements acquisition and requirements modelling, so a combination of techniques is necessary (darke and shanks 1995). it is also essential to ensure that viewpoint representation focuses on the semantics of the viewpoints and not just the syntax of the representations. the viewpoint development process model. viewpoint development is an exploratory and evolutionary process. the process model defined within the conceptual framework identifies six core activities: viewpoint identification (identify the relevant viewpoints and acquire their content), viewpoint representation (represent each viewpoint using appropriate representation techniques), viewpoint analysis (identify intra-viewpoint inconsistencies and determine the completeness of each viewpoint), viewpoint comparison (compare all viewpoints to identify inter-viewpoint inconsistencies and conflicts), management of conflicts and inconsistencies (use inconsistency handling techniques and conflict resolution strategies), viewpoint integration (negotiate a common viewpoint if it is considered desirable and feasible to do so). management of conflicts and inconsistencies. techniques to identify conflicts and inconsistencies between viewpoints are needed so that relationships and differences between viewpoints can be understood. easterbrook (1991) defines conflict as "any interference in one party's activities, needs, or goals, caused by the activities of another party", whereas inconsistency is indicated by apparent contradictions between the stated requirements of involved parties. this distinction is useful, but in practice it is not always easy to recognise whether differences between viewpoints are the result of conflicts or inconsistencies. it is important also to ensure that the emphasis is on identification of semantic differences between viewpoints rather than just on syntactic differences between viewpoint representations, otherwise viewpoint comparison and integration activities will be compromised and important differences in understanding of the problem domain may be suppressed (darke and shanks 1995). 58 viewpoint integration. one of three viewpoint integration methods may be used: one representation scheme (one representation scheme is used for the viewpoints and for the integrated viewpoint), pre-integration translation (viewpoints are developed in various representation schemes which are translated into a different common representation scheme for comparison and integration), post-integration translation (one representation scheme is used for viewpoints, a common viewpoint is negotiated and then translated into a different representation scheme). incorporating viewpoint development into the niam csdp viewpoint development is "method independent", i.e. it can be used within the context of any approach or methodology. the viewpoint development process model which is an element of the conceptual framework developed by darke and shanks (1995) is depicted in figure 3. 1. viewpoint identification 2. viewpoint representation j. viewpoint analysis 4. viewpoint comparison 5. manage conflicts and hiconslslendes tf required 6. viewpoint integration figure 3 viewpoint development process model (from darke and shanks 1995) the incorporation of viewpoint development into the niam csdp would provide a mechanism for exploring, understanding and representing a system's socio-organisational context in terms of user viewpoints, as an integral part of the requirements definition process. this can be achieved by adding the viewpoint development process model to the front of the csdp as "step 0", thus including viewpoint development as a specific activity to be carried out. the resulting enhanced design procedure would address the limitations of step 1 identified earlier in this paper. viewpoint development would help to ensure that requirements definition within the csdp is an evolutionary and exploratory process which focuses on social interaction between users and analysts, that requirements themselves are 59 seen to be emergent and not always predefined, and that representations of requirements are not formalised too early in the requirements definition process. conclusion and future research this paper has identified limitations of the requirements definition step of the niam conceptual schema design procedure (csdp) which result from its lack of focus on the socio-organisational dimension of information systems development. the limitations relate specifically to the csdp's underlying assumptions about the nature of the requirements definition process, the nature of system requirements, and the nature of system requirements representations. four different approaches to exploring the socio-organisational contexts of information systems were described: the soft systems methodology, the use of concepts from semiotics (as in the measur methodology), the use of concepts and techniques from the social sciences, and the use of viewpoint development. it was proposed that viewpoint development, which is a methodology independent approach, be incorporated into the csdp to provide a means of exploring and understanding a system's socio-organisational context and to ensure that contextual information is a major input to the information requirements definition process. viewpoint development can be integrated into the csdp by including the viewpoint development process model at the front of the csdp as "step 0", to create an enhanced design procedure. further research is underway to investigate more fully the proposed approach to integration of viewpoint development within the csdp, including techniques for converting representations of user viewpoints into niam fact types. empirical studies will be conducted to test the usefulness of the enhanced design procedure. general strategies for incorporating viewpoint development within system development approaches and methodologies other than niam are being explored as well. research is also in progress to investigate specific techniques for integrating viewpoint development activities with other requirements definition activities. the focus of this research is on development of user viewpoints. it involves investigation of the types of contextual information which need to be identified and represented in order to ensure that the "local" socio-organisational perspectives of relevant user groups are captured as part of requirements acquisition and modelling activities. the development of techniques for the representation and structuring of user viewpoints to support this process is an important component of this research. a notation for representing user viewpoints is being developed which will incorporate the use of multiple and diverse representation techniques to allow a broad range of types of contextual information to be identified and captured. integrating viewpoint development activities within requirements definition should ensure that local perspectives of problem situations are not ignored or suppressed. in this way alternative viewpoints and perspectives can be explored. this should facilitate the development of systems which are better able to meet the needs of all involved stakeholder groups. 60 sommerville, i., rodden, t., sawyer, p., bentley, r. and twidale, m. (1993) "integrating ethnography into the requirements engineering process", proc. ieee international symposium on reqts. engineering, san diego, calif., january. stamper, r. (1992) "signs, organisations, norms and information systems", proc. 3rd australian conference on information systems, wollongong. verheijen, g.m.a. and van bekkum, j. (1982) "niam: an information analysis method" in t.w. olle, h.g. sol and a. verrijn-stuart (eds.) information systems design methodologies: a comparative review, ifip, north-holland 62 ajis special issue requirements engineering 1999 a hybrid parallel execution model for logic-based requirement specifications* jeffrey j.p. tsai and bing li department of electrical engineering and computer science university of illinois at chicago chicago, il 60607-7053 tsai@eecs.uic.edu abstract it is well known that undiscovered errors in a requirements specification is extremely expensive to be fixed when discovered in the software maintenance phase. errors in the requirement phase can be reduced through the validation and verification of the requirements specification. many logic-based requirements specification languages have been developed to achieve these goals. however, the execution and reasoning of a logic-based requirements specification can be very slow. an effective way to improve their performance is to execute and reason the logic-based requirements specification in parallel. in this paper, we present a hybrid model to facilitate the parallel execution of a logic-based requirements specification language. a logic-based specification is first applied by a data dependency analysis technique which can find all the mode combinations that exist within a specification clause. this mode information is used to support a novel hybrid parallel execution model, which combines both top-down and bottom-up evaluation strategies. this new execution model can find the failure in the deepest node of the search tree at the early stage of the evaluation, thus this new execution model can reduce the total number of nodes searched in the tree, the total processes needed to be generated, and the total communication channels needed in the search process. a simulator has been implemented to analyze the execution behavior of the new model. experiments show significant improvement based on several criteria. keywords : logic-based requirements specification language, data flow analysis, parallel execution model, and-or parallel execution. this research was supported partially by national science foundation and darpa under grant no. ccr9633536, by usaf rome laboratory under grant no. f30602-95-1-0035, by fujitsu network communication. introduction undiscovered errors in a requirements specification will be very expensive to be fixed in the software maintenance phase. errors in the requirement phase can be reduced through the validation and verification of the requirements specification. through the execution of the logic-based requirements specification, customers and developers can validate their disagreement or misunderstanding in the requirement phase. by reasoning over the logic-based requirements specification, developers can verify that desired properties such as, consistency, liveness, etc. have been preserved. however, the execution and reasoning of a logic-based requirements specification can be very slow. an effective way to improve their performance is to execute and reason the logic-based requirements specification in parallel. to reach this goal, in this paper we present a hybrid parallel execution model for logic-based requirements specifications such as frorl (tsai, j.j.p., 1987). a logic-based specification consists of a set of clauses. for each clause, the clause head and body are formed by predicates. the execution of a logic-based specification is based on the resolution principle. a data value is transferred among clauses and within the clauses' bodies based on the unification rule. the execution of a logic-based specification is similar to a depth first search of an and-or tree (corneryj.s., et al, 1983), where and nodes represent the predicates within a clause, and or nodes represent the clauses unifiable within a calling predicate. the parallelism of a logic-based requirements specification can be extracted from this and-or execution model. it follows the top-down searching strategy of the and-or tree. the parallel execution of or-branches is relatively easy to implement because the execution of clauses unifiable with a calling predicate are independent of each other. the parallel execution of and-branches is much more complex since it may create binding conflicts during evaluation (degroot, d., 1988). an effective and common method to solve this problem is to rely on data flow analysis, which allows exactly one producer and multiple consumers for each parameter in a clause. currently, most approaches adopt the method of dynamic mode inference to do data flow analysis (chen, a.c. at al, 1991). the basic idea of this approach is to rely on the user's query to generate data dependency information, then use this information to spawn processes which can be executed in parallel. the nature of all these methods is a demand-driven model which executes in a top-down fashion. even though these 13 ajis special issue requirements engineering 1999 approaches can extract and-parallelism during the execution, their common drawback is that the dynamic determination of parameters' mode and parallel execution combination puts tremendous overhead on the model. this may jeopardize any advantage these parallel execution models might obtain. a few approaches for data dependency analysis by static mode inference (chang, j.h., et al, 1985) is based on the analysis of the structure of a logic program to generate the parallel executable components. but again they suffer the problems of either being unable to generate all the solutions to the input logic program, or they sacrifice the degree of parallelism of the program. for example, in (chang, j.h., et al, 1985), chang et al. implement static data dependency analysis by considering only the worst case clause invoking pattern, which eliminates the possible advantage of generating multiple solutions of a logic program at the same time. as a result, the approach has to explicitly handle the backtracking problem during the parallel execution of the and-or tree. in this paper, a novel parallel execution model for a logic-based requirements specification based on a new static data dependency analysis is presented. this model can preserve maximum parallelism while guaranteeing to generate all the solutions of a logic-based specification without backtracking. the power of this model comes from the fact that the static data dependency is capable of representing all execution sequences and all multiple solutions implied by a logic-based specification (tsai, j.j.p. et al, 1998). since the complete mode information is available through mode inference even without actual evaluation, the new parallel execution model can apply some extra analysis processes during compiling comparing with other approaches and obtain more parallel execution information of the underlying logic-based specification. the overall parallel execution behavior of the logic-based specification can thus be improved. specifically, the new execution model extracts the necessary mode information from the static data dependency analysis to facilitate the combination of top-down and bottom-up searching of the and-or tree of a logic-based specification. in this process, all solutions implied by the logic-based specification can be explored and generated based on the same data dependency information, which may result in the nice property of preserving the correctness of the results while eliminating backtracking from consideration. regarding the top-down evaluation direction of this hybrid model, it adopts the method similar to the current approaches, i.e., a predicate is executable only if all of its input parameters have received values from some other predicates within the same clause. this evaluation direction mainly reflects the interrelation among the predicates within a clause. on the other hand, the new evaluation along the bottom-up direction relies on the detection of ground variables from the static data dependency information to facilitate the evaluation of those predicates with all of its required value being available. this direction mainly deals with the possible adjustment of evaluation sequence among clauses of a logic-based specification. by adopting this strategy, the overhead of the parallel execution of a logic-based specification can be reduced. in this paper, other related issues of parallelizing a logic-based specification are also discussed. among them are the elimination of backtracking in the process of searching and the multiple binding synchronization problem which exists in or-parallelism. the paper is organized as follows. section 2 summarizes the current approaches. section 3 provides an overview of the proposed new system and shows how it handles major problems in the parallelizing process. section 4 discusses the static data dependency analysis method and the mode information it generates. section 5 explains in detail the and-or parallelism based on the static data dependency information. section 6 considers the efficiency of the new approach and compares the experimental results of the new model with other parallel approaches. section 7 gives a brief summary. related works when realizing the parallelism of a logic program, various approaches exist for dealing with the three main problems: 1) realization of orparallelism; 2) realization of and-parallelism; and 3) reduction or elimination of backtracking. or-parallelism if an or-branch has more than one clause which can unified with a predicate (a goal), then these clauses can be executed concurrently. the successful evaluation of any one of these clauses indicates the successful evaluation of the invoking predicate. this kind of parallelism is called or-parallelism. because the execution of unifiable clauses in this case is independent of each other, the implementation of or-parallelism is relatively simple. the main problem in handling the or-parallelism is how to represent the different branches. from our observation, the following two classes of approaches exist in this area. solution synchronization almost all systems and approaches realizing or-parallelism fall into this category. the major problem which needs to be handled is the synchronization of the multiple solutions generated from different clauses (butler, r. et al 1986). warren provided a concise summary of the current approaches available within this category (warren, d.h.d, 1987). these methods developed various synchronization and ordering algorithms to identify and control the solutions generated from the different branches of the clause. the final 14 ajis special issue requirements engineering 1999 result of the or predicate is basically a cartesian product of all possible results generated from unifiable orclauses. in order to insure the proper cross production, various data structures and identifications attached to a result have been presented which determine the bindings generated by different clauses. the major data structures used under this context can be categorized as • binding array (butler, r. et al 1986) • synchronization timestamp (halperin, m. et al, 1985) • version vectors (hausman, a, ciepielewski, a., haridi, s., 1987) the data structures have the following common characteristics: • binding data structures defined at each or-branch point exists when the or-parallel point is first entered and exists throughout the parallel execution process until all the branches of the orparallel point have finished execution. • binding data structures are dynamically modified along with the execution at each branch of the or-parallel point. this modification may come from the deeper-level predicates in the or-parallel branches when that branch finishes execution and generates the corresponding binding. the first point forces a system to bear a large overhead by maintaining many complex synchronization data structures. to make things worse, some of the data structures may be necessary for the current or-parallel execution if some processes are not interested in the current variables at the current or-branch point (warren, d.h.d, 1987). the second point also makes the control of the synchronization data structures complex. specifically, the cost for the creating and accessing variable bindings becomes prohibitively high. in order to cope with this problem, some of the current approaches consider a and-or search tree as a hierarchical structure and use the inheritance mechanism to reduce the complexity of maintaining the synchronization data structures (hailperin, m. et al, 1985). since all possible or-parallel branches are explored simultaneously, if all the solutions are guaranteed to be found while the or-branches are explored, there is virtually no need for the backtracking in this case. the merging of or-parallel processes has to co-operate with the spawning of and-parallel processes. for example, in approach (li, p. peyyum & martin j. alain, 1986), the results are gathered at or-branches by taking a cross production, and only selected information is passed to the above and-branches. or-parallelism combined with backtracking this model (westphal, h et al, 1987) adopts the lay approach in spawning the process. only when a processor idle and asks for a process will another branch at an or-parallel point be generated and executed. if a failure occurs, the remaining untouched branches of the or-parallel point are searched. so it is a combined approach of the traditional sequential and the advanced parallel execution model. the incompleteness in the parallelism which originates from the inadequate number of process at evaluation time require the approaches can be considered as a special case of the first class with a special focus on the parallelism and a limitation on the number of process which can exist at the same time. so the implementation has to consider those controls usually occur in the sequential realization, like task switching and backtracking. this is a mixture of the or-parallelism and the general sequential evaluation of a logic program. when the number of process exists at any time is limited to one, the execution is equal to a sequential approach. and-parallelism with and-parallelism, all predicates within a clause body must be true for a clause to be successful. the exploration of the predicates within a clause body can be carried out concurrently. handling the binding conflict (degroot, d., 1988) is the major task in any implementation, and a number of approaches exist. programmer's annotation this is the simplest method to handle and-parallelism. it requires the users to explore and define explicitly the parallel component of a logic program (clark, k, et al, 1984). even though the control is relatively simple, but human's involvement is definitely not our intention. independent and-parallelism in this kind of parallelism, there is no variable dependency between and-parallel goals. predicates which share variables must have these variables bound to ground upon entering the clause (hermenegildo, m.v., jacobs, d., et al, 1986). this approach restricts the andparallelism to a relatively simple case similar to or-parallelism. a comparison of different independent and-parallel models can be found in (gupta, g., 1991). data flow analysis for dependent and-parallelism one of the most popular methods is to assign 15 ajis special issue requirements engineering 1999 only one producer for each parameter and allow multiple consumers of the same parameter in the program (chang, a.c., et al, 1985). data flow analysis is used to determine the producers. with a single producer, the dependency of variables between predicates can be monitored. the spawning of process is based on this data flow information. the data flow analysis method can be done either dynamically or statically, with the dynamic analysis being the dominant analysis method at the current time. dynamic analysis generates binding information from the user's input at each step of unification. this variable instantiation changes during execution and is adjusted dynamically. static analysis (chang, a.c., et al , 1985) generates binding information which is independent of the user's input, so that the overhead of the parallelization can be reduced. but currently available static analysis approaches impose various limitations or restrictions on the results, because they either limit the number of solutions generated or the degree of parallelism. other approaches impose the similar sequential restrictions to a logic program to achieve dependent and-parallelism (shen, k., 1992). backtracking handling backtracking is not so critical as and-parallelism and or-parallelism, because it is possible to totally eliminate backtracking from the parallel execution model (chen, a.c., wu, c.l. et al, 1991) if • and-or parallelism is supported so that all branches at all choice-points can be searched simultaneously, and • all solutions implied by the logic program can be expressed and generated from the parallel execution model. even though, some of the interesting development in various approaches which handle backtracking, not necessarily under the context of parallel logic program model, it is still worthy to be listed here. also, there still exist a lot of approaches in the current logic program community which explicitly handle backtracking because of the insufficient power to guarantee the above two criteria. basically, there are two different kinds of backtracking mechanisms, shallow backtracking and deep backtracking. shallow backtracking backtracks to alternative candidate clauses when the head of the current candidate clause cannot be proved a success by the unification with the calling literal. deep backtracking backtracks to any previously invoked procedure which has a choice-point. this is also called intelligent backtracking. intelligent backtracking in conventional backtracking scheme, where a failure occurs during the search, it backtracks to the nearest choice-point and tries other branches. in intelligent backtracking, instead of backtracking to the nearest choice-point, backtracking goes to the choice-point where the binding of the variable which causes the failure can be bounded to another variable (kumar, v. & lin, y.j, 1987). research in intelligent backtracking which handles other related aspects, like cut, tail recursion, etc., also exists. in (codognet, c, 1988), the pruning of the parts which could be explored several times by the conventional backtracking scheme is studied. improving backtracking by using data dependency information was first proposed in (conery, j.s., 1983). various researches considered intelligent backtracking handling in the and-parallelism of the and-or parallel execution model of a logic program were presented in (borgwardt, p., codognet, c. etal 1986). 1.1.1 semi-intelligent backtracking semi-intelligent backtracking uses static data dependency analysis for the and-parallelism. chang and despain (chang, j.h. & despain, a.m., 1985) use this technique to find the optimal choice-point for intelligent backtracking. since the mode information is not sufficient to support the multiple solutions exploration, backtracking is necessary. the binding status considered is only the worst case binding combination, which may involve in inaccurate binding results, so it is called semi-intelligent backtracking. elimination of backtracking the and-or parallel execution model can completely eliminate backtracking from the execution of a logic program (chen, a.c., wu, a.m. et al, 1991). few results have been produced because most current approaches are unable to find all the solutions implied by the underlying logic program. this has forced them to adopt some degree of the backtracking. 16 ajis special issue requirements engineering 1999 overview of a new approach in this section, we outline a new approach and discuss how it handles the problems mentioned in section 2. figure (1) shows the major components of the model and their interrelations. our approach consists of the following four techniques. s . i logic-based requirements specification j [ static data dependency analysis und dependency matrix i ^ ' i . -k . [ hybrid and-or parallel execution mode i / user's query) figure 1: an overview of the parallel evaluation system static data flow analysis in this step the multiple execution sequences in a logic-based specification are found. these results are stored as two-dimensional data dependency matrixes and are consulted during execution to determine predicates that can be evaluated in parallel. hybrid and-or parallel execution model this and-or parallel model uses information gathered statically and the user's input to find the largest number of the predicates that can be executed in parallel. these predicates may come from different levels of the corresponding and-or tree. the parallelism of the predicates determined by the static data dependency information will be carried out in a bottom-up fashion. if no predicate can be executed in this way, the general top-down execution model will be used. simplified synchronization data structure the data structure used to support or-parallelism can be realized with reduced complexity comparing with current approaches according to the hybrid execution model. specifically, the duration of the synchronization data structures and the complexity of them can be reduced. backtracking elimination since our approach can generate all the solutions from the static data dependency analysis, and all the branches at each choice point are searched simultaneously, we believe that our model can eliminate the backtracking by carefully implementing the andand or-parallelism. data flow analysis of a logic-based specification a logic-based requirements specification is highly dynamic. a specification clause may be used to perform computations in several directions. the arguments of a clause may serve both as sources of input for the computation described by the clause, and as the location of output of that very computation. a predicate in a logic-based specification may have different meanings based on different environments. most obviously, many predicates in a logic-based specification can be used in several directions, depending on the mode of the arguments. considering the following example, append ([1,2], [3,4], x) appends two lists and returns the result in x, append ([1,2], x, [1,2,3,4]) obtains lists which could be appended to [1,2], in order to yield the list [1,2,3,4], and append ([1,2],[3,4],[1,2,3,4]) tests whether the two lists appended indeed yield the list [1,2,3,4]. as a consequence, terms in a logic-based specification may have multiple meanings. most prominently, the list notation «equation:datafl» can either denote the construction of a list from an element h and a list t, or it can denote the destruction of an existing list in its head (h) and tail (t) components. which of the two meanings is intended by the 17 ajis special issue requirements engineering 1999 specification depends on the sequence of clause evaluation. the order of the clause evaluation can be determined within the semantic domain of logic-based specification. the sequence of evaluation may change from input (query) to input (query). such multi-directionality is the nature of a logic-based specification language and constitutes the main difficulty in parallelizing a logic-based specification. in order to preserve the completeness of a logic-based specification, we have to explicitly locate all possible data transformation paths from it in the parallelizing process. this phenomenon can be characterized as the mode information of parameters and data flow analysis in a logic-based specification (chang, j.h. et al, 1985) traditionally, data flow and data dependency analysis have been investigated within the framework of logic programming. one approach to deal with the multi-directionality of a logic program has been to provide mode information for each clause (e.g., in parlog, concurrent prolog). bruynooghc (bruynooghe, m., 1982) and smolka (smolka, g., 1984) have investigated the issue of verifying the consistency of mode declarations for arguments to predicates, as supplied by programmers. automatic mode inference has been discussed by mellish (mellish, c.s., 1986) and reddy (reddy, u.s., 1984) recently by bruynooghe (bruynoogle, m. et al, 1987) and mannila and ukkonen (mannila, h. & ukkonen, e., 1987) most promising approach to mode inference so far has been presented by debray in (debray, s.k., & warren, d.s., 1988). debray uses an interpreter to reason upon the four-valued lattice of modes don't know, empty, free, and ground. its purpose is to enable a compiler to optimize a logic program. mannila and ukkonen (mannila, h. & ukkonen, e., 1987) limit themselves to the simple modes ground and unground. reddy relies on syntactic analysis to assign modes to predicates. mode representation in the new model in this section, we present a new approach to mode inference which relies on a matrix representation of clauses to perform data flow and dependency analysis to support the new parallel evaluation model. in the new approach, we reason over the four-valued lattice of the following mode values: • don't know ("?") • input mode ("+") • output mode ("--") • empty ("") for each clause in the specification, we construct a two-dimensional matrix, with the literals in the clause along one dimension, and the variables of the clause along the other. this matrix is filled in with mode values by applying algorithms and constraining rules. boundary of c pi . . . .ar jc . ... ;. t to clause hc:ml boundary of clause c2 + 1 input co clause h«i output fiom clause -t-1, lupin to va>\ 1 qi.... vufi, . . . /. figure 2: an illustration of data transformation by a clause head literal 18 ajis special issue requirements engineering 1999 initially arguments of literals are marked as don't know ("?"), since no mode information about the parameters in a predicate is available. arguments that do not occur in a corresponding literal are marked with " ". the next step is to collect and apply all possible known mode constraints of the literals within the rule clause such as those of user-defined predicates and the mode information for all built-in operators used in this rule clause. the mode values of the arguments for all the literals in a clause body follow the normal definition of data flow: input is marked with "+", and output is marked with "--". in other words, an argument which generates data (exports a value for the argument) is said to have output mode and an argument which consumes data (needs a value to bind to) is said to have input mode (data is viewed to flow into the predicate across its "boundary"). but for a predicate in a clause head, the mode convention for its arguments should be the opposite of that in the arguments of predicates in the clause body. this is because a clause head serves dual roles: on the one hand it unifies with the calling (invoking) literal (and thus passes data between the calling literal and the clause). on the other hand it communicates data to and from the clause body through variables sharing the same name. we can interpret the communication of data to a clause in two steps: first we have a unification between the calling literal and the clause head of the invoked rule clause, then the data is passed between the head of the clause and the literals forming the clause body. the modes for the arguments of a clause head are normally viewed as based on the former role. however, when performing the analysis of a particular clause we treat the clause as a single unit, and therefore consider the modes for the arguments of the clause head from the latter point of view. figure (2) gives an example of the two-staged process of data communication. if a clause c2 with the clause head p (..., vark,...) is called within the body of a clause p' in c\, then the literal p in c\ may provide data to the clause c2 via unification of the argument vark of p in c2 against the parameter argk of the call to p within the clause c\. in that case, the mode of argk of p in c\ is output, whereas vark in c2 has mode input, for the balance of flow to be preserved. however, in addition to receiving data from argk in c\, the variable vark of p in the clause c2 also provides data to the predicates in the body of c2 that share this variable. since mode analysis is performed within the boundaries of a clause, the mode conventions for the clause head have to match that for the clause body. a "+" ("--") in the clause head therefore has opposite meaning in the clause body, and vice versa. the same holds true for data flow being communicated from the head of the clause c2 to the calling literal p in c\. data flow and dependency analysis algorithm after we establish the matrix representation of mode information, we then apply the data dependency and flow analysis algorithm to the mode matrix of each clause. applications of the algorithm are interleaved with applications of the constraining rules. through this process, we will gradually build up the mode information for the clause under consideration. in order to handle multiple mode combinations more conveniently, we also introduce a new mode status "b", which stands for "both." it has a similar meaning as "?". the difference between "?" and "b" concerns the stage of analysis."?" represents that we have no idea about the mode of an argument; "b" implies that we have already explored the argument and that both input and output modes are possible for this argument. if we restrict our attention to an argument position in a literal, then we can refer to the literals as "producer" and "consumer." if a literal consumes the value for a variable generated by other literals, we call this literal as "consumer." if a literal produces (binds) the value for an ungrounded variable, we call this literal a "producer." the algorithms are based on two fundamental assumptions. these assumptions are also used to solve binding conflicts in and-parallelism. unique-producer assumption any argument, other than a constant, with the mode of "+" will demand exactly one producer for the value to be consumed by it. existence-of-consumers assumption a producer of a value (with the exception of dead-end values, which are generated but not used by any other literal in the clause) has at least one consumer. from the initial matrix, the algorithm below calculates the mode matrix for a given clause. the matrix initially contains only " " and "?" information, with the exception of built-in predicates, and predicates for which information is already available from previous runs of the algorithm on other clauses. the algorithm is exhibited in two parts, the initialization of the matrix, and the dynamic analysis. in the following, let lit be a sequence formed from the multi-set of the literals mentioned in a clause (note that a literal could occur more than once in the sequence), and let arg be a sequence constructed from the set of arguments in the clause. further, let arg (t) be the subsequence of those elements in arg, that occur in the literal with predicate symbol /(i.e., «equation:dataf2»" "} where mmt<v denotes the mode status of the variable v in the literal 1 in the matrix). in the algorithm, «equation:dataf3» ! represents " there exists only one/'. 19 ajis special issue requirements engineering 1999 the data dependency and flow analysis algorithm deduces mode information by reasoning from the modes already known from the first part (1 — 16) to generate the remaining unknown modes in the matrix (17 — 29). algorithm 1 (data dependency and flow analysis) 1. types mm: lit x arg -»{ "+", "-", "?", "b", " "}; arg: set of argument(empty) 2. for each literal 1 e lit 3. push 1 onto todo-stack 4. arg = arg + argument(l); 5. if 3! v e arg(l). v is not a constant 6. then mm; v«"-" 7. for each argument v e arg 8. if v constant 9. then for each 1 e lit a mm,.v"?" 10. mm|.v<-"+" 11. if v is predicate 12. then if all of v's arguments depend on constants 13. then for each 1 e lit a mm,.v"?" 14. mm|.v<-"+" 15. else for each 1 e lit a mm,./'?" 16. mm,,v<-"b" 17. while todo-stack is not empty 18. i «— pop todo slack 19. for each v 6 arg(l) 20. if mm|,v= "+" a v is not a constant 21. then if vi' € lit . mmr,v = "+" 22. then fail 23. i f 3 ! l ' e lit .mm r .v="?" 24. then mm,.v <"-" 25. i fmm r .v="?" 26. then if 31' e lit, 1' * 1. mmr.v = "b" v mmr.v = "?" 27. then mm,.v <"-" 28. else. mmr.v ="b" 29. call rule-based flow analysis the algorithm examines the mode matrix in two directions. in the horizontal direction, the analysis is mainly based on the mode constraints existing within the arguments of a single literal, which may include built-in user defined predicates. in the vertical direction, the analysis is based on the above mentioned two assumptions. line 1 initializes the data dependency and control flow matrix "+" and "-" modes at this step are generated from user provided mode constraints and from decomposition in the rewriting process. this line also creates a set which holds all the arguments within the clause and initializes it to empty. line 2 to 4 push each literal in the clause onto stack, todo-stack, which contains all remaining literals to be analyzed, at the same time these lines pick each argument from literal 1 and save it in arg. line 5 and 6 set a variable's mode to "-" if it occurs only once in its column. since in this case it cannot have the "+" mode, it requires another appearance of the same variable to provide a value. line 7 to 16 analyze the argument in arg one by one to decide their mode if possible, based on the available modes. among them, lines 8 to 10 set the modes of all occurrence of a constant within its column to "+", since a constant always provides values to others. lines 11 to 16 handle the case of a predicate. if all the predicate's arguments depends on constant, then the mode of the predicate itself is set to "+" (line 14), otherwise its mode should be "b" (line 16). lines 17 to 29 apply mode inference operations and constraints to find new modes from the modes already in the matrix. it does this by checking each literal inside todo-stack. note that i represents the current literal being checked (line 18). line 0 to 22 force failure if an argument other than a constant has all its appearances in different mode "+"; the intuitive idea behind it is that all the occurrences are consumers, and no producer exists. line 23 to 24 change the only occurrence of the argument with unsearched mode to "-" (a producer) if all the remaining occurrences of the same argument are consumers. line 25 to 28 deal directly with the undecided modes in the matrix. if there is no other cell within the same column which has undecided mode of "b" or unsearched mode of "?" other than the current cell, then the mode of the current cell should be "-" (line 27), otherwise there is nothing we can say about its mode (line 28). line 29 calls the subroutines defined by rule-1 through rule-5. those rules can also be written as condition statement inside the algorithm similar to the above cases. because some of them 20 ajis special issue requirements engineering 1999 are too long to be put in the algorithm, and some of them deserve some special attention for their importance, we write them in the format of a rule. note again the effect of line 29 is identical to a case check from rule-1 to rule5. beside the operations described in the algorithm, the following rules constitute additional mode constraints and result in adjustment of modes within a single column or row of the mode matrix. among them, some are used to eliminate mode status "b" from the matrix. rule-1 (flow analysis) mm|,v="-" => (mm,.v= "b" v mmr,v = "?") a w let mm,..v = "+" rule-1 (flow analysis) mm,,v="b" letmmr,v = "-" these two rules reduce the number of arguments marked with mode "b". further mode information to trim "b" could consist of mode declarations provided by the user, or mode information obtained by queries to the user. an example of the former is given in the following example. example 1 consider the predicate append ("+", "-", "-") and append ("-", "+", "-") makes no sense when considering the procedural meanings implied by them. by eliminating them from the mode analysis, we can generate more accurate mode status. ' from here we can see that the user should describe explicitly the mode constraints among the arguments of a userdefined predicate in order to ease the analysis process. when the user builds a logic-based specification, he should also provide the mode constraints with each predicate he defines. alternatively, when the operation of the transformed program does not meet the user's requirements, the user can force the selection among the multiple procedural interpretations generated due to several possible mode combinations. further rules constituting mode constraints are: rule-3 (flow analysis) mm,,v = "-" =>mm,v="-"aw fail according to the unique-producer assumption, any mode information that includes more then one literal marked "" for the same variable can be discarded. thus this rule can be used to eliminate choices for modes marked "b", or terminate a particular search branch during flow analysis. rule-4 (flow analysis) mm,,v = => 1 is a clause head a 1' is a recursively literal in the clause body and matches with 1 a v and v' are the corresponding arguments let mm)iv = opposite mm|iv this rule is built to handle the case that a recursive literal in the clause body calls the same clause head. based on the different sign conventions between predicates in the clause head and body, we have to adjust the sign when this case occurs. 1 this applies to append as a user-defined predicate, rather than append as obtained from the decomposition of a list construct since the latter has a determined mode assignment. 21 ajis special issue requirements engineering 1999 rule-5 (flow analysis) mm|.v => l i s a clause head a 1' is a recursively literal in the clause body and matches with 1 a v and v' are the corresponding arguments a mm|.v * opposite mmi.v fail if the corresponding modes are not opposite, as discussed earlier, then the mode assignments constructed so far are inconsistent, and we have to backtrack and attempt to find a different assignment. in contrast to the data dependency and flow analysis technique presented here, debray and warren's approach lacks the power to handle multiple-mode combinations. as a result, the mode pattern of arguments of a predicate have to be generated by taking the least upper bound of all the possible success patterns generated by different calling patterns. our approach explicitly considers a multiple-mode possibility of a logic-based specification and is flexible enough to determine all possible mode combinations. for debray and warren, a logic program may just mean a single mode transformation process, whereas in our approach it may result in different data dependencies and execution sequences. therefore different target programs may be generated. in addition, our approach performs significantly better than debray and warren's method. the general procedure of mode analysis in debray and warren's approach consists of a computation of the set of success patterns and calling patterns of a predicate, and a compulation of the mode of a predicate by taking the least upper bound of all success patterns. the first one contributes most to the complexity of the algorithm. in order to compute the success pattern, the control of the algorithm must go from left to right, according to the evaluation order assumed, through each predicate of a clause, and then calculate the possible mode changes of all parameters of the predicates in the clause body. this will contribute o (n2) to the algorithm. but in order to calculate the mode modifications of each predicate in the clause body, the control must reclusively transfer to the analysis of success patterns of clause with the predicates as their head. so the overall complexity is o (n2). the best case occurs when all the predicates in the clause body are defined by fact clauses. in this case the complexity will reduce to o (n2). in the worst case (if there us a recursive call to the clause itself), the algorithm may not terminate. mode analysis in our approach proceeds sequentially through the matrix. if there exists a "b" in the matrix, then some modifications of the entries of matrix need to be applied. however, because of the close relationship between modes in the column direction, which is represented by the two assumptions, and the row direction, which is represented by the constraints of arguments within a predicate, a change of only a few of the "b"s to either "+" or "-"can completely specify all the modes in a matrix. so the average complexity of our algorithm is o (n2). the best case occurs when al the parameters of the predicates in a clause body are constants or ground variables, in which case all the predicates become test conditions and the mode analysis can be accomplished in linear time. the worst cases arises when all the cells marked with mode 'b are totally independent such that the analysis of each predicate in the clause body must return to modify the matrix of the clause itself. such backtracking may raise the complexity of the algorithm to o (n3). example 2 in this example, we will use some clauses to illustrate the effect of the data dependency and flow analysis. the format of the clauses has been adjusted by rewriting rules to help facilitate the mode analysis. for details of these rewriting rules, please refer to (tsai, j.j.p & weigert, t., 1993). these clauses are: 1) p(a,) < 2) r(a,,a2) <s(a,,n,w) a t(w,a2) 3) s(a,,a2,a3) <= (a,,[]) a = (a2,0) a (a3,[]) 4) s(ai,a2,a3) <car (a,,x) a cdr (a,,l,) a "f ' (n,f(n)) a = (a2,f(n)) a listcnstr(a2,[a2]) a append([a2],l2,a3) a s(l|,n,l2) 5) t(a,,a2)^ = (a1,[])at(a2.[]) 6) t(a|,a2) <— car(a|,h) acar(a|,l]) a "g" (h,g(h)) a listcnstr(g(h),[g(h)]) a append([g(h)],l2,a2) ,l2,a2)a t(l,,n,l2) 22 ajis special issue requirements engineering 1999 for clause (2), we select one of two possible choices (see figure 3), in this case, the choice results in r has mode "+" in clause (1). from the mode information of r, we further partially infer the modes of the parameters of s and t. because in column w there is no information about the mode status, it is marked with "b" in the two entries of the column. mode status may change due to more specific mode information obtained at a later stage. since the only entry in column w is not a constant, its mode has to be "-". (this entry is a "dead end", in that the value of this variable is not needed during the computation.) this information can be used to reduce the possible mode combinations for the clauses with the head s. in the following data dependency matrices shown, we mark entries that result form either the results of data flow analysis on other clause, or form alg. 1 by circling them. figure 4 shows the data dependency matrix obtained for clause (4). the selected decomposition for the list constructs of the forol specification allows us to eliminate from consideration the situation where the argument a3 of clause (4) has mode "+". this in turn determines completely the mode information for clause (2). the final matrix for clause (2) is given in figure 5. to analyze, clause (6) we obtain the mode of the clause head t from the matrix of clause (2). however, as figure 6 shows, the resulting matrix is not consistent. rule-4 of the second stage of the data flow analysis detects that the entries for the argument literal a2 of clause as a3 jv vv l"_"v^-'t "+" 0 "b" v { »+" ^"b" figure 3: the partial result matrix for clause 2 (6) conflict in both having mode "-". however, the selection of this particular decomposition for the list construct in clause (6) does not jeopardize mode analysis. when we fail to find a consistent mode assignment, we will backtrack to the decomposition phase of rewriting and choose an alterative decomposition.2 as a consequence, we also have to change this clause to 7) 7) t(a,,a2) <car(a,,h) a car(a,,l,) a "g" (h,g(h)) a listcnstr(g(h),[g(h)]) a append([g(h)],l2,a2) ,l2,a2)a t(l,,n,l2) 2 the decomposition of the list may also need to be changed in case the user requests other possible mode combinations, in which case we might have to explore all alternatives that can be generated that can be generated from using different ways of decomposing lists. 23 ajis special issue requirements engineering 1999 f\.< a: a! x l; a' li [..\ 2 1 car car "f appvna ii"-; i'v; -v-x' 'ex1 -*-'v a '+" o, figure 4: the final result matrix for clause 4 a: ,-! t -o, figure 5: the final result matrix for clause 2 24 ajis special issue requirements engineering 1999 car cdr a > a 2 listconstr h l, i* "b" "b" fail b>rulc-22 j> _i fa ."1 " '1 ^/ \> x fa "b1 'b1 figure 6: initial attempt to construct a matrix for clause 6 figure 7 shows the successful decomposition choice and the obtained mode assignments. in figure 8 and figure 9, we show the final data dependency matrices for cla~uses (3) and (5), respectively. note that for clause (3), the meaning of the = predicates vary based on different mode combinations of their arguments. the = predicate in the second row is that of a logical comparison since both of its arguments are grounded; the ones in the third and fourth rows are simply assignment expressions (e.g., a2 will receive a value form the constant "o"). car cdr car cdr a 2 11 © li l"-r".l ? figure 7: the final result matrix for clause 7 hybrid and-or parallelism implementation 25 ajis special issue requirements engineering 1999 after the static mode analysis is finished, the system then uses the generated mode information to implement the parallelism of the original logic-based specifications. the usage of mode information in the parallel model as mentioned above, most current approaches put tremendous overhead on the parallel execution system by doing dynamic analysis of the data flow, which is also called demand-driven. few of them do static data flow analysis. but none of them solves the problem satisfactorily, mainly because their static approach cannot guarantee that it will generate all the solutions of a logic program implied. in our approach, by using the four-valued mode combination and generating all possible mode combinations using bi-directional mode "b" in the matrix for each of the parameters of a predicate, all the solutions of a logic-based specification can be found from the matrix statically. the most significant advantage of this data dependency analysis is that the data flow information is available before any execution of the logic-based specification and any parallelization is attempted. making use of this early available mode information is the key contribution of the new approach. at a: a.i i i 0 |o, (o, ^ figure 8: the final result matrix for clause 3 a.a: ( ] •= figure 9: the final result matrix for clause 5 at each step of the dynamic evaluation of a logic-based specification, when the binding of a variable changes, the static data flow and dependency matrix is consulted to see if there exist some predicates in the logic-based specification clauses (not necessarily clauses from the same level) which can be evaluated in parallel, and the results generated from this simultaneous evaluation will enable further evaluation of other predicates in those clauses. these predicates can be executed in parallel and can come from different clauses which may reside in different levels in the searching tree. if there are no predicates which can be evaluated in the bottom-up fashion, then the general top-down evaluation process is invoked. the current approaches which employ mode information to solve binding conflicts generate and modify mode information at each step of the logic program. as a result, after each step of evaluation, mode analysis process is re-employed and mode status of arguments is changed. this changed mode information is used to facilitate further parallel evaluation of a logic program. so the parallel evaluation goes in a level by level fashion from the 26 ajis special issue requirements engineering 1999 respective of a search tree, after each step, mode analysis has to be invoked to find a more specific mode for arguments of a predicate in the lower level. instead in our model, since the complete mode information is available before any actual evaluation of the logic-based specification, the parallel evaluation can go more than one level in the search tree. so a predicate in a deeper-level of the search tree can be invoked if all its arguments which require value have been bounded by other predicates within the same clause, and this decision is made by checking the static mode information. in this case, the evaluation goes in a bottom-up fashion. that is, the parallel evaluation will explore predicates in a deeper-level in the search tree, collect the value there, and pass to other predicates within the same clause. the decision of when and where this bottom-up evaluation should be applied is solely decided by the static mode information. so the static mode information here serves to find predicates which can be executed in parallel from various levels of the search tree. and-or parallel execution in order to handle and-or parallelism, a bottom-up combined with a top-down evaluation scheme is used. current approaches always work from the top of an and-or tree and search all the way down to the tree. in contrast, our approach can find all the mode combinations before the user's query is available, it is possible that while the execution continues, some information about the predicates which will be executed several steps later in the old top-down model is available in advance. if we carry out those operations first, then more information of mode for unsearched predicates can be gathered in advance and the efficiency of the whole process can be improved. the approach uses the results generated from the static data flow analysis, combined with the user's input, to find out the portion of the and-or tree of which the execution behavior is deterministic under the current variable binding. this portion of the and-or tree is executed in a bottom-up fashion. compare this to other approaches, which rely completely on the user's input query to initiate the analysis, their execution is completely based on the dynamic determination of the data flow information. so the analysis is completely dynamic and executes in a top-down manner, which puts a tremendous overhead on the system. the overhead for the new model is the cost of the static mode analysis. in the previous section the cost for static mode analysis has been discussed and shown that when a mode can be obtained, the cost for the new approach is better than current approaches in the best and average cases, and is the same in the worst case. so among the approaches applying mode to resolve binding conflicts, our approach is substantially better than the current approaches in obtaining the mode information. the new hybrid evaluation model also uses the mode matrices during execution to obtain the actual mode status for arguments of the predicates. this portion of the cost is the same among all the approaches which adopt mode information. the cost to finish a mode check for a predicate is o ( n * m ), where n is the average predicates within a clause, m is the number of arguments within a clause. if we assume there is a central process which contains mode matrices for all the specification clauses, then an extra overhead is two communication channels to sent the current binding to and receive the currently executable predicates from the centered process. in the other word, one of these two channels sends the mode status for input query, the other returns generated parallelly executable predicates. these predicates will then be assigned processes and begin evaluation. the new approach shows significant improvement over the current approaches when the corresponding search tree has a large height value. the algorithm to check for predicates which can be executed in parallel begins by generating grounded variables. dynamic checking of parallelly executable predicates focuses only on the detection of newly generated grounded variables. in our new model, instead of only dynamic evaluation, all possible variable bindings and execution information have been included in the statically generated data dependency matrix. a very important property of our data dependency matrix is that it has all the possible execution sequences based on the multiple data dependency information. the only role a user's query plays is to initiate the analysis. at each step of the execution, especially whenever a new binding of a variable is generated, this information is used to restrict the multiple data dependency choices existing in the data dependency matrix and the possible execution sequences they imply. this restriction may result in only one possibility for the execution of a clause. so a large part of the analysis of our approach is done statically instead of dynamically. the execution starts from the root of an and-or tree, which corresponds to the user's query, then at each step of the unification, a variable binding is generated which will restrict the meaning and execution behavior of the clause. the system locates the branch of the and-or tree, which may not be the predicates from a single clause based on the multiple binding, so that the execution behavior of this branch is restricted to a single choice, and then executes this branch in a bottom-up fashion. all the remaining branches which cannot be restricted to a single choice will be executed top-down. this is how multiple solutions of a query are derived. even in the top-down case, the static data dependency matrix can still help. since all the solution have been predicted and represented in the data dependency matrix, the place where this multiple choice occurs can be determined from the same matrix. so the generation of multiple choices is still under the control of the system. 27 ajis special issue requirements engineering 1999 synchronization in or-parallel execution model the problem of preserving the multiple choices generated from the or-branches can be handled with the help of the same data dependency matrix. the major problem of or-parallelism is to synchronize the multiple bindings of the same variable generated from different branches. a simple synchronization mechanism must be built according to the mode information included in data dependency matrix, considering that all the binding information is supported by these matrices. based on the analysis of the and-or parallelism, there are two execution models, top-down and bottom-up. the realization of orparallelism is also closely related to these two models. during execution, if we find that the parallel execution of the logic program is in the top-down fashion similar to the current approaches, then the orparallel synchronization structure is identical to the current approaches using any of the currently available data structures and methods. if the execution proceeds in a bottom-up fashion, then the binding data structure will also be generated in a bottom-up fashion, which will simplify the data structures proposed for synchronization to only one level in the search tree. compare with other approaches, which have to keep a binding data structure for all the processes existing at the same time, this is a major improvement in our approach. also the creation of all these binding data structures can be delayed until all the variable bindings have been generated from all branches of an or-parallel point. the current approaches have to generate a synchronization data structure upon the first entry of a or-parallel branch point, and continue changing the synchronization data structure to reflect changes in the binding status of variables when the execution in any of its or-parallel branches changes. our approach reduces the life cycle of the synchronization data structures by waiting until they are actually needed to generate the complete binding combinations from orbranches. also the complexity of the or-parallel binding data structure is sharply reduced because we use a one level control structure. in contrast, current approaches have to use a synchronization hierarchy when dealing with the or-parallel synchronization problem. the simplification of the synchronization data structure comes from the control structure of the and-or parallelism. the choice of a specific synchronization data structure does not affect the new model's performance. calculation of the exec set in order to find predicates which can be evaluated in parallel, a search of the data dependency matrices is necessary to find all predicates which can be evaluated concurrently. the predicates are calculated dynamically according to the binding status of the clause and the static data flow and dependency matrix. all these currently valuable predicates are gathered into a data structure which has enough space to hold the syntax structure of any of the clauses in the logic-based specification. this data structure is called exec, which represents all the predicates at the current stage whose parameters' requirements for value in order for them to be valuable have been satisfied by the other predicates within the respective clauses. this set will include the parallelly executable predicates at any time in the process. the formal definition of exec is followed. definition exec is a set which contains the predicates from the logic-based specification clauses which can be executed in parallel. a predicates in exec if all of the input variables of the predicate have been bounded by other predicates within the same clause. the predicates within this set can come at any time during the process from clauses of different levels in the search tree, based on the binding status of the parameters within the predicate. the binding generated from the execution of this set will enable both the calculation of the new exec set and the general top-down evaluation of the logic-based specification, according to the binding status of the other predicates in the clause. if the predicates form more than one level are found to be executable, then they will be included in the new exec set, otherwise, the general top-down evaluation is invoked. so the proper calculation and application of this set become the center point of the whole approach. in the following, an example is used to illustrate how this mechanism works. the example is based on the data flow and dependency matrices generated in example 2, where generating the mode information is discussed. example 3. consider the case where the specification is called with a mode combination of r (bound, unbound), i.e., r is called with the first parameter bounded to a list and the second parameter unbounded. current approaches will first check the predicates in the body of the clause (1), which corresponds to the and-parallelism at the first level. in this case, there is no possible parallel execution, so a process 28 ajis special issue requirements engineering 1999 corresponding to predicate s will be created and the or-parallelism of the predicate s is explored, i.e. checking continues in a top-down fashion. in our approach, after the user's query is available, the input mode combination of r (-, +) is checked with the data dependency matrices to find the possible execution set exec. from the matrices for clause (1), we can see that the list provided from the clause head predicate r is consumed by the first parameter of the predicate s. since the predicate is a user defined predicate, the search for exec will go into the or-branches. based on the algorithm described below, all the predicates in the clause (2) can be executed simultaneously since for all of these predicates, all their parameters with the input mode have been satisfied. thus after examining the data dependency matrix for clause (2), exec = {=1, =2, =3}. meanwhile the search for executable predicates is also done at clause (3). in this case, the two predicates car, and cdr are found to be executable. finally exec becomes {=1, =2, =3,car, cdr}. in this example, instead of only the predicate s in the clause (1) being executed, the five predicates in exec can be executed in parallel. after the execution of the predicate s in clause (1) is completed, the value bounded in x] by the execution of predicate s is passed to the predicate t in the same clause, and the same process continues for t. the exec set corresponding to the first step of the execution of predicate t will be {=1, =2,car, cdr}, which comes from clause (4) and clause (5) respectively. please notice that within a clause, the behavior of our approach is similar to chen and wu's model [chen, a.c. & wu, c.l., 1991]. a predicate within a clause has to have all its parameters with input mode be matched with a producer in order for the predicate to be executable. so after the parallel execution of the above exec set, the next exec set of s will be executed. the same process continues until the parameter b2 and b3 in clause (3) receive a value from the execution of the clause. as this example shows, the execution of the new model relies on the statically built data dependency matrix to spawn as many parallelly executable processes as early as possible during the dynamic evaluation of the logic-based specification. in the following, an algorithm is introduced, which calculates the data structure exec set at each step whenever a new binding is introduced in the execution of the logic-based specification. the algorithm uses the exec to hold the predicates which are found to be parallelly executable by the algorithm. in the algorithm, exec is an external variable, similar to that in language c. the reason is that in any invocation of the algorithm, predicates which may be included in exec may come from various clauses in the logicbased specification, and the algorithm has to recursively call itself for each of the clauses involved in the calculation. in order to preserve the predicates in the exec set in the process of a multiple clause call, and form the complete image after the calling is finished, the data structure must be external. the same thing happens with the data structure clause, which holds the clauses which contributes to the current exec set. other predicates in the clause in this set will be searched for further evaluation when the current parallel evaluation of the exec is over. expanded is a two-dimensional data structure. it is built up for each of the clauses in the logic-based specification, which forms the first dimension. the second dimension is similar to the exec, which holds the predicates in each clause which has been expanded or searched. it is similar to the dead set in the conventional dps or bfs [luger, g.f. & stubblefield, w.a., 1989] algorithms. the only difference is that we have to consider the case when the clause is recursively called, the predicate may be moved from the expanded set, which means that the predicate has to be reevaluated under the new binding status in the new round of evaluation. expandedi is a duplicate of expanded except that it is defined locally. it is used to hold the partial results during the execution. in the algorithm, we assume there are n clauses in the logic-based specification, and the specification is called with the predicate predj, with the parameter of a, in the clause body of clausej. further assume that clause clausej has p predicates within its body, marked from 1 to p. algorithm 2 calculate_current_exec (predj (parai, ...... paraa),clausej) external: exec; external: clause; static: expanded[l..n] expanded i[l..n]; for each 1, (l<l<pal^i) spred, e expanded|j] a arity(pred,)==b if for each v e arg(predi) if mmp^^ "+" a v is not a constant then 3predk € clause;, s.t. (predk e expanded[j] v k == i) a (mmpredk,v== " — "); then if predi is not a user defined predicate 29 ajis special issue requirements engineering 1999 exec = exec + pred,; expanded, [j] = expanded] [j] + pred,; clause = clause + clause/, else if pred) is a user defined predicate for each clausem,s.t. head(clausem) == predi calculate_current_exec(pred|(para, parab),clausem); if none of these predicates exist, then report "evaluation finished"; expanded[j] = expandedi[j] the algorithm is called with a clause and its clause-head predicate as parameters. first the algorithm checks each of the predicates within the same clause which are not expanded predicates to see if any of them has all of its consumer parameters satisfied by some producer within the expanded set of the same clause. if no such predicate exists, then the evaluation of the clause has been finished. otherwise if a predicate found is not user-defined one, then the predicate and the corresponding clause are added to the data structure. if the predicate is user-defined one, then the corresponding clause is called with the appropriate parameters. with user-defined predicates, the parallelly executable predicates found in the subsequent evaluation will be added to the exec set, and the corresponding clauses will be added to the clause set. but the expanded set will be filled by the procedures called later. hybrid execution algorithm in this subsection, another algorithm is introduced, which explores the hybrid and-or parallelism of a logic-based specification. this algorithm is initially called when the user's query is available. the user's query predicate and the clauses unifiable with the predicate consist of the initial calls to the previous algorithm to calculate the first exec set. after the set is constructed, all predicates in this set are evaluated in parallel by creating a process for each of them. then the same exploration for the exec continues for the clauses which have contributed to the previous set and the parallel evaluation process proceeds for another exec set generated. in this recursive process, if the look-ahead process goes only one level deep, then we know that the new scheme does not contribute to the current evaluation of the clause, and the conventional top-down evaluation process is adopted. if the evaluation of a clause terminated, then the results generated from the evaluation of the clause is gathered to form a complete answer set and passed to the upper level in the search tree. this part is discussed further in the next section when the data structure for the or-parallel synchronization is discussed. when there is no clause in the current clause set, which contributes to the calculation of the previous exec set, then the process terminated. suppose the query is a predicate pred with a parameter of m. both sets clause and clause, hold the clauses which contribute to the calculation of the previous exec set. since clause will change in the process of the calculation for the next exec set, a duplicate clause data structure is created to keep the set consistent in the calculation process of the next exec set. algorithm 3 parailel_evaluation((pred(parai ,param=}) exec = 0; for each clause clause;, s.t. head(clausei) == pred calculate_current_exec(pred(para, .paraj.clauseo; for each of the predicates in exec, create a parallel process to evaluate it; do exec = 0; clause, = clause; for each clause clausej in clause),s.t. the head of clausej is predj(par|, ,paraj) calculate_current_exec(predj(parai ,paraj), clause^; if the evaluation of the current clause goes only one level deep the general top-down parallel evaluation scheme is adopted for the clause; if the evaluation of clausej has been finished form the cross product for the result of the clause from the results generated from each predicates in the clause; 30 ajis special issue requirements engineering 1999 add the result generated at this step to the storage area of the or-parallel processor one level higher; for each of the predicates in exec, create a parallel process to evaluate it; until all of the clauses in clausei have finished evaluation. efficiency considerations and experimental results in this section, the simulator is used to demonstrate the executional behavior of the conventional and the new parallel searching methods. for each set of simulations, the percentage of improvement of the new model over the conventional model is given, i.e., (1 new_val / old_val)%. the parameters for describing the search tree are adjusted in their full range to explore all possible combinations of the searching. the three constraints used are: 1. error_rate of the leaf nodes in the search tree. the higher the value, the more the leaf nodes will have fail truth value. 2. deep_jump_distance of the minimum and maximum deep jump distances. this serves as the basis for the new evaluation model. the higher the range of the value, the deeper the jump can be. 3. deep_jumping_rate of deep jumping. the higher the value, the more likely deep jumping will occur within the searching tree. in the table, mh means the minimum height of the search tree, xh means the maximal possible height of the tree, mw means the minimum width of the and-parallel branches, which is also the minimum number of predicates within a clause, and xw is the maximum width of and-parallel branches. 31 ajis special issue requirements engineering 1999 simulation for various values of error_rate the following table shows the effect of the error rate on the execution behavior. results for the testing of various values of error_rate mh=3 xh=5 mw=6 xw=10 mh=6 xh=9 mw=5 xw=7 mh=8 xh=13 mw=3 xw=5 mh=15 xh=20 mw=l xw=3 mh=20 xh=30 mw=l xw=2 mh=30 xh=50 mw=l xw=2 avg. error_rate = 0.00 total evalua. steps total nodes searched total cpu rounds total channel costs ttl wtd channel cst 5.88% — 28.57% -27.27% -15.29% 2.07% — 6.04% -2.96% -0.63% 2.50% ... 13.67% -0.28% 0.05% 15.79% ... 53.51% 12.90% 13.61% 29.17 % — 64.99 % 30.43 % 30.43 % 30.30 % ... 71.28 % 31.25 % 31.25 % 9.76% ... 30.79% 3.28% 14.68% error rate = 0.50 total evalua. steps total nodes searched total cpu rounds total channel costs ttl wtd channel cst 14.29% — 26.67% 14.29% 22.81% 20.00 % 21.43 % 44.90 % — 12.90 % 45.45% 41.67% 70.00% 45.00% 51.32% 57.89% 75.76% 89.03% 67.90% 65.77% 51.85 % 54.17 % 79.38 % 55.32 % 58.45 % 57.14 % 60.61 % 83.48 % 60.00 % 61.29 % 48.63% 59.48% 81.59% 53.28% 59.62% error rate = 0.99 total evalua. steps total nodes searched total cpu rounds total channel costs ttl wtd channel cst 14.29% — 26.67% 14.29% 22.81% 20.00 % 21.43 % 44.90 % 4.17% 27.02 % 45.45% 41.67% 70.00% 45.00% 51.32% 57.89% 75.76% 89.03% 67.90% 65.89% 51.85 % 54.17 % 79.38 % 55.32 % 58.45 % 62.86 % 60.61 % 83.48 % 60.00 % 61.29 % 48.63% 59.84% 81.59% 53.69% 59.99% from the table, we can see that with increasing error rate, all of the average improvements of the new execution model have values substantially better than the previous case in the corresponding column. for each row, where the depth of the evaluation changes, it can be seen that the improvement for deeper evaluation is generally better than for the shallow evaluation. this is true because the new approach can go into deeper levels more quickly than the general approach. so while the error rate increases, those leaf nodes with error as their truth value can be discovered earlier than the general approaches can discover them, and short cuts can also be discovered much earlier. the new approach focuses on deep jumps, which is directly related to the depth of a tree. for the shallow search tree, the search does not go deep into the tree, and the advantage of the new approach cannot overcome the overhead of the new approach. so for deep search processes, the advantage for the new approach is obvious. 32 ajis special issue requirements engineering 1999 simulation for various values of deep_jumping_distance in the following, the effect of the various depths and widths of search trees is analyzed using the deep jump distance. results for the testing of various values of deep_jumping_distance mh=3 xh=5 mw=6 xw=10 mh=6 xh=9 mw=5 xw=7 mh=8 xh=13 mw3 xw=5 mh=15 xh=20 mw=l xw=3 mh=20 xh=30 mw=l xw=2 mh=30 xh=50 mw=l xw=2 avg. v!in_deep_jump_distance = 1 , max_deep_jump_distance = 2 total evalua. steps total nodes searched total cpu rounds total channel costs til wtd channel cst 14.29 % — 26.67 % 14.29 % 22.81 % 10.00 % 42.86 % 32.65 % 16.67 % 16.13 % 18.18% 41.67% 40.00% 35.00% 30.79% 31.58 % 46.79 % 62.92 % 44.44 % 40.08 % 33.33 % 33.33 % 57.85 % 36.17 % 37.19 % 31.43 % 33.33 % 54.71 % 33.85 % 33.65 % 27.52% 39.87% 56.10% 35.66% 34.94% min deep jump distance = 2, max deep jump distance = 4 total evalua. steps total nodes searched total cpu rounds total channel costs ttl wtd channel cst 14.29 % — 26.67 % 14.29 % 22.81 % 20.00 % 21.43 % 44.90 % — 12.90 % 18.18% 41.67% 40.00% 10.00% 15.79% 47.37 % 65.15 % 80.94 % 56.79 % 56.17 % 40.74 % 45.83 % 67.69 % 44.68 % 47.29 % 51.43 % 54.55 % 78.33 % 53.85 % 55.16 % 39.45% 52.29% 73.86% 43.03% 50.84% vlin_deep_jump_distance = 4, max_deep_jump_distance = 7 total evalua. steps total nodes searched total cpu rounds total channel costs ttl wtd channel cst 14.29 % — 26.67 % 14.29 % 22.81 % 20.00 % 21.43 % 44.90 % — 12.90 % 54.55% 50.00% 78.00% 55.00% 61.84% 57.89 % 75.76 % 89.92 % 69.14 % 67.88 % 55.56 % 58.33 % 82.77 % 59.57 % 62.68 % 54.29 % 57.58 % 80.99 % 58.46 % 59.71 % 49.54% 60.13% 81.59% 54.92% 60.49% obviously the behavior of the new execution model improves substantially when the deep jumping distance increases. this best demonstrates the improvement of this new model over traditional evaluation models. the execution also improves with increased depths. this point was analyzed when the effect of error_rate was discussed. 33 ajis special issue requirements engineering 1999 simulation for various values of deep_jumping_rate the next table examines the effect of the rate of deep jumps during the search process. results for the testing of various values of deep_jumping_rate mh=3 xh=5 mw= 6 xw=l 0 mh=6 xh=9 mw=5 xw=7 mh=8 xh=13 mw=3 xw=5 mh=15 xh=20 mw=l xw=3 mh=20 xh=30 mw=l xw=2 mh=30 xh=50 mw=l xw=2 avg. deep_jumping_rate = 0. 1 total evalua. steps total nodes searched total cpu rounds total channel costs ttl wtd channel cst — — — — 7.02 % — — — — — 18.18 % 16.67 % 34.00 % 15.00 % 16.58 % 15.79 % 28.79 % 41.51 % 22.22 % 17.62 % 11.11% 12.50% 22.15% 12.77% 13.59% 5.71% 6.06% 11.55 % 6.15% 7.03% 9.18% 16.99% 22.60% 12.71% 10.49% as excepted, increase in the deep jumping rate improves the performance of the new execution model more and more substantially. since the advantage of this new approach comes solely from the deep jumping in the execution and hybrid execution models, the results is quiet noticeable. in other simulations, we found that with increases in the variable dependency rate, the improvement of the model decreases. the improvement, in the worst case, still achieves an average rate of about 20%. the result also show that as the values of non-determinism rate increases, the new parallel evaluation model demonstrates better evaluation behaviour. conclusion to improve the performance of the validation and verification process of logic-based requirements specifications, we presented a hybrid parallel execution model. it handles and-parallelism, or-parallelism, and eliminates of backtracking related to a logic-based specification. the significance of this model is • it is the first model to discuss the parallel execution of a logic-based requirements specification. current approaches all focus on the parallelism of various logic programs. • the new model uses a static data dependency approach to generate mode information for each of the variables in the predicate of a clause. this static mode information is the basis for the whole parallel evaluation process. the new model is guaranteed to find the maximum degree of andand or-parallelism by adopting a new hybrid execution model. this model finds a parallel executable set of predicates at each step when a new variable binding is created based on the static mode information. • because the static data dependency analysis is guaranteed to find all possible execution paths and all solutions of a logic-based specification, this model can eliminate the backtracking structure from the execution control structure. • at each step of the execution, the correct processes are divided into several clusters by the clauses. these clauses have predicates which are currently executable or have been active because of the static analysis. as a result, the total idle time of communication channels needed for the parallel processes is substantially lower than the idle time of the current approaches, where it is possible for a communication channel to be idle for a long time before a message is collected from deep predicate of one branch and been sent to the deep predicate of another branch. • during the and-parallel execution, if intermediate nodes do not contribute to the binding of the related variables, we do not need to generate these intermediate processes, which reduces the overhead of 34 ajis special issue requirements engineering 1999 the parallelizing scheme. during the or-parallel execution, the creation of the binding structure can also be delayed until the variable bindings actually are generated form the branches of the or-parallel points. • since at each step the search goes deep into the and-or tree of a logic-based specification and the evaluation is applied to the clauses which have predicates currently executable, it is possible to apply the classical short-cut algorithm here earlier than other approaches can and thereby reduce the space of the and-or tree searched to generate the complete multiple solutions. as a consequence, the time and space saved using the short-cut algorithm will be maximized. • the hybrid execution of or-parallelism reduces the complexity of the synchronization data structures used for management of different bindings generated from different branches. this is reflected in the possibility of delaying the creation of binding data structures and the one-layered binding data structure synchronization. the bottom-up execution will first go deep into the and-or tree hierarchy, a data structure at an or-parallel merging point will not be created until all the variable bindings are actually generated from the evaluation of the or-branches below the or-parallel point. according to the bottom-up execution model, all the bindings generated from different branches of an or-parallel predicate are already available before the merging algorithm at the same or-branch point is called. so there is no need to maintain synchronization data structure for more than one level below the or-parallel branch point. this will sharply reduce the complexity of creation and maintenance of all the synchronization data structures used by most of the current approaches without any limitation on the level of information a binding list needs to maintain. the only overhead we have here is a tag which distinguishes solutions generated one level deep from one or-parallel sub-branch from others. references: p. borgwardt, "distributed semi-intelligent backtracking for a stack-based and-parallel prolog," ieee 1986 symposium on logic programming, pp.211-222, salt lake city, utah, sept. 1986. m. bruynooghe, "adding redundancy to obtain more reliable andmore readable prolog programs," proceeding of the first international logic programming conference, marseille, france, 1982. m. bruynooghe, b. demoen, a. callebaut and g. janssens, "abstract interpretation: towards the global optimization of prolog programs," proceedings of the 4th ieee symposium on logic programming, san francisco, california, september 1987. r. butler, e.l. lusk, r. olson and r.a. overbeek, "anlwam: a parallel implementation of the warren abstract machine/'internal report, argonne national laboratory, u.s.a., 1986. r. butler et. al., "anlwam: a parallel implementation of the warren abstract machine," (\em internal report, argonne national laboratory, u.s.a., 1986. j.-h. chang and a.m. despain, "semi-intelligent backtracking of prolog based on static data dependency analysis," ieee 1985 symposium on logic programming, pp.10-21, boston, massachusetts, july 1985. j.-h. chang, a.m. despain and d. degroot, "and-parallelism of logic programs based on a static data dependency analysis," digest of papers, compcon spring '85, pp.218-225, feb. 1985. c. chen and c.-l. wu, "a parallel execution model of logic programs," ieee trans, on parallel distributed sys., pp. 79-92, vol. 2, no. 1, january 1991. k. clark and s. gregory, "parlog: parallel programming in logic," research report doc , department of computing, imperial college of science and technology, 1984. codognet and p. codognet, "non-deterministic stream and-parallelism based on intelligent backtrackings," logic programming, proceedings of the sixth international conference}, pp.63-80, lisborn, portugal, june 1989. codognet, "yet another intelligent backtracking method," logic programming , proceedings of the fifth international conference and symposium}, pp.~447-465, seattle, washington, august 1988. j. s. conery, "the and/or model for parallel interpretation of logic program," phd thesis, dept. of information and computer science, univ. of california, irvine, 1983. t. conlon, "programming in parlog," publisher: addison-wesley publishing company, 1989. s. k. debray, "static analysis of parallel logic programs," logic programming, proceedings of the fifth international conference and symposium, pp. 711-732, seattle, washington, august 1988. s. k. debray, "flow analysis of dynamic logic programs," the journal of logic programming, vol. 7: pp. 149-176, 1989. s. k. debray and d. s. warren, "automatic mode inference for logic programs," the journal of logic programming, vol. 5: pp. 207-229, 1988. degroot, "alternate graph expressions for restricted and-parallelism," compcon spring '85, pp. 206210, feb. 1985. 35 ajis special issue requirements engineering 1999 degroot, "restricted and-parallelism and side effects," 1987 ieee symposium on logic programming , pp. 80-89, san francisco, california, august 1987. d. degroot, "restricted and-parallel execution of logic programs," parallel computation and computers for artificial intelligence, pp. 91-107, editor: janusz kowalik, publisher: kluwer academic publishers, 1988. p. dembinski and j. maluszynski, "and-parallelism with intelligent backtracking for annotated logic programs," 1985 ieee symposium on logic programming), pp.~29-38, boston, massachusetts, july 1985. gupta, et. al., "idiom integrating dependent and-, independent and-, and or-parallelism," logic programming, proceedings of the 1991 international symposium), pp. 152-166, san diego, california, october, 1991. m. hailperin and h. westphal, "a computational model for pepsy," technical report ca-16, ecrc, 1985. b. hausman, a. ciepielewski and s. haridi, "or-parallel prolog made efficient on shared memory multiprocessors," 1987 ieee symposium on logic programming), pp.~69-79, san francisco, california, august 1987. m. v. hermenegildo, "an abstract machine for restricted and-parallel execution of logic programs," third international conference on logic programming), "lecture notes in computer science," springerverlag, vol. 225, 1986. m v. hermenegildo, r. warren and s. k. debray, "global flow analysis as a practical compilation tool", journal of logic programming pp. 349-366, 1992:13. d. jacobs and a. langen, "compilation of logic programs for restricted and-parallelism," european symposium on programming), nancy, france, 1988. v. kumar and y.-j. lin, "an intelligent backtracking scheme for prolog," 1987 ieee symposium on logic programming, pp.406-414, san francisco, california, august 1987. peyyun p. li and alain j. martin, "the sync model: a parallel execution method for logic programming," ieee 1986 symposium on logic programming, pp.223-234, salt lake city, utah, sept. 1986. f. luger and w. a. stubblefield, artificial intelligence, benjamin/cummings co. 1989. mannila and e. ukkonen, "flow analysis of prolog programs," proceedings of the 4th ieee symposium on logic programming, san francisco, california, september 1987. c. s. mellish, "some global optimizations for a prolog cormier", (\it journal of logic programming, vol. 2, no. 1: pp. 43-66, april 1986. p. raman and w. stark, "fully distributed and/or-parallel execution of logic programs," logic programming, proceedings of the fifth international conference and symposium), pp.~l 189-1203, seattle, washington, august 1988. u. s. reddy, "transformation of logic programs into functional programs", proceedings of the 1984 international symposium on logic programming, atlantic city, new jersey, ieee computer society: pp. 187-196,february 1984. e. y. shapiro, "a subset of concurrent prolog and its interpreter," (\em tr-003), icot-institute for new generation computer technology, japan, jan. 1983. k. shen, "exploiting dependent and-parallelism in prolog: the dynamic dependent and-parallel scheme (ddas)," logic programming, proceedings of the joint international conference and symposium on logic programming), pp.717-731, 1992. g. smolka, "making control and data flow in logic programs explicit," proceedings of the 1984 symposium on lisp and functional programming, austin, texas, august 1984. k. steer, 'testing data flow diagrams with parlog," logic programming, proceedings of the fifth international conference and symposium, pp. 96-110, seattle, washington, august 1988. p. tinker and g. lindstrom, "a performance-oriented design for or-parallel logic programming," logic programming, proceedings of the fourth international conference, pp. 601-615, melbourne, australia, may 1987. j.-p. tsai, t. weigert and h. c. jang, "a hybrid knowledge representation as a basis of requirement specification and specification analysis," ieee transactions on software engineering, vol. se-18, no. 12, pp. 1076-1100, december 1992. j.j.-p. tsai and t. weigerl, knowledge-based software development of real-time distributed systems), world scientific, 1993. j. p. tsai, b. li, and t. weigert, "a logic-based transformation system," ieee transactions on knowledge and data engineering), vol. 10, no. 1, pp. 91-107, jan. 1998. j. p. tsai, a. liu, e. juan, and a. sahay, "knowledge-based software architecture: acquisition, specification, and verification," ieee transactions on knowledge and data engineering, vol. 11, no. 1, pp. 187-201, jan./feb. 1999. d. h.d. warren, "the sri model for or-parallel execution of prolog-abstract design and implementation issues," 1987 ieee symposium on logic programming, pp.92-102, san francisco, california, august 1987. 36 ajis special issue requirements engineering 1999 h. westphal, et al., "the pepsys model: combining backtracking, andand or-parallelism," 1987 ieee symposium on logic programming, pp. 436-448, san francisco, california, august 1987. n. s. woo and k.-m. choe, "selecting the backtracking literal in the and/or process model," ieee 1986 symposium on logic programming, pp.200-210, salt lake city, utah, sept. 1986. 37 ajis vol. 4 no. 2 may 1997 the need for an applied computer ethics handbook daniel salber clips-imag, university of grenoble b.204/bp53 38041 grenoble cedex 9 france email: daniel.salber@imag.fr abstract information technology is developing at an astounding pace. computers are now common and their use is switching from number-crunching tools to "information and communication appliances". the growing success of (he internet is just an example of how information and communication facilities are now available to the general public. but today's internet is just the precursor of more innovative applications. new advances in computer science disciplines such as communications networks and mobile devices are suggesting radically new uses of information technology. these new uses may raise new ethical issues. we give a few examples and show that these issues are not explicitly dealt with in existing ethical frameworks such as mason's papa or huffs impactcs. we also discuss the usability of these frameworks by software designers and suggest that information technology practitioners need more explicit tools such as handbooks to help them understand and deal with ethical issues. introduction this paper presents a computer scientist's perspective on ethical issues that may arise in future computing environments. we believe that ethical issues should be given thought very early, even before building a prototype or developing a new computing paradigm. following software engineering discipline, high-level requirements such as ethical concerns raised by features of the system should be considered in the very first phases of systems development. if ethics are taken into account as an afterthought, the new system or paradigm may break a number of ethical principles and may be very difficult and costly to modify. ethical concerns should help shape tomorrow's computing environments. we first describe a few examples of systems where ethical concerns had to be taken into account after the fact. we then present some examples of current active research areas which question existing ethical frameworks. we then argue that it practitioners need better tools to help them understand what and where ethical issues are. ethical issues in recent and future it developments information technology (it) is developing fast. the world-wide web concept, for example, was first implemented only three years ago and is now widely popular among internet users. the java language now extends the world-wide web concept from an hypertext document base to a collection of documents and executable programs. java has brought answers to old computing problems for which solutions where not readily available less than a year ago. examples include cross-platform portability of executable code or secure execution of remote code. these capabilities bring to front ethical issues related to transaction security, authentication and privacy and also to software and networks reliability. in a first paragraph, we present examples of commercial systems where privacy has obviously not be taken into account in the design phase. these systems had then to be redesigned. in the second paragraph, we discuss some issues related to software reliability. in the third paragraph, we introduce two examples of recent technological advances which raise serious ethical concerns. let us state right now that we don't intend to be exhaustive but we focus on areas that we have rarely seen mentioned in the computer ethics literature. two privacy-related examples many new privacy-related issues surround the recent developments of the internet and computer-mediated communication. we first give two examples of systems which have been designed without caring for ethical issues and which had to be modified due to users' concerns. in the first example, the designers recognized they didn't think about the issues. the second case raises questions about the designers' intent. magic cap business cards magic cap is an operating system running on mobile personal digital assistants. magic cap provides facilities for communication between users through phone, e-mail or fax. with this system, each user has a personal 141 ajis vol.4 no. 2 may 1997 electronic business card in an address book which is used for two purposes. first, information stored in the business card may be used by the system, for example for phone call or email forwarding. second, the electronic business card is sent to other users, in the same way one gives a business card to a colleague who asks for contact information. in the first version of magic cap, giving a business card to someone also meant giving away any personal information that was stored on it. thus, users either were reluctant to give away their business card which contained fields such as "private personal phone number" or wouldn't fill out personal information and thus couldn't use some advanced features of the system. due to users' requests, these two uses of the electronic business card had to be separated. in the newer version of the system, each field of information has an "unlisted" checkbox which allows the user to prevent private information to appear on the business card as related by jerney (1995). in this case, the mistake is blatant and was even recognized by the designers. personal information should be dealt with carefully and the electronic business card function should be handled differently from the personal information function. designers should be better warned about the risks of a system dealing with user's personal information as is the case with most personal digital assistants today. we'll show in the next paragraph that mobile systems also bring up new privacy issues. this example shows that customer concern over privacy may lead software designers to redesign software, and, of course, at a cost. the next example brings up a different issue: if the system includes a mechanism that deals with some user personal information, the user should be made aware of this fact. netscape cookies netscape "cookies" are a software mechanism through which world-wide web servers can store information on the computer of the visiting client browser. this information may be any code or string sent by the server as a cookie. although the cookie mechanism is intended for harmless uses such as a server remembering a client's last visit, it may be diverted and thus raises ethical concerns. the most worrying potential use of this capability is the possibility for a server to "know" if another given page or server has been visited by the client. there is one condition though for this to be possible: the previously visited server or page and the server which asks for the information should be in the same "url range" (url stands for uniform resource locator, i.e., the unique address of a web page). but an url range may be as large as "all sites ending with acme.com" as defined in netscape (1996). this possibility raises a privacy risk if used improperly: a server may get information about which sites a user has previously visited. until recently, cookies transactions were not even noticeable to the user. everything happened "behind the user's back", giving him absolutely no control over the information that was stored in the cookies file. this design choice, when discovered, was questioned by the internet users community: cookies are a mechanism which may benefit servers, for example for tracking customer fidelity or for presenting different advertisements every time a user connects to a server. why did the designers hide this feature from the user, especially if it may involve privacy risks? this speculation has remained unresolved, but in the new netscape navigator version 3, netscape added the option for the user to be notified of new cookies and to allow or not cookies to be stored on his/her machine. although this option is disabled by default thus leaving the novice user ignorant of cookies, the informed user has a way to track the storage of cookies and can even forbid them. we'll show in section 3 that a basic human-computer interaction principle could have helped analyse this problem. it is interesting to note that the cookie blocking capabilities of netscape version 3 were not considered convincing by internet users. systematic confirmation/denial of cookies is required from the user and is largely intrusive (some web documents send several cookies, requiring user confirmation/denial for every cookie). this poor design has triggered reactions: on the apple macintosh platform alone, six different freeware or shareware utilities are available to clean up cookies after every netscape session. these examples show that computer-mediated communication software is prone to involve privacy risks and that software designers should be better informed of the issues. but the possibilities brought by computer-mediated communication are rapidly changing the whole software industry. a worrying aspect of this change is the lower reliability of some software distributed over the internet. software reliability beta software is unreliable software a recent trend on the internet is the wide distribution of unfinished software also known as "public beta". in software engineering, the word "beta" to qualify the development stage of a software product has a precise meaning. beta software is considered feature-complete by the development team and has no known serious bugs. 142 ajis vol.4 no. 2 may 1997 according to marciniak (1994), "the purpose [of beta-testing] is to find problems that show up in actual operation before many copies of the system are released". when a product is in beta stage, it is usually distributed to a group of technically-aware users ("beta-testers") with the aim of testing the product on a wide range of software and hardware combinations to detect possible incompatibilities that the quality assurance team couldn't have found under normal conditions. beta-testers are asked to "stress-test" the software and report technical details, should they find a remaining bug. with the advent of "public betas" freely available on the internet, the situation has changed radically. any user, whatever her/his technical proficiency, can download and use a beta version. most users aren't even aware that they use beta software, because beta software is not often clearly labeled so, except for the version number which usually contains a "b". moreover, most "public betas" software is at a development stage that would rather be called "alpha" (feature-complete but known bugs still remaining) or even "development release" (not feature-complete and still in development). the reasons for this fury over beta versions are outlined in a recent issue of an electronic newsletter duncan (1996). in an environment where communication is fast and ubiquitous, software companies are making every effort to get attention and distributing new beta versions of their software every few weeks is a way of making news. however, this behaviour has some consequences for users and may be damaging to the software industry in the long term. every user is now a potential beta-tester, even if s/he lacks the necessary technical background, and most of all, the willingness to be one. actually, development or alpha versions of software may even contain dangerous bugs. the user is thus left at risk on his own, since most software products are distributed with a license agreement by which the software company declines responsibility in case of software misbehaviour. users are now commonly using software that is known to be defective, since they won't always upgrade to the final version if they have a public beta. in the long term, this strategy may be disastrous: confidence in software is already lower and lower and "crashing software" is a common experience among users. giving away buggy software to inexperienced users may be rewarding for a company's publicity but carries the risk of diminishing seriously the confidence users have in software and the software industry in general. new approaches in building software such as the component software approach we discuss next may make this problem even worse. reliability of component software maner explains that a unique characteristic of information technology lies in its complexity maner (1995). because of the large number of interacting components, even in a basic system, hardware and software are prone to errors and bugs which are difficult to isolate and correct. a current trend in software engineering aims at building a software product from smaller software parts (or components) put together. this approach, named "component software", has some definitive advantages from a software engineering point of view. small pieces can be designed to be highly reusable across a variety of software projects, thus reducing development time and cost. java or opendoc are examples of this approach. however, one may wonder at the implications of this approach with regard to software reliability. increasing the number of components increases the complexity of software and thus the un-foreseeable or unwanted interactions between components. although some approaches, for example the use of mathematical tools such as formal methods, alleviate these risks, these methods are costly and used only for critical systems such as flight decks or power plant control systems. we don't deny the clear advantages of the component software approach: it allows reusability of ready to use software components and facilitates standardization of software. however, we believe it is misleading to advocate this approach for cost reasons. if component software actually cuts software coding costs, it requires a much more thorough testing approach and thus costs a lot more in testing. we should emphasize that software testing and stronger software engineering practices are the only key to software reliability. future computing environments and ethical concerns when looking at trends of computer science technology, one may wonder at the ethical implications of some recent developments. in this paragraph, we outline a few of these recent advances and the ethical issues they raise. who is responsible for "intelligent agents"? "intelligent agents" are a new computing paradigm. it aims at changing the way the user works from "doing" to "letting do". most current systems follow the traditional "direct manipulation" paradigm of graphical user 143 ajis vol.4 no. 2 may 1997 interfaces: users point at objects (such as text paragraphs or shapes of a drawing) and act on them. similarly, when using the world-wide web, users point and click on links they want to visit and have to elaborate information retrieval plans when looking for specific information. with agents, users are able to delegate tasks to software agents that will then act on behalf of the user. for example, the user may delegate the task "make plane and hotel reservations for a two-day trip to london on monday next week". based on the user's preferences or habits, the agent would then roam the network, visiting the airline company's and the hotel's servers and making appropriate reservations, then getting back to the user to inform her/him of the results. general magic's telescript system described in white (1995) already allows for the use of such agents on a dedicated network. the agents paradigm raises issues related to privacy and responsibility. a software agent wandering on a network carries a great deal of information on the user whose behalf it is acting on: personal information such as name or credit card number for handling electronic transactions, but also information required to carry the agent's task, for example the user's daily schedule or airline and hotel preferences. attention should be given to the privacy of this information. security mechanisms should prevent an hostile host from "hi-jacking" a software agent or from gathering data from visiting agents. another issue is responsibility: if an agent performs unwanted transactions whose responsibility is involved? the difference here with the traditional software paradigm of "doing" is that the agent acts following instructions given by the user. one should expect users to make errors when giving instructions to agents and also software agents to be sometimes faulty. should the user or the software company or the visited host be held responsible in case the agent misbehaves? another area of concern is the use of mobile systems. new risks to privacy occur due to the very nature of mobile devices. ubiquitous computing and privacy of location with mobile systems, such as portable phones or personal digital assistants (pda), users are now free to use information technology while on the road and in different places. the "ubiquitous computing" paradigm proposes the implantation of fixed infrastructures in buildings that allow mobile systems to communicate, whatever their location inside the building or whatever building they are in. this paradigm raises serious issues with regard to privacy. whenever a mobile device establishes a communication through the fixed infrastructure, the location of the entry point nearest to the user is known to the system. thus, it is possible for the system to know the user's location with great accuracy: a typical entry point range is a few meters or hundred meters. a similar problem already occurs with cellular phones: the cellular phone infrastructure divides geographical space in small cells. it is then possible to know which cell a user carrying an active cell phone is in. not only can users be located, but more worryingly, they are often not aware of this possibility. once again, system designers should inform the users of ethical issues raised by the use of their systems. these examples cover a broad range of issues but are symptomatic of a common problem. software designers and computer science researchers aren't aware of ethical issues that may arise in the systems they develop. practical tools are needed to help designers identify and assess potential ethical issues. towards an it practitioner's computer ethics handbook a possible first step towards a better understanding of ethical issues is a categorization of the areas of ethical concern. ethical frameworks such as papa in mason (1986) or impactcs in huff (1995) are certainly adequate to classify ethical issues into categories. but their intent is not practical use in software design. impactcs is intended for teaching ethics and therefore, as the authors recognize, requires a good knowledge of the ethics literature to identify issues and refine the framework's categories. papa suffers some similar limitations: without previous exposure to computer ethics, these frameworks are of little value to the practitioner. in the case of new technologies, even previous experience with computer ethics may not be sufficient. for example, the problem of privacy of location raised by the use of mobile systems is not commonly dealt with in the computer ethics literature. computer ethics codes, such as the ones proposed by the acm or ifip certainly are valuable to the practitioner. but here again, their scope is too broad for them to be practically usable in software design. they present general principles but are of little help when confronted to a software design decision. another approach is to establish a bridge with human-computer interaction. human-computer interaction deals with the design of user interfaces. the user interface of a computer system mediates the interaction between the user and the system. in the terms of norman, the user interface presents to the user the image of the system from 144 ajis vol. 4 no. 2 may 1997 which the user builds a cognitive model of the system behaviour and functions norman (1986). user interface design relies heavily on collections of rules often devised by ergonomics specialists, which practitioners apply when designing a system. we first introduce a simple basic rule of human-computer interaction and show how computer ethics could provide input for its use. we then show how it applies to an example of the previous section. the observability rule in human-computer interaction the rule of observability states that the user interface of a system should inform the user about the current internal state of the system that is relevant to the user's current task (abowd (1992)). this rule is sometimes called "feedback principle" because it implies that any action from the user must trigger a response from the user interface (a user action modifies the internal state of the system). the difficult part in applying this rule is deciding what part of the internal state of the system is actually relevant to the user's task. computer ethics could help decide what information should be considered relevant and thus be presented to the user. in the example of netscape cookies, observability applies easily. in this case, the system to be considered is the user's browser. when handed a cookie by a server, the client browser's internal state is modified: an information is received and then stored in a file. when a server asks for a previously stored cookie, the internal state is again modified since this information is fetched in the file by the browser and then transmitted to the server. is this information relevant to the user's task? when netscape first didn't give any possibility to the user to know about cookie transmission, they considered that it was not relevant. in the current version of netscape's browser, information about cookies is presented to the user only if a server hands in a cookie. if the server fetches a previously stored cookie, the user is not informed about it. there is here a clear discrepancy between the presentation of both operations to the user. if privacy concerns were taken into account consistently, both operations would be considered relevant to the user's task and the user interface of the browser would notify the user in both cases. to be of value to practitioners, computer ethics could provide them with practical rules to be taken into account when designing systems. these rules could build on the observability rule. towards a practitioner's computer ethics handbook to take into account computer ethics in system design, designers need clearly outlined issues expressed in terms of their practical concerns instead of broad categories of ethical concerns. a first step could be to recommend the use of basic human-computer interaction principles. human-computer interaction prescribes a "user-centered" approach to design. it emphasizes that computers are designed for users to accomplish tasks in a human way, and not for users to adapt to the peculiarities of the software they use. the observability rule should also be presented. it doesn't prevent a system to break computer ethics principles such as privacy, but it requires the user to be informed about the capabilities of the system. it forbids a system to perform actions "behind the user's back". this applies to netscape cookies but also to the problem of privacy of location we have mentioned above. if a mobile system is able to transmit the location of the user, at least the user should be made aware of this. practical recommendations would be helpful to designers. for example, they should be warned about the potential issues of systems dealing with the user's personal information such as software agents or mobile systems. not only should the system inform the user when personal information is about to be disclosed, but the system should provide a way of keeping personal information really personal if the user wishes so. systems permitting remote execution such as software agents should also be mentioned as raising issues related to privacy of personal information and responsibility. finally, emphasis should be put on software quality and the risks of unreliable software. software engineering could certainly be helpful here, but better software quality really requires a profound change in current software development practices. conclusion we have presented a few examples of recent systems and technologies that raise ethical issues. we have also shown how practitioners need to be better educated about ethical concerns. there is definitely a need for practical recommendations that software designers could apply without possessing a background in computer ethics. human-computer interaction rules could provide a first basis for taking computer ethics principles into account in software design and disseminating them efficiently to designers. 145 ajis vol.4 no. 2 may 1997 references abowd, g., coutaz, j. & nigay, l. (1992) "structuring the space of interactive systems properties", ehci'92, ifip tc2/wg2.7 working conference on engineering for human-computer interaction, ellivuori, finland, pp. 113-130. duncan, g. (1996) "waiting with beta'd breath", tidbits, n° 328, vol 1. <http://www.tidbits.com/tbissues/tidbits-328.html>. huff, c. & martin, c.d. (1995) "computing consequences", communications of the acm, n° 12, vol 38, pp. 75-84. jerney, j. (1995) "magic cap: straight from andy hertzfeld", mobilis, n° 8, vol 1, pp. 1-9. <http://www.volksware.com/mobilis/august.95/hertzl.htm>. maner, w. (1995) "unique ethical problems in information technology", proceedings of ethicomp95, an international conference on the ethical issues of using information technology, leicester, uk. marciniak, j.j. (1994) "categories of testing", encyclopedia of software engineering. new york: john wiley, pp. 90-93. mason, r.o. (1986) "four ethical issues of the information age", mis quarterly, n° 1, vol 10, pp. 486-498. netscape (1996) "persistent client state http cookies", netscape communications corporation. <http://www.netscape.com/newsref/std/cookie_spec.html>. norman, d. & draper, s. (1986) "user centered system design: new perspectives on humancomputer interaction", hillsdale, nj: lawrence erlbaum associates. white, j.e. (1995) "mobile agents", general magic corporation. <http://www.genmagic.com/telescript/whitepapers/wp4/whitepaper-4.html>. 146 microsoft word ajis special 2002.doc ajis special issue december 2002 40 the foundational role for theories of agency in understanding information systems design robert b johnston & simon k milton department of information systems the university of melbourne r.johnston@dis.unimelb.edu.au s.milton@dis.unimelb.edu.au abstract in this paper we argue that theories of agency form a foundation upon which we can build a deeper understanding of information systems design. we do so by firstly recognising that information systems are part of purposeful sociotechnical systems and that consequently theories of agency may help in understanding them. we then present two alternative theories of agency (deliberative and situational), mainly drawn from the robotics and artificial intelligence disciplines, and in doing so, we note that existing information system design methods and ontological studies of those methods implicitly adhere to the deliberative theory of agency. we also note that while there are advantages in specific circumstances from utilising the situated theory of agency in designing complex systems, because of their differing ontological commitments, such systems would be difficult to analyse and evaluate using ontologies currently used in information systems. we then provide evidence that such situational information systems can indeed exist, by giving a specific example (the kanban system), which has emerged from manufacturing practice. we conclude that information systems are likely to benefit from creating design approaches supporting the production of situational systems. keywords: information systems design, theories of agency, ontology, situated action introduction there has been much recent effort towards finding a theoretical basis for the information systems discipline. some of this work draws on ontology and focuses on the representational role of information systems with particular emphasis on design tools and methodologies. this work is based on the assumption that computerised information systems in part record and track states of a ‘real world’ system and of real world objects of interest to an information system. in this work, design methodologies and tools have been examined using two specific ontologies. this work is important in that it is leading to a deeper understanding of tools and methodologies. however, it is only part of the story. in this paper, we argue that a more fundamental understanding of information systems design methodologies will emerge from recognising that information systems are not merely representational, but are embedded in socio-technical systems that are designed to automate the achievement of certain goals. the socio-technical wholes, of which information systems are the representational part, can thus be usefully thought of as intentional or goal-directed agents. we therefore propose that information systems design can be studied at a deep level by analysing their design assumption in terms of theories about how intentional action is possible (theories of agency). this proposal cuts deeper than prior ontological analysis of systems since it examines roles other than representation for information systems. however, it also complements that approach since theories of agency contain ontological commitments at a deep level. in particular we will argue that there are two basic theories of agency the “deliberative theory” and the “situational theory”. only the deliberative theory has had any explicit influence on approaches to information system design. its inherent ontological commitments are similar to those of the objectivist/realist ontologies that have been mainly used in the systems evaluation work. by contrast, despite its influence within robotics and computer science, the situational approach has had almost no explicit influence on the design of management or information systems, although elements of the approach can be detected in certain non-computerised operational systems. the ontological commitments of this approach are radically different and present a challenge for the ontological analysis of these information systems. we begin by arguing our substantive proposal, that theories of agency should be viewed as a deep theory of information systems design. we then describe the deliberative theory of agency and show that it is this conception of agency that informs much systems design practice. we then put forward a radically different conception of agency that has been recently articulated in robotics, artificial intelligence and the cognitive sciences. we use an example from the artificial intelligence literature to illustrate this approach, in particular to emphasise the challenge it poses for traditional theories of representation. we give an example of a system from current supply chain management practice (the kanban replenishment system), which we believe shows traits of the situational agency theory. our conclusion is that existing ontological studies assume a deliberative theory of agency for information systems design. further, we conclude that the ontology-based approach is insufficient for a comprehensive theoretical understanding of information systems design and that theories of agency form a theoretical basis to which the ontological studies must relate. moreover, the analysis points to an opportunity to develop new methods for analysing task domains in such a way as to encourage the production of information systems based on the situational approach to agency, which may display the robustness and efficiency that has been demonstrated for this approach in other disciplines. the main contribution of the paper is the novel proposal that information systems should be studied in terms of their explicit or implicit commitments about the ajis special issue december 2002 41 nature of intentional agency and the working through of the consequences of this proposal for ontological evaluation of systems designs. prior work previous attempts to judge the soundness of design methodologies have focused on identifying a meta-theory that encompasses information systems design principles as a subset, and which thus provides an external reference theory against which the methodologies can be judged and refined. noting that these methodologies are concerned with the possible entities and states of affairs that can exist in a task domain, which is also the topic of a branch of philosophy known as ontology (honderich, 1995), several researchers have made use of ontologies from the philosophical literature for this purpose. specifically, bunge’s ontology (bunge, 1977; bunge, 1979) has been adapted and used to study information systems design and development tools ranging from data flow diagrams through to case tools (wand, 1989; wand and weber, 1993; wand et al., 1995; wand and weber, 1995; weber, 1997). this ontology is realist (assumes the existence of an external reality independent of human experience) and also, in lakoff's (1987, p159) sense, objectivist (assumes that the world consists of entities, properties and relations that can be put in one-to-one correspondence with symbolic settheoretical counterparts). chisholm’s ontology (chisholm, 1996), which is a less formalised but more expressive commonsense realist ontology, has been used to evaluate specific data modelling tools (milton et al., 2001; milton and kazmierczak, 2002). one difficulty with these approaches is that there are a number of philosophical ontologies available, with differing levels of formality and a wide range of philosophical commitments. the problem of how to select the appropriate one has not been resolved. the choice is thus largely a matter of taste and philosophical bias, which introduces a degree of arbitrariness in this line of research. recently, we (johnston and milton, 2001) proposed an alternative meta-theory of information systems design. our point of departure is the observation that information systems are never designed to represent what is in the world for its own sake; they are designed to be used by human actors to achieve certain goals. thus, an information system is just one component of a larger socio-technical system intended to allow one or more human actors to bring about certain desirable states of affairs and which is therefore a goal-directed system. this point of view is particularly compelling for operational systems where the information system is used to control or direct human action in repetitive, time-constrained and largely routine activity, in pursuit of persistent goals such as: “maximize return on investment”, “minimize late delivery”, “minimize stock-outs”. an appropriate meta-theory for information systems is thus the theory of intentional (goal-directed) behaviour in general, or theories of agency. it is important to note that we are viewing the combination of human actors and the information systems they use, as a single intentional agent denoted “the socio-technical system”. many kinds of complex systems ranging from robots, software agents, humans and animals, to organisations can be characterised as intentional agents (dennett, 1978), and this approach is particularly useful in designing artificial systems, because different theories of intentional agency suggest different decompositions of these complex systems into parts (brooks, 1986; agre, 1997). a theory of agency is a theory about: • how an agent can obtain data about (sense) the world in which it acts; • how an agent can represent the world it senses; • how an agent can manipulate this representation to select actions that will allow it to close in on its goal; and • how actions are effected. recently, a number of authors (brooks, 1986; agre and chapman, 1987; suchman, 1987; hendriks-jansen, 1996; johnston and brennan, 1996; agre, 1997; clancey, 1997) from diverse disciplines, have argued that there are essentially two radically different main views on the nature of intentional action, which we will call the “deliberative” and the “interactional” theories of agency. what they have in common, though, is that agent actions are presumed to be derived from a representation of the agent’s condition in the world of action. to represent the world of action requires commitment to an ontology – a theory about what is, and can be, in the world. the two main theories have radically different ontological commitments and thus differ on the type of representational scheme that is most efficient for intentional action. our argument is that the fundamental connection between ontology and system design is through the place of ontology in theories of agency and their role in system design. it is because of this connection that ontology is relevant to system design and the evaluation of system design tools. we will argue that existing approaches to computerised information system development are implicitly informed by the deliberative theory. on the other hand, we suspect that many noncomputerised operational systems make informal use of the situational approach to agency. it should be possible to design computerised operational information systems using the situational theory. as a long-term goal of our research, we propose to develop and test a methodology and set of tools to assist designers to do this. ajis special issue december 2002 42 doing by thinking: deliberative agency the deliberative theory of agency posits that an agent acts in a goal-directed way by building and maintaining an abstract model or representation of the world and applying deductive processes to this representation to determine what to do. this stance commits the theory to a certain position about the nature of the world that is both ‘realist’ and ‘objectivist’. the world consists of objects with properties. a particular object is of a given kind because it displays certain properties characteristic of that class of object, and differs from other objects of that class by virtue of its particular values for the properties. objects stand in certain relations to one another. the world is characterised by a set of possible states of the world, which are allowable configurations of objects (including the agent) under these relations. the representation of the world is of a particular kind, a “symbol/object” representation (agre, 1997), in which configurations of symbols, their attributes and relations, are isomorphic to objects, properties and relations within states of the world (figure 1). the agent uses sense data to define the current state within this world model and then applies deductive processes to find a sequence of symbolic “actions” that will transform the model current state to the model goal state. this sequence of symbolic actions is a “plan” that is handed to an effector system that implements it in the real world. sensorseffectors world world model figure 1. the agent and its relation to the world as envisioned in the deliberative theory of agency we can identify this deliberative theory of agency with the conventional approach to the design of the sociotechnical systems of which information systems are a part. sensing is achieved by recording data about events in the world (transactions). databases, or more precisely the data models used to design databases, provide the representational scheme. this representational scheme is conventionally of the symbol/object type: files represent object classes, records in files represent instances of objects and the rules of manipulation of records by programs enforce symbol manipulation rules, which mimic the lawful relation among objects in the world. in the strongest version, programs also deduce goal-attaining actions and human actors are only required to define the goal state, execute the actions (according to computer-generated schedules) and record transactions. a good example is manufacturing resource planning (mrp ii) (wight, 1981). mrp ii is a computer-based approach to manufacturing operations control. mrp ii embodies the deliberative model in a very literal way (johnston, 1995; johnston and brennan, 1996). a computerised master production schedule represents the company-wide production goal. this is processed against a product structure database that describes how raw materials and bought components can be converted into saleable items. the current availability of these items is checked against an inventory database and, by purely deductive means, schedules for buying and assembling parts to meet the master schedule are produced and handed to operators for execution. such a strong action selection role for computer systems is now not fashionable. more typically, the information system merely supports action selection (decision making) by human actors. nevertheless, the overall architecture of the combined socio-technical systems (human actors + information systems) is assumed to function according to the deliberative model. in the past 30 years a number of methodologies have been developed to assist information systems designers to produce and document such systems designs, especially the data models that underpin these systems. they can be referred to collectively as information engineering methodologies (iem), with structured systems analysis and design methodology (ssadm, the british government standard) being a typical example. these design methodologies share the ontological assumptions of the deliberative theory, namely, that systems should represent the environment in which the system acts in terms of external, independent and objective entities, properties and relations. given this focus on symbol/object representation, use of these methodologies encourages designs for the socio-technical systems, of which the information systems form the representational scheme, which mimic the deliberative approach to agency. therefore, evaluation of these approaches against ajis special issue december 2002 43 philosophical ontologies such as bunge’s that also share the same objectivist assumption is appropriate and may lead to refinement of these tools and methodologies for the task of building systems along these lines. routine action: situational approaches to agency in the past decade or so researchers in diverse disciplines have agued against the efficacy of the deliberative theory of agency (brooks, 1986; suchman, 1987; brooks, 1991a; johnston, 1995; hendriks-jansen, 1996; johnston and brennan, 1996; clancey, 1997; agre, 1997, p392), on the grounds that even if the world can be represented in such an abstract symbolic way – a debatable ontological assumption (dreyfus, 1992) – it does not provide a manageable engineering solution to the operation of artificial, autonomous agents, particularly in the face of time-constrained repetitive action. for instance, in a series of proof-of-concept robot designs, rodney brooks and co-workers (brooks, 1986; brooks, 1991b; mataric, 1992) have shown that goal-achieving behaviour is possible in robots that do not make use of deliberation or centralised symbolic representations of their environments of action. their robots use more or less reactive behaviours to establish complex interactions between the agent and features of its environment. they have demonstrated that the performance of such agents in complex environments is more robust than that of deliberative agents and degrades more gracefully in the face of the unexpected. in a similar vein, agre and co-authors (agre and chapman, 1987; chapman, 1991; agre and horswill, 1992) have argued for a theory of routine action, based on the interaction of relatively simple action selection rules with the structure of environments and have demonstrated the possibility of robust goaldirected action in this way using computer-simulated agents. emerging from this technical work, is an alternative theory of agency with substantially different representational and ontological commitments from the deliberative theory. the key insight is the essential infeasibility of the continual translation, required by the deliberative theory, from the world of action to the world model and back again to action (johnston and brennan, 1996; agre, 1997), as a principle for timeconstrained action in complex environments (this problem is referred to as the “symbol binding problem” in artificial intelligence literature). this translation is required because the world model is an “aerial” or “transcendental” objective view of the world from outside the agent and independent of its goals. any real agent has available only an incomplete, skewed “ground” view from its sensors with which to construct and update its world model. but according to the deliberative theory, it is only by building this aerial world model that its own goal can be expressed and lead to action selection. in addition, the strategy of instantiating the world model independently of the agent’s goal means that it is difficult to bound the size of the world model, which typically leads to a computational complexity explosion during action planning (chapman, 1988). the solution to these dilemmas proposed in “situational” theories of agency is to provide an agent with more or less reactive responses based on sense data obtainable directly from the agent’s ground view of the world, and to introduce the agent's intentions and point of view early on in the theory at the level of the ontology and representation schemes. according to this theory, agents respond reactively to “situations” without deliberation. situations, in contrast to “worlds states”, are accounts of the world from the agent’s point of view. they are parsimonious descriptions because they need only include features of the world that are relevant to the agent’s purposes (agre and chapman, 1987). these features consist of the relations of things to the agent given its goals. actions are selected from a repertoire of more or less reactive rule-like responses to situations. such a simple approach to action selection will only work (be goal attaining) if the action environment exhibits structure (“affordances”) that makes complex planning unnecessary (agre and chapman, 1987), and analysis and exploitation of environmental structure is an important part of designing situated agents (agre and horswill, 1992; hammond et al., 1995; horswill, 1995; agre and horswill, 1997). we can see the differing roles of representation in the two theories. with the deliberative approach the agent must acquire and make use of a description of the entire universe of discourse from a transcendental point of view. objects in reality are bound to symbols and states are instantiated. decisions about action are made taking into account a complete description of reality. once actions are taken, sensing of the state of reality must continue in order to continue pursuing the goal under consideration. in contrast, because situations are agentcentred and intention-laden, representation of situations with a symbol/object isomorphism is neither possible nor necessary. however, aspects of situations do need to be marked in some way in order to fire situation-action responses. agre and chapman argue that the appropriate representational scheme should be “indexical” and “functional” in nature (agre and chapman, 1987). indexical representations describe the disposition of things relative to the agent and functional representations select things according to their relevance for the purposes of the agent. agre and chapman (1987) argue that the ability to rebind indexical/functional representations to situation features that are relevant to the agent’s goals on the fly, eliminates the computational complexity explosion inherent in using all-encompassing aerial world models as a basis of action. the reliance of the situational theory on indexical/functional rather than symbol/object representation, indicates it is based on entirely different ontological assumptions and thus systems designed in this way would be difficult to analyse using objectivist ontologies such as bunge’s. ajis special issue december 2002 44 example of a situated agent: pengi in order to make this idea of an agent based on the situational theory more concrete, we now describe the famous “pengi” simulation, devised by agre and chapman to illustrate their ideas about the role of representation and ontology in situated and routine agency. pengi is a computer simulation of an agent that plays an arcade game called “pengo” (which is also simulated). in pengo, a penguin moves in response to commands from a button and a joystick in a two dimensional video screen “world” of ice-blocks, which effectively form a maze. the world is also inhabited by bees whose behaviour is almost entirely random. if a bee gets too close to the penguin, the penguin dies and the game ends with a loss. both the penguin and the bees can kick the ice-cubes and they slide in the kicked direction until they reach an obstacle. if the obstacle struck is a bee it dies and if all bees die the game ends with a win. if the obstacle is the penguin it dies, and the game ends with a loss. the environment is thus fairly complex, dynamic, and has a high degree of unpredictability. pengi’s behaviour, in controlling the actions of the penguin, is clearly goal directed. the game environment is illustrated in figure 2. figure 2. a pengi game in progress (from (agre, 1997)) pengi does not use any planning: it is merely equipped with a set of appropriate responses to each type of situation, which have been put in by hand by the designers on the basis of an analysis of the dynamics of the game. in fact pengi’s action selection is done using a network of binary logic gates that have no internal state. these generate a series of simulated joystick (left, right, up, down) and button (press) movements that are passed to the game simulator. thus, pengi’s executor system is essentially reactive and rule-based. however, the visual (sensing) system is quite sophisticated. it is designed to detect characteristic features of the situation on the game board that are relevant to pengi’s goals of avoiding defeat and winning. while the vision system has access to the data structure representing the positions of the various objects on the game board, this symbolic representation is not made available to the simulated agent. instead, this data structure is used like a retinal image and various features in it are extracted by “visual routines” which are used to pick out objects of relevance, such as bees threatening the penguin, ice-cubes that can be kicked, and other spatial relationships. the visual system can also place a small number of markers on objects to keep track of them as a simulation of visual attention. most of the visual operators ask binary questions about the relationships between markers, the answers to which are used as input to the action selection system. according to agre, “pengi plays a pretty decent game of pengo. in its present state it is a little better than i am, which is to say it wins from time to time and puts up a good fight.” (agre, 1997, p265). pengi can clearly be viewed (and was explicitly designed) as an intentional agent that acts in its simulated world to achieve the persistent goal of survival of the penguin and killing all bees. however, pengi differs from our characterisation of a deliberative agent in the following ways: 1. pengi does not make use of an objective representation of its world of action. that is to say its does not build or update a model of the pengo world in terms of symbols that stand in one-to-one correspondence to the ice-blocks, bees and the penguin and their positions on the board. nor does it make use of an explicit representation of states of this world in its action selection. rather, to the extent that pengi has access to its world of action through symbolic means, the symbols stand for aspects of situations of concern to the pengo player. for instance, a marker might be interpreted as representing “the-bee-that-ischasing-me” or the “the-ice-block-between-me-and-the-bee”. unlike “bee-29” and “ice-block-9”, such ajis special issue december 2002 45 references are not objective for two reasons. firstly, they are indexical since they point to objects relative to the penguin's position. they are continually rebound to new individuals as the game progresses. they are also essentially intention-laden, or functional as agre and chapman put it, since they label features that are relevant to the penguin's goal. features that are too remote from the penguin to be relevant to its survival are not represented at all and thus not available to the player simulation. 2. pengi does not make use of the notion of “state of the world” in the sense of a configuration of objects in its world. in fact it does not “know” the state of the game at any time. instead it bases its action selection upon detection of characteristic features of the world viewed from its myopic agent-centred point of view. 3. pengi is not deliberative in the usual sense of the word, which implies searching the space for future possible states of the world. pengi reacts in a rule-based way to aspects of situations it encounters in realtime. 4. pengi's goal-directed behaviour is an interaction between its reactive rule set and recurring structure in its world of action. the structure of its environment is a co-agent, which cooperates with pengi in goal attainment/non-attainment. this active participation of environmental structure in intentional action is a hallmark of the situational approach to agency. in this simulation it is easy to see that the overall behaviour is dually determined in this way since one set of programs is determining the penguin’s action while another is a simulation of the way the game (its world) behaves in response. thus in a very direct sense the penguin’s moves alone are insufficient to account for its goal achievement. they only work because of the non-arbitrariness of the game world (the rules of the game and the physics of the game world, such as two ice blocks cannot occupy the same square). these points analyse the pengi simulation in relation to choices of ontology, representation, action selection and the role of environmental structure for goal attainment. we would expect to find similar traits in any agent system informed by the situational theory. combining this analysis and our earlier discussion of deliberative agent designs, table 1 sets out systems traits that might allow one to distinguish the theory of agency which informs the design of any particular intentional system. trait deliberative situational type of representation symbol/object indexical/functional what represented objects (entities and properties) represented as though external to and independent of the system situations and aspects of situations relevant to goal-attainment persistence of representation persistent account of the state of the world from a dispassionate view point transient representations intended only for initiating immediate action underlying ontology objectivist non-objectivist action selection involving deduction on representations of the state of the world and resulting in plans and schedules for future execution immediate (reactive) rule-like responses to aspects of situations significance placed upon environmental structure structuring of the social and physical environment of work marginalised structuring of social and physical environment of work manifest as essential to system functioning table 1. traits that might allow the underlying theory of agency to be discerned in any given intentional system design pengi also illustrates our main point about ontology. in designing pengi the designers could have dissected pengi's world using a realist objectivist ontology that would talk of objects, relations, possible states and state transitions but such an analysis is not pertinent to the design of the agent, since pengi does not know its world in these terms. to pengi the world consists of situations, detectable aspects of which determine its action. pengi never needs to make the difficult translation from this ground view of the world to the objective aerial view of individuated objects. such an objectivist ontology is not part of the theory of agency that pengi implements. kanban: a situational inventory replenishment system? can we find examples of socio-technical systems that can be viewed as intentional agents operating on the situational theory of agency? we suspect there are many examples and they should be most often found in environments where the situational theory is known to be most appropriate, namely, routine interaction in time critical environments. given our suggestion that we have a predilection to designing systems using the deliberative approach, we also suspect they will have evolved rather than been explicitly designed. as a result they may often be unofficial systems and may well be manual information systems. nevertheless, we would ajis special issue december 2002 46 argue that these systems contain important lessons for formal design of computerised systems that support routine action. we will present one example, which is a manual operational system that is widely used in the automotive industry for replenishing parts for production, namely the japanese kanban system (schonberger, 1987; womack et al., 1990). kanban is the japanese word for “card” and in this system, stock levels and replenishments are controlled by the movement of cards associated with containers of parts. each part is stored in a fixed size container and in each container is placed a kanban card that has printed on it minimal information about the item, usually product-id, primary supplier and location (see figure 3). there are a fixed number of kanbans in existence for a given item and in normal operation they are neither created nor destroyed. figure 3. a typical kanban card goods are taken from the container (which is usually stored right at the work-centre) until the container is emptied. the kanban in that container becomes free and it then becomes a signal that the items need replenishment. these free kanbans are taken to a kanban-board near the goods receiving area and are placed on hooks that are sequenced by supplier-id. the board traditionally has the kanban system operation rules (kanban rules) clearly displayed. when a given supplier arrives with shipments of items to deliver, they check the kanban board and take the kanbans on their hook back to the supplier site to authorise replenishment of these items next time round. when the items are subsequently supplied the kanbans are returned in the appropriate containers with the items and they are taken to the work-centre where they are used. the process repeats. unlike pengi, which is a computational intentional agent, if we are to discuss the kanban system in terms of agency we must recognise that the goal-achieving agent is socio-technical. it consists of the kanban cards, the kanban board, the kanban rules and the operators (those using the materials and taking the kanbans to the kanban-board according to the rules, and those taking the kanbans as an authorisation to supply and returning them with the goods). together these elements implement an intentional system that achieves the goal of keeping the parts available for production. the kanbans are the representational part of this system. we can make the following observations about this system, which show the connection between its design and the situational theory of agency: 1. although the kanbans form the representational scheme in the system, kanbans are not associated with objects (the parts or the suppliers) in any simple way. when a kanban is in the container it stands in oneto-one correspondence with the objects in that container. however, since the system is a visual manual system these kanbans provide no information to the operators. only free kanbans have a symbolic significance. when a kanban is freed by the exhaustion of these items, it represents a shortage of a particular quantity of items and when it is on the kanban-board, it represents an authorisation to replenish a certain quantity of these items. the physicality of the kanban as a symbol is crucial to the functioning of the system. firstly, the constancy of the number of kanbans, ensured by their physical robustness, effectively controls the item stock level, according the desired system goal. secondly, the multiple representational roles of the kanban are enabled by its changing association with physical places and things. 2. the socio-technical system does not function by acquiring and manipulating an account of the state of the objects in the domain of action. the only information that is apparent to the operators is that conveyed by the free kanbans. the state of most parts is not available through this visual system, nor is it of any interest to the operators. the free kanbans are best thought of as marking situations of concern to the operators, namely, shortages of certain parts. this contrasts to conventional computerised systems where the state of stock of any item is available through on-line enquiries made possible by elaborate transaction processing. just like the markers in pengi’s visual system, the kanbans are finite in number and constantly rebound during system operation. they are indexical/functional because they mark only aspects of situations relevant to the socio-technical agent with its specific stock replenishment goal, and do not provide an all-purpose account of parts appropriate to a “disinterested” observer such as an accountant. 3. kanban is a purely reactive and rule-based system. actions are triggered by situations of shortage marked by free kanbans. the operators cannot and do not need to plan in the sense of searching through possible future states of the system. materials requirements planning mentioned above is an example of ajis special issue december 2002 47 a conventional transaction-based computerised inventory system that does use extensive state-space search. 4. kanban will only work in a structured environment. it is well known that kanban will not tolerate fluctuations in daily item demand greater than about 10% (schonberger 1987). this environmental constraint is achieved in japanese practice by smoothing the release of assembly work to the shop floor in such a way as to have a constant mix of variants each day (so-called hijunka) (monden, 1983). this practice protects the kanban system from external demand fluctuations, and contrasts markedly with fordist practice of batching production to achieve economy-of-scale. because of the manual, visual and socio-technical nature of the system many other simplifications of the physical and social work environment (including cellular machine layouts, teams, quick tooling changeovers, etc.) are required for its smooth running (monden 1983). the kanban rules alone are insufficient to ensure the goal state, but interact with these affordances of the environment to achieve the goal as an emergent outcome. this analysis shows that the kanban part replenishment system has marked similarities in its functioning to pengi. the characteristics of the kanban system enumerated above indicate particular commitments in relation to ontology, representation, action selection and the role of environmental structure for goal attainment, which are quite characteristic of situational agency in general. we hypothesise that many other manual card or token systems, particularly those that control routine action, function by implementing the situational theory of agency. many have evolved from practice and have survived because of the inherent robustness and cognitive efficiency that has been documented for the situational approach in other areas concerned with intentional action. however, because of their origins in praxis they are often marginalized as “practical wisdom” rather that manifestations of a genuine theory of “how to get things done” efficiently with minimum fuss. the kanban system is perhaps an exception because it is now widely implemented in western and japanese car companies (often replacing deliberative approaches) (muffatto, 1999; spears and bowen, 1999) because the phenomenal success of japanese manufacturing since the 1973 oil crisis can be largely attributed to its use of just-in-time, of which kanban is a part (womack et al., 1990). it is interesting to speculate on why the situational approach has had so little impact on information system design. it is probably partly due to the deep influence in our intellectual tradition of the cartesian duality of body and mind, which encourages system approaches that begin with an objective representation of the environment of action. but robotics researchers operate in the same tradition. in a sense it is easier to detect the limitation of the deliberative approach when designing a totally autonomous system. bad performance of sociotechnical systems can always be repaired (if not totally worked around) by the tacit knowledge of their human components. a second major reason is that information system design is so intimately linked to database design. existing database designs almost force a symbol/object (even set-theoretical) view of the world on designers. notice that the kanban system does not implement database principles at all. although the kanban system evolved out of practice, it might equally be reinvented by systematically applying situational agency concepts to the inventory replenishment domain. there is no reason to suppose that computerised systems cannot also be devised, possibly using information technologies other than databases, in this way, that would provide similarly novel designs for control of operations in other domains of routine, time-constrained action, such as retail distribution, emergency dispatch and hospital wards, to name but a few. significance and conclusions if we are right in our contention that many routine operational systems, such as the kanban system, are best understood as implementing the situational theory of agency, it will generally not be useful to analyse these systems using objectivist ontologies such as bunge’s. we would need an ontology that included at least the nonobjective (that is agent-centred and intention-laden) notions of “situations” and “aspects of situations”, such as the ontology that has at least been partly worked out by agre (1997). furthermore, it is unlikely that systems design methodologies such as those from the information engineering paradigm would be useful in designing such systems, since they would encourage the designer to decompose the world of action for these systems in a way that is totally inappropriate to the theory of agency they implement. that is to say, most systems designers trained with these tools would come up with system designs similar to familiar transaction-based database systems, which are known to be difficult to make routine in operational environments (johnston, 1995). this suggests that by using objectivist ontologies to benchmark systems design methodologies and tools we may be perpetuating, in our information systems practice, a particular style of systems design implicitly informed by the deliberative approach to agency, and that the general conformance of these methods to the ontologies may be a self-fulfilling prophesy. on the other hand, the robustness and efficiency of the situational approach, which has been exploited in other disciplines concerned with automating goal-directed behaviour, could have potential to solve many of the performance and acceptance problems that have dogged attempts to use information systems to automate operational activity. to test this hypothesis, we need to study robust systems that have grown out of practice in operational domains to see if their functioning can be attributed to the situational theory of agency. at the same time we need to develop system design methods which would allow a designer to analyse the design problem in ways that might encourage the discovery of solutions (computerised as well as ajis special issue december 2002 48 manual) exploiting the situational theory of agency. such an approach to designing a socio-technical system using the situational theory of agency would involve: 1. an analysis and specification of situations (relationship between the system and its environment) important to goal attainment. this contrasts with the emphasis of the deliberative approach upon objects and properties. 2. defining a scheme for representing key aspects of situations that trigger action. these schemes will be indexical/functional in nature. 3. defining a rule-set for actions to be taken in response to the occurrence of situations of importance. this emphasis upon reactivity contrasts with the emphasis on planning and scheduling in deliberative theories. 4. analysing and defining structuring features of the system environment that will enable goal-attainment through interaction with such reactive responses. our contention is that employing such an approach will produce systems designs that are paradigmatically different from those produced using conventional methods. as with conventional systems, the information system will form the representational part of the socio-technical whole, but now using different representational principles. although the kanban system evolved out of practice, it could equally be reinvented by applying the above method to the inventory replenishment domain. there is no reason to suppose that computerised systems cannot also be devised in this way that would provide similarly novel designs for control of operations in other domains of routine time-constrained action, such as retail distribution, emergency dispatch and hospital wards, to name only a few. references agre, p. e. (1997) computation and human experience, new york, cambridge university press agre, p. e. & chapman, d. (1987) “pengi: an implementation of a theory of activity”, proceedings of aaai87. the sixth national conference on artificial intelligence, menlo park, ca., morgan kaufman, los altos, ca., pp 268-272 agre, p. e. & horswill, i. (1992) “cultural support for improvisation”, proceedings of aaai-92. the tenth national conference on artificial intelligence, menlo park, ca., aaai press, pp 363-368 agre, p. e. & horswill, i. (1997) “lifeworld analysis”, journal of artificial intelligence research 6, pp 111145 brooks, r a. (1986) “a robust layered control system for a mobile robot”, ieee journal of robotics and automation 2(1), pp 14-23 brooks, r. a. (1991a) “intelligence without representation”, artificial intelligence 47(1-3), pp 139-159 brooks, r. a. (1991b) “new approaches to robotics”, science 253(9), pp 1227-1232 bunge, m. (1977) treatise on basic philosophy: vol. 3: ontology i: the furniture of the world, boston, reidel bunge, m. (1979) treatise on basic philosophy: vol. 4: ontology ii: a world of systems boston, reidel chapman, d. (1988) “planning for conjunctive goals”, artificial intelligence 32(3), pp 333-377 chapman, d. (1991) vision, instruction, and action, cambridge, ma., the mit press chisholm, r. m. (1996) a realistic theory of categories-an essay on ontology, cambridge, cambridge university press clancey, w. j. (1997) situated cognition: on human knowledge and computer representations, cambridge, cambridge university press dennett, d. c. (1978) brainstorms: philosophical essays on mind and psychology, brighton, harvester dreyfus, h. l. (1992) what computers still can't do, cambridge, massachusetts, the mit press hammond, k. j., converse, t. m. & grass, j. w. (1995) “the stabilization of environments”, artificial intelligence 72(1), pp 305-328 hendriks-jansen, h. (1996) catching ourselves in the act: situated activity, interactive emergence, evolution, and human thought, cambridge, ma, the mit press honderich, t. (ed.) (1995) the oxford companion to philosophy, oxford, oxford university press horswill, i. (1995) “analysis of adaption and environment”, artificial intelligence 73(1-2), pp 1-30. johnston, r. b. (1995) “making manufacturing practices tacit: a case study of computer aided production management and lean production”, journal of the operational research society 46(10), pp 1174 1183 johnston, r. b., & milton, s. k. (2001) “the significance of intentionality for the ontological evaluation of information systems”, proceeding of the americas conference on information systems, boston johnston, r. b. & brennan, m. (1996) “planning or organising: the significance of theories of activity for the management of operations.”, omega, international journal of management science 24(4), pp 367384 lakoff, g. (1987) women, fire and dangerous things: what categories reveal about the mind, chicago, the university of chicago press mataric, m. j. (1992) “integration of representation into goal-driven behavior-based robots”, ieee journal of robotics and automation 8(3), pp 304-312 ajis special issue december 2002 49 milton, s. k. & kazmierczak, e. (2002) “on the study of data modelling languages using chisholm's ontology”, information modelling and knowledge bases xiii. kangassalo, h. et al (eds.). amsterdam, ios press: pp 19-35 milton, s. k., kazmierczak, e. & keen, c. d. (2001) “data modelling languages: an ontological study”, proceedings of the 9th european conference on information systems, bled, slovenia: pp 304-315 monden, y. (1983) toyota production system, norcross, institute of industrial engineers muffatto, m. (1999) “evolution of production paradigms: the toyota and volvo cases”, british journal of management 10(1), pp 15-25 schonberger, r. j. (1987) “the kanban system”, just-in-time manufacture. voss, c.a. (ed.) london, ifs ltd. uk: pp 59-71 spears, s. & bowen, h. k. (1999) “decoding the dna of the toyota production system”, harvard business review, pp 1-11 suchman, l. a. (1987) plans and situated actions, cambridge, cambridge university press wand, y. (1989) “a proposal for a formal model of objects”, object-oriented concepts, applications, and databases, kim, w. & lochovsky, f. (eds.) reading, addison-wesley, pp 537-559 wand, y., monarchi, d. e., parsons, j. & woo, c. (1995) “theoretical foundations for conceptual modelling in information systems development”, decision support systems 15(4), pp 285-304 wand, y. & weber, r. (1993) “on the ontological expressiveness of information systems analysis and design grammars”, journal of information systems, pp 217-237 wand, y. & weber, r. (1995) “on the deep structure of information systems”, information systems journal 5(3), pp 203-223 weber, r. (1997) ontological foundations of information system, blackburn australia, buscombe vicprint wight, o. w. (1981) manufacturing resource planning: mrp ii, essex junction, vt., oliver wight publications, inc womack, j. p., jones, d. t. & roos, d. (1990). the machine that changed the world, new york, rawson associates cyberspace versus citizenship: it and emerging non space communities s.e. little department of business systems, university of wollongong this paper was developed from one presented at the cities cultures conference, brisbane, december 1992 abstract in 1964 melvin webber challenged the notions of community and centrality used in urban studies by demonstrating that "community without propinquity" was emerging within certain social networks. he argued that individuals were enmeshed in an overlapping range of groups, and that increasingly these social networks were not limited by physical or geographical location. his definition of community acknowledges a differentiated range of "non-place" cultures. it reflects a change to a process, rather than product oriented view of urban form, triggered in part by the influence of general systems theory. webber influenced and was influenced by the emergence of an orientation towards non-physical aspects of community, and a participatory approach to design which emerged strongly during the seventies. reexamination of webber's work in the light of current information technology offers some insight into the nature of the globalisation of the world economy, and consequent impacts on nationality and sovereignty. the technologies that will be commonplace by the end of the century can both empower and disempower and it will be necessary to reconsider our current notions of both citizenship and of access to and control of such crucial resources. opportunities offered by it for marginalised or peripheral groups, whether at the level of nation, region or local community, will challenge existing definitions of centre and periphery. the moral panics surrounding such activities as "hacking" and its supporting "cyberpunk" sub culture demonstrate a growing awareness of the importance of emerging non-space communities. webber's formulation of non-place realms webber's 1964 paper addresses "metropolitan planning", in an attempt to move away from the agenda of city planning with its focus on "small scale physical aspects of urbanization" (webber; 1964, p.80). he suggests his view is "oriented to metropolitan processes (a verb view) from which it seeks to identify the matching spatial form (a noun view), and hence it seeks to pose a dynamic portrait of metropolitan form in action (a gerund view). (webber; 1964, p.80, emphasis in original) webber's argues that the physical preoccupation of urban theory creates inadequacies. a processoriented re-evaluation of the components of urbanism that planners and architects claim to support and promote is therefore required. urban communities are presented as spatially structured processes, with the physical fusing of us atlantic seaboard settlement providing a striking example of the outcome of such processes. in order to move from the physical bias of established planning conceptions, webber proposes the "city as communications system" (webber; 1964, p.84). planners "share a conviction that the physical and locational variables are key determinants of social and economic behaviour and of social welfare" (webber; 1964, p.85). according to webber, the unique commodity offered by the city is accessibility and he cites meir's (1959) development of a system of social accounts as the basis of an index of cultural wealth reflecting the volume and variety of information flowing through public communication channels (webber; 1964, p.87, note 9). such an approach switches the emphasis of urbanity from physical built form to the quality of interaction in cultural life through the 38 ajis exchange of information. webber argues that this definition implies that suburban and exurban dwellers enjoy a measure of urbanity not previously acknowledged. in established practice plans are formulated in spatial terms, with a view of social and spatial interaction derived from an examination of population distribution and density. however, webber argues that such an approach is poorly equipped to account for "dynamic, locational patterns of human communication that occur through space but transcend any given place" (webber; 1964, p.90). increasing sophistication in measures and classification of activities still leaves them short of transcending an essentially static picture, even if such "snapshots" may be strung together to reveal changes over time. webber argues that planning must deal effectively with three components of metropolitan social structure: 1: spatial flows of money, people and goods 2: location of the physical channels and adapted spaces that physically house activities 3: locations of activity places (webber; p.96) he suggests that first component can be derived from the effectiveness with which communications systems may substitute messages for physical movement of persons or goods (p.97). he cites the significance of railroad location in north american development as an indication of the importance of the second component in determining urban form. the subsequent significance of street car and suburban lines, and freeways suggests that developments in telecommunication capability and capacity had been equally instrumental in freeing access and range of locational choice. face-to-face communication is still seen as a special need, and traditional central locations are still sought for many forms of business. the permanence of building stock, as well as the character of interaction is significant here, but the creation of this building stock is still regarded as an end in itself by architects and urban planners. the third component is approached through the traditional land-use view of development. webber provides a descriptive schema for spatial structure using these three components, and points out that "(p)atterns of functional interdependence will become increasingly complex at the same time that major developments in transportation and communications systems will be opening up unprecedented possibilities for whole new spatial patterns." (p. 107). earlier notions of "place" in planning are less than helpful: the functional processes which must be acknowledged are not "place-like" or "regional-like" at all. webber therefore formulates a "nonplace community" in terms of interest-communities. accessibility, rather than the propinquity aspect of "place" is the necessary condition for this form of community (p. 109). extensive webs of specialised professionals can be regarded as communities without propinquity. while specialised professionals are acknowledged to be at one end of the spectrum of residents in a metropolitan area, many other similar association patterns may be detected in non-professional communities. the pure or traditional community is assumed to be relatively heterogeneous, with a unity arising from the pursuit of overlapping interests at a common place. webber argues that such a "place community" is in fact a special case of a larger genus. with developments in technology and education allowing wider participation in non-place ' groups a hierarchical continuum from highly specialised communities spanning the entire world via less specialised intra-national networks, to metropolitan and neighbourhood networks may be envisaged. individuals can expect to play roles at a number of these levels at any one time. at each level, the appropriate spatial field is shared by a number of interest communities, and webber calls such levels of interdependence and interaction "urban realms" (p. 114). these are distinguished from the established notion of an urban region by its lack of specific spatial location. webber argues that if an analysis of the distribution of each individual's time between realms were possible, it would reveal that rich and diverse human communication was present in conditions of low density and low concentration. he suggests that, in his terms, the urbanity of los angeles may not be very different from that of new york. webber argues that certain approaches to the classification of urban centres were more amenable to the consideration of the range of interactions which he identifies, but that any reconsideration of definitions of centrality in the terms outlined by him would call into question the traditional notions of centre and hinterland, citing locational decisions of emerging high technology companies in california at that time. september 1993 39 webber discusses the empirical difficulties of estimating proportions of working hours in an economy by roles and realms, however, any number of the range of qualitative techniques may be appropriate to the task of evaluating the range and intensity of participation of individuals. he ends by suggesting that emerging institutional changes and technological developments coupled with ever increasing mobility and specialisation are likely to involve urban dwellers in increasingly wide area communications, consequently both behavioural models of individual locational decisions, and descriptive models of overall spatial structures would benefit from an orientation to communication patterns. reading webber in the nineties clearly much has changed in the thirty years since webber's paper appeared. rosenberg (1982) illustrates the problem of being even partially right in technological forecasting by quoting a late nineteenth century prediction that ultimately every american city would be provided with a telephone. webber (1968) cites significant high speed surface and air transport developments, although the sst and not the jumbo is mentioned and mass air travel is not discussed. however, improved communications, including public real-time access to computerised databases are identified as potentially critical developments. toffler's (1970) pro-technology assertions were subsequently given fictional form by science fiction writer john brunner. however the predicted world of "shockwave rider" (1975) in which it was possible to exist in the interstices of a massive computerised domestic credit and surveillance systems had been exceeded by clifford stoll's factual account from the end of the eighties (stoll; 1989). billed as a new kind of detective story, the latter describes the global pursuit of miscreants across several international computer networks, while brunner's fictional twenty-first century network was confined to the u.s.a. information and communication technologies are by no means the first spatially significant technologies to impact on the nature of national sovereignty. headrick (1981) demonstrates the key role of technologies ranging from transportation to medical prophylaxis in this extension of the influence and sovereignty of the european powers and the globalisation of that model of the state. continued development in technology and economic scale has led both to the intemationalisation of economic activity and to the emergence of environmental consequences beyond the capacity of individual states to manage. the emergence of transnational corporations (tncs) and the intemationalisation of both financial and labour markets have created a rapidly evolving world system currently characterised by progressive integration at a world scale. camilleri & falk (1992) argue that power and authority have become diffused with national states participating in a variety of multilateral arrangements covering not just trade, production and finance but also increasingly interrelated environmental and security issues. the key issues which have emerged since webber wrote can be categorised as those of the development of the big technologies driving the processes of globalisation and those of the small scale implementations of those technologies facilitating local and regional initiatives. the big technologies include the complex wide area networks which support global business exchanges and global broadcasting: synchronous satellites, fibre optic networks, massively distributed computing facilities, such as internet. here an institutional perspective is needed to understand the driving concerns which are shaping the emerging "new world order". the small implementations are those points at which entry cost is low enough for individuals and small groups, in some cases because they piggy-back from existing infrastructures. the yuppie symbols of the fax and mobile phone, for example are sufficiently close to existing technical experience for users such as self-employed trades people to successfully appropriate. the size of the australian economy has been one of several factors influencing the rapid and complete diffusion of earlier forms of information technology. within this economy the significance for employment of small and very small business is becoming more widely recognised. some of the most radical changes to work practice may be found in the smallest, one and two person business organisations. joining the non-space community castells (1990) has produced one of the most influential analyses of the locational implications of information technology. he argues that a range of contingencies is required to create an appropriate milieu which will allow a region to participate fully in the range of infomrational activities. however, if local and regional initiatives are to be possible, then the notion of an intermediate or 40 ajis appropriate entry point to the technology becomes central.. engagement in a non-place community does not depend on the big technologies, rather on appropriate use of a range of technical opportunities. the question of what can be achieved by a relatively small player in terms of influence over the emerging new "techno-economic paradigm" (perez; 1984) is crucial to a highly urbanised country the size of australia. in the social use of this technology we must also ask what leverage we can achieve in the influence of outcomes. the perspective that is appropriate here is that of the seventies "intermediate technology " movement which developed in the decade following webber's paper, in conjunction with concepts of participative design and community action. suchman (1986) has looked at the collective and collaborative use of computer systems by groups of users, breaking with a human-computer interaction research tradition which has emphasised useability of systems in terms of the performance of individuals. the concept of computersupported collaborative work (cscw) reflects the move from an individual to a collective view of users in a socially constructed workspace which includes computer-based information systems. such technically created non-spaces for both work and social activity can be achieved with relatively unsophisticated equipment. the evaluation of the use and impact of an experimental computer bulletin board on people with disabilities (cerebral palsy) illustrates one paradigm for a diverse range of minority users and their relationship with information technology and the opportunities it affords. a full account of the study in which clients and therapists in an australian spastics centre were linked with computing students for a period of three months is available in earls (1990). one striking feature of the findings was the contrast between short term results which reveal no change in attitude over a three month period and the significant changes revealed by a repeated application of the measurement of attitude towards disabled persons test (yuker, block & campbell; 1960) after two years. initial results revealed a slight lowering of self image by the board users at the end of the experimental period. after two years, however, case-studies revealed individuals who had become politically active, gained employment (over 50% of the experimental group), taken up further study and begun to live independently. these clear benefits also evoked positive changes in their peer group, structural changes in the organisation and a response from the federal government. earls' findings emphasises the importance of longitudinal studies. vitalari (1985) argues the need for such an approach in the broader investigation of computing environments and sproull and kiesler (1991) contrast the short and long run implications of computer-based communication within groups. for the mainstream too, benefits are both long term and qualitative in nature. arguments are currently being advanced that quantitative evaluation often fails to reveal any measurable increases in organisational performance following investment in information technology. despite the continually falling costs and growing flexibility of computer technology there is considerable debate over the realisation of its potential benefits by organisations (eg. hartamis & lin; 1992, kling; 1993). much of the argument is around the way in which the core concerns of computer science should be broadened to encompass the useability and usefulness of its products. the skills required to assess and monitor the practices and requirements of end-users are now recognised as very different from the technical skills required for the development of a technical infrastructure. strategies favourable to minority users within a local community may be appropriate to national players in the global community. for example, the impact of the relatively simple bulletin board technology highlights the need for any minority group to utilise features of mainstream technical developments as far as possible in order to minimise the use of relatively expensive small volume specialised hardware and software. however, such windows of opportunity may be inadvertently closed by the momentum of mainstream technical development the features which were of advantage to this particular group, such as the "narrowness" of the communication bandwidth which produced a leveling and bland alphanumeric interface, or the asynchronous nature of communication are currently being eliminated in a technology driven push for greater sophistication. considerable sophistication in the evaluation and use of elements of mainstream technology is needed by any relatively small group wishing to access the potential benefits of a mass solution. similarly policies of smaller national players must be based on realistic assessments of opportunities and relative advantages. september 1993 41 non space, cyberspace and cyberpunk/new spaces, new institutions, new cultures webber's contribution to the centre versus periphery issues raised by recent developments in information technology is to question the value of physical centrality. there is, however, a marked contrast between the telecommuting scenarios of technologically optimistic futurists such as toffler (1970) and bell (1979) and the reality of technical achievements. the potential of computer-based information systems to facilitate or even substitute for organisational structure and standards has been evaluated from a variety of social and organisation theory perspectives. little (1988) shows how very different strategies may be pursued with the same equipment, but that technical developments may lag behind organisational ambitions. sproull and kiesler (1991) emphasise the difference between immediate technical gains from such technologies and the longer term process of social gains for organisations. specific technologies like computeraided design may substitute for organisationally enforced standards, while software standards and electronic date interchange are increasingly presenting organisations with externally derived standards and procedures. the ethnocentric focus of webber's original paper is reflected by the equation of modernism with western (us) urban forms and structures which are seen as the end product of a process of social evolution. castells (1990) readily acknowledges a us focus in his data. clegg (1990) indicates how north american organisational forms are still regarded as a norm, and argues that cross cultural examination of alternative organisational form, building on lammers and hickson (1979) is necessary to evaluate the full range of available possibilities. belussi (1989) describes the utilisation of a number of technical strands by the benetton company which result in a computer-based network which is both transnational and trans-organisational in extent. in this example, ownership and control of the core of the information network substitutes for ownership of production and distribution and for a formal hierarchy. the core of this enterprise is an information system linking a network of worldwide franchised retail outlets and a network of subcontractors who provide the wholesale products. developments in transnational work organisation and telecommunications have allowed real-time off-shore location of white collar work from the u.s. mainland to the caribbean and less dramatically have facilitated a division between "front office" tasks which remain in the prestigious cbd and "back office" tasks, often involving part-time and female workforces relegated to the periphery of suburbia (nelson; 1988). while information technology appears to offer a means of redressing relative locational disadvantage, experience so far suggests that the impact on locational mobility has been overestimated for the majority of the working population. webber himself discusses the relative disadvantage of the "by-passed preindustrial locals" (webber; 1968, p.1101), and the elite component of the workforce required by transnational corporations (tncs) may be the only actors fully to meet webber's definition of non-space interest communities. the critics of the urban planning of the seventies attacked redevelopments that privatised previously public spaces, yet the virtual space of computer networks is primarily private, with the deregulation and privatisation of common carriers widening this trend. the process of globalisation and deregulation has accelerated the privatisation of this infrastructure. emerging technologies and concepts such as virtual reality and computer supported collaborative work (cscw) offer the prospect of the annihilation of physical separation, but at a price, and at the cost of excluding a significant minority of a region's citizens. any attempt to redress such shifts on behalf of local or regional interests requires an alternative paradigm to that of the technological optimists. recognition of the existence of "cyberspace": a non-place realm of computer-supported relationships has taken webber's original conception considerably further. benedikt (1991) traces the term to william gibson's dystopian novels (gibson; 1984). the cyberpunk culture which has grown around the themes first articulated by gibson has both utopian and dystopian strands. the physical non-space aspects of arpanet, the original national research network in the united states are a striking indication of how far such tendencies may take organisations. the operating software could relocate itself across its own network to the least busy host machines. such non-space aspects of "netland" have been exploited by "cyberpunk" enthusiasts but have also attracted more respectable attention from a number of researchers. for example, canter and perin are applying respectively their cognitive mapping and environmental psychology methods developed n the seventies to the mental images and organisational relationships developed by users of complex interactive computer systems (see canter; 1977, perin; 1970, 1991). 42 ajis in a post-cold war environment it is clear that a number of institutions are reassessing their roles. some cold warriors are viewing this non-space arena as their new fiefdom. the national security agency has become deeply involved in the development of data verification and encryption, to the extent of proposing software and standards for commercial transactions, control of the encryption standards would enable them to monitor traffic. the proposals have been vigorously opposed by groups such as computer professionals for social responsibility (cpsr). nasa has similarly become involved in a reconfiguration of sdi or "star wars" projects which are directed not at the defence of the united states, but at denial of near-earth space to competing nations. they are effectively policing the infosphere for one nation's benefit. if the cpsr can be regarded as a nineties equivalent of the aclu or the naacp, then the demonised hackers of the networked world may yet emerge as the equivalent of the black panthers and urban guerrillas of the sixties. the phenomenon of "hacking" is indicative in part of resistance to emerging patterns of ownership and control. in their naivety these technophiles have become luddites subverting rather than attacking the technological base of a "new world order" with slogans such as "information wants to be free". the demonisation focuses on genuinely criminal and destructive activities by a minority of the "cyberpunk" sub-culture within computing circles. however, such activities themselves grew out of the fringes of the anti-war movement in the u.s.a. in the late sixties and early seventies, with the traditional telephone system being the first target for the "liberation" of resources (see bowcott & hamilton; 1990). this sub-culture has become the first example of an increasingly global non-space community inhabiting the "virtual space" of communication networks. it became global as the technological base has diffused from the leading nations. a great deal of traffic across the networks in the summer of 1991 originated in russia and described the events of the abortive coup to the outside world. in china, following the suppression of the democracy movement in 1989 there was a massive increase in the incidence of computer viruses. in britain at the height of the anti-poll tax campaign an elaborate hoax was staged to persuade local authorities that a computer virus had been introduced in order to corrupt the databases essential to the collection of this unpopular charge. the cyberpunk literature emphasises the intellectual and cultural possibilities of the emerging global "non-space realm". it provides an alternative vision to the privatising economic rationalism that has arisen with the mainstream. paradoxically it also draws attention to the critical qualitative features of the technology that will be central to its effective commercial and economic use. conclusion the "global village" concept, itself a cousin of webber's non-place realm, and the culturally acceptable face of globalisation, represents a top down diffusion of cultural hegemony. only now can the diffusion of information technology at various levels of sophistication support a possible twoway traffic. at the same time the concept of nation state has become problematic. the nineties is developing as the decade of the revival of subnational and regional sovereignties. these are readily supported by developments in global communication technology. the most striking example was the briefly canvassed suggestion during the iraqi occupation of kuwait that the dispossessed national government might continue to function on the world financial markets without access to its country, but with electronic access to its funds. jane jacobs was one of the first to report the negative effects of applications of modern urban planning theory in terms that affected the popular consciousness of the sixties (jacobs; 1962). she illustrated how the strict hierarchical decomposition of mainstream land-use planning failed to acknowledge either long established economic mechanisms, or more recent changes in the nature of industry and employment which made rigid separation of housing, industry and commerce less important than accessibility and diversity. twenty years later jacobs (1982) discusses the development of quebec separatism in a global context. subnationalism can be seen as an inevitable outcome of changes wrought since 1964. webber (1968) argues that the replacement of the city state by the nation state weakened the power of the traditional urban community. despite european community funded research includes a substantial programme addressing definitions of citizenship, the difficulties over ratification of the maastricht treaty indicate the lack of any clear consensus between and within member states. supranational government may revive city and regional status, as it erodes the status of national governments. experience with the development of transport informatics in support of road pricing policies suggests that direct negotiations between european community institutions, metropolitan september 1993 43 governments and trans-national corporations may diminish the influence of national governments over crucial standards for information and communications technologies. ohmae (1990) argues that regional and city level interests have replaced direct international competition. however the question remains how powerful these new "city states" might be in the face of direct competition with their neighbours for the attention and resources of tncs. he suggests that tncs strive for a position of "currency neutrality" so that relative shifts of exchange rates have a negligible impact on their activities. similarly location policy is also informed by the potential formation of trading blocks, and facilities and resources are planned so that operations are self-sufficient within each potential block. webber cites the growing importance of non-governmental and specialised governmental groups and bodies in urban life. industry and de-facto standards as well as formally agreed international standards are key elements of the emerging supranational infrastructure, just as railways, roads, and national telecommunication carriers were cited by webber as historically significant influences on locational decisions by individuals and organisations. in this context, the widespread deregulation of national telecom carriers may mark the final decline of purely national sovereignties. as these are prepared for privatisation their social role of equalisation of spatial disadvantage is usurped by the need to compete for the core, profitable markets. already one national australian bulletin board has had to relocate to a capital city because of a concentration on lucrative urban markets and a reduction of its redistributive role by the national carrier. local governments must now compete for the early availability of new communication resources, just as they one competed for access to railway routes. the situation is even more problematic in countries such as russia where unfettered privitisation has been unleashed in a situation of much greater existing inequity between centre and periphery (giglavyi; 1993). press (1992) cites landweber's calculation that 129 of 236 political entities still lack any direct connection to this global club. they effectively lack a basic ingredient of world citizenship. gender was little theorised in relation to the application of technology in the sixties, and aungles (1992) demonstrates how it can still be largely ignored in technological debates through her examination of the impact upon families of electronically monitored domestic detention. when coupled with sewell and wilkinson's (1992) analysis of electronic surveillance in the workplace, aungles' work shows us a complete penetration of both the domestic and employment sphere by the technologies of commerce. clearly at the levels of both national and personal life there is an imbalance in power reflected in access to the key technologies of ther emerging non-space realms. the technology itself is not the threat to established conceptions of citizenship, however. margaret thatcher attributed the 1992 u.k. general election result to the removal of up to one million voters from the register through their avoidance of the poll tax (hodgson; 1993). this is an example of the application of user-pays principles to what previously were regarded a basic civil rights. in this context there may be a positive role for the cyberpunk sub-culture, with its assumption of open access to the non-space of the emerging networks in the facilitation of the collective community action of the seventies through the technologies of the nineties. webber's paradigm of non-place realms can be linked with that of appropriate technology and windows of opportunity in the formation of that technology. this can provide guidance for the development of appropriate local responses to the globalising pressures resulting from the continuing development and coalescence of advanced information technologies. references aungles a. (1992) "electronic surveillance and organisational boundaries: the home as the prison" in aungles s. (ed) information technology in australia kensington: university of new south wales press bell d. (1979) "the social framework of the information society" in dertouzos m.l. & moses j. (eds) the computer-age: a twenty year view cambridge: mit press belussi f. (1989) "benetton: a case study of corporate strategy for innovation in traditional sectors" in dodgson m. (ed) technology strategies and the firm: management and public policy longman, london. benedikt m. (ed) (1991) cyberspace, first steps cambridge ma: mit press bowcott o. & hamilton s. (1990) beating the system: hackers phreakers and electronic spies london: bloomsbury. brunner j. (1975) the shockwave rider london: dent. 44 ajis camilleri j.a. & falk j. (1992) the end of sovereignty ? the politics of a shrinking and fragmented world edward elgar, aldershot. canter d. (1977) the psychology of space london: architectural press. castells m. (1990) the informational city clegg s. (1990) modern organisations: organisation studies in the post-modern world london: sage. earls j.a.m. (1990) "social integration of people with disabilities: the development of an information technology model based on personal growth and achievement" unpublished phd thesis, university of wollongong. gibson w. (1984) neuromancer grafton press, dublin giglavyi a. (1993) "it landscape in russia" international conference on people and information technology rtap'93, moscow, may 1993. hartamis j. & lin h. (eds) (1992) computing the future: a broader agenda for computer science and engineering washing d.c.: national academy press. headrick d.r. (1981) the tools of empire: technology and european imperialism in the nineteenth century oxford university press, oxford. hodgson g. (1993) "down a slippery poll" new statesman & society 21 may 1993, pp.18-19. jacobs j. (1982) the question of separatism london: junction books, kling r. (1993) "computing our future for a social world" communications of the acm, february 1993. lammers c.j. & hickson d.j. (1979) "a cross-national and cross-institutional typology of organisations" in: lammers c.j. & hickson d.j. (eds) organisations alike and unalike: international and interinstitutional studies in the sociology of organisations london: routledge & kegan paul. little s.e. (1988) organisational implications of computing for professional work, aldershot: gower/avebury. meir r.l. (1959) "measuring social and cultural change in urban regions" journal of the american institute of planners, vol.25, august 1959, pp.180-190. nelson k. (1988) "labor demand, labor supply and the suburbanization of low-wage office work" in: scott a.j. & storper m. production work and territory: the geographical anatomy of industrial capitalism boston: unwin-hyman. ohmae k. (1990) the borderless world: power and strategy in the interlinked economy collins, london. perez c. (1985) "microelectronics, long waves and world structural change: new perspectives for developing countries" world development 13(3) 441-63 press l. (1992) "the net: progress and opportunities" communications of the acm, vol.35 no. 12 december 1992,pp.21-25 rosenberg n. (1982) inside the black box: technology and economics cambridge: cambridge university press. sewell g. & wilkinson b. (1992) "'someone to watch over me': surveillance, discipline and the just-in-time labour process" sociology 26.2, may 1992, pp.271-290. sproull l. & kiesler s. (1991) connections: new ways of working in the networked organization cambridge: mit press. sterling b. (1992) the hacker crackdown new york: bantam. stoll c. (1989) the cuckoo's egg london: the bodley head. suchman l. (1986) plans and situated action cambridge: cambridge university press. toffler a. (1970) future shock new york: bantam. vitalari n.p. (1985) the need for longitudinal designs on the study of computing environments in mumford e. hirschheim r. fitzgerald g. & wood-harper a.t. research methods in information systems amsterdam: north holland. webber m. (1964) "the urban place and the non-place urban realm" in: webber m.m. dyckman j.w. foley d.l. gutenberg a.z. wheaton w.l.c. & wurster c.b. explorations in urban structure philadelphia: university of pennsylvania. webber m.m. (1968) "the post-city age" daedalus vol.97 no.4, fall 1968, pp.1091-1110. yuker h.e. block j.r. & campbell w.j. (1960) a scale to measure attitudes towards disabled persons human resources study no 5 albertson n.y.: human resources centre september 1993 45 ajis vol. 7, no 2 may 2000 eftpos impacts on branch banking: an extraorganisational analysis gregory e.p. shailer department of commerce, the australian national university, canberra 0200 kenneth j. stevens school of information systems, technology and management, the university of new south wales hargo utomo department of commerce, the australian national university abstract perceptions and experiences of bank branch managers and merchant who have installed eftpos are surveyed to investigate the impact of eftpos on branch banking activities using internal and extraorganisational perspectives. impacts are identified by reference to banking activities and interpreted as efficiency, effectiveness and competitiveness impacts. it appears that bank managers may have difficulty in isolating the impacts of eftpos on banking activities from concurrent changes in banking practices and might overestimate some beneficial impacts. results indicate a much richer view can be obtained using an extraorganisational perspective of a systems' impacts. key words eftpos, impacts, branch banking, small business, extraorganisational perspective introduction this study investigates the impact of the electronic funds transfer at point of sale (eftpos) system on branch banking activities by considering both the banks' and their merchant customers' perspectives. the results, while based on australian data, have implications for similarly developing systems. the approach used is potentially relevant to studying the impact of information technology (it) on other activities in other contexts. eftpos has become an important element in consumer markets with potentially powerful social influences on how individuals and organisations engage in retail transactions. australian banks have invested heavily in eftpos with significant take-up by retail merchants. individuals are increasingly using the system for everyday payment transactions. the system is now regarded as an established part of retail banking (gart, 1992) with extensive adoption in australia.[4] while some authors have speculated as to the impact of eftpos on banking and users (e.g. barras, 1991; chorafas, 1987; clarke and walters, 1989; .fraser, 1985), there appears to have been little empirical investigation of such impacts in any context. such empirical work is desirable because of the likely influence exerted by the technology on many aspects of retail banking and trading and the absence of published explicit knowledge. in assessing it impact, most studies are concerned with changes from an internal perspective and typically focus on aggregated financial outcomes that relate to cost or revenue objectives for the organisation as a whole (e.g. alpar and kim, 1990; harris and katz, 1991; mahmood and mann, 1993). barua et al. (1995) criticise this approach and propose that a more disaggregated approach be adopted. following weil and olson (1989) they canvas the notion that impacts should be identified at the strategic business unit level. this remains an introspective (or intraorganisational) approach which still appears inadequate when viewing a system that substantially interfaces with external parties. in a more general context of performance evaluation, kaplan and norton (1992) propose a 'balanced score-card' approach that requires consideration of customers' perspectives of service or product delivery this 'extraorganisational' approach is also in sympathy with developments in other domains, such as those dealing with interorganisational information systems (see gupta, 1995; johnston and vitale, 1988; kumar and van dissel, 1996) and those concerned with environmental impacts (see lederer and mendelow, 1990). in providing eftpos, banks compete at the institutional level. a substantial proportion of banks' other activities, such as delivery of the traditional retail banking products and services, occurs primarily through branches. branch banking activities, therefore, is an appropriate reference for the sub-unit approach for identifying impacts, with customers for those products and services providing the focus for the extraorganisational perspective. assuming banks can more easily identify the dollar costs attributed to eftpos per se and the revenues flowing from its adoption, the current study is restricted to looking only at the juxtaposition of these local (branch level) and extraorganisational (customer) perspectives. while this is only an intermediate phase of impact assessment, it serves to provide a richer view of impacts than if they are subsumed in the effects of other simultaneous developments (such as organisational restructuring and ongoing automation of transaction processing) through aggregated measures. the kaplan and norton (1992) approach of incorporating customers' perspectives focuses on the nature and delivery of the product. in the context of eftpos, the primary product is the eftpos facility for engaging in eftpos was introduced in australia in 1984 although the take up rate by merchants increased significantly only after 1990. the number of eftpos terminals installed by merchants grew from 26,314 terminals in 1992 to 116,704 by the end of 1996. similarly, the number of eftpos transactions increased from 129 million in 1991-92 to 470 million in 1995-96 (australian payment systems council, 1993,1996). 70 ajis vol. 7, no 2 may 2000 transactions, not the transactions per se. the main customer in this regard is the merchant who has installed eftpos, rather than the bank account holder who uses a debit card for purchasing goods and services from merchants using eftpos. note that merchants need not have acquired their eftpos facilities from the bank with which they conduct their branch banking. eftpos nevertheless may have impacted on branch banking activities as a result of changes in both bank processing of transactions and the substitution of eftpos for other types of retail transactions by merchants. in particular, bank branches which previously carried a substantial burden of transaction processing may have experienced changes in the nature of their functions and relationships with customers. the nature of impacts is outlined in the next section. issues in identifying and measuring eftpos impacts on branch banking activities are explored in the third section, followed by a description of the research design and data collection. the analysis is presented in the fifth section while implications and limitations are considered in the conclusion. impacts of eftpos the nature of impacts of most systems are necessarily complex, even at localised levels. nonetheless, it is analytically convenient to label impacts according to some schema regardless of any lack of exclusivity. three popularly identified labels for impacts of it investment (see earl, 1989 among others) are efficiency, effectiveness and the promotion of competitiveness. efficiency refers to resource utilisation within a particular it application (earl, 1989) at the operational level of an organisation (singleton et al., 1994). efficiency is generally associated with cost, accuracy and timeliness (mahmood, 1994). effectiveness refers achievement of organisations' goals (earl, 1989) typically it describes the extent to which it applications achieve business objectives (srinivasan,1985; lyytinen et a/., 1991; haga and zviran,1994). promotion of competitiveness refers to the use of it to respond to the changing nature of the business environment (earl, 1989). competitiveness is also described as arising from efficiency and effectiveness (galliers, 1993). this latter perspective seems more salient in the case of eftpos and banking particularly in the context of extraorganisational impacts. this is explored further within the empirical analysis. use of these labels to aid interpretation of changes wrought by it developments must recognise that some impacts will have multiple connotations for efficiency, effectiveness and promoting competitiveness. impacts on bank branch activities branch banking is the quintessential interface between banks and their retail customers. the principal branch functions in this regard are the delivery of identifiable products and services and ongoing interaction with customers. branch level impacts of eftpos should be identifiable by reference to activities associated with these functions. the literature already offers some speculation and evidence as to the nature of such impacts. there is some suggestion in fraser (1985) and chorafas (1987) that the spread of the eftpos system was expected to significantly reduce the paperwork involved in cash and paper-based payments. removing clerical aspects of transaction processing from banks' branch activities should affect timeliness and accuracy of payment transactions (following mahmood, 1994). two possible consequences for branches of this change are reduced staff numbers and increased staff activities in improving customer service and promoting other banking products. barras (1991) reports reduced demand for clerical staff that had previously undertaken these tasks. this, however, does not negate the possibility of changes in staff activities. increased staff activity in promoting bank products may involve extending the skills of individual staff and the introduction of additional products or services through the branch. if staff are successful in this regard, there should be an identifiable impact in the form of increased business from existing customers. the provision of centralised telephone assistance for eftpos and access to account balances combined with reduced cash and paper-based transaction volumes may impact on merchants' banking practices by changing the nature and extent of contact with bank branches. for example, clarke and walters' (1989) suggestion that merchants may benefit by reducing their costs of holding and transferring large amounts of cash to banks also implies reduced merchant contact with the bank. this may conflict with the possible impact of redirecting bank branch staff activities to improving customer service and promoting other products by reducing the potential for contact between branch staff and merchant customers. fraser (1985) argues that merchants will adopt eftpos to increase sales. if this dominates changes, rather than substitution of eftpos for other forms of transactions, then no impact on merchant contact with the bank would be expected. where eftpos is provided to a merchant by a separate agency, it may be seen as a "foot-in-the-door" threat to the banks' competitiveness. if this is the case, there should be an identifiable impact in the form new merchant accounts. the potential complexity of impacts on various branch activities, and their interaction, reduces the strength of a priori reasoning as to their cumulative impact on the bank-merchant relationship, leaving it as an issue to be explored empirically . how indicators of potential impacts are sought is described below in the context of the empirical research design. 71 ajis vol. 7, no 2 may 2000 research design and data collection indicators of potential impacts were sought by surveying bank branch managers and merchants who use eftpos. separate questionnaires[ ] were developed for bank managers and merchants which largely mirrored their perceptions or experiences of impacts of eftpos. the key information sought through the survey is listed in table 1. table 1 information sought regarding perceptions of eftpos impacts on branch banking activities eftpos impact on: bank managers' perceptions: merchants' perceptions of: transaction volumes • number of cheques transactions • number of cheques transactions • number of credit card transactions » number of cash transactions » number of cash transactions contact with banks • frequency of visits to bank • frequency of visits to bank * frequency of account queries * frequency of account queries branch staff • number of branch staff • staff skills regarding other bank products products offered • the number of bank services or products offered through the branch branch business • attracting more business from • use of other bank existing bank customers products/services • attracting business accounts to the branch • the bank's business relationships • the merchant's business with eftpos merchants relationship with their bank the bank manager survey focussed on the impact of eftpos on branch banking, distinguishing impacts on staff, customers, products or services, and internal activities. the merchant survey distinguished on the eftpos installation decision, reasons for adopting eftpos and the impact of eftpos on the business. both questionnaires used a mix of open-ended, categorical and scalar questions. many of the questions in regard to changes in banking practice and transaction volumes where common to both surveys. in developing the bank manager questionnaire, a consultative group of three bank managers and a merchant survey manager of a major bank group provided assistance in regard to the feasibility of information sort and served as a pilot group testing the questionnaire. three merchants provided similar assistance in the development of the merchant questionnaire. piloting resulting in some structural and "language" changes to the instruments.6 in generating the sample for bank branches, the policies for participation in surveys for each national and regional bank was firstly established thought contact with representatives of each bank. all banks with act branches were approached. four declined to participate. one bank required regional office approval and the others approved direct individual contact with branches. the questionnaires where mailed directly to branch managers in august 1996. it was necessary that the individuals completing the merchant questionnaire have detailed knowledge of the business and its experiences with branch banking. large retail corporations typically organise their banking via centralised accounts and facilities rather than through local branches. consequently, the study was limited to small merchants because they are most likely to operate branch accounts as their main bank accounts and only one or two people will hold most of the management functions. a prospective sample of 700 merchants was created from local business directories[7] and the merchant survey was mailed in september 1996. respondents twenty branch managers across four banks agreed to participate. most branch managers had at least three years experience, with 50% having more than 5 years as a branch manager. most (75%) had managed their current branch for at least 2 years. 5 initial discussions with bank representatives indicated that questionnaires would be more acceptable than interviews. 6 the consultative/pilot group participants were not included in the final survey. 7 the 700 merchants were identified from three main sources: the membership directories of the act chamber of commerce and the queanbeyan business council (nsw), and the telstra yellow pages. merchants were selected if they satisfied the small business criteria of the australian bureau of statistics (<100 employees). 72 ajis vol. 7, no 2 may 2000 of the 700 selected small merchants, 291 (41%) responded. of these, 105 (36%) currently used eftpos. these are described by industry and size in table 2. the profile of merchants shows substantial representation of responding eftpos adopters for a broad range of retail categories. most firms are quite small (less than 25 employees) so that the respondent managers/owners are likely to have close supervision of eftpos use and banking activities. table 2 industry and size profile of merchant respondents number of % of respondents total main business: department & general stores clothing & fabrics homewares etc petrol retailers food specialty retail hospitality services pharmacies number of staff 1 5 5 25 • 25 7 21 14 7 5 35 3 6 7 105 72 27 6 105 6.7 20.0 13.3 6.7 4.8 33.3 2.9 5.7 6.7 100.0 68.6 25.7 5.7 100.0 generally, the profiles of both bank managers and merchants suggest they are well placed to comment on the various issues and impacts considered in the questionnaires. the major limitation on the data set is the uncertainty of its representativeness of merchants and bank branches outside the act. analysis of responses the analysis of the responses are organised under the headings used in table 1. in each case, banks' and merchants' views are assessed in terms of implications for efficiency, effectiveness and competitiveness impacts. transaction volumes bank managers and merchants categorised the extent to which eftpos had substituted for other types of transactions (cheque, credit card and cash).[8] this is a difficult task for the bank managers who may not have adequate information regarding the composition of branch transactions or their breakdown by customer type. half of the bank managers were unable to estimate these effects. of those that gave estimates, results were so diverse that no clear view of changes can be obtained from banks' responses alone. the impact of eftpos on cheque deposits was described as positive (increasing) by 5 respondents, negative by 3 and no change by 2; impact on cash deposits were described as 3 positive, 2 negative and 5 no change. cash withdrawals had greater consensus with 7 positive and 3 negative. overall, this is contrary to expectations. while bank managers may have difficulty in addressing such details if all transaction levels for branches has increased substantially in recent years, the absence of data regarding changes in transaction volumes for individual branches leaves this as conjecture. those managers that identified an increase in cheque or cash deposit activities, which is logically inconsistent with eftpos, are evidence of the difficulty in separating the impacts of specific changes including generally increasing activity levels. the apparent increase in atm withdrawals at branches may indicate a product complementarity impact for banks whereby eftpos encouraged greater take-up and use of electronic facilities by non-merchant customers. however, the extent of other concurrent banking changes that may pertain to this effect places it beyond the scope of this investigation. the merchants' perceptions offer a more consistent and seemingly more reliable guide as to the extent and nature of these impacts. table 3 describes merchants' responses, adjusted for those that do not accept cheques or credit cards. approximately three quarters of respondents described reductions in cheques and credit card transactions as a result of adopting eftpos while about half (48%) identified some substitution for cash transactions. while most of these ' bank managers were not be expected to have knowledge of the timing individual merchant adoption of eftpos and therefore were asked to evaluate changes over the past three years, which coincides with the spread of eftpos through the merchant sample 90% of whom indicated that they had installed eftpos within the last three years (75% within the last 18 months). 73 ajis vol. 7, no 2 may 2000 substitutions, were placed in the lesser category ("some decrease"), "large decreases" were attributed in 32% of cases for cheques, 19% of cases for credit cards and 14% of cases for cash. the implication of these merchants' responses for the bank perspective is important. while banks identified little overall change in cheque deposits, merchants' reports of decreased cheque volumes imply efficiency gains in the clerical aspects of cheque processing at branches with fewer cheques to be entered by counter staff when accepting deposits and less sorting and validation activity. the lower incidence of substitution of eftpos for cash transactions for merchants and apparent increases in bank dealings in cash suggests that any efficiencies arising for merchants and banks through reduced cash handling may be minor at best. this is discussed below in terms of the frequency merchant contact with banks. the flow-on effectiveness and competitiveness impacts of transaction type substitution also may arise with the nature of bank contact with merchants and delivery of other products and services as discussed below. table 3 merchants' descriptions of substitution of eftpos for other transaction types % of merchant respondents transaction type cheque credit card cash some decrease 40 56 35 little or no change 8 2 35 some increase 32 19 13 unsure 20 23 17 total 100 100 100 contact with banks while the reduction in cheque volumes suggests merchants have opportunities for reducing the frequency of branch deposits, this may continue to be moderated by the degree of risk attached to holding cash. a small proportion of merchants (18%) reported that they reduced the frequency of their visits to bank branches. this appears to conflict with the 40% of bank managers who believed there had been some reduction in the frequency of such visits as a consequence of installing eftpos. some explanation of this disparity may lie in the aggregation of effects at the bank branch. bank managers were not reporting how many merchants had reduced the frequency of their visits, but rather whether the bank had experienced a net reduction in visits from merchants who had installed eftpos. it is unlikely, however, that the differences are caused by a large reduction in visits by some merchants given that less than 3% reported that their frequency had greatly decreased. similarly, there appear to be differences in perceptions of the changes in the frequency of account queries following the installation of eftpos. only 9% of merchants reported any decrease and another 9% reported some increase, whereas 25% of bank managers believed there had been a decrease and 10% believed there had been some increase. subject to the previous qualification, there are four possible explanations for these various differences in perceptions: • the results are distorted by sample differences if a significant proportion of the responding merchants deal with banks or branches not covered by the bank branch sample. • bank managers have a poor perception of the experiences and practices of their small merchant customers. • banks are obtaining efficiencies through reductions in such transactions that are not enjoyed by a corresponding number of merchants. • bank managers impute to eftpos effects from other systems. overall, responses indicate that efficiency impacts at bank branch level appear to be minor or have been subsumed in other coincidental changes to banking. if a substantial proportion of merchants are experiencing negative efficiency impacts, or at least failing to obtain the expected degree of positive impacts, then this may have significant implications for assessments of effectiveness or competitiveness impacts for banks. while it is not obvious as to whether merchants' or banks' views are the more reliable in this regard, if branch managers are overestimating the decrease in contact with merchants then this may cause inappropriate bank decisions in areas such as staffing levels training and branch-oriented marketing activities, with consequences for competitiveness. branch staff changes the potential efficiency gains in transaction processing (discussed above) raise expectations regarding staffing. half of the branches reported a decrease in staff numbers over the last 2 years, 10% reported an increase and 40% had no change. the changes in staff numbers had no statistical association with the changes in counter staff skills. most bank managers (90%) reported that staff had increased their skills or knowledge in other bank products, half of these being described as "greatly increased". this effectiveness impact has major implications for customer servicing (competitiveness), including providing the bank with opportunity to increase business and enhance the bankcustomer relationship. 74 ajis vol. 7, no 2 may 2000 there are substantial difficulties in separating restructuring and other industry developments that have also affected staffing practices. consequently, it is difficult to attribute these changes wholly to eftpos although 40% of bank managers believed that eftpos has a positive impact on the number of products or services offered through branches (45% believed there was no effect). branch business it has been suggested that the provision of eftpos facilities arose more through competitive necessity than as a new market willingly pursued by banks. evidence of this is scant but myers (1997) indicated that new zealand banks had first proposed eftpos systems to large retailers in the 1980s but had then sought to withdraw from the market, claiming unexpectedly high unit costs of transaction processing. key large retailers threatened to withdraw their custom from particular banks, thus forcing them to persevere with the establishment and provision of eftpos. this appears to have forced other banks to then follow suit to retain their positions in the broader banking services market. this view is not supported by the survey of merchants, although the situation may have changed substantially from the 1980s to the time of the survey. a significant proportion of merchants (29%) indicated that their eftpos provider was different from their provider of other banking services. this selection of a specific eftpos provider by merchants appears to treat eftpos as separable from other banking services. this is further evidenced by the relatively minor part played by eftpos in decisions to switch banks. while a modest proportion of merchants (25%) indicated that they had changed banks during the eftpos expansion period, most of these indicated that eftpos was not a consideration in their decisions to do so. the few that indicated that eftpos was particularly important in their decisions to switch banks (4 cases in total) all indicated at least one other equally important factor, and most identified 3 or 4. another 4 respondents indicated that eftpos was a factor but was much less important than other factors such as bank fees, debt provision and quality of branch service. there is no evidence that the provision of eftpos to merchants provides banks with a "foot-in-the door" . indeed, a remarkably consistent proportion (30%) of merchants used a separate eftpos provider, regardless of whether or not they had recently changed banks (whereas a "foot-in-the door" argument suggests that recent switchers should have a much lower proportion of separate providers). when combined with the reasons of merchants' switching banks, this provides support for the view that merchants do not associate eftpos services with their traditional banking services or products. this suggests that bank branches will not experience a significant competitiveness impact from eftpos. this is contrary to implications that may be drawn from the specific impacts (on the level and nature of branch business with merchants) identified in the context of transaction type substitution, branch-merchant contact and staff skills. this issue is examined using five factors as proxies for branch business changes: branch relationship with merchants; number of services or products offered through the branch; change in branch business from existing customers; merchants' view of their change in use of products ; and new branch business accounts. the first two of these may pertain to either retaining existing business (competitive necessity) or generating new business (competitive advantage), while the remainder pertain only to generating new business. most bank managers (66%) reported that eftpos had no impact on the branches' relationship with merchants. of the balance, 3% reported a deterioration and 31% reported an improvement. these proportions are reasonably consistent with merchants' views (72% no effect, 9% deteriorated and 19% improved). forty per cent of bank managers believed that eftpos increased the number of products and services offered through branches (45% believed there was no effect and 15% believed the impact was negative). while most merchants (95%) reported no change in their use of other banking products or services as a consequence of eftpos, 20% of bank managers perceived an increase in their branch business from existing merchant customers as a result of eftpos (70% of bank managers reported no change). similarly, 20% of bank managers reported an increase in business obtained through new business accounts as result of eftpos and 65% reported no change. while the proportion reporting increased business from existing customers is consistent with that reporting an improved relationship, there are only 2 respondents common to these categories. on balance, some bank managers believe that their branches have increased their business as a result of eftpos but this is not supported by the analysis of merchants' actions. as with other differences identified above, this may reflect the twofold aggregation problem in branch perspectives: that involving aggregation across merchant customers and that involving difficulty in separating eftpos impacts from other concurrent changes in banking. nonetheless, it highlights the desirability of incorporating the extraorganisational perspective in analysing impacts on business units. conclusion this study had two elements in its objectives. the first was to investigate the impact of eftpos on branch banking. the second was to assess the merit of analysing external parties experiences and perceptions in determining the impacts of an extraorganisational system on sub-units of an organisation. 75 ajis vol. 7, no 2 may 2000 possible branch level impacts of eftpos were identified by reference to changes to activities associated with branch functions (the delivery of identifiable products and services and ongoing interaction with merchant customers). such changes are labelled as efficiency, effectiveness and promoting competitiveness impacts. eftpos appears to have provided some efficiency gains to bank branches, identified in the context of transaction type substitutions and staff changes. this is identified from merchant responses, with bank managers' perceptions suggesting difficulty in separating such impacts from concurrent changes in other areas of banking for transaction substitution. the merchant perspective is supported by bank managers reporting reduced staff numbers or increased (substitute) activities by staff. these efficiencies, however, appear to be confined to the transaction processing activities and not in the frequency of counter transactions or other dealings with customers. effectiveness impacts for bank branches were identified by reference to dealings with customers, including the nature and frequency of contact with merchants and the bank-merchant relationship. while some bank managers believed there were identifiable decreases in merchant visits to or inquiries through the branches (40% and 25% respectively), merchant perspectives indicate that eftpos had little or no impact in this regard. whether this represents overstatements by bank managers or differences in perspectives caused by bank managers' views being aggregations across their merchant portfolios cannot be determined. in either case, this demonstrates that a more accurate or richer view of impacts might be obtained by analysis of extraorganisational perspectives. reasonably consistent views were obtained in the case of the bank-merchant relationship, with improvements reported by substantial minorities of both merchants and bank managers. importantly, however, this does not appear to have flowed through to the competitiveness impacts. if banks have experienced any competitiveness impacts from eftpos, it seems unlikely that this was at the branch level. again, this was evidenced most clearly by reference to merchants' behaviour, rather than the internal perspective. again, it cannot be determined whether these differences are due to overstatements by bank managers or bank managers' views being aggregations across their merchant portfolio. while a significant minority of bank managers reported increases in branch business, 95% of merchants indicated that they had not changed their use of other bank products or services as a result of eftpos. when combined with the analysis of merchant behaviour in choosing eftpos providers or switching banks, it further illustrates the potential value of incorporating extraorganisational perspectives in assessing organisational impact. limitations several data limitations reduce the generalisability of these results. all of the data is regional although any differences between the act region and the rest of australia in relation to banking or the use of eftpos cannot be identified. likewise, the nature and extent of any response bias for either banks or merchants is unknown. however, the range of merchant types in the sample and variety of individual responses suggest little likelihood of particular responses biases in respect of the issues considered. the number of bank branches in the sample limit the extent to which responses can be disaggregated. nonetheless, it provided a reasonable basis for this exploratory study of the experiences and perceptions of key participants in a largely overlooked extraorganisational it application. methodologically, it is important to note that the assessment of impacts were based primarily on perceptions. banks should be able to pursue a more rigorous analysis of impacts using quantitative data from internal records, but it seems this may still be unduly limited if they do not take account of extraorganisational perspectives of impacts. implications the implications of this study for assessing the impacts of eftpos are modest given the limitations noted above. however, it does suggest that, even at the sub-unit level, bank managers may have difficulty separating the impacts of individual it changes from the general melee of changes on their activities that is to be expected in a dynamic commercial environment. a more general implication of this exploratory study is its highlighting of the potential of an extraorganisational approach when assessing impacts. while this has been demonstrated at the sub-unit (branch) level for banks in the case of eftpos, it may be as or more valuable where internal perspectives are based on more aggregated measures or perceptions. this remains very much an exploratory study with much yet to be resolved. application of an extraorganisational approach in other cases, with greater access to internal data or with reference to multiple external perspectives (such as non-merchant customers in the case of eftpos) should advance understanding and lead to more comprehensive treatments of it impacts on organisations. references alpar, p. and kim, m. (1990), "a microeconomic approach to the measurement of information technology value", journal of management information systems, vol 7 no 2, pp 55-67. australian payment system council (1993), annual report, canberra: reserve bank of australia. australian payment system council (1996), annual report, canberra: reserve bank of australia. 76 ajis vol. 7, no 2 may 2000 barras, r. (1990), "interactive innovation in financial and business services: the vanguard of the service revolution", research policy, vol 19, pp 215-237. barua, a., kriebel, c.h. and mukhopadhyay, t. (1995), "information technologies and business value: an analytic and empirical investigation", information systems research, vol 6 no 1, pp 3-33. chorafas, d. (1987), strategic planning for electronic banking: from human resources to product development and information systems, butterworths, london. clarke, r. and walters, m. (1989), "consumer efts in australia: an introduction", computer law and security reporter, vol 4 no 4. delone, w. and mclean, e. (1992), "information systems success: the quest for the dependent variable", information systems research, vol 3 no 1, pp 60-95. earl, m. (1989), management strategies for information technology, new york: prentice hall. frazer, p. (1985), plastic and electronic money: new payment systems and their implications, cambridge: woodhead-faulkner. galliers, r. (1993), "it strategies: beyond competitive advantage", journal of strategic information systems, vol2no4,pp.283-291. gart, a. (1992), "how technology is changing banking", journal of retail banking, vol xiv. no.l, pp 35-43. gatian, a. (1994), "is user satisfaction a valid measure of system effectiveness?", information and management, vol 26, pp 119-131. gupta, a. (1995), "a stakeholder analysis approach for interorganisational systems", industrial management and data systems, no 6, pp 3-7. haga, w. and zviran, m. (1994),"information systems effectiveness: research designs for causal inference", information systems journal, vol 4, pp 141-166. harris, s. and katz, j. (1991), "organisational performance and information technology intensity in the insurance industry", organisation science, vol 2 no 3. pp 263-295. johnston, h. and vitale, m. (1988), "creating competitive advantage with interorganisational information systems", mis quarterly, vol 12 no 2, pp 153-165. kaplan, r. and norton, d. (1992), "the balanced scorecard measures that drive performance", harvard business review, january february, pp 75-85. kumar, k. and van dissel, h. (1996), "sustainable collaboration: managing conflict and cooperation in interorganisational systems", mis quarterly, vol 20 no 3, pp 279-300. lederer, a. and mendelow, a. (1990), "the impact of the environment on the management of information systems", information systems research, vol 1 no 2, pp 205-222. lincoln, t (1990), managing information systems for profit, chichester: john wiley & sons. lyytinen, k., klein, h. and hirschheim, r. (1991), "the effectiveness of office information systems: a social action perspective", journal of information systems, vol 1, pp 41-60. primozic, k., primozic, e. and leben, j. (1991), strategic choices: supremacy, survival, or sayonara, mcgrawhill, new york, as quoted in sprague, jr., r. and mcnurlin, b. (1993), information systems management in practice, third edition, englewood cliffs, nj: prentice-hall. mahmood, m. (1994), "evaluating organisational efficiency resulting from information technology investment: an application of data envelopment analysis", journal of information systems, vol 4, pp.93-115. mahmood, m. and mann, g. (1993), "measuring the organisational impact of information technology investment", journal of management information systems, vol 10 no 1, pp 97-122. singleton, j., mclean, e. and altman, e. (1994), "measuring information systems performance: experience with the management by results system at security pacific bank," in gray, p., king, w., mclean, e. and watson, h. management of information systems, second edition, new york: the dryden press. srinivasan, a. (1985), "alternative measures of system effectiveness: associations and implications", mis quarterly, vol 9 no 3, pp 243-253. 77 ajis vol. 6 no. 2 may 1999 refinement of research surveying in software methodologies by analogy: finding your patch eugene eric doroshenko department of computer science and software engineering university of melbourne email: eueene@cs.mu.ozau felicity lear school of information systems university of tasmania email: felicitv.lear@infosvs.utas.edu.au abstract to enhance research surveying in software methodologies, a model is introduced that can indicate field maturity based on vocabulary and relevant literature. this model is developed by drawing analogies with software methodologies. two analogies are used: software models and software life cycles or processes. how this model can reduce research surveying problems for researchers is described using extracts from application results as examples. although the model does support research surveying activities, it cannot choose the subject for the researcher. introduction part of research in software methodologies is reviewing of literature and identification of candidate projects based on the literature review. the literature review process assists knowledge acquisition by the researcher about a subject. two foundations of candidate projects are the maturity of the field and unresolved issues in the literature. areas that are immature are likely to have more candidate projects. the more unresolved issues there are, the more projects that can be derived from them. hence, the literature review is a foundation for knowledge acquisition as well as project identification based on field maturity and unresolved issues. all of these are part of research surveying activities. researchers in software methodologies still face some problems. firstly, a researcher may have little experience in research surveying; honours, masters, and phd students are three examples. secondly, beyond the scope of keywords, number of literature sources, and types of literature sources, much surveying tends to be ad hoc in nature and highly dependent on the experience of the researcher. thirdly, knowledge acquisition cannot be avoided, but tracking and organising such knowledge for a literature review can be difficult, particularly in mature fields. some frameworks and models (fms) classify software methodologies for various purposes, such as research surveying of case technology (dart, et al., 1987; fuggetta, 1993), research surveying of modelling methods (doroshenko, 1994; livari, 1995; monarchi and puhr, 1992; roper, 1995), evaluation of software process modelling (mcchesney, 1995), and technology transfer (korson and vaishnavi, 1992). although the fms assist knowledge acquisition and research surveying of a subject, they tend to be specific to their subject and lack sufficient generalisation for application in other subjects. in any case, research surveying activities are not dealt with explicitly. one fm1 has shown some success in more generalised application. hence, to deal with the above issues, a model for research surveying in software methodologies is presented using doroshenko (1994) as a starting point. a number of features are introduced based on fms that classify software methodologies. a model for construction of research surveying tools that incorporates these features is outlined. the sum of these components are analogous to a software model. a development process is described to show how the model components support research surveying activities. the development process is analogous to a software life cycle. stages in the development process are literature acquisition, component formation, data gathering, and data analysis. literature acquisition is done as preparation for component formation. component formation and data gathering assist knowledge acquisition in a subject. the results of data gathering are used in data analysis to support identification of candidate projects via field maturity and unresolved issues. in the same way that software applications for a problem are built using software models, a research surveying tool for a subject is made using 1 doroshenko (1994) described a framework for classifying software methodologies using object-oriented methods. this was applied to structured process and structured data methods by roper (1995). both doroshenko and roper received first class honours, and doroshenko went on to further research obtaining a university of tasmania scholarship with apa conditions and stipend. 13 ajis vol. 6 no. 2 may 1999 the model components. for supervisors, this work can be presented to research students to assist their research surveying activities and lessen supervisor workload. limitations and issues relating to alternative application and specification conclude this paper. model foundations table 1 summarises the foundations for a research surveying tool in software methodologies. note well, it is the features themselves that are important, not the examples of the features. for example, "dimension" as a feature, not the example of a dimension, such as "aspect dimension" or "viewpoint dimension" in livari (1995). table 1: feature identification based on sources source (beringer, 1994) (curtis, et al., 1992) (doroshenko, 1994) (doroshenko, 1996) (fernstrom, et al., 1992) (fuggetta, 1993) (henderson-sellers and edwards, 1990) (livari, 1995) (korson and vaishnavi, 1992) (omg, 1996) (lowry, et al., 1994) (mcchesney, 1995) (monarch! and puhr, 1992) (snyder, 1993) location 1-3, 5-7 76-80 19-37 162-168 38 28-38 143 145 156-158 106 109 5-7, 14-16 223 224 364 368 36-40 38-42 features dimension, subdivision, vocabulary, concept/representation dimension, subdivision subdivision, vocabulary, concept/representation, dependency dimension, subdivision, concept/representation, dependency aggregate dependency subdivision, vocabulary, aggregate dependency, classification dependency aggregate dependency, subdivision dimension, subdivision, vocabulary, prerequisite dependency subdivision, classification dependency, vocabulary subdivision, vocabulary dimension, subdivision classification dependency subdivision, prerequisite/classification dependency, concept/representation concept/ representation location refers to page numbers, concept/representation), in others in some cases the feature is discussed (e.g. beringer (1994) for it is only evident (e.g. monarchi and puhr (1992) for dependency) dimensions divide an fm into mutually exclusive areas, e.g. lowry et al (1994) have methodology, organisation and linkage dimensions. livari (1995), based on de champeaux and faure (1992), has aspect and viewpoint dimensions. these are usually illustrated as lines at right angles with words (e.g. figure 1; stages, components) subdivisions divide a dimension into smaller parts, e.g. beringer (1994) has a model level dimension with analysis and design subdivisions. livari (1995) has structure, function, and behaviour subdivisions for the aspect dimension, along with individual and object community subdivisions for the viewpoint dimension. these are usually illustrated as intervals along the lines representing dimensions (e.g. figure 1; analysis, design, modelling, method process). vocabulary form the basic units for subdivision intersections, e.g. livari (1995) has vocabulary for individual structure (object, attribute) and object community structure (inheritance, subsystem). monarchi and puhr (1992) have a process subdivision with identify, placement and specification of classes. these are usually represented as words enclosed by areas delineated by subdivisions (e.g. figure 1; class, object, operation, aggregation, identify class, identify object, identify aggregation). korson and vaishnavi (1992) enumerate the notion of vocabulary description. a conceptual vocabulary item has a "name:" as the conceptual term, a "type" analogous to its subdivision, source contributions identified as "references", and information defining the term in "description", "notation:", and "example". an fm can have a division between concepts and their representation, e.g. snyder (1993) indicates division between concept meaning (object) and its varying terms (instance, class instance, object). doroshenko (1994) 14 ajis vol. 6 no. 2 may 1999 has a conceptual model of the method process, and a representation model of modelling notation. monarchi and puhr (1992) associate modelling concepts with their graphic representation. dependencies can further organise fm parts. these include include: classification; one fm part is a more general form of another fm part. e.g. mcchesney (1995) has classification dependency for specific concepts, such as style (ai, petri net, functional, oo) and notation (text or graphic). aggregation; one fm part is composed of, or part of, another fm part. e.g. henderson-sellers and edwards (1990) have subdivision aggregate dependency, where a phase (build) is composed of more specific phases (coding, program testing, program use). prerequisite or existence; one fm part is dependent on another fm part for its own definition. e.g. monarchi and puhr (1992) have the process and representation subdivisions dependent on objectoriented modelling concepts. doroshenko (1994) has the notation and method process for objectoriented methods dependent on their modelling concepts. components modelling method process class, object, operation aggregation identify class, identify object, identify aggregation analysis \ dimensions are modelled using the axis component conceptual vocabulary is modelled using the conceptual vocabulary component design stages subdivisions are modelled using the subdivision component figure 1: illustration of research surveying tool, analogous to a software application the above features appear across many fms. but each fm usually contains only a subset of the features described, and lacks generalisation for research surveying in other subjects. one fm does indicate a means of tool generalisation using selfdefinition (doroshenko, 1996), but it is specific to case technology. doroshenko (1994) is more applicable to research surveying and suggests how some features can be integrated, but lacks sufficient generalisation for other subjects. to integrate the above features into a model for construction of research surveying tools, the author uses doroshenko (1996) as a foundation for generalisation, and doroshenko (1994) as a foundation for structure. based on the approach in doroshenko (1996), the model is described using three tools, resulting in a number of model components. the model components extend and generalise the framework in doroshenko (1994) with the features summarised in table 1. the specification tools are table templates (appendix b), er modelling (figure 2), and an ebnf variation called the text definition language (tdl, appendix a) based on four sources ((backus, 1959; doroshenko, 1996; naur, 1963; sethi, 1989), see appendix d for an overview of tdl). the er model gives an overall picture of the model. the tdl and table template specifications are notation alternatives to forming a research surveying tool. 15 ajis vol. 6 no. 2 may 1999 f u r t h e r research component figure 2 illustration of model components based on e-r modelling model components as a software model a) axis generic term: stage [ description: a software methodology is divided into a number of phases that represent periods in time, with eac phase setting some aim or purpose for work done in the phase based on the phase name ] term used phase stage activity source vlasbolm et al 1996; henderson-sellers and edwards 1990; jones 1990; fuggetta 1993; evidence page 597, figure la, page 597; page 144; "...the normal stages of the software life cycle are shown in figure 1-8. each product passes through these stages although the duration... of each stage may vary..." (p 1 1), pages 1 119 page 28, page 32 b) axis generic term: component [ description: a software methodology is divided into a number of components that describe its processes and acts. ] [ term used h source ter hofstede et al 1997 evidence pages 404 405 c) stage axis. dependencies: type: component. design; subdivision generic term: logical design [ description: the logical design stage is where a system is developed according to how it will solve a problem, independent of a programming language and hardware environment ] term used logical design source firesmith 1993; monarchi and puhr 1992; evidence pages 15 16; page 36 -37 figure 3: example axis and subdivision component instances the axis component supports formation of axes. each axis has a name, indicating the conceptual name of the axis, and some interpretation. for example, in figure 3a) the name of the axis is "stage", its interpretation starts as "description: a software methodology is divided into a number of phases that represent periods in time...". 16 ajis vol. 6 no. 2 may 1999 an axis has a number of representation terms based on evidence in literature sources. for example, in figure 3a) the representation terms are "phase", "stage", and "activity". a literature source for the "activity" representation term is "fuggetta 1993". the evidence in "fuggetta 1993" is "page 28, page 32". figure 3a) and b) illustrate examples of the axis component using its table template specification. see, table 6 for details on the relationship between the axis component and its specifications in appendix a and appendix b. a) analysis modelling conceptual vocabulary dependencies: analysis modelling concetual vocabulary type: aggregate; object. vocabulary generic term: operation [ description; an operation can either access or change the state of an object. ] term used operation service behavior source booch 1994 henderson-sellers and edwards 1994 monarchi and phur 1992 evidence page 43 page 5 1 pages 39 40 b) analysis modelling conceptual vocabulary dependencies analysis modelling conceptual vocabulary type: classification. aggregation ; vocabulary generic term: owns aggregation [ description; a class is composed of classes, such that deletion of the class implies deletion of its parts. term used owns aggregation existence dependent source maciaczek et al 1996; rahayuetal 1996; evidence page 435, paragraph 7, starter quote: ". aggregation..." .the owns pagees 524 525, figure 2, page 524 c) analysis method process conceptual vocabulary dependencies: analysis modelling conceptual vocabulary type: prerequisite. class; method process conceptual vocabulary type: prerequisite identify vocabulary generic term: identify class [ description: an activity that identifies classes in an object model ] term used finding classes source monarchi and puhr 1992; evidence "...an important process in ooad is the identification of classes..." page 39, table 3 page 39, figure 4: example conceptual vocabulary component instances the subdivision component supports partitioning of axes based some axis component an axis is linked to one or more subdivisions based on the axis name. for example, in figure 3c) "stage axis" links the "stage" axis to the logical design subdivision. each subdivision has a name, indicating the conceptual term for the subdivision, and some interpretation. for example, in figure 3c) the name for the subdivision is "logical design". the interpretation for the logical design subdivision starts as "description: the logical design stage is where a system is developed according to how it will solve a problem...". a subdivision has one or more representation terms based on evidence in literature sources. for example, in figure 3c) the logical design subdivision has the representation term "logical design". two literature sources for this term are "monarchi and puhr 1992" and "firesmith 1993". the evidence in "monarchi and puhr 1992" is "page 36 37". subdivisions can have dependency on other subdivisions from the same axis. dependency types include aggregate, prerequisite or existence, and classification described earlier. for example, figure 3c) the logical design subdivision is a part of the design subdivision (aggregate dependency). this is indicated by 'type: component design;". figure 3c) illustrates an example of the subdivision component using its table template specification. 17 ajis vol. 6 no. 2 may 1999 see appendix c table 5 for details on the relationship between the subdivision component and its specifications in appendix a and appendix b. the conceptual vocabulary component supports grouping of related vocabulary concepts based on a vocabulary area. the vocabulary area is formed using subdivision names from subdivisions. for example, in figure 4a) the vocabulary area is "analysis modelling conceptual vocabulary". this means there are two subdivisions named "analysis" and modelling". a vocabulary area has a number of vocabulary concepts. each concept is given a name, indicating its conceptual term, and some interpretation. for example, in figure 4a) operation is a vocabulary concept, its name is "operation"! its interpretation is "description: an operation can either access or change the state of an object.". a concept has one or more representation terms based on evidence in literature sources. for example, in figure 4a) the vocabulary concept operation has "operation", "service", and "behavior" representation terms. the literature source for the "operation" representation term is "booch 1994". the evidence for this in "booch 1994" is "page 43". concepts can have dependency on other concepts in vocabulary areas. dependency types include aggregate, prerequisite or existence, and classification. for example in figure 4a) the vocabulary concept "operation" is part of the concept "object" (aggregate dependency). this is indicated by "analysis modelling conceptual vocabulary type: aggregate.object;". in figure 4b) the "owns aggregation" concept is a more specific kind of "aggregation" concept (classification dependency). this is indicated by "analysis modelling conceptual vocabulary type: classification.aggregation;". in figure 4c) the "identify class" concept depends on the "identify" concept (prerequisite dependency). this is indicated by "method process conceptual vocabulary type: prerequisite. identify;". figure 4a), b), and c) illustrate examples of the conceptual vocabulary component using its table template specification. see appendix c for details on the relationship between the conceptual vocabulary component and its specifications in appendix a and appendix b. the further research component supports identification of research issues based on axes, subdivisions, vocabulary areas, and conceptual vocabulary. a further research component is linked to an axis, subdivision, vocabulary area, or vocabulary concept(s). for example, in figure 5a) the "stages activities phases further research" component is linked to the "stages" and "phases" axes, and the "activities" subdivision. one or more research issues are indicated, along with evidence from literature sources that contribute to issue identification. for example, in figure 5a) the description of an issue starts as "notes: some authors use phase and stage interchangeably...". a literature source that contributes to issue identification is "vlasbolm et al 1995". evidence for the issue in "vlasbolm et al 1995" includes "figure 5 page 599". figure 5a) gives an example of the further research component using its table template specification. see appendix c table 10 for details on the relationship between the further research component and its specifications in appendix a and appendix b. the literature source component specifies the various sources of information. instances of the literature source component specify the literature in a subject for reference by instances of other model components. (usually <evidence entry> or <literature source>/<evidence statement appear as part of the specification). a literature source has a unique identifier using some referencing system. a literature source is linked to an author or authors. for example, in figure 5b) the identifier for the literature source in "booch 1994". the author of the literature source "booch 1994;" is "author: booch, grady;". this is indicated by "author: booch, grady;" following its identifier. literature sources can be of a particular type. literature sources are linked to the evidence extracts they contain. for example, in figure 5b) the type of the source is a book, indicated as 'type: book;". two evidence extracts are "p 88." and "figure 5-24 (p. 209). a number of sources can be included as a literature source component. korson and vaishnavi (1992) figure 2, page 103) illustrate a number of examples. figure 5b) illustrates an example of the literature source component using its tdl specification. see appendix c table 12 for details on the relationship between the literature source component and its specification in appendix a. specification of instances can be minimised by using a citation formatting package like endnote®. the key identifiers for literature sources in endnote® can be used as instances of <literature source> in instances of other model components. see the second row of the table in figure 6a) for an example. evidence extracts linked to a literature source still need consideration. specifically, certain terminal and non-terminal sequences in the <literature source entry> production can be left out. these are "authors: { <author entry> ; }", <source title>, <source typo, <year published>, and <publisher>. the "<author declaration>"can also be left out. but the terminal/non-terminal sequences "evidence: { <evidence statement> } ;" and "<literature source> ;" cannot be left out. 18 ajis vol. 6 no. 2 may 1999 a) stages activities phases further research notes: some authors use phase and stage interchangeably. fuggetta (1993) shows explicit discrimination between phase and activity, with activities composed of tasks. in contract, vlasbolm et al (1995) have phases composed of activities. hence, use of fuggetta for assessing case tools for methodologies formed using vlasbolm (1995) could be difficult. source fuggetta 1993 vlasbolm etal 1995 monarchi and puhr 1992 henderson-sellers and edwards 1990 evidence "...tasks are formed to group activities... the activity concept is not to be confused with the phases of a waterfall life cycle. activities are not necessarily carried out in strict sequence: they can be composed to form any type of life cycle..." (page 28), "...products for analysis and design activities..." (page 32) "...the most striking feature of many methodologies is their way of dividing up the development into phases... development phases are subdivided into activities..." (page 597), figure 5 page 599 "...while identifying clases, attributes and behavior has been fairly throughly covered, placing them has been largely negelected..." (p 41) "...the design stage is perhaps the most loosely defined since it is a phase..." (page 144), "...undertake system requirements specification. this stage..." (page 149) b) author: booch, grady; booch 1994; author: booch, grady; title: object-oriented analysis and design with applications; type: book; city: redwood city; edition: 2nd; number of pages:589; year: 1994; publisher: addison-wesley; evidence: p88. figure 5-24 (p. 209). "...figure 5-24 shows the icon we use to represent an object in an object diagram..." (p. 208). figure 5: example further research and literature source component instances figure 6a) elucidates application of the table template in appendix b for the conceptual vocabulary component, resulting in an instance of the conceptual vocabulary component. figure 6b) illustrates the same instance using the tdl specification in appendix a. note the relationship between figure 6a) & b) and the vocabulary row in table 2 and table 3. both the table template and tdl specifications use common constructs with a different style. note also the use of type and instance. for example, using figure 6a), the name of a concept is specified using "vocabulary <generic term>", with an instance being "vocabulary generic term: identify class". analogical view of model components when using the component specifications, model components can be viewed as as a software model, with the tdl and table template specifications being two forms of notation (author's preference). alternatively, the tdl specifications can be viewed as a programming language, with the specifications as record types and the instances as record variables; or the table templates as tables in a database that are filled out with data. table 2 summarises the relationship between the features in table 1 and tdl specifications for model components in appendix a table 3 summarises the relationship between the features in table 1 and table templates for model components in appendix b. the specifications usually reflect the descriptions of the components above. for example, the axis component has an axis name, which is specified using the non-terminal <generic term> in <axis name>. further details about model component specification can be found in appendix c. 19 ajis vol. 6 no. 2 may 1999 <vocabulary depdenency> <conceptual vocabu la ry area> a) analysis modelling conceptual vocabulary dependencies: analysis modelling conceptual vocabulary type: classification. operation; analysis modelling conceptual vocabulary type: aggregate. object; vocabulary generic term: modifier operation 'description: a modifier operation modifies the state of an object] -^ — " ----..,,"~.. „.....,, term used modifier operation source booch 1994 [henderson-sellers, 1994 #38] evidence page 88 "...operations are services of a class that lead to a change in the state of an object of that class..." (p 52) <term used> vocabulary <generic term> <literature source> v <evidence s ta tement> <vocabulary depdenency> <conceptual vocabulary area> b) analysis modelling conceptual vocabulary • dependencies: analysis modelling conceptual vocabulary type: classification. operation; • analysis modelling conceptual vocabulary type: aggregate. object; vocabulary generic term: modifier operation ^ vocabulary <generic term> <description entry> [ description: a modifier operation modifies the state of an object ] ->. vocabulary representation terms ^ modifier source: booch 1994; evidence entry: page 88 ; operation source: henderson-sellers and edwards 1994; _^. evidence entry: "...operations are services of a class that lead to a change in the state of an object of that class..." (p52);; vocabulary representation terms> <vocabulary entry> figure 6: example conceptual vocabulary component specification table 2 features in model components (tdl specifications) aggregate features/components axis/ <axis declaration> subdivision/ subdivision declaration> vocabulary/ <conceptual vocabulary declaratioro further research/ <further research declaration> literature/ <literature source declaration> concept <generic term> <description entry> <generic term> <description entry> <vocabulary entry> <generic term> <description entry> <notes> n/a representation <term used> <evidence entry> "source/evidence" <term used> <evidence entry> <term used> <evidence entry> <evidence entry> <evidence statement> dependency n/a <axis subdivision title> <subdivision dependency> <conceptual vocabulary area> <vocabulary dependency> <further research title> n/a 20 ajis vol. 6 no. 2 may 1999 table 3 features in model components (table templates) aggregate features/components axis/ axis component subdivision/ subdivision component vocabulary/ conceptual vocabulary component further research/ further research component concept <axis name> <description entry> <generic term> <description entry> <generic term> <description entry> <notes> representation <term used> <literature source> <evidence statement> <term used> <literature source> <evidence statemeno <term used> <literature source> <evidence statement> <literature source> <evidence statement> dependency n/a <axis subdivision title> <subdivision dependency> <conceptual vocabulary area> <vocabulary dependency> <further research title> the model component in figure 6a) is usually described as a type (e.g the conceptual vocabulary component), with examples described as instances (e.g. analysis modelling conceptual vocabulary). other model component specifications use a similar pattern, ie one component has many instances similar to a structured process or object model having many application models; or the concept of a class in an object model (the type), with classes such as myclass and queueclass (the instances). in this work, model components are types, whereas instances are associated with one of these types. for example the axis component is the type, and the stages axis is an instance of the axis component or an axis instance. the illustrations in figures 3 6 are all instances of model components. the tdl and table template specifications define the model components and their instances to some systematic degree. these specifications in appendix a and appendix b are like type definitions. the instances of these type definitions are like type variables (e.g. figures 3 6). each named component is a kind of model component, in the same way that a superclass has subclasses. for example, the axis and subdivision components are kinds of model components. the instances of model components together represent a research surveying tool in a subject for software methodologies (see figure 1). the model components can be viewed as a meta-model of research surveying tools, with instances as particular research surveying tools. this is similar to metamodelling in software methodologies (doroshenko, 1997; nissen, et al., 1996; steele and han, 1996), where an object model is a meta-model of application models and application models are instances of this meta-model. development process as a software life cycle the process for applying the model is illustrated in figure 7a). the literature acquisition stage is where relevant literature is identified and read based on a subject using standard methods (e.g. leedy (1993) or howard and sharp (1983)). the purpose of this stage is to start the knowledge acquisition process in a subject and prepare the researcher for component formation. the products of this stage are instances of the literature source component. the literature source instances are similar to a bibliography of the literature. two examples of this are shown in figure 5b) and figure 7b). the component formation stage is where a number of dimensions, subdivisions, vocabulary areas, vocabulary concepts, and dependencies are identified as suitable for a research surveying tool. the purpose of this stage is to further knowledge acquisition of a subject of the researcher through the act of classification, and prepare a suitable framework for the data gathering stage. the products of this stage are draft axis, subdivision and conceptual vocabulary instances. the draft instances contain the names, dependencies, and interpretations, but no reference to identifiers for literature sources or representation terms (ie excluding instances of representation terms>, <literature source>, <term used>, and <evidence statement). an example of this is shown in figure 7c) using the conceptual vocabulary component. the data gathering stage is where literature sources are linked to axes, subdivisions, and conceptual vocabulary along with representation. the purpose of this stage is to further refine the researchers knowledge in a subject by classifying the literature using the instances and prepare the instances for the data analysis stage. by modelling the vocabulary and literature sources, researchers create a reference resource for revision via vocabulary and navigation based on literature and component instances. the products of this stage are complete axis, subdivision, and conceptual vocabulary instances. these are based on the draft instances from the component formation stage. examples of this are shown in figure 3, figure 4, figure 6, and figure 7d). 21 ajis vol. 6 no. 2 may 1999 a) fc literature aquisition 1 -+[ j ~ 1 component formation | i j"~"---^ o) mal ^-^^ uep ^^ anal typ i f t y p vnrf \ voc' data gathering [de 1 j\ chan author: booch, g author: henderson-sellers, b... b) booch 1994; authors: booch g; ~~ ^" title: object-oriented analysis and design; henderson-sellers and edwards 1994; authors: henderson-sellewrs, brian; edwards, julian; title: book ii of object-oriented knowledge ysis modelling conceptual vocabulary endencies: ysis modelling conceptual vocabulary e: classification; operation, e: aggregate; object. abulary generic term: selector operation scription: a selector operation modifies the state of an object but does not geit] \ "> 1 i analysis modelling conceptual vocabulary data analysis j analysis modelling a ^ j type: classification ! type: aggregate; o 1 vocabulary genen . . 1 1 description: a sel 1 term used 1 selector 1 property ~oncetual vocabulary ; operation, bject. c term: selector operation ector operation accesses the state of an object but does not change it. ] source evidence booch 1994 j>age88 henderson-sellers and edwards 1 994 page 5 1 4 operations classes further research notes: terminology conflict for operations, monarch! and puhr point to the need for placing operations. 3 representation terms for operation. source evidence booch 1 994 page 88, page 43 hendnerson-sellers & edwards 1994 page 5 1 , page 52 monarch! and puhr 1 992 "...while identifying classes, attributes and behavior has been fairly thoroughly covered, placing them has been largely neglected. . ." (p 4 1 ) figure 7: development process for model components maturity indicators and data analysis the data analysis stage is where component instances are analysed using a number of maturity indicators outlined in table 4. the purpose of this stage is to identify where there may be candidate projects based on the maturity of the field or unresolved issues. the products of this stage are a number of numerical figures based on instances, along with instances of the further research component. names, vocabulary areas, and vocabulary concepts support naming of project topics whereas maturity indicators along with further research instances help determine significance of project topics. maturity indicators are used to analyse instances of model components from the data gathering stage as a foundation for identification of candidate projects. in relating the level of maturity (low, high) to the quantified indicator (many, few) it must be remembered that these are somewhat subjective terms which depend on the size of the research surveying tool. to get some objective idea of the relationship between field maturity and the indicators would require either an expert system or statistical analysis based on historical data. this is outside the scope (and space) of this paper. as a rule of thumb, few is less than ten, and many is ten or more. 22 ajis vol. 6 no. 2 may 1999 table 4: maturity identification guidelines item literature source reference conceptual term representation term dependency homonym synonym concept absence identification in model <literature source> instance in axis, subdivision, & conceptual vocabulary component instances <term> instance in <generic term> instance in axis, subdivision, & conceptual vocabulary component instances <term used> instance in axis, subdivision, & conceptual vocabulary component instances subdivision dependency> instance or <vocabulary dependency> instance two component instances with differing <generic term> and differing <description entry> instances, have common <term used> instance for a number of <literature source> instances a component instance with a given <generic term> and associated <description entry> instance, has a number of <term used> instances for a number of <literature source> instances a <literature source> instance does not appear in an axis, subdivision, & conceptual vocabulary component instance maturity indication low few literature source references few conceptual terms many representation terms few dependencies many homonyms many synonyms many incidences of concept absence high many literature source references many conceptual terms few representation terms many dependencies few homonyms few synonyms few incidences of concept absence literature source reference: the literature source reference maturity indicator is derived from instances of axis, subdivision and conceptual vocabulary components. by counting the number of literature source references, an indication of the work done based on the vocabulary is quantified. the value is obtained by counting the instances of <literature source> in the model component's tdl specification, or counting the instances of <literature source> in the "source" column of the model component's table template specification. for example, in figure 7d) there are two literature source references. many literature source references indicate a high level of maturity, whereas few of these indicate a low level of maturity. for example, in figure 7d) the number of literature source references would suggest that research related to the concept of selector operations for analysis modelling may be immature. calculation of this maturity indicator may require removal of duplicates. the literature source reference maturity indicator is a typical indicator of field maturity. for example, monarch! and puhr (1992) talk about the work done in placement and identification of operations. wynekoop and russo (1997) discuss work done based on research methods used (action research, field experiment etc.) in studying software methodologies. conceptual term: the conceptual term maturity indicator is derived from instances of axis, subdivision, and conceptual vocabulary components. by quantifying the size of the real vocabulary, as opposed to apparent terminology, an indication of the knowledge established in the field is quantified. the value is obtained by counting the instances of <term> in instances of <generic term> for the model component's tdl specification, or counting instances of <term> in instances of <generic term> in the model component's table template specification. for example, figure 4a), b), and c) have three conceptual terms (operation, owns aggregation, identify class). many conceptual terms indicate a high level of maturity, whereas few of these indicate a low level of maturity. the conceptual term indicator points to the size of the real vocabulary in a subject. representation term: the representation term maturity indicator is derived from instances of axis, subdivision, and conceptual vocabulary components. by quantifying the size of the apparent terminology, an indication of the level of awareness and consensus between researchers based on excessive vocabulary is indicated. the value is obtained by counting instances of <term used> in the model component's tdl specification, or counting instances of <term used> in the 'term used" column of the model component's table template specification. for example, based on figure 4a), figure 6a), and figure 7d), there are seven representation terms. (operation, service, property, selector, operation, modifier, behavior). many representation terms indicate a low level of maturity, whereas few of these indicate a high level of maturity. the representation term indicator is one 23 ajis vol. 6 no. 2 may 1999 indicator that assess the volume of apparent vocabulary. note that limiting application of this indicator to one axis name, subdivision name, or vocabulary concept yields the same result as the synonym maturity indicator (see below). this indicator by itself is rarely useful and should be used in conjunction with other indicators. dependency: the dependency maturity indicator is derived from instances of subdivision and conceptual vocabulary components. by counting the dependencies, an indication of the knowledge areas in a subject is quantified. the value is obtained by counting instances of "[ type: <phrase>.] <term>;" in the model component's tdl specification, or counting instances of "[ type: <phrase>.] <term>;" in the model component's table template specification. for example, based on figure 4a), figure 6a), and figure 7d), there are five dependencies, (operation and object, selector operation and object, modifier operation and object, selector operation and operation, modifier operation and operation). many dependencies indicate a high level of maturity, whereas few of these indicate a low level of maturity. dependencies reflect the amount of knowledge building that has occurred in a subject. homonym: the homonym maturity indicator is derived from instance of axis, subdivision, and conceptual vocabulary components. by counting the number of homonyms, an indication of the level of consensus between researchers is quantified. the value is obtained by finding two instances of a <generic term> in two model component's tdl specifications and comparing the instances of <term used> that are equal in value, or by finding two instances of a <generic term> in two model component's table specifications and comparing the instances of <term used> in the 'term used" column that are equal in value. each pair of values counts as one homonym. for example, looking at the <term used> instances in figure 6a) and figure 4a), there is one homonym (operation). many homonyms indicate a low level of maturity, whereas few of these indicate a high level of maturity. this indicator by itself is the most effective for assessing the level of maturity. homonyms reflect the level of awareness between researchers and sometimes divergence and conflict in views about a subject. synonym: the synonym maturity indicator is derived from instances of axis, subdivision and conceptual vocabulary components. by counting the number of synonyms, an indication of the level of awareness between researchers is quantified. the value is obtained by counting the instances of <term used> for the same instance of <generic term> in the model components tdl specification, or by counting the instances of <term used> in the 'term used" column for the same <generic term> in the model component's table template specification. for example, figure 7d) has two synonyms (selector, property). synonyms reflect the level of awareness between researchers. in immature fields, researchers usually resort to introducing terminology that looks different, but may be the same as that of others. concept absence: the concept absence maturity indicator is derived from instances of axis, subdivision, and conceptual vocabulary components. by counting concept absence, an indication of the division between research and divergence of views is quantified. the value is obtained by comparing the instances of <literature source> in a model component's tdl specification to instances of <literature source> in instances of the literature source component that are deemed relevant, or by comparing the instances of <literature source> in the ""source" column of a model component's table template specification to instances of <literature source> in instances of the literature source component that are deemed relevant. for example, monarchi and puhr (1992) are not included in figure 7d) or figure 6a), creating two concept absences. many concept absences indicate a low level of maturity, whereas few of these indicate a high level of maturity. this indicator can converge with the literature source reference indicator, but sources must be relevant. determining relevance does rely on the judgement of the researcher. however, one rule of thumb is to use the instance of <title> for the literature source instance as well as evidence from the text. for example, booch (1994) and shlaer and mellor (1988) have "object-oriented" and "analysis" in their titles. both sources also discuss modelling. hence, they would be relevant to the conceptual vocabulary instance in figure 6a) due to its vocabulary area ("analysis modelling"). concept absence reflects primarily the division of knowledge in a subject with more and more work becoming increasingly fragmented, particularly with journal and conference sources. sometimes it can reflect a lack of work done in an area. however, this indicator by itself is not enough to assess maturity and other indicators should be used in conjunction with it. grouping and calculation of maturity indicators although some indicators are better than others, no single maturity indicator can point to the maturity of the filed by itself. hence, maturity indicators must be compared against each other to assess field maturity. for example, although a subject may have many instances of concept absence and representation terms. a high number of 24 ajis vol. 6 no. 2 may 1999 dependencies and conceptual terms, along with a low number of homonyms and synonyms really reflects an ordered knowledge building process. the consensus on views is indicated by consistent vocabulary (homonym, synonym, conceptual term and indicators), and more and more people are building on previous work (dependency indicator). a basic method is presented for obtaining and classifying values for maturity indicators suitable for analysis. although it is not necessary to rely on statistical methods, they can be applied to the values. numerous books on statistics can be found, for an introduction to the subject two sources are recommended (huff, 1954; moroney, 1965). the maturity indicators are only applied to axis, subdivision, and conceptual vocabulary instances. basic application of the maturity indicators is to sum the values for all instances. for example, using the instances in figure 3, figure 4, figure 6a), and figure 7d) the following values are obtained for the above maturity indicators: ten literature source references, eight conceptual terms, fourteen representation terms, nine dependencies, one homonym, nine concept absences, and for synonyms a mean of one point five, a median of two, and a mode of two. figure 8a) illustrates presentation of data analysis for these results. note that the synonym maturity indicator is one exception. it must be applied to each vocabulary concept, each axis name, and each subdivision name. to gain a single value for a number of instances, the mode, median, and mean calculations for the synonym maturity indicator are recommended. the values for the synonym maturity indicator are the values averaged, the number of instances are the number of samples (n). obtaining values for maturity indicators using all instances is not the only way to analyse the instances. typically, values for indicators are partitioned based on the kinds of model components and their relationship to each other. one method for partitioning values is hierarchical selection. figure 8 and figure 9 show some results of this method. values for indicators are partitioned based on the hierarchy of model components illustrated in figure 2. to demonstrate this, the instances in figure 3, figure 4, figure 6a), and figure 7d) are used as if they represent an entire research surveying tool. the literature source reference maturity indicator is used for obtaining values. indicators for all instances (the subject) a) indicator literature source references conceptual terms representation terms oepedenedcies homonym concept absences synonym value 10 8 14 9 1 9 m e a n : l . s , median:!, mode:2 b) all axes indicator li tera ture source reference (other indicators) value 4 y c) axis names axis name-* indicators! literature source refeneces (other indicators) stages 4 y components 0 z d) stages axis name representation term — » indicators! literature source refeneces (other indicators) phase 2 x stage 1 y act iv i ty 1 z figure 8: example data analysis for subject and axis instances a research surveying tool for a subject has a number of axes, subdivisions, and vocabulary areas (figure 1). values can be partitioned for all axes, all subdivisions, and all vocabulary areas. for example, there are four literature source references for all axes (figure 8b)), two literature sources for all subdivisions, and five literature source references for all vocabulary areas (figure 9a)). where applicable, other indicators can be used in the same table (e.g. figure 9a)). 25 ajis vol. 6 no. 2 may 1999 an axis has a name (generic term), the name has a number of representation terms (term used), a representation term is used in evidence from literature sources (figure 1). values can be obtained for each axis name and each representation term attached to an axis name. for example, using each axis name there are four literature sources for the stages axis, and zero literature source references for the components axis (figure 8c)). using each representation term attached to an axis name, the stages axis has two literature source references for the phases representation term, one for the stages representation term, and one for the activity term (figure 8d)). a subdivision has a name (generic term), the name has a number of representation terms (term used), a representation term is used in evidence from literature sources (figure 1). values can also be obtained for each subdivision name, and each representation term attached to a subdivision name. for example, using all subdivisions for an axis, there are two literature source references for the phases axis. using subdivision names there are two literature source references for the logical design subdivision. using representation terms attached to a subdivision name, the logical design subdivision has two literature sources for the logical design representation term. a) all vocabulary areas indicator literature source reference depdendency value 5 8 b) vocabulary areas vocabulary area-* indicators! literature source refeneces depdendency analysis modelling 5 6 analysis method process 1 2 c) analysis modelling vocabulary area vocabulary name-* indicators! literature source refeneces dependency operation 3 1 selector operation 2 2 d) analysis modell ing vocabulary area selector operation vocabulary name representation term-* indicators! literature source refeneces (other indicators) selector 1 x property 1 y figure 9: example data analysis for vocabulary area and vocabulary concepts an axis is composed of a number of subdivisions, a subdivision is used in a number of vocabulary areas, a vocabulary area has a number of vocabulary concepts (generic term), a vocabulary concept has a number of representation terms (term used), a representation term is used in evidence from literature sources (figure 1). values can be obtained for all subdivisions attached to an axis, each vocabulary area, each vocabulary concept attached to a vocabulary area, and each representation term attached to a vocabulary concept in a vocabulary area. for example, using each vocabulary area the analysis modelling vocabulary area has five literature source references, the method process vocabulary area has one literature source reference (figure 9b)). using each vocabulary concept attached to a vocabulary area, the operation vocabulary concept in the analysis modelling vocabulary area has three literature source references (figure 9c)). using representation terms attached to a vocabulary concept in a vocabulary area, the selector vocabulary concept in the analysis modelling vocabulary area has one literature source reference for the selector representation term, and one for the property representation term (figure 9d)). not all maturity indicators can be applied effectively at all levels for hierarchical selection. appendix e table 10 recommends which indicators can be applied at the given hierarchy levels. further research instances are used to capture any significant results a researcher wants to highlight. for example see the "notes" entry in figure 7e). analysis of a subject using the model and processes can indicate a high level of maturity in some places, and a low level of maturity in others. literature sources attached to component instances that indicate evidence of a low level of maturity are likely candidates for further examination. for example, (monarchi and puhr, 1992) is one such candidate (figure 7e)). further research instances can also 26 ajis vol. 6 no. 2 may 1999 specify any information found about unresolved issues. for example, the (monarch! and puhr, 1992) entry in figure 7e) discuss the lack of research in placement of classes, attributes and behavior. candidate issues identified by the researcher can also be included. the "notes:" entry in figure 5a) is one example. early in the research process, iteration from component formation to data analysis may result if a research surveying tool cannot provide adequate results. in these circumstances, use instances of representation terms (<term used>) as keywords and instances of dependencies (table 2, dependency column) to guide keyword combinations for literature acquisition. for example, based on figure 6a), figure 7d), and figure 7e), "identification and (selector or property) and (operation or service or method)" could be a search string entered into a bibliography search engine such as ovidweb current contents®. to assist formation of topics for candidate projects, the vocabulary can be used in combination with further research issues previously identified. for example, monarchi and puhr (1992) raise the issue of research in placement of classes, attributes and behavior. this can be extended into projects for placement of owns aggregation from the instance in figure 4b), or selector and modifier operations from the instances in figure 6a) and figure 7d). research surveying, if time permits, can result in a large number of candidate projects offered to other researchers in addition to the researcher doing the surveying, further assisting the task of project identification. if an application of results can be found (e.g fuggetta (1993)), or the size of the survey identifies a significant number of issues (e.g doroshenko (1994)), the survey tool can become the project for an honours student. for phd and masters students, such a project can be part of a thesis based on some issues identified using the model and above processes. vocabulary conflict can also be a foundation for a project (examples can be found in four sources (barki, et al., 1993; doroshenko, 1994; snyder, 1993; soley and stone, 1995)). analogical view of the development process the development process can be viewed as a software life cycle with modelling of a problem and evaluation of the proposed model to verify its usefulness for solving a problem. in this development process the model components are used to make a research surveying tool. literature acquisition and component formation for a subject are similar to domain analysis of a problem with the resulting model. data gathering and data analysis are analogous to an evaluation process to apply and evaluate the model. the surveying tool is evaluated to see if it generates useful results in the same way a software model is evaluated to see if it meets requirements. iteration occurs if the model components do not generate useful results. this is similar to iteration occurring in software development if requirements are not fulfilled. discussion dimension has much use in classifying software methodology aspects (see table 1). incorporation of these dimensions can reduce time in component formation, but some conflict in views may arise, e.g. livari (1995) has two dimensions for object-oriented analysis, compared to lowry, et al. (1994), who have software development methodologies as one dimension. one possible solution is to describe dimension as a pattern in terms of subdivisions and dependency (the example is based on henderson-sellers and edwards (1990)). a subdivision (phase) has classification dependency with a number of subdivisions below it (analysis, design). these subdivisions have aggregate dependency with subdivisions below themselves (user req specification, user req analysis; logical design, physical design). the axis is a projection, with the axis name being the top most subdivision (phase), with the classification dependency subdivisions attached to it (analysis, design etc). although this simplifies the model and still allows inclusion of dimensions from other fms, maturity indicators for data analysis are effected by such a change. specifically, there would be no calculations for the axis, and the value for dependencies would increase and possibly change the interpretation of the data. conclusion a model is presented to assist research surveying activities of researchers in software methodologies. knowledge acquisition of a subject is assisted by classifying it using the model components, resulting in a research surveying tool. this tool can assist project identification by subjecting it to processes that indicate the maturity level in an area, examining more literature in that area, capturing research issues identified in the literature, and using the issues as a basis for candidate projects. these projects can contribute to department research if supervisors offer relevant subjects for research surveying. the model components and processes reduce a student's dependence on supervisor knowledge about research surveying activities. some issues are still open for inquiry. firstly, the specification in doroshenko (1994) provides much more data for classifying aspects of software methodologies. how these specifications relate to research surveying activities is not discussed in this paper. secondly, although some papers in other is research areas suggest 27 ajis vol. 6 no. 2 may 1999 evidence of model components (e.g. classification dependency and subdivision in pervan (1994, pp 563 564), dimension and subdivision in berci (1996, p 61)), successful application of the model in these areas for research surveying or in general is unknown. thirdly, if the analogy is reasonably accurate, then automation of the model should be possible. this is similar to case technology support for developing software. fourthly, there are two fundamental risks in using this approach to research surveying. creation of a research surveying tool that is too large and contains data that is comprehensive but obsolete (an old phone book), or one that is tool small and contains data that is recent but trivial (a gossip magazine). this paper does not deal with management of these risks. fifthly, although the model can support and corroborate a research topic, it cannot provide the "initial bright idea". instances of model components must still be formed by the researcher in a given subject (the author doesn't know of a software model that can build itself)this requires imagination and creativity, both of which are a part of research work (research, 1997; whitehead, 1929). references backus, j. w. (1959) "the syntax and semantics of the proposed international algebraic language of the zurich acm-gamm conference", , international conference on information processing, paris, france, unesco, pp 125132. barki, h., rivard, s., andtalbot, j. (1993) "a keyword classification scheme for is research literature", mis quarterly, vol, no, pp 209 226. berci, b. j. (1996) "strategic information management vs. strategic business management", , australasian conference on information systems, hobart, australia, australian computer society inc, pp 57 68. beringer, d. (1994) "limits of seamlessness in object-oriented software development", , tools europe, , prentice-hall, pp 1 -11. booch, g. (1994) object-oriented analysis and design with applications, 2nd, redwood city, benjamin/cummings, curtis, b., kellner, m. i., andover, j. (1992) "process modelling", communications of the acm, vol 35, no 9, pp 75 90. dart, s. a., ellison, r. j., fieler, p. h., andhabermann, a. n. (1987) "software development environments", ieee computer, vol, no 11, pp 18 28. de champeaux, d. and faure, p. (1992) "a comparitive study of object-oriented analysis methods", journal of object-oriented programming, vol , no march/april, pp 21 32. doroshenko, e. e. (1994) common concepts in object-oriented methodologies, applied computing & mathematics, university of tasmania. doroshenko, e. e. (1996) "toward a language-graphic model for case tool construction: helping the corner store ot vendor", , australasian conference on information systems, hobart, australia, australian computer society inc, pp 161 -172. doroshenko, e. e. (1997) "a refined meta-model approach to education for the transition to object technology", , international conference on object-oriented information systems, brisbane australia, springer-verlag, pp 466 476. fernstrom, c, narfelt, k.-h., andohlsson, l. (1992) "software factory principles, architecture, and experiments", ieee software, vol 9, no 2, pp 36 44. fuggetta, a. (1993) "a classification of case technology", ieee computer, vol 26, no 12, pp 25 38. henderson-sellers, b. and edwards, j. m. (1990) "the object-oriented systems life cycle", communications of the acm, vol 33, no 9, pp 142-159. howard, k. and sharp, j. a. (1983) the management of a student research project, hampshire, gower, huff, d. (1954) how to lie with statistics,, london, victor gollancz ltd, livari, j. (1995) "object-orientation as structural, functional and behavioural modelling: a comparison of six methods for object-oriented analysis", information and software technology, vol 37, no 3, pp 155-163. korson, t. d. and vaishnavi, v. k. (1992) "managing emerging software technologies: a technology transfer framework", communications of the acm, vol 35, no 9, pp 101 111. leedy, p. d. (1993) practicl research: planning and design, 5th, new york, macmillan, lowry, g. r., godfrey, r., lear, f. a., andmorgan, g. w. (1994) "a 4th generation information systems development research framework",, 5th australasian conference on information systems, melbourne australia, department of information systems, monash university, australia, pp 221-236. mcchesney, i. r. (1995) "toward a classification scheme for software process modelling approaches", information and software technology, vol 37, no 7, pp 363 374. monarchi, d. e. and puhr, g. i. (1992) "a research typology for object-oriented analysis and design", communications of the acm, vol 35, no 9, pp 35-47. 28 ajis vol. 6 no. 2 may 1999 moroney, m. j. (1965) facts from figures, 2nd, london, penguin books, naur, p. (1963) "revised report on the algorithmic language algol", communications of the acm, vol 6, no l ,ppl-17. nissen, h. w., jeusfeld, m. a., jarke, m., zemanek, g. v., andhuber, h. (1996) "managing multiple requirements perspectives with metamodels", ieee software, vol 13, no 2, pp 37 47. omg, 0. m. g. (1996) object analysis & design facility rfp-1, farmingham ma, object management group. pervan, g. p. (1994) "the measurement of gss effectiveness: a meta-analysis of the literature and recommendations for future gss research", , hawaii international conference on systems sciences, maui, hawaii,, pp 562 571. research, o. f. (1997) research higher degrees handbook, , hobart, university of tasmania, roper, n. (1995) common concepts in structured process methodologies, applied computing & mathematics, university of tasmania. sethi, r. (1989) programming languages: concepts and constructs, , reading, ma, addison-wesley, shlaer, s. and mellor, s. j. (1988) object-oriented systems analysis: modelling the world in data., , englewood cliffs, new jersey, prentice-hall, snyder, a. (1993) "the essence of objects: concepts and terms", ieee software, vol 10, no 1, pp 31 42. soley, r. m. e. and stone, c. m. (1995) object management architecture guide, 3, new york, wiley & sons, steele, p. m. and han, j. (1996) "a layered architecture for describing information systems development methodologies", , australasian conference on information systems, hobart, australia, australian computer society, pp 677-688. whitehead, a. n. (1929) the aims of education, , new york, macmillan company, wynekoop, j. l. and russo, n. n. (1997) "studying system development methodologies: an examination of research methods", information systems journal, vol 7, no 1, pp 47 65. acknowledgments eugene doroshenko would like to thank his honours supervisor, ms felicity lear, phd supervisor, professor glenn lowry, the scolarships officer, vanessa folvig, the head of school, professor chris keen, and professor robert delbourgo, for their assistance at the university of tasmania. endnote® is a registered trademark of niles & associates, inc 29 ajis vol. 6 no. 2 may 1999 appendices appendix a component specifications <axis declaration ::= { <axis name> <description entry> <axis representation> } <axis name> ::= axis <generic term> <axis representation ::= axis representation terms> • <term> in <axis name> unique within <axis declaration> <subdivision declaration> ::= { <axis subdivision title> <subdivision dependency> subdivisions: { subdivision entry> } } <axis subdivision title> ::= <term> axis. • <term> in <axis subdivision title> there must exist <term> in <axis name> in <axis declaration> <subdivision dependency> ::= [ dependencies: { [ type: <phrase> . ] <term> ; } ] • <term> in <subdivision dependency> there must exist <term> in subdivision entry> in <subdivision declaration> <subdivision entry> ::= subdivision <generic term> <description entry> subdivision representation terms> • <term> in <generic term> in <subdivision entry> * <term> in <axis title> in <axis declaration> • <term> in <generic term> in <subdivision entry> unique within <subdivision declaration> <conceptual vocabulary declaration> ::= { <conceptual vocabulary area> <vocabulary dependency> { <vocabulary entry> } } <conceptual vocabulary area> ::= <title> conceptual vocabulary. <vocabulary dependency> ::= [ dependencies: { <conceptual vocabulary area> { [ type: <pbxase> ] <term> ; } } ] • (<conceptual vocabulary area> and <term>) in <vocabulary dependency> there must exist ( <term> in <generic term> and <conceptual vocabulary area>) in <conceptual vocabulary declaration> • <term> in <generic term> in <vocabulary entry> * <terrn> in <conceptual vocabulary dependency> <vocabulary entry> ::= vocabulary <generic term> <description entry> vocabulary <representation terms> 5 • <conceptual vocabulary area> (<term> in <generic term in <vocabulary entry>) unique <further research declaration> ::= { <further research area> <further research entry> <notes> } <further research entry> ::= issue evidence: { <evidence entry> } <further research area> ::= <term> { <term> } further research <notes> ::= notes: [ <phrase> ] • <term> in <further research title> there must exist (<term> in <generic term> in <subdivision entry>) or (<term> in <generic term> in <axis title>) or (<term> in <generic term> in <vocabulary entry>) <literature source declaration> ::= { <author declaration> } { <literature source> ; <literature source entry>. } <literature source> ::= <author last name> [ ((and i & ) <author last name>) i (et al)) ] <year> { <letter> } <literature source entry> ::= authors: { <author entry> ; } <source title> <source typo <year published> <publisher> evidence: { <evidence statemeno } ; <source typo ::= type: ( book; <book details>) i (journal article; <journal article details> ) i ( conference paper; <conference paper details>) i (other; <informal>) <author declaration> ::= { <author entry> ; } <author entry> ::= author: <author last name>, cauthor first namo [ <author middle name> ] <year published> ::= year: <year> ; <source titlo ::= title: <phrase> ; <publisher> ::= publisher: <phrase> ; <year> ::= <number> [ ad ibc ibce ]; • <literature sourco in <literature source declaration> unique within <literature source declaration> • <author entry> in <literature source entry> there must exist <author entry> in <author declaration> • <author last namo in <literature sourco there must exist <author last namo in <author entry> in <literature source entry> 30 ajis vol. 6 no. 2 may 1999 • <year> in <literature sourco there must exist <year> in <year published> in <literature source entry> universal terminals and non-terminals •^description entry> ::= '[' description: { <informal> i <evidence entry> } ']' <titie> ::= <term> { <term> } <generic term> ::= generic term: <term> <representation terms> ::= representation terms: [ <term used> { <evidence entry> } { , <term used> { <evidence entry> } } ]; <evidence entry> ::= source: <literature sourco evidence entry: { <evidence statement } ; <evidence statement ::= (<figure referenco i <quotation> i <page referenco i <phrase> i <informal>) [. i , ] • <term> in <title> there must exist <term> in subdivision entry> • ( <literature sourco and <evidence statement> ) in <evidence entry> there must exist (<literature sourco and <evidence statemeno in <literature source entry> ) in <literature source declaration> <term used> ::= <phraso i ? appendix b nb the table templates make use of some terminals and non-terminals found in appendix a. the literature source component does not have a table template specification axis component <axis narno <description entry> term used <term used> source { <literature sourco ; } evidence { <evidence statement ; ) subdivision component <axis subdivision titlo <subdivision dependency> subdivision <generic term> <description entry> term used <term used> source { <literature sourco ; } evidence { <evidence statement> ; } conceptual vocabulary component <conceptual vocabulary area> <vocabulary dependency vocabulary <generic term> <description entry> term used <term used> source { <literature source> ; } evidence { <evidence statemeno ; } further research component <further research area> <notes> source <literature sourco evidence <evidence statement> 31 ajis vol. 6 no. 2 may 1999 appendix c table 5: subdivision component and specification component item subdivision component axis link to subdivisions name interpretation dependency representation terms literature source for representation term evidence in literature source tdl subdivision declaration> <axis subdivision title> <generic term> in subdivision entry> <description entry> <subdivision dependency> <subdivision entry> (<term used> in representation terms>) <literature source> in <evidence entry> <evidence statement> in <evidence entry> table template subdivision component <axis subdivision title> <generic term> <description entry> <subdivision dependency> <term used> <literature source> <evidence statement> table 6: axis component and specification component item axis component name interpretation representation terms literature source for representation term evidence in literature source tdl <axis declaration> <axis name> <description entry> <axis representation> (<term used> in representation terms>) <literature source> in <evidence entry> <evidence statement> in <evidence entry> table template axis component <axis name> <description entry> <term used> <literature source> <evidence statement> 32 ajis vol. 6 no. 2 may 1999 table 7: conceptual vocabulary component and s component item conceptual vocabulary vocabulary area vocabulary concepts name for vocabulary concept interpretation for vocabulary concept representation terms literature source for representation term evidence in literature source dependency tdl <conceptual vocabulary declaration <conceptual vocabulary area> <vocabulary entry> <generic term> in <vocabulary entry> <description entry> in <vocabulary entry> <vocabulary entry> (<term used> in representation terms>) <literature source> in <evidence entry> <evidence statemeno in <evidence entry> <vocabulary dependency> pecification table template conceptual vocabulary component <conceptual vocabulary area> one instance per concept <generic term> <description entry> <term used> <literature source> <evidence statement> <vocabulary dependency> table 8: further research component and specification component item further research component further research component link to axis, subdivision, conceptual vocabulary area, vocabulary concepts) issue literature source evidence from literature source tdl <further research declaration> <further research area> <notes> <literature source> in <evidence entry> <evidence state ment> in <evidence entry> table template further research component <further research area> <notes> <literature source> <evidence statement> table 9: literature source component and specification component item literature source component author literature source identifier literature source link to authorfs) type literature source link to evidence extracts tdl <literature source declaration> <author declaration> <literature source entry> <literature source> <author entry> in <literature source entry> <source typo in <literature source entry> <evidence statemeno in <literature source entry> table template n/a n/a n/a n/a n/a n/a n/a appendix d the tdl is used to describe itself. non-terminals are enclosed by angle brackets (<a non-terminal>), terminals are written in bold face (a_terminal), production rules have a non-terminal name followed by "::=" and a series of non-terminals and terminals (<a non-terminal> ::= <a non-terminal> a_tenninal). braces represent zero or more options for terminals and non-terminals ({ ajterminal <a non-terminal> }), brackets represent an optional item ([ <a non-terminal> ]) parentheses are used to group items ((<a non-terminal> a_terminal)), a vertical bar represents a choice between the items (<a non-terminal> i a_tenninal). reserved symbols used as terminals are enclosed by single quotes ('[' or ')'). relationships between instances of non-terminals can exist, called semantic statements (see table below). each semantic statement is preceded by a "•". an element is a 33 ajis vol. 6 no. 2 may 1999 valid instance of a terminal/non-terminal sequence (e.g. "an_identifier" is an element of <identifier>, where <identifier> ::= <letter> { <letter> i <digit> } ). semantic statement existence semantic (<existence semantio) uniqueness semantic (<uniqueness semantio) alias semantic (<alias semantio) cardinality semantic (<cardinality semantio) equality semantic (value) (<equality semantio) equality semantic (number) (<equality semantio) enclosure semantic (<enclosure semantio) premise clause (<premise clauso) purpose dependency between two elements uniqueness of elements duplicate naming of same element number of instances between two elements value equality between two elements number equality between two elements limits application of uniqueness contingent application of semantic statement example application type identifier in a variable declaration global variable identifier in a program variable passed as actual parameter a type identifier & variable identifiers type identifier and variable identifier number of type and variable identifiers local variable in procedure declaration private keyword for class member <text definition statement> ::= { <production rule> i <semantic statemeno } } <production rule> ::= <non-terminal name> '::=' <non-terminal structuro <non-terminal structuro ::= { '{' <production rule item> '}' i'[' <production rule item> ']' i'(' <production rule item> ')' i ( <production rule item> [ t <production rule item> ]) } <production rule item> ::= <non-terminal structuro i <non-terminal namo i <terminal namo i <notation definition statement i <diagram description> <non-terminal namo ::= '<' <plain character> { <plain character> } '>' <terminal namo ::= { ('" <reserved symbol>'") i <bold character> } < semantic statement ::= • <semantic typo { <logic operator> <semantic typo } . <semantic typo ::= [ <premise clauso ] (<existence semantio i <uniqueness semantio i <alias semantio i <equality semantio i <enclosure semantio ) <uniqueness semantio ::= <semantic referenco unique [ within <semantic referenco ] <existence semantio ::= <semantic identification> there must exist <semantic identification> <alias semantio ::= <semantic referenco alias for <semantic reference> <cardinality semantio ::= for <cardinality> { <semantic referenco } there are <cardinality> { <semantic referenco } <equality semantio ::= ['(' <bound> ')' ] <semantic identification> <operator> ['(' <bound> ')' ] <semantic identification> <enclosure semantio ::= <semantic identification> encloses <semantic identification> <operator> ::= '=' i '<' iv i < i > i * <premise clauso ::= if <semantic identification> i <semantic typo => <semantic identification> ::= <semantic referenco { <logic operator> <semantic reference> } <semantic referenco ::= (<semantic element> {in <semantic element> } ) ! ( ' ( ' <semantic identificatioro ')' ) <semantic element> ::= { <terminal namo i <non-terminal namo i <phrase> i <instance identifier } ['(' <number> ')' ] <instance identifier>::= " { <bold character> i <space> } " <reserved symbol> ::='{' i'}' i '<' i v i t i'(' i')' i '"=' i'[' i t i'" i '=>' i '•' <cardinality> ::='(' <bound> : <bound>')' <bound> ::= <capital letter> i <number> • <non-terminal name> in <production rule> unique within <text specification definition statement> 34 ajis vol. 6 no. 2 may 1999 appendix e table 10: partitioning of values for hierarchical selection indicators -* level j> axis axis name axis rep. term subdivision subdivision name subdivision rep. term vocabulary areas vocabulary area name vocabulary concept vocabulary rep. term literature source reference x x x x x x x x x x conceptua itenn x x x x x x x rep. term x x: same as synonym x: same as synonym x: same as synonym depdend. x x x x x homonym x x x x synonym x x x x x x x concept absence x x x x x 35 a novel user interface for information exploration and visualization max m. north, sarah m. north, and nazir a. warsi human-computer interaction group clark atlanta university, atlanta, ga 30314, (404) 880-6942 max@acm.org, snorth@cc.gatech.edu, warsi@cau.auc.edu abstract the primary objective of this study is to investigate the design, implementation, and evaluation of a graphical user interface prototype, called infovis, for direct manipulation of databases. this prototype information exploration and visualization interface allows users to explore a database with graphical widgets such as a novel dynamic slider. specifically, the infovis interface enables the user to search a database without the need to create or formulate complex syntactical query statements. the dynamic slider and other widgets are utilized to assist users in mental visualization and representation of objects and actions. a pilot empirical study was conducted to assess the effectiveness of the infovis interface prototype. the primary results suggest that subjects who used the infovis interface performed at a higher level than their counterparts who did not use infovis interface. introduction generally, information exploration of databases is performed using query languages. most query languages such as structured query language (sql), are based on relational algebra or calculus. while relational algebra and calculus provide a powerful means to formulate and specify queries, their usage is an extremely tedious and complex task (cha 1990), (mayhew 1992), (north et al. 1993), (nowell et al 1993), (zloof 1975) for computer users, especially for naive users. in addition, query languages are redundant in the sense that the same query may be expressed in many different ways (kim et al 1988). in fact, empirical research indicates a wide variation in response times in the implementation of different query languages. this research argues that providing a graphical visualization of the databases, queries, and search results will empower users with the complex task of information exploration (ahlberg et al 1992), (cha 1990), (johnson 1992), (north et al 1993), (nowell et al 1993), (sarker et al 1992). to provide and enhance graphical visualization, a new multiple attribute presentation widget called dynamic slider is introduced (north et al 1993). the dynamic slider enables the user to present multiple value ranges rather than a single or anchored value to minimum or maximum points. dynamic slider as a novel multi attributes widget traditionally, sliders have been used as a metaphor to assist the user in entering either a single value or a single range value anchored to minimum or maximum points of a field. the concept of dynamic queries (ahlberg et al 1992) has been extended so that the users can implicitly construct complex queries by utilizing visual and graphical techniques and tools such as dynamic sliders. by utilizing dynamic sliders, infovis interface is able to represent dynamic range(s) rather than a single or anchored value to provide the mechanism for assisting the user in formulating more complex queries. the use of novel dynamic sliders widget distinguishes the infovis interface from other database interfaces. in infovis, the query is represented primary by number of dynamic sliders widgets and other widgets. a dynamic slider consist of a name, size, type, global relationship, minimum and maximum values, a slider with two drag boxes, and ranges. figure 1 denotes only a simple dynamic slider. a dynamic slider (figure 1-a) is capable of mimicking the simple or traditional slider by just manipulating (clicking on the slider bar) one of the drag boxes at a time. for instance, by manipulating the top drag box values from the minimum point to the desired point is represented (figure 1-b). by manipulating the bottom drag box values from the maximum point to the desired point is represented (figure 1-c). representing range of values that is not anchored to minimum or maximum points could be accomplished by manipulating both drag boxes to desired points. figure 1-d demonstrates this representation. a dynamic slider was built with the user's ease of use in mind. although, it may be simply understood and operated by the user, it is a very powerful widget. because of its property of being a basic block of widget, it may be included in many interfaces to revise the static nature of interface to a more dynamic nature thus provide more powerful and efficient interface. dynamic sliders widget within frames and clusters frames were introduced in the mid-1970's by minskey to present knowledge. since then, the minskey's frame concept have been extensively utilized by other applications, such as databases. simply, frames are structured forms of attributes (gupta et al 1991), (leong et al 1989). one of the advantages of the frame is that users do not need 52 ajis prior knowledge of the attributes for an object (gupta et al 1991), (koh et al 1989). in essence, frames reduce the burden of typing and eliminate need for user knowledge of acceptable attribute values. frames also provide several other advantages. the default values of attributes may be explicitly stated (koh et al 1989); however it may be possible to set a default for other query interface (e.g., sql) but a user has to memorize the specific of the default. frames also provide for consistent placement of attributes. this visual consistency should ease the task of understanding sets of attributes. on the contrary, sql or natural language does not enforce a consistent ordering of a. b. c. d. min : max 0 100 0 100 0 100 0 100 dynamic slider [default and] f3 _i' t=t ' c31 |d , ^^^ ranges. ..[default or] 0 « 1 47 37 100 | 60 | j h ~l j figure 1. a simple dynamic slider. attributes. finally, frames can be graphically and visually extended to allow for building more complex queries. frames with all of the advantages appears to be static and bounded to the specified attributes values thus allowing the user to set attributes that are totally depended on the widgets capabilities used within a frame. the dynamic slider utilized within the frame extends the power and characteristics of the frame from a static state to a dynamic state. this new dynamic state of the frame, called dynamic frame allows for more efficient use of the frames and possibly combines several static frames to one dynamic and more general specifications and constrains of the attribute values. clusters have been introduced and successfully utilized in v1msys query interface for image databases (klinger et al 1993). klinger, et al, define the clusters as follows: "clusters are abstract query structures which encapsulate any combination of query objects. a cluster is simply a box with a user-provided natural language label." in general, clusters possess the following characteristics and can be: (i) linked to other clusters, or frames; (ii) expanded to reveal their contents; (iii) modified for customization; (iv) moved around the screen; and (v) retrieved, saved, grouped, renamed, or removed. clusters reduce the burden on the user for manipulating groups of attributes. specifically, clusters reduce any group of attributes into a single unit with a customized natural language label, such as: "aircraft query base on range of external fuel"; or "aircraft' performance at high altitude." since the label is customized as chosen by the user, it will provide a better cue for recalling and constructing more complex query rather than remembering and interacting with several frames and set of attributes at a time. clusters also provide a mechanism for the user to graphically manipulate while forming queries. moreover, the small size of the clusters allow more manageable and meaningful information such as frames and other clusters to be displayed on the same screen (klinger 1993). clusters contribute extensively to the domain of module reusability. clusters may be retrieved and combined together to form a new query. these characteristics of clusters increase the efficiency of formulating especially complex queries. finally, clusters are linked via a specific attribute. this limitation will enhance the user's understanding by simplicity of the link element for more detail information of clusters the reader may refer to the work of klinger, et al at uist '1993 (klinger et al 1993). clusters power also may be extended by the dynamic nature of the dynamic frames it represents. in other words, clusters inherent the properties of the frame or frames that are within them, called dynamic clusters, and in turn frames inherent the properties of dynamic sliders within themselves. this hybrid architecture gives birth to the new interface for exploration and visualization of information. the basic block of the dynamic sliders empowers the frames and clusters which empowers the user for facing the challenges of graphically forming complex queries with a very low cognitive load. the infovis capabilities and characteristics several attempts to use direct manipulation for databases have been made. examples include the user friendly query language picasso (kim et al 1988) and another product, query-by-example (zloof 1975). although these approaches are powerful, they do not provide visual presentation of information and actions. the infovis interface provides a more powerful interface by implementing the following guidelines: (i) continuous graphical representation of database, query, and information exploration outcome; (ii) visible range of the object by utilizing september 1994 53 dynamic sliders and other widgets (ahlberg et al 1992); and (iii) immediate feedback as the user physically manipulates the sliders, selection buttons, etc. specifically, the intbvis interface possesses the following capabilities and characteristics: (i) objects and graphical widgets of interest are continuously visible to the user (beard et al 1991), (cha 1990); (ii) the outcome of the information exploration is produced by physical actions such as manipulation of the dynamic sliders and other widgets rather than utilizing the complex query syntax (ahlberg et al 1992); (iii) the provision for incremental, rapid, and reversible actions that will be immediately displayed for users; and (iv) a minimal learning curve to aid naive as well as expert users in the exploration of information. in essence, the infovis interface enables the users to visualize and explore information similar to human cognitive information processing using minimal cognitive load (chimera 1992). in addition, the infovis interface enables the user to see in one view objects, actions, and results, and infovis interface assists the user in extracting the meaning and relationship of objects (sarkar et al 1992). the infovis interface capabilities include the following (ahlberg et al 1992): i. basic data manipulation consists of the select, update, delete, and insert operations. select operation the data retrieval operation that specifies the method of selecting tuples of relation(s) and constrained by predicates. update operation the operation modifies one or more records in a specified relation and consideration of predicates. delete operation this operation provides the deletion of one or more records from relation constrained by predicates. insert operation the insert operation allows the user to insert a new tuple into a specified relation. ii. basic data retrieval provides three basic operations, selection, join, and projection of multiple tuples queries. ie. condition specification supports the following boolean and comparison operators utilizing the dynamic sliders widget: and, or, not, =, not =, > , 3 , <, and £ to allow the formulation of more complex predicates. iv. arithmetic and aggregate operators support the following arithmetic operators and functions: average, minimum, maximum, sum, and occurrence count. furthermore, the following general aspects are included: i. multiple constraints upon single attributes such as the predicate value between a and b. ii. implicit conjunctive form of predicates. by default, the specified predicates are assumed to be of a conjunctive nature (and) unless otherwise indicated by the user such as disjunctive connection (or) (kim et al 1988), (nowell et al 1993). hi. set operations such as in, contains, set difference, set intersection, and set union. the infovis interface at work the user is presented by the graphical meta structure of the database. figure 2 denotes an example of aircraft database which is comprised of six databases in this particular case. the graphical meta structure at this time is defined by the database designer and an administrator. aircraft role performance specification avionics |  weapons  | figure 2. initial graphical meta structure of the aircraft database. 54 ajis by clicking on the middle point, shaded square, infovis interface will reveal the basic structures and an example of each database. the databases are linked via one or more attributes. if the user is interested in only individual or several databases then the user may click on any desired box on the screen to activate and reveal the structure and sample data. this mechanism also allows the user to select the desired database or databases for direct manipulation. this global view of the database reduces the mental load on the user and allows the user to view the entire databases and their relationships in a general form first and gradually move to more detail as desired. at this point, each database or combination of databases may be selected (see figure 3). name:  boeing ec­137d/e sentry manufactured by:  boeing code:  bec137des strategic recomaisance specialized code:  bec1370es performance all­weather capability airspeed:  250mph­mach1 ceiling:  20,80040,000ft range:  3,000 miles plus code:  bec137des specification usa type:  airborne early­warning and c3 aircraft powerplant:  four 9525­kg thrust pratt&vttittney tf33­1brt08a turbofan engines weights:  maximum take­off 147418kg dimensions:  span 44.42m; length 46.61 m might 46.81 m;vwngarea  283.35m" code:  bec137des weapons none code: bec137des electronic counter measures electronic support measures search radar code:  bec137des figure 3. the global view of the aircraft database structure and sample data. for simplicity, the query part of the infovis interface is introduced through examples on the following tuple: aircraft (service ceiling, take-off run, speed at high altitude, range of external fuel, rate of climb at sea level). where the aircraft tuple specifies the service ceiling, take-off run, speed at high altitude, range of external fuel, and rate of climb at sea level (table i). the query frame (see figure 4) consist of structure of the database, such as name, size, type, etc. each attribute is aircraft service ceiling [ft] take-off run [ft] speed at high altitude [mach] range/external fuel [km] rate of climb sea level [ft/min] f/a-1 8 hornet f-1 4 tomcat f-1 5 eagle f-1 6 fighting falcon f-4b phantom mig-23 flogger mig-25 foxbat-a saab ja37 viggen tornado f.mk2 50,000 50,000 60,000 50,000 62,000 61,000 80,000 60,000 50,000 1,400 1.300 900 1,200 4,390 2.950 4,525 1,310 2,500 1.8 2.34 2.5 2.0 2.25 2.35 2.83 2.1 2.16 3.706 3.220 4.500 3.800 3.700 2,500 3,000 3.000 4,800 40.000 30,000 50,000 40,000 28,000 30,000 40,950 40,000 30,000 september 1994 table 1. aircraft tuple. 55 linked with a dynamic slider which specifies single range, dynamic range, or ranges in general. the sliders are placed at the same line with attributes to minimize the users visual tracking. by clicking on the attribute select box the user may choose (i) the attribute(s) selected for imposing constraints; and (ii) the attribute(s) selected for outcome output of the query window. the other novel idea in infovis interface project was to allow the user to specify the order of the relationship by simply entering a numerical ordering number in the global relation box and to select the kind of relationship of the attributes as desired. by default the attributes of the databases are connected together by conjunctive predicate (and). if the query needs to be connected by disjunctive predicate (or), the user may click on the default box to change the and to or. the user is provided minimum and maximum values of each attribute. minimum and maximum values allow the user to extract meaning from the collective data in the database and assists the user in building queries. it also reduces the cognitive load of the user by providing ranges. at this point the user may manipulate the dynamic sliders to set and lock desired range or ranges of values of each attributes. after the range or ranges are selected it may be locked and then the predicates "and" or "or" (user's perspective) to the next set of ranges as desired by the user. this dynamic mechanism enables the users to build a more complex query without being worried about sql or other query languages. figure 4 shows an example of aircraft database infouii reunion  bynamfc slider field­name  size type  [default and] min:man  [default  and] 20  charaircraft q  service ceiling  7  nfft h take­off run  5  wft b speed at high altitude  3  wmach q  rangelekternalfuel  5  nfcm •  rate climwsea level  6 mmflvlin ranges... [befault or] q | | | 51  i  i  i |300  |1500  | i or i aircraft query: select aircraft database and display all the items that meet the following condition: all the airerafts with take-off run from 900 to 1500 ft or 2000 to 3000 ft; and speed at high altitude greater than 2.4 mach or between 2.1 and 2.34 mach; and rate of climb/sea level less than 40,000 ft/min or greater than 40,950 ft/min. i  help  ml  insert «| delete  m| update query print  •[organizem return dynamic outcome of the aircraf t query 1 aircraft f-14 tomcat f-15 eagle tornado f.nk2 take-off run 1,300 ft 900 ft 2,500 ft speed at high altitude 2.34 mach 2.5 mach 2.16 mach rate climb/sea level 30,000 ft/min 50,000 ft/min 30,000 ft/nin figure 4. the query dynamic frame of the aircraft database. direct manipulation of aircraft database that responds to the following query: select aircraft database and display all the items that meet the following condition: all the aircraft with take-off run from 900 to 1500 feet or 2000 to 3000 feet; and speed at high altitude greater than 2.4 mach or between 2.1 and 2.34 mach; and rate of climb at sea level less than 40,000 feet/minute or greater than 40,950 feet/minute. the experiment this experiment compared two different interface for query building for databases. the first interface was infovis that was just briefly described above. the other interface was sql, the traditional query building interface. the major hypotheses was that, there would be a significant difference between the performance of subjects who use infovis interface and the subjects who use the traditional sql interface. the independent variables in this experiment were the interfaces that subjects utilized to build queries: (i) infovis interface; and (ii) sql interface. the dependent variables were: (i) time to build query in order to get the desired output; and (ii) usefulness, effectiveness, learnability, and attitude (mayhew 1992). the prior data collected from the users revealed that users' 56 ajis skills vary from novice to expert with novice users dominating the population. the experimental subjects were twenty individuals (n=20) who were randomly selected for this study. ten subjects were placed in the experimental group and ten subjects were placed in the control group. a typical subject was 18-23 year-old (76%), and female (72%). initially, a prototype infovis interface was designed, developed, tested, and implemented on a spare sunworkstation™. each group was given an overview of experimental procedures and a short instructional period, primary to acquaint users with the domain subject and interfaces. the subjects from the experimental and control groups were presented with ten different queries to perform on aircraft database. these queries started with simple query and gradually became more complex. for instance, subjects were given the following queries: find aircraft(s) that have service ceiling of 60,000 feet and above. find aircrqfi(s) that have speed at high altitude of above mach 2.16 or equal; and take off run of1,300 feet; or rate of climb at sea level above 40,000 feet. find aircraft(s) that have range of the external fuel between 3700 km and 4,800 km. find the performance ofmig-25 foxbat-a. preliminary evaluation evaluation of the prototype system is critical to ensure the quality of the final product. to determine the desirable characteristics and features of the infovis prototype, a comprehensive survey was conducted. prototype development and its evaluation were tightly interleaved. each prototype was evaluated in realistic settings, and the outcomes was accommodated in the design of the next prototype, thus ensuring continuous quality improvement during each stage of system development. in summary, to ensure the usefulness, effectiveness, learnability, and attitude of the infovis interface prototype, several usability tests for each intermediate prototype design was conducted (mayhew 1992). preliminary results the data were subjected to appropriate statistical procedures. these procedures included a measure of central tendency, and the t-test. the result of the t-test indicated that there exists a significant difference between the performance of the experimental group and the control group (t=2.69, df=18, p<0.05), (table 2). control group (sql) experimental group (infovis) t df n mean s.d. 10 266.20 96.71 10 170.01 58.93 2.69 18 table 2. means, variance, standard deviations, degree of freedom, and t-test of subjects in experimental and control group. although the preliminary results of this research indicated that the subjects in experimental group performed at higher levels than subjects in the control group, a larger sample is recommended for future research. figure 5 shows the mean time to complete each task utilizing sql and infovis interfaces. in general, figure 5 shows the superior performance of the infovis interface over sql interface for each task that was performed by experimental and control groups. september 1994 57 tiiie <sec*ndx> 588 4 5 6 task ib sql infouis its 184 162 113 235 139 161 118 238 166 235 133 289 184 393 251 335 234 439 258 figure 5. mean time to complete each task utilizing sql and infovis interfaces. conclusion although we have built several prototypes to test out the ideas, the work is still in its early stages. infovis interface is an ongoing project with clark atlanta university's human-computer interaction group. the preliminary results of the pilot study (t=2.69, df=18, p<0.05) so far are very encouraging, and users' feedback is positive. using twenty subjects, several testings of the interface have been performed, and the infovis interface is now being tested with larger populations and more complex queries. further prototyping and testing will improve the overall performance of infovis interface. finally, the infovis interface is an operational prototype though functionally quite a simple system. in other words, the preliminary results so far are very encouraging, and users' feedback has been positive. in essence, the empirical results demonstrate that infovis interface appears to be a promising approach. discussion and future research the hypotheses that there would be a significant difference between the infovis interface and the traditional query procedure was confirmed. thus, it appears that infovis interface provides much better and faster learning and building queries for information exploration. of course, this is only a prototype of infovis interface, and many significant problems must be resolved. for instance, graphical meta structures provided to the user at this time is build by database designer and administrator, further work must be undertaken to allow the user to build this initial stage of the interface. also provisions must be provided for additional customization capabilities for the user. at this point the number of links between the databases is limited to one link. additional investigation is needed to provide the user with the mechanism for multiple linkages. in addition, as the system is used over time, many dynamic frames and dynamic clusters will be stored in the library. libraries may be tagged private or public depending on the person who creates it. eventually, the number of dynamic frames and dynamic clusters in libraries either private or public will increase and re-using these stored dynamic frames and dynamic clusters will become increasingly difficult (klinger et al 1993). also, each dynamic frames or dynamic clusters may have different versions with different attribute sets that may add to the problem at hand. an object (refers here to dynamic frames and dynamic clusters) management interface will be needed to assist users in manipulation of objects in the libraries. the dynamic slider concepts have been applied to the relational database at this time. further research is necessary to find the effect of dynamic frames and dynamic clusters on other type of databases, such as hierarchy and network databases. finally, it is desirable to compare the infovis interface with two or more graphical interfaces to find out if there are any significant differences among their performance. 58 ajis acknowledgement this project is sponsored by boeing computer services and partially supported by the u.s. army center of excellence in information science (contract number daal03-92-6-0377) and david and lucile packard foundation. the views contained in this document are those of the authors and should not be interpreted as representing the official policies of the u.s. government, either expressed or implied. references ahlberg, c, williamson, c. and shneiderman, b. (1992) "dynamic queries for information exploration: an implementation and evaluation", proceedings of acm chi'92 human factors in computing systems conference, pp 619-626. beard, d. and walker, j. (1991) "navigational techniques to improve the display of large two-dimensional spaces", behaviour & information technology, vol 9, pp 451 -466. cha, s. (1990) "kaleidoscope: a cooperative menu-guided query interface (sql version)", proceedings of ieee artificial intelligence applications, pp 304-310. chimera, r. (1992) "value bars: an information visualization and navigation tool for multi-attribute listings", proceedings of acm chi'92 human factors in computing systems conference, pp 293-294. gupta, a., weymouth, t. and jain, r. (1991) "semantic queries with pictures: the vimsys model", proceedings of the 17th international conference on very large data bases. johnson, b. (1992) "treeviz: trcemap visualization of hierarchically structured information", proceedings of acm chi'92 human factors in computing systems conference, pp 369-370. kim, h. j., korth, h. f. and silberschatz, a. (1988) "picasso: a graphical query language", softwarepractice and experience, vol 18, pp 169-203. klinger, j., swanberg, d. and jain, r. (1993) "concept clustering in a query interface to an image database", proceedings of uist '93, pp 1 1 2 1 . koh, t. and chua, t. (1989) "on the design of a frame-based hypermedia system", hypertext d. leong, s., sam, s. and narasimhalu, d. (1989) "towards a visual language for an object-oriented multi-media database system", visual database systems, north holland publishing. mayhew, d. j. (1992) principles and guidelines in software user interface design, new jersey: prentic-hall, inc., a simon & schuster company, englewood cliffs. minsky, (1988) the society of mind, simon & schuster. north, m. m. and north, s. m. (1993) "sarah: an information exploration and visualization interface for direct manipulation of databases", proceedings of graphics interface '93 conference, pp 163. nowell, l. t., hix, d. and labow, e. d. (1993) "query composition: why does it have to be so hard? ", proceedings of east-west international conference on human computer interaction, pp 226-241. sarkar, m. and brown, m. (1992) "graphical fisheye views of graphs", proceedings of acm-sigchi'92 human factors in computing systems conference, pp 83-91. zloof, m. m. (1975) "query-by example", proceedings of national computer conference, afips press, pp 431-437. september 1994 59 ajis vol. 7, no 2 may 2000 cursor control device characteristics james g. phillips' and thomas j. triggs' 'psychology department, monash university, clayton vic australia, email: james.g.phillips@sci.monash.edu.au abstract an analysis of cursor positioning may provide guidelines for improvements in cursor control. four experiments addressed efficiency of cursor control devices (mouse, digitising pen, accupoint, trackball). participants moved a cursor leftwards, upwards or rightwards, positioning it in large or small targets situated in near or far space on the computer screen. cursor coordinates were sampled every 5 ms. the number of submovements and the proportion of time spent in deceleration were analysed. participants could not plan movements controlled by an accupoint. cursor trajectories were more variable in near space for detachable manipulanda due to potential cartesian coordinate system incompatibilities. key words: cursor, positioning, kinematics, mouse, accupoint, trackball, pen introduction cursor control devices differ in their intuitive appeal, their directness, and the demands they make on space and posture (baber, 1997). although the mouse is commonly used in personal computers, recent developments in laptop computers and personal organisers have led to an increase in the use of alternative cursor control devices, and a reemergence of choice (dix, finlay, abowd, & beale, 1998), with concerns that good ergonomic practice may be sacrificed in some instances for compactness. there is thus a need to characterise the properties of cursor control devices when positioning cursors upon computer screens. a better understanding of the nature of such positioning movements may provide informed choice as to the appropriate cursor control device for a specific application, and/or assist in the development of real-time algorithms that can predict movement endpoints. pointing devices are typically assumed to be adequately described by pitts' (1954) law, which describes the rate of gain of movement precision with longer movement durations. however, pitts' law is a simplification, rather than an explanation of the underlying mechanisms (mackenzie, 1992). indeed, longer durations for movements of greater precision may be due to a greater need: 1) to overcome the variation in the initial impulse; 2) for terminal visual guidance; 3) for successive corrective responses; 4) to overcome incompatiblities of display and control (wickens, gordon, & liu, 1998). and while pitts' law holds for a variety of tasks (e.g. langolf, chaffin, & foulke, 1976), it does not apply to less precise movements (can & hoffman, 1988). nor does pitts' law provide guidelines for any interventions to improve positioning movements (mackenzie, 1992), because it does not address the underlying mechanisms responsible for the coordination of movement (mackenzie, 1992), thereby overlooking the rich potential associated with current technology. since cursor coordinates are monitored in real time there is the potential to reduce the duration of positioning movements if the intended endpoint can be predicted effectively (murata, 1995). there has only been a limited application of kinematic techniques to address the underlying nature of cursor positioning movements using a restricted number of cursor control devices. walker, meyer and smelcer (1993) examined the number of submovements required when a mouse was used to move a cursor over different distances to rectangular targets of varying horizontal and vertical dimensions. walker et al. found the number of submovements increased when movements were made to targets requiring greater precision. subsequently, douglas and mithal (1997) addressed the efficacy of mouse and isometric joystick (trackpoint). they reported that the trackpoint was less efficient, there being an appreciable variability in the trajectories produced by this device. they attributed the poorer performance to high frequency noise within the device (removable by filtering), and oscillations due to operator tremor. and while douglas and mithal thought tremor was a contributing factor, there was little evidence for the rhythmic motions characteristic of tremor, indicating other factors are likely. indeed increased variability in cursor trajectories might reflect an increased need for visually based corrective responses. unfortunately, the proportion of time spent in terminal guidance was not considered, and so it was difficult to address such issues. however, it may be possible to answer these issues by a judicious comparison of cursor control devices. there are a number of cursor control devices available (dix et al., 1998), which vary in their efficacy (mackenzie, 1992). differences in their efficacy may reflect their mode of operation. devices such as isometric joysticks and trackballs afford fewer cues to their operation, while the mouse and digitising pen are somewhat more intuitive and direct (dix et al., 1998). in the present paper we summarise 4 of our studies in which we used kinematic techniques to characterise properties of cursor control devices. we report the impact of movement distance and target size upon cursor trajectories controlled by mouse, digitising pen, accupoint, and trackball. since direction of motion has been observed to affect positioning time (trankle & deutschmann, 1991), we compared leftwards diagonal, vertical and rightwards diagonal movements. as geometrical factors have had an impact upon cursor positioning times (walker et al., 1993), 115 ajis vol. 7, no 2 may 2000 circular targets were used to control target approach angle. kinematic analysis of cursor positioning movements can document: 1) the variability of the accelerative phase of movement; 2) the duration of the decelerative phase of movement; 3) the numbers of submovements and accelerative/decelerative impulses. this analysis should provide insights into relative strengths and weaknesses associated with each cursor control device. method twelve right handed adults participated in each experiment. the experiments were performed using: 1) microsoft mouse and a 486pc; 2) a digitising pen, wacom sd420 graphics tablet and 486pc; 3) an accupoint on a toshiba pentium laptop computer; 4) the trackball on a mitac 486 laptop computer. in each case the computer sampled cursor coordinates every 5ms. participants sat in the midline of the computer, viewing the screen from a distance of 40cm. participants used the cursor control device to move a cursor from a home position at the middle of the bottom of the screen, to a target of either 6 or 12 mm diameter, that could be situated 5.35cm or 10.7cm away. the target could appear in the vertical midline of the screen, directly above the home target, or could be situated 30 degrees to the right or 30 degrees to the left of the midline. these combinations of target sizes (big, small), distances (near, far) and directions (leftwards, middle, rightwards) produced a 2x2x3 design. participants were instructed to position the cursor in the home position upon the computer screen. a target (either big or small) was then presented in one of six possible randomly determined locations. trials terminated after the cursor entered the target. after a block of 24 practice trials, there were 5 blocks of 24 experimental trials, providing 10 trials for each combination of target size, distance, and direction. kinematic indices were calculated for each trial, and then averaged for each condition. kinematic features were determined along the main axis of movement (i.e. in the direction of the target). high frequency noise affecting the coordinates generated by the devices were filtered out. the coordinates of the resulting displacement function were then differentiated twice to produce velocity and acceleration functions. reaction time was defined as the duration of time from initial button click to onset of cursor movement. movement time was defined as the interval of time from onset of cursor movement to the point in time at which maximum displacement occurred. the point of maximum extent along the main axis of movement was used to determine the extent to which people overshoot the target. the proportion of total movement duration spent in deceleration was determined using the point at which peak velocity occurred: the period of time from peak velocity to maximum displacement was divided by total movement duration. the angular error of aiming was determined from an angle defined by lines joining: the centre of the target and the point at which movement started; and the point at which movement started and the point at which peak velocity occurred. the number of submovements was determined by counting zero-crossings in the velocity function, and the number of cycles of acceleration and deceleration was determined by counting zero-crossings in the acceleration function. these kinematic indices were then subjected to separate 2x2x3 repeated measures analysis of variance. results findings are summarised in table 1. the flatter slope for pitts' law for accupoint and trackball indicated that operators require movements of longer duration to place the cursor upon the computer screen with equivalent precision to that of mouse and digitising pen. some additional insights may be obtained from response latencies. accupoint and trackball had longer response latencies indicating that fewer cues are afforded as to their operation. the accupoint showed no systematic changes in reaction time as a function of task difficulty, implying there are problems planning cursor movements with this device. this may be reflected in the predictability of the cursor trajectory. mouse and pen require fewer accelerative/decelerative impulses per second than accupoint or trackball. the trackball is particularly notable for the large number of submovements it requires. this jerkiness means that cursor trajectories are less stable and predictable for accupoint and trackball. the percentage of movement time spent in deceleration, provides an index as to the amount of terminal guidance required with each device. for most of these cursor control devices, the values are greater than those associated with ordinary pointing movements (e.g. 55%-65%), this indicates a greater reliance upon visual feedback to execute these movements. the accupoint is the exception here, but this may actually arise from the prolonged periods of time spent in acceleration for this cursor control device. as a lump in the middle of the keyboard, the accupoint affords few cues to its operation, and this would appear to lead to slower initiation of responses and greater uncertainty during the initial accelerative and terminal guidance phases of positioning movements. the greater degree of overshoot associated with the accupoint emphasises the problems people have with this cursor control device. 116 ajis vol. 7, no 2 may 2000 slope pitts' law reaction time ms movement time ms overshoot mm no. of submovements force inefficiency1 percent time in deceleration angular error2 mouse 10.8 bits/s 239.7 (4.8) 643.2 (10.7) 2.67 (0.41) 2.20 (0.05) 7.85 (0.10) 71.0 (0.52) 4.3 (0.25) pen 6.94 bits/s 284.8 (4.7) 646.8 (16.8) 0.67 (0.31) 2.40 (0.06) 6.29 (0.08) 81.4 (0.37) 4.5 (0.24) accupoint 5.72 bits/s 460.6 (8.0) 833.9 (24.5) 27.8 (2.0) 2.09 (0.05) 10.80 (0.15) 56.1 (1.18) 4.9 (0.54) trackball 3.30 bits/s 588.9 (21.4) 974.1 (39.6) 9.07 (0.61) 14.58 (0.46) 11.90 (0.07) 73.2 (0.93) 5.74 (0.31) 'number of accelerative/decelerative impulses per second 2degrees error at peak velocity from the direct path to the target table 1. relative efficacy of cursor control devices each of these cursor control devices tends to show the expected changes in performance as greater movement precision is required, that is greater movement durations for more precise (i.e. smaller targets) movements. however, there were some unexpected effects. cursor positioning times tended to be faster, but less accurate to targets on the left of the screen, and there were problems with diagonal movements, which we attribute to additional problems dealing with two coordinate axes (both x and y) instead of one (y). in addition, while we expected cursor positioning to be easier for near targets, the mouse and digitising pen exhibited the reverse for two kinematic indices. as may be seen in table 2, there were increased cycles of acceleration and deceleration per second for near targets for the mouse and pen. in addition, there was increased error associated with the point at which peak velocity was attained. there was a greater angular error of aiming for the near target for the mouse and pen. in contrast, for the accupoint and trackball the greater angular error was associated with the further targets. this means that the cursor trajectories were less predictable in near space for the mouse, but that cursor trajectories were less predictable in far space for the accupoint. there are clearly additional factors influencing cursor control using mouse and pen. force inefficiency1 near far angular error2 near far mouse f( 1,1 1)=35.0, p<0.001 8.26 7.44 f(l,ll)=5.28, p<0.05 4.83 3.81 tablet f(l,ll)=140.9, p<0.001 6.85 5.72 f(l,ll)=30.6, p<0.001 4.94 4.13 accupoint f(l,ll)=12.3, p<0.005 10.18 11.45 f(l,ll)=5.94, p<0.05 3.67 6.16 trackball f(l,ll)=0.9, p>0.05 11.86 11.96 f(l,ll)=3.93, p<0.07 5.46 6.02 'number of accelerative/decelerative impulses per second 2degrees error at peak velocity from the direct path to the target table 2. differential effects of movement amplitude discussion the increasing variety of alternative cursor control devices available for laptop or desktop computers necessitates further considerations of the grounds upon which choice can be made between devices. baber (1997) distinguishes between cursor control devices on the basis of their learnability, effectiveness (speed, accuracy), acceptability and flexibility. to some extent, the effectiveness of a cursor control device can be indicated by pitts' law, with the efficiency of cursor control impacting upon the acceptability of the device as far as sustained use in relatively fixed postures is concerned. nevertheless, pitts' law does not address the underlying effectors or the associated trajectories during cursor control, and this can be a problem when considering possible improvements to hardware or enhancements to software (mackenzie, 117 ajis . vol. 7, no 2 may 2000 1992). in this regard, our kinematic analysis of cursor trajectories produced with a variety of cursor control devices provides some insights into the characteristics of the controllers and the predictability of the trajectories they produce. pitts' law provides a rate of gain of movement precision for movements of longer duration, and this can be used to compare the four cursor control devices in this study. in this regard the mouse is superior to digitising pen, accupoint and trackball in keeping with reviews by mackenzie (1992). the values for the slopes of pitts' law are also in keeping with those of previous studies, which is pleasing given the degree of discrepancy observed between studies (baber, 1997). and while pitts' law is reported to hold for a variety of effectors over a range of tasks, the slope also varies as a function of the muscle groupings employed (langolf, chaffin, & foulke, 1976). a number of factors may contribute to the effectiveness of cursor control devices. mouse, digitising pen, accupoint and trackball vary in terms of the grip (palm, finger, finger-tip) and joint combinations (forearm/wrist, finger/wrist, finger) employed during their operation (baber, 1997). and as the different cursor controllers require different amounts of reaching and grasping, they are liable to employ different neurophysiological systems (goodale, jakobson, & servos, 1996). goodale et al. (1996) suggested that a dorsal system involving posterior parietal cortex was more responsible for on-line control of goal directed action, while a ventral system involving inferotemporal cortex was more devoted to object properties. the distinction is important, as it implies the dorsal system is better suited to the processing of distance and the control of reaching movements, while the ventral system is better equipped to deal with orientation and the control of grasping movements. as detachable manipulanda, the mouse and digitising pen can be picked up in a variety of orientations, many of which do not match the coordinate system of the screen. these variations in orientation lead to potential screen/manipulandum cartesian coordinate system incompatibilities, in which operators need to correct for variations in the orientation of these cursor control devices relative to the screen. we suspect the operator initially needs to engage in on-line corrections to establish the mappings between screen and manipulandum, and that this would be a continuing problem, since the standard 2:1 gain means that slight variations in orientation are magnified upon the computer screen. in this regard, relationships of controller to the screen, and the coding of distance varied, with the accupoint and trackball having a fixed relationship to the screen in laptops, but at the same time affording a less direct means of coding the distances to be moved. such factors would appear to contribute to the observed differences in the slope of pitts' law for these cursor control devices. the present data suggest that variations in biomechanical and neurophysiological systems manifest as greater uncertainty in the initial cursor trajectories (as variations in orientation are mapped in) or greater irregularity during the entire cursor trajectory (as distance is mapped out). the emergence of hybrid cursor control devices comprising mouse and cursor wheel or trackball may indicate manufacturers are sensitive to these issues. an alternative solution to these problems might involve the use of touchsensitive screens or display tablets, but parallax errors and expense currently render these options unsuitable. while cumulative trauma disorder is more commonly linked to keyboard use (harms-ringdahl & schuldt, 1990), it does occur with mouse use (fogleman & brogmus, 1995), particularly when mouse placement promotes inappropriate postures (english & andre, 1999; fogleman & brogmus, 1995). some therapeutic interventions have replaced a cursor control device responsible for cumulative trauma disorder with alternative cursor control devices (kittle, 1999). however, there are reasons to suspect some of the replacement devices (e.g. joystick, trackball) may not be as effective, or be more appropriate for certain specific applications, and indeed be just as likely to lead to problems if used in sustained awkward postures (kittle, 1999). from our data, a principle advantage of joystick and trackball would seem to be the relatively fixed relationship between controller and screen in laptops, which might mean a reduced need to adjust for differences in orientation between screen and controller in the early part of the cursor trajectory. nevertheless, while chosen for compactness, these devices are poorer for full screen movements and other approaches may be desirable. attempts have been made to improve cursor positioning times by means of software interventions which focus upon the accelerative or decelerative phase of movement. for example, murata (1995) attempted to predict cursor endpoint from its initial movement trajectory. conversely, kabbash and buxton (1995) sought to improve cursor positioning by using a larger area cursor which captured icons and reduced the amounts of terminal guidance required. this study documents the proportions of time the cursor spends in acceleration or deceleration. most cursor control devices produce cursor trajectories with considerable amounts of terminal guidance, and where this is not the case (accupoint) the cursor trajectory itself may not be as predictable. our data implies a potential uncertainty with the initial cursor trajectory would make an area cursor (kabbash & buxton, 1995) more effective for mouse and digitising pen, while a predictive algorithm based upon the initial cursor trajectory could be more beneficial for accupoint and trackball (murata, 1995). unfortunately the greater instability of cursor trajectories for some devices might make implementation difficult. conclusions cursor positioning typically involves prolonged amounts of terminal visual guidance, indicating that enhancements that reduce terminal visual guidance would be effective. accupoint and trackball are inefficient cursor control devices, difficult to plan with, and prone to greater overshooting, and with less predictable cursor trajectories. the mouse and digitising pen are more efficient cursor control devices, nevertheless, as detachable manipulanda, there may be problems establishing mappings between device and screen orientation, which are magnified on the screen, and so any interventions should not be engaged for nearer targets for these devices. 118 ajis vol. 7, no 2 may 2000 references baber, c. (1997). beyond the desktop. san diego: academic press. dix, a.j., finlay, i.e., abowd, g.d., & beale, r. (1998). human-computer interaction. london: prentice hall. douglas, s.a., & mithal, a.k. (1997). the ergonomics of computer pointing devices. london: springer. english, j.d., & andre, a.d. (1999). posture and internet navigation: an observational study. proceedings of silicon valley ergonomics conference & exposition, san jose: san jose university, 126-135. fitts, p.m. (1954). the information capacity of the human motor system in controlling the amplitude of movement. journal of experimental psychology, 47, 381-391. fogleman, m. & brogmus, g. (1995). computer mouse use and cumulative trauma disorders of the upper extremities. ergonomics, 38, 2465-2475. can, k.c. & hoffmann, e.r. (1988). geometrical conditions for ballistic and visually controlled movements. ergonomics, 31, 829-839. goodale, m.a., jakobson, l.s., & servos, p. (1996). the visual pathways mediating perception and prehension. in a.m. wing, p. haggard, & j.r. flanagan (eds.), hand and brain. san diego: academic press, 15-31. harms-ringdahl, k. & schiildt, k. (1990). neck and shoulder load and load-elicited pain in sitting work postures. in m.i. bullock (cd.), ergonomics: the physiotherapist in the workplace. edinburgh: churchill-livingstone, 133-147. hittle, j.m. (1999). using alternative input devices and techniques to reduce repetitive strain injury. proceedings of silicon valley ergonomics conference & exposition, san jose: san jose university, 221-223. kabbash, p. & buxton, w. (1995). the "prince" technique: pitts' law and selection using area cursors. chi'95 mosaic of creativity, may 7-11, 273-279. langolf, g.d., chaffin, d.b., & foulke, j.a., (1976). an investigation of fitts' law using a wide range of movement amplitudes. journal of motor behavior, 8, 113-128. mackenzie, i.s. (1992). fitts1 law as a research and design tool in human-computer interaction. human-computer interaction, 7, 91-139. murata, a. (1995). improvement of performance by method for predicting targets in pointing by mouse. ieice transactions fundamentals, 78, 1537-1541. trankle, u., & deutschmann, d. (1991). factors influencing speed and precision of cursor positioning using a mouse. ergonomics, 34, 161-174. walker, n., meyer, d.e., & smelcer, j.b. (1993). spatial and temporal characteristics of rapid cursor-positioning movements with electromechanical mice in human-computer interaction. human factors, 35, 431-458. wickens, c.d., gordon, s.e., & liu, y. (1998). engineering psychology and human performance. new york: harper collins. 119 ajis vol. 7, no 2 may 2000 editorial -special section from ozchi99 conference the following six papers have been selected from the recent ozchi99 conference held at csu in november 1999. the theme for the conference, interfaces for a global community, reflects the importance of system and interface designs that facilitate effective communication between computers and the human population, with all of its diversity. it also reflects the mutlidisciplinary nature of systems and interface design. the quality and scope of the conference papers was encouraging and the papers included in this edition illustrate some of the excellent research currently being undertaken in human-computer interaction. jan scott and barney dalgarno, editors, ozchi99 conference proceedings the australasian journal of information systems (ajis) began in 1993 and was based on the simple realisation that there was a need for an academic ‘outlet’ for is research in australia australasian journal of information systems volume 18 number 2 2014 165 identifying the education needs of the business analyst: an australian study deborah richards macquarie university deborah.richards@mq.edu.au mauricio marrone macquarie university mauricio.marrone@mq.edu.au abstract the business analyst (ba) plays a key role in ensuring that technology is appropriately used to achieve the organisation’s goals. this important mediating role is currently in high (unmet) demand in many english-speaking countries and thus more people need to be trained for this role. to determine the educational and/or training needs of a ba we conducted a survey in the information and communication technology industry in australia. the survey items are based on prior studies of information systems educational requirements and the internationally-developed skills framework for the information age (sfia) that has been endorsed by the australian computer society. from the literature we identified three types of skills: soft, business and technical. with the increasing importance of greenit and the pivotal role that the ba could play in green decision making, we added a fourth type of skill: green. the survey considers 85 skills, their importance, the level of attainment of that skill, skill gaps and types of skills. results show that all soft skills were considered to be important with the smallest knowledge gaps. selected business skills and green skills were seen to be important. technical skills were considered less important, but also where the largest knowledge gaps existed. further we asked respondents whether each skill should be acquired via an undergraduate or postgraduate degree and/or industry training and experience. we found that the workplace was considered the most appropriate place to acquire and/or develop all skills, except the ability to innovate. while we found that softskills should be taught almost equally at the undergraduate and postgraduate level, business and green skills were more appropriate in a postgraduate degree. in contrast, technical skills were best acquired in an undergraduate program of study. keywords: business analyst, educational needs, australia, greenit education, introduction according to the australian computer society (acs) (2011), the role of a business analyst is to “assess the overall business and information needs of an organisation. they co-ordinate the design of new it solutions to improve business efficiency and productivity”. for this paper, and since the study is based in australia, we adopt the definition established by the australian computer society which we explore further in the next section. according to recruiters, skills shortages in ict are becoming critical with “the most in-demand ict roles nationally are network engineers (in all states), project managers, australasian journal of information systems volume 18 number 2 2014 166 business analysts and developers6”. this shortage of bas is also being experienced in other countries such as the uk7. education and training can be used to address skills shortages. to ensure that the skills taught in such a program will address the current gap in industry, it is important to ask industry what knowledge and skills are needed, where they should be acquired and where gaps currently exist. presentation of an industry-based study asking these questions is the focus of this paper. typically, at the undergraduate level, a ba would undertake a degree within the information systems (is) field. we thus focus on the educational needs of a ba within the context of information systems programs. the challenge, among others, facing learning and teaching (l&t) in the is field is the dynamic and cross-disciplinary nature of the material to be taught. due to a rapid change in technologies, l&t in this field must find a balance between teaching the fundamentals which are stable and core and changing course content and delivery to meet the current and future needs for knowledge and skills in the industry. also, it is essential that the program is coherent, broad, and yet have sufficient depth so that the is graduate can take on careers requiring them to understand the needs of business, turn those needs into a computer based solution and convey the design of that solution to the technical team. the definition of the role of a business analyst is elusive. alexander (1999) argues that this is the least defined it role. some find that a business analyst is “largely expected to ‘protect’ the users and it staff from each other” (vashist et al. 2010, p. 2). others may perceive that business analysts “are responsible for the business and technical analysis and design of the business solution and to manage the soft issues arising from the relationship” (evans 2004, p. 2). evans also suggests that there might be some confusion of the title, role and required knowledge of a business analyst. this study contributes to researchers, educators, employers and practitioners to identify relevant sets of skills enabling professional bas in the modern technological driven business environment. via the literature and an industry based survey we seek to identify skill sets and the alignment between the perceived importance of these skills and their knowledge level. subsequently, our research questions are: r1: which skills are viewed by industry as important for bas? r2: do bas have knowledge of and competence in these skill sets? r3: is there a gap between what bas perceive as important and their attainment of that knowledge? r4: where should the skills identified as important be acquired/learned? (i.e. in an undergraduate or postgraduate degree or in the work-place) to answer these questions, the next section draws upon the literature to develop a potential set of skills needed by the ba. our methodology section outlines the survey instrument to measure the importance 6 http://www.theaustralian.com.au/careers/skills-shortage-to-become-critical/story-fn717l4s1226063877709, accessed 5-12-12 7 kpmg, july 8,2011 http://news.globalcareerlink.com/united-kingdom-news/business-analyst-skillsshortage-continues-as-uk-it-job-market-grows/ accessed 10-8-13 or http://www.computerweekly.com/news/2240104980/business-analyst-skills-shortage-continuesas-it-job-market-grows australasian journal of information systems volume 18 number 2 2014 167 and knowledge for each of those skills. our results and discussion follow, concluding with limitations and future research. literature review first, we consider an initial definition of the ba role in the australian context. according to the australian computer society (acs) (2011), a ba may need to: • “evaluate business needs, and contribute to the design and development of a business solution; • work closely with developers and end users to ensure technical compatibility and user satisfaction; • ensure timelines and budgets are met, and oversee the implementation of a new system; • write user manuals and provide or co-ordinate training to users of a new system”i. the australian department of immigration and citizenship states that the ba position “identifies and communicates with users to formulate and produce a requirements specification to create system and software solutions8”. the site goes on to note that “this occupation has a level of skill commensurate with a bachelor degree or higher qualification”. so while in the past it may not have been an entry level position, this is changing and it is unclear just what skills may be included in the role and where they should be acquired. despite the increasing demand, we could not find industry-based research to understand the skills needed specifically by a business analyst. however, there has been some research concerning the skills that an is graduate must have when entering the working environment both in australia and other countries. in the absence of a focus on the ba previously, we have used the set of skills needed by an is graduate in general to provide a starting point and broader base for our survey of ba specific skills. these studies can be divided into three categories: (1) industry surveys, (2) faculty and students’ survey, and (3) job advertisements studies. we cover each of these areas in the next sub-sections. industry surveys though universities have tended in the past to concentrate on technical skills to prepare their graduates, research has found that personal characteristics, or “soft skills,” are often considered by employees to be more valuable than technical skills. fang et al. (2005) surveyed 51 u.s. recruiters and found that interpersonal skills (e.g. teamwork, communication) and personal traits (e.g. critical thinking, personal motivation) were regarded as more valuable than is skills and organizational knowledge. research from bullen et al. (2007) surveyed 77 companies in the u.s. and found that firms required capabilities in project management and the business domain over technical capabilities. simmons and simmons (2010) interviewed 20 is professionals in fortune 500 businesses, small-to-medium businesses, and non-profit organizations to conclude that an emphasis is placed on negotiation skills, time management, managing cultural differences, outsourcing management, and information assurance as well as technical skills. aken and michalisin (2007) developed a comprehensive survey to identify, from an employer’s perspective the skills that management information systems (mis) graduates are lacking. they established four categories of skills: soft, business, technical and programming. other researchers, such as burns et al. (2004), ferguson (2005), doucek et al. (2007), lee et al. (2008) and litecky (2009) have also surveyed the industry to pinpoint the required skills. the skills they identified can be found in table 1. 8 http://www.immi.gov.au/asri/occupations/i/ict-business-analyst.htm#desc accessed 5-12-12 australasian journal of information systems volume 18 number 2 2014 168 faculty and student surveys academics and curriculum researchers have been interested in understanding the skills and knowledge that university faculty and students view as important for the workplace. weber, mcintyre, and schmidt (2001) compared is students with the is industry’s perceptions of the importance of various skills. in general, they found that is students’ ratings of the importance of skill and competence were similar to those rated by the industry. in another study, medlin, dave, and vannoy (2001) surveyed students to understand their views on the importance of technical versus nontechnical skills for a successful it career. four categories of skills were included in this study: technical, organizational, creative thinking, and analytical skills. overall, students rated communication skills the highest, followed by analytical skills, technical skills and organizational skills. the study results showed that the students and the practitioners’ views were similar, which indicated that is professionals need more than technical skills to be successful in the workplace. csapo and featheringham (2005) looked at is alumni’s experience once they entered the work place and found, that alumni view communicating skills as crucial to their future career success. researchers that have compiled a comprehensive list of skills required include mutch (1996), calitz (1997), miertschin et al. (2006), and beard et al. (2007). these skills are aggregated in table 1 and used as part of our survey instrument. job advertisement studies to examine the development of is job skills from the early 1980s to the late 1990s, todd, mckeen, and gallupe (1995) analysed the number of technical, business and systems phrases in 1,234 job advertisements from four leading newspapers. three types of is jobs were included: programmers, systems analysts, and is managers. the results showed that while the programmers’ job requirements had changed very little over the 20 year period and the is managers’ job category remained relatively stable; the systems analysts category showed the greatest change. more recently, however, liu, liu, koong, and lu (2003) analysed 300 is job ads posted on job advertising websites such as monster and hot jobs and suggested that the types of skills is changing. contemporary programming languages and web development skills had increased in importance. lee and han (2008) analysed 837 systems analysts job advertisements posted on the website of 230 fortune 500 organizations between 2001 and 2003. they found that an equal emphasis was required with regards to skills in design, software development, business and social understanding. huang et al. (2009) looked at 241 job ads posted in monster.com between april 2008 and june 2008 and concluded that there is an overall increase in demand for capabilities in business, specifically in project management, financial analysis, and communication skills. conversely, kennan et al. (2007) found that in australia, job advertisements focused more on technical skills. however, communication skills were mentioned in close to 75% of the ads. importance of skills the reviewed literature indicates that there is an emphasis on soft skills by industry, which tend to be treated as generic skills or graduate attributes by universities. for example, the acs (2011) states the ba “career would appeal to people who: • have an ability to conceptualise and think creatively; • have excellent oral and written communication skills; • effective negotiation and customer management skills; • sound administrative skills and good analytical and reporting abilities; • effective time management and personal organisation skills; • an understanding of user needs and a business outcome approach; australasian journal of information systems volume 18 number 2 2014 169 • can articulate visions; • and a logical approach to problem solving and an investigative and inquisitive mind”. csapo and featheringham (2005) estimate that “employees spend more than 75 percent of their time communicating in interpersonal situations, as well as on a variety of other levels within the organization and externally” (p. 1). the researchers suggest that this may be a reason why communication and other soft skills may be seen by recruiters as important. generally, research has attempted to identify the skills that are needed by is graduates overall. in addition, a previous study by rai, vatanasakdakul and aoun (2010) investigated the alignment between which is skills are perceived as important to accountants and their knowledge levels of such technologies. it is interesting to note that none of the literature surveyed included green-related skills. with sustainability high on the agenda in many nations, graduates who will act as communication bridges between technology specialists and businesses will play a vital and leading role in ensuring a sustainable future for our planet. other studies, shown in table 1, suggest that the ba is a communicator, requirements gatherer and solutions architect. the ba thus potentially plays a key role in making sustainable choices, providing direction to business and influencing demand for specific technologies. the authors believe bas will potentially play a vital role in decisions around environmental sustainability and will need to be educated in this area. in this study, we wanted to test whether current practitioners held such a view. table 1 shows the extended set of skills and competences, the identify of the authors that have studied (number explained in legend) and the total number of authors (count) that identified the respective skill. australasian journal of information systems volume 18 number 2 2014 170 skills # skill label 1 2 3 4 5 6 7 8 9 10 11 12 cou soft 1 ability to learn/lifelong learning x x x 3 2 attention to details x x x 3 3 business problem solving x x 2 4 creativity x x x x 4 5 analytical /critical / logical thinking x x x x x x 6 6 general problem solving x x x x x x 6 7 ability to innovate x x x x x 5 8 conflict resolution x x 2 9 teamwork x x x 3 10 leadership x x x x x x 6 11 self-esteem x x 2 12 negotiation skills x x 2 13 customer-oriented outlook/service oriented x x 2 14 initiative/motivation to work x x x x 4 15 integrity/honesty/ethics x x x 3 16 professional ethics x x 2 17 responsibility x x x 3 18 time management x x 2 19 general communication x x x x x x 6 20 written communication x x x 3 21 oral communications x x x x 4 22 training others x 1 23 organization skills x x x 3 business 24 domain/industry specific knowledge x x x 3 25 enterprise/organization specific knowledge x x x x 4 26 general business skills x x x 33 27 accounting x x x x 4 28 business process design /re-engineering x x x x 4 29 contracting and legal x x x 3 30 finance x x x 3 31 marketing x x x 3 32 supply chain management x x 2 33 asset and human resource management x 1 34 management theory and practice x 2 35 change management x 1 36 outsourcing management x x x 3 37 user/3rd party relationship management x 1 38 working in global or virtual teams x x x x 4 39 project mgt/planning/budgeting/scheduling x x x x x x 6 40 project risk management x 1 41 business intelligence x 1 42 business strategy x x x 3 43 project integration x x x 2 44 it service management (e.g. itil, mof) x 1 45 it governance x 1 46 technology auditing x 1 australasian journal of information systems volume 18 number 2 2014 171 technical 47 e-commerce x x 2 48 sdlc/development methodologies x x x x x 5 49 requirement specification x 1 50 analysis x x x x x x x 7 51 design x x x x 4 52 implementation x x x x x x 6 53 testing x x x 3 54 integration and deployment x 1 55 software quality assurance x x 2 56 case tools x x x x 4 57 user-interface design x 1 58 web-based application development x x 2 59 programming x x x 3 60 applying it to business problems x x 2 61 customer relationship management x 1 62 dss/gdss x 1 63 enterprise resource planning (erp) x x 2 64 transaction processing systems x x 2 65 data mining x x x x 4 66 data warehousing x x x x 4 67 database administration x x x 3 68 electronic data interchange (edi) x 1 69 online analytical processing (olap) x x 2 70 business continuity planning (bcp) x 1 71 operating systems x x x x 4 72 web application servers x x x x 4 73 web servers x x x x 4 74 it architecture / standards x x x x x 5 75 network administration x x x x 4 76 security x x x x x 5 77 client-server architecture x x x x 4 78 printer/storage x 1 79 voice/data telecom x x x x 4 green 80 sustainability strategy x 1 81 sustainability management of it 0 82 training for sustainability awareness x 1 83 sustainability assessment x 1 84 sustainability engineering 0 legend: 1. mutch 1996; 2. calitz, watson, kock 1997; 3. burns, hopper, yazdifar 2004; 4. ferguson, 2005; 5. miertschin et al. 2006; 6. aken, michalisin 2007; 7. doucek and novotný 2007; 8. beard et al. 2007 9. kwoon-lee et al. 2008; 10. litecky et al. 2009; 11. sfia, 2011; 12. simmons and simmons 2010 table 1. set of ba skills based on the literature methodology there has been no research that attempts to study the ba’s perspective and map the alignment between is skills importance and knowledge. therefore, in order to fully understand the skills needed by bas in australia, it is important to investigate both knowledge and importance of skills. by identifying a gap between knowledge and its perceived importance, the research will help educators and practitioners to recognize any deficiencies in it skills needed by bas. to achieve this objective, we have identified 84 factors based on the set of skills reported in the literature and aligned them with those established by the skills framework for the information age (sfia) (2011) guidelines which has been adopted by the acs for defining employment and educational requirements. also, informed by the literature, we divided these factors into four categories: soft, business, technical and green skills. the first three australasian journal of information systems volume 18 number 2 2014 172 correspond to aken and michalisin’s (2007) four categories. we saw programming as a type of technical skill and in itself a small category. thus, we have subsumed the programming category into the technical categeory. green skills were added to reflect this new and emerging area of concern that did not fit clearly under one of the other categories and we also wanted to specifically identify the perceived need for training in these skills. this study adopts a quantitative survey approach. a study conducted by scott and yates (2002) looked at identifying capabilities that were seen to be most important for the engineering field. in this survey, engineering graduates rated the importance and performance (extent of this being studied in university) of each item for their successful professional practice. other fields such as accounting (crebert et al., 2004; rai, vatanasakdakul & aoun (2010) have also used a similar approach. preceding the data collection, a pilot survey was carried out to ensure the reliability of the survey instrument. this pilot was conducted by a total of 6 academics, students and business analysts. their feedback was used to polish the survey and identify any ambiguous skill or trait terminology. as a result of comments from bas enrolled in our master of it who participated in our pilot study, we added the skills working under pressure, research skills, and questioning skills. review of the skills before and after the pilot resulted in modification of some of the labels in table 1 to improve comprehensibility and improve alignment with the terms in sfia. for example, rather than just the word design, analysis, implementation or testing (skills 50-53 in table 1), we used the terms software design, software analysis, software implementation and software testing. programming was dropped as a separate category because it was seen to be covered by software implementation. user/3rd party relationship management was relabeled to stakeholder relationship management. we also added requirements elicitation to differentiate this skill from requirements specification because they involve very different activities and are treated as separate sections and chapters in most textbooks. we dropped general communication skills, people skills and general business skills as we had more specific skills in each of these areas already included in our survey and the length of our survey was a concern. this resulted in an amended set of 85 skills. the online survey was made available in the month of june 2011. an invitational email was sent to members of the australian computer society (acs), the major professional body for the australian it industry. it was distributed initially to the nsw branch to be further distributed by other states. the survey targeted experienced business analysts, those who had been business analysts, and those that managed business analysts, but it was also open to others working in ict. as the ba is part of an it team, we thought it was also important to gain an understanding of what others working with bas also believed the role entailed. this also allowed us to analyse if differences in perception existed. the first part of the survey captured biographical data such as gender, age, experience and their relationship to the ba role. following the biographical section, the questionnaire listed the many skills under investigation. these skills were gathered from the literature, and covered the questions and categories as per table 1. the categories were not identified in the survey, and thus participants were not aware which questions belonged to which category. the research aimed at understanding the importance of each of the skills listed. importance ratings for each of the skills was on a 4-point likert scale (0= no importance to 3=very important). this scale was chosen to eliminate the neutral or undecided answers. the survey also aimed at understanding if the respondents perceived that they possessed these skills. respondents were asked to self-report their attainment of this skill and we have given this the label of ‘knowledge’ in the tables. for this answer, a 4-point likert scale was used (0=no skill to 3=highly skilled). we asked where these skills should be taught, where the gaps are. we concluded the survey with the question australasian journal of information systems volume 18 number 2 2014 173 did we miss any skills needed by a business analyst? where they should be learnt? any other comments? the results to each of these questions are presented and discussed next. results and discussion respondents out of the 163 it professionals who began to fill out the survey, 131 completed and provided valid responses. partially completed surveys were not used in the study. the sample size is 131, unless specified otherwise. respondents were asked about their company’s size and industry, age, gender, and years of work experience. we did not collect data about which state they worked in as this was not considered a variable that would contribute to our understanding. table 2 shows the profile of the respondents. the technology, public and education and the financial and banking sectors represent close to 75% of the industries covered in the survey. other industries include manufacturing and construction, non-profit, retail and distribution. the respondents represented small medium and large companies. the largest population of respondents was between the ages of 30 and 39 (26%), followed by those between 40 and 49 (22.1%). the majority of the respondents were male (78%) with female (22%). the average number of years of experience was 13.26. of those surveyed, 25.2% were currently business analysts and 21.4% worked with business analysts. close to 15% aspired to be business analyst and 15.3% had previously been a business analyst. industry percent information media & telecommunications 24.4 finance & management services 26.7 public sector & education 13.7 other 22.9 manufacturing & construction 3.8 non-profit 0 retail & distribution 2.3 healthcare 2.3 utility 0.8 age percent 1824 9.2 25-29 19.8 30-39 26 40-49 22.1 50 and above 19.1 gender percent male 78 female 22 company size percent less than 500 36 501 to 10,000 32 more than 10,000 35 years of experience percent less than 10 19.2 10 to 20 33.1 21 and over 46.7 job role percent i am a business analyst 25.2 i work with business analysts 21.4 i want to be a business analyst 14.5 i have been a business analyst 15.3 i manage business analysts 9.2 none of the above 14.5 table 2. respondents’ profile (n=131) empirical results an exploratory analysis was conducted for each variable to test for normality. both the kolmogorov smirnov and the shapiro-wilk showed significance for variables of importance of skills (p<0.001) and for variables on perceived attainment of skill (p<0.001). due to the data being non-normal, the mannwhitney u test was used. australasian journal of information systems volume 18 number 2 2014 174 perceived importance and knowledge in order to illustrate the gaps between the perceived importance and the perceived knowledge, each of the sets of skills is presented in figure 1. this figure shows the average score of the perceived importance and knowledge of all skills within each category; soft skills comprising 24 sub-skills, business skills comprising 25 sub-skills, technical comprising 31 sub-skills and green comprising 5 sub-skills. a further analysis of individual skill items, including the mean values and the ranks of all the skills studied are shown in table 3. in terms of perceived importance, the soft skills were perceived to be the most important, with a mean score of 2.57. this is followed by business skills (2.08), green skills (1.91) and technical skills (1.79). in addition, the scores of perceived knowledge also show a similar pattern that soft skills are the highest (2.45), followed by business skills (1.95), green skills (1.7) and technical skills (1.67). figure 1. bar chart of importance and knowledge of sets of skills moreover, to understand if there is a difference in perception of importance of skills between those that manage bas, work as bas or have been bas and those that are not, mann-whitney and wilcoxon tests were conducted. results are shown in table 3. results show a significant difference in perceptions of the importance of technical and green skills between those people (93) directly involved with the ba role (i am, have been, work with, manage bas) and those (38) that are not (“i want to be a ba”, “none of the above”). no difference was observed in the perceptions of these cohorts for knowledge attained or exhibited by bas. 2.57 2.08 1.79 1.91 2.45 1.95 1.67 1.7 0 0.5 1 1.5 2 2.5 3 soft business technical green skill type av er ag e importance knowledge australasian journal of information systems volume 18 number 2 2014 175 soft skills business skills technical skills green skills importance knowledge importance knowledge importance knowledge importance knowledge mannwhitney u 1396.5 1262.5 1226 1223 804 947 1327 1656 wilcoxon w 4882.5 1727.5 4712 1818 3730 3503 5698 2359 z -0.087 -0.356 -1.551 -0.652 -2.137 -0.091 -2.242 -0.145 asymp. sig. (2-tail) 0.931 0.722 0.121 0.514 0.033* 0.927 0.025* 0.885 asymp. sig. (1-tail) 0.4655 0.361 0.0605 0.257 0.0165* 0.4635 0.0125* 0.4425 table 3: difference in perceptions between those directly involved with the ba role and others a further analysis of individual skill items including the mean values and the ranks of all the skills studied are shown in table 4. soft skills are rated as one of the most important skills to have as a business analyst. the five most highly rated skills are: ability to learn/lifelong learning, analytical / critical / logical thinking, responsibility, business problem solving and working under pressure. soft skills are considered to be of such high importance that they comprise the top 9 ranked skills and 92% of them are in the top 36%. top ranking business skills are it governance (ranked 10th) and it service management (ranked 12th). the next business skill, business process improvement/reengineering is ranked 21. the highest ranked technical skill is integration and deployment at position 16. software quality assurance is the next most important technical skill which is ranked 37 followed by other technical skills within rankings in the range 40-85. green skills, though an emerging area, were comparable in importance to technical skills and not so far behind business skills. the most important green skill was seen to be sustainability strategy ranked 33 with sustainability engineering and assessment the least important, ranked 70 and 71, respectively. it appears that the ba role is very broad spanning the strategic, high level decision making and management activities of an organization as well as the activities at the operational and implementation levels. in terms of perceived knowledge, the respondents rate their knowledge as very high in the soft skill area. the top 16 rankings of knowledge are in soft skills with the exception of the technical skill of integration and deployment which is ranked 11. this skill is somewhat of an outlier as the next highest ranked technical skill, requirements elicitation, is ranked 38. knowledge attainment for business skills is highest for organisation skills and it governance, ranked 17 and 18, respectively. the first five skills in which respondents perceived themselves to be stronger were: responsibility, working under pressure, time management, analytical /critical / logical thinking and leadership. australasian journal of information systems volume 18 number 2 2014 176 the largest gap between perceived knowledge and importance were found in the technical skills category which had the top five ranked skills for gaps (client-server architecture, voice/data telecom, printer/storage, information security and network administration). what this result indicates is that even though none of these skills ranked higher than 58/85 in importance for the ba role, the level of knowledge needed was greater than what they possessed. this highlights the challenge of educating is professionals to be sufficiently broad to be able to fulfill their role as communicators with business and technical stakeholder. requirements elicitation was the next gap in this category and ranked 10. in the soft skills category the biggest gap was in business problem solving (mean=0.35) followed by ability to learn/lifelong learning (0.3) and were ranked 9th and 12th respectively. in the business skills category the largest gaps in necessary knowledge were identified in the areas of it service management and it governance, ranked 6th and 8th. as these skills were ranked more highly for importance, it seems that these gaps should be addressed most urgently. stakeholder relationship management ranked 11th. the gap in knowledge for the skill sustainability strategy was ranked 7th which is not surprising as little training and information would be readily available for the emerging area of green skills. some skill gaps were negative, meaning that respondents saw themselves as having the skills but not needing them for the ba role. skills with small negative gap values, in increasing order, (<0.15) include: general problem solving; ability to innovate; management theory and practice; ecommerce; dss/gds; it architecture/standards; self esteem; database administration; project integration; edi; continuity management. other skills even less utilised included (in ascending order); web servers (0.28); training others (-0.27); and data warehousing (-0.41). importance knowledge gap skill acquisition soft skills mean rank mean rank mean ran ugrad pgrad work ability to learn/lifelong 2.83 1 2.53 10 0.3 13 54.20% 55.70% 83.20% analytical /critical / logical 2.82 2 2.6 4 0.22 28 59.50% 64.10% 77.90% responsibility 2.77 3 2.75 1 0.02 68 48.10% 45.80% 80.20% business problem solving 2.76 4 2.41 16 0.35 9 30.50% 44.30% 85.50% working under pressure 2.76 5 2.72 2 0.04 64 41.20% 44.30% 85.50% research skills 2.76 6 2.57 8 0.19 34 67.90% 64.90% 72.50% leadership 2.76 7 2.59 5 0.17 39 46.60% 42.70% 85.50% time management 2.74 8 2.71 3 0.03 65 48.90% 45.00% 85.50% questioning skills 2.74 9 2.44 15 0.3 14 59.50% 56.50% 74.80% attention to details 2.72 11 2.47 13 0.25 20 42.00% 41.20% 83.20% oral communications 2.67 13 2.45 14 0.22 29 58.00% 55.70% 81.70% integrity/honesty/ethics 2.66 14 2.37 19 0.29 15 45.80% 52.70% 74.80% professional ethics 2.66 15 2.58 6 0.08 51 36.60% 38.90% 84.70% initiative/motivation to work 2.63 17 2.57 9 0.06 56 19.80% 21.40% 89.30% general problem solving 2.57 18 2.58 7 -0.01 73 58.00% 45.00% 74.80% written communication 2.52 19 2.49 12 0.03 67 41.20% 44.30% 87.80% customer-oriented 2.47 20 2.22 26 0.25 21 32.10% 38.20% 84.00% teamwork 2.44 22 2.15 31 0.29 17 25.20% 34.40% 82.40% negotiation skills 2.43 23 2.37 20 0.06 57 42.00% 43.50% 81.70% conflict resolution 2.33 27 2.18 29 0.15 43 34.40% 47.30% 71.80% creativity / innovation 2.28 29 2.22 27 0.06 59 47.30% 44.30% 72.50% ability to innovate 2.25 31 2.26 23 -0.01 71 57.30% 68.70% 49.60% self-esteem 2.18 38 2.24 25 -0.06 76 30.50% 43.50% 83.20% training others 2.03 47 2.3 21 -0.27 83 27.50% 39.70% 80.90% category average 2.57 2.45 0.13 43.92% 46.75% 79.71% australasian journal of information systems volume 18 number 2 2014 177 importance knowledge gap skill acquisition business skills mean rank mean rank mean ran undgra upgrade work it governance 2.73 10 2.38 18 0.35 8 46.60% 51.90% 78.60% it service management (e.g. 2.68 12 2.27 22 0.41 6 43.50% 51.10% 76.30% business process 2.45 21 2.16 30 0.29 16 39.70% 46.60% 74.80% organization skills 2.42 24 2.4 17 0.02 69 42.70% 43.50% 84.70% stakeholder relationship 2.4 25 2.09 34 0.31 11 22.10% 34.40% 79.40% domain/industry specific 2.37 26 2.25 24 0.12 45 25.20% 38.90% 85.50% enterprise/organisation 2.31 28 2.2 28 0.11 46 9.90% 16.00% 87.80% project management / 2.28 30 2.08 36 0.2 33 40.50% 56.50% 80.90% project risk management 2.23 32 2.07 37 0.16 40 28.20% 48.10% 78.60% technology auditing 2.21 34 2.13 32 0.08 52 58.00% 51.10% 67.20% change management 2.19 35 2.01 39 0.18 36 31.30% 47.30% 77.90% working in global or virtual 2.19 36 2.1 33 0.09 50 21.00% 37.40% 77.90% business strategy 2.18 39 1.94 44 0.24 24 21.00% 48.10% 78.60% business intelligence 2.12 41 1.91 46 0.21 31 26.00% 42.70% 79.40% business planning 2.08 44 1.87 53 0.21 32 26.00% 41.20% 76.30% asset and human resource 1.99 51 1.97 43 0.02 70 60.30% 50.40% 67.90% project integration 1.98 52 2.09 35 -0.11 78 43.50% 40.50% 71.80% applying it to business 1.88 56 1.66 60 0.22 26 43.50% 41.20% 71.00% management theory and 1.81 62 1.91 47 -0.1 77 46.60% 53.40% 73.30% supply chain management 1.64 68 1.54 69 0.1 47 34.40% 40.50% 68.70% outsourcing management 1.6 73 1.55 68 0.05 62 21.40% 35.10% 74.00% contracting and legal 1.59 74 1.5 74 0.09 49 31.30% 37.40% 74.80% finance 1.59 75 1.56 67 0.03 66 46.60% 41.20% 67.90% marketing 1.56 77 1.48 76 0.08 53 45.00% 41.20% 66.40% accounting 1.47 81 1.6 64 -0.13 79 52.70% 42.00% 61.80% category average 2.07 1.95 0.13 36.37% 43.11% 75.26% australasian journal of information systems volume 18 number 2 2014 178 importance knowledge gap skill acquisition technical skills mean rank mean rank mean ran ugrad pgrad work integration and deployment 2.65 16 2.49 11 0.16 42 38.90% 48.90% 80.20% software quality assurance 2.19 37 2.01 40 0.18 37 30.50% 38.90% 75.60% software implementation 2.13 40 1.87 52 0.26 19 52.70% 36.60% 72.50% data mining and analysis 2.1 42 1.78 57 0.32 10 30.50% 44.30% 74.80% requirement elicitation 2.09 43 2.04 38 0.05 63 52.70% 42.00% 73.30% requirement specification 2.05 45 1.98 41 0.07 54 32.80% 37.40% 79.40% software analysis 2.03 48 1.89 50 0.14 44 47.30% 42.00% 76.30% operating systems 2.03 49 1.81 55 0.22 30 54.20% 49.60% 76.30% online anaytical processing 1.98 53 1.92 45 0.06 58 50.40% 48.90% 68.50% web-based application 1.95 54 1.9 48 0.05 60 40.50% 40.50% 71.00% web application servers 1.95 55 1.9 49 0.05 61 53.40% 38.20% 69.50% user-interface design 1.86 57 1.64 61 0.22 25 30.50% 41.20% 71.00% information security 1.86 58 1.38 81 0.48 4 28.20% 48.10% 63.40% network administration 1.85 59 1.43 79 0.42 5 32.80% 44.30% 66.40% ecommerce 1.83 60 1.84 54 -0.01 72 45.80% 47.30% 64.10% client-server architecture 1.82 61 1.25 83 0.57 1 29.80% 44.30% 64.90% customer relationship 1.81 63 1.62 62 0.19 35 45.00% 41.20% 67.90% software design 1.8 64 1.62 63 0.18 38 43.50% 43.50% 64.10% transaction processing 1.78 65 1.47 77 0.31 12 38.90% 42.00% 64.10% software testing 1.77 66 1.52 71 0.25 22 54.20% 37.40% 62.60% voice/data telecom 1.75 67 1.21 84 0.54 2 32.10% 37.40% 67.20% printer/storage 1.63 69 1.12 85 0.51 3 32.10% 42.70% 62.60% enterprise resource planning 1.61 72 1.45 78 0.16 41 38.20% 35.90% 66.40% data warehousing 1.57 76 1.98 42 -0.41 85 55.70% 42.70% 67.90% case tools 1.51 78 1.29 82 0.22 27 33.60% 38.20% 57.30% database administration 1.48 79 1.52 72 -0.04 75 52.70% 35.90% 68.70% dss/gdss 1.48 80 1.68 59 -0.2 82 47.30% 35.90% 70.20% it architecture / standards 1.38 82 1.41 80 -0.03 74 42.70% 28.20% 66.40% electronic data interchange 1.38 83 1.52 73 -0.14 80 45.80% 32.10% 68.70% continuity management 1.34 84 1.49 75 -0.15 81 45.00% 35.10% 69.50% web servers 1.31 85 1.59 65 -0.28 84 46.60% 26.70% 67.20% category average 1.81 1.67 0.14 42.07% 40.23% 68.97% importance knowledge gap skill acquistion green skills mean rank mean rank mean ran undergr pgrad work sustainability strategy 2.23 33 1.87 51 0.36 7 35.10% 52.70 74.00% training for sustainability 2.05 46 1.8 56 0.25 23 31.30% 50.40 71.80% sustainability management 2.03 50 1.75 58 0.28 18 34.40% 36.60 78.60% sustainability engineering 1.63 70 1.56 66 0.07 55 26.70% 42.00 68.70% sustainability assessment 1.63 71 1.53 70 0.1 48 24.40% 36.60 71.00% category average 1.91 1.70 0.21 30.38% 43.66 72.82% table 4. importance of each skill, participants’ level of competency (knowledge) and the gap, by skill type in summary, to answer research question 1 (i.e. the skills viewed by the industry as important), we have identified the sets of skills that are important to bas. the results are quite clear that soft skills are perceived as the most important attributes for bas. almost all soft skills are more important than business, technical or green skills. the results are in line with previous research, particularly the australasian journal of information systems volume 18 number 2 2014 179 analytical /critical / logical thinking skills which are the skills most frequently identified by recent is studies, refer to table 1 (e.g. doucek and novotný 2007, beard et al. 2007 kwoon-lee et al. 2008, sfia 2011). business skills are the next most important set of skills. responding to research question 2 (i.e. do they have the knowledge of and competency in these skills), the findings are more mixed across categories. bas perceive that they have higher ability in the soft skills compared to either the business, technical and green skills. respondents ranked their knowledge highest in integrity/honesty/ethics and professional ethics. this finding is consistent with the previous studies by aken and michalisin (2007), beard et al. (2007) and litecky et al. (2007). this leads us to consider research question 3 (i.e. the difference between perceptions of importance and knowledge). gaps between importance and knowledge can be found within all skill categories. the biggest gap between skill importance and ba knowledge is in technical skills, however, the knowledge gap for skills that were ranked high in importance should be addressed first. we found it governance and it service management to be areas for possible concern. we also found that skill and knowledge gaps should be addressed in the area of green is/it. we recommend that green related topics be included in the is curriculum. green it content could replace content relating to those skills that showed a negative gap. alternatively, educators need to make students more aware of how and where these skills can be applied and also some education of employers is required to make them aware of the untapped skills of their employees. bas are expected to have knowledge in the business domain, not just in the technical area, and they are also required to interact with multiple parties as part of the requirements gathering process and design of systems. hence, to address skill demands and reduce these gaps, we encourage educational programs to include training in it governance, it service management, stakeholder relationship management and business problem solving skills for future bas. this leads us to consider where our respondents believe these skills should be acquired. education and training needs to acquire these skills we see in the skill acquisition column in table 4 that all skills, with the exception of ability to innovate, should be learnt or extended at work (69-76%). perhaps respondents felt that at work they had little opportunity to innovate in their current it positions or that they should have been taught how to nurture this ability before they became more molded into their current work habits. while the differences between the results in the undergraduate and postgraduate columns are not as great as the differences to be seen in the ug/pg columns with the “work” column, we do see differences for each category and individual skills. in general for soft skills, 43.92% of respondents believe these should be taught in undergraduate courses and this increases only slightly to 46.75% at the postgraduate level. we see the opposite effect for technical skills, where the undergraduate percentage (42.07%) is slightly higher than the postgraduate percentage (40.23%). larger gaps are found for the other two categories. for business skills the undergraduate percentage is 36.37% compared to 43.11% for postgraduate. we see a similar pattern with green skills (undergraduate 30.38%, postgraduate 43.66%). it seems that greater maturity and probably some work experience make postgraduate study of business concepts more appropriate. there are numerous interesting individual results that are obscured when looking at category averages. for example, in the case of the technical skill it architecture / standards the percentage of respondents who believed the skills should be acquired at the undergraduate level was 42.70% compared with 28.20% who believed this skill was appropriately taught at the postgraduate level. other technical skills that respondents believed are better taught as part of the undergraduate curriculum moreso than at the postgraduate level include: software implementation (52.70%, 36.60%); web application servers australasian journal of information systems volume 18 number 2 2014 180 (53.40%, 38.20%); software testing (54.20%, 37.40%); database administration (52.70%, 35.90%). the biggest differences between undergraduate and postgraduate percentages in the other direction can be found in the green skills category were sustainability strategy (35.10%, 52.70%); training for sustainability awareness (31.30%, 50.40%); and sustainability engineering (26.70%, 42.00%). skills missing from the survey and further comments with 85 skills already listed in our survey and five things to consider for each one, we could not include all possible skills and terms for consideration. we believe the length of the survey impacted the number of willing participants and was the direct reason why only 131/163 respondents chose to complete the survey. to address the possibility that we had missed essential skills we asked our participants if there were any skills missing from our survey and where the skill should be acquired. thirty-five participants entered a response to this question in the free text textbox provided. some of the skills we deemed to be equivalent to skills in our survey. for example, we thought “analyzing the current business status” was overly specific and covered by other skills such as business problem solving and/or business planning. similarly, we think that “experience in using ba tools” is covered by case tools, though we appreciate that this old term may imply such tools are for software engineers only. table 5 lists those skills that seemed sufficiently different to those we had included. not that the absence of the skill “software development methodologies (waterfall, agile, etc)” was an oversight in our online survey and technical skill #48 in table 1 was sdlc/development methodologies. we also include any likert scale responses provided regarding the importance, knowledge level or appropriate place to acquire the knowledge if provided by our participant. the first row includes two skills suggested by two respondents and hence importance shows the two different responses. missing skills importance knowledge undergrad pgrad work political awareness/influencing skills 2+3 2 x x modelling/data modelling 3 3 x x basic database querying (not mining) 1 3 x x software development methodologies (waterfall, agile, etc) 3 3 x x x documentation/knowledge sharing and management skills stress management bilingual ? for some cases systems thinking & design thinking impact of systems on the society and their interaction the business analysis process 3 x x x language skills (not oral rather, acuity for ambiguity) extremely important u/p/w 3 x x x rapport building 3 x x x active listening 3 x x x business risk identification and mitigation 3 x x legal and regulatory compliance awareness 3 x x user experience specialist documentation management systems table 5. additional skills identified by respondents australasian journal of information systems volume 18 number 2 2014 181 some respondents provided a detailed description of what they believe the ba role to be, or further information to clarify their viewpoint and responses to the survey items. we present those comments below as a source of insight into some of our respondents, most of whom are highly experience it professionals familiar with the ba role. after each comment we provide some information about the person who made the comment. i expect students would only get a taste of some of these things at undergraduate study level. i also believe that business analysis as a discipline is not something people should go into straight out of university, but that they progress to after acquiring experience in another role (whether it or line of business).i expect students would only get a taste. (40 year old male ba with 17 years experience) unfortunately many of the thinking and understanding skills required to excel as a business analyst aren't taught in universities. this might be because of the considerable effort required to teach somebody else how to think and understand things. only one lecturer has taught me skills related to thinking and understanding. this lecturer has exhausted a respectable effort and it has resulted in my extended capabilities. (18 year old male who wants to be a ba) the ability to see big picture stuff and look at trends and provide data analysis that is useful. so much is spent of finer points that we dont see the big picture. (56 year old male ba with 35 years experience) many of the skills cannot be ‘acquired’ – you have them or you don’t (57 year old male ba with 25 years experience) the successful analyst needs to be able to create solutions out of nothing using whatever resources are available often this means nothing more then excel, access, vba and sql. the majority of analysts cannot participate effectively in discussions because they cannot grasp the data model, or communicate with those who will be writing the sql at the end of the day. (32 year old male ba with 3 years experience) industry specific skills generally can be learned on the job, although it is beneficial to have knowledge/experience at the u or p stage. work experience at u or p is not always possible. more technically-based skills such as bi, network admin, software design etc is better learned at the u or g level, as they are very specialist and detailed. to give you a background of my situation, i completed an undergrad degree in arts informatics, majoring in art history and theory, government and international relations and information systems. my first job was at a financial services institution who was also a wealth management and risk product provider. i went from a policy administrator to business analyst role in 3 years, where i focused on management reporting using bi tools. my second role was as a business performance analyst in another financial services institution similarly representing the business in dealing with it in developing management reporting using bi. my current role now is as a management consultant with an information management / bi focus. i'd like to highlight that i believe that there is a generic set of ba skills (eg communication skills, requirements gathering process and documentation etc) that need to be developed in order to be an effective ba. i think that there are many routes to becoming a ba, and by the same token, there are different types of bas. once these exist, then bas should be able to work across industries. some of the skillsets in the survey above are more for certain australasian journal of information systems volume 18 number 2 2014 182 technical specialists, rather than bas eg i would expect a software developer to do software design, not a ba. (29 year old female) wit and humour; diplomacy and tact; intellectual honesty and integrity -all acquired and continually honed from the early years (e.g. childhood) onwards (51 year old female ba with 28 years experience). the ability to analyse, problem solve and communicate at conceptual, logical and physical levels as appropriate to the phase of the project. (65 year old male with 40 years experience who has been a ba) consistent with our quantitative survey findings, we see that the ba role is indeed a complex one where people skills are essential and must be applied in the specific business context. the ba is more likely to interact with others who need to have the appropriate technical skills, but some understanding of these skills is needed to avoid miscommunication and aid communication with the business stakeholders. future directions & challenges this study sought to answer a number of research questions. as a practical implication of the study, these findings shed light on what should be included in a program of study in australia aimed at business information systems students and with a specific graduate destination in mind, the ba. hence this study focussed on australia with participants being recruited via the acs and therefore there may be some limitations on the generality of the findings to a broader global context. in particular, the high ranking of integrity/honesty/ethics and professional ethics may reflect the code of conduct established by the acs and be a key reason which attracted the individual to become an acs member, though we noted our findings were similar to other studies. the skills measured in our survey were derived from international frameworks (i.e. sfia) and other skills found relevant by researchers. however, to provide more internationally validated results, and to determine if differences exist, in the future we may replicate the study drawing on the association of information systems (ais) or the international institute of business analysis (iiba). certainly our study would benefit from increased participant numbers. we observed that earlier questions were (on average) ranked higher than later questions indicating a potential bias. future studies would need to ensure that the list order is rotated to account for this error. also, we had to discard one-third of our participants because of non-completion which we think may be due to an exhaustion factor given the large number of questions, however this would need to be tested by researching a smaller subsection of completed and not completed surveys. an option may be to reduce the number of questions by removing those skills that were seen to be unimportant in this study. finally, we have some concerns whether our participants understand what was meant by sustainability. perhaps use of the term greenit would have been more meaningful. sustainability is often associated with organisational viability related to competitiveness, not related to preserving the environment. in any case, we believe our findings demonstrate the current lack of agreement, awareness and emphasis in this area that needs to be addressed. it would potentially be interesting to compare our findings with reputable sources and professional bodies that we had not consulted in the development of our survey. initially we performed a comparison with the is2010 curriculum guidelines for undergraduate degree programs in information systems that was developed by the association for computing machinery (acm) and the association for information system (ais) (topi et al. 2010). however, since an is undergraduate degree is not specific to the role of the ba only, we looked for a more specific set of skills and knowledge. we found that australasian journal of information systems volume 18 number 2 2014 183 the iiba had produced a document for this purpose for its members. we performed a post survey analysis of the babok guide which is described as follows9 a guide to the business analysis body of knowledge® (babok® guide) is the collection of knowledge within the profession of business analysis and reflects current generally accepted practices. as with other professions, the body of knowledge is defined and enhanced by the business analysis professionals who apply it in their daily work role. the babok® guide describes business analysis areas of knowledge, their associated activities and the tasks and skills necessary to be effective in their execution. the babok® guide is a reference for professional knowledge for business analysis and provides the basis for the certified business analysis professional™ (cbap®) and the certification of competency in business analysis™ (ccba®) designations. in the babok guide v.2 the knowledge areas (kas) relevant for business analysis are divided into the phases: enterprise analysis; elicitation; requirements analysis; and solution assessment and validation. spanning these phases is business analysis planning which are underpinned by the ka requirements management and communication. while the guide is very useful in providing a flow of tasks (for example, as part of the enterprise analysis phase we have identify business needs, determine solution approach, define solution scope and develop the business case) and does give a good idea of what a ba needs to do, it does not tell us what knowledge is needed to perform these tasks or where that knowledge should be acquired. empowering students with the knowledge, desire and confidence to be drivers of sustainability cannot be done in isolation from the rest of the curriculum. the literature review and survey presented seek to identify what knowledge and skills are required to ensure that they will be successful their careers and responsible citizens. it is important to design a curriculum to allow those identified knowledge and skills to be learned and constructed, progressively via a process of experience and reflection throughout their studies. while not available to use at the time of this study, the literature review of green is by brooks, wang and sarker (2012) might suggest other green skills. gaining a deep appreciation of sustainability and the role the student can play, cannot be done using traditional teaching methods only and may require alternative teaching strategies such as those suggested by the unesco (1998) project on teaching and learning for a sustainable future: experiential learning, storytelling, values education, inquiry learning, appropriate assessment, future problem solving, learning outside the classroom and community problem solving. conclusions the emphasis on graduate attributes (also known as graduate capabilities and formerly better known as generic skills) for is graduates, particularly in the role of bas, is noteworthy. through the use of an industry based survey, this study identified skills relevant to the current ba role and compared respondents’ perception of the importance of and their ability in each of these skills. eighty five (85) 9 http://www.iiba.org/imis15/iiba/professional_development/business_analysis_body_of_knowledg e/iiba_website/professional_development/business_analysis_body_of_knowledge_pages/busines s_analysis_body_of_knowledge.aspx?hkey=d0891e0a-996a-431f-a6f5-a7d644e23a5c accessed 512-12 australasian journal of information systems volume 18 number 2 2014 184 skills were identified and categorised into four skill sets: technical, business, soft and green skills. soft skills were perceived to be the most important and the knowledgeable area for bas, followed by business, green and technical skills. responding to a public debate on sustainability and green issues, our research confirms that there is currently a lack of awareness of the future role the ba is likely to play in this area. our research also shows there is a need for training to improve their understanding about these skills and the implications in their careers. consequently, exploring the area of green is curriculum could be a potential area for future research. we suggest that the set of ba skills and knowledge gaps identified in this study can be used by other researchers, teachers, employers and even students to challenge and change current curricula so that information systems graduates are better equipped to carry out their pivotal decision making role in a world in which business and technology will continue to change rapidly and a world in need of sustainable solutions. references aken, andrew and michalisin, m.d., 2007. "the impact of the skills gap on the recruitment of mis graduates". in proceedings of the 2007 acm sigmis cpr conference on computer personnel research: the global information technology workforce. sigmis cpr ’07. new york, ny, usa: acm, pp. 105–111. alexander, s., 1999. "what’s in a job title? less and less, some say". infoworld, 21(21), p.94. australian computer society, 2011. australian computer society business analyst. available at: http://www.acs.org.au/ictcareers/index.cfm?action=showandconid=businessanalyst [accessed 15-7-2011]. beard, d., schwieger, d. and surendran, k., 2007. "incorporating soft skills into accounting and mis curricula". in proceedings of the 2007 acm sigmis cpr conference on computer personnel research: the global information technology workforce. sigmis cpr ’07. new york, ny, usa: acm, pp. 179–185. brooks, s., wang, x., & sarker, s. (2012). unpacking green is: a review of the existing literature and directions for further research. in jan vom brocke, stefan seidel, & jan recker (eds.), green business process management towards the sustainable enterprise. heidelberg, germany: springer. isbn 978-3-642-27487-9. bullen, c., abraham, t. and galup, s.d., 2007. "it workforce trends: implications for curriculum and hiring". communications of the association for information systems, 20(1), p.34. burns, j., hopper, t. and yazdifar, h., 2004. "management accounting education and training: putting management in and taking accounting out". qualitative research in accounting & management,1(1),pp.1-29. calitz, a.p., watson, m.b. and de v de kock, g., 1997. "identification and selection of successful future it personnel in a changing technological and business environment". in proceedings of the 1997 acm sigcpr conference on computer personnel research. sigcpr ’97. new york, ny, usa: acm, pp. 31–35. crebert, g., bates, m., bell, b., patrick, c. j., & cragnolini, v., 2004. "developing generic skills at university, during work placement and in employment: graduates' perceptions". higher education research & development, 23(2), 147-165. csapo, n. and featheringham, r.d., 2005. "communication skills used by information systems graduates". issues in information systems, vi (1), pp.311–317. australasian journal of information systems volume 18 number 2 2014 185 doucek, p. and novotnỳ, o., 2007. "ict education and requirements for ict graduates in the czech republic" in proceedings of the 2007 computer science and it education conference. pp. 215–31. evans, n., 2004. "the need for an analysis body of knowledge (abok): will the real analyst please stand up." journal of issues in information science and information technology, 1, pp.314–329. fang, x., lee, s. and koh, s., 2005. "transition of knowledge/skills requirement for entry-level is professionals: an exploratory study based on recruiters’ perception" journal of computer information systems, 46(1), p.58. ferguson, e., 2005. "changing qualifications for entry-level application developers". journal of computing sciences in colleges, 20, pp.106–111. huang, h. et al., 2009. "synthesizing it job skills identified in academic studies, practitioner publications and job ads". in proceedings of the special interest group on management information system’s 47th annual conference on computer personnel research. sigmis cpr ’09. new york, ny, usa: acm, pp. 121–128. kennan, m.a. et al., 2007. "is early career job advertisements: a content analysis". pacis 2007 proc, p.51. lee, c.k. and han, h., 2008. "analysis of skills requirement for entry-level programmer/analysts in fortune 500 corporations". journal of information systems education, 19(1), p.17. litecky, c. et al., 2009. "skills in the mis job market". amcis 2009 proceedings, p.255. liu, x. et al., 2003. "an examination of job skills posted on internet databases: implications for information systems degree programs". the journal of education for business, 78(4), pp.191– 196. medlin, b.d., dave, d.s. and vannoy, s.a., 2001. "students’ views of the importance of technical and non-technical skills for successful it professionals". journal of computer information systems, 42(1), pp.65-69. miertschin, s.l. et al., 2006. "developing information technology specializations in growing is environment areas". in proceedings of the 7th conference on information technology education. sigite ’06. new york, ny, usa: acm, pp. 65–70. mutch, a., 1996. "information management: a challenge for business education". international journal of information management, 16(6), pp.445-455. rai, p., vatanasakdakul, s. and aoun, c., 2010. "exploring perception of it skills among australian accountants: an alignment between importance and knowledge." in proceedings of amcis 2010 paper 153. sfia foundation, 2011. skills framework for the information age. available at: http://www.sfia.org.uk/ [accessed july 15, 2011]. simmons, c.b. and simmons, l.l., 2010. "gaps in the computer science curriculum: an exploratory study of industry professionals". journal of computing sciences in colleges, 25, pp.60–65. scott, g., & yates, k. w., 2002. "using successful graduates to improve the quality of undergraduate engineering programmes". european journal of engineering education, 27(4), 363-378. todd, p.a., mckeen, j.d. and gallupe, r.b., 1995. "the evolution of is job skills: a content analysis of is job advertisements from 1970 to 1990". mis quarterly, pp.1–27. topi, heikki; valacich, joseph s.; wright, ryan t.; kaiser, kate; nunamaker, jr., jay f.; sipior, janice c.; and de vreede, gert jan (2010) "is 2010: curriculum guidelines for undergraduate degree programs in information systems," communications of the association for information systems: vol. 26, article 18. australasian journal of information systems volume 18 number 2 2014 186 vashist, r., mckay, j. and marshall, p., 2010. "the roles and practices of business analysts: a boundary practice perspective". acis 2010 proceedings. available at: http://aisel.aisnet.org/acis2010/50. weber, j.e., mcintyre, v.j. and schmidt, m., 2001. "explaining is student and is industry differences in perceptions of skill importance". journal of computer information systems, 41(4), pp.79–83. an information technology tool to support negotiating teams jose a. pino* and jose m. montanana depto. de ciencias de la computaci6n, universidad de chile casilla 2777 santiago, chile e-mail: jpino@dcc.uchile.cl abstract this paper discusses computer-supported large-scale negotiation, in particular, negotiation with advisers. it is claimed that better communication within negotiating teams should lead to longer, more productive sessions than the current ones. to this end, an information technology environment should be provided for the negotiation. the paper introduces shine, a collaborative software system developed at the university of chile. this software has many features to allow rich interactions among advisers belonging to the same team, among negotiators and also between a negotiator and his advisers. emphasis is placed on the design features to enable and ease these interactions. the facilities include wysiwis windows, enhanced electronic mail to send and receive text or video messages with several urgency levels, an evaluation procedure and various ways to state comments and ideas. shine has been implemented as a prototype on sun spare workstations. introduction this paper deals with designing a computer-based system to support negotiations among teams of people. such system should enable users to accomplish their task in a more efficient way than without it. the system is not intended to model the negotiation or try to optimize the outcomes in a game-theory approach, but rather to improve the communication among participants. it is important to note that communication during a negotiation is needed not only among negotiating teams, but within the teams themselves. the objective of the system to be described below is to enable such communication while the negotiation is in progress. this would make the negotiation more efficient because the members of each team would be in * this work has been partially supported by fondo nacional de investigaci6n cientifica y tecnol<5gica (fondecyt), project no. 92-1162, and a grant from synapsis s.a. 17 permanent, close and private communication among themselves as well as being talking with the other parties in a face-to-face setting. this computer-based system can be considered a computer-supported collaborative work (cscw) development. according to ellis et al. (1991), cscw reflects a change in emphasis from using the computer to solve problems to using the computer to facilitate human interaction. challenging issues in the design of good cscw systems are the coordination among members (malone and crowston (1990)), effective collaboration devices and humancomputer interfaces (ellis et al. (1991)). collaborative systems have been applied to brainstorming (stefik et al. (1987)), document writing (fish et al., (1988); pino and fabres (1993)), drawing (bly and minneman (1990)), group decision making (nunamaker et al. (1991)), spreadsheet use (fuller et al. (1993)), folder circulation (karbe et al. (1990)) and software development (hahn et al. (1991)). among their advantages over stand-alone systems, include freer participation, increased productivity, and in the case of remote systems, no burden of personal transportation to a meeting site. negotiation with advisers according to sociologist julien freund, conflicts among groups of people arise because they become hostile over maintaining, reaffirming or restoring their rights (freund (1983)). from the point of view of one of the groups, it is necessary to break the resistance of the members of the other groups. the latin word hostis and the greek word polemics describe this type of rival or enemy, rather than inimicus (latin) or ejzroz (greek), which characterize a private opponent (schmitt (1985)). social conflicts may arise because different groups hold opposed economic, political, national or religious views. for example, an economic conflict occurs when the organized workers of a certain sector ask for salary increases the organized managers consider excessive. an example of a political/international conflict may arise because two governments claim sovereignity over a certain geographical region for their respective countries. in both cases, there may be mediators (e.g., the ministry of labor in the former and the united nations in the latter). a conflict of this kind may be settled by force. alternatively, there may be negotiation, that is, each party agrees to talk in order to accept part of the others' point of view if its own claims are partly accepted as well. a typical negotiation setup is composed of two or more parties (eventually one of them may be a mediator). in the technical literature, there is a distinction between the terms "negotiation" and "bargaining" although in everyday english usage they are considered synonyms. negotiation involves human interactions defying quantitative analysis whereas bargaining refers to situations that can be modeled and analyzed (nierenberg (1973)). both terms are not necessarily disjunct, because a bargaining model can be used within a negotiation (fraser and hipel (1984)). as an example, fraser and hipel describe the holston river negotiations which occurred in the early 70's in the state of tennessee (u.s.a.); they 18 apply conflict analysis to develop a bargaining model which could have been used in these negotiations. according to bacharach and lawler (1981), power is the essence of negotiation, and the negotiating power is subjective power. this means that it is important for a party to convey the impression of controlling significant exchange commodities. in particular, a way to alter the opponent's definition of the power situation is to manipulate information, argument, do bluffing, etc. therefore, each party is best represented by a group head or negotiator and several assistants or advisers. the speaker of each group is the negotiator. the assistants, given their specialization in various fields, provide a negotiator with the necessary expert information to apply good negotiating power tactics. thus, for an international peace talk, for instance, assistants may be experts in subjects such as international law, geography, military affairs, national politics, etc. typically, assistants are seated behind their negotiator. in the holston river example, there were three major participants, each party having consultants and advisers. during a negotiation session, an individual assistant may approach his negotiator and deliver a short verbal comment or quickly pass him a written note. in many cases, that is all the information the negotiator gets from his group during the talk. a negotiator may request a break or a new meeting in order to have enough time to discuss new proposals with his advisers. therefore, the negotiator-assistants interaction during a meeting is poor, and the negotiator may get worried of saying too much because he does not have enough input from the experts of his side. as a consequence, interruptions slow down the negotiation. on the other hand, if a negotiator does indeed give away what it is more than convenient to the other party due to lack of counsel, later on he may be unwilling to give any other concession and this may result in stalling progress in other aspects of the negotiation. ouu nnn bbs (a) (b) figure 1. negotiating room and nine advisers' rooms, (a) the electronic board is on the right wall of the negotiating room; the shaded area represents seats for journalists and public. the diagonal line in each advisers' room represents the tv monitor, (b) this view of the negotiating room shows the board and part of an advisers' room window. 19 an improvement over the previous situation is to furnish a voice communication system linking negotiators and their assistants. this system would complement the traditional voice communication system designed to listen the current speaker, in more than one language if translators are provided. however, if multiple communication channels were available, we could have a better comprehension, since it is well-known that the use of several channels eases the communication among people (bettinghaus (1980)). in this case, the communication channels might be voice, written text, visual icons and video. a computerbased multimedia system could provide these channels and an appropriate software might allow the negotiator to continue listening to the current speaker while asking counsel with a few hand movements over a computer mouse, or quickly see the first opinions of his experts on a recent proposal presented by the opposing party. the negotiator would have the option of interacting with his workstation whenever he thinks that the complexity of the current speech is low enough to do so. thus, the negotiator-assistants interaction could become intense and efficient. an additional shortcoming of the traditional approach to negotiation is that the advisers do not enjoy privacy or isolation to exchange views during the meeting. as a result, the input they can provide to the negotiator is only based on personal opinions. these considerations led to the design of shine (shared interactive negotiation environment). this software system was developed as part of a larger project aimed at investigating the potential of adequate human-computer interfaces to build useful collaborative systems. design of the environment our proposal calls for a redesign of the physical negotiating environment (fig. 1). a separate room is provided for each group of assistants. this room has a window to the negotiating room and a video monitor to receive the audio and close-ups from the speaking negotiators. each assistant is provided with a workstation and a video camera focusing on him. the negotiators remain on the main floor, each with a workstation in front. workstation screens are only visible by their corresponding negotiators. there is a large electronic board, controlled from a terminal by a technician. a negotiator may speak as usual, but he may also send short text messages to designated negotiators by typing on his workstation and send text, pictures or tables to the technician to be displayed on the electronic board. he may also have a contents-rich communication with his assistants. up to seven advisers engage in a permanent face-to-face meeting in their room and they communicate through their workstations among themselves and with their negotiator. they can have private or shared views over text documents and send tables or pictures to each other. the idea of the assistants' reduced participation in the meeting was inspired by an early experience in group dialog reported by sheridan (1971). to have a group of interconnected computer-supported people do a joint task requiring discussion in a face-to-face meeting seems to be a successful arrangement, as reported by nunamaker et al. (1989; 1991) in the case of the group decision-making area. 20 a method to move a stalled negotiation is to generate new ideas in a brainstorming process (fisher and ury (1981)). this can easily be done by the advisers in the privacy of their room, according to this design. a negotiator should be able to ask some or all advisers to have a small brainstorming session about a specific subject while the negotiation is in process. the advisers may send the negotiator written proposals, short telegrams of high priority and evaluations of various kinds previously asked by the negotiator. they may also send a video message consisting of a recording of one of the advisers speaking. the latter type of message has the advantage of letting the negotiator watch the facial emphasis the assistant wants to provide. however, it has the disadvantage the negotiator has to switch audio channels and perhaps miss another negotiator's speech. interface design the human-computer interface of shine was carefully designed considering that first, the users are supposed to be computer novices, and second, the negotiator can not afford much time to interact with his workstation. thus, the user interface is icon-based. the negotiator communicates with two basic paradigms displayed on his screen: a pushbutton panel and a bulletin board. the panel is very simple to manipulate: the user chooses an icon, presses a mouse button and the action occurs. for example, if an icon signals that mail messages have arrived, choosing it causes a window to be opened showing the contents of the messages. the bulletin board is of simple operation as well and it is explained in the next section. display of results is also visual. for example, evaluation results are shown using faces, with obvious meanings: smiling, annoyed, etc. figure 2 shows the negotiator's workstation screen. the control panel is a private window shown on the left hand side. in its upper part, there are icons depicting the assistants; the first one represents the whole group of advisers. these icons are useful to selectively ask the services explained below. an arbitrary number of advisers can be chosen for a service by pointing with the cursor over each icon in turn and depressing the first mouse button (selection is indicated by an icon shade change). a second button depression deselects the corresponding icon. people icons are also useful to show their current state and keep the negotiator informed. in fig. 2, adriana is out of the room while alejandra is in the assistants' room but she is busy with another task. besides a bulletin board and a control panel, an adviser screen includes a strict wysiwis window (stefik et al. (1987)) called ours. this window allows advisers to have a place to propose tentative writings, exchange comments, etc. the system automatically inserts a cursor with the author's name whenever an adviser begins to write in this window. once the writing is finished, the author's name remains as a signature in the last place occupied by the cursor. 21 j o! '5 0) ^ t x^ & 0> j_, > o iii ts si* o) > ms; *"« = (/) oq^-q = 151 i © j^ ii . 5 £ <d >, n o **s +3 oj "*~-d * = 3*^in sin 01 q rt 3 s«•a i o 5 »*-i o i>< in 2j 3> e dq ©: 1̂ .2 i 22 send maked send un poll po i'd like you evaluate the 1 min 2 min gjog)' 0 5 min definitions cancel a) we close a contract for 2 years, with yearly salary increases of 3* for'95 and '96. b) we close a contract for 3 years, with yearly salary increases of 2.5* for'95/96 and'97. c) we close a contract for 3 years, with yearly salary increases of 2% for '95 and '96, and a yearly salary increase of 505 of the '95 productivity growth rate for '97 and '98. ( roberto) figure 3. dialog box at the negotiator's screen used to ask for an evaluation. the bulletin board the bulletin board is a non-strict wysiwis window shared with the advisers. there are pinned down items on this board which can be opened by the negotiator or any adviser. if one person opens an item by selecting its pin, another window pops up only on his screen showing the contents of the item; selecting that pin again closes the window and a "closed" item icon appears on the board. figure 2 shows three closed items: a request, an agreement and a note, and one open evaluation. the closed request is ready. the open evaluation of fig. 2 illustrates the type of service the negotiator may need from his advisers. the assistants are asked to vote on one or more texts. when the negotiator presses the evaluation button on the control panel, a dialog box appears (shown on figure 3) and he writes the texts on which he wants feedback, chooses the urgency of the matter (explained in the next chapter), selects the advisers and then presses the send maked poll button on the dialog box. thereafter, the box disappears. the "maked poll" is the way of doing this service in which the negotiator formats the options himself. the dialog box has a vertical line separating two columns: the left one to contain text and the right one to place urns. urns are placed in front of the text to be evaluated, simply by depressing the mouse button when the cursor is in the right place: the system automatically inserts an urn; in fact, in order to further help the user, the cursor changes shape to a shaded urn when the cursor is in the right column. an alternative way of doing the evaluation is to ask one of the advisers to prepare the questions for his colleagues. in this case, the negotiator simply writes an outline of the options to be evaluated and presses the send unmaked poll button before choosing the helping assistant. 23 once a maked evaluation reaches the chosen advisers, a window appears on their workstation screens. perhaps they exchange views briefly and then each one casts his votes. the vote format is very simple: only five choices are available and range from excellent to unacceptable. an adviser simply selects a "face" representing his choice and drags it to the urn at the side of the text being evaluated. when finished, each adviser presses a send button. at any time during this process, a team member can open the evaluation item and see the votes already cast. the open evaluation in fig. 2 shows such results: the faces at the right hand side represent the vote average each option received; clicking on one face shows the individual votes for the corresponding option. (request ' send edit definitions vfe 3 min subject: how about increases in fringe benefits? i agree, we should offer this beforehand and reject cash increases. in any case, our best estrategy should be to talk about the whole package.<<eduardo> yes. we should compute an updated package cost and talk about cost related to workers' productivity. jose should be able to keep this updated cost with a spreadsheet<edgardo> (eduardo) figure 4. wysiwis window provided to the assistants working on a request. another service available for the negotiator is the request. it is a petition to one or more advisers to work on a subject and provide an opinion or position. this feature allows the negotiator to ask his advisers to begin a brainstorming session to generate optional solutions to a particular issue. the system provides a wysiwis window similar to the "ours" window to the selected assistant (shown on figure 4). when they are finished writing, one of them presses a send button. since it is a shared service, the system asks for confirmation, and once received, the shared window is gone and a "ready" signal appears in the respective closed item of all bulletin boards of the negotiating group. communication using shine important features of communication in the context of negotiation are the specification of task urgency by the sender and the perception of this specification by the recipient of the corresponding message. in requests and evaluations, the negotiator specifies one out of three urgency levels when completing the dialog box. the three levels of urgency have pre-assigned suggested completion times, and when the task related window appears at the advisers' workstations, a sand clock in the upper part of the window depicts the remaining time to complete the task. therefore, it is easy for the negotiator to choose an icon illustrating the very urgent concept to ask the assistants maximum priority to get a task done quickly, as it is 24 the case of the open request in fig. 4 which was a very urgent task with a expected reply time of three minutes. private communication is done with mail and telegram messages. the difference between them is again, the urgency the sender assigns to them. these messages can be sent from the negotiator to the advisers and viceversa and among negotiators. in the case of negotiators, separate icon buttons are provided for mail and telegram messages from/to negotiators and advisers, in order to avoid confusions (figure 2). these four buttons are reduced to two team mail and telegram in the case of advisers, since they are not permitted to communicate with other negotiating teams. the operation of mail and telegram messages is similar; they will be referred as messages in what follows. the receiver selection and message sending has a similar procedure to the one needed for sending a request. if a negotiator chooses to send a message to another negotiator, shine automatically opens another window with the pictures of the negotiators and selection is enabled. this window is also automatically closed when the operation is finished. reception of a message is signaled by a mark at the corresponding icon in the control panel. alternatively, each workstation can be configured using the definitions button of the telegram or mail window to display messages as soon as they arrive. immediate display of telegram messages is recommended in the case of the advisers' screens in order not to miss any urgent message from the negotiator. definitions buttons are provided in all other types of shine windows to accommodate various preferences or situations. to read or write messages, a user simply chooses the mail or telegram icon button on the control panel. a window is open displaying the first unread message. next and previous buttons on the window provide the controls to read other messages. another button labelled cancel lets the user close the window. three other buttons write, edit and send require further explanation. the write option of the message window clears the text part of the window and enables the user to type a message. edit provides several services to easily prepare the message. edit has a menu with options to paste a text from a buffer, which has previously been copyed from another text, include the contents of a file and read from clipboard. the clipboard is a private window, and the text transferred to the message window can be all the contents of the clipboard or only a part of it, as chosen with the definitions button of the clipboard window. other options of the edit menu enable the user to delete a part of the text been written. undo, of course, reverses the last action. other edit options are useful also when reading messages: write to clipboard , save to file , copy to buffer. the edit environment is also provided in the evaluation, request, agreement and share windows (the last two are described below). edit services are also provided with the function keys on the left keypad of the keyboard in the open windows implementation for compatibility with other applications running on this environment. once a message is prepared, it can be sent using the send button. again, the activity can be customized with the definitions button. for instance, it can be specified that the system asks confirmation before sending a message. 25 an adviser message may also be a video recording. his window has an additional button called video ; when chosen, a familiar vcr control panel is drawn on the window: record, play, pause, rewind and advance forward have the obvious meanings. the negotiator gets the control panel and a time duration indication when reading the message (the record button is disabled); selecting the play button creates a window where the video is shown. shine also provides a tool to support face-to-face discussion and brainstorming. the share button in the control panel opens a shared window for people in the same room: negotiators or advisers. after choosing the share button, the user chooses the people with whom the sharing is to occur (it may be all). the shared information is saved when the person who initiated the window selects the unshare button in the window. previous share windows can be viewed again, saved in private files, and deleted using the read button of the window. negotiators also have a tool to improve their presentation of ideas: the electronic board. this device is an additional visual channel which may be used to present complex material in a redundant way (bettinghaus (1980)). a negotiator wishing to place text or a drawing in the electronic board simply selects this item to be displayed and sends it to a technician. the technician appropriately distributes the various displayed items on the board. the negotiators can remotely point to a place in the electronic board using a telepointer (stefik et al. (1987)). another related service is the extraction of a piece of text being currently displayed. to access these features, the user must select the electronic board button of the control panel; a menu offering the magnifier and marker choices pops up. the marker activates the telepointer, with the negotiator's name; its movement is controlled by the negotiator's mouse and it is cancelled off by pressing the mouse select button. the magnifier option opens a window on the negotiator's screen showing an outline of the electronic board; the cursor shape is now a magnifier. the mouse select button, in this case, opens another window showing a part of the board pointed by the magnifier cursor. this latter window has normal font size letters and vertical and horizontal scrolling bars; it has edit and cancel buttons with the usual meaning. further bulletin board items figure 2 shows another type of item pinned down on the bulletin board: an agreement. this is a presentation made by the assistants to the negotiator. it may be a suggestion or a general proposal made by all the assistants. to write it, one assistant chooses the agreement button from his control panel and writes it similarly to a message. the negotiator also has a mechanism to make a general remark to the advisers: it is the note which gets also pinned down on the bulletin board. to write it, the negotiator must select the typewriter icon on the top of the bulletin board and gets a window resembling a message window. comments can be placed by the advisers on any item of the bulletin board. they simply select the item on the bulletin board and then select the comment button. at this point, they are allowed to write a comment which is afterwards concatenated with other previous 26 comments. these comments can be viewed by choosing the comment icon in the respective bulletin board item window. face votes can also be placed over requests, evaluations, notes and agreements. a user drags one of the faces shown on the lower part of the control panel and drops it over the desired item. the system will place the person's name under the face on the control zone of the item. any comment or vote appearing on bulletin board items are signaled in their closed (pinned down) format. for example, in fig. 2 the closed request has votes and comments. history management as it was mentioned, share sessions can be reviewed afterwards. read messages are not automatically deleted by the system and thus they can be reviewed as well. the log feature in the control panel displays when selected a listing of all shine interactions of the negotiating team including the time of occurrence. implementation a prototype of the system was developed using networked sun spare workstations running sunos 4.1.2 and open windows 3.0. internal communication was implemented with rfc (bloomer (1991)), which uses a client-server architecture. messages from all users are transmitted via one "message router", which is a software process running in a machine. messages from the user programs to the message router are sent using the asynchronous onc rfc technique. messages from the router to the user programs use a simplified asynchronous follow-up rfc strategy (bloomer (1991)). this approach allows the router to handle communication errors, avoiding unwanted messages blocking. another function of the router is to enforce the permitted communication links: among negotiators or among team members only. finally, the router guarantees the correct sequence of messages through the network. the interface has been written using guide 3.0 (sun microsystems (1991)) and rpcgen (sun microsystems (1988)). the programming language was standard c. the objective of this prototype is to show the capabilities of the system and let some people involved in real negotiations try it. the prototype accepts up to 72 users, but it has been tested only up to 12. each user requires more than 3 mb of core. another restriction of this implementation is that all users must sign on at the beginning of the negotiating session in order to maintain data consistency. this prototype has no built-in tight security measures which a system applicable to real negotiations should have. lastly, the approach taken to issues such as performance, reliability and data management is simple and not appropriate for a final software product. 27 conclusions the design of a collaborative multimedia system to assist large-scale negotiation has been described. a prototype implementation has been tested but it has not been used in real negotiations yet. shine provides multi-channel communication features between a negotiator and his individual assistants, between a negotiator and the group of advisers, and among negotiators. this communication can occur while a negotiation session is in progress. the system is not bound to any bargaining strategy or model and thus, a negotiating team may choose a suitable tactical approach at will. the divergent tasks (as defined by nunamaker et al. (1991)) of sending messages, making evaluations, doing brainstorming and writing comments are supported by the system. the hard problem of reaching consensus in distributed environments (chen et al. (1994)) has been circumvented by seating people who need to do convergent tasks in a face-to-face environment. the advisers are provided with enough privacy to have their discussions, removing in this way a physical restriction of computerless negotiations. the negotiators get tools to communicate ideas efficiently through the electronic board. the system implements these functions through an intuitive, quick-to-use interface. simplicity of use is one of the main design objectives and perhaps many negotiators may get used to the system. in the worst case, at the beginning, one assistant may sit beside the negotiator and handle the workstation, showing him all interesting input the other assistants may provide him. in the long run, even the most technology-opposed negotiators may feel attracted to ask for services from his collaborators through the workstation. it is not clear the usefulness of the video-voice messages. it may become valuable in some cases and not in others. other tools may be ignored by a negotiating team: they may not use agreements and notes at all, and decide to use telegrams only for very important messages, for example. in this same flexibility of tools vein, the definitions feature makes possible adaption to different preferences and cases. acknowledgement alejandra galvez helped in the production of drawings used in this paper. our appreciation to two anonymous referees whose comments helped to improve a draft of the paper. references bacharach, s., and lawler, e. (1981) bargaining: power, tactics and outcomes. josseybass, san francisco, ca bettinghaus, e. (1980) persuasive communication, holt, rinehart and winston, new york, ny, 2nd. ed. bloomer, j (1991) power programming with rfc. o'reilly & associates, inc. sebastopol, ca. 28 bly, s., minneman, s.(1990) commune: a shared drawing surface. acm conf. on office information systems, (april), cambridge, ma, pp 184-192. chen, h., hsu, p., orwig, r., hoopes, l. and nunamaker, j.f.(1994) automatic concept classification of text from electronic meetings. communications of the acm, vol. 37, no. 10, pp56-73. ellis, c.a., gibbs, s.j. and rein, g.l.(1991) groupware: some issues and experiences. communications of the acm. vol. 34 no. 1, pp39-58. fish, r., kraut, r., leland, m. (1988) quilt: a collaborative tool for cooperative writing, sigois bulletin, vol. 9 nos. 2 & 3, pp30-37. fisher, r. and ury, w.(1981) getting to yes. houghton-mifflin. boston, ma. fraser, n. m., and k. w. hipel (1984) conflict analysis: models and resolutions. north-holland. new york, ny. freund, julien (1983) sociologie du conflict. presses universitaires de france. paris, france, (in french). fuller, d., mujica, s., pino, j. a.(1993) the design of an object-oriented collaborative spreadsheet with version control and history management, acm symposium on applied computing (sac 93), indianapolis, ill (february), pp416-423. hahn, u., jarke, m. and rose, t.(1991) teamwork support in a knowledge-based information systems environment. ieee trans, on soft. eng,. 17(5), pp. 467-482. karbe, b., ramsperger, n., weiss, p.(1990) support of cooperative work by electronic circulation folders. acm conf. on office information systems (april), cambridge, ma, pp!09-l 17. malone, t.w. and crowston, k.(1990) what is coordination theory and how can it help design cooperative work systems. in halasz, f. (ed): cscw 90: proceedings of the conference on computer-supported cooperative work. los angeles, ca (oct.), pp357-370. nierenberg, g.i. (1973) fundamentals of negotiating. hawthorn books. new york, ny). nunamaker, j.f., vogel, d., heminger, a. and martz, b.(1989) experiences at ibm with group support systems: a field study. decision support systems, vol. 5, pp!83196. nunamaker, j.f., dennis, a., valacich, j.s., vogel, d. and george, j.f.(1991) electronic meeting systems to support group work. communications of the acm, vol. 34, no. 7,pp40-61. pino, j. a., fabres, e.(1993) design of the user interface of a collaborative text writing system, 5th. int. conf. on human-computer interaction (hci 93), orlando, fl (aug.) in: salvendy, g. and m.j. smith (eds.): human-computer interaction software and hardware interfaces", elsevier, amsterdam, pp. 50-55. schmitt, carl (1985) der huter der verfassung. 3. aufl. duncker & humblot. berlin, germany, (in german). sheridan, t. (1971) technology for group dialogue and social choice, afips fall joint computer conference (nov.), las vegas, nev., afips press, reston, va, pp327-335. stefik, m., foster, g., bobrow, d., kahn, k., lanning, s., suchman, l.(1987) beyond the chalkboard: computer support for collaboration and problem solving in meetings, communications, of the acm, vol. 30, no. 1 (jan.), pp32-47. sun microsystems, inc.(1991) open windows developer's guide 3.0 user's guide. mountain view, ca. sun microsystems, inc.(1988) network programming. mountain view, ca. 29 the australasian journal of information systems (ajis) began in 1993 and was based on the simple realisation that there was a need for an academic ‘outlet’ for is research in australia australasian journal of information systems volume 17 number 1 2011 5 dimensions of autotelic personality and their effects on perceived playfulness in the context of mobile information and entertainment services felix b. tan business information systems department aut university, new zealand email: felix.tan@aut.ac.nz jacky p.c. chou school of computing and mathematical sciences aut university, new zealand email: mr_lo8@hotmail.com abstract autotelic personality represents an important individual construct in flow theory yet little is known about its underlying dimensions. csikszentmihalyi (1988) suggested that the higher the autotelic nature of an individual, the more likely for him/her to experience flow. this study explores autotelic personality, its underlying dimensions and its relationship with perceived playfulness in the context of mobile information and entertainment services (mies). this study found autotelic personality to be an important quality in order for users to experience perceived playfulness in the mies context. several underlying dimensions of autotelic personality were identified, including personal innovativeness, self efficacy and control. introduction autotelic personality characterizes a person “who is able to enjoy what he is doing regardless of whether he will get external rewards from it and who thus is more likely to experience flow for a given activity” (hoffman & novak, 1996). flow represents an optimal experience when the challenges of a situation match the skills of the participant (csikszentmihalyi, 1988). flow is considered important in many technology settings including www, electronic library, e-commerce as well as computer-mediated environments (ghani, 1991; hoffman and novak, 1996; koufaris, 2002). in his study of happiness, csikszentmihalyi (1988) identified what an ‘autotelic’ personality is – a person who sets their own goals, whether short-term or long-term, and then has great fun in achieving them. as csikszentmihalyi stated: “...the complexity of a flow activity is limited by the degree of challenge it can provide, and by the willingness and “creativity” of the person to create challenges in an activity. a person who can do this well, or who has the ability to enter a flow state relatively easy, is said to have an “autotelic personality” although autotelic personality was identified as an essential part of the flow model by csikszentmihalyi, very little research has explored its underlying dimensions (finneran and zhang, 2005; nakamura and csikszentmihalyi, 2002). this study treats and explores autotelic personality as a second-order construct reflected by four first-order factors: personal innovativeness; self efficacy; control; and focused attention. mailto:felix.tan@aut.ac.nz mailto:mr_lo8@hotmail.com australasian journal of information systems volume 17 number 1 2011 6 the convergence of mobile commerce and internet technologies has promised users unprecedented convenience and greater enjoyment. in recent years, mobile information and entertainment services (mies) have been gradually gaining popularity among mobile phone users (garcia-macias et al., 2003; baldi and thaung, 2002; van de kar et al., 2003). research into mobile internet to date has by and large focused on extrinsic use of these services, for example, productivity and usefulness (cheong and park, 2005; pedersen 2005; wu & wang, 2005) and has largely ignored some important end-user characteristics. to better understand users’ acceptance of mies, we argue that it is equally important to examine an intrinsic motivator “perceived playfulness”. several studies have demonstrated the importance of this construct especially in the wider internet context (chung and tan, 2004; lee et al., 2005), online retailing (ahn et al., 2007), web portals (lin et al., 2005) as well as mobile internet services (nysveen et al., 2005; fang et al., 2006). these studies agree that perceived playfulness is an important factor motivating users to use technologies like the internet, online retailing sites and mobile internet services – confirming when the individual’s perception that attention is focused on the interaction, is curious during the interaction, and finds the interaction intrinsically enjoyable or interesting, the more likely the individual is to use that technology. perceived playfulness was proposed by moon and kim (2001) based on the theory of flow (csikszentmihalyi, 1988). individuals experience flow when they are unconsciously engaged in an activity such that s/he becomes so engrossed thereby losing the sense of self (chung and tan 2004). these authors contend that playfulness occurs when individuals experience concentration, curiosity and enjoyment – concepts that are consistent with flow theory. however, some researchers have recently pointed out that individual differences have been ignored in many flow related studies (finneran and zhang, 2005; nakamura and csikszentmihalyi, 2002). therefore to gain a better understanding of the role of autotelic personality in mies, it is important to identify those individual differences likely to be related to autotelic personality and which contribute to the emergence of perceived playfulness. this study contributes to both research and practice. by focusing on the enduser characteristics, this work extends our understanding of the individual differences determining users’ perceived playfulness as dimensions of one’s autotelic personality in the context of users’ intrinsic use of technology. in terms of practice, a clearer understanding of the individual differences in users’ personalities has wide implications for market segmentation and providers of mies should pay attention to the unique needs of different user groups. literature review and research model over the past few years, progress in mobile internet services has been remarkable, both in terms of technology and applications development (shim and shim, 2003). the introduction of real-time video-calling, mobile tv, multi media services and many other exciting offerings has seen the realization of faster mobile internet services through third generation mobile networking technologies (3g). mobile internet service is considered as one with high potential for its ability to carry out multiple services and to be truly convenient from a user’s point of view. among these services, mobile information and entertainment services (mies) are gaining in popularity among mobile phone users (garcia-macias et al., 2003; baldi and thaung, 2002; van de kar et al. 2003). with increasing popularity, companies providing mies will therefore benefit from sound conceptual and empirical research that enhance the understanding of the factors critical to user acceptance of mies. we adopt van der kar et al.’s (2003) definition of mies as “the delivery of information and entertainment from specially formatted content sources (internet sites, sms, mms, etc.) via the mobile telecommunication network to a mobile user” (p. 413). according to the authors, there are four basic functions mobile services provide – communication, transaction, information and entertainment. unlike other types of mobile internet services, the focus of mies is on the delivery of information and entertainment, made possible by recent developments in mobile telecommunications australasian journal of information systems volume 17 number 1 2011 7 technology. early mies were based on wireless application protocol (wap). more recently, a variety of services including short message service (sms)-based information services and now multimedia message service (mms)-based information services are available. as suggested earlier, research into mobile internet services to date tends to focus on the extrinsic use of these services and has largely ignored important intrinsic uses like perceived playfulness which is based on the theory of flow (csikszentmihalyi, 1988). flow theory and perceived playfulness flow represents an optimal experience when the challenges of a situation match the skills of the participant such that s/he becomes unconsciously engaged in the activity as though losing all sense of self (csikszentmihalyi 1988). flow experience can usually occur in structured activities such as games, ritual events, sports, artistic performances and etc. the flow theory assumes that the world is essentially in a state of chaos and that humans are intrinsically motivated to seek out those experiences that add order to consciousness to account for this chaos. therefore, humans will tend to seek out those experiences with the most opportunity to achieve a flow experience. the theory assumes that it is within the ability of the subjective conscious to control, or provide order to, external stimuli. it claims that a flow state occurs when the challenges of a situation match the skills of the participant. if the challenges of a certain situation are above the participant’s skills, then participants will perceive anxiety. conversely, when the skills of the person are higher than that of the challenges of the situation, boredom will result. the importance of flow is recognized in various technology settings, such as personal computers, e-commerce, www, electronic library and computer–mediated environment (cme) (ghani, 1991; koufaris, 2002; hoffman and novak, 1996; trevino and webster, 1992). one of the most widely employed framework of it/is adoption is the technology acceptance model (tam) proposed by davis (1989). tam proposed two major determinants of user’s intention toward it/is usage: perceived usefulness and perceived ease of use. their importance has been widely supported by many technology acceptance studies. however, one of the weaknesses of tam lies in the fact that it is largely extrinsically focused. on the other hand, flow as an intrinsic motivator has attracted researchers’ attention in it/is studies in recent years. one of the most phenomenal trends in personal computer usage during the last ten years has been the internet. a group of researchers have employed tam to investigate user acceptance of internet-based technologies, such as email, virtual stores, online-shopping and general information searching websites (trevino and webster, 1992; chen et al., 2002; bhatnagar et al., 2000; chung and tan, 2004). these studies validated and extended the original model, providing a richer and more comprehensive picture of tam to explain user acceptance of information systems. moon and kim (2001) extended the tam in the context of world wide web and found that perceived playfulness had a significant positive relationship with attitude toward using and behavioral intention. they also suggested people use the internet not only for utilitarian purposes but also for leisure and recreation. based on the theory of flow, they consider playfulness as an intrinsic belief or motive, which is shaped by the individual’s experience with the environment. a new intrinsic motivation factor—perceived playfulness, is introduced into the original tam. moon and kim concluded that it is important for developers to include intrinsic and extrinsic motivational factors in user interface design, thus helping to improve user acceptance. chen et al. (2002) supported this claim and found playfulness to be an important factor motivating users to use a virtual store. we adopt moon and kim’s (2001) definition of perceived playfulness: “the extent to which the individual perceives that his or her attention is focused on the interaction with the world-wide-web; is curious during the interaction; and finds the interaction intrinsically enjoyable or interesting” australasian journal of information systems volume 17 number 1 2011 8 several researchers have studied intrinsic motivator in the context of mobile internet services (nysveen et al. 2005; fang et al. 2006). in their study of four mobile services, nysveen et al. (2005) found enjoyment plays an important role in determining user acceptance of these services. fang et al. (2006) categorized mobile tasks that can be performed on handheld devices into three broad types: general task, transactional task and gaming task. they found perceived playfulness is significant when users are performing gaming tasks. individual differences and autotelic personality individual differences refer to factors such as personality, situational, and demographic variables that influence user’s beliefs about and use of information technology. in the context of flow, webster and martocchio (1992) studied microcomputer playfulness and suggest microcomputer playfulness act as an individual’ tendency to interact spontaneously, inventively and imaginatively with microcomputers. it is a situation specific individual characteristic, which represents a type of cognitive playfulness. moon and kim (2001) considered perceived playfulness as an intrinsic belief or motive, which is formed by an individual’s subjective experience with is/it. hence, identifying those individual characteristics that lead to perceived playfulness may provide insight into this (stable) individual belief construct. this study also explores whether these individual differences are likely to be the dimensions of autotelic personality. it draws upon previous research on flow as perceived playfulness is fundamentally based on flow. prior work related to the state of flow with information technologies has highlighted four key factors that influence individuals to experience flow – namely:  personal innovativeness (aggarwal and karahanna, 2000; finneran and zhang, 2005),  self efficacy (ghani et al., 1991; hoffman and novak, 1996; novak et al., 2000; lim, 2001; koufaris, 2002),  control (ghani et al., 1991; trevino and webster, 1992; webster et al., 1993; ghani and despande, 1994; webster and ho, 1997; chen 2000; koufaris, 2002; chung and tan, 2004),  focused attention (trevino and webster, 1992; webster et al., 1993; hoffman and novak, 1996; novak et al., 2000; koufaris, 2002; chung and tan, 2004). the information systems (is) and marketing literature on intrinsic motivators was drawn to identify these underlying dimensions. these dimensions were mainly ones found to be important to users experiencing playfulness, enjoyment, flow and engagement in the literature. these are described below. personal innovativeness flow experience usually occurs in structured activities such as games, ritual events, sports, artistic performances, etc. (csikszentmihalyi 1988). it does not normally occur in everyday life because challenges and skills are rarely balanced. however, even if skills and challenges are balanced, it does not guarantee a flow experience occurring. this is because activities only provide the challenges; it is still up to the individual to recognize the challenge, provide the skills, and extract enjoyment from the activity. therefore, challenge is more related to the perceived complexities provoked by the activity rather than the individual per se. as csikszentmihalyi (1988) stated; the complexity of a flow activity is limited by the degree of challenge it can provide, and by the willingness and “creativity” of the person to create challenges in an activity. because of this reason, personal innovativeness introduced by agarwal and prasad (1998) is deemed important in the context of our study. personal innovativeness in the domain of information technology is conceptualized as an individual trait reflecting a willingness to try out any new technology. furthermore, agarwal and karahanna (2000) provide empirical support of its influence on cognitive absorption, which is a construct similar to australasian journal of information systems volume 17 number 1 2011 9 flow. in this regard, pagani (2004) suggests that individual innovativeness can be seen as a willingness to adopt 3g multi-media services. self efficacy self efficacy is similar to skill which has been well studied in research on flow (koufaris, 2002; ghani et al. 1991; hoffman and novak, 1996; novak et al. 2000). recall that the most important condition for a flow state to occur is when the challenges of a situation match the skills of the participant. prior research has drawn distinction between general self-efficacy and computer selfefficacy (compeau and higgns, 1995). while the former being an overall judgement of an individual on efficacy across multiple computer application domains, the latter represents the judgement on specific task in the domain of general computing. agarwal et al. (2000) pointed out that there is significant support for a relationship between self-efficacy and individual beliefs about it. their result suggests that software specific self-efficacy will have a stronger effect than the initial general self-efficacy due to the “carryover” effect, i.e. the accumulated application specific self-efficacy will eventually displace the effects of initial belief with the passage of time. because of this reason, computer self efficacy has been preferred over skill as the latter may convey the meaning of general competency on everyday tasks. control and focused attention hoffman and novak (1996) developed a theoretical model of flow within the hypermedia context. in this model, challenges, skills and focused attention were modelled as the primary antecedents of flow. other secondary antecedents (interactivity and telepresence) were also added in accordance to the literature of hypermedia. the consequence of flow leads to increased learning, perceived control, exploratory behavior and positive experience. however, their earlier work was exploratory in nature therefore all the hypothesized relationships were not empirically tested. novak et al. (2000) later revised the original model and added control as a primary antecedent of flow. the revised model was then tested and results showed all these four antecedents exert positive and significant impact on flow. a somewhat interesting finding is that they modeled control and skill together as a higherorder construct (i.e. skill/control). however, a distinction should be made between these two control capturing an individual’s perception that s/he exercises control over the interaction with environment (webster et al.. 1993), whereas self efficacy (skill) is the judgement on specific task in a specific domain prior to that interaction. chung and tan (2004) in their study also proposed focused attention and control as two individual cognitive aspects that lead to perceived playfulness. from the review of the relevant literature, the following research question is developed: does autotelic personality (constituted by personal innovativeness, self efficacy, control, and focused attention) influence perceived playfulness in the context of mobile information and entertainment services? figure 1: research model australasian journal of information systems volume 17 number 1 2011 10 figure 1 presents the research model. from the flow literature and the discussion above, we contend that (i) autotelic personality is a second-order reflective construct consisting of the following individual differences – personal innovativeness; self efficacy; control; and focused attention, and (ii) that these characteristics are its first-order reflective indicators that individually and collectively influence the perceived playfulness experienced by the user. as depicted in figure 1, autotelic personality is portrayed as a multidimensional construct. it is considered a reflective construct for several reasons (petter, straub and rai, 2007): (i) its indicators manifest the higher-order construct and as such the direction of causality is from the construct to the reflective indicators (see figure 1); and (ii) its measures are interchangeable and have a common theme (see appendix for the questionnaire). the study therefore proposes that: p1: autotelic personality is a second-order reflective construct consisting of personal innovativeness, self efficacy, control and focused attention, p2: autotelic personality is positively related to perceived playfulness. research methodology sampling the sample consisted of students who attended information system and business lectures at a new zealand university. students from six classes were invited to complete the questionnaire (see appendix). students are appropriate target sample because they are likely to be literate and familiar with mobile internet, and are likely to have experience using mobile internet on a regular basis. familiarity with using mobile internet services permit attitudes and beliefs to be measured based on direct behavioural experience (davis, 1993). a total of 186 questionnaires were returned. incomplete questionnaires were discarded, leaving 149 usable samples. most of these were from respondents who claimed not to have used mies at all. no significant differences were found to exist between the group who claimed not to have used mies and the group who did. of those who did 94 were males and 55 were females aged mainly between 21-30 years (60.4%) and 20 years or less (30%). most have an undergraduate degree (72.5%) and vary considerably in terms of years of computer experience – with 32% 8 or more years experience, 23% 6-7 years, 22% 4-5 years and 15% 2-3 years. on average, most of the respondents used mies for no more than an hour a day (63%). in terms of phone categories, about 60% of all the respondents specified they have wap-enabled mobile phone or general packet radio service (gprs) phone. respondents holding a 3g mobile phone came second in the list, with a significant proportion of 12% among the respondents. users of the code division multiple access (cdma) mobile phone are relatively few, only 7% compared to other phone categories. at the same time, 21% of all the respondents are not sure about the phones they are using. when asked whether their phone provides a feature with which to access mies services, 83% of the respondents answered “yes” to this question. this indicates the wide availability of mobile internet capable phones in the new zealand market. mms services were the most popular among the respondents with a total of 38% indicating using this mies most recently. sms-based services (i.e. ringtones, subscription-based information alerts and sending/receiving text messages) were the next most recently used mies. the numbers of respondents using mobile email, information searching and mobile downloading most recently were 9%, 10% and 17% respectively. australasian journal of information systems volume 17 number 1 2011 11 measures in terms of measurement, all items were constructed as agree-disagree statements on a seven-point likert scale, which is consistent with the scales applied in prior research. since the variables of interest have been previously validated under different contexts, mirroring the same items in a new context is straightforward. to ensure measurement reliability, items validated in previous research have been used (see table 1). the measures for personal innovativeness, self efficacy, control, focused attention and perceived playfulness were taken from previous is research and were modified to suit the mies context. adopting the same measures as others instead of creating new measures enhances the comparability of the paper with others, specifically when existing measures already become a field standard (churchill, 1999). variable source personal innovativeness agarwarl and prasad (1998) self efficacy hung et al. (2003) control agarwal and karahanna (2000) focused attention webster et al. (1993) perceived playfulness moon and kim (2001) table 1: measurements items data analysis and results the data analysis technique employed in this study is partial least squares (pls). pls is a recent technique that generalizes and combines features from principal component analysis and multiple regressions (thompson et al. 1995). the pls approach provides a means for directly estimating latent variable component scores. it is a technique comprised of measurement and structural models (gefen et al. 2000). the aim of testing the measurement model is to specify how the latent variables are measured in terms of the observed variables, and how these are used to describe the measurement properties (validity and reliability) of the observed variables. the structural model investigates the strength and direction of the relationships among theoretical latent variables. autotelic personality as a second-order factor can be approximated using various procedures. one of the easiest to implement is the approach of repeated indicators known as the hierarchical component model suggested by lohmöller (1989, pp. 130-133). in essence, a second-order factor is directly measured by observed variables for all the first-order factors. this procedure works best with equal numbers of indicators for each construct. measurement validation to measure internal consistency of a given block of indicators, internal composite reliability (icr) scores were obtained through pls-graph to assess the reliabilities of each latent variable. where the internal consistency of any latent variable exceeds 0.70, this indicates tolerable reliability (fornell and larcker, 1981). as shown in table 2, all latent variables in our model have internal consistencies greater than 0.7, indicating all constructs have high reliabilities. in addition, the average variance extracted (ave) of each latent variable in our model are more than 0.50 (table 2), guaranteeing a more valid variance explained than error in its measurement (fornell and larcher 1981). two validities were captured in the measurement model: convergent validity and discriminant validity. essentially, they estimate how well the measurement items relate to the constructs. pls performs confirmatory factor analysis to establish factorial validity regarding these two validities (gefen and straub, 2005). to test convergent validity of the measures associated with each construct, australasian journal of information systems volume 17 number 1 2011 12 the loadings and cross loadings of each indicator on the latent variables must also be examined. convergent validity is shown when the t-values of these loadings are above 1.96 (gefen and straub, 2005). the results (table 3) indicate almost all measurement items exhibit very high convergent validity on their measured latent variables. latent variables icr ave personal innovativeness 0.896 0.683 self efficacy 0.953 0.870 control 0.864 0.679 focused attention 0.751 0.520 perceived playfulness 0.866 0.576 table 2: measurements items however, it has been noticed that the first measure of focused attention (foa1) and the first as well as the second measure of perceived playfulness (pp1 and pp2) exhibit very low loadings. the way to establish discriminant validity is to examine the square root of the ave of each construct to the correlations of this construct to all other constructs. personal innovativeness self efficacy control focused attention perceived playfulness personal innovativeness (pi1) 0.774 0.257 0.331 0.190 0.177 personal innovativeness (pi2) 0.869 0.410 0.417 0.261 0.438 personal innovativeness (pi3) 0.816 0.492 0.412 0.104 0.443 personal innovativeness (pi4) 0.826 0.372 0.357 0.213 0.364 self efficacy (se1) 0.410 0.880 0.443 0.041 0.468 self efficacy (se2) 0.408 0.877 0.443 0.012 0.477 self efficacy (se3) 0.475 0.959 0.470 0.045 0.488 control (ctrl1) 0.401 0.430 0.846 0.304 0.459 control (ctrl2) 0.281 0.344 0.769 0.256 0.329 control (ctrl3) 0.442 0.415 0.818 0.369 0.532 focused attention (foa1) -0.021 0.017 0.123 0.546 0.025 focused attention (foa2) 0.067 -0.058 0.168 0.698 0.088 focused attention (foa3) 0.225 0.052 0.379 0.866 0.281 perceived playfulness (pp1) 0.152 0.076 0.226 0.300 0.434 perceived playfulness (pp2) 0.107 -0.063 0.093 0.218 0.105 perceived playfulness (pp3) 0.432 0.357 0.453 0.290 0.779 perceived playfulness (pp4) 0.316 0.418 0.404 0.278 0.775 perceived playfulness (pp5) 0.291 0.483 0.404 0.208 0.768 perceived playfulness (pp6) 0.379 0.488 0.498 0.230 0.846 table 3: loading and cross loadings in the pls-graph, the aves were obtained by performing a bootstrap re-sampling. fornell and larcker (1981) suggest that the square root of ave should be greater than the corresponding correlations among the latent variables. the results shown in table 4 demonstrate all latent variables australasian journal of information systems volume 17 number 1 2011 13 exhibit high discriminant validities. the diagonal cells in the correlation matrix shown in table 4 are the square root value of ave for each latent variable. personal innovativeness self efficacy control focused attention perceived playfulness personal innovativeness 0.824 self efficacy 0.496 0.934 control 0.468 0.498 0.822 focused attention 0.287 0.074 0.412 0.808 perceived playfulness 0.486 0.534 0.557 0.316 0.739 table 4: correlation matrix and the square root of ave the initial test of the measurement model using confirmatory factor analysis indicated that some construct revisions were needed. the loadings and cross-loadings of indicators foa1 and pp2 in table 3 showed relatively low correlations on the latent constructs they were meant to describe. essentially this result presents a need to re-specify the instruments. the approach described in churchill (1979) is to purify the measures. items that do not share equally in the common core were eliminated. as suggested by straub et al. (2004), this approach can be applied to pca, pls and covariance-based sem. as a result, foa1, pp1 and pp2 were eliminated in the in the subsequent analysis. the structural model figure 2 presents the results of the structural model. the structural model has an r2 of 0.439 thereby accounting for nearly 44% of the variances in the model. figure 2: the structural model as mentioned earlier, autotelic personality is a second-order reflective construct represented by the four first-order factors and their own indicators. when testing second-order constructs, the percentage of the paths should be at 0.70 or above to establish the convergent validities of the first-order factors (chin, 1998). the steps outlined by pavlou and el sawy (2005) were employed to test the existence of a second order factor. firstly, the correlations between the first order factors and the second order factor were tested. as shown in figure 2, the correlations between personal innovativeness, self australasian journal of information systems volume 17 number 1 2011 14 efficacy, control and autotelic personality are highly correlated because the 0.70 threshold is met. since a reflective model would assume the first and second-order constructs are extremely highly correlated, a formative model seems less likely for autotelic personality. the low loading from autotelic personality on focused attention (after measurement revision) casts doubt on its role in reflecting autotelic personality. the item loadings and loadings of the first order factors on autotelic personality are shown in table 5. item loadings with a t-value above 2.0 are considered significant. all items have large and significant loadings on their corresponding factors indicating evidence of good construct validities (doll et al, 1994). the r-square for each factor indicates that all first order factors, except for focused attention (i.e. personal innovativeness, self efficacy and control) are reliably explained by autotelic personality. these results partially support the first proposition p1. observed variables (pi, se, ctrl & foa) latent variables (autotelic personality) item item loading tstatistics factor factor loadings r-square (reliability) perinnov1 0.7744 16.3283 personal innovativeness 0.820 0.672 perinnov2 0.8685 39.6413 perinnov3 0.8166 28.0138 perinnov4 0.8469 22.1562 selfeffy1 0.9354 72.7521 self efficacy 0.797 0.636 selfeffy2 0.9073 27.8973 selfeffy3 0.9603 91.6832 control1 0.8370 31.1400 control 0.792 0.627 control2 0.7903 22.3422 control3 0.8157 20.3454 focattn2 0.6491 1.9574 focused attention 0.398 0.158 focattn3 0.9623 3.7664 table 5: validities and t-values for autotelic personality (4 first order factors, 1 second order factor) next, a mediation test was performed to see if the second order construct fully mediates the relationship between the first order factors and the theorized dependent variable (chin, 1998). fig 33(a) shows the impact of each first order factor on perceived playfulness individually. personal innovativeness, self efficacy, control and focused attention all show significant effects on perceived playfulness at the 0.01 level. in fig 3-3(b), autotelic personality is used as a more parsimonious second order factor and it shows significant correlation with all first order factors. as a surrogate of its first order factors, autotelic personality strongly influences perceived playfulness and is highly significant (β=0.647, t=11.2900). this aggregate measure is the only significant predictor of perceived playfulness when all first order factors are controlled for. this is indicated by the weak correlation (dotted line) between first order factors and perceived playfulness in the presence of autotelic personality. based on this analysis, there is support for the second proposition p2. australasian journal of information systems volume 17 number 1 2011 15 figure 3-3 the mediation effect of autotelic personality (pi: personal innovativeness se: self efficacy ctl: control fa: focused attention) discussion chin and gopal (1995) suggest the relative importance of the reflective model is established by contrasting the loadings from the overall latent belief with each of the individual beliefs. each belief represents a separate attitudinal dimension, which reflects an existing overall attitude. our findings suggest all underlying factors (personal innovativeness, selfefficacy, control and focused attention) significantly correlated with autotelic personality. the loading of focused attention indicates it is not an important underlying belief in reflecting autotelic personality compared to other factors. according to buchanan et al. (2001), navigating the mobile internet via mobile phones can be a daunting experience given the constraints of small screen display and cumbersome input mechanisms. a common criticism of early wap sites was that they involved too many selections and moves between menus and submenus, for the user to get to their desired contents. therefore, user’s focused attention is likely to be affected by the screen size of mobile phones (buchanan et al. 2001; sweeney and crestani, 2006). our findings suggest all underlying factors (personal innovativeness, selfefficacy, control and focused attention) significantly correlated with autotelic personality. on the other hand, only personal innovativeness, self efficacy and control significantly determine perceived playfulness. high correlations were also identified between these three factors and autotelic personality. this is consistent with asakawa’s (2004) finds that autotelic students are those who feel more in control of the situation and positive about challenges as compared to their non-autotelic counterparts. acknowledging that an autotelic person perceives challenges differently from a non-autotelic person, this study integrated personal innovativeness as one of the underlying dimension of autotelic personality (agarwal and karahanna, 2000). the higher loading of autotelic personality on personal innovativeness and control indicates both constructs are important dimensions of autotelic personality. this study also provides empirical evidence that self efficacy is a significant dimension of autotelic personality in the context of mies. first order factors of ap perceived playfulness first order factors of ap perceived playfulness autotelic personality 0.496 0.570 0.561 0.417 0.824 0.795 0.790 0.396 (a) (b) -0.405 -0.141 -0/015 -0.122 0.657 pi se ctl fa pi se ctl fa pi se ctl fa australasian journal of information systems volume 17 number 1 2011 16 asakawa (2004) discovered autotelic students have more balanced level of perceived challenges and perceived skills suggesting the latter is an important dimension of autotelic personality. in this study, we provide empirical evidence that self efficacy, which is similar to skill, is a significant dimension of autotelic personality in the context of mies. more importantly, autotelic personality is reflected in only one combination of these measures: personal innovativeness, self efficacy and control and is positively related to perceived playfulness. this study thus empirically supports the existence of this important construct and the predisposition of its core dimensions to lead to the occurrence of perceived playfulness in the context of mies. it is suggested that these individual differences within autotelic personality stimulate users to engage in mies activities for the perceived playfulness that accompanies these activities. while users’ personalities remain a complex phenomenon, this study shows that an understanding of their autotelic personality sets a new direction for market segmentation. the marketing literature suggests that there are two main reasons to segment a market – to explore new product opportunities and to develop improved advertising messages by better understanding one’s customers. from the findings, these users are genuinely high on personal innovativeness – regarding themselves among their peers to usually be the first to try out and experiment with new technologies; are confident and independent users of mobile-based application; and who place importance on feeling in control when using mies. autotelic users will recognise new challenges (e.g. new features and functionalities) that confront them by providing the skills in order to extract enjoyment out of the activity. providers are advised to consider identifying this segment, formulating effective marketing strategies to better target this user group. as commonly agreed by many other researchers in mobile commerce, the youth segment is likely to be the segment with the greatest potential, as they have more opportunities to interact with new technologies, have better education and are more innovative. this study was however limited in a number of ways. first, this study is cross-sectional in nature and uses a convenient sample. thus, some caution must be exercised when generalizing the findings. second, this study did not examine the adoption of mies. therefore, the importance of perceived playfulness in the context of using mies cannot be established. future research on this can shed new light on user adoption of mies. third, common method bias may be introduced as the questionnaire employed may affect the scores or measures that are being gathered. attempts to minimize the impact of common methods bias were implemented by using procedural remedies, including guaranteeing respondent anonymity and assuring respondents that there were no correct or incorrect answers (podsakoff, mackensie, lee, & podsakoff, 2003). in a study using flow theory, ghani and deshpande (1994) established that certain cognitive behaviour was related to different task characteristics. for example, perceived control was more important for individuals working on high uncertainty tasks. similarly, respondents working on experientially directed tasks had a stronger tendency to experience perceived playfulness confirming that flow was more apparent in experientially directed activities than goal-directed ones (chung and tan, 2004). others have also found, among other variables, task characteristics and task variety influence students use of the internet (seyal and rahman, 2003). we encourage further investigation into the moderating effect of task characteristics in mies adoption. conclusion this paper investigated those individual differences determining user’s perceived playfulness in the context of mobile information and entertainment services. based on the flow literature, this study also verified an important individual construct—autotelic personality and its significant influence on perceived playfulness. our findings suggest individuals with autotelic personality are more likely to experience perceived playfulness in relation to these services. they also exhibit some characteristics australasian journal of information systems volume 17 number 1 2011 17 in common, such as innovativeness and confidence about their ability to perform and feeling of control when using these services. more importantly, the validation of autotelic personality in the context of this study provides evidence for the existence of this higher order construct which is jointly reflected by its first-order beliefs (personal innovativeness, self efficacy and control). in summary, the important dimensions of autotelic personality are: personal innovativeness, selfefficay and control. these dimensions are positively related to perceived playfulness. this study thus empirically found the existence of this important construct and the predisposition of its core dimensions that can lead to the occurrence of perceived playfulness in the context of mies. our findings suggest individuals with autotelic personality are more likely to experience perceived playfulness in relation to these services. in terms of practice, it is possible to use these dimensions to address the unique needs of different user groups. for example, it might be a possibility to categorize mies users according to the importance of personal innovativeness, self efficacy and control when designing marketing strategies for service offerings. as hoffman and novak (1996) suggest, this has crucial implications for market segmentation of the mobile marketplace. copyright copyright © australian computer society inc. general permission to republish, but not for profit, all or part of this material is granted, under the creative commons australian attributionnoncommercial-noderivs 2.5 licence, provided that the copyright notice is given and that reference is made to the publication, to its date of issue, and to the fact that reprinting privileges were granted by permission of the copyright holder. reference agarwal, r., & karahanna, e. (2000). time flies when you’re having fun: cognitive absorption and beliefs about information technology usage. mis quarterly, 24, 665-694. agarwal, r., & prasad, j. (1998). a conceptual and operational definition of personal innovativeness in the domain of information technology. information systems research, 9(2), 204-215. ahn, t., ryu, s. & han, i. (2007). the impact of web quality and playfulness on user acceptance of online retailing. information & management, 44(3), 263-275. asakawa, k. (2004). flow experience and autotelic personality in japanese college students: how do they experience challenges in daily life?. journal of happiness studies, 5(2), 123-155. baldi, s., & thaung, h. p. p. (2002). the entertaining way to m-commerce: japan's approach to the mobile internet – a model for europe? electronic markets, (12)1, 6-13. bhatnagar, a., misra, s., & rao, h. r. (2000). on risk, convenience and internet shopping behavior. communications of the acm, 43(11), 98-105. buchanan g, jones m., thimbleby h., farrant s. & pazzani m. (2001). improving mobile internet usability. proceedings of the 10th international conference on world wide web, hong kong, may 2001, pp 673-680. acm press. chen, l.-d., gillenson, m. & sherrell d. (2002). enticing online consumers: an extended technology acceptance perspective. information and management 39, pp. 705–719. cheong, j.h. & park, m-c. (2005). mobile internet acceptance in korea. internet research, 15(2), 125-140. australasian journal of information systems volume 17 number 1 2011 18 chin, w. w. (1998). issues and opinion on structural equation modeling. mis quarterly, 22(1), pp. vii – xvi. chin, w. w., & gopal, a. (1995). adoption intention in gss: relative importance of beliefs. the data base for advances in information systems, 26(2/3), 42-64. chung, j., & tan, f. b. (2004). antecedents of perceived playfulness: an exploratory study on. user acceptance of general information-searching websites. information & management, 41(7), 869-881. churchill, g. (1979). a paradigm for developing better measures of marketing constructs. journal of marketing research, 16, 64-73. churchill, g. (1999). marketing research: methodological foundation. new york: the dryden press. compeau, d. r., and higgins, c. a. (1995) computer self-efficacy: development of a measure and initial test, mis quarterly, (19:2), 189-211. csikszentmihalyi, m. (1988). the future of flow. in csikszentmihalyi, m., & csikszentmihalyi, i. s. (eds.), optimal experience: psychological studies of flow in consciousness (pp.364-383). new york. davis, f. (1989). perceived usefulness, perceived ease of use, and user acceptance of information technology. mis quarterly, 13, 318-339. davis, f. (1993). user acceptance of information technology: system characteristics, user perceptions and behavioural impacts. international journal of man-machine studies, 38, 475-487. doll, w.j., xia, w., & torkzadeh, g. (1994). .a confirmatory factor analysis of the end-user computing satisfaction instrument. mis quarterly, 18, 453-461. fang, x. w., chan, s., brzezinski, j., & xu, s. (2006). moderating effects of task types on wireless technology acceptance. journal of management information systems, 22(3), 123-157. finneran, c. m., & zhang, p. (2005). flow in computer-mediated environments: promises and challenges. communications of the association for information systems, 15, 82-101. fornell, c., & larcker, d. f. (1981) evaluating structural equation models with unobservable variables and measurement error. journal of marketing research, 18, 39-50. garcia-macias, j. a., rousseau, f., berger-sabbatel, g., toumi, l., & duda, a. (2003). quality of service and mobility for the wireless internet. wireless networks, 9, 341-352. gefen, d., & straub, d. (2005). a practical guide to factorial validity using pls-graph: tutorial and annotated example. communications of the association for information systems, 16(5), 91-109. gefen, d., straub, d., & boudreau, m.c. (2000). structural equation modeling and regression: guidelines for research practice. communications of the association for information systems, 4(7), 1-80. ghani, j. (1991). flow in human computer interactions: test of a model. in: carey, j. (ed.), human factors in information systems: emerging theoretical bases. new jersey: ablex publishing corp. ghani, j., & deshpande, s. (1994). task characteristics and the experience of optimal flow in human-computer interaction. the journal of psychology, 128, 381-391. ghani, j., supnick, r., & rooney, p. (1991). the experience of flow in computer-mediated and in face-to-face group. in degross, j., benbasat, i., desanctics, g., & beath, c. (eds.), proceedings of the twelfth international conference on information systems (pp. 229-238). new york. australasian journal of information systems volume 17 number 1 2011 19 hoffman, d. l., & novak t. p. (1996). marketing in hypermedia computer-mediated environments: conceptual foundations. journal of marketing, 60, 50-68. hung, s. y., ku, c. y., & chang, c. m. (2003). critical factors of wap services adoption: an empirical study. electronic commerce research and applications, 2, 42-60. koufaris, m. (2002). applying the technology acceptance model and flow theory to online consumer behavior. information systems research, 13(2), 205-23. lee, m.k.o., cheung, c.m.k. & chen, z. (2005). acceptance of internet-based learning medium: the role of extrinsic and intrinsic motivation. information & management, 42(8), 1095-1104. lim, n. (2001) customers' beliefs behind businesstoconsumer electronic commerce. australasian journal of information systems, 9(1), 70-78. lin, c.s., wu, s. & tsai, r.j. (2005). integrating perceived playfulness into expectation-confirmation model for web portal context. information & management, 42(5), 683-693. lohmoller, j. b. (1989). the pls program system: latent variables path analysis with partial least squares estimation. multivariate behavioral research (23), pp. 125-127. moon, j. w., & kim, y. g. (2001). extending the tam for a world-wide-web context. information & management, (38)4, 217-230. nakamura, j., & csikszentmihalyi, m. (2002). the concept of flow. in c.r. snyder & s.j. lopez (eds.), handbook of positive psychology (pp. 89-105). new york: oxford university press. novak, t. p., hoffman, d. l., & yung, y.-f. (2000). measuring the customer experience in online environments: a structural modeling approach. marketing science, 19(1), 22-42. nysveen, h., pedersen, p. e., & thorbjørnsen, h. (2005). intentions to use mobile services: antecedents and cross-service comparisons. journal of the academy of marketing science, 33(3), 330-346. pagani, m. (2004).determinants of adoption of third generation mobile multimedia services. journal of interactive marketing, 18(3), 46-59. pavlou, p. a., & el sawy, o. a. (2005). from it competence to competitive advantage in turbulent environments: a dynamic capabilities model (working paper). university of california at riverside. pedersen, p.e. (2005). adoption of mobile internet services: an exploratory study of mobile commerce early adopters. journal of organizational computing and electronic commerce, 15(2), 2003-222. petter, s., straub, d. & rai, a. (2007) specifying formative constructs in information systems research. mis quarterly, 31(4), 623-656. podsakoff, p. m., mackensie, s. b., lee, j-y., & podsakoff, n. p. (2003). common method biases in behavioral research: a critical review of the literature and recommended remedies. journal of applied psychology, 88(5), 879–903. seyal, a.; rahman, m. (2003) student use of the internet: an extension of tam in technical and vocational institutions in brunei darussalam. australasian journal of information systems, 10(2), 91-104. shim, j. p., & shim, j. (2003). m-commerce around the world: mobile services and applications in japan, korea, hong kong, finland, and the united states. decision line, september/october issue. sweeney, s., crestani, f. (2006), effective search results summary size and device screen size: is there a relationship? information processing & management, 42(4), 1056-1074 australasian journal of information systems volume 17 number 1 2011 20 thompson, r., barclay, d., & higgins, c. (1995). the partial least squares approach to causal modeling: personal computer adoption and use as an illustration. technology studies: special issue on research methodology, 2(2), 284-324. trevino, l. k., & webster, j. (1992). flow in computer-mediated communication. communication research, 19(5), 539-573. van de kar, e. a. m., maitland, c., wehn de montalvo, u., & bouwman, h. (2003). design guidelines for mobile information and entertainment services; based on the radio538 ringtunes i-mode service case study. in sadeh, norman (ed.); proceedings of the fifth international conference on electronic commerce; 414-422. webster, j., & martocchio, j. (1992). microcomputer playfulness: development of a measure with workplace implications. mis quarterly, june, 201-226. webster, j., trevino, l. k., & ryan, l. (1993). the dimensionality and correlates of flow in humancomputer interactions. computers in human behavior, 9(2), 411-426. wu, j-h. & wang, s-c. (2005). what drives mobile commerce? an empirical evaluation of the revised technology acceptance model. information & management, 42(5), 719-729. australasian journal of information systems volume 17 number 1 2011 21 appendix: questionnaire research on users’ experience with mobile information and entertainment services purpose the objective of this questionnaire is to investigate users’ experience with mobile information and entertainment services via mobile phones (pda and pocket pc are not included). participation will only take ten minutes and your response to this questionnaire will provide information for our study. all responses from the survey are anonymous and are kept strictly confidential. definition of mobile information and entertainment services (mies) mobile information and entertainment services (mies) may be defined as the delivery of information and entertainment from specially formatted content sources (such as e -mail, mms, sms-based services, screen graphics and ring tone downloading, new, weather reports, and internet sites etc) via the mobile telecommunication network to a mobile phone user. 1. which of the following applies to your mobile phone? (please circle the one applicable) a) wap-enabled mobile phone or gprs capable phone b) 3g mobile phone c) cdma mobile phone d) none of these or not sure 2. does your mobile phone provide a feature (for example: wap) that allows you to access mobile information and entertainment services on your phone? yes □ no □ (please tick one only) 3. which of the following mobile information and entertainment services did you use most recently? (please circle one only) a) mobile e-mail b) mms (e.g. pxt) c) information services (news, entertainment, sports, lifestyle etc) d) download (music and tones, screen tattoos or java games etc) e) none of these services 4. have you ever used sms-based services on your mobile phone? (examples: request information of account balance, unused free minutes or sending/receiving text messages etc) yes □ no □ (please tick one only) if you have never used any of those services specified in questions 3 and 4, you are not required to answer the following questions below. please return the questionnaire back to instructor. otherwise, please continue. the following questions relate to the mobile information and entertainment service (mies) you have selected in question 3. however, if you selected “none of these services” in question 3, you can use sms-based services as a mobile information and entertainment service to answer the following questions. please indicate your agreement with the next set of statements by circling the appropriate number: item rating scale strongly agree neutral strongly disagree personal innovativeness australasian journal of information systems volume 17 number 1 2011 22 among my peers, i am usually the first to try out new information technologies 1 2 3 4 5 6 7 if i heard about a new information technology, i would look for ways to experiment with it 1 2 3 4 5 6 7 in general, i hesitate to try out new information technologies 1 2 3 4 5 6 7 i like to experiment with new information technologies 1 2 3 4 5 6 7 self efficacy i would feel comfortable using mies on my own 1 2 3 4 5 6 7 i will be able to use mies even if there was no one around to help me 1 2 3 4 5 6 7 i would be able to use mies reasonably well on my own 1 2 3 4 5 6 7 focused attention when using mies , i thought about other things 1 2 3 4 5 6 7 when using mies, i was aware of distractions 1 2 3 4 5 6 7 when using mies, i was totally absorbed in what i was doing 1 2 3 4 5 6 7 control when using mies i feel in control 1 2 3 4 5 6 7 i feel that i have no control over my interaction with mies 1 2 3 4 5 6 7 the mies allow me to control my mobile phone interaction 1 2 3 4 5 6 7 perceived playfulness when interacting with mies i did not realise time had elapsed 1 2 3 4 5 6 7 when interacting with mies, i often forgot the task i was doing 1 2 3 4 5 6 7 interacting with mies lead to exploration 1 2 3 4 5 6 7 when interacting with mies i had fun 1 2 3 4 5 6 7 interacting with mies was enjoyable 1 2 3 4 5 6 7 overall, i encountered a positive experience from using mies 1 2 3 4 5 6 7 please provide information about your background for our study by answering the following questions. this survey is anonymous and confidentiality will be strictly adhered. please circle the one that is applicable to you: gender a) male b) female age a) 20 or less b) 21–30 c) 31–40 d) 41–50 e) 51 or above highest education a) primary school b) secondary school c) undergraduate degree d) postgraduate degree computer experience (years) a) 1 or less b) 2–3 c) 4–5 d) 6–7 e) 8 or above average time of using mies per week a) none b) 1 or less c) 2–4 d) 5–7 e) 8 or above average time of using mies per day a) never/almost none b) less than 1/2 hour c) 1/2~1hour d) 1~2hrs e) 2~3 hours f) more than 3 hours ethnic origin a) nz european b)nz maori c) european d) asian e) indian f) other a.iis vnl.3. nu. 2 editorial this is the second issue of volume 3 of the australian journal of information systems. as editor, i am pleased to report the on-going interest in the journal, both from authors and more recently from a number of international conferences who have kindly agreed to advertise the journal to their participants. in this issue we have three articles from australian authors, two from new zealand, one from scotland, one from the usa and one co-authored by an australian and an israeli academic. our first paper 'software quality certification: identifying the real obstacles' by megan baker, department of business management and anne rouse, department of information systems, monash university examines the difficulty of certifying software quality beyond superficial assessment. the paper takes a case study of a consulting arm of one of australia's leading accounting firm and shows that by using a number of management strategies, the company was able to far more quickly gain certification. the next paper 'grammatical conversion of descriptive narrative an application of discourse analysis in conceptual modelling' by bruce calway and james sykes, centre for information systems research, swinburne university of technology, examines the use of discourse analysis techniques as an approach to developing elementary fact based sentences for information systems conceptual schema development. the paper suggests that the application of discourse analysis techniques provides a narrative which will retain its meaning and contextual integrity while at the same time providing a simplified and independent clause representation for input to the fact-oriented conceptual schema modelling technique. bill doolin, school of management studies, university of waikato, contributes a paper entitled 'alternate views of case research in information systems'. using different case studies drawn from the literature, this paper discusses the differences between the positivist research philosophy and the interpretive research philosophy. the paper shows that the differing ways of looking at reality help in the understanding the nature and use of case studies as a method of conducting research. stephen draper, glasgow interactive systems centre, university of glasgow contributes a paper entitled 'facing up to the plurality of goals, methods needs and resources in hci'. this paper argues that most analyses of human artifacts and interactive software in particular tend to be carried out in a 1:1 manner, while in fact most really require a multiplicity of goals, methods, information needs and information resources. the paper suggests that to address this plurality we perhaps must draw on such concepts as activity theory to address the users' mental organisation of such plurality. the next paper is 'groupware and business process improvement: technology enabled organisational learning' by nereu kock jr. and robert mcqueen, department of management systems, university of waikato. this paper qualitatively examines the introduction of groupware into a service company based in brazil. the paper argues that improvements on business redesign efficiency and effectiveness can be obtained from asynchronous groupware support as well as public sharing of historical information. nava pliskin, department of industrial engineering and management, ben gurion university of the negev, israel, and celia romm, department of management, university of wollongong contribute the next paper, entitled the role of e-mail in an industrial dispute'. the paper examines the use of email communications in a two and a half month long strike by academic staff in israel's universities. the paper argues that analyses of the data suggests that the emailing list was instrumental in enabling strikers to succeed in what was presumed to be a hopeless battle. the next paper 'philosophical and conceptual perspectives on the design of group support systems' by karma sherif and james courtney, department of business analysis and research, texas a&m university, suggests that even more elusive than the design of information systems is the design of group support systems (gss) since they involve groups who vary in behaviour, process and culture. in an attempt to shed new light, the paper examines gss design from four philosophical perspectives functionalism, relativism, radical structuralism and neohumanism. the final paper in this issue is 'computerised support for requirements modelling in foom' by terence tan and paul swatman, centre for information systems research, swinburne university of technology. this paper firstly introduces an information systems development/acquisition method, foom. the paper describes a workbench which can be used in conjunction with foom and reports on the experiences gained through the use of foom in requirements analysis and modelling processes. once again i would like to pass on my thanks to the authors and consulting editors for the high quality of material in this issue. i would also like to thank david dodds for his assistance in producing final copy of the journal and the maintenance of the home page. r.macgregor editor home page: http://www.uow.edu.au/public/faculties/commerce/business/ajis/ajis.html ajis vol. 7, no 2 may 2000 editorial -special section from ozchi99 conference the following six papers have been selected from the recent ozchi99 conference held at csu in november 1999. the theme for the conference, interfaces for a global community, reflects the importance of system and interface designs that facilitate effective communication between computers and the human population, with all of its diversity. it also reflects the mutlidisciplinary nature of systems and interface design. the quality and scope of the conference papers was encouraging and the papers included in this edition illustrate some of the excellent research currently being undertaken in human-computer interaction. jan scott and barney dalgarno, editors, ozchi99 conference proceedings ajis vol. 5 no. 2 may 1998 a ceo view of the key issues in australian information systems management 1997 graham pel-van school of information systems curtin university of technology gpo box ul 987, perth, australia 6845 email: pervang@cbs.cuitin.edu.au abstract as part of a research programme on key information systems management issues, a survey of australasia's largest 500 organisations was conducted to identify which issues were perceived by their chief executive officers (ceos) as being important, problematic and critical over the next three to five years. the results reported are based on a relatively low response rate (though perhaps reasonable for the target group) so should be regarded as only exploratory. the most critical issues were revealed to be a mix of technology management issues (it infrastructure, disaster recovery), strategic management issues (bpr, competitive advantage, information architecture), people management issues (organisational learning), systems development and data management issues (effective use of the data resource, effectiveness/productivity of is, effectiveness of software development), and their own support systems (eis/dss). this reflects their interest in a wide range of issues, but all with an organisation-wide focus. non-critical issues were mostly related to the individual technologies and the place and role of is in the organisation. while the ceos showed some agreement with their chief information officers (cios) on issues such as the it infrastructure, competitive advantage, and organisational learning some distinct differences exist, particularly in relation to the quality and effectiveness of systems and processes and also how ceos and cios perceive each other. keywords is management issues; data resource management; organising is; is research issues. introduction in recent years, and for the foreseeable future, organisations are facing rapidly changing business environments which challenge their executives (both is and non-is) to handle issues such as downsizing, outsourcing, leveraged buyouts, strategic alliances, flexible manufacturing, just-in-time scheduling, globalisation, business process reengineering and total quality management. these environmental changes place demands on their is departments to support product innovation, new production techniques, and changing organisational designs, and provide timely, high-quality information. identifying and handling key issues in the management of information systems is essential for is executives to support their organisations efficiently and effectively. the investigation of these key issues by researchers serves to enhance understanding of the concerns of is executives and suggest relevant areas of investigation by is management researchers. this particular study, which focuses on the views of ceos on the key is management issues, is part of a research program which aims to: • determine the is management issues most important, problematic and critical to australasian executives (cios and ceos) over the next 3-5 years; • examine differences between the views of cios and ceos to better understand their problems in relation to is/it; • examine trends in the status of different issues over time; • inform industry (is and non-is) of current and future issues; • inform is researchers of relevant issues to investigate. other key is management issues studies many key issues studies have now been carried out, but most (for example, pervan 1996, brancheau et al. 1996) have been concerned with the views of is executives only. an historical summary of the few major studies which have included (and specifically presented) non-is executive views is been provided in table 1 below: the most common research approach used has been the postal survey, either single-round or in three-round delphi studies. all of the above studies have obtained opinions from is managers and other non-is managers. in most cases the organisations sampled have been quite varied in size and industry, and the sample sizes, number and type of issues have varied significantly. the studies have generally sought opinions of the major 'key' or 'critical' information systems management issues facing these organisations. this particular study has followed the methodology of the recent australian cio study (pervan 1996) in considering issues as important (they have a significant impact within the organisation), problematic (they are difficult to manage), and critical (they are both important and problematic). 51 ajis vol. 5 no. 2 may 1998 authors brancheau & wetherbe moynihan caudle, gorr & newcomer galliers, merali & spearing wang & turban davis, menon, munday, thomson & young year of study 1987 1990 1991 1992 1994 1995 country u.s.a ireland u.s.a united kingdom taiwan new zealand sample (response rate) is & gen. mgrs. (68/180; ceos/tt mgrs. (49/??) ps mgrs. (354/1054) managers (incl. is) (98/??) is & gen. mgrs. (297/928) various (118/392) total no. of issues 26 ?? 37 26 30 23 research method three round delphi survey structured interviews survey survey survey three round delphi survey table 1: a history of key issues studies involving non-is executives the key issues as indicated in table 1 (column 5), there are many different is management issues. while most researchers accept that is executives must handle many technology management (tm) issues, there is a growing acceptance that there are many strategic management (sm) issues which are equally, if not more, important other categories of issues include systems development and data management (sddm), people management (pm), and end-user computing (euc). the issues in each category are listed in table 2 below. (for a thorough description and explanation of each issue, see pervan (1997).) these issues have changed over time, but those included below are identical to the most recent australian study of cios (pervan 1996,1997). research design in order to provide information concerning the key is management issues, a postal survey of the ceos of the top 500 australasian organisations was conducted in february/march, 1997. the mailing list used was identical to that used for the cio study in 1996. in the course of the development of this mailing list, some organisations were removed because they were either subsidiary to other (larger) organisations on the list or because we were aware they had a company policy against participation in any surveys. a final list of 452 ceos was produced. each questionnaire was identified by printing the organisation's rank on each page so that the responses were not anonymous (but confidential of course). this enabled key financial indicators for each responding organisation to be extracted from the original source of the top 500 list (jaques 1995). the aim of this study was to identify the key is management issues for these organisations over the next 3-5 years. the approach taken was similar to the most recent australian cio study (pervan 1996) where the respondents were asked to rate both how important and how problematic each issue was for their organisation. a list of 31 issues with an associated paragraph of explanation for each issue was produced (the same list as for the cio study). in addition to the brief title for each issue, a paragraph explaining the issue (its 'rationale') was included to ensure that the meaning of each issue was interpreted consistently across all respondents. these 31 issues have been discussed previously (see pervan 1996, 1997). the questionnaire also contained a brief set of questions seeking information on current characteristics of the organisation (ownership, business activity, structure, management style) and of future plans (in market share, staffing, acquisitions, diversification, and it outsourcing). the results of the study valid questionnaires were received from 33 of the 452 organisations. while the response rate of 7.3% is low, it may be considered reasonable for the group targeted (namely the ceos of australia's largest organisations, and hence very busy people). table 3 below shows that these are indeed large organisations with revenue, assets and profits measured in the billions of dollars. table 4 shows the location of company headquarters and the sector to which that organisation's primary activity belongs. these tables show that, though the sample is relatively small, a wide range of the organisations targeted (from the 13th and 486th ranked organisations) did respond and the results may be moderately generalisable for large australian organisations. nevertheless, the discussion which follows should be read as only 'indicative' because of the response rate. 52 ajis vol. 5 no. 2 may 1998 technology management issues building a responsive it infrastructure measuring is effectiveness and productivity improving data integrity and quality assurance developing and managing electronic data interchange planning and integrating multi-vendor open systems technologies integrating data processing, office automation, and telecommunications; managing data and document storage planning and managing communications networks implementing and managing collaborative support systems establishing effective disaster recovery capabilities strategic management issues improving is strategic planning developing and implementing an information architecture aligning the is organisation within the enterprise outsourcing selected information services determining appropriate is funding levels; facilitating and managing business process redesign using information systems for competitive advantage people management issues recruiting and developing is human resources facilitating organisational learning educating senior management in relation to it increasing understanding of is role and contribution systems development and data management issues improving the effectiveness of software development selecting and integrating packaged applications software making effective use of the data resource managing the existing portfolio of legacy applications developing and managing distributed systems improving information security and control planning and managing the applications portfolio planning and using case technology end-user computing issues facilitating and managing end-user computing facilitating/managing executive and decision support systems table 2: list of key issues and categories variable revenue assets profit mean ($b) 0.722 0.905 0.048 sl dev. ($b) 1.222 1.800 0.127 min. ($b) 0.172 0.054 -0.092 max. ($b) 6.536 9.218 0.579 table 3: respondent organisations financial data hq sector resources/mining retail/trading services manufacturing government total vic 1 1 9 1 12 nsw 1 1 1 2 5 sa 1 1 2 4 qld 1 2 3 wa 1 4 5 tas 1 1 act 2 2 nt 1 1 total 4 2 6 17 4 33 table 4: respondent organisations ho and industry sector 53 ajis vol. 5 no. 2 may 1998 as table 4 above indicates, responses were received from all states and territories. the largest sector represented in the sample (and in the target top 500 population) was manufacturing (51%), while major government instrumentalities for energy, water, and transport made up the 12% from the government sector. the two most populous states, new south wales and victoria, made up 51% of the sample, confirming that the majority of company headquarters of large australian organisations are in sydney and melbourne. the representation of industry sectors, states, and company finances (assets, profit, revenue) was similar to the corresponding groups in the cio study. ownership of most of these organisations is australian (64%), as is the focus of their business activity (only 18% international). organisational structure was mostly flat (63% vs 37% hierarchical), mostly along divisional/functional lines (82% vs 18% cross-functional), centralisation was evenly split between centralised and decentralised, and management style varied between 'formal procedures and rules' (19%), 'few rules, greater autonomy' (37%), and 'co-operative and group-oriented' (44%). most were planning expansion of sales/market share (90%), more acquisitions (80%), greater diversification of products and services (61%) and greater it outsourcing (67%), while reducing staff numbers in the organisation (59%). critical issues as indicated earlier, respondents were asked to rate both how important an issue will be for their organisation over the next 3-5 years, and how problematic the issue might be. these were both rated on a 1 to 10 scale where 1 represented a lowest priority issue and 10 represented a highest priority issue. all respondents provided a score for both on all issues. as in the cio study, how critical an issue was to the organisation's ceo was calculated as the arithmetic mean of the ratings for important and problematic. the ten most critical issues, with their ranking, mean and standard deviation of critical rating, are shown in table 4 below. a full list showing mean and standard deviation for all 31 issues is provided as appendix a. rank 1 2 3 4 5 6 7 8 9 10 issue making effective use of the data resource building a responsive it infrastructure facilitating organisational learning facilitating and managing business process redesign using information systems for competitive advantage measuring is effectiveness and productivity developing and implementing an information architecture establishing effective disaster recovery capabilities improving the effectiveness of software development facilitating/managing executive and decision support systems issue category sddm tm pm sm sm tm sm tm sddm euc mean rating 7.16 6.97 6.84 6.83 6.80 6.77 6.75 6.73 6.64 6.58 std. dev. 1.78 1.34 1.64 1.70 1.85 1.65 1.40 1.78 1.94 1.91 table 5: the ten most critical issues the relative ranking of issues within the top 10 should not be overly highlighted because their mean ratings range only by 0.58 on a 10-point scale and the average standard deviation is 1.70. however, it is interesting to note that each of the five issue categories have one or more issues in this top group. while it might be supposed that ceos would not be overly concerned with technology management, this is contradicted by the appearance of it infrastructure, disaster recovery, and is effectiveness measurement (all technology management issues). however, this should come as no surprise in such a rapidly changing technological environment. it is clear that the provision of a responsive it infrastructure (ranked #2) is a fundamental concern for the organisation and that the ability of that infrastructure to respond to disaster situations is an organisation-wide problem in the 1990s. a closer examination of the data revealed that a responsive it infrastructure was rated significantly higher (in a one-way anova at the 1% level) by ceos planning expansion in staffing and greater diversification of products and services in the next 3-5 years. in these organisations the it infrastructure would depended on to support this increased organisational size and complexity. many organisations would not be able to survive more than a few days without their it facilities (o'brien 1993) so disaster recovery (ranked #8) might be expected to be a major issue. measuring the effectiveness and productivity of the organisation's information systems and technology (ranked #6) is clearly of major concern to the ceos given the scale of investment in this infrastructure. clearly, the ceos are expecting 'value for money', 54 ajis vol. 5 no. 2 may 1998 and rightly so. sensibly, the ceos recognised that this issue was particularly critical if they were planning increased it outsourcing (as indicated by a significantly higher critical rating for measuring is effectiveness among organisations planning more it outsourcing). educational issues also rank highly, with organisational learning ranked 3rd and eis/dss (which involves similar motivations and contains a significant user training/education component)) ranked 10th. organisations (and the people in them) require continuous learning about ways to better utilise the information resource and integrate new technologies into the organisation (niederman et al. 1991). it education is therefore critical and can lead to increased productivity and reduced applications backlogs (brancheau and brown 1993). significant increases in innovation and productivity from an organisation's it investment can be derived from an emphasis on supporting and managing organisational learning in the organisation (henderson and lentz 1996). the high ranking of eis/dss may be interpreted as a concern by ceos for their own personal support, but these systems are aimed at supporting managers and decision makers in organisations (laudon and laudon 1991) and ceos would see this as key to the effective operation of the organisation. strategic management issues which ranked highly were business process redesign, is for competitive advantage, and information architecture. business process redesign (ranked #4) has been a top issue among ceos and cios for several years (watson et al. 1997). many organisations are focusing on a radical redesign of their business processes in order to achieve dramatic improvements in performance and innovative and effective applications of information technology are seen by many as a vehicle for achieving this (hammer and champy 1993). further analysis revealed that bpr was seen to be significantly more critical for organisations planning to concentrate their activities more, perhaps reflecting that some radical redesign of existing processes may be necessary to achieve this concentration effectively. further, through the innovative use of it, organisations may find new ways to realise competitive advantage. a corporate information architecture (ranked #7) is a high-level map of the information requirements of an organisation (brancheau and wetherbe 1987) which is used to identify key information needs and how this information relates to key business processes. its development is necessary in order to guide the development of these innovative applications and facilitate the integration and sharing of data (niederman et al. 1991). further analysis revealed that this issue was significantly more critical for more decentralised organisations, an indicator that this high level information map may be more difficult to develop in more decentralised organisations. the ceos also perceive the effective use of the data resource and the effectiveness of software development as critical issues. the data resource in most organisations is becoming larger, more complex, and greater in cost and value. matheus et al. (1993) claim that the data resource is often poorly recognised, difficult to access, and poorly utilised. effective information systems can provide users with timely, accurate, and relevant information, but the backlog in the development of these systems remains at unacceptably high levels in many organisations (laudon and laudon 1994). this is exacerbated by changing hardware platforms and new software development tools arriving on the market, which the developers have to learn and utilise. the ceos, understandably, are particularly concerned with maximising the return from their substantial investment in it and having it available on time, in budget, and with the right form and content. in summary, the most critical issues were revealed to be a mix of technology management issues, strategic management issues, people management issues, systems development and data management issues, and end-user computing. this reflects a broad range of concerns for the ceo in relation to it. however, a closer examination of the issues reveals a major concern for organisation-wide issues and optimising the return from their it investment non-critical issues rank 22 23 24 25 26 27 28 29 30 31 issue managing existing portfolio of legacy applications developing and managing distributed systems outsourcing selected information services increasing understanding of is role and contribution managing data and document storage determining appropriate is funding levels recruiting and developing is human resources multivendor open systems technologies planning and managing the applications portfolio planning and using case technology issue category sddm sddm sm pm tm sm pm tm sddm tm mean rating 6.05 6.02 5.92 5.91 5.83 5.75 5.63 5.48 5.31 4.58 std. dev. 2.19 1.97 2.02 1.88 1.85 1.87 1.97 2.34 1.97 2.32 table 6: the ten least critical issues 55 ajis vol. 5 no. 2 may 1998 the issues which were rated the ten least critical are shown in table 6 above. overall, it may be observed that individual technology management and systems development and data management issues are of little concern to the ceo. six of the ten least critical issues relate to individual information technologies (#26 data/document storage, #29 multi-vendor integration, and #31 case technology) and three relate to specific systems management issues (#22 legacy applications, #23 distributed systems, and #30 applications portfolio). while these individual systems and technologies are part of the overall infrastructure, they may be seen to be the province of the cio (as is the recruitment/development of their is human resources, #25), while it is their overall responsiveness, usefulness and stability which is of greater concern to organisation as a whole. the same argument may hold true for it outsourcing (#24) as it may be seen as the cios job to determine what is actually provided in-house or outsourced and the ceo is interested only in results. (not surprisingly, the further analysis revealed that it outsourcing was seen to be a significantly more critical issue for those organisations planning more it outsourcing in the next 3-5 years.) the results in table 6 also reveal that the ceos show little concern for understanding is role and contribution (#25) and determining appropriate is funding levels (#27). these issues may also be seen by the ceo as a job for the cios and their area. comparison of australian ceos and cios the ranking of the critical issues as perceived by the australian cios in the final round of the 1996/97 study have been added in the final column of appendix a. the spearman's rank correlation calculated on the paired (ceo and cio) rankings was 0.6956 which is significant at the 0.1% level. this confirms that there is substantial agreement between the views of the cios and ceos on the key is issues in these large organisations. that there is fairly strong consensus on the key issues is a positive sign and may indicate the fairly advanced stage of it growth in these large organisations and sound lines of communication between cios and ceos in these organisations. however, there are some distinct differences in ceo and cio rankings on a few issues and these are summarised in table 7 below. ceo rank issue cio rank issues more critical to the ceo than to the cio 4 6 7 9 12 facilitating and managing business process redesign measuring is effectiveness and productivity developing and implementing an information architecture improving the effectiveness of software development improving data integrity and quality assurance 18 12 15 22 19 issues more critical to the cio than to the ceo 13 15 16 19 25 educating senior management in relation to it planning and managing communications networks improving is strategic planning aligning the is organisation within the enterprise increasing understanding of is role and contribution 3 5 6 10 13 table 7: issues seen differently bv ceos and cios the first five issues above are those much more critical to ceos than to cios. all of these issues can be interpreted to have a focus on ensuring that the organisation's is and it delivers on its promises. quality and effectiveness of data and systems are the common themes in these issues. while it is clearly appropriate for the ceos to be concerned with these issues it is perhaps a little alarming that the cios do not show quite the same concern for providing the services whose delivery is their responsibility. is departments and their managers in these large australian organisations may need to be more focused on service and on perceiving the non-is people in the organisation as 'customers' rather than 'users' (pitt et al. 1995). the last five issues above are those more critical to the cios than to ceos. it is quite reasonable that communications networks, the key technology underpinning the it infrastructure now and in the future, should be ranked higher by cios as it is their specific function to ensure that this technology works effectively. it is also reasonable that issues such as is planning, is organisational alignment, and increasing understanding of the is role and contribution should all be ranked higher by the cios as they are essentially their responsibility. however, it is interesting (perhaps even amusing) to observe that cios see a greater need for the it education of senior management than do the senior managers themselves! this difference of view about the senior 56 ajis vol. 5 no. 2 may 1998 management's it knowledge may reflect the long-held perception by is professionals that a poorly educated (in relation to it) senior management is a continuing obstacle to it growth. interestingly, the same difference of opinion on senior management it education occurred in the most recent uk study (galliers et al. 1994). however, their is inherent danger in this apparent conflict of views on the senior management's need for greater it education which may need to be addressed with better communication. in summary, there is a good degree of consensus between australian ceos and cios on the key is management issues and, except for the it education of senior management, any differences are explainable by their roles and so could be considered reasonable. conclusions, limitations and further research this paper has presented some results from a study of the key issues facing chief executives in australia's largest organisations. the most critical issues were a mix of technology management issues (it infrastructure, measuring is effectiveness, disaster recovery), strategic management issues (business process redesign, competitive advantage, responsive information architecture), people management issues (organisational learning), systems development and data management issues (effective use of the data resource, effectiveness of software development), and end-user computing (executive and decision support systems). this range of is management issues are of major concern to ceos. however, it is quite clear that ceos are greatly concerned with 'the bottom line' in relation to their it infrastructure with effectiveness in relation to the data resource, software development, and the measurement of the is function generally. ceos want a sound return on their it investment and it is the role of the it professionals to ensure that they are effective and can be shown to be so (with appropriate measurement). issues perceived by the ceos as non-critical were mostly related to the management of specific systems and several individual technologies which must be integrated and managed to ensure a responsive it infrastructure, as well as a number of clearly cio roles. it outsourcing was ranked surprisingly low (considering its apparent popularity), but it did rate as a more critical issue for those organisations actually planning to implement it outsourcing in the next few years. these same organisations considered the measurement of effectiveness of their is as more critical as this would be a key factor in what functions to outsource and to what extent. on a positive note, cios and ceos were in broad agreement on the majority of the issues though the ceos had, as would be expected, a more organisation-wide focus and were clearly more concerned about receiving a good return from their substantial it investment. on the other hand, for cios some of the more critical issues were directly concerned with their specific role in managing the organisation's is/it. these differences are as would be expected according to the roles filled by ceos and cios. however, cios perceived the senior management's it education as a more critical issue than the ceos themselves and this may imply a lack of communication or at least of understanding of each other. this survey has some limitations associated with the response rate, but the results were interesting and the comparisons between ceos and cios of some value. it is hoped that by providing both target groups with the results of both surveys, they will see where their differences of opinion are, and this may lead to better communication between them so that these differences might diminish over time. further analysis of both surveys is being undertaken and comparisons are being followed up with recent similar studies in other countries. it is hoped that this and other follow-up studies will serve to assist ceos, cios, and is researchers identify areas of interest in the management of information systems and information technology. further, to assess the limitation imposed by the low response rate, a follow-up mailout to non-respondents is planned and non-response bias will be measured. references brancheau, j.c. and brown, c.v. (1993) "the management of end-user computing: status and directions", acm computing surveys, 25,437-481. brancheau, j.c., janz, b.d. and wetherbe, j.c. (1996) "key issues in information systems management: a shift toward technology infrastructure", mis quarterly, 20, 2,225-242. brancheau, j.c. and wetherbe, j.c. (1987) "key issues in information systems management", mis quarterly, 11,23-45. caudle, s.l., gorr, w.p. and newcomer, k.e. (1991) "key information systems management issues for the public sector", mis quarterly, 15,170-188. clark, t.d., jr. (1992) "corporate systems management: an overview and research perspective", communications of the acm, 35,61-75. 57 ajis vol. 5 no. 2 may 1998 davis, j.g., menon, r., munday, s.r., thomson, b.c. and young, l.w. (1995) "key issues in information systems management: a new zealand perspective", proceedings of the prdsm'95 international conference, maui, hawaii, january 2-3,187-195. galliers, r.d., merali, y. and spearing, l. (1994) "coping with information technology? how british executives perceive the key information systems management issues in the mid-1990s", journal of information technology, 9, 223-238. hammer, m. and champy, j. (1993) re-engineering the corporation: a manifesto for business revolution. harper collins publishers, inc., new york. henderson, j.c. and lentz, c.m.a. (1996) "learning, working, and innovation: a case study in the insurance industry", journal of management information systems, 12,43-64. jacques, b. (1995) 'top 500: overview of the year", the bulletin, december 5,4-50. laudon, k.c. and laudon, j.p. (1991) management information systems: organisation and technology. macmillan, new york. matheus, c.j., chan, p.k. and piatetsky-shapiro, g. (1993) "systems for knowledge discovery in databases", ieee transactions on knowledge and data engineering, 5, 6, december. moynihan, t. (1990) "what chief executives and senior managers want from their it departments", mis quarterly, 14,15-26. niederman, f., brancheau, j. and wetherbe (1991) "information systems management issues for the 1990s", mis quarterly, 15,474-500. o'brien, j.a. (1993) management information systems: a managerial end user perspective. irwin, usa. pervan, g.p. (1994) "information systems management: an australian view of the key issues", australian journal of information systems, 1, 32-44. pervan, g.p. (1996) "results from a study of key issues in australasian is management -1996", proceedings of the 7th australasian conference on information systems, university of tasmania, december, 509520. pervan, g.p. (1997) "information systems management: an australasian view of the key issues 1996", australian journal of information systems, 5,1, 55-68. pitt, l.f., watson, r.t. and kavan, c.b. (1995) "service quality: a measure of information systems effectiveness", mis quarterly, 19, 173-187. watson, r.t., kelly, g.g., galliers, r.d. and brancheau, j.c. (1997) "key issues in information systems management: an international perspective", journal of management information systems, (to appear). 58 ajis vol. 5 no. 2 may 1998 appendix a: ceo's critical scores for all key issues rank 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 issue making effective use of the data resource building a responsive it infrastructure facilitating organisational learning facilitating and managing business process redesign using information systems for competitive advantage measuring is effectiveness and productivity developing and implementing an information architecture establishing effective disaster recovery capabilities improving the effectiveness of software development facilitating/managing executive and decision support systems improving information security and control improving data integrity and quality assurance educating senior management in relation to it facilitating and managing end user computing planning and managing communications networks improving is strategic planning selecting and integrating packaged applications software developing and managing electronic data interchange aligning the is organisation within the enterprise integrating data processing, office automation, telecommunications implementing and managing collaborative systems managing existing portfolio of legacy applications developing and managing distributed systems outsourcing selected information services increasing understanding of is role and contribution managing data and document storage determining appropriate is funding levels recruiting and developing is human resources multi-vendor open systems technologies planning and managing the applications portfolio planning and using case technology issue category sddm tm pm sm sm tm sm tm sddm euc sddm tm pm euc tm sm sddm tm sm tm tm sddm sddm sm pm tm sm pm tm sddm tm mean rating 7.16 6.97 6.84 6.83 6.80 6.77 6.75 6.73 6.64 6.58 6.53 6.50 6.48 6.47 6.33 6.22 6.19 6.19 6.19 6.17 6.05 6.05 6.02 5.92 5.91 5.83 5.75 5.63 5.48 5.31 4.58 std. dev. 1.78 1.34 1.64 1.70 1.85 1.65 1.40 1.78 1.94 1.91 1.75 1.63 1.91 1.94 1.72 1.71 1.98 1.90 1.92 1.54 2.10 2.19 1.97 2.02 1.88 1.85 1.87 1.97 2.34 1.97 2.32 cio rank1 2 1 7 18 4 12 15 8 22 14 11 19 3 9 5 6 16 28 10 20 30 27 21 29 13 24 23 17 26 25 31 1 source: pervan (1997) 59 ajis vol. 5 no. 2 may 1998 editorial this is the second issue of volume 5 of the australian journal of information systems. as editor i am pleased to report the ongoing interest in the journal both in australia and overseas. our first paper opportunities for knowledge research in spatio-temporal information systems tamas abraham and john roddick, school of computer and information science, university of south australia examines the exploration of large volumes of geo-referenced data. the authors examine how data mining techniques not only aid in the examination of the data, but allow time to be added to the spatial applications. our second paper towards guidelines for development and management of transnational information systems, angele cavaye, department of information systems, university of southern queensland, paul mantelaers, department of computer science delft university, wander van de berg, department of systems engineering, delft university and arre zuurmond, department of public relations, delft university, examines transnational information systems which cross both organisational as well as international boundaries. the paper examines a number of transnational information systems and describes some of the problems encountered in their implementation and use. andrea engelhardt, department of information systems, friedrich-alexander university provides our next paper a fresh look at integrated business applications neither package nor custom-made. the paper begins by describing the problems both with custom-made software as well as the package equivalent. the paper then goes on to describe a possible middle ground solution. our fourth paper efficient consumer response (ecr): a survey of the australian grocery industry, by john harris and paula swatman, electronic commerce research group, monash university presents a summary of a survey examining ecr knowledge and usage in the australian grocery industry. the paper examines the traditional grocery format, the supermarket and the supply chain, examining implementation strategies for ecr. barry jackson and sony sulaksono, department of information systems, massey university present our next paper going soft on information systems evaluation. the authors suggest that despite the rapid change of both business and its accompanying technology, little real change has taken place in the techniques of systems design. the paper proposes alternate approaches such that a wider share of the stakeholders of the information system are included. our sixth paper a ceo view of the key issues in australian information systems management, graham pervan, school of information systems, curtin university of technology, examines those issues which over the next five years appear as important, problematic and critical to chief executive officers. among the most critical issues were systems development and data management issues, in particular their effective use within the organisation. our seventh paper the dynamics of establishing organisational web sites: some puzzling findings, celia romm and jeanne wong, department of business systems, university of wollongong begins by re-examining the model proposed by jarvenpaa and ives in 1996. the paper presents data gathered in an australian university and compares this data to the jarvenpaa and ives model, concluding that a more generic approach is applicable. our final paper the australian information systems research community: an analysis of mainstream publication outlets, gail ridley, school of information systems, university of tasmania, paula golding, department of information technology, murdoch university, glenn lowrey, college of business administration, university of bahrain and graham pervan, school of information systems, curtin university of technology report on an investigation of the nature of and the interaction among is academics. the paper examined factors such as collaboration within and between departments and universities in australian is research. once again i would like to pass on my thanks to the authors and consulting editors for the high quality of material in this issue. i would also like to thank david dodds for his assistance in producing final copy of the journal and the maintenance of the home page. r. macgregor editor ajis special edition electronic commerce november 1998 content provision on the web: an economic challenge for tv stations claudia loebbecke erasmus university rotterdam school of management 50 burg. oudlaan, ml3000 dr rotterdam cloebbecke@fac.fbk.eur.nl abstract increasingly, tv stations are introducing online services as they seem to hold economic potential and open new markets. nevertheless, networks have apparently until now been neglecting a careful evaluation of online services' strategic and economic potentials, which ultimately is a prerequisite to develop these markets and thus to profit from them. objective of this paper is to analyse some exemplary tv online services offered by the clobo tv network, the largest television network in brazil, as well as to assess their strategic potential and their economic benefit. key words content provision, online services, strategic potential, cost-benefit analysis, brazil introduction the increase in the private and commercial use of the internet has encouraged a large number of companies in different business sectors to conduct business over the internet and to offer their own content and services on the web. while some of the internet-active companies follow well-prepared business plans regarding their online activities, others have just got started. similarly, while numerous companies gained remarkable benefits from their internet activities (benjamin, wigand 1995; angell, heslop 1995; jones 1994), others are learning their lessons 'the hard way' (loebbecke 1996; riefer 1996). therefore, an in-depth analysis of the economic contribution and the strategic potential of online services is of importance: competitive opportunity, competitive necessity, or just 'nice-to-have'? the recent rush to establish an internet presence has also been followed by tv stations. private and public stations are either already using the internet or they are starting to build up an online presence. the strategic potential of tv online services is particularly interesting, there is a high growth potential as currently about ten to twelve percent of european households have access to the internet, while almost 100 percent have access to public tv (e.g. media perspektiven 1995). three developments in the tv sector lead tv stations to investigate the strategic potential and the economic performance of their recently introduced or upcoming online services. (1) for about fifteen years, private tv stations have been allowed to enter the markets in their home countries (meffert, hensmann 1993). (2) with the diffusion of cable tv and the increasing popularity of satellite tv, end consumers can watch programs on tv channels of 'foreign' tv stations. thus, the number of available and accessible tv stations has increased continuously. (3) this increase in 'accessible1 tv stations has been accompanied by a rise in the number of broadcasted channels per station. with the development of digital technologies, tv stations start to broadcast several channels (niche programs). to tackle this growing competitiveness, tv stations are increasingly introducing tv online services, as they seem to hold economic potential and open new markets. nevertheless, networks have apparently until now been neglecting a careful evaluation of online services' strategic and economic potentials, which ultimately is a prerequisite to develop these markets and thus to profit from them. objective of this paper is to analyse some exemplary tv online services offered by the brazilian globo tv network as well as to assess their embedded economic challenge as well as their strategic and economic potential. framework for investigation following general economic potentials of content provision on the web (see loebbecke 1996), tv online services may contribute to a tv network's revenue in four ways (see figure 1): 97 ajis special edition electronic commerce november 1998 general increased number of products sold due to internet-based marketing pr product information larger margin due to lower internal cost (efficiency) higher prices based on value added for the customer content offered on the internet as additional product line space on web pages offered as additional source of income / product line tv stations increased share of viewership / advertising time sold due to internet activities pr (—> tv station) show, news, sports information larger margin due to lower cost for mailings / internal communication, etc. higher prices (for advertising) based on value added for the customer content offered on the internet as additional product line space on web pages offered as additional source of income / product line figure 1: economic potential of content provision on the web (1) online services optimise business processes and therefore contribute to cost reductions (efficiency). tv stations could reduce costs by using the internet as an internal communication tool and for efficient communication with viewers. (2) online services increase either the number of units sold or the price that can be charged per unit. tv online services could increase the share of audience of particular programs and consequently lead to a higher advertising income on the tv advertising market by having more advertising minutes sold or by charging higher prices for a given number of minutes. (3) advertising facilities on web pages offered to third parties represent an additional source for income. (4) consumers pay directly for specific online services which thus turn into stand-alone products sold separately in the market. this paper aims primarily at assessing the strategic potential of tv online services for keeping current and gaining new customers. therefore, issue (1) efficiency of internal processes is not investigated further, as is issue (4) due to the lacking application of secure payment systems on the internet. though conceptually different, issues (2) and (3) for tv stations can be considered together. for private tv stations, advertising income is the major revenue source and the analysis focuses on the online-driven increase in advertising income resulting either from an increase in audience share (program-specific advertising income) or from a greater number of hits on web pages (internet-specific advertising income). figure 2 depicts the media-interdependence with regard to advertising income and thus economic potential of tv online services (see also loebbecke, powell, trilling 1998). there is an obvious link between audience share and program-specific advertising, as the price of advertising time is determined by the number of viewers. web-based online services that can attract additional viewers on the one hand, and television programs which encourage viewers to make use of online-services on the other, are connected in a reciprocal or even complementary way. lastly, the price of an advertisement placed on a web page is linked to the number of hits, as they are usually used as an indicator for the reach of the advertisement (loebbecke, powell 1997). 98 ajis special edition electronic commerce november 1998 keeping current and gaining new customers increased or stabilized share of audience ^^^ more hits on web pages ^m ^ ^ program-specific . advertising income ^ i ^l increased ^v advertising ^f income m ^^ internet-specific w ^^ advertising income figure 2: media-interdependencies and advertising income (source: loebbecke, powell.trilling 1998) as a basis for further detailed investigation and comparison of tv online services loebbecke and powell (1997) suggest an 'online success per program typology' (ospt) integrating (1) different kinds of programs, (2) success factors and (3) different kinds of tv online services (see figure 3). figure 3: 'online success per program typology' (ospt) tv programs are distinguished according to the two dimensions contents (entertainment and information) (krueger 1995) and emission regularity (once/irregular and repeated/regular). the four identified success factors contemporaneity, emotional tie, individuality/interactiveness and image are used to analyse qualitative factors that cause the different levels of audience (toffler 1980; ishikawa, muramatsu 1991; mudie, cottam 1991; albers 1992; greenberg, busselle 1992; collins 1993; leggatt 1993; schenk, gralla 1993; gleich, groebel 1994; duvinage 1996; weise 1996). contemporaneity is especially important for information programs and mainly depends on up-to-dateness of information and its availability to 99 ajis special edition electronic commerce november 1998 viewers. the emotional tie encompasses the viewer identification and the ability of the network, program or actor to influence him; it is a form of viewer commitment. this is a vital element for regular programs such as news or soap operas and can be based on content or people. individuality and interactiveness, as a function of customer integration, are important for the perceived quality of services. the factor image is of significance for the competitiveness of the network and for the success of a particular program. table 1 summarises the four success factors and shows their importance for the different types of tv programs based on in-depth interviews in germany, france, and the netherlands. program type tv success factor program -specific nonprogram -specific contemporaneity emotional tie and viewer identification individuality and interactiveness image regular irregular regular irregular enterenterinforinfortainment tainment mation mation ++ -h++ very important + important not important table 1: tv success factors and their relevance for program types (source: loebbecke, powell, trilling 1998) concerning tv online services, the approach integrates 'supportive1 versus 'stand-alone' online services (alpar 1996) and the focus on the 'unit covered' (schroeter, ewald 1996), e.g., station, channel, and program differentiates four main kinds of tv online service (loebbecke, powell, trilling 1998): (1) program-specific online services concern particular tv programs and may be further classified according to their interactiveness. if viewers desire information, the 'interaction' only consists of clicking on web pages (passive internet use). however, online services may also encourage viewers to express opinions (active use). (2) channel-specific, but program-independent online services, assist consumers to familiarise themselves with the tv station. (3) stations can place segments or entire programs on the web. such 'programs on demand not only provide the opportunity to view contents individually, but also facilitate the acquisition of viewers who cannot access conventional transmissions (e.g. those living abroad). (4) tv stations increasingly offer channel and program-independent online services, e.g., information services, online games, to enlarge their business scope. the case of rede globo*0 in the following two sections, we will introduce brazil's red globo tv network, the largest television network in brazil, and apply the proposed framework to analyse some exemplary tv online services as well as to assess their strategic potential and their economic benefit. case setting: brazil and the brazilian telecommunications and television industry demographic and economic particularities of brazil brazil is the fifth largest country by area in the world and the only state in south america where portuguese is spoken. there is a very sharp division in society between the rich and the poor. only about 60% of the 162.6 million population dispose of any buying power beyond their most basic needs. a concentration of economic wealth lies along the coast, especially in the southern part (economist 1995; baratta 1996; cia 1997). brazil is the world's eighth largest economy with an estimated gdp at purchasing power parity of us$ 976.8 billion in 1995 (cia 1997). until 1990, brazil was a relatively closed economy, but then adopted an aggressive liberalization strategy which immensely increased the competitiveness of most sectors. efforts were made to 20 an earlier version of the red globo case and its analysis appeared in [butzbach, loebbecke 1998]. 100 ajis special edition electronic commerce november 1998 privatise state-owned monopolies. this led to dramatic changes in technology, especially telecommunications (economist 1995, kpmg 1996, un 1996, bndes 1997). brazil has a very young demographic structure. table 2 shows the estimate for july 1996; life expectancy at birth is 61.62 years (cia 1997). a high degree of illiteracy is still a major problem with only an estimated 83.3% (1995) of the population at age 15 and over being able to read (cia 1997). age 0-14 years 15-64 years 65 years and over share of population 31% 65% 4% table 2: age structure in brazil (estimate for july 1996) brazilian telecommunications and television industry brazil's telecommunications industry is controlled by formerly state-owned companies that have just recently been privatised. in the 1980's, most of the necessary investments were not carried through as the capital was used by the government to pay for the nation's debt instead of developing and maintaining the telecommunications infrastructure. as a consequence, the provision of telephone lines is a severe problem (economist 1995; pinhanez 1995; gouvea neto 1997). nevertheless, the existing telephone net is in a good state. in 1994, there were an estimated 8.1 lines per 100 inhabitants compared to, for instance, 60.2 per 100 inhabitants in the united states (un 1996). brazil is a country with one of the highest daily per-capita television consumption world-wide. daily tv audience in brazil's economic centers (sao paulo and rio de janeiro) in 1996 was 420 minutes (paoletti 1996) compared to 241 minutes in the united states, 251 minutes in japan and 196 minutes in europe (baldauf 1996). an estimated 82% of brazilian households had television (corresponding to 32.65 million television sets) as per december 1993 (paoletti 1996; un 1996). there are currently around 4 million personal computers installed (santos 1996;marins 1996b). television industry is mainly financed by revenues from advertising (wilke 1992). the estimated number of internet users in brazil was at 600,000 for november 1996 (godoy 1996). the average net user pays roughly us$30 per month for the basic service. this limits internet usage to persons with a fair amount of buying power (marins 1996a). company overview rede globo is the television network of the globo media giant, a company of great domestic influence and importance in brazil (wilke 1992; gouvea neto 1997). the conglomerate also holds print media, record labels (to commercialize globo's soap opera soundtracks) and radio stations. in the last decades, it has been following an aggressive national and international expansion strategy. as a consequence, globo tv has turned into a global player in a sophisticated and fast-moving high-technology sector which is becoming increasingly competitive. being the fourth largest private television network in the world and one of the largest in-house producers of television programs, globo tv is highly involved in the globalization of television and program distribution (grosse-kracht 1992; netz 1996; paoletti 1996; gouvea neto 1997; rede globo 1997). globo is currently trying to position itself as the technology leader in the brazilian and latin american television markets, preparing to exploit new potential markets, products and services (netz 1996; salamanca 1996; paoletti 1996; gouvea neto 1997; rede globo 1997). rede globo's audience share in brazil is approximately as follows: time of day morning afternoon evening (prime-time) late-night share of total viewers 56% 59% 74% 69% table 3: rede globo audience shares (source: gouvea neto 1997; rede globo 1997) rede globo has been following a strategy of technology leadership, launching new products and introducing new technology at a high pace and with a long-term approach. an emphasis lies on state-of-the-art equipment. this led to an environment which quickly adopted new developments and technologies (herz 1987; grossekracht 1992; wilke 1992; gouvea neto 1997; rede globo 1997). almost all of brazil (99.84%) is covered by 101 ajis special edition electronic commerce november 1998 the rede globo television network with 107 broadcasting and affiliate stations. as a consequence of its national approach, it has traditionally had the biggest market share of brazil's advertising budgets for television (65%) as shown in figure 4. sistema brasileiro de televisao others 13% rede globo 65% tv bandeirantes 7% figure 4: network market shares in television advertising (source: gouvea neto 1997) rede globo's online services rede globo's web-based online services (http://www.redeglobo.com.br) started in 1996. as of november 1997, there were no commercial or non-globo advertisements. television program schedule on this page, the daily television program times are shown. some of the programs offer hyperlinks to the individual home page of that program. there is the possibility to retrieve the schedule for the remaining days of the week and for the following week. additional information on programs this is a platform to access the content offered by the individual programs, such as science and news programs, soap operas, etc. for instance, for these soaps the tv online services provide an introduction to the story, the plot, information about the author and the director as well as a presentation of the production team, the cast and the technical staff. besides portuguese, some of the pages also offer the content that does not change on a daily basis, in english and spanish. most of the pages also allow for feedback via a dedicated e-mail-address hyperlink. e-mail contact this page allows viewers to contact the makers of the most popular programs via e-mail by clicking on icons with the program-specific logos, letting them recognise their program visually rather than in text form. company information and sales by choosing this link, interested parties can obtain company information (in portuguese, english and spanish) as well as information on in-house productions currently for sale (in english and spanish). besides the plots of the production, the names and full addresses of the sales representatives are supplied. also available is information on merchandising possibilities for soap operas. news from the recording sets behind this link, viewers can obtain the latest news from the recording sets. this includes information on what is currently being recorded, small and big problems at the studios, updates on stones, and gossip about the actors. 102 ajis special edition electronic commerce november 1998 planned expansion of red globo's online services rede globo is planning to expand its online services to increase the benefits created. online services are to be used to lower the cost of interaction. for example, instead of reading, processing, printing and mailing replies to viewer letters, e-mails will be processed and suggestions for replies made automatically before being sent via the internet. this will not only save the postage and handling, but also reduce human processing time to a minimum. the internet will then substitute part of the amount of telephone calls on toll free lines (antunes 1997). the offering of additional information will be enlarged in the near future and is being implemented for breaking news headlines and brazilian economic news. rede globo already uses its pages to supply company information and present its efforts in charity and welfare programs. these online services are to be enlarged and it is planned to add links to charity programs encouraging viewers to (possibly online) support them (antunes 1997). also, the usage of online services is planned to be more closely monitored to establish individual viewer profiles for certain social segments or age groups. a good example for the possibility of gathering quantitative data is the 'angelica1 page. a doll of the children's program presenter 'angelica' can be dressed by the young viewers online from a selection of clothes. besides being a fun activity, the information will be gathered and used to dress 'angelica1 according to the preferences of her viewers. the information will also be used for advertising purposes and trend research, as this social segment is the main addressee of non-basic consumer goods (antunes 1997). analysis of rede globo's online services according to the proposed framework success factor importance per product category in the following analysis of examples from the rede globo online services, the classification of tv online services shown above will be used. the importance of the introduced success factors is assessed for each of the three categories. programand channel-independent online services are disregarded, as none are offered by rede globo. program-specific online services the majority of the offered online services are program-specific. examples for this are 'jornal nacionat (daily news program), 'por amor' (telenovela) and 'angelica' (children's program with a live audience of elementary school-aged children) pages. contemporaneity is an important success factor for the 'jornal nacionat. the online offerings include discussions and background information on a selected number of subjects treated in the tv edition, but there is no separate and comprehensive online edition emotional tie and viewer identification are vital for tor amor' and 'angelica1. 'por amor' needs an emotionally involved audience to keep its share of audience which determines the amount and price of advertising. a drop in audience may even make it necessary to start a new soap. the 'angelica' page tries to bind its young audience by offering them additional activities, pictures, music, video clips and a contest to increase the level of involvement. individuality and interactiveness are important for 'por amor' and 'angelica', less for 'jornal nacionat, as viewers have no influence on the happenings that constitute the news. 'por amor' is, as any soap, permanently in progress and each episode is written just ahead of recording. as the story develops and becomes more concrete, rede globo has to keep track of audience profiles, checking on their expectations to let the story run in that direction. the viewers become virtual co-writers of the soap-operas they watch and thus their personal preferences can be taken into account. consequently, the interaction with them is vital for the program's success. this is achieved by offering the possibility to read previews of the episodes just ahead and to give e-mail feedback on the development. finally, image is a very important success factor for the online services of all of the three programs, but the conveyed image has to be different. whereas 'por amor' must be up-to-date on social concerns and public opinion, 'angelica' must be trendy and innovative. 'jornal nacionat, on the contrary, should convey an image of a reliable and trustworthy supplier of information. channel-specific, but program-independent online services examples for channel-specific, but program-independent online services are 'programagao' (tv program guide) and 'institucionat (company information) pages. 103 ajis special edition electronic commerce november 1998 of course, contemporaneity is of vital importance to the 'programa9ao' tv program guide. viewers expect to see the valid version of the programming. thus, any sudden changes in the schedule must be posted in this online service without delay. for the 'institucionat company information pages, this is less important, as the history and development of the company can't change, only and rarely its perception. emotional tie and viewer identification as well as individuality and interactiveness have little importance for the 'programa?ao' and the 'institutional' online services. lastly, image is the dominating factor for the 'institucionat pages. these pages are used for public relations purposes and convey a positive, non-controversial view of the network. for 'programac.ao', image can only have a negative effect, if the program guide gives wrong information. its potential to convey a positive image is very limited. programs-on-demand no programs on demand are currently offered except for short clips on the 'angelica' page which cannot really be considered as such. table 4 shows the relative impact of rede globo's program-specific and channel-specific tv online services on the different success factors. program-specific service barely contribute to contemporaneity and individuality /interactiveness. also less important are channel-specific services for emotional tie and viewer identification, as well as image. channel-specific services promote contemporaneity, on the other hand, more than previous investigations in europe had suggested. online service tv success factor program-specific non programspecific contemporaneity emotional tie and viewer identification individuality and interactiveness image programchannelspecific specific ++ ++ +++ ++ + +++ ++ +++ high impact ++ high/modest impact + modest impact no impact table 4: impact of tv online services on tv success factors assessment of the strategic and economic impacts the assessment of the additional cost created through online services can only be estimated, no precise fugures are available. one time costs to set-up were around us$ 250,000, and minimum yearly maintenance costs are us$ 50,000 (maintenance costs are an own estimate based on the necessary manpower of two full-time positions for current online services offering). the set-up includes procured hardware and software as well as installation and training costs (antunes 1997). on the positive side, according to globo tv, there is limited direct return (direct additional revenues or savings) from its online services yet. the currently expected economic potential is a savings of up to 1.6% of total cost of toll-free telephone lines. indirect revenue from online services-based marketing and a more positive image cannot be estimated, but seems to be rather limited due to the fact that the number of internet users in brazil is still very small in comparison to the total number of tv program viewers. thus online services can only have a slight influence on increasing the number of viewers and in turn raising advertisement prices (antunes 1997). nevertheless, rede globo tries to support a rapid expansion of internet-usage by specifically addressing young viewers as target group. investments in online services are seen as a further step in slowly developing this new potential and in conveying a technology leader image. the company's technology leader approach makes it a strategic necessity to pursue any promising new technologies, at least in the perception of the public. if tv online services turn into a competitive necessity or if new markets open, rede globo needs to have the new technologies up and running. the current offering of online services allows the testing of reliability and the gaining of experience with a still limited user base. due to the novelty of the services, a comprehensive cost-benefit-analysis is not yet possible. 104 ajis special edition • electronic commerce november 1998 conclusion and further research in the case of red globo it seems likely that online services were implemented without a clear idea of the objectives pursued. a trend was followed without a clear answer to the question of 'why?'. generally speaking, even though offered online services are constantly increasing in proficiency and user utility, they do not seem to be generating any direct additional income for 'their suppliers' yet, nor do they seem to actually be developing new markets and potentials. as in the case of red globo, there are often no precise figures on the cost of the necessary investments available. apparently, companies lack a serious investment plan and do not define a minimum expected return on investment. the (seemingly) low cost of online activities is frequently regarded as a sufficient justification. consequently, two crucial questions for tv stations considering the launch of tv online services should be answered before entering online activities: (1) is a positive return on investment to be expected? (2) are tv online services considered a competitive necessity for the station's core business? if at least one of these questions can be affirmed, the set-up of online services is to be recommended. otherwise, tv stations should at first refrain from doing so. a positive answer to at least one of the two questions seems likely if content provision on the web, and tv online services in particular, are embedded in a comprehensive business strategy and business model that explicitly takes into account the opportunities of innovative technologies. if, however, chances are sought only along traditional operational processes and lines of business, content provision might be economically risky. the case of tv stations is quite obvious. red globo and other stations offer a wide variety and significant depth of content on their web pages. currently those pages are attractive and well taken, but do not generate income. however, the obvious demand for the content provided should allow for business models that lets tv stations and other content providers benefit from their extra performance: possible solutions may include income generation via relevant additional advertising income on web pages or directly charging consumers for the content and the convenient delivery (see issues (3) and (4) in figure 1). only if none of the two options is seriously pursued, in the long run tv online service may turn into a competitive necessity that mainly reduces the available profit margin an extra service expected by customers for free. what developments can be expected in the near future? the general globalization tendency is a factor that will foster the development of tv online services. tv stations will be able to reach out to new viewers via the web. this will open the door to a more global advertising market, but will also increase competition. improvements in the infrastructure will lead to a higher interest in programs-on-demand via the web. as can be concluded from the current boom in pay-tv demand (in brazil, for instance, the growth rate of pay tv was 212% between 1992 and 1996 and generated revenues of more than us$ 1 billion in 1996), tv online services such as programs-ondemand have a particular potential in a country with a high degree of street violence. tv online services will probably follow this trend, but due to higher up-front cost for users (investments in pc or web computer, telephone line, internet provider, etc.) the development will be much slower. the proposed framework and the presented case study raise several questions for further empirical and theoretical research: what insights gain be gained from applying the framework to a larger number of tv stations and other content providers such as newspapers, radio stations, or even very enterprises in different business sectors such as travel agencies or financial services? what are opportunities and limits for interorganisational comparisons or cross-sector analyses? should the given classifications be further divided (complexity versus feasibility)? what other culture-specific factors play a role in different environments? is there a correlation between online usage and the share of audience and how can it be measured? on a more general level: is there a correlation between 'online usage' and the 'number of units sold' or the 'price per unit sold1, and how can it be measured? last, but not least, how could the framework be transformed into a business planning tool for content providers on the web? answers to these and other questions will provide valuable insights into a challenging area of business. references albers, r. (1992). quality in television from the perspective of the professional program maker. studies of broadcasting, 28,7-20. alpar, p. (1996). kommerzielle nutzung des internet. berlin. angell, d.; heslop, b. (1995). the internet business companion, growing your business in the electronic age. cambridge, massachusetts. antunes, r. (1997). member online services team. interview on august 6 in sao paulo, brazil. baldauf, s. (1996). briten sind europameister im tv-konsum. tendenz 4, 30-33. banco nacional de desenvolvimento econdmico e social (bndes) (1997). the brazilian privatization program, http://www.bndes.gov.br. 105 ajis special edition. electronic commerce november 1998 baratta, m.v. (ed.) (1996). der fischer weltalmanach 1997, frankfurt, germany. benjamin, r.; wigand, r. (1995). electronic markets and virtual value chains on the information superhighway. sloan management review, 36,62-72. central intelligence agency (cia) (1997). worldfactbook. http://www.odci. gov/cia/publications/nsolo/factbook/br.htm. collins, a. (1993). intellectuals, power and quality television. cultural studies 7, 1/93, 28-45. duvinage, m. (1996). wdr, head of'online service center1. interview on oct. 30 in cologne, germany. economist (1995). survey brazil a glass half full. 7912(335) 74. gleich, u.; groebel, j. (1994). ard-forschungsdienst beurteilung der programmqualitat im fernsehen. media perspektiven, 5/94, 253-258. gouvea neto, r. de (1997). case study: tv globo the brazilian media giant. the international executive, 2(39), 255-269. godoy, n. (1996). com a porta na cara. istoe. http://www.zaz.com.br/istoe/ ciencia/141725.htm. greenberg, b.; busselle, r. (1992). television quality from the audience perspective. studies of broadcasting, 28,157-194. grosse-kracht, h.-j. (1992). joint-venture und militarische protektion zur entstehungsgeschichte des brasilianischen fernsehsenders tv globo. rundfunk und fernsehen, 2(40), 247-256. herz, d. (1987). a historia secreta da rede globo, porto alegre. ishikawa, s.; muramatsu, y. (1991). the assessment of quality in broadcasting: research in japan. studies of broadcasting, 27,7-20. jones, r. (1994). digital's world-wide web server. a case study. computer networks & isdn systems, 272, november, 197-206. kpmg (1996). investment hi brazil. sao paulo. krueger, u.m. (1995). trends im informationsangebot des femsehens. programmanalyse 1994 von ard, zdf, rtl, sat.l und pro sieben. media perspektiven 2(95), 66-78. leggatt, t. (1993). quality in television: the views of professionals. studies of broadcasting, 29, 37-70. loebbecke, c.; powell, p.; trilling, s. (1998). investigating the worth of internet advertising, international journal of information management, forthcoming. loebbecke, c.; powell, p. (1997). understanding internet advertising. international conference of information systems (icis). loebbecke, c. (1996). content providers benefiting from the internet. fourth strategic information systems network (sisnet) conference, lisbon, portugal. marins, m. (1996a). net dumb no more, brazzil. http://www.brazzil.com/ p22sep96.htm. marins, m. (1996b). reborn on the web, brazzil. http://www.brazzil.com/ p20apr96.htm. media perspektiven (ed.) (1995). basisdaten, frankfurt, germany. meffert, h.; hensmann, j. (1993). die entwicklungsdynamik des europaischen fernsehmarktes in den 90er jahren. vienna, austria. mudie, p.; cottam, a. (1993). the management of marketing services. oxford, uk. netz, c. (1996). a globo plim-plimpor plim-plim. exame, http://www2.uol. com.br/exame/19emprel.html. paoletti, r. (1996). fall season for globo, brazzil. http://www.brazzil.com/ aug96.htm. pinhanez, c. (1995). internet in developing countries: the case of brazil. http://www.cea.mdx.ac.uk/cea/researchers96/clau/internet_brl.html. rede globo (1997). company information, http://www.redeglobo.com.br/inst/ english/instl .htm. riefer, k. (1996). zeitungen online chance oder risiko?. media perspektiven 10/96, 537-549. salamanca, d. (1996). lateinamerika: medienkonzerne unter sich. media perspektiven 12/96, 639-647. santos, a. (1996). personal computers are hot. ieee spectrum 7(33), 34-39. schenk, m.; gralla, s. (1993). qualitatsfernsehen aus der sicht des publikums. media perspektiven, 1/93, 8-15. schroeter, c.; ewald, k. (1996). onlineangebote von rundfunkveranstaltern in deutschland. media perspektiven, 9/96,478-486. toffler, a. (1980). die zukunftschance. munic, germany. united nations department for economic and social information and policy analysis statistics division (un) (1996). statistical yearbook 1994, new york. weise, d. (rtl manager 'online'). interview on nov. 5 in cologne, germany. wilke, j. (1992). massenmedien in brasilien. wilke, j. (ed.). massenmedien in lateinamerika, frankfurt, germany. 106 the evolution of an information system for managerial use: a longitudinal study helen m. hasan and edward l. gould department of business systems, the university of wollongong, wollongong australia email: h.hasan@uow.edu.au fax: 61 42 214474 abstract the failure of many information systems (is) designed for use by managers may be due to the fact that traditional is methodologies were used in their development. in this paper we describe an organisation's efforts, over a period of four years, to develop an is for use by senior management and show how traditional methodologies have impeded the involvement of the intended users of the system from the development process resulting in poor specification of user requirements and inflexible systems. from this experience we verify the superiority of an evolutionary prototyping methodology for the development of these types of systems. introduction in the 1990's the process of management has become information intensive (broadbent et al 1992) and organisations are now having to make changes to their information systems (is) in order to meet the information requirements of managers at all levels. companies have not always been successful in managing the changing demands of information technology (it) from the self-contained business support units of the sixties to the distributed strategic business functions of the nineties (selig 1991, hasan & cheung 1993). organisations continually face the trade-off between it processing power, human effort and different investment decisions in order to best meet organisational objectives and opportunities (galliers 1991). it is reported that at least half of the executive information systems (eis) developed in the nineteen-eighties have not lived up to expectations (crockett 1992) and that this failure is due in part to the methodology used for their development. current literature in this area suggests that an eis can only be successful if it is created using an evolutionary prototyping methodology (guimares & saraph 1991, rainer & watson 1992, watson et al 1991, wetherbe 1991). with this approach executive user involvement can be sustained, ensuring that their changing information requirements can continue to be met. in order to study this process, we have instigated, in one organisation, a longitudinal investigation of the use of it by senior management for their own information needs. the findings reported here support the use of evolutionary prototyping for development of this type of system. 88 background to the study the organisation participating in the study is a medium sized public organisation of around 1000 employees and 200,000 customers. the organisation has had a long history of it use. the senior management had no formal is training but all have a technical (engineering) background. interviews were conducted over a period of four years, from 1991 to 1994, with a total of 16 subjects, 5 at executive level, 6 at middle management level and 5 members of the is staff. these interviews focused on the following: • the sources of the information used by senior management, • the current and anticipated flows of information throughout the organisation • the information systems used by managers at various levels. the responses to four years of interviews have been recently collated and a picture of change has emerged as shown in figure 1. the study began in 1991 when there was a recognition by second-tier managers (immediately below the chief executive officer) of information as an organisational resource and an awareness of a need to overcome short-comings in their managerial information systems. we have documented their reaction to this over the ensuing three years. figure 1 an overview of changes to managerial is stace 1 1991 awareness of management information deficiencies stage 2 1992 attempt at a quick fix solution staee 3 1993 rationalisation of the existing information systems staee 4 1994 evolutionary development of an eis prototype results four main stages were identified in the longitudinal study, coinciding roughly with the four years 1991 to 1994 as shown in figure 1. these were: stage 1 in 1991 the state government, in an endeavour to stimulate a more competitive corporate environment, imposed a set of performance indicators (pi) on the organisation. this resulted in the launch by management of a performance improvement program aligned with the corporate goals shown in figure 2. an analysis of these goals shows that additional management information systems would be required to monitor progress on achievements in each of the four areas. consequently there was at that time an emphasis on better strategic use of it throughout the organisation. this led to the development of new user-oriented operational computer 89 systems that could keep much more comprehensive information on customers and employees. the development of these systems was carried out by teams consisting of middle management users and is staff but which left out senior management. so just when there was an increased pressure for timely and accurate information by senior management, their needs were not considered during the development of the new organisational information systems. figure 2 corporate goals • excellent customer service by discovering and responding to customers' expectations and providing a level of service and reliability that meets customer expectations. • sound business management that optimises market share, maintains financial viability and seeks new business opportunities. • good people management that sustains high level of employee commitment and a skilled and flexible workforce. • respect for the environment. stage 2 in 1992, as a reaction to the crisis in managerial information, there was an attempt at a cheap quick-fix solution to executive information needs. with a large amount of company it resources being invested in upgrading the operational and middle management is, the provision of a system to supply information to senior executives, was low on the priority list of the is department. no resources were available for this project despite pressure from one senior executive who was keen on sponsoring an executive information system (eis). at the same time the organisation was given a pc based eis package that was about to be superseded and, coincidentally, a research fellow from the local university wanted to undertake a study with an organisation on current management uses of it in real world situations. the executive sponsor mentioned above instigated a project to use these free resources (the pc package and consultation time from the research fellow) to implement an eis. his approach to systems development was to follow the traditional life cycle method of ascertaining requirements then designing and building the final system. the plan was that the research fellow should interview all executives to ascertain their information needs and determine the sources of this information. a member of the is staff would then set this information up in the eis package. the result of these interviews was a vague and conflicting list of executive user requirements and a long list of problems with the existing sources of information as has already been reported (hasan & cheung 1993). it was clear that many of the problems on this second list needed to be addressed before anything further could be done. this 90 included the consolidation of duplicate sets of data, particularly on staff and customers, held in different departments and a co-ordination of the timing and format of the various reporting systems. following a traditional development path the options were either to wait until all these problems were fixed or to go ahead and build a system that would be of no use. stage 3 towards the end of 1992 and into the beginning of 1993 there was a rationalisation of the situation. it was seen that many, but not all, of the problems identified in the previous stage could be solved without much difficulty and outlay of resources (which was still a concern). in particular a good deal of mis-information was identified as due to the number of small end-user developed systems kept mainly on pc's by individuals through out the organisation. most of these had been set up as a reaction to problems with the old operational systems that were now being replaced. however many employees still produced reports for managers based on the data kept on these small systems in preference to the new centralised systems. so it was often a question of whose figures did you believe. the is department was asked to locate and rationalise all organisational systems. also in 1992 the implementation of some of the new operational systems was completed and a decision was made on the purchase of suitable hardware for the executives (networked pc's). at the same time one member of the is staff was allocated to work with the executive sponsor on the problem of a suitable is software for the executives. a small prototype executive query system was set up on top of the most critical of the existing operational systems (the customer billing system) using q&a. this was done to stimulate discussion among the executives and enable a more concise list of their information requirement to be gathered. stage 4 late in 1993 the organisation decided to allocate some resources to continuing this project and a new pc based eis package was purchased. this followed pressure from the executive sponsor who was convinced that the "easy to use" graphical user interface (gui) of these systems was required by busy executives. there was also a long term objective that this type of interface would be the ultimate goals of all systems for managerial use such as email, wordprocessors, spreadsheets and also for queries on organisational databases. an evolutionary approach has been used in the implementation of the eis package so as not to repeat some of the errors made in attempting the quick fix solution of 1992. the most critical area for management information is based on the organisation's customer database so this is being set up first, with summarised information broken down into various categories. other data, both internal and external, will be added as the system grows. the eis package allows a "drill-down" facility to enable mangers to investigate problem areas. it is recognised that the system needs to be flexible to meet changing needs of the organisation and the different executives. the is department, in conjunction with senior management, has come up with an organisational policy for data responsibility. user departments are responsible for the 91 accuracy and timeliness of the data contained in their systems whereas the is department is responsible for their technical support, and for data security. all departments are aware that the data is an overall company resource and should be used to supply information wherever needed as determined by management. discussion the literature shows that there is no clear definition of is for managerial use with the terms management information systems (mis), decision support systems (dss), executive support systems (ess), eis and many others being used (see for example millet & mawhinney 1992, whymark 1990). in the course of this study we identified conflicting pressures on the use of information by different levels of management within the organisation. as a result there is no an easy solution to the problem of developing managerial information systems. these pressures result from continually changing technology, organisational structures, personality differences and the external environment. the organisation in the study in common with most others of the nineties is adopting a more distributed strategic business function structure with delegation of authority, accountability and greater autonomy of decision making to managers throughout the organisation. with the process of management itself becoming so dispersed and information intensive, there is pressure for quick fix solutions to problems of management information. this used a traditional approach to is development and a minimum of resources. in this study we have seen that this can lead to a lack of specification of real executive information needs and the supply of misinformation if there are problems with the underlying data systems. it would appear that a cautious approach using an evolutionary prototyping methodology will help to identify and remedy such problems and give the flexibility to built a more suitable is for management (guimares & saraph 1991). the prototyping approach has the advantage of encouraging greater involvement of senior management in the setting up of organisational information systems. their involvement results in better quality of management information and its alignment with the organisation's strategic objectives. conclusion despite the progress of four years the organisation of this study is far from claiming to have an ideal is for use by managers. their experience has however greatly improved their knowledge of the issues involved and has given them a way forward that is well accepted. it is yet to ascertain whether all senior managers will use the system that is being developed, in fact it is unlikely that the ceo will ever make any use of it. the advantage of this system is that it is considered as a continually evolving prototype so that it is able to adapt where necessary. more importantly, it is still not clear whether such a system actually aids managerial work and decision making. this will be an important issue as this study continues, although the criteria on which this should be measured have not been decided. the prevailing executive view is that, in the nineties, a competitive organisation must make every possible use of it at all levels, including that of senior management, for strategic advantage. 92 references broadbent m. hansell a. lloyd p. & dampney c.n.g. (1992) "managing information systems in australia and new zealand: requirements for the 1990's" the australian computer journal. vol 24 no 1 pp 1-11 crocket f (1992) "revitalizing executive information systems" sloan management review summer 1992 pp 39-47 galliers r.d. (1991) "information technology management's boon or bane?" journal of strategic information systems vol 1 no 2 pp 50-56. guimares t & saraph j. (1991) "the role of prototyping in executive decision systems", information & management vol 21 pp 257-267. hasan h & cheung l.(1993) "executives, users and is in the 1990's " proceedings of the fourth australian information system's conference. brisbane pp 553-565. millet i & mawhinney c (1992)"executive information systems a critical perspective" information and management vol 23 pp 83-92 rainer r. kelly jr & watson hugh (1992) "the evolution of executive information systems software" decision support systems vol 8 pp 333-341. selig f (1991) "managing information technology in the nineties" editorial, information & management vol 21 pp 251-255 watson h, rainer r jr & chang e.koh (1991) "executive information systems: a framework for development and a survey of current practices" mis quarterly march 199 i p p 13-30 wetherbe j (1991) "executive information requirements: getting it right" mis quarterly march 1991 pp51-65. whymark g (1991) "development of the eis concept and its implementation in the ran" the australian computer journal vol 23/3 pp 110-118 93 synthesizing expert systems and operations research techniques: system classification and review h. raghav rao* b. p. lingaraj** maithili rao*** •school of management suny/buffalo buffalo, ny 14260 **school of business and management sciences indiana univ-purdue university fort wayne, in 46805 ***planning & development fallen community health plan chestnut place 10 chestnut street worcester, ma 01608 abstract expert system (es) and operations research (or) nave many fundamental characteristics that can be used to completnent and supplement each other. this paper surveys the synthesis of es and or and discusses and classifies various expert systems that synthesize the two for managerial problem solving. key phrases: expert system. operations research, synthesis, classification. introduction problems in industry and organizations can be highly complex, sensitive to change, subject to uncertainties, and may need fast, accurate responses to aid the decision making process. traditional approaches to solving such problems include operations research methods that are either algorithmic in nature or involve simulation (hertz, 1990). such methods simplify the problem sufficiently (by constraining the scope) to be able to compute the optimum. however, a key to effective problem solving is to exploit the knowledge of the actual environment and its particular constraints. expert system (es) methods are particularly useful in such contexts. ever since the evolution of expert systems from its focus on research problems to real world problems, there has been considerable interest in the synthesis of es techniques and operations research (or) methods. the synthesis of the two disciplines could leverage human capabilities resulting in substantial progress in techniques for decision making in complex managerial situations. es and or can both be considered as part of the normative theory of procedural rationality (simon, 1987). that is, they both deal with how decisions are made, or about how decision makers should decide about optimal methods for use in any problem solving process. the goal of both es and or is to find powerful problem solving methods that give the least-cost or best-cost decision, net of computational costs. therefore, it is no wonder that in recent years there has been a spate of interest in the integration of es and or. this paper briefly reviews the characteristics of es and or methods that allow each to complement the other. it then focuses on real world applications that synthesize both expert systems and operations research techniques, and reviews such integrative attempts. a framework is developed for classification in order to provide a "simple method for discovering order in the bewildering multiplicity of nature" (phillips, 1963; rao and lingaraj, 1988). such a classificatory framework may help in finding trends in es-or integration efforts, and may help in finding areas where expert systems and operations research techniques may be successfully developed. problem solving in expert systems and operations research in the recent past, several authors have expounded on the links between artificial intelligence and operations research (finlay, 1990; fordyce, et. al., 1987; grant, 1986; ignizio, 1990; liebowitz, 1988; o'keefe, 1988; phelps, 1986; grover and greenberg, 1989). for the purposes of this paper, we focus on the expert systems aspects of artificial intelligence. (the areas of genetic algorithms, simulated annealing, tabu search, among others, will not be dealt with in this paper). in this section, we briefly review some of the characteristics of es and or in terms of the activity of problem solving. problem solving in es expert systems are founded upon the part of computer science concerned with designing intelligent computer systems, that exhibit the characteristics we associate with intelligence in human behavior: understanding language, learning, reasoning, solving problems. they deal with symbolic, non-algorithmic methods of problem solving. a common thread that runs through definitions of 45 ajis expert systems articulated in literature is the activity of problem solving (ban, cohen and feigenbaum, 1989). problem solving in es research is viewed in terms of three primary metaphors (simon, 1987): a) search b) reasoning c) constraint satisfaction. these metaphors often overlap, and thus yield powerful techniques for taming complex problems. the search metaphor is concerned with the dynamic analysis of situations. every situation is defined in terms of a state space. the state space is then searched for a solution point or a final state, through the application of a sequence of operators to transform the initial state to the goal state. the problem solution then consists of the sequence of operators that traverse the state search space from some initial state to a final goal state. reasoning is the process of accumulating information by inference till the answer to the problem is found. logical inference is normally done by theorem proving, through a procedure called resolution. two mechanisms that are most often used for deductive inference are forward and backward chaining. the forward chaining process starts with a-priori information, and basic axioms and moves to the goal state. in contrast, the backward chaining process starts with the goal and proceeds to the facts. constraint satisfaction starts with a large state space, and eliminates large chunks for applying successive constraints until it has narrowed down the original state space to a subset or unique element that satisfies all of the constraints. expert system approaches use heuristic search procedures. they are highly knowledge based, and take into account subjective judgment of the decision maker. thus es techniques emphasize knowledge elicitation and knowledge representation. analysis in expert systems means developing, modeling and programming a human's cognitive and mental processes. it is an effort to understand and develop models for resolving and dealing with complexity, ambiguity, and ill formed problem structures. problem solving in or or is defined (grant, 1986) as the application of quantitative techniques to determining an optimal solution for complex problems. it focuses on mathematical modelling and optimization techniques, with a numeric, algorithmic and procedural way of solving problems (hertz, 1990). thus, or problem solving emphasizes optimization, derivation of the analytical properties of the mathematical model, and understanding the implications of the interaction of the model with the environment an or approach to a problem involves the construction of a model that extracts the essential elements of a real-life decision problem which is inherently complex and uncertain. the relative merit of alternative actions against the decision maker's objectives are compared, and the relationship analyzed. a solution technique including the mathematical theory is developed such that it can yield an optimal value based on the decision maker's objective. this is opposed to es problem solving methodologies that try to mimic human cognitive processes. problems tend to be seen as a mapping of or tools or techniques onto a wide variety of domains. traditional or techniques are simulation, network methods, combinatorial and heuristic approaches. the choice of techniques depends on problem complexity, type of model, choice of alternative and other factors. there are problems using each of the techniques; for example, network methods cannot be applied to situations which change dynamically; combinatorial procedures cannot be used when the problems are too complex, and simulations need to have skilled interpretations. a synthesis the characteristics of es and or complement and supplement each other. both es and or models are used to help the decision maker. or methods have stressed on quantitative knowledge, es techniques incorporate qualitative knowledge. the two fields have developed different solution techniques, one precise and numerical, the other inexact and inferential. hence problem solving approaches that integrate es and or would allow the incorporation of both quantitative and qualitative aspects of a problem. qualitative theories of es can help or researchers to expand their inventory of software tools (kusiak, 1987; leibowitz, 1988). es search techniques are often satisficing, thus relaxing optimality criteria of or methods. these heuristic search methods can be applied to the solution of complex problems that cannot be modeled using optimization techniques, involve large knowledge bases, are characterized by ill-specified goals and constraints, or incorporate the design and discovery of alternatives of choice. heuristics can, for example, be used to provide structure to ill-structured problems, which can then be formally modeled and solved optimally (duchessi, et. al., 1988; fordyce et. al., 1987). [a case in may 1994 46 point is the use of a heuristic: elimination by aspects, to eliminate all dominated alternatives from a solution space, and then use or techniques to optimally solve from the non-dominated set of alternatives.] other beneficial interactions can be achieved through the use of optimization models for those parts of a system capable of mathematical description together with human style heuristic reasoning for the more complex and behavioral parts (phelps, 1986). classification based on operational modes, expert systems are categorized into two classes: (1) stand-alone systems (kusiak, 1987) and (2) tandem systems (kusiak, 1988). 1. an es in a stand-alone mode uses data and constraints pertinent to the problem and solves it using simple procedures: problem > es >solution it does not use or procedures. many existing systems fall into this class. 2. a tandem system combines the or and es approaches to solve problems. thus a tandem system may be thought of as an es linked to a database of models and algorithms. it can handle qualitative as well as quantitative data. since the models and algorithms are independent, various models and algorithms can be stored in an individual database. problem —> tandem system (models + algorithms + heuristics) —> solution simulation based expert systems and algorithm based expert systems are two subsets of tandem systems that employ various representations to model some aspect of an uncertain world. simulation based es capture the change in the status of the system by focusing on the behavior of the individual components. in contrast, the algorithm based es deal with the aggregate system behavior directly (loparo and widman, 1990; o'keefe, 1988), and are designed to compute actions that are optimal. the unking between es and or techniques can also be described in terms of two levels of coupling: shallow and deep. in shallow coupling, the or models are treated as black boxes by the es knowledge base, and are called as needed. in deeply coupled systems, the es program has, for example, intelligent front end access to a knowledge base describing the or models. the additional information allows it to select the best model for a given task, interpret the routine, and perhaps modify the models to match the problem better. real world applications of a combination of es and or methods need to be classified within a framework for better analysis and understanding. gorry and scott morton (1989) have presented a framework based on levels of managerial activities: strategic and tactical, and the degree of structure in the decision being made: structured, semi-structured and illstructured. expert systems work well in structured problem solving scenarios and reasonably well in the case of semistructured ones. however, illstructured problems are too difficult for expert systems to solve primarily because of their unpredictable characteristics (andriole, 1990). es and or approaches apply to both the tactical and strategic problems of the organization. tactical problems that are concerned with daily ongoing activities, and are repetitive in nature include production scheduling and inventory control, balancing assembly-line facilities, facility maintenance and repair, and inspection for quality control. strategic problems have a planning and more global orientation, thus bearing on the daily operations only indirectly. for example, selection of plant sites, plant-expansion programs, allocation of resources for space exploration are all strategic problems (o'keefe, 1985,1986,1988). raiszadeh and lingaraj (1986) propose a two dimensional framework of classification of es/or applications in research literature based on (i) orientation, whether strategically oriented or tactically oriented and (ii) decision, the degree of structure in the decision being made. in the framework proposed here, we adapt the raiszadeh lingaraj, and the gorry and scon-morton frameworks to include the operational mode of the es in real world applications. thus the resultant framework consists of three dimensions. the first dimension separates strategically oriented applications with long term implications and tactically oriented applications with short and medium term implications. the second dimension refers to the degree of structure in the decision being made. the distinction is 47 ajis explore eft choices, coats illustrates the use of simulation and es technology to create an eft decision support model. the simulation model emulates the behavior and economics of actual eft networks. the interaction between the user and the model is handled with a query-response sequence using a structured-interactive natural language dialogue. the es feature guides the user through the input process, provides explanation of input categories and reminds the user of the previous input its chief contribution, however, is its flexibility in understanding requests and responses from the user. domain knowledge about queuing system and modeling rules drives the determination of what information is solicited from the banker during the interactive terminal session. tactical oriented: the systems discussed below deal with decisions which are of tactical importance to the organization. es in logistics: (helferich et. al., 1990) many decisions in logistics are made under complex circumstances by acquiring information about particular problems and the applying "rules of thumb" obtained through experience. there are four broad areas in logistics where es are well suited: (a) monitoring and control, (b) exception processing assistance, (c) as an on-line assistant in assisting routine decisions and (d) fault diagnosis of machines. here, we focus on the first. monitoring and control of functions provide a method of assessing and evaluating functional performance. for example, the weekly reports from an inventory management control system can be used as an input into an es designed to evaluate inventory performance in terms of percentage fill rates, backorders, stockouts, inventory value on hand etc. the"rules of thumb" input to the es knowledge base are used during the inference process where the actual report results are compared to the facts in the knowledge base. the comparison leads to the computation of an inventory performance grade. depending on the grade, the inventory manager can ask the es for the reasoning behind the grade or request recommended actions for improvement rectangular cutting stock problem: (daniels and ghandfaroush, 1990) es can be used to solve the two dimensional, rectangular cutting stock problem for determining how best to lay out a number of rectangular or square packages on a rectangular pallet this system uses 13 basic rules to recursively examine the pallet space and the preferred layout the rules determine how to arrange packages and how many packages may then be positioned on the pallet; keep track of the number of packages laid out; vary termination conditions and also handle situations where there is usable space left on the pallet navy equipment failure : (wauchope, 1990) terse is a knowledge based text reduction system for highlighting important information in the narrative portion of navy equipment failure messages. the system contains two knowledge bases for message evaluation, one equipment specific and the other, general. the system also contains general heuristics that use the equipment model network to infer causal relationships believed to be implicit to the message. the general portion of the system performs semantic normalization, infers and tags key categories of information and finally ranks the causes of equipment failure by applying user evaluation criteria, represented as numeric scores assigned to various patterns of information types. fault diagnosis and repair: (pate-cornell, et. al., 1988) a system may fail in such a way that it may be unclear from the symptoms how the failure occurred and what repairs are needed. traditionally optimization methods have been used to conduct a diagnosis search given some criterion (minimisation of costs or minimization of time). a number of practical problems result when fault problems arise due to a system's complexity which make analytical methods impractical. for example, how to handle a large number of potential failures or how to treat probabilistic dependencies and functional dependencies among failures of different parts are typical problems. (probabilistic dependencies mean that along the diagnosis path, the probability that a yet unchecked part has failed, depends on the information obtained thus far. functional dependencies would mean that the system could be dependent on other physical parts or subsystems). systems that solve the fault diagnosis and repair problems rely on the recommendation of an expert whose expertise is encoded in a knowledge base and used when a failure occurs. time warp operating system (twos): (reiher, and jefferson, 1990) this operating system, developed at the jet propulsion laboratory for the caltech mark iii hypercube multiprocessor 49 ajis has the primary goal of concurrent execution of large, irregular discrete event simulations at maximum speed. the main innovation that distinguishes twos from other systems is its complete commitment to an optimistic style of execution, and to process rollback. the optimistic style of execution assumes at each moment that the messages already in the input queue are the "true next" ones and proceeds accordingly. process rollback involves setting back the process to a time just before time t for a message that arrives with a timestamp t less than some that have been already executed. systems for semi-structured problems: the following systems are used for semistructured decision problems. strategically oriented: the systems discussed here are used in decisions which are of strategic importance to the organization. aurora: (yu, 1988) the automatic reasoning with operations research applications (aurora) is a system which decomposes and solves planning problems with operations research and expert system techniques. the problems addressed are general planning problems, including assignment, routing and scheduling with knowledge of a variety of solvable subproblems and available techniques. aurora attempts to stimulate the human thought process. it uses a sophisticated metaplanner to solve the scheduling subproblem of tactical air planning problems. a pattern matcher is used to identify multiple occurrences of a specified pattern and in particular a linear subproblem an automatic programming system produces an executable code to solve the linear subproblem using the simplex technique. process modelling for decision making: (hall, 1988) this approach is used to conceptualize an organization as a system of resource flows controlled by an interacting set of policy making processes. the process model in an organizational context describes how things are decided and done by people and groups. the model involves two parts: the first part attempts to model and simulate how the organization works as an integrated flow of resources and die second part attempts to provide a rule based system that captures the essence of the collective decision making behavior of managers in an organizational setting. the model can be used as an alternative policy tool for companies in crisis where the effect of the natural policy making process on corporate systems is not giving the desired results and the reason is not obvious. strategic level gaming: (davis et. al., 1989) the rand corporation has developed a large scale program modelling and simulation system in the domain of game structured military strategic analysis that combines features of human political-military war gaming and analytical modeling. known as mark i, it involves a synthesis of knowledge-based modeling and more traditional simulation modeling. it replaces human teams by automated models called "agents". mark i assumes that the decision making agents would depend primarily on qualitative heuristic rules rather than on optimizing algorithms. the system is characterized by four agents: a red agent who makes decisions for the soviet union and the warsaw pact countries; a blue agent who makes decisions for the us and nato; a scenario agent who makes decisions for non superpower countries; and a force agent who simulates results of combat and other military operations and also controls simulation time. the red agent makes decisions by (a) comparing a is dimensional characterization of the current world state with a set of world states previously considered and identified in data (b) choosing the world state that is closest to the current one and (c) following rules associated with that state. the scenario agent is a rule based model built in an english-like rand es language called rose that allows users to review and change rules interactively. the force agent consists of simple combat simulation models developed in previous rand work. tactical oriented: the systems discussed below deal with decisions which are tactical in nature. urogen: (scherer. et. al., 1987) urogen is an es that is intended to aid the physician in the treatment and diagnosis of female urogenital complaints. it is also intended as a learning aid since it enables the user to observe the probabilistic results of different management strategies on hypothetical patients. it uses a combination of probabilistic information and expert rules to provide information regarding the value of certain diagnostic tests and effectiveness of numerous treatment strategies. may 1994 50 urogen contains four sub-models of the female urogenital complaint problem. the sub-models determine the probabilities of the various underlying causes of the complaints, as a function of the current patient history. in addition, a fifth model, the global model, calculates the probabilities of the patient having combinations of the sub-problems and the probability of the patient having none of the considered problems, using a bayesian updating algorithm. gate assignment problem: (brazille, and swigger, 1991; srihari, and muthukrishnan, 1991) the es is used to monitor the allocation of gate space to arriving aircraft at an airline hub. it advises on how to allocate the gates to aircraft in a cost effective manner and helps recover a portion of hub operation costs by reducing ground delay for aircraft waiting for an available gate at a terminal. this translates into increased aircraft availability and crew utilization and increase in customer satisfaction. the gate assignment procedures obtained at the particular site become the rules of knowledge base. the system uses an external file known as the terminal record that describes the current scenario of individual gates occupied by specific flights for a specific time. a second file known as the arrival record describes the actual arrival sequence of the incoming flights. the system is capable of determining the suitability of all gates when a single arriving flight is specified. in order to arrive at a suitable gate for the arriving aircraft, es utilizes a priority list as a knowledge based system parameter. navy personnel assignment problem: (clark, et. al., 1988) there are about 500,000 enlisted personnel and 70,000 officers in the us navy. the task of assigning the personnel to jobs every month, was done manually till the development of the es. by using the es to provide an interface to a numerical optimization program, a composite system which allows effective development of numerical transshipment models for navy personnel assignment has been produced. the es is able to examine the functioning and solution produced by the algorithm, including the effects of slightly changing the inputs, and report the findings to the user. the es can also explain comprehensively why a particular component of the solution was selected by the algorithm. es for flexible manufacturing systems: (kusiak, 1987; kusiak and chen, 1988) there are two basic flexible manufacturing system problems: the design problem and the operation problem. the design problem can be further decomposed into product design consisting of part design and process planning, and system design consisting of equipment design and layout design. the fms operational problem may have the following structure -aggregate planning, resource planning, disaggregate planning and scheduling. both es and or approaches have been applied to almost all design and operational problems. a typical problem in cellular manufacturing involves the grouping of machines and parts, where each partial solution generated by the algorithms suitable for solving grouping problems is evaluated by the expert system, until the final solution is determined. similarly es have been used to solve the fms scheduling problem. bruno (kusaik and chen, 1988) has developed an expert scheduling system for scheduling parts in a flexible machining environment the parts are grouped into batches, each containing 100 to 200 parts. in order to generate a release time for each batch a dynamic priority scheme is used. the reasoning process is partially controlled by the simulation subsystem. bensana (kusaik and chen, 1988) has presented a job shop scheduling system called opal. when operations cannot be ordered by a constraint based analysis module, and no more precedence constraints can be discovered, the rule based decision support module is called upon to select a new pair of operations. the control strategy of the decision support module is based on fuzzy set methodology. the description of the various expert systems are tabulated in table 2. conclusion expert systems that synthesize es and or have been used in multiple ways, ranging from model selection and subsequent data extraction for satisfying the data requirements of the model, to constructing appropriate models or modifying existing models for use with specific problems, and even modifying solutions for ease of implementation. the majority of the systems surveyed, fall into the tactically oriented application category, and are algorithm based tandem systems. the fact that strategic applications do exist, (though small in number) is evidence of the promise of the applicability of tandem systems. 51 ajis the fact that expert system and operations research complement each other in many ways is a feature that has far reaching prospects for human-computer problem solving. the approaches that have been effectively used in real life problems, as shown in this paper, are evidence for the fact. specifically, one can see interesting prospects when a decision maker and a decision support system that incorporates es and or techniques, interact to solve a problem. such systems can supplement the knowledge processing capabilities of decision makers. for instance, they can give novices expert guidance, and thus help them to come up with the same quality of decisions as an expert in a specific domain. the synthesis of the disciplines could leverage capabilities resulting in substantial progress in techniques for decision making in complex managerial situations. adding es methods to the inventory of tools for or researchers opens up new domains of application. it provides powerful methods for making use of non-quantitative information. since es uses a cognitive style compatible with that of managers, the es/or systems can give rise to a better expert that can deal with more complex problems effectively. for instance, es production techniques in or simulations is an area that has much promise. the handling of decision rules in or simulations is generally fixed. an embedded es system would enable the simulation to incorporate an intelligent choice of rules to better model human decision making. alternatively, optimization techniques can be used to help in resolving some of the limitations of es methods. they could help in increasing an es systems performance and produce better recommendations for decision making. or can contribute its experience in knowledge acquisition and model building to the development of es systems thus resulting in the production of es systems based on or expertise. such systems may be useful in situations requiring models based on heuristics. they can also deliver guidelines derived from models, choose and formulate appropriate models, and interpret model results by formalizing the knowledge embedded in the applications of or methods. expert system (and system type) aurora (deep coupling) urogen (shallow coupling) logistics (shallow coupling) gate assignment problem (shallow coupling) fms (shallow coupling and some deep coupling) rectangular cutting stock problem (deep coupling) vehicle routing (shallow coupling) terse (deep coupling) es method automatic programmer, metaplanner and pattern matcher probabilistic information, expert rules and probabilistic rules knowledge base, evaluation rules feasibility rules, knowledge base knowledge based rules and frames knowledge based rules knowledge based rules knowledge based rules or/ simulation method lp algorithms bayesian updating algorithms algorithms lp or ip algorithm optimization algoritl algorithms route algorithms or model description decomposes the planning problems using me pattern niatchcf and the metaplanncr into subproblems that can be solved using or algorithms makes use of the information provided by the patient's case history and makes inference using probabilistic information regarding the diagnosis. uses the expert knowledge base and solves problems such as, exception processing assistance or on-line assistance in routing decisions, using heuristics the knowledge base finds the gates suitable for a flight and then uses the or algorithm for gate assignment used in operations such as job shop scheduling, planning where priorities are established using tl are used the rules examine the pallet space and then uses the algorithm to give the preferred layout this process is repeated until an optimal layout is achieved the knowledge base helps select the best route based on experience. the system uses two systems, one equipment specific and the other equipment general to infer the messages that are sent and inferring the causal relations for equipment failure lang. used zetalisp na ml na prolog, lis ops5 fortran prolog insight! pascal ops5 may 1994 52 electronic money networks (deep coupling) fault diagnosis and repair (deep coupling) time warp operating system (shallow coupling) process modelling (shallow coupling) strategic level gaming (deep coupling) modelling rules rules derived from probabilistic analysis rules for process rollback rule based rule based simulation analytical optimization discrete event simulation simulation simulation the rules formulate the user inputs in a teller machine into the form accepted by the model and offers explanation to the user of various input categories the symptom-fault knowledge base, fault fixing knowledge base and verification knowledge base are used to both diagnose and offer solution to faults the system helps concurrent executions of large, irregular, discrete event simulations at maximum speed by using distributed roll back mechanism for synchronization the rules form corporate goals, maps environment, formulates plans and help add the qualitative information into the model which simulates corporate decision making simulated multiple agents war game in which rules are used to hrfmg the role of the multiple agents prolog na c c randabel rose table 2: expert system details *nanot available acknowledgement the authors would like to thank ariyapadi krishnaraj of suny at buffalo for his excellent research assistance. references andriole, s.l. (1990) "command and control information systems engineering: progress and prospects", advances in computers, vol 31, pp 2-98. barr, a., cohen, p.r. & feigenbaum, e.a. (1989) the handbook of artificial intelligence, addison wesley. brazille, r.p. & swigger, k.m. (1991) "generalized heuristics for the gate assignment problem", control and computers, vol 19, no 1, pp 27-32. clark, p., niblett, t. and thompson, t. (1988) "the application of expert systems to the "black box' problem: a case study for the navy personnel assignment problem", report for the office of naval research, contract n00014-85-g-0243, presented at tims/orsa joint national meeting, washington d.c., april 27. coats, p.k. (1989) "a banker's use of simulation and artificial intelligence for assessing the economics of electronic money networks", european journal of operational research (netherlands) august, vol 41, issue 3, pp 290-301. daniels, jj. & ghandforoush, p. (1990) "an improved algorithm for the non-guillotine-constrained cuttingstock problem", journal of the operational research society, february, vol 41, 2, pp 141-149. davis, p.k. (1986) "applying artificial intelligence techniques to strategic-level gaming and simulation", in m.s. elzas, t.i. oren, and b.p. zeigler (eds.) modelling and simulation methodology in the artificial intelligence era, north holland, amsterdam. duchessi, p., belardo, s. & seagle, j.p. (1988) "artificial intelligence and the management science practitioner: knowledge enhancements to a decision support system for vehicle routing", interfaces 18:2, marchapril, pp 85-93. finlay, p.n., (1990) "decision support systems and expert systems: a comparison of their components and design methodologies", computers and operations research, vol 17, no 6, pp 535-543. fordyce, k., norden, p. & sullivan, g. (1987) "artificial intelligence and the management science practitioner links between operations research and expert systems", interfaces, 17:4, july-august, pp 34-40. fordyce, k., norden, p. & sullivan, g. (1987) "review of expert systems for the management science practitioner", interfaces, 17:2 march-april, pp 64-77. grant, tl. (1986) "lessons for or. from ai: a scheduling case study", journal of operational research society, vol 37, no 1, pp 41-57. 53 ajis grover, f., & greenberg, h.j. (1989) "new approaches for heuristic search: a bilateral linkage with artificial intelligence", european journal of operational research, 39, pp 119-130. gorry, g.a. & scott morton, m.s. (1989) "a framework for management information systems", sloan management review, reprinted, spring. hall r.i. (1990) "building an artificial intelligence model of management policy making: a tool for exploring organizational issues," in masuch, m. (ed.) organizations, management and expert systems: models of automated reasoning, walter de guyter, ny, pp 105-121. helferich o.k., taylor, l.a. & trecha, s.j. (1990) "artificial intelligence/expert systems in logistics: putting the theory into practice", technical report, dialog systems, east lansing, mi. hertz, d.b., (1990) "the knowledge engineering basis of expert systems," expert systems applications, vol 1,1, pp 79-84. ignizio, j.p., "a brief introduction to expert systems", computers and operations research, vol 17, no 6, pp 523-533. kusiak, a. (1987) "artificial intelligence and operations research in flexible manufacturing systems", infor, vol 25, no 1. kusiak, a. & chen, m. (1988) "expert systems for planning and scheduling manufacturing systems", european journal of operational research, vol 34, pp 113-140. liebowitz, j. (1988) "beyond decision support systems: the role of operations research in expert systems", computers ind. eng, vol 14, no 4, pp 415-418. loparo, k.a. & widman, l.e. (1990) "artificial intelligence, simulation and modeling", interfaces, 20:2, march-april, pp 48-66. o'keefe, r.m. (1988) "artificial intelligence and the management science practitioner: expert systems and ms/or methodology", interfaces 18:6, november-december, pp 105-113. o'keefe, r.m. (1985) "expert systems and operational research mutual benefits", journal of operational research society, vol 36, no 2, pp 125-129. o'keefe, r.m., belton, v. & ball, t. (1986) "experiences with using expert systems in o.r.", journal of operational research society, vol 37, no 7, pp 657-668. pate-cornell, e., tse, e., lee, h. & shachter, r. (1988) "fault diagnosis and repair in complex systems: heuristics vs optimization", research report to ibm, department of industrial engineering, stanford university, presented at tims/orsa joint national meeting, washington d.c., april 27. phelps, r.i. (1986) "artificial intelligence an overview of similarities with o.r.", journal of operational research society, vol 37, no 1, pp 13-20. phillips, w.h. (1963) a primer on book classification, association of assistant librarians, london. rao, h.r. & lingaraj, b.p. (1988) "expert systems in production and operations management: classification and prospects", interfaces, vol 8, no 6, december. raiszadeh, f.m.e. & lingaraj, b.p. (1986) "real-world o.r7m.s. applications in journals", journal of operational research society, vol 37, no 10, pp 937-942. reiher, p.l. & jefferson, d. (1990) "dynamic load management in the time warp operating system", transactions of the society for computer simulation, vol 7,2, pp 91-120, june. simon, h.a. (1987) 'two heads are better than one: the collaboration between ai and or", interfaces 17:4, july-august scherer, w.t., white, c.c. ffl., wilson, e.g. (1987) "urogen: an expert system for the diagnosis and treatment of female urogenitial complaints", proceedings of the eleventh annual symposium on computer applications in medical care, november 1-4, washington d.c. srihari, k & muthukrishnan, r. (1991) "an expert system methodology for aircraft-gate assignment",computers and industrial engineering, vol 21, pp 101-105. wauchope, k. (1990) "structural domain modeling for understanding equipment failure messages", ieee proceedings of southeastern x90technologies today and tomorrow, new orleans, la. yu, d. (1988) "a system for utilizing operations research techniques in knowledge-based planning systems" technical report d78, the mitre corporation, bedford, ma. zahedi, f. (1987) "artificial intelligence and the management science practitioner: the economics of expert systems and the contribution of ma/or", interfaces 17:5, september-october, pp 72-81. may 1994 54 towards the proper integration of extra-functional requirements hike hochmuller department of telematics/network engineering university for applied sciences cannthia tech institute / technikum karnten primoschgasse 3, a-9020 klagenfurt austria hochm@cti.ac.at abstract in spite of the many achievements in software engineering, proper treatment of extra-functional requirements (also known as non-functional requirements) within the software development process is still a challenge to our discipline. the application of functionality-biased software development methodologies can lead to major contradictions in the joint modelling of functional and extra-functional requirements. based on a thorough discussion on the nature of extra-functional requirements as well as on open issues in coping with them, this paper emphasizes the role of extra-functional requirements in the software development process. particularly, a framework supporting the explicit integration of extra-functional requirements into a conventional phase-driven process model is proposed and outlined. keywords: extra-functional requirements, process modelling, integration framework motivation many advances have been achieved in the area of software engineering during the last 30 years. the main breakthrough was certainly the shift from the highly artistic way of programming to a systematic and phasestructured process of software development. many process models were proposed and applied so far. in order to support the various activities within the software development process, a broad spectrum of methods, techniques, and tools is currently available. most of them concentrate on functionality. but functionality is not the only dimension that matters in software development (cf. mittermeir 1996; potts 1997(a)). a whole bunch of additional software product dimensions is put up by product quality requirements (like security, performance, usability). these requirements may be at least as important as the functional ones since a product must possess an acceptable degree of quality such that it becomes worth being used (cf. fitness for use: potts 1997(b)). in the sequel, these extra categories of requirements will be simply called extra-functional requirements. the deficiencies of giving extra-functional requirements proper treatment can mainly be traced back to inadequate techniques to grasp as well as to express them in an appropriate manner because of their rather informal nature. currently available and applied (analysis and design) methods, techniques and tools mainly focus on objects, functions and states. as a consequence, quality requirements are often simply forgotten or even neglected during analysis. they then will turn up again far too late during acceptance test or even during operation, but obviously quality cannot be added to a software product a-posteriori (bertolino 1994). moreover, quality has to be ensured from the very beginning of the software life cycle (rai et al. 1998). this requires to inherently incorporate quality related aspects into each software development process. this paper serves four purposes. first, it shows that there definitely are extra-functional requirements which cannot be treated as mere attachments to functions (cf. chapter 2). second, the process-related challenges in dealing with extra-functional requirements are elaborated (cf. chapter 3). third, the role of extra-functional requirements within the software development process as well as within the software product is discussed (cf. chapter 4). fourth, a framework is proposed for a better integration of extra-functional requirements with existing phase-oriented process models (cf. chapter 5). extra-functional requirements while functional requirements describe the intended purpose of a system component, extra-functional requirements (efrs) are constraints regarding quality (e.g. performance, maintainability, usability) as well as economics (e.g. time, cost) of process or product components. distinguishing between function and behavior (cf. chandrasekaran et al. 1993; kaindl 1995), a function is what is necessary or desired (e.g. a car should be able to transport persons) and the behaviors are how this result is attained (e.g. a car should run on streets). in our car example, extra-functional requirements may be stated as environmental conditions like: • the car need not run on water but on sealed roads. • with a lower speed it should also be able to run on bumpy roads. 98 similar conditions may also matter in the case of software systems. table 1 contains some examples which illustrate the analogy between extra-functional requirements of 'real-life' systems (e.g. car) and software systems (e.g. distributed flight reservation system). car the car should run at least 8000 miles without oil change. break, clutch and accelerator pedals should be arranged according their standard positions. indicating instruments should be independent of the driver's nationality (e.g. usage of icons). experienced drivers should be able to use the car within 10 minutes of preparation. the car should still operate in case of 20 percent of overweight. the car should dispose of two independent break mechanisms. car driving should not keep the driver fully occupied such that not even a little chat with the passengers is possible. a service man should be able to check 4 cars per hour. the car should satisfy legal provisions (e.g. exhaust gas limits). flight reservation system (frs) the frs should be available 99.99 percent of the time in any given calendar year. standard elements should be used for the user interface (e.g. windows, icons, mouse). the user interface should be multilingual (e.g. english and german; icons). experienced computer users should be able to use the system after a training period of 1 hour. if the initial amount of users increases by 50 percent, the response time must not increase by more than 20 percent. in case of a network failure, the local system should be able to operate independently. the usage of the frs should not hinder from talking with clients in the office or via the phone. the technician should be able to carry out his maintaining tasks on the whole system within his 20 office hours. the frs must be certified concerning the specified security system. table 1. extra-functional requirements examples analysing functional requirements in a top-down manner leads to the definition of behavioral and structural models which nowadays can be represented in a quite straight-forward way using adequate modelling techniques, e.g. sa/sd (yourdon 1978; yourdon 1989), er (chen 1976), jsd (jackson 1983), ooa (coad & yourdon 1991), omt (rumbaugh et al. 1991), sadt (ross & schoman 1977), uml (rumbaugh et al. 1998). as opposed to functional requirements, extra-functional requirements are not embodied thoroughly in the analysis process. they are difficult to elicit and to represent, have a multi-faceted structure, may constrain special functional requirements, the system as a whole or the development process on its own, influence (positively or negatively) other extra-functional requirements. compliance of extra-functional requirements cannot easily be qualified as right or wrong, the results can rather be compared along an ordinal scale giving good or bad solutions (loucopoulos & karakostas 1995). the structure of extra-functional requirements can soon overcharge our capabilities in dealing with complexity. table 2 illustrates the contrast between functional and extra-functional requirements. functional requirements describe what a system should do purpose are mainly focused by analysis process sufficient methods for analysis available standard techniques/notations for representation easy to elicit decomposition possible module test sufficient are largely independent of each other extra-functional requirements delimit the solution space constraints are not inherently embodied in analysis process few methods for analysis known no representation standards difficult to elicit multi-faceted structure often complete system necessary for verification can contradict each other table 2. functional vs extra-functional requirements 99 extra-functional requirements are different within the computer science community, the interpretation of extra-functionality varies considerably. symptomatic for this situation are the many different definitions in the literature (cf. section 2.2.) and recent debates (refsq'94 (starke 1994), iwssd-8 (kramer & wolf 1996), bcs refsg&facs joint workshop on requirements engineering and formal methods, refsq'97 (dubois et al. 1997)) about the existence of any independent extra-functional dimension. certainly, there are some kinds of extra-functional requirements which can simply be attached to functional requirements. for example, performance requirements can be quantified in terms of time periods; existing modelling techniques like state transition diagrams or data flow diagrams can easily be extended to express such time constraints (cf. davis 1993). other kinds of extra-functional requirements, however, cannot be simply modelled as attachments to functional requirements. in the sequel, this will be illustrated by examples from the treatment of synchronization constraints in object-oriented concurrent programming (oocp) and security constraints in oodbmss representing two very different categories of extra-functional requirements originating from two different fields of computer science. inheritance contradictions because of synchronization constraints in oocp in object-oriented concurrent programming, messages are allowed to arrive at an object in any order and even simultaneously. for the purpose of maintaining its internal integrity, an object in a certain state may only accept a subset of its entire set of messages. such a restriction on the acceptable message set is called synchronization constraint. the object behavior that satisfies a synchronization constraint is implemented by a particular portion of the method code which is referred to as synchronization code. the fact that synchronization code cannot be effectively inherited without non-trivial class re-definitions was identified as inheritance anomaly and is deeply discussed by matsuoka & yonezawa (1993) using the example of a bounded buffer to illustrate the three different categories of anomalies: • state partitioning anomaly • history-only sensitive anomaly • state modification anomaly using boundedbuffer as common superclass, the omt object schema in fig. 1 outlines an example for each of the three different anomalies arising on subclass definition. boundedbuffer in bufferfsizel put() get() lock locked lock() unlockq fig. 1. inheritance anomalies in oocp 1 boundedbuffer2 get2(x/ get 2 items state partitioning 1 hboundedbuffer after-put gget(>/ get after get puto gct() history-only sensitiveness 1 1 lboundedbuffer put() get() state modification subclass boundedbuffer2 inherits the methods put() and get() from boundedbuffer and additionally defines a method get2() which automatically removes two elements from the buffer. the associated synchronization constraint allows instances of boundedbuffer2 only to accept message get2() if the buffer contains at least two elements. hence, the state allowing the acceptance of both get() and put() has to be partitioned into two states in order to be able to distinguish the case where just one element is in the buffer and get2() is not allowed. this is a simple example for a state partitioning anomaly. while state partitioning anomalies can be overcome by using 100 method guards as synchronization mechanism, the next two types of inheritance anomalies are much more problematic to deal with. subclass hboundedbuffer defines a new method gget() which behaves like method get() inherited from boundedbuffer but cannot be accepted immediately after the invocation of put(). a new instance variable afterput for recording the necessary history information has to be defined. as after-put has to be adjusted in the methods get() and put(), the synchronization code of both methods have to be overridden in subclass hboundedbuffer and history-only sensitive anomaly arises. state modification anomalies can arise in the presence of multiple inheritance. let us consider subclass lboundedbuffer which additionally inherits the instance variable locked from a second superclass lock. the inherited method lockq orthogonally restricts the set of states in which each method inherited from boundedbuffer can be invoked. in order to be able to evaluate the instance variable locked, the synchronization parts of methods get() and put() must be overridden in subclass lboundedbuffer. classes hboundedbuffer and lboundedbuffer are expressive examples for method overriding which becomes necessary in order to guarantee compliance of the corresponding synchronization constraints. however, the pure functionality of get() and put() remains the same (get() removes one element from the buffer; put() adds one element to the buffer). the original work (matsuoka & yonezawa 1993) on the problem of inheritance anomaly did not explicitly address extra-functional requirements. nevertheless, it is a proof of the existence of a particular additional dimension which is something else than just pure functionality. although the used example is very simple, it did already demonstrate that tight interweaved treatment of functional requirements and extra-functional requirements (like synchronization constraints) can lead to major development problems resulting in rather complex and unreadable software components which are difficult to maintain. schema distortion through security constraints in oodbmss similar anomalies can arise in the context of modelling security for object-oriented database management systems (oodbmss). especially mandatory security models which represent multi-level real-world entities as single-level objects may have grave impacts on conceptual schema design. this can be illustrated even by such a small example as shown in fig. 2. mandatory access control requires to label all objects in a database based on their sensitivity (classification). the active processes requesting access to database objects are called subjects and are assigned to a clearance level. a strictly enforced central security policy (bell-lapadula concept) disallows the information flow from (data) objects classified for a higher security level to subjects with a lower security clearance. most mandatory security models for oodbmss (e.g. message filter model described by jajodia et al., 1995) require each object class to be classified at one single level such that all attributes and methods of an object have the same classification. fig. 2 represents the realization of a simple security constraint. let us assume type employee and subtype manager to be part of an omt object schema. type elements can be either unclassified (u) or secure (s). employee has unclassified attributes ssn and name and a secure attribute salary. type manager has an additional unclassified attribute bonus. most multilevel approaches split each of the conceptual types employee and manager into two separate single-level parts, a secure and an unclassified one. in fig. 2, the unfamiliar dashed lines linking the secure and the unclassified components of each of the two conceptual types may be interpreted either as security induced inheritance (jajodia et al. 1995) or as association between composite objects (bertino & jajodia 1993). both variants provide means to view just the unclassified parts or to access the whole conceptual object. 101 fig. 2. simple security constraint and multilevel classes in the presence of more classification levels and/or more complex security constraints (cf. dobrovnik & hochmuller 1998) the single-level implementation strategy can lead to rather complicated structures in the conceptual schema. the pure conceptual semantics of the schema are heavily distorted by the realization of security requirements. the following anomalies resulting from tight interweaving of semantic (conceptual) and security dimensions were identified and discussed elsewhere (hochmuller 1998): • class partitioning • class rearrangement • object reassignment how to avoid the contradictions between both dimensions by using object-oriented views as security mechanism was already outlined previously in hochmuller 1998 and thoroughly demonstrated by dobrovnik & hochmuller (1998). an in-depth discussion of limits in inheritance structures in object-oriented system development (synchronization in oocp and security in oodbmss) arising from tight interweavings of functional and extrafunctional requirements which cannot be simply overcome by multiple inheritance can be found in hochmuller 1998. other kinds of extra-functional requirements like the often cited cost and performance criteria are less problematic because they can usually be attached to specific functions quite easily. the examples from the security and synchronization areas given above are qualitatively different since no function can be identified to attach them naturally to. thus, there are definitely extra-functional requirements which cannot be treated as addons. taxonomy previous work in the area of extra-functional requirements was mainly concerned with their classification. in the literature, various synonyms for extra-functional requirements (efrs) are used, e.g. non-functional requirements (nfrs), non-behavioral requirements, or (software) quality requirements. although the term 'non-functional' is used more often than the others, it has a far too negative touch. also, it is misleading in the sense that it suggests the existence of requirements which have nothing to do with functionality, at all. however, non-functional requirements will sooner or later be incorporated in functional solutions. the debates within the scientific community about the (non-)existence of non-functional requirements mainly originate from that misinterpretation. hence, the term 'extra-functional' which originally was coined by mary shaw in the context of software architectures (shaw 1996) seems to be more appropriate to denote something which does not directly address the functionality/purpose of a system and which may additionally impose constraints on the functionality dimension. consequently, throughout the sequel, the term 'extra-functional' will be used systematically to avoid the possible irritation due to the term 'non-functional'. however, since the semantics of both terms do not differ, they could also be used as synonyms, if the reader prefers to do so. one of the first characterization approaches dates back to 1976 when boehm introduced a hierarchical list of quality attributes (boehm 1976). his recent work (boehm & in 1996) includes the mapping of primary concerns of stakeholders onto these quality criteria in order to get a better starting point for negotiations in case of conflicting requirements. davis (1993) prefers the notions of behavioral and non-behavioral requirements and focusses primarily on reliability and efficiency issues. the ieee std. 830 (1993) defines non-functional requirements in terms of performance, external interfaces, design constraints, and quality attributes. in addition to functionality, the iso 9126 standard (1991) distinguishes five classes of non-functional requirements as main characteristics for evaluation of product quality, namely reliability, usability, efficiency, maintainability, and portability. roman (1985) classifies non-functional requirements according to interface constraints, performance constraints, operating constraints, life cycle constraints, economic constraints, and political constraints. a methodology (keller et al. 1990) developed by the rome air development center (radc) considers nonfunctional requirements from two perspectives and distinguishes between user-oriented software quality factors (e.g. efficiency, correctness, interoperability) and system-oriented software quality criteria (e.g. completeness, anomaly management, functional scope). all approaches mentioned above address quality of and constraints on a particular software product they are so-called 'product-centered' approaches. less attention, however, is paid to the quality of and constraints on the software development process on its own. only few approaches address also requirements upon the development process, like those of mittermeir (1990) and of sommerville (1992) distinguishing between product requirements, process requirements, and external requirements, which can be legal or economic 102 constraints placed on both product and process. dobson & mcdermid (1991) try to avoid an explicit distinction between extra-functional and functional requirements and propose 4 kinds of (product as well as process) concepts (commitments to specific policies, obligations which are functionality-specific constraints, external constraints, and derived properties). here, classical extra-functional requirements can be mainly perceived as obligations as they are specific goals for particular systems but may also be assigned to commitments because of their generality. as can be seen from the amount of and differences between all of those classification approaches, there does not exist one single taxonomy which is powerful enough to identify and classify comprehensively all types of extrafunctional requirements regardless of the prevailing context. because of the multi-faceted and contextdependent nature of extra-functional requirements, chances are low for establishing a fully fledged classification schema which is suitable for any kind of project. existing product-centered classification approaches propose various enumerations of software quality attributes which are sometimes ordered within a hierarchical structure (e.g. boehm 1976; iso/iec 1991). the actual hierarchy depends on the selected quality attributes and their relationships (as they are commonly accepted or perceived by the author). a further ordering can also take place according to different viewpoints like primary concerns of stakeholders (boehm & in 1996). approaches which additionally consider process and external requirements (e.g. mittermeir 1990; sommerville 1992) also classify extra-functional requirements hierarchically and distinguish between product, process, and external requirements at the topmost level leading to three different trees. all these classification approaches have certainly their eligibilities, but lack in the ability to structure extrafunctional requirements along more than one single dimension. the generic classification framework for extrafunctional requirements presented in hochmiiller 1997(a) is based on the faceted classification technique which has already been successfully applied in the area of software reuse (prieto-diaz 1991). faceted classification is a synthetic approach and enables the definition of more than just one single dimension. classes are assembled by selecting predefined keywords from faceted lists. this approach provides higher accuracy and flexibility in classification. a faceted schema may have several facets (dimensions) and each facet may consist of several terms. the core dimensions of the faceted classification schema can be roughly summarized when regarding the different ingredients of requirements engineering. it is obvious, that requirements engineering is a process to establish a (composite) product. different stakeholders (with different viewpoints) are involved in the requirements engineering process. they can be regarded as sources of requirements on certain features of parts of the product or/and of parts (phases) of the development process. these requirements are captured in terms of representations. hence, the following five facets are proposed as core dimensions: • the part of the product, for which the requirement is a constraint (e.g. functions and procedures, behavioral and structural components, user interface) • the phase of the process, for which the requirement is a constraint (e.g. analysis, design, or programming phases) • the source, which is the origin for the requirement (e.g. different stakeholders, law, standards) • the feature, which the requirement addresses (actual efr category) • the form of representation, which describes the requirement (e.g. just free text or annotations in entityrelationship diagrams, state transition diagrams,...) product whole product hardware interface whole software function behavior structure database user interface documentation design code process procurement project management analysis design programming component test system test installation maintenance source domain general objective client user project manager law economics standards internal guidelines feature time cost performance usability availability security reliability efficiency flexibility expandability portability maintainability testability understandability modifyability representation er diagram object diagram state transition diagram data flow diagram structure chart functional specification free text test plan 103 table 3. faceted classification schema for extra-functional requirements table 3 shows the resulting core classification framework. it consists of the five facets together with an initial set of related terms. these terms should be regarded just as examples for the purpose of illustration. an actual instance of the classification scheme will consist of far more specific terms with usually finer granularity. for example, 'time' is a rather general term which can be specialized to 'response time', 'development time', 'training period', and so on; similarly, the term 'cost' can be further refined and also attached to product as well as process components. additionally, a further dimension 'domain' with an initially empty set of terms will help in getting together domain-specific extra-functional requirements. thus, the terms within this dimension will strongly depend on the particular development projects. the core facets selected in applying and adapting the faceted classification technique to extra-functional requirements are similar to those proposed in the goal question metric (gqm) approach (basili et al. 1994) which defines goals along the dimensions objects (products, processes, resources), viewpoints (actors), and issues. gqm starts with the definition of a goal which can be refined into several questions each of which can itself be refined into various metrics. gqm mainly concentrates on the refinement of (rather abstract) goals through questions leading to better quantifyable goals. faceted classification does not only establish different dimensions but also semantically refines them into terms according to specific system development needs. thus, each organization can establish its own classification schema. the actual classes of extra-functional requirements can then be assembled by selecting the most appropriate set of terms within the given facets. its extensibility and the possibility to tailor its contents according to organization and project-specific needs are also the main advantages of the generic classification schema. its multi-faceted nature allows for taking into account the multi-dimensional nature of extra-functional requirements leading to dynamically assembled classes. as the terms within each classification schema instance will apply to the whole organization, a common vocabulary (concerning development phases, product and document types, efr features) together with common development strategies will be valuable byproducts. process challenges in order to aim at acceptable approaches in dealing with extra-functional requirements, a better understanding of the challenges to be addressed during the software development process is necessary. for this purpose, let us first identify the problems which become evident when extra-functional requirements are involved in the software development process. • vagueness. especially in early-phase requirements engineering, extra-functional requirements tend to be fuzzy in their nature. they may be difficult to quantify and may lack in further details (e.g. product or process component to be addressed, reason, volatility). hence, it is inevitably difficult to express such requirements in an unambiguous, accurate and complete manner as should be striven for in order to be able to handle them properly. • functionality-bias. it is a common practice to use abstractions in terms of object types and classes, functions and states from the very beginning. these are certainly well-approved concepts enabling us to cope with (functional) complexity, but are biased towards the functionality dimension. this leads to a functionality-minded development process where extra-functional requirements are degraded to second class citizens which will have no chance to become more than just some attachments to the used abstractions. this might be feasible for some categories of extra-functional requirements like time constraints, but other (multi-faceted) categories of extra-functional requirements such as usability criteria cannot be sufficiently addressed with this strategy. • contradictions. as already shown by the examples of section 2.1., tight interweaved modelling of functional and extra-functional requirements may lead to severe problems in development and maintenance processes. we have to cope with the contradictions which may arise between functional and extrafunctional requirements. on the other hand, we have to take into account that the compliance of functional as well as extra-functional requirements will manifest in the final software product. although a distinction between functional and extra-functional requirements will be feasible especially in early phases of software development, extra-functional requirements, however, will be accomplished in later phases by getting incorporated into 'functional' components of the product to be developed (be it in terms of decisions on the overall architecture or during actual implementations). the moment and manner of this incorporation is the crucial issue in the integration of extra-functional requirements. 104 the above problems lead us to the following aims which we should try to achieve concerning extra-functional requirements: • the 'function-biased' focus during the software development process has to be extended to a wider context which also covers extra-functional requirements. • too tight interweavings between functional and extra-functional requirements during early phases have to be avoided. • the integration of extra-functional requirements must take place in a planned and controlled manner. potential issues to be addressed in achieving these aims regard capturing and modelling efrs as well as the integration of efrs in the software development process. these issues will be subsequently discussed. efr capturing processes, methods, techniques and tools currently used in practice did prove to be effective means for supporting the engineering tasks in software development. nevertheless, real-world experience confronts us with a substantial amount of failed projects, dissatisfied users and unused products (cf. loucopoulos & karakostas 1995), all factors which indicate that some essential requirements were not taken into account properly. some kinds of extra-functional requirements are fuzzy in their nature and are difficult to handle. this is certainly one reason why these requirements are likely to be neglected or simply forgotten. additionally, our functionality-biased understanding of software development is all but conducive to turn our attention towards extra-functional requirements. however, adding forgotten functionality to a final software product is far less a challenge than adding quality. hence, it is highly advisable to pay adequate attention to extra-functional requirements from the very beginning of the software development process. although requirements engineering tasks are already considered to be the most crucial activities in software engineering, even the relevance of requirements elicitation alone will increase because of the additional challenge to address extra-functional requirements explicitly. we all know that a question like "what extra-functional requirements are relevant for this project?" will in most cases be extremely inadequate for getting any useful answer from a stakeholder. it rather requires a systematic efr-specific preparation of the elicitation process. it certainly cannot be denied, that extra-functional requirements are highly dependent from the domain which is relevant for the software product. the identification of efr categories which are of particular interest from the domain point of view will help in collecting elicitation entries for initial questions referring to these categories. these questions will be at a higher level of abstraction though, but are already touching upon potential efr clusters which might turn out to be worth looking at in detail during elicitation. the usage of the faceted classification schema as introduced in section 2.2. throughout many different development projects will help in finding domain-relevant efr categories by experience. once the relationships between domains and efr categories are identified, templates for each efr category consisting of category-specific questions will provide a more detailed starting point for efr-specific elicitation activities. efr modelling as already mentioned, currently used modelling techniques have realization-focussed abstractions (like types, classes, functions, states) as basic concepts. they deal with structures and behaviors. complexity can be handled through compositions and different levels of detail. each modelling technique has its conceptual view, be it structural or behavioral. different modelling techniques can be used in an additive manner by allowing for multiparadigm connectors in terms of particular concepts (e.g. objects as common concept in static and dynamic modelling). while functional requirements are dealt with by using abstractions, extra-functional requirements can be (at best) expressed as values attached to process and product attributes (e.g. time constraints). some categories of extra-functional requirements cannot even be quantified because of their fuzzy and inaccurate nature (e.g. usability); they need additional refinements. the goal question metric (gqm) model (basili et al. 1994; van latum et al. 1998) is an example for a method which supports the refinement of (rather abstract) goals by means of questions leading to better quantifyable goals and associated metrics for measurement. the huge variety and diversity of different categories of extra-functional requirements makes it infeasible to rely on a special single modelling technique which is capable of handling functional and extra-functional requirements together. trying to extend existing methods used for functional requirements modelling with components for expressing all the different categories of extra-functional requirements would soon overload these methods. as discussed in section 2.1., too tight interweavings of extra-functional with functional 105 requirements should be avoided in favor of readability, flexibility and maintainability. so far, only few approaches exist which at least enable modelling functionalities together with one isolated dimension of extrafunctional requirements, e.g. only performance (opdahl 1994) or only security (dobrovnik & hochmiiller 1998). one approach which particularly focusses on modelling extra-functional requirements is the nfr framework (chung 1993; mylopoulos et al. 1992; chung et al. 1996). it uses goal graphs to represent satisficing decomposition and correlation structures of goals. the system quality model proposed by harrison et al. (1996) is based on the nfr framework. it refines the notion of correlation in 'supports' and 'promotes' relationships and introduces a template notation for recording possible kinds of quality relationships. one drawback of these methods is their non-applicability in case of complex efr interrelationships. the main issue in efr modelling is to find a balance between the necessity of the representation of relationships between efrs and to functional requirements and the applicability of methods extended with efrspecific constructs. efr integration process the most crucial challenge is the integration and accomplishment of the proper extra-functional requirements in the appropriate software development phases. regarding the rumors about the (non-)existence of extrafunctional requirements it is no wonder that this problem has not been very extensively tackled by the scientific community so far. the rest of this paper is dedicated to this particular challenge. for the purpose of efr integration, two different strategies would be conceivable: a revolutionary or an evolutionary approach. • the revolutionary approach would require to throw overboard our traditional software development strategies. we should have to explicitly avoid the above-mentioned functionality-bias by starting with (functional) abstractions too early in the software life cycle. new methods would be required which at least handle both, functionalities and extra-functionalities, in a like manner. these methods are likely to turn out to be incompatible to each of the currently applied methods. as a consequence, our overall way of thinking would have to be changed radically. • the evolutionary approach would take into account existing development strategies including our preferences regarding abstractions and functionalities. it would accept one driving development process which can be an instance of the currently applied phase-oriented process models. this setting would allow for a smooth integration of extra-functionalities. the main aim of such an approach would be the integration of extra-functional requirements in the right phases. continuous verification and validation would have to take place in all phases of the development process in order to detect potential inconsistencies (which are most likely to arise and have grave impact on the whole project with extrafunctionalities rather than in case of mere functionalities) as early as possible. the advantages of a smooth efr integration and the possibility to mainly adhere to classical, already approved and repeatedly used development strategies rather call for an evolutionary approach. the concept for such an approach will be outlined in the next section. efrs in the software development process an integrated view in the sequel, the role of extra-functional requirements within the software development process will be discussed. initially, the different efr categories will be put into relation to the functional dimension of software systems. then, the different kinds of relationships between these dimensions will be looked at and a general schema for the evolution of extra-functional requirements within the software life cycle will be presented. after outlining the effect of extra-functional requirements on the potential solution space for a software product, the evolutionary process of efr integration will be introduced by using the cylinder-to-stalactite metaphor. a different view of viewpoints a software system usually will be a product which is initiated, developed and used by different persons with different concerns. there are not only different perspectives on a definition or characterization of the notion of viewpoints (cf. leite 1996), but also various approaches tackling the problems of view(point)s in requirements engineering (e.g. kotonya & sommerville 1996) and their integration (cf. nuseibeh et al. 1994). although some of these approaches already address the integration of viewpoints across different modelling techniques, 106 common to them is again their functionality focus because of the concentration on structural and/or behavioral view integration. regarding our distinction in dimensions of concern, the traditional view of viewpoints which is limited to only one particular category of viewpoints has to be extended to cover also extra-functional dimensions as viewpoint categories on their own. this would lead to a more complete picture which is given in fig. 3. classical viewpoint integration mainly takes place within just one viewpoint category. a comprehensive integration approach, however, should also bear in mind this variety of viewpoint categories. a fig. 3. viewpoint categories efr relationships and dynamics extra-functional requirements cannot be regarded in isolation. they may constrain the software development process or directly address the software product or its components. nevertheless, we should be aware of the fact that process constraining requirements (e.g. documentation requirements, given programming language, standards) will finally also manifest themselves in the resulting product. in general, efrs are constraining the functional solution space. they may constrain particular functional requirements, compositional parts of the system, the system as a whole or a special contextually coherent part of the system which cannot be directly mapped to a component within the functional scope. the different categories of extra-functional requirements can neither be regarded in isolation. the fulfillment of some requirements of one efr category may support or hinder the fulfillment of requirements of another one. beyond that, situations are possible where single efrs are not influencing each other, but particular combinations thereof may have different impacts. as extra-functional requirements have their effects throughout the software development process, conflicts between extra-functional requirements may arise dynamically in all phases of development. this means that some requirements engineering activities (e.g. analysis, negotiations, conflict resolution) which usually are carried out in early phases will have to be effected also during later phases (on demand). let us consider a simple example. a flexible and maintainable information system should be developed to be used by various persons of different security levels. requirements engineering activities include the elicitation of static and dynamic requirements and associated security constraints. the integration of different information requirements result in a conceptual model as structural basis for further design steps. certain design decisions are required to tackle the security issues. for reasons of better flexibility and maintainability, a strict boundary between the pure semantic parts of the conceptual model and the security-relevant design elements is striven for. this could lead to the decision in favor of a three layered architecture in the sense of ansi/spare using views as security mechanism. this decision may have in turn impacts on other categories of efrs, e.g. performance problems might arise during the physical design phase. here we can see, that the accomplishment of a combination of extra-functional requirements of different categories (flexibility, maintainability, security) could cause contradictions to other efr categories. these contradictions may become evident even only in subsequent development phases. sometimes, even extra-functional requirements of a new category (e.g. performance requirements) arise and become relevant because of particular results of design decisions in accomplishing existing ones. another important aspect is the dichotomy on the border between functional requirements as elicited and analysed during requirements engineering activities and the functional design elements satisficing these requirements. usually, a one-to-one mapping will not be possible. therefore, a mere attachment of efrs to functional requirements is not sufficient. 107 in general, it is quite reasonable to make a distinction between two different types of extra-functional requirements with respect to their expressiveness: • crisp efrs: this type of efrs comprises requirements which can be accurately stated (quantified) as well as checked for accomplishment (measured). the term 'crisp' was chosen to emphasize a certain degree of precision in the formulation of efrs. as an example for an efr category which is likely to consist of crisp efrs, we can assume any kind of time constraints. these constraints will usually concern particular functions and will be stated in terms of time limits. • vague efrs: especially when extra-functional requirements start to evolve, they might not be tangible items from the very beginning. often, they will take shape in a later phase as they may also depend on other decisions made during the development process. examples for categories of vague efrs are especially those without directly applicable metrics, like usability, flexibility, maintainability, and so on. often, the refinement process will require to use proxies (e.g. 'easy to learn', 'easy to use' for usability) and associated metrics (e.g. average training period, average amount of help requests) which together serve as a substitute and approximation of the original vague efr (wieringa 1996). vague efrs are most difficult to handle. because of their fuzzy nature and our restricted possibilities in grasping them properly, they often tend to be neglected. additionally, if the refinement process from rather vague to more crisp efrs was successful, the resulting crisp efrs might substantially diminish the degrees of freedom for the ensuing software development process. the evolution of extra-functional requirements within the software development process can be described as follows. efrs can be stated in terms of vague or crisp efrs. vague efrs have to be refined to result in crisp efrs. the refinement process to be applied depends on the actual efr category and the initial efr statement. possible means for refinement are additional efr-specific interviews (e.g. using proxies) or gqm. the decision about the achievement of an acceptable degree of detail is also situation-dependent. the fulfillment of the refined efrs and the compliance check take place during the rest of the software development process. at any development step before the actual delivery of the software system, it may become obvious that a particular efr (which already might be fulfilled so far) cannot be met any more. this would require to trigger a conflict resolution process involving certain negotiations. three kinds of results of this process are possible: • the efr is considered to be still important as it is. adaptation activities will only concern other (conflicting) requirements. • the efr is still important but has to be changed because of a compromise. adaptations to the requirement specification and possibly to already existing software components will be necessary. • the efr is abandoned because of other requirements which are more important. cancellation activities might be necessary. the state transition diagram of fig. 4 illustrates these efr dynamics. this representation slightly deviates from the notation as used in omt (rumbaugh et al. 1991). while an arrow from a composite state indicates that the source state can be any of the substates, an arrow to a composite state should indicate that the target state can also be any of the substates (and not a certain initial state), irrespectively of the original source state. valid statement/ elicitauon staled. [complin}! [ok]/adaptation system delivery ® fig. 4. efr dynamics and evolution 108 fitness for activation vs. fitness for use concerning the result of a software development process the software product we can assess its quality depending on the spectrum of requirements met by this product. • fitness for activation. a software product which satisfies only functional requirements can be considered as a system fit for activation. it does what it should do in terms of input/output relationships as governed by pure functionality. the maturity of systems fit for activation rather refers to functioning prototypes than to systems of some quality. • fitness for use. a software product which satisfies functional as well as extra-functional requirements is a system fit for use. it does not only do was it should do with respect to input/output relationships but also behaves according to the constraints specified by extra-functional requirements. the notion of fitness for use should not be interpreted in the narrow sense of usability. it rather refers to overall system quality features (cf. potts 1997(b)). the solution space for systems fit for activation allows more degrees of freedom in software development. therefore, it will be larger than the solution space for systems fit for use. hence, a system fit for use is certainly also a system fit for activation but not necessarily vice versa. a graphical representation of the relationship between both basic solution spaces is given in fig. 5a. leaving it at just the two archetypes of systems would, however, oversimplify the complete picture. as already mentioned, extra-functional requirements may also contradict each other. this would mean that there is no 'ideal' solution space for a system fit for use. such possible trade-offs between extra-functional requirements have to be managed with adequate negotiations and decision processes. the decisions about preferences and compromise solutions will actually set up an 'agreed' solution space in which a system fit for use should be placed. fig. 5b illustrates such a situation with three constraint spaces (a, b, c) restricting the original functional solution space. here, the intersection between a and b was selected to be the agreed solution space. systems fit for activation systems fit for use apreed solution fig. 5. solution spaces the cylinder-to-stalactite metaphor in the sequel, the metaphor of a cylinder which is tailored to a stalactite (fig. 6) is used to explain the evolution which takes place during the process of software development. aiming at an evolutionary approach, we assume a phase-driven process followed for software development which is initially symbolized by an imaginary cylinder. the initial circular base of the cylinder represents the initial solution space which is headed for. the time dimension is partitioned roughly into basic process phases. 109 requirements analysis design implementation fig. 6. cylinder-to-stalactite metaphor during the phase of requirements analysis, the activities of requirements elicitation, specification, and validation are carried out. the result of this phase is the functional solution space (for systems fit for activation) which embeds an initial solution space constrained by already identified extra-functional requirements. but the influence of extra-functional requirements does not stop at the end of the requirements analysis phase. moreover, requirements of different efr categories have their effects during the whole driving development process (symbolized in fig. 6 by little arrows from outside). within the metaphor, we assume that extrafunctional requirements restrict the solution space by cutting off pieces from the cylinder. while the shape of the resulting stalactite characterizes the evolution of the solution space within the development period, decisions made during the development process can be observed as directed arrows within the stalactite. at the end of the software development process, the final software product is symbolized by the tip of the stalactite. the cylinder-to-stalactite metaphor allows us to observe and describe three phenomena: • the solution space may be partitioned into more than just one part. this can be symbolized by divided stalactites. it requires for decisions choosing the best fitting stalactite to follow. • overwhelming efrs can restrict the solution space in a radical manner such that the tip of the stalactite gets cut off before it has a chance to touch the base of the (initial) cylinder. this would mean that there is no ideal solution possible which meets all the requirements. an immediate conflict resolution is required. • at any time in the software development process, too big tensions between competing efrs can lead to negotiations and conflict resolutions. in case of resulting compromises, it might be possible that the already restricted solution space will be expanded again to some extent. sticking already removed parts back onto the stalactite can symbolize this situation. at any time within the development process, a sectional view of the stalactite allows for an assessment of the development process. the constrained solution space can be anticipated giving the range in which the final product is likely to be situated. the shape of the stalactite mirrors the sequence of influences originating from extra-functional requirements during the software development process. efr integration framework it is rather impossible to think of just one single, fully fledged model which fits perfectly for all software processes. however, many of the frequently used process models are phase-driven. the following ideas on efr integration aim at the efr-specific extension of such phase models and are therefore largely independent of the particular activities within the various phases. a kind of meta framework is proposed to enable the smooth integration of extra-functional requirements into software development processes. this framework concentrates on the fact that extra-functional requirements are eminent throughout a software development process and need continuous attention because of their ongoing influence on the whole process. principally, we can distinguish between two classes of efr-related activities within the software development process: 110 • requirements engineering activities which should also deal with the elicitation, analysis and validation of extra-functional requirements • further software development activities which are influenced by efrs and which may in turn affect the degree of efr compliance according to this distinction, the efr integration framework provides efr-specific hints for early-phase requirements activities but also suggests a reference model for continuously keeping track of extra-functional requirements in later phases of development. efr-specific requirements engineering in the types of actual activities to be carried out during requirements engineering, no distinction is necessary between kinds of requirements. project-relevant requirements be they functional or extra-functional have to be elicited, analysed and validated. as already mentioned, our functionality-biased mind as well as our greed for abstractions soon will put our emphasis on functional requirements leading to behavioral and structural requirements models. being thoroughly convinced, however, that extra-functional requirements are at least as important as functional ones and that these requirements have to be treated properly too, is already a fertile ground for openness regarding additional effort for efr-specific analysis activities. the early identification of the set of extrafunctional requirements constraining the potential solution space for the software product is certainly the most crucial challenge. the following suggestions for identifying extra-functional requirements will not address particular efr categories nor give any advice in efr specification. the purpose is just to give some general practical hints for their explicit consideration. a first information about extra-functional requirement categories which might be of importance for the software product can be obtained from analysing the problem domain. some of these domain-relevant efr categories may be conflicting giving an initial conflict set at category level. elicitation of project-specific priorities regarding these categories will result in additional information about the potential conflict proportions. the actual instances of project-relevant efr categories have to be identified during the elicitation process in terms of vague and crisp extra-functional requirements (cf. section 4.2.). vague requirements need further refinement (as described in section 4.2.) as soon as possible as it is easier to identify potential conflicts between crisp requirements than in cases when vague requirements are involved. upon identification of conflicting extra-functional requirements, negotiations become inevitable. all decisions resulting from these negotiations must be made evident and documented to be able to be referenced later on. the additional role of an efr advocate (hochmiiller 1997(b)) can help in explicitly focussing on extrafunctional requirements during requirements engineering and the rest of the software development process. question plan action for a smooth integration of extra-functional requirements into subsequent development phases, we propose the question plan action (qpa) approach. qpa does not explicitly address special categories of extra-functional requirements neither, but rather enables us to identify milestones where extra-functional requirements particularly need our consideration. each phase-oriented software development process can be extended by qpa. for such an extension, the underlying process model is assumed to dispose of the following granules: • process phase • development step within a phase • decision making process a process phase is a period of time in which a particular kind of development task is fulfilled. it has a certain entry point at which the phase begins and a certain exit point which means that all the activities foreseen for the phase were carried out. the elementary activities within a phase correspond to so-called development steps (fig. 7). decisions are particular development steps which influence subsequent activities of the same or later phases. i l l development step i i j phase j ( development step j2 j development step 13 j fig. 7. process granules the structure of the qpa approach can be outlined as follows: • question (q): special questions referring to efrs serve as connection points between the driving software development process and extra-functional requirements. their purpose is the identification of particular extra-functional requirements which are of importance for the actual process granule. • plan (p): depending on the answers to the questions, the appropriate actions regarding the involved extrafunctional requirements have to be planned and scheduled. a plan can also define a new project-dependent question which can be scheduled to be asked at a later stage of development either within the same granule or within a subsequent one. • action (a): the actions actually carry out the planned efr integration. these actions will take place in terms of development steps. in general, an answer to a question will be immediately followed by the definition of a plan. the action carrying out the plan usually will be executed at a later time. there are two kinds of questions: • predefined meta questions: these questions are general enough to be applicable in any project regardless of the actual development project. the rest of this section is dedicated to give an overview of these questions which are stated in analogy with the evolution of efrs (cf. sec. 4.2.). • project-specific questions: in addition to the rather generally stated meta questions, the qpa approach also allows for questions which can be defined on demand. these questions will be formulated specific to the actual project. qpa for process phases regarding a phase as elementary unit in software development, there are two possible points of connection for qpa: the entry and the exit of a phase. the meta questions to be answered at the beginning and the end of a phase refer to already identified extrafunctional requirements and their different states (cf. sec. 4.2.). especially in case of process phases, actions are of pure operative nature. hence, the following qpa description will be confined to the identification of connection points together with the relevant meta questions and the plans to be developed. • on entry the following meta questions are proposed at the very beginning of each phase. for each question, a rough sketch about the contents of the associated plan is outlined. qen,i: what efrs have to be directly obeyed in this phase? pen,i: plan for continuous observation of phase relevant efrs for each identified efr: when has it to be obeyed? who has to obey it? how should it be obeyed? 112 qent2: what efrs have to be fulfilled in this phase? pent2: plan for efr fulfillment for each identified efr: in which development step(s) should it be fulfilled? when should the fulfillment be checked (earliest, latest date)? what efrs have to be checked in this phase for compliance? pent3: plan for efr compliance check for each identified efr: in which development step should the compliance be checked? who is responsible for the check? • on exit after having carried out the activities of one development phase but before passing over to the next phase, the following questions are relevant: qcxl: what efrs of the set identified in qeml have not been obeyed in this phase? pexi : plan for assessment of effects of neglecting these efrs for each identified efr: why have they not been obeyed? which effects (risks) could/did arise? decision about further steps. qex2: what efrs of the set identified in qent2 have not been fulfilled in this phase? pex2: plan for assessment of effects of not fulfilling these efrs for each identified efr: which effects (risks) could/did arise? decision about further steps; eventually immediate fulfillment. qm3: what efrs of the set identified in qent3 did fail during check? pex3: plan for assessment of failed check for each identified efr: which effects (risks) could/did arise? decision about further steps; eventually immediate fulfillment and repeated check. qpa for development steps within each development step, actual efr-related actions have to be planned and carried out. the difference to the qpa elements for process phases lies in the granularity of the plans leading to actual actions. qdevp which efrs have to be addressed in the step? pdevl : plan actual treatment (fulfillment, continuous observation) which efrs are influenced by the actions in the development step? pdev2: plan the assessment of how these efrs are influenced qpa for decisions a decision making process involves decisions which might be influenced by efrs or which in turn may influence certain efrs. as decision making processes usually do not last a very long time, actions are likely to follow immediately upon answering the questions. qdeci: which efrs influence the decision? continuously observe these efrs during decision making process which efrs are influenced by the decision in what aspect? a<iec2: analyse and evaluate these influences. which conflicts become evident during decision making process? adec3: in case of conflicts: 113 negotiate resolve conflicts (within actual decision making process) document the decision making process (and the changed/new efrs) document the results representation considerations as the question plan action approach inherently influences and requires the coordination of efr-relevant activities throughout the whole software development process, appropriate support in the administration of those activities together with the necessary qpa steps is desired. such a support should provide predefined meta questions and help in stating project-specific questions on demand as well as in expressing plans. generally, it should be possible to establish links between corresponding qpa elements as well as between software development granules and qpa elements. in addition to the assistance in integrating qpa steps into the software development process, also the administration and representation of extra-functional requirements, their relationships and interdependencies need appropriate support. usually, the multi-dimensional structure of all these relationships will be rather complex and difficult to survey. hence, any attempt to get along with just one universal graphical representation keeping track of all the details is likely to fail. similarly, it is useless to apply just a linear textual description. hypertext concepts with different types of links between different types of nodes (including graphics) will certainly better meet the representation requirements outlined above. the benefits of semiformal hypertext as means to support requirements engineering were already demonstrated (e.g. reth (kaindl 1993), hydra (pohl & haumer 1995)). for our purposes, hypertext will not only be utilized to structure requirements information during requirements engineering. it likewise will serve as a mechanism to relate this information to software development activities throughout the development process. this enables the navigation between process elements and respective efr information. while the hypertext paradigm is suitable as a presentation mechanism, the amount of data together with multiple concurrent access and update operations call for a consistent and powerful database for efr information management. for example, the efr information base (hochmiiller 1997(b)) helps in keeping track of extrafunctional requirements, their origins, relevance, (inter)relationships and compliances. it can provide the data required for qpa and can also be used for efr-related documentation purposes. usefulness of the efr integration framework the main goal of the efr integration framework is its applicability in a wide variety of traditional phase-driven development processes. we explicitly waived the introduction of any new formalisms in favor of the emphasis on crucial facts and activities dealing with efrs. so, software development units (teams, enterprises) can decide on their own how much effort they are willing to pay for the treatment of actual efrs. the effort will depend on the perceived risks emerging from particular efrs. its intensity can range from developing and using simple checklists to the development of own methods for dealing with particular efr categories (e.g. simulation in case of performance requirements) together with the reuse of project experiences for continuous refinement and adaptation of the integration framework (e.g. additional domain-specific questions for future project within the same problem domain). tool support should be taken into consideration even in case of rather simple methods (like checklists) since they will enhance productivity of and overall method acceptance by project members as well as improve the development process by better traceability and documentation of efr-related activities. related work comparison and synergies the presence of the computer science community's disagreements regarding the existence and taxonomy of extra-functional requirements is already an indicator for the lack of substantial relevant research results. however, there are few approaches which also tackle the issue of integrating extra-functional requirements into the software development process. the approach by dobson & mcdermid (1991) proposes two process-related reference models which describe the different facets of the overall software development process. one of them assumes a canonical development stage and outlines the activities of analysis, evaluation and synthesis and their input-output relationships to policies and external constraints. the interrelationship to any conventional phase-based model, however, is not explicitly addressed at all. the other reference model of dobson & mcdermid (1991) considers the different facets of a specification (in terms of the 4 roles of (extra-functional) requirements) within the development process and how they relate at different life cycle stages. although a distinction between extra-functional and functional requirements is avoided, its main problem the potentially complex interplay between both types of requirements leading to that rather unsatisfactory dichotomy remains which becomes evident in the attempt to 114 find the appropriate categorization for classic efrs in terms of commitments or obligations. the main drawback of this approach is the missing managerial aspect of efr integration. the goal-oriented nfr framework (chung 1993; mylopoulos et al. 1992; chung et al. 1996; nixon 1993) has its origin in the context of information systems design. it deals with the decomposition of goals but does not address the integration of extra-functional requirements into the overall software development process. however, it serves as basis for another process-oriented approach dealing with extra-functional requirements integration. mike (model-based incremental knowledge engineering) is a framework for the development of knowledge-based systems which explicitly treats extra-functional requirements within its proposed process model (laudes & studer 1995). extra-functional requirements are described semiformally as nfr context in a hypermedia-based fashion. the impact of extra-functional requirements on decisions taken in the design phase is explicitly described in a particular process model. while the nfr framework mainly deals with the decomposition of goals and stops at a higher level of granularity, mike also handles elementary design decisions. the qpa framework tries to glue extra-functional requirements on a given phase-driven software development process. it is independent of any project domain and the actual activities carried out within the process. nevertheless, the phase dealing with requirements engineering will include special activities for efr capture and modelling. the related tasks can be fulfilled by applying existing approaches. methods like gqm (basili et al. 1994; van latum et al. 1998) or other quality measures (gillies 1992) can be used to identify and quantify high-level efrs (goals) and to proceed from vague to crisp efrs. synergies can also be achieved by applying organization-specific requirements management with particular focus on extra-functional requirements. an efr information base (cf. section 5.3.) which is used throughout the software development process will additionally support traceability of efrs back to their origins and rationales and forward to their compliances. conclusion based on two rather simple examples in modelling synchronization constraints in oocp and security constraints in oodbmss, the existence of an 'extra' dimension of requirements was demonstrated and the problems connected with tight interweaved treatment of functional and extra-functional requirements were pointed out. beside the issues of capturing and modelling extra-functional requirements, the integration of this kind of requirements into the software development process was identified as the most crucial challenge. systems which should be fit for use call for such an integration. this paper presented an evolutionary approach for the integration of extra-functional requirements in an overall phase-driven development process. this efr integration framework is universal and flexible in the sense, that it can supplement a wide range of phase models currently in use. additional benefits can be achieved by synergies from combining this framework with other approaches (like the management of extra-functional requirements) or techniques (e.g. hypertext representation of multidimensional efr structures and process documentation for better traceability of negotiations and decisions). references basili vr, caldiera g., rombach hd. goal question metric approach. in: marciniak jj (ed). encyclopedia of software, vol 1. 1994. pp 528-532 bertino e, jajodias. modeling multilevel entities using single level objects. in: ceri s, tanaka k, tsur s (eds). deductive and object-oriented databases (dood'93). springer. lncs 760. pp 415-428 bertolino a. achieving quality in software. journal on systems and software 1994; 26(1): p 13 boehm bw. software engineering. ieee transactions on computers 1976; 25(12): pp 1226-1241 boehm bw, in h. identifying quality requirements conflicts. ieee software 1996; 13(2): pp 25-35 chandrasekaran b, goel ak, iwasaki y. functional representation as design rationale. ieee computer 1993; 26(1): pp 48-56 chen p. the entity-relationship model: toward a unified view of data. acm transactions on database systems, 1976; 1(1): pp 9-36 chung l. dealing with security requirements during the development of information systems. proceedings of the fifth international conference, caise'93, paris, 8-11 june 1993. springer-verlag, berlin, 1993. pp 234-251 chung l, nixon ba, yu e. dealing with change: an approach using non-functional requirements. requirements engineering journal. 1996; 1(4): pp 238-260 coad p, yourdon e. ooa object-oriented analysis. prentice-hall, englewoods cliffs, 1991 davis am. software requirements: objects, functions, and states. 2nd edn. prentice-hall, englewoods cliffs, 1993 115 dobrovnik m, hochmiiller e. views as security mechanism in object-oriented database systems. proceedings of the international workshop on issues and applications of database technology (iadt'98), july 1998 dobson je, mcdermid ja. an investigation into modelling and categorisation of non-functional requirements (for the specification of surface naval command systems). tr 320, computing laboratory, university of newcastle upon tyne, november 1991 dubois e, opdahl al, pohl k. refsq'97 workshop summary, proceedings of the third international workshop on requirements engineering: foundations of software quality (refsq'97), barcelona, spain, 16-17 june 1997. presses universitaires de namur, pp 1-12 gillies ac. software quality theory and management. chapman & hall, london, 1992 harrison r, pratten ch, sivess v. quality decomposition for large-scale systems". technical report, department of electronics and computer science, university of southampton, may 1996 hochmiiller e. requirements classification as a first step to grasp quality requirements. proceedings of the third international workshop on requirements engineering: foundations of software quality (refsq'97), barcelona, spain, 16-17 june 1997(a). presses universitaires de namur, pp 133-144 hochmiiller e. quality improvement through quality requirements management. proceedings of the international conference on software quality (icsq'97), maribor, november 1997(b), pp 71-78 hochmiiller e. inheritance contradictions between functional and extra-functional requirements. journal of integrated design and process science. sdps. june 1998; 2(2): pp 22-31 ieee recommended practice for software requirements specifications. ansi/ieee std. 830-1993. ieee, new york, 1993 iso/iec. international standard iso/iec 9126. information technology software product evaluation quality characteristics and guidelines for their use. geneva, 1991 jackson m. system development prentice-hall, 1983 jajodia s, kogan b, sandhu rs. a multilevel secure object-oriented data model. in: abrams md, jajodia s, podell h.j (eds). information security an integrated collection of essays. ieee-csp. 1995. pp 596-616 kaindl h. the missing link in requirements engineering. acm software engineering notes 1993; 18(2): pp 30-39 kaindl h. an integration of scenarios with their purposes in task modeling. proceedings of the acm symposium on designing interactive systems (dis'95), ann arbor, mi, august 1995, pp 227-235 keller se, kahn lg, panara rb. specifying software quality requirements with metrics. in: thayer rh, dorfman m (eds). system and software requirements engineering. ieee-csp tutorial. 1990 pp kotonya g, sommerville i. requirements engineering with viewpoints. software engineering journal 1996; 11(1): pp 5-18 kramer j, wolf al. succeedings of the 8th international workshop on software specification and design. acm software engineering notes 1996; 21(5): pp 21-35 laudes d, studer r. the treatment of non-functional requirements in mike. proceedings of the 5th european software engineering conference (esec'95), sitges, september 1995, pp 294-306 van latum f, van solingen r, oivo m, hoisl b, rombach d, ruhe g. adopting gqm-based measurement in an industrial environment. ieee software 1998; 15(1): pp 78-86 leite jc. viewpoints on viewpoints. proceedings of the international workshop on multiple perspectives in software development (viewpoints 96), san francisco, october 1996 loucopoulos p, karakostas v. system requirements engineering. mcgrawhill, new york, 1995 matsuoka s, yonezawa a. analysis of inheritance anomaly in object-oriented concurrent programming languages. in: agha g, wegner p, yonezawa a (eds). research directions in concurrent object-oriented programming. mit press, 1993, pp. 107-150 mittermeir rt. requirements engineering. in: kurbel k, strunz h. handbuch wirtschaftsinformatik. poeschel verlag, 1990 mittermeir rt. dimensions of software design from algorithms to systems. proceedings of the second world conference on integrated design & process technology (idpt'96), vol 1, sdps, austin, december 1996, pp 82-89 mylopoulos j, chung l, nixon b. representing and using non-functional reqirements: a process-oriented approach. ieee transactions on software engineering 1992; 18(6): pp 483-497 nixon ba. dealing with performance requirements during the development of information systems. proceedings of the ieee international symposium on requirements engineering (re'93), san diego, 4-6 january 1993, pp 42-49 nuseibeh b, kramer j, finkelstein a. a framework for expressing the relationships between multiple views in requirements specification. ieee transactions on software engineering 1994; 20(10): pp 760-773 opdahl a. requirements engineering for software performance. proceedings of the first international workshop on requirements engineering: foundations of software quality (refsq'94), utrecht, netherland, 6-7 june 1994, pp 16-32 116 pohl k, haumer p. hydra: a hypertext model for structuring informal requirements representations. proceedings of the second international workshop on requirements engineering: foundations of software quality (refsq'95), jyvaskyla, finland, 12-13 june 1995, pp 118-134 potts c. requirements models in context. proceedings of the third ieee international symposium on requirements engineering (re'97), annapolis, maryland, 6-10 january 1997(a), pp 102-104 potts c. fitness for use: the system quality that matters most. proceedings of the third international workshop on requirements engineering: foundations of software quality (refsq'97), barcelona, spain, 16-17 june 1997(b). presses universitaires de namur, pp 15-27 prieto-diaz r. implementing faceted classification for software reuse. communications of the acm 1991; 34(5): pp 88-97 rai a, song h, troutt m. software quality assurance: an analytical survey and research prioritization journal on systems and software 1998; 40(1): pp 67-83 roman gc. a taxonomy of current issues in requirements engineering, ieee computer 1985; 18(4): pp 14-23 ross dt, schoman ke. structured analysis for requirements definition. ieee transactions on software engineering 1977; 3(1): pp 1-65 rumbaugh j, blaha m, premerlani w, eddy f, lorensen w. object-oriented modelling and design. prenticehall, englewoods cliffs, 1991 rumbaugh j, jacobson i, booch g. the unified modeling language reference manual.addison-wesley, 1998 shaw m. truth vs knowledge: the difference between what a component does and what we know it does. proceedings of the eighth international workshop on software specification and design (iwssd-8), schloss velen, germany, march 1996, pp 181-185 sommerville i. software engineering. addison-wesley, 1992 starke g. session summary: non-functional requirements. proceedings of the first international workshop on requirements engineering: foundations of software quality (refsq'94), utrecht, netherland, 6-7 june 1994,pp4-6 wieringa rj. requirements engineering frameworks for understanding. john wiley & sons, chichester, 1996 yourdon e, constantine ll. structured design. prentice-hall, 1979 yourdon e. modern structured analysis. prentice-hall, 1989 117 a virtual organisation model for e-government janice bum and greg robins edith cowan university perth, w.a. australia e-mail: i.bum@ecu.edu.au abstract this paper looks at the implementation of a new customer value alliance model in e-govemment. firstly we review the issues of e-govemment and the drive towards customer centric organisations in the context of multiple government agencies. a model of e-government is introduced and examined within the context of a virtual organisation model which can be applied along the customer value chain across multiple service agencies. a case study is used to demonstrate how this concept of a virtual organisation as a customer value-alliance model can effect a successful transition to e-govemment from a traditional government model. finally, we examine how the aboriginal affairs department, a western australian government agency is implementing this model for improved customer service and the implications of this model for the management of change in a developing e-community. introduction it is now well accepted that with the emergence of the world wide web, a totally new business environment, ebusiness, has evolved where companies must work together to create online networks of customers, suppliers and value-added processes (ticoll, lowry and kalakota, 1998; el sawy et al. 1999). such radical transformations are not solely restricted to the business world but are now impacting on the public sector where the latest "e'volution is e-government (stratford and stratford, 2000). within the next five years the internet will transform not only the way in which most public services are delivered but also the fundamental relationship between government and citizen (von hoffman, 1999, sprecher, 2000, zulfikar et al, 2001). with few exceptions, however, governments have arrived late on the scene. as monopoly suppliers, none were worried about being "amazoned" by a new web-based competitor. transactions with government are rarely a matter of choice and government employees are unlikely to be rewarded for devising innovative web based strategies to replace them in their jobs. nevertheless the drive is now on for radical government change (morin, 1999; sprecher, 2000). a major driver has been the desire to reduce costs and make revenues go further. savings of 20% are not unusual in the e-business community as they network their supply chains (burn and hackney, 2000), and such concepts transfer easily into government services. u.s. federal, state and local procurement spending on materials and services in 2000 was estimated at around $550 billion, and in the european union member states' combined procurement spending was around $778 billion (symonds, 2000). with a 20% cut in costs we are looking at savings of around $250 billion. an additional driver comes from customer expectations. customers now have far greater access to information and demand personalised experiences as opposed to simply acquiring goods and services. a customer driven organisation is one that maintains a focus on the needs and expectations of customers both spoken and unspoken in the creation and/or improvement of the product or service provided. successful organisations, state or municipal governments and federal government departments and agencies have recognised that developing customer focus is an absolute necessity (cavanagh and livingston 1997; schoeniger, 2000). one of the proposed solutions has been the creation of government portals such as the singapore or uk portals. these have been designed around "life events" such as changes in marital status and allow users to find what they are looking for by using "how do i ?" type questions rather than by forcing the client to search through complex organisational structures possibly linking up to 50 different departments in one search. in reality the government portal acts as a virtual organisation front interacting with customer driven demand. this type of solution requires major changes within and without the government organisation and as yet, there is no clear evidence of success. (jellinek, 2000; traunmuller, 2001). the failure of a massive government it outsourcing project in australia in 2001, has highlighted the enormous difficulties of implementing cross-agency collaboration. the proposed solution is to return autonomy to the individual government agencies but to focus on the development of inter-agency systems based on customer services and added customer value. this paper looks at a specific e-government solution in the context of the west australian (w.a.) government. firstly we review the issues of customer focus and utilising external organisations in the context of government agencies. ostensibly government agencies are service driven organisations with a major goal of providing a service to the public. we then discuss how traditional government organisations are set up with a focus on w.a. and how the concept of a value alliance network can improve customer service. finally, we examine how the aboriginal affairs department, a w.a. government agency is implementing a value-alliance model as a virtual organisation and the implications of this model for the management of change. 104 developing a customer focus prahalad and ramaswamy (2000) suggest that organisations need to "create their future by harnessing competence in an enhanced network that includes customers". as a minimum they propose four fundamental realities which managers must come to grips with: • they have to engage their customers in an active, explicit, and ongoing dialogue; • they have to mobilise communities of customers; • they have to manage customer diversity; and • they have to engage customers in co-creating personalised experiences. they developed a three-stage model which we have adapted to a government context and summarised below in table 1. unit of analysis resources basis for access to competence added value of managers value creation sources of managerial tension the agency the government agency what is available within the agency internal agencyspecific processes nurture and build competencies autonomous service-unit autonomy vs leveraging core competencies network of agencies b2b the extended enterprise:the agency, its agency partners and other service providers access to other agencies' competencies and funding privileged access to agencies within the network manage collaborative partnerships collaborate with partner agencies partner is both collaborator and competitor for value enhanced value network e-government the value alliance:the agency, its partners, other funding and service providers and its customers access to other agencies' competencies and funding, as well as customers' competencies and investments of time/ effort infrastructure for active ongoing dialogue with diverse customers harness customer competence, manage personalised experiences, and shape customer expectations collaborate with partner agencies and with active customers customer is both collaborator and competitor for value table 1: developing model of e-government table 1 shows that the idea of extending the government services network and changing the nature of its usage to improve core competencies is a central component of this model. in the past, most government agencies had a traditional focus where they have embraced the concept of the extended enterprise but have been primarily concerned with alliances, networks, and collaborations among other agencies and services. the old idea of the "extended enterprise" should give way to the idea of an enhanced network of traditional agencies, other services, funding bodies and most importantly, customers. government managers need to recognise that consumers are a source of competencies. they must focus on developing relationships with the customer as the agent that is most dramatically transforming government as we know it and leading the e-government and governance revolution (stamoulisetal, 2001). traditional situation organisations, particularly government organisations are typically structured in a top down bureaucratic style, creating a barrier between the customer and the organisation and forcing customers to develop a knowledge of the structure to be able to seek services (barreyre, 1988). the western australian government has established 53 agencies to provide a variety of government services to the public. each agency reports to a cabinet minister, has a chief executive officer accountable for all aspects of the agency and a corporate executive team responsible for the operation of the divisions within the agency. each agency is charged with a specific function or service and has responsibility for setting policy in relation to their function, providing the minister with responses to correspondence and assisting the public. as an example the aboriginal affairs department has responsibility for assisting all aboriginal people within the state to access government services; ensuring the 105 welfare of aboriginal people, their culture and heritage and maintaining traditional aboriginal sites. the ministry of sport and recreation are responsible for increasing the participation of all west australians in sport, maintaining sporting venues throughout the state and assisting elite west australian athletes. the education department is responsible for the education of all primary and secondary aged children in the state. all western australian government agencies have had similar structures that comply to a traditional organisational structure and for the main part work in isolation from each other. both the education department and ministry of sport and recreation have an aboriginal affairs section which work in isolation of each other and the aboriginal affairs department. o oo people having to fit the services tttt* figure 1 traditional agency this approach forces the customers to fit or break into this structure if they require a service (figure 1). customers must seek out the area that deals with their particular requirement. they must move between many different agencies to gain access to all the services they need. as an example a company within western australia seeking to explore mineral deposits on aboriginal lands would need to seek approval and apply for appropriate permits through the department of minerals and energy, the department of land administration and the aboriginal affairs department. this requires the customer to discover which departments need to be approached, to approach each individually and to locate the appropriate section within each of these to obtain the correct advice. _ i **•»" 1 services fitting the people figure 2. new agency there is a need particularly within a service environment such as government agencies to move the customer into the centre and to offer a wide range of services across agencies (hopkins and jamil, 1997). this requires agencies to develop close working relationships and implement a structure based on the idea of collaboration. (figure 2). achieving a service driven environment many companies already focus on core value adding processes, working with external partners to jointly bring forward a service. these companies believe that a more flexible organisation built around a series of alliances and business relationships, is the most effective way to respond quickly and creatively to constantly changing 106 market conditions (miles and snow, 1995). the conventional, vertically integrated corporation may be too slow, or have too much retained infrastructure to allow it to compete with companies who can quickly put together a customised response to its clients (campbell and dinicola 1997). if government agencies are to provide a public service then they must embrace wholeheartedly the notion of the value alliance. the value alliance emphasises the decentralisation of control, the creation of more flexible patterns of working, a greater empowerment of the workforce and the customer, the displacement of hierarchy by teamwork, the development of a greater sense of collective responsibility and the creation of more collaborative relationships among co-workers and customers. to initiate such developments an agency needs to perform a full customer value chain analysis in order to set up a number of different agency alliances through an electronic network. this may form the basis for a one-stop portal where the alliance combines a range of services and facilities in one package forming one single customer supply chain. participants may come together on a project by project basis but generally the general contracting agency provides coordination. where longer term relationships have developed the value alliance often adopts the form of value constellations where agencies and funding services have multiple interactions and a complex and enduring communications structure is embedded within the alliance (burn and barnett, 2000). substitutability has traditionally been a function of efficiency and transaction costs: searching for, evaluating, and commencing operations with potential partners has been a costly and slow government procedure, relying as it does on information transfer, the establishment of trust and policy rules across states, time zones, culture, and legal frameworks. these have determined the relative positioning of partners on the chain and the reciprocity of the relationship. this value-alliance will be built around customer value chains and enable the sharing of resources, skills and knowledge to produce a 'best' customer solution and enable agencies to be more responsive to customer requirements and offer superior quality of service. each agency may be required to form several value-alliance virtual organisations depending on what has been identified as the requirements for one stop processes for clients. managing a virtual agency model in order to achieve a successful value-alliance it is essential that a business planning model is established that ensures each member agency has 'buy-in' to the desired outcomes. the first step to achieve 'buy-in' is to establish a high level committee comprising of the chief executive officers from each of the member agencies. the committee is charged with the responsibility of identifying the virtual organisation's goals. these goals must then be meshed into the individual agency's processes. any business planning must be built on services, delivery goals and objectives that focus on its customers through direct customer and front-line employee input. to achieve this there must be a fundamental shift in management and workforce thinking and practices that include: • pervasive knowledge sharing, feedback and communication; • integration of environmental considerations at the earliest stages of design; • effective partnerships with customers. • commitment to using customer feedback to drive changes in operations, goals and vision; and • frontline employees given the authority to deal with customer issues. it is essential that each agency is represented by the chief executive officer. commitment grows as employees understand what is being developed, this understanding is achieved through communication and commitment from the top. in order to achieve the most appropriate goals that focus on the customers' requirements and establish a one stop shop from the clients viewpoint a model must be established that passes information between all levels both within each agency and between the agencies. it is important to recognise the customers as integral members of the virtual organisation glassey, 2001). this model establishes information flows that: • ensure customers and front line staff can impact on the strategic planning process through passing information upwards; • agreed goals are passed to all levels of each agency; • these goals are articulated to clients; and • planning takes place across agencies at all levels a key to the success of an organisation is a network of open communication, a combination of sharing and listening flowing both horizontally and vertically through the organisation. management must share details with employees. a workforce that is involved is much more likely to 'buy in' to management's vision and work together for results. management must be able to combine the differences in diversity and organisation structure, in order to make the virtual organisation reach its target. virtual teamwork places a particular emphasis 107 on communication and the development of 'awareness' skills. it is critical that front-line employees have immediate access to current information. the key groups in this model are the strategic planning committee (spc) and the local staff. key responsibilities of the spc are to develop goals that reflect customer needs, all members are committed to the goals and each member ensures that their agency implements processes aimed at addressing the goals. local staff have two key responsibilities: 1. ascertain needs and provide information, advice and advocacy support to groups, communities and individuals within their area; and 2. inform government of unmet needs and priority issues case study the aboriginal affairs department (aad) aad organisation the aboriginal affairs department (aad) is a w. a. state government agency, hi 1994 a taskforce on aboriginal social justice was formed. the terms of reference for this taskforce was to review the activities of the government of western australia in relation to the social conditions and development of aboriginal people and to recommend a strategy for implementation of government's programmes. recommendations of the taskforce included: • the need for high calibre regional coordinators with a role to include breaking down barriers between government agencies and reducing waste and duplication; • a regional structure be implemented to undertake regional liaison and co-ordination across government agencies in co-operation with local aboriginal communities; • the establishment of an aboriginal affairs department structured as a planning, advisory, coordinating and monitoring agency and not responsible for administration of specific programmes. in order to implement these recommendations the government established aad. aad has utilised the virtual organisation planning model (figure 4) and established the aboriginal affairs co-ordinating committee (aacc). the aacc consists of chief executive officers from all state government agencies that have a role in aboriginal affairs including education, justice, police and housing. the major role of the aacc is to establish a set of strategic goals for aboriginal affairs and to ensure that each of their agencies implement processes aimed at achieving the goals. government and aacc figure 4 aboriginal affairs virtual organisation within aad a structure known as a local area co-ordination approach (lac) has been established to ensure: 108 • pervasive knowledge sharing, feedback and communication; • integration of environmental considerations at the earliest stages of design; • effective partnerships with customers. • commitment to using customer feedback to drive changes in operations, goals and vision; and • frontline employees are given the authority to deal with customer issues. this overall structure was implemented in the aad and evaluated through interviews and on-site assessments by the authors over a period of one year. one of the authors then had the opportunity to participate as manager of a specific customer-centric project and this is discussed below. the aboriginal heritage land management system the aad is responsible for the implementation of the aboriginal heritage management act which states that all aboriginal sites in w.a. be recorded and that prior to any development a search of sites must be conducted to determine the impact of development on sites in the area. it should be noted that these sites may not be inhabited but traditionally have strong spiritual significance for the aboriginal community and non-aboriginals and indeed aboriginals from different tribes outside this community would normally not be permitted access to these sites. for this reason much secrecy often surrounds the location of these sites. if sites are to be impacted then the developer must consult with the site custodians (typically through the regional commission of elders (rce)) and apply to the minister for aboriginal affairs to either destroy or move the site depending on the nature of the site and the type of development. as described previously, aad has a history of each division working separately with very little interaction between divisions or branches. originally the heritage and culture division (hcd) had full responsibility for all aspects of aboriginal sites. the division was strongly opposed to providing information on these sites to mining companies and land developers as they believed this was the most appropriate way of protecting sites. one manager openly labelled mining companies and some government agencies as "red necks" and was strongly opposed to forming alliances with these organisations and sharing information. following the restructure of the aad the management of the information component of aboriginal sites was moved to the information management division (imd). this prompted the evaluation of the current status of the sites register and the development of a project to completely revamp all aspects of the system. this necessitated not only close cooperation between the two divisions but the development of a network of agencies and an extension of these to an enhanced value network (evn). organisations that formed part of the network include: • aad imd (ict, gis, data management), hcd (archaeology, anthropology) • state agencies main roads w.a., department of conservation and land management, waters and rivers commission, department of resources development, department of minerals and energy • commonwealth agencies national native land tribunal, indigenous land corporation, department of defense, csiro • other agencies aboriginal legal services, goldfields land council, wesfarmers, awi administration services. shire of busselton, acmc these agencies all had different standards and procedures and often competed for budgets and authority. two main customer groups were identified (other groups include consultants and researchers): • aboriginal communities • mining and resource development companies robe river mining, alcoa, western mining, bhp iron ore, acacia resources, packman mining, normandy mining both groups required accuracy of information about location of sites and also cultural heritage and use of sites. however, aboriginal people required protection of information about closed sites while developers needed to know where these were in order to avoid them. many longstanding issues of political, cultural and economic tensions served to make collaboration a less then easy task. nevertheless, compromises were reached, one example being where boxes which masked the exact location of closed sites were reduced from 100 square kilometres to four and sites would be designated closed only at the request of aboriginal people. full paper based files on these would be designated "red files" to prevent unauthorised access. 109 the system the gis was designed as a web-based application using internet technology. the database is linked to maps of the site locations and boundaries and access comes via the database. clients often assign their own site references and these are cross-indexed with the aad references to allow searching by client reference number. access is only available to organisations who have signed a licensing agreement in accordance with the endorsed policies, and access automatically ceases on expiry of the agreement. customers will be provided with site maps within the constraints implemented for preservation of closed sites. the new system streamlines site registration and site searches resulting in significant reductions in turnround time from six to ten weeks to two to three minutes and provides much greater accuracy. companies can also download mapping and site index data into their own systems with monthly updates. the library catalogue allows for a variety of different reports accessible to the public. for example, a consultant can specify a mining tenement of interest and all heritage survey reports within this area can be identified by streamlining site registration and searches, improving the relevance and accuracy of stored data, and making the information more accessible the new system has had a positive impact on the whole enhanced value network. lessons learned customer-centric model of aad the primary aim of the restructured aad is to ensure that all government services revolve around its customers as opposed to establishing bureaucracies that customers need to break into to obtain the services they require. that is to establish a mechanism that will: • assist aad's customers receive equitable services from government agencies; • inform government of unmet needs and priority issues; and • facilitate co-ordination of services across agencies. to achieve this the department has looked at ways to best share resources, skills and knowledge to provide better access to a wide range of government services for their clients. this fits with the developing model of egovernment (table 2) as follows: unit of analysis resources basis for access to competence added value of managers value creation sources of managerial tension — the agency aad: imd hcd itc heritage knowledge aad specific processes nurture and build cross divisional (virtual) teams autonomous divisional autonomies and different core competencies network of agencies b2b + state, commonwealth and other agencies legal land use statutory research privileged access to system for all agencies within the network manage multiple, complex and conflicting agency relationships for all partner agencies agencies' different regulations and conflicting purposes enhanced value network e-government + aboriginal communities mining and resource development development plans mineral sources closed and open sites tradition and customs secure authorised access for searching and updating accurate site info preservation of culture harness aboriginal knowledge and culture manage land exploitation and development more effectively for both aboriginals and developers aboriginal customs (preservation) corporate development plans (exploitation) table 2: customer value alliance model of heritage management system 110 that is each partner in the value alliance : • brings its core competence aboriginal affairs department is not required to have expertise in areas such as education, health, housing or justice; • has access to other competencies and customer life cycles; • develops flexible collaborative partnerships the amount of involvement each agency has in each of the goals varies depending on the agency's specific skills and over time the amount of involvement will change as progress is made towards achieving the goals; • trusts and is trusted for this co-alliance to be successful mutual cooperation will need to develop. this has started to develop through the coordinating committee. all chief executive officers have participated in the meetings and not sent a representative, and agreement has been reached on the primary goals. it remains to be seen if this mutual cooperation does flow down through each agency. the local area coordinators are working with their local counterparts on common goals however sections of each agency's central office are yet to develop mutual cooperation; • establishes appropriate communication links where all stakeholders have evidence of information flowing up and down the model; • must see immediate benefits and be able to impact on the goal setting process this is particularly important for clients and the lacs are pivotal to this process and must be recruited with care. value alliance model of heritage system for this alliance to work, trust was the most important factor. aad took the first step towards this by reversing the policy that "protection means exclusion to information" and allowed their information to be vulnerable. this vulnerability of one party to another was a crucial concept that had to be seen to be shared by all players to ensure collaboration. (mayer et al, 1995). protection was defined as disclosure of information and developers were portrayed as organisations who inherently wanted to do the "right thing", adhere to legislative requirements and assist with protection of sites. trust was engendered in the aboriginal communities by ensuring they had input to all policies and procedures and that they were regarded as the senior decision makers to any changes in these. the second success factor came from the decision made from the outset to centralise the approach and establish a steering committee representative of all stakeholders to oversee the knowledge gathering and management. this committee was charged with the coordination of internal and external networks as well as the effective use of resources to ensure a positive outcome for the project. the processes that were followed matched closely the guidelines proposed by berry and parasuraman (1997) for developing an effective service-quality information system: 1. measure service expectations 2. emphasise information quality 3. capture customer words 4. link service performance to business results 5. reach every employee. throughout the project and beyond implementation customer satisfaction surveys were conducted, stakeholder groups were interviewed, the steering committee formed a customer advisory panel and all processes were constantly evaluated and altered as necessary. the perceived success of the system was based on technical achievement (the old system was paper-based and had remained unchanged for 15 years) but more importantly on the successful collaboration with a number of diverse players. the building block approach assisted this greatly since the aad had previously restructured its agency and established inter-agency collaboration around customer services. this meant that in the development of this particular system priority could be given to building the enhanced value network through the customers with appropriate inter-agency communication channels already established. the lacs assumed the most important role of customer interface and support with full authority and support from the agency network. conclusion as a study in progress the establishment of this model is in its infancy. it is currently too early to tell what impact this model has had on the goals aad have set out to achieve nor the effect it has had on the department's primary client base aboriginal people. further research is required as aad further implements its lac model and establishes appropriate communication procedures. the authors are also now evaluating this model in the context of two different government agencies who are attempting to implement customer-centric, e-government systems. il l there is much research left on the subject of the implementation of the value alliance model in a government agency, especially with regard to the distribution of information and communication. managing a virtual organisation may require a whole new set of virtual information leadership skills (morin et al, 2000). storing knowledge and expertise from both partners and customers are also important areas of consideration. references barreyre, p. y. (1988). "the concept of'impartition' policies: a different approach to vertical integration strategies." strategic management journal 9: 507-520. bum, j. m. and hackney, r. (2000) strategies for i-business change in virtual markets: a co-evolutionary approach. international journal of e-business strategy management, vol, 2, no. 2, 123-133. burn, j. m. and barnett, m. l. (2000) emerging virtual models for global e-commerce world wide retailing in the e-grocery business. special issue on global e-commerce, special millennium issue of journal of global information technology management, vol 3, no. 1, pp 18-32. campbell, a. and p. dinicola (1997). "virtual organisations. problem setting and research agenda." . cavanagh, j. j. and d. j. livingston (1997). serving the american public: best practices in customer-driven strategic planning. el sawy, o. a., malhotra, a., gosain, s. young, k. m. "it-intensive value innovation in the electronic economy: insights from marshall industries," mis quarterly, (32:3), 1999, pp. 305-335. glassey, o. (2001). model and architecture for virtual one-stop public administration. 9th european conference on information systems, bled, 969 -977. hopkins, c. and j. k. jamil (1997). serving the american public best practices in one-stop customer service., federal benchmarking consortium. jellinek, d. (2000).e-government reality or hype? e-government bulletin, october. avaialble at http://www.cisp.org/imp/october_2000/10_00jellinek.htm miles, r. e. and c. c. snow (1995). "the new network firm: a spherical structure built on a human investment philosophy." organizational dynamics 23 (4): 4-18. morin, t. devansky, k. little, g. and petrum, c. (2000). the future of information leadership. in information leadership a government executive's guide, pricewaterhousecoopers publications. prahalad, c. k. and v. ramaswamy (2000). co-opting customer competence. harvard business review 78(l)(79-87). schoeniger, e. (2000). the future of electronic government. unisys exec, sep-oct, 2000, 8-14. sprecher, m. h. (2000) racing to e-govemment: using the internet for citizen service delivery. government finance review, vol. 16, 5, 21-22. stamoulis, d. s., gouscos, d., georgiadis, p. and martakos, d. (2001). re-orienting information systems for customer-centric service. the case of the greek ministry of finance. 9th european conference on information systems, bled, 977-986. stratford, j. s. and stratford, j. (2000). computerised and networked government information. journal of government information, 27, 385-389. symonds, m. (2000). government and the internet: the next revolution. the economist, jun 24. ticoll, d., lowry, a. and kalakota, r. (1998) joined at the bit, in blueprint to the digital economy creating wealth in the era of e-business don tapscott, alex lowy and david ticoll, mcgraw-hill traunmuller, r. (2001). relevance of portals in e-govemment. 14th bled electronic commerce conference, 35-45. von hoffrnan, c. (1999). the making of e-government. co! enterprise magazine, nov. 15. zulfiker, k. a., pan, s, l., lee, j. n. and huang, j. c. (2001) e-government: an exploratory study of on-line electronic procurement systems. 9th european conference on information systems, bled, 1010-1022. 112 the australasian journal of information systems (ajis) began in 1993 and was based on the simple realisation that there was a need for an academic ‘outlet’ for is research in australia australasian journal of information systems volume 18 number1 2013 5 paradigmatic approaches used in enterprise resource planning systems research: a systematic literature review kevin burgess cranfield university kevinburgess@cranfield.ac.uk don kerr university of sunshine coast dkerr@usc.edu.au luke houghton griffith university l.houghton@griffith.edu.au abstract the purpose of this paper is to determine the range of research paradigms employed in a subset of the information systems (is) literature, namely enterprise resource planning (erp) systems. a systematic literature review based on papers that mentioned erps was drawn from eight of the most highly ranked journals according to their hindex. the findings indicate that the majority (96.6%) of the papers on erp systems described research that was conducted within a positivist research paradigm, which is a far higher proportion than is suggested by other research in the general is literature (approximately 81%). paradigm selection has a profound impact upon the subsequent nature of the body of knowledge that is developed in any area of inquiry. given the findings of this study demonstrated the dominance of a single paradigm this paper suggests that there is a strong case for erp researchers to re-examine paradigm selection. in particular, to review how effectively this present paradigm selection helps or hinders to the development of the erp body of knowledge, especially in respect to the issues of importance to managers within organizations (notably those concerned with social and change management issues). this research also identifies areas where existing paradigm evaluation methods could be enhanced and refined in respect to nonpositivist classifications. keywords: erp, research paradigms, epistemology, ontology. introduction for over 20 years, information systems (is) researchers (e.g. orlikowski and baroudi, 1991; goles and hirschheim, 2000; mingers 2004; chen and hirschheim, 2004; richardson and robins, 2007) have engaged in a debate as to the importance of knowing what research paradigms were used when researching is issues. in addition, researchers such as orlikowski and baroudi (1991) and koch, olsen and tambo (2010) consider this debate to be important as it helps the discipline better understand the implications of paradigm selection in respect to theory development. in this paper, we explore the question ‘where does enterprise resource planning (erp) systems research sit in relation to the paradigms used by is researchers in general?’ the rationale for seeking to answer this overall research question is based on a view that erp systems are sufficiently mailto:kevinburgess@cranfield.ac.uk mailto:dkerr@usc.edu.au mailto:l.houghton@griffith.edu.au australasian journal of information systems volume 18 number1 2013 6 distinctive to warrant being treated as a separate discipline in its own right rather than a subset of is. if this suggestion is correct then it follows that erp research may also differ in the usage rate of research paradigms employed compared to the rate found in the main is literature. we took the view that it was important to determine if any differences existed, for the following reasons: i) the selection of research paradigms has a profound impact on how a body of knowledge in a particular field will develop. ii) if significant differences were found, these would provide secondary evidence to support the view that erp systems do represent a distinct discipline, sufficient to warrant developing a separate body of knowledge rather than treating it as subset of the wider is body of knowledge. factors which differentiate erp from normal is literature include a typical erp implementation being not only the most costly information and communication technology (ict) investment for an organisation, but also the most demanding in terms of the organisation-wide change management requirements it creates. every individual who is connected to the erp will, to varying degrees, have to be involved in workplace changes during implementation. these changes can vary from having to learn how to use the new system through to fundamentally changing work processes and practices in order to align with the new system's requirements. the scale of change generated by erps, would therefore suggest that this particular strand of is research would be better disposed to the application of social science methodologies involving qualitative research paradigms there is a body of opinion in the erp literature which claims that erps are distinctly different from traditional is in many dimensions, and particularly with respect to implementation issues (robey, ross and boudreau, 2002; morris and venkatesh, 2010). these issues include business reengineering; dependence on external consultants; high cost and disruption caused by widely reported failure rates; having to manage greater complexity as erps have been describes as “typically the largest, most complex and most demanding information systems implemented by firms” (grabski, leech and schmidt, 2011, p.37); and very high costs. it is claimed that implementations can often represent the single largest item of organizational is expenditure (hawari and heeks, 2010) where it is estimated between three and seven times more money is spent on erp implementation and associated services compared to the initial purchase of the software license (scheer and habermann, 2000). critically these issues, as reported, are clearly tied to a body of knowledge which in turn is influenced by the research paradigm used. because a research paradigm carries with it several assumptions (e.g. about social structures) which in turn shape the knowledge used and the issues reported in the erp literature, it is of great importance that erp researchers fully understand the implications of their paradigm selection. there is also a growing body of erp literature that has highlighted the need to pay greater attention to social factors when considering research into erps (nandhakumar, rossi and talvinen, 2005; wood and caldas, 2001; markus, 1983; orlikowski, 2000). we speculated, in our research questions, that if the differences between erp and is were sufficiently distinctive to warrant separating the two into different research categories, then this might also reflect the differences in extant research paradigm usage rates. such differences, if proven to exist, would in turn reflect in the kinds of knowledge derived from research which, in turn, is used to inform practical concerns about erp systems. we were also eager to examine areas of similarity in the choice and usage rates of research paradigms. most of the studies conducted in the generalised is literature to determine the range of research paradigms employed have concluded that there is a strong bias toward positivism (orlikowski and baroudi, 1991; hirschheim and klein, 1992; goles and hirshheim, 2000; mingers, 2004). however, despite the dominance of a single paradigm there is also evidence that the is australasian journal of information systems volume 18 number1 2013 7 literature has been expanding the range of research paradigms employed. for instance, orlikowski and baroudi’s (1991) study claimed positivism accounted for 96.8% of studies. slightly over a decade later chen and hirschheim’s (2004) study had reduced this figure to 81%. richardson and robins (2007) contended that chen and hirschheim (2004) overestimated positivism and that a more exact figure was just below 50%. this decline in positivism has been attributed to a recognition within is to expand the research paradigms used in order to better understand critical social issues such as the role played by people (chen and hirschheim, 2004; richardson and robins, 2007). because erp literature had suggested a need to pay more attention to social factors, we assumed that investigating social factors would involve greater use of social science research paradigms which sit outside the positivist tradition. as the social sciences often employ a diverse range of research paradigms (jarvie, 2011) which also tend to change within short time periods (baum and rowley, 2002) then we speculated that the erp literature would also be accompanied by an increase in diversity of the research paradigms used. for all the aforementioned reasons, we were eager to conduct a basic comparison of research paradigm usage rates between erp and is, as suggested by our research question. furthermore, given the raft of social issues associated with erp implementations, we were curious to explore if erp research tended to favour paradigms that were more aligned with the social rather than the natural sciences. in order to examine these matters in greater depth we conducted research designed to answer the following two questions. first, what types of research paradigms are employed in the erp literature? second, are there any differences in the usage rates of research paradigms in erp literature compared to mainstream information systems (is) literature? background literature review several studies have been conducted in the generalised is literature to determine the range of research paradigms employed. most studies have suggested there is a strong bias toward positivism. (orlikowski and baroudi, 1991; hirschheim and klein, 1992; goles and hirshheim, 2000; mingers, 2004). most of the major studies investigating the research paradigms used in is literature have employed chua’s (1986) classification framework (orlikowski and baroudi,1991; chen and hirschheim, 2004; richardson and robins, 2007). chua’s (1986) classification framework consisted of three possible research paradigms a paradigm was defined by chua as a community of scientists who share ‘a constellation of beliefs, values, and techniques’ and these beliefs ‘circumscribe definitions of' worthwhile problems and acceptable scientific evidence’ (chua, 1986, p.602) the three distinct research paradigms are positivist, interpretivist and critical studies. positivist studies are premised on the existence of a priori fixed relationships within phenomena which are typically investigated with structured instrumentation. interpretive studies, by contrast, assume that people create and associate their own subjective and inter-subjective meanings as they interact with the world around them. finally, critical studies seek to expose deep-seated structural contradictions within social systems and the use and abuse of power (chua, 1986). in the orlikowski and baroudi (1991) study, their classification of positivism was in accordance with chua’s framework; however, they also classified certain descriptive studies as positivist. orlikowski and baroudi (1991, p.5) defended this categorisation by stating that ‘in "descriptive" work, researchers attempted no theoretical grounding or interpretation of the phenomena; rather, they presented what they believed to be straightforward "objective”, “factual”, accounts of events to illustrate some issue of interest to the information systems community’. the criteria adopted by australasian journal of information systems volume 18 number1 2013 8 orlikowski and baroudi (1991, p.5) to classify interpretive studies were ‘evidence of a nondeterministic perspective where the intent of the research was to increase understanding of the phenomenon within cultural and contextual situations; where the phenomenon of interest was examined in its natural setting and from the perspective of the participants; and where researchers did not impose their outsiders' a priori understanding on the situation.’ classification of “critical” occurred where evidence was found of ‘a critical stance towards taken-for-granted assumptions about organizations and information systems, and a dialectical analysis which attempted to reveal the historical, ideological, and contradictory nature of existing social practices’ orlikowski and baroudi (1991, p.6). orlikowski and baroudi (1991) reviewed 155 is research articles from four sources in the period between 1983 and 1988. they found 96.8% of these research articles were positivist, 3.2% were interpretive and none was critical. in another study, goles and hirschheim (2000, p.254) concluded that ‘although the proportion of positivist to non-positivist articles varies somewhat from study to study, the indisputable consensus is that positivism dominates information systems research’. chen and hirschheim (2004) conducted a follow up study of orlikowski and baroudi’s (1991) work, based on 1893 articles published between 1991 and 2001 and drawn from eight journals (four european and four us-based journals). their study found that ‘positivist research still dominates 81% of the published material’ (chen and hirschheim, 2004, p.197). while using chua’s (1986) classification framework, they restricted their analysis to either positivist or interpretive paradigms, the rationale being that they concurred with the orlikowski and baroudi (1991) study, and its conclusion that there was a lack of interest shown in critical is research. a third study conducted by richardson and robins (2007) challenged chen and hirschheim (2004) on three grounds. first, they claimed that the lack of interest in critical is research (cisr) was restricted to higher ranked is journals. they implied that higher ranked journals may exercise an institutional bias which systematically excludes cisr. hence they concluded that cisr was the missing paradigm because ‘the development of research taking a critical approach in is has been represented in a number of the top publications on a very small scale between 1991 and 2001’ (richardson and robins, 2007, p.264). second, they challenged the selection method used, claiming it resulted in an inflated figure. their reasoning here was that the conclusion reached by chen and hirschheim (that 81% of the research was positivist) was based on a sample of 1131 (60%), while the original population was 1893. richardson and robins claim that it is not entirely clear why the other 40% were excluded but it is assumed that it was because they consisted of ‘materials such as conceptual studies and other forms of research carried out without a guiding methodological framework’ (richardson and robins, 2007, p.252). when richardson and robins adjusted chen and hirschheim’s results relative to the total population (1893), the proportion of positivist categories dropped from 81% to 48.3%. the analysis undertaken by richardson and robins (2007) appears to be that if 40% were excluded on the grounds that they could not be classified, then they assumed they could be classified as non-positivist. as they did not explain how they determined that a sample which could not initially be classified under any heading could then be conclusively classified as non-positivist, it is difficult to comprehend the reasoning used to justify their conclusion about overestimation. the third challenge was around interpretation of classifications. richardson and robins (2007) argued that chen and hirschheim’s exclusion of the critical research paradigm was unjustified because of the claim that the number of studies using critical research had been increasing. to demonstrate the basis for this challenge, they examined ‘a similar set of publications over the same time period (1991-2001) for papers which fall in the critical paradigm’ (richardson and robins, 2007, p.254). nine sources were used, including seven of the eight sources used in the chen and australasian journal of information systems volume 18 number1 2013 9 hirschheim (2004) study. richardson and robins (2007) also used the chua (1986) classification schema and they were able to identify a total of 31 articles that fell under the cisr category. however, without knowledge of the total sample it is not possible to say what percentage this figure represented. similarly, no comment was made on the percentage occupied by the positivist and interpretivist paradigms. the conclusion offered was a wide range of concerns about the lack of cisr papers in highly ranked journals and suggested that ‘critical research is likely to remain the preference of a minority of is researchers for the foreseeable future’ (richardson and robins, 2007, p.265) and that the critical school should ‘begin to develop its own voice’ (richardson and robins, 2007, p.265). the different results reached by the chen and hirschheim (2004) and richardson and robins (2007) studies highlight that variations in interpretation are problematic when it comes to paradigmatic categorization research. despite richardson and robins making strenuous efforts to replicate the chen and hirschheim study by using similar data and the same classification scheme (namely, chua, 1986) they reached very different conclusions. while it is unclear as to what caused these differences, it is widely acknowledged that all paradigmatic classification schemas tend to be open to a wide range of interpretations (deetz, 1996; chua, 1986; willmott, 1993). a common criticism is “reification” – the tendency to mistake research abstractions for reality and producing biased and simplistic representations (chia, 1995). another is the tendency to change over time as ‘the criteria for paradigm comparison and evaluation are essentially judgmental, open to change and grounded in social and historical practices’ (chua, 1986, p.606). given the aforementioned limitations of classification schema the ‘question is not: are these the right categories or who fits in each? but: are these differences that make a difference?’ (deetz, 1996, p.192). classification framework developed for the research while a great deal has been written on the philosophy of science (for example – okasha, 2002) and social science research (for example – law and lodge, 1984) far less seems to be available by way of practical tools which can be consistently applied for the purposes of classification. burrell and morgan’s (1979) four sociological paradigm framework is the one of the earliest and most widely used of all such schema. however, it is now rarely used and has been challenged on several fronts, including its departure from kuhn’s original concept of paradigm, maintaining adherence to the incommensurability of paradigms view and reification of the dimensions used (deetz, 1996; lewis and kelemen, 2002; willmott, 1993). chua’s (1986) framework was developed for accounting research purposes, and despite its spread to is, does not appear to have been widely adopted elsewhere. johnson and duberley (2000) are the latest to have developed a classification system of research paradigms. from an is research perspective, their framework does appear to have some distinct advantages over previous categorization tools. it was developed specifically in the context of management research (which aligns well with erp issues); it examined the criticisms of previous attempts including burrell and morgan (but not chua); it matches almost identically with chua’s definition of positivism; and, importantly, appears superior to chua’s framework in that it provides a clearer delimitation of the absolute foundational elements of a paradigm, namely the epistemological and ontological stances taken. australasian journal of information systems volume 18 number1 2013 10 figure 1 below provides a graphic representation of the johnson and duberley schema (adapted from johnson and duberley, 2000, p.180) figure 1. johnson and duberley schema a key aspect of the above classification schema is that a research paradigm is determined by the possible combinations of ontology and epistemology as defined in either objective or subjective terms. johnson and duberley (2000) call epistemology the finding of ‘frameworks’ of warranted scientific knowledge that acts as a guideline for representations of what knowledge claims can be made. ontology is defined by the same authors as the philosophy of being or what researchers think reality, being and existence are thought to be. for example, a positivist ontology claims that reality is an objective set of causal laws (see figure 1, quadrant 1). a postmodernist conceives of reality as being dependent on the observers who create and sustain its meaning. an objective stance implies that there is a reality that can be accurately assessed while the subjective position is far more sceptical about what researchers can determine to exist and what knowledge and truth claims they can make. classification of quadrant 1, with an objective ontology and epistemology is defined as a positivist research paradigm. no theory is given in quadrant 2 as none exists which claims a subjective ontology and an objective epistemology. quadrant 3, with an objective ontology and subjective epistemology covers many theories, but tends to be dominated by than those commencing with the adjective ‘critical’. the term ‘critical’ as described in quadrant 3 therefore captures a wide range of theories including critical theory, critical realism and others. just what is meant by the term ‘critical theory’ is subject to many interpretations. richardson and robinson (2007) and sim and van loon (2009) view critical theory as covering a wide range of epistemological and ontological positions which include poststructuralists such as derrida (1973) and deconstructionists such as belsey (2002). johnson and duberley’s model, by contrast, makes it clear that there are very important ontological distinctions between critical theory and postmodernism. these ontological and epistemological distinctions have profound implications for how research should be conducted and what truth claims can be made. therefore, johnson and duberley’s model does not accept that the term should be so broad as to include theories which take entirely opposite ontological positions. under this model, a definition of critical theory is restricted to the original work of marx, the frankfurt school and australasian journal of information systems volume 18 number1 2013 11 habermas, or subsequent followers, which fit into an objective ontology and a subjective epistemology. postmodernism is used to classify theorists such as derrida, baulrillard, foucault and lyotard who, according to johnson and duberley (2000), share a subjective ontology and epistemology, quadrant 4 refers to a research paradigm which assumes that both the ontological and epistemological stances are subjective, and is categorised as postmodernism. therefore, johnson and duberley define theorists such as foucault and derrida as postmodernists in contradistinction to authors such as richardson and robinson (2007) and sim and van loon (2009) who would define these theorists as critical theorists. another position illustrated in figure 1, but not shown as a quadrant, is the rectangle entitled ‘conventionalists’. this term has been included because johnson and duberley claim that while many research approaches concede the difficulty of having either an objective ontology or epistemology, they are not prepared to embrace the postmodernist position. to do so would force them into accepting a relative position to all truth claims. johnson and duberley go on to claim that the seminal thinker on paradigms, kuhn, was a conventionalist. because of difficulties associated with how to apply and interpret different classification schema, findings on the topic of which research paradigms are being used are open to challenge. it is unclear if variation in findings to date is due to differences in interpretation or in the actual data. this problem is unlikely to be resolved in the near future as johnson and duberley (2000, p.9) state ‘in the epistemological literature we will find that there is considerable, and it would seem irresolvable, disagreement over epistemology and the standards by which we may discriminate warranted knowledge’. research method the authors used the h-index to determine what is journals were most highly cited from a list of highly ranked journals. the h-index (hirsch 2005) has gained acceptance as a measure of the distribution of citations for both individual authors and journals with mingers, macri and petrovici (2012, p.244) stating that ‘the h-index is a better citation-based metric for measuring the research contribution of a journal than the impact factor’. the h-index was developed by hirsch (2005) and is used to measure the productivity and impact of the published work of a scholar or a journal; it is based on cited papers and the number of citations that they have received; for example, an h-index of ten would mean that ten papers have each been cited ten times or more. h-index selection process the h-index for each journal was obtained using the software package publish or perish (harzing, 2010) and analysis was conducted on the top two tiers (a* and a) of journal rankings from the australian council of professors and heads of information systems (acphis 2010). the journal list and h-index for each journal is shown in appendix one. due to resource constraints, the selection of journals was restricted to those with an h-index of over 100 and a timeframe from 2000 to 2010. a total of nine journals were selected, the details of which are shown in table 1. australasian journal of information systems volume 18 number1 2013 12 journal h-index communications of the acm 292 harvard business review 223 mis quarterly 183 european journal of operational research 158 information systems research 120 california management review 113 information and management 109 journal of management information systems 108 human-computer interaction 104 table 1 – list of h-index journals in descending order of ranking at the time of selection in 2011. the rationale for selecting higher ranked journals was based on the assumption that the higher the ranking of the journal, the more likely it was that it would give emphasis to declaring its research stance. this in turn would make it easier to identify research paradigms. a potential limitation in selecting the h-index is that it has been criticised on several grounds. for instance, that it can be manipulated. as this critique applies to all journal ranking systems, our selection of the h-index was based on other criteria which suggested that, overall, it offered the most rigorous ranking system (e.g. refer mingers et al., 2012). another potential limitation is that there are discrepancies across the journals in terms of their research effort and emphasis given to erps. however, we concluded that to start making judgements as to what journals should be in or out based on our opinion would add more bias than staying within our original sampling design based on the h-index. a final criticism is that poor articles can end up being highly ranked because they are subject to frequent criticism. we sought to reduce this risk by two means. first, the h-index was used to select journals rather than specific articles. second, we took the view that because we only used articles drawn from the higher ranked journals the chance of introducing such a bias was further reduced. third, the focus of the research was on research paradigms while criticisms of articles tend to be more around methodological issues and errors. even if these articles were included in our sample the fact that their high rating was due to criticism would in itself ensure a sense of balance. furthermore such articles still provided insights into the paradigms being used to generate or refute knowledge within the erp literature and was therefore of interest to our study. for all of these reasons we were reasonably confident that the use of the h-index would not have an adverse impact on the overall aim of our study. each journal in the aforementioned list (table 1) was searched issue by issue for articles that had erp or enterprise resource planning as part of either the keywords or abstract. to limit the search to the most recent research, we selected journal articles from the period 2000-2010. the rationale for this was that most of the work done in erps has taken place in the previous ten years. secondly, given that this research represents a decade of progress, it is representative of the broad paradigmatic concerns and frames of references in the literature. this process yielded 90 articles. selection from this population was further refined to include only those papers that consisted of either primary research or a detailed meta-analysis of other studies. thirty-one articles which fell outside these selection criteria were excluded on the grounds that they were, for example, opinion pieces with no empirical or theoretical content or concerned with issues outside the scope of the study, such as capital investment decision. the remaining 58 articles used in the study were then classified against the criteria in johnson and duberley’s framework. australasian journal of information systems volume 18 number1 2013 13 classification framework it could be conjectured that by not using the chua model, we may have created difficulties for making a comparative analysis of the findings of our study with prior studies. however, we took the view that using the johnson and duberley model would help avoid the interpretation problems as exemplified by the differences in findings reached between the richardson and robinson (2007) and chen and hirschheim (2004) studies. it will be recalled that both teams of researchers used the same classification schemas and data yet reached different conclusions. while it was conceded that the introduction of a new classification framework would initially inhibit the ability to do comparative studies, the aspiration was that ‘using a model which pays more attention to differences that make a difference’ (deetz, 1996, p.192) may go some way toward providing a tighter classifications schema which may in turn reduce some of the interpretation problems. in addition, as this framework’s definition of positivism is very similar to that used by most other classification schemas, and given that most literature to date has suggested that the positivist research paradigm dominates, then the use of the johnson and duberley framework would not inhibit comparative analysis for contrastive purposes. the orlikowski and baroudi (1991) study highlighted that one area of potential definitional variations was associated with the concept of constructivism. constructivism is ‘an ontological position that asserts that social phenomena and their meanings are continually being accomplished by social actors. it implies that social phenomena are not only produced through social interaction but that they are in a constant state of revision’ (bryman, 2008, p. 19). orlikowski and baroudi (1991, p.15) highlighted that the ‘difference between weak and strong constructionist positions has implications for how interpretative research relates to research conducted in the positivist mode’. however, orlikowski and baroudi were less forthcoming on how to map these distinctions back to their epistemological and ontological assumptions. this may in part be a reflection of a reticence to move beyond seeing critical theory as a very broad continuum. for the purposes of this research, a strong constructivist position was mapped back to the postmodernist position in quadrant 4 of figure 1. while a weak constructivist position accepts realism, it further divides into two types – positivist and a broader camp which includes critical theorists and critical realists. weak constructivist positions were mapped back to either quadrant 1 or quadrant 3 on table 2. a naïve positivist would reject a weak constructivist position; however most modern positivists would accept that weak constructivism is involved in most observations (weber, 2004). while johnson and duberley’s (2000) taxonomy provides clear guidance on how to ultimately classify the data for the purposes of this research (at the paradigmatic level), the reality is that erp researchers rarely define the philosophical research paradigm they employed. in such circumstances, we had to infer the paradigm from other information provided in the papers reviewed. in order to make clear the logic used for drawing such inferences and to also increase consistency in how individual researchers went about making inferences, a slightly modified version (we added ontology) of crotty’s (2004) research process diagram was used. figure 2 provides a graphic outline of the ideal line of sight that should apply to the major elements involved in sound research. australasian journal of information systems volume 18 number1 2013 14 weak constructivists strong constructivists quadrant 1 (objective ontology objective epistemology) positivists believe in both realism (there is an external reality that is separate from our descriptions of it), and objectivism (an ontological position which assets that social phenomena and their meanings have an existence that is independent of social actors). despite our tendency to impose weak constructions on observations, through the use of appropriate methods reality can be understood in an objective manner quadrant 2 in the (subjective ontology, objective epistemology) no advocates of this position exist quadrant 3 (objective ontology subjective epistemology) critical theorists believe in realism but are cautious about objectivism arguing that conceptualisations provide a way of knowing reality rather than an objective representation of reality quadrant 4 (subjective ontology subjective epistemology) post-modernists – as social actors are always creating meanings; these meanings are antithetical to both types of realism maintained by weak constructivists. table 2 – weak and strong constructivist positions mapped to johnson and duberley’s (2000) paradigmatic framework. figure 2 – basic elements of the research process (adapted from crotty (2004)) definitions of the terms used in figure 2 are as follows: ontology, as mentioned previously, is concerned with the theory of being and, accordingly, addresses issues such as what entities exist or can be said to exist (hartwig, 2007). epistemology is the theory of knowledge and covers such topics as what we can know and how (scruton, 2004). crotty (2004) defines the theoretical perspective as the philosophical stance informing the methodology and thus providing a context for the research process and grounding its logic and criteria. methodology is defined as the strategy, plan of action, australasian journal of information systems volume 18 number1 2013 15 process or design lying behind the choice and use of methods for the desired outcomes. methods are defined as the techniques or procedures used to gather and analyse data related to some research question or hypothesis. for reasons that will be explained in more depth in the operationalization of this framework, at times we used all basic four levels of crotty research process to initially interrogate the data. despite this usage, it needs to be stressed that the unit of analysis for this study was the first level shown in figure 2 (ontology and epistemology) and not the other levels. while acknowledging that our research interest was on the knowledge creation process and that all four levels of the research process can exert an influence over knowledge creation, our study was strictly limited to the role that the research paradigm played in knowledge development. as johnson and duberley’s (2000) framework defined paradigms in terms of the combinations of ontological and epistemological stances, for the purposes of answering our research question it followed that our final analysis had to be restricted to the first level of crotty’s research process. operationalization of the framework the research was operationalized by each article being read independently and assessed by two of the authors of this paper. these readers first scanned articles at the highest level to see if ontological and epistemological positions were declared. if nothing definitive could be found at this level then the reviewers would move to the next level (theoretical perspective), and so on, until they found some evidence that could be used to determine what research paradigm was being deployed. this approach assumed researchers would always strive for consistency across all four dimensions of crotty’s model such that there would be a coherent line of sight from the highest level to the lowest level of the research concept. using this logic, it should be possible to draw inferences which could in turn be used to determine the research paradigm. several steps were taken to mitigate the risks associated with misclassification of research paradigms. to begin with, the first two readers re-read each article and checked for other contextual variables that may have been missed in the first reading. second, even if the individuals felt they could classify an article but had minor doubts, they flagged their concerns. third, a comparison of individual results was compiled. where there were disagreements, or agreements but minor concerns had been raised (by either or both reviewers) or if no classification was agreed to, then these articles were sent to the third reviewer for additional adjudication. fourth, when individual researchers had any major concerns, they did not record a classification. this procedure ensured that the third reviewer would have to get involved in a group process with the other two reviewers in order to help resolve anomalies. the use of this research approach has two limitations. the first is that the researchers in the papers we read may not have had, or even have wished to use, the clear line of sight that we used in our approach. the second is that the further down we had to go down on crotty’s adapted model (figure 2) the greater the level of inference involved and, therefore, the greater the risk of misclassifying the research paradigm. findings on erp literature table 3 summarizes the findings as mapped against johnson and duberley’s (2000) framework, while figure 3 is a graphical representation of the findings of paradigm usage scaled within that framework. australasian journal of information systems volume 18 number1 2013 16 epistemology article number 1 count % positivist 1-7, 9-24, 26-37, 39, 41-46, 48-58 53 91.4 % post-modernist (strong constructivist) 8 1 1.7% critical theory 40 1 1.7% unclear 2 25,38,47 3 5.2% table 3 research paradigms used in erp research figure 3: graphical representation of the findings of paradigm usage scaled within johnson and duberley’s (2000) framework. note this figure has deliberately been drawn large in order to clearly display the ratios between each approach and thereby graphically illustrate the finding as to the dominance of a single paradigm in erp research. 1 please refer to appendix two for article numbers. 2 this includes both articles that were not able to be classified and those where the two reviewers could not agree on an epistemology. australasian journal of information systems volume 18 number1 2013 17 discussion the results of this study fall between the orlikowski and baroudi (1991) study’s finding of 96.8% and the chen and hirschheim (2004) study’s findings of 81%. however, if we exclude the unclear results (as did both orlikowski and baroudi, and chen and hirschheim), then the percentage of positivist research rises to 96.6% and this is identical to the orlikowski and baroudi study conducted over twenty years ago. for the reasons already outlined, the differences in classification schema and interpretation make comparisons of non-positivist paradigms difficult. these overall findings are generally in line with several other studies and commentaries, such as those by orlikowski and baroudi (1991), goles and hirshheim (2000), mingers (2004) and chen and hirschheim (2004), which claim that positivism dominates is literature. the findings of our study strongly suggest that the choice of research paradigms used in erp literature is similar to that found in the wider contemporary is literature (albeit slightly more biased towards positivism). it will be recalled that richardson and robins (2007) used the same data and classification but different sampling methods to those of chen and hirschheim (2004). richardson and robins (2007) concluded that the figure of 81% was grossly inflated; when adjusted for the total, the figure fell to 48.3%. however, just as these authors were correct to highlight why the 81% was an inflated figure, we would argue they have, in turn, underestimated the percentage of positivist articles. while using richardson and robins’s (2007) classification of the primary data used in this study would deflate the dominance of positivism, it would not undermine the overall conclusion that erp literature employs research paradigms with a higher positivism bias than would be found in the wider is literature. furthermore, irrespective of which of the two sampling methods is applied to this study, positivism dominates (91.4%, if we declare the unclear articles as non-positivist, or 96% if we exclude them). the three papers we could not classify fell under the heading of weak constructivist. the difficulty was in deciding if they should be deemed positivist or critical, or both, as they could easily be classified as being multi-paradigmatic. our study did not initially consider multi-paradigmatic approaches. however, as far back as 2000, goles and hirshheim analysed other studies (notably orlikowski and baroudi, 1991 and walsham, 1995) and concluded that while is studies were heavily guided by the positivist paradigm, they advocated the need to end the ‘paradigm wars’ and move instead to a stance based on paradigmatic pluralism. other authors such as mingers (2004) and lewis and kelman (2002) have advocated similar stances. future studies would need to consider how to classify multi-paradigmatic stances. while none of the studies that used chua’s (1986) model explained the criteria used to exclude articles, if they used logic similar to that used in our study, then it would suggest to us that many of these excluded articles may be positivist. we therefore have reservations about richardson and robinson’s claims as to an overestimation of positivism. just because there is not enough information to classify an article as positivist, it doesn’t follow that it automatically becomes non-positivist. although this study used a different classification schema to that employed by other studies, the findings did align with the interpretation applied by chen and hirschheim (2004) and orlikowski and baroudi (1991). variations in interpretation will remain a difficulty with these types of studies until a more stable and widely agreed classification schema is accepted. therefore, developing a more rigorous and widely accepted classification schema for categorising paradigms would be a worthwhile enterprise in order to improve the usefulness of future research in terms of making meaningful trend comparisons. we consider our study using the johnson and duberley (2000) framework to be a useful start in that direction. the overwhelming bias towards positivism in the erp domain suggests that the increasing interest in social factors associated with erp implementations has not been accompanied by changes in the australasian journal of information systems volume 18 number1 2013 18 research paradigms used. while this study did not track paradigmatic changes over time in the erp literature, the wider is literature does suggest a remarkably stable pattern in respect to the dominance of positivism over the past decade. given this trend, and the strong alignment of is and erp on research paradigms, it would appear reasonable to imply that what is happening in both bodies of literature is similar. while there are many possible explanations for why erp literature maintains a higher rate of positivism than the wider is literature, there are two that appear to be worthy of investigation. the first is that erps are in some sort of lag mode relationship to is, possibly due to factors such as having little flexibility. the second is that while some researchers (robey, ross and boudreau, 2002; morris and venkatesh, 2010) have claimed that the differences between erps and is may be significant (especially in social terms), consistent with the claims of positivism, both can still be researched under the one paradigm. indeed, positivism’s claim of strength is that it can be applied to all forms of research. however, there are many other plausible explanations and there is a need for further research before any strong conclusions can be drawn. the contribution of our paper has been to establish the ratio of research paradigms used in the selected erp literature. this has helped inform what erp’s paradigmatic stance is, relative to the paradigmatic debate. the findings of our study give rise to the question: is the dominance of a single research paradigm in the erp literature important? as there is already a vast body of is literature which argues that it is (orlikowski and baroudi, 1991; richardson and robins 2007), we accept that this is most likely the case. it is our hope that having demonstrated that erp literature is dominated by a single research paradigm, other researchers will use our findings to explore what implications, if any, follow from continuing to develop a body of knowledge overwhelmingly within a positivist paradigm. if richardson and robinson (2004) are correct in their claim of institutional bias in higher ranked journals, then the sample of journals with an h-index of over 100 that we used may simply be reflecting a bias. until this claim is investigated in depth, the strength of the findings in this study can be easily challenged. future research in this area should include a larger cross-section of journals and, as suggested by richardson and robinson (2004), may include lesser-ranked journals. conclusions this research set out to determine the types of research paradigms employed in erp literature. the intent of our meta-level analysis of the paradigms used was descriptive, with no intent to offer a detailed prescription of research methodologies and tools which could be used within each paradigm. our descriptive findings led us to draw the conclusion that the erp literature is in lag compared to the wider is literature. in the last thirty years of is literature, the first two decades were dominated by positivism. in the last decade, however, there has been a widening of the paradigms used. our findings show that a similar broadening is not reflected in the erp literature. in fact, the rate of usage of the positivist paradigm in the erp literature is almost identical to that found in the general is literature of the first two decades. this difference between the erp and is literature is made all the more stark when it is recalled that the sample used for the erp literature was drawn from 2000 to 2010. if the wider is literature has found its way to a broadening of research paradigms used in order to enhance theory development, it begs the question of why this is not similarly reflected in erp literature. this findings seems all the remarkable given the previously mentioned call for more socially aware research into erp implementations there would appear to be, therefore, good grounds to justify erp researchers engaging more fully in this debate on paradigm usage. such an engagement would help determine if erp activities are sufficiently different from general is activities to warrant the development of its own body of knowledge or if it should simply be subsumed under existing is theories. australasian journal of information systems volume 18 number1 2013 19 this study has two main limitations. the first is that the data gathered was limited to those journals with an h-index of 100 or over and this may be a biased sample. this limitation can be overcome with wider sampling. the second limitation has to do with interpretation and is not so easily resolved. as shown, there are many ways to interpret research paradigms and this task is not made any easier by the common practice of using the same terms to mean vastly different concepts. developing a more robust classification scheme would therefore be a useful development in helping improve the process for conducting comparative studies into the paradigms being used in is and erp research. references baum, j.a.c. and rowley, t.j. (2002). companion to organizations: an introduction. in the blackwell companion to organizations, joel a.c. baum (ed) blackwell publishers ltd. oxford. belsey.c. (2002), poststructuralism: a very short introduction, oxford university press, oxford. bryman, a. (2008), social research methods (3rd edition), oxford university press, oxford. burrell, g. and morgan, g. (1979). sociological paradigms and organisational analysis, ashgate publishing, aldershot, england. chen, w. and hirschheim, r. (2004). a paradigmatic and methodological examination of information systems research from 1991 to 2001. information systems journal, 14 197-235 chia, r. (1995). from modern to postmodern organizational analysis. organization studies, 1995, 16(4), 579-605 chua, wai fong. (1986). radical developments in accounting thought. the accounting review 61, 601-632 crotty, m. (2003). the foundation of social research: meaning and perspective in the research perspective, sage london derrida j. (1973). speech and phenomena. evanston, il; northwestern university press deetz s. (1996), describing differences in approaches to organization science: rethinking burrell and morgan and their legacy. organization science 7(2), 191-207. grabski, s.v., leech, s.a. and schmidt, p.j. (2011). a review of erp research: a future agenda for accounting information systems. journal of information systems, 25 (1), 37-78. goles t,and hirshheim r. (2000). the paradigm is dead, the paradigm is dead … long live the paradigm: the legacy of burell and morgan. omega, 28, 249-268 harzing a.w. (2010). the publish or perish book: your guide to effective and responsible citation analysis. published by tarma software research pty ltd, melbourne, australia hawari, a and heeks, r. (2010). explaining erp failure in a developing country: a jordanian case study.,journal of enterprise information management, 23 (2),.135 – 160 hirsch j. e. (2005). an index to quantify an individual's scientific research output, proc natl acad sci u s a. 2005 november 15; 102(46), 16569–16572. hirschheim, r. and klein h.k., (1992). paradigmatic influences on information systems development methodologies: evolution and conceptual advances. in yovits, m. (ed.), advances in computers, 34, 293-392 jarvie, i, (2011). the philosophy of social sciences from mandeville to mannheim in ian c. jarvie and jusús zamora-bonilla (eds) the sage handbook of social sciences. sage publications, london australasian journal of information systems volume 18 number1 2013 20 johnson, p and duberley, j.(2000). understanding management research, sage publications, london. koch c., olsen m. and tambo t. (2010). how little do we need to know about enterprise resource planning (erp)? a critical review of information systems research on erp. in: proceedings / information systems research in scandinavia (iris), 33, august, 2010. law, j. and lodge, p. (1984), science for social scientists, london: macmillan. lewis, m.w. and kelemen m. l. (2002). multiparadigm inquiry: exploring organizational pluralism and paradox, human relations 55(2), 251-275 light, b. and wagner, e. (2006), integration in erp environments: rhetoric, realities and organisational possibilities. new technology, work and employment, 21 (3), 215 – 228. markus, m. lynne. (1983),. power, politics, and mis implementation, communications of the acm, 26(6), pp 430-444. mingers, j, macri, f and petrovici, d. (2012). using the h-index to measure the quality of journals in the field of business and management. information processing and management 48, 234-241. mingers, j. (2004). future directions in management science modelling: critical realism and multimethodology.in s.fleetwood and s.ackroyd (eds) critical realist applications in organisation and management studies. routledge, london morris, m. g. and venkatesh, v. (2010). job characteristics and job satisfaction: understanding the role of enterprise resource, mis quarterly, 34 (1), pp.143-161 nandhakumar, j., rossi, m., and talvinen, j. (2005). the dynamics of contextual forces of erp implementation, journal of strategic information systems, 14 (2), pp. 221–242. okasha, s. (2002). philosophy of science: a very short introduction. oxford university press, oxford. orlikowski, w. (2000). using technology and constituting structures: a practice lens for studying technology in organizations, organization science, 11(4), 404–428. orlikowski, w, j. and baroudi, j.j. (1991). studying information technology in organizations: resaerch approaches and assumptions. information systems research, 2, (1), 1-28 richardson, h. and robins, b. (2007), the mysterious case of the missing paradigm: a review of critical information systems research 1991-2001, information systems journal, 17, 251-270 robey, d, ross j. w. and boudreau, m.c. (2002), learning to implement enterprise systems: an exploratory study of the dialectics of change. journal of management information systems, 19 (l), pp. 17-46. scheer, a.w. and habermann, f. (2000), “enterprise resource planning: making erp a success”, communications of the acm, 43 (4), pp. 57-61. sim, s. and van loon, b. (2009), introducing critical theory. icon books, london wood, t. and caldas, m.p. (2001), reduction and complex thinking during erp implementations. business process management journal, 7, (5) pp. 387-393 walsham, g. (1995), "the emergence of interpretivism in is research." information systems research, 6(4) pp. 376-394. weber r (2004), the rhetoric of positivism versus interpretivism: a personal view (editor’s comment) mis quarterly 28 (1) pp. iii-xii. australasian journal of information systems volume 18 number1 2013 21 appendix 1 – a and a* journal list from acphis with h-index derived from publish or perish journal acphis rank h-index decision support systems a* 85 european journal of information systems a* 62 information and management a* 109 information systems journal a* 80 information systems research a* 120 journal of information technology a* 86 journal of management information systems a* 108 journal of the association for information systems a* 68 mis quarterly a* 183 behaviour and information technology a 63 california management review a 113 communications of the acm a 292 communications of the association for information systems a 88 data and knowledge engineering a 68 data base for advances in information systems a 11 electronic commerce research a 39 electronic markets a 43 european journal of operational research a 158 harvard business review a 223 human-computer interaction a 104 ibm systems journal a 96 information and organization a 38 information and software technology a 59 information systems a 96 information systems frontiers a 33 information technology and people a 47 international journal of electronic commerce a 59 international journal of forecasting a 45 international journal of medical informatics a 58 internet research: electronic networking, applications and policy a 53 journal of behavioral decision making a 59 journal of computer information systems a 29 journal of global information management a 58 journal of information systems a 61 journal of organizational computing and electronic commerce a 40 journal of strategic information systems a 54 misq executive a 6 mit sloan management review a 60 omega a 86 scandinavian journal of information systems a 28 australasian journal of information systems volume 18 number1 2013 22 appendix 2 – analysed papers 1. soh, c., kien, s.s. and tay-yap, j. (2000). cultural fits and misfits:is erp a universal solution? communications of the acm, vol. 43, no. 4, pp. 47-51 2. willcocks, l.p. and sykes, r. (2000). the role of the cio and it function in erp. communications of the acm, vol. 43, no. 4, pp. 32-48 3. van everdingen, y., van hillegersberg, j. and waarts, e. (2000). erp adoption by european midsize companies: searching for erp systems offering a perfect fit. communications of the acm, vol. 43, no. 4, pp. 27-31 4. markus, m.l., tanis, c and van fenema, p.c. (2000). multisite erp implementations. communications of the acm, vol. 43, no. 4, pp. 42-46 5. kumar, k and van hillegersberg, j. (2000). erp experiences and evolution. communications of the acm, vol. 43, no. 4, pp. 22-26 6. sprott, d. (2000). componentizing the enterprise application packages communications of the acm, vol. 43, no. 4, pp. 63-69 7. kumar, k. (2001). technology for supporting supply chain management. communications of the acm, vol. 44, no. 6, pp. 58-61 8. davison, r. (2002). cultural complications of erp. communications of the acm, vol. 45, no. 7, pp. 58-61 9. lee, j., siau, k. and hong, s (2003). enterprise integration with erp and eai. communications of the acm, vol. 46, no. 2, pp.54-60 10. arinze, b. andanandarajan, m. (2003). a framework for using oo mapping methods to rapidly configure erp systems. communications of the acm, vol. 46, no. 2, pp.61-65 11. liang, h., xue,y., boulton, w.r. and byrd, t.a. (2004). why western vendors don’t dominate china’s erp market: examining cases of failed erp system implementation in china and explaining the unique circumstances. communications of the acm, vol. 47, no. 7, pp. 69-72 12. delen, d. dalal, n.j. and benjamin, p.j. (2005). integrated modeling: the key to holistic understanding of the enterprise. communications of the acm, vol. 48, no. 4, pp.107-111 13. shore, b. (2006). enterprise in across the globally disbursed service organization. communications of the acm, vol. 49, no. 6, pp102-106 14. morris, g. and venkatesh, v. (2010). job characteristics and job satisfaction: understanding the role of enterprise resource planning system implementation. mis quarterly vol. 34 no. 1, pp. 143-161 15. ko, d-g., kirsch, l.j., and king w. r. (2005). antecedents of knowledge transfer from consultants to clients in enterprise system implementations. mis quarterly vol. 29 no. 1. pp. 59-85 16. gattiker, t.f and goodhue, d.l (2005). what happens after erp implementation: understanding the impact of interdependence and differentiation on plant-level outcomes. mis quarterly vol. 29 no. 3, pp. 559-585 17. cotteleer, m.j. & bendoly, e. (2006). order lead-time improvement following enterprise information technology implementation: an empirical study, mis quarterly vol. 30 no. 3, pp. 643-660 18. karimi, j., somers, t.m. and bhattacherjee, a (2007). the role of information systems resources in erp capability building and business process outcomes. journal of management information systems, vol. 24, no. 2, pp. 221–260 19. karimi, j., somers, t.m. and bhattacherjee, a (2007). the impact of erp implementation on business process outcomes:a factor-based study. journal of management information systems, vol. 24, no. 1, pp. 101–134 australasian journal of information systems volume 18 number1 2013 23 20. kauffman, r.j and tsai, j.y. (2009). the unified procurement strategy for enterprise software: a test of the “move to the middle” hypothesis. journal of management information systems vol. 26, no. 2, pp. 177–204 21. seddon, p.b., calvert, c. and yang, s. (2010). a multi-project model of key factors affecting organizational benefits from enterprise systems. mis quarterly, vol. 34, no. 2 pp. 305-328 22. boh, w, f. & yellin, d. (2007). using enterprise architecture standards in managing information technology. journal of management information systems, vol. 23, no. 3, pp.163–207 23. gefen, d. (2004). what makes an erp implementation relationship worthwhile: linking trust mechanisms and erp usefulness. journal of management information systems, vol. 21. no. 1. pp. 263-288. 24. hitt, l.m., wu, d.j. and zhou, x. (2002). investment in enterprise resource planning: business impact and productivity measures. journal of management information systems, vol. 19, no. 1, pp. 71-98. 25. robey,d., ross, j.w. and boudreau, m-c. (2002). learning to implement enterprise systems: an exploratory studyof the dialectics of change. journal of management information systems, vol. 19, no. l, pp. 17-46. 26. wang, e.t.g., klein, g. and jiang, j.j (2006). erp misfit: country of origin and organizational factors. journal of management information systems, vol. 23, no. l, pp.263-292 27. tarantilis, c.d., kiranoudis, c.t. and theodorakopoulos, n.d. (2008). a web-based erp system for business services and supply chain management: application to real-world process scheduling. european journal of operational research vol. 187, iss. 3, pp.1310– 1326 28. hong, k-k. and kim, y-g.(2002). the critical success factors for erp implementation: an organizational fit perspective. information & management, vol. 40, pp. 25-40 29. amoako-gyampah, k. and salam, a.f. (2004). an extension of the technology acceptance model in an erp implementation environment. information & management, vol. 41, pp.731-745 30. davis, c. and hikmet, n. (2008). training as regulation and development: an exploration of the needs of enterprise systems users. information & management, vol 45, pp. 341–348 31. connell, n.a.d. and young, t.p. (2007). evaluating healthcare information systems through an ‘‘enterprise’’ perspective. information & management, vol. 44, pp. 433–440 32. bernroider, e.w.n. (2008). it governance for enterprise resource planning supported by the delone–mclean model of information systems success. information & management, vol. 45, pp.257–269 33. bendoly, e. (2003). theory and support for process frameworks of knowledge discovery and data mining from erp systems. information & management, vol. 40 pp. 639–647 34. bechor, t., neumann, s., zviran, m. and glezer, c. (2010). a contingency model for estimating success of strategic information systems planning. information & management, vol. 47 pp.17–29 35. aloini d., dulmin r. and mininno v. (2007). risk management in erp project introduction: review of the literature: information & management, vol. 44, iss. 6, pp. 547–567 36. goeke, r.j and faley, r.h. (2009). do sap successes outperform themselves and their competitors? communications of the acm vol. 52 no. 10 pp. 113-117 37. romero, j.a., menon n., banker r.d, and anderson m. (2010). erp: drilling for profit in the oil and gas industry: communications of the acm vol. 53 no. 7 pp. 118-121 australasian journal of information systems volume 18 number1 2013 24 38. ligh,t b. (2005). potential pitfalls in packaged software adoption: communications of the acm, vol. 48, no. 5 pp. 119-121 39. scott, j.e. and vessey i (2002). managing risks in enterprise systems implementations: communications of the acm, vol. 45, no. 4, pp74-81 40. behrens, s. (2009). shadow systems: the good, the bad and the ugly: communications of the acm, vol. 52 no. 2 p 124-129 41. fayad, m.e., hamu d.s. and brugali d. (2000). enterprise frameworks characteristics, criteria, and challenges: communications of the acm vol. 43, no. 10 p 39-46 42. hasselbring, (2000). information system integration: communications of the acm vol. 43, no. 6, pp. 33-38 43. ramanathan, j. (2005). fractal architecture for the adaptive complex enterprise: communications of the acm, vol. 48, no. 5 pp. 51-57 44. kremers, m. and van dissel, h. (2000). erp system migrations: communications of the acm, vol. 43, no. 4 pp. 53-56 45. scheer, w. and habermann f. (2000). making erp a success: communications of the acm, vol. 43, no. 4 pp. 57-68 46. martinsons, m.g. (2004). erp in china: one package, two profiles: communications of the acm, vol. 47, no. 7 pp. 65-68 47. akkermans, h.a., bogerd, p., yucesan, e. and van wassenhove, l.n. (2003). the impact of erp on supply chain management: exploratory findings from a european delphi study: european journal of operational research 146, pp. 284–301 48. ash, c.g. and burn j.m. (2003). a strategic framework for the management of erp enabled e-business change: european journal of operational research 146 pp. 374–387 49. su, y.f. and yang, c. (2010). why are enterprise resource planning systems indispensable to supply chain management? european journal of operational research 203 pp. 81–94 50. olhager, j. and selldin, e. (2003). enterprise resource planning survey of swedish manufacturing firms: european journal of operational research 146 pp. 365–373. 51. al-mashari, m., al-mudimigh, a. and zairi, m. (2003). enterprise resource planning: a taxonomy of critical factors: european journal of operational research 146 pp. 352–364 52. mandal, p. and gunasekaran, a. (2003). issues in implementing erp: a case study: european journal of operational research 146, pp. 274–283 53. abdinnour-helm s., lengnick-hall m.l and lengnick-hall c.a.(2003). pre-implementation attitudes and organizational readiness for implementing an enterprise resource planning system: european journal of operational research 146, pp, 258–273 54. umble e.j, haft r.r and umble m.m (2003). enterprise resource planning: implementation procedures and critical success factors: european journal of operational research 146, pp. 241–257 55. jacobs r and bendoly e (2003). enterprise resource planning: developments and directions for operations management research: european journal of operational research 146, pp. 233–240 56. mabert v.a. soni a. and venkataramanan m.a. (2003). enterprise resource planning: managing the implementation process: european journal of operational research 146, pp. 302–314 57. sundarraj r.p., talluri s (2003). a multi-period optimization model for the procurement of component-based enterprise information technologies: european journal of operational research 146, pp. 339–351 58. somers t.m. and nelson k.g. (2003). the impact of strategy and integration mechanisms on enterprise system value: empirical evidence from manufacturing firms european journal of operational research 146, pp 315–338. cinderella gets an invitation: recognising and organising systems maintenance & support c. james bacon. department of information systems university of canterbury. christchurch, new zealand jamesb@accy.canterbury.ac.nz abstract systems maintenance of existing systems is a 'cinderella' subject in the field of information systems & technology (1st). although it accounts for the majority of systems work, the subject is generally neglected and taken for granted. based on studies carried out in a large australasian organisation, evidence is presented for recognising the strategic role of systems maintenance, as a complement to new development. alternative means of organising systems development work is examined, with the conclusion that a separate systems support group for carrying out maintenance may have net advantage. a re-definition of maintenance, according to a taxonomy of eight different types, is proposed. effective methodology and metrics for maintenance are also addressed, together with the general phenomenon of "entropological deterioration', and 12 factors that need to be used in assessing existing systems. introduction the subject of systems or software maintenance is a " cinderella' in the field of information systems & technology (1st). in the academic environment, teaching is oriented almost exclusively to new systems development. what happens to those new systems after they are installed is of little concern. also, compared to the plethora of textbooks on new development, there would be few books suitable for a course on systems maintenance. similarly, it is a relatively neglected subject in the journal literature on 1st. in the business environment, managerial focus tends to be on new development, with little recognition of systems or software maintenance as a key activity. as for systems developers, maintenance tends to be something you do when you get the time, or delegate to trainees. it might even be contracted out; anything so that you don't have to do the "dirty work1 yourself. and software tools, notably case tools, concentrate on new development. thus, just like cinderella in the children's story, systems maintenance is generally neglected, abused and taken for granted. ' systems maintenance is the process of modifying existing, operational systems. it is referred to alternatively as software maintenance, but is herein referred to as systems maintenance. in most organisations (apart from vendors of systems software), it applies largely to application software rather than systems software, and it incorporates not only the software but the documentation and procedures, overall design, and hardware platform as well, i.e. the whole system. also, in terms of the organisational activity, there is some evidence that "maintenance" is now being replaced by the term "support" in referring to the activity of modifying systems. therefore, "systems support" may become the more prevalent term, rather than systems maintenance, at least in referring to the function that carries out systems maintenance. returning to the storybook parallel, there are a number of reasons why cinderella needs to receive an invitation to the party, and get the recognition she deserves from the people at the top. first, various estimates indicate that between 50 per cent and 80 per cent of systems work is taken up with systems maintenance and support (gibson and senn, 1989 and vail, 1991). systems maintenance therefore represents the majority of systems work. related to this point is that, as time goes on, systems are becoming bigger, growing at about ten per cent per year (harrison and cook, 1990). this growth may accelerate, to the extent that systems become oriented to graphics, and the needs of human-computer interface. as systems become bigger, they generally become more complex. consequently, effective systems support is becoming more critical, and a major requirement, in many organisations. a more fundamental reason why systems maintenance and support needs to receive better recognition and management is that, although they are not normally represented as such in annual reports, existing information systems represent a significant corporate asset. for most organisations, especially those of any size, day-to-day survival and excellent customer service can depend upon the 2 ajis organisation's information systems. therefore, somewhat like a building but more so, a businesss application system will quickly deteriorate and lose value if it is not properly maintained and managed. in 1992, the author was invited to undertake an organisational review of the systems development and maintenance activities in a large australasian organisation operating in the services sector. the basic need was to enhance the effectiveness of the organisation's 1st function. this was especially needful at a time when the business sector concerned was undergoing rapid change and an intensifying level of competition. the organisation therefore needed full and effective support from 1st. the review was an opportunity to study practice, and apply as well as validate relevant research findings. it represented a response to the call for greater research investigation in the field (computer science and technology board, 1990). in the event, it represented a case study in recognising and organising systems maintenance and support. the role of systems maintenance & support the basic role of systems maintenance & support is indicated in a proposed systems service-product grid, as shown in figure 1. this is based on the strategic grid (cash, mcfarlan and mckenney, 1988). high strategic impact of systems » j • . i%nfn* & support low service environment yesterday's environment intensely competitive environment new product environment low high figure 1. the system service-product grid where the strategic impact of systems maintenance and support is high, there is a service environment. that is, the organisation is not just dependent on the existing installed systems for its day to day operations and survival. it is also dependent on its users receiving effective service, in responding to their continually changing needs with those systems. thus, a more appropriate term for users, in this context, is customers or clients (the term "clients" denoting a somewhat closer business relationship than "customers"). this dependency is a characteristic of the situation where excellent service to the organisation's external customers is a critical success factor. for that service to be excellent, users/clients are dependent, directly or indirectly on the service they receive from the systems support function. implicit in the continually changing needs of users/clients is the need for continuous improvement. this is a fundamental concept in total quality management (tqm). thus, evolutionary or incremental improvement, in small steps, is important to systems quality, and a key to excellent service in systems support. in contrast, where there is a continual and strategic need for new systems development, there is a new (systems) product environment. this is likely to be the case where the business sector or type of business is obliged to engage in a continuous process of bringing out new or differentiated business products and/or services. it also occurs where "orders of magnitude' improvement in business functions are the focus, as in business process re-engineering. a third case is where the organisation needs to be at the leading edge', in bringing in new information systems to implement new information technology. if systems maintenance and new systems development are both strategically important, then the organisation is operating in an intensely competitive environment. however, where maintenance and new development both have a low impact on the organisation, it is evident that the organisation september 1993 is not utilising information systems & technology as a strategic resource, and is operating in 'yesterday's' environment. the implication is that both systems maintenance and new systems development need to be aligned with organisational goals. both activities need to be synchronised with the organisation's overall strategy (swanson and beath, 1989). therefore, the primary task of the organisation's 1st function may not be new development, nor systems maintenance. rather the concept of a dual task may be more appropriate. this implies that when efforts are directed at reducing the size and expense of the maintenance task, the motivation may be wrong, if it is to free up resources for new development. the implication in this case is that maintenance is inherently a secondary consideration, with limited strategic impact. the limited role in which maintenance is traditionally viewed is largely based on the traditional lifecycle, as represented in the "waterfall" model (boehm, 1981). that is, maintenance is "tacked on' at the end of new development, almost as an afterthought. the alternative view is that of maintenance integrated into the whole lifecycle. systems design includes an orientation to effective maintenance, and there is a methodology of/for maintenance. the alternative is also a wider view of existing systems as a portfolio of strategic assets. under this view, the systems support function has a key responsibility in maintaining and protecting the portfolio in good order (swanson and beath, ibid., and hayes, 1991). this is reported to be the view in "leading edge' organisations (layzell and macauley, 1990). in such organisations, the role of systems support is now recognised as being of key importance in the overall 1st strategy. it has been said that the major challenge for software engineering in the 1990s is systems maintenance (corbi, 1989). to the degree that this is true, systems maintenance & support needs to be properly recognised, properly organised, and properly managed. this applies not only to "leading edge' organisations, but any organisation that aims to use 1st effectively in an increasingly competitive environment. challenges involved in the common forms of systems organisation in the organisation that was the subject of the organisational review, the division of work among its systems staff was, first, according to business application area, and second, by type of work. staff specialised in specific business domains and, within these domains, there were analysts and programmers. this has been described as an application-work or a-w type of systems organisation, which is a common form of organising systems staff (swanson and beath, 1990). the "pure1 w type of organisation is the more traditional type of organisation, wherein programming expertise is required in third or second generation languages, and separated from analyst work. there is no specialisation by business domain. the contrary form of specialisation by business domain (type a), is more common in today's organisation, since it allows analyst-programmers (both types of work combined in a single type of position), to develop specialised application and business area knowledge, as well as closer relationships with a given user clientele. thus, in the organisation concerned, the application (a) and work (w) forms of organisation were combined in hybrid form, which is not unusual. a third form of organisation is the lifecycle or l type organisation (swanson and beath, ibid.). in this case, analyst-programmers specialise in either new development or systems support (maintenance). in order to enhance the effectiveness of the subject organisation's 1st function, it was recommended, and it was agreed after discussion, that it should to move to an l type of organisation, thus separating maintenance work from new development there were a number of reasons for this, based on the challenges found with present structures. one of the challenges faced was the inherent lack of flexibility in the a form. there were "peaks and valleys' in the need for new development in each business application area. systems development needs were "lumpy1. this meant that systems staff could be overworked in one period while waiting for work in another. it also meant that systems staff could be overworked in one application area while waiting for work in another. the overall business impact was that there was a frequent short-fall of resources needed to support business needs, together with user concerns about systems expense and lack of support from the 1st function. thus, there was a need for greater flexibility and response in covering the different application areas. another challenge faced by the organisation was that users/clients were not happy with the level of service they were receiving, on seemingly simple change requests, on current systems. in some cases, the response to such requests was so long in coming that the user/client had given up on and 4 x ajis forgotten the request, with the change having been superseded by others. thus, there was a need to enhance the level of service in response to change requests on current systems. generally, management of maintenance was limited. statistics were kept on the amount or "rework" or corrective maintenance being done. however, they had limited use, since there was no specific programme or response for preventive measures. maintenance was seen as being equivalent to re-work, made necessary by new development not being done properly. maintenance was therefore seen in very much of a negative light, as in many 1st environments. that is, there was a stigma attached to systems maintenance. the pros and cons of separating maintenance from new development owing to the negative view attached to systems maintenance in the subject organisation, at least according to the traditional view held at the time, there was resistance to the idea of separating it from new development no-one wanted to be assigned to a separate maintenance function, having to clean up * other people's mess1. the specific reasons initially given against the recommendation to separate maintenance from new development were as indicated in figure 2. these reasons constitute the * cons' of separating maintenance from new development. 1. separate maintainance staff may not understand a system and the effects of requested changes as well as those who developed it, so that systems quality may suffer. 2. separate maintenance people would be correcting other people's mistakes, and it would be better if people corrected their own. 3. response to users/clients might be slower, since separate maintenance staff may not be as familiar with the specific busines&area concerned, and wouldhave to study and be dependent on systems documentation more than those who developed a given system. 4. communication costs would be higher, since there would be a need for maintenance staff to refer to the new development staff concerned, and there would be duplicate interface with users/clients. figure 2. the cons of separating maintenance from new development there are advantages and disadvantages in most organisational forms. based on observation and study in the subject organisation, advantages or "pros' were advanced in favour of a separate systems support (maintenance) group, as indicated in figure 3. the organisation concerned did not produce any tangible product, but as a service oganisation was in the "information business'. it was a veritable "information factory'. its information systems therefore represented its "productive equipment' and the engine that "powered" its operations. the portfolio view of installed application systems was therefore of particular relevance to the organisation concerned, and this concept was emphasised in arguing the case for effective maintenance, and a separate maintenance function. however, in moving to a separate maintenance function, management felt it was important to distance itself as far as possible from the maintenance "stigma1, by avoiding the word "maintenance" in naming the new group. it was therefore given a more client-centred name, focusing on rapid service and quality support. a separate systems support group also addressed the need for greater flexibility in allocating new systems development staff across different business domains. essentially, it was recognised that systems staff needed to become cross-functional professionals, able to operate in different business application areas. in this way, project teams could be pulled together for a given new development project, which would mean more effective response to and support for the organisation's business needs. to make this happen, all staff were expected to do a "tour of duty1 in the systems support group, where they would be exposed to the maintenance needs of several business areas, thus obtaining "hands on' training across different application areas. it was important that such "hands on' experience in systems support then became a basis for further promotion. it was also important that systems support was be staffed by experienced people. it was not to be used as a training ground for beginners. implicit in this was the implication, and the "signal" from management, that maintenance was in no way secondary to new development work. september 1993 1. there is reduced risk exposure for the organisation, in terms of: (a) loss of knowledge when a person leaves, and (b) lack of availability in the event of absence due to various reasons, when at least two people (a new development person and a sytems support person), have knowledge about a given application area. 2. system documentation is forced to improve when people other than those who originally developed a system are maintaining it. 3. there is reduced interference with and distraction from new development, thus enhancing the prospect of meeting project schedule needs. 4. since effective maintenance is the focus in systems support, rather than new development, "quick and dirty' fixes, or "patches', tend to be avoided, and the design integrity and maintainability of installed systems is enhanced. 5. the principle of continuous, incremental improvement in the quality of existing systems, as per total quality management (tqm), receives enhanced focus. 6. ongoing response to users/clients becomes a primary focus, with positive implications for user/client satisfaction, since maintenance change requests are not seen as a distraction and a "nuisance" interfering with new development, but are instead the main job. 7. specific responsibility can be assigned for "custodianship1 of the organisation's installed systems, within a systems support group, in accordance with the asset/portfolio view. 8. effective management of systems maintenance becomes a specific responsibility, and effective methodology and metrics become more feasible. figure 3. the pros of separating maintenance from new development re-defining maintenance a fundamental reason for the commonly negative view of systems maintenance is its traditional orientation to corrective work, and this at the coding level rather than the design level. in the subject organisation, maintenance was generally known as "re-work", i.e. corrective maintenance. but this is only one kind of maintenance need and, from a strategic perspective, not the most important. three kinds of maintenance have been defined: corrective, adaptive and perfective (lientz and swanson, 1978). these three kinds have wide acceptance at the present time, in representing the totality of systems maintenance work. corrective maintenance is defined as that carried out to repair or correct processing performance and implemented failures; adaptive maintenance responds to required changes in the data and processing environments; perfective maintenance eliminates processing inefficiencies and enhances performance, (swanson and beath, 1989). but other types of maintenance have also been defined. for example, there is functional maintenance, which specifically refers to minor changes to or enhancements of system functions, (hall, 1987). two further kinds of maintenance that have been proposed are: (1) preventive maintenance, to improve future maintainability and reliability, and (2) deletion maintenance, to delete capabilities that are no longer needed (computer science and technology board, ibid.). thus, there have been at least three others added to the three kinds of maintenance commonly seen as constituting systems maintenance work. in addition, two further kinds or types of maintenance are herein proposed, namely mandated maintenance and post-implementation maintenance. the different kinds are indicated in figure 4, which represents a taxonomy of systems maintenance types. the taxonomy re-labels deletion maintenance as rationalisation maintenance, and includes definitional enhancement of the other types. it also distinguishes between discretionary and nondiscretionary maintenance. if systems maintenance work is to be properly recognised, organised and managed, the specific kinds of maintenance need to be identified and articulated. the essential reason for this is "granularity1, in which respect there is a parallel with project management. if a systems development project is not divided into discrete phases, activities and tasks, planning and control cannot be effective. also, metrics maintained on such projects are of limited value, since there can be no meaningful validation (jones, 1991). thus, effective management of systems maintenance needs to include, specifically, planning and control of the different kinds of maintenance required. distinction between the different kinds is also needed because the orientation and response to each needs to be different. 6 ajis non-discrctionarv maintenance 1. corrective maintenance: the repair or correction of design errors, including emergency changes 2. adaptive maintenance: modification made necessary by changes in the processing, data base or network environments. 3. mandated maintenance: modification made necessary by changes in the organisation's external environment, as in legislative and tax changes. discretionary maintenance 4. perfective maintenance: the enhancement of processing, data base, or network efficiency. 5. rationalisation maintenance: the deletion of superfluous output, input, programs, data, or redundant processing. 6. preventive maintenance: design and documentation changes for the purpose of improving future maintainability and/or reliability. 7. functional maintenance: minor additions to or enhancements of system functions, in response to evolving user/client needs. 8. post-implementation maintenance: the addition or enhancement of functions following system implementation, as agreed during the process of new development. figure 4, a taxonomy of systems maintenance types efforts to reduce the amount of maintenance work need to focus on non-discretionary maintenance, since this is the form of systems maintenance that the organisation is forced to do. in the case of mandated maintenance, for example, this could be combined with types of discretionary maintenance, so that the time and effort expended obtains some value. also, rationalisation maintenance reduces system size and complexity, by getting rid of ^petrified code1 (carlyle, 1989). consequently, it helps to reduce corrective maintenance, and makes other kinds of maintenance easier. preventive maintenance can have a similar effect on non-discretionary maintenance. discretionary maintenance is the form of systems maintenance which the organisation chooses to do, and for which, it needs to plan on a continuous basis. for example, if project target dates are to be met, it is necessary to to plan for post-implementation maintenance, as defined. otherwise, inclusion of functions that users/clients request after project schedules have been agreed can nave a negative impact in meeting such target dates. also, therjs needs to be planning for functional maintenance, particularly on those application systems which are central to business functions and activities. implementation of a taxonomy of systems maintenance types, as a chart of accounts in planning, recording and controlling maintenance work, will provide a clearer picture of maintenance, and facilitate effective management. for example, it will facilitate meaningful metrics, as in the proportion of resources consumed by discretionary as opposed to the non-discretionary form of maintenance, and in the different types of maintenance. the taxonomy may also help to remove the traditionally narrow view of systems maintenance. the taxonomy was therefore recommended, and agreed, as a means of improving overall management of systems maintenance in the subject organisation. it was screened and reviewed by experienced, senior-level analysts and programmers, and found to be useful as a means of approaching systems maintenance, and as a basis for providing overall measures for the systems support group. maintenance methodology and documentation under the commonly narrow view of systems maintenance, the term "maintenance methodology" may seem academic. under this view, you: (1) study the source code to understand the problem, (2) make the needed changes to the code, and (3) test the changes, which is hardly a methodology. this perspective and approach is unlikely to be associated with the portfolio view of existing, installed systems. september 1993 under the portfolio view, and as part of an overall maintenance methodology, it would be necessary to periodically assess each installed system, with respect to its strategic, functional and technical value (hayes, 1991). the assessment would need to include factors such as those listed in figure 5. the implication in all this is that there may be benefit in some form of impact analysis, prior to coding work. this involves an analysis of the effects of the prospective changes on overall system design. there are therefore a number of considerations that need to be incorporated in an effective maintenance methodology. these considerations were articulated and discussed in recommending certain enhancements in the approach to systems maintenance in the subject organisation. strategic factors 1. pertinence to and/or role in organisational plans. 2. effectiveness in supporting business critical success factors. 3. user/client satisfaction. 4. importance and value of the business function concerned. functional factors 5. the dependence of the business function on the system. 6. effectiveness in supporting the business function. 7. costs of operation and maintenance. 8. back-log of change requests. 9. utility and use made of respective screen and printed output. technical factors 10. maintainability (modularity, simplicity, consistency and self-descriptiveness). 11. reliability, (as per operations records/logs). 12. viability and vendor support. figure 5. assessment factors for an installed application system the assessment factors need to part of a maintenance methodology, and be included at the * front end'. but maintenance also begins early in new development rather than at its "back end' (pfleeger and boehner, 1990). thus, maintenance and the inevitability of future change need to be a consideration throughout new systems development that is, evolving user/client needs to be explicitly recognised. a further maintenance methodology consideration is that a system tends to lose its architectural integrity, if maintenance changes are at the code level only, with overall design excluded. this phenomenon might be referred to as entropological deterioration. fourth generation languages (4gls), and object orientation may help to reduce this. however, to the degree that they encourage a vlazy' development methodology, and prototyping as a substitute for (instead of a complement to) formalised methology, they may exacerbate the phenomenon. with this deterioration the system becomes more difficult to understand, in making changes to it. in the latter respect, it has been estimated that this traditional first step of understanding existing code and documentation consumes 47 to 62 per cent of prgrammer's/analyst's time (hayes, ibid.). as part of the recommended changes, the iterative or transformation approach to systems maintenance was recommended, for inclusion in a revised methodology, as an alternative to the extremes of the quick fix and total re-write. this is based on the idea of recovering design abstractions, in re-using design and code components from the existing system (arango et al, 1986, and harrison and cook, ibid.). thus, instead of starting with and focusing on source code, the approach starts with the design of the existing system, starting at the highest or overall level then working down, re-using existing design and code as much as possible. the iterative/transformation approach is not a re-write of the existing system, but it is more than the quick fix approach. it is a middle course between the two. the benefit is that it promotes continuous evolution of the system design, thus avoiding entropological deterioration, and making future maintenance easier. the result of these considerations was a recommended maintenance methodology for the subject organisation. the portfolio & design-based maintenance methodology implies that there are certain documentation needs. for example, at the very beginning there is a need to document pertinent aims, objectives and critical success factors, and to document the present system and/or procedures. there is also a need to document each maintenance change, or change release, including its 8 ajis justification, reason and purpose, some of which can be summarised in the source code itself. this type of documentation is often missing in the traditional approach to systems maintenance, which generally provides little more than the latest code (baxter, 1992). with this approach, there is a virtual need for the original developer(s) to continue personal involvement in the installed system. thus, effective maintenance documentation is complementary to effective maintenance methodology. as part of the enhanced maintenance methodology for the subject organisation, systems support staff were to be involved in new systems design. as 'custodians' of the portfolio of installed systems, they were to participate in structured walkthroughs, and in post-implementation maintenance. there was also a formalised maintenance handover phase, wherein systems support staff were to become familiar with the new or enhanced system, and obtain agreed documentation for ongoing maintenance. maintenance metrics it is difficult to know how well you are doing in a given area of activity, or the difficulty of what you are trying to do, without appropriate and meaningful measurement. the organisation and management of systems maintenance and support can only, therefore, be deemed effective and succesful to the extent that it is measured and validated. the need for measurement in the area of information systems & technology is directly related to its economic impact (jones, ibid.). the surging interest in software quality measures, project productivity, and 1st value-for-money indicators is therefore an indication, if any were needed, that 1st has become a strategic economic resource, and a vital area of activity for today's organisation. it is likely that such measures will become increasingly important. however, there are a number of imperatives in any maintenance measurement programme. first, it needs to be fully supported by senior management, because: (a) it involves cultural change and therefore needs authoritative endorsement, and (b) it requires resources, in a similar way to which cost and management accounting require resources. second, maintenance measurement requires learning and training. third, it needs to be positive rather than punitive. in the latter respect, people need to rewarded for finding errors, rather than being faulted for making them (robinson, 1993). the fourth imperative is, therefore, that effective measurement and targets need to be built into the reward system for management and staff. there are four general areas where metrics can be applied to maintenance. these are shown in figure 6. 1. taxonomical: measure and compare the different kinds of maintenance. 2. quality & maintainability: measure ease or difficulty of maintenance. 3. soft factors: measure human and environmental factors. 4. productivity: measure performance: figure 6. general areas of/for maintenance metrics the taxonomy and chart of accounts for the different types of maintenance work being done provides a means of comparing the proportion of maintenance effort being taken up by each type or each account. it also facilitates metrics specific to each type a second area for maintenance metrics is quality and maintainability, although this applies as much to new development as it does to maintenance. for maintenance, systems quality applies specifically to maintainability. the four attributes of systems software and documentation that optimise maintainability are consistency, simplicity, modularity and self-descriptiveness. consistency refers to the uniformity and standardisation of design, notation and techniques. simplicity refers to ease of understanding and avoidance of complexity. modularity refers to a design structure of independent modules. self-descriptiveness refers to the degree to which the software and documentation provide explanation of functions. a third area is that of "soft1 factors, which involves measurement of: such things as: (1) the morale of systems support staff, (2) the level of service perceived by clients/users, (3) the cooperation and assistance received from clients/users, (4) the skill, experience and training needs of systems support staff, and (5) environmental assistance/constraints, as in the provision of technical and/or physical resources. september 1993 a fourth area is that of productivity. however, although metrics have been developed for measuring new development, there has been very little research in applying productivity metrics to systems maintenance. function point analysis (fpa), was originally developed for new systems development (albrecht, 1984). with the increasingly strategic importance of 1st, and the need for optimising systems development productivity in supporting business needs, fpa is enjoying increased usage. there is even an active international function point users group (ifpug). fpa appears to hold the greatest promise for measuring maintenance productivity. it has had some application, specifically to three kinds of maintenance defined in the taxonomy, namely perfective, functional, and post-implementation maintenance (cote and st-pierre, 1990). from an overall perspective, metrics for maintenance need to focus on those areas that represent or include critical success factors (csfs) for the systems support group or maintenance activity (as well as for the 1st function as a whole). for example, if user/client satisfaction is a csf, then this needs to be regularly measured, likewise "turnaround time' for maintenance changes. in the organisation concerned, metrics were being maintained on the amount of corrective maintenance being performed, and it was recommended that the area(s) of measurement be enlarged, particularly in the areas of maintainability and soft factors. this was agreed, and it was accepted that the systems support group depended, for its effective management, upon the development of a metrics programme suitable to the organisation. optimising user/client service a fundamental aim and csf for a systems support group is user/client satisfaction. this will depend to a large extent on the level of service and support received and perceived by the user/client. "perceived" is the operative word, because systems staff may be exerting great effort to provide good service and support to users/clients, but still the perception could be that of poor service. two basic reasons for this are: (1) lack of user/client awareness, and (2) 'red tape'. to become aware, users first need to be involved in the maintenance methodology, and be part of if not leading the assessment of existing systems. second, users need to be involved in the planning and prioritisation decisions. third, users need to be educated, so that they can participate as partners with systems management and staff in these decisions. a fourth area of needed user awareness is that of change request status. users/clients need to know whether and when a given change request is scheduled for implementation, its expected release date, and its progress (in the event of enquiry), as it progresses toward implementation. with respect to "red tape', the maintenance methodology must not become so bound up that it obfuscates and inhibits communication between users and systems staff. where necessary, communication needs to be direct, as between the initiator of a change request and the systems support group. both awareness and "red tape' needed to be addressed in the organisation concerned, and recommendations were made and agreed in these areas. the aim was an enhanced level of service and support to users/clients, both in reality and in the perception of users/clients. managing and motivating change where an organisation is oriented to the traditional view of systems maintenance, a process of cultural change is required, in moving to a strategic asset, portfolio-based view of systems maintenance and support. this involves the management of change, both organisational and behavioral. the management of change needs to incorporate two basic principles: (1) the participation of those affected by or involved in the change, and (2) an orientation to identifiable results (mankin et al, 1988, and schaffer and thomson, 1992). the principle of participation was embraced in a wide consultation process during the study, involving all levels in developing the recommendations. in addition, the very first recommendation embraced both principles. it recommended a process whereby the recommendations were divided up and allocated among teams of systems staff, with each team being asked to assess the following, for each recommendation allocated to the group: (1) whether the recommendation should be accepted or rejected, (2) whether and where it needed to be refined, modified or improved, (3) whether it needed to be replaced by a better alternative, as specified and justified, (4) other things that needed to be done or considered for effective implementation, (5) additional or re-oriented resources that might be needed for effective 10 ajis implementation, (6) specific measures or criteria that could be used in determining whether implementation was proceeding successfully, and (7) priority for implementation. the teams were asked to submit a one-page report on each recommendation. management also decided to establish an overall change process team, representing different levels and areas, to oversee the change process. conclusion the subject of systems maintenance and support is potentially a large subject area. to date, it has been treated as a x cinderella' subject. application systems are growing, becoming more complex, and increasingly represent a strategic asset for most organisations. but poorly maintained systems may represent a "time bomb' for many organisations. thus, systems maintenance needs to receive a higher profile, and considered complementary rather than secondary to new systems development. this paper focuses on organising and managing systems maintenance and support, and directs itself to the main areas involved. not more than a handful of researchers have focused on systems maintenance, and this paper owes a debt to them since it builds on their work. but, given the increasing significance of maintenance, there is scope for further research; a lot more work needs to be done. references albrech, alan j. (1984) productivity measurement and estimate validation, purchase, ny: ibm corporate systems & administration. arango, g., baxter, i., freeman, p., and pidgeon, c. (1986) "tmm: software maintenance by transformation", ieee software , may, vol. 3, no. 3, pp 27-39. baxter, ira d. (1992) "design maintenance systems", communications of the acm, april, vol. 35. no. 4, pp 73-81." boehm, barry w. (1981) software engineering economics, englewood cliffs nj: prentice-hall. carlyle, ralph e. (1989) "fighting corporate amnesia" datamation, 1 feb., vol. 35, pp 41-44. cash, j.i., mcfarlan, f.w. and mckenney, j.l. (1988) corporate information systems management: the issues facing senior management homewood, illinois: dow jonesirwin. computer science and technology board (1990) "scaling up: a research agenda for software engineering, communications of the acm, march, vol. 33, no. 3, pp 281-293. corbi, t. (1989) "program understanding: challenge for the 1990s" ibm systems journal, .vol. 28, no. 2, pp 294-306. cote, vianney and st-pierre, denise (1990) "a model for estimating perfective software maintenance", conference on software maintenance 1990. pp 328-334. los alamitos, ca: ieee computer society press. gibson, virginia r. and senn, james a. (1989) "systems structure and software maintenance" communications of the acm march, vol. 32, no. 3, pp 347-358. hall, rand p. (1987) "seven ways to cut software maintenance costs" datamation, 15 july, pp 81-84. harrison, warren, and cook, curtis. (1990) "insights on improving the maintenance process through software measurement" conference on software maintenance 1990, pp 37-45. los alamitos, ca: ieee computer society press hayes, ian (1991) "software as assets increases returns", computerworld new zealand, 23 sep, pp 13-15. jones, capers (1991) applied software measurement, ny: mcgraw-hill layzell, p.j. and macaulay, l (1990) "an investigation into software maintenance-perceptions and practices", conference on software maintenance 1990, pp 130-140. los alamitos, ca: ieee computer society press lientz, b.p. and swanson, e.b. (1978) "characteristics of software maintenance", communications of the acm, june, vol. 6.-481 pp 466.. 21, no mankin, d., bikson, t., gutek, b. and stasz, c.(1988) "managing technological change: the process is key", datamation, 15 sep. pp 69-75. september 1993 11 pfleeger, s. and bohner, s (1990) "a framework for software maintenance metrics" conference on software maintenance 1990, pp 320-327. los alamitos, ca: ieee computer society press robinson, jeffery a. (1993) "the dark side of software metrics", information strategy: the executives journal, winter, pp 44-47. schaffer, rober h. (1992) "successful change programs begin with results", harvard business review, jan-feb., pp 80-89. swanson, e. burton and beath, c. (1989) maintaining systems in organisations, ny: john wiley and sons swanson, e. burton and beath, c. (1990) "departmentalisation in software development & maintenance, communications of the acm, june, vol. 33, no. 6, pp 658-667. vail, simon (1991) "happiness is a fault-free system", management today, nov. pp 115-116. 12 ajis information systems contribution to business performance: a study of information systems executives' attitudes dieter fink the australian national university canberra act 0200 australia falantina tjarka edith cowan university pearson street churchlandswa6018 australia abstract a shift is occurring in the information systems (is) paradigm towards integrating is activities with business activities. to investigate this development further, the study examined business performance measurement frameworks, and adopted these in compiling is effectiveness measures relevant to the business environment. the is executives of australia's 200 largest companies were surveyed to establish how important these is effectiveness measures and the traditional is efficiency measures are in reflecting is performance. it was found that is executives continue to focus on is efficiency and rate internal-oriented is effectiveness significantly higher than businessand finance-oriented is effectiveness. the findings appear to indicate that is executives at least in australia are technology-oriented rather than business-performance-oriented in their attitudes to is performance. in contrast, the finding that qualitative is measures were regarded significantly more important than quantitative is measures may indicate that they believe that many benefits of is are intangible and strategic. introduction little agreement exists on measures of information systems (is) performance. the contribution of information systems has, in the past, been generally measured in terms of efficiency ("doing things right"). more recently, attention is being given to measuring information systems effectiveness ("doing the right things"). there is a growing expectation that information systems need to make a contribution to the overall performance of the business ("improving the bottom line"). a study was conducted with the aim of identifying the role of is in today's business environment, in particular the contribution is makes to overall business performance. suitable is performance measures were established, followed by the conduct of a survey to establish australian is executives' attitudes to the importance of such measures. the article discusses the research conduct and findings. it commences with a literature review of the topic. information systems and business performance recently a shift in the is paradigm has occurred with significant implications for the measurement of is effectiveness. this is reflected in farwell et al.'s (1992) description of two is worlds: the new and the old. in the old is world, it was assumed that information systems are developed and directly controlled by is professionals; specialised technical knowledge was needed which only the is professionals possess. in the new is world, there are changes in the business computing environment with users having more direct control over is applications, and demanding is support and services as opposed to is products. this paradigm shift implies that there is a need for the is executive to have an "integrative perspective on corporate computing and the management of corporate information resources" (farwell et al., 1992, p 9). according to symons (1991), the evaluation of is effectiveness in today's business environment requires the consideration of two separate but related areas: the alignment of is with business strategy, and the contribution of is to organisational effectiveness. various attempts have been made to provide frameworks to integrate is with business (a good overview is provided by baets, 1992). the key problem remains in that the success of the integration process is dependent on the is analyst's experience and knowledge of the it/is and his/her ability to infer the is objectives, characteristics, and design principles from the organisation's mission, objectives, strategies, and strategic attributes. in order to assess the contribution of is to organisational effectiveness, it is necessary to conceptualise it in terms of implementation issues which include specifications of requirements, assessment of costs and benefits, managing processes of change, offering organisational support and conflict management. both the linkage of is to business goals and the consideration of the implementation process aim to integrate it with the rest of the business. september 1994 29 recent attempts to measure business performance provide potentially promising new frameworks to measuring is effectiveness in business terms. the business value of is may thus be derived, which according to rouse (1991) is usually measured by the changes in indicators such as profitability, market share and market size. four proposals have been identified for contemporary business performance measurement frameworks. kaplan and norton's (1992) balanced scorecard approach offers a balance of financial and operating measures for establishing business performance. financial measures reflect what has taken place. non-financial measures are the drivers of future financial performance and include operational data on customer satisfaction, internal processes, and the organisation's innovation and improvement activities. goals from each perspective are specified and appropriate measures are identified. it is claimed that this approach integrates diverse, complex information in an easy-to-read manner, presenting those measures in each category that management wants emphasised. rubin (1991a, 1991b) introduced the business value framework where is performance, in terms of its business contribution, is arrived at in three stages. in the first stage, key measures for evaluating technical and software processes are defined in terms of quality, productivity and impact on customer satisfaction. in the second stage, the key technical indicators are linked to business performance. for example, the is organisation should be able to make the following assertion about its performance in business terms: "if we show a productivity increase of n% this year, the business will be able to lower product costs by y% or produce z new products". in the third stage, is can directly express changes in its performance in terms of such key measures as cycle time, quality, profitability, shareholder value, process improvement and yield. in the enterprise-level measurement by berger (1988) (also cited in belitsos, 1988), it is not the sole responsibility of one department (the is department). other departments, such as engineering, sales and customer service, should be able to buy and operate their own it/is. the approach firstly requires the determination of the enterprise's business objectives and goals. secondly, the decision is made whether or not it is needed to accomplish these objectives and goals and, if so, what strategy is needed. the effectiveness of it is measured through business actions that will result in increased market share, new market penetration, and lower product costs. according to strassmann (1990), understanding how successfully an organisation uses its resources and measures its success are prerequisites for analysing the effectiveness of it. specifically, his concept of return on management (rom) measures performance based on the value provided by management to an organisation. the approach assumes that in the modern organisation, the cost of managing the enterprise is the cost of providing information. improvements in the management value-added (or rom) can be realised by investing in appropriate it which enables management, inter alia, to evaluate effectively the competitive environment, to develop business strategies, and to motivate staff. research design as seen above, business-performance models provide a wide range of is measurement criteria to select from. to assist in the compilation of the research material, the expert opinions of four is professionals were solicited through discussion and involvement in the pilot testing of the material. the appendix shows the is performance measurement items that were compiled and their categorisation under fourteen headings whilst table 1 provides an overview of this breakdown. table 1 categorisation of is performance measures measure throughput by is utilisation of is cost of is programming by is availability of is timeliness of is accuracy of information quality provided by is return on is increase attributed to is comparisons time to develop is is personnel is enables efficie effecti intern ncy v'ss al x x x x x x x x x x x x x x x x x busine financ ss e x x x x x x x x x x quanti tat'e x x x x x x x x x qualit ative x x x the measures will be discussed below together with the research hypotheses. 30 ajis research hypotheses as stated in the introductory section, is have in the past been measured in terms of efficiency which focuses on is performance along the lines of throughput, utilisation, costs, and programming. however, since attention is increasingly being given to effectiveness, it was hypothesised that australian is executives rate the importance of is higher in terms of effectiveness than in terms of efficiency. accordingly, hypothesis 1 was formulated. hypothesis i: is executives place a greater importance on is effectiveness than on is efficiency. to gain further insight into the importance of is effectiveness measures, we categorised them according to the organisational activities they support. first, we distinguished between internaland business-oriented items. second, the items were also classified as being eithei quantitativeor qualitative-oriented. third, within business-oriented items, those measuring financial performance were identified as being finance-oriented. these groupings reflect the broad approaches to measuring business performance discussed in the earlier section and are briefly defined below. internaland business-oriented measures: the former measure the contribution that is makes within the organisation and include the availability, timeliness and accuracy of information produced. the latter measure the business contribution of is, for example the quality provided to products and services, financial returns and is enabling characteristics. quantitativeand qualitative-oriented measures: a distinction was made according to whether or not the measure could be quantified. examples of quantifiable measures are the time availability and financial returns of is, whilst qualitatively-oriented measures include improved communications and better decision-making. finance-oriented measures: they comprise measures that can be expressed in financial terms, namely the returns on, and increases attributable to, is. in line with our expectation that businesses, and therefore is executives, increasingly adopt a business, financial and qualitative focus towards is we framed the following hypotheses: hypothesis 2: is executives place a greater importance on business-oriented than on internal-oriented is effectiveness. hypothesis 3: is executives place a greater importance on finance-oriented than on internal-oriented is effectiveness. hypothesis 4: is executives place a greater importance on qualitative measures than on quantitative measures of is effectiveness. as discussed earlier, a shift in the is paradigm is occurring in which is is managed in a decentralised way and is integrated with business activities. we expected therefore that is effectiveness would be more important to decentralised is organisations than to centralised ones. this led to the following hypothesis. hypothesis 5: is executives in decentralised is management structures place a greater importance on is effectiveness than is executives in centralised is management structures. task data was collected through questionnaires. the three sections of the questionnaire captured information on the organisations' information systems, the responses to the importance of is performance measures, and demographic data of respondents. participants were asked to consider the various measures and indicate how important they regarded each was in reflecting it/is performance. a 7-point category-numerical scale was used because the sensitivity of a scale is important when changes in attitudes or other hypothetical constructs are under investigation (zikmund, 1988). this was the need in this study since the views of is executives in considering the relative importance of newer forms of is effectiveness were determined. september 1994 31 subjects the sample included australia's top 200 companies by turnover. it was argued that such organisations would likely be large enough to be concerned whether or not their it/is products and services are regarded as satisfactory by the rest of the organisation. the sample was selected from the listing of companies in the database available from the australian stock exchange (1993). the subjects of the study were the is executives in these organisations. the sample reduced from 200 to 194 because six organisations either declined to participate or the parent company was based overseas. forty-three useable responses were eventually received after one reminder letter was mailed. this gave a response rate of 22%. validity and reliability content validity of the questionnaire was assumed on the bases that an exhaustive search of the literature for items to be included in the questionnaire had been conducted. construct validity1 was achieved by soliciting the expert opinions of four is professionals through discussion and involvement in the pilot testing of the questionnaire. reliability was examined in terms of internal consistency of the questionnaire responses, i.e. the extent to which all of the subparts of an instrument or scale measure the same characteristics. this was established through the cronbachalpha technique; a technique where the mean reliability coefficient estimates for all possible ways of splitting a set of items in half are computed. table 2 shows the cronbach's alpha value for each group of is performance measurement in the questionnaire. the largest number of responses were from manufacturing companies (27.9%), followed by mining with 20.9% of responses. to test the representativeness of the sample, the industry profile of the respondents was compared with the same profile of australia's top 200 organisations. a chi-squared goodness of fit test (x2 = 6.572, p = 0.765) showed that there was no significant difference in the frequency distribution of industries represented in the sample and the underlying population. table 2 cronbach's alpha value for each group of is performance measure alpha throughput by is 0.632 utilisation of is 0.568 cost of is 0.822 programming by is 0.655 availability of is 0.704 timeliness of is 0.503 accuracy of information 0.857 quality provided by is 0.944 return on is 0.840 increase attributed to is 0.874 comparisons 0.738 time to develop is 0.830 is personnel 0.916 is enables 0.901 construct validity can also be established through f a c t o r a n a l y s i s ( d a v i s and consenza, 1988). in our study, at a cut-off level of 0.30, the fourteen categories (factors) that were used to divide the measures in the questionnaire accounted for 75.3% of the total variance of the original measure as explained by each factor. the low number of responses, however, raises concern over the use of this statistic. 32 aj1s data analysis demographics the industry distribution of the sample and population organisations is shown in table 3. table 3 distribution of the population and sample by industry industry manufacturers mining wholesale/retail trade finance, insurance and business services construction entertainment and personal service electricity, gas and water transport, storage and communication publishing agriculture, forestry, fishing and hunting total population n % 66 41 36 21 10 6 1 9 6 3 200 33.0 20.5 18.0 10.5 5.0 3.0 0.5 4.5 3.0 1.5 100.0 sample n % 12 9 6 6 5 2 1 1 1 0 43 27.9 20.9 14.0 14.0 11.9 4.7 2.3 2.3 2.3 0 100.0 the questionnaires were mailed to the information systems executives of australia's largest companies. of the respondents, 88% were above 30 years old, and 46% were older than 40 years. however, most (67%) had been in their position for 5 years or less and only 5% for more than 10 years. their is experience was mostly above 10 years (67%), while 28% had more than 20 years experience. with respect to the is management structure, 58% of organisations used a centralised structure, 37% a decentralised one, and the remaining 5% indicated that they were employing outside contractors and consultants. hypotheses testing histograms produced showed that research variables followed a normal probability distribution. since the sample is also representative of the population (see table 2), the parametric t-test and analysis of variance were used to test the hypotheses. the outcomes of the statistical analysis are provided in tables 4 and 5 below. table 4 t-tests of efficiency and effectiveness measures measure efficiency effectiveness efficiency measures effectiveness measures effectives : internal business internal-oriented measures business-oriented measures effectiveness: internal finance internal-oriented measures finance-oriented measures effectiveness: quantitative qualitative quantitative-oriented measures qualitative-oriented measures mean 4.72 4.86 5.46 4.65 5.46 4.25 4.78 5.06 st. dev. 0.82 0.77 0.82 0.87 0.82 0.97 0.78 0.93 p-value 0.211 0.000 0.000 0.004 t-value 1.269 6. 145 8.083 3.090 table 4 shows that whilst there was no significant difference in the importance attributed by is executives to the overall measures of is efficiency and is effectiveness, difference existed when effectiveness measures were further categorised. internal-oriented effectiveness measures were considered more important than both business-oriented and september 1994 33 finance-oriented effectiveness measures, and is qualitative-oriented measures were considered more important than is quantitative-oriented measures. as a result hypothesis 1, 2, and 3 have to be rejected and only hypotheses 4 can be accepted. the is management structure did not affect the overall is effectiveness measure, according to the analysis of variance contained in table 5 below. when t-tests were carried out on the various is effectiveness measures (for example internal-oriented versus businessoriented) by is management structure, there was a significant difference between qualitative-oriented and quantitative-oriented measures in centralised is structures (t = 3.222, p = 0.004) but not in decentralised ones (t = 0.989, p = 0.338). hypothesis 5, therefore, can only partially be accepted. the findings are discussed below. table 5 anova of is effectiveness measures by is management structure source is management structure error total sum of squares 2.707 22.798 25.505 degrees of freedom 2 40 42 mean square 1.353 0.570 fvalue 2.375 pvalue 0.106 discussion our study attempted to establish the attitudes of is managers to measuring the contribution of information systems by using business performance measurement frameworks. below follow a discussion of the findings, the study's limitations and further research required. findings is executives in australia's largest organisations still focus heavily on is efficiency and on internal measures of is effectiveness. according to our sample, nearly half the is managers are older than 40 years and 28% have is experience in excess of 20 years. their backgrounds may therefore have had an influence on their attitudes to is business performance. they could be following the behaviour of their colleagues in northern america reported in a recent study by bryce (1992). the comparison of american with japanese companies found that american managers placed importance on programming technology and tools whereas their japanese counterparts addressed management issues. it was reported that most is managers in the usa graduated from the ranks of programming and do not entirely understand or appreciate management issues, hence their continuing focus on efficiency. australian is managers may be similar in career backgrounds and appear to possess the same technical inclination. this requires further study (see next section). our study showed that is internal-oriented measures play a significantly more important part in reflecting is performance than businessand finance-oriented measures. the latter may present unfamiliar concepts to many australian is executives as it appears to be the case in northern america. according to a study of chief information officers (cios) and chief executive officers (ceos) in the usa and canada by plewa and lyman (1992), the ceo was found to consistently focus on such outside factors as market share and customer satisfaction, whereas the cio measured the success of is performance by internal measures. when asked to report their department's progress, the cio usually discussed the following: systems and network uptime, reports delivered on time, number of errors, number of aborts, and control over expenses. it appears that the cio is still focussing on his/her department's performance rather than on the performance of the organisation. concerns about the business proficiency of australian is managers have been raised in previous research. broadbent et al. (1992) found that australian organisations had difficulty in lining up business and is strategies because of the skills gap on the part of the is managers. more specifically, hunt and dampney (1990) found that australian is managers have to possess a general business background, and that in the 1990s is were too important to be managed by "edp" experts, a term used for the older, more technology-oriented generation of managers. qualitative measures were regarded as more important than quantitative measures, particularly in centrally is managed organisations. possible reasons can be found in other studies. according to katz (1993), whilst most senior executives believe that the benefits of is expenditures are quantifiable and measurable in some way, they tend to offer only vague and general guidelines when asked what quantifiable measures are being used. kaplan (1986) and willcocks (1992) argue that many of the benefits of is (such as quality, flexibility, responsiveness, and functional integration) are intangible and, therefore, the measures for these benefits cannot be quantified and have to be 34 ajis qualitative. willcocks (1992) goes further in that he believes that many is/it investments are justified by faith alone and notional figures are used instead of rigorous methods to calculate the benefits of it/is investments. more positive views are held by singleton et al. (1988). they suggest that measures of is effectiveness tend to become more qualitative and less quantitative as the focus moves from operational to managerial to strategic concerns. hence, this could indicate that australian is executives, despite their internal focus, may be adopting, or be involved in, strategic approaches to the use of it/is. this presents an opportunity for further research as outlined below. study limitations and further research our study was subject to limitations because of the new-ground-breaking nature of the research. first, it was innovative in adopting elements from different business performance measurement frameworks to construct indicators of is effectiveness. the questionnaire design and responses, therefore, need to be examined rigorously in terms of validity and reliability. whilst procedural design requirements were met through a thorough literature review and the involvement of is professionals in constructing and pilot testing the questionnaire, some concern is reflected in the internal groupings with cronbach's alpha values below 0.80, the generally regarded satisfactory level (sekaran, 1984) (see table 2). however, it was reasoned that these values were acceptable in view of the newness of the research topic. the relatively low response rate of 22% may be a concern but was very similar to the response rate in the well-know australian is key issues study by watson (1989). the population, being australia's top 200 companies, are frequently approached by researchers and they may have become selective by way of response. furthermore, it was learned during the study that some companies have a policy of not responding to surveys. general weaknesses of a survey design approach obviously exist, such that it only collects self reports, recall may be selective, and respondent may not be wil l ing to express attitudes or beliefs on sensitive topics. the study acknowledges that "it value research is still in its adolescence" (kauffman & weill, 1989, cited in sethi et al., 1993) and considerable additional research is required. as a first step, the study explored the attitudes of is executives to a range of business performance measures. as the next step, it recommends that case studies/action research be carried out to capture the complexities and richness of organisational behaviour towards measuring is contribution to business performance. concluding comment is too much expected of is executives? can they be talented business people working as equal partners with other executives and managers of business units to design systems that will benefit the organisation? according to the findings of waldman (1992), this is an unrealistic expectation in the real world where generally 80% of is staff are technology-oriented and only 20% are business-oriented. it could be argued, therefore, that is should be judged by how effectively it builds the system that the business side has decided on and not on the benefits that its systems return to the organisation. on the other hand, in today's economic climate, it is reasonable to expect that is executives investigate and consider adopting business-performance-oriented is approaches. their focus should be on the organisation as a whole, and not simply on the is department. to achieve this, they need to increase their knowledge of business management. with the changing structures of is management from centralised to decentralised and dispersed, and the growth in importance of aligning is with business activities, the is executive should become a contributor to the organisation's "bottom line". references australian stocks exchange (1993) cd rom data disc of companies for the month of may 1993, release 3.14. baets, w. (1992) "aligning information systems with business strategy", the journal of strategic information systems, 1(4), pp 205-214. belitsos, b. (1988) "can we measure what we do", computer decisions, august, pp 60-63. berger, p. (1988) "selecting enterprise-level measures of it value", in berger, p., kobielus, j.g. and sutherland, d.e. (ed.), measuring business value of information technologies, icit press. broadbent, m., hansell, a., lloyd, p. & dampney, c.n.g. (1992) "managing information systems in australia and new zealand: requirements for the 1990s", the australian computer journal, 24(1), pp 1-11. bryce, m. (1992) "focus on effectiveness", information systems management 9(4), pp 70-72. davis, d. & consenza, r.m. (1988) business research for decision making, pws-kent, boston. farwell, d.w., kuramoto, l., lee, d., trauth, e.m. & winslow, c. (1992) "a new paradigm for is: the educational implications", information systems management, 9(2), pp 7-14. september 1994 35 hunt, r & dampney, c.n.g. (1990) "future management information systems: a delphi study of australian industry", referenced in broadbent, m, hansel), a., lloyd, p. and dampney, c.n.g. (1992) "managing information systems in australia and new zealand: requirements for the 1990s", the australian computer journal, 24(1), pp 1-11. kaplan, r.s. (1986) "must cim be justified by faith alone?", harvard business review, 64, pp 87-95. kaplan, r.s. & norton, d.p. (1992), "the balanced scorecard measures that drive performance", harvard business review, january-february, pp 71-79. katz, a.i. (1993) "measuring technology's business value", information systems management, 10(1), pp 33*39. kauffman, r.j. & weill, p. (1989) "an evaluative framework for research on the performance effects of information technology investment", proceedings of the 10th international conference on information systems, boston, pp 377-388. plewa, j.m. & lyman, h.t. (1992) "keeping the customer satisfied: the new cio mandate", information systems management, 9(1), pp 54-61. rouse, d.r. (1991) "measuring the business value of information technology investments", proceedings share europe anniversary meeting, 2, pp 239-245. rubin, h. (1991a) "measurement: shifting the focus to business value", capacity management review, 19(1), pp 1-3. rubin, h. (1991b) "measure for measure", computerworld, april 15, pp 77-78. sekaran, u. (1984) research methods for managers a skill-building approach, wiley and sons. sethi, v., hwang, k.t. & pegels, c. (1993) "information technology and organisational performance: a critical evaluation of computerworld's index of information systems effectiveness", information & management, 25(4), pp 193-205. singleton, j.p., mclean, e.r. & altman, e.n. (1988) "measuring information systems performance: experience with the management by results system at the security pacific bank", mis quarterly, 12(2), pp 325-337. strassmann, p. (1990) the business value of computers, the information economics press. symons, v. (1991) "a review of information systems evaluation: content, context and process", european journal of information systems, 1(3), pp 205-212. waldman, r.a. (1992) "a misguided effort: trying to evaluate the bottom-line contribution of is is a waste of time", cio, july, p 22. watson, r.t. (1989) "key issues in information systems management: an australian perspective", the australian computer society, 21(2), pp 118-129. willcocks, l. (1992) "evaluating information technology investments: research findings and reappraisal", journal of information systems, 2(4) pp 243-268. zikmund, w.g. (1988) business research methods, the dryden press. 36 ajis appendix mean importance ratings for is efficiency measures measure throughput hardware software is personnel utilisation hardware software is personnel cost hardware software is personnel programming lines of code delivered function points overall mean mean 4.41 4.44 5.16 4.72 4.79 5.16 5.00 5.27 5.11 4.16 3.76 4.72 september 1994 37 mean importance ratings for is effectiveness measures measure availability hardware software is personnel timeliness hardware software is personnel accuracy of information produced by hardware by software by is personnel quality provided by is functional requirements met user/customer complaints improved service level overall user/customer satisfaction user/customer perception of ease of use meeting critical success factors returns on is investment on equity attributable to is on assets attributable to is on management (value-added) is yield overall cost reduction attributable to is increase attributable to is in net earnings per share in net income in profit margin in market share in sales comparisons industry: is budget as % of revenue % of is resources applied to strategic business time to develop new is applications adopt new is methodologies is personnel education/training morale level agreement with is strategic directions is enables improved communications better decision-making expanded access to information enhanced reporting capabilities overall mean mean 5.79 5.67 4.90 5.51 6.00 5.30 5.37 5.65 5.00 5.30 5.07 5.20 5.48 5.20 5.16 4.93 4.32 4.11 4.72 4.48 5.20 3.83 2.74 4.11 4.25 4.20 4.09 4.14 5.04 4.25 4.69 4.90 4.76 4.81 5.18 5.00 5.04 4.86 int. bus. fin. 5.79 5.67 4.90 5.51 6.00 5.30 5.37 5.65 5.00 5.30 5.07 5.20 5.48 5.20 5.16 4.93 4.93 4.32 4.32 4.11 4.11 4.72 4.72 4.48 4.48 5.20 5.20 3.83 3.83 2.74 2.74 4.11 4.11 4.25 4.25 4.20 4.20 4.09 4.09 4.14 5.04 4.25 4.69 4.90 4.76 4.81 5.18 5.00 5.04 5.46 4.65 4.25 q'nt. q'lt. 5.79 5.67 4.90 5.51 6.00 5.30 5.37 5.65 5.00 5.30 5.07 5.20 . 5.48 5.20 5.16 4.93 4.32 4.11 4.72 4.48 5.20 3.83 2.74 4.11 4.25 4.20 4.09 4.14 5.04 4.25 4.69 4.90 4.76 4.81 5.18 5.00 5.04 4.78 5.06 38 ajis microsoft word ajis special 2002.doc ajis special issue december 2002 86 a theoretical basis for is? the contribution of ant jim underwood university of technology sydney, australia jim@it.uts.edu.au abstract representation is a key issue of is design and operation that is often ignored. actor-network theory (ant), a semiotic theory of stakeholders, provides a way of dealing with representation. combining aspects of ant and foucault's discourse theory allows us to include concepts as actors and promises a flexible and durable foundation for is practice, but ant itself indicates that the search for a purely theoretical foundation for is is misguided. introduction an information system is a system – an interaction of people, devices and procedures; a technology – for providing information: news, facts which are useful and previously unknown, experiences that change our view of the world (avison and fitzgerald, 1995; boulding, 1961). such systems usually operate by acquiring information form other sources such as people, documents, sensing devices, traces of events, and representing it as data (possibly highly structured) and making these data available to potential users (interpreters) by either sending messages or storing the data for later retrieval. many introductory texts (e.g. oz, 2000) deal with the storage and transmission of data, the uses to which they might be put and sometimes the way in which data may be collected. representation and interpretation are seldom treated comprehensively in information systems (is) texts, except technically as coding and decoding. in this paper i will argue that representation is a key issue in is design and use, and that semiotic theories provide an important foundation for the study of is. i will further argue that actor-network theory (ant) is a semiotic approach that is particularly suited to understanding information systems. representation in information systems development in traditional life-cycle approaches systems development takes place through a number of fairly clearly differentiated stages. transitions between successive stages are marked by "sign-offs" where all stakeholders agree that the work so far has been completed satisfactorily. what is being agreed upon is some sort of document, usually a mixture of text and diagrams, which is supposed to represent in a certain level of detail (depending on the particular development phase) the system that will eventually be installed. from a semiotic point of view these text elements are signs that represent parts or features of the proposed system. there is considerable anecdotal evidence that different stakeholders have widely varying understandings of what they have agreed to in signing-off. we have also noticed, at least in student projects, that developers see the documentation as representing existing business rules and development work already completed, and that their idea, if any, of the final system, is imagined privately and separately from the agreed documents. prototyping has come to be used as a supplement, or in some cases a replacement, for life-cycle approaches. with prototyping, representation of the future system is in the form of hardware and software artefacts as well as documents. agreement on the design is negotiated continuously rather than at a number of discrete sign-off points. concerns have been raised over the extent to which prototypes can represent issues of system management, maintenance and performance under normal operating conditions (davis, 1984). from our point of view prototypes are also signs, and raise similar issues of understanding to documents. beyond prototyping there is evolutionary development. here requirements construction, implementation and operation occur simultaneously. an embryonic system is built and put into use. with use some features are found to be helpful and are enhanced, others are ignored and gradually disappear from the system (although they often remain buried in unused code). as with prototyping, the speed of evolutionary change is facilitated by rapid software development environments, particularly where changes can be easily made by end users. in a sense, evolution of information systems is what always happens over time anyway. adopting evolutionary development as a deliberate policy is a matter of realising that we will never be able to faithfully represent a future system, of removing controls that attempt to make the future system represent the original design, and allowing the evolutionary process to speed up. we see design activities as social activities, exercises in communication. where there is a designated "designer" their role involves at least two communication relationships assuring the users that the system-to-be will make their lives better and guiding the system builders (often programmers) in their construction work. when prototyping is used extensively the designer might take a less dominant role, mainly facilitating communication between the users and the builders. successful information systems design would seem to depend on establishing mutual understanding either between users and designers and designers and builders or between users and builders and preferably among all three groups. an appreciation of how information systems are constructed through representation and shared understanding is critical for anyone attempting to manage the ajis special issue december 2002 87 systems development process. this is so even with an evolutionary approach: design, building and use occur simultaneously, but even within one user there are conceptual shifts as they move among these three modes of system construction. peirce's semiotic model for these reasons, an important theoretical foundation for research and teaching in is is semiotics (nöth, 1990), the theory of signs and meaning. fundamentally, semiotics is simply about representation, about how signs stand for things in the world of objects. one popular semiotic theory is peirce's triadic model. a sign, or representamen, is something, which stands to somebody for something in some respect or capacity. it addresses somebody, that is, creates in the mind of that person an equivalent sign, or perhaps a more developed sign. that sign which it creates i call the interpretant of the first sign. the sign stands for something, its object. it stands for that object, not in all respects, but in reference to a sort of idea, which i have sometimes called the ground of representation. (peirce collected papers of c.s. peirce (2.228) quoted in spinks, 1991) in the case of is development some of the things represented, systems-to-be, do not yet exist and we might expect to find that the same sign could have different meanings for different stakeholders. hence is development is about networks of meaning the existing network of the organisation, the project development network where the information system becomes an object of shared understanding, and the changed organisation network which will include the completed information system. documents, models and discussions are the links, which carry, or fail to carry, meaning in these networks. traditional descriptions of information systems rely not only on natural language but also on diagrams (structured or unstructured) and restricted or formal languages (such as pseudocode). some authors attempt to use semiotics as a basis for yet another specialised is specification language (stamper, 1992), or use semiotic concepts to classify user communications in existing systems (clarke, 1998). our image of information systems development is strongly constructivist. this means that the world, in this case the supposed reality underlying the system design the business rules and the technical possibilities, is created by the participation of the participants (users, designers, programmers, managers) (watzlawick, 1984). although some methodologies give the impression that system design follows logically from predetermined requirements, it is fairly obvious that information systems are in fact designed and built by people, almost always working with others, and sometimes with computers, in a social setting. but we also believe that the requirements themselves are socially constructed during the design process. in this situation aspiring is developers do not need a semi-formal semiotics based methodology, nor a theoretical model of "best practice" organisational communication, but a knowledge, sensitivity and attitude which will keep them continuously aware of representation issues during the development process. one representation issue, which (as we have mentioned) is of particular significance in design and development, is that of signs referring to future objects. peirce provides some light on this issue when he distinguishes objects as immediate, dynamic and real. the immediate object is that which takes part directly in the sign occurrence, and is in a sense created by the act of interpretation (semiosis). the dynamic object has an existence independently of the particular sign occurrence and to some extent can be thought of as co-producing the sign. if there is anything real that “sufficiently corresponds” to the immediate object then this can be called the real object (oehler, 1987, p10). the introduction of an immediate object avoids or at least defers a difficult semantic question: can a sign refer to an object that does not exist (such as a unicorn) or which is nonsensical (such as a square circle)? for peirce anything referred to in a sign is meaningful, because by definition it means in some respect something to somebody (perhaps as an example of impossibility); the sign itself validates the immediate object. but can these unreal objects also be dynamic objects? we can regard them as such if the same object seems to take part in a variety of signs. a unicorn can represent good fortune, we can say “a unicorn is a mammal” and many children can draw one. it is reasonable to believe that these cases all refer to the same (dynamic) object. the idea of a dynamic object is clarified when signs are used in a design process. suppose we have a data-flow diagram of part of an existing payroll system. the data-flow diagram and the phrase "add new employee" are two sign vehicles referring to the same dynamic, and presumably real, object. in another context the data-flow diagram may signify a quite different object: if someone is photocopying the system documentation this diagram could indicate the middle page of the documentation. with an existing system we assume that the objects are to some extent already established; when designing a new system the objects are gradually brought into being. suppose now that we begin our design by drawing boxes representing subsystems of the payroll system. we label one of these "add new employee". by this action we create an immediate object, which we imagine as a process, to which the sign refers. as we think about this object, mention it in reports, discuss it with others and perhaps ask someone else to draw a data-flow diagram for it, the immediate object gradually achieves the status of a dynamic object, existing independently of any particular sign occurrence. if we spend a large amount of money on hardware, software and training and persuade several people to change their payroll processing behaviour we might eventually feel confident in calling the object real. ajis special issue december 2002 88 the interpretant is what the sign means, or how the sign works. it is the “respect in which” the sign vehicle stands for the object. but what exactly is it? for peirce the interpretant is that which the sign produces in the interpreter; it may be a feeling, an action or another sign. since peirce was an empiricist and pragmatist the ultimate meaning of the sign must be its observable effect, so the feeling must be eventually expressed by speech, action or symptom; in the end all interpretants are available as vehicles for further signs. as with objects, there are various types of interpretants. the immediate interpretant is the actual understanding of the sign (perhaps a process), the dynamic interpretant is the reaction provoked and the final interpretant is the “proper” reaction to which all interpreters converge (oehler, 1987, p6). the combination of sign vehicle, final interpretant and dynamic object gives a general sign that is understood through objective consensus, not individual thought. this is important for peirce because it removes signs from the unverifiable mental space of the interpreter. the disadvantage is that signs can only be understood in terms of other signs and in theory this leads to an infinite regress; but in practice we always manage to stop. until the object becomes real the assumed existence of a dynamic sign shared by a number of sign occurrences depends on the existence of a stable mental image. agreement among sign users depends on two links: the object in question must invoke the same mental image in all users, and they must use the same sign to indicate that image. because the common element, the mental image, is private and cannot be accessed except through other signs, it is impossible to tell whether agreement has been reached or how it has broken down. management and workers may agree that workers should be "more productive" but management may have an image of more output per dollar while the workers imagine more output per person (and more pay). if, referring to the same events, managers say "we have down-sized by 10%" and workers say "500 people were sacked" management may be using a euphemism (different sign, same object and image) or may have a quite different mental image of the events. in practice, although our mental images related to a sign differ and although any clarification must be based on equally ambiguous signs, we eventually build a network of connected signs that we are fairly confident others understand. that this confidence could be grossly misplaced is apparent when we discuss "red" with a colourblind person, or are told by a teenager that certain behaviour is "sick". nevertheless, such confidence is exactly what keeps communication going. different ways of using signs (morris) the pragmatic semiotics of peirce and james, which believes "that, to develop a thought’s meaning, we need only determine what conduct it is fitted to produce: that conduct is for us its sole significance" (james, 1991, p23), is taken to its logical conclusion by charles morris. morris has been credited with defining the terms syntax, semantics and pragmatics with reference to signs (posner, 1987, p25). his own theory of signs is based on behavioural psychology. for morris all behaviour is goal-seeking: the organism selects a desirable object (orientation), performs appropriate actions (manipulation) and fulfils its desire (consummation). of course while this is going on many other organisms are doing the same thing so it is easy to become distracted or completely diverted; and organisms can have very obscure strategies where even they find it difficult to distinguish ends from means. the goal-seeking model can, however, still provide the basis for a definition of sign. thus a sign is a preparatory stimulus that, in the absence of an impulse-satisfying object, causes in an organism a sequence of responses of the same type that would be caused by the object itself. (posner, 1987, p28) here morris was probably thinking of something like pavlov’s bell and he would most likely expect his definition to expand to include more complex situations. if we imagine a specification as a sign for an information system then this definition could work in some cases. a customer might, under strict conditions, pay for the specification as if it were the system and a court of law might (again only in some circumstances) accept the specification as signifying the system. but an end user could not actually use the specification to do their work and a software engineer faced with the specification might be tempted to build the system, something they wouldn’t do if the system already existed. in the same place, however, posner compares the sign to the behaviour sequence itself. the sign vehicle alerts the organism to the desired object, which will satisfy the impulse, while the interpretant is the impulse itself. in this case we can say that the meaning of the specification to the engineer (or the client) is the satisfaction they anticipate when using the completed system. thus we might respond to an advertisement (the sign vehicle) for a new operating system by imagining that obtaining it will lead to an experience (the object) that will give us greater reliability or will make our friends think we are top computer scientists (the interpretant or imagined consummation). corresponding to the behavioural sequence, morris defined three dimensions of signification. in the orientation phase it is important to assess the facts of the situation and designative signs are most helpful; in the manipulative phase we need rules or guidelines about what to do through prescriptive signs; and when considering consummation we need to evaluate the desired object using appraisive signs. we would normally, for instance, use a prescriptive sign if we wished someone else to do something for us ("please back up the disks regularly"); but in some circumstances it may be more effective to use designative ("did you hear how fred lost all his data last week?") or appraisive ("our database is our most valuable resource") signs for manipulative purposes (posner, 1987, p41). ajis special issue december 2002 89 representation in actor-network theory from the above we can see that meaningful signs, particularly for yet to be constructed objects, depend on the development of shared meaning, but that ambiguity of meaning can be used for a form of manipulation. systems development methodologies often assume that all parties to the development share goals or, in the case of soft systems methodology, that a common direction is established at an early stage (checkland, 1972). if various participants continue to have different interests then the likelihood of shared meaning developing is greatly reduced. stakeholder theory is a widely accepted model for describing conflicting interests, but gives us little insight into the expression, communication and manipulation of these interests. actor-network theory (ant) provides a semiotic approach to stakeholder analysis. actors, networks and meaning actor-network theory (ant) was developed as a method in science and technology studies. it attempts to anchor meaning to the desires of people and the constraints of nature (latour, 1993, p64) by granting status to hybrids or quasi-objects, which are heterogeneous networks of phenomena. the hole in the ozone layer, for instance, "links the most esoteric sciences and the most sordid politics, the most distant sky and some factory in the lyon suburbs, dangers on a global scale and the impending local elections or the next board meeting" (latour, 1993, p1). similarly, a management information system is "an integrated user-machine system for providing information to support operations, management, and decision-making functions in an organization. the system utilises computer hardware and software; manual procedures; models for analysis, planning, control and decision-making; and a database" (davis and olsen, 1984, p6). by tracing the activity of these hybrid networks we can see how the world is interpreted, reinterpreted and changed. these networks of actors are the hybrids ("network", "hybrid" and "quasi-object" are interchangeable terms). from the point of view of much traditional philosophy, confusing machines, people, ideas and organisations would be regarded as a category error. ant makes category error its major virtue. latour (1993, p11) explains this in the following diagram. figure 1 purification and translation he says that the project of modernism attempted to divide the world of thought into the natural and the social, a world of objects, forces and velocities, and a world of feelings, beauty and community. in one world statements may be true or false, in the other right or wrong. before we can discuss or even think of some phenomenon we must decide on which side of the vertical divide it lies. this decision may vary over time, and may even divide different aspects of what appears to be one being at a particular time in history, human anatomy may have been considered a natural phenomenon, while human thought may have been cultural. purification ensures that we talk about the right things with the right word, that we avoid such "errors" as anthropomorphism. but below is the hidden world of translation. practical people are building factories, musical instruments, information systems and cloned sheep; artefacts in which the natural and the cultural are hopelessly mixed. the modern project requires that we don't think of the two levels, the worlds of purification and translation, at the same time. otherwise the belief that thought can be clear, that ontology is possible, that our life can be controlled by the right ideas, would dissolve. and because we cannot use our much refined tools from the world of thought, the world of hybrids does run out of control and is filled with all sorts of monsters on which we depend but which we don't understand. how can we bring meaning to these networks of translation? ajis special issue december 2002 90 scripts and translation ant is a type of stakeholder analysis where the stakeholders may be other than the usual people and organisations and the stakes may be reinterpreted as negotiations progress. typical stakeholder phenomena such as the formation of alliances are dealt with in detail by callon (1986) and will not be covered here. we will concentrate on the key concepts of ant: actor, script and translation. in is development the obvious candidates for actors are the stakeholders such as owners, clients, operators and regulators; and the potential components of the future system are hardware, software, people and processes. as callon comments, "... first ... the list of actors is not restricted to social entities; but second, and most important, their identities and their wishes are all constantly negotiated during the process of translation" (callon, 1986, p227, p21). scripts represent the interests of the actors. notice that we have just uncovered another level of ambiguity in "representation". a script may be as simple as "close the door" or a million line computer program. the design of technical artefacts (quasi-objects) involves delegation of actions to various actors by inscribing. this process is complicated with a collection of script-based activities such as conscription, prescription, proscription and subscription, all of which may be ascribed to a particular situation by researchers (or project managers) (akrich and latour, 1992, p259). technical artefacts such as database servers or application programs are judged by their actions; they make performative statements. in systems development they will be conscripted to the purposes of other actors by being inscribed with scripts in a language, which they (pragmatically) understand. this conversion of scripts (perhaps into machine code) as they are inscribed from one actor to another is one of the several meanings of translation in ant. translation can model the operation of is components as well as their design, for example when client requests are translated into keystrokes. it is widely accepted that failures in translation, both during design and operation, are a common source of system "malfunction". ant allows us to model this mistranslation as possibly intentional (betrayal) and as an essential part of system design and operation, rather than as an exogenous disturbance, which can be removed. in the pure world of modern thought, where humanity and nature occupy separate domains, such a removal might be imaginable. in the messy world of modern practice, the world of technology in general and information systems in particular, such a removal would render translation impossible (latour, 1993, p11). representing concepts in ant latour gives us a vivid picture of the way scripts move and change among the network of actors, but does not address how the meaning of the scripts might change with the interests of various classes of actors. we have proposed (underwood, 2001) that, using foucault's (1972) concept of discursive formation, we can see how particular actors may translate scripts with reference to a "home discourse" such as accounting or computer science. this translation takes place in the "world of thought" and is facilitated by the use of learned concepts such as "efficiency" or "attribute". but in the "world of practice", where actors from different discursive backgrounds must engage in at least minimal collaboration if any progress is to be made at all, these concepts must be reinterpreted or betrayed to develop some level of shared meaning. as this process continues, some type of local language or "hybrid discourse" will develop and actors may gradually detach from their home discourse ("go native"). the ability to straddle these two worlds, shown in figure 2, with relative comfort is a critical skill for any system developer. ajis special issue december 2002 91 figure 2 concepts as actors from a theoretical point of view, the fate of the concepts as they are betrayed and transformed can be followed in ant by regarding the concepts themselves as actors (underwood, 2001). this move was anticipated in stakeholder theory by mitroff (1989). for practical purposes, however, we still need to quarantine the concepts from the rest of the network. this recognises that although concepts can inscribe other actors they are not, as long as they remain in their home discourse, themselves inscribable. in the long run the construction of multitudes of computer-based accounting information systems may change the discourses of both accounting and computing, but for any given development project these discourses have to be taken as given. dealing with key concepts in is development over the years a number of concepts have been proposed as, or assumed to be, important to is development. most technically oriented approaches either explicitly or implicitly advocate rationality and objectivity. more socially oriented values are often relegated to treatment as "social impacts" or "professional responsibility" but a number of methodologies have attempted to build in social concepts from the beginning. stafford beer's (1985) viable systems model focuses on control; enid mumford's (1979) ethics methodology emphasises freedom; checkland's (1972) ssm is based on consensus; and current approaches to is design emphasise knowledge. with ant as a basis for our understanding of is development we have neither to leave these desirable values for an "impact assessment" at the end nor to choose one and use it as the (somewhat disguised) basis of a complete methodology. these and other concepts can be seen as additional actors, represented in various guises during system design, development and operation, with a shifting band of allies and enemies. we can follow their fortunes and assess their influence on the "final" system, while all the time remembering that we are also actors in the network. if we can inspire this attitude in our students we may contribute to the growth of is developers who are ethical, effective, and sometimes even both. conclusion actor-network theory and foucault's discourse theory have already been used by a number of authors (eg bloomfield et al, 1997; cooper and bowers, 1995; hanseth and monteiro, 1997; hocking, 1996; poster, 1990; remenyi et al, 1999; woolgar, 1996) to better understand is development and use. a combination of these theories, emphasising representation, translation and concept may provide a durable foundation for he study of information systems. but actor-network theory is not, according to latour (1999), really a theory at all. it is either a framework, providing a series of tricks to aid our understanding, or a meta-theory, allowing us to compare other theories or even eras. many have searched for a foundation theory for information systems. davis (2000, p78) found that at least ten different major theoretical areas have been used recently as a basis for is research and these are divided more or less equally between "hard" (if not natural) and social sciences. if latour's analysis of modernism is correct, theories from these two "sides" will have been more or less deliberately constructed to be incompatible. to simplify, we may imagine "computer science" represents nature in figure 1, and "sociology" represents culture. then the hybrid networks will be represented by "information systems". the only way these networks can exist is by not referring back to "foundations"; otherwise they will be mired in contradiction and controversy. it follows that the only way the putative foundational disciplines can be represented is by some of their concepts taking part as actors, on an equal basis with all others (computers, bank tellers, financial regulations, etc). the practical and very useful role of ant in any particular systems development or operation is to promote understanding, and to facilitate collaboration and betrayal. its equally ajis special issue december 2002 92 useful theoretical role in the “discipline” of information systems is to show us that there can never be any foundations, except in practice. even though speaking of social research, the comment of schratz and walker (1995, p105) is apposite: our aim is to resist generalising, to scrutinise every statement for values, to search for blind spots in our own perceptions in order to understand both the espoused theories and theories-in-use of all those involved and implicated in a particular setting, event or process. we see theory as implicit in every social action, not simply the concern of academic theorists, and rather than seeking to replace these everyday theories with those we consider better, we seek to explicate and understand them. references akrich, m., latour, b. (1992) "a summary of a convenient vocabulary for the semiotics of human and nonhuman assemblies" in w. e. bijker, j. law (eds) shaping technology / building society mit press avison, d. e, fitzgerald, g. (1995) information systems development, mcgraw-hill beer, s. (1985) diagnosing the system, wiley bloomfield, b. et al (eds). (1997) information technology and organizations, oxford university press boulding, k. e. (1961) the image, university of michigan callon, m. (1986) "some elements of a sociology of translation: domestication of the scallops and the fishermen of saint brieuc bay" in j. law (ed) power, action and belief: a new sociology of knowledge? sociological review monograph, routledge and kegan paul checkland, p. b. (1972) "towards a system-based methodology for real-world problem solving" journal of systems engineering 3(2) clarke, r. j. (1998) "systemic semiotic development using sfx: prototyping genre-based representations of is workpractices" proceedings of 1998 ieee international symposium on intelligent control (isic) sept 14th-19th 1998, pp846-851 cooper, g. & bowers, j. (1995) "representing the user: notes on the disciplinary rhetoric of human-computer interaction" in p. j. thomas. (ed) the social and interactional dimensions of human-computer interfaces cambridge university press davis, g. b. (1984) "caution: user developed systems can be dangerous to your organization" joint international symposium on information systems acs/ifip davis, g. b. & olson, m. h. (1984) management information systems mcgraw-hill davis, g.b. (2000) "information systems conceptual foundations" in r. baskervilleet al (eds) organizational and social perspectives on informational technology kluwer, boston foucault, m. (1972) the archaeology of knowledge tavistock hanseth, ø. & monteiro, e. (1997) "inscribing behaviour in information infrastructure standards" accounting, management and information technology 7(4), 183-211 hocking, l. (1996) "systems analysis and early design as the negotiation of meaning and interest" in c. d. keen et al (eds) proceedings of 7th australasian conference on information systems acis’96 james, w. (1991) pragmatism prometheus books latour, b. (1993) we have never been modern, harvester latour, b. (1999) "on recalling ant" in j. law, j. hassard (eds) actor network theory and after, blackwell, oxford mitroff, i. i. (1989) stakeholders of the organizational mind, jossey-bass mumford, e. & weir, m. (1979) computer systems in work design, associated business press nöth, w. (1990) handbook of semiotics, indiana university oehler, k. (1987) "an outline of peirce’s semiotics" in m. krampen et al (eds) classics of semiotics, plenum press oz, e. (2000) management information systems, thomson learning posner, r. (1987) "charles morris and the behavioral foundations of semiotics" in m. krampen et al (eds) classics of semiotics, plenum press poster, m. (1990) the mode of information, polity press remenyi, d., white, terry & sherwood-smith, m. (1999) "language and a post-modern approach to information systems" international journal of information management 19(1), february 1999, 17-32 schratz, m. & walker, r. (1995) research as social change routledge spinks, c.w. (1991) peirce and triadomania, mouton de gruyter stamper, r. (1992) "signs, organisations, norms and information systems" in r. macgregor (ed) proceedings of third australian conference on information systems, university of wollongong underwood, j. (2001) meaning in is development, unpublished phd thesis, university of wollongong watzlawick, p. (ed) (1984) the invented reality w.w. norton woolgar, s. (1996) "the user talks back: directed reading at the front line" crict discussion paper no. 57, brunel university 69 the role of conferences and refereed journals in australian information systems research 15 gail ridley department of information systems university of tasmania gpo box 252-87 hobart tasmania 7001 email: gail.ridley@infosys.utas.edu.au abstract the premise of this paper is the need for the is discipline in australia to identify its nature and traditions before promulgating both to gain greater access to political power and resources. the survey results reported help characterise australian is research by assessing the significance of conference publications and determining which international and national conferences and refereed journals are perceived by leading is researchers to be the most appropriate and of the highest quality as publication outlets. comparison is made between the results of this study and north american rankings. results from the study support the claim that refereed conference publications are significant to is researchers and help legitimise this style of publication. introduction information systems (is) is an emerging field. over the last decade there has been an explosion in research undertaken in the area. the number of new is journals and recently introduced university is research programmes provide evidence of this claim (galliers 1992). however, as australian is research education has a short history and current researchers come from disparate employment and educational backgrounds, researchers bring different interests and approaches to australian is research (shanks, rouse & arnott 1993) which influence the content and style of research undertaken. partly too because reference disciplines play a significant role in is, its boundaries are not well established. as a result of these factors and others, little is tradition or orthodoxy have been established and recognised in australia. as the is discipline is not certain of its own nature and lacks social standing, it has a limited ability to lobby and exert political influence (king 1993). one result has been a limited representation on relevant decision making bodies that control access to resources, power and status, and a dependence on the decisions of researchers from other disciplines. for example, initially there was no is representative included in the working party to undertake a study to develop a discipline research strategy for information technology in australia (arnott 1996). as is gives little guidance on appropriate bases for making determinations, the author considers that de facto decisions are made, often based on the traditions laid down in the disciplines of the decision makers. because is research is unlikely to score well on measures established for other disciplines, the consequences are likely to be negative. for example, even though only three of the sixteen subcategories from the 'information, computers and communication technologies' section of the 1995 australian research council (arc) initial awards of large grants appear to cater for is research, only one of eight grants awarded within the three subcategories was awarded to a researcher generally acknowledged as belonging to the is research community (arc 1995). despite is's applied nature there is an urgent need to assess the character and traditions of is research and then develop and proclaim explicit guidelines to assist in the recognition of quality research (king 1993). the first australian attempts to do so and form potential lobby groups are now apparent, for example the australasian is survey 1995 (keen 1995), the is heads of discipline meeting at the australasian conference on is 1995 and 1996 and the is curriculum working conference 1996. however, the author considers that australian is research lacks both an awareness of a need for a political identity and background analyses of its nature that will allow the discipline to present a cohesive front. both issues have been addressed by the computer science community in australia since 1981 with the formation of the australian association of professors of computer science as a lobby group, and the collection and publishing of relevant statistics (bennett et al. 1994). the research described in this paper partly addresses this need in is. it has been claimed that for the communication of research results in computing, refereed conference papers are more important than refereed journal papers (lowe 1995). without empirical evidence that compares the significance of the two publication forums like that presented in this paper, it may be difficult to convince 15 a previous version of this paper was presented at the 7th australasian conference on information systems and appeared in the proceedings of that conference. 70 academics from other disciplines of this, for example when they are involved in the appointment of an is academic or consider an is research project for an arc grant. similarly, the ranking of appropriate journals by quality for australian is researchers in this paper will assist those from both within and outside the discipline to make better informed decisions that may affect the future of is researchers. rankings of is journals and conferences in the literature have been developed by north american researchers and deal with north american journals. there has been a call for analyses that rate journals from both north america and elsewhere (morrison & george 1995). it is not known how appropriate north american rankings are for australian researchers and how non-north american journals or conferences rank against those from north america. if australian is researchers are making de facto use of north american rankings of publication outlets, there is a need to learn whether the rankings are relevant. the research reported describes the first phase of a larger study to investigate the nature and development of is research in australian universities. a preliminary stage is to determine the most highly ranked journals and conferences for australian is researchers. establishing a ranked list of appropriate quality publications and an empirical assessment of the significance of conference publications are considered significant enough tasks to be reported independently. it is understood that neither has been done before. as publication acts as a 'legitimator of academic performance' (king 1993) it is critical for is that its publication patterns be validated. it is remarkable that the importance of conference publications appears not to have been compared to that of journals in an empirical study before, despite its obvious value to the advancement of is research throughout the world. background overview of literature a review of literature was conducted where the ranking of literature relevant to is research had been established or employed. some research ranks publications using empirical investigation while other research assumes the leading publication outlets for the field. although investigation that establishes the ranking of publication outlets based on empirical work (doke & luke 1987; koong & weistroffer 1989; gillenson & stutz 1991; hamilton & ives 1983; holsapple et al. 1993; omar & goodwin 1991; vogel & wetherbe 1984; walstrom et al. 1995) is of more relevance than studies that adopt leading journals for further investigation the latter group reveals expert perceptions of the significance of is journals and conferences and reinforces the standing of journals and conferences. nord and nord (1995) combined both approaches in their study which determined top-ranked journals from an examination of the results of three previous investigations, and then grouped them into two tiers. the four studies, gillenson and stutz (1991), holsapple et al. (1993), nord and nord (1995) and walstrom et al. (1995) provide the most meaningful comparison as they are recent and consider journal rankings by, or for management information systems (mis or is) academics. other recent empirical studies examine subsets of mis research (doke & luke 1987; omar & goodwin 1991) or address practitioner perceptions (tsai et al. 1991). this study restricts itself to is university researcher perceptions as the outcomes are designed to serve that group. methodology of previous studies some studies have ranked publication outlets using a citation analysis methodology rather than relying on expert perception (hamilton & ives 1982; holsapple et al. 1993; omar & goodwin 1991; vogel & wetherbe 1984). holsapple el al. claim that citation analysis is a less subjective technique to rank publication outlets. however, as the quantity of australian research published in international journals is unknown and publication may be limited to particular outlets, such an approach for this study may unnecessarily limit the publications ranked. use of expert perception to represent the quality of journals is an established practice in strategic management, behavioural management and mis (walstrom et al. 1995). by expert perception it is meant people who work as professionals in the area. gillenson and stutz (1991) polled chairmen and senior is researchers to rank is journals. in the gillenson and stutz study and possibly the walstrom et al. (1995) study, 'not familiar' responses to journals have been ignored rather than treated as a zero. as holsapple et al. (1993) and nord and nord (1995) use criterion based filtering processes that incorporate previous research findings, their results may also have been influenced by the treatment of 'not familiar' responses. the result may have been to increase the rating of less well known journals. statistically it would be more meaningful to treat each of these responses as a zero. 71 however, to do so is likely to yield a different journal rating for this reason alone, making comparison with prior research impossible. many studies have grouped research outlets into two or three 'tiers' (holsapple et al. 1993). north american research has shown that when publication outlets are grouped into tiers there is some consistency in the membership of a journal to a tier. gillenson and stutz (1991) reported that naturally occurring 'breaks' occurred in their data to form the boundary between tiers. although this technique may be of value in clustering publication outlets for limited samples, any boundary established by 'breaks' is an arbitrary division as ranges are continuums. a researcher's publication history and editorial association with a journal has been observed to have little or no influence on ratings of publications (extejt & smith 1990; walstrom et al. 1995). findings of previous studies gillenson and stutz (1991) and walstrom et al. (1995) grouped the journals into three tiers. the first tier of the gillenson & stutz study was made up of mis quarterly, communications of the acm and management science. note that 7s research had been announced but not published at the time that data collection took place. the walstrom et al. paper also grouped the publications into three tiers, in the first of which were mis quarterly, communications of the acm, is research and management science. in 1993, holsapple et al. grouped journals into two tiers, the first of which contained 19 journals while the second tier had 25. nord and nord (1995) also grouped mis journals into two tiers, with nine journals in the first and eight in the second. other studies indicate the strong performance of communications of the acm and mis quarterly at or near the top (doke & luke 1987; culnan 1987; jackson & nath 1989; koong & weistroffer 1989; vogel & wetherbe 1984). holsapple et al. (1993) report that there is a consensus of the top four publications in six studies. each discipline has a particular publication pattern in australia. between 1988 and 1992 the average refereed journal output per researcher for all disciplines was 3.7 papers, for physical sciences it was 8.5 and for computing and information technology (it), 0.5 (lowe 1995). in computing and it, refereed conference papers are 'much more important' than refereed journal papers (lowe 1995). as change occurs rapidly in computing the more rapid publication in conference proceedings may account for the dominance of this kind of publication. the majority of the literature attempts only to rank journals (gillenson & stutz 1991; holsapple et al.; nord & nord 1995). the walstrom et al. (1995) study considered both journals and conferences, but ranked the two types of publication outlets separately. methodology epistemology, research question and methods employed this project is conducted within a positivist framework using both descriptive and inferential statistics. gathering data using an interpretivist approach would have been difficult as the respondents are in different locations in australia and elsewhere. as australia has a small population and is is not well established in australian universities, the number of eminent experts on australian is research is restricted. as a result, the number of subjects from whom data are collected for the study is limited. the study aimed to rank international and australian journals and conferences appropriate as publication outlets for australian is researchers by perceived quality on a single scale. in this way a comparison may be made between the journals and conferences. distinction needs to be drawn between appropriateness and quality. some journals may be of high quality, but due to the emphasis of australian is research, may be perceived to be unsuitable as a publication outlet. respondents were required to consider the appropriateness of each journal and conference listed before rating it by quality using the criteria provided. the major method employed in this project was an emailed survey. one disadvantage is that respondents are not anonymous. respondents may be reluctant to declare openly their ratings for publication outlets, and in particular to indicate that they are not familiar with certain journals and conferences. this problem is a limitation of the current study. selection of publication outlets it was necessary to develop or adopt a list of publications for inclusion in the survey instrument. the core of journals used for analysis in this investigation was derived from a study by walstrom et al. (1995). walstrom's list was used as it was based in turn on previous journal studies (doke & luke 1987; gillenson & 72 stutz 1991; hamilton & ives 1982; vogel & wetherbe 1984). the journals and conferences in the list derived for this study contain all those in lists of leading journals recorded by alavi and carlson (1992), cheon (1993), grover et al. (1992), jackson and nath (1989) and orlikowski and baroudi (1991). the list contains all the journals identified by grover et al. (1993), which are consistent with the leading journals identified by culnan (1986), culnan and swanson (1986), nolan and wetherbe (1980) and vogel and wetherbe (1984). all the journals in the list by holsapple et al. (1993) appear in the list developed for this study. the journals identified as being in the first tier ranking from the study by nord and nord (1994) all appear in the list derived for this study, with the exception of is management. it is clear that there is strong support in the literature for the core of publication outlets derived for use in the current study. in the list of journals and conferences, no distinction was made between researcher and practitioner publications. this approach echoes that taken by walstrom et al. (1995). there is a reliance on north american-based journals and conferences in this investigation. however, other studies of the is literature display a similar bias (morrison & george 1995). because this is an australian study the core of north american based journals and conferences were supplemented by more regional journals and conferences. as a second method of including publication outlets seen as appropriate for australian researchers, respondents were invited to nominate other unlisted journals and conferences and rate them. the survey instrument was developed, piloted and implemented, and required respondents to rate 38 journals and conferences. this compares with a different list of 38 journals in the gillenson & stutz study (1991) and 27 publications in the walstrom et al. (1995) study. three australian journals have been added to the list developed for this study, the australian computer journal, the australian journal of is and the australian journal of management. the least popular five journals from those listed in the walstrom et al. (1995) study have been omitted in order to avoid making the list unwieldy. the five most highly ranked conferences identified in the walstrom et al. study have been included, plus the recently introduced america's conference on is. the international federation for information processing world congress and ifip working group tc8 conference, as well as two conferences from the australian region, the pan pacific conference on is and the australasian conference on is have been included. the conferences and journals from the australian region were selected for their broad appeal, and the choice was validated when no generally suggested regional publication outlets were nominated by the respondents. the result is a list of 26 refereed journals and twelve conferences. the list from the walstrom et al. (1995) study included only journals. as conferences and journals are both outlets for research publication, it was decided to test how researchers would rate one against the other. survey instrument the wording of the ranking instrument has been adapted from the walstrom et al. (1995) study. a five point rating scale was used instead of the four point scale used by walstrom, to be consistent with recommended questionnaire design techniques (davis & cosenza 1988). it was considered that walstrom et a/.'s scale required respondents to rate publications according to two dimensions, one the explicit request to consider how appropriate the journal was as a publication outlet, and the other an implied request to rate the quality of the journals. the reliability of a measure is reduced by multi-dimensional questions. accordingly, in the modified scale developed for this study only those journals and conferences considered appropriate as an outlet for australian is research were rated, and the scale referred only to the publication quality. in the modified measure the respondent was asked to place a cross where they were not familiar with the publication outlet. as a result, the modified measure is likely to be more reliable than the original measure where it was assumed that where a respondent did not rate a journal, the respondent was unfamiliar with the publication. in addition to the rating instrument the questionnaire asked for the position and location of the researcher, and for a measure of the familiarity of the respondent with australian is research. as no publication bias was found in the rankings of publication outlets by researchers in two previous studies (walstrom et al. 1995; extejt & smith 1990), the publication history of the respondents was not investigated. the rating instrument is included in appendix 1. procedure and subjects the questionnaire was reviewed by three experienced researchers and extensively edited before it was emailed to a pilot group of eighteen, drawn at random from a list of senior is researchers from australian universities. respondents were requested to complete the questionnaire and suggest modifications. no significant modifications were generally suggested, and no changes to the survey instrument were made for the next round 73 of the survey. as the survey instrument for the next round of the survey was not different to the pilot, the authority of the pilot participants was sought to use their data for this study. the questionnaire was released to another group of 16 eminent is researchers. this group was again drawn from senior australian researchers, while six were from outside australia where the researcher had an apparent interest in australian is research. all undertook research within, or in association with universities and were involved with at least one editorial group. a limitation of the current study is that australian researchers are likely to be more familiar with international is journals and conferences than are non-australian researchers with australian journals and conferences. in addition, there is a possibility that some of the respondents may have been influenced by previous research that ranks journals. analysis descriptive statistics related to the appropriateness and quality of each publication were determined. to be consistent with previous recent research, when calculating the means of the ratings, the 'not familiar' and 'not appropriate' responses to publication outlets have been ignored rather than treated as a zero. the publication outlets were then ranked from the ratings according to the respondents' perception of quality. student's t-tests were performed on the means of the first and second ranked publication outlets, and then between the first and third, first and fourth and so on, until a significant result was obtained. the publication outlets above this point of significance were clustered to form tier 1. this technique was not continued to determine the boundary of further tiers as the differences between the means of the publication outlets became close and were not large enough to reach significance. below the boundary of tiers 1 and 2, the data were scrutinised to see whether any naturally occurring breaks existed which would could be used to divide subsequent tiers of publication outlets. a t-test was performed on the means of the journals and the means of the conferences to determine whether there was any significant difference between the two groups. a further t-test was conducted to compare the mean rankings of the australian respondents with the means for the international researcher group. results and discussion response twelve of the 17 pilot questionnaires and 14 of the 16 questionnaires from the expert round were returned, giving a response of 26 out of 33 or 79%. twenty of the respondents held positions of (australian) professor or associate professor, while six held other positions significant to is. all respondents in the pilot gave authority for the data to be used for analysis. it is considered that there is little chance of error introduced into the results due to perfunctory completion of the pilot round. the respondents were senior researchers, and the excellent response rate suggests that they found the survey relevant. each pilot respondent recorded the length of time it took to complete, which reinforced the validity of the pilot data. ranking of publication outlets table 1 summarises the responses for each publication outlet, presented in order of decreasing means. the column headed 'not appropriate counts' indicates where a respondent considered a publication to be inappropriate for australian is research. the column headed 'not familiar counts' indicates where a respondent placed an x to register a lack of familiarity with a publication. the number of rankings (n), mean, median, and standard deviation are given for each publication. the mean, median and mode or a subset of the three have been used in prior research to rank publications (walstrom et al. 1995). the mode has not been used in this study as it may not be typical of the data, and several modes may exist (pollard 1972). for table 1, the following coding scheme from the rating instrument has been used: 1 = very low quality; 2 = low quality; 3 = of about average quality; 4 = high quality; 5 = very high quality. 74 name of publication outlet mis quarterly is research management science communications of the acm international conf on is acm computing surveys deee trans on software ens. acm trans on d'base systems harvard business review sloan management review academy of management jnl academy of management rev journal of mis decision sciences journal of computer is dtp tc8 working conf. european jnl of is operations research data base dtp world congress decision support systems information and management jnl of is management america's conf on is (ajs) european conf on is hawaii int conf on ss (hicss) oopsla conferences omega australian jnl of is jnl of systems management australasian conf on is (acis) australian computer journal australian jnl of management decision sci inst conf (dsi) interfaces (orsa/tims) pan pacific conf on is (pacis) decision suppt. s. conf (dss) informs (orsa/tims ) conf. type of outlet journal journal journal journal conference journal journal journal journal journal journal journal journal journal journal conference journal journal journal conference journal journal journal conference conference conference conference journal journal journal conference journal journal conference journal conference conference conference not appropriate counts 0 0 1 0 0 0 4 0 1 0 1 1 0 1 1 0 1 4 1 1 1 0 0 0 0 0 3 1 2 1 2 2 2 1 0 0 1 4 not familiar counts 2 5 7 0 2 12 1 2 0 3 12 12 3 8 12 1 5 9 4 4 11 7 6 3 5 4 12 13 3 10 1 1 13 12 11 3 12 9 no. of rankings (n) 24 21 18 26 24 12 21 24 24 23 12 12 23 17 12 24 20 13 20 20 13 19 19 23 20 19 10 12 21 15 23 23 10 11 15 23 11 13 mean rating 4.58 4.52 4.50 4.31 4.21 4.17 4.14 4.13 4.13 4.00 3.92 3.92 3.91 3.88 3.83 3.63 3.60 3.60 3.55 3.55 3.54 3.42 3.42 3.39 3.35 3.32 3.30 3.25 3.14 3.13 3.09 3.00 3.00 3.00 3.00 3.00 2.82 2.50 median rating 5.0 5.0 5.0 4.0 4.0 4.0 4.0 4.0 4.0 4.0 4.0 4.0 4.0 4.0 4.0 3.5 3.5 4.0 4.0 3.5 4.0 3.0 3.0 3.0 3.0 3.0 3.0 3.0 3.0 3.0 3.0 3.0 3.0 3.0 3.0 3.0 3.0 2.5 std. dev. 0.67 0.67 0.67 0.51 0.83 0.79 0.83 0.67 1.00 1.00 0.79 0.67 0.67 0.79 0.58 0.97 0.75 0.79 0.79 0.83 0.80 0.90 0.67 1.07 1.00 0.78 0.95 0.75 0.83 1.00 1.14 0.74 0.82 0.63 0.60 0.79 1.17 1.00 table 1 summary of responses for each publication outlet, in order of decreasing means the majority of the publication outlets were considered appropriate as outlets for australian research, with the highest 'not appropriate count' of four being for specialised publication outlets such as the ieee transactions on software engineering and operations research. researcher familiarity varied considerably for the publication outlets, as did the standard deviation. communications of the acm and harvard business review were the only publications with which every respondent was familiar. the australian journal of management and omega were the least familiar publication outlets with half of the respondents (13) indicating that the journals were unfamiliar to them. twelve respondents or almost half of those surveyed indicated that they were not familiar with acm computing surveys, the academy of management journal, journal of computer is, oopsla conferences, decision science institute conference and decision support systems conference. among the least familiar of the journals and conferences were some of the most specialised. however, the first three ranked journals each had a low standard deviation with very similar means, possibly indicating agreement on the quality of the journals. the standard deviations of other publication outlets varied considerably. a high standard deviation may suggest a lack of agreement regarding quality. the two practitioner publications, harvard business review and sloan management review both had high standard deviations, possibly indicating confusion regarding the role of practitioner journals for researchers. in walstrom et al.'s (1995) study, a similar inconsistency was found as the same practitioner journals had high standard deviations. other publication outlets with a high standard deviation included the general is 75 conferences, america's conference on is and the european conference on is. this result may indicate a lack of agreement regarding the role and quality of general is conferences, both local and international. some respondents rated the general is conferences high, while other ranked them low. although the walstrom etal. (1995) study ranked conferences, it did not rank conferences against journals. the ranking of the conferences from the results of this study is similar to that indicated by walstrom et al. (1995), and there is agreement that the best quality conference is the international conference on is. after the international conference on is, walstrom et al. ranks in order decision science institute, the hawaii international conference on systems science, decision support systems and informs (formerly orsa/tims). apart from those in the second and third positions where the order has been transposed, the conferences ranked in the current study appear in the order proposed by walstrom et al. however, other conferences appear in the ranking of conferences from the current study, including one recently introduced conference and three from outside north america. other publication outlets survey respondents were asked to suggest other noteworthy journals or conferences and rate their quality. five journals were rated at least three times, while many others received one rating each. the is journal was rated ten times, the journal of strategic is was rated highly by nine respondents and the journal of information technology and acm transactions on is were each rated three times. the most frequently mentioned journal, the is journal was suggested as frequently as the least well rated publication outlet from the list in table 1 (n =10). however, it is not valid to compare the two sets of results. recalling an item is more difficult than that of recognising and selecting an item (preece et al. 1993). for this reason the ranking of the journals suggested by the respondents cannot be compared with the other rankings. however, it is evident that the is journal and the journal of strategic is should have appeared on the list presented to the respondents, although this was not obvious from the pilot. the is journal was first published as the journal of is in 1990 and was ranked poorly by holsapple et al. (1993) from data gathered earlier. the journal of strategic is was first published in 1991, but did not appear in the lists of publications put to respondents in walstrom et al. (1995), and other prior studies. it is suggested that both journals are likely to be well considered. ranking position and tiers the means of the rankings have been used for an initial differentiation between the publications, as they largely correspond with the median, but allow better discrimination. however, as differentiation of ranked publications on the basis of very small differences in the means may not be reliable the publications have been grouped into tiers. student's t-tests were performed to compare the subjects' ranking of the first ranked journal, mis quarterly, and the second ranked journal, 75 research, and then again between the first and the third, and the first and the fourth ranked publication outlet and so on, until a significant difference was found. non-significant results were obtained for the first five comparisons (p = 0.186, 0.135, 0.103, 0.088 and 0.119), which indicate that the probability of a difference arising by chance between mis quarterly and each of 75 research, management science, communications of the acm, international conference on is and acm computing surveys was high. however, when a t-test was conducted to compare the ranking of the first journal, mis quarterly, and the seventh publication outlet, ieee transactions on software engineering, a significant result of p = 0.008 was obtained. the result indicates that the respondents perceive a significant difference in quality between the first six publication outlets and the seventh and subsequent outlets. accordingly, the first six publication outlets listed above have been clustered to form tier one. from the seventh publication outlet (the start of tier two) the means of the publication outlets were close, making it unlikely that the differences in the means were large enough to be significant. as a result, and despite the inconsistency, a different method was needed to differentiate between tiers subsequent to tier one. below the boundary between tiers one and two the data were scrutinised to see whether any naturally occurring breaks existed which would divide subsequent tiers of publication outlets. after tier one, the next nine ranked publication outlets, ieee transactions on software engineering, acm transactions on database systems, harvard business review, sloan management review, academy of management journal, academy of management review, journal of management is, decision sciences and journal of computer is form a second tier, with a break in the means after the last journal (3.83 to 3.63). the last 23 publication outlets have been clustered into a third tier. 76 membership of a publication outlet to a tier is a more reliable measure of its quality than use of a specific ranked position, due to the restricted number of respondents in the study. however, when a publication outlet is located at or near the margin of a tier, the membership of the outlet to that tier is not as reliable as when a publication outlet is located more centrally within a tier. it can be seen that the top ranked conference, international conference on is is placed in the first tier. this ranking supports couger's claim (1985) that the international conference on is is the top ranked is conference. it appears that this finding is applicable to australian as well as to north american researchers. the other ten conferences have all been placed into the third tier, although the ifip tc8 conferences appear in the first position in the third tier, while the informs (orsa/tims ) conference ranks last. the last conference is not refereed. comparison of journals and conferences a t-test was conducted to compare the ratings of the conferences against those of the journals and a significant result was obtained (p = 0.001). it can be seen that there is a real difference in researchers' perceptions of the quality and appropriateness of is journals and conferences for australian research. certainly some journals are considered the most prestigious publishing outlet for australian is researchers. however, one conference, international conference on is, was not clearly of less quality than the journals in tier one. there is other evidence to support the importance of conferences to is researchers. many respondents have ranked general is conferences highly, which has contributed to a large standard deviation for these publication outlets. comparison of international and australian researchers a p value of 0.014 was obtained when a t-test was performed to determine if there was a significant difference between the ratings of the international researchers and those of the australian group. at the 0.05 level of significance, this represents a significant result. however, as the numbers of researchers were small, they may not be representative. the standard deviation of the mean ratings of the international group at 0.69 was higher than that of the australian group at 0.50. international researchers rated the publication outlets lower, but with less consistency than australian researchers. this finding suggests that australian and international researchers have different perspectives on the quality of the publication outlets. comparison of findings with north american studies comparison of the results was made with findings from four previous studies, gillenson & stutz (1991), holsapple et al. (1993) nord and nord (1995) and walstrom et al. (1995). these four comparisons were the most meaningful as they were recent studies that considered journal rankings by, or for mis academics. the holsapple et al. (1993) and nord and nord (1995) studies ranked the publications into two tiers, not three which makes comparison with the current study more difficult. the first tier in the 1993 study contained 19 journals while the second held 25. the nord and nord study placed nine journals in the first tier and eight in the second. all the studies ranked other journals that are not listed in the table. table 2 compares the tier allocation with those from the four previous studies. despite the difficulties in comparing the five studies, it can be seen that there is considerable consistency, particularly if only the first two tiers are considered. membership of tier 1 is identical apart from the comparatively recent appearance of 75 research, inclusion of the international conference on is which has not been ranked against journals previously and acm computing surveys which appeared in the second tier in three of the four comparison studies. if allowance is made for the two tiers in the 1993 and the nord and nord study with the resulting larger number of publications in the first tier, there is a remarkable degree of consistency between the second tiers of the current study and the others. however, it is more difficult to compare the third tier publication outlets in the five studies as different journals were used, and the current study is the only one of the five to rank journals and conferences together. study limitations; recommendations for future research a larger survey will overcome the limitations of the small number of respondents and check any impact of nonanonymity of the subjects on the data due to the email data collection method. future research will need to include the is journal and the journal of strategic is. further investigation is needed to examine the cause of the researchers' disagreement regarding the role of general is conferences, and to discover how well 77 conferences and journals rank in other fields. it may be that in fields where conference publications are considered less highly, no conferences appear in a ranking of the first 38 publication outlets. some cautions need to be stated. many of the journals and conferences have different aims and each probably fulfils its goals. exposure to similar north american ranking exercises may have influenced the respondents in their ratings. one limitation of relevant research is that non australian researchers are likely to be less familiar with the australian publication outlets than are australian researchers with non-australian journals and conferences. conclusions a ranking order of journals and conferences as determined by leading australian is researchers was produced in this study, but more reliable is an allocation of publication outlets to three tiers. australian senior is researchers considered the study to be relevant. the ordering of journals and conferences in tiers one and two is consistent with those of prior research, and the prior rankings of journals and conferences were found to be applicable to australian is researchers. australian perception of is publication outlets does not appear to be unique. however, prior research does not rank non-north american publication outlets, and the quality of australian journals and conferences clearly is of interest to australian is researchers. australian journals and conferences, and other local conferences were not ranked highly. in general international researchers rated publication outlets lower than did australian researchers. 'colonial cringe' may contribute to a higher rating of international is publication outlets and the lower rating of local outlets by australian researchers. the two highest ranked publication outlets were north american general is journals. also included in the first tier were two acm publications, a management science journal and a general international conference on is. the lowest ranked outlets were other conferences. the journals and conferences considered to be the least appropriate and familiar as outlets for australian is research were specialised rather than of general appeal. both the researcher's familiarity and the level of agreement of the researchers varied considerably with the publication outlet. there was widespread agreement from the researchers on the top ranked publication outlets in tier one. while practitioner journals rated well, there was greater variation in how they were rated. the rankings of general is conferences suffered because of a marked inconsistency in the ratings by researchers. australian is researchers perceived one conference to be appropriate and of similar quality to top ranked journals as a publication outlet. this and other indications provide evidence that it is possible for is conference publication outlets to be at least as significant to is researchers as refereed journals. the findings of this study, particularly the last, contribute to a greater understanding of the nature of is research and help legitimise its publication patterns which will in turn assist to validate its academic performance. these steps are necessary before is research in australia will gain a higher political profile and greater access to resources acknowledgements i would like to thank james alexander of the department of psychology, university of tasmania and dr pauline o'connor of the department of computer science, james cook university for their helpful advice regarding statistical analysis of the data. i am also indebted to dr chris keen, associate professor glenn lowry and other members of the department of information systems, university of tasmania for their valuable suggestions on an early draft. co rso> *"̂ "«5 •w q) o ^3« i ^^^ u) 0) o) to 00 ^j 0 ^m^ co cd <n t"5 q) 0) a aa jo o x *^s c3 tn 08 c o ,0) o i c ur re nt s tu dy -•:5 . •ft 2 •ai a> £ a, i m er rib e • wo ir•:.;':•: s 1 •£ 0) ^£ v q i« •q ffl s 'm ¥-<• '•• ''f': •t™£ ^ t*•r| m is q ua rte rly _ ? 2 i e 2 <5 c o u "o 8 i 3 to c cd f •q cd q o c i in fo rm at io n s ys te m s r es ea rc h ^ tt| m an ag em en t s cie nc e ^ •r^ i c om m un ic at io ns o f t he a c m to c 'co ra co t3 cd c 2 o c o e~ 0 « a ? §1 t3 t s ur ve ye re nc es n o £ 1 cd cd" co i to cd o c cd £ c o u t3 s. cd £ 3 co o c to cd o c cd _2 "c o u in te rn at io na l c on fe re nc e on is cm cm icm i a c m c om pu tin g s ur ve ys •: '''•: •;•:•'' -.-.. ' . %;. i?:': '.;" i •x jo cm cm rcm cd co 1 ie ee tr an sa ct io ns on s o e ng in ee rin g cm cm tcm cd co « 1 ac m tr an sa ct io ns on d at i s ys te m s cm cm -cm i h ar va rd b us in es s r ev ie w cm cm -cm i s lo an m an ag em en t r ev ie w co c co o c t3 cd co o c 0> c co 1 to i a ca de m y of m an ag em en t j ou rn co 1 cd 1 •o 1 co o c t3 <£ c cc c c j i a ca de m y of m an ag em en t r ev ie cm tcm i jo ur na l o f m is cm r*cm i d ec is io n s ci en ce s co ,ico i jo ur na l o f c om pu te r i is 1 ^ j y _« to c co 01 co 1 c co o c to cd u ô to ii 8.1 •d cd a? to re nc es n o cd §u t3 cd 1 o c co cd u c cd cd "c 8 tj §. cd to o c to 8 cd cd § o t eo •2 0 1— in te rn at io na l fe de ra tio n in fo rm at io n p ro ce ss in g (if ip ) w or ki ng c on ff. "8s. c 2 co £ 3 o s.s 1 € o z o c to 75 e iri ca n ]o u to rth a m e a 6 t3 cd l£ co co co | o c1 cd 1 o 2 -0 s^ c. s co co § c 1 cd £ z 1 i e ur op ea n jo ur na l o f iis co •o cd c 2 c cm "8 co o c i o pe ra tio ns r es ea rc h co t3 cd c co o c cm t3 » 1 o c i d a ta b a s e to c 'co c» co jc c co o c co cd o § <£ •d t s ur ve ye o c 8 o c cd £ 1 •o cd co i tos cd cd "£ 8 13 §. 0> 3 co o c co 0> u c cd £ o u l_ ^ **"" o1 in te rn at io na l fe de ra tio n in fo rm at io n p ro ce ss in g (if ip ) c on gr es s cm •d 1 co o c "s £* c co c5 c i d ec is io n s up po rt s ys te m s cm tcm i in fo rm at io n an d m an ag em en t co c co 1 *co i jo ur na l o f is m an ag em en t to c co 0? cd .*c co o c 8 § to 11 8.1 t3 t s ur ve ye re nc es n o cd 8 •o cd c co o c to cd o c cd £ 15 8 •o§. cd £ 3 <0 o c to cd cd 8 <n \ a m er ic a' s c on fe re nc e on is ( al to c 'co o> co 1 co 0 c 8 c 0) ot £ c? 11 •q t s ur ve ye re nc es n o cd 8 ĉd c s to co 1 o c 1 cd 1 ^* o 1 oe 3 co c co cd £ cd £ i e ur op ea n c on fe re nc e on is to c co co 1 c co o c o fcd to ii 11 •a t s ur ve ye re nc es n o cd 8 ^3 cd § o c co cd o § £ 1r 8 1 cde 3 co 1 co cd £ cd £ 8 c o cd ti h aw ai i in te rn at io na l c on fe re ni s ys te m s s ci en ce ( h ic s s ) 2 1 co cd 1 g 8 si « ih §1 t3 cd >, cd | co 'o cz fe re nc es c o u s.* c co o c co 'e re nc e i tj cd1 co o c u c cd 1 co 8 1 o o p s la c on fe re co t3 a c e c c eg a s c cz 1 o m e g a 1 3 co ! 5 o 1 r* cz o s cz co co co £ 3o 1 cd / n or th a o •d a c e co a e c c co c a <c t "c t3 a c n a "5 £ c a ^t a 4 £ tz_c 2 "c co b 1 a us tra lia n j ou rn al co co ,co 'c a e 0 c 0 c a co 1 jo ur na l o f s vs te m % 15 3) co -o cd s i §s «ii 8° •o cd | 1 fe re nc es u t3 cd c s co to j^ j§ 1<c 1 6 •g>, cd 2 3) ! co cd cd cd 1 ^^co o co c o er en ce a us tra la si an c on f 1 b co c 3 o ne ric an < ^ cz o 1 c 2 co co £ 3o 1 co y n or th a c •d a c 2 .2 ct e c c 1 a ^ ^ i "c •̂ 0 c5 u a | c c cl c a 4 t̂z o 2 "c 7 £ j c 1 a us tra lia n c om pu 1 0 co i 3 s 1 d e cz co co to c 3o 1 cd / n or th a c •oa c 2 "cc c c c i a ^ € ?; •£ 0 cz 2 "5 e c c a c a <i t̂z c ^"c , cc c; g c co cz 0 75 ti c j c ^ t | 5 o) co 5 8 1 su i « 11 8.i -q cd>, cd 3 ca o c fe re nc es c o o •a cd.*c 2 o c co 8 i cd i "s>, cd | co o c co cd u cz £ cd § u ^ co * — "c oo o z3 1 d ec is io n s ci en ce co cm cm tja cz 2 tc 5 f 1 in te rfa ce s (o r sa 2 1 co t3 ]> 5 5 i1 « §1 -jj cd cd 3 co 0 fe re nc es i cd c 2 o c o i £ "c o o •o cd £ 3 co o c co cd 8 £ 1 co § cd u c £ 8 o co °-co i! 1 d) co i 3 i co <d cj jl t3 cd >, cd | o fe re nc es c o u cd c 2 o c co c cd & o o to>, cd | co 1 co cd 8 £ i co e cd s,co t:a 0. d ec is io n s u c on fe re nc e( d s s ) 0 5 5° t3 d 5 5 co 1) 5 w i i •o cd ĉd 3 co 1 fe re nc es 8 ti.* 2 o cz co fe re nc e 1 cd cd £ 3 co 1 co cd o cz cd £ 8 8 1 cd o r s a /t im s c on f .2 1 en c5 '5 •̂c oz ig s ^ _c 1pg .« •s gi •e cs mm e 0 u n (u « f"r 80 references alavi, m. & carlson, p. (1992) "a review of mis research and disciplinary development", journal of management information systems, vol 8, no 4, pp 45-62. amott d. (19%, aug 15) "discipline research strategy for ft", is-depts [online], available: is-depts@filquledu.au. australian research council (1995) report on research funding programs 1995 volume 2 grants and fellowships awarded, department of employment, education and training. bennett, j. m., broomham, r. murton, p. m., pearcey, t. & rutledge. (eds) (1994) computing in australia, sydney: hale & iremonger pty limited for australian computer society inc. cheon, m. j., grover, v. & sabherwal, r. (1993) "the evolution of empirical research in is: a study in is maturity", information & management vol 24, pp 107-119. culan, m. (1986) "the intellectual structure of management information systems, 1972-1982: a co-citation analysis", management science, vol 32, no 2, pp 156-172. culnan, m. j. & swanson, e. b. (1986) "research in management information systems, 1980-1984: points of work and reference", mis quarterly, vol 10, no 3, pp 289-301. davis, d. and cosenza, r. m. (1988) business research for decision making, 2nd edn., boston: pws-kent pub. co. doke, e. r. & luke, r. h. (1987) "perceived quality of cis/mb journals among faculty: publishing hierarchies", journal of computer information systems, vol 28, no 4, pp 30-33. farhoomand, all f. (1992) "scientific progress of management information systems", in information systems research: issues, methods and practical guidelines, (ed) robert galliers, oxford: blackwell scientific publications. galliers, r. (1992) preface in information systems research: issues, methods and practical guidelines, ed galliers, r., oxford: blackwell. gillenson, m. l. & stutz, j. d.(1991) "academic issues in mis: journals and books" mb quarterly, vol 15, no 4, pp 447452. grover, v. cheon, m. and sabherwal, r. (1993) "the evolution of empirical research in is: a study in is maturity" information and management vol 24, no 5, pp 107-119. grover, v., segars, a. r& simon, s. (1992) "an assessment of institutional research productivity in mis" data base, vol 23, no 4, pp 5-8. hamilton, s. & ives, b. (1982) "mis research strategies", information and management vol 5, no 6, pp 339-347. hamilton, s. & ives, b. (1983) "the journal communication system for mis research", data base, vol 14, no 2, pp 314. holsapple, c, johnson, l., manakyan, h. & tanner, j. (1993) "a citation analysis of business computing research journals", information and management vol 25, no 5, pp 231-244. jackson, w. m. & nath, r. (1989) "publication patterns of mis researchers", interface, vol 11, no 2, pp 15-220. keen, c.d. (1995) australasian information systems survey 1995, hobart department of computer science, university of tasmania. keen, p. (1991) "relevance and rigor in information systems research: improving quality, confidence, cohesion and impact" in information systems research: contemporary approaches and emergent traditions, eds he. nissen, k. heinz & r. hirschheim, amsterdam: north-holland. king, jl. (1993) "editorial notes", information systems research, vol 4, no 4, pp 291-298. koong, k s. & weistroffer, h. r. (1989) "faculty usage of management information systems journals: a survey", journal of computer information systems, vol 30, no 1, pp 1-4. lowe, l (1995) "data points to research split in so-called "unified system", campus review, mar 23-29, p 9. morrison, j. & george, j. f. (1995) "exploring the software engineering component in mis research", communications of the acm, vol 38, no 7, pp 80-91. nord, j. r & nord, g. d. (1995) "mis research: journal status assessment and analysis", information and management vol 29, no l,pp 29-42. omar, m. r & goodwin, j. s. (1991) "an investigation of decision support systems literature", interface, vol 13, no 2, pp 18-23. orlikowski, w. j. & baroudi, j. j. (1991) "studying information technology in organizations: research approaches and assumptions", information systems research, vol 2, no 1, pp 1-28. pollard, a. r (1972) introductory statistics, rushcutters bay, nsw.: pergamon press. preece, j., benyon, d., davies g., keller, l. & rogers, y. (1993) a guide to usability, wokingham: addison-wesley. shanks, g., rouse, a. & amott, d. (1993) "a review of approaches to research and scholarship in information systems", working paper series, 3/93. department of information systems, faculty of computing and information technology, monash university. 81 tsai, r., richards, t. & yellen, r. (1991) "practitioners' issues: perceptions of information systems journals and issues", interface, vol 13, no 3, pp 65-69. vogel d. r. & wetherbe, j. c. (1984) "ms research: a profile of leading journals and universities", data base, vol 16, nol,pp3-14. walstrom, k. a., hardgrave, b. c. & wilson, r l. (1995) "forums for management information systems scholars", communications of the acm, vol 38, no 3, pp 93-107. appendix 1 rating instrument for those conferences and refereed journals that you consider to be appropriate as a publication outlet for australian is research, rate the following using the rating scheme given below (adapted from walstrom, hardgrave & wilson, (1995). if you consider the journal or conference to be inappropriate as a publication outlet for australian is research, leave its line blank. 1 = very low quality 4 = high quality 2 = low quality 5 = very high quality 3 = of about average quality mark an x where you are not familiar enough with the journal or conference to be able to rate it academy of management journal oopsla conferences academy of management review operations research acm computing surveys orsa/ems conference acm transactions on database systems pan pacific conference on is (pacis) america's conference on is (ais) sloan management review australian computer journal australasian conference on is (acis) australian journal of information systems australian journal of management communications of the acm database decision sciences decision science institute conference (dsi) decision support systems decision support systems conference (dss) european conference of information systems european journal of information systems harvard business review hawaii international conference on systems science (fflcss) ieee transactions on software engineering international federation for information processing (ifep) tc8 working conference international federation for information processing (mp) world congress information and management information systems research interfaces (orsa/tims) international conference on is (icis) journal of computer information systems journal of information systems management journal of mis journal of systems management management science mis quarterly omega if you can, suggest any additional noteworthy journals or conferences. rate their quality using the scale given above. ajis vol. 3, no. 2 the role of e-mail in an industrial dispute nava pliskin • & celia t. romm»* 'department of industrial engineering and management, ben-gurion university of the negev, beer-sheva, israel. fax: (07)280776 , tel: (07)472203. e-mail address: npliskin@black.bgu.ac.il "department of management, university of wollongong. wollongong, nsw, australia. fax:: (042) 272785, tel: (042)214046, email address: c.romm@uow.edu.au abstract the purpose of this paper is to show, based on a case study, how electronic communication networks have affected the process of a large scale industrial dispute. the case study describes a two-and-a-half month long strike of academic staff members at all universities in israel. during the strike, electronic mail (e-mail) was used as a major means of communication between faculty members. sources of data for this paper were mass-media reports, including reports from national radio, tv, and newspapers, participant observations, and messages that flowed on the e-mailing list throughout the lengthy strike. analysis of the data from the various sources suggests that the c-mailing list was instrumental in enabling the strikers to succeed in what was presumed to be a hopeless battle. the paper is concluded with a discussion of the lessons to be drawn from this case for e-mail research and practice theoretical background e-mail is a communication medium which can facilitate exchange of addressed messages between computers via electronic transmission paths (culnan and markus, 1987; gaukroger, 1988). much of the early research on e-mail focused on its diffusion process. the vast majority of the early papers tended to look at the diffusion process of e-mail as a dependent variable, i.e., as a process that is caused by other social phenomena. within this perspective, early papers tended to attribute successful diffusion projects to technological aspects (pliskin, 1989; pliskin, ball, and curley, 1989). it is only in the last five years that non-technological explanations, individual and collective, have started to receive attention. lynne markus, in a recent paper on e-mail use (1994), contrasts individual and collective level explanations for e-mail diffusion (or lack of it). among the individual-level obstacles for diffusion she lists: fear of change, lack of user involvement, lack of motivation, and difficulties related to user cognitive style. among the collectivelevel obstacles to diffusion she mentions: avoidance of e-mail adoption due to cultural reasons, situational factors, and structural pressures from other organisational members. markus concludes that individual-level considerations are not enough to explain why members of organisations choose e-mail for some of their communication tasks. she strongly recommends that future research should explore the collective nature of email use. it is only in recent years that the collective nature of e-mail use has begun to be researched extensively. in a review by rob kling (1995) he singles out several studies as the major contributions in this area. the first among these studies is sproull and kiesler's book, "connections" (1991), in which they set the stage for examining e-mail in organisations. these authors were among the first to argue that e-mail has social features that distinguish it from other communication technologies, i.e., it enables people who are at the periphery of organisations to become more visible, and facilitates communication between people at the bottom of the organisational hierarchy and those at the top. in these ways e-mail can actually have a "democratising" effect on organisations. a related pioneering work on the collective-level aspects of e-mail was undertaken by finholt and sproull (1990). in this research they investigated how e-mail can facilitate group decision making and bring about group unity and cohesion. rice's series of investigations ( rice, 1987; rice, 1990; rice and aydin, 1991; rice, 1992; rice, 1993) have also been important in demonstrating the effect of networks on group behaviour in the workplace, with particular emphasis on how membership in networks affects members' attitudes about the new technology, and promote group innovation. the purpose of our paper is to build on the growing body of research on the collective-level aspects of e-mail by exploring the effect of e-mail on the successful outcome of an industrial dispute. the main focus of the paper is on the impact that an e-mail list, established by a group of strikers to support their struggle, had on the process and outcome of the strike. the qualitative methodology used in this study (detailed in the next section) was case analysis. the case study (in section 3) describes a two-and-a half months long strike that was the culmination of a much longer struggle 47 a,|is vol.3, no. 2 for improved pay and status of faculty members at all state universities in israel. the industrial dispute started before the strike and ended only after a two-and-a-half months long period of post-strike negotiations. the final contract between the union and the universities was signed about five months after the strike began. the discussion (in section 4) elaborates on how the e-mailing list, which was established by the academics to support their struggle, enabled them to initiate and manage the simultaneous negotiation processes that were involved in this large scale labour dispute. the discussion suggests that, in retrospect, e-mail can be seen as an important "weapon" which enabled the strikers to maintain their unity in the face of extreme pressures from all directions. the discussion is concluded with suggestion for directions for future research and implications for is policy makers and practitioners. data collection data for this study was collected by the authors. one of the authors, a faculty member at ben-gurion university of the negev, was able to gain access to materials that would otherwise be inaccessible. sources for data included: the mass media, participant observations, written communication distributed by the universities and the unions, and e-mail messages which appeared on the various e-mail lists established by the strikers. the different sources of data were complementary and mutually supportive. both authors were involved in all data analysis activities and attempted to reach agreement whenever findings or their interpretation of the findings were in dispute. mass media the media was highly involved in the strike. both sides to the dispute tried to harness the media to strengthen their bargaining position. the authors collected many newspaper reports and followed radio and television coverage, including interviews with strike activists, students, and other involved figures. participant observations one of the authors participated in the strike. in her capacity as a striker she attended all faculty meetings called by the local union and received all memorandums distributed before and after such meetings. impressions from those observations were shared with the other author soon after the observation took place. e-mail messages electronic communication emerged early during the strike in the form of several local e-mailing lists. faculty members were encouraged by local unions to subscribe to academia, an e-mail-based user group which was opened to all interested faculty members. academia provided a forum for exchange of ideas and distribution of information from strike activists. one of the authors subscribed to academia, forwarding all messages (over 600 messages were collected and analysed) to the other author. both authors engaged in textual analysis of all electronic messages exchanged on the network. case study general background in early january 1994, shortly before the end of the fall semester, faculty members in all seven israeli universities initiated a strike, suspending all work related to the teaching of 90,000 students, including: lectures, labs, exams, grading, and counselling. the strike was declared by the co-ordinating council of faculty unions (co-ordinating council, hereinafter) which included representatives from each of the seven local unions. other parties who were involved in the dispute were the treasury ministry, the committee of university presidents, and the planning and grants/budgeting committee of the council for higher education (singer, 1994). the main reason for the strike was that salaries in the academic sector eroded continually over the decade which preceded the strike to well below other salaries of researchers in the public sector. this had been the situation despite many previous faculty strikes, which were relatively brief and did not yield satisfactory results. a two-week long strike, declared by the co-ordinating council at the end of 1992, led the committee of university presidents to recognise that the entire israeli higher education system was at stake and to nominate 48 a.iis vol. 3, no. 2 an objective commission charged with in-depth investigation of faculty reward structure. the commission's report, filed toward the end of 1993, recognised the need to increase salaries by 30%. the commission also recommended tactics for executing this policy, including linking faculty salaries to nation-wide salary index (to prevent salary erosion in the future) and funding raises by reduction of sabbatical privileges. both sides, however, rejected parts of the report. the treasury was unwilling to go beyond the 20% raise offered to others in the public sector and refused to link faculty salary to any index. the co-ordinating council demanded more than 80% increase and objected to any reduction in sabbatical benefits. all parties involved in the strike did not expect it to be long. based on many previous instances in which israeli academics went on short strikes that did not yield any substantial results, the expectation was that this strike too will be short and ineffectual. consequently, the fact that the strike lasted 75 days and resulted in a 70% increase to academics' salaries was a surprise to all. the remainder of this section will present the events of the strike as they unfolded. the remainder of this section will present the events of the strike as they unfolded chronologically. sources of data for this discussion included the mass media, e-mail flow, and participant observations. events are presented in sequence following the three major stages of the strike, initiation, escalation, and resolution. events of the strike initiation the media's involvement in the strike started on its first days. both sides to the dispute used the media to gain public support for their position. each side posted announcements in newspapers and gave interviews that were broadcast on radio and television. the heavy coverage by the media also included daily reports on the strike, with headlines that included a day count since the strike started. newspapers also carried editorials and letters to the editor from striking faculty members, students, and interested members of the general public. the public was exposed to frequent statements issued by the treasury ministry, attempting to prove that the cause of the strikers was unjustified and that faculty members were being irrational, irresponsible, and impossible to negotiate with. for instance, in order to show that faculty salaries were fair, the media was informed that faculty members only teach eight hours per week. the strikers were also involved in the war of words. in the early stages of the strike they too devoted resources to combat public opinion. for instance, interviewed faculty members revealed that teaching involves twice as many hours of preparation and grading, and that promotion decisions are mainly based on publication output, rather than teaching, forcing faculty members to invest much more than eight hours per week in their work. the co-ordinating council took pan in the public debate by providing the media with tables, underlining the gap between faculty salaries and salaries of other researchers in the public and private sector. even though the strike received much attention from the media, the public was not exposed to the information that flowed internally among faculty members within the israeli campuses. shortly after the strike started, an email-based user group entitled academia was created. in addition to word of mouth and announcements during faculty meetings, the existence of academia became known to the strikers electronically. it was advertised through e-mailing lists established by the local unions on each campus, even before its creation. the academia e-mailing list distributed messages from the co-ordinating council and messages which originated from academia subscribers. local mailing lists distributed messages from their own constituency, from the local union, from the tel-aviv union, and from the co-ordinating council. thus, while academia subscribers were exposed to more strike-related e-mail traffic than non-academia subscribers, it is safe to assume that non-academia subscribers who subscribed to local e-mailing lists at their campuses were exposed to the major messages distributed by the co-ordinating council as well as to the tel-aviv bulletins. it should be noted that even though information from the co-ordinating council (as opposed to inaccurate information published by the media), was available to strikers almost in real time, initially, the local e-mailing lists and academia were perceived by union activists as primarily a one sided channel of information. this was reflected in the fact that in the messages transmitted during the first weeks of the strike (all of which were from the co-ordinating council) there were no requests for strikers' input, suggestions, or opinions. it was only later, as the strike lingered, that both the co-ordinating council and the strikers realised that the network could be used for many purposes other than top down dissemination of information. following is a typical example of the early "report type", one sided messages issued by the co-ordinating council.. 49 aj1s vol. 3, no. 2 17th day dear comrades, yesterday was too tiring (13 hours devoted to the strike) for me to sit down and report. here is a brief summary for your information. discussions in our "bunker" are getting into swing. we are becoming organised. help is welcome at any time of the day. a circular letter to all members of the union from us has been dispatched. it includes "real data" about our salaries, as opposed to the false data disseminated by the other side. make as much use of it as possible. we are trying to complete the "network of internal communications" within our university and other universities as wellif you know of isolated faculties, universities etc. let us know and eventually suggest people who could be used as "contacts" for them. escalation within the first month of the strike it became apparent that neither party to the dispute was likely to back down easily. the escalation of the struggle involved high-level figures including: the prime minister, the minister of education, the minister of finance, and many members of the parliament. the prime minister spoke against the strikers, backing the minister of finance who threatened to close the universities. parliament members tried to bring forth a compromise solution but were unsuccessful. as the strike continued, the student unions became increasingly vocal about the strike and its grave implications to them. their protest demonstrations, some of which were quite violent, conveyed the message that they were the innocent victims of the strike. in addition to requesting an end to the strike, students also demanded passing grades for fall semester subjects. in confronting their opponents, the strikers turned to a consulting company whose speciality was public relations. the firm was retained by the co-ordinating council throughout the strike, advising members of the council on ways of dealing with the situation. the faculty meetings, which were called by local unions every other week to report on (lack of) progress, were exposed to the public eye through the media. in every meeting, the local representative of the co-ordinating council asked for a vote of confidence, which was always unanimously forthcoming. as the struggle was escalating, the use of the e-mailing lists evolved and became more sophisticated. many of the informative messages from the union were now written immediately after latenight negotiations and were available to strikers early in the morning. this helped strikers cope with attempts by the other side to break their unity by distributing false information. messages from the union now reflected the fact that the list had transformed from a one sided channel of communication to a multi-channel forum of discussion and debate. following is a message transmitted by the co-ordinating council at this stage for the purpose of responding to the on-going debate on strategy. 31st day friends. people are sending us interesting and important ideas for things to do. i will mention two ideas that seem worth pursuing. first, the labour party is holding a meeting next week in tel-aviv. all the party ministers will be there. we should stage a demonstration in front of the building. our experts on demonstrations please pick up the idea and go with it. second, we might consider organising an "alternative university system". similar to what the physicians did in their strike (they established an alternative privately financed hospital system). does anyone know how to do it? we need volunteers to develop this. another use of e-mail by the unions was to minimise the effect of tactics employed by the other side in order to break the strike. for instance, strikers were electronically notified of arrangements made by the union with the banks to provide loans to faculty members whose salaries were withheld by the universities. following is an example of such messages: 45th day dear comrades. it seems that the top political figures are in panic. let's stand strong and united. on another matter. i just returned from my bank. they have special loans for the faculty members that are on strike. their loan is for 12 months with a fixed interest of 13%. return of the loan starts two months from receipt. in 50 ajis vol. 3. no. 2 addition you can increase your credit line from $3000 to $6000 with an interest of 15%. if necessary, our representative can sit with you and help those with problems. keep the faith. as a solution to the dispute did not seem eminent, the use of e-mail as an important communication vehicle spilled over to nonisraeli institutions. the e-mailing lists were now being used to recruit support for the strike from academics around the world. the strikers believed that introducing international pressures would have an effect on the government, which was at the time very sensitive to world public opinion due to the peace process. a long message, entitled "crisis in israeli higher education" was put together, detailing the state of academia in israel in general, including salaries and faculty-to-student ratios. academia subscribers were asked to share the "crisis in israeli higher education" file with their colleagues overseas and to encourage them to respond. indeed, academics from all over the world were moved to write supporting letters, addressing them, as requested by the council, to leading politicians in israel, including the prime minister. following are two examples of these supporting letters as they appeared on academia. 42nd day is there a way that friends here in the us can actively help you? i'm wil l ing, as i am sure many of my colleagues here are, to do what we can to help. reading the messages on the internet help us understand. but they don't directly help the cause. peter. us. 46th day a copy of a letter sent directly to the israeli prime minister having taught as a visiting professor of comparative literature at the hebrew university of jerusalem and at tel aviv university, i am extremely dismayed by the erosion of higher education in israel. 1 feel that the issue reaches beyond the question of israeli cultural identity. it involves, indeed, the future of a whole region of the world where peace and those human values necessary to sustain peace must come to prevail if the rest of the world is to address the new imperatives of global politics that are emerging after the cold war. i have learnt from experience that israeli universities are centres of intellectual excellence and of enlightened thinking that make them invaluable to a larger international community. it is in the perspective of this larger community that a new generation of leaders as world citizens must be formed if israel is to endure in years and centuries to come. i believe that the israeli government should listen carefully to the concerns of scholars throughout the world who support the cause of higher education in israel, (name) university of washington. it is important to note that as the dispute escalated and as strikers developed confidence in the e-mailing list, messages from strikers (as opposed to activists) became prevalent. the e-mailing lists were used by nonactivists to discuss objectives of the struggle, methods for achieving these objectives, and ways of increasing the effectiveness of the strike. in particular, the issues of securing supportive public opinion nationally and internationally and the means toward this goal were discussed in much detail. also important in the public debate were discussions of negotiation strategy, i.e., issues that should or should not be compromised on. following are three messages which reflect the heated public debate on e-mail on how the struggle should be managed. 44th day it was reported on tv this morning that we are ready to settle for 40% over 4 years. to those of you who are not familiar with finances (i am just an amateur myself), let me explain the actual meaning of this offer. consider mr. x who is right now having our salary. mr. x will not get any real raise over the next 4 years. his salary will be monthly updated according to the rate of inflation which will actually give him an addition to his salary of over 40%. in fact settling for 40% over a period of 4 years actually means giving up some of the money that we would have gained anyway. this is totally ludicrous!!! 51 a.iis vol. 3, no. 2 44th day why is 40% in three years plus indexing and without giving up any benefits that we now have so bad? perhaps a "salami" (step by step) approach is better? in 1997 we'll be able to renew the battle. somebody really thinks that we will indeed get a 102% raise right now? whoever thinks so, must be nuts!! 46th day i wish to point out that the calculations of the "average salary of academics in four years" is highly inaccurate. when you take into account seniority, the figures are much higher than the $1200 quoted by the member of the council. my colleagues' suggestions that we should compromise are not as ridiculous as the council is trying to make us think. in fact, the last few letters expresses a widespread unease that many of us apparently feel about the way in which our leaders (mis)lead us. resolution during the last phase of the strike, it became clear to all parties involved that if the dispute was not resolved within days, the school year would be lost. the co-ordinating council persisted in telling the public that, for the first time in the history of the state of israel, one semester, or even a full academic year is about to be lost. the uncertainty surrounding the academic school year was of major concern to the students and their families, who together constituted a large segment of the israeli public. by now, students became more aggressive, attempting to disrupt non-teaching activities that the academics continued despite the strike. student demonstrations were primarily aimed at blocking the entrances to campuses in order to stop work on research projects. for instance, several professors received threatening phone calls, and there were attempts by students to interrupt faculty meetings. students were not the only group trying to interrupt non-teaching activities of faculty members. when the ministry of finance announced that funds will no longer be transferred to the universities, forcing the universities to close down, nonacademic employees were affected as well. their unions which at the beginning of the strike were supportive of the academics, were alarmed about the situation as it meant loss of income to their members. this caused additional pressure on the strikers as well as on the government. when the israeli president offered help, summoning the parties to the dispute in an effort to bridge the gap between them, he was asked by the prime minister to back off. from this point on, however, not only the strikers but also the government felt under pressure to produce results. with the president's involvement and the rift between the prime minister and the president, the dispute was developing into a political crisis that called for a much more serious approach by the government. another source of pressure to end the strike was related to the massacre in hebron and the fact that, for a while, the peace process was halted. it made the strikers realise that they were not that important in the general scheme of things. it made the government realise that, following the hebron events, the futile struggle with the academics was an embarrassment and that to please public opinion locally and internationally the strike had to end. the hebron massacre has changed the two sides' perspective. the strikers realised that they had to compromise because public opinion was moving away from them to more important issues. the government realised that with public opinion in the us going against israel, and with the peace process being threatened, the conflict with the academics, who by virtue of their profession have better contacts in the us than any other employee group in the country, had to be resolved immediately. following are a few messages which appeared on academia during the last days before negotiations leading to the resolution of the crisis were resumed. the messages reflect the mounting pressures on the council from the members of the union, the various other stake holders (students, administrative staff) and other local and international interested parties. 47th day dear comrades. the weeks before the holidays will be decisive, since it is clear to everybody that after the holidays, it will be too late to salvage the academic year. we need now more patience and resilience. on the public front colleagues have complained that the strike disappeared from the news in the course of the week. it is likely that in the wake of the hebron tragedy, it will be even less in the public awareness. we shouldn't 52 a.i is vol. 3, no. 2 worry too much about this, however. in all likelihood, the students and the media will realise the gravity of what is at stake (the fate of the academic year) and then suddenly the subject will return to the media in ful l force. finally, about our network. 54th day here is an interesting question raised in my department. are we working or are we not? for sure we are not getting a salary. so the question is are the universities paying our other benefits? for example, i come to work daily. what will happen if on my way to work tomorrow i get involved in a car accident? is my insurance still paid for? do the universities continue to pay for our pension funds? those who pay for medical insurance through the university, should they go and pay personally, or is it taken care of? is anybody in the union giving any thought to these issues? is anybody taking care of us? 6()th day i just received a phone call from a concerned colleague. someone called his home this morning, shouted "death to the professors" and hung up. it is not surprising that some frustrated students should vent their frustration this way. however, coupled with the violent and abusive verbal behaviour of the students who disrupted the tau senate meeting last week, it may be a cause for concern. there is little we can do to counter such phone calls except, perhaps to monitor their frequency. thus, i am volunteering to keep track. please let me know if you receive any abusive calls or if someone you know receives one. 61 st day it seems to me that our pm, several members of his cabinet, many members of the parliament and top administrators in the finance ministry are very much against any reasonable settlement with uswith no real, clear-cut reason. while their explanations for their position make no sense, their public announcements suggest a strong and unexplained, emotionally based, and absolute, aversion towards negotiating with us. we need a different approach to these people, one based on a psychological analysis of their personality. such analysis may produce better results than the total deadlock that we are facing now. as the above messages suggest, events at this stage of the strike created an atmosphere of "now or never", with all parties realising that time was getting short and was not on any body's side. the two sides were thus open to suggestions and receptive to innovative approaches. with this background, two nonactivists faculty members came up with a proposal that for the first time was acceptable to both sides. based on this proposal, an agreement which brought the strike to an end was drafted. during the final stages of the strike, the e-mailing lists served several important purposes. the most important role was probably in boosting morale. numerous messages transmitted by the co-ordinating council were aimed at promoting and maintaining faculty solidarity and support. many non-activists used the emailing lists to distribute messages that not only supported the union but also encouraged fellow strikers to keep the faith and to believe that time is on their side and that the other side will not be able to take a very long strike. following is a typical message from the council, incorporating information on the progress of the negotiations with comments that are intended to boost morale. 71st day yesterday and today's round of negotiations was possibly the longest in the history of the present dispute. from 3.30pm to 5.00am this morning we were sitting together. all these long hours were dedicated to the discussion of the criteria that will entitle faculty members to receive the special addition to their salary. the other side wants criteria that will accomplish two things at once: differentiate clearly between academics and other groups, differentiate among the academics themselves. we, on the other hand, contend that the present task is only to define how we differ from others. for we see the special addition as a way for the government to give us the 60% increase to our salary without causing the domino effect the government is so concerned about. we believe that differentiating between academics is something that should be done by academics and at this point should not be discussed at all. i know that many of you are tired of this long strike, but hold on. keep the faith. justice is on our side and we will win this war. we have to. 53 a.i is vol. 3, no. 2 the e-mailing lists have become an essential life-line, gaining an unprecedented level of user awareness and appreciation. many messages were devoted to describing how important the network has become to users. when employers threatened to cut off the electricity, the e-mailing lists were used to consider alternative means of communication. messages exchanged at that time reflected an awareness that interruption of e-mail services could be a fatal blow to the strikers' ability to keep fighting. following are several short messages, reflecting strikers' dependence on e-mail and the important role they felt e-mail was playing in the strike. note that the first letter is being written in the dark, by an activist who feels that sharing his views with fellow strikers even under these extreme conditions is essential. 65th day fresh news: today at 12.00 the administration workers shut down the electricityno electricity!! i am writing from the computation centre where we have a bit of light, but most of the other places are dark. i talked to some students a few hours ago and they are organising some demonstrations here too. they burned some tires earlier... the academics are amazed and lots of tv and radio people are here too. there are many arguments in the corridors between administrative staff and academics. at least we too (at bgu) got some action now. bye for now. i gotta get back to the centre of events. 71st day it looks as if the universities will be totally closed soon. with e-mail not available as a means of communication, we have to establish alternative ways of communication with our colleagues. stay tuned for information on emergency arrangements for the union operation (new office, phone, fax numbers etc.) at this point, it is absolutely crucial not to rest during the coming holidays but to conduct a well concerted and clever pr campaign to take full advantage of the irresponsible conduct of the negotiations which led to the present deadlock. they (the other side) are fully responsible for not allowing an agreement that was on the brink of being achieved. 71st day the universities will be closed this weekend and as a result most of the computers will shut down. accordingly, we must establish a communication procedure that is based on home phones. please start a communication chain with five members at the top for each faculty/school and then one or two members for each department. please start establishing this chain as soon as possible. keep the faith and stay tuned for more information during the day. as pressure from all directions to stop the strike mounted, the e-mailing lists were increasingly used to discuss the leadership of activists. this included support for leaders, on the one hand, and complaints about lack of progress because leaders were not doing enough or were not doing the right thing, on the other hand. some emailing list subscribers complained that the struggle was not managed well and even demanded a change of leadership. this culminated in a proposal to conduct a referendum among faculty members in order to resolve issues of direction and leadership. the referendum was heatedly debated and finally rejected by the majority of strikers. during this electronic exchange, messages of e-mail included numerous expressions of support for the strike leaders but also a significant number of vicious personal attacks on leaders and their supporters. one implication of the leadership debate was that some subscribers, disgusted by the language used over the network, withdrew from the e-mailing list. this, in turn, led to a discussion of whether the organisers of the list should screen messages and prevent "unacceptable" messages from reaching the whole subscriber group. the debate was still going on when the strike ended. after the strike ended, however, academia was restructured. based on lessons from the message flow during the strike, a limited form of censorship was instituted, with managers of the list screening messages that "were not of interest to the group". discussion and conclusions the description of the three-phased history of the strike in the previous sections was seems to suggest that email played an important role in shaping the events and bringing forth what was hailed by observers as the greatest industrial relations victory in israeli academia's history. when we compare the events of the 1994 israeli academics' strike to previous similar israeli academics' strikes, it becomes apparent that it was characterised by two aspects that made it unique. first, in contrast to previous 54 ajis vol. .v no. 2 strikes which were short and ineffective, this one was very long and highly successful in terms of its outcome for the strikers. the second aspect that made this strike unique was the fact that for the first time e-mail was used as the major communication media between strikers. is there a link between the two aspects? can we claim, on the basis of the data provided here, that e-mail made the strike so successful? although it is a tempting possibility, we believe that the data that we have presented here cannot substantiate such a claim. there are, in fact, several other possible explanations to the unprecedented achievements of this strike. these list of possible explanations include: the excellent negotiation skills of the union leaders, the wide support from members of the public locally and internationally, israeli internal politics (the conflict between the prime minister and the president), and even the tragic events of the hebrew massacre. all of these put pressure on the government to close the gap between its ini t ial position and the strikers'. and, yet, it is our claim that e-mail's role in shaping the strike's events was crucial. in fact? based on the data that was presented in the previous sections, it appears that e-mail played several important roles, including: contribution to unity within the strikers' group the first and most important role that e-mail played in this case was in helping the strikers maintain their unity in the face of pressures from all directions. e-mail was the means through which strikers boosted each other's morale, aired differences, exchanged jokes about their opponents, and eventually reached agreements on issues that were initially in dispute. e-mail was also the means through which strikers offered practical help to members in need and the vehicle through which they consolidated their support of their leadership. the unity that e-mail helped maintain made it possible for the strikers to keep on fighting un t i l the group of stake holders became too large to ignore by the employers and the government. facilitation of communication between strikers and their leaders the role that e-mail seemed to have played in this case is that of a mechanism which closed the communication gap between the constituents and their representative negotiators. it facilitated a high level and quality of communication throughout the dispute, enabling the constituents to debate and argue and yet consistently provide their representatives with a united front that they could work from. thanks to e-mail, the risk of negotiators becoming "separate" from their constituents was minimised. constituents remained united as a group and were consequently able to maintain their support of the negotiators. in that sense, e-mail helped create an illusion of closeness between the constituents and their representatives. this illusion allowed views that were offensive and even threatening to the negotiators (particularly, had they been expressed face to face), to be openly exchanged on e-mail. these included detailed scrutiny of the negotiators' bargaining behaviour and even questioning of the overall legitimacy of their mandate. provision of feedback on the negotiation process. e-mail served as a continuing source of input to the negotiating team on acceptable and unacceptable strategies for resolving the dispute. this role has changed throughout the dispute. in the early stages, the major contribution of e-mail was in debating and then initiating the international support campaign. in later stages messages transmitted on e-mail alerted the negotiating team to solutions that were unacceptable and solutions that were acceptable. for example, the numerous messages from constituents calling for compromise on the last days of the strike may have helped the negotiators agree on the principle of criteria for the "special salary increase" to which they were vehemently opposed at the beginning of the strike. despite the fact that our case study took place in a specific and rather unique t ime and place, it has general implications that can apply to much broader contexts. the first implication has to do with the role of e-mail as a contributor to group uni ty and cohesion. the process by which groups gain a sense of unity and cohesion is well-documented in the organisational behaviour literature in which several models of group development have been proposed (tuckman. 1965; gersick, 1989, 1990). our research suggests that virtual communities, such as the group of strikers in our case, may reach similar levels on unity and cohesion thorough e-mail. future research may explore the degree to which existing models of group development do indeed apply to virtual communities and help identify further the unique patterns that characterise virtual communities as groups. the second implication from this research relates to e-mail's political potential. past research has demonstrated the importance of charging mechanisms to regulate e-mail use (pliskin & romm. 1990). for instance, a liberal 55 a.i is vol. 3, no. 2 charging scheme that rewards new users for using e-mail might increase diffusion. the data from this study suggests that political considerations could affect diffusion as well. in an environment where employees are in conflict with management, e-mail's diffusion can be a result of emplyees' desire to use the new technology as a weapon against management. it remains for further research to explore the ways that e-mail can indeed be put to political uses within and between organisations. finally, the data from this research raises some intriguing ethical dilemmas for is researchers and practitioners. as indicated in the case, e-mail can be used by employees for purposes other than the ones intended by management. indeed, it can be used as a vehicle for staging a revolt against the very people who establish it, whether they are members of management or the leaders of a striking union. what should be the role of is practitioners once e-mail is used as a political tool by either side? should is practitioners facilitate the regulation of e-mail as was the case here when censorship was introduced to academia toward the end of the strike? should is practitioners advise managers against introducing e-mail to an organisation that has particularly volatile industrial relations? should is practitioners side with management when employees decide to sabotage e-mail as part of an industrial dispute? all of these are ethical dilemmas that will have to be addressed by future research on the use of e-mail within and between organisations. references culnan, m. j. and markus, m. l., (1987), information technologies: electronic media and interorganizational communication , in f. m. jablin, l. l. putnam, k. h. roberts, and l. w. porter (eds.), handbook of organisational communication: an interdisciplinary perspective, 420-443, newbury park, ca: sage publications. finholt, t. and sproull, s. (1990) electronic groups at work , organization science, vol. 1, no. 1, pp. 41-64. gaukroger, j., (1988), information technology in the office, mc-graw hill book company, london. gersick, c. j. k., (1988), time and transition in work teams: toward a new model of group development, academy of management journal, 31,1, 9-41. gersick, c. j. k., (1989), marking time: predictable transitions in task groups , academy of management journal, 32, 2, 274-309. kling, r., (1995), social relationships in electronic forums: hangouts, salons, workplaces and communities, lead article for section iv of computerization and controversy: value conflicts and social choices, by r. kling (ed.), academic press, san diego (forthcoming). markus, m. l., (1994), electronic mail as a medium of managerial choice , organization science, vol. 5, no. 4., pp. 502-527. pliskin, n., (1989), interacting with electronic mail can be a dream or a nightmare: a user's point of view , interacting with computers, vol. 1, no 3: 259-272. pliskin, n., ball, l. d., and curley, k. f., (1989), impediments to proliferation of electronic mail: a study from the users' perspective , human systems management, vol. 8, no. 3: 233-241. pliskin, n. and romm, t., (1990), design of charging mechanisms according to the interaction between information technology type and diffusion life cycle phase , database, vol. 21, no. 3: 34-40. rice, r. e., (1987), computer-mediated communication and organizational innovation , journal of communication, vol. 37, no. 4, pp. 65-94. rice, r. e., (1990), individual and network influences on the adoption and perceived outcomes of electronic messaging , social networks, vol. 12, pp. 27-55. rice, r. e., and aydin, c., (1991), attitudes toward new organizational technology: network proximity as a mechanism for social information processing , administrative science quarterly, vol. 36, pp. 219-244. rice, r. e., (1992), contexts of research on organizational computer-mediated communication , in m. lea (ed.) contexts of computer mediated communication, harvester wheatsheaf, england, pp. 113-144. rice, r. e., (1993), using network concepts to clarify sources and mechanisms of social influence , in w. richards jr. and g. burnett (eds.) progress in communication sciences, vol. 12, norwood, nj: ablex. singer, a., (1994) faculty strike in the israeli universities , israel studies bulletin, vol. 9, no. 2, spring , 16-20. sproull, l. and kiesler, s. (1991) connections: new ways of working in the network, mit press, cambridge, ma. tuckman, b. w., (1965), developmental sequences in small groups , psychological bulletin, june, 384-99. 56 using annotated conceptual models to derive information system implementations anthony berglas the university of queensland brisbane 4072 australia berglas@cs.uq.edu.au. abstract producing production quality information systems from conceptual descriptions is a time consuming process that employs many of the world's programmers. although most of this programming is fairly routine, the process has not been amenable to simple automation because conceptual models do not provide sufficient parameters to make all the implementation decisions that are required, and numerous special cases arise in practice. most commercial case tools address these problems by essentially implementing a waterfall model in which the development proceeds from analysis through design, layout and coding phases in a partially automated manner, but the analyst/programmer must heavily edit each intermediate stage. this paper demonstrates that by recognising the nature of information systems, it is possible to specify applications completely using a conceptual model that has been annotated with additional parameters that guide automated implementation. more importantly, it will be argued that a manageable number of annotations are sufficient to implement realistic applications, and techniques will be described that enabled the author's commercial case tool, the intelligent develope to automated implementation without requiring complex theorem proving technology. keywords: information systems, conceptual models, case, extended ni am, reuse, user interface design introduction there has been considerable research into the development of sophisticated techniques that capture abstract requirements and refine them into functional prototypes. good examples include the programmer's apprentice reubenstein (1991) which is concerned with resolving informal descriptions that are ambiguous, inconsistent and incomplete by referencing a library of reusable cliches, and the daida project jarke (1992) which uses advanced knowledge representation techniques (telos) to capture general requirements and map them to conceptual and implementation descriptions. daida also uses a general system of non-functional goals to determine optimum implementations. both of these systems describe transactions using first order logic which requires advanced theorem proving technology. while designing detailed conceptual models may require human like intelligence that is difficult to reproduce, much of the cost of developing applications using conventional case technology is incurred in the writing and maintenance of 4gl code and end user documentation that implements these models. tools such as oracle (1993), or described in ovum (1992) effectively implement a waterfall model in which tools partially automate the mapping from conceptual to logical models and then 4gl code, but the programmer/analyst must extensively edit each intermediate stage. this is an expensive process and makes the resultant applications difficult to maintain. object oriented interface tools such as borras (1992), flynn (1992), and king (1993) facilitate more complex graphical manipulation of objects, but still required programmers to essentially edit generated code. this paper argues that by recognizing the limited types of operations that are commonly performed by conventional information systems, it is possible to largely automate the implementation of production quality applications from conceptual models that contain a manageable number of annotations. although the emphasis of this paper is on how applications are technically implemented, automating implementation can also be used to empower more sophisticated end users to usefully tailor conceptual models of large systems in a controlled manner. the next section describes the types of decisions that need to be made, how rules can be used to automate them, and mechanisms to handle exceptions. the ifep conference example ifip (1982) will then be used to give examples of specific annotations. techniques for working with annotated models will be discussed including the possibility of enabling end users to tailor conceptual models. ajis decisions, rules and annotations the decisions that are required to implement information systems may be classified in terms of where they would be made in a waterfall model, which include:• generating relational schemas: when denormalizations should be introduced; whether derived facts (attributes) should be evaluated eagerly or lazily; whether constraints should be maintained in the applications or an active database; how data should be distributed over a network. • outlining user interfaces: how many user interfaces there should be and the scope of each one; determining which attributes should be manipulated together; controlling end user access to sensitive data. • detailing user interfaces: how many windows should be used; how should the fields be laid out within each window; which widgets1 should be used for each field; how can non-form interfaces be used. • 4gl coding: how to code unclassified business rules and special cases. many of these decisions can be made on the basis of well defined rules. for example, the decision to select an appropriate widget largely depends on the number of valid values a field may have — a "ding bat" is good for two alternatives, "radio buttons" for up to four, thereafter a conventional text field is more appropriate. another rule states that when a screen includes a foreign key, the attribute(s) that identify the foreign entity to a user (ie. the descriptive ones) should also be displayed if there is sufficient room. for example, students' names would normally be displayed next to their number on a class list, rather than their age, say. at the relational level, the use of keyword tables to allow efficient searches for words within strings is a common reason to introduce denormalizations2. the decision to use one could be made by the following rule:if a column is used for sub-word searching, and the number of rows is > = 500 and searching is at least four times more common than updating, and this column is often used as the sole search criteria then create a keyword table. conceptual schemas and dynamic models provide the base parameters required by the rules. some parameters such as an attribute's data type form part of most modeling techniques while others may be introduced to the models using additional annotations. typical annotations indicate which attributes describe foreign keys, how and when derived attributes are calculated, and when keyword searching is useful. annotations can be loosely classified in terms of their "conceptual purity" as follows:• conceptual constraints that are pan of a niam model, such as uniqueness and exclusion constraints. • structured business rules that enforce more complex constraints. • conceptual parameters that are independent of specific implementation techniques, but do not directly affect conceptual semantics. examples include specifying which attributes are descriptive, which uniqueness constraint defines the primary key, or whether a subtype is collapsible3. • overrides to the standard derivation rules. these can be quite specific, for example, to specify that a particular attribute on a specific interface should be represented using radio buttons rather than the default ding bat • ad hoc 4gl code attached to interfaces. this classification is orthogonal to the waterfall approach used for decisions, and the more conceptual annotations may be used by several decisions. for example, an enumeration of the valid values an attribute may contain requires an integrity constraint to be created during table definition while also affecting the widget used in the user interface. 1. ie. low level user interaction mechanisms, eg. scroll bars and radio buttons. 2. eg. if a column contains "peter john smith" the corresponding keyword table would have three indexed rows for "peter", "john" and "smith".] . ie. stored in the same table as its supertype. may 1994 while implementation rules can produce useful prototypes directly from conceptual constraints and parameters, override and ad hoc code annotations enable analyst/programmers to override rules or to add arbitrary procedural extensions when developing production quality applications. unusual modules could require numerous annotations4, but in practice most modules either require no annotations or have specific, minor idiosyncrasies. specifying these idiosyncrasies with impure annotations at the conceptual level is undesirable, but when the need arises it is much easier to maintain a few lines of ad hoc annotations in a conceptual model than to find and understand the few changed lines interspersed amongst hundreds of lines of generated 4gl code produced by a waterfall approach. moreover, if the schema changes in ways unrelated to the overrides, the complete 4gl program can be easily regenerated5. the effectiveness of this approach is dependent upon the ability of a manageable number of conceptual annotations to provide significant automation. if new special case annotations were required to effectively implement each new application module, the problem of designing a conceptual model would degenerate into that of understanding the meaning of innumerable different annotations. the author's experience with the intelligent developer (id) berglas (1993b), bha 1992, a commercial 4gl generator, shows that the following types of requirements account for the majority of 4gl code:• enabling rows to be selected, edited and reported within the basic relational structure. this is usually achieved through forms oriented interfaces that process related tables, with most of each form's fields corresponding directly to columns in the database. complications include using non-standard indexing systems such as keyword tables, and looking up descriptive values for foreign keys. • making salient derived data conveniently available, such as totals of detail rows. this is similar to the functionality provided by conventional views, but there are subtleties such as deciding whether to use lazy or eager evaluation, and if eager the use of delta rules6, or perhaps the derived data is a default that can be overridden. • different users have different access to sensitive data. this is also similar to views whose queries can relate the accessed data to user profiles, but additional details need to be addressed, such as telling a user that they have no access rather than just pretending a row does not exist • invoking structured business rules. • producing elegant user interfaces. this involves connecting windows together, providing convenient default values, context sensitive help, and designing reasonable form layouts. the types of requirements are limited because most of the processing involved in an information system can be described in terms of accessing, updating or disseminating the information that they contain using standard algorithms contained within a database manager. most constraints on the data are of a standard type, such as referential integrity and uniqueness, while simple finite state machines can be used to provide dynamic constraints and trigger actions upon state transitions. the relatively small number of business rules that cannot be expressed as constraints to conceptual schemas account for a very small fraction of the 4gl code used. the dozen specific types of annotations described in the following sections account for most individual annotations used in practice because they correspond to these requirements. further, the data centred approach makes it unnecessary to detail many transactions such as "enroll a new student" if they mainly involve accessing and editing data subject to constraints, in this case editing details about each student. by capturing common constraints and implying basic transactions, annotations reduce the need for formal specifications using languages like taxisdl (part of daida). it is acceptable to specify remaining ad hoc definitions procedurally because there are not many of them hi an information system, so the limitations of technologies that automatically refine formal specifications into production quality applications can be circumvented. this technology is inappropriate for applications that contain many operations that do not fit the model of updating a structured database subject to relatively simple rules. for example, an application where most transactions involved complicated procedural interfaces to other systems rather than just 4. techniques for managing unusual modules will be discussed below. -5. a warning should be issued if this independence cannot be shown.. 6. eg. it is not necessary to completely recalculate a total if one detail record is changed. ajis updating a local database would be unsuitable — indeed a language like c++ might be more appropriate than a 4gl for such a system. the ifip example the ifip example defines an information system for managing conferences. authors submit papers, which may have been invited to conferences. the information system should keep track of papers and authors, allocate the referees, produce exception reports of overdue papers etc. a typical intelligent developer style user interface is shown in figure 1, which could be used to submit papers. the user enters the paper nr of an existing paper, or creates a new paper record by allowing it to default to the next available number. when the id of the conference being submitted to is entered the conference name is shown to the user. the paper's title, subjects and authors may then be entered, the latter in scrolling regions. an acknowledgment is sent to the first author once the paper has been logged. paper submissions paper nr i submitted to [ tide status subject code description author nr institution e-mail figure 1: window for submitting papers to a conference. if the user does not know the conference id, they may press the "select" key to pop up the conference selection window shown in figure 2 . conferences may then be selected by entering a fragment of a conference name or a period during which one may have been held. most id forms are variations of either either maintenance forms (like figure 1) or selection ones, but although these interfaces are fairly simple and have a common structure, they take hundreds of lines to implement properly in a conventional 4gl conference selection word in name l date from f£ i to conference id name ' j1 ( . ,» -^ , * j,_ j1 _ j a0 figure 2: window for selecting conferences figure 3 shows part of the corresponding niam or object role modeling conceptual schema halpin (1991). entities are circled, and/actt, the relationship between them, are represented as rectangles. the bar next to a square indicates the role(s) that identify each fact. thus many people may work for may 1994 an institution, but each person may only work for one institution (at a time); while a paper may have several authors, each of whom may write several papers. niam is similar to entity relationship attribute (era) models, but era "attributes" and "relationships" are considered to be just a type of fact in niam. for example, conference is an era entity with attributes id, name and date, while paper-has-conference is a relationship because conference has attributes. case tools can provide convenient era abstractions of niam schemas, and may also suppress non-conceptual annotations. i tag: "conference id" i i i i tagsubmission: "submitted to" i j i size: (20,40) i subject thesaurus obsolete figure 3: the ifip conference conceptual schema. paper abandonded | submitted | params: (subjects, authors) post-action:{ let e-mail = (select... from authors where ...) call e-mailer(e-mail, "your paper has been received.")} figure 4: the paper jsd-like entity life cycle. ajis figure 4 shows the life cycle of a paper in a jsd style notation jackson (1984). papers are either invited or offered by the authors, in which case they are submitted, refereed a number of times and possibly accepted before the camera ready copy is printed; but this could be abandoned at any stage. as no predicates have been specified to define what state a paper is currently in, an implicit state fact is created on the niam schema. niam entities can have several independent life cycles, and two state life cycles with explicit predicates encompass active database "when/if/then" rules. examples of specific annotations table 1 lists common annotations, some of which were used in figure 3. many of these annotations describe the conceptual models in ways that do not strictly constrain the data and so cannot be naturally described using first order logic. however, they provide important information for the production of effective implementations, especially user interfaces. for example, "3" indicates that papers typically concern three subjects but does not prevent some papers from referring to five subjects. this can be used to determine whether indexes should be created and the number of records to display in an interface's scrolling region7. all conceptual objects (facts, entities, constraints and states) may be annotated, with facts inheriting annotations from entities. design rules can determine the purpose and basic structure of each user interface by examining an era abstraction of the niam diagram8. by default, each era entity has a corresponding form to view and update it that contains a field for each attribute, foreign key and any associated descriptive facts. if the entity was embolded (ie. paper and person) a scrolling region is created for each referencing foreign key (eg. about-subjects and authors). selection forms and simple reports are created in a similar mannerthe paper submission form in figure 1 was produced via a different mechanism, namely to provide a special purpose interface for transition to the submitted state because it is embolded in the dynamic model. the annotation indicates that subjects and authors (but not referees) are involved and how the acknowledgment should be sent. the details about each author may be edited directly on paper maintenance forms because the author role is embolded, while only the reference to subjects is shown. note that most of the interface's functionality concerns the editing of information about papers, even though it implements the transaction of logging new papers. annotations are used to describe user interface layouts rather than specifying them explicitly using a screen painter. screen painting involves using a pretty interface to issue very low level commands such as "place field x at (12, 34)" rather than "place field x near field y". this makes it difficult to generate new layouts when the underlying schema changes, and curtails the interesting option of generating layouts at run time in response to the subtype instances being manipulated or the end user resizing graphical windows. screen painting is also unsuitable for non-forms interfaces such as node/arc graphs or gantt charts. therefore details of each layout should be determined by an algorithm, with user interface annotations providing hints such as which facts are related or partial orders of fields. these gentle annotations will usually be applicable for all interfaces that reference a fact. for example, a report will probably want fields to be laid out in the same order as an interactive interface even though their actual positions may be quite different. the intelligent developer's simple layout algorithm is adequate for simple interfaces, while techniques such as wiecha (1990) and pizano (1993) could provide greater automation. annotations may be addressed to specific interfaces when the need arises by using different "facets". thus "conference id" has been specified as the boiler plate tag in figure 39 for all interfaces except paper submission, for which it has been specialized to "submitted to". as the conceptual model is used as a complete specification, it also becomes desirable to be able to restrict annotations to specific subtypes or dynamic states. for example, it might be mandatory for all papers to be refereed, except invited ones. this is orthogonal to facet specialization. 7. this annotation could have been deduced by examining the populations of live databases, but affects implementation like other annotations and can aid an analyst's understanding of a schema. 8. this brief description only provides an approximation to the complete algorithm and ignores subtypes, problems with large forms, keys generated from the data etc. '. the tag defaults to the fact name (eg. "name") in the intelligent developer, unless it is specifically "id", "ni" or "code" in which case it is prepended by table name, ie. "conference id" is actually the default may 1994 7 annotations can also improve the behaviour of other database interfaces because they usually describe the nature of the schema itself. for example, qbe10 could use a descriptive annotation to automatically join person.name by default whenever the author table is queried rather than just showing the uninfonnative person.nr, and the sql code required to calculate derived facts ("*") could be incorporated into queries as required. berglas (1994) argues that the definitions of active business rules should be structured in terms of the conceptual schema using kl-one brackmann 1985 style classification and method combination. thus there are annotations conceptual constraints a mandatory role, ie. every entity instance must have a corresponding fact instance. exclusion constraint eg. one cannot referee ones own paper. fixed value, fact instances cannot be changed once created (not standard niam). ~ n o x size conceptual parameters primary key (always unique). descriptive, eg. the conference name is shown in figure 1 rather than its date (say), although the descriptive id (eg. "cikm93") would suffice (cf. person-nr "92364"). selective, so is useful for a selection criteria (eg. in figure 2) and suggests an index could be created. selective fact usually used in combination with others. this affects user interfaces, and may create multi-column indexes. typical populations, eg. there are usually 3 subjects per paper. a role that is usually null. affects user interfaces and database sizing. exclude this fact from a (specific) interface. typical and maximum sizes of each field/column. not all instances are stored ie. they may be purged. user interface options embolding affects user interface structure, described below. tag fixed boiler plate text on forms and reports. &x in the fact group named x, should be displayed close to other members of the group. >afact defines a partial ordering of fields in user interfaces. wsize actual, fixed size to use for a widget. offset (vertical, horizontal). explicit position in a window relative to previous field. widget name of the widget to use, possibly with specific parameters. popup name of a pop up program, eg. conference selection. non-standard business rules * lazy derived fact (the actual 4gl code is shown on request). ~* eager derived fact (stored in the database). + default values provided .for a fact instance if it was null. + default value calculated once upon a state transition and stored. 0 conditions that determine when an instance belongs to a subtype. ® database action to performed when an entity is (re)classified. ® add hoc 4gl code to be hacked into forms interfaces that contain the corresponding field or table. 6 like © but only affects the user interface. schema management a user configurable conceptual object. v conceptual object owned by an end user. t, t (very) "important" object for schema abstraction. conceptual library invocation e total of pointed fact, a special case of *. likewise e, average, count, max, etc. # automatic key generation, eg. give papers the next available number. thesaurus indicates that a multi-parent thesaurus should be created. obsolete obsolete records may be flagged rather then simply deleted. sum-purge use summarize and purge techniques to remove old data. table 1: commonly used annotations . query by example, ibm's end user graphical query interface. ajis that effectively provide precedents and consequents that can describe any business rule, and these can be executed in the database itself. as most fields and blocks in 4gl programs are directly related to columns and tables in the database, ad hoc user interface annotations can also be placed on the conceptual schema. modem 4gl structuring devices such as exception handling enable different code fragments to be sensibly combined. programs are harder to elegantly define than relational data structures so 4gls contain more constructs than the dozen or so used by standard niam models. this has been reflected in the relatively greater number of annotations proposed in this paper. however, by concerning themselves mainly with the five general requirements listed in the introduction, and by using overrides and 4gl procedural annotations to cover unusual cases, a manageable number can significantly automate the production of realistic information system. intermediate schemas each form in the finished application can be described by a subschema that is extracted from the main schema as shown in figure 5 experience with the intelligent developer showed that information systems could be designed in terms of three main interface paradigms, namely the selection, maintenance and reporting of central entities (embolden on the figures, eg. paper) together with selected facts that are related to that entity. each fact in the subschema corresponds to a field in the form, with the paradigm defining how they are used in the interfaces. main schema referees 1 tag: "conference id" [ i tag.submission:"submitudto" ' i 1 / ct^/* •' / / ^ \ ™es\ *r' ' ^<v\ «~ '• — • vnr \ /'date i # j '• ' '. / "̂' vr 'nr i ^/^~ l "7,1"-) : size: 20. .40 '. \ wsae:25 ' offset: (50, 80) ', .' ( name ' conference , authors "••; farf/y"! ; pup-up: customer selection paper submission subschema figure 5: the paper submission interface subschema. unlike pizano (1993), the subschemas are usually temporary data structures that are automatically extracted from the main schema during application generation, and destroyed once 4gl code has been produced. however it was found to be convenient to allow programmers to store and edit the ajis subschemas for the few interfaces that would otherwise have required excessive annotations on the main schema. the problem of maintaining stored subschemas as the main schema evolves is reduced by representing schemas by semantic nets, with each node in the subschema inheriting slot values from the corresponding node in the main schema, as shown in the figure. thus stored subschemas only contains information that overrides information in the main schema so non-structural changes to the main schema will usually be automatically reflected in any stored subschemas. conceptual macros enable commonly used high level constructs to be defined in terms of annotated conceptual schemas. for example, the obsolete flag on subjects in figure 3 is used to flag subject categories that are no longer considered useful after librarians change the classification scheme. obsolete subjects cannot be simply deleted from the database because existing papers that refer to them would need to be reclassified, which may not be considered worthwhile. obsolescence is fairly common in practice, for example institutions, status codes, and products may also be considered obsolete. figure 6 shows a conceptual macro that can automate the implementation of entities that can be flagged obsolete. it indicates that if an "&entity&" such as subject is marked obsolete, then an obsolete fact should be attached to it and the associated code executed whenever a reference to entity is created. different code may be generated for different entities, in this case "&long-name(entity)&" will expand to the long-name meta-value of the primary entity. generalized functions that could be implemented using conceptual macros include multi-parent thesauruses, mailing labels, summarizing and purging old data, and maintaining statistical histories. obsolete on-being-referenced(.-fetch obsolete): { if obsolete = "y" then message ("? this %1 record is obsolete" "&long-name(entity)& "); raise-exception obsolete ; endifj figure 6: the obsolete module. conceptual macros are similar to super-classes in an object oriented system — saying that an entity can be obsolete is analogous to saying that the entity inherits the obsolete superclass. however, conceptual macros can access and manipulate the conceptual model quite generally — the intelligent developer uses the full power of common lisp as its macro language, which was found to be useful when developing realistic modules. although implementing conceptual macros requires care, it is much easier than developing conventional macros because they augment conceptual models rather than generating 3gl code. for example, having attached the obsolete fact to the subject entity, the obsolete module does not have to decide how to rearrange user interfaces to include the corresponding obsolete field. this has enabled application programmers using the intelligent developer to effectively extend the tool to suit their specific environments. normally only the invocation of the macro is stored in the main schema, and it is expanded each time it is needed. this makes it easy to change parameters to the macro, or even the macro's definition. however, programmers may elect to store and then edit a macro's expansion in the same way that they can edit stored subschemas. using the annotated models realistic information systems may involve hundreds of entities, so a major difficulty in developing them is to understand their overall structure. it seems unlikely that any single diagram like figure 3 could display all the details of a specification and at the same time be suitable for gaming an overview may 1994 10 of a large system. in order to resolve this conflict, case tools can display different abstractions of a specification, with changes made to one view automatically reflected in the others. an initial design can be sketched out using an era view of the schema, and be partitioned into overlapping "scopes". the analyst can then expand a scope into an annotated niam diagram similar to figure 3 , but without the detailed annotations and expanded conceptual models shown in dashed boxes in that diagram. those details can be displayed on demand when the analyst selects specific facts, entities or conceptual module invocations. new facts added to the niam view would have any corresponding relations automatically included on the era abstraction11. the analyst may also annotate certain era entities and relationships to be important, in which case an overview diagram containing just those components can be displayed12. this style of interaction in which the analyst/programmer "zooms" in and out through different levels of abstraction is in sharp contrast to the one way journey through a waterfall model offered by conventional case tools. end users may also wish to edit conceptual schemas. for example, different organizing committees may wish to store different fact types about papers that are submitted to a conference — one conference may be interested to know if a paper could qualify for a student paper award, while another might be interested in each paper's security clearance. in order to avoid having to modify a conference management system package for each individual customer, a trend in the design of information systems is to include generalized fact types such as paper-has-value-for-attribute so that the end user can configure the names of any additional paper attributes in special code tables simsion (1994). this fact type would produce a table that contains tuples like (<paper-123, security, x23>, <paper-456, student, true>). however, it can be inefficient to implement the "verticalized" tables that result from generalized data structures, the resultant user interfaces are often inelegant, application programs become more complex and the resultant application can still only be configured in ways that were conceived when the application was developed. as an application is made more generalized to enable users more configuration choices, the implementation problems increase. these issues are developed in detail in berglas (1993a). by enabling applications to be largely defined at the conceptual level, it becomes possible to enable end users to edit the conceptual models themselves and so avoid these generalized structures. data entry operators may not have the skills to do this, but most organizations contain "spreadsheet literate" users that can build the simple spreadsheet applications upon which many important management decisions are based. these users could not design complex information systems or edit 4gl programs, but they could certainly learn how to add a simple fact to an existing schema and then have it automatically included in all the relevant interfaces. in order to prevent spreadsheet literate users from corrupting the essential integrity of an application, special annotations may be used to specify how different end users may alter the conceptual models themselves. adding new fact types is usually safe, as is changing user interface annotations on existing ones. end users cannot normally change the semantics of existing facts or entities unless the programmer specifically annotates them "a", although they may change any facts they created themselves and are therefor annotated "v". thus annotated models enable simple applications to be highly configurable without recourse to complex generalized designs. conclusion by recognizing the fact that most information system functionality involves accessing, editing and disseminating data using common techniques, this paper shows that annotations can be used to augment conceptual models so that they provide a complete description of production quality applications without the need for advanced theorem proving technology. it also argues that a manageable number of annotations are sufficient and that a minimal amount of procedural code needs to be specified in practice. annotations provide additional parameters that make it feasible to develop sophisticated rules that can significantly automate implementation. many of these rules are concerned with producing elegant user interfaces, the construction of which has been studied outside the information system context 1'. ideally, the case tool should contain a schema layout algorithm. . le. important annotations describes how the schema itself is used. 11 ajis annotated conceptual models are at the same time concise because they are used in conjunction with these rules and complete because impure annotations can override and augment any rule. subschemas that describe individual interfaces can be explicitly extracted from the main schema, and may be stored and edited to avoid excessive annotations where necessary. it is also possible to produce libraries of conceptual macros that enable reuse at the conceptual level. however, as far more resources are spent maintaining applications rather than in their initial development, a major benefit of using annotations is to enable analyst/programmers to "zoom" between different levels of abstraction rather than taking the one way journey through a waterfall model. another important benefit is to enable end users to edit conceptual models in a well defined manner and so produce more flexible applications without recourse to complex generalized data structures, acknowledgment assistance in preparing this paper was gratefully received from bob colomb of the university of queensland. the intelligent developer was written for and is marketed by bha computer of 1/303 coronation drive, milton, brisbane, australia. bibliography berglas, a. (1993a) "enabling end users to tailor conceptual models in a disciplined manner", n. sarda (ed.) conference on information systems and management of data. insdoc. berglas, a. (1993b) 'the intelligent developer— towards the automated implementation of information systems" in m. orlowska and m. papazoglou (eds.) advances in database research, world scientific. berglas a. (1994) "using kl-one knowledge representation techniques to structure active database rules" in r. sacks-davis (ed.) advances in database research. world scientific. bha computer ltd. (1992), brisbane. the intelligent developer reference manual borras p., mamou j., plateau d., poyet b., and tallot d. (1992) "building user interfaces for database applications: the 02 experience" sigmod record, vol 21 no 1. brachman r. and schmolze j. (1985) "an overview of the kl-one knowledge representation system" cognitive science, vol 9 pp 171-216. flynn b. and maier d. (1992) "supporting display generation for complex database objects" sigmod record, vol 21 no 1. halpin t. and orlowska m. (1991) "fact-oriented modeling for data analysis" journal of information systems, vol 2 no 2 ifep (review of information systems design 1982) "the dtp example" in w. olle and h. sol (eds.) comparative methodologies, pp 8-9. north-holland. jackson m. (1984) system development. prentice-hall. jarke m., mylopoulos j., schmidt j., and vassiliou y. (1992) "daida: an environment for evolving information systems." acm trans, on information systems, vol 10 no 1. king r. and novak m. (1993) designing database interfaces with dbface. ieee trans. software engineering, vol 11 no 2. oracle corporation. (1993) the case*generator reference manual. ovum ltd (1992), london. ovum evaluates case products pizano a., lizawa a., and shiroto y. (1992) "an automatic screen layout generator for database applications" in y. yesha (ed.) proceedings of the conference on information and knowledge management, cikm-92, pp 239-247. pizano a. (1993) "automatic generation of graphical user interfaces for interactive database applications" in b. bhargava, t. finin and y. yesha (eds.) proceedings of the conference on information and knowledge management, cikm-93. reubenstein h. and waters r. (1991) "the requirements apprentice: automated assistance for requirements acquisition"ieee trans. software engineering, vol 13 no 3. simsion g. (1994) "implementation of very generalized data structures" in r. sacks-davis (ed.) advances in database research. world scientific. wiecha c. (1990) its: a tool for rapidly developing interactive applications. acm trans, on information systems, vol 8 no 3. may 1994 12 the australasian journal of information systems (ajis) began in 1993 and was based on the simple realisation that there was a need for an academic ‘outlet’ for is research in australia australasian journal of information systems volume 17 number 2 2012 57 from action research to practice research göran goldkuhl linköping university goran.goldkuhl@liu.se abstract action research (ar) has gained more acceptance as an approach to qualitative research in information systems (is). the complexities of organisational and technical change makes this approach a suitable one in is research. there are, however, still some controversies and confusions about the relation between “action” and “research”. the many types of ar and similar approaches (not labelled as ar) that have emerged demand further conceptual clarification of ar. a conceptual inquiry of ar, presented in the paper, has led to the identification of several unresolved issues concerning intervention research like ar. an alternative research approach is presented: practice research. this research approach is well founded in pragmatism and it is founded on the two premises: 1) to contribute to general practice through abstract and useful knowledge and 2) to study the empirical field as interconnected practices. several important concepts of practice research are described as: local practice contribution vs. general practice contribution; theorizing vs. situational inquiry; abstract vs. situational knowledge. these notions and their pragmatist foundations can help to sharpen ar as a qualitative research approach. practice research is defined as a broader notion encompassing ar and other research approaches as e.g. design research and evaluation research. two case examples of practice research are briefly presented and compared: one ar-based study in the social welfare sector and one evaluation study of a taxation e-service. introduction action research (ar) has evolved as an important approach of qualitative research within information systems (is). there seems to still be many sceptics, but the acceptance and prestige of this research approach is growing. the two special issues in information technology & people (kock & lau, 2001) and mis quarterly (baskerville & myers, 2004), together with the compilation of articles in kock (2006), are signs of this growing recognition. action research has indeed strengthened qualitative research in is with its specific qualities of closeness to the empirical field, exploration of change and ensuring practical relevance. ar seems to be especially appropriate in the is area since we are so concerned with issues of organisational, informational and technical change. since ar deviates from classical scientific endeavours in several ways, there have been several efforts to clarify the scientific character of ar. there are many examples of this, such as classical ones like e.g. rapoport (1970), susman & evered (1978) and hult & lennung (1980) and specific is efforts like e.g. checkland (1991), baskerville (1999), mckay & marshall (2001), davison et al (2004). one core issue has been the relation between action and research. several scholars have tried to clarify how research and problem solving interests intersect and interact (marshall & mckay, 2001; davison et al, 2004). what we call action research is, however, not the only research approach that deals with intervention in practice. there are several other research approaches that include intervention and collaboration mailto:goran.goldkuhl@liu.se australasian journal of information systems volume 17 number 2 2012 58 with practitioners; e.g. action science (argyris et al, 1985), action learning (revans, 1982), interactive research (lundin & wirdenius, 1990), participative research (elden, 1983), participatory research (parks, 2001), participative case study (baskerville, 1997), collaborative practice research (mathiassen, 2002), collaborative management research (pasmore et al, 2008), co-operative inquiry (heron & reason, 2001), clinical inquiry (schein, 2001), development action inquiry (torbert, 1999), appreciative inquiry (ludema et al, 2001), pragmatic-systemic inquiry (cronen, 2001), practical inquiry (stevenson, 2005; goldkuhl, 2008) and pragmatic inquiry (metcalfe, 2008). this means that there are complementary views that have not found their place within approaches called action research, although several of the approaches mentioned above are sometimes seen as variants of action research. there are needs for further investigation of action research and other intervention approaches. how should we conceptualise such approaches? what criteria exist for making them scientific? sometimes ar is confused with plain consulting and it is therefore important to state criteria for what makes ar scientific and distinguishable from consulting although some similarities exist (gummesson, 1988; baskerville, 1997). there is also a growing debate within is about the relations, similarities and differences between ar and design research (e.g. cole et al, 2005; järvinen, 2005; iivari & venable, 2009; sein et al, 2011). in order to clarify how ar and design research relate to each other it is necessary to have a clear conceptual image of ar. it is also the case that ar has over the years been diversified in a large number of variants (baskerville & wood-harper, 1998; casell & johnsson, 2006; chandler & torbert, 2003; lau, 1997). reason (1999, p 222) is drastic concerning implications of this diversification: “my experience of the term ‘action research’ is that it means so many things to so many people that it is methodologically useless to distinguish one strategy from another”. i would definitely not go so far as to state that the term ‘action research’ has lost its meaning and become useless. it is in good use, but there seems to be a need for further conceptual clarification concerning its characteristics and limits. the paradigmatic foundations of ar are not always clear. in many accounts of ar there are no explicit references to any paradigm or scientific school of thought. one typical example is braa & vidgen (1999), who have stated an is research framework through comparison of three different research approaches (prediction/reduction, interpretation/understanding, intervention/change). two of them have been positioned paradigmatically in clear ways (prediction/reduction as positivism; interpretation/understanding as interpretivism), but the third one (intervention/change) has no identified paradigmatic home. however, there exist clear accounts of the epistemological basis for ar. baskerville & myers (2004) have clearly positioned ar within pragmatism; cf. also oquist (1978). the full consequences of this paradigmatic positioning are, however, still to be found. the overall purpose of this paper is to investigate intervention-oriented strategies that are relevant in qualitative research on information systems and organisations. it is natural and appropriate to take the point of departure in action research. this inquiry on ar will focus  “action”, i.e. the problem solving and how it is constituted as a part of the research process  “research”, i.e. what makes ar scientific (the research legitimacy of ar). through this inquiry a number of ar issues emerge that comprise controversies, obscurities or are unresolved in other ways. this inquiry and its identification of important issues have informed the articulation of a partially alternative qualitative research approach. this alternative approach will be labelled practice research (pr). the articulation of practice research implies also a clarification of ar as a viable research orientation. the formulation of practice research should not at all be interpreted as a rejection of ar as an appropriate research approach for qualitative research in is. ar has important qualities which should be acknowledged and defended. however, as the discourses on ar indicate, there are needs for clarification and this paper contributes with some clarification of such a kind. australasian journal of information systems volume 17 number 2 2012 59 the conceptual development of pr is illustrated by two case studies in e-government. one study is an evaluation of a taxation e-service. the other one is an action research study comprising development of an it artefact for information-transfer between state agencies and municipal welfare offices. the two case studies are compared in relation to action research and practice research. this also means that this paper uses a combined approach of conceptual inquiry and empirical case studies in order to formulate its contributions. to summarize the purposes of this paper: it contributes with  knowledge about important unresolved issues concerning action research (and other intervention strategies) that need clarification  an articulation of an alternative research approach, “practice research”, based on the ar inquiry that includes the formulation of several concepts and dualities and a pragmatist foundation  a comparative clarification of action research and practice research and relations between these approaches  a partial reformulation of action research based on practice research notions action and research in action research: a conceptual inquiry the main knowledge interest in this inquiry has been on “action” and “research” in ar; i.e. issues of intervention, collaboration and what makes ar scientific and other related questions. in order to identify and investigate such issues a literature review has been conducted. important publications on ar in is and generally on ar have been studied. different themes have been identified based on the main knowledge interest during the review of the literature. it has been important to disclose alternative views within these themes. this search for alternative views has also guided the selection of literature. this investigation has been labelled a conceptual inquiry, which means that it has been guided by an interest for improvement and clarification. goals and target groups action research deviates from classical research in several ways. this makes it necessary to make a proper conceptualisation of what is done in ar (mckay & marshall, 2001; davison et al, 2004). one key feature of ar is serving “two masters” (kock & lau, 2001). ar is aiming to contribute both to a research community and to practitioners in a specific problematic situation according to classical definitions by rapoport (1970) and hult & lennung (1980). this is expressed in the following way by rapoport (1970): “action research aims to contribute both to the practical concerns of people in an immediate problematic situation and to the goals of social science by joint collaboration within a mutually acceptable ethical framework”. this definition includes the conjoint goals of action research; that of practical problem solving and contribution to scientific knowledge. it is notable that rapoport talks about “an immediate problematic situation”, which means that general practical relevance is not explicitly considered. a more elaborated definition, but still in the same spirit, is formulated by hult & lennung (1980). they have reviewed an extensive amount of literature in ar and refined the definition from rapoport. in their definition, the practical goals and procedures in action research are elaborated, but not the ones concerning scientific knowledge. hult & lennung emphasise collaboration, mutual learning and local understanding, which are goals often referred to in ar. the goals of scientific results, i.e. the contribution to the scientific body of knowledge are not problematized. the character of the scientific contribution is simply taken for granted. the main target groups of ar seem to be the people in the local operational practice and the scientific community. other practitioners outside the local operational practice do not seem to be a specified target group. there has actually been critique against ar from the perspective of improved practical relevance of is research. if the results from an ar project are too specific concerning its problems of that specific company, these results are not considered to be practically relevant in a general a sense. australasian journal of information systems volume 17 number 2 2012 60 there seems, however, to be a misconception, that generalizable results need to be stated in terms of practical problems solved. nielsen (2007) has argued for a transferability criteria in ar (instead of generalizability). it is claimed that “we need to explicate the general characteristics of the findings and the conditions for transferring them to other situations” (ibid 365). dual processes working in cycles this dual purpose has led mckay & marshall (2001; 2002; 2006) to describe ar in terms of two interrelated process cycles. there is one problem solving cycle that describes the problem solving process and there is one research cycle that describes the management of the research interests. these cycles are described in a similar generic way (figure 1) consisting of some initial steps and then a cycle of steps that can be pursued until a satisfactory result is reached. identification of problems, questions fact finding planning action steps implementation monitoring evaluation amend plan exit figure 1 a generic process cycle for problem solving and research in ar (based on mckay & marshall, 2001) first, it is important to note that this cycle deviates from the classical ar cycle in susman & evered (1978); cf. figure 2. problem analysis and diagnosis are in the process of mckay & marshall left out from the cycle and defined as an introductory step that is not iterated. it is hard to see any reasons for eliminating the diagnosis/problem analysis from the iterative cycle. later work with implementation can give rise to needs in order to better understand the original problem-situation. action planning considering alternative courses of action for solving a problem development of clientsystem infrastructure specifying learning identifying general findings evaluating studying consequences of an action action taking selecting a course of action diagnosis identifying or defining a problem figure 2 a classical cycle description of action research (from susman & evered, 1978) there are several other problematic issues in this dual cycle conceptualisation of ar. the two process cycles are described as working in tandem. they are superimposed upon each other. what this exactly australasian journal of information systems volume 17 number 2 2012 61 means is not clarified. in a later paper (mckay & marshall, 2002) the two processes are described in an integrated way, but this still leaves questions with regard to how the two cycles work in tandem. for example, are there actions that are multi-functional and thus belong to both cycles? are there other actions that only belong to one cycle? should it not be necessary to clearly describe what actions belong to a single cycle and what actions belong to both cycles? the problem solving cycle is described from the researcher’s perspective. “the first cycle relates to the researcher’s problem solving interests and responsibilities…” (mckay & marshall, 2001, p 50). why is this not done from the problem owner’s perspective? would that not be more adequate? the research cycle does not seem to be properly described. where is data collection, data analysis and theory development performed in the research process cycle? is it so that the (implicit) aim to describe the two processes in a similar way (based on a generic process structure) has hampered the formulation of adapted process descriptions? a generic structure has, in an obtrusive way, been imposed on to the two processes of problem solving and research. we should rather try to understand each process and then try to integrate them in a proper way. an alternative conceptualisation will be presented as a way to resolve this obscurity. the research process is divided into two fundamental activities of theoretical and empirical work (figure 3). this division is valid for action research and for all other types of research that is empirical in nature. there is a difference between action research and more classical empirical research that cycling between theorizing and empirical investigation is much more intense in ar. in a classical research approach there will be an initial theoretical part with hypothesis formulation and preparation of investigation and then a gathering of data followed by concluding analysis of data. there will be just one cycle; first theory, then empirics and then back to theory. in an action research context there will be many movements back and forth between theory and empirics. this is also the case for most qualitative research approaches; see e.g. descriptions of grounded theory (e.g. strauss & corbin, 1998) where a continual interplay between data collection and data analysis is prescribed. what are the corresponding fundamental constituents of practical problem solving? the generic structure (from mckay & marshall, 2001) seems to be more adapted to problem solving than research so it would be more appropriate to use this structure here than for research. however, is it possible to boil down practical problem solving and change into two fundamental activities? practical problem solving originates from some problems in operational work. these problems give rise to a developmental process followed by implementation of changes into the operational practice. we can describe practical problem solving as a movement from operational work to developmental work and then back to operational work (figure 4). theoretical work empirical work operational work development work figure 3 research process cycle figure 4 problem solving cycle described in this way, there is a one-cycle movement like in classical research; first operation, then development, then implementation which means back to operation. there might be other types of australasian journal of information systems volume 17 number 2 2012 62 change processes (like experimental, incremental and agile approaches) that work with many small cycles going back and forth between operational and developmental work. if we take these two cycles and use them as a lens when looking at ar, what can be seen? can we put the two cycles on top of each other? ar works with practical problem solving. the researchers’ work together with practitioners to develop some operational practice is empirical work when looking at it from a research perspective. from the practitioners’ perspective this is researcher involvement in their development work. this implies that developmental and empirical work may coincide to a large degree. what is done in a collaboration process between researchers and practitioners is at the same time both 1) development of a local operational practice and 2) empirical research. this follows from the dual purposes of ar. this means that action research can be described in terms of the interrelatedness of three practices (cronholm & goldkuhl, 2004): 1) theoretical work, 2) empirical and developmental work and 3) operational work (figure 5). the operational work is the object of problem solving, development and change. theoretical work empirical & development work operational work figure 5 integration of research and practical problem solving it should be noted that development and empirical work do not fully coincide (goldkuhl, 2011). there may be empirical data collection made by researchers (in ar projects) that is mainly done in the service of theorizing (i.e. empirical work separated from development). there will also be development work performed by the practitioners that lies outside the interest of the researchers (i.e. development work outside the object of empirical research). research interest vs. problem solving interest albeit conceptual problems in the dual cycle model, mckay & marshall (2001, 2002) have contributed with an in-depth analysis of problem solving vs. research interest in ar. peters & robinson (1984) have in a survey study of ar identified that there does not seem to be consensus what interest should have the privilege over the other. this has been described as a classical dilemma in action research (rapoport, 1970). there may be a risk to be too compliant to practitioners’ problem definitions. these can be unreflective and block an unbiased inquiry. on the contrary schein (2001) makes strong claims in his clinical approach that the inquiry process should be guided by the practical problems. “it is my argument that some of the best opportunities for such an inquiry actually arise in situations where the setting is created by someone who wants help, not by the researcher deciding what to study” (ibid p 228). the issue of practical relevance to have a practical problem solving interest governing the research process, as in ar, is a way to respond to the demands for more practical relevance in is research (baskerville & myers, 2004; davison et al, 2004). there has been a long debate on practical relevance in is research and its relations to research rigour (e.g. keen, 1991; benbasat & zmud, 1999; davenport & markus, 1999; lee, 1999; lyytinen, 1999). relevance has been divided into several dimensions. benbasat & zmud australasian journal of information systems volume 17 number 2 2012 63 (1999) have made a division into interesting and current (timely) topic, applicable results and accessible and readable publications. this division has been used by other scholars with some slight conceptual/terminological modification. klein et al (2006) speak of relevance in terms of importance, applicability and accessibility and rosemann & vessey (2008) use the terms importance, suitability and accessibility. relevance can be seen as an issue of input to research (selection of an important topic) and output from research (production of useful results). ar has important methodological features for warranting both input relevance and output usefulness. the selection of a practical problem to be solved (i.e. a problem solving interest) is a way to ensure a selected topic of relevance. however, objections may be raised against this. something that is of concern in one organisation might not be of concern to many other organisations. to be responsive to one single organisation does not ensure a broad practical importance of the topic selected. there are other issues concerning the relevance of input than of topic selection. after selecting a relevant topic you need data that is relevant to this topic. an ar endeavour works usually with data generation of in-depth character and over a long time-span. this is a way to ensure a nuanced body of data as a basis for theoretical generation. for example, lyytinen (1999) pinpoints the need for close cooperation with practitioners on a long-term basis to ensure relevance. one demand in action research is the search for a workable change in the client-organisation. to have a solution that works in at least one organisation is a way to ensure some applicability of results. however, criticism can also be raised that this is a narrow application. here, the idea of viewing research in terms of inquiry in the professions (davenport & markus, 1999; lee, 1999) seems adequate as a response. this implies formulation of abstract knowledge of professional value and use. empirical vs. theoretical activities the structure of ar into five phases as described by susman & evered (1978) – see figure 2 – has according to davison et al (2004) reached a canonical status. this means that action researchers should adhere to this structure (ibid.), otherwise clear reasons should be stated. the main parts of this structure follow principles of developmental work (diagnosis, planning, implementation, evaluation). this means that these activities are clearly empirical in nature. the fifth activity, originally labelled “specifying learning”, is not very clear in the original text of susman & evered (1978). it is only further specified as “identifying general findings” (ibid p 588). davison et al (2004) have contributed with more substance to this phase. from their description, it is obvious that this activity (here labelled “reflection and learning”) contains theorizing parts. they specify requirements for this phase as “informing/re-informing theory” (ibid p 77). the place as the last phase in the ar phase model (figure 2) thus seems to be inadequate. a key feature of many ar projects is the continual interplay between empirical and theoretical work. researchers will now and then reflect and analyse the different findings. theorizing (reflection and learning) will not only be the last phase in ar. it will run in continual parallel with diagnosis, planning, implementation and evaluation. the description of these empirical and theoretical activities should rather be performed following the principal structure of figure 5. the canonical ar model by susman & evered (1978) can be seen to be refinement of earlier ar models. in his original description of ar kurt lewin (1947) made a division into three phases: 1) investigation of the initial situation, 2) intervention, 3) investigation of the resulting situation. in another classical article, blum (1955) made a division into two stages: 1) a diagnostic stage and, 2) a therapeutic stage. the logic of all these accounts is the solving of a practical problem. there is no or little emphasis of the role of theorizing activities. in order to find an interplay between a practice problem focus and a theoretical reflection we need to move further back to another model of integrated practical problem solving and knowledge production: the inquiry model by dewey (1938). australasian journal of information systems volume 17 number 2 2012 64 in this model (cf. also cronen, 2001) there is an explicit inclusion of theorizing activities. this will be further discussed below when articulating practice research. the role of theory how is the researcher informed by theory and other kinds of pre-knowledge in the ar process? there does not seem to be a unanimous view of this matter (lau, 1997). davison et al (2004) state the “principle of theory” including arguments for an active use of theory in the ar process. they explicitly state that “action researchers need to rely on one or more theories to guide and focus their activities” (ibid p 74). this is also a key argument in checkland (1991 p 397) who claims that “there must be an intellectual framework, declared in advance, in terms of which learning will be defined. without such a framework action research can quickly be indistinguishable from mere action”. there are however other views. elden & crisholm (1993 p 127) claim that “the usual theory to practice sequence for formulating research is reversed” in ar with direct reference to dewey’s (1938) thesis of a problematic situation as a trigger to inquiry. baskerville & pries-heje (1999) has described an ar approach using grounded theory (strauss & corbin, 1998); cf. also cassell & johnsson (2006) for inductive ar practices. an inductive generation of theory implies a minimum use of pre-categories from extant theories. this is to be contrasted to the statement that “a clearly articulated theoretical framework must be imposed on the phenomenon of interest” (davison et al, 2004 p 73). mumford (2001) makes claims of both using extant theory and producing theory in new areas. it is also important to recognise that there are different types of knowledge that are used and generated during ar. besides abstract theories there will be a use of local and situational knowledge and development of enhancing such knowledge through the ar process. some scholars talk about development of “local theories” (e.g. elden, 1983; israel et al, 1992) when creating a new diagnostic understanding of the local practice. how to conceive the empirical field ar means research into a social setting. it is not research in a reductionist fashion on pre-structured and fixed variables. ar means studying a contextual whole in order to discover and reveal unknown phenomena, properties and relations. although not always labelled so, ar endeavours are intervention oriented case studies; cf. vidgen & braa (1997) on “action case approaches”. holism is a basic stance in many ar studies. there are scholars making claims to study phenomena in relation to its surrounding environment (e.g. hult & lennung, 1980; davison et al, 2004). there is however not much discussion on what this means ontologically for inquiries. what is actually meant by context, environment or surrounding? kemmis & mctaggart (2000) discuss different extensions of the studied practice; from a more individual level (focus on human actions) to a social level (focus on social interaction). there are some remaining questions: how should we conceive the empirical field? what should govern the inclusion of studied phenomena? interaction and collaboration how is the interaction between researchers and practitioners conducted? the collaborative nature of ar is nearly always seen as a key feature. the collaboration can however be pursued in different ways. there are scholars who prefer the term participatory action research (e.g. kemmis & mctaggart, 2000) in order to emphasise the collaborative nature of the ar process; how practitioners participate in research. there are some scholars (e.g. heron & reason, 2001) who claim that all decisions, even about clear research matters like method choice should be made by researchers and practitioners together. the practitioners should rather be seen as co-researchers. this can be contrasted with how mckay & marshall (2001) describe the separated researcher-based research interest. davison et al (2004 p 73) describe diagnosis as the first part of ar in the following way: “the researcher has a responsibility to conduct an independent diagnosis not only so as to confirm the australasian journal of information systems volume 17 number 2 2012 65 nature of the problem(s), but also to determine its/their causes”. the collaboration seems to be weak in such an independent diagnosis. huxham & vangen (2003 p 385) take a clear instrumental view on collaboration with practitioners. they state that “the concerns to do with empowerment, participation, or learning … are therefore not regarded as important unless they happen to be connected with the research or action agendas”. in a review of different reports of ar endeavours, avison et al (2001) conclude that there is a great variation in how collaboration between researchers and practitioners is accomplished. scientific issues for clarifying action research and other intervention approaches this conceptual inquiry has led to the formulation of a number of issues that needs to be dealt with in order to clarify ar. the following issues have been identified:  the issue of contribution; to whom (research community vs. local practice vs. practice community), what kind of contributions (practical vs. scientific) and how contributions are generated  the issue of interests and topics; research interest vs. local problem solving interest vs. broader practical interest/importance  the issue of research activities; what kinds of activities and relations between them, what kinds of underlying conceptualisations of these activities  the issue of use of pre-knowledge in research; inductive knowledge generation vs. active use of extant theories  the issue of different types of knowledge and their functions; e.g. local/empirical knowledge vs. theoretical knowledge  the issue of how to conceive the field of study; conceptions of wholes (holism) vs. conceptions of fragments (atomism)  the issue of interaction with those researched about; the need for collaboration between researchers and practitioners  the issue of different research approaches; how ar is related to other research approaches these issues will guide the further development and clarification of research approaches below. this list should not be seen as exhaustive concerning unresolved ar issues. a deeper literature review might reveal more issues. the claim is that the list covers important issues from the ar discourse within is. i will explicitly return to these issues when comparing action research and practice research below; cf. especially table 1. articulating practice research the investigation of ar above has been a generative force to articulate the paradigm of practice research (pr). the identified issues will be used to clarify the core elements of pr and also to compare the two research orientations. practice research as a pragmatic research approach practice research is based on some essential assumptions. the issue of contribution is fundamental. for whom is something being done? what is done for someone? the basic attitude from pragmatism to improve existence through knowledge (dewey, 1938) is the key motivator. the creation of abstract and useful knowledge for practice communities is seen as the main purpose. this means that practice communities (or what will be called general practice below) are seen as a main target group. there is a second basic assumption complementing this issue of contribution. it concerns the character of empirical field. the notion of practice is here essential. in is and other related disciplines we should study practices. this means that we should acknowledge the practice character of the empirical field. australasian journal of information systems volume 17 number 2 2012 66 what is done in the design of it artefacts and when using such artefacts will always be parts of human and social practices. this follows what is sometimes called the practice turn in science (e.g. schatzki 1996; schatzki et al, 2001; gherardi, 2000, 2009; scollon, 2001; reckwitz, 2002; whittington, 2006; miettinen et al, 2009; goldkuhl, 2011). a practice is seen as a meaningful and coherent assemblage of human actors, actions, utterances and documents, and material artefacts. practices are often arranged with purposes to serve other actors outside the practice itself. a practice is shaped by humans as an organised, artificial and continually evolving arrangement, enabled and restricted by human knowledge and financial, semiotic and material conditions. practice is thus a holistic notion. it avoids however the danger of reification in holistic approaches (e.g. cuff & payne, 1979; goldkuhl & röstlinger, 2002) when acknowledging a practice’s basic constituents of human actors, their actions and action objects of semiotic and material character. this makes it possible, when studying practices to move back and forth between the holistic level of the practice and the micro elements of such a practice. this is thus an application of the well-known principle of the hermeneutic circle; a continual shifting between viewing the whole and viewing its parts (bleicher, 1980). the practice notion has also been acknowledged in is research; for example in contributions by orlikowski (2000), levina & vaast (2006), goldkuhl (2008; 2011) and gregor (2008). a key contribution from pr is, as stated above, abstract and useful knowledge for general practice; i.e. a general practice contribution (gpc). there are other target groups of practice research: research community and local practice. pr contributes with abstract knowledge as additions to the scientific body of knowledge (sbk). a main function of the research communities in relation to proposed new additions to sbk is quality assurance through review and dialogue. the research community has a responsibility for the accumulation and dissemination of knowledge generated through research. practice research means research on and for practices and will often be research conducted in (or close to) practices. in order to understand and reveal the internal meaning and logic of a practice it will be necessary to be close to the practice and to get access to what is not immediately visible or reportable (gummesson, 1991). in pr there is an interest for what is actually going in studied practices, not only what people think is happening or should happen (argyris et al, 1985). in most cases, practice research will contribute to studied local practices. there may be different local practice contributions (lpc) as diagnosis results, design/change proposals and participation in implementation of real practice changes which may include the furnishing of changed or new artefacts (goldkuhl, 2008; 2011). in ar there is a generation of diagnosis results, design proposals (from action planning) and the implementation of changes (action taking) and evaluation of implemented changes. this follows the canonical ar model (susman & evered, 1978; davison et al, 2004); cf. figure 2 above. all these four types of results count as lpc. in ar, following the demands of canonical ar (ibid), it is necessary to produce all four types. in practice research, one or several of these local practice contributions can be generated. it is not mandatory to create local changes through such research. in pr it is necessary to produce knowledge of practical value following the ideals of pragmatic inquiry (dewey, 1938). this means aiming for knowledge that is useful for general practice. the ideal of pr is to strive for inquiry topics that are relevant to both local and general practice and for situational knowledge that is useful for local practice and abstract knowledge that is useful for general practice. the research work directly related to local practices is called situational inquiry (goldkuhl, 2011). it is empirical research and it will be driven by a concern for the local practice. as an inquiry, this kind of activity contains a knowledge interest that is guided by a problematic situation and a need to resolve this problematic (indeterminate) situation into a resolved determinate situation (dewey, 1938). australasian journal of information systems volume 17 number 2 2012 67 the problematic situation that drives the inquiry is located in the local practice, otherwise the local practice should not be used as catalyst for inquiry and knowledge development. however, the comprehension of the problematic situation can in some cases mainly be framed by the researcher’s interest for knowledge development, i.e. an inquiry mainly guided by research interests. this research interest can of course be determined by an analysis of gaps in the scientific body of knowledge, but it can also be influenced by knowledge needs apprehended in practitioner communities (general practice needs). however, in many cases the problematic situation emerges directly from the local operational practice and will be encountered as such by its the ordinary members. this will drive the need for inquiry and possible changes. the role of the researchers will then be to take part in this inquiry process as knowledge resources. they will address the problematic situations in the local practices, but they may also add other inquiry drivers based on specific research interests. this relates to the division into problem solving interest and research interest discussed above. confer chiasson et al (2009) for an analysis of how research interest vs. problem solving interest can guide the inquiry process. pr is based on a differentiation into theorizing and empirical work following figure 5 above; cf. goldkuhl (2008; 2011). it also based on a differentiation between operational practice and its development (figure 4 and 5). a more enhanced model describing practice research in its context is found in figure 6. the empirical and developmental work is conducted as a situational inquiry. what is done in a situational inquiry is instrumental in relation to theorizing and the creation of abstract knowledge for research and practice communities. situational inquiry (which may be driven by a practical problem solving interest) is generative for researchers’ theorizing. theorizing local operational practice situational inquiry research community general practice practice research figure 6 practice research in context (based on goldkuhl, 2011) from the perspective of the local practice, situational inquiry functions as an arena for practical problem solving. the situational inquiry generates situational knowledge for the sake of the local operational practice. from the perspective of practice research and theorizing, the situational inquiry functions as a generator of empirical data and an arena for tests and trials of ideas and hypotheses. theorizing (in practice research) generates abstract knowledge in order to generalise findings and make them transferable to other practice contexts as well as additions to the knowledge base in the research community. this abstract knowledge, can during an on-going situational inquiry, be used as instruments for data collection and practical problem solving. the situational inquiry will in such cases be conducted in a theory-informed way. the used theories in the inquiry can be emergent australasian journal of information systems volume 17 number 2 2012 68 (generated during theorizing working in parallel with the situational inquiry) or can be extant from the knowledge base of the research community. after the empirical work (the situational inquiry) has been terminated, theorizing further refines the abstract knowledge aimed for research and practice communities. how a situational inquiry is conducted will always be a synthesis and continual balancing of problem solving interests (of local practice) and research interests. there may be more or less use of abstract pre-knowledge. how and when abstract knowledge is generated is also dependent on the combinations of these different interests. there may be use of pre-existing practical theories in situational inquiries (cronen, 2001; goldkuhl, 2008). such pre-knowledge can guide the inquiry process and make it theory-informed. there can also be reasons to test a certain theory, as its practical applicability and/or some hypothetical constructs. a situational inquiry is performed by researchers. practitioners can be more or less active in situational inquiries. there may be situational inquiries driven by researchers with a low degree of participation from practitioners. sometimes practitioners want an independent view from outside investigators. a design proposal may be requested for delivery by independent researchers-asdesigners. there may be other situations with full collaboration between researchers and practitioners. the degree of participation and collaboration is dependent on the purpose of the inquiry, the research interests and the type of intervention asked for by the practitioners. there are many good reasons for a close and active collaboration between researchers and practitioners. there may, however, be resource reasons, knowledge reasons or other reasons against active collaboration. action research in the light of practice research as can be read from the above description of pr, there are many resemblances between ar and pr. as a matter of fact, ar can be seen as a special kind of practice research. there are some differences both in view and properties which need to be emphasised. a comparison can be found in table 1. the articulation of practice research has been driven by both the identified needs for clarification of ar and ideas not sufficiently elaborated in ar. there are issues that are not dealt with in ordinary ar, as for example the ontology of the field of study. pr is based on an explicit view on the empirical field as consisting of a set of interrelated practices. such an explicit stance is not found in ar, although ar studies may adopt a view of this kind. the purpose to contributing to general practice in pr is not stated in ar (goldkuhl, 2008). pr avoids the hard and definitive demands on how the research process should be performed (high collaboration, implementation of changes) in ar. instead, the research design should be adapted to how the research interests and local practice interests can be combined in a constructive and generative way. pr has contributed with important clarifications which are also needed for further conceptualisation of ar:  activities: theorizing vs. situational inquiry  knowledge: situational knowledge vs. abstract knowledge  practices: local vs. general practice these categories should be used, not only when describing pr, but also in order to clarify ar. a refined model of ar, based on the pr conceptualisation, is found in figure 7. it is important to clearly distinguish between the theoretical work and the empirical work (that is performed mainly through a situational inquiry). the canonical ar cycle (susman & evered, 1978; davison et al, 2004) should be re-structured. reflective learning (theorizing) should not be seen as the last activity in a linear cycle. it should rather be seen as a continual activity working in tandem with the different developmental and empirical activities of situational inquiry (figure 7). it is thus important to distinguish between situational knowledge that is generated in situational inquiries and abstract knowledge that is generated through theorizing. australasian journal of information systems volume 17 number 2 2012 69 practice research action research main idea production of useful knowledge for general practice. research through local changes. contribution and target groups contribution of abstract knowledge to general practice and research community. situational contribution to local operational practices. problem solving contribution to local practices and scientific knowledge to research community. conceptualisati on of research activities theorizing and situational inquiry and continual interplay between these. different cyclical models exist with emphasis on practical problem solving. research interest vs. problem solving interest both interests guide situational inquiry. generated/furnished abstract knowledge is instrumental for problem solving. situational knowledge (=empirical data) is instrumental for theorizing. both interests are acknowledged. most emphasis in descriptions on problem solving. field of study practices. situations (wholes). use of preknowledge practical theories and methods can inform the situational inquiry. inductive as well as theoryinformed approaches can be used. different types of knowledge a differentiation into situational vs. abstract knowledge clearly related to the activities of situational inquiry and theorizing. local/empirical and theoretical knowledge. collaboration between researchers and practitioners can vary dependent on combinations of different purposes and interests. demanded, but can vary. relations between different research orientations encompassing view. pr can be conducted as action research, design research, evaluation research, case studies or other approaches. action research may be combined with other approaches e.g. design research. contribution to local practice different kinds of interventions (diagnosis, design proposals, implementation of changes/artefacts) dependent on agreements between researchers and practitioners. implementation of changes based on diagnosis and action plans. contribution to general practice mandatory to be done according to pr definition. different kinds of contributions (practical theories, generic models, methods). optional, not mandatory (not stated as a demand in definitions of ar). table 1 practice research and action research compared australasian journal of information systems volume 17 number 2 2012 70 even if there is a continual interplay between theorizing and empirics in ar, these activities and their distinct properties should not be blurred in ways that make these differences collapse. the focus on local practices in ar approaches should be accompanied by a complementary focus on general practice. the generation of abstract knowledge should not only have research community as the target group. the abstract knowledge should be useful for general practice. the conduct of ar should be informed by needs and problems of the local operational practice as well as of general practice. diagnosis intervention (action planning & taking) evaluation theorizing (reflection & learning) empirical/developmental work (situational inquiry) research community local operational practice general practice local practice contributions local needs & problems abstract knowledge contributions additions to sbk gpc extant theories general practice needs & problems figure 7 a refined view of action research based on practice research views and concepts empirical illustrations two empirical illustrations will be given below. the purpose of these case accounts is to illustrate practice research and action research and their similarities and differences. there are two cases from research in e-government; one in taxation and one in the social welfare sector. a taxation case this study concerns a taxation e-service. since several years, there exist an e-service for companies to declare vat and staff taxes every month. this e-service ran parallel with an old routine consisting of a paper form declaration. the national taxation agency had hoped that the e-service should have been very popular and that most companies would use this type of communication medium instead of the paper form. however, after several years, there were only about one third of the companies that used the e-service. the rest used the paper form. the taxation agency had conducted several investigations (through questionnaires) in order to inquire why more companies did not use the eservice. the national taxation agency participated in a larger research project concerning methods for service development. they asked the researchers to study this taxation e-service in order to obtain a better understanding of this partial failure. an evaluation study was initiated by one researcher (this author) in the research team. a thorough investigation was done comparing different service alternatives; the e-service and the paper form and different variants within these. it was necessary to disclose and model the two interconnected workpractices of the companies’ accounting routines and the processing of taxation information at the taxation agency. an inter-organisational service modelling was conducted. this service modelling was made from a specific theoretical perspective australasian journal of information systems volume 17 number 2 2012 71 which was labelled socio-instrumental (goldkuhl, 2009). the idea was to investigate affordances in the services and service pre-conditions 1) at both parties, 2) of both positive and negative kinds and 3) of both socio-communicative and techno-instrumental character. this analysis revealed not yet known problems with the different service alternatives. new insights arise especially concerning the lack of integration with the accounting systems at the companies and also some deficiencies in the e-service. a diagnosis report was written with the taxation agency as the main target group. this report is one input for considering different alternative measures for the taxation agency. the taxation study was a pr study, but it was not an ar study. it was limited to an evaluation study. being an ar study it should include proposals for changes (perhaps both it redesigns and organisational improvements) as well as more close cooperation between researchers and practitioners. the direct evaluation of the two service alternatives should be seen as a situational inquiry in terms of pr. the evaluation case was one empirical part in a research development of service modelling methods. there existed some methods prior to the evaluation. other method components (concepts, notations) were developed during the evaluation process due to discovered needs and generated insights. these new methods thus emerged as responses to practical and conceptual needs. the development of these new concepts and notations was carried out in a “theorizing” activity supporting the service evaluation. the evaluation of the e-service had one function of a testbench of methods for modelling e-services in context. after the evaluation, experiences from the e-service modelling have been reported (to the research community and to general practice). these experiences have also been fed into further development of methods for e-service design and evaluation (goldkuhl & röstlinger, 2007). the evaluation was also very generative concerning an enhanced conceptualization of e-services as coservices and socio-instrumental pre-conditions for e-service use (goldkuhl, 2009). these are examples of theorizing occurring both during and after the evaluation process (situational inquiry). a social welfare case this case study is from a project working with it development in the social welfare sector. several municipalities participated in the project in order to develop joint it solutions. these solutions comprise both an e-service application for clients and a multi-query application for collecting information about clients from state agencies. one impetus for starting the project was a new regulation that gives the municipalities better possibilities to obtain information about clients. the handling of client information within the public sector is severely restricted due to data protection regulations. the new regulation makes it easier for municipalities to obtain information about the clients’ economic situation. a multi-query application was developed and launched to the participating municipalities. information can now, on demand, be transferred electronically and immediately from state agencies (like the social insurance agency and the board for study support) to the social welfare offices at the municipalities. this communication was earlier mainly conducted through telephone calls and a slow batch query application. two researchers (one of them the author) participated in this project as action researchers and design researchers. the integration of action research and design research in this project has been described in goldkuhl (2012). the roles of the two participating researchers have been to actively conduct development tasks (like process modelling, information modelling, user-interface design, xml schema design, program coding, testing and deployment) besides traditional research tasks like data collection and analysis. the original research interest was to study inter-organisational development of e-infrastructure in e-government. australasian journal of information systems volume 17 number 2 2012 72 this research is an example of practice research that is also action research. the work in the development project followed a traditional problem solving model of ar: an initial diagnosis was conducted, mainly through process modelling, conceptual modelling, legal analysis and studies of existing legacy systems in the social welfare offices. design proposals were elaborated for a new it artefact (the multi-query application) and for revised work processes. lo-fi prototypes of the userinterface design and new process models were produced, which corresponds to action planning in the model of susman & evered (1978); figure 2 above. the next step (action taking) comprised coding, testing and deploying the new artefact. after implementation in several municipal welfare offices, evaluations of use have been conducted. this has been a typical ar project with intense collaboration between researchers and practitioners. several ideas from the researchers and other concepts from prior research have been used in the design process. a particular research interest was to study principles for e-infrastructure evolution (e.g. hanseth & lyytinen, 2010; markus et al, 2006). during the process we also formulated design principles about conceptual, processual and legal transparency. the formulation of these design principles was performed by the two researchers when reflecting on the design process. this reflection and formulation of design principles were dealt with in a “theorizing” activity that supported the design process in the situational inquiry. these different design principles guided the design of the multi-query application. the design case (i.e. the situational inquiry) has produced a lot of empirical data which are used in theorizing. the theoretical analysis has generated several important insights about legal issues, interoperability and e-infrastructure evolution in e-government. case comparison the two cases are both good examples of practice research. they comprise both local practice contributions and the generation of abstract knowledge aimed for general practice and research community. the taxation study is not an ar study. it is restricted to an evaluation study. there are no changes implemented as a result of this study so far. the level of collaboration was rather low. the social welfare case is a typical ar project. this case is also an example of design research. in table 2 there is a comparison between the two cases. it can be seen that the two cases fulfil the demands of what counts as practice research. from the comparison a variation between the two cases as practice research can also be seen. conclusions this paper has contributed with a conceptual inquiry into action research leading to identification of several (partially unresolved) issues concerning such an approach. there are several obscurities and ambiguities within ar that need to be resolved. these identified issues have informed the articulation of practice research, which is another important contribution of this paper. practice research has been described concerning its two main activities (theorizing and situational inquiry) and its contributions to different target groups (local practice, general practice and research community). the conceptualisation of pr can be used to clarify issues of ar. a comparison has also been made between ar and pr in order to reveal resemblances and differences. this comparison has also clarified that ar can be seen as a special kind of practice research. australasian journal of information systems volume 17 number 2 2012 73 taxation case social welfare case type of research practice research as evaluation research. practice research as combined action research & design research. researcher role and collaboration an independent researcher study. no active involvement of practitioner in the study. collaborative design between researchers, it designer and social welfare officers. problem solving interest (practical interest) knowledge needed why an eservice was not sufficiently used. new legal conditions gave a window of opportunity for new it-based inter-organisational information transfer. studied practices taxation practice. parts of companies’ accounting practices. e-service evaluation practice. municipal social welfare practices. state agencies’ practices of handling client information. practices for design of e-infrastructure. local practice contribution diagnosis of an e-service (use vs. non-use). a new it artefact, with designed new work processes based on diagnosis/modelling of current situation. research interests initially & lasting: to test modeling methods for e-services. emerged: new views on eservices and service preconditions. initially & lasting: to study e-infrastructure development in egovernment. emerged: design principles about transparency. use of prior abstract knowledge socio-instrumental perspective on e-services. initial methods for eservice modelling. methods for process and conceptual modelling. design principles for e-infrastructure development. emergence of new abstract knowledge new concepts and methods for eservice modelling. design principles about different kinds of transparency in it artefacts end products aimed for general practice and research community a new method package for e-service modelling. a developed definition of e-services. new knowledge on legal issues, interoperability and einfrastructure evolution in e-government. table 2 comparison of two practice research cases the articulation of practice research should not at all be seen as a rejection of action research or neglecting the importance of this research approach. on the contrary, ar should be acknowledged for its contribution to local practices and its use of practical problem solving as a catalyst for the generation of scientific knowledge. the needs for the articulation of practice research can partially be found in some unresolved issues concerning ar described above. baskerville & myers (2004) state australasian journal of information systems volume 17 number 2 2012 74 that the paradigmatic foundations of ar can be found in pragmatism; cf. also oquist (1978). however, the full implications of this pragmatist foundation have not been made. practice research should be seen as an approach to bring pragmatism a step further in concrete research. practice research is based on one pragmatist pillar to improve practice through knowledge (dewey, 1938). this means not only through direct intervention into local practices, but more importantly a contribution to general practice through scholarly writing. practice research rests also on a second pragmatist pillar that conceives the empirical field as a set of interconnected practices (e.g. schatzki et al, 2001). the title of this paper “from action research to practice research” should not be interpreted as a quest to move away from ar in the direction of pr. action research should be preserved, but at the same time be further developed through the aid of this practice research articulation. “from action research to practice research” should be interpreted in the following way: action research with 1) its fundamental pragmatist properties and 2) its unresolved issues and needs for clarification have contributed to the articulation of practice research. this means that this paper has made contributions to both the clarification of action research and to the articulation of practice research. this should also be seen as an important contribution to further development of qualitative research in information systems. there are of course needs for further research of both conceptual and empirical kind concerning the application and development of ar and pr. this should comprise the study of not only action research as practice research, but also other forms of pr as design research and evaluation research. references argyris c, putnam r, mclain smith d (1985) action science. concepts, methods and skills for research and intervention, jossey bass, san francisco avison d, baskerville r, myers m (2001) controlling action research projects, information technology & people, vol 14, no 1 p 28-45 baskerville r (1997) distinguishing action research from participative case studies, journal of systems and information technology, vol 1 (1), p 24-43 baskerville r (1999) investigating information systems with action research, communication of ais, vol 2 baskerville r, myers m (2004) special issue on action research in information systems: making is research relevant to practice – foreword, mis quarterly, vol 28 (3), p 329-335 baskerville r, pries-heje j (1999) grounded action research: a method for understanding it in practice, accounting, management & information technology, vol 9, p 1–23 baskerville r, wood-harper t (1998) diversity in information systems action research methods, european journal of information systems, vol 7, p 90–107 benbasat i, zmud r w (1999) emprical research in information system research: the practice of relevance, mis quarterly, vol 23 (1), p 3-16 bleicher j (1980) contemporary hermeneutics. hermeneutics as method, philosophy and critique, routledge & kegan paul, london blum f (1955) action research – a scientific approach?, philosophy of science, vol 22 (1), pp 1-7 australasian journal of information systems volume 17 number 2 2012 75 cassell c, johnson p (2006) action research: explaining the diversity, human relations, vol 59 (6), p 783–814 chandler d, torbert b (2003) transforming inquiry and action. interweaving 27 flavors of action research, action research, vol 1 (2), p 133–152 checkland p (1991) from framework through experience to learning: the essential nature of action research, in nissen h-e, klein h, hirschheim, r (eds, 1991) information systems research: contemporary approaches and emergent traditions, north-holland, amsterdam chiasson m, germonprez m, mathiassen l (2009) pluralist action research: a review of the information systems literature, information systems journal, vol 19 (1), p 31–54 cole r, purao s, rossi m, sein m (2005) being proactive: where action research meets design research, proceedings of the twenty-sixth international conference on information systems, las vegas, p 325-336 cronen v (2001) practical theory, practical art, and the pragmatic-systemic account of inquiry, communication theory, vol 11 (1), p 14-35 cronholm s, goldkuhl g (2004) conceptualizing participatory action research – three different practices, electronic journal of business research methods, vol 2 (2) cuff ec, payne gcf (ed, 1979) perspectives in sociology, george allen & unwin davenport t, markus l (1999) rigor vs. relevance revisited: a response to benbasat and zmud, mis quarterly, vol 23 (1), p 19-23 davison r m, martinsons m g, kock n (2004) principles of canonical action research, information systems journal, vol 14, p 65–86 dewey j (1938) logic: the theory of inquiry, henry holt, new york elden m (1983) democratization and participative research in developing local theory, journal of occupational behaviour, vol 4 (1), p 21-33 elden m, crisholm (1996) emerging varieties of action research, human relations, vol 46 (2), p 121-142 gherardi s (2000) practice-based theorizing on learning and knowing in organizations, organization, vol 7 (2), p 211-223 gherardi s (2009) the critical power of the 'practice lens', management learning, vol. 40 (2), p 115–128 goldkuhl g (2008) practical inquiry as action research and beyond, in proceedings of the 16th european conference on information systems, galway goldkuhl g (2009) socio-instrumental service modelling: an inquiry on e-services for tax declarations, in persson a, stirna j (eds. 2009) poem 2009, lnbip 39, pp. 207–221, springer, berlin goldkuhl g (2011) the research practice of practice research: theorizing and situational inquiry, systems, signs & actions, vol 5 (1), p 7-29 goldkuhl g (2012) pragmatism vs. interpretivism in qualitative information systems research, european journal of information systems, vol 21 (2), p 135-146 australasian journal of information systems volume 17 number 2 2012 76 goldkuhl g, röstlinger a (2002) the practices of knowledge – investigating functions and sources, in proc of the 3rd european conference on knowledge management (3eckm), dublin goldkuhl g, röstlinger a (2007) clarifying government – citizen interaction: from business action to generic exchange, in proceedings of the 4th scandinavian workshop on e-government, örebro gregor s (2008) building theory in a practical science, in hart d, gregor s (eds, 2008) information systems foundations: the role of design science, anu e press, canberra gummesson e (1991) qualitative methods in management research, sage, newbury park hanseth o, lyytinen k (2010) design theory for dynamic complexity in information infrastructures: the case of building internet, journal of information technology, vol 25, 1-19 heron j, reason p (2001) the practice of co-operative inquiry: research ‘with’ rather than ‘on’ people, in reason p, bradbury h (eds, 2001) handbook of action research, sage, london hult m, lennung s-å (1980) towards a definition of action research: a note and bibliography, journal of management studies, vol 17, p 241-250 huxham c, vangen (2003) researching organizational practice through action research: case studies and design choices, organizational research methods, vol 6 (3), p 383-403 iivari j, venable j (2009) action research and design science research – seemingly similar but decisively dissimilar, 17th european conference on information systems, verona israel b, schurman s, hugentobler m (1992) conducting action research: relationships between organization members and researchers, journal of applied behavioral science, vol 28 (1), p 74-101 järvinen p (2005) action research as an approach in design science, the euram (european academy of management) conference, munich keen p g w (1991) relevance and rigor in information systems research: improving quality, confidence, cohesion and impact, in nissen h-e, klein h, hirschheim, r (eds, 1991) information systems research: contemporary approaches and emergent traditions, northholland, amsterdam lee a (1999) rigor and relevance in mis research: beyond the approach of positivism alone, mis quarterly, vol 23, (1), p 29-33 levina n, vaast e (2006) turning a community into a market: a practice perspective on information technology use in boundary spanning, journal of management information systems, vol 22 (4), p 13-37 ludema j, cooperrider d, barrett f (2001) appreciative inquiry: the power of the unconditional positive question, in reason p, bradbury h (eds, 2001) handbook of action research, sage, london lundin r, wirdenius h (1990) interactive research, scandinavian journal of management, vol 6 (2), p 125-142 lyytinen k (1999) empirical research in information systems: on the relevance of practice in thinking of is research, mis quarterly, vol 23, (1), p 25-27 kemmis s, mctaggart r (2000) participatory action research, in denzin nk, lincoln ys (eds, 2000) handbook of qualitative research, 2 ed, sage, thousands oakes australasian journal of information systems volume 17 number 2 2012 77 klein g, jiang j, saunders c (2006) leading the horse to water, communications of ais, vol 20, article 13 kock n, (ed. 2007) information systems action research. an applied view of emerging concepts and methods, springer kock n, gray p, hoving r, klein h, myers m, rockart j (2002) is research relevance revisited: subtle accomplishment, unfulfilled promise, or serial hypocrisy?, communications of ais, vol 8, pp 330-346 kock n, lau f (2001) information systems action research: serving two demanding masters, information technology & people, vol 14 (1), p 6-11 lau f (1997) a review on the use of action research in information systems studies, in lee a, liebenau j, degross j (eds, 1997), information systems and qualitative research, chapman & hall, london lewin k (1947) frontiers in group dynamics ii. channels of group life; social planning and action research, human relations, vol 1 (2), pp. 143-153 mckay j, marshall p (2001) the dual imperatives of action research, information technology & people, vol 14 (1) p 45-59 mckay j, marshall p (2002) action research: a guide to process and procedure, in proc of the european conference on research methods in business & management, university of reading mckay j, marshall p (2006) driven by two masters, serving both: the interplay of problem solving and research in information systems action research projects, in kock n, (ed. 2007) information systems action research. an applied view of emerging concepts and methods, springer markus l, steinfield c, wigand r, minton g (2006) industry-wide information systems standardization as collective action: the case of the u.s. residential mortgage industry, mis quarterly, vol 30, p 439-465 mathiassen l (2002) collaborative practice research, information technology & people, vol 15 (4), p 321-345 metcalfe m (2008) pragmatic inquiry, journal of the operational research society, vol 59, p 10911099 miettinen r, samra-fredericks d, yanow d (2009) re-turn to practice: an introductory essay, organization studies, vol 30 (12), p 1309–1327 mumford e (2001) advice for an action researcher, information technology & people, vol 14 (1), pp 12-27 nielsen p a (2007) is action research and its criteria, in kock n, (ed. 2007) information systems action research. an applied view of emerging concepts and methods, springer oquist p (1978) the epistemology of action research, acta sociologica, vol 21 (2), pp 143-163 orlikowski w j (2000) using technology and constituting structures: a practice lens for studying technology in organizations, organization science, vol 11 (4), pp 404-428 parks p (2001) knowledge and participatory research, in reason p, bradbury h (eds, 2001) handbook of action research, sage, london australasian journal of information systems volume 17 number 2 2012 78 pasmore w a, stymne b, shani a b, mohrman s a, adler n (2008) the promise of collaborative management research, in shani a b, mohrman s a, pasmore w a, stymne b, adler n (eds, 2008) handbook of collaborative management research, sage, los angeles peters m, robinson v (1984) the origins and status of action research, journal of applied behavioral science, vol 20 (2), pp 113-124 rapoport r n (1970) three dilemmas in action research, human relations, vol 23 (6), p 499-513 reason p (1999) integrating action and reflection through co-operative inquiry, management learning, vol 30 (2), p 207–226 reckwitz a (2002) toward a theory of social practices. a development in culturalist theorizing, european journal of social theory, vol 5 (2), p 243–263 revans r w (1982) what is action learning?, journal of management development, vol. 1 (3), p 6475 rosemann m, vessey i (2008) toward improving the relevance of information systems research to practice: the role of applicability checks, mis quarterly, vol 32 (1), p 1-22 schatzki t r (1996) social practices. a wittgensteinian approach to human activity and the social, cambridge university press schatzki t r, knorr cetina k, von savigny e (eds, 2001) the practice turn in contemporary theory, routledge, london schein e (2001) clinical inquiry/research, in reason p, bradbury h (eds, 2001) handbook of action research, sage, london scollon r (2001) mediated discourse. the nexus of practice, routledge, london sein m, henfridsson o, purao s, rossi m, lindgren r (2011) action design research, mis quarterly, vol 35 (1), p 37-56 stevenson c (2005) practical inquiry/theory in nursing, journal of advanced nursing, vol 50 (2), 196–203 strauss a, corbin j (1998) basics of qualitative research. techniques and procedures for developing grounded theory, 2 nd edition, sage, newbury park susman g i, evered r d (1978) an assessment of the scientific merits of action research, administrative science quarterly, vol 23 (4) p 582-603 torbert w (1999) the distinctive questions developmental action inquiry asks, management learning, vol 30 (2) p 189–206 vidgen r, braa k (1997) balancing interpretation and intervention in information systems research: the action case approach, in lee as, liebenau j, degross j i (eds, 1997) information systems and qualitative research, chapman & hall, london whittington r (2006) completing the practice turn in strategy research, organization studies, vol 27 (5), p 613–634 microsoft word ajis special 2002.doc ajis special issue december 2002 50 systems thinking about purpose gaye lewis university of south australia sa, australia gaye.lewis@unisa.edu.au abstract this paper argues that in the context of human activity systems, the concept of purpose is critically important and that giving purpose a central role in the development and management of human activity systems can lead to more flexible, effective and autonomous systems. the systems thinking literature is reviewed in order to consolidate and assess current thinking about purpose. the importance of intrinsic purpose is highlighted. implications for practice in terms of information systems design are illustrated by contrasting two different approaches to the design of knowledge management systems. an analysis of corporations as purposeful systems highlights some of the benefits of a purposeful systems approach. directions for future research with a focus on purpose are suggested. introduction systems thinking research has long argued that the management of human activity systems requires a different management approach than that used to manage simple, mechanistic systems (eg. checkland, 1994; ackoff, 1999; ulrich, 1983). human activity systems are made up of people and one of the distinctive characteristics that separates people from other species and non-living objects is their ability to formulate and act in accordance with a purpose. the relevance of purpose to human activity systems has been mentioned before in the systems thinking literature, but often using different language and not always with purpose as the main focus of the research. this literature will be revisited with the explicit intention of synthesizing the thinking about purpose into one paper, which it is hoped will act to highlight a topic that is foundational to those who think about system design. stebbing (1952) wrote about the importance of purpose in thinking and said “to think logically is to think relevantly to the purpose that initiated the thinking: all effective thinking is directed to an end.” elder and paul (2001) argue that “raising human goals and desires to the level of conscious realization is an important part of critical thinking”. in order to help with the development of effective human activity systems, suitable for encouraging innovative and adaptive ‘thinking’ managers and employees, the concept of purpose and its application will be explored. more specifically, this paper will explore the argument that purpose is a key concept that must be incorporated into the development and management of human activity systems. the importance of purpose has also been emphasised in the management literature. bartlett and ghoshal (1994) write that, “senior managers of today’s large enterprises must move beyond strategy, structure, and systems to a framework built on purpose, process, and people”. briskin (1999) writes that “organizations facing a volatile market must address purpose as they attempt to undo much of what they once found to be successful.” the argument for incorporating purpose into management thinking has even stronger validity in the context of information systems development. as bartlett and ghoshal (1994) point out, the underlying problem with traditional approaches is in their objective “to create a management system that minimise(s) the idiosyncrasies of human behaviour”. with information systems, the tendency to focus on the technology components of the system accentuates the problems inherent in traditional approaches. so, although this paper focuses on purpose in human activity systems in general, the conclusions are especially applicable to the design and development of information systems. the paper begins with a discussion of the concept of purpose, and then develops a classification of different systems in terms of types of purpose. this classification is used to review the literature on systems thinking in order to critically evaluate the importance of purpose in the management and development of human activity systems. the main aim of the paper is to argue for the importance of purpose in the management and design of human activity systems by drawing on the systems thinking literature. some implications for practice are suggested by analysing and assessing the purposefulness of some example information systems and corporations. the final section includes a summary and some potential directions for future research. purpose the oxford english dictionary (1991) defines purpose as, “the object for which anything is done or made, or for which it exists; the result or effect intended or sought; end, aim”. in common language, the same word – purpose – is attributed to inanimate objects and to humans – a clock’s purpose is to tell the time in the same way that a human’s purpose in setting the alarm is to wake up on time. for the analysis here, it is important to distinguish between these two different uses of the word purpose, as will be discussed further later in this section. suffice it to say here that purpose can only be exhibited by human beings and represents a form of internal motivation – why people do what they do. another important characteristic of humans that enables the formation of purpose is our ability to be self-reflective. it is in the process of reflection that we develop and become aware of our individual purposes. ajis special issue december 2002 51 aristotle (strong, 2000) believed that to understand the existence of artificial (man-made) phenomena, four general principles (causes) must be explored. his description of the final cause is “the reason for the phenomenon’s existence or that purpose for which it was created”, which is consistent with the modern definition of purpose above. aristotle's other causes are the material cause which represents the phenomenon’s physical manifestation, the efficient cause which identifies the processes involved in producing the phenomenon and the formal cause which embodies the essence of the phenomena. these causes are concerned with what the phenomenon is and how it is made rather than exploring why it exists. taking aristotle's point that in order to understand a phenomenon it is critical to explore its purpose, the distinction between artificial and natural phenomena and their relation to purpose will be explored in detail later in this section. further support for the importance of purpose is evidenced by the way it is applied within the legal system. the degree to which intent and self-reflection can be established in a criminal act can have a profound influence on the severity of the resulting sentence. a verdict can range from manslaughter (no intent to kill) to second degree murder (intent to kill is not pre-meditated) to first degree murder (pre-meditated intent to kill). the harshest sentences correspond with the highest degree of self-reflection and purpose. types of purpose for the analysis in this paper, it is important to explicitly distinguish between two types of purpose – extrinsic and intrinsic. extrinsic or ascribed purpose is the purpose of a system as seen by an observer or designer. a system with extrinsic purpose has been designed intentionally for fitness of purpose and is perceived to serve a purpose. ‘purposive’ is the terminology often used to describe this type of purpose. intrinsic purpose can only be attributed to systems involving people and represents the free will of the person/people within the system to act. ‘purposeful’ is the terminology used to describe this type of purpose. designed systems natural systems i. extrinsic + intrinsic purpose ii. intrinsic purpose human activity systems iii. extrinsic purpose iv. no purpose non-human systems figure 1: system classes defined according to types of purpose. figure 1 characterises two dimensions of system classes according to whether they exhibit extrinsic and/or intrinsic purpose. designed systems always have extrinsic purpose – they are designed and used for a reason. human activity systems are the only type capable of exhibiting intrinsic purpose. a river is an example of a non-human, natural system with no purpose (quadrant iv). the argument that a river does have a purpose presupposes some form of designer and, since this paper is only concerned with improving the design of manmade systems, the river and other natural, non-human systems can be classified as having no purpose. a person or a family are both examples of human, natural systems with intrinsic purpose (quadrant ii). an alarm clock is an example of a designed, non-human system which therefore has extrinsic purpose only (quadrant iii). an information system or a football team are both examples of designed, human activity systems potentially exhibiting both extrinsic and intrinsic purpose (quadrant i). figure 1 will be used to review some of the literature on systems thinking in reference to purpose. the focus in this paper is on designed, human activity systems – quadrant i. management thinking as applied to designed human activity systems, such as a corporation or an information system, can be enhanced if it gives a central role to both extrinsic and intrinsic purpose. potential enhancements are a result of focussing on this uniquely human aspect of human activity systems and include increased understanding, flexibility and motivation. observer’s purpose before reviewing the systems thinking literature in terms of intrinsic and extrinsic purpose, the concept of the observer’s purpose in systems thinking will be described. the observer's purpose is the starting point for using systems concepts to describe an external reality, and an understanding of observer purpose is required in order to interpret systems thinking literature in terms of purpose. based on his or her particular purpose (which may be to describe, use or design something), an observer will define the entities that make up the system, describe the relationships between the entities and so delineate the system's boundary with the external environment. jordan’s (1968) analogy of “recognising various different figures and backgrounds, depending upon, among other things (our) interests” provides a visual image of this concept. with physical systems such as an alarm clock, it can be difficult to separate the external reality from our system conception of it. with abstract systems ajis special issue december 2002 52 such as a political system, it is more readily apparent that our conception of the system is the basis for its existence. delving into the systems concept in more detail, checkland (1981) differentiates between two approaches for describing or analysing a system. the ‘black box’ approach identifies inputs and outputs that cross the boundary between the system and its external environment requiring a description of the transformations which must occur within the system to produce the outputs from the inputs. the second approach describes the internal state changes of the system as a result of inputs from the environment. the first approach looks from the outside in, whereas the second approach looks from the inside out. with human activity systems, the second approach provides more opportunities for the description and development of intrinsic purpose – the purposes of the people within the system. extrinsic purpose this section will explore the concept of extrinsic purpose as referenced by the systems thinking literature. although the terminology may be inconsistent, several authors have looked at the concept of extrinsic purpose as defined in the previous section. in general, the authors referred to in this section have focussed on extrinsic purpose only and not considered intrinsic purpose. a richer, more comprehensive conceptualisation of systems can be developed with consideration of both types of purpose. rosenblueth and wiener (1950) present an in-depth discussion of purposeful vs. non-purposeful behaviour. they argue that “if the term purpose is to have any significance in science, it must be recognizable from the nature of the act, not from the study of or from any speculation on the structure and nature of the acting object”. in accordance with their argument, they only consider extrinsic purpose, and account for intrinsic control mechanisms without considering intrinsic motivation, which they would consider to be part of the structure or nature of the acting object. however, intrinsic motivation or purpose is also recognisable in the ability of purposeful ‘objects’ to change their actions even in a stable environment. the approach in this paper differs from theirs in that intrinsic motivation is considered critically important. another difference in approach relates to the method of study. their approach to scientific study is similar and limited to checkland’s ‘black box approach’ whereas checkland’s second approach offers more opportunities for the study of intrinsic motivation. dennett (1997) presents an argument for not differentiating between intrinsic and extrinsic purpose. his “intentional stance” for describing entities is similar to rosenblueth and wiener’s approach to studying the nature of acts rather than the structure of an object. my interpretation of his argument is that since we are made up of parts and have evolved from parts with no intrinsic purpose, it is conceivable that we could design robots (once again made of parts with no intrinsic purpose) that would in effect evolve and display behaviour as purposeful as ours. this paper assumes that there is a fundamental difference between free will or intrinsic purpose and extrinsic purpose. in any case, the line of reasoning taken by dennett is not relevant here. whether or not ‘purposeful’ robots can be designed does not affect the discussion of purposeful people and groups of people in this paper. cell characteristics example 1 structural, purposive, mechanical a road network 2 structural, purposive, organismic a suspension bridge 3 structural, non-purposive, mechanical a mountain range 4 structural, non-purposive, organismic a bubble 5 functional, purposive, mechanical a production line 6 functional, purposive, organismic living organisms 7 functional, non-purposive, mechanical changing water flow 8 functional, non-purposive, organismic the space-time continuum figure 2: jordan’s (1968) system classification. jordan (1968) uses three different characteristics to classify systems. he proposes that systems differ importantly in their rate of change (static or dynamic), purpose (purposive or non-purposive) and connectivity (sparse/mechanical or dense/organismic). eight system classes (cells) are defined by using all possible combinations of the three characteristics, as illustrated in figure 2. jordan only considers extrinsic purpose (purposiveness) and so cells with the purposive characteristic map into the category of designed systems and cells that are non-purposive map into the natural systems category of figure 1 as diagrammed in figure 3. since there is no consideration of intrinsic purpose, the difference between human and non-human systems is not relevant, and jordan’s classification does not specifically address the key area of interest in this paper – the purposefulness of human activity systems. ajis special issue december 2002 53 designed systems natural systems i. & iii. cells 1,2,5,6 ii. & iv. cells 3,4,7,8 human activity systems & non-human systems figure 3: jordan’s systems taxonomy mapped onto figure 1. intrinsic purpose ackoff and emery’s (1972) definition of a purposeful system is “one that can change its goals in constant environmental conditions; it selects goals as well as the means by which to pursue them. it thus displays will. human beings are the most familiar examples of such systems”. this definition is consistent with the human designed and natural systems in figure 1 (quadrants i and ii). however, their approach to the study of purposeful systems is more in line with rosenblueth and wiener’s tactic in that it studies purpose by analysing the nature of acts or the behaviour of the system. their assessment of purposeful systems attempts to define an objective, quantifiable account of purposeful behaviour from an outsider’s point of view – checkland’s ‘black box’ approach. the approach used by ulrich (1983) (described in a later section) asks questions about the system’s intentionality and self-reflectiveness which is more in line with checkland's second approach and more appropriate to the study of systems with intrinsic purpose. ackoff (1999) develops a classification of systems where “the critical classifying variable is purpose.” his classification consists of a hierarchy of four system types – deterministic, animated, social and ecological and he differentiates the types based on whether the parts and/or the whole of the system are purposeful. deterministic systems such as alarm clocks are not purposeful and do not have purposeful parts. animated systems such as people are purposeful, but their parts are not. social systems are purposeful and so are their parts. ecological systems are not purposeful, but their parts are. ackoff is only concerned with intrinsic purpose, and so makes no distinction between designed and natural systems. his system classification maps to figure 1 as illustrated in figure 4. ackoff then describes the progression of thinking about organisations originally as deterministic systems then animated systems and now as social systems. as deterministic systems, the main purpose of an organisation was to make a profit and no other purposes were considered important. as an animated system, the main purpose of the enterprise became one of survival, but the parts of the system (the employees) were still not considered purposeful. he emphasises that a systemic transformation from an animated system to a social system (where both the whole and the parts are purposeful) is required “if an enterprise is to be successful in the current environment which is characterized by an increasing rate of change, interdependence, complexity, production, and dependence on knowledge and information.” moving from an animated system to a social system in terms of figure 1 involves moving from thinking in terms of non-human systems (quadrant iii) to human activity systems (quadrant i) with the main difference being the incorporation of the intrinsic purposes of employees. designed & natural systems i. & ii. ecological systems social systems human activity systems iii. & iv. animated systems deterministic systems non-human systems figure 4: ackoff’s (1999) system classification mapped onto figure 1. boulding (1956) developed a classification of system types based on an intuitive judgement of the degree of complexity. his types range from structures and frameworks at level 1 to transcendental systems at level 9 (refer to figure 5) with new characteristics emerging at each level. intrinsic purpose or motivation would be considered an emergent property of level 7 systems, which are represented by man. some might argue that it also exists at level 6 (animals), but that discussion is not relevant here. self-consciousness and self-reflection are two characteristics that also emerge at level 7. these characteristics of purposefulness, self-consciousness and self-reflection would then also exist at level 8 and level 9 (socio-cultural and transcendental systems). ajis special issue december 2002 54 level type of system example 1 structures, frameworks bridge 2 clock-works clock 3 control mechanisms thermostat 4 open systems flames 5 lower organisms plants 6 animals birds 7 man human beings 8 socio-cultural systems families 9 transcendental systems ? figure 5: boulding’s (1956) system classification. n terms of figure 1, boulding’s classification makes no distinction between natural and designed systems, since both can exhibit varying degrees of complexity and that is his main concern. since purpose (intrinsic purpose) emerges as a characteristic in systems at level 7 and above, boulding’s levels 1 through 8 can be mapped onto figure 1 as illustrated in figure 6. level 9 (transcendental systems) appears outside of the figure 1 classification. designed & natural systems i. & ii. levels 7 & 8. human activity systems iii. & iv. levels 1 – 6. non-human systems figure 6: boulding’s system levels as mapped onto the classes of figure 1. boulding argues that any problem that exhibits the characteristics of a certain level cannot be solved with a lower level system. one of the arguments of this paper is that approaches such as those used in mbo (management by objectives) programs and goal-oriented systems are not capable of solving highly complex management problems, because they are situated somewhere within levels 4-6 and the problems they are trying to solve are level 8 problems in boulding’s classification. purposefulness, self-consciousness and self-reflection of systems are not considered to be critical characteristics in these goal-oriented approaches. checkland (1981) defines a set of systems classes intended to be helpful in developing research approaches to management problems – not simply in classifying different types of systems. his classes are as follows: • natural physical systems • designed physical systems • designed abstract systems • human activity systems • transcendental systems checkland’s system types map to those of figure 1 as illustrated in figure 7. transcendental systems are once again outside of the classification. checkland does not differentiate between natural human and natural nonhuman systems. designed systems natural systems i. human activity systems ii. natural physical systems human activity systems iii. designed, physical systems designed, abstract systems iv. natural physical systems non-human systems figure 7: checkland’s (1981) system types mapped onto figure 1. checkland defines human activity systems as activities ordered in wholes as a result of some underlying purpose or mission. an important realisation (checkland, 1994) is that “all real-world problem situations are characterized by the fact that they reveal human beings seeking or wishing to take purposeful action”, which once again supports the incorporation of purpose into the management and design of human activity systems. he also states that the perception of human activity systems is often emphasised by the existence of other ajis special issue december 2002 55 designed systems. in the case of information systems, the activities are emphasised by the technology that supports them. checkland stresses that it is important to separate the two but doesn’t elaborate. with information systems, the focus often tends to be the technology and separating the two systems encourages more appreciation for the human aspects of information systems, including purpose. ulrich (1983) developed a purposeful systems paradigm as a conceptual framework to encourage designers of purposeful systems to be social planners rather than tool designers. ulrich’s definition of a purposeful system is as follows: “we designate a system s a purposeful system if s if self-reflective with respect to its own normative implications, seen from the point of view not only of the involved but also the affected, and if s has at least partial autonomy in determining its client, its purposes, etc. “partial autonomy” precludes the possibility that a system is purely extrinsically motivated; it means that the system can exercise its own will in choosing its goals.” ulrich goes on to reason that since only humans are self-reflective and autonomous, purposeful systems are made up of groups of individuals where some of their purposes are convergent or interdependent. the task of the purposeful designer is then to design for the purposeful individuals who make up the system i.e. enhance and facilitate critical reflection and allow for intrinsic motivation within the system. purposeful system design is about the design of (for) social systems rather than the design of tools. ulrich’s concern is with the design of systems involving people, represented by quadrant i in figure 1. ulrich then provides a taxonomy for categorising a system’s purposefulness in terms of the three basic kinds of problem-solving processes that a system must perform – inquiry, action and valuation. for each of the processes the degree of intrinsic motivation and opportunity for critical reflection should be examined to assess the system’s achievement of purposefulness. inquiry assesses whether the system produces meaningful knowledge (with respect to its purpose). action assesses how the system uses this knowledge purposefully. valuation measures the systems’ ability to responsibly evaluate the effects it is having on clients and anyone else who might be affected through the production and use of its knowledge. these categories add another dimension to quadrant i of figure 1 and can be used to assess the purposefulness of designed human activity systems. this completes the section of the paper reviewing systems thinking literature related to purpose. support has been drawn from the literature for the argument that both extrinsic and intrinsic purpose should be considered critically important in the management and design of human activity systems. the next section will look at purpose within specific contexts by using ulrich’s taxonomy to assess the purposefulness of information systems and corporations. implications for practice purposeful information systems as an illustration of the application of ulrich’s paradigm, knowledge management systems as classified by alavi (2000) will be analysed. alavi defines a knowledge management system (kms) as “an it-based system developed to support and enhance the primary organizational knowledge management processes of knowledge generation, knowledge codification, and knowledge transfer”. two underlying models of kmss are described “(1) the repository model aims at codification of knowledge” and “(2) the network model aims at using the power of information and communication technologies to support the flow of knowledge in organizational settings and among networks of individuals”. using ulrich’s assessment approach, we can first ask whether either of these types of systems are purposeful inquiring systems, i.e. do they produce meaningful knowledge with respect to their purpose, where purpose as defined above is to support and enhance organizational knowledge management processes. looking first at the repository model, it does not produce any knowledge – it is only concerned with codifying or collecting knowledge and so cannot be considered a purposeful inquiring system. the network model, however, does potentially allow for the production of knowledge by facilitating communication among individuals. to evaluate whether each type of kms is a purposeful action system, the question to ask is how the system makes purposeful use of the knowledge produced. since the repository model is not concerned with producing knowledge, but only with storing and retrieving knowledge, this question does not apply. the network model does provide the opportunity for purposeful use of knowledge produced, but this can only be evaluated in the context of a specific system. alavi provides an example of a network model kms at british petroleum where the knowledge produced by the system was used to quickly diagnose and efficiently solve an equipment problem experienced by a mobile drilling ship on the north sea. finally, a purposeful valuation system is able to gauge the effects that knowledge production and use have on stakeholders. since the repository kms model does not produce any knowledge, the question once again is not applicable. kmss that follow the network model provide the potential for increasing awareness of possible effects by stimulating discussion and facilitating communication among stakeholders. whether or not this potential is realised depends on the specific kms and the particular context in which it is being used. in summary, the repository model of kms is more about tool design than social system design, whereas the network model provides opportunities for social system design. the repository model exhibits only extrinsic ajis special issue december 2002 56 purpose – it is developed to serve a purpose. the network model incorporates both extrinsic and intrinsic purpose – it is developed to serve a purpose and to enhance and facilitate critical reflection by the people within the system. there are opportunities for the individuals within the system to creatively develop ways of achieving the extrinsic purpose. the network model represents a more flexible, autonomous paradigm with more capabilities for adapting to complex, dynamic environments. purposeful corporations in order to assess the purposefulness of a corporation, it is important to realise that the corporate vision or mission statement represents the extrinsic purpose of the corporation. intrinsic purpose is manifested in the desires and actions of the employees, which cannot be directly controlled. however, purposeful corporations can be facilitated by defining and operationalising an appropriate corporate purpose. the corporate purpose defines why the corporation exists and, if it is value laden, it can attract and retain employees with similar values. this alignment of intrinsic and extrinsic purpose can lead to increased motivation and commitment of employees. defining the corporate purpose in broad terms and encouraging employee articulation of and action towards specific goals that support the corporate purpose can lead to increased flexibility and more autonomous employees. bartlett and ghoshal (1994) cite several examples of recently successful companies whose success they attribute in large part to the emphasis placed on developing and implementing an appropriate corporate purpose. using ulrich’s paradigm, these examples will be analysed to highlight how the development of purposeful corporations was facilitated. • at&t developed a corporate purpose to become the world’s best at bringing people together with the corresponding slogan ‘anytime, anywhere’. employees participated in strategy forums to develop ideas about how to achieve the stated purpose. this encouraged the production of meaningful knowlege with respect to the corporation’s purpose, satisfying ulrich’s definition of a purposeful inquiry system. • komatsu’s corporate purpose to become a total technology enterprise was embodied by the slogan ‘growth, global, groupwide’. management at all levels were encouraged to find and explore opportunities consistent with the corporate purpose – once again qualifying as a purposeful inquiry system. komatsu also instituted a committee to look at the broad social contributions of the company which would classify it as a purposeful valuation system according to ulrich. • at 3m, employees were encouraged to spend 15% of their time on bootleg projects – projects which represented original ideas of employees. this encouraged both the production and use of meaningful knowledge with respect to the corporation’s purpose and would be considered a purposeful inquiry and action system according to ulrich’s definition. • the body shop, with its commitment to environmental causes and corning with its focus on producing world-class quality products both espouse values that employees can identify with and are willing to commit to. alignment of intrinsic and extrinsic purpose is thus facilitated. whether these corporations qualify as purposeful according to ulrich’s definition depends on how these values are operationalised within the corporations. in the above examples, the adoption and interpretation of the corporate purpose by employees enabled the development of purposeful corporations, by increasing the alignment and contribution of intrinsic purpose. in some cases, these companies had moved from specific, value-free statements to the broad, value-laden purposes outlined above (bartlett and ghoshal, 1994). for example, komatsu’s previous strategy had been to catch up and surpass their major competitor – caterpillar – and at&t’s previous strategy had been phrased in very specific technical terms. a corporate purpose which states what the corporation does only leaves room for employees to decide how to achieve goals. a corporate purpose which defines why the corporation exists leaves room for employees to decide both what can be done to achieve the purpose and how to go about it. this leads to the idea that well-defined goals represent a specific instantiation of objectives that relate to an overall purpose. the crucial benefit that arises out of this realisation is flexibility. many different goals can help to achieve the same purpose. this difference of including both why and what becomes increasingly important as management tasks become more complex and less well defined/quantifiable and as more of the work force is represented by autonomous decision-making managers. there is a significant amount of research centred around locke’s theory of using goals to increase employee motivation and performance. a review of this research was conducted by latham and yukl (1975). one of their main criticisms is that “goals that are assigned to a person ... have an effect on behaviour only to the degree that they are consciously accepted by the person.” they proceed to point out that the “greatest deficiency of locke’s theory is the failure to specify the determinants of goal acceptance and goal commitment”. this commitment versus compliance to goals evolves from the added value of purpose. while goals relate to an external object or state to be realised, purpose represents an internal desire. goals must be aligned with intrinsic purpose in order to maximise motivation/performance improvements. fitzhardinge (1997), in an environmental context, highlights the difference between goals and values where he says values are related to purposeful activity. “goals are desired ends or states, some are self-sufficient and ajis special issue december 2002 57 others instrumental to gaining more desired ends. their main defining feature is that they can be uniquely satisfied. values, however, are more personal property, and a value system is said to predetermine behaviour in various areas.” once again the inflexibility of goals and the potential lack of internalisation are emphasised. angell (1990) states that “environments are continuously changing, and to be effective a system must be capable of a variety of responses to match those changes in the environment (ashby’s law of requisite variety)”. thinking in terms of purpose helps to enable this type of flexibility – specific goals and objectives can be adapted to changing environments and clear, consistent purpose can be used to drive their selection. conclusion this paper has argued that both extrinsic and intrinsic purpose are key concepts that should play a central role in the development and management of human activity systems. definitions for extrinsic and intrinsic purpose have been presented and a classification of system types in terms of purpose outlined. this classification was then used to review the systems thinking literature in terms of purpose. support for the importance of purpose in human activity systems was drawn from the systems thinking literature. ackoff (1999) emphasises “that a systemic transformation from an animated system to a social system (where both the whole and the parts are purposeful) is required if an enterprise is to be successful in the current environment which is characterized by an increasing rate of change, interdependence, complexity, production, and dependence on knowledge and information.” boulding argues that any problem that exhibits the characteristics of a certain level cannot be solved with a lower level system. the properties of purposefulness, self-consciousness and self-reflection are required in order to deal with the high level of complexity of today’s management problems. checkland (1994) states that “all real-world problem situations are characterized by the fact that they reveal human beings seeking or wishing to take purposeful action”. different approaches for studying human activity systems were briefly discussed. checkland describes two approaches – the ‘black-box’ approach which studies systems from the outside in and his second alternative with an emphasis on proceeding from the inside out. it was pointed out that the second alternative offers more opportunities for the study of intrinsic purpose. accordingly, the literature, which takes a ‘black-box’ approach to the study of human activity systems, is not as rich or comprehensive in terms of dealing with purpose as literature which takes the second approach. ulrich provides a practical approach for evaluating the purposefulness of human activity systems in line with checkland’s second approach. two different models of knowledge management systems were analysed to illustrate some of the implications for practice of incorporating both extrinsic and intrinsic purpose in information systems design. the analysis revealed that the model that incorporates both types of purpose has more potential for producing flexible systems that can creatively adapt to complex, dynamic environments. more detailed empirical analysis of information systems would help to validate the claims made in this paper about the benefits of purposeful systems design. further exploration of the contexts in which purposeful systems design is most appropriate is another potential direction for future research. several examples were highlighted to illustrate how the definition and implementation of an appropriate corporate purpose (extrinsic purpose) could lead to more opportunities for aligning intrinsic purpose with corporate purpose, thereby increasing employee motivation and commitment. it was argued that ‘thinking’ managers are better able to creatively formulate alternatives and make decisions when guided by purpose than when carrying out specific goals. this section of the paper touched on some of the potential benefits of focussing on purpose rather than goals. further development and substantiation of these ideas is a possible area for future research. one final potentially fruitful direction for future research will be mentioned here. the problem of alignment between corporate strategy and information systems strategy involves two types of human activity systems – corporations and information systems. as argued in this paper, both systems can benefit from management thinking with a focus on purpose. the concentration on purpose in both systems could then be used as a focal point to allow for more flexibility and creativity in strategy formulation and alignment. aligning purposes is a logical extension to focussing on purpose and it may provide a novel and insightful approach to research in the area of strategic information systems planning. ajis special issue december 2002 58 references ackoff, r.l., emery, f.e. (1972) on purposeful systems, london: tavistock and chicago: aldine-atherton ackoff, r.l. (1999) “transformational leadership”, strategy and leadership, 27(1), 20-25 alavi, maryam (2000) “chapter 2 systems for managing organizational knowledge,” in framing the domains of it management: projecting the future through the past, r.w. zmud and m.f. price (eds.), pinnaflex educational resources inc., cincinnati, 15-28 angell, i.a. (1990) “systems thinking about information systems and strategies”, journal of information technology, 5, 168-174 bartlett, c. a. & ghoshal, s. (1994) “changing the role of top management: beyond strategy to purpose”, harvard business review, 72(6), 79-88. boulding, k. (1956) “general systems theory – the skeleton of science”, management science, 2(3), 197208 briskin, a. (1999) “in pursuit of purpose”, journal for quality and participation, 22(1), 6-12. checkland, p.b. (1981) systems thinking, systems practice, john wiley and sons ltd. checkland, p.b. (1994) “systems theory and management thinking”, american behavioral scientist, 38(1), 75-89 dennett, d.c. (1997) kinds of minds – the origins of consciousness, london: phoenix elder, l. & paul, r. (2001) “critical thinking: thinking to some purpose”, journal of developmental education, 25(1), 40-41. fitzhardinge, g. (1997) “a sense of knowing, a sense of caring, a sense of place...”, ‘reading the land’ workshop proceedings, canberra jordan, n. (1968) themes in speculative psychology, london: tavistock krebs, w.a. (1997) collins compact australian dictionary, harper collins publishers latham, g.p., yukl, g.a. (1975) “a review of research on the application of goal setting in organizations”, academy of management journal, 18(4), 824-845 murray, j.a.h., bradley, h., craigie, w.a., onions, c.t., burchfield, r.w. (eds.) (1991) the oxford english dictionary, second edition, volume xii, oxford university press, oxford, 878 rosenblueth, a., wiener, n. (1950) “purposeful and non-purposeful behavior”, philosophy of science, 17, 318-326. stebbing, s. (1952) thinking to some purpose, london: penguin books strong, k.c. (2000) “a voice from the past: aristotle on the mission of the firm”, business and professional ethics journal, 19(2), 83-94 ulrich, w. (1983) critical heuristics of social planning, john wiley and sons ltd ajis vol. 4 no. 2 may 1997 living with the internet: ethics in the noosphere emma rooksby and diarmuid pigott murdoch university po box 776 nedlands perth west australia 6006 email emma@merriweb.corn.au abstract in this paper we present a holistic alternative to the conventional ground of existent ethical discourse on the internet. we analyse this discourse and the model of the 'net and netizens from which it is derived, and finding it unsatisfactory, present a discourse of co-operative ethics, based on a model of the internet as a noetic hyper entity. we then examine the tradition of such entities in western philosophy, and decide which of three traditions (the precxistent. the emergent and the convocational) is suited to our discussion finally we make suggestions as to how awareness of this entity can provide incentive to act co-operatively, and conclude that the existent form of netiquette is the best model for appropriate behaviour on the 'net. a lone boundary-rider at the ethical frontier it is tempting to portray interaction with the internet as a solitary indulgence, or even a solitary vice not intrinsically good, but at best useful. pessimistic visions of the internet as composed of millions of self-interested users in darkened rooms in front of glowing screens direct most ethical debate to restraining individuals from acting out their base natural inclinations. similarly, defence of the internet is usually framed in terms of the rights of the individual user to enjoy certain freedoms, and to be protected from the incautious exercise of freedom by other individuals. the pessimists' predictions focus on images of an increasingly selfish and alienated information-empowered elite who disenfranchise, or exploit, an information-hungry majority (see for instance, roszak, 1989; pawley, 1973; mcluhan and powers, 1989; rheingold, 1990; or even gibson, 1985,1993). exploitation is taken to be an inevitable outcome of conflicts among solitarily indulgent individuals. the pessimist's only distinction between the elite and the masses is in the distribution of power, in terms of the individual's ability to enact his or her selfish desires. if the internet can be retailed as allowing the individual to be a nietszchean superman in an infinitude of self-created virtual worlds without fear of consequence, then entirely new forms of voluntary alienation, and concomitant retreat from community life are envisaged. dystopian futurologist pawley sees the self-interested future in such bleak terms: "alone in a centrally heated air-conditioned capsule, drugged, fed with music and erotic imagery, the parts of his consciousness separated into components which reach everywhere and nowhere, the private citizen of the future will have become one with the end of effort and the triumph of sensation divorced from action." (pawley, 1973) the internet is seen by the pessimist as consisting of millions of such users, logged on for the sole purpose of gratification, and not contributing to it for any other reason. optimistically such users are exercising their right to self-expression or self-improvement. either way, when the moral agent is taken to be solitary and self-interested, then the individual's autonomy and gratification coalesce as the only stable criteria for ethical decision-making. for critics and defenders alike, individual desires and satisfactions are taken to be the yardstick of what is good and right. ethics based on individual-rights or perhaps utilitarian principles predominates on both sides of the debate about the future of the internet. the pessimistic solution is to guard against the emergence of moral and legal no-man'sland by legislating against new forms of vice the 'net is seen as disparate nodes wherein the storage or transmission of information forms the basis for legal and ethical evaluation. even when ethical theorists take a positive approach to the internet, they also take the nodal perspective. mitchell, for instance, inclines this way in city of bits: "the technological means to create private places in cyberspace are available but the right to create these places remains a fiercely contested area" (mitchell, 1995). implicit in the approaches of both the pessimist and the optimist is a conceptual map of the internet as consisting of self-determined nodes: the separate users and their workstations, connected via networking to hosts which facilitate the relay of information throughout a global network. this nodal map leads us to treat the internet as a passive network permitting efficient communication between individuals engaged in isolated and discrete transactions. 133 ajis vol.4 no. 2 may 1997 the nodal picture of the internet relies on a reductionist ontology, in which the only respectable explanations are those that reduce systems to their constituents elements. attached to this is a rather unpleasant portrait of human nature, a sort of autonomous enlightenment rationality gone sour: nodal images of human society remain as a hangover from enlightenment values of autonomy and self-determination which have governed accounts of the self and of social interaction since descartes. schopenhauer commented approvingly on descartes' treatment of the self as a basis for philosophy: "by [descartes'] taking the cogito, ergo sum as the only certainty, the essential and only proper starting-point of all philosophy was found, at the same time its true focus, which is essentially and inevitably the subjective, the individual consciousness" (schopenhauer, 1819). while for descartes, and for schopenhauer too, this image inspired hope in the power of the self to overcome difficulty, it painted social values as irrelevant, or even oppressive to the concerns of the self. at the hands of nietzsche, the whole of human existence was depicted as a struggle between noble self and a small-minded herdlike community. the evolutionary paradigm emerging in the life sciences in the nineteenth century reinforced this inaccurate picture of the essential nature of human relations, presenting accounts of society as embodying such antagonism (bronowski, 1973). even those opposed to the newtonian-cartesian paradigm (mcmaster, 1996) have tended to presuppose its validity see for example the work of rouse (rouse, 1989; also latour, 1986). it is this nexus of misperceptions which besets us in ethical speculation about the internet. a holistic focus on co-operation and community is long overdue. the internet treated as a dynamic whole there are three main problems resulting from the nodal image of the internet: it is incorrect ontologically, it is misleading in its expectations of the behaviour of the internet community, and finally it seduces people into inappropriate expectations. firstly, a problem of focus: it depicts the internet as built of fixedly separate nodes, thereby neglecting the co-operative and dynamic aspect of the 'net as a whole. this derives from a reductionist metaphysics which holds that knowledge of systems is a matter of understanding their constituent parts, not of holistic apprehension of them. we can get some way towards dissolving the first problem by reconceiving the internet as co-operative and dynamic. the co-operative element is already present in the networking protocol, including the hardware basis, of the entire internet. the edifice of the 'net stands because it has at its heart a cooperative and adaptive design paradigm. in the same way a cat's cradle cannot be completely described by analysing some string and ten fingers, concentrating on isolated nodes for the purpose of analysis of the 'net loses sight of the net as a co-operative whole. all networks have a common protocol for sharing data which is in some measure holistically consistent this is not a tautology, but rather a ideological description. the failsafe mechanism built into the original arpanet established a networking protocol that was not only co-operative, but adaptive, so that destruction of part of the network would not destroy the whole: "from its inception, the network was designed to be a decentralised, self-maintaining series of redundant links between computers and computer networks, capable of rapidly transmitting communications without direct human involvement or control, and with the automatic ability to re-route communications if one or more individual links were damaged or otherwise unavailable." (cda decision, 1996) as the arpanet evolved into the internet, the older protocols of mail and file transfer gave way to the more complex protocols that make up the web as we have it now. as we watch, these go to the increasingly rich and complex html exception set, and the interactivity of the net-based languages and operating systems. at the same time, there has from the outset been a willingness on the part of the designers of the 'net's protocols to be inclusive and co-operative (see also mitchell, 1995). at the functional limits of the more formal internet standards we see a proliferation of user-created consortia developing and implementing additional standards (conklin and newcomer, 1995). the mapping of the internet through domains and domain names is functionally co-operative, as are the higher level protocols ftp, uucp, http etc. this means the internet is an entity fractally co-operative, without any central control mechanism, but with an evolving set of consistent information transfer interfaces (conklin and newcomer, 1995). despite numerous varieties of os, connections can be established and maintained. none of these protocols is easily typified as an object like the terminals described in the nodal picture of the internet. rather, the interaction of several levels of protocol interfaces suggests a picture of the internet as an entity composed of cooperation-driven processes rather than of objects (vaskevitch, 1995). the mechanisms for change on the internet involve co-operation: examples of co-operation in establishment of new protocols are the self-adjudicating internet engineering task force, and the many ad hoc user consortia. the process for change can proceed with great vigour, through the innocuous sounding calls for discussion, calls for votes, and requests for comment. any ietf proposal at any point in time preserves its mantle of co134 ajis vol. 4 no. 2 may 1997 operation, maintaining the name request for comment even when it has been de facto accepted by millions of hosts around the world. co-operation is also strongly encouraged in the establishment of new newsgroups, where calls for discussion are used; all these mechanisms are non-centralised co-operative measures used in the development of internet forums. most developments are public and can be commented on or contributed to by anybody. by being co-operative and adaptive, the internet employs non-centralised mechanisms which allow it its own rate of evolution. part of those mechanisms is the co-operative nature of the human interactions: " no single entity academic, corporate, governmental, or non-profit administers the internet. it exists and functions as a result of the fact that hundreds of thousands of separate operators of computers and computer networks independently decided to use common data transfer protocols to exchange communications and information with other computers (which in turn exchange communications and information with still other computers)" (cda decision, 1996). both the co-operation inherent in the evolution of the internet and its current non-centralised dynamic structure point to the appropriateness of a model which incorporates this dynamic co-operative aspect of the 'net. the original nodal picture simply ignored this aspect. we have here indicated a new conception of the internet as an entity comprised of myriad non-centralised dynamic cooperative interactions. asynchronous manifold connections is also a processing paradigm in cognitive science. co-operation is implicit in neural and other systems; antagonism is only introduced at higher levels, and as such is still considered a pathogen. similar accounts of complex information systems are provided by connectionism and by complexity theories (see for example johnson-laird, 1988; lewin, 1993; kurzweil, 1990). in addition, there is support for the idea that the order apparent in a phenomenon, such as an ant colony (resnick, 1994), or the growth of a crystal structure (penrose, 1989), need not be the result of a single ordering "agent", but may result from the conformity of the whole system to codes of networked communication (mcmaster, 1996; changeux, 1985). cognitive scientist philip johnson-laird argues that the apparently symbolic or logical functioning of the conscious human mind may actually overlie a level of coordinated parallel processing of non-symbolic material. at the subconscious level mental processes may, he writes, "be the parallel processing of distributed representation created by the merging of many separate experiences." (johnson-laird, 1988) the overall order of such systems is maintained by regular networked communication, not by the autonomy of each individual cell or process. a more detailed account of connectionism or of markov systems, could be used to give further support to the picture of the internet as a self-regulating and co-operative system, though there is insufficient space to do so here. the fiction of the self-interested individual a second problem with the nodal picture of the internet is its treatment of individuals as self-determining independent agents rather than as primarily social beings. this problem is compounded by the assumption (also common to much irl ethical speculation) that human beings are almost entirely self-interested, and can only be induced to socially responsible or community-minded behaviour by coercion. examination of the nature of interaction on the 'net between the people who are its users shows it not to be the self-centred exercise the nodal picture paints. the essential process of the 'net is one of dialogue even stateless transactions such as web-pages partake of the basic communication form of the 'net, rapid asynchronous communication (mitchell, 1995). it could be objected that people are ultimately co-operating with machines, not each other, and that machine cooperation still leaves us with isolated, or even alienated users. but this ignores the fact that terminal interfaces through the internet are only the latest in a series of asynchronous communication devices, and that previous such devices letters and telephones have reduced isolation and alienation. moreover, some writers have even proposed a new evolution of consciousness into human/machine hybrids. such systems have the capacity to move away from a myopic focus on discrete individuals. for instance, donna haraway's "manifesto for cyborgs" links optimism toward technological innovations with a manifold increase in subjective possibilities. her "cyborg" is a potent image of the ongoing immersion of humanity in a compound multiple self whose essential nature lies in the mental realm (haraway, 1989). the nodal picture assumes a very stark view of man as wholly selfish and unwilling to co-operate except for selfish reasons. apart from any effect such pessimism might have on our behaviour, it is unlikely that human society could function without at least a modicum of co-operation. the co-operative nature of human society is focal to communitarian ethics (see etzioni, 1995; glendon, 1991). it is taken to be the basis of all ethics by ewin in co-operation and human values (ewin, 1981). he argues that co-operation is fundamental to human society not just in obviously co-operative activities like helping one another, or using a mutually agreed strategy to sort out a disagreement. even agreeing to abide by the conventional rules for determining a strategy, or using the linguistic conventions which allow us to communicate without great misunderstanding are co-operative 135 ajis vol.4 no. 2 may 1997 activities. "in each case, we aim at some good, the resolution of the dispute [or the sharing of opinions], and we gain it by following a procedure that requires us to accept mutual limitations" (ewin, 1981). co-operation is presupposed by human society tools such as language and reasoning are intrinsically dependent on the ability of human beings to work co-operatively. (ewin, 1981). so although each person benefits overall from social co-operation, his or her decisions in particular circumstances are not based merely on judgements as to which action will best serve that person in the future. that is, self-interest is a minor consideration besides the good brought to all by social co-operation, and is never the only factor in decision making. he notes: "cooperation is always inter-personal, a matter of several people doing something, and not simply a matter of individual action by a lone person" (ewin, 1981). clearly the interactions among the many co-operative concerns which constitute a society will involve more complex issues such as loyalty but preservation of the benefits of co-operative life necessitates self-moderation and control. this brings us to the issue of ethics in the co-operative society. virtues are, for ewin, simply those qualities which contribute to or sustain the co-operative nature of society; vices are simply those qualities which detract from or damage the co-operative enterprise. because co-operation is essential to the continuation of society, reasoning about virtues and vices is fundamental and takes priority over all other reasoning. so, for example, the smaller co-operative enterprise (say a software company) can be assessed only in relation to the greater good of society (or whatever is taken to be the greatest co-operative enterprise in existence). because the basis of society is co-operative rather than ruleor law-based, determinations of the best course of. action are loose, and depend on considerations of various virtues and vices which can never be formulated into strict rules (ewin, 1981). thus, for example, a willingness to cause pain to animals is seen as a vice, not because it may lead to ill treatment of people, but because "what is at work in both cases is a delight in causing pain" a vice, not an infringement of a law (ewin, 1981). so, by ewin's account of society, co-operation is seen not merely as conducive to individual gratification, but as constitutive of the social fabric. the holistic picture of social interaction which he presents, with dynamic multi-level co-operation as conceptually fundamental, and values (or virtues) rather than rules governing behaviour, is remarkably similar to the account of networks used above to characterise the internet. a holistic picture of the internet as a noetic hyper-entity a third problem to tackle is the net effect of treating individuals as selfish, assuming that how we think about the world helps make the world. the nodal picture informs users of the 'net as received wisdom, and an ethical fortress mentality is induced by the apparent lack of common goals uniting users. people reason: "if others can't be relied on to co-operate, why should i?" without a suitable holistic or dynamic concept of the internet, ethical speculators and legislators fall back on the nodal picture of individuals as ethically autistic and in need of strict regulation. for the optimist, the best course is to legislate to ensure the survival of the internet; for the pessimist, the clear path is to legislate to ensure the 'net's control. the internet can acceptably be treated as a proliferation of dynamic co-operative interactions which are self-regulating. but can this amorphous mass of co-operative and dynamic interactions provide a more satisfactory basis for ethics than a conception of interested selves conducting pecuniary transactions? ewin's account of human society as intrinsically co-operative may be more optimistic than traditional accounts of persons as selfish. but how can a dynamic and co-operative, yet inherently chaotic and unlocalised, internet provide a sufficiently coherent entity with which to guide ethics? the answer to this is that it must be conceived as an entity greater than any single human person. descriptions, from both experts and novices, of the complexity and vitality of the internet support the claim that it may best be grasped holistically, as a gestalt entity. (so, incidentally, does the willingness of many people to believe that a centralised controlling body "in charge" of the internet (cda decision, 1996)). one of the defining characteristics of any gestalt entity is that a grasp of its essential nature includes understanding its intentional structure, its telos. a simple example of a gestalt entity is a human being. although we know people to be composed of various atoms, cells and organs, we best understand them as social beings with beliefs and aspirations. we could never properly understand someone we did not know by, say, dissecting their corpse. what is significant about the internet as a gestalt entity is that human interactions are its constituent parts, along with the information we have put into it, and the protocols we have established for that information's conveyance. this means that understanding the internet is a matter of examining, and attempting to understand, something similar to human beings, but of a different order of magnitude. ludwig wittgenstein vividly describes this rather difficult concept as "the feeling of depending on an alien will" (wittgenstein, 1961). but even if we do treat the internet as a semi-autonomous agent, we do not treat it as a human being. it is, for one thing, primarily immaterial. so we have chosen the term noetic hyper-entity to describe entities of the order 136 ajis vol. 4 no. 2 may 1997 of the internet. referring to the internet as a hyper-entity describes its status as an agent of some sort. the term noetic, meaning mental, captures the immaterial nature of the agent with which we are dealing. western philosophy contains at least three traditions which employ the idea of a noetic hyper-entity. the oldest species is the pre-existent nhe, first documented in plato's timaeus: "in the centre [of the world] god put the soul, which he diffused throughout the body, making it also to be the exterior environment of it; and he made the universe a circle moving in a circle, one and solitary, yet by reason of its excellence able to converse with itself, and needing no other friendship or acquaintance. having these purposes in view he created the world a blessed god" (plato, 1892). plato's world soul is a higher order of being in the world, prior to human existence: not merely a social construct or an explanatory mechanism, but part of the world's composition. it exists in the noetic realm, and so can only be apprehended by pure intellect, the faculty appropriate to that realm. the other two lower faculties were spirit (which apprehended thought) and appetite (which apprehended the material world). however, we can have only a dim conception of the world soul because humans are not made up purely of intellect. ethics is a matter of attuning the human organism to the structure of the world soul, controlling and ordering the lower faculties of spirit and appetite by use of the intellect. the idea of a pre-existent nhe described by plato derived from the pythagoreans, and was adopted by the neoplatonists. a minor heretical doctrine in mediaeval theology, it re-emerged in late nineteenth century idealism, in fechner's panpsychism, bergson's vitalism, and the neohegelianism of royce. the pre-existent nhe is eternal and unchanging this was part of its appeal to the ancient greeks. a contrasting model is the emergent nhe, as formulated in hegel's phenomenology of spirit: "the living substance is...truly realised and actual solely in the process of positing itself, or in mediating with its own self its transitions from one state or position to another...true reality is merely this process of reinstating self-identity, of reflecting into its own self in and from its other, and is not an original and primal unity as such, not an immediate unity as such" (hegel, 1910). this notion finds resonance in the work of modern emergent pantheists (whitehead, hartshorne and lovelock) and eschatologists such as vernadsky and teilhard de chardin. it also forms the basis of much philosophical speculation in complexity theory, though rarely as an explicit part of the theory itself (waldrop, 1992; lewin, 1992). emergent nhes are biological or ecological formations, seen as still in the process of evolution. as in the case of the pre-existent nhe, human desires and actions cannot influence the form the emergent nhe lakes. thus the cellular sub-entities of the emergent nhe (that is, individual minds) cannot prefigure its final form, nor influence its development in any way. for example, for lovelock, consciousness may well be parasitic, while for teilhard consciousness is a transient organic phase. because the nature of the emergent nhe cannot be known in advance, the ethical systems associated with it are primarily ad hoc pragmatic schemes. the third species to be considered is the convocational nhe, formed through human association, and not existing prior to that association. hobbes' nation-state, the sovereign, is the archetypal convocational nhe, arising from the mutual political interests of the citizens of a state, and it can be seen in ewin's "greatest cooperative enterprise" and the "common ground" of the communitarian ethical schemata. st. paul's vision of the christian church as the bride of christ is also a convocational nhe, arising from fellowship and a belief in the common good. a convocational nhe can be conceived of as existing, but unacknowledged: the idea of an unnoticed convocational soul of the state first occurs in plato's republic, and also occurs in aristotle's politics, rousseau's volume ge'ne'ral, and marx's proletariat consciousness. nor is the convocational nhe a solely political entity. a human culture, including its arts, could, for instance, be treated as a convocational nhe, and indeed has been in fludd's artes liberaliores (godwin, 1979) and fraser's nootemporal umwelt (eraser, 1975). there is no pre-existent order to which a convocational nhe will conform, means that it can be formed to some extent by all those who participate in its being, and leaves humans partly responsible for the qualities of the nhe which eventuate from their activities. the actions of the nhe can only be traced indirectly to the actions and interactions constituting it. it should be emphasised that the convocational nhe is usually described anthropomorphically. this makes it easier to treat holistically as a psychologically consistent entity. ethics which employ a convocational nhe tend to be contractual and to focus on social co-operation. the existence of an nhe is a continuous thread of the western intellectual tradition, and is frequently used in ethical and political theory. all the above traditions posit a nhe as a metaphysical means of guiding human ethics and politics, though each has its own way of so doing. we need to decide which one to use as a basis for ethical conduct on the internet. the pre-existent nhe relies on a gnostic metaphysical doctrine of creation, the emergent nhe on the acceptance of a post-gnostic cosmology. this immediately makes them less appealing to people who do not accept such doctrines. more importantly, in order for an nhe to be of any use in guiding our conduct, a clear conception of that nhe is needed. and for both the pre-existent and the emergent nhes, this conception has only ever been undertaken by ad hoc appeal to an anthropomorphic characterisation of the nhe. 137 ajis vol.4 no. 2 may 1997 for example, the pre-existent world soul has been described by considering it as the semi-human demiurge and teilhard's emergent noosphere is predicted to emerge as a point omega consciousness. the anthropomorphic nhe is, as we have seen, primarily associated with the convocational species of nhe. in effect, then, the preexistent and emergent nhes best function in ethics when treated as convocational nhes. an additional drawback to both the pre-existent and the emergent nhes is that they are taken to be entirely determined by forces other than human intentions. these forces are either purely theistic (as in plato's timaeus) or purely organic (like teilhard's noosphere). they leave little or no room for human input into the form taken by the nhe. thus neither gives us freedom to negotiate an ethics with the nhe in question. for instance, what there is of ethics in plato's timaeus suggests that the human soul's goal is the attainment of the same condition as the world soul. a pre-existent nhe is a fixed entity, upon which the actions of humans have no influence. we cannot negotiate with a pre-existent or an emergent nhe. using either as a model for the internet would give us an ethics with very little theoretical flexibility. however, the convocational nhe is by definition composed of deliberate human association. this model leaves at least a degree of freedom for humans to negotiate with and to determine the precise form of the nhe. this can be accomplished by treating the internet as a semi-autonomous entity, rather than ignoring its presence and activity in favour of consideration of individual human will. in conclusion of this section, the internet is best conceived of as a convocational nhe, a semi-autonomous entity produced by the intentional actions of a large community of people, but whose precise nature is only indirectly related to the interactions of individual members of that community. the nhes ethical function is as a guide or reminder of our duty towards its other members. the covenant the context of internet ethics is that of an evolving convocational contract with a noetic-hyper-entity, involving both value-based ethical norms and a set of explicit rules delineating injustice, the latter lagging some way behind normative values, (i.e. standards for virtuous behaviour). for instance, going on-line initially involves agreement to a set of service-provider conditions. as regular usage is established, the important behavioural rules and norms come into play in increasingly recognisable forms. convocational behaviour is exhibited, for example, in the willingness of old hands to help out and guide those new to the 'net, as well as to admonish them when ethical norms are contravened. similar normative constraints regulate the behaviour of up-stream service providers. all behaviour which falls inside the very broad category not-unjust is at least acceptable, if not the best course of action (ewin, 1981). in those cases where an action is considered unjust, it is with regard not only to those directly affected but to the convocational nhe itself, and by implication to all members of the nhe, including the instigator of the unjust action. vice is unco-operative behaviour, and, as such, is damaging to the nhe as a whole. so, by our very use of the internet we find ourselves in a social contract with the nhe and, within the context of the nhe, with our peers. tacit consent is established in the act of benefiting from membership in the community of users. and a sense of being part of a larger entity a convocational nhe provides overwhelming encouragement for co-operation and responsible behaviour. the co-operative origin of the internet in scientific and academic circles has given it a firmly community-minded co-operative basis. the essence of a contract with the nhe is that each of us makes certain sacrifices in exchange for the benefits provided by the nhe. in the case of the internet, we forgo unlimited expression and unlimited bandwidth usage in exchange for such expression and bandwidth as common usage allows. by treating this agreement as between the self and the nhe, we find ourselves responsible to all elements of the nhe, even those whom we do not know. responsibility to a larger community than our immediate acquaintances is, as already noted, a characteristic of convocational nhes. on the pauline model we act responsibly towards a larger whole through cooperating and willing the good of the enterprise. on the hobbesian model this is achieved through voluntary restraint and respecting the importance of the nation (the largest co-operative community to which one belongs). for the internet, the community is potentially a worldwide one. in conclusion, by treating conformity to the co-operative nature of a convocational nhe as the norm, we have a strong intersocial ethical basis for determining which new behaviours are virtuous and which vicious, and for responding accordingly. anything which preserves the sense of the community of the 'net, without great detriment to other co-operative entities, is to be promoted. an on-line virtue-based ethics is already maintained by peer censure and voluntary restraint and is best maintained thereby. netiquette and rfcs are appropriate and hitherto effective means of maintaining a regulated environment for 'net interaction. believing in the force of the nhe to maintain and encourage co-operation with accepted 'net practises can only provide us with greater incentive to co-operate. self-regulation, and governmental reticence are important to keeping the internet fertile, preventing the sterility which comes with legal wrangles and administrative conformism. some regulations are no doubt required to prevent abuse, but are unnecessary (and probably ineffective) as a means of pre-emptive 138 ajis vol.4 no. 2 may 1997 control. the nhe has come into existence as a co-operative dynamic entity, and it, not individual persons or corporations, should be given priority in ethical decision-making. references aristotle (1972), de generatione animalium (trans. balme, d.m.), clarendon press. aristotle (1959), politics (trans. warrington, j.), j.m.dent & sons. bergson, h. (1914), creative evolution (trans. mitchell, a.), macmillan & co. bronowski, j. (1973), the ascent of man, bbc. campbell, j. (1982), grammatical man, simon & schuster. changeux, (1985), neuronal man: the biology of mind, (trans. carey, l.), oxford university press. conklin, p. and newcomer, e. (1995), "the keys to the highway" in the future of software (ed. leebaert, d.), mit press, 77-104. cunard, j.(1995), "property of the mind" in the future of software (ed. leebaert, d.), mit press, 227260. etzioni, a. (1993), the spirit of community: the reinvention of american society, touchstone. ewin, r. (1981), co-operation and human values, st. martin's press. fechner, g. (1966), elements of psychophysics, holt, rinehart and winston. foucault, m. (1980), knowledge and power: selected interviews and other writings, harvester 'press. fraser, j. (1975), of time, passion and knowledge, george braziller. gibson, w. (1985), neuromancer, grafton. gibson, w. (1993), virtual light, viking. glendon, m. (1991), rights talk: the impoverishment of political discourse, free press. godwin, j. (1979), robert fludd: hermetic philosopher and surveyor of two worlds, thames & hudson. haraway, d. (1989), simians, cyborgs and women, routledge. hartshorne, c. (1984), omnipotence and other theological mistakes, suny press. hegel, g. (1910), the phenomenology of mind (trans. baillie, j.), swan sonnonschein. hobbes, t. (1914), leviathan, dent. hobbes, t. (1962), body, man and citizen, collier books. johnson-laird. p.n. (1988), the computer and the mind: an introduction to computer science, fontana. kline, g. (ed.), (1963), alfred north whitehead: essays on his philosophy, prentice-hall, inc. kurzweil, r. (1990), the age of intelligent machines, mit press. latour, b. (1986), laboratory life: the construction of scientific facts, princeton. lewin, r. (1992), complexity: life at the edge of chaos, macmillan. lovelock, w. (1988), the ages of gaia: a biography of our living earth, norton. mitchell, w. (1995), city of bits; space, place and the infobahn mit press. mcluhan, m., and powers, b. (1989), the global village: transformations in world life and media in the 21st century, oxford university press. mcmaster, m. (1996), the intelligence advantage, butterworth-heinemann. pawley, m. (1973), the private future: causes and consequences of community collapse in the west, thames & hudson. penrose, r. (1989), the emperor's new mind: concerning computers, minds, and the laws of physics, vintage. plato (1892) dialogues (trans. jowett, b.), random house. plotinus (1929), selected works (trans. mead, g.), g bell & sons ltd. resnick, m. (1993), turtles, termites and traffic jams: explorations in massively parallel microworlds, mit press. rheingold, h. (1991), virtual reality, martin, seeker and warburg. roszak, t. (1986), the cult of information: the folklore of computers and the true art of thinking, lutterworth. rouse, j. (1987), knowledge and power: towards a political philosophy of science, cornell university press. rousseau, j. (1913), the social contract, dent. royce, t. (1908), the world and the individual, macmillan. 139 ajis vol.4 no. 2 may 1997 schopenhauer, a. (1819), the world as will and idea (trans. berman, j.), dent. sheldrake, r. (1983), a new science of life: towards a hypothesis of formative causation, fontana. teilhard de chardin, p. (1959), the phenomenon of man, harper and brothers. vaskevitch, d. (1995) "is any of this relevant?" in the future of software (ed. leebaert, d.) mit press, 45-76. wallis, r.t. (1972), neo-platonism: classical life and letters, charles scribner's sons. waldrop, m. (1992), complexity, simon & schuster. wildiers, n. (1968), an introduction to teilhard de chardin, fontana. wills, c. (1995), the runaway brain: the evolution of human uniqueness, flamingo. wittgenstein, l. (1961), notebooks 1914-1916 (trans. anscombe, g.) basil blackwell. acknowledgements we would like to thank callan ledsham, sheila renfrey, peter bryant, pasha parpia, jo simpson, brent parkin, james coleman, patrick pigott and paul mackin-brown for their helpful advice and criticisms of earlier drafts of this paper. many of their comments have been incorporated into the final piece. 140 microsoft word vol 15 2 final.doc australasian journal of information systems volume 15 number 2, 2009 45 an ontology for mobile situation aware systems paul o’brien university of queensland business school email: p.obrien@business.uq.edu.au abstract this paper introduces a novel design artefact, namely a generic situation management ontology based on situation theory. this ontology contributes to the foundation knowledge base of mobile service delivery systems for future research and systems design. it demonstrates the applicability, and feasibility of using situation theory in the design of reactive information systems. the support within the ontology for context based filtering for situation detection also contributes to the efficiency of implementation and operation of situation driven reactive information systems. highly mobile people (hmps) require flexible, reactive service delivery due to their regularly changing location and activities and the lack of a wired network connection. a mobile service delivery system should be able to detect relevant events that occur such as change of location, availability of new last-minute specials, sales opportunities and safety issues and then reactively take action in response to these events. this paper describes a generic situation management ontology that was developed in owl using the ontology development tool, protégé. the ontology is combined with domain specific classes in the travel domain to create a travel situation management ontology that can be used as the basis for a ubiquitous mobile travel service application. using a typical independent traveler scenario, the travel situation management ontology is instantiated to demonstrate its effectiveness. keywords situations, situation awareness, situation management, context, mobile information systems introduction today, personal computers are available that are powerful, portable, use little energy, do not need special environments and are cheap. the original computers were good for large scale information processing. they were not suited to the tasks that highly mobile people (hmps) typically carry out as they go about their work or play. “they have not been aware of our needs or even of whether we were in the room with them” (mit, 2004). the next generation of computing devices will be faster, cheaper, smaller, more powerful, and more portable than their predecessors. wireless data technologies are maturing and improving in bandwidth, performance and reliability (wimax, australasian journal of information systems volume 15 number 2, 2009 46 2006) so the next generation of portable computing devices will be networked via high speed, ubiquitous wireless technologies. problems with current automated service delivery techniques current automated service delivery systems tend to be location-aware, context-aware within a particular room or single location, or temporally aware at a single location or within a single system. their designs also tend to be restricted by consideration of narrow mobile bandwidth and limited handset capability and are generally designed for a specific task or a specific domain. context dimensions of time alone or location alone or even a combination of time and location do not allow independent decisions to be made about situations that occur. location, type of location, time, type of time, local bindings and user preferences are required as a minimum for effective decisionmaking for hmps. domain specific and application specific dimensions are not generalisable to other domains or applications. application and domain knowledge is essential for any working system, however, this information can be provided by creating a domain and application specific instance of a generic situation ontology that includes, of necessity, ontology elements that relate to context. by designing the architecture and system logic to be driven by a situation ontology, new domains and applications can be accommodated by creating a new instance of the situation ontology, thus eliminating the need to hard-code what is essentially object data into the application itself. to automate the delivery of services the system must also be able to identify situations that require action (reactive situations) from the set of existing situations, and apply the appropriate action rules. the focus of this research is therefore the design of a generic mobile situation management ontology that can be used as the basis for an architecture for a ubiquitous reactive service delivery system for hmps. research questions the research problem that this work addresses is how to improve the delivery of services to hmps continuously while they are travelling. research questions arising from this problem include: 1. how can a highly mobile person’s need for a mobile service (a reactive or proactive situation) be detected? 2. what mechanisms can be used to reduce the complexity of the detection and response to reactive situations? 3. how can the concepts related to situations be represented in an information system? 4. is it possible to design a generic mobile situation management ontology that is domain independent? previous work by the author investigated the first two of these questions (o'brien & burmeister 2003; o'brien & colomb 2005). this paper focuses on the last two questions; the design of a generic ontology for mobile situation management that provides a base for the future development of a prototype of a ubiquitous mobile service delivery system. australasian journal of information systems volume 15 number 2, 2009 47 philosophy of approach the key to the provision of an effective, high added value mobile service is the ability to detect relevant events and to independently take the action that is most appropriate for the resulting situation. this situation triggered action approach relies on active context filtering to minimise the processing of situations (o’brien, 2006). a review of previous work suggests that an automated ubiquitous mobile service delivery systems requires: • an agreed ontology of situations that allows the specification of situations, contexts and situation action rules • an agreed domain ontology for the application domain in which the system will be operating. the mobile situation management ontology described in this paper forms the basis for the implementation of a system that addresses these requirements. methodology the goal of this work is to find a satisfactory solution, not necessarily the best or optimal solution from amongst all the possible solutions. generalisability of the artefacts produced by this work is not a key consideration as they will be the starting point for a line of future research in situation aware mobile service delivery systems and will be refined in future research. for these reasons, a design research approach, as proposed by hevner et al (2004) has been used. design research aims to create innovative artefacts that extend human or organisational understanding or capabilities (hevner et al., 2004). considering this approach, the following tasks were undertaken: 1. to develop an awareness of the problem, interviews were undertaken with a small number of hmps (o’brien & burmeister, 2003); 2. one service was selected from each of angehrn’s four virtual market spaces. the selected services are representative of the types of services in each space (o’brien & colomb, 2005). this approach is recommended by hevner et al (2004); 3. design of a generic situation ontology that can be used in the development of mobile service delivery systems.; 4. evaluation of the generic situation management ontology by, • instantiating a travel domain ontology using a typical hmp scenario to ensure that the situation ontology has adequate expressiveness for real world situations. • developing an academic situation management ontology to demonstrate the adaptability of the generic mobile situation management ontology. where necessary minor design changes were made. australasian journal of information systems volume 15 number 2, 2009 48 mobile situation management context & context awareness decisions and assertions that are made regarding hmps must consider their current state and context. for example, information about current location and activity only allows decisions and assertions to be made about this location at this time. it does not allow decisions to be made about other times and locations. lenat (1998) provides a detailed definition of the twelve dimensions of context that he considers to be essential for automated reasoning. include are the dimensions of time and space, but also a number of other dimensions such as granularity and culture. there has been considerable debate in the literature about the value of these additional dimensions but there is general agreement that time and location are essential. assertions about something are generally only applicable in specific contexts. therefore, an answer to a question about something must be a pair <context, answer>. that is, “the answer is …… in this context”. for example, two possible answers to the question “who is the prime minister” would be <1972-75, gough whitlam> and <1996 – 2007, john howard> (lenat, 1998) lenat’s (1998) experience with contexts or micro-theories has shown that it results in simpler assertions and extensive re-use of assertions but also imposes the additional burden of choosing the most appropriate context from a large number of small contexts. consequently, a trade-off needs to be made between coding time within the contexts and time to select the best context. mobile data technologies allow data to be captured that simplifies the selection of the most appropriate context and hence, simplifies the automated decision-making process. dey et al (2001) define a context aware system as a system that “uses context to provide relevant information and/or services to the user, where relevancy depends on the user’s task”. although a person may be in a particular physical environment, for example, his bedroom, the activity that he is engaged in within his bedroom environment determines what in the bedroom is important to him. if he is relaxing and reading then the light being on is important, however, if he is sleeping it is important for the light to be off and, perhaps for the telephone to be turned off. knowledge of the user’s current activity and preferences are therefore essential for context-aware automated decision-making. despite time being identified as a primary dimension of context, dey et al (2001) did not provide for temporal dimensions of context in their framework and toolkit for context-aware applications. as awareness of both temporal and location dimensions of context are essential for mobile service delivery systems, both have been included in the situation management ontology developed in this paper. gross and specht (2001) show that delivery of services to highly mobile, or nomadic, users can be enhanced by using context awareness, user modelling and by localisation of service delivery. they show that context aware systems enable the delivery of services to mobile users that are filtered or adapted depending on the user and service contexts. however, the types of services that highly mobile users require are not heterogeneous. angehrn (1997) proposed that these services can be categorised as information, communication, transactional and distribution services. kanter (2000) and schmidt (1999) further showed that the adaptation and/or filtering that is required differs according to the category of the service being delivered. by categorising services into angehrn’s four categories, a hierarchy of service types can be defined within each of the four dimensions australasian journal of information systems volume 15 number 2, 2009 49 thereby facilitating more functionally cohesive service definitions and consequently more functionally cohesive software modules and/or agents to deliver the services. gross & specht (2001) further assert that action should be taken and/or information should be provided ‘if an event occurs in the same context as a user’s current context’. while this does provide a level of filtering that eliminates actions for events that are “out of context”, it may still result in irrelevant information or unnecessary actions. a further level of decision-making is required. information and/or actions should only be taken if the contexts are the same and a particular state of affairs exists within that context(s). that is, a reactive situation exists. situations loke (2006) provides a very good summary of previous work on context aware systems and situations. he clearly distinguishes context and activities from situations, which he asserts is a similar but higher level concept. he argues that “situations can be useful….in that the situation abstraction allows the modeller to effectively carve the world up into manageable pieces”. these situations can be combined to compose more complex situations. there are a number of different definitions of the concept of a situation in the literature. while they are similar, they differ according to the viewpoint of the researcher. as my work is from an information systems viewpoint, i have focussed on the information systems definition of situations. barwise (1987), akman and surav (1996), adi et al (2002) and fischer et al (2002) provide definitions that initially appear to be quite different, however they all take context into account, either explicitly or implicitly, and they all consider the state of things in the context. from a theoretical perspective, situations do not necessarily require any action. situations that do require some action to be taken, that is, reactive situations, require action rules to be defined. in this work i have therefore adopted the following, slightly modified version of barwise’s (1981) definition: “a situation is a state of affairs in a particular context” but i draw on etzion & adi’s work to define reactive situation. rules need to be defined to specify the appropriate actions that should be taken in response to occurrences of each reactive situation (adi et al., 2002). where the same response is required for a number of reactive situations, these situations form a class or type of situation (cherry, 2001) for which reaction rules must be defined. barwise (1987) also shows that situations are first class citizens (or objects), that can have properties and relationships with other objects including other situations. the relationships can be expressed by constraints or background conditions. barwise and perry (1983) found that states of affairs required spatio-temporal parameters to address seemingly incoherent situations such as paul is talking and paul is not talking which can both be true in different contexts that are not always persistent. this supports lenat’s (1998) position that the truth of any state of affairs can only be determined if we know its context. situation theory situation theory was introduced by barwise (1981). barwise’s ideas were then applied to logic by barwise and perry (1983). perry (1993) then built on his early work with barwise. he clearly distinguishes situations from worlds. “a world determines the answer to every issue, the truth-value of every proposition. a situation corresponds to the limited parts of reality we in fact percieve, reason about, and live in.” (perry, 1993) consequently, situations only allow limited decisionmaking about events and states of affairs within particular contexts and domains. australasian journal of information systems volume 15 number 2, 2009 50 situation semantics is normally applied to the theory of natural languages, but it can be useful to define real world context dependent situations. situation semantics is based on the concept that we classify parts or views of reality by considering the things, their properties and the relationships between them in that part of reality. the events that occur in a situation alter the values of the properties of things in that part of reality, that is, they change the state of affairs, in that context. a state of affairs may exist within a context because of events outside that context, but determination of whether a situation exists depends only on the state of affairs of the things within that context. in situation theory, a key concept is that of a type of situation. the type of situation in which your car has a flat tyre (s0) and the type of situation in which a car can’t move (s1) are examples. one type of situation may be related to or depend on another. for example, every time one situation occurs, another situation will also occur. barwise and perry (1983) say that s0 involves s1; cars with flat tyres can’t move. they call these situations constraints, that is, a car that has a flat tyre is constrained from moving. devlin’s (1991) book built on the work of barwise and perry and introduced a modern notation and terminology for describing situations such as these. in situation theory, infons are basic discrete pieces of information. using devlin’s (1991) notation, an infon is represented as: «p, a1, …,an, i » where p is an n-place relation, a1, …, an are objects of the relation p and i is the polarity (0 or 1) indicating whether the relation holds. situations are related to infons by the supports operator ( ╞ ). s ╞ α indicates that the infon α is supported in situation s, or in other words, s makes it the case that α is true. for example, s ╞ «bites, fido, mary, l, t, 1 » represents the situation where fido bites mary at some location (l) and time (t). devlin’s notation and terminology are used to represent situations in this paper with the exception that the words true or false are used to represent polarity rather than the numbers 1 and 0. this has been done simply for clarity and to avoid confusion related to the representation of boolean values in different programming languages. devlin (1991) also specified a number of basic types that are necessary for situation specification. they include tim to specify time, loc to specify location, ind to specify parameters, rel to specify relationships, sit to specify situations and inf to specify infons. for example, ss = [ s’ | s’ ╞ << slaps, a’, b’, l’, t’, 1 >>] st = [ s’ | s’ ╞ << touches, a’, b’, l’, t’, 1 >>] means that if a’ slaps b’ at l’ (a loc type) & t’ (a tim type) then a’ also touches b’ in the same l’ & t’. factual infons and constraints are also required because some facts are only true under certain background conditions. a constraint c may only allow situation s2 to be true when s1 is true if the background condition b is true, that is c = s1 → s2 | b can also be written as an infon: <<involves, s1, s2, b, 1 >> or <<involves, s1, s2, 1 >> | b akman and surav (1996) show that contexts provide a useful way to apply context specific “rules and pre-suppositions related to a particular point of view” to simplify the specification and detection of situations. they show that when binding parameters in situations, “only objects that are in one of the current contexts need to be considered.” furthermore, they show that contexts can be considered as objects, that is, as “first class citizens”. this means that they can be used in the same way as any other object so ‘they can be denoted by constants in the logical language” and “variables can range over them (akman and surav, 1996)”. combining the work of devlin and akman & australasian journal of information systems volume 15 number 2, 2009 51 surav, i have combined tim and loc, together with other dimensions where necessary, into a single, complex type, context (con). this simplifies the specification, filtering and comparison of situations. situation management reactive systems are systems that react automatically to changes in the environment. they have been used in command and control, active databases, system management tools, customer relationship management systems and electronic commerce systems. this paper extends previous work that tended to provide mechanisms for reacting to single events only to one which reacts to situations. to be able to detect situations some key information must be included in the situation definition or in related objects: • the events that can participate in situation detection; • context during which situation detection is relevant; • the semantic conditions that must be satisfied in order to detect a situation; • whether and under what conditions an event is consumed (adi et al., 2002) adi and etzion’s event class does not include any location attributes. for mobile users it must be extended to include at least location attributes and most likely, other attributes of context. this is particularly important when inferring events from the world’s state and concrete events such as user movement. adi and etzion’s lifespan class defines a set of events that can initiate a lifespan, a set of events that can terminate a lifespan, conditions for initiation and termination and maximum lifespan length. to take account of location i have added a locationspan class with similar attributes to lifespan but related to location, namely, • a set of events that can initiate a locationspan; • a set of events that can terminate a locationspan; • the conditions for initiation and termination of a locationspan; • maximum locationspan radius; • granularity of location. i have also added a granularity of time attribute to the lifespan class. adi and etzion group collections of semantically associated event instances into event groups. they are used to match different event instances that refer to the same entity or concept. an event collection is the collection of event instances that must be considered for situation detection. they are only evaluated if they occur during the time that a context that is associated with situation detection is active. i have used event collections to group together sets of events that result in a context and/or situation transition. a single event collection may include temporal events, locationbased events and other service and/or user events each of which causes the same transition. although my work is based on adi & etzion’s concepts, there are many differences, namely, • context is used to filter out irrelevant situations rather than just lifespan; • an object reference attribute is used to relate a situation to event collection objects rather than simply an event collection name; australasian journal of information systems volume 15 number 2, 2009 52 • background conditions associated with a situation are specified using references to situation objects rather than using a simple situation expression. • adi & etzion’s situation manager uses lifespan to filter out irrelevant situations. if there is no open lifespan for a particular situation, the situation is ignored. this is clearly insufficient because a situation may still be irrelevant, even if there is an open lifespan, if there has been a change in some other dimension of context such as location, or even a change in user preferences. therefore, for hmps, situations need to be filtered by context, not just lifespan. cherry (2001) proposed the use of pre-specified rules, situation-action rules, to specify the actions that are required to be taken in response to the detection of a reactive situation.. he shows that rules can be defined for classes, or types, of situations rather than individual situations, thereby reducing the number of rules required. i have taken this approach, using semantic web rule language (swrl) to specify the rules (swrl, 2004). ontologies in artificial intelligence (ai), an ontology is “the specification of a conceptualisation. that is, defined terms and the relationships between them, usually in some formal and preferably machine readable format” (hendler, 2001). guarino (1998) points out that the ai concept of an ontology generally refers to “an engineering artefact, constituted by a specific vocabulary used to describe a certain reality, plus a set of explicit assumptions regarding the intended meaning of the vocabulary words.” these formal ontologies provide a means of representing, or at least approximating, a conceptualisation of the world. an ontology of situations matheus et al (2003) proposed a core ontology for situation awareness that defines a situation as a collection of goals, situation objects and relations between situation objects. situation objects are “entities in a situation – both physical and abstract – that can have characteristics (i.e. attributes) and can participate in relationships” (matheus et al., 2003). this ontology was designed to be the core of a situation awareness system that assumes that the recognition of the existence of a particular situation will occur in the mind of a human operator who interfaces to the system through a human-computer interface. the inclusion of goals as attributes of a situation is inconsistent with the theoretical definition of a situation as “a state of affairs within a given context” (barwise, 1987). a situation is independent of, but of course related to, the goals and action rules that determine the actions that should be taken when that situation occurs. however, from a practical perspective, if inclusion of goals as attributes reduces implementation time and/or efficiency then there is no reason to not include them. devlin (1991) defined a basic ontology of situation theory, based on barwise and perry’s work (barwise,1987; barwise and perry, 1983) and documented it using a new notation that he developed. devlin’s situation theory ontology was later used to develop a number of new situationoriented languages (tin et al., 1995) including baby-sit (kee, 1993; tin and akman, 1994). barwise and perry’s situation theory and devlin’s notation are now well accepted within the situation theory research community. the adaptation of this ontology for use on the worldwide web provides a solid theoretical basis for the development of a situation management ontology for use in semantic web applications. owl has become the language of choice for most of the recent work that involves the development of ontologies for the worldwide web (fukazawa et al., 2006; australasian journal of information systems volume 15 number 2, 2009 53 jacobson et al.,2005; matheus 2005). since it was published, owl has also been successfully used by matheus (2005) to develop his core ontology for situation awarenesss. for these reasons, it was used to implement the core situation management ontology in this work. relationship between events, context and situations in this work, i have combined barwise’s tim and loc parameters, and a number of other dimensions of context into a complex parameter, context (or con). for example the following situation type, specified using devlin’s notation, s = [ŝ | ŝ ╞ «cancelled, flight, ĺ, ť, true»] could be rewritten as, s = [ŝ | ŝ ╞ «cancelled, flight, c’, true»] where s = situation type, ŝ = situation, ╞ = supports, c’ = context (composed of ĺ = location and ť = time) and « …. » = an infon. the relationship between events, states, contexts, situations and actions is summarised below: a e e δs δs δc or δs δs δc δs δc sr | b δs sr | b sr a1, a2….an where e = event, δc = change in context, δs = change in state, s = situation, sr = reactive situation, δs = change in situation, b = background situation (a situation as described in section 0 above), a = action, = “causes”, = “may cause” and | = “subject to”. service or hmp actions cause events to occur. events cause changes in the state of a hmp or a service. the change in state may cause a change in context, which in turn may cause a change in situation to occur. the change in state may also directly cause a change in situation without a change in context. a change in context may cause a change in situation. a change in context may cause a reactive situation to occur subject to a background situation existing. similarly a change in situation may cause a reactive situation to occur subject to a background situation existing. in response to the occurrence of a reactive situation one or more actions are taken. an event that can cause a transition to a new context is included in an event collection for that transition. background conditions cause normal and/or reactive situations to occur. situation management example a simple example of a typical mobile user scenario and how the situations that occur would be handled by the system are given below. the date is 26 march 2007. fred is traveling around the islands of the south pacific on business. he is currently in brisbane and is scheduled to travel to vanuatu tomorrow and then on to the solomon islands on 28 march 2007. fred is unaware that a serious conflict has broken out in solomon islands. he is not contactable because he is in a high level meeting. the us state department issues a travel warning for the solomon islands. the australasian journal of information systems volume 15 number 2, 2009 54 system detects the warning, selects relevant contexts and checks to see if there are any hmps registered in these contexts and sends the required alert through to them. the system then checks fred’s itinerary for planned solomon islands trips in the near future. the system identifies that a reactive travel safety situation exists for fred’s planned solomon islands trip and takes appropriate actions to cancel this leg of his trip. the next time fred switches on his handheld device, the alert is displayed advising him of the situation and the changes to his itinerary. the scenario above assumes that fred has set his preferences to allow the system to automatically change his itinerary without consulting him. each hmp will have different preferences regarding this and other automatic responses. hmp’s must be able to set their own preferences regarding what actions the system is allowed to take before consulting them including level of automation of itinerary changes, granularity of location and time for alerts, type of alerting preferred etcetera. the generic situation ontology described in this paper includes a class service_preference that stores the hmps preferences for a particular type of service, such as the transport service that would manage the situation described in the scenario above. the mobile situation management ontology must include information regarding these preferences so that system designers can use them to determine what, if any, independent action should be taken. the class (or type) of reactive situation, itinerarychangerequired (sr1) can be specified by the following situation type definitions: s1 = [ŝ | ŝ ╞ « travelsafetywarning, c’, true»] s2 = [ŝ | ŝ ╞ «hasflightreservation, p’, c’, true »] sr1 = [ŝ | ŝ ╞ «itinerarychangerequired, p’, c’, true »] c1 = [s1 sr1 | s2] the specific instances of the situations that relate to the example scenario above are: s1 ╞ « travelsafetywarning, solomonislands070328, true» s2 ╞ «hasflightreservation, fred, solomonislands070328, true » sr1 ╞ «itinerarychangerequired, fred, solomonislands070328, true » although this is a very simple example it does demonstrate how the system would handle a typical hmp scenario. a mobile situation management ontology a working mobile situation management system requires the definition of a task ontology for situation management and a domain ontology for the domain in which situations need to be managed. a high level, or generic, situation management ontology can be specified independent of the domain in which it will be used. the primary concern of this high level situation management ontology is to specify the concepts that are important to the generic task of managing situations. similarly, domain ontologies can be initially specified at a high level for a particular domain. the high level situation management and domain ontologies must then be combined and specialised to create a situation management ontology for a particular domain. for example, a nuclear power station safety system would require that specific situations, and the required responses to those situations, related to nuclear power station safety be defined. while the generic situation management ontology provides the underlying mechanism for reacting to any situation, the specific situations related to nuclear power stations and the rules defining how to react to them must be specified within the domain ontology of nuclear power stations. situation constraints can be australasian journal of information systems volume 15 number 2, 2009 55 specified within the ontology itself or they can be specified using ruleml (ruleml, 2003) or swrl (swrl, 2004). the generic, high level situation management ontology is described below. an example application ontology that combines this generic situation management ontology with a domain ontology for the travel domain can be found in appendix 2. a concrete ontology containing instantiations related to the travel domain, detailed owl specifications of these three ontologies, and sample rules defined in semantic web rule language (swrl) are provided in o’brien (2006). generic situation management ontology the generic situation management ontology is shown in figure 1. australasian journal of information systems volume 15 number 2, 2009 56 figure 1 generic situation management ontology hierarchy each of the classes in the generic situation management ontology and their properties, together with suggestions on how to use each are described in appendix 1. more detailed descriptions of each of the classes and the full owl specification of the ontology can be found in o’brien (2006).generic situation action rules the situation-action rules are stored on a server and are accessed via a unique uri that is encoded into the action_rule class. they can be specified in a number of different formats such as australasian journal of information systems volume 15 number 2, 2009 57 ruleml, a description logic language such as prolog or in semantic web rule language (swrl, 2004). swrl is recommended for all future rule development as it is now a world wide web consortium recommendation and is therefore likely to become the defacto standard in the future. specific action rules for particular reactive situations have not been defined in this work, however, the logic of reactive situation detection and action can be described as: if servicesituation(c) then for all hmpsituations(c’) or servicesituation (c’) where (c = c’ or c intersects c’) if hmpsituations(c’) is a background condition on reactivesituation (c’) or servicesituations(c’) is a background condition on reactivesituation (c’) then reactivesituation(c) is true endif endif where servicesituation(c) is a situation that exists because of a hmp’s state of affairs of a service in context c, hmpsituation(c’) is a situation that exists because of a hmp’s state of affairs in context c’, background condition is a required condition of a defined situation constraint and reactivesituation(c) is a consequential situation that exists that requires some action to be taken. similarly, if hmpsituation(c) then for all hmpsituations(c’) or servicesituation (c’) where (c = c’ or c intersects c’) if hmpsituations(c’) is a background condition on reactivesituation (c’) or servicesituations(c’) is a background condition on reactivesituation (c’) then reactivesituation(c’) is true endif endif problems and issues encountered as can be expected in an “inherently iterative and incremental activity” such as design science research (hevner et al., 2004), some problems and issues were encountered when developing the ontology. these are discussed below together with the adaptations of the original design that were made or are recommended to address the problems and issues. ontology development in the process of developing the generic situation management ontology some unexpected, but resolvable, issues and problems were encountered. they were generally related to the ability to represent things and relationships in different ways. nothing was encountered that could not be represented in the ontology. this section discusses these issues and the decisions that were made in relation to them. specification of location as previously discussed, a key dimension of context is spatial location. to effectively represent spatial location in the real world requires the ability to specify a point in space, absolute location, and an area or volume in space, that is, a location span is necessary. for example real world things australasian journal of information systems volume 15 number 2, 2009 58 such as rooms, buildings, city blocks, cities, countries and continents all have a two dimensional footprint, and some are three dimensional. specification of a point in two dimensional space can be achieved by simply specifying a geographical co-ordinate such as latitude and longitude. i decided to use latitude and longitude to specify absolute location as it is universally understood and accepted. to allow specification of a point in three dimensional space an additional dimension of altitude was added as it is required in some domains, such as air travel. specification of an area on the earth’s surface can be achieved in many ways such as a bounding rectangle, a bounding circle or a bounding polygon. a simpler, and more flexible geocoding system, natural area coding (nac) was chosen as it can be used to specify an area on the earth of virtually any size, and if necessary, a volume of any size anywhere in the universe (nac, 2006). while it is a proprietary encoding scheme, it has been gaining wide acceptance and is flexible enough to enable specification of everything from a continent to an absolute location. however, since conversion between nac format and latitude/longitude can be achieved with simple algorithms i decided to use nac encoding for specification of areas in conjunction with latitude and longitude. i have called this dimension of context locationspan. specification of time another key dimension of context is time, or temporal location. similar to spatial location, a temporal location can be absolute, that is a specific point in time, or it can span many points in time, that is, it can have a duration. as local time is relative to location, a universal absolute time is necessary to allow comparison of time in different locations. the most common absolute time for civil usage is co-ordinated universal time (utc). i have used utc as my ontology is expected to be primarily used to implement civil applications. in addition to specification of absolute time, the local timezone must be specified to allow appropriate reaction to situations. for example, opening and closing times of hotels and restaurants, arrival and departure times of flights and flight curfews are generally determined by local time, not by utc. the timezone entity in the generic ontology allows the number of hours ahead of utc to be specified for the local timezone. duration, which i have called timespan in this work, is specified by an absolute start time specified in utc and an absolute end time, also specified in utc. for ease of reading and automation an additional property, granularity, is included, which can take values such as second, minute, hour, half-day, day etcetera. specification of contexts as the reaction to situations can depend on the temporal and spatial subsumption relationship between the user context and the service context, the context class includes object properties for specifying which contexts it subsumes (spatiallysubsumes and temporallysubsumes) and contexts by which it is subsumed (isspatiallysubsumedby and istemporallysubsumedby). for the situation type s where s = [s | |= <<isprimeminister, c’, john howard, true>>], an example of a situation that is a member of s would be one in which the context was 31 march 1996 and australia because the timespan “2 march 1996 to today” subsumes 31 march 1996. a context of 31 march 1996 and brisbane would also create a situation that supports the infon because australia subsumes brisbane. australasian journal of information systems volume 15 number 2, 2009 59 to specify the above, each context with a locationspan of australia would have brisbane included in its set of spatiallysubsumes properties and each context with a locationspan of brisbane would have australia included in its set of isspatiallysubsumedby properties. similarly the timespan of john howard’s term of office as prime minister would temporally subsume all days, months and years within his term and this would be specified in the temporallysubsumes property of each context that has a timespan of howard’s term in office. context property of infons and situations the difference between an infon and a situation is that an infon can exist without being situated. when it becomes situated it becomes a fact. for example, the infon <<isprimeminister, john howard, true>> and <<isprimeminister, john howard, false>> are both valid infons although they appear to be contradictory. this is because they are not situated. when we put either of these infons into a particular context, it becomes either a truth (or fact) or a falsehood depending on the context. if the context is the united kingdom in 2006 then it is clearly false but it is true, or factual, in australia in 2006. a situation is a state of affairs (infon) in a particular context, so the infon <<isprimeminister, john howard, true>> is supported in all contexts where country equals australia, and the date is between 2 march 1996 and today. that is, s = [s | |= <<isprimeminister, c’, john howard, true>>] an infon in a specific context is a situation. for this reason the infon class of the situation ontology does not have a context property. the context property is specified in the situation class, together with the associated infons and background conditions. this simplifies the specification of the generic situation ontology and the action rules associated with it. specification of situation and context transition rules event collections cause a transition from one context to another but do they also cause a transition from one situation to another? events can cause a change in context and they can directly cause a new situation to occur with or without a change in context. an event will create a new state of affairs, that is a new infon, in an existing context. the combination of this new infon in an existing context will result in a new situation if that state of affairs did not previously exist. therefore, both context transition rules and situation transition rules should include any relevant event collections. situation constraints are still required to specify the transitions. ontology instantiation when instantiating the ontology for the travel domain it became clear that instantiating all of the objects and their properties for the sample scenario would be a repetitive task that would not provide any additional insights in the evaluation of the ontology. although the entire scenario was instantiated as part of the project, only one instantiated object of each class is included in o’brien (2006) as examples. similarly for the second concrete ontology for the academic domain. rule specification as production of a prototype was outside the scope of this work, detailed action and context subsumption rules were not developed. however, consideration was given to the rule specification methods that were available. the contemporary choices for web based situation management systems include ruleml, semantic web rule language (swrl) and the use of situation constraints. it was clear from the development of the situation ontology that the use of situation australasian journal of information systems volume 15 number 2, 2009 60 constraints to specify complex rules was not an appropriate choice. as an swrl add-on is now available for the ontology editor, protégé (protégé, 2006), and protégé can export its data in a format that is compatible with agent and expert system development tools such as the java expert system shell (jess) (jess, 2006), i recommend that swrl be used to specify the rules for the prototype. scalability rules and actions are defined for classes or types of reactive situations, not individuals. the complexity of the rules versus the granularity of the situation classes and contexts is a trade-off. the choice of granularity depends on the power of the end-user devices, the context granularity of the service to which the user subscribes and the users desired level of automation. if the context granularity is kept large, the complexity of the situation detection process is minimised, however, the complexity of the action rules will increase as more logic will be required to determine the most appropriate actions. this is appropriate for today’s mobile infrastructure where the system components would execute on large internet based servers and handheld devices are of limited capability. in the future, as handheld devices become more powerful, the context granularities could be reduced to provide more specific situation detection and management. also, many of the current service providers only provide information at quite coarse spatio-temporal granularity, making it pointless to have a situation detection and management system that uses a finer granularity. for example, the australian department of foreign affairs and trade (dfat) issues travel warnings by country, not by city, state or region. however, transport situations need to be at the granularity of city and day to be able to manage flight cancellation and delays, train trips etcetera. summary this research contributes a novel design artefact, namely a generic situation management ontology, thus satisfying the first type of contribution. it also contributes to the foundation knowledge base of mobile service delivery systems by providing a generic situation management ontology, based on situation theory, that can be used in future research and systems design. this demonstrates the applicability, and feasibility of using situation theory in the design of reactive information systems. furthermore, the support within the ontology for context based filtering for situation detection contributes to the efficiency of implementation and operation of situation driven reactive information systems. hevner et al (2004) state that any artefacts produced by design science research “must be implementable”. representations of contexts, situations and rules must be available in an agreed digital form before an automated reactive system can be developed. recently xml has become the standard for encoding data that will need to be shared across heterogeneous systems. however, xml by itself does not carry any semantic information related to the data that is included in the document. the semantic information comes from the agreed meaning of the tags surrounding the data. as the recent research in contexts, situations, ontologies and rule specification has been carried out in a number of different projects, in a number of institutions by researchers with quite different viewpoints on the subject areas, there is neither a single ontology nor a single markup language that covers all these areas. bearing this in mind, the situation management ontology developed in this work, which covers contexts, situations and situation action rules were encoded in the de facto standard ontology and rule markup languages, owl and swrl. this ensures that they are relevant to current and future work being undertaken in web-based information systems. ontology australasian journal of information systems volume 15 number 2, 2009 61 development and management tools such as protégé (protégé, 2006) that have been used in this research have built-in support for owl and swrl, simplifying the development of owl based ontologies. furthermore, the ontologies developed in these tools can be exported in a form suitable for use in agent development systems such as jess. thus, the requirement of implementability is satisfied. in summary, this paper presents a generic situation management ontology and generic action rules that are: • based on etzion and adi, cherry and lenat’s work; • semantically consistent; • specified in the xml based, industry standard specification languages, owl and swrl; • able to be adapted to specific domains such as travel and academic support services this work has also introduced the concept of a reactive situation management system that utilises contexts to eliminate irrelevant situations from consideration for action. the aim is to improve system efficiency and accuracy. it has brought together concepts from ontology theory, context theory and situation theory in the design of a general situation management ontology that can extended to a domain specific situation management ontology. although the travel domain was used to exemplify how the system works, it is not restricted to the travel domain. as the system is ontology driven, it can be adapted to a new domain by replacing the travel related ontology tags with tags related to the new domain. to further validate the generic ontology, an academic situation management ontology was developed by simply replacing the travel specific components in the ontology with academic domain components. see appendix 3. very few changes to the generic travel domain ontology were required to change it to the generic academic domain ontology, confirming that the generic situation management ontology is flexible enough to be adapted to any domain. this was a key goal of this research. more detailed descriptions of each of the classes and the full owl specification of the concrete academic ontology can also be found in o’brien (2006). limitations due to the limited time and resources available, some limits must be set to the scope of any research project. similarly, technical, financial and organisational factors also limit their outputs and the implementability of the outputs in practice. in this section, the key limitations and their effects are discussed. design evaluation and generalisability as no working prototype has been developed to test the effectiveness and efficiency of the proposed ontology, it was not possible to verify the effectiveness of the ontology in an operational system. however, the ability to easily specify the required concepts and rules has been demonstrated. all that is left to be done to develop a working prototype is the coding to implement the required agents and service managers. while this is not a trivial task, it is also not a difficult task for an experienced software developer using modern software development tools. for this reason, this work has concentrated on the novel and complex tasks of ontology design. a consequence of this is that testing of the design had to be done on paper by working through a scenario. while this has indicated that the design is effective, more rigorous testing will require the development of a live working prototype. australasian journal of information systems volume 15 number 2, 2009 62 recommendations for future work this work provides a foundation on which future work can be carried out to assess the effectiveness of context filtered situation management systems. there are a number of areas where work still needs to be carried out before working systems can be deployed in practice. the next step will be the development of a working prototype of a mobile situation management system that is based on the generic situation management ontology. this will most likely be developed using jess and jatlite as the ontologies that have been developed in this work can be exported in a format that can be directly accessed by these tools. once developed, the effectiveness of the prototype system can be evaluated through controlled experiments. references adi, a., et al. (2002) “the situation manager component of amit active middleware technology”, lecture notes in computer science (2382), london, springer-verlag, pp 158168. akman, v. and surav, a. (1996) “steps towards formalizing context”, artificial intelligence magazine, vol 17 no 3, pp55-72. barwise, j. (1981) “scenes and other situations”, journal of philosophy, vol 78 no 7, pp369397. barwise, j. (1987) the situation in logic, csli lecture notes 17, pp 59-77, centre for the study of language and information, stanford university, california barwise, j. and perry, j. (1983) situations and attitudes, mit press, cambridge. cherry, g. (2001) extending and formalizing the object-oriented paradigm with situation-driven modeling, thought**tools corp. http://www.sdml.com accessed 5 september 2007. devlin, k. (1991) logic and information, cambridge university press, cambridge. fischer, k. et al. (2002) “multimedia workplace of the future map”. http://informatiksysteme.ptit.de/ mti-2/cd-rom/projects/map/beitrag_map.pdf accessed 7 september 2007. fukazawa, y. et al. (2006) “situation-aware task-based service recommendation”, in proceedings of the fourth international conference on mobile systems, applications and services, mobisys 2006, uppsala, sweden. guarino, n.(1998) “formal ontology and information systems”. in formal ontology in information systems (n. guarino ed.), ios-press, amsterdam. hendler, j. (2001) “agents and the semantic web”, ieee intelligent systems, vol 16 no 2, pp 3037. hevner, a. et al. (2004) “design science in information systems research”, mis quarterly, vol 28 no 1, pp 75-105. jakobson, g. lewis,et al. (2005) “overview of situation management at sima 2005”. in proceedings of sima 2005, atlantic city, new jersey. jess (2006) “jess, the rule engine for the java platform, the java expert system shell”. http://www.jessrules.com/ accessed 6 september 2007. kee (1993) “kee tm (knowledge engineering environment) software development system, version 4.1”, intellicorp, inc., mountain view, ca. australasian journal of information systems volume 15 number 2, 2009 63 lenat, d. (1998) “the dimensions of context-space”, cycorp, austin. www.cyc.com accessed 8 september 2007. loke, s. (2006) “context-aware pervasive systems: architectures for a new breed of applications”, crc press, boca raton. matheus, c. (2005) “using ontology-based rules for situation awareness and information fusion”. in proceedings of the w3c workshop on rule languages for interoperability, washington. matheus, c. et al. (2003) “a core ontology for situation awareness”. in proceedings of sixth international conference on information fusion, pp 545-552, cairns, australia. mit (2004) “mit oxygen project”. http://oxygen.lcs.mit.edu accessed 6 september 2007. nac (2006) “natural area coding, nac geographics inc. http://www.nacgeo.com/ accessed 10 september 2007. o’brien, p. (2006) “an architecture for ubiquitous mobile service delivery”,phd thesis, school of itee, university of queensland, australia. http://adt.library.uq.edu.au./public/adtqu20060929.120135 accessed 12 october 2007. o'brien, p. & burmeister, j. (2003) “ubiquitous travel service delivery”, international journal of information technology and tourism, vol 5 no 4, cognizent, new york. o’brien, p. & colomb, r. (2005) “generic context driven situation detection and management”. in proceedings of the iasted international conference on internet and multimedia systems and applications 2005, grindelwald, switzerland. perry, j. (1993) the problem of the essential indexical and other essays, oxford university press, new york. protégé. (2006) “the protégé ontology editor and knowledge-base framework”, stanford university. http://protege.stanford.edu/ accessed 5 october 2007. ruleml (2003) http://www.dfki.uni-kl.de/ruleml accessed 1 october 2007. swrl (2004 ) “swrl: a semantic web rule language combining owl and ruleml”. http://www.w3.org/submission/2004/subm-swrl-20040521/ accessed 15 september 2007. tin, e. et al (1995) “towards situation-oriented programming languages”, acm sigplan notices, vol 30 no 1, pp 27-36. wimax (2006) the wimax forum. http://www.wimaxforum.org accessed 6 october 2007. australasian journal of information systems volume 15 number 2, 2009 64 appendix 1 generic situation management ontology class descriptions absolute_location the geocode for an absolute geographical location. latitude and longitude are standard geocode values expressed in degrees. both are mandatory. altitude is optional. all three are single precision floating point number. absolute location is used to specify a point on or above the surface of the earth. absolute location is needed to specify centres, the start and end of a locationspan and the corners of areas and volumes. absolute_time absolute time specifies the absolute time in utc irrespective of the location. this can be combined with timezone to determine local time. action_rule this class is used to initiate an swrl action rule. the uri of the rule is specified together with the name of the function that is to be executed and any required parameters for the function. address this is the street address of a place, typically a building or landmark. it is composed of the names of the country, city, suburb, street name and number and the postcode. all are strings. street, city and country are mandatory. agent the agent class is used to specify the unique identifier, name and uri of software agents that provide a virtual presence for hmps and service providers. up to 20 functions can be specified for each agent, allowing requests such as reservation requests, to be sent to the service provider. compound_infon this class allows an infon to be specified that is a logical or or and combination of two individual infons. if negation is required for one of the infons, this can be achieved by setting the opposite polarity in that infon. context the context class is used to fully define a unique context. each instance of the class has a unique context identifier that can be used for querying and comparison. each context instance must have one or more timespans and one or more locationspans specified. if the context subsumes other contexts these can be specified with the context_subsumes object property. if the context will change from this context to other contexts or situations, the rules under which these transitions will occur can be specified in the context_transition_rules object property. context_to_situation transition_rule this allows one or more transitions rules to be specified for transitions from the context in which this rule is specified to situations. each transition rule is composed of the destination situation and the conditions under which the transition occurs. australasian journal of information systems volume 15 number 2, 2009 65 context_transition_rule this allows one or more transition rules to be specified for transitions from the context in which this rule is specified to other contexts. each transition rule is composed of the destination context and the set of events each of which causes a context transition (the event collection). event the event class is used to define an event that causes a change of state of a hmp or service. the time the event occurs, the time it is detected, the certainty that the event occurred and the length of time that the event will remain active are common properties to both the hmp_event and service_event subclasses. these properties can be used to determine whether an event should be added or removed from an event collection. event_collection all the events that can individually cause a transition to a particular new context are included in a set of events, event_collection_events. the event_collection_group_membership_condition object property is used to specify the context_to_situation_transition_rules and/or context_transition_rules that determine what the new situation or context will be if one of the events in the event collection occurs. hmp the highly mobile person class (hmp) is a sub-class of person and is used to specify whether a particular hmp wishes to have visible and/or audible alerts and to set the service time and place granularity preferences. hmp_event defines an event that causes a change of state of a hmp. this is a sub-class of person that includes one additional property that identifies the hmp that generated the event. hmp_event_changed context defines an event where a hmp’s current context changes. this is a sub-class of hmp_event that includes additional properties that specify the context the hmp has moved from and the context the hmp has moved to. hmp_event_pref_changed defines an event where a hmp’s service preferences change. this is a sub-class of hmp_event that includes additional properties that specify new service preferences. infon this class allows one unit of information to be specified, as defined by barwise (1987). the properties of this class are: • infon_element that allows one or more objects to be specified as parameters. any object can be specifed as an element. australasian journal of information systems volume 15 number 2, 2009 66 • infon_polarity is a boolean that takes the values true or false and is applied to the relation property. • infon_relation specifies the relationship between the elements. infon_arrived an infon that asserts that something such as a hmp has arrived in a location. this is a sub-class of infon. infon_future_location an infon that asserts that something such as a hmp will be in a location in the future. this is a subclass of infon. it is used by situation_hmp_future_location class to specify the future location in which a hmp situation could exist. infon_inlocation an infon that asserts that something such as a hmp is currently in a location. this is a sub-class of infon. it is used by situation_hmp_in_location class to specify the current location in which a hmp situation exists. infon_intransit an infon that asserts that something such as a hmp is travelling between two locations. this is a sub-class of infon. it is used by situation_hmp_intransit class to specify the context that a hmp has left. the context that the hmp is transiting to is specified in the situation property situation_context. infon_past_location an infon that asserts that something such as a hmp was in a location in the past. this is a sub-class of infon. it is used by situation_hmp_past_location class to specify the past location in which a hmp situation existed. location_span this class is used to specify a geographical area. the area is specified by a corner which is an absolute_location object, a width and a length in metres. these three properties are mandatory. additional optional properties can be used to specify other location_spans that are spatially subsumed by this object, other location_spans that spatially subsume this object, altitude in metres and natural area code (nac). person a human being who is currently living, will be alive in the future or was alive in the past. the properties of this class include a unique person identifier, the person’s name and the contexts in which the person was, is or will be alive. place a type of physical location. the location is specified using a location_span object and a name. place_bloc a group of countries that form a political, economic or social group. this is a sub-class of place that includes an object property that allows a place_country object to be specified for each country in the bloc. australasian journal of information systems volume 15 number 2, 2009 67 place_building a physical structure. this is a sub-class of place that includes an object property that allows the address of the building to be specified. place_city_block a city block typically bounded by a number of streets. this is a sub-class of place that includes an object property that allows a place_street object to be specified for each of the streets that bound the block. place_continent one of the seven continents. this is a sub-class of place that includes an object property that allows a place_country or place_bloc object to be specified for each country or bloc on the continent. place_country a sovereign state. this is a sub-class of place that includes properties that allow cultural and economic details of the country to be specified. these properties are its name, international three letter country code, international telephone country code, three character currency code for each currency that is legal tender, official spoken languages, religions practiced, and the location_state objects that it contains. place_lga a local government area. typically a city or shire. this is a sub-class of place that includes properties that allow the timezone of the lga and the suburbs that it contains to be specified. place_locality a part of a city. this is a sub-class of place that includes properties that allow the postcode of the locality and the city blocks that it contains to be specified. place_region a part of a state that generally contains a number of lgas. this is a sub-class of place that includes a property that allows the lgas that it contains to be specified. place_state a province of a country. this is a sub-class of place that includes a property that allows the lgas and regions that it contains to be specified. place_street a named road within a city. this is a sub-class of place. place_suburb a named area of a city that generally contains a number of localities. this is a sub-class of place that includes a property that allows the localities that it contains to be specified. service an information, communication, distribution or transaction service provided by a service provider. this is specified within the service_provider class. australasian journal of information systems volume 15 number 2, 2009 68 service_event defines an event that causes a change of state of a service provided by a service provider. this is a sub-class of event that includes a property to specify the service_manager object that was the source of the event. service_manager a software agent or program that manages one or more service providers of a particular type. this class includes properties that allow the type of service that the agent manages, the service providers that it manages, the uri of the agent and a unique service manager identifier, to be specified for each service manager. service_preference this is a sub-class of situation_hmp that allows situations that result from hmp service preference settings to be specified. properties are provided to set the granularity of time and space for a particular service type, that is, a particular service manager. service_provider a physical provider of a particular service such as an airline. properties are provided to specify its name, its street address, the software agent (sa) that interfaces to it, its code (such as iata code), the contexts in which it operates, the service manager that manages it, and the uri of the service provider’s website. situation a state of affairs that is true in a particular context. this class allows a situation as defined in situation theory (barwise 1987) to be specified. all of the elements that are required to define a situation using devlin’s (1991) notation can be specified using the properties of the situation class. the background condition or constraint required for the situation to exist, the context in which the situation exists, the infons the situation supports, the conditions under which a transitions will be made from this situation to other situations and a unique situation identifier can all be specified. situation_hmp a state of affairs of a hmp that is true in a particular context. this is a sub-class of situation that allows a situation that is associated with a hmp to be specified. this is achieved by setting the hmp property. situation_hmp_future_location a state of affairs, or an infon, where it is true that a particular hmp will be in a location at a particular time in the future. this is a sub-class of situation_hmp. situation_hmp_intransit a state of affairs, or an infon, where it is true that a particular hmp is in transit between two locations. this is a sub-class of situation_hmp. situation_hmp_in_location australasian journal of information systems volume 15 number 2, 2009 69 a state of affairs, or an infon, where it is true that a particular hmp is in a location at a particular time. this is a sub-class of situation_hmp. situation_hmp_past_location a state of affairs, or an infon, where it is true that a particular hmp was in a location at a particular time in the past. this is a sub-class of situation_hmp. situation_reactive a situation that requires some action to be taken. this is a sub-class of situation. the additional property situation_reactive_reaction allows one or more reactions to be specified for each reactive situation. situation_service a state of affairs, or an infon, of a service that is true in a particular context. this is a sub-class of situation. the service that the situation is related to is specified by setting the situation_service_service object property to the service manager for this type of service. the severity of the situation can be set through the situation_service_severity property. this can be used to match user preferences with service situations by severity level. situation_transition_condition a rule defining what background conditions can cause a transition from the current situation to a new situation. one or more background conditions or constraints can be set any one of which, if true, will cause a transition the new situation. the background conditions are themselves situations. time_span a period of time. this class allows the start and end time, the type of time and the granularity of the time period to be specified. granularity can be set to a value in an enumerated type that has common english descriptions of time periods such as second, minute, hour etcetera. similarly, type of time can be set to a value in an enumerated type that has common english descriptions of time periods in which particular types of activities take place such as worktime, leisuretime, sleeptime etcetera. additional descriptions can be added to the enumerations as required. properties can also be set to specify other time periods that this time period subsumes or that it subsumes. timezone a standard international timezone. this class allows a timezone code to be associated with a period of time ahead of utc. as all times are specified in utc, objects of this class can be used to convert utc to local time. australasian journal of information systems volume 15 number 2, 2009 70 appendix 2: additional travel domain specific classes based on the identified high value services and a real world independent traveller scenario, the generic situation management ontology was extended to include additional classes that are specific to the travel domain. the new classes that were added are described below. hmp_event_fare_paid event where hmp pays fare for a reservation hmp_event_res_made event where hmp pays makes a reservation infon_has_reservation infon describing state of affairs where a hmp has a reservation place_building_airport a building categorised as an airport place_building_hotel a building categorised as an hotel reservation a reservation for accommodation or transport services reservation_accommodation a reservation for accommodation reservation_transportation a reservation for transport services reservation_transportation_air a reservation for air transport services reservation_transportation_bus a reservation for bus transport services reservation_transportation_train a reservation for train transport services service_event_arrival_delay event where arrival of something has been delayed to time later than the scheduled time service_event_departure_delay event where departure of something has been delayed to time later than the scheduled time australasian journal of information systems volume 15 number 2, 2009 71 service_event_health_warning event where a health warning has been issued by a health information service provider service_event_safety_warning event where a safety warning has been issued by a travel safety information service provider service_event_trip_cancelled event where a scheduled trip has cancelled situation_hmp_reservation_required state of affairs in a particular context that requires action to be taken to make a reservation for a hmp situation_service_accommodation a state of affairs of an accommodation service provider in a particular context situation_service_accommodation_available a state of affairs of an accommodation service provider where accommodation is available in a particular context situation_service_accommodation_price_change a state of affairs of an accommodation service provider where the price of accommodation has changed in a particular context australasian journal of information systems volume 15 number 2, 2009 72 appendix 3: additional academic domain specific classes situation_reactive_cancel_lecture a state of affairs, or an infon, where it is true that a particular lecture must be cancelled in a particular context. situation_service_evacuation_required a state of affairs, or an infon, where it is true that evacuation of a particular building is required in a particular context. situation_service_lecturer_unavailable a state of affairs, or an infon, where it is true that a particular lecturer is unavailable in a particular context. situation_service_lecture_cancelled a state of affairs, or an infon, where it is true that a particular lecture has been cancelled in a particular context. situation_service_noparking a state of affairs, or an infon, where it is true that no vacant car parking spaces are available in a particular context. situation_service_traffic_delay a state of affairs, or an infon, where it is true that a traffic route is congested and experiencing delays in a particular context. situation_service_fire_alarm a state of affairs, or an infon, where it is true that a fire alarm is ringing in a particular context. course a course of study. software quality management and organizational fit s.h.nielsen school of computing and information technology, griffith university nathan campus, kessels road, brisbane, qld 4111, australia suen@cilgu.edu.au abstract: this paper describes some of the findings, of an ongoing ethnographic study of a computer operations section in an information technology centre. the study finds that after an initial period of staff acceptance of prescribed quality management procedures, certain features of organizational culture, structure and power, work against continued conformance. procedures will be modified, firstly to resolve any inconsistencies between the prescribed procedures and strongly held beliefs and values about work practices and organization, and secondly to reduce or eliminate perceived threats. the paper argues that software quality management is based on a unitarian approach to organization, which ignores the plurality of beliefs and work contexts which exist in an organization, and which assumes that organizational features can be managed and changed in predictable ways. this paper suggests that a pluralist approach to organizational analysis helps to reveal the nature and extent of changes required to the quality management system and the requirements for implementing changes. introduction although the importance of software quality, as a strategic and competitive tool is now widely accepted, the achievement of improvements in quality is still problematical. many of the hard' problems, the development of tools and systems, seem to have been solved, but the 'soft' problems of values, attitudes, and organization remain, so that the implementation of software quality systems has not been so successful. a new approach to these problems has been developed by the total quality movement, which views quality as a philosophy of management, and sees the organization as a series of interrelated processes which can be assessed for their effectiveness in contributing to quality management. in the software quality arena, considerable effort is being invested in the definition and assessment of software processes. (humphrey and sweet 1987) the purpose of the study discussed in this paper is to gain an understanding of how and why actual software quality practices differ from the policies and procedures prescribed by management in a particular organizational context. the findings discussed relate to the acceptance of change, and the features of organizational culture, structure and power which affect the implementation of prescribed procedures. the study research setting the research setting and methods of this study are reported fully in davies and nielsen (1992). a brief summary is presented here. the organization selected for the field study, the operations section of an information technology centre (itc) in the division of information services of an australian university, was chosen for ease of access and because of the pivotal role it appeared to play in the itc. the operations section maintains a help desk, is involved in multiskilling, organizes work schedules, coordinates much of the documentation which deals with system development and use, deals with internal system problems and system failures and is generally the interface between the users and the other sections of the itc. the field study focuses on the configuration management practices of the section. these practices were chosen because in the author's experience the implementation of such procedures has often been problematic, and because configuration management occupies an important place in the development of a quality management system. according to the capability maturity model proposed by humphrey (1990), configuration management processes are required for an organization to move from the initial to the repeatable and defined levels of process maturity. ultimately, at the level of process and work systems, it is suggested that configuration management can provide a continuous source of information on software development practices in an organization. "according to the sei [software engineering institute], an important criterion for assessment of an organization's software maturity level is the degree to which its software development process is defined (and, therefore, documented)." (hopkins and jernow 1990, p 125) 38 ajis the discipline of configuration management is currently undergoing some redefinition in relation to its role in system development (olsen 1993). however, it is generally agreed that configuration management has four primary goals: to identify and define the items in a system (identification), to control changes to these items (change control), to record and report the status of these items (configuration status accounting), and to audit the items to verify completeness and conformance to specifications (buckley 1993, whitgift 1991).this paper focuses specifically on configuration identification and change control, which were continuing concerns of the operations section during the period of the study. research methods the research methods used in the study are ethnographic, including participant observation, unstructured interviews, attendance at regular section meetings, diary recordings, and analysis of documentation. the study commenced in 1991 with an intense two month period of 'shadowing' one member of the section. the interpretive perspective is used, to enable the author to gain an understanding of the meaning of work practices within their organizational context how people make sense of their everyday practices in relation to their goals and their working situation. this sense making is crucial to the understanding of what is meant by quality, and also to how it can be achieved in ways that do not conflict with other beliefs and values. the particular interpretive approach used by the author is that of critical hermeneutics, the interpretation of the text produced by people in their work contexts, and of the texts produced by the author in the act of interpretation. (thompson 1981) thus the interpretation is seen as a recursive act, which is potentially infinite. this recursiveness is also important to the ongoing nature of meaning construction, which continually reacts to and creates change in the sense making environment. it is expected that this approach will contribute more to an understanding of why quality management systems are not successfully implemented, than the acontexrual view of much of the positivist research currently undertaken in the software quality arena. alhough there is the problem of generalization in such research, which can only be overcome by further research and the development of a larger body of knowledge about the organizational contexts of software quality management practices. the specific approach to interpretation taken in this study is based on the intertexuality relationships between texts and contexts. themes, such as issues and problems, are identified by lexical and syntactic foregrounding which continues across a number of texts. this continuation is explained by the notion of cohesion in text, which shows the links from one specific textual unit to other units (con-texts) and to the wider situational and cultural contexts (halliday and hasan 1985). research findings the author's identification and interpretation of themes was challenged and modified by feedback sessions between the author and the staff of the operations section and these differences in interpretations became further texts to be studied. the themes which are discussed in this paper are concerned specifically with the problems of establishing satisfactory configuration management practices and associated documentation. to explore the possible meanings of these themes we need to consider the features of the situational context of the field study and to place these in the wider context of organizational culture, structure and power. the significance of this for the implementation of software quality management practices is also discussed. the theoretical outcomes of this study so far are that initially staff may accept the quality management procedures, but that ultimately these procedures will be modified by staff, even when ownership of the procedures appears to have been accomplished. the reasons for these modifications being made are, first to resolve any inconsistencies between the prescribed procedures and strongly held beliefs and values about work practices and organization, and secondly to reduce or eliminate perceived threats. the gap that arises between prescribed procedures and actual work practices results from a unitarian perspective on organizational culture, structure and power which ignores the plurality of beliefs and work contexts which exist in an organization. the practical outcomes of the study are a greater understanding of the different perspectives on organization within itc, and of the organizational features which work against the acceptance of prescribed practices. if this understanding can be confirmed and shared by itc management, then the chances of successfully implementing software quality management systems may be improved. september 1995 39 the context of change resistance to change the implementation of software quality management systems means the introduction of significant changes to the way an organization or group of people operates. the way that changes are handled by management and staff is an important feature of the organizational context. during the period 1992-1993, the general manager of the itc initiated work on the implementation of practices conforming to the australian standard for software quality management (as3563) with the objective of gaining official certification. most of the staff of the operations section were involved in the formulation of section policies and procedures, as well as vision statements and strategic plans for other sections in the division. during this time a sequence of events was seen to recur several times. one or more members of staff would be involved with management in the formulation of an objective and a consequent set of procedures to enable the objective to be carried out. (for example, a new procedure for recording software change requests was proposed, so that information could be collected about the frequency and nature of these requests, as well as the time required to make the changes.) the objective and the procedures would be discussed and agreed on or modified at section meetings and would be documented for future reference. many of these procedures were such that they would need to be carried out very frequently, on a daily, or at least weekly, basis. after some period of time the procedures would be raised again at a section meeting, and in many cases the procedures would have been changed significantly or ignored in the interim. there would be disagreement amongst staff about the importance, correctness or usefulness of the procedures and after further discussion over one or more meetings, a new set of procedures would be put into place. these changes were often not documented and were rarely communicated to management. there were very few occasions during this two year period in which officially instituted procedures were accepted and maintained without modification, or where management were involved in discussions about subsequent modifications. acceptance of change the way that changes were handled in this organization can be considered in relation to the literature on the way that people react to attempts to introduce change. acceptance of change is usually characterised as taking one of three forms compliance, identification or internalization (e.g handy 1985). compliance means that change is accepted quickly, but unwillingly, because the recipient is given no choice. in other words, some source of power money, status, etc., is exchanged for the recipient's compliance. the recipient may be prepared to accept the change without the exchange offered. however the use of power denies the recipient this choice, so compliance is the only response which the employee can offer. such acceptance requires the manager to continually check and monitor the employee and to offer further rewards. commitment to the change by the employee remains low. identification, as a form of acceptance occurs when staff admire or identify with the source of the change, be it an individual or a group. where managers are close to their employees and have considerable personal power they may obtain this kind of acceptance readily. however it requires the continued presence of the initiator of the change. if the group disbands or the initiator leaves, commitment to the change may disappear. the last type of acceptance is internalization, where an individual accepts the change as their own idea. this creates a high level of commitment and is self-maintaining, however it takes great effort to obtain and requires granting the member of staff considerable independence. the information systems literature which deals with the impact of systems on organizations indicates that the internalisation of change through 'ownership' is essential for successful implementation (knight 1989, land 1987). in this study, members of the section were fully involved in formulating the original changes. the verbal and written language used by management and staff during these negotiations showed that participants were attempting to achieve changes in a way that was acceptable to everyone, and the operators appeared to have internalised these changes. it would therefore seem reasonable to conclude that monitoring by management was not required. however, since the implementation of most of these changes was shortlived, another explanation must be sought. what is also significant in this study is that there were often considerable differences between the official procedures which were not accepted and the procedures which were finally implemented. an understanding of the nature of these differences and the reasons for them occurring, may be looked for in the analysis of the operators discourse within the organizational context, discussed in the next sections of this paper. 40 ajis organizational culture perspectives on organizational culture before discussing the organizational culture as a context for the development of configuration and documentation practices, it is important to refer briefly to some of the assumptions about culture, which are found in the literature of quality management. the assumption that organizational culture can be changed and managed lies behind much of the literature. such an assumption is called 'integrationist' (meyerson and martin 1987), and its adoption, either implicitly or explicitly, is consistent with the notion that culture is something an organization has, that it is a set of variables which can be manipulated to achieve desired objectives. according to this view, the effective implementation of quality management practices can be achieved by the sharing of vision and values across the organization. there is an emphasis on achieving common objectives and the idea of ambiguity or subcultures is disregarded. "quality as an organizational truth is the outstanding emotive force which can unify everyone within the organization" (lascelles and dale 1988). over a two year period there were a number of interventions into the practices of the operations section, and the author was able to observe the effects of these actions. there have been no significant empirical studies of how quality management interventions affect organizational culture and so far there is no evidence that change does occur except at the surface level (dale and cooper 1992). this study supports this, in that no effects were observed, except in the creation of some artefacts (such as procedure manuals) for official use. indeed most of the interventions were adapted to be consistent with strongly held beliefs and values, for example about the amount of autonomy professional staff should have. so that although the staff accepted in principle the new practice, it was implemented in such a way that the 'new' system was virtually the same as the system it was intended to replace. different perspectives on organizational culture are provided by the differentiation and fragmentation views (martin and meyerson 1988) which recognize that different and sometimes incompatible beliefs and values exist within organizations, and that these are not always easy to identify. so that interventions into the organizational culture may have no effect, or that the effects may be unpredictable. this study supports these latter views, by showing that differences continued to exist between management and the staff of different sections, in spite of the considerable effort to involve staff in the development of software quality policies and procedures. these three different perspectives on organizational culture are strongly related to the unitarian, pluralist and radical frames of reference for considering organizational power, proposed by burrell and morgan (1979). these are referred to again in section 5 of this paper. the importance of organizational subcultures the fragmentation view of organizational culture is supported by the following incident. a major intervention occurred when a requirement was made for a common depository for all written documentation, relating to a new system to support the help desk service. general problems experienced in compiling documentation have been discussed in a previous paper (davies and nielsen 1992). in this case, the need for visibility of system development was considered urgent by management as the help desk was under considerable criticism and the new system was expected to alleviate some of these problems. staff from several sections were involved in the development of the system, and so a procedure for change control was proposed so that all staff would be aware of which version of the documentation was the current one. the most difficult stage of development was expected to be that of information requirements analysis, and it was at this stage that the change control procedure broke down. because of the difficulty of having staff meet face to face, much of the discussion occurred on email. however, informal work groups were formed, whose documentation was not able to be managed under the new change control system. ultimately a prototype was developed which satisfied the most urgent requirements of the help desk staff, but which did not include the functions which were intended to supply information to management about the efficiency and effectiveness of the service. the development of the 'official' system which included these reporting functions was delayed indefinitely, and when changes to the prototype were proposed it was after the systems programmer who had undertaken most of the development work had left the organization and it was found that there was no extant documentation of the requirements. an analysis of the discourse that occurred during this incident showed that the objectives of the various groups were irreconcilable. consequently the informal work group was used to avoid disclosing officially some of the concerns which the different sections had with the proposed system. these objectives reflected the different values of management and staff and their different notions of how quality should be achieved. these differences were not resolved by the consultation processes used during the development of the change control system. in particular, a conflict persisted between the quality control perspective, which requires staff to recognise the need for continuous group inspection, and the perspective of the professional computer specialist who expects to be judged on the basis of results rather than on compliance with specific procedures. professionally educated personnel september 1995 41 generally expect a high degree of discretion and control in their work. they expect some autonomy in the way they organize their work practices and less outside supervision than people who have received less education. development staff expressed great impatience with procedures for change control which would slow down the pace of their work, and which intimated that they were unable to perform high quality work without outside supervision. computer support staff valued most highly their responsiveness to client needs, and would not sacrifice this responsiveness to the putative long term benefits of a new system. the operations section who were charged with coordinating and controlling the new documentation procedures suffered considerable stress during this period, because of the conflicting demands of management and the other sections. however they were half hearted in their attempts to maintain control, showing their strong commitment to the values held by the staff in the support and systems development sections. as one operator expressed it "if you're filing change reports, and the telephone's ringing and there's a customer at the help desk, i know what i'd do first." organizational structure official and emergent structures another explanation for the differences between prescribed and actual work practices can be sought in the way that control is maintained over and within groups. within the itc there are some conflicts between official control mechanisms and actual organizational structures. the overall structure resembles a bureaucracy and the official systems in place to support this structure are vertical in design. however, the actual organizational structures in some sections in itc are more organic in form, adapting and changing according to the various projects in hand. this adaptive structure adopted by the operations section is discussed below. although the operations section had very clearly defined responsibilities and official authority in relation to documentation and configuration management it had no real authority. that is, the section did not feel empowered to use this authority considering its relationship with the other sections and its own mode of operation. in relation to the project for the development of the help desk system, the status of the operations staff was generally lower than that of other sections. (in other projects, such as systems maintenance, it had a far higher status). it was therefore difficult for operations staff to convincingly argue for the adoption of the new procedures. in addition operations staff felt uneasy providing information to management which might result in criticism being directed at the staff who had produced it. in an organic style organization documentation is produced and used mainly for communication and education. in a bureaucratic type of organization this same documentation may be used for evaluation, review and performance assessment. staff are unlikely to be accurate in documenting activities which reveal problems in performance, if this information may affect their chances of promotion. this confusion over the purpose of software quality practices is likely to affect their successful implementation. organizational performance and organizational structure change whether it be the slightest deviation from an established procedure or the orderly progression from one stage of development to the next, will have some effects on the way that work is carried out in an organization. the motivation for organizational change is to improve the performance of the organization, in relation to, for example, the production of quality software. the proposed change will be based on certain assumptions about the relationship between organization and performance, and notions of what constitutes 'good' organization. an explanation for the difficulties experienced by the operations section in their attempts to comply with official procedures may be found in the conflict between different models of 'good' organization, held by the operations staff, individual section managers and itc management. the operations staff appear to be torn between a consistency and a contingent approach to organization for quality management. the consistency approach is shown by their reluctance to institute any formal changes, such as the change control system, which are inconsistent with firmly held beliefs about what is important in their work. the initial acceptance of procedures initiated by management seems to derive from their desire to show respect for their opinions, and their understanding or the political realities of the organization. but as inconsistencies become evident, the new procedures are modified without any communication to management. however, the section also takes a contingency approach by continually reorganizing on a short-term basis to cope with the frequent changes in technology, complicated work schedules, and most especially customer demands. because this organic approach is not supported by any management resources, such reorganization occurs in an ad hoc manner and damages any mutual reinforcement provided by the organization. indeed, it is often referred to by staff of other sections as an inability by operations to organize themselves properly. for example, the section has great difficulty in organizing staff meetings because they are concerned that the standard of service will drop if inexpert staff from other sections are used as stand-ins. the operations staff face daily conflict between their need to meet as a group to communicate and discuss important changes, and their desire to maintain service. thus, the 42 ajis contingent approach used by the section does not derive from any management philosophy but from the section's desire to remain responsive to urgent customer requirements. this approach is not supported by management. for the management of itc, a 'model' approach appears to be in use, reinforced by the government requirement for universities to demonstrate their commitment to quality management. in some sections, staff have greater control over their workload, and more stability in their work schedules and practices. the best ways of operating to implement quality practices, objectives and procedures are worked out over longer periods of time. however, this is often achieved at a cost to the operations section, who are most affected by the deployment of staff to such temporary arrangements as working parties on quality management. these different approaches to organization reflect different views of 'quality1; even though the notion of quality is defined and specified in official documents, the exact meaning of quality is constructed quite differently by different groups according to features of their work context and their shared values and beliefs. since the operations staff have the greatest role in the production and coordination of documentation, the incongruencies in ideas about how best to achieve quality between the different sections is expected to have long term effects on the ability to implement properly documented quality management practices. each of these approaches entails problems for software quality management. the 'model' approach seems to underlie many software quality notions such as standards, "best practice', etc. but a well identified problem with organizations attempting to emulate such models to achieve quality improvements, is that each organization varies so greatly in terms of size, history, market and so on, that the model may be inappropriate. the 'consistency' approach may entail the organization maintaining structures and practices inappropriate to external and internal contingencies. the operations section seems to be in this situation, ignoring long-term strategic considerations in order to maintain the current order. however their use of the 'contingency' approach aggravates the situation, as it prevents them from maintaining stability in work practices and puts them at odds with the management. organizational power perspectives on power the integrationist approach to organizational culture, discussed earlier, is similar to the unitarian approach to organizational power which characterises much of the discussion of software quality management. from the unitarian viewpoint, software quality management is proposed as a rational and technical process which should become a common objective for all members of an organization. conflict is viewed as dysfunctional, and able to be removed through appropriate managerial actions. however, the experience of the author in this field study was that the configuration management processes which were introduced to facilitate communication between sections and between staff and their clients, were a source of unease for staff in the operations section. this was in spite of the involvement of staff in formulating these processes. the rationality of management discourse was not challenged by staff at the time of formulating new procedures, but management authority was undermined by changes which were made by staff to protect themselves from the consequences of adopting the new procedures. this reaction by staff may be explained by viewing organizational power from a pluralist or radical point of view. the radical perspective views the organization as a political battleground where "rival forces strive for the achievement of largely incompatible ends" (morgan 1986). power according to this perspective is distributed unequally, conflict is often latent, and will ultimately result in radical change. the radical perspective provides little hope for quality improvement under present circumstances, since such a quality management programme is viewed as an instrument for management control.(boje and winsor 1993) the pluralist perspective recognizes the diversity of interests within an organization and the inevitability of conflict, which can be used positively or negatively. it also proposes that power derives from a number of sources and that individuals can draw from these sources. for example, the professional knowledge of the operations staff could be used to influence management policy. configuration identification one process which occupied considerable time at section meetings was that of establishing systems for identifying and naming configuration items. identification as a configuration management activity involves the creation of new names, since each configuration item (software, hardware, process) must be identified as unique. many of these names will form part of an artificial language, which will require a key for decoding (for example, hls/dex3.2ec3.1: key hls = mnemonic for system/programme; dex =module(subsystem); 3 = baseline; .2 = version; ec = error correction (type of change; 3 = sequential number of approved change; .1 = version of change). other names will be formed from everyday words and phrases. both types of names should be descriptive, in that if the rules of the language are understood, the item that the name refers to should be unambiguously recalled. thus identification september 1995 43 is prerequisite for one of the major functions of configuration management, that is, to be able to communicate the current state of any software product to the development team and to clients, during the development process. however, the ability to create names in this way has political consequences: what can be identified can be used for control, unless its real significance can be hidden. the search for appropriate names may be viewed as a rational attempt to understand the nature of change in software development, and to develop a classificatory scheme which will appropriately reflect the reality of change. however, such a scheme also creates a knowledge structure and presents it for scrutiny to the outside world. this structure may have a constraining effect on future practices and therefore may be conceived of in threatening terms. this is strongly demonstrated in the problem of establishing the terminology for describing types of change; error correction, enhancement, etc. during the meetings on how to record software change requests, discussion of what change is and how it should be categorised soon broke down and was replaced by a search for names that were acceptable to the group. however the operators did show awareness that the names chosen could have meaning for other staff. hence the criteria for acceptability of names was discussed. these criteria derived more from the impression that the operations staff thought the name might make on other staff, than from its applicability to the phenomenon to which it was meant to apply. a persistent approach to identification took the form of "hiding1. for example, names were sought for routines, such as tests, that were not performed any more, but which the operators did not like to ignore in case someone official remembered instituting the practice and looked for evidence that it still existed. identification and control a preoccupation of staff in many meetings was the question of how to deal with the 'merging' of previously separate systems and practices. this discussion often revealed that the current usages in the identification of systems and practices were deemed unsatisfactory by some staff they did not want their current names to be revealed to staff from other sections. some effort was appb'ed to finding new names for these items. effort was also expended on finding names for practices and systems that so far had remained nameless or had unique identification but no descriptive identification. several reasons for the preoccupation with identification emerged from the discussions at meetings. firstly, the prospective reorganization of work groups led to a feeling of going public and therefore the need to tidy up the shop, and to put on a good face. a desire was also expressed to maintain current practices as far as possible and to be in the strongest, most invulnerable position, before the reorganization occurred to have everything sewn up. the operations staff did not explicitly discuss the idea that the classification of changes to configuration items might have an impact on the way the section was able to control its own work, or that it might affect management perceptions of their efficiency. however, the solutions proposed by operations staff to the problems of classifying changes and recording the status of changes showed that these were major concerns. much of this discourse was at odds with rational notions of configuration control by making configuration items invisible to the uninitiated, and of enabling clients to understand the status of the system development project. however, when the identification activity is viewed as an instrument of management control, it is likely to be used by operational and production staff to conceal the nature of some of their activities. conclusion the tensions between prescription and practice, the lack of a common understanding of what is meant by 'quality' within an organization, and the barriers presented by traditional structures and divisions between management and operational staff are all problems which will make the implementation of quality management practices difficult in the organization studied. these problems, or similar ones are reported in the literature and are experienced by practitioners and consultants to such an extent that there have been a number of attacks on the quality movement in the software development and other industries (e.g. boje and winsor 1993). these problems have been partly explained in this paper by examining their situatedness in organization. firstly, the introduction of software quality management requires change of some form or other. in the literature of organizational change, considerable emphasis has been put on managing the process of change, to involve those affected by the change in the process of design and implementation, so that they come to 'own' the change. however this paper argues that unless the organizational context supports this approach, this will be just as ineffectual as more authoritarian approaches. the prescribed practices will appear to have been accepted, but non-prescribed practices will emerge which enable staff to maintain a reasonable working environment which is consistent with their beliefs and values. this will make it difficult for an organization to achieve any improvements in quality. secondly, actual work practices will differ from prescribed practices in a way that reflects features of the organizational context. procedures which are inconsistent with the beliefs and values of staff will be modified, to reduce inconsistencies. conflicting organizational structures will require changes to systems to accommodate 44 ajis ajis special edition-electronic commerce november 1998 electronic markets ontology: ideal architecture for a global capital market davide khali! edinburgh business school heriot-watt university edinburgh, scotland eh14 4as email: nine@cyberservioes.com abstract when approaching electronic capital market design and microstructure with the focus of analysing and improving existing markets with end-state analysis, it is necessary to name an ideal objective. this serves the purposes of technology evaluation and the development of a standard framework for structural measurement in modeling and language paradigm design. an ideal capital market architecture is presented in this paper that is feasible with current technology based on the end-to-end functionality of existing capital markets including internal requirements of participants. various architectural and ethical issues are introduced and discussed sketching a framework for further work in quantifying electronic markets. introduction electronic capital markets have been developing since the systematic, commercial use of high-speed computers began in the 1970's. with the end of the cold war and the spread of global capitalism, nations throughout the world are attempting to install and develop capital markets. in the engineering of a capital market software system, the marketplace microstructure codified into the software is reflected in the real world in terms of the efficiency of the market and in implicit advantages for certain market participants (cohen, 1986). to evaluate these influences an end-state model is proposed for objectives and ideals with which technologies can be described and measured for their suitability of purpose and quality (prindl, 1994). this process involves sifting through the entire set of requirements and perceived values currently provided by all stakeholders in a capital market (fabozzi, 1996). a simplified analysis is presented as a root model. an electronic capital market is a virtual version of a physical market. there are goods for sale, buyers, sellers, the auction process and agents of the market. the primary object moving in and out of the market is cash. unlike other marketplaces, the goods for sale are delivered virtually and converted to cash only if the entity wishes to take them from the market with the exception of certain contracts where delivery of physicals is part of the contract. every dealing party is concerned with their inventory of goods, bringing goods to market, the auction market for the goods, qualities of the goods themselves and their membership status and accounts. because no entity in the marketplace keeps consolidated information as a service, each entity keeps its own record of the portion of data that is relevant to its own needs and thus spends enormous sums on information technology to maintain what is already recorded elsewhere in the marketplace. over years, each entity's interests with this data has lead to specialised representations to each user community on a need-to-automate basis (reddy, 1995)(pardo, 1992)(o'leary, 1997). this has lead to databases throughout the industry keeping a patchwork of data, none of which is an absolute for a marketplace. these factors have spearheaded a specific demand from computer science in the development of artificial intelligence software ontology's for the representation of a whole trading market. in effect, the knowledge demands of the various dealing parties are optimally represented in an ideal virtual market ignoring the locality of the underlying corporate entities. the ideal market model presented is designed to provide the optimum marginal benefit for all market participants. this ideal is considered to be a multi-image tightly distributed simultaneous^ global capital 12 a simultaneous market actually exchanges value at the moment of trading. there is no settlement delay as all payments and transfers happen at the point of exchange. in order to have a simultaneous market; the depository and other aspects of the "processing" of today's trade are the execution process itself. a simultaneous market is the ultimate evolution of the current settlement system. it also greatly simplifies issues in modem computerized finance to dispense with the delay between front office actions and actual accounts. for these reasons, an ideal market is a simultaneous market. 70 ajis special edition electronic commerce november 1998 market on a high bandwidth secure network with images in every sovereign constituency. the model is first discussed in terms of its selection and is then broken down into a top-level software paradigm that could be constructed using a variety of underlying physical hardware and software sub-systems such as supercomputers and databases (example in appendix a). this is composed of a market engine, required media services and network services to support such an ideal global market. the market knowledge required to support these services is presented along with workflow ontology to bring together the various electronic market functions into a discrete market "instruction". the further ontological development of each portion of the market is outlined including its requirements as a basis for ongoing work. this paper can be expected to be of interest to the is community helping to formalise the joint discipline of electronic markets as a field involving finance, accountancy, economics, law, ethics, telecommunications, computer science and industry current practices. this broad knowledge scope combined with the vertical structure of university departments and disciplines has restricted research in this area. further research will yield models of optimal economic markets capable of testing various market structures for advantage as well as models for re-engineering the business in marketplace media, capital markets themselves and in financial organisations. in the macroeconomic sense, it is possible to introduce a capital markets efficiency and effectiveness multiplier that can geometrically stimulate economic output though further innovation along the lines of the technologies described in this paper. within the software domain, design parameters can vary quite widely to accommodate an optimal representation. the following conceptual model (figure 1) is presented in simplicity to keep the paper short and to facilitate comprehension of a broad domain of knowledge. were a further notch of real-market complexity introduced, this paper would develop into an extremely complex software design. postulates & assertions there can be significant debate among economists on what qualities represent an ideal market. the following 18 postulates are underlying assumptions underlying subsequent arguments. 1. an ideal market maximises the marginal benefit of each entity connected to the marketplace and it is in their best interests to participate in the marketplace even if a dealing party has an absolute advantage in the production of goods (smith, 1776). 2. ideal market governance is representative democracy of its member constituents and operated under no single sovereign, thus no agency motivation from the directors. ̂ seller buyer o x y z \ \ \ 10,000 us$ \ 1000 xyz "i 5,000 sell 1000 xyz € uss "i 5 uss j market state t(n)sell 2000 xyz @ 5 us$ buy 1000 xyz ® market received 5000 uss received: 1000 xyz transaction transferred: 1000 xyz transferred: 5000 uss 1000 xyz \ \ \ 5,000 uss \ market state figure 1 . note how a simultaneous market is always in a state of equilibrium. when a matching event occurs, the market changes state and again resumes a state of equilibrium. 1 3 the definition of directorship is limited to reasonable representation or accepted asymmetry by capitalization. below a certain capitalization threshold, there is no marginal benefit for participation in a capital market and thus no representation in the directorship. 71 ajis special edition electronic commerce november 1998 3. an ideal capital market has minimal barriers to entry. this includes the cost of the connection to the marketplace and the knowledge and comprehensibility of the market structure which is a more likely realworld barrier-to-entry. efficiency of system 5. replication coat it 2 number of market images 4. (assertion) one single large distributed system is the ideal market. replicated systems wilh heterogeneous software environments introduce inefficiency to an ideal market. the replication cost is perpetuated due to: a) the replication delay in maintaining multiple equivalent data images and the additional network traffic of that replication. b) human error factor forcing systems to reconcile through manual processes. c) market technology evolutions involve risk and the nature of finance is risk-adverse. d) conflict-of-interest's in the is industry as the business model for profitability in software is based on quantity of installations or licenses sold, not efficiency of end-application function. e) successful standards come from small groups or individuals largely due to the intensity of knowledge that must be brought together to innovate new standards (kahin, 1995). (e.g. unix, c, c++) an ideal capital market is completely secure through the intelligent use of encryption, secure physical access, and policing (marinelli, 1991). 6. clean-slate consideration any knowledge systems innovated prior to the development of global networks and modern supercomputing are accepted in terms of their essential discipline and not in terms of the methodologies themselves. this is especially true of procedural computer science, but also in accounting (kam, 1986), law (susskind, 1996), economics, current business process (reddy 1995) and finance (arbnor, 1997). 7. there is one time zone for a simultaneous market. a global market is operating across time zones. current telephone networks have the same consideration and use "zulu time" as a geocentric time (bellcore 1984). 8. an ideal market does not have an asymmetry in favour of any one entity. many existing capital markets have asymmetries in favour of the well-capitalised dealers and customers, sovereign nations or exchanges themselves. though it is the standard practice, asymmetries only weaken a market and for the body of the paper make the assumption that neutral asymmetries improve the climate for frictionless exchange (prindl, 1994)(cohen, 1986). 9. markets are always open. the open/close times of regional markets are related to the human resources involved in operating the market. a global frictionless computerized market has no such concerns. it is also to the benefit of all connected parties if news expectations are simultaneously realised in the market rather than forcing up volatility expectations for listed instruments during that time that markets are closed. 10. an ideal capital market is instantaneously replicated to multiple secure physical sites to avert concerns over risks of catastrophe and sovereign control. for a quantifiable replication cost in terms of system complexity, multiple images of a complex system can be maintained (tandem, 1989). 11. a capital market's software does not make mistakes. though not the case over the past 25 years, it is possible to create highly complex systems that are error-free for their function as they are tested so well. as of today, these include space vehicle computers, the telephone network in western nations, bank atm machines and military systems. 72 ajis special edition electronic commerce november 1998 12. an ideal market facilitates the development and fair pricing of financial products by making them simpler to bring to market and easier to disclose and measure. 13. an ideal market is simultaneous and has no settlement risk or contraparty credit risk as all transactions clear before acknowledgment of execution. 14. an ideal market has minimum execution risk. every participant may enter the market with confidence that they will get the best price from the ideal market. this is a function of liquidity and the design of the market auction process (optimark, 1998). 15. an ideal market is not taxed internally, deriving all internal operating revenues from connection charges. this "commission" issue is a major source of complexity in existing market systems (reddy, 1995). 16. an ideal capital market supports the interests of both clever participants and innocent participants, loosely defined in the current marketplace as sell-side and buy-side (prindl 1994)(boyd, 1997). 17. an ideal capital market has a user-interface that has no learning curve and that allows each user to execute their business efficiently and effectively (barber, 1997). 18. human interests are best served by a government that abstains from interference in free enterprise, putting checks only on undue strife and competition (smith, 1776). simultaneous frictionless market financial theory uses the concept of a "frictionless market", walrasian auction, when considering ideal market valuation across changes in supply and demand. the following section describes a frictionless market in terms for a computer model of an "ideal" in modeling and language design. the market described here is technologically feasible today and is derived in function from today's capital markets. as the model argument is developed, it departs from the physical model of the market into an abstract world where a transaction can take place simultaneously. an electronic financial market is always in equilibrium. value is never in transit; it is always accounted for. (kam, 1986) such a world is represented differently. the paper refers to this "different" representation as a market state of knowledge. a simultaneous market is the official accounting record of ownership from moment to moment as events change the market. stakeholder scope the classic definition of a frictionless market includes its end users, the marketplace participants. an electronic market has a less visible set of stakeholders with widely varying requirements that must be considered for an ideal capital market architecture (freixas, 1997). these stakeholders are: 1. financial users 2. system operators 3. network administration. 4. backup/recovery. 5. legal 6. accounting 7. taxation agencies 8. software development & test (upgrade migration) abstract meta-parameters market design has a broad range of parameters that become implicit within the capital market architecture. the following parameters are thematic influences that must considered either deliberately or endemically in the architecture definition. 1. aggregation the unitary level at which every object interacting in the market is recognised. a) should the market keep record of individuals or just corporations. b) should it be easy or difficult to aggregate cash flows and instruments to create new products? 2. complexity what level of complexity should be presented within the market? a) the user interface be so complex that only highly trained experts can use it. 73 ajis special edition electronic commerce november 1998 b) should the market architecture be intuitively obvious to any educated person? 3. national influence the relationship between the sovereign and the market a) the market is an extension of a sovereign legal and political system. b) the market is country-independent and is not directly influenced by sovereign entities. 4. liquidity the market can be designed to increase or diminish participation. a) should every component of a market be fit for end-user facility? b) should the order-match process incorporate specialist-type liquidity insurance? 5. capitalisation what is the effect of capital leverage on the fairness of the market? a) none. each participant has the same transaction charges for participation removing all non-capital related barriers-to-entry for arbitrage. b) very high. firms with more money should be able to purchase reasonable advantage in a marketplace with their leverage. c) cronyism. capital leverage can buy leverage depending on its relationships. 6. governance what is the fairest way to control a market and its legitimacy? a) virtual nation the capital market is its own country with participants as citizens b) hegemony one country runs the market for others c) plutocracy one vote for one unit of capital leverage d) democracy each human being affected by a capital market has a vote in the market e) computocracy the electronic capital market software is self-governing. 7. privacy what is the trade-off between the marketplace interest and participant liberty? a) no market-internal information is ever disclosed (mill, 1859). b) only reasonable disclosures to the public with total disclosures made to surveillance c) all market information is public to all members capital market knowledge domains transactions in a market currently take days to propagate from exchange computers to the other systems in the broker-dealer chain that make up the current capital markets from initiation of deal to settlement. (reddy, 1995) this data is marketplace domain knowledge and the delay is caused by antiquated computer and business infrastructure. (jang, 1998) in a simultaneous architecture, representing all data in a single environment sustaining all requisite knowledge functions eliminates any propagation delays. capital market knowledge domains administration membership sec dtb treasury iscc telephony & networks media expert systems los alamos national laboratories financial operations sybase figure 2. companies with their market segments shown in reference to the knowledge areas in commercial capital markets. the dotted line encloses the scope of this frictionless market. the number of corporate barriers that are crossed to form a capital market is so large, that different parts have developed different ontologies for the same market data representation. in automation terms, they all process the same information, but break it up and view it differently. 74 aj1s special edition electronic commerce november 1998 reference frame each entity involved in the market uses a common, underlying reference frame of knowledge (illustrated in figure 3) to participate in that market. in this reference frame, there are certain archetypal bodies of knowledge that have standard interrelated properties. (xticket, 1995) to computerise such a market, the systems representation must accurately model the market-state knowledge in each domain. below is an illustration of how these domains are represented in the software architecture of a capital market. t(h) financial. horizon instruments t(0) current moment current reference f rame and active state of the market figure 3. the current frame of the market is a joint representation of the state of accounts, members, instruments, markets and positions all interlocked in a state of exchange. the net-market position extends to a future time horizon beyond which no products are leveraged. interrelationship the "market x account x instrument x participant = position" equation of the reference frame is the core abstract structure of a capital market. by keeping perfect reference-data integrity within such a logical marketplace, reconciliation and the repetitive back-office operations of the past can be eliminated. (om gruppen ab, 1996, nicolle, 1998) the ultimate critical result of isolating the market equation within the boundaries of pure-data market, however, is that the market can be safely driven by a discrete set of end-user instructions without risk of corruption or failure. snapshot market images the market is referenced in either its current state, past states and by changes between states. figure 3 illustrates only the current state. any number of market images and derivative data can be maintained and generated from the market transaction log for reporting and accounting purposes. marketplace objects a marketplace is a body of expert system knowledge that participants must interact with to obtain their objectives. based on this definition, the knowledge bodies that compose a marketplace are visible in figure 4. each of these "objects" has an active state representing the current moment and the capability of showing historical states and the transaction logs. all messages that do not actually change the state of the market, are parts of what this paper calls marketplace media services. events such as "checking an account balance" or 75 ajis special edition • electronic commerce november 1998 "checking the p&l in account a-l" do not change the market and thus are not market commands, underlying objects in the diagram are each described in a subsequent section. the mode-1 market commands secure domain (perfect data) market 'instruction cache & engine^ (service control point) marketplace query & multicast media services figure 4. the box represents the main intelligence engine of a simultaneous market. media services provide channels for global distribution and query regarding all aspects of the market. market commands are actual instructions to buy, sell, open accounts or other actions that cause the market to change. the existing financial services industry operates similarly per the diagram except the workflow processes are implicit in the physical computer-to-computer connections between systems and in the manual processes of the marketplace. membership every connection to a market is defined as "membership". one broker dealer could have 15000 connections. the process of connecting to a capital market begins with membership. although there are significant aspects of membership that involve non-market aspects of finance (e.g. human resources), they do not affect the electronic market internals. for an efficient electronic market, membership is considered similarly to telephony standard practices of the "service order". (wolfe, 1988) (d'sylva, 1991). the services that each member has access to, their billing status for marketplace services and all related support information are kept in the object. easily a large and highly complex database, the object nevertheless contains only membership information for electronic processing purposes. there are types of market commands and hence "messages" that affect membership: a) query member inquire membership information, permissions, limits, etc. b) new member add a new market connection c) change member change services, address, permissions, etc. d) delete member disable member connection e) timer period based membership functions 76 ajis special edition • electronic commerce november 1998 membership also includes the individuals, corporate entities and non-entities that access the market on behalf of an institution. this membership object also contains the security and permissioning for the market. instruments the financial products traded in the marketplace are kept in a depository and virtually transferred by their entry in the instrument database object. this object represents all knowledge about the products including special issuance criteria and a market standard valuation algebra for derivative products. instruments issue cash flows to their owners. such instructions are driven by the instrument object automatically by a timer as in the case of a bond coupon payment. in the case of variable payment such as a dividend or a call on an issue of callable bonds determined by sources external to the marketplace, the instrument command is issued like an order as a marketplace instruction. instruments each have an issuing party that is responsible for such instructions. instrument messages: a) query instrument inquire regarding the instrument b) new instrument the instrument has passed physical criteria for depository and is being formally introduced c) change instrument call bonds, pay dividend (notifies marketplace media of relevant instrument change). d) delete instrument all positions must be flat and instrument is disabled e) timer this marketplace command is issued by the instrument object timer itself for coupon payments and other internal "built-in" payments. in order to trade an instrument unambiguously, a market transaction must have accurate instrument reference data. the instrument object must also be capable of representing newly engineered off balance sheet products without which, an ideal market could not exist. it is my position that any instrument including its valuation criteria can be represented using a quantum algebra involving expert system rules, and a "molecule" from a set of quantum finance "particles" including: time, probability, and value. in order to list the instrument, the financial dealer would have to use a published legal financial standard algebra and program the instrument. accounts an account is a logical grouping known in the marketplace to hold a position. each member will have many accounts and control the respective permissions for access, limits and transfer rules. for managed accounts, allocations are a method within today's markets for electronically managing account inventories (flx, 1998). the ownership of all the instruments on the market is distributed between the accounts, but not kept in the account object as that is a position, not an account. for these purposes, cash is considered a "position in a currency instrument" rather than a "balance in an account". much of the media services reporting uses accounts as a driver to produce position and activity reports. since all member inventory in the market is held in accounts, accounts are implicit in every market transfer transaction. account messages: a) new account create new account for member b) change account address, reporting-currency, permissions, allocation scheme, etc. c) delete account verify that account has no position and retire account position a position is financial leverage in the market. in practice, it is the marketplace inventory of financial instruments, (e.g. 2 yen, 4 dollars, 3 bonds, 1 digital option) the position is the current quantity of instruments held and the related transactions outstanding, including the purchase price(s), that created the position, but not the current market price. a buy order for ibm at 90 dollars per share with a time-in-force of 1 month from yesterday is also a position. like an option, an order is a leveraged market commitment and is treated as such. all attempts to measure the value of marketplace holdings are part of assessing a position and thus incorporate all p&l and time-based profitability & risk media as requisite functionality of the market position-media services. position messages: a) buy/sell/cancelreplace/exercise/cancel change position in the market b) allocate, bot/sld off-market c) deposit/withdrawal import/export cash from the market market 77 ajis special edition • electronic commerce november 1998 a number of buyers currently wants to buy. a number of sellers currently wants to sell of the same instrument. the market in an instrument is inseparable from the instrument itself as the bid-offer pool represents supply and demand for the instrument and related accounting values of corresponding positions. order match including market transfer of ownership in goods takes place when the conditions of the seller's offer agree logically with the conditions a corresponding buyer's offer. (mckendrick, 1990) conditions of entry, status and interconnection exist in this market space that the order match takes place in a [random ordered] way. (bernstein, 1995) if it is purely random, patterns of matched orders vs the mean of the current pool of outstanding orders with the same order match criteria will show randomness. if it is ordered, then the market has asymmetries which are the focal point of marketplace design. a market is an expert system with the current frame as all buy and sell orders for a particular instrument. when a new fact is asserted (a new order), the system evaluates the current orders to see if there is a match. if so, then order match has taken place and the object issues a market command to transfer. if not, then the order is added to the current frame. market messages: a) suspend/resume market surveillance action stops all order match in an instrument. (debatably, this is a sovereign asymmetry and should not be considered as root functionality.) b) timer order xx has reached its time-in-force without executing, delete. a flexible systems architecture would support multiple interchangeable auction processes as part of the market object. the result would be that the market object is sophisticated and can support real requirements rather than enforcing any one set of auction market rules. period of offer current markets use a time-in-force (tip) in thenorder-match systems (nyse, 1998). this is the life of an order in the market. this property gives a limit order the quality of an option contract. from a software point of view, there is no difference between a financial instrument and an order, leading one to consolidate non-oms14 details regarding orders into the instrument metabase rather than the market metabase. all-or-none (aon) some exchanges offer an order attribute that means an order is never broken up. (rosenburg, 1987) if you are a small dealer, you may not be able to match the order quantity of a large order. thus, the best market may not be available depending on capitalisation. without the rule, an order for sale of 1000 at 40 per share would sell 500 to a buyer of 500 at 40 per share. such a rule is a market asymmetry albeit a small one on behalf of larger capital institutions. this type of rule allows markets to cross (where the best bid is higher than the best offer). in a newer order match system such as optimark (optimark,1998), crude parameters like aon are replaced by urgency and price-sensitivity indicators clearly advancing along the line of allowing an end-user to express their strategy effectively and efficiently to the market. nevertheless, the result is a market message described by quantitative parameters to be processed by a market order-match. market entry triggers current order-match environments offer the concept of an order that only becomes active contingent on an event within the market such as the last-sale price. the design of these trigger rules in an order-match environment, combined with the block sizing and order-breakup rules of the market are pivotal to the efficiency of the ordermatch system itself. since orders "flow" at a certain rate of "traded-volume/second" (cme, 1997), a measurable standard flow moves in a market. to place orders into the flow without disturbing its price or getting a fair price in the flow are objectives to the end user. administration often implicit in the human business processes related to a market, the electronic market, network and software have innumerable issues that impact the state of the market. these include operations, billing, utilisation auditing, startup, shutdown, performance tuning, surveillance, servicing member requests and other actions regarding the marketplace that ultimately affect the market state (kse, 1996). under some circumstances, these activities can change the state of the market. a) shutdown market b) start market 14 order match system 78 ajis special edition electronic commerce november 1998 c) suspend trading in <xyz> workflow every "market command" is a workflow template enforcing a chain process through a synchronised sequence. these sequences are parallel with one thread for each instrument. the workflow bus walks the transaction through its phases very similar to the sub-processor micro-code instructions that make up a cisc 15 processor (sun microsystems, 1993). each template is composed of individual messages to market objects that the workflow subsystem enforces to either be executed in order, concurrently or a combination. mn=(current financial state of the marketplace in instrument n) g = (state of world market) = mms™,,.̂ . . minstnimem.n e = workflow message event w = workflow template = cq . . en mn = marketframe{n, accounts, membership, instrument, market, position, administration) w = eo(accounts, membership, instrument, market, position, administration) .. en(accounts, membership, instrument, market, position, administration) a workflow template is a type of service administrated in the administration object as the marketplace instruction template for a transaction. example marketplace transaction: abc company of south africa entered the market (after having registered and established a connection prior) to buy 100,000 shares of xyz at 51 privately negotiated market", this indication of interest was routed by abc preference, to two dealers, one, cayman islands ltd., and the other, frankfurt mainz ag both seeing abc's need, each with significant holdings in xyz, they each reply "sell 100,000 xyz but the cayman island dealer places a limit of 51.15 on his sell and the frankfurt dealer's message matches with 5,100,000. post-match media services are contacted of the transfer, the abc company has been debited 5,100,000 the frankfurt dealer pulls up a statement showing his reduced balance of 100,000 shares of xyz and a payment of 5,100,000. the depository record in instrument now shows abc company as 2.5% owners in xyz. membership abc company abc company abc company abc company cayman island cayman island frankfurt m. frankfurt m. frankfurt m. xyz company position media snapshot market account media subscribe media snapshot indication buy 100000@51 xyz implicit implicit order buy 100000@51 xyz implicit implicit xfer 5100k & receive 100k xyz order match implicit media snapshot media subscribe media snapshot expire position media snapshot cancelled -outbid implicit media subscribe media snapshot order sell >100000 shares of xyzimplicit xfer 100 xyz & receive 5100k order match implicit media snapshot instrument admin media snapshot verify user & log implicit log implicit log implicit log media snapshot verify user & log implicit log media snapshot verify user & log implicit implicit ownership change log log log marketplace media services the marketplace media support all requests for knowledge in the marketplace that do not involve changing the marketplace itself. these include historical price activity, position value-at-risk, year-to-date p&l, accounts statements, quantitative views of derivative positions and the vast majority of the data on a current trading desk. market media requests fall into two categories, snapshots and subscriptions. a snapshot is a single view of a market data image without updates for future changes even though it is on the screen. all paper reports in the 15 complex instruction set 79 aj1s special edition electronic commerce november 1998 financial industry today are snapshots. subscriptions are continuously updating images of the market in a window on the user screen. as neither of these requests involves access to the central workflow bus, a media image can be generated from an integrated parallel reporting sub-system diagrammed in figure 3 as "media services". snapshot media are similar to that supported current database access methods (sybase, 1998). subscription media are more complex due to the problem of simultaneous delivery of changes. if an order is matched, all subscriptions whose scope includes that portion of the market must be concurrently updated as discontinuous updates allow unfair advantage. this fan-out problem exists in today's markets, and is managed with high speed networking and intelligent design (nyse, 1987). the bid-offer pool, a media design issue it is common practice for an exchange to quote the market as best bid & offer price*size. a participant is inquiring into the media to find out what is the state of the market. should i participate? how deep is the market? how fast can i execute this order without moving the market? how volatile is the instrument? a participant comes to the marketplace media with these questions. if the media describing the bid-offer pool is misleading, this benefits the "clever" participant over the innocent participant (prindl, 1994). depending on a media designer's ethical position, this microstructure asymmetry can deliberately or unconsciously influence the overall market's effectiveness (shenk, 1997)(mcluhan, 1989). marketplace media gives market participants the sensation of "being there" in the market and allows them to see and feel the state of the market. the effectiveness of the media interface is largely dependent on how well it uses currently-available technology to communicate the essence of the knowledge a participant needs to achieve their objectives. there is an argument for complete secrecy in a market to disguise large pension rebalancings and the dumping of blocks into the market. entering the market with a large block can be difficult without secrecy as competitive bidders benefit from the disclosure of the strategy that one is forced to sell (straffin, 1993). if real-time data costs drop regarding a market, does this reduced threshold increase participation? if the cost of participation is the ability to interpret and understand the market, does providing lucid media regarding the potential execution prices draw or diminish interests on willing participants to trade in the market? the issues presented are suggested merely to promote the ethical argument over whether such design should be completely opaque to the inquiring economist or studied and modelled. profit & loss when marking a position, there is some consideration as to the time-value of the marking if the market liquidity will not support that value within that timeframe. consider the example of the firm with 200,000 corporate bonds seeking a marking price from the market to assess the value of the holding. the market for the bonds would fetch a price of 88 if sold over 1 week and a price of 93 if sold over 3 weeks due to market liquidity. p&l mark-to-market^ pricing media should support mark-to-depth sensitivity as part of the incorporated p&l media services to afford the users of an ideal market a realistic standard for marking assets. (wurtz, 1998) the complexity of this area can be awe-inspiring. risk and profitability media represent the cutting edge of financial engineering design and thus are unlikely to be easily incorporated into an open framework without significant forethought. the intent to include risk and volatility standards as services of the market opens up the ethical question of whether it is in the stakeholder's best interest to have these things be vague and un-standardized (prindl, 1994) example media queries: • inquire history of account • view position • view position changes • view p&l marked to 1-month liquidation • view transactions • view market in gbp/usd • view cash flow p&l's, duration & convexity • view value-at-risk for account • view quantitative valuation modelling • duplicate market 16 using the market assessment as the "price" and book value of the position 80 aj1s special edition electronic commerce november 1998 capital markets efficiency and effectiveness multiplier current macroeconomic theory is heavily entwined with the concept of free, relatively-frictionless capital markets. a simultaneous market yields a higher efficiency in the conversion of savings to investment (s-i) in macroeconomic modeling by reducing the transaction premium of the marketplace through reduced risk, reduced transaction cost and better information flow. efficiency is improved by pure throughput of the capital market through the combination of more participants and more products introduced to each other on a vastly more efficient commerce medium than is in use today. effectiveness would also be improved by better information flow and reduced short-term volatility leading to more intelligent expenditure and return on funds. this information flow also reduces the potential for hyperreal finance (mcgoun, 1997) by reducing the opportunity for agency interests within the marketplace itself. such considerations include the potential to perform macroeconomic adjustments through the introduction of simultaneous electronic markets. asymmetries & considerations following are discussed several factors contributing to market asymmetries and some considerations in the tradeoff between frictionless objectives and current standard practices. exchanges there was a celebration in the siac (nyse's automation company) during one of my research interviews after the close of the largest equity market crash since 1987, october 13, 1989. it was explained on inquiry that the celebration was due to the record high transaction volume in the electronic systems during the market downturn and that that would increase the salary bonuses. the order-match system designers in this exchange have a motivation to design a market that pays their company well and that sustains their jobs building more systems that work within the exchange framework. they do not have a motivation to increase the wealth of the stakeholders in the market through efficient software microstructure design, nor do they have the liberty. they also have a strong corporate motivation to over-sell the complexity of market system design to guard against potential entrants to their business. sovereign the question in marketplace architecture is should sovereign nations be treated like any other participant, selling and buying bonds for currency. other matters regarding sovereign control are not electronic such as public disclosure, adherence to the rule of law, the privacy of physical records, and member policing. accounting in terms of a financial market, all public accounting standards match a transaction log to the public record. they are completely important in describing assets through standards that are used in the public media of the market. in terms of our efficient market, accounting standards are a type of media view on assets. the accounting discipline is thus split in usage in this perfect market accounting as a matter of public record (financial accounting) and accounting as a discipline of measuring real value. the former of the account domain and the latter part of the position domain. legal issues in a simultaneous market, contract law is implicit in the market thus the technology of a simultaneous market is (under perfect conditions) law in itself legitimately transferring real ownership of goods and services in realtime. parties can conflict and resolve using transactions, but transactions are never backed out. though this seems trivial, it is an important assumption omitted in earlier work. the legal dominance of a sovereign entity over a marketplace is an implicit asymmetry toward the sovereign state. the market-domicile sovereign has the power to change the rule of law governing products owned and represented by parties foreign to the state. since a real-time depository is considered in the ideal model, the ontology's described herein are legally binding contractual terms within the market. if legal challenges are allowed to tamper with past transactions in a depository marketplace, then all dealing parties will have to hedge the risk of market failure from legal action, undermining the legitimacy of a unchallenged depository market-record. 81 ajis special edition electronic commerce november 1998 backing out transactions in current operations, it is common for an account number to be mis-keyed by a clerk keying in a trader's ticket or for a ticket to be misplaced. carry calculations and end-of-period dividend and tax calculations all involve having a proper transaction record at the time of calculation. when a transaction is applied "as-of' it requires the recomputation of the entire market subsequent to the transaction as a missed transaction changes fifo p&l matching, carry-cost calculations, depository transfers and such. missed or incorrectly entered transactions^ currently justify days of delay in proper bookkeeping if we keep with the premise of "perfect fit-to-function and failure-free" market software with user-impervious end screens, invalid keying can be dropped as a manualsystem side-effect. since a very high percentage of incorrect or lost transactions are related to human involvement in the process in the first place, the ideal market guarantees internal integrity by having the endcustomer enter a transaction once, correctly, with no subsequent human involvement. the valid transaction is then part of a forward time rolling legal depository. netting, stock borrow/loan, margins and commissions each of these areas of real-world markets usually involves a separate software environment and special treatment. netting does not exist in a simultaneous market as each transaction clears individually. stock loan, margins and commissions are administrative sub-components of market transactions. as each component transaction is a market position in its own right, no separate consideration is given these as they can be effectively included within the architecture given by simply extending the workflow definition of a transaction. references arbnor, ingeman; bjerke, bjom (1997) methodology for creating business knowledge, sage publications, london uk barber, rick (1997) torex™ trading station derivative trading functionality, toronto stock exchange, toronto, ontario canada bellcore 1981-1989 ccitt #7 (signal switching protocol #7) protocol message & state design bellcore publications, new jersey usa bernstein, richard (1995) portfolio trading guide, part 1 adding value using basket trading methods merrill lynch & co., global securities research & economics group, quantitative & equity derivatives research department, new york city, usa boyd, i; smith, b. (1997, april) capital market imperfections, international credit markets, and nonconvergence journal of economic theory, vol. 73, no. 2, pp. 335-364 cohen k., maier et. al. (1986) the microstructure of securities markets new jersey: prentice-hall cme chicago mercantile exchange; cbot chicago board of trade (1997) order routing api developer guide cme & cbot chicago usa d'sylva, ray (1991) "artificial intelligence/gui for dynamic service creation under ss7" nynex ai technology laboratories, white plains, new york fabozzi, frank; modigliani, franco (1996) capital markets institutions and instruments; second edition prentice hall, new jersey usa ftx financial information exchange (1998) www.fixprotocol.org freixas, xavier; rochet, jean-charles (1997) microeconomics of banking mit press, cambridge, mass. usa jang, jeff (1998) scotiamcleod equity trading platform (tables) scotiabank internal, toronto ontario canada kahin, brian; abbate, janet (1995) standards policy for information infrastructure the mit press cambridge massachusetts usa pp. 378-615 kam, vernon (1990) accounting theory second edition, john wiley & sons, new york usa pp. 8-30, 149152 khalil, d. (1997) universal financial transaction protocol discrete arbitrage nine llc, new york usa 17 human errors including switching off modems, power failures, programmer error, human clerk key entry error, incorrect receipt of clearing ticket, incorrect booking of error transaction by staff are all common errors happening every day in the industry each slowing down the pipeline of whatever marketplace transaction is in queue for system services. end-to-end media integration limits run-time errors to the market participants themselves by eliminating the source of the majority of errors, disparate, complex systems based on the accounting and international trade market of the cold war. 82 ajis special edition electronic commerce november 1998 kse korea stock exchange (1996) outline of korean stock price index futures to be introduced in 1996 korea stock exchange, stock index futures market organising committee, seoul, korea marinelli, j (1991) bess system: swift security access architecture westpac bank, new york city usa mill, john stuart (1859) on liberty hackett publishing company, inc. indianapolis, indiana usa mit (1992) x window protocol definition the x consortium mtt, cambridge, mass. usa mcgoun e.g. (1997, feb.) hyperreal finance, critical perspectives on accounting, vol. 8, no. vz, pp. 97-122 mckendrick, ewan (1990) contract law macmillan press ltd., london, england mcluhan, marshall; powers, bruce r. (1989) the global village, transformations in world life and media in the 21st century oxford university press, london uk nicolle, t. (1998) business plan for a typical citadel client, demica ltd. london, england nyse (1987) mds market data systems, test plan for weekend testing siac, 55 water street, new york usa nyse (1998) common message switch (cms) api programmers guide, siac corp. 55 water street, new york, ny 10005 usa o'leary, k & facih, odile, (1997) equity trading system request for proposal, nikko europe pic., london uk v 1.0 om technology ab (1996) the omnet application programmer's interface manual stockholm, sweden optimark inc. (1998) www.optimark.com pardo, robert (1992) design, testing, and optimisation of trading systems john wiley & sons inc. new york usa prindl, andreas r.; prodham, bimal (1994) ethical conflicts in finance blackwell, uk reddy, michael t. (1995) securities operations; second edition new york institute of finance press. rosenburg, ellen (1987) multibond system design securities industry automation corporation, metrotech center, brooklyn usa shenk, david (1997) data smog, surviving the information glut harpercollins, new york usa smith, adam (1776) the wealth of nations prometheus books, new york usa pp. 377 straffin, phillip d. (1993) game theory & strategy the mathematical association of america, washington dc, usa sun microsystems (1993) sunship, queue management and workflow design sun microsystems computer corporation, milpitas, california usa susskind, richard (1996) the future of law; facing the challenges of information technology, clarendon press, oxford uk sprint (1988) ttsi dsp-ms accountcode interface specification us sprint development laboratories, burlingame, california usa sybase inc. (1994) open clienttm and open servertm common libraries reference manual emeryville, california usa tandem computers (1989) tns-ii interprocessor bus and the non-stop architecture, tandem computers inc., cuptertino california usa wolfe, nancy (1988) e911-ms, error processing for service order parsing failures pacific bell telephone corp., san ramon, california usa wurtz, charles (1998) "marking for depth", veristics inc., new york usahttp://www.optimark.com/ xticket systems inc. (1995) documentation of the physical data model, position database (xticket internal) manhattan, new york usa acknowledgements this paper is dedicated to the memory of dr. frederick lenz. so much depends on the red wheelbarrow in the rain next to the white chickens. this research has supported by the following: * sybase inc., london & toronto 9 petra kloppenburg, edinburgh * the royal bank of canada, toronto » dr. charles wurtz, new york » scotia capital markets, toronto « reuters america, new york 83 aj1s special edition electronic commerce november 1998 appendix a composition of media each market participant engages in their connection(s) with the electronic marketplace via network media services. from the point of view of the digital market, all participants are merely electronic messages requesting services that it subsequently provides. although the actual substance of the media can vary widely depending on the participant and their interactions with the marketplace, the types of media modalities are few. a physical network of adequate bandwidth to support multiple video, audio & digital data multicast and point-to-point communications is required. electronic market engine & media switch bandwidth marketplace media mode-1 transaction mode-3 (video conference) mode-2 media subscription (market of ibm access device access device access device figure 5. shown are the three modes of media services within this electronic market. the diagram parts can be related to the ccitt#7 service control point, signal transfer point and switch with a high bandwidth local loop. access devices the device that a user employs to engage a market (via its media services) incorporates bi-directional audio (telephone, orders), video (television, video-conference), text (news and interactive services) an x-window (mit, 1994) paradigm of end-user windows each driven by a media stream is adequate. the technology of switching media between windows is well researched in the operation of the current public internet. switching services the marketplace must provide a mechanism for the network to dynamically change itself as membership changes and to engage any media modalities with any access point in the network. this switching service is supported and made available via a command center that is instructed by software to engage various media between various points from instructions in a dynamic database/expert system. this paradigm is better supported by an extension of the (bellcore, 1988) signalling system #7 protocol18 (sprint, 1988) than the current web technology due to die separation of the signalling network from the "voice" or data network within die telephone network and protocol-integrated load balancing. also because signalling information shares bandwidth with data on the internet and because no reliable point-to-point services can be ensured, die medium is a weak standard for this virtual marketplace. 18 the functionality described in this paper is programmable using current commercial technology between a hybrid multimedia telephony switch, an upgraded signal transfer point, a supercomputer service control point and a high-bandwidth global network. the entire switching engine would reside in the service control point operating a global market. the service control point is itself a virtual network intelligence engine simultaneously replicated to other sites via an "inter-processor bus" (tandem, 1996). 84 ajis special edition electronic commerce november 1998 the telephony paradigm is also selected to overcome a difficult hurdle in current financial operations and internetworking, weak transaction reference data. when using a current open connectivity standard for financial interconnection such as (fix, 1998)1", me homogeneity of the reference data cannot be guaranteed and is accepted as weak. when interconnecting with 10 dealers, the problem is manageable. when scaled to 10,000 dealers, the problem is unmanageable and the standard breaks down. in telephony, all connections use the internal network databases, and system services are both hardware and application fault-tolerant. it is then possible for the telephone company to guarantee the quality of their services due to this closed architecture and the same must be true in an ideal market. modality any communications link in the market is via one of three types of modalities. they each represent three fundamentally different communication services required in a virtual market. this paper really only is concerned with the first mode as switching and transmission technologies for the latter two are in use today. however, it is important to recognise the latter two as they make up the vast majority of marketplace services visible today with the first mode an opaque process within the dealing community. 1. multi or single part discrete transaction. this channel requires guaranteed message delivery in a point-topoint fashion. the transaction is always initiated by the end user access device. it is composed of discrete packets of information that dialog between the source and the destination, (e.g. "what happened in account 123 last month") 2. subscription. the channel requires information to be subscribed to until a request is made to discontinue the subscription. the data are delivered as a packetized stream of whatever media composition is requested, (show the active ibm open order book, watch cnn television) (1998) reuters ssl documentation 3. multi or single message full-duplex streaming connection. this channel requires point-to-point connection services for omni-directional data communications until terminated, (e.g. telephone call or 5-way video conference) as is evident in figure 4, the marketplace entities must be able to complete every desired transaction with the marketplace via these marketplace media devices (workstations). the first type of media is in all cases a request from the signalling network for intelligent services, being it address resolution on an email to placing an order into the market for xyz. it is the focus of the technology in the paper as the latter two media systems are already standardised. !9 each user must maintain the logical address database of all interconnected parties 85 the australasian journal of information systems (ajis) began in 1993 and was based on the simple realisation that there was a need for an academic ‘outlet’ for is research in australia australasian journal of information systems volume 17 number 1 2011 5 an institutional perspective on the adoption of green is & it 1 adela j. chen richard t. watson marie-claude boudreau elena karahanna department of management information systems terry college of business university of georgia athens, georgia, usa abstract this article examines how institutional pressures affect the adoption of green is & it across organizations. from the natural-resource-based perspective, it examines green is & it practices with strategic foci on pollution prevention, product stewardship, and sustainable development. each category incorporates the separate roles played by it (as a problem) and is (as a solution). the partial least square method was employed to analyze the survey replies from 75 organizations. the results show that mimetic and coercive pressures significantly drive green is & it adoption. in particular, outcomebased imitation and imposition-based coercion represent major institutional processes. the results also suggest the complementary relationship between mimetic and coercive pressures. such interaction significantly motivates the green is & it adoption focusing on product stewardship. these findings contribute to existing knowledge on the proenvironmental behaviors of organizations, demonstrate the interaction between institutional forces, and further current understanding of green is & it adoption the study concludes with a general discussion of eco-goals and their relationship to institutional theory before considering the implications and directions for research and practice. introduction environmental sustainability has drawn significant interest among management researchers, as demonstrated by special issues in the academy of management review (in 1995) and academy of management journal (in 2000), and by targeted articles in specialized journals such as the journal of industrial ecology, journal of environmental management and environmental management. with a 1 an earlier version of this paper was accepted by the international conference on information systems (icis) in 2009. australasian journal of information systems volume 17 number 1 2011 6 few exceptions (see yi and thomas, 2007, for a summary), however, researchers have neglected the role that information system (is) and information technology (it) have played in such endeavors (watson et al. 2010; chen et al. 2008). indeed, much remains to be explored about the role played by is & it in the worldwide pursuit of ecological sustainability. such neglect may reinforce the implicit assumption that environmental sustainability lies outside of the is research realm. through this research, we seek to challenge this assumption by examining the adoption of green is & it across organizations, and thus to inform practice and research about the role of is and it in the natural environment. although is and it are often used interchangeably, we clearly distinguish between them in our research. whereas it specifically focuses on devices that store, transmit, or process information, is is more encompassing, as it includes the integrated and cooperating set of people, processes, software, and information technologies to support individual, organizational, or societal goals. as is and it are distinct with is being more encompassing (watson et al. 2010), we posit that it and is contribute to the environmental issues differently, for the most part, it appears that it is part of the problem (e.g., data center’s energy consumption, electronic waste, etc.) while is can be part of the solution (e.g., environmental management systems, telecommuting programs, etc.) (watson et al. 2009b; boudreau et al. 2008). an organization’s decision to adopt green is & it is often based on a complicated mix of both pragmatic (e.g., financial and legal) and idealist (e.g. moral and ethical) considerations (lampe et al. 1991). on the one hand, green is & it create financial concerns, as they may lead to reduced cost or incur additional expenses. being green is not necessarily cost-efficient, though in many cases it is (e.g., watson et al. 2009c). green is & it also have legal ramifications, as governments have started creating policies and regulations to penalize organizational waste and reward decrease in emissions. on the other hand, the “green” orientation also highlights the idealist implications of these technologies, both moral and ethical, as an organization’s adoption may generate positive or negative consequences for others (velasquez et al. 1985). given the public-goods nature of the natural environment, organizational reactions to an environmental problem become a moral issue because of the potential to harm or benefit other stakeholders. it can also become an ethical issue for the same reasons (e.g., jones 1991; chan et al. 2008). these idealist and pragmatic considerations relating to the adoption of green is & it are not necessarily at odds, or conflicting, with one another. indeed, the adoption of green is & it has consequences for both the organization (e.g., impact on cost and productivity and legal fines) and others (e.g., moral and ethical impact on all stakeholders, including the natural environment and future generations). this research leverages the institutional approach (dimaggio et al. 1983) to better understand organizational adoption of green is & it. the institutional perspective provides a useful theoretical lens to study the organizational response to environmental issues, because it understands that institutional forces beyond the market play a critical role in making organizations responsive to the interests of others (scott 2003). in institutionalized organizations, legitimacy and efficiency do not necessarily co-vary (zucker 1987). this is often true in the case of adopting green is & it. institutional theory explains how organizations adapt to institutional change through three different australasian journal of information systems volume 17 number 1 2011 7 mechanisms – mimetic, normative, and coercive isomorphism2. the main effects of the three pressures are firmly defined. however, there are few studies investigating the interaction among them, which this study does. particularly, this study focuses on the interaction between mimetic and coercive pressures. the other major contribution is to inform the research and practice of green is & it by reviewing the level of environmental friendliness of organizations in a wide range of practices. understanding the adoption and diffusion of green is & it across organizations informs the design of technological applications and institutional interventions to support ecological sustainability. the literature on green behaviors within organizations largely focuses on a single practice, such as recycling (cheung et al. 1999). however, we believe that the level of an organization’s environmental friendliness is better evaluated against a variety of relevant practices rather than a single one. furthermore, reality requires that firms have a coherent portfolio of actions to enhance sustainability. therefore, we adopt the natural-resource-based view of the firm (hart 1995) to categorize organizational green is & it practices into three groups based on their different strategic orientations, i.e., pollution prevention, product stewardship, and sustainable development. the three categories differ in terms of their resource requirements and contributions to ecological sustainability. we surveyed 75 organizations to examine how institutional forces motivate organizational adoption of green is & it. the results of our study provide an overview of the status quo of organizational green behaviors involving a variety of is & it. this paper is structured as follows. first, we provide the necessary theoretical background (i.e., institutional theory and the natural-resource-based view) relevant to our research. next, we present the research model, along with the propositions, and describe the indicators of the central constructs. after presenting and discussing the results, we conclude with implications for both research and practice. theoretical background we draw upon institutional theory and the natural-resource-based view of the firm. institutional theory explains how institutional isomorphism occurs through three different mechanisms – mimetic, coercive, and normative pressures. the natural-resource-based view of the firm differentiates among strategies that underlie different categories of green is & it practices. we believe that the synergy of both theoretical perspectives yields a finely grained understanding of the effects of institutional pressures on organizational adoption of green is & it. institutional theory institutional theory provides a rich, complex view of how organizations become homogeneous in broad institutional environments, sometimes due to external sources, other times from within the organization. these include factors from the technical environment, such as economic performance, and various widely practiced sociological and cognitive elements, such as professional certification 2 mimetic isomorphism happens when organizations model other organizations’ behaviors in pursuit of legitimacy or taken-for-granted practices. normative isomorphism occurs when organizations feel compelled to honor certain cultural expectations from professional circles or the larger society. coercive isomorphism is often driven by powerful stakeholders upon whom a focal organization depends. australasian journal of information systems volume 17 number 1 2011 8 and prevalent activities across other organizations. institutional theory has been used to explore an organization’s environmental behaviors (campbell 2007; chen et al. 2008; jennings et al. 1995). by developing structures or taking actions that are isomorphic with institutional pressures, organizations gain legitimacy, resources, and survival capabilities (dimaggio et al. 1983; meyer et al. 1977). institutional isomorphism is diffused through three mechanisms – normative, mimetic, and coercive isomorphism. the three mechanisms are not necessarily empirically distinguishable (dimaggio et al. 1983; mizruchi et al. 1999). compliance under normative pressures occurs when organizations feel compelled to honor certain cultural expectations from professional circles or the larger society. normative pressures can diffuse through dyadic and multilateral relational channels. mimetic isomorphism happens when organizations model other organizations’ behaviors in pursuit of legitimacy or taken-for-granted practices (dimaggio et al. 1983; tolbert et al. 1983). when a clear course of action is not available to an organization, it might decide to mimic others. coercive pressures are often associated with powerful actors upon whom a focal organization depends. there has been significant recognition of institutional forces as important predictors of the adoption and diffusion of is products and practices (e.g., liang et al. 2007; orlikowski et al. 2001; teo et al. 2003; tingling et al. 2002). while many is studies with an institutional perspective focus on a specific technology (such as electronic data interchange) with a direct bearing on adopters, we apply institutional theory to the adoption of a variety of is & it practices with both pragmatic and moral implications. natural-resource-based view of the firm organizational studies previously took little interest in the environment and focused on economic, social, political, and technological factors, with the natural environment being an absence of the performance puzzle (shrivastava et al. 1992; shrivastava et al. 1995). recognizing the natural environment as an important emerging source of competitive advantage, hart (1995) proposes a natural-resource-based view of the firm by integrating the natural environment into the resourcebased view. an organization’s competitive advantage is built upon its capabilities to engage in green economic activities. from the natural-resource-based view, hart (1995) distinguishes between three inter-connected green strategies with different orientations (table 1). goals foci pollution prevention to generate significant savings, especially during early stages, resulting in cost and productivity advantages over other organizations (hart et al. 1996). on reducing the cost of installing and operating emission-control equipments (smart 1992), shortening cycle times (hammer et al. 1993), and reducing the organization’s compliance and liability costs (rooney 1993). product stewardship to reduce the overall life-cycle environmental costs of a product by disciplining the design and development process with the objective of (shrivastava et al. 1995). on mitigating the environmental footprint of activities at each step of the value chain and achieving system transformation from cradle-to-grave to cradle-to-cradle. sustainable development to reduce the environmental impact of organizational economic activities across the world. on establishing long-term solutions rather than short-term profits by envisioning and developing clean technologies (e.g., replacing synthetic chemicals with biological substitutes). table 1. three green strategies (hart 1995) australasian journal of information systems volume 17 number 1 2011 9 green is & it green is & it refers to is & it products (e.g., software that manages an organization’s overall emissions) and practices (e.g., disposal of it equipment in an environmentally friendly way) that aim to achieve pollution prevention, product stewardship, or sustainable development (boudreau et al. 2008; molla et al. 2009a). green is & it can play a critical role in enabling green business process management (ghose et al. 2009) and in driving the shift to a sustainable society (watson et al. 2009a). there is a growing awareness among professionals that is & it can contribute to both the problem and the solution of environmental issues (molla et al. 2009a). however, organizational investment in green is & it is still at the early stage of maturity (molla et al. 2009b), and such investment may take longer to yield a return (olson 2008). adoption and diffusion of green is & it can be driven by a multitude of internal and external factors, such as financial, technological, organizational, regulatory, and ethical factors (gonzález 2005; kuo and dick 2009; molla 2008). this study conceptualizes and empirically assesses the impact of institutional factors on the adoption of green is & it, recognizing that green is & it can both be part of the problem and the solution. research model and propositions this study focuses on the mimetic and coercive mechanisms of isomorphism. with the fervent concern for environmental issues in both academic and popular media, there is hardly any doubt about the existence of widespread consensus among members of organizational fields with respect to the necessity and urgency of green practices. despite the lingering concern that environmentally friendly practices may erode competitiveness, the need of environmental protection is widespread because everyone wants a sustainably livable environment (porter et al. 1995). therefore, we propose that the foremost consideration in diffusing green practices is not how such diffusion is influenced by normative pressures, but how it is motivated through mimetic and coercive pressures when normative pressures are already established to some extent. excluding normative pressure from this study allows us to eliminate a great amount of potential confounding, making the effects of mimetic and coercive pressures more likely to be detected. this decision is based on both theoretical and empirical considerations (see table 2). by drawing upon institutional theory and the natural-resource-based view, we propose a research model including the main and the interaction effects of mimetic and coercive pressures on organizational adoption of green is & it (see figure 1). frequencybased imitation outcome-based imitation inducementbased coercion impositionbased coercion mimetic pressures coercive pressures adoption of green is & it h1a h2 h2b h2a h1 h1b h3 mimetic x coercive figure 1. research model australasian journal of information systems volume 17 number 1 2011 10 sources of normative pressure theoretical consideration empirical consideration suppliers and customers (dyadic relational channel) given the generic nature of green behaviors of interest to this study, suppliers and customers become a source of mimetic pressure at the same time. the same scale, namely, the extent of adoption, measures both pressures in the literature. professional affiliation (multilateral relational channel) accumulation of normative pressure from professional affiliation can heighten environmental issues, resulting in coercion such as public policies. the temporal sequence will be masked by correlation in a cross-sectional study. table 2. potential confounding of normative pressure with mimetic and coercive pressures mimetic pressures mimetic isomorphism is considered a standard organizational response to uncertainty when the course of action is unclear (dimaggio et al. 1983). the adoption of green is & it often involves considerable uncertainty. deviation from the single bottom line of profitability requires a mindset shift among managers and induces uncertainty. addressing the moral component of environmental issues while maximizing profitability represents a new challenge for practitioners. the moral component of an environmental issue can be converted into one that incurs instrumental consideration through the enforcement of regulations or industrial standards. relentless punishment such as steep fines or suspension of operation licenses for dumping industrial waste and toxic carries an immediate monetary implication for organizations that violate the mandate. in the absence of such coercive forces, other organizations’ behaviors and the corresponding outcomes play a critical role in determining an organization’s decision with respect to a moral issue. mimetic isomorphism as an organization’s strategic response to uncertainty has been observed in diverse studies, such as the adoption of a certain organizational form (lee et al. 2002) and the diffusion of automated teller machines prior to the establishment of its business value (banker et al. 1988). p1: mimetic pressures will be positively related to the adoption of green is & it. the two fundamental modes of imitation of interest here are frequencyand outcome-based imitations (haunschild et al. 1997). frequency-based mimetic pressure arises from the number of other organizations that have adopted a certain practice. with outcome-based imitation, organizations are motivated to adopt a given practice because of the favorable results achieved by other adopters. with trait-based imitation, organizations mimic the behaviors of other organizations with whom they share important attributes. frequency-based imitation when driven by frequency-based mimetic pressures, organizations make adoption decisions based on the prevalence of a practice. frequency-based imitation is considered the purest form of mimetic isomorphism, as all organizations in a focal organization’s institutional context have the same effects on its decision-making (lu 2002). on the one hand, such prevalence is strong evidence of the legitimacy of the practice. when a practice has been adopted by a growing number of organizations, it becomes increasingly taken-for-granted so that some organizations may adopt such practice without thinking (march 1981; zucker 1977). on the other hand, a frequently adopted practice can become difficult to change and reduce the likelihood of alternatives through the imprinting process (mezias 1990). in the face of high uncertainty associated with adopting green is & it, the sheer number of adopters in an organization’s institutional context provides the decision base, indicating the australasian journal of information systems volume 17 number 1 2011 11 desirability of a given green is & it. past research has shown that an organization’s likelihood of adopting a given practice is positively related to the proportion of prior adoption by other organizations (e.g., burns et al. 1993; haunschild et al. 1997) p1a: frequency-based imitation will be positively related to the adoption of green is & it. outcome-based imitation when outcome-based pressures are at play, organizations tend to imitate others when the observed consequences of implementing these practices are considered favorable. unlike the frequency-based imitation, outcome-based imitation is considered a selective process of mimetic isomorphism (lu 2002). the lack of immediate economic gains represents a barrier to organizational adoption of green is & it. therefore, it is difficult for such practices to be immediately accepted by organizations, especially the myopic profit-focused ones. when this is the case, the adoption outcomes of other organizations will greatly reduce the search costs and the uncertainty faced by a potential adopter, leading to adoption (or non-adoption) decisions. an organization looks to the success or failure of other adopters in its decision-making. outcome-based imitation has been observed in various settings, such as market entry mode (lu 2002) and investment banker choice (haunschild et al. 1997). p1b: outcome-based imitation will be positively related to the adoption of green is & it. coercive pressures coercive isomorphism is driven by both formal and informal forces exerted by other organizations, upon which an organization relies (dimaggio et al. 1983). from a resource-dependence perspective, coercive pressures arise from the exchange relationship between an organization and others, which may include regulatory institutions, funding bodies, and resource-dominant organizations. given the institutionalized pattern of dependency (e.g., regulations, policies, contracts, and formal programs), a resource-dependent organization complies with the resource-dominant organization to sustain the exchange relationship and secure its own survival (pfeffer et al. 1978). an organization’s adoption of green practices arises from both regulatory bodies and important supply chain partners through imposition and inducement respectively. p2: coercive pressures will be positively related to the adoption of green is & it. imposition-based coercion the adoption of green practices may be imposed upon organizations, as when regulatory authorities such as governments or agencies mandate such behaviors by law or industrial standards. regulatory institutions utilize coercive power to create institutional elements when they perceive that organizational practices are in conflict with the societal good. with regulatory authority, these institutional elements, such as industrial regulations and threat of legal sanctions, are powerful tools to govern organizational behaviors. the consequences for noncompliance may include suspension of an operating license or a steep monetary penalty. coercion in the form of public policy plays an important role in effective environmental management (kilbourne et al. 2002). in the us, regulatory institutions have enacted ordinances, regulations and laws in response to the growing awareness of australasian journal of information systems volume 17 number 1 2011 12 environmental issues (clemens et al. 2006). imposition-based coercion has been the most prevalent approach in the us (delmas and terlaak, 2001). the mandate of the low emission vehicle by the californian air resources board has stimulated active investment in r&d across automobile manufacturers (frenken et al. 2004). p2a: imposition-based coercion will be positively related to the adoption of green is & it. inducement-based coercion the second manner in which coercive isomorphism may happen is that of inducements when supply chain partners make the fulfillment of certain criteria an eligibility requirement for collaboration. an organization may develop the dependence on certain customers and suppliers when these organizations account for most of its sales and purchases, and are hard to be replaced by others. important supply chain partners do not have the authority or power to impose regulations or laws, but they often possess the power to create strong inducements for a focal organization to comply with their demands (meyer et al. 1992). supply chain partners generate forces for conformity to certain standards, which translate into coercive pressure by providing incentives (or disincentives). they can specify the conditions for the receipt of future contracts or ongoing collaboration. resource-dominant organizations that have adopted green is & it may influence their supply chain partners to behave in an environmentally friendly way as well in order to reap more benefits from their own adoption. for example, important customer or supply chain partners, as “dominant” or “definitive” stakeholders (mitchell et al. 1997), may exert pressures over organizations to be certified to iso 14000, which entails guidelines for different aspects of environmental management. wal-mart has successfully reduced the shipping cost through persuading its suppliers to reduce packaging and increase product concentration. despite its lack of regulatory power, wal-mart drives the adoption of environmentally friendly packaging among its suppliers who aim to retain the business exchange. in the context of adopting green is & it, an organization can be subject to the pressures from its supply chain partners that have adopted green is & it and attempt to maximize the financial (e.g., cost reduction) and social (e.g., reputation of being environmentally responsible) benefits of the adoption. funding decisions by stakeholders with added requirements fall within the purview of inducementbased coercion. some individuals and organizations behave in an environmentally friendly way in order to win funding opportunities, given the growth in funding on innovations that address environmental issues. environmental defense fund has induced a number of organizations to reduce emissions and heavily polluting materials with financial support and allowances (bonifant et al. 1994). p2b: inducement-based coercion will be positively related to the adoption of green is & it. interaction between mimetic and coercive pressures both mimetic and coercive pressures aim at motivating institutional isomorphism. however, they trigger different reasoning mechanisms behind an organization’s decisions to adopt green practices. on the one hand, coercive isomorphism is an organization’s conforming response to mandated standards. under coercive pressures, the threat of sanction by powerful organizations such as regulatory authorities and critical supply chain partners drives an organization’s adoption choices. on the other hand, mimetic isomorphism represents an organization’s response to uncertainty as a barrier to adoption of green practices. with mimetic pressure, organizations use the frequency and outcome of others’ adoption as a proxy indicator of the legitimacy of a given practice, when there is lack of australasian journal of information systems volume 17 number 1 2011 13 adequate information to validate the feasibility and profitability of such a practice through a cost-andbenefit calculation. the institutional pressures do not necessarily operate in isolation, especially in a dynamic environment (roy et al. 2000). between coercive and mimetic pressures, the presence of one is very likely to add to the institutional legitimacy suggested by the other. in organizational fields where coercive pressure is not directly involved, mimetic forces represent a significant factor in an organization’s green decision-making (jennings et al. 1995). the prevalence of a green practice among organizations may reflect the urgency and validity of existing or anticipated coercive forces. legal enforcement of a regulation or pressures from supply chain partners may suggest the legitimacy and criticality of a green practice, easing the uncertain conditions faced by potential adopters. moreover, when evidence of the value of a given practice comes from multiple sources rather than a single one, it is very likely to be perceived as more convincing by an organization. therefore, the presence of one pressure reinforces the effect of the other. we expect mimetic and coercive pressures to synergistically combine, lowering the uncertainty faced by potential adopters of green is & it and demonstrating the regulatory validity. p3: interaction between coercive and mimetic pressures will have a positive effect on the adoption of green is & it. controls organizations are subject to different regulatory sanctions across industries. for example, the food and drug industries are more tightly regulated than textile manufacturing because of the public health consequences. therefore, we include industry as a control variable. financial resources are an important precursor to innovation adoption (iacovou et al. 1995; riggins et al. 1994). organizations with sufficient financial resources are able to experiment with new practices and cope with adoption failures. as a result, revenue is also included as a control. methodology a questionnaire-based, cross-sectional field study was conducted to test the research model, as the objective of this research is to understand the effects of institutional pressures on the adoption of green is & it by organizations. data were collected through the cutter consortium in 2008. the total sample size was 75 participating organizations, which were from 18 industries, including both manufacturing and service industries, and from 22 countries, with about one third of the sample in the u.s. of the 75 responding organizations, 37% are headquartered or based in north america, 28% in europe, 24% in asia/australia/pacific, and the remainder in other regions. of these organizations, 12% have over 50000 employees, 16% have between 5000 and 50000 employees, 17% have between 1000 and 5000 employees, 29% have between 100 and 1000 employees, and the remainder have no more than 100 employees. the majority of the organizational representatives who filled out the questionnaires were in a position of is management (26.7%), consulting (21.3%), or senior management/policymaking (14.7%). we aimed to examine general rather than industry-specific green is & it adopted by organizations. therefore, our target population consisted of organizations across industries and with diverse attributes (see table 3). australasian journal of information systems volume 17 number 1 2011 14 annual revenues of organizations/divisions (us $) response percentage annual revenues of organizations/divisions (us $) response percentage less than 1 million 25.3% more than 100 million to 1 billion 18.7% 1 million to 10 million 12% more than 1 billion to 10 billion 13.3% more than 10 million to 50 million 12% more than 10 billion to 50 billion 5.3% more than 50 million to 100 million 10.7% more than 50 billion 2.7% table 3. annual revenue of organizations/divisions operationalization all the dependent and independent variables were operationalized formatively, according to the jarvis et al. (2003) criteria. the measures are summarized in table 4. dependent variables based on empirical studies of green it practices (such as telecommuting), on interviews with professional and managerial employees, and on discussions with colleagues interested in green is & it, we generated the items tapping each of the categories discussed in the preceding theoretical background: green is & it practices focusing on pollution prevention, product stewardship and sustainable development. thus, we operationalized organizational adoption of green is & it through three dependent variables. for item clarity, we retained the distinction between the separate roles played by is & it, as both a problem and a solution. the items themselves did not combine both aspects, each with a single focus either on practices to curb the environmental impact of it or on practices enabled by is to enhance the environmental friendliness of other business operations. this distinction is important because it allows an explicit empirical investigation of the seemingly contradictory roles in environmental issues. thus, the three dependent variables are measured formatively, each with two dimensions capturing the roles played by is & it. this study examines a wide variety of green is & it practices, which may differ from each other along multiple dimensions. for example, they range from low-frequency practices such as investing in energy-efficient it hardware and software to high-frequency ones such as using green is to manage overall emissions. moreover, they include both intra-organizational practices such as using renewable energy to support it infrastructure to inter-organizational practices such as enhancing the environmental friendliness of supply chain activities through green is. given the diversified nature of the green is & it practices, we used the level of institutionalization (i.e., the existence of policies/regulations/incentives) of such practices as a proxy of adoption. this measure, as opposed to the traditional adoption measures (e.g., frequency and scope), captures the stabilized organizational behaviors. each item asked respondents to indicate the adoption status of their organizations on a 3-point likert scale, with 1 representing no adoption, 2 representing adoption plan, and 3 representing existing adoption. dependent variables – adoptions of green is & it with a focus on pollution prevention, product stewardship, and sustainable development – are each measured with two formative indicators, with the first indicator addressing it as a cause of the environmental issues and the second indicator presenting is as a solution of environmental issues (table 4). independent and control variables mimetic and coercive pressures were both measured formatively with scales adapted from teo, wei and benbasat’s study (2003). each scale asked respondents to indicate the degree to which they australasian journal of information systems volume 17 number 1 2011 15 agreed with the statements regarding the institutional pressures on a 5-point likert format, with 1 representing strongly disagree and 5 representing strongly agree. mimetic pressure. we measured mimetic pressure through two formative indicators: frequencyand outcome-based mimetic forces. to measure the former, we used the mean of two reflective indicators capturing the extent of adoptions by an organization’s competitors and supply chain partners. to measure the latter, we used the mean of four reflective indicators that capture the perceived success of adoptions by other organizations. coercive pressure. we measured coercive pressure through two formative indicators: impositionand inducement-based coercive pressures. we measured the former by asking respondents to indicate whether their organizations are pressured to adopt green is & it by current and foreseeable regulations. two reflective indicators on whether the organizations are pressured to adopt green is & it by major customers and suppliers were used to gauge the latter. control variables. for the control variables (i.e., industry and revenue), we used two ordinal variables that indicate the industry and revenue range of an organization respectively. data analysis we used the partial least squares (pls), a variance-based structural equation modeling (sem) tool, to test the research model in view of pls’s ability to operationalize a latent construct either formatively or reflectively. we adopted smartpls with a 500 sample bootstrapping technique for model assessment. all statistical tests were assessed with one-tailed t-tests because of the unidirectional nature of our hypotheses and corollaries. an important concern for formative indicators is multicollinearity (diamantopoulos et al. 2001; petter et al. 2007). since mimetic pressure and coercive pressure are formatively measured, we examined variance inflation factors (vif). with the maximum vif being 1.495, neither exceeds 3.3 as recommended by diamantopoulos and siguaw (2006). this indicates that multicollinearity is not a concern. the inter-construct correlations for the second-order constructs are presented in table 5 and the correlations between the first-order dimensions (as well as control variables) are shown in appendix a. we assessed the correlations among the formative dimensions of the institutional pressures. the extent of adoption by supply chain partners, as one dimension of the normative pressure in the literature, is significantly and highly correlated (>=0.6) with the extent of adoption by competitors, as one dimension of the mimetic pressure. professional affiliation, as the other dimension of the normative pressure in the literature, is significantly and highly correlated (>=0.6) with both impositionand inducement-based coercive pressures. the results support our rationale for excluding the normative pressure from this study. prior to assessing the structural model, we assessed the psychometric properties of our first-order dimensions. factor analysis (see table 6), comparison of the average variance extracted (ave) to inter-construct correlations (see appendix a), and reliabilities (see table 4) show that our scales exhibit good psychometric properties. australasian journal of information systems volume 17 number 1 2011 16 constructs measure (reliability) items mimetic pressure frequency-based imitation: extent of adoption by competitors, suppliers and customers (0.820) what is the current extent of the adoption of sustainable is business practices by your organization's competitors (adp_comp)? supply chain ecosystem (adp_sucu)? outcome-based imitation: perceived success of competitors, suppliers and customers that have adopted green is & it (0.852)  our main competitors who have adopted sustainable is business practices o have benefited greatly financially. (suc_c1) o are perceived favorably by customers. (suc_c2)  within my organization's supply chain management ecosystem, those who have adopted sustainable is business practices o have benefited greatly financially. (suc_sucu1) o are perceived favorably by customers. (suc_sucu2) coercive pressure imposition-based coercion: pressure from regulatory bodies current and foreseeable regulations are pressuring us to adopt sustainable is business practices. (policy) inducement-based coercion: pressure from major customers and suppliers (0.801)  our suppliers are pressuring us to adopt sustainable is business practices. (press_sup)  our major customers are pressuring us to adopt sustainable is business practices. (press_cus) pollution prevention polpre_prob: organizational action on reducing energy consumed by it infrastructure and hardware (it as a problem) to what extent does your organization have policies  to reduce the energy consumed by its it infrastructure (through virtualization, thin clients, etc.)?  to purchase energy-efficient it hardware (e.g., energy star, 80 plus power supply, electronic product environmental assessment tool, etc.)? polpre_solu: organizational adoption of is to reduce overall emissions, waste and hazardous materials (is as a solution) to what extent does your organization have policies that encourage installation of software for which the main goal is to reduce your organization's overall emissions? waste? use of hazardous and toxic materials? product stewardship proste_prob: organizational action on disposing of it equipment in an environmentally friendly way (it as a problem) to what extent does your organization have policies that encourage  purchasing products based on an it vendor's end-of-life/recycling program? (proste2)  disposing of its it equipment in an environmentally friendly manner? (proste3) proste_solu: organizational adoption of is to enhance the environmental friendliness of upstream and downstream supply chain management (is as a solution) to what extent does your organization have policies that encourage installing software for which the main goal is  to make its upstream supply chain management (material sourcing and acquisition) more environmentally friendly?  to make its downstream supply chain management (product distribution and delivery) more environmentally friendly? sustainable development susdev_prob: organizational action on seeking renewable energy to support it infrastructure to what extent does your organization have policies that encourage use of renewable energy (solar, wind, hydro, etc.) to support its it infrastructure? susdev_solu: organizational adoption of is to transform business operations (is as a solution) to what extent does your organization have policies that encourage  online collaboration tools (beyond email) to substitute for travel (e.g., video conferencing, etc.)?  employee telecommuting?  transforming its business processes to be paperless? table 4. operationalization of constructs australasian journal of information systems volume 17 number 1 2011 17 coercive pressures mimetic pressures pollution prevention coercive pressures 1.000 mimetic pressures 0.488 1.000 pollution prevention 0.502 0.527 1.000 coercive pressures mimetic pressures product stewardship coercive pressures 1.000 mimetic pressures 0.455 1.000 product stewardship 0.476 0.450 1.000 coercive pressures mimetic pressures sustainable development coercive pressures 1.000 mimetic pressures 0.487 1.000 sustainable development 0.422 0.462 1.000 table 5. intercorrelations among latent variables pollution prevention product stewardship sustainable development (1) (2) (3) (4) (1) (2) (3) (4) (1) (2) (3) (4) adp_com p 0.87 0.161 0.086 0.038 0.884 0.178 0.086 0.04 0.813 0.181 0.086 0.037 adp_suc u 0.959 0.216 0.054 0.175 0.95 0.222 0.054 0.171 0.984 0.216 0.054 0.177 suc_c1 0.119 0.853 0.335 0.411 0.122 0.873 0.335 0.404 0.106 0.897 0.335 0.416 suc_c2 0.105 0.834 0.299 0.287 0.099 0.8 0.299 0.271 0.123 0.79 0.299 0.298 suc_suc u1 0.307 0.859 0.388 0.461 0.306 0.889 0.388 0.451 0.307 0.9 0.388 0.467 suc_suc u2 0.174 0.781 0.486 0.384 0.172 0.756 0.486 0.377 0.179 0.708 0.486 0.389 policy 0.003 0.45 1 0.585 0.002 0.448 1 0.575 0.022 0.432 1 0.59 press_cus 0.136 0.502 0.598 0.935 0.131 0.508 0.598 0.913 0.155 0.505 0.598 0.95 press_sup 0.106 0.32 0.454 0.889 0.103 0.332 0.454 0.914 0.116 0.33 0.454 0.868 (1) – frequency-based imitation, (2) – outcome-based imitation, (3) – imposition-based coercion, (4) – inducement-based coercion table 6. pls confirmatory factor analysis results as shown in figure 2, figure 3, and figure 4, pls results provide strong support for h1 and h2. the lines marked with an asterisk represent the paths that are significant at 0.05. the corollary h1b is consistently supported across the models, indicating strong evidence for the role of outcome-based pressure in organizational adoption of green is & it. the corollary h1a is not supported in all models. hypotheses h2a, h2b and h3 are partially supported. mimetic and coercive pressures, but not the control variables (i.e., industry and revenue), are significant determinants of organizational adoption of green is & it, explaining 35.6%, 29.7%, and 27.6% of the variances in practices focusing on pollution prevention, product stewardship and sustainable development, respectively. we first ran these models including the control variables one at a time. because neither of them is significant at 0.05, we exclude the control variables from the models presented in this paper. furthermore, all formative indicators of the dependent variables, except for the “it as a problem” dimension of sustainable development, have significant weights on their corresponding constructs. table 7 and figure 5 show the results of our tests of the interaction hypotheses. the hypothesized complementary australasian journal of information systems volume 17 number 1 2011 18 effect (i.e., positive interaction) between mimetic and coercive pressures is significant only in the product stewardship model. the interaction effects, which are presented in figure 6, explain an additional 6.8% of variance in the adoption of green is & it focusing on product stewardship. frequencybased pressure outcome-based pressure inducementbased pressure impositionbased pressure mimetic pressures coercive pressures pollution reduction (35.6%) 0.174 (0.191) 0.321* (0.000) 0.455* (0.026) 0.666* (0.001) 0.370* (0.000) 0.950* (0.000) it as a solution it as a problem 0.545* (0.001) 0.658* (0.000) path coefficient (p-value) figure 2. pls results (pollution prevention frequencybased pressure outcome-based pressure inducementbased pressure impositionbased pressure mimetic pressures coercive pressures product stewardship (29.7%) 0.345 (0.102) 0.342* (0.000) 0.692* (0.003) 0.427 (0.066) 0.294* (0.009) 0.871* (0.000) business process transformation telecommuting 0.637* (0.002) 0.500* (0.017) path coefficient (p-value) figure 3. pls results (product stewardship) frequencybased pressure outcome-based pressure inducementbased pressure impositionbased pressure mimetic pressures coercive pressures sustainable development (27.6%) 0.011 (0.482) 0.276* (0.010) 0.073 (0.415) 0.956* (0.000) 0.337* (0.003) 0.998* (0.000) is as a solution it as a problem 0.874* (0.000) 0.273 (0.151) path coefficient (p-value) figure 4. pls results (sustainable development) frequencybased pressure outcome-based pressure inducementbased pressure impositionbased pressure mimetic pressures coercive pressures product stewardship (36.5%) 0.381 (0.077) 0.458* (0.000) 0.748* (0.002) 0.360 (0.101) 0.170 (0.057) 0.850* (0.000) it as a solution it as a problem 0.748* (0.000) 0.373* (0.048) mimetic x coercive 0.291* (0.011) path coefficient (p-value) figure 5. pls results (product stewardship with interaction effect) australasian journal of information systems volume 17 number 1 2011 19 independent variable model 1 model 2 mimetic pressures ➔ product stewardship 0.294* (0.009) 0.170 (0.057) coercive pressures ➔ product stewardship 0.342** (0.000) 0.458** (0.000) mimetic pressures x coercive pressures ➔ product stewardship 0.291* (0.011) adjusted r2 29.70% 36.50% ∆ r2 6.80% f 7.603* table 7. interaction effects for product stewardship (p-value** p<0.001 * p<0.01) figure 6. interaction of mimetic and coercive pressures (product stewardship) discussion this study examines institutional pressures that can motivate the adoption of green practices across organizations. given the theoretical and empirical difficulty of differentiating the effects of mimetic and normative pressures, we focus on mimetic and coercive pressures and propose that both pressures are important factors that drive green is & it practices. adoption of general rather than industryspecific is & it is of interest. we control for the effects of certain organizational attributes such as industry and revenue. we consider frequencyand outcome-based imitations as two important mechanisms of mimetic isomorphism. the extent to which other organizations have adopted green is & it and the perceived success of their adoptions serve as valid proxy indicators of the mimetic pressure. regulatory authorities and supply chain partners represent two important sources of coercive pressures. our analysis provides strong support for the main effects of mimetic and coercive pressures. the hypothesized complementary relationship between the two constructs receives partial support, being significant only in the product stewardship model. one explanation for the partial support for the complementary relationship between mimetic and coercive pressures may be that the reliance on supply chain partners and the lack of regulatory guidance in product stewardship practices present high uncertainty so that organizations take extra precaution in adoption decision-making. australasian journal of information systems volume 17 number 1 2011 20 according to our analysis, outcome-based imitation consistently represents a significant source of mimetic pressures across the models of pollution prevention, product stewardship and sustainable development, while frequency-based imitation is consistently insignificant. this suggests organizations cautiously adopt green is & it: the sheer number of adopters is not strong enough to reduce the uncertainty associated with green practices. rather, the perceived favorable outcomes provide a more convincing rationale for adoption. in contrast to the consistent pattern of mimetic pressures, different indicators of coercive pressures turn out to be significant in different models. regulations are a significant source of coercive pressures only in models predicting pollution prevention and sustainable development practices. this reflects the effectiveness of regulatory efforts in guiding green behaviors across organizations, especially when such behaviors have an organization-wide, as opposed to supply-chain-wide, impact. pressures from supply chain partners, rather than regulations, turn out to be significant in the model of product stewardship. on the one hand, this suggests a lag in regulatory efforts to motivate a fulllifecycle eco-friendliness. on the other hand, the significance of coercive pressures from supply chain reflects the supply-chain-dependent nature of practices oriented towards product stewardship. product stewardship emphasizes the full lifecycle of a product, striving to turn the traditional cradle-to-grave production into cradle-to-cradle eco-design that builds upon the collaborative agreements among supply chain partners. it calls for a mindset shift from a partial-lifecycle to a full-lifecycle perspective. overall optimization does not necessarily mean local optimization. as stakeholders become salient at different stages of a product lifecycle, they tend to maximize financial gains during the part of product lifecycle that matters to their financial performance rather than seeking a fulllifecycle optimization. for example, manufacturers are the dominant stakeholders during the production stage. obtaining materials, manufacturing products, and distributing products through retail channels are of immediate concern to manufacturers. therefore, they aim to optimize those activities to increase financial gains during the production stage. manufacturers are seldom interested in how individual consumers deal with the products at the end of their lifecycle, especially in financial terms. likewise, consumers tend to care more about the price and performance of a product than about how it is produced and how raw materials are obtained. creating regulations that effectively drive collective efforts across supply chain can be a challenging task for regulatory authorities. given the relatively small sample size (i.e., 75 responses) and relatively large presence of us-based organizations (i.e., 33% of the responding organizations based in us), we conducted additional analysis. first, we calculated post-hoc statistical power for our models, all of which turned out to have sufficient power (over 0.99). second, we analyzed alternative models by including moderating effects to assess whether being based in us affects the observed relationships. we did not detect any significant differences (at 0.05) between us-based organizations and non-us-based ones in the hypothesized relationships. beyond the adoption of green is/it: studying the eco-goals this study has focused on the adoption of particular is & it related practices. more importantly, we believe, is the adoption of a mindset focused on the implementation of sustainable organizational practices. solving global climate change requires a wholehearted acceptance of the general principles of ecological change, rather than just adopting specific practices. we need general executives who think green as well as acting green. we need is executives who envisage how is can create a sustainable corporation and can move beyond greening the data center or installing power monitoring for personal computers. institutional theory and the natural-resource-based view of the firm are not the only theoretical lenses that can be used to examine green is & it. adoption of the three eco-goals australasian journal of information systems volume 17 number 1 2011 21 (eco-efficiency, eco-equity, and eco-effectiveness) identified in the sustainability literature (dyllick et al. 2002) can be used as a framework for understanding green is & it. eco-efficiency is “the delivery of competitively-priced goods and services that satisfy human needs and bring quality of life, while progressively reducing ecological impacts and resource intensity throughout the life-cycle to a level at least in line with the earth’s carrying capacity” (desimone et al. 1997). eco-efficiency is aligned with organizational efficiency and profitability. reducing energy consumption and resource usage is clearly also a way of cutting costs and improving revenues. thus, we would expect institutional theory to explain the adoption of eco-efficiency, because although it aligns with economic performance, there is also strong social pressure for economic efficiency in liberal democracies. it is part of the social fabric to expect efficiency from economic actors. the pursuit of eco-efficiency goals is not likely to change current institutional forces on is leaders. eco-equity refers to the “equity between peoples and generations and, in particular, the equal rights of all peoples to environmental resources” (gray et al. 2000). implicitly, sustainability requires a fair distribution of resources both within and across generations and thus eco-equity is a core principle for those advocating sustainability. eco-equity is driven by social pressures and may be misaligned with short or even medium term eco-efficiency goals. eco-equity focuses on the current generation’s social responsibility for those who inherit what it leaves in place. in the pursuit of economic goals, many societies would appear to be willing to let their descendants bear the consequences of excessive current consumption of scarce resources. in order to understand what adoption of eco-equity principles means for is leaders, we need to identify the consequences of seeking eco-equity and the impact on our field. the concept of eco-effectiveness was introduced late last century (mcdonough et al. 1998) without a clear statement of its meaning. the coiners explained that “our concept of eco-effectiveness means working on the right things—on the right products and services and systems—instead of making the wrong things less bad” (mcdonough et al. 2002). the parallels to drucker’s (1954; 2006) life-long concern with distinguishing between efficiency (doing things right) and effectiveness (doing the right things) are evident. achieving an eco-effective civilization, which is the only long-term goal we can sensibly pursue for the survival of our species, requires a radical transformation of current thinking on most matters (mcdonough et al. 2002), including is (watson et al. 2010) we contend that organizations need to move beyond eco-efficiency. sustainability, restoration, and regeneration should be the prevailing organizational aspirations (mcdonough et al. 2002). in particular for our domain, we need to understand what these goals mean for is leaders. for perhaps the last half century, is leaders have managed the technology that has transformed business and society (e.g., networks, relational database, programming languages, graphical user interface, and so forth). we need to translate these goals of sustainability, restoration, and regeneration into is actions and unleash the transformative power of is to promote sustainability. rather than applying theoretical perspectives reactively, as most social scientists do and as we have done in this study, we need to become solution scientists and apply theories proactively to accelerate the adoption of highly desirable organizational traits and is practices for eco-effectiveness. contributions and implications for research and practice this study contributes to research and practice in several ways. the contribution to research is threepronged. first, it represents one of the few studies focusing on the role played by is & it in green practices. the heightened importance of ecological sustainability has generated a body of research on green practices. however, is & it have been a missing piece of the eco-sustainability puzzle. by drawing upon the natural-resource-based view of the firm, we identify three types of is & it-based green practices. we also make an important distinction between is and it in driving ecological sustainability. second, the research contributes to the literature of institutional theory by examining australasian journal of information systems volume 17 number 1 2011 22 the interaction between institutional forces. although such interaction has been theoretically recognized (delmas and toffel, 2004), there is a paucity of empirical research on it. in particular, this study provides empirical support for the complementary effects between mimetic and coercive pressures in driving the adoption of is & it-based product stewardship by organizations. third, this study also enriches the research on green practices by assessing organizational action on adopting a multitude of green practices. a wide variety of practices can better capture the eco-friendliness of an organization. in other work, we have introduced the idea of a new is subfield, energy informatics (watson et al. 2010), to focus on using is & it to improve energy efficiency. it is important that is scholars start working in this area. we need to make a contribution, and we believe a good starting point is to discover what causes the adoption of the fundamentally different types of green behaviors. the research model and the hypotheses developed in this study provide avenues for future research. first, by examining a multitude of generic is & it-based green practices, we have embarked on a study across various industries. future studies can take a finely grained approach by focusing on a particular industry. thus, the findings based on industry-specific green is & it may provide more insights in industrial heterogeneity regarding the adoption of green practices. alternatively, future research can also explore emerging green is & it practices, which may not exist or gain prevalence when this study was conducted. the advancement of is and it, coupled with the growth of institutional pressures, may further change the industrial landscape and give rise to new practices across organizations. therefore, revisiting the list of green is & it practices identified in this study is important for keeping our understanding of the field current. we should expect more is practices to emerge in the near future as attention has been mainly focused on it. to some extent greening it has not required much imagination – cut energy use – and is-driven greening will require greater creativity and could produce a greater variety of practices. second, given the cross-sectional nature of research design, this study focuses on the synchronic effects of the institutional pressures. to complement the findings yielded in this study, future research can take a longitudinal approach to demonstrate the interactive operation of the institutional forces over time, and thereby account for the diachronic effects. additionally, our understanding of green is & it in this study is built upon a small sample size (75 organizations). a future study based on an extended sample can be conducted to refine our understanding of this increasingly important phenomenon. additionally, the majority of the respondents were from high per capita income, liberal democracies. given that china is now the major contributor to green house gases, we need to replicate this study in china, which has a different culture, lower per capita incomes, and is a command economy. our study has important implications for practitioners. first, the outcome-based mimetic isomorphism is an important determinant of organizational adoption of green is & it. due to the inherent uncertainty of the outcomes of green practices, making successful adoptions known to potential adopters will motivate their mind-set shift and provide effective guidance in their decision-makings. thus, part of general research in this area is to publish case studies on leading green is practices (watson et al. 2009b) and a chapter for introductory is classes (watson et al. 2008). second, it highlights the importance of the complementary effects between mimetic and coercive pressures. the complementary effects between the two kinds of pressure may inform regulatory authorities in developing effective interventions in driving the diffusion of green is & it across organizations. finally, this study also brings to the attention of organizations and regulatory bodies the separate roles played by is and it in the pursuit of ecological sustainability. this careful differentiation helps organizations to find the right positions for is and it in their green business strategies. australasian journal of information systems volume 17 number 1 2011 23 references american psychological association online (2009). peer pressure best motivator when it comes to energy saving, psychologists tell house panel. accessed last october 2009 at http://www.apa.org/releases/peerpressure.html. banker, r.d., and kauffman, r.j. (1988). strategic contributions of information technology: an empirical study of atm networks. proceedings of the ninth international conference on information systems, minneapolis, mn, pp. 141-150. bonifant, b.c., and ratcliffe, i. (1994). competitive implications of environmental regulation in the pulp and paper industry. working paper, management institute for environment and business, washington, d.c., 1994. boudreau, m., watson, r.t., and chen, a. (2008). from green it to green is. cutter benchmark review, vol. 8(5) 2008, pp. 5-11. burns, l., and wholey, d. (1993). adoption and abandonment of matrix management programs: effects of organizational characteristics and interorganizational networks. academy of management journal, pp. 106-138. campbell, j. (2007). why would corporations behave in socially responsible ways? an institutional theory of corporate social responsibility. academy of management review, vol. 32(3), pp. 946-967. carey, j. (2009). the tug-of-war over cap and trade. businessweek, the mcgraw-hill companies, pp. 20-22. chen, a., boudreau, m., and watson, r. (2008). information systems and ecological sustainability," journal of systems and information technology, vol. 10, pp. 186-201. cheung, s., chan, d., and wong, z. (1999). reexamining the theory of planned behavior in understanding wastepaper recycling. environment and behavior, vol. 31(5), pp. 587-612. clemens, b., and douglas, t. (2006). does coercion drive firms to adopt ‘voluntary’ green initiatives? relationships among coercion, superior firm resources, and voluntary green initiatives. journal of business research, vol. 59(4), pp. 483-491. delmas, m., and terlaak, a. (2001). a framework for analyzing environmental voluntary agreements. california management review, vol. 43(3), pp. 44-63. delmas, m., and toffel, m. (2004). stakeholders and environmental management practices: an institutional framework. business strategy and the environment, vol. 13(4), pp. 209-222. desimone, l.d., and popoff, f. (2000). eco-efficiency : the business link to sustainable development mit press, cambridge, mass. diamantopoulos, a., and siguaw, j. (2006). formative versus reflective indicators in organizational measure development: a comparison and empirical illustration. british journal of management, vol. 17(4), pp. 263-282. diamantopoulos, a., and winklhofer, h. (2001). index construction with formative indicators: an alternative to scale development. journal of marketing research, pp. 269-277. dimaggio, p.j., and powell, w.w. (1983). the iron cage revisited: institutional isomorphism and collective rationality in organizational fields. american sociological review, vol. 48(2), pp. 147-160. drucker, p.f. (1954). the practice of management, harper, new york, pp. 404. drucker, p.f. (2006). what executives should remember. harvard business review, vol. 84(2), pp. 144-153. australasian journal of information systems volume 17 number 1 2011 24 dyllick, t., and hockerts, k. (2002). beyond the business case for corporate sustainability," business strategy and the environment, vol. 11(2), pp. 130-141. frenken, k., hekkert, m., and godfroij, p. (2004). r&d portfolios in environmentally friendly automotive propulsion: variety, competition and policy implications. technological forecasting & social change, vol. 71(5), pp. 485-507. ghose, a., hoesch-klohe, k., hinsche, l., and le, l.-s. (2009). green business process management: a research agenda. australian journal of information systems, vol. 16(2), pp. 103-117. ginsberg, a., and buchholtz, a. (1990). converting to for-profit status: corporate responsiveness to radical change. academy of management journal, pp. 445-477. gonzález, p.d.r. (2005). analysing the factors influencing clean technology adoption: a study of the spanish pulp and paper industry. business strategy and the environment, vol. 14(1), pp. 2037. gray, r., and bebbington, k. (2000). environmental accounting, managerialism and sustainability: is the planet safe in the hands of business and accounting? advances in environmental accounting and management, vol. 1(1), pp. 1-44. greening, d., and gray, b. (1994). testing a model of organizational response to social and political issues. academy of management journal, pp. 467-498. hammer, m.c., and champy, j. (1993). reengineering the corporation: a manifesto for business revolution. harper business, new york. hart, s., and ahuja, g. (1996). does it pay to be green? an empirical examination of the relationship between emission reduction and firm performance. business strategy and the environment, vol. 5(1), pp. 30-37. hart, s.l. (1995). a natural-resource-based view of the firm. academy of management review, vol. 20(4), pp. 986-1014. haunschild, p., and miner, a. (1997). modes of interorganizational imitation: the effects of outcome salience and uncertainty. administrative science quarterly, vol. 42(3), pp. 472-500. huff, s., and munro, m. (1985). information technology assessment and adoption: a field study. mis quarterly, pp. 327-340. iacovou, c., benbasat, i., and dexter, a. (1995). electronic data interchange and small organizations: adoption and impact of technology. mis quarterly, pp. 465-485. jennings, p.d., and zandbergen, p.a. (1995). ecologically sustainable organizations: an institutional approach. academy of management review, vol. 20(4), pp. 1015-1052. kilbourne, w., beckmann, s., and thelen, e. (2002). the role of the dominant social paradigm in environmental attitudes: a multinational examination. journal of business research, vol. 55(3), pp. 193-204. knoke, d. (1982). the spread of municipal reform: temporal, spatial, and social dynamics. american journal of sociology, pp. 1314-1339. kuo, b., and dick, g. (2009). the greening of organisational it: what makes a difference? australian journal of information systems, vol. 16(2), pp. 81-92. lampe, m., ellis, s. r., and drummond, c. k. (1991). what companies are doing to meet environmental protection responsibilities: balancing legal, ethical, and profit concerns. proceedings of the international association for business and society, pp. 527-537. lee, k., and pennings, j.m. (2002). mimicry and the market: adoption of a new organizational form. academy of management journal, vol. 45(1), pp. 144-162. australasian journal of information systems volume 17 number 1 2011 25 lewin, k. (1947). frontiers in group dynamics. human relations, pp. 5-41. liang, h., saraf, n., hu, q., and xue, y. (2007). assimilation of enterprise systems: the effect of institutional pressures and the mediating role of top management. mis quarterly, vol. 31(1), pp. 59. lu, j. (2002). intra-and inter-organizational imitative behavior: institutional influences on japanese firms' entry mode choice. journal of international business studies, vol. 33(1), pp 19-38. march, j. (1981). decisions in organizations and theories of choice. in a.h. van de ven and w.f. joyce (eds.) perspectives on organization design and behavior, wiley, ny, pp. 205-244. mcdonough, w., and braungart, m. (1998). the next industrial revolution. in the atlantic monthly. mcdonough, w., and braungart, m. (2002). cradle to cradle: remaking the way we make things. north point press. meyer, j., and rowan, b. (1977). institutionalized organizations: formal structure as myth and ceremony. american journal of sociology, vol. 83(2), pp. 340-363. meyer, j., and scott, w. (1992). organizational environments. sage publications, newbury park, ca. mezias, s. (1990). an institutional model of organizational practice: financial reporting at the fortune 200. administrative science quarterly, pp. 431-457. mitchell, r., agle, b., and wood, d. (1997). toward a theory of stakeholder identification and salience: defining the principle of who and what really counts. academy of management review, pp. 853-886. mizruchi, m., and fein, l. (1999). the social construction of organizational knowledge: a study of the uses of coercive, mimetic, and normative isomorphism. administrative science quarterly, vol. 44(4), pp. 653-683. molla, a. (2008). gitam: a model for the adoption of green it. proceedings of the 19th australasian conference on information systems, christchurch, new zealand, pp. 658-668. molla, a., cooper, v., deng, h., and lukaitis, s. (2009a). a preliminary report on green it attitude and actions among australian it professionals. working paper, school of business information technology, rmit university, melbourne, australia, pp. 1-13. molla, a., pittayachawan, s., and corbitt, b. (2009b). green it diffusion: an international comparison. working paper, school of business information technology, rmit university, melbourne, australia. olson, e. (2008). creating an enterprise-level "green" strategy. journal of business strategy, vol. 29(2), pp. 22-30. orlikowski, w., and barley, s. (2001). technology and institutions: what can research on information technology and research on organizations learn from each other? mis quarterly, pp. 145-165. perrow, c. (1986). complex organizations: a critical essay. random house trade. petter, s., straub, d., and rai, a. (2007). specifying formative constructs in information systems research. mis quarterly, vol. 31(4), pp. 623-656. pfeffer, j., and salancik, g. (1978). the external control of organizations: a resource dependence perspective. harper & row, new york. porter, m.e., and van der linde, c. (1995). green and competitive: ending the stalemate. harvard business review, vol. 73(5), pp. 120-133. riggins, f., kriebel, c., and mukhopadhyay, t. (1994). the growth of interorganizational systems in the presence of network externalities. management science, pp. 984-998. australasian journal of information systems volume 17 number 1 2011 26 rooney, c. (1993). economics of pollution prevention: how waste reduction pays. pollution prevention review, vol. 3(summer), pp. 261-276. rowan, b. (1982). organizational structure and the institutional environment: the case of public schools. administrative science quarterly, pp. 259-279. roy, c., and séguin, f. (2000). the institutionalization of efficiency-oriented approaches for public service improvement. public productivity & management review, pp. 449-468. scott, r. (2003). organizations: rational, natural, and open systems (5th ed.). prentice hall, upper saddle river, nj. shrivastava, p., and hart, s. (1992). greening organizations. proceedings of the annual meeting of academy of management, las vegas, nv, pp. 185-189. shrivastava, p., and hart, s. (1995). creating sustainable corporations. business strategy and the environment, vol. 4(3), pp. 154-165. smart, b. (1992). beyond compliance: a new industry view of the environment. world resources institute, washington, dc. teo, h., wei, k., and benbasat, i. (2003). predicting intention to adopt interorganizational linkages: an institutional perspective. mis quarterly, pp. 19-49. tingling, p., and parent, m. (2002). mimetic isomorphism and technology evaluation: does imitation transcend judgment? journal of the association for information systems, vol. 3(1), pp. 113143. tolbert, p., and zucker, l. (1983). institutional sources of change in the formal structure of organizations: the diffusion of civil service reform, 1880-1935. administrative science quarterly, vol. 28(1), pp. 22-39. velasquez, m., and rostankowski, c. (1985). ethics: theory and practice. prentice-hall, englewood cliffs, nj. watson, r., aronson, j., donnellan, b., and desautels, p. (2009a). energy + information < energy. 15th americas conference on information systems, san francisco, ca, usa, pp. 448. watson, r. t., boudreau, m.-c., chen, a., and huber, m. (2009b). green is: building sustainable business practices. the global text project, athens, ga, usa, http://globaltext.terry.uga.edu/books#greenis. watson, r. t., boudreau, m.-c., li, s. and levis, j., (2009c). ups: en route to energy informatics. working paper, university of georgia, ga. watson, r. t., boudreau, m.-c., chen, a. (2010). information systems and environmentally sustainable development: energy informatics and new directions for the is community. mis quarterly, vol. 34(1), pp. 23-38. yi, l., and thomas, h. (2007). a review of research on the environmental impact of e-business and ict. environment international, vol. 33(6), pp. 841-849. zucker, l. (1977). the role of institutionalization in cultural persistence. american sociological review, pp. 726-743. zucker, l. (1987). institutional theories of organization. annual review of sociology, vol. 13(1), pp. 443-464. http://globaltext.terry.uga.edu/books#greenis http://www.sciencedirect.com/science/journal/01604120 australasian journal of information systems volume 17 number 1 2011 27 appendix a: correlation matrix mean (std dev) 1 2 3 4 5 6 7 8 9 10 11 12 1.frequencybased 1.30 (1.21) 1.000 2.outcomebased 3.16 (0.77) 0.204 1.000 3.impositionbased 3.32 (1.25) 0.015 0.454 ** 1.000 4.inducementbased 2.70 (1.05) 0.117 0.447 ** 0.575 ** 1.000 5.polpre_prob 3.84 (1.46) 0.108 0.485 ** 0.390 ** 0.320 ** 1.000 6.polpre_solu 4.35 (1.87) 0.225 0.367 ** 0.384 ** 0.386 ** 0.376 ** 1.000 7.proste_prob 3.64 (1.40) 0.133 0.387 ** 0.405 ** 0.333 ** 0.579 ** 0.486 ** 1.000 8.proste_solu 2.93 (1.38) 0.268 * 0.365 ** 0.302 ** 0.442 ** 0.518 ** 0.690 ** 0.540 ** 1.000 9.susdev_prob 1.52 (0.76) 0.181 0.230 * 0.291 * 0.13 0.356 ** 0.584 ** 0.496 ** 0.408 ** 1.000 10.susdev_solu 6.41 (1.98) 0.058 0.460 ** 0.403 ** 0.268 * 0.569 ** 0.439 ** 0.575 ** 0.584 ** 0.340 ** 1.000 11.industry 11.72 (6.18) -0.15 0.088 0.127 0.109 -0.03 -0.08 0.033 -0.06 -0.11 -0.03 1.000 12.revenue 3.60 (2.10) 0.075 0.004 0.06 0.043 0.151 0.04 -0.00 0.084 0.047 0.05 0.156 1.000 *. correlation is significant at the 0.05 level (2-tailed). **. correlation is significant at the 0.01 level (2-tailed). the ave values of frequency-based imitation are 0.839, 0.843, and 0.814 for the models of pollution prevention, product stewardship, and sustainable development, respectively; the ave values of outcome-based imitation for the three models are 0.693, 0.691, and 0.685, respectively; and the ave values of inducement-based coercion for the three models are 0.832, 0.834, and 0.828, respectively. ajis vol. 8 no. 1 september 2000 a methodology for evaluating user perceptions of the delivery of ict services: a comparative study of six uk local authorities les worrall, dan remenyi and arthur money prof. les worrall, associate dean (research), management research centre, wolverhampton business school, university of wolverhampton, telford campus, telford, tf2 9nt, uk. l.worrall@wlv.ac.uk prof. dan remenyi, visiting professor, department of information systems and computing, brunei university, uxbridge, middlesex ubs 3ph, uk. address for correspondence: remenyi@compuserve.com prof. arthur money, henley management college, greenlands, henley on thames, oxfordshire, rg9 3au, uk. arthur.m@henleymc.ac.uk abstract: evaluating and managing the effective delivery of ict services is an issue that has been brought into sharper relief recently. this has been particularly prevalent in the uk public sector where the growing emphasis on formalised client-contractor relationships, outsourcing and benchmarking (both between local authorities and between local authorities and private sector organisations) has meant that the definition of service standards and agreeing performance criteria has attracted considerable practitioner attention. this research is based on 295 interviews conducted in six uk local authorities. the investigation used both gap analysis and perceptual mapping techniques to develop an understanding of the aspects of ict service delivery that users' value most in conjunction with an assessment of how well they perceive their ict department is performing on these criteria. the paper exposes considerable differences in the relative performance of the six local authorities from both the gap analysis and the perceptual mapping elements of the investigation. the methodology is shown to provide an effective way of identifying key performance issues from the user perspective and benchmarking service performance across organisations. introduction this paper proposes a methodology for evaluating the performance of an information and communications technology (ict) department or service provider. although the research has been conducted within uk local government, we propose a methodological framework for die evaluation of effectiveness of ict service delivery, incorporating the views of all the main stakeholders, which we believe is generally applicable. this subject is important because it is increasingly clear that the current approaches to managing ict and managing ict departments have not always produced satisfactory results. specifically, computer systems take too long to develop, user departments often voice their dissatisfaction with the quality and timeliness of support they receive, systems often run over budget (allingham and o'connor 1992; lester and willcocks, 1993; banker and kemerer 1991; remenyi, 1991) and systems are frequently perceived not to deliver the benefits on which they were originally business-justified (attewell 1993; brynjolfsson 1993; wilson 1993; peters 1988; parker, 1989; hitt and brynjolfsson 1994; remenyi et al. 1997; remenyi et al. 2000; willcocks 1991). these circumstances are not new and they have been a central challenge for ict management and the management of useroriented ict services for the past thirty years: they continue to be a major problem in many organisations today. the ict function of an organisation is involved in the development, implementation and maintenance of numerous information technologies and systems and supporting a wide range of users of differing ability and ict awareness. more important, certainly in the uk, the power in the user-provider relationship has moved significantly from the provider to users as initiatives such as "market testing", "compulsory competitive tendering" and "best value" have been introduced in uk central and local government (see goss et al, 1993 for a description of these terms). the information systems in most organisations aim to meet needs at all levels within the organisation from operational, through tactical to strategic. in evaluating the success or effectiveness of the ict department it is necessary to evaluate the performance of the individual systems and the performance of the ict department in supporting the users of those systems. in addition, it is important to assess users' overall view of the effectiveness of the ict department (silk, 1990). however, in recent years, and particularly in local government, users have become far more concerned about the effective delivery of an ict service and, in many instances, this has become enshrined in the production of service level agreements negotiated between ict users and the service providers. in addition to assessing satisfactions with systems, and given the changes in power relationships, we also need to be able to assess satisfaction with the broader quality of service provided defined on parameters of most relevance to the user. 73 ajis vol. 8 no. 1 september 2000 there have been continuous attempts to improve both the efficiency and effectiveness with which computer systems are conceptualised, developed and commissioned: although considerable progress has been made the situation is still fraught with many problems (remenyi and money, 1991; remenyi and money 1993; remenyi et al, 1997; remenyi et al., 2000). however, the problems identified require urgent resolution as it becomes more and more apparent that traditional information systems methods and traditional, provider-oriented approaches to service management and service delivery no longer fit the way organisations need to conduct their business in an ever increasingly competitive environment. from a functional perspective, which has implications for managing the ict service, local authorities in the uk are highly complex organisations (worrall, 1994; 1995). it has been estimated that each year, collectively, local government spends over £1 billion on information technology (socitm, 1996). this is a substantial sum, yet, apart from regular analysis by the audit commission, we are unaware of any systematic cross-local authority comparisons of how the users of ict view the relative importance of aspects of their organisation's ict management policies or how satisfied they are with how their organisation performs on those criteria. the research presented here moves some way to filling this gap in our knowledge and is based on questionnaire interviews with ict users in six local authorities. the interviews were completed in september 1997 and used a variant of a questionnaire that had been designed and tested elsewhere (see remenyi and money, 1991; 1993). this paper comprises seven main sections. the first section contains a description of the approach used to collect the data and a description of the respondents to the sample in terms of their level in the organisation and their intensity of use of ict. the second section contains an analysis of responses on the importance of attributes affecting is/t effectiveness and a user assessment of organisational performance on these attributes. the third section uses factor analysis to highlight key themes in both the importance and performance data. the key importance factors are then used to compare the performance of local authorities. in the fourth section, the relationship between an organisation's performance on the importance composite variables produced by the factor analysis are compared in order to assess the performance. in the fifth main section, the local authorities are compared based on their scores on the eight derived variables. in the sixth section, a cross-local authority analysis of a number of attitudinal variables is conducted in order to try to explain some of the differences in performance across local authorities that the analysis has exposed. in the seventh, and final main section, there is an analysis of the open responses which the questionnaire sought to obtain. this final section is based on a content analysis of the open responses to which correspondence analysis (greenacre, 1994; remenyi and money, 1994) has been applied in order to identify similarities between local authorities and the issues that users have identified within them. the concluding section summarises the main findings and discusses the responses of the ict directors of the six participating local authorities to the findings following a debriefing session held with them on the outcomes of the research. the approach to evidence collection the research project uses a previously validated self-completion questionnaire. the questionnaire was developed in an earlier study (remenyi and money, 1994): the attributes included with it (see table 2) were derived from a review of existing literature, from the authors' extensive experience in ict management and from discussions with users prior to the implementation of the initial study. prior to the execution of the study reported here, consultations were held with several it experts from uk local government and the it directors from the six participating local authorities. on the basis of this the attributes derived from remenyi and money's 1994 study were modified slightly to reflect the uk local government context. the analytical techniques used include satisfaction-importance gap analysis as developed by parasruraman et al. (1985) to explore levels of user satisfaction and user assessed importance of attributes and established techniques such as regression analysis, factor analysis (lehmann 1989; kerlinger 1969) and correspondence analysis (greenacre 1984). these techniques are used to develop an understanding of the perceptions which shape and underpin user satisfaction (see hair et al, 1998). much of the data collected was derived using likert scales and we are aware of the problems that may arise using regression and factor analysis on this type of data. however, this form of analysis now seems to have become commonplace in much management research literature. 74 ajis vol. 8 no. 1 september 2000 given the perennial problem of gaining access to organisations in business and management research, initial contact was made through the society of information technology managers to six local authorities who then agreed to participate in the study: they have been given the fictitious names of the london borough of banton, dunton county council, eston county council, posham unitary council, wrenton unitary council and brinton district council. the local authorities included in the exercise thus represent a variety of types and sizes of local authority. the data collection process was completed in september 1997. the number of respondents from each local authority is shown in table 1. table 1: responding local authorities and the number of responses local authority banton brinton dunton eston posham wrenton total type of local authority london borough district council county council county council unitary authority unitary authority responses 52 52 49 82 31 29 295 the questionnaire comprised seven major sections. the first part sought information about the respondents such as their level in the organisation, their years of experience working with ict, their intensity of use of ict and 'how comfortable' they felt using ict. the second section asked respondents to rate (using a six point likert scale) how important each of thirty eight attributes were in ensuring the effective delivery of their organisation's is/t strategy and is/t service (see table 2 for a list of the attributes). the third section asked respondents to rate (using a six point likert scale) their organisation's performance on the same thirty eight attributes. in sections four and five respectively, respondents were asked to identify how intensively they used various systems in their organisation and how effective they thought those systems were. the sixth section contained questions to identify how well users had been involved in the process of is/t development; how well existing ict met users' managerial and operational needs; users' satisfaction with the training they had received; users' views on how well service level agreements work; and, users' overall opinion of their ict department. the final section gave the users the opportunity to state how they thought the effectiveness of the is/t service in their organisation could be improved using an open ended question. an assessment of the importance attributes affecting ict effectiveness and a user assessment of organisational performance on these attributes the main objective of the research was to identify the degree of importance that users ascribed to each of thirty eight attributes integral to the design, development and delivery of an is/t strategy in their local authority. the mean score on each of the thirty eight attributes was calculated and the attributes were then ranked to give an impression of the relative importance that users ascribed to each of the attributes. the results of this analysis, ranked by the perceived importance of attributes, are shown in table 2. it is noticeable that the most highly rated attributes related, not to the technological components of the is/t strategy, but to the 'softer' aspects of the is/t strategy such as a high degree of technical competence from support staff, ease of access to facilities, fast response from support staff to remedy problems, user confidence in systems and a positive attitude by support staff to users. the majority of the top ten rated attributes were related to the quality of the support service that the ict department provided. a second task was to measure how well users' perceived their organisation's is/t strategy to be performing on the same thirty eight attributes. the average performance score and the average importance score on each of the thirty eight attributes are shown in table 2. a trace of these scores, both performance and importance, for each of the thirty eight attributes is shown in figure 1. 75 ajis vol. 8 no. 1 september 2000 table 2: respondent's assessment of the importance of each attribute and an assessment of current performance. attribute a high degree of technical competence in systems support staff. ease of access for users to computing facilities. user confidence in systems. fast response time from systems support staff to remedy problems. provision for disaster recovery. positive attitude of information systems/ict staff to users. a low percentage of hardware and software downtime. system's response time. systems responsiveness to changing user needs. extent of user training. participation in the planning of the system's requirements. users' understanding of the system. ability of the system to improve personal productivity. the quality of the reports delivered to the user. the use of windows type software. systems analysts who know the user's business. flexibility of the system with regards to both data and reports. users' willingness to find time to learn the system. prompt processing of requests for changes to the existing systems. up-to-dateness of software. confidentiality of user's own data. flexibility to produce professional reports (e.g. graphics and desktop publishing). overall cost-effectiveness of information systems. the monitoring of the ict department's performance in delivering services to users. the alignment of the information systems plan with the overall corporate plan. up-to-dateness of hardware. documentation to support training. the degree of personal control users have over their systems. procedures for avoiding software piracy. short lead times for the development of new systems. the measurement of benefits derived by the user from the system. ability of the system to enrich the working experience of the user. help with database or data model development. standardisation of hardware. the use of a service level agreement with the ict department. increasing the portfolio of applications. access to external databases through the system. ability to conduct computer conferencing with colleagues. importance mean score 5.6 5.5 5.5 5.5 5.3 5.3 5.2 5.2 5.1 4.9 4.9 4.9 4.9 4.9 4.8 4.8 4.8 4.7 4.7 4.6 4.6 4.6 4.6 4.6 4.6 4.5 4.5 4.3 4.2 4.2 4.2 4.1 4.1 3.9 3.9 3.9 3.4 2.8 performance mean score 4.2 4.5 3.9 3.8 3.6 4.2 3.9 3.9 3.5 3.3 3.6 3.7 4.1 4.1 4.6 3.5 3.7 3.7 3.4 4.1 4.4 3.9 3.6 3.1 3.4 4.1 3.5 3.7 3.6 2.8 3.1 3.8 3.2 3.6 3.1 3.5 2.7 1.9 76 ajis vol. 8 no. 1 september 2000 figure 1: a trace of the average performance (lower line) and importance (upper line) scores on each of the thirty eight attributes 1 • 10 15 20 25 attributess 30 35 40 table 2 and figure 1 show that in all cases the average importance score was higher than the average performance score and that the two curves mirrored each other closely. what is most important is the distance between these two measures evaluated in the context of the relative importance of attributes. to this end, a "gap" measure was constructed which was the individual's performance score minus their importance score on each attribute. the gap between importance and performance on each attribute is shown in table 3. a negative score is considered to denote the width of the "performance deficit" on each measure. 77 ajis vol. 8 no. 1 september 2000 table 3: the ranked importance of attributes and the difference between importance and performance attribute a high degree of technical competence in systems support staff. ease of access for users to computing facilities. user confidence in systems. fast response time from systems support staff to remedy problems. provision for disaster recovery. positive attitude of information systems/ict staff to users. a low percentage of hardware and software downtime. system's response time. systems responsiveness to changing user needs. extent of user training. participation in the planning of the system's requirements. users' understanding of the system. ability of the system to improve personal productivity. the quality of the reports delivered to the user. the use of windows type software. systems analysts who know the user's business. flexibility of the system with regards to both data and reports. users' willingness to find time to learn the system. prompt processing of requests for changes to the existing systems. up-to-dateness of software. confidentiality of user's own data. flexibility to produce professional reports (e.g. graphics and desktop publishing). overall cost-effectiveness of information systems. the monitoring of the ict department's performance in delivering services to users. the alignment of the information systems plan with the overall corporate plan. up-to-dateness of hardware. documentation to support training. the degree of personal control users have over their systems. procedures for avoiding software piracy. short lead times for the development of new systems. the measurement of benefits derived by the user from the system. ability of the system to enrich the working experience of the user. help with database or data model development. standardisation of hardware. the use of a service level agreement with the ict department. increasing the portfolio of applications. access to external databases through the system. ability to conduct computer conferencing with colleagues. importance 5.6 5.5 5.5 5.5 5.3 5.3 5.2 5.2 5.1 4.9 4.9 4.9 4.9 4.9 4.8 4.8 4.8 4.7 4.7 4.6 4.6 4.6 4.6 4.6 4.6 4.5 4.5 4.3 4.2 4.2 4.2 4.1 4.1 3.9 3.9 3.9 3.4 2.8 gap -1.38 -1.01 -1.49 -1.65 -1.67 -1.07 -1.32 -1.20 -1.64 -1.64 -1.39 -1.09 -0.75 -0.73 -0.23 -1.25 -1.07 -1.02 -1.35 -0.51 -0.13 -0.66 -0.99 -1.56 -1.17 -0.32 -0.97 -0.58 -0.58 -1.32 -1.00 -0.31 -0.89 -0.24 -0.85 -0.43 -0.69 -0.90 table 3 was used to identify those measures which were rated both as important and as having a high importance-performance gap ("performance deficit"). an arbitrary importance score of 4.7 or above and an importance-performance gap score of-1.20 or above were taken as cut-off points and eleven of the thirty eight measures were isolated. these attributes are listed below (ranked by importance): 78 ajis vol. 8 no. 1 september 2000 • a high degree of technical competence in systems support staff • user confidence in systems • fast response time from systems support staff to remedy problems • provision for disaster recovery • a low percentage of hardware and software downtime • system's response time • systems responsiveness to changing user needs • extent of user training • participation in the planning of the system's requirements • systems analysts who know the user's business • prompt processing of requests for changes to the existing systems the outcome of this analysis was the isolation of those measures deemed most important by users where the organisation's is/t department has the highest performance deficit. factor analysis of the importance-performance scores the interpretation of the thirty eight importance, satisfaction and gap scores proved to be somewhat difficult and so it was decided to apply a data reduction technique (factor analysis) to the importance and performance data in an attempt to: • identify the latent structure of the satisfaction and importance scores; • create a set of composite variables (factors) to make the analysis more understandable; and, • provide a base for the comparative analysis of the six local authorities. the results of the factor analysis of the importance measures are shown in table 4. table 4: "importance" factors and their related attributes factor 1: systems support increasing the portfolio of applications. flexibility of the system with regards to both data and reports. systems analysts who know the user's business. short lead times for the development of new systems. prompt processing of requests for changes to the existing systems. factor 2: training and service monitoring the monitoring of the ict department's performance in delivering services to users. documentation to support training. extent of user training. factor 3: support staff positive attitude of information systems/ict staff to users. user confidence in systems. a high degree of technical competence in systems support staff. factor 4: up-to-datedness up-to-dateness of software. up-to-dateness of hardware. factor 5: enhancing personal effectiveness and alignment ability of the system to improve personal productivity. overall cost-effectiveness of information systems. the alignment of the information systems plan with the overall corporate plan. factor 6: job enrichment and control ability of the system to enrich the working experience of the user. the degree of personal control users have over their systems. importance score l_ 3'9 4.8 4.8 4.2 4.7 4.6 4.5 4.9 5.3 5.5 5.6 4.6 4.5 4.9 4.6 4.6 4.1 4.3 79 ajis vol. 8 no. 1 september 2000 factor 7: confidentiality and security confidentiality of user's own data. provision for disaster recovery. procedures for avoiding software piracy. factor 8: system responsiveness a low percentage of hardware and software downtime. system's response time. 4.6 5.3 4.2 5.2 5.2 in any data reduction process, information is inevitably lost but, in the above case, the eight factors in table 4 accounted for over 53% of the variance in the data set. each of the factors was given a label to reflect the common characteristics of the variables which correlated most strongly with that factor. the first factor grouped together a set of variables related to increasing the portfolio of applications, increasing system flexibility, having systems analysts who understand the user's business, having short lead times and the prompt processing of requests for changing existing systems. the factor comprising these variables, accounted for over 23% of the variance in the data set. the factor to account for the next highest percentage of the total variance related to training and to the monitoring of the is/t department's performance while the third factor related almost exclusively to the attitude and technical confidence of support staff. a similar type of analysis was undertaken on the performance data set. however, in this case, the factors which emerged from the analysis were somewhat different from those that emerged from the importance data set though there were some similarities. table 5: performance factors and their related attributes factor 1: desktop and reports the quality of the reports delivered to the user. the use of windows software (graphical presentation, use of mouse). flexibility to produce professional reports (e.g. graphics and desktop publishing). factor 2: support staff positive attitude of information systems/ict staff to users. fast response time from systems support staff to remedy problems. a high degree of technical competence in systems support staff. factor 3: training and support help with database or data model development. documentation to support training. participation in the planning of the system's requirements. extent of user training. factor 4: alignment the alignment of the information systems plan with the overall corporate plan. factor 5: facilities up-to-dateness of software. up-to-dateness of hardware. ease of access for users to computing facilities. standardisation of hardware. importance 4.9 4.8 4.6 5.3 5.5 5.6 4.1 4.5 4.9 4.9 4.6 4.6 4.5 5.5 3.9 the five factors listed in table 5 accounted for over 49% of the variance in the data set with factor 1 accounting for 30.8% of the total variance. the factor which accounted for the highest share of the variance related almost exclusively to reporting at the desktop as the three variables which contributed most to this factor are the quality of reports, the use of windows type software and system flexibility to produce professional reports. the second factor on the performance data mirrored factor 3 using the importance data with the positive attitude of ict staff and high degree of technical competence being shared by both factors. factor 3 on the performance data isolated variables related to training (the 80 ajis vol. 8 no. 1 september 2000 extent of user training and documentation to support training) and thus has some similarities with factor 2 of the importance data set. the correlation and regression analysis of overall satisfaction and performance scores on the importance factors indentifeed by the factor analsis if an ict facility is to be managed to improve the overall level of user satisfaction with that organisation, it is important to be able to assess the relative contribution of each of the eight factors to overall satisfaction. to conduct this analysis, eight new variables were constructed (labelled pfac 1 to pfac 8) reflecting the eight importance factors shown in table 4. an average score for each respondent based on their performance scores was constructed for each of the attributes that were shown to load onto each importance factor. the variables constructed and their descriptions are shown below: pfac1 systems support and the effective meeting of user needs pfac2 training and service monitoring pfac3 support staff attitudes and technical competence pfac4 up-to-dateness of hardware and software pfac5 enhancing personal effectiveness and alignment pfac6 job enrichment and control pfac7 confidentiality and security pfac8 system responsiveness the correlation of each of the above factors with the overall satisfaction score is given in table 6. it can be seen that pfac3, pfac1 and pfac2 were most strongly correlated with users' overall view of their ict department. table 6: overall satisfaction and its relationship with the pfac variables factors pfac1 pfac2 pfac3 pfac4 pfac5 pfac6 pfac7 pfac8 correlation with overall satisfaction 0.486 ** 0.400 ** 0.585 ** 0.300 ** 0.310** 0.257 ** 0.309 ** 0.190** 1 significant at the 5% level regression analysis was then used to explain statistically overall satisfaction with the organisations' ict department and its delivery of services by reference to individuals' scores on the constructed pfac variables. the output of the regression analysis (using the backwards elimination procedure1 ) to explain users' overall satisfaction with their ict department is shown below: os = 0.78 h (t=4.14) 0.25pfac1 (t=4.38) 0.54pfac3 (t=10.10) r2 = 0.365 16 by using backwards elimination, the regression analysis begins with an equation comprising all the variables included in the initial regression set (the eight factors). variables which are found to have little explanatory power are then successively excluded until the most parsimonious model is derived. 81 ajis vol. 8 no. 1 september 2000 the above equation shows that pfac3 (support staff) and pfac1 (systems support) largely explain the overall user satisfaction with the is/t department and the services it provides. pfac3 (support staff attitudes and technical competence) explained most of the variation in the overall satisfaction variable. however, as the r2 derived from the equation was only 0.365 this implies that the explanation of overall satisfaction is somewhat more complex than our most parsimonious regression model would suggest. the analysis has revealed that the thirty eight importance attributes can be collapsed into eight factors which collectively account for 53.4% of the total variance in the data set. the eight factors were then used to construct composite variables with the objective of examining the explanatory power of performance of these variables on a respondent's overall level of satisfaction with the organisation's ict department. it was discovered that two composite variables systems support and support staff accounted for almost 40% of the variation in overall satisfaction with the organisation's ict department and its delivery of an is/t service. these factors comprised the following raw variables: • positive attitude of information systems/ict staff to users. • user confidence in systems. • a high degree of technical competence in systems support staff. • increasing the portfolio of applications. • flexibility of the system with regards to both data and reports. • systems analysts who know the user's business. • short lead times for the development of new systems. • prompt processing of requests for changes to the existing systems. of most importance here is the finding that a positive attitude of is/t staff, user confidence in systems and a high degree of technical competence in support staff emerge as the three most powerful variables which need to be acted upon to secure a high level of user satisfaction with the overall delivery of is/t services in the uk local government setting. a comparison of the six local authorities and their performance on the importance factors in order to assess the relative strengths and weaknesses of the six local authorities within the study, an analysis of how each local authority performed on the eight composite variables derived from the factor analysis was undertaken. the mean score of each local authority on each of the eight factors was computed and the results are shown in table 7. the data in table 7 are shown graphically in a radar diagram (figure 2) as departures from the overall mean of each of the eight factors. table 7: local authority scores on the eight factors la banton brinton dunton esston posham wrenton total pfac1 2.94 3.89 3.52 3.60 3.32 3.21 3.41 pfac2 3.34 3.48 3.25 3.70 3.21 2.86 3.31 pfac3 3.68 4.24 4.07 4.41 4.14 4.25 4.13 pfac4 3.49 4.46 4.13 4.16 4.53 4.05 4.14 pfac5 3.53 3.89 3.84 4.05 3.51 3.41 3.70 pfac6 3.33 4.06 3.50 3.91 4.09 3.77 3.78 pfac7 3.45 4.23 4.11 4.12 3.74 3.72 3.89 pfac8 3.51 4.00 3.73 4.51 3.78 4.16 3.95 82 ajis vol. 8 no. 1 september 2000 figure 2: radar diagram depicting relative authority performance on the eight factors pfac8 pfac2 pfac7 i pfac3 pfac6 pfac4 pfac5 table 7 and figure 2 reveal that statistically significant differences existed between the six local authorities. (these differences were identified using anova together with the bonferroni multiple comparisons test at the 5% level of significance). from this analysis, the london borough of banton emerged as the local authority which fared worst on the majority of the eight factors (pfac1, 3, 4, 6, 7, 8). on pfac4 (up-to-dateness of hardware and software), the bonferroni procedure also showed that banton was significantly different from the other five local authorities; on pfac6 (job enrichment and control) and pfac3 (support staff attitudes and technical competence) banton was significantly different from all local authorities except dunton; on pfac1 (systems support: the prompt and effective meeting of user needs) banton was significantly different from all local authorities other than wrenton; and, on pfac7 (confidentiality and security) and pfac8 (system responsiveness) banton was significantly different from three of the other five local authorities. the radar diagram (included as figure 2 where scores are plotted in terms of distance from the average scores for all local authorities) shows clearly that there were radical differences between the traces of the six local authorities. eston and brinton scored highly on the majority of the derived variables while wrenton, posham and dunton occupied the middle ground with banton being some distance behind. our analysis has revealed substantial differences across the six local authorities which, in an era when benchmarking and cross-organisational comparisons are becoming the norm, indicates the potential use of these tools both to assess the relative strength of the determinants of user satisfaction with overall performance and to expose differences between organisations on the same set of variables derived from the eight importance factors. user involvement, meeting managerial and operational needs and overall satisfaction the preceding analysis has exposed significant differences between local authorities on the derived variables constructed out of the thirty eight measures. here, we augment the analysis by looking for differences on a number of attitudinal variables about the extent to which users felt that they had been involved in influencing ict developments; about whether users felt that ict met their managerial and operation needs; and, users' overall opinions about the ict department. an analysis of responses to these questions is presented in tables 8, 9, 10 and 11. 83 ajis vol. 8 no. 1 september 2000 table 8: degree of user involvement in influencing ict development as a % high 2 3 4 5 low no reply eston 31.7 29.3 19.5 11.0 6.1 1.2 1.2 banton 19.5 25.6 17.1 15.9 11.0 7.3 3.6 dunton 14.5 18.1 24.1 14.5 20.5 8.3 0.0 brinton 11.1 17.3 27.2 11.1 21.0 9.9 2.4 posham 9.9 21.0 17.3 13.6 28.4 9.8 0.0 wrenton 15.9 22.0 28.0 12.2 12.2 9.7 0.0 total 17.1 22.2 22.2 13.0 16.5 7.7 1.3 responses on the user involvement in influencing ict developments measure varied considerably across the six local authorities ranging from a high of 61% scoring their organisation at 1 or 2 in eston to a low of 28% in brinton. a regression analysis of this variable with the eight pfac variables, however, has low explanatory power (r2 0.124). an analysis of the inter-relationship between the degree of user involvement and overall satisfaction does reveal that the independence hypothesis must be rejected and that user involvement is a strong component in building overall satisfaction (chi square score of 185.8 with 36 degrees of freedom). table 9: how well does ict meet managerial needs as a % high 2 3 4 low no reply eston 7.3 42.7 23.2 18.3 4.9 3.6 banton 6.0 16.9 19.3 36.1 16.9 2.8 dunton 12.3 16.0 40.8 24.7 6.2 0.0 brinton 2.4 30.5 32.9 20.7 9.8 3.7 posham 0.0 27.7 37.3 16.9 7.2 10.9 wrenton 0.0 25.3 39.8 15.7 15.7 3.5 total 4.7 26.5 32.2 22.1 10.1 4.4 response on this measure again varied considerably from a high of 50% in eston (scoring 1 or 2) to a low of 22.9% in banton. the relationship between this variable and overall satisfaction was also relatively strong (with a chi square score of 222.7 at 36 degrees of freedom) indicating that an inability to meet managerial needs has a substantial effect on perceived overall satisfaction. table 10: how well does ict meet operational needs as a % high 2 3 4 5 low no reply eston 8.5 47.6 22.0 14.6 4.9 0.0 2.4 banton 3.7 17.1 23.2 32.9 13.4 6.1 3.6 dunton 9.8 32.9 32.9 15.9 6.1 2.4 0.0 brinton 3.7 26.8 30.5 25.6 11.0 0.0 2.4 posham 3.7 41.5 30.5 7.3 9.8 3.7 3.5 wrenton 3.6 31.3 31.3 18.1 6.0 3.6 6.1 total 5.5 32.9 28.4 19.1 8.5 2.6 3.0 there were very strong differences between local authorities in terms of the extent to which ict was perceived to meet the organisation's operational needs. this ranged from 56.1% (scoring 1 or 2) in eston to 20.8% in banton. banton was substantially below the other five local authorities. the relationship between meeting operation needs and overall satisfaction with the ict department was particularly pronounced (with a chi square score of 233.6 at 36 degrees of freedom). 84 ajis vol. 8 no. 1 september 2000 table 11: overall opinion of the ict department in your organisation poor 2 3 4 5 excellent no reply eston 1.2 8.5 13.4 39.0 32.9 2.4 2.6 banton 6.0 13.3 22.9 42.2 13.3 0.0 2.3 dunton 0.0 2.4 18.3 46.2 30.5 2.4 0.0 brinton 0.0 3.7 17.1 32.9 43.9 2.4 0.0 posham 0.0 16.9 7.2 48.2 24.1 3.6 0.0 wrenton 0.0 12.0 21.7 31.3 31.3 0.0 3.7 total 1.2 9.5 16.8 40.0 29.3 1.8 1.4 given the analysis of the three preceding variables, it is not surprising that there were substantial differences in users' overall opinion of their organisation's ict department and its ability to deliver services to meet users' expectations. reported satisfaction (i.e. the percentage recording a score of 5 or 6) was highest for brinton (46.3%) followed by eston (35.3%) with banton (13.3%) recording the lowest level of overall satisfaction by a substantial degree. an analysis of attitudes and the production of perceptual maps the questionnaire gave respondents an opportunity to give their views on how they thought the effectiveness of their organisation's is/t service could be improved. what follows is a distillation of their responses. the first phase of the exercise comprised a content analysis of the issues that respondents raised: in all seventeen themes were identified (see table 12). table 12: issues identified from the content analysis of the open ended question number 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 name bserv data decent dskill ecom hwork liaise mgt olds plan prod resc sta suppt trn uneeds user description problems with the quality/level of service from ict dept data management problems need for more decentralisation ict dept does not have the skills needed e-mail and internal communications need for better access to systems need for better user/ict dept liaison problems with management of ict problems with old systems concerns about ict planning and strategy productivity concerns problems with resourcing need to standardise on soft/hardware problems with support from ict dept need for better training ict dept does not understand user needs low user knowledge/skill the second stage of the exercise was to construct a 2-way contingency table (table 13) which counted the number of times the above seventeen issues were cited in each local authority. the third stage of the process used correspondence analysis (see greenacre, 1994 for a full discussion of this technique and remenyi and money 1993 and 1994 for its application in is/t strategy evaluation) to explore two themes: • first, to identify which local authorities were alike in terms of the issues raised; and, • second, to analyse the joint relationship between local authority and the issues raised to identify if certain issues tended to cluster around certain local authorities. 85 ajis vol. 8 no. 1 september 2000 table 13: contingency table of responses by the six local authorities name bserv data decent dskill ecom hwork liaise mgt olds plan prod resc sta suppt trn uneeds user sum brinton 3 0 0 1 1 0 2 2 0 3 2 0 1 2 5 3 3 28 dunton 0 0 0 2 2 0 0 1 3 1 0 0 0 3 1 1 2 16 posham 0 4 0 0 0 3 0 2 0 2 0 0 1 2 0 1 1 16 banton 7 2 0 8 8 2 5 13 6 6 3 7 7 8 3 3 4 92 wrenton 5 1 2 3 4 2 5 10 5 6 5 2 3 2 4 3 0 62 eston 6 6 5 11 1 1 4 14 3 9 4 15 4 8 8 3 5 107 total 21 13 7 25 16 8 16 42 17 27 14 24 16 25 14 14 15 321 while a technical discussion of correspondence analysis is beyond the scope of this paper, the graphical output in figure 3 was generated to explore the topics listed in table 12. the points numbered 1-17 in figure 3 relate to the issues listed in table 12 and the triangles represent the locations of the six local authorities whose codes are shown in the table 14. table 14: local authority codes used in figure 8.1 local authority banton brinton dunton eston posham wrenton code ba br du es po wr the graphical output of the correspondence analysis revealed that brinton, dunton, banton and wrenton tended to cluster together thus displaying a degree of similarity with eston being a slight outlier to this group. posham (a rural welsh local authority), was significantly detached from this main group. the advantage of correspondence analysis is that it allows us to explore the relationship between the rows and columns of a contingency table simultaneously. our analysis reveals, for example, that: • in posham, the need for better home-working facilities and data management were viewed as being more problematic than in other local authorities; • in eston, the need for more decentralisation, problems with resourcing, the lack of ict skills, the need for better training and low user skills were seen as relatively problematic 86 ajis vol. 8 no. 1 september 2000 figure 3: perceptual map: local authorities and key issues 1.5 1.0' .5 0.0 -.5 -1.0 -1.5 15,,•4 6 194 17 issue raised local authority -1.0 -.5 0.0 .5 1.0 1.5 2.0 2.5 3.0 dimension 1 conclusions our research has revealed considerable variability in how users perceived their organisation's is/t department as a deliverer of services. analysis has revealed that there were substantial gaps between importance and performance on measures that users considered to be highly important determinants of is/t delivery effectiveness. the use of factor analysis, regression analysis and correspondence analysis have allowed us to focus in on those issues which determine users' overall level of satisfaction with the is/t department in their organisation. in many cases we have found that it is the softer issues such as the positive attitude of information systems/ict staff, user confidence in systems and a high degree of technical competence in systems support staff that do most to explaining overall satisfaction with the ict service. the framework we have developed has been used to provide a diagnostic tool that can be used to help managers focus more clearly on those aspects of their is/t and service delivery strategies that users perceive as important and on improving aspects of service delivery that are most undermining users' perceptions of overall performance. the analysis presented here is exploratory but highly appropriate in an era when local government ict managers are being confronted with outsourcing and the need to benchmark their services against other local authorities and external service providers to demonstrate that they are delivering value. this paper has been derived from a longer research paper which was presented to and then discussed with the ict managers of the six participating local authorities. the outcomes of this session allowed us to assess the extent to which ict managers felt our methods were robust, to discover if managers empathised with our findings (as revealed by their own research, for example, and in the context of their managerial experiences) and to elicit how managers felt our questionnaire and methodology could be refined to meet the particular circumstances of local government. this process of engagement with our research subjects allowed us to "close the circle" as a basis for improving both our questionnaire and our ability to communicate our research findings clearly and effectively. the feedback derived from the six ict managers proved to be supportive of our findings. all ict managers agreed on the need to develop a generic framework that would allow them to benchmark their performance against other local authorities and ict service providers in the private sector as a means of demonstrating their capacity to 87 ajis vol. 8 no. 1 september 2000 deliver value. the research was seen as useful in that it provided a base of evidence for the continued improvement of their service. essentially, the managers indicated that they wanted a framework that would enable them to identify best practice and enable them to understand "why the best were the best". the prevailing view was that while the questionnaire we had used was not perfect and could be made more incisive, the base of evidence we had developed had given the managers a useful insight into how an effective diagnostic tool could be developed which would give them a clear view of the issues on which they needed to focus to deliver satisfaction to the user. more important, they expressed a view that while developing and implementing a research exercise such as ours was not the "end game", it would provide them with the evidential basis for developing a constructive dialogue with users that might subsequently modify the initial research findings as clearer insights were generated from the conduct of that dialogue. the exploration of effectiveness, particularly is/t effectiveness, is a difficult research problem and we hope we have cast some light on this difficult issue. our approach has shown that these issues and problems do have a definable structure and that within uk local government there are considerable inter-authority differences in performance. perhaps most important, we have exposed that the knowledge base and user orientation of ict staff are the most important features in explaining users' overall opinion of their ict department as a deliverer of quality services: an effective is/t strategy can be badly damaged if the people who deliver it do not have good "people skills". references allingham p. & o'connor m. (1992) mis success: why does it vary among users? journal of information technology, vol 7, pp 160-168. attewell p. (1993) technology and the productivity paradox, mimeograph, city university of new york. banker r. & kemerer c f. (1991) research undertaken on 'performance evaluation metrics for information systems development: a principal-agent model', october, 1991. brynjolfsson e. (1993) the productivity paradox of information technology. communications of the acm, vol. 36(12), pp 67-77. goss s. miller c. & symons b. (1993) exploiting the strategic potential of best value. office of public management briefing paper. office of public management. london. greenacre m.j. (1994) the theory and application of correspondence analysis, academic press, new york. hair, j. f., anderson, r.e., tatham, r.l. and black, w. c. (1998) multivariate data analysis with readings. fifth edition, prentice-hall international inc., new jersey hitt l. & brynjolfsson e. (1994) the three faces of it value: theory and evidence'. proceedings of the fifteenth international conference on information systems, vancouver, british columbia, canada, december 14-17,1994. kerlinger f.n. (1969) foundations of behavioural research, holt rinehart and winston, london. lehmann d.r. (1989) marketing research analysis, 3rd edition, richard d. irwin, new york. lester s. & willcocks l. (1993) how do organisations evaluate and control information systems investments? recent uk survey evidence. adapted from a study by the same authors: evaluation and control of is investments: recent uk survey evidence, oxford institute of information management, templeton college, oxford, 1993. parasuraman a. zeithaml v. & berry l. (1985) a conceptual model of service quality and its implications for research. journal of marketing, fall 1985, pp 271-293. parker m.m. (1989) information strategy and economics, prentice-hall, new york. peters g. (1988) evaluating your computer investment strategy, journal of information technology, vol 3(3), pp 154-163. remenyi d. (1991) introducing strategic information systems planning. national computer centre, blackwell, london. remenyi d. & money, a.h. (1991) a user-satisfaction approach to is effectiveness measurement. journal of information technology, vol 3, pp 162-175. remenyi d. & money, ah. (1993) service quality and correspondence analysis as diagnostic tools in determining the problems associated with the effective use of computer services. proceedings of the first european conference on information systems, march 1993 88 ajis vol. 8 no. 1 september 2000 remenyi d. & money a.h. (1994) service quality and correspondence analysis in determining problems with the effective use of computer services. european journal of information systems, vol. 3(1) pp 2-12. remenyi d. money a. sherwood-smith m. & irani z, (2000) the effective measurement and management of it costs and benefits, butterworth heinemann, oxford. remenyi d. sherwood-smith m. with white t. (1997) achieving maximum value from information systems: a process approach. john wiley, chichester. silk d.j. (1990) managing is benefits for the 1990s. journal of information technology, vol. 5, pp 132-147. socitm. (1996) it trends in local government. society of information technology management. university of birmingham. willcocks l. (1991) unpublished chairman's introduction to a conference on managing it investment, conducted by business intelligence, london, 20 may, 1991. wilson d. (1991) assessing the impact of information technology on organisational performance'. in banker r. (ed) strategic information technology management. kauffman p. & maymood m. a. (1993) ideas group, harrisburg, pennsylvania. worrall l. (1994) current issues in information management: a local government perspective. journal of industrial affairs, vol. 3(2), pp 107-119. worrall l. (1995) managing information in complex organisations: a local government perspective. local government studies, vol. 21(1), pp 115-129. acknowledements: the assistance of martin greenwood, consultant to the mapit initiative of the society of information techology managers in negotiating access to the six particpating local authorities is acknowledged. the willingness of the it directors of the six local authorities to participate in the research programme and their constructive comments about our research is also acknowledged. 89 ajis vol. 7, no 2 may 2000 adapting the locales framework for heuristic evaluation of groupware saul greenberg', geraldine fitzpatrick2, carl gutwin3 and simon kaplan2'4 'department of computer science, university of calgary, alberta, t2n 1n4, canada saul@cpsc.ucalgary.ca; www.cpsc.ucalgary.ca/-saul 4department of computer science and electrical engineering / 2crc for distributed systems technology, university of queensland, st. lucia, qld 4072, australia [ger, s.kaplan]@dstc.cdu.au; www.dstc.edu.au 'department of computer science, university of saskatchewan, saskatoon, canada gutwin@cs.usask.ca; www.cs.usask.ca/faculty/gutwin abstract heuristic evaluation is a rapid, cheap and effective way for identifying usability problems in single user systems. however, current heuristics do not provide guidance for discovering problems specific to groupware usability. in this paper, we take the locales framework and restate it as heuristics appropriate for evaluating groupware. these are: 1) provide locales; 2) provide awareness within locales; 3) allow individual views; 4) allow people to manage and stay aware of their evolving interactions; and 5) provide a way to organize and relate locales to one another. to see if these new heuristics are useful in practice, we used them to inspect the interface of teamwave workplace, a commercial groupware product. we were successful in identifying the strengths of teamwave as well as both major and minor interface problems. key words: groupware evaluation, heuristic evaluation, inspection methods, locales framework. introduction hci researchers and practitioners now have a good repertoire of evaluation methodologies that they can use to determine the usability of conventional single user systems. these methodologies range from: those generating quantitative analyses vs. qualitative subjective descriptions; those that are precise but expensive to perform vs. those that are approximate but cheap to do; and those that require user involvement vs. inspection techniques that can be done without user participation. for example, controlled experimentation typically produces objective quantitative results, is precise but expensive, and requires observations of users. in contrast, inspection techniques such as heuristic evaluation and task-centered walkthroughs typically produce qualitative and subjective descriptions of usability problems, are inexact but very cheap to do, and require no user involvement. the point is that evaluators can choose an appropriate methodology to fit the needs and nuances of their situation. in contrast, there is no equivalent set of evaluation methodologies readily available to groupware researchers and practitioners. while some methods have been adapted from other disciplines, these are typically too expensive to use for everyday evaluations, are difficult to learn, or are not appropriate for the questions being asked. as one example, evaluators using ethnographic methods are usually experienced ethnographers, and the studies are almost always expensive in terms of time, logistics and analyses. similarly, controlled experimentation applied to groupware evaluation is difficult to design and analyze due to the variability inherent in group composition and interaction, the lack of proven metrics that are capable of measuring things of interest in groupware, and the resilience of groups in adapting their behaviors to different controlled situations. the consequence is that groupware is difficult to evaluate. as a result, the cscw community now includes as part of its research agenda the development of evaluation methodologies for groupware. for example, some researchers are developing a 'discount' version of ethnography called design ethnography, where observations and analyses are specific to design implications rather than to a general understanding of a culture. other researchers are trying to identify metrics specific to groupware. in this vein, our goal is to see if heuristic evaluation, a cheap but effective evaluation methodology, can be adapted to groupware evaluation. we recognized that existing heuristics, while excellent for evaluating single user systems, do not provide the evaluator with any guidance for discovering problems specific to groupware. thus our first step in pursuing this goal was to create a new set of heuristics. fortunately, we felt that an existing theory of group behavior called the locales framework was a promising place to begin. in this paper, we present our adaptation of the locales framework as heuristics for evaluating groupware. in section 2, we give a brief summary of both heuristic evaluation and the locales framework. in section 3, we list the groupware heuristics adapted from the locales framework. in section 4, we summarize our experiences using these heuristics to inspect the interface of teamwave workplace, a commercial groupware product. we close by outlining the next steps in our research agenda. 102 ajis vol. 7, no 2 may 2000 background in this section, we briefly review the basic premises of heuristic evaluation and the locales framework. further details are available through the references: both are well documented and well-known to the research community. heuristic evaluation heuristic evaluation (he) a widely-accepted discount evaluation method for diagnosing potential usability problems in user interfaces (nielsen 1993, 1994a,b). he defines a particular interface inspection process (nielsen and mack 1994) where several evaluators examine an interface and judge its compliance with recognized usability principles called 'heuristics' (nielsen, 1994a, p. 26). non-compliant aspects of the interface are captured as interface bug reports, where evaluators describe the problem, its severity, and perhaps even suggestions of how to fix it. he is popular with both researchers and industry for a variety of reasons. first, it is low cost: it can usually be done in a few hours, and it does not require any end-user involvement. second, it is easy to learn: the heuristics are well documented and worked examples are available (e.g., nielsen 1994a, b). third, it is becoming part of the standard hci curriculum (e.g., greenberg 1996) and thus known to many hci practitioners. fourth, it can be used fairly effectively by trained non-usability experts (nielsen 1994a). finally, it has a good success rate, where the pooled results of 3-5 evaluators typically identify about 75-80% of all usability problems (nielsen 1994b). what makes he particularly powerful is that considerable thought has gone into the selection of heuristics. starting with a large body of literature of interface guidelines, nielsen (1994b) examined how well particular guidelines explained usability problems held in a database. the result was a set of about ten recommended heuristics, where the set collectively describes the majority of problems commonly found in today's graphical user interfaces. the catch is that these heuristics are very general, such as "prevent errors" (nielsen 1994a). this level of generality means that heuristics arc motherhood statements that serve only to guide the inspection rather than prescribe it: an evaluator must exercise his or her judgement and interpretation skills to uncover particular problems. the locales framework the locales framework was developed as a principled approach to help people understand the nature of social activity and work, and how a locale (or place) can support these activities (fitzpatrick 1998; fitzpatrick, mansfield, and kaplan 1996). in this framework, a social world is a group of people with some common purpose, a site for collaboration, and some means to communicate. a locale is the actual site in which a group collaborates, the actual means by which people communicate, and the actual means by which the work is achieved. for example, a team with the goal of designing a software system would form the social world. if the team met in a meeting room, the room and all its artifacts—its visual and auditory communication, its whiteboard, its table, and the paper within it— would form a physical locale. if instead the team met through a groupware system in conjunction with a telephone, then that would form a virtual locale. while a virtual locale may not be actually realized in physical space, it still provides the site and means for the social world to collaborate. more formally, the locales framework comprises five aspects, as summarized here. further details are in fitzpatrick, mansfield, and kaplan (1996) and the framework is described completely in chapter 8 of fitzpatrick (1998). • locale foundations define a collection of people, resources and artifacts (tools, objects, information) in relation to the central purpose of the social world. • mutuality considers those interactions within locales that maintain a sense of shared place. • individual view over multiple locales acknowledges that individuals can be participating in many locales. • interaction trajectories concern how courses of action evolve over time. • civic structures concern how interactions fit within a broader communal level. deriving groupware heuristics from locales framework nielsen's analysis that resulted in his recommended heuristics were derived from how well they explained usability problems found in single user environments, but not in groupware. to understand why this makes existing heuristics insufficient for groupware evaluation, we first describe the differences between single user usability and groupware usability. we then present new heuristics based on the locales framework. single-user usability is the degree to which a system is effective, efficient, and pleasant to use, given a certain set of users and tasks (e.g. shackel 1990). usability thus emphasizes 'task work': how a person does the domain tasks and activity that produces things like drawings, documents, or models. clearly, a groupware system must also allow task work to proceed effectively, efficiently, and pleasantly. however, groupware must go beyond task work and support teamwork—the work of working together—in order to be truly usable. teamwork involves several activities. for example, group members must communicate, organize joint action, provide assistance, coordinate activity, divide labor, and monitor each other's work. our conception of groupware usability focuses on teamwork, and we define it as the degree to which a groupware system supports the activities of collaboration (gutwin and greenberg 1998). 103 ajis vol. 7, no 2 may 2000 although teamwork also involves social and affective activities, we limit our definition to those that accomplish the mechanics of collaboration: communication, coordination, planning, monitoring, and assistance. while existing usability heuristics could identify problems in task work, they say nothing about teamwork. yet developing new heuristics for group work is not straight forward. nielson had a good corpus of interface development guidelines to work with, but there is no equivalent corpus specific to groupware and teamwork. as an alternative, we decided to use the locales framework as the basis for new heuristics. it seemed amenable to this because (and similar to a corpus of guidelines) it is a general rather than specialized framework for understanding fundamental aspects of teamwork, and because it describes a small set of different but inter-dependent perspectives of the characteristics of teamwork, where each could be reconsidered as a heuristic. it is also one of the few theories or frameworks specifically created with groupware in mind. it has also been validated as both a way to understand existing work practices, and to motivate the design of new systems. thus it is reasonable to expect that it can also be used as a standard against which to evaluate groupware, particularly those that claim to support long-term team activity within some kind of 'space' or 'place'. the locales framework was not developed as a usability evaluation method. rather, it was developed to understand the nature of social practices in the workaday world, and to help inform the design of groupware. still, converting the framework into heuristics is straightforward. using the text and descriptions found in chapters 1 and 8 in fitzpatrick (1998), we recast them as statements that ask whether a groupware's interface allows certain social phenomena to occur. as with nielsen's heuristics, these statements are brief reminders to help guide evaluators. detailed explanation describing the nuances behind each statement are found in fitzpatrick (1998). provide centers (locales) that collect people, artifacts and resources in relation to the central purpose of the social world. a locale provides the site, means and resources for a group to pursue team and task work. locales should be dynamic so they can evolve along with the people, the artifacts, and the purposes that defines them (harrison and dourish 1996). provide awareness (mutuality) within locales that helps people maintain a sense of shared place and that keeps them informed about shared activity. mutuality includes one person's awareness of others, the artifacts comprising the locale, where things are located, and how things are changing (gutwin & greenberg 1999). allow individual views so one can view a locale or aggregate multiple locales as they relate to one's responsibilities, activities, and interests. a particular person should be able to view locales from his or her particular perspective and in a way that reflects their degree of focus and participation. allow people to manage and stay aware of their evolving interactions over time. this includes a group's control over past, present and future aspects of routine and non-routine work; how people coordinate and negotiate plans and activities over time; how people leverage past experiences; how breakdowns are noticed and repaired; and how processes are supported. provide a way to organize and relate locales to one another (civic structures). locales are rarely independent of one another: people need a way to structure the locales in a meaningful way, to find their way between locales, to create new locales, and to remove old ones. 104 ajis vol. 7, no 2 may 2000 ji-'lnl x| ', f8e -edit room tack hep .! ̂ , : nowr> teamwave heuristic evaluation j institute for ihe future foyer'yererttf patrick] draft note) lex 02chi papei ^udy louise^ pde20t) ' 'pointer: * ! •other connected users judy louise (idle 20s) • teamwava heuristic evaluation1 jeteroy patrick. (idte4<n)foyer :;. :''forit' rsae ;!'styfe heuristic evaluation of groupware '•*; understanding and applying groupware heuristics go through the liatnng material to learn how to do heuristic evaluation of groupware then use the concept map tools to do the heuristic evaluation of teamwave the paper can be found here jj draft notes for ozchi paper iuudy louise says: can you walk me through the baining session? l|saul says: sue just phone me at 222-3456 so we can talk figure 1. a snapshot of a teamwave workplace room the locales framework would not claim that all groupware applications should support all heuristics. rather, the heuristics point to aspects that might be useful in a total collaborative work context, depending on the specific shared task needs. still, evaluation of groupware independent of known tasks and total use context can suggest areas of system strengths, and also areas of weakness that might need to be compensated for in other ways. case study: evaluating teamwave workplace we performed a heuristic evaluation on a groupware system to give us a sense of whether the new heuristics are useful in practice we chose as a case study teamwave workplace (http://www.teamwave.com), a state-of-the-art commercial internet groupware product based on a room metaphor. there were several reasons for our choice. 1. team wave is designed on a spatial metaphor, while the locales framework promotes a place metaphor. because there are strong similarities (as well as strong differences!) between these metaphors, we felt that the heuristics would be particularly appropriate for evaluating teamwave. 2. teamwave is a groupware environment designed to support teamwork; users then craft this environment to fit specific task work needs. this emphasis of teamwork over task work matches the emphasis of the heuristics. 3. some of the authors were involved in the development of teamrooms, a research precursor of teamwave (roseman and greenberg 1996). others had contributed to the locales framework. although independently produced, we were collectively familiar with the framework and the design rationale behind the software. 4. two of the authors had previously debated against each other as members of a conference panel: one argued for groupware design based on the locales framework, while the other argued for design based around a room metaphor as provided in teamwave. we wanted to see if the heuristics would help us overcome our biases and come to a common understanding of the strengths and problems found in team wave's room metaphor. 5. we believe teamwave to be one of the best collaborative environments and thus appropriate to analyze. a brief description of teamwave workplace teamwave workplace is a groupware environment designed around a spatial metaphor. its users can create any number of 'rooms', each being a bounded space. when two or more people enter a room, they are immediately connected together: all people within the room then see the same thing. within a room, people can write and draw 105 ajis vol. 7, no 2 may 2000 on the back wall, and can add one or more groupware applications into a room. for example, figure 1 illustrates a room crafted for a team whose purpose is to learn groupware evaluation and to perform a heuristic evaluation. the 'slide presentation' application contains a training presentation; several 'postit' applications along with arrows drawn on the back wall give instructions; a 'concept map' tool records the group's evaluation, and a 'doorway' leads to a related room that contains a draft of this paper. because rooms are large, people can only see part of its contents: a radar overview (bottom left) provides awareness of the space by showing a miniature of the entire room. people within a room can textually chat with each other (bottom pane). they can also see: who else is in this room (top right pane); a list of other rooms and their occupants (lower top pane). people are also notified of who else is in the community and their comings and goings (top right). finally, rooms are persistent: when people leave and reenter a room at a later time, its contents remain unchanged (unless someone had changed things within a room in the interim). there are many other features of teamwave: full descriptions as well as trial versions of the software can be obtained from www.teamwave.com. problems identified through the heuristic evaluation, we found that teamwave workplace supports many, but not all of the properties described in the locales framework. its major strengths and problems are summarized below. provide centers (locales). one of the strengths of teamwave is that it provides a locale to the group through its familiar room metaphor. as a locale, the room is the site, and the applications added to a room are the means for doing the work. most real time collaborative resources are almost free within a room: all people have to do is enter a common room. because new applications can be added and removed at will, it seems reasonable that the room as locale can evolve along with the people and purposes that define it (as illustrated in figure 1). on the down side, teamwave's only resource for direct communication between people within a room is a chat facility. because audio and video communication must be done through other means, the room as locale lacks an essential resource to support rich communications. to be fair to teamwave, we should add that this was a conscious decision on their part: the developers were constrained because they wanted teamwave to be able to work over modest speed communication links (e.g., modems); they wanted to avoid the many platform dependent issues with today's audio and video standards. they also felt that existing communication channels, such as a telephone, would be an adequate replacement for the chat facility when required. provide awareness within locales. teamwave provides several mechanisms for awareness. people looking at the same portion of a room see all actions as they occur in real time, including each other's telepointer movements. the radar overview provides awareness of the other parts of the room,. as well, people are kept informed via notifications in the top panes of other people's coming and goings. in spite of this, there are problems. the radar overview is too small to give a good sense of another's activity in a different corner of the room. the text-based notifications of people's arrival and departures are easily missed, especially if a user decides to save screen space by resize this information pane to a single line. finally, telepointer tracking (which provides awareness of where others are and what they are about to do) is difficult because they are too small and their movements somewhat jerky. allow individual views. teamwave allows people within a room to look at different parts of it, and lets an administrator assign different levels of access permissions to particular people within a room. aside from this, different people cannot have differing views of a room. thus the teamwave approach is only reasonable if people consistently share similar responsibilities, activities and interests. teamwave also provides no good way to aggregate multiple locales. although users can copy an application to another room, the hard walls of a room essentially bind and separate locales into discrete entities. people should be able to manage and stay aware of their evolving interactions. because rooms and their contents persist, each room is a dynamic entity that evolves over time. thus being aware of the current room state is easy. however, a large and perhaps serious problem is that people have almost no awareness of what others are doing when they do not share the same room: there is no indication of what those people are working on, whether they are active, and so on. in terms of understanding past actions, teamwave provides a version control system that allows people to list and visit a set of previous 'snapshots' taken of a room and its contents. however, the interface to the version control system is awkward, making it unwieldy to stay aware of prior changes e.g., what changes happened since a person was last in the room, who else had entered the room previously, who had performed particular changes, etc. finally, teamwave provides little in the way of process management; processes must be defined and supplied (if at all) by social convention or via individual applications. provide a way to organize and relate locales to one another. except for the linking of rooms by doorways, there is no way to organize and relate rooms together. at the top level, all rooms are presented as a single, flat list. nor is there a way to provide a different structure for different people. 106 ajis vol. 7, no 2 may 2000 discussion after applying these heuristics to teamwave workplace, we felt that we had identified several design problems. in particular, the 'hard walls' inherent in its room metaphor limited awareness across rooms. it also prohibited a person's ability to aggregate multiple locales to fit a particular perspective. the fixed nature of space within a room also disallowed individual perspectives of a room. as well, the heuristics helped us realize that teamwave did not have any good ability to structure rooms together. rooms also tended to present themselves as very much in the here and now; it was awkward to see how things changed over time, and no process could be established to create a structure for how people would manage themselves and their artifacts as the room evolved. finally, the heuristics helped identify smaller usability bugs, such as the fact that the radar overview is too small to provide good awareness, and that notifications of people's arrival and departures may be overlooked. of course, the severity of these problems has not been discussed; if these aspects prove important for the task at hand e.g., rich communication channels, individual views, etc., then teamwave should be modified, or complementary tools and facilities added, or social workarounds developed. finally, we still believe teamwave to be one of the richest collaborative environments today. while the heuristics identified problems, they also suggest that teamwave supports the basic idea of locale; the system just lacks a few of the subtle aspects that accompany the idea of locale. summary and future work our initial development and application of groupware heuristics are promising, as indicated by our case study. although familiar with teamwave, the heuristic evaluation indicated potential problems that we would have missed (although the problem severity is likely to be task and situation dependent), and hinted at what else might be needed to support rich collaboration. of course, much is left to do. in keeping with nielsen's development of the heuristic evaluation method, we (as a community) need to create databases of groupware usability problems and use this database to see how well these particular heuristics, as well as others, serve to identify major problems. we must also examine how well people can use these heuristics in practice; while conventional heuristics are easy to learn and apply, the subtle nuances of teamwork implied by these groupware heuristics may require a greater level of evaluator training and experience. finally, we have to understand whether these heuristics are applicable to task-based groupware, which comprise the majority of existing groupware systems today (e.g., shared whiteboards, shared editors, brainstorming tools, etc.). most of these applications would fail miserably when judged by these heuristics. for now, it is likely that the locales-based heuristics are best used for evaluating design subtleties in general groupware environments (such as teamwave workplace) rather than specific groupware applications. acknowledgements this research was sponsored in part by the national institute of standards and technology, by microsoft research, by the national science and engineering research council of canada, by the cooperative research centre program through the department of industry, science & resources of the commonwealth of australia, and by the united states defence advanced research projects agency under grants f30602-96-2-0264 and f30603-94-c-0161. the views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, expressed or implied, of the defence advanced projects research agency or the u.s. government. we are indebted to teamwave inc. for supplying the software. references fitzpatrick, g. (1998) the locales framework: understanding and designing for cooperative work. phd thesis, department of computer science and electrical engineering, the university of queensland. fitzpatrick, g., mansfield, t. & kaplan, s. (1996) locales framework: exploring foundations for collaboration support. proceedings of the ozchi '96 sixth australian conference on computer-human interaction,hamilton, new zealand, november 24-27, 34-41. greenberg, s. (1996). teaching human computer interaction to programmers. acm interactions, 3:4, 62-76. gutwin, c. & greenberg, s. (1999) a framework of awareness for small groups in shared-workspace groupware. tr99-1, dept comp science, u saskatchewan, canada, www.cpsc.ucalgarv.ca/grouplab/papers gutwin, c. & greenberg, s. (1998). the effects of workspace awareness support on the usability of real-time distributed groupware. rr-98-632-23, dept computer science, u. calgary, canada. www.cpsc.ucalgary.ca/grouplab/papers harrison, s. & dourish, p. (1996) re-place-ing space: the roles of place and space in collaborative systems. proc conference on computer supported cooperative work, 67-76, acm press. nielsen, j. (1993). usability engineering. academic press. 107 ajis vol. 7, no 2 may 2000 nielsen, j. (1994a). heuristic evaluation. in nielsen and mack (1994) nielsen, j. & mack, r.l. (1994). usability inspection methods. wiley nielsen, j. (1994b). enhancing the explanatory power of usability heuristics. proc conference on human factors in computing systems, 152-158, acm press. nielsen, j. & mack, r.l. (1994). usability inspection methods. wiley. roseman, m. & greenberg, s. (1996). teamrooms: network places for collaboration. proc conference on computer supported cooperative work. acm press. shackel, b. (1990). human factors and usability. in human-computer interaction: selected readings. j preece and l keller, eds. prentice hall, 1990. 108 evaluating the benefits of electronic commerce in small and medium enterprises peter marshall and judy mckay mt eliza business school melbourne pmarshal(ti)inteliza.com.au monash university melbourne iudv.mckav@sims.monash.edu.au abstract this paper outlines and analyses the behaviours of small and medium enterprises with respect to the evaluation of electronic commerce investments, and the subsequent realisation of the anticipated benefits from those investments. the responses of executives who participated in the study suggested that on-going involvement in electronic commerce did not come cheaply. nonetheless, our study suggests that there were generally ad hoc approaches to evaluation of the proposed electronic commerce investments, almost non-existent post-implementation reviews, few measures of success, and generally speaking, there was little evidence of there being proactive management of the realisation of benefits of those investments. perhaps not surprisingly, the group interviewed seemed somewhat disappointed with the outcomes of their electronic commerce initiatives. the implications of these findings are discussed, particularly for countries where small and medium enterprises contribute a large proportion of total economic activity, and where thus it is vital for small and medium enterprises to successfully engage in electronic commerce if the country is to derive the benefits of the new economy. keywords: electronic commerce, it evaluation, benefits management, small and medium enterprises introduction it appears that somewhat of a dichotomy exists in many contemporary organisations with respect to the question of investment in information and particularly in information technology (it). on the one hand, discussion of the new information-based economy and the promise of the new e-business domain lead inevitably to enormous faith being placed in it, or perhaps more accurately, on the critical, appropriate utilisation of it to deliver business benefits. such faith is illustrated by quotes such as: "across all industries, information and the technology that delivers it have become critical, strategic assets for business firms and their managers" (laudon and laudon 2000). but such enthusiasm is tempered by another view or concern that it is not delivering on its promises, that it is "oversold and under delivered" (earl 1994), and that demonstrating the business value of it investment is difficult in many instances. this concern that managers do not perceive that they are deriving value for money when it comes to it investments is troubling when information and it are often presented as the very backbone of the new economy. such cynicism is reflected in quotes such as: "there are many different ways to ruin a company. speculation is the fastest, it is the most reliable" (kempisetal. 1999). why do we experience such conflicting attitudes? why is there so often a gap between aspirations with respect to it, and the reality of it implementations in many organisations? more importantly, can a sensible way forward be found, such that managers can develop greater confidence in their it investment decisions? this paper presents and discusses the results of a survey on the practices of small and medium enterprises (smes) with respect to the evaluation of their electronic commerce (ec) investments (both pre-investment and post-implementation) and their behaviours in terms of managing the realisation of the anticipated benefits of those ec investments. the study aimed to find out whether, and how, managers in smes evaluated proposals for investment in ec technologies, and whether they have mechanisms in place to ensure the realisation of business benefits to the organisation once that investment has been made. our aim was to better understand whether these managers felt they were deriving "value for money" with respect to their ec initiatives, and to better understand whether (and how) expected benefits from ec investments are realised in practice. for the purpose of this study, the focus was on ec investments, and thus a distinction was drawn between ec and is/it investments. some would argue that this distinction is false, that it is unfruitful to attempt to separate out interorganisational and intraorganisational business processes and activities, and more specifically the is/it investments which support and facilitate such business transactions, interactions and decision making (zwass 2001). while we concede that such a distinction is not always nor necessarily clear cut, and that indeed, over time, it may fade altogether, previous research has lead us to the view that firstly, owner-managers of smes are quite conceptually clear on those is/it investments which have been initiated essentially through the advent of the internet (see marshall et al. 2000, valli and marshall 2001, for example), and those which are being made to 135 support more traditional business activity. secondly, experience has suggested that the internet has directly motivated is/it expenditure in smes, in some cases in organisations where comparatively little is/it expenditure had ever taken place before: it was on this type of investment that we wished to focus. thus, the former we call "ec investments" for the purpose of this research, essentially "front office" investments which enable an internet presence, are required to sustain that presence, and which were essentially driven by the rise of the internet. the authors subscribe to the view that benefits do not automatically accrue to an organisation from investment in technology. rather, the view held is that technology investment should be guided by clear business objectives indicating the need for is/it for their achievement. furthermore, not only should it requirements be based on business imperatives, but they should also be subjected to a rigorous, strategy-driven process of on-going, multifaceted evaluations linked to proactive benefits realisation, which acknowledge and imply the need to be linked to organisational change management. thus, an is/it strategy process links is/it investments to business strategy and then these proposed investments are subjected to a process of evaluation, benefits management and change management. in this way, through appropriate is/it investment and sound governance of those investments will business benefits accrue to the organisation (mckay and marshall 2001). thus, pre-investment, benefits identified as stemming from an ec investment must be regarded as potential benefits only (that they are expected to accrue to the organisation), but subsequent events and managerial actions will impact on whether or not they are actually realised. arguably, appropriate evaluation practices and proactive management of the realisation of benefits will help translate potential benefits into benefits derived for the organisation (ward et al. 1996) (see figure 1). evaluation required potential benefits of '• ' i -ec investments • • / ; pre-investment development and/or acquisition post-implementation time figure 1: need for evaluation and active realisation of benefits from ec investments background there can be little doubt that ec has had, and will continue to have, a significant impact on the commercial, social and economic fabric of society. the past decade has seen exponential growth rates in most metrics of ec (us department of commerce 1999; andersen consulting 1998, docita 2000a), and generally speaking, most forecasts predict that this growth will continue, if not increase, over the next few years (docita 2000b, ernst & young 2000a). this rise in the importance of the internet both as a source of information exchange and commerce leads us inexorably closer to a truly global community, and hence notions such as globalisation and a global economy increasingly manifest themselves in actual commercial and information exchanges. for small and medium enterprises (smes), the internet has much to offer as a business vehicle as it can serve as a relatively efficient and effective channel for information provision and exchange, advertising, marketing, completing transactions, and in some cases facilitating or directly supporting the distribution of goods and services to global trading partners and customers (czerniawska & potter, 1998, ernst & young, 2000b, oecd 1998). however, concerns are being voiced in many quarters about the relatively slow uptake of electronic commerce by smes (small business index 1999). it appears that smes are yet to be convinced of the merits or potential of ec, nor do they have appreciation of the value of global communication and information infrastructures (behendorf et al 1996). some pundits argue that smes do not always seem to appreciate that ignoring ec (because the current level of transactions is relatively low) puts them at risk of becoming uncompetitive in the medium and long-term, and hence of failing in the future (crawford 1998), perhaps succumbing to more aware and aggressive international operators. this is of particular concern in counties and regions where a substantial proportion of business (and hence economic) activity can be attributed to the enterprise of smes. in such countries and regions, then, it could be argued that becoming a vibrant player in this new information economy, will rely on smes becoming a part of that growth and development. if the currently perceived barriers to involvement in ec are to be overcome, then it would seem that well integrated and embracing strategies that take cognisance of a range of interacting challenges and issues are vital. in addition, being persuaded of the merits of ec and feeling empowered to manage their ec investments such that benefits 136 for the business are realised may be vital steps in encouraging appropriate levels of participation in ec amongst smes. research approach the research reported in this paper is the first stage of a planned programme of research into the management and realisation of benefits from electronic commerce (ec) investments. this was a partly exploratory, partly descriptive study, aimed at finding out about current practices amongst smes with respect to the evaluation and realisation of benefits from ec investments. the research was deemed to be mainly exploratory for a number of reasons. there was little published literature to provide insights into this area, a point previously noted in the background to this paper. as researchers, we were aiming to become more familiar with the practices in organisations, and wanted to become more confident in our knowledge of "what was actually going on". it was believed that through establishing a broad "picture" of current practices, the beliefs of the researchers would be confirmed or disconfirmed, questions and issues would emerge for a subsequent, more systematic enquiry, as would appropriate directions for future research (neuman 1997). however, the study could also be claimed to be descriptive, in the sense that it aimed to describe practices, provide a "picture", and document information about an issue. given these factors, an exploratory/descriptive study using survey interview method gathering both qualitative and quantitative data seemed an appropriate way to proceed (neuman 1997). while some writers express concerns about the suitability of survey interview method for exploratory research (fowler and mangione 1990), there is generally more support for its usefulness in capturing pictures of practices or descriptions of real-world situations (galliers 1991, bryman 1989, leedy 1993). thus, the major purpose of the research can be stated as follows: • to identify current practices amongst smes with respect to the evaluation and realisation of benefits from ec investments; • to investigate whether smes attempt to, and are able to accurately assess the impacts of their ec investments; and • to see whether smes have a methodical approach to benefits management. the survey was developed from an instrument previously used by ward et al. (1996), but was modified somewhat to make it more suitable for the context of this particular study. this survey was limited to ec investments (as previously defined), and not to is/it investments in general as the original one was. secondly, the focus was on smes, not large organisations, and some changes were required to reflect this. thirdly, as the original survey was intended for it directors or chief information officers, some modifications were required for our intended target of general senior managers. the fourth point to note is that the survey in this research was conducted by phone, not by mail, as in the original research. the nonprobability sampling technique used in this research was convenience sampling (leedy 1993). nonprobability sampling means that there can be no guarantees that participants in the study are representative of the population at large, and a convenience sample is selected on the basis of availability to the researcher or happenstance (neuman 1996, bryman 1989). clearly in studies requiring statistical analyses, this approach would be inadequate, but given the exploratory nature of the research and the interest of the researchers in getting a "feel" for industry practices with respect to benefits management, it seemed adequate for our purposes, especially given some of the defining criteria imposed on participants (and discussed in the next paragraph). furthermore, as no attempt was planned to make statistically-based deductions from the data collected, and as any quantitative results are expressed only as proportions or frequencies, a convenience sample seemed adequate for the purposes of this study. the participants were all enrolled in the mba programme at the researchers' university and to be included in the study had to meet all of the following criteria: • they had to work in, or direct, or own a non-government sme; • they had to demonstrate that they were at least in senior middle management ranks or above; • they had to be in non-is/it management (i.e. senior line managers or in a senior staff function other than is/it); • their organisation had to have implemented at least a rudimentary ec system (a "publish" web site as a minimum); • they had to have had direct involvement in the evaluation and implementation of the ec investment, and with its subsequent management; • they had to be willing to participate in a phone interview. out of the total cohort of 43 mba students potentially available to the researchers, 18 met all the criteria and were selected for inclusion in this study. clearly there is no way of knowing whether this group is representative of the wider population, and indeed, the decision to study an mba may indicate an atypical sme manager. 137 however their involvement in ec investment decisions may suggest their responses are knowledgeable of evaluation practices with respect to ec investments. participants were telephoned at a prearranged time and location, and were interviewed by the researchers guided by the survey questionnaire. while there were occasions when the interview became more expansive, by and large, the interviews were quite structured, and were limited in the elaborations sought. the interviews were conducted during april and may 2000. while the researchers did clarify the meaning of "ec investment" for the participants, no attempt was made to define "benefits management" or "benefits realisation", thus enabling participants to apply their organisational understanding of benefits in their responses. at the time of the interviews, the participants had not studied either it evaluation or benefits management in their mba coursework. the survey contained a total of 57 questions, a mix of closed questions (yes/no), scale items (multiple choice, ranking, etc.) and with some open-ended items providing opportunities for descriptions, explanations, justifications and so on. there were four broad sections in the survey. the first aimed to elicit background information on the participants and their organisations. the second section contained general questions about the nature of their involvement with ec, their expenditure on ec, and their perceptions about the benefits of ec. the third section went into more details regarding the planning that was done prior to making the ec investment decision, while the fourth section concentrated on the realisation and management of benefits from the ec investment. research findings background of the 18 participants, 12 regarded themselves as non-is/it managers, with job titles such as ceo, business manager, and financial manager, for example. the remaining 6 preferred to describe themselves as hybrid managers, with an excellent knowledge of is/it although officially now employed in a senior, non-is/it staff management position. more than half the participants (12) worked in medium-sized enterprises, with the remainder divided evenly between small and micro enterprises. they were drawn within a variety of industry sectors including such diverse fields as tourism, recreation and hospitality, business consultancy and training, environmental services, marine services, engineering, logistics, import/export businesses, and medical and welfare services. involvement in electronic commerce the executives were asked to identify their organisation's level of involvement with ec. the levels described for them were based on those of o'connor and galvin (1998) who defined four levels as publish (essentially a web site enabling one-way communication only), interact ( a web site enabling two-way flows of information), transact (a web site supporting the completion of full business transactions including payment), and integrate (a web-enabled network of trading partners integrated with internal business processes and systems). the majority of these organisations are still at relatively immature levels of electronic commerce currently, but it is interesting to note that 3 of the larger businesses were already engaged in full-scale b2b and b2c electronic commerce through their involvement at an integrate level (see figure 2 below). not surprisingly therefore, it was seen in a support or key operational role in their organisation by most participants, although 5 suggested that it was being deployed strategically by their organisations, and 4 claimed it was being used in a developmental capacity. what role does is/it play support role • in your organisation currently? enables key */ t/ operational process e> o strategic importance tr % % p % future development > % ji $ (r&d) publish interact transact integrate figure 2: level of involvement with ec start up costs for ec were defined for the participants as including the costs associated with hardware purchases and/or upgrades, software purchases (including development and/or tailoring costs if appropriate), and consultancy fees. the estimates of these costs are detailed (in australian $) in figure 3 below. while estimates of start-up costs ranged from $2500 to more than $100,000, it is interesting to note that while only 3 138 organisations were involved in more sophisticated activity than interacting with customers on the web, 6 organisations had reportedly spent more than $100,000 in establishing electronic commerce in their organisations. our study would suggest that while some organisations are able to engage in ec with relatively little expenditure, for others, quite extensive costs are involved. to i <2,500 2,5005,00010,00020,00050,000>100,000 5,000 10,000 20,000 50,000 100,000 figure 3: estimated start-up costs for ec for some organisations there are also expectations of considerable on-going costs for operations and maintenance of the ec activity. on-going operations and maintenance were defined to include isp and asp costs, web site maintenance costs, order fulfillment costs and so on. figure 4 below details the executives' estimates of their organisation's annual expenditure. <5,000 5,00010,00020,00050,000>100,000 10,000 20,000 50,000 100,000 figure 4: estimated annual expenditure on operations & maintenance for ec in contrast to the costs of being involved in electronic commerce, participants were asked to estimate the proportion of weekly revenues generated through their electronic commerce involvement. revenues generated was defined to include direct web sales as well as orders/sales motivated by the web site. details are contained in figure 5 below. of interest is the high proportion who claim not to be generating any revenues, although this may be a function of the fact that a large proportion (>40%) of organisations in this study at this stage, had established a publish-only web site. of concern too with these estimates of revenues is how accurately webmotivated transactions are able to be identified amidst all other business activity8 to§ 13to "c to 6 'o <2% 2-5% 5-10% 10-20% >20% none being generated figure 5: proportions of weekly revenues generated from ec in terms of cost savings, a scattered picture emerged of savings in telephone and fax costs, post, printing and paper, and reduced costs of advertising over the internet. other reported savings were from time saved, lower transaction costs, and lower service delivery costs. once again, however, by far the most common response was that no cost savings had been achieved (and 1 respondent was uncertain as to whether costs savings had been achieved), and one executive wryly added that "the cost of web site maintenance more than overrides any cost savings achieved". figure 6 below contains the details of participants' perceptions of cost savings. 8 a previous study of car dealers found that no mechanisms existed to reveal web-motivated "walk-in" customers and subsequent sales (see marshall et al. 2000). 139 3 'c ras b o telephone/ transaction printing/ more timely reduction none being fax „ costs . . . . paper ,_ receipt of ,. in staff .. . derived post advertising k ̂ time payments travel uncertain saved costs of cost savings figure 6: cost savings derived through ec although the participants did not paint an overly optimistic view of their experiences to date in terms of increasing revenues and reducing costs, they nonetheless perceived electronic commerce to offer a number of potential benefits (both tangible and intangible) for their organisations. further details of the participants' responses are provided below in figure 7. access to, and provision of timely, relevant information was the most widely recognised benefit of electronic commerce, and interestingly, this may not contribute directly to increasing revenues or reducing costs. however, many of the remaining perceived benefits (reduced cycle time, improved efficiencies, etc.) would be expected to translate directly into improved organisational performance. given the contrast then, between their perceptions of benefits and their relative lack of deriving costs savings or increased revenues, it begs the question of how confident the participants were that they were actually successfully realising these perceived benefits. there was an even range of responses to this from "not at all confident" to "very confident". how confident of actually verv confident> > > realising these benefits? quite mnment. moderately confident not very confident not at all confident v v v provision of improve improve improve increase information efficiency , image/ , service staff skill ., improved increase fite improve provision none processes market v competitiveness v communication figure 7: major perceived benefits of ec the expressed reasons for their confidence that benefits were being achieved were mixed. those who were not very confident commented that they had not had nearly the response they expected following their ec investment (for example, they were not getting as many "hits" or customers as forecast), or admitted that they had not assessed whether benefits were being derived. many commented in their replies that they were finding it difficult to measure where and what benefits were being achieved. by contrast, those expressing great confidence were usually able to cite some metric having been applied, such as improved customer satisfaction surveys following the ec initiative (and arguably directly attributable to that investment). it must be noted however, that some participants expressing confidence could not actually substantiate their feelings with any tangible evidence, and seemed to be relying on "gut feel". when questioned about their main concerns with respect to their ec initiatives, their responses seemed to reflect a relatively high awareness of some current concerns surrounding ec. for example, the single most mentioned issue was "is electronic commerce worthwhile? will electronic commerce contribute benefits to my business (either reducing costs or increasing profitability)?". security concerns, including the need to maintain confidentiality of important corporate and customer data, and technical issues were also reported to be of concern to a number of respondents, particularly those in micro and small enterprises and those in not-for-profit organisations. figure 8 below details the concerns expressed. 140 10 i.2 o o worthwhile technical rewards/ maintenance lack of undertaking ̂ concerns ^ profitability trajning ofwebsrte ^^ planning lackqf confidentiality expertise understanding figure 8: major concerns with respect to ec planning considerations for ec the smes represented in this study were in large agreement over the main drivers of their involvement in ec (see figure 9 below). issues to do with competitiveness (ec was seen as essential to match or better their competitors) and market forces were by far the most frequently cited reasons for engaging in ec. participants reported increasing awareness of and interest in the internet amongst both their customers and trading partners and a number reported that they viewed ec simply as an extension of their business (it was a part of being in business). in contrast to previous studies (marshall et al 2000), comparatively few (3) said there were involved in ec because "everyone else is doing it". greater business maturity seem to underlie their decision-making about making investments in ec than was revealed in previous case studies (marshall et al. 2000). $ i s>o increased market increased part of everyone improve communication staff competitiveness forces interest being in is doing it customer reduction in web business service figure 9: drivers of ec investments despite the seeming business imperatives driving the ec activity of these smes, a significant proportion of respondents (>50%) did not include ec considerations in their business plans, nor did they have any apparent mechanism for linking their ec investments with their business objectives. only 4 participants (22%) said that planning ec initiatives was an integrated part of their business planning activity. planning was conducted for individual ec projects. economic and financial benefits figure highly in their planning activity, as do benefits that would accrue from better management of their customers. other motivators seem to be associated with competitiveness and the promotion of brand and/or corporate image through ec initiatives. indeed improving their image or profile and signalling to competitors were important intangible benefits that were considered. figure 10 below details to planning of benefits in these ec ventures. 141 o •3 o economic/ increased access to synergies to financial customer """p^ness promote formation |ncfeased organisation relationships image business opportunities figure 10: benefits considered in planning ec participants were asked a number of questions about their up-front evaluation practices, and a somewhat mixed response emerged. five participants said their organisations used a fairly traditional roi and cost/benefit approach to their evaluations. two participants said evaluation was part of building a business case for ec prior to investment, and one respondent had sought professional advice prior to making the investment. for the remaining organisations, a much more haphazard approach seems to a been adopted. five said they had relied on "gut feel", essentially the beliefs of their management team or the project champion, and thus no formal evaluation had been employed. two had sought advice from other similar organisations, two had sought advice from their customers, and two had not done any evaluation at all. one respondent said the only criterion had been the time to implement (they wanted the web site quickly) while one other participant said that evaluation had been based solely on perceptions of the ease of maintenance of the proposed ec project. i a .2 co cd 6 •5 o cost/benefit business seek advice from consulted time to ease of no gut roi case professional similar customers implement maintenance evaluation feeling advice organisation figure 11: approaches to up-front evaluation of proposed ec investments generally speaking, however, only 3 participants (17%) expressed satisfaction with their approach to evaluation. with all the wisdom of hindsight, many now recognised some shortcomings in the approach they had adopted. there were concerns expressed that they felt they needed to do much more research up-front (2), they had grossly overestimated the benefits that would accrue (4), they had overstated expected hits on the web site (and hence their ability to raise revenues from advertising) (3), they realised they had to spend much more in marketing their web sites and hence costs had blown out, to the point where one respondent commented that "more than six months after implementation, we have lost control of the costs". one executives also acknowledged that there had been no sense in which pre-investment evaluation had been linked to their postinvestment desired outcomes. figure 12 below summarises some of these concerns. 142 c (0 e> o o o benefits costs hits conflicting more no linkage yes overstated inaccurate overestimated advice from research to desired . consultants upfront outcomes . no (15) yes(3) figure 12: perceived helpfulness of approach to ec evaluation employed not only was there some concern amongst executives about their evaluation practices, but there were additional concerns about the efficacy of the approach used. so, for example, 13 respondents (72%) felt that their evaluation process had failed to identify all possible or potential benefits from their ec investments, and perhaps more importantly, felt that they had been unable to satisfactorily quantify the benefits identified. 11 of the respondents (61%) conceded that maybe benefits had been overstated in order to ensure approval was given for the project. planning for the realisation of identified benefits also proved to be of some concern. in 12 of the organisations (67%), no one was assigned responsibility for realising the business benefits claimed in the justification. in those cases where someone was allocated that responsibility, the types of expectations held of that person were a post-completion audit (1), training (2), processing orders on a daily basis (1), controlling costs (1), and "keeping watch on the web site" (1). when asked how they would ensure that the ec initiative would deliver benefits to all relevant stakeholders, a variety of responses were received (see figure 13). it is interesting to note that 7 organisations (39%) were just assuming benefits would be delivered, thought that a nice web site was all that was required, or were willing to leave it to chance. id § o o customer financial leave it to have a nice assume feedback forecasts chance website it will figure 13: how do you ensure ec initiatives deliver benefits? participants were also questioned about whether organisational changes (such as changes to structure or creation of a new department, etc.) or process changes were planned to accommodate the ec initiatives. varied responses were received indicating that planning of such changes sometimes took place before approval of the investment was given, but there was a clear indication that planning change also went on during the design and implementation stages. of interest to note was the fact that 8 respondents (44%) still claimed that changes were still being planned after the web sites and associated systems were operational, perhaps indicating that a reactive approach to change was being adopted in a substantial proportion of the cases. realisation and management of benefits from ec 15 of the respondents (83%) agreed that they had no formal plan to ensure the realisation of benefits from the ec project. (a limitation of the research was perhaps our failure to find out what form the plan took in those three organisations who suggested that a delivery plan did exist). the executives interviewed overwhelmingly felt that responsibility for delivering benefits lay with senior management and to some extent with line/departmental managers, but were generally less clear as to precise procedures in place to ensure the benefits identified preinvestment were actually delivered post-implementation. 6 of the executives (33%) admitted that in reality, no steps were taken to ensure the realisation of benefits. other respondents claimed that web site statistics were analysed in some way, but only one respondent then added that this analysis was then considered against objectives in the organisation's strategic plan. the others conceded that such analysis still may not ensure that 143 all stated anticipated benefits were actually realised. two of the executives reported six monthly postimplementation analyses of performance, which they argued identified where benefits were occurring and which benefits were not being realised. a third organisation claimed to have rigorous, systematic post-implementation measures in place which gave it confidence in claiming that many of the anticipated benefits were being delivered. participants in the study were asked questions about what constituted "success" with an ec project (see figure 14). 3 respondents (17%) considered the impact of the ec initiative on sales, with 2 defining success as increased sales and/or revenues generated on-line, with the third, comparing total sales performance (on-line and traditional channels) post-implementation against historical data. this organisation was willing to attribute an upward trend to the ec project, although the executive did concede that the success may actually have been achieved in the traditional sales channels. 3 organisations defined the ec initiative as successful if project objectives were met, 2 others (22%) relied on market feedback, while 2 others looked to (upward) changes to service levels as indicative of success. of concern, however were the 8 organisations (44%), who simply had not defined success, who did not make any measurements, and who sometimes complained that it was very hard to measure success. s § s> o "5 i increase project market increased no measures in sales objectives met feedback service levels of success figure 14: how do you measure success with ec investments? the executives were also questioned about intangible benefits, specifically how they knew they were achieving intangible benefits, and whether they attempted to quantify intangibles. 12 (70%) agreed that intangibles should be included in considerations of benefits derived and the success criteria for ec, but only 11 claimed they had tried to quantify such benefits. when asked how they did this, the general answer (10 out of 11) was that they were still attempting to work out appropriate ways of quantifying intangible benefits! 7 organisations (39%) relied on informal feedback or anecdotal evidence as indicative that intangible benefits were being achieved, 3 looked to changes to business performance, but again, the larger group (44%) admitted they had no idea as to whether they had achieved or were achieving any intangible benefit from their ec investment. only 7 organisations conducted (or were in the process of developing) formal post-implementation reviews. these were typically in the form of committee assessments, 3-6 months after the implementation of the web site and associated systems. the stated objectives of these reviews were most often project management effectiveness (7), and technical conformance (6), with only 4 organisations seeking to confirm the delivery of benefits to the organisation, and 2 stating that they were just trying to check that the web site was doing the job required of it. what needs to be remembered at this juncture are the 11 organisations (61%) who do not conduct any formal post-implementation reviews whatsoever (see figure 15). 144 coc o .2 project mgmt technical delivery checking no posteffectiveness conformance of benefits website implementation reviews a with what objective(s)? a yes (7) no (11) post-implementation review? figure 15: post-implementation reviews of ec initiatives to conclude the interviews, the executives were asked whether their organisation had derived "value for money" from their ec investment. no definition of this concept was offered, so hopefully, their overall "gut reaction" was being expressed. figure 16 reveals the picture of responses. those toward the "yes" end of the spectrum reported increases in efficiency, competitive positioning, and increased public awareness of their enterprise. 2 said "yes", but added that there had been no measurable benefits, 1 said "yes", only to quickly qualify the claim by adding that "it will in the future". of interest to note however, were the 6 respondents (31%) who were not persuaded that value for money had been derived. co i 3 c 6 o increased competitive increased yes, but no yes, in no efficiency position awareness measurable future . . benefit yes (9) no (6) figure 16: have you derived "value for money" from ec investments? additional comments were proffered by some to support their feelings about getting value for money from ec. for a minority of the executives in this study, the venture into ec has been a positive and successful one, bringing lasting benefits and repositioning their organisations. more of those who chose to make a closing comment, however, seemed family to distinctly disappointed with their ec initiatives. one comment, in essence repeated by 3 or 4 of the participants, related to concerns that the initial costs for ec were perceived to be high, and once that hurdle was overcome, the expected benefits were not easy to achieve. some talked about the difficulties they had experienced in firstly, anticipating all the costs they actually encountered, and secondly, in establishing appropriate measures for benefits. one executive described what he called an "expectation gap", the perception that what had been delivered to the organisation had been less than that anticipated. echoing a similar sentiment, one executive commented that "/ have been seduced by media hype!". discussion of research findings the study reveals an interesting picture amongst smes with respect to their ec initiatives, but does not inspire confidence in terms of their approach to evaluating their investments and managing the realisation of benefits from those investments. although many participants expressed quite a degree of confidence that they would derive benefits from their ec investments (see figure 7), other components of their responses did not really 145 confirm this confidence, and indeed, their overall responses to the issue of deriving value for money did not present such assuredness (see figure 16). the organisations represented in this study in the main were at a relatively immature stage with respect to ec, had spent fairly substantial amounts in getting established (certainly enough to de-bunk the myth that ec is cheap!), and were anticipating fairly high on-going expenditure on ec. despite this, they were generating little or no revenue from the ec initiative, and outlays did not appear to be matched by cost savings in most cases. interestingly, and perhaps not surprisingly given these data, along with security, the most frequent concern about ec expressed by these executives had to do with whether there was any purpose in engaging in ec, whether it was "worthwhile". it also appeared that they felt somewhat "pushed" into ec, with external factors such as their competitiveness, market pressures and increased interest from trading partners and customers featuring highly in driving their ec investments. a picture emerges of organisations feeling impelled to become involved in ec at least in a minimal way, spending quite a lot of money in establishing and maintaining a presence, but deriving neither substantial revenues nor cost savings. why might this be the case? previous argument suggests the need to consider the relationship between planning (establishing close links between it investments and business objectives), an on-going life cycle approach to evaluation, and the proactive management of the realisation of business benefits from it investments (mckay and marshall 2001). if we consider the reported behaviours of respondents in this regard, an interesting picture emerges. comparatively few claimed that planning for ec was integrated with their business planning, nor were there reported to be any mechanisms to link ec investments with business objectives. it is tempting to conclude therefore, that in many instances, planning for ec could best be described as an ad hoc activity, without there necessarily being a coherence between the overall business direction and objectives and the ec initiative. evaluation of the ec proposal also appeared somewhat haphazard: those that did rely on some recognised approach such as cost/benefit analysis (and they were in the minority), alighted upon a mechanism the efficacy of which has been questioned in the is research literature (willcocks and lester 1997, farbey et al. 1992). many relied on intuition. if we consider the large proportion who subsequently felt that their evaluations had not proved satisfactory, it could be argued that neither approach (cos^enefits or intuition) appears to have been particularly effective. of particular concern was their failure during the evaluation activity to identify costs and potential benefits accurately, and to adequately quantify the "value" of those benefits. when it came to the realisation of benefits, again a disturbing picture emerges of executives assuming that benefits will flow to their organisations just by making an investment: chance and personal belief that benefits would accrue were reasonably common. this is coupled with the fact that generally speaking, there were few measures of success in place, there were comparatively few post-implementation reviews, and when these were held, they generally focussed on checking technical specifications and project management performance. while the business world is far too complex to safely draw any definite causal relationships, could it be postulated that inadequate planning, evaluation and benefits management may have contributed to perceptions that organisations were not deriving value for money, and the view that they "had been seduced by media hype"? the answer to that question lies outside the scope of this research, as we set out to explore and describe, not to explain. but the picture that emerged from this study does not suggest that smes are currently managing their ec initiatives in a particularly efficacious way, and thus, it is perhaps not completely surprising that they themselves echo some disappointment with the results they are achieving with respect to ec. of interest was the fact that the results of our study mirror fairly closely the findings of ward et al (1996), despite the different foci of the research (ward et al. (1996) were considering is/it investments in large, private organisations). conclusion while the study reported here has revealed some interesting and disturbing practices, it also seems to point to the need for further research. in the sample of organisations we looked at, there seemed a disturbing lack of planning, evaluation and proactive management of benefits with respect to their ec initiatives. is this practice replicated across all their is/it investments? can a stronger indication of causation be established between these behaviours and their disappointment with their ec initiatives? what sort of strategies and approaches could be adopted in order that better business outcomes for the organisation might be achieved? this study seems to pose more questions than it answers. it was earlier argued in this paper that for countries in which a large proportion of their economic output is dependent on the activity of smes, participation in and deriving benefits from the new economy depends on individual smes successfully engaging in and managing a whole range of ec initiatives for positive business outcomes. our study of the small sample of smes questions the extent to which this is being accomplished. of vital importance, therefore, is to work towards helping smes achieve better outcomes with their ec endeavours. 146 references anderson consulting (1998) e-commerce: our future today, http://www.ac.com australian department of communications, information technology and the arts (docita) (2000a) australia's e-commerce report card, http://www.noie.gov.au/reportcard australian department of communications, information technology and the arts (docita) (2000b) ecommerce beyond 2000. http://www.dcita.gov.au behrendorff, g., fisher, j., goldsworthy, m. a., corbitt, b.and perks, g. (1996) advice on electronic commerce programmes for small to medium-sized enterprises (smes), http://www.cec.buseco.monash.edu.au/reports/dist/report.htm canberra, australia: department of industry science and tourism bryman, a. research methods and organizational studies. routledge, london, 1989. crawford, j. (1998) news: a project to get smaller enterprises on-line. ultimo, nsw: department of industry, science and tourism (dist), and tradegate eca. czerniawska, f. and potter, g. (1998) business in a virtual world: exploiting information for competitive advantage, macmillan press, london. earl, m.j. (1994) putting information technology in its place: a polemic for the nineties. in r.g. galliers and b.s.h. baker (eds.) strategic information management: challenges and strategies in managing information systems. butterworth heinemann, oxford, pp76-90. ernst & young (2000a) global on-line retailing, http://www.ey.com.au ernst & young (2000b) virtual shopping in australia, http://www.ey.com.au farbey, b., land, f. and targett, d. (1992) evaluating investments in it. journal of information technology, 7(2): 100-112. fowler, fj. and mangione, t.w. standardized survey interviewing: minimizing interviewer-related error. sage, newbury park, 1990. galliers, r.d. "choosing appropriate information systems research approaches: a revised taxonomy." in information systems research: contemporary approaches and emergent traditions. h.e. nissen, h.k. klein and r. hirschheim (eds.) elsevier science publishers, north holland, 1991. kempis, r. and ringbeck, j. (1999) do it smart: seven rules for superior information technology performance. free press, new york. laudon, k.c. and laudon, j.p. (2000) management information systems: organization and technology in the networked enterprise. 6th ed. prentice hall, n.j. leedy, p.d. practical research: planning and design 5th ed. macmillan, new york, 1993. marshall, p., sor r. and mckay, j. (2000) electronic commerce and car retailing: an industry case study. international journal of electronic commerce research. 1(1). mckay, j and marshall, p. (2001) the it evaluation and benefits management life cycle. in van grembergen, w. (editor) it evaluation methods and management. idea publishing group. neuman, w.l. social research methods: qualitative and quantitative approaches. 3rd ed allyn & bacon, boston, 1997. o'connor, j. and galvin, e. (1998) creating value through ecommerce. financial times management, london. oecd (1998) smes and electronic commerce, working party on small and medium-sized enterprises, directorate for science, technology and industry, organization for economic co-operation and development. small business index (1999) survey of computer technology and e-commerce in australian small and medium business, http://www.pacificaccess.com .au us department of commerce (1999) the emerging digital economy n, http://www.ecommerce. gov/emerging.htm valli, c. and marshall, p. "an initial study of australian on-line bookstores." paper accepted for presentation at 2nd annual global information technology management 2001 world conference, dallas, texas, 10-12 june, 2001. ward, j., taylor, p. and bond, p. (1996) evaluation and realisation of is/it benefits: an empirical study of current practice. european journal of information systems, 4, 214-225. willcocks, l. and lester, s. (1997) assessing it productivity: any way out of the labyrinth? in l. willcocks, d. feeny and g. islei (eds.) managing it as a strategic resource. mcgraw hill, london. zwass, v. (2001) structure and macro-level impacts of electronic commerce: from technological infrastructure to electronic marketplaces, http://mhhe.com/busmess/mis/zwass/ecpaper.html 147 84 executive support of information technology and information systems in australian hospitals: an empirical study16 graeme rose school of business charles stun university albury nsw 2640 australia grose@csu.edu.au http://www.csu.edu.au/faculty/commerce/business/bushp.htm robert reeve school of economic and financial studies macquarie university sydney nsw 2109 australia rreeve@efs.mq.edu.au http://www.efs.mq.edu.au/accg/index.html abstract little is known regarding the form executive support should take for the progressive use of information technology and information systems [ft] within organisations. this study applies the theory developed by jarvenpaa and ives (1991) who examined two forms of support provided by chief executive officers. these were executive participation, a set of it-related activities, and executive involvement, a psychological state reflecting the importance of it for the organisation's success. our research, using data obtained from a questionnaire mailed to a sample of australian hospitals, measures the relationships between these two forms of support and the progressive use of it. our statistical analysis supports the findings of jarvenpaa and ives (1991), who found a stronger relationship between executive involvement and the progressive use of it. using australian hospitals allowed jarvenpaa and ives' (1991) theory to be applied in a different environment, increasing its external validity. firm size was also found to have a positive relationship with the progressive use of it independent of the two forms of executive support. introduction many writers have identified the link between information technology and information systems {it], and the improvement of business operations. for instance rockart and crescenzi (1984, 3) argue that "... [it] gives managers an opportunity (1) to improve delivery of their products and service and (2) to potentially increase their effectiveness and productivity in managing the businesses". despite the benefits to be derived from it by organisations, many instances can be found in the literature where it initiatives have been unsuccessful. table 1 presents a selection of references to problems associated with the progressive use of it. by the progressive use of it we mean the extent to which progress and continual improvement are made in the identification, development, implementation, management, evaluation of opportunities, and use of information systems and information technology in achieving the goals of australian hospitals. this meaning is consistent with the way this term was used by jarvenpaa and ives (1991). the difficulties that have been encountered with various aspects of progressive use of it indicate the need for research aimed at identifying factors that will facilitate successful it initiatives. the main aim of this study is to assess the relationship between the progressive use of it and one of these factors, executive support for it; in particular support for it by the chief executive officer (ceo). ceos need to consider the nature and extent of their support for it. organisations may have different strategies for achieving their goals. the progressive use of it is important in all organisations but particularly important in firms with information intensive value chains (porter & millar 1985). 16 an earlier version of this paper was presented at the 1996 aaanz conference in christchurch, new zealand, where constructive comments were received, especially from the discussant dr. zahirul hoque of griffith university gold coast. helpful suggestions were also received from one anonymous reviewer. all this assistance is gratefully acknowledged. the authors are grateful to the school of business and the faculty of commerce at charles sturt university for providing funds to support this research project. ajis vol s, do 1 september 1997 85 table 1: problems associated with the progressive use of it date 1971 1976 1985 1987 1992 1993 problem it has long been recognised that sisp [strategic information systems planning] is an intricate and complex activity fraught with problems (p 78). [studies on the impact of mis across industries] indicated that current, or planned, computer-based management information systems and their usage fall far short of their theoretical capabilities (p 579). top management and mis directors have been increasingly urged to improve the way in which their information systems development efforts are being managed (p 17). reports abound of systems implemented late and over budget with the anticipated savings being unrealised (p 390) ... failure to execute the [sisp] plan is a serious problem ... (p26). the track record for [it] implementation is not very good (p 23). authorfs] mcfarlan schewe doll lederer & mendelow lederer & sethi benjamin & levinson journal harvard business review academy of management journal mis quarterly mis quarterly journal of mis sloan management review our study applies the theory developed by jarvenpaa and ives (1991) (j&i hereafter) which examines the role played by the ceo in supporting the progressive use of it. that support provided by top management is a necessary ingredient in the progressive use of it is well documented (j&i, p 204). however, the appropriate role of ceos for achieving the progressive use of it is unclear. should ceos personally engage in it-related management activities such as participation in it committees, developing it skills, and keeping abreast of it developments relevant to their organisations? alternatively, can ceos better support it by maintaining a psychological state in which they communicate to staff the critical importance of it to the success of the organisation? our study aims to assess which role has the stronger relationship with the progressive use of it. by applying j&i's theory in a different setting, our study will make a useful contribution to the literature on the appropriate role of ceos in the progressive use of it. specifically, we expect to show that j&i's theory will apply in a different industry (hospitals), in smaller organisations (j&i's study had been carried out in fortune 500 firms), in a different country (australia), and at a different time (1995), thereby increasing its external validity and strengthening our general understanding of executive support for it. as recommended by j&i (p 220), this study extends their work by also examining the relationship between organisation size and the progressive use of it. the remainder of the paper is organised as follows. first j&i's theory is reviewed, and hypotheses are developed to guide the empirical research. then the methods used in the empirical research are explained and the data analysis is summarised. finally the results are discussed, and their limitations and implications are considered. theory and development of hypotheses the main research question in this study is how ceos can best support the progressive use of it in their organisations. in recognising the need to be progressive in the use of it, ceos need to communicate their belief in the role to be played by it effectively. this study compares two ways in which ceos can do this. is it more effective for ceos to take an active hands-on approach in the progressive use of it, or is it more effective for ceos to communicate their belief that progressive use of it is critical for the success of the organisation in other ways? this study also considers whether organisation size influences the progressive use of it. larger hospitals, with their greater input and output diversity, leading to greater uncertainty and a greater need for information (galbraith 1973, pp 4-5), are expected to make more progressive use of it than smaller hospitals. smaller hospitals should be able to be administered with less sophisticated it more easily than larger hospitals, although it has been shown that higher performing small firms have a higher sophistication of it (raymond, pare & bergeron, 1995). larger hospitals are more likely to require sophisticated it as this may assist the larger and more complex systems such as administration, communication, scheduling, human resource management and ajis vol 5, no 1 september 1997 86 patient care. merchant (1981, p 815), in commenting on the effect of size, observed that "larger organizations ... face an exponentially increasing number of channels requiring information flows for coordination purposes ...". to develop an understanding of executive support, j&i built on the theoretical foundations of the user involvement construct developed by barki and hartwick (1989). barki and hartwick (1989) undertook a review of user involvement as it has been used in it research and compared this with the way it has been developed in other disciplines. they examined user involvement as applied in it research, believing that future research should adopt a common definition in line with that used by other disciplines, such as psychology, marketing, and organisational behaviour. they compared their findings with the way in which involvement has been defined in the literature of these disciplines, and showed that not only are these disciplines undertaking a refinement of the construct, but that they seem to be coining to uniform conclusions (barki & hartwick 1989, p 61). they argue that the term user participation should be used instead of user involvement when referring to the behaviours and activities of the users, and that the term user involvement should be used to refer to the user's "...subjective psychological state" (pp 59-60) regarding it. j&i used barki and hartwick's (1989) framework to examine involvement and participation empirically as forms of executive, rather than user, support. to carry out their examination of executive support, j&i developed empirical measurements for executive participation and executive involvement, and developed theoretical models to describe the relationships between these factors and the progressive use of it. they then measured the strength of each relationship to identify the more effective form of support role for the ceo and found that "... the ceo's psychological state about it appeared to be a more powerful predictor of the firm's progressive use of it than a ceo's personal participation in it management." (p 216). the main research question, which deals with the relative strengths of two sub-sets of executive support is: which subset of executive support, executive participation (a set of it-related management activities of the ceo) or executive involvement (a psychological state reflecting the degree of importance placed on it by the ceo) has a stronger relationship with progressive use of it in australian hospitals? the secondary research question is: is there a relationship between organisation size and the progressive use of it within australian hospitals? hospitals have a value chain of relatively high information intensity (porter & millar 1985) and are therefore a suitable context for this analysis. the choice of australian hospitals as an industry of high information intensity is supported by johnston and carrico (1988). they identified three industry factors that significantly influence the likelihood of strategic it potential, and based their analysis on the operation of the value chain as developed by porter (1985, p 39). the industry factors identified were said to arise from the nature of the primary products and services and the structure of the processes that are used to produce value. the three factors influencing strategic it potential as identified by johnston and carrico (1988) are significant information content in key relationships (pp 39-40), products or services that have limited life (p 40) and increased competitive pressure within the industry (pp 40-41). these factors all apply in the hospital industry in australia. the dependent variable in this study is the progressive use of it in australian hospitals. other writers have considered similar concepts. boynton and zmud (1987, pp 59-60), for example, referred to "information technology managerial efforts" which involved planning, organising, controlling and directing the introduction of it within an organisation. organisations can therefore be progressive in use of it with acquisition of hardware and software, implementation of it systems and undertaking strategic it planning. progressive use of it in australian hospitals was designated as progit and defined as: the extent to which progress and continual improvement are made in the identification, development, implementation, management, evaluation of opportunities, and use of information systems and information technology in achieving the goals of australian hospitals. the main independent variables in this study are executive participation and executive involvement. executive participation is defined as the ceo's activities or substantive personal interventions in the management of it within the hospital and executive involvement is defined as the psychological state of the ceo regarding it, reflecting the degree to which the ceo views it as being critical for the hospital's success. organisation size is defined as the size of the hospital in terms of number of maintained beds. ajis vol 5, no 1 september 1997 87 executive participation is defined as personal engagement in it-related management activities such as membership it committees, developing it skills and keeping abreast of it developments relevant to their organisations. this entails the ceo's investment in time and energy in it-related matters and would be expected to contribute to the progressive use of it within an organisation. this leads to hypothesis one: hi: there will be a positive relationship between executive participation and progit. executive involvement refers to the psychological state of the ceo regarding it, reflecting the degree to which the ceo views it as critical to an organisation's success and the way in which he or she communicates its importance to staff. this is expected to contribute to the progressive use of it within an organisation. this leads to hypothesis two: h2: there will be a positive relationship between executive involvement and progit. based on j&i's findings it is expected that the executive involvement factor would influence the progressive use of it more strongly than the executive participation factor. this leads to hypothesis three: h3: the relationship between executive involvement and progit will be stronger than the relationship between executive participation and progit. figure 1: the relationships between executive participation, executive involvement, organisation size and progit executive participation executive involvement organisation size figure 1 shows the relationships between executive participation, executive involvement and progit, taking into account the relationship between organisation size and progit. it is hypothesised that, after allowing for the effects of executive participation and executive involvement, larger hospitals would be more likely to be progressive in the use of it than smaller hospitals. this leads to hypothesis four: h4: there will be a positive relationship between organisation size and progit, after controlling for the effects of executive participation and executive involvement. finally it is hypothesised that the relationship between executive participation and the progressive use of it, and the relationship between executive involvement and the progressive use of it, would be independent effects and would not be moderated by organisation size. this leads to hypothesis five: h5: organisation size will not moderate the relationship between executive participation and progit, or between executive involvement and progit. research method data were collected through a mailed questionnaire addressed to ceos and senior information technology managers (sitms) in 236 australian hospitals. these hospitals were randomly selected from the 1994 edition of isaacson's year book of australian hospitals. the sample was selected from both public and private australian hospitals whose size was listed as greater than 100 maintained beds. cohen and cohen's (1983) procedure was used for determining the required sample size. using multiple regression with the significance criterion (a) set at the conventional 95% level, the desired power for the f-test set at 0.80, 5 regressors, and an expected r of 0.16 (the smallest variance explained in j&i's study), a sample size of 74 useable responses was required. j&i's study produced response rates from ceos of 45%. allowing ajis vol 5, no 1 september 1997 for a response rate of around 40%, this indicated that at least 155 survey forms should be mailed. survey forms were mailed to 236 australian hospitals in order to minimise the need for follow-up procedures. eighty-four useable responses were received; a response rate of 35.6%. two survey forms were sent to the ceo of each hospital in the sample using the title for that person as identified in isaacson (1994). one was to be completed by the ceo, and the other (similar) survey form was to be completed by the senior information technology manager (sitm). appendix a contains a copy of the survey form sent to sitms, the responses to which provided most of the data for this study. the form sent to ceos is similar and is available from the authors. both were mailed to the ceo with a covering letter asking that the envelope containing the covering letter and survey form for the sitm be forwarded to the relevant person. stamped addressed return envelopes were included for both the ceo and the sitm to preserve the confidentiality of each respondent. a single question, developed by j&i, was used to measure progit. respondents were asked "how would you describe your hospital's use of it (information technology and information systems)? they were asked to choose from the following range of responses: • a leading hospital • close follower • middle of the pack • somewhat behind • laggard although subjective in nature this measure was found to have convergent validity with at least one objective measure of this variable (j&i, p 351). also, dess and robinson (1984) found that firms' relative performance within an industry can be appropriate substitutes for objective performance measures. j&i (p 213) experienced demand syndrome in the responses submitted by the ceos. that is, ceos appeared to have responded in the way they thought the researchers, their it managers, or others, expected them to answer. analysis of the responses from ceos of australian hospitals revealed the likely existence of demand syndrome. therefore the average of the ceos' and the sitms' responses to this question were used as the score for this variable. organisation size was measured by taking the average of the ceos' and the sitms' responses regarding the number of maintained beds in the hospital. executive participation and executive involvement were measured with multi-item scales. the ten items used were adapted from j&i, and were reverse-coded and re-coded to a 5 point scale where appropriate. each of the items used to measure executive participation tapped into the it-related activities of the ceo as reported by the sitm. each of the items used to measure executive involvement tapped into the ceos' prevailing thinking about it as reported by the sitms. table 2 contains a varimax rotated factor matrix for the ten items included in the factor analysis. inspection of table 2 reveals that factor 2 contains four items with loadings above .40. these were the four items that made up j&i's measure of executive involvement. factor 1 also contains four items with loadings above .40. these four items were included in j&i's measure of executive participation. factor 3 contains two items with loadings above .40. these two items were also included in j&i's measure of executive participation but, based on the data collected in this study, were not unidimensional with the four executive participation items in factor 1 and were dropped. deletion of items 5 and 6 from the executive participation scale can be further justified as follows. the question identifying the number of levels between the sitm and the ceo does not necessarily measure the it-related management activities of the ceo. where the number of levels between the ceo and the sitm is low, the itrelated management activities will not necessarily be high, nor would the converse necessarily be true. the question relating to the ceo's role in the hospital steering committee is also problematic. the low end of the scale required the respondent to indicate that no steering committee exists. the high end of the scale required the respondent to indicate that the ceo is the defacto steering committee. at both ends of the scale, therefore, no steering committee exists. the multi-item scales developed for executive participation and executive involvement were tested for internal consistency reliability using cronbach's (1951) alpha. the four-item executive participation scale returned an alpha value of .76 and the four-item executive involvement scale returned an alpha value of .58. these values of alpha are considered acceptable for exploratory research (nunnally, 1967). ajis vol 5, no 1 september 1997 89 table 2: varimax rotated factor matrix for the ten items used to measure executive participation and executive involvement factor ceo's personal participation in firm's use of it ceo's informal contacts with it management ceo's knowledge of it opportunities ceo's knowledge of other hospitals' use of it number of levels between sitm and ceo ceo's role in corporate it steering committee ceo's perception of the importance of it ceo's prevailing thinking about it spending ceo's endorsement of it not meeting traditional criteria ceo's vision for it question part hi, question 2 part iii, question 3 part iii, question 4 part hi, question 5 part ii, question 2 part hi, question 8 part ie, question 1 part iii, question 6 part iii, question 7 part m, question 9 eigenvalue variance explained factor 1 .788 .562 .817 .778 .143 .203 .351 .178 -.011 .106 3.137 31.4% factor 2 .037 .061 .317 .178 -.015 -.047 .673 .533 .690 .776 1.392 13.9% factor 3 .108 .523 -.043 -.058 .659 -.761 -.026 .261 -.047 .012 1.247 12.5% data analysis and results table 3 contains a summary of descriptive statistics for the variables used in this study. frequency histograms of the four continuous variables were prepared with normal curves superimposed. inspection of these revealed that all had approximately normal distributions except for organisation size, which was positively skewed. a square root transformation of this variable was undertaken which resulted in an approximately normal distribution. table 3: summary of descriptive statistics [n=84] variable name progit executive participation [4-item scale] xi executive involvement [4-item scale] x2 organisation size x3a mean 3.262 3.161 3.321 199.98 standard deviation .883 .719 .814 157.63 minimum possible actual 1 1.50 1 1.75 1 1.20 n/a 40 maximum possible actual 5 5.00 5 4.50 5 5.00 n/a 925 table 4 contains a summary of the correlations between the variables. in each cell the top line shows pearson's r, and the second line shows the associated level of significance (two-tailed pvalue). ajis vol 5, no 1 september 1997 table 4: 90 correlation matrix [n=84] progit executive participation [4 item scale] xl executive involvement x2 organisation size [square root trans] x3 organisation size x3a progit 1.00 xl .115 p=.296 1.00 x2 .389 p=.oq\ .376 p=.00\ 1.00 x3 .189 p=.084 -.098 p=.371 -.043 p=.002 1.00 x3a .169 p=.124 -.065 p=.554 -.034 p=.\92 .984 p=.00l 1.00 regression 1 shows the simple regression model developed to test hypothesis one. progit = j8o + /3ixi +£; regression 1 regression 2 shows the simple regression model developed to test hypothesis two. progit = j3o + 02x2 + £j regression 2 regression 3 shows the multiple regression model developed to test hypothesis three. progit = #) + /3ixi +£2x2 + £/ regressions regression 4 shows the multiple regression model developed to test hypothesis four. progit = /3o + 0ixi+02x2 + /33x3 + £/ regression 4 regression 5 shows the moderated multiple regression model developed to test hypothesis five. progit = ft) + 01 x i + &x2 + £3x3 + 04 x i x3 + £5x2 xs + £/ regression 5 in all the regression analyses the following terms are used: x2 x3 x2x3 = ej = intercept (constant) executive participation [revised 4 item scale] executive involvement organisation size [square root transformation] product of xi x3 product of x2 x3 residual (error) table 5 summarises the results of the simple regression analyses of executive participation and executive involvement on progit. ajis vol 5, no 1 september 1997 91 table 5: results of simple regression analyses of executive participation and executive involvement on progit [n=s4] r2 adjusted r2 f significance of f & beta t significance of t regression 1 progit = a) + 01*1 + ei 0.013 0.001 1.108 0.295 0.141 0.115 1.053 0.295 regression 2 progit = ft) + p&2 + ei 0.151 0.141 14.691 0.001 0.422 0.389 3.833 0.001 table 6 summarises the results of regression 3, the multiple regression analysis of both executive participation and executive involvement on progit. table 6: results of multiple regression analysis of executive participation and executive involvement on progit [jv=84] regression 3 progit = #) + 01 xi + & x2 + e{ variable name executive participation xi executive involvement x2 ft -.044 .437 r2 0.153 beta -.036 .403 adjusted r2 0.132 t -.329 3.656 f 7.320 significance of t .743 .001 significance of f 0.001 hypothesis one predicted a positive relationship between executive participation and progit. that is, the more that a ceo participated in it related activities the more progressive use of it would be in that hospital. inspection of regression 1 in table 5 shows that executive participation has a beta of 0.115 at the 0.295 level of significance and explains only 1.3% of the variance (r = 0.013) in progit. this provides no support for hypothesis one. this was confirmed in the analysis in table 6 showing the results of regression 3 (the multiple regression analysis) where the beta for executive participation is -0.036 at a level of significance of 0.743, when the effect of executive involvement on progitis taken into account. hypothesis two predicted a positive relationship between executive involvement and progit. that is, the more that a ceo reflected the importance placed on it the more progressive the use of it would be in that hospital. inspection of regression 2 in table 5 shows executive involvement has a beta of 0.389 at the 0.001 level of significance and explained 15.1% of the variance (r = 0.151) in progit. this provides strong support for hypothesis two. this was confirmed in the analysis in table 6 showing the results of regression 3 (the multiple regression analysis) which shows a beta for executive involvement of 0.403 at a level of significance of 0.001, when the effect of executive participation on progit is taken into account. hypothesis three predicted that the relationship between executive involvement and progit would be stronger than the relationship between executive participation and progit. this could be tested formally by estimating whether a significant difference exists between the two correlation coefficients. this procedure is unnecessary as, from the regression analyses in tables 4 and 5, it can be seen that the relationship between executive involvement and progit is positive and significant, but the relationship between executive participation and progit is not significant. this provides strong support for hypothesis three. ajis vol 5, no 1 september 1997 92 table 7 summarises the results of regression 4, the multiple regression analysis of executive participation, executive involvement, and organisation size, on progit. table 7: results of multiple regression analysis of executive participation, executive involvement, and organisation size, on progit [/v=84] regression 4 progit = ft) + ft xi + fc x2 + & x3 + ei variable name executive participation xl executive involvement x2 organisation size x3 ft -.020 .438 .038 r2 0.194 beta -.016 .404 .205 adjusted r2 0.164 t .151 3.739 2.039 f 6.456 significance of t .880 .001 .044 significance of f 0.001 hypothesis four predicted a positive relationship between organisation size and progit, after controlling for the effects of executive participation and executive involvement. that is, the larger the hospital the more progressive the use of it in that hospital, regardless of the level of executive participation and executive involvement. inspection of regression 4 in table 7 shows organisation size has a beta of 0.205 at the 0.044 level of significance, providing strong support for hypothesis four. table 8 summarises the results of regression 5, the moderated multiple regression analysis of executive participation, executive involvement, and organisation size, on progit. hypothesis five predicted that organisation size would not moderate the relationship between executive participation and progit, or between executive involvement and progit. that is, the relationship between executive participation and progit, and the relationship between executive involvement and progit, would be independent of the relationship between organisation size and progit. to test this hypothesis regression 5 was developed which contains product terms containing the interactions between organisation size, executive participation and progit [xix^] and organisation size, executive involvement and progit [xix^] (jaccard, turrisi & wan, 1990). inspection of regression 5 in table 8 shows that the coefficients of both of these product terms are not significant. this provides strong support for hypothesis five. ajis vol 5, no 1 september 1997 table 8: 93 results of moderated multiple regression analysis of executive participation, executive involvement, and organisation size, on progit [n=84] regression 5 progit = a) + pi xi + fo x2 + fo x3+ ft xi x3 + ei variable name executive participation xl executive involvement x2 organisation size x3 product term xlx3 product term x2x3 ft .033 .776 .129 -.003 -.023 beta .027 .715 .682 -.072 -.517 r2 adjusted/?2 0.208 0.158 t .074 1.757 1.523 -.108 -.757 f 4.119 significance of t .941 .082 .131 .914 .451 significance of f 0.002 discussion and conclusions table 9 contains a summary of the simple regression analyses of the relationships between executive participation and progit, and between executive involvement and progit in both the j&i study and this study. table 9: comparative results of simple regression analyses: australian hospitals and jarvenpaa & ives (1991) beta f-statistic significance off *2 australian hospitals jarvenpaa and ives [1991) australian hospitals jarvenpaa and ives (1991) australian hospitals jarvenpaa and ives (1991) australian hospitals jarvenpaa and ives (1991) executive participation 0.11 0.49 1.10 10.54 0.29 <0.05 0.01 0.16 executive involvement 0.38 0.46 14.69 22.02 <0.01 <0.01 0.15 0.32 the results in table 9 show that executive involvement has a statistically significant relationship with progit in both research environments. j&i found executive participation to have a statistically significant relationship with progit, but this study found that no statistical significance can be attributed to the relationship for australian hospitals. in both research environments executive involvement was found to have a stronger relationship with progit than did executive participation. this finding supports the first two hypotheses developed by j&i (1991) and also supports the recommendations of barki and hartwick (1989). these findings indicate that a ceo's psychological state concerning the importance of it for the success of the organisation (executive involvement) is a better predictor of the progressive use of it (progit) in the organisation than it-related ajis vol 5, no 1 september 1997 94 management activities (executive participation). in practical terms, it is better for ceos to communicate their view of the importance of it for an organisation's success than to engage in it-related management activities. consideration is now given to the weak results for executive participation in the study of australian hospitals. there are two possible explanations for this phenomenon. firstly, it is possible that the it-related activities of ceos do have no effect on the progressive use of it within an organisation. this is an unlikely alternative given the strong relationship identified by j&i. a better explanation is that ceos in australian hospitals behave differently to ceos in the large private sector firms surveyed by j&i. for instance ceos in australian hospitals may not have the same level of it exposure or training or do not feel comfortable with it and hesitate to support it. it is also possible that the low importance of the ceo's participation in hospitals may be due to the high importance of doctors for whom the participation is crucial to the effective implementation of it. the hypothesis that larger hospitals would be more progressive in their use of it than smaller hospitals was confirmed. as size increases, so does the level of sophistication of systems necessary for efficient and effective operations. the administrative systems in small hospitals need only be operated manually. they would not necessarily require payroll, patient records or communications systems to be computerised. larger hospitals would be more likely to justify the use of computerised systems in terms of cost/benefit analyses, and would benefit from the production of information in a timely manner for purposes of external reporting, internal decision-making, and communicating. progressive use of it would produce economies of scale whereby complex it would be justified on the basis of the increased complexity of systems that arise from the size of the organisation. ceos need to identify the best way of promoting the progressive use of it in their organisations. ceos have ultimate responsibility for the performance of their organisations, and if they are to operate efficiently and effectively they need to take advantage of the benefits offered by it. however their time is scarce and valuable. they need to identify the most appropriate and effective way of supporting the progressive use of it within their organisations. the research in this study supported the findings of j&i that it is more appropriate for ceos to communicate their belief in the importance of it for the success of the organisation than to actively participate in it-related management activities. this finding could come as a pleasant surprise to ceos who do not feel comfortable with the developing technology, even though they do understand its importance. the message to these ceos is that they do not necessarily need to engage in it-related activities. instead, they can communicate their perception of the value of it in other ways. the findings in this study suggest that the most crucial aspect of this communication is the psychological support that ceos are able to provide to the organisation. that is, it will be enough for ceos to communicate their commitment and support for such projects, rather than becoming personally active in an area in which they may have little expertise. although this study provides support for the recommendations made by barki and hartwick (1989, 59-60), and for the generalisability of the findings of j&i (p 219), it is necessary to continue to explore the conditions and mechanisms governing the effects of participation and involvement on the progressive use of it. research focusing on the support provided by the ceo of a single hospital could provide useful insights. measures of the form and value of the support provided by the ceo as identified by other members of the hospital organisation would be also be valuable, particularly considering the demand syndrome demonstrated by ceos. j&i (p 219) also recommend that attention be given to better measures of the progressive use of it, and further suggest that individual level factors such as decision-making style and leadership style be brought in to future research. we suggest that additional organisational-level variables, such as ownership mode (public or private), and in the case of hospitals the method of funding, be included in future research. for instance, are private hospitals, being more strongly influenced by the need to recover costs, more likely to be involved in the progressive use of it? because casemix funding formulae rely upon the operation of sophisticated information gathering, storage, and analysis techniques a further question arises. that is, are hospitals that are funded using a casemix funding formula likely to be more progressive in their use of it than hospitals that derive their funds from the more traditional, historically-based budget allocation methods? aj1s vol 5, no 1 september 1997 95 references barki, h. & hartwick, j. (1989) "rethinking the concept of user involvement", mis quarterly, 13(1), pp 5364. benjamin, r.i. & levinson, e. (1993) "a framework for managing it-enabled change", sloan management review, summer, pp 23-33. boynton, a.c. & zmud, r.w. (1987) "information technology planning in the 1990's: directions for practice and research", mis quarterly, 11(1), pp 59-71. de vaus, d.a. (1991) surveys in social research, sydney: allen & unwin. cohen, j. & cohen, p. (1983) applied multiple regression/correlation analysis for the behavioral sciences, second edition, hillsdale, new jersey: lawrence erlbaum associates. cronbach, l.j. (1951) "coefficient alpha and the internal structure of tests", psychometrika, 16, pp 297-334. dess, g.g. & robinson, r.b. (1984) "measuring organizational performance in the absence of objective measures: the case of the privately-held firm and conglomerate business unit", strategic management journal, 5, pp 265-273. doll, w. (1985) "avenues for top management involvement in successful mis development", mis quarterly, 9(1), pp 17-35. galbraith, j.r. (1973) designing complex organisations, reading, massachusetts: addison-wesley. isaacson, p. (1994) hospital and health services year book (18th edition), prahran, victoria: peter isaacson publications. jaccard, j., turrisi, r. & wan, c. (1990) interaction effects in multiple regression, newbury park, california: sage publications. jarvenpaa, s.l. & ives, b. (1991) "executive involvement and participation in the management of information technology", mis quarterly, 15(2), pp 204-227. johnston, h.r. & carrico, s.r. (1988) "developing capabilities to use information strategically", mis quarterly, 12(1), pp 37-48. lederer, a.l. & mendelow, a.l. (1987) "information resource planning: overcoming difficulties in identifying top management objectives", mis quarterly, 11(3), pp 389-399. lederer, a.l. & sethi, v. (1992) "root causes of strategic information systems planning implementation problems", journal of management information systems, 9(1), pp 25-45. mcfarlan, f.w. (1971) "problems in planning the information system", harvard business review, 49(2), pp 75-89. merchant, k. (1981) "the design of the corporate budgeting system: influences on managerial behaviour and performance", the accounting review, 56(4), pp 813-826. norusis, m.j. & spss inc. (1993) spss® for windows™: base system user's guide, chicago, illinois: spss inc. nunnally, j.c. (1967) psychometric theory, mcgraw hill, new york. porter, m.e. (1985) competitive advantage, london: the free press. porter, m.e. & millar, v.e. (1985) "how information gives you competitive advantage", harvard business review, 63 (4), pp 149-160. raymond, l., pare, g, & bergeron, f. (1995) "matching information technology and organizational structure: an empirical study with implications for performance", european journal of information systems, 4,pp3-16. rockart, j. f. & crescenzi, a. d. (1984) "engaging top management in information technology", sloan management review, summer, pp 3-16. schewe, c.d. (1976) "the mis user: an exploratory behavioural analysis", academy of management journal, 19(4), pp 577-590. ajis vol 5, no 1 september 1997 96 appendix senior it manager survey form thank you for agreeing to provide information for our research project. this survey form is to be completed by the senior it manager. it has been designed to collect information about your hospital and about the role your ceo plays in the use of information technology and information systems (it) within your hospital, and contains nineteen questions. the survey form is designed to be completed in around ten (10) minutes, to ensure that not too much of your time is taken. nevertheless, the information you provide is important in assessing the role your ceo plays with respect to it within your hospital, and we ask that you give it careful consideration. on completion could you please return the survey form to us by post using the enclosed stamped addressed envelope. section i: asks about your hospital. section ii: asks about your organisational status as senior it manager within the hospital. section iii: asks about the approach your ceo takes to it within your hospital. for each of the following questions you are asked to circle the number representing the most appropriate response. section i hospital profile 1. what is the size of your hospital? (approximate number of maintained beds) optional comment: 2. what is the proportion of funds received by your hospital from a casemix formula? 5 mostly all 4 more than half 3 about half 2 less than half 1 very small 3. state the estimated proportion of funds received via casemix formula. 4. how would you describe your hospital's use of it (information technology and information systems)? optional 5 a leading hospital 4 close follower 3 middle of the pack 2 somewhat behind 1 laggard comment: 5. is your hospital a private hospital or a public hospital? section n your organisational status as the senior it manager 1. what is the title of the person to whom you report? 2. how many levels in the organisation hierarchy are you from the ceo? 1 1 1 aj1s vol 5, no 1 september 1997 97 3. do you have an it, administration or other (eg. medical) training? 1 it training only 2 administration training 3 other training [specify below] other training [please specify] 4. for how many .years have you served as a senior it manager? 1 2 3 4 5 more than 5 5. for how many years have you served as a senior it manager in this hospital? 1 2 3 4 5 more than 5 section m chief executive officer [ceo] characteristics 1. which of the following statements best describes the importance that your ceo perceives it (information technology and information systems) to be for your hospital? 6 considers it as the single most critical factor for the hospital 5 considers it as one of the vital parts of the competitive strategy 4 considers it to be vital for smooth functioning of operations 3 considers it to be one of the many ways to cut costs in the hospital 2 considers it to be the concern of technologists, not managers, although is supportive of it 1 has little concern for the potential utility of it optional comment: 2. how often does your ceo get personally involved in matters related to the use of it within your hospital? optional 5 daily 4 weekly 3 monthly 2 few times a year 1 less than once a year comment: j 3. how frequent are informal contacts between the ceo and the hospital's senior it management? optional 5 daily 4 weekly 3 monthly 2 few times a year 1 less than once a year comment: ajis vol 5, no 1 september 1997 98 4. how knowledgeable is the ceo about it opportunities and possibilities for your hospital? optional 5 extremely knowledgeable 4 very informed 3 well informed 2 somewhat informed 1 weakly informed comment: 5. how knowledgeable is the ceo about it innovations that have been developed by other hospitals? optional 5 extremely knowledgeable 4 very informed 3 well informed 2 somewhat informed 1 weakly informed comment: 6. which of the following best describes the ceo's prevailing thinking about funds the hospital spends on it? 3 views it as an expense to be controlled 2 views it as a resource to be allocated fairly across organisational units 1 views it as a strategic investment optional comment: 7. how often does your ceo endorse major it investments that have not been endorsed by traditional justification criteria and procedures? optional 3 rarely 2 occasionally 1 frequently comment: 8. which of the following best describes the ceo's role in the hospital it steering committee? 5 is the defacto steering committee 4 chairs an it committee and actively participate in meetings 3 is a member of the it steering committee 2 it committee exists, but with minimal input or awareness from the ceo 1 no steering committee exists optional comment: 9. what is your ceo's vision for it? 4 a strong, but generic vision ("we will be the leader in hospital use of advanced it') 3 a technical vision of how the hospital will use it ("we will install a database that keeps track of all patients") 2 a functional vision of how the hospital will use it ("95% of patient information will be correctly coded") 1 no vision for it optional comment: ajis vol 5, no 1 september 1997 99 thank you for completing this survey form. please return it in the attached stamped addressed envelope. write your name and address on the back of the envelope if you would like a copy of the findings of this research project. ajis vol 5, no 1 september 1997 ajis vol. 7, no 2 may 2000 employee perception of the impact of information technology investment in organisations: a survey of the hotel industry bruce w.n. lo school of multimedia & information technology southern cross university, lismore, nsw australia 2480 email: blo@scu.edu.au cede sri darma pendidikan national university (undiknas) jl. tukad yeh aya p o box 2361, denpasar, bali (80225) indonesia email: sridarma@indo.net.id abstract to improve their operational efficiency and to maintain their competitiveness in the market place many organisations continue to invest substantially in their information technology (it) capability. but how do the employees themselves view the impact and the value of it investment? this paper considers this question in the context of the hospitality industry. more specifically, it investigates the different perception of hotel employees on the impact of organisational it investment on (a) it usage by employees, (b) employee satisfaction with the it systems, (c) changes in the level of employee performance, and (d) organisational performance of the hotel. 945 hotel employees in bali, indonesia were surveyed to assess their perception of the organisational impact of it. factor analysis, analysis of variance and regression analysis were performed on the data to examine the' range of employee viewpoints. results showed that there were significant differences among the employees' perception with respect to age, educational level, position in the hotel, and individual income. these findings suggest that even senior management may regard the investment in it is worthwhile, there may be a divergence of opinion among the employees as to what are the real benefits of it. in order to fully leverage the potential of their it investment, managers should pay particular attention to shaping the views of the employees who ultimately determine whether the it capabilities in an organisation are put to use. introduction the impact of information technology (it) investment in organizations has received considerable attention in the literature during the past few years (kauffman & weill, 1989; harris & katz, 1991; weill, 1992a; 1992b; wilcox, 1994; raymond et al., 1995; byrd & marshall, 1997; sriram et al., 1997; sohal & ng, 1998). on the one hand ft is seen as the enabling agent through which organisations can improve their operational efficiency and strategic position in an increasing competitive business environment. on the other hand, in the current economic climate, many are questioning the real benefits of the ever increasing it investment in the organization. past research on the impact of it investment on organisational performance has been rather inconclusive. research literatures do not give reliable generalisation about the relationships between it investment and organisational changes (powell & dent-micallef, 1997; fitzgerald, 1998). some studies found a positive relationship between the two, while other actually found no relationship at all. it is generally believe that there are other factors, which come into play. unfortunately, too often the decision to invest was made based on fear of being left behind by competitors rather than on a genuine understanding of the real benefits that it can bring to the organisation (weill 1992a). this lack of understanding can lead to the organisation's inability to generate a shared vision among employees on how to leverage the real potential of it in that organisation. consequently the huge investment in it committed by the organisation sometime did not produce the full benefits that was expected at the time when the decision to invest was made. the purpose of this study is to examine the different employee's perceptions and expectations of the impacts of ft investment on the organisation, and to develop an instrument to measure these perceptions. it is hoped that the findings will assist senior management to have a more realistic approach in making decisions on organisational investments in it and the associated expectation of the likely benefits. the findings will also provide some insights to senior management as to how to harness the human elements of the organisational information system to ensure that the technological capabilities of the system are fully utilised. in this study, the hotel industry was chosen as the subject of investigation. there are several reasons for this choice. firstly to the best knowledge of the authors, this industry has not been the central subject of study in this research area. secondly, during the last few years hotels have become increasingly dependent on computer and communications technology to manage the diverse range of service provision, needed in a highly competitive and globalised market. instead of conducting the study in a technologically developed country, the authors have chosen to study the organisational impact on hotels at a regional location, viz. bali, indonesia, which is considered as a part of the developing region of the world with rapidly changing national information infrastructure. but in terms of tourist demand, bali is no less significant than any other international destinations. hence the need of technology support to maintain a highly competitive customer service is just as real as in other parts of the world. this challenge is clearly recognised by the local tourism authority (mardani, 1997). the remainder of this paper is organized into seven sections. the next (second) section begins with a brief review of previous research in this area. the third section provides a brief description of the research methodology employed in this study. descriptive data analysis, and the reliability and validity of the instrument are reported in the fourth section. the demographic differences on employee perception are examined in the fifth section. finally the different perceptions by employees of the impacts of information technology investment in the organisation are reported in the sixth section. 32 ajis vol. 7, no 2 may 2000 the paper concludes with a discussion on the implications of these findings for management. previous research given the high level of financial stakes involved, the investigation on the impact of information technology investments on organisational performance and organisational processes has been and continues to be a major research concern for practitioners and academics (willcocks, 1996). previous studies have evolved from early anecdotal reports to more recent empirically-based research (e.g. sriram et al., 1997). they also covered a wide range of perspectives including investigation on research methodologies (orlikowski & baroudi, 1991), the extent of it use (weill 1992b), the concept of organisational performance and business value (mukhopadhyay et al., 1995), the strategic advantage of the firm (weill, 19992a, mata et al., 1995), the impact on inter-organisational relationships, the impact on the management function and its structure, the change in organisation's work environment, and even the very nature of the organisation's core business in the marketplace (willcocks, 1994). the relationship between it and the organisation is multi-facet and highly complex. the impact of it investment in organisations is far reaching, and may affect not only the internal attributes but also the external attributes of a firm. attempts to a isolate particular aspect of the relationship often lead to different or even seemingly contradictory results. for example, a careful review of research studies on the impact of organisational it investment on organisational performance is likely to result in an inconclusive view of the relationship, as studies reported both positive and negative impacts, as well as no relationship at all. to highlight the diverse perspectives on this relationship, three tables are presented which provide a summary on the findings of major studies by different researchers on this issue. table 1 summarises those studies, which yielded a positive relationship. table 1: studies showing positive relationship between it investment and organisational performance references bender (1986) banker, kauffman & morey(1990) harris & kats (1991) weill (1992b) santos, peffers & mauer(1993) mahmood(1994) raymond, pare & bergeron (1995) lubbe, parker & hoard (1995) siram, stump & banerhee(1997) industry insurance fast food stores insurance valves manufacturing finance and manufacturing 8 1 firms from computerworl d in a nonrandom sample manufacturing insurance purchasing key variables and key findings optimum level of performance achieved at a range of it investment from 15% to 25% of total operating cost of the firm. stores with the technology and large breakfast sales (more complex breakfast menu) performed significantly better in terms of cutting materials handling costs. firms with the most improvement in organisational performance allocated a significantly higher proportion of their non-interest operating expenses to it. transactional it investment correlated positively with organisational performance market value of the firm reacted positively to the announcements of it investment by the organisations. firms in the efficient group had a much higher return on their it investment than the inefficient group. it usage is positively related to organisational performance and the relationship between it management and structural sophistication is stronger among the better performing firms. positive relationship found between the computerisation index and the financial ratios; the most profitable firms were more likely to spend a higher proportion of the non-interest operating expenses on it. positive relationship was found: it used in purchasing is not a homogenous phenomenon, but can be represented by three componentsbasic computer systems & support, purchasing-specific applications, and 33 ajis vol. 7, no 2 may 2000 rai, patnayakuni & panayakuni (1997) purchasing vender communications. it investment made a positive contribution productivity. to the firm's output and labor table 2 summarises those studies which gave a negative relationship. finally, table 3 lists those studies where the authors found no relationship exists between it and organisational performance. for other recent reviews on the subject, readers are referred to the articles by powell & dent-micalleff (1997), mahmood (1997), and sohal & ng (1998). table 2: studies showing negative relationship between it investment and organisational performance references cron & sobol (1983) roach (1988) kauffman & weill (1989) weill (1992b) hitt and brynjolfsson (1996) industry warehousing finance valves manufacturing valves manufacturing finance key variables and key findings firms that make extensive use of computer have either very week financial performance. it investment had negative and dysfunctional effects performance. a very strong or a on organisational very little evidence of the value contributed by it investments. strategic it investment had a negative relationship performance to organisational small negative relationship between computer capital and total shareholder return in at least one year. table 3: studies showing no relationship between it investment and organisational performance references turner (1985) strassmann (1985) santos, peffers & mauer(1993) powell and dentmicallef(1997) industry banking service sector firms finance and manufacturing retail industry key variables and key findings no relationship was found between organisational performance and relative proportion of resources allocated to data processing. no significant relationship was found between high performing firms the size of it investment. the and no excess returns was found for either the full sample or for any one of the industry subsamples. it alone had not produced sustainable performance advantages. recognising the complexity of the issues under consideration, some authors attempted to investigate the intervening factors in the it investment-organisational performance relationship. floyd and wooldridge (1990) examined the evidence of it investment having a moderating effect on the organisational competitive strategy and financial performance. on the other hand, parthasarthy and sethi (1993) investigated how strategy and structure moderate the relationship between capital outlay in computer automation and firm performance. in their study, they found that the flexibility of business and manufacturing strategies, quality of leadership, and cost leadership had a significant moderating effect. more recently, sohal and ng (1998), in their study of 530 companies in the australian manufacturing, banking and finance, and insurance industries, found that the greatest impediments in unleashing the full potential of it in an organisation are the misalignment of business and it strategies, lack of change in historic it structure, lack of a full understanding of the potential of it, and lack of senior management support. in the context of these findings, the present study attempts to go beyond simple financial measures of organisational performance and consider a number of user-organisation variables as perceived by the employees which may be affected 34 ajis vol. 7, no 2 may 2000 by the organisational investment in it. therefore, as well as examining the employees' (in their roles as the user of the it systems) perceptions on the organisation performance, this study also probe their perception on (a) user satisfaction with the organisational information system, (b) the level of it usage and (c) the effect of it on employee performance. the basic premise is that, the employees' perception of the benefit of it investment plays an important role in determining whether the investment itself will lead to actual improvements in organisational performance. methodology this study follows the survey research methodology. based on previous research in related areas, a questionnaire was constructed to measure the perceptions of the employees on each of the following five variables: • the level of information technology investment (iti) by the organisation, • the amount of information technology usage (itu) in the organisation, • the level of user satisfaction (us) with the organisational information system, • the level of employee performance (ep), and • the level of organisational (in this case the hotel) performance (hp). after pilot testing, the questionnaire was administered to a group of subjects consisting of the employees in twenty 4and 5-star hotels in bali, indonesia. the data were analysed using standard techniques of factor analysis, analysis of various and multiple regression to examine the different employee perceptions on these variables. the remainder of this section gives a brief description of the sample, the survey instrument and the survey procedure. sample the target group chosen for this study, were hotel employees in bali, indonesia, who regularly used computers in the performance of their duties. they include those working at both the front counters and at the back offices as well as people in management positions. in addition to standard hotel reservation and accounting, these employees used a variety of computerised applications in their daily work. it is thought that the different employment situations will provide a wider and more representative viewpoint in the survey. only hotels with 5and 4-star rating were chosen rather than the lower ranking hotels, because typically hotels in the former group are better equipped with it facilities and they tend to cater for international tourists. a larger proportion of employees in these hotels was supplied with pcs. all pcs and minicomputers in the hotels surveyed in this study were connected to the corporate computer network. survey instrument a four-page questionnaire consisting of six subscales was developed. in the first subscale, demographic information such as age, gender, marital status, position in hotel, educational background, and individual income level were sought. in the remaining five subscales, questions were adapted from similar instruments reported in the literature by previous researchers, to measure the perception of the subjects on the five variables under consideration, viz. information technology investment by the organisation (iti), information technology usage by employees (itu), user satisfaction with the it facilities (us), level of employee performance (ep) and level of hotel performance (hp). the structure of the questionnaire is summarised in table 4. the main references on which the subscales are based or adapted are also indicated in the table. each question in the last five subscales of the questionnaire was scored on an 11-point likert scale ranging from (0) "not at all/almost never" to (10) "extremely large/almost always". the two "frequency" questions in the itu subscale, naturally involve counting of the number of applications and is not based on the likert rating scale. 35 ajis vol. 7, no 2 may 2000 table 4: structure of the questionnaire subscale 1 demo 2 iti 3 hp 4 itu 5 us 6 ep what it measures demographics information technology investment hotel performance information technology usage user satisfaction employee performance key elements of the questions age gender marital status position in hotel educational background income level hardware investment software investment training costs support costs total investment hotel financial performance hotel room occupancy number of visitors frequency of use time of use criticality of use feeling of use number of business task used number of computerised applications provide precise information information content meet need provide reports needed sufficient information computer system accurate satisfied with accuracy output in useful format information clear user friendly ease of use information on time up to date information productivity job performance effectiveness of the job decision making quality source references sriram, stump & banerhee (1997) jeffrey & hubbard ( 1994) straub, limayem & karahanna-evaristo (1995); igbaria & tan (1997) doll &torkzadeh( 1988) igbaria & tan (1997) information technology investment to measure information technology investment, it is not sufficient to consider the total cost of it equipment. sriram et al. (1997) pointed out that, in addition to hardware, software and telecommunications costs, there are other costs associated with the support and operation of it facilities in an organisation. this study identified four components: computer hardware cost; computer software cost; the cost of training it users; and the cost of supporting it users during normal operation. telecommunication costs have been absorbed into the other components. the authors believe that by measuring each of these separately provides a more complete and reliable picture on the organisational it investment. information technology usage several researchers (straub et al., 1995; igbaria & tan, 1997) have reported instruments with reliability • = 0.7 or higher to measure information technology usage. from their works, six measures were chosen to be used in this study: (i) frequency of use; (ii) time of use; (iii) criticality of it usage; (iv) feeling of it usage; (v) number of business tasks for which the system was used by employees; and (vi) number of computerized applications used by employees. a mixture of semantic differential scales and likert-type scales was used to record responses to the survey questions. the number of business tasks involving the computer and the number of different computer applications used were employed as an indication of the extent of it usage. user satisfaction delone and mclean (1992) argued that user satisfaction occupies a key role in information system success. in the present study, user satisfaction is taken to mean the employees' attitude towards the use of computer facilities as they 36 ajis vol. 7, no 2 may 2000 perform their daily duties in the hotel. the scale, developed by doll and torkzadeh (1988) with • = 0.8, was adapted here to measure the overall user satisfaction that includes a measure of the extent to which organisational information systems meet the end-user's needs with regards to information content, accuracy, timeliness, format and ease of use. employee performance employee performance was measured based on a scale developed by igbaria and tan (1997) (a = 0.8). what is being measured here is the employees' perception of their own performance improvement (or the lack of it) in four indicators as a result of the use of it facilities. the four indicators are (i) productivity on the job, (ii) job performance, (iii) effectiveness on the job, and (iv) decision-making quality. hotel performance organisational performance is a complex phenomenon. any finite set of measures for hotel performance is likely to be too simplistic. financial indicators alone are not sufficient to capture the broad scope of hotel performance. therefore three different global measures were chosen in this study: (i) hotel financial performance; (ii) room occupancy rate; and (iii) the number of visitors. this set of measures provides a performance profile in terms of occupancy and visitation rate as well as financial performance. it was found to be a better indicator of the overall performance of hotels (jeffrey and hubbard, 1994). survey procedures the instrument was first pilot tested on a small group of 12 hotel employees selected from lismore and gold coast areas in australia and from bali in indonesia. preliminary analysis of the pilot data showed that those completed the survey form were generally happy with the questions asked. minor changes were made to the item statements to improve clarity of presentation. prior to the conduct of the full survey in 1998, the approval and support of the director of the department of tourism, post, and telecommunication at bali was obtained. as a result, the researchers were able to claim official sanction and legitimacy of the research project, thus improving the response rate. to assist with the data collection process, the department also provided useful information on the addresses of the twenty-eight 4and 5-star hotels in bali, their phone and facsimile numbers, the names of the hotel general manager, the number of employees, and the number of rooms in each hotel. with a letter of introductory from the bali tourism director, initial telephone contacts were made with the general manager of all 28 hotels in bali, and permission was sought to conduct a survey in their hotels. out of the 28 hotels, twenty were willing to participate in the study representing 71% of the population group (in hotel counts). it is believed that this should provide an adequate sample of the target population. these 20 hotels employ a total of 11,529 employees. of these, 1,131 or approximately 10% used computers directly in their daily duties. questionnaire forms were then hand-delivered to these 20 hotels for distribution to the 1,131 employees. the survey package consists of a 4-pages questionnaire, a covering letter, a guide on how to fill the questionnaire, and the letter of introduction from the director of the bali department of tourism, post and telecommunication. hotel general managers who had agreed to participate were urged strongly to encourage their employees to complete the questionnaire. as an incentive, a souvenir key ring from australia was offered as a token of appreciation to all those employees who have returned a completed questionnaire, a gesture that proved to be very popular. to ensure a degree of objectivity in the survey data, selected employees were personally interviewed by the researchers to verify the accuracy of the self-reported data. as far as the researchers were able to ascertain, the self reported data were found to be reliable. data analysis the data collected from the survey were scored and entered in the computer for analysis by the spss package. some preliminary results relating to the sample characteristics, the reliability of the questionnaire, and the validity of the measures are reported in this section. respondent characteristics of the 1,131 questionnaire forms distributed, 945 completed forms were returned, representing a response rate of 83%, which is considered an acceptable level of response in this type of research. listed in table 5 are the id of the hotels that participated (names were suppressed for confidentiality), the number of employees in each, the number of completed questionnaires returned, and the number of persons interviewed for validation purpose. 37 ajis vol. 7, no 2 may 2000 table 5: questionnaire response rates hotel id. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 total/ave star level 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 4 4 4 4 4 total employees 985 804 784 773 758 751 717 708 693 670 628 592 580 418 386 309 292 234 227 220 11,529 it end-user employees 90 78 80 70 76 75 100 55 60 70 40 55 60 40 40 30 30 30 27 25 1,131 questionnair es returned 80 61 60 55 60 71 78 35 56 62 36 50 55 32 35 28 25 24 22 20 945 percentage of returns 88.9 % 78.2 % 75.0 % 78.6 % 78.9 % 94.7 % 78.0 % 63.6 % 93.3 % 88.6 % 90.0 % 90.9 % 91.7% 80.0 % 87.5 % 93.3 % 83.3 % 80.0 % 81.5 % 80.0 % 83.6 % persons interviewed 5 4 4 4 4 4 5 3 3 4 2 3 3 2 2 2 2 2 1 1 57 an examination of the individual response rate for each hotel, all except hotel no. 8, have a response rate of 75% or above. even in the lowest case, the response rate is 63.6%. therefore, the data collected may be considered a reasonable representation of all the it end-user employees in these hotels. the biographical characteristics of the respondents in this survey are summarised in figures 1 to 6. with respect to age (figure 1), the majority of the employees surveyed were relatively young (70.6 % of the respondents fell in the age range of 25-40). the mean age of the respondents was 34. it is generally believed that younger workers are figure 1: age distribution 37% • 18-24 q 25-30 d31-40 41-50 d>56 figure 2: gender distribution likely to be more receptive to the introduction of new information technology into their workplace environment. as shown in figure 2, the majority of the respondents, 64% were male and only 36% were female. this gender distribution reflects the typical trend of the workforce in indonesia. figure 3: position in hotel 20% 12 12% figure 4: educational level 23% 42% 38 ajis vol. 7, no 2 may 2000 as for the nature of their employment positions, the largest group of respondents (27%, see figure 3) were working in the accounting and finance sections of the hotel, signifying that this job function is probably the most computerised group in the hotel. the second largest group of subjects worked in the front office. it is interesting to note that a significant proportion (20.4%) of the respondents worked in others departments such as engineering, executive office, banqueting, food and beverage, kitchen, laundry, housekeeping, material, and time keeping. this seems to indicate that the application of information technology is reasonably wide spread among these hotels. figure 4 shows the educational levels of the subjects. over two-thirds of the hotel employees participated in this survey (68%) have at least either a tertiary diploma or university degree. about 30% have only completed high school education. the trend of the hospitality industry in employing more professionally trained workers was evident here. in terms of the monthly income level (figure 5), the largest group (53.7%) fell in the range rp. 500,001 to rp. 1,000,000, while a further 40.7% was remunerated at a higher level. according to statistics office of bali province 12 16 figures: monthly income (in 1000's rp) 13% 6% 3% figure 6: marital status (1997) the modal monthly income in bali for all employment is in the range rp. 300,000 to rp. 450,000. this means that the people employed in the hospitality industry enjoyed a much higher level of pay compare to all employment groups. finally figures 6 lists the marital status of the respondents. reliability of the scale table 6 gives the descriptive statistics: means, standard deviation and factor loading (to be discussed later) for each item in the questionnaire. it is noted that all items were found to have a mean value ranging from 7.7 to 9.1 in the 11-point scale, where a value of 5 is regarded as the neutral point. this indicates that the ratings from the respondents tend to lie on the positive side of the rating scale. furthermore, the standard deviations were found to range from 0.81 to 1.49, indicating a relatively high degree of consensus among the respondents in their perception of the rating of the five variables in question. factor analysis was carried out on each of the five research variables with respect to items in the questionnaire. in each case only one factor was identified, accounting for 74% to 84% of the variance in the set of questionnaire items. the factor loading for each item is shown in table 6. the homogeneity of the factor loading coefficients is to be expected as most items are adapted from other similar instruments. to assess the reliability of the instrument, the cronbach (1981) alpha coefficients for the total questionnaire and the five subscales were calculated and reported in table 7. the cronbach alpha is the most widely used index for determining internal consistency (kerlinger, 1986). it has been generally accepted that in the early stages of research on hypothesized measures of a construct, reliabilities of 0.70 or higher are needed, while for widely used scales, the reliabilities should not be below 0.80 (nunnally, 1978). in the current survey, all subscale alpha coefficients exceed 0.90 with an overall alpha value of 0.94 for the entire questionnaire. the high alpha value in all five subscales confirms the homogeneity of the items comprising them, and indicates an acceptable level of reliability. 39 ajis vol. 7, no 2 may 2000 table 6: descriptive statistics for the five variables (iti, itu, us, ep and hp) code iti-1 iti-2 iti-3 iti-4 iti-5 hp-1 hp-2 hp-3 itu-1 itu-2 itu-3 itu-4 itu-5 itu-6 us-1 us-2 us-3 us-4 us-5 us-6 us-7 us-8 us-9 us10 us11 us12 ep-1 ep-2 ep-3 ep-4 variables hardware computer investment software computer investment training cost of using information technology job support cost of using information technology total investment in information technology hotel financial performance hotel occupancy room the number of visitors frequency of use time of use critical of it usage feeling of it usage number of business tasks using it number of computerized applications provide precise information information content meet your need provide reports what you need provide sufficient information computer system accurate satisfied with the accuracy the output in an useful format the information clear system user friendly ease of use information need on time up to date information productivity job performance effectiveness of the job decision making quality factor loading 0.93 0.94 0.82 0.90 0.91 0.79 0.95 0.96 0.90 0.90 0.92 0.92 0.77 0.73 0.92 0.91 0.89 0.91 0.90 0.90 0.93 0.94 0.87 0.87 0.89 0.85 0.90 0.95 0.91 0.85 mean (scale 0-10) 8.19 8.18 7.76 8.07 8.15 8.11 7.91 7.93 9.03 8.79 9.06 9.11 9.15 8.54 8.60 8.53 8.60 8.51 8.55 8.64 8.61 8.62 8.42 8.53 8.55 8.49 8.32 8.30 8.42 8.30 standard deviatio n 1.35 .35 .50 .41 .34 .24 .20 .21 0.84 0.83 0.85 0.82 1.13 0.99 1.07 1.10 1.11 1.12 1.10 1.10 1.05 .07 .23 .10 .12 .21 .36 .32 .29 1.32 table 7: cronbach's alpha coefficient of reliability name of subscale information technology investment (iti) hotel performance (hp) information technology usage (itu) user satisfaction (us) employee performance (ep) entire questionnaire variable code j.ti1 iti5 (5 items) hp1 hp3 (3 items) itu1 itu6 (6 items) us1-us12 (12 items) ep1 ep4 (4 items) all 30 items reliability (cronbach's alpha ) 0.938 0.920 0.934 0.970 0.938 0.939 validation of the perception measures in using perception rating to assess user-organisation variables, a reasonable question to ask is whether the employee's perception is an accurate measure of the reality? for example, how much one can rely on the employee's perception to measure the organisation's actual investment on it? or to measure the organisation's actual performance? to answer these questions, permissions were obtained to access the actual annual records of selected hotels to extract the objective measures of it investment and hotel performance. in all, six hotel general managers gave their permission to the researchers to search their company records. actual figures of the hotels' investment in it in the previous year were obtained, including hardware costs, software costs, it training costs, user support costs, and total it investment. also, actual figures of the hotels' performance in the previous years were obtained, including the net financial return, hotel occupancy rate and the number of visitors to the hotel. these actual values were compared with the subjects perception data obtained from the questionnaire by regressing the actual measures on the perception measures. table 8 gives the results of regression analyses. the "r square" values are all greater than 0.77, indicating that at least 77% of the variance in the actual measures can be accounted for the 40 ajis vol. 7, no 2 may 2000 perception data. in fact, most of the items are above 90% showing a strong relationship between the two. this means, according to the actual data from the hotel records, the perception ratings by the employees could be considered to be reliable indicators for the variables under consideration. table 8: regression of actual data on employees' perception var iti hp question items computer hardware costs (iti-1) computer software costs (iti-2) costs of training it users (iti-3) costs of supporting it users (iti-4) total costs of it investment (iti-5) financial performance (roi) (hp-1) occupancy rates (hp-2) number of visitors (hp-3) r square 0.927 0.926 0.962 0.771 0.811 0.897 0.972 0.933 df 4 4 4 4 4 4 4 4 f 50.76 49.98 102.60 13.47 17.20 34.66 136.60 55.68 sig. 0.002 0.002 0.001 0.021 0.014 0.004 0.000 0.002 bo constant 6.816 7.2189 6.293 6.847 1.121 7.2996 5.970 7.079 bl 2.4e-09 3.9e-09 1.5e-07 3.2e-08 4.7e-10 0.0174 0.0301 1.9e-05 the regression relationships between the actual data and the employees' perception are illustrated in figure 7. these graphs clearly show the close relationship between the actual and perception data. for the remainder of this paper, the perception measures will be used in subsequent analyses. figure 7: regression of actual data on perception measures computer hardware costs perception data (iti-1) 77 actual data (itm) computer software costs perception data (iti-2) actual data (iti-2) imoooooo 100000000 200000000 costs of training it users perception data (iti-3) actual data (iti-3) costs of supporting it users perception data (iti-4) 20000000 actual data (itm) 41 ajis total costs of it investment perception data (iti-5) vol. 7, no 2 may 2000 300000000 400000000 500000000 600000000 700000000 boooooooo actual data (iti-s) financial performance (roi) perception data (hp-1) actual data (hp-1) occupancy rates perception data (hp-2) number of visitors perception data (hp-3) actual data (hp-2) actual data (hp-3) here "actual data" is shown as the independent variable (x-axis), and "employee perception" is shown as the dependent variable (y-axis). factor analyses were performed on each of the subscales. all five subscales were found to contain only 1 factor, indicating that each subscale is relatively homogeneous. this is not surprising because the subscales were constructed based on items selected from validated instruments in the literature for these variables. this means in subsequent analysis, one may use a single factor score rather than each individual item score. demographic differences among respondents' perception the purpose of this section is to examine whether there are differences in the five dependent variables, viz. it investment, it usage, user satisfaction, employee performance, and hotel performance, with respect to groups in the category variables age, gender, position in hotel, education level, and individual income as independent variables. the factor scores of the dependent variables were used in the analysis. some of the smaller frequencies in the original categories have been merged in the analysis. first, a multivariate analysis of variances model was fitted for all five dependent variables with the five category variables as independent variables. effects tested were main effects only, i.e. the tests for each effect was adjusted for possible confounding with all other effects in the model (they are unique effects). post-hoc comparisons among the levels of variables showing significant effects were performed by means of the bonferroni procedure (hair et al., 1995). the results of the multivariate analysis are shown in table 9. table 9: test of multivariate significance effect age gender position in hotel education level individual income wilk's lambda 0.912 0.993 0.947 0.947 0.933 f 3.408 1.068 1.986 1.948 2.575 d.f 20,2419 5,7290 20,2419 10,1458 20,2419 sie. p p= 0.000 p= 0.377 p= 0.006 p= 0.035 p= 0.000 significant differences are shown in bold. significant differences (shown in bold in the table) in the set of dependent variables were found among the categories with respect to four of the five categories variables viz. age, position in hotel, education level and individual income. there were no significant differences between females and males. 42 ajis vol. 7, no 2 may 2000 univariate analyses of variance were then performed to identify the dependent variables on which independent variables had significant effects. the results of the univariate tests are shown in table 10. an examination of the table shows, • different "age" groups differ significantly re their perception of the level of it investment, user satisfaction and employee performance; • different "individual income" groups differ significantly re their perception on the level of it investment and hotel performance; • different "position in hotef' groups differ significantly re their perception on the level of user satisfaction; and • different "education lever groups differ significantly re their perception on the level of hotel performance. table 10: test of univariate significance dependent variable | effects | f | d.f. it investment it usage user satisfaction employee performance hotel performance age position in hotel education individual income age position in hotel education individual income age position in hotel education individual income arc position in hotel education individual income age position in hotel education individual income 5.605 2.226 0.233 5.671 2.394 2.233 1.640 2.375 4.443 3.049 0.343 0.731 4.285 1.082 0.981 1.070 2.061 1.630 6.108 2.806 4,733 4,733 2,733 4,733 4,733 4,733 2,733 4,733 4,733 4,733 2,733 4,733 4,733 4,733 2,733 4,733 4,733 4,733 2,733 4,733 sje.p p= 0.000 p= 0.065 p= 0.792 p= 0.000 p= 0.051 p= 0.064 p=0.195 p= 0.051 p= 0.001 p= 0.017 p=0.710 p= 0.571 p= 0.002 p= 0.364 p= 0.375 p= 0.370 p= 0.084 p=0.165 p= 0.002 p= 0.025 significant differences are shown in bold. this appears to indicate that age has the widest significant influence affecting the respondents' perception in three out of the five dependent variables, while individual income level affects two dependent variables, and position in hotel and education level each only affects one. however, there is no category variable which has a significant effect on it usage. this suggests that an employee's age, position in hotel, education level and income do not affect their perception of the level of it usage by that employee. this seems to indicate that the level of it usage is more likely to be determined by the need of the tasks at hand rather than by any of these category variables. post-hoc test by the bonferroni procedure were employed to identify which categories (or groups) of these independent variables differ significantly on the dependent variables identified above. age effect in this section, the differences in employee perception for the different age groups will be examined in greater detail. the three research variables under consideration relates to the employees' perceptions of it investment, user satisfaction and employee performance. the results are summarised in figure 8 below where the mean levels (deviation from the total mean) of each age group are plotted. with respect to the employees' perception of the level of it investment, both the youngest group, i.e. age "up to 24 years" and the oldest group, i.e. age "56 years & above", were found to be significantly higher than that of the three age groups in the middle. the perception of the oldest group was particularly positive. it is interesting to observer that even the actual level of it investment is a relatively objective measure, the subjective perceptions of employees still differ significantly among the groups. this highlights the need for the senior managers to explicitly inform and properly promote awareness among their employees of their decisions to invest in it resources for the organisation. 43 ajis vol. 7, no 2 may 2000 one possible explanation of this difference in perception may be that the oldest group still remembers the former days when things are totally done manually and hence are grateful for the improvement they are now experiencing. next, it is observed that the mean user satisfaction level of the age group "up to 24 years" was significantly lower than both that of the age groups "41-55 years" and that of "56 years above". the mean satisfaction levels of both the age figure 8: differences in perception by different age groups t • it investment • user satisfaction employee performance age groups groups "25 to 30 years" and "31 to 40 years" were significantly lower than that of "56 years above". generally speaking, the younger groups are less satisfied than the older groups. there is a clear trend from these data to indicate that the employees' perception of user satisfaction increases with increasing age. the variable being measured here is not general satisfaction, but the user satisfaction with the computer facilities or the company's information system. this seems to further support the earlier assumption that, the older group tends to "appreciate" the new technology more because it is likely that they started their working career in a less automated, and hence more administratively cumbersome work environment similarly it was found that the perception of the three younger age groups "up to 24 years", 25-30 years, and "31-40 years" on employee performance were significantly lower than those of the older age groups "41-55 years" and "56 year above". however the perceived differences were not as marked as in the previous two cases. this means the older employees had a more positive feeling about their level of performance than the younger ones. these findings suggest a possible positive relationship among these three dependent variables, viz. it investment, user satisfaction and employee performance. more positive perception of it investment results in more positive user satisfaction. a more positive user satisfaction is associated with a more positive perception of employee performance. these relationships have been investigated and reported in greater details elsewhere (sri darma & lo 1998). 44 ajis individual monthly income vol. 7, no 2 may 2000 in this section, the differences in employees perception with different income levels are being examined. as shown in figure 9, significant differences were found among the different income groups on the perceived levels of it investment and hotel performance. figure 9: differences in perception by different income groups • it investment —•hotel performance s -0.5 i -1 0.5-1.0 1.0-1.5 1.5-2.0 >2.0 monthly income in millions of rp's with the exception of the two highest income groups, the two series on perceived it investment and hotel performance, were remarkably similar. generally speaking, those in the higher income groups were more positive in their perception of the levels of it investment by their hotel as well as the levels of hotel performance. however, there appears to be a "cross-over" in the transition from the second highest to the highest income groups. (the graph shows a "dip" on the right hand side.) there does not appear to be any logical explanation for this. it is also interesting to note that, the perception of the lower income group with "less than 500,000" rps. per month, was found to be significantly less than any other groups. generally speaking, as the monthly income increase, the perceived levels of it investment and hotel performance also improves. as is in the case of the age effect, this also suggests a positive relationship between the amount of it investment in an organisation and the perceived level of performance in that organisation. position in hotel in this section, the differences in the perception of employees in different positions in the hotel (different job functions) are examined. figure 10 shows the mean user satisfaction levels as perceived by employees who work in different departments of the hotel. 45 ajis vol. 7, no 2 may 2000 one group stands out prominently in this analysis. the perceived user satisfaction level of those working in the accounting/finance department was significantly higher than the satisfaction levels of employees who work in the other sections in the hotel. their satisfaction levels were even higher than those working in the information systems department, which ranked second in this comparison. this probably reflects the reality that computing facilities have been traditionally used to support mainly the accounting/finance functions in hotels. it is only in relatively recently years that other applications of the computers have been explored in other job functions in an organisation. this seems to indicate that familiarity or greater experience with computers is often associated with greater use satisfaction (and less figure 10: difference in perception by different employee positions d ev ia tio n f ro m m ea n 0 0 _.£. 0 1 . 0 _ -0.1 n •> _ — •— user satisfaction / . / marketing front offce humanrsi>x*x'lnfo sys accounting ^^^^ -\j,^ position of employees in hotel anxiety) with respect to information technology. on the other hand, the low user satisfaction level of the marketing group is probably related to the relatively low penetration of it in the marketing function in these organisations. educational levels with respect to educational level, there were significant differences found in the employees' perception on hotel performance (see figure 11). it appears that the higher the level of education of the employees, the less positive is their perception on the level of hotel performance. this may be attributed to the more critical attitudes and higher expectation level that are often associated with university graduates. figure 1 1 : differences in perception by different education levels d ev ia tio n fr om m ea n 0 4 -,.h o o . 0 0 . 0 -1 _ 0 . n 1 1 » hotel performance •: *̂ ~~~~'~~~^~---------̂ ^^-^ , . ,, _ . ^""-^x^' 1 -u.i -1 nign sen uipioma educational level 1 . "> • % , _ _ , ,uegree 46 ajis vol. 7, no 2 may 2000 relationships among the user-organisation variables in this section the relationship between the five research variables, viz. information technology investment (iti), information technology usage (itu), user satisfaction (us), employee performance (ep), and hotel performance (hp) will be examined. a correlation analysis was performed to explore the strengths of the relationship between pairs of variables. table 11 summarizes the pearson product-moment correlation coefficients between each pair among the five variables: table 11: pearson product-moment correlation between pairs of variable iti itu us ep hp iti 1.000 0.203 0.261 0.286 0.376 itu 1.000 0.387 0.549 0.341 us 1.000 0.588 0.402 ep 1.000 0.465 hp 1.000 all correlation coefficients are significant at the 0.01 level. while all variables are significantly and positively correlated at the 0.01 level, a closer examination of the table reveals that the strength of correlation appears to fall into three groups: • relatively strongly correlated with coefficient of determination r2 > 30%: viz. ep & itu (0.549), and ep & us (0.588). these are indicated in bold in table 11. • moderately correlated with 10% < r2 < 20%: viz. hp & itu (0.341), hp & iti (0.376), us & itu (0.387), and hp & us (0.402). these are indicated in italic in table 11. • weakly correlated with r2 < 10%: viz. iti & itu (0.203), iti & us (0.261), and iti & ep (0.286). three observations may be made from the results so far. firstly, the strongest relationship is among the three variables: information technology usage (itu), user satisfaction (us) and employee performance (ep). this appears to suggest that there is a strong belief among the employees that their performance is enhanced by higher computer usage and their satisfaction with the organisation's information system. this relationship will be explored further later in this section by regression analysis. the second observation is that, information technology investment (iti), while is moderately related to hotel performance (hp), correlates only rather weakly with three (itu, us, ep) out of the other four variables. this weak relationship may account for the reason why past research in linking it investment to organisational performance sometimes led to inconclusive/ambivalent results. the third observation is that, hotel performance (hp) appears to be roughly equally correlated to each of the other four variables (iti, itu, us & ep). this seems to imply that none of these variables taken singly (that includes it investment as well) can fully account for the variation in hotel performance. this supports the earlier argument that in order to fully leverage the it capabilities of an organisation, managers must also pay attention to fostering other aspects of the organisation-employee relationships. to further clarify these observations, several multiple regression analyses were performed. the results are reported below. determinants of employee performance in relation to the first observation mentioned above, a multiple linear regression analysis was performed by regressing employee performance (ep) on information technology usage (itu) and user satisfaction (us). the results are shown in table 12. 47 ajis vol. 7, no 2 may 2000 table 12: regression of employee performance on itu & us dependent variable user performance (up) regression equation regression coefficient information technology usage (itu) 0.378 (t= 14.654) user satisfaction (us) 0.441 (t=17.111) constant 1.632e-16 (effectively zero) up = 0.378 itu + 0.441 us + error other statistics r = 0.68 r2=0.47 f = 412.58 significant at 0.01 level this analysis shows that both information technology usage and user satisfaction contribute positively to employee performance. but the effect of user satisfaction is greater than that of information technology usage. this indicates that psycho-social factors such as user satisfaction with the organisational information system, are more important determinant of employee performance than the objective measures such as actual it usage. therefore it is not sufficient to just provide access to it use. it is very important to ensure that employees are satisfied (or feel satisfied) with the it facilities provided. what affects hotel performance? the next analysis investigates the main determinants of hotel performance. the results of the regression of hotel performance (hp) on information technology investment (iti) and employee performance (ep) are shown in table 13. table 13: regression of hotel performance on iti & ep dependent variable hotel performance (hp) regression equation regression coefficient information technology investment (iti) 0.264 (t=9.167) employee performance (ep) 0.390 (t=13.514) constant 1.932e-16 (effectively zero) hp = 0.264 iti + 0.390 ep + error other statistics r = 0.53 r2=0..28 f= 183.92 significant at 0.01 level while both it investment and employee performance significantly and positively affect hotel performance. the effect of employee performance is greater than that of it investment. this is to be expected, as intuitively speaking, one would expect the impact of employee performance is more direct. it investment can affect hotel performance only in an indirect way. what affects it usage? to examine what affects information technology usage (itu) in an organisation, a multiple linear regression analysis was performed by regressing itu on information technology investment (iti) and user satisfaction (us). the results are shown in table 14. 48 ajis vol. 7, no 2 may 2000 table 14: regression of it usage on iti & us dependent variable information technology usage (itu) regression equation regression coefficient information technology investment (iti) 0.110 (t=3.557) user satisfaction (us) 0.358 (t=l 1.590) constant 2.935e-16 (effectively zero) iyu = 0.110 itu + 0.358 us + error other statistics r = 0.40 r2=0.16 f = 90.40 significant at 0.01 level the interesting finding here is how (relatively) small the regression coefficient of information technology investment (iti) is compared to that of user satisfaction (us). this confirms our earlier assertion that in order to maximise the potential of the it capabilities of an organisation, simply raising the level of it investment, essential it may be, is not enough. senior manager must do what they can to raise the satisfaction level of the users to ensure better utilisation of the it capabilities. this highlights the importance of continuing post installation support of it users to improve user satisfaction. summary and discussion the perception of employees on the organisational impacts of investment in information technology by the organisation is crucial to the success or failure of such an investment. in this paper, the demographic variations in employees' perception in five organisation-user variables, viz. investment in it, level of it usage, user satisfaction with it, employee performance, and hotel performance, and the interrelationships among them were examined. in the two cases where objective data were available, employee perception measures were found to be reliable indicators of the actual level of it investment and hotel performance. it is therefore assumed that perceptual measures may be used to provide a reasonable representation of the actual measures in these two areas. age was found to account for significant variations in the perceived level of organisational it investment, user satisfaction with it facilities, and perceived employee performance level. the older the employee the more likely is their perception to be more positive. the only exception is the case of it investment, where the youngest group also have a significantly more positive perception. the fact that there were significant differences in perception among the different age groups highlights an important point. even though the amount of it investment in an organisation can be objectively measured, the perceptions of the different individuals may still differ. it is important for senior managers not to neglect explicitly informing the employees of investment decisions so that they may be more aware of what actually took place in their organisation. the age differences in the perceived level of user satisfaction with the it facilities could be due to the "contrast effect", because older employees are likely to compare the relative ease of doing their tasks in the current computerised environment with the early days where all tasks had to done manually. thus they are more positively oriented toward the organisation's information systems (which may also be more user friendly compared to the early days). the age differences in the perception of employee performance, though much smaller than the previous two, are a bit more difficult to explain. nevertheless the results in the current study seems to suggest a "positive" age effect in these perception measures. the different perception of the different income groups presented a more intriguing situation, as the positive perception measures did not follow a "monotonically increasing" trend at all income levels, particularly at the higher income end. however, both the it investment level and the hotel performance level, were perceived to be the lowest by the lowest income group. this may have to do with the fact that employees at the lowest income group are likely to have the least affinity with the organisation and hence perceived these two variables in the least favourable light. the effect of the employee's position in the organisation provides a rather interesting trend. those employees in positions that expose them to more computer usage or that had the history of longest exposure to computers, tend to have a higher level of user satisfaction than employees from other areas. thus accounting department employees are the most satisfied followed by the information systems (is) department employees. it is interesting to observe that the is employees did not have the highest level of user satisfaction. it is not clear whether this is due to the possibility that in many hotels, the account office is better equipped than the is department, or whether this is due to the higher expectation of the is staff. further research is needed to determine the real reason. however, employees in the human resources departments, the "front" offices, and the marketing departments in these 49 ajis vol. 7, no 2 may 2000 hotels have a much lower user satisfaction level, with the marketing employees have the lowest of them all. this highlights the need to expand the use of it facilities to these applications areas. the findings here support the generally accepted concept that the longer a person's experience with using the computer will reduce computer anxiety and improve one's attitude to computer usage which ultimately leads to better user performance (igbaria & tan 1997). finally, it was observed that the perception of hotel performance decreases as the education level of the employee increases. this is probably due to the higher expectation of the employees as their educational level increases. presumably they are taught to think more critically and independently. results of correlation and regression analyses in the present study clearly demonstrated the existence of close relationships among the five research variables: iti, itu, us, ep and hp. in particular the level of employee performance (ep) is strongly affected by user satisfaction and it usage, both combined together accounts for close to 50% of the variances. this clearly highlights that the attitude of employees (us) and the hands-on factor (itu) are most critical to the employee performance, more so than the original it investment itself. both employee performance (ep) and it investment (iti) were found to contribute significantly to the level of hotel performance (hp), accounting for nearly 30% of the variance in the latter. however, the effect of employee performance (ep) was again found to have a greater effect than it investment. similar results were obtained when it usage (itu) was regressed onto ft investment (iti) and user satisfaction (us). again the us was found to have a stronger influence on itu than iti. these results confirm the importance of the attitudinal factors such as user satisfaction when one examines the impact of it investment in organisations. the findings thus far suggest a rather complex relationship among the five research variables, requiring further investigation. this shall be the subject of a separate path analysis and/or structured equation modeling study to be reported in a forthcoming paper. however, based on results obtained thus far, it is possible to conclude that, when managers consider the decision to invest in information technology in an organisation, it is important to cultivate and nurture a sense of "user satisfaction" among the employees with the it systems. otherwise it would be difficult to fully leverage the potential of the it capabilities as a result of the investment. mere putting more money in it equipment or software, dose not necessarily lead to increased usage of it systems, let alone ultimate employee or hotel performance. the need to attend to the human side of information systems can never be over emphasised. references banker, r.d., kauffman, r.j. and morey, r.c. (1990) "measuring gains in operational efficiency from information technology: a study of positran deployment at hardee's inc.", journal of mis, vol 6, no 2, pp 59-81. bender, d. (1986) "financial impact of information processing", journal of mis, vol 3, no 2, pp 232-248. byrd, t.a. and marshall, t.e. (1997) "relating information technology investment to organisational performance: a causal model analysis", omega, vol 25, no 1, pp 43-56. cron, w. and sobol, m. (1983) "the relationship between computerization and performance: a strategy for maximizing economic benefits of computerization", information and management, vol 6, pp 171-181. cronbach, w. and meehl, p.e. (1981) construct validity in psychological testing. in mason, r.o. and swanson, b.e. (editors.), measurement for management decision, reading, m.a: addison wesley, pp. 335-359. delone, w.h. and mclean, e.r. (1992) "information systems success: the quest for the dependent variable", information systems research, vol 3, no 1, pp 60-95. doll, w. j. and torzadech, g. (1988) 'the measurement of end-user computing satisfaction", mis quarterly, vol 12, no 2, pp 259-274. fitzgerald, g. (1998) "evaluation information systems projects: a multidimensional approach", journal of information technology, vol 13, no 1, pp 15-27. floyd, s.w. and wooldridge, b. (1990) "path analysis of the relationship between competitive strategy, information technology and financial performance", journal of management information systems, vol 13, no 1, pp 1527. hair, j.r. anderson, r.e., tathan, r.l. and black, w.c. (1995) multivariate data analysis, englewood cliffs, nj: prentice-hall. harris, s.e. and katz, j.l. (1991) "organizational performance and information technology investment intensity in the insurance industry", organization science, vol 2, no 3, pp 263-295. hitt, l.m. and brynjolfsson, e. (1996) "productivity, business profitability and consumer surplus: three different measures of information technology value", mis quarterly, vol 20, no 2, pp 121-142. igbaria, m. and tan, m. (1997) "the consequences of information technology acceptance on subsequent individual performance", information and management, vol 32, no 3, pp 113-121. jeffrey, d. and hubbard, n.j. (1994) "a model of hotel occupancy performance for monitoring and marketing in the hotel industry", international journal of hospitality management, vol 13, no 1, pp 57-71. kauffman, r.j. and weill, p. (1989) "an evaluative framework for research on the performance effects of information technology investment", proceedings of tenth international conference on information systems, boston, usa, pp. 198-218. kerlinger, f.n. (1986) foundations of behavioural research, new yourk: holt, rhinehart and winston. 50 ajis vol. 7, no 2 may 2000 lubbe, s., parker, g. and hoard, a. (1995) "the profit impact of it investment", journal of information technology, vol 10, no 1, pp 44-51. mahmood, m.a. (1994) "evaluating organizational efficiency resulting from information technology investment: an application of data envelopment analysis", journal of information systems, vol 4, no 2, pp 95-115. mahmood, m.a. (1997) "how information technology resources affect organizational performance and productivity", information resources management journal, vol 10, no 1, pp 4-5. mardani, n.l.k. (1997) bali dalam angka semester i tahun 7997, department of bali tourism, post and telecommunication, denpasa, bali, indonesia. mata, f.j., fuerst, w.l. and barney, j.b. (1995) "information technology and sustained competitive advantage: a resource-based analysis", mis quarterly, vol 19, no 4, pp 487-505. mukhopadhyay, t., kekre, s. and kalathur, s. (1995) "business value of information technology: a study of electronic data interchanges", mis quarterly, vol 19, no 2, pp 137-156. nunnally, j.c. (1978) psychometric theory, new york: mcgraw-hill. orlikowski,, w.j. and baroudi, j.j. (1991) "studying information technology in organizations: research approaches and assumptions", information systems research. vol 2, no 1, pp 1-28. parthasarthy, r and sethi, s.p. (1993) "relating strategy and structure to flexible automation: a test of fit and performance implications", strategic management journal, vol 14, no 4, pp 529 -549. powell, t.c. and dent-micallef, a. (1997) "information technology as competitive advantage: the role of human, business, and technology resources". strategic management journal, vol 18, no 5, pp 375-405. rai, a., patnayakuni, r. and patnayakuni, n. (1997) "technology investment and business performance", communications of the acm, vol 40, no 7, pp 89-97. raymond, l., pare, g. and bergeron, f. (1995) "matching information technology and organisational structure: an empirical study with implications for performance", european journal of information systems, vol 4, no 1, pp3-16. roach, s.s. (1988) "technology and the service sector: the hidden competitive challenge", technology forecasting and social change, vol 34, no 4, pp 387-403. santos, d.w.l., peffers, k. and mauer, d.c. (1993) "the impact of information technology investment announcements on the market value of the firm", information systems research, vol 4, no 1, pp 1-23. sohal, a.s. and ng, l. (1998) "the role and impact of information technology in australian business", journal of information technology, vol 13, no 3, pp 201-217 sri darma, g. and lo, b.w.n. (1998) information technology and organisational performance: a survey of the hospitality inudstry. working paper somit 98/1, school of multimedia and information technology, southern cross university, lismore, nsw, australia. sriram, v., stump, r.l. and banerhee, s. (1997) "information technology investments in purchasing: an empirical study of dimensions and antecedents", information and management, vol 33, no 2, pp 59-72. strassmann, g. (1985) information payoff, new york: free press. straub, d., limayem, m., and karahanna-evaristo, e. (1995) "measuring system usage implications for is theory testing", management science, vol 41, no 8, pp 1328-1342. turner j.a. (1985) organisational performance, size and the use of data processing resources. working paper #58, center for research in information systems, new york university, new york. weill, p. (1992a) managing the it investment pyramid for competitive advantage, graduate school of management, the university of melbourne, australia. weill, p. (1992b) "the relationship between investment in information technology and firm performance: a study of the valve manufacturing industry", information systems research, vol 3, no 4, pp 307-333. willcocks, l. (1994) information management: the evaluation of information systems investments. london : chapman & hall. willcocks, l. (1996) investing in information systems, london: chapman & hall. 51 ajis vol. 6 no. 1 september 1998 speech act based communication and information modelling with demo victor e. van reijswoud hans b.f. mulder delft university of technology essential action engineers b.v. dept. of information systems p.o. box 58 p.o. box 356, 2600 aj delft 2280 ab rijswijk the netherlands vreijsw@is.twi.tudelft.nl the netherlands hans_mulder@inventive.nl abstract the field of speech act based modelling of information systems has attracted an increasing attention in recent years. on the basis of the research work of winograd and flores, several modelling approaches have been proposed. in this paper we discuss the speech act based modelling approaches in general and the demo approach in particular. next to the discussion of theoretical foundations of this modelling approach we also illustrate the application of demo to a case study, and we report on the status of the resulting models for information systems design and business process optimisation. keywords speech act based modelling, business analysis, information systems design, business process redesign/reengineering introduction theories for modelling and designing information systems have evolved through different stages and on different foundations. hirschheim, et al. (1995) describe seven generations that evolve from the developer's private methodological approach in the mid-1950's, through life-cycle, structured, evolutionary and participative approaches, to the emergence of emancipatory approaches to information systems development. this latest generation is very much in the making and gets considerable attention in both the practitioner and academic literature. although these alternative approaches are a controversial topic in the field, they also offer new perspectives on information systems development (livari et al., 1996). approaches for information systems analysis and design that receive an increasing attention in the past years are the approaches that are founded in the language/action perspective (see dignum et al., 1996; dignum, dietz, 1997). the language/action perspective (for a large part based on searle's speech act theory) which was introduced in the field of information systems by flores and ludlow in the early 1980's, can be considered to be a new basic paradigm in the information systems arena (livari et al., 1996). in contrast to traditional views of "data flow", the language/action perspective emphasises what people do while communicating; how they create a common reality by means of language and how communication brings about a co-ordination of their activities. in this article we will discuss a full grown modelling approach in the language/action perspective called demo® (dynamic essential modelling of organisations). demo provides a theory for the understanding of the working of an organisation and an explanation of its business activities. demo also provides a modelling approach based on this understanding. after a brief introduction to the speech act theory of which the basic principles of the language/action perspective are derived, and to some approaches that are founded on this theory, the fundamentals of the demo approach are discussed in section 3. in section 4 we illustrate, using a case study, how the modelling facility of demo is applied. in section 5 two application domains (business process optimisation and information systems design) are discussed. finally, some general conclusions are drawn in section 6. speech act based modelling approaches the focus on communication as the key concept for the understanding and modelling of organisations requires a theory explaining language and its functions. the speech act theory (austin 1962; searle 1969; searle 1979; searle, vanderveken 1985) has proven to be a strong frame of reference for this purpose (e.g., flores, ludlow 1981; winograd, flores 1986; taylor, cameron 1987; winograd 1988; taylor 1993). the main characteristic of the speech act theory is that it considers the use of language to be a form of rulegoverned behaviour. uttering a sentence is the performance of an act, a so-called speech act. the most important type of speech act in an organisational context is the illocutionary act. falling through the speech act theory's prism, the concept of saying something spreads out a spectrum of illocutionary acts and illocutionary forces. these forces specify how an utterance in question is intended to be taken that is, what natural effect (cognitive, motive, social or legal) it is intended to have, and, accordingly, in what dimensions (truth, ajis vol. 6 no. 1 september 1998 feasibility, propriety, and so on) it is supposed to be assessed. for instance, an utterance has the "force" of a statement, a warning, a promise, an order, and so forth (vendler 1967). the general structure of the illocutionary act is f(p): a proposition p with an illocutionary force f. on the basis of their force, illocutionary acts have been grouped into five families: assertives commit the speaker to something's being the case (e.g., stating); directives try to get the hearer to do something expressed in the proposition p (e.g., questioning and commanding); commissives commit the speaker to some future course of action (e.g., promising); declaratives bring about a (new) state of affairs by merely declaring it (e.g., declaring); expressives express the speaker's attitudes/feelings about a state of affairs (e.g., apologising). the speech act theory has been the foundation for a number of theories and modelling approaches in the area of information systems. these approaches consider the utterance of speech acts to be the backbone of business processes and consequently their modelling effort focuses on speech acts. the initial impetus to a speech based conceptualisation of organisation has been the work by flores and ludlow (1981). the authors propose to perceive organisations as networks of inter-related commitments created by directives, commissives, assertives and declaratives (expressives were not included in their analysis). the 'commitment analysis' of flores and ludlow was extended in great detail by winograd and flores (1986). according to this approach, the conversation for action (cfa) is the central co-ordinating structure for human organisation. the cfa is conceptualised as an interplay of requests and commissives directed to explicit co-operative action. next to the cfa, the conversations for clarification, possibilities, and orientation are identified. for more details on the different types of conversations the reader is referred to (winograd, 1988). the conversation for action theory has been developed into a full-grown speech act based modelling approach within the business design language by action technologies (medina-mora et al., 1992, schal, zeller, 1993; schal, 1996). the basic modelling unit is the four step action workflow protocol. the basic unit can be connected with loops in articulated processes (see figure 1). together these building blocks are used to model an organisation as a network of inter-related speech acts grouped together in workflow loops. clarification/ negotiation \/ j 1 client asks for auction (opening ph 2 supplier agrees to do it (commitment phase) basic workflow4clie accepts report and declares satisfaction (closing phase) 3 supplier fulfills the work and reports it done hase) .k., thank you" delegation/ confirmation figure 1 the interconnection of action workflow loops in articulated processes another important speech act based approach is sampo (lehtinen, lyytinen, 1986; auramaki et al., 1988; auramaki et al., 1991; auramaki, lyytinen, 1996). sampo studies organisational activities as a series of speech acts creating, maintaining, modifying, reporting, and terminating commitments. it offers a method for modelling organisations as systems of communicative action. the approach uses concepts of the speech act theory and discourse analysis for the information analysis of organisations. the modelling approach entails two domains: the entity domain (ed) and the action domain (ad). the ed comprises of static entities that persist over longer periods of time. the ad includes dynamic entities called acts. 90 ajis vol. 6 no. 1 september 1998 two types of acts are studied: instrumental acts (iacts) and speech acts (sacts). iacts accomplish changes in the ed. these changes are modelled. sacts are defined as symbolic deeds that result in linguistic expressions having a "meaning". sacts form larger wholes called conversations or discourses. the systematic regularities of these conversations are analysed. sampo introduces an analysis and modelling method for discourses in organisations. after an analysis of the quality of the discourses, in which coherency, completeness, and ambiguity is defined, two main graphical models are used. the first model is the discourse graph. this model describes the overall structure of the discourse, and it delineates discourse objects and their properties and relationships. it can be seen to represent a partial "script" or "schema" for communications. the graph defines necessary and sufficient conversation possibilities for each discourse participant. the conversation graph is introduced to characterise the dynamic discourse features. it describes the stages and moves in a conversation as well as the conditions (in sampo called predicates) that restrict and control the performance of acts. both graphs describe institutionalised networks of acts and help to detect: 1. principles needed in the set-up and control of commitments 2. inconsistencies in the co-ordination of commitments 3. possibilities for organisational development that simplify communication and control mechanisms in the next section we introduce the demo approach. this approach builds on the same theoretical foundations as the cfa/business design language approach and sampo, but is extended with a complete and integrated communication, information and behaviour modelling facility that provides both the basis for information systems design as well as business process optimisation. modelling with demo demo (dynamic essential modelling of organisations) is a cross-disciplinary theory describing and explaining the communicational dynamics of organisations, as well as an analysis method based on this theory. a relevant set of fragments describing demo is constituted by (dietz, 1994a; 1994b; 1996a; 1996b; dietz, mulder, 1996; reijswoud, 1996; reijswoud, rijst, 1995). in demo, the functioning of organisations is viewed from three levels: the documental, the informational and the essential level. at the documental level, an organisation is viewed as a system of actors that produce, store, transport and destroy documents. in other words, at the documental level the substance and form by which co-ordination becomes visible is considered. at the informational level one abstracts from this substance and form (i.e. documents) and focuses on the actual meaning. the organisation is observed as a system of actors that send and receive information, and perform calculations on this information in order to create derived information. at the essential level an organisation is conceptualised as a system of actors that are engaged in the executions of business transactions. at the essential level organisations are considered as networks of business transactions, which are composed of interrelated communicative acts. the essential transaction is a core concept in demo. a transaction is a pattern of activity that is performed by two actors: the initiator and the executor. it is important to note that actors are roles in an organisation and not persons. a transaction is composed of three phases: the order phase in which two actors come to an agreement about the execution of some future action; the execution phase, in which the negotiated action is executed; and the result phase in which the actors negotiate an agreement about the result as brought about in the execution phase. the successful execution of a transaction in the subject world (the world of communication) results in a change in the object world (the world of facts) in which the actors exist. the basic pattern of a transaction is displayed in figure 2. 91 ajis vol. 6 no. 1 september 1998 sw 0-phase /«? uest e remise e(ju fcf 1 i j r i *l 1 j *l ' j requested promisee ow ;. . .', , ', , ;, . e-phase . « <» , o r-phase e;s/afe ^ ', 4 stated teccept,— accepted fact n /̂ time figure 2 the basic pattern of the demo transaction the execution of a transaction can be described and consequently modelled at all three levels of abstraction. at the essential level the transaction is described as a pattern of performative communication. at the informational level the execution of a transaction is described as the exchange of information (information flows), and at the documental level the materialisation of the transaction in tangible objects (documents, files etc.) is described. the demo approach hypothesises that the transaction at the essential level allows multiple realisations at the informational level and the documental level. it is important to realise that these realisations are ideally deliberate organisational choices. the principle idea is displayed in figure 3. essential transaction design of the organisation allows informational transaction design of the organisation allows documental transaction design of the organisation figure 3 transaction design and levels of abstraction the modelling facility of demo provides an understanding and graphical representation of the transactional structure of organisations. this transactional structure is modelled in five partial models: the action model, the interaction model, the process model, the facts model and the interstriction model. the models are developed incrementally. the interaction model contains a description of the transaction types and the actors in an organisation. the actors are displayed as transaction initiating or transaction executing actors. the graphical notation used for the interaction model is the communication diagram. the process model describes to aspects of the transactional structure. in the first place to display the causal and conditional relationships between the transaction types, and in the second place to display the course of individual transaction processes. the relationships between the transaction types is expressed in the process diagram, and the course of individual transaction processes in the transaction diagram. the facts model is the complete and precise specification of the state space of the object world. the facts diagram is used to represent the facts model. the interstriction model is a specification of the actors and the information that is needed by these actors to execute a transaction type. the interstriction model is also expressed in 92 ajis vol. 6 no. 1 september 1998 the communication diagram. finally, demo includes the action model of an organisation. the action model is called the 'mother of the models' because it comprises the most detailed specification of the transaction structure of an organisation. it allows a specification of transactions at the essential, informational and documental level. the action model is expressed in the action diagram. in the next section we present the analysis of a case study in which demo is applied. the different models and the corresponding diagrams are illustrated in this section in more detail. the case study: sgc the conciliation board for consumers (in dutch abbreviated to sgc) is a non-profit organisation with the aim to negotiate quick, less costly and easy solutions for disagreement between customers and suppliers. for suppliers the participation in a conciliation board for consumers is an important pan of the quality management. an optimal mediation of the board in consumer-supplier disputes requires an optimal availability of internal and external information. the primary question of sgc was to reengineer the internal and external information handling processes of the organisation. automation of these processes in a workflow management system was considered as one of the possibilities. the first step that was taken was to initiate an analysis of the current way of working. the demo approach was used to describe the current situation of sgc. before the demo analysis of sgc is presented, a brief description of the current way of working is presented. a description of the current way of working a request for mediation of the conciliation board for consumers needs to be started with a letter in which the nature (concerning housing, computers, travels etc.) and the magnitude of the complaint is explained. there are about 12000 letters received every year. on the arrival of a letter, a file is opened. the file is identified by the committee the complaint relates to, a unique identification number of the complaint, the name of the complainer and the date that the complaint was submitted. in the course of the procedure the file is used to archive additional information. on the basis of the first letter a first selection is made as to whether the complaint is taken into consideration. when the complaint is taken into consideration (which holds for about 70% of the complaints), the complainer is requested to fill out a questionnaire, to pay complaint fee (based on the amount of the invoice of the complaint), and to deposit the remaining amount of the invoice. at the same time the supplier is requested for a bank guarantee. when the questionnaire is returned and the money is transferred by the complainer, and the bank guarantee is provided by the supplier, the procedure continues. when the complainer fails to meet (one of) these conditions within one month, the request to meditate in the conflict is turned down. in some circumstances exceptions on this rule are made, but the complainer has to submit a request for dispensation with reasons within this one month period. next to exclusion on the basis of failing to meet the requirements of the standard procedure of submitting complaints there are some other reasons for turning down a request for mediation by sgc. a complaint is turned down when the article the complaint relates to is used professionally, involves physical injury, or the supplier has suspension of payment or has already gone bankrupt, or when the supplier is not a member of a branch organisation. when the request for mediation by sgc is granted, the supplier is informed by mail and the execution of the procedure is filed in the, so-called, complaint book. this complaint book was set up for the purpose of progress monitoring. at the same time the supplier is supplied with the documents of the file and is requested for his defence or may propose an agreement. in addition to the complaint of the customer and the defence of the supplier, the board can initiate an expert examination. all the documents of the customer, supplier and possibly the experts form the input for a meeting of a special committee of the board for which all the parties involved are invited. in this meeting the committee reaches a decision. about one month after the meeting the parties involved are informed by mail about the judgement of the committee. after the judgement of the committee, the financial matters between the customer and the supplier are settled. this winding up comprises the complaint fee, the deposit of the remaining amount of the invoice, and the expenses of the members of the committee and the experts. if the supplier fails to comply with his terms of payment, the consumer can appeal to a regulation that assures payment. at the same time the branch organisation of the supplier is informed. then, the file is closed. 93 ajis vol. 6 no. 1 september 1998 the demo analysis ofsgc the first step in the demo analysis of the conciliation board for consumers is the description of the activities at the essential level. this means that the business is described as a network of inter-related business transactions. these transactions are formulated at type level and are displayed in a transaction table. in the transaction table the resulting object world facts of the successful execution of the transactions are also included. the resulting facts are represented in a high level manner, more details of these facts are revealed in the facts model. (nb. time variables like the date the complaint is submitted or the date that the committee passes judgement have been left out for reasons of clarity.) transaction type transaction result tl t2 t3 t4 t5 admitting_complaint mediating_complaint paying_consumer_fee depositing_invoice_ amount depositing_bank_ fl f2 f3 f4 f5 the complaint <co> is declared to be admissible the complaint <co> is mediated the consumer complaint fee concerning complaint <co> is paid the remaining invoice amount of complaint <co> is deposited the bank-guarantee concerning complaint <co> is deposited guarantee t6 dealing_with_complaint f6 the complaint <co> is dealt with t7 defending_complaint f7 the complaint <co> is defended t8 obtaining_expert_advice f8 the advice from expert <ex> concerning complaint <co> is obtained t9 passingjudgement f9 the judgement concerning complaint <co>is passed t10 paying_supplier_fee f10 the supplier complaint fee concerning complaint <co> is paid table 1 the transaction table of the current way of working of sgc the transaction structure is displayed by means of a communication diagram as presented in figure 4 below. the actors are represented by numbered a-squares. the grey s-squares represent complex (external) actors of which we do not know (or do not want to know yet) the composition. the transactions are depicted by the t-numbered circlein-the-diamond. the arrow connecting the transaction symbol to the actor points at the executor of the transaction, while the plain line connects the initiator to the transaction symbol. in figure 4 we have also included some dotted lines to represent informative links between actors and transactions types. the informative links are part of the interstriction model. these links represent an actor's need for (some of) the information stored as results of the execution of these transaction types for the execution of its transactions. for example, for the checking of the mediation requirements, and thus the possible initiation of the transactions by which the supplier is requested to provide a defence, the actor a6 needs the resulting facts that are stored in the tl, t2, t3 and t4 relating to a particular complaint co. in the communication diagram there is also a external fact bank added (ef1). this bank contains the rules for the mediation that are laid down in the regulations of sgc. for more syntactical and semantic details on the graphical notation of the communication diagram see (dietz, 1996a; 1996b; reijswoud 1996). 94 ajis vol. 6 no. 1 september 1998 figure 4 communication diagram of sgc the demo communication diagram provides an overview of the organisation as a network of communicative commitments. it does, however, not show the relationships in time and interdependencies between the transaction types. the process model is used to highlight these relationships. below the process diagram is presented (figure 5) in which the relationships between the transaction types of sgc are depicted. transaction types are represented by circles or stretched circles. the point of initiation is represented as a tiny circle. causal relationships are presented as solid arrows while conditional relationships are presented as dotted arrows. optional relationships are indicated with a small horizontal line on the causal relationship arrow. the different stages of a transaction (order, execution and result phase) are represented as a suffix to the identifier of the transaction type. if relationships between the transaction types of sgc are considered, a more detailed understanding of the working of the organisation is achieved. we can see that not all of them need to be executed for a successful mediation. transactions t4 (depositing_invoice_amount), t5 (depositing_bank_guarantee), t8 (obtaining. expert_advice), and t10 (paying_supplier_fee) are optional. for example, if the complainer has already paid the total amount of the invoice, the payment of a deposit is not requested. similarly, when the committee decides that the complaint of the consumer would not lie, the supplier does not have to pay for the mediation. 95 ajis vol. 6 no. 1 september 1998 figure 5 the process diagram of sgc insights in the object world of sgc are obtained with the facts model. this model provides a complete and precise specification of the fact types that are created and/or used as well as their mutual relations. the facts model is represented with the niam-like facts diagram (for details on niam see: nijssen, halpin, 1989; the facts diagram is explained in dietz, 1996; reijswoud, 1996; rijst, reijswoud, 1995). in figure 6, a small part of the facts diagram of sgc is presented. in the example below the facts diagram of the actor al is presented. the circles denote object classes with their class names written above. the objects play different roles in a fact. in the facts diagram these roles are indicated by numbers in rectangles and referred to in the sentences below the roles. in figure 6 the fact types used and/or created by actor 2 mediation of sgc are expressed. central in the diagram is the object class complaint. the facts diagram also shows that there is a description related to a complaint (this is a special object class containing textual description of the complaint), that a complaint is requested for mediation on a particular date and has been mediated on a particular date, that a complaint is mediated by a particular committee, and finally that a complaint is submitted by a complainer and relates to a particular supplier. the uniqueness constraints depicted as bold line above roles describe that every complaint is unique (identified with an unique identification number in the daily practice of sgc) and thus may not appear more than once in the extension. 96 ajis vol. 6 no. 1 september 1998 description 1 2 committee 1 2 <1> was mediated on <2> supplier 1 2 <1> was requested on <2> consumer 1 <2> relates to <1> <1> was requested by <2> figure 6 the facts diagram of actor al a complete understanding of the transaction structure of an organisation is achieved in the action model, the 'mother' model of demo. in the action model not only the transactional structure of an organisation at the essential level is represented, the essential transactions are also extended with their realisations at the informational and documental levels. in figure 7 the procedure regarding the execution phase of 't2 mediating_complaint' is depicted (t2/e in figure 5). the arrows represent precedence relationships. the execution of the procedure consists of the following of the route from the start of the procedure at the top to the end of the procedure. the first white box on the left hand side shows the communicative act by which sgc requests the complainer to pay the complaint fee. the waiting condition below indicates that the complaint fee has to be paid before the procedure can continue. the two elements next to the initiation of t3 are optional. t4 and t5 are only initiated when the question in the triangular shapes are answered with 'yes', otherwise the procedure continues to the synchronisation. after the synchronisation the execution procedure of transaction t2 proceeds to initiation of transaction t6. on reaching this point the procedure will halt until the transaction t7, t9 and the optional t8 and t10 as part of t6 are completed. the execution of the transactions are described in the action diagram of actor a6 handle-complaint. when t6 is carried through, the actual objective action of t2, the mediation of the complaint is executed. the result of the execution of this objective action is realised in the notification (statement) of the outcome of the mediation of actor a2 of sgc to the consumer. normally, this becomes tangible in a notification letter of sgc to the consumer. 97 ajis vol. 6 no. 1 september 1998 begin procedure execute t2: mediating_compaint request t3 the consumer complaint fee concerning complaint <co> is paid to consumer tnloraiotlonil: calculation rutat complaint faa pocumantal: aand brochurt »0»807 to coniumti the consumer complaint fee .concerning complaint <co> is paid informational: chaek bank account sgc documantal: r*e«lv« balance tram bank no yes request t4 the remaining invoice amount of complaint <co> is deposited to consumer nfermatlonal: calculate total outstanding in voice docurnantafraend broehur* *09607 toconaumar the remaining invoice amount of complaint <co> is deposited .informational: ehaekbankaccounfsgc *, documental: receive balanca from bank request t5 the bank guarantee amount of complaint <co> is deposited to supplier informational: atata coat to raeolve complaint doeumantal: send peraonanaod letter moot the bank guarantee amount of complaint <co> is deposited informational: check bank account sgc documental: receive balance from bank request t6 complaint <co> is dealt with to handle complaint informational: cheek reciiu&tt3ft4 and ts and to ', lhandle complaint documental: compiata form *9603 complaint <co> is dealt with |tlnform atlona i: fudgamanf;from coraniltta* : ̂ dbcumantal: racatva'copy of ludganiant do mediate complaint <co> state t2 complaint <co> is mediated to consumer .informational;judgaoafltfrom contmlttea docum.ntali «.nd (ugosmant to conmlm.r end procedure execute t2: mediaung_connpaint figure 7 action diagram of the execution phase of transaction t2 on the basis of the presented demo models in which the transactional structure of the organisation is described, a follow up project can be started. two possible directions for follow up are presented in the next section. application areas the speech act based business process and information modelling approach demo is applied in two main areas: 1. business process optimisation, 2. information system design. each of the two application areas is considered in more detail in this section. business process optimisation the introduction of the term bpr has led to several approaches for business process optimisation under different names (davenport, 1993; hammer, champy, 1993; keen, 1991; scott-morton, 1991). bpr lacks a commonly accepted definition but the underlying claim that organisational change is necessary to maintain flexibility and 98 ajis vol. 6 no. 1 september 1998 competitiveness of a business is clear (davenport, 1995; teng et al., 1992). by means of the introduction of the three levels of abstraction, the demo approach is able to provide a differentiated and well-founded definition of bpr and by means of the models a structured working approach for bpr is provided. in the demo approach bpr is viewed as activities related to the three levels of abstraction. this means that organisational change can focus on the documental level, i.e. the production, storing transporting and destruction of documents performed by the organisational system of actors. change can also focus on the informational level. this means that the system of actors that sends, receives, calculates and derives information in an organisation is changed. it is, however, important to realise that changes at the informational level necessarily cause changes at the documental level. in line of this reasoning, organisational changes can also focus at the essential level. these changes focus at the system of actors that are engaged in the execution of the business transactions. these changes are the most drastic for the organisation since they have also effects on the informational and documental structure of the organisation. at the same time it is important to note that only changes of transactional structure of the organisation at the essential level can cause drastic increases in efficiency and effectiveness. the demo models provide a structured approach to more or less radical business process optimisation. business process optimisation with demo primarily focuses on changes to the essential transaction design of the organisation. on the basis of the interaction model decisions can be made about the adding or deleting of transaction types. these changes have a major impact on the organisation. the business process model can be used to reduce the cycle time of the entire business process. cycle times can, for example, be reduced by parallelisation of transaction types or changes to the conditional and optional relationships of the transaction types or transaction phases. changes to individual transaction processes can be made to impose incremental organisational changes. when the efficiency and effectiveness of the case study is considered several optimisation possibilities can be considered. on the basis of the communication diagram it was considered to delete transaction tl admitting_complaint. the questionnaire could be the medium to be used by the consumer to start a mediation in t2. another proposal concerned the cycle time and was based on the process diagram. it was considered to reduce the cycle time by moving the conditional relations between t3, t4, t5 and t6/o (see figure 5) to t9, meaning that the committee will only gather to pass a judgement (t9) when the complaint fee and the deposit has been paid by the consumer, and when the bank guarantee is given by the supplier. the introduction of standardised (electronic)forms for the starting of a mediation was proposal for the optimisation of an individual transaction process. information system design modelling for the design of information systems is a well developed area (e.g., lundeberg et al., 1982; checkland, scholes, 1990; yourdon, 1989; 1990). however, designing on the basis of speech act models is hardly explored (hirschheim et al., 1995). most endeavours in this area have aimed at the understanding and optimisation of business processes (e.g., denning, medina-mora, 1995; dietz, 1994a; 1994b). since the demo approach both integrates models of the subject world and the object world, it is well suited for the design of information systems. when using the demo approach for the design of information systems, the aim is to develop a blueprint for an information system. this blueprint contains a description of the essential (core) information that is necessary for conducting the business, as well as the location of the creation of this information. the models also specify the external information, that is the information that is created outside the system that is under consideration. the core information of the business is described by the facts model. the model provides a complete and precise specification of this information. therewith it contains the basic specification of the data model of an organisation. the interstriction model describes the location of the information and the actors that make use of this essential business information. it specifies the information support the individual roles in the organisation need. in other words, it provides the blueprint of the information architecture of an organisation. these blueprints can be used to determine the fundamentals a new information system, but more importantly, they can be used to evaluate the value of existing information systems. by determining a complete and precise understanding of the essential information in an organisation, criteria for evaluating the design of existing systems become available. when the demo analysis of the case study is taken as a starting point, the blueprint of an information system can be determined. based on the facts diagrams of the two actors (e.g., figure 6) the basic information needs are determined. some of this information is created by the actor as a result of the successful execution of a transaction, and some of the information is obtained from other sources, like the rules for mediation from the regulations of sgc. from the communication diagram (figure 4) we can conclude that the facts that are created as result of the execution of transactions t1-t4 are important for the actor a6. for the information system architecture this implies that this 99 ajis vol. 6 no. 1 september 1998 information needs to be made available to the persons in the organisation that play the role "checking-mediationrequirements". the goal of a demo analysis for the purpose of information systems design is to specify the conceptual scheme of the information system and the information architecture. therewith it forms the starting point for informational (e.g., derived and aggregated information like management information) and documental (e.g., user-interface) filling-in resulting in an actual system. conclusions speech act based modelling approaches provide an understanding of organisations that extends on information and document oriented modelling approaches. by focusing on the communication a richer understanding is obtained of the dynamics of an organisation, and therewith creates richer models. the demo approach supports modelling of an organisation from different perspectives. the distinction between the essential level on the one hand and the informational and documental levels on the other presents a new perspective on both business process optimisation and engineering of information systems and infrastructure. it creates a direct relation with the core of the business (the essential level) and the informational and documental realisations of the core. the different models also allow different inter-related perspectives at the essential level. the interaction and the interstriction model describe an organisation as a structure of customers and suppliers, products / services, while the process model describes the organisation as a structure of activities. the facts model is concerned with the information producing and storing characteristics of an organisation. finally, the action model highlights the organisation as a decision-making system. together these models of demo provide a more detailed and richer description of an organisation than other speech act based modelling approaches do. the demo modelling approach has been applied in some large scale projects and several smaller ones. examples of large scale projects are the dutch telecom, and the rotterdam police force. smaller projects are projects like sgc, but also some small (production) companies.9 the purpose of the projects is diverse, however most of them concerned the optimisation of the business processes and the evaluation of the current information systems. the sgc project concerns the modelling for the implementation of a workflow management system, which is, at the time of writing, being implemented. references auramaki e, e. lehtinen, k. lyytinen, 1988. a speech act based office modeling approach. acm transactions on office information systems, vol. 6, no 2, pp. 126-152. auramaki, e., k. lyytinen, 1996. on the success of speech acts and negotiating commitments. in: f. dignum, j. dietz, e. verharen, h. weigand (eds.), communication modeling the language/action perspective, electronic workshops in computing springer. http://www.springer.co.uk/ewic/workshops/cm96/ auramaki, e., r. hirschheim, k. lyytinen, 1991. modeling offices through discourse analysis: the sampo approach. the computer journal, vol. 35, no. 4, pp. 342-352. austin, j.l, 1962. how to do things with words. clarendon press, oxford. checkland, p. j. scholes, 1990. soft systems methodology in action. john wiley, chichester. davenport, t.h., 1993. process innovation. harvard business school press, boston. davenport, t.h., 1995. business process reengineering: where it's been, where it's going. in: v. grover, w.j. kettinger (eds.). business process change: reengineering concept, methods and technologies. idea group publishing, harrisburg, pp. 1-13. denning, p.j., r. medina-mora, 1995. completing the loops. interfaces, vol. 25, no. 3, pp. 42-57. dietz, j.l.g., 1994a. business modeling for business redesign. proceedings of the 27th hawaii international conference on system sciences, ieee computer society press, los alamitos, pp. 723-732. dietz, j.l.g., 1994b. modelling business processes for the purpose of redesign. in: b.c. glasson, i.t. hawryszkiewycs, b.a. underwood, r.a. weber (eds.), proceedings of the ifip tc8 open conference on 9 most of the reports on these cases are in the dutch language and in most cases confidential, but the authors can be contacted for english summaries. 100 ajis vol. 6 no. 1 september 1998 business process re-engineering: information systems opportunities and challenges. elsevier, amsterdam, pp. 249-258. dietz, j.l.g. 1996a. the what and the why of modelling business processes. in: r.m. van es, a. post (eds.), dynamic enterprise modeling. kluwer bedrijfsinformatie, deventer. dietz, j.l.g., 1996b. introductie tot demo: van informatietechnologie naar organisatietechnologie. samson, alphen a/d rijn. dietz j.l.g., h.b.f. mulder (1996). realising strategic reengineering objectives with demo. in: proceedings of the international symposium on business process modelling, springer-verlag. dignum, f., j. dietz (1997). communication modeling • the language/action perspective, proceedings of the first international workshop on communication modeling, computer science reports, eindhoven university of technology, http://www.win.tue.nl/win/cs dignum, f., j. dietz, e. verharen, h. weigand (eds.), (1996). communication modeling the language/action perspective, proceedings of the first international workshop on communication modeling, electronic workshops in computing springer. http://www.springer.co.uk/ewic/workshops/cm96/ flores, f., j.j. ludlow, 1981. doing and speaking in the office. in: g. pick, h. spraque jr. (eds.). decision support systems: issues and challenges, pergamon press, new york, pp. 95-118. hammer, m., j.a. champy, 1993. reengineering the corporation: a manifesto for business revolution, nicholas brealy, london. hirschheim, r., h.k. klein, k. lyytinen, 1995. information systems development and data modelling: conceptual and philosophical foundations. cambridge university press, cambridge, livari, j., r. hirschheim, h.k. klein, 1996. five emerging approaching to information systems development: an analysis of paradigmatic foundations. manuscript. keen, p.g.w., 1991. shaping the future: business design through information technology. harvard business school press, harvard. lehtinen, e., k. lyytinen, 1986. action based model of information system. information systems, vol. 11, no. 4, pp. 299-317. lundeberg, m., g. goldkuhl, a nilson, 1981. information systems development: a systematic approach. prentice hall, englewood cliffs. medina-mora, r., t. winograd, r. flores, f. flores, 1992. the action workflow approach to workflow management technology. in: j. turner, r. kraut (eds.), proceedings of the 4th conference on computer supported cooperative work. acm, new york. nijssen, g.m., t.a. halpin, 1989. conceptual schema and relational database design: a fact oriented approach. prentice hall, sidney. reijswoud, v.e. van, 1996. the structure of business communication: theory, model and application. phd thesis delft university of technology, delft. reijswoud, v.e. van, n.b.j. van der rijst, 1995. modelling business communication as a foundation for business process redesign: a case of production logistics. in: proceedings of the 28th hawaii international conference on systems sciences. ieee computer society press, los alamitos ca, pp. 841-850. rijst, b.j. van der, v.e. van reijswoud, 1995. comparing two speech act based modeling approaches for the purpose of information systems development. in: g. doukidis, r. galliers, t. jelassi, h. krcmar, f. land (eds.), proceedings of the third european conference on information systems, ecis'95, athens, pp. 353-365. schal, t., 1996. workflow management systems for process organisations. lecture notes in computer science 1096, springer, berlin. schal, t., b. zeller, 1993. workflow management systems for financial services. proceedings of the conference on organizational computing systems, coocs'93, acm, new york, pp. 142-153. scott morton m.s., 1991. the corporation of the 1990's: information technology and organizational transformation, sloan school of management, oxford university press, new york. searle, j.r., 1969. speech acts: an essay in the philosophy of language. cambridge university press, cambridge. searle, j.r., 1979. meaning and expression. cambridge university press, cambridge. searle, j.r., d. vanderveken, 1985. foundations of elocutionary logic. cambridge university press, cambridge. taylor, j.r., 1993. rethinking the theory of organizational communication: how to read an organization. ablex, norwood. 101 ajis vol. 6 no. 1 september 1998 taylor, t.j., d. cameron, 1987. analysing conversation: rules and units in the structure of talk. pergamon press, oxford. teng, j.t.c., w.j. kettinger, s. guha, 1992. business process redesign and information architecture: establishing the missing links. in: j.i. degross, j.d. becker, j. j. elam (eds.), proceedings of the international conference on information systems. dallas. yourdon, e., 1989. modern structured analysis. prentice hall, englewood cliffs. yourdon, e., 1993. yourdon systems method: model driven systems development. prentice hall, englewood cliffs. vendler, z., 1967, linguistics in philosophy. cornell university press, ithaca ny. winograd, t, f. flores, 1986. understanding computers and cognition: a new foundation for design. ablex, norwood nj. winograd, t., 1988. a language/action perspective on the design of cooperative work. in: i. greif (ed.), computer supported cooperative work: a book of readings. morgan kaufmann, san mateo. 102 ajis special issue requirements engineering 1999 realizing requirements in product-line development using o-o frameworks garry froehlich, h james hoover, paul sorenson software engineering research laboratory department of computing science university of alberta edmonton, alberta, canada t6g 2h1 (garry, hoover, sorenson)@cs.ualberta.ca abstract an existing, rapidly evolving strategy for realizing requirements in product line development is described. the position taken and the proposed new directions are based on our substantial experience in building and using o-o frameworks important new directions include the use of hooks to encapsulate product-line requirements and the management of requirements using a hook evolution model and experience base construction. it is also proposed that requirements traceability can be supported in an obvious manner by following prescribed processes for product development that are based on enacting hooks. introduction the paper begins with an overview of product-line development, o-o frameworks and the relationship between the two. the importance of requirement capture and management in software product-lines is identified and strategies for determining product-line requirements are reviewed. a form of prescriptive documentation, embodied in the hooks model [froehlich97), is introduced as a mechanism for encapsulating requirements. hooks use script style descriptions to assist the developer in extending a framework into a product using a set of intentional requirements that are anticipated by the o-o framework developers. the issues of how hooks are changed to support evolving requirements is discussed. a proposal for how requirements traceability can be supported using hook enactment is presented. this approach to traceability can be supported through a tool like hookmaster [liu99). although much of this work is still "work-in-progress," it is based on our substantial experience in building and using o-o frameworks in areas such as engineered products [froehlich99a) and personal assistants [liew99) for the web. product lines and o-o frameworks product-line architectures, or product families [parnas79), move beyond the traditional notion of building a single product in isolation, and instead focus on building whole suites of related products within a single domain. the concept takes the view that most development is actually redevelopment. most products have been built before at some level. industries such as telecommunications build the same or similar products over and over again. although the core requirements evolve slowly, there may be significant feature changes from one product to another. domain analysis techniques, such as fast [weis99) or foda [kang90), capitalize on the similarities between programs within a domain. using commonality analysis techniques, they examine products within and across a domain or rely on the knowledge of experts to discover the core requirements that apply to the entire domain. these are divided into two basic, but important, types. commonalties are the same across the products, such as the event model in a window based user interface. variations are the parts that differ between products, such as the actual menus or buttons within that user interface. similarly, analysis patterns [roberson93, fowler97) capture common requirements and their solutions within a particular domain such as accounting. they document problems and solutions, and capture the experience of developers within the domain. analysis patterns are at an abstract enough level to capture both the commonalties and variations of the problem. ajis special issue requirements engineering 1999 analysis patterns domain analysis object-oriented framework figure 1: relationship of analysis techniques to frameworks these domain analysis techniques and patterns are used to produce the domain specific representations of the product-line. these can take the form of tools that support domain specific languages such as its [batory98), or domain 'jargons' as in fast [nakatani99). as shown in figure 1, the products of domain analysis and analysis patterns can also be used as the basis for building object-oriented frameworks. o-o frameworks consist of an architecture that covers an entire domain, and an implementation of that architecture. the commonalties are built into the architecture and implementation so that they need not be developed over and over again. the variations are included as hooks into the framework. product developers can easily attach their own components through the hooks. the framework forms a solid basis for quickly developing new products within the product line. strategies for determining product-line requirements product-line requirements differ from the requirements of a single product in their breadth and their generality. in order to produce a successful product-line, the requirements must cover an entire domain and they must be general enough to capture both the commonalties and variations within that domain. new strategies are needed and these are explored in the next section. one strategy is to examine or to build products within a domain in order to gain experience in that domain (taligent 1995). obviously several products must be built or analyzed to determine the commonalties and variations across them. typically, at least three such products are needed. another strategy is to employ a domain expert. key to our success in constructing a product-line framework for engineering sizing and selection tools (froehlich 1999a) was the close collaboration with the product line champion: a domain expert in pressure safety relief valve sizing and selection his familiarity with not only the relief-valve domain but the broader domain of engineering design was indispensable in the development of our first product sizemaster mark iv (see www.avrasoft.com/sizemaster) and the current eaf (engineering application framework). we were also influenced by the expert users, practicing chemical engineers, who raised requirements that neither the domain export nor we anticipated. building products and using domain experts complement one another, as we have discovered in the development of our frameworks. a third approach that supports both the determination and elaboration of requirements is the creation of an experience base. in this approach, product-line development is treated as a continuous case study in which results from using the framework in several product developments is captured. these results are later used to improve the framework and the product development process that is based on the framework. the requirements for our csf (client server framework see http://www.cs.ualberta.ca/~garry) are undergoing this type of refinement as part of the ongoing framescan case study (froehlich 1999b). the csf supports the development of simple web-based client server applications in our fourth-year software engineering class cmput 401 see http://www.cs.ualberta.ca/courses/cmput401.html. the ultimate goal of this approach is to provide an experience base that complements and links to the requirements (encapsulated in hooks as described below), examples and other design documentation of a framework. our work to date in this area is preliminary but promising. documenting product-line requirements in this section, we describe how the requirements that are common across products are encapsulated, and how those that vary within the product-line architecture are documented. the variations, which are particularly important to product developers, are documented prescriptively using hooks. finally, we examine how ajis special issue requirements engineering 1999 product-line requirements evolve as the domain of the family of products evolve and why they must be managed carefully. hooks one means of encapsulating these requirements is through the use of hooks (froehlich 1997). each hook documents a narrowly focused means of using the framework to fulfill a particular requirement. for example, a hook will indicate how to create a menu within a user interface framework, or which packages to choose in a command and control framework. the suggested hook documentation consists of a structured template that elicits all of the required information needed to select and use the hook, including a section on the requirement fulfilled, the parts of the framework it uses, and any conditions on the use of the hook. since the product-line architecture and implementation already exists, the requirements must be mapped onto that implementation. as shown in figure 2, each hook maps its requirement back to the place in the framework that it affects. in addition, the hook defines in a systematic manner, how the design and code base of the framework should be extended. figure 2: hooks and classes in csf figure 3 shows the space of possible variations, that is, the changes, modifications or additions needed to produce a product within a particular domain based on an 0-0 framework. it is divided into four overlapping regions. option hooks are provided when the variations of a particular aspect of a domain can be explicitly catalogued. they are the narrowest in scope, but are also the easiest for the user. option requirements are fulfilled by selecting one or more items from a pre-defined set of components and linking them into the object-oriented framework. they can be completely automated and tested to ensure conformance with the framework. open unsupported figure 3: the space of possible extensions to a framework ajis special issue requirements engineering 1999 pattern hooks capture variations that can be parameterized and/or restricted. they may include options, but they also present the user with a restricted set of operations that the product developer is allowed to perform. patterns can be partially automated and tested. open hooks use a much less restricted set of operations. open hooks allow entirely new functionality to be added to the framework, and represent wide variations within a domain. unsupported changes go beyond the uses the framework builders envisioned for the framework. they may arise because the framework does not completely cover the domain or that the requirements of the product-line have changed. typically, the discovery of unsupported changes is an indication that a framework needs to be updated or evolved. examples of all four kinds of hooks can be found in our engineering application framework (eaf) which is used to build engineering worksheets. each worksheet is a collection of data elements and calculations that represent the contents of an engineering code or standard. each engineering application that we build has one or more worksheets that form the core of the application. the user interface of the application, provided by another framework, navigates through the worksheet but does not do any engineering computations. in this way we keep the robust core engineering part of an application relatively isolated from the more fragile user interface. an engineering standard is embodied into a worksheet by following the eaf hooks. option hooks are provided for adding data elements to the worksheet. each item of data in the standard is identified, and then classified according to whether it is numeric or text, and whether it is an external input from the user or an internal computed value, and whether its range of values is fixed or user modifiable. for example, one combination of options can be used to add a text data element that has a predetermined set of user-selectable values. for numeric data, there is the additional option of what basis units it has, such as length or temperature, and what its normally displayed style should be (for example, english or metric). pattern hooks are provided for adding new data element types, basis units and calculations. the hooks for new elements and units involve constructing new classes directly by writing code. the pattern for adding new calculations is embodied in a custom calculation language that hides many of the details. calculations are written in this higher-level custom language and then compiled into the class definitions that implement the calculation. this allows us to alter the implementation of the calculation hooks without having to rewrite the calculations. we just recompile them. open hooks are provided for calculations that cannot be described by our calculation language. worksheets are by nature acyclic in order to enforce a consistent evaluation order. this means that iterative components of the calculations must be individually programmed, and then attached to the framework through open hooks. unsupported changes are more an issue of good framework architecture. if the architecture is well-factored, we find that adding hooks for unanticipated requirements is a natural, although non-trivial, activity. evolving requirements when the requirements of the product-line evolve, it can have several different effects on the underlying framework: changes to the underlying framework these occur when the common requirements change, new ones arise, or the framework design needs to be made more robust. such changes typically include refactoring (fowler 1999) the framework classes without affecting the hooks. the variable requirements themselves don't change so the hooks don't change either; the only changes are in the common parts of the framework and their underlying implementation. changing hooks to make them more flexible. sometimes the variable requirements supported by the framework are not flexible enough to accommodate the requirements of the products within the domain. generally, one or more hooks need to be evolved along with parts of the underlying framework in order to support the new flexibility. as shown in figure 4, to make a hook more flexible, it must be made more open as well. changing hooks to provide more support for the product developer. as a framework matures, the exact variations between products within a domain can be catalogued and support for the different variations built directly into the framework. building in variations makes the framework easier to use as it contains more support for the product developer. as shown in figure 4, this is the opposite effect of increasing the flexibility of the framework. ajis special issue requirements engineering 1999 increased flexibility •̂ open _ ŵ pattern ŵ option increased ease of use figure 4: evolving from one type of hook to another adding new hooks. new hooks can be added in two ways: • as users gain more experience with a framework they may discover ways to support a requirement that was not documented by the original framework developers. these experiences can be documented in a hook description, usually an open hook, and are supported in new releases of the framework. existing products need not be affected. • new functionality can be added to the framework itself, and this functionality accessed through hooks. this type of change can have pervasive effects on the framework and can affect existing products that rely on new releases of the framework. refining requirements and requirements traceability in our experience, we have found that one of the difficulties in using a product-line architecture is mapping the requirements of the product to the requirements supported by the architecture. when developers are familiar with the architecture and the framework, they can perform this mapping intuitively. but when developers are new to the framework, it can be difficult to see the relationships between the focused requirements represented by the hooks and the coarse requirements of the product. we define a high level process consisting of a series of steps that guides the new product developer through the use of the framework. figure 5 shows an example for the client-server framework (csf). each of the steps is written in natural language and discusses a key concept of the framework, much like framework patterns (johnson 1992). the steps link to the hooks themselves, which in turn link to the framework. these steps provide three benefits: mapping. the high level process guides the user in deciding how to map their requirements onto the requirements supported by the framework. for example, the first two steps, discover data and identify persistence, guide the framework user in deciding what they need to communicate across a network, and which information needs to be persistent (stored in files or in a database). they are then pointed to specific hooks in the framework, specifically new data and read/write data, which handle transmitting and storing information in the csf. order. the steps of the process explain the order in which things should be done to successfully use the framework, which isn't clear from a simple list of supported requirements. even in the case of the csf, we found that when new framework users were presented with the unordered set of hooks and the dependencies between them (e.g. the send message hook depends on the new data hook), they needed some coaching in where to get started and how to proceed further. the ordering shown in figure 5 is somewhat simplistic, but appropriate for the csf framework. however, in practice for larger and more complex frameworks, choices made later in the steps may affect earlier steps, so more complex orderings and mappings may be required. 10 ajis special issue requirements engineering 1999 discover data 1r ^ identify persistence 1r communicating information communication performance process steps hooks figure 5: process for using csf details. this approach is similar to work done by lajoie and keller (1994) in which high level descriptions called motifs are mapped to the design patterns of the framework. the process steps themselves are based on the hooks and point to specific hooks in order to make the changes. the hooks map into the framework itself, so the transition from requirements, to architecture to implementation can be made smoothly. for example, in the communication performance step, the product developer decides on the specific communication needs, such as performance. the communication performance step directs the developer to, in this case, the choose mailserver hook, in which an 'engine' best matching the requirements is chosen. the hook itself points to the places in the framework that it affects. ideally, the steps will identify single requirements (or sets of similar requirements) which can then be mapped directly to a hook. the refinement performed potentially offers an additional benefit. requirements mapped to the framework during the process steps allow for requirements tracability. that is, each requirement identified is mapped to a single hook, which then points to a specific part of the framework that fulfills that requirement. additionally, support for this type of process can be incorporated into a tool, not just for a single framework as in the case of gui builders, but for frameworks in general. we have constructed the prototype of such a tool called hook master (liu 1999) for supporting the enaction of hooks on frameworks. conclusions using hooks to encapsulate requirements in a generic design and implementation is extremely promising for product-line development. it helps to localize the effects of requirement changes and focuses a developer's attention on those requirements that are unique for the product under development. traceability can be supported in a straightforward manner and with the assistance of a tool such as hookmaster. our approach has been tried out and refined in the development and use of several o-o frameworks including the eaf (engineering product framework), prothos (web delivery of database-centric applications), sandwich (personal web assistant proxy framework), and csf (client server framework). we are continuing to experiment and improve many aspects of our approach. future work includes: i) formalizing the hooks model so that model checking can be applied to ensure consistency and completeness of a product-line architecture, ii) extending the hookmaster tool to more explicitly support requirements traceability, iii) investigating the use of generic processes for o-o framework evolution, and iv) developing support for the creation and evolution of experience bases that can be associated with o-o frameworks. references d. batory, b. lofaso and y. smaragdakis. its: tools for implementing domain-specific languages. in proceedings of the 5th international conference on software reuse. (victoria, canada, june 1998). m. fowler. analysis patterns: reusable object models. addison-wesley, menlo park, california. 1997. m. fowler. refactoring: improving the design of existing code. addison-wesley, menlo park, california. 1999. 11 ajis special issue requirements engineering 1999 g. froehlich, h.j. hoover, l. liu, and p. sorenson. hooking into object-oriented product frameworks. in proceedings of the 1997 international conference on software engineering (boston, mass, 1997), 491501. g. froehlich, h.j. hoover, l. liu and p. sorenson. reusing hooks. in building application frameworks. m. fayad, d. schmidt and r. johnson, ed. wiley computer publishing, new york. 1999, 219-235. g. froehlich, a. kamal and p. sorenson, framescan: exploring o-o framework usage. located at www.cs.ualberta.ca/frames/papers. r. johnson. documenting frameworks using patterns. in proceedings of oopsla'92 (vancouver, canada, 1992), 63-76. k. kang, s. cohen, j. hess, w. novak and a. peterson. feature-oriented domain analysis (foda) feasibility study. (cmu/sei-90-tr-21, ada 235785). software engineering institute, carnegie mellon university, pittsburgh, pa. r. lajoie and r. k. keller. design and reuse in object oriented frameworks: patterns, contracts, and motifs in concert. in proceedings of the 62nd congress of the association canadienne francaise pour 1'avancement des sciences. (montreal, canada, 1994). w. liew. a personal web assistant framework msc. thesis, university of alberta, 1999. l. liu. hook master: a tool to support the enactment of hooks. msc. thesis, university of alberta, 1999. l. nakatani, m. ardis, r. olsen and p. pontrelli. jargons for domain engineering. in proceedings of the 2nd conference on domain-specific languages (dsl '99). (october, 1999). d.l. parnas. on the design and development of program families. ieee transactions on software engineering, se-2:l-9, march 1976. s. robertson and k. strunch. reusing the products of analysis. in proceedings of the second international workshop on software reusability. (lucca, italy, march 1993). taligent. the power of frameworks. addison-wesley publishing company, reading, ma. 1995. d. weiss. commonality analysis: a systematic process for defining families. in proceedings of the second international workshop on development and evolution of software architectures for product families. (february 1998). 12 the australasian journal of information systems (ajis) began in 1993 and was based on the simple realisation that there was a need for an academic ‘outlet’ for is research in australia australasian journal of information systems volume 16 number 2 2010 5 not so ‘invisible’: a qualitative case study exploring gender relations and farm management software dale carolyn mackrell faculty of information sciences and engineering university of canberra, bruce act 2601 maree veronica boyle griffith business school griffith university, nathan qld 4111 abstract this paper reports on a qualitative case study which explored farm management practices by women cotton growers who used computer-based information systems, most particularly the agricultural farm management software, cottonlogic, within the australian cotton industry. the study found that, although gender differences and inequalities persist, the agency of women cotton growers ensures not only a sustainable future for themselves and their families, but also for the cotton industry as a whole. the study was informed by connell‟s theoretical framework of gender relations (2002). the findings suggested that, women‟s active participation in family farm partnerships and their acquisition of technological skills through the use of farm management software like cottonlogic, mean that all cotton growers benefit through the feminizing of specific farm management practices in family farm enterprises. this, therefore, has significant implications for developing the cotton industry into a truly sustainable entity. introduction the study aims to understand how the use of farm management software, namely the agricultural decision support system (dss), cottonlogic, affects the decision making roles of women as farm partners within the australian cotton industry. in simple terms, a dss may be identified as a computer-based information system that supports decision makers. the extensive use of computers has become essential for farm decision makers, not only for communication, and information acquisition and transfer, but also for farm management (hearn & bange, 2002). although recent research indicates that farm women have become the primary users of computers, they are hesitant to use computers for farm management and hence, their decision-making roles on family farms are unclear (bryant, 1999; james, 1990; stewart, 2004). this paper aims to provide clarification with regards to the influence of the gendering of farm management software usage and the potential impact this may have on farm partnerships within the regional cotton industry. australasian journal of information systems volume 16 number 2 2010 6 this study is theoretically informed by connell‟s theory of gender relations (2002) which provides a framework for analysing the gendering of cotton farm decision-making roles. hence, the key question we seek to answer here is: „how does farm management software impact on gendered relationship between farm partners on family cotton farms?‟ women and farm management australian rural social researchers are concerned about the limited qualitative data available on the roles of australian farm women. alston (1995; 2000; 2003) found that the participation of women in agriculture is „invisible‟ with a persistent and regrettable absence of women from positions of authority and power. board (1997) recognized that harnessing the expertise and diverse skills of rural women leads invariably to better decisions. nevertheless, the focus of government, agribusiness, and even the rural community itself, still rests with male primary producers as the main recipients and users of information and services related to primary production (stewart, 2004). studies have indicated that farm women lead busy lives, and time is a major constraint in the performance of their many functions. their multiple roles range from farm manager and office worker right through to tractor/header driver and farm animal carer, in addition to traditional caring roles for children and elders (board, 1997; james, 1989). despite some evidence that male farmers are making a greater contribution to domestic tasks and childcare to compensate for the increased involvement of women in the running of the farm business (kilpatrick, johns, murray-prior & hart, 1999), women on farms are still required to do the bulk of domestic work (james, 1989; schaffer, 1988). many australian rural women bring to farming communities a myriad of non-agricultural skills such as teaching, nursing, retail, administrative and clerical. furthermore, 65 percent of farm women who work off-farm have been tertiary educated (alston, 2003; kilpatrick et al, 1999). even though women may have higher formal educational levels than their male counterparts, they are less likely to have undertaken formal studies and employment in rural-based scientific and technical areas (board, 1997). this is due, in part, to a generational effect, since, prior to the 1970s, women were forbidden from gaining college degrees in australia in the fields of agronomy, horticulture, agricultural economics, and animal science (alston, 2000; 2003). indeed, prior to the mid-1990s, little data was gathered on australian women working in agriculture. this omission stemmed from a sense of shame that a developing nation such as australia had women working out in the fields (broad, 1997). yet by the mid-1990s, there was evidence of changing gender identities, in that 70,000 women defined themselves as farmers or farm managers rather than as farmers‟ wives (alston, 2003). several reasons are given for this change of designation. when economic conditions in the rural sector deteriorated and structural adjustments were an imperative, farm women provided vital labour to replace more costly hired farm labour on the family farm. this was especially evident on smaller family farms. as a consequence, women‟s work has been crucial for the survival of the family in farming. furthermore, some farm women have chosen or were forced to pursue careers outside of farming to generate additional income for the family when long term viability of the farm business was doubtful (alston, 1995; bamberry, dunn &; kilpatrick et al., 1999; james, 1989; lamont, 1997; poiner, 1990). the family farm mode of ownership remains dominant with over ninety percent of australian farms being family-owned despite various recent rural adjustment schemes to facilitate the exit of small farmers and the consequent consolidation of small family farms into larger holdings (alston, 2000). furthermore, there has been an increase in farming partnerships, most of which are between husband and wife (james, 1990). with the gaining of legal partnership status, it appears that farm women are moving away from predominantly supportive roles to become more involved with the business side of the family farm enterprise (fisher & hutchison, 1997). all the same, james (1989; 1990) claimed australasian journal of information systems volume 16 number 2 2010 7 that the role of women in decision making on the family farm is often overlooked and the scope of the contribution by women to family farm management remains vague. the theme of the „invisibility‟ of women farmers emanated from north american research, especially by sachs (1983) and resonated through feminist studies of australian rural women. it is well documented that farm women‟s subordinate social position has been imposed by the patriarchal social relations of family farming which rendered women and their work „invisible‟ (james, 1989; poiner, 1990; schaffer, 1988). for example, poiner‟s (1990) explanation of farm women‟s lack of visibility was the „pervading male hegemony‟ of a conservative rural society. alston (2000) argued that an inferior role for women has been facilitated by patrilineal inheritance practices in agriculture where only five percent of farms are inherited by daughters. she contended that farm roles have developed based on gender stereotypes, that male farmers are participants in the „more important‟ public sphere of outdoor work while farm women have become associated with the less visible private sphere of housework and children (alston, 1995). alston (1995) claimed that domestic work is often devalued since it is unpaid and perceived as not contributing directly to agricultural production and the marketplace. paradoxically, farm women are perceived to cling strongly to their farm roles defined by gender. bamberry et al. (1997) stressed that farm women have a strong commitment to their farming lifestyles, to the viability of the family farming operation, to the nurturing of their family, and to the community which supports them. evidence indicates that farm women tend to be suspicious of feminism as constructed by the media and are more likely to view it as a threat to their solidarity with their male partners (alston, 1995; board, 1997; james, 1989; poiner, 1990; rowe, 1997). in brief, an overview of the literature suggests that farm women are, to some extent, an underappreciated and underutilized resource with a commitment to maintaining traditional family ideologies on family farms. nevertheless, their identities are fluid in the context of emerging social trends. joint legal ownership of family farms is becoming a reality. many farm women have tertiary educational qualifications and a broad range of skills and experiences derived from both within and outside the farming sector. therefore, structural changes are underway and farm women are poised to become a more creditable and visible force. women and computers for farm management several studies from both industry and academia have explored women‟s use of computers for farm management in australia (mackrell, von hellens & nielsen, 2009). in her account, bryant (1999) reported that the use of software reflects the traditional gender divisions of labour on farming properties in that farm women are associated with financial data entry and record keeping, while male farmers provide the input data, analyse, and plan the farm business. bryant (1999) contended that while many rural women are increasingly aware of the decision making and farm management possibilities of computer programs, there is still a high level of dependence upon the male farmer, with his more detailed day-to-day outdoor farm knowledge. stewart‟s (2004) study of gender, technology and cotton farming in the late 1990s found that ideologies of family farming, like technology, are socially shaped as a male domain. that is, both farm roles and technology are gendered according to whether or not they are predominantly indoor or outdoor activities or skills. male farmers are more likely than farm women to make sophisticated use of computers for the purposes of farm decision making and management with farm women almost invariably recognized as lesser status, data entry operators. stewart (2004) argued that this resulted in a lack of confidence amongst women as controllers of data, which sometimes led to avoidance of responsibility for developing information systems for decision-making purposes. australasian journal of information systems volume 16 number 2 2010 8 bellamy, webb, mayocchi and leitch (2002), in their report of a more recent cross-industry study of the use of technologies for natural resource management, identified an improved adaptive ability in farm management by farm women through increased use of computer-based decision support tools. bellamy et al. (2002) found that seventy to eighty percent of women in rural industries self-reported as being jointly involved in major (strategic) decision making with their farm partner / spouse and / or other family members, while women in the beef industry self-reported a contribution to all farm decisions. in addition, they were the greatest users of technology. this confirmed the findings of an earlier study by lewis (1998) of a strong association between innovative decision making by farmers and being better informed through the use of computer-based management information systems. in essence, a review of the literature indicates that cooperation in the use of the farm office computer by both farm partners (generally husband and wife) is critical for a more effective outcome. collaborating is an opportunity for the male farmer, who frequently works outside in the paddock, to combine his skills with his women farm partner, who frequently works in the farm office. the study in this paper extends knowledge from earlier studies temporally and theoretically as explained in the following section. theoretical framework: connell’s theory of gender relations in this paper, we use connell‟s (2002) theory of gender relations as an analytical framework. in theorising gender relations, connell (1987) established several units of analysis, namely gender orders and gender regimes. the wider social sphere, represented by patterns in gender arrangements of social institutions such as „the state‟, „the church‟, and „the military‟, contains a structure which connell calls the gender order of society. within gender orders are gender regimes which are established in smaller social settings such as families, neighbourhoods, schools, clubs, organizations, and workplaces. within these social domains, connell identified three main social structures: the division of labour; the exercise of power; and „cathexis‟ which encompasses the domain of sexual social relationships (connell, 1987; stewart, 2004). connell (2002) revised his original theory of gender relations (1987) to include a fourth social dimension. thus, his 2002 theory of gender relations encompasses four main dimensions of gender: production, emotion, symbolism and power relations. production relations apply to the gender divisions of labour both at home and in occupational employment. emotional relations are particularly applicable for emotional attachments, for example, marriage commitments. symbolic relations refer to gender symbolism in life and in marketing, language, fashion, film, and even architecture. power in the form of oppression operates through institutions where patriarchal dominance exists. connell (2002: 68) explained that while these four components are analytically separate, in practice they constantly intermingle. connell‟s (1987) original gender relations model was used recently by alston (2000; 2003) with research into australian rural women for analysing the construction of gendered power relations within agriculture. additionally, stewart (2004) used connell‟s theory of gender relations (1987) with its social structures of labour, power and cathexis. the contemporary theory of gender relations (connell, 2002) is more recent and its use to date has been less extensive. research context, design and methodology the management of cotton growing in australia has become increasingly complex with the need to sustain reliable crop production while making the best use of limited water and soil resources, to utilize effective pest and weed management, and to limit detrimental environmental impacts (the australasian journal of information systems volume 16 number 2 2010 9 australian cottongrower, 2006). a study by hearn and bange (2002) found that innovative farm management technologies such as the agricultural decision support system, cottonlogic, are considered keys to the adoption of sustainable farming systems. research context cottonlogic is an advanced farm management suite of software programs to aid the management of cotton production. the software was developed in australia in the late 1990s by the australian commonwealth scientific and industrial research organization (csiro) and the australian cotton cooperative research centre (crc), with support from the australian cotton research and development corporation (crdc). cottonlogic consists of record-keeping and decision support modules to assist cotton growers and their advisors in the management of cotton pests, soil nutrition, and farming operations. mackrell (2005) found that the design of cottonlogic into record-keeping and decision support modules contributed to adoption and implementation success with cotton growers being more likely to use the record-keeping modules for recording and reporting crop inputs and yields while cotton industry professionals are more inclined to use the relatively complex decision support modules for predictive purposes. design and methodology the empirical work was carried out by the first author and was based on a qualitative case study covering areas in south-east queensland and northern new south wales, in australia. the rationale for a qualitative study was that an in-depth study of a situated experience was expected to provide deeper understanding than gathering standardised quantitative data from a large sample of the population. the study was steeped in an interpretive paradigm, and as such, themes emerged from the interview data rather than being pre-determined beforehand as for a survey (patton, 2002). the unit of analysis was at the individual level, predominantly the australian women cotton grower in her role on the family cotton farm. participants were either cotton growers or cotton industry professionals and were selected according to a purposeful sampling strategy. the cotton growers were selected based on the following criteria: 1) farmed in the australian eastern states of queensland or northern new south wales; 2) were responsible for, that is, owned and/or managed family farms irrespective of size (as distinct from farms owned by large corporations); 3) indicated an awareness of environmentally sustainable and high-technology farming practices; and 4) were registered on a cottonlogic and/or wincott (women in cotton industry network) database. the cotton industry professionals were cotton agronomists and consultants, rural extension officers, researchers and educators, rural experimental scientists, and cottonlogic developers, located in queensland or northern new south wales. these participants were required to have some knowledge of agricultural dss either through development, usage, research or teaching, and, to some extent, were observers and/or advisors of cotton growers. data gathering took place during three field studies and one telephone study with 32 participants over three years. further details can be obtained from the first author. an interview guide was prepared to steer the interviews which were conducted at locations selected by participants, with each interview lasting at least an hour, and recorded on audio tape with permission. interviews varied between semistructured and conversational depending on circumstance: semi-structured when the interviewee needed considerable guidance, and conversational when that was not the case. notes on each interview were recorded daily in an activity log and interviews were transcribed from audio tape into microsoft word as soon as possible. analysis was manual rather than computer-assisted since the number of interview transcripts was workable and the obligation to stay closely connected with the data was fundamental. codes used in analysis were based on concepts or themes drawn from both the literature and theoretical framework. australasian journal of information systems volume 16 number 2 2010 10 in short, this study of cotton growers using farm management software in the australian cotton industry was an interpretive single case using ideographic methods. it allowed a first-hand investigation, involving several field trips to study participants in their natural setting, taking place over an extended period of time, and producing a textual analysis of rich insights after a period of reflection. data analysis this section reports the analysis of gender relations in the context of farm management on the australian family cotton farm using, as a conceptual lens, connell‟s (2002) theory of gender relations with its four social structures of gender: production, power, emotional, and symbolic relations. the following analysis commenced with production relationships. this is justified because the family farm as a commercial enterprise is production centric. that is, if production ceases so eventually will the farm as a business unit. production relationships according to connell (2002), production relationships pertain to the gender divisions of labour both at home and in occupational employment. the australian rural literature indicates that women‟s roles extend from household worker/manager to farm worker/manager as well as to the carer of children and the elderly (board, 1997; james, 1989). men‟s roles are traditionally associated with the production and marketing aspects of farming with the majority of time spent outside (alston, 2000). it was evident that some women growers were becoming more involved in marketing, selling and buying, as well as investment decisions on family farms, tasks which have traditionally been in the male domain. selma, a cotton grower, explained her roles: i’ll deal with interest rates. i’ll sell cotton. i’ll make decisions about options, and things like that. he’ll [farm partner] have no hassles with that. he’s a fair bit more liberal than the traditional guys. we work as a team really. i’m more focussed on the administration side of things. bill [farm partner] is more focussed on the day-to-day running, and keeping out of the office which he shouldn’t do. he needs to be more involved. in the above scenario, selma takes on the marketing, selling, and investment tasks that expose her to financial risks. in the recent past, this function would probably have been the domain of the male farmer or farm manager, and, women operating outside conventional rural gender norms would have been treated with suspicion (alston, 2003). however, as disclosed later, selma‟s farm partner husband, bill, would rather be out in the fields and not in the office. beyond the farm gate, selma is involved in local grower associations. her industry knowledge means that her role on the farm extends beyond bookkeeping and human resources towards strategic production decisions. she has very firm views on the running of a successful cotton and cattle property, and the importance of good management. therefore, it was selma not bill, who noted that he should have more involvement in farm management with a reduced emphasis on outdoor work. selma and bill‟s comments implied a reversal of the commonly accepted attitude that inside office work (unlike outside physical work) is not considered to be real work. therefore, for this family farming team, this is a win-win situation where tasks, strategic and operational, are allocated according to skills and preferences rather than gender. the notion of being part of a family farm management team was reiterated by several women growers in relation to a australasian journal of information systems volume 16 number 2 2010 11 husband and wife joint partnership. meg, a grower, explained that her role on the „team‟ was as the farm bookkeeper: you were asking about women’s roles but in the cotton industry, it’s more of a team. the bookkeeping is the biggest factor [of my role]. we use the computer for that. i do all the bookkeeping. despite a growing recognition of changing responsibilities in the context of cotton growing, commonly the woman farm partner performed tasks in the conventional mould such as doing the farm accounts while the male farm partner worked outside in the fields. however, traditional gender divisions of labour did not diminish the importance of the task or a partner‟s appreciation of the other‟s contribution to the team effort. brad (meg‟s farm partner) recognized meg‟s valuable contribution to production when he expressed his relief and appreciation of her willingness to help reduce the management burden. brad was grateful that decision making in farm management could be shared with someone having the same goals. furthermore, brad acknowledged the importance of office work and his conviction that the inside / outside balance should shift so that less time was spent outside: meg does all the stuff inside here and i basically do most of the stuff outside. i’ll tell you what i find. there are so many decisions that you have to make and so many things in the modern agricultural world, or certainly on cotton farms, it’s a big load for any one particular person, and i just reckon that it really improves the efficiency of the whole thing. i can just rely on meg to look after all the accounts, all the human resources sort of stuff. i don’t have to worry about them… we should be working more on the business and be less hands-on. both meg and selma did not limit their involvement to the farm office and both women worked willingly alongside their male farm partners when circumstances dictated the need for their labour out in the paddocks: meg: and it really is a team. i actually drive out and i work with brad. we do irrigating together and selma: if they need an extra hand, i’m down there or if cultivating and trying to get around something, i’ll jump on the cultivator despite the evidence of traditional gender divisions of labour in the home and outside, there was a demonstrated softening of role distinctions. as more women were shouldering the administration tasks of the farm business, some men were more willing to help out in the house: interviewer: and does he ever help you in the house? julia (grower): yes things are pretty good actually. most men won't even make a cup of tea let alone cook a meal, whereas if i'm busy doing paper work and he's not busy, it's not this thing where a lot of men will ask … what are we having for lunch? but he will go ahead and make it. several industry professionals such as reese and kylie recognized the husband and wife partnership style of farm management as becoming a norm across smaller family farms. reese: i think that is a pattern across the industry. the smaller growers, the family farms, their children including the girls go out and work in the fields. the benefits to the farm enterprise in combining limited human resources were identified. kylie: they [women] are very much a partner in the business and unless you have both the financial and production sides of things working very well, you’re not going to make money today because farming’s a business … if you didn’t have a wife to do it, you’d have to pay somebody else and nine australasian journal of information systems volume 16 number 2 2010 12 times out of ten, they [the women] know the business so well that they are really adding value with what they are doing. there is a growing expectation that women‟s roles will be more varied as youth become adults in rural society with the prospect of fewer gendered boundaries. toni, a grower with two young daughters, explained: interviewer: you’re saying that the girls work outside. do you think that is an indication of things changing? toni (grower): yes definitely. i see no problem with that. my mum never did anything and i probably do a bit more than her. the loosening of gender-based restrictions was evident to reese in her work for a cotton agribusiness. since australian agricultural colleges opened enrolment to women in the 1970s (alston, 2000; 2003), working in the rural sector became a career choice for an increasing number of women. … but i think there are more women becoming involved in the industry as a whole, from being agronomist or growers though to the chemical companies and sales people and even out in the field, there are a few. maybe not near as much as what males are but i think they are becoming more involved in the office side of things. within this section, gender divisions of labour in production units on family cotton farms have been explored revealing adjustments to established societal norms and notions of teamwork. it is clear that some behavioural changes are generational with many farm women unwilling to accept codes of behaviour from previous generations which were unfavourable towards them. as women adopt new responsibilities, such as preparing financial budgets, and marketing and selling cotton, it is expected that, over time, these roles become sanctioned as normal codes of behaviour. power relationships connell (2002) stated that power relationships in the form of oppression operate through institutions where patriarchal dominance exist. foucault, cited in connell (2002: 58-59), proposed another form of power where „power is widely dispersed, and operates intimately and diffusely. especially it operates discursively, through the ways we talk, write and conceptualise‟. this power is less authoritative, oppressive and confrontational. both institutional and discursive power are used as analytical concepts to facilitate an understanding of gender in relationships of authority. in the context of the cotton industry, institutional patriarchal power could be identified as existing within an older generational pattern. for example: helena (grower): with women, some husbands stop them going into the fields. they are not allowed to make decisions, but i think women’s input is just as important as the men’s. and selma (grower): i think bill’s a bit different to most of the farm blokes around. i know blokes who don’t let their wives get mail from the post office. interviewer: is that your age group? selma (grower): no, i’m talking another ten years older. oh, yeah, twenty years older. discursive power was more in evidence in this study. sigrid, an experimental scientist, explained how the financial knowledge which the women acquired as they prepared the farm accounts provided some women with the means to challenge existing farming operational decisions. australasian journal of information systems volume 16 number 2 2010 13 whereas a consultant might tell a [male] grower to spray such-and-such, they might not even question how much that will cost or if there is an alternative. whereas the women, because they know how much it was, and they put it in [the budget], they are possibly more challenging to their consultant. several of the decisions were of strategic importance to the farm business, thereby providing the farm women with a greater measure of control. this supported the findings by bryant (1999) that some women are using data from record keeping to participate in and influence decisions about farm management. sigrid exemplified the point through the light-hearted banter between her parents regarding the purchase of machinery, generally an expensive capital investment. women seem to get their power and ability to make decisions on the farm because they know exactly what the budget was, because they’ve done the budget. a lot of the men wouldn’t have a clue. what dad often says is ‘i think i’m going to buy a round-bale machine’? mum’s like ‘yeah right, no, it’s not in the budget. we can’t. but we can put it in next year’. elle, a grower, illustrated a structured decision-making process whereby the four partners jointly decided on the more significant strategic management issues while the two male partners together made the operational decisions. any decision that we make regarding management decisions, there are four of us in our partnership, we have a meeting and the four of us make the decision. if it’s a big management decision then the four of us must agree. the two men make the decisions on the day-to-day running of the farm, this is what we plant, and this is where we’re going to plant it and all that sort of thing. certainly, many of the women cotton growers, although claiming to be part of the family farm management team left the day-to-day decisions connected with production as applied to crop planting, spraying, or harvesting to the men. reese, a cotton professional, explained that women on family farms are not so involved in operational decisions on production but more with record keeping for budget reasons. george, an independent consultant, confirmed the lesser impact of women on the operational day-to-day decisions. reese: but i think for a lot of growers: the males take on the decision making themselves in conjunction perhaps with their agronomist. i think that women are taking more of a role in the record keeping though. and george: women on the farms always have had a strong orientation to the strategic positions, not the day-to-day decisions. diane, a service manager for an agribusiness, explained that in order for the women to participate in operational level decisions, there was the necessity to be acquainted with chemicals, product costings, as well as the science of crop and soil management. … but as far as the actual agronomic side of things, most of them [the women] don’t have that background to be able to discuss it with them [male partners]. this was an important remark. notwithstanding, reese, a consultant, explained that the farm woman is often well-positioned to offer an alternative perspective. that’s right and there are also things that women will see in management that a man can’t see. like a man is looking at the things all the time whereas a fresh face, not necessarily the wife or the women involved, but any fresh face but it’s normally the woman … australasian journal of information systems volume 16 number 2 2010 14 several times during interviews, it was implied that management dynamics had changed once extended family, such as brothers and brothers-in-law, had left the farm partnership and the controlling entity had reverted to the male farmer and his wife: toni (grower): … now that the boys [farm partner and his brother] have split their partnership and we’re working separately, and now the girls are old enough to help, they drive the tractors, move the irrigators …. alston (1995: 63) claimed that if a farm is owned in partnership by a male farmer and his family, the farm woman remain marginalised from the farm business. she described the male-oriented culture in farming as follows: the structure of the farm family shapes the very nature of the farm woman’s life. if the farm is owned by her father-in-law or in partnership with her husband’s brothers, she will remain marginalised within the family and the business for much of her life. as stated in the literature review, there has been an increase in farming partnerships, most of which are between husband and wife. australian rural women now make up forty percent of all farm partners (james, 1989; 1990) with farm women moving away from a predominantly supportive role to become more involved with the business side of the family farm enterprise (fisher & hutchison, 1997). the mobilization towards legal partnership has given the farm woman the opportunity to assume a greater role in strategic decisions of farm management. this was illustrated by sarah as she justified her increased participation in marketing decisions since her brother-in-law had exited the partnership: my husband is not chauvinistic. we work collaboratively. with regards to decision making, we know we will grow cotton. i have input into every bale sold and i’m involved in marketing. but that didn’t happen when we were with gordon’s brother. now we are out on our own. this year is the first crop in our partnership without my brother-in-law. in this section, while there was little verification of institutional power, except by hearsay, discursive power was in evidence and usually it was wielded by the women who were familiar with the financial circumstance of the farming business. the situation of farm women varied immensely. factors affecting their participation in farm management included age, skills and talents, educational background, the presence of relatives in the partnership, and the employment of agronomists or consultants. nevertheless, the women interviewees were strongly aware of their rights as farm partners and disapproved of patriarchal and patronising attitudes. emotional relationships emotional relationships apply when attachments are based on emotions (connell, 2002), as in family commitments, and in business and lifestyle choices such as the running of the family farm. in the example below, selma expressed the strength of the attachment to the farm: we have to work as a team. we’ve had to fight a lot of hard battles to have what we’ve got. we’re determined to stay viable and on the land. we have to take an active role. three of the farm women explained that historically women have not always been so connected, nor have they participated by sharing the farm management load. this was described by julia, a grower. i suppose my mother was never really involved in any of the farm stuff but it was more the point of view that if they [husbands] know you can do it then they get you to do it … and she’s often said to me don’t always say you can do everything (because) then you will have to do it. julia‟s mother appeared to lack emotional dedication to the farm. julia, however, had adopted a greater sense of loyalty represented by willing and active participation in farm management decisions. australasian journal of information systems volume 16 number 2 2010 15 this situation aligned with many of the interviewees who expressed strong emotional commitment to the family, farm, business, community and lifestyle, despite challenging times in rural australia. symbolic relationships symbolic relationships in gender relations refer to gender symbolism in many spheres such as language, fashion, film, marketing and architecture, and its contribution to the construction of identities (connell, 2002). connell (2002: 73) stated that „patriarchy has long been legitimated by belief systems which picture gender as a timeless unchanging division … over the last century and a half, social and intellectual movements have chipped away at these assumptions‟. diane, a woman cotton agribusiness manager, opined that the marketing of the farm management software cottonlogic was directed mainly to men in the cotton industry: the thing that i see why cottonlogic, … and those sorts of things, haven’t been picked up, the marketing of the extension of those programs has been directed towards men and not so much the women. one of the reasons, and it’s just a theory, it’s safer and easier [to direct towards the men] …. as discussed, stewart (2004) found that rural services such the marketing and extension of agricultural products had been traditionally focussed towards the male farmer. diane perceived this was also the case with cottonlogic. paradoxically, the cottonlogic courses were run by a woman, sigrid, an experimental scientist, and much of the material promoting cottonlogic showed women as cotton scouts. all the same, george, an independent consultant, observed that in many cases, the perception of farm women is still unflattering: the women are starting way behind the eight ball. even if they are intellectually far ahead of the farmer, they are perceived as way behind the eight ball. there is evidence that this practice is changing to become more family focussed with the entire family invited to participate in field days. according to naomi, a cotton extension coordinator, this encourages the women to be involved and not left behind at home caring for children: but certainly if you’re running field days, mostly men attend. we’re really trying to encourage – same with our ipm [integrated pest management] courses (and) we make provisions for the family to come at a discounted rate. we’re really trying to encourage as a choice, decision making and learning. as stated earlier, recent studies have found that farm women are defining themselves as farmers or farm managers rather than simply as „farmers‟ wives‟ (alston 2003). this study has disclosed that the involvement of women cotton growers in a diverse range of farm related activities justifies their self-styled description of themselves as cotton growers rather than simply cotton growers‟ wives. in summary, relationships of production, power, emotions and symbolism from connell‟s (2002) theory of gender relations illuminated that teamwork was highly valued by farm partners, especially by the women themselves. the women cotton growers were confident of the value of their labours in the farm office and paddocks. they were committed to their way of life, and aware of their identity as cotton growers not just as wives, important as that function was. while women‟s tasks on farms generally fell within traditional gender norms being inside work, the blending of male and female roles through necessity meant that gender distinctions were becoming blurred. even so, farm women‟s participation in decision making associated with the production aspects of farming was low. australasian journal of information systems volume 16 number 2 2010 16 discussion the social dimensions of gender relations as defined by connell (2002), namely, production, power, emotion and symbolism, are highly relevant in the context of the study. combined together, these concepts signified the relations between farm partners, usually husband and wife, as together they managed the family farm as a small business enterprise. relations were productive because the farm partners laboured together to sustain the family farm as an economically viable business entity. power was characterised by the allocation of decision-making roles. the bonds of the farm partners were emotional because of marital responsibilities and all that family life on a farm entailed. symbolism had an important association with marketing and influenced perceptions within and outside of the cotton industry. the analysis of the data elicited some interesting findings such as the prevalence of teamwork in the farm partnership, the conviction of the women regarding the value of their roles as farm managers, and the changing focus of marketing towards women and families in the industry. these are discussed further below. multidisciplinary teams in farm management few studies have investigated multidisciplinary teamwork in the rural sector. studies into multidisciplinary teamwork in health care highlight the value of diversity, as well as the obstacles when health care professionals have differing attitudes towards the outcome. cott (1998), in a study of multidisciplinary teamwork in the canadian health care system, claims that definitions of teamwork emphasise some of the basic assumptions of multidisciplinary teams. these are that team members should have a shared understanding of roles, norms, and values so that the team functions in a cooperative, egalitarian but interdependent manner, and that cooperative decision making should benefit the purpose (or patient) rather than the individual team member. in brief, while a group works together to achieve a certain outcome, a multidisciplinary team is a group of people with different training and experience working together with a common purpose. therefore the tasks they perform as team members may not be the same but instead are related to their expertise in benefiting the collective outcome. for this study, the concept of multidisciplinary teams essentially describes partnerships on family cotton farms in the australian cotton sector. the evidence suggests that teamwork on the family farm is of a collaborative but interdependent nature. moreover, since roles may overlap, there is certain flexibility about which tasks are to be performed, when and by whom, in order to achieve a viable and sustainable family farm business. it was apparent that the women had no hesitation in regarding themselves as team members in the family farm business. all the same, in a few cases, the women astutely declared that the functioning of the team would improve with greater cooperation of farm partner husbands. women as decision makers in farm partnerships like most rural sectors, the cotton industry is subject to volatility, as in the weather, market supply and demand, global costs and prices, legislative compliance, environmental policies, and community expectations. it is essential for women as farm partners to have the resourcefulness to respond to these fluctuations. the knowledge and skills that women bring to the workplace are diverse, enrich rural society, and enable them to adapt and cope. women‟s roles on farms vary from farm to farm, and from season to season, with on-farm and off-farm work placing full demands on their times. certainly, for most farm women, their time is in short supply due to the extensive and varied roles they assume. some women, especially when children are young, assume traditional roles inside the home while others spend more time on outside duties. australasian journal of information systems volume 16 number 2 2010 17 several women growers recognized that the use of financial software, along with knowledge of the farm budget, influenced their farm management roles, notably for strategic decisions. however on the production side of the farming operation, the women were consulted less. for them, there were numerous constraints. aside from lacking confidence in their agronomic knowledge, the main one was that the male grower and the farm agronomist/consultant made decisions frequently and in haste in the fields based on their experience and intuition. this was to the disadvantage of many of the women partners who were often not present. bellamy et al. (2002) found that only ten percent of cotton women participated in joint decisions at an operational level compared with those in other rural industries such as beef and grains where sixty percent of women self-reported as contributing to production decisions (bellamy et al., 2002). this meant that the women in the cotton industry are considerably less involved in the day-to-day decisions associated with production or marketing than in other rural industries. this situation highlighted the fact that technology courses (such as for cottonlogic) for women farm partners may need to include an agronomy component to supplement software training. this would enable the women to be better informed and, if they aspire to it, to participate more usefully in farm decision making on a day-to-day (operational) level. in the literature on women on farms in australia, personal conflicts between self fulfilment and farm wifely duties have been recurrently recorded (allan, 2005; alston, 1995; james, 1989; sachs, 1983; stewart, 2004). nevertheless for the women in this study, the sense of isolation, unhappiness and despair regularly documented could not be substantiated. the recent establishment of wincott (women in cotton industry network) as a resource and self-support organization by the women of the industry denotes a recognition of their self-identity, their agency, their achievements, as well as constraints in accessing knowledge. wincott encourages members to ask question and seek answers without embarrassment. it is a source of motivation and confidence for many of its women members despite some initial resistance from men in the industry. there are claims in the literature that farm women are an underutilized resource. however, their changing status as legal farm partners has given them the opportunity to assume a greater role in farm management (alston, 1995; 2000). two women growers hinted at discordant relations in farm partnerships when extended families were involved. for these women, the position improved when, with their farm partners, they became the controlling entity and dominant decisions makers after extended family such as brothers and brothers-in-law left the farm partnership. this confirmed the findings by stewart (2004) that farm women were able to build more equal partnerships with their farmer husbands when other males were not involved in the farm partnership. gender stereotypes in marketing symbolism the perception of the supremacy of masculine discourse in rural services, such as rural extension and marketing, may be partly responsible for farm women‟s reluctance to utilize computers for production decision making. stewart (1997) provided numerous instances where „hegemonic masculinity‟ was represented in advertisements in specialist farm literature by depicting older, weathered male farmers and heavy, powerful machinery. conversely, emphasised femininity was symbolized in industry journals by young models on the catwalk wearing fashionable cotton garments. rarely was the reverse seen where women were depicted driving heavy machinery or men modelled cotton garb. within the ideologies of farming, the symbolism of hegemonic masculinity versus emphasized femininity is associated with connell‟s notion of power relations. nonetheless, the rural sector is being transformed, as in the latest „cotton yearbook‟ (2006) by the australian cottongrower. images of wholesome family scenes are becoming more common. this atmosphere is less threatening to women as it encourages their involvement alongside that of their farm partner husbands. australasian journal of information systems volume 16 number 2 2010 18 conclusion in this section, the response to the research problem is discussed as well as the contributions of the paper to theory and practice. the research problem was identified as: „how does farm management software impact on gendered relationship between farm partners on family cotton farms?‟. while results of past studies were conflicting (bellamy et al, 2002; stewart, 1997; 2004), this study found that women cotton growers, alongside women in other rural industries, were willing participants in most long-term and strategic decisions on the family farm, although their involvement in day-to-day production and operational decisions was generally less. the study also found that many women cotton growers were motivated to increase their contribution, and that agricultural dss usage was expected to assist in providing improved reflexivity in their farm decision-making roles. the findings also suggested that women cotton growers were no longer „invisible‟. while reluctant to challenge traditional notions of identity, farm women were not immune to social trends with generational change taking place on family cotton farms. with their diversity of skills and experiences, women were becoming valued and „visible‟ members of family farm management teams. ever since farm women took over the bookkeeping roles on family farms, the knowledge acquired had assisted them in influencing financial decisions at every management levels. implications for theory distinct from the representativeness of findings (external validity) in quantitative studies, generalisability in a single interpretive case study such as this, involves generalising from empirical, rich descriptions to theoretical statements (walsham, 1995). for this study, the outcome is rich insights as theory rather than the generation or testing of theories. as explained earlier, this paper extends extant scholarly literature in the research domain of the study, namely gender, farm management software, and farming, certainly within the context of australian cotton. furthermore, connell‟s (2002) gender relations theory as a conceptual lens has provided novel views of the social world of women cotton growers. these are the theoretical contributions of the paper. implications for practice the practical implications through capturing the increased decision choices of women cotton growers are innumerable. as acknowledged farm management team members, farm women need to be considered when rural goods and services are promoted; when positions for farm managers on rural properties are advertised; and when government policy affecting the rural sector is being prepared. it is possible that greater skills in the use of farm management software may be a means of improving the self-confidence of farm women as decision makers, and that involving farm women in decision making may help in finding sustainable farming solutions outside those in existing practice. references allan, j. (2005). farmers as learners: evolving identity, disposition, and mastery through diverse social practices, rural society. 15(1): 4-21. alston, m. (1995). women on the land: the hidden heart of rural australia. university of new south wales, kensington nsw. alston, m. (2000). breaking the grass ceiling: women, power and leadership in agricultural organizations. harwood academic publishers. amsterdam the netherlands. alston, m. (2003). women's representation in an australian rural context. sociologia ruralis. 43(4): 474-487. australasian journal of information systems volume 16 number 2 2010 19 bamberry, g., dunn, t. & lamont, a. (1997). a pilot study of the relationship between farmer education and good farm management. rural industries research and development corporation (rirdc), canberra act. bellamy, j., webb, v., mayocchi, c., & leitch, a. (2002). improving resource management through rural women's use of new technology: a pilot study in impediments and opportunities for learning activities. csiro sustainable ecosytems, brisbane qld. board, h. (1997). rural women, policy and the process of change in australia agriculture. proceedings of rural australia: towards 2000 conference. centre for rural social research, charles sturt university, wagga wagga nsw. broad, l. (1997). the untapped resource. proceedings of rural australia: towards 2000 conference. centre for rural social research, charles sturt university, wagga wagga nsw. bryant, d. l. (1999). computers on the farm: farmers' usage patterns and impact on farm management. rural industries research and development corporation (rirdc), canberra act. connell, r. w. (1987). gender and power: society, the person and sexual politics. stanford university press, stanford ca. connell, r. w. (2002). gender. polity press, cambridge uk. cott, c. (1998). structure and meaning in multidisciplinary teamwork. sociology of health and illness. 20(6): 848-873. fisher, m. and hutchison, p. (1997). involving rural women makes for powerful partners. proceedings of rural australia: towards 2000 conference. centre for rural social research, charles sturt university, wagga wagga nsw. hearn, a. b. and bange, m. p. (2002). siratac and cottonlogic: persevering with dsss in the australian cotton industry. agricultural systems. 74: 27-56. james, k (ed.) (1989). women in rural australia. university of queensland press, brisbane qld. james, k. (1990). women's decision-making in extended family farm businesses. key papers number 1 rural women. centre for rural welfare research, charles sturt university, wagga wagga nsw. kilpatrick, s., johns, s., murray-prior, r. & hart, d. (1999). managing farming: how farmers learn report 99/74. rural industries research and development corporation (rirdc), canberra act. lewis, t. (1998). evolution of farm management information systems. computers and electronics in agriculture. 19: 233-248. mackrell, d. 2005. „we work as a team really‟: gender homophily on australian cotton farms, issues in informing science and information technology. 2: 77-92. mackrell, d., von hellens, l. and nielsen, s. (2009). “harnessing diversity: individual differences in the use of farm management software”, proceedings of the 15th americas conference on information systems (amcis), san francisco, ca. patton, m. (2002). qualitative research and evaluation methods. sage publications, thousand oaks ca. poiner, g. (1990). the good old rule: gender and other power relationships in a rural community. sydney university press. sydney nsw rowe, f. (1997). women contributing to agriculture and resource management. proceedings of the national forum on agriculture and resource management. standing committee on agriculture and resource management. canberra act. australasian journal of information systems volume 16 number 2 2010 20 sachs, c. e. (1983). the invisible farmers: women in agricultural production. rowman & allanheld, totowa nj. schaffer, k. (1988). women and the bush: forces of desire in the australian cultural tradition. cambridge university press, cambridge uk. stewart, j. (1997). „i don't touch it without the cook here‟: a case study of gender and technology on family cotton farms. phd dissertation, dept of sociology and anthropology, university of queensland, brisbane qld. stewart, j. (2004). gender as a factor in the uptake and use of icts on family farms in southern queensland australia. gender, technology and development. 8(1): 97-117. the australian cottongrower. (2006). cotton yearbook 2006. toowoomba qld. walsham, g. (1995). interpretative case studies in is research: nature and method. operational research society. 4: 74-81. ajis vol. 7 no. 1 sept 1999 discourse strategies model: an initial phase for discovery of the factbased statements from descriptive text. bruce a. calway and ross smith swinburne at lilydale swinburne university of technology email: bcalwav@swin.edu.au abstract fact-based conceptual modelling approaches seek, as one goal, to express detail about a universe of discourse (uod) as elementary declarative statements, generalised from collections of data. a further resource available to the systems analyst, as the fact-modeller, is to discover fact-based statements from descriptive natural language information systems specifications. however, such extant formalised approaches as exist, that can assist the fact-based analyst to process textual resources, are incomplete. this paper proposes a discourse strategies model, for use as an initial process for the fact-based analyst processing descriptive text. the model is synthesised from extant linguistic and fact-based modelling approaches represented in literature. the model proposed is exercised using an example descriptive text. keywords fact-based modelling; discourse analysis. overview each textual resource is inherently compressed through linguistic referencing and ellipsis techniques. the application of these techniques by an author is aimed at making the resource readable, and within a particular universe of discourse (uod), communicable. the author of a text uses many linguistic methods for this process, and it might be argued that for the fact-based analysis of a textual resource to be more thorough the reversal of these processes should be of primary importance. the discourse strategies model (dsm), proposed as an approach in this paper, calls for all implicit lexical material to be made explicit within the textual resource and that the simple sentence, as a single clause, forms the largest sentential element. this approach typifies that of a data oriented representation of textual content. in that each clause is taken to represent a proposition and therefore a fact without regard for the clause's relative position to any other clause which exists within the text, nor regard for how the elements of texture function relative to each other. to achieve a clause based narrative of a textual resource involves the simplification or unpacking of discourse as written text into clause based descriptions of either simple (i.e. one clause sentences) or cosub-ordinate clause sentences. this work was first reported in (calway and sykes 1996) and is further detailed as a series of strategies in this paper. this study takes as its premise the concept that a simple clause is expressed through one proposition (itself being a set of atomic propositions) and expressing one fact as a declarative statement (cf. wintraecken 1990; sykes 1994; van dijk and kintsch 1983; clark and clark 1977). having unpacked and simplified a text then allows the analyst to approach the text to isolate and report propositions to be found in the clause based statements. several theories apply and are discussed as optional approaches (i.e. syntactic and semantic strategies (cf. clark and clark 1977; van dijk and kintsch 1983), and lexico-grammatical processes (cf. eggins 1994; winter 1982; halliday 1994). in combination with fact-based processes, syntactic and semantic processes could be incorporated as a production system and therefore be developed using an approach similar to augmented transition networks. the significant theoretical variance to previous research, which uses transformational approaches (cf. dunn 1992; meziane 1994; su 1988), is the use of psycholinguistic theories for proposition comprehension (van dijk kintsch 1983); functional linguistics for clause complex theories (cf. halliday 1994; winter 1982); and strategies theory as a processing model (cf. bever 1970; clark and clark 1977; van dijk and kintsch 1983). further, underlying the discourse strategies model is that "understanding sentences as part of a discourse is a different process from understanding sentences in isolation". also as yet there is no system able to parse arbitrary english text reliably", (van dijk and kintsch 1983:32) the linguistic processes developed in the following sections are a synthesis of strategies taken principally from van dijk and kintsch (1983), clark and clark (1977), eggins (1994), halliday et al. (1976, 1978, 1985, 1994), and martin (1992). in the next section a background to the research is established and a formal research question is proposed. arising from this the structure of the remainder of the paper emerges, as a discourse strategies model (dsm) is synthesised and subsequently executed. ajis vol. 7 no. 1 sept 1999 natural language models there are many examples of models of natural language and language use which take account of linguistic objects syntactically, semantically and pragmatically. a major assumption underpinning the model of discourse processing proposed in this paper is that information from each level interacts in and with each other. semantic knowledge does not explicitly follow or equate to a syntactic analysis of sentential surface structure. however, both semantic and syntactic structure would seem to be required for conceptual modelling. calway and sykes (1996) have shown that information can be lost when concentrating on text from a context free syntactic position. also, van dijk and kintsch (1983) among others have empirically shown that we do not break discourse into elementary pieces and parse them independently. chunks of discourse are processed relative to each other using a combination of structural and semantic strategies. conceptual modelling not only seeks structure but also meaning or conceptual intent. it is important that textual cohesion be retained as a step in the model because, unlike day to day discourse in which we might engage for the purpose of communication and comprehension, descriptive discourse places significant emphasis on the entities within the discourse and the role relationships which are extant. that is, this model places greater emphasis on the objects of direct interest by placing them at the focal point of a sentence (thematic, psychological subject) rather than concentrating for example on the actor undergoer (informal, logical subject) or subject/verb/object (grammatical subject) (eggins, 1994:141). derivational rule governed processes like those of transformational grammars will produce structural descriptions of a sentence by syntactic parsing rules (an example of this approach for fact modelling is dunn (1992)). however the process may be complex, and guaranteed success is only available if all the rules are correct and applied correctly. some linguists argue (cf. quirk et al. 198s) that language is an organism and therefore subject to evolutionary change, having a direct impact on the veracity of a derivational rule based approach. this point is further highlighted by the fact that no universal grammar is available for parsing natural language, as yet. quirk, et al. (1985) would not claim to have solved all issues of grammar, even though they have documented in excess of 1500 pages of detail (including a 110 page index). this highlights the extremes which must be considered if technology is employed in a prescriptive and deterministic way. the model proposed in the present work is rather like a hypothesis testing scenario, where no unique representation of text is assumed. this means that an initial working hypothesis may be understood but that this may be re-interpreted by further processing. strategies theory (cf. bever 1970) also takes account of the textual characteristics and the language user in terms of goals and world knowledge (a theme acknowledged by interaction of a systems analyst and domain expert in fact-based methods). the strategies detailed in this paper are an expression of the procedural knowledge about understanding discourse as information systems descriptions. however the work in no way forms the totality of possible processing strategies. it is plausible that strategies can be learnt and formalised, and therefore available to be automated as a production system shell. a textbase (as an outcome of the application of dsm) is generated as a collected declarative representation formed from the input discourse. the textbase will be defined in terms of elementary statements expressed as declarative clauses and cohesive relations between entities/entity-types. this approach is similar to those proposed by fillmore (1968), wintraecken (1990), and vendler (1967) for example. therefore the underpinning question which is investigated, in this paper, through the use of a literature survey and exercising of the proposed model using and annotated example is: can a discourse strategies model be synthesised from extant fact-based data modelling and linguistics literature which enables the development of elementary fact-based statements from descriptive textual information systems specifications? the balance of this paper addresses this question as follows: • review of the fact-based processes (section 3); • discussion of the foundations upon which the discourse strategies model is built (section 4, 5 & 6); • detailing of the discourse strategies model (section 7 and 8); and • exercising the proposed model as an annotated example of the application of the model (section 9 and 10). fact discovery using fact-based processes hirschheim et al. (1995:177-186) outline the fundamentals of conceptual modelling as typically represented in fact-based approaches. fact-based approaches argue that the meaning of language is determined by the way in ajis vol. 7 no. 1 sept 1999 which the elements of language correspond to entities and facts, as elements acting in some world. this accords with the goal of context free grammatical parsing through generative structures. currently generative grammar, as algorithmic rule-governed processes, is used for parsing natural language declarative sentences, however, this procedure depends on automation and a holistic context free grammar. for the proposed discourse strategies model it is important to note that the linguistic processes proposed are manually applicable, non-deterministic, open ended and highly context sensitive. as van dijk and kintsch (1983:31) suggest this does not preclude the rigour and objectivity required for a scientific theory or formalism. they remain open to automated strategies parsing by suggesting that both the rule based (context free) and strategy based (context sensitive) models can be implemented as an augmented transition network, currently a preferred parsing approach for context free grammars (cf. wood 1970; wanner and maratsos 1977; dunn 1992; kaplan 1972; meziane 1994). the issue of context free versus context sensitive parsing offers a significant theoretical dichotomy that remains unresolved in conceptual modelling schools of thought (hirschheim et al. 1995:198ff). it is not resolved or advanced further in this paper, which draws on existing theoretical assumptions of the fact-based school. when considering sentence parsing approaches, understanding sentences as part of a textual discourse is different from understanding sentences in isolation from one another. while studying sentences in isolation may tell us something, it may also mislead us. with textual discourse there may be an elementary clause that is part of a complex sentence which itself forms part of a sentence sequence. this observation requires contextual considerations to be observed in terms of uod coherence, texture and reference normalcy. the proposed discourse strategies model does not seek a 100 per cent grammatical model of sentence analysis, but rather a normative approach which the fact-based analyst might plausibly apply to assist fact discovery when dealing with information systems specifications as descriptive text. further, the model proposed does not deliver elementary facts or fact-type sentences sufficient for the immediate application of data base design approaches. the proposed model could best be described as providing an induced proposition set and consequent declarative sentence narrative. these statements could be used as input to conceptual modelling methods, where such generalised facts would be expressed or verified using specific uod examples, sufficient that a data base designer would be sure of the accuracy of a deduced design framework. wintraecken (1990:41-104), when describing the niam information analysis method, offers the theoretical foundation for using natural language on which this dissertation also draws. this is substantially the same as that of clark and clark (1977) and van dijk and kintsch (1983), where propositions are offered as the representation of semantic detail expressed as a limited sentence type and syntax. once propositions as declarative statements are catalogued as a textbase then the application of a conceptual modelling formalism can follow. for example, one could apply conceptual schema design process (csdp) steps 1 and 2 from the niam conceptual schema design procedures of nijssen and halpin (1989) to the textbase generated from the application of the discourse strategies approach. a fact-based information model is stated as expressing entities and their relationships, which occur within a specified uod. this being the case, there are three indicators to start the process strategies framework for fact discovery and declarative statements development: • entities, entity-types; • relationships between entities, fact-types; and • a specified uod for the context of the previous elements. there are a number of general processes which are defined for a fact-based analysis of discourse. burg and van de riet (1996) have stated that the starting process is to: 1. transform the user sentences into elementary sentences; 2. find common nouns in the elementary sentences (as entity-types); 3. find transitive verbs in the elementary sentences (as actions); 4. identify adjectives (as attributes); and 5. identify adverbs (as attribute relationships). similarly edmond (1992:220) suggests that the fact-based analysis stages should be: 1. uncover the relevant entity-types and the fact-types that join them; 2. look for any uniqueness constraints involved in each fact-type; and 3. construct record types by merging fact-types, where appropriate. however these approaches rely upon a sentence-by-sentence analysis with no regard in the first instant to inter textual detail. ajis vol. 7 no. 1 sept 1999 strategies theory in addressing alternative theoretical foundations psycholinguistic approaches as proposition comprehension strategies form a further theoretical focus for the present research, for comprehension of discourse. they were introduced by bever (1970), extensively developed and experimentally tested by clark and clark (1977) and form the basis of strategies for discourse comprehension of van dijk and kintsch (1983). reasons for advancing a strategies based model were that rule based (a term used by van dijk which is different in intent from earlier references to the term rule-based used by hirschheim et al.) systems which linguists use to parse sentences were implausible as psychological process models. semantic proposition comprehension strategies best emulate the heuristics of the intuitive fact-based analysis approach (the subject of a further paper by calway, yet to be published). the present research suggests that fact-based analysts using intuitive heuristics to discover facts in textual discourse actually use psychological processes. secondly, the calculation resource demanded for a rulegoverned grammatical parsing simply exceeds human processing limitations, as demonstrated by noble (1988). moreover, as stated earlier, a definitive grammar of english does not exist and therefore context free grammar parsers remain incomplete. dunn (1992), and meziane (1994), et al., have shown the complexity of attempting to implement context free grammars through automated augmented transition networks. van dijk and kintsch (1983:28) suggest that "strategies were simpler". as they say, strategies do not guarantee the right results, but it is plausible that people can, and in fact do, parse sentences on the basis of strategies. what then is the nub of the psycholinguistic approach as a strategy? it is using the proposition as the basis for parsing, as demonstrated by bever (1970), clark and clark (1977) and confirmed in van dijk and kintsch (1983) where they concluded that evidence for the psychological reality of prepositional units is overwhelming. howard (1983:304) commented that there is a failure in the derivational theory, due to complexity, which indicates that such approaches have not worked. howard continues, "... psycholinguists have tried a different approach. they argue that perhaps language comprehension is more accurately viewed as involving a set of heuristics, i.e. a set of rules of thumb that provide guidelines for uncovering the propositions encoded in the string of words." therefore processing strategy theory assumes that people use heuristics rather than algorithms during language comprehension. the derivational theory of complexity posits syntactic processes (e.g. transformational rules) that are largely independent of semantic strategies. whereas the processing strategy approach assumes that both syntactic cues (such as word order) and semantic information (such as word meaning) are used throughout the processing of a text (howard 1983:306,307) text and texture using proposition comprehension strategies only provides a partial approach to text processing for conceptual modelling. within descriptive text there could be several referent and cohesive elements/items that are not explicit in terms of lexical labels. to emphasise these items from a textual rather than a sentential perspective, the proposed model also draws on strategies from functional linguistics disciplines (halliday 1994; eggins 1994; martin 1992) referencing elements contained in sentences act as textual elements to provide cohesion. (halliday and hasan 1976:1) state that text is "any passage (of language), spoken or written, of whatever length, that does form a unified whole". for any research to move from a single clause to a clause complex as a sentential structure, and further to sentence sequences, requires recognising or differentiating between text and non-text. to analyse discourse requires examination of the nature of texture. texture results from contextual coherence (coherence of register and genre) and cohesion (internal semantic ties through which parts of the text depend for their interpretation or comprehension on other parts (eggins 1994:95)). eggins suggests (1994:95-109) that text can exist as a single clause or multiples of clauses and depends on texture created by: • referencing endophoric (retrieved from within the text) and exophoric (retrieved from shared immediate context); • lexical relations how lexical items (nouns, verbs, adjectives) and event sequences are used to relate the text consistently to its focus; and • conjunctive relations how the author creates and expresses logical relationships between the parts of text (elaboration, extension, enhancement). ajis vol. 7 no. 1 sept 1999 elementary facts, and formal facts (as rules or fact-type sentences) are to be expressed as simple declarative clauses and constraints which act to modify the relationships expressed in and between entities. the strategies envisaged for expressing these items as a fact oriented resource include sentence unpacking and simplification by splitting clause complexes and recording statements expressing the main, property, and constraint propositions. strategies selection proposition development the strategies framework provides for specific selection and application of processes which underpin the development of the discourse strategies model for use by a fact-based analyst desiring a more methodical approach to declarative statement discovery from a textual resource. central to this position is that the intuitive and documented processes for fact discovery displayed, by fact-based analysts when using a textual information systems description, were in fact reflected in psycholinguistic discourse comprehension processes for discourse analysis. the starting point for developing strategies is the definition of the relationships between clausal or sentential structures and consequently prepositional structures. as noted, a schema is given by van dijk and kintsch (1983:120) and wintraecken (1990:42-43) where one simple clause corresponds to one proposition, that proposition representing one fact which is the referent of the clause in some possible world. this is the elementary level; above this level exists the clause complex that will reflect a composite schema (of clauses, propositions and facts), i.e. one schema may be coordinated or subordinated with respect to another schema. van dijk and kintsch (1983) gives the example: the professor hired an assistant who had written a dissertation on discourse comprehension. this sentence is said to denote one fact, i.e. the action of hiring somebody who has certain properties. this fact is complex due to the property of one of the arguments being described in terms of another (previous) fact. the sentence structure suggests the embedded fact has no independent function other than to specify a (main) fact. to illustrate this, a further example might be: a car is of a particular model and is given a serial number by its manufacturer that is unique among the cars made by that manufacturer. here the main clause and therefore main proposition and main fact is that the manufacturer gives a serial number to a car. there also exists one coordinated clause and therefore fact (1 below) and one subordinated clause and therefore fact (2 below), as follows: 1. a car is of a particular model made by a manufacturer, 2. a serial number is unique among the cars made by the manufacturer. these three facts (i.e. one main and two relative facts) are sufficient for a fact modeller to begin the conceptual modelling process as outlined by authors such as wintraecken, halpin, and edmond, et al. this small exercise points to the strategies required above the clause or proposition level, i.e.: • how to discover the proposition hierarchy in the clause complex; and • the importance of knowing how semantic information is placed in, or distributed across, several clause complexes. before stating the processing strategies for discovery of the main and relative clauses there is a question of clause independence, where the sentence as a clause complex is re-phrased into individual clauses. this rephrasing process creates a situation where each clause identified contains a main proposition and therefore denotes an independent fact. the proposition schemata for such a structure could then be linked textually by degrees of closeness (van dijk and kintsch 1983:122-123). a sentence or clause complex is the highest level of grammatical and therefore syntactic structure. however in text, propositions as semantic structure extend beyond those bounds. the semantic relation can be achieved through text forming resources such as conjunctive relations, i.e. relations between messages or between larger complexes. there are two conjunctives, external (ideational) which are seen in a series of events or as a narrative, and internal (interpersonal), setting semantic relations between steps in an argument. there are three levels of cohesion that will require analysis: ajis vol. 7 no. 1 sept 1999 • cohesion between entities within a clause; • cohesion between entities within a clause complex; and • cohesion between entities within a sentence complex (text). the grammar of a clause accounts for how various parts participate or function within a clause. this can be extended to include how clauses function together. once the textual resource moves beyond a single sentence then traditional grammars offer little support. a fourth level of cohesion, that is not dealt with in this work but which forms the basis of considerable debate in information systems development research, is that of context free and context sensitive text best expressed by hirschheim et al. (1995) in their book. this traces the various philosophical and practical developments in data modelling. a distinction must be drawn between what is the actual outcome from a textual simplification or reconstruction of a textual resource and that of a data driven intuitive approach. if, as is the case with the present research, the input to the process is descriptive text, then consideration must be maintained as to the information content through cohesion in the text, as distinct from the data structures which neglect the cohesion that may exist between clause and sentence complexes. this is exemplified when sentences are split on a coordinating conjunction boundary, as suggested by falkenberg (1986:6.1). hirschheim et al. (1995:26,27) make little distinction between information, conceptual and data modelling, preferring their definition of the later to be inclusive of the former. they do state in passing that it is the information or conceptual level that deals only with linguistic modelling. it is this which forms the basis of the output from the discourse strategies model approach. therefore it is the information model as a textbase that forms the input to the modelling formalism of fact-based approaches. a discourse strategies model there have been several strategy groupings identified, in this paper, for the fact discovery process of a factbased analyst: 1. clause and therefore declarative statement discovery strategies sufficient to identify inter and intra textual cohesion elements within the text; 2. sentence clarification and simplification strategies which remove all implicit referencing and ellipsis by substitution; and 3. declarative statement development strategies for identification and collection of expressed elementary statements. the expression of an elementary fact is described in terms of a simple declarative sentence as a single clause. the availability of descriptive discourse for expressing information systems structure is common, however as indicated earlier, documented heuristics used by the fact-based analyst for comprehension and modelling of fact statements are lacking. therefore, this section begins with strategies, which it is believed will help disclose the clausal and prepositional content of a descriptive textual discourse. it is plausible that discourse simplification should be considered as the starting point for expressing propositions within and across the sentential content of text. each of the strategies discussed below, individually or when combined, allow the fact-based analyst a plausible model for analysing textual material for the development of a collection of statements which express the uod information system specification. the previous sections have indicated that the starting point for fact discovery from a descriptive text is to make explicit how the text is implicitly structured as constituent parts. an important component of this approach is to understand that although comprehension, and to a degree interpretation, are important, these are only a means to an end, which is a detailed set of declarative statements made available for the conceptual modeller. as with any text it is shown that considerable content is sacrificed through methods such as ellipsis and substitution, endophoric and exophoric referencing, etc. for a more complete treatment of propositions, and therefore facts, as much content as possible is retrieved prior to the declarative statement textbase development. the desired sentence structure for the proposed textbase is as: • indicative affirmative clauses (i.e. declarative); and • indicative conditional clauses. each clause will be asserted as true (cf. nijssen and halpin 1989:16). ajis vol. 7 no. 1 sept 1999 also, all property facts (i.e. facts which have only an attribute relationship expressed (e.g. a car is of a particular model)) should be tested for an associated activity fact to which the property ascribes. the identified detail may not be relevant to the conceptual modeller but does give opportunity to discover how a state of affairs came about. further, proper names or data examples used in sentential material are to be associated with their entitytype and entity-label detail, as prepositional statements, as this forms an integral part of developing fact sentences of the kind expressed in a fact-based formalism (e.g. niam): e.g. john is the first-name of a person person with first name john. or within descriptive text it is unlikely for the analyst to observe sufficient (or any) examples sufficient for a full understanding of all data constraints, unless they are explicitly mentioned within the text, as is often the case: e.g. the maximum number of employees per department is 10. textual resources, when considered conceptually, can be handled in one of two ways to deliver the prepositional structure as a semantic representation of the text. one is to take the text and represent the semantic detail as a narrative of elementary clauses and therefore elementary propositions which express a one-to-one fact sentence relationship. the other is to deal with the text as a set of sentences (clause complexes) which will contain one main proposition per sentence and therefore one main fact as a proposition hierarchy. the first requires that each clause be identified and made independent of others within the text, whereas the second, would suggest one main clause which is supported by statements which relate to, or qualify, the main clause. it is the first option which is most like the outcome proposition statement set of the fact-based approach. the proposition comprehension strategies discussed in calway (1998:116-146) were shown to take content words and create a series of elementary propositions without regard to any implicit context or hierarchy within the text. also, if an analyst were to look for one main clause, and therefore one main fact, that clause would be the one which expresses the main finite predicate structure. all other clauses would be considered relative, in that they are qualifying the main clause. this is exemplified in the functional approach to mood analysis as an understanding of sentences eggins (1994). as a proof-of-concept it is necessary to express the discourse processing findings of the present research as a synthesised theoretical strategies model which can be aligned to existing fact-based analysts heuristics (to be reported in calway, yet to be published). the discourse strategies model will be exercised subsequently in this paper, using the textual examples given by wintraecken (1990:41). as a future work, the model could be automated and tested in a longitudinal study as to the process and product of the theorised approach to analysing descriptive text. descriptive text jimplifsimplify split coordination re-phrase sub-ordination record clause's as a narrative textbase figure 1 dsm text processing sequence the discourse strategies model described in the following sections has a series of steps, with each step having a set of discourse strategies to apply for various textual problems as they may be encountered by the fact-based analyst. the text processing sequence as shown in figure 1 includes: splitting where compound sentences (i.e. co-ordinated clauses) are split at points where conjunctions are operative and clause boundaries are identified; ajis vol. 7 no. 1 sept 1999 • unpacking replacement of linguistic items of referent, substitution and ellipsis for each simplified sentence (clause complex); • re-phrasing taking each clause and phrase complex and identifying clause/phrase boundaries. the contents are re-phrased into independent clauses as a narrative where the above steps can be re-applied; and • extraction extracting the content words, identifying the main predicate and developing an elementary proposition as a declarative simple sentence. a flow diagram (figure 2) provides a step by step indication of the strategies that might be applied, in what order, and what decisions might be taken during the analysis process. the process steps need not be applied in a linear form; it is more important to choose the appropriate processing strategies than to attempt a prescriptive hierarchy of actions. the dsm strategies are summarised in the following sections (a detailed analysis of the dsm strategies development can be found in calway (1998). input descriptive text, as a cohesively linked set of sentences for a uod information systems specification 1 select each sentence in sequence top to bottom and apply (dsmstrategy 1) label each sentence and part (e.g. sla,b,s2, etc.) for ease of referencing if a sentence is a single clause and no substitutions are required (go to step 5) if a sentence is a single clause with substitutions required(go to step 4) simplify clause complexes by re-phrasing constituent analysis (dsmstrategy 2) replace substitution and referent items (dsm-strategy 3) identify content words and express these as simple declarative statements (dsm-strategy 4) identify and express implicit clause/s for textual relationships (dsmstrategy 5) place statements as a set of tagged declarative expressions into a textbase figure 2 model flow diagram (note: strategy numbers shown i.e. strategy 1, 2, etc. refer to specific, related strategies discussed later in this paper) 10 ajis vol. 7 no. 1 sept 1999 elementary statement development there are at least two approaches, which can be considered when developing elementary statements from unconstrained textual discourse. of the two approaches it is the first which is taken as constituting the discourse strategies model. the second approach is more likely to replicate the current fact-based text processing heuristics of the fact-based analyst. this second approach could form the basis of a further research project that could reveal in more detail specific text processing heuristics or conditional approaches. approach one • sentential splitting and unpacking the text is analysed for opportunities to split compound sentences into less complex structures. this is achieved by observing conjunctive items which join clauses together (e.g. and, but, or, that). divide the sentence into two and discard the conjunction, making each new part into an independent clause or clause complex. unpacking of text means replacing significant contextual items in each new or existing clause complex so that each new structure can make an independent sense. this is achieved by replacing non lexical items, substituted items and ellipsis items within the structure. each unpacked clause or clause complex may offer new opportunity for further splitting before moving to declarative statement development, although, it is not mandatory to split clause complexes into smaller parts. • extracting propositions, facts, rules, constraints once the text is developed so that clause complexes are simplified, then elementary declarative statement can be extracted. it will be noticed that each clause or clause complex will have one main proposition and possibly one or more relative propositions as attributes or constraints on the entity/entities in the main fact. as one approach the content words (i.e. nouns and predicate) are selected and written as a declarative simple clause based sentence. although declarative statements will be extracted for every clause in the text initially, this does not mean that they must be used in any conceptual schema modelling process. approach two a second approach is to identify each main clause at the surface level of a sentence and dissect each clause identified. this approach accepts the main clause in-situ, with the development of propositions emulating the semantic strategies approach expressed earlier. such an approach relies on the ability of the fact-based analyst to identify the main proposition through the content words. this approach is more likely to replicate the current text processing heuristics of the fact-based analyst (research reported in the phd dissertation (calway, 1998)). approach one sentence splitting and unpacking strategies before any attempt is made to process textual material it is necessary to determine a starting point. within the literature reviewed there is no explicit statement as to whether a top down approach should be taken, when processing text. the use of a top down approach is implicit in the work of eggins 1994; wintraecken 1990; and van dijk and kintsch 1983); and therefore, for the present research, a top down, sentence by sentence sequential approach has been taken as a basis for the initial research. the goal of the following strategies is to identify how the various parts of a sentence are functioning. the text, and therefore a sentence, can be split into components and have all implicit detail made explicit. two main conjunctive relationships exist which can be identified and allow sentence simplification: namely co-ordination and subordination. coordination is expressed as follows: dsm-strategy 1 whenever co-ordinated clauses are detected joined by a coordinating conjunction, then begin a new clause and remove the conjunction. co-ordination is where co-equal clauses are joined by a co-ordinating conjunction (and, or, but, nor) in order to create a single sentence. this strategy reflects the observation that conjunctions are used to connect ideas or to remove repetition between sentences in a textual discourse. sub-ordination is expressed as follows: 11 ajis vol. 7 no. 1 sept 1999 dsm-strategy 2 whenever a sub-ordinate clause or clause component is detected; 1) expand the word or phrase and add a co-ordinating conjunction to produce a connected clause sentence which can then be split at the co-ordinating conjunction or (2) move the sub-ordinate clause to the head of the clause complex and separate the clauses as if coordinated (refer dsm-strategy i.) it is possible to simplify a sentence structure by taking account of relative, subordinating and complement elements of a sentence. often these elements can be made independent through re-expressing the clauses or clause constituents as co-ordinated clauses which can then be split further, as follows: dsm-strategy 2a use the first word (or major constituent) of a clause to identify the function of that clause in the current sentence. dsm-strategy 2b assume the first clause to be a main clause unless it is marked at, or prior to, the main verb as something other than a main clause. this strategy also distinguishes main from qualifying clauses by noting the clause markers. additional optional syntactic strategies include: strategy 2c whenever a junction word is used, begin a new constituent larger than one word. function words (e.g. pronouns, determiners, qualifiers, propositions) are easy to detect and very reliable cues to constituent structure. second, function words signal the type of constituent. strategy 2d after identifying the beginning of a constituent, look for content words appropriate to that type of constituent. as with strategy 2c, there is a correlation between the variety of function words with those of content words (e.g. nouns, verbs, adjectives, adverbs). most content words cannot be identified unambiguously and are therefore dependent on the use of function words for disambiguation. although strategies 2c and 2d can clarify the constituents of a clause there is a requirement for additional strategies to identify the clause and how it functions. (halliday 1994; eggins 1994; and martin 1992), when developing functional linguistic concepts, suggest that the mood constituent of a clause identifies what the function of a clause complement will be, as follows: dsm-strategy 3 replace all implicit lexical details for the text. (ie. identify all ellipsis, substitutions and referent co-ordination and subordination (endophoric/exophoric referencing). the application of this strategy allows the analyst to unpack the implicit entity and role relationship substitutions encapsulated textually within the discourse. sub-strategies include: dsm-strategy 3a when a pronoun is identified look for the referent to which it is associated and replace the pronoun with the referent. dsm-strategy 3b endophoric and exophoric referencing analyse the text for the referent and replace the reference with the referent lexical items. dsm-strategy 3c ellipsis analysis and substitutions analyse each sentence for ellipsis clauses or phrases expanding the sentence with the ellipses clause or phrase. approach one strategies for extracting propositions, facts, rules, constraints the goal of the following strategies is to determine how each sentence was meant to be utilised. this is achieved by following two working principles (van dijk and kintsch 1983; clark and clark 1977:72-85): • reality concerned with the substance; and • co-operative concerned with the way ideas are expressed. 12 ajis vol. 7 no. 1 sept 1999 as stated earlier, for fact-based approaches, there is the assumption that the writer of the information systems description means what is stated and that sufficient uod knowledge is available to interpret sentences. the following strategy is suggested: dsm-strategy 4 using content words alone, build declarative statements that make sense using the subject/finite/predicated/complement/adjunct structure. within the sentence which has been subjected to strategies 1 to 3, there are contentive (content words) items which constitute the substance of the sentence and therefore the substance of the propositions. it cannot be stated that a declarative statement at this point constitutes an elementary fact until further conceptual schema design processes have been undertaken. one should apply the functional analysis processes to determine the mood constituents and therefore assist in determining the subject, finite, predicate, complement and adjunct structure. this allows consequent parsing of these elements into elementary (declarative) statements. sub-strategies include: dsm-strategy 4a identify the subject (psychological) and finite mood items for a sentence or clause. dsm-strategy 4b identify the predicate, complementary and adjunct mood items for a sentence or clause. these processes (strategies 1-4) in large part satisfy the stages of fact-based analysis suggested by burg and van de riet (1996), and edmond (1992), detailed earlier in this paper. to identify and express, implicitly, clauses for textual relationships the following strategy is suggested: dsm-strategy 5 develop elementary statements to represent the indicative relationship which exists cohesively between clauses/entities(types) within the text. sub-strategies include: dsm-strategy 5a whenever a proper name or literal (i.e. a name or literal given to represent one real or imagined person, place, or thing) is identified also discover the referent label and entity type for that item(entity). there are two processes required as part of this sub-strategy: • identify the lexical noun, as entity type and label type, for the proper name or variable of the lexical noun. (e.g. person with last-name, johnson) • record a statement which expresses this proposition. dsm-strategy 5b whenever a thematic connection is implicit within a discourse create a linking proposition and tag the proposition as implicit. the most common text construction indicates some relationship at the type level for entities and then leaves the connection between specific examples within the text as implicit thematic references. dsm-strategy 5c review the adjunct items for constraint and qualification phrases and create a proposition statement to express the detail. resultant textbase the outcome from the analysis is a collection of elementary statements as declarative representations. these statements represent both elementary and formal fact options, with referent constraints or qualifying propositions: • elementary facts in the textbase contain a declarative statement with one or more of the identified entities being a literal or proper noun as an instance; • formal facts in the textbase represents declarative statements identifying non instance entities as label or type entities; 13 ajis vol. 7 no. 1 sept 1999 • fact constraints as qualifier or conditional statements operating on an entity and expressed as an indicative conditional statement; and • implicit facts being those statements derived when checking for entity label and type detail expressed for an identified entity as a literal or proper noun. all detail is expressed as declarative clause based simple statements that can be asserted as true for a given uod. however, in no way should they be taken as other than indicative of the possible statements for a conceptual model until such time as a full set of specific examples and formalisms have been applied as a part of any fact-based conceptual modelling approach. equally, as stated earlier, the elementary statements expressed within the textbase should not be equated to elementary facts as expressed by any fact-based formalism (e.g. niam). approach two strategies although the use of a psychological model, where propositions are discovered and recorded as a hierarchy, is a plausible model there have been no strategies synthesised or tested in this dissertation. however, strategies could well be incorporated into further research which tests the veracity of one model synthesis over another (i.e. narrative vis-avis hierarchy). annotated example the following is a step by step rendering of declarative detail using the discourse strategies model as described above. as such, this example should facilitate understanding of the dsm synthesised in earlier sections. the example text is taken from wintraecken (1990:4 1) which states: "in this company, there are employees and departments. employees have last names and departments have department names. employees can only work for one department. johnson and peterson work for the purchasing department and king works for the personnel department. a department may not consist of more than 10 employees." a step by step rendering of declarative detail using the discourse strategies model could proceed as follows: • apply strategy 1 to the body of text as far as possible, splitting sentences at the co-ordinating conjunction. in this company, there are employees and ... departments. employees have last names and departments have department names. employees can only work for one department. johnson and peterson work for the purchasing department and king works for the personnel department. a department may not consist of more than 10 employees. • strategy 2 allows for splitting of clause complexes, however there are no obvious candidate sentences in the analysis. • apply strategy 3 as the approach to filling the void left by splitting the sentence and number the sentences as a reference back to the originating text. also it is practical to sequence the new text structure for further reference and connection with the textbase. 14 ajis vol. 7 no. 1 sept 1999 sla in this company, there are employees. sib in this company, there are departments. s2a employees have last names. s2b departments have department names. s3 employees can only work for one department. s4a johnson works for the purchasing department. s4b peterson works for the purchasing department. s4c king works for the personnel department. s5 a department may not consist of more than 10 employees. • apply strategy 4 to the body of text in order to discover content words (subject. finite. predicate, complement, adjunct). s1 a in this company, there are employees. sib in this company, there are departments. s2a employees have last-names. s2b departments have department-names. s3 employees work for one department. s4a johnson works for purchasing department. s4b peterson works for purchasing department. s4c king works for personnel department. s5 department consists of<= 10 employees. the first and main proposition can be determined and expressed as an indicative (declarative) clause structure. these clauses represent the main elementary propositions and therefore facts in a textbase. for the purpose of comparison with the set of propositions offered by wintraecken (1990:43) some re-phrasing and article elimination have been activated on the above declarative statements: pi company has employees. p2 company has departments. p3 employee has last-name. p4 department has department p5 employee works for (one) department. p5a employee works for department. p5b maximum 1 department per employee. p6 johnson works for purchasing department. p7 peterson works for purchasing department. p8 king works for personnel department. p9 department consists of <= 10 employees. p9a department consists of employees. p9b maximum 10 employees per department. • using strategy 5c produces a representative set of elementary statements for sub-ordinate clause/s. these propositions, when expressed as indicative clauses, represent property and constraint facts and are represented in the textbase (refer to the propositions 5 and 9 above for an indication of what could exist). in some statements data is present as proper names, etc. where these are identified they also should be represented by a fact statement which expresses the example, its entity-label, and entity-type where available by induction of plausible propositions. plo peterson is the last-name of employee pi 1 johnson is the last-name of employee pi2 king is the last-name of employee p 13 purchasing department is the department-name of department p!4 personnel department is the department-name of department 15 ajis vol. 7 no. 1 sept 1999 having completed the discovery process from the textual resource it is important to pass the outcome textbase on to the conceptual modelling processes to further be developed using example data which operate within the uod and which form the fundamental input into fact-based conceptual schema formalism. annotated example comparative analysis table 1 is a comparative analysis of the results for the proposition set offered by wintraecken (1990:43) and the demonstration study argued above: (wintraecken 1990) proposition set demonstration equivalent wl there are employees. w2 there are departments. w3 employees work for departments. w4 an employee may only work for one department. w5 there are last names, (this would be considered an atomic proposition and part of (w7) w6there are department names. (as for w5 but part of w11) w7employees are denoted by their last names. wsjohnson is the last name of a certain employee. w9peterson is the last name of a certain employee. w10 king is the last name of a certain employee. wl 1 departments are referred to by department names. w12 purchasing is the department name of a certain department. w13 personnel is the department name of a certain department. w14 johnson works for the purchasing department. w15 peterson works for the purchasing department. w16 king works for the purchasing department. w17 a department may not contain more than 10 employees. p9a,b study pi p2 p5a p5b therefore p3 pll p10 p!2 p4 p13 p!4 p6 p7 p8 table 1 demonstration study statement comparison it can be seen from this example that a simple textual example can be easily developed into an independent set of declarative statements each expressing a single fact. however, it can also be seen that wintraecken has not identified fully the content or constraints operating. this is observed by the lack of reference to this company' in wl and w2; and the inclusion of atomic propositions w5 and w6. as a fact-based conceptual modelling approach, wintraecken may be eliminating non relevant data items (e.g. where only one value is valid for a data item). however, as has been argued earlier, design issues should not be considered until after a full analysis of the text has taken place. it can only be assumed that w3 is meant to represent the relationship of employee and department (p5a) and the inverse (p9a), and that w17 expresses a constraint only. summary in this paper, the fact-based processes have been reviewed, and the foundations upon which one might build a discourse strategies model have been summarised. building upon this base, a discourse strategies model has been synthesised and illustrated with an annotated example. as a consequence, the question posed at the commencement of this paper (i.e. can a discourse strategies model be synthesised from extant knowledge which enables this development of elementary statements from descriptive textual information systems specification?) is answered in the affirmative. this paper has been limited to reporting the discourse strategy model synthesis. further aspects of the wider research project, of which this is part, form the content of a phd dissertation (calway 1998), and further investigation of the model, and its application, is the subject of continuing research. references abbott, r. (1983) "program design by informal english descriptions", communications of the acm, vol. 26 no. 11, pp. 882-894 allen, j. (1987) natural language understanding, benjamin cummings publishing company. 16 ajis vol. 7 no. 1 sept 1999 sever, t. g. (1970) "the cognitive basis for linguistic structures", in hayes, j. r. (ed.), cognition and the development of language, john wiley and sons. beverage, r. (1981) a structured methodology to ascertain requirements for computer based decision support systems, phd thesis, george washington university, usa. bolinger, d., and sears, d. a. (1981) aspects of language, 3rd edition, harcourt brace jovanovich publishers. burg, j.f.m., and van de rffit, r.p. (1996) "analyzing informal requirements specifications: a first step towards conceptual modelling", in van de riet, r.p. et al. (eds.), application of natural language to information systems, ios press. calway, b. a. (1998) a discourse analysis and strategies based model for discovering facts from natural language information systems specifications, phd thesis swinburne university of technology. (submitted) calway, b. a., and sykes, j. a. (1996) "an application of discourse analysis in conceptual modelling", australian journal of information systems, vol. 3 no. 2, pp. 1019. chen, p. p. (1983) "english sentence structure and entity relationship diagrams", information sciences, no. 29, pp. 127-149. chomsky, n. (1956) "three models for the description of language", ire transactions, no. 2, pp. 113124. chomsky, n. (1965) aspects of the theory of syntax, mit press. clark, h. h., and clark, e. v. (1977) psychology and language an introduction to psycholinguistics, harcourt brace jovanovich publishers. darke, p., and shanks, g. (1994) defining systems requirements a critical assessment of the niam conceptual schema design procedure, working paper 8/94, department of information systems, monash university, australia. van djjk, t. a., and kintsch, w. (1983) strategies of discourse comprehension, academic press. dunn, l. j. (1992) relational database engineering a natural language driven approach, phd thesis, university of queensland, australia. edmond, d. (1992) information modelling specification and implementation, prentice hall. eggins, s. (1994) an introduction to systemic functional linguistics, pinter publishing. falkenberg, e. d. (1976) "concepts for information modelling", in nijssen, g. m. (ed.), modelling in data base management systems, north-holland publishing company. falkenberg, e. d. (1986) data base and information systems, lecture notes. fillmore, c. j. (1968) "the case for case", in bach, e., and harms, r. t. (eds.), universals in linguistic theory, holt rinehart and winston inc., pp. 1-88. flavin, m. (1981) fundamental concepts of information modelling, yourdon press. fodor, j. a., bever, t. g. and garpett, m.f. (1974) the psychology of language: an introduction to psycholinguistics and generative grammar, mcgraw hill, van grtethaysen, j. j. (ed.) (1982) concepts and terminology for the conceptual schema and the information base, iso/tc97/sc5-n695, ansi. halliday, m. a. k. (1978) language as a social semiotic the social interpretation of language and meaning, edward arnold. halliday, m. a. k. (1994) introduction to functional grammar, 2nd edition, edward arnold. halliday, m. a. k., and hasan, r. (1976) cohesion in english, longman publishing. halliday, m. a. k., and hasan, r. (1993)language, context and text aspects of language in a social-semiotic perspective, deakin university, australia, (reprint from 1985). halpin, t. a. (1995)conceptual schema and relational database design, 2nd edition, prentice hall. fflrschheim, r., klein h. k., and lyytinen, k. (1995)information systems development and data modelling, cambridge university press. hoffman, r.r. (1987) "the problem of extracting the knowledge of experts from the perspective of experimental psychology", ai magazine, vol. 8 no. 2, pp. 53-67. hoffman, r.r., shadbolt, n.r., burton, a.m., and klien, g.a. (1995) "eliciting knowledge from experts: a methodological analysis", organization behaviour and human decision processes, vol. 62 no. 2, pp. 129-158. howard, d. v. (1983) cognitive psychology memory, language and thought, macmillan. iso/tr 9007 (1987) information processing systems concepts and terminology for the conceptual schema and the information base, international organization for standardization. technical report udc 681-3-02, published 1987-07-01. kaplan, r. w. (1972) "augmented transition networks as psychological models of sentence comprehension", artificial intelligence, vol. 3, pp. 77-100. 17 ajis vol. 7 no. 1 sept 1999 kerl, s. a (1985) common-school grammar of the english language (1878), scholars' facsimiles and reprints, delmar, ny. kimball, j. (1973) "seven principles of surface structure parsing in natural language", cognition, vol. 2, pp. 15-47. martin, j. (1992) english text system and structure, john benjamin publishers. meziane, f. (1994) from english to formal specifications, phd thesis, university of salford, uk. moulin, b., and creasy, p. (1992) "extending the conceptual graph approach for data conceptual modelling", data and knowledge engineering, vol. 8, pp. 223-248. nesfield, j. c. (1936) outline of english grammar, macmillan and co. nijssen, g. m., and halpin, t. a. (1989) conceptual schema and relational database design a fact oriented approach, prentice hall. noble, h. m. (1988) natural language processing, blackwell scientific publications. quirk, r., greenbaum, s., leech, g., and svartvik, j. a (1985) comprehensive grammar of the english language, longman. reed, a., and k-ellogg, b. (1886) higher lessons in english 1886, scholars' facsimiles and reprints, delmar, ny. van de riet, r. p., and meersman, r. a. (eds.) (1992) linguistic instruments in knowledge engineering, elsevier science publishers. riloff, e., and lehnert, w. (1994) "information extraction as a base for high-precision text classification", acm transactions on information systems, vol. 12 no. 3, pp. 296-333. rothe, h. j., timpe, k. p., and warning, j. (1989) "psychological methods of knowledge elicitation within the domain of mechanical engineering", in klk, f., streitz, n. a., and waern, y. (eds.) man computer interaction research macinterii, north holland publishing company. stamper, r. k. (1992) "language and computing in organised behaviour", in van de reet, r. p., and meersman, r. a. (eds.), linguistic* instruments in knowledge engineering, elsevier science publishers. su, m. (1988) the design of a natural language modelling environment for manufacturing system simulation, phd thesis, the university of iowa, usa. sykes, j. a. (1994) english grammar as a sentence model for conceptual modelling using niam (working paper 10/94), centre for information systems research, swinburne university of technology, australia. vadera, s., and meziane, f. (1994) "from english to formal specification", the computer journal, vol. 37 no. 9, pp. 753-763. vendler, z. (1967) linguistics in philosophy, ithaca cornell university press. wanner, e., and maratsos, m. (1977) "an atn approach to comprehension" in bresnan, j., and hale, m. (eds.), linguistic theory and psychological reality, mit press. winter, e. 0. (1982) towards a contextual grammar of english, allen and unwin. westtraecken, j. j. v. r. (1990) the niam information analysis method theory and practice, kluwer academic publishers. wood, d.d. (1993) "process-tracing methods for the study of cognition outside the experimental psychology laboratory", in klffin, g., orasanu, j., colderwood, r. and zsambok, e. (eds.), decision making in action: models and methods. norwood, pp. 228-251. wood, w. a. (1970) "transition network grammars for natural language analysis" communications of the acm, vol. 13, pp. 591-606. 18 information systems and c orporate memory: design for staff turn-over2 colin a. sharp & neil r. lewis school of accounting finance and management , flinders university, south australia abstract the "information age" is reliant upon the information skill-base of people and the storage and access of complete data. staff turn-over and "down-sizing" as well as the rate of organisational change places increased pressure on the designers of corporate information systems to keep up with the organizational demands. especially relevant are the "soft" aspects of the corporate information needs of organizations. the term "corporate memory" is introduced to explain the relatively informal collective and individual knowledge of employees gained through their experience and position in the organization. this paper elaborates the issues concerned with the "corporate soft data", and makes suggestions for research, as well as for planners of corporate information systems in organizations facing major down-sizing or related changes. it is concluded that there are a number of readily obtainable sources of soft data in the organization and these data should be analysed for inclusion in system development. if organizations are to become learning systems and not just keep repeating the same mistakes, they need to review all aspects of corporate memory, especially the soft data sources and losses. introduction staff turn-over, and "right-sizing" of organizations are commonplace in the public sector and private sector bureaucracies in economic down-turns. for example, the ceo of eastman kodak told the world management congress in one strategic change he had to "down-size" 15,000 of his employees from the company town, rochester, new york (samper, 1989). in australia almost 200,000 (or about 1 in every 8) jobs have been lost in the first two years of the current recession with a further wave of "down-sizing" occurring currently. this painful trend has been the subject of research by psychologists, concerned with "survivor sickness" (marks & mirvis, 1992), i.e., the stressful effects on employees of the uncertainty of change, as well as the anxiety of unemployment for those retrenched (e.g., cartwright & cooper, 1990, 1993) and the focus of management researchers concerned with the associated problems of productivity and organization effectiveness (cartwright & cooper, 1990; hunt, 1988). increasingly, the consequences of continual organizational change are becoming a matter for concern of information system managers. the computing and information systems are expected to substitute for personnel more than ever. information systems managers increasingly have to deal with the difficulties of qualitative information storage in these circumstances. this paper provides a review of some of the experiences of public sector organizations which have undergone major organizational changes, and some of the key issues concerning design of information storage in anticipation of such changes. the findings of the review suggest implications for design and implementation of preventative approaches. the organisational problems to be addressed in australia, the economic down-turn and government policies have led to the redundancy, retrenchment or redeployment of thousands of personnel from the public and private sectors. public sector "reforms" and cut-backs in state and commonwealth bureaucracies have led to many retrenchments from the public sector. indeed, according to a recent report from the economic planning advisory council (cited by colebatch, 1993), 38 state and federal government business enterprises cut about 50,000 jobs in two years. these unemployed are not merely from the ranks of the lower levels of the public sector, but since the abolition of the public service board in 1987, have included the ranks of the senior executive service (jacka, 1988). for example, according to the 2. an earlier version of this paper was presented to the third australian conference on information systems at the university of wollongong october 1992. we thank the participants for their encouraging comments. september 1993 75 former director of the senior executive service unit, harvey jacka (1988, p. 45), "the recent machinery of government changes have had a significant impact on the ses". the re-structuring of government departments from august 1987, "displaced" at least 125 ses officers within the following three months. even though the commission was said to regard these "ses staff [as] a valuable resource" it was prepared to "cut the ses by 90, that is, about 5 per cent." (jacka, 1988, p-45) in some ways the public sector provides an opportunity to examine the implications of these potential problems for design of information systems. this kind of staff loss and turn-over, as well as the economizing of managers, coupled with the technological improvements of the "information age"^, has led to an increasing reliance on automated quantitative management information systems (mis) at the expense of the human qualitative systems. in order to understand what can be done to address the problems of continuing organizational restructure and the role of information systems, it is important to examine the underlying concepts of the "information age". the two relevant components are the intellectual skill-base and the learning organization which theoretically enhance the quality of the organization's database. lessem (1990) sees the intellectual skill-base as an organization's most precious resource. it is the intellectual skill which is relevant to, and enhances the qualities of "wholeness", "interdependency" and "integration" of information in an organization. the second major component of the information age will be the "learning organization" (degeus, 1988; senge, 1990); the organization's ability to learn faster than its competitors. zuboff (1988) considers that learning and integration constitute the two most vital priorities in such a work-place. consequently it is the knowledge that people have, and how they use the database of information available to them, which are the keys to a learning organization and yet in "down-sizing" it is a diminishing resource with which information managers must cope. the key to these "soft systems" (checkland, 1981) problems associated with organizational restructuring are related to the concept of "total corporate memory". this concept is implicit within the analogies used to explain effective organizations, i.e., "cybernetics" (wiener 1961). components of corporate memory walsh and ungston (1991) have reviewed the literature on what has been called "organizational memory", in an attempt to provide a more coherent theory of the storage of information about an organization's history which can be brought to bear on decision making in the present and future. they recognise the concern about anthropomorphism in the reification of the analogy of the organization (and its memory) with the human brain or mind. nevertheless they attempt to investigate the locus of storage of the information in organizational memory; to examine how information is acquired, stored and retrieved from memory, and the manner in which the recalled information is able to be used in decision making and performance of the organization. also recently tsoukas (1991) has addressed the development of the organism analogy, and the brain analogy by continuing the work of beer (1972). the present paper is less ambitious than that by walsh and ungson (1991). here the authors attempt only to define the various components of data which could be the basis for information storage and retrieval, and to examine the implications of some staffing practices on the potential for retention of that data. in this endeavour we focus on some of the lessons identifiable in the experience of public sector organizations in australia. the derivation of organizational memory is uncertain, but the origin of the concept of information storage in systems has been implicit in systems theory since norbert wiener's (c. 1948, see 1961) exposition of cybernetics, defined as the science of communication and control in the animal and the machine. the first part of this definition has to do with the role of information in regulatory systems. the second part to the existence of laws or principles of control that apply to complex learning systems. one aspect of an information system that this concept of cybernetics implies is the concept of memory, or storage, of the information, in order to regulate or control the system and to enable it to learn (simon, 1976; wiener, 1961). herbert simon (c. 1945, see 1976) was among the first to specifically refer to memory as a relevant factor in administrative decision making. in the case of corporations we refer to this as "corporate memory". while "corporate memory" has not been satisfactorily identified or investigated in the 3. also called the post-industrialist society: a shift to a more complex information and communication environment with a shift away from goods production to information based services (bell, 1973; lessem, 1990; masuda, 1980). 76 ajis literature, in this paper we define it as being a function of data (both hard and soft) of the corporation as well as the employees of that corporation, which can be expressed as follows: cm =f(ch, cs, ph, ps), where: cm = corporate memory, while: c/j = corporate hard system data, i.e., transactional data (e.g., in the accounting information systems) and electronically encoded textual data, stored centrally as well as distributed through the various network and system hardware; cs = corporate soft system data, which include: * knowledge surrounding the hard data; * "organizational culture" (collective, or shared beliefs and values about the organization by the individuals comprising the organization); pfr = personal hard copy data, i.e., individual personal records of the personnel (gained by virtue of their position or idiosynchratically) stored in hard copy, usually printed or written on paper (e.g., notes in their diaries); ps = personal soft data, i.e., the implicit meaning and individuals' personal views or recollections stored in their own mind's memory. the distinction between the corporate and personal concepts were coined to expose the underestimation of the information available which has potential value for the organization. other authors have referred to similar aspects of the holistic meaning of the contextual aspects of the system by the term "weltanschauung", literally world view (e.g., checkland, 1981). however, it has not been common to clarify the sense of corporate memory per se, nor its components. to appreciate the significance of the problem is it useful to review (in the following sections) some of the organizational issues which have surfaced in the public sector and private sector since the 1980's. research on the "human factor" in organizational restructuring broadly speaking corporate mergers, acquisitions and down-sizing can be described as organizational re-structuring. there is much research work in the difficult, commercially competitive and confidential fields of organizational structure and design (see mintzberg, 1989) organizational change (see argyris, 1985; argyris, 1990) and organizational culture or climate (see rousseau, 1988). however, there has been little research on the effects of corporate acquisitions, mergers and downsizing on corporate memory and management information systems. one major research program on the conditions for successful corporate acquisitions by the london business school (see hunt, 1988) has shed some light on part of the problem. professor john hunt and his team conducted longitudinal studies, including interviews and quantitative information from the buyers and sellers about the whole acquisition process involving 40 british firms. they found that the four most frequently identified factors which tended to affect difficult or failed post-acquisition implementation were (in order of decreasing frequency) the personnel systems and practices (such as differences in pay and conditions), the clash of management styles and the organizational cultures (e.g., differences in the values and belief systems), the apparent risk aversion of the seller's managers (according to the buyer's view they did not accept the challenge presented to them by their new owners). the fourth problem area is the most relevant here, as the researchers put it: "the excessive demands for information from the buyer's head office. ... the seller has the information and the buyer needs it. information is the only source of power the seller's managers have in a decidedly unequal distribution of power. whether the demands were excessive or not is less important than the fact that in over one third of our total sample the sellers were convinced demands were excessive and were irritated by them." (hunt, 1988, p. 10, emphasis in original) september 1993 77 it is the organizational power pertaining to the manager's knowledge and access to information which is of major concern in this paper and the research being undertaken. the combination of the managers' knowledge and the complete organizational data is seen as a form of intellectual technology. this is different from the research focused on managers' needs for information (cf mcfarlane, 1984; mintzberg, 1979, 1989). the present research pertains to the information the managers already have gained through their experience, and which is capable of being lost on downsizing. it is hypothesised that the organization needs such ps as well as p^ information for its continuity of effectiveness. thus far research on organizational memory and research on the information needs of managers have neglected these personal information storage issues and focused on the c/j and cs information systems (especially that held by computers). certainly, there is little evidence available about what the system designers in the organization have done or should do in addressing the implications of senior staff turn-over for the information needs of organizations. information: a key to organizational effectiveness and competitiveness information is a generally accepted variable of organizational effectiveness and is seen as one of the parts of a process that leads to the competitiveness of an organization. pettigrew (1973) points out that strategically relevant information is required for any effective influence attempt. the development of information that is relevant to the situation requires analysis and insight which in turn implies that all relevant data is available at the time. competitive advantage is seen to derive from the organization's ability to retrieve and use organizational information, which in turn leads to decision making which provides customers with services or products in a manner superior to the organization's competitors (e.g., see porter, & millar, 1985; mcfarlane, 1984). this competitive advantage leads to superior performance by that organization. a problem inherent in the design and implementation of information systems is prediction of usage. this depends on whether there exists a model of all of the relevant actions which is sufficiently complete with respect to the later use of data produced from that model for decision making. however, as authors such as alter (1980) have pointed out, there is a difference between the relatively "poor" performance of information systems, which are generally described as complex, and the relatively "good" performance of simple data processing systems. this difference is known as the management information systems gap. such a gap commonly exists, for example, between the transactional systems and the decision support systems. information has value a concept which economists use to measure an opportunity lost is the "opportunity cost" of that opportunity. without minimising the work done in this area, it is clear that a gap in a management information system represents an opportunity cost. an emerging area of research on this topic is called "information economics". this area of thought argues that business performance is the measure of value for the enterprise and its management, and for the information system and its staff (parker & benson, 1988, p. 8). while the simplistic elemental system (consisting of a planning, an action, an output, and a feedback loop as fundamental units) provides the means for controlling an organization and is the model for giving the organization a competitive advantage which will create value, it is the flow of information which provides the coherence and meaning to that process. however, information is the conversion of data by a human which gives meaning to the transformation process, i.e., weltanschauung (checkland, 1981, p.215). as mintzberg (1978, pp. 49-50 emphasis in original) put it so well: "most work cannot get done without some informal communication. ... at the managerial levels, study after study shows that managers of all kinds favour the verbal channels of the informal system over the documents of the formal ... and they spend almost as much time communicating outside the formal chain as inside it. ... the soft information, intangible and speculative, is simply ignored in the formal mis despite clear evidence that managers depend on such information. ... the result is that managers bypass the mis to get much of their own information. 78 ajis they build their own networks of informal contacts, which constitute their real information and intelligence systems." consequently, managers build their own personal databases which are maintained and controlled by them in both hard (p/j) and soft (ps) form to enhance organizational data which is stored in hard (c/,) form in the computer and in soft form in the organisational culture (cy) and networks (cs). it is this form of "added value" or "opportunity cost" which current information systems tend to ignore as if this information has no value. the human factor in organizational memory data for information systems are extracted from two separate domains or spheres. that is the internally developed data of the organization which is collected from its own knowledge and the data that is resident in the domain outside the organization (e.g., see mintzberg, 1979; kirzner, 1979; porter & miller, 1985). the ability of an organization to use this data is dependent upon its information systems which track both domains of data and the storage facilities of the organization which hold the data. however, when one reads the information analysis and design literature there is rarely (if ever) any discussion of how to gather, store, or update data of the external variety or how that data should be integrated or interfaced with internally generated data. clearly, with a body of knowledge available and generally accepted methods of planning which include external data, organizations must gather, store and update this data in order for it to be used in information processing. if the computer systems are not the "memory" of the organization then the logical storage of such data lies in physical hardcopy storage and its access systems are totally reliant upon human memory banks (i.e., cs and ps data). further, data which is non-existent in the formal computerised information system (either through oversight or poor design) must also be stored in, and also relies upon, human memory and hardcopy physical storage banks before it can be updated or retrieved into the management information system, in order to provide support for decision making. two aspects are noticeable with respect to data which eludes capture into the formal computerised information system. the first aspect is that of design error and the second a more subtle form of aggregated information loss due to the nature and design of computer systems. these errors and losses are considered below in turn. in the first instance, data collection systems are limited by the designer's ability to forecast currently unknown requirements or by inability to correctly interpret the current user requirements. for example, take a university which had a new system designed for managing student enrollments. then the department of employment education and training changed its requirements and the university was amalgamated with a college. both changes were not anticipated at the time the design tender was let and both have a major impact on mis as well as the "soft systems" issues like incompatible corporate cultures. in the second situation the computers ability to store data is limited by the designer's tools in the form of limited syntax in computer languages and the disaggregation of data into machine readable code. in both the limited syntax and the disaggregation there is the potential for information loss through simplification. in both design errors and aggregation loss the organization depends upon the "user workaround' to interpret or modify the data through utilisation. these "user workaround" practices are human dependent, hence, it is the human minds of selected users which substitute for the organizational memory. human aspects of information fragmentation butler et al (1981) propose that there is a connection between fragmentation of information, and the ever changing areas of market available for opportunity exploitation, and the organization's level of success with these opportunities. they argue that the collection, processing and assessment of information is a matter of maintenance of human resources as the area of human resource management is in the best position to gauge the degree of variance in the organization's perceptions. while one could argue that it is data which are collected and processed, not information, there is an intuitive and observational nicety about the relationship between perceptions and an organization's market opportunities (see kirzner, 1979). september 1993 79 the solution to information fragmentation proposed by butler et al (1981) is to recognise the role that humans play in the initiation, facilitation and enhancement of intended strategies. the consequence for organizational data is that if the system design does not accommodate the storage of data from these positions, then data will be stored under personal control and maybe even stored within the individual (ps) rather than in some form of hard copy (p^ or corporate memory loss in the public sector a number of commentators on "public sector reform" (e.g., winston, 1991, 1992) have pointed out that the turn-over of staff and the many changes of policy in government, have combined to create a myopia of the effected organizations; they are often unable to detect when they are reinventing the wheel or falling in the same pits as their predecessors. they have become like the punch-drunk boxer, whose continual minor brain damage (from external pounding and internal haemorrhaging of key personnel) has rendered them unable to realize their own inappropriate behaviour, or incapable of remembering enough of their mistakes to learn from them. as in the private sector, public sector organizations have a duty to meet the needs of their shareholders, and clients, i.e., the tax payers, the general community. it seems reasonable to draw the parallel to include the need for the public sector to develop "learning organizations" (argyris, 1990; garratt, 1987; senge, 1990) to continually assess its delivery towards those needs. indeed, one of the most important reviews of the public sector in australia, chaired by h.c. coombs, identified organizational improvement as a vital part of public sector administration (royal commission on australian government administration, 1976). a public service restructured in the public sector case study being researched, a human service organization was drastically restructured by the minister, leading first to a large proportion of staff lost through redundancy and then an expansion in the new direction. both of these moves caused drastic effects on the corporate memory. an example, of how this affected the inefficiency was that the efforts to establish a quality assurance and programme evaluation training program were cut short due to most of the key staff leaving and the system developed to monitor the clients' progress in their program was also disrupted and then later was reinvented by different people with apparently little or no increase in productivity nor increased satisfaction of the staff. mis design issues raised from the experience of the case studies of public sector organizations being examined it seems that there arise at least three potential system design limitations which take on significance in the downsizing and restructuring of organizations, viz.: * neglect of the attrition of "personnel memory' (p^ and ps data) resulting from the dislocation of staff; * a focus on managers' needs in the organizational diagnosis and mis design at the expense of the needs of the workers at the client or market interface; and * a lag between when the system designers conduct their organizational diagnosis and obtain the organizational information needs, to when the design is fully implemented and the errors become noticeable, the degree of lag can be exacerbated by prevailing conditions of (internal and external) environmental uncertainty (and organizational cultural factors), which are the circumstances that require the most appropriate strategic response of the organization and its managers, and when they are most likely to need an up-to-date mis. if this lag is exacerbated by internal confusion and resulting staff turn-over, there is a risk that the original goals of the mis may mis-match the needs of the staff replacing them, as the new comers may not have access to the same degree of inside knowledge or corporate memory. 80 ajis implications and conclusions in attempting to rectify some of these problems it seems that the analysts and designers of management information systems need to consider the following points for the benefit of their audience: 1. exit interviews. the organization's personnel manager should conduct exit interviews, or in their absence they should be taken up by the managers responsible for the management information systems. only by interviewing the staff who are leaving will there be an opportunity to learn what they are taking with them, not simply in physical, hard copy form (taken or destroyed as superfluous by the exiting employee), but also in that ex-employees "implicit" memory of the organization. this also raises the need for research into the processes of obtaining such information, and research regarding the extent of knowledge that leaves the corporate memory when staff have departed. 2. disposal of personal records. when staff leave in large numbers it is probably too late to consider the need for a systematic review of the personal records which may have been disposed of, with the consequent loss of data. managers responsible for management information systems need to consider the strategic significance of the confidential information and crucial implicit memory data which could be lost during the down-sizing or re-structuring. formulating a policy (including intellectual property agreements with staff) on corporate memory review and retention as part of the restructuring or merger etc. could be a contribution to the strategic planning. 3. limitations of new technology. the expectation that the installation of a state-of-the-art mass optical storage system will solve the corporate memory problems is naive. the point of this paper is that the personnel themselves are the vital memory resource. it is their experience and know-how which even the "expert systems" are not able to entirely replace. even if the new systems could store all the relevant information, the goodwill of the retrenched staff would be a crucial factor in accessing the departing information. 4. strategic significance of externally sourced data. individuals who deal with the external environment acquire and generate valuable data on the strategic environment. the data gathered externally by these personnel is not usually entered into management information systems, which are usually internally or control-oriented. organizations need to consider the literature on strategic external environmental scanning and establish conduits for obtaining and storing such external data. this also puts an emphasis back on the information needs of crucial personnel with strategic roles to play in the organization's planning and information systems. the other side of the coin about external competitiveness is that individuals who leave the organization are available to be recruited by the organization's competitors. the obvious significance of this goes beyond the corporate memory as it is damaging externally as well as internally. 5. on-going consultancy. some of these problems can be lessened or avoided by retaining the exiting personnel on a consultancy basis to be recalled and paid when doubts arise about information needed or solutions to old problems. redeploying staff as contractors is sometimes feasible in the public sector as well as the private sector. of course, this again reflects on the mode and rationale for dismissal and ' the degree of good will, and confidentiality involved. 6. appropriate control aspects. it should be possible to redesign or accommodate the personnel aspects of corporate memory in the management information systems. of course there are control issues that the designers need to address with respect to this new form of data (as to its update, storage and access). these control issues pertain especially to the strategic types of data (e.g., information about competitors), as well as the data perceived as confidential or personal (e.g., personal opinion). 81september 1993 7. collaboration in information economics research. there are no simple solutions for the problems found at the interface between management, computing, personnel psychology, and information economics. however, there is a need for managers in the information systems area to liaise with researchers in emerging disciplines in order for the practical edge of application to adopt concepts which are relevant or to exert influence over the agenda of researchers where appropriate. further research the lack of relevant australian case studies concerning the effects of organizational change on management information systems is a serious deficiency, for academics and practitioners. the difficulty in obtaining access to the qualitative, personal, confidential data, and other pertinent information, is a severe stumbling block to research in this field. the present paper has briefly mentioned two case studies, which are interesting in their introduction to this topic, but greatly in need of elaboration, replication and extension. there are also logistical difficulties of obtaining information from staff who have now left as they are either uncontactable, unwilling or who themselves are not aware of the information that was relevant. information would have been destroyed in the hard copy form (pfr), or unrecognised and/or forgotten in the soft form (ps and/or cs). there is a need for research on such practicalities if the anecdotal evidence of gaps in knowledge or missing information, such as was found in the organizations studied, are to be more precisely investigated. references alter, s.l. (1980) decision support systems: current practices and continuing challenges. new york: addison-wesley. argyris, c. (1990) overcoming organizational defences: facilitating organizational learning. boston: allyn and bacon. beer, s. (1972) the brain of the firm: the managerial cybernetics of organization. london: allen lane. bell, d. (1973) the coming of post-industrialist society. new york: basic books. butler, j. ferris, g & napier,n. (1981) strategy and human resources management. ohio, usa: south western publishing co. cartwright, s. & cooper, c.l. (1990) "the impact of mergers and acquisitions on people at work: existing research and issues." british journal of management, vol. 1, pp. 6576. cartwright, s. & cooper, c.l. (1993) "the psychological impact of merger and acquisition on the individual: a study of building society managers." human relations. vol. 46, no. 3, pp. 327347. checkland, p. (1981) systems thinking systems practice. london: wiley. colebatch, t. (1993) "governments big job shedders" the age, 29th april, p. 17. de geus, a. (1988) "planning as learning", havard business review, march/april, pp. 7074. fayol, h. (1916) general and industrial management, (transl. from french) london: pitman, 1949. garratt, b. (1987) the learning organization. london: fontana/collins. jacka, h. (1988) "the personnel dimension managing redeployment". canberra bulletin of public administration, may, vol 54, pp. 45 50. kirzner, i. (1979) perception, opportunity and profit. chicago: university of chicago. lessem, r. (1990) "introduction" to y. masuda, managing in the information society: releasing synergy japanese style. oxford, u.k., basil blackwell, pp. i xxv. marks, m.l. & mirvis, p.h. (1992) "rebuilding after the merger: dealing with 'survivor sickness"'. organizational dynamics, autumn, pp. 18 -32. marsh, p.(1986) "are profits the prize of the prey or the predator?" mergers and acquisitions, may, vol. iv -vi, masuda, y. (1980) the information society as post-industrialist society. institute for the information society, tokyo. mcfarlane f. (1984) "information technology changes the way you compete" harvard business review, may/june. mintzberg, h. (1979) the structuring of organizations: a synthesis of the research. englewod cliffs, new jersey: prentice-hall. 82 ajis mintzberg, h. (1989) mintzberg on management: inside our strange world of organizations. n.y.: free press. parker m, benson r, (1988) information economics, englewood cliffs new jersey: prentice hall. pettigrew, a.m. (1973) the politics of organizational decision-making. london: tavistock. porter, m. & millar, v., (1985) "how information gives you a competitive advantage" harvard business review, vol. 4, july/aug, pp 149-160. puxty, a.g. (1986) organization and management: an accountant's perspective. london: pitman. review of commonwealth administration, (1983) report, canberra: australian government publishing service. rousseau, d. (1988) "the construction of climate in organizational research", in cooper, c.l. & robertson, i. (eds) international review of industrial and organizational psychology 1988. chichester: wiley., pp. 139-158. royal commission on australian government administration (1976) report. canberra: australian government publishing service. samper, j.p. (1989) "changing face of the global corporation: reflections on the fabled phoenix" proceedings of the world management congress, new york: world management council september 22. senge, p.m. (1990) the fifth discipline: the art and practice of the learning organization. new york: doubleday. simon, h.a. (1976) administrative behaviour: a study of decision-making processing administrative organization, third ed. new york: free press. tsoukas, h. (1991) "the missing link: a transformational view of metaphors in organizational science." academy of management review, vol. 16, july, pp. 566585. walsh, j. p. & ungson, g.r. (1991) "organizational memory" academy of management review, vol. 16, no. 1, pp. 57-91. wiener, n. (1961) cybernetics or control and communications in the animal and the machine. second ed., cambridge, mass.: m.i.t. press. winston, j. a., (1991) "on-going issues: performance management review paper: linking evaluation and performance management". proceedings of the national evaluation conference, adelaide: australasian evaluation society and state print, pp. 599 606. winston, j. a. (1992) "performance indicators for program evaluation: the case against." unpublished draft paper presented at the conference on program evaluation in the public sector. sydney: iir conferences, 6th april. zuboff, s. (1988) in the age of the smart machine. london: heinemann. 83 september 1993 the australasian journal of information systems (ajis) began in 1993 and was based on the simple realisation that there was a need for an academic ‘outlet’ for is research in australia australasian journal of information systems volume 17 number 1 2010 83 impact of pressure for environmental sustainability on grid assimilation – empirical results from the financial services industry jens vykoukal, roman beck, martin wolf e-finance lab university of frankfurt, germany {jvykoukal, rbeck, mwolf}@wiwi.uni-frankfurt.de abstract due to the significant increase in it-related power consumption and the resulting higher co2 emissions, green it has gained considerable attention in industry and society in recent years. green it as an engineering paradigm encompasses the multifaceted, global effort to reduce power consumption and the promotion of environmental sustainability. due to several similarities between green it objectives and the environmental benefits of grid technology, this article provides empirical evidence from the financial services industry emphasizing that grid technology is capable of reducing the environmental impact of it hardware. furthermore, the article analyzes the extent to which pressure for environmental sustainability as well as different types of institutional forces impact on the intention of enterprises to use grid technology as a means to reduce energy consumption of it hardware, which is one of the key green it objectives. introduction during the last few years, the acceptance that co2 emissions are a major cause of global warming and changes of weather patterns has grown steadily. therefore, enterprises, governments, and society at large are beginning to consider environmental issues in the process of technology adoption, leading to increased application of environmentally sound (“green”) practices (murugesan 2007). the adoption of such green practices is mainly influenced by economic forces (i.e., rising energy costs), environmental regulations imposed by governments or inter-governmental organizations that allow or prohibit certain practices, and social pressure that stems from the need to meet social obligations and enforces moral governance (molla et al. 2009b). one of the most important green practices is the reduction of electric power consumption of it hardware that needs significant amounts of electricity (murugesan 2008). according to a report provided by the u.s. environmental protection agency (2007), data centers were estimated to have used 1.5 percent of all electricity in the united states in 2006, and their power demand is projected to grow 12 percent per year through 2011. in addition, the cooling and air-conditioning problems caused by global warming or energy supply difficulties, especially in over-populated areas, further enhance ecological pressure on it-intensive industries. consequently, the concepts of environmental sustainability have been among the most important themes in industry to emerge over the last decade australasian journal of information systems volume 17 number 1 2010 84 (petrini & pozzebon 2009) and the key objective is to reduce the environmental impact of it and to facilitate the emergence of a more sustainable environment (e.g., by reducing co2 emissions). in order to meet this goal, the it industry increasingly considers green it as a way to address environmental issues of it and envisions environmentally sustainable it as the key to future success, as suggested by several industry research reports (e.g., gartner 2008; idc 2008). accordingly, ryan (2008) and porter and van der linde (1999) state that enterprises exhibiting the technology and vision to provide products and services that address environmental issues are likely to achieve sustained competitive advantage. another driver of green it initiatives is that it investors and consumers are beginning to look at the carbon footprint of an it company and its products. in this context, a carbon footprint is a measure of the total set of greenhouse gas emissions caused directly and indirectly by an individual, an organization, a process, or a product (kurp 2008). as a result, it companies increasingly advertise their environmental credentials and disclose their carbon emissions in order to be recognized as being an environmentally responsible enterprise. moreover, besides social pressures, recent research suggests that mimetic and coercive pressures significantly drive green it adoption (chen et al. 2009). this means that organizations are motivated to adopt a given green practice because of the favourable results achieved by other adopters (mimetic pressure) and because of governmental regulations, laws, and industry standards (coercive pressure). since especially it-intensive industries are increasingly exposed to pressure for environmental sustainability in the technology assimilation process, more and more enterprises are searching for ways to reduce it-related energy consumption without huge investments in new, energy-efficient hardware. as will be shown in this article, grid technology is capable of reducing the environmental impact of it hardware while at the same time providing enterprises with large computing and storage capacity. although the extant literature on grid computing suggests that grid technology is capable of minimizing the number of it hardware by automatically adjusting the provision of hardware resources according to the demand, grid-based it infrastructures have not yet been considered to be an implementation of green it concepts. therefore, the research objective is to provide first empirical evidence for the analogies between green it objectives and the characteristics of grid technology by depicting the results of a questionnaire-based field study conducted in the financial services industry. to the best of our knowledge, there is only little empirical evidence of the environmental impact of grid technology in prior literature, wherefore this research is one of the first quantitative studies on this topic and thereby contributes to both theory and practice. the field study analyzes the relationship between perceived pressure exercised on financial institutions for environmental sustainability and their motivation to use grid technology to respond to this pressure by reducing it-related energy consumption. furthermore, the role of different types of institutional pressures (mimetic, coercive, and normative pressure) is analyzed in the context of grid assimilation as a means to reduce environmental impact of it hardware. although the term “green it” is multifaceted and encompasses the manufacturing and purchasing of energy-efficient it equipment, the efficient operation and utilization of hardware devices, as well as its proper disposal (murugesan 2008), this article mainly focuses on the minimization of energy consumption of it equipment as the key green it objective. other green practices include the design and manufacturing of environmentally sound and energyefficient it equipment, which can be achieved by adopting new techniques and materials that are both environmentally friendly and economically advantageous. prominent examples are the move from 65 to 45 nanometer chips that have increased energy efficiency and an improved performance per watt ratio (allarey et al. 2008) or the implementation of new power management features as well as the production of recyclable materials (kurp 2008). moreover, green practices include the reuse, refurbishing, and recycling of old it equipment in environmentally sound ways (murugesan 2008). australasian journal of information systems volume 17 number 1 2010 85 the remainder of this article is organized as follows. first, the article emphasizes the need for green it infrastructures in order to reduce the power consumption in data centers and provides an introduction into grid technology that is expected to be an effective and efficient implementation of green it. subsequently, the theoretical background of the research is illustrated and an introduction into the research model that is used in the survey is provided. the following sections depict the research model and the hypotheses derived from literature and describe the empirical study conducted to validate the model. the results of the empirical study are presented and discussed subsequently. finally, the article concludes with a summary and limitations of the findings and an outlook to further research. need for green it infrastructure due to the increasing competition in the industry and the current market dynamics, enterprises are forced to adopt high-performance computing (hpc) technology in order to stay competitive. according to a study by joseph et al. (2004), 97 percent of the u.s. businesses surveyed could not exist, or could not compete effectively, without the use of hpc technology. although data centers achieve economies of scale in management and power supply, they consume large amounts of energy and have high carbon emissions. due to the high cost of power consumption and cooling, enterprises are facing a dual challenge of adopting more hpc technology to meet dynamically changing business needs while at the same time delivering such capabilities cost effectively as well as power-efficient (scaramella & healey 2007). as already outlined, green it can support enterprises to use it equipment in an environmentally friendly way by reducing the power consumption of it hardware. although the benefits of green it solutions are numerous and the rate of green it adoption is increasing, biros et al. (2008) and olson (2008) state that most green it efforts take too long since many enterprises cannot afford a short-term replacement of their existing systems with newer ones. here, grid technology provides a remedy to this problem by giving enterprises the opportunity to build up a powerful green it infrastructure by connecting existing it hardware into a grid without the necessity to invest in new hardware. enterprise grid computing (strong 2005) provides users and applications with immediate access to large computing power and storage capacity, such as server clusters, servers, desktop computers, storage systems and databases that can be accessed as a unified resource across an industry, enterprise, or workgroup (foster & kesselman 1999). grid technology in combination with virtualization technology automatically handles fluctuating workloads and peak demands by adding it resources to or removing them from the grid without affecting the resilience and stability of the grid infrastructure (vykoukal et al. 2009). enterprises thereby benefit from a “breathing” it infrastructure with constantly high resource utilization and reduced power consumption due to minimal hardware utilization. this is especially important because of the fact that in typical data centers, resource utilization is only 20-30 percent on average (bohrer et al. 2002). therefore, the goal is to minimize the number of under-utilized or idle it resources that still consume significant amounts of energy. hence, grid virtualization and minimized hardware use are a key strategy to reduce it-related energy consumption. additionally, simulation results provided by beck et al. (2008) indicate that virtualization of grid resources can significantly reduce the number of it resources and thus produce a cost reduction of about 40 percent compared to dedicated servers. another benefit of grid technology is that it allows it service providers to interconnect a large number of it resources into a grid and to offer these grid-based it resources to their customers over the internet on a use-on-demand, pay-per-use basis. in extant literature this concept of on-demand computing is often associated with cloud computing (foster et al. 2008). since most of the external service providers have already invested in green it initiatives, they can offer grid resources that are more energy and cooling efficient, leading to reduced carbon emissions. in addition, grid providers australasian journal of information systems volume 17 number 1 2010 86 increasingly adopt energy-efficient servers, apply virtualization technology to maximize the utilization of hardware, and build new data centers at locations with a cool climate (e.g., iceland or siberia). by using cold outside air for cooling of the data center, there is lesser need for power to operate mechanical chillers to produce cool air, thus reducing overall energy consumption. for example, microsoft, that offers grid resources on-demand to customers, has built a large data center in ireland which is, due to the moderate climate in ireland, air cooled and therefore 50 percent more energy-efficient than other comparably sized data centers (kurp 2008). consequently, grid technology is shown to be suitable for the implementation of green it concepts and the development of a green it infrastructure. research model and hypotheses in order to empirically validate the impact of ecological pressure and institutional pressure on the assimilation of grid technology, the research model depicted in figure 1 was developed. the research model draws on institutional theory (dimaggio & powell 1983; meyer & rowan 1977) and analyzes the extent of institutional pressure for grid assimilation. the model also incorporates a newly developed construct that measures the pressure for environmental sustainability in the industry domain. lastly, the model analyzes the impact of these different types of pressures (institutional pressure for grid assimilation and pressure for environmental sustainability) on the motivation of enterprises to use grid technology to respond to this pressure. the constructs, their relationships, and the resulting hypotheses are discussed in the following subsections. grid assimilation as green it strategy h1 h4 + + normative pressure coercive pressure mimetic pressure pressure for environmental sustainability h3 + h2 + control variables: country firm size earliness of grid adoption figure 1: research model pressure for environmental sustainability during recent years, especially it-intensive industries, (e.g., financial services, manufacturing, and retail industry) have identified environmental sustainability as a way to proactively address and handle environmental issues of it. huang (2009) states that one possible reason for the former lack of environmental concerns in industry is that it has been too progressive and too successful for enterprises to worry about being efficient. however, today, the efforts on green it initiatives in industry are increasing and environmental sustainability is becoming a critical topic for is research australasian journal of information systems volume 17 number 1 2010 87 (elliot 2007). the maturing literature in this research field suggests mainly three types of pressure for environmental sustainability in the industry domain: (1) economic, (2) political and governmental, and (3) social pressure for environmental sustainability. economic pressure mainly stems from rising energy costs, leading to the need for enterprises to reduce power consumption of it hardware. lowering energy costs is often associated with the aim to achieve competitive advantage (hendry & vesilind 2005; kuo & dick 2009; porter & van der linde 1999; ryan 2008). political and governmental pressure for green it mainly stems from various forms of environmental standards and regulations imposed by governments around the world (chen 2001; chen et al. 2009; clemens & douglas 2006; hendry & vesilind 2005; kurp 2008; molla 2009; rugman & verbeke 1998). social pressure is exerted by the increasing customer demand for green it solutions (chen 2001; molla et al. 2009a) and the increased positive public perception of green it initiatives (daly & butler 2009; unhelkar & dickens 2008). by changing the it strategy towards an increased assimilation of green technology, organizations accommodate the ecological demand and increase their resilience to ecology-driven shocks while reducing their vulnerability at the same time. adaptation due to perceived ecological pressure can be regarded as adjustments in social-ecological systems in response to actual, perceived, or expected ecological changes (janssen & ostrom 2006). due to the aforementioned pressures for environmental sustainability and since grid technology is likely to have the potential to reduce energy consumption of it, we propose: hypothesis 1: enterprises that are exposed to a higher level of pressure for environmental sustainability are more likely to assimilate grid technology as an implementation of green it. institutional pressure for grid assimilation institutional theory in general posits that structural and behavioural changes in firms are rather driven by an inherent organizational need for legitimacy than sole considerations of competitive advantages and hidden efficiency potentials (dimaggio & powell 1983; meyer & rowan 1977). this continuous search for organizational legitimacy eventually facilitates the process of institutionalization and organizational isomorphism especially against the background of an uncertain and turbulent environment. due to dimaggio and powell (1983) basically three different types of institutional pressure can be distinguished: mimetic, coercive, and normative pressure. mimetic pressure reflects the pressure to imitate structurally equivalent successful organizations in the same industry without necessarily considering the firm-specific context (dimaggio & powell 1983). these so-called “bandwagon phenomena” can be induced by competitors in the same industry having already successfully adopted grid technology that resulted in a reduction of it-related power consumption and a positive perception of the adopting enterprise and the technology. due to this, swanson and ramiller (2004) suggest that most enterprises “borrow” mindfulness from peers that have successfully introduced a new technology for their business processes and achieved significant benefits. this pressure can even be enforced by rising energy costs and increased customer demand for green it solutions. with these (perceived) ecological benefits of grid technology, mimetic pressure is supposed to positively impact the intention of enterprises to assimilate grid technology for the purpose of reducing energy consumption of it hardware. this is in line with the findings by chen et al. (2009) that found a positive impact of mimetic pressure on the adoption of green practices (pollution prevention, sustainable development practices, and product stewardship). thus, we hypothesize: hypothesis 2: enterprises that are exposed to a higher level of mimetic pressure are more likely to assimilate grid technology as an implementation of green it coercive pressure is defined by the pressure grounded in societal expectations and dependencies towards other enterprises (bela & venkatesh 2007; dimaggio & powell 1983). furthermore, various government and industry regulations exert coercive pressure on enterprises and decisively drive the australasian journal of information systems volume 17 number 1 2010 88 assimilation of new technologies (ang & cummings 1997; zhu et al. 2006). with regard to the increased governmental pressure for sustainability, grid technology is an effective implementation of green it since it can be used to reduce the environmental impact of it by providing large storage and computing capacity in a power efficient way (scaramella & healey 2007). due to this, grid technology provides enterprises with the opportunity to encounter coercive pressure imposed by government and industry regulations. the empirical findings provided by chen et al. (2009) as well as clemens and douglas (2006) indicate that coercive forces encourage organizations to focus on green practices, which reflects the effectiveness of regulatory efforts in guiding green behaviours across organizations. hence, we propose: hypothesis 3: enterprises that are exposed to a higher level of coercive pressure are more likely to assimilate grid technology as an implementation of green it pressure that is rooted in the ongoing process of professionalization is encompassed by normative pressure (dimaggio & powell 1983). this pressure arises from the exchange of best practices among business partners, suppliers, and the government. this ongoing information exchange within the value chain provides enterprises with guidelines how to assimilate grid technology efficiently and provides them with access to first-hand experience with grid technology and its ecological and economic benefits. in line with this, molla (2009) indicates that green practices are driven by normative pressure in terms of pressure for eco-effectiveness and eco-legitimacy. thus, we hypothesize: hypothesis 4: enterprises that are exposed to a higher level of normative pressure are more likely to assimilate grid technology as an implementation of green it grid assimilation as green it strategy to the best of our knowledge, the environmental impact of grid technology has never been empirically analyzed. due to this, the dependent variable of the research model incorporates the suggested potential of grid technology to reduce it-related power consumption, which can be interpreted as a green it strategy. the environmental benefits of grid technology are especially provided by (1) the application of virtualization technology to the grid and (2) the purchase of it resources from external grid resource providers. as already outlined, grid virtualization technology is able to automatically handle fluctuating workloads and peak demands by adding resources to the grid infrastructure or by removing them from the grid. this self-reconfiguration capability of a virtualized grid infrastructure leads to a reduction of redundant hardware components and hence maximizes hardware utilization and minimizes energy consumption (pernici et al. 2008). furthermore, computational workloads can be shifted to more energy-efficient resources so that less energy-efficient resources can be powered down (patel et al. 2003). additionally, since third-party grid resource providers (also called cloud computing providers) increasingly expand their network of massive data centers with hundreds of thousands of servers, petabytes of data, and hundreds of megawatts of power, they tend to be more conscious concerning power consumption than enterprises that utilize less it resources. due to the exploitation of economies of scale in provisioning, powering, cooling, and recycling of it equipment, grid service providers are able to invest in energy-efficient practices and technologies, leading to reduced power consumption compared to traditional data centers (kurp 2008). this leads to the suggestion that grid technology is an effective way to implement the concepts of a green it infrastructure with reduced power consumption, thereby enhancing environmental sustainability. empirical study because our research in the domain of green it is in its early stages and profound theoretical background as well as empirical work is scarcely available, the nature of this research is exploratory. australasian journal of information systems volume 17 number 1 2010 89 the research model depicted in the previous section was operationalized as a structural equation model (sem). the sem approach is a methodology of multivariate data analysis that allows for modeling complex cause-effect relationships involving unobservable latent variables (constructs, factors) and observable variables (measurement items, indicators). these models seek to analyze the underlying causal process that is assumed to generate some phenomenon of interest. as will be explained in the subsequent section, two different sem methods can be distinguished: covariancebased and component-based sem methods. for our research, the components-based sem approach, i.e., the partial least squares (pls) method was applied. the following subsections provide details on the measures used for the empirical field study, and on the data collection as well as the sample profile. measures as depicted in table 1, each construct of the research model is represented by a set of two to three measurement items. the operationalization of the measurement items is provided in table 5 in the appendix. construct abbreviation item source scale pressure for environmental sustainability (reflective measures) sust sust1 based on porter and van der linde (1999) 7-point likert sust2 based on chen et al. (2009) sust3 based on chen (2001) mimetic pressure (reflective measures) mp mp1-3 adapted from liang et al. (2007) 7-point likert coercive pressure (reflective measures) cp cp1-3 adapted from liang et al. (2007) 7-point likert normative pressure (reflective measures) np np1-3 adapted from liang et al. (2007) 7-point likert grid assimilation as green it strategy (reflective measures) assm assm1 based on pernici et al. (2008) 7-point likert assm2 based on kurp (2008) table 1: set of constructs and measurement items in pls analyses, the lack of emphasis on the measurement properties of the constructs makes it more amenable to the use of constructs with fewer items than are required for covariance-based sem (hair et al. 2010). generally, each construct should be measured by at least two items so that both measurement reliability and construct validity can be assessed (gerbing & anderson 1988; nunnally 1978). however, measurement models in which factors are defined by only two items per construct can be problematic so that more items (maccallum et al. 1999) or larger samples are required to obtain a converged and proper solution (anderson & gerbing 1988). due to our sample consisting of 359 responses we deemed the use of only two items for the construct assm as appropriate for our study. australasian journal of information systems volume 17 number 1 2010 90 whenever possible, existing measures from prior empirical studies were adopted as was the case for the three institutional pressures (mimetic, coercive, and normative pressure). these measures were informed by existing measures in the literature (i.e., liang et al. 2007) and adapted to our context. in case of the sust and assm constructs, our new operationalizations were deductively informed by extant literature on green it (i.e., chen 2001; chen et al. 2009; porter & van der linde 1999) and grid technology (i.e., kurp 2008; pernici et al. 2008). in doing so, the nature of causality between the (unobservable) latent constructs and their (observable) measurement items was assessed since two different types of relationship between constructs and their measures can be distinguished: reflective and formative. reflective relationships indicate that the measurement items are a reflection of the construct (reflective measurement), whereas in formative relationships, the items describe and define the construct (formative measurement) rather than vice versa (petter et al. 2007). a fundamental characteristic of reflective measurement models is that a change in the latent construct causes variation in all measures simultaneously. furthermore, all measures in a reflective model must be positively inter-correlated (diamantopoulos et al. 2008). in formative measurement models, the indicators characterize a set of distinct causes which are not interchangeable as each indicator captures a specific aspect of the construct's domain (jarvis et al. 2003). thus, omitting an indicator potentially alters the nature of the construct (bollen & lennox 1991). moreover, in formative models, there are no specific expectations about patterns or magnitude of inter-correlations between the indicators (diamantopoulos et al. 2008). in our study, all constructs were modelled using reflective indicators (measured on a fully anchored 7point likert scale, ranging from “strongly agree” to “strongly disagree”) since the items used were designed to tap into the same concept or phenomenon and have the tendency to move in the same direction. in detail, the items of “mimetic pressure”, “coercive pressure”, and “normative pressure” are reflective indicators, which is consistent with previous literature (e.g., liang et al 2007). the items of the sust construct are also operationalized in a reflective manner since high pressure for sustainability exerted on organizations is reflected by high energy cost awareness, increased environmental regulation, and increased positive public perception of green practices. furthermore, we assessed the inter-item correlations. our results suggest high correlations among the three indicators, ranging from r=0.59 to r=0.70, which supports our hypothesis of reflective indicators. moreover, both items of the assm construct were modeled as reflective indicators. both indicators are inter-related since they reflect the capability of grid technology to reduce it-related power consumption. in this context, it is irrelevant whether the grid-based resources are provisioned inhouse or by an external grid resource provider. the inter-item correlation between both indicators is high (r=0.62) and further supports our hypothesis of reflective indicators. to ensure content validity of the measures drawn from the literature on green it and grid technology, several expert interviews were conducted and the survey instrument was provided to a panel of judges of both practitioners and academics to refine the wording of the measures. data collection and sample profile in order to validate the research model presented in figure 1 and the aforementioned hypotheses, a questionnaire-based, large-scale field study was conducted. the study aimed at it decision makers that work for a financial institution with more than 1000 employees. moreover, the financial institution had to be a grid adopter in order to ask the study participants for their experience with grid technology. the decision to conduct the field study in the financial services industry was driven by the fact that this industry is one of the most promising application domains of grid technology because of its information-driven business processes and its high computational demands (schwind et australasian journal of information systems volume 17 number 1 2010 91 al. 2007). in addition, the financial services industry is the largest user of it among industry sectors, which is reflected in the fact that the annual it spending in the financial services industry is twice as high as in other industries (zhu et al. 2004). in essence, the grid adoption rate is likely to be higher than in other industries. thus, we deemed the financial services industry an appropriate test bed for our research model. from an empirical perspective, focussing on a single industry also allows to control for extraneous industry factors that could otherwise confound the analysis, thereby enhancing internal validity (zhu et al. 2004). since the field study was conducted internationally, the backtranslation method proposed by brislin (1986) was used to translate the questionnaire into different target languages. in doing so, first, professionals translated the questionnaire, originally written in english, into german, french, and dutch. in a second step, different bilinguals translated the questionnaires back into english allowing for a comparison of the original and the back-translated questionnaire and a re-translation of discrepant passages. in august 2009, 3096 potential study participants were selected by a large market research company that runs several expert panels (e.g., it business panel). the research company applied the criteria defined by the authors (it decision maker in a financial institution with more than 1000 employees) for the selection of the panel members and invited them (on behalf of the authors) to respond to the survey by sending an email invitation. the email contained an embedded, individualized url link to the online questionnaire to ensure that each respondent completed the survey only once. moreover, it allowed the research company to track all study participants so that they could send an email reminder to all non-respondents after one week. however, information about the respondent‟s profile (name, email address, etc.) was kept confidential and was not disclosed to the authors. the potential participants were asked to completely fill-out the questionnaire to avoid missing values that can cause bias due to systematic differences between observed and unobserved data. as an incentive for their participation, the respondents that completely filled-out the questionnaire received airline or hotel discount vouchers. in total, 855 responses were returned, indicating a response rate of 27.6 percent. since the study aimed at grid adopters, the study participants were asked at the beginning of the questionnaire to indicate whether they have already adopted grid technology or not. in the latter case, the non-grid adopters were directly excluded from taking part in the survey. 393 responses from nongrid adopters and 103 responses that exhibited missing values were removed, leading to a final sample of 359 valid responses. the key characteristics of the sample are shown in table 2. country number of employees (firm size): invited responded u.s. 2034 (65.7%) 211 (58.8%) 1,001 5,000 5,001 10,000 10,001 50,000 50,000+ 48 (13.4%) 47 (13.1%) 92 (25.6%) 172 (47.9%) u.k. 788 (25.5%) 111 (30.9%) germany 132 (4.3%) 13 (3.6%) canada 44 (1.4%) 11 (3.1%) france 78 (2.5%) 3 (0.8%) netherlands 20 (0.6%) 10 (2.8%) respondent’s position: year of first grid adoption: cto | coo | cio chief systems architect other it decision maker 53 (14.8%) 16 (4.6%) 290 (80.8%) < 2000 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 21 (5.8%) 22 (6.1%) 20 (5.6%) 46 (12.8%) 112 (31.2%) 138 (38.4%) table 2: sample characteristics australasian journal of information systems volume 17 number 1 2010 92 data analysis and results to validate the research model, two main approaches can be considered (tenenhaus 2008): covariance-based sem methods and components-based sem methods. covariance-based sem methods involve various constraints regarding the distributional properties, e.g., multivariate normality (fornell & bookstein 1982) and are mainly applied to test whether the hypothesized relationships among the latent variables and between the latent variables and the measurement items are consistent with the empirical data (diamantopoulos & singuaw 2000). covariance-based sem methods are usually employed when prior theory is strong and further theory testing and development is the goal. in this case, covariance-based estimation methods, e.g., the maximum likelihood (ml) or the unweighted least squares (uls) method, are more appropriate to validate a research model (chin et al. 2003). in contrast, components-based sem approaches, i.e., the pls method, require fewer distributional assumptions about the data (cassel et al. 1999). especially in new areas of applied research and in early stages of measurement instrument development, little is known about distributional characteristics of observed variables. although pls can be used for theory confirmation, it can also be used to suggest where relationships might or might not exist and to suggest propositions for later testing (chin et al. 2003). therefore, the pls method is a predictionoriented approach (chin 1998) that not only supports confirmatory but also exploratory research (gefen et al. 2000). this can be seen as an advantage since theory construction is as important as theory verification (deshpande 1983). due to these facts, we deemed the pls method appropriate for the validation of our research model since our research is still at an early stage and the proposed research model (i.e., that contains two newly-developed constructs sust and assm) has not been tested in the literature. in the data collection stage, we aimed at gathering a large number of responses to address the criticism raised by marcoulides and saunders (2006) of insufficient sample sizes in pls studies. the authors note that even moderate non-normality of data requires a sufficiently large sample size, even if the indicators are highly reliable. in line with this, hui and wold (1982) found, in a simulation study, that the average absolute error rates of pls estimates diminish as sample size increases. thus, the larger the sample, the more reliable the pls estimates. although the appropriate sample size depends on many factors (marcoulides & saunders 2006), chin et al. (2003) state that large sample sizes are needed for unbiased parameter estimates. thus, our data sample of 359 valid responses seems to be sufficiently large to validate our research model. for the data analysis, the software implementation smartpls (version 2.0 m3) was used, which is a components-based path modeling software application based on the pls method. smartpls (ringle et al. 2005) is comparable to pls-graph (chin 2001) since it is based on the same method. to estimate the parameters in the measurement and the structural model, we used pls path modeling with a path weighting scheme for the inside approximation (chin 1998; tenenhaus et al. 2005). because pls does not directly provide significance tests, the non-parametric bootstrap (davison & hinkley 2003; efron & tibshirani 1993) re-sampling method was conducted to provide confidence intervals for all parameter estimates, building the basis for statistical inference. with this method, the performance of an estimator of interest is judged by studying its parameter and standard error bias relative to repeated random samples drawn with replacement from the original observed sample data (marcoulides & saunders 2006). we pre-specified the number of 500 bootstrap samples and conducted the bootstrap procedure with construct level changes pre-processing to test the significance of the path estimates and factor loadings. the default number of bootstrap samples is 100 but a higher number may lead to more reasonable standard errors of the estimates (tenenhaus et al. 2005). in the following subsections, the results of the pls analysis are discussed in detail. australasian journal of information systems volume 17 number 1 2010 93 validation of the measurement model since the research model contains only reflective constructs, the validity of all constructs was assessed in terms of convergent validity and discriminant validity (campbell & fiske 1959), which will be presented in the following. assessment of convergent validity convergent validity signifies that a set of indicators represents one and the same underlying construct (henseler et al. 2009) and is examined by the magnitude of correlation between the indicators of a construct (gefen 2003). convergent validity is demonstrated by (1) indicator reliability and (2) construct reliability. indicator reliability is a measure for the degree to which the variance of an item can be explained by the underlying construct and can be assessed by the indicator loadings. as can be seen in table 3 (bold numbers), all loadings of the reflective constructs are above the recommended threshold of 0.707 (barclay et al. 1995; chin 1998), indicating that there exists more shared variance between the construct and its indicators than error variance (hair et al. 2010) and that the measurement items used were adequate for measuring each construct. construct reliability measures the degree to which items are free from random error, and therefore yield consistent results. construct reliability was assessed by using the average variance extracted (ave), the composite reliability, and the cronbach‟s alpha. the ave measures the amount of variance that a construct captures from its indicators relative to the amount due to measurement error (chin 1998). it is used to assess how well a theoretical latent construct explains the variance of a set of items that are supposed to measure this construct. as indicated in table 4, the ave of each construct is above the recommended threshold of 0.5 (fornell & larcker 1981), meaning that at least 50 percent of measurement variance is captured by the construct. the composite reliability (werts et al. 1974) is an aggregate measure of the degree of inter-correlation or internal consistency among measurement items of the same construct and indicates how reliably the construct is represented by the indicators (chin 1998; fornell & larcker 1981). table 4 shows that the composite reliability scores of the constructs is above the recommended threshold of 0.7 (straub 1989) providing evidence for sufficient reliability. cronbach's alpha (cronbach 1951) is an alternative measure for estimating internal consistency and assumes equal weights of all the items of a construct and is influenced by the number of items. in contrast to this, composite reliability relies on actual loadings to compute the factor scores and thus provides a better indicator for measuring internal consistency (henseler et al. 2009; teo et al. 2009). however, as presented in table 4, the cronbach‟s alpha values were computed and exceed the critical value of 0.7 (nunnally 1978) providing further indication of internal consistency among the measurement items. assessment of discriminant validity discriminant validity measures the degree to which the specified constructs differ even though they are correlated (hair et al. 2010) and was assessed by analyzing (1) the cross-loadings and (2) the fornell-larcker criterion. the first criterion of discriminant validity requires that the loading of each indicator exceeds all of its cross-loadings (chin 1998) and thereby ensures the appropriateness of the measurement model (henseler et al. 2009). the cross-loadings shown in table 3 reveal that each indicator loading is much australasian journal of information systems volume 17 number 1 2010 94 higher on its assigned construct than on the other constructs, providing evidence for sufficient discriminant validity on the indicator level. the fornell-larcker criterion (fornell & larcker 1981) posits that a construct shares more variance with its assigned indicators than with any other construct and is assessed by the relationships between the inter-construct correlations and the square root of aves. in statistical terms, the square root of the ave for each construct should exceed the correlations involving the construct (fornell & larcker 1981; gefen et al. 2000). table 4 shows that the square roots of all the aves (i.e., the numbers on the diagonal) are greater than the correlations among constructs (i.e., the off-diagonal numbers), indicating satisfactory discriminant validity of all constructs. consequently, the results demonstrated that all constructs in the model were indeed different from each other. sust mp cp np assm sust1 0.84 0.00 0.15 0.12 0.39 sust2 0.87 0.10 0.20 0.16 0.38 sust3 0.90 0.08 0.18 0.12 0.44 mp1 0.06 0.88 0.48 0.33 0.20 mp2 0.08 0.94 0.48 0.38 0.21 mp3 0.06 0.93 0.50 0.37 0.24 cp1 0.15 0.42 0.80 0.52 0.32 cp2 0.19 0.43 0.89 0.51 0.37 cp3 0.17 0.50 0.88 0.48 0.41 np1 0.17 0.31 0.50 0.89 0.35 np2 0.09 0.39 0.50 0.90 0.31 np3 0.12 0.30 0.50 0.73 0.20 assm1 0.46 0.20 0.42 0.31 0.92 assm2 0.37 0.23 0.36 0.33 0.89 table 3: loadings and cross-loadings for measurement items. bold numbers indicate indicator loadings on the assigned constructs. mean sd ave cr alpha sust mp cp np assm sust 4.76 1.57 0.76 0.90 0.84 0.87 mp 4.62 1.22 0.85 0.94 0.91 0.07 0.92 cp 4.83 1.46 0.74 0.89 0.82 0.20* 0.53* 0.86 np 4.47 1.44 0.71 0.88 0.80 0.15* 0.39* 0.58* 0.84 assm 5.01 1.29 0.81 0.90 0.77 0.47* 0.24* 0.43* 0.35* 0.90 table 4: means, standard deviations, average variances extracted (ave), composite reliabilities (cr), cronbach‟s alphas, and correlations among constructs; diagonal elements represent the square root of ave; * p < 0.01 (two-tailed). australasian journal of information systems volume 17 number 1 2010 95 validation of the structural model since all constructs showed convergent validity and discriminant validity and all indicators satisfied various reliability and validity criteria they were used to test the structural model and the proposed hypotheses. in doing so, the hypothesized relationships between the constructs were tested by estimating the path coefficients and the r² value. path coefficients indicate the strengths of the relationships between the independent and dependent variables, whereas the r² value is a measure of the predictive power of a model for the dependent variables (chin 1998). as can be seen in figure 2, three of the four path coefficients are above the threshold of 0.1 (sellin 1994) at a significance level of at least 0.05 (two-tailed). therefore, three hypotheses h1, h3, and h4 are supported by the survey data, whereas hypothesis h2 is not supported by the survey data due to the insignificant path coefficient. the explanatory power of the structural model is measured by the squared multiple correlations (r 2 ) of the dependent variable. the r 2 value of 0.354 indicates that, according to chin (1998), the model explains a moderate amount of variance for the dependent variable. normative pressure coercive pressure mimetic pressure pressure for environmental sustainability r 2 = 0.354 h1 0.386 ** (t=7.016) h2 0.019 (t=0.308) h3 0.268 ** (t=4.076) h4 0.133 * (t=2.079) control variables country dummy (u.s.) earliness of grid adoption firm size grid assimilation as green it strategy -0.029 (t=0.662) country dummy (u.k.) country dummy (germany) country dummy (canada) country dummy (france) -0.025 (t=0.430) -0.125 (t=0.780) -0.095 (t=0.626) -0.089 (t=1.178) -0.033 (t=0.488) -0.018 (t=0.284) figure 2: structural model with results; ** p < 0.01, * p < 0.05 (two-tailed); netherlands sample served as reference group for five country dummy variables. australasian journal of information systems volume 17 number 1 2010 96 effect of control variables in order to account for differences among the financial institutions, three control variables “firm size” (zhu et al. 2006), “earliness of grid adoption” (fichman 2001), measured as the number of years since first grid adoption, and “country” (zhu & kraemer 2005) were included in the model. rogers (1995), tornatzky and fleischer (1990), as well as slade and van akkeren (2002) suggest that firm size may be positively related to innovation adoption since large firms are more likely to exhibit slack resources. in contrast to this, nord and tucker (1987) argue that smaller firm size can also be expected to facilitate innovation assimilation since small firms require less communication, less coordination, and less influence to gather support. the second control variable “earliness of grid adoption” captures the fact that firms that initiated grid implementation activities earlier will have had more time to reach later stages of assimilation, leading to different magnitudes of it-related energy consumption. therefore, it is likely that this measure involves some degree of commingling of behaviours across different assimilation stages (fichman 2001). the third control variable “country” accounts for country-specific characteristics and differences in the survey data. due to the categorical nature of this control variable, it was formed into five dummy variables (aiken & west 1991; cohen & cohen 1983). the responses from the netherlands were used as the reference group and the responses from the remaining five countries were dummy-coded with a “1” on the respective country dummy variable. thus, the dummy coding for the netherlands sample served as a comparison group against which the samples drawn from the other countries are contrasted. for the data analysis, we estimated the model parameters with each of the control variables as well as with all control variables. however, the analysis reveals that none of the control variables is significantly related to the decision of enterprises to assimilate grid technology as green it strategy (see figure 2). this means that the survey data does not support the hypothesis that large firms are more (or less) innovative than small firms in terms of the use of grid technology for the purpose of reducing the environmental impact of it hardware. moreover, the hypothesis that firms that have more experience in the use and utilization of grid technology are more likely to use this technology for reducing it-related energy consumption is not supported by the survey data. lastly, countryspecific characteristics and differences between the six surveyed countries do not show significant effects on the survey results. common method bias analysis method biases are one of the main sources of measurement error and most researchers agree that common method variance is a potential problem in behavioural research (podsakoff et al. 2003). in this context, common method variance refers to the variance that is attributable to the measurement method rather than to the constructs the measures are supposed to represent. a common method bias, which is a subset of method bias (burton-jones 2009), can occur if the same individual is asked to assess both the independent and dependent variables in a field-based study. this research design flaw is a frequently encountered problem especially with survey studies and constitutes a major threat to the validity for reported empirical findings (henderson & lee 1992). to test for common method bias, the statistical approach described by podsakoff et al. (2003), liang et al. (2007), and williams et al. (2003) was employed. in doing so, a common method construct whose indicators included all the indicators used in the empirical research model was included in the pls model. then, the variances explained by the common method construct relative to that explained by the substantive constructs were assessed. as depicted in table 6 in the appendix, the average explained variance by the substantive constructs is 0.772 while the average variance explained by the common method construct is 0.001. given the low and in all cases insignificant method variance values, it can be concluded that common method bias did not impact the results. australasian journal of information systems volume 17 number 1 2010 97 discussion the results of the data analyses suggest that three of the four proposed hypotheses (h1, h3, h4) are supported by the survey data. the r 2 value of the dependent variable is satisfactorily high, indicating a high degree of explanatory power of its predecessors. the results suggest that pressure for environmental sustainability as well as coercive and normative pressure positively impact grid assimilation of enterprises for the purpose of reducing energy consumption. unfortunately, due to the insignificant path coefficient, it cannot be concluded that mimetic pressure drives grid assimilation for the purpose of reducing the environmental impact of it hardware. the analysis of the newly developed construct sust and its measurement items leads to the conclusion that the financial services industry is facing economic, political/governmental, and social pressure for environmental sustainability. due to this, the construct sust and its highly significant indicators are shown to be suitable for measuring the pressure for environmental sustainability and can therefore be used in future empirical studies on green it adoption. the analysis of the first hypothesis leads to the conclusion that pressure for environmental sustainability positively affects grid assimilation as a means to decrease the environmental impact of it usage. this finding is supported by the fact that when dropping (or adding) the sust construct from the research model, the change in r 2 is 0.14, which further emphasizes the strong ecological pressure (financial) institutions are facing. this is reasonable due to the fact that the rising energy costs, governmental regulations, and the increasing positive public perception for green it initiatives force enterprises to implement environmentally sound it infrastructures. since grid technology supports enterprises in developing green infrastructures relatively fast, it is likely that pressure for environmental sustainability significantly drives grid assimilation. in addition, the study results indicate that coercive pressure drives the assimilation of grid technology in the industry domain for the purpose of reducing energy consumption of it hardware. since enterprises are forced to meet the regulatory requirements for environmental protection and due to the capability of grid technology to effectively implement a powerful and at the same time an energyefficient it infrastructure, coercive pressure positively impacts grid assimilation. this is consistent with extant literature that hypothesizes that government and industry regulations exert coercive pressure on enterprises and decisively drive the assimilation of new technologies (ang & cummings 1997; zhu et al. 2006). moreover, our study results support the empirical findings by chen et al. (2009) as well as clemens and douglas (2006) that coercive forces encourage organizations to focus on green practices, which reflects the effectiveness of regulatory efforts in guiding green behaviours across organizations. furthermore, the survey data emphasizes that normative pressure positively affects grid assimilation. this can be explained by the fact that, due to the competitive conditions in the industry, the demand for more storage and computing capacity rises. this leads to the need of enterprises to build up a scalable it infrastructure that thereby contributes to the reduction of it-related energy consumption. grid technology is a relatively new technology but the number of grid adopters is rising rapidly, as is evidenced by the sample characteristics depicted in table 2. due to this, the move to a grid infrastructure can be seen as a process of professionalization that is encompassed by normative pressure. the finding that normative pressure positively impacts on the adoption of green practices is consistent with the empirical results provided by molla (2009) indicating that green practices are driven by normative pressure in terms of pressure for eco-effectiveness and eco-legitimacy. unfortunately, the hypothesis that mimetic pressure positively affects grid assimilation as a means to reduce the environmental impact of it hardware is not supported by the survey data. this might be explained by the fact that the ecological benefits of grid technology are not well known or not even australasian journal of information systems volume 17 number 1 2010 98 discussed in the industry and in scientific literature. due to this lack of knowledge about the ecological benefits of grid technology and the relatively small but increasing number of grid adopters, the so-called “bandwagon phenomena” are currently not induced by competitors. this result is not consistent with the findings provided by chen et al. (2009) that found a positive impact of mimetic pressure on the adoption of green practices (pollution prevention, sustainable development practices, and product stewardship). regarding the newly developed construct assm and its highly significant indicators, the results of the analysis support the hypothesis that grid technology is capable of reducing energy consumption of it hardware components. due to this, the construct assm and its measurement items are shown to be suitable for measuring the motivation of enterprises to use grid technology as an implementation of green it infrastructure and can therefore be used in future empirical studies. conclusion and further research as will be presented in the following subsections, the results of our study provide some interesting contributions to theory and practice, but also contain some limitations. finally, opportunities for further research are discussed. theoretical and practical contribution prior literature mainly emphasized the benefits of grid technology with regard to increased computing and storage capacity although the results of the study show that there are also benefits of grid technology towards environmental sustainability. therefore, the reduced environmental impact of grid technology in contrast to non-grid it environments should be promoted as another central benefit of grid computing in the extant literature. the results of the empirical field study provide a valuable contribution to both theory and practice. since little empirical research has been conducted on green it so far, this article provides empirical evidence on the relationship between green it and grid technology. moreover, this article is one of the first empirical researches conducted to analyze the role of institutional pressure as well as pressure for environmental sustainability in the context of grid assimilation. the main theoretical contribution of the article is the analysis of the pressure for environmental sustainability in the context of it assimilation. the results indicate that this kind of environmental pressure complements institutional pressures, which, to the best of our knowledge, has not been analyzed in the prior literature. besides the theoretical contribution, the results are of interest for enterprises that are planning to reduce power consumption of their it systems while not cutting back on compute and storage capacity for their departments. grid technology has shown to be suitable for the implementation of green it concepts leading to increased competitiveness of the assimilating enterprise. most green it efforts take too long since many enterprises cannot afford a short-term replacement of their existing systems with newer ones. our empirical results indicate that grid technology provides a remedy to this problem by giving enterprises the opportunity to build up an energy-efficient it infrastructure with large computing and storage capacity by connecting existing it hardware without the necessity to invest in new hardware. limitations and further research despite rich implications, this exploratory study has some limitations to acknowledge. although gerbing and anderson (1988) as well as nunnally (1978) state that each construct should be measured by at least two items so that both measurement reliability and construct validity can be assessed, the average number of indicators used is literature is three (chin et al. 2003), which is equal australasian journal of information systems volume 17 number 1 2010 99 to the minimum recommended indicators per construct in the sem literature (bollen 1989). therefore, future studies may employ additional items to adequately assess the newly developed assm construct that was measured using two items. another limitation is that our study results do not support our hypothesis that mimetic pressure positively impacts on the adoption of green practices (as, e.g., indicated by chen et al. 2009). although the measures were adapted from well established literature (i.e., liang et al. 2007, teo et al. 2003), future research is needed to analyze whether the measurement instrument items are appropriate for measuring mimetic pressure in the context of grid assimilation in the financial services industry. moreover, it has to be analyzed if there are mediating agencies that are involved in the causal relationship between mimetic pressure and grid assimilation, such as, e.g., top management support. empirical results provided by liang et al. (2007) in the context of erp assimilation indicate that top management serves as main human agency for absorbing and conversing institutional pressure arising from the environment to operational course of action. therefore, the inclusion of top management support as mediator between mimetic pressure (as well as the other three pressures analyzed in our study) and grid assimilation might provide more insights about the motivation of (financial) institutions to adopt grid technology for the purpose of reducing the environmental impact of their it systems. moreover, to refine the current research model, further research will be performed to incorporate additional predecessors of green technology assimilation in the model and to identify mediating or moderating factors. since the study was conducted in the financial services industry, the results are only indicative for other industries and limited to a specific technology (i.e., grid technology). further research is needed to analyze if the different types of pressures exert the same impact on grid technology assimilation in other industries compared to the financial services industry. moreover, future studies may analyze other technologies that are useful in reducing the environmental impact of it equipment. lastly, further research is needed to establish a research model that analyzes the predecessors of green it practices in general, thereby extending the recent empirical work on green it adoption provided by, e.g., chen et al. (2009) and molla (2009). although the study has some limitations, the findings provide the basis for future research to further investigate the environmental impact of grid technology or it in general. references aiken, l.s. & west, s.g. (1991) multiple regression: testing and interpreting interactions, sage publications, newbury park, ca, usa. allarey, j., george, v. & jahagirdar, s. (2008) “power management enhancements in the 45nm intel core microarchitecture”, intel technology journal, vol 12 no 3, pp 169-178. anderson, j.c. & gerbing, d.w. (1988) “structural equation modeling in practice: a review and recommended two-step approach”, psychological bulletin, vol 103 no 3, pp 411-423. ang, s. & cummings, l.l. (1997) “strategic response to institutional influences on information systems outsourcing”, organization science, vol 8 no 3, pp 235-256. barclay, d., thompson, r. & higgins, c. (1995) “the partial least squares (pls) approach to causal modeling: personal computer adoption and use as an illustration”, technology studies, vol 2 no 2, pp 285-309. beck, r., schwind, m. & hinz, o. (2008) “grid economics in departmentalized enterprises”, journal of grid computing, vol 6 no 3, pp 277-290. australasian journal of information systems volume 17 number 1 2010 100 bela, h. & venkatesh, v. (2007) “assimilation of interorganizational business process standards”, information systems research, vol 18 no 3, pp 340-362. biros, d., sikolia, d. & hass, m. (2008) “the perceived dichotomy between current green it initiatives and information security”, cutter it journal, vol 21 no 2, pp 26-31. bohrer, p., elnozahy, e.n., keller, t., kistler, m., lefurgy, c., mcdowell, c. & rajamony, r. (2002) “the case for power management in web servers”, graybill, r. & melhem, r. (eds.) power aware computing, series in computer science, kluwer academic publishers, norwell, usa, pp 261-289. bollen, k.a. (1989) structural equations with latent variables, john wiley and sons, new york, usa. bollen k.a. & lennox, r. (1991) “conventional wisdom on measurement: a structural equation perspective”, psychological bulletin, vol 110 no 2, pp 305-314. brislin, r. (1986) “the wording and translation of research instruments”, lonner, w.j. & berry, j.w. (eds.) field methods in cross-cultural research and methodology series, sage publications, beverly hills, usa, pp 137-164. burton-jones, a. (2009) “minimizing method bias through programmatic research“, mis quarterly, vol 33 no 3, pp 445-471. campbell, d.t. & fiske, d.w. (1959) “convergent and discriminant validation by the multitraitmultimethod matrix”, psychological bulletin, vol 56 no 2, pp 81-105. cassel, c., hackl, p. & westlund, a. (1999) “robustness of partial least-squares method for estimating latent variable quality structures”, journal of applied statistics, vol 26 no 4, pp 435-446. chen, c. (2001) “design for the environment: a quality-based model for green product development”, management science, vol 47 no 2, pp 250-263. chen, a.j., watson, r.t., boudreau, m.-c. & karahanna, e. (2009) “organizational adoption of green is & it: an institutional perspective”, proceedings of the 30th international conference on information systems (icis‟2009), phoenix, usa, december 2009. chin, w.w. (1998) “the partial least square approach to structural equation modeling”, marcoulides, g.a. (ed.) modern methods for business research, lawrence erlbaum associates, mahwah, nj, usa, pp 295-336. chin, w.w. (2001) “pls-graph user‟s guide, version 3.0”, http://www.pubinfo.vcu.edu/carma/documents/oct1405/plsgraph3.0manual.hubona.pdf accessed 12 march 2010. chin, w.w., marcolin, b.l. & newsted, p.r. (2003) “a partial least squares latent variable modeling approach for measuring interaction effects: results from a monte carlo simulation study and an electronic-mail emotion/adoption study”, information systems research, vol 14 no 2, pp 189-217. clemens, b. & douglas, t.j. (2006) “does coercion drive firms to adopt „voluntary‟ green initiatives? relationships among coercion, superior firm resources, and voluntary green initiatives”, journal of business research, vol 59 no 4, pp 483-491. cohen, j. & cohen, p. (1983) applied multiple regression/correlation analysis for the behavioral sciences, lawrence erlbaum associates, hillsdale, nj, usa. cronbach, l.j. (1951) “coefficient alpha and the internal structure of tests”, psychometrika, vol 16 no 3, pp 297-334. australasian journal of information systems volume 17 number 1 2010 101 daly, m. & butler, t. (2009) “environmental responsibility and green it: an institutional perspective”, proceedings of the european conference on information systems (ecis‟2009), verona, italy, june 2009. davison, a.c. & hinkley, d.v. (2003) bootstrap methods and their application, 2nd edition, cambridge university press, new york, usa. deshpande, r. (1983) “„paradigms lost‟: on theory and method in research in marketing”, journal of marketing, vol 47 no 4, pp 101-110. diamantopoulos, a. & singuaw, j.a. (2000) introducing lisrel, sage publications, london, uk. diamantopoulos, a., riefler, p. & roth, k.p. (2008) “advancing formative measurement models“, journal of business research, vol 61 no 12, pp 1203-1218. dimaggio, p.w. & powell, w.w. (1983) “the iron cage revisited: institutional isomorphism and collective rationality in organizational fields”, american sociological review, vol 48 no 2, pp 147-160. efron, b. & tibshirani, r.j. (1993) an introduction to the bootstrap, chapman and hall, new york, usa. elliot, s. (2007) “environmentally sustainable ict: a critical topic for is research”, proceedings of the pacific conference on information systems (pacis‟2007), auckland, new zealand, july 2007. fichman, r.g. (2001) “the role of aggregation in the measurement of it-related organizational innovation”, mis quarterly, vol 25 no 4, pp 427-455. fornell, c. & bookstein, f.l. (1982) “two structural equations models: lisrel and pls applied to consumer exit-voice theory”, journal of marketing research, vol 19 no 4, pp 440-452. fornell, c. & larcker, d.f. (1981) “evaluating structural equation models with unobservable variables and measurement error”, journal of marketing research, vol 18 no 1, pp 39-50. foster, i. & kesselman, c. (1999) the grid: blueprint for a new computing infrastructure, morgan kaufmann, san francisco, usa. foster, i., zhao, y., raicu, i. & lu, s. (2008) “cloud computing and grid computing 360-degree compared”, proceedings of the ieee grid computing environments workshop, austin, usa, november 2008. gartner (2008) “gartner identifies the top 10 strategic technologies for 2009”, http://www.gartner.com/it/page.jsp?id=777212 accessed 17 march 2010. gefen, d. (2003) “assessing unidimensionality through lisrel: an explanation and example”, communications of the association for information systems, vol 12 no 2, pp 23-47. gefen, d., straub, d. & boudreau, m.c. (2000) “structural equation modeling and regression: guidelines for research practice”, communications of the association for information systems, vol 4 no 7, pp 1-77. gerbing, d.w. & anderson, j.c. (1988) “an updated paradigm for scale development incorporating unidimensionality and its assessment”, journal of marketing research, vol 25 no 2, pp 186192. hair, j.f., black, w.c., babin, b.j. & anderson, r.e. (2010) multivariate data analysis: a global perspective, 7th edition, pearson education inc., upper saddle river, nj, usa. henderson, j.c. & lee, s. (1992) “managing i/s design teams: a control theories perspective”, management science, vol 38 no 6, pp 757-777. hendry, j.r. & vesilind, p.a. (2005) “ethical motivations for green business and engineering”, clean technologies and environmental policy, vol 7 no 4, pp 252-258. australasian journal of information systems volume 17 number 1 2010 102 henseler, j., ringle, c.m. & sinkovics, r.r. (2009) “the use of partial least squares path modeling in international marketing”, advances in international marketing, vol 20, pp 277-319. huang, a.h. (2009) “a model for environmentally sustainable information systems development”, journal of computer information systems, vol 49 no 4, pp 114-121. hui, b.s. & wold, h. (1982) “consistency and consistency at large in partial least squares estimates”, jöreskog, k.g. & wold, h. (eds.) systems under indirect observation: prediction, structure, causality, part ii, north-holland publishing co., amsterdam, netherlands. janssen, m.a. & ostrom, e. (2006) “resilience, vulnerability, and adaptation: a cross-cutting theme of the international human dimensions programme on global environmental change”, global environmental change, vol 16 no 3, pp 237-239. jarvis, c.b., mackenzie s.b. & podsakoff, p.a. (2003) “critical review of construct indicators and measurement model misspecification in marketing and consumer research”, journal of consumer research, vol 30 no 2, pp 199-218. joseph, e., snell, a. & willard, c. (2004) “council on competitiveness study of u.s. industrial hpc users”, technical report, idc, july 2004. kuo, b.n. & dick, g.n. (2009) “the greening of organisational it: what makes a difference?”, australasian journal of information systems, vol 16 no 2, pp 81-92. idc (2008) “idc predictions 2009: an economic pressure cooker will accelerate the it industry transformation”, http://www.idc.com/getdoc.jsp?containerid=215519 accessed 05 march 2010. kurp, p. (2008) “green computing”, communications of the acm, vol 51 no 10, pp 11-13. liang, h., saraf, n., hu, q. & xue, y. (2007) “assimilation of enterprise systems: the effect of institutional pressures and the mediating role of top management”, mis quarterly, vol 31 no 1, pp 59-87. maccallum, r.c., widaman, k.f., zhang, s. & hong, s. (1999) “sample size in factor analysis”, psychological methods, vol 4 no 1, pp 84-99. marcoulides, g.a. & saunders, c. (2006) “pls: a silver bullet?”, mis quarterly, vol 30 no 2, pp iii-ix. meyer, j. & rowan, b. (1977) “institutional organizations: formal structure as myth and ceremony”, the american journal of sociology, vol 83 no 2, pp 340-363. molla, a. (2009) “organizational motivations for green it: developing and exploring a green it matrix and motivation models”, proceedings of the pacific conference on information systems (pacis‟2009), hyderabad, india, july 2009. molla, a., cooper, v.a. & pittayachawan, s. (2009a) “it and eco-sustainability: developing and validating a green it readiness model”, proceedings of the 30th international conference on information systems (icis‟2009), phoenix, usa, december 2009. molla, a., pittayachawan, s., corbitt, b. & deng, h. (2009b) “an international comparison of green it diffusion”, international journal of e-business management, vol 3, no 2, pp 3-23. murugesan, s. (2007) “going green with it: your responsibility toward environmental sustainability”, cutter business-it strategies executive report, vol 10 no 8, pp 1-25. murugesan, s. (2008) “harnessing green it: principles and practices”, it professional, vol 10 no 1, pp 24-33. nord, w. & tucker, s. (1987) implementing routine and radical innovation, lexington books, lexington, ma, usa. nunnally, j.c. (1978) psychometric theory, 2nd edition, mcgraw-hill, new york, usa. australasian journal of information systems volume 17 number 1 2010 103 olson, e.g. (2008) “creating an enterprise-level „green‟ strategy”, journal of business strategy, vol 29 no 2, pp 22-30. patel, c., sharma, r., bash, c. & graupner, s. (2003) “energy aware grid: global workload placement based on energy efficiency”, proceedings of the international mechanical engineering congress and exposition, washington, dc, november 2003. pernici, b., ardagna, d. & cappiello c. (2008) “business process design: towards service-based green information systems”, proceedings of the ifip 20th world computer congress, milan, italy, september 2008. petrini, m. & pozzebon, m. (2009) “managing sustainability with the support of business intelligence: integrating socio-environmental indicators and organisational context”, journal of strategic information systems, vol 18 no 5, pp 178-191. petter, s., straub, d. & rai, a. (2007) “specifying formative constructs in information systems research”, mis quarterly, vol 31 no 4, pp 623-656. podsakoff, p.m., mackenzie, s.b., jeong-yeon, l. & podsakoff, n.p. (2003) “common method biases in behavioral research: a critical review of the literature and recommended remedies”, journal of applied psychology, vol 88 no 5, pp 879-903. porter, m.e. & van der linde, c. (1999) “green and competitive: ending the stalemate”, harvard business review, vol 73 no 5, pp 120-134. ringle, c.m., wende, s. & will, a. (2005) “smartpls 2.0”, http://www.smartpls.de, accessed 28 march 2010. rogers, e.m. (1995) diffusion of innovations, 4th edition, the free press, new york, usa. rugman, a.m. & verbeke, a. (1998) “corporate strategies and environmental regulations: an organizing framework. strategic management journal”, vol 19 no 4, pp 363-375. ryan, e.j. (2008) “building sustainable it”, cutter it journal, vol 21 no 2, pp 6-12. scaramella, j. & healey, m. (2007) “service-based approaches to improving data center thermal and power efficiencies”, idc white paper, may 2007. schwind, m., hinz, o. & beck, r. (2007) “a cost-based multi-resource auction for serviceoriented grid computing”, proceedings of the 8th ieee/acm international conference on grid computing (grid‟2007), austin, texas, usa, september 2007. sellin, n. & keeves, j.p. (1994) “path analysis with latent variables”, husen, t. & postlethwaite, t.n. (eds.) international encyclopedia of education, elsevier publishers, london, uk, pp 4352-4359. slade, p. & van akkeren, j. (2002) “business on-line? an empirical study of factors leading to the adoption of internet technologies by australian smes”, australasian journal of information systems, vol 10 no 1, pp 50-64. straub, d. (1989) “validating instruments in mis research”, mis quarterly, vol 13 no 2, pp 147169. strong, p. (2005) “enterprise grid computing”, acm queue, vol 3 no 6, pp 50-59. swanson, e.b. & ramiller, n.c. (2004) “innovating mindfully with information technology”, mis quarterly, vol 28 no 4, pp 553-583. tenenhaus, m. (2008) “component-based structural equation modelling”, total quality management & business excellence, vol 19 no 7, pp 871-886. tenenhaus, m., vinzi, v.e., chatelin, y-m. & lauro, c. (2005) “pls path modeling”, computational statistics and data analysis, vol 48 no 1, pp 159-205. australasian journal of information systems volume 17 number 1 2010 104 teo, h.h., wei, k.k. & benbasat, i. (2003) “predicting intention to adopt interorganizational linkages: an institutional perspective”, mis quarterly, vol 27 no 1, pp 19-49. teo, t.s.h., srivastava, s.c. & jiang, l. (2009) “trust and electronic government success: an empirical study”, journal of management information systems, vol 25 no 3, pp 99-131. tornatzky, l.g. & fleischer, m. (1990) the processes of technological innovation, lexington books, lexington, ma, usa. unhelkar, b. & dickens, a. (2008) “lessons in implementing „green‟ business strategies with ict”, cutter it journal, vol 21 no 2, pp 32-39. u.s. environmental protection agency (2007) “epa report on server and data center energy efficiency”, http://www.energystar.gov/index.cfm?c=prod_development.server_efficiency_study accessed 20 march 2010. vykoukal, j., wolf, m. & beck, r. (2009) “services grids in industry: on-demand provisioning and allocation of grid-based business services”, business & information systems engineering, vol 1 no 2, pp 177-184. werts, c.e., linn, r.l. & jöreskog, k.g. (1974) “intraclass reliability estimates: testing structural assumptions”, educational and psychological measurement, vol 34 no 1, pp 25-33. williams, l.j., edwards, j.r. & vandenberg, r.j. (2003) “recent advances in causal modeling methods for organizational and management research”, journal of management, vol 29 no 6, pp 903-936. zhu, k. & kraemer, k.l. (2005) “post-adoption variations in usage and value of e-business by organizations: cross-country evidence from the retail industry”, information systems research, vol 16 no 1, pp 61–84. zhu, k., kraemer, k.l. & xu, s. (2006) “the process of innovation assimilation by firms in different countries: a technology diffusion perspective on e-business”, management science, vol 52 no 10, pp 1557-1576. zhu, k., kraemer, k.l., xu, s. & dedrick, j. (2004) “information technology payoff in e-business environments: an international perspective on value creation of e-business in the financial services industry”, journal of management information systems, vol 21 no 1, pp 17-54. australasian journal of information systems volume 17 number 1 2010 105 appendix pressure for environmental sustainability (1 = strongly disagree; 7 = strongly agree) sust1 rising energy costs force our it departments to reduce it-related power consumption sust2 political pressure and governmental regulation force our it departments to reduce itrelated power consumption sust3 social pressure and positive public perception of green it initiatives force our it departments to reduce it-related power consumption mimetic pressure (1 = strongly disagree; 7 = strongly agree) mp1 our main competitors who have adopted grid technology have greatly benefited mp2 our main competitors who have adopted grid technology are favourably perceived by others in the same industry mp3 our main competitors who have adopted grid technology are favourably perceived by their suppliers and customers coercive pressure (1 = strongly disagree; 7 = strongly agree) cp1 the increasing regulatory pressure requires our firm to use grid technology cp2 the increasing customer demand requires our firm to use grid technology cp3 the competitive conditions require our firm to use grid technology normative pressure (1 = strongly disagree; 7 = strongly agree) np1 our firm‟s it service providers have already adopted grid technology np2 our firm‟s business partners have already adopted grid technology np3 the government‟s promotion of it influences our firm to use grid technology grid assimilation as green it strategy (1 = strongly disagree; 7 = strongly agree) assm1 with a virtualized grid environment, it-related power consumption of my firm could significantly be reduced assm2 by on-demand purchasing it resources from external cloud computing providers, itrelated power consumption of my firm could significantly be reduced table 5: measurement items and scales australasian journal of information systems volume 17 number 1 2010 106 construct indicator substantive factor loading (r1) r1 2 method factor loading (r2) r2 2 pressure for environmental sustainability (sust) sust1 0.862* 0.743 -0.046 0.002 sust2 0.861* 0.741 0.040 0.002 sust3 0.891* 0.794 0.004 0.000 mimetic pressure (mp) mp1 0.903* 0.815 -0.016 0.000 mp2 0.951* 0.904 -0.009 0.000 mp3 0.907* 0.823 0.025 0.001 coercive pressure (cp) cp1 0.780* 0.608 0.039 0.002 cp2 0.948* 0.899 -0.068 0.005 cp3 0.841* 0.707 0.034 0.001 normative pressure (np) np1 0.846* 0.716 0.010 0.000 np2 0.903* 0.815 -0.011 0.000 np3 0.783* 0.613 0.002 0.000 grid assimilation as green it strategy (assm) assm1 0.895* 0.801 0.015 0.000 assm2 0.908* 0.824 -0.015 0.000 average 0.772 0.001 table 6: common method bias assessment; * p < 0.01 (two-tailed) ajis special edition-electronic commerce november 1998 electronic commerce and collaboration between competing firms eberhard stickel dept. of information systems, viadrina university frankfurt (oder), po box 776,15207 frankfurt (oder), germany phone: 449 335 5534 358, fax: +49 335 5534 357, email:stickel@euv-frankfurt-o.de abstract investments in electronic commerce technology typically require large sums of money and the realisation of possible benefits is often highly uncertain. possible investors may also be confronted with the so-called free-rider-problem. innovators have to bear all development costs. once standards are established followers (free-riders) may easily imitate the investment. hence, innovators may not be able to build up sustaining competitive advantages that make their investments worthwhile. as a result, available technology may not be used in an efficient way. a typical prisoner's dilemma scenario prevails. pre-competitive collaboration may be a possible solution to this problem. the term ,,pre-competitive" refers to the possibility of joint application development and/or sharing of information, knowledge and ability. it should not be confused with collusion which may be legally restricted or even forbidden. the goal of the paper is to analyse whether there are economic incentives for pre-competitive collaboration as sketched above. the analysis is carried out with the help of a microeconomic model and techniques from game theory. introduction investments in electronic commerce technology typically require large sums of money and the realisation of possible benefits is often highly uncertain. possible investors may also be confronted with the so-called freerider-problem. innovators have to bear all development costs. once standards are established followers (freeriders) may easily imitate the investment. hence, innovators may not be able to build up sustaining competitive advantages that make their investments worthwhile. sustainability, however, is of crucial importance if itinvestments are pursued with the goal of building up competitive advantages (kettinger et al., 1994). as a result, available technology may not be used in an efficient way since potential innovators may abstain from investment. this corresponds to a typical prisoner's dilemma situation. a possible solution to this problem would be some sort of pre-competitive collaboration commitment between competing firms. pre-competitive refers to the possibility of joint application development and/or sharing of information, knowledge and ability. pre-competitive collaboration should not be confused with collusion which may be legally restricted or even forbidden (tirole, 1997). the goal of the paper is to analyse whether there are economic incentives for pre-competitive collaboration as sketched above. the analysis is carried out with the help of a microeconomic model and techniques from game theory. collaboration between firms that are located in different phases of the value chain (typically supplier-purchaserrelationships) is a quite well-studied problem. the institutional structure of supplier-purchaser-relationships has changed significantly over the last few years (bauer & stickel, 1996). more often coordination through markets and collaboration (network organisations, strategic alliances) instead of pure hierarchical coordination may be observed. the role of it in this process was investigated in the literature. malone et al. (1987) propose the socalled "move-to-the-market'-hypothesis. basically it is stated that complex products are more likely to be obtained through hierarchical man through market coordination due to communication costs. since it may decrease communication costs, malone et al. argue that a shift towards market coordination should occur. also, product specificity should decrease. unfortunately, the authors do not investigate the specificity of itinvestments themselves. highly specific it-investments may allow opportunistic behaviour and may incur additional costs (agency costs). this increase may very well exceed the savings with respect to communication costs. more work on the subject was done by clemons & reddi (1994). they proposed the so-called "move-tothe-middle'-hypothesis. it is argued that the use of it involves a learning curve. hence, long-term collaboration will be preferred to short-term market arrangements. long-term collaboration will allow companies to longer enjoy the benefits of their it-investments. empirical research basically supports this hypothesis (bauer & stickel, 1996, p. 49). the work mentioned above is concerned with issues of vertical integration and not with collaboration among competing firms (horizontal integration). the latter issue is addressed in the work presented here. it is investigated under what conditions information and/or knowledge sharing is beneficial. investments in electronic commerce technology typically are strategic investments. such investments are considered to be long-term investments with the goal to exploit possible competitive advantages. according to the literature strategies of firms may be seen as a mixture of cost reduction, product differentiation and improvement of decision making and/or planning (porter, 1985). the case of product differentiation will not be considered in this paper. as will be indicated, results are similar to the results presented here, however. 114 ajis special edition electronic commerce november 1998 investments in electronic commerce technology may have effects on costs. the primary focus, however, is associated with the demand side. possible benefits include e.g. increased customer bonding, better customer services, anticipation of changes in demand and/or consumer behaviour. generally, this implies better support for planning and decision making. also note, that demand risk is one of the most important uncertainty factors faced by today's'firms. to keep our model simple, we necessarily have to abstract from reality. based on the previous remarks we assume that the main focus of investments in electronic commerce technology lies on better support for planning and decision making on the demand side. this, in turn, may be viewed as decreasing uncertainty by means of suitable investments. uncertainty is introduced into our model by means of stochastic demand parameters. the possibility of cost reductions through investments in electronic commerce technology is addressed shortly in section 5. the paper is organised as follows. in the next section the assumptions underlying the microeconomic model will be outlined in more detail. also, a detailed literature review will be provided. section 3 sketches the derivation of our results. methods from game theory will be applied. the main steps will be motivated so that the presentation is also accessible to the non-technical specialist. the main conclusions will be drawn in the fourth section. section 5 considers issues related to cost reductions. finally we conclude by presenting a summary and pointing out further areas of possible research. for the sake of completeness and easier reference rather technical derivations have been placed in the appendix of the paper. construction of the model and literature review a review of the microeconomic literature (see e.g. fried, 1984, gal-or, 1985, li, 1985, hviid, 1989, kao & hughes, 1993) indicates that relevant factors affecting collaboration decisions between competitors are: • market structure. if perfect competition prevails collaboration is of no use. no single firm or proper subsets of firms may influence market prices and/or quantities. in a monopolistic environment there obviously is no room for collaboration. consequently the interesting market structure is an oligopoly. in oligopolistic markets decisions of a firm may directly influence prices and quantities offered of the firm and its competitors. • product relationship. products offered may be substitutes or complements. in general, we would expect that products of competing firms are substitutes. product differentiation, however, allows to vary the degree of possible substitution. in microeconomic theory homogeneous and heterogeneous markets are distinguished (see e.g. pindyck & rubinfeld, 1992, pp. 430-444). in homogeneous markets goods are perfect substitutes for each other. as a consequence, there is no product differentiation. in such markets usually quantities are set by competing firms. then, a unique equilibrium price, which holds for all competing firms results. in heterogeneous markets firms usually set prices and apply a product differentiation strategy. klemperer & meyer (1986) have shown that generally price competition is preferred by firms in an oligopolostic market if the slope of their marginal cost curve is rather flat. for the sake of simplicity we will assume constant variable costs. hence, the slope of marginal costs is zero everywhere and the result of klemperer and meyer applies. also note, that price competition prevails in a lot of interesting markets. examples are the airline and automotive industry, as well as the financial services sector. in markets with price competition, each firm acting in the market sells a certain quantity of the good offered, depending on prices set and consumer demand. • risk preferences of competing firms. in general decisions are delegated by absentee owners to managers. this is especially true in large companies. the compensation of such managers is usually tied to profits. this fact, as well as possible opportunistic behaviour and asymmetric information suggest that managers behave more or less risk averse (see e.g. kao & hughes, 1993). hence, they will not maximise expected profits as if they were risk neutral but expected utility. a utility function basically accounts for the degree of risk associated with a certain payment stream (copeland & weston, 1983). for risk averse decision makers a secure payment h is always preferred to a random payment with expected value h. hence, there is a positive risk premium. as can and will be shown the consideration of risk aversion significantly influences the results. • kind and degree of uncertainty faced by competing firms. basically we may distinguish uncertainty with respect to common or private parameters. as an example consider demand parameters. they are called common variables since they are the same for all firms and directly affect profits of each firm. on the other hand variable costs are an example of private variables. they are unique to each competing firm (note however, that some components of variable costs may be considered as being common (e.g. costs influenced by the overall economy), while others (e.g. firm-specific production costs) typically are private). 115 ajis special edition-electronic commerce november 1998 • distribution of knowledge and ability. knowledge and abilities to develop, adapt and use a certain technology may not be equally distributed. the interesting question is whether sustaining competitive advantages may or may not be built on specific knowledge and ability (see choudhoury & sampler, 1997 for a discussion of information specifity). it will be shown that under quite reasonable assumptions sharing of knowledge and ability between competitors is beneficial for ail of them. again it should be pointed out that collusion is not intended (e.g. price and/or quantity fixing). this incentive for sharing is a strong case for collaboration between competitors. note, that in particular competitive effects need to be monitored carefully. in the following the homogeneous product case where quantities are set and prices determined accordingly will be analysed. the analysis of the heterogeneous product case (i.e. the case of product differentiation) proceeds along the same lines but is technically more complicated. results will be reported elsewhere in a short while. note, however, that the spirit of the results does not change significantly in the heterogeneous product case (stickel, 1998). to simplify the analysis the case of two competing firms (duopoly) will be considered. it is assumed that both firms are risk averse and that the utility function of firm fj is given by (1) m,(*) = l-exp(-a,*) for i = 1,2. this utility function corresponds to the case of constant risk aversion according to the arrow-pratt risk measure (copeland & weston, 1983 and dossani et al., 1995). the smaller <xj the less risk averse is the corresponding firm fj. in the limiting case as cq tends to zero the case of a risk neutral decision maker is obtained. throughout the paper we will assume that firm f\ is less risk averse than firm f2. consequently, o.\ < 0/2 holds. the price-demand function faced by the two firms is given by (2) p = a-(xl+x2). in (2) a is a stochastic demand parameter that introduces uncertainty into our model. note, that the decision of a firm influences the price p and hence the optimal decision with respect to the quantity offered on the market by its competitor. the parameter a is assumed to be normally distributed with mean p. and standard deviation o. obviously, negative prices and quantities should be ruled out. this certainly requires positivity of a. from elementary statistics it is well-known, that the probability of a being negative is less than 0.5% if u > 2.57 a holds (pindyck & rubinfeld, 1991, p. 31). therefore, if the mean is large enough with respect to the standard deviation a may be assumed to be positive with near certainty (in fact the probability of error may be made arbitrarily small by increasing the mean and/or decreasing the standard deviation). if variable costs of the i-th firm are denoted by c; its profits are given by n; (x} ,x2) = pxf c,.*,. = (a-x}-x2cf ) x. . a risk neutral firm would maximise the expected value of this expression given the action xj (j * i) of its competitor. a risk averse firm, however, would maximise its expected utility where u\ is given by (1). a formal analysis of this optimisation problem would be difficult due to the exponentials in (1). the normal distribution of the parameter a and the special form of the utility functions in (1) allow to restate the optimisation problem. the i-th firm maximises the objective function (3) g /(n j(jrpac2)) = £ni.(x1,x2)-^-var(n,(*,,*2)). the expression in (3) is the so-called certainty equivalent of the corresponding expected utility. a decision maker is indifferent between a normally distributed payment stream with mean u, and standard deviation a and a secure payment in the amount of the certainty equivalent (copeland & weston, 1983). consequently, maximising expected utility and maximising the certainty equivalent will lead to the same optimal actions. note, that the problem given by (3) is much simpler than the original problem. in particular, (3) allows the derivation of closed-form optimal solutions. this greatly simplifies the analysis. we also assume that both firms produce at the same constant variable costs c. hence, there is no cost leader. the cost leader usually is rewarded with higher profits and a higher market share. this effect may strengthen or weaken the effects we wish to study in this paper. since our goal is to analyse incentives of pre-competitive collaboration and not primarily the effects of cost leadership the assumption of zero variable costs is certainly justified. it is then very easy to show, that c = 0 may be assumed without loss of generality (see the appendix, lemma 1 for a proof). finally, we assume that both firms announce their quantities ki simultaneously. hence, there is no early announcement by one of the competing firms. an early announcement is usually rewarded with a larger market share. such effects may again strengthen or weaken the results presented in the paper. the assumption of 116 ajis special edmon-electronic commerce november 1998 simultaneous price announcement allows to compute the so-called cournot-equilibrium (tirole, 1997, pp. 218221). more details of this computation will be outlined later. both firms may now use information technology to reduce demand uncertainty. to be specific they may invest and eventually predict the stochastic parameter a. in that case all uncertainty is removed for the investing firm. another possibility would be the use of it to reduce production costs. the case of cost reductions is discussed briefly in section 5. the interested reader is referred to stickel (1997) where a detailed treatment may be found. the options faced by the competing firms lead to four possible scenarios: 1. none of the firms tries to reduce risk by predicting the stochastic parameter a (peace, scenario 1), 2. firm fi tries to predict a, while firm f2 does not do so (offense, scenario 2), 3. firm f2 tries to predict a, while firm fi does not do so (defense, scenario 3), 4. both firms try to predict a (war, scenario 4). n case only one firm invests the rival firm knows about this investment. of course it does not know the results of the prediction (this may be changed by means of collaboration, however). the model, although equipped with quite restrictive assumptions, is a quite flexible one. various extensions, e.g. use of a more realistic price-demand-function or use of different distributions of a, ait possible and will be indicated later in more detail. most of these extensions do not alter the results presented in the third section, however. variants of the model used in this paper have been applied in the literature to study the problem of information transmission in a duopolistic (more generally oligopolistic) market environment. fried (1984) investigates incentives for information production and disclosure in a duopolistic environment. firms are assumed to be risk neutral. the focus is on unknown variable costs. in fried's paper variable costs have two components of uncertainty. first, there is a common component related to the economy, second, there is a firm-specific private component. fried assumes, that decisions are first made about information production and disclosure. the decisions made by each firm are known and enforceable. production decisions will be next. fried shows that disclosure of private cost components usually is beneficial, while disclosure of common components generally does not pay off. gal-or (1985) and li (1985) study the effects of information sharing associated with an uncertain demand parameter a. note, that this is a common parameter. as fried, they assume risk neutrality of the competing firms. when a firm observes a signal of low demand (this corresponds to a low value of the parameter a) disclosure of this information may prevent its competitor from overproducing. on the other hand, disclosure of a signal indicating high demand may result in a higher production of the competitor. both authors investigated which effect dominates. their result suggests that information sharing is not an optimal strategy. gal-or assumed a normal distribution of the parameter a, li allowed more general probability distributions. li also analysed the case of sharing private cost information and confirmed the results of fried. to summarise, sharing information about private parameters pays off, sharing information about common parameters does not. these results change significantly if the assumption of risk neutrality is relaxed. as was already indicated, the delegation of decisions from owners to managers naturally introduces risk aversion. palfrey (1982) provided first insights into this situation. he showed that even the private use of information may be harmful. in particular, a firm that is less risk averse than it competitor(s) is rewarded for taking higher risks. if such a firm tries to reduce risk by using information the more risk averse competitor does not have, it may very well be worse off afterwards. stickel (1995) has extended the analysis of palfrey to all four possible scenarios and provided an explanation for the productivity paradox of information technology. the more risk averse firm fi has incentives to collect information and to reduce uncertainty. the less risk averse firm f\ should then act as a follower. its expected profits (and expected utility) will be lower than in the case when none of the firms collects information but slightly higher than in the case when only firm f2 collects and uses information. more work on the subject stems from hviid, 1989 and kao & hughes, 1993. hviid investigates the incentives of a risk averse firm to share information about the parameter a. it is assumed that information production in general is beneficial. he showed that the results obtained in the case of risk neutrality need not be true anymore. basically, the results of palfrey are confirmed. the main focus of hviid's paper is on mechanisms to assure correct information transmission in case information sharing is beneficial. in particular hviid investigates the roles of information brokers (the term used in his paper was auditor). dynamics of decision making (analysis of all possible four scenarios) and costs, as well as incentives for information production and acquisition (e.g. costs to implement a suitable information system and to maintain it) are not considered, however. kao and hughes extend the analysis of hviid to the case of sharing firm-specific cost information. the case of heterogeneous goods is not treated in the literature as extensively as the case of homogeneous goods. in case of risk neutral decision makers the results published indicate that it now pays off to share information about common parameters while it is not beneficial to share information about private variables (gal-or, 1986). 117 ajis special edition-electronic commerce november 1998 to summarise, in case of homogeneous goods there seem to be incentives for sharing information about common variables if both decision makers are risk averse. then there is room for (pre-competitive) collaboration in the form of information and/or knowledge sharing. it is left open in the literature, however, whether there are any incentives at all for information production if risk aversion is assumed. but collaboration issues need to be closely tied to investment incentives and decisions. this point will be analysed in more detail now. note, that the problem investigated is a key one in the area of electronic commerce investments. firms have to decide whether it is beneficial to pursue often costly and highly uncertain it-investments. collaboration may be a way to reduce uncertainty and to share costs. this, in turn may influence the initial investment decision. hence, the issues of investment incentives and collaboration may not be separated but should be studied together. derivation of results within this section the model presented will be formally analysed. the section is rather mathematical and may be skipped by the non-technical reader. important equations have been numbered and will be referenced later on. we begin the analysis by considering the situation at the beginning (status quo, scenario 1). both firms face uncertain demand introduced by means of the normally distributed parameter a. as was already indicated each firm maximises its objective function (3) given the quantity offered by its competitor. the result is a reaction function showing the optimal response of firm i to an arbitrary quantity set by firm j (i * j). to be specific, consider firm f\. its profits depending on a and x2 are given by ul(xl,x2) = pxl = (a-xl-x2)xl. the expected value of the profits is given by (4) eti, cx,,jc2) = ol-x,-x,)*,, the variance by (5) vflr(n,(x,,x2)) = o2*i2. combining (4) and (5) immediately yields the objective function of the first firm. it is given by a, , . (6) gl(xl,x2) = (\i-xl-x2)xi-—axl. in the next step (6) has to be maximised with respect to x\ holding x2 fixed. first order conditions for a maximum yield the reaction function u.-jc, the situation for f2 is identical. by exchanging indices we immediately get its reaction function , n i1"*'(8) *2 (*,) = -2-2 + a2a up to now two reaction functions have been determined. in an ideal case (7) and (8) coincide. suppose e.g. that f2 sets a quantity x2. from (7) the optimal answer of f! in the form of its quantity x\ set may be computed. using this in (8) yields another quantity x2 which in an ideal case coincides with the earlier choice of f2. then an equilibrium would have been obtained. by assumption, prices are set simultaneously. an equilibrium may thus be computed by solving the two reaction functions simultaneously for the unknown quantities. this is easily done, since the reaction functions form a system of two linear equations in two unknowns. equilibrium quantities are given by x\ ~ 1 + cc20 2 l + a,cj2 3+2(a1+a2)a2+a,a2a4 ' 2 3+2(a1 +a2)a2 +a,a2c4 ' first note, that in equilibrium the quantities offered are positive and hence meaningful. also, note, that the less risk averse firm offers a higher quantity and hence has higher (expected) sales, higher (expected) profits as well as a larger market share. to see this divide x\ by x2 and note that di < 0% by assumption. the ratio is larger than 1. from this we may conclude that there is a reward for taking risk. the less risk averse firm is rewarded by higher sales, higher expected profits and a larger market share than its more risk averse competitor. from (9) and (6) expected profits and expected utilities (to be precise the certainty equivalent of expected utility) may be easily computed. for future reference the results are stated in equations (10) and (11). here p^ and cy denote expected profits and certainty equivalent for firm ft in scenario j: (h-tt2o2)a(l + a,a2) (l + a,a2)2(l + a2a2) (io) ^,-j i (3 + 2(0, +a2)o2+a1a2o4)2 ' 118 ajis special edition electronic commerce november 1998 + a2a2)2(2-kx,q2) 2 (l + a,a2)2(2-kx2a2)= ' 21 2 (3 + 2(a1+a2)a2+a1a2a4)2 ' 21 2 (3 + 2(a, + a2)a2 + a,a2a4)2 ' this completes the analysis of the first scenario. next, we assume that firm f, predicts a by introducing a suitable information system, while firm f2 does not. the analysis basically proceeds as in the first case discussed previously. for firm fi all uncertainty is removed. let a* denote the prediction of the parameter a. given this prediction ni(jci,x2) is no random variable anymore. hence its variance is zero and f, simply maximises profits. this leads to the objective function gl(xl,x2) = (a*-xl-x2)xl. first order conditions for a maximum lead to the reaction function (12) xl(x2>a*) = -(a*-x2). profits for the second firm are given by (13) n2(xi(d),x2) = (a-xl(a)-x2)x2. taking expectations in (13) yields £tt2 (jc, (a), x2 ) = (|i xl (ji) x2 )x2 . finally, we need to compute the variance of it2. for this we get (14) var(u2 (x} (a), x2 ) = x2 2var(a jc, (a)) = x2 2 [a 2 + var( x, (a)) 2cov(a,x} (a))}. from (12) it follows and 1 1 , cov(a,xl(d)} = -cov(o,a) =-a2. collecting terms in (14) finally yields the objective function cx (15) g2(x1(a\x2) = (\l-xl(ll)-x2)x2— g x2. the difference in the objective function stems from the fact that f2 knows about the prediction of fj. it does not know the predicted value a*, however. since both firms announce their quantities offered simultaneously we may not substitute (12) in the formula for #2. otherwise we would assume that f] announces its quantities earlier than f2 (cf. pindyck & rubinfeld, 1992, pp. 438-439). using first order conditions (15) is maximised with respect to *2. after differentiation (12) is used and the equilibrium price results. using this in (12) yields the equilibrium on average fi charges the price ot,a2) note again, that expected prices (16) and (18) of both firms are positive and hence meaningful. also, if a2 is small enough the expression in (17) is positive with probability arbitrarily close to zero. in order to compare the results to the ones in scenario 1 we need to take expectations. note, that uncertainty for the first firm is only resolved in a single period. to decide about the benefits of a suitable it-investment that allows information production more periods need to be considered. thus, over time there still is uncertainty for both firms. expected profits of both firms may be easily computed. we get ^ ,j2)2+4a2(5+a2az) (4 + a2a2) (19) f '2~ 4(6 + a2a2)2 ' ^ ~ ^ 119 ajis special edition electronic commerce november 1998 the certainty equivalent for the second firm is also easily derived. a straightforward computation yields , (8 + a2g2) (20) ^•"vo.o')'the situation for firm fi is a little bit more tricky and complicated. the utility function given by (1) was substituted by the objective function (3). it was argued that the maximisation of (3) is equivalent to the maximisation of expected utility. this is true since (3) is the corresponding certainty equivalent in case that the underlying random variable is normally distributed. clearly this holds for a. in the first scenario profits and objective functions have been linear functions of a. this is also true for the profits of firm f2 and the objective functions of both firms in the second scenario. hence, instead of computing expected utility of normally distributed profits we might as well use (3) with expected value and variance of the profits. profits of firm fi in scenario 2 are quadratic functions of a, however. to be specific, we have ( 6 a 2 2 clearly, i~ii(a*) is not normally distributed any more. it is still possible, however, to compute the certainty equivalent of (21). then it is possible to compare the outcomes of scenario 1 and scenario 2 for firm f], too. the important formula that allows to derive the certainty equivalent of (21) is stated in the appendix (lemma 2). the result of an application of this formula to (21) is (4 + oc2a2)2 (22) c12 = 2 2 ioo(6 + a2a't 2a, in (22) ln() denotes the natural logarithm. this completes the analysis of scenario 2. scenario 3 is symmetric to scenario 2. in order to get results we may simply switch the roles of firm fi and f2. in the formulas only indices need to be interchanged. finally, in the fourth scenario both firms are able to ex ante predict the parameter a. in a single period all uncertainty is removed. hence, a symmetric solution with the same prices and expected profits should be expected. the objective function of the i-th firm is given by g, (*,, x2 ,a*) = (a * -xl x2 ) x f . the computation of the reaction function yields xl(x2,a*)=-(a*-x2), x2(x},a*)=-(a*-x}) which clearly are symmetric. equilibrium prices are given by * _£i!*,(**)-*2(fl*)3 . the profit functions of both firms are a =fl 1 2 9 hence, ( 2\ j (23) p14 = p24 = e|j-j = -ai2 +a2). it remains to compute the certainty equivalent. an application of lemma 2 of the appendix yields (24) c,4 = this completes the analysis of scenario 4. 120 ajis special edition electronic commerce november 1998 discussion of results for easier reference we will collect together expected profits and expected utilities for the four scenarios analysed. table 1 contains expected profits, table 2 the certainty equivalents. 2 (l + a2a2)2(l + a,a2) 11 (3 + 2(a, +a2)a2 +a,a2cr4)2 21 r'2 4(6+ a2a2)2 (4 + a.a2) (n 2+a 2 ) / is ^ (6+a.c2)2 23 ^,4 = ^4=-9-(m 2 (l + a,a2)2(l + a2a2) ^ (3 + 2(a,+a2)c2+a,a2a4)2 •a2<*2) r 2 (4 + a2c2) 22 (6 + a2a2)2 (4 + a,a2)2+4a2(5 + a,a2) 4(6 + a,a2)2 table 1: expected profits py for firm i in scenario y 2(a,+a2)a2+a,a2a4)2 21 2 (3 + 2(a,+a2)a2+a,a2a4)2 ,414 ln u+^2aa2 2a 9— t+^ — ln i+^a2a,a2 2a, i, 9 ' table 2: certainty equivalents cy for firm i in scenario; note, that both firms decide on the basis of their certainty equivalents. instead of performing a formal analysis of the differences in various scenarios we will graph the certainty equivalents and expected profits for selected parameters u., a, <xi and 02the results derived will hold in general, however. let cti = 0.4, a2 = 0.7, n = 15 be given. fig. 1 shows the differences cn ci2 as a function of the standard deviation a. due to the non-negativity constraints it does not make sense to consider values of a > 5. for small and medium risks which correspond to small values of a there is no incentive for firm f\ to be a first mover. moreover, the competitor of f} would benefit, although it does not undertake any investments. if risk increases and development as well as maintenance costs are small enough the situation changes. then f\ may have first mover advantages. 121 ajis special edition electronic commerce november 1998 -0.5-" f21(0) -1.5' -2• fig. 1: comparison of first and second scenario for firm f\ (left) and f2 (right) 3.5 — 2.5" 1.5' 0.5' -0.5' 122 ajis special edition electronic commerce november 1998 fig. 2: comparison of first and third scenario for firm f} (left) and f2 (right) fig. 2 compares scenario 1 to scenario 3 by considering the differences cii-ci3. the second firm has first mover advantages for smaller and larger risks if development and maintenance costs are small enough. for medium risks the first mover advantages disappear. by comparing the graphs in fig. 1 and fig. 2 it is evident, that for medium risks (around a =2) none of the firms has first mover advantages even if development and maintenance costs are zero. next, we will compare the first to the fourth scenario. fig. 3 visualises the differences circw for the two firms. for larger risks both firms benefit, for smaller risks only the more risk averse firm does (this result holds in general if the difference in risk aversion of the firms is not too small, otherwise there is an interval [0,o*] with small a* such that firm f2 does not benefit for risks within that interval). finally fig. 4 compares the third to the fourth scenario by graphing the differences cjj-c^. except for small risks f} always benefits and consequently should act as a follower. f!3(0) -0.5-' f23(0) 1 -• -1.5-' fig. 3: comparison of first and fourth scenario for firm fi (left) and f2 (right) the last observations made have some interesting consequences. neglecting development and maintenance costs, if the differences in risk aversion are sufficiently high the second firm should always try to be a first mover. firm fi should then be a follower except if risks are small. then the fourth scenario would constitute an equilibrium (in game theory this is called a nash equilibrium, none of the firms has an incentive to change its behaviour, cf. pindyck & rubinfeld, 1992, pp. 431-435 and 470-476). for small risks the third scenario would constitute an equilibrium. 123 ajis special edition electronic commerce november 1998 -0.2' -0.4' -0.6' -0.8--1' -1.2' -0.5-' f25co) -1.5-' fig. 4: comparison of third and fourth scenario for firm f[ (left) and f2 (right) we will now continue to analyse the cases presented in fig. 1 to fig. 4. our goal is to indicate possible incentives for collaboration. for the sake of simplicity we consider specific examples by choosing suitable values of the parameter a. the other parameters remain unchanged as in fig. 1 to fig. 4 (<xi = 0.4, 0.2 = 0.7, \i = 15). table 3 contains the certainty equivalents of the two firms for the case of small a. note, that neither development nor maintenance costs are considered. the certainty equivalent is linear in secure payments. hence to get the certainty equivalent in the presence of costs we merely have to deduct these costs from the corresponding numbers. scenario/firm scenario 1 scenario 2 scenario 3 scenario 4 f, 25.981 24.010 23.879 23.871 f2 21.593 23.099 21.778 23.048 table 3: results for a = 0.75 without consideration of costs only the more risk averse firm f2 has first mover advantages. if it invests the third scenario is reached. f\ should not follow, however. thus the fourth scenario which is the best possible situation for the second firm is not reached. we now consider costs. it is assumed that the costs are incurred periodically (a single investment may be transformed into an annuity with a suitable number of periods). scenario 3 will be obtained as long as the development/maintenance costs of the second firm are less than a2 = 21.778-21.593 = 0.185. the second firm has a strong incentive to reach the fourth scenario. this would increase the certainty equivalent by 1.270 units per period. a possible strategy would be to share the development efforts with the first firm and to compensate this firm for the losses of 0.008 units associated with the use of the system. the second firm has collaboration incentives as longs as its investment costs are lower than 1.447 (difference of fourth and first scenario minus the minimum transfer payment necessary to compensate the first firm for moving from the third to the fourth scenario). up to now we have assumed that it-use is merely a matter of costs. suppose next, that knowledge and ability with respect to developing a suitable system and/or using a specific technology are not equally distributed. 124 ajis special edition electronic commerce november 1998 assume that the first firm has a competitive advantage. in that case f) has no incentive to share its knowledge. of course, it will not invest and use the new technology. note, that the first scenario is optimal for this firm. then the status quo will prevail as long as f2 is not able to efficiently use the technology. collaboration will not occur since there are no (economic) incentives. scenario/finn scenario 1 scenario 2 scenario 3 scenario 4 f, 22.065 19.394 18.698 18.823 f2 13.773 15.690 13.350 15.757 table 4: results for a = 2 without consideration of costs next we increase a by setting a = 2. table 4 contains the results with zero costs. none of the firms has first mover advantages. it was indicated above that the more risk averse firm should nevertheless invest. then, the third scenario is reached. now f, should follow and we reach the fourth scenario. scenario 4 will be obtained as long as the development/maintenance costs of the first firm are less than ai = 18.823-18.698 = 0.125 (difference between scenario 4 and scenario 3; there is an incentive to be a follower) and the costs of the second firm are less than a2 = 15.757-13.773 = 1.984 (difference between scenario 4 and scenario 1). collaboration would make sense in order to cut down development costs. suppose next, that knowledge and ability with respect to using the new technology are not equally distributed. assume that the second firm has a competitive advantage. it may develop the system at costs below a2, while a similar development project would cost f] more than a]. it is important to note that f2 has incentives to transfer its know-how to firm fi. otherwise it would get stuck in the third scenario which is worse than the original status quo. on the other hand fi should accept the know-how transfer since it may improve. obviously transfer payments in either direction are possible. this is a strong incentive for collaboration among competitors. if the first firm has a competitive advantage it will not use its know-how and it will not transfer it. again, the status quo will prevail as long as f2 is not able to efficiently use the new technology. as in the previous case there is no incentive for collaboration. scenario/firm scenario 1 scenario 2 scenario 3 scenario 4 f, 11.650 12.253 10.536 11.427 f2 6.735 7.301 7.343 8.058 table 5: results for a = 4 without consideration of costs finally, we set a = 4. table 5 contains the results for this case. again, both firms face first mover advantages as long as costs remain below ai = 0.603 and a2 = 0.608. suppose firm fi has a competitive advantage in developing and/or maintaining a suitable information system. this firm has no incentive to share its know-how since the second scenario is the best possible world for this firm. the opposite statement holds if firm f2 has a similar competitive advantage. this firm has vital interest in reaching the fourth scenario. before concluding this section the effects of technology use on consumers are analysed. as was shown the third and fourth scenarios are candidates for an equilibrium. hence, quantities offered by the two firms need to be compared to the quantities offered at the beginning. higher quantities result in lower prices and vice versa. hence, consumers benefit if and only if quantities offered increase. for the first scenario the total quantity offered may be computed using (9). this yields 2 + (a l+a2)q2 (25) *' ~ similar computations for the third (<?3) and fourth scenario (q^ using expected quantities give u(8 + a,c2) 2 (26) q, -by computing the differences of the expressions in (25) and (26) it is possible to show that €4 > 4j > q\ holds. hence, consumers always benefit from the use of the technology by the competing firms. clearly, precompetitive collaboration should be encouraged from a consumer's point of view. this once more demonstrates 125 ajis special edition • electronic commerce november 1998 the difference between pre-competitive collaboration (which is beneficial for consumers) and collusion where prices and/or quantities are fixed or negotiated, respectively. the latter, generally is not beneficial for consumers. effects of cost reduction in this section the effects of cost reduction by using suitable information technology in a homogeneous duopolistic market environment are analysed. the starting point is the price-demand function (2). now a is assumed to be fixed and consequently non-random. suppose that the i-th firm produces at variable costs cj. its objective function is then given by 8i (*i ,*2 ) = (fl ~ *i ~ *2 c, )*, using first order conditions for a maximum the corresponding reaction functions are given by *i(*2)=^(a-ci-*2)> x2(xl)=-(a-c2-xl}. from this we may compute equilibrium quantities. the result is (27) * , we assume that the quantities in (27) are positive (this amounts to the assumption that a is sufficiently large with respect to variable costs). from (27) it is obvious that the firm that produces with lower costs has a higher market share and consequently also higher profits (since there is no uncertainty the certainty equivalent corresponds to profits). profits for the firms are given by (28) pn=-(a + c2-2cl) 2, p21=-(a + c,-2c2)2. consider next the case of cost reductions. suppose the first firm is able to reduce costs by an amount of a. then its profits will increase while the profits of the competitor decrease as (28) reveals. if one of the firms has a competitive advantage there is no incentive to share its know-how with its competitor. if knowledge and ability are equally distributed, profits of both firms increase if each firm is able to lower costs by an amount of a. then, collaboration would be useful in order to cut down development costs. in contrast to the situation in section 4 the results are as one would expect. competitive advantages should be exploited, knowledge and ability become a strategic resource that should not be shared. in such cases firms will be extremely cautious when it comes to possible (pre-competitive) collaboration. it is interesting to note that consumers benefit from cost reductions. addition of the equilibrium quantities in (27) yields hence, cost reductions lead to an overall increase in supply and consequently to lower prices. this corresponds to the result of section 4. summary and conclusion the use of electronic commerce technology may affect cost structures and/or demand/supply parameters. decisions of two firms acting in a homogeneous duopolistic market environment have been analysed. it was investigated whether economic incentives for pre-competitive collaboration exist. if the focus is primarily on cost reductions firms will be very cautious when the issue of collaboration arises. know-how and ability are resources that allow to build up sustaining competitive advantages. only if these resources are equally distributed between the competing firms collaboration is useful. then development and/or maintenance costs may be split and therefore reduced. the case where technology primarily influences the demand side is the more interesting one. the influence on the demand side was modelled with the help of a stochastic demand parameter. using available technology the firms were able to reduce uncertainty and hence to exactly predict the uncertain parameter. the results derived are surprising in some cases. it was shown that situations are possible, where it is beneficial to share know-how with a competitor. know-how refers to knowledge and ability about the technology under investigation and should not be confused with collusion. the term pre-competitive was used to characterise this form of collaboration between competing firms. whether collaboration makes sense depends primarily on the market structure and the degree of risk aversion of the two firms. various situations with different parameters have been analysed. in some situations technology use was beneficial for both firms, in some situations only the more risk averse firm had benefits. generally, the less risk averse firm has less incentives to share know-how than the more risk averse firm. an explanation for this 126 ajis special edition electronic commerce november 1998 observation may be the fact that the less risk averse firm is rewarded for taking excess risks. hence, its strategy should be the protection of its market dominance. the results presented have interesting implications on the diffusion of electronic commerce technology. it indicates which kind of parameters may be important to firms in order to decide on the use of electronic commerce technology. an empirical validation of the model, eg. by considering case studies, is an important topic of future research. generally, consumers benefit from technology use. total quantities offered increase and as a consequence price decreases. hence, consumers receive more goods at lower prices. in the paper the homogeneous duopoly case was analysed. in the same way the case of competition in a heterogeneous market environment may be tackled. the analysis is more complicated, the spirit of the results, that will be reported elsewhere, remains the same (stickel, 1998). the model presented, although very simple, may be extended in a variety of ways. it may be extended to the case of an arbitrary oligopoly. the results basically remain the same, except that we get an ordering of the firms according to their degree of risk aversion. the firm that has the least risk aversion is in the same situation as f\, the firm with the highest degree of risk aversion resembles f2. the assumptions about the distribution of the parameter a may be relaxed. the results hold for arbitrary single peaked distributions (eg. a beta distribution on the finite interval [a,b]). also, more general utility functions may be used as long as an ordering with respect to risk aversion is possible. then, in most cases, closed-form solutions may not be computed any more. instead, numerical techniques need to be used. it is also possible to consider more general demand function. simulations with more complex demand functions (eg. quadratic ones) have been carried out without changing the results. as was already indicated in the text different variable costs, as well as the case of a market leader who announces quantities earlier than its competitor may be considered. cost effects and effects due to market leadership may ..overlay" the results presented and may strengthen or weaken the effects described. the derivations relied on the existence of a cournot-equilibrium. other price mechanisms are possible and studied in the literature. caldas & coelho (1994) eg. use organisational learning concepts together with artificial economic agents. it remains to be explored whether these concepts change our results. the model constructed allowed the consideration of different time periods. these time periods were treated as being independent. hence there was zero correlation between parameter values of a in different time periods. usually, this cannot be expected to hold in reality. small demand in period r is usually followed by a small demand in period r+1. this implies that there is some sort of correlation. if such assumptions are introduced decision makers are equipped with so-called real options (dixit & pindyck, 1994). investments may eg. be deferred, speeded up, slowed down or abandoned. flexibility usually implies higher benefits of investments. influences on the results presented may result. this interesting topic is the subject of current research. references bauer, s., stickel, e: contributions of it to the genesis of network organisations. in khosrowpour, m. (ed.): information technology management and organisational innovations. proceedings of the 1996 information resources management association conference. harrisburg (1996), pp. 46-52. caldas, j. and coelho, h.: the simulation of trade in oligopolistic markets. in gilbert, n., doran, j. (eds.): simulating societies: the computer simulation of social phenomenas. london (1994), pp. 245-266. choudhoury, v., sampler, j.: information specifity and environmental scanning: an economic perspective. mis quarterly 21, no. 1, 1997, pp. 25-53. demons, e., reddi, s.: the impact of i. t. on the degree of outsourcing, the number of suppliers, and the duration of contracts. in nunamaker, j., sprague, r. (eds.): proceedings of the 27th annual hicss, vol. iv. hawaii, 1994, pp. 855-864. copeland, t., weston. j.: financial theory and corporate policy. 2"* ed., reading, ma. (1983). dixit, a. and pindyck, r.: investment under uncertainty. princeton, nj. (1994). dossani, r., hamid, s., prakash, a., smyser, m.: a strong case for the arrow-pratt risk aversion measure. in ghosh, d. (ed.): new advances in financial economics. oxford, 1995, pp. 87-96. fried, d.: incentives for information production and disclosure in a duopolistic environment. quarterly journal of economics 99, 1984, pp. 367-381. gal-or, e.: information sharing in oligopoly. econometrica 53, no.2 , 1985, pp. 329-343. gal-or, e.: information transmission coumot and bertrand equilibria. review of economic studies 53, 1986, pp. 85-92. hviid, m.: risk-averse duopolists and voluntary information transmission. journal of industrial economics 38, no. 1,1989, pp. 49-64. kao, j., hughes, j.: note on risk aversion and sharing of firm-specific information in duopolies. journal of industrial economics 41, no. 1, 1993, pp. 103-112. 127 ajis special edition-electronic commerce november 1998 kettinger, w., grover, v., guha, s.: strategic information systems revisited: a study in sustainability and performance. mis quarterly 18, no. 1, 1994, pp. 31-58. klemperer, p., meyer, m.: price competition vs. quantity competition: the role of uncertainty. rand journal of economics 17, no. 4, 1986, pp. 405-415. li, l.: cournot oligopoly with information sharing. rand journal of economics 16, no. 4, 1985, pp. 521-536. malone, t., yates, j., benjamin, r.: electronic markets and electronic hierarchies. cacm, 30, 1987, 484-497. palfrey, t. r.: risk advantages and information acquisition. bell journal of economics 13, no. 1, 1982, pp. 219-224. pindyck, r., rubinfeld, d.: econometric models & economic forecasts. 3rd ed., new york (1991). pindyck, r., rubinfeld, d.: microeconomics. 2nd ed., new york (1992). porter, m. e.: competitive advantage. 2nd ed., new york (1985). stickel, e.: wettbewerbsorientierte informationssysteme und produktivitatsparadoxon. wirtschaftsinformatik 37, no. 5, 1995, pp. 548-557. stickel, e.: problems associated with cost/benefit analysis for strategic information systems. in wojtkowski, g., wojtkowski, w., wrycza, s., zupancic, j. (eds.): systems development methods for the next century. new york (1997), pp. 101-114. stickel, e.: an economic analysis of collaboration between competing firms in a heterogeneous market environment unpublished manuscript (1998). tirole, j.: the theory of industrial organisation. cambridge, ma. (1997). appendix lemma 1. if both firms have the same variable costs c we may assume without loss of generality that c = 0 holds. proof. suppose both firms have the same variable costs c. the profits of the i-th firm are given by n,(*, , x2 ) = px, cxf = (a-xl-x2-c')xi. put a* = a c. then a modified optimisation problem with a stochastic parameter a* instead of a and variable costs c = 0 results (note that a* is independent of /)• the parameter a* has a normal distribution with mean u, c and unchanged standard deviation a. this demonstrates the equivalence of the problems with and without zero variable costs. + lemma 2. let a have a normal distribution with mean p. and standard deviation a. suppose further that a decision maker has a utility function as given in (1) with constant risk aversion y. the certainty equivalent c of this decision maker for the payoff c2a + c\a + cq is then given by 1 r , = — inl+2c27(yl 2 «2 0 .2y l 2 « j o 2(l + 2c2ya2) proof. for the density of a normal distribution with mean \i and standard deviation a we have expected utility r of bur decision maker is given by 1 e x o 1 °° 1 = 1 — r=j exp(--(— -)2)exp(-y(c2a 2 +c,a + c0)rfa = 1-7 v27ta z az a <j?g standard techniques from calculus the last integral / may be evaluated to get , ... 2c2n,2+2c1p.-c1 2ya2 7 =<1 + 2c21pv" exp(-t(c0+ ^ by assumption the utility function of our decision maker is given by m(x) = l-exp(-yr). consequently ln(l -«(*)) 128 ajis special edition electronic commerce november 1998 for the certainty equivalent this yields ln(/) c = it1 (/?) = it1 (!-/) = —l-l. y from this the claim readily follows. ̂ 129 ajis vol 4 no. 1 data warehouse architecture for dss applications v. j. anand and himawan gunadhi* department of decision sciences national university of singapore 10 kent ridge crescent singapore 119260 abstract a data warehouse facilitates the integration of disparate operational databases in an enterprise into a single store. the warehouse then provides knowledge workers with easy access to historical, summarized and other forms of aggregated data. a major flaw in present warehouse architectures is the de-coupling of the warehouse database from its underlying operational databases. this creates two problems: the difficulty in updating the warehouse and the inability to provide users with a drill-down feature from warehouse to current data. we propose an architecture that is based on a federated database system extended to incorporate a materialized warehouse. a federated database provides the basic mechanism to tightly couple heterogeneous databases, which in this case are the operational and warehouse databases. the warehouse database is a special component in the federation, made up of historical data, external data and materialized views of the operational data. this approach enables users to access historical and current data when required, and provides a method of maintaining the warehouse as an integrated view of the underlying operational data sources. introduction database management systems have helped enterprises to efficiently collect and manage data, but have fallen short in providing an environment where this data can be used to gain a better business insight and making quality decisions. the problem arises from the manner in which databases have been built and deployed primarily for storing transaction data generated by the day-to-day activities of an organization. in addition, large corporations have built disparate and heterogeneous database systems, each fine tuned for a specific function. consequently, several requirements of knowledge workers have not been satisfied: 1) access to historical data that cut across functional boundaries and involve complex aggregates; 2) quick turnaround between the request and availability of information; and 3) the ability of operational systems to simultaneously cater to the needs of both operational and decision support applications. data warehousing [inmon (1992); inmon & hackathorn (1994); kelly (1994); gupta et. al (1995); hackathorn (1995)] provide a framework for integrating data from multiple databases in a logical and unified fashion. it provides users with easy access to enterprise data in a consistent manner and a method for storing historical, summarized and other data aggregations. separating the data required for decision making activities from those needed for operations provide the following advantages: 1) integration of data from multiple sources in a consistent manner; 2) a single point access for enterprise-wide data; 3) enhanced efficiency in terms of availability and response time for user queries; and 4) real-time analysis of data for decision-making, i.e., online analytical processing (olap). to provide users with consistent and current information, a data warehouse has to be updated constantly. present methods [inmon & hackathorn (1994)] use programs that periodically extract data from different sources. since these are batch processes, the data warehouse is updated only periodically, say once a day. thus, the warehouse is as current as the last extraction cycle. also, these programs have to be altered against any schema change either in the operational databases or data warehouse, since these affect the mapping information. while the de-coupling of decision-support and transaction-oriented databases have resulted in enhanced response time for complex queries, it has also created an island between them. there are situations when a data warehouse user requires access to the most current data, but there is no easy access path from the warehouse to the sources residing in the operational databases. research has recently been carried out in the area of warehouse maintenance as a materialized view of the underlying data sources [zhou et. al (1995)]. this enables existing view maintenance methods [hanson (1987)] to be adapted. but, a warehouse is not created simply by joins or projections among tables from a single data source some data are derived by complex aggregations that span multiple databases. moreover, many of the sources are not relational databases but legacy-based systems, e.g., hierarchical and network databases, which may not support views. the stanford whips project [hammer et. al (1995)] models a warehouse as a materialized view that can be incrementally maintained via an integrator and monitors. whenever a change is detected in the data source, the corresponding monitor sends a message to the integrator to update the warehouse. the integrator may subsequently require additional data either from the same or additional sources. the drawback of this model is that it becomes difficult to maintain the integrator against changes in the schema t author's present address: 1691 ruther place ct., san jose, ca 95121, usa. internet: h.gunadhi@usa.net. 43 ajis vol 4 no. 1 at the data source or warehouse. moreover, the integrator has to respond to a change detected by a monitor by sending appropriate queries to the data sources, resulting in increased maintenance costs. research in heterogeneous databases [sheth & larson (1990)] have provided methods to integrate disparate databases into a single unifying architecture — the federated database model. we propose to extend the federated architecture to include a data warehouse modeled as a materialized view of the underlying federated schema. in addition, it adapts a deferred view maintenance approach, rather than immediate approach adopted by whips. this approach is preferred, because a great deal of decision-making may not require current data, but for those that require them, the model provides a mechanism to obtain them without adding too much overhead. this approach provides the following advantages: • a unified architecture that ties the data warehouse to multiple heterogeneous databases. • provides a method of maintaining the data warehouse as an integrated materialized view of the underlying data sources. • provides flexible access to current data residing in the data sources. • ease of maintenance against any change to the schema in the data source or warehouse. the rest of the paper is organized as follows: section 2 gives an overview of related research that forms the basis for this paper. section 3 introduces the integrated data warehouse model and the concept of schemes within this framework. section 4 describes how view management is handled by the architecture, while section 5 concludes by outlining future research. related research in this section we give an overview of research in federated database systems, data warehousing and view maintenance which are closely related to the work we describe in this paper. federated database systems federated database systems provide methods of integrating heterogeneous databases into a unified system, thereby increasing the accessibility. in other words, users have to interact with a single system rather than individual databases. federated database systems provide: 1) methods for data integration — reconciling, cleaning and removing inconsistencies between the various schemes to generate a federated schema [sheth & larson (1990)], and 2) transparent access to users — the system takes care of coordinating in splitting the query and subsequently combining the results. federated databases are implemented as either loosely or tightly coupled architectures [sheth & larson (1990)] depending on the level of autonomy. though they provide ease of access they are limited in as much as: 1) data that can be seen by all users is dependent on individual databases, 2) federated schema is non-materialized, which means that queries have to be evaluated in the individual database, resulting in slower response time, and 3) data from external sources are not integrated within the federated schema. data warehouse data warehouses [inmon (1992)] are large repositories for analytical data. they are primarily designed to improve query response time and provide data in a form required by the knowledge workers. they offload the query overhead from operational databases into a separate specialized database providing improved efficiency. in addition, they also provide a means of: 1) integrating data; both external and enterprise in a consistent manner, and 2) storing different forms of data, e.g., data cubes [weldon (1995)], aggregates of data, etc. presently these systems are deployed separately, without any link to the underlying databases and data is periodically pushed or pulled into the warehouse. but for such queries that require access to the operational data from the warehouse, it does not provide a method. view maintenance materialized views [gupta & mumick (1995)] provide a means of caching the result of a query and therefore, providing increased query response time. this technique has been widely used in distributed databases [segev 6 park (1989)], where base relations and views are physically separate. materializing the view requires it to be constantly updated to reflect the present state of the base relations. a trivial and most often used method is to re-compute the view each time the base relation undergoes change. the problem with this approach is that base 44 ajis vol 4 no. 1 relations that change often can trigger excessive re-computation of the view, although it is efficient for certain operations as deleting the whole base relation. alternatively, only changes that occur on the base tables are propagated incrementally. based on when the view is updated, the algorithms [hanson (1987)] can be classified into: 1) immediate — which propagates the change within the transaction that affects the base relation, and 2) deferred — which propagate the change after a time lag. immediate propagation [blakely et. al (1986); shmueli (1984); zhuge et. al (1995)] methods have been well researched and a wide range of algorithms that implement such a technique is reported in the literature. in a number of applications, immediate view maintenance is simply not possible. because applications cannot tolerate any slow down, systems would like to keep the transaction overhead to the minimal. moreover, in a distributed scenario, the de-coupling of the base relation from the view can result in update anomalies [zhuge et. al (1995)] and high network cost. deferred view maintenance [agarwal & dewitt (1983); hanson (1987); griffin & libkin (1995)] has however not been studied as extensively as the immediate case. they propagate the change periodically or on-demand when certain conditions arise. therefore, the materialized view is inconsistent with the base relations for a period of time, which is tolerable for a number of applications. moreover, they do not add any overhead to the transaction as they require only to insert an entry into a log file. in addition, efficiency is gained by batching several updates together and replacing several small queries and updates into one. the integrated data warehouse model (idwm) integrated data warehouse model integrates a data warehouse into a tightly coupled federated database architecture (figure 1). this provides a three tier architecture: operational data sources and data warehouse forms the bottom and top layers respectively, while the interface layer provides the mapping between the two. when access to the current data is required it is retrieved transparently from the operational data sources by accessing the interface layer. in the model, a component database (cdb) refers to an operational data source. data that is of interest is first represented uniformly through the export schema, which defines data that is exported from the operational sources. after reconciling, integrating and removing semantic heterogeneity between various export schemes, these are then integrated along with the external data into an integrated data schema (ids) depicted near the top of figure 1. here, we make a distinction between ids and the federated database schema as defined in the literature. the former refers to a schema that integrates all data from the participating operational sources, while the latter may not. in addition, we distinguish export schema from component schema only in the type of data model that is employed in defining them, since we assume that all data from the component database is for export. for example, component schema can be defined as hierarchical and the export schema can be defined as relational. between the operational sources and the data warehouse lies the global view management system (gvms), and the local view manager (lvms). an lvm interfaces the operational source into the framework, and provides utilities to generate differential files and specific mapping utility between the export and local schema. in addition, it periodically, sends updates from the operational source to the warehouse. gvms services request for data from the warehouse and coordinates lvms during periodic updates. when updates are received at the warehouse, gvms has to propagate the change to the materialized export views and the warehouse views. although, information flow appears to be exclusively "upwards" in direction, there are situations when gvms requests for data from one or more lvms. the model does not make any assumption on the type of operational sources: they can be any database employing any data model e.g., hierarchical, network or object-oriented. also, they may or may not have a view management facility. moreover, the model views the warehouse not just as a source of data for decision-making purposes, but also as a repository of reconciled and integrated enterprise data. thus, there is a natural evolution of a federated database into a data warehouse. 45 ajis vol 4 no. 1 process legend schema description software component data warehouse schema global view management system data warehouse st figure 1. the integrated data warehouse model export schema as different data models may be employed by the underlying component databases, it becomes imperative to map them to a common data model. the export schema provides a common representation to which each component schema is mapped. for example, one can represent an individual component schema as a relational export schema, with appropriate mappings between them. this helps in localizing the schema changes, and makes the local database administrator responsible for keeping the export schema consistent. in addition, we consider the export schema not just as a mapped representation of the underlying tables, but as meaningful views. this helps in resolving what needs to be exported from a component database. unlike a typical federated database system where the local administrators are responsible for exporting data, here the responsible administrator or users can determine that a certain data item is to be obtained from a particular database. as an illustration, there may be two databases maintaining customer names, and both can be exported; but it is decided that only one will export it. thus, the data item gets included in one, and excluded from the second export schema. integrated data schema this schema is generated by combining and reconciling the export schemes from the component databases. it provides a consistent and reconciled view of all the data existing in the enterprise. also, it is a means of integrating unstructured and external data within the organization. generally, the ids follows the same data model as the export schema, but may differ. we consider the former, as this removes an additional mapping between two different data models. in addition, ids forms the basis from which the schema for the warehouse is extracted. changes to the warehouse are localized since ids provides the interface between the warehouse and export scheme of the component database. 46 ajis vol 4 no. 1 thus, ids maintains: • information regarding the source from where a certain data is obtained. • mapping information to the export schemes: this refers to any transformation that is required between the two representations. • mapping information to the warehouse schema: this refers to any transformation that is required between the two representations. data warehouse schema the data warehouse schema consists of aggregates, summaries and other data groupings based on different dimensions. time, regions and product lines are examples of common dimensions. ids provides the data warehouse with an enterprise data schema from which one can generate such views. initially, the warehouse schema is small and grows as users become aware of the advantages of using a warehouse. the warehouse may not have all the data to respond to user queries at first; data may have to be obtained from the cdbs transparently, and subsequently integrated into the warehouse schema. this allows a natural evolution of the warehouse schema where usage determines what has to be stored. we formally define a warehouse as a collection of materialized export and warehouse schemes. materialized export views are stored over time for access to historical operational data. at any given time, an instance of a warehouse view will correspond to a set of materialized export views and external data mapped through the integrated data schema. any schema change at the warehouse will not have retrospective effect on its earlier definition. hence, the affected view will be considered as a new inclusion in the warehouse schema. this allows users to access older versions of the schema as it is still maintained in the warehouse. but, previous versions of the same schema is not materialized in subsequent refreshes. let {v; , v2 ,..., vm} be the set of views exported from a single component database. let [gi , gi ,..., gm } be the set of transformation functions between the local schema and the export views; gi is defined as selection, projection, join or combination thereof, over the local schema. let {di, . . ., dw } be the views defined at the data warehouse. let {// , /2 ..... /„} define the set of transformation functions between the exported views and the views in the warehouse; jj is defined as selection, projection, join or a combination thereof, over the exported views. let {tt , t2 ..... tp} define time intervals when the warehouse is refreshed. the set of export views from a single component database at the warehouse is defined as: 1=1 export views materialized at the warehouse is given by: y yv-«1 = 1 1 = 1 where v, refers to intention and v,{t) to extension. views in the warehouse schema defined over export views is given by: the materialized warehouse schema can then be defined as: w f m \ y.m yv; + (external data) m u=i ) we illustrate the above through an example: let rt (an , a12, ..., ah, ) and r2 (a2i, a22 ,..., a2<) be two relations in a component database. let v/ = <7/>i( r/) and v2 = k(au, am ..... 42,) (r2) be the export views defined over relations ry and r2 respectively. let <4 =opi(vi) x ^((jpscva)) be a view defined in the warehouse over the exported views, where p/ , p2 and p3 are condition predicates. then it follows that m y(w) = {v,,v2} 1=1 x y=i the above definitions and example illustrate the case for a single component database. for multiple component databases, we derive the following equations: 47 aj1s vol4 no. 1 views in the warehouse schema defined over export views is given by: h> / c n \ y.m yyvfa wnere & = {1,..., c} are the component databases, each exporting n views. materialized warehouse schema is given by: w f c n \ y./5' yyvfa + (external data) view management local view manager the local view manager (lvm) is a software module that provides the interface between a component database and the data warehouse. this module is provided for each component database that participates in the federation. lvm implements the specific mapping utility between the local and export schemes, e.g., a function to map hierarchical to relational data model. generally, there may not be a direct mapping between the local and export schemes, e.g., a table in the export schema can be a view defined by a join between tables. therefore, a comprehensive catalog is maintained which provides mapping information. moreover, it makes it easier for the local administrator to maintain the catalog for any change in the local schema. the other major component of lvm is the generation of differential files for views defined in the export schemes. the implementation of this function depends on the individual component database. for a database that support views, lvm generates a transaction file for each table. at the time of propagation, a differential file is generated for every view. legacy systems that do not support views will either require modification to the applications such that an entry to a transaction log is made before committing changes, or data can be moved to another database that support views. alternatively, differential files can be generated periodically by comparing the current and previous snapshots of source tables, but they become inefficient as they grow in size. lvm has to interact with the global view management system (gvms) for refreshing the data warehouse, and in addition process requests from gvms for ad-hoc data. we will describe the data structures that help lvm in this process, adopting the assumption that component databases support view-definitions. view-manager the view-manager maintains information regarding views exported from the local database defined as: (viewname, view-definition, differential file-name, time created, last refresh-time, next refresh-time, last requesttime from gvms), where: • view-name — uniquely identifies each view that is exported from the local database. • view-definition — defines the view, select-project-join and attributes of the view. • differential file-name — uniquely identifies the file to store the net changes that has occurred for the view, between the last and next refresh time. • time created — a time-stamp indicating when the differential file was created. • last refresh-time — a time-stamp indicating when the view was last involved in updating the data warehouse. • next refresh-time — a time-stamp indicating when the view will be next involved in updating the data warehouse. • last request-time from gvms — a time-stamp indicating when there was a direct request from gvms for the view differential file. table-manager maintains information regarding tables and the attributes that are taking part in a view defined by: (table-name, view-name, attribute-name), where • table-name — identifies the table taking part in the view definition. • view-name — identifies the view in which the table is used. • attribute-name — identifies attributes of the table that are included in the view definition. 48 ajis vol 4 no. 1 transaction-manager maintains information regarding the changes on the table between the last and the next refresh times defined by: (table-name, transaction file-name, trigger-name), where: • table-name — identifies the table. • transaction file-name — uniquely identifies the file name that records the transaction for the table, from the last refresh time to the next refresh time. • trigger name — identifies the trigger that will be activated from the table when changes occur on it. here we have assumed databases that support triggers and view definitions to give a flavor of the book-keeping functions required. but, for databases that do not support these functions, a different implementation is required. as we are focusing more on framework rather than actual implementation, we do not discuss this issue further. lvm has to respond to two basic events: 1) refresh request from gvms, and 2) ad-hoc request from gvms. we treat them as separate events, since (1) would require a generation of a differential file for the complete export schema, while (2) requires the generation for a specific view. refresh from local view manager at the next refresh cycle, lvm propagates to gvms a differential file for its export schema. the algorithm (see figure 2) used for generating these files may vary depending on the type of implementation chosen. algorithms that generate differential files can be used in systems that support view definitions, while a snapshot differential will have to be used for systems that do not support views. our model is not constrained by any specific algorithm, as they can be chosen based on the requirements of the data source. algorithm refresh for (export views) loop {send-message (site-name, refresh); /* message to lvm */ a,-<f(gi(trans-log(l), trans-log(2),...,trans-log(n)) /* f is view maintenance algorithm trans-log(/') is the transaction file-name of table j between refresh intervals. a, is the differential file-name to be propagated. */ send-message( a,) /* message to gvms */} updated materialized export views at the warehouse: y yv.(0 +y(v.+a,) ; i = 1 i = 1 1 = 1 materialized warehouse schema after the refresh: w ( m \ tv f m \ yjfl yv; l yj5 yv' + a* + (external data) 7=1 \i=l j j=l \i=l ) figure 2. algorithm refresh 49 ajis vol 4 no. 1 request from global view management system other than periodic refresh, lvm may be requested to send the changes for a specific view defined in its export schema (see figure 3). this would require the propagation of a differential file for the requested view. we foresee such requests in the earlier stage of the warehouse's existence, since these data will be integrated into the warehouse schema subsequently, and will be updated through the periodic refresh cycles. but we treat these queries as non-preserving. to summarize, lvm provides: • mapping between export and local schema of each cdb. • generate differential files for the views exported from each cdb. • communicate with the global view management system (gvms). • maintains the transaction and differential files for each table and view respectively. algorithm; ad-hoc request (m }vdi & warehouse schema, 3 / i di=f,\ yv> k* transformation */ u=' ) for (export views in </,-) loop {send-message (view-reference, site-name); /* message to lvm */ a , <f(gi(trans-log(l), trans-log(2) trans-log(n)); /* f is the view maintenance algorithm */ send-message (a / ) /* message to the gvms */} (m \ materialized d, = /• y(v-/ + a/) u=' ) figure 3. algorithm ad-hoc request global view management system (gvms) gvms is a software module that provides similar functionalities to the data warehouse as lvm provides to the component databases. but the gvms is a global master that coordinates the lvms and determines the next refresh cycle. functionally, gvms provides: • a utility to map ids to the warehouse schema, and also ids to the export schema of component databases. • algorithm to generate differential files for updating the data warehouse schema. it takes as input differential files from lvms and outputs a differential file corresponding to the warehouse schema. • coordination of propagation of differentia] files from lvms. • requests to lvms to propagate differential files for ad-hoc data. the idwm employs a two stage process for update propagation: in the first stage, a differential file for every export schema is generated; in the second stage, these are combined to generate a differential file for the warehouse. this helps in implementing different algorithms at the warehouse and component databases. we do not see this as a necessity as the model allows for differing implementations, thus providing a framework without making assumptions about the underlying methods. gvms consists of various software modules (see figure 4) that perform different functions. the mapper transforms the warehouse schema to the underlying export views by interacting with ids. the assembler consolidates results and propagates updates to materialized views, and it also implements the view maintenance algorithm in gvms. coordination between these modules is performed by the coordinator, while monitor provides interface between gvms and lvms. for ad-hoc queries the coordinator passes the definition to the mapper in order to find relevant export views that fulfill the request. next, the mapper passes the site addresses of the relevant export views to the monitor to retrieve the differential files. these are buffered, until all requests are obtained by the monitor and passed on to the assembler, which then assembles them according to the query definition and submits the result to the coordinator. 50 ajis vol 4 no. 1 i coordinator i \ r [assembler! 1 mapper \ i monitor integrated data schema network i/f lvm lvm (operational! [database! operational [database figure 4. global view management system for periodic update propagation from the component databases, the monitor has an elaborate tracking mechanism through which it ensures that all the differential files are received by the warehouse before the assembler begins propagating. in the event of a network failure, the monitor requests the corresponding local view manager to re-send the differential files. conclusion and future research a great deal of work has been carried out in heterogeneous databases, view maintenance and data warehousing, but only recently have there been some research in integrating them. our aim is to develop a model and define interfaces required to integrate these methods into a unified architecture. in this process many open and challenging issues that requires further investigation were brought to our focus, such as: 1. view maintenance algorithms for updating aggregates. as the warehouse consists of aggregated data along many dimensions, there are not many generalized algorithms to handle these operations. 2. algorithms for generating differential files for views that are defined as select-project-join. 3. handling data changes and update propagation from databases that do not support view definitions. 4. characterization of data warehouse — present characterization looks at the warehouse as a repository solely for decision-making purposes. what about those applications that require access to enterprise data in a consistent form? our framework has taken the first step in this direction. 5. we assume the warehouse as an append only database, but what about those that can have updates as well, e.g., to enable error-correction and schema changes. 6. mechanism to integrate data into the warehouse schema when it is not present when requested for the first time. we are presently working on defining the various functional modules that have been defined in the idwm in terms of functions and data structures. further work is required in defining gvms and lvm in terms of: • functional modules for mapping schemes, • functional module to generate differential files, and • book-keeping functions. in addition, we are focusing on developing algorithms to generate differential files on the export and data warehouse schema that are defined as select-project-join. this will be further extended to include aggregates at the data warehouse schema. to conclude, we have shown that research in the areas of heterogeneous databases and data warehouse have looked at the same problem of data integration quite differently. the former looked at the problem of providing easy access to data residing in different databases, while the latter looked at the problem of bringing the 51 ajis vol 4 no. 1 relevant data to a central repository. we have proposed a data warehouse architecture that is integrated to the data sources, by extending the federated database architecture. our architecture provides: • flexibility — underlying databases can be any legacy system, including those that do not have a view management facility, as long as modules that can supplement the underlying database view facility is included in the lvms and gvms. • transparent access to operational component databases to enable response to queries needing current data. • a method of maintaining the data warehouse as a materialized view. any view management method can be used, as they are pluggable in the modules in lvms and gvms. references adiba, m. & lindsay, b.c. (1980) database snapshots, proceedings of the conference on very large databases, pp 86-91. agarwal, r. & dewitt, d.j. (1983) updating hypothetical databases, information processing letters, vol 16, pp 145-146. blakely, j.a., larkson, p. & tompa, f.w. (1986) efficiently maintaining views, proceedings of the acmsigmod international conference on management of data, pp 61-71. ceri, s. & widom, j. (1991) deriving production rules for incremental view maintenance, proceedings of the conference on very large databases, pp 577-589. gilgor, v.d. & luckenbaugh, g.l. (1984) interconnecting heterogeneous database management systems, computer, vol 17, 1, pp 33-45. griffin, t. & libkin, l. (1995) incremental maintenance of views with duplicates, proceedings of the acm-sigmod international conference on management of data, pp 328-339. gupta, a. & mumick, i.s. (1993) maintaining views incrementally, proceedings of the acm-sigmod international conference on management of data, pp 157-166. gupta, a., harinarayan, v. & quass, d. (1995) aggregate-query processing in data warehousing environments, proceedings of the conference on very large databases, pp 358-369. gupta, a. & mumick, i.s. (1995) maintenance of materialized views: problems, techniques, and applications, data engineering bulletin, vol 18, 2, pp 3-18. hackathorn, r. d. (1995) data warehousing energizes your enterprise, datamation, feb, pp 38-43. hammer, j., gracia-molina, k, widom, j., labio, w. & zhuge, y. (1995) the stanford data warehousing project, data engineering bulletin, vol 18, 2, pp 41-48. hanson, e. n. (1987) a performance analysis of view materialization strategies, proceedings of the acm-sigmod international conference on management of data, pp 440-453. inmon, w. h. (1992) building the data warehouse, qed. inmon, w. h. & hackathorn, r.d. (1994) using the data warehouse, john wiley. kelly, s. (1994) data warehousing: the route to mass customization, john wiley. lindsay, b., haas, l., mohan, c, pirahesh, h. & wilms, p. (1986) a snapshot differential refresh algorithm, proceedings of the acm-sigmod international conference on management of data, pp 53-60. roussopoulos, n., chen, c.m., kelly, s., delix, a. & papakonstantinou, y. (1995) the maryland adms project: views r us, data engineering bulletin, vol 18, 2, pp 19-28. segev, a., park, j. (1989) updating distributed materialized views, ieee transaction on knowledge and data engineering, vol 2, 1, pp 173-184. sheth, a.p. & larson, j.a. (1990) federated database systems for managing distributed, heterogeneous and autonomous databases, acm computing surveys, vol 22, 3, pp 183-236. shmueli, 0. & itai, a. (1984) maintenance of views, proceedings of the acm-sigmod international conference on management of data, pp 240-255. weldon, j. (1995) managing multidimensional data: harnessing the power, database programming and design, aug, pp 24-33. woodfill, j. & stonebraker, m. (1983) an implementation of hypothetical relations, proceedings of the conference on very large databases, pp 157-165. yan, w. & larson, p. (1995) eager aggregation and lazy aggregation, proceedings of the conference on very large databases, pp 345-357. zhou, g., hull, r., king, r. & franchitti, j. (1995) supporting data integration and warehousing using h2o, data engineering bulletin, vol.18, 2, pp 41-48. 52 ajis vol 4 no. 1 zhuge, y., gracia-molina, h., hammer, j. & widom, j. (1995) view maintenance in a warehousing environment, proceedings of the acm-sigmod international conference on management of data, pp 316-327. 53 inquiring organizations james courtney, david croasdell & david paradice business analysis and research department texas a&m university college station, texas 77845 409-845-9541 j-courtney@tamu.edu dcroasdell@tamu.edu d-paradice@tamu.edu please direct all correspondence to james courtney abstract churchman (1971) developed five archetypal models of inquiring systems in an effort to expand the field of management information systems along a philosophical path. contemporary businesses can use the ideas developed by churchman to become productive and efficient inquiring organisations. this paper explores the relationship between inquiry and learning in organisations and how information technology can be used to support the process of knowledge creation in the context of inquiring systems. introduction in order for organisations to maintain a competitive edge, they must be capable of continuous learning (senge, 1990). learning organisations must have well-developed core competencies, show continuous improvement, and have the ability to fundamentally renew or revitalise themselves. many authors and researchers are coming forward to describe roles, skills, and tools necessary for the development of learning organisations (e.g., dibella, 1995; nevis et al.; 1995; slater and narver, 1995; huber, 1991). this paper provides a new perspective on learning organisations by viewing them as inquiring systems: systems whose actions result in the creation of knowledge. churchman (1971, p. 18) recasts the theories of knowledge of philosophers leibniz, locke, kant, hegel and singer "in the language and design of inquiring systems," providing "a description of how learning can be designed, and how the design can be justified." reflections on creating knowledge are shaped and interpreted in the context of designing inquiring systems. mason and mitroff (1973) suggested that churchman's models of inquiry be used as the basis for creating evidence in the context of management information systems. hodges (1995, p. 1) believes that the "understanding of the knowledge accumulation process which philosophers have developed throughout the history of human thought is directly applicable to the development of an understanding of knowledge accumulation and use in organisations." this paper follows the leads of churchman, mason and mitroff, and hodges, and interprets the churchmanian inquiring models in the language of the design of learning organisations, or what we call "inquiring organisations." first, we briefly review churchman's descriptions of archetypal inquiring systems, then we describe epistemology and the concept of learning in organisations. finally, we employ churchman's models of inquiring systems to make inferences about the design of inquiring organisations in order to support organisational learning and discuss how information technology is useful in this context. a review of inquiring systems churchman (1971) discusses the writings of philosophers leibniz, locke, kant, hegel and singer in the context of inquiring systems. each of the philosopher's approaches provides for a different way of gathering evidence and building models to represent a view of the world (mason and mitroff, 1973). the basic features of each philosophical inquiring system are summarised in table 1 and elaborated below. the models of inquiry, being systems, have inputs, processes, and outputs. the output of an inquiring system is "true" knowledge, or at least knowledge that is believed not to be false. one of the most distinctive features of inquiring systems design is the inclusion of elaborate mechanisms for "guaranteeing" that only "valid" knowledge is produced. such a concept is not new. the "guarantor" in scientific inquiry is generally based on use of the "scientific method," and scientists in general include many checks and balances and usually exhaust a great deal of time and effort to ensure that the results of their inquiries are acceptable to the rest of the scientific community. each of the approaches described below contain analogous provisions for ensuring that its outputs are consistent with the underlying philosophy, so that the knowledge generated may be considered "valid" for all time. theleibnizian inquirer a leibnizian inquiring system is a closed system with a set of built-in elementary axioms that are used along with formal logic to generate more general fact nets or tautologies. the fact nets are created by identifying hypotheses, each new hypothesis being tested to ensure that it could be derived from, and is consistent with, the basic axioms. once so verified, the hypothesis becomes a new fact within the system. the guarantor of the system is the internal consistency and comprehensiveness of the generated facts. the lockian inquirer inquiring systems based on lockean reasoning are experimental and consensual. empirical information, gathered from external observations, is used inductively to build a representation of the world. elementary observations form the input to the lockean inquirer which has a basic set of labels (or properties) which it assigns to the inputs. the lockean system is also capable of observing its own process by means of "reflection" and backwards tracing of labels to the most elementary labels. agreement on the labels by the lockean community is the guarantor of the system. input given process output guaranto r leibniz none built-in axioms formal logic sentence generator fact nets tautologies contingent truths internal consistency locke elementary observations built-in labels (properties) assign labels to inputs communicatio n taxonomy consensus kant some empirical space-time framework theories construct models from theories interpret data choose best model fact nets fit between data and model hegel some empirical theories construct theses, antithesis dialectic synthesis objective observer singer units and standards system of measurement strategy of agreement sweeping-in new standard exoteric knowledge simplistic optimism replicability hegelian overobserver table 1. summary of inquiring systems the kantian inquirer the kantian system is a mixture of the leibnitzian and lockian approaches in the sense that it contains both theoretical and empirical components. the empirical component is capable of receiving inputs, so the system is open. it generates hypotheses on the basis of inputs received. a clock and kinematic system are used to record the time and space of inputs received. perhaps the most unique feature of kantian systems is that the theoretical component allows an input to be subjected to different interpretations. this occurs because the kantian theoretical component maintains alternative models of the world (alternative world views). representations and interpretations are based on causal connections maintained in the models. the theoretical component contains a model building constituent which constructs leibnizian fact nets. it tests the alternatives by determining the best "fit" for the data, and the guarantor in this approach is the degree of model/data agreement. the use of alternative models permits, for example, one piece of economic data to be interpreted differently by different econometric models (e.g., competing models proposed by different political parties). additionally, an "executive routine" turns the kantian models on and off and can examine their outputs in terms of the degree of satisfaction with their interpretations. thus, if a model is not producing satisfactory results it can be turned off, while those which are more successful proceed. the hegelian inquirer hegelian systems function on the premise that greater enlightenment results from the conflict of ideas. the hegelian dialectic is comprised of three major players. the first player begins the dialectic with a strong conviction about a fundamental thesis. this player or subject, besides holding a strong belief in the thesis, constructs a view of the world in such a way that information, when interpreted through this world view, maximizes support for the thesis. the second player is an observer of the first subject. the observer generates an opposing conviction to the original thesis. in fact, the observer is "passionately dedicated to destruction of the first subject's conviction" (churchman, 1971, p. 173). the final player in the hegelian dialectic is a "bigger" mind and an opposition to the conflict between the thesis and the antithesis. this "bigger" mind synthesizes a new (larger) view of the world which absorbs the thesis/antithesis conflict. synthesis generated by the objective "bigger" mind acts as guarantor of the system. objectivity is based on a kind of interconnection of observers (churchman, 1971, p. 149). they promise that "the movement from thesis-antithesis to synthesis is a soaring to greater heights, to self-awareness, more completeness, betterment, progress" (churchman, 1971, p. 186). the singerian inquirer two basic premises guide singerian inquiry (churchman, 1971, 189-191). the first premise establishes a system of measures that specify steps to be followed in resolving disagreements among members of a community. measures can be transformed and compared where appropriate. the measure of performance is the degree to which differences among group member's opinions can be resolved by the measuring system. a key feature of the measuring system is its ability to replicate its results to ensure consistency. the second principle guiding singerian inquiry is the strategy of agreement. disagreement may occur for various reasons, including the different training and background of observers and inadequate explanatory models. when models fail to explain a phenomenon, new variables and laws are "swept in" to provide guidance and overcome inconsistencies. yet, disagreement is encouraged in singerian inquiry. it is through disagreement that world views come to be improved. complacency is avoided by continuously challenging system knowledge. singerian inquiry provides the capability to choose among a system of measures to create insight and build knowledge. a simplistic optimism drives the community toward continuous improvement of measures. however, the generation of knowledge can move the community away from reality and towards its own form of illusion if not carefully monitored. conceptual foundations of organizational learning to understand how churchman's inquiring systems can be used to support organisational learning, it is useful to have a fundamental understanding of organisational learning concepts. this section presents organisational learning concepts and is followed by a discussion of how inquiring organisations can utilise information technology to support organisational learning processes. epistemology and learning corporate epistemology is the theory of how and why organisations know (von krogh et al., 1994). learning is no longer focused on attaining a "right knowledge," but at least three co-existing pieces of knowledge: syntactic knowledge, pragmatic knowledge, and semantic knowledge. syntactic knowledge pertains to grammar or structure. pragmatic knowledge relates to the situated context within which learning takes place. semantic knowledge deals with the meaning of words and symbols. the contest between different elements of knowledge continuously increases the complexity of the total knowledge conveyed (von krogh et al., 1994). an organisation's knowledge comes in part from the organisation's employees. "individuals have private knowledge that can be a basis for organisational knowledge ... knowledge of the organisation is shared knowledge among organisational members" (von krogh et al., 1994 p. 59). individuals have private knowledge that can be an advantage for organisations, because knowledge from various sources contributes to meaning (white, 1990). ultimately, knowledge is the assimilation and utilisation of some kind of integrated learning system to support "actionable learning" (nevis et al., 1995). creation of a corporate knowledge base allows core competencies to be developed and shared, further allowing actions to be taken that result in incremental or transformational change. learning learning occurs by improving actions through better knowledge and understanding, encoding inferences from history into routines that guide behaviour, and developing insights, knowledge, and associations between past actions, the effectiveness of those actions, and future actions. it involves the understanding of reasons beyond immediate events. learning and cognition are highly context dependent. learning involves the understanding of reasons beyond immediate events. learning facilitates behaviour change that leads to improved performance (fiol & lyles, 1985; senge, 1990; garvin, 1993). while learning can certainly be improved through the use of lessons, lessons are prone to equivocality: the multiple, varied and conflicting interpretations arising from a single source of material. in addition to the use of lessons, learning occurs by improving actions through better knowledge and understanding (foil and lyles, 1985), encoding inferences from history into routines that guide behaviour (levitt and march 1988), and developing insights, knowledge, and associations between past actions, the effectiveness of those actions, and future actions (fiol and lyles, 1985). discovery and affirmation (dibella, 1995) may encourage learners to employ trial and error experimentation or searching mechanisms in order to gain new knowledge. however, structure and organisation (mayhew, 1992) facilitate learning. walsh and ungson (1991) maintain that cultivating and expressly maintaining memory increases learning. they might argue that equivocality reduction is achieved by ensuring the validity of memory. many dichotomies have been used to describe learning styles. descriptions of styles are similar, but include subtle differences. learners can be considered to be adaptive vs. generative, behavioural vs. cognitive, or singleloop vs. double-loop. adaptive learning styles imply an ability to adjust to correct a given situation. the emphasis in generative learning, however, is an attempt to expand capabilities. focus on generative learning occurs in order to make transformational changes (nevis et al, 1995). learning characterised as behavioural development occurs by adopting new responses and actions, while cognitive development occurs through deeper processing and understanding. similarly, argyris and schon (1978) compare single-loop and double-loop learning. the former is a process that results in a change in factual understanding. the latter attempts to resolve incompatible norms by setting new priorities and weighing norms. learning is considered to be low-level when a given set of rules is followed producing consequences for a particular behaviour (argyris and schon, 1978). it is a result of repetition and routines and involves association building. single-loop learning maintains central features or sets of rules. learning at low-levels is restricted to simple error detection and correction. high-level learning is more consistent with double-loop learning in that it aims at adjusting overall rules and norms. the desired consequence is the development of frames of reference and interpretive schemes of cognitive frameworks. high level learning develops an understanding of causation. multi-level learning mixes elements of low-level and high-level learning. multi-level learning occurs when frequently used procedures and specific rules are used to develop competency and understanding (levitt and march, 1988). multi-level learning produces increasing returns to experience via the persistent use of procedures or technologies. multi-level learning is susceptible to competency traps whereby superior procedures are ignored once an inferior procedure or technology is learned and used repeatedly. finally, learning can be considered from the perspective of learning frameworks, learning sources, and developmental orientation. shanks (1995) describes a framework for organisational learning which produces results at three levels: procedural, which engenders continuous incremental improvement; architectural, which attempts to change how work is done; and strategic, which represents change in or reinvention of the business. white (1990) emphasises that knowledge from various sources contributes to meaning. syntactic knowledge has to do with grammar or structure. pragmatic knowledge involves the situated context within which learning takes place. semantic knowledge deals with the meaning of words and symbols. four levels of professional development can be transferred to the context of inquiring organisations. an apprentice is able to understand and apply basic concepts. apprentices move from rote memorisation to understanding on their way to becoming specialists. analysis allows a specialist capable of doing independent work, to move into the generalist realm and to integrate knowledge (possibly across disciplines), creating links between previous unrelated items of information. as a generalist moves toward the renowned, learning becomes synthesis and new knowledge is created. organisational learning organisational learning is the development of new knowledge and insights that have the potential to influence behaviour (e.g. fiol & lyles, 1985; huber, 1991; slater and narver, 1995). when members of an organisation share associations, cognitive systems, and memories, organisational learning is taking place. learning by organisations relies on the people and groups as agents for the transferral of knowledge. over time, what is learned is built into the structure, culture, and memory of the organisation. lessons (knowledge) remain within the organisation even though individuals may change. shanks et al. (1995) theorise that organisational learning improves performance, enhances value, and creates new beginnings. he argues that well designed learning programs improve mental models, facilitate effective analysis, forge commitment, and open senses to the real world. dibella (1995) makes a case for understanding learning organisations using normative, developmental, and capability perspectives. the normative view, typified by senge (1992) and garvin (1993), supports the notion that learning is a collective activity that only takes place under a certain set of circumstances. there is a focus on traits or processes that must be present to ensure learning. this perspective requires some form of managed leadership in order to achieve learning. the developmental perspective considers evolutionary changes and learning through on-going interpretations of experience. organisations pass through developmental stages in order to learn. this is consistent with models presented by nevis et al. (1995) and huber (1991), which sort learning into knowledge acquisition and assimilation, dissemination and sharing, and utilisation. another view describes developmental learning as movement from rote memorisation to understanding of concepts, integration of ideas, and finally synthesis of new ideas. a capability perspective posits that there is no one best way for organisations to learn. according to this perspective, learning processes are embedded in organisational structure and culture. learning occurs through self-discovery and reaffirmation. as new models are presented to the system, it considers where they fit and revises its world view accordingly. inquiring organizations inquiring organisations are learning organisations modelled on the theories of inquiring systems. the first maxim for inquiring organisations is that their actions should be based on valid knowledge. yet organisations faced with the dynamic, and often hostile, environments of today may not have the time nor the resources required to validate knowledge in the scientific sense. moreover, beliefs that were valid yesterday may no longer hold in rapidly changing organisational environments that are socially constructed and subject to change by many forces. thus, while an inquiring organisation should take reasonable and prudent steps to ensure that its actions are based on valid knowledge, in many cases, the only reasonable guarantor is a lockean type of consensus among its members. this assumption would appear to be a potentially fruitful research area, as validation of knowledge for inquiring organisations seems to be substantially different from validation of knowledge in scientific inquiring domains. learning is clearly a fundamental aspect of the inquiring organisation, and a well-designed learning subsystem is critical to its success. as indicated previously, organisations must develop core competencies, improve continuously, and fundamentally renew or revitalise themselves. churchman's descriptions of inquiring systems can be applied to enhance an organisation's ability to achieve success in these areas. the roles of inquiring systems in creating inquiring organisations are briefly described in table 2 and elaborated below. leibnizian-based inquiring organizations the leibnizian inquirer learns by using formal logic to make inferences of cause and effect. a leibnizian system in an inquiring organisation would rely on the theory of autopoiesis for its existence. the theory, taken from cell biology, maintains that the components of a system are used to produce new components and their relations, so as to recreate the system. everything that the system needs for its reproduction is already in the system (von krogh et al., 1994). leibnizian systems are created in a recursive, self-generating, closed and autonomous manner. being a closed system, it only has access to knowledge generated internally. an application of the leibnizian approach may be observed when the policies, goals, ideas of purpose, and core values, established by the organization's designers, serve as leibizian axioms. 'truth" is determined in a procedural manner, with focus on structural or procedural concerns, and with error detection and correction being a direct consequence of comparing inputs with the accepted "axioms" of the system (i.e., organization). the organization's basic theorems, so defined, must be mutually consistent, lending themselves to rote memorization and direct application. furthermore, new ideas, plans, and visions, (i.e., hypotheses) developed within the organization must be compatible with the existing policies, goals, and core values of the organization. as creative tension is exercised to bring the organization closer to its vision, this test of consistency must be continuously reviewed. a consequence of leibnizian inquiry may be the creation of what fiol and march (1985) refer to as competency traps; as learning produces increasing returns to experience via the persistent use of procedures or technologies, superior procedures or technologies are ignored once an inferior technology is learned and used repeatedly. if several "generations" of procedures or technologies have been ignored, conversion to the newer approaches may become more costly and difficult. the gap between employees' existing skills and skills needed for new approaches may cause retraining to be difficult. lockean-based inquiring organizations a community of lockean inquirers learns by observing the world, sharing observations, and creating a consensus about what has been observed. the organisation's culture or subculture (a lockean community) determines the nature of learning and the way in which it occurs. equivocality refers to the multiple, varied and conflicting interpretations about an organisational situation. the lockean inquirer attempts to reduce equivocality by building consensus among team members. agreement by lockean communities helps to establish new direction, agreement, and organisational knowledge. the lockean inquirer is able to support both adaptive and generative learning. lockean systems are open to outside influences and have no built-in preconceptions of the world. these characteristics enhance the firm's generative learning by fostering new ways of looking at the world and preventing rigid adherence to existing standards and ideas. by accepting observational inputs without a biased view, the lockean inquirer may more clearly see, not only how events occur, but also the systems which control the events. this is critical information to facilitate generative learning. the lockean system, with its ability to observe its own process and trace back any label to the most elementary set of labels, supports this need. an application of the lockean approach is demonstrated in a "needs assessment" exercise in which, for example, an organization's managers decide the characteristics needed in a new hire. while complete agreement may not be possible, consensus may be reached at least to the point that an effective advertisement may be created and a search process begun so that a new employee may be identified who embodies the generally agreed upon skill set needed by the organization. the lockean concept of consensual agreement in or between systems enhances the ability of the learning organization to develop a shared vision. examination of causal relationships also enhances understanding, thereby moving the participants in a lockean approach from apprentice-like rote memorization of procedures to a level of specialist understanding and perhaps on to generalist integration. by observational input and inductive labeling, the lockean inquiring system is also able to improve the firm's picture of current reality. these two factors, a shared vision and agreement on current reality, are necessary ingredients for creative tension. an inability to reach consensus, however, marks failure for the system. in this case, decision-making processes may be unacceptably delayed or impossible to complete. in highly emotional or divisive decision-making situations, a lockean approach may not be suitable due to the reduced likelihood of achieving consensus. system learning style learning mechanism learning level learning framework learning source learning orientation developmental orientation obstacles leibniz closed behavioural adaptive single loop simple error detection and correction low procedural syntactic normative apprentice competency traps incorrect inferences about cause and effect locke open consensual generative reduction of equivocality high strategic architectural pragmatic developmental capability specialist generalist failure to reach consensus kant open/closed cognitive generative knowledge scan model matching recursiveness multi-level procedural architectural pragmatic semantic developmental capability specialist generalist competency traps lack of fit between tasks and goals hegel open generative double-loop synthesis by objective mediator high architectural semantic developmental renowned learning disconnected from purpose singer open generative double-loop trial and error agreement and partition multi-level procedural strategic syntactic pragmatic developmental renowned complacency incorrect standards lack of oversight table 2. properties of learning associated with churchmanian inquiry kantian-based inquiring organizations recognizing that knowledge is time variant, a kantian system of inquiry constantly scans internal and external environments for purposeful knowledge. the recursive nature of the kantian inquirer suggests that hew knowledge is created from existing knowledge. environmental scanning and creating knowledge from existing knowledge give the kantian approach characteristics of both closed and open systems. the search for alternate representations, basic to a kantian inquiring system, is a form of generative learning. kantian inquiry acknowledges that inputs received from various knowledge sources may have different interpretations. the kantian inquirer is able to use explicit knowledge and tacit knowledge (i.e., hunches, intuition, experience, insights) to consider the many interpretations of inputs. incoming knowledge is compared to organizational memory allowing the inquirer to consider ways to create and incorporate new knowledge. by considering associations between extant knowledge and new information, the kantian inquirer establishes new world views. an application of the kantian approach can be seen in market testing of new advertising campaigns. different advertisements, exploiting different types of cues, are often tested to determine which advertising approach generates the best response. each advertisement alternative provides a different model to be evaluated. ultimately, one (or perhaps a few) advertisements are selected for general use based on responses from the test subjects. simultaneously, the company represented in the advertisements and the marketing agency producing the advertisements has an opportunity to learn about the product market. failure in a kantian system occurs when no fit exists for the input data in the alternative models or when one model is used repeatedly due to a history of good fit with the data. in the former case, no model emerges as superior and decision-making processes will be stalled. in the latter case, the same competency trap issues emerge as described earlier in a leibnitzian approach. hegelian-based inquiring organizations hegelian synthesis of two opposing models into a new world view is the epitome of an open system and generative learning. hegelian organizations rely upon the dialectic to resolve diametrically opposing viewpoints, the thesis and antithesis. in the hegelian component of an inquiring organization, arbitration is used to evaluate and synthesize contributions from opposing viewpoints resulting in a larger mind which absorbs the thesis/antithesis conflict. knowledge gained through hegelian inquiry may result in an entirely new strategic direction for a given organization, as mason (1969) has shown in his work on dialectical planning systems. the concept of vision versus current reality in a learning organization may be represented by the broad perspective of the observer and the narrow world views of the thesis and the antithesis. in other words, creative tension is analogous to the force which drives the dialectic toward synthesis. an application of the hegelian approach can be seen in the arbitration of employment contracts. the most visable example might be salary and benefit disputes between employers and trade unions. each side of the dispute is representative of the thesis and antithesis. an objective observer is brought in to help promote dialogue and find overlap in the opposing views. eventually, the overlapping views result in an arbitrated agreement between the parties. another example is provided by managerial debate about forecasts. honest debate regarding the meaning of various economic indicators, for example, and their related cause and effect issues, would be representative of hegelian inquiry, assuming the new world view is attained. note that hegelian inquiry depends on synthesis of opposing viewing presenting a win/win scenerio to the organization. in contrast, solutions which offer compromise reflect a lose/lose scenerio to participants. dialectic cannot exist without dialogue. in addition, the hegelian system will fail when learning is disconnected from the purpose of the dialectic debate. as discussed by hodges (1991): 'to understand a statement, it must be interpreted in some context of meanings. in isolation, a statement has no intrinsic truth. contexts of meanings may be built around any statement, including a thesis and antithesis. each is given its true value by its own context of meanings. when the contexts of meanings overlap, a process of translation becomes possible, i.e. the synthesis. if finding the overlapping of meaning was not possible then not only would the thesis and antithesis be contradictory, but their contexts of meanings would be contradictory as well. the latter would in fact be two mutually exclusive worlds." singerian-based inquiring organisations the singerian organisation has the purpose of creating knowledge for choosing the right means for one's end. knowledge must be connected to measurable improvements. measures of performance are judged not only by organisational standards, but also by what is good for all of society. a company has to know the kind of value it intends to provide and to whom. knowledge is generated to be useful for all. in this regard, singerian organisations model contemporary management trends where employees are empowered to contribute in the decision-making process. working environments stress cooperation with fuzzy boundaries where teamwork and common goals are primary driving forces. anyone may act as designer and decision-maker. applications of singerian inquiry are evident in standards making bodies such as ieee and iso. organisations that use metrics as a measure of performance are constantly searching to create and modify existing models. marketers consider models of consumer behaviour, operations management simulates newer, more efficient production facilities, and computer chip manufacturers search for faster processors and methods for utilising greater processing capabilities. rapidly advancing technologies create impetus for singerian organisations to constantly monitor and update methods and models for conducting business. finally, a singerian organisation keeps one eye turned to the needs of society to measure what is possible against what is good for humankind. it support of inquiring organizations having merged churchman's inquiring system descriptions with learning characteristics, we are now in a position to offer suggestions regarding information technology (ft) support for the inquiring system processes described 10 above. table 3 presents a summary of these suggestions. ̂ organisations may employ one or many of these methods of inquiry. technological changes and shifting demands make rapid learning essential in organisations. the advent and increasingly wide utilisation of wide area network tools such as the internet and world wide web provide access to greater and richer sources of information. local area networks and intranets give organisations a way to store and access organisation specific memory and knowledge. used effectively, these tools support the notion of learning in organisations. coupled with the constructs described in this paper, they help support the notion of inquiring organisations. we describe more general aspects of it support for each of the methods of inquiry below. it support for leibnizian inquiry leibnizian inquiring organisations are essentially closed systems. they do not accept inputs from outside sources. however, they are still capable of "learning" by using formal logic to create knowledge. many expert systems operate with a static set of rules. interrogation of the system results in suggested course(s) of action for problem resolution. unlike a database, an expert system can draw upon its rule base to make inferences. some of these systems "learn" by updating the knowledge base as new situations are encountered. organisations employing leibnizian inquiry rely heavily on standard operating procedures and formal instruction mechanisms. computer operations, machine shops, and assembly lines are examples of systems within an organisation with characteristics that lend themselves to leibnizian knowledge creation. rule-based expert systems are easily employed in such environments. it support for lockean inquiry the goal of lockean inquiry is consensus among members of a decision community. in organisations, consensus may be reached by disseminating information among decision-makers with solicitation for feedback, and compilation of results. formal and informal gatherings are often held to iron out any differences. several technologies have been developed to support consensus building among team members. information technologies, which aid lockean decision-makers, include group support systems (gss), computer-supported cooperative work (cscw), computer networks, and distributed databases. these technologies provide mechanisms to bring decision makers together, giving them access to information on-line to support the decision making process. some of these systems allow anonymous participation to encourage non-biased and unencumbered input. lockean inquiry can be used to help define a problem or elements of a task. inputs are classified according to given properties (locke calls this process "labelling"). participants in lockean inquiry make observations about the problem or task and develop methods to solve the problem or carry out the task. lockean inquiry classifies aspects of the task domain allowing decision-makers to decompose the problem and build a solution by reaching agreement. within organisations, lockean inquiry is exhibited when members of the organisation are brought together to perform a task or solve a given problem. unlike hegelian inquiry, synthesis of a new solution is not necessarily required. decision-makers may only need to agree on an existent option (e.g. where they are going for lunch). the boundries drawn in table 3 do not form discrete categories. properties described in the table may in fact overlap between forms of inquiry. similarly, forms of inquiry may overlap between functional levels within an organization. 11 input given process output it support leibniz none standards operating procedures rule base cause and effect analysis inference systematic problem solving simple error detection and correction suggested course of action expert systems locke goals, decisions, standards, policies, and procedures) organisational history organisational structure and culture negotiation communication consensus building equivocality reduction database gss networking kant knowledge sources organisational memory tacit knowledge explicit knowledge working theories knowledge scanning association building recursive learning integrated, timely knowledge internet knowledge base model base hegel mission statement opposing views arbitration conflict resolution renewal enlarged perspective new strategic direction gss dialectron singer units, standards system of measures sweeping-in variables to overcome inconsistency new measures exoteric knowledge case expert systems www objects table 3. summary of it support of inquiring organisations it support for kantian inquiry a kantian inquirer compares what is known with new inputs. learning occurs in the kantian inquiring organisation when inputs are fit into existent models creating new perspectives and generating additional context for decision making. organisational memory plays an important role in kantian inquiry because it provides the model bases and knowledge structure with which inputs are compared. knowledge from a kantian perspective is dynamic by virtue of changing environments and solution requirements. kantian inquiry may be viewed as a method for interpreting inputs to provide direction. in organisations, middle management is responsible for interpreting inputs from upper management and providing direction for lower level organisational members. middle managers use the resources at their disposal to determine how best to fit 19 tasks into the ongoing operations of the organisation. executive information systems, decision support systems, and group support systems that employ organisational models and knowledge sources (e.g., data warehouse, corporate databases, etc.) illustrate ways in which kantian inquiry could manifest itself in learning organisations. corporate intranets and newsgroups are a rich resource for comparing current issues with past decisions. it support for hegelian inquiry hegelian inquiry in organisations has little structure or formal mechanisms to guide it. group support systems that include negotiating and arbitration elements assist organisations in hegelian inquiry. applegate et al. (1987) developed plexsys, a sophisticated planning system, to facilitate idea generation, idea structuring and analysis, and choice of alternative solutions to problems. conklin and begeman (1988) designed gibis (graphic issue based information systems) to facilitate argumentative dialogue among stakeholders in order to help them understand the specific elements of each other's proposals. mason (1969) demonstrates strategic planning as another example of hegelian inquiry within organisations. hodges' (1991) dialectron system can manage the dialogue necessary to generate synthesis between problem domains incorporating thesis and anti thesis. dialectron's architecture results from combining concepts from several works: reseller's (1977) characterisation of dialectic as two parallel disputations, and his inventory of moves and countermoves; ackoff s (1981) concept of interactive planning; axelrod's (1976) cognitive mapping technique; and mitroff, et al.'s (1970, 1971) mathematical model of the dialectic process. the design follows an extensive exploration of the dialectic process. it is capable of supporting two opposing models of the same phenomena, thereby facilitating the dialectic process. it support for singerian inquiry systems and organisations that use metrics practice singerian inquiry. accounting systems are perhaps the sine qua non of measurement, as every enterprise must have one. however, accounting systems measure only the financial health of the firm. to understand and explain the organisation fully, it is necessary to "sweep in" variables from a wide variety of sources both inside and outside organisational boundaries. managers in a singerian organisation should develop measurement standards, continuously compare organisational performance to those standards, and modify models of performance as is required to achieve the standards. numerous examples of metrics exist in information technology. telecommunications standards, reuse libraries, code generators, objects and software metrics all incorporate standards and systems of measurement. the metrics and standards are constantly evolving due to the rapid pace of emerging and improving technologies. organisations who become complacent can loose in a competitive marketplace. other organisational elements that fit a model of singerian inquiry include training offices and marketing departments. training provides a forum for creating and measuring knowledge necessary for work-place activities. marketing departments assign and evaluate sales quotas used to measure the success of organisational members. the internet and world wide web serve as resource and dissemination agents for singerian inquiry. during the sweeping-in process, inquirers are able to use the web to gather and assimilate information that helps refine variables and reduce inconsistencies in the system of measurement. once defined, new measures and standards can be posted to the web and distributed to all interested parties. in this way, the exoteric knowledge goes forward to be useful "for all men in all societies" (churchman, 1971, p. 200). summary to be successful, the modern organisation must be capable of continuous learning. churchman's inquiring system models provide the basis for a new perspective on learning organisations. examining organisations from the point of view of inquiring systems reveals the need for a guarantor of knowledge in learning organisations. a learning system constitutes the core of the learning organisation. by exploring models of inquiry, learning organisations can gain insight into how learning systems may be designed. ways in which information technology may be used to support inquiring organisations have been suggested. references ackoff, r.l. (1981) creating the corporate future: plan or be planned for, john wiley and sons, new york, ny. applegate, l.m., chen, t.t., konsynski, b.r., & nunamaker, j.f. (1987) "knowledge management in organisational planning", journal of management information systems, vol. ill, no 4. argyris, c. & schon, d. (1978) organisational learning, addison-wesley, reading, ma. argyris, c. & schon, d. (1996) organisational learning n, addison-wesley, reading, ma. axelrod, robert, ed. (1976) structure of decision, the cognitive maps of political elites, princeton university press, new jersey. badawy, m. (1993) chapter 5, management of new technology, mcgraw-hill, new york, ny. churchman, c.w. (1971). the design of inquiring systems: -basic concepts of systems and organisations, basic books, inc, new york, ny. conklin, e.j., & begeman, m.l. (1989) "gibis: a tool for all reasons", journal of the american society for information science, may, vol. 40, no 3,200-213. dalton, g., & thompson, p. (1986) novations: strategies for career advancement, scott forsmen, glenview, m. dibella, aj. (1995) "developing learning organisations: a matter of perspective", acadamy of management journal, iss. best paper proceedings 1995, 287-290. fiol, c.m., and lyles, m.a. (1985) "organisational learning", association of management review. vol. 10, no 4. 803-813. garvin, d.a. (1993) "building a learning organisation", harvard business review, july-august, 78-91. hodges, w.s. (1995) five roles which philosophy can play in mis research. panel discussion, philosophy as a reference discipline for mis, ais95. hodges, w.s. (1991) dialectron: a prototypical dialectic engine for the support of strategic planning and strategic decision making. phd dissertation, texas a&m university. huber, g. (1991) "organisational learning: the contributing processes and literature", organisation science, vol. 2, 88-115. levitt, b. and march, j.g. (1988) "organisational learning", annual review of sociology, vol. 14, 319-340. mason, r.o. (1969) "a dialectical approach to strategic planning", management science, april vol. 15, no 8. mason, r.o. & mitroff, i.i. (1973) "a program for research on management information systems", management science, vol. 19, no 5,475-487. mayhew, d.j. (1992) principles and guidelines in software user interface design, prentice hall, new jersey. mitroff, i. (1971) "a communication model of dialectical inquiring systems a strategy for strategic planning", management science, june, vol. 17, no 10, b634-b648. mitroff, i., betz, f. & mason, r.o. (1970) "a mathematical model of churchmanian inquiring systems with special reference to popper's measures for 'the severity of tests'", theory and decision, vol. 1, 155-178. nevis, e.g. dibella, a.j., & gould, j.m. (1995) "understanding organisations as leaning systems", sloan management review, winter, vol. 36,73-85. rescher, n. (1977) dialectics: a controversy-oriented approach to the theory of knowledge, state university of new york press, new york, ny. senge, p. (1990) "the leader's new work: building learning organisations", sloan management review, fall, 7-22. senge, p.m. (1992) the fifth discipline: the art and practice of the learning organisation, doubleday, new york, ny. shanks, d.c. and olsen, d.a. (1995) "the learning organisation", chief executive, march, vol. 101 57-64. slater, s.f. and narver, j.c. (1995) "market orientation and the learning organisation", journal of marketing, july, vol. 59, no 3. 63-74. stein, r. g. and pinchot, g. (1995) "building intelligent organisations", association management november, 32-44. von krogh, g., roos, j., slocum, k. (1994) "an essay on corporate epistemology" strategic management journal, vol. 15,53-71. walsh, j.p. & ungson, g. r. (1991) "organisational memory", academy of management review, vol. 16, no 1, 57-91. white, g., (1990) "natural language understanding and speech recognition," communications of the acm, vol. 33, no. 8, august, 72-82. 14 microsoft word ajis special 2002.doc ajis special issue december 2002 14 design theory in information systems shirley gregor school of business and information management australian national university canberra act 0200 shirley.gregor@anu.edu.au abstract the aim of this paper is to explore an important category of information systems knowledge that is termed “design theory”. this knowledge is distinguished as the fifth of five types of theory: (i) theory for analysing and describing, (ii) theory for understanding, (iii) theory for predicting, (iv) theory for explaining and predicting, and (v) theory for design and action. examples of design theory in information systems are provided, with associated research methods. the limited understanding and recognition of this type of theory in information systems indicates that further debate concerning its nature and role in our discipline is needed. introduction the aim of this paper is to explore the body of information systems knowledge that is referred to by terms such as “design theory” (markus, majchrzak, and gasser, 2002; walls, widmayer, el sawy, 1992), “design science” (march and smith, 1995), and a “theory of artifacts” (weber, 1987). discussion of the nature of theory in general in information systems has been relatively limited, especially in comparison with the considerable debate that occurs over research methods and paradigms. the question of what constitutes theory appears to be a more fundamental question – one that is worthy of more attention (gregor, 2002). the idea of design theory in particular requires critical elaboration. examination of what is meant by theory has occurred in other disciplines. an issue of the academy of management review (1989, vol. 14, no. 4) focussed on theory and theory development. similarly, an issue of administrative science quarterly (1995, vol. 40, no. 3) had articles about what theory is, what theory is not and how theorizing occurs. descriptions of theory in the social sciences can also be found in dubin (1978), freese (1980), kaplan (1964), merton (1967) and weick (1989). more established disciplines have considerable histories of enquiry into the nature of theory. in the philosophy of science there has been discussion of scientific knowledge and the formulation of theory over a very long period (locke, 1689; hume, 1748; popper, 1980). information systems has a number of contributing disciplines, including mathematics, logic, philosophy, computer science, psychology, sociology, economics and management, from which theoretical bases are drawn and adapted. it is not surprising, given that many of our community have come from these different disciplines, that there are different views on what constitutes theory. the information systems discipline is now, however, recognized as a distinct discipline and we should devote some effort to considering the nature and form of theory in our own discipline a definition of information systems suitable for our purposes is that it concerns: the effective design, delivery, use and impact of information technology in organizations and society (avison and fitzgerald, 1995, p. xi). we do not agree with webster and watson (2002) that information systems is another management field, like organizational behaviour. a characteristic that distinguishes information systems from these fields is that it concerns the use of artifacts in human-machine systems. thus, we have a discipline that is at the intersection of knowledge of the properties of physical objects (machines) and knowledge of human behaviour. information systems can be seen to have commonalities with other design disciplines such as architecture or engineering, which also concern both people and artifacts, or with other applied disciplines such as medicine, where the products of scientific knowledge (drugs, treatments) are used with people. in this paper a wide rather than a narrow view of theory is adopted, to encompass a number of different views. dictionary definitions show that the word “theory” can take on many meanings, including: “a coherent group of generic propositions used as principles of explanation for a class of phenomena”; “a proposed explanation whose status is still conjectural, in contrast to well-established propositions that are regarded as reporting matters of fact”; “that department of a science or art which deals with its principles or methods, as distinguished from the practice of it”, “a system of rules or principles”; or “conjecture or opinion” (macquarie dictionary, 1981). thus, the word theory will be used here rather broadly to encompass what might be termed elsewhere conjectures, models, frameworks, or “body of knowledge”. the remainder of this paper proceeds as follows. first, it considers briefly five different types of theory, which are labelled: (i) theory for analysing and describing, (ii) theory for understanding, (iii) theory for predicting, (iv) theory for explaining and predicting, and (v) theory for design and action. the fifth type of theory, design theory, is then explored in more detail. illustrations of relevant work in information systems are provided, as are related research methods, and the form a contribution to knowledge could take. the paper concludes with some implications that arise from discussion of design theory. ajis special issue december 2002 15 the nature of theory in general, theory answers a human need to make sense of the world and to accumulate a body of knowledge that will aid in understanding, explaining, and predicting the things we see around us, as well as providing a basis for action in the real world. in the words of popper (1980, p. 59): scientific theories are universal statements. like all linguistic representations they are systems of signs or symbols. theories are nets cast to catch what we call ‘the world’; to rationalize, to explain and to master it. we endeavour to make the mesh ever finer and finer. theories are seen as uncertain and as approximate representations of reality (fallibilism): the empirical basis of objective science has thus nothing ‘absolute’ about it. science does not rest upon solid bedrock. the bold structure of its theories rises, as it were, above a swamp. it is like a building erected on piles. the piles are driven down from above into the swamp, but not down into any natural or ‘given’ base; and if we stop driving the piles deeper it is not because we have reached firm ground. we simply stop when we are satisfied that the piles are firm enough to carry the structure, at least for the time being (popper, 1980, p. 111). the ontological position adopted here recognizes theory as having an existence separate from the subjective understanding of individuals. a theory can be regarded as: constructions about which there is relative consensus (or at least some movement towards consensus) among those competent (and in the case of more arcane material, trusted) to interpret the substance of the construction (guba and lincoln, 1994, p.113). this position corresponds to ideas expressed by both habermas and popper. habermas (1984) recognizes three different worlds the objective world of actual and possible states of affairs, the subjective world of personal experiences and beliefs, and the social world of normatively regulated social relations. these three worlds are related to popper’s worlds 1, 2 and 3 (popper, 1986). world 1 is the objective world of material things. world 2 is the subjective world of mental states, and world 3 is an objectively existing but abstract world of man-made entities – language, mathematics, knowledge, science, art, ethics and institutions. thus, theory belongs to world 3. a computer program as an electronic artifact exists in world 1, yet the articulation of the algorithm or design principles that are represented in the program belongs to world 3. theory and theoretical knowledge are invented by human beings rather than being discovered. we invent concepts, models and schemes to make sense of experience and, further, we continually test and modify these constructions in the light of new experience. theoretical terms are abstractions, human inventions that are simply convenient devices for managing and expressing the relations among observables. concepts and ideas are invented but correspond to something in the real world. a number of different views of theory are encompassed in this paper with a view to being inclusive. there is still a limit, however, to what is classed as theory. from the definitions above it can be seen that abstraction and generalisation about phenomenon, interactions and causation are thought to be at the core of a theory. we do not regard a collection of facts, or knowledge of an individual fact or event, as theory. “data are not theory” (sutton and shaw, 1995, p. 374), though data may form the foundation for theoretical development. thus, a description of the method used in one case for the construction of an artifact is not theory. views differ on the degree to which generalization or universality is required in theory. popper (1980) provides a detailed coverage on universality and the problems associated with this concept. his view is that the natural sciences should aim at strictly universal statements and theories of natural laws (covering laws), though these laws can never be held with certainty. in the social sciences, however, it is thought unlikely that social phenomena are determined in accordance with strict laws of nature. for example, wars do not result from antecedent political tensions in the same way that earthquakes result from antecedent conditions of plate tectonics (little, 1999). hume (1748) illustrated this view rather well by saying, that in comparison to the universal laws, or laws of nature, knowledge of human affairs is more uncertain, so that: there are other causes, which have been found more irregular and uncertain; nor has rhubarb always proved a purge, or opium a soporific to everyone, who has taken these medicines. (p. 206). nevertheless, we expect in the social sciences (and information systems) that theory should include generalizations to some degree. types of theory theory can be considered on many different dimensions and can be classified in ways that differ from the categorization adopted in this paper. for example, neuman (2000) believes theory can be categorized in terms of (i) the direction (deductive or inductive), (ii) the level of the theory, (iii) whether it is formal or substantive, (iv) the forms of explanations it employs, and (v) the overall framework of assumptions and concepts in which it is embedded. in this paper the categorization of theory depends on the primary purpose that the theory under consideration is to serve. this purpose will in turn depend on the focal question of any research undertaken. for example, if we are interested in describing and categorizing the entities that we believe are relevant to a particular area of enquiry, development of descriptive theory is appropriate. ajis special issue december 2002 16 the following sections describe five different types of theory that are seen as relevant to information systems. a fuller discussion of each theory type is given in gregor (2002). these five different categories of theory are not regarded as mutually exclusive, but are inter-related. type i: theory for analysing and describing descriptive theory says “what is”. descriptive theories are the most basic type of theory. they describe or classify specific dimensions or characteristics of individuals, groups, situations, or events by summarizing the commonalities found in discrete observations. they state ‘what is’. descriptive theories are needed when nothing or very little is known about the phenomenon in question. (fawcett and downs, 1986, p. 4) there are two categories of descriptive theory – naming and classification (stevens, 1984). a naming theory is a description of the dimensions or characteristics of some phenomenon. a classification theory is more elaborate in that it states that the dimensions or characteristics of a given phenomena are structurally interrelated. the dimensions may be mutually exclusive, overlapping, hierarchical, or sequential. classification theories are frequently referred to as typologies, taxonomies or frameworks. research approaches for building descriptive theory include analysis of existing evidence or data, philosophical and historic enquiry and empirical observation. type ii: theory for understanding this type of theory explains “how” and “why” something occurred. it is not formulated in such a way, however, that predictions about the future are made so that they can be tested. at least two types of work may be distinguished here. in the first, theory is used as a “sensitising device” to view the world in a certain way (klein and myers, 1999, p. 75). similarly, dimaggio (1995, p. 391) describes “theory as enlightenment” where theory serves as: a device of sudden enlightenment. from this perspective theory is complex, defamilarizing, rich in paradox. theorists enlighten not through conceptual clarity… but by startling the reader into satori. the point of theory, in this view, is not to generalize, because many generalizations are widely known and rather dull. instead, theory is a ‘surprise machine’ .., a set of categories and domain assumptions aimed at clearing away conventional notions to make room for artful and exciting insights. examples of theory used in this way in information systems are structuration theory and actor-network theory (klein and myers, 1999). structuration theory is a meta-theoretical social framework developed by giddens (1984) that argues that action and structure operate as a duality, simultaneously affecting each other. orlikowski (1992) developed a structurational model of technology which made the claim that technology is both constituted by human agency and constitutes human practice. in a second type of theory for understanding, “conjectures” are drawn from a study of how and why things happened in some particular real world situation. these conjectures could form the basis of subsequent theory development, or be used to inform practice. research approaches that can be used to develop this type of theory include case studies (yin, 1994), surveys, ethnographic, phenomenological and hermeneutic approaches (denzin and lincoln, 1994), and interpretive field studies (klein and myers, 1999). type iii: theory for predicting theories aiming at prediction say “what will be”. these theories are able to predict outcomes from a set of explanatory factors, without necessarily understanding or explaining the causal connections between the dependent and independent variables. associated research approaches include statistical techniques such as correlational or regression analysis. correlational work can be longitudinal, that is, we can show how y varies with a number of independent variables (x1, x2, …) over a time period. correlation studies can also be multi-directional, that is we can say larger values of x are related to larger values of y, and also larger values of y are related to larger values of x (as in height and weight of the population). type iv: theory for explaining and predicting this type of theory says “what is”, “how”, “why” and “what will be”. to many it is the real view of theory (the traditional view). a theory is a set of interrelated constructs (concepts), definitions, and propositions that presents a systematic view of phenomena by specifying relations among variables, with the purpose of explaining and predicting the phenomena. (kerlinger, 1973, p.9) ajis special issue december 2002 17 this theory type implies both prediction and understanding of underlying causes, as well as good description of theoretical constructs. authorities can be found for the dimensions and specification of theories of this type (for example, dubin, 1978). in the social sciences the predictions made are not expected to take the form of universal or covering laws, but rather to be probabilistic-type propositions (little, 1999, p. 705). almost all research methods can be used to investigate aspects of theory of this type, including case studies, surveys, experiments, statistical analysis, field studies, and also interpretive methods if they are used to build theory with predictive power. the grounded theory approach (strauss and corbin, 1994a, 1994b) can be used to develop theory, which is at some point capable of prediction, and thus being tested. type v: theory for design and action this type of theory has two aspects. first, it can concern the methodologies and tools used in the development of information systems. second, it can be about “design principles”, which are design decisions and design knowledge that are intended to be manifested or encapsulated in an artifact, method, process or system. in both cases it must be possible to articulate the principles instantiated in the method, tool, process, or design. it is this articulation, whether in natural language, diagrams or similar, that constitutes the design theory. this fifth type of theory is discussed in greater detail in the section that follows. design theory elaborated design theory is a normative or prescriptive type of theory – it gives guidelines or principles that can be followed in practice. what discussion exists of this type of theory in the information systems literature is scattered and appears under different labels. associated research has been referred to as engineering type research (cecez-kecmanovic 1994), as a constructive type of research (iivari, hirschheim and klein 1998), as prototyping (baskerville and wood-harper, 1998), and a systems development approach (lau, 1997; burstein and gregor, 1999). march and smith (1995) use the term “design science”. the term “design theory” is used here, following walls et al. (1992) there appears to be reluctance by some in the information systems discipline to accept the importance of this type of knowledge. for example, galliers (1992) excluded engineering-type research as a research methodology. it is clear, however, that it occupies an important place in information systems. a review by morrison and george (1995) of three leading management information systems journals showed that softwareengineering related research represented about 45% of the information systems articles found in the six-year period from 1986 to 1991. iivari et al. (1998) also argue for the particular importance of these types of methods for applied disciplines, such as information systems and computer science. design theory concerns both how to undertake the building of an artifact (development process knowledge) and what the artifact should look like when built (design principles). these aspects or design theory are intertwined. artifacts based on information technology are not static but evolve – through human actions and from new technological development. in addition, the nature of the artifact is likely to reflect the tools and methodologies used in its construction. some design theory links particular types of artifacts to particular methods for their construction. for example, truex et al. (1999) make the claim that organizations that are not stable, but continuously adapting to their environment, need system development methods that assume that systems should be under constant development, can never be fully specified, and are subject to constant adjustment and adaptation. there are a number of views of design theory, both in information systems and in related disciplines. cook and campbell (1979. p. 28) identified the requirement for knowledge of this type in the social sciences, but used the term “recipe”: knowledge of causal manipulanda, even the tentative, partial and probabilistic knowledge of which we are capable, can help improve social life. what policy makers and individual citizens seem to want from science are recipes that can be followed and that usually lead to desired positive effects, even if understanding of the micromediational processes is only partial and the positive effects are not invariably brought about. in 1987 weber indicated the difficulties with some instances of design work. he saw the “lure of design and construction” as one of three factors inhibiting the progress of information systems as a discipline. the conundrum posed by design research for progress in a discipline emerges clearly when a paper describing such research must be evaluated for publication in a learned journal. what are the quality standards the reviewer must apply to decide upon its acceptability? typically the paper contains no theory, no hypotheses, no experimental design, and no data analysis. traditional evaluation criteria cannot be used. the paper’s contribution inevitably requires an inherently subjective evaluation (p. 9). landauer (1987), points out problems with the argument that systems design can be guided by an underlying design theory: ajis special issue december 2002 18 previous research findings and theories are likely, at best, to be relevant only to a small portion of the total situation. it means that testing the effectiveness of one total system against that of another is likely to yield information of extremely limited generality (p. 157). walls et al. (1992) considered that an information systems design theory had two distinctive characteristics: a theoretical base and explicit guidance for practitioners. design theory should have three interrelated components: a set of user requirements, a set of system features (or principles for selecting system features), and a set of principles deemed effective for guiding the process of development. the theory underlying an information systems design theory (referred to as “kernel theory”) could be an academic theory (e.g cognitive psychology) or a practitioner theory-in-use (sarker and lee, 2002). these authors argued that the value of an is design theory lay in the diminution of developers’ uncertainty through a reduction of the range of allowable system features and development activities to a more manageable set, thereby increasing the likelihood of success. in the field of management accounting, kasanen, lukka and siitonen (1993) argued for the usefulness of a “constructive approach” to research, which promoted managerial problem solving through the construction of models, diagrams, plans, organizations and so on. these authors recognize that in their field such research suffered from a perception that it is “non-scientific” and is liable to be confused with consulting. march and smith (1995) discussed “design science” in terms of the development of understanding of the natural laws governing information technology systems and the natural laws governing the environments in which they operate. they considered that two groups of activities are needed for design science. the first includes the research activities of: build, evaluate, theorize and justify. the second includes the outputs of representational constructs, models, methods, and instantiations. in the field of software engineering, glass (1996) discussed the close relationship between theory and practice, showing how practice can lead to theory, and theory to practice. this relationship is illustrated in aerodynamics, where the invention of the airfoil preceded and motivated the science of aerodynamics. glass points out that practice may lead to theory –the best methodologies and representations for software design in the world will not necessarily lead us to superior designs. there is something else in the act of design – the art, skill and experience of the designer. limited discussion of research approaches for design theory can be found. exceptions are burstein and gregor (1999), march and smith (1995), nunamaker, chen and purdin (1990-91), nunamaker and chen (1990). action research is seen as particularly appropriate (baskerville and wood-harper, 1996). lau (1997) reviewed the use of action research in information systems studies over a twenty-five year period. eleven of the thirty articles reviewed were categorized as “systems development”, covering the areas of analysis, design, development and implementation of information systems and decision support systems. the above discussion shows a range of thoughts on design theory in information system and allied fields. it can been seen that to date a coherent unified view has not emerged, and that there is some unease as to whether design theory is “scientific” and properly distinguishable from consulting. towards a better understanding of design theory the view adopted here is that design theory is, in essence, a variation of traditional theory that both explains and predicts. in this view design theory should have well-defined constructs, definitions and propositions that both explain and predict phenomena. the difference is that this type of theory has an additional role in that it is normative or prescriptive. the propositions go a step further and say “how something should be done”. the paper by markus et al (2002) is a good example of a theory of this type. it gives explicit design and development principles for the class of application systems that supports emergent knowledge processes. an example advanced of a design principle is: “design for implicit guidance through a dialectical development process”. this principle (proposition) has a predictive element and can be phrased in a fashion similar to propositions in traditional theory. that is, this principle implies that “a dialectical development process is better (in some sense) than alternative development processes for the class of systems that support emergent knowledge processes”. other researchers can test this proposition with other systems of this nature. in their own research the authors show that an alternative development process “of explicit guidance” was faulty. the design theory proposed consists of general principles to solve a class of business problems, rather than a unique set of system features to solve a unique business problem. the abstraction and generalization that occurred here distinguishes the study from what would occur in consulting. design theory can be informed by all the other types of theory. a general, formalized methodology can build on particular idiographic studies of what has worked in practice, on predictive relationships that are known but not fully understood and on traditional theories such as those relating to data representation or human behaviour. figure 1 illustrates these relationships. ajis special issue december 2002 19 (v) design theory (normative, prescriptive) (iv) traditional theory (explanatory and predictive) (ii) explanatory theory (i) descriptive theory (iii) predictive theory figure 1: interelationships among theory types the interrelationships between the theory types can be bi-directional. work on design theory could lead to advances in other types of theory. for example, principles of interface design that are tested in software trials could lead to improvement in underlying theories of cognitive psychology (of the traditional type). similarly, practical implementations of methods may lead to refinement of frameworks, taxonomies or construct definition. an apparent divergence in opinions can be detected in the views of design theory. one view places emphasis on the development of deep theories of information systems of the traditional type (type iv), which can then inform design practice. for example, weber (1987), saw a solution to the dilemma posed by design and construction work if: researchers show how their designs are based in some theory of information systems. the theory can be used to predict the likely success or failure of a design, and these predictions can be tested empirically. traditional evaluation criteria can then be applied (p. 9). weber’s work is clearly motivated by this aim. he provides what could be termed a “theory of representation in information systems” using a formalization of a system based on the ontology of mario bunge (weber, 1997). this theory has implications for design practice in that it allows predictions about the strengths and weaknesses of various forms of information system representations (grammars) that can be tested in empirical work. cumulative work on this theory has continued with studies aimed at testing propositions derived from the theory (for example, bodart et al., 2001). march and smith (1995) discuss “design science”, but do not regard theory as a product of design science: “rather than posing theories, design scientists strive to create models, methods and implementations that are innovative and valuable.” (p. 254). markus et al. (2002), following walls et al. (1992) recognize design “theory”, but believe that “is design theory is not a radical departure from established is practice and theorizing” (p. 181). we argue for an alternate view that recognizes design theory (type v) as special case of traditional theory (type iv), but distinguished from it as a separate category, so that its particular characteristics can be discussed. in much work of the traditional-theory type the primary focus is an integrated body of knowledge – the design implications are secondary. in contrast, with design theory the primary focus is the general design principles that inform practice, as cook and campbell (1979) noted with their idea of “recipes”. the knowledge needs to be stated in such a way that it provides guidance in practice for the design, development and use of information systems. design theory can arise from a situation in which imagination, art, or a designer’s insights result in a new way of solving problems being “discovered”. this discovery can be instantiated in an artifact, providing “proof-ofconcept”. the new design theory could be based on existing theories of type iv – what walls et al. (1992) ajis special issue december 2002 20 term “kernel” theory” – that are combined together or interpreted in new or novel ways. this situation is similar to “theory-building” with traditional theory, and is congruent with views in the philosophy of science. popper (1980) says that the act of conceiving or inventing a theory is a matter of psychology and not the province of science. what constitutes a contribution to knowledge with design-type research? some criteria adapted from march and smith (1995) are: • in the theory-building phase, the value or utility to a community of users is important, especially with the first artifact of any type: building the first of any set of constructs, model, method or instantiation is deemed to be research, provided the artifact has utility for an important task. the research contribution lies in the novelty of the artifact and in the persuasiveness of the claims that it is effective. (p. 260) • with research that builds subsequent constructs, models, methods and instantiations, judgement is based on ‘significant improvement”; whether the artifact offers, for example, improvements such as better performance, efficiency, understandability or ease-of-use. • instantiations that apply known constructs, methods and principles, to novel tasks may be of little significance. the prime concern is with the uncertainty of the success of the artifact – for example, applying expert system technology to build a system in a well-structured domain is likely to be of little significance when there is little uncertainty about its chance of success. • in theory-testing, the evaluation of artefacts is worthwhile design theory research. different artefacts can be evaluated with respect to different metrics. for example, evaluation of constructs (such as data modelling formalisms) tends to involve completeness, simplicity, elegance, understandability and ease of use. models (such as database design models) are evaluated in respect of their fidelity with realworld phenomena, completeness, level of detail, robustness, internal consistency and usefulness in practice. evaluation of methods considers operationability (the capacity of an algorithm to perform a task or of humans to effectively use the method), efficiency, generality, and ease of use. for example, systems development methods or tools can be evaluated with respect to the quality of results obtained by analysts applying the methods or tools. in addition, it is assumed that the general design principles represented in the artifact are articulated clearly so that they can be followed and tested by others. criteria for software development (design) research are discussed further in burstein and gregor (1999). the study by markus et al (2002) provides a good example of design theory. the claim is made, and justified, that there was no suitable existing design theory for a particular class of systems. thus, the claim is made that the design principles proposed are new and novel. design principles were drawn from the development of a tool that was shown to be successful in two pilot studies (though little detail of evaluation or use is given). the design theory (principles) is well articulated and suggestions are given as to how this theory can be tested in further work. discussion and conclusions this paper has distinguished five different types of theory important for the discipline of information systems: (i) theory for analysing and describing, (ii) theory for understanding, (iii) theory for predicting, (iv) theory for explaining and predicting, and (v) theory for design and action. the fifth category of design theory was discussed in detail. prior discussion of this type of theory in information systems is relatively sparse and what work does exist is not entirely congruent. we argue that design theory should be distinguished from other types of theory to allow its nature to be examined in some detail. design theory is seen as a special case of traditional theory that allows for both explanation and prediction and testing of propositions. the distinctive feature of design theory is that it is normative and gives guidance (design principles) for action in the real world. these design principles relate to artifacts that include system or subsystem models, artifact features and development methodologies and tools. the importance of design theory in information systems should be recognized. we are essentially an applied discipline, one in which our work should provide guidance to people who have to take action in the world. thus, a critical examination of what design theory entails is valuable. it would be advantageous if we could come to some common understanding of the form that knowledge claims in this theory can take, and the means by which such knowledge claims are developed and tested. ajis special issue december 2002 21 references avison, d. & fitzgerald, g. (1995) information systems development: methodologies, techniques and tools, 2nd ed., mcgraw-hill, london. baskerville, r. & wood-harper, a. t. (1998) “diversity in information systems action research methods”, european journal of information systems, 7, 90-107. bodart, f., patel, a., sim, m. & weber, r. (2001) “should optional properties be used in conceptual modelling? a theory and three empirical tests, information systems research, 12(4), pp 384-405. burstein, f. & gregor, s. (1999) “the systems development or engineering approach to research in information systems: an action research perspective” in b. hope & p. yoong (eds) proceedings of the 10th australasian conference on information systems, victoria university of wellington, new zealand, pp 122-134. cecez-kecmanovic, d. (1994) “engineering type information systems research: a discussion on its position and quality”, proceedings of the 5th australian information systems conference, caulfield, vic., monash department of information systems, pp 767-770. cook, t. d. & campbell, d. t. (1979) quasi-experimentation design and analysis issues for field setting, boston, mal houghton mifflin. denzin, n. k., & lincoln, y. s. (eds.), (1994) handbook of qualitative research, thousand oaks, ca: sage. dimaggio, p. j. (1995) “comments on ‘what theory is not’”, administrative sciences quarterly, 40(3), pp 391-397. dubin, r. (1978) theory building. (rev. ed.). london: free press. fawcett, j. & downs, f.s. (1986). the relationship of theory and research, norwalk, ct: appletoncentury-crofts. freese, l. (1980) “formal theorizing”, annual review of sociology, 6, pp 187-212, palo alto, ca. galliers, r. (1992) “choosing information systems research approaches” in galliers, r. (ed.) information systems research: issues, methods and practical guidelines, blackwell scientific publications, oxford, pp 144-162. giddens, a. (1984) the constitution of society. cambridge: polity press. glass, r. l. (1996) “the relationship between theory and practice in software engineering”, communications of the acm, 39(11), pp 11-13. gregor, s. (2002) “a theory of theories in information systems” in gregor, s. & hart, d. (eds.), information systems foundations: building the theoretical base, australian national university, canberra, pp 1-20. guba, e. g. & lincoln, y. (1994) “competing paradigms in qualitative research” in denzin, n. k. &. lincoln, y. s. (eds.), handbook of qualitative research, thousand oaks, ca: sage, pp 105-117. habermas, j. (1984) theory of communicative action., vol 1: reason and the rationalization of society. heinemann, london, u.k. hume, d. (1748) “an enquiry concerning human understanding”, reprinted in perry j. & bratman, m. (eds.), introduction to philosophy classical and contemporary readings (3rd ed.). new york: oxford university press , 1999. iivari, j., hirschheim, r. & klein, h. k. (1998) “a paradigmatic analysis contrasting information systems development approaches and methodologies”, information systems research, june , pp 164 – 193. kaplan, a. (1964) the conduct of enquiry. new york: harper row. kasanen, e., lukka, k., & siitonen, a. (1993) “the constructive approach in management accounting research”, journal of management accounting research, 5, pp 243-264. kerlinger, f.n. (1973) foundations of behavioural research, (2nd ed.) new york: holt, rinehart and winston. klein, h. & myers, m. (1999) “a set of principles for conducting and evaluating interpretive field studies”. mis quarterly, 23(1), pp 67-93. landauer, t. k. (1987) “relations between cognitive psychology and computer system design” in carroll, j. m. (ed.), interfacing thought, mit press. lau, f. (1997) “a review on the use of action research in information systems studies” in liebenau, l. a. & degross, j. (eds.), information systems and qualitative research, chapman & hall, london, pp 3168. little, d.e. (1999) “philosophy of the social sciences” in audi, r. (ed.) the cambridge dictionary of philosophy (2nd ed.) cambridge, uk: cambridge university press, pp 704-706. locke, j. (1689) “essay concerning human understanding” reprinted in perry, j. & bratman, m. (eds.) introduction to philosophy classical and contemporary readings (3rd ed.) new york: oxford university press, 1999. march, s.t. & smith, g.f. (1995) “design and natural science research on information technology”, decision support systems, 15, pp 251-266. markus, l., majchrzak, a., & gasser, l. (2002) “a design theory for systems that support emergent knowledge processes”, mis quarterly, 26(3), pp 179-212. merton, r.k. (1967) theoretical sociology. new york: free press. ajis special issue december 2002 22 morrison, j. & george, j.f. (1995) “exploring the software engineering component in mis research”, communications of the acm, 38,7, pp 80-91. neuman, w. l. (2000) social research methods (4th edn.) boston: allyn & bacon. nunamaker, j., chen, m. & purdin, t. (1990-91) “systems development in information systems research”, journal of management information systems, 7 (3), pp 89-106. nunamaker, j.f. & chen, m. (1990) “systems development in information systems research”, ieee, pp 631639. orlikowski. w. (1992) “the duality of technology: rethinking the concept of technology in organizations”, organization science, 3(3), pp 398-427. popper, k. (1959) the logic of scientific discovery, london: unwin hyman. popper, k. (1986). unended quest an intellectual autobiography, glasgow: fontana. sarker, s., & lee, a.s. (2002) “using a positivist case research methodology to test three competing theories-inuse of business process reengineering”, journal of the ais, 2(7) (online). stevens, b.j. (1984) nursing theory. analysis, application, evaluation, (2nd ed.). boston: little, brown. strauss, a., & corbin, a. (1994a) basics of qualitative research: grounded theory procedures and techniques, newbury park, ca: sage. strauss, a., & corbin, a. (1994b) “grounded theory methodology: an overview” in denzin, n.k., & lincoln, y. s. (eds.), handbook of qualitative research, thousand oaks, ca: sage, pp 273-285 sutton, r.i & staw, b.m. (1995) “what theory is not”, administrative sciences quarterly, 40(3), pp 371384. the macquarie dictionary (1981) macquarie library, mcmahons point, nsw, australia. truex, d. p., baskerville, r., & klein, h. (1999) “growing systems in emergent organizations”, communications of the acm, 42(8), pp 117-123. walls, j. g., widmeyer, g. r., & el sawy, o. a. (1992) “building an information system design theory for vigilant eis”, information systems research, 3(1), pp 36-59. weber, r. (1987) “toward a theory of artifacts: a paradigmatic base for information systems research”, journal of information systems, spring, pp 3-19. weber, r. (1997) ontological foundations of information systems, melbourne, vic.: coopers & lybrand. webster, j. & watson, r. t. (2002) “analysing the past to prepare for the future: writing a literature review”, mis quarterly, 26(2), xiii-xxiii. weick, k. e. (1989) “theory construction as disciplined imagination”, academy of management review, 14(4), pp 516-531. yin, r. k. (1994) case study research design and methods (2nd ed.), thousand oaks, ca: sage. understanding support lessons from a case study chris sauer fujitsu centre for managing information technology and organisation australian graduate school of management university of new south wales kensington nsw australia abstract support from top management and others is agreed to be an important factor in information systems success and failure, but little is written about how it has its effect and how it might be managed to a project's advantage. a recently developed conceptual framework is described. it covers the nature and forms of support, the way support affects project outcomes, the bases on which support is provided, and the strategies by which support may be managed. the framework is used to analyse a case study in several stages. at the end of the analysis of each stage, the framework's utility is assessed in terms of its explanatory value and the practical advice it suggests. areas for further research are identified. introduction top management support has long been recognised as an important factor in information systems success (ackoff 1960, ginzberg 1975 cited in keen & scott-morton 1978, walton 1989, jarvenpaa & ives 1991). some writers have recognised the need for support from other sources such as the senior mis executive (morgan & soden 1973), user managers (argyris 1971, lucas, ginzberg & schultz 1990), and the organisation itself (lucas, ginzberg and schultz 1990). for reasons that remain unclear, despite apparently widespread agreement on the importance of support, there has been relatively little research into its role in information systems projects (ein-dor & segev 1981). assuming that support does function in ways which are of benefit, it follows that project managers would be advised to consider how to ensure that they receive the support they need. support, then, needs to be managed just as does the technical effort of building and maintaining a system. sauer (1993a) has started the process of constructing a conceptual framework for theorising about support and support management. this will allow us to employ a richer conceptual vocabulary, and, in the longer term, to increase our understanding through a better appreciation of the ways in which support works and how it can be managed to the benefit of a project organisation. as it stands, sauer's conceptual framework is unrefined. it is a pudding awaiting proof. it is the aim of this paper therefore to test it out on a case study. this will do three things. one, it will allow us to observe its applicability how useful is it, what does it help explain, what does it not explain? two, it will raise some interesting research problems. and three, it will suggest some practical morals for project managers. the next section of this paper summarises sauer's framework. it is followed by a section which briefly introduces the case study. the case can be readily divided into temporal segments, and so is analysed segment by segment in the final section the outcomes of the analysis are summarised and conclusions drawn. the conceptual framework sauer's conceptual framework has four main aspects. the first is constitutive and originative it says what kind of resource or behaviour constitutes support and indicates its possible sources or origins. the second is functional: it begins the task of showing how support works. the third identifies the bases of support. then, equipped with knowledge of the way support helps a project and the bases on which it may be provided, it becomes possible to start to identify and analyse the elements of the fourth aspect of the framework, namely those practices and strategies by which support may be managed by a project organisation. starting with the first aspect of the framework, what constitutes support? sauer (1993a) defines it recursively as follows: september 1993 63 1) the supply of resources, services, and assistance (in controlling contextual contingencies) which contribute to the innovation process conducted by a project organisation, or 2) activities which generate support anything satisfying the first part of the definition counts as primary support. thus, the provision of resources (funding), the control of organisational contingencies (fixing), and the technical effort that goes into the innovation process of developing, implementing, operating, and maintaining a system (normal technical work) are all forms of primary support. sauer also proposes that as system use is usually regarded as critical to success (lyytinen & hirschheim 1987), it also be regarded as a primary form of support. behaviours which satisfy the second part of the definition count as secondary support. there are three forms of secondary support. mediation between a project organisation and another source of support counts as power-broking. activities which promote the social legitimacy of the system or project is another form of support. commitment is the third. the framework identifies two types of source of support internal and external. it is quite apparent that much support will be external to the project organisation from managers, users and others. the purpose of this distinction is largely to raise awareness of the fact that internal support is a crucial variable too. it should not be overlooked. how does support work? plainly, its ultimate value lies in providing necessary inputs to a project. the value of secondary support comes from facilitating this. support also works by helping the project organisation cope with flaws in the innovation process. this can either be by solving the problem that constitutes the flaw (for example, correcting a bug) or by absorbing the flaw (for example, by 'working around' it). such support has two possible modes. it can be anticipatory in being ready for activation when required budgets are anticipatory in this sense. or, support can be reactive if it is mobilised only in reaction to the appearance of a problem. what is the basis for support? why is it provided? sauer (1992) argued that support was given in exchange for system benefits. sauer (1993a) views this as too limiting, and suggests that structural and cultural factors are also significant. where there is an exchange of benefits for support, the benefits may be direct when enjoyed by the supporter or indirect if they are enjoyed by some other party in a network of relationships whom the supporter wishes advantaged. when support is supplied by virtue of some standard organisational arrangement (for example, the organisation's software group maintains system software common to all projects) then it can be said to be structural. when support is supplied because of values shared by a supporter and the project organisation or where certain values are implicit in the system (for example, a believer in organisational decentralisation might support a networked system because it advances the philosophy of decentralisation even though it does not offer her any direct benefits) then it can be said to be culturally based support. the final component of the framework asks what kinds of support management strategy and practice are available to a project organisation. the support management practices are divided into the structural and non-structural. the structural are those which attempt to institute organisational practices which will assure certain forms of support. they are likely to be the most systematic. the non-structural consist of those activities which attempt to manage support from a single individual and those which are more general in their purpose. thus, practices such as cooption will be in the former category while attempts to make an organisation's culture more generally supportive will be in the latter category. support management strategies will combine various kinds of management practice to achieve a particular emphasis. four are listed in the current framework. the targetted strategy singles out a small number of important/powerful supporters. the sprayshot approach tries to win support from as many sources as possible. the capture-a-champion strategy involves finding an appropriate stakeholder to champion the project. the decoupling strategy involves trying to reduce the project organisation's dependence on its innovation efforts by finding a sponsor who is less interested in the system's benefits a central government granting agency might fit the bill (see kraemer et al 1989 for examples). this taxonomy of strategies helps organise the different ways in which support management practices may be employed. the framework described in this section is summarised in table 1. it is valuable both for before-thefact analysis and planning for information systems innovation, and for after-the-fact understanding and explanation. it is therefore both a research tool and an aid for the practitioner. an assessment can be made of what support is needed in terms of the different forms of support, its different 64 ajis possible sources, and the modes in which it will need to operate. analysis of a project organisation's access to support can be achieved via an assessment of the actual bases of support. the map that this analysis provides of the organisational terrain can be used to guide and critically assess the selection of a support management strategy and the practices by which it is to be implemented. the framework will be tested on the case study by using it for explanatory analysis. what is support? primary funding, fixing, normal technical work, use secondary power-broking, promotion of social legitimacy, commitment how does support work? provides necessary inputs copes with flaws overcomes them, absorbs them modes anticipatory, reactive what are the bases of support? exchange of benefits for support, structural arrangements, cultural norms what types of support management structural, non-structural practice are there? what support management strategies targetted, sprayshot, capture-a-champion, decouple system from are there? support table 1: summary of a framework for understanding support and support management the case study the case to be used was researched by the author through interviews and extensive file research mostly between 1985 and 1988. the intention was largely to explore the problem of information systems failure. one of the insights it threw up was that support had played a major role in the project's life. a more thorough account of the research method can be found in sauer (1992). the organisation that sponsored the project was the office of the australian public service board (referred to here as the public service board). the public service board was directed by a formal board consisting of a chairman and two commissioners (referred to here simply as the board). its general responsibility covered all matters to do with employment, industrial relations, training and the like for the australian public service. the system, called mandata, was intended to automate the administration of personnel and establishments processes for the whole of the australian public service. the project ran throughout the 1970s before being finally terminated in 1981. what makes it particularly interesting for our present purpose is that support for the project varied considerably over its lifetime. at one stage the project was given a new lease of life by an injection of support while its final demise can be seen to have resulted in part from a chronic haemmorhage of support. the project's history can usefully be divided into a sequence of temporal segments or stages. the first was an extended initiation stage which involved the gathering of support. the second was an initial development stage that was bedevilled by resource problems. this ended in a review which resulted in a major reorientation of the innovation strategy and a reinjection of support. during the subsequent consolidation stage the innovation strategy began to give way under the strain imposed on it while the project as a whole was subjected to a barrage of criticism. a further review was instituted resulting in a partial abandonment of the project, trimming it back to a minimum viable system, an outcome which left it vulnerable to the environment of public spending cuts. the abbreviated analysis given here necessarily simplifies and finesses much of the project's complexity. however, the full details are not critical to this paper. a far fuller account and analysis is given in sauer (1992). september 1993 65 analysis of the case the analysis of the case tests the conceptual framework as an after-the-fact aid to explanation. it is also expected to identify problems for research and useful lessons or principles for practice. each stage of the case will be analysed in turn through the following eight questions: 1. what support was needed? 2. what was the support management strategy? 3. what support did it generate? 4. did the support generated match the support needed? 5. what was the outcome for the project of the answer to 4? 6. what explanations does the framework yield? 7. what further research is needed on the framework? 8. what practical advice is suggested? stage one initiation the goal of this stage was to obtain a strategic decision to proceed to develop the mandata project. formally this was to be a decision by the cabinet of the commonwealth government to fund equipment and system software. to obtain this, a feasibility study had to be launched and funded; its report had to be helped through a lengthy consultation and approval process; a revised proposal had to be funded; and the process of preparing for system development had to be started. the feasibility study report indicated a particular timeframe for this stage which was expected to result in a submission to cabinet for funding in september 1972. as well as needing support for these essentials of the initiation stage, the project ideally needed a supportive context to be established (walton 1989) anticipatory of future support needs. the strategy adopted for winning support in this stage was targetted although it had elements of the sprayshot approach as well. its central concern was to navigate a path through a sequence of formal decisions. the board and then various specialist committees had to approve before a proposal could be put to cabinet. these approvals were substantive decisions which would affect primary support, particularly resourcing. so, the formal decision process imposed a degree of targeting. the strategy was also sprayshot in that some degree of verbal support was required from the permanent heads of public service departments and statutory authorities and from the staff associations. this aspect of the strategy was geared to assuring a measure of secondary support for the project, especially social legitimacy, but not to developing primary support. initially, the public service board's head of management services was a champion for the project. he was able to persuade the board to support a feasibility study through funding and obtaining secondments. in february 1972, just after completion of this study he retired to be replaced by a less actively supportive manager in approximately the same gatekeeping role. from then on during this stage support had a largely structural basis in that it was obtained by virtue of passage through a formal decision-making process. the adp departmental systems branch of the public service board received passive support insofar as it was able to persist with the initiation process even though it had no special funding to do so and was required to maintain a full workload at the same time. the formal process of consulting the permanent heads and the staff associations yielded some enhanced legitimacy for the project in the form of a number of statements of support although some responses were qualified or hesitant. for example, one said, 'in our experience, the development of an organisation of the size envisaged would take many years before producing a worthwhile output'. and, one reason given for delays in putting forward the cabinet submission was the tardiness of the staff associations. however, by mid-1973, the board was sufficiently convinced that the project would go ahead that it agreed to resource twenty-eight new positions for mandata in the financial year 1973-74. the strategy of targetting key decision-makers through the formal decision process ensured that the project received a measure of primary and secondary support sufficient to meet the goal of a favourable cabinet decision. however, the lack of earmarked funding for initiation between the end of the feasibility study and mid-1973 combined with the loss of the project's champion meant that 66 ajis there was no individual or group with sufficient status, access to the board and time available to enable them to meet the schedule laid down in the feasibility study report. moreover the formal approach of seeking approval from permanent heads and staff associations yielded only an increase in perceived legitimacy for the project. in not seeking any primary support or even any formal commitment this approach failed to build up a supportive context for future stages. perhaps most damaging to the project's future was the precise form of the strategic decision. it was a decision by cabinet to resource mandata's equipment including some software. it was not an explicit decision to provide either adequate staff or accommodation. the outcome of this stage was success insofar as the project was approved. however, it was success bought at a price that could not have been easily foreseen or avoided. because of limited support the project was delayed by nearly eighteen months. this had three undesirable outcomes. first, it set up a situation in which the project's broader context was to be particularly damaging it was to be caught by funding restrictions as a result of changing national economic and political circumstances. second, the delay was to be used later as a criticism when the project became subject to external scrutiny. third, it had not been possible to build up a strongly supportive context. no agreements had been obtained for primary or secondary support beyond the general approvals from permanent heads and staff associations. no special level of commitment had been built up nor had primary support been promised. thus when the project found itself squeezed it had no reserves of support to call upon. the lessons of this stage are several. in terms of explaining the project's later difficulties, it is important to understand that limitations to the support available after the feasibility study generated delays which were to subject the project to very damaging contextual circumstances which it lacked the capacity to combat completely. apparent success in initiation masked what was to become a more deep rooted problem. in terms of the researcher/theorist's interests a number of problems emerge. first, there is a problem with assigning a strategy type to a set of management practices. in this case, the strategy has been assumed to be largely targetted in that it pursued a formal decision and approval process in which primary support was explicitly asked for from cabinet alone. the support sought from the other parties appears to have been nominal. however, this may be influenced by the acuity of hindsight. was it truly a targetted support management strategy? at the time it would have been difficult to estimate what support had in fact been garnered by the initiation process. this constitutes a second, more general problem for work which uses the concept of anticipatory support. how can support be identified in advance of its being called upon? this appears to be a major problem in advancing from the explanatory use of the current conceptual framework to its use predictively to guide practice. it plainly warrants further research. there are several lessons for the practising manager. the first is that most projects need a variety of forms of support, and apparent success in obtaining that form of support which has most visibility may hide shortfalls elsewhere. second, it is worth noting that delays can be damaging for more reasons than that benefits are deferred. uncertainty about the future increases with distance the further into the future a project is delayed the less certainty there can be about the difficulties likely to arise and therefore about the support that will be needed. a third point related to the second is that it is usually easier to keep established funding than to win new funding. the earlier in a project that support can be won, particularly funding, the better. consequently, obtaining funding should be a very high priority for most projects. fourth, as walton (1989) has proposed, it is important to build up a supportive context. but, that support needs to be balanced. it is not sufficient to have, say, strong legitimacy but no fixers or power-brokers willing and able to help out with unforeseen difficulties. because some of the effects relating to support developed in this stage only emerged subsequently, further practical morals arising out of initiation will be drawn in the next section. stage two initial development the initial development stage concentrated on establishing the social and technical infrastructure for the project and on setting in train the application design work. the former involved issuing a call for tenders for a variety of hardware and software contracts, evaluating the tenders, managing the contracts as they were let, building up a central mainframe facility, designing and installing a continent-wide network, putting together the staff for a project organisation, and designing the application system. the cabinet proposal had indicated what resource funding would be needed principally, money for the technology, staff, and accommodation. because of public service september 1993 67 structures the project organisation was dependent on a number of other departments and agencies such as the australian government supply and tender board for management of the formalities of the tender process, the national capital development commission (ncdc) for provision of accommodation, the department of housing for site works, and the department of services and property for fitting out and maintaining accommodation. consequently, the project organisation needed them to fix a number of tasks which were outside its own control. as the stage progressed and it became apparent that not all the support needed was forthcoming, there became a requirement for power-broking. the support management strategy was essentially more of the same from the previous stage. it relied on the cabinet decision and assumed that that decision implied that support would be forthcoming for all the other needs outlined in the proposal on which it was based. when it became apparent that more support was needed than was provided the main target was the board itself as both funder and as power-broker. the approach to managing its support was structural inasmuch as it mostly followed formal channels of communication with the board through the head of the adp division. one development in the management of secondary support during this stage was the establishment of a consultative committee of middle managers from the user departments. this had the potential to build support among staff at levels lower than permanent head by encouraging commitment, and promoting legitimacy. the support actually provided was variable. in some cases, such as the tender process, support appears to have been satisfactory. but, in crucial ways it was less than was needed. the project organisation lacked a champion with both access to and influence over the board. the head of adp was slow to open the gate he kept. the public service board and other departments were strongly affected by stringency measures introduced by successive governments in response to difficult economic circumstances nationally. for the whole period the staffing level was well below that desired because of restrictions on establishing and filling new positions. in the financial year of 1975-76, the branch responsible for the system's technical development had a monthly average staff of 32 as against 74 needed according to earlier estimates. in another instance, the ncdc stopped site planning for the central computer because of funding restrictions. in this case, the project was able to summon up sufficient support from the board and, via it, ministers to overcome the problem, but the result was a facility considerably less than originally specified. the effects of the limitations to support for this stage were various. the support that was provided was sufficient for some progress to be made but too little to avoid undesirable outcomes. the shortage of staff when combined with contractual imperatives to service the establishment of the technical infrastructure meant that the application design received a lower priority. as a result this aspect of the system development was seriously delayed by may 1976 the board was being told that pursuing the then current implementation strategy without a significant injection of resources no implementation date could be promised. project organisation morale, and hence internal support in the form of technical work and commitment, was affected and the project became vulnerable to a burst of external criticism from the auditor-general (auditor-general 1976) and the press. such criticism became a chronic problem over the following four years and provoked further external scrutiny. the problem with the computer accommodation meant that the computer equipment took longer than expected to accept and suffered continuing performance problems at least in part because of its lack of adequate accommodation. a particularly interesting outcome for the project was that the build up of all these problems for the project created a political crisis for the board. faced with external criticism and no clear implementation date, it had to make a decisive response. it either had to cut its support for the project altogether and abandon it, or it had to revive it with an injection of support. it called in a reviewer who proposed a strategy which would achieve early implementation of some application functions. this strategy was to employ on behalf of a number of client departments the department of productivity's existing personnel and establishments system. initially, the system would be run on spare machine capacity in several other departments' installations. over time, the system would be converted and ported to the mandata mainframe. after that the original mandata system would be implemented. the strategy was agreed, and the reviewer was appointed project director at a level more senior than that of the project's other managers, a position which gave him direct access to the chairman of the board, and concomitantly greater board support. so an inadequate level of support ultimately created the conditions by which more support would be generated. but this was only possible because public criticism made abandonment potentially embarrassing and because an acceptable alternative implementation strategy was found. the cost to the project was that the new 68 strategy introduced significant new problems because the system adopted was flawed and because porting it to the mandata computer proved significantly harder technically than was anticipated. in terms of explaining the project's history an understanding of the problems of support during the initial development stage show how the project reached that critical point at which the board had to take some radical decision about the project's future. the board's decision in the face of crisis to increase its support also helps to explain the continuing momentum the project maintained over the next two stages in the following three years. the board had now made a very visible commitment and would need to justify its investment. for the researcher, the problem of explaining why the board decided to continue its support remains outstanding. the best the current framework can offer, deriving from its account of the basis of support, is the suggestion that the board stood to benefit more from the project's continuation than from its termination. this benefit might have been in terms of embarrassment avoided or political damage from loss of face or a variety of other possible considerations. it might have been subject to a form of escalating commitment (staw & ross 1990). what is clear is that research is needed to refine the concepts which define the bases of support so as to better explain such decisions. a second problem for the researcher is that the strategies and practices for managing support outlined earlier in the paper do not address the problem encountered by the mandata project organisation of the withholding of support such as staff resources. how can it be avoided and how should it be dealt with when it does occur? the strategy of decoupling the system from its supporters will only help if alternative support is available. this issue warrants considerable further attention. for the practitioner, several lessons emerge. two relate back to the strategic decision taken by cabinet at the end of the previous stage. first, expressions of support cannot necessarily be relied on to convert to substantive provision of that support. second, one of the reasons for this is that there is often ambiguity in the interpretation of what is promised. in this case the cabinet only explicitly supported expenditure on equipment. the project organisation drew the inference that the staff and accommodation and other costs included in the proposal were thereby also sanctioned. however, this left potential supporters such as the ncdc room for manoeuvre when economic restrictions were imposed. they had neither promised support nor had the cabinet explicitly instructed that they should give it. a further related lesson is that support is contingent in that had economic circumstances not changed it is quite likely that the staffing and accommodation resources would have been readily provided. two further lessons concern the project organisation's ability to manage support. in automating personnel and establishments record keeping mandata did not constitute a strategic application for any major stakeholder. it lacked leverage. it also lacked ready access to its most obvious source of support, the board, because its structural route was through a gatekeeper and because it had no champion. either of these conditions is likely to be enough to prevent a project organisation maintaining its support in adverse circumstances. the case of the board's intervention to secure accommodation for the central mainframe when it was endangered by the ncdc's decision to stop site planning is also instructive. that the project organisation did not get what it really wanted serves to indicate that even where reactive support is provided, it may not be sufficient to retrieve the situation fully. the outcome was consequential problems with the both the equipment and its accommodation. finally, a practitioner faced with difficulties in managing support, in particular in obtaining promised or expected support, would do well to dwell on the mandata experience. each effort to obtain the desired support is costly to the project organisation. it is impossible to know and very hard to estimate what the outcome will be. will this effort be successful and yield all the desired support or will it merely lead to another effort because it only yields some proportion of what is needed? this uncertainty can be both debilitating and inhibiting to the formulation of some more rational project strategy based on the limited support actually available. stage three reorientation the reorientation stage instituted a major change in project strategy in order to achieve earlier implementation. for this it needed considerable support. all forms of primary support were needed. it needed funding to get the project organisation to a size where it could implement, operate and maintain a large application system for a number of departments at once. in adopting its new strategy the project placed itself in the hands of the technical staff of the departments whose system it was implementing and whose computers it was using. being knowledgeable about the system the september 1993 69 department of productivity's technical staff would be crucial fixers of applications problems. naturally, internal technical work was required to get this strategy off the ground and to subsequently convert the system to run on the mandata mainframe. with the new strategy emphasising implementation it also became important that users actually use the system. also all forms of secondary support were needed. power-broking was necessary to help manage the critical external environment. the promotion of social legitimacy was required to combat criticism. and commitment was needed because the new strategy, having been formulated quickly, was a calculated risk and inevitably would require continuing support to help solve unimagined difficulties. the new implementation strategy was in part a support management strategy as well. it targetted three critical groups the project organisation itself, the board, and users. by bringing forward implementation of an application, it disarmed the concerns of some staff internal to the project and enhanced their morale. it gave the board a response to the critics and a chance to avoid further damage to its prestige. it aimed to win the support of users by involving them further, most obviously by providing them with a service which would then serve as leverage for future support. the project was mostly well supported during this stage. the new director was positioned to command more board support in the form of funding for more staff and power-broking to acquire new office accommodation. the strategy itself served to win back support from disaffected members of the project organisation. users were mobilised to contribute resources to facilitate the implementation and so increase the probability of active use. at the same time, though, the project continued to be under resourced. it requested loans of computer systems officers from departments, but all refused. the departments providing technical support for the operational system generally contributed what was needed but their willingness to do so without compensation declined over time. mandata's public stock rose with the new momentum although not sufficiently to avoid some continuing criticism. the support provided was only slightly less than was needed. in consequence the new strategy got under way with few hindrances. this met many of the political difficulties facing the board. the cost lay in the risk associated with the new strategy. the lessons of this stage relate mostly to the possibility of the transformation of a troubled project given sufficient support. this is not a matter which receives much attention in the traditional systems development literature. in view of the number of projects that run into trouble it would seem that the support perspective has something distinctive to offer. in this case it took a political crisis to create the conditions in which such a transformation could occur. how necessary this is is unclear although there is some evidence in other literatures that a crisis is an important precondition of radical change (pettigrew 1985 child, ganter & kieser 1987). although the point has not really emerged in the above analysis, another lesson, if more minor, is that the improved image of the mandata project indicates that some problems and criticisms can be dispersed by a radical shift in strategy. by eradicating the original basis for criticism, the critic is disarmed. the other major lesson is that although a radical shift in strategy can be beneficial in terms of its immediate effect on support it needs to be recognised that it is unlikely that the new strategy will be as well thought out and its implications as well understood as if it had been the strategy followed since the project's inception. when the decision to reorient is taken under political pressure this danger will be acute. finally, a lesson which is apparent in the next stage, too, is that structural positioning is important in managing support. the improved access of the new director undoubtedly benefited the project in assuring improved support from the top. and this in turn benefited legitimacy with consequential effects for commitment. stage four consolidation where the third stage introduced a major change of direction, the consolidation stage aimed at bringing the new implementation strategy to fruition. this involved implementation of the interim system in many user departments and development of a new system along the lines originally envisaged. the management of this stage was undertaken by a new project director. initially, the main needs for support were funding and technical work to maintain project momentum; fixing by application staff in departments to get their personnel and establishments records into a condition from which they could be computerised as well as continued fixing by the departments providing system support; and, when implementation was achieved in a department, the system needed to be used. secondary support was also necessary to promote the provision of the primary 70 ajis support legitimacy to combat criticisms from external sources and departmental users, commitment to generate other forms of support, and power-broking to facilitate its provision where not immediately available. a high level of anticipatory support was needed as the uncertainty arising from the implementation strategy and contextual contingencies was high. as the stage progressed, new problems emerged requiring previously unanticipated support. the support management strategy continued to target key supporters but also included a greater element of the sprayshot approach. the new director was known for maintaining a wide network of contacts throughout the public service. at the same level as his predecessor, he was able to maintain good links with the board through regular formal and informal reporting sessions. he also emphasised the continuing involvement of users at the management and supervisory levels in two committees which met regularly. when faced with increased external scrutiny from the auditorgeneral and then the public accounts committee, he called upon supporters to defend the project. the support provided by the board, the project's own staff, and by user departments was sufficient to sustain the technical effort involved in implementing the system in twelve departments and to the new development of several sub-systems. conversion to the mandata computer ran only a few months late. but, there was not enough slack to allow the project organisation to compensate for the flaws which persisted in both the converted system and in some newly developed sub-systems. these same flaws caused user departments to continue to run clerical systems in parallel thereby undermining much of the legitimacy their use of the system promoted. under critical attack from the auditor-general (auditor-general 1976, 1977, 1978, 1979, 1980), the public accounts committee (jcpa 1979), the press, and the staff associations, the board gave considerable support in an effort to counter the damage the criticisms caused to legitimacy and commitment. however, even this support proved sufficient only to effect a rearguard action and to moderate the worst of the criticisms. overall, then, the support given to the project organisation was sufficient to sustain continuing development, operation and maintenance, but it was not sufficient to permit eradication of all flaws and shortcomings nor to generate the supportive context necessary to the development of the original system. the outcome was a situation in which a seriously flawed system was operational but for which there continued to be a clerical alternative. departments were, at most, only partially dependent on the system. at the same time the project's legitimacy had been seriously damaged by the auditorgeneral's annual critique, by a severely critical report by the public accounts committee and by sensational press coverage. the project organisation's ability to leverage support for further development to replace the flawed system was therefore seriously impaired despite the fact that the public accounts committee ultimately supported continued development of the project. the final recognition that key flaws could not be rectified without redeveloping the core system and that there was not support for that redevelopment forced upon the project organisation a scaling back of ambitions while at the same time leaving it wide open to termination as part of governmental cost cutting. analysis of the support management strategy and the adequacy of the actual support received helps us to see how mandata was able to continue as a functional project into the late 1970s but at the same time how its prospect of surviving long into the 1980s was rendered poor. the crucial contrast here is between the support needed and available to pursue a continuing implementation strategy and that needed and available to rectify flaws in the strategy. the project organisation was unable to build up a level of support sufficient for future needs. no doubt part of the reason for this was its inability to satisfactorily manage perceptions of its performance. the critics were too vocal. thus one way in which the conceptual framework might be usefully extended would be to develop an account of the factors which influence what support can be built up. how important is performance? how important are criticisms? then, the research might be extended to consider practices which help combat negative influences on support. finally, the chief lesson of the consolidation stage for the practising manager is that however well support may be managed in the present, negative perceptions of performance can undermine the support needed for future efforts particularly if they involve any shift in type of activity. where performance is to be evaluated in terms of the quality of an implementation which has not yet been completed it will not be known until that implementation is complete how support will be affected. in other words implementation may cause a significant reduction in the support from a currently satisfactory level to one which constrains future options. a subsidiary moral is that criticism is important. it can be seriously detrimental even if there is considerable support to counteract it. september 1993 71 stage five scale back and termination in the first half of 1980 the project organisation, the auditor-general's staff and the departmental users independently of each other reached the conclusion that the existing core system was inadequate as a basis for future extension. there was very little support for developing a replacement even within the project. a consensus emerged that the project's ambitions should be scaled back to whatever minimum was necessary to achieve operational viability. in the short term, the project continued to need much the same level of resources because there continued to be a large amount of work. the project continued to rely upon users to help it fix flaws in the system. public criticisms and their fall-out continued to command attention and responses. so considerable support was still needed despite the decision to scale back. the strategy for maintaining support, though still involving attention to the board and users, now centred on satisfying them through providing a valued service. this continued to yield the level of resources needed for the new scaled back project. moreover, users were now sufficiently supportive that they could be coordinated into a joint rebuttal of the auditor-general's most stinging criticisms in his april 1980 report (auditor-general 1980). whether or not as a result of this or as a result of continuing lobbying by the board, a few months later the auditor-general agreed to take the pressure off the project. so, once again support was adequate for the project's continuing existence. crucially, though, in finally abandoning the original project objectives the project organisation could no longer offer future benefits that would justify the total investment that had been made in the project. in this it forfeited the basis for its funding. thus, when funding became threatened by a governmental cost cutting exercise it had no adequate ground on which to defend itself. indeed, for its chief sponsor, the board, termination through the need for cuts was probably welcome. it gave a face saving excuse for relieving the board of a public embarrassment. in an earlier section it was concluded that the conceptual framework exemplified in this paper might be usefully enhanced by an elaboration of its account of the bases of support. the suggestion that the board might have supported the project because it could see no politically acceptable way out underlines this. support need not be based solely on benefits, structure and culture, but also may be based on the avoidance of the disadvantages incurred by not giving support. the experience of the termination of mandata also highlights the contingency of support. had economic conditions been less straitened the cost cutting exercise might never have happened and the strategy of bedding down a minimum viable system might have been successful. clearly, context is an important source of influence on support. analysis shows that the project was terminated because it no longer had a basis for justifying its funding. the obvious lesson for practitioners is that the loss of a financial rationale may render funding vulnerable although the decoupling strategy for managing support seeks to avoid this. the other lesson is that partial abandonment (as in the decision to scale back mandata) is a realistic option when a project is in trouble (see sauer 1993b). in this case, it did not manage to assure the long term survival of the project. however, it did permit its continuance long enough for a face saving resolution to emerge. this is not a negligible consideration. for members of a project organisation concerned about the effects on their careers as a result of association with a failed project or for supporters who have had long association with the project such an outcome may be highly desirable. conclusion testing the conceptual framework on a case study was advertised as having three benefits it would demonstrate the framework's value in offering explanatory insights; it would highlight ways in which the framework could be improved; and it would offer useful lessons for the practising information systems project manager. at the end of each section analysing a stage of the case study conclusions have been drawn. it is now appropriate to consider what more general conclusions can be reached. first, the framework has demonstrated that it can help explain the ups and downs of the mandata project, in particular why it was able to continue when it encountered difficulties and why ultimately it could not survive in its scaled down form. indeed, in view of the fact that the project and system were in better shape in 1981 than at the time when it was decided to scale back, it is only through analysing the project's support that its termination can be understood. this is not to say that 72 ajis analysing support explains all facets of a project's history. the technical development of the innovation is still important but, the support analysis perspective sheds light where otherwise matters might remain obscure. second, the adequacy of the framework has been tested. all aspects of it have been used. one facet which has not received much attention but which could usefully be explored is the linkages among different forms of support. for example, visible primary support is likely to have a positive effect on perceived legitimacy. perceived legitimacy is likely to have a positive effect on other forms of support. however, perceptions of supporters could influence these effects. stakeholder x might have a negative opinion of y and view y's support of a project as a reason for not supporting it. so, to understand the likely influence of support we need a more sophisticated understanding of how forms of support interact and how they may be modified. this leads to further consideration of the bases of support. a better account of why supporters support a project is needed. as noted earlier, it is more complicated than a rational exchange. in addition to structural and cultural bases we have discovered that support may be given because the alternative looks less attractive. the contingency of support has been noted. even where a potential supporter sees benefits from giving support there may be external factors which discourage or prevent this from being converted into something usable. the framework has nothing to say about contextual influences on support this needs to be rectified. another aspect that receives no attention in the framework is the matter of the balance of different forms of support. not only does the level of support vary over time, the levels of the forms of support may differ at one time. it may therefore be useful to think in terms of a profile of support to do this, and to make practical use of the matching of needs for support and available support it will be necessary to attempt some kind of operationalisation of levels of support. third, various lessons have been drawn for the practitioner. the central lesson is that support is crucial to projects. it is necessary in different amounts of its various forms at every stage. it is a major reason that projects are able to continue even under adverse conditions. as the reorientation of mandata demonstrates, it can be used to radically change the direction of a project, perhaps saving it from termination and loss of all potential benefits. the moral for the practitioner is that it is important to attempt to manage support in order to smooth a project's path and enhance its prospects of continuing when faced with adverse conditions. to manage only the technical development of a system and to leave support to chance is to manage only half of an information systems project. at present, then, although the conceptual framework presented and tested here has been shown to be valuable in a number of ways, it has also been shown to be limited. future empirical research on the ways in which support works in projects and on the ways by which it is managed will contribute to a more sophisticated conceptual framework. an improved framework will provide a basis for a theory of support from which soundly based advice can be formulated. until then researchers and practitioners have at least a partial framework by which to think about support, and that is a significant advance. references ackoff, r.l. (1960) unsuccessful case studies and why, operations research, 8,4, march-april, 259-263. argyris, c. (1971) management information systems: the challenge to rationality and emotionality, management science, 17,6, feb, b-275-292. auditor-general (1976) report of the auditor-general for the year ended 30 june 1976, agps, canberra. auditor-general (1977) report of the auditor-general for the year ended 30 june 1977, agps, canberra. auditor-general (1978) report of the auditor-general for the year ended 30 june 1978, agps, canberra. auditor-general (1979) report of the auditor-general for the year ended 30 june 1979, agps, canberra. auditor-general (1980) report of the auditor-general, agps, canberra, april 1980. september 1993 73 child, j., ganter, h-d. & kieser, a. (1987) technological innovation and organizational conservatism, in pennings, j.m. & buitendam, a. (eds), new technology as organizational innovation: the development and diffusion of microelectronics, ballinger, cambridge, ma.. ein-dor, p. & segev, e. (1981) a paradigm for management information systems, praeger, new york. ginzberg, mj. (1975) a process approach to management science implementation, phd dissertation, mit. jarvenpaa, s.l. & ives, b. (1991) executive involvement and participation in the management of information technology, mis quarterly, june, 205-227. jcpa (1979) 775th report of the joint committee of public accounts of the parliament of the commonwealth of australia: use of adp in the commonwealth public sector the mandata project, australian government publishing service, canberra, september. keen, p.g.w. & scott morton, m.s. (1978) decision support systems: an organizational perspective, addison-wesley, philippines. kraemer, k.l., king, j.l., dunkle, d.e. & lane, j.p. (1989) managing information systems: change and control in organizational computing, jossey-bass, san francisco. lucas, h.c. jr, ginzberg, m.j. & schultz, r.l. (1990), information systems implementation: testing a structural model, ablex publishing corp, norwood, new jersey. lyytinen, k. & hirschheim, r. (1987) information systems failures: a survey and classification of the empirical literature, oxford surveys in information technology, vol 4,257-309. morgan, h.l. & soden, j.v. (1973) understanding mis failures, data base, 5, winter. pettigrew, a.m. (1985) examining change in the long-term context of culture and politics, in pennings, j.m. and associates, organizational strategy and change, jossey-bass, san francisco, 269-318. sauer, c. (1992) information systems failure: the problem of managing support for a rawed innovation process, phd thesis, university of western australia, perth, wa. sauer, c. (1993a) support and support management in information systems projects: building a conceptual framework, mimeo. sauer, c. (1993b) partial abandonment as a strategy for avoiding failure, in avison d.e., degross, j. & kendall, j. (eds) information systems development: human, social and organizational aspects, proceedings of ifip wg8.2 working conference, noordwijkerhout, 17-19 may 1993, elsevier (north holland). staw, b.m., & ross, j. (1990) behavior in escalation situations: antecedents, prototypes, and solutions, in staw and cummings, l.l. (eds) the evolution and adaptation of organizations, jai press, greenwich , conn, 1990,191-230. walton, r.e., (1989) up and running: integrating information technology and the organization, harvard business school press, boston, massachusetts. 74 ajis ajis vol. 4 no. 2 may 1997 structure and extensions of the user information satisfaction construct27 clived. wrigley faculty of business simon fraser university email: cwrigley@sfu.ca don h. drury faculty of management mcgill university email: drury@management.mcgill.ca ali f. farhoomand school of business the university of hong kong pokfulam road, hong kong email: alii @hkucc..hku.hk abstract user information satisfaction (uis) remains one of the most important constructs in information systems research. this paper investigates the application of the uis instrument across a number of key variables including respondent characteristics. several findings emerge from a survey of 379 is and non-is managers: first, the uis factors are stable and generalizable. second, the level of satisfaction varies quite substantially when compared to prior uis reports, although the is product satisfaction dominates. third, is managers, not surprisingly, evaluate their systems significantly higher than non-is managers, indicating the importance of stakeholders in evaluating information systems success. finally, respondents evaluate their internal systems differently than their inter-organizational systems, indicating that in the is evaluation process system type must be considered as a moderating variable. introduction the user information satisfaction construct has pervaded the information systems literature for over ten years. delone and mclean (1992) identify 33 separate empirical studies, conducted between 1981 and 1987 that measured some aspect of the information system (is) success construct. in most cases user satisfaction was measured during the study. its use, primarily as a surrogate for is success, stems from the recognition that one of the primary stakeholders in organizations are the users themselves. with this view, user satisfaction indicates that systems' objectives are being met, thereby allowing managers to differentiate between more or less successful systems. the logical appeal to determining system success comes from two primary sources. the first comes from the belief that successful is products lead to improved organizational performance. while the second comes from the desire to improve the process by which systems come into existence. recently, the software engineering literature has placed greater research focus on the system development process in an attempt to understand the factors that lead to well built maintainable systems as well as managing the resulting process (rombach, ulery, & valett d 1992, madhavji & schafer 1991). these, and other researchers, maintain that the process of developing, implementing and supporting is in organizations is as important as the resulting product of development. thus we see growing recognition for the need to integrate process and product measurement and evaluation (tate, verner & jeffrey 1992, chroust, goldmann & gschwandtner, 1990, pfleeger & mcgowan 1990, lanphar 1990). by retrospectively looking at the development process we may begin to determine, for example, which tools and methods were more effective in coping with the enormous task of implementing is. establishing a linkage between measures of is product and process, and subsequent organizational performance, while appealing, has been empirically elusive. the much more tractable task lies in the measurement of uis. this paper presents the results of a questionnaire survey, which replicated the uis instrument in its entirety with added scales to measure respondent demographics and respondent evaluations of their interorganizational and intra-organizational systems. the data from this study provides the opportunity to i) compare the results to previous uis reports and ii) investigate the role of system type and user type in the evaluation of is success. in the next section we will focus on the properties and application of the uis instrument in order to develop our hypotheses. following this we will provide background of our survey characteristics and properties. next we will present our analysis with comparative studies and structural refinements. the final section will summarize our results and discuss their implications. the support of the edi council of canada (edicc) and the canadian information processing society (cips) in providing their membership lists is greatly appreciated. 114 ajis vol. 4 no. 2 may 1997 conceptual development uis instruments have undergone extensive revision since their inception. gallagher (1974) developed an instrument to ascertain the value of information output, focusing solely on the is product. the bailey and pearson (1983) instrument contained 39 scales which were factor analyzed into four or five factors. ives and olson (1983) further reduced bailey and pearson's questionnaire into 13 scales which resulted in three factors. this version of the instrument has been replicated in at least five independent studies (joshi 1990, galletta & lederer 1989, baroudi & wanda 1988, montazemi 1988, raymond 1987). these three factors have been labelled: 1. information product 2. mis/edp staff and services 3. user knowledge and involvement recent work by a number of researchers has further clarified the relationship between the independent, moderating and dependent variables. two important variables to emerge from this literature are: 1) is type and 2) user type, as having contingent effects on user satisfaction. measurement of uis should, a priori, be different depending upon the nature of the system with which users are being satisfied, and the tasks that the users perform (goodhue 1988). accordingly, a number of specific instruments or modifications to existing instruments have been developed and used. the uis instrument was developed during the period in which mainframe computing was the dominant platform of is. all scales pertaining to is product deal exclusively with output data, specifically: reliability, relevance, accuracy, precision and completeness. the remainder of the questions take the view that users are separate from the builders and that therefore the relationship between the two contains the source of satisfaction or dissatisfaction. more recently, because of developments in information technology, researchers have focused on the evaluation of specific types of information systems, introducing measures for decision support, ease of use, learnability etc. (cats-baril & huber 1987, barki & huff 1985, sanders & courtney 1985, benbasat & dexter 1982). it is important to note that these studies have focused on specific properties of the respective type of systems and users. for example, the specific properties making up the success of dss are related to the improvement of decision quality. similarly studies related to success of end user computing (euc) have been based on the basic premise that since end-users assume more responsibility for their own applications, their satisfaction should be measured through certain scales applicable to conventional computing environments, but also certain scales specific to euc environment (doll & torkzadeh 1988, rivard & huff 1988). as seen in figure 1 systems can be classified according to a number of criteria. one broad distinction is based on the relationship that systems have with their organizational environment, specifically, either having an intra or inter-organizational focus. from an ontological perspective, it is reasonable to assume that different kinds of systems will have different stakeholders, different properties, and different criteria for evaluation. we would expect, therefore, that different kinds of systems within the same organization may be viewed differently by the same person. recent advances in national and international standards have promoted wider adoption of electronic data interchange (edi) systems. the orientation of class of system is external electronic linkage with trading partners [4, 14]. edi is recognized as a natural extension of interorganizational relations because it facilitates exchange of information and resources among the participating firms. implementation and use of these systems are likely to lead to different perception and subsequent evaluation among users. with this view, it is possible to consider two broad classes of systems: those that are built having inter-organizational linkages as their primary focus, and those that have an infra-organizational focus. more specifically, we maintain that information systems can be classified in a hierarchy, composed of at least three components: a) properties shared by all is, b) properties shared only by internal or external systems, but not both and c) properties unique to specific systems. the top level of the model covers the basic properties of all information systems. the intermediate level focuses on the unique properties that differentiate internal systems from the external ones. finally, the bottom level relates to properties of specific information systems. the framework in figure 1 highlights the role of systems classification in the is success assessment process. at the environment level, the parameter differentiating internal systems from external ones is related to the specificity of the internal environment as constrained by the internal operations, management and decision-making functions of the organization. at a specific system level, the success of a system is influenced by many specific factors unique to the system's functionality, economic goals and its stakeholders. this leads to the following general hypothesis: 115 ajis vol. 4 no. 2 may 1997 hi: managers evaluate the success of their internal systems differently than their external systems. in addition to system type individual differences play an important role in the systems evaluation process (zmud 1979). users exist at all levels in organizations and interact with is in a variety of ways. the central notion is that of fit between the technology and the job to be performed, whether it be decision making, data entry or other. of interest in this study is the effect of respondent profile on the reported level of satisfaction. specifically, the degree to which uis scores vary across respondent groups reduces the ability to compare uis surveys and hence stability and generality of the instrument. this leads to the following hypothesis: h2: is managers evaluate their is differently than other functional area managers. being directly involved in development and management of systems, we would expect to see is managers evaluating their systems higher than others. for our purposes it is sufficient to demonstrate the stability, or instability, of uis measures among respondent groups. figure 1. a classification model of information systems generic level all information systems environment level specific system level system a 116 ajis vol. 4 no. 2 may 1997 survey to investigate the research questions, which we have stated as hypotheses, a survey instrument was constructed comprising demographic questions, the standard 13 scale uis questionnaire, and six additional questions. these six questions asked respondents to differentially evaluate their internal systems and their edi program. respondents were provided with the following definition of edi at the beginning of the questionnaire: edi is defined as the corporate to corporate exchange of business documents in a structured format. edi is not electronic transmission of data in a free form. therefore, it excludes facsimile (fax) transmission, which requires rekeying of data by the receiving party, and electronic mail (e-mail), which requires rekeying or editing of data. however, it includes tape exchange of business documents in an edi related format. respondents were requested to rate their internal systems and their edi program according to three criteria each, namely: 1. overall degree of success 2. the overall extent to which objectives are met 3. overall satisfaction. the complete instrument was designed, therefore, to measure managers' subjective evaluations of 1) all their computer based information systems (uis instrument), 2) their internal systems (three questions), and b) their edi systems (three questions). the membership lists of the edi council of canada and the canadian information processing society (cips) were used in the survey phase of the study. this questionnaire was mailed to 875 members of these two organizations. by the deadline, 208 usable questionnaires had been returned up to that date. in order to gain further insight into non-response bias, another wave of surveys was mailed out to 460 companies. the new mailing list was based on the one used in the final survey, but excluded the names of a) those who had responded to the survey and asked for the results (235), b) those on the undelivered envelopes (33), c) those who had not filled out the questionnaires for various reasons but returned it (15), and d) those who had indicated in the telephone follow-up that they were not interested in the survey (25). 78 questionnaires were received after the first deadline from the original mailing. by the second deadline, an additional 94 usable questionnaires were received, resulting on an overall usable response rate of 43% (379/875). the questionnaires were divided into three groups, or waves, according to their dates of arrival. the first group contained all questionnaires that had been received before the deadline of the first mailing (208). the second group consisted of questionnaires received after that date (78). the last group contained the second mailing's surveys (94). these three groups were cross tabulated with several demographics questions. the wave procedure assumes that late respondents are close in characteristics to nonrespondents. evidence of no differences between the waves provides support for the assumption of group similarity, thereby allowing pooling of responses. in this survey, no evidence of non-response bias was found based on the firm's size, stage of edi adoption, or respondents' functional area. none of the chisquare tests of independence could be rejected. the demographic information of the sample are shown in tables 1-4. table 1: distribution of firm size annual sales (millions) frequency percent <$5 $5 $9 $10 $24 $25 $49 $50 $99 $100 -$249 $250 $999 $1000 or > 30 13 30 32 28 39 60 107 3.8 8.8 9.4 8.3 11.5 17.7 31.6 8.8 total 339 100.0 117 ajis vol. 4 no. 2 may 1997 table 2: industry classification chemicals communications financial food manufacturing government insurance metals, mach. & equip. mining oil & gas pharmaceutical & health printing & publishing pulp & paper retail stores textile & apparel transportation utilities wholesale trade other freq. 18 16 30 27 27 14 15 11 17 14 9 16 4 23 6 23 109 cum. percent freq. 4.7 18 4.2 34 7.9 64 7.1 91 7.1 118 3.7 132 4.0 147 2.9 158 4.5 175 3.7 189 2.4 198 4.2 214 1.1 218 6.1 241 1.6 247 6.1 270 28.8 379 cum. percent 4.7 9.0 16.9 24.0 31.1 34 38.8 41.7 46.2 49.9 52.2 56.5 57.5 63.6 65.2 71.2 100.0 table 3: stage of edi adoption no edi program underway edi feasibility study technical specifications legal & audit requirements pilot program edi in operation mode finance information systems production/manufacturing purchasing sales/marketing transportation/logistics other freq. i l l 45 12 3 46 162 table 4: freq. 44 205 5 15 46 16 48 cum. percent freq. 29.3 111 11.9 156 3.2 168 0.8 171 12.1 217 42.7 379 cum. percent 29.3 41.2 44.3 45.1 57.3 100.0 respondent functional area cum. percent freq. 11.6 44 54.1 249 1.3 254 4.0 269 12.1 315 4.2 331 12.7 379 cum. percent 11.6 65.7 67.0 71.0 83.1 87.3 100.0 results the uis literature over the years has repeatedly stressed the need for internal reliability of instruments. galletta and lederer (1989) present evidence that the test retest reliability of the uis instrument may be weak, although, hawk & raju (1991) challenge this evidence. one of the main concerns, however, is that the items used in the scales are internally consistent. in order to address the internal consistency issue, table 5 compares the scale reliability of this study with the baroudi & orlikowski (1988) study, and the 118 ajis vol. 4 no. 2 may 1997 ives et. al. (1983) study28. as can be seen in table 5, all the chronbach's coefficients of alpha are in the acceptable range (higher than 0.90). table 5: uis scale reliability comparison: chronbach's alpha uis scale 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. relationship with mis staff processing of requests for changes degree of mis training provided user's understanding of systems user's feeling of participation attitude of mis staff reliability of output information relevancy of output information accuracy of output information precision of output information communication with the mis staff time for new system development completeness of the output information sample size this study .95 .94 .95 .94 .91 .95 .95 .95 .97 .95 .93 .94 .97 330 baroudi et. al .92 .88 .93 .88 .89 .92 .91 .91 .89 .84 .88 .94 .93 358 ives et. al .94 .90 .97 .92 .92 .88 .95 .95 .95 .94 .90 .90 .96 100 we factor analyzed the 13 scales of the uis instrument in our sample data. for each of the 13 scales, the linear composite of the two corresponding items was calculated by averaging the 2 scores. the 3-factor pattern was produced using varimax rotation. table 6 presents this structure with the factor scores reported by baroudi and orlikowski (1988) in the adjacent column, surrounded by []. 28 .95. although raymond (1987) does not provide detailed scale reliability he reports that all alpha coefficients were between .81 and 119 ajis vol. 4 no. 2 may 1997 table 6; replicated three factor structure uis scale 7. 8. 9. 10. 13. 1. 2. 6. 11. 12. 3. 4. 5. reliability of output information relevancy of output information accuracy of output information precision of output information completeness of the output relationship with mis staff processing of change requests attitude of mis staff communication with the mis staff time for new system development degree of mis training provided user's understanding of systems user's feeling of participation eigen value variance explained by factor information product .66 [.82] .60 [.77] .80 [.82] .79 [.82] .63 [.75] .26 [.24] .24 [.15] .31 [.18] .35 [.21] .28 [.21] .19 [.23] .23 [.12] .27 [.37] 3.04 3.57 24% 28% mis staff & services .35 [.23] .35 [.00] .23 [.30] .20 [.28] .38 [.30] .77 [.84] .58 [.49] .62 [.82] .75 [.84] .56 [.49] .41 [.08] .19[.05] .56 [.37] 3.19 3.02 25% 28% knowledge & involvement .20 [.11] .27 [.11] .18 [.13] .16 [.14] .24 [.20] .17 [.09] .27 [.49] .18 [.15] .21 [.11] .35 [.40] .50 [.82] .71 [.80] .54 [.59] 1.58 2.20 12% 17% as can be seen the factor structures are strikingly similar, and there are small differences in the factor loadings. appearing below each factor column is the factor's eigen value and the proportion of total variance explained by each factor. for the 3-factor structure in this study the explained variance is .61, while the baroudi and orlikowski (1988) reported an explained variance of .68. an examination of the factor loadings and the relative eigen values indicates strong concordance between the two structures. specifically, using the cut-off criteria of .50, 37 of the 39 factor loadings match. the probability of this occurring by chance is less than .001. further, the fisher test applied to the equality of the factor variances could not be rejected at .001. we also compared the uis mean scores across studies. it should be noted that different researchers have used different scales in their studies. for example, raymond (1987) takes the average of two items reporting scores from 1 to 7, galletta & lederer (1989), report scores scaled from 2 to 14 by summing the two items, while baroudi & orlikowski (1988), re-scale the figures to fall between -3 and +3. this later scale is more visually appealing as the zero point may be used to differentiate between unsatisfied and satisfied, assuming item midpoints are meaningful. although all three scales are simple linear transformations, without the transformation, the job of comparison is much more abstruse. in this light we recalibrated previous mean satisfaction results to the -3 to +3 scale for comparison purposes. table 7 presents a comparison of five separate studies that used the uis 13 scale instrument29. several observations can be made about these five studies. first, the wide range of uis total satisfaction (-15.1 to +11.59) is reasonable considering that the questionnaire was administered to five different reference groups at different times. however, it is interesting that both low reports (galletta & lederer and raymond) were from mba students and small firms respectively. second, in the three studies that report detailed scales values, the lowest scores on all three are for the two questions relating to time: the time for 29 since neither baroudi & orlikowski (1988) or joshi (1989) report detailed scale values, these have been marked with an '*'. 120 ajis vol. 4 no. 2 may 1997 new system development and the processing of requests for changes. clearly, time expectation has a negative effect on user satisfaction. the third observation of note is that the information product factor dominates positively across all studies. perhaps a conclusion that can be drawn is that users are reasonably satisfied with the product once it is working, but arc less pleased with the process of building or maintaining them. all in all, these results provide further evidence in support of the generalizability of the uis instrument across different samples. table 7: uis mean satisfaction scores comparison this study galletta & lederer raymond baroudi & orlikowski joshi system product factor 7. 8. 9. 10. 13. reliability of output information relevance of output information accuracy of output information precision of output information completeness of the output factor total 1.29 1.27 1.72 1.37 1.03 1.34 -0.40 -0.45 0.05 -0.50 -1.00 -0.46 -1.03 -0.96 -0.86 -0.92 -1.31 -1.01 * * * * * 1.30 * * * * * 1.08 mis support & services factor 1. 2. 6. 11. 12. relationship with mis staff processing of requests for changes attitude of mis staff communication with the mis staff time for new system development factor total 1.22 0.16 1.15 1.15 -0.05 0.73 -0.45 -2.50 -.070 -.050 -2.45 -1.32 -0.92 -1.58 -0.91 -1.08 -1.67 -1.23 * * * * * 0.20 * * * * * 0.34 user involvement factor 3. 4. 5. degree of mis training provided user's understanding of systems user's feeling of participation sample size subjects 0.17 0.44 0.67 330 mangers -1.5 -1.3 -1.7 27 mba students -1.55 -1.25 -1.08 464 small manufac. * * * 358 clerical * * * 100 prod. man. in order to test the first research hypothesis, we used matched paired t-test of the respondents evaluation of their internal and edi systems, using the three variables related to overall system success, objectives, and satisfaction (table 8).30 we accounted for the firm size by dividing the sample into small and large firms. the larger number of missing values for small firms is due to the lower frequency of edi programs in small companies. 30 a positive difference indicates higher satisfaction with internal systems. 121 ajis vol. 4 no. 2 may 1997 table 8: evaluation differences firm size <$100m n 173 variable deff_suc diff_obj diff_sat n 57 59 59 nmiss 116 114 114 mean 0.49 0.58 0.25 std error 0.12 0.14 0.14 t 3.99 4.02 1.87 prob>|l| .000 .000 .066 firm size >$100m n 206 variable diff_suc diff_obj diff_sat n 142 138 141 nmiss 64 68 65 mean 0.18 0.63 0.26 std error 0.08 0.09 0.08 t 2.23 6.77 3.11 prob>|r| .027 .000 .002 the results clearly indicate differences between manager's evaluations of their systems based solely on system classification. one further observation is appropriate. in all cases the evaluation of internal systems is higher than for external systems. we may speculate that familiarity with existing systems may lead to more favourable responses, and that in time, any unfavourable evaluations of edi systems due to, for example, uncertainty, time delays etc., will ameliorate. this, however, goes beyond the current data and remains a topic for further investigation. to test the second hypothesis the sample was divided into two groups: is managers and all other managers. the results of this test appear in table 9. non-parametric tests were performed on all variables. not surprisingly, is managers report significantly higher uis scores than other managers, perhaps providing prima facie evidence for key respondent bias. several observations are worthwhile. first, ten of the thirteen uis scales show clear differences in levels of satisfaction; precision of output, degree of training, and user's understanding being the notable exceptions. second, is managers' evaluations dominate on all uis summary questions. third, the largest discrepancies come from the mis attitude and time for system development items, both being process measures. summary this paper has examined several structural aspects of the 13 scale uis instrument. comparison of these results with four previous studies indicates that users are relatively more satisfied with the products of system development than they are with the process of system development. within this sample of respondents, significant differences in level of satisfaction were detected when respondence functional area is taken into account. specifically, is managers were, not surprisingly, more satisfied with their systems than non-is managers. further, managers rated the success of their intra-organizational systems higher than their inter-organizational systems. we would like to point out some of the issues that subsequently need to be addressed. first, as is products have made steady improvement in quality, this dimension of satisfaction may be becoming increasingly related to satisfaction with the is development process. in this light we need to directly question the assumption of orthogonality of the three factors and use structural modelling techniques to test higher order structures underlying uis. in the process we need to understand that the uis factor structure is not in itself a final research result. rather, each of the underlying dimensions must be translated into management agendas for change if the objective is to improve satisfaction, rather than simply accepting it. 122 ajis vol. 4 no. 2 may 1997 table 9: respondent differences is managers (n=205) non-is managers (n=174) prot»|t| system product factor 7. 8. 9. 10. 13. reliability of output information relevancy of output information accuracy of output information precision of output information completeness of the output 1.47 1.41 1.83 1.45 1.26 1.07 1.10 1.59 1.28 .77 .010 .032 .055 .240 .001 mis support & services factor 1. 2. 6. 11. 12. relationship with mis staff processing of requests for changes attitude of mis staff communication with the mis staff time for new system development 1.37 0.42 1.40 1.29 0.26 1.05 -0.15 0.84 0.98 -0.44 .058 .004 .000 .047 .000 user involvement factor 3. 4. 5. degree of mis training provided user's understanding of systems user's feeling of participation 0.31 0.48 0.88 -0.01 0.38 0.42 .121 .621 .004 summary questions is product mis/edp staff and services user knowledge and involvement overall satisfaction uis total (-39 to +39) 1.23 1.25 1.43 1.14 13.75 0.83 0.56 0.64 0.54 8.93 .010 .000 .000 .001 .001 references bailey, j.e., & pearson, s.w. development of a tool for measuring and analyzing computer user satisfaction, management science, volume 29, number 5, may 1983, 530-545. barki, h., & huff, s. l. change, attitude to change, and decision support system success, information & management, 1985. pages 261-268. baroudi, j. j. & orlikowski w.j., a short-form measure of user information satisfaction: a psychometric evaluation and notes on use, journal of management information systems, vol. 4 no. 4, spring 1988, 44-59. 123 ajis vol. 4 no. 2 may 1997 barrett s. and konsynski b. inter-organization information sharing systems, mis quarterly, september 1982, 93-105. benbasat, i., and dexter, a. individual differences in the use of decision support aids, journal of accounting research, spring 1982, 1-11. cats-baril, w. l. and huber, g. decision support systems for hi-structured problems: an empirical study, decision sciences, summer 1987, 350-372. chroust, g.; goldmann h.; gschwandtner, o., the role of work management in application development, ibm systems journal, vol. 29, no. 2, february 1990, 189-208. delone, w. h. and mclean, e. m., information systems success: the quest for the dependent variable, information systems research, march 1992, 60-95. doll, w. j., and torkzadeh, g. the measurement of end-user computing satisfaction, mis quarterly, june 1988, 259-274. gallagher, c. perception of the value of a management information system, academy of management journal, 17, 1, 1974, 46-55. galletta, d. f. & lederer, a. l., some cautions on the measurement of user information satisfaction, decision sciences, vol. 20, 1989, 419-438. goodhue, d., i/s attitudes: toward theoretical and definitional clarity, database, fall/winter 1988, vol. 19 no. 3/4, 6-15. hawk, s. r., & raju n.s., test-retest reliability of user information satisfaction: a comment on galletta and lederer's paper, decision sciences, vol. 22, 1991, 1165-1170. horner r. b. & benbasat i. an empirical investigation of factors influencing the success of customer-oriented strategic systems, information systems research, september 1990, 325-347. ives b., olson m. & baroudi j. the measurement of user information satisfaction, communications of the acm, vol. 26, no. 10,1983, 785-793. joshi, k., an investigation of equity as a determinant of user information satisfaction, decision sciences, vol. 21,1990, 786-807. lanphar, r., quantitative process management in software engineering, a reconciliation between process and product views, journal of systems & software, vol. 12, no. 3, july 1990, 243-248. madhavji, n. h., schafer, w., prism methodology and process-oriented environment, ieee transactions on software engineering, vol. 17, no. 12, december 1991, 1270-1283. montazemi, a. r. factors affecting information satisfaction in the context of small business environment, mis quarterly, vol. 12, no. 2,1988, 239-258. pfleeger, shari lawrence, mcgowan, clement, software metrics in the process maturity framework, journal of systems & software, vol. 12, no. 3, july 1990, 255-261. raymond, l. validating and applying user satisfaction as measure of mis success in small business organizations, information and management, no. 12,1987, 173-179. rivard, s. and huff, s. factors of success of end user computing, com. acm, may 1988, 552-561. rombach, h. dieter, ulery, bradford t., valett, jon d., toward full life cycle control: adding maintenance measurement to the sel, journal of systems & software, vol. 18 no. 2, may 1992. sanders, l. g., and courtney, j. f. a field study of organizational factors influencing dss success, mis quarterly, march, 1985. pages 77-89. software engineering journal, special edition on software process and its support, edited by madhavji, n. h., vol. 6, no. 5, september 1991. tate, g.; verner, j.; jeffery, r., case: a testbed for modeling, measurement and management, communications of the acm, vol. 35, no. 4, april 1992, 65-72. zmud, r. individual differences and mis success: a review of the empirical literature, management science, vol. 10 no. 10., october 1979, 966-979. 124 ajis special edition knowledge management december 2001 theory and practice thinking styles in engineering and science hanspeter schmid bernafon ag, morgenstrasse 131, 3018 bern, switzerland. e-mail: h.p.schmid@ieee.org abstract this paper describes knowledge as an element of thinking styles, which are properties of thinking collectives. according to the theory outlined here, the choice of a thinking style to solve a certain problem is relative, but once the thinking has been chosen, realism prevails. this paper also describes the genesis and development of thinking styles and, with them, of facts. the theoretical concepts are illustrated with two examples of thinking styles: a description of the thinking styles of circuit theorists and circuit designers (theory vs. practice), and a comparison of the thinking styles of two closely related technical societies of the institute of electrical and electronics engineers (ieee). applications of the theory are also presented in this paper; they include information management, documentation tools, and writing styles, and mainly draw from the author's own experience with these topics. background much has been said about the relationship between theory and practice, and it has been said by people who are both technically more experienced and philosophically better educated than i am. nevertheless, i regularly thought and debated about the philosophical background of science for the better part of the time i spent at the signal and information processing laboratory of the swiss federal institute of technology, mainly for three reasons. one is that i soon noticed the incredible discrepancy between the ideal science producing facts and objective truths, as it is portrayed at high school and even in university lectures, and the chaotic mess of contradicting and inconsistent views i found at the front of research. then it also occurred to me that one often finds seriously flawed papers even in the most renowned of technical journals, so often in fact that i simply cannot believe that the time pressure on the referees is the only reason that such papers get published. the most important reason for my decision to write this paper was, however, my experience with writing my paper (schmid 2000). for three years, i discussed several versions of the paper with my doctoral father (this is the swiss name for the supervisor of a doctoral student). we argued for hours about the paper and its quality, but i never managed to write the paper in a way that he would agree to publish it. it will become apparent in this paper that this was not actually a personal argument, effectively we argued from two very different perspectives, the perspectives of a designer and a theorist. the same text that is clear, valuable, and true from my perspective must have looked unclear and trivial from his perspective. finally, he stopped our discussions by saying that he could still not agree with me, but that he trusted me with having found something, and he suggested that i publish the paper on my own. knowing about the ultimate importance of perspectives, i saw to it that the reviewers would use my perspective, not his. this could easily be done, because the theory-practice divide has been enforced a few years ago in my field when the most important publication of the ieee circuits and systems society, the "ieee transactions on circuits and systems," was split into two independent parts, "fundamental theory and applications" (tcas-i) and "analog and digital signal processing" (tcas-ii). so all i had to do was to explicitly declare my theoretical ideas as rationalised analog-ic-design experience and submit the paper to tcas-ii, where i knew that the associate editor who would get my paper would be an analog-ic designer as well. and the paper got accepted. the problems are still not quite over, though. it turned out that the title of the paper, "approximating the universal active element," was badly chosen. the term "universal active element" clearly comes from the theoretical realm, and one of the reviewers told me later that at first he thought "oh no, i don't have the time to look at this kind of paper!" in his review, he wrote "i am very enthusiastic about this paper ...." now he was a reviewer, meaning he really had to read the paper, but many people who would find it interesting will not read it because they already lose interest when they see the title. coming back to theory and practice: maybe i would never have seen the difference between the worlds of what i call theorists and designers, had it not been for a remarkable circumstance. during the years i worked at it, i showed the paper to many colleagues and found it very polarising: some liked it a lot, some saw not much of a point in it, but there was almost nobody in between these extremes. it was even more remarkable that the ones who liked it were the ones who had experience in 1c design, the ones who did not like it were the more theoretically inclined, and this in 106 aj1s special edition knowledge management december 2001 spite of the fact that the main contribution of the paper is of a theoretical nature. most remarkably, the designers suggested improvements in the theory, while the theorists asked for design criteria. this was when i started thinking about the difference between theorists and designers, and since i like writing and technical communication, it was inevitable that my findings would find an application in writing too. in this paper, i will provide very few citations in the text, because i cannot easily tell where i acquired all these ideas, but i will now mention the authors whose work had a major influence on mine and cite some of their work. i will also very briefly describe the main idea they conveyed to me. these descriptions are caricatures; to get the real picture, one should rather read the cited works. first to the people whose views 1 used in this paper. yehuda elkana: there is truth in both realism and relativism, but since every person routinely uses both at the same time, it is better to see the two as two sides of the same coin (elkana 1978, 1981). ludwik fleck: thinking is not done by individuals, but by groups, so intellectual individualism is mainly wishful thinking (fleck 1977). paul k. feyerabend: the only methodology that is common to all scientists is that they do what looks promising to them, so the only conclusion one can reach when one tries to describe science by its methods alone is that anything goes (feyerabend 1981, 1975). clifford geertz: lean descriptions are not suitable to describe cultures, one must give thick descriptions; reasoning rationally is not sufficient, one must also demonstrate things and tell stories (geertz 1973). arthur schopenhauer: using cunning tricks, one can win a debate no matter whether one is actually right or not (schopenhauer 1995, schmid 1999). william broad, federico di trocchio: it is rather easy to cheat in science without being found out for a very long time (broad & wade 1982, trocchio 1999). i also studied the "enemy," mainly through feyerabend's work, but also directly. karl popper: the scientific method is to make bold assumptions and then try to falsify them; verification is logically impossible (popper 1972). gunnar andersson: science can be understood in terms of formal logic (andersson 1988). what i did not know was that "distributed cognition" is actually a field of study that has a few things in common with the ideas presented in this paper, as one of the reviewers of this paper pointed out. not knowing that body of literature, i cannot provide good references, but would like to invite the readers to follow that link by themselves. last, but not least, i can only recommend the very cynical treatise "plan your fame" by daniel vischer (1992), in which he shows what you have to do to become famous and maybe respected in a scientific community, and how little it has in common with the public image of science. facts according to a wide-spread, old view of science, which can be caricatured by " 'a fact is a fact' is a fact," the truth of a theory or a statement does not depend on who looks at it. facts are there, just waiting to be discovered, and it is mainly a matter of chance who discovers them and when. this old view is still found in many popular science texts and text books used at university, either explicitly or between the lines. it is, however, a very inhuman view, since it will, in its extreme form, lead to the belief that somebody (often the person looking at the world) is right and everybody else is wrong. the view is also historically absurd, most scientific views that were believed to be true four hundred years ago have been replaced by now, and there is no reason to assume that any of today's theories will persist longer. the other extreme view, namely that there is no reality at all, that the truth of everything is always relative, is equally absurd. if this were the case, it would be impossible to assert even the most obvious of facts, like who ate the last cookie or whether it was the butler who murdered the duke, let alone anything more complicated, like how large the threshold voltage of a certain mos transistor is. both an extreme realist view and an extreme relativist view lead to absurd conclusions. nevertheless, both views contain some truth as well. the best solution to this dilemma is to assume that every single person uses both views. this mode of thinking was called two-tier thinking in (elkana 1978), and works as follows: once a person has selected a certain perspective from which to look at the world, realism prevails. facts are really facts from one perspective, and as long as the perspective does not change, the facts will not change either. the choice of a perspective, however, can only be made relativistically, since there is no absolute, external-to-all perspective from which all other perspectives can be judged and compared. note that a relativistic choice is not the same as an arbitrary choice. sometimes, a person may have the conscious choice between a very small number of possible perspectives, but in most cases the perspective is given to the per' son. but given by whom? 107 ajis special edition knowledge management december 2001 the thinking collective (denkkollektiv) we are living in a very individualistic time, and people (laymen as well as scientists working in the field of artificial intelligence) generally have adopted the main assumption of individualist psychology, namely that thinking is done by individuals. however, this view has at most bio-mechanical relevance; all thinking is deeply rooted in the social environment of a person, and because these social factors by far outweigh the individual ones, one may just as well say that thinking is done collectively. this collective thinking is something that most people experience quite often. when close friends chat, they say things and express ideas which they cannot say, or even think, when they are on their own. when they part, this "magic" is lost, but as soon as they meet again, it is back as if it had never been away. together, they are more than just the sum of the individuals. they think together, they are a thinking collective, and the "magic" is the thinking style they share (fleck 1977). what distinguishes one thinking style from another are words that have special meanings, ideas that only have a meaning to the people belonging to the same thinking collective, insider jokes, values, ideologies, and so on. and there is also something else: in a thinking style, some things can only be thought in a certain way, and some things are even unthinkable. so while people transcend themselves when they are in a thinking collective, they also become subject to thinking constraints. this becomes clearer when we have a look at larger thinking collectives. first, let us compare two large ones, namely the analog signal processing experts from the ieee solid-state circuit (ssc) society and from the ieee circuits and systems (cas) society. the comparison is oversimplified, mainly because the thinking style of the ssc society leaks into the cas society through the people who contribute to both societies. but the difference between the two societies is quite clear because their members maintain that difference on purpose. most articles appearing in ssc publications and at ssc conferences describe a working microchip. it is very difficult to get a paper published without having a working chip. exceptions are only tutorial or overview papers written by scientists who have already published several conventional ssc papers. on the other hand, the analog papers in cas publications that describe a working chip are in the minority. the reason for this difference is that the two societies have differing ideas of what knowledge actually is, they use different images of knowledge (elkana 1981). in both societies, knowledge can come from many different sources, e.g. experience, evidence of the senses, clear and distinct ideas, tradition, authority, novelty, beauty, intuition, analogy, and so on. the difference is that experience with 1c design is the most important source of knowledge in the ssc society, but only of minor importance in the cas society. in cas publications, even people with no design experience at all can publish new concepts for integrated circuits, provided that these concepts are intellectually pleasing, novel, or derived by analogy from cas tradition. this again is explicitly discouraged in ssc publications: "[the abstract] must be factual [and include] performance data. claims such as 'new,' 'advanced,' 'high-performance,' and 'high-speed' are not acceptable" (ieee 2000). the difference between the thinking style of ssc and cas scientists is a fundamental one, because it makes them see science itself differently. the two thinking styles cannot be reconciled and must not be reconciled, since having different views that exist so closely together is the key to scientific progress, as will become apparent later in this paper. let us now look at a smaller thinking collective, the group of scientists who deal with high-gain broadband amplifiers and feedback around such amplifiers. their thinking style has a much less fundamental trait, namely the tendency to express concepts in terms of two-port theory (c.f. moschytz & carlosena 1994, cabeza et al 1994, cabeza & carlosena 1997, payne & toumazou 1996, arbel 1996, choma 1995, wilson 1999). it became apparent in (schmid 2000) that this tendency used to be a thinking constraint that actually made it impossible for the collective to comprehensively classify the most wide-spread of operational amplifiers. thinking styles are very important because they actually determine what facts are. something is accepted as a fact by a thinking collective if and only if it fits into its thinking style. this is obvious in the ssc society: no chip-no paper. so if something becomes a fact only if it is in proper style, why was (schmid 2000) accepted for publication? one reason is that one person belongs to a vast number of thinking collectives, and the context decides which thinking styles are active at the time the person thinks about something. a change of perspective is sometimes easily possible, it can be caused by a perceived context change, or also deliberately or even playfully. in the case at hand, the reader is forced to shift his perspective (i.e., to abandon the two-port thinking style) by a comparison of two perspectives which shows that the one used up to that time is inadequate to tackle the problem. this is possible only because there is another perspective that fully embraces the two compared perspectives. being in the proper style is not only necessary if something should become a fact, but also sufficient. if a paper is in accord with the thinking styles a reviewer activates when he reads it, he will accept the paper. several papers published recently in the ieee transactions on circuits and systems describe new integrated circuits which simply do not work. for some reason, the reviewers did obviously not use an analog-ic designer's thinking style, in which it is 108 ajis special edition knowledge management december 2001 immediately apparent why the circuits do not work. however, the papers in question were in accord with several other thinking styles that are wide-spread in the cas society, so they were accepted anyway. alan d. sokal played with other people's thinking styles in an impressive way when he submitted what he later called a "parody" paper to a renowned sociology publication and got it accepted ([23], c.f. [24]). he did this to show that sociology is not really science because it suffers from the fact that one can deliberately (or unwittingly) publish nonsense as long as it is in a proper style. what sokal completely neglected was that the same is probably possible in any kind of science; it is certainly possible in engineering. it is also an inevitable consequence of the collective nature of all thinking. the mechanism that prevents such things from happening too often is psychological: trust. if an author publishes too many bad papers, his reputation gets damaged, and he is not trusted anymore. another possible mechanism is to fix some criteria that a paper must fulfil if it should be accepted for publication at all, but since such criteria should be valid for a wide range of thinking styles, they can only be of a quite general nature, e.g., that papers on integrated circuits should only be accepted if at least ac, dc, and transient simulations are provided, or of didactic nature, e.g., that a dc characteristic showing only a straight line does not contain enough information and should be replaced by a figure showing the deviation from linearity. note that introducing such criteria may be dangerous; they are effectively thinking constraints, and if they are introduced into an existing thinking style, the facts will change! the dynamics of thinking styles thinking styles are maintained by a thinking collective. as such collectives form and develop, the thinking styles form and develop as well, and together with them the facts. fleck described this extensively in his 1935 book whose title, translated into english, is "the genesis and development of a scientific fact" (fleck 1994, 1977). four aspects of the dynamics of thinking styles are particularly important in the context of this paper: the acquisition of a thinking style, its tenacity, how thinking styles interact, and how a thinking style develops from the front of research towards popular science. at the beginning of a new thinking style is chaos. the people involved in the genesis of the thinking style move around in that chaos by trying out many things and thinking many thoughts, without understanding what they are doing. the first non-chaotic thing they find when moving around in this chaos is resistance, they find it easier to move in some directions than in others. then they make first assumptions about what they perceive as a chaos, they decide what they want to investigate first, or, in other words, they deliberately introduce thinking constraints. sooner or later they will see patterns in the chaos, patterns which, once perceived, will from then on always be immediate appearances, "gestalten," which are then called facts. since the source of these facts is not only the chaos, but also the thinking constraints that were introduced deliberately, facts are rather made than discovered. therefore, the normal development of a thinking style is towards more and stronger thinking constraints and towards more intensive "gestaltsehen." it is interesting that a person who learns to use a new thinking style or a scientist tackling a new, complex problem normally live through a very similar process, starting with the feeling of facing a big chaos and arriving at seeing many immediate appearances. thus acquiring knowledge goes hand in hand with both seeing a larger number of immediate appearances (in other words, knowing more facts) and being more constrained in ones thinking (fleck 1977). all this is very similar to the concepts of information theory, which says in mathematical terms that information is gained by reducing entropy (shannon 1998). the reason why somebody introduces thinking constraints in the first place when he faces chaos is that he needs some continuity in thinking. thus, the further a thinking style develops, the more tenacious it gets. there is always a tendency to adjust everything to fit into the thinking style, but this can be done more passively or more actively. the most passive kind of tenacity is that contradictions against the thinking style become unthinkable. tenacity becomes more active when something that does not fit into a thinking style remains unseen, or is even kept secret. for example, a one-input rail-to-rail input ota was recently published in tcas-i, in which one half is only switched on for positive signals and the other for negative signals. a single transient simulation would have shown the authors that their circuit has enormous dynamic zero-crossing distortion, or, in non-engineering terms, that it does not work, but either they did not see it, or they saw it but omitted it. in the latter case, it can be that they thought it was a minor problem or a problem with the circuit simulator software, or it can even be that the omission was deliberate, although this was most probably not the case in this particular example. it was pointed out in (trocchio 1999) that such "small frauds" occur more and more frequently in modern economy-driven science, because nowadays many scientists need to present good results in order to get money for further research. in some cases, tweaking results unwittingly or deliberately may become essential for a scientist if he wishes to survive. an even more active way of dealing with contradictions is to declare them not to be contradicting, sometimes with great effort. one example is the amplifier classification made in (payne & toomazou 1996). the widespread current109 ajis special edition knowledge management december 2001 feedback operational amplifier (cfb opamp) does not fit in that classification, but it is treated as a special case in (payne & toomazou 1996), as a connection of two amplifiers that do appear in the classification. thus the authors of (payne & toomazou 1996) did their best to save the two-port thinking style; if they had been ready to break it, they would easily have arrived at a classification containing the cfb opamp like the one that was later presented in (schmiod 2000) by the author of this paper. the most active form of thinking style tenacity is when people see certain facts although they contradict other important thinking styles they use. this has happened in (cabeza et al 1994), where the authors state that they show how the four high-gain two-port sources can be implemented, but then actually end up with implementations of only one of these sources plus three devices that are not two-port sources, but still useful, properly working amplifiers. the last two examples must not be taken as reproaches, both groups of authors did fine work. the examples above were about an idea, the idea of amplifier classification, that was looked at from two different thinking styles (classical two-port theory and ic-design-related theory). time will show whether the new thinking style will replace the old one, which will only happen if some of the main proponents of amplifier-related thinking styles integrate it into their perspectives. this again will only happen if they find the new thinking style valuable, which is the only criterion that always plays a role for the integration of new thinking style elements into existing thinking styles. this brings us to the interaction of thinking styles. one can never say that the same thought is both true for person a and false for person b. either both a and b have the same thinking style, in which case the thought is either true for both of them or false for both of them. or the two have different thinking styles, and then the thought is not actually the same thought for the two, since it either must be unclear to one of them or they understand it differently. one of the main elements of scientific progress is certainly that thoughts move from one thinking style to another, which can only happen if a person gets in close contact with both thinking styles. when a thought moves into a different thinking style, it necessarily changes, becomes a different thought, and this is the main source of scientific progress. this idea can be expressed in a much briefer and more provocative form: scientific progress only happens when one person misunderstands the ideas of another person (c.f. schmid 1999). therefore one task of knowledge management must be to make sure that people are exposed to a wide variety of thinking styles. all that has been said up to here makes it sound as if it were easy to distinguish between two related thinking styles, and as if it were easy to tell whether a new thinking style starts to exist or whether a branch of an old one develops radically. but it is not easy, and the reader should resist the urge to use my theory to make divides between thinking styles. divides that are clear to some degree only ever exist if there are two organisations like the ssc and the cas who are interested in maintaining such a clear divide. the ideas about how science works that were discussed up to here are very much different from the image most people (including many scientists) have of science. how could so many people possibly be wrong about their own business? the reason for this is the way in which a well-developed thinking style develops further. when "gestaltsehen" develops from the initial chaos, scientists start to send around e-mails, sometimes with attached scanned hand notes, and soon the first conference and journal articles are published. the papers related to the thinking style are, however, very far from forming an integrated whole. there are many contradictions, and every scientist has a different idea of which aspects of reality are more important, and which ones should be treated as parasitic effects, or deferred for later inclusion into the theory. at some stage, the thinking style becomes important enough for the people involved that a handbook is made, a technical book whose main purpose is to portray the thinking style as an integrated whole. then a clear divide is made between basic theory and practice, second-order effects, and negligible parasitic effects. when a thinking style becomes handbook science, it becomes more rational, but also moves farther away from the chaotic reality. a further, more extreme change is the move from handbook to textbook science. in a textbook, everything is made as rational and logical as possible, and as many contradictions as possible are removed. in order to do this, many details have to be omitted. the thinking style is thus fundamentally reconstructed for teaching and becomes far less closely related to reality. one last, but rather important step, is to express a thinking style as popular science. for this, the reconstructed style is further simplified and expressed in popular terms. almost everything that is really important for the people who have helped to develop the thinking style is by then omitted from the description, but now it is accessible to everybody and becomes known far beyond the group of scientists and students who are seriously interested in it. this means: the more wide-spread a thinking style becomes, the more rational, integrated, and consistent it gets, but it also becomes far less complex and thus less realistic. now it becomes apparent why it is that most people, even scientists, say that science is essentially rational and scientific theories are wholly integrated and consistent. people know a very small part of science, if any at all, from the front of research, a larger part from textbooks (lectures, high school), but the major part from popular science. if they are not aware of the differences between front research, textbook and popular science, they will unwittingly see their 110 aj1s special edition knowledge management december 2001 own project as the exception and the distorted textbook and popular views as real. this also has important consequences in how scientific papers are written, as will be shown below. a final remark: i believe that a lot of what rational reconstructivists said about science is based on their reading mainly handbooks, textbooks, and papers written in handbook and textbook style, i.e., publications that intentionally are rational reconstructions. about theorists and designers so why is (schmid 2000) so polarising? the reason is that theorists and designers have two fundamentally different thinking styles. for a designer, every new project starts with chaos, he is used to gaining experience until he achieves sufficient "gestaltsehen" to solve the problem. for the designer, immediate appearance, sensory evidence, and experience are the most important sources of knowledge, and theories are just tools, like maps to a hill-walker, which give a rough impression of where to go but do not tell when and how one has to jump over a rock. if designers develop theories, they do it by rationally reconstructing their "gestaltsehen." this is the work methodology i used when i wrote (schmid 2000). theorists are different. for them, the theories are the most important elements of science, and their most valued sources of knowledge are tradition, analogy, theoretical beauty, and logical reasoning. they are like birds, for whom the map used by the hill-walker is more than detailed enough, and who only dive down to look at details in places that look promising on the map. they often do a systematic search for such places. if theorists discuss design, they start from theories and incorporate more and more non-idealities to approach reality. a similar work methodology has been described in (moschtyz 1977a,b). both the theorist and the designer can write works that have both theoretical and practical relevance, but they are not likely to really appreciate each others work. in the case of (schmid 2000), i was asked to use my new theory to derive some general rules telling the reader which amplifier he should select for a certain application. from a designer's point of view, this request is just as meaningless as it is for the hill-walker when the bird asks him to grow wings and do some flying in order to better understand how to use ropes, crampons, and ice axes. vice versa, the hill-walker can ask the bird to walk a few miles over scree in order to practice flight manoeuvres. this also explains a remarkable circumstance: the main contribution of (schmid 2000) is theoretical, but the theory is the rationalised description of a designer's "gestaltsehen." so it happened that the designers who read the paper found it very interesting and worthwhile, but the theorists did not really see the point. furthermore, the designers made many valuable suggestions on how to improve the theory such that it is a better rational account of the immediate appearances they see as well, but the theorists asked for more design-related rules, since they did not, even could not see that the theory itself was already a rationalised thinking style of a designer thinking collective. at present, the divide between theorists and designers is a very real one in the circuits and systems society, since the former mainly serve as reviewers for tcas-i, while the latter mainly serve for tcas-ii. naturally, knowing this, i submitted (schmid 2000) to tcas-ii to place it in a designers' context. it was accepted, but it might well have been rejected if i had submitted it to tcas-i. this shows how very important cunning reasoning (elkana 1981) is, the counterpart to epistemic reasoning that is deemed ultimately important by many scientists and philosophers of science. again, these explanations are oversimplified. in reality, every scientist is basically capable of thinking like a theorist or like a designer. there are very few scientists who have only learnt to use one of the two thinking styles. from this point of view, calling a scientist a theorist means saying that he is at present more inclined towards using the theorist thinking style. this inclination can well change during a scientist's professional life, and some people can even switch back and forth between the two thinking styles. science as theatre seeing the colourful complexity and irrationality of science, one starts to wonder why most scientific papers describe the results in such a rational way. all that happened along the long way to the results, all the reasons for decisions made along this way, everything irrational, disappears from the final publication, and with it disappears most of the experience of the scientist. only a very little part of his experience remains, and // is not the part that made "gestaltsehen" possible for him. as a result, one could never become a good scientist by reading papers alone, and this is why conferences are so important: there one can gather the information omitted from the papers by talking to the people who wrote them. ajis special edition knowledge management december 2001 i think however, that one could easily provide some of this essential information in scientific papers, in the form of background information, like the first section of this paper. the image that guided me is that papers are mostly written like greek dramas, but should be written like epical theatres. a scientific paper is conventionally written like a greek drama: everything comes out as it has to come out, destiny rules, and the point of the greek drama is merely to describe how the inevitable happened. from the perspective of the above discussion, however, it would make much more sense to write scientific papers like brecht's epical theatres, the point of which is to describe why it came out as it did, when it could so easily have come out differently. the main differences between a greek drama and an epical theatre are the following: in the greek drama, the actors deceive the spectators, but in the epical theatre, they demonstrate and tell stories. the audience identify themselves with the characters in the greek drama, but in the epical theatre, the audience are kept at a distance and are forced to think for themselves. in the greek drama, the actors remain in their roles, in the epical theatre, the actors come out of their roles. the greek drama draws the audience into the plot, but in the epical theatre, the audience is cosciously distanced through narrative comments by the story teller, inserted songs, or by other means. finally, the greek drama has a clear end, while the epical theatre only ends in time, not in thought. the question "why did it come out as it did, when it could so easily have come out differently?" is of ultimate importance to the person who wants to adopt the thinking style used by the authors of a paper. the reader of a paper can answer this question better if the paper contains more epical elements. although the better part of a paper will still have to be written in the conventional, rational form, the authors should come out of their role as describers of the inevitable, by demonstrating how they work and think, and by telling the story behind their paper. this can either be done in special sections, as in this paper, or by short, personal comments by the author that are inserted into the descriptive material of the paper. the latter is done remarkably often in the papers that are considered very good and important by the scientific community. the stepping out of the role must be made clear to the reader. while it is in fact conceivable to include songs into technical papers (c.f. mason 1956), which had a major impact on circuit theory and also features a rhyming abstract), it is preferable to use a different writing style and, if the publisher permits, also a special layout for epical elements. finally, the 'conclusions' section should only mark the end of the paper, but not the end of the thought, and should thus contain open questions and indicate possible ways in which the reader may direct his thoughts. i think that technical and editorial committees should seriously discuss the possibility of writing scientific papers in an epical style and of explicitly asking for such elements in the author's guidelines of journals and conferences. until then, it is up to every author to think about how he can introduce epical elements in his papers in a way that the result is not too unconventional to be accepted by the reviewers of a certain publication. how the author applies his theory to show the reader how the ideas described in this paper can be applied, i will now describe a few of my own projects. conflict resolution in the past, i solved a few conflicts between my thinking styles and others simply by telling and showing people that i value their view even though i do not share it. and it also helps me that i believe in the value of all thinking styles, even if i don't see it yet. so if somebody i talk to says something that sounds stupid, i work until i understand them. the main tool i use for this is to try and determine the other person's image of knowledge and compare it to mine. of course, anger and laziness sometimes prevent me from doing this, but that is a different topic. individuality in organisations carol steiner once asked me an interesting question: why is individual knowledge valued highly in some organisations but not in others? a possible answer follows from the discussion above. what other people call "individual knowledge" is, in my words, "knowledge that an individual brings into the organisation's thinking style from another thinking style." if this transfer should succeed, the other people in the organisation must trust the person transferring the knowledge through the time during which that person does the transfer. this can really take quite long, so first above all the person must be trusted. so individual knowledge is valued highly in some organisations but not in others because people are trusted highly in some organisations but not in others. 112 ajis special edition knowledge management december 2001 i think that real trust only occurs in an organisation when the management of the organisation value people highly and have true respect and understanding for all people. the management are so critical because they determine the company thinking style, so if, and only if, they really trust people, the other people in the organisation will trust too. 1 think that if this is not the case in a certain company, only radical measures help. at least somebody who has this true humanity and understanding must come into the management, and everybody who cannot tolerate such human views must be fired. in my company, this happened by chance a few years ago: after it was sold by the former owners, who had acquired it as a part of a telecom company they bought up, but who were not really interested in hearing aids, the whole management was replaced, with the exception of the person who now is the managing director. the management we have today meet my two criteria, and both the company thinking style and lots of practical aspects of daily company life are exceptionally human. knowledge management speaking of management, what about knowledge management? in my view, knowledge is very dynamic and only a product of thinking styles and thinking collectives. it cannot be managed directly, it can only be managed through the people who make it happen. as an amateur who by chance got into the role of a part-time knowledge manager, i do only three things and nothing more: i try to understand how knowledge happens in my company or organisation, i help people see how knowledge happens, and i try to give people a good environment in which they can make knowledge happen. the latter can mean providing people with a documentation system that lets them organise documents as best fits their thinking style. as the thinking style develops, the system may have to adapt itself. the best i could think of is to set up a web server that works like the internet, and to give it the potential to mimic chaotic and journal science, handbook science, and textbook science. to account for the chaotic and journal science, all engineers can add content to the web site as they please, from scanned hand notes over e-mail messages to papers from journals, and they can interrelate their work with the work of others as they see fit by using hyper-links. then there is a second layer of documentation, where things that have actually been built are described, or where concept studies are kept for the future. these documents are ordered in a more planned way, in a tree structure, by editors who are responsible for a certain part of the document tree; these documents correspond to the "handbook science." finally, there are also official documents used to give to managers, to users of the technical components, and for quality control; they are entered into a rigid, well-defined frame. these documents correspond to "textbook science." structure of single documents this can also be done with single documents. my documents normally start with a list of changes that enables readers of previous versions to skip what they have already seen. then there comes a chapter in the style of textbook science, where managers and engineers having very little time get a brief, rational, straightforward explanation of what i have done and what my integrated circuit does. they have to trust me on that if they only read the textbook chapter. if they need more, they can read the handbook science chapter where i describe the assumptions, results, and tradeoffs in detail, so that the readers can see why i think what i said in the previous chapter is true. with this chapter, the readers can also use or re-design my circuits. if they want to check on this too, they can read the journal and chaotic science chapter, which is nothing more than scanned hand-written notes, paper excerpts, and computer plots. intuition in documents finally, i have also started to use writing style elements that should capture the intuitive part of science. i have not yet used them in my company, mainly because i could not figure out how to do it with the publishing software we use. in my doctoral thesis (schmid 2000a), which i wrote in l^tex, i wrote a very brief margin paragraph for every single paragraph of the main text. this margin paragraph tells the reader what the most important aspect of the main paragraph is for me. reading just the margin, a reader can get an intuitive grasp of the structure of my text. an additional benefit is that if i have trouble writing the margin text for a certain paragraph, this indicates that the paragraph does not contain a single, clear thought! an additional style element is that every chapter has an introductory section called "background" where i describe how i got the results. this is not a rational reconstruction, but the story of what i actually did and why. then the subsequent sections can be in traditional scientific style. 113 aj1s special edition knowledge management december 2001 the clear mark-up of the intuitive parts, plus an explanation of what it is, in the introduction to my dissertation, allows the reader who only wants to have traditional scientific texts to skip the intuitive material. this may be very important for the acceptance of a publication. brechtian refinement i am presently working on a refinement of these style elements. one major problem of science is that people identify themselves far too much with the ideal view of rational and objective science, i am especially interested in realising brecht's estrangement effects (the "coming out of one's role" described above) in technical documentation. i have not yet found a satisfactory solution that does not disturb the text flow too much. getting the estrangement effects right is one of the main challenges of a brecht play, so i don't expect it to be easy to do in technical writing. conclusion the theory of thinking presented in this paper is a kind of middle way between two extremes, in three important aspects: facts have no inherent existence. people have tried to find objective, absolute facts in the past, but failed. some concluded from it that there are no facts at all, but this is absurd as it contradicts daily experience. the way out of this dilemma is to state that facts exist only within a thinking style, which a person chooses to apply in a specific situation for various reasons. facts are therefore both relative and real. individual thinking has no inherent existence. people have tried to find individual, absolute thinking in the past, but failed. some concluded that there is no individual self, but this is absurd too. the way out is to accept thinking as a collective process and search for individuality in the way that thinking is performed by certain people in certain situations. thinking is therefore both individual and collective. last, but not least, thinking styles and facts are not timeless. people have tried to find timeless knowledge in the past, but failed. some concluded that there is no such thing as knowledge or facts, but again, this nihilistic view is absurd. the way out is to recognise knowledge as a process rather than a state, and facts as things that change with time. on the other hand, people using a well-developed thinking style strive to prevent changes. the nature of thinking styles is therefore both dynamic and static. classical technical writing does, however, only emphasize the real and the static. it either neglects the people altogether or emphasizes the individual side of thinking. how can we re-integrate relative, collective, and dynamic aspects of knowledge into technical writing? i have made a few suggestions of how to do it, derived from personal experience, mainly related to technical writing and to a theory-practice duality i found in a technical society of the ieee. these are just my first small attempts. clearly, lots of work still needs to be done before the ideas presented here can have an effect on technical documentation. in this sense, the main purpose of this paper was to encourage the readers making their own attempts to transfer the ideas presented here into the thinking styles they use themselves. acknowledgements i wish to thank my friend prof. martin hanggi (university of notre dame, indiana, u.s.a.), who is a fine theorist according to my definition. he disagreed with me in many very interesting philosophical discussions. without him, i could not have brought my ideas to a focus. i also thank christoph a. schenker (student of theatre and drama, university of bern, switzerland) for discussing brecht with me and helping me to form a clear idea of brecht's style elements. references gunnar andersson, kritik und wissenschaftsgeschichte — kuhns, lakatos' und feyerabends kritik des kritischen rationalismus, j. c. b. mohr (paul siebeck), tubingen, 1988. arie f. arbel, "negative feedback revisited," analog integrated circuits and signal processing, vol. 10, no. 3, pp. 157-178, aug. 1996. william broad and nicholas wade, betrayers of the truth, simon and schuster, new york, 1982. rafael cabeza, alfonso carlosena, and luis serrano, "unified approach to the implementation of universal active devices," electronics letters, vol. 30, no. 8, pp. 618-620, apr. 1994. 114 ajis special edition knowledge management december 2001 rafael cabeza and alfonso carlosena, "analog universal active device: theory, design and applications," analog integrated circuits and signal processing, vol. 12, no. 2, pp. 153-168, feb. 1997. john choma, "feedback amplifier configurations," in the circuits and filters handbook, wai-kai chen, ed., chapter 25, pp. 820-850. crc press, inc., boca raton, florida, 1995. yehuda elkana, "two-tier-thinking: philosophical realism and historical relativism," social studies of science, vol. 8, pp. 309-326, 1978. yehuda elkana, "a programmatic attempt at an anthropology of knowledge," in science and cultures, vol. 5, pp. 1-76. d. reidel publishing company, dordrecht, holland, 1981. paul karl feyerabend, philosophical papers: realism, rationalism and scientific method; problems of empiricism, vol. 1-2, cambridge university press, cambridge, 1981. paul karl feyerabend, against method, new left books, london, 1975. ludwik fleck, genesis and development of a scientific fact, university of chicago press, chicago, 1977, ludwik fleck, entstehung und entwicklung einer wissenschaftlichen tatsache—einfiihrung in die lehre vom denkstil und denkkollektiv, b. schwabe und co., verlagbuchhandlung, basel, 1935. ludwik fleck, entstehung und entwicklung einer wissenschaftlichen tatsache—einfiihrung in die lehre vom denkstil und denkkollektiv, stw 312. suhrkamp, 1994. clifford geertz, the interpretation of cultures, fontana press (harper collins), london, 1993, ieee ssc soc., "isscc 2001 call for papers," 2000. samuel j. mason, "feedback theory-some properties of signal flow graphs," proceedings of the institute of radio engineers, vol. 41, no. 9, pp. 1144-1156, sept. 1953. samuel j. mason, "feedback theory-further properties of signal flow graphs," proceedings of the institute of radio engineers, vol. 44, no. 7, pp. 920-926, july 1956. george s. moschytz, "die morphologische methode in der elektrotechmk mit beispielen aus der theorie der aktiven netzwerke (teil 1)," bulletin sev, vol. 68, no. 6, pp. 275-280, mar. 1977. george s. moschytz, "die morphologische methode in der elektrotechnik mit beispielen aus der theorie der aktiven netzwerke (teil 2)," bulletin sev, vol. 68, no. 8, pp. 401-406, apr. 1977. george s. moschytz and a. carlosena, "a classification of current-mode single-amplifier biquads based on a voltage-to-current transformation," ieee transactions on circuits and systems-h: analog and digital signal processing, vol. 41, no. 2, pp. 151-156, feb. 1994. alison payne and chris toumazou, "analog amplifiers: classification and generalization," ieee transactions on circuits and systems-i: fundamental theory and applications, vol. 43, no. 1, pp. 43-50, jan. 1996. karl raimund popper, objective knowledge, oxford university press, 1972. hanspeter schmid, single-amplifier biquadratic mosfet-c filters for video frequencies, ph.d. thesis, swiss federal institute of technology, zurich, nov. 2000b, hanspeter schmid, "approximating the universal active element," ieee transactions on circuits and systemsii: analog and digital signal processing, vol. 47, no. 11, pp. 1160-1169, nov. 2000. hanspeter schmid, "black tools-how to win a debate," ieee professional communication society newsletter, vol. 43-44, no. 2/1999-2/2000, 1999-2000, arthur schopenhauer, die kunst, recht zu behalten, number 1658 in insel taschenbuch. insel verlag, frankfurt am main, 1995, claude e. shannon, "communication in the presence of noise," proceedings of the ieee, vol. 86, no. 2, pp. 447457, feb. 1998, (reprinted from proc. ire, vol. 37, no. 1, pp. 10-21, jan. 1949.). alan d. sokal, "transgressing the boundaries: toward a transformative hermeneutics of quantum gravity," social text,, no. 46/47, spring/summer 1996. alan d. sokal, "web site on the 'social text affair'," url: http://www.phvsics.nvu.edu/facultv/sokal/. federico di trocchio, der grosse schwindel—betrug und falschung in der wissenschaft, rohwolt taschenbuch verlag, reinbek bei hamburg, 1999, daniel vischer, plane deinen ruhm!, verlag der fachvereine, zurich, 4 edition, 1992. b. wilson, "transconductance feedback amplifier configuration exhibiting bandwidth-independent voltage gain," iee proceedings circuits, devices and systems, vol. 146, no. 5, pp. 242-248, oct. 1999. 115 ajis vol 4 no. 1 concepts of cognition and consciousness: four voices bonnie a. nardi apple research laboratories apple computer 1 infinite loop cupertino, ca 95014 phone:+l -408-974-8708 fax: +1-408-974-8414 http:/avww.atg.apple.com/personal/bonnie_nardj/ abstract this paper considers theories of cognition and consciousness in four traditions: neuroscience, cognitive science, activity theory and the distributed cognition approach. it is most concerned with social theories of consciousness—activity theory and distributed cognition—but briefly considers biological and computational models as a foil or backdrop against which the social theories stand out more clearly. introduction of all the many artifacts we've produced in our two million years on the planet, computers are the most humanlike. our encounter with computers has given a new edge to ancient philosophical questions about what it means to be human. while taking up such questions might seem a long way from information systems design and evaluation, as an anthropologist and designer, i believe that the way we describe what it means to be human is ultimately reflected in our designs. thus i have found it important and useful to try to keep up with the major attempts to define who we humans might really be. while there are a great many such attempts, i would like to analyze four powerful paradigms whose vocabulary and concepts permeate today's discourse: neuroscience, cognitive science, activity theory and the distributed cognition approach. philosophers also have much to say about cognition and consciousness, but i'll concentrate here on the scientific approaches. i will not deal with the practical matter of exactly how to apply these theories to design and evaluation (but see nardi, 1996 for the application of activity theory to problems of human-computer interaction); rather i hope to raise consciousness about the implications of the concepts and vocabulary that become part of the way we talk about the people we study or design for, whether in a research or applied context. two concepts that capture modern cutting edge efforts to describe the nature of humankind are cognition and consciousness. neuroscience, cognitive science, artificial intelligence, psychology, philosophy and anthropology have all taken as one of their greatest challenges explaining consciousness or cognition. the question asked in this paper is: what is the view of humanity that emerges in each paradigm as the difficult problems of consciousness and cognition are tackled? reflecting on this question will, i hope, help to move our dialogue forward as we define who we are, so that we can reflect that definition in our designs. i offer these analyses of cognition and consciousness in the spirit of what donald schon calls the "reflective practitioner" (schb'n, 1983)—they are meant to raise our level of awareness about the concepts we employ and the assumptions we make about people as we study and design. consciousness, part i neuroscience has lately set itself the task of explaining consciousness, marshaling its resources to try to understand "those states of sentience and awareness" that are an "inner, first-person" phenomenon, as john searle defines it in his review of research on consciousness (searle, 1996). according to this view, the seat of consciousness is the brain, and the way to understand consciousness is to understand how the brain works. consciousness is a fundamentally biological phenomenon; its secrets will yield to biological analyses. as searle puts it, "it is an amazing fact that everything in our conscious life, from feeling pains, tickles, and itches to—pick your favorite—feeling the angst of postindustrial man under late capitalism or experiencing the ecstasy of skiing in deep powder—is caused by brain processes." francis crick (1993) tells us that our joys, sorrows, memories, ambitions, identity and free will are "no more than the behavior of a vast assembly of nerve cells and their associated molecules." consciousness is as biological as "growth, digestion, or the secretion of bile," deadpans searle. a very strong claim is made that the brain causes consciousness. under a biological framework, the problems of understanding consciousness lie in the complexity of the brain itself (those mind-boggling 100 billion neurons and their complicated connections), the moral impossibility of controlled experiments and the difficulty of talking rationally about what it feels like to be pinched (or to ski in deep powder, or whatever). the task before neuroscience is to find a brain architecture that explains how 64 ajis vol 4 no. 1 something that fits in the palm of the hand can ski and remember and plot and plan and feel sad and try to come up with architectures to describe itself. crick, for example, gives a go at sketching an architecture by describing the brain as possessed of "emergent properties" that "arise in the brain from the interactions of its many parts." gerald edelman (1992) is more specific in detailing a brain that actively recategorizes memories, learns by valuing some stimuli over others, discriminates self from others, categorizes temporal events, forms concepts, and provides reentrant connections between memory and perceptual categorization. in all of these architectures, the brain and its physical structures are the locus of study, the home of consciousness. as edelman (1992) says, "...mind depend[s] on special arrangements of matter." "we need not reach beyond biology itself to mount any exotic explanations of the mind" (edelman, 1992). in the neuroscientific view, the problems of consciousness are tidily contained beneath the skull, in one known place, with physical structures that can be unambiguously defined and studied. sure the brain is complex, but at least we know where to look. cognition, part i the cognitive scientists, by contrast, have chosen to concentrate on something far more abstract than neurons and synapses. they take up not consciousness (which would sound a bit mystical to most of them), but cognition, defining symbolic representations as the mainstay of cognition. no secretions of bile here. for cognitive science, cognition is, to give a talk show definition, all the neat mental stuff that people can do. problem solving is a big favorite of the cognitivists (and they love games and logic puzzles, the kind of thing the math whizzes were always best at), but they embrace any kind of mental activity as their turf: classification, remembering, decision making, judging, calculating and so forth. their game is to spell out how people represent and manipulate representations. this clean disembodied activity can be simulated on computers, and showing that a program can solve a problem is an important activity for cognitivists. they don't worry too much about consciousness per se because their heart belongs to the more precise image of the person as rational problem solver, systematically crunching away at those representations and "mental models." a focus on consciousness, rather than cognition, forces one to the ambitions and sorrows and free will and skiing in deep powder that a francis crick or gerald edelman will take a crack at, but these squishy things quickly prove intractable in a representations-centric perspective. the cognitivists' strategy has been to first figure out human problem solving capabilities (as they define them), and then to someday move on to the rest of life, after the cognitive ship has come in. like the neuroscientists, the cognitive scientists confine their attention to what's under the hood, that is, to what goes on inside the head of a single individual—who is supposedly the locus of cognition. they firmly locate cognition "beneath the skin" as lave (1988) puts it, and there is a neat input-output loop between external stimuli and the resulting representations that the individual creates, stores and manipulates. the body, other people and things like tools are not especially interesting for a cognitivist account, except as "stimuli." the theory of cognition provided by cognitive science is profoundly cartesian, separating "mind" from everything else. consciousness, part ii what if there were a theory of mind that was not cartesian, that did take account of the role of other people and tools and the environment in which cognition takes place? what if there were a social theory of consciousness in which consciousness was seen as inextricably woven together with the social, the social being defined as interactions with other real people, as well as interactions with the tools other real people have designed and left for others as part of their culture? such a theory would most certainly look beyond "biology itself to understand consciousness and would not regard thinking about other people and tools as "exotic," as our neuroscience friends would have it. there is such a theory, and it is called activity theory. i'll sketch out its five main principles and then move to a discussion of another more recent theory —"distributed cognition"—that shares some insights with activity theory but which does not, in my opinion, capture its depth or richness. the reader is likely to run across the recent work of ed hutchins or jean lave and perhaps less likely to find vygotsky or leont'ev, and the other activity theorists, so i'll make the pitch here that vygotsky and leont'ev and their colleagues have already covered most of the ground today's distributed cognition theorists have, and in fact gone well past it. the vagaries of russian history, the home of activity theory, have meant that activity theory has been slow to reach the west. for those who do know this work, i'll note that i use the term "activity theory" broadly to include the culturalhistorical school founded by vygotsky in the 1920's, as well as the activity theory work of luria and leont'ev 65 ajis vol 4 no. 1 and their students, and the continuing work in these traditions in many countries. key activity theorists include the russian psychologists vygotsky (1978, 1934/1986), rubinshtein (1940), and leont'ev (1978). there is a thriving activity theory tradition with computer-science related work in scandinavia (bertelsen, 1994; b0dker, 1989, 1991; christiansen, 1996; engestrom and escalante, 1996; kaptelinin, 1993, 1996; kuutti, 1991, 1996; saarelma, 1993; sjoberg, 1996), as well as increasing interest in activity theory applied to information systems the u.s. (bellamy, 1996; blumenthal, 1995; 1996; holland and reeves, 1996; nardi, 1996a), europe (bannon, 1990; bannon and b0dker, 1991; draper, 1993; imaz and benyon, 1996; raeithel, 1992; raeithel and velichkovsky, 1996), canada (cohen, candland and lee, 1995), and australia (bourke, verenikina, and gould, 1993), and also continuing work in russia (tikhomirov, 1991; zinchenko and munipov, 1989; zinchenko, 1996). a key point i wish to alert the reader to is that activity theory insists that consciousness is social. consciousness can never be reduced to brain mechanisms or representations inside one person's head, because without other people and artifacts, there is no consciousness. other people and artifacts are not just "stimuli" as cognitive science would have it, nor are they epiphenomonal to the firing of synapses; rather they are crucial to any kind of consciousness. if it weren't for a social world, there would be no you. your "mind" does not belong strictly to you; it is not really located under your skull; rather it functions only so long as you are in a social matrix in which it can find expression. remember the old joke about the person who answers the door and says, "there's no one home." they really mean it; that's why it's funny. according to activity theory, the mind is like that: without the rest of the family or roommates around, there is literally "no one home," no consciousness. i may be sitting here by myself in california struggling to explain activity theory, but this would be impossible if i did not have some notion of you, dear readers, in my consciousness to motivate and channel my activity. my mind is really stretched across the ocean, all the way to australia, in an absolutely real sense. activity theory does not in any way deny the importance of the brain or its structures. in fact a number of eminent activity theorists such as luria were neuropsychologists. activity theory sees the brain as "the material basis for all complex psychological processes" (luria, 1972). activity theory insists however that the brain does not cause consciousness; rather it provides the physical matrix in which it can develop and change. and, most crucially, we must look beyond the brain itself to understand consciousness. one more point before we plunge in to the principles of activity theory: activity theory says you are what you do. if you spend your days digging ditches, your consciousness will be thusly shaped. if you find yourself writing computer programs as your life's work, that is quite another kind of consciousness. "doing" is very broadly defined in activity theory to include things like speaking, meditating, remembering, as well as activities more centered in the body and its movements. this flattening of "doing" to include all kinds of things is disconcerting at first, but it scores big points in helping to make us aware that our consciousness will be shaped just as much by praying as by bungee jumping. it helps to reduce the seduction of cartesianism which wants to yank the mind away from everyday life and set up a life of its own. activity theory posits that you are a unique individual because you've had and chosen for yourself a unique set of "doings" in your life. saying you are what you do is a lot different than saying you are your neurons, or you are the mental representations that sit under your skull. saying you are what you do includes the way you physically move through the world, and the tools you pick up and put in your hands and use, and the hugs and kisses you give and get and everything that makes you human. activity theory asserts that the consciousness created by your everyday activities spills over into other seemingly unrelated activities, so that activities like formal education and reading have tremendously powerful effects on consciousness. now let's look systematically at the five principles of activity theory: hierarchical structure of activity, objectorientedness, internalization/externalization, tool mediation, and development. hierarchical structure of activity in activity theory the unit of analysis is an activity. leont'ev, one of the chief architects of activity theory, described an activity as being composed of subject, object, actions, and operations (1974). a subject is a person or a group engaged in an activity. an object (in the sense of "objective" or motive) is held by the subject, and motivates activity, giving it a specific direction: "... behind the object there always stands a need or a desire, to which [the activity] always answers" (leont'ev, 1974). the term "objectified motive" is a useful mnemonic. one might also think of the "object of the game" or an "object lesson." actions are goal-directed processes that must be undertaken to fulfill the object. they are conscious (because one holds a goal in mind), and different actions may be undertaken to meet the same goal. for example, "a person may have the object of obtaining food, but to do so he must carry out actions not immediately directed 66 ajis vol 4 no. 1 at obtaining food..: his goal may be to make a hunting weapon. does he subsequently use the weapon he made, or does he pass it on to someone else and receive a portion of the total catch? in both cases, that which energizes his activity and that to which his action is directed do not coincide" (leont'ev, 1974). goals can have lower level goals, which can have lower level goals, and so forth, much like the concept of goals/subgoals in ai and other traditions. for example, making the hunting weapon is an action which then entails finding suitable materials and tools for the manufacture of the weapon, and so on. objects can be transformed in the course of an activity; they are not rigid structures. as kuutti (1996) noted, "it is possible that an object itself will undergo changes during the process of an activity." objects do not, however, change on a moment-by-moment basis (see holland and reeves, 1996). actions are similar to what are often referred to in the human computer interaction literature as tasks (e.g., norman, 1991). activities may overlap in that different subjects engaged together in a set of coordinated actions may have multiple or conflicting objects (kuutti, 1991). moving down the hierarchy of actions we cross the border between conscious and automatic processes. actions have operational aspects, that is, the way the action is actually carried out. operations become routinized and unconscious with practice. operations do not have their own goals; rather they provide an adjustment of actions to current situations. when learning to drive a car, the shifting of the gears is an action with an explicit goal which must be consciously attended to. later, shifting gears becomes operational, and "can no longer be picked out as a special goal-directed process: its goal is not picked out and discerned by the driver" (leont'ev, 1974). operations depend on the conditions under which the action is being carried out. if a goal remains the same while the conditions under which it is to be carried out change, then "only the operational structure of the action will be changed" (leont'ev, 1974). activity theory holds that the constituents of activity are not fixed, but can dynamically change as conditions change. this is an important distinction between activity theory and cognitive science-based techniques such as goms (card, moran and newell, 1983). in activity theory, all levels can move both up and down (leont'ev, 1974). as we saw with gear-shifting, actions become operations as the driver habituates to them. an operation can become an action when "conditions impede an action's execution through previously formed operations" (leont'ev, 1974). for example, if one's mail program ceases to work, one continues to send mail by substituting another mailer, but it is now necessary to pay conscious attention to using an unfamiliar set of commands. notice that here the object remains fixed, but goals, actions and operations change as conditions change. object-orientedness the principle of "object-orientedness" (not to be confused in any way with object-oriented programming) is one of the most important principles of activity theory. every motive is an object (as defined above) but there is another related sense of the word object, i.e., a prospective outcome toward which activity is directed, around which activity is coordinated, and which will be crystallized in a final form when the activity is complete. for example, a computer program is an object of a programmer's activity. it is not her "motive"; the software program is that toward which she directs her activity so she can attain a motive, say, becoming the best programmer in california. line by line she writes the code; the program itself channels her attention and thought and motion. objects can be physical things (such as the bull's eye on a target) or ideal objects ("i want to become a brain surgeon."). leont'ev noted that the concept of object is not limited to physical, chemical, and biological properties; it crucially includes social properties as well. internalization/externalization activity theory differentiates between internal and external activities. the traditional notion of mental processes in cognitive science corresponds to internal activities. activity theory emphasizes that internal activities cannot be understood if they are analyzed separately, in isolation from external activities, because there are mutual transformations between these two kinds of activities. internalization is the transformation of external activities into internal ones. activity theory argues that it is not just mental representations that get placed in someone's head; it is the holistic activity including motor activity and the use of artifacts that are crucial for intemalization. for example, learning to calculate may involve counting on the fingers, in the early stages of learning simple arithmetic. once the arithmetic is internalized, the calculations can be performed in the head without external aids. internalization provides a means for people to try potential interactions with reality without performing actual manipulation with real objects (mental simulations, imaginings, considering alternative plans, etc.). in some cases external components can be omitted in order to make an action more efficient, e.g., in the case of 67 ajis vol 4 no. 1 performing calculations in the head. internalization can help to identify an optimal way to perform action before performing the action externally. externalization transforms internal activities into external ones. extemalization is often necessary when an internalized action needs to be "repaired," or scaled, e.g., when a calculation is not coming out right when done mentally, or is too large to perform without pencil and paper or calculator (or some external artifact). externalization is also important when a collaboration between several people requires their activities to be performed externally in order to be coordinated. while the concept of internalization shares much with traditional cognitive science's notions of information processing, extemalization is not emphasized in cognitive science. furthermore, activity theory says that it is the constant transformation between external and internal that is the very basis of human cognition and activity. mediation activity theory's emphasis on social factors and on the interaction between people and their environments explains why the principle of tool mediation plays a central role within the approach. first of all, tools shape the way human beings interact with reality. and, according to the principle of internalization/externalization, shaping external activities results in shaping internal ones. second, tools reflect the experience of other people who encountered and solved similar problems and invented or modified a tool to make it effective and efficient. this experience is accumulated in the structural properties of tools (their shape, size, material, and so forth) as well as in the knowledge of how the tool should be used. the use of tools constitutes an accumulation and transmission of social knowledge. tools influence the nature not only of external behavior but also internal mental functioning. vygotsky identified two kinds of tools: technical tools and psychological tools. technical tools manipulate physical objects (e.g., a hammer) while psychological tools are used to influence other people or oneself (e.g., the multiplication table, a calendar, or an advertisement). development finally, activity theory requires that activity be analyzed in the context of development. development continuously reforms and develops practice. in activity theory development is not only an object of study, it is also a general research methodology. when activity theorists do experiments, they always hang around long enough to see what happens as the person gets used to the tool being tested, or the material being internalized. for example, in a simple but classic study, kaptelinin (1993) investigated how ordinary computer users adapted to two different forms of a menu. users were first presented with a menu with all menu items spelled out fully and given in the same order on repeated trials. then one group of users got the menu with the items fully spelled out but in scrambled order on successive trials. the second group got a menu with dots for the menu items but with the items always in the same order. the second menu was harder to use at first, but proved to be faster and easier once people got used to it (they could look up the bindings till they learned them). a typical cognitive science analysis might have stopped the experiment prematurely, or worse, counted on a theory of visual "affordances" (norman, 1991) for which the first menu clearly was superior (i.e., because you could identify the actual items and didn't have to take time to learn them; they could be directly "picked up" from the environment as gibson (1979) would have said). but developmentally, users were able to process the ordered dotted items better over time. this experiment also shows the relationship of the brain to real activity: clearly there is something about the brain that does linear processing efficiently, once a person has had the experience of a particular activity; here using the menu over time. activity theory is quite happy to look at the wetware aspects of the brain as neuroscience does, but it insists on viewing those aspects in the context of some real activity, and developmentally, over time, and not as a question reduced to nerve cells and molecules in the way crick talks about it. integration of the principles these basic principles of activity theory should be considered as an integrated system. a systematic application of any of the principles makes it eventually necessary to engage all the other ones, just as we saw that mediation calls upon internalization/extemalization. activity theory insists on the unity of these principles and does not abstract out any single process (e.g., internalization), because then the whole activity—and consciousness itself—cannot be understood. 68 ajis vol 4 no. 1 summary in activity theory then, what it means to have a human consciousness is to be a part of a web of social activities and to live and act in a culturally elaborated environment that is profoundly artificial, populated by a wealth of tools, including language. as vygotsky noted, the use of artificial means, the transition to mediated activity, fundamentally changes all psychological operations just as the use of tools limitlessly broadens the range of activities within which the new psychological functions may operate. in this context, we can use the term higher psychological function, or higher behavior as referring to the combination of tool and sign in psychological activity (1930/1978). vygotsky reckoned that people start out with a set of basic capabilities such as attention, will, intention, and then, through human activity, develop what he called the higher psychological functions. these functions include language, decision making, abstraction, generalization, classification, problem solving, and more advanced forms of will, attention, memory, intention. vygotsky saw the potential for development as limitless. he very much believed in formal, especially scientific, education, as a way to create a person possessed of the higher mental functions (seeing education as a necessary but not sufficient condition for development). vygotsky's very definition of consciousness, then, is the active processes of the higher and lower psychological functions. because these functions arise, develop and change within a social matrix (as detailed in the principles of mediation, internalization/externalization, etc.), they cannot be seen as residing strictly "under the skull"—consciousness is social, simultaneously beyond and within the individual. because of externalization, there is not a one-way path from outer to inner activity; rather, inner activity gets concretized and made public through external activity. the emphasis on the artificial pits activity theory directly against the neuroscientific view which is fundamentally biological. edelman (1992) talks about "putting the mind back into nature," while activity theorists (in direct descent from marx) see the mind as fundamentally the product of the social relations in which people spend their everyday lives. activity theory wants to get the mind into culture, while neuroscience wants to nail it down in nature. the neuroscientific view does not account for the role of artificial tools, including language, in the development of consciousness. (the ability to learn language obviously has a biological substrate, but words and the concepts they convey are patently manufactured, artificial things). while it might be argued that a neuroscientist would of course think that language is important to consciousness, there is in fact an unambiguous divide between the two approaches: a theory that is profoundly biological does not find causality in the artificial. the emphasis on biology leads one away from the artificial. one never gets from brain architecture as cause to activity as cause. activity never gets into the discussion at all. i argue that it is necessary to adopt the activity theory strategy of acknowledging activity as the primary shaper of consciousness, and then to get to the brain and biology. activity theory declares that consciousness depends directly and profoundly on the mediation provided by human activity. to say that consciousness is fundamentally about nerve cells and molecules would be impossible in activity theory. a limitation of the activity theory approach is that while activity theorists posited the importance of the wider culture in shaping consciousness, their analyses in practice have not included culture in very interesting ways. this is an area for further development within activity theory. it is not an easy area as 150 years of anthropology has shown the difficulty of elaborating what culture is all about, and especially how culture affects the individual. understanding culture remains a future challenge for activity theory. another limitation of activity theory is that it provides few tools for understanding social organization and its impact on activity. this is an area of active research in europe and progress is being made (see especially engestrom, 1987 andraiethel, 1992). activity theory is optimistic about the potential for human development. while recognizing that everyone starts out with some basic equipment such as the ability to pay attention, activity theory says that consciousness is the result of development. what you spend your time doing is what shapes your consciousness. because you have some say over how you spend your time, you are in charge of your consciousness. if you design mediating tools for others (such as computer hardware or software), you are also responsible, in part, for the consciousness of others. our tools make us who we are, says activity theory. as designers this gives us tremendous power and tremendous responsibility. 69 ajis vol 4 no. 1 cognition, part ii activity theory has provided a social theory of consciousness. a much newer paradigm, "distributed cognition" has to some extent revisited much of what the activity theorists discovered and explicated. but there are some crucial differences in the two approaches that i will explore here. distributed cognition has taken a sharp turn away from vygotsky's notion of the individual as a being with unbounded potential, the notion of consciousness as capable of limitless development. distributed cognition is much less optimistic concerning human development and sees tools as a means of reducing the complexity of human tasks. in my opinion, edwin hutchins is the most articulate proponent of the distributed cognition approach, so i will consider his work in this brief paper rather than provide a full overview (but see also halverson, 1993; salomon,1993; sandberg and wielinga, 1993; rogalski, 1994; rogers and ellis, 1994; and lave, 1988 though there are major differences in lave's approach which i discussed in nardi, 1996b). hutchins' book cognition in the wild (1995) presents the most carefully worked out theory of distributed cognition with which i am familiar. while i provide a critique of distributed cognition, i acknowledge that it is a relatively new body of work that has not yet had time to ferment and mature. but its potential impact merits a close critical look. the points of similarity between distributed cognition and activity theory are (1) cognition/consciousness are mediated by tools, so the artificial is profoundly important to any theory of what it means to be human; (2) cognition/consciousness are social in nature and (3) the "functional system" of a person or group with a set of tools is an important unit of analysis. each approach differs in (1) what is meant by "cognition" (2) the attitude toward the potential for human development (3) the notion of objects (4) concepts of awareness. hutchins (1995) details the functioning of navigation aboard an amphibious helicopter transport in the u.s. navy. he studied how the sailors on board the ship used tools and human communication to navigate the ship. cognition in the wild is a tour de force ethnography, with beautifully detailed descriptions of the minutiae of navigation and the best discussion of how people use tools that i have ever seen. i will now look in more detail at the points of similarity between activity theory and distributed cognition. i will have to use the narrower "cognition" rather than "consciousness" here as that is what the distributed cognitivists deal with. distributed cognition and activity theory: connections cognition is mediated by tools hutchins has a real feel for tools (technical, rather than psychological tools, to use vygotsky's terms) and his descriptions of the use of nautical tools such as hoeys, alidades, fathometers, compasses, charts, and so forth are brilliant. through these rich descriptions, hutchins makes clear that cognition is mediated by tools. this finding is in complete concert with what vygotsky proposed in 1930. hutchins has done the ethnographic work to demonstrate—in the most vivid manner—the importance of tool mediation. like the activity theorists, hutchins understands that the crucial role of tool mediation in cognition means that cognition is embedded in the artificial. as hutchins says, "the environments of human thinking...are artificial through and through" (p. xvi). (see also leont'ev, 1981.) cognition is social in nature, embedded in practical human activity hutchins expends lavish ethnographic attention on the social arrangements by which the cognition involved in navigation gets done. he makes clear that cognition is a social affair involving nuanced communication, learning, interpersonal interaction. again, this insight is not new to hutchins but his ethnographic descriptions bring the point alive. but i would like to point out that vygotsky proposed a social theory of consciousness more than 60 years ago. vygotsky theorized, for example, that learning takes place in what he called the "zone of proximal development"; that is, the "space" of what a learner can accomplish with the help of a more experienced peer or teacher vs. what she can accomplish by herself. we learn much more rapidly under the tutelage of a more experienced person than solo. (it isn't merely a matter of rich paths between neurons. nor can we just open up the head and pour in the right representations as a cognitive scientist would have it.) activity theorists (and distributed cognitivists) understand that there is a whole process by which the teacher assesses what it is the learner knows or doesn't know; decides how to shape representations so the learner can grasp them; uses the physical world to embody the task (perhaps guiding the hand to trace a letter of the alphabet, for example); gives the learner exercises to practice the new task, and so forth. cognition is not fundamentally a matter of clever representation, or forming those neural pathways (though that certainly is a physical result of activity) 70 ajis vol 4 no. 1 but of a special kind of social interaction embedded in a practical activity, whether attending school or navigating a large ship. after vygotsky, activity theorists continued to stress the point that cognition is rooted in the social and practical. as luria (1971) noted, "cognitive processes are not independent and unchanging 'abilities'...; they are processes occurring in concrete, practical activity and are formed within the limit of this activity." leont'ev (1974) observed, "...a person's mental processes acquire a structure necessarily linked to sociohistorically formed means and modes, which are transmitted to him by other people through teamwork and social intercourse." this could have been the epigram for hutchins' chapter 6. hutchins reprises (without attribution) these basic activity theory points throughout the book as he describes how quartermasters learn as they work on a team, how tools are used in navigation, how the practical activity of navigation shapes cognition. hutchins' points resonate clearly with activity theory when he talks about "locating cognitive activity in context," (p. xiii), language as a mediating technology (p.300), a cognition that is "not entirely internal to individual persons" (p. 118), and "learning that happens in the doing " (p. 373). the functional system a key point hutchins makes in cognition in the wild is the importance of functional systems which he defines as "systems composed of a person in interaction with a tool" (p. xvi). his point is that these systems mean that what a person can do with a tool is profoundly different than what a person can do without the tool. to talk about the person without the tool—cognition without tools—is to make a huge mistake. this is the most telling point hutchins makes against the traditional cognitive science approach. while cognitive scientists would not deny the importance of tools, in practice their analyses are confined to abstract representations. simulating on a computer, or even taking seriously, how to count on your fingers, or the hug the teacher gives you, or the way you close your eyes when you are trying to retrieve a memory are not stunts a cognitivist would attempt. these actions are just not amenable to being reduced down to abstract representations. but such actions count among the tools of human cognition. (activity theory forces you to look much more broadly at the notion of "tools.") i don't know why squeezing your eyes shut helps when you're trying to remember something, but sometimes it does. the human need for encouragement means that you may keep at a task when you get the hug and give up in frustration when you don't. the hug is a real tool in getting the task done. counting on your fingers is a great way to learn simple arithmetic (even if you have to do it under the desk because the teacher is not a proper activity theorist). cognition depends on these tools in a very material and everyday way. hutchins gives many wonderful examples of functional systems on board the helicopter transport. he points out that such systems may be very temporary, coming together at a particular moment to solve a particular problem. again, this insight was provided earlier by activity theorists. and it has been elaborated beyond hutchins' discussion. the notion of a "functional organ" as the activity theorists refer to it, goes back to marx (and hegel), as zinchenko (1996) explains: from marx's point of view, the major property of any living system—whether individual or social—is its capability to create needed organs in the course of its growth and development. similar ideas were being developed by the russian physiologist ukhtomsky, who defined a dynamic, integral functional organ: "usually we associate the name 'organ' with the notion of something that has already formed, something static and constant. it is not necessarily so. any temporary combination of forces which is capable of attaining a definite end can be called an organ" (ukhtomsky, 1978, cited in zinchenko, 1996). zinchenko (1996) elaborates the notion of a functional organ building on the work of the russian physiologist bernshtein (1924) and the psychologist zaporozhets (1986). (see also leont'ev, 1981 and engestro'm, 1991 for discussion of functional organs.) distributed cognition and activity theory: disconnects as we have just seen, in many ways, distributed cognition and activity theory are singing the same song. but the human that lies beneath the surface of a distributed cognition account is not the human that activity theory speaks of. let's look at the ways in which distributed cognition veers radically away from activity theory. what leads to this divide is the insistence in distributed cognition that people and things are fundamentally the same, that the same language can be used to describe how people and things behave, that both are similar parts in a larger computational/cognitive system. this notion that people and things are the same plays out in many ways throughout hutchins' book. it is to these manifestations of the notion of the symmetry of people and things in the distributed cognition approach that we now turn. 71 ajis vol 4 no. 1 cognition, computation, task performance and the division of labor although an anthropologist, hutchins has been very influenced by cognitive science; indeed he states that he has written his book for cognitive scientists. while hutchins has much new to say to that community, at the same time he wishes to preserve their definition of cognition as computation. computation, as hutchins defines it, is "the propagation of representational state across a variety of media" (p. xvi). cognitive science has not paid special attention to different kinds of media, but the idea of propagating changing state across representations is squarely within the cognitive science view of the world. for hutchins, a person is a "medium" just as surely as a fathometer or chart is a medium. cognition takes place, at the highest level of abstraction, indiscriminately across media of all types. that hutchins believes this is evident in his statement that he seeks a "concept of computation that does not require a change of theory to cross the skin" (p. 117). in other words, what's cognitive for the hoey is cognitive for the human. to understand cognition we must understand how state is propagated across various media, be they people or tools. hutchins' theory of propagating representations has been a point of great confusion for me in reading his book. 1 cannot make out why he applies the term "cognition" to inanimate tools as well as to people. in activity theory, a tool mediates a relationship between a cognizing person and reality, but the tool does not in itself exhibit any cognition. the tool bears information but it does not think. the neuroscientists and philosophers speak of cognition and awareness as intimately related; for example, searle (a philosopher), defines consciousness as materially involving awareness, as we discussed earlier in the section on neuroscience (searle, 1996). (common sense concepts of cognition also posit awareness, e.g., my dictionary defines cognition as "the act or process of knowing including both awareness and judgment" (webster's ninth new collegiate dictionary)). flattening cognition to obliterate awareness and judgment seems to me to profoundly miscast what cognition is all about. the problem may be that hutchins confuses the cognitive and the functional. a tool functions; it does not think. further evidence for this confusion lies in hutchins' notion of distribution. he goes back and forth between descriptions of what individual people can do and the tasks that can be accomplished by the group, wanting all the while to call both "cognitive." group properties are different than individual properties he says. no one would argue with such a statement. whether you look at classical systems theory (wiener, 1948; ashby, 1956; bertalanffy, 1968) or work on the division of labor which makes clear that a society can accomplish things that an individual cannot (and hutchins himself cites this work), it is clear that a group functions differently—in terms of the work that can be accomplished—than a single individual. hutchins however wants to call this group function "cognitive." that seems obscurantist. where is the awareness and judgment in a fathometer? or in the system as a whole? hutchins presents no data that a system has awareness or judgment beyond that of the individuals in the system. the system can be organized to take maximal advantage of human awareness and judgment, and set things up so that complex tasks get done by leveraging that awareness and judgment, but it is obscure to assert that a social system, or the media within it (apart from people, if you insist on calling people media), are cognizing entities. a system can evolve over time such that human cognition is effectively leveraged, but that is not to say that the system itself cognizes anything. i think hutchins gets tangled up in the notion of assigning the term "cognitive" to a system because he so profoundly recognizes that large scale tasks like navigation involve the transmission and transformation of a huge amount of information across people and their tools. information is part of cognition, but it is not cognition. the practice of navigation itself does not "know" anything or cognize in any way, but it is ordered to take advantage of the cognitive contributions of individuals as they use their many tools. hutchins is aware, at some level, that his description of "cognition" is problematic. in chapter 8, "organizational learning," he attempts to demonstrate how the shipboard organization "learned" something that no individual team member learned. the learning was not achieved by "conscious reflection about the work" (p. 317) as he says; rather, a good solution to a problem on the ship was achieved by small adjustments to local conditions, rather than being designed by any one person. but this just tells us that sometimes such local adjustments work in certain situations. it does not tell us that the system exhibited cognition. what if there had been a disaster (as in other complex environments such as nuclear power plants) and the local adjustments had not worked? would we say the organization had failed to learn? we could, but i prefer the explanations of those such as perrow (1984) who point to specific instances of human error, badly designed tools, problematic power relations, and so forth in analyzing system failure. (see also linde, 1988 for a fascinating account of pilot error caused by communication breakdowns attributable to the pilots' status hierarchy.) it seems much less mystical to me to talk about real people making real mistakes with bad tools (and bad lines of communication, etc.) than to assert a cryptic level of system cognition. 72 ajis vol 4 no. 1 a great deal of hutchins1 book is devoted to how individuals learn and how they use information-bearing tools. when hutchins talks about individual "task performers" as he sometimes refers to the sailors on the ship, he's on solid ground. it is here that the rich descriptions of tool use shine. when he attempts to talk about the "cognition" in the other media, stretched across a socio-material system, or the "cognitive properties of human groups" (p. 176) the discussion is less lucid. the "cognitive properties of human groups" are nothing more than a division of labor and the functional interdependencies of the components of any system. hutchins again senses his own confusion and he begins, well into the book, (p. 176) to talk about "cognitive labor." (the plethora of terminology in the book is an indication of muddles in the models. hutchins keeps trying to invent new terms to describe what he's talking about, without saying what the distinctions between the terms might be.) what hutchins really means is that knowledge is distributed across the different people on the ship as is the information contained in the tools of the ship. knowledge and information are then used by various people in their own cognitive tasks. cognitive tasks are distributed in a social system—as are manual, social and emotional tasks. at times hutchins himself make this crucial distinction, e.g., when he speaks of "computational tasks [that] are socially distributed" (p. 185). here the book is fluent and coherent. that the system is organized to bring labor and information into coordination is undeniably true. and hutchins' wonderful contribution is that he details this process with great care, precision and clarity. but such coordination is not cognition. (in a review of cognition in the wild, latour (1996) calls coordination "one of the fetish words of the book"). this sentence from the book sums up the confusion: "these systems are simultaneously cognitive systems in their own rights and contexts for the cognition of the people who participate in them" (p. 288). but only the latter makes any sense. the system is not cognitive any more than it is manual or emotional; it is rather an organizing framework in which various kinds of labor—done by real humans—are accomplished in an organized way. the system itself does not exhibit cognition just as it does not perform manual labor or feel emotion or behave sociably. a fetish of long-standing in cognitive science has been that cognition can be wrenched away from body and soul and reified as something apart from the total person (such that it can be simulated on computers). and that cognition is different in kind than other human capabilities such as physical skill (remember skiing in deep powder?) or spirituality. hutchins makes this very mistake of reification in thinking that a system can be a cognitive entity because it can instantiate a cognition apart from body and soul. i don't think hutchins would suggest that a system performs manual labor or that it has a good time at parties or prays to its god(s). while hutchins yearns for one big flat system in which people do not have their pesky peculiarities and are really no different than any other "medium," he is nevertheless aware that this really isn't quite right. he smells the fishiness of a theory in which a person is not so different than a hoey (which, by the way, is a protractor like device). hutchins thus declares people a "special medium": "the thinker...is a very special medium that can provide coordination among many structured media—some internal, some external, some embodied in artifacts, some in ideas, an some in social relationships (p. 316). of course the whole game is blown if the thinker must be "special." we then do not have a theory that does not have to change when it "crosses the skin." at the end of the book hutchins takes cognitive science to task for ignoring perception, motor behavior, emotion, the body, history, context and culture. so he sees the complexity of real people. but the theory of propagating representations across media (animate and inanimate) could not possibly account for any of these things either. it's not just that hutchins was looking at something else, or that he'll get to emotion and so forth sooner or later; it's that a view of a person as a computing entity propagating representational state cannot find its way to a view of a person who knows the tango, or writes poetry or laughs at beavis and butthead. and it cannot find its way to a description of a cultural system in which such things are possible. scaling back the person what most disturbs me about the distributed cognition point of view is the idea that people are not too bright and that our tools exist to prop up our pathetic intellects. (how we are smart enough to create these tools to make up for our stupidity has not yet been addressed by distributed cognition theorists.) activity theory looks to the potential for development inherent in all of us and says that we start out needing a lot of help from other people to learn things, but then we acquire the psychological tools to do a lot of developing on our own. other people and artifacts continue to be important but there is a very real sense in which the individual bootstraps his way to greater and greater development (at least under certain historical conditions). thus activity theory has no problems with an albert einstein or a martha graham or a martin luther king. the distributed cognition perspective, by contrast, is chary of expert performance (and here is a real reaction to cognitive science which loves those smart puzzle solvers and brilliant chess masters). distributed cognition 73 ajis vol 4 no. 1 sees the world as being structured so that most tasks are pretty easy (see also lave, 1988; norman, 1991). hutchins details the simple tasks performed by the sailors and the way the tasks are broken down and organized so that nothing is terribly difficult. of course terms like "difficult" are extremely relative; to me the literacy of the sailors is a major cognitive achievement as are the social skills necessary to interact with their fellow sailors. hutchins does not agree with me on this point. he says, "...the cognitive abilities that navigation practitioners employ in their use of the forms and inscriptions are very mundane ones—abilities that are found in a thousand other task settings" (p. 133). he doesn't mention social skills a great deal, but after describing a fascinating non-verbal exchange between two sailors who negotiated a complex, novel alignment of tools, he merely remarks, "the social skills required to enter into shared task-performance relationships probably develop fairly early in life." this offhand comment implying that the social skills were not the result of a great deal of development dismisses the accomplishment of the two men. this kind of complex coordinated behavior is a threat to the distributed cognition approach because it is so clearly something that only humans can do. it means the theory has to change when it crosses the skin. to underscore the point that navigation is organized to be easy, hutchins describes how the use of algebra and arithmetic are avoided on board ship: "...tools and techniques permit the task performer to avoid algebraic reasoning and arithmetic...rather than amplify the cognitive abilities of the task performers...these tools transform the task the person has to do by representing it in a domain where the answer or the path to the solution is apparent...the existence of such a wide variety of specialized tools and techniques is evidence of a good deal of cultural elaboration directed toward avoiding algebraic reasoning and arithmetic..[the tasks] are part of a cultural process that tends to collect representations that permit tasks to be performed by means of simple cognitive processes" (p. 155, emphasis in original). i find this line of reasoning unsatisfying. algebra and arithmetic are human artifacts and lots of people understand and use them. ordinary people such as carpenters use the pythagorean theorem when calculating dimensions for stairs. insurance sales people use probabilities in figuring out what products their customers should buy. civil engineers design bridges using algebra. there is nothing beyond human capabilities about algebra or arithmetic. i would have been quite happy to see a cultural analysis in cognition in the wild in which the fact that many people who end up in the american armed forces do not have much formal education (or did badly in school) means that algebra is to be avoided. but there is no such analysis. rather, it is suggested that culture tends, in a general sense, toward making things simple. the very title of the book—cognition in the wild—asserts that what is at stake is a "natural" depiction of cognition as it "really is." not cognition in a particular culture, but cognition as it naturally occurs. others (e.g., brazerman, 1996) have pointed out that what happens in specific navigational tasks within the confines of an amphibious helicopter transport in the highly elaborated culture of the u.s. military is not necessarily what happens elsewhere. (see ratner, 1997 on the propensity for cognitive anthropologists to take the culture out of cognition.) while hutchins avoids making a blanket claim that tools do not amplify cognition, it is worth looking at an opposing view since hutchins does not dwell on expert performance. kaptelinin (1996), an activity theorist, responds to norman's (1991) argument about amplifying cognition vs transforming tasks (exactly the same argument hutchins makes, as described above). kaptelinin points out that, "tools not only change the task but often empower the individual even if the external tool is no longer used." he provides the example of a novice pool player who has to actually hit a ball with the cue to see where the ball will go, vs the expert who knows in advance where the ball will go and can plan a shot accordingly. the expert pool player has not substituted a simple task for a hard one; he has learned how to do a hard task through the practical activity of shooting a lot of pool. his cognition has truly been amplified (and so would say the marks of pool sharks who have learned the hard way, as memorialized in many american movies). citing rumelhart et al. (1986) hutchins goes on to enumerate the things he thinks "people are good at: recognizing patterns, modeling simple dynamics of the world and manipulating objects in the environment." this is entirely too simplistic, and says nothing about expert performance and the amazing and surprising things people do all the time. lave (1988) has a similar perspective in her description of people who she labels "just plain folks." (her book details how "just plain folks" avoid arithmetic when grocery shopping). sandberg and wielinga (1993) note the danger of reduction in the distributed cognition approach where "mind is a simple organism interacting with its environment and producing complex behavior through the application of simple behavioral rules." the inadvertent elitism in the distributed cognition stance is unfortunate; surely a hutchins or a lave would not regard their own work as "simple" or "mundane" (a word lave also uses) or merely a matter of doing tasks where a "solution is apparent." a theory of cognition in which some people are smart and some just slog through the easy tasks seems wrong to me. it is obvious that people have amply proved throughout human history that they are good at all kinds of things. some people make beautiful gold jewelry and some people tell fabulous stories and some write terrific code. 74 ajis vol 4 no. 1 none of these activities is possible without a great deal of development. and yet none of these activities is rare or bizarre. and people are always looking for new things to be good at. i recently saw a television show on "skysurfing" in which people train themselves to jump out of airplanes, and using a snowboard-like board, "surf" through the air (using the board as a wing) until the last possible second when they have to open a parachute. while i admit this isn't the stuff of everyday life, neither is it somehow not human. hutchins offers a discussion of difficult cognitive tasks that are not "representative" that he thinks researchers have paid too much attention to. he paraphrases d'andrade (an american anthropologist) who has noted that, "if we want to know about walking, studying people jumping as high as they can may not be the best approach" (p. 367). but an activity theorist would say you have to walk before you can jump and there is a developmental path that anyone might choose in going from walking to high jumping. it would be a matter of defining an "object" in the activity theory sense (wanting to be a high jumper), and then going for it, full bore. and if you couldn't walk, you couldn't high jump. they are utterly related to one another. high jumping does not have to be seen as "unrepresentative"; it is something a person might very well choose to pursue, if they already know how to walk and if they decide they want to do it. the view of culture as tasks-made-simple reaches a nadir in hutchins' analogy (borrowed from simon, 1981) between people and ants. extending simon's original analogy, hutchins suggests that what history means to an ant is that an ant occurring later in ant history can follow chemical trails left by its predecessors earlier in ant history and can then find food sources much more readily than the earlier ants. "is this a smart ant?" asks hutchins. "no, it is just the same dumb sort of ant, reacting to its environment in the same ways its ancestors did. but the environment is not the same...generations of ants have left their marks on the beach, and now a dumb ant has been made to appear smart through its simple interaction with the residua of the history of its ancestors' actions" (p. 169). the same dumb ant! please. it pains me deeply to hear a fellow human referred to this way, even in an analogy. the notion of a person "reacting to its environment" was what behaviorism was all about; we seem to be revisiting that sad view of humanity again. the analogy with the ant falls precisely into the trap sandberg and wielinga (1993) warn of: the person is made to seem a "simple organism interacting with its environment and producing complex behavior through the application of simple behavioral rules." but in the ant world, the chemical trails are automatic biological excreta of the ant; neither producing nor placing the trails involves any creativity or awareness or intelligence on the part of the ant. in the human world, tools need to be dreamed up, manufactured, distributed and explained to other people. there is no real analogy between biologically produced chemical trails and culturally produced and reproduced human tools, especially in the need people have to creatively design tools that will meet future needs. the question remains: how can people be so clever as to create a world in which they can be so dumb? hutchins does not ask how the tools that mediate task performance were invented. tools appear full blown, like venus on the half shell, as "crystallizations of knowledge and practice in the physical structure of artifacts" (p. 96). latour notes that "hutchins should have countered the objection that there must be a huge difference in applying routine knowledge and producing new knowledge" (latour, 1996). hutchins asserts that the whole social organization of navigation on board ship is dedicated to bringing the ship in safely. but again, the procedures and arrangements in the social organization must be devised and enforced by someone. even if a system did the right thing without any larger conscious agent designing a solution, someone must recognize a solution as useful and codify and enforce it in the future. otherwise such innovations will be one-time occurrences, not cultural accomplishments. hutchins uses the word "residua" to describe the accomplishments collected in a culture; it is an odd choice of words, suggesting a set of inadvertent by-products (as in the residua from a chemical process). by contrast, in talking about how a person encounters culture leont'ev (1974) observed, on a more positive note, "a person's activity assimilates the experience of humanity." the object of the game every game has an object. every murderer has a motive. one of leont'ev's principal achievements was pointing out that the very definition of an activity comes from its object—that to which the activity "always answers." in distributed cognition while there is a systemic goal, such as bringing the ship safely into harbor, there are no human motives. to introduce human motive would be to introduce conflict, contradiction, tension, ambiguity, dislocation, discoordination. it would be to admit that individuals count, or can count, for a lot in the system. it would mean that the person has to become an even more "special medium" because having a motive is not 75 aj1s vol 4 no. 1 something a thing has (though latour, 1993, who sees people and things as even more equivalent than hutchins does, gives it the old college try with his assertion that tools have goals "built into them" by their designers). the whole thrust of the distributed cognition approach is that transitions between media happen in a smoothly coordinated way. as halverson (1993) notes, distributed cognition accounts work best for "highly rationalized systems." hutchins does describe a couple cases of breakdowns in ship function, but they are cast as a depiction of how quickly the system gets back on its feet, how rapidly it damps out any dysfunction—not how raw open conflicts can escalate and profoundly change a system. but even if there were open conflict on the ship (billy budd or captain ahab, say), a distributed cognition account would have few tools for describing or analyzing what was happening. activity theory has a strong notion of contradiction, straight out of marx, and looks for movement and growth in systems as a partial result of conflict and tension (see kuutti, 1996). raiethel (1996) has expanded this notion to describe how organizations go through cycles of coordination (the level of smooth functioning hutchins describes) to cooperation (small disruptions in the organization that do not require major realignment) to co-construction (major breakdowns that require extensive communication and realignment within the organization). this is a much more expansive picture of how a system functions than confining analysis to the coordination level and proposing that coordination is the major modus operandi of a system—and then calling this "cognitive." raiethel (1996) does not call his levels "cognitive"; he is comfortable in describing system dynamics, at the level of the organization, and calling it just that. awareness again i have proposed that we follow philosophy and neuroscience and webster's in asserting that cognition has to do with awareness. hutchins has set out to do something else, but he does use the word cognition—a rather special word—to label the coordinated flow of media that he describes so richly. it would be more precise, in my opinion, to describe that phenomenon as information coordination (or some such), leaving "cognition" a larger space in which to range. hutchins' contribution is in drawing attention to the intricacies of information flow between people and tools, rather than in providing a general framework in which to conceptualize cognition. hutchins does not talk much about awareness, but when he does, it is uneasily. he notes that "...human institutions can be quite complex because they are composed of subsystems (persons) that are 'aware' in the sense of having representations of themselves and their relationships with their surroundings" (p. 350). the rhetoric here is telling: people are "subsystems" (a term he continues with throughout the long paragraph)—not flesh-and-blood people with active intellects. the term "aware" is placed carefully in quotes by hutchins. to make the propagating-representations theory work, people have to be "media" or "malleable media" or "subsystems" or "adaptive systems" (all terms hutchins uses at different times in the book for people) and even sometimes ants. hutchins also refers to the navigational team as a "computational machine" (p. 185). when we call people people, we are less likely to forget how messy and unpredictable and creative and surprising we are. media and systems are things—not people. a vision of people as things just makes it harder to figure out who we are. it makes it harder to design and evaluate technology too, if we forget about whose needs we are really addressing. the neuroscientists do not make the mistake of eliminating awareness or equating people and things. edelman's (1992) language, for example, at some points is very close to that of activity theory.2 he observes that to get started in the game of understanding consciousness, or mind, a few basic presuppositions are in order. first, things do not have minds. second, normal people have minds. third, beings with minds can refer to other beings or things. edelman calls the third proposition intentionality (after the german philosopher brentano). as edelman says, there must always be an "awareness of something;" "it always has an object" (italics in original). this is object in the second activity theory sense: that toward which activity is directed, around which activity is coordinated, and which will be crystallized in a final form when the activity is complete. " edelman proposes a strongly biological theory of mind, but at the same time he is aware that "no amount of neuroscientific data alone can explain thinking" and that social and cultural interaction are crucial. but his focus on biology inhibits him from going far enough to grasp the basic tenet of activity theory—that consciousness is quite literally manufactured through our interactions with others and with tools. that is the essential basis of mind, though the brain and its wonderfully complex structures provide the material substrate on which such interactions are stored, used, organized and reorganized. 76 ajis vol 4 no. 1 where next? activity theory provides, in my estimation, the best conceptual framework for a scientific study of consciousness. it has been fruitfully applied to many problems of human-computer interaction which materially involve consciousness (see nardi, 1996) and has potential for further growth and development. there is a strong recognition of the importance of culture in activity theory, though it has not yet been realized in a large body of empirical work (ratner, 1997). modern activity theorists such as zinchenko (1996) propose a spiritual dimension to humanity and describe in secular terms how we might talk about spirituality (seen as an aspect of human development). activity theory not does not fall into the cartesian trap as cognitive science does. coming out of a squarely materialist tradition, activity theory is quite prepared to go down to the level of the wetware, as the neuroscientists want to, as long as the importance of practical activity in the everyday world is a fundamental tenet of any theory of consciousness. activity theory is optimistic concerning human development, celebrating achievement and the social arrangements that are necessary for achievement and development. activity theory is set up to account for people engaged in the whole range of human activities, from those of great ordinariness to those of great genius. while i believe the neuroscientists are off a bit on the wrong track in assuming that the brain causes consciousness, they are asking the right questions, the big questions. they are ready to ski in deep powder—to look at the messy problems of ambition, joy, sorrow, free will—and they deserve tremendous credit for that. they understand that people are not things, that a fundamental feature of people is their intentionality. perhaps a marriage of neuroscience and activity theory will one day show that we can understand those 100 billion neurons if we come to understand the way everyday practical activity shapes human consciousness. in the meantime, it is exciting to live in a time when science is tackling the big questions and, i think, making some progress. all of the four traditions we have visited—neuroscience, cognitive science, activity theory and the distributed cognition approach—make substantial contributions to the investigation of cognition and consciousness. continuing the investigation will keep us busy for some time to come. a focus on consciousness, in particular, is especially likely to bear fruit as it is an expansive, grand, three-dimensional, somewhat vague concept, requiring analyses that include an account of culture, a sense of the individual as much more than an engine of representation, and notions of intentionality, spirituality and physicality. as nabokov (1980) put it, "the brain only continues the spine: the wick really goes through the whole length of the candle." acknowledgments i am indebted to yvonne rogers for a careful, critical reading of an earlier draft of the paper. thanks also to carl ratner for his helpful comments. errors and omissions are my own. references ashby, w. r. (1956) introduction to cybernetics, london: chapman and hall. bannon, l. (1990) a pilgrim's progress: from cognitive science to cooperative design. ai and society, 4, 259-275. bertalanffy, l. (1968) general system theory, new york: george braziller. bannon l. & b0dker s. (1991) beyond the interface: encountering artifacts in use. in: j. carroll (ed.): designing interaction: psychology at the human-computer interface, cambridge: cambridge university press, pp 227-253. bellamy, r. (1996) designing educational technology: computer-mediated change. in nardi, b., ed. context and consciousness: activity theory and human-computer interaction, cambridge: mit press. bertelsen, o. (1994) pitts' law as a design artifact: a paradigm case of theory in software design. proc. east-west human computer interaction conference, 2~6 august. st. petersburg, russia. vol. i, pp 37-43. blum, b. i. (1995) beyond programming. to a new era of design, new york: oxford university press. blumenthal, b. (1995) industrial design and activity theory: a new direction for designing computer-based artifacts. in blumenthal, b. et al., (eds). selected papers: human-computer interaction. 5th international conference, ewhci '95, moscow, russia, july 3-7. lecture notes in computer science. v. 1015. springer. b0dker, s. (1989) a human activity approach to user interfaces. human computer interaction, 4,3, 171195. 77 ajis vol 4 no. 1 b0dker, s. (1991) through the interface: a human activity approach to user interface design, hillsdale, nj: lawrence erlbaum. bourke, i., verenikina i., gould, e. (1993) interacting with proprietary software users: an application for activity theory? proc. east-west international conference on human-computer interaction, moscow, russia, 3-7 august moscow, v. 1, pp 219-225. brazerman, c. (1996) review of cognition in the wild. mind, culture and activity 3, 1. card, s., moran, t. and newell, a. (1983) the psychology of human computer interaction, hillsdale, nj: lawrence erlbaum associates. christiansen, e. (1996) tamed by a rose: computers as tools in human activity. in nardi, b., ed. context and consciousness: activity theory and human-computer interaction. cambridge: mit press. cohen, a., candland, k. & lee, e. (1995) the effect of a teacher-designed assessment tool on instructors' cognitive activity. proceedings interact'95, london: chapman & hall, pp 405-409. crick, f. (1993) the astonishing hypothesis: the scientific search for the soul. new york: touchstone. draper, s. (1993) activity theory: the new direction for hci? international journal of man-machine studies 38. edelman, g. (1992) bright air, brilliant fire, new york, basic books. engestrb'm, y. (1987) learning by expanding: an activity-theoretical approach to developmental research, helsinki: orienta-konsultit oy. engestrom, y. (1991) developmental work research: reconstructing expertise through expansive learning. in: m. i. nurminen and g. r. s. weir (eds.): human jobs and computer interfaces, amsterdam: north-holland. engestrom, y. & escalante, v. (1996) mundane tool or object of affection?: the rise and fall of the postal buddy. in nardi, b., ed. context and consciousness: activity theory and human-computer interaction, cambridge: mit press. gibson, j. j. (1979) the ecological approach to visual perception, boston: houghton mifflin. halverson, c. (1993) distributed cognition as a theoretical framework for hci. cogsci tech report # 94-03. san diego: university of california at san diego. holland, d., & reeves, j. (1996) activity theory and the view from somewhere: team perspectives on the intellectual work of programming. in nardi, b., ed. context and consciousness: activity theory and human-computer interaction, cambridge: mit press. hutchins, e. (1994) cognition in the wild, cambridge: mit press. imaz, m. & benyon, d. (1996) cognition in the workplace: integrating experientialism into activity theory. ecce-8: proceedings of the 8th european conference on cognitive ergonomics, t. r. g. green, j. j. can-as and c. warren (eds.) granada, spain: university of granada press. kaptelinin, v. (1993) item recognition in menu selection: the effect of practice. adjunct proc. intercht93 conference on human factors in computing systems. amsterdam 24-29 may, pp 183-184. kaptelinin, v. (1996) computer-mediated activity: functional organs in social and developmental contexts. in nardi, b., ed. context and consciousness: activity theory and human-computer interaction, cambridge: mit press. kaptelinin, v., kuutti, k., bannon, l. activity theory: basic concepts and applications. (1995) in blumenthal, b. et al. (eds.) human-computer interaction. 5th international conference, ewhci '95, moscow, russia, july 3-7, lecture notes in computer science. v. 1015. springer. kuutti, k. (1991) the concept of activity as a basic unit for cscw research. proceedings second european conference on cscw (ecscw'91), amsterdam: kluwer pp 249-264. kuutti, k. (1996) activity theory as a potential framework for human computer interaction research. in nardi, b., ed. context and consciousness: activity theory and human-computer interaction, cambridge: mit press. latour, b. (1993) on technical mediation. three talks prepared for" the messenger lectures on the evolution of civilization" ithaca, new york, cornell university, april. latour, b. (1996) review of cognition in the wild. mind, culture and activity 3, 1, 54-63. lave, j. (1988) cognition in practice. cambridge: cambridge university press. leont'ev, a. (1974) the problem of activity in psychology. soviet psychology 13, 2, 4-33. leont'ev, a. (1978) activity, consciousness, and personality. englewood cliffs, nj: prentice hall. leont'ev, a. (1981) problems of the development of mind, moscow: progress. linde, c. (1988) the quantitative study of communicative success: politeness and accidents in aviation discourse. language and society 17, pp 375-399. luria, a.r. (1972). the man with a shattered world, new york: basic books. 78 ajis vol 4 no. 1 nabokov, v. (1980) lectures on literature, new york: harcourt, brace, jovanovich. nardi, b., ed. (1996a) context and consciousness: activity theory and human-computer interaction. cambridge: mit press. nardi, b. (1996b) studying context: a comparison of activity theory, situated action models, and distributed cognition. in nardi, b., ed. context and consciousness: activity theory and human-computer interaction, cambridge: mit press. norman, d. (1991) cognitive artifacts, in j. carroll, ed. designing interaction: psychology at the human computer interface, new york: cambridge university press. perrow, c. (1984) normal accidents, new york: basic books. raeithel, a. (1992) activity theory as a foundation for design. in: floyd c. et al. (eds.) software development and reality construction, berlin: springer. raeithel, a. (1996) from coordinatedness to coordination via cooperation and co-construction. san diego: workshop on work and learning in transition, january. raeithel, a. and velichkovsky, b. (1996) joint attention and co-construction of tasks: new ways to foster user-designer collaboration. in nardi, b., ed. context and consciousness: activity theory and human-computer interaction, cambridge: mit press. ratner, c. (1997) cultural psychology and qualitative methodology: theoretical and empirical considerations, new york: plenum press. rogalski, j. (1994) analyzing distributed cooperation in dynamic environment management: the "distributed crew" in automatized cockpits. proceedings seventh european conference on cognitive ergonomics (ecce 7), bonn, germany, september 5-8, pp 187-199. rogers, y. and ellis, j. (1994) distributed cognition: an alternative framework for analysing and explaining collaborative working. journal of information technology 9,119-128. rubinshtein, s. l. (1940) osnovy obschei pshychologii [basics of general psychology], moscow: the state teaching-pedagogical publishers (uchpedgiz). (in russian) rumelhart, d. (1986) schemata and sequential thought processes in pdf models. in j. mcclelland, ed. parallel distributed processing, cambridge: mit press. saarelma, 0. (1993) new work new tools. information system development in the context of development of work activity. cec esprit project 6225 comic report oulu-2-2. oulu: university ofoulu. salomon, g. (1993) distributed cognitions: psychological and educational considerations. cambridge: cambridge university press. sandberg, j. & wielinga, r. (1993) situated cognition: a paradigm shift? jaie 3,2. schon, d. (1983) the reflective practitioner, new york: basic books. searle, j. (1995) the mystery of consciousness. new york review of books, november 2 and november 16. simon, h. (1981) the sciences of the artificial, cambridge, mit press. sjoberg, c. (1996) activities, voices, and arenas: participatory design in practice. linkoping studies in science and technology. dissertation no. 439. linkoping, sweden. tikhomirov, o. k. (1991) the psychological consequences of computerization. in: j. wertsch (ed.).the concept of activity in soviet psychology, armonk, ny: m. e. sharpe. vygotsky, l. s. (1930/1978) mind and society, cambridge: harvard university press. vygotsky, l.s. (1934/1986) thought and language, cambridge: mit press. wiener, n. (1948) cybernetics, new york: wiley. zaporozhets, a. v. (1986) zbrannye psihologicheskie trudy v 2-h t.t. [selected psychological works in 2 vols.] in v. v. davydov and v. p. zinchenko, eds. vol. 2, moscow: pedagogika. zinchenko, v. & munipov v. (1989) fundamentals of ergonomics, moscow: progress publishers. zinchenko, v. (1996) developing activity theory: the zone of proximal development and beyond. in nardi, b., ed. context and consciousness: activity theory and human-computer interaction. cambridge: mit press. 79 the australasian journal of information systems (ajis) began in 1993 and was based on the simple realisation that there was a need for an academic ‘outlet’ for is research in australia australasian journal of information systems volume 17 number 2 2012 161 storytelling as a qualitative method for is research: heralding the heroic and echoing the mythic julie e kendall rutgers university julie@thekendalls.org kenneth e kendall rutgers university ken@thekendalls.org abstract we all like a good story, but as researchers we too often dismiss a story as being tangential or superfluous. in this paper we review how early adventurous information systems researchers attempted to use organizational stories in their work, but in doing so may have used mere fragments of stories as an alternative method to gathering data for the design, development, or documentation of information systems. from a theoretical perspective formed by the use of the social construction of technology, we reexamine the purpose or functions of myths and relate these to the functions of stories (which can be experiential, explanatory, validating, and prescriptive) and illustrate how myths are similar to the stories (that are normative or descriptive, practical or philosophical) we hear in organizations today. we explore the elements of a story and show why researchers need to be aware of not just the purpose of the organizational story (through elaboration of the myth), but also the telling of the story (through vividness) and the order in which the story is told (through the use of episodes). a researcher gleans understanding through storytelling by reacting, matching, eliciting, and collaborating with organizational participants, but above all the researcher must appreciate the story as a whole. in this paper we develop a new research method based on storytelling. in doing so we aim to inspire is researchers to realize the depth and richness found in organizational stories and to appreciate and use stories in their own work. keywords: qualitative research methods; storytelling; social construction of technology; information systems research; narrative; myths; mythology; discourse analysis introduction as children we get our first exposure to stories as fables. these early cautionary tales are stories that convey a lesson to young people. they warn about the consequences of adopting certain values (like the greediness of king midas) or show by comparison that certain behavior will produce better results than other behavior (as in aesop’s fable of the “ants and the grasshopper”). some stories are so good, that we hear them told and retold throughout our lives. many organizational stories are like this too, taking on a life of their own. stories originate in the workplace, and many are shared with, and repeated by, coworkers. organizational stories, like myths and fables, are used for specific purposes. in the past, information systems (is) researchers have dismissed these stories or even treated stories as if they impede data gathering. mailto:julie@thekndalls.org mailto:ken@thekendalls.org australasian journal of information systems volume 17 number 2 2012 162 in this paper we explore early is research using stories and story fragments, reveal what we learn from observing how researchers in other disciplines use storytelling, and adopt the tenets of social construction of technology (scot) theory as a useful lens for understanding organizational members’ creation of shared meaning through is stories. realizing that organizational storytelling today is much like the telling of myths, we propose that researchers need to appreciate the story as a whole. we also assert that researchers need to identify the purpose, the telling, and the order in which the story is told. our goal is to inspire is researchers to realize the depth and richness found in organizational stories and to appreciate and use stories in their own work. early use of storytelling in is documentation we can trace interest in is and storytelling back to the days of mainframe computing. an early attempt that applied storytelling to solving information systems problems was proposed by kendall and losee (1986). they faced a legacy system with sketchy documentation and supplemented the manual with new, user-driven documentation they called folklore. their approach asked users to contribute their thoughts and memories to create the information system’s documentation (the result was not unlike the discussion and support forums like mactalk or microsoft internet explorer forums that we have today in support of current software and computer products). their contribution, however, was to provide a unique form or structure for user contributions. users were asked to contribute to one of four categories, which were based on those found in the webster’s new collegiate dictionary definition that folklore is “traditional customs, tales, sayings, or art forms preserved orally among a people,” (as quoted by kendall and losee, 1986). customs were stories about how people behaved. tales were stories in the first person, where the teller was often the hero. sayings were, according to kendall and losee, “distillations of tales,” mainly brief exclamations, and the category of art forms was relaxed to include diagrams and flow charts found on bulletin boards and in file cabinets. kendall and losee noted that customs and tales were easier to obtain than sayings and art forms. the folklore approach was later adapted for used on macintosh computers using a hypertext system to link parts of the documentation for software together. this computer-based system was called softlore and encouraged contributors to freely add material in each of the four categories that would help users understand the system better (kendall and kerola, 1994). storytelling in information requirements analysis the following review of literature shows that systems analysts and information systems researchers have long believed that stories obscured, rather than revealed, critical information. almost as a reflection of that perspective, interviewing evolved into a highly structured, almost ritualistic, data gathering approach. with that mindset, interviewers mistakenly discarded, overlooked, or downplayed the importance of stories, since they consumed valuable time. myers and newman (2007) point out in their examination of qualitative interviewing in information systems research published from 2001-2005, using six articles chosen at random from mis quarterly, information systems research, journal of ais, and information and organization for a total of 22 articles, that they found only two examples out of 22 that made mention of the types of questions posed and answers received. in other words even studies using the approach of qualitative interviews did not specifically mention what myers and newman asserted should be part of open ended questioning, i.e. “to focus on common, vividly-held events and stories,” (p. 17). to further underscore the point, neither of the two studies they cite (silva & backhouse, 2003 and shim, shin & nottingham, 2002) refers to elicitation of stories. while the shim et al. paper published a list of australasian journal of information systems volume 17 number 2 2012 163 questions used in interviewing; the authors do not make explicit mention of events or stories, asking instead about internet “experiences.” alvarez and urla (2002) noted “while analysts may be inclined to dismiss narratives as messy or uncodeable data, the insights they provide merit attention,” (p. 38). they continued, “our interest in stories was sparked by their frequent dismissal by the analyst. consultants often expressed frustration at storytelling and often dismissed it as trivial, irrelevant, or meaningless,” (p. 42). the scorn for stories was not limited to those in our field. as denning (2000) recalled, “i quickly found that i was living in an age when storytelling was suspect. scientists derided it. philosophers threatened to censor it. logicians had difficulty in depicting it. management theorists generally ignored it,”(p. xv). sabherwal and robey (1993), boland and schultz (1995), kendall and kendall, (1995), and later brown (1998), who were all working in information systems research, noticed there was valuable information in the stories told by organizational members. one problem with early research into stories was the focus on “data” gathering of fragments. the mindset that accompanied data gathering (before it slowly became information analysis) was to locate pieces of data, both physically in reports and memos, and orally in interviews. the aforementioned early authors tended to discuss fragments of narratives gleaned from interviews, rather than capturing an entire story. while they made a contribution by introducing stories into is research, they stopped short of treating the story as a whole. often stories are discovered as part of an interview where other questions are asked. moody, blanton, and cheney (1998) used a field study to compare interviewing techniques, using standard information requirements interviews with cognitive interviews. cognitive interviewing, a method they described in detail, was “found to be more effective and more efficient than standard interviewing techniques in eliciting episodic knowledge from reference librarians,” (p. 79). working in the same era, kelly and zak (1999) noted that even though narratives excel at helping organizational participants make sense of reality, they are not used nearly enough. although still working with fragments, researchers have started to examine narratives in terms of interaction. working in the domain of research into collaborative work systems, pentland (1999) argued for conscious use of narrative features (drawn from bruner, 1990) that go beyond the standard inclusion of “sequence in time” to include “focal actor or actors, identifiable narrative voice, and canonical or evaluative frame of reference in the study of collaborative work systems,” (pp. 2-3). pentland (1999) asserted, “to the extent that questions about collaborative systems are connected to questions about social interaction, however, narrative will be intrinsic to the phenomena of interest,” (p.6). more recently, alvarez and urla (2002) examined interview data to determine how stories function for participants in an erp implementation. they noted that interviewees: organized their experience, sought to persuade listeners, and conveyed information to analysts using stories, habitual, and hypothetical narratives in their interviews (p. 38). they wrote, “that is, in this approach both interviewer and client are engaged in creating the meaning of the questions and answers that constitute the narrative as they negotiate understanding through language,” (p. 40). fincham (2002) used stories taken from case study interviews to examine success and failure in systems development. in undertaking his study he noted about storytelling that, “the realm of social interpretation, however, has yet to be more broadly applied to systems or to the analysis of computer success and failure,” (p.5). australasian journal of information systems volume 17 number 2 2012 164 kuechler and vaishnavi (2006) used a series of experiments to determine the importance of including explicit goal-based information in narratives. their goal was to improve the capability of decision support systems in supporting rapid, ad hoc decisions about dynamic business situations. they recommended designing systems using narratives that state goal information (why an action is taken), actors (who), activities (what is being done), temporal information (when), and means-ends (strategic information) along with presentation of “retrieved narrative so that the information is made more comprehensible and memorable.” (p. 974). in summary, there are two observations that can be drawn from past research in is that used storytelling. firstly, the use of narrative in information systems research has been largely embedded in the context of studying it practitioners eliciting information requirements or in capturing stories post hoc about documentation and systems implementations. in other words, is researchers have predominantly used storytelling in a limited way to study aspects of the systems development life cycle. our intention in this paper is to broaden the use of storytelling as a legitimate and useful research approach for studying information systems as organic systems, holistically within organizations. the second observation is that, many of the stories that have been interpreted in is research have turned out to be mere fragments of stories. the intent or purpose of the story is not reported and the reader is not able to fully understand the importance of the story as presented in the research. we believe that many of the stories discussed in is research have been shared during the interviewing process yet the reader is not informed about whether this is an enduring story that others in the organization share and repeat, or whether it is an isolated story told in a single interview. isolated stories are welcome when the researcher is looking for facts, while enduring stories capture all aspects of the organization, from practical to philosophical and from descriptive to normative. let’s discuss and elaborate on these two observations in order. first, is researchers need to focus more on systems holistically rather than pieces of systems (holons). at one time it was acceptable to use stories to find data for part of a systems lifecycle. now the is researcher needs to look at the organization, its members, its interactions, its purpose, how it manages to survive, and what good it does for society and individuals. a holistic approach to is research is now possible and stories can help in this regard. to understand the importance of a holistic approach, it can be beneficial for us to first understand how the appreciation of stories by researchers outside of the is field has evolved. early use of storytelling was modest in purpose. for example, mitroff and kilmann (1975) proposed that storytelling was a new tool for organizational problem solving. they claimed that companies could not exist without stories and thus they set the stage for more meaningful storytelling research. weick and browning (1986) furthered research in storytelling by observing that as narration changes, organizational structures change. ready (2002) noted that storytelling can be used for building strategic competence and strengthening organizational character and silverman (2004) showed that storytelling was used in communicating organizational strategy. chreim (2005) observed that storytelling was useful in both continuing and changing organizational identity. in these later studies, the purpose of storytelling was clearly identified in a broader holistic context. our second observation concerns the fragmentation of stories as reported in is research. we suggest that the entire story is greater than the sum of its parts. we assert that by analyzing the entire story as told, retold, shared, and passed down from one person to another, researchers will gain a deeper, more meaningful understanding of the organization and its members. australasian journal of information systems volume 17 number 2 2012 165 in his retrospect and prospect article on is research, lee (2010) does a superb job of pointing out the importance of making a “major effort” to take a general systems approach such as that of von bertalanffy (1968) or using checkland’s systems thinking approach (checkland, 1999) for accomplishing future information systems research that would make systems concepts an essential “part of such an undertaking,” (p. 345). once again we can learn from scholars outside of the field of is. an early book by tichy (1997) noted that storytelling was useful in creating emotional energy and energizing others. in a review of organizational storytelling, james and minnis (2004) examined the role of stories in organizational culture; they looked at sense making and stories, as well as the managerial uses of stories. hodgett and deneulin (2009) observed that storytelling was useful in enriching human policy assessment. barker and gower (2010) championed “narrative paradigm theory” as a way to model storytelling as a complete organizational communication tool, using the diversity of the work environment to illustrate their argument. in these instances, the researchers searched for deeper meaning in enduring stories. in sum, our purpose here is to promote the use of storytelling as a research method for is researchers intent on analyzing and interpreting the social construction of technology (as discussed in the next section) concerning the use and management of is in organizations through the stories that are told by organizational members. unlike researchers who might use narrative analysis to study the narrative of a personal life story, a formal speech, a fragment of a story, or a printed text in order to study form and categories of the narrative (lieblich, tuval-mashiach, and zilber, (1998, p. 6), we specify the telling of a complete story as our unit of analysis (moen, 2006). adopting a perspective grounded in sociocultural theory and using narratives to facilitate educational research on classroom behavior, moen commented on the narrative as a unit of analysis, “narratives are not broken into elements; they are neither reductionistic nor static,” (moen, 2006, p. 4). she concludes her study by stating, “i reiterate that narratives retain all the characteristics of the whole and that they occur as an integrated and living part of the whole. when appearing as a whole rather than elements, narratives are not abstract, remote, or inaccessible. instead, they can rather be perceived as familiar, informative, and relevant for those who hear about or read them,” (moen, 2006, p. 9). we propose storytelling as a qualitative research method. our argument is supported by boje (1991) who expanded on the importance of considering the text in conjunction with the telling or performing of a complete story: the focus in traditional organization story research has been on texts … these studies ignore performance and streamline the stories or treat them as so many variables that can give empirical explanations of the organization. the story telling organization theory posits story text and performance as two sides of the same coin and gives us insight into the complex and varied ways organization members use storytelling in their work world, (p. 125). therefore, in our approach to using storytelling as a qualitative method, we elicit a complete story as identified by both moen (2006) and boje (1991). in developing the research method, elements of a story that a researcher will be alert for are provided later in this article. those elements can guide the researcher in capturing the telling of a complete story. the qualitative researcher who uses this approach is interested in the text of the story, the order in which the events are told, and the telling of the entire story by the organizational participant. this approach is distinct as it moves away from studying the text alone as previously suggested by kendall, kendall, and kah (2007). these authors used dramatistic analysis to study the text of listserv messages rather than using the telling or performance of the story. australasian journal of information systems volume 17 number 2 2012 166 conceptual framework the storytelling method we develop fits well within the interpretivist tradition as characterized by creswell (2007), gioia and pitre (1990), kaplan and maxwell (1994), klein and myers (1999), and walsham (1993). gioia and pitre (1990) noted, “the interpretive researcher collects data that are relevant to the informants and attempts to preserve their unique representations.” they went on to add that analysis, the generation of theory and further data collection go hand in hand (p. 588). additionally, creswell (2007) stated in his account of how interpretive researchers proceed with their work by focusing on the “multiple perspectives of stories and who tells the stories,” (p. 24). in this paper we critically examine stories through the lens of the social construction of technology (pinch & bijker, 1984) framework to understand how organizational stories are created among organizational participants and information systems researchers. we illustrate how is researchers can identify how stories are functioning for organizational members. examples of relevant organizational stories include descriptions of corporate culture and the use of information systems; revelations about how it problems are solved; articulation of company values toward employees as well as toward adoption of new technologies; and promoting the image of the company as technologically ground breaking or the company’s leader as technologically advanced. we chose the social construction of technology (scot) as our lens because it is highly compatible with the purposes of the interpretive research we are conducting. as myers (1997, updated november 12, 2011) noted, interpretive methods of research are “aimed at producing an understanding of the context of the information system, and the process whereby the information system influences and is influenced by the context”(walsham 1993, pp. 4-5 as quoted by myers). we believe the lens of the social construction of technology has the power to help us view and even magnify the process of technology adoption through the elicitation and examination of stories. in addition, scot is widely known, used, and understood by our intended audience, is researchers. this type of ready recognition in the is community furnishes many shorthand associations for those attempting to learn our new methodology. indeed, while oft criticized, the social construction of technology approach still provides one of the dominant approaches to studying technological innovation (“social construction of technology,” n.d., para.1). the social construction of technology (pinch & bijker, 1984) framework is useful in understanding how the interplay of individuals, groups, and technology create shared meaning (in our case, stories) about technology processes and artifacts. the last 30 years or so have seen the development and use of the social construction of technology framework (mackenzie and wajcman, 1998; misa, feenberg, and brey, 2003; moisander and eriksson, 2006; pinch and bijker, 1984; and shin, 2006). in basic terms, the social construction of technology (scot) proposes that technological innovations are constructed socially (negotiated) through human interactions taking place among individuals, organizations, and society. the four principles of the social construction of technology (as paraphrased from pinch and bijker, 1984) are: 1) interpretive flexibility, 2) relevant social groups, 3) closure and stabilization, and 4) technological frames, all of which we elaborate on in an upcoming section. the is researcher using scot looks for the negotiated meanings of technological innovations created by social processes. in our case, as is researchers, scot is useful because it resonates with the interpretive research stance adopted here, and also it is widely recognized as a legitimate way to conceptualize technological innovation throughout the is research community. the social construction of technology theory reminds us that information systems are usefully conceived as a coconstruction of meaning that result in a variety of is artifacts, acceptable practices, actions to be http://www.qual.auckland.ac.nz/interp.aspx#walsham,%20g.%20interpreting australasian journal of information systems volume 17 number 2 2012 167 taken, and alternative frames for sociological and technological thinking (“social construction of technology,” n.d., para.3). the first principle of social construction of technology is that of interpretive flexibility, which visualizes technology design as an open process that results from negotiations within the group. according to shin (2006, p. 86), when groups negotiate among themselves, different results are possible, depending on the social circumstances of development. the second principle of social construction of technology says that meaning exists in the relevant social groups who create and abide by the same meaning of an artifact, story, or myth, by virtue of the fact that they exist in a shared social grouping. since stories are often passed down person to person and even generation to generation, in this paper we focus on the shared social groupings implied by users and managers of it within an organization and the meaning they impart to stories they tell and re-tell about technological artifacts. indeed, the story itself can be considered an artifact, and a unit of analysis. in the social construction of reality framework, features of closure and stabilization characterize the third element. how can inevitable conflicts within groups be resolved? this third principle of social construction of technology emphasizes the resolution of is design conflicts. the end of a design phase is signaled when no social group brings up a problem with an artifact; i.e. the technological meaning is no longer the center of discussion. following the phase of closure, the technological design can be characterized as frozen or stabilized, and the participants agree that the artifact, and the story about the artifact are in their final form (shin 2006, p. 86). in the fourth and final element of the social construction of reality framework we are working with technological frames. in the case of our organizational stories, we assume that this component reflects the shared exchanges or interactions of the group as they communicate with each other and the information systems researcher about the stories’ widespread applicability. scot informs our study by reinforcing the notion of conducting interpretivist research in is. since the first principle of scot is “interpretive flexibility” our storytelling method utilizes the principle of technology design as an open process that arises from negotiated meanings within a group. we leverage the second principle of scot , which is that of relevant social groups, in our storytelling method by interviewing shared social groupings comprised of users and managers of it within a company. we use the third principle of scot, that of closure and stabilization, to understand where a story is in its life cycle; thereby identifying when an innovative technological artifact and the story about an artifact are in their final forms. technological frames are the focus of the fourth and final principle of the social construction of technology approach. the is researcher using storytelling from a scot perspective is actively seeking the technological relevance of stories being told. for is researchers, the four principles of scot encompass clear concepts to explore the questions of how stories of it managers and users create shared meaning of information systems in a social context. without scot, the is researcher who is fortunate enough to identify a story may still be perplexed by it, with few ways to begin interpreting it. the theory of the social construction of technology helps us build our method of storytelling as an interpretive, qualitative approach that can be widely understood. australasian journal of information systems volume 17 number 2 2012 168 why storytelling differs from narrative analysis a typology of myths as being useful in organizations in this paper we draw much of our inspiration from the use of myths. myths simply are stories (the word mythos is taken to mean plot by aristotle (doty 2004, p. 1)). over time myths have accumulated and have also been passed on from generation to generation. mythology is a repository for our knowledge in story form. while there are many instances where mis uses the term myth to refer to untruths (denning, 2004; grover & ramanlal, 1999; mintzberg, 1972; powers & dickson, 1973; seffah & metzker, 2004; and vergin & grimes, 1964), we return to the classic meaning of the term myth. we agree with doty (2004), who objects to what is seen as a misappropriation of the word myth. on the other hand, myths are complex narratives, not simple folktales or song-texts, by which human cultures stabilize and encode their deepest ambiguities" (robinson 1993: 136). they are "deep stories" of a culture (wagner and lundeen 1998: 3) that reflect the human tendency to organize its representation of the past, present, and future according to specific emphases and moral highlights. (doty 2004, p 13) the classical concept of the myth is critical to the development and enrichment of the storytelling method we are proposing. myths are stories that are deep and enduring. myths represent the shared belief of people within a society (or in our case, members within an organization). myths can address the past, the present, or the future by telling the listener how things have been (levy, 1994), why we behave a certain way (belk & tumbat, 2005), how to behave (whelan & o’gorman, 2007), or even what happens if we fail to act in a certain way (livingston, 1965). stories take on mythic qualities when they represent a shared vision, are carefully retold, and passed to the next generation. over the years, the authors have heard many mythic stories when they visited organizations. we assume therefore that knowledge of myths can help us in the development of the storytelling method contained in this article. joseph campbell has written extensively on the use, meaning, and purpose of myths (campbell, 1964) and discussed in detail four functions of myth. the four functions were summarized succinctly by young (2004). they are: 1) a metaphysical function, 2) a cosmological function, 3) a sociological function, and 4) a psychological function. the metaphysical function describes awakening and experiencing the awe of the universe. the cosmological function explains how the universe is organized. the sociological function validates and maintains the social order. the psychological function turns to the individual and directs how an individual should behave. occasionally this fourth function appears as the pedagogical function. we find campbell’s four functions to be a good starting point in the development of a typology of organizational stories. we observe that campbell’s four functions of myth can be said to correspond to storytelling in organizations and identify four functions of common stories as follows: 1. the experiential function. stories that describe experiencing what the organizational universe is like. 2. the explanatory function. stories that attempt to explain the organizational universe. 3. the validating function. stories that try to maintain the organizational value structure. 4. the prescriptive function. stories that recommend correct and proper behavior in the organization. australasian journal of information systems volume 17 number 2 2012 169 table 1 compares campbell’s functions of myth with storytelling functions that we based on campbell’s work. the concepts are similar but the researcher should be able to relate more with the storytelling functions we’ve identified. campbell’s (1964) myth functions young’s (2004) description of each myth function storytelling functions defined here description of the story as it relates to modern organizations metaphysical describes awaking and experiencing the awe of the universe experiential stories that describe experiencing what the organizational universe is like. cosmological explains how the universe is organized explanatory stories that attempt to explain the organizational universe. sociological validates and maintains the social order validating stories that try to maintain the organizational value structure. psychological (pedagogical) directs how an individual should behave prescriptive stories that recommend correct and proper behavior in the organization. table 1. how campbell’s functions of myths relate to the functions of storytelling the first function of organizational storytelling, the experiential function, examines experiencing the business universe. these stories retell the observations of the storyteller, who describes what the business world is all about. the emphasis is on experiencing the organization, not in trying to understand it. the storyteller doesn’t try to explain why something happens; rather they tell a tale that describes how things unfold. the ability of the group of people coming together to create something is told but not questioned. qualitative researchers understand this concept. examples of the experiential function shown in table 2 use stories to describe corporate behavior (smillie, 2010), use stories to enhance camaraderie (berman, 1995), and use stories to encourage social inclusion (gillard, mitev, and scott , 2007). the explanatory function, the second function of organizational storytelling, looks a little deeper into how an organization works. examples of the explanatory function are stories that reveal how problems were solved (vlachoutsicos, 2011), stories that analyze culture and behavior (mascarenhas et al., 2010), and stories that explain why certain decisions were made (frohman, 2006). the third function of organizational storytelling is the validating function. these stories are told to support the organization, its structures, and its decisions. validating stories seek to maintain the existing value structure. in validating the way the organization behaves, they seek to preserve the organization well into the future. the main difference between the third function and the previous two is that the validating function is normative, while both the experiential and exploratory functions are descriptive. examples of the validating function include stories that articulate company values (mainiero, 1994), stories that manage change or even slow it down (macaulay, yue, & thurlow, 2010), and stories that extend corporate knowledge (stewart & kaufman, 1995). australasian journal of information systems volume 17 number 2 2012 170 storytelling function common stories found in organizations example found in the literature describing corporate culture smillie (2010) experiential enhancing team camaraderie berman (1995) encouraging social inclusion gillard, mitev, and scott (2007) revealing how problems are solved vlachoutsicos (2011) explanatory analyzing culture and behavior mascarenhas et al. (2010) explaining how decisions are made frohman (2006) articulating company values mainiero (1994) validating managing or slowing down change macaulay, yue, & thurlow (2010) extending corporate knowledge stewart and kaufman (1995) educating future decision makers désilets (2008) prescriptive changing or controlling behavior higgs and rowland (2010) promoting image of the company hanson (2004) table 2. common stories found in organizations fall into one of four functions the prescriptive function, the fourth function of organizational storytelling, regulates how individuals and the company as a whole should behave. the storyteller directs the listener and might go so far as to prescribe certain activities that must be done in the future. occasionally this story could be told as a cautionary tale, warning of dire consequences if an individual doesn’t act a certain way, or it can be told as a simple parable. in any case, this type of storytelling is normative. examples of the prescriptive function in an organizational setting are stories that educate future decision makers or leaders (désilets, 2008), stories that help in changing or controlling behavior (higgs & rowland, 2010), and stories that promote the image of the company or leader (hanson, 2004). figure 1 depicts the differences among the four functions served by organizational stories by separating the experiential, explanatory, validating, and prescriptive functions into one of four quadrants. they are drawn as existing on an axis that runs from practical to philosophical (top to bottom) and from descriptive to normative (right to left). on the practical side of the continuum we hear stories recommending actions such as educating, describing, and controlling, while on the philosophical side we hear stories that explain, analyze, reveal, and articulate concepts rather than actions. on the normative side of the continuum, we hear stories that tell listeners how to act, while on the descriptive side we hear stories that describe how things are. notice that stories performing an experiential function for their tellers are thus categorized as most practical and most descriptive, while stories that serve the explanatory function are descriptive and philosophical. stories serving a validating function are categorized as normative and philosophical, while in the fourth quadrant, stories serving the prescriptive function are described as practical and normative. the benefit of developing and sharing this typology with researchers is that the researcher who is familiar with the typology is more likely to understand the context from which the stories emerge. contextualization is a key principle in interpretive field research. australasian journal of information systems volume 17 number 2 2012 171 based on our work in organizations, we recognize that most organizational stories feature a hero wherein the storyteller often identifies herself or himself as the hero or protagonist who experiences a struggle only to come up with a positive solution to a problem. there are, however, occasional negative stories that have the potential to do damage and are persistent. some examples of negative stories are stories that make the mythmaker or storyteller the center of the business universe (experiential), stories that exclude or stereotype people or groups (explanatory), stories that encourage behavior that deviates from the norm (validating), and stories that attempt to negatively modify behavior (prescriptive). figure 1. organizational stories serve one of four functions (experiential, explanatory, validating, or prescriptive). the four functions of storytelling distinguish it from other information gathering methods. a key practice for is researchers using the story method is to identify the function a story is serving for organizational participants. analysis progresses as the function of a story is identified. understanding the four functions of storytelling is useful to the researcher for many reasons. first it helps the researcher put stories into context. functional context is just as important as historical or social context. second, knowledge of storytelling functions reminds the researcher that they themselves may also harbor preconceived notions about stories as they are told. third, understanding the functions of storytelling can explain why the storyteller often describes himself or herself as the hero of the story. fourth, understanding storytelling functions can explain why different participants tell the same story in different ways. australasian journal of information systems volume 17 number 2 2012 172 the storytelling method encourages the researcher to revise their initial understanding of the informing narrative based on several stories collected from participants. when researching why information systems fail, the is researcher using this storytelling methodology may be drawn to interview participants in an organization with a high rate of system failure based on a preconception that several new systems implemented in the organization have failed due to resistance to change in the organization. after collecting several stories from participants, the researcher may revise their conception repeatedly thus revising their main impression in the process. for example, one person’s narrative may feature a story that makes the operations department the hero because of their championing of new technology, specifically their use of supply chain management applied to services; another may share a narrative that they are convinced that a new is will radically alter the company’s external relationships, while a third person’s story may highlight fears associated with implementing a new supply chain management system too rapidly. in response to these findings in the field, the researcher using the storytelling method may revise their central concept, moving to examine alignment of new information systems with overall organizational strategic goals, and in this way revising their initial, narrower theoretical conception of resistance to change. the elements of a story researchers who embrace storytelling as a research method also need to understand the basic elements of a story. the following is a guide that explains commonalities found in complex stories. it lists 18 elements, each followed by a description. the 18 elements were compiled by the authors from their experiences in organizations, observations about stories and plots in theatre productions, and a continuing discussion about stories and myths with playwrights they have met. 1 common elements of stories include: 1. mythical quality. stories that have an enduring quality have a larger meaning. they take on a mythic quality. 2. call to adventure. every story has a call to adventure. this can be a crisis, a trigger, or an opportunity. 3. the hero. every mythical story has a protagonist. 4. the goal is revealed. a hero is assigned a quest. 5. hero’s inner problem surfaces. a complex story reveals both a hero’s outer and inner problems. outer problems are observable challenges and obstacles obvious to all of us. inner problems describe challenges to the hero such as learning about oneself. 6. a new situation arises. a complex story will describe the hero leaving their normal world and entering a new world that may have different rules, expectations, and timetables. 7. the hero is tested. along the way, the hero will be challenged. 8. enemies or forces of nature intervene. the hero may encounter enemies or forces of nature that need to be overcome. 9. hero encounters new obstacles. in addition, certain obstacles may need to be overcome as well. 10. hero experiences a transformation. the hero may experience an ordeal, may receive a brilliant idea, and may even experience a change along the way. 11. hero experiences a setback. the story may detail a series of failures along the way. 12. hero may be deceived. the hero may experience deception on their journey. 13. the situation is resolved. at some point there is a resolution. either the hero fails (and metaphorically dies) or the hero succeeds. 14. lessons are learned. in either case, there are lessons learned. 15. better problem understanding. the hero or someone else now understands the problem better and will hopefully succeed (especially if the hero experiences failure). australasian journal of information systems volume 17 number 2 2012 173 16. quest is easier next time. in the future, the hero or someone else might encounter a similar problem but will have an easier time succeeding. 17. celebration time. often the story ends with recognition and joy. 18. epilogue. sometimes a story ends with an epilogue where the storyteller will explain what has happened since the problem was resolved. examples of each of the above elements are given in table 3 later on in this paper. there are numerous reasons that stories linger in the memories of those who hear them. human beings are equipped with special capacities to recall stories, read into them in personally meaningful ways, and efficiently store information in the form of episodes that can be recalled later. stories involving organizations can be just as memorable as myths and legends because organizational stories have similar characteristics to myths. there are three mechanisms that distinguish storytelling from narratives. researchers swap, leonard, shields, and abrams (2001) suggest that these three underlying cognitive mechanisms aid people in remembering the stories have heard earlier to then use them in appropriate situations. the three cognitive mechanisms are 1) the availability heuristic (also called the vividness effect), 2) elaboration, and 3) episodic memory. they are important because they provide the abstractions needed so that the researcher interprets the story as a whole, rather than trying to interpret the 18 elements. in the availability heuristic, listeners hear a story that is memorable because it is told vividly via a story that they remember. in the elaboration mechanism, the listener is given enough detail so that the story is meaningful to them in their own experience. they can elaborate the story from their own background, and thus have an easier time remembering it. episodic memory refers to the idea that it is easier for listeners to recall information organized in a story or episode than it is to try and remember a list of items without the context provided. the elements of a story we outlined above exhibit all three of these mechanisms. first it is complex. each of the 18 elements makes a unique point that is similar to a myth or fable the listener is familiar with (vividness). next, the story is told in such a way, that the listener can empathize, or even identify with, the hero (elaboration). finally, it is complex enough to have 18 scenes or episodes that are witnessed in an unmistakable and crucial order (episodic). so in using storytelling as a research method, the researcher needs to appreciate the entire story and to be aware of the telling (through vividness), the purpose (through elaboration of the myth), and the order (through the use of episodes). these three mechanisms emphasize why using storytelling as a research method differs from other types of narrative analysis that focus on the analysis of transcribed interviews, or on the examination of reports, blogs, email, and other text found in organizations. an illustration of how a storyline is used in an organizational setting is found in table 3. the story elements are mapped to events and plot points from a situation that occurred from a 5.9 magnitude earthquake near whittier, california in 1987. this story appeared in an abbreviated form in an article about disaster recovery planning for an organization’s software, data, and systems (kendall, kendall, and lee, 2005). the authors recommended that organizations adopt a workshopping metaphor from evolving theatre practices. in this article we break down the story about an earthquake, a heroic it team leader, and a plot involving atms (an abbreviation for automated teller machines; also known as cash points) into segments and match them to the elements that are common to all stories. table 3 provides a roadmap for the interpreter to follow, but if each element was taken alone, it would not be representative of the story as a whole. here is where the three mechanisms come in. they provide the abstraction needed so that the interpreter can understand the entire story. the mechanism of vividness provides realism and a comparison to other experiences the researcher has had, the mechanism of elaboration helps identify the function so that the researcher can better understand the australasian journal of information systems volume 17 number 2 2012 174 purpose of the story, and the mechanism of episodes helps the researcher understand the order in which the story is revealed. story element example from whittier earthquake atm problem mythical quality our company is a family so we work together. call to adventure a 5.9 magnitude earthquake struck whittier, east of los angeles on october 1, 1987. the hero the it department came to the rescue. the goal is revealed the atm machines in the area are down and need to be restored. hero’s inner problem surfaces the team leader never worked under such time pressures. a new situation arises the team had to evacuate the building. the hero is tested the phone lines were down. how could he communicate? enemies or forces of nature intervene other banks would steal customers if atm machines were not restored. hero encounters new obstacles the power lines went down as well. hero experiences a transformation the team leader realized that an employee had a cell phone (rare in 1988). coordinated and communicated with others. hero experiences a setback no one was allowed to enter the building. hero may be deceived (did nor occur in this story.) the situation is resolved at the last minute (as payday arrived) power was restored and access to the building was allowed. lessons are learned store data off-site. appoint an it team member to examine disaster scenarios, organizational resources, forecasts, and the interconnectedness among them. better problem understanding the it staff realized that it does not control everything at all times. quest is easier next time anticipating problems and planning for them will help the coordination of the recovery efforts. celebration time atms were restored in time for friday payday. it is unlikely that the bank lost customers. epilogue new regulations stipulate that banks must store data at off-site locations at least 100 miles away. table 3. storyline example from the 1987 whittier narrows earthquake. using storytelling as a research method developing the method when using storytelling as an approach, the is researcher might identify stories through informal interactions, or identify stories appearing in interviews or corporate literature such as websites or advertising campaigns. unlike interviews, that customarily take place in offices and meeting rooms, the process of obtaining stories can also take place in cafeterias, parking lots, warehouses, and shipping docks. these stories can be documented and can serve as the basis for reacting, matching, or eliciting performances of stories, or collaborating on a story with organizational participants. these activities form the basis for eliciting information in the proposed new method. recall that when australasian journal of information systems volume 17 number 2 2012 175 storytelling is used as a qualitative method, the is researcher is using the story as the unit of analysis (moen, 2006, p.2). interaction between the researcher and the participants will have an effect on the data that the researcher obtains and the storytelling method recognizes that the participants are in many cases both storytellers and interpreters while the researchers are sometimes co-contributors to stories as well as interpreters. in seeking stories, the researchers want more than a description of a procedure or a description of how a problem was solved. they are therefore required to interact with the participant so that they can obtain richer, more meaningful stories. four interactive story activities researchers can be involved in, include reacting, matching, eliciting, and collaborating. reacting means the is researcher is requesting reactions to events or data learned through other methods. for example, the researcher might say, “now that you, as a member of the company, have seen this data, what is a story you would tell to respond to this event?” matching means that the researcher asks for a “matching” or “bookend” story that mirrors a story or incident provided. for example, the researcher might say, “here is a story about the push for more and more advanced use of it in a company. please tell a story that reveals your own reaction to the significance of increasing it budgets on an annual basis.” eliciting means that the researcher is consciously communicating with an organizational member to request information in the structure of a story. for example, the researcher might say, “tell me a story about the time you created a workaround to get your work done when the system went down and you were trying to meet a deadline.” collaborating with organizational participants in the creation of a meaningful story is a distinct aspect of storytelling as a qualitative research method in that it demands that the researcher be functioning on a very high level of proficiency, sophistication, and honesty and that they can contribute both technical and soft skills to collaboration efforts. take for example the story about the whittier narrows earthquake that appears in table 3. the researcher could participate in the research because they knew facts about the earthquake. in this instance, the story became richer than if it were simply told by a single storyteller. the interaction described above will undoubtedly have an effect on the story being told, but we already know that any story will change to some degree depending on the audience. even a carefully structured story such as the agony and ecstasy of steve jobs, told by the author monologist mike daisey, changes from night to night depending on the reaction of the attendees (personal discussion with mike daisey, 23 october, 2011, new york city, usa). the researcher must therefore assume that the storyteller will tell a story a certain way while in front of the researcher. in order to better understand how a story is told, the researcher needs to be aware of and apply the above four interactive story activities, not only to obtain richer and deeper stories, but also to be more cognizant of the effects of the interaction between the participant and researcher. the first two interactive story activities can be used to guide the participant into telling a story on a particular topic and have less effect on the stories than the other two activities. eliciting, the third story activity seeks more details from the participant and is therefore more intrusive. johnson (2008) gives a prime example of eliciting a story, when she wrote about storytelling and software testing, noting she asks software testers that she hires: have you seen bugs you can’t reproduce? do you think there are issues still lurking in this product? are there bugs you haven’t reported because you know the bugs that cannot be reproduced will be disregarded? give me your impressions. i often ask the testers i hire – i ask direct hires, contractors australasian journal of information systems volume 17 number 2 2012 176 and outsourced teams that i’ve managed – what are your impressions. i guess what i’ve been asking for is, tell me a story. tell me about your experience with the software. eliciting will obtain information on the specific details the researcher is looking for, but caution is advised as the story must change dramatically to include what the storyteller thinks are sub-plots. collaborative storytelling places the researcher in a creative partnership with the organizational participants and therefore is the activity that involves the most interaction. often the activity of storytelling can be accomplished orally in a group, team, or panel setting rather than one-on-one. in this way, the participants are free to do what is called “fantasy theme chaining” (bormann, 1972), building on each other’s comments to collectively create a hero, place blame on a villain, solve problems, or resolve the tensions of situations encountered with information systems. 2 how storytelling differs from other qualitative approaches it is useful to examine how storytelling is distinct from other types of familiar qualitative approaches in information systems research. while researchers may initially be attracted by similarities between storytelling and other methods such as narratives, interviewing, discourse analysis, observation, and therefore find it appealing, it is also wise to become cognizant of the distinct facets of storytelling as a method in its own right. in this section we briefly compare and contrast interviewing, discourse analysis, metaphors, and observation with the storytelling approach proposed here, surfacing some advantages of the storytelling method in contrast to more familiar narrative approaches. denning (2000) observes “the narrative way of thinking is internal and immersive and self-forgetting and attached to the full richness of tacit understanding. through a story, life invites us to come inside as a participant,” (p. 70). interviewing is a mainstay for data gathering in information systems research (schultze and avital, 2011). the investigative stance or perspective of the is researcher is typically one of elicitation: seeking opinions, feelings, organizational and personal goals, and informal procedures. these are often fragments of stories, not complete stories. the researcher also needs to draw out opinions about human-computer interactions and ergonomics of the system including usability, pleasantness, enjoyment of use, and how well the system supports accomplishment of individual tasks (kendall & kendall, 2011, pp. 103 and 104). but in order to do so, interviewing has become more structured over time, for example see the work of tan and hunter (2003). while this allows for easy comparison among participants and organizations, the increased structure may not allow the participants to tell their own stories. personal priorities and key impressions may remain hidden. myers and newman (2007) summarized previous research and explained several of the pitfalls in using the qualitative research interview (p. 4). discourse analysis begins with text and narratives, but tends to break texts and conversations into manageable pieces suitable for analysis. this approach is evident in the work of ellingsen (2003) who studied the development of knowledge work in hospitals in norway. in another study, dube and robey (1999) interpreted narratives that emerged from a software project and the complexity of negotiating multiple perspectives. one problem is that discourse analysis does not show concern for the order in which statements are made. this order is essential in the telling of a story. furthermore, boland and schultze (1996) suggested that although one narrative may be convincing, there is always an alternative narrative waiting to step in as the prevailing narrative. information systems researchers often identify, gather, analyze, and interpret metaphors (which are figures of speech that assert that one thing is another, e.g. an organization is a well-oiled machine) as australasian journal of information systems volume 17 number 2 2012 177 a way to understand and interpret what is happening with information systems in an organization; as a way to understand information systems development methods; as well as a way to interpret organizational settings (kendall & kendall, 1993, 1994). an approach seeking organizational metaphors is indeed a valid approach since it attempts to view a different dimension of the organization. however, using metaphorical analysis is complex and time-consuming. it is also possible to miss the subtler metaphors that occur daily in organizations. observation is an important method for eliciting information. unless observation is structured, it can also be selective, similar to the fragmentation in interviews. researchers and management consultants alike have been following around decision makers for decades. they can observe and document their behavior with the analyst’s playscript, a document that describes each action taken by each organizational member (shultis, 1964). in the uml (the unified modeling language) exceptions to main activities are documented in use case scenarios (jacobson, booch, and rumbaugh, 1999). strobe, structured observation of the environment (kendall & kendall, 1984a, 1984b, and 1985) is an unobtrusive method for observation that looks at common physical elements in an office to determine the use of information by executives. still, observation has its limits. would you prefer to observe einstein, or listen to his stories? a comparison of the above methods can be found in table 4. method strengths weaknesses storytelling  the story as a whole is greater than the sum of its parts.  knowing intended function of the story gives more context.  stories that possess vividness, elaboration and episodic order are long-lasting.  stories can be handled down (as myths) from organizational generation to generation.  requires the researcher to learn about storytelling.  print-based journals cannot print entire story.  requires the researcher to interact, thus changing the story or its effect.  stories may be true, untrue, or partially true. interviewing  excels at obtaining opinions, feelings, organizational and personal goals, and informal procedures.  can be managed and structured.  makes it easy to compare participants or organizations.  tends to result in obtaining fragments of stories.  has a tendency to be too structured.  participants may not have the opportunity to tell the stories they want to tell. discourse analysis  views language as social interaction.  can be aware of the social context in which discourse occurs.  emphasizes naturally occurring language.  may ignore the order in which statements are made.  can become too mechanical unless care is taken.  focuses more on text than actions or cognitive maps of participants. metaphors  excels in understanding the social and historical context.  can explain which information system is likely to succeed or fail.  difficult for some researchers who have difficulty identifying predominant metaphors.  listening for and identifying metaphors is time-consuming. observation  behavioral observation is used to document participant actions.  structured observation of the environment (strobe) can be done while participant is not in the room.  behavioral observation is intrusive and time-consuming.  strobe is effective and negating the narrative but is less effective at confirming the narrative. table 4. strengths and weaknesses of five types of information gathering methods australasian journal of information systems volume 17 number 2 2012 178 caveats and limitations of storytelling as a research method as is the case with the use of any research approach, be it a qualitative method or not, certain caveats apply to the use of storytelling. they include: 1. storytelling as a qualitative research method for is researchers is best used in conjunction with other research methods (i.e., using a multiple methods approach to avoid the limitations listed in table 4). 2. fruitful application of storytelling as a qualitative research method requires full participation of the organizational members involved. 3. the researcher needs to reveal the stories, not just fragments to the reader and to follow a method as suggested. 4. the act of storytelling as a qualitative research method engaged in by is researchers demands honesty from the researcher as well as organizational participants. we now discuss each of the four caveats in more detail. we underscore the importance and worthiness of a multiple methods approach (patton, 2002), which will allow the weaving of a bright tapestry rather than spinning of one single strand. an is researcher relying solely on stories is unlikely to accomplish all of their research goals. for example, some compelling narratives will not unfold accurately, or even be comprehensible, without the benefit of quantitative performance reports or monthly sales figures, as another way to understand the story. as denning (2000) points out, “storytelling doesn't replace analytical thinking. it supplements it by enabling us to imagine new perspectives and new worlds, and is ideally suited to communicating change and stimulating innovation. abstract analysis is easier to understand when seen through the lens of a well-chosen story and can of course be used to make explicit the implications of a story,” (p. xviii). when a researcher seeks meaningful interpretation of stories told by organizational members, the company participants must be fully on board. this might entail creating a longer timeline for project completion, since learning storytelling could require special types of coaching, as well as the development and use of instructive templates, websites, and seminars dedicated to heightening awareness of the importance of storytelling to the is function (ohara and cherniss, 2010). storytelling research has been problematical for many print-based journals, because of the demands of lengthy passages and unusual grammatical notations. typically authors and journals in the past have resorted to quoting fragments of stories, not complete ones. instead, authors can provide links in their papers that send the reader to their own research sites and data (a policy that federal granting institutions in the us are now requiring), which can in turn display entire stories. as storytelling in is becomes more widely adopted, is editors and reviewers will doubtless adjust to the demands of accommodating lengthier pieces. using a storytelling approach requires both individual and organizational candor, which is not apparent, possible, or (as some would say) even desirable in all settings. for the researcher’s part, they must heed the ancient greeks advice of “know thyself,” in order to harbor the substantial selfawareness necessary to interpret stories in a way that is meaningful to organizational participants as well as the is researcher. just as there are reasons participants tell their stories, researchers have reasons for interpreting them as they do. they need to be aware of their prejudices when they begin the study. gabriel (2000) explained, “the most evident danger of story-based research is the selective use of organizational narratives to amplify or reinforce the researcher's preconceived ideas or assumptions,” (p.151). australasian journal of information systems volume 17 number 2 2012 179 it is wise to remember that the presence of the is researcher using qualitative research methods changes the organization. for researchers involved in precipitating stories, this idea of the interactions among the researcher, the study organization, and any change in information systems should shape and inform the is researcher’s behavior from the outset. the caveats and limitations outlined in this section provided could spur many new research questions that provide ample substance for future studies. so, in essence the storytelling method consists of the following: 1. the main point of the storytelling method is to realize the whole story is greater than the sum of its parts. realizing that there are parts common to every story (as represented in our typology of 18 story elements) is useful to understanding the whole, and understanding the whole helps explain the reason why specific actions occurred. 2. the mechanisms of vividness, elaboration, and episodic order are useful for the researcher in the process of abstraction. these three mechanisms help explain why a story may take on mythic qualities. 3. understanding the storytelling functions (experiential, explanatory, validating and prescriptive) is a critical part of the storytelling method. it helps the researcher put the story in functional context along with an historical and social context. 4. knowledge of storytelling functions also helps the researcher overcome any preconceptions they bring to the study. researchers need to realize the reason they are interpreting a story as well as why the story is being told. 5. researchers need to be aware that storytellers often place themselves in the story as the hero. once again realizing the function of the story can help understand the motivation of the storyteller. 6. sometimes a story may be told in different ways. in the classic 1950 film rashomon, multiple storytellers recall a crime in very different ways. in each case they are motivated by their own self-interest (imdb). the researcher needs to be aware of this possibility and needs to place the stories in social, historical, and functional context. knowledge of the functions of stories is helpful to the researcher once again. 7. the researcher must recognize that interaction with participants changes stories to some degree. we describe four story activities (reacting, matching, eliciting, and collaborating) that researchers can use to obtain stories. the story elements, mechanisms, functions, and activities all play a part in the storytelling method. conclusion for too long, is researchers have been immersed in minutiae when conducting interviews, examining metaphors, and observing organizations. in doing so they often willfully disregard the whole story in favor of structured fragments. many methods facilitate gazing at the trees, rather than marveling at the forest. is researchers can gain useful insights into organizational practices surrounding information systems and their socially constructed meaning by using the qualitative research method of storytelling. is researchers can use storytelling to engage organizational participants in reacting to stories, matching one story told by recounting another, eliciting a new story, and collaborating in interpreting organizational stories about information systems use, problems, systems development, and systems adoption, as well as the strategic use of information systems. in developing our storytelling method, we developed a typology where stories can be described on a continuum from practical to philosophical, and from normative to descriptive. we propose that stories australasian journal of information systems volume 17 number 2 2012 180 are rooted in myths, and so can be usefully discussed in a typology of four powerful functions: experiential, explanatory, validating, and prescriptive. understanding the purpose of an organizational story (through elaboration of the myth) is not sufficient, however. stories follow certain patterns and contain many elements told in a customary way. the researcher using storytelling as a research method must also be fully aware of the telling of the story (through vividness that resonates with well-known myths and fables) and the order in which it is told (through the use of episodes). these three mechanisms, along with the importance of appreciating the story as a whole rather than a sum of its parts, emphasize why using storytelling as a research method differs from other types of narrative analysis. information systems researchers need effective research methods like storytelling that heighten their awareness and understanding of how information systems and their users, managers, and developers are interacting in organizations. these insights enable is researchers to interpret the meaning of organizational stories and how they are told. stories have structure and purpose and researchers can begin to examine the stories they hear for patterns and commonalities. while many standard research methods afford brief glimpses of information systems and the relationships among several organizational actors, storytelling presents an intuitively appealing, rich, and relatively unexplored method in is that can unveil insights that are often unavailable through the use of other, traditional methods. traditional methods often result in customary framing of analysis and codification of results that actively militate against, or at least methodologically inhibit creativity in interpretation. information systems researchers using storytelling are in many ways liberated from the constraints of framing imposed by traditional methods, and thus may infuse their studies of stories with fresh insight. a story that resonates is reiterated over and over, told to many people (not just researchers) and often handed down to generations (in organizations, among teams, and even to outsiders such as customers and suppliers). the story as a whole, its meaning, its purpose, its reach, and its effectiveness has rarely been addressed by researchers in is. it is for these reasons that this paper calls for is researchers to embrace storytelling and in doing so to recognize how storytelling can become a powerful way to gain an understanding of information systems in organizations. footnote 1 the authors thank these master storytellers, the playwrights we have met in the last five years, without whom we could not have written this paper: woody allen, david auburn, frank blocker, charles busch, jez butterworth, ethan coen, mike daisey, christopher durang, brian hargrove, jeffrey hatcher, bill irwin, david javerbaum, tony kushner, david lindsay-abaire, david mamet, tarell alvin mccraney, martin mcdonagh, denis o’hare, terrance mcnally, richard nelson, suzan-lori parks, david pittu, james rado, paul rudnick, sarah ruhl, gabriel shanks, and lucy thurber. 2 the authors thank the editor who was a true collaborator on this paper. australasian journal of information systems volume 17 number 2 2012 181 references alvarez, r. & urla, j. (2002) “tell me a good story: using narrative analysis to examine information requirements interviews during and erp implementation,” the data base for advances in information systems, vol 33 no 1, pp 38-52. belk, r. w., & tumbat, g. (2005) “the cult of macintosh,” consumption, markets & culture, vol 8 no 3, pp 205-217. berman, p. (1995) “harry's a great storyteller,” forbes, vol 155 no 5, 02/27, pp 112-115. barker, r. t. & gower, k. (2010) “strategic application of storytelling in organizations,” journal of business communication, vol 47 no 3, july, pp 295-312. boje, d. m., (1991) “the storytelling organization: a study of story performance in an officesupply firm,” administrative science quarterly, vol 36, pp 106-126. boland, r. j. & schultze, u. (1995) from work to activity: technology and the narrative of progress. in w. j. orlikowski, g. walsham, m. r. jones, & j. i. degross (eds.), information technology and changes in organizational work, pp 308-323. london: chapman & hall. boland, r. j. & schultze, u. (1996) “narrating accountability: cognition and the production of the accountable self,” in accountability: power, ethos, and the technologies of managing, r. munro & r. mouritsen (eds.) london: international thompson business press. bormann, e. (1972) “fantasy and rhetorical vision: the rhetorical criticism of social reality,” quarterly journal of speech vol 58, pp 396-407. brown, a.d. (1998) “narrative, politics and legitimacy in an it implementation,” journal of management studies, vol. 35 no 1, pp 35—58. bruner, j. s. (1990) acts of meaning, harvard university press, cambridge, ma. campbell, j. (1964) the masks of god, vol. 3: occidental mythology, penguin books usa inc., new york. checkland, p. (1999). soft systems methodology in action: a 30-year retrospective, john wiley & sons, new york. chreim, s. (2005) “the continuity-change duality in narrative texts of organizational identity,” journal of management studies, vol 43 no 3, 567-593. creswell, j. w. (2007) qualitative inquiry & research design, sage publications, thousand oaks, ca. denning, p. j. (2004) “the field of programmers myth,” communications of the acm, vol 47 no 7, 07, pp 15-20. denning, s. (2000) the springboard: how storytelling ignites action in knowledge-era organizations, butterworth-heinemann, woburn, ma. désilets, a. (2008) “tell me a story,” ieee software, vol 25 no 2, mar, pp 14-15. doty, w. g. (2004) myth: a handbook, greenwood press, westport, ct. australasian journal of information systems volume 17 number 2 2012 182 dube, l. & robey, d. (1999) “software stories: three cultural perspectives on the organizational practices of software development,” accounting, management, and information technologies vol 9, pp 223-259. ellingsen, g. (2003) “knowledge work in hospitals” in global and organizational discourse about information technology, e. wynn, e. whitley, m. myers, j. degross (eds.), kluwer academic publishers, boston, pp 466-484. fincham, r. (2002) “narratives of success and failure in systems development,” british journal of management, vol 13, pp 1-14. frohman, d. (2006) “leadership under fire,” harvard business review, vol 84 no 12, 12, pp 124131. gabriel, y. (2000), storytelling in organizations: facts, fictions, and fantasies, oxford university press, new york. gillard, h., mitev, n. & scott, s. (2007) “ict inclusion and gender: tensions in narratives of network engineer training,” information society, vol 23 no 1, jan, pp 19-37. gioia, d. a. & pitre, e. (1990) “multiparadigm perspectives on theory building,” academy of management review, vol 15, no 4, pp 584-602. grover, v. & ramanlal, p. (1999) “six myths of information and markets: information technology networks, electronic commerce, and the battle for consumer surplus,” mis quarterly, vol 23 no 4, 12, pp 465-495. hanson, k. (2004) “keeping a sense of community alive,” strategic communication management, vol 8 no 4, jun, pp 6-7. higgs, m. & rowland, d. (2010)"emperors with clothes on: the role of self-awareness in developing effective change leadership,” journal of change management, vol 10 no 4, 12, pp 369-385. hodgett, s. & deneulin, s. (2009) on the use of narratives for assessing development policy, public administration vol 87, no 1, pp 65-79. imdb, rashomon, accessed at http://www.imdb.com/title/tt0042876/, january 10, 2012. jacobson, i. , booch, g. & rumbaugh, j. (1999) the unified software development process, addison-wesley, reading, ma. james, c. h. & minnis, c. (2004) “organizational storytelling: it makes sense,” business horizons vol 47 no 4, july-august, pp 23-32. johnson, k. n. (2008) “storytelling & software testing,” http://www.testingreflections.com/node/view/6700 accessed 22 march 2011. kaplan, b. & maxwell, j.a. (1994) “qualitative research methods for evaluating computer information systems,” in evaluating health care information systems: methods and applications, j.g. anderson, c.e. aydin & s.j. jay (eds.), sage, thousand oaks, ca, pp 4568. kelly, c. & zak, m. (1999) “narrativity and professional communication: folktales and community meaning.” journal of business and technical communication, vol 13, pp 297-317. kendall, j.e. & k.e. (1985) “development, application and enrichment of strobe: refinement of an observational tool for the information analyst,” in research methods in information http://www.imdb.com/title/tt0042876/ http://www.testingreflections.com/node/view/6700 australasian journal of information systems volume 17 number 2 2012 183 systems, by e. mumford, r. hirschheim, g. fitzgerald, & a. t. wood-harper (eds.), elsevier science publishers b. v., amsterdam, pp 119-129. kendall, j.e. & kendall, k.e. (1993) “metaphors and methodologies: living beyond the systems machine,” mis quarterly, vol 17 no 2, pp 149-171. kendall, j.e. & kendall, k.e. (1994) “metaphors and their meaning for information systems development, european journal of information systems, vol 3 no 1, pp 37-47. kendall, j.e. & kendall, k.e. (1995) “the impact of using hypertext-based information systems on organizational narrative: changing the structure of stories in d.m. olson (ed.),” proceedings of the third international meeting of the decision sciences institute, puebla, mexico. kendall, j. e., kendall, k.e. & kah, m. m. o. (2007) “examining virtual organizations using fantasy theme analysis: a study of ict policy advisors’ discourse about developing countries,” jitta (journal of information technology theory & application) vol 8, no 2, pp 1-20. kendall, j. e. & kerola, p. (1994) “a foundation for the use of hypertext-based documentation techniques,” journal of end user computing, vol 6 no 1, pp 4-14. kendall, k. e. & kendall, j.e. (1984a) “structured observation of the decision-making environment: a validity and reliability assessment,” decision sciences, vol 15 no 1, pp 107-118. kendall, k. e. & kendall, j.e. (1984b) “strobe: a structured approach to observation of the decision-making environment,” information & management, vol 7 no 1, pp 1-11. kendall, k. e. & kendall, j.e. (2011) systems analysis and design, eighth edition, prentice hall, upper saddle river, nj. kendall, k. e., kendall, j. e., & lee, k. c. (2005) “understanding disaster recovery planning through a theatre metaphor: rehearsing for a show that might never open,” communications of the association for information systems, vol 16, article 51. http://aisel.aisnet.org/cais/vol16/iss1/51 accessed 22 march 2011. kendall, k. e. & losee, r. d. (1986) “information system folklore: a new technique for system documentation,” information & management, vol 10 no 2, 1986, pp 103-111. klein, h. k. & myers, m. d. (1999) “a set of principles for conducting and evaluating interpretive field studies in information systems,” mis quarterly, special issue on intensive research, vol 23 no 1, pp 67-93. kuechler, w.l. & vaishnavi, v. (2006) “so, talk to me: the effect of explicit goals on the comprehension of business process narratives,” mis quarterly, vol 30 no 4, pp 961-979. lee, a. s. (2010) “retrospect and prospect: information systems research in the last and next 25 years,” journal of information technology, vol 25, pp 336-348. levy, s. j. (1994) “interpreting consumer mythology,” marketing management, vol 2 no 4, spring94, 1994, pp 4-9. lieblich, a., tuval-mashiach, r., & zilber, t. (1998) narrative research, sage publications, inc., thousand oaks, ca. livingston, j. s. (1969) “pygmalion in management,” harvard business review, vol 47 no 4, jul, pp 81-89. http://aisel.aisnet.org/cais/vol16/iss1/51 http://www.qual.auckland.ac.nz/klein-myers.pdf http://www.qual.auckland.ac.nz/klein-myers.pdf australasian journal of information systems volume 17 number 2 2012 184 mackenzie, d. and wajcman, j., (eds.) (1998) the social shaping of technology, 2nd ed. open university press, buckingham, uk. macaulay, k. d., yue, a. r. & thurlow, a. b. (2010) “ghosts in the hallways: unseen actors and organizational change,” journal of change management, vol 10 no 4, 12, pp 335-346. mainiero, l. a. (1994) “getting anointed for advancement: the case of executive women,” academy of management executive, vol 8 no 2, 05, pp 53-64. mascarenhas, s., dias, j., prada, r. & paiva, a. (2010) “a dimensional model for cultural behavior in virtual agents,” applied artificial intelligence, vol 24 no 6, 07, pp 552-574. mintzberg, h. (1972) “the myths of mis,” california management review, vol 15 no 1, fall72, 1972, pp 92-97. misa, t.j., feenberg, a., & brey, p. (eds.) (2003) modernity and technology, mit press, cambridge, ma. mitroff, i. i. & kilmann, r. h. (1975) “stories managers tell: a new tool for organizational problem solving,” management review, vol 64 no 7, pp 18-28. moen, t. (2006) “reflection on the narrative research approach,” international journal of qualitative methods, vol 5 no 4, article 5, http://www.ualberta.ca/~iiqm/backissues/5_4/pdf/moen.pdf accessed 4 march, 2011. moisander, j. & eriksson, p. (2006) “cooperative narratives of information society: making up the mobile consumer subject,” consumption, markets and culture, vol 9 no 4, pp 257-275. moody, j.w., blanton, j. e. & cheney, p.h. (1998) “a theoretically grounded approach to assist memory recall during information requirements determination,” journal of management information systems, vol 15 no 1, pp 79-98. myers, m. d. "qualitative research in information systems," mis quarterly (21:2), june 1997, pp. 241-242. misq discovery, archival version, june 1997, http://www.misq.org/supplements/. misq discovery, updated version, last modified: november 12, 2011 www.qual.auckland.ac.nz myers, m. d. & newman, m. (2007) “the qualitative interview in is research: examining the craft,” information and organization vol 17, pp 2-26. ohara, s. c. & cherniss, m. (2010) “storytelling at jupiter networks connects a global organization to the values and behaviors of success,” global business and organizational excellence, vol 29 no 5, pp 31-39. patton, m. q. (2002) qualitative research & evaluation methods, 3rd edition. sage publications inc., thousand oaks, ca, pp 339-427. pentland, b. (1999) “narrative methods in collaborative systems research,” proceedings of the 32nd hawaii international conference on systems sciences, hawaii: ieee computer society, pp 19. pinch, t. & bijker, w. (1984) “the social construction of facts and artifacts: or how the sociology of science and the sociology of technology might benefit each other,” social studies of science, sage, vol 14 no 3, pp 399-441. powers, r. f. & dickson, g. w. (1973) “misproject management: myths, opinions, and reality,” california management review, vol 15 no 3, spring73, pp 147-156. http://www.ualberta.ca/~iiqm/backissues/5_4/pdf/moen.pdf http://www.misq.org/supplements/ http://www.qual.auckland.ac.nz/ australasian journal of information systems volume 17 number 2 2012 185 ready, d. a. (2002) “how storytelling builds next-generation leaders,” mit sloan management review, vol 43, no 4, pp 63-69. sabherwal, r. & robey, d. (1993) “an empirical taxonomy of implementation processes based on sequence of events in information system development,” organization science, vol 4 no 4, pp 548-576. seffah, a. & metzker, e. (2004) “the obstacles and myths of usability and software engineering,” communications of the acm vol 47 no 12), pp 71-76. schultze, u. & avital, m. (2011) “designing interviews to generate rich data for information systems research,” information and organization, vol 21 no 1, pp 1-16. shim, j.p., shin, y. b., & nottingham, l. (2002). “retailer web site influence on customer shopping: exploratory study on key factors of customer satisfaction,” journal of the association for information systems, vol 3 no 3, pp 53-76. shin, d. (2006) “effective design in the development of public information infrastructure: a social constructionist approach,” information polity, vol 11, pp 85-100. shultis, r. (1964) “playscript – a new tool accountants need,” naa bulletin, vol 45 no 12, pp 310. silva, l. & backhouse, j. (2003). “the circuits-of-power framework for studying information systems. journal of the association for information systems,” vol 4 no 14, pp 236-294. silverman, l. i. (2004) “strategic storytelling,” association management, vol 56, no 11, 41-48. smillie, k. (2010) “notes on canadian computing, 1950s,” ieee annals of the history of computing, vol 32 no 2, apr, pp 58-83. social construction of technology (scot) (n.d.) on social construction of technology sts wiki. retrieved from http://www.stswiki.org/index.php?title=social_construction_of_technology_%28scot%29, january 6, 2012. stewart, t. a. & kaufman, d. c. (1995) “getting real about brainpower,” fortune, vol 132 no 11, 11/27, pp 201-203. swap, w., leonard, d., shields, m., & abrams, l. (2001) “using mentoring and storytelling to transfer knowledge in the workplace,” journal of management information systems, vol 18 no 1, pp 95-114. tan, f. & m.g. hunter (2003) “using narrative inquiry in a study of information systems professionals,” in 36th hawaii international conference on system sciences, hawaii: ieee computer society, p 235. tichy, n. (1997) the leadership engine, harper business, new york. vergin, r. c., and a. j. grimes. (1964) “management myths and edp,” california management review, vol 7 no 1, fall64, pp 59-70. vlachoutsicos, c. a. (2011) “how to cultivate engaged employees,” harvard business review, vol 89 no 9, 09, pp 123-126. von bertalanffy, l. (1968). general system theory, g. braziller, new york. walsham, g. (1993) interpreting information systems in organizations, wiley, chichester, pp. 4-5. http://www.stswiki.org/index.php?title=social_construction_of_technology_%28scot%29 australasian journal of information systems volume 17 number 2 2012 186 weick, k. e. & browning, l.d. (1986) “argument and narration in organizational communication, journal of management,” vol 12 no 2, pp 243-259. whelan, g., and c. o'gorman. (2007) “the schumpeterian and universal hero myth in stories of irish entrepreneurs,” irish journal of management, vol 28 no 2, 10, 2007, pp 79-107. young, j. (2004) “joseph campbell – a scholar’s life,” the center for story and symbol, http://www.folkstory.com/ accessed on 22 march 2011. http://www.folkstory.com/ australasian journal of information systems volume 17 number 2 2012 187 appendix klein & myers (1999) principles for interpretive field research how the kendall & kendall storytelling as a methodology paper addresses each of the seven klein and myers principles 1. the fundamental principle of the hermeneutic circle the storytelling method described in this article reiterates the fundamental principle of the hermeneutic circle that individual parts of the narrative cannot be understood without understanding the whole narrative. kendall & kendall suggest that in order to better understand the 18 distinct elements of a story a researcher needs to appreciate the story as a whole. in turn, the whole story helps the researcher understand what the story elements mean. 2. the principle of contextualization in this article, kendall & kendall articulate four functions of storytelling and the storyteller typically sets the story. while interpretation of stories require the interpreter to be aware of the social and historical context, kendall & kendall also recognize that interpreters should be aware of the functional context. 3. the principle of interaction between the researchers and the subjects the storytelling method recognizes that the storytellers, not just the researchers, serve as interpreters and analysts. in this article, kendall and kendall suggest four activities a researcher can undertake to help elicit stories that are accurate and detailed. they are reacting, matching, eliciting, and collaborating. 4. the principle of abstraction and generalization this principle compels the researcher to relate theoretical abstractions and generalizations to the details experienced or collected by the researcher. kendall & kendall identify 18 elements common in complex stories, then assess these to conclude that each of these contribute to three cognitive mechanisms (vividness, elaboration, and episodic) that together make a story memorable and enduring. 5. the principle of dialogical reasoning each researcher may approach storytelling differently based on their knowledge of myths and folklore as well as prior experience in organizations. storytellers, however, tell stories because they believe the stories have a function. kendall & kendall recognize that identification of the intended function may cause the researcher to revise their theoretical preconceptions. 6. the principle of multiple interpretations the same story can be told by different storytellers from conflicting viewpoints. the storytellers may be motivated by their own self-interest. kendall & kendall recognize that there may be multiple interpretations of the same story. 7. the principle of suspicion since storytellers often place themselves as the hero in a story, multiple storytellers may be in conflict with one another. in this case, kendall & kendall recommend that the researcher needs to reexamine the context and scrutinize which function is intended. table 5. the storytelling method observes the principles for interpretive research. ajis vol. 6 no. 1 september 1998 evaluation of workflow management systems a meta model approach michael rosemann, michael zur muehlen department of information systems westfaelische wilhelms-universitaet muenster steinfurter str. 107 d-48149 muenster germany {ismirolismizu) @ wi.uni-muenster.de phone 449-251-83-3 8100 fax+49-251-83-38109 correspondence: dr. michael rosemann july 22-october 31, 1998: queensland university of technology school of information systems 2 george street gpo box 2434 brisbane qld 4001 abstract the automated enactment of processes through the use of workflow management systems enables the outsouring of the control flow from application systems. by now a large number of systems, that follow different workflow paradigms, are available. this leads to the problem of selecting the appropriate workflow management system for a given situation. in this paper we outline the benefits of a meta model approach for the evaluation and comparison of different workflow management systems. after a general introduction on the topic of meta modeling the meta models of the workflow management systems workparty (siemens nixdorf) and flowmark (ibm) are compared as an example. these productspecific meta models can be generalized to meta reference models, which helps to specify a workflow methodology. exemplary, an organisational reference meta model is presented, which helps users in specifying their requirements for a workflow management system. basics of workflow management systems similar to the transfer of data management away from application systems toward database management systems the transfer of control flow enabled by workflow management systems has a profound meaning for the future architecture of information systems. joosten defines a workflow management system (with reference to the workflow management coalition (wfmc)10) as "[...] a system that defines, manages and executes workflows through the execution of software whose order of execution is driven by a computer representation of the business process logic."11 workflows12 are processes temporal and logical sequences of functions that are necessary to perform operations on economically relevant objects with automated transitions, namely processes, whose control logic lies within the control of an information system. every workflow is based on a process model that has been enhanced with additional object types (eg. data structures) and attributes (eg. start conditions) that allow its automation, the so called workflow model. workflow models are usually described using directed graphs whose knots represent (elementary or composite) functions. from the point of view of the workflow management system elementary functions are encapsulated, ie. the workflow management system has an external view on the automated or manual execution of the functions but is not concerned with the execution of the functions themselves. 10 cf. wfmc (1996). 11 joosten(1995),p.4. 12 for a variety of interpretations of the term workflow cf. e.g. georgakopoulos, hornick, sheth (1995), pp. 122f. 103 ajis vol. 6 no. 1 september 1998 by now a large number of commercial workflow management systems are available. these systems differ with regard to their efficiency due to their distinct historical background.13 the selection of an appropriate workflow management system demands an efficient selection process. during the traditional process of software evaluation the main focus lies on technical aspects, such as reliability, useability, maintainability and adaptability14 as well as on economic aspects of the systems analysed. the use of criteria catalogues represents the common practice of software evaluation. however, they fail during the evaluation of modeling methods, because they do not reflect the vast number of modeling alternatives offered by a method. the complexity of these alternatives can be reduced by the formalisation of the method's description. in the next sections we will analyse how meta models can help (pre-)selecting an appropriate workflow management system. to begin with, we discuss meta models in general and meta data models in particular. in the following chapter the product individual meta data models of workparty (sni) and flowmark (ibm)15 are presented. the comparison and generalisation of the two meta data models leads to a generic organisational reference meta model, which can serve as a benchmark for system specific meta data models. the analysis performed allows us to draw some substantial conclusions. the discussion of this approach is the subject of the last section. definition and intention of meta models characteristics and occurrences of meta models a model is an immaterial representation of a relevant part of the real world. it is created for the purpose of a subject, relates two systems and therefore consists of three components:16 • the object system represents the subjective interpretation of a selected part of the real world (universe of discourse) including the relevant part of the environment. • the model system represents the subjective image of the object system. a syntax (also: notation, language) is needed to create the model system. • the projection formulates the relationship between the object system and the model system. the complexity of the real world, that is generated by the number of real world elements and relationships, is reduced by eliminating irrelevant elements. the variety of real world elements and relationships is controlled by clustering elements and relationships (type generation). if a model system ml represents the object system of a model system m2, the model system m2 represents the meta model system of the object system ml is based upon.17 because of this degree of abstraction a meta model can be seen as "a design framework, that describes the basic model elements and the relationships between the model elements as well as their semantic. this framework also defines rules for the use and specialisation of model elements and relationships."18 in the context of information modeling we distinguish meta models describing a notation and meta models describing a procedure.19 meta data models characterise notations that can be used for information modeling purposes. meta process models describe the modeling process using a specific method. every meta model is based upon another 13 cf. joosten (1996), p. 2: "the [...] field of workflow management suffers from confusion caused by weakly defined concepts and a lack of consensus about the way in which these concepts are used." 14 cf. dunn (1993); kazman et al. (1995). 15 cf.leymann,altenhuber(1994). 16 cf.hars (1994), pp. 7-11. 17 cf. hars (1994), p. 11. cf. nissen et al. (1996), p. 38. 18 cf. ferstl, sinz (1994), p. 86; steele, zaslavsky (1994), p. 317: "meta models [...] might be expressed using one or more modelling techniques, that in combination are able to adequately model all relevant aspects of any given modelling technique." 19 cf. saeki (1996). 104 ajis vol. 6 no. 1 september 1998 meta model, which can be of the same kind, eg. the notation of the entity relation diagram may be explained using another entity relation diagram. if the similarities of a number of meta models are consolidated in one universal model that also claims a high degree of semantic quality this model is called a reference meta model and specifies the common methodology. in the past a number of different meta models for workflow management systems have been introduced. in most cases these models are reference meta models. examples for such reference meta models can be found in derungs (1997), galler (1997), jablonski, bussler (1996), joosten (1996) and rupdetta (1997). the main objective of this paper is the evaluation of product individual meta models as well as the discussion of the results delivered by a comparison of these (detailed) meta models. the integration of the presented product individual meta models into an inductively generated reference meta model is outlined further on. description of the notation the meta data models in this paper are designed using extended entity-relationship models.20 an extension of this method are variable-based integrity constraints21, which allow instance specific definitions of semantic relationships (eg. exclusive-or) between cardinalities. this is achieved by replacing the cardinalities with variables and relating the cardinalities in separate integrity constraints. although variable based integrity constraints raise the complexity of the meta data models, they are useful as they enable the description of specific rules of a certain method. furthermore, generalisations are established for reasons of clarity, eg. groupings of all information objects that are relevant for the staff resolution. for every generalisation a description of the disjointness constraint (d (disjoint) or n (not disjoint)) and the completeness constraint (p (partial) or t (total)) is given.22 comparison of meta models conflict identification and resolution prior to the evaluation of meta models it has be made sure that all possible conflicts between the models have been resolved. three major kinds of conflicts can be identified:23 naming conflicts occur, if the naming conventions of the models to be compared show synonyms or homonyms. these are the most common conflicts during the evaluation of meta data models. a synonym can be found if two or more linguistic expressions share the same meaning. a homonym results of the ambiguity of a linguistic element. in its context the meaning of a homonym is unique, but not because of its notion alone. the determination of synonyms requires an analysis of different terms with identical meaning and associations. hints about potential synonyms can be found by tracing similar structures embedding information objects with different names (concept likeness)24. homonyms contradict the clarity of a model, because the notion of the term can be determined depending on the user and the context, but not in general. a potential indicator for the identification of homonyms in meta models are information objects with the same name that are embedded in different structures (concept unlikeness). a type conflict can be located, if the same fact is represented semantically correct in two models through different methodical concepts. structural conflicts arise if the meta models to be integrated depict the same facts using different semantics. this is a violation of the semantic correctness. structural conflicts often arise if different people are involved in the modeling process. type and structural conflicts are not relevant at this point, because the meta data models to be compared and to be evaluated were created by the same group of people. objects of a meta model evaluation 20 cf. chen (1976). 21 cf. rosemann (1996). 22 cf. elmasri, navathe (1995), p. 618. they use the term overlapping instead of not disjoint. 23 cf. hars (1994), pp. 194-206. 24 cf. batini, lenzerini (1984); batini, ceri, navathe (1992), p. 124. 105 ajis vol. 6 no. 1 september 1998 when two conflict-free meta data models are compared, the following information objects can provide useful information: entity types the comparison of the number and kind of entity types provides the most essential information for the comparison of meta data models. within a given degree of abstraction the adaptability of a workflow management system increases with the number of entity types the meta data model consists of. the validity of a qualitative and quantitative rating of entity types is determined by the degree of detail of the models in the first place. relationship types another metric concerning the flexibility of a workflow management system are the relationship types. the flexibility of a workflow management system increases with the number of relationship types, if the number and kind of entity types stay the same. an example for different relationship types can be found in the organisational model of a workflow management system, where the relations between actors, organisational units and roles differ between the systems. cardinalities the cardinalities provide useful information about the design options a specific system provides. a workflow management system, that allows a (0,l)-(0,n)-hierarchy of organisational units, simultaneously prohibits the design of multi-dimensional organisations (eg. matrix-structures). the introduction of variablebased cardinalities enhances the semantic capability of a model, but complicates the comparability of the model. attributes beyond the elements an erm usually includes, the comparison of attributes provides further information about the system characteristics a workflow developer can use. attributes have to be distinguished into attributes used by the system and additional ('free') attributes. user defined attributes allow a higher flexibility than predefined attributes. evaluation of the organisational meta models before the meta data models are compared, a brief description of the central entity types is given. potential naming conflicts have already been resolved during the modeling of the two entity-relationship-diagrams depicted in fig. 1 and 2. for example, the entity type role of workparty was renamed position type, while the entity type competence was renamed role in order to avoid a homonym conflict with the corresponding terms of flowmark.25 an organisational position in workparty is the domain of a person within an organisational unit and is assigned to exactly one organisational unit. an organisational position can be occupied by one person, but it may be left empty, too. in workparty an organisational position is an abstraction of a specific person. zero, one or several persons can act as a substitute for a specific organisational position. an organisational position can belong to a certain position type. this entity type cannot be found in the meta model of flowmark. an organisational unit in workparty is a part of the company organisational structure, eg. a department. it is created as a composition of organisational positions or subordinate units. organisational units can be temporary or permanent, the former are called project units, the latter are called line units and form the core of the company organisational structure. the hierarchical structure of organisational units is restricted to a tree structure. an organisational unit may be associated with several roles and positions and can be the owner of several resources. similar to workparty, an organisation in flowmark represents an administrative unit of the company organisational structure. however, a person in flowmark can only be a member of one organisation. an organisation has one specific manager and may have an unlimited number of members with different roles, but at least one (the manager). 25 one can identify a homonym conflict between the entity type role of workparty and flowmark as well as a synonym conflict in the renamed entity type competence and the original entity type role of workparty. a correct resolution of these conflicts asks for a new context free term for the entity types concerned. however, the term role has a generally accepted meaning in this context and is used by the wfmc as well. therefore, in this particular case the conflict resolution is passed up in order to use generally accepted terminology. 106 ajis vol. 6 no. 1 september 1998 resources (only in workparty) are items that ultimate units of responsibility may dispose in order to fulfil their tasks. each resource may be assigned to one or more roles. this leads to the definition of access rights for certain persons. each resource has one specific owner, which may be a person, an organisational unit, or an organisational position. resources can be organised in an is-part-of-relationship, thus being superior or subordinate to other resources. levels (only in flowmark) represent a hierarchy inside the organisation that need not be identical with the company organisational structure (eg. years of membership or tariff). when compared directly (fig. 1 and 2), the workparty meta model shows a larger number of entity types than the flowmark meta model, which points to a higher flexibility of modeling different organisational structures. flowmark misses especially the entity types organisational position and position type. in contrast to this, flowmark offers two relationship types between the entity types person and organisation or person and role, while workparty offers only one relationship type between these entity types. flowmark shows more restricted cardinalities than workparty. especially the rigid (l,l)-(l,n)-relationship between the entity types person and organisation is expected to be a handicap when modeling organisations. the possibility of assigning several positions to one person in workparty enables the modeling of multi-dimensional organisations and shows greater flexibility than the solution provided by flowmark. both products show a restricted (0,l)-(0,n)hierarchy over the entity type organisation. in addition, the substitute-relationship for the entity type person in flowmark is restricted to only one substitute per person ((0,l)-(0,n)-relation), while workparty allows an unlimited number of substitutes. in workparty a person has to be assigned to a position and therefore to a specific organisational unit. this prohibits the modeling of external workers, such as contractors. the interface between the organisational meta model and the process meta model is provided by an address expression. this entity type enables a dynamic staff resolution at runtime, eg. according to the criteria given in the address expression and the current system users, all persons are determined, that match the given criteria. in general, direct and indirect address expressions can be distinguished. while the former explicitly reference the authorised actors in person, the latter contain criteria like organisational units or positions which are used to determine a subset of authorised persons at runtime. the use of indirect address expressions makes processes more resistant against changes in the organisational model of a company than it would be possible with a direct assignment of persons to activities. evaluation of the process meta models both flowmark and workparty provide the user with a graphical modeling method for designing workflow models. in flowmark workflow models are drawn as graphs with weighted transitions through the explicit design of knots and connectors. workparty offers the user pre-defined process components, which provide elementary control flow elements, such as iterations, forks and joins. the process meta model of workparty in workparty a process is a directed graph that determines the sequence of activities or programs respectively. it consists of a starting activity and one or more ending activities, knots and connectors. the starting activity contains global information about the process that are relevant for all activities. knots contain activities or programs, connectors determine their sequence. several ending activities can be used to model alternative endings of a process, the starting activity of a process has to be unique. an activity is an elementary step in a process. activities can be manual or they can reference an application program. furthermore, programs may be associated with activities that are only executed in case of an exception. workparty supports ad-hoc activities, ie. at runtime the user may insert an unlimited number of new activities at a specified point, before the modelled process is resumed. a file is a database used by workparty for storing process and activity templates. while a theme file contains all process and activity templates that are related to a specific field of business, a template file is a subset of a theme file and contains process and activity templates for a certain kind of business process. process files are instances of template files and contain attachments and parameters relevant for the specific processes. 107 ajis vol. 6 no. 1 september 1998 the process meta model of flowmark a process in flowmark is similar to a process in workparty. in addition to the sequence of the activities the process designer in flowmark has to specify the flow of data between the activities through the use of data connectors. each process can be assigned to one process category in order to restrict the number of people authorised to execute the process. furthermore, a process may contain an address expression that is valid in addition to activity individual address expressions. for every process a process administrator can be defined, who is contacted in case of an exception. an activity is a single step in a process and corresponds to the activity in workparty. activities contain a start condition and an end condition as well as an address expression for the individual staff resolution. flowmark supports three kinds of activities: • process activities are used to invoke sub processes within a process. it is possible to recursively invoke the same process. • program activities are (semi-)automated activities, that invoke an application program during their execution. • loop elements are used for modeling iterations. if a single activity shall be executed several times the end condition of the activity can be used as a loop condition. if the iteration shall contain several activities a block has to be used. blocks are equivalent to sub-processes, but their contents are modelled individually. bundles are used if a single activity has to be instantiated several times. containers are used for storing structured data that is transmitted from one activity to another. each activity, process and block has one input container and one output container. the contents of a container can be addresses using various apis (c, rexx, cobol etc.). 108 ajis vol. 6 no. 1 september 1998 integrity constraints v,e{0, l ) v 2 e { v , , l ) (vn) \ (o.m) address •tprtssion (o.n) /\ fig. 1: organisational meta model of workparty 109 ajis vol. 6 no. 1 september 1998 integrity constraints '. ' ' : • ' ,^1 ax,. €{0,1} vie {1,2} •r=l 3. fig. 2: organisational meta model of flowmark 110 ajis vol. 6 no. 1 september 1998 fig. 3: process meta model of workparty ill ajis vol. 6 no. 1 september 1998 integrity constraint ition ilion 1 end condition start co v"clivity loop element process fig. 4: process meta model of flowmark a connector is a directed edge that determines the control flow and the data flow of a process. control connectors can have a transition condition that must be fulfilled, otherwise the target activity will not be executed and the path of the connector will be eliminated during a dead path elimination. default connectors have no transition condition and are followed if no transition condition of the other control connectors evaluates to true. data connectors are used 112 ajis vol. 6 no. 1 september 1998 for connecting the inputand output containers of activities. the flow of data in flowmark need not to be identical with the control flow. evaluation of the process meta models similar to the conflict resolution during the comparison of the organisational meta models several conflicts had to be resolved before the process meta models could be compared. for example the entity type process of flowmark is called sequence in workparty. the resolution of conflicts was done by the selection of wfmc-compliant terms. the grey reverse t shapes in figure 3 and 4 indicate characteristic desip elements that can be found in the process meta models of different workflow systems. in flowmark control and data flow are modelled separately and are therefore visible as distinct elements. in workparty the control flow is modelled by customising predefined components while the data flow is not visible in the process model. instead, the data flow is modelled for each activity individually. the control flow elements available in both systems are not completely represented in the meta models. for example, the bundle activity of flowmark can be implemented in workparty as well, using certain iteration components in combination with optional process branches. therefore, the semantic power of the meta models cannot be compared without taking the control flow elements into account. contrary to the organisational meta model a direct comparison of the number of entity types does not seem reasonable for the process meta models. because of the different workflow philosophies the mere number of entity types does not lead to conclusions about advantages or disadvantages of a specific system. the selection of the appropriate level of detail is difficult, too, because on the one hand the meta models should not be subject to unlimited growth and on the other hand sufficient aspects for comparison should be available. the great variety of workflow modeling methodologies shown by both systems makes the comparison of the process meta model a lot more difficult than the comparison of the organisational meta models. therefore, an enhancement of the evaluation with a catalogue of basic control flow elements seems to be useful. the results of such an enhancement are a starting point for the evaluation of the semantic modeling power of different workflow management systems. reference meta models as a benchmark for meta data models an organisational reference model for workflow systems the evaluation of the meta data models described above as well as the analysis of two additional workflow management systems led to the conclusion, that an independent benchmark had to be established in order to enable a more objective analysis of workflow systems. the evaluated workflow systems showed substantial weaknesses regarding their organisational modeling component, whereas the process modeling capabilities were much more sophisticated. therefore an organisational reference model has been developed, which serves as a benchmark for the evaluation of meta data models (fig. 5). this reference meta model includes several aspects which enhance the ideas found (or missed) in the systems analysed. the main objective during the development of the reference meta model was to obtain a maximum of flexibility in order to use the model for a broad variety of organisations. consequently, the meta data models of existing workflow systems served as a starting point for the reference model, but it finally turned out that a flexible meta model could be designed with a relatively small number of entity and relationship types. additional entity types most workflow systems provide the modeller with a role concept for the modeling of organisational entities. roles serve as a container for the qualifications an actor has to have in order to fulfil a given task. however, in real life organisations there is a difference between qualifications and competencies. while the former are attributes of a specific actor and cannot be removed (ie. "speaks spanish"), the latter depend on the position the actor occupies in a given organisation (ie. "may sign orders over $ 50.000") and can be assigned and removed. consistently, the entity type role is subdivided into the two entity types qualification and competence. the modeling of committees, expert groups and other temporary organisational units is enabled by two elements. first, the strict organisational hierarchies [(0,l)-(0,n)] found in workparty and flowmark are replaced by structures 113 ajis vol. 6 no. 1 september 1998 which allow the modeling of multidimensional relationships between roles, positions and organisational units. secondly, the relationship between actors and organisational units is separated by the introduction of positions, which form the static organisational structure of a company. a person can occupy several organisational positions and can therefore be a member of several organisational units, those being temporary or permanent. since an organisational position serves as a placeholder for a specific person, the competencies related to a position should be assigned to the position itself instead to the person occupying it (although it is possible to assign competencies to a person directly as well). with the increasing size of a company, the number of similar positions is likely to increase. in this case position types serve as a template for positions (eg. "secretary") which can be refined in a position instance (eg. "secretary of prof. becker"). fig. 5: organisational reference meta model substitute relationships not all workflow systems offer the possibilities of defining substitute relationships between employees. in most cases the assignment of an activity to a role instead of a specific person solves the problem of substituting an absent employee. in some cases (especially during an authorisation or an inspection activity) it is necessary to assign an activity to a specific person. substitutes are necessary in these cases in order to preserve the flexibility of the original role system. most substitute mechanisms allow an actor to inherit all qualifications of the original actor. in some systems (eg. workparty), the substitute mechanism relates two organisational positions instead of two actors. however, the complete inheritance of all rights can have serious impact on the security policies of an enterprise. therefore it is desirable to be able to restrict a substitute relationship to specific roles of an actor. this is enabled by introducing a ternary relationship between the entity type person and the entity type role. a person can have several substitutes as well as be the substitute for several other people. the substitution may be reduced to certain roles of a person. hence it is possible eg. to create a substitute for a manager that is not allowed to sign orders over $ 50.000, because this specific role of the manager was excluded from the substitute relationship. 114 ajis vol. 6 no. 1 september 1998 process dependent staff resolution during the instantiation of a workflow model it may be necessary to assign actors to activities depending on the state of process objects (eg. invoice, offer) or the state of activities already performed. it should therefore be possible to assign an activity not only to the entity types of an organisational meta model, but also to the relevant relationship types, such as "supervisor of the actor that performed the last activity". this option is useful in case of an exception, when an escalation mechanism redirects the current activity to the superior of the actor performing the activity. another example of this mechanism is the double check of an invoice. in this situation it has to be made sure that the actor performing the second check is not the same actor that already performed the first check. for reasons of clarity, these relationships are not depicted in fig. 5. an important aspect of process dependent staff resolution is the use of process object attributes as variables for the activity definition, which is outlined by means of an example. during the auditing of accounts, it is necessary to model separate activities for every origin of a foreign bill (english, french, spanish etc.) and to assign certain roles (clerk with english language skills, clerk with spanish language skills etc.) to each of these activities. with the use of process objects it is possible to reduce the activities to one, where the origin of the bill itself serves as a variable for the role definition, ie. if an american bill is the process object of the current process instance, the required generic role ("x language skills") is replaced by a certain instance enhanced with the attributes of the process object ("english language skills"). since it affects the interface between the meta process and the meta data model of a workflow management system, this aspect is not depicted in fig. 5 as well. evaluating individual meta models against a reference model the comparison of the organisational meta models with the organisational reference model outlined above reveals some additional aspects. while the organisational meta model of workparty (fig. 1) shows entity types similar to those of the reference model (position type, organisational position etc.), the high number of relationship types tend to make the meta model unnecessarily complex. contrary to this observation the entity types of the flowmark meta model (fig. 2) show only little resemblance to those of the reference model. on the other hand some relationship types may be addressed directly in flowmark (coordinates, is member of), which is not possible in workparty. the use of a reference model helps directing the attention of the observer to aspects that would not have been analysed otherwise. thus a reference model can improve the results of a meta model comparison. reference meta models can serve as a foundation for the design of a process information system, used for process monitoring and controlling, as well as a guideline for information system designers deriving meta models from given systems. summary and future directions a comparison of meta models can provide useful information for the evaluation of systems, but it depends on the context of the analysis. organisational meta models of workflow management systems are limited to a well documented domain with little product-individual differences. in this context a meta model evaluation provides the analyst with important information about the modeling capabilities of the methods analysed. process meta models, however, are only one instrument during the evaluation of workflow management systems. they are useful for the analyst to gain an overview about the modeling methodologies and to get acquainted with the individual systems, but they are not sufficient for a final decision for or against a specific system. the use of reference meta models, such as the organisational reference model can help the user find the aspects relevant to his decision. some extensions for existing modeling methodologies have been introduced, such as variable-based cardinalities, which enhance the suitability of these models. future work should emphasise this aspect. further fields of research are besides the inclusion of other workflow management systems the design of an inductively generated process reference meta model as well as the development of a catalogue of basic control flow elements. references batini, c., ceri, s. & navathe, s. b. (1992) conceptual database design, an entity-relationship-approach, redwood city et al. batini, c. & lenzerini, m. (1984) "a methodology for data schema integration in the entity relationship model"" ieee transactions on software engineering, 6, vol. 10, pp 650-664. 115 ajis vol. 6 no. 1 september 1998 chen, p. p.-s. (1976) "the entity-relationship model: toward a unified view of data", acm transactions on database systems, 1, vol 1, pp 9-36. derungs, m. (1997) kundenorientierte workflowprojekte, duv, wiesbaden. dunn, r. h. (1991) software-quality. concepts and plans, englewood cliffs. elmasri, r. & navathe, s. b. (1995) fundamentals of database systems, 2nd ed., benjamin/ cummings publishing. ferstl, o. k. & sinz, e. j. (1994) grundlagen der wirtschaftsinformatik. band 1, 2nd ed., vahlen (in german). caller, j. (1997) vom geschaftsprozefimodell zum workflow-modell, gabler, wiesbaden (in german). georgakopoulos, d., hornick, m. & sheth, a. (1995) "an overview of workflow-management: from process modeling to workflow automation infrastructure", distributed and parallel databases, 2, vol 3, pp 119153. hars, a. (1994) referenzdatenmodelle. grundlagen effizienter datenmodellierung, gabler, wiesbaden (in german). ibm flowmark modeling workflow. version 2 release 3. document number: sh19-8241-02, ed.: ibm germany, wien. jablonski, st. & bussler, c. (1996) workflow-management: modeling concepts, architecture and implementation, international thomson computer press. joosten, s. (1995) "conceptual theory for workflow management support systems". technical report. centre for telematics and information technology, university of twente, enschede. joosten, s. (1996) "workpad: a conceptual framework for process analysis and design". acm transactions on office information systems (under submission). kazman, r., bass, l., abowd, g. & webb, m. (1994) "saam: a method for analyzing the properties of software archietectures", proceedings of the international conference on software engineering icse'16. ed.: b. fadini, sorrento, pp 81-90. leymann, f. & altenhuber, w. (1994) "managing business processes as information resources", ibm systems journal, 2, vol 33, pp 326-348. nissen, h. w., jeusfeld, m. a., jarke, m., zemanek, g. v. & huber, h. (1996) "managing multiple requirements perspectives with metamodels", ieee software, 3, vol 13, pp 37-48. rosemann, m. (1996) komplexitatsmanagement in prozebmodellen, gabler, wiesbaden (in german). rupietta, w. (1997) organisation and role models for workflow processes, workflow handbook 1997, ed.: p. lawrence, john wilsey & sons ltd, chichester et al., pp 165-172. saeki, m. (1995) "object-oriented meta modelling", proceedings of the 14th int conference on object oriented and entity relationship modelling, ed.: m. p. papazoglou, berlin et al., pp 250-259. siemens nixdorf (1995) organisationsund ressourcenmanagement, version 2.0, muenchen (in german). siemens nixdorf (1994) workparty benutzerhandbuch, version 2.0, muenchen (in german). steele, p. m. & zaslavsky, a. b. (1994) "the role of meta models in federating system modelling techniques", proceedings of the 12th international conference on the entity-relationship approach er *93, eds.: r. a. elmasri, v. kouramajian, b. thalheim. berlin et al., pp 315-326. wfmc (1996) workflow management coalition terminology & glossary. document nr. wfmc-tc-1011, 2nd ed., bruessel. 116 ajis special edition electronic commerce november 1998 analysis of expectations on electronic grocery shopping for potential customer segments4 hejkkjla jukka kallio jukka saarinentimo tuunainen vjjrpikristhna helsinki school of economics. electronic commerce institute p.o. box 1210, fin-00101 helsinki, finland heikkila@hkkk.fi jkallio@hkkk.fi saarinen @ hkkk.fi tuunaine @ hkkk.fi abstract the special nature of groceries, especially perishables, challenges the possibilities of digital channels i.e. computers and networks to improve customer service. we report the findings of theme interviews on the potential of electronic commerce (ec) of groceries with 33 experts of grocery industry and ec, most of them senior executives, in 23 organisations in finland during the autumn of 1997. the specific areas of interest in the study are the predicted volume of ec in grocery shopping, the most potential customer segments, the anticipated benefits of ec to customers, and supplier strategies and infrastructure alternatives. benefits of ecs (electronic grocery shopping) are analysed for three selected potential customer groups in terms of phases in consumer buying process as well as two different infrastructure solutions. it seems that an egs build on top of current grocery industry infrastructure can only satisfy one major consumer group. in order to gain momentum egs needs a dedicated infrastructure within which logistic efficiency can be increased. introduction this article reports the results of a study on the electronic commerce (ec) of daily grocery goods on the net. the study of grocery shopping is of value, because groceries seem to be one of the most difficult objects of trade for electronic commerce: material flows are distinct from information flows, the number of potential customers is huge, and an average purchase consists of many items (e.g. gould and silberzahn, 1996). the diffusion of electronic trade of groceries is not only subject to customer behaviour but to national and trade area regulation, irreversible investments in logistics by current suppliers, and to the existence of standardised platforms for the electronic transactions, e.g. tjn/edifact. this is in contrast to the commerce of digital products (such as information services, computer games, video and gambling) that are easily accessible throughout the world, as the information about the product and the product itself are in digital format (choi et al., 1997). ec of daily groceries is also more difficult than electronic commerce of many other physical products like books or clothing, because groceries are prone to spoil, if not handled properly. the finnish case highlights this issue exceptionally well, because the technology is readily available to the finnish customers: most companies and many households are linked to internet. more than one third of households in finland already have computers, and the host to population ratio is 87.3 per 100 000 habitants, the highest in the world (telmo^, 1997). the density of servers and capacity of telecommunications lines are estimated to be high enough to serve the customer population. yet, the consumer services on the networks have not kept up with the technological possibilities: a fact that has urged us to launch this study together with the ministry of trade (kallio et al. 1997). the specific areas of interest of this article are the predicted volume of ec in daily grocery shopping, the best potential customer segments, the anticipated benefits of ec to customers, and suppliers strategies and infrastructural choices. we addressed the topics by theme interviewing the key informants from the supply side of daily shopping: representatives of current producers, suppliers, and potential service providers to the grocery market. before going further, we must ask: what is electronic commerce of daily grocery goods? to most of the interviewees it is about ordering a basket of commodities over digital network. however, there has been mail 4 this article is based on the paper presented in european conference on information systems, aix-en-provence, france, on the 7th of may, 1998 (heikkila et al., 1998a) 5 telmo ry, the finnish association for interactive network services, was established in 1992 to promote the development of telematic services in finland. telmo operates as a co-operative forum and working organisation of its members: the users of the services, service providers, network operators, system vendors and governmental bodies. 56 ajis special edition electronic commerce november 1998 ordering and other means e.g. electronic devices such as phones, faxes, etc., available for a long time. but after easy access to internet, it is feasible to provide information about products, place orders, etc. in a profitable manner. this is because all the information is in digital format for further analysis and processing for the development of future value-added services. the article is organised as follows: firstly we describe the research methodology: selection of key informants, the themes, and the data collection procedure. we then report the findings of the study in the context of trade of grocery goods in finland. in the following section we outline the ec services in the different phases of consumer's shopping process, and then describe egs and its benefits from the point of view of three selected potential customer groups. supplier strategies and two infrastructures as the basis for egs are also discussed. the potential benefits for different consumer groups are evaluated against these structural choices. the last section summarises and concludes the article. research method the study is based on theme interviews among the key informants around the supply chain of the commerce of grocery goods and related services during the autumn of 1997. in addition, we analysed the past development paths to the present structure of the trade of daily grocery shopping in finland. the key informants were chosen by the researchers based on the following criteria: first, they are related to the commerce of groceries, or related services (e.g., common carriers, upstream food processing industry, itvendors, and ec-marketplace providers). secondly, the individuals have been following closely, or are directly responsible for ec in their companies. and finally, they have insight to the current structure and forms of trade in the commerce of grocery goods. the saturation of the information structure of the responses was not used to determine when to cease data collection, because we wanted to cover the whole range of respondents. we were well received among the professional community: not one of the contacted companies turned down the possibility for an interview. another implication for the importance of the topic is that most of the interviewees held high rank positions in their organisations. thus the sample of our respondents represents the views of senior management from various industries related to the commerce of daily grocery goods in finland. the translated, detailed list of the 33 interviewees from 23 companies" is presented in appendix a. we also made two test interviews with the help of academic experts on the research topic in order to test and revise the approach, themes, and interview checklist. the interviewees were sent a list of themes in advance. english translation of the list of themes is in appendix b. the idea of the first theme was to cover particular cases, or pilots of ec that the interviewees have been following or are currently participating in. we also asked about their current efforts and future plans for ec. the most interesting topics to us were their visions on the trade volume in the near future, and on the segments of potential customers to ec. any hindrances or critical factors were also covered by the interviews. finally, we encouraged the interviewees to predict the effects on the industry structure and on the demand of labor in daily shopping and related services, if their estimates would come true. the interviewers worked mostly in pairs, sometimes in a group of four. the interviewers were the same researchers of the project that planned the questionnaire. the discussions during the interviews were transcribed to text allowing us to develop the measures used to analyse the data. the texts were circulated and analysed by the interviewers. the answers are tabulated, but as the sample is not random, and the cases are only 23 in number, we did not engage in any statistical analyses. the responses are used to triangulate the theme from various angles of the supply chain. findings this paper describes grocery, or as it is called in finland, daily shopping, so that the results can be interpreted in context. it then concentrates on the following four topics: predictions of the growth of the volume of ec of daily consumer goods in the near future, most potential segments of customers for ec of groceries, and the potential benefits to consumers and suppliers. 6 two producers of grocery goods, seven wholesalers, retailers, trade associations, seven providers of related services (logistics, financing, delivery), and seven it & telecommunications solution providers. 57 ajis special edition electronic commerce november 1998 outset of the research daily shopping in finland daily consumer goods are defined to be groceries or products with reasonably low unit prices that consumers are accustomed to buy in the same shop (kasso, 1995). groceries can be roughly divided into perishables7 and nonperishables^; nowadays they include also non-food commodities^ (ltt, 1981). in 1996 the industry grew about 3%, and was about usd 10.31 billion. hypermarkets continued to increase their market share from 15.6 to 17.8 % (ac nielsen, 1997) the growth has continued in 1997 (ac nielsen, 1997), and is expected to stay at 1-2% per annum until year 2000 (wires, 1997). most of shopping (in terms of volume) is cash-and-carry there is little delivery of goods to homes. an average purchase ranges from usd 5.5 at corner stores to hundred of usd at shopping centres, with the mean value of the purchase being usd 15 in 1995 (ltt, 1995). the shopping frequency of daily consumer goods is on average 4.6 times a week (ltt, 1997). the retail of daily consumer goods are classified into nine types of stores as shown in table 1 (ac nielsen, 1997). table 1. types of daily consumer goods outlets type of store hypermarkets department stores large supermarkets small supermarkets large self-service stores small self-service stores small stores specialised stores market halls all % of number of outlets 2% 2% 5% 15% 21 % 29% 13% 12% <1 % 100% % of sales 18% 8% 21 % 25% 16% 9% 2% 1 % <1 % 700% excluded from the table are kiosks, i.e. small convenience stores with unlimited business hours, and gas stations that sell approximately usd 1 billion daily consumer goods per year, accounting for almost to 9% of the market share (ac nielsen, 1997). the commerce of daily grocery goods in finland is largely similar to other european countries, except that the market is more concentrated, the population in the countryside has become sparser, and the efficiency of logistics has consequently played a major role. these special features deserve closer attention. the geographical distances have an effect on the cost of logistics, which, by international comparison are quite high (ministry of transport, 1994). however, delivery is nowadays more cost efficient, thanks to the development of highly sophisticated routing of transport and efficient logistics within large units. the greater distances are offset by larger units of transport, and less traffic on the roads, as compared to other european countries. for the past three decades the primary trend in finnish daily consumer goods industry has been to increase the floor-area of the outlets (koski et al., 1995; ltt, 1996). increasing floor-area is very investment intensive, thus the grocery industry is very concentrated as the wholesaler market structure is pure oligopoly: the largest companies directly control the maturity of the market. this concentration has led to an investment race of building massive warehouses and malls at the junctions of motorways, railways, etc., during the 1990's (koski et al., 1995). this trend in such large scale has no comparison in any other west-european country (santasalo, 1995). on the other hand, in 1980 there were almost 10,000 grocery shops in finland: by comparison, today there are only about 5,000 left, and the number is still expected to drop down to about 4,000 (pulkkinen, 1998). another side effect of the concentration is that it is difficult for an up-stream producer to get into the consumer market: it has to convince the wholesaler about the quality of the product and about its capability to deliver in quantity if required. also the retailers of the wholesalers are in an awkward position, because the superior efficiency of the big wholesalers' logistics and financial resources are reducing the number of economically feasible alternatives (most often it is the wholesaler who invests in a new store, mall, or shopping center). the bargaining power of the wholesaler does not make life any easier for an individual retailer (hyvonen, 1990). 7 perishables are the commodities with shortest shelf-life (e.g., bread, vegetables, fruits, some dairy products, fish). semi-perishables have longer shelf life, e.g., ready-made, processed food, frozen food. 8 non-perishables include canned and dried food stuff, beverages (excluding wines and spirits), 9 household products, cosmetics, magazines, tobacco, etc. 58 ajis special edition electronic commerce november 1998 the finnish grocery goods market used to be highly protected and subsidised before joining eu. there were times when the prices were regulated and determined by the state and the farmers' union. after the collapse of soviet trade and the recession at turn of the decade, the borders opened as finland joined eu, leading to a rise in imports up to usd 5 billion in 1995. the potential demand for wider assortment and selection of commodities has been satisfied with this increase of import, and the prices of daily commodities to the customer has reduced on the average about 20%. the food processing industry has recovered from the recession, and been able to improve its efficiency and regain the exports especially to russia, and baltic countries. the structural changes and last recession has led to the largest movement of rural people to towns and cities since the 1960s and 1970s 10. this has also boosted the growth of the retail stores together with the increased mobility of the consumers, motorising of the society. it has also increased levels of consumers' demands and purchasing power as well as industry's strive for rationalisation and the emphasis on price as a means of competition instead of service (kilpio & pantzar, 1993; ltt, 1996). against this backdrop, it is easy to understand the keen interest in the new forms of trade among the suppliers. prediction of the growth for ec diffusion of the internet and the availability of cheap computer equipment has created a new type of customer who wants an easy and convenient alternative to regular shopping in supermarkets (schuster & sporn, 1998). according to a recent study by andersen consulting, the number of us households using on-line services to buy food and other household goods and services is expected to grow from fewer than 200,000 now to 15-20 million by 2007 (kutz, 1998). our respondents were asked to estimate the volume of electronic commerce of grocery goods within the next few years. the estimates varied, as one could expect, notably. the general impression was, yet, that ec will account for 5-10% of the total volume of the groceries in 2010. the most optimistic forecasts were found among potential providers of new services to the electronic commerce, and the most moderate among the current retail chain masters who operate also as wholesalers. these estimates are lower than some other common publicly presented estimates. there are a number of reasons for this: first of all, many of the other estimates are not about some specific sector of trade, thus the object of assessment is simply different. secondly, the trade of grocery goods is tightly regulated and controlled in terms of preservation and quality of the delivered goods. despite the advances in packaging and material handling, the logistics are not up to date with the it-possibilities. and finally, customers find the current delivery channels unproblematic (ltt, 1995), with the only major problem being the limited hours of operation of the grocery stores due to the national legislation. furthermore, past commentaries on the potential impact of the internet on consumer marketing have typically failed to acknowledge that consumer markets are heterogeneous and complex and that the internet is but one possible distribution, transaction, and communication channel in a world dominated by conventional retailing channels. (peterson et al., 1997). potential customer groups the andersen consulting study identified six major groups of potential on-line grocery shoppers, based upon survey respondents' attitudes toward time, shopping and technology (kutz, 1998). these groups were shopping avoiders, who dislike grocery shopping; necessity users, who are limited in their ability to go to the store for some reason; new technologists, who are typically young and comfortable with technology; time starved, who are insensitive to price and will pay extra to free up time on their schedules; responsibles, who have available time and who get an enhanced sense of self-worth from shopping; and traditional shoppers, who are older, avoid technology and genuinely enjoy shopping in a store. all except the traditional shoppers group showed at least some willingness to use on-line grocery shopping services, and most of the groups showed strong interest (kutz, 1998). these households will represent a wide range of demographics and will spend as much as $85 billion on groceries and related purchases through on-line services (kutz, 1998). an austrian study identified especially younger and better educated people in urban areas to be more inclined to use online shopping (schuster & spom, 1998) the interviewees in our study were asked about potential customer groups for electronic commerce of groceries. a clear tendency of the respondents was to form the consumer groups on the basis of price or time savings, or improved service. the groups below are derived directly from the interviews. table 2 summarises the expected benefits for each customer segment that was identified. j0 finland is known for its dispersed population. the situation is changing in cycles. in the 60's large 44 % of the population lived in the countryside (due to the resettlement after ww2), in the 70's 36 %, and at present about 19 % (statistics finland, 1997a). moving from countryside to the cities is partly compensated by the increasing leisure time habitation in the lake district and in lapland (statistics finland, 1997b). 59 ajis special edition electronic commerce november 1998 table 2. potential customer groups of electronic grocery shopping potential customer groups 1. sub-urban family commuters around large cities 2. bargain seekers in large cities 3. wealthy adults seeking for high quality services 4.rural area dwellers 5. elderly, or disabled people relying on social services 6-computer literate generation 7. business-to-business customers expected benefits • reduced time • reduced effort • placing orders independently of time of the day • lower prices • price comparability • value added services • broad selection of special and high quality products • accessibility of shopping services • broader selection of products • accessibility of shopping services • broader selection of basic products • indirect cost savings to the public sector • first, easy access to the shopping services • later, same benefits as groups one and two • cost savings • efficiency • new possibilities for services * n 17: 1 producer 5 trade 5 related 6 it&tc 1: 1 trade 2: 1 trade 1 it&tc 5: 3 related 2 it&tc 12: 1 producer 5 trade 4 related 2 it&tc 9: 2 producers 1 trade 4 related 2 it&tc 4: 1 trade 2 related 1 it&tc • number of mentions by the interviewees (trade = wholesalers, retailers and trade associations; related = logistics, financial and alternative service providers; it&tc = it and telecommunications solution providers) for the purpose of this paper we will concentrate on the three groups ̂ of customers mentioned by the interviewees (table 2): two most potential groups of customers in the early stage of egs: sub-urban family commuters and the elderly or disabled, and as third group the bargain seekers in large cities. the third group is, theoretically speaking, the segment that is expected to benefit most from the introduction of egs, as the customers can get better price and quality information both from the providers and other consumers as well (hagel & armstrong, 1997). in other words, unlimited information on the web makes price discrimination difficult, if not impossible (provided that there is competition). yet, on the basis of our interviewees, this kind of benefits to the customers are likely to be realised only later in the diffusion of egs. sub-urban family commuters around large cities this group consists of working families with strict time constraints. therefore they are expected to value the new services of electronic commerce in reducing the time and effort needed for daily shopping. flexible timing in placing the orders via networks, home deliveries, and pick-up terminals would be most useful for them. this was the group that most of the interviewees mentioned first. elderly or disabled people relying on social services many elderly or disabled people are willing to spend their lives at home. if they can continue living at home, the society saves in costs, because of the decreased need to invest in hospitals and nursery homes. in addition, skilled nurses for this group are in short supply, and they spend a great deal of time taking care of daily tasks of 11 other groups mentioned in the table 2 are explained in detail in heikkila et al., 1998a. 60 ajis special edition electronic commerce november 1998 the nursed. electronic commerce and related services would increase the capacity of these services significantly (more detailed description of problems of the current infrastructure, alternative service models and benefits, see heikkila et al., 1998b). importance of these issues will grow as the large post-war generation is getting to the age of retirement at the turn of the century. bargain seekers in large cities the expected late adopter group looking forward to ec is the bargain seekers. this is basically the same group of people taking advantage of the sales and special offers today. this group will become customers of ec only if they can get better prices. therefore, easy price comparisons, enabled by search capabilities are essential to this group. according to the it providers these features are technically possible, but for obvious reasons the chain masters do not intend to implement them. thus, this customer group was discussed in almost all interviews, but rarely considered among the most potential ones. services of ec in the different phases of the consumer's buying process the paper next outlines the consumer buying process of groceries, and then describe egs from the point of view of the three customer groups: sub-urban family commuters, bargain seekers, and the elderly and disabled. the paper illustrates the suitability of the egs-solutions to meet the needs of the specific customer groups and to solve the problems in their current grocery shopping process. consumer buying process of groceries as a starting point for the interviews we used the six stage consumer shopping process of groceries (gould, 1997), which differs in many parts from the models describing consumer behaviour in context of selecting a single product. while the traditional models (such as aida (see e.g. kotler et al., 1996)) start with attention and searching information about the desired product or products, the consumer purchasing a basket of groceries makes the planning in a more routine manner and selects the store based on previous experiences, the contents of the basket, the image of the price level, etc. (wilkie, 1990). however, soon in the interviews it turned out that the model should be amended to cover additional considerations such as the creation of the image of the store, price level information, and recycling. after the store is decided upon, the current supermarket shopping process for groceries continues step by step basically so that the consumer transports herself to the store, which is in convenient reach on the way home from work, or at the city center. the store has often recycling facilities for paper and board, glassware, cans, etc. she takes a shopping cart, and starts to pick up the foods according to a prepared shopping list, or makes the decisions ad hoc in the store. most often the customer picks up the goods by herself. only in special cases the store personnel picks any goods for the consumer. when all the planned and impulse buys have been collected in the cart, the customer goes to the cashier, moves the goods from the cart onto the cashier's table, the cashier reads the optical ean-stripes. the consumer pays for the goods, normally on debit card, or on credit, and packs them by herself in bags. finally, the consumer carries the goods home by him/herself, and either starts consuming them or stores them for later use. the steps of the buying process are depicted in table 3 to illustrate some observations on the grocery shopping process. based on our interview data, it seems that the process will be changed due to the introduction of the electronic channel. and most strikingly, the chances of splitting supply chain into more specialised parts seems high, which is in contradiction to the predictions of the integrating supply chain (e.g., benjamin & wigand, 1994). the paper subsequently analyses the shopping process of groceries of the commuters, the bargain seekers, and the elderly in turn, and highlight the potential benefits creating cost savings or value-add to the customers. in the analysis we describe the alternative consumer buying processes for grocery shopping. grocery shop services appreciated by the consumers the interviewees mentioned services which will be appreciated by the different consumer groups. a clear tendency of the respondents was to. form services that are associated to the consumer groups on the basis of price or time savings, or improved service. the table illustrates this, and is derived from the interviews by the researchers. the consumer groups should be understood more like consumer roles. consumers may actually be in several roles during their lifetime or even during their weekly life. 61 aj1s special edition electronic commerce november 1998 table 3: consumer buying process of groceries steps 1. recognise/identify needs 2. information search and evaluation of alternatives 3. make a shopping list 4. choose the grocery store 5. travel to the chosen store 6. walk in the store, i.e. select and pick up the goods from the shelves to the can wait in the service line, be served by the service personnel 7. operate in the checkout point, i.e. wait in the line, be served by the cashier, pack purchases 8. payment 9. transport of the purchases to home 10. use, maintain, evaluate, feedback (complaints, return) and dispose or recycle observations in finland the average weekly visiting rate of grocery store was 5.4 in 1980, but has decreased to 4.6 in 1994 (a.cnielsen, 1994). ninety percent of the groceries are sold via self-service retail outlets (a.c. nielsen, 1997), as many consider current grocery shopping convenient (spare & pulkkinen, 1997). 75 percent of grocery consumers ignore newspaper ads before shopping (wilkie, 1990) 70 percent of the grocery consumers do not prepare a shopping list (wilkie, 1990) eighty percent of households buy their daily consumer goods almost always from the same store (spire & pulkkinen, 1997), i.e. few buyers choose daily or weekly a different store. however, price level, service and product range most important criteria the choice. about half of the trips to the store are done by car, the other half is divided into other means, e.g., by foot, public transport, etc. ( lit, 1995). over all consumers and products in the supermarket, the decision times obtained ranged from almost instantaneous to 5.5.minutes, but almost all purchases were at the low end of the range (wilkie, 1990). up to 30% of the purchases are impulse buys (marjanen, 1997). customers find queuing the most annoying thing in current purchase process (spare & pulkkinen, 1997). there are more than one debit card per a finn. debit cards are used annually in 177 million transactions (pankkiyhdistys, 1997), private credit cards in 50 million transactions (luottokunta, 1998). see step 5 above. in finland 98% of bottles, over 80% of cans, and 61% of paper are recycled (keraysviesti, 1997). the current finnish infrastructure seems to meet the needs of the commuters 'customer group'. the needs of the shopping process stages of bargain seekers, however are not as often met, and even less often the needs of the elderly and disabled. the next section discusses the potential benefits of egs first in general terms, then by the customer groups. expected benefits of egs to the customers in the austrian study the consumers saw the major benefit of online shopping as factors of convenience: possibility to save time and the independence from the store hours were named as major advantages of buying groceries over the internet. additionally, the internet's ability to allow easy comparison of prices and to offer up-to-date product information were appreciated (schuster & sporn, 1998). generally, the current customers are both expected and willing to pay the extra charges for this convenience (kalakota & whinston, 1997). the respondents in our study described their expectations about the potential benefits to the customers from electronic grocery shopping. they include direct time and cost savings from improved purchasing processes and better information about the prices and other features of the goods. on the other hand, some of the respondents considered that the customers, or some subset of customers, might be willing to pay for the new value-added services. 62 ajis special edition electronic commerce november 1998 table 4 services appreciated by the consumers in the buying process process steps 1. recognise/identify needs 2. information search and evaluation of alternatives 3. make a shopping list 4. find source i.e. choose the grocery store 5. travel to the chosen grocery shop 6. select and pick up the goods from the shelves to the cart, getting served by the personnel at service counters 7. operate in the checkout point, i.e. wait in the line, be served by the cashier, pack purchases 8. payment 9. transport of the purchases to home 10. use, maintain, evaluate, feedback (complaints, return) and dispose or recycle commuters family shopping history, knowledge of lacking groceries and predicted need familiarity of the store 'basic' lists and 'customised' shopping lists (e.g. recipes) according to the needs convenient location and premises, image, quality, additional services ease of access by car, good parking facilities alternative routes through the store according to the shopping situation (e.g. with/without kids) short queuing times, help in packing multiple payment methods, i.e., cash, credit cards, debit cards, and credit account. short distance and easy access to the parking facility no data available bargain seekers methods of planning and following budget easy access to price information of multiple stores shopping lists for several stores price level and distance (shopping route) easy access either by car, by foot or by public transportation clearly visible guidance like product group signs, price tags, sales offers and other in store information. cash and credit card availability easy access (see step 5) no data available elderly and disabled local newspapers and direct mail ads as information sources, (these steps are sometimes done together with a home helper) familiar products and previous shopping lists real choice between approximate stores (now the nearest store, or the one chosen by the municipality) easy access by foot (sometimes the shopping is done by the home helper on behalf of the person) prefer service to picking up. if no help in picking, short routes through the basic supplies help in packing cash and credit account home delivery services no data available time savings the reduction of time and effort are expected to be the most obvious benefits to the customer thanks to ec of groceries. consumers save time and have more control because they can shop from home or work whenever they want (e.g. kalakota & whinston, 1997). all interviewees mentioned time savings as a major benefit. these savings will be achieved in the forms of shorter ordering time, reduced need for commuting, and reduction in time spent in picking up the products and the payment of them. cost savings the tremendous growth of the internet and the www is bringing significant changes to the economics of marketing and is likely to lower prices for retail consumer products (lee, 1998). consumers can save money, despite the extra overhead, because they can do better comparison shopping, and buy fewer impulse items than 63 ajis special edition electronic commerce november 1998 they would if they shopped at a real supermarket (alba et al, 1997; hoffman et al, 1996; kalakota & whinston, 1997). by lowering the cost of searching for alternative and substitute products in a commodity-like bargaining atmosphere, electronic markets encourage greater price competition for relatively generic products (elofson & robinson, 1998; lee, 1998). nevertheless, with the advent of systems such as bargainfinder (which searched for lowest-priced cds on the web) some suppliers have quickly responded by making their formats difficult to search in order to prevent price-shopping (elofson & robinson, 1998). interviewees believed that the cost savings can be achieved by gaining access to better information on the prices of the daily basket of groceries, and consequently, from keener price competition. price decline can also be due to the avoidance of some part of the present cash-and-carry logistics. in addition, lower total expenditure on grocery goods can be achieved also indirectly by getting better control on consumption with the information gathered and reported by the shop keeper, or an agent. many of the interviewees could not imagine the breakthrough of egs until the costs will drop to the level of traditional shopping. most of the current online consumers are, however, described as being interested in making better-informed purchases more quickly, rather than necessarily getting the lowest price (the economist, 1997). value added services in planning, selecting and placing orders egss can offer a variety of value added services for planning, executing and placing the orders that are new to the current shopping process. shoppers can browse through virtual aisles based on the way they want to shop, searching for foods offered as specials or scanning the dairy counter (kalakota & whinston, 1996). within categories, customers can choose to have the items arranged alphabetically by brand or sorted by lowest cost per ounce, package size, unit price, or nutritional value (choi, et al., 1997). these services may also include advanced facilities for information processing to make planning for shopping more effective, such as ready-made shopping lists, or capability of building a personal shopping list for one-shot or repeated ordering (kalakota & whinston, 1997). moreover, recipes can be turned to shopping lists according to the number of persons joining the meal. if the supply of electronic commerce develops as expected, far more exhaustive selection of goods than in the traditional shopping environment is also available to the consumers through the electronic channel. there is potential of wider availability of hard-to-find products and wider selection of items due to the width and efficiency of the channel (hoffman et al, 1996). in 1966, the number of articles in a selection of an average store was 2600, in the beginning of the 1990's 6100. today in a finnish hypermarket there are on average about 10000 to 12000 articles (daily consumer goods; all products included about 20000 articles), in germany about 15000, and in the united states up to 25000 (kilpio and pantzar, 1993). clearly presented special offers and possibilities to price comparisons between competing stores will provide opportunities to cost savings, at least in theory. time and place independence egs provides consumers with new and enhanced abilities to shop anywhere, anytime, and any place (sherman & topol, 1996). ordering can be done from distant places around the clock. many alternative stores are accessible, which makes it possible to order the bundle to a local store close to a remote location (e.g., a summer cottage). new forms of delivery in the early stage of egs, two forms, of delivery are used: home delivery or picking up from the store. these forms do not serve all customers well and therefore new solutions, like drivein-type pick up terminals, neighbourhood pick up stations (perhaps in the connection with a small convenience store or gasoline station) or unmanned low temperature storage facilities outside the apartments, should be developed in the future. benefits for the three selected customer groups as illustrated by the literature excerpts and expert opinions above, electronic channels are expected to create new opportunities to serve and create value-added to the customers. ultimately, it can provide wide selection at low prices 24 hours a day. however, the specific features are not equally important for every consumer groups, as is implied already in table 4. therefore, there is a need to systematically evaluate the importance of the service features the customer segment that values most of the time saving is the commuters. this is natural as their families are typically in an active stage of life; parents working full time and children participating in leisure activities. thus, travelling to the store has never been a problem for them, as they are commuting and taking their children to their activities. the shorter and time independent ordering time are highly valued, if the ready made bundle of groceries can be conveniently picked up (saving the burden and time of selecting and picking up the goods). 64 ajis special edition • electronic commerce november 1998 home deliveries are actually not so important, as the time window of the delivery is difficult to synchronise to family activities. costs and cost control are important, but not the primary issue, as the needs and search for variety are driving to the shop. thus customised shopping lists, or lists created from recipes are of interest to them, which eventually also means the need for broader selection. for the bargain seekers a more planned approach to shopping is needed to keep the grocery expenses under control. thus, they value the information facilities that help in planning, comparing prices, and finding special offers from a number of stores are high. ordering in advance is not the primary issue, as they are willing to do the legwork of getting to the stores, picking up the sales goods, packing and taking them back to home by themselves. as a consequence, home delivery may not be valued by them either, particularly if it presents an additional cost item. the elderly and disabled form a two faceted group, as they can be self-supporting or in need of help (temporary or permanent). the current infrastructure of grocery shopping does not meet their needs of proximity, service, and trusted relationship at a favourable price. thus, if they are able to do the shopping by themselves, they would appreciate both customised and standard shopping lists and prefer home delivery to other means of getting the basket of purchases. service and trust is often more important than money, but a credit account is preferred. but if the customer is relying on community help, the situation is different. currently, the customer is authorising the home helper to do shopping for him. with the help of egs diey can together plan and order the goods online either to be delivered to home, or being picked up by home helper, or a trusted third party, e.g., by coupling it to the meal services. there is no need to hand over the cash to the home helper for shopping, and the electronic channel can provide a feedback channel about the service and quality of products. thus, in this case the customer can benefit indirectly from the time savings of the home helper, who can then spend more time with the elderly instead of doing shopping. this would also mean the possibility to get a real choice of the store; currently it is normal that the authorities make a long-term contract with a local entrepreneur. but how can these benefits realised, if the suppliers are transforming to egs? in the following section we depict suppliers' structural and strategic view on egs. suppliers strategies and infrastructure alternatives as consumers grow accustomed to using on-line services for ordering groceries and related purchases, traditional retailers and manufacturers will face pressure to adapt to changing consumer demands (kutz, 1998). in this section it is outlined how the strategies of the electronic grocery stores are derived from the benefits to the supply chain estimated by interviewees. the strategies can be based roughly on two categories of infrastructure: those which could be realised on top of the current bricks-and-mortar retail structure (e.g. during the transition period), or on building on new structural solutions and capabilities of the supplier. the respondents discussed the advantages of ec in the present situation and then moved on to their vision on what the advantages might be in the future. these are documented in detail in kallio et al., 1997. infrastructure alternatives for egs egs based on current infrastructure one advantage for current retailers revealed by the andersen consulting study is consumers desire for their current, primary grocery stores to also be their egs providers simply because they prefer a name they know and trust this advantage will not last long, however, as today's emerging consumer direct providers become more established (kutz, 1998). when egs is based on either existing hypermarket, supermarkets, retail, or even wholesale outlets, only the virtual marketplace is created in the electronic network. the upstream logistics remain mostly the same all the way down to the retail outlet, where part of the personnel has to be dedicated to picking, packing and delivering the goods, which may actually help in leveling the use of labor during the workday. some parts of this work can be outsourced as well. however, in this structure there will be some redundant work: the goods are first stored in the shelves of the store or wholesale outlet, from where the personnel collects them for further delivery to the customer. this manner means additional costs to the retailer and consequently higher prices, or at least the fees for delivery, to the customers. building electronic commerce using the current infrastructure as a platform means that aside from improved leveling of resources and shop floor utilisation the advantages can be achieved primarily by enhanced marketing and improved information on purchasing behaviour. dedicated egs infrastructure a dedicated egs requires totally new design of the logistics system and layout of the store. this is because the current infrastructure is built on customer self-service, and the stores are to attract customers for impulsive buying. they are not suitable for efficient pick-up as such. 65 aj1s special edition • electronic commerce november 1998 it became obvious during the interviews that the current delivery channels of groceries cannot realise the potential of egs. according to our respondents, there seem to be opportunities emerging for new forms of logistics at warehouses (for packing, loading, pricing, collecting and paying), and at retail outlets. ec will also bring along new types of home delivery services. new dedicated platforms are expected to emerge as a basis for a new infrastructure. for the major players in the industry there are possibilities to win new markets and customers without sacrificing performance. it seems evident that building egs on the current infrastructure is a short term solution for the transition period before more efficient logistic solutions. only dedicated solution can hope to meet simultaneously all the prerequisites of large scale egs, i.e., lower prices, wider selection, and around-the-clock opening hours. in a large enough solution also non-technically oriented consumers can be served by providing different order and delivery channels, e.g. call centers, or fax order services. differences of egs structures from the customers' point of view time, and to some extent also place independence are distinctive features of egs. both egs infrastructure solutions represented above can offer the same benefits in terms of reduced time and effort in shopping: ordering times are shorter, there is no need to travel to the store and no time is spent in collecting the goods and queuing at the cashier's desk. home (or office) delivery is, at least so far, the only delivery choice in the existing egss. with a dedicated structure other types of solutions, such as drive-in-type pick up terminals or manned or unmanned pick up terminals or facilities can be built more efficiently. this option would be particularly useful to the time pressed commuters. new value adding services to aid in planning, such as standard and customised shopping lists, recipe services, additional information on nutritional values, are key features in egs, regardless of the infrastructure solution. in addition, independently of the infrastructure solution, the consumer can gain better control on the costs of grocery shopping with egs. the total cost of a basket of groceries, however, can be less than in a traditional brick-and-mortar shop only in a dedicated egs solution, within which the extra costs of inefficient logistics can be eliminated. lower costs are particularly valued, by definition, by the bargain seekers, as well as at least by some of the elderly and disabled. the commuters are likely to be less sensitive to the price differences. there is, nonetheless, a major difference in the number of products in the selection. in an egs built on top of an existing outlet, the selection is smaller or at the most it can be as large as in the traditional store, whereas in the dedicated structure the potential selection can be significantly broader. the broad selection is valued especially by the commuters, as well as some of the elderly and disabled. thus, it seems that there is a need for the new dedicated egs solution in order to serve different consumer groups. when electronic commerce diffuses more in grocery markets with large numbers of buyers and sellers, it can improve the welfare of both the sellers and the buyers by increasing market efficiency, rather than transferring welfare from sellers to buyers. (lee, 1998). in our opinion, in a competitive environment this calls for the efficiency increase in logistics of the value-added services (pick-up, delivery) suggested by the dedicated structure. summary and conclusions this analysis of the potential for electronic commerce of groceries is based on theme interviews of 33 people in 23 organisations in finland during the autumn of 1997. the specific areas of interest of the study were the predicted volume of ec in grocery shopping, the most potential customer segments, and the anticipated benefits of ec to customers and suppliers. the key findings indicate that the interviewees estimate the ec of grocery goods to reach 5-10% of the total volume within the next twelve years (estimates varying, however, between 2% and 20%). the most potential groups of customers in the early stages of ec of groceries are likely to be found among suburban family commuters, elderly and disabled people, and computer literate youth. the rural area dwellers, high-income middle-aged people and some business customers (e.g., commercial and public catering units, small restaurants, grills) were also mentioned. the differences between respondents' views of the potential groups are, however, obvious. basically, the anticipated benefits to the customers are savings in the current shopping time and effort, new value-added services or lower prices. those operating in a direct customer contact emphasise the first two, and service providers see also lower prices as a driving force. yet, only few respondents mentioned the bargain seekers as the beneficiaries of ec. the emergence of value-added services and lower prices are evidently subject to fundamental changes in ec platforms, logistics and market structure; not to a simpler delivery chain as suggested by the literature, but more complex chain of value-added service providers. if, in the near future, the customer orders would be received electronically, the information systems for grocery shopping could be integrated to the current supply chain. this would complement the current logistic processes of collecting and delivering the goods. the rationalism for this development path lies in the fact that grocery 66 ajis special edition electronic commerce november 1998 industry in finland resembles an oligopolistic market giving the major players such a strong bargaining power, that it is extremely difficult for the new entrants to enter the market, or outperform the chain masters in efficiency. in addition, as the current retail chains would not gain from rapidly changing structure, the modest growth and conservative development of ec in the grocery industry is easy to understand. as soon as customers gain experience in electronic shopping, they are likely to call for better services and lower prices despite the demanding logistics involved in grocery shopping. by then, it is most probable that software for price comparisons between electronic grocery shops will appear. furthermore, the demands of the numerous segments of potential customers are different, or even contradictory. polarisation of markets might follow, i.e. we may have bigger chain masters than ever, but on the other hand there is substantial potential for a decentralised market structure, especially concerning logistics, retail outlets, and value-added services. if the current grocery chain masters developed the dedicated electronic grocery shops, they would end up cannibalising their own shop-keepers. possible developers of these dedicated electronic channels could be small regional or local entrepreneurs or a big international chain with its own procurement sources and good reputation to attract and convince the consumers. given the existing infrastructure and the forecasted 10% share of the daily shopping for ec is an incentive high enough to attract various investors into ec-development within the next years. acknowledgments the authors wish to thank ministry of trade and technology development centre (tekes) for financial support, and the organisations for their willingness to participate in the interviews. references ac nielsen (1997) newsletters 1/97. alba, j., lynch, j., weitz, b., janiszewski, c, lutz, r., sawyer, a. & wood, s. (1997) "interactive home shopping: consumer, retailer, and manufacturer incentives to participate in electronic marketplaces", journal of marketing, july, vol 61, pp 38-53. benjamin, r. i & wigand, r. (1994) "electronic markets and virtual value chains on the information superhighway", sloan management review, winter 1994, pp 62-72. choi s.-y, stahl d.o. & whinston a.b. & (1997) the economics of electronic commerce. the essential economics of doing business in the electronic marketplace, macrrullan technical publishing, indianapolis, in., usa. the economist may 10 1997, pp 3-26. elofson. g. & robinson, w.n. (1998) "creating a custom mass-production channel on the internet", communications of the acm, march, vol. 41, no. 3, pp 56-62. gould, j. & silberzahn, p. (1996) electronic home shopping, a review of evidence and expert opinion from the usa and uk, centre for marketing working paper, london business school, no. 96-802. hoffman, d.l., novak, t.p. & chatterjee, p. (1996) "commercial scenarios for the web: opportunities and challenges", journal of computer-mediated communication, vol 1., no. 3 hyvonen, s. (1990) integration in vertical marketing systems: a study on power and contractual relationships between wholesalers and retailers, doctoral dissertation, helsinki school of economics, series a:72. hagel, j., and armstrong, a., (1997) net gain: expanding markets through virtual communities", harvard university press, usa. heikkila j., kallio j., saarinen t., and tuunainen v., (1998a) "diffusion of electronic grocery shopping: expectations of current suppliers and potential service providers in finland", in the proceedings of the 6th european conference on information systems, june 4-6 1998, in aix-en-provence, france, baets w.r.j., (editor), university of aix-marseille hi, vol. i, pp 218-232. heikkila j., kallio j., saarinen t., and tuunainen v.k., (1998b) "ec of groceries for elderly and disabled: comparison of alternative service models", in the proceedings of the 21st information systems research seminar in scandinavia, 8-11 august, saeby s0bad, denmark, buch, n.j., damsgaard, j., eriksen, l.b., iversen, j.h. & nielsen, p.a. (eds.), department of computer science, aalborg university, vol 1, pp 337350. kalakota, r. & whinston, a.b. (1996) frontiers of electronic commerce, addison-wesley pub co. kalakota, r. & whinston, a.b. (1997). electronic commerce, a manager's guide. addison-wesley. kallio j., saarinen t. & tuunainen v.k. (1997) "elektroninen kaupankaynti paivittaistavarakaupassa potentiaaliset kuluttajaryhmat, jakeluratkaisut, rakenteet ja tyollisyysvaikutus (electronic commerce in grocery shopping potential consumer groups, forms of delivery, structures and employment effects)", ltt series b 137, helsinki (in finnish). 67 microsoft word vol 15 2 final.doc australasian journal of information systems volume 15 number 2, 2009 73 modelling the interactions among factors that influence successful computerisation of small businesses gerard j fogarty faculty of sciences university of southern queensland email: fogarty@usq.edu.au bruce armstrong school of multimedia and information technology southern cross university email: bruce.armstrong@scu.edu.au abstract identification of factors that contribute to computer-based information systems success is of crucial importance to the small business sector. central to this process is the task of building conceptual models of interacting variables that contribute to success. the present study used structural equation modeling to test a model that embraced four broad groups of variables, including background characteristics of the organization, background characteristics of the chief executive officer, decision making processes, and a cluster of variables relating to the performance of the system itself. the outcome variable was user satisfaction. a self-report instrument called the implementation survey for computer based information systems (is-cbis) was used to collect data from 171 regional small businesses located on the east coast of australia. results showed that, although the performance of the system was the immediate determinant of satisfaction, the background variables had both direct and indirect (mediated) effects on user satisfaction. these findings emphasise the importance of going beyond the immediate surrounds of a computing environment if one wishes to explain the factors that influence cbis success in small businesses. introduction many small businesses are improving their competitiveness through deployment of information systems and information technologies. information systems are now essential for small business innovation (redoli, mompo, garcia-diez, & lopez-coranado, 2008). properly handled, the investment will improve the competitiveness of the company. badly handled, the investment will handicap the company and perhaps even lead to its closure. it is therefore not surprising to see an increasing amount of research devoted to identifying the factors that lead to successful computerisation. despite these efforts, a search of the literature indicates that there are still gaps in australasian journal of information systems volume 15 number 2, 2009 74 the knowledge relating to success factors in small businesses, especially regional small businesses (patullock, 2005; schubert, fisher, & leimenstoll, 2007). small businesses typically lack the expertise and financial power of larger businesses, both of which are major impediments when it comes to adopting information systems. regional small businesses face additional concerns and experience different pressures to their metropolitan counterparts, especially in relation to backup support and servicing. the main aim of the present study was to identify and model the network of factors contributing to successful implementation of computer-based information systems (cbis) in small wholesale and manufacturing businesses in a regional area of australia. a second gap in the literature concerns the fact that many studies have concentrated on a small part of the cbis success chain, focussing on the decision to adopt a cbis, the operational characteristics of already-purchased systems that contribute to satisfaction, or the measurement of satisfaction itself. technology adoption and implementation is a multistage process. the focus of the present study was the stages following the decision to adopt cbis through to the point where the businesses were able to rate their satisfaction with the cbis they had adopted. the key constructs investigated were the characteristics of the chief executive officer (ceo), organisational characteristics, the decision criteria used in the acquisition and implementation of cbis, and the operating characteristics of the system installed (e.g., whether it produced the required information). the study used survey methodology in conjunction with structural equation modelling (sem) to test implementation success models. we begin by stating our definition of small business and then review the literature on cbis success. the term “small business” is not the only one used to describe this section of the economy. the term “small and medium enterprise” (sme) is also widely used. throughout this paper, we will refer to small business but, for convenience, use sme where appropriate in our review of the literature. in australia, the widely accepted definition of small business is still that offered by the australian bureau of statistics (abs, 1995). this definition categorises manufacturing small businesses as those with fewer than 100 employees and all other categories of small businesses as those with fewer than 20 employees. the abs (2006) estimated there were over one million small businesses operating in australia in 2004. these small businesses employed over 34% of the nonagricultural, private sector workforce, inclusive of owners. studies from neighbouring countries emphasise the importance of small businesses to their economies and the increasing adoption of cbis in their small business sectors (e.g., kartiwi, 2006; locke, 2001). clearly, the sector is a very important part of the australasian economy (patullock, 2005). despite the positive nature of these growth statistics, rapid advances in information technologies and benefits obtainable through efficient and effective use of these technologies is often not realised in the small business sector. a decade ago, small businesses lacked technical expertise in information systems and related technologies (chau, 1994; burgess, belcher, paull, & singh, 1997) and it seems that the situation has not changed in the intervening years with more recent studies reporting that small businesses rarely position themselves close to the cutting edge of technology and generally don't have the time, resources, or expertise to develop appropriate plans to incorporate technologies into their business operations (morgan, colebourne, & thomas, 2006; lawson, alcock, cooper, & burgess, 2003). this problem is not confined to cbis adoption, but to any situation where small businesses take on new systems or technologies (chong, 2007). consequently, most small businesses use information systems technologies for operational support rather than for strategic purposes (harindranath, dyerson, & barnes, 2007). alongside these observations, a growing body of literature is adding to our knowledge of what factors will help businesses to adapt successfully to computerised work environments. in the paragraphs that follow, we review these key factors. australasian journal of information systems volume 15 number 2, 2009 75 background (distal) factors contributing to cbis success when examining the factors that contribute to cbis success, some researchers (e.g., delone & mclean, 1992, 2003, 2004) have focused on the characteristics of the is system itself, such as the relevance of the information provided. we call these proximal factors to distinguish them from the more distal factors identified by other researchers, such as the broad economic landscape in which the business operates (e.g., thong & yap, 1995). a distal factor is therefore one that is likely to have an indirect effect on cbis success. the present study covered both proximal and distal factors in an attempt to gain a better understanding of the various influences on successful cbis implementation. the terms proximal and distal can have another meaning in this context. as redoli et al. (2008) pointed out, technology is usually adopted in stages and an sme can be at different stages of implementation for different parts of a cbis system. the first stage is the decision to adopt a cbis. in their meta-analysis of the literature dealing with sme e-business, parker and castleman (2007) pointed out that a high proportion of the literature on this topic deals with factors influencing the adoption decision itself. although our focus is on implementation, it is likely that factors that led to adoption will continue to influence implementation, but perhaps with decreasing strength over time. some of the rationale for our implementation success model will therefore draw upon the cbis adoption literature. our three distal factors, ceo characteristics, organisation characteristics, and decision processes, span both adoption and implementation phases. we begin our review of cbis success factors by looking at these three factors. thong and yap (1995) found that small businesses that adopt it are more likely to have ceos who possess a positive attitude towards adoption of it, who are innovative, and who are knowledgeable about it. burgess et al. (1997) found that 56% of the respondents in their study of small businesses in australia considered a progressive owner to be a success factor in the implementation of cbis. they also found that in small companies the owner is generally the manager and the sponsor of the it projects. aside from this empirical evidence, one would expect that in a small business environment, the owner or ceo will determine the characteristics and organisational culture of the business. we therefore included the variable in our modelling, focussing particularly on the ceo’s knowledge of cbis and the ceo’s innovativeness. the next set of factors relate to the characteristics of the organisation. businesses that have technical expertise should be in a better position to capitalise on it innovations. recent research recognises that for small businesses, this technical expertise is likely to come from external advisors, in which case availability and quality of external advice becomes a key factor in cbis success (goode, 2002; morgan et al., 2006). we labelled this construct organisation cbis technical capacity, and included it as part of the network of factors influencing cbis success. recognising that technical expertise could come from outside or inside the business, we phrased the questions in our survey so that respondents indicated the capacity of the business to deal with each of the technical issues identified in the items. we also included the source of the capacity in the question stem. a second organisational factor likely to impact on cbis success concerns the growth outlook of the business. a cbis is more likely to be rated positively by a business that is expanding and able to capitalise on the opportunities offered by computerisation. in a similar vein, successful small businesses must be innovative to enhance or maintain their competitive advantage over rivals (redoli et al., 2008). compared with countries such as sweden and germany which follow a “coordinated” business philosophy, australian small businesses operate in a highly “competitive” environment, supported by government policies directed at entrepreneurship (parker, 2002). in regional areas, in particular, competition for scarce resources is likely to be a major factor in australasian journal of information systems volume 15 number 2, 2009 76 business success (chong, 2007; macgregor & bunker, 1996). thus, many businesses fail simply because they cannot keep up with their rivals. however, competitiveness is not an inherently good or bad quality. it can be a major asset in some industries (e.g., service) and a liability in others (e.g., technical innovation). its inclusion in our cbis success model as a third characteristic of organisations was therefore somewhat exploratory. the last of the distal factors concerns the decision-making processes and/or decision criteria used by small businesses to make decisions relating to cbis. working in an australian context, macgregor and bunker (1996) identified four main reasons why small businesses acquire cbis. they are to increase productivity, to streamline work procedures, to improve customer service, and to achieve better record keeping. more recently, bunker, macgregor, carlsson, and magnusson (2002) added organisational competitiveness to this list. these reasons could be classified as a mixture of strategic and operational aims. there are other types of decision-making processes that should also be taken into consideration, such as the criteria used to make the choice between different it solutions (e.g., vendor support), and whether or not formal decision aids, such as cost-benefit analysis, were used. although the focus of the present study is not on the decision to adopt cbis, it is apparent from macgregor and bunker’s research that this initial decision can also embrace decisions about the characteristics of the system to be adopted, and that these characteristics have an influence on cbis success. we therefore included a range of decision making measures in the present study and positioned them in our model as potential mediators between ceo characteristics and the more proximal factors, such as system characteristics and user satisfaction, which are the subject of the next section of this review. proximal factors contributing to cbis success delone and maclean (1992) classified all measures of cbis success under the six major categories of system quality, information quality, information use, user satisfaction, individual impact, and organisation impact. the delone and mclean model has subsequently been updated and expanded (see delone & mclean, 2003, 2004). the expanded model retained system quality and information quality as key input variables but added service quality as a third major influence on the outcome constructs of intention to use, actual use, and user satisfaction. seddon and kiew (1996) employed a slightly different set of factors when modelling the network of constructs surrounding cbis usefulness and satisfaction. they introduced the constructs of system importance and task importance to their conceptual model on the basis that systems that perform more tasks that are important are perceived as more useful, irrespective of the quality of the actual system. in their model, the immediate predictors of user satisfaction were usefulness, information quality, system quality, and importance of the system. usefulness also acted as a mediator variable through which the other three variables exerted an indirect influence on user satisfaction. this model, minus the system importance variable, was nested within the conceptual model to be tested in the present study. defining the outcome variable different studies have used a variety of measures of cbis success and although there is continuing debate about the operationalisation of some of the constructs underpinning cbis success (burtonjones & straub, 2006; sedera & tan, 2007; zumpe & van der heijden, 2007), certain core elements are present in most studies; namely information quality, system quality, system use, and user satisfaction. the last of these is particularly important because it has been used extensively as a surrogate measure of cbis success (seddon & kiew, 1996; thong & yap, 1995). seddon and kiew australasian journal of information systems volume 15 number 2, 2009 77 (1996) defined user satisfaction as the net feeling of pleasure or displeasure that users experience based upon their individual expectations of the expected benefits of the cbis (p.22). the extent to which the system meets or fails to meets this expectation is the extent to which the user is more or less satisfied. following in this tradition, user satisfaction was chosen as the dependent variable for the present study. conceptual model of factors influencing cbis success the model tested in this study draws upon much of the research described above. the model has both measurement and structural components but, for the sake of clarity, only a broad conceptual overview of the structural model is shown in figure 1. starting from the top right-hand side, ceo characteristics (knowledge and innovativeness) is shown as having a direct effect on organisational characteristics and decision-making criteria, as one would expect in a small business environment where the ceo is closely involved with the business. the model also shows a direct link to user satisfaction, reflecting the fact that the ceo is likely to be one of the main users of the cbis. at the top left-hand side, organisation characteristics (technical capacity, competitiveness, and capacity for growth) has a direct influence on system characteristics (software, information quality, usage characteristics) because the needs of the organisation will largely determine the software and hardware choices. decision criteria (reasons for purchasing, criteria used to make decision, and decision making tools) is influenced by the characteristics of both the organisation and the ceo and, in turn, has a direct effect on system characteristics and user satisfaction. this last link reflects the belief that the processes used to make purchase decisions, sometimes years ago, will continue to affect users’ satisfaction. the model shown in figure 1 is much broader than proposed by delone and mclean (2003) and seddon and kiew (1996) in that it explores background variables not featured in their models. conversely, it presents a simplified view of constructs like user satisfaction, which are treated in more detail in other recent models of cbis success (e.g., seen, rouse, & beaumont, 2007). the conceptual model was tested using structural equation modelling (sem). version 7.0 of the amos package (arbuckle, 2006) was used for this purpose. method sampling procedure the participants came from an initial pool of 598 small businesses randomly selected from the business enterprise register (ber) for the mid north coast region of new south wales, australia. this pool was narrowed to 240 when phone contact indicated that 358 of these small businesses, either did not use a cbis to support their business functions or they were unwilling, or were unable to participate in the survey. two hundred and forty businesses met the criteria, and received the survey form. out of the 171 businesses that responded (a response rate of 71.3%), 97 were manufacturers, 64 were wholesalers, and 10 combined both operations. the average age of these businesses was 16 years and almost two-thirds (63%) had fewer than five employees. australasian journal of information systems volume 15 number 2, 2009 78 organization characteristics decision criteria ceo characteristics system characteristics user satisfaction figure 1: conceptual overview of factors influencing cbis success description of the implementation survey for cbis (is-cbis) to measure constructs relevant to the aims of this study, we developed the implementation survey for cbis (is-cbis). the scales and their internal consistency reliability estimates (cronbach’s alpha) derived from the present study are presented below. the first two scales represent ceo characteristics, the next three scales capture the characteristics of the organisation, followed by the four scales capturing the different categories of decision-making criteria, then the well-known cbis success scales and, finally, the outcome variable for this study, user satisfaction. 1. ceo cbis knowledge. six items were used to assess the ceo's level of cbis knowledge on a seven-point scale ranging from never to very often (e.g., i use a computer at work). alpha = .93. [source: items adapted from thong and yapp, 1995]. 2. ceo cbis innovativeness. three items were used to assess the level of the ceo’s innovativeness and resourcefulness on a seven-point scale ranging from strongly disagree to strongly agree (e.g., i often risk doing things differently). alpha = .87. [source: items adapted from thong and yapp, 1995]. 3. organisation cbis technical capacity. eight items were used to assess the technical capacity of the organisation to deal with cbis issues on a seven-point scale ranging from very poor to excellent (e.g., designing new information systems to carry out business tasks). alpha = .94. [source: items adapted from macgregor and bunker, 1996]. 4. organisation expansion and growth. five items assessed the opportunity for expansion and growth in the industry. the anchor points of the seven-point scale were strongly disagree and strongly agree (e.g., there is a lot of scope for the business to grow). alpha = .85. [source: items adapted from delone and mclean, 1992]. 5. organisation competitiveness. three items assessed the competitiveness of the industry in which the company operated. again, the anchor points of the seven-point scale were strongly disagree and strongly agree (e.g., the rivalry among companies in this industry is very intense). alpha = .85. [source: items adapted from thong and yapp, 1995]. cbis success australasian journal of information systems volume 15 number 2, 2009 79 6. operational reasons. six items were used to assess the importance of improving operational functions through implementing a cbis on a seven-point scale ranging from not important to very important (e.g., improve availability of information). alpha = .85. [source: items adapted from mcgregor and bunker, 1996]. 7. strategic reasons. a further six items assessed the extent to which the importance of improving strategic functions through implementing a cbis was a goal of the organisation (e.g., improve competitive advantage). the same scale seven-point scale was used as for operational reasons. alpha = .87. [source: items adapted from mcgregor and bunker, 1996]. 8. decision criteria. this 22-item scale assessed the various sources of information and evaluation criteria used to arrive at the purchase decision (e.g., confidence in the vendor). the anchor points for the seven-point scale were not important and very important. responses to all items were summed so the scale reflects the range of sources consulted and the extent to which they were consulted. alpha = .89. [source: items adapted from mcgregor and bunker, 1996]. 9. cost benefit techniques. eight items were used to assess adherence to cost benefit analysis techniques during cbis acquisition (e.g., return on investment). the anchor points for the seven-point ratings were no emphasis and a lot of emphasis. alpha = .91. [source: this study]. 10. information quality. six items assessed the quality of the information provided by the cbis (e.g., the information i get from the system is clear). anchor points for the sevenpoint rating scale ranged from strongly disagree to strongly agree. alpha = .91. [source: items adapted from seddon and kiew, 1996]. 11. system usefulness. five items assessed system usability (e.g., using the system saves time). the items employed the same scale format as information quality. alpha = .92. [source: items adapted from seddon and kiew, 1996]. 12. system usage characteristics. seven items assessed the extent to which the system met expected usage characteristics (e.g., the system is easy to learn). the items employed the same format as the other two cbis success scales. alpha = .85. [source: items adapted from seddon and kiew, 1996]. 13. user satisfaction. three items with anchor points ranging from never (1) to always (7) were used to gauge the overall feeling of satisfaction with the system (e.g., overall, how often are you satisfied with the system?). alpha = .80. [source: items adapted from seddon and kiew, 1996]. procedure we mailed the surveys to the participating firms, and the completed surveys were returned in a reply-paid envelope. we entered the data into excel and then imported to spss. the survey took approximately 30 minutes to complete. results descriptive statistics many of the items used in the is-cbis were adapted from scales used to assess cbis adoption, rather than implementation. it was therefore necessary to assess the factorial structure of the australasian journal of information systems volume 15 number 2, 2009 80 instrument before forming scales and assessing their psychometric properties. in a two-step process, principal axis factor (paf) routines from the spss package were used to weed out items with low communality. confirmatory factor analytic routines from the amos package were then used to check the fit of the measurement model for individual scales. the factorial validation process resulted in the deletion of a number of items and 13 factors being identified. scales were formed from the various sets of items defining each of the factors. because the number of items differed from scale to scale, the means and standard deviations are based on the total score for each scale divided by the number of items in that scale. all items used a seven-point response format, so a mean of 4.0 represents the midpoint of the scale. the next stage of the validation process involved checking the distributional properties of the scales. scales that have floor or ceiling effects or too little variance are obviously not very useful for either theoretical or practical purposes. the psychometric properties of the scales are shown in table 1. scale no. of items m sd α ceo characteristics ceo innovativeness 3 4.63 1.29 .87 ceo cbis knowledge 3 4.06 1.36 .93 organisation characteristics competitiveness 3 4.92 1.59 .85 expansion & growth 5 5.08 1.14 .85 cbis technical capacity 8 4.04 1.36 .94 decision criteria operational reasons 6 5.63 1.00 .85 strategic reasons 6 4.33 1.55 .87 decision criteria 22 3.84 1.04 .89 cost benefit techniques 8 2.03 1.41 .91 cbis success information quality 5 5.24 1.07 .91 system usefulness 5 5.01 1.37 .92 system usage characteristics 3 4.86 1.21 .85 user satisfaction user satisfaction 3 5.47 1.00 .80 table 1: descriptive statistics for all scales (n = 169) the summary statistics establish that all scales had satisfactory internal consistency reliability estimates. although a degree of positive skewness was evident for the cost benefit techniques scale and negative skewness for the operational reasons and user satisfaction scales, in general, a reasonable spread of scores was obtained. this quality is important if the scores are to be used to explore relations among these constructs. table 2 shows these relations in the form of pearson product moment correlations among all 13 scales. the first point to note about the correlations is that they were mostly positive and that there were many significant relationships. the second point to note is that all but one of the variables correlated with user satisfaction, the main outcome variable in this study. the presence of so many significant australasian journal of information systems volume 15 number 2, 2009 81 correlations suggests that the study has been successful in capturing constructs that are conceptually distinct from user satisfaction but potentially useful in predicting this important outcome variable. the next section of the paper describes the testing of the expanded form of the structural model shown in figure 1. scale 1 2 3 4 5 6 7 8 9 10 11 12 ceo characteristics 1. ceo innovativeness 2. ceo cbis knowledge .41 org. characteristics 3. competitiveness .05 -.05 4. expansion & growth .40 .27 .05 5. cbis tech. capacity .39 .47 .00 .55 decision criteria 6. operational reasons .15 .16 .12 .41 .31 7. strategic reasons .13 .09 .23 .43 .32 .46 8. decision criteria .19 .23 .06 .52 .48 .44 .50 9. cost ben. techniques .09 .16 .06 .33 .37 .25 .36 .51 cbis success 10. information quality .07 .18 -.05 .39 .35 .29 .14 .19 .09 11. system usefulness .20 .29 .03 .40 .41 .41 .25 .29 .24 .57 12. system usage chars .14 .36 .02 .35 .43 .17 .06 .31 .18 .58 .54 user satisfaction 13. user satisfaction .25 .34 .04 .56 .57 .42 .29 .44 .22 .74 .68 .61 table 2: correlations among scales (n = 169) nb: correlations in boldface are significant at .01 level (2-tailed). testing the cbis implementation success model for these sem analyses, we followed kline’s (2005) recommendations in reporting a number of indices reflecting different aspects of model fit. these indices were the sample discrepancy function (cmin) and the ratio formed with its degrees of freedom (cmin/df), the comparative fit index australasian journal of information systems volume 15 number 2, 2009 82 .36 information quality .21 system usage characteristics .44 system usefulness .72 user satisfaction .39 organisation technical capacity .16 expansion & growth .17 operational reasons .36 decision criteria ceo innovativeness ceo cbis knowledge e4 e2 e3 e7 e5 e6 e12 e13 .27 .55 .41 .49 .31 .31 .40 .27 .20 .29 .24 .46 .17 .41 .20 .33 .36 .22 .25 figure 2: the c bis im plem entation success m odel (c bis-ism ) australasian journal of information systems volume 15 number 2, 2009 83 (cfi) as an index that describes the overall proportion of explained variance, the adjusted goodness of fit index (agfi) as an index that adjusts the proportion of variance based on model complexity, and the root mean square standard error of approximation (rmsea) as an index based on standardised residuals. to represent good fit, the cmin should be non-significant (p > .05), the cmin/df less than 2.00, the cfi and the agfi greater than .90, and the rmsea not greater than .05. the sample used for the analyses in the present case was 169 after two cases were eliminated from the original 171 because of missing data. aggregated scales were used in place of items because of limitations in sample size and the fact that we used a two-step process to establish the psychometric properties of the scales before testing the structural model based on those scales. the final model contained 29 free parameters, meaning that the sample size was consistent with guidelines. figure 2 shows the full structural model with standardised parameter estimates and r2 values (in boldface). during the model-testing process, some variables and pathways were deleted. these changes were based on theoretical and statistical grounds (modification indices and correlations). we will deal with these before going on to discuss the final model. the variable organisational competitiveness was deleted because it related to just one other variable in the model (strategic reasons) and that relationship, although significant, was not particularly strong (see table 2). cost benefit techniques and strategic reasons also failed to contribute to the prediction of any other variables in the model, and were therefore, deleted. two other non-significant pathways were identified during model testing (cr < +/-1.96). the two pathways were from organisation technical capacity to information quality and from decision criteria to system usage characteristics. both of these pathways were removed. the final model shown in figure 2 has the distal variables ceo innovativeness, ceo cbis knowledge, organisation technical capacity, and expansion and growth positioned in the centre of the model. from there, they influence the cbis success variables at the bottom of the model and the decision-making processes at the top of the model. this model fitted the data (chi-square = 44.03, p > .01; cmin/df = 1.69; agfi = 0.90; cfi = 0.97; rmsea = 0.06). the rmsea value, although above .05, still falls within the guidelines for reasonable fit (kline, 2005). furthermore, all pathways shown in the model are significant. the most salient feature of the model is the high proportion of variance explained in user satisfaction (r2 = 0.72), the surrogate measure of successful cbis implementation. large effect sizes were obtained for all other endogenous variables in the model. discussion the primary aim of this study was to develop and test a model of the factors influencing cbis success in a regional small business setting. to achieve this aim, it was first necessary to assemble a questionnaire to collect data on the key variables selected for inclusion in the model. the construction and validation of is-cbis therefore became a subsidiary aim, which we discuss first, followed by descriptive statistics of data gathered. although survey instruments are in wide use in the cbis literature, very few of them have been subjected to adequate statistical validation procedures. the is-cbis used in the present study covers the period from the decision to adopt cbis right through to the period where users are able to rate their satisfaction with the system. to achieve this coverage, it contains 13 scales grouped into five major categories. where appropriate, the is-cbis drew upon the literature to australasian journal of information systems volume 15 number 2, 2009 84 source items and scales, and modified to suit the small business context and the different phases of cbis implementation. table 1 provides the means, standard deviations, and alpha reliabilities for the 13 scales. the scales have good psychometric properties including demonstrated factorial validity, satisfactory means and variances (i.e., absence of floor and ceiling effects and spread of scores), and satisfactory internal consistency reliability estimates. four of the scales showed evidence of skewness but we believe that in each of these cases, the skewness reflects properties of this regional small business sample rather than weaknesses in the instrument. positive skewness was most evident for the cost benefit techniques scale. the reality is that these businesses do not place much emphasis on management applications and they do not employ cost benefit techniques as reported by other researchers (e.g., ekanem, 2005). negative skewness was evident in the operational reasons and user satisfaction scales. for operational reasons, it is well known that small businesses are more likely to make decisions for operational rather than strategic reasons (macgregor & bunker, 1996; patullock, 2005), so the skewness is a feature of the sample, rather than something that has been induced by the items themselves. the development and validation of the is-cbis scales allowed us to test the conceptual model shown in figure 2. as expected, the major direct contributors to the variance in user satisfaction were the proximal variables information quality and system usefulness with other significant direct contributions from organisation technical capacity and decision criteria. what is also clear from the model is that a number of variables had an indirect effect on user satisfaction. an indirect effect can occur where a variable is connected to an output variable through a mediating variable. system usage characteristics, for example, does not have a direct pathway to user satisfaction but has potential indirect influences through system usefulness and information quality. amos employs a bootstrapping technique to estimate standard error terms for these indirect effects, thus permitting statistical tests of the indirect effects. when these indirect pathways were tested, all of the variables in the model had some degree of influence on user satisfaction, with the largest indirect contribution coming from system usage characteristics. it is interesting to note the pathways by which these indirect effects operate. there are too many to describe here, but it is informative to trace one or two for didactic purposes. starting with the ceo, by following one pathway we can see that the innovativeness of the ceo has a direct effect on the growth of the business, which in turn leads to a concern about operational issues, a concern which is eventually reflected in the quality of the information and overall usefulness of the installed cbis system, and thence to feelings of user satisfaction. this is just one of the indirect pathways connecting ceo innovativeness to user satisfaction. the point is that these pathways have been validated statistically and they make sense from a conceptual point of view. to illustrate this last point more clearly. ceo innovativeness links with expansion and growth but does not have a direct link with organisation technical capacity. on the other hand, ceo cbis knowledge does not have a direct link with expansion and growth but links with organisation technical capacity. the linkages one would expect are innovativeness affects company growth but not technical capacity (other than indirectly), and cbis knowledge affects technical capacity and system characteristics but not company growth. these effects occur because the ceo is the major influence in a small business and much more involved in operational activities than counterparts in larger businesses. the ceo’s level of understanding of cbis would be expected to have a major influence on the capacity of the business to deal with cbis planning and management issues. similarly, it makes sense that the ceo's perceived level of innovativeness and resourcefulness would be a major factor in the identification and exploitation of opportunities for growth and expansion of the business. the strong relationship between ceo innovativeness and ceo cbis australasian journal of information systems volume 15 number 2, 2009 85 knowledge (r = 0.41, p < .01) also makes sense as more innovative and resourceful ceos would be expected to have a better grasp of cbis. looking at organisational characteristics, we can see that expansion and growth had a direct effect on operational reasons (b = 0.41, p < .01), decision criteria (b = 0.29, p < .01), and organisation technical capacity (b = 0.46, p < .01). these pathways are no doubt due to the need for an expanding company to have more computer support for its operations. thus, small businesses that identify growth and expansion opportunities place more emphasis on planning for cbis acquisition and implementation, leading indirectly to more user satisfaction. the direct links involving organisation technical capacity are similarly plausible. this variable had a direct effect on user satisfaction (b = 0.22, p < .01), decision criteria (b = 0.25, p < .01), and system usage characteristics (b = 0.33, p < .01). a small business that has technical capacity is likely to consult more widely (decision criteria), to implement systems that have better usage characteristics, and ultimately to appreciate the cbis more because the cbis that was chosen met the needs of the business. decision making processes completed the set of distal variables measured in this study. we have already noted the predictors of these variables (organisational characteristics) and observed that these pathways make good conceptual sense. operational reasons influenced both information quality (b = 0.20, p < .01) and system usefulness (b = 0.27, p < .01) and thereby had a significant indirect effect (b = 0.22, p < .01) on user satisfaction. we therefore know that although small businesses may overemphasise operational reasons at the expense of strategic reasons (harindranath et al., 2007; patullock, 2005), there is a direct payoff in terms of the how useful the system is perceived to be and how satisfied the users feel. it is interesting to note that the other variable relating to decision processes, and decision criteria does not influence any variable in the model other than user satisfaction. we know from the table of correlations (table 1) that decision criteria is related to system usefulness, but that pathway is not supported in a model that reflects only unique variances. the sole pathway to user satisfaction indicates that a small business that has undertaken more market research and investigation of cbis options is likely to be more satisfied with its cbis. the final set of variables in the model belongs to the proximal set of indicators of cbis success. this set of variables has been well-researched (e.g., delone & mclean, 1992, 2003, 2004; seddon & kiew, 1996) and we found no more than what has already been found. indeed, there is a remarkable similarity between the parameter estimates reported by seddon and kiew and those obtained in the present study for this part of the model. system usage characteristics, information quality, and system usefulness are quasi-indicators of cbis success and, not surprisingly, they account for a large proportion of the variance in user satisfaction. the various outcomes reported in this paper are of importance to researchers examining cbis success in small business. relationships among variables measuring aspects of ceo characteristics, organisation characteristics, decision-making processes and criteria, and cbis success in small business have been identified and shown to be relevant to user satisfaction. the statistical and theoretical relationships among the variables build upon and support the work of delone and mclean (1992, 2003, 2004), seddon and kiew (1996), thong and yap (1995), macgregor and bunker (1996). the model itself is an aggregation of several key outcome variables from the work of each of these researchers plus the constructs dealing with cbis decision-making processes and decision criteria. the importance of scales measuring organisational characteristics and ceo characteristics as determinants of cbis success in the model provides confirmatory support for the findings of thong and yap (1995) who found that both these scales were significant in predicting the adoption of it in australasian journal of information systems volume 15 number 2, 2009 86 small business. it also supports recent research on the investing role of ceos in small businesses (ekanem, 2005). we can say that they are also important in predicting successful implementation of cbis in small business. overall, the model is a sound statistical and theoretical representation of several factors that lead to the perceived success of cbis in small business, accounting for 72% of user satisfaction. it shows that decision criteria, information quality, system usefulness, and organisation technical capacity are direct predictors of user satisfaction. it is quite clear how information quality and system usefulness contribute to user satisfaction and the pathway between these variables is supported by the research of seddon and kiew (1996). thong and yap (1995) hypothesised the relationship between organisation characteristics and the adoption of it. the current study has identified a relationship between organisation technical capacity and the primary outcome variable user satisfaction. the relationship between decision criteria and user satisfaction also makes sense, as a business that invests more time and effort on cbis market research would be more likely to be satisfied with their cbis. in their recent research expanding their cbis success model, delone and mclean (2003, 2004) added variables to improve the measurement of outcomes. other researchers have followed their lead (sedera & tan, 2007; zumpe & van der heijden, 2007). we have moved in the other direction, accepting user satisfaction as the outcome variable but adding to the set of variables that predict satisfaction. our findings establish the relevance of additional constructs – namely ceo characteristics, organisational characteristics, and decision-making criteria to predicting cbis success. limitations of the study this study has a number of limitations. first, like many that have preceded it, used a proxy variable – user satisfaction – as the measure of cbis success. satisfaction does not ensure success. second, although there is an implied time sequence in the categories included in the survey – ceo characteristics, organisational characteristics, decision processes, system characteristics, and then user satisfaction –the respondents actually completed the survey at a single point in time. they must therefore, have relied upon their memories to answer questions about the pre-implementation stages. we know from research in psychology that such memories are likely to be flawed and influenced by subsequent experiences, thus creating artificial shared variance between distal and outcome variables. a third limitation is that we have treated the stages of cbis acquisition and implementation as discrete when in reality firms may be at various stages for different parts of the cbis system. a particular firm may have acquired a cbis but not be engaged in e-commerce, which may have been one of the original reasons for adopting cbis. these things take time, especially in the small business sector. a fourth limitation applies to our findings and concerns the fact that there may be alternative models that fit the data equally well or better. our response to this potential criticism is that part of our model, involving what we call the proximal variables, is well replicated (delone & mclean, 2003). the newer parts either are, based on research into it adoption or, in the case of decision processes, sound theoretical considerations. thus, while alternative models could also fit the data, our model has strong empirical and theoretical foundations, and fits well to the data. australasian journal of information systems volume 15 number 2, 2009 87 directions for future research when looking at avenues for further research, the instrument used in the present study contains sections of instruments used in previous studies, most notably raymond’s (1985) user information satisfaction (uis) survey. the uis has extensively measured cbis success in small organisations. we included the core components of the uis in our study, and added constructs relating to the ceo, the organisation, and the decision-making processes that preceded the purchase of the cbis. while we have validated the expanded instrument, it could benefit from further developmental work. as we noted above, there is not a lot of room for improvements in the prediction of user satisfaction because current models, including the one reported in this study, are already capturing most of the shared variance (72%) in this outcome variable. however, there is certainly scope for improving the prediction of other variables in the model and for the inclusion of new constructs and improved measurement techniques. further investigation is also required to examine those constructs that were included in our survey but omitted from the measurement model. this included the competitiveness of the organisation, strategic focus is another, and cost benefit techniques, which were not included in the present study. there is evidence from the literature that these constructs play a role in determining the adoption of cbis but the conditions under which they help to predict successful cbis implementation remain unclear. conclusion small businesses are an important part of the australian and world economy, and every attempt must be made to improve their prospects of success. the adoption of cbis by small business is recognised as one of the difficult steps in the growth phase. because of the disruption and expense often involved in computerisation, we need to learn more about the key factors that help to ensure success. previous studies have drawn attention to the characteristics of the installed systems and the influence they have on user satisfaction. we have drawn attention to an important set of background factors that will help to ensure that the characteristics of the system are suited to the business. we encourage other researchers to take up the challenge of discovering more about these backgroundenabling conditions so that more small businesses are able to make a smooth transition to computer based information systems. references bunker, d.j., macgregor, r.c., carlsson, s., and magnusson, m. (2002), ‘electronic commerce and market focus: some findings from a study of swedish small to medium enterprises’, australian journal of information systems, vol. 10, no. 1, pp. 110-119. burgess, s., belcher, b., paull, s. and singh, m., (1997), ‘information technology and australian small business enterprises’, proceedings of seaanz '97, coffs harbour, nsw, norsearch ltd. burton-jones, a. and straub, d.w. (2006), ‘reconceptualizing system usage’, information systems research, vol. 17, no. 3, pp. 228-246. australasian journal of information systems volume 15 number 2, 2009 88 chau, p.y.c. (1994), ‘selection of packaged software in small business’, european journal of information systems, vol. 3, no. 4, pp. 292–302. chong, s. (2007), ‘business process management for smes: an exploratory study of implementation factors for the australian wine industry’, journal of information systems and small business, vol. 1, nos. 1-2, pp. 41-58. delone, w., and mclean, e. (1992), ‘information systems success: the quest for the dependent variable’, information systems research, vol. 3, no. 3, pp. 60–95. delone, w. and mclean, e. (2003), ‘the delone and mclean model of information systems success: a ten-year update’, journal of management information systems, vol. 19, no. 4, pp. 9-30. delone, w. and mclean, e. (2004), ‘measuring e-commerce success: applying the delone and mclean information systems success model’, international journal of electronic commerce, vol. 9, no. 1, pp. 31-47. ekanem, i. (2005), ‘bootstrapping’: the investment decision-making process in small firms. the british accounting review, vol. 37, pp. 299-318. goode, s. (2002), ‘exploratory analysis of the effect of consultants on the use of the world wide web sites in smes’, australasian journal of information systems, vol. 10, no. 1, pp. 20-28. harindranath, g., dyerson, r. and barnes, d. (2007), ‘ict adoption and use in uk smes: a failure of initiatives?’ electronic journal of information systems evaluation, vol. 11, no. 2, pp. 91-96. kartiwi, m. (2006), ‘case studies of e-commerce adoption in indonesian smes: the evaluation of strategic use.’ australasian journal of information systems, vol. 14, no. 1, pp. 69-80. kline, r. b. (2005), ‘principles and practice of structural equation modelling (2nd ed.)’, ny: guilford press. lawson, r., alcock, c., cooper, j. and burgess, l. (2003), ‘factors affecting adoption of electronic commerce technologies by smes: an australian study’, journal of small business and enterprise development, vol. 10, no. 3, pp. 265-276. locke, s. (2001), ‘adoption of information communication technology by new zealand smes’ proceedings of 14th conference of seaanz, wellington, 13-15 sept. macgregor, r. c. and bunker, d.j. (1996), the effect of priorities introduced during computer acquisition on success with information technology in the small business environment. in proceedings of the information resource management association international conference, 271 – 277, washington d.c. morgan, a., colebourne, d. and thomas, b. (2006), ‘the development of ict advisors for sme businesses: an innovative approach’, technovation, vol. 26, pp. 980-987. parker, r. (2002), ‘coordination and competition in small business policy: a comparative analysis of australia and denmark’, journal of economic issues, vol. 36, no. 4, pp. 935-952. parker, c.m. and castelman, t. (2007), ‘new directions for research on sme-ebusiness: insights from an analysis of journal articles from 2003 to 2006’, journal of information systems and small business, vol. 1, nos. 1-2, pp. 21-40. australasian journal of information systems volume 15 number 2, 2009 89 patullock, e. (2005), ‘the management of computer systems in small business: a preliminary analysis of the issues of importance to managers’ australian journal of information systems, vol. 13, no. 1, pp. 101-119. raymond, l. (1985). ‘organizational characteristics and mis success in the context of small business’, mis quarterly, vol. 9, no. 1, pp. 37–52. redoli, j., mompo, r., garcia-diez, j. and lopez-coronado, m. (2008), ‘a model for the assessment and development of internet-based information and communication services in small and medium enterprises’, technovation, vol. 28, pp. 424-435. schubert, p., fisher, j. and leimenstoll, u. (2007), ‘ict and innovation in small companies’, in h. osterle, j. schelp, and r. winter (eds.), proceedings of the 15th european conference on information systems, pp. 1226-1239. june 7-9, st gallen, switzerland. seddon, p.b. and kiew, m.y. (1996), ‘a partial test and development of delone and maclean's model of is success’, australian journal of information systems, vol. 4, no. 1, pp. 90-109. sedera, d. and tan, t.c.f. (2007), ‘reconceptualising usage for contemporary information success’, in h. osterle, j. schelp, and r. winter (eds.), proceedings of the 15th european conference on information systems, pp. 1344-1343. june 7-9, st gallen, switzerland. seen, m., rouse, a. and beaumont, n. (2007), ‘explaining and predicting information systems acceptance and success: an integrative model’. in h. osterle, j. schelp, and r. winter (eds.), proceedings of the 15th european conference on information systems, pp. 13561367. june 7-9, st gallen, switzerland. thong, j.y. and yap, c.s. (1995), ‘ceo characteristics, organizational characteristics, and information technology adoption in small business’, omega – international journal of management science, vol. 23, no. 4, pp. 429–442. zumpe, s. and van der heijden, h. (2007), ‘on the use of variable user goals to measure perceived usefulness’, in h. osterle, j. schelp, and r. winter (eds.), proceedings of the 15th european conference on information systems, pp. 1334-1355. june 7-9, st gallen, switzerland. visual navigation of complex information spaces sarah m. north, max m. north, and nazir a. warsi human-computer interaction group & virtual display systems laboratory computer and information science department clark atlanta university, atlanta ga, 30314 e-mail: sarah@acm.org, max@acm.org, warsi@cau.auc.edu abstract the authors lay the foundation for the introduction of visual navigation aid to assist computer users in direct manipulation of the complex information spaces. by exploring present research on scientific data visualisation and creating a case for improved information visualisation tools, they introduce the design of an improved information visualisation interface utilizing dynamic slider, called visual-x, incorporating icons with bindable attributes (glyphs). exploring the improvement that these data visualisations, make to a computing environment, the authors conduct an experiment to compare the performance of subjects who use traditional interfaces and visual-x. methodology is presented and conclusions reveal that the use of visual-x appears to be a promising approach in providing users with a navigation tool that does not overload their cognitive processes. introduction data visualisation has received a great deal of attention recently, and the need for better information visualisation tools is widely recognized (beard & walker 1991, chimera 1992, mclnlay et al 1991, north & north 1993, north et al 1994, north & north 1993b). one of the main reasons for better information visualisation tools is the characteristic of the human cognitive process (gardiner & hitch 1987, johnson 1992). specifically, humans have difficulty recognizing specified objects among many homogenous objects even with the aid of semantic attributes (e.g., a directory listing just by words) (byrne 1993, north et al 1994, north & north 1993b). consequently, icons are commonly used to facilitate the information visualisation and assist users referencing, locating, and selecting objects. although the graphical user interfaces (guis) extensively use icons to represent objects such as files and directories, there exists little systematic empirical research to support the foundation of the interaction process between humans and icons (byrne 1993, north et al 1994). furthermore, there exists almost no empirical data or guidelines to assist interface designers in the graphical presentation of object's attributes (e.g., in file directory, size of file, generation time) (north et al 1994, north & north 1993b). theoretically, factors such as size, color, form, etc. seem to be useful in locating objects; they are not commonly used in most computer displays or generally are ignored (byrne 1993, chimera 1992). in some instances, unfortunately, user interface designers have exaggerated the hypothetical guidelines of icon usage and employed complex icons that seem to overwhelm users. byrne (1993) reports, "in general, simple icons (those discriminable based on a few features) seem to help users, while complex icons are no better than simple rectangles." surprisingly, several other researchers have shown that most of the visual factors have little or no effect on users's ability to locate objects. jones, et al have expressed concern regarding the difficulty many computer users experience in locating objects (documents) through "spatial filing". moreover, byrne (1993) reports that pictorial icons do not effect the users' efficiency in locating objects (documents). consequently, it appears that there exists no unified theories about underlying interaction between human and icons. a systematic investigation of all the proceeding factors and theories is needed in order to develop guidelines for assisting users in the creation of more effective and efficient mental model of objects and object searching. need for information visualisation tools modern resource discovery systems, and even more so those of the future, will need visual displays (including icons and text) that will effectively and efficiently convey directory structure, organization, and contents. these systems (wais (kahle & medlar 1991), gopher (mccahill 1992), www (berners-lee 1992), etc., and their interfaces) often construct "virtual directories" (berners-lee 1992) in the user's work space that may contain subdirectories and files from distant servers. thus the user may have little knowledge of the directories's contents other than they are the result of particular keyword or textual searches. if we add the fact that future searches will be much more extensive, involve many more servers, and glean many more files, the need for information visualisation tools that convey this information becomes paramount. the expansion in the types of data is an important instance where information visualisation tools will be needed. whereas now resource discovery searches yield mostly text files, in the future they will also yield image files in 46 ajis various formats (e.g., postscript, tiff, sgi rgb, etc.), executables, animation sequences, program source codes, make or build files, more complicated file structures such as hypertexts containing multimedia components, and results from observations, experiments, or computer simulations. the observation results from the earth observing system provide a useful case in point. by the end of the decade, this system of satellites will be pouring observations about the earth's atmosphere, climate, ocean currents, pollutants, and their interrelationship into archives at the rate of one trillion bytes per day. nasa will make this data, and correlated data such as the results of comparative computer simulations, available to researchers world-wide. the advent of the federal initiative in high performance computing and networking and several government-backed "grand challenge" investigations in the sciences and engineering mean that there will be the infrastructure (e.g., distributed databases and servers), the delivery system (e.g., ubiquitous high bandwidth networks) and the uses (e.g., by researchers who must retrieve and compare massive amounts of data from many sources) requiring information visualisation tools in many different fields. conceptual framework and design this research project supports the following theories reported by gardiner and hitch (1987) concerning the human cognitive processes, specifically the short term memory (stm) and long term memory (ltm) characteristics. stm is a memory buffer with an extremely low capacity (7±2 units) and only moderate trace duration (a matter of seconds). gardiner and hitch (1987) postulate that the characteristics of human stm and ltm as follows: (i) letters or words that rhyme are particularly difficult to differentiate in stm (directory file names that rhyme visually such as xy.012, xy.015, etc.). (ii) small, manageable amounts of information will be lost from stm in a matter of seconds if that interval is filled with distracting information or activity, (iii) stm recall of item names is usually better when names are presented as pictures (visual), rather than words (semantic), (iv) the probability of retrieval of information from ltm increases as a function of the match between the cues and context present at the time of storage and the cues and context available at the time of recall, (v) the greater the compatibility between the operations carried out to store information and the operations carried out to retrieve it, the greater the probability of successful recall, (vi) the more distinct and unique the information stored in ltm, the greater the probability that it will be successfully recalled. cha and wiederhold [4] independently agree with gardiner and hitch,"... the knowledge in human long-term memory is imprecise, incomplete, and often incorrect." by adhering to the proceeding postulates, visual-x enables users to visualize and browse through different attributes of an object using low cognitive load. the visual-x is a collaborative tool that will extract some of the attributes of objects automatically while allowing the user to fill in additional attributes. some of the important features of visual-x are: (i) it enables the user to see in one view different attributes of the object (north et al 1994, north & north 1993b), (ii) it provides low cognitive load for the user, (iii) it assists the user in extracting meaning about objects, and (iv) it provides a global view of the distribution of attributes of the object (johnson 1992). the design of visual-x is based on the concept of direct manipulation (schneiderman 1993). the direct manipulation is achieved by utilizing a novel widget called dynamic slider (north & north 1993, north et al 1994) and other widgets. the direct manipulation and dynamic slider are briefly described below: direct manipulation basic philosophy the distinguishing feature of direct manipulation interfaces is that their structures corresponds to the structure of the domain. direct manipulation allows the user to observe the results while directly manipulating the domain structure. shneiderman reports a direct correlation between objects presentation and innate human capabilities. shneiderman (1986) writes: "dealing with representations of objects may be more "natural" and closer to innate human capabilities: action and visual skills emerged well before language in human evolution. psychologists have long known that spatial relationships and actions are grasped more quickly with visual rather than linguistic representations. furthermore, intuition and discovery are often promoted by suitable visual representations of formal mathematical systems." the direct manipulation architecture provides the following: (i) continuous graphical representation of information, and information exploration outcome; (ii) visible range of the object by utilizing dynamic sliders and other widgets (shneiderman 1993); and (iii) immediate feedback as the user physically manipulates the sliders, selection buttons, etc. september 1995 47 dynamic slider as a novel multiattributes widget traditionally, sliders have been used as a metaphor to assist the user in entering either a single value or a single range value anchored to minimum or maximum points of a field. a dynamic slider is a slider with two drag boxes which represent dynamic range(s) rather than a single or anchored value to provide the mechanism for assisting user in direct manipulation of the complex information spaces. the detailed description of the dynamic slider can be find in the article by north and north (1993) and north, et al (1994). design approach we have extended this work in a unique way by adding techniques that have proved quite successful in scientific and engineering data visualisation. the idea is to generalize the usual iconographic display in, for example, a directory window. the generalized icons would have attributes that are bound by the user to data variables causing a mapping of that variable onto changes in the iconic attribute. thus in this view, the icons would have attributes that in this case are bound to file size. the essential point here is that the object attributes need not be bound only to file size (or any single property); at the user's discretion it could be bound to another property such as the number of times a keyword appears in the file's text. icons with bindable attributes are commonly called "glyphs". glyphs are 2or 3-d graphical objects whose elements (e.g., position, size, shape, color, orientation, intensity, etc.) are bound to data. they have proven quite useful in depicting spatially complex, multivariate data such as is found in current calculations and experiments. as we have pointed out, the size, complexity, and number of variables in this type of data will only increase in the future. glyphs have been effective in depicting macromolecular structure and interactions, variables such as wind speed and direction in atmospheric dynamics simulations and observations (treinish 1993), complex 3-d structure and several variables simultaneously from molecular dynamics simulations (ribarsky et al), plastic flow in injection molding processes (elson & cox 1988), and many other types of data. based on ideas originally presented by foley & mcmath (1986), ribarsky and colleagues have recently extended the power and scope of the glyph concept by developing tools to allow the user to make and change bindings of glyph attributes to sets of related data variables (ribarsky et al 1993, ribarsky et al). this approach implies that we have a set of data objects and separately have a set of graphical objects, and we can delay decisions about how they are interactively joined together until those decisions need to be made-as the data is being viewed and explored. this is the most flexible way to create data visualisations. ribarsky and colleagues have made an easy-to-use, programmerless environment, called glyphmaker, (ribarsky et al 1993) that allows the creation of glyphs and their subsequent binding to data variables by direct manipulation. visual-x capabilities and characteristics visual-x enables users to visualize and browse different attributes of objects using low cognitive load. visualx is a collaborative tool that extracts some of the attributes of objects automatically while allowing the user to fill in additional attributes. some of the important features of visual-x: (i) enable the user to see in one view different attributes of the objects, (ii) provide low cognitive load for the user, (iii) assist the user in extracting meaning about objects, and (iv) provide a global view of the distribution of attributes of the objects (johnson 1992). specifically, visual-x interface possesses the following capabilities and characteristics: (i) objects and graphical widgets of interest are continuously visible to the user (schneiderman 1993); (ii) the outcome of the information exploration is produced by physical actions such as manipulation of the dynamic sliders and other widgets; (iii) the provision for incremental, rapid, and reversible actions that will be immediately displayed for users; and (iv) a minimal learning curve to aid naive as well as expert users in the exploration of information. in essence, visual-x interface enables the users to visualize and explore information similar to human cognitive information processing using minimal cognitive load. in addition, visual-x interface enables the user to see in one view objects, actions, and results, and visual-x interface assists the user in extracting the meaning and relationship of objects (sarkar & brown 1992). visual-x interface at work (an application) in many instances, users list the directory contents of a computer system and cannot recognize or extract meaningful information by looking at the file names (using unix is command). consider a unix directory listing using visual-x tool (figure 1). 48 ajis visual-x does not only depicts file name, creation time, and size. it also provides other attributes such as volume of modification, last date of modification, frequency of use, number of key words appearing in the file, etc. each of these attributes are presented with the dynamic slider. the user can select any range of the attributes by just manipulating the dynamic slider. for example, file size between 10k and 100k bytes may be chosen. each attribute also corresponds with a series of visualisation primitives. visualisation primitives assign the attributes of the object (icon) in the view window. for simplicity, we have used only four visualisation primitives for our prototype. however, these visualisation primitives could be extended or rearranged as the user desired. the visualisation primitive of visual-x prototype is size, color, intensity, and depth. the user can select one of these primitives by clicking and binding it to object attributes. for example, the intensity primitive is binded with creation time. attributes may be classified as primary or secondary. two primary attributes are allowed in our prototype to represent the x and y axis of the view windows. for example, "size of file" and "frequency of use" are selected as primary attributes. the user may bind different attributes to different visualisation primitives with the limitation of the two primary bounding. this selection allows users to customize the icons in the view window in turn, users can explore the objects attributes and visually extract meaning about the objects and their relationship to each other. the experiment this experiment compared visual-x interface with traditional interface (i.e., unix interface). the major hypotheses was that there would be a significant difference between the performance of subjects who use visual-x interface and the subjects who use the traditional unix interface. the independent variables in this experiment were the interfaces that subjects utilized for referencing, locating, and selecting objects: (i) visualx interface; and (ii) traditional unix interface. the dependent variables were: (i) time to reference, locate, and select an object or group of objects; and (ii) usefulness, effectiveness, learnability, and attitudes. the prior data collected from the users revealed that users' skills vary from novice to expert with expert users dominating the population. the research subjects were forty individuals (n=40) who were randomly selected for this project. twenty subjects were placed in the experimental group and twenty subjects were placed in the control group. a typical subject was 19-24 year-old (64%), and female (82%). initially, a prototype visual-x interface was designed, developed, tested, and implemented on a personal computer. each group was given an overview of experimental procedures and a short instructional period, primary to acquaint users with the interfaces. the subjects from the experimental and control groups were presented with twenty different information exploration tasks to perform on visual-x interface. each subject was timed and recorded for the successful performance of each task. the following ia a partial listing of information exploration tasks: • locate files created after october 1993; and the frequency of file use exceeded ten; and volume of modification is greater than 1000 bytes. • locate text files ( . txt) with the file size is between 10k and 20k bytes. • locate files with five to eight keywords; and the last modification to the file was made after december 1992; and the file size is between 20k and 30k bytes. preliminary results the data were subjected to appropriate statistical procedures. these procedures included a measure of central tendency and the t-test. control group n mean s.d. experimental group (visual-x) 20 169.35 51.27 t 20 92.85 37.68 df 5.37 38 table 1. means, variance, standard deviations, degree of freedom, and t-test of subjects in experimental and control groups. september 1995 49 i 2 3 4 5 6 7 8 9 10 ii 12 figure 1. a unix directory listing using visual-x tool. the result of the t-test indicated that there exists a significant difference between the performance of the experimental group and the control group (t=5.37, df=38, p<0.001), (table 1). although the preliminary results of this research indicated that the subjects in the experimental group performed at higher levels than subjects in the control group, a larger sample is recommended for future research. figure 2 shows the mean time to complete each task utilizing visual-x and traditional interfaces. in general, figure 2 shows the superior performance of visual-x interface over traditional interface for each task that was performed by experimental and control groups. me (seconds) r 100 .,— . . _ . _jj:.llvisual-x interfacei a ! * |w a5[14°!"9| * a ; traditional interface! » i"0]*10^"'"'**'1"11*" 1 n 11 iieoi '23hiji sa ai a io*< <«< x) bbm : 1 i 1 •visual-x interface cdtraditional interface j task figure 2. mean time to complete each task utilizing sql and visual-x interfaces. 50 ajis gardiner, m. and hitch, g. (1987) principles from the psychology of memory: part 2-working memory, applying cognitive psychology to user-interface design, new york: john wiley & sons, inc., pp. 135-163. johnson, b. (1992) tree viz: treemap visualisation of hierarchically structured information, proceedings of acm chi'92 human factors in computing systems conference, pp. 369-370. jones, w. and dumais, s. the spatial metaphor for user interfaces: experimental tests of reference by location versus name, acm transactions on office information systems, vol. 4, no. 1, pp. 42-63. kahle, b. and medlar, (1991) a. an information system for corporate users: wide area information servers. connexions-the interoperability report, vol. 5, no. 11, pp. 2-9. mackinlay, j., robertson, g., and card, s. (1991) the perspective wall: detail and context smoothly integrated, proceedings of acm cht91 human factors in computing systems conference, pp. 173-179. mccahill, m. (1992) the internet gopher protocol: a distributed server information system. connexionsthe interoperability report, vol. 6, no. 7, pp. 10-14. north, m., and north, s., sarah: (1993) an information exploration and visualization interface for direct manipulation of databases, proceedings of graphics interface '93 conference, pp. 163. north, m., north, s., and warsi, n. (1994) a novel user interface for information exploration and visualization, australian journal of information systems, vol. 2, no. 1, pp. 52-59. north, s., and north, m. (1993b) an information visualization tool for attributes presentation of objects, proceedings of graphics interface '93 conference, ribarsky, w., ayers, e., eble, j., and mukherjea, s. (1993) using glyphmaker to create customized visualisations of complex data. report git-gvu-93-26, submitted to ieee computer. ribarsky, w., hodges, l., minsk, r., and bruchez, m. visual representations of complex, discrete multivariate data. the visual computer, accepted for publication. sarkar, m., and brown, m. (1992) graphical fisheye views of graphs, proceedings of acm chi'92 human factors in computing systems conference, pp. 83-91. shneiderman, b. (1993) designing the user interface: strategies for effective human-computer interaction: new york: addison-wesley publishing company, treinish, l. (1993) inside multidimensional data. byte 18 (4), pp. 132-133. 52 ajis microsoft word 8 ajis 14-2.doc australasian journal of information systems volume 14 number 2 june 2007 5 a ranking list for information systems journals julie fisher monash university, australia email: julie.fisher@infotech.monash.edu.au graeme shanks university of melbourne, australia email: gshanks@unimelb.edu.au john lamp deakin university, australia email: john.lamp@deakin.edu.au abstract there has been an increasing focus internationally on the quality and impact of research outputs in recent years. several countries, including the united kingdom and new zealand have implemented schemes to base the funding of research on research quality. the australian government is planning to implement a research quality framework (rqf) in the next few years that will impact greatly on funding of research in australian universities. a key issue for australian researchers is how the quality and impact of research is defined and measured in their discipline areas. although peer review is widely used to assess the quality of research outputs, it is expensive and labour intensive. other surrogate quality measures are often used. this paper focuses on measuring the quality of research outputs in the information systems discipline. we argue that measures such as citation indexes are inappropriate for information systems and that the publication outlet is a more suitable indicator of quality. we present a ranking list of journals for the information systems discipline, and discuss the approach we have taken in developing the list. we discuss how the ranking list may be used in defining and measuring the quality of information systems research outputs, the limitations inherent in the approach and discuss lessons we have learned in developing the list. australasian journal of information systems volume 14 number 2 june 2007 6 introduction the quality and impact of research outputs have received increasing attention internationally in recent years. governments are concerned that “public money is invested in research of the highest quality that delivers benefits to both research communities and the wider community” (australian commonwealth government 2006, p11). a further concern is to “identify and reward areas of research excellence” (australian commonwealth government 2006, p11) enabling research funding to be more focused. we adopt the definition of research from the oecd (2002) that is used in the proposed australian research quality framework (rqf), “creative work undertaken on a systematic basis in order to increase the stock of knowledge” (australian commonwealth government 2006, p12). furthermore the quality of research is defined as its “intrinsic merit and academic impact” (australian commonwealth government 2006, p12). the australian government has planned to implement the rqf in the next few years and it will greatly impact funding of research in australian universities. several other countries, including the united kingdom and new zealand have also implemented schemes to base the funding of research on research quality. defining and measuring the quality of research is a contentious issue and varies considerably between different discipline areas. in this paper we focus on how to define and measure the quality of research in the discipline area of information systems. there are several approaches to defining and measuring the quality of research. the traditional and widely accepted approach is academic peer review. this is used in the highest quality journals and conferences and in research grant assessments, and is based on the critical review of research outputs by peers within the discipline area. peer review is labour intensive and time consuming so proxy indicators have been used in defining and measuring the quality of research. the most important of these is the quality of the journal (or conference proceedings) in which the research output is published. this may be measured using impact factors or ranking lists. in this paper we argue that for the discipline of information systems, the use of impact factors is inappropriate and we develop a ranked list of information systems journals for use as a proxy for the quality of research outputs. the paper is structured as follows. the next section discusses how the australian government has proposed to measure research quality within the rqf. we then present a critical assessment of the three main approached to developing journal ranking lists. this is followed by a critical assessment of existing is journal ranking lists establishing the need for an australian journal ranking list. we then discuss the approach we have adopted in developing our information systems journal ranking list, and present the list together with evidence for the rankings. finally we discuss how the ranking list may be used in defining and measuring the quality of information systems research outputs, the limitations inherent in the approach and discuss lessons we have learned in developing the list. rqf quality measures when announcing the rqf, the honourable brendan nelson, then minister for education, science and training said “once implemented, the rqf will provide the australian government with the basis for redistributing research funding to ensure that areas of the highest quality of research are rewarded.” (nelson 2005, 3). the focus for the rqf was to be on the quality of the research and its academic and broader impact. academic impact was described as how well the research was received by our peers and the broader impact was described as the impact and usefulness of the research outcomes on those the research was designed to assist. in 2006 there was a change in australasian journal of information systems volume 14 number 2 june 2007 7 minister and the new minister, the honourable julie bishop, reinforced the government’s intention of introducing a scheme and argued that the rqf “would measure quality through a combination of metrics and review by domestic and international peers” bishop (2006). in order to assess the quality of research outputs, a quality ranking scale of 1 to 5 has been proposed, where a ranking of 5 signifies that “the majority of research outputs were considered to be in at least the top 20% of research in their field internationally, with a significant percentage (>25%) in the top 10%. there was evidence of high international peer esteem and significant impact on the international academic community” (nelson 2005). the most widely accepted measure of peer esteem and impact is the publication of papers reporting research activity in academic journals. mort (2004) argues that “the overall quality of the journal acts as a proxy for the quality of individual papers within that journal.” in august 2006, the rqf departmental advisory group released the rqf guiding principles (dest, 2006). the main measure of research quality proposed in this document is as follows: 1. citation measures (where relevant to the discipline) as: 1.1 citations per publication listed in the research grouping’s ‘body of work’; 1.2 the proportion of publications in the ‘body of work’ which have citations that would put them in the top decile for the discipline (to be derived from thomson scientific (isi) citation benchmarks); the rqf guiding principles does however provide that: 2. in those disciplines where citation measures cannot be readily applied, an alternative measure should be derived through assessment of selected research output from the research grouping’s ‘body of work’. these outputs should be assessed against benchmarks relevant to the discipline (for example, ranked journals, book publishers, conferences, performance venues); in this paper we argue that the use of thomson scientific (isi) citation index is not a useful measure of research quality for the is discipline and develop a journal ranking list as a step towards an alternative measure in accordance with the rqf guiding principles option 2 quoted above. approaches to developing journal ranking lists a number of approaches have been used to develop journal ranking lists. three commonly used approaches are the thomson scientific (isi) impact factor, the use of rankings produced by others and polling eminent scholars in the area for their ranking of journals. in this section of the paper we critically assess these three approaches and argue for the development of an australian is journal ranking list. thomson scientific (isi) impact factor. the isi is a measure of the frequency with which the “average article” in a journal has been cited. the impact factor of a journal is calculated by dividing the number of current year citations to the source items published in that journal during the previous two years. the impact factor can be used to provide a gross approximation of the prestige of journals in which individuals have been published (thomson isi, 2005). listing with isi is not a simple matter of notification. isi aims “to provide comprehensive coverage of the world’s most important and influential research” (thomson isi, 2004). in order to undertake this they are guided by bradford’s law (bradford 1934) which they interpret (thomson isi, 2004): australasian journal of information systems volume 14 number 2 june 2007 8 that the core literature for any given scientific discipline was composed of fewer than 1,000 journals. of this 1,000 journals, there are relatively few with a very strong relevance to the given topic, whereas there are many with a weaker relevance to it. those with a weak relevance to the given discipline or topic, however, typically have a strong relevance to another discipline. thus, the core scientific literature can form itself around various topics, with individual journals becoming more or less relevant depending on the topic. hence isi confine themselves to tracking approximately 8,700 journals from all fields which they believe accounts for 85% of published research and 95% of cited publications. there is an element of irony in that bradford’s investigations were motivated by his disquiet over the inadequate coverage of abstracting and indexing services. isi recognises that this core is not static and their editorial team focuses on evaluating journals on an ongoing basis to determine their eligibility for membership of this core. while they may evaluate around 2,000 new journals per year, only 10-12% become members of the core, and although it is not explicitly stated in their document, it can be assumed that there is an ongoing rate of exit from their core. publishers may submit journals for evaluation, but inclusion is subject to evaluation by the editorial team. being listed is not simple, nor is it speedy. the use of isi as a measure of quality for is research would have a major impact on the information systems (is) discipline. table 1 below shows the number of is journals by year of first publication and the percentage of those journals which are currently in the isi index. the table was developed using data from the index of information systems journals (lamp 2004) as at october 2006. a number of observations may be made from the table: • approximately 50% of journals that began publishing 10 years ago (1990-1994) are in isi • approximately 80% of journals that began publishing 25 years ago (1975-1979) are in isi • approximately 90% of journals that began publishing 40 years ago (1960-1964) are in isi number of is journals and percentage in isi 0 20 40 60 80 100 120 140 19 40 1 94 4 19 45 1 94 9 19 50 1 95 4 19 55 1 95 9 19 60 1 96 4 19 65 1 96 9 19 70 1 97 4 19 75 1 97 9 19 80 1 98 4 19 85 1 98 9 19 90 1 99 4 19 95 1 99 9 20 00 2 00 4 year of first publication 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% no of journals percentage in isi table 1: information systems journals and the isi index australasian journal of information systems volume 14 number 2 june 2007 9 overall only 36% of is journals are on the isi database. it should be noted that no australian is journals are presently in the isi database and neither of the association for information systems (ais) flagship journals (journal of the ais and communications of the ais) are in the isi database. furthermore, most specialist is journals have a history of less than 10 years. clearly, isi cannot provide a useful measure for rqf assessment of is research quality as its coverage of the discipline is currently inadequate. using citation indexing and impact factors derived from them have been criticised from a number of perspectives. seglen (1997) lists 22 factors which he considers limit the utility of impact factors derived from citation indexing. tellingly, he also notes that he has found little evidence of the “free ride” belief that publishing in a high impact journal will enhance the impact of an article. in a study focussing on citation indexing and ranking of is journals katerattanakul et al. (2003) found inconsistencies between the two measures. the continued validity of many of the assumptions on which links between citation indexing, impact factor and quality are based are also undermined by the growing number of open access journals. open access publishing is showing up as a major influence on impact factor (jasco 2006). furthermore, the choice of which index to use (isi, google scholar, citeseer, abi/inform, or ebscohost’s computer science index, to name a few) is a key issue. using existing ranking lists there are a number of existing journal ranking lists for the information systems discipline and a variety of ways have been used to develop these lists (see lowry, romans and curtis (2004) for a useful summary). some important existing lists are include the association for information systems (ais) mis journal ranking list (isworld 2006), the rainer and miller (2005) ranking list and the journal quality list of (harzing 2006). several australian institutions have also developed their own internal is journal ranking lists which tend to reflect the research proprieties of the institution rather than the broader focus of the whole australian is discipline. both the ais list and the rainer and miller list are composite and based on an average of other lists and present clear evidence for their structure. however both of these lists have a clear north american bias and do not adequately include european and asia-pacific journals. the north american bias problem has also been recognised by katerattanakul and han (2003) who found that european is journals are generally underrated as compared with north american journals. this is not unique to the is discipline: mort et al. (2004) note that in the discipline of marketing the evaluation of journals has typically emanated from the united states and consequently “reflect the views of american academics”. mort (2004) goes on to argue that marketing in australia and new zealand has continued to develop and mature, reflecting a dynamic fusion of the disciplines’ american, european and asia pacific characteristics”. this is similar to the is discipline. the journal quality list is currently in its 23rd edition and collates eighteen different rankings of 909 journals in the field of management, including is, however it does not attempt to produce a composite ranking list and has a european bias, not adequately including american or asia-pacific journals. an additional problem with journal ranking lists is that many were developed some time ago and do not reflect the current quality of journals (rainer and miller 2005) or include more recent journals. furthermore, composite lists that synthesise other lists include some from many years ago with more recent lists, yielding unreliable results. australasian journal of information systems volume 14 number 2 june 2007 10 polling eminent scholars another common approach to developing journal ranking lists is to ask eminent scholars in the area how they would rank journals in their area. mort (2004) sought input from the senior marketing academics in australia and new zealand on their perceptions of quality of 73 identified journals in the discipline of marketing. this approach has also been used in is (walczak, 1999; whitman et al, 1999; mylonopoulos and theoharakis, 2001; walstrom and hargrave, 2001; bharati and tarasewich, 2002; peffers and ya, 2003). there are some difficulties in polling eminent scholars to create journal ranking lists. first, as noted by uncles (2004), there are problems with drawing up the initial list of journals. the is studies cited above varied from 23 to 326 journals. some restricted themselves to refereed publications, while others included professional magazines (lamp 2006). another problem information systems also faces, is where we draw the boundaries of the discipline. existing categorial schemes are unhelpful, as they do not adequately cater for the is discipline (lamp and milton, 2003). even within the discipline, there are differences on the definition of is; the discipline draws on many reference disciplines and uses many different research methodologies, models and frameworks (lamp and milton, 2004). this lack of clear definition may explain the extreme variation in the number of journals used in the studies mentioned above. it also highlights a further problem: given this large range of journals publishing is research, are we really able to assess the quality of journals such as, for example, ieee and acm journals as well as our colleagues in computer science? uncles (2004) also notes the difficulty respondents face in ranking journals with which they are not familiar. process for developing an australian is journal ranking having examined the different approaches to developing an is journal ranking list we decided on a combination of methods, including the development of a composite ranking list using a set of carefully selected existing lists and seeking feedback and confirmation from senior scholars in the australian information systems community using email and discussion at a workshop of senior australian is academics. the process was undertaken in four stages. stage one: identification of existing rankings the first stage involved identifying the ranking lists that would be used for the composite list. to do this we drew on our own knowledge of existing ranking lists. we also undertook a general internet search for information systems journal rankings and finally approaches were made to other leading european and north american professors in the information systems. the ranking lists were selected to ensure that both north american and european lists were included and that the lists were recent and reputable. the initial journal ranking lists used to develop the first composite ranking list the rationale for their selection and the approximate number of is journals each list ranked are described below. • the ais represents the lead is body internationally. the isworld ranking was created based on other journal rankings that had been published by a reputable is academics (135 journals ranked). • vienna university is a very reputable european based university that provides a european perspective. their ranking includes all journals in the business field including is (52 journals ranked) australasian journal of information systems volume 14 number 2 june 2007 11 • lousiana state university (united states) ranking list was developed by a leading north american university (25 journals ranked). • london school of economics, is a leading european university and their ranking list, provides another european perspective. the london school of economics has a highly regarded is dept (25 journals ranked).. • st gallen university: hsg-habilliste this is a journal ranking list for the purpose of getting "habilitation" at st. gallen university. this list has been included again because it is from a leading european university (50 journals ranked). • peffers and ya (2003) published ranking (50 journals ranked) • rainer and miller (2005) published ranking (29 is journals ranked) stage two: initial ranking list drawn up the second stage of the process was to develop an initial composite ranking list. this involved the following: • definitions of three tiers (named tier 1, tier 2 and tier 3) were developed based on the existing is journal ranking lists. journals were assigned to one of the tiers based on their average rankings within the selected lists and an initial composite ranking list established. • this ranking was circulated to a small group of senior is academics within monash university for feedback and minor modifications were made. stage three: feedback on the initial ranking list the third stage of the process involved seeking feedback and confirmation from senior scholars in the australian information systems community. this involved the following: • the initial ranking list was circulated to the heads of is academic units and professors within australia and we sought input from the australian is community. • a request for other australian university rankings was made and three universities provided their rankings. these were incorporated into the composite ranking list and changes were made based on these rankings. • the updated ranking list was circulated within the australian is community prior to the australian council of professors and heads of information systems (acphis) workshop held in canberra in september 2006. stage four: information and finalisation of ranking list the final stage of the process involved a discussion at the acphis workshop in canberra about the ranking list and the development process. the workshop was attended by approximately 30 senior australian is scholars. a number of outputs resulted from the discussion: • definitions of tiers were discussed and confirmed. • labels for tiers were renamed as level a, level b and non-ranked. • journals that were considered to be outside the is discipline were removed from the list (for example ieee journals, international journal of accounting is and the ibm systems journal. this is consistent with the journals rainer and miller (2005) identified as being australasian journal of information systems volume 14 number 2 june 2007 12 computer science or management oriented). it should be noted that these journals will be ranked in the ranking lists of other disciplines, for example computer science and software engineering. • it was also agreed that a further journal ranking list relating to electronic commerce (bharati, & tarasewich, 2002) would be considered. this was done but most journals in the list were not ranked highly and so not included. the new ranking list was distributed through the heads of is academic units and professors email list for comment. it was also made available on the acphis website and the is community were able to provide further feedback for a number of weeks. while we feel that the journal ranking list is current as at march 2007, we acknowledge that over time journal editorships change, new journals become available and journals change. the journal ranking list should therefore not be regarded as final. the list should be regularly revisit and updated based on changes in other ranking lists used in its development and through feedback within the australian is community. the approach we have taken overcomes some of the problems identified by uncles (2004) and mort (2004) in relation to the marketing discipline. the initial lists used were drawn from both north america and europe to ensure the breadth of our discipline was well represented. the process used was an iterative one ensuring there was maximum input from the community. the australian information systems journal ranking list the final ranking list for is journals is presented below. journals within each level are presented in alphabetical order. evidence for the composite ranking is provided in appendix b. we have not included the non-ranked journal list as this is large and is academics will be aiming to publish in the level a and level b journals. level a journals these are the journals we all should aspire to publish in, but it is not expected that many scholars in australia will have more than one or two articles in these journals. those that do will have a deservedly outstanding reputation as the best in their field. decision sciences decision support systems european journal of information systems information systems journal information systems research information and management journal of the association for information systems journal of management information systems management science mis quarterly level b journals these are still top quality journals, and cover a broad range of fields, but they are not strictly in the top level. good scholars can be expected to publish fairly often at this level. australasian journal of information systems australasian journal of information systems volume 14 number 2 june 2007 13 behaviour and information technology communications of the association for information systems data and knowledge engineering database electronic markets human computer interaction information and organisation (formerly, accounting, management and it) information systems (elsevier) information technology and people international journal of electronic commerce journal of the association for information systems journal of computer information systems journal of database management journal of is (acct) journal of information technology journal of strategic information systems journal of the operational research society scandinavian journal of is premier professional journals these are high quality journals with broad circulation that are highly visible within the practitioner community and can potentially have high impact. academy of management executive communications of the acm california management review harvard business review interfaces mis quarterly executive sloan management review discussion and conclusion the journals ranking list presented in this paper has been developed specifically for the australian is community. it includes a balance between north american and european influences and provides a useful basis for assessing the quality of research output. the ranking list may be used in two main ways. first, papers published in journals ranked in level a or level b may be assumed to be of outstanding or top quality respectively and the judgements used in research quality assessment exercises. second, the list provides a useful guide for academics to target their research publications. there are a number of limitations inherent in the approach we have used to develop the list. first the quality of the list is dependent on the accuracy, completeness and timeliness of the list we have selected as the basis for the development of our composite list. second, our list is current as of march 2007 but will rapidly become out-of-date unless periodically updated. a mechanism for updating the list needs to be developed. thirdly, it is difficult for recently initiated journals to join the list until they are well established. there are a number of lessons we have learned in developing the list. first it is important to be very clear about the scope of the is discipline and to decide which journals may be included and which australasian journal of information systems volume 14 number 2 june 2007 14 will be left to other discipline areas to rank. second, individuals hold very strong beliefs about the quality of journals in which they publish and clear evidence needs to be provided to justify the list. third, great care needs to be taken in developing the lists as journals not ranked at level a or level b may no longer be desirable places in which to publish for is researchers. references bharati, p., & tarasewich, p. (2002). global perceptions of journals publishing e-commerce research. communications of the acm, 45(5), 21-26. bishop j (2006) keynote address: knowledge transfer and engagement forum sydney june, http://www.dest.gov.au/ministers/media/bishop/2006/06/b001160606.asp (accessed july 2006) bradford, s. c. (1934). “sources of information on specific subjects.” engineering 137 85-86 (reprinted j information science (1985) 10(4) 173-180). dest (2006) rqf guiding principles. canberra, department of education, science and training (excerpts from this can be found at http://www.dest.gov.au/sectors/research_sector/policies_issues_reviews/key_issues/research _quality_framework/rqf_development_2006.htm#rqf) harzing a journal quality list [online] available: http://www.harzing.com/resources.htm#/jql.htm accessed october2006 isworld journal rankings [online] available: http://www.isworld.org/csaunders/rankings.htm accessed oct 2006 jacso, p. (2006). “savvy searching: open access to scholarly full-text documents.” online information review 30(5): 587-594. katerattanakul, p. and b. han (2003). "are european i. s. journals underrated? an answer based on citation analysis." european journal of information systems 12: 60 -71. katerattanakul, p., m. razi, et al. (2003). “is journal rankings versus citation analysis: consistency and concerns”. ninth americas conference on information systems, tampa, florida. lamp, j. w. index of information systems journals. geelong, deakin university. [online] available: http://lamp.infosys.deakin.edu.au/journals/ accessed: august 2006 lamp, j. w. (2006). “recognition as a distinguishing characteristic of is journals”. australasian journal of information systems, 13(2), 7-16. lamp, j. w., & milton, s. (2003). “an exploratory study of information systems subject indexing”. fourteenth australasian conference on information systems, perth australia. lamp, j. w., & milton, s. (2004). “the reality of information systems research”. second workshop on foundations of information systems, canberra australia. lowry, p., romans, d. and curtis, a. (2004) global journal prestige and supporting disciplines: a scientometric study of information systems journals, journal of the association for information systems 5(2), 29-77 australasian journal of information systems volume 14 number 2 june 2007 15 mort, g., j. mccoll-kenndy, g. kiel and g. soutar (2004). “perceptions of marketing journals by senior academics in australia and new zealand.” austral asian marketing journal 12(2): 51-61. mylonopoulos, n. a., & theoharakis, v. (2001). “global perceptions of is journals: where is the best is research published?” comm. acm, 44(9), 29-33. nelson research quality framework: assessing the quality and impact of research in australia the preferred model september http://www.dest.gov.au/sectors/research_sector/policies_issues_reviews/key_issues/research _quality_framework/rqf_preferred_model.htm#the_rqf_preferred_model accessed july 2006 peffers, k. and t. ya (2003). “identifying and evaluating the universe of outlets for information systems research: ranking the journals.” j information technology theory and application 5(1): 63-84. rainer, k.and miller, m. (2005):"examining differences across journal rankings.", communications of the acm 48.2 91(4). seglen, p. o. (1997). "why the impact factor of journals should not be used for evaluating research." british medical journal 314(feb 15): 498-502. thomson isi the isi® database: the journal selection process [online] available http://www.isinet.com/essays/selectionofmaterialforcoverage/199701.html accessed may 2005 thomson isi the isi impact factor [online] available http://scientific.thomson.com/free/essays/journalcitationreports/impactfactor/ accessed august 2006 uncles, m. (2004). “journal rankings: how much credence should we gives them?” australasian marketing journal 12(2): 67-72. uq business school uq business school journal rankings [online] available: http://www.business.uq.edu.au/research/resources/jnl_rankings.phtml accessed: october 2006 walczak, s (1999) “a re-evaluation of information systems publications forums” journal of computer information systems 40(1) 88-97 walstrom, ka and hargrave bc (2001) “forums for information systems scholars: iii” information and management 39 117-124 whitman, m. e., hendrickson, a. r., & townsend, a. m. (1999). “research commentary. academic rewards for teaching, research, and service: data and discourse”. information systems research, 10(2), 99-109. australasian journal of information systems volume 14 number 2 june 2007 16 appendix a is journal ranking lists used details of the ranking lists used to develop our composite ranking list are provided below. isworld / ais (united states-based), the leading international information systems academic web site includes many is and is related journals. their ranking scheme ranks journals individually and 124 journals are ranked. we ranked the top 12% (15) as tier 1, the next 25% (28) as tier 2 http://www.isworld.org/csaunders/rankings.htm vienna university provides a european perspective and has extensive rankings of all journals in the business field http://bach.wu-wien.ac.at/bachapp/cgi-bin/fides/fides.aspx?journal=true;lang=de lousianna state university (lsu), united states, provides a north american perspective but with few journals ranked. lsu journal ranking isds department (december 2003) http://www.isworld.org/csaunders/rankings.htm london school of economics (lse) provides a european perspective, highly regarded is dept. journals and conferences include only those ranked as ‘first ranked' of which there are 24 only. (not available on-line provided by prof. leslie willcocks in september 2006.) appendix b evidence for composite journal ranking list it should be noted that by the end of stage four there was a significant convergence of the rankings, that is, there was very little difference in how each of the journals was ranked between the individual lists. for this reason only the rankings where there were 50 or more journals ranked have been included, two are european and two are north american. evidence for journals ranked in levels a and b is provided below. level a journals (listed in alphabetical order) ais (is world) vienna uni univer sity of st gallen peffers & ya (2003) decision sciences 1 nr b nr decision support systems 1 a nr 1 european journal of information systems 1 a a 1 information systems journal 2 a a 1 information systems research 1 a+ a+ 1 information and management 2 a nr 1 australasian journal of information systems volume 14 number 2 june 2007 17 journal of management information systems 1 a b 1 management science 1 a+ a nr mis quarterly 1 a+ a 1 operations research 2 a a nr level b journals (listed in alphabetical order) ais (is world) vienna uni univer sity of st gallen peffers & ya (2003) australian journal of information systems 3 b nr 2 behaviour and information technology 2 b nr 3 communications of the association for information systems 2 b b 1 data and knowledge engineering nr a a nr database 2 a nr 1 electronic markets 3 a a nr european journal of operations research 3 a a nr human computer interaction 2 nr nr nr information and organisation (formerly, accounting, management and it) nr b nr nr information systems (elesvier) 3 b nr 2 information technology and people 2 b nr 2 international journal of electronic commerce 2 a b 2 australasian journal of information systems volume 14 number 2 june 2007 18 journal of the association for information systems 2 b b 1 journal of computer information systems 3 nr nr 2 journal of database management 2 nr nr 2 journal of is (acct) 3 nr nr nr journal of information technology 3 a nr 3 journal of strategic information systems nr b b 2 journal of the operational research society nr a nr nr scandinavian journal of is 4 b nr 3 ajis vol 4 no. 1 how strategic are strategic information systems?1 david avison*, alan eardley** and philip powell* * department of management, university of southampton, southampton so17 1bj, uk. ** school of computing, staffordshire university stafford, st18 oad, uk. # information systems research unit warwick business school, university of warwick, coventry c v4 7al, uk. key words: business strategy, competitive advantage, information technology, strategic information systems, strategy frameworks. abstract there are many examples of information systems which are claimed to have created and sustained competitive advantage, allowed beneficial collaboration or simply ensured the continued survival of the organisations which used them these systems are often referred to as being 'strategic'. this paper argues that many of the examples of strategic information systems as reported in the literature are not sufficiently critical in determining whether the systems meet the generallyaccepted definition of the term 'strategic' that of achieving sustainable competitive advantage. eight of the information systems considered to be strategic are examined here from the standpoint of one widely-accepted 'competition' frameworkporter's model of industry competition . the framework is then used to question the linkage between the information systems and the mechanisms which are required for the enactment of strategic business objectives based on competition. conclusions indicate that the systems are compatible with porter's framework. finally, some limitations of the framework are discussed and aspects of the systems which extend beyond the framework are highlighted introduction for almost as long as computers have been used in business, the strategic potential of automated information systems has been recognised. kriebel (1968) and whisler (1970) were among the first to identify the strategic importance of information systems and to discuss the necessary linkage between the information systems and a business framework. systems which exhibit this linkage, and which enable companies to carry out specific business strategies, are termed 'strategic information systems' (huff and beattie 1985, ciborra 1991, krcmar and lucas 1991). there are several contexts within which information systems have been said to play a strategic role, and a number of business frameworks have been invoked to describe the phenomenon of the strategic information system. this variety is less than helpful in defining the meaning of a strategic information system. doyle (1991) considers the frameworks which have been used in this context to be diverse and allencompassing but that 'each may illuminate a particular aspect of the [system's] environment'. there has however, tended to be a consensus, with some notable exceptions (huff and beattie 1985), that strategic information systems are those systems which enable firms to achieve strategies of sustained competitive advantage over their rivals (ciborra 1991, krcmar and lucas 1991). some authors have attempted to draw a distinction between strategic and competitive' systems (huff and beattie 1985). the distinction centres upon the supposition that strategic systems help senior managers to create and monitor the strategic plan, whereas competitive systems help them to make the strategy operational. this view, while having merit, has tended not to garner the support of the academic or business communities. therefore, the definition of strategic information system used here is an information system which is intended to enable its operator to achieve and sustain competitive advantage over its rivals under a variety of environmental conditions. this view is important to the choice of strategy framework for this study. there is a considerable body of literature describing examples of information systems which have been instrumental in achieving business success for their operators. however, on examining these cases individually, it is often not clear to what extent the systems' success in achieving advantage is due to the features or attributes of the information system . in other words, is there something about these systems which has been crucial in enabling their operators to gain and to maintain an advantage over their rivals, or are the information systems in question simply a part of the overall resource portfolio of a successful company? this paper sets out to identify a suitable framework for describing a business strategy based on competition, and then applies that framework to a range of information systems which are said to be 'strategic', in order to assess whether the systems show features which match those of the framework. if this matching is evident, then it is reasonable to assume that the systems are 'strategic' in the sense of being instruments of competitive advantage. in short, it is 1 an earlier version of this paper was presented at the 5th international forum on technology management. 11 a.iis vol 4 no. 1 the intention to examine the applicability of a widely-accepted competition framework to some of the frequently-reported examples of strategic information systems, using a comparison of the features of the framework with those of the systems to establish the match. the framework as described, the intention here is to select a framework which may be used to define an information system as 'strategic' (rather than merely important or successful) and then apply the chosen framework to some of the 'classic1 examples of strategic information systems. the choice of framework is clearly paramount. of the available frameworks, porter's model of competitive forces (porter 1980) is cited most frequently in the literature (boon 1988) and was developed initially as a tool for helping firms choose a strategy which is appropriate for a given market environment. the use of porter's framework implies that the firm's management will seek to implement 'strategic moves' which seek to position the firm in its environment by balancing five environmental forces: competitors, customers, suppliers, substitute products and/or services and new entrants to the market or industry. the potential of these forces is to allow the firm to act in one or more of the following ways: • to change the basis of competition among rivals; • to reduce the bargaining power of suppliers; • to reduce the mobility of customers in seeking alternative suppliers; • to prevent the introduction to the market of substitute products or services; • to manipulate entry to the market by creating or demolishing barriers to entry. these strategic moves may be carried out within the context of three generic business strategies: • establishing cost leadership as a way of acquiring market share; • differentiating its products or services from those of its rivals; • focussing on one particular market niche or sector. in the context of strategy development, the strategy serves the long-term aims which the organisation has, the strategic moves or 'thrusts' are options which will take the organisation in the chosen direction, and the moves are implemented by 'mechanisms' which are directed at a particular target (wiseman 1985). porter's framework, as extended by others (porter and millar 1985, cash and konsynski 1985, mcfarlan 1984), suggests and identifies it-based mechanisms by which the objectives of the strategy may be implemented. earl (1989) summarises this work and suggests two further uses for the framework: in identifying strategic applications of information technology; and for testing ideas for gaining competitive advantage against predetermined business objectives. earl's explanation of the uses of porter's framework in line with the work of other authors, is depicted in table 1. table 1. porter's framework in use (after earl 1989) force potential mechanism 1. competition among rivals change basis of competition compete or collaborate 2. mobility of customers lock-in to firm differentiate or create switch costs 3. supplier bargaining reduce bargaining erode or share power 4. substitute products innovate develop new product or services or add value to existing product 5. new entrants to market create barriers to entry demolish or erect own a review of the literature shows the impact which porter's framework has had on research in the link between business and information technology. an early application of porter's framework by parsons (1983) assessed the strategic importance of it applications and cash and konsynski (1985) use the same framework to evaluate the strategic importance of inter-organisational information systems. the framework also forms a basis for work which explores the links between corporate strategy and it (bacos and treacy 1986) and an examination 12 ajis vol 4 no. 1 of the strategic risks to information systems development (vitale 1986). porter's framework is a 'convenient and useful tool for communicating the potential of it to managers and practitioners' (boon 1988). porter's model is termed 'the most fundamental of these (frameworks)' in an empirical study into the strategic uses of information and information technology (king, hufnagel and grover 1988). it is also a useful benchmark against which to evaluate the success of strategic information systems as the theory on which the framework is based are grounded in the business and cultural environment within which most of the 'classic' strategic systems were developed. ciborra (1994) sees drawbacks to the use of porter's framework in an examination of four strategic information systems cases (three of which are re-examined here). he emphasises the discepancy between a well-defined approach to developing strategic information systems (such as would be achieved with a framework) and the realities which often include chance, serendipity, trial and error, and gross negligence, suggesting that successful competitive systems need to emphasise the creation of value, and that such systems should be rare and difficult to imitate. he concludes, 'the question is, then, how to achieve rarity and imperfect imitability in designing a sis [strategic information system]? the analysis of the competitive environment as prescribed by porter (1980) is lacking in this respect' (ciborra 1994 p. 15). the methodology a large body of rather unstructured material exists which describes individual instances of strategic information systems. a search of the relevant journals and the index of the harvard business school case service reveals twenty such systems which are specifically described in terms of their strategic success. for the puposes of this study a set of criteria was devised to screen these candidate cases, the criteria are intended to clarify the issues and to ensure compatibility between the chosen cases, rather than to test compliance with porter's framework. cases which did not meet the following criteria are excluded: • there must be a means of comparing one case with another. in practice at least two cases from each of a number of business sectors are included; • there must be a description of the background or business context within which the system was developed; • the combined references have to describe the examples in sufficient detail to allow a reasonable level of understanding of the systems; • the information systems must have been developed in the first instance by a single entity, as opposed to a collaborative venture. after the screening process there remain eight well-documented cases which can be placed into three business sector classifications. these are presented in table 2. a secondary role of this paper is to re-evaluate some of these cases, which have often been the subject of 'secondary analysis', and to take the opportunity is taken to go back to the primary sources to assess whether the claims made for these systems are justified. a number of issues emerge when the literature on strategic information systems is examined: • a small number of examples of strategic information systems constitutes the majority of the published cases. these cases have been described frequently and used as exemplars, often going on to become regarded as 'classic' in their field. this may have affected the reviewed sample, as other cases which may be equally deserving of analysis are not sufficiently well-reported to meet the inclusion criteria; • many of the references to the systems are anecdotal and few would claim to be original studies. there is a shortage of comparative or analytical work and few publications seek to justify formally the claims of strategic importance which have been made on their behalf. exceptions to this are boon's (1988) research into 14 cases of strategic information systems and the empirical study by king, hufnagel and grover (1988) of 84 companies which claimed to have developed such systems; • most of the published cases (and all those which meet the inclusion criteria of the present study) are from the usa, and the majority of the systems were developed progressively over more than twenty years, from the late 1960s to the early 1990s. 13 ajis vol 4 no. 1 table 2. strategic information systems cases sector/company system type airlines american airlines sabre reservation/booking united airlines apollo reservation/booking distribution baxters (ahs) asap order/stock control federal express cosmos consignment/routeing mckesson economost order/stock control financial services citicorp gtn global trading merril lynch cma integrated cash a/c. philadelphia nat. bank mac atm network the methodology of the review is to examine the cases and the framework in two ways. first, the systems are categorised and analysed to determine if a correlation exists between their features as described in the cases and porter's framework, in order to test the validity of the claim that the systems have strategic dimensions. second, the results of the analysis are used to evaluate porter's framework as a valid instrument for the identification and definition of strategic information systems. case analysis one problem apparent in the study is to match the method of analysis to the degree of detail available in the published data. one study of strategic systems uses economic analysis techniques to examine the deployment of information technology and the levels of customer switching cost which would be involved in reponding to some of the forces (barua, kriebel and mukhopadhyay 1991), but this is at an aggregated industry level, and no such data is available for the individual cases. in the end, it was decided to use a form of feature analysis (law 1988) to match the published features of the chosen systems with those of porter's framework. this involved searching the literature for evidence of information systems-based mechanisms which the firms had put into place to exploit or to counter the potential strategic moves suggested by porter's model and to record the evidence in narrative form. although it is not possible to reproduce the richness of data which is present in the published cases, the findings on each of the five forces are briefly as follows, and are reinforced in table 3: competition between rivals it is apparent that the key to success in this area is to use is to change the basis of competition in an industry. the systems which were the most successful at changing the basis of competition were those which had 'first mover' or 'early adopter1 status and achieved an early technological advantage. of the eight cases examined, five had changed the basis of competition in their respective industries. the exceptions were united airlines and philadelphia national bank (pnb), both of which entered systems into a sector which had already seen the effects of systems introduced by rivals. in such cases a competitive struggle ensued which only resolved itself when a state of stability was reached, in the former case by gaining control of the booking channels and in the latter case by assimilating many of the competing atm systems. in the case of the sabre and apollo systems the two systems were in direct competition since potential passengers preferred to fly on whichever airline could offer them the first booking. sabre was, however, the pioneering system in the airline industry and was the first to gain widespread acceptance with travel agents when the agents and the airline companies failed to agree on a joint nationwide reservation system (emery 1990). american airlines had its system in an advanced stage of development and was therefore able to control the booking channels of a large part of the u.s. air travel industry. it soon began processing reservations for its competitors for a fee, which came to be an important new form of revenue, amounting to almost 50% of profits in 1991 (senn 1992). the american hospital supply (later baxter's) asap (later called asap express) and mckessons' economost systems have similar functionality, but were operating in different markets (hospitals and pharmacies). both companies were therefore able to revolutionise the way in which orders were placed in their respective sectors, and both systems were judged to be strategic successes. federal express corporation gained a first-mover advantage by effectively creating a new industry the overnight package delivery service (senn 1992) through the use of its cosmos parcel routing and tracking system. the advantage was soon wiped out, however, as a 14 ajis vol 4 no. 1 number of companies sought to exploit the new overnight market by developing competing systems. these 'follower' companies, which included united parcels service, airborne freight corporation and emery air freight, had the advantage of being able to observe the functioning of the cosmos system and to take advantage of more recent technological developments. federal express, therefore, did not find that changing the basis of competition by using a strategic information system has given it the degree of long-term advantage which might have been expected, and it had borne the research and development costs for the entire industry in return for a temporary competitive advantage. three things are apparent from this scenario; first, that it is particularly important for 'first-movers' to gain the maximum benefit from their systems as soon as is possible, and second, that the initial competitive advantage may be at best temporary unless very unusual circumstances apply. third, any move involving strategic applications of is may, unless defended well, invoke a superior response from competitors. dos santos and peffers (1991) concur with this view in their analysis of it adoption by us banks in which they find that smaller banks and banks with a slow rate of growth were not able to make gains from the early adoption of it, while other banks found the advantage to be temporary (particularly in terms of market share) although some long-term gains were made in terms of efficiency. customer mobility several of the systems show clearly attempts to achieve control of the firms' supply chains. in particular, the two airline booking systems and the ahs and mckesson systems show evidence of mechanisms designed to 'lock-in' customers by opening a channel which differentiated their service from that of their competitors. this created switching costs if the customer sought to exercise mobility by dealing with competitors. in return it has been claimed that the benefits which resulted from the use of the system were passed on to the customer (demons and row 1988). merrill lynch's cash management account (cma) system offered a complete range of financial services to individual customers for the first time, based upon the use of a computerised personal account. the range of features which the system offered was intended to make it unnecessary for the bank's customers to use competitors' services. the customer would also incur switching costs in the event of a change of service provider, as most of the competing financial products carried a front-end charge. american airlines was also able to lock-in customers in two ways, both of which made possible by sabre. first, there was the securing of the channels of services which locked-in travel agents by providing a dedicated it system at the agents' premises and second, this was achieved by using the system as the basis for running the american advantage frequent flier programme, which created a club of over 11 million regular users who were 'rewarded' for not using the services of competitors (hopper 1990). bargaining power of suppliers there is little direct evidence that any of the chosen systems were developed with a strategy in mind of reducing supplier bargaining power, although laudon and laudon (1988) cite the examples of sabre and apollo as examples of strategic information systems which have such an affect upon suppliers. they state, 'for instance, united has begun to include in its contracts with travel agents provisions that prevent them from using any other airlines (sic) system if they want to keep apollo. this is an example of how a strategically oriented supply system gives a supplier a market advantage over a distributor and a customer'. similarly, the asap system could also be considered to affect supplier relationships. cash et al. (1992) quote the president of baxter's hospital systems division, 'asap express opens up a new era of electronic order entry in which suppliers must compete in other areas such as product line breadth, distribution capabilities, and value-added services' (p. 162). asap express therefore promotes not only the sales of baxter's products but also those of its suppliers. baxter's also initiated 'vip' a system which linked the firm to its suppliers. this was a later system and it may be that the mechanisms for making such a strategy operational did not exist at the time the systems were originally initiated and so do not 'show up' in the case studies. substitute products and services several of the organisations described in the cases appear to have used a strategy of incremental development and improvement of their systems as a counter to the threat of competitors entering the market with new products or services. two strategies can be applied by a firm to counter this force. first, the firm which is being attacked may try to introduce a new strategic information system with which it hopes to restore the state of competitive advantage which it previously enjoyed, and second, the firm may seek to develop an existing 15 ajis vol 4 no. 1 system to give it extra functionality with which it may outperform its rivals. a third strategy, which is less well researched, is to create illusory barriers to entry by undertaking visible high spending on it (powell 1992). it is interesting to compare the way in which two of the companies had to cope with the force of competition from the substitute services of rivals. federal express and merrill lynch are almost at opposite ends of the spectrum in terms of the need to stay one step ahead of the competition. merrill lynch's cma was a radically new financial service (developed from ideas put forward by the stanford research institute) which enabled the firm to capture 90% of the market for investment account products in the first year of operation. perhaps because of its initial success, the system was not enhanced for several years and when systems were developed by competitors which offered similar facilities in different ways its share of the market dropped to 50% (boon 1988). in contrast, the american airlines' sabre system and federal express corporation's cosmos show distinct evidence of 'value adding' developments being made as a direct response to innovations made by the competition in order to stay one step ahead. some of the improvements federal express made to the original cosmos system include organisational enhancements (e.g. a hub-and-spoke distribution complex), operational features (e.g. on-line parcel tracking and direct aircraft assignment) and technological developments (e.g. display units in truck cabs). these competitive developments allowed the firm to avoid competing on price alone and have been 'instrumental in allowing it to maintain its position in the market' (senn 1992, p.10). new entrants to the industry proponents of porter's framework suggest that there are three main reasons why a first-mover's development of a strategic information system should act as an effective barrier to prevent rival firms from entering a given industry or market sector: • information systems of the scale that is necessary to achieve anindustry-wide strategic effect are bound to have long planning and development times. this alone should guarantee a considerable period of freedom from competition; • the development of such systems is extremely costly in terms of risk, finance and human resources, so that only firms with a large resource base are able to produce them; • the technical and organisational expertise which is needed to produce this type of system is in short supply. in fact, experience shows that these barriers can be illusory or at most temporary. time and cost factors are constantly being reduced by methodological and technological developments and the supposed monopoly of technical expertise has been broken (if it ever existed) by the fluidity of the it labour market and the prevalence of consultancy services. therefore, the evidence from the cases to confirm or deny the effectiveness of information systems as barriers to entry is inconclusive. one of the criteria for including the cases in this study is that there should be at least two examples in each business sector all the companies, therefore, have competitors, suggesting that the systems may have reduced the number of entrants or have delayed entry but have not been effective in preventing it completely. there is more evidence of the success of information systems at demolishing barriers than at erecting them. several of the cases describe a company which has been able to enter a market by using information technology as a substitute for some other factor: 'high street' bank branches in the case of merrill lynch and a distributed sales operation in the case of american hospital supply. companies have chosen to collaborate in the development of strategic information systems for a variety of reasons. merrill lynch, for instance, had to collaborate with bane one in order to provide the current account and credit facilities for its cma system, as existing u.s. legislation did not allow an investment bank to offer such services. it is interesting to note that, in view of merrill lynch's long dominance of the market, this collaboration appeared to have no effect on other firms entering the market, as bane one subsequently acted as a partner to several of merrill lynch's competitors of in a number of ventures. nevertheless, the combination of merrill lynch's investment expertise and bane one's technological innovation went almost unopposed in their market sector for five years. philadelphia national bank used collaboration as a way of enabling its shared atm system 'mac' to overcome the effects of scale and first-mover advantages which were enjoyed initially by girard (later named mellon bank) in the philadelphia area with its 'george' atm system. pnb maintained ownership of the network as a service provider and was able to attract enough banks into its mac network for the system to overtake george (by then called cashstream) and pnb was eventually able to absorb cashstream into its own system when it took over part of mellon bank's operations (demons 1990). 16 ajis vol 4 no. 1 discussion by comparing porter's framework with the details of the cases it is possible to draw general conclusions about the framework which cast light upon the nature of strategic information systems. the findings indicate that the strategic information systems described provide a partial match with porter's framework in that they define strategic moves, identify strategic potential and suggest mechanisms in relation to some, but not all, of the forces of the model. the results are summarised in table 3 and are described below: • six of the eight cases show evidence of the characteristic strategy which porter terms 'changing the basis of competition'. all the systems have been cited as the direct cause of the relative success of the companies which operated them. in the case of direct competition between companies for a defined market (for instance american airlines and united airlines) it is apparent that one of the companies has achieved a measure of competitive advantage over the other. in the cases of collaboration between companies (for instance pnb and merrill lynch) the degree of success is more difficult to define, although it is notable that cma was pre-eminent in its market for five years and mac did absorb most of the rival atm systems in the pittsburg area. table 3. summary of strategic moves in response to forces company american airlines apollo airlines baxters ahs federal express mckesson citicorp merrill lynch pnb system sabre forces in framework 1 2 3 .response mechanisms change basis asap cosmos economost gtn cma mac change basis change basis change basis change basis change basis incentive switching costs switching costs switching costs switching costs range of services added value erect added demolish value erect demolish key to forces: 1. competition among rivals 4. substitute products or services 2. mobility of customers 5. new entrants to the market 3. supplier bargaining power • six of the cases exhibit signs of the strategy of reducing customer mobility by an apparent attempt to lock-in customers (as opposed to an increase in market share through competitive pressure). attempts were made to exercise such power over the customer in several ways; american hospital supply and mckesson used the strategy of sharing the technology with the customer to create switching costs should the customer seek to develop an in-house equivalent of the system. american airways secured the loyalty of air travellers by using sabre to log their flights and to provide a frequent flier bonus as an incentive and merrill lynch used the range of financial services which are offered by cma to make it unnecessary for customers to open specialised accounts with competitors. • none of the cases exhibit the related strategy of responding to suppliers' bargaining power by sharing or eroding that power. this may be due to the size of the sample or to some characteristic of the sample, such as the era in which the systems were developed. there are a number of instances of this feature being displayed by strategic information systems (e.g. stalk 1988, gooding 1988). 17 ajis vol4 no. 1 • as the literature about the cases fails to describe the market situation which existed before the development of the systems, it is impossible to assess how many of the systems were part of a strategy of response to competitive products or services. neither is it clear how many firms had tried to introduce competitive systems and failed, since these cases go unreported. there is direct evidence that the management of the companies which operated four out of the eight systems made value adding enhancements to the systems' functions in order to stave off the threat of substitution. it is clear that these enhancements are strategic moves, independent of any technology-led improvements which may have been made to the systems. • there is evidence that four out of the eight cases had responded to the force relating to market entry. two of the cases show evidence of the use of the system to create a barrier to other companies entering the market. in these cases the barrier is related to the scale of the system and to the first mover effect rather than the difficulty of development. in fact, few of the systems used more sophisticated technology than the average transaction processing system of the relevant era. the proportion of systems which exhibit this characteristic is less than might be expected. two of the cases (federal express and pnb) show clear evidence of the use of systems to demolish barriers which had been erected by competitors with first-mover advantage. pnb applied the classic strategy of barrier demolition by using a shared atm system to enter the market and to gain market share. the cases of american airlines and citicorp (with its gtn system) also exhibit this characteristic quite strongly. it is clear from the above that the majority of the systems included in the survey map onto porter's framework to an extent which goes beyond coincidence. it is immediately apparent that there is a disparity in the number of forces to which the companies have responded. all the cases presented in the previous sections exhibit evidence of compliance with at least one dimension of the framework, and three of the eight comply with three dimensions. the most common factor which influenced the firms' strategic moves is the force of competition from rivals. the systems in seven of the eight cases had mechanisms which addressed this. the only force which was possibly not present in the systems was that of seeking to control the bargaining power of suppliers. of course, it could be argued that porter's framework has been derived from practice. indeed, perhaps derived from analysis of some of the cases analysed here. hence, a good mapping between the framework and reality would be expected. it may be tempting to construe from these findings that the systems which best fit the framework are more strategic or more successful. certainly in the cases where a direct comparison within a business sector between a high-scoring and a low-scoring system is possible (for instance american airlines and united airlines), the company in the former category has been more successful in several important business areas. limitations in applying the framework one view is that since the systems in the review comply strongly with some of the competitive forces with porter's model (i.e. those relating to rivals and to customers), and since the model is widely accepted as a strategy framework, then by definition these systems must have a significant strategic value to their operators. however, there are other aspects of the systems in the cases which are not covered by porter's framework but which are worth examining in relation to their strategic or competitive potential: • it is possible to confuse the concept of 'strategic' with 'important' or 'pervasive' when discussing information systems. all the cases describe large, complex systems which reach into many areas of the businesses that use them. there is no doubt that some parts of these systems impart a strategic advantage to the firms, but to state that the information system as a whole is strategic or 'competitive' would seem to go against common sense. it may, therefore, be more accurate to state that the majority of the systems in these cases are information systems with strategic dimensions or strategic components. to do otherwise would confuse importance to the business with 'strategic value', a fault which one author has called 'a mischievous misconception' (emery 1990); • it may be possible to refer to any system in a successful, competitive firm as 'strategic'. just as it is possible to confuse the terms 'strategic' and 'important' when referring to information systems, so it is easy to refer to an information system as 'strategic' when what is meant is 'successful'. a review of the literature finds few descriptions of strategic information systems which are failures (at least in the short term). exceptions are the case of olin chemical group (senn 1992) and "winchester tyres'. it may be pointed out that the firms which operate some of the notable strategic systems already dominated their respective markets before the systems were developed. however, in the 18 ajis vol 4 no. 1 cases from the distribution sector under study here (asap, economost, and cosmos), the applications (and in the case of cosmos, the industry) simply could not have existed without the information systems to drive them; • it may be said that a system can only be strategic if it is an indigenous part of an organisation's long-term strategy, and many of the examples in the study do not appear to have been planned from their inception, but were developed in an evolutionary manner (ciborra 1994, krcmar and lucas 1991). the example which best illustrates this is american airlines' sabre, which was conceived in 1959 as an internal data processing system. relatively few of the enhancements which were to give the system its strategic value to the company over the next 16 years appear to have been planned in any long-term way, but were developed as a result of piecemeal enhancements in response to changes in the air travel market such as the breakdown of negotiations for a shared system and deregulation of the industry. in the case of economost, mckesson's former is manager admits that the system owes its development not to a top-level, company-wide strategic plan but to a local initiative by one of its business units (demons and row 1988). this in no way diminishes the strategic impact which the systems had upon the organisations and porter's framework models this dynamic with the concepts of 'strategic moves' in response to one or more of the forces. • it could be argued that porter's framework does not describe one of the most important aspects of a strategic information system, that of providing information from which strategic plans may be formed and upon which strategic decisions may be based, which is one definition of a strategic information system (fredericks and venkatraman 1988),. all the systems have reporting and interrogation facilities but in most cases these can only be seen as improving the efficiency of that system (for instance, asap allows for the optimisation of stock and re-order levels). there is no evidence that any of the systems provide the sort of information which would enable top-level management to identify, for instance, the sort of opportunities and threats to which the companies respond in formulating strategies. sabre has a facility which monitors competitors' fares and optimises its own ticket prices to maximise revenue, but this could hardly be called strategic, although it may help american airlines to be more competitive. conclusions the information systems described in these cases show a significant degree of compliance with at least part of porter's framework. this framework is one of the most commonly accepted models for describing the strategic actions of companies in an industry. the dynamic of tension, response and balance which this model implies is echoed in the incremental way in which most of these systems described in the cases have been developed over time. however, there are other perspectives by which the strategic nature of such information systems may be judged. one view is that the nature of an information system (in terms of scale, success and importance to the business) is often confused with strategic worth. another is that the cases are in some way tied in to the use of the framework, and that the definition of strategic value which is attributed to the systems is rooted in the common cultural and temporal background which gave rise to both the framework and the systems. the business-driven need to gain a strategic advantage from the use of information technology and to justify the development of such systems may have led to claims that the systems are strategic in nature. there is clear evidence from the cases of the use of information systems as mechanisms to exploit and to secure business opportunities. there is, however, an obvious problem in the use of porter's framework in this study, which is related to the definition of the term 'sustainable', as the cases show that competitive advantage is rarely sustained without changes being made to the information systems which are intended to achieve it (senn 1992, ciborra 1991). in fact, this should not detract from the strategic value of the systems in the study, or from the relevance of porter's framework, as the model describes adequately the need for continual change in response to a dynamic environment. strategies such as 'locking-in' customers and changing the basis of competition in a market sector can be effective over a prolonged time period, providing that continuous improvements are made to the strategy and to the mechanisms which are in place to enact it. porter's framework can provide a valuable tool for scanning the environment and for prompting such changes to reestablish competitive advantage by the use of information systems. references bacos, j.y. & treacy m.e., (1986), i.t. and corporate strategy: a research perspective, mis quarterly, 10(2), pp.106-119. 19 ajis vol4 no. 1 barua, a., kriebel, c.h. & mukhopadhyay, t. (1991), an economic analysis of strategic information technology investments, mis quarterly, september, pp.313-331. boon, s.n., (1988), factors facilitating the use of information technology for competitive advantage: an exploratory study, information and management, (15), pp. 191-201. cash, j.i. & konsynski, b.r., (1985), i..s. redraws competitive boundaries, harvard business review, 63(2), pp. 134-142. cash j., mcfarlan w., mckenney & applegate l., (1992), corporate information systems: text and cases, new york, irwin. ciborra, c, (1991), the limits of strategic information systems, international journal of information resource management, 2(3), pp.11-17. ciborra c., (1994), the grassroots of i.t. and strategy, in ciborra c. and jelassi t., strategic information systems; a european perspective, chichester, wiley, pp.3-24. demons, e.k. & row, m., (1988), mckesson drug company: a case-study of economost a strategic information system, journal of management information systems, 5(1), pp.37-50. demons, e.k., (1990), mac-philadelphia national bank's strategic venture in shared atm networks, journal of management information systems, 7(1), pp.5-25. dos santos, b. & peffers, k., (1991), the effects of early adoption of it: an empirical study, proceedings of 12th. annual conference on information systems, new york, pp. 127-140. doyle, j., (1991), problems with strategic information systems frameworks, european journal of information systems, 1(4), dec., pp.273-280. earl, m.j., (1989), management strategies for i.t., london, prentice hall international. emery, j.c., (1990), misconceptions about strategic information systems: editor's comments, mis quarterly, june, pp. vii-viii. fredericks, p. & venkatraman, n. (1988), the rise of strategy support systems. sloan management review, 29 (3) pp. 47-54. gooding, g., (1988), exploiting i.t. in business development: ford in europe, in m.j. earl (ed.), information management: the strategic dimension., oxford, oxford university press, pp.87-97. hopper, m., (1990), rattling sabre: new ways to compete on information, harvard business review, 68(3), pp. 118-125. huff, h.l. & seattle, e.s., (1985), strategic versus competitive information systems, business quarterly, 50(4), pp.97-102. king, w., hufnagel, e. & grover, v., (1988), using information technology for competitive advantage, in m.j. earl (ed.), information management: the strategic dimension, oxford, oxford university press, pp.87-97. krcmar, h. & lucas, h.c., (1991), success factors for strategic information systems, information and management, 21(3), pp. 137-145. kriebel, c.h., (1968), the strategic dimension of corporate systems planning, long range planning, pp. 812. law, d., (1988), methods for comparing methods, manchester, n.c.c. laudon k. & laudon j., (1988), management information systems. new york, macmillan. mcfarlan, f.w., (1984), it changes the way you compete, harvard business review, 62(3), pp.98113. parsons, g., (1983), it: a new competitive weapon, sloan management review, 25(1), pp.3-14. porter, m.e., (1980), competitive strategy, new york, free press. porter, m.e. & millar, v.e., (1985), how information gives you competitive advantage, harvard business review, july-aug, pp. 140-160. powell, p., (1992), information technology evaluation: is it different? journal of the operations research society, 43(1), pp.29-42. scott morton m., (1992), reshaping the corporation of the future, revue des systemes de decision, 1(2-3), pp.121-126. senn, j.a., (1992), the myths of strategic systems; what defines competitive advantage, information systems management, 9(3), pp.7-12. stalk, g., (1988), time the next source of competitive advantage, harvard business review, julyaug, pp.41-51. vitale, m.r., (1986), the growing risks of is success, mis quarterly, 10(4), pp.327-334. whisler, t., (1970), information technology and organisational change, new york, wadsworth. wiseman, c., (1985), strategy and computers, new york, dow jones irwin. 20 ajis vol. 6 no. 2 may 1999 object-oriented analysis: a voyage of discovery elizabeth a. kemp institute of information sciences and technology college of sciences massey university palmerston north, new zealand email: e.kemp@massey.ac.nz abstract in this paper the experiences of a business analyst who worked on several projects using the object-oriented (o-o) approach are described. the business analyst in question had initially no background whatsoever in o-o and worked within organisations which were also making the same transition. her reflections on the events that transpired are discussed and some suggestions made for organisations migrating to o-o. introduction employing the object-oriented (o-o) approach for application development has been acknowledged to have benefits for certain applications: higher quality software, more flexibility, code re-use, greater resilience to change etc (graham, 1994; booch 1996). nonetheless, for individuals and organisations, the migration to this paradigm is non-trivial and involves a considerable effort (meyer, 1995; goldberg and rubin, 1995; booch, 1996). in this study, a business analyst was interviewed on three separate occasion over a fifteen month period about her experiences learning the object-oriented paradigm. the underlying objective was to find out how one individual working in the computing industry came to grips with the move to the object-oriented paradigm. the business analyst described her role in various projects and reflected on her experiences. because she was working within businesses that were also engaged in the transition to o-o, it also proved possible to think about lessons to be learned for organisations when implementing the 0-0 approach. two voices can be heard in this paper, that of the business analyst and that of the author. initially the emphasis is on the business analyst with her account of the three projects on which she worked and what she felt that she had learned. the author then analyses the findings, taking account of the literature currently available to organisations migrating to o-o. research of the kind described here falls within the interpretivist method of research where knowledge is socially constructed. it is not possible to make any statistical generalisations from the results. instead, qualitative research offers the opportunity to describe and understand what is happening in the world. as walsham (1995, p 6) notes "there are no correct and incorrect theories but there are interesting and less interesting ways to view the world." it is hoped that the research described here will make a contribution to the debate about the migration to object-oriented technology and will be more rather than less interesting. methodology there are a variety of philosophical views in relation to the nature of reality. positivists assume the existence of a single objective reality that can be observed and expressed in terms of universal laws and empirical generalisations (keat, 1981). others hold a subjectivist ontological position where they believe in multiple realities that exist as mental constructions whereby people interpret and make sense of the world (white et al., 1992). human behaviour is continually constructed and reconstructed on the basis of individual's interpretation of the situations they are in (hammersley and atkinson, 1983). to access people's meanings, the researcher cannot be objective and external to the phenomena under investigation, rather the researcher must play the role of the participant observer (hammersley and atkinson, 1983). it is this role which is played by the author of this paper who both describes and comments on what occurred. the author has several years experience of both teaching and research in software engineering, focusing recently on the o-o paradigm. an in-depth, longitudinal case study approach was followed. walsham (1995) observes that this is appropriate in the interpretivist tradition. a longitudinal study allows for the unfolding of events over time to be studied. benbasat et al. (1987) maintain that case research is particularly appropriate for practice-based problems where research and theory are still being developed. it can be argued that studies of o-o practitioners are still at that stage. walsham, (1995, p 15), notes that any extrapolations from case studies of this kind rely on the "plausibility and cogency of the logical reasoning used and in drawing conclusions from them." it is also possible, however, to compare the findings with the literature on making the transition to the object-oriented paradigm. 76 ajis vol. 6 no. 2 may 1999 one business analyst was chosen as the subject of the case study. she was a suitable candidate for various reasons: her degree included papers in systems analysis and she had several years experience in the computing industry, specialising in data modelling. the business analyst, therefore, had a suitable background for someone learning the o-o approach. she was, moreover, keen to participate and would make herself available when required. during a fifteen month period, the business analyst was interviewed on three separate occasions: about her experiences as a complete novice, when familiar with the main concepts of the o-o approach and after completing the analysis phase of a large project. the interviews were unstructured, allowing the business analyst to describe her own experiences and reflect on coming to grips with the o-o paradigm. the business analyst checked that the account of events outlined in this paper was correct. for reasons of confidentiality, details of examples have been changed when necessary. organisational concerns advice on managing the transition to object-oriented software development is provided by various authors. hares and smart (1994) talk of crossing the "rubicon" when the move is made to an object-oriented environment. they discuss what types of computer systems to migrate and when to make the transition. confidence in the technology is an important prerequisite for making the move. booch (1996) addresses the project management issues that may arise in his text 'object solutions." he believes that ideally an organisation should start with a pilot project since this allows important learning to take place in a controlled environment. staff should be adequately trained and suitable tools made available to them. even so, booch warns that it might take several months for programmers to be at ease with the paradigm. using experienced members of the team to mentor others may alleviate some of these problems. booch also describes a macro and micro process that can be followed when developing o-o software. in particular he emphasises the need for a requirements phase during which an o-o model of the problem should be built. involving real users throughout the project is seen as vital. finally the project manager is warned that if more than two existing practices (such as the terminology, the platform, the method, the staff, the tools, the development language etc) are changed then the outcome is uncertain. cockburn has recently written a book called "surviving object-oriented projects." he comments that organisations have claimed significant reduction in development time but that people "rarely seem to want to name the actual costs of making the move to objects, perhaps to avoid scaring away future newcomers. there is, therefore, a lack of information on what sorts of unpleasant surprises await one when starting an o-o project" (cockburn, 1998, p xv). rebecca wirks-brock has also commented that introducing objects may create more problems than it solves. (cockburn, 1998, p xiii). cockburn provides a wealth of advice to managers working on their first o-o project. a suitable project must be selected, "not too big, not too small, not too important, not too unimportant" (cockburn, 1998, p 33), and an appropriate methodology selected. careful planning is required with standards established; incremental and iterative procedures should be followed. he warns managers that they must invest in training their staff. managers should not have unrealistic expectations but expect programmers converting to the o-o paradigm to take 6 to 9 months to become productive. access to people experienced in the technology is also required. both consultants and mentors are needed. consultants can help an organisation avoid pitfalls whilst the role of the mentor is to help developers acquire good habits. as cockburn points out, however, experience is a relative term in an immature industry. "more object projects are starting up than there are experts to lead them, and even large consultancy houses are having difficulty finding experienced o-o developers...afteryou train novices for a year, they may quit and become self-declared experts "(cockburn, 1998, p 44). consequently, managers are at risk of hiring experts with only one year's experience. ambler (1998) goes beyond organisational issues to discuss ways in which individuals can help themselves overcome the o-o learning curve. the following activities were seen as useful: reading books, magazines and journals on the topic, attending courses and conferences, browsing the internet and obtaining a mentor. getting started with a degree in english and information systems, the first position held by the business analyst was in the area of data administration for a large organisation. she carried out a wide variety of tasks including data modelling and the migration of software modules into production from maintenance environments. code tables also had to be kept up to date. she was a member of a team that built a corporate data model for the organisation. in addition she was also responsible for reviewing the standards and methodology manuals for software development. 77 ajis vol. 6 no. 2 may 1999 at this point, with around three years of experience, a move was made to a position with a major consulting organisation. one of her first tasks was to produce the analysis for a small system to be written in java. there was no time to become familiar with the object-oriented approach that would have been most appropriate in the circumstances. instead the analysis was centred initially around a textual description of user requirements and the production of an entity relationship diagram (erd). the erd formed the basis of the normalised, physical data model for a relational database. the programmer took the entities as the basis for classes and objects. subsequently dataflow diagrams and an event list were drawn up. there was also the requirement to identify the associated methods. the dataflow diagrams proved to be of little value in this respect. instead the methods were taken from the screen designs (derived from the event list) produced by the business analyst. the process of developing the class diagram and physical data model was more iterative than strictly necessary due to inexperience of the team. the system was quickly up and running, however, and proved its value to the organisation. the business analyst recognised, though, that she was too dependent on the tools and techniques of structured analysis. when the analysis for a second (prototype) system was required, she researched various approaches to the object-oriented paradigm, reading several texts as well as material from the web and trying out the rational rose case tool. she based her approach to this project on the method proposed in "case studies in objectoriented analysis and design" by yourdon and argila (1996). first the business analyst produced the written user requirements documentation with the aim of specifying ultimately the relevant business objects, attributes and methods. then, following yourdon and argila's process of textual analysis, she identified the significant nouns, about 15 to 20. these represented the candidate objects. to recognise significant objects, the 3-view approach of building entity relationship diagrams (erd) etc was used. a large erd diagram was drawn up before developing a class diagram. entities, however, could not just be mapped across into objects. one difference, according to yourdon and argila is that an object does not have to hold data eg. "funds transfer." nor do all associative entities have to be represented as objects. object models can show many to many relationships. nonetheless, the business analyst found it difficult as a novice analyst to move away from the relational approach. there was the requirement to produce a physical data model for the relational database which was based on the erd as well as a class diagram. this caused her some problems. she initially moved across an attribute such as "account type" into a class diagram as an attribute of an object when the different types of account should have been represented by an "is-a" relationship. her advice to others was to "be aware when you are thinking relationally and be aware when you are thinking objectoriented." the attributes of the objects were taken directly from the logical/physical data models. they had been obtained initially from the user requirements specification and also research on the subject such as an analysis of forms from banks. experience indicated that attributes should be checked with users but that they will never have a full list of attributes in their head. the methods to be associated with objects were also identified by the business analyst, following a procedure suggested by yourdon and argila (1996). this involved going through the life cycle of an object imagining what it does from start to finish. the analyst also drew up an event list which assisted with this process. she believed, though, that the resulting class diagram was in some ways too detailed (all objects had create, update and delete associated with them) and in others incomplete. whilst she identified all major methods it proved difficult to think of all the optional activities that might occur. associating a method with the appropriate object also proved problematic, "sometimes you are not sure where to put the methods." in one case, there were three objects to which a particular method could be attached. the event list, however, whilst it did not prove particularly useful for method identification and placement, was seen as invaluable for validation purposes. a check was made that every event was supported by the business object diagram. the business analyst also had to verify that the activities of the class diagram were supported by the physical data model. only a few small omissions were made which were quickly rectified. overall, the business analyst was concerned about the problems caused by having to develop both erds and a class diagram. she asked herself whether yourdon's recommended starting point (erd views) was appropriate. another member of the consultancy advised her to discard this approach and move to one that was more objectoriented. how though, the analyst wondered, would she be able to move from a textual user requirements documents to a class diagram and be sure that she was retaining everything of importance. even though the business analyst was at this point essentially a novice she was still able to stand back and comment on her experiences to this point. she quickly realised after the first project that she needed to familiarise herself further with o-o techniques and turned to yourdon and argila (1996). she had been taught during her university days yourdon's (1989) approach to structured systems analysis and thought that it made sense to move in a systematic way from a structured approach to o-o. on reflection, however, she believed that this approach ultimately proved confusing, especially with regard to the identification of objects and methods after discussing with others in the consultancy the problems of moving from an erd to a class model, she was 78 ajis vol. 6 no. 2 may 1999 advised to start in future from a purer o-o approach, underpinning the o-o approach by a relational database. there was still the underlying problem, however, of a practice run by the time the business analyst started work, as a member of a team, on a large (two year) transaction processing project, she had, therefore, had experience of using one approach to o-o analysis. she, together with other analysts, had also attended a course on uml (unified modelling language). a comprehensive o-o analysis methodology had been worked out for this project by the analysis group. a great deal of research had been undertaken which included searching the web for relevant information. the project involved the rewriting and upgrading of an existing system. the first stage involved a practice run of tools and techniques to show that the approach selected would work. this proof of concept for one subsystem (customer management) was expected to take six weeks from initiation to implementation (using java and corba2). for the organisation concerned, this was its first venture into o-o. some important techniques that can be used in analysis, taken from a guide to unified modelling language (uml) by booch, rumbaugh and jacobsen (1998), will be described here. one of these is the use case. a use case essentially is a description of a sequence of actions carried out by a system which provides a result to an actor. a use case diagram, therefore, is one which shows a set of use cases and actors. actors in their turn are defined in terms of "a coherent set of roles that users of use cases play when interacting with the use cases" (booch et al, 1998, p 457). figure 1 shows a use case diagram for the situation where a system is being developed for a garage. the system, inter-alia, will handle the recording of repairs made to vehicles and the ordering of parts required. the actors ("manager" and "mechanic") are represented by stick figures and the use cases ("place order" and "record repair details") as ellipses. any individual can play more than one role, for example the manager of the garage can also act on occasion as a mechanic. manager mechanic figure 1: an example of a use case scenarios describe the flow of events for a use case. there may be primary and secondary scenarios. essential sequences of actions are incorporated into primary scenarios and alternative sequences in secondary scenarios. to help identify the important objects (together with their attributes and behaviour) for an application, crc cards can be used. crc cards were developed by ward and cunningham (1989) to document collaborative design decisions in an object-oriented environment. they have subsequently been used to help create the design since these cards can be used to model classes (booch et al, 1998). the responsibilities or obligations of a class can be translated into a set of operations and attributes. to return to the proof of concept phase, a scoping diagram was built, in consultation with users, showing the selected customer management sub-system at the centre and arrows, representing interfaces, to other 79 ajis vol. 6 no. 2 may 1999 subsystems/actors. then, from a prior user requirements analysis, six high level processes were identified and the links between them were shown in a context diagram. the detail of each of the six processes was modelled using role activity diagrams (rads), following huckvale and ould (1994). for each high level process defined, rads were produced by walking through all stages of each process with the users. "as is" rads (how things work now) and "to be" rads (how things will be done in the future) were produced. several member of the organisation were seconded to work full time on the project and so were available whenever required by the analysis team. triggers, activities and states were shown and the business analyst observed that users found such diagrams easy to read. a goal was associated with each rad which enabled a check to be made that the specified activities met that goal. there were various problems, however, that occurred when using this notation according to the business analyst. people tended to think about what they would like rather than the current systems and the "as is" diagrams and "to be" diagrams were virtually identical. the value of the "as is" diagrams was rather limited. problems arose because of the inexperience of the group. the primary and secondary scenarios were not identified. this meant that the users had to work at a high level of abstraction when modelling the detail of the processes. in these circumstances, the business analyst commented, "it was difficult to get people thinking thoroughly enough." a great deal of useful detail, it subsequently transpired, was omitted. furthermore the roles of users were not sufficiently teased apart and one group of workers who appeared to carry out the same tasks had later to be separated into two. nonetheless, producing the rads helped the team understand the business. the use cases were identified and the use-case diagram built after producing the rads. the business analyst believed that it was essentially too late as, by this time, the process view was dominant and it was difficult to come in with a fresh perspective. the use cases should have been developed earlier, the analyst believed, when the high level processes were identified. the analysis team debated at length the problem of handling a processbased system within the o-o paradigm. it was decided that the use case approach allowed commonalties to be identified where a process oriented approach uncovered differences. both were required. one possible solution, the use of activity diagrams to model processes, was not investigated since this notation was not at that time incorporated into the rational rose case tool nor did the literature indicate how these could be fully integrated into an o-o methodology. after completing the use cases, the candidate objects for the class diagram were identified by the analysis group so that interaction diagrams could be developed. this was done by a textual search of the rads which produced about 20 candidate objects. after a brief walkthrough, invalid objects were eliminated and the remainder put into a class diagram. class responsibility collaboration (crc) cards were then employed to allow users to select attributes and behaviour for each object. crc cards, the business analyst believed, should also have been used earlier to help select/validate object selection. interaction (sequence) diagrams were then drawn. it became apparent at this point, according to the business analyst, that "we needed more information about which attributes were passed on from the objects at what point. so it was not enough for us to know that an object had 5 attributes." information was needed about which attributes to update at particular times. this information could have been obtained by using crc cards again but time did not permit this. interaction diagrams for this application proved to be very confusing as there were so many exceptions to be handled. the class model was, though, refined in the light of this exercise as were the use cases. eventually there were only two principal use cases with various extensions. these formed the basis for the written specifications which were handed over to the designers. with regard to interface design the approach referred to as lo-fi prototyping (rettig, 1994) was employed. instead of building a system to demonstrate the interface, lo-fi prototyping is paper based. a paper prototype of the interface is developed with a user playing the role of the computer. for the proof of concept project, users walked through a particular scenario following this approach. this was done, however, with a very limited goal which was to find out what knowledge had to be captured where. someone was chosen to act the part of the computer and handed out the attributes as indicated by the scenario. this proved to be a very valuable exercise as users were able to provide even more relevant detail. this process was not carried out early enough, however, for this detail to be included within the analysis. the business analyst believed that walkthroughs of this kind should be carried out much earlier on. finally, with regard to the interface, state transition diagrams were used to model volatile objects. there were not many of these, however, in this transaction processing system. the sequence in which the outputs were produced during the proof of concept phase and the dependencies between them are shown in figure 2. a major problem occurred, however, once the class diagram was handed over. the designers wished to minimise the use of inheritance. consequently according to the business analyst "we had a huge political debate at this time." the decision was made that the designers would make a limited use only of inheritance and roll the objects up, that is amalgamate inheritance hierarchies such as customer, customer type etc, in their own version of the class diagram. this enabled them to have fewer objects, each with a large number of 80 ajis vol. 6 no. 2 may 1999 attributes and methods. they were able in this way to reduce the number of messages passed. the detailed structure provided by the analysts, however, had been necessary to validate business rules so it had not been a waste of time drawing it up. where a problem did arise was with the interaction diagrams which contained information useful for system development. the interaction diagrams, however, were linked to the analyst's class diagram and so the designers either had to lose this information or adapt the interaction diagrams to match their own version of the class diagram. another problem which faced the designers was having to match up the attributes with those in the logical data model (produced at an earlier stage.) only a very small part of the system specified was successfully implemented. overall, the business analyst felt that "the most important thing i learned was that there is not any mature o-o methodology." she felt that during the proof of concept phase she was effectively doing research to find a methodology that could be used for a process-oriented application. she realised that there had been major problems with the sequencing of the stages in the methodology. for instance, use cases and crc cards were not used early enough. moreover, the full power of a technique such as crc cards was not employed because of a failure to understand its potential. these problems occurred because all of the members of the team had a background in structured systems analysis and were probably too wedded to a process view and process-oriented techniques. use cases, however, she believed were vitally important. they assisted with identifying roles and actors, segmenting functionality and allowing for both components, such as screens, and code to be re-used. in the future she would suggest that they be built earlier in the proceedings. the analysis proper the third interview took place after the first stage of the analysis proper. for this phase of the project uml (3.5) was the standard followed and the tools used were rational rose 98 and requisite pro version 3.0.7. even at this point the team was "still a bit confused about how they were going to marry business processes with use cases." however, advice from an expert in o-o allowed this critical issue to be resolved. the team followed the process he suggested with some additions of their own. first, a context diagram was produced which showed all the actors outside the business and their interactions with it. from this, 6 high level processes were identified for the business. no links were shown between them. instead, for each process, the associated business cases were specified. in one case there were 40 of these. in effect, a hierarchy of use cases was built although this, as the business analyst observed, is not a solution generally accepted in the literature. the top level (the 6 processes) had to be supported by the package facility of the rational rose case tool. the only problem that arose was that packages do not interact with actors. all the use cases were named and any relevant interactions between the actors shown in a use case diagram, again drawn in using the rational rose case tool. all this was achieved in one week. the team was also advised to produce the detail of the business use cases first and, based on these, build system use cases. teams were set up composed of users seconded to work on the project and analysts. these had the responsibility of developing the business use cases: writing the associated descriptions (scenarios) and identifying flows, alternate flows and exceptions. a clear distinction was made between the actions of the actor and those of the organisation. only three weeks were allowed for this stage and the business use cases were not completely filled out. the intention was to use them to derive system use cases and not to become swamped with detail. the sequence of actions involving the actor and the organisation was shown in a two column table (figure 3). one column showed the actor's action and the other the business's response including the title (membership services officer) of the person who carried it out. this idea worked so well that users were able to complete the relevant sections themselves and just bring them back for review. as the business analyst remarked " these tables focused people's minds on who was doing the action and that you are going to get responses." from these business use cases, it was possible to move directly to system use cases which followed the two column format previously described. in these, the workers within the organisation become the actors. their activities are now shown in the left hand side of the table with the system activities on the right (figure 4). in system use cases the actors act directly with the system. these system use cases included all relevant data but did not include interface details such as "open window." this was left to the interaction diagrams. to avoid incorporating many alternative courses of action per use case, each separate option was specified following the "extends" feature of uml which allows multiple scenarios per use case. 81 ajis vol. 6 no. 2 may 1999 figure 2: outputs for the proof of concept phase the business users (ten of whom were seconded to work on the project) assisted by members of the analysis team went through the business use cases, constructed the system use cases and added any necessary detail. this information came from various sources such as the current staff procedures manual and from the detail incorporated within existing software documentation. for the use cases which handled extensions to the existing system, members of the organisation brainstormed together with analysts on the team and built decision trees of the knowledge to be incorporated. managers who had to sign off the use case specifications also added further information. the only problem was that some users were out of touch with what was happening within the organisation. as the analyst commented "it's nice having users totally accessible to you, sitting by your side but because they have been out of action for at least a year it is not ideal." the process of building the use cases would have been completed more quickly if the users had had more recent experience. nonetheless the managers who signed off the specifications were always aufait with the situation and could make suggestions when necessary. 82 ajis vol. 6 no. 2 may 1999 once the system use case specifications had been signed off it was necessary to include relevant calculations within them. a team of programmers went through the code of the existing system and documented all of these. when incorporated within a systems use case, the calculations became a list of system actions or a "uses" use case if the calculation was complex or used by more than one case. the business analyst remarked that "our use cases are becoming quite different beasts now from how they were before we put this detail in. it is not just this interaction between the actor and the system any more." several steps may be involved in the response to one worker action. actor 1. customer submits change of address 4. customer receives confirmation of address change organisation 2. membership services officer updates address 3. membership services officer sends confirmation figure 3: flows in a business use case batch processing also had to be handled within this framework. all the batch processes were represented by a use case with a list of system actions associated with an actor referred to as "father time. " all in all, there were more than a hundred use cases. some of these were for essentially non-functional requirements such as the setting up, for security purposes, of access rights. auditing requirements such as those for performance or disaster recovery were described in a separate requisite pro table as a "non-functional" requirement type. system actor 1. 3. 5. membership services officer enters member number membership services officer enters new address the membership services officer confirms the change system response 2. the system displays the customer details including address 4. the system requests confirmation of the change 6. the system updates the customer record and prints off a letter to the customer confirming the change of address figure 4: system use case the signing-off of the specifications was not the final stage of the analysis. it was still necessary, in another iteration, to complete the specification of the "uses" use cases, that is to identify common functionality. the team had been advised to leave this to a comparatively late stage so that they fully understood the business and avoided coming to premature conclusions. finally, state transitions diagrams were used to show particular sequences of actions that would have to be carried out by a staff member. this notation was only employed since there was no support in the rational rose case tool for activity diagrams. as well as taking part in the development of the context diagram, business and system use cases, the business analyst also had the responsibility of writing the guidelines for developing the use cases. the business analyst was not involved with the building of the class diagram but did participate in two reviews of it. the initial plan had been to derive the business object model from interaction diagrams drawn for the core 83 ajis vol. 6 no. 2 may 1999 use cases, the top 20%. these core use cases were those most critical to the business and incorporated the major functional requirements. this process proved very tedious and was abandoned. instead a crc card approach was used to build the business model. a team walked through the 30 selected use cases and produced a crc card for each object. in this way reasonable cover was provided. this diagram, built in the rational rose case tool, showed the relationship between objects, attributes and methods. it was only expected at this point that 80% of the objects would be identified. other objects and methods would be determined later when the other use cases were considered. the class diagram would, in the future, have to be matched against the logical data model. once the object model was in place, interaction diagrams could be built to link actions to classes and also show interface details such as "window." coding could then commence with all the necessary detail present in the system use cases. the outputs for the process described above and the order in which they were produced are shown in figure 5. overall the business analyst now felt confident that there was a way to handle a large scale transaction processing system in o-o. "analysis paralysis" had been avoided by the iterative nature of the process used for o-oa. she re-iterated, too, her opinion that system use cases were invaluable for system specification, allowing the relevant parts to be sliced off for implementation by programmers. nonetheless, there were strengths, she believed, in other approaches. the business analyst thought that it was due to her data modelling background that all relevant data attributes were incorporated into the system use cases. this was important in a transaction processing systems. discussion in this section, the experiences of the business analyst and the organisations described above are reviewed in light of the literature on migrating to o-o technology. to summarise, the business analyst emerged from the fifteen months spent working on o-o projects feeling quite confident of her ability in this area; the various pieces of the jigsaw coming together to form a whole. during this period she had read widely in the area and attended courses on both uml and the rational rose case tool. she had also had the benefit of discussions with a consultant in the area of 0-0. lengthy debates with analysts and designers also helped to clarify many important issues. ambler (1998) listed many ways in which an individual could improve their knowledge of oo technology: reading books, magazines and journals on the topic, attending courses and conferences, browsing the internet and obtaining a mentor. the analyst (whilst not aware of the ambler text) had carried out all of these activities save for obtaining a mentor. unfortunately there was no-one available on any of the projects with the expertise to perform this role. this situation is quite common. cockburn (1998, p 72) noted that "market demand currently exceeds the supply of properly trained mentors." whilst not ideal, the situation was alleviated by extensive interaction with other analysts, designers and the consultant. whilst booch (1996) and cockburn(1998) both comment on the time taken for programmers to become effective using o-o techniques, it can be argued that analysts are in much the same position. considerable time and effort are required. business analysts have to have wide ranging expertise and need to be familiar with techniques from both structured systems analysis and the 0-0 paradigm for systems requirements. a synthesis of knowledge has to be achieved so that analyst knows which techniques to use for which projects or parts of projects. organisations, as booch (1996) and cockburn(1998) recognise, are faced with many problems when making the transition to o-o technology. these were particularly severe for the organisation that wanted to develop the large transaction processing system. more than two existing practices were changed (method, staff, tools, development language). according to booch (1996) the outcome is uncertain in these circumstances. nonetheless, progress was made on the project. many of the actions taken by the management on the large transaction processing system correspond to the suggestions made by booch (1996) and cockburn (1998). a pilot project (the proof of concept phase) was undertaken. an appropriate methodology was developed after outside expertise was brought in. an objectoriented model was built during the analysis phase. a process of incremental development was followed. this allowed a divide and conquer approach to be employed. first the use cases were identified and the business use cases were built. this was only done, however, at a level of detail that was sufficient to form the basis for building the system use cases. it was at this point that detailed analysis commenced. when the flows for the system use cases were finalised the required calculations were added. non-functional requirements were also associated with them. finally, the "uses" use cases were developed. what was a very large task was made manageable by breaking the analysis process into several stages and progressively adding more detail. iteration around the requirements allowed time to be set aside to re-visit parts of the analysis. the system use cases were checked by business managers and mistakes and omissions had to be rectified. the class diagram, too, was being developed iteratively, in at least two passes. real users were involved from the start, although not all of them 84 ajis vol. 6 no. 2 may 1999 were, by the time of the analysis proper, up to date with what was happening in the organisation. finally, staff were provided with the appropriate training and tools. on-functional requirements figure 5: outputs for the analysis phase it is worth examining one of these issues in more detail, the development of an appropriate o-o methodology. the major difficulty that arises for a new problem is the selection of a suitable process. tailoring a methodology from the many notations available in order to specify the solution is a non-trivial task. for the transaction processing application described above there were various features of the solution that do not arise in o-o texts (which often deal with real-time systems.) the differences include the hierarchy of use cases and the introduction of an actor such as "father time'. the system use case descriptions (scenarios) were somewhat unusual too, often involving a long list of system actions and a complex calculation in response to just one actor action. developing the methodology for the project, however, took a great deal of time. initially, it was hoped during the proof of concept phase to finalise the process that would be used on the project. it became clear to all the members of the analysis team, during this stage, that the approach suggested was 85 ajis vol. 6 no. 2 may 1999 unsuitable for o-o analysis. team members had become aufait with the nature of business problem and various notations but had not brought together the different aspects of the o-o analysis process. at this point additional expertise was brought in. it is debatable whether bringing in the consultant earlier would have had much benefit. when attending the uml course, assistance had been sought for the proof of concept phase but only the use of activity diagrams was suggested as a possible solution. by the time of the consultant's arrival, the team had carried out a preliminary analysis and knew exactly what their problem was. the solution proposed, the hierarchy of use cases, is not one found generally in the literature and had to be tailored to their problem. with the major problem solved, the team were able, themselves, to deal with issues such as how to handle batch processing. it was vital, though, to obtain assistance in a timely fashion too soon and the problem would not have been sufficiently understood, too late and the project would have failed at the analysis stage. conclusion for the business analyst involved in moving across to the 0-0 paradigm from a data modelling perspective, the journey involved many stages. she learned first that a gradual move from structured analysis (data flow diagramming and data modelling) to o-o was fraught with problems especially with regard to the identification of objects and methods. a purer o-o approach was needed. use cases provided this. she also realised that they should be produced earlier rather than later in the analysis as happened in the proof of concept phase. of most importance, the business analyst discovered that the o-o approach selected has to be tailored to fit the problem. descriptions of 0-0 usage in textbooks do not always map to real world problems. it was easy at the start of a large project to feel overwhelmed by the complexity. in the transaction processing system described above, there were functional and non-functional requirements to be considered. batch processing was still necessary and had to be handled within an o-o framework. seeing how the different aspects of the system could be specified in a way that allowed the whole to be comprehended was an invaluable learning experience. overall, the analyst now feels that she can see the advantages and disadvantages of the various approaches with which she is familiar. she does not believe that her previous experience of structured analysis can be discounted even within the o-o framework. at times a process or data focus can prove very useful. just as individuals are having to come to terms with o-o, so are organisations. changing over to the o-o paradigm is not a straightforward matter as many authors have pointed out. a large number of methodologies can be found in the literature and many case tools are available. even the advent of the standard, uml, has not resolved this situation; there are still competitors around. small projects are one way for an organisation to advance (see the prototype system). another way of learning about o-o is to have a proof of concept phase. some might argue that an organisation should not commence a large project without having in place people at every level with requisite experience. unfortunately, due to 0-0's lack of maturity, it is impossible to know in advance that the right people are in place. success on projects of one kind (real-time) does not ensure success on others (transaction processing). organisations have to appreciate that solutions may have to be tailored to the problem in hand. what is essential, however, is that when assistance is required it is made available. a divide and conquer approach is also recommended for larger projects. this fits in well with the iterative nature of o-o which allows the detail to be added in at later stages when the problem is properly understood. staff training is another important issue for organisations migrating to o-o as it not only provides practice in the use of tools and notations but allows for discussions with those experienced with the technology. the business analyst was fortunate that she not only received adequate training but had the opportunity to develop her skills incrementally, moving from smaller to larger projects. this is an ideal situation which should as far as possible be replicated by organisations. an organisation should also endeavour to ensure that those starting off in o-o analysis have the opportunity to consult with those more experienced than themselves even if this expertise cannot be found within the organisation. it is a major challenge for organisations to recognise that their analysts may not be fully productive from the start but require time to adapt to the o-o paradigm. given the current shortage of trained staff, a related difficulty may be that of retaining staff once they are capable of acting as mentors and consultants to others. references ambler, s.w. (1998) building object applications that work, cambridge: cambridge university press. beck k. & cunningham w. (1989) "a laboratory for teaching object-oriented thinking", oopsla'89 conference proceedings, october 1989, new orleans, pp 1-6. benbasat, i., goldstein d. k. & mead, m. (1987) "the case research strategy in studies of information systems", mis quarterly, 11(3), pp 369-386. 86 ajis vol. 6 no. 2 may 1999 booch, g. (1996) object solutions, menlo park, ca: addison-wesley. booch, g., rumbaugh, j. & jacobsen, i. (1998) the unified modeling language user guide, menlo park, ca: addison-wesley. cockbum, a. (1998) surviving object-oriented projects, reading, mass: addison-wesley. goldberg, a. & rubin k. s. (1995) succeeding with objects, reading, mass: addison-wesley. graham, i. (1994) migrating to object technology, wokingham: addison-wesley. hammersley, m. & atkinson p. (1983) ethnography: principles and practice, london: routledge. hares, j. s. & smart j.d. (1994) object orientation : technology, techniques, management, and migration, chichester: john wiley & sons. huckvale, t. & quid m. (1994) "process modelling: why, what and how" in software assistance for business re-engineering, eds, k. spurr, p. layzell, l. jennison and n. richards, chichester: john wiley & sons. keat, r. (1981) the politics of social theory: habermas, freud and the critique of positivism, oxford: blackwell. meyer, b. (1995) object success, london: prentice hall. rettig, m. (1994) "prototyping for tiny fingers", communications of the acm, 37(4), pp 21-27. walsham, g. (1995) interpreting information systems in organisations, chichester: john wiley & sons. white, l., gamble, d. & blunden, s. (1992) an examination of the epistemological foundations of an actionresearch project, discussion document, faculty of agriculture and rural development and the social ecology centre, university of western sydney, hawkesbury. yourdon, e. (1989) modern structured analysis, englewood cliffs, n j: prentice-hall. yourdon, e. & argila c. (1996) case studies in object-oriented analysis and design, englewood cliffs, n j.: prentice hall. 87 ajis vol. 6 no. 2 may 1999 integrating is and hci using activity theory as a philosophical and theoretical basis helen hasan, department of business systems, university of wollongong, australia helenjiasan @ uow.edu .au abstract the fields of information systems (is) and human-computer interaction (hci) are both multidisciplinary in nature and each is searching for an identity as a legitimate discipline in its own right, with a firm theoretical basis. activity theory offers a set of concepts, structures and terms which are eminently suited to both fields. it has its origins in the cultural historical psychology of russia, and avoids the dichotomies between thought and action, and between individuals and society, that are prevalent in western thinking. this paper describes the basic principles of activity theory as they apply to is and hci, with particular emphasis on those systems which provide knowledge about the progress of their organisation to senior managers. activity theory principles will be used in a longitudinal case study to analyse an extensive project which attempted, with limited success, to produce management support systems in an organisation. the activity theory analysis indicates that an integrated is and hci approach would have improved the outcomes of this project. introduction the fields of information systems (is) and human-computer interaction (hci) have much in common. over the past 10 to 20 years there has been an emergence of new conferences, journals and professional associations in both is and hci but both fields are still searching for identities as legitimate disciplines in their own right. davies and ledington (1991) assert that information systems is a hybrid field which is attached to many other disciplines. they claim that, as information is central to our lives, attempts to restrict it to a single area of study are futile. at the same time carroll (1991) describes human-computer interaction as an interdisciplinary area of applied research and design practice. mantovani (1996) warns however that the necessary multi-disciplinary work, crossing barriers separating scientific cultures, can be "highly effort demanding, time consuming, and sometimes distressing, requiring as it does negotiation among researchers with different ways of speaking and thinking". information systems the field of is grew in an era when large transaction processing systems were created and maintained in organisations by specialised is staff. the field's formal methods of systems analysis and design (sa&d) are best suited to the development of operational systems which automate tedious work processes. in the traditional linear sa&d lifecycle, users were viewed as people from whom to get initial requirements and, later on, were there to test and use the final product. a popular is textbook by alter (1996) states that "an information system is a system that uses information technology to capture, transmit, store, retrieve, manipulate, or display information used in one or more business processes". this is a common usage of the term but contains several statements which can be questioned. must an is be computer-based or even rely on technology at all? what is "information"? are is only found in a business context? where is the boundary of an is; does it just include the information processing technology, or should it include people and work processes in the "system"? the fact that these questions arise is symptomatic of the confusion regarding the philosophical and theoretical foundations of is. if is research has any traditional base it must surely be general systems theory (gst). the systems approach would treat the computer technology, its human users and any relevant part of the organisation as one whole integrated purposeful system. systems theorists insist that the system's approach may be the only valid treatment of some problems and the only way to "create coherence out of chaos" (van gigch 1978), even though they admit that such systems are inherently complex and hence difficult to manage. however in practice, many researchers and most practitioners restrict their view of is to little more than the textbook definition of alter (1996) as stated above. human-computer interaction the advent of the personal computer has brought software products into use by the wider community, and the use of computers has become an object of research by ergonomists, psychologists and others. the result has been the emergence of the field of human-computer interaction (hci) which focuses on the users, and the usability, of systems. a significant proportion of hci research has looked at commercial software packages 44 ajis vol. 6 no. 2 may 1999 such as word-processors (see for example the seminal work of card et al 1983) and the introduction of "userfriendly" operating systems (such as the famous star interface from xerox, the forerunner of the macintosh) (preece 1994). hci has been strongly influenced by the fields of computer science, cognitive psychology and ergonomics (human factors). according to carroll (1991) its key concern is to understand and facilitate the creation of "user interfaces", that is, of computers as manipulated by human users. this places the human and the machine on an equal footing as two interacting information processing systems where the output of one is the input of the other (aboulafia & gould 1995). this cognitive science view of hci has reached its limitation from the point of view of prominent hci researchers such as norman (1991) and carroll (1991). system which "informate" zuboff (1988) makes a useful distinction between computerised systems which "automate" as opposed to those which "informate". examples of systems which "informate" are executive information systems (eis), decision support systems (dss), expert systems (es) and intranets. neither is or hci have a strong record in dealing with these types of systems and so they are used here to highlight current concerns in these fields. the strength of hci is its focus on, and insistence on the involvement of, users in every phase of system development. it favours an evolutionary prototyping approach using participatory design (pd) or joint application design (jad) techniques (hasan & gould 1996, hasan et al 1996). however when users are senior managers it is difficult to get them involved in the time consuming pd or jad process. the strength of is is its incorporation of business needs in systems development, but it relies on the establishment of a thorough set of user requirements before the design and implementation phases. with informate-type systems, the standard is techniques of observation and interview, may only elucidate vague requirements from users when they are busy executives (hasan & lampitsi 1995). a comprehensive, holistic approach is essential for the study of projects such as the development of systems to informate, where the user information requirements are unclear and cooperation is required across many organisational units. it will be shown in this paper, that there are at least two complex activities interacting with each other in such projects: the activity of system production (the focus of is) and the activity of system use (the focus of hci). too often those involved in each one of these activities does not appreciate the importance and the complexity of the other. research in this area must contend with dynamic interactions between individuals and groups, within a complex environment and served by sophisticated technology. activity theory offers a set of concepts, structures and terms which are eminently suited to this type of research. it has its origins in the cultural historical psychology of russia as propounded by vygotsky in the early 20th century. for this reason it avoids the dichotomies between thought and action and between individuals and society that are prevalent in western thinking (6 ladder 1993). this paper will investigate the special characteristics of informate-type systems and describe the basic principles of activity theory as they are applicable to systems which provide information or knowledge about the progress of the organisation. activity theory principles will be used to draw parallels between is and hci, and a study will be presented where these principles have been used. mss: information systems support for management with the increasing recognition that information is an important organisational resource, has come a demand for systems that bridge the gap between the huge quantities of raw data in company databases and the provision of useful information to support management. business intelligence (bi), executive support systems (ess), executive information system and more recently intranets and knowledge management, are some of the commonly used names for systems which aim to provide information in a form that suits the styles and tasks of senior managers. these, together with dss and expert systems, are now among the most strategic systems in organisations and will be collectively referred to in this paper as management support systems (mss). in the late 1980s, the work of rockart and de long (1988) on ess contested the notion that rigid computer systems were essentially alien to the unstructured and flexible work of senior managers and provided case studies where ess were becoming indispensable to the work of many executives. however many organisations still do not have a functional information system that is relevant to the work of senior executives and there remains the complex problem of determining the requirements of the system which differ from industry to industry, organisation to organisation and from manager to manager. the work of strategic decision makers has always been inherently complex and "unstructured" (mintzberg 1975, mintzberg et al 1976), but more so today than ever, as the political, economic and social climate is constantly changing. to establish initial specifications for a mss, it would be worthwhile to have an understanding of the 45 ajis vol. 6 no. 2 may 1999 characteristics of the managers, the roles they adopt and the activities in which they engage, using a list of generic management styles, roles and activities such as those published by fidler and rogerson (1996). these general characteristics could then be used to determine the composition of a mss that would support the identified management styles, roles and activities. the contentious issue here is the degree of processing of the data supplied to the mss. do executives require highly summarised data, or something more in the form of information or knowledge? this question raises a debate on the definition of these 2 terms. data, information or knowledge? there has been debate in the is community on the is distinction between data, information and knowledge (isworld 1997). the discussants in this debate agreed that data are the raw facts, events and figures, in fact, the carriers of information and knowledge. most definitions of information implied that information is data which has been sufficiently processed as to be immediately useful and thus is relative to its use. there was even the suggestion that, in common use, information is simply the output of a computer. knowledge was defined in a number of ways such as "information which has been internalised and results in enhanced operations" or "the application of a combination of instincts, ideas, rules and procedures that guide actions and decisions" and that knowledge "normally causes some change in your internal mental map of the external world". the point was also made in the discussion that the field of is should no longer be concerned with data processing or information processing but knowledge management. these concepts are relevant to the design of computer support for senior executives. cognitive research shows that people use map-like structure to make sense of information (fiol 1992) and cognition can be defined as the activity of knowing: the acquisition, organisation and use of knowledge (neisser 1976). norman (1991) states that managers have working mental models of their organisation and fiol (1992) argues that the broad strategic concerns of managers require a portfolio of different kinds of cognitive maps which are used as a decision aid. she contends that the interaction among these maps is important, as is the recognition that many of these maps are in the form of ingrained, and taken-for-granted, assumptions. wagner (1995) observes that decision makers in organisations continually face "messy", wide-and-open, non-quantitative problems and proposes techniques that rely on the human problem solver to "make sense" of the problem in a structured way. a related concept, used by kuutti and virkkunen (1995), randall et al (1996) and others, is the term "organisational memory", a metaphor relating to the collective memories of an organisation's members. the need for "organisational memory" may be a consequence of restructuring and downsizing when organisations discover that much of the organisation's "knowledge" leaves with sacked or released employees. kuutti and virkkunen (1995) propose that there should be methods and systems which can capture this "knowledge" and make it available to new members of the organisation. in summary an ideal mss should provide knowledge management that enhances the accuracy of managers "mental maps", support the solution of "messy" problems and captures organisational memory. it should be relevant to the management styles of real people in the roles they play and the activities in which they engage. mss designers must anticipate changing requirements and be sensitive to the context in which it will be used. a framework in which this can be done is that provided by activity theory. the reasons for this choice will be elaborated below but are well articulated by blackler (1993) in his review of the relationship between knowledge organisation and the management, where he says "activity theory reframes management by modelling the recurrent and embedded nature of human activities, by revealing the tentative nature of knowledge and its action orientation and by highlighting the opportunities for individual and collective development that engagement and dilemma can provide". principles of activity theory activity theory has its roots in the work of the russian psychologist vygotsky during the first half of this century, and is based on the idea that human activity is mediated by cultural signs: words and tools, which cause changes in that activity, and thus its internal mental reflection (vygotsky 1978). vygotsky's work was continued by others including leontjev who developed a conceptual framework for the theory of activity. (leontjev 1981). the theory therefore has a well-established tradition and has been applied in education, linguistics, anthropology, cultural research and more recently hci (kaptelinin 1994), computer science, (bodker 1991) and is (kuutti 1990). an excellent compilation of work on activity theory and computer-based systems can be found in the book by nardi (1996). according to kuutti (1996), activity theory is "a philosophy and cross-disciplinary framework for studying different forms of human practices and development processes, with both individual and social levels interlinked". it requires a paradigm shift for most western researchers and it is not easily comprehended on a first encounter. the following is an overview of key principles that could be applied to mss. 46 ajis vol. 6 no. 2 may 1999 a comprehensive unit of analysis in activity theory the basic unit of analysis of all human endeavour is "activity", a broader more embracing concept than individual goal-oriented actions. the word "activity" has been chosen in english to be the best, but not perfect, equivalent of the russian word "deyatelnost" roughly meaning human "doing" (performing or operating both physically and mentally). an activity is the minimum meaningful context for individual human actions and unless the whole activity is the unit of analysis, the analysis is incomplete. this is not unlike the general systems theory model which assumes that life in society is organised around complex systems which, when studied, should be treated as whole entities (hasan 1998). an mss project usually requires an organisation-wide data model, cross-functional integration and cooperation from many sources and is often related to data-warehousing (hasan 1995). the "activity" of mss development must be studied as a whole and failure to take this broad view of the project can often be its downfall. political rivalry and conflicts over "data ownership" must be anticipated and an mss project team should realise that a system which informs management decisions can impact on the progress of the whole organisation. the use of tools central to activity theory is the concept that all human activity involves the use of tools and that this distinguishes human activity from the activities of animals. activity theory recognises that there are physical, material tools, such as computer and psychological tools, such as language and ideas. while physical tools produce changes in the material object, psychological tools influence the psyche and behaviour of subjects. modern computer systems act as both material and psychological tools to support human activity but this tool concept is not always recognised in hci or is. most of the initial work in hci, such as that by card et al (1983), was based on the cognitive science model of humans as information processors with sensory inputs and motor outputs not unlike the computer (aboulafia & gould 1995). the interaction between human and machine was comparable to the interface between two computers where the output of one was the input to the other. this perceived equality of person and computer is evidenced by the term artificial intelligence. some eminent hci researchers have challenged that image. norman (1991) uses the term "cognitive artefact" to describe a computer system and suggests that the real "interface" of interest is that between the artefact and the task. carroll (1991) likewise proposes a dynamic taskartefact cycle that serves the needs of the human user. in the field of is, task analysis of work process is more likely to look for those tasks which can be automated, than those that can be done by people with the computer as a support tool. an effective methodology for developing the latter type of systems is missing from most is training curricula. one approach that has merit is that suggested by ciborra and francesco (1989), where a minimal set of hardware and software is put in place and further development is done as needed, either by end-users themselves, or by a professional team. activity theory opens up the concept that, together with the physical mss, the information and knowledge it provides should also be viewed as tools. this is related to the concept of internalisation that is discussed below and very relevant to the way a manager might use an mss. subject v. object * outcome figure 1. the structure of an activity the structure of activity an activity is undertaken by a human actor motivated towards an object and mediated by a tool as shown in figure 1. activities are distinguished from each other according to their objects. the word object is used in the sense of the "object of the exercise" and is related to the motive driving the activity, hence the term object47 ajis vol. 6 no. 2 may 1999 oriented activity is often used. in other words, transforming the object into an outcome motivates the existence of an activity (kuutti 1996). in the context of mss use, it is proposed that the principal "activity" has the components shown in figure 2 where the manager uses the mss to make sense or understand the organisation and its environment in order to make more appropriate decisions. the object-orientedness of activity. activity theory recognises that there are two kinds of objects: real, physical (material) objects and ideal (mental) objects, present in the subject's mind. the motive of an activity is always objective whether real or ideal and activity theory considers social and cultural properties of the environment to be as objective as the physical and biological ones. the object "sense-making" in figure 2 is an example of a valid ideal object. there is no activity without an object, that is activity is always purposeful although sometimes even the subject is not fully aware of that purpose. activities can be polymotivated, an issue that must be addressed in mss use, where the motivation may include the desire to be better informed and make better decisions but may also for status, to impress one's competitors or a variety of other reasons. manager v. s decisions rgure 2. the activity of mss use the active nature of subject another dynamic addressed by activity theory is the active nature of the subject of an activity. a person brings with them a whole range of biases that will affect how they approach the activity: previous experience, cognitive skills, personality and culturally determined traits. these will continue to evolve as the person undertakes the activity. in the activity of management decision making the subjects are the mss users, usually high powered company executives. the principal tenet of hci has been "early and constant focus on the user" (preece 1994) and user involvement in product design has been a constant theme. communication between the triad of management, is staff and system users, has often been difficult (hasan & cheung 1994) and the hci techniques of pd and jad have proved to be effective in facilitating this communication. the users of a mss can be a heterogeneous mix with different management styles, personalities and cognitive skills. a review of the research literature in this area could produce a checklist of possible combinations of characteristics with guidelines as to the possible systems design that may be suitable for each. the term "executive profiling" is becoming popular in the organisational management community. the hierarchical structure of activities participating in an activity involves performing conscious actions, that have an immediate and definite goal, and operations according to the hierarchy in figure 3. activities are always related to motives, whereas actions are always related to goals. goals are always conscious, but the operations which make up actions are usually non-conscious and automatically performed. most importantly, although an activity can consist of many actions and operations in context, it is more than the sum of these. 48 ajis vol. 6 no. 2 may 1999 activity < > motive h u actions < > goals u u operations < *• tasks figure 3 the hierarchical structure of activity the hierarchical structure shown in figure 3 is a dynamic one. one activity usually is performed by several actions and operations. the same activity can be performed by different sets of actions. the same action can be used in different activities, as most subjects are often engaged in several concurrent activities. this activity theory hierarchy has two significant messages for is and hci. one is that focussing on the whole activity, and not the detailed actions, may facilitate dialogue between the different stakeholders. at this top level, the purpose of the system is at issue, and cross-disciplinary communication should be easier. secondly the interplay between the levels of actions and operations is important with the desire for more usable, transparent computer systems. the aim in interface design is to allow the user to concentrate on conscious actions that are a significant part of the activity and relegate the use of the machine to the realm of unconscious operations. context activity theory provides a model of humans in their social and organisational context. one of the main criticisms of the current cognitive science bias of hci research is that it does not provide an appropriate conceptual basis for computer use in its wider social and organisational context. activity theory demands that activities are always studied in situ, taking into account the political, economical, social and cultural realities. two examples of contextual concepts are collective activity and tool mediation. collective activity the basic activity structure applies to an individual subject. however as kaptelinin (1996) observes the cultural-historical tradition emphasises the social nature of human beings and it would be natural to expect that activity theory would support the study of group activities. the scheme of engestrom (kuutti 1996) as shown in figure 4 extends the basic structure of activity theory to include group activities. in this scheme the entity of community is added to those of subject and object in the structure of activity. whereas the subject-object relationship is mediated by the tool, the community-subject relationship is mediated by rules. the relationship between community and object is labelled "division of labour" but this term may need rethinking outside the scandinavian tradition. it is this model of activity that offers most promise for the study of mss which are used in organisational contexts and may include communication facilities and tools for data manipulation to support decision making processes. tool mediation the structure of activity is strongly cognisant of the power of the tool to mediate and change the activity it supports. the mediating property of the tool modifies existing activities and opens the possibilities for new ones. developers rarely consider how the tool they design will mediate activities, changing work practices as well as social and cultural norms. it is common that, only as users become familiar with the system, can they appreciate the possibilities it offers. this is why an evolutionary prototyping method of development is recommended so that the system can grow with the user. this however means an ongoing commitment of resources as the system may not be signed off as complete even after a long period of real use. 49 ajis vol. 6 no. 2 may 1999 subject ««. object s outcome rules division of labour figure 4 the contextual model of activity internalisation and externalisation a fundamental principle of at is the unity of consciousness and activity, where consciousness is considered to be the human mind as a whole and activity is considered as the interaction of the human with objective reality. this leads to the object oriented model of a subject's interaction with the environment. the concept of internalisation is described by vygotsky (1978) as the'underlying mechanism for the origin of mental processes. mental processes are derived from external actions through the course of internalisation. this is in apposition to conventional cognitive psychology which emphasises the mind as the source of action. it is through activity that we learn and internalise concepts that then become a psychological tool that is manipulation in the internal plane of action (epa). the ipa is a concept developed within activity theory that refers to the human ability to perform manipulation with an internal representation of external objects before starting actions with these objects in reality (kaptelinin 1996). ipa is more general than the cognitive concepts of working memory and mental models and is well suited to the analysis of the processes of dealing with the "messy", unstructured problems of senior managers. the information provided and the knowledge managed by an mss should support the user's ipa. the study: background and method the benefits of applying activity theory to mss from both an is and hci perspective, is demonstrated in the following longitudinal case study which analysed the attempts made in a project to provide relevant and useful information to support quality management initiatives in an australian university. following considerable restructuring of the tertiary education sector in the 1980s, in the early 1990s the australian government turned its attention to making the system work effectively and efficiently and it was felt that quality assurance could best be achieved through the use of performance indicators. this would require each university to vastly improve those information systems which provided teaching and research performance data. the university project began in 1993 as a quality assurance initiative and evolved into a data-warehouse development (1994-1997). though useful outcomes were produced, the project did not succeed in producing a useable mss. the case study presented in this paper investigated both system development (is) and usability (hci) issues of the mss project. the activity theory approach gave new insight into the processes and outcomes. the author conducted the case study by observation and periodic interviews with principal stakeholders, over the five year period of the project. data was collected in the form of transcripts of interviews, notes from observation and documents. the documents covered initiatives for quality assurance and performance indicators, planning and review, information systems policy and implementation as well as government and industry reports. as presented earlier in this paper, activity theory was considered the most appropriate method of qualitative data analysis suitable for this lengthy study, based on similar work by boedker and groebak (1996), kuutti (1996) and engestrom and escalante (1996). whereas these previous applications of activity theory as a research tool were highly descriptive in nature, a much more categorical approach to data analysis was used in this study using the following steps: 1. the activities of interest were chosen and identified by defining their objects (in the activity theory sense). 2. activity theory diagrams were then drawn to display this data. 3. the structure of activity was used to filter and organise the data into categories of subject, outcome, community and tool. 4. the resulting structure was analysed as one or more activities 50 ajis vol. 6 no. 2 may 1999 5. other activity theory concepts, such as the action/operation hierarchy, tool mediation and internalisation/extemalisation, could then be used in a focussed way to interpret the data and draw conclusions. s".r the organisation & its environment •j?l! ^\\ s figure 5. the relationship between the activities of mss use and development the study: analysis and results the 5 steps listed above will now be demonstrated in a detailed analysis of the university mss project. words used with their specific activity theory meaning will be written in italics. stepl. the first step here is critical and, using an approach similar to that of boedker and groebak (1996), it was recognised that the activities central to this study were firstly the activity of creating the mss, for which the mss is the outcome, and secondly the activity of being informed, which uses the mss as a tool. these activities correspond respectively to an is and an hci perspective. step 2. these 2 activities are interrelated, as shown in figure 5, using engestrom diagrams for both activities. the top activity (figure 4 inverted) is the use (hci) activity and the lower one is the development (is) activity, the object of which is the tool for the first activity. the terms use and development will be used for these two activities in the remainder of the paper. step 3 the quality assurance initiative and data warehousing development are now described using activity theory language and structure. the first stage of the project included in this study was begun in early 1993 when a group, coordinated by the head of the education policy research program, was awarded a grant for the investigation of institution-based quality assurance, in line with government policy. the project group included representation from top university management, and had expertise in tqm as well as education and research performance evaluation. these were all potential subjects of the use activity. there was no permanent member of the group with is development expertise and only occasional representation from the university's administration information systems (ais) department, who maintained the student database and other operational systems. several group members, mainly those who had experience in the design of educational software, took on the role of subjects of the development activity. their knowledge of is development, in particular data analysis and design, was very limited. the outcomes of this phase of the project were: (1) a set of research papers, (2) an information needs audit for the broad spectrum of university staff and (3) a "user-friendly information system prototype to be accessible on line by staff of all ranks, in academic as well as administrative units, to support informed decision-making" (fasano 1994). unfortunately this prototype lacked credibility, as the technology, and is development expertise, used for the prototype was restricted to the authoring systems that were used in the education department for computer-aided learning applications. the information needs audit was a lengthy list of information requirements gathered from representatives of various groups on campus but fell short of identifying a workable set of performance indicators. there was no evidence that this list was comprehensive and there was no allowance for the way in which the needs would change over time. however it was a useful document. 51 ajis vol. 6 no. 2 may 1999 in 1994 there was a decision, driven by ais, to create a data-warehouse in an effort to meet the rapid growing need for easily accessible information on students as identified in the information needs audit. this data warehouse has been successfully implemented by ais staff using an oracle database. the main reason for this choice was to be compatible with the anticipated release of a new student enrolment system by a consortium of australian universities. however development of this system is still a long way behind schedule and the data-warehouse continues to be used with regular daily updates from the legacy operational student database. this development activity, performed by subjects with is skills, has produced an outcome that is technically sound but not very useable. control of end-user access to the data warehouse was not left with ais but given to the planning and marketing department (pmd) on the assumption that they would know what the users want. employing contract programmers pmd have made three attempts to create a user-interface to the data warehouse: once using downloads into excel spreadsheets, once using a web browser interface and a third enduser system using a suite of oracle queries and reports. none of these systems have passed the prototype stage and pmd continue to use oracle programmers to generate ad hoc reports from the data warehouse, in response to specific requests from management, and for the periodic government reporting. the use activity in this case has virtually failed. the most useful outcome of the whole project has been the realisation of the complexity of the information needs of university management at all levels. the quality assurance project's conclusion highlighted the need by all decision-makers in the university to access information "well beyond what is provided by pre-established lists of indicators" and that there is a "need of staff to access information processing capabilities, as well as just information, allowing individual and specialised reporting" (fasano 1994). step 4. the language of activity theory is useful when describing this situation. it is clear that the objects of the two activities, development and use, are quite distinct. the object of one is to produce an artefact, the mss, while the object of the other is to get a sense of how the organisation is preforming using that artefact. it is well known that the subjects of these two activities are often worlds apart and do not communicate well. the mss developers are familiar with technical tools whereas managers often have little in the way of computing skills. it was apparent in the two projects of the study, that the activities are often confused, with project teams contain both sets of subjects, and one or other group tends to dominate. at the quality assurance stage of the project, the activity of mss use was dominant and the outcome was a somewhat useful "information needs audit" but a weak system. the data warehouse development phase had a successful outcome in the data warehouse but still no useable interface that could provide an mss for managers. using figure 5 as a model it is clear that subjects of the mss development activity and the subjects of the mss use activity have a different perception of the aim of an mss project. the developers aim to produce a robust, working system, while the managers want an easy-to-use tool that will provide a clear picture of the organisation's performance. senior managers usually find it difficult to specify what sort of tool would be useful for their activity in a language that is meaningful a technical person. both sets of subjects however are members of the same community which strongly depends on the outcomes of both activities. identifying the activities and their components is only the first phase in an activity theory analysis. all the other concepts of the theory, as described in section 3 above can be brought to the analysis of the problem. three of these are now briefly discussed. the hierarchical nature of activities. this study is typical of most mss projects run by teams composed of members, from different backgrounds, to whom different aspects of the project are important. the hierarchical structure of activity theory provides a means of dealing with this problem by insisting that the "unit of analysis" or "focus of study" should be a single identified purpose or object of the whole activity. an mss project comprises development and use activities whose objects should be made clear to all. once the main purpose is established and agreed by the team, then all points of disagreement should be on lower implementation issues at the action or operation level (see figure 3). activity theory stipulates that an activity may be undertaken by different sets of actions and operations which may vary depending on the state of the subject, the tool and the community at the time. less importance can then be placed on deciding which is the best set of actions to achieve the object of the activity. team members, aware that there is no necessity to agree on one correct set of actions to carry out the activity, should find it easier to communicate and cooperate. 52 ajis vol. 6 no. 2 may 1999 the dynamics of activities. the concept of tool mediation anticipates that changes will take place in an activity through the use of the tool. participants in an mss project usually focus on what sort of system would be useful for an activity as it is now. activity theory states that the question should be: "what sort of system would adapt with the users as they became more competent and reliant on the tool". the internalisation of information the process by which managers use information in the decision-making process is not well understood. the activity theory concept of ipa, introduced in section 3.6 of this paper, deals with the way in which information is internalised by subjects as a mapping from the external world. this concept could be used to guide the design of information structure and presentation in an mss. further research in this area is strongly recommended. conclusion mss are organisational systems that are normally the province of is. however as systems which "informate" rather than automate, they differ from traditional organisational systems, in that they have uncertain specifications and a nonhomogeneous user population. the field of hci specialises in problems of users and the field of is could, in general, benefit from some of its teachings. however both is and hci lack a sufficiently contextual and flexible methodology to adequately support the development and use of information systems to support senior managers. the principles of activity theory have much offer both is and hci and could be an appropriate philosophical and theoretical foundation for the study of systems such as mss in the following ways: • the holistic nature of activity provides an understandable focal point for cross-disciplinary discussion without considering the lower level detail where language and emphasis may differ. • the model of activity, comprising subject, tool, object and community provides a useful framework for identifying stakeholders in an mss, the activities in which they are engaged and the relationships between them. • the activity theory hierarchy enables subjects to consider alternative sets of actions and operation to carry out the activity in a way which suits the specific goals and conditions of the situation. • the principle of tool mediation is much more useful to the study of mss than the traditional is concept of the computer as a machine which automates work processes. • the principle of internalisation is an apt description of the way managers use a support tool in the decision making process. nardi (1997), an anthropologist, believes that activity theory provides a common vocabulary to facilitate crossdisciplinary discussion and activity theory has already proved to be applicable for descriptive and explanatory studies of hci and is problems (for example the studies of bodker 1991 and engestrom & escalante 1996). attempts are now being directed at a more prescriptive application of activity theory to problems of hci using a design checklist (nardi & kaptelinin 1997). the study presented in this paper suggested that another prescriptive use of activity theory could be to follow the 5 steps described here and to use the diagram shown in figure 5 which relates system development to system use. this diagram, together with the dynamic and contextual principles of activity theory, could be a vehicle for describing the situation, identifying important issues and warning of common pitfalls. references aboulafia, a, gould, e., (1995) information processing vs human cognition: a critical review of cognitive approaches in it and hci, international society for the systems sciences 39th annual meeting, amsterdam,, 293-304. alter s. (1996) information systems: a management perspective, 2nd edition, benjamin/cumrrungs. bladder f. (1993) knowledge and the theory of organisations: organisations as activity systems and the refraining of management. journal of management studies 30/6 863-884. boedker, s., (1991) through the interface a human activity approach to user interface design, lawrence erlbaum. boedker s. and groenbak k. (1996) users and designers in mutual activity: an analysis of cooperative activity in systems design, in y.engestrom and d.middleton (eds) cognition and communication at work cambridge university press. 53 ajis vol. 6 no. 2 may 1999 card s, moran t. and newell (1983) the psychology of human-computer interaction. lawrence erlbaum, hillsdale. carroll jm. 1991 introduction: the kittle house manifesto in j.m carroll ed. designing interaction: psychology at the human-computer interface. cambridge, 1-16. ciborra c. and francesco l. (1989) designing networks in action: formative contexts and post-modern systems development proceeding of sost'89 ifip tc9 conference, terrigal. davies l and ledington p (1991) information in action: soft system's methodology macmillan. engestrom y. and escalante v (1996) mundane tool or object of affection? the rise and fall of the postal buddy, in b. nardi ed context and consciousness: activity theory and human computer interaction. mit press 325-374. fasano c (1994) planning and management information systems for quality enhancement and assurance and effective resource utilisation in the higher education system. debt fidler c and rogerson s (1996) strategic management support systems, pitman, london. fiolc. (1992) maps for mangers: where do we go from here? journal of management studies 29/3 267285. hasan h. (1995) organisational issues and implications of eis, proceedings of acis95 perth 207-218. hasan h. (1998) activity theory: a basis for the contextual study of information systems in organisations. in h. hasan, e. gould & p. hyland eds information systems and activity theory: tools in context, university of wollongong press. hasan h. & cheung l. (1993) executives, users & is in the 1990's proceedings of the 4th acis conference. brisbane hasan, h. and gould, e (1996) a framework for eis development in the public sector to appear in the proceedings of erma, washington. hasan h, jones c and e gould (1996) prototyping tools for expert and novice application development proceedings of the empirical studies of programmers conference, washington hasan, h. and lampitsi, s. (1995) executive access to information in australian public organisations, journal of strategic information systems 4/2.213-223 is world 1997 international electronic discussion group on is kaptelinin, v. (1994) activity theory: implications for human computer interaction. in: brouwer-janse, m.d., harrington, t. (eds.) human-machine communication for educational systems design, nato asi series. series f: computer and systems sciences, v.129, berlin : springer-verlag, 5-16. kaptelinin v (1996) activity theory: implications for human-computer interaction in b. nardi ed context and consciousness: activity theory and human computer interaction. mit press 103-116. kuutti, k. (1990) activity theory and its application to information systems research and development, in: isra-90 proceedings vol. i, ifip tc 8 wg 8.2. working conference on the information systems research arena of the 90's, copenhagen. kuutti k (1996) activity theory as a potential framework for human-computer interaction in b. nardi ed context and consciousness: activity theory and human computer interaction. mit press, 17-44. kuutti k and virkkunen j (1995) organisational memory and learning network organisations: the case of finnish labour protection inspectors proceedings of the 28th annual hawaii international conference on systems science 313-322. leontjev, a.n. problems of the development of mind moscow: progress 1981 mantovani g, (1996) social context in hci: a new framework for mental models, cooperation and communication cognitive science 20 237-269. mintzberg h (1975) the manager's job: folklore and fact, harvard business review, july-august 1995, 49-61. mintzberg h raisinghani d and theoret a. (1976) the structure of "unstructured" decision processes, administrative science quarterly 21, 246-275. nardi b (1996) (editor) context and consciousness: activity theory and human computer interaction. mit press nardi b (1996) studying context: a comparison of activity theory, situated action models and distributed cognition in b. nardi ed context and consciousness: activity theory and human computer interaction mit press 69-102. nardi b (1998) where to from here? in hasan, gould & hyland (eds) information systems and activity theory; tools in context, university of wollongong press nardi b and kaptelinin v (1997) activity theory: basic concepts and application, tutorial at chi97, atlanta. 54 ajis vol. 6 no. 2 may 1999 neisser u. (1976) cognition and reality, principles and implications of cognitive psychology, w.h. freeman. norman d (1991) cognitive artifacts in j.m carroll ed. designing interaction: psychology at the humancomputer interface. cambridge, 17-38. preece j (1994) human-computer interaction addison-wesley. randall r, o'brien j, rouncefield m and hughes j. (1996) organisational memory and cscw: supporting the 'mavis phenomenon' proceeding of ozchi96, hamilton nz 26-33. rockart, j and delong, d.,(1988) executive support systems, dow jones-irwin. vangigchj. (1978) applied general systems theory harper & row new york vygotsky, l.s. (1978) mind and society, cambridge ma: harvard university press wagner c. (1995) decision support for "messy" problems information and management 28 393-403. zuboff s (1988) in the age of the smart machine oxford 55 ajis special edition knowledge management december 2001 knowledge management and organisational effectiveness: balancing the mild, the wild, and the crazy greg marsh defence science & technology organisation po box 1500 edinburgh, south australia, 5111 phone: 08 8259 5362 fax: 0882595619 email: greg.marsh@dsio.defence.gov.au martin burke defence science & technology organisation and systems engineering and evaluation centre, university of south australia po box 1500 edinburgh, south australia, 5 1 1 1 phone: 08 8259 7004 fax: 0882595619 email: martin.burke@dslo.defence.gov.au abstract in the knowledge age, effective knowledge management (km) is seen as fundamental to the survival of many organisations. information systems (is) increasingly underpin km in such organisations. the value of the contribution of is to the goals of these organisations depends on balancing various conflicting requirements. for example, stable, robust and reliable business processes are often fundamental to their efficient operation. however, less structured environments are conducive to the innovation and knowledge creation that are considered to be fundamental to their on-going effective operation. the paper presents a model developed to relate these conflicting requirements to organisational effectiveness and describes how it was used to examine the outcomes of a knowledge management study conducted in the australian defence organisation. introduction it has been observed (eg the price waterhouse llp change integration team 1996; shaw 1997; turner and crawford 1998; hidding 2001), that change is the imperative of the knowledge age. in this epoch, both public and private organisations operate in environments of constant, and often unpredictable, change. this paper contends that, as the dynamics of environmental change have become more chaotic, traditional methods for its analysis and prediction have become less effective. accordingly, conventional approaches of organisational response to change have lost their relevance. it argues that, in order to survive and thrive in such dynamic and uncertain circumstances, organisations need to develop new ways of balancing stability and dynamism in their functions, processes, and operations. the paper presents a new model that addresses organisations interacting with external environments subject to dynamic change. the model applies insights from the fields of complexity science, systems thinking and knowledge management to treat organisations as systems comprising inter-related heterogenous components described in terms of attributes and relationships. the paper describes the use of the model in attempting to resolve a paradox that emerged in a recent investigation of knowledge management in the australian defence organisation. paradigm shift in understanding some management writers (freedman 1992; stacey 1996) see developments in the science of complexity as providing a new paradigm for understanding organisations. the concept of the "paradigm" in scientific research was used by thomas kuhn (1970) to describe a set of unquestioned assumptions held by those who, at a particular point in time, profess a particular science2. revolutionary changes in the dominant scientific paradigm profoundly influence how society views the world. "the historian of science may be tempted to exclaim that when paradigms change, the world itself changes with them"(kuhn 1970). it can be said that our view of the business world has changed as we move from the industrial age to the knowledge age. the success of newtonian mechanics contributed to a socially constructed phenomenon in which science was seen as "exact" and the presumption that once all fundamental laws had been discovered, the future behaviour of systems could be predicted with certainty. in the 19th century, many scientists (notably laplace) were confident that even the 2 usage of the term "paradigm" has changed over time to include the prevailing modes of thinking as well as the set of unquestioned assumptions. 67 ajis special edition knowledge management december 2001 most complex behaviours could be reduced to simple laws, which could be used to predict future behaviour exactly. this conviction spilled over into other areas of thinking. for example, f.w. taylor's (1923) analysis of organisations and his prescriptions for "scientific management" held the implicit assumption that once the fundamentals of any task were discovered, the one best way of performing it could be calculated (pugh and hickson 1996). "but during the past few decades, more and more scientists have concluded that this and many other of science's traditional assumptions about the way nature works are fundamentally wrong. far from being as predictable as clockwork, nature appears as random as a throw of the dice"(freedman 1992, p 30). physics led the way as quantum mechanics and relativity theories showed that the newtonian paradigm was limited and, in the limit, that physical parameters could not be measured exactly. more recently, the mathematics of chaos theory has shown that even when the causal relationship is known, some systems exhibit dramatic and unpredictable outcomes in response to very small changes in initial conditions. it has been observed that complex systems, like the weather for example, are sensitive to initial conditions and can develop dramatically different behaviour from very small changes to initial conditions. this is known as the butterfly effect the notion that a butterfly stirring the air today in peking can transform the storm systems next month in new york (gleik 1988). contemporary organisations have to operate in a rapidly changing environment, dealing with new and unexpected forms of competition, and working in new forms of partnerships and alliances. corporations no longer only compete with traditional, similar organisations, but are subject to competition from unexpected quarters. for example ibm, accustomed to competing in a relatively stable mainframe market, had to change rapidly its strategy and processes to cope with the totally different competitive environment arising from the advent of the pc and microsoft's business strategies. microsoft in turn has had to adapt its strategy very rapidly, to counter a new form of competition from netscape (rebello 1996). it would seem that changes of thinking are occurring in other fields of study, including organisational and management science, that are analogous to the complexity/uncertainty paradigm shift in the physical sciences. the recognition of such limitations of organisation and management theory is leading to a new paradigm that acknowledges the reality of operating with complexity/uncertainty. strategic planning for example, was developed hi the 1960s with the implied assumption that the future was predictable, and only a few years ago mintzberg reported that "strategic planning, as it is still practised, is still heavily slanted toward the 'predict-and-prepare' mode of coping with the future" (mintzberg 1994, p228). in the new paradigm of uncertainty, organisations are positioning themselves to meet likely contingencies and developing the flexibility to respond quickly to the unexpected threat or opportunity. the ability to manage the unpredictable is the objective of this repositioning. "in the new thinking, success is defined not as ending up where you want to be, but as wanting to be where you end up"(sauer and yetton 1997). figure 1 shows how the complexity/uncertainty analogy can be used to draw some insights about the interactions within an organisation and with its environment. conventional relationships are in this "mild" region close to agreement and close to certainty. in this region we can be confident of causal relationship and can have high confidence of predicting future outcomes from the measured inputs. moving further out into the "wild" region, there is more complexity. here there is some control, but the unexpected is likely. this is the area of creativity, new ideas, insights and innovation. as we move further out the ride becomes wilder as we move towards the edge of chaos. here creativity and innovation is at its greatest, but levels of control and efficiency are relatively low. in the "crazy" region of highest uncertainty and greatest disagreement, chaos reigns. here we expect confusion and disarray. the good ideas from here have to be pulled back towards the controlled region to gain the benefits. 68 aj1s special edition knowledge management december 2001 far from agreement close to agreement crazy intuition insights heuristics anarchy, disintegration, s confusion 'chaos creativity s,_ ^ innovationx o tf planning control wisions missions close to certainty far from certainty based on: ralph stacey 1997 figure 1. mild, wild, and crazy knowledge in organisations as society moves from the industrial age to the knowledge age, the predominant competitive mode is moving from efficiency in process to effectiveness in innovation and flexibility. success of organisations depends not just on efficient management of physical and information resources but increasingly on the effective management of knowledge. this raises issues of the nature of the knowledge within organisations; how new knowledge is created and embodied in the innovations, and how existing knowledge might to be brought together either physically or virtually. in practice, the terms "data", "information" and "knowledge" are often confused (boisot 1998). data and information can be transmitted directly, but knowledge is "...anchored on the commitment and beliefs of its holder" nonaka (1994, p 15). it is the interaction between the information and the existing knowledge of the agent that is defining in this case. data becomes information when the agent receiving it can ascribe meaning to the data; knowledge is generated through the interaction of existing knowledge with the flow of information to the agent. in contrast to information, knowledge cannot be directly observed. its existence can only be inferred from the action of agents (boisot 1998 p!2). for the purposes of this discussion, we will use the conception of tacit and explicit knowledge described by nonaka (1994) in his seminal paper on knowledge creation34. explicit knowledge is codified knowledge or information derived from knowledge. tacit knowledge is personal knowledge; embodying elements like mental models, experience, skills, know-how and refers to individuals images of reality and visions for the future,(nonaka 1994). in the view of boisot (1998, p57) tacit knowledge comes in 3 distinct variants: 3 although this conception has gained some popularity in the literature, it should be emphasised that it conflicts with distinctions between tacit, explicit and implicit knowledge drawn by polanyi (1962; 1966) and subsequently adopted by sveiby (1998), baumard(1998), etc. we consider that there are some important consequences of these differences in conception and intend to address them in detail in a forthcoming paper. 4 burke(2000) provides a conceptually coherent discussion of knowledge in the context of competition and collaboration. 69 ajis special edition knowledge management december 2001 • things that are not said because everybody understands them and takes them for granted • things that are not said because nobody fully understands them. they remain elusive and inarticulate, see (polanyi 1962)) • things that are not said because while some people understand them, they cannot costlessly articulate them, see nonaka and takeuchi (1995). knowledge creation nonaka (1994, p 14) postulated that a stream of innovation indicates that an organisation should be studied with regard to how it creates information and knowledge, rather than how it processes problems. this insight suggests that we move away from the mechanistic model of organisations to seek a more dynamic, organic view. using the proposition that new knowledge is created through conversion between tacit and explicit knowledge nonaka and takeuchi (1995), have proposed that knowledge creation is a "spiralling process of interactions between explicit and tacit knowledge". their seci model, shown, in figure 2, suggests that there are four (dominant) knowledge conversion processes: • socialisation; • £xternalisation; • combination; • /nternalisation tacit knowledge from explicit knowledge modes of knowledge creation externalization combination explicit knowledge socialization internalization to tacit knowledge figure 2. modes of knowledge creation (nonaka and takeuchi 1995) from studies of new product development in japan, nonaka (1994) found that the knowledge creation process needs a mix of creative chaos, redundancy of ideas, and requisite variety (towards the "crazy")these conditions are quite different to those found in the efficient machine bureaucracy type of organisation (handy 1986, p 192) that characterises industrial age production organisations and the traditional military structure (striving towards the "mild"). in the knowledge age, innovation is driving change at an increasing rate, which in turn is shaping changes in the organisational form. miles et al (1997, p 7) describe how the organisational form has evolved as dominant competitive forces have changed. "the locus of organisational exemplars has shifted from capital intensive industries ... to information intensive industries, ... and now towards innovation-driven industries, where competitive advantage lies mostly in effective use of human resources". information systems the interplay between tacit and explicit knowledge provides a frame from which to look at the role of information technology in knowledge creation and knowledge management. examination of figure 2, shows that computers 70 aj1s special edition knowledge management december 2001 inherently operate in the lower left hand quadrant, where explicit codified knowledge is combined and manipulated into different and potentially more meaningful forms of explicit knowledge. the early stages of the knowledge age saw is used to automate these processes, but as information and communications technology became more pervasive, new applications started to move into knowledge creation processes in the other quadrants. web technology, for example, supports the learning process, the internalisation quadrant, by providing a mechanism for finding explicit knowledge that can be absorbed to add to an individual's tacit knowledge. video-conferencing, groupware and meeting support software moves it into the externalisation quadrant, and to some extent into the socialisation quadrant. these systems assist in making explicit some of the tacit knowledge held by those interacting with and through these systems. a systems view a systems view of the organisation is used to frame this study of knowledge management to establish a conceptually coherent basis for examining the issue. in this approach the organisation is viewed as a system operating within a wider system. the common conception of "system" within the disciplines of engineering and computer science is a physical, human-made, dynamic, combination of technologies, which acts in a coordinated way. examples of such systems can range from desktop computers to submarines. however, natural and social scientists use the term to describe vastly different phenomena. indeed, it was within the natural sciences that the term "system" was first popularised (von bertalanffy 1968). thus the term "system" is now used to describe things as diverse as ecological systems, human-activity systems, systems of ideas, and technical systems. we contend that systems thinking is hierarchical and self-similar in nature. for example, the entire universe may be considered as one large system made from component galaxies, whilst the galaxies themselves can be viewed as systems of solar-systems, which in themselves are systems and so on and so forth. this reduction of abstraction can continue until we reach the level of the atom, which was once thought to be the smallest building block of the universe, but is now widely understood to be a system (of sub-atomic particles) as well. such a structure of systems within systems can be thought of as a systems hierarchys. each system within the hierarchy is a "whole" with respect to its component elements, in addition to being a component element with respect to a system at a higher level in the hierarchy. the art of thinking systemically about a part of the world is in the process of bounding both the object of study from its environment and the levels of abstraction (upper and lower) within the systems hierarchy. this process is described in detail in matthews et al. (matthews, burke et al. 2000). for our purposes the process is important in that it brings into being the system of interest. the validity of systems thinking and analysis often depends on the appropriate choice of abstraction and an understanding of the mutual influences between the entity under study and the environment. matthews et al. (matthews, burke et al. 2000) define five essential characteristics of a system (or systems thinking), these are component, component attributes, relationships, boundary and emergent properties (or system attributes). these characteristics are the recursive elements within the systems hierarchy, as shown in figure 3. a key issue in this conception of systems is that of emergence. an emergent property of a system is a property that is meaningful when attributed to the whole system, but not to its component elements (checkland 1981). thus it can be viewed as an attribute of the whole system that does not reside in any of the component elements6. 5 a systems hierarchy is a self-similar structure; the pattern that recurs are is the five characteristics (described below) which make up the system concept. 6 according to capra (1996), the term "emergent properties" was coined by the philosopher c. d. broad (1923), to refer to those system properties that emerge at a certain level (of hierarchy) but do not exist at lower levels. with this definition of emergence, a systems hierarchy can then be defined as an architecture view of a system from a structural perspective made on the basis of the existence of emergent properties. (burke 2000). 71 ajis spe environment owledge management december 2001 mv1ronment >-f^ environment tatt. -k x v catt. figure 3: a repesentation of the systems hierarchy organisations as systems organisations can be thought of as socio-technical systems composed of a complex interaction of humans and their tools. kline defines socio-technical systems as "systems that link people with human-made hardware to perform tasks that humans want done" (kline 1995, p. 171). thus organisations are defined by the group of people in the organisation, the equipment they use and produce, as well as the procedures that govern the interaction of these components. another feature of the systems hierarchy is the recursive nature of the representation. thus the emergent attributes of a system are the component attributes of the next highest system. similarly, the relationships between the components within a system become the relationships with the environment when the component is the system of interest. the emergent attributes of the system under study is a property that emerges from the attributes of its components, the relationships among the components and the relationships between the systems and other systems in its environment. these characteristics are evident in organisations, which inherently have components with attributes and relationships, and are seen to have boundaries and organisational properties (emergent attributes). an organisation's emergent system attributes include tangible products and services as well as less tangible attributes of organisational effectiveness, performance or strength. components are organisational sub-groups in the organisation and the tools and procedures that are used in the organisation. information systems, for example, are an important socio-technical component of modem organisations, and their attributes and relationships with other parts of the organisation contribute to the emergent properties of the whole organisation. for many organisations in the knowledge age, information systems also mediate the external relationship that organisation has with other organisations in its environment. it is emphasised that this simple description of an organisation is a representation of the organisation as a systems hierarchy from a limited perspective, namely that of organisational sub-groups, tools and procedures. from a 72 ajis special edition knowledge management december 2001 broader perspective, organisations are made up of interlocking systems of components and relationships within an organisation. by selecting another viewpoint, different systems hierarchies can be seen within any organisation. nonaka and takeuchi, for example, describe this as the "hypertext organisation" (1995, p 169). figure 4 shows one way of viewing components of an organisation with different relationships depending on which systems hierarchy view is taken. this leads to a richer, more complex view of the organisation as a system, compared to the "engineering" model of a system with components having well defined fixed relationships with each other, and producing predictable outputs. thus a richer set of views provides a perspective that can generate insights into those components of an organisation operating in the "mild" the "wild" or the "crazy" regions. figure 4multiple system hierarchies in an organisation • proiectcollaboration among _ project teams to promote knowledge creation teams are loosely-coupled around organisational vision team members form a hyper network across business-systems dynamic knowledge cycle continuously creates, exploits and accumulates organisational knowledge high accessibility to knowledge-base by individual members organisational vision, culture, databases, etc. similarly, (recursively) an organisation interacts in a complex way with its environment (the other components of the next highest system). this can be viewed from different systems hierarchy perspectives to gain further insights into the interactions. for simplicity, we will restrict discussion to a single view, using this as the archetype for other views. acknowledging the potential for other views enables a more dynamic view of the organisation than is possible from a single view, particularly one drawn from a machine analogy viewpoint. in recent times, the business and is strategy literature has evolved to take account of the more dynamic environment (stacey 1999; kidding 2001). models have evolved from structural forms (porter 1980) to resource-based models (prahalad and hamel 1990; barney 1991); moving from a reliance on predictions of the future environment, to developing capabilities that can be flexibly adapted to meet a range of contingencies. effectiveness effectiveness of the organisation can be viewed as the "fit" of its emergent attributes with those of its environment. how well system inputs are transformed into outputs to suit the environment determines the value of the organisation within the higher order system. miles and snow (1994) describe this as the process of "external fit", while internal organisational effectiveness is linked with how well the components of the organisation achieve "internal fit", "company success depends on putting together a complete and complementary package of ingredients: strategy, structure, process, and a managerial ideology that holds these together and gives them meaning. further, this pack73 ajis special edition knowledge management december 2001 age of characteristics must be widely understood and disseminated so that it becomes part of the daily behaviour of everyone in the organization. we call this process 'achieving fit'" (miles and snow 1994, p 7) this description of what it takes to be a successful organisation emphasizes knowledge intensive issues of meaning, understanding and behaviour and "fits" with kline's succinct definition of socio-technical systems as "systems that link people with human-made hardware to perform tasks that humans want done" (kline 1995, p 171). an effective organisation then is a system that has a complementary set of components linked in such a way that the emergent properties of the system achieve desired goals. balancing near the edge to achieve internal "fit" the organisation has to achieve the complementary balance of components. both the attributes of the component subsystems and their relationships can range from stable and predictable to volatile and unpredictable. in terms of the characteristics described in figure 1, they include the mild, the wild, and the crazy. the mild state represents stability, order and predictability, where well engineered components will have predicable behaviour. however, socio-technical systems can never achieve such certainty, because outcomes are not determined by fixed relationships, but emerge from a complex mix of rules, ethics, emotions and human drives interacting with each other and with human constructed tools. nevertheless, systems can be designed and people trained and motivated to operate with a level of predictability and consistency that can tend towards the "edge of order". other socio-technical parts of the organisation are at the other end of the spectrum and (by design or accident) are out of control and unpredictable. these components of the organisation react faster and less predictably and are likely to drive the emergence of new and useful organisational attributes. at the same time, operating at the "edge of chaos" puts the organisation at risk of going over the edge into the "crazy" region of confusion, anarchy and disintegration. thus to be effective, an organisation needs components with attributes and relationships balanced in the "wild" band between stifling stability and potentially destructive chaos. some organisations, for example, might at times operate in a stable environment while at other times have to cope with fluctuating market conditions. like its components, the organisation itself operates in the "wild" striving to achieve an effective balance between the edge of order and the edge of chaos in its relationship with its environment. figure 5 shows a view of the interplay of the internal and external attributes and relationships, as they range between the edge of order and the edge of chaos. the mild and crazy states represent the limits of where we would expect to find components in an organisation or organisations in an environment. figure 5 organisational relationship internal/external mix crazy edge of chaos stability of: external wiid attributes/relationships edge of order mild reconfigurable modular stable efficient automation single purpose new situations, innovation, individually tailored continuous improvement, best practice bpr mild edge of order wild edge of chaos crazy stability of: internal attributes/relationships 74 ajis special edition knowledge management december 2001 this modelv is of the organisation, but can be considered a general view of a typical layer of the systems hierarchy in which the sub-systems within, and the environment of, the organisation form a recurring self-similar structure. at the organisational level this model provides a tool to examine how its emergent properties in its environment relate to the relationships and attributes of its components. for example, to be effective in quadrant 2, the organisation, finding itself in an unstable rapidly changing environment, responds by moving itself towards the "crazy" to achieve the necessary innovation, adaptability and speed of response. however, such an organisation still needs components from quadrant 4 to provide stable infrastructure systems for the other components of the organisation. lack of stable infrastructure can constrain effectiveness as much as an overly rigid and bureaucratic infrastructure. thus, a quadrant 2 organisation will need components from all four quadrants, but in a mix that enhances organisational competitiveness. as discussed above, knowledge plays a key role in the attributes and relationships within the organisation. in terms of the relationships described in figure 5, the quadrant in which the organisation finds itself will be closely related to the predominant knowledge creation modes employed in the organisation. knowledge management study of the australian defence organisation background for 40 years following the end of world war ii, the cold war balance of the two world superpowers dominated military planning. in terms of our model, this external environment was near the "edge of order": it was predicable and stable. accordingly, military forces could work to hone their internal processes to meet this threat efficiently. inventions and technical developments, such as the developments in it during this period, were incorporated to improve the efficiency of the huge cold war "machines". during this time, the military world was solidly in quadrant 4 with inputs from quadrants 3 and 2. the collapse of the soviet union brought complex changes in the military environment: it swung from the mild to the crazy. traditional approaches to military planning, organisation and practice were found wanting in this rapidly changing environment. new and unpredicted conflicts erupted, creating unexpected alliances and unstable coalitions. military forces, structured for large-scale conflict against similar forces, had to be quickly adapted to handle asymmetric conflict. the attributes and relationships optimised over the years to be effective in the stable, balanced force world were now inappropriate to cope effectively with the changing attributes and relationships of military conflict. the military world was now in quadrant 2, with military planners having to cobble together quickly one-off solutions for unexpected situations. the australian defence organisation (ado), like its counterparts across the world, has been working hard to understand and adapt to the unsettling changes in the military landscape since the end of the cold war. also, like all organisations, the ado continues to evolve its practices and processes to make best use of developments in information systems. one response of the ado to these pressures was to appoint, in mid 2000, its first chief knowledge officer who commissioned a team from the defence science & technology organisation (dsto) to investigate knowledge management in the defence headquarters. aspects of the findings from this study motivated much of the thinking behind this paper. method during the study the team interviewed 60 staff comprising a broad cross section from the ado headquarters located in russell, in the australian capital territory. interviewees were selected for their interest in km and came from the three single services and the civilian staff. the rank of those interviewed ranged from chief of the defence force and secretary of the department to trainee graduate administrative assistants, but centred around the "one star" (brigadier and equivalent) level. initially, the team planned to conduct structured interviews, but moved to a more open format in order to capture the rich range of experience, opinions and ideas from the staff interviewed. although this adaptation was expedient in achieving cko's immediate goals in commissioning the study, it has consequences for the scientific rigour of the approach and consequently its findings need to be interpreted accordingly. to make sense of the wealth of information contained in the interview notes, the team aggregated the interviewee comments into various themes and groupings. general themes emerged around issues of culture, organisation, learning and technology. the comments in these groups were further divided into comments on km enablers, inhibitors, initiatives and recommendations. this analysis provided the cko with a structured snapshot of perceptions 7 this model also builds on pine's (1993) "mass customization" model of product/process change in manufacturing. 75 ajis special edition knowledge management december 2001 of km within the canberra headquarters. it also gave the study team a valuable perspective of km issues from a key part of the australian defence organisation that, amongst other things, provoked the thinking leading to the development of the model presented above. selected observations many of those with whom we spoke had formed clear ideas on how to best to improve knowledge management, but no clear consensus emerged. in fact the perceptions and recommendations, on both cultural and technology issues, seemed to be in conflict. it is emphasised that the observations reproduced below have been selected by the authors to illustrate the contrasting ideas. they are (often strongly worded) opinions of individuals on particular issues of interest to them. it is important that they should not be interpreted as representing: the formal position of the ado; a complete and coherent description of km in ado; or the consensus view of those interviewed. those interviewed believed that the culture of the organisation was central to improving knowledge management in the ado, but voiced conflicting requirements. some, who perceived a requirement for the organisation to be able to survive in an environment of rapid change, identified the need for more flexibility and even chaos, to promote a culture in which innovation and flexibility could flourish. others, who understood the need for strong risk management in a business with as much at stake as defence of the country, identified the need for a more structured, highreliability culture. some, who advocated a more open culture, saw promise in the attitudes of those entering the service: 'youngpeople are more open in their interactions with senior officers', but others believed that the military culture could potentially constrain the innovation and flexibility needed to be effective in a changing environment a civilian offered the opinion that defence is: 'not open, not sharing'. on the other hand, others saw more structure and control as the key to improved km and organisational effectiveness. particular concerns were held in regard to records control and management that were seen as necessary to restore efficiency, reduce duplication and retain corporate knowledge. one officer commented on the need 'to facilitate cultural change to get people to understand that reporting is part of business'. 'compliance with the archives act is not optional'. the responses in relation to the information systems showed the same paradox. some advocating relaxing some controls on is, while others saw tighter controls as the way to improve organisational effectiveness. ideas for opening up the is constraints included improving linkages with outside agencies and more freedom to introduce or experiment with new tools and databases. as it was put succinctly by an interviewee, there is wo provision of technology to support sharing, defwebs is not searchable, cannot see outside branches.' another military officer related his experience with setting up a chat room tool, which 'was closed down unilaterally by cismo' (the central is management organisation). the contrary view was put by those who believed improved km would come from tighter procedures to control document storage and sharing, standardisation of tools to enable sharing across the organisation, as well as tighter controls to improve system reliability and reduce the risk of unauthorised access to information. one officer suggested 'the configuration control board in dieb is the way to enforce compliance'. another could see the need for standards, as a pre-requisite for establishing km. to quote, there is a 'need to define standards and procedures: e.g. common web environment and images. develop systems across board'. discussion these informal comments do not constitute fully rigorous scientific findings9, but do highlight an interesting question. how is it possible for concerned, dedicated and intelligent people, who have all thought deeply about knowledge management in their organisation, to come to such divergent opinions on the best way forward? the model in figure 5 provides insights into this paradox. during the cold war there was a relatively stable military strategic environment in which the external attributes and relationships of the ado evolved slowly over time, but within a known superpower framework. internal attributes and relationships in the ado also evolved and adapted in a relatively stable way, with changes in technology and organisation focussed on more efficient operation within the predictable external framework. many of the people interviewed had received their training and much of their expedefweb is the ado intranet. 9 they do, however, constitute a preliminary survey that would provide a useful basis for a more systematic, scientific investigation. work of this nature may be undertaken at some stage in the future. 76 aj1s special edition knowledge management december 2001 rience in this era and some had come to see a world with a predictable framework and efficient internal processes, as "how things should be". this situation is identified with quadrant 4, where the rate of change of the external, attributes and relationships is "mild" and internal characteristics also have an orderly evolution. both external and internal relationships are close to the "edge of order". in this mode, streamlining the organisation is represented by excursions into quadrant 3 to improve internal processes, before returning to quadrant 4 with improved efficiency to meet the known external environment. coping with a disruptive new invention or event was achieved by stabilising a quadrant 2 development in such a way that it could be incorporated back into the quadrant 4 world. with the dramatic change in the strategic military situation in recent times, external relationships have become wildly unpredictable, balancing on the "edge of chaos". internal attributes and relationships, developed in response to the cold war influences, have consequently become unstable. what was best practice in the cold war era becomes inappropriate or inefficient when meeting emerging situations. here the ado is operating in quadrant 2. the interfet peacekeeping mission was an example of this. new and different relationships had to be created very quickly. with australia leading the coalition, the relationship with the usa was nothing like that honed to a high state of efficiency during the cold war. simultaneously, working relationships within a diverse 22 nation coalition had to be developed. also, although the command arrangements within the australian defence force had already undergone significant change to accommodate the post cold war situation, they had to be changed quickly and adapted to meet this unpredicted interfet situation. it seems inevitable from recent events that external attributes and relationships for the ado are likely to remain on the edge of chaos. the ado would be more effective if it could move from quadrant 2 to operate in quadrant 1. to operate in this mode, stable internal attributes and relationships need to be developed in such a way that they can be reconfigured flexibly to accommodate unexpected requirements. the problem becomes one of determining which attributes, relationships and processes should be stabilised to form the efficient backbone on which capabilities can be efficiently mixed and matched to meet sudden changes in the external environment. striving to operate in this quadrant is what, we believe, led to the apparently contradictory comments from those interviewed in the knowledge management study. those interviewees who commented on the need for more flexibility were identifying the need for organisation relationships and attributes that could emerge quickly to accommodate the external environment effectively. those who sought more control and stability were focussed on the need to develop an organisation with a strong and stable framework that could deal efficiently with the external environment. within this framework, the planning issue for this organisation comes down to devising processes, attributes and relationships that can be managed tightly to provide efficiency, without restricting the flexibility needed to quickly adapt to unpredictable changes to the external systems. in terms of the km study, this is the recognition of the need for underlying modules of technology and associated operating procedures operating efficiently "on the edge of order", but with characteristics that allow for rapid reconfiguration to interact effectively with whatever attributes emerged to change relationships with the external environment. in summary, the ado is a quadrant 4 organisation that has been thrust into quadrant 2, and is moving to achieve greater organisational effectiveness by attempting to move to quadrant 1. implications for knowledge management and systems thinking knowledge management the knowledge management debate has tended to be dominated by the software and information systems issues, which has distorted the approach to the management of knowledge in an organisation. berkman (2001), for example, sees this a consequence of the marketing activities of software vendors. as discussed above, knowledge management, particularly when seen as a knowledge creation issue, involves dynamic interplay between tacit and explicit knowledge. km needs to include cultural as well as technological elements. the risk of not mobilising knowledge in the organisation is compounded if knowledge management is perceived as a requirement that can be met with a traditionally engineered software solution. a simplistic software approach to knowledge management is likely to not only leave out the management of tacit knowledge, but is also prone to focus on solving a particular issue, rather than developing a capability. systems thinking the systems thinking concept used in this paper is a useful structure within which to frame the problem of balancing the relationships among the "mild, wild, and crazy" components of an organisation. the self-similar concept of the hierarchy simplified the understanding of the emergent attributes and the relationships at each level. the model 77 ajis special edition knowledge management december 2001 described above can be applied recursively to the organisation and its environment and down through the layers of systems within the organisation. this model can be used as an explanatory tool, and as a framework for developing future strategy for an organisation and the subsystems within it. defining the upper and lower boundaries of the organisation, its components and environment is a difficult issue. the upper boundary is that which defines the system of interest, in our case the organisation. the upper boundary of a government department, for example, might seem to be well defined, but aspects of outsourcing and consultants, for example, cloud the boundary definition. similarly, the lower boundary, which is that which defines the separately identifiable "atomic" components of the system (organisation), is difficult to define exactly. invariably, such boundaries change over time and also differ within alternate system views of the organisation, such as those in figure 4. further work needs to done to extend the model to take account of the other systems hierarchies that simultaneously exist within the organisation. this was briefly discussed in relation to figure 4 that shows diagrammatically two of the many structures that exist in an organisation. useful insights come from considering the single hierarchy, but in many cases the cross linking (termed 'hypertext organisation' by nonaka and takeuchi) reveal other emergent attributes of the components of the organisation and of the organisation itself. novel approaches (e.g. shaw 1997), beyond those of established system analysis techniques, will need to be developed if we are to tap into this richer systems understanding of the organisation. conclusion as the world moves into the knowledge age, the rate of change of the environment in which organisations operate is developing at an ever-increasing pace driven, in large measure, by growth in the capabilities of information and communications technology. with issues of knowledge and innovation now critical to performance in many organisations, traditional approaches to improving organisational performance, based on concepts of organisations as mechanistic or "newtonian" systems have been found wanting. this paper draws together concepts from complexity science, systems thinking and knowledge management to develop a model relating the effectiveness of an organisation to the complexity of its internal and external attributes and relationships. despite its simplicity, the model proved to be a useful lens for observing the defence organisation: at a coarse level of resolution it clarified important differences in the perception of knowledge management in defence. we believe that it might be similarly useful in analysis of other organisations. further work is needed to improve understanding of how the "mild, wild, and crazy" component parts interact within organisations. references barney, j. b. (1991). "firm resources and sustained competitive advantage." journal of management vol. 17, no. 1: pp. 99-120. baumard, p. (1998). "from infowar to knowledge warfare: preparing for the paradigm shift", http://www.iae.univaix.fr/cv/baumard/infowar knowledge warfare.htm. accessed: 2 nov 2001. berkman, e. (2001). "when bad things happen to good ideas-knowledge management", http://www.danvinmag.com/read/040101/badthings.html. accessed: 14 june 2001. boisot, m. h. (1998). knowledge assets: securing competitive advantage in the information economy. oxford, oxford university press. broad, c. d. (1923). the mind and its place in nature. london, kegan paul. burke, m. (2000). "thinking together: new forms of thought system for a revolution in military affairs", defence science and technology organisation. capra, f. (1996). the web of life: a new synthesis of life and matter. london, flamingo. checkland, p. (1981). systems thinking, systems practice. chichester, john wiley & sons. freedman, d. h. (1992). "is management still a science?" harvard business review(nov-decl992): pp. 26-38. gleik, j. (1988). chaos. london, william heinemann. handy, c. b. (1986). understanding organisations. hammondsworth, england, penguin books. kidding, g. j. (2001). "sustaining strategic it advantage in the information age: how strategy paradigms differ by speed." journal of strategic information systems 10(3): 201-222. kline, s. j. (1995). conceptual foundations for multidisciplinary thinking. stanford, stanford university press. kuhn, t. s. (1970). the structure of scientific revolutions. chicago, the university of chicago press. matthews, d., m. burke, et al. (2000). "core concepts of joint systems". australasian systems engineering, test and evaluation conference, brisbane. 78 ajis special edition knowledge management december 2001 matthews, d. b., m. burke, et al. (2000). "assessing the value of a joint systems capability". australasian systems engineering, test and evaluation conference, brisbane. miles, r. e. and c. c. snow (1994). fit, failure and the hall of fame: how companies succeed or fail. new york, the free press. miles, r. e., c. c. snow, et al. (1997). "organising in the knowledge age: anticipating the cellular form." academy of management executive vol. 11 no 4: pp 7-20. mintzberg, h. (1994). the rise and fall of strategic planning: reconceiving roles for planning, plans, planners. new york, the free press. nonaka, i. (1994). "a dynamic theory of organizational knowledge creation." organization science vol. 5, no. l(feb 1994): 14-37. nonaka, i. and h. takeuchi (1995). the knowledge-creating company. new york, oxford university press. pine ii, b. j. (1993). mass customization: the new frontier in business competition. boston, harvard business school press. polanyi, m. (1962). personal knowledge : towards a post-critical philosophy. chicago, university of chicago press. polanyi, m. (1966). the tacit dimension. london, routledge & kegan. porter, m. e. (1980). competitive strategy: techniques for analysing industries and competitors. new york, free press. prahalad, c. k. and g. hamel (1990). "the core competence of the corporation." harvard business review(mayjune 1990): 79-91. pugh, d. s. and d. j. hickson (1996). writers on organisations. london, penguin books. rebello, k. (1996). "inside microsoft: the untold story of how the internet forced bill gates to reverse course", http://www.businessweek.com/1996/b34841.htm. accessed: 16 december 1997. sauer, c. and p. w. yetton, eds. (1997). steps to the future: fresh thinking on the management of it-based organisational transformation. san francisco, jossey-bass. shaw, p. (1997). "intervening in the shadow systems of organizations consulting from a complexity perspective." journal of organizational change management 10(3): 235-250. stacey, r. d. (1996). complexity and creativity in organizations. san francisco, ca, berrett-koehler. stacey, r. d. (1999). strategic management and organisational dynamics : the challenge of complexity. new york, financial times. sveiby, k. e. (1998). "tacit knowledge, sveiby knowledge management", http://www.sveibv.com.au/polanyi.html. accessed: 15nov2001. taylor, f. w. (1923). the principles of scientific management. new york, harper and brothers. the price waterhouse llp change integration team (1996). the paradox principles : how high-performance companies manage chaos, complexity, and contradiction to achieve superior results. chicago, iii., irwin professional. turner, d. and m. crawford (1998). change power: capabilities that drive corporate renewal. warriewood, nsw, business and professional publishing, von bertalanffy, l. (1968). general systems theory: foundations, development, applications. new york, george braziller. 79 data modelling and conceptual modelling: a comparative analysis of functionality and roles t.william olle t.william olle associates ltd., walton on thames, surrey, united kingdom abstract this paper reviews the functionality associated "data modelling" and "conceptual modelling". the history and origins of each term are considered, together with the current interpretation of each term. the term "information modelling" is also taken into account. alternative representation forms are presented and reviewed. the merit of diagrams as a basis for a dialogue with a subject area expert is indicated. the paper suggests that a clear distinction is needed between data modelling and conceptual modelling. both analytic modelling and prescriptive modelling are reviewed. the requirements for a conceptual schema modelling facility over and above the functionality provided by currently available data modelling facilities are presented. the need is emphasized for a conceptual schema modelling facility to support a representation form easily assimilatable by a subject area expert not familiar with information system. based on the distinctions made, the paper suggests a way in which a data modelling facility and a conceptual schema modelling facility can be positioned in an information systems life cycle. introduction to data modelling data modelling is a widely practised skill. it is usually associated with the preparation of diagrams showing entity types and the relationships between pairs of entity types. there are innumerable variations on the basic theme of what in this paper are given the generic name of data structure diagrams. the term "entity type" is not always used and is sometimes replaced by the term "object type" or some related variant. the term "data model" is used in two ways. one way refers to a model for a specific subject area (such as claims handling in an insurance company) as a data model. the other way follows the practice established by date (1975) of referring to the "relational data model" and the "hierarchical data model". some very confusing discussions can take place if this conflict is not resolved. this paper distinguishes by using two terms. the first term is "application data model" which relates to a subject area. some people regard an application data model and a data structure diagram as essentially synonymous. in this paper the argument is put forward that any given application data model may be represented as a diagram or possibly in some other form. hence the diagram is merely one of a number of options. the second term is "data modelling facility" which is an approach used to prepare an application data model. these two terms are based on the iso reference model of data management (1993) which makes considerable use of the latter and introduces the former merely to clear up any possible confusion. this is discussed later in this paper. introduction to conceptual modelling the term "conceptual model" is much less widely used. in some cases, it seems to have the same meaning as "application data model" introduced above. in other cases, it is clearly attempting to be much more. the term "conceptual schema" was first introduced in the "draft interim report" prepared by an ansi/x3/sparc study group (1975). their work was picked up and elaborated some two years later by an iso committee (tc97/sc5/wg3) which prepared a seminal report entitled "concepts and facilities for the conceptual schema and the information base" (1977). (the use of the definite article in the title was presumably not intended to imply global uniqueness.) when the idea of a conceptual schema was first introduced in 1975, it was seen as a solution to a practical problem. it became a springboard form much research. the practical problem was that there was considerable debate about the relative merits of the three main approaches in use at that time to structuring a database. these approaches were identified as network, hierarchical and relational. the ansi study group rightly recognized that data in a subject area had an inherent structure which was quite independent of the type of database management system (dbms) which might 46 ua subsequently be used to structure and manipulate the data. this dbms independent structure could be represented in a "conceptual schema". the dbms dependent schema was called an "internal schema". in passing, it should be noted that the 1977 document borrows a term used in philosophy by wittgenstein, namely "universe of discourse". the more down to earth term "subject area" is preferred in this paper. one of the most important added value concepts in the 1977 document is that of the 100% principle. this is defined as follows: "all relevant static and dynamic aspects, i.e. all rules and laws, etc. of the universe of discourse should be described in the conceptual schema. the information system cannot and should not be held responsible for not meeting those described elsewhere, including in particular those in application programs." the clarification in the second sentence is clearer than the statement of the principle in the first sentence. the 100% principle may be paraphrased as follows. the business rules applicable to a given subject area must be specified declaratively. the place where they are captured is called a conceptual schema. the business rules should not be embedded in application programs. in absence of the widespread recognition of the 100% principle and the availability of commercially available products based on this principle, there has typically been no choice available to the designer of an information system. even those who recognize and accept the principle have had to fall back on representing many of the business rules for a subject area using a programming language. there have been numerous attempts to design and promote facilities for preparing a conceptual schema. in essence, these amount to modelling approaches which go somewhat further than the more widely used data modelling facilities towards meeting the 100% principle. it is a matter for evaluation and conjecture whether the 100% principle is indeed fully attainable using such approaches. information modelling the origin of the term "information modelling" is obscure and it is impossible to point to any first usage. it is certainly not defined in the 1977 document. the distinction between the term "data" and the term "information" as established by mp (1966) ought to help clarify the useful distinction between "information modelling" and "data modelling". the terms "data" and "information" were defined as follows: "data: a representation of facts or ideas in a formalized manner capable of being communicated by some process. note: the representation may be more suitable either for human interpretation (e.g. printed text) or for interpretation by equipment (e.g. punched cards or electrical signals)." "information: in automatic data processing, the meaning that a human ascribes to data by means of the known conventions used in its representation. note: the term has a sense wider than that of ordinary information theory and nearer to that of common usage." one may be able assume from these definitions that data modelling has something to do with modelling the representations of the data and that information modelling has something to do with modelling the meaning ascribed to the data. however, the definition of "information" makes it clear that the meaning is ascribed "by means of the known conventions used in its representation." the crux of the distinction between information and data, appears to be found in the phrase "the meaning that a human ascribes to data". if no known human ascribes any meaning to the data, should september 1993 47 it still be included in a model? if different humans ascribe different meanings to the same data, should both (or all) be modelled? looking at the "data modelling facilities" which use the term "data model" (ssadm4 (1990), rochfeld & morejon 1989), and those which use the term "information model" (veryard 1992, jardine & van griethhuysen 1989), not to mention those which use the term "entity relationship modelling" (barker 1990), there does not appear to be a significant distinction in functionality. in other words, the choice of term used to qualify the kind of modelling involved is essentially arbitrary. an in-depth comparative analysis of functionality provided by each approach would certainly provide valuable insight. however, it is not clear that it would justify the disparity of terminology. in this paper the terms "data modelling" and "data modelling facility" are preferred to other variants. more on data modelling the iso reference model of data management (1977 clause 5.4) describes a data modelling facility as follows: "a schema is prepared according to a set of data structuring rules. each set of data structuring rules may have an associated set of data manipulation rules which define the processes which may be performed on data structured according to the data structuring rules. the data structuring rules and data manipulation rules together are called a data modelling facility." this description of a data modelling facility clearly sets bounds around the concept of a data modelling facility to data and the associated processes. the iso reference model of data management (1977 clause 8.4.2) also identifies three ways in which a data management standard can involve a data modelling facility as follows "a) a data modelling facility may itself be the subject of a standard b) a data modelling facility may be used explicitly as a definition tool in the specification of another standard c) the data structuring rules of a data modelling facility may be implicit in another standard (such as a standard for a database language)." express (1993) is an example of alternative a). the irds services interface (1992) is an example of alternative b). sql92 (1992) is an example of alternative c). an underlying thought here is that an application data model prepared using a data modelling facility may have different representation forms. some typical representation forms are the following a) diagrams b) language (based on a clearly identified character set) c) narrative proses d) structured tables this also implies that one may consider an application data model independently of any particular representation form. the same application data model may be represented as a diagram, in linguistic form and in tables. the following is a trivial but abstract example of the first three representation forms by far the most common kind of relationship which needs to be expressed using a data modelling facility is the well known one to many relationship (where many is to be interpreted as zero, one or more) in narrative prose, it might be represented as follows: for each a, there exists zero, one or more b's and for each b, there must exist exactly one a. in diagrammatic form, it might be represented in a data structure diagram as shown in figure 1. 48 ajis e. a data modelling facility which is different from and therefore "neutral" with respect to broadly similar data modelling facilities used in commercially available data base management systems. these alternative interpretations may be analyzed as follows. results of analysis as indicated earlier in this paper, data analysis facilities are widely used and there is both disparity and commonality among the techniques in use in various parts of the world. some techniques are being energetically promoted by minority groups. the situation is complicated by the recent rapid developments in case (computer aided software engineering) tools. the protagonists of some data analysis facilities choose to use the terms containing the word "conceptual" rather than "data" to refer to the approach used. in terms of the definition of "conceptual schema" cited earlier in this paper, it would be difficult to provide a justification for the terminology chosen. 100% principle the 100% principle expounded in the 1977 document is an important principle which indirectly has had a major influence in the development of database language standards such as the 1989 and 1992 versions of the iso database language sql standard (1992). this language specifications make it possible to specify declaratively a very large percentage of the constraints on the data which a database designer is ever likely to want to define. while sql is never thought of or promoted as a means of defining a conceptual schema, it is, in this very important respect, functionally superior to many of the approaches labelled conceptual modelling. storage independent data definition some groups, whose view of information systems in general and data concepts in particular, is normally on the level of stored representations, have adopted the term conceptual schema to refer to some kind of representation of the data definition which is above the level of stored representations. they tend to think that sql92 is concerned with stored representations of data because it is used for defining databases. sql92 is in fact a language which enables the definition of data in a way which is independent of how the data is represented in storage. the traditional term "logical data structure" is used in the standard (rather than the term "conceptual") in order to emphasize the difference with "physical data structure" which is used to connote both representation in storage and also the inclusion of data constructs such as indices and pointers which are included for performance reasons but of which the user does not need to be aware. the designers of the sql92 standard (1992) have in fact made a point of excluding from the standard certain aspects, such as the definition of indexes, which are seen as belonging to the physical data structure. the fact that such facilities are to be found in commercially available products is misleading. common view for electronic data interchange it is clear that in electronic data interchange (edi), one needs a definition of the data to be interchanged which is common to all sites involved in a set of interchanges. much of the edi work has been concerned with the specification of standard formats for an industry area, such as banking or travel. as edi work tends to adopt a more generalized approach to its standardization, the need for a common definition facility becomes apparent. it is important to note the distinction between the requirement to have a data definition which is common to several sites and the requirement to be able to prepare such a definition easily. the representation form suitable for one is not suitable for the other. more specifically, to support the person to person dialogue necessary during the preparation, it is normal to use some form of diagram. 50 ajis however, to support the computerized interchange, some form of computerized representation form is needed and it is not likely to be a graphics file. neutral data definition the relative merit of various data modelling facilities has proved to be a somewhat emotive topic among data base researchers and practitioners. the debate goes back to the early seventies. various advantages have been claimed for each approach. when it is difficult to evaluate competing approaches, there is a tendency to develop yet another "neutral" approach. an interesting and potentially far reaching example of this kind of thinking is to be found in the international standard for directory services. to avoid getting involved in what was seen at the time as a domestic wrangle among database experts, a "neutral" data modelling facility was developed which was claimed to be more suitable for the task than those used for commercial information systems. since the term "conceptual schema" was originally introduced for this purpose, the choice of label to indicate neutrality is perhaps understandable. analytic and prescriptive use of a modelling facility an important distinction needs to be made at this point between the analytic and the prescriptive modes of use of a modelling facility. the term "modelling facility" is used in this paper when discussing aspects which are common to the use of both a data modelling facility and a conceptual schema modelling facility. analytic models a modelling facility may be used to prepare an analytic model of a subject area. such an analytic model is a statement by an analyst (or team of analysts) of how the subject area is to be interpreted. unless the analyst is an expert in the subject area, then the analyst has to conduct a dialogue with one or more subject area experts, on the basis of which the analytic model can be prepared. in practice, some subject area experts describe the subject area as it is and other experts describe it as they would like to see it at some time in the future. to be purely analytic, a model must avoid any statement about the information system which may be needed to support organizational and personal activities in the subject area. prescriptive models a modelling facility may be used to prepare a prescriptive model for a subject area or indeed of a subject area. a prescriptive model for a subject area is a prescriptive for an information system to be built to support some or all of the activities being performed in the subject area. a prescriptive model of a subject area is a model of how the subject area ought to be quite independently of any information system which may be necessary to support the activities which are to be performed in the subject area. the situation of more interest here is the former. a prescriptive model in effect prescribes, in subject area terms, the requirements for an information system to support the activities being performed in the subject area. this model is independent of how the information system is to be constructed. data modelling and conceptual modelling as indicated, the discussion of analytic models and prescriptive models applies equally to both data modelling and conceptual modelling. however, a data modelling facility has to be seen (in terms of this paper) as functionally more restricted than a conceptual modelling facility. a data modelling facility is restricted to data and the processes which may be performed on the data. while many self styled conceptual schema modelling facilities do no more than this, it is surely deskable for future progress in the filed of modelling to regard a conceptual schema modelling facility as not restricted to data and the processes to be performed on the data. 51 september 1993 in order to pursue this analysis further, it is useful to examine the role of modelling facilities in an information systems life cycle. role of modelling facilities em information system life cycle there is no widely accepted standard for an information system life cycle. numerous alternative breakdowns into stages have been proposed. the breakdown into the following major stages is used in this paper and is based on the one used in the ifip framework for understanding information systems methodologies (olle, hagelstein, macdonald, rolloand, sol, van assche & verrijn-stuart 1991). planning analysis system design construction design construction operation evolution outline of stages the planning stage is a preliminary stage in which decisions can be made about the subject area such as establishing boundaries for the ensuing analysis. the analysis stage is a stage in which a subject area (which may in practice be a complete enterprise) is examined with the aim of building a formalized understanding of what activities are performed in the subject area and the kinds of data which are needed to support such activities. it is necessary to distinguish between two stages containing the word "design". "system design" is the stage in which the externally visible aspects of the information system are prescribed, quite independently of how the system is to be constructed and which construction tools are to be used. this could also be called logical design or even conceptual design. the construction design stage is that which takes into account the construction tools to be used and prescribes how the system is to be constructed using these tools. where appropriate, this sub-stage may consider such aspects of design as the breakdown into programming modules and the designation of performance enhancing indexes to be used when accessing the database. this stage could also be called implementation design. role of a conceptual schema modelling facility a conceptual schema modelling facility used in analytic mode is used only in the analysis stage. in the prescriptive mode, it may subsequently used in the system design stage. it is not useful to consider a conceptual schema modelling facility in a purely prescriptive role (to the exclusion of the analytic role). conceptual schema in evolution stage the role of a conceptual schema in the evolution stage is significant and indicates that there are different kinds of evolution. 1. new or changed business rules revise conceptual model developed in analysis stage 2. improved conceptual design based on unchanged business rules 3. conceptual design unchanged, but performance enhancing changes required to construction design 4. construction design unchanged but construction errors to be corrected these requirements emphasize the need for the conceptual schema, in both analytic and prescriptive use, to be easy to modify. 52 ajis role of a data modelling facility in different stages a data modelling facility may also be used in the same stages as those in which a conceptual schema modelling facility is used. however, a data modelling facility would need to be supplemented in various ways. in addition to these uses, a data modelling facility may be used in the construction design stage and in the operational stage. a construction tool may be based on a database language standard. an important component in the operation of a computerized information system would normally be a database server which interprets and enforces specifications expressed in that language. a useful distinction between a conceptual schema modelling facility and a data modelling facility can be made in terms of the potential role of each in an information systems life cycle. however, where there is an overlap, as in the analysis stage a more important distinction can be made in terms of how much of the complete analysis necessary can be done using a data modelling facility (as currently understood) and how much needs to be done in terms of a conceptual schema modelling facility (as perceived to be needed). there are several requirements for a conceptual schema modelling facility over and above the functionality provided by a typical data modelling facility and these are presented in detail in the following section. requirements for a conceptual schema modelling facility this section itemizes requirements for a conceptual schema modelling facility and relates these where applicable to the capability provided by data modelling facilities. provide user friendly representation form a conceptual schema modelling facility must be usable as an elicitation technique for determining the business rules in a given subject area. this means that it is necessary that one form in which a conceptual schema may be represented is assimilatable by subject area experts who are not familiar with informatics oriented representation forms and certainly not with forms such as predicate logic. typically, this representation form would be a graphic representation form. as such, the form would have a prescribed syntax and semantics. it is also a requirement that a conceptual schema modelling facility should provide three or four "capability levels" to enable dialogue between a modeller and subject area experts when eliciting business rules of varying complexity. in addition, the level of skill for understanding conceptual models which is available among subject area experts may vary considerably. there may be other standardized representation forms for the conceptual schema in addition to the user friendly form. in order to preserve the user friendliness, it may be possible to represent more business rules in the non-graphic form than in the graphic form. the standard must clarify this difference. business activities a conceptual schema modelling facility must be able to model business activities. a business activity is an activity performed in a business area. it may be computerizable, computer supported or a purely human activity. one aim of the use of the conceptual schema modelling facility is to determine in which category to place a business activity with respect to computerization. a conceptual schema modelling facility must provide for the identification and representation of business activities, namely activities which are performed (somehow or another) in a given business area. depending on the scope of the business area, it must be possible to represent a hierarchical decomposition of each business activity into two or more lower level business activities. there should be no limit on the number of levels of decomposition allowed. a conceptual schema modelling facility must be able to model other inter-relationships between business activities as well as decomposition. such inter-relationships include the following september 1993 53 a) satisfactory completion of one business activity may impact the initiation of another business activity b) one business activity may not be in progress at the same time as another business activity a conceptual schema modelling facility must also be able to model (prescriptively) the users' perception of the user tasks which are required to be performed by an information system to be based on the conceptual schema. it must also be possible for a conceptual schema modelling facility to be able to model the work patterns in the business area in order to determine the most appropriate grouping of user tasks to support such work patterns. the is also a requirement for a conceptual schema modelling facility to be able to relate the business processes modelled with the data concepts. this would normally be in the form of a bidirectional cross reference table (also called a matrix). the representation forms used to depict business activities must include a graphic option (to satisfy the user friendly requirement) and may have another representation form. handle analysis of distribution considerations a conceptual schema modelling facility must be able to analyze the distribution aspects of the business area. these include the following a) what locations are relevant to the business area b) what business activities are performed at each location c) what business events can happen at each location d) what data is required at each location to support business activities it is necessary to achieve an understanding of the distribution aspects of the business in order to be able to prescribe the best approach to distribution for an information system to support the business area. as far as can be ascertained, none of the conceptual schema modelling facilities in use address the problem of distributed systems (or the more demanding problems of inter-operable systems to be discussed in the next section). in the case of distributed systems, the lack of existing facilities is due to the fact that conceptual schema modelling facilities traditionally avoid implementation issues and whether to distribute or not is typically regarded as an implementation issue. it is reasonable to expect that a conceptual schema modelling facility should be able to represent the concept of distribution. in analytic mode, representing the status quo for a business area, it should be possible to represent how various activities are performed at different physical locations. in prescriptive mode, it should be possible to define where certain activities should be performed in a reorganization of the way in which business activities are to be performed. if the distribution aspects of the business activities in a business area can be represented using a standard conceptual schema modelling facility, then this would provide a distinct benefit to users of products based on the standard. a conceptual schema modelling facility must provide for different representations of the different physical locations at which business activities are performed and also which activities are performed at each location. possible representations of location might be as follows latitude and longitude name of town, city or country identification of work site this aspect of "representation" is orthogonal to that considered in section 4 of this paper. temporal considerations a conceptual schema modelling facility must be able to model business events. a business event is an event which can happen in a business area. examples are 54 ajis 1. send job offer to applicant 2. receive insurance claim 3. dispatch goods. the difference between business activities and business events should be perceived as follows. a business activity is on-going and consumes measurable resources. a business event occurs at a specific point in time, can be date time stamped and does not consume measurable resources. a business event may be initiated by human action, calendar clock or a situation occurring in the business. the aim of the use of the conceptual schema modelling facility is to determine which business event are of sufficient importance to merit consideration in the design of a computerized information system. a conceptual schema modelling facility must provide for the representation of temporal aspects of the business area. these aspects include, but are not limited to, the following a) definition of business events (as distinct from business activities) b) definition of temporal constraints between business activities c) definition of temporal constraints between business events d) definition of time zones associated with locations at which business activities are performed and business events happen both a graphic and non-graphic representation form are needed. handling data concepts while data concepts are handled to varying degrees of completeness in existing standards for data modelling facilities (express and sql92), it is necessary to address the added requirements for a conceptual schema modelling facility standard and relate these requirements to the functionality provided by these two standards. the requirement for a user friendly representation of data concepts is of major importance. both express and sql92 are standards for a concrete syntax. however, it should be noted that the introduction to the express standard contains the following statement (page xiv) "express is consistent with the requirements specified in iso tr 9007 for a conceptual schema language." it is also clear from the document that use of the express "data specification language" is intended to precede use of possible selected "implementation methods", one of which is "database implementations". in addition to the express data specification language, there is also included express-g which is a graphical representation form. the requirements for the handling of data concepts in a standard schema modelling facility are one of the more complex issues associated with the proposed standard and the following factors must be taken into account. one of the major aims is to be able to achieve the 100% principle stated in tr9007. this means that business rules which can be represented as constraints on data need to be represented declaratively rather than embedded in application programs. (it is noted that not all business rules can necessarily be stated in terms of constraints on data.) with the inclusion of the check clause and ten types of predicate (for example comparison, like, match, overlaps, exists, unique), sql92 has gone a long way towards allowing a user of its data modelling facility to achieve the 100% principle. a requirement could be stated for a standard conceptual schema modelling facility that it should be able to represent the superset of all data constraints expressible in sql 92 and all data constraints in express. however, it is not clear that it is desirable to use scarce standards development resources to reinvent a new concrete syntax for this purpose. it is also not clear that existing graphic standards (such as express-g) can be extended to enable representation of the full set of data constraints necessary to achieve the 100% principle. the requirement for handling data concepts in the conceptual schema modelling facility standard is therefore stated as follows. the requirement must be to provide facilities for a user friendly graphical representation for sets of carefully selected "constructs" (such as a zero or one to many relationship september 1993 5s or an exclusivity constraint). the subsetting should be standardized in terms of difficulty levels (possibly three or four such levels) in order to place bounds on the dialogue which can take place between the conceptual schema modeller and a subject area expert. it would be useful to provide a technique in the graphic representation form for designating the existence of a data constraint on data in the entity type rather than attempting to designate the possible multiplicity of other entity types which are referenced in formulating the constraint. handling process concepts as indicated earlier, the conceptual schema modelling facility standard should provide facilities for modelling business activities (sometimes called business processes or business functions). it is also noted that a data modelling facility "may have an associated set of data manipulation rules". the term "manipulation" is intended to refer to basic processes performed on the data (create, retrieve, update, delete) rather than to any application process which makes use of these basic processes. it is not a requirement for a standard conceptual schema modelling facility to be able to specify application processes which can be regarded as handled in terms of application programming. it is a requirement for a conceptual schema modelling facility to be able to specify the users' view of the processes which the information system can perform. the proposed name for these processes is "user tasks. the concept of a user task is between a business activity and an application process. a user task must be meaningful to a user in the sense that a user may be able to select it from a menu containing several such tasks. alternatively a user who makes use of the same user task repetitively may be able to press a function key or enter a short code identifying the task. how a user task is initiated is clearly an implementation issue and not appropriate for a conceptual schema modelling facility standard. however, which user tasks may be initiated is part of a conceptual schema modelling facility when it is used in the prescriptive mode, but not when it is used in the analytic mode (see analytic models earlier). access control the requirement for handling access control in a conceptual schema modelling facility is to be able to develop an analytic model of the different user roles and to identify which user roles are authorized to perform which business activities. the business activities which need to be considered for this purpose are normally low level activities identified as a result of decomposing higher level activities. when a conceptual schema modelling facility is used in a prescriptive mode, then the consideration of authorization of user roles would need to relate to the user tasks which may be initiated by a user at the computer human interface. the conceptual schema modelling facility standard used in analytic mode must provide a means of analyzing the requirements for access control in the business area. these requirements would need to be expressed in terms of business activities on the lowest level in the decomposition hierarchy. the conceptual schema modelling facility used in prescriptive mode must provide a means of prescribing the requirements for access control in the information systems to be designed for use in the business area. these requirements would need to be expressed in terms of the user tasks relevant to the information system to be designed for use in the business area. provide both analytic and prescriptive mode of use a conceptual schema modelling facility is intended for use in an early stage of the information system life cycle. it may be used to prepare an analytic model of a business area (universe of discourse). it may also be used to prepare a prescriptive model of a business area. an analytic model of a business area may be used as a basis for specifying a prescriptive model of an information system for the business area. a prescriptive model of a business area may be used for reorganizing the business activities to be carried out in a business area. certain conceptual schema modelling facilities are already in use in this way. however, these appear to have no common basis and there are perceivable incompatibilities among them. 56 ajis conclusions the following points developed in this paper can be formulated as conclusions. 1. in terms of approaches currently available, there appears to be no significant difference among the following a data modelling facility, an information modelling facility and a conceptual schema modelling facility. the traditional term "data model" and its associated derivatives are preferred. 2. a data model (or a conceptual model) may have more than one representation form diagram, language, narrative. 3. since the term "conceptual schema" was introduced, it has come to mean different things to different people. 4. it is useful to recognize a clear distinction between a data modelling facility and a conceptual schema modelling facility, based on the overlapping and differing roles that each can fulfil in an information systems life cycle. 5. for a conceptual schema modelling facility to be useful in practice, it is required to be able to embrace a number of aspects of a subject area such as activities, temporal consideration and distribution, as well as data. acknowledgement the work on this paper has been motivated by the author's participation in the iso sc21 special working group on modelling facilities which was formed with a finite life span to determine (among other things) whether an international standard for a conceptual schema modelling facility would be beneficial. the group completed its work in june 1993 and drafted a new project proposal for such a standard. the views expressed in this paper are the author's own and should not be construed as representing a consensus view of the special working group. references ansi/x3/sparc (1975) study group on data base management systems: interim report 75 0208. acm sigmod newsletter, fdt, vol 7 no 2 barker r. (1990) case*method entity relationship modelling addison-wesley. date c.j. (1975) introduction to database systems addison wesley mp icc (1966) vocabulary of information processing north holland is 10027 (1993) reference model of data management is 10303-11 (1993) product data representation and exchange part 11: the express language reference model. is tr 9007 (1977) concepts and facilities for a conceptual schema language is 10032 (1992) irds services interface is 9075 (11992) database language sql is 9594 (also known as x500) directory services jardine d,.a. and van griethuysen j.j. (1989) infomod volume 1 concepts and information structure mmj information engineering olle, t.w., hagelstein, j., macdonald, i.g., holland, c, sol, h.g., van assche, f.j.m., verrijn-stuart, a.a. (1991) information systems methodologies a framework for understanding addisonwesley rochfeld a. and morejon j. (1989) la methode merise tome 3 gamine operatoire les editions d'organisation. ssadm version 4 reference manuals. published 1990 by national computing centre and blackwells veryard r. (1992) information modelling: practical guidance prentice hall september 1993 57 431-article with all pages.pdf scan2022-04-01_152404 operational complexity of direct manipulation tasks in a windows environment john r. maltby faculty of business and computing southern cross university p.o. box 157, east lismore nsw 2480, australia jmaltby@scu.edu.au abstract a method to quantify operational complexity of direct manipulation tasks in a windows environment is discussed. the method utilises a formula from communication theory due to shannon and weaver which describes the uncertainty h(p\,p2,...pn) in outcome of an event which is the result of a markov process where the individual events have probabilities of occurrence pi,p2,-pna taxonomy of basic windows operations is developed for microsoft windows and used to show that a windows dialogue can also be described as a markov process. the h formula is then applied to determine the total complexity of a sequence of basic windows operations and thus provide some measure of the complexity of a given task as seen by the user. an estimate of the total entropy of a microsoft windows language source is obtained, indicating that the redundancy of windows dialogue is about 28%. introduction guidelines for graphical user interface (gui) and screen designs are reasonably numerous (galitz 1989; rivlin, lewis and davies-cooper 1990; powell 1990; marcus 1992; mayhew 1992; schneiderman 1992; hix and hartson 1993). most of these guidelines appear sound and are clearly constructive in helping eliminate aspects of poor interface design, but they are all qualitative and are in general directed towards the interface design process. some formal methods do exist for analysing and documenting user actions and operations: these include the keystroke-level model (card and moran, 1980), command language grammar (clg) (moran, 1981), the goals, operators, methods and selection (goms) model (card, moran and newell, 1983), task action grammar (tag) (payne and green, 1986) and user action notation (uan) (hartson, siochi and hix, 1990); however, these methods are also in general directed towards the design process. neither the guidelines nor the formal methods listed above can provide a quantitative method of analysis of interface design which will enable different interfaces to be compared on the basis of their complexity, usability and efficiency, as seen by the user when performing specific tasks. in this paper, we attempt to address this problem for guis, firstly by describing a windows dialogue as a markov process and secondly by utilising a formula from communication theory, originally derived by shannon and weaver (1949), in order to 30 describe the complexity of a gui. specifically, we will choose the microsoft windows interface for our analysis and assume that the interface is directly manipulated using the usual one or two button mouse pointing device. the windows user as a language source we begin by describing the nature of the communication process between a user and a gui. in order to complete a task in a windows environment, a user has to sequentially undertake a series of actions and operations. this sequence is similar to transmitting a series of symbols down a half-duplex communication channel and waiting for a response after each symbol has been transmitted: in the case of a gui, the interface will respond after each operation is transmitted by changing its state. the messages transmitted to the interface from the user in this way are the result of user actions and operations. thus we can consider the user as the source of a sequence of actions and operations which we will refer to as basic windows actions (bwas) and basic windows operations (bwos). bwas are the actions undertaken by the user on the mouse and/or keyboard, with which the user manipulates instances of windows object classes (wocs) in a wimp gui. bwos are the operations performed on the interface which are necessary in order to complete a task, such as copying a file, cutting and pasting text, or creating a document with a word processor. thus title bar is an example of a woc, click is an example of a bwa and open window is an example of a bwo. in constructing a sequence of bwos, the user is bound by the rules of the windows language in the same way that an author is the source of an english sentence and is bound by the rules of the english language. we will thus refer to the user as a language source for the particular gui being utilised. any language source can be considered as a stochastic system and thus be analysed as a markov process in which the occurrence of a future state depends upon the probability of transition from an immediately preceding state. by assigning a symbol of the english language to each state 5; of a communication system, shannon and weaver (1949) use a markov process to describe the transmission of a message which consists of a series of such symbols, where a symbol can be either a letter or a word unit. the transition probability for each state (i.e. the chance of occurrence of each symbol) is determined as a series of approximations: in the zero-order approximation, each symbol in a message is independent and has an equal probability of occurrence; in the first-order approximation, the symbols are independent but occur with the frequency of english text; in the secondorder approximation, the probability of occurrence of each symbol depends upon the previous symbol in the message according to the structure of english text (this is known as a digram structure); in the third-order approximation, the probability of occurrence of each symbol depends upon the previous two symbols in the message according to the structure of english text (trigram structure); etc. in order to use the same technique to describe a windows dialogue, it is necessary to define the equivalent of alphanumeric 31 and special symbols (or word units) in a windows system. we approach this problem by first describing user actions at their most primitive level using the user action notation. user action notation user action notation (uan) (hartson, siochi and mix, 1990) can be used to describe gui operations in a high degree of detail. its purpose is to enable such operations to be explicitly detailed so that they can be unambiguously implemented by an interface programmer. by way of example, we will use it here to analyse a simple task with microsoft windows: that of copying a file from a directory on one drive to a directory on another. we will imagine that the task is carried out by a user with a specific microsoft windows set up in which the file manager icon is minimized and that the user performs the steps described in table 1 and illustrated in figures 1 through 6. steps step 1 (figure 1) step 2 step 3 (figure 2) step 4 (figure 3) step 5 (figure 4) step 6 (figure 5) step 7 (figure 6) step 8 step 9 bwo [xi] [mw] [of] [vd] [mo] [mo] [ma] [cw] [nw] operation description execute the file manager by double clicking on file manager icon. maximise the file manager window by clicking on the maximise button. open the folder files in the c:\ drive directory tree by double clicking on the files folder icon. this operation makes visible the icon of the file which is to be copied to a:\ drive, which is ass4.xls. view the directories of drive a:\ by double clicking on the drive icon and opening a new window. this operation obscures the file icon ass4.xls which is to be copied. move the a:\ window out of the way of the source file, ass4.xls drag the ass4.xls icon from the c:\files window to the a:\ window. this operation makes the target window inactive. it also completes the file copy task. all future operations are to restore the system to its original state. make active the a:\ window again so that it can be closed. close the a:\ window. minimise the file manager window. the system is now back in its original state, as represented in figure 1 . table 1: 9 steps of a file copy task 32 hng. figure 1: step 1, bwo [xi] file qlsk iree ylew options tool] window help fflhd dtant opngcov e. ok q« t. owe ocartii gtdg oeum @«ntnt<loc dmot diofcgo qttuiuq c:8.03bkbtree. lie.l36kbtotal total 89 m«(!) (883.672 by1»f) figure 3: step 4, bwo [vd] pie qlik icce flew options too|i wlnjow help ai.«ikfltiee. l.«3kblotal total ule(s)pbyles) figure 5: step 6, bwo [mo] lie bisk iree view options too|i fllniow 3« bc| t|or.john| c]«w donda oiboiwl gdenme qfin ogmwn qimax qdn d«a« drew orcb.hbkbtiee. h8.13skbtotal total ?4 file(s) (53.882 bytes) figure 2: step 3, bwo [of] pit qltk iree yiew qptloni tooli ttlntow help iie. 1.423kbtdtol total lhlc(i)(0bytoi) figure 4: step 5, bwo [mo] efat •ocartie •dconj •(±1*123 •o»» •qhoi •qto» qrta elixpni qmaach •dtejogo •ommi • n .̂ qcertn qond q*m q™» oho qhom drt>. dm*" optogcorp cd'bmch diojogo c]iai>wt di ddg fd«oitî a) b»4doc [f]«te|»td)c ©•nan. doc dtfcdomtyd bocfcidot b«it»ntpdac ^&^h^ qbdutud ntatl.m db«2ea el selbobd 1 we(i) (<.212 bylsj) totej 89 tle(«) (883.672 byws) figure 6: step 7, bwo [ma] 33 a windows user will have noted that the above file copy task could have been completed in fewer operations: instead of opening a new window for the a drive (as in step 4, figure 3), the user could have simply dragged the source file icon to the a drive icon. however, we will imagine in this case that either the user did not know that this was possible, or that the user wanted to check the contents of the disc in drive a before performing the copy. it is also of interest to note that the move object operation of step 6 (figure 5) makes and leaves active the c drive window which consequently overlaps the a drive window. not only this, but windows immediately repositions the dragged icon to the top left of the right hand partition. in this case, the icon and accompanying text turns out to be totally obscured by the source window, as shown in step 7 (figure 6), thus denying the user any feedback as to the result of the copy process. a uan description of the file copy task is given in table 2, using the symbols recommended by hix and hartson (1993). as can be seen, the uan details tasks at the user action level: the actions specified are primitive user actions (puas), so called because they cannot be broken down into sub-components. these actions include a mouse button press (mv), a mouse button release (ma) and a mouse move (~). in uan, the typographic symbols are chosen to symbolically represent the actual action; thus v is similar to a down arrow, a to an up arrow, while ~ gives the indication of movement and * is used to indicate repetition. a prime (') is used to indicate a specific instance of an object, such as a particular file. other symbols to depict the interface feedback to the user include > to represent follow the cursor and ! to represent highlighting. the uan symbols used in this example are summarised in table 3. it is seen that the analysis of even a simple task like copying a file can reveal design defects in the interface. in the task just described, at least one unnecessary action is required of the user because of step 4. if the destination window had been tiled with the source window, rather than cascaded, then the source file icon might still be visible, thus removing the need for step 5. if uan were used by an interface designer to design an interface for copying a file, then the lines in table 2 which depict step 5 would presumably not have been included. such a design might not then have explicitly specified exactly how the secondary windows were to be arranged. this illustrates an important difference between using a notation for design and using it for analysis: when used for analysis, the notation can emphasise poor design because it explicitly requires the analyst to document all user actions; when used for design, however, the notation cannot compensate for any omissions of the designer. thus certain user operations may be dictated by the nature of the interface rather than chosen by the user as direct steps towards goal accomplishment. although uan provides a method of describing primitive user actions in detail, its analysis is at too low a level for our purposes. we require to describe a gui dialogue in terms of more coarse actions and operations, such that each "symbol" in the dialogue is the equivalent of a word in a natural language. towards this end, we look at classifications of basic windows actions and operations. 34 gui: microsoft windows task: copy file user actions -[icon'] mva (ten) mva -[folder icon'] mva (ten) mva -[disk icon'] mva (ten) mva -[title bar'] mv ~[x,y]* ~[x',y'] ma -[secondary window'] mva -[file icon'] mv ~(x,y)* ~(x",y") ma -[secondary window"] mva -[close icon"] mva (ten) mva -[minimise icon'] mva interface feedback display(primary window") display(secondary window') folder icon! display(folder tree) display(folder contents) display(secondary window") outline(title bar') > @x',y' redisplay(secondary window") file icon! file icon > file icon-! minimise(primary window) interface state selected = folder' selected = title bar" selected = title bar' selected = file icon' selected = title bar' ' selected = title bar' table 2: uan sequence for a file copy operation in microsoft windows ~ [x] v a m t * > (t<n) ! move the cursor the context of the object x depress release mouse device specific instance of an object iterate follow cursor within a specific time highlight object table 3: some common uan symbols 35 basic windows actions we will define a bwa as an action commonly associated with manipulating a pointing device (such as a mouse), or with manipulating a key on the keyboard, and use the symbols < and > to denote a specific bwa. with a mouse, there are three basic actions that can be undertaken: these are a click, a double click and a drag. these actions can be defined using the uan in the following way: < c > <mva < cc> <— mva (t< n) mva <d> <-mv ~(x ,y)* ~(x 'y ' )m a thus a click is the action of both pressing and releasing a mouse button, a double click is the action of doing this twice within a set time period, and a drag is the action of pressing a mouse button, moving the mouse to a specific position, and then releasing the mouse button. with a keyboard, there are two basic actions that can be undertaken: these are a single key press and holding down one key whilst pressing and releasing another (as when using the shift, alt or ctrl keys, for example). these actions can be defined in the uan as follows: < k > <-kva < k k > <-k'vk"va k'a note that the actual motion of a mouse prior to selecting a target is not defined as a bwa, although it is normally represented in uan. moving a mouse prior to target selection is here considered similar to moving a hand over a keyboard before deciding which key to press. although such actions, known as homing, are considered in some models such as the keystroke level model (card, moran and newall, 1980), they are not relevant to the gui language we are trying to develop, which does not include time as an independent variable. neither have we included the symbols for motion (x,y)* etc. in the definition of <kk>, even though, strictly speaking, a finger (perhaps on the same hand) will be moved to a different key. finally, an action can be performed similar to <kk> which requires a key to be pressed and held down whilst a mouse button is pressed and released. this action can be represented in uan thus: < kb > 4kv mva ka 36 basic windows operations the windows interface consists of a number of objects each of which is an instance of a particular windows object class (woc). these objects can be manipulated by basic windows operations (bwos) which in turn are performed using bwas. there are some problems in defining and identifying window operations and a decision must be made on the granularity of the dialogue analysis. for example, should scroll up be a single operation, or should it be a goal (or sub-task) which can be achieved by a variety of different operations? if we consider scroll up to be a bwo, then it can be achieved by using a variety of bwas; if we consider it to be a goal, then it can be achieved by using a variety of bwos. we will choose to consider scroll up to be a goal because it can be achieved by manipulating one or more of a variety of different wocs. let us represent goals by using curly braces, so that scroll up is written {scroll up}. given this starting point, it is then clear that in microsoft windows the goal of {scroll up} can be reached by performing one of the following operations: 1. click on the scroll up icon at the top of the scroll bar; 2. slide (by dragging) the elevator on the scroll bar in the up direction; let us identify each of these operations by using two letter mnemonics in square brackets. thus we will let [su] stand for the bwo scroll up icon and [ve] stand for the bwo slide vertical elevator. using this nomenclature, the relationship between basic windows operations, basic windows actions and primitive user actions for these two methods of reaching the goal {scroll up} is as shown in figure 7. each method shown requires a different bwo and each bwo requires various combinations of puas. in the first method, the mouse is clicked upon the scroll up icon; in the second method, the vertical elevator is dragged upwards. f[su] «— < c> <— [scroll up icon] mva up j^ <_< d > <— [vertical elevator] mv ~ (x, y) * ~ (x1, y') ma figure 7: decomposition of the goal {scroll up} proceeding in this manner, we identify 26 basic window operations as listed in table 4. this list is clearly not exhaustive and is subject to addition and refinement through further investigation. the window object classes and the basic window operations required to manipulate window objects to perform tasks are listed in table 4. the basic window action or actions necessary to perform each operation are given in the last column of this table. in this column, alternative bwas for the same bwo are indicated by the i symbol. thus 37 [d]l[c] indicates that the desired bwo can be achieved by either dragging or by a single click. this particular alternative occurs for the bwo [so], where the operation can be performed by opening the menu and then dragging the mouse pointer to the required option in a single operation, or by clicking on a menu option of an already open menu. bwo close folder close window scroll left/right make active move object maximize window minimize window open folder open menu open window position cursor resize frame restore window bwo code [cf] [cw] [he] [ma] [mo] [mw] [nw] [of] [om] [ow] [pc] [rf] [rw] bwo scroll down select icon scroll left select option scroll right select text scroll up type text view directories scroll up/down execute icon execute option execute task bwo code [sd] [si] [si] [so] [sr] [st] [su] [tt] [vd] [ve] [xi] [xo] [xt] table 4: basic window operations different bwos can be performed on the same woc. for example, a program group icon can be moved [mo] or opened into a window [ow]. in this case, these different operations require different basic window actions: to perform [mo] a <d> must be executed; to perform [ow] a <cc> must be executed. but this is not always the case. for the program group icon, both [mo] and [rw] can be performed with the same basic window action, that of <d>. however, the action is performed in different ways. in order to move the group icon, the drag must be performed on the icon itself; in order to restore the program group window, the drag must be performed on the program group menu. note also that different bwos can be performed to achieve the same functional goal and obtain the same response from the gui. for example, it is possible to execute a program from the program manager by performing any one of 3 different operations: [xi], or the combined operations [si] + [so] or [so] + [tt] + [xt]. it is possible to execute a program from the file manager by performing the operation [xo], and in the case of an application such as microsoft office, it is also possible to execute a program by performing the operation [xt]. it is not altogether clear at this stage how different bwos should be selected and classified; this is still the subject of on-going research. given that we can identify a finite number of basic window operations, it becomes possible to record the sequences of such operations which are necessary to perform specific tasks. thus the sequence of operations of the file copy task described in steps 1 through 9 and depicted in figures 1 though 6 is 38 [xi][mw][of)[vd][mo][mo][ma][cw][mw], as detailed in table 1. this includes those operations required to return the system to its original state. in order to analyse the complexity of this task, it is necessary to know the initial and digram frequencies which dictate the structure of a windows dialogue. woe title bar close button minimize button maximize button restore button window window frame active menu item prog, group icon possible bwos [ma] [mo] [cw] [om] [so] [nw] [mw] [rw] [ma] [rf] [so] [si] [om] [so] [ow] [mo] [rw] [mw] required bwas <c> <d> <cc> <c> <d>kc> <c> <c> <c> <c> <d> <c> <c> <c> <d>kc> <cc> <d> <d> <d> woe program icon scroll up icon scroll down icon scroll left icon scroll right icon hor. elevator butn. ver. elevator butn. disk icon folder/file icon tool icon document text keyboard possible bwos [xi] [si] [mo] [su] [sd] [si] [sr] [he] [ve] [vd] [of] [cf] [mo] [xt] [pc] [st] [xo] [tt] required bwas <cc> <c> <d> <c> <c> <c> <c> <d> <d> <okcc> <okcc> <okcc> <d> <c> <c> <d> <cokc+c> <k> table 5: window object classes and corresponding operations and actions construction of a transition probability matrix for basic window operations comprehensive data has yet to be collected in order to determine initial and digram frequencies of each bwo. however, a pilot study of 462 bwos over several different tasks was made by the author logging his own operations during normal work sessions on a computer. the tasks included file management sessions and document production using microsoft word, excel and powerpoint. the study yielded the frequency information in tables 6 and 7. cf 1 cw 14 he 0 ma 3 mo 8 mw 5 nw 6 of 21 om 0 ow 2 pc 103 rf 1 rw 0 sd 48 si 1 si 3 so 25 sr 1 st 25 su 18 tt 78 vd 2 ve 2 xi 7 xo 5 xt 83 total 462 table 6: initial frequencies for basic window operations 39 the initial frequencies of occurrence of the various bwos are as shown in table 6. note that during these short sessions there are 3 operations that were never performed by the user, these being [he], [om] and [rw]. this may be a reflection of the user's windows language style as well as a reflection of the very small sample size. the digram transition frequencies are given in table 7. in this table, the frequency of occurrence of each bwo is dependent upon the previous bwo; thus each cell contains the frequency of occurrence with which a bwo j follows a bwo i in the sample sequence of 462 bwos. the data can be normalised to produce a table of transition probabilities by applying the condition for all i to each row. pi(j) i cf cw he ma mo mw nw of om ow pc rt rw sd si si so sr st su m vd ve xl xo xt cf 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 cw 0 2 0 2 0 1 0 0 0 0 0 0 0 0 0 0 1 0 0 1 0 0 0 1 0 6 he 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 ma 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 mo 0 0 0 0 2 0 0 1 0 1 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 2 mw 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 1 2 nw 0 3 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 0 0 1 of 0 0 0 1 0 1 0 2 0 0 0 1 0 9 0 0 1 0 0 3 1 0 0 1 0 1 om 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 ow 0 1 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 pc 0 0 0 0 0 1 0 0 0 0 22 0 0 3 0 0 6 0 1 1 56 0 0 1 0 13 i rt 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 rw 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 sd 0 0 0 0 0 0 0 10 0 0 0 0 0 23 0 0 1 0 0 2 5 0 1 0 3 3 si 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 si 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1 0 0 0 0 0 0 0 0 0 2 so 0 0 0 0 0 0 0 1 0 0 3 0 0 1 0 0 1 0 8 1 2 0 0 2 0 2 sr 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 st 0 0 0 0 1 0 0 1 0 0 0 0 0 2 0 0 6 0 0 1 5 0 0 0 1 8 su 1 0 0 0 0 0 0 0 0 0 0 0 0 5 0 0 1 0 0 5 1 0 0 0 0 6 m 0 0 0 0 0 0 0 0 0 1 62 0 0 1 0 0 3 0 3 1 0 0 0 0 0 7 vd 0 0 0 0 0 1 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 ve 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 xl 0 3 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 xo 0 0 0 0 0 0 0 3 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 1 xt 0 4 0 0 1 1 2 1 0 0 15 0 0 0 1 1 3 1 13 2 6 0 1 1 0 29 table 7: digram frequencies from a sample of 462 basic window operations the entropy equation we next require some method of quantifying the uncertainty in outcome at any point in a markov chain of bwos. for this we utilise a formula from communication theory due to 40 shannon and weaver (1949). consider a message transmitted using a message alphabet consisting of n different types of symbol, where at any given point in the message the symbol types have the probabilities of occurrence p\, p2,---pnshannon and weaver proposed that the uncertainty in occurrence of a given symbol type at any particular place in such a message is given by h(p\, p2,...p<d, where 1=1 and k is a positive constant. this form of h is identical to that of entropy as defined in statistical mechanics, where entropy is a measure of the disorder of a system which can be arranged in a large number of different ways. by considering a long sequence of n symbols, shannon and weaver also obtain an expression for what they call the entropy of an information source. in the first-order approximation of a message transmitted from such a source, there will be a high probability that there will be p\ n occurrences of each ith symbol. it follows that the probability of the message will be approximately: where there are n different possible symbols. \np= n^pi^pi ...(2) 1=1 substituting for h from equation (1) and re-arranging gives k\nph'—r ~<3) it is of interest to determine the maximum entropy that could be associated with the same source. this will occur when all the pi$ are equal and given by p\ = \/n. from equation (1) this is: hmax = the relative entropy of the information source is then given by: entropy of a windows screen display a modification of equation (1) has been utilised by bonsiepe (1968) to provide a measure of the complexity of a typographically designed page. bonsiepe classified a page of text and graphics in terms of objects of different sizes. objects were classified by width, height, and x and y spatial coordinates. by considering the proportion of objects in each class bonsiepe calculated an aggregate value for h; this was then interpreted as a measure of the complexity of a given page layout. the same method has also been 41 applied by tullis (1983) to a layout of text on a computer terminal and by comber and maltby (1994) to objects in a microsoft windows graphical user interface. in order to demonstrate the approach used by these workers, and to further clarify the nature of h, we will briefly consider how h might be calculated for a gui screen which consists of different classifications of on-screen objects. setting k equal to unity for convenience, the entropy of the screen display hd is given by: ifapi -..(6) 1=1 where c = number of different classes of screen objects, and where n, = number of objects in class i and n = total number of objects in all classes, pi now represents the proportion of screen objects in a particular class, rather than the probability of their occurrence. to demonstrate possible uses of equation (6), consider static displays of objects on a windows screen which are classified by size and shape. some examples are shown in figure 8. (a)fld = n 0 (d) ha = 1.099 nnnnnnnn \-\ nnn nnn ^ nnn d (b) ha = 0 1 ii 1 nnnn nnnn nnnn nnnn (e) tfd = 0.349 p~| (c) #d = 0.693 g n g $ar zicn — |im cu — (f) //d = 0.693 figure 8: hd values for screens of objects classified by size and shape 42 each of the screens (a) to (f) in this figure contains a number of objects of different classes. the hd values are determined in the following manner. screens (a) and (b) contain only 1 class of object and so we have c = 1. in both cases ni/n'is also 1 and so hd is zero: both of these screens thus have the same minimum entropy. in contrast screen (c) has 2 classes, with 1 object in each. in this case ha is given by -ln+ -in2 2 2 2 this is the maximum value of hd possible with 2 classes; thus the entropy of a 2 object screen such as screen (a) is considerably greater is than the entropy of a 20 object screen such as screen (b). clearly, if we add another class of object as in screen (d), the entropy will further increase, thus: #,=--ln+ -in=0.693d since it is the proportion of objects in each class that is the determining factor, it is clear that for screen (e) we will have some value of hd which is between the values of hd for screens (b) and (c), thus: 2 2 2 ^ the final example of screen (f) has equal numbers of objects in 2 classes; hd is therefore the same as for screen (c), i.e. 0.693. in the context of a windows screen, we can interpret these figures in several ways; they can be interpreted as a measure: (a) of the degree of disorder of the screen; (b) of the complexity of the screen in the context of different categories of information. (c) in the uncertainty in choice of a windows object class by a user; interpretation (a) leads to the conclusion that figures 8(a) and 8(b) have the same amount of disorder, and that figures 8(c) and 8(0 have the same amount of disorder. also, figure 8(c) is more disordered than figure 8(e), because there is a greater proportion of one object than the other. although these disorder rankings do not appear to be intuitively the case, we shall see shortly that these interpretations may have validity in the context of different classes of user. in terms of interpretation (b), a user may well see a screen consisting of a large number of uniform objects (such as icons) to be less complex than a screen consisting of a smaller number of objects of a variety of types (such as icons, windows, scroll bars and menus). comber has suggested that a novice user would view all objects in a windows screen as separate entities, thus seeing a complex screen, whereas an expert user would be more likely to group items of a similar type (such as options of a menu) into an entity which is seen as a single object. thus an expert user may see figure 8(f) as figure 8(c), which has the same h value. these assumptions need to be tested by collecting data on different classes of users. 43 interpretation (c) suggests that it might be profitable to look at how entropy changes along different task paths by comparing different options open to the user, such as whether to perform an [xt] or an [so] operation. we will now apply this method of determining display entropy to the file copy task summarised in table 1. the results are given in table 8. woe title bar close button minimize button maximize button restore button window window frame active menu item program icon scroll up icon scroll down icon scroll left icon scroll right icon horizontal elevator button vertical elevator button disc icon folder/file icon total number classes c = total number objects n display entropy hj = steps 1 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 1 2 0 2 2 2 2 2 0 1 2 8 1 1 1 1 1 1 1 3 36 16 65 1.45 3 2 2 2 1 1 1 2 8 0 1 1 1 1 1 1 3 36 16 64 1.75 4 2 2 2 1 1 1 2 8 0 1 1 1 1 1 1 3 39 16 67 1.12 5 3 3 3 2 1 2 3 8 0 0 0 0 0 0 0 6 4 10 35 1.72 6 3 3 3 2 1 2 3 8 0 1 0 0 0 0 1 6 35 12 68 1.41 7 3 2 3 2 1 2 3 8 0 1 1 1 1 1 1 3 39 16 75 1.31 8 3 3 3 2 1 2 3 8 0 1 0 0 0 0 1 6 37 12 70 1.56 9 2 2 2 1 1 1 2 8 0 1 1 1 1 1 1 3 39 16 59 0 .94 table 8: display entropies and number of objects in each class for steps 1 to 9 it is seen from these figures that the screen in step 1 (figure 1) possesses the minimum display entropy and that the screens in step 3 (figure 2) and 5 (figure 4) possess the maximum. we can liken the screen in step 1 to figure 8(a), in step 3 to figure 8(f) and in step 5 to figure 8(d). this latter case is of interest, since it highlights that hd is a measure of disorder or complexity of object class, the high value in this case being due to the additional window which has been opened. we might postulate that a novice user would find the screen in step 5 the most confusing, because it offers the most even distribution of different object classes, thus demanding of the user the maximum amount of knowledge about the interface. in contrast, the entropy of the screen in step 4 (figure 3) is significantly lower, even though it contains a greater number of different classes, because of the uneven distribution of different object classes. in this screen there is a predominance of one particular class, that of folder/file icon. we might deduce that there is thus more certainty as to its purpose. this deduction leads us to consider ways in which the entropy of basic windows operations can be determined. 44 entropy of basic windows operations the above determinations of screen entropy relate to static displays in a similar (but not identical) manner to the work of bonsiepe and tullis. in this paper we further extend the application of entropy to an analysis of the basic windows operations required to complete a specific task, and hence determine what we will call the operation entropies h0 associated with each screen. we do this by substituting the initial and digram probabilities from tables 6 and 7 into equation (6). this enables us to determine the entropy of a windows operation at any point in a markov chain of basic windows operations. consider one again the file copy task described in table 1 and depicted by the screens in figures 1 through 6. given the sequence of bwos necessary to complete this task, it is possible to calculate the entropy h0 of each operation by applying equation (6) and hence, since entropy is an additive quantity, to determine the total entropy of the task. the results of this determination are given in table 9, which also includes the values of the display entropies and the total the number of windows objects present in each step. the entropy and object number data is plotted against step number in figure 9. in this graph, the number of objects n has been normalised for comparison purposes. although there are some similarities in trend between all three parameters as we move from screen to screen, the correlation between n and each type of entropy is low, being r 0.66 for hd and r = 0.62 for h0. it is clear that entropy is measuring something more significant than simply the number of objects on a given screen. step 1 2 3 4 5 6 7 8 9 task [xi] [mw] [o f ] [vd] [mo] [mo]* [ma] [cw] [nw] number of objects n 2 65 64 67 35 68 75 70 59 display entropy hd 0 .45 .75 .12 .72 .41 .31 .56 0.94 operation entropy h<, 1.6138 3.1192 2.8832 3.2477 1.7918 3.3286 3.4012 2.6701 1.3108 table 9: h values for each screen display and bwo in a file copy task (at * file copy task is completed) the arrow in figure 9 indicates the last operation required to complete the task. the total entropies of the operation up to this point are hd = 7.45 and h0 = 15.93. the operations after this point are those required to return the system to its original state and the total entropies of the complete task are hd = 11.30 and h0 23.32. 45 l:(xp) 2:(xw) 3: (of) 4:(vd) 5: (mo) 6: (mo) 7: (mo) 8:(cw) 9:(nw) step n or bwo • ho figure 9: h versus bwo for a file copy task figure 10 highlights the differences between the 3 parameters by plotting the change in n, hd and h0 as a function of each transition as the user goes from screen to screen. the greatest differences occur in the transitions between the screens of steps 4 to 5 and steps 5 to 6, where hd and h0 change in opposite directions. in the transition from step 4 to 5, hd increases while h0 decreases; in the transition from step 5 to 6, these changes are reversed. once again the key screen is that of step 5: in terms of possible user operations, hd indicates that this is a complex screen; however, in terms of probable user operations, h0 indicates the opposite, because there is a high probability that having just opened a window, the user will then move it to a new position. perhaps hd is an appropriate measure of complexity for the novice user and h0 is an appropriate measure of complexity for the expert user. entropy or # objects screen transition figure 10: differences in n, h^ and ho as a function of screen transitions 46 we also note from both figures 9 and 10 that the file copy task undertaken both starts and finishes with low h values. the opening values of hd and h0 are a consequence of the fact that we chose the simplest possible type of opening screen. however, h0 decreases as the task approaches termination because the sequence of bwos is a predictable one. entropy of a windows language source we can use equation (4) to determine to the first-order the entropy of a windows language source by using the bwo frequencies given in table 6. the total number of operations performed in the pilot survey is n = 462. from equation (3) we can also calculate: =1083.4 1=1 substituting this and n = c = 26 into equation (5) yields a relative entropy of hr = 72.0%. discussion shannon and weaver (1949) classify communication problems into the following three categories: category a: the accuracy of transmission of symbols in a communication category b: the precision with which the transmitted symbols convey their desired meaning category c: the effectiveness with which the received meaning affects the receiver if we consider our problem to be mainly that of quantifying the operational complexity of a gui and of also providing a measure of its usability, then all three of these categories become relevant to gui design and to a windows dialogue. category a involves issues not only of software accuracy, but also of interface design. category b relates to the complexity and usability of an interface as seen by a user, quantities which are very dependent upon the user's interpretation of transmitted symbols (the windows object classes). category c relates to the efficiency of a windows dialogue, which is dependent upon the effectiveness of a user's response to these same transmitted symbols. to give an extreme example, if a user mistakes a save icon for a close icon, and the application does not warn the user that the file is unsaved, then it is possible to lose an unacceptable amount of work. in a discrete communication system, an information source is considered to select a desired sequence of objects (such as alphanumeric characters, picture or sound entities, or bits) in order to convey a particular message or amount of information. this sequence of objects is transmitted by the source to a receiver. the receiver translates the sequence into a form understandable by the information destination. if the destination is a human 47 being, then the human being attempts to interpret the final presentation of objects. in this context, the word information has a wider meaning than message content. in terms of an information source, it relates to what can be transmitted rather than to what actually is transmitted. for sequential objects in a markov chain, what can be transmitted is determined by the markov transition probabilities. thus when we determine an entropy value h0 at a particular point in a markov process, we are obtaining a measure of the uncertainty in the outcome of the next event in that process. this uncertainty can be a reflection of the freedom of choice of outcomes available to the information source, or it can be a measure of the predictability of outcomes as seen by the information destination. in the context of a windows language source, the entropy h0 at any given point in a markov chain of bwos can be interpreted as a measure of the: • uncertainty in choice by the novice user of the next bwo to transmit • freedom of choice by the expert user of the next bwo to transmit • complexity of the task being undertaken, as seen by the user • ability of the windows transceiver to predict the next bwo to be received we have seen that the relative entropy of a windows language source is hr = 72.0%. this figure can be interpreted as the amount of freedom experienced by the user in choosing a windows operation to perform when undertaking a particular task. the complement of this quantity is the redundancy of the windows destination, given by /? = 1 hr = 28.0%. this quantity is the proportion of the sequence of windows operations which is determined by the statistical structure of the dialogue, and not by the free choice of the user. it is interesting to compare the redundancy of a windows language source to the redundancy of the english language, which is about 50%. in other words, when composing a meaningful english sentence, an author has around 50% freedom in the choice of each successive word. weaver notes that if the redundancy of the english language were much higher, it would be difficult to construct crossword puzzles, whereas if it was much lower, then it would be possible to construct three-dimensional crossword puzzles. according to the data from the pilot study reported here, a windows user has about 72% of freedom in the choice of each successive bwo. whether we would expect a windows language to have a lower redundancy than a natural language such as english is an interesting question. in terms of freedom of choice of operations by the user, the redundancy of windows is a reflection of the versatility of the interface. we might speculate that low redundancy is desirable for an expert user, who would wish for the freedom to complete a task in a variety of ways. however, such freedom may be an added complication for the novice user, because it increases the uncertainty in the choice of each operation to be undertaken in order to complete a given task.. 48 conclusions a taxonomy of basic windows operations has been presented for the microsoft windows interface, enabling interface dialogue to be described as a markov process. a pilot study has been undertaken to obtain a stochastic matrix for a windows language source and this matrix has been used to calculate the entropy h of a specific windows task and of the language source itself. it has been found that the relative entropy of the source and its redundancy are hr = 72.0% and r = 28.0% respectively. various interpretations of these parameters have been briefly discussed in the context of a gui. it is concluded that additional research is necessary to further clarify the meaning of h and to develop methods by which the model can be applied to guis in general in order to quantify their operational complexity. references bonsiepe, g. a. (1968) a method of quantifying order in typographical design, journal of typographic research vol 2, pp 203-220. card, s. k. and moran, t. p. (1980) the key stroke-level model for user performance time with interactive systems, communications of the acm vol 23, pp 396-410. card, s. k., moran, t. p. and newell, a. (1983) the psychology of human-computer interaction, hillsdale, nj: erlbaum. comber, t and maltby, j. r. (1994) screen complexity and user design preferences in windows applications, proceedings of ozchi '94, chisig of the ergonomics society of australia annual conference, melbourne university (in press). galitz, w. o. (1989) handbook of screen format design, wellesley, ma: qed information sciences, inc. hartson, h. r., siochi, a. c. and hix, d. (1990) the uan: a user-oriented representation for direct manipulation interface designs, acm transactions on information systems, vol 8(3), pp 181-203. marcus, a. (1992) graphic design for electronic documents and user interfaces, new york: acm press / addison-wesley. mayhew, d. j. (1992) principles and guidelines in software user interface design, englewood cliffs, nj: erlbaum. moran, t. p. (1981) the command language grammar: a representation for the user interface of interactive computer systems, international journal of man-machine studies, vol 15, pp3-51. payne, s. j. and green, t. r. g. (1986) task-action grammars: a model of the mental representation of task languages, human-computer interaction, vol 2, pp 93-133. powell, j. e. (1990) designing user interfaces, san marcos, ca: microtend books. rivlin, c., lewis, r. and davies-cooper, r. (1990) (ed) guidelines for screen design, oxford: the university press. shannon, c. e. and weaver, w. (1949) the mathematical theory of communication, urbana, illinois: university of illinois press. schneiderman, b. (1992) designing the user interface: strategies for effective humancomputer interaction, reading, ma: addison-wesley. tullis, t.s. (1983) the formatting of alphanumeric displays: a review and analysis. human factors, vol 25(6): pp 557-.6v9 49 microsoft word ajis special 2002.doc ajis special issue december 2002 23 ownership as an issue in data and information sharing: a philosophically based view dennis hart school of business and information management australian national university dennis.hart@anu.edu.au abstract it has long been an aim of information management and information systems development to enable more effective and efficient data and information sharing within organisations. a commonplace assertion has been that data and information belong, or should belong, to the organisation as a whole as opposed to any individual or stakeholder within it. nevertheless, despite the potential benefits of data and information sharing within organisations, efforts to achieve it have typically run into more difficulty than expected and have frequently been less successful than the technological capabilities would, at least prima facie, allow. this paper is based on the proposition that perceptions of ownership can have an important influence on data and information sharing behaviour, and explores philosophical theories of ownership and property with the aim of better understanding the origins of such behaviour. it is further proposed that what are here called “implicit” theories of information ownership on the part of different individuals or parties within an organisation can lead to varying perceptions as to who is the legitimate owner of particular data or information, and that this view is illuminating of the difficulties that have often been experienced in trying to achieve effective organisational data and information sharing. introduction many modern organisations are critically dependent on their data and information resources for effective and efficient operation. for example, beynon-davies (2002, p435) says “information is a critical resource for organisations, particularly in modern information societies and economies” and consequently, as applegate, austin and mcfarlan (2003, p1) note, “it has become a strategic part of most businesses”. reflecting the increasingly critical and strategic nature of data and information, one of the main aims of information management and systems development has been to “ensure that data [and information] with the appropriate quality characteristics … can be delivered to the right person at the right time in the right form to meet that individual’s needs” (stair & reynolds, 1998, p67). but such availability of course implies a willingness to share on the part of those who have the data or information concerned. according to nolan’s (1990) stages of growth model, this is expected to occur in the fifth of the six stages, which is named “data administration” and which is characterised by, among other things, “shared data”. despite the claimed benefits of sharing data and information in organisations, and the undoubted and ever increasing capabilities of information technologies to enable it, such sharing evidently remains remarkably problematic. for example, davenport, et al (1992) say that “the rhetoric and technology of information management have far out-paced the ability of people to understand and agree on what information they need and then to share it [so] the information-based organisation is largely a fantasy” and, arguably, the situation has not changed much since. for example, kendall & kendall (2002, p73), discussing the management of e-commerce projects, say “organisational politics can come into play, because often units feel protective of the data they generate and do not understand the need to share them across the organisation”. various explanations have been put forward to account for these difficulties in achieving organisational data and information sharing, but one issue that, although recognised early (keen, 1981), has received relatively little attention is that of ownership. data and information ownership the focus of this paper is organizational data and information sharing behaviour and the role issues of ownership play in such behaviour. accordingly, it is the ownership perceptions of involved parties rather than any formal allocations of “ownership” by external agents or legal issues relating to ownership that are of primary interest here because it seems evident that it is internally felt perceptions of ownership that are likely to drive behaviour at a more fundamental level than any externally or legally imposed ownership might do. while data and information ownership have been recognised as an issue, particularly with respect to sharing behaviour in an organisational information management and systems development context (e.g. hart, 1999, hart & whymark, 1999; koh & watson, 1998; allen & wilson, 1996; van alstyne, et al, 1995; wigan, 1992), there has apparently been little, if any, attempt to develop a substantive theoretical understanding of the reasons why this should be so. constant et al (1994) have proposed a theory of information sharing based on social interdependency theory in which they acknowledge the role of “norms of organisational ownership”, but do not investigate the foundations or origins of such norms other than to suggest that they arise from work experience and training, and are affected by individual self interest as well as attitudes to reciprocity of behaviour. moreover, they apparently assume that there is one such ownership norm per organisation and that it influences sharing behaviour generally. similarly, jarvenpaa and staples (2000), who base their work on that of constant et al, take organisational norms relating to information ownership as one of their independent variables without attempting to explicate its origins. ajis special issue december 2002 24 this paper explores theories of ownership and the closely related concept of property, in an historical and philosophical sense, in an attempt to understand where the foundations of norms of organisational ownership lie. this exploration is intended to fill in some of the gaps identified above, and it is further contended that the theories of ownership and property investigated have the potential to underpin a better understanding of data and information sharing phenomena of interest and importance in the discipline of information systems. the paper extends the work of constant et al (1994) by investigating the basis of the organisational ownership norms that they identify as a factor influencing information sharing. in outline, it is argued here that individuals or groups within an organisation hold implicit internal theories of ownership that tell them who owns what and why. these theories could then differ and conflict in their outcomes when applied to the question of who owns certain data or information. furthermore, data and information are argued to be somewhat different in character from other items that we are used to regarding as being owned, and this adds further complexity and confusion to the issue of who has legitimate ownership of them. lastly, the implications of the argument for organisational information sharing and conflict over ownership are briefly discussed. attentive readers may have noticed that no distinction has so far been made between data and information. it is, however, usual to make such a distinction along the lines that data is a set of uninterpreted symbols whereas information is “data interpreted in some meaningful context” (beynon-davies, 2002, p 4). moreover, it is now also common to differentiate, as separate concepts, knowledge (e.g. stair & reynolds, 1998, p 5-6) and increasingly wisdom (e.g. liang, 1996) in addition to data and information. but while these distinctions might be valid they will not in most cases be made here because they are, in large part, inessential to the argument to be put forward. the concepts of ownership and property the first step in the argument is to consider what it means to “own” something or the closely related question of what it means for something to be “property” (in the sense of private property). there are various views and often differences of detail, but what all have in common is that ownership entails rights (and responsibilities too, but we ignore these) and property is that over which these rights are exercised. the next question, then, is what rights it is that ownership entails. according to honoré (1961), ownership is “the right to possess, viz. to have exclusive control of a thing, or to have such control as the nature of the thing admits … and the right to remain in control, viz. the claim that others should not, without permission, interfere”. but while any definition of ownership would likely have similar concepts at its core, it is still the case that “all attempts … to provide a univocal explication of the concept of ownership, applicable within all societies and to all resources, have failed” (harris, 1996, p5). even so, as harris also notes, while the concepts and foundations of ownership and property – that which is or can be owned – might be theoretically elusive, we all still seem to have a well developed intuitive sense of what they mean. and for the most part we operate successfully with this intuitive sense in our ordinary everyday dealings with one another. nevertheless, we shall here follow honoré and, while recognizing that ownership has been argued to encompass other significant rights as well as responsibilities, at the very least it entails as its foundation: • the right to use whatever is owned • the right to control the use of whatever is owned • the right to remain in control of whatever is owned, without interference from others. indeed, as carter (1989, p127) says, also quoting honoré, these characteristics are “perhaps the most central aspect of our conception of private property” and logically therefore, of ownership also. now, having briefly considered what ownership and property mean, the question of on what basis ownership is to be assigned arises. that is, on what grounds does ownership of some item of property reside in one set of hands as opposed to another? the attempt to answer this question has occupied philosophers as well as others such as legal theorists over a long period, resulting in a range of different proposals as to the basis of the concepts of ownership and property and it is to the major ones of these that we now turn. theories of ownership and property theories of property and ownership have a long history dating from at least ancient greece (ryan, 1987; schlatter, 1951) and various philosophical foundations have been proposed as underlying these concepts (becker, 1977; grunebaum, 1987). loosely based on the classification of becker (1977), the major philosophical bases are, with their originators and principal proponent(s): • first occupancy theory (immanuel kant) • labour theory (john locke) • utility theory (jeremy bentham, john stuart mill) • libertarian theory (robert nozick, john rawls) • personality theory (georg wilhelm friedrich hegel) we now give a very brief thumbnail sketch of each of these foundations in turn. ajis special issue december 2002 25 first occupancy theory as the name suggests, first occupancy theory says that ownership is established on the basis of the principle that the initial appropriator of a thing is thereby entitled to it. any child who ever said: “i had it first, so it’s mine!” understands the fundamentals of this theory. there are other issues, of course. in particular, whatever it is that is claimed has first to be unowned; the “occupation” has to be actual as opposed to notional (whatever “actual occupation” may in fact mean); and the bounds of where “occupation” begins and ends need to be clear (becker, 1977). but despite such points of detail and other complexities that are beyond the scope of this discussion, the basis of the assignment of ownership according to this theory is apparent. being first to possess or “occupy” whatever the entity in question is, is the basis for its ownership. labour theory the labour theory of john locke can be considered a variant of the first occupancy theory outlined above (grunebaum, 1987). however, a critical difference is that in the labour theory it is the adding of value through the “mixing” of one’s labour with something that confers ownership. the basic premise underlying this theory is that one owns one’s person and labour and, therefore, when that labour is applied to something the thing laboured upon thereby moves from an unowned state to an owned one. as locke says: “the labour that was mine, removing them [the items claimed] out of that common state they were in, hath fixed my property in them” (locke, 1689). but, according to locke, the labour-based claim to ownership has nevertheless to be subject to the requirement that there be “enough, and as good left in common for others” (locke, 1689). again, there are complexities and problems relating to how much labour and/or added value is required to confer ownership and how far that ownership extends but, as with the first occupancy theory, the fundamental basis of ownership according to this theory is obvious: one owns the results of one’s labour. utility theory utility theories of ownership are founded on the notion that property should be assigned in such a way as to maximise some quantifiable and, at least in principle, calculable “good” (the utility). the same basic principle underlies consequentialist theories of ethics – notably utilitarianism – and a considerable portion of traditional decision theory. differences arise depending on how utility is defined. is it an economic concept or a noneconomic one such as happiness, or some combination? while, as becker (1977, p57) notes, the economic view has traditionally taken a higher profile in discussions of utility theory, this is neither necessary nor logical since economic utility is simply a special case of a more general concept of utility. but in any case, the fundamental essence of the theory is the same no matter how utility is conceived: ownership is to be allocated in such a way as to achieve maximum benefit for all involved, whichever way this might be defined and calculated. libertarian theory libertarian theories of ownership are founded on the idea that “a general right to private property is in some sense constitutive of freedom” (waldron, 1988, p290). indeed, it has been argued that it “is plausible to construe all rights as property rights” [emphasis in original] (narveson, 1988, p66). people must, the argument goes, be allowed to establish ownership, though only by just means (nozick, 1974, p178 et seq), over whatever they want or need because to prevent them from doing so would infringe their liberty in the sense of being able to function effectively as autonomous agents. liberalism derives from locke and comes in two main varieties – classical liberalism or libertarianism and welfare liberalism (sterba, 1999). the difference between these is that, in the former, liberty is conceived in terms of the absence of coercions that would restrict one’s freedom of action and in particular, in our context, to establish ownership over whatever one requires, albeit without infringing the rights of others in so doing. nozick’s liberalism is of this kind. welfare liberalism on the other hand asserts that in addition to avoiding actions that would restrict an agent’s freedom of action, it is also required to act positively to help them if their ability to act as they wish is below some acceptable minimum. in relation to property, this implies some redistribution in favour of the most disadvantaged may be necessary and there is thus some affinity between this version of liberalism and utilitarian approaches. rawls’ liberalism is of this type (see rawls, 1971, chapter v). nevertheless, the fundamental principle is clearly that property rights are justified by their close and intimate contribution to the liberty of agents to act autonomously, and are to be determined accordingly. personality theory in his book the philosophy of right hegel (1821) put forward a theory of ownership and property based on personality. in his words: “a person has as his substantive end the right of putting his will into any and every thing and thereby making it his, because it has no such end in itself and derives its destiny and soul from his will. this is the absolute right of appropriation which man has over all ‘things’”. this theory means, as ryan ajis special issue december 2002 26 (1984, p122) explains, that it is only humans who have wills and purposes whereas the other stuff of nature is “blankly material and, as it were, ‘unoccupied’”. therefore, the argument goes, a human who invests an object with a purpose through their will to use it for the achievement of some end thereby establishes ownership over it: it becomes their property as a result. there is apparently some affinity here with the first occupancy theory in that ownership presumably resides with the first human to invest the “unoccupied” object with such a purpose. however, there is also an important difference in that the first occupancy theory says nothing of will and purpose whereas this is prime in hegel’s account. the essential feature of this theory is, then, that “propertyowning is [important] since it is only through owning and controlling property that [one] can embody his [or her] will in external objects” (waldron, 1988, p377-8). it is the agent that invests an object with a purpose who is that object’s legitimate owner. as a simple example highlighting three of the five theories above, consider the case of three people walking together. the first picks up a stick lying beside the path, subsequently just swinging it for a while in rhythm with the pace of the walk before passing it to the second who, when they sit for a rest, proceeds to idly whittle it until it has a sharp point. the third then says “hey, that would make a really good stake for my tomato plant at home!”. according to the first occupancy theory the stick should be the property of the first person whereas the labour theory would accord ownership to the second, and finally the personality theory would vest its ownership with the third. implicit theories of ownership it was noted above that despite the lack of theoretical agreement as to what underlies the concepts of ownership and property, we nevertheless have an intuitive understanding of what these mean in practice. in other words, it could be said that we each have developed an implicit internal theory (or more probably an amalgam of theories) of ownership. for example, if someone unversed in philosophical theories of ownership is asked on what basis that concept rests then the answer will in all likelihood contain aspects of several of the theories described above. indeed, the author has made just such an informal enquiry of a friend and received a reply that contained, in the first sentence, essential elements of both the first occupancy and labour theories. now, as noted above in the quote from harris (1996), no single theory of ownership turns out to be completely satisfactory in the sense of being able to correctly account for our intuitive notions of ownership across all cases. the implication is that in resolving an issue of ownership we apply different theories, or at least aspects of them, according to context and circumstances. but what context and circumstances? first, an important aspect will be the perspective from which we view the ownership issue. what relationship do we have to the item for which ownership is being decided? more specifically, for example, were we the first to come across it? did we create it? do we have it in our possession? second, what is the nature of the item for which ownership is being decided? is it tangible or intangible? are there existing and accepted precedents and conventions applying to the kind of object with which we are dealing that can guide and help in resolving ownership issues? since the target of interest to this paper is ownership as it affects data and information sharing, it is now necessary to turn to these, viewed from what might be called a “property” perspective. that is, in what ways, if any, are data and information similar to other things to which we normally ascribe ownership and, more particularly, how might the different ownership theories described above apply to them? data and information as property can data and information be owned? it is often assumed so (e.g. brynjolfsson, 1994), but it remains to ask in what ways are they are like and unlike other more traditional forms of property such as land and personal effects. tackling differences first, unlike most “normal” items of property, data and information are typically characterised by: • intangibility – while data and information may be, and often are, instantiated in some tangible form such as hardcopy reports or computerised databases, it remains true that fundamentally they are intangible. it may be, for example, that i write the telephone numbers of my friends in a personal telephone book or store them in a personal organiser, but this is not an essential feature of this data or information. i could, if i’m good enough, simply commit them to memory and in such a case they are still the same data/information. • non-exclusivity – unlike a physical object that can only be used (at any particular time) by its owner or someone to whom it has been temporarily loaned, use of data and information by one does not prevent its simultaneous use by another. • usage tolerance – data and information do not (normally) deteriorate through use. there are, however, important exceptions to this. for example, insider knowledge of business performance data that will affect stock values cannot be used, at least by an arbitrary number of others, and remain completely unaffected by such use (hettinger, 1989). ajis special issue december 2002 27 • age susceptibility – this characteristic is complementary to the usage tolerance aspect in that data and information often do deteriorate with age even if they have never been used. this is, again, to be contrasted with usual items of property for which this is not normally the case. to the extent, then, that data and information exhibit these characteristics they are unlike usual items of property and therefore it may be expected that the concept of ownership should not apply to them, at least not in the same way that it applies to other items. however, in other important ways data and information do exhibit characteristics that permit the different theories of ownership to be applied to them. in the first instance, it is possible for data and information to be discovered. then, according to the first occupancy theory, being the discoverer confers ownership of the data or information that has been uncovered. second, effort and resources are often expended in collecting data and producing information and therefore, according to the account of the labour theory, this should confer ownership of the result upon the producer. third, the value of data and information is frequently if not always dependent on to whom and when it is available and the utility theory could therefore be applied in order to determine ownership from this value-based perspective. fourth, data and information are often important enablers of personal or organisational activities that would not be possible in their absence. that is, they contribute to the liberty of the agent to pursue activities in an autonomous way, and therefore the libertarian theory could be applicable and used to determine ownership on this basis. lastly, a distinction has for a long time been made in the information systems literature between data and information on the basis of purpose. as just one recent example: “information is data that has been refined and organised by processing and purposeful intelligence. the latter, purposeful intelligence, is crucial to the definition – people provide the purpose and the intelligence that produces true information” [emphasis added] (whitten, et al, 2001). it is clear that information at least (as opposed to data, which is considered its raw material) has purposefulness as one of its characteristics and for this reason the personality theory of ownership should be applicable to it. to sum up: data and information have some characteristics that distinguish them from what we normally regard as items of property that can be legitimately owned. nevertheless, there are also important aspects of them that are closely related to the various theories of ownership that emerge from the philosophical literature, and which allow the meaningful application of those theories in determining their ownership. therefore, given that we each carry within us an implicit theory of ownership built around these philosophical foundations, that theory could in many respects be justifiably applied to considering who owns data and information. ownership and data and information sharing in thinking about the implications of ownership perceptions for organizational data and information sharing it is perhaps necessary to first consider whose ownership perceptions are relevant. ultimately organizations consist of individuals. however, in discussions of organizational behaviour it is groups as well as individuals that play a prominent role. what, then, of group versus individual perceptions of ownership? indeed, is it meaningful to speak of a “group perception” at all? but this is not the place to enter such debates. it will instead be assumed in what follows that individual ownership perceptions are key, and where it might be meaningful to speak of a group’s perception then this can be equated to a common set of individual perceptions across all of the members of that group. we have seen that there are reasons for regarding data and information as being capable of being owned. however, depending on the (implicit) theory upon which one bases that ownership assessment as well as the perspective from which it is viewed, it may well be that a particular piece of data or information will turn out to have different owners. for example, a particular organisational group may have invested effort and resources in acquiring or generating certain data or information and therefore – on the basis of the first occupancy and labour theories – lay claim to it as its own. on the other hand, another organisational group (management, for example) might rightly see that the data or information concerned would provide them with substantial benefit for their purposes, and therefore – by the utility and personality and possibly libertarian theories – consider that its ownership legitimately resides with them. in such a case, what would be the effect? to see this, consider again what ownership entails. it has already been seen that ownership confers certain rights; namely, and at the very least, the right to use, the right to control use, and the right to remain in control without interference from others. therefore, where two or more different parties each see themselves as legitimate owners of particular data or information, albeit based on different underlying theories, it is natural to expect dispute and conflict concerning who should exercise these rights. in the example case above the original producers of the data or information – its actual possessors – would be expected to resist attempts by external parties to access and use it without their explicit consent. moreover, they would also be expected to fight any attempts to wrest control of access or use from them, as well as viewing such attempts either to use or control use without their permission as unjustified interference with their ownership rights. on the other side, the other party would see such a response as entirely illegitimate, uncooperative and obstructive given their understanding that they are the valid owners of the data or information concerned. in their turn, they would not only seek to use the data or information as they saw fit (according to ajis special issue december 2002 28 them they own it, after all), but would also seek to take control of it and regard attempts to block them in this as unjustified interference with their ownership rights. conflict would be a natural outcome. an analysis of this type could be applicable to, for example, the well-known fis case study of markus (1983) in which she describes the battle between divisional and corporate accountants over divisional finance data. while markus describes the fis case in power and political terms, the issue of ownership is nevertheless alluded to in her description. she says, for example: “prior to fis … divisions retained control of their own data” whereas “after fis … the divisional accountants still had to enter the data, but they no longer ‘owned’ it” [emphasis added]. it is the contention of this paper that just such disputed ownership issues of the kind illustrated above may be a fundamental element underlying conflict, power and political situations of the type described in markus’ paper. how data and information sharing can occur if the analysis presented so far has any validity, it is reasonable to ask how it might account for the fact that data and information sharing occurs at all, under any circumstances. manifestly, there are many situations in organisations where data and information are shared and this would seem to be at best incompatible with, and at worst destroy, the argument above. but there are at least two and possibly three responses to this. first, the party that is the producer or possessor of data or information may deem itself, according to their implicit ownership theory, to be the legitimate owner of it but nevertheless consent to share it with various other parties. after all, one of the rights entailed in ownership is to control use of whatever is owned, and this includes allowing others to use it in circumstances considered appropriate by the owner. second, even though one party may be the originator or possessor of certain data or information, their implicit ownership theory may tell them that its legitimate owner is in fact a different party and, therefore, not only should that data or information be shared with this other party, they should also cede the control the use of it to them. in both cases, the implicit ownership theories to which the involved parties subscribe do not serve to interfere with data and information sharing, but rather to further it. a remaining possibility is that the implicit ownership theory of no party in the organisation leads it to lay claim to the data or information concerned. that is, everyone decides that ownership of it lies elsewhere and it is therefore someone else’s prerogative to use it, and control its use. the data or information consequently remains unclaimed and uncontrolled, so anyone who wishes to can access and use it. it is, therefore, in a sense “shared” although in this case the term lacks the implication of intentionality it normally carries, that there is some (owning) actor actually initiating and doing the sharing. conclusion achieving data and information sharing in modern organizations has proven notably more difficult than expected despite the undoubted existence of technological capabilities to make it happen. one issue underlying these difficulties, it has been argued, is ownership perceptions on the part of organizational stakeholders (whose data or information is it anyway, and who therefore has the right to control it?). the original foundations of ideas about ownership and property lie in considerations relating to land and physical objects and their long history of application in this domain has meant that there is now extensive intersubjective agreement, at least intuitively and taking due note of cultural context, as to how they work and when they apply. however, when intangible goods such as data and information, as captured and instantiated in modern computerized systems, are considered there is a much more abbreviated history and evidently less agreement as to who should own what, and on what grounds. it has been argued in this paper that we carry “implicit” theories of ownership that contain elements from the various different philosophical bases that have been proposed as underlying the concepts of ownership and property in general, and it is on the basis of these implicit theories that we decide who legitimate owners are. moreover, data and information, while different in a number of respects from traditional items of property, nevertheless exhibit important characteristics that make ownership theories applicable to them. now the crux of the argument is that different parties within an organisation may, on the basis of their implicit theories of ownership and the particular characteristics of the data or information concerned as it relates to them, deem that ownership of it resides in different places. ownership carries rights, notably those of use, control of use, and immunity from interference. therefore, if different parties in the organisation each lay claim to ownership of the same data or information (albeit based on different grounds and perspectives) then conflict may arise over the exercise of these rights. moreover, it has been argued that such differences of opinion over where ownership legitimately lies will potentially have important effects on sharing behaviour, although not necessarily so. the contribution of this paper is that it illuminates the origin of organizational ownership norms with respect to data and information by applying philosophical theories of ownership more completely than has been done to date within the information systems literature. in doing so, it has drawn out more fully some reasons for information and data sharing behaviour, or the lack of it, in an organisational and information systems context. ajis special issue december 2002 29 references allen, d. k. & wilson, t. d. (1996) “information strategies in uk higher education institutions”, int. j. of information management, 16, pp 239-251 applegate, l. m., austin, r. d. & mcfarlan, f. w. (2003) corporate information strategy and management, 6th ed, boston: mcgraw-hill irwin becker, l.c. (1977) property rights: philosophic foundations, routledge & kegan paul beynon-davies, p. (2002) information systems: an introduction to informatics in organsations, palgrave brynjolfsson, e. (1994) “information assets, technology, and organization”, management science, 40, pp 1645-1662 carter, a. (1989) the philosophical foundations of property rights, harvester wheatsheaf constant, d., kiesler, s. & sproull, l. (1994) “what’s mine is ours, or is it? a study of attitudes about information sharing”, information systems research, 5, pp 400-421 davenport, t. h., eccles, r. g. & prusak, l. (1992) “information politics”, sloan management review, pp 53-65 grunebaum, j. o. (1987) private ownership, routledge & kegan paul harris, j. w. (1996) property and justice, clarendon hart, d. n. (1999) ownership, organizational politics and information systems, phd thesis (unpublished), unsw – australian defence force academy; http://www.library.unsw.edu.au/~thesis/adt-adfa/public/adt-adfa20020619.095737/ hart, d. n. & whymark g. k. (1999) “re-arranging the chairs: tackling the ownership and political aspects of organizational transition and information systems support” proceedings of the 1st international conference on enterprise information systems, pp 453-461. hegel, g. w. f. (1821) the philosophy of right, trans t. m. knox, reprinted in hutchins, r. m. (1952) great books of the western world, vol 46, encyclopædia britannica inc hettinger, e. c. (1989) “justifying intellectual property”, philosophy and public affairs, 18, pp 31-52 honoré, a. m. (1961) “ownership” in a. guest (ed) oxford essays in jurisprudence, oup jarvenpaa, s. l. & staples, d. s. (2000) “the use of collaborative electronic media for information sharing: an exploratory study of determinants”, strategic information systems, 9, pp 129-154 keen, p. g. w. (1981) “information systems and organizational change”, communications of the acm, 24, pp 24-33 kendall, k. e. & kendall, j. e. (2002) systems analysis and design, 5th ed, prentice-hall koh, c. e. & watson, h. j. (1998) “data management in executive information systems”, information & management, 33, 301-312 liang, t. y. (1996) “the basic entity model: a theoretical model of information processing, decision making and information systems” information processing and management, vol 32, no 4, pp 477-487 locke, j. (1689) “of property”, in c. b. macpherson (ed), (1978) property: mainstream and critical positions, university of toronto press markus, m. l. (1983) “power, politics, and mis implementation”, communications of the acm, 26, pp 430444 narveson, j. (1988) the libertarian idea, temple university press nolan, r. l. (1990) “managing the crisis in data processing” in the information infrastructure, harvard business review nozick, r. (1974) anarchy, state and utopia, basil blackwell rawls, j. (1971) a theory of justice, oup ryan, a. (1984) property and political theory, basil blackwell ryan, a. (1987) property, open university press schlatter, r. (1951) private property: the history of an idea, george allen & unwin stair, r.m. & reynolds, g.w. (1998) principles of information systems: a managerial approach, 3rd ed, course technology-itp sterba, j. p. (1999) “political philosophy” in r. audi (ed), the cambridge dictionary of philosophy, 2nd ed, cambridge university press van alstyne, m., brynjolfsson, e. & madnick, s. (1995) “why not one big database? principles for data ownership”, decision support systems, 15, pp 267-284 waldron, j. (1988) the right to private property, oup whitten, j. l., bentley, l. d. & dittman, k. c. (2001) systems analysis and design methods, 5th ed, mcgraw-hill wigan, m. r. (1992) “data ownership” in r. clarke & j. cameron (eds), managing information technology’s organisational impact, ii, north-holland ajis vol. 4 no. 2 may 1997 the effects of normalisation on the satisfaction of novice end-user querying databases' conrad benedict paul bowen colin ferguson fiona rohde department of commerce the university of queensland, qld, 4072 e -mail: rohde@commerce.uq.edu au abstract this paper reports the results of an experiment that investigated the effects different structural characteristics of relational databases have on information satisfaction of end-users querying databases. the results show that unnormalised tables adversely affect end-user satisfaction. the adverse affect on end-user satisfaction is attributable primarily to the use of non atomic data. in this study, the affect on end user satisfaction of repeating fields was not significant. the study contributes to the further development of theories of individual adjustment to information technology in the workplace by alerting organisations and, in particular, database designers to the ways in which the structural characteristics of relational databases may affect end-user satisfaction. more importantly, the results suggest that database designers need to clearly identify the domains for each item appearing in their databases. these issues are of increasing importance because of the growth in the amount of data available to end-users in relational databases. introduction the complexity and pervasiveness of information systems based on relational database management software continues to grow. end-users are increasingly expected to query these systems to obtain the information they need to perform their jobs. in addition, relational database technology has enhanced the ability of end-users to (a) design their own database structures, and (b) maintain their own data (doll and torkzadeh, 1993). these data have value only to the extent end-users can obtain the information they need from these databases. the growth in end-user computing warrants an investigation of factors that improve end-user performance and satisfaction because, for example, end-user information satisfaction (uis) is associated with systems success (doll & torkzadeh, 1988b). an understanding of how these factors affect uis enables designers and managers to enhance system effectiveness through improved user education and training. data normalisation and, through it, task complexity are the foci of this study. earlier research found that the perceived ease-of-use of a system is an important component of uis (doll & torkzadeh, 1988b). perceived ease-of-use of a database is diminished when the complexity of tasks increases. therefore, through its effect on perceived ease-of-use, task complexity affects uis. in turn, task complexity associated with the use of databases is affected by the level of normalisation. this study focuses on these relationships (as depicted in figure i ) and investigates how different levels of normalisation and, therefore, task complexity affect novice end-users' satisfaction when querying1 relational databases. the authors gratefully acknowledge the assistance of the editor and the two anonymous referees for their comments on earlier drafts of this paper. the authors also gratefully acknowledge the financial assistance provided by the university of queensland via a new staff grant. end users can interact with the database through the use of a query language. queries can include insen, delete, update and data selection statements. this research limits the use of the word query/querying to include only data selection statements. ajis vol. 4 no. 2 may 1997 level of normalisation task complexity perceived ease-of-use end-user information satisfaction (uis) figure 1: normalisation, task complexity and end-user information satisfaction although normalisation, originally outlined by codd (1970) when he introduced the relational model, has been the focus of earlier research (see, for example, maciaszek, 1990), little empirical research has examined the effects of various levels of normalisation on uis. through an experiment, this study examines uis associated with querying databases exhibiting three different data structures: not normalised ("'nf), first normal form (inf), and third normal form (3nf). the paper proceeds as follows. the next section develops the theory on which the experiment was based and describes the key research constructs. the next two sections present example queries and develop the research hypotheses. the subsequent section outlines the research method. results are then reported and analysed. the paper concludes with a discussion of the implications of the study's findings, its limitations, and suggestions for future research. theory development research model the model depicted in figure 1 shows that for databases, user information satisfaction is influenced by the level of normalisation through its effect on the level of task complexity. the following subsections discuss end-users and the components of the model. novice end-users end-user computing is the direct interaction with application software by managerial, professional, and operating level personnel in user departments (doll and torkzadeh, 1993). end-users are often novices and frequently reluctant adopters of information systems technology (rockhart and flannery, 1983; doll and torkzadeh, 1993). some recent research (see, for example, mackay & elam, 1992; kieras & bovair, 1984; soloway et al, 1982) has focused on novice end-users. overall, these studies suggest that end-users are not aware of the analysis required to solve complex problems, and that this lack of awareness is heightened in novice end-users. novice end-users are likely to perform poorly and be dissatisfied in complex task environments. improving their performance and increasing their levels of satisfaction are likely to lead to more effective and efficient use of information systems resources in organisations. task complexity a complex task has three primary properties: the number of dimensions of information requiring attention (information load), the number of alternatives associated with each load (information diversity), and the degree of uncertainty involved (rate of information change) (schroder et al., 1971). as the magnitude of each property increases, so does task complexity (campbell, 1988). this implies that a change in the structural characteristics of databases changes the magnitude of the dimensions of task complexity. therefore, changes in the level of normalisation in a database structure change the level of task complexity for users of that database. normalisation normalisation is a simple elimination procedure (codd, 1970). codd initially proposed three normal forms that he called first, second, and third normal form. other normal forms include boyce-codd, fourth, and fifth ajis vol. 4 no. 2 may 1997 normal forms2. each level of normalisation is generally considered more desirable than the levels below it (date, 1986). increasing the level of normalisation changes specific structural characteristics, e.g., atomicity, repeating fields, and fragmentation (see figure 2). for a relation to be in inf, "at every row-and-column position within the relation, there is always exactly one data value, never a set of multiple values" (data, 1990, p 378). therefore, when the level of normalisation changes from ->nf to inf, all repeating fields are eliminated and non-atomic3 data items are decomposed into multiple atomic elements. when the level of normalisation increases from 1 nf to 3nf, all partial and transitive dependencies (two examples of functional dependencies) are eliminated. the removal of repeating fields, non-atomic data items, and these two types of functional dependencies, however, results in an increase in the fragmentation of the database. the existence or elimination of these structural characteristics is likely to affect the complexity of the database tasks and, consequently, the level of perceived ease-of-use of the database system. >n-at icity repeati ws .avh 1h f«3 o> p. e ields inf 2nf 3nf • nf inf 2nf 3nf fragm^it ition « i<4-l inf 2nf 3nf figure 2: structural characteristics and anomalies of normalisation example questions and the resulting queries this section presents two of the questions the experimental participants were asked to answer and provides queries that answer these questions using each of the three data structures. the differences in complexity between these queries relative to the three data structures are then discussed. the experiment required participants to obtain information about a furniture manufacturer's just-in-time operations. the participants queried a database that contained data about raw materials inventory items including receipts, issues, and current amounts on hand. see appendix b for the data structures and appendix c for details of the experimental setting. for readability, sql keywords appear in upper case letters; table names appear in small capital letters; and attribute names appear in lower case, italicized letters. find vendors with discount rates > 5% or discount days > 45. ->nf query select distinct vno, vname, terms from itema where to_number(substring(rcrm5,l,instr(rerffw,'/')-l) > 5 or to_number(substring(terms,instr(term.r1'/')+l,2) > 45; because the experiment only used ->nf, 1 nf, and 3nf, the definitions of boyce-codd normal form (bcnf), fourth normal form (4nf), and fifth normal form (5nf) are not presented in this paper. the atomicity constraint requires that each individual data value be atomic in that they have no internal structure (i.e., are non-decomposable (date, 1990). for example, a single address field that contained both city and state data violates the atomicity constraint. ajis vol. 4 no. 2 may 1997 sources of complexity one source of complexity in the query is that multiple records contain terms data for eachvendor, i.e., each vno. assuming that each vendor uses the same discount rate and discount days for all items, the output of the query should contain only one item per vendor. hence the keyword distinct is required to satisfy this requirement. indeed for the ->nf data structure, there can be multiple records for each item supplied by an individual vendor. the primary source of complexity in the above query is the lack of atomicity of the attribute terms. as can frequently be the case, this lack of atomicity is also associated with less specific data types. that is, rather than two numeric attributes, the discount rates and days are combined into one character attribute. this means that integrity constraints are much more difficult to implement (for example, range checks) and that a particular format must be assumed. the assumed format may be difficult to enforce or ensure. in this case, the format is assumed to be the discount rate, a slash as a separator, and the discount days. if this format is not followed for each and every record in itema, then the results of the query may not be correct. hence, to answer the above question using the ->nf data structure requires making a strong assumption about the format of terms, parsing the contents to extract the desired subset of characters, and converting those characters to a numeric value. inf query select distinct vno, vname, termdisc, termdays from itemb where termdisc > 5 or termdays > 45; relative to the queries of the other two data structures, the only additional source of complexity present in the inf query is that multiple records can contain terms data for each vendor, i.e., each vno. unlike the ->nf data structure, there are no multiple records for each item supplied by an individual vendor. 3nf query select itemc.vno, vname, termdisc, termdays from itemc, vendorc where (termdisc > 5 or termdays > 45) and itemc.vwo = vendorc.vno; because the discount rate and discount days data for each vendor only exists in a single record, the keyword distinct is not needed in the 3nf query. relative to the inf query, however, the 3nf query does contain at least three additional sources of complexity. first, because the attribute vno exists in both itemc and vendorc, vno in the select clause requires a table name qualifier to remove the ambiguity. second, an additional table, vendorc, must be specified in the from clause. third, an additional condition in the where clause is required to join the itemc and vendorc tables so that the vendor's name can be displayed, i.e., the condition and itemc.vno = vendorc.vno. find items where the quantity on hand is < the average quantity issued. ->nf query 1st attempt select distinct itemno, idesc, vno, vname, cqtyoh, avg(qtyiss]+qtyiss2+qtyiss3)/3 from itema alias 1 where alias i.cqtyoh < (select avg(qtyissj +qtyiss2+qtyiss3)/3 from itema alias2 where aliasl.itemno = allas2.itemno group by alias2.ifemno); sources of complexity one source of complexity in the query is that there can be multiple records for each item supplied by an individual vendor. the keyword distinct is required to display only one line per item supplied by each vendor. the primary source of complexity in the above query is the presence of repeating fields, i.e., qtyissl, qtyiss2, and qtyisss. these repeating fields increase complexity in a number of ways. first, instead of specifying a ajis vol. 4 no. 2 may 1997 single attribute in the select clause, e.g., qtyiss in the queries below of inf or 3nf, an average of the three attributes must be taken. second, because the subquery in the where clause will require joining the itema table to itself, the from clause must specify an alias for itema. third, an intricate subquery is required to extract the data needed for the where clause. although the queries of both the 1nf and 3nf data structures also require subqueries, the subquery for the ->nf data structure contains more terms, e.g., select avg(qtyiss]+qtyiss2+qtyiss3)/3 versus select avg(qtyiss). fourth, the results of the query will be incorrect except for those items with exactly three issues, e.g., dividing qtyissl + qtyiss2 + qtyisss by 3 when fewer than three issues have been made will not yield the correct average quantity issued for that item. one of the easiest ways to formulate a correct query using the ->nf data structure is to create a view that places the required data into a first normal form data structure as shown below. after creating this view, the query is almost identical to the query of the inf data structure. -<nf query, correct create view issuesa as select itemno, qtyiss] as qtyiss from itema where qtyissl not null union select itemno, qtyiss2 as qtyiss from itema where qtyiss2 not null union select itemno, qtyisss as qtyiss from itema where qtyisss not null; select distinct item a. if emno, idesc, vno, vname, cqtyoh, avg(qtyiss) from itema, issuesa where ytema.itemno = issuesa.ifemno and cqtyoh < (select avg(qtyiss) from issuesa where yyema.itemno = issuesa.i/emno group by itemno); inf query select itemb.itemno, idesc, vno, vname, cqtyoh, avg(qtyiss) from itemb, issuesb where itemb./femno = issuesb. itemno and cqtyoh < (select avg(qtyiss) from issuesb where \temb.itemno = issuesb.iremno group by itemno); relative to the queries of the other two data structures, the inf query is the least complex, e.g., requires the least number of terms, qualifiers, and conditions. 3nf query select itemc.itemno, idesc, itemc. vno, vname, cqtyoh, avg(qtyiss) from itemc, issuesc, vendorc where itemc.itemno = issuesc.//emno and itemc.vno = vendorc.vno and cqtyoh < (select avg(qtyiss) from issuesc where itemc.iremno = issuesc.ifemno group by itemno); relative to the inf query, the 3nf query contains the same additional sources of complexity as noted in the previous example. 8 ajis vol. 4 no. 2 may 1997 hypotheses formulation the foregoing theory development and example queries suggest task complexity, normalisation, and uis provide foundations on which to analyse task complexity-data structure relationships and predict end-user attitudes. these constructs motivate research hypotheses that investigate end-users' query satisfaction at different levels of normalisation. end-users' uis responses are explored relative to variations in task complexity caused by (a) lack of atomicity, (b) repeating fields, and (c) data fragmentation. end-user satisfaction: inf versus -inf non-atomic data and repeating fields cause complexity to increase. the increased complexity decreases enduser satisfaction. normalisation removes repeating fields and makes all attributes conform to the atomicity constraint. hence: hi: end-users are more satisfied querying inf data structures than -inf data structures. h2: the increased dissatisfaction of ->nf end-users relative to inf end-users is associated with nonatomic data. h3: the increased dissatisfaction of ->nf end-users relative to inf end-users is associated with repeating fields. end-user satisfaction: 3nf versus inf higher levels of normalization cause fragmentation, i.e., increasing normalization from inf to 3nf results in a larger number of tables. this fragmentation requires joins to reassemble the data. these joins increase complexity and decrease satisfaction. hence: h4: end-users are more satisfied querying inf data structures than 3nf data structures. h5: the increased dissatisfaction of 3nf end-users relative to inf end-users is associated with increased data fragmentation. end-user satisfaction: 3nf versus -inf relative to inf, complexity associated with non-atomic data and repeating fields in -<nf data structures exceeds that associated with the increased fragmentation in 3nf data structures. hence: h6: end-users are more satisfied querying 3nf data structures than ->nf data structures. h7: the increased dissatisfaction of ->nf end-users relative to 3nf end-users is associated with nonatomic data. h8: the increased dissatisfaction of ->nf end-users relative to 3nf end-users is associated with repeating fields. research method research design, participants, and data collection this study uses a laboratory experiment to control for various extraneous variables that may confound the observed results. the laboratory experiment facilitates control of data structures, equalises subject motivation, optimises subject participation, and allows random assignment of subjects to experimental groups (campbell and stanley, 1963). the experiment uses the posttest-only control group design explicated by campbell and stanley (1963)4. using this design enhances the study's internal validity by controlling for potential problems such as history, maturation, statistical regression, testing, instrumentation, and selectionmaturation5. the experiment required participants to record their attitudes after querying a database established for a justin-time inventory system. the experiment was developed by liu (1995). the databases of these information systems used three data structures: not normalised (-inf), first normal form (inf), and third normal form (3nf). this is one of three true experimental designs discussed by campbell and stanley (1963). in general, they prefer this experimental design over the other two: the pretest-posttest control group design and the solomon four-group design. for a detailed discussion of these potential threats to internal validity, see campbell and stanley, 1963. ajis vol. 4 no. 2 may 1997 eighty undergraduate and masters level commerce and information technology students participated in the experiment6. most participants had little experience with relational database querying yet were familiar with general computing concepts and activities. prior to the experiment, all participants completed two projects in which they were required to create tables and forms and then query the tables in sql. based on their information systems background, participants were randomly assigned to one of the three treatment groups as follows: an information systems expert ranked each participant in descending order, i.e. the person considered to have the most extensive information systems background was ranked number 80, the person with the next most extensive background was ranked 79, etc. the order of the three treatments was then randomised, with the order being: ->nf, 3nf, and inf. participants were randomly7 assigned to the groups according to their information systems background, so as to eliminate any experience effect. data used to measure the end-user satisfaction variable was collected using a questionnaire. existing uis instruments were not used because they could not investigate specific database characteristics without substantial modification and extension. a number of these instruments, however, were used as the basis for developing the uis scales used in this study (see, for example, ives, olson & baroudi, 1983; doll & torkzadeh, 1988). the questionnaire was administered after the completion of the experiment. the response rate was 95 percent, i.e., 76 questionnaires were returned of the 80 distributed. measures participants performed queries on parts of the experimental data structures that exhibited characteristics of: (a) non-atomic data, (b) repeating fields, and (c) increased fragmentation. the level of end-user satisfaction for querying activities associated with each characteristic was measured using specifically developed scales8. a two-item scale was used to measure the uis of users querying data structures with varying levels of fragmentation. item two's responses were reversed, added to item one, and their total divided by two. a single item scale was used to measure uis (non-atomicity), a seven item scale was used to measure uis (repeating fields), and a three item scale was used to measure uis (overall). these scales are included in appendix a. results table 1 summarises the hypotheses and their results. hypothesis statement of hypothesis result hi h2 h3 h4 end-users are more satisfied querying inf data structures than ->nf data structures the increased dissatisfaction of ->nf end-users relative to 1nf end-users is associated with nonatomic data. the increased dissatisfaction of ->nf end-users relative to inf end-users is associated with repeating fields. end-users are more satisfied querying inf data structures than 3nf data structures supported supported not supported, however, in the predicted direction. not supported eining and dorr (1991) argue that students are appropriate participants for research concerned with novice decisionmakers. 7the method of randomisation was to assign participant 80 to ->nf, participant 79 to 3nf, 78 to 1 nf, 77 to 1 nf, 76 to 3nf, 75 to ->nf, 74 to -inf, and so on. q cronbach alphas were calculated to ensure the reliability of scales with more than one item. only those questions that loaded significantly together in the factor analysis and had cronbach alpha coefficients greater than 0.80 were selected as combined scales to measure specific uis responses. 10 ajis vol. 4 no. 2 may 1997 h5 h6 h7 h8 the increased dissatisfaction of 3nf end-users relative to inf end-users is associated with increased data fragmentation. end-users are more satisfied querying 3nf data structures than ->nf data structures. the increased dissatisfaction of ->nf end-users relative to 3nf end-users is associated with nonatomic data. the increased dissatisfaction of ->nf end-users relative to 3nf end-users is associated with repeating fields. not supported, however, in the predicted direction. not supported, however, in the predicted direction. supported not supported, however, in the predicted direction. table 1: summary of research hypotheses and results table 2 provides descriptive statistics of the uis measures. counts vary between measurements because of incomplete responses on the questionnaires. model non atomicity uis -•nf inf 3nf repeating fields uis --nf inf 3nf fragmentation uis inf inf 3nf querying uis -.nf inf 3nf mean 2.5385 3.7391 3.2692 3.9167 4.5431 4.0500 3.5962 4.6737 4.3800 3.1250 4.0434 4.4400 std dev 1.4486 1.9590 1.5889 1.2084 1.6841 1.5170 1.0051 1.3020 1.0025 1.4166 1.4165 1.3613 count (n) 26 23 26 20 21 20 25 23 26 25 23 24 table 2: end-user information satisfaction descriptive statistics uis (non-atomicity) analysis of variance (anova) was used to determine whether end-users are more satisfied querying databases containing purely atomic data. as hypothesised (h2, h7), the results show that participants who used either inf or 3nf databases are more satisfied than those who used ->nf databases (^2,12 3-25, p = 0.0444). the least square means, reporting the significance of the pairwise comparisons of the uis values, are shown in table 3. 11 ajis vol. 4 no. 2 may 1997 uis (repeating fields) statistically significant support was not found for the hypothesis that end-users are more satisfied querying databases with data structures containing no repeating fields (h3, h8) (f2, 58 = -14, p 0.3009). the least square means (table 3) show, however, that the relationships were in the predicted directions. inspection of their queries indicated that users of ->nf, inf, and 3nf data structures entered similar queries. therefore, the disadvantages of querying databases containing repeating fields was overcome by users repeatedly executing the same query and simply substituting field names. uis (fragmentation) anova results suggest a significant relationship between levels of normalisation and uis arising from fragmentation (h5) (f2,7l = 3.39 p = 0.0197). table 3 shows that no statistical difference was found between inf and 3nf, however, the relationship was in the predicted direction. table 3 also reveals that users querying the -inf data structure were less satisfied than users querying the inf data structure. similarly, users querying the -inf data structure were less satisfied than users querying the 3nf data structure. these results are the opposite to those predicted. these results suggest that users are more satisfied querying databases with a small amount of fragmentation than querying databases containing non atomic data times and repeating fields. this assertion is further supported in the next subsection. model uis (non atomicity) = f (normjevel) uis (repeating fields) = f (normjevel) uis (fragmentation) = f (normjevel) uis (querying overall) normalisation level -.nf inf 3nf -.nf inf 3nf -inf inf 3nf -inf inf 3nf least square means 2.5385 3.7391 3.2692 3.4714 3.8980 3.2929 3.5962 4.6737 4.3800 3.1818 3.6232 3.5278 significance of pairwise comparison -nnf .0141 .1183 .2885 .6593 .0075 .0341 .0599 .1082 significance of pairwise comparison inf .3280 .1340 .2514 .3644 (note: normjevel = level of normalisation) table 3: least squares means analysis uis (overall) statistically significant support was not found for the hypothesis that end-users are more satisfied querying databases with data structures in inf than either -.nf or 3nf (hi, h4, h6) (f2,69 = 1-37, p = 0.1305). the least square means (table 3) show, however, that the relationship was in the predicted direction. the least square means results reported in table 3 show a significant difference, in the direction predicted, between the ->nf and inf data structures (hi). users were more satisfied querying databases in inf than databases in ->nf. statistically significant support was not found for the proposition that users querying 3nf databases were less satisfied than users querying inf databases (h4). investigation of the users' query commands also revealed that many users did not perform the query task intended to test the structural disadvantages of querying in 3nf, i.e., few questions were attempted that involved the fragmentation inherent in higher levels of normalisation. statistically significant support was not found of the proposition 12 ajis vol. 4 no. 2 may 1997 that users querying 3nf databases were more satisfied that users querying -inf databases (h6), however, the results were in the predicted direction. discussion & conclusions this study provided empirical evidence about the effects of various levels of normalisation on uis. the results show that lack of atomicity adversely affects user satisfaction. the results also show that different levels of fragmentation affect user satisfaction when querying databases. the results show that unnormalised tables adversely affect end-user satisfaction. the adverse affect on end-user satisfaction is attributable primarily to the use of non atomic data. in this study, the affect on end user satisfaction of repeating fields was not significant. the negative effects on uis increased fragmentation associated with normalising databases to first normal form are overshadowed by the positive affects on uis of eliminating non atomic data and repeating fields. the increased fragmentation resulting from further normalisation, however, appears to be associated with decreased uis. overall, the results suggest that end-user satisfaction is greatest when querying databases in inf, as this is the level with the least complex queries, i.e., atomic data, no repeating fields, and very little fragmentation. this reduction in task complexity culminates in greater enduser satisfaction. these results are important to organisations because they impact the relationship between database designers and end-users. database designers must be aware that normalisation levels affect both end-users' performance and satisfaction when querying databases. the results also emphasise the need for the provision of specific training programs for end-users. end-user training is of increasing importance as organisations increase their investment in information technology. investigating the effects of normalisation on uis enables organisations to properly direct their training programs and to maximise the benefits they receive from their information technology investments. two specific areas should be the focus of these training programs. first, end-users creating their own databases should be educated on the advantages afforded by creating databases in inf so they as end-users, as well as other end-users, can query the databases with relative ease. users should also be aware of other problems associated with inf databases in relation to update anomalies and learn about the advantages and disadvantages of each level of normalisation. they should also be instructed on the creation of views to emulate a inf database so that querying, one of their more frequent tasks, can be accomplished with efficiency and effectiveness. second, information technology professionals need to be aware that the databases created by them may be highly fragmented. the negative effect that this fragmentation may have on query complexity and uis suggest that end-users should be educated on how to overcome the difficulties of querying fragmented databases. when end-users consistently use databases created in higher normal forms further training may help them create views (as was required for example in one question discussed in section 3) of the data that may be approximately equivalent to inf to make querying easier and more satisfying. another issue arising from this research is the significant negative affect that non atomic data has on user satisfaction. this finding has implications for database designers in that they need to clearly identify the domains for each item appearing in their databases, and ensure that they have no internal structure, that is, are atomic in nature. this study adopted doll and torkzadeh's (1993) assumption that the majority of end-users have low skill levels and generally fall outside the category of experts in most common computer applications. future research should assess cognitive differences between novice and expert end-users. this assessment would assist information technology professionals who create databases for end-users. such research would also help such professionals to better assist end-users who develop and query their own databases. greater knowledge of cognitive characteristics of end-users could also improve the content and relative emphasis of the training provided to them. future research should also test user satisfaction in across a broader range of user queries. completion of this research in future would enable an overall strategy to be adopted that would lead to both efficient and effective database operations and high levels of user satisfaction. references batra, d., hoffer, j.a., & bostrom, r. p. (1990) "comparing representations with relational and her models", management and computing, vol 33 (2) pp 126-138. bowen, p.l., (1995) "the effects of different levels of normalisation on data entry and data entry controls: an experimental evaluation", working paper, university of queensland. box, g.e.p., hunter, g.w., & hunter, j.s., (1979) statistics for experimenters: an introduction to design, data analysis, and model building, john wiley & sons, brisbane. 13 ajis vol. 4 no. 2 may 1997 references batra, d., hoffer, j.a., & bostrom, r. p. (1990) "comparing representations with relational and her models", management and computing, vol 33 (2) pp 126-138. bowen, p.l., (1995) "the effects of different levels of normalisation on data entry and data entry controls: an experimental evaluation", working paper, university of queensland. box, g.e.p., hunter, g.w., & hunter, j.s., (1979) statistics for experimenters: an introduction to design, data analysis, and model building, john wiley & sons, brisbane. campbell, d.j., (1988) "task complexity: a review and analysis", academy of management review, vol 13 ( l )pp 40-52. campbell, d.j., (1984) "the effects of goal-contingent payment on the performance of a complex task", personnel psychology, vol 37, pp 23-40. campbell, d., & gingrich, k.f., (1986) "the interactive effects of task complexity and participation on task performance: a field experiment", organisational behaviour and human decision processes, vol 38, pp 162-180. campbell, d.t., & stanley, j.c., (1963) experimental and quasi-experimental designs for research, houghton mifflin company, boston. card, s.k., moran, t.p. & newell, a. (1983) the psychology of human-computer interaction, erlbaum, hillsdale. codd, e.f., (1970) "a relational model of data for large shared data banks", communications of the acm, vol 13 (6) pp 377-87. codd, e.f., (1974) "recent investigations in relational data base systems", information processing, vol 74 pp 1017-87. cook, t.d., & campbell, d.t., (1979) quasi-experimentation design and analysis and issues for field settings, houghton mifflin, boston. date, c.j., (1986) relational database: selected writings, reading addison-wesley publishing company, massachusetts. date, c.j., (1990) an introduction to database systems, 5th ed. addison-wesley publishing company, massachusetts. doll, w.j., & torkzadeh, g., (1993) "the place and value of documentation in end-user computing", information and management, vol 24 pp 147-158. doll, w.j., & torkzadeh, g., (1988a) "the quality of user documentation", information and management, june, pp 258-274. doll, w. j., & torkzadeh, g. (1988b) "the measurement of end-user computing satisfaction", mis quarterly, june, pp 259 274. elmasri r., & navathe s. b., (1989) fundamentals of database systems, the benjamin-cummings publishing company, redwood city, california. ives, b., olson, m., & baroudi, s., (1983) "the measurement of user information satisfaction", communications of the acm, vol 26 (10) pp 785-793. kent, w. (1983) "a simple guide to five normal forms in relational database theory", communications of the acm, vol 26 (2) pp 120-125. kieras, d.e., & bovair, s., (1984) "the role of mental model in learning to operate a device", cognitive science, vol 8 pp 254-273. liu, m.r., (1995) "the effects of normalisation on data error finding: an experimental evaluation", unpublished masters thesis, university of queensland. maciaszek, l.a., (1990) database design and implementation, prentice-hall, sydney. mackay, j.m. & elam, j. j., (1992) "a comparative study of how experts and novices use a decision aid to solve problems in complex knowledge domains", information systems research, vol 3 pp 150-172. rockart, j.f., & flannery, l.s., (1983) "the management of end-user computing", communications of the acm, vol 26 (10) pp 776-784. schroder, h., & suedfeld, p., (1971) personality theory and information processing, ronald press, new york. schwab, d.p., & cummings, p.l., (1973) "theories of performance and satisfaction: a review", readings in organizational behaviour and performance, w.e. scott, and l.l. cummings (eds.), richard d irwin inc. homewood, pp 130-153. shasta, d.e. (1992) database tuning a principled approach. prentice hall, englewood cliffs, new jersey. 14 ajis vol. 4 no. 2 may 1997 soloway, e., lochhead, j., & clement, j., (1982) "does computer programming enhance solving ability? some positive evidence on algebra word problems", in r. seidel, r., anderson, b., eds, computer literacy, academic press, new york. smith, h.t., & green, t.r.g., eds., (1980) human interaction with computers, academic press, london. tsai a.y.h. (1988) database systems management and use, prentice-hall, scarborough, canada. wood, r.e., (1986) "task complexity: definition of the construct", organisational behaviour and human processes, vol 37 pp 60-82. 15 ajis vol. 4 no. 2 may 1997 appendix a description of scales used in the study variable scale items and response formats no. of items and source uis (non-atomicity) querying terms of payment data was: 1 = extremely difficult; to 7 = extremely easy. circle the number that you feel is most appropriate. 1 item; developed by the researchers. uis (repeating fields) • querying issue date data was: • querying quantity issued data was: • querying quantity defective data was: • querying production variance data was: • querying received date data was: • querying received quantity data was: • querying unit cost data was: 1 = extremely difficult; to 7 = extremely easy. circle the number that you feel is most appropriate. 7 items; developed by the researchers. uis (fragmentation) • the database contains too many tables. • the database contains loo few tables. 1 = strongly disagree; to 7 = strongly agree. circle the number that you feel is most appropriate. 2 items; developed by the researchers. uis (overall) the overall quality of the data structure was: 1 = very poorly designed; to 7 = very well designed. overall querying the database was: 1 = extremely difficult; to 7 = extremely easy. 1 = extremely inefficient; to 7 = extremely efficient. circle the number that you feel is most appropriate. 3 items; developed by the researchers. 16 ajis vol. 4 no. 2 may 1997 appendix b description of data structures used in the study non-normalised (-inf) abbreviation desciption comments itema table itemno vno vname idesc terms ctqtyoh prodjnol issdatel qtyissl qtydefl prodvarl char(6) char(5) char(30) char(30) char(5) number char(5) date number number number prodjno2 issdate2 qtyiss2 qtydefz prodvar2 prodjnos issdates qtyisss qtydefs prodvars recrepnol recdatel qtyrecl unitcostl pay date 1 pay arm* 1 recrepno2 recdate2 qtyrec2 unitcost2 paydate2 payamt2 char(5) date number number number char(5) date number number number char(6) date number number date number char(6) date number number date number (violates atomicity -violates inf) item number of item vendor number name of the vendor description of item terms of payment current quantity on hand production job numberl (repeating fields violates inf) date the iteml was issued quantity of the item issued quantity of the defective items 1 production time variance 1. e.g. 1.10 indicates production required 110% of standard time, i.e., a 10% unfavorable variance. production job numberl date the item2 was issued quantity of the item issued quantity of the defective items2 production time variance!. production job numbers date the items was issued quantity of the item3 issued quantity of the defective itemss production time variances. no of the first report (repeating fields violates inf) date shipmentl received quantity of iteml received cost per unitl date payment 1 made (corresponds to receiving report) the amount 1 paid no of the second report date shipment2 received quantity of item2 received cost per unit2 date payment2 made (corresponds to receiving report) the amount2 paid 17 ajis vol. 4 no. 2 may 1997 first normal form (inf) abbreviation itemb table itemno vno vname idesc termdisc termdays cqtyoh type char(6) char(5) char(30) char(30) desciption item number of item vendor number name of the vendor description of item number terms of payment discount percent number terms of payment number of days discount applies number current quantity on hand issuesb table prodjno itemno issdate qtyiss qtydef prodvar char(5) production job number char(6) item number of item date date the item was issued number quantity of the item issued number quantity of the defective items number effect on production time, i.e., production time variance. for example 1.10 indicates production required 110% of standard time, i.e., a 10% unfavorable variance. receiptsb table recrepno char(6) itemno char(6) recdate date qtyrec number unitcost number paydate date payamt number the number of the receiving report item number of item date shipment received quantity received cost per unit date payment made (corresponds to the receiving report) the amount paid 18 ajis vol. 4 no. 2 may 1997 third normal form (3nf) abbreviation type desciption itemc table *itemno char(6) item number of item idesc char(30) description of item vno + char(5) vendor number cqtyoh number current quantity on hand vendorc table *vno char(5) vendor number vname char(30) vendor name termdisc number terms of payment discount percent termdays number terms of payment number of days discount applies issuesc table *{ prodjno }+ char(5) production job number { itemno + } char(6) item number of item {issdate date date the item was issued qtyiss number quantity of the item issued qtydef number quantity of the defective items productionc table *{ prodjno char(5) production job number {itemno + char(6) item number of item prodvar number effect on production time, i.e., production time variance. for example 1.10 indicates production required 110% of standard time, i.e., a 10% unfavorable variance. receiptsc table *{ recrepno }+ char(6) the number of the receiving report { itemno + } char(6) item number of item { recdate date date shipment received qtyrec number quantity received unitcost number cost per unit paymentsc table *{ recrepno char(6) the number of the receiving report { itemno + char(6) item number of item paydate date date payment made (corresponds to the receiving report) payamt number the amount paid * primary key for the relation + foreigh key for the relation 19 ajis vol. 4 no. 2 may 1997 appendix c instructions given to participants for-<nf, infandsnf some of the purposes of this exercise are: 1. to test how well you can use sql to search for data errors within a database. 2. allow you to relate input control to specific data errors by searching for data errors that input controls should have prevented. task overview task 1 read the scenario and chris's questions task 2 record your session part 1 answer chris's stated questions task 3 locate errors and problems chris (your boss) head of the internal audit department, specifically asked you to look for. (40 mins.) task 4 report records that contained errors or problems chris specifically asked you to lookfor.(10 mins.) part 2 use your ingenuity task 5 use your ingenuity to locate errors and problems not specifically requested by chris (40 mins.) task 6 report records that contained errors or problems not specifically requested by chris (jo mins.) task 7 transmit log files task 8 complete the survey scenario background comfortable furniture limited manufactures household and office furniture for distribution throughout the world. the company operates from 9:00am to 5:00pm, monday through to saturday. comfortable furniture adopted the just in time (jit) ii method a little over a year ago for their inventories. jit ii involves the use of contractual agreements between an organisation and its suppliers where the suppliers assume direct responsibility for entire categories of the organisation's inventory. vendors: (1) provide the required items on a just-in-time basis for production schedules; (2) provide these items at favourable, if not preferential prices; and (3) over the long term, make innovations in their products, production, and pricing to better match the organisation's requirements. in jit ii situations, vendor representatives often occupy offices in the organisation's facilities. the organisation grants the vendor's representatives access to the organisation's data and freedom to inspect physical inventory. the vendor representatives, rather than personnel in the organisation's purchasing department, place the orders for the input materials needed for the organisation's production runs. jit ii reduces ordering costs, delivery times, handling costs, and inventory holding expenses. for these systems to yield the intended benefits, suppliers must not abuse their direct ordering capabilities. converting to jit ii means many of the internal controls for traditional purchasing procedures are eliminated. management expects the internal auditors to query the information systems to determine that jit ii vendor relationships meet internal control objectives. 20 ajis vol. 4 no. 2 may 1997 task on 2 july 1995, chris kaniuk, the manager of the internal audit group (your boss), calls you into her office. she wants you to scrutinise the jit ii database using sql select queries to locate any data errors before deer, price, and persnickety, the external auditors, begin their investigation of the system. furthermore, ian mcmurdy, president of comfortable furniture, asked chris to look for problems or potential problems with the jit ii system. he particularly wants to know about potential overpayments to vendors and whether the overpayments arose from errors or irregularities. because you must investigate the data anyway, chris just delegated you this task as well. after completing your investigation, prepare a memo (in bullet format) to chris pointing out the errors and problems you found in the jit ii database. part 1 answer chris's stated questions . 1. items with no item description 2. negative quantities issued for items 3. production variances less than 0.8 or greater than 1.5. 4. (a) term discount rates greater than 5%. (b) term discount days greater than 45 days. 5. same item description for different item numbers 6. items that have current quantity on hand less than the average quantity issued. 7. items with unit prices that have differences greater than 50% between different receipts. 8. discount [or possibly even an incorrect discount %] taken after the discount period 9. production variance greater than 1.1 and quantity defective = 0 10. payment date less than receipt date plus half the number of term days 11. items that have total receipts greater than 10 times the current quantity on hand. 12. vendor with the highest percentage of defective items part 2 use your ingenuity. chris realises that more errors and problems may exist. she would like you to identify as many of the errors as possible in your investigation. the questions asked by chris in part 1 should suggest other errors and problems. your knowledge of input controls should help you generate some questions. your knowledge of business and your common sense should suggest additional questions. 21 aj1s vol. 4 no. 2 may 1997 c x e z a ;*>uat *njo u. f*^ >û v 5 b. z >>uu 6 b. z z 0 l-^™ 0/3 u o1 j" 3z on <ĵ %j:g qio £ "ii s ^j 5 ^ 2•s ss h -uh ^ 0s tu s w-j o t uj os?on lj. ^ j" 5z on sj •^ ^j ^h ô s "n s vj * 1 -s e!= a u ^ qi uj 2 2 _! o tuj qi 5 c^ u. > • " ^ se le c t d is t in c t it em no f r o m i tk m a w h e r e id es c= ' o r i de sc i s n u £ o cc -c c 'i 'i.« 'n c u || .̂2 &. qm" r» •§ %1 1 0 ov h u s c tn -2 *) * r^.-•s = s v5 *̂ h w u ^ osu s wj o i tu os > on u. ^ .___ .5' r*«w^ 4i" <^ '•0 -5 c tf v ^ «• ^ x 7, .'2 ^ * c•~ 3: <sh w o -^ os u s ujj o ia os 5on u. > <n ^ -s ot 1 ° se l e c t it em no , is sd at el , qt yi ss i, qt yi ss 2, i ss da te 3, q ty is s3 f r o m i tf .m a w h e r e qt yi ss i < 0 o r q ty is s2 < qt yi ss i < 0 ; «o •— v; 7^ c5 tes u 3 —ct" "™ u t_ d c> i: so 3 0 s z .22 do" 0 v i^ ^ 3 a s§ -q ^3 s c c fc ft. *• £<s o •v ^*f '•§ "1 t.2 a s § gs u -q ft = s^ c ir* o ̂ te w u ^ os pj ^ tu ° ipj os 5 on u. > do" c v b 1 -s "* i £l. a_ *• o7h-m c n •v ^^ ^> l/~l •w 1 a s 1 1 si si d £ •?> ft. e£ uju .5 oiuj 2 g j o nr uj os > on a. > oo cr -~ e= • x-c c c •^ v v v -.-? "5 ^ t h -5 s ? 3 se l e c t i te m no , pm dj no l, pr od vi pr od jn o2 , p ro dv ar 2, p ro dj no 3, p ro f r o m i tk m a w h e r e ( pr od va rl > \ .5 o r p ro d o r ( pr od va r2 > \ .5 o r p ro d\ o r ( pr od va r3 > \ .5 o r p ro d\ = s ? -^jz •"•* ^ (/i u(« —fli ra c — 0 .2 s ob u y os= £ n ^ •-o b oo <n 0> c — io tf a to .̂ "§ ? k. ^> a: o wl a ^j y^ioe si s i s s z, si e> uj u ^ os w s ujj o t uj os ton u. > i/^ •<t a b« •s c si qi 0 w, a o • 5 "^w s = s *. 5 aif si et kj u ^ qiuj a ^ — c zu a >on u. > u-. .-. a 5 !~ t 25 i ^ cn c *+ *> ^~" se l e c t d is t in c t v xo , »v i« /h «. j f r o m it k m a w h e r e t o _n u m b e r (s u b st r (/ er w .v , 1 ,i n st r (/ e/ vw o r t o _n u m b e r (s u b st r (/ er w i5 ,3 ,i n st r (/ er /? ii ,'/ m _ c _ ^r c ts = c 3 -c 3 ra c "~ c rbs « ?! r ^3 ra -5 °_ j> _ «j | 00 £ ji a u oo ^s-g^^-2 s^e-g-s c c o £ 2 -~ ^ "x3 cd ^ '~< s a^ . s v b o u; c ii a fc s 'i p. £• t < s < ? s ••s u --sl < i 5-^ &|gow £ s z j o -r <• uj qi £ ̂ on u. > 5̂^j %j . » "> .= vj *~ io ^3 _ oe <u '^ m a ̂* | 0 5 s e = n i <1 s•*i •" *— ^ • 5 '~ ^3< g < -: &|s^w s s z_) o t; < u « >on u. > (^t̂ se le c t d is t in c t a .it em no , a .. f r o m i tk m a a . it k m a b w h e r e a . i te m no < > r. it em no a n d a . i de sc = r . i de sc ; l. 15 e <l) _ c _ e o — s '•= c s£•cl o o u 'c "-; a e £ ̂ era o ;t: 3 on -a -o c 22 t o fc. z u 0z • o> u. r z o h ed cb s e l e c t it em c. i te m no ,a v g (q ty is s) f r o m i t e m c , i ss u e sc w h e r e i te m c. it em no = \s su e sc .it em no a n d c qt yo h < (s e l e c t a v g (q ty is s) fr o m i ss ue sc w h e r e i ss ue sc . i te m no ^r re m c .it em ni g r o u p b y i te m c. it em no ; k •̂ 2 * cx) ^ kli ^cd i £ i^ ^ .^2 s "̂ ^ s o '̂ e ^ ii 5t « s | ffl | v 0 | b = c *m -5 s 3 r . cs s rrj ^ rr i ^ 1 b s ^ s ^ s h p j < s p ^ ^ o u ^ pa "~ gj § 3c 00 u. ̂ x op*( _2 ?^_ ^~ca + <n ^ c*3 ^ 1 § i ac si . '— "^ s e l e c t d is t in c t a .i/ < f r o m i t e m a a w h e r e a .c qt yo h < (s e l e c t a v g (( b .q ty is sl + u. f r o m i t e m a i ) w h e r e n .i te m no = \. g r o u p b y b .it em iw c o u <u .i1 *"* /s a "c a 5 "5 § — 3 iii?!o 3 cs > <" .= cj j= re .s3 s q 2 se l e c t r ec ei pt sc . i te nm o, r ec ei pt sc . r ec re pt re ce ip ts c. re cd at e, p ay m en ts c. p ay da te , pa ym en ts c. p ay am t f r o m r e c e ip t sc , v e n d o r c , pa y m e n t sc , i t e m c w h e r e i te m c. it en m o= r e c e \r rs c. ite m no a n re ce ip ts c. re cr ep no = pa \m eh ts c . re cr ep i a n d i te m c. v/ 70 =v en d o r c . v no a n d p ay da te > r ec da te + t er m da ys a n d p ay am t < u ni tc os t* qt yr ec ; \\ ^j q ^ q j* ^" s "q (j . m fc c qj cc c/3 ^j ^ .̂ts> h.t: ^ ^-, °3 0 3 * 7 ^ s jj co + # u lj r ^ &; g 5. _ 1 1 2 £ _g a g ^j ' 5 s "3 §> s: n *» a 5 s q s £ a v §i s s "". s q § s e e >r £ 3 §b u u fe s c.^a: o: o: u h q ~ fa ffl 5 < pj s qj5 pj ci 5 00 tt. > ... os u w ce ̂ cd ̂ <m" s r7 2 ?t >c •^r ^r 7 ^"^ "̂ *7 ^~^ ^j •si ~x3 —1 "̂ k) 1 ~"~>» is § 1 |r 1 §* . + m j ? ' v ' * j *-«g ^^t + ^^ -. § 5 i ^ t ; < n « t ;s tx a f c ^ j f c ; 1. s "s 9 ~ -s % m ' * % s * $ g ~§^j a r*^ £: ro* . v ' v ^-t:^ ~ ~ 2 ^ a 2 c a a t s c — t m s ' s :~ a c s ' u a ' u ^ a a >-. **. s o c ' s c ^ ' s s 8.8.1 gg «"§g «'•^ ^ s cxh s. c'h d, h -"fn" t m ^ ̂ § ̂ ̂u ^ ~ «r 2 s q ̂ g q s-§ -§ i«gzggz u 5 ' ^ f t i5^ < 0 ^ < 00 cx cxp> *c ^c i ̂ca 5 — -o_ s io,_ c y o .s o > -^ c jj ^j u ^3 ra c. if hi&̂ ^ s e l e c t is su es c. i te nm o f r o m i ss u e sc , p r o d u c t io n c w h e r e i. s s u e s c ./7 ro r/ /o ft h o = p r o d u c t io n c ./7 ro £/ // 10 a n d p ro dv ar > 1 . 1 a n d q ty de f= 0; q z ^2 c i n || 3 — . 2 !< 3 s 5 ~ •̂ | •'s a 2 j* 2 ^ h ? p "§ h ^ pj s pj ^ pj "" pj qi •e 00 b.^ §^^ ii ^* ^" "̂ r!l "^^ **a **-l "̂ ; _^j' xi' ?*. s; s! si!: |sc 1 111 s e l e c t d is t in c t i te n f r o m i t e m a w h e r e ( (p ro dv ar l > 1. 1 o r ( (p ro dv ar s > 1 . 1 ) o r ( (p m dv ur 2 > 1 . 1 ) s "2 n c q ^ .2 _ •— g-| c « -o o ^—•1 fll •" •a s g o o ro >— i— 3g. 60 o"c 55 s e l e c t i te m c ./t em no , re ce ip ts c. r ec re pn o f r o m i t e m c , v e n d o r c , r e c e ip t sc , pa y m e n t w h e r e i te m c. w io =v en do rc . v no a n d re ce ip ts c. it en m o\t e m c . i te nm o a n d pa ym en ts c. re cr ep )io =r ec ei pt sc . re cr ep no a n d p ay da te < ( re cd at e+ (0 .5 * te rm da ys )) ^ ^ js ~ ss w •t: * e ""' 1 l!u y ^ *•• ^ "b s s i * = x si -s ^ 2^ s ^-. e § e s. h = pj c u ^ ot z 3 qi ?e 00 p> s ^ ^ ^7 7 <n r~ xs -*•. *•^ ^c^ ~fv ^ jj pj c uj s <j cq ^ c2 * s. ^ 2 s § "s z c/2 z fe s e l e c t i te nm o, re cr ep n f r o m i t e m a w h e r e p ay da te l < (r ec da te / + (0 .5 *( t o _ (s u b st r (/ e/ -m s, 3, in o r p ay da te 2 < (r ec da te 2+ (0 .5 *( t o (s u b st r (f er w .v ,3 ,i n ! u. urt^ p tu ^ £ u •§ c w i.b-i 1 c u — _rti pi c3 p c sj= c ^ s 3 o. •£ "e. o 23 aj1s vol. 4 no. 2 mav 1997 o u. z u. z z z c 12 =1 fc^l lu oc *o ua az i ** d y •r. 3 00 2 -j. l£ x so £se-< *a. •?,i' o "p -5 1,1 < c. 7. e ~ « u. o ii + i ^ p ll!^ll<j s.'i ^ =a o 0£a u q. is £g c a •3 y o c. o q z u 24 ajis vol. 7, no 2 may 2000 arguing for phd coursework dr. mike metcalfe, information systems doctoral school, university of south australia (city west), adelaide, sa 5000 dr. margaret kiley, advisory centre for university education, adelaide university, south australia 5005 correspondence to mike metcalfe, tel: 08 8302 0238, mike.metcalfe@unisa.edu.au abstract the lone-scholar reputation of the traditional uk style phd program is the result or a misunderstanding by some course designers. getting a phd is as much about making friends, forming a collegiate group, as it is about writing a book. the loneliness should be about being an independent thinker, not to being physically lonely. an increasing number of researchers are now reasserting aristotle's calls for informed public argument to advance inquiry. research is an argumentative process, requiring claims and a universal audience of independent thinkers to constructively question those claims. this paper argues for the establishment of semi-structured programs for phd students, but importantly, one that undergrirds any program with an appreciation that research is an argumentative process, trying to develop knowledgeable audiences, and thus a social activity. introduction most readers will be aware of the british style phd program, well characterised by phillips and pugh's [1987] phrase "the lone scholar," with students having little regular contact with either a supervisor or other students. typically this approach has been adopted by australian universities. in contrast, the typical doctoral programs in the us (with minnesota, indiana and george state excepted) includes a large coursework component where a range of staff and students interact in seminars. the failure of the "lone-gentleman" approach to produce well trained, enthusiastic, researchers within the allocated three years has attract criticism, yet there is a reluctance to move to the full us approach. a compromise, involving some formal schooling, ( a semi-structured program) but still requiring a large thesis is being considered by an increasing number of universities in australasia. the argument this paper will argue for the establishment of a semi-structured program within is doctoral schools, but importantly, one with a significant social agenda based on an understanding of the central role of argument in research. discussion on what subject material (topics) should be included in the program is not of issue here but rather the philosophy undergirding the semi-structured program. for topics, individual school will need to look to their expertise. typically, though it is assumed to be a first year of optional, flexible, seminars designed to provide phd students with research skills, writing skills and/or literature criticism skills culminating in a fairly detailed research proposal. the undergirding philosophy should be aimed at providing students with a perception of research being a local and international collegiate opportunity, derived from understanding that research is a social activity organised around argument for constructive dialectic discourse, not a lonely quest. first though, is there a problem? some background the history of phd's described by noble [1994] points out that the first doctoral degree was conferred in paris circa 1150. it was very much based on the idea of providing more teaching to academics to enable them to become teachers or lecturers. it would have involved a "class" activity. the minister for education in berlin, circa 1810 started the first doctoral degree as a schooling for researchers, but it was still very much a group taught degree. these european taught degrees spread to the us by the mid 19th century. in britain the first doctoral degrees were nearly 100 years later, post first world war; at oxford in 1918 and cambridge in 1919. unlike the european or us degree the english postgraduate research degree was modeled on the oxbridge tutoring philosophy used in undergraduate degrees which allowed students a great deal of freedom1. it did not expect to dictate to unruly lone gentlemen how or when they had to study, provided they eventually produced a credible piece of written-up research. as these institutions would have assumed that they were very much the centre of the universe, issues of developing a teaching style that encouraged collegiate research would not have been explicitly considered. it seems unfortunate that both types of phd used the same label given their very different process. 'as an aside it is interesting to reflect on a similar difference in attitudes towards students, stemming from the same period, between the codes for "english" rugby and american grid-iron. the latter is really a "chess game" played between the two coaches, characterised by the coach's power to stop the game every few minutes to issue instructions. in the english game the coach gives broad tactic advice at the beginning and is then not allowed to "interfere" with the flow of the of the game. the players are treated as "lone scholars." 52 ajis vol. 7, no 2 may 2000 the purpose, then of the us style phd is to train lecturers, future supervisors and researchers, an extension of the undergraduate program. on the other hand, the purpose of the lone-scholar uk style phd seems mainly to be to offer appreciation or recognition for good research. both expect a thesis, which in the us maybe thought of as one, maybe two, publishablc articles plus a large literature review. the uk-type thesis was expected to produce at least two publishable articles, maybe even be a database for a life's work. however there must be similarities. administrators of both phd programs would most likely say they also hope their graduates are, able and keen to undertake a wide range of further research, well networked, well aware of other research, persons able to initiate with the skills and will to design, and able to complete and effectively communicate a research program. despite these potential similarities the different philosophies has led to very different experiences, and types of stress. the us notion of a cohort of doctoral students following a shared and closely monitored framework of research development runs contrary to the individualistic ethos of the lone-gentleman emerging as a "lone-thinker." the idea of teaching students in fully structured programs would be considered to be likely to inhibit both their broader learning, their development, and the encouragement of independent thought. most uk-type academics would argue that there is something seriously wrong with a doctoral program that is not based on the concept of students having to prove themselves, independently, in the research arena. indeed uk students have failed if the external examiners feel their work has not been sufficiently independent of even their supervisors thinking. the culture of individualism within uk trained academia is such that postgraduate students are expected to make their own way in the research world, albeit under the wing of a sole supervisor who will occasionally point them in an appropriate direction. for their part, doctoral students are expected to evaluate and select research areas by themselves, and to produce proposals and plans for research. thus, the emergent approach emphasises individualism and self-reliance, thereby seeking to develop students confidence in their own abilities. students are given "enough rope to hang themselves," and many do. of course there will be mistakes, learning comes from mistakes. some research is misconceived, and students sometimes only recognise this after considerable effort has been expended. it is argued, however, that such realisations, together with the resulting reappraisal of the research plan, encourage some considerable depth of independent thought. they facilitate an appreciation of the importance of having clear research objectives, and help to develop the student's understanding of the subject matter, fostering the creativity, flair, and strength of character that is vital for good inquiry. it is only excusable to do this if students clearly understand the process of research. this includes that research is an argumentative process [metcalfe, 1995] which requires constant interactions with others, else, the research cannot be effective. the problem evidence that the old lone-gentleman approach may need some revision is provided by completion rates as low as 30% in some areas reported by deetya, the british research councils and by studies by christopherson et al [n.d.] who report a study conducted across a range of uk universities which indicated that over 50% of all phd students took longer than 3 years to complete. they suggest that this rate is lower when students had some kind of framework consisting of regular meetings, formal training and discussion on the literature. they go on to suggest that at the end of one year of this type of semi-structured program the students and supervisors were in a better position to judge whether the student should continue to the thesis stage. in the australasian and scottish system, where potential phd students normally take an extra "honours" year (see later), this can also act as a very effective filter, while allowing the student an interim qualification. put another way, christopherson et al concur with philips and pugh's finding that student should not proceed to project work (ie a phd thesis) without the necessary project management skills, and a collegiate base. baddeley [1979] and wood-harper et al [1993] also point out that there has been criticism of the lone-scholar doctoral program. they emphasise the "social problems" such as time wasted by students making changes in research direction, students feeling isolated due to access to insufficient peer networks, vulnerability to the personality, leadership style and standing of their supervisor, inapplicability of research findings (lack of general relevance), and a general lack of appreciation of a range of research skills, leading to low completion rates. that a semi-structued program may be useful has also been confirmed by asking students. kiley [1996] interviewed 32, including overseas, south australian students, and found 38% said they would have liked a group coursework component in their phd. the exact form of the group work needed to be spelt out before other respondents would commit themselves to the idea. this result needs to be read remembering that anything that extends a phd for overseas and fee paying students is an added expense. a student in kiley's study is quoted as saying the leap from undergraduate to the honours year is enormous but the leap from "honours" to phd nothing. the australiasian honours year has evolved from the "scottish" system. it requires an extra year taken after graduation to allow excellent students to be further trained in research methods, further exposed to the research journal publications and undertake a small "trial" dissertation. it can either be used as "pre-training" for a phd or simply used to make the student more attractive for employment. it is a similar program to what will later be suggested for a "semi-structured program" in the doctoral school. clearly the name "honours" is of no use to uk universities as they offer an honours title to students who get good grades in a three year undergraduate program. however, the impact of this extra "structured" year after undergraduate study, but 53 ajis vol. 7, no 2 may 2000 prior to the phd, means that students who have successfully completed the study program were considered by staff to be desirable phd candidates as they knew the department, its research ethos, the collegiality, and thus has inducted students into a collegiate group of like-researchers, ready for the phd study. not only do potential phd students ask for a semi-structured program, but so do employers. clarke [1996] found that many phds did not make good employees as they were over-specialised, with little skill outside those required for their own phd, unable to plan and execute experimental work, poor at problem solving, poor at team work, having a lack of appreciation of intellectual property, trademarks, and information-property issues. it is suggested a good structured program would give the students an umbrella of skills, and colleagues, from which they could conduct their phd, and then later, use for any research required by their future employers. the uk economic and social research council [esrc] [1989], the 1996 harris report and the 1997 dearing report, also call for more structure on doctoral programs as support for the students. harris suggests these support mechanisms should be auditable. shannon [1995] argues, "supervision clearly involves research, but... it also involves teaching, teaching of a special sort." a further argument for having a semi-structured program is that it makes the teaching element in a supervisors work more explicit. a phd supervisor has both a teaching role and a monitoring role. often the teaching role is implicit. lee and green [1995] agree, saying that the teaching component needs to be taken more seriously. it needs to be more explicit so as to provide a conduit for dialogue between supervisors to discuss the teaching aspect of their work, which can often be considerably enriched by group interaction. the need for a change to semi-structured program approach to phd education is strongly supported by the enormous change in the demographics of students over the last two decades. these changes include that the number of students doing phds has increased enormously. previously, departments may have only had one or two postgraduate students in the past now there may be 10 or 20. the average age has risen, with more mature aged, experienced, managers returning to study. more students enrol in a phd's that is very different from their initial studies, or else the material they learnt as undergraduates is no longer at the research edge. this is particularly true in is. research methods appropriate to their old studies may no longer be appropriate. there are more international students with a range of different skills depending on the undergraduate course structure from their first degree, more students are coming from industry where the criteria for producing "good" reports can be very different to that required from academia. furthermore, the motivations for doing a phd can be very different. some come "to get a better job," others want it to be a life changing social experience. some are under a lot of pressure to complete in 3 years, others are willing to take 10 plus years. these differing backgrounds suggest that either care has to be taken in the selection of students or a flexible induction program is required to bring students "up to speed" in whatever deficiency they bring, including forming a collegiate atmosphere. in addition to variation in students background, the same can be said of supervisors. a semi-structured program can be of considerable support to inexperienced supervisors. a frequent counter to suggestions for a semi-structured program is that students should not be forced to undertake study they don't want. if a student applies for a doctorate with a concise, detailed and well written research proposal then forcing them to go through a semi-structured program may not be appropriate. the proposal would need to exhibit the full range of skills a semi-structured program might have offered. this concern can only be countered by emphasising that any semi-structured program should be flexible, and to a large extent, optional. importantly, the student also needs to exhibit some appreciation that doing a phd should leave him or her, and the department with a group of research associates, as well as a phd. having a phd student that lacks the necessary technical skills can be a considerable draw on the resources of a department, but there is an opportunity cost of having a student that is making no contribution to the development of a collegiate atmosphere in the department's subject area. in perelman and olbrecht's [1969] terms the departments needs to develop a universal audience through recognising research to be an argument process. a solution so how can some of these problems be overcome? one obvious solution is for department heads to simply have some humanity for the students personal problems, especially overseas students. however, a more academic response is to suggest that the explicit nature of research being an argument be used as the major design driver. this typically will lead to a semi-structured program that, besides technical knowledge, provides a research collegiatcy that satisfies the socialisation needs of students, and the development of the research topic. how does argument provide the opportunity to develop a collegiate atmosphere? argument is the noun of arguing or argumentation2, the directed construction of convincing evidence [perelman and olbrechts tyteca, 1969; eemeren et al., 1987], the argument being the one line conclusion that is supported by various evidence. does'nt 2 some care has to taken over the word argumentation. here it is simply being used to mean the process of preparing and presenting an argument; arguing. some sociologists use the words argumentation theory to mean argumentology, the study of past arguments. 54 ajis vol. 7, no 2 may 2000 this sound like research? a courtroom3 analogy can be used. court protocol has been designed around the open public forum approach also suggested by aristotle. a court case is a research activity, the courts are also searching for knowledge, indeed, the truth. evidence is presented to a universal audience (jury) to convince. society has accepted this research method to take human life, so it would appear to be acceptable, maybe even the best research system on offer. evidence presented in order to support an argument can be initially sub-divided into, observation and reasoning (logic). observation includes participant observation, ethnography, experiments and psychoanalysis, and can be made with any of the human senses, eyes, ears, taste, smell or touch. as anyone who has seen a stick half in water knows, observations are unreliable, discourse is needed to reason why it looks bent. the second type of evidence, reasoning (or logic) , uses the mind. in a courtroom example the observation may be forensic evidence, eg. blood on the suspects clothes. the motive would provide the reasoning evidence. so it may be logically concluded that the suspect did the stabbing. so, for example, when presenting research arguing that: system design needs consideration of world views, observation evidence might include what users said about a system and conversations evidence from experienced designers. reasoning evidence may take the form of explaining what world views are and how they affect perceptions, which in turn affects learning. it is not possible to say what will be sufficient to convince the audience. hopefully it is sufficient just to say: in an argument, reasons are oriented in favour of the conclusion... argument is accepted only if the reasons it provides seem plausible, relevant, oriented in favour of the conclusion, and sufficient to support it [apotheloz et al., 1993]. the important message in this is that research involves argument and argument is a social activity. eemeren et al. [1987] also provides a definition of argument: argu[ing] is a social, intellectual, verbal [spoken or written] activity serving to justify or refute an opinion [idea, conception, policy], consisting of a constellation of statements and directed towards obtaining the approbation of an audience. eemeren et al go on to explain their definition. they start by pointing out that arguments are a social activity because there needs to be two people present. argument is an intellectual activity, "an activity of reason" where emotion is "subordinate to that of reason." "argu[ing] requires the use of language." eemeren et al. contrast this with communication by means of force and associates argument with report writing, logic, mathematics and the use of symbols generally. in research reports, the language usually has to be carefully defined to avoid disputes that turn out to be definitional. so then, the act of arguing aligns itself with the accepted ethos of research. eemeren at al. go on. argu[ing] always refers to a particular subject concerning which opinions can and do differ... all scientific, scholarly, areas of human knowledge... without any exception whatever, may be the subject of argument. they provide examples of opinions... (1) litmus reacts to acid by changing colour. ... (2) amsterdam is the capital of the netherlands... they continue, "argufing] is offering, defending against, taking account of, and anticipating criticism." "argu[ing] consists of a constellation of one or more statements ... "the common feature to both pro-arguments and contra-arguments is that both forms are directed towards testing the opinion." research, and convincing are the cornerstones of good argument. ... the logical articulation or intellectual cohesion that makes an argument tight comes from thought and not from the mere recitation of the facts or instances [ziegelmueller and dause, 1975]. they also put this the other way around: research skills stand at the very heart of the inquiry phase of argufing] so, research is arguing which is a social activity. cross white [1995] goes so far as to say that it is one that contributes to world peace by offering an alternative to force. if students understand that research is a social activity, involving discourse, and their supervisor organises the students semi-structured program around this understanding, doing a phd should not be a isolating experience. for 3 it is better to think of the royal commission "courtroom" format, or that used by french and german courts, that is, judges and lawyers co-operating to find the truth. this contrasts with the english and us approach involving separate lawyers for the defense and the prosecution. 55 ajis vol. 7, no 2 may 2000 example, any literature review aspects of a semi-structured program may want to do these reviews explicitly using the argument perspective (see appendix 2). if done in a constructive way students can engage the research community in a sociably acceptable way. moreover, it is a very powerful way to review the literature, especially the management literature that contains more assertions (like this!) than evidence. the argument strategy is therefore useful at the "planning my phd activities" level and at the more pragmatic level of layout for the thesis. under this perspective a thesis is presented as an instrument of the argument, one that presents credible evidence to persuade an informed (hostile) audience partly by anticipating their counter argument. this perspective of research trying to convince the reader leads to a distinctive language for the research proposal, the brief. (see appendix 1). not only does the argument approach provide a strategy for designing a thesis, it also considerably assists with the problems of writing. kiley and mullin's [1998] conference into issues affecting quality in postgraduate education reported that writing skills were a problem. grammar and spelling can be mechanically corrected, that a piece of writing has no flow, theme, path, direction, purpose, construction; call it what you will, often stems from the writer not appreciating that having an argument gives structure to writing. lastly, a short comment about epistemology and bias. this argument approach encourages eclectic empirical methods, aimed at convincing a universal audience, and sidesteps whether discourse is searching for truth or creating it, it assumes the problem being studied, and the audiences requirement to be convinced will select the appropriate sources of knowledge. moreover, it does not hide researcher bias, unlike the "independent observer" ideology. conclusion this paper offered support for adopting a "semi-structured program" approach to phd studies in australia. the traditional uk approach to phd students has produced some very eminent scientists but it has also produced a large number of tired, lonely "also-rans". the differences may be due to a lack of consideration of the philosophy that undergrirds phd programs. this paper suggested that this should be that research involves an argumentative process and is therefore a social activity. it is important that both students and those running doctoral schools appreciate that the end product is not just a phd, but a college of competent researchers. all this is best achieved by relistening to aristotle, more than one world view is required to interpret observation, making research a social activity, advance by well reasoned argument. the loneliness of phd scholars should refer to their independence of thought as one questioner in a socially active community of scholars offering dialectic questions to each other arguments. references apotheloz d. et al., (1993) the function of negation in argumentation, journal of pragmatics, vol: 19, pp. 2338. baddeley, a. (1979) is the british ph.d. system obsolete? bulletin of the british psychological society, 32, 129-131. christopherson s.d., et al. (n.d.) research student and superviser, discussion paper, swansea, university of wales, dept. of civil engineering. clarke j.a., (1996), preparation of the phd for the real world, report to the 2nd seminar of the council of deans and directors, adelaide university, south australia. crosswhite j. (1996), the rhetoric of reason, university of wisconsin. davies, g. (1990) new routes to the ph.d. the psychologist: bulletin of the british psychological society, 6, 253-255. economic and social research council (1989) discussion paper on research training in the 1990s. swindon, uk: esrc. economic and social research council (1991) guidelines on the provision of research trainingfor postgraduate research students in the social sciences. swindon, uk: esrc. eemeren f.h. at al., (1987), handbook of argumentation theory, dordrecht:foris publications. kiley m. (1996), evaluation of the structured program, report to adelaide universty, south australia. kiley m, and mullins g., (1998) quality in postgraduate research, adelaide: conference proceedings. lee l., and green b., (1995) postgradutae studies / postgradute pedagogy, the australian universities review, vol: 38(2), pp:2-4. noble k.a., (1994) changing doctoral degrees: an international perspective, england: open university press. perelman ch. and olbrecht-tyteca l., (1969), the new rhetoric, university of notre dame. phillips, e. m. & pugh, d. s. (1987) how to get a ph.d. milton keynes, uk: open university press. shannon a., (1995) research degree supervision, the australian universities review, vol: 38(2), pp.12-15. walton d. (1998), the new dialectic, toronto university press. wood-harper t. at al. (1993), designing research education in information systems, included in: khosrowpour m., loch, k., (eds) global information technology education: issues and trends. harrisburg, pennsilvania, usa 56 ajis vol. 7, no 2 may 2000 ziegelmueller g.w. and dause c.a., (1975) argumentation: inquiry and advocacy, new jersey: prentice hall. 57 ajis vol. 7, no 2 may 2000 appendix 1 research proposal format in an attempt to demonstrate what the argument approach means in terms of laying out a phd proposal, an outline is presented. however, this should be read asking yourself if you have identified the research community or a set of inter-related communities you wish to work with, and who is likely to counter you. research proposal outline this guide-line for laying out a research proposal is only a suggestion. the headings indicate possible headings are followed by text providing brief explanation. it is important that any evidence be presented as if addressed to the worse opponents of your argument. so, any counter arguments should be fully anticipated. any empirical work should, therefore, be designed to counter your worse opponents. the argument approach used below assumes the thesis is being written in order to convince these opponents that your argument is correct. the argument (thesis, theory) the argument approach requires that the research question be initially stated as a claim (conclusion). in about one sentence state what you intend to argue. having an argument, rather than a research question, indicates less arrogance, and some familiarity with the topic. the more surprising your argument the more interesting your thesis. definitions: briefly explain any technical keywords used in the argument. this raises the question of, for which audience should the thesis be written. with postgraduate work, the best answer might be, a recent graduate in the discipline of the thesis. introduction and motivation. this section is a paragraph explaining why the reader should spend time reading your work. its aim is to persuade the reader to keep reading. it should also explain why you bothered doing the research. the evidence the next step might be to list briefly what evidence will be brought to support your argument. this might take the form of other writers' comments, other researchers' findings, sonic logical reasoning on your part and normally some form of empirical study, designed and conducted by you. this part not only outlines your evidence but it also demonstrates your knowledge of the literature. in larger works, this evidence might be summarised as chapter headings. previous arguments here the comments of previous writers can be aired. counter arguments might be introduced. care must be taken to avoid "appeals to authority," so the background of any commentators (expert witnesses) used should be explained. findings from previous empirical studies this section will be characterised by the phrase, "jones 119851 found that..." the experimental method of previous studies could be explained here, maybe with the more general findings presented before the more specific findings. the criticisms of any empirical studies should be mentioned. any counter arguments (rival hypotheses) would also be countered here. a failure to be able to counter some arguments effectively by use of previous writings might guide the choice of empirical study used in the next section. own empirical evidence your own evidence needs to include empirical evidence as this separates "science from religion." give details of the methods and expected results. such evidence might best be designed so as to satisfy your worse critic. a discussion of the criticisms (limitations) in the study design is normally given. conclusions and implications as in the summation of the evidence in a trial, the main evidence is reviewed and the conclusion normally that the argument is "proved" lastly, the implications of the study might be re-emphasised. these answer the "ok, i agree with you but so-what" questions. to some extent the implications will have been mentioned in the "motivation" section but reiteration and elaboration her can provide a useful conclusion. 58 ajis vol. 7, no 2 may 2000 the title present a short title which contains any keywords that would be used in a computer search for this study. appendix 2 preparing a critique of research articles below are some notes intended to assist your thinking when preparing a critique. context in the way of ethnographic studies, what demographic details are known about the paper, the year written, the institutions, motivations for writing, the authors other work, what literature the references are drawn from and how often the paper has been cited (see ssci). this, and the nature of the paper's argument, may help to establish the author's perspective. the argument • identify the argument of the paper being critiqued. • does the argument have any surprise element? that is, does the study verify or challenge ? • does the author explain all the terminology used in the argument adequately? • does she state any limitations in the argument? does it offer any predictive opportunities? • is the argument well integrated, or at least acknowledge the arguments in the existing literature? how so? and, • does the author state why the argument is important? the evidence • what type of logical evidence is presented? is it convincing? outline the steps, and/or • what type of empirical/observational evidence is presented? is it convincing, docs it use appropriate methods? conclusions / implications • do you (dis)agree with any stated implications? • is it possible to make any type of generalisation s (predictions) from the paper? 59 ajis vol. 7, no 2 may 2000 increased availability for networked software services peter clutterbuck faculty of business university of the sunshine coast maroochydore, queensland, australia email: pcluner@usc.edu.au abstract the availability of networked software services is an increasingly important quality of service issue. a service user gauges service availability in terms of initial access time and subsequent service delivery time. this paper develops from a user perspective a quality of service framework for the availability of networked software services. the paper utilises this quality of service framework to suggest the functional characteristics of a new transport layer software specification that would facilitate the increased availability of networked software services. keywords availability, quality of service, network, services, denial of service. introduction business computing is becoming more networked oriented. universities allow candidates from faraway places to earn qualifications online. service industries encourage customers to track delivery progress from their very own home of office pcs. companies are taking their entire supply chains to the web. networked software services are increasingly important within this business-computing paradigm. name, file, mail and web services are examples of that software set considered essential for efficient business operation. the availability of these services is an increasingly important quality of service issue to all major stakeholders associated with the network. idc (1998) reports that 99.999% availability equals five minutes of downtime per year, whilst 97% availability equals a loss of 263 hours per year. whilst hardware availability has received significant research and development focus, no broadly accepted solutions exist for the comprehensive, automated management of networked software service availability. software availability research to date has produced two availability management paradigms. initial availability studies described in gligor (1983), nyberg (1988), che et al (1992), amoroso (1990a, 1993b), bacic et al (1991), and millen (1992) pursued the goal of denial of service protection. these studies focused upon the services and resources directly managed by a single operating system within a multi-user, multi-tasking computing environment. denial of service protection aimed to provide to all authorised users guaranteed rates of progress in completing their current computing task. user rates of progress were theoretically quantifiable by limiting the upper number of users operating within the system, and by establishing for each service or resource the maximum time interval for which it could be held for exclusive use by any one user. a second availability paradigm has subsequently developed with a focus upon services within networks and is described in karamanolis et al (1995), cristian et al (1994) and most completely in pfister (1998). this paradigm has been shaped by two interlinked network characteristics: the frequency of hardware, system or application crashes and the resource cost in reconfiguring and recovering following these crashes. the paradigm proposes a service availability strategy pivoting upon service replication across the network. the service replicates combine as a group to ensure the service always exists at some accessible and publishable point within the network. this second availability paradigm, under the label 'clustering architecture', is gaining some acceptance as the choice of industry for business solutions. this paper provides a comprehensive analysis of the networked software service availability challenge. the paper opens with a review of concepts and theory provided by denial of service research and also the clustering architecture availability solution. the paper continues by defining networked-service availability and by proposing a quality of service framework to support this definition. the availability expectations of a generic service user shape this definition and framework. the paper then proposes a conceptual blueprint for transport layer functionality that would support this quality of service. a brief discussion of future work to implement the quality of service framework concludes the paper. existing availability theory historically availability management research has not been prolific. considered as a body of work, the existing research spans two paradigms: denial of service protection and clustered server availability. denial of service protection denial of service protection was initially examined in gligor (1983), incrementally developed in amoroso (1990a, 1993b), nyberg (1988), bacic et al (1991), and most fully described in millen (1992) and lei wo et al (1997a, 1998b). denial of service involves deliberate or accidental software service monopolisation, destruction or withholding via the usage patterns of other software users. denial of service is a security concern and is a subset of 13 ajis vol. 7, no 2 may 2000 availability management. denial of service protection guarantees to credentialed users service access and provision within acceptable, well defined and advertised wait times. service access and provision guarantees are provided by a denial of service protection base (dpb) which forms part of, or works closely with, the trusted computing base (tcb) or host operating system. the dpb is tamperproof, cannot be circumvented or prevented from operating, and guarantees authorised access to services under its control. the dpb consists of waiting time policies, user agreements, a resource allocation system, and a resource monitor. waiting time policies quantify when a user may expect service access. a maximum wait time prescribes a fixed time bound within which the required service access will be provided. a finite wait time specifies that required service access will eventually be provided. a probabilistic wait time specifies an averaged wait time for service access. user agreements or usage policies constrain the behaviour of users when accessing the service. a resource allocation system controls the timeliness of service allocation the holding, surrender and revocation of resources. a resource monitor implements the service-scheduling algorithm appropriate to the waiting time policy of the dpb. the operational logic of a dpb as described in millen (1992) pivots upon several main points. firstly, two user agreements are stipulated:1) a service request cannot exceed the system capacity for that service and the request must be announced by the user upon entry to the system, and 2) all service users must engage and use the service for a minimum time before user voluntary deactivation (this ensures users progress toward task completion / service completion). secondly, each service or resource within the protected system has a maximum holding time (mht) associated with it. this mht represents the longest continuous time for which the service may be 'held' or utilised by any user. the mht does not cap the total service time a user may require service requests exceeding the mht will be provided as discrete multiples of the mht. from these starting points, the dpb knows the maximum number of users permissible within the system and the feasible service needs of those users when they enter the system. the dpb (and operating system) then runs each user job allocating the service required in time unit multiples of the service maximum holding time. this allows the dpb to quantify an upper bound for the maximum wait time that will be experienced by each user job. the dpb as outlined in millen (1992) is similar to the operation of real time operating systems as described in levi et al (1990) and agrawala et al (1992). the dpb and real time operating systems exercise quota setting on resources/services and acceptance testing on user needs prior to admitting new users to the system. this strategy effectively caps the number of users, their service requests that may be active at any one time and the level of operational services existing within the system. the dynamic scaling of services existing within the system or the operational reality of unbounded numbers of service users is outside the dpb model. clustered server availability clustered server availability has been theoretically described in karamanolis et al (1995), cristian et al (1994), and most completely discussed in pfister (1998). the paradigm guarantees service availability within distributed systems and is motivated by the frequency of hardware, system or application crashes and the resource cost in reconfiguring and recovering from these crashes. clustered server availability assumes an asynchronous network consisting of multiple host machines or nodes linked together into an arbitrary network topology. the primary entities within the model include the distributed service, a service group, an availability policy, and a management service. the distributed service is a facility of the network, and is characterised by its state, type and operational implementation. a service group is a collection of replicated operational implementations. the service group consists of a single primary implementation and several backup implementations. a group is formed by consensus on the single criteria of speedy communication amongst all nodes hosting the group members. group membership is dynamic. an availability policy is a 2-tuple (replication, synchronisation). replication defines the number of service backup implementations maintained within the distributed system. synchronisation defines the number of updates by which backup state differs from the primary state. the model categories close (one to one update) and loose (update arrears) synchronisation. a management service is the system entity that implements the availability policy of the distributed service. the model guarantees distributed service availability by ensuring operational primary and backup implementations exist within the distributed system at all times. the only threat to model viability is total network failure (the model suggests that the last surviving network node will hold the primary-backup group). the model makes no distinction between communication faults at a link level, faults within a node itself, or service implementation failure. when group reconfigurations occur for any of these failure reasons, the newly formed group select a replacement primary server and then create an additional backup on a suitable host node within the distributed system. the model approaches availability management at the macro or system level. service access is always available to users because the service implementation is replicated across the system. the model however does not address any form of service degradation other than service failure. as long as a service implementation is functioning and maintaining communication with group members, the model assumes service integrity and efficiency. availability under this model is all about service access service delivery is assumed to follow as a natural consequence. 14 ajis vol. 7, no 2 a definition and framework may 2000 there are two strategic issues most prominent in researching the service availability challenge a definition of service availability, and the quality of service framework that would enable a networked-service achieve this definition. defining service availability a comprehensive, clear definition of service availability underpins much of the discussion within this paper. an availability definition facilitates a more accurate appraisal of relevant completed research and contemporary industry trends and may well suggest other appropriate solution strategies. within the computing vocabulary, availability is a sparingly used, and poorly defined, term. the international standards tcsec (1985), itsec (1991), ctcec (1993) that traditionally have served as a baseline for evaluating security and quality of service have lightly treated system and application availability. of these standards, ctcec (1993) is most specific defining availability as the access to a specific resource or service within a time frame prescribed by the resource or service. the major distributed system standards and rm-odp (1995) corba (1996) superficially discuss availability. rm-odp (1995) declares availability to be an important quality of service consideration, and corba (1996) defines availability as service accessibility and usability upon demand by an authorised user. the availability research publications referenced in this paper contain differing availability definitions. the clustering architecture publications define availability as service access assurance, the goal being the high probability at all times of service operational existence at some contact location within the network. clustering aims to guarantee service existence user contact and usage progress are then assumed to follow. alternatively, the denial of service research publications view availability in terms of initial access and subsequent user progress in using the service. the availability discussions considered as a body of work also reflect inconsistencies in describing the relationship between availability assurance and other computing architectural design and operational characteristics. is availability assurance a security concern of the hosting operating system or a quality of service challenge for each hosted service-providing entity? does service availability encompass fault tolerance and reliability engineering? can the obvious operational interdependencies of individual services, host software systems, hardware platforms and connecting networks be demarcated and related in describing the availability challenge? the views, experiences and expectations of any service user may well help clarify this complicated scenario. firstly, a service is viewed by a prospective user as a contact point for some functionality or capability. secondly, service use is experienced by a user as a sequence: service contact, request feasibility, and finally the complete provision of this functionality or capability all activities unfolding over some time line. this time line may be highly specified or loosely approximated but is essential for meaningful service use. thirdly, service availability is a gauge as to how this service usage time line meets that timetable expected by the user. this user timetable expectation may be individually derived or evolved via some explicit standard or generalised community understanding. this generic user-driven definition of availability pivots on the timeliness of service contact and complete service provision. this must now be specifically reshaped within the context of a networked service. figure 1 will assist with this development. user network service (host) environment boundary advertised service interface service provision figure 1: network service timeliness service supporting environment (e.g. network host) 15 ajis vol. 7, no 2 may 2000 the generic distributed service is represented in figure 1. the user requests service at the advertised service contact point. from this point, service availability pivots on the time taken to accept the request (tj), verify request feasibility (tf) and complete processing the request (tp) and how this total (plus reply times through tj if appropriate) compares with the user's expectation of total service usage time (te). this scenario applies regardless of network topology, the physical positioning of user and service within this topology, or the type of service offered. this analysis is summarized within the following definition: networked service timeliness is an accumulating measurement of the following activities: j) accepting a service request at the advertised service contact point and 2) analyzing the feasibility and validity of the service request and 3) establishing a contractual timetable for feasible service provision and 4) progressing and completing service provision as contracted. networked service timeliness should be monitorable, verifiable, and publishable. maximised networked service availability maps the timeliness needs and expectations of users to the historically achieved and forecast levels of distributed service timeliness. quality of service framework maximised service availability as defined in this paper maps user timeliness expectations to historically achieved and forecast levels of service timeliness. the historical and forecast service timeliness is an aggregated measurement of the time taken to admit, verify, progress and complete service requests. this definition serves as the single strategic goal for this availability-maximising analysis. it is now necessary to consider the quality of service framework that will achieve this goal. the availability definition suggests five areas of generic functionality: service contact point, service request validation, service throughput, service state, and service accountability. consequently a quality of service framework must directly focus on these functionality areas. • the service contact point or service interface is viewed by users as the contact and delivery point for some specific behaviour or facility of the entire network. this interface consists of three design dimensions: a referencing representation or publishable dimension (e.g., a service name appropriately recorded within a network name server or trader server); a usage protocol or user-calling dimension (e.g., a tcp message of well defined format); and an instantiation or physically functioning dimension (e.g., a tcp program and port mapping to user space). service users require the following quality of service framework for the service interface: a highly publicised and well-known interface throughout the users' network. the interface representation must be readily registrable and detectable within all forms of service listings advertised through the network regardless of whether these listings be based upon service name, service type or some other characteristics. a simple and easily used interface. interface usage protocol must be concise, unambiguous and lightweight. a consistently addressable interface when viewed from any user location within the network. the interface representation must remain constant to all users across network space and time. a temporally efficient interface. the interface must quickly admit new service requests, dispatch both ongoing service dialogue and completed service results. the interface must be operationally efficient for all traffic volumes queues and delays must be avoided. • request validation describes a service capability whereby appropriate service requests are admitted for service provision whilst inappropriate service requests are refused at the earliest opportunity. in general, a networked-service must manage two broad classes of inappropriate service requests. one class consists of requests that are infeasible, incomplete or incorrect in some way. a second class consists of requests that are appropriate in format and content but whose volume or load stresses the service or its supporting infrastructure as in the case of spamming. both classes may be traced to either accidental or malicious origins. the maximally available service must manage this traffic any legitimate and certainly fee-paying customers have little appeal for cuts in service quality that result from the non-screening of inappropriate service requests. to this end, service users require a quality of service framework that centres upon the concept of usage agreements defined as the combination of interface usage protocol with all relevant service utilisation conditions and quotas. these usage agreements must be concise, unambiguous and lightweight. in functional terms, it is difficult to be more prescriptive in terms of generalised usage agreements. it is clear however, that 16 ajis vol. 7, no 2 may 2000 additional functionality is required within the generalised networked-service if request validation is to be fully implemented. this additional functionality is a service awareness of non-corruptible user identity. this user identity enables the service to establish state information linking each user identity to a request history. with this state information, a networked-service may dynamically prioritise service requests down a range of decreasing service levels the minimum of which is service refusal. • request throughput within this model describes a timetable for completed service provision. throughput is a dynamically calculated cumulative time measurement of the following service activities: request receipt, request validation, and complete service provision. service users have a very clear quality of service requirement in this area: service delivery timeliness that meets or exceeds the users' timeliness expectations. throughput consistency service delivery timeliness standards are maintained during all peak workload times. • service state, to a user, facilitates seamless service completion a very important quality of service expectation. even in problematic instances, a single legitimate service request from each user must ultimately trace to completed service provision. in functional terms, service statelessness, however, will mean discontinuous service provision in problematic instances several legitimate service requests will be required for completed service provision. the availability definition derived in section 3.1 views meaningful service provision as completed service provision. in the general case, users are not satisfied with access only to the service contact point, or with other partially completed service results. whilst reliability engineering can never completely offset operational difficulties or failures, a graceful degradation of service is always preferable from a user's perspective. the ideally available service must always complete service provision, and this implies fault tolerance. this fault tolerance must exist at two levels. at the macro level, the service must always exist in some operationally ready form within the network to meet the needs of prospective users. at the micro level, the currently progressing service requests from users must also be cushioned so that the service quality for each request experiences managed degradation, not extinguishment, in the event of operational difficulties within the service or its environment. • service accountability defines how a service describes its availability levels to prospective and actual users. service users, especially those operating within an economic paradigm, would have the following quality of service expectations: the ready supply of up-to-date, accurate, obligation-free "quotes' in respect of service timeliness. service contracts that comprehensively define service usage. these contracts fully specify the usage conditions that must be met by the client, the timeliness guaranteed by the service, the conditions and mechanisms for service level renegotiation, and the domain or scope in which the contracts remain binding and enforceable. these accountability quality of service expectations clearly involve two different levels of implementation. a distributed service can readily issue quotes and contracts but how are these recognised as credible and binding? a trusted third party is required to authenticate contracts and ensure they are honoured. service accountability is an area where functionality is significantly shaped by the network security model and in particular that model's strategies for confidentiality, correctness, and non-repudiation. the five quality-of-service characteristics outlined above translate to four generic capabilities within the service contact point: separation, redirection, relocation, and identification. interface separation maximises the processing power for admitting service requests and dispatching request results. separation describes an interface instantiation that is physically and operationally distinct from other service provision components. separation means that the interface instantiation whilst possibly initiated by, managed by, and certainly coupled with other service provision components operates in an immediate resource space distinct from that of other service provision components. consequently, the throughput of the separated interface instantiation is impacted by the admittance and dispatching of incoming/outgoing service requests and not by the workload that the ensuing service provision entails. interface redirection facilitates replication and therefore scaling of the service at multiple points across the network. redirection describes a capacity for an interface instantiation to be replicated at different, addressable points within the users' network and for each replicated interface instantiation to redirect, 17 ajis vol. 7, no 2 may 2000 where necessary, service requests to other instantiations within the group. the service requires a single, high profile contact point within the users' network. this is the primary interface instantiation and it is supported by one or more secondary interface instantiations. redirection within a group of replicated interface instantiations is hierarchical and rule based. the redirection rules are established initially, and subsequently varied, within the instantiation by that service component responsible for overall service throughput and load management. the instantiation, however, must autonomously exercise rule-based redirection decisions without further reference to other service provision components. an instantiation must also recognise and remember redirection 'chains' in which the instantiation is either the initial or final point of redirection. interface relocation implements address transparency for the overall service. where redirection facilitates service availability via interface load management, relocation facilitates existential availability ensuring the publicised service contact point (the primary interface) operationally exists within the users' network at all possible times. interface identification implements a basic level of protection against the accidental or malicious overuse or spamming of the networked-service. interface identification also provides a basic framework upon which the networked-service may differentiate the levels of service attracted by individual service clients. transport layer functionality the definition and quality of service framework derived in this paper suggest a networked-service interface very similar to that achieved in business via the telecentre concept. the telecentre concept combines telephony infrastructure and business personnel to achieve a business public contact process that is versatile and professional. the telecentre involves mapping all telephone inquiries to a single, well-known, relocatable telephone contact point from this point, incoming inquiry traffic is distributed to a well-defined group of operator contact points. these contact points may be geographically centralised or distributed. the set of contact points is sufficiently scalable to respond quickly to spikes or decreases in the numbers of received inquiries. the management of inquiry workload across the telecentre is very flexible the differing characteristics and time requirements of individual calls can readily be factored into the workload distribution. clustered software services or server farms is the closest analogy that business computing has with the telecentre concept. this paradigm of service delivery is comprehensively discussed in pfister (1998). commercial clustering products are available from microsoft, sun, and a combined offering from hewlett-packard and cisco. all of these solutions impose restrictions in terms of operating systems and/or supporting network topology. the basic operational logic of all these solutions is to share load across the cluster or farm on the basis of incoming request numbers. existing cluster solutions lack the versatility and simplicity of the telecentre concept. transport layer protocols glue together networked software services. the ready implementation of the telecentre concept depends on the operational capabilities of the relevant transport layer protocol. the defacto standard, transport control protocol (tcp), is well described in feit (1998). tcp's popularity is largely due to the quality of service it offers to users. tcp provides a reliable, flow-controlled, connection-oriented communication service. tcp is easily utilised by applications and is implemented within all commercially available operating systems. tcp does not support the full quality of service framework developed in this paper. tcp clearly offers interface separation tcp executes as part of the host operating system and is not tied to the resources allocated to the user application. tcp offers a very coarse-grained identification facility tcp identifies the type, and not the identity, of the application user. tcp does not support redirection or relocation. tcp's relative simplicity has been significantly responsible for its popularity. it is felt, however, that minimal changes to tcp's specification would be needed for it to fully support the desired quality of service framework. these specification changes will evolve from the future work outlined in the next section. future work the distributed file server is usually the most heavily used service within the network and consequently an excellent vehicle with which to benchmark and demonstrate maximised availability. distributed file systems present a single name space to the user and hide the physical location of the file from the file system user. the user references the required file via a single pathname and cannot and need not differentiate physically local and remote file locations. microsoft's distributed file system (dps), sun's network file system (nfs), and the coda file system are all influential file systems and follow the client/server paradigm. in nfs and dps a small set (usually cardinality one) of server processes provide a storage repository utilised by a much larger set of users. this server set provides no file locking or synchronisation consequently nfs and dfs do not provide one copy update semantics for files held uniquely within, or shared across the server set the nfs and dfs server set aims to enhance scalability and also reduce the impact on users of server failure or network component problems. the coda file system has evolved from the andrew file system and is substantially more resilient to file server and network component difficulties. in coda a 18 ajis vol. 7, no 2 may 2000 small set of replicated file servers provides one-copy file updates to a large set of distributed file users. the future work with the research outlined in this paper will apply the quality of service framework previously described to distributed file server design. the resulting distributed file system will resemble coda in its operational goals and its focus on replicated file storage but differ significantly via a state-based but more lightweight client-side agent and greater server interface functionality. conclusion networked-service availability is an increasingly important quality of service issue and is essential in supporting the paradigms of network and user-pays computing. despite this situation, no clear definition or solution strategies exist for managing service availability. this paper has defined networked-service availability and described the service quality level expected by service users. the paper has broadly summarised the strategic changes necessary to transport layer protocols to provide these service levels. the paper also describes the construction of a distributed file system that is being used to demonstrate the practicality and accuracy of the model. references amoroso, e.g. (1993a) a policy model for denial of service. proceedings computer security foundations workshop hi, franconia, n.h. amoroso, e.g. (1993b) ndu(c): a mandatory denial of service model. proceedings of the 16* national computer security conference. agrawala, a.k. and gordon, k.d and hwang, p. (1992) mission critical operating systems. case studies in os. ios press netherlands. bacic, e. and kuchta, m. (1991) considerations in the preparation of a set of availability criteria canadian system security centre, ottawa canada 1991. che, f. and gligor, v.d. (1990) a specification and verification method for preventing denial of service proceedings of the ieee symposium on security and privacy. corba (1996) object management group common object request broker architecture. url http://www.corba.org ctpcec (1993) canadian trusted computer product evaluation criteria, version 3.0. communications security establishment, government of canada. cristian, f. and shivakant, m. (1994) automatic service availability management in asynchronous distributed systems. proceedings of the second international workshop on configurable distributed systems ieee. dod. (us department of defense, 1987,) trusted network interpretation of tcsec. us national computer security center feit,s. (1998) tcp/ip. mcgrawhill. new york. gligor, v.d. (1983) a note on the denial-of-service problem. proceedings of the symposium on security and privacy, ieee. idc (1998) preparing for i-commence. url http://www.idc.com karamanolis, c.t. and magee, j.n. (1995) configurable highly available distributed services. imperial college of science, technology and medicine, london. leiwo, j. and zheng, y. (1997a) a method to implement a denial of service protection base. monash university. leiwo, j. and zheng, y. (1998b) layered protection of availability. monash university. levi, s.t. and agrawala, a.k. (1990) real time operating systems. mcgraw-hill publishing company. new york. millen, j.k. (1992) mitre corporation. a resource allocation model for denial of service. proceedings of the ieee symposium on research in security and privacy. nyberg, n. (1998) denial of service flaws in sdi software : an initial assessment. proceedings [of the ieee symposium on security and privacy. pfister, g.f. (1998) in search of clusters. prentice hall, new jersey. rm-odp (1995) reference model open distributed processing. iso (international standards organization). tcsec. (1985) dod (us department of defense) trusted computer security evaluation criteria (tcsec). us national computer security center, department of defense. 19 ajis special issue requirements engineering 1999 an integrated, probabilistic framework for requirement change impact analysis simon lock and gerald kotonya department of computing, lancaster university. lancaster. uk tel: +44 1524 65201 exl 93140 fax: +44 1524 593608 e-mail: (lock,gerald/@comp.lanes.ac.uk abstract impact analysis is an essential part of change management. without adequate analysis it is not possible to confidently determine the extent, complexity and cost of proposed changes 10 a software system. this diminishes the ability of a developer or mamtainer to make informed decisions regarding the inclusion or rejection of proposed changes. the lack of coherent impact analysis can also hinder the process of ensuring that all system components affected by a change are updated. the abstract nature of requirement level entities has meant that current impact analysis techniques have focused largely on design and code level artifacts. this paper proposes a novel approach which integrates traditional impact analysis with experience based techniques to extend current approaches to support requirement level impact analysis. central to this approach is the use of probability to assist in the combination and presentation of predicted impact propagation paths. an auto teller machine (atm) example is used to illustrate the approach. keywords: requirement, change, impact analysis, impact propagation probability introduction most requirements are defined during the early stages of system development and evolve throughout the system life cycle to reflect the changing needs of the system stakeholders, the customer organisation and operational environment. changes to single requirements may ripple through a system and impact on other requirements and broader organisational goals. requirement changes have to be managed and assessed to ensure that they are feasible, make economic sense and contribute to the business needs of the customer organisation. most development approaches pay insufficient attention to the problems of uncertainty and change (marker et al 1993). a number of approaches address these problems by freezing requirements at fixed points within the life cycle so that system development can be undertaken in an orderly and controlled manner. this, however, often leads to systems which fail to meet the real business needs of the system procurer. other approaches seek to identify the minimum set of stable requirements which must be satisfied and which will lead to perceived benefits for the user (sommerville 1996). it is hoped that by providing incremental delivery, which assists developers in meeting subsequent requirements, it is possible to cope with changes arising from requirements. however, this only prevents certain types of change by focusing on the correctness of the design and implementation of the requirements, rather than supporting changes to the requirements themselves. the problem of requirement change management is further compounded by size of the information and complex relationships between requirement artifacts. this can make the process of assessing the effect of change expensive, time consuming and error-prone. however, without proper assessment it is impossible for developers and maintainers to fully appreciate the extent and complexity of proposed changes. this makes cost estimation, resource allocation and change feasibility study impractical. in addition to this, a lack of adequate impact analysis can lead to difficulties in ensuring that all affected artifacts are updated for all changes made to a system. in order to perform impact analysis, extensive lateral traceability information regarding the system must first be obtained (bennett 1996). lateral traceability links indicate potential relationships between the artifacts, which make up a system and the system's environment. these relationships can cause changes to be propagated from one entity to another. current impact analysis techniques (ajila 1995, goradia 1993, li et al 1996, mccrickard et al 1996, moriconi et al 1990, han 1997) have focused mainly on design and code level artifacts. the main reason for this is that these artifacts are less abstract than requirements level entities, thus making the identification of lateral traceability links simpler (bennett 1996). in this paper we propose a novel new approach which integrates traditional impact analysis techniques with our own experience based approach in order to support change impact analysis at the requirement level. this is supplemented by features to reflect impact certainty and probability values derived during analysis. the use of probabilities in the impact analysis process provides us with useful information regarding the likelihood of occurrence of impact propagations. this allows use to gain a certain degree of contrast between the often large number of predicted propagation paths which are a feature of any realistically sized system. a prototype tool has been developed to support the impact analysis approach described here. a brief description of the tool is given in a later section of this paper. throughout this paper, we will use of the word 'impactable' to denote any previously specified entity, which is affected by a proposed change. for the purposes of this paper the set of impactable entities is limited to the requirements of a system and to any existing or proposed changes to those requirements. by including changes 38 ajis special issue requirements engineering 1999 as impactable entities, we can support their evolution in the change management process, as well as predicting the impact of other changes upon them. in addition to this, we can use the experience of previously enacted changes to try to shed light on the impact effect of new changes. the notion of impactables provides a mechanism for extending the approach beyond the requirements stage to include design level and code level artifacts. this would allow the approach to support impact analysis of development artifacts at all stages of the software lifecycle. old system revise request impact _ analysis refect request r 1 system release planning _l delay request fig. 1 change management process change management process the process of impact analysis is dependent on the availability of a mechanism for change management that provides the information required for assessing proposed changes. the main activities of a management process are illustrated in figure 1 (moreton 1996). the work described in this paper is limited to the impact analysis stage of the change management process. impact analysis process accurate impact determination is crucial for effective impact analysis. because the information produced is often large and complex, care must be taken to ensure that the information generated is both relevant and adequate. a careful balance must be maintained between producing impact analysis results with too much information and not supplying enough to allow proper appreciation of change impact. the main stages of the impact analysis process are illustrated in figure 2 (moreton 1996). single change request fig. 2 impact analysis process impact determination the aim of impact determination is to predict the probable effect of a proposed change on a system. impact determination uses lateral traceability information to determine impact propagation paths between impactables. these paths are then composed into a single structure, which may then be used as a basis for direct investigation, or for the production of a visualisation of the effect of the proposed change (lock et al 1998). figure 3 shows the impact determination process. 39 ajis special issue requirements engineering 1999 change data traceability extraction traceainhy traceability combination propagation sliuctura ^ investigation & visualisation fig. 3 impact determination process i) traceability extraction denotes the extraction of individual traceability relationships from the representation of the proposed system. ii) traceability analysis involves the processing of the raw traceability information extracted previously to produce a representation of the impact of a change on a model of the system. example we will use the example of an automated teller machine (atm) to illustrate our approach to impact analysis. the atm contains an embedded software system to drive the machine hardware and to communicate with the bank's customer database. the initial atm requirements are as shown in table 1. in the remainder of the paper we will refer to these requirements by their short labels rather than descriptions to conserve space. id 1 2 3 4 5 6 7 8 9 10 label cash balance databa se valid printou t encrypt risk avail failure simple description the system will allow customers to withdraw cash the system will allow customers to display and print their current balance the system will access a remote database in order to obtain customer pin and account information a customer will require a valid cash-card and personal identification number (pin) issued by the bank, access the atm services to the system will print a paper record of all transactions bank standard encryption algorithms must be used for all transactions the system security risks must be explicitly identified, analysed and minimised according to the alarp principle the cash-withdrawal service should have an availability of 99% the system failure rate should be no more than 1 in a year the system should be simple to use for all customers. customers should spend no more than 10 sees locate required information to table 1 atm requirements table 2 shows four possible requirement changes that may be proposed even as the initial atm requirements are formulated. id 1 2 3 4 label affiliat e new pin mainta in transfe r description in addition to its own customers the bank would also like customers from affiliated banks to access the atm for cash withdrawal the bank would like its customers to be able to change their pins interactively on the atm to help ensure fault-free operation, the bank proposes to do complete atm maintenance once a month the bank would like to add a funds transfer facility to the atm for its customers to transfer funds between accounts. table 2 requirement change list in this example, that the first three change requests (affiliate, new pin and maintain) are assumed to have already been enacted and their total resultant impact is known. the last change (transfer) has yet to be enacted and must be assessed for its impact on the system. traceability extraction the first step in impact determination is to extract traceability information from a model of the proposed system. our approach uses 5 different schemes to extract traceability information at requirements level, namely: • pre-recorded traceability analysis • dependency analysis • plain experience analysis 40 ajis special issue requirements engineering 1999 • extrapolation analysis • certainty analysis we will use the atm example to illustrate each of these techniques, and later demonstrate how we have integrated them into a single framework that provides a more sensitive and informed impact analysis. pre-recorded traceability analysis pre-recorded traceability analysis is an existing technique which uses information recorded by developers, maintainers and end users to identify potential traceability links between impactables (bohner et al 1996). prerecorded traceability analysis can be done throughout the system life cycle, from initial requirements formulation through to implementation. this information captures the relationships between the entities that make up the requirement problem space. each recorded relationship has the potential to cause an impact to propagate from one impactable to another. thus all relationship types, including all inverse relationships, must be considered when assessing a proposed change. the information used as a basis for performing pre-recorded traceability analysis must be recorded by the requirements engineer when the requirements are being specified. early traceability information can be extracted from user requirement documents by manually scanning or automatically parsing them for specific keywords representing relationships. the identifiable pre-recorded traceability relationships, which are present in the atm example, are illustrated in figure 4. j const'edby | avail ] const'edby | failure j impeded by i valid j supported by | avail ] supported by | failure ] const'edtjy | risk ] const'edby | risk ] eonst'edby i risk ] const'edby | risk 1 const'edby | risk dependency analysis fig. 4 pre-recorded traceability relationships dependency analysis is an existing technique which aims to extract traceability information from behavioural models of the system in order to predict possible impact propagation paths (bohner et al 1996). the basis for extracting such information is normally a call graph representing dependencies between the system entities. both forward (depends on) and backward (is depended on by) calling relationships are extracted from the models to ensure that all possible impact propagation paths are identified (paakki et al 1996). once this has been done it is possible to determine potential paths by tracing altered functional requirements (impact sources) to impacted functional requirements (impact targets) via the call graph. this is illustrated in figure 5, where a change (cl) requires an alteration to the source requirement (fr5) which could propagate an impact to the target requirements (fr1,fr2,fr3,fr4,fr8,fr9) via the model calling graph. 41 aj1s special issue requirements engineering 1999 1 1 requirf mtnl 1 1model ( c l ) change 1 o ta^ .ta.0. j,. ! impact* fig. 5 identifying propagation paths the models used to extract call graphs range from structured textual descriptions through use-case scenario diagrams to pseudo code. the extraction of such call graphs may be performed manually, although improvements in efficiency and accuracy can be attained if automated analysis tools are available. the dependency relationships, which can be extracted from the atm example are illustrated in figure 6. fig. 6 dependency relationships fig. 7 plain experience data plain experience analysis existing plain experience analysis approaches use a record of the effect of previous changes to the system as a basis for extracting traceability relationships. for a given change, traceability relationships can be assumed to exist between the change and impactables which it has previously effected. thus each previous change provides us with a cluster of impacts which represent a small segment of the entire system traceability structure. this technique is known as plain experience analysis because previous impact propagation paths are used to construct a path map which is then directly applied to the current change to the potential impact propagation patterns. no attempt is made to identify similarities between the current change and the previously proposed changes and no 42 ajis special issue requirements engineering 1999 additional analysis or intelligent filtering takes place. although simplistic, such an approach is quick and reliable and provides a sound basis for impact analysis. figure 7 shows the effect of three currently enacted changes (affiliate, new pin and maintain) on the atm example system. extrapolation analysis extrapolation analysis is a new technique which utilises past experience data to perform analysis. the aim of this type of analysis is to project the small number of direct impacts specified for a proposed change into a complete set of both direct and indirect impacts. extrapolation analysis is achieved by comparing the direct impacts specified for the proposed change with the actual total impacts of all previous changes. in this way it is possible to identify a number of previously enacted changes which contain the direct impacts of the proposed change as a sub-set of their total direct and indirect impacts. it is possible to combine the total impact effect of a number of similar, previously enacted changes to develop a composite prediction of the impact effect of the proposed change. this not only maximises the set of projected impacts, but allows us to identify the most common, and therefore most likely, impacts. the direct impacts of a proposed change are compared with the set of all impacts (direct and indirect) of previously enacted changes. the previous changes can then be ranked in order of similarity with the new change. the impact effects of most similar (i.e. highest ranking) previous changes are combined to produce a final set of impacts with which to extrapolate the effect of the proposed change. the number of previous changes used in this combination is configurable, so that the predicted effect could be based on any number of previous change effects. each of a the given number of most similar previous changes contributes a number of potential impacts. the more previous changes that agree on a particular impact, the higher will be the estimated probability given to that impact. this forms the basis for a scoring system in which the most popular and most likely impacts is given the highest probabilities, and conversely, impacts which are unpopular given lower probabilities. the proposed change (transfer) best matches the impact effect of affiliate. this is because the direct impacts of transfer are a subset of the total impact of affiliate. we can therefore designate both changes to be of the same class. as a result, we can infer that the total impact of transfer is equal to that of affiliate. this is illustrate in figure 8. direct impacts direct and indirect impacts lnfered\ xi y ĵ-f impacts \ ' encrypt fig. 8 extrapolation data certainty analysis we have developed a certainty analysis technique as part of this work to try to minimise the effect of the inherent incompleteness and uncertainty involved in performing impact determination with requirement level artifacts. this should not be confused with the concept of certainty factors as used in the field of expert systems. our only aim is to try to measure the reliability of the information held about the requirement artifacts which make up a system. certainty analysis provides us with additional information to offset the deficiencies of the recorded information. to this end, the following additional items of information must be collected for each impactable by the developers specifying the system: • degree of definition the extent to which the impactable has been completely specified (eckland et al 1996). • certainty of definition the confidence with which the developers believe the entered information is correct using these two measures, it is possible to derive a quantitative estimation of the reliability of impactable information and the traceability relationships between them. this additional information makes it possible to gain an appreciation of which impact predictions are most likely to be correct. conversely, it is also possible to identify those predictions, which are less likely. the assignment and maintenance of values for 'degree of definition' and 'certainty of definition' relies to some extent on the judgement and experience of individual engineers. however, provided that a coarse scale (e.g. 3 or 4 point) is employed the assessment of the impactables is relatively straightforward. the values that these 43 ajis special issue requirements engineering 1999 measures can hold range from 0 for complete uncertainty to 1 for complete certainty. to produce a total certainty value for a particular impact, the two measures are combined using the following formula: qoui = dod * cod where c^i is the total certainty value, dod is the degree of definition and cod is the certainty of definition. we shall investigate how certainty values are used to adjust propagation probabilities in a later section. the total certainty values for each requirement and change specified for the atm system are present in figure 9. impaaawe cash balance database risk avail failure simple total certainty 0.7 0.8 0.6 0.7 o.e 0.8 0.5 impactame valid printout encrypt affiliate new pin maintain transfer total certainty 0.7 0.9 0.8 0.6 08 0.7 0.7 fig. 9 certainty values integrated approach as mentioned earlier, the abstract nature of the entities makes it difficult for any one of these techniques to extract enough traceability information to allow for viable impact analysis. this lack of adequate information can adversely affect the accuracy of impact determination and result in: • the failure to distinguish between relationships, which propagate change, and those which do not • the prediction of impact effects that are larger than would actually be caused by a change. this is due to the difficulty in separating relevant propagation paths from large sets of heterogeneous traceability information. our proposed solution to these problems is an integrated framework, which combines a number of individual techniques. this is underpinned by a consistent structure of probability and certainty analysis as a basis for the analysis technique. we believe such a framework provides a basis for better impact analysis by: • providing a more sensitive analysis than the single paradigm methods. this results from the fact that although each individual approach may predict spurious propagation, those on which the methods agree will be presented as the most likely impacts. • ensuring completeness through improved system analysis coverage. • counteracting the abstract nature of requirements through a scheme that integrates the individual strengths of traditional techniques. • providing potentially extensible approach that be used to support change impact analysis across the entire system lifecycle. • providing a mechanism for distinguishing between potential impacts due to the fact that propagation paths identified by more than one analysis technique are more probable than those identified by a single technique. impact propagation structure once the raw traceability information has been extracted from the system by one or more of the traceability extraction techniques it is combined into a single propagation structure. disparate traceability links are combined to form a concrete structure showing all direct and indirect propagation paths for the particular change being analysed. this structure is known as an 'impact propagation structure' and constitutes the end product of the impact determination process. figure 10 shows an example of an impact propagation structure. 44 © bnpxt propagation pith irwlivi dual traceability relationships 0-<b) ©-<f) 0—<§) ©—© ©—© predicted impact propagation structure fig. 10 an impact propagation structure structure showing impacted components \ctualuniqueimpacts structure showing actual impacts fig. 11 unique structure nodes a single impactable may be associated with any number of nodes within the structure. however, every node is unique because the impact which it represents is defined by the total path to that node. this is shown graphically in figure 11. propagation probability due to the potentially large size of the produced propagation structure, the hybrid approach utilises estimated probability values to provide some distinction between the many predicted propagation paths. in addition to this, probabilities are used during the combination of individual extraction techniques to reflect the differing reliability of the results produced by each. each propagation path identified is allocated a relative probability by the extraction method which identified the path. this probability value reflects the confidence with which the method has suggested the path. the value indicates the likelihood that, if the impact source was altered, the target would be impacted (ajila 1995). these probability values are independent of each other in that the occurrence of an impact propagating between two impactables does not effect the probability of propagation between any other impactables. it is however important to take other probabilities into account when composing the final propagation structure, but this is a different matter and will be discussed in more depth in a later section. it is not possible to differentiate between the significance of relationships derived from pre-recorded and dependency analysis. this is because such links are derived from simple entity relationships. thus the probability of each impact propagation relating to a single call path will all be equal. however, if multiple relationships exist between two impactables, then the probability assigned to the impact path will be correspondingly higher. probability values given to propagations identified by past experience techniques depend on the number of times each impact path has been previously experienced. for example, if one change causes many impacts to a particular impactable, then the predicted propagation path which is produced will be associated with a relatively high probability. conversely, if an impactable is rarely effected by a change then the predicted path will have a relatively low probability. it is obvious that much of the generation of probability values is based on previous experience. when analysing previous experience, it is not possible to distinguish between the probabilities of impact propagation through an impactable and from one impactable to another. these two components of impact propagation are illustrated in figure 12. 45 key: o impactablo propagation from one impaciabto to another propagation through an importable composite propagation path fig. 12 composite propagation paths we cannot distinguish between the propagation probabilities of these two components because, when recording the impact effect of changes, only a single probability is actually observable and we cannot easily decompose it into its component parts. this is not a problem in its self in that such decomposition is not essential to the generation of probability values, however it should be borne in mind that all impact propagation paths actually represent such composite probabilities. the production of probability values is particularly useful in providing answers to 'what if questions. that is, when a developer wishes to observe the effect of a theoretical or experimental change, without the need for the enactment of that change. the tool developed to support the hybrid approach is particularly useful in such cases, as it provides the engineer with a rapid and accurate way of experimenting with different change scenarios. the probability values for the atm past experience relationships are given in figure 13. mtn 9 probabkty of 0 t mtn a proba&kty of 0 8 mm a prooabkry of 0 5 mtn a probablry ol 0 7 won a orobaouv of 0 6 mtn e probabkty o* 0 9 mtnaorobatmttvof 07 mm a probaoiltv of 0 8 mtn a protoatxk^ of 0 5 mth a prombkry of 0 0 tnaorobat>ihyof07 a probathkry of 08 a prooamry of 0 5 ih a protawty of 07 a promcnlty of 0 9 tn a diottftnkty of 0 7 o probatxkty of 0 6 tn a [xobatxkty of 0 5 a probatxtaty of 0 6 fig. 13 probability values 46 certainty adjustments impact impact impact change | »•[ impactable a |—»•[ impactable b| »-|lmpaclable c | probability of impact 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 \n impactable a \ >. \\ \ \ impactable b impactable c \ \ \ \ \ impactable c 1 impactable depth figure 14 the effect of certainty adjustment the certainty values generated by the tool can be used to adjust the calculated propagation probabilities in two distinct ways. the two types of probability values produced by the application of the certainty information are as follows: • sureness probabilities these represent the estimated likelihood of impact calculated from the information available. such values indicate the sureness with which the analysis technique predicts that impacts will occur. thus, with low certainty values, predicted probabilities are reduced due to the high uncertainty of the information on which the predictions were made. • cautious probabilities these represent the risk of impact calculated from the information available. such values indicate an estimation of the danger of an impact occurring. thus with low certainty values, predicted probabilities are increased due to the high uncertainty of the information on which the predictions were made. the effect of these two types of probability value are illustrated in figure 14 which shows the two probability estimations for each of three hypothetical impactables. the diagram shows the effect of the different probability values on the values of the 'downstream' (indirectly impacted) impactables. we can see from figure 14 the large range of values that the impact probability of impactable c may have, depending on how certainty is used. the cautious and sureness probability values for the three impactables a, b and c are show in figure 15 impactable a b c cautious 0.875 0.825 0.775 sureness 0.775 0.475 0.175 figure 15 cautious and sureness probability values 47 in order to achieve either cautious or sureness probability values, we must apply the results of certainty analysis to the propagation path probabilities in a certain manner. the formulae used to calculate probability values is as follows: for sureness probabilities, we reduce the probability of the propagation by scaling it using the calculated certainty value: adjusted probability of impact = probability of impact * certainty value of path ( 1 ) "adjusted = "path ^-path for cautious probabilities, we reduce the probability against propagation by scaling it using the calculated certainty value: adjusted probability of no impact = probability of no impact * certainty value of path (2) —i "adjusted = ~~™ path *~piih however, standard probability theory allows us to state that: (3) — "ppath = 1 " ppath and that: (4) ~~i "adjusted = 1 " 'adjusted substituting (3) and (4) into (2) we get: 1 "adjusted = ( ' " "path) cpath ' adjusted = 1 ( 1 p path) cpat|, 'adjusted = 1 {-p&th + (lpjtj, 'path/ where ppalh is the probability calculated by the extraction mechanism, cpath is the certainty of the target impactable of the extracted relationship and pasted is the final probability assigned to the path. for simplicity's sake, we will concentrate on the generation of sureness probabilities for the atm example. traceability analysis traceability analysis involves the processing of the extracted traceability information to produce a single impact propagation structure. once the traceability data has been extracted from the system representation, the individual traceability links are combined into a single structure. this does not simply mean combining the results of each method for the entire system, but rather interleaving them so that they can work together in unison. a diagram showing the process used is illustrated in figure 16. impact propagation stnjcluni to impact visualisation fig. 16 propagation structure generation the generation of the impact propagation structure involves four distinct stages, the first three of which are interleaved and performed repetitively. we will first investigate the cyclic nature of the process and then move on to discuss the four key stages in more detail. analysis cycles in order to identify the total potential impact of a change on a particular system, any analysis technique must be repeatedly applied to the system, forming a number of analysis cycles. this is true for the hybrid technique and in fact forms an important aspect of the combination process. each analysis cycle takes a set of impact sources (entities which must be altered to accommodate the change proposed) as inputs and produces a set of impact targets (entities which are impacted as a result of the alterations made to the sources) as outputs. repeated analysis is required because the targets of the impacts can propagate the change impact to other entities and must therefore be fed into the next analysis cycle as impact sources (thompson 1994). this is illustrated in figure 17. 48 key a ) r«qu«f«m«nl cycle one cycle two to=b,c ouud.e.f cyde three © full propagation stiucture full propagation structure fig. 17 analysis cycles lateral composition lateral composition is the stage in which the outputs of individual methods are combined. this is achieved by summing all the propagation paths detected by the individual methods at each cycle of analysis. figure 18 shows the combination of the results of two separate methods for a single cycle of analysis. key 0 hybrid ol method 1 and method? hybrid result incaoutcb.c fig. 18 lateral composition in order to allow propagation paths identified by different methods to be combined in this way, the results of each method must be weighted appropriately. this is done because the accuracy with which impacts can be predicted may vary depending on the reliability and focus of the method being used. it is important to note that these weightings are relative and could thus have any range. the suggested method weightings for each of the techniques used in the framework are given below: • pre-recorded traceability analysis 0.8 • dependency analysis 0.8 • plain experience analysis 1.0 • extrapolation analysis 0.6 by weighting the results of each individual method it is possible to adjust them to reflect the relative accuracy of each approach. in this way the results of disparate techniques may combined while still ensuring approximate parity between predicted impacts. thus the probability of predicted impacts identified by dependable methods will be generally higher than impacts identified by less reliable methods. vertical composition vertical composition involves the chaining together of the results of each cycle of analysis in order to construct the complete propagation path structure. each cycle of vertical composition adds an additional layer of predicted impacts to the propagation tree. this additional layer is the total set of impacts identified by the lateral composition stage of the current cycle. figure 19 shows how lateral composition combines the results of each method and how vertical composition then takes these impacts to construct the next layer of the propagation structure. key © roquromonl impact cydeone cycle two cyde three '©© ©0© hybrid ol method 1 and method 2 ln=b,c out=o.e.f ln=d.e.f outeg.h fig. 19 lateral and vertical composition 49 figure 33 in the appendix shows the interleaved vertical and lateral composition stages as applied to the atm example. the figure illustrates the predicted impact of a newly proposed change, referred to as transfer, on the atm system. the impact propagation structure produced for the example system after lateral and vertical composition is illustrated in figure 34 in the appendix. duplication resolution it is possible for duplicate propagation paths to be identified between two requirements. this results from the same impact being predicted by two different extraction mechanisms or multiple predictions from the same method. during duplication resolution these are converted into a single propagation path in order to minimise the size of the final propagation structure. the probability of the single path is calculated from a combination of the values for the two original propagation paths. the probability value assigned to the predicted impact will be relatively high, reflecting the fact that the propagation has been detected by two independent mechanisms. this duplication resolution is done at the end of each cycle to minimise the number of inputs to the next cycle of analysis and prevent the replication of analysis. a number of duplicate paths are present in the structure shown by figure 34 in the appendix. in order to resolve these duplicate paths, an adjustment formula must be used for combining multiple probabilities. we shall now describe the adjustment formula which is used and in particular examine the purpose and rational behind that formula. let us imagine that we have a impactable 'a' and a impactable 'b1 and that two impact propagation paths exist between them as shown in figure 20. path 1 path 2 fig. 20 impactables and propagation paths between them key: q decision node © decision node at wttcn b is impacted —-j unexpended branch fig. 21 decision tree for an impact on impactable b in order to combine both paths into a single link, we must integrate the impact propagation probabilities of the two paths into one. if we examine the decision tree for the impact on b (see figure 21), we can see that an impact on b may occur if: • the impact propagates along path 1 (node 2 on the decision tree) or • the impact does not propagate along path 1, but does propagate along path 2 (node 4 on the decision tree) we can illustrate the probabilities of these conditions using the venn diagrams shown in figure 22 50 ifflpsct along pathl impact path « no impact impact along impact along path 2 no impact no impact probability of an impact on b probability of an impact via path 1 probability of no impact along path 1, but an impact via path 2 fig. 22 venn diagrams showing the probability of impact on b the shaded areas of the venn diagrams represent occurrences in which b is impacted. the diagram illustrates how the total probability of impact on b may be broken down into two separate areas. if we further decompose the rightmost venn diagram from figure 22, we reach a stage where we may formalise the discussion and derive a formula for calculating the probability of impact on impactable b. this is illustrated in figure 23, which shows equivalent venn diagram, natural language and formula for calculating the probability of impact on impactable b. n probability of an impact on b probability of an impact on b probability of an impact via path 1 p, probability of an impact via path 2 probability of no impact via path 1 (1-pi) fig. 23 equivalent venn diagram, natural language and formula we can generalise this case and state the following formula for integrating the probabilities of any two impact propagation paths: padjusted = pold + pnew (1-pold) where pnew is an additional probability, p0id is the current probability before the assimilation of the additional probability and padjusted is the total probability after adjustment and integration has taken place. if many propagation paths are to be integrated together, then the formula is applied incrementally, combining each additional path in turn to the composite path. the resolved paths and adjusted probabilities for the atm system are illustrated in figure 35 in the appendix. application of decay as mentioned before, the impact propagation probabilities associated with each propagation path are independent of all other probabilities. the occurrence of one impact along a propagation path will not effect the probability of the propagation paths between any other pairs of impactable. despite this fact, when analysing the total effect of a given change, we cannot simply combine all the previously extracted traceability relationships without first considering their relative position in the final impact propagation structure. as we move through the layers of propagation in the generated structure, the probability of reaching a particular point decreases. this is because each propagation in a transitive path will usually have a probability of less than 1. thus, when calculating the probability of a single propagation path, it is important take account of the probability of reaching the path in the first place. this process is known as probability decay and is illustrated in figure 24. 51 single relationships impact propagation stnjcture ~n a 0.8'0.7 / \ 0 8 0 2 fig. 24 probability decay in the diagram, the probability of an impact on impactable 'c propagating to impactable 'e' is 0.9. however, this value needs to be adjusted when the propagation path is part of the full propagation structure because impactable 'c' has a less than one probability of being reached. thus, for the given change, the probability of an impact upon impactable 'e' is the probability of an impact upon 'c' (i.e. 0.56) multiplied by the probability of that impact propagating to 'e' (i.e. 0.9) which gives a final result of 0.504. figure 36 in the appendix shows the effect of applying probability decay to the atm propagation structure. combating complexity the size of the impact propagation structure shown in figure 36 is significant, despite the relatively small size of the example system. it is thus obvious that the impact propagation structure produced from any non-trivial system will potentially be very large. a number of mechanisms can be employed to help minimise the extent of this structure. these include filtering methods such as: • the prevention of recursive paths • the use of a probability cut-off level • the application of behavioural thread filtering • the employment of viewpoint partitioning these four mechanisms will now be discussed in more detail in the following sections. in addition to mechanisms which aim to reduce the physical size of the structure, it is also possible to ease the process of construction and investigation. this can be done by providing tool support for the impact analysis process, which can include facilities for collection and analysis of lateral traceability, as well as the visualisation of the produced impact propagation structures. an overview of the tool support for the integrated approach is provided in section 4 of this paper. prevention of recursive paths full impact propagation structure pruned impact propagation structure a) figure 25 a pruned impact propagation structure a feature of the process aimed at reducing propagation structure size and complexity is the detection and prevention of recursive impact paths. these are cyclic paths which occur if one impactable is traceable back to 52 itself, either directly, or indirectly via a transitive path. if these cyclic paths are incorporated into the generated propagation structure, it can result in structures of infinite depth. preventing these recursive paths involves recording which impactables have previously been visited when performing traceability analysis. this ensures that no impactable is expanded more than once in any single transitive path. by preventing these cyclic paths from appearing in the structure, it is possible to remove all recursive paths, and thus all duplicate information. this is illustrated in figure 25 which shows the effect of 'pruning' an impact propagation structure in this manner. probability cut-off one possible way to help minimise the size of the generated impact propagation structure is to allow the specification of a probability cut-off value. this value represents a level below which the probability of impact is considered to low to be of significance. only propagations with probabilities above the cut-off level will be included in the propagation structure. in this way we can minimise the size of the final impact propagation by removing propagations which are unlikely to occur and thus do not contribute significantly to the assessment of impact. figure 26 shows the effect of two different probability cut-off values on a complete propagation structure. full impact propagation structure probability cut-off = 0.3 probability cut-off = 05 ,0-7 behavioural thread filtering fig. 26 probability cut-off values behavioural thread filtering is an abstraction technique which allows us to focus on a particular aspect of system behaviour when assessing the impact of change. this leads to a reduction in the complexity of the final impact propagation structure produced which can ease investigation and appreciation. in addition to this, utilising behavioural threads can provide a useful tool for investigating the impact of change upon a system. such analysis can assist the requirements engineer in: • assessing the relative effects of different, potential changes on the same portion of system functionality • comparing the effect of a number of different changes on a number of different portions of system functionality • evaluating the resilience of different implementations of the same functionality to given changes • identifying potential impact 'hotspots1 in the implemented functionality of the system • experimenting with a number of potential changes to try to identify which ones have the most significant effect on system functionality • determining which of a number of different change request implementation alternatives will most effect the critical operations of the proposed system. to perform behavioural thread filtering, the requirements engineer must first specify a pattern of execution of the proposed system. this pattern of operation is referred to as a single 'behavioural thread' and represents a single path of flow control through the impactables which make up the proposed system. the behavioural thread is actually made up of a single slice through the behavioural models used to represent the operation of the system. once a behavioural thread has been specified, we must then identify the system impactables associated with that thread. each model representing the system helps define one or more functional requirements. the functional requirements defined by the models in the behavioural thread are then all collected. this subset of all system functional requirements is then used as a basis for filtering the generated propagation structure. all requirements 53 not present in this selected set are then filtered out of the propagation structure, so that only those associated with the behavioural thread are present. it is important to note that a complete analysis takes place and all possible transitive paths are derived, but only impacts on the identified subset of requirements will be present in the final structure. viewpoint partitioning a useful technique for reducing the size and complexity of the generated impact propagation structure is the use of viewpoints to partition the system. viewpoints are individual perspectives of a system and can include end users, external systems, managerial staff and other system stakeholders. each viewpoint is associated with a subset of the entire collection of impactables of a system. by limiting the impactables present in the impact propagation structure to those of a particular viewpoint, we can produce a assessment of the impact of a change from a single perspective of the system. this can not only greatly reduce the complexity of the propagation structure, but provides us with a valuable tool for isolating desirable segments of impact information. when assessing the effect of change upon a system, we may often only be interested in how a change impacts a single viewpoint of that system. for example, it may be desirable for the user to assess how a particular change will effect the security viewpoint of a system. even if we need to identify the effect of the change from all perspectives of a system, the use of viewpoints can assist in breaking the total impact into logically grouped, more easily digestible segments. these can then be explored sequentially in order to investigate the impact of the change on the system in it's entirety. it is useful to classify the viewpoints of a system exist into a hierarchical structure such as that show in figure 27. i console operator [[customer |[rfttertefence || manager || merkeong fig. 27 a viewpoint hierarchy it then becomes possible to select specific viewpoints (such as console operator), abstract viewpoints (such as interactor), or even the root viewpoint. this gives us the flexibility to focus on individual viewpoints or alternatively general viewpoints, as well as the ability to generate a single propagation structure of the whole system. tool support we have implemented an extensive prototype tool to support the integrated approach described here. the tool provides facilities for: • recording, managing and analysing requirement artifacts • abstract system modeling • recording, managing and analysing proposed changes • automated traceability extraction • automated propagation structure creation • advanced impact visualisation • impact investigation and appreciation features figures 28 to 30 in the appendix show screenshots of the implemented systems. these include the interface components for the specification, maintenance and examination of functional requirements (figure 28), non functional requirements (figure 29) and changes (figure 30). in addition to these interface elements, figures 31 and 32 in the appendix depict the impact propagation structure visualisation interface and magnification facilities respectively. conclusions this paper has investigated the important process of impact determination and analysis within the general context of requirements change management. we have presented the various traceability extraction mechanisms available for performing requirements level impact analysis. we have discussed shortcomings of individual traceability extraction techniques and proposed an integrated approach to addresses these problems. 54 our proposed solution achieves the composition of multiple traceability extraction approaches while maintaining approximate parity between predicted probabilities. the result of this work has been the development of a novel framework, which successfully integrates past experience based approaches with more standard dependency and pre-recorded techniques. the issue of uncertainty has also been addressed by the framework, which uses certainty values to adjust generated probability values. to ensure the scalability of the approach a number of filtration mechanisms have been developed. these can be used to both minimise the set of impacts predicted by the process as well as providing important techniques for revealing different aspect of the system. in addition to this, the analysis of large systems is also eased by the availability of an integrated tool which supports all of the features described in this paper. we are currently exploring ways of extending the change impact visualisation to integrate use cases and event scenarios. by using scenarios we hope to incrementally explore the change process and the wider context in which the change occurs. we intend to integrate static and dynamic visualisation. dynamic visualisation will show how control ripples through the system and how change affects exceptions. we are also investigating ways of extending our approach to support impact analysis in the later stages of system development. in this respect we are exploring ways of extending the notion of impactables1 to include artifacts from the design, implementation, testing and maintenance stages of the lifecycle. references ajila, s. , (1995), software maintenance: an approach to impact analysis of objects change, softwarepractice and experience 25(10) october, comer, d. e. ; wellings, a. j., ppl 155-1181 bennett, k., (1996), software evolution: past, present and future, information and software technology 38(11) november, dyer, m.; shepperd, m., pp673-680 bohner, s. a., arnold, r. s., (1996), software change impact analysis, ix-xii, ieee computer society, los alamitos ecklund, e. f., delcambre, l. m. l., freiling, m. j. , (1996), change cases: use cases that identify future requirements, sigplan notices 31(10), october , berman, a. m., pp342-357 goradia, t. , (1993), dynamic impact analysis: a cost-effective technique to enforce error-propagation, proceedings of the 1993 international symposium on software testing and analysis, june, pp!71-181 han, j. , (1997), supporting impact analysis and change, propagation in software engineering environments, proceedings of the eighth ieee international workshop on software technology and engineering practice, july, ppl72-182 marker, s., eason, k., dobson, j., (1993), the change and evolution of requirements as a challenge to the practice of software engineering, proceedings of the 1993 requirements engineering symposium, january, pp266-272 li, l. , offutt, a. j. , (1996), algorithmic analysis of the impact of changes to object-oriented software, proceedings of the international conference on software maintenance, november, ppl 71-184 lock, s., kotonya, g., (1998), requirement level change management and impact analysis, internal technical report no. cseg/21/98, october, lancaster university computing department mccrickard, d. s. , abowd, g. d., (1996), assessing the impact of changes at the architectural level: a case study on graphical debuggers, proceedings of the international conference on software maintenance, november, pp59-67 moreton, r., (1996), a process model for software maintenance, software change impact analysis, bohner, s. a., arnold, r. s., pp29-33, ieee computer society, los alamitos moriconi, m. , winkley, t. c. , (1990), approximate reasoning about the effects of program changes, ieee transactions on software engineering 16(9), september, basili, v. r., pp980-992 paakki, j., salminen, a., koskinen, j., (1996), automated hypertext support for software maintenance, the computer journal 39(7), august, van rijsbergen, c. j., pp577-597 sommerville, l, (1996), software engineering, addison-wesley, wokingham thompson, j . , (1994), what you need to manage requirements, ieee software 11(2), march, sharon, d., pp!15-118 55 1 i > iiiii it 3 1 m en ts aun ct io na u iu c t—« so n ob b b £ 56 u a '3 ^o uc5 c o i u c »-h o\ c* oi x "3 ia 58 m archlvist prerecorded dependency plain experience "inference he 13 cu1-offlevel: magnlf: 0.42 weight 15.3 vfj cut-off prob.: generate | dynamic | : compress extraction mechanisms: pj prerecorded traceabllity p" dependency p" plain experience p" extrapolation no.: s visualisation mode: | black and white (s tree overlay type: (• sureness r feasibility c graph c cautiousness o priority minimise close appendix fig. 31 visualisation interface appendix fig. 32 magnified propagation structure 59 » 3 ii § s ,.? i* ^.s •3 i ii c o '« o q. "au <u t3 § _m i •3c 0»aa. < o. o o. 3 •o bo c o .c vi 3 o co 'i i s bjd 62 •a xie d. do c o j= ca u 00 ffj q. o e i x 1 &n. a.i is vol. .1, no. 1 alternative views of case research in information systems kill doolin school of management studies, university of waikato, hamilton, new zealand e-mail: wrd@waikato.ac.nx abstract this paper discusses the differences between a positivist research philosophy and one which is increasingly emerging in is research an interpretive research philosophy, as they relate to case research, by examining selected case studies drawn from recent published is case research. each perspective places a different emphasis on case research, reflecting different ways of viewing reality and the pursuit of valid knowledge. this is an important realisation that can help us understand the debate that often arises over the nature and stature of case studies as a method for conducting research. introduction case studies are becoming increasingly popular in information systems (is) research (benbasat et al. 1987, benbasat and nault 1990). boland and o'leary (1991), in an editorial to accounting, management and information technologies, emphasise the fruitfulness of studying information technologies in the field. such studies enable an examination of the interrelationship of information technologies with organisational activities and management practices. frequently, however, discussions on the use of case studies in is (e.g. benbasat et al. 1987, lee 1989) have used criteria derived from the application of "scientific method" to develop their methodology. judging case studies by such standards unnecessarily limits their use in is research. fundamental philosophical assumptions about the nature of reality, knowledge and human behaviour underlie any piece of research, and influence the researcher's notion of acceptable research methods (chua 1986, hopper and powell 1985). orlikowski and baroudi (1991) suggest that a single set of philosophical assumptions, which they label positivist, dominates is research. they argue that there are additional, alternative philosophies that can inform is research, specifically interpretive and critical research traditions. assumptions about the nature of physical and social reality (ontology), together with assumptions about what constitutes valid knowledge (epistemology), influence what are considered acceptable methods for obtaining that knowledge (methodology). differences in these underlying philosophical assumptions mean that case research enjoys a different status under different research traditions. despite the obviousness of this argument, there is still misunderstanding over the nature and stature of case studies as a method for conducting research. examples of the debate over the conduct and value of case studies can be seen in the is literature (benbasat et al. 1987, dutton 1988, benbasat et al. 1988) as well as in the management literature (eisenhardt 1989, dyer and wilkins 1991, eisenhardt 1991). the aim of the paper is to illustrate the differences between a positivist research philosophy and one which is increasingly emerging in is research an interpretive research philosophy, as they relate to case research, by examining selected case studies drawn from recent published is case research. the intention is to contribute an understanding of the differences in approaches to case research under these two research traditions, which may assist members of the australasian is community who are reading the product of case research or contemplating conducting case research themselves. first, the philosophical assumptions underlying the positivist and interpretive research traditions are reviewed, and their implications for the conduct of case research are discussed.' second, case research is defined for the purposes of this study, and the process used to review recent is case research is described. research design characteristics of the case studies reviewed are then discussed, and the different purposes of selected case studies under positivist and interpretive research philosophies are examined. '. for a fuller discussion of each research philosophy, its underlying assumptions and its application to is research, the reader is directed to orlikowski and baroudi (1991). for reviews of the "scientific" approach to case study method in is see benbasat et al. (1987) and lee (1989). for a recent discussion of the interpretive use of case studies in is see walsham (1993). for coverage of alternative research approaches in is more generally, see nissenera/. (1991). 21 ajes vol. 3, no. 2 positivist and interpretive case research a positivist research philosophy presupposes a reality which exists independently of our knowledge of it. however, experience with this singular, tangible reality can result in knowing it fully, given enough time and sound investigative procedures. individual studies may only be approximations, but eventually convergence will occur (lincoln and cuba 1985). a positivist research philosophy understands organisations as having a structure and reality beyond the actions of its members. this objective structure is capable of being discovered and measured using researcher-devised constructs. the validity and reliability of these identifying and measuring instruments is of particular concern, as is the passive, neutral role played by the researcher in the investigation. large sample surveys and controlled experiments are primarily used, to deductively establish causal relationships and universal laws which form the basis of generalised knowledge, and from which patterns of behaviour can be predicted across a range of situations. the data revealed are assumed to be identifiable and exist independent of the method used to measure them. importantly, participants in the research are required to express their experiences in terms of the researcher's constructs, facilitating replication in other settings (orlikowski and baroudi 1991). case research under a positivist research philosophy tends to be evaluated using criteria common to experimental and survey research (button 1988). the concern is to conduct case studies in such a way so as to satisfy the standards of the natural science model of scientific research controlled observations, controlled deductions, replicability and generalisability (lee 1989). while acknowledging the complexity of organisational processes, researchers in this tradition tend to assign case studies a preliminary role in developing measurement constructs or as an exploratory tool for developing hypotheses for subsequent, more rigorous, investigation. case studies are often viewed as particularly appropriate where research and theory are at their early, formative stages and lack a strong theoretical base (benbasat et al. 1987). however, the use of case studies to build theory which is generalisable across settings has been proposed by authors such as yin (1989) and eisenhardt (1989). it involves the use of multiple cases to conduct cross-case comparisons under a replication logic analogous to scientific experiments (as opposed to the sampling logic behind survey research, for example). an interpretive research philosophy asserts that reality and our knowledge of it are social products which cannot be understood independent of the social actors who construct and make sense of that reality. a shared social reality is produced and reproduced through ongoing social interaction, and can only be interpreted, rather than "discovered" (orlikowski and baroudi 1991). actions cannot be understood without reference to their meaning (chua 1986): through this process of continuous social interaction, meanings and norms become objectively (intersubjectively) real. they form a comprehensive and given social reality which confronts the individual in a manner analogous to the natural world (pp. 613-614). of course, constructed realities are often related to tangible entities, and while constructions differ between different individuals, they are often given a similar, agreed definition which constitutes a partial description of the entity. an information system can have physical components in the form of hardware and data communication cables. nevertheless, an information system is understood (constructed) differently by different individuals and is given meaning by the shared understanding of such phenomena which arises out of social interaction. events, persons, objects are indeed tangible entities. the meanings and wholeness derived from or ascribed to these tangible phenomena in order to make sense of them, organize them, or reorganize a belief system, however, are constructed realities (lincoln and cuba 1985, p. 84). the interpretive tradition encourages a less strongly structured approach to the case research, favouring ethnographic techniques and participant observation as principal modes of data collection and emphasising the perceptions and explanations of participants in the organisation studied. such research is experiential and interpretive (ansari 1991). the researcher approaches the organisation studied with less preconceptions and prior categories. the primary purpose is to offer an interpretation of and for human conduct. a review of recent is case research in order to gather a rich set of is case research, the publications of four journals between 1989 and 1993 were examined. the intention was to examine examples of case research in the is area, rather than attempt an exhaustive survey of published is research. mis quarterly (misq) and information and management (i&m) are core is journals which publish the majority of is empirical research (cheon et al. 1993). information systems research (isr), is relatively new, but is already well regarded. the fourth journal, accounting, management and information technologies (amit), is also new and was selected because of its editorial commitment to publishing field 22 a.iis vol. .1. no. 2 research. it was first published in 1991 and publishes research concerning information systems and organisational practices. table 1 shows that of the journals surveyed, the primary publishing outlets for is case research were mis quarterly and information and management. of course, the journals vary considerably in the number of years they have been in existence and the number of articles published in them per year. table 1: journals reviewed for the period 1989-1993 journal publication period misq 1989-1993 i&m 1989-1993 isr' 1990-1993 amit* 1991-1993 total ' first published in 1990 6 first published in 1991 number of is case research articles 20 18 6 6 50 percentage of total sample 40% 36% 12% 12% 100% percentage of total journal output 16% 6% 10% 18% two standard identifying characteristics were used to distinguish case research from other types of research. first, the researcher has direct contact with organisational participants, and these contacts provide the primary source of research data. second, the research focuses on actual organisational tasks or processes (ferreira and merchant 1992). case research is the study of social practices in the field of activity in which they take place (scapens 1990). the implication here is that the phenomenon under study is investigated within its "real-life context" (yin 1989). experiments and surveys are thus excluded from this definition, since an experiment divorces the phenomenon from its context, and surveys are limited in their ability to investigate context (yin 1989). this close involvement with the organisation means that interviews and direct observations of activities tend to be the primary means of data collection in case research. fifty studies published since 1989 met the above criteria for case research. where studies involved multiple data collection methods, the decision to include or exclude them in the review was, admittedly, a subjective one. for example, studies where a survey was the primary method of data collection, and in which interviews were used only for instrument validation or in which follow-up interviews and site visits were used only to clarify results, were excluded. studies where interviews were used only to administer a structured survey instrument were also excluded. the extent of organisational contacts and the importance the authors placed on the results they gained from them were used as criteria for making the decision. the focus was on empirical research, defined by benbasat and nault (1990) as "research that uses qualitative or quantitative data as a basis for the investigation of research questions" (p. 211). cases where the motivation was simply to describe the development or implementation of a particular information system, in the absence of an explicit research objective, were excluded from the survey. these "application descriptions" (benbasat et at. 1987) frequently conclude with a list of lessons or recommendations for the development or implementation of similar systems. cases based on publicly available, secondary data were also excluded. table 2 provides characteristics of selected case studies referred to in the subsequent discussion. the table is intended as a guide for readers who wish to examine in more detail is case studies which illustrate the range of approaches to case research recently utilised under positivist and interpretive research philosophies. 23 a.iis vol. 3, no. 2 table 2: recent examples of is case research study positivist alavi (1993) aydin and rice (1991) blanton et al. (1992) doll and doll (1992) earl (1993) feeny et al. (1992) lederer and mendelow (1990) reich and benbasat (1990) zack (1993) interpretive bloomfield etal. (1992) hirschheim and newman (1991) orlikowski (1991) topic evaluation of electronic meeting system predicting attitudes towards a medical is it organisation is downsizing strategic is planning ceo/cio relationship impact of environment on is management factors influencing success of customer-oriented strategic systems interactivity and communication mode choice construction of hospital is symbolism and is development it and organisational control site(s) 1 company headquarters (us) 1 university student health service 2 banks (us) 1 distribution company (us) 21 companies (uk) 14 ceo/cio pairs (uk) 20 is executives 1 1 systems in 9 companies (canada) 2 daily newspapers 3 public hospitals (uk) 4 organisations (us, canada, uk) 1 software consulting firm data sources questionnaire interviews observation questionnaires structured interviews observation document review structured interviews questionnaire structured interviews document review structured interviews structured interviews structured interviews structured interviews document review interviews observation email capture questionnaire interviews document review observation nondirective interviews observation unstructured and semi-structured interviews document review social contact findings extends prior laboratory research to an organisational setting adds new variables to prior implementation research exploratory contribution to a contingency theory of it organisation exploratory description of a recent phenomenon exploratory construction of a sisp taxonomy exploratory insights into ceo/cio relationships exploratory groundwork for testable hypotheses identifies factors for hypothesis generation; develops a model of coss success exploratory study; formulates propositions for future research explains interpretive flexibility surrounding implementation of resource management systems uses symbolism to better understand is development provides an understanding of control and deployment of it in organisations 24 ajis vol. 3, no. 2 wastell and behavioural 2 universities interviews uses a stress model to newman dynamics of is (us) gain insights into the (1993) development systems development process research design most of the studies reviewed were descriptive in that the authors described organisational contexts and practices in their presentation of their research. however, most researchers went beyond just describing the case or cases they studied, and attempted to contribute to theory in some way. most commonly they described their observations and then reflected on their observations' theoretical significance (ferreira and merchant 1992). the majority of the studies examined (about 80%) fell within the mainstream, positivist orientation that dominates is research (orlikowski and baroudi 1991).^ that is not to say that they were necessarily good examples of positivist epistemology. the descriptive approach of many of the studies lacked strong methodological development of any kind. case research designs can be categorised as case studies, multiple case designs and case surveys (den hertog and wielinga 1992). single case studies allow in-depth analysis of one setting with regard to a large number of aspects, allowing a broad and detailed analysis of organisational dynamics, and the production of the rich descriptions favoured by interpretive researchers. a multiple case design usually sacrifices detail and richness of description for the opportunity to make comparisons across several settings. multiple case designs are necessary for generating generalisable theory under the replication logic of positivist case research (eisenhardt 1989, yin 1989). case surveys are capable of generating information from a large number of settings, but are usually restricted to a small number of narrowly defined and often quantitatively expressed variables. the research designs of the studies reviewed are shown in table 3. multiple case designs were mainly used to increase the diversity of settings examined, and only occasionally for cross-case comparisons (e.g. blanton et al. 1992). the relatively large proportion of multiple case designs among the interpretive studies is interesting, although the median number of cases was two, which is lower than the median of four cases for the positivist multiple case designs. often, the interpretive studies used more than one setting without an apparent loss of depth. for example, bloomfield et al. (1992) conducted field work in three different sites over a three year period. the case surveys in this review typically interviewed a single individual in each organisation using very structured interview techniques, and gave comparatively less information on organisational contexts. table 3: research designs case study multiple case design case survey total positivist 13 17 9 39 interpretive 5 6 0 11 total 18 23 9 50 as expected, interviews were the primary method of data collection in the studies reviewed. however, the interviews varied widely in terms of both their structure and their extent. unstructured, semi-structured and structured interviews were all used. unfortunately, many authors failed to describe what they meant by these terms. some interviews were relatively unstructured or "nondirective", but with guidance over topics to be discussed: a nondirective interviewing technique was used, as this allowed the respondents to express their own views in their own words rather than force their experience into predetermined categories. however, the interviews focussed on certain topics which are important in the literature (hirschheim and newman 1991, p. 42). 2. in their review of is research literature published between 1983 and 1988, orlikowski and baroudi (1991) found that a dominant set of assumptions reflecting a positivist perspective informs is research. they found only 5 interpretive studies in the 155 research articles they surveyed. 25 ajis vol. 3, no. 2 other interviews were more structured, using a list of open-ended questions as an interview protocol. for example, lederer and mendelow (1990), in their study of the impact of environment on the management of is, structured their interviews by asking a similar set of three questions about each of five dimensions of the environment. the most structured interviews solicited quantitative responses as well as qualitative ones, often using likert-type scales. these very structured interviews were frequently a feature of the case surveys (e.g. earl 1993, feeny et al. 1992). less than a third of the studies fully disclosed the extent of their interviewing. the practice was more typical of positivist oriented studies, presumably in an effort to provide sufficient information for readers to judge the adequacy of the time spent in the organisation (ferreira and merchant 1992). the interpretive studies tended to rely on the richness of their descriptions to convince the reader of their validity. while there were extremes among the studies that provided information, 20-30 interviews of 1-2 hours duration was typical. other data collection methods were used to supplement and corroborate the evidence obtained from interviewing. a number of studies reported examining archival data and company documentation. for example, doll and doll (1992) examined 78 items which included the is strategic plan, budgets, management correspondence, organisational charts, consultant reports and system architecture diagrams. other reported sources of documentary evidence included annual reports, job descriptions, minutes of meetings and project documentation. several studies directly observed is activity, including electronic meeting systems sessions (e.g. alavi 1993) and project meetings (e.g. bloomfield et al. 1992). sometimes observation had a primary role in the research. zack (1993) attended 16 meetings and spent 140 hours observing the publishing process in 2 daily newspapers. some studies used a multiple phase approach to combine field interviews with questionnaires. commonly, the quantitative and qualitative results were used in a confirmatory way as a form of triangulation, but in other instances the two types of evidence complemented each other (alavi 1993) or a comparison between them yielded additional information (aydin and rice 1991). research purpose the most common purpose given in the positivist studies was exploratory. usually the motivation was simply to describe and explore the research subject, providing useful insights or identifying common factors. sometimes the study focused on a recent problem or issue without an established literature or on a unique situation. for example, in the study by doll and doll (1992), the objective was to describe and explore a phenomenon, is downsizing, which is not well understood. some of the exploratory studies went a step further and generated formal hypotheses or propositions from the results of their research. this motivation seemed to arise out of the positivist concern for "scientific" testing of theory, and was demonstrated by feeny et al. (1992): given the small size of the sample, it is inappropriate to formally test the research propositions implied by the model... the study was instead intended to build insights using the depth rather than breadth of the data. this, we felt, would lead to stronger propositions that could be tested by subsequent work (p. 438). lederer and mendelow (1990) commented similarly on the contribution made by their study of the impact of environment on is management: [t]he paper contributes to is theory and research by providing the groundwork for some testable hypotheses ... future research should test the model... using a larger sample and more objective (i.e., survey-based) data (p. 219). and again, zack (1993): the results provide a foundation on which to build more focused and controlled (e.g., social survey or behavioral laboratory) studies of communication context and its relationship to communication mode characteristics and communication effectiveness (p. 235). in many respects, the study by blanton et al. (1992) is a typical example of a positivist treatment of case research. because prior research on the issues they wished to study was limited, the authors selected a case-oriented methodology. their research was exploratory, suggesting certain attributes of the relationship between it organisational structure and it support effectiveness, and generating three propositions which required further empirical studies to either confirm or deny them. the limitations in applying these propositions were clearly stated; in this case, only to organisations with similar external environments and competing in industries with similar technological sophistication. validity was explicitly addressed through the use of multiple sources of evidence and 26 ajis vol. 3, no. 2 reviews by key informants. a case study protocol was developed, and an expert panel used to analyse the case descriptions, in order to improve reliability by reducing researcher bias.3 a few studies went a step further and attempted to develop theory in some way. some found evidence to support theoretical propositions. others attempted to extend prior theory to a new setting. for example, alavi's (1993) assessment of the effectiveness of electronic meeting systems in a field setting was motivated by the limitations that the laboratory setting of previous studies imposed on their generalisability to more complex settings. sometimes, the study introduced new variables to an existing theoretical base, such as when aydin and rice (1991) suggested that membership in occupational and social worlds also influences individual attitudes towards information systems. other studies constructed theoretical frameworks or models. for example, earl (1993) identified a taxonomy of five approaches to strategic is planning using descriptive classification and inductive interpretation of organisational experiences. reich and benbasat (1990) proposed a multi-stage model of customer-oriented strategic systems success to guide future research. the studies reviewed which had their primary motivation in interpretation attempted to describe, interpret and understand is phenomena, often utilising an alternative perspective to the technical or "rational" view of the mainstream, positivist tradition. for example, hirschheim and newman (1991) challenged the orthodox view that systems development is a normative process reflecting conventional economic reality. as an alternative basis for understanding the actual practice of systems development, they used the concept of symbolism to interpret the social actions and behaviour of developers and users in the development process. many of these interpretive studies attempted to convince the reader of the perceived limitations of the orthodox approach and that an alternative perspective could provide new insights into is problems. for example, wastell and newman (1993) examined the behavioural dynamics of systems development, utilising a stress perspective, and concluded that: the rational-technical view of systems development presented in most textbooks is a caricature. information system development is a complex, protean, social phenomenon; conflict is endemic ... simple prescriptions such as "get top management support" or the pious call for user involvement grotesquely oversimplify the exacting realities of actual systems work (pp. 142-143). the study by orlikowski (1991) which described how the deployment of an information technology affected production workers is a useful illustration of the interpretive approach to case research, one which she terms "organizational ethnography". her study employed ethnographic techniques such as observation of participants, researcher interaction with the information technology, documentation review, social contact, unstructured and semi-structured interviews, over an eight month period.'* she used a theoretical framework to focus her questions and observations, but used no structured instrumentation or statistical analysis of her data. the aim was to understand the implications of the information technology through understanding the norms and meanings that the workers shared around their work, how they interacted among themselves and with the information technology, and how these meanings and experiences changed with the deployment of the information technology. her study highlighted the tendency for information technology to be implemented so as to reflect existing forms of knowledge, legitimacy and authority, thus supporting the organisational status quo. discussion by comparing and contrasting is case research conducted in an interpretive tradition with that of the dominant positivist research philosophy, i have attempted to illustrate the different treatment of case research by researchers utilising different philosophical assumptions. the differences in approach are fundamental. the positivist assumption of an objective, measurable reality, with fixed causal relationships, encourages the use of more structured interviews to collect data for relevant variables. in this way inferences about a phenomenon can be generalised from the study setting to a population. an objective reality also implies that there is one correct answer or truth, so when conflicting answers from the multiple individuals interviewed arose in the study by reich and benbasat (1990), the interviewer "resolved" them during the interview or made a judgement about the final answer based on the individual's perceived reliability. 3. other validation approaches used in some of the studies included seeking support for the research findings by comparing them to prior research in is and other management disciplines, and using multiple interviewers, who compared and cross-checked their notes after each session to increase control. 4. participant observation, although not utilised by orlikowski, would have assisted in weighing intentions and impressions against actual behaviour (orlikowski and baroudi 1991). 27 a.jis vol. 3, no. 2 since in the socially-constructed, subjective reality of the interpretive tradition no one perspective is more valid than another, multiple perspectives are actively sought by interviewing participants who span the organisation's hierarchic levels (e.g. orlikowski 1991) or different levels in the system development process (e.g. hirschheim and newman 1991). less structured interviewing techniques are used to encourage the participants to express and develop their own views unrestricted by the interviewer's preconceptions. conflict and tension in research evidence are exploited to reveal differing aspects of the phenomenon (covaleski and dirsmith 1990). interpretive research does not seek to generalise in the statistical sense (ansari 1991). instead, a deeper understanding of the phenomenon is sought, which can then be used to inform other settings (orlikowski and baroudi 1991). from an interpretive position, the validity of doing so relies on the plausibility of the logical reasoning used in describing the results and in drawing conclusions from them (walsham 1993). the positivist concern with objectivity is reflected in the comment of benbasat et al. (1987) on the attempts of some researchers to generate explanations of their observations: while on the positive side the explanations were grounded in observed facts, our concern is that these facts were filtered through the subjective lenses of the investigators (p. 378). if we acknowledge the subjective nature of all research, a positivist epistemology itself becomes a subjectively selected "lens" through which the world is viewed. researchers need to understand the implications of their research perspective and be open to the possibility of other research traditions (orlikowski and baroudi 1991). this implies that we need to acknowledge the assumptions, beliefs and value judgements that we use in addressing research questions and in making knowledge claims (humphrey and scapens 1992). methodologies informed by a positivist research philosophy tend to treat case research as an exploratory tool for developing hypotheses in the construction of "scientific" theory. by contrast, interpretive methodologies give a more central role to case research, utilising case studies to obtain a deeper understanding of the phenomenon and of the meaning of human behaviour. each perspective places a different emphasis on case research, reflecting different ways of viewing reality and the pursuit of valid knowledge. this is an important realisation that can help us understand the debate that often arises over the use of case studies and appropriate criteria for evaluating them. case studies of organisational aspects of is have a broader role to play than merely as "necessary precursors to more elaborate, large sample hypothesis testing" (humphrey and scapens 1992, p. 1). the contextual complexities involved in studying information systems in organisations and the need to interpret is activities in the context of other organisational activity demand a depth of understanding which can only be achieved through the close involvement of the researcher with the organisation of interest. by conducting research in this way, the researcher is more likely to find a satisfactory basis for understanding and providing meaning to the research subject. references alavi, m. (1993) an assessment of electronic meeting systems in a corporate setting, information and management, vol 25(4), pp 175-182. ansari, s.l. (1991) field research in management accounting: the ethnographic tradition , paper presented at the conference on field research skills in management accounting (american accounting ssociation), boulder, colorado, march 1991. aydin, c.e. & rice, r.e. (1991) social worlds, individual differences, and implementation: predicting attitudes toward a medical information system , information and management, vol 20(2), pp 119-136. benbasat, i., goldstein, o.k. & mead, m. (1987) the case research strategy in studies of information systems , mis quarterly, vol 11(3), pp 369-386. benbasat, i., goldstein, o.k. & mead, m. (1987) letter to the editor, mis quarterly, vol 12(4), p 522. benbasat, i. & nault, b.r. (1990) an evaluation of empirical research in managerial support systems, decision support systems, vol 6(3), pp 203-226. blanton, j.e., watson, h.j. & moody, j. (1992) toward a better understanding of information technology organization: a comparative case study , mis quarterly, vol 16(4), pp 531-555. bloomfield, b.p., coombs, r.. cooper, d.j. & rea, d. (1992) machines and manoeuvres: responsibility accounting and the construction of hospital information systems , accounting, management and information technologies, vol 2(4), pp 197-219. boland, r.j., jr. & o'leary, t. (1991) technologies of inscribing and organizing: emerging research agendas , accounting, management and information technologies, vol 1(1), pp 1-7. cheon, m.j., grover, v. & sabherwal, r. (1993) the evolution of empirical research in is: a study in is maturity , information and management, vol 24(3), pp 107-119. chua, w.f. (1986) radical developments in accounting thought, the accounting review, vol 61(4), pp 601632. 28 ajis vol. 3. no. 2 covaleski, m.a. & dirsmith, m.w. (1990) dialectic tension, double reflexivity and the everyday accounting researcher: on using qualitative methods , accounting, organizations and society, vol 15(6), pp 543573. den hertog, f. & wielinga, c. (1992) control systems in dissonance: the computer as an ink blot, accounting, organizations and society, vol 17(2), pp 103-127. doll, w.j. & doll, m.w. (1992) downsizing at cbs/fox video, information and management, vol 23(3), pp 123-139. dutton, w.h. (1988) letter to the editor, mis quarterly, vol 12(4), p 521. dyer, w.g., jr. & wilkins, a.l. (1991) better stories, not better constructs, to generate better theory: a rejoinder to eisenhardt, academy of management review, vol 16(3), pp 613-619. earl, m.j. (1993) experiences in strategic information systems planning, mis quarterly, vol 17(1), pp 1-24. eisenhardt, k.m. (1989) building theories from case study research , academy of management review, vol 14(4), pp 532-550. eisenhardt, k.m. (1991) better stories and better constructs: the case for rigor and comparative logic , academy of management review, vol 16(3). pp 620-627. feeny, d.f., edwards, b.r. & simpson k.m. (1992) understanding the ceo/cio relationship, mis quarterly, vol 16(4), pp 435-448. ferreira, l.d. & merchant, k.a. (1992) field research in management accounting and control: a review and evaluation , accounting auditing and accountability journal, vol 5(4), pp 3-34. hirschheim, r. & newman, m. (1991) symbolism and information systems development: myth, metaphor and magic , information systems research, vol 2(1), pp 29-62. hopper, t. & powell, a. (1985) making sense of research into the organizational and social aspects of management accounting: a review of its underlying assumptions , journal of management studies, vol 22(5), pp 429-465. humphrey, c. & scapens, r. (1992) theories and case studies: limitation or liberation, accounting and finance working paper 92/4, manchester: university of manchester. lederer, a.l. & mendelow, a.l. (1990) the impact of the environment on the management of information systems, information systems research, vol 1(2), pp 205-222. lee, a.s. (1989) a scientific methodology for mis case studies , mis quarterly, vol 13(1), pp 33-50. lincoln, y.s. & cuba, e.g. (1985) naturalistic inquiry, beverly hills: sage publications. nissen, h-e., klein, h.k. & hirschheim, r. (1991) information systems research: contemporary approaches & emergent traditions, amsterdam: north-holland. orlikowski, w.j. (1991) integrated information environment or matrix of control? the contradictory implications of information technology , accounting, management and information technologies, vol 1(1), pp 9-42. orlikowski, w.j. & baroudi, j.j. (1991) studying information technology in organizations: research approaches and assumptions , information systems research, vol 2(1), pp 1-28. reich, b.h. & benbasat, i. (1990) an empirical investigation of factors influencing the success of customerorientated strategic systems , information systems research, vol 1(3), pp 325-347. scapens, r.w. (1990) researching management accounting practice: the role of case study methods , british accounting review, vol 22(3), pp 259-281. walsham, g. (1993) interpreting information systems in organizations, chichester: john wiley & sons. wastell, d. & newman, m. (1993) the behavioral dynamics of information system development: a stress perspective, accounting, management and information technologies, vol 3(2), pp 121-148. yin, r.k. (1989) case study research: design and methods, rev. ed., newbury park, california: sage publications. zack, m.h. (1993) interactivity and communication mode choice in ongoing management groups , information systems research, vol 4(3), pp 207-239. 29 ajis special edition electronic commerce november 1998 editorial this is the first special edition of the australian journal of information systems. we hope to issue one special edition each year on a topic which is relevant both to the practitioner as well as the academic. this year's topic area is electronic commerce. this issue showcases some of the foremost researchers in the area of electronic commerce and i would like to thank each of them for their contribution. i would also like to thank st george bank for their support in the production of this issue of the journal. our first paper the long term effects of electronic commerce and data interchange on firms francois bergeron, laval university, canada and louis raymond, university of quebec examines the determinants of electronic commerce effectiveness. the results show that a number of factors, including control procedures, imposition can have long term effects upon the organisational acceptance and use of electronic commerce. andrew burton, andersen consulting, chicago and john mooney, university college dublin provide our next paper the evolution of electronic marketplaces: an exploratory study of internet-based electronic commerce within the american independent insurance agency system. this paper draws upon conceptual and empirical literature to develop a framework for analysing potential influences on market and industry structure. this framework is then applied to the american independent insurance agency system. our third paper inter-organisational systems, strategy and structure: the case for planning environments pat finnegan, university of cork, robert galliers, warwick business school and philip powell, university of london looks at inter-organisational co-operation and organisational strategies. the paper examines the development of systems in three inter-organisational networks in ireland, in particular the planning that went into them. our fourth paper analysis of expectations on electronic grocery shopping for potential customer segments heikkila jukka, kallio jukka, saarinen timo and tuunainen kristiina, helsinki school of economics examines the use of electronic commerce technologies in the grocery industry. several infrastructure solutions are examined as bases for electronic grocery shopping in finland. davide khalil, heriot-watt university, edinburgh, presents our fifth paper electronic markets ontology: ideal architecture for a global capital market. this paper proposes an ideal capital market architecture which is both feasible with current technology and functional in terms of the participants and their internal requirements. our sixth paper planning and designing web-based electronic commerce: a case in the insurance industry rolf teubner and stefan klein, university of meunster, germany begins by showing that many web-based approaches have often been disappointing to the users and organisations amploying them. the authors propose two frameworks which are tested on a medium sized insurance company. claudia loebbecke, erasmus university, the netherlands presents the seventh papercontent provision on the web: an economic challenge for tv stations, this paper begins by asserting that many television companies have neglected careful evaluation of the strategic and economic potentials of online services. the paper examinesan exemplary tv network, globo, in brazil. our eighth paper towards new web application development practices angeliki poulymenakou, william drakos, anastasia papazafeiropoulou and georgios doukidis, athens university, greece begins by examining the increasing sophistication of web applications. the authors argue that system developers need to reconsider their professional practices in the context of these new technologies. our ninth paper electronic commerce and collaboration between competing firms eberhard stickel, viadrini university, frankfurt examines investment in electronic commerce. the paper uses microeconomic techniques to analyse the economic incentives pre-competitive collaboration in the development and use of electronic commerce. our final paper is electronic commerce: a battle on regulation, standards, transportation media and business integration kim andersen and niels bjorn andersen, copenhagen business school. this paper examines the effect of regulations, standards etc. on the acquisition and use of electronic commerce technologies. the paper concludes that the effect of many of these factors are still ambiguous. once again i would like to pass on my thanks to the authors and consulting editors for the quality of material in this issue. i would particularly like to thank deborah bunker and david dodds for their assistance in planning and producing this special issue of the journal rob macgregor editor microsoft word ajis vol 15-1 m.doc australasian journal of information systems volume 15 number 1 december 2007 5 ajis featured theme australian women in information technology editor: julie fisher monash university, australia email: julie.fisher@infotech.monash.edu.au in 1992 the first australian women in information technology (auswit) conference took place in launceston tasmania with the theme ‘women in computing – keeping the doors open’. auswit formed with the mission of informing, sharing, community building, and re-energising those involved in the recruitment and retention of women in information technology. since then the conference has successfully run every two to three years in different states across australia. as the conference has grown it has attracted women from europe and elsewhere reflecting the ongoing need for such dialogue to continue. as well, the conference now attracts funding from sponsors such as ibm and google suggesting a growing recognition of the issues and the field. last year in december the auswit conference was held in adelaide. the theme for the conference was "participation one year on". the theme was selected as it was one year on from the federal government's participation summit hosted by senator the hon helen coonan, minister for communications, information technology and the arts. the conference provided an opportunity to reflect on what was achieved by the summit as well as in the year since. . the best papers from that conference have been selected for this special edition of australasian journal of information systems. abs data reported that from 1999-2000, 23% of those enrolled in it courses were women, by 2005 this had dropped to 20%. we know anecdotally that the number of women participating in the information technology industry has been declining in recent years. we need only look around us to see the dearth of female colleagues. additionally the number of women looking to undertake a computing program at the tertiary level is also declining. part of the ongoing debate in the research literature is around why this continues to happen. at least since 1992, australian researchers have been investigating and reporting on issues of gender and it. people such as professor liisa von hellens and sue nielson at griffith university, dr joy teague and professor val clarke then at deakin, dr lorraine staehr and mary martin, bendigo la trobe, myself and associate professor angela scollary then at victoria university, associate professor ruth christie at qut, charmaine ryan university of southern queensland, debbie clayton at the then university of central queensland are just a few of the researchers who have been active in the area over the last 15-18 years. if we reflect on where research has focused we can see emerging trends. the 1992 conference reported research on issues for women in the work force, changing girls’ perceptions of it and how to attract more women to the industry. subsequent conferences published papers along similar themes. in 1998 we published a paper (craig et al. 1998) which reported on a range of initiatives and research in australia designed to support women and encourage women join the it workforce. it is worth reflecting on the early research work that was undertaken. it can be summarised briefly as: australasian journal of information systems volume 15 number 1 december 2007 6 • research which studied the perceptions secondary school girls have of it and how this might be changed. • at the tertiary level, research was conducted into why women are more likely to drop out of it courses compared with men, identifying the factors and how can these issues can be addressed. • once women have entered the it workforce the research focus shifts to the barriers women face, factors that lead to women leaving the industry and programs that support and / or encourage of women to stay. much of the research published at early auswit conferences involved reflections by women working in the it industry, experiences of teaching it in individual institutions, proposals of how things might be changed and examples of programs designed to change attitudes or improve the decline in numbers. reflection and commentary were common approaches taken by the authors. today as a research field, women in it has matured, this is reflected in the fact that although there have only been a handful of papers focusing on gender published, in the australasian journal of information systems, most of these have been published since 2000. we can see the further maturing of the field in australia in the papers presented at the 2006 auswit conference, with a stronger research basis than perhaps, was the case earlier. this special edition is therefore timely. the papers published in this special edition were selected as the best from the conference. each paper has been reworked and extended and undergone a second review process. our reviewers included international scholars in the field and we are appreciative of their time and contribution. the themes that attracted attention in the early years are still common today and highlight the issues of the low participation rate of women in the industry, issues relating to education and the declining number of women studying it. the first paper reports on a significant survey of students involved in online learning at deakin university. coldwell, goold, craig and mustard have analysed the data and identified the different characteristics of female students interacting online compared with the male students. this study is important because of the large data sets with which they were working. they conclude that irrespective of gender students are comfortable and confident working in the online environment. women however, are more likely to participate in discussions compared with males. further the female students placed greater value on using the online environment for communication and collaboration. the second paper by stockdale and stoney presents research conducted in both australia and new zealand universities. the research sought to identify why women are not studying information systems, from the perspective of recent graduates and final year students. both male and female students were involved in focus groups. the research is important because despite universities and academics being aware of the declining interest of women in studying it, little progress has been made on any of the underlying factors that might be causing this decline. the authors recommend a stronger focus on curriculum and better marketing of information systems as possible approaches to arresting the decline. mentoring has long been considered an important support mechanism in organisations. for women in a male dominated industry mentoring can be the critical, providing support and guidance in the workplace. the third paper by login and crumps makes an important contribution to the discussion of why mentoring is important for women and how it can be successfully implemented. the authors' research was extensive involving interviews with 90 women working in the new zealand ict industry. they conclude that mentoring, at least in new zealand, was successful but most relationships developed informally. a significant contribution of this paper is the reflection they provide on mentoring generally and recommendations to ensure success. australasian journal of information systems volume 15 number 1 december 2007 7 the final paper explores the issues facing the tertiary sector with regard to women in ict. lewis, lang and mckay explore some of the reasons behind what they term the ‘invisibility of women in ict’. from theory and their research they reflect on what they have learnt from the swinburne experience and propose a number of recommendations to address the issues they identify. auswit attracts people from across the it spectrum from computer science and engineering to information systems, researchers, primary and secondary teachers and women from industry. the conference provides encouragement to those thinking about a career in it, supports women already in the industry and gives those currently studying good reasons to continue. we are confident that future auswit (now ozwit http://www.ozwit.org/) conferences will be as successful as the last conference and that the quality of research and papers presented will continue to demonstrate the high-quality work that is published in this special edition. we therefore encourage all researchers and other women interested in the area of women in it to continue the good work and produce solid research in the future. craig, a., j. fisher, a. scollary and m. singh (1998). "closing the gap: women education and information technology courses in australia." journal of systems software 40: 7-1 australasian journal of information systems volume 15 number 1 december 2007 8 ajis vol. 4 no. 2 may 1997 the tasks of information systems professionals in philippines, thailand, indonesia and malaysia graham k. winley ang y. ang sim k. lau department of business systems university of wollongong northfields avenue wollongong nsw 2522 abstract this paper presents the results of an empirical study into the present and future tasks expected of information systems professionals in a range of organisations in south-east asian nations. information was collected using a three round delphi study technique across a period of one year. the views of senior personnel working in information systems positions in organisations in these nations are analysed and compared. the results are related to the present and expected future profiles of these organisations and are of relevance to those responsible for developing is curricula in academic institutions as well as commercial providers of education and training courses in these important south-east asian nations. the study also contributes to an emerging body of knowledge concerning the development of the is profession in the developing nations of south-east asia. introduction a considerable body of literature has developed, particularly in the past ten years, which examines the nature of the is discipline, is curricula and the education and training needs of is professionals in an environment which is dynamic in terms of technological innovations and changing organisational structures. many studies have focussed on the design of is curricula and have been influenced by information gathered from is academics and practising is professionals. these studies have identified broad areas of skills and knowledge often categorised as systems development, systems management, technical and user support. some of the major issues identified across these studies include: a perceived gap between is curricula objectives and the needs of practising professionals; the need for graduates to have high level communication skills; the current balance between the emphasis placed on these four categories of skills, especially the balance in curricula between systems management and technical and user support skills; the need to distinguish between appropriate curricula content at undergraduate and postgraduate levels; the need to provide problem-based realistic experiences in is courses and the need to anticipate future developments in is in order to produce graduates prepared for a dynamic environment. the majority of studies relating to these issues have been conducted in a range of developed nations. the following sample of references report on studies of this kind and have been selected because they are quite recent, they have been conducted in a range of developed nations, they address the range of critical issues described above and they include comprehensive references which collectively provide the interested reader with a thorough overview of previous related studies (athey and wickham (1996), cassidy (1990), (1992), crockett et al (1993), deet (1992), fabri and mann (1993), jones and dick (1994), knapp (1993), mis quarterly vol 19, number 3 (1995), monin and deive (1994), quarstein et al (1994), richards and pelley (1994), sallis (1994)). studies conducted in developing nations are less numerous and, based on the same selection criteria, include the recent publications by: ang (1992); ang and winley (1993), ang et al (1993), (1994a), (1994b), (1995); goodman and press (1995), goslar and deans (1993), gupta et al (1994), lu et al (1994), winley et al (1995), yusof and rahim (1994). it is noted that an extensive literature search did not reveal any additional documented studies conducted in developing south-east asian nations similar or relevant to the study reported here. the study presented here examines the tasks expected at the present and in the near future of is professionals in a range of organisations in the developing south-east asian nations of philippines, thailand, indonesia and malaysia. this delphi study, with three rounds, involved senior is personnel in order to obtain an informed view of the role of the is professional in these organisations. a detailed analysis of the results, including comparisons of the views from these four nations is presented in order to address the current lack of documented information about the is profession in these important southeast asian nations and to identify key issues relevant to those responsible for is curriculum development and commercial providers of education and training courses. a common approach to the problem of ensuring that is curricula are in step with the present and future requirements of the profession has been to examine directly the relevance of understandings, knowledge and skills defined in the objectives and content of subjects included in is courses. this approach has usually involved input from the is academic community as well as practising professionals. the present study has adopted a different and complementary approach by examining directly the present and future 101 ajis vol. 4 no. 2 may 1997 tasks expected of is professionals. this information is valuable in ensuring that the objectives and content of is curricula and training courses are specifically designed to address the tasks expected of practising is professionals. this approach has been adopted successfully in a recent extensive australian study (cassidy (1990)). methods and analysis initially forty five tasks were identified as being representative of the range of duties commonly performed by is professionals. these tasks were developed from the most important tasks identified in the extensive australian study (cassidy (1990). as part of the first round of the delphi study participants were invited to add any tasks they considered should be included or to delete any tasks they considered were not relevant. five additional tasks were added and no tasks were deleted at the completion of the first round. consequently, rounds two and three of the study included fifty tasks. senior is personnel were involved in order to ensure a well informed view was expressed. they were asked to rate each of the tasks on a scale from 1 (least important) to 4 (very important) in terms of the duties expected of is professionals in their organisations at the present and in five years time. the three round delphi study approach was used to allow participants to reconsider their views in the light of the results indicating the mean ratings for each task from their nation from earlier rounds and this approach also encouraged participants to be involved and more attentive to the study. the results presented are based on the responses obtained at the conclusion of the third round of the study. each organisation which participated in the study has been provided with the results from this final round. for each of the four nations (philippines, thailand, indonesia, malaysia) the mean of the ratings assigned to each task was calculated together with the associated standard deviation. these means were then used to rank all of the fifty tasks from most important to least important. these ranked lists indicate the relative importance of tasks at the present and for the future in relation to all of the other tasks. in addition, each task was categorised into one of the four task categories (systems development (sd), systems management (sm), technical support (ts), user support (us)) and each task was ranked within each category in relation to the other tasks in the same category again using the means to rank each task within these categories. the allocation of tasks to categories was based on the results of the study by cassidy (1990) and for convenience the categorised tasks together with their rankings in relation to all other tasks and in relation to other tasks in the same category are included in the appendix table a. since the mean for a task tells us nothing about the relative importance of that task in relation to other tasks our analysis focusses on the present and future ranked lists for each of the four nations. consequently, the means for each task are not shown but it is noted that all the associated standard deviations were less than 1 which indicates that for each of the four national groups there was a very consistent view about the rating of each of the fifty tasks. analysis of the level of agreement amongst the ranked lists involved spearman's rank order correlation coefficient ( ) where only two lists were being compared and kendall's coefficient of concordance (w) where four ranked lists were being compared (daniel 1978)). using these techniques the following comparisons amongst ranked lists were made: a) using the ranked lists for the importance of tasks at the present (i) all four nations were compared on their rankings of all fifty tasks and (ii) on their rankings of tasks separately within each of the four task categories. b) the same comparisons were made as in a) but using the ranked lists for the importance of tasks for the future. the five comparisons made in each of a) and b) involved w. c) for each of the four nations their ranked list for the present importance of all fifty tasks was compared to their ranked list for the future importance of all fifty tasks. the same comparison was repeated using the ranked lists for each of the four separate categories of tasks. this involved twenty separate comparisons each involving r. further analysis identified for each national group the ten most highly ranked tasks amongst all fifty tasks and compared these for the present and for the future. in addition, tasks are identified which have changed significantly in relative importance by a movement involving 10 or more ranked positions from the present ranked list to the future ranked list for any of the four national groups. the results of the analyses outlined above together with related discussion are presented in the section on results and discussion which follows a description of the profile of the respondents. 102 ajis vol. 4 no. 2 may 1997 profile of respondents prior to the first round of the study a list 100 to 150 organisations in each of the four nations was compiled using the directory of key business enterprises in the asia pacific region (duns and bradstreet (1992)). in order to involve organisations with a substantial investment and interest in is those with a high financial turnover or a large number of employees were selected from the directory and each was asked to nominate a senior is professional in the organisation who would participate in the three rounds of the study. for each nation about 50% responded and were willing to participate. however, as the rounds progressed there was an attrition rate of about 50% for reasons mostly unknown to the researchers. the most reliable group of respondents were from organisations in the philippines where almost all of those expressing an interest in participating remained involved throughout the study. the results reported here are based on responses from participants who completed all three rounds of the study and involved 45 (philippines), 25 (thailand), 24 (indonesia) and 37 (malaysia). in each of the four nations the majority of participants were from organisations designated as manufacturing/engineering or banking/finance with others representing wholesale and retail, transport and public service organisations. in each case participants represented organisations with a very large number of employees, typically one to two thousand, with some having more than nine thousand employees. each organisation had a very significant investment in is and the respondents were typically is managers (or equivalent) responsible for large is departments most often distributed across several sites. these organisations use systems based on mainframes, minicomputers and microcomputers with a strong emphasis on minicomputers and microcomputers at the present and an increased emphasis on microcomputers expected for the future. local area networking is a dominant feature at the present and although wide area networks are used slightly less frequently at the present they were seen as of significant importance in the near future. there was also significant emphasis on open architectures and client server technology. the majority of organisations currently develop their own applications as well as purchasing systems from vendors although they predicted a decreased emphasis on developing their own systems in the future. they follow standard systems development methodologies when developing their own systems and currently make use of project management and case tools as well as 4gl and application generators. they indicated an increased high use of project management tools, 4gl and application generators for the future and an increasingly important role for case tools. cobol and rpg were the most common languages used with a high and uniform continuing use of word processing, spreadsheets and database software packages. all respondents emphasised the present and increased future importance of databases. results and discussion we begin by considering the level of agreement between the four national groups with respect to the rankings of tasks at the present and for the future. next we compare the present rankings with future rankings of tasks by each national group before examining the tasks that have been highly ranked by the national groups. we conclude with a discussion based on those tasks which have changed significantly in relative importance. each of the tables 1 to 4 which follow involve information derived from the rankings of tasks displayed in detail in appendix table a. table 1: comparison of national groups on the ranking of tasks (w = kendall's coefficient of concordance) rankings compared for: all 50 tasks sd tasks sm tasks ts tasks us tasks present (w) 0.78 0.73 0.78 0.79 0.78 future (w) 0.85 0.85 0.88 0.94 0.59 * all values of w are significant at a level of 0.01. from table 1 we see that there is significant agreement between the four national groups on their rankings of tasks at the present and also for the future. this agreement occurs with respect to their rankings of all fifty tasks and also with respect to their rankings of tasks within each of the four task 103 ajis vol. 4 no. 2 may 1997 categories. this result is supported by the very consistent responses from all the nations to the issues reported in the previous section relating to the profile of the respondents and indicates that organisations in these four developing nations have very similar expectations of is professionals at the present and also for the future. table 2: comparison between each national group's present and future ranking of tasks using rank order correlation coefficients (). comparison of present with future rankings of: all 50 tasks sd tasks sm tasks ts tasks us tasks philippines 0.75** 0.56* 0.76** 0.75* 0.94* thailand 0.74** 0.53* 0.89** 0.88** 0.39 indonesia 0.66** 0.46 0.75** 0.68* 0.96* malaysia 0.70** 0.45 0.86** 0.55 0.94* * significant at 0.05 level. ** significant at 0.01 level. from table 2 we see that for all fifty tasks and for tasks within each category there is generally significant agreement between each national group's present and future rankings of tasks. this is particularly true for the rankings involving all of the fifty tasks and those within the systems management category. since the respondents are senior is personnel, who are most likely heavily involved with tasks in the systems management category as part of their workload, this indicates that they have fixed views that the priorities they assign to their current duties will be much the same over the next five years. with regard to systems development tasks there is neither significant agreement (nor disagreement) between the rankings of tasks within this category at the present and for the future by respondents from indonesia and malaysia. an examination of the rankings of tasks within this category (see appendix table a) reveals that for the future both these national groups have highlighted the increased importance of determining client/user needs, the development of test procedures and the variety of tasks related to the development and maintenance of databases. also both these groups have highlighted the decreased future importance of the range of programming tasks involving program specification, coding, compiling and maintenance. these changes to the priorities amongst systems development tasks are also evident to a lesser extent for the philippines and thailand but are emphasised much more by respondents from indonesia and malaysia. the changes are commensurate with issues noted in the profiles of the organisations particularly the increased importance of databases, the decreased importance of developing their own applications, the use of 4gl and application generators and the adoption of client server technology. there is significant agreement between the present and future rankings of tasks within the technical support category assigned by all the national groups except for malaysia where the level of agreement is not quite strong enough to be significant (at 0.05 level). this is mainly due to the marked future importance assigned by the malaysians to tasks related to consulting with internal and external groups and maintaining networks. they also highlight the decreased future importance of tasks involving the identification, selection and maintenance of hardware. these views are shared by the other national groups but have been emphasised strongly in the rankings of technical support tasks by the malaysian respondents. it may be that the respondents regard the detailed technical aspects of hardware selection and maintenance as being outside of the responsibilities of is professionals and of greater relevance to specialised technical staff. however, the emphasis on network activities and consultation with related groups on is issues do conform with the future profiles of is activities in these organisations. tasks within the user support category are ranked in very much the same order at the present as they are for the future by all national groups except thailand and where there is agreement between the rankings but it is not significant. this is very much a result of the movement of the task related to analysing training needs being moved from last to first priority by the thai respondents. the other national groups did not change the priority of this task very much at all but for reasons that are not clear this task was seen to have quite varied future importance being ranked 4, 1, 3 and 2 amongst the five tasks in this category by respondents from philippines, thailand, indonesia and malaysia respectively. 104 ajis vol. 4 no. 2 may 1997 many of the previous observations from tables 1 and 2 are confirmed and highlighted by the following discussion of tasks that were highly ranked (i.e. in the first 10 positions) and those that changed significantly in relative importance (i.e. rankings changed by 10 or more positions. table 3: highly ranked present or future tasks category/task systems development determine client/user needs document information needs identify problems in the existing system identify user functional requirements define data requirements design the data model design database code and compile program maintain existing programs systems management identify information strategy identify key support systems for business environment provide advice to management establish procedures to restore the database after system failure plan for development, implementation and installation phases manage network plan and manage project teams technical support identify and select hardware alternatives design local area networks maintain network keep abreast on new technologies user support assist users present p x x x x x x x x x t x x x x x x x x i x x x x x x x x x m x x x x x x x x x x future p x x x x x x x x x x t x x x x x x x x x i x x x x x x x x x x m x x x x x x x x x p = philippines, t = thailand, i = indonesia, m = malaysia. in table 3 a cell marked x indicates that the corresponding task was ranked in the first 10 positions by respondents from the identified national group. a blank cell means the task was not ranked in the first 10 positions by that group. it is important to note that when considering these highly ranked tasks none of the fifty tasks in this study were considered unimportant or irrelevant by these is professionals. for example, assisting users has a high priority for most respondents at the present but not for the future. this however, does not mean that these is professionals consider that this task is not a duty they will be expected to perform satisfactorily in the future. from table 3 in the systems development category there is general agreement amongst the national groups that identifying problems in the existing system and user functional requirements together with designing databases are very important tasks at the present. for the future they continue to emphasise the design of databases and assign very much increased emphasis to determining client/user needs and documenting associated information needs. these indications are not unexpected particularly in a predicted organisational environment where many more users are networked and require access to information presented in a form which matches their particular needs. also with less in-house systems development predicted for the future is professionals can re-focus some of their attention from dealing with problems in existing systems to meeting satisfactorily the information needs of users. 105 ajis vol. 4 no. 2 may 1997 for the systems management tasks at the present most agreed emphasis amongst the national groups focusses on providing advice to management, establishing procedures to restore the database after systems failure and planning the phases of systems development. advising management, database activities, identifying information strategies and key business support systems together with managing networks are the agreed major activities for the future. we note that for the future there is increased agreement amongst the national groups as to which systems management tasks are most important. this reflects increased emphasis on networking, a continuing emphasis on databases and the resulting need for strategic planning for the future development of distrubuted is resources. the technical support task of keeping abreast on new technologies is predictably very important at the present and for the future and again we note the continued emphasis on networking related tasks in this category. at the present there is general agreement that assisting users is a very important task. however, none of the national groups ranked any user support tasks amongst the most important 10 tasks for the future. as noted above, this does not mean that such tasks are unimportant or irrelevant but that amongst competing priorities these tasks are not included amongst the most important future tasks. table 4 identifies tasks which have either increased (i) or decreased (d) significantly in relative importance across the present and future ranked lists for any of the national groups. a significant movement corresponds to a change in rank of 10 or more positions. where necessary in table 4 the notations i(d) have been used to complete the cells in a row of the table and indicate a slight increase (decrease) in relative importance respectively with n representing no change in relative importance. table 4: significant changes in relative importance category/task systems development determine client/user needs document information needs identify problems in the existing system write functional specifications prototype systems, including screens and reports design the data model review structure of database write program specifications code and compile program prepare and run test data maintain existing programs systems management identify information strategy identify key support systems for business environment identify and prepare overall technical architecture plan define human, equipment or machine resource requirements provide advice to management write cost/benefit analysis plan a preventative maintenance program manage network negotiate contracts with vendors audit computer systems plan cost reduction programs write a request for proposal from vendors and suppliers technical support act as a technical consultant for internal or external groups identify and select hardware alternatives identify systems software alternatives philippines n i d d i d i d d d d i i i d i i i i d i i d i d d thailand i d d d i i i i d d d i i i d i i i i d i i i d d d indonesia i i d d d i i d d d d i i i d i i i i d i i d d d d malaysia i i d d i i i d d d d i d i ' d i i i d i i d i d i 106 ajis vol. 4 no. 2 may 1997 identify and select software application alternatives maintain existing equipment design wide area networs design local area networks maintain network quote the cost of an item of equipment keep abreast on new technologies user support identify other business units affected by project prepare and/or maintain user documentation assist users analyse training needs d d i i i d i i i d i d d i d d d i i d d i i d i i i d i i i d i i d i i i d d i d d i amongst the systems development tasks in table 4 we see significantly increased importance being assigned to reviewing the structure of databases and the adoption of prototyping as an approach to systems development. this is in line with the continuing importance of other database activities and the increased use of 4gl and application generators identified in the profile of these organisations. programming tasks related to specification, coding, compiling, testing and maintenance have decreased significantly in relative importance along with writing functional specifications and the identification of problems in existing systems. as noted earlier, this is in accordance with less in-house development of systems and more emphasis on strategic planning and management tasks for is professionals in the future. the majority of tasks in the systems management category have increased in relative importance and in most cases the increase is significant. tasks related to defining human or equipment requirements and preparing and negotiating proposals with suppliers and vendors have decreased in relative importance. in fact activities related to dealing with vendors and suppliers are relegated to almost last priority by all the national groups for the future. this may reflect the often difficult and risky nature of such tasks and the need for such negations to be carried out by some other functional group within the organisation. it is noted, however, that determining equipment and other resource requirements are tasks that these is professionals continue to see as important aspects of their duties. the systems management tasks that have increased significantly in relative importance relate to strategic planning issues and increased management control rather than emphasising technical systems development and implementation issues. these strategic planning and management control tasks include planning cost reduction and preventative maintenance programs, identifying key business support systems and business wide information strategies as well as auditing systems and managing networks. the previous comment is supported by the significant decrease in the relative importance of technical support tasks related to identifying hardware and software alternatives and the maintenance of existing equipment. keeping abreast of new technologies is of significantly increased future importance and the future importance of networking is highlighted by the increased importance assigned to the tasks of designing and maintaining wide and local area networks. these key issues have already been detected in previous results and in the profiles of the organisations. it is interesting to note that for thailand only there is a significant decrease in the relative importance of designing local area networks (but not wide area networks). from appendix table a we see that at the present this task has a much higher priority in thailand than in the other nations but in the future all groups have this task amongst those in the top 16 positions. it would appear that local area networking in thailand may be receiving much more attention at the present than in the other nations and that within five years thailand will be comparable to the other nations in terms of their local area networking requirements. assisting users is a very important user support task at the present in all of the nations. the fact that this task decreases significantly in relative importance for the future for all national groups has been commented on previously and it must be remembered that it still has a reasonably high priority for the future. the user support tasks related to analysing training needs and identifying business units affected by a system development have significantly increased future importance and this supports some earlier comments about the general tendency for important is tasks in the future to reflect the need for is professionals to take a broader view of the organisation's information needs in terms of strategic planning, management and support. from appendix table a we see that for the present respondents from philippines, thailand and indonesia have ranked the user support task related to preparing and maintaining user documentation quite lowly at about position 36 our of 50 while for malaysia it has a much higher priority at position 21. respondents from the philippines and indonesia indicate that this task will be more important in the future and moved it upward while malaysian respondents moved it 107 ajis vol. 4 no. 2 may 1997 downward to about the same relative position as all the other nations except for thailand who actually decreased its importance even further giving it very low priority for the future compared to the other groups. thus, perhaps with the exception of thailand, it is reasonable to argue that this task is seen to be an important is task for the future but probably not as important as one might have expected. conclusion this study has focussed on the relative importance assigned to a wide range of is professional tasks related to systems development, systems management, user and technical support by respondents from organisations with considerable investments in information systems in the developing south-east asian nations philippines, thailand, indonesia and malaysia. the relative importance of these tasks at the present and in five years time is presented and provides insights into the stage of development of the is profession in these developing south-east asian nations as well as the key areas where present and future curriculum planning and training should focus in order to foster that development. the main issues that have emerged from the analysis and discussion of the results indicate that in these nations there is a strong need to support the increased development and management of wide and local area networks, databases, prototyping approaches to systems design, an awareness of new technologies, training for users and techniques and tools to assist the is professional to strategically plan, manage and control the information needs of an organisation's business units. there was clear and consistent evidence based on the responses from each of these nations that key is professional tasks in the future will move away from detailed technical aspects of programming, systems design, implementation and maintenance and focus on strategic planning, management and control of the whole organisation's information needs. systems development tasks will not be abandoned but instead will hopefully be done more efficiently and effectively through the use of prototyping methodologies, case tools, 4gl's and application generators in an environment where projects are better managed and wherever possible applications are purchased from vendors rather than being entirely developed in-house. it is noted that the same issues have been identified in other studies relating to so-called developed nations (e.g. cassidy (1990)) and if possible these developing nations should endeavour to learn from the experiences and the approaches that have been adopted in these developed nations. those readers interested in particular tasks or categories of tasks are referred to the section on results and discussion for detailed comment as well as the information in appendix table a. it is hoped that this information, especially the identification of tasks that have a high priority at the present or for the future, will be of assistance to those designing is curricula and training programs. acknowledgements this study was completed with support from the research group: information systems in organisations at the university of wollongong, australia. financial support was provided through a research grant from the australian research council. references ang, a., 'information systems curricula: a south-east asian perspective', journal of computer information systems, 32,4, pplo-19, 1992. ang, a.y., winley, g.k., 'information systems education: comparison of views from australian and southeast asian academics', journal of computer information systems, xxxxiv, 1, pp30-36, 1993. ang, a.y., winley, o.k., fan, j.p.o., 'information systems education: a malaysian industrial perspective', international business schools computing quarterly, 5, 2, pplo-21, 1993. ang, a.y., winley, g.k., lau, s.k., 'information systems curriculum: a comparison of the views of professionals in australia and malaysia', international business schools computing quarterly, 6, 2, pp!4-19, 1994a. ang, a.y., winley, o.k., lau, s.k., 'towards an information systems curriculum for developing asian nations', proceedings apitite'94, brisbane, pp507-513, 1994b. ang, a.y., winley, g. & lau, s.k., 'information systems curricula in the people's republic of china' submitted to journal of information systems education, 14pp, 1995. athey, s., wickham, m., 'required skills for information systems jobs in australia', journal of computer information systems, vol xxxvi, number 2, 1996. 108 ajis vol. 4 no. 2 may 1997 cassidy, m.j., education and training needs of computing professionals and para-professionals in australia, vol one: data and findings, vol two: methodology and appendices, department of employment, education and training, canberra, australia, vol 1 1990, vol 2 1992. crockett, h.d., hall, g.r., jeffries, c.j., 'preferred information systems skills: are undergraduate is programs serving their markets?', interface, vol 15, issue 2, 1993. daniel, w., applied non-parametric statistics, houghton mifflin, boston, 1978. department of employment, education and training (deet), report of the discipline review of computing studies and information systems education, 3 volumes, canberra, australia, 1992. duns and bradstreet., dun's asia/ pacific key business enterprises 1992, duns and bradstreet services, sydney, australia, 1992. fabri, t., mann, r.a., 'a critical analysis of the acm and dpma curriculum models', journal of computer information systems, 34: 1, 1993. goodman, s.e., press, l.i., 'computing in vietnam: an asian tiger in the rough', communications of the acm, vol 58, no 1, 1995. goslar, m.d., deans, p.c., 'a pilot study of information systems curriculum in foreign educational institutions', journal of computer information systems, 34: 1, 1993. gupta, j.n.d., wang, p., ravichandran, r., 'assessment of information systems education needs in taiwan', international journal of information management, 14, 1994. jones, s., dick, g., 'business information technology program at university of new south wales: meeting the needs of industry?' ifip transactions a: computer science and technology, a54, 1994. knapp, j.a., 'information systems: educational offerings versus industry needs. how well do they match?' proceedings of the 1993 acm sigcpr conference, 1993. lu, m., qiu, y., lu, d.h., 'information management issues in the prc: a comparative study', asia pacific journal of information management, 2, 1994. mis quarterly, volume 19, number 3, 1995. monin, j.d., deive, p.j., 'skills in an environment of turbulence: a survey of information systems professionals in new zealand', proceedings of the 1994 sigplan conference, 1994. quarstein, v.a., ramakrishna, h.v., vijayaraman, b.s., 'meeting the it challenges of business. the changing mandate for mba's', information systems management, spring, 1994. richards, m., pelley, l., 'the ten most valuable components of an information systems education', information and management, 27, 1994. sallis, p.j., 'management perceptions of is research and development issues', asia-pacific journal of information management, 2, 1994. winley, g., lau, s.k., ang, a.y., 'a pilot study of the tasks of information systems in indonesia' to appear, journal of computer information systems, 1995. yusof, m. rahim, m.m., 'problems in computer-based information systems development: malaysian perspective', asia-pacific journal of information management, 2, 1994. appendix table a indicates the rankings of tasks in relation to all other tasks (all) and in relation to tasks in the same category (within). where necessary tied ranks have been untied in the usual manner with ranks of 1 through to 50 corresponding to a decrease in relative importance. 109 ajis vol. 4 no. 2 may 1997 :nrs vi h cm o sd ea -3 cq •4 as —i 3 h za a os .2 cs s 1 •g •3 p t/3 y 'n a ~ a. as c/3 >d * .ag 3 -3 4 s "3 r-: o '5 q :—n" c a t e g o r y /t a s k £3 5 ^ 5 a 3 ^ b 3 * 3 ^ —< a •5 ĵ j —<; •3 > __ <£ a ^ _ <jj _g ir ^ _ <£ s y st em s d ev el op m en t _ ^ cs ^ vi r-h 5 — _h no m "•* en on r*h u-i .̂ -h ~ , d et er m in e cl ie nt /u se r ne ed s en on cs m en in en m t-̂ ^ ĥ *"̂ ^ _ •— ' m in,_• n 1-1 in m cs d oc um en t i nf or m at io n ne ed s r̂ 8 ' cs m o en cs oo on — • in — ĥ in m in oo id en u' fy p ro bl em s in tli e ex is tin g sy st em in m •n 00 in in cs •n oo no m cs' en o oo no cs in m * in m id en u fy u se r fu nc u' on al re qu ire m en ts m on en en ^ m en m tf m on cs 0 _^ en m no in ™^ m en in en in vo cs in m ocs w rit e fu nc do na l sp ec ifi ca tio ns m on en en in en en •n r~ lh ^ ^̂in vg r̂ in en in en en m on m ocs pr ot ot yp e sy st em s, in cl ud in g sc re en s an d ^e po rts m on en en m en en m r~ rrr̂ ™^ in ŝr m en m en en in on in 0 cs pr ot ot yp e sy st em s, in cl ud in g sc re en s an d re po rts ^ en f-h m in m o cs vo m ^ on 00 oo "h vo encs en m en cs cs d ef in e da ta re qu ire m en ts m m m 00 in in m d cs in en m vo m ĥ 1-1 en ên vo m cs m ^̂̂ ^ in 0 1 d es ig n tli e da ta m od el ] cs 00 en o m — i m rf cs ^ cs m vd cs ^ * m m en _,. i d es ig n da ta ba se i oo m c)en o en on cs on on cs vo f-h ^ ^ in in in cn j^vo u-i vo x ooen r ev ie w s tr uc tu re o f da ta ba se m •t' in "* m m in̂f m r~ r1̂ csen — _h en m en ^ m cs' cs m ~~" m m cs w ri te p ro gr am sp ec ifi ca tio ns vo "h vo vo oo m o m en cs m r-en on m on on m oen vo m o in in oo c od e an d co m pi le pr og ra m s _^ *"** en in en en en ~" •n on cn en inen — _^ en •n in in cs tf in en in cs 2 ^ •<t w ri te p ro gr am te st in g pl an f or e nt ir e sy st em or in di vi du al pr og ra m s in •t in "* 06 m o en cs in cs s vnen 2 in en •<t 2 0•̂ cs in cs 2 m vo en i d ev el op te st p ro ce du re s m cn m on en in oo in c3 en m tt in on cn cs in en en ^̂ co vo en cs m on" on r— » en m t̂ cs c/2 32 "5 ajis vol. 4 no. 2 may 1997 .-o y x— i => r z a m 2£ — a </i ea •̂•i e3 y •3̂ 5 § fs s 'sc —ol s3 -) .3 § 3 j -3 =3 2 r̂ mu 'na i c a t e g o r y /t a s k c •3 :j f —. ^ c 5 ^ _ <c 2 3 * ^ ^ 3 ^ —< 3 > h! •5 * =: <: -5 —̂i ^ 3 * = s y st em s m an ag em en t m cm m ov m cm m 0 tf *0 o\ " £ 2 ^ m cm m in ^ in o\ o\ oo 2 m ai nt ai n ex is tin g pr og ra m s m cm m oven cm in o ^ \0 0\ ^ s q £̂ m cm en "1 ^ ""; o-> on oo 2 m ai nt ai n ex is tin g pr og ra m s „ „ in cm cm cm ~̂ m cs m v) oo *> "1 2; m m in o en cm id en tif y in fo rm at io n st ra te gy en en -' cm m •* m —~ cm m ~̂ — \o "1 2; in ĵ. in 2 id en tif y ke y su pp or t sy st em s fo r bu si ne ss en vi ro nm en t m o cm cs 2 ên in o\ in m cm m m cs o\ •̂ in en m tt ^̂̂ m r-en m in id en tif y an d pr ep ar e ov er al l t ec hn ic al ar ch ite ct ur e pl an m oo m m •m ^ »-m m f̂ m ov 0 ĉm \o m m oo ov r-o\ cm in o fli d ef in e hu m an , eq ui pm en t o r m ac hi n re so ur ce r eq ui re m en ts -am ĵ.' m ^ _ m ô cm oo o\ ••• in cm m cs m m ^ — in f̂ . oo in cs pr ov id e ad vi ce t o m an ag em en t •01 ' in en "* s m m m m m "̂* c: om m vd m en o\ m in oo e o m 1w ri te a f ea si bi lit y st ud y en 1 ' m o — m m cs en m >̂ en in m en tt in o in $ — m r-en in m in 1w ri te c os t/b en ef it an al ys is cm cm in •t' ^ v£, m *"̂ "t in m cm m cs in en cm m f«̂ m in " 2 es ta bl is h pr oc ed ur es re st or e th e da ta ba se af te r s ys te m f ai lu re m \o „ "~ ' o m cm cm cm m cm en o\ ^̂cm rin o fm tf m ^ o a pl an a p re ve nt at iv e m ai nt en an ce p ro gr am m 0 cmcs in cm in "* m ~^ in m oo r— m m ^ __ m in — 00 pl an f or d ev el op m en t im pl em en ta tio n an d in st al la tio n ph as es m *> ,̂ rt en m _ — en \o \o in en \o \o o\ in cm i m an ag e ne tw or k ^̂̂ m en in in o ^ 0in in i— i in en rjin *̂ in oo o\ en ^̂ inen n eg ot ia te c on tra ct s w itl i v en do rs 111 ajis vol. 4 no. 2 may 1997 mcn h 3 u. h z ti) c/3 w at e. c« w3 >, cfl •5 « « % t3 *— ( t3 i "3 1w3 0) s o 1̂0, cs c/3 >l •a £ .3 * 3 •n i r[— i on (1) s d aa. c a t e g o r y /t a sk c 3 ^̂[̂ e__ < a 3 ^ ^̂ s € ^ — . < e 3 >̂?"_ < c 3 _̂ < c 3 = c 3 ^ —* < 1 ^̂" —< s y st em s m an ag em en t in so os 00 ^h en os in sb in ,-).' r— « ^ m so so m in en m o *m pl an a nd m an ag e pr oj ec t te am s cs so cs cs 00 cs ĵ. en ^ cs' m,_; "* m en *"h in en en 3 t̂ o 2 ^ ^ a ud it co m pu te r sy st em s m t— 2 m f̂ en m os in en cs cs u-i * in sb ™̂* in en * s 5 os en en in cs' m os en pr es en t r ep or ts a nd de m on str at io ns in oo in in oo so *"* in ^ m os m en en cs m m en cs oo en oo in os cs s ,-. r̂ pl an c os t r ed uc tio n pr og ra m s r. m * r-t̂ ^ m m m -̂ so so m en " ' m en — csen m m in oo m cs in os en w rit e a re qu es t f or pr op os al f ro m v en do rs an d su pp he rs c •3 <^^ [̂j —. < ,& •s ^ 3̂ ^̂ b 3 ^ -—< 3 "̂ —< c 3 ^ " —< c 3 ~ g ^ _ < g •̂* —< i t ec hn ic al s u p p o rt so so cs os en ^ m 00 m o * m cs cs 0 m enn00 m en" os en en 00 cs en a ct s as a t ec hn ic al ^c on su lta nt fo r i nt er na l •% e xt er na l g ro up s 00 os cs ^ m o en t̂ oo en oo os cs m ĉs en in ^ m os cn "* m ocs id en tif y an d se le ct ha rd w ar e al te rn at iv es so so cs so m m cs so ên so ĉs t en in in os m m cs cs in m en cs id en tif y sy st em so ftw ar e al te rn at iv es ^ ^ m m cs cs m in csen ^ os en cs cs rso cs m en os cs >n cs id en tif y an d se le ct so ftw ar e ap pl ic at io n al te rn at iv es os m en 00 m r»; en in oo m 0 os os en cs m os ^ m ^ m os cs f_ en m ai nt ai n ex is tin g eq ui pm en t so so cs ^ in ĵ.' ĥ ^ ^ — * ^ m in cs oo in in en os o f̂ in os cs os en en d es ig n w id e ar ea ne tw or ks cs , en cs -h en .̂— i en so * .̂cn in m os ĥ m en en in so d es ig n lo ca l ar ea ne tw or ks en ^ cs os cs m _^ cs !~> so oo cs — ' en cs in o so m r~ cs i m ai nt ai n ne tw or k t-h om o os tt ,̂ oo '3m c5 m oo r}os os en o t-h oo o m t̂ en ^ oo *3" q uo te tl ie c os t o f a n ite m o f e qu ip m en t 0 ootfr — 0 m o in so m c3 in oo ^ — 0 in ^ o in _h m so 0 l-h m m i w rit e te ch ni ca l re po rts ajis vol. 4 no. 2 may 1997 l. u, l. z u u a; "• .2 03 s i ^̂ ~3 a 1 fc/5 0 'e u 's a. ••z 13 s 1 o « -3 3 ^ h 1 f̂ac a t e g o r y /t a s k s 2 "> 5 •5 « ^ •5 > _ , < •n ^ _ < •3 i •3 ^ —< c 3 p̂ •* * i —< i t ec hn ic al s up po rt | — . 5 t̂— m m tn "* fn m m m os m l/"l k ee p ab re as t o n ne w te ch no lo gi es c -a *• 3 •5 ! ^ ^ _ < •5 _̂ * •3 5 3 ^ -̂ « < _ 3 ">••* * •5 «• < ~oaa 3 u in ^ rm oo ™̂ m sc r1"c. os fn ^ 5 s in oo fn m tn m £ fn vn os fn id en tif y ot he r bu si ne ss un its a ff ec te d by th e pr oj ec t ro 8 in rr r«i 5 in o rr fn m in' fn tn fn in r*f' in fn m r— m ô 1pr ep ar e an d/ or m ai nt ai n us er d oc um en ta tio n — tn fn -' oo " ' fn vn ^ m fn oo — fn „ f̂n m 1/5 uiu 3 en fn fn m m i/-j fn — m ."7s fn ^ oc m ~' fn in ey> fn m m ' cn *j i a na ly se tr ai ni ng n ee ds vo »n r-. ""' tt u-i o rr \o ^ ^ \0 5 so o ^ m m , on f̂ pr ep ar e an d m ai nt ai n m at er ia l f or t ra in in g m vn o * " v) so <nj<3' m oo * û o m m tn ^ c on du ct tr ai ni ng se ss io ns ajis vol 4 no. 1 editorial this is the first issue of volume 4 of the australian journal of information systems. as editor, i am pleased to report the on-going interest in the journal, both from authors and more recently from a number of international conferences who have kindly agreed to advertise the journal to their participants. in this issue we have 9 articles, 2 from australia, 1 from australia and malaysia, 2 from the usa, and one each from germany, the uk, singapore and hong kong. our first paper 'a pattern-oriented approach to a methodical evaluation of modeling methods' by michael amberg, department of business information systems, university of bamberg, germany describes a patternoriented approach to evaluate modeling methods. using a set of defined patterns, the strengths and weaknesses of modeling methods regarding methodical aspects can be identified. the next paper 'how strategic are strategic information systems? by david avison, department of management, university of southampton, uk, alan eardley, school of computing, staffordshire university, uk and phillip powell, information systems research unit warwick business school, uk argues that many examples of strategic information systems reported in the literature are not sufficiently critical in determining whether the systems meet the generally accepted definition 'strategic. the paper examines eight information systems. akhilesh bajaj and sudha ram, department of mis, university of arizona, contribute a paper entitled 'a content specification for business process models'. this paper suggests that the problem with most process models is that ultimately the user need to integrate several models to completely represent the business process, and that in some cases business processes contain concepts not found in software processes. the paper presents an alternative model aimed at overcoming the problems of other techniques. the next paper 'information systems success in small and medium enterprises: an evaluation by australian public accountants' by dieter fink, school of management information systems, edith cowan university examines the adoption of packaged software by sme's from the perspectives of user practices as well as design features. a number of important findings, including the need for better documentation and user education, are identified. the next paper 'data warehouse architecture for dss applications' by v.j. anand and himawan gunadhi, department of decision sciences, national university of singapore examines the integration of disparate databases into a single store. the authors provide a method for maintaining a data warehouse of current as well as historical data. trevor moores and ralph champion, department of information systems, city university of hong kong contribute a paper entitled 'a methodology for measuring the risk associated with a software requirements specification'. this paper presents a six step metrics based methodology which eliminates the use of subjective probability assessments. the authors show that the class of risks can be tailored to a given organization using a delphi method. bonnie nardi, apple research laboratories, california, contributes a paper entitled 'concepts of cognition and consciousness: four voices'. this paper considers theories of cognition and consciousness in four traditions: neuroscience, cognitive science, activity theory and the distributed cognition approach. the next paper is 'focusing information systems post-graduate research projects' by gail ridley, department of computer science, university of tasmania. this paper reports on an investigation of mechanisms that assist information systems post-graduate research students to focus their projects and concludes that systematic expert review is considered by most students to be more effective than the doctoral consortium process. the final paper in this issue is 'a partial test and development of delone and mclean's model of success' by peter seddon, department of information systems, university of melbourne and min-yen kiew, rating agency, kuala lumpur, malaysia. this paper critically examines four of the six is success constructs developed by delone and mclean. the paper shows that factors such as systems quality, information quality and usefulness are found to explain 75% of the variance in the user satisfaction measure. once again i would like to pass on my thanks to the authors and consulting editors for the high quality of material in this issue. i would also like to thank david dodds for his assistance in producing final copy of the journal and the maintenance of the home page. r. macgregor editor home page: http://www.uow.edu.au/public/faculties/commerce/business/ajis/ajis.html ajis vol. 6 no. 1 september 1998 tool based psychology as a philosophy of technology irina veienikina' & edward gould2 'russian academy of sciences 2department of business systems, university of wollongong email: e.gould@uow.edu.au abstract this paper examines the influence of psychology in the area of human computer interaction and information technology in general. cognitive psychology is critically examined and its shortcomings in guiding technological development outlined. as a possible alternative to the cognitive approach the vygotskian psychological framework of activity theory introduced. this russian developed theory may provide a more extensive basis for human computer interaction and holds some promise as a framework for the development of computer systems more compatible with the humans that use them. keywords activity theory, human computer interaction, psychology introduction the basis for a philosophy of technology has always been hampered by a tendency to use the most 'advanced' machine of the day as a paradigm for human cognition as b0dker (1997) points out, it was once the steam engine but in the last few decades the computer has provided an example of a machine which is seemingly able to perform mental tasks in contrast to the preceding 'all-manual' machines. it has even given rise to a branch of psychology known as cognitive science. this concept of the machine as a model for human cognition is a rather curious reversal of the true role of any machine, that of as an assistant or mediator of daily activities. there is no reason to regard the computer as anything more than a machine developed from a long line of tools used to assist human activity. background the explosive growth in the use of information technology is having a profound effect on the work practices of humans. in order to increase the benefits of information technology computer systems and their interfaces must be designed as tools which are compatible with the capabilities of their human users. much research has been aimed at providing technicians and programmers with a proper picture of the user, most specifying the psychological features which should be taken into account when developing an interface. however, there is still no agreement on the optimal psychological approach to this complicated area. up to recent times western cognitive psychology has been the main influence in the development of information technology interfaces but attempts to apply it to human computer interaction have had limited success due to the narrow focus of this discipline. it has concerned itself more with the cognitive process and projected this into a philosophy of human-computer interaction which is based on the computer analogy of input-processing-output to describe the functioning of the human mind. the reason for this is the apparent unanimity between this branch of psychology and the architecture of the computer both in concept and vocabulary. there is a readily identifiable compatibility between human and computer "components" when computer devices such as the keyboard, memory and vdu are compared to disembodied human devices such as the eyes, brain and fingers. this tempting analogy has led to a concept of human-computer interaction (hci) which treats the computer and its human operator as equals. the logical extension of this line of reasoning is that ultimately it would be possible to build a computer with the ability to outstrip the human mind. this has given rise to fields dedicated to the 'cybernaut' quest such as the study of artificial intelligence and knowledge based systems. they have mostly had limited success although recently the perception of their ability has been exaggerated by the development of a machine capable of humiliating the world chess grandmaster. philosophically the dream of the technophile was realised in this chess contest, the role of the machine having been reversed from that of a tool for enhancing life to that of a machine which seeks to dominate it. the practical outcome of this general approach is that any experiment to examine the compatibility of a computer with a human is restricted to an interaction between these two components alone to the exclusion of the context of the real life interaction of the human work activity. this can be seen in the classic hci experimental model shown in figure 1. 136 ajis vol. 6 no. 1 september 1998 experiment human i — » computer figure 1 hci experimental model is there an alternative model? this general cognitive approach has led hci experimenters to a frustrating lack of design tools suitable for predicting human reaction to information technology. it is essential to explore the processes where information technology intercedes in the work practices of humans. a branch of psychology developed by lev vygotsky in the beginning of this century provides a different philosophical approach and opens new perspectives for constructing experimental research into this problem. vygotskian psychology and activity theory take a much broader view of human psyche as being the product of cultural and social forces (vygotsky, 1978). from its perspective people are not a collection of cognitive processes but in a complex interaction with the world directed to the process of living. in this approach the main feature of the psyche is the active position of human beings toward the world in which they live. this world and its social context is referred to as 'objective reality' and consists of all the things (objects) which contribute to human existence such as events, happenings, interactions, etc. humans are continually changing the objects and creating artefacts tools. this complex interaction of individuals with their surrounding has been called activity and regarded as the fundamental unit of analysis of human psyche (a.leontiev, 1978, c.tolman, 1988). activity is motivated by the objects to be changed. object orientedness and mediation by tools is one of the most distinguishing characteristics of activity. tools are seen as having extended human ability to achieve the goals of an activity, that is, to change objects in the world. this theory treats tools as a means of satisfying real needs and achieving corresponding goals. this leads us to a different experimental scheme, one with many more elements which emphasise the role of the computer as a tool embedded in human activity, both mental and physical. social context objects/motives 4 i i i t elements of human activity 4 computerr figure 2 activity theory framework cognitive psychology as a philosophical basis for technology recent trends in hci research heavily favour exploration of the cognitive processes, due to what is seen as the qualitative new form of artefact embedded in the use of the computer the automation of brain work compared to the automation of hand-work. many of the problems associated with the information processing model of cognitive science, evident in such areas as expert systems and artificial intelligence, can be seen to be related to the incompleteness and narrow base of this cognitive approach. computerised automation, where the operational aspects of human mental work are removed from the 'human sphere' and transferred to technology, are part of, and not 137 ajis vol. 6 no. 1 september 1998 significantly different to, the transference of human physical work to technology which has been going on for thousands of years. when operations are carried out by technology, it still realises the wishes of the subject, and hence involves the human psyche. the first real possibility of building a nervous system computationally equivalent to the human mind was elevated from the realms of science fiction by the construction of electronic computers in the late 1940's. the view of a computer as a 'thinking' machine was based on turing's concept of thinking as an algorithmic process with the compelling analogy between human and machine of input, processing and output. this view, held by many artificial intelligence theorists who see computers as substitute humans with identical capabilities is, however, fraught with problems. as karpatschof (1992) explains: 'it is meaningless to expect a full correspondence between a human and a machine. not only is a machine lacking in the most important human characteristics, the machine has also many attributes we don't find in a human being. that is actually the reason for making it in the first place.' dreyfus and dreyfus (1986) also share similar beliefs when they say that: 'computers as reasoning machines can't match human intuition and expertise, so in determining what computers should do we have to contrast their capacities with the more generous gifts possessed by the human mind.' in spite of comments like these, a great deal of effort has been put into the field of artificial intelligence over the last decades without, it seems, ever turning to the various branches of psychology as a source model for ' human intelligence', the very thing they were purporting to duplicate artificially. cognitive science cognitive science emerged in the mid 1970s out of a realisation that a multi-disciplinary approach was required if we were to understand higher mental processes and structures. it attempts to bring together what is known about the mind from many academic disciplines: psychology, linguistics, anthropology, philosophy, and computer science. the assumption is that all important human insight can be reduced to principles and rules that machines can think (mccorduck, 1979). cognitive science asks about thought and thinking, about consciousness and computation. in line with cognitive science the concept of cognitive psychology emerged in the usa and uk as an outcome of computer technologies by assuming that the computer can provide a new paradigm for psychology. cognitive psychology therefore, analyses human mental processes with the aim of understanding human behaviour. information theories of perception, attention, memory, emotions and personality are developed. humans are viewed as an 'active process of information eternally striving to sum up and interpret the incoming data and to interpret and reproduce the information stored in its memory through a variety of algorithms and strategies' (tikhomirov, 1988). in this trend psychology at large was declared a science of information processing. although artificial intelligence, cognitive science and cognitive psychology have different aims and methods of investigation, they basically all share an understanding of human thinking as information processing. in addition they interpret cognition or intelligence as the most important constituent property of the psychic processes. when using the concept of cognitive approaches in the following we do not distinguish between the three of them. artificial intelligence artificial intelligence (at) aims at creating computer software and hardware that imitates the human mind or functions of the human brain. the role of the computer is to replace the human in intellectual spheres, such as mathematical calculations, manipulation of numbers and letters, decision-making, problem-solving, and so on. according to feigenbaum (1977) some work towards the construction of intelligent artefacts such as expert systems, which purport to be models of experts' problem solving and expert knowledge automatised in information programs. others view artificial intelligence as 'theoretical psychology' seeking information processing models of human thoughts leading us to a view of ai as the study of 'cognitive' phenomena within the machine. two related points of critiques have emerged. one argument is in opposition to the very rational conception of human beings embedded in the cognitive approaches emerged from a meta-theoretical point of view (e.g. dreyfus and dreyfus 1986). also some empirical studies suggest that the design process is characterised by intuition and imagination as the driving force of the process, rather than a rational problem-solving strategy (aboulafia et al, 1993). in most cases the designers relied upon their intuitive understanding of the system, gained from previous experience. another design study showed similar results. bansler and boedker (1993) found a gap between the way systems development is represented in structured analysis by yourdon and demarco and the way in which it was carried out. designers did not follow the design procedures prescribed but, in general, had a very pragmatic attitude towards using it. 138 ajis vol. 6 no. 1 september 1998 as well as arguing against the limited understanding of problem-solving as being a rational rule-based process of thinking, the studies indicate that the way methods are used is related to the context of application. a general view by the designers in one of the studies were that a main hindrance in modelling techniques was that these techniques didn't take into account social aspects, such as team work and organisational issues (aboulafia et al 1993). they suggested that 'a bag of tools' from which they could pick and choose, would be useful, so they could adapt different methods to their own purposes and integrate these into their own design processes. another criticism is related to the attempt to use the same concepts and methods as properties of both systems and human beings. bansler and boedker's (1993) study found a possible limitation of modelling methods 'inherent' in the methods themselves. the basic idea of structured analysis is to model organisations and work processes as information processing systems with the aim of producing detailed functional descriptions of tasks and operations. there is no distinction made between the way people act and the way machines function. consequently, the method has a number of 'defects' such as reducing problem solving and judgement to mere rule following, ignoring informal communication, underestimating of errors, giving no help to analysing work organisation, etc. and as bansler and boedker (1993) stress, while is makes sense to understand a computer system in terms of data processes, it is doubtful whether it makes much sense to understand work in these same terms. it is becoming more obvious to even die-hard aj proponents that the information processing units in humans and computers are indeed different. much criticism therefore, has emerged of this cognitive science understanding of human cognition, and a question is if it possible to overcome the problem within the framework of cognitive science. limitations of the cognitive science approach winograd and flores (1986) argue that 'contrary to widespread current belief one cannot construct machines that either exhibit or successfully model intelligent behaviour'. they go on to say that 'the essence of intelligence is to act appropriately when there is no simple pre-definition of the problem or the space of states in which to search for a solution. rational search within a problem space is not possible until the space itself had been created, and is useful only to the extent that the formal structure corresponds effectively to the situation.' their analysis is formulated on the philosophy of heidegger which proposes as a basis for understanding cognition should be built around our 'being-in-the-world' which challenges the dominant view of cognition based on the systematic manipulation of representations. brooks (1991) states that 'out of the soup of ideas on how to build intelligent machines the disembodied and nonsituated approach of problem-solving search systems emerged as dominant, at least within the ai community'. search was extended to make use of heuristics to prune the search space and before long the focus became the control of search. no doubt the popularity of search was due to the ease with which its basic tree structure can be programmed on a digital computer but it is this opportunism that has taken it further away from the very thing that it set out to imitate by amplifying the abstraction away from connectedness to the world. he criticises the branch of ai known as knowledge engineering for concentrating much of its energies on anomalies within formal systems which are never used for any practical task. he goes on to say that there is an implicit assumption that someday the inputs and outputs will be connected to something that will make use of them. the heuristic approach to problem solving is examined by dreyfus & dreyfus (1986) who found that only beginners use this approach. as people become more proficient in their field to the level of expert they intuitively move away from using abstract rules to being able to recognise thousands of special cases. brookes (1991) also criticises the view of knowledge as being stored in a way that is independent of the circumstances in which it was acquired. in comparing human and artificial thought he says: 'biological systems run on massively parallel, low speed computation, within an essentially fixed topology network with bounded depth. almost all artificial intelligence research, and indeed almost all modern computation, runs on essentially von neumann architecture, with a large, inactive memory which can respond at very high speed over an extremely narrow channel, to a very high speed central processing unit which contains very little state. when connections to sensors and actuators are also considered the gap between biological systems and our artificial systems widens.' these sentiments are echoed by suchman (1987) when she says that 'the question of theoretical versus practical criteria of adequacy for machine intelligence is rendered moot by the problems involves in constructing a device that 139 ajis vol. 6 no. 1 september 1998 even appears to behave in ways that are purposeful or intelligent'. dreyfus (1992) points out that the belief that both human beings and computers are information processing units presupposes that the mechanisms underlying them are essentially the same. computers, he says deal with facts but humans are the source of facts, in a world of facts created by them in the process of living. he goes on to say: this human world with its recognisable objects is organised by human beings using their embodied capacities to satisfy their embodied needs. there is no reason to suppose that a world organised in terms of these fundamental human capacities should be accessible by any other means.' this concept of embodiment or situatedness has been addressed by cognitivists as a possible solution to the ai problem. given this criticism and concern about the use of cognitive science as a philosophical basis for information technology the next section looks at an alternative approach. vygotskian psychology and activity theory introduction activity theory is a psychological theory that had its beginnings in the german philosophy of kant and hegel and was later developed by the russian psychologists l.s. vygotsky, a.n. leontiev and others during the first half of this century. it is a theory which treats the individual as an outgrowth of social forces rather than the autonomous being. there is evidence of the influence of a great many other philosophers such as janet, wallon, piaget, wertheimer, tolman with his neobehaviourism and particularly john dewey with his work on pragmatism. others include james with his work on substantialism and spinoza, a favourite of vygotsky, with his belief that mental and neuropsychological phenomena were two attributes of the one substance. although vygotsky and his followers were accustomed to citing marx frequently, it is difficult to judge whether his ideas had much influence on the development of the theory, due to the fact that it was generally accepted at the time that marxist-leninist philosophy had to be used as the basis for all branches of scientific study irrespective of whether it had any relevance to the topic being studied. the theory has become familiar to western psychologists through the work of cole and maltzman, 1969, wertsch, 1981, 1994, raeithel, 1990, kuutti, 1990, 1992, engestrom, 1990, kaptellinin, 1992, 1994 and zinchenko, 1992. overview vygostkian psychology (vygotsky, 1978, 1981) stated that human mind is constructed through a subject's interactions with the world and is an attribute of the relationship between subject and object. consciousness is neither reducible to behaviour nor separate from it, but instead is an attribute of the organisation of practical activity. it is the process that organises behaviour. vygotsky also claimed that consciousness is not an attribute of any particular state or process, but is an attribute of the way in which states and processes such as attention and memory, are organised and functionally related both to behaviour and to each other. it was consciousness that established the connection between the various processes, it both creates them and transforms them. in particular the nature of a goal-directed activity transforms its user. he thus introduced the idea of externally mediated activity, actions that involve the use of external means to reach a goal. this led to the idea that mental processes could only be understood if we understand the tools and signs that mediate them. tool mediation a basic tenet of vygotsky's theory was that tools occupy a mediating role, they mediate human reaction with the world. tools therefore are social objects with certain modes of operation developed socially in the course of labour and are only possible because they correspond to the objectives of a practical action. tools can be either external (physical, technical) such as artefacts, instruments and machines or internal (psychological) such as laws, signs, procedures, methods and language. physical tools are designed to manipulate physical objects (e.g. hammer) while psychological tools are used by humans to influence other people or themselves (e.g. concepts, advertisements, calenders). since psychological tools are included in the process of behaviour they alter the entire flow and structure of mental functions. similarly the physical tool (including computers) alters the process of natural adaptation by determining the form of labour operations. a physical tool can be seen as an instrument of labour, a thing which is interposed between a person and the object of their labour and which 'serves as the conductor' of their activity. 140 ajis vol. 6 no. 1 september 1998 although tools expand our possibilities to manipulate and transform different objects they also have a limiting effect in that the object can only be manipulated within the limitation of the tool. his basic idea was of historically evolving object-oriented practical activity carried out by humans determining the genesis, structure and contents of the human mind. vygotsky derived his original ideas from an analysis of the features of specifically human activity work activity, productive activity carried on with tools, activity that is indigenously social i.e. developed under conditions of cooperation and sharing by people. he had isolated the two principal interrelated features basic to psychology. these were the 'instrumented' (tool mediated) structure of human activity and its incorporation into the system of interrelationships with other people. this means that the higher mental functions in humans originate only in the interaction of people with people. vygotsky did not make a complete analysis of the specific concept of activity, but his theoretical approach pre-supposed the concept as one of its fundamental building blocks. the theoretician who followed up vygotsky's work and is most closely associated with the development of a coherent activity theory framework was a.n.leontiev. activity theory structure activity, according to leontiev (1978, 1981) is not a reaction and not a totality of reactions but a system that has structure, its own internal transitions and transformations, its own development. activity brings together in one unit of analysis both internal, psychological phenomena and an external, objective world. leontiev was opposed to the separation of what goes on in our mind with what goes on around us and sought to examine the profoundly unique nature of the psychic phenomena that distinguishes it from other 'objective phenomena of reality' (the real world). he sums it up thus: '...our activity seems to occur as it were on two primordially different (and) opposite planes: • on a subjective ideal plane (i.e. the activity of our consciousness and thought) and • on an objective plane the plane of the phenomena of the material world, i.e. our external material activity, the movements of our body, and those internal processes which are their physiological basis.' the initial and fundamental form of human activity is external, practical activity. the internal plane of activity, inner mental operations and actions, is formed in the process of internalisation. internalisation is the transition in which external processes with external, material objects are transformed into processes that take place at the mental level, the level of consciousness. during this transition these processes undergo special changes they become generalised, verbalised, abbreviated; and most importantly, they become the means for further development that transcends what is possible with external activity. activity here has its general collective meaning, we need to deal with specific activities, each of which answers a definite need of the subject, is directed toward an object of that need, is extinguished as a result of its satisfaction, and is produced again in other altogether different conditions. activity types can differ among themselves depending on the difference between their objects. the object of an activity leontiev says is its true motive, which may be either real or imagined, thus activity does not exist without motive. he called a process an action if it is subordinated to the representation of the result that must be attained, that is if it is subordinated to a conscious purpose. he goes on to say that just as the concept of activity is related to the concept of motive then the concept of purpose is related to the concept of action. human activity he explains, does not exist except in the form of action or a chain of actions with one and the same action accomplishing various activities and may be transferred from one activity to another. this means that activity is usually accomplished by a certain complex of actions subordinated to particular goals that may be isolated from the general goal. in other words, these actions are seen as goal-directed processes but, in society, these processes are rarely related directly to the goal where even the simplest division of labour leads to 'goal-isolation'. the needs of a person working on an assembly line are not satisfied by the immediate results of producing something, but by a share of the results of the collective activity in the form of salary. the actions that realise activity are aroused by its motive but directed towards a goal. the means of accomplishing actions leontiev called operations. just as activity is related to goals, operations he saw as related to conditions. in a case where the goal remains the same but conditions change then it is only the operational content of the action that changes. actions and operations he saw as the 'units' of activity. thus leontiev introduced a three level structure of activity. associated with each level is a specific type of unit, the first of which is the unit of activity (detyatel'nost). here 'activity' means the real activities as opposed to human activity in general and includes such things as labour, formal education and play. it is not determined by the physical context in which humans function but is a socio-cultural creation imposed on the context by the participants. the 141 ajis vol. 6 no. 1 september 1998 next level of analysis is the action (deistvie) which, although related to an activity at a particular time can vary independently of it and is transferable to other activities. a particular action could be executed while a person participates in different activities such as work, play or education. the third level of analysis is the operation (operatsiya) and this relates to the concrete conditions under which the action is carried out and includes the resources for reaching the goal. the goal, which is embedded in a specific set of conditions, comprises the task of the activity. this can be summarised schematically: activity • motive actions goals operations conditions (tasks) this structure can however, be divided in a way which depends on the characteristics of each component and on their inter-relationships which in turn are mobile and changeable. in an integrated activity the within-level and betweenlevel organisations are realised by the integration and unification of the functionally defined processes. human computer interaction and activity theory the diverse nature of the many current cognitive techniques used in hci design and evaluation makes the job of classifying and identifying them as models, methods or architectures difficult. over the last fifteen years a large number hci design tools and techniques have been proposed. howes (1994) has identified twenty three different approaches, each oriented towards a small and manageable part of the overall issue of the way humans interact with computers. the psychological theory expounded by vygotsky and extended by followers such as leontiev and others has been discovered by the international community of researchers in human-computer interaction and computer supported cooperative work over the last ten years. for example see bannon & b0dker, 1991; b0dker, 1991; draper, 1993; engestrom. 1990; gould & verenikina, 1995; kaptelinin, 1992; kuutti, 1992; nardi, 1992, 1997. its generally welldeveloped frameworks may prove more beneficial than traditional cognitive psychology in the analysis of the reaction between humans and machines. kaptelinin (1992) has identified four points which are of central importance to activity theory and the special role that tools in general play within this framework. • the identification of the specific psychological functions of computer tools and how they differ from other kinds of artefacts, • consideration of the psychological nature of the functionally equivalent processes involved, more specifically whether they are activities, actions or operations since this can affect the design and training strategies, • an analysis of the development of users from novice to expert as they develop new skills and abilities with experience, • the relationship between computer tools and cultural influences which can render successful systems in one social setting ineffective in another it stresses the vertical integration of psychological analysis and stresses the following points usually missed by the cognitive approach: • the goal which is determined by the nature of human actions, their structure and performance, • the role of mechanisms underlying the integration of a computer tool into the structure of activity, • the importance of developmental aspects of human computer interaction, • the significance of social interactions surrounding computer use (kaptelinin, 1992). activity theory provides a paradigm for the description and understanding of the way humans deal with computers within the context of the user's environment. recently, hci researchers came to understanding of the importance of the context in which computer-mediated activities take place mainly due to the work of lucy suchman (1987). however, contextual factors are rather elusive and difficult to identify (nardi, 1997, fennel et al, 1997). it is here 142 ajis vol. 6 no. 1 september 1998 that activity theory helps by providing a theoretical framework for describing context of human activity through the structural analysis of the activity, in particular the highest level of it motivation. definition of the motive as the object of transformation within the activity brings together psychological factors (drive, motive, meaning of the activity) and contextual factors (objects as artefacts belonging to a particular social cultural-historical context). in this respect the activity theory approach is not a theory which substitutes for other approaches but it brings a more broader, holistic, integrative philosophical view of the use of information technology by humans. it also should be noted that activity theory shares a few common ideas with contemporary developed task analysis, which presents a detailed description of individual actions and operations. activity theory is based on a much broader set of structures dealing with the higher levels of activity and paying attention to integrative relationships between the actions of the activity. executive information systems individual interface design recently activity theory has been applied to the development of executive information systems interfaces ( hasan , gould and hyland , 1998, verenikina, gould, 1997). this is a particularly promising area for applications since the development of executive information systems (eis) present particular problems for the designer. not only are the clients for whom the system is being designed not known, but they are of a disparate group with different ways of conducting business, making decisions, using data, arriving at conclusions, etc. for an executive to make use of an information system to improve management efficiency an interface compatible with individual management style is essential. software developers have been producing systems for many years which purported to be executive information systems. unfortunately very few of these systems have been successful. the problem cited in the past has always been that the spreadsheet software on which the systems were usually based was not sufficiently sophisticated to deal with the problem. this, however, is no longer the case. very sophisticated multidimensional databases are now capable of extracting complex combinations of the most obscure of data and presenting it in every imaginable way, all at considerable speed. the problem has changed from one of software engineering to one relating to the psychology of executive users. it has long been known that executives manage in their own particular style. it is also well known that these different styles need different types of information presented in different ways. the question is how do we go about finding out the best mesh of information and business management style so that an eis is useful to an individual executive. existing research which studied personalities and tried to apply it to interface design were unsuccessful because of the difficulty in applying psychological studies of personality to this particular case they were too far removed from the problem. one possible approach is based on the study of a personal profile of an executive. this profile will consist of an ensemble of the executive's business activities relating to the use of the eis as a tool embedded in the activities. this will enable the information system to function as a tool for performing the activities and will enable designers to devise ways that an interface can be designed to extract, amalgamate and present information in a way that suits the business management style of the executive. kaptelinin and nardi (1997) have recently created an 'activity check list' for analysing the structure of activities. it is a good example of the operationalisation of activity theory concepts for practical use. an example of the influence of identification of different levels of activities is shown in an information retrieval system design (hyland, 1998). activity theory is used to present a more integrated view of the process of information retrieval and suggest how these problems and their solutions fit together. hyland shows that implicit in much of the information retrieval research is the assumption that the person is engaged in the activity of locating data from a visualisation of some set of the data which is used to navigate and locate that subset which is required. however, using the specific meaning of 'activity' as applied in activity theory this assumption is incorrect. in such cases, the search for data would be an action that is carried out to complete the activity of writing a report or other broader based undertaking. hyland goes on to say: "the actual process of searching, the keystrokes used and so on, are the operations used and these depend on the conditions encountered by the user. this distinction between activity and action is not a trivial one. the supposition that most users are involved in the activity of locating data has led to many problems that have been raised in the literature. for users who have a clearly defined search and are familiar with the system being used, the main concern is that the system provides an appropriate level of recall and precision as quickly as possible. for such users it could be reasonably assumed that much of the search process would be carried out an operational level. for a user with a 143 ajis vol. 6 no. 1 september 1998 poor knowledge of the system, it cannot be assumed that this is the case. such a user may shift the focus of his or her activity to the use of the information retrieval systems, placing the original activity in abeyance for the time being. if systems are to be used not at an operational level but as an activity in its own right, then system developers need to adopt a different focus in their system design. one example of this would be a change in the help that the system must provide to such a user." it is important to distinguish between the various levels of interaction. bannon (1991) has highlighted the case of discretionary users who might use computers to assist with everyday tasks but who have alternative means of accomplishing them. if an information retrieval systems is built on the assumption that all users work at an operational level then discretionary users, for whom the use of the system is in itself a significant activity, will reject the system conclusion psychology has a long way to go before it will be readily accepted by technology developers as a basis for their design. it is possible that part of the problem has been the incomplete model inherent in cognitive psychology although there is strong evidence that most developers pay scant regard to any form of psychology when developing new systems. in vygotskian activity theory designers have a model which is more complete and could certainly provide some benefits to human centred interfaces. a philosophical basis for the inclusion of technology into our lives must seek to place human considerations first a possibility only if sound human psycholgy is available to technology builders. references aboulafia, a., nielsen, j., jorgensen, a.h. (1993) the ambiguous reality and formal methods in user interface design, interchi'93 research symposium, amsterdam. april 1993. bannon, .l. (1991) from human factors to human actors, in greenbaum & king, (eds), design at work, erlbaum, nj. bannon, l. and bodker, s. (1991) beyond the interface: encountering artifacts in use, in carroll j. (ed), designing interaction, psychology at the human-computer interface, cambridge university press. bansler, j. p. and boedker, k. (1993) a reappraisal of structured analysis: design in an organisational context, acm transactions on information systems, vol. 11, no.2, april 1993. b0dker, s. (1997) computers in mediated human activity, mind, culture and activity, 4(3), pp!49-158. b0dker, s. (1991) through the interface: a human activity approach to user interface design, lawrence erlbaum, hillsdale, nj. brooks, r.a. (1991) intelligence without reason, proceedings of the 12th international joint conference on artificial intelligence, vol 1, sydney brown, d.e. and white, c.c. (1990) operations research and artificial intelligence, kluwer academic press, mass, usa cole, m. and maltzman, i., (eds) (1969) a handbook of contemporary soviet psychology, basic, ny. draper, s. (1993), activity theory: the new direction for hci?, intl journal of man machine studies, vol 24. dreyfus, h.l. and dreyfus s.e. (1986) mind over machine, free press (macmillan), ny dreyfus, h.l. (1992) what computers still can't do, a critique of artificial reasoning, mit, cambridge, mass. engestrom, y. (1990) learning, working and imagining twelve studies in activity theory, orientakonsultit oy, helsinki feigenbaum, e. a. (1977) the art of artificial intelligence: themes and case studies of knowledge engineering. 5th intl conference on artificial intelligence ai-77, pp!014-1024. gould, e. and verenikina, i. (1995) population modelling design based on an activity theory, 6th congress on activity theory, moscow, pp 134-138. hasan h, gould e. and hyland p. (eds.) (1998) information systems and activity theory: tools in context, university of wollongong press, wollongong, australia howes, a. (1994) learning to use an interface: articulating the space of models, proceedings of chi'94 research symposium, boston mass. 144 ajis vol. 6 no. 1 september 1998 hyland, p. (1998) exploring some problems in information retrieval: an activity theory approach in hasan, h., gould, e and hyland, p. (eds), information systems and activity theory: tools in context, university of wollongong press, wollongong, australia, pp 93-108. kaptelinin, v (1992) human-computer interaction in context: the activity theory position, proceedings eastwest intl. conference on human-computer interaction, st petersburg, russia. kaptelinin, v. (1994), activity theory: implications for human computer interaction, in brouwer-janse, m.d. and harrington, t.l., (eds), human-machine communication for educational systems design, nato asi series. series f: computer and systems sciences, v.129, springer-verlag, berlin, pp 5-16. kaptelinin, v and nardi, b. (1997) the activity checklist: a tool for representing the "space" of context, personal communication with authors. karpatschof, b. (1992) the control of technology and the technology of control, multidisciplinary newsletter for activity theory, 11/12, international standing conference for research on activity theory (iscrat), berlin. kuutti, k. (1990) activity theory and its application to information systems research and development. in: isra90 proceedings vol. i, ifip tc 8 wg 8.2. working conference on the information systems research arena of the 90's, copenhagen. kuutti, k (1992) hci research and activity theory position, proceedings east-west intl. conference on human-computer interaction, st petersburg, russia. leontiev, a.n. (1978) activity, consciousness and personality, prentice-hall, engelwood cliff, nj. leontiev, a.n. (1981) problems of the development of the mind, progress publishers, moscow. mccorduck, p. (1979) machines who think, freeman and company, san francisco nardi, b. (1992) studying context: a comparison of activity theory, situated action models and distributed cognition, proceedings east-west intl. conference on human-computer interaction, st petersburg, russia. nardi, b (1997), context and consciousness: activity theory and human computer interaction, mit press, boston, mass pennel, r, durham, m, conrad, o. and spark, t.(1997) writing in context: situated learning on the web, proceedings of ascilite '97, perth, wa. raeithel, a (1990) production of reality and construction of possibilities: activity theory answers to the challenge of radical constructivism, multidisciplinary newsletter for activity theory, vol 5/6, international standing conference for research on activity theory (iscrat), berlin. suchman, l.a. (1987) plans and situated actions: the problem of human machine communication, cambridge university press. tikhomirov, o.k. (1988) the psychology of thinking, progress publishers tolman, c.w. (1988) the basic vocabulary of activity theory, multidisciplinary newsletter for activity theory, vol 1 , international standing conference for research on activity theory (iscrat), berlin verenikina, i. and gould, e. (1997) activity theory as a framework for interface design, asdlite'97, perth wa,pp 611-615. vygotsky, l.s. (1981) the instrumental method in psychology. in wertsch, j. v., (ed), 1981, the concept of activity in soviet psychology, armonk, ny: sharpe. vygotsky, l.s. (1978) mind in society: the development of higher psychological processes, harvard university press. warren, c. (1992) hciwhere's the practice?, int. j. man-machine studies , vol 37, p 811-812. wertsch, j. w. (1981) the concept of activity in soviet psychology, an introduction, in wertsch, j. v.(editor), the concept of activity in soviet psychology, m. e. sharpe, inc., armonk, new york. wertsch, j.v. (1994) the primacy of mediated action in sociocultural studies, in mind, culture and activity: an international journal, vol 1 no 4, fall, laboratory of comparative human cognition, ucsd, ca. winograd, t and flores, f. (1986) understanding computers and cognition: a new foundation for design, ablex, nj. zinchenko, v.p.i(992) activity theory: retrospect and prospect, proceedings east-west intl conference on human computer interaction, st petersburg, russia. 145 ajis vol. 6 no. 1 september 1998 book review keith curie caminer, d., aris, j., hermon, p., and land f., (contributors & editors). (1998). leo: the incredible story of the world's first business computer. mcgraw-hill new york. 392 pp. as a book devoted to the pioneering days of computing, this would have to be a nostalgic journey to anyone who was involved in the data processing and computer industry in the 1950s and early 1960s. the four contributors whose names appear on the cover are only some of the early pioneers who have participated in writing their versions of those early days when they were involved in the painstaking development of this, the first computer specifically designed for business. in all, twelve of the team have put their memories on paper and contributed to this book. they discuss the struggle and difficulty of building the leo (lyons electronic office) computer considering there were no true business computers, in fact only a few scientific computers existed and they were mostly sitting in electronics laboratories in universities. in fact the leo computer was a variation of the edsac, a scientific computer developed in the mathematics laboratory of cambridge university. it is interesting to note that the internal political manoeuvrings within the company played a role in the development process and are looked at in retrospect. the writers look at lyon tea houses, an entrepreneurial british company who were the country's largest distributor of food and beverages and were running a nation-wide complex of restaurants and tea houses. lyons had been around since the 1890s and had always been innovative in their approach to business and by 1947 felt the need for a better method of processing their large quantities of statistical as well as other data. by giving cambridge university a grant to enable them to complete edsac they were able to borrow from their expertise when the university's computer was completed and tested in 1949. at this time, the board of lyon tea houses approved the building of leo. david caminer then describes the construction of leo by following the cambridge drawings and modifying parts of the circuitry and architecture to that which suited a business environment. one quote which seems very pertinent is, "the first machine we built, leo mark 1, was based on the logic of edsac mark 1. the philosophy we had was that we would not change anything if we didn't understand why it was done the way it was. so to start with, since we didn't understand very well why it was done the way it was, we didn't make very many changes at all." the description of their experiences is very entertaining, one thing that the reader will note is the very british way in which people are addressed when discussing their roles. in many cases, given names are not used, instead only the formal title of mister or miss. if there is one complaint about the content, it would have to be the way david caminer is intent on mentioning the name of every person he can recollect being associated with the project, even if they have only a minuscule involvement. the first computer was completed in 1951 and once tested was used for running a large statistical program for lyons. surplus time was sold in a bureau type operation programming for which involved not only leo staff but advisers from the user company. when you consider that all the programming in these early days was carried out in machine language it isn't difficult to imagine what a mammoth task it was. of course some subroutines for repetitive operations were written and called when required, but nothing was easy one of the slowest processes was program writing and testing. because of its workload, testing time for new programs was difficult to obtain especially when there was a considerable amount of bureau work for other companies such as the ford motor co. and stewart & lloyds. the most hype in the industry was centered in the usa where ibm were working frantically to match the remington rand univac, a struggle described in the book father son & co. [(1990) bantam] where thomas j. watson jnr., describes ibm's frantic efforts to make up lost ground. probably neither company paid a great deal of attention to the leo project which was technically superior to anything produced at that time by either of the american corporations. for readers who have only entered the computer industry over the last decade or so it is interesting to note that leo was using a stored program (by way of mercury vapour delay lines) and even though input was by way of punched paper tape it was really years ahead of its time. however by 1954 it had run out of capacity and there were many areas of the architecture which the leo staff realised could be improved upon, consequently leo mark ii was given the go-ahead. once again it used paper tape and punched cards for input and mercury vapour delay lines for storage, output however was greatly improved. once this new leo was introduced in 1957 it became the most 146 ajis vol. 6 no. 1 september 1998 advanced stored program computer in the business environment. the most advanced of the ibm computers was the 700 series which used ferrite core memory, but it still wasn't up to the performance of the leo ii. later productions of the leo mark ii/2 replaced the mercury vapour delay lines with ferrite core storage as well. the narrative goes on to describe how the ibm 1401 entered the market in 1959 and orders were taken which made severe inroads into the british computer market. the sales motto being expressed by the aggressive ibm team was, "nobody has ever been fired for buying ibm." in the next phase of the story, the reader is taken through the halcyon days of leo with their second generation model, the leo hi. this was designed in 1960 and by 1961 it had been tested and orders taken. they began selling them in 1962. at this point in time leo iii was the most advanced business computer in the world, with the ability to multi-program and with fast peripheral equipment. later, in 1963 the last of the leo models was produced, the leo 326. leo's were being sold overseas at this time and would probably have continued developing except for the introduction of the ibm 360 together with ibm's aggressive marketing policy. the lyon's company were very profit oriented and were disappointed that they were still not seeing any profits from their computer subsidiary, and sold off leo to the english electric company in two stages. the leo hi and 326 were still produced until 1968 at this time english electric merged with ict to form icl. david caminer's account ends half way through the book whereupon others from the group of development pioneers begin telling their stories. their accounts are mostly associated with applications for specific customers and details how they organised their programming and operations. one of the most interesting sections for me, having come from a background in hardware, was by t.r.thompson and o.w.standingford, two of lyon's executives who travelled to the usa in 1947 to evaluate the state of development of electronic computers. they undertake technical descriptions and drawings of the devices employed at that time and for anyone who remembers vacuum tube operated computers, it is intensely interesting. after reading this section it would be a good exercise to sit down at your pentium 266 mmx with 64mb ram and its cd rom drive and 6gb hard disk, svga screen and possibly a colour scanner and laser or inkjet printer and ponder the progress made since leo mark 1 first ran a program 47 ago. 147 information systems management: an australian view of the key issues graham p. pervan school of information systems curtin university of technology gpo box u1987, perth 6001 email: pervan@bal.curtin.edu.au abstract studies investigating the key issues in is management serve to better understand the concerns of is managers and help to guide is researchers in choosing is management problems worthy of investigation. this paper presents results from a study of the key issues facing the is managers of australia's largest 300 organisations which is part of a three-yearly longitudinal study. in order to gain consensus on the relative importance of the key issues, a three-round delphi method was applied. the results reveal that is strategic planning continues to be the paramount issue in our industry, as are many issues associated with is strategic planning, including responsiveness of the it infrastructure, effective use of the data resource, is for competitive advantage, and a comprehensive information architecture. a greater emphasis on quality is also revealed in highly rated issues such as effective use of the data resource, data integrity and quality assurance, and the quality of software development introduction in the face of expanding information technology (it) capabilities information systems (is) professionals need a greater range of knowledge and skills. the current it environment includes large mainframe computers, minicomputers, microcomputers, powerful workstations, servers and communications networks, local and wide, unking some or all of these together. this range of it supports a variety of applications, development tools (such as case and 4gls), expert systems, database systems and end-user tools which are growing in quantity, range and complexity. the role of the is manager has shifted from being the technical expert responsible for solving technical problems to a provider of information to support organisational objectives and goals. as such, the is manager operates in the nexus between a changing it environment and an organisational environment changing with mergers and takeovers, a recessionary economy, downsizing, outsourcing, total quality management and intemationalisation. this paper describes a recent study of the key information systems management issues facing the information systems managers of large australian organisations. a detailed description of the research method (a three-round modified delphi approach) is provided for information systems researchers with a particular interest in the research methods applied to this research area. the overall results of the study showing the relative importance of all issues considered is then presented and discussed. these results are then compared across sectors of australian industry (government, service, manufacturing). a historical analysis is then discussed in terms of the changes indicated in this study relative to the only previous australian study. finally, the key results of the study are summarised and recommendations for future work are presented. studying the key is management issues identification of the information systems management issues which concern australian is managers in the current technological and economic environment is of value to researchers and practitioners alike. in 1988 such a study was undertaken and published in the australian computer journal (watson, 1989). this new study, undertaken in 1991, is a follow-up of the 1988 study and part of a longitudinal research project to study trends in the issues impacting upon australian is managers at approximately three-yearly intervals. the key aims of this research project are to determine: • which is management issues are expected to be the most important to australian is managers over the next three to five years and thus the most deserving of attention. may 1994 32 • how much consensus exists about the relative importance of specific issues. • why some issues deserve more attention than others. • what trends are emerging in the status of different issues over time. • how the relative importance of these issues compares between australian is managers in different sectors (service, manufacturing, and government). • arising from the above, what aspects of is management should then receive greater attention from australian is researchers. • how the relative importance of these issues to australian managers compares with their importance to their counterparts in the us and uk. all of these aims, except the last, are discussed in this particular paper. the research method in order to provide information concerning the key is issues, a three-round delphi study of the is managers of the top 300 australian organisations was conducted from june to november, 1991. the aim of the study was to identify and rank in order of importance the major issues facing australian is executives over the next 3 to 5 years. the approach taken was identical to the previous (1988) study and based on the methodology used by the mis research center (misrc) at the university of minnesota where three previous key issue surveys have been conducted (dickson et al, 1984; brancheau and wetherbe, 1987; niederman et al, 1991). delphi, a group decision making process, is particularly useful for assessing the judgement of a large, dispersed group of experts (bass, 1983). a typical delphi study starts with an open-ended questionnaire that solicits opinions on the issue of the study. succeeding rounds of the study feed back the previous round's results so that participants can revise their judgment in theory, the delphi process should continue until the group has reached consensus. in practice, however, the researcher frequently decides to end the process after two or three rounds, by which time firm trends have generally emerged. in this particular study the open-ended questionnaire was replaced with the final set of issues from the 1988 study and additional issues which had come to light in the 1990 misrc study (niederman et al, 1991). the previous australian study showed the same issues were of concern to is managers in both australia and the us, though not in exactly the same order. it was felt, therefore, that an update of the previous set of australian issues with newer us issues would provide a fairly complete list of starting issues. this starting list is provided as appendix 1 which shows all 34 initial issues and a one paragraph rationale for the issue provided to all participants so that all would have the same understanding of the issue. the data collection in the previous australian study, modifications were made to the misrc approach. first, participants were asked to rate each issue rather than identify and rank the top ten issues. second, the delphi method was modified to increase response rate. because of the likelihood of a significant rate of attrition in a three-round postal survey, two modifications to the standard delphi method were made. in round two, round one respondents were fed back their results, but round one non-respondents were also fed back the summary results. also, the round three questionnaire was sent to anyone who had responded to either round one or round two. as these modifications kept the response rate stable (at about 25 percent) the same modifications were used in the 1992 study. the value of this approach was further borne out when the misrc acknowledged its usefulness by modifying their own methodology in the same way (niederman et al, 1991, p476). in our 1991 study these modifications helped to maintain the response rate at about 30 percent for each round (see table 1). round one the business magazine, australian business, provides an annual review of the top soo australian organisations. from the issue of december 5, 1990 (parry, 1990), the top 500 (by revenue) was obtained and the largest 300 (on the basis of revenue) selected from these. this number was chosen as 33 ajis a compromise between increasing the sample size from the previous study (from 200 to 300) to improve response rate and cost and time constraints associated with larger samples. the list included publicly listed and private australian companies, foreign-owned companies, and government authorities with headquarters across all states and overseas. the various current city telephone directories were used to obtain the address of the australian head office of each organisation and each head office was telephoned to identify the name and title of their is manager (or equivalent). the 25 highest ranked issues of the third round of the 1988 australian study formed the basis for the first round questionnaire and were augmented with nine new issues used in the 1990 misrc study for a total of 34 round one issues. the issues were then randomly sorted to ensure the previous studies had no influence on the ratings for the 1991 study and a copy of the questionnaire was sent to each of the 300 is managers. a full listing of the issues is shown in appendix 1. participants were asked "what do you consider to be the most critical issues facing is executives over the next three to five years?" and were then-invited to rate each of the 34 issues on a 10-point scale where a 10 represented a highest priority issue and a 1 indicated a lowest priority issue. they were also offered the opportunity to change the wording of any issues or modify the rationale. further, space was provided at the end of the form to write in additional issues and their rationale. after posting to the 300 in the sample, useable responses were received from 88 respondents (see table 1), producing a response rate of 29.3 percent round two after analysing the 88 responses from round one, all 300 of the original sample were sent feedback showing the results of this analysis. the round one analysis simply involved calculating the mean and standard deviation of the ratings for each issue and then sorting them in order of average importance from highest to lowest these sorted lists were returned to the 300 participants and showed the individual issues and associated rationale (in order) with their average rating. all 34 round one issues were retained and the (few) additional issues were examined. there was no commonality among these additional suggestions and they involved issues which were already represented in other forms, so no others were added to the list of 34. round one respondents received their individual responses to each issue in addition to the average rating, whereas non-respondents to round one received only the averages. all 300 were again asked to rate the issues on the 10-point scale. in round two, useable responses were received from 97 participants (see table 1), producing a response rate of 32.3 percent. round three the round two results were analysed in the same way as for round one. the mean and standard deviation were calculated and the issues were again sorted into order of importance from highest to lowest the nine lowest average rating issues were deleted from the list and only the top 25 were retained for round three. in an attempt to explain a significant shift in importance (from 1988) for the issue "specifying, recruiting, and developing human resources for is", two additional issues were included which separated the recruitment element of the human resources issue from the development element. these 27 issues were sent to respondents and they were asked to rate the issues one last time. no non-respondents were included in round three and responses were solicited only from the 126 managers who had responded to either round one or round two (or both). useable responses were received from 88 respondents (after two were discarded as part of the questionnaire was not returned), providing a response rate of 29.3 percent the responses by round are shown in table 1 below. table 1: number of responses for each round round 1 52 12 7 . 17 . 88 round 2 52 . 7 24 14 97 round3 52 12 . 24 88 may 1994 34 representativeness of the sample the target population for this study was the top 300 australian organisations from which 88 finally completed round three. while this represents a slightly higher response rate than the 1988 study (29.3 percent versus 24 percent) and the total number of responses is much higher (88 versus 48), the possibility always exists that the final group of respondents is not representative of the target population. in order to validate the representativeness of the participant sample its profile was compared with the target population in relation to both industry category (alcohol and tobacco, automotive, banking and finance, etc.), type of organisation (publicly listed, non-listed, government, foreign-owned, other), and spread of organisations over the top 300. a jpgoodness of fit test was applied to the distributions in each case. this statistical test provides a measure of how closely a set of observed frequencies (the sample) fits the frequencies expected from a given probability distribution (the population). the sample was found to be representative of the population on all three classifications. in the case of industry distribution the calculated x was 12.4916 and for type of organisation the calculated % was 6.0062, both of which were not significantly different from zero at the 5% level of significance. in order to see that there was an even spread of organisations according to size (measured by the ranking according to gross revenue) the population was divided into six groups of 50 rankings (1-50, 51-100, 101-150, 151-200, 201-250, 251-300). the calculated x2 was 5.125, also not significantly different from zero at the 5% level of significance. in summary, all three tests on the sample indicate that it is representative of the top 300 australian organisations. consensus achieved by the delphi method if a one shot sample had been used for this study, there would have been some danger of a large variation in opinion on the issue ratings and so the use of mean rating to place issues in order may have been subject to this uncertainty. through the use of the delphi method it is argued that the multiple rounds and feedback leads to greater consensus among those surveyed. the average standard deviation (asd) of the issue ratings in the three rounds was calculated. the round one aso was 2.08, which decreased to 1.90 in round two, and finally 1.65 in round three. the decrease in variation in each round confirms that consensus did in fact improve with each round of the delphi. the results of the study the average ratings in each round of the study for all issues in the final round are shown in table 2 below. the issues eliminated after round 2 because of their low importance rating were those relating to factory automation, data and document storage, office automation, relational dbms, image technology, computer graphics, global systems, the asset value of information and software, and, lastly, artificial intelligence. with such a spread of issues there may be a number of reasons for their low importance, but almost all are more technical issues. it may be argued that their technical nature inherently makes them of less concern to is managers, but the more likely explanation is that the is managers are the best judges of what is needed for is and they recognise that the organisational issues are more problematic in the long run for both the is function and the organisation. the issues appearing in the top ten make it clear that the organisational issues are indeed much more important to these is managers. is strategic planning is the top ranking issue. there is increasing pressure to do is strategic planning (henderson and sifonis, 1988) despite the difficulties in doing it well (lederer and sethi, 1992). it can help to identify high return applications, provide better forecasts of requirements, find ways of improving the is function, improve is-user communication, increase top management support for is and develop an organisation-wide information architecture (mclean and soden, 1977). many of the other highly ranked issues, therefore, such as a responsive it infrastructure, aligning the is organisation with the enterprise, effective use of the data resource, is for competitive advantage, and a comprehensive information architecture, all arise from, or are promoted by. effective is strategic planning. in examining the link between the closeness of the is manager (ism) to the chief executive officer (ceo) of the organisation (measured by the number of intervening organisational levels) and the issue ratings, it was found that the strongest (positive) correlations were with is strategic planning, 35 ajis competitive advantage, and information architecture. that is, these issues were more important where the ism and ceo were further apart the implication, therefore, may be that closeness of the ism to the ceo leads to better control over these organisation-wide problems making them (literally) "less of an issue". table 2: average rating for all issues by round issue is strategic planning responsive it infrastructure aligning is organisation effective use of data res is for comp advantage information architecture data integrity/quality ass quality of software dev't role/contribution of is disaster recovery end user computing org learning/use of it eis/dss is effectiveness/productivity telecommunications recr/dev human resources edi/mvi info security & control integrating dp/oa/comms applications portfolio is funding levels distributed systems integrating packages case is influencing org structure recruiting human resources developing human resource round 1 7.931 7.482 7.333 6.780 7.188 7.174 7.024 7.082 6.705 6.902 6.613 6.964 6.855 6.635 6.159 6.566 6.280 6.427 6.500 6.095 5.964 6.305 6.119 5.982 5.805 round 2 7.964 7.840 7.567 7.229 7.537 7.263 7.052 7.211 6.939 6.688 6.823 6.800 6.871 6.813 6.386 6.220 6.113 6.134 6.160 6.033 5.853 5.913 6.140 5.828 5.408 round 3 8.125 7.784 7.773 7.414 7.384 7.273 7.126 7.115 6.909 6.837 6.788 6.779 6.703 6.674 6.565 6.235 6.200 6.153 6.023 5.965 5.965 5.835 5.814 5.552 5.337 5.294 6.518 in examining the link between the closeness of the is manager (ism) to the chief executive officer (ceo) of the organisation (measured by the number of intervening organisational levels) and the issue ratings, it was found that the strongest (positive) correlations were with is strategic planning, competitive advantage, and information architecture. that is, these issues were more important where the ism and ceo were further apart. the implication, therefore, may be that closeness of the ism to the ceo leads to better control over these organisation-wide problems making them (literally) "less of an issue". quality issues have gained greater pre-eminence in organisations, both in management and the development of systems. approaches such as total quality management (tqm) emphasise customer satisfaction and quality of service (tobin, 1990). assuring quality in systems development and data maintenance improves product quality and cost effectiveness (humphreys, 1989). it is not surprising, therefore, that a number of quality issues appear in the top ten in this study, including "effective use of the data resource" (4th), "data integrity and quality and quality assurance" (7th), and "quality of software development" (8th). one particular issue, "specifying, recruiting, and developing human resources for is" was of particular interest. when this issue showed up as a fairly low-rated issue (compare its 1988 rating in section 5) in the first two rounds, two additional issues were included in the final round to see of there was any significant difference between the recruitment and development aspects of this issue. the ratings may 1994 36 shown at the bottom of table 2 show that, as suspected, recruiting staff in the 90's economic environment of high unemployment was much less of an issue that developing the existing staff. the issues of significance to the different sectors of australian industry were then examined. in the final 88 respondents, 7 organisations were from the government sector (mainly airlines and state government insurance providers), 49 from the services sector, and 32 from the manufacturing sector. table 3 below shows the issues appearing in the top ten in these sectors. the list of issues in the table is in the order of the top issues overall, and shows both the average rating for each issue in the given sector and the issue's rank in that sector. table 3: top ten issues in each industry sector issue is strategic planning responsive it infrastructure aligning is organisation effective use of data res is for comp advantage information architecture data integrity/quality ass quality of software dev't role/contribution of is disaster recovery end user computing eis/dss is effectiveness/productivity telecommunications dev human resources government 7.714 (5=) 8.286 (1) 7.857 (2=) 7.857 (2=) 7.143 (9=) 7.714 (5=) 6.571 (14=) 7.143 (9=) 7.857 (2=) 6.571 (14=) 7.143 (9=) 6.423 (18=) 7.429 (7) 5.714 (23=) 7.286 (8) service 8.020 (1) 7.674 (3) 7.834 (2) 7.438 (4) 7.388 (5) 7.061 (8) 7.286 (6) 7.163(7) 6.918(10) 6.937 (9) 6.770 (12) 6.698 (14) 6.708(13) 6.500 (15) 6.292 (16) manufacturin g 8.375 (1) 7.844 (2) 7.656 (3) 7.281 (6) 7.433 (5) 7.500 (4) 7.000 (8) 7.032 (7) 6.687 (14) 6.742(11) 6.733 (12) 6.774 (10) 6.452 (16) 6.867 (9) 6.700(13) the results in each sector are quite similar to the overall results, but not identical. understandably, the service sector's top ten issues were the same as the overall top ten (since the service sector was the largest group in the total sample), though the order was slightly different. in the manufacturing sector, telecommunications squeezed into the top ten, indicating a greater reliance on this technology. executive information systems and decision support systems also squeezed into the top ten in this sector, perhaps indicating a greater concern for productivity at the executive level. disaster recovery rated highest in the services sector, indicating their reliance on critical systems and their sensitivity to down time (for example, in banking and finance). comments about the government sector should be taken carefully since the number (7) and the type in this group may make the sample unrepresentative of the government sector as a whole. however, it is interesting to note that data integrity and quality assurance rated much lower than in the other sectors. further investigation must be made to determine whether this is due to the impact of the government software standard as3563 and, if so, whether its low importance implies a job done well or a misplaced assumption that it is no longer a problem. on the other hand, is effectiveness and productivity rated much higher than in the other sectors, possibly indicating the demand on this sector to perform in the face of political and community pressure. finally, the high rating of the human resources development issue in the government sector may indicate some concerns in that sector with is staff skills and their training programmes. an historical comparison as discussed earlier, this research project closely followed the 1988 study (watson, 1989), using the same methodology, tackling basically the same issues (with a few updates), and sampling from the same population (though the target population was increased from the top 200 to the top 300 australian organisations by gross revenue). a direct comparison between the top ten issues in each 37 ajis study throws light on some changes over that time. table 4 lists the top ten issues in each study and their comparative ratings and rankings. two new issues, responsive it infrastructure and disaster recovery, were both found to be of great importance (ranked 2nd and 10th, respectively). it is perhaps comforting to see that the top ten issues from 1988, except for the human resources issue, all appeared in the top twelve in 1991, so there has not been a lot of change overall. there are quite significant changes, however, in their relative positions, with effective use of the data resource (6.61 and 9th to 7.414 and 4th) and data integrity and quality assurance (6.31 and llth to 7.126 and 7th) both rising significantly. these testify to the rising concern with quality and effectiveness throughout all organisational activities (not just is). in addition, and perhaps as expected, is strategic planning remains the top issue, since, as discussed above, it is central to the effective handling of so many of the other issues. table 4: top ten issues -1988 & 1991 issue is strategic planning responsive it infrastructure aligning is organisation effective use of data res is for comp advantage information architecture data integrity/quality ass quality of software dev't role/contribution of is disaster recovery end user computing orgl learning/use of it recr/dev human resources 1991 8.125 (1) 7.784 (2) 7.773 (3) 7.414 (4) 7.384 (5) 7.273 (6) 7.126(7) 7.115(8) 6.909 (9) 6.837 (10) 6.788(11) 6.779 (12) 6.235 (16) 1988 8.09 (1) new issue 7.18 (5) 6.61 (9) 6.83 (7) 7.42 (3) 6.31(11) 7.20 (4) 6.98 (6) new issue 6.74 (8) 6.38 (10) 7.61 (2) the most significant fall in importance is the specifying, recruiting and developing of human resources for is, which has slipped from 7.61 (2nd) to 6.235 (16th). as previously discussed, the main reason for this decline is economic and this is borne out by the fact that the developing staff aspect of the issue is rated more important (at 6.518) than the recruiting aspect (at 5.294). when there is a tight job market with many applicants, it is understandable that is managers have few concerns about attracting and retaining good staff. conclusions and future work this paper has presented some results from a study on the key issues facing the is managers of australia's largest 300 organisations and has concentrated on the current status and trends in these issues in australia. it has made no comparisons with similar studies overseas. the data presented, however, has been more than rich enough to paint an interesting picture of these issues from a purely australian point of view. is strategic planning continues to be the paramount issue in our industry, and the many other issues it impacts upon also rate highly in importance. these issues, which include the responsiveness of the it infrastructure, effective use of the data resource, is for competitive advantage, and a comprehensive information architecture, are all organisational and signify the growing need for is people to look beyond the technical solutions to technical problems and to pay close attention to organisational requirements. further, there is an increasing need for emphasis on quality and effectiveness in is activities and for the organisation as a whole. the analysis presented in this paper has, of course, some limitations. firstly, no direct comparisons of the results have been made with recent overseas studies in the usa (niederman et al, 1991) and in europe (doukidis et al, 1992). secondly, the government sector element of this study is very small and may not be representative of the sector as a whole. a follow-up study is planned for the commonwealth government (initially) to see how is in our government sector compares with overseas may 1994 38 experiences in the public sector (for example, caudle et al, 1991). finally, the historical evaluation possible from only two australian studies (1988 and 1991) is obviously limited. however, the authors intend to continue using this research method to obtain snapshots on the key issues at about threeyearly intervals. in time, this longitudinal study will provide an interesting picture of the changing face of australian is management acknowledgments i wish to thank dr. richard watson of the university of georgia for providing the research instrument and materials from his 1988 study and for providing advice on the execution of the delphi. further thanks are also due to the western department of state services and the curtin business school for joint funding of this project. finally, i wish to acknowledge the useful comments provided by the anonymous reviewers of this paper. references bass, b.m. (1983) organizational decision making. homewood, illinois: irwin. brancheau, j.c. and wetherbe, j.c. (1987) "key issues in information systems management", mis quarterly, 11,1, march, pp 23-45. caudle, s.l., gorr, w.l. and newcomer, k.e. (1991) "key information systems management issues for the public sector", mis quarterly, 15,2, june, pp 171-188. dickson, g.w., letheiser, r.l., wetherbe, j.c. and nechis, m. (1984) "key information systems issues for the 1980's", mis quarterly, 8,3, september, pp 135-148. doukidis, g.i., smithson, s. and naoum, g. (1992) "information systems management in greece: issues and perceptions", the journal of strategic information systems, 1,2, march, pp 63-75. henderson, j.c. and sifonis, j.g. (1988) "the value of strategic is planning: understanding consistency, validity and is markets", mis quarterly, 12,2, june, pp 187-200. humphrey, w.s. (1989) managing the software process, reading, mass.: addison-wesley. lederer, a.l. and sethi, v. (1992) "root causes of strategic information systems planning implementation problems", journal of management information systems, 9,1, summer, pp 25-45. mclean, e.r. and soden, j.v. (1977) strategic planning for mis, new york: wiley. niederman, f., brancheau, j.c. and wetherbe, j.c. (1991) "information systems management issues for the 1990's", mis quarterly, 15,4, december, pp 474-500. parry, a. (1990) "the top 500", australian business, december 5, pp 88-103. tobin, l.m. (1990) "the new quality landscape: total quality management", journal of systems management, 41,11, november, pp 10-14. watson, r.t. (1989) "key issues in information systems management: an australian perspective1988", australian computer journal, 21,2, pp 118-129. 39 ajis appendix 1 key information systems management issues what do you consider to be the most critical issues facing is executives over the next three to five years? please indicate your views by rating each issue on the scale 1 to 10, where 1 indicates a lowest priority issue and 10 indicates a highest priority issue. the more important the issue the higher the score. use the space below to assign your ratings. please feel free to change the wording of the issues or modify the rationale. space is provided at the end of the form to write in additional issues. your rating key issues and their rationale 1. issue: planning, implementing, and managing office automation. rationale: office automation is being implemented by many organizations to improve "white collar" productivity. problems arise in determining how this should be done and what the role of the is should be. 2. issue: planning, implementing, and managing factory automation. rationale: factories are being automated to establish an internationally competitive position. many functions are involved manufacturing, engineering, research & development, etc. coordination of automation efforts is critical. what should the role of the is department be? how should these new systems be integrated within themselves and within traditional is applications? issue: developing an information architecture. rationale: a corporate/global information architecture is needed to identify the major information categories used within an enterprise and their relationships to business processes. it is necessary to guide applications development and facilitate the sharing of data. issue: improving information security and control. rationale: as organizations increase their dependence on information systems, there is a greater risk from destruction and alteration of data, disclosure to outside sources, and disruption of information services. tight security controls and fault tolerant information delivery are becoming a necessity. issue: improving is strategic planning. rationale: it is increasingly critical to an organization's success that it align its long-range is plan with its strategic business plan. rapidly changing business environments, increased involvement of end-users, and accelerated technological change underscore the need to continue improving strategic planning skills. issue: facilitating organizational learning and use of is technologies. rationale: organizations that prosper will be those that make use of appropriate new is technologies in their entire operation. business practices and organizational structures will need to be modified in many cases. is also must demonstrate its own ability to learn and use new technology. issue: specifying, recruiting, and developing human resources for is. may 1994 40 rationale: current and future shortages of qualified information systems personnel threaten the is department's ability to keep up with the information needs of its parent organization. career paths need to be clarified. more emphasis needs to be put on developing business skills. 8. issue: measuring is effectiveness and productivity. rationale: the measurement of is performance is crucial to its effective management this is becoming more important as organizations invest more and more money in information systems. 9. issue: improving data integrity and quality assurance. rationale: mainframe applications, timesharing extracts, and personal computer applications are all used to analyze data and prepare reports. frequently there are discrepancies among these different data sources due to lack of controls in is and user departments. too often computer printouts are assumed to be accurate. such assumptions can lead to faulty business decisions. 10. issue: improving the quality of software development rationale: the application development backlog remains at unacceptably high levels. users are getting impatient add to this the increasing costs of human resources and the need for improved effectiveness in systems development becomes clear. 11. issue: facilitating/managing executive and decision support systems. rationale: increasing the ability to exploit situations for competitive advantage depends on enhancing the ability of management to "experiment" with decision possibilities. many other issues also depend on this capability. decision support tools have long been viewed as a method for introducing modelling tools to executives to improve their decision making, however those efforts have met with mixed success. 12. issue: selecting and integrating packaged application software. rationale: the changing relationship among costs of hardware, packaged software, and human resources makes managing this aspect of the business critical. using purchased application software can cut costs, boost productivity, and increase user satisfaction, but it can result in poor integration of systems and serious maintenance problems. 13. issue: converting and using relational database management systems rationale: ibm once suggested that its major users convert to relational dbms by 1990. how? is it really necessary for end-user applications? can organizations afford the additional hardware and conversion costs? too little is known about this technology. 14. issue: aligning the is organization with that of the enterprise. rationale: the effectiveness with which is can support the enterprise's information needs is dependent on the is department's position within the enterprise. too often is is not located appropriately within the organization. 41 ajis 15. issue: promoting effective use of the data resource. rationale: the organization's data resource is growing in size, complexity, and value. despite this, it remains largely unrecognized, inaccessible, and underutilized. is must develop a climate within its department and throughout the organization which values the data resources as a corporate asset. 16. issue: enabling electronic data interchange and multi-vendor integration. rationale: electronic communications with vendors and customers may offer tremendous competitive advantage to a company. is executives must push for basic industry standards. standards would enable electronic data interchange and cost effective multi-vendor environments without major conversion costs or loss of functionality. 17. issue: determining appropriate is funding levels. rationale: there is no generally accepted way of establishing the level of is funding relative to the other funding needs of the organization. this can put both is and general managers at a disadvantage. 18. issue: increasing understanding of the role and contribution of is. rationale: is is sometimes viewed as an overhead expense with little appreciation of its contributions to the organisation. this can lead executive management to make infeasible demands and to cut funding resulting in missed opportunities for the organisation. 19. issue: integrating data processing, office automation, factory automation, and telecommunications. rationale: the capability now exists to integrate systems that are based on these diverse technologies. as organizations try to integrate their technologies, organizational and managerial problems will need to be solved. 20. issue: managing data and document storage. rationale: information stored in both electronic and hardcopy form is accumulating at a staggering pace. there is a need to provide for future data and document storage requirements. 21. issue: planning and implementing a telecommunications system. rationale: communication is the lifeblood of the organization. using is for competitive advantage often depends heavily on telecommunications. rapid and major changes in the industry complicate this task. 22. issue: managing the impact of artificial intelligence. rationale: artificial intelligence may be the tool to integrate is with other functions in the organization. its potential is not well understood. although it is becoming increasingly feasible, too little is known about managing this technology. may 1994 42 23. issue: using information systems for competitive advantage. rationale: in many businesses, long-term survival is dependent upon using information systems to gain competitive advantage. the business can be lost without it competitive advantage results from recognition of opportunities through creativity and innovation, followed by rapid implementation. these are historical weaknesses of is. 24. issue: planning and management of the applications portfolio. rationale: the applications portfolio is rapidly increasing in size, complexity, and maintenance cost. despite the longevity of the maintenance problem, too little is known about managing it effectively. 25. issue: making effective use of computer graphics. rationale: computer graphics offer an effective way to present information. problems need to be resolved in using graphics interactively and remotely, and integrating it with standard reporting mechanisms. 26. issue: facilitating and managing end-user computing. rationale: the proliferation of end-user computing through personal computers offers the promise of improved productivity but also the dangers of inadequate management control. information systems management must balance control against the need for slack. clarification of is and end-user roles is a necessity. 27. issue: planning and using case technology. rationale: significant progress has been made automating business functions within organisations; however, a vast productivity gain is possible if the automation process itself is automated. in principle, software systems can provide support for integrating the design efforts of project teams, for standardizing representation methods, and for generating code. while this technology is still being refined, providing support for systems development is extraordinarily complex and will require major changes within the is function. 28. issue: developing and managing distributed systems rationale: distributed systems present a variety of challenges including: establishing and maintaining consistent data; physically connecting computers linked by various networks with seamless gateways; responding to increased demands for joint development projects with users; controlling wasteful duplication of efforts; and establishing guidelines for providing data resources to multiple versus local users. 29. issue: building a responsive it infrastructure rationale: the combination of maintenance problems, evolving corporate architecture, package integration, user-developed systems and quality of the application portfolio will require extensive automation of the it processes required to coordinate action and manage the business. building an integrated technology platform is the key to enterprise and industry productivity. 43 ajis 30. issue: using is to influence organizational structure rationale: it is commonly accepted that new information technologies offer the potential to flatten organizational structure thereby creating a more responsive, flexible, and innovative firm. information systems may also enable new dynamic organizational forms, further improving the firm's responsiveness to react to new opportunities. 31. issue: establishing effective disaster recovery capabilities rationale: down-side risks are increasing daily from the potential loss of business due to a disaster. effective recovery plans must be in place and tested regularly to insure losses are minimized as organizational applications grow and become more integrated, the greater the risk becomes. 32. issue: planning and using image technology rationale: the utilization of image technology is growing in firms. there is an increasing need to integrate these into the is arena and to consider their impact on technology planning and acquisition, hardware and software architecture, and user policies. 33. issue: moving toward global systems rationale: with increasingly global orientation of markets, suppliers, customers, etc. business will require global information planning, designs and implementations to permit effective and efficient operations. this will challenge current systems of data transfer, time concepts, cooperative work, and language utilization. 34. issue: accounting for the asset value of information and software rationale: currently, most internally developed software and information databases are expensed, not amortized. effective long-term management requires an acceptable method of accounting for the asset value of software and databases. may 1994 44 ajis vol. 7 no. 1 sept 1999 constructing multidatabase collections using an extended odmg object model3 adrian skehill mark roantree interoperable systems group dublin city university dublin 9, ireland. abstract collections are an important feature in database systems. they provide us with the ability to group objects of interest together, and then to manipulate them in the required fashion. the oasis project is focused on the construction a multidatabase prototype which uses the odmg model and a canonical model. as part of this work we have extended the base model to provide a more powerful collection mechanism, and to permit the construction of a federated collection, a collection of heterogenous objects taken from distributed data sources introduction the oasis project (odmg architectures for specification of interoperable systems) is focused on the construction of a multidatabase prototype for usage in a healthcare environment (roantree 1998). research is conducted at dublin city university with assistance provided by lona technologies with their common object request broker architecture (corba) products, versant through the usage of their object-oriented database, and st james's hospital through the use of their extensive healthcare applications as a suitable test environment for the planned prototype. this paper describes one of the sub-projects within the oasis project where the collection mechanism of the odmg model was extended to provide three key features. firstly, oasis dynamic collections are automatically updated through our collection manager; secondly our collections are capable of containing heterogenous objects; and thirdly collections from different databases can be merged to form a federated collection. we will refer to oasis dynamic collections as dynamic collections during the rest of the paper. the paper is structured as follows: the remainder of this section briefly describes odmg collections and motivates our need to extend the base model; §2 looks at the design architecture for implementing dynamic federated collections; §3 provides information on constructing local and global collections; §4 describes the construction of dynamic collections; §5 describes the prototype; and finally §6 offers some conclusions. motivation the odmg object model provides a generic collection class, which defines the basic properties for each collection type (catell & barry 1997). specific collection types inherit from this object providing more specific collections. the collections supplied by the odmg object model are listed in table 1. type description set unordered collection of elements, no duplicates allowed. bag unordered collection of elements, which may contain duplicates list ordered collection of elements. array dynamically sized ordered collection of elements. dictionary unordered sequence of key value pairs with no duplicate keys. table 1: odmg collection types the elements of a collection can be of any type, primitive literals, structures and abstract data types, objects and references to objects. however, a collection can hold one type, and all elements in the collection must be of that type. in order to extract information from the collection, an iterator is used. this is a mechanism for accessing elements by traversing the collections. iterators are defined in the standard template library (musser et al 1996) specification. this defines standards for collections, and how to extract information from them using iterators. there are two types of iterator defined by the odmg standard, a generic iterator which moves forward through the elements of a collection, and a bidirectional iterator which allows bidirectional traversals of a collection. 3 this work has been funded by forbairt strategic research grant st/98/014. 84 ajis vol. 7 no. 1 sept 1999 the odmg collection handling mechanism is static. a collection is defined, and must be initialised and maintained by the programmer. the contents of a collection may become invalid for any number of reasons, e.g. objects can be deleted, modified or new objects can be added. our requirement is to export object collections from various hospital information systems to provide the global user with federated collections of objects of interest. although no facility exists to perform manipulations on the data inside federated collections, an import facility can download the federated collection into a local database as snapshot data where oql can be used to execute meaningful queries. notification mechanism uatsoase t •v object cobecbon database objects figure 1: architecture for oasis dynamic collections oasis dynamic collections solve the problem of static collections by allowing the application developer to associate a query with each collection, and then forcing the database to maintain the contents of the collection based on this query. figure i demonstrates the architecture for dynamic collections. each component has a specific task which will be explained over the course of this paper. a dynamic collection can contain different types of objects whereas odmg collections can hold only one type of object. a number of queries can be used to define a collection, and their result is held internally in the collection. this allows a meaningful and diverse set of objects to be stored within a collection. the oasis multidatabase kernel also allows dynamic collections to be merged, thus forming federated collections of data. architecture for dynamic collections maintenance interface the maintenance interface (mi) facilitates the specification of queries for collections. multiple oql queries can be specified for each collection. the mi also requires a name for each collection constructed. • naming the collection. after creating an instance of a dynamic collection, it can be named, so that other applications will be able to easily retrieve a reference to this collection (and hence its contents) from the database. to name the collection, the method name_collection is used. this saves the name supplied in the database, and other applications will be able to query based on this name. • supplying the query definitions. a dynamic collection can have a number of queries defined in order to place heterogenous types inside the collection. after constructing the dynamic collection, any number of queries can be defined. a query is associated with a collection by calling add_query where the query is a string representing an oql statement. if the oql statement is either syntactically or semantically 85 ajis vol. 7 no. 1 sept 1999 incorrect, then an exception is thrown (in c++) and the query is refused. • initialising the collection. once all the queries have been added to the dynamic collection, it can then be initialised. this is a process where each of the queries associated with the collection are executed, and each object reference is added to the collection. it is achieved by calling the initialise method. since this call actually triggers the initial creation of the contents of the collection, it may take some time as it has to execute each oql query defined for the collection, and build a reference to the objects returned. the following code segment, demonstrates how to use the maintenance routines to create, name, define and initialise a collection: example 1 maintenance interface cdyncollection *col; col = new (db_name, cdyncollection); col->add_ query ( "select inamefrom patients where pl5_ test>500 "); col->name_ collection ("p15 failures); col->initialise (); in this example, a dynamic collection cot is created. the name of the collection is "p15 failures", and the query that defines it is added using the add_query method. the initialize method is used to activate the dynamic collection, so that its contents will be updated as changes occur in the database. structural interface the structural interface (si) provides information of the structure of the collection. it provides applications with details of the different types that are present in the collection. calling the retrieve_object_names method returns a list of the names of classes that are stored in the collection. information about each type supplied by the interface can be used to select objects of that type using the data interface. browsers or other mechanisms that display the contents of the collection can use this information to format the display of data from the collection. the following code segment shows how the structural interface can be used. this is based on the code shown in the previous example. example 2 structural interface d_set <d_ref <d_string> > classes = col->retrieve_object_names (); for (i = 0; i < classes .size(); i cout « "class name is < " « classesfi] « "> « endl; } the class d_set <d_string> is an array of strings, and is the storage for the list of types returned by the call to retrieve_object_names. data interface the data interface (di) is used to extract objects (data) from the database. access to data is through the get_obj_list method, which returns object references. the standard database locking mechanism is used to protect the list of objects retrieved. the application that processes the collection will have a lock on the objects to ensure that no other application can delete objects contained within the dynamic collection. if any objects are modified the change will be shown immediately after the commit point to the database. once the lock has been removed from the list of objects returned, get_obj_list will have to be called again to refresh this list. the dynamic collection uses the standard odmg collection mechanism to store its elements. refresh mechanism once a dynamic collection has been constructed, the database engine is used to maintain the contents of the collection. a list of references to objects is materialized and stored so that the collection is not regenerated each time it is used by an application. however, this means that the collection must be refreshed each time an object associated with a collection is created, modified or deleted. recalculating the entire collection each time could represent a large overhead for the database if there were a large number of objects involved. from a refresh point of view there are two issues here that have to be discussed: how the dynamic collection gets notified when 86 ajis vol. 7 no. 1 sept 1999 a change has occurred, and how the collection refreshes itself in such a way as there is minimal impact on the performance of the database. we address both issues in the remainder of this section. notification procedure applications which use an object oriented database generally retrieve objects from the database, and execute methods provided for the classes. since dynamic collections are persistent classes containing object references, the database will not be able to refresh these collections. a refresh operation is quite complex as it will have to determine what objects to change in the collection (if any). an external component (the collection manager) was constructed to refresh dynamic collections, running continuously as a daemon program. dynamic collections provide an interface for the database to trigger a refresh. a method called refresh is invoked by the collection manager (cm) for this purpose, and is discussed in the next section. in order for the cm to refresh collections, a notification (or event) mechanism exists to provide details of change. the more refined this mechanism is, the more efficient the refresh mechanism will be. the odmg object model does not contain a standard for event notification so proprietary event mechanisms must be built. the following features, embedded in the database engine attempt to optimise the refresh mechanism: • the ability to define the events of interest only e.g. instance modification, creation and deletion, of a particular class. • the ability to register the predicates of oql statements which the dynamic collections are based on. only events that effect these predicates will cause a notification message to be sent to the appropriate dynamic collection. • the notification mechanism should supply the object identifier of an object that has been created, changed or deleted. when the database using dynamic collections is started, a daemon process capable of defining and receiving events, the collection daemon (cd), must be started. the implementation of this application is dependent on the database containing the dynamic collections, as different databases have different event notification mechanisms. when the collection daemon starts, it queries the database for each dynamic collection that has been created in the database. for each collection found, the collection daemon will query the collection for 1. the predicates from each oql statement defining the collection; 2. the class name corresponding to each predicate. for each query which defines the dynamic collection, the collection daemon will retrieve the predicates for each oql query. for example, in the following oql query: define extent patients for patient; select p from patients p where p.address.city = cork in this case the predicate is 'p.address.city = cork'. so if a patient moves out of cork, the collection will be notified that this object will need to be re-examined to confirm whether it should remain in the collection, or have its reference deleted. the collection daemon will also retrieve the names of classes corresponding to the predicate contained in the oql statement. in the above example the class name will be patient. the collection daemon uses the event message received from database to determine which collection should be refreshed and uses the collection manager to refresh this collection only. using this mechanism, we obtain the following advantages: • the database engine is responsible for "watching" the objects of interest, which should be more efficient than having an application constantly checking the objects of interest in the database. • only the classes which are of interest to dynamic collections are being monitored. the predicate is a further refinement. the database will only notify the collection for objects satisfying the predicate. 87 ajis vol. 7 no. 1 sept 1999 the collection manager the effort of the collection manager depends on the power of the database's event handling mechanism. with versant (versant 1997) it has a powerful event mechanism which is capable of returning the object reference for modified objects. the collection manager simply checks its collections to determine if the object reference is present, and re-evaluates for that object only. dynamic collections maintain a state designed to provide information on the correctness of the list of objects it contains. table 2 provides a list and description of each of the five possible states. state description ready the dynamic collection has been created but not initialised. the initialise method is invoked to ensure the collection has been activated. activation of a dynamic collection means the collection manager can refresh its contents empty the collection has been initialised, and no objects were returned as a result of the queries. the collection is still active so if any changes occur in the database that affect this collection, the collection will be updated. being_refreshed an event has occurred which has caused the collection to begin a refresh. no data will be available from the collection while it is being refreshed. valid_collection the collection is ready for queries. it has been successfully initialised and has a valid list of object references. coljerror there is an error with the collection and no data is available from the collection. a typical cause of this error is that the structure of the schema has changed, and one of the oql queries is longer valid. table 2: dynamic collect: on states handling schema changes another issue that dynamic collections must address is changes to the database schema. if a type structure is changed so that an oql query will no longer execute (for example, the attribute that the query is based on is removed from the class definition), then the collection will no longer be able to refresh its contents based on the oql query. because of this, the user should check the state of the collection before attempting to access any elements in the collection. if the user attempts to retrieve elements from the collection and the state returned is col_error, then an exception is thrown which the application must address. since dynamic collections deal only with objects references they do not need to know the actual structure of the object types that are referenced by the collection. if new attributes are added to an object, this will not affect the operation of the collection. the collection can be updated using queries on this new attribute by calling the add_query method to add a new query to the collection. constructing and using dynamic collections this section examines how dynamic collections can be used to construct a 'cutdown' federated schema and serve a useful purpose in a multidatabase prototype. dynamic collections represent a convenient method to model export and federated schemes as they refresh themselves when changes occur. figure 2 illustrates how dynamic collections can be used to build a limited form of federated schema. each database has had its local data model converted to the cdm of the multi database system. in this case the cdm is the odmg object model. figure 2 starts at the second layer of the traditional architecture of these systems (sheth & larson 1990) where the component schema resides. in order to construct export schemes it is necessary to query the schema repository of the component schema. construction of a local dynamic collection (export schema) requires knowledge of the structure and meaning of the participating database. when constructing the federated dynamic collection the required information is provided by the schema server objects which provide distribution for local collections. thus, a federated collection becomes a federation of exported collections of objects. 88 ajis vol. 7 no. 1 sept 1999 dynamic collection federated collection figure 2: dynamic collections in a multidatabase environment. constructing the export schema a component schema must be generated for each local database participating in the multidatabase system (pitoura et al 1995, sheth & larson 1990). a dynamic collection can then be constructed to act as an export of data. as the component schema changes, so do the objects in the exported collections. this subsequently provides dynamic updates to federated schemes to ensure that schemes presented to the user is always correct. metadata information is also present in dynamic collections so that a description of exported objects also exists. some process is required to export metadata to the multidatabase kernel and hold information on each database participating in the multidatabase system. this exists as an external application, illustrated as the schema server in figure 2. the schema server allows metadata to be queried and exported to the multidatabase kernel. accessing federated data using both the structural interface and the data interface, a constructing processor (sheth & larson 1990) is able to build a federated schema from any number of export collections. for any database participating in the multidatabase system, the schema server can access the local collections that can be used to construct the federated schema. for each export schema present in the database, the server imports the appropriate metadata into the multidatabase kernel with a unique id for the source of each metadata subset. when the schema server has completed the importing process the mdb kernel will contain: • metadata representing schemes from the participating databases; • the various exported collections. it is necessary for the multidatabase administrator to assist the constructing processor, as decisions will need to be made regarding name overlaps etc. the problems associated with schema integrations can be found in various research publications including (batini et al 1986). at its most simple, it may be possible to define the federated schema as a collection of collections (i.e. export schemes). however, in most non-trivial situations, there will be some mediation required between the contents of the different export schemes that are to be integrated to form the federated collection. federated collections a federated collection consists of a collection of collections from local databases. a browser can easily be constructed to view data from multiple sources. alternatively, the imported metadata can be used to create a 89 ajis vol. 7 no. 1 sept 1999 schema at the mdb kernel which can be used the store actual data, imported from local databases, and construct a multidatabase snapshot. either application will need a bridge to allow the data to be retrieved from local databases. this bridge called the data server, exists as a corba object, and facilitates the supply of data from local sources. for each collection present in the multidatabase collection, the data server will supply the contents of that collection to the calling application. the calling application will be responsible for decoding this information, and performing any additional processing before the data is displayed to the user. application prototype a working prototype has been built using microsoft visual c++ 5.0, orbix 2.3c (1998), and versant 5.0.8 (1997) and runs on a windows nt platform. it comprises a number of components as follows: • collection daemon (cd). this observes any events in the database that may affect the contents of any dynamic collection (c++/versant). • collection manager (cm). when notified by the collection daemon of a change, this application refreshes the dynamic collection(s) affected by the change (c++/versant/orbix). • data server (ds). this application provides a data for application requests (c++/orbix). • schema server (ss).generates the metadata to model the schema from each component database, and creates this information in the oasis kernel database for applications to be able to query (c++/orbix). the last three components (cm, ds and ss) are corba servers, and must reside on an orb which has access to the component database. multiple implementations of these applications can exist, to allow for different platforms, database types etc. however, the interface will be common regardless of the implementation chosen. data was exported from legacy systems at st james' hospital and imported directly into three component schema4 which exist as versant databases. local collections were defined using our extended odmg collection mechanism and subsequently merged to form federated collections. a browser was built to view federated collections. test collection were specified by the hospital based on the imported data set. figure 3: information flow through the protototype 4 a model converter was hard-wired specifically for this task. future oasis prototypes will facilitate both component schemes containing only mappings, and those containing imported snapshot data. 90 ajis vol. 7 no. 1 sept 1999 conclusions in this paper we described our extension to the odmg object model to provide an enhanced collection handling mechanism. this research was motivated by the need to have collections automatically refreshed, and to provide federated collections of objects from distributed sources in a healthcare environment. our work extends the odmg model in three ways: • collections can contain heterogenous types; • collections are automatically refreshed when changes to the database require; • collections from separate databases can be merged to form a federated collection. a prototype has been constructed and collections (supplied by st james hospital) were used to test the system. references batini c., lenzerini m., and navathe s. a comparative analysis of methodologies for database schema integration. acm computing surveys, 3:(4), 1986. catell r. and barry d.(eds). the object database standard: odmg 2.0. morgan kaufmann, 1997. orbiz 2.5 documentation suite. lona technologies, dublin ireland, 1998. landers d. target information systems for the oasis prototype: a description of the systems and solutions for interoperability. oasis report no. oas-07, dublin city university, february 1999. musser d., saini a. and stepanov a. stl tutorial & reference guide: c++ programming with the standard template library addison wesley, 1996. e. pitoura, o. bukhres and a. elmagarmid. object orientation in multidatabase systems. acm computing surveys 27:2, 1995. roantree m. the oasis multidatabase architecture. oasis report no. oas-01, dublin city university, may 1998. a. sheth and j. larson. federated database systems for managing distributed, heterogenous and autonomous databases. acm computing surveys 22:3, 1990. versant c++ reference manual 5.0. versant sofware, 1997. 91 tailor-made concurrency control distributed transactions as a case prof. dr.tcchn. mads nygard the database technology section, sintef delab the norwegian inslilute of technology, trondheim norway abstract this paper applies a model for distributed databases and transactions with a distinction between global and local correctness criteria. the global requirements per system are weaker than the local requirements per site. the paper presents an application which suits such a two-level division. the main motivation for our investigation is based on the fact that the commonly used correctness criteria for concurrency control and recovery, serializability and total recoverability, are very strict criteria. the use of more relaxed criteria (allowing more true parallel behaviour and more true partial behaviour) is therefore very appealing as long as this can be achieved without compromising safety or applicability. the main paradigm in our approach is based on the observation that relatively little knowledge about the databases and transactions can lead to major gains in system throughput. this allows specific systems to have more tailormade correctness criteria. we analyse a specialized type of distributed database, the skeleton-database, and a specialized type of distributed transaction, the wander-transaction. wander-transactions accessing a skeleton-database allow breaks with both the common serializability criterion and the common total recoverability criterion. our main emphasis here is on the nonserializability aspect. the primary goal of this work is to designate correctness criteria for controlling local and global parallelism. the secondary goal is to specify priority rules for handling local and global criteria breaks. wandertransactions accessing a skeleton-database experience dynamic priorities. our resulting concept, priority serializability, gives increased parallelism without compromising safety. introduction background: for many distributed databases the type of global integrity constraints implied by the databases collectively is under discussion. any such constraints come in addition to the local integrity constraints implied by the databases separately. by integrity constraints we mean any associated rules which couple the given database items in some way. likewise, for many distributed transactions the type of global semantic requirements induced by the transactions per system is under discussion. any such requirements come in addition to the local semantic requirements induced by the transactions per site. by semantic requirements we mean any overall information describing the assumed transaction results in some way. see for example elmagarmid (1992), breitbart (1992) and bhargouti (1991). it is a fact that in some distributed databases global integrity constraints do exist but are weaker than the local integrity constraints. it is also a fact that in some distributed transactions global semantic requirements do exist but are weaker than the local semantic requirements. this opens up for non-serializability globally and partial recoverability globally on top of serializability locally and total recoverability locally. context: a knowledge of the concurrency control material in bernstein (1987) and papadimitriou (1986) will benefit the reader but it is not a prerequisite. contents: the "alternative database and transaction types" chapter presents the skeleton-database and wandertransaction concepts and investigates the new notions from a general viewpoint. the "relaxed correctness criteria" chapter then designates the correctness criteria for concurrency control and specifies the priority rules for criteria break handling, while the "mechanisms and examples" chapter investigates corresponding mechanisms and examples. the skeleton-database represents sets of close substitutes for diverse articles. effectively we investigate an organized coupling of cooperating databases. each specific database contains a complementing set of articleoffers and the different databases contain substituting sets for these article-offers. further, the databases may be accessed simultaneously in the sense that article-offers in one database may be "held" while article-offers in another database are checked and the accesses to all participating databases in a skeleton-database will be collectively controlled. our approach allows non-serializability and partial recoverability. we arrive at multilevel correctness criteria with respect to both consistency preservation and atomicity assurance. the wander-transaction traverses a skeleton-database trying to seize an optimal set complying with any combined conditions on the attribute-values of specific articles. actually we discuss a traversing race among september 1995 53 competing transactions. a transaction may require diverse articles of several types and several articles of a single type. and a transaction/buyer may acquire articles directly i.e. it corresponds to an on-line read-andwrite service and not only an off-line read-only service. further, a buyer may consider more static attributes like quality and price as well as more dynamic attributes like no-left and who-purchased. our concept allows transactions to build up priority during their traversing race. acquiring an article-offer early may increase the priority of the corresponding buyer while checking an article-offer later may decrease the priority of the corresponding buyer. alternative database and transaction types skeleton-databases imagine a collection of separately run department stores selling similar types of articles. the current available qualities, prevailing prices, remaining quantities and confirmed reservations of articles naturally vary among the department stores. their article information will have to exist in n variants and may have to be stored at n sites. a typical database corresponding to three department stores offering three specific articles is illustrated in fig. 1. here x, y and z signify different occurrences of sites, while a, b and c signify different types of items. so ya, yb and yc represent different item-types available as article-kinds at a unique site or department store. we talk about a complementing set of article-offers. further xb, yb and zb represent a unique item-type available as article-variants at different sites or department stores. we talk about a substituting set of article-offers. types sites figure 4. complementing sets of article-offers at single sites & a substituting set of article-offers of a specific type consider the article-information in the previous figure as a distributed database consisting of a set of local databases. each set of items connected with horizontal lines corresponds to the intra-base view, while the set of items encircled by a vertical oval corresponds to an inter-base view. the different local databases of this nonreplicated global database are not interrelated by normal integrity constraints. but within each local database there may be any kind of integrity constraints relating the local items in some way. the general picture will be as in fig. 2. 54 ajis global database collection of non-interrelated local databases figure 5. lacking interrelations in the alternative distributed database we are actually specifying an alternative database type where some integrity constraints (i.e. those between the separate local databases in a global database) are completely missing. hence we have complete knowledge about a well-defined part of the integrity constraints. the non-existence of integrity constraints between clearly identifiable database parts is easy to represent and manipulate. we choose to call a global database of this type a skeleton-database. in a skeleton-database there are no inter-site integrity constraints. this means that the substituting set of articleoffers of for example type b (i.e. xb, yb and z,,) has no attached rules restricting the sets of possible attributevalues that may coexist. but there may be some intra-site integrity constraints. this means that the complementing set of article-offers at for example site y (i.e. ya, yb and yc) may have some attached rules restricting the sets of possible attribute-values that may coexist. examples of such integrity constraints are: value(attributei(yb)) > value(attributei(yc)) value(attributej(yb)) + value(attributej(yc)) < value(attributej(ya)) note that a typical database item may have more attributes than the four we have introduced. wander-transactions imagine a simple but interesting operation requiring the cheapest article-variant of type b. the specification of the corresponding transaction will be: ti: acquire onee: lowest-price the meaning of such an access-predicate is that the corresponding transaction should, after first checking (i.e. retrieving) some or all of the available article-offers, then acquire (i.e. update) the one article-offer fulfilling the given condition in the best way. this is illustrated in fig. 3. here the operation checks the current offers by retrieving all existing variants of the specified article-kind, finds one cheapest offer by comparing the prices and consulting the quantities, and acquires the best offer with at least one instance left by updating this variant of the article-kind. 3) retrieve all 0 designate one update this figure 6. actions of the alternative distributed transaction september 1995 55 to check an offer means to retrieve its quality, price, no-left (i.e. quantity) and who-purchased (i.e. reservationlist) attributes. a reservation-list reflects the sequence of reservations already made. to acquire an offer means to decrement the quantity (> 1) by one and add the buyer-identity to the reservation-list. the who-purchased attribute will produce a delivery-list for the department store, so the sequence of buyer-identities is important. the execution result with the article-offer at site y designated as best will be: we are actually specifying an alternative transaction type where the relationship between the readand writesets of the transactions is known (i.e. a reads-before-write case) and where the functional objective of the transactions is fixed (i.e. to acquire articles). hence, we have both some high-level information about the syntax of the transactions and some overall information about the semantics of the transactions. we are particularly concentrating on a specific type of transaction. we choose to call a transaction of this type a wandertransaction. in a wander-transaction there are no inter-site value-dependencies. this means that values read among several sites are only used to decide which subset of items to update and will not be used to calculate which values to write to this subset of items. but there may be some intra-site value-dependencies. this means that values read within a single site may have to be applied to calculate which values to write to its designated subset of items. for example, assume that all the nine database items in fig. 1 in the "skeleton-databases" section are checked, and assume that yb is both the only database item acquired of type b and the only database item acquired at site y. then the values read from the substituting set of article-offers of type b (i.e. xb, yb and zb) are only used to decide that yb is to be updated. in contrast, the values read from the complementing set of article-offers at site y (i.e. ya, yb and yc) have to be applied to calculate which value to be written to this yb. initial illustrations now we only consider the simplest case i.e. with one item per site. in general there are several items per site, and the different sites must not necessarily have entries for exactly corresponding sets of items. let us imagine a skeleton-database containing the following three item-variants: x: quality = price = no-left = 85% 5$ 1 who-pur. = quality = 90% price = 10 $ no-left = 1 who-pur. = z: quality = price = no-left = 95% 15$ 2 who-pur. = the item-variants represent the one and only item-type occurring at the three different sites. or to put it another way: each specific quadruple corresponds to a specific department store's offer of the single article-kind. we look at the following two wander-transactions: ti: acquire one: lowest-price & quality > 85 % + acquire one: lowest-price & quality > 80 % t2: acquire one: lowest-price & quality > 75 % recall that a buyer may be interested in more than one article-instance as t] is here. the item-variants that the two wander-transactions will acquire if executed in isolation are tv x + y and t2: x. a possible schedule corresponding to the two wander-transactions being executed in parallel is: h,= t,: ri(x)w,(x) t2: r2(z,r2(y) r,(y)ri(z)w,(z) r2(x)w2(y) the description of a schedule is split among the involved transactions. thus, the horizontal axis represents "time", while the vertical axis represents "space". that the wander-transactions access the item-variants in different sequences is not important. it has only been employed for simplicity reasons. one reason for not 56 ajis checking all article-offers before acquiring an article (as t! does with x here) is that a very good offer should be acted upon as soon as possible. note the possible race-situation among wander-transactions for a favourable offer.6 the results stem from the facts that initially there is only one instance left of both item-variant x and itemvariant y. when t2 would like to acquire x, there is none left and it has to settle for y. further, when t, would like to acquire y, there is none left and it has to settle for z. the item-variants that the two wander-transactions will acquire if executed in the hrschedule are t,: x + z and t2: y. observe that these are not the same as in the isolated cases. in the "initial evaluations" section we shall see that the hi-schedule will not be allowed by the designated correctness criteria. typical examples of schedules that will be allowed by these correctness criteria shall be shown in the "concurrency control examples" section. inherent freedom we will consider some consequences of defining and using skeleton-databases and wander-transactions. we will specifically look at the possible needs for serializability and total recoverability. the normal correctness criterion for concurrency control corresponds to the one-after-the-other effect. this is illustrated in fig. 4. such serializability means that the effect of handling several transactions being run in parallel should be as if they where executed in some, unknown, serial order. this is a very limiting criterion. its necessary strength stems from the difficulty in knowing, representing and manipulating both general integrity constraints of databases and general semantic requirements of transactions. the lack of some integrity constraints in a skeleton-database leads to a certain degree-of-freedom in synchronizing accesses to such databases. the existence of some semantic information about a wandertransaction then fixes a particular point in this solution span for synchronization of such transactions. consider for instance using locking as a concurrency control mechanism. is 2phaselock synchronization then necessary for wander-transactions accessing a skeleton-database? the answer is no! it is quite adequate to always hold the best offer currently found and constantly free already obsolete offers. holding corresponds to locking and freeing to unlocking. carrying out this principle recursively means that eventually an article is locked after another is unlocked. hence 2phaselocking is violated. <] t2 |r2w2r2w2 n"n"n"nl^7 tk -*• riwirjwi timing of transactions execution effect of transactions execution figure 7. the one-after-the-other effect of transactions execution relaxed concurrency control has been used in for instance garcia-molina (1982) (allowing special arrangements for read-only transactions in centralized systems) and du (1989) (allowing special arrangements for local-only transactions in distributed systems). our aim is to arrive at relaxed concurrency control for all wander-transactions. effectively, true parallel behaviour may still give sensible results. this will be the main theme of the "relaxed correctness criteria" and "mechanisms and examples" chapters.^ the normal correctness criterion for recovery corresponds to the all-or-nothing effect. this is illustrated in fig. 5. such total recoverability means that the effect of dealing with a transaction being interrupted in the middle should be as if it was finished totally or as if it was not started at all. this is again a very limiting criterion. its 6. hence "lowest-price" or "highest-quality" is not always to be interpreted literally as might be indicated in the "wander-transactions" section. see also later examples. 7. note that we require normal serializability between the group of "selling transactions" and the group of "buying transactions" and within the group of "selling transactions". it is within the group of "buying transactions" that we may allow some non-serializability. september 1995 57 necessary strength once more stems from the difficulty in knowing, representing and manipulating both general integrity constraints of databases and general semantic requirements of transactions. rw rw ! halting of transaction execution effect of transaction execution figure 8. the all-or-nothing effect of transaction execution wander-transactions accessing a skeleton-database may apply partial commitment instead of total commitment. this means that at particular points during its execution a specific transaction may commit its hitherto purchased items instead of having to commit all its purchased items together at the end. each purchase corresponds to a single write, so committing a set of purchases corresponds to committing a set of writes. a reason for applying partial commitment instead of total commitment is that at particular points a specific transaction may be able to live with the items acquired by then without necessarily having to get hold of the remaining required items. allowing such step-wise committing transactions does not mean that corresponding subtransactions become totally independent. actually these will be priority-connected: early writes in a specific transaction build up priority for later writes in the same transaction. further, early commits secure purchases from being aborted if the transaction waits too long. partial commitment has been applied with nested transactions, and this context has been analysed in bancilhon (1985), beeri (1986) and korth (1988). our point is not that partial commitment is to be used but rather to discuss how it should be controlled. effectively, even true partial behaviour may still give sensible results. nygard (1994b) has covered this. relaxed correctness criteria basic concurrency control we must first look at the transaction conflict patterns that may occur corresponding to two transactions accessing a single item-variant. here wr, rw or ww indicates a write-read, read-write or write-write conflict respectively see bernstein (1987) or papadimitriou (1986). one or more "&"s couple a set of conflicts that have to occur together in a specific schedule. in contrast, one or more "/"s separate sets of conflicts that have to occur alternatively in a specific schedule. further, a conflict marked with an "*" contributes in the reverse way of a non-marked conflict. hence, if a non-marked conflict contributes ti— »t2, then a *-marked conflict contributes t2— >ti. each separate alternative is accompanied by a corresponding example schedule. the set of possible results is as follows for the reads-before-write case: both read-only => <none> ha = t,:r,(y) t2: r2(y) one reads-only, one reads-and-writes => rw / wr hb = t,:r,(y) t2: r2(y)w2(y) t,:r,(y)w,(y) t2: r2(y) 58 ajis both read-and-write => wr & rw & ww / rw & nv* & ww* hd = t,: r,(y)w,(y) t2: r2(y)w2(y) t,: r,(y) w,(y) t2: r2(y) w2(y) the alternative with "-marked conflicts exactly represents the possible lost update situations. the corresponding schedule has been marked with a ^-symbol. • with respect to the synchronization of typical wander-transactions, which of these transaction conflict patterns should we pay attention to and which should we not? an asymmetric assumption is that whenever a transaction acquires an item-variant, the corresponding article-offer can become only less attractive and never more attractive to later transactions. this means that the lower the current quantity and the longer the current reservation-list, the less interesting the article-offer appears. for simplicity reasons we have also anticipated that each wander-transaction only needs one instance of an article. in the following, a reads-and-writes block being changed into a reads-only action further means that another variant of the corresponding item will/must be acquired instead. • the rw & rw* & ww* alternative embedded in the he*-schedule represents a lost update situation. to disallow such a pattern guarantees that a transaction which, after having checked an article through a read, decides to acquire the article through a write, actually gets an instance of this article. in contrast, to allow such a pattern does not guarantee this property. this pattern is termed ap for parallel acquiring. actually, a tentative serialization of the corresponding transactions has two typical results: r2(y) r.(y) w2(y) w,(y) => r,(y) w,(y) r2(y) w2(y) r2(y) r.(y) w2(y) w,(y) => r,(y) w,(y) r2(y) the first outcome occurs when there are article-instances enough for both buyers, and the second buyer does not mind that the first buyer gets the article before him. in contrast, the second outcome occurs if there are not article-instances enough for both buyers or the second buyer does mind that the first buyer gets the article before him. • the wr & rw & ww alternative embedded in the hd-schedule reflects a situation where two transactions in turn check and decide each to acquire an instance of a specific article. to take notice of such a pattern means that the sequence in which instances of a specific article are acquired does matter. in contrast, not to take notice of such a pattern means that the sequence does not matter. this pattern is termed a, for sequential acquiring. actually, a tentative reversal of the corresponding transactions has two typical results: r.(y) w,(y) r2(y) w2(y) => r2(y) w2(y) r,(y) w,(y) ri(y) w,(y) r2(y) w2(y) => r2(y) w2(y) r,(y) the first outcome occurs when the first buyer does not mind that the second buyer gets the article before him. in contrast, the second outcome occurs if the first buyer does mind that the second buyer gets the article before him. • the wr alternative embedded in the he-schedule indicates that a transaction, after having checked an article through a read, decides not to acquire the article. this may be because the article is not the optimal choice or because the conflicting transaction has acquired the last instance of it through a write. the last possibility makes it important to pay attention to such patterns. if the transaction conflict had been reversed, the original transaction could have acquired the last instance of the article instead. this pattern is termed ca for checking after. september 1995 59 actually, a tentative reversal of the corresponding transactions has three typical results: ri(y) w,(y) r2(y) => r2(y) r,(y) w,(y) ri(y) w,(y) r2(y) =» r2(y) w2(y) r,(y) w,(y) ri(y) w,(y) r2(y) => r2(y) w2(y) r,(y) the first outcome occurs when the second buyer's negative decision is not related to the first buyer's positive decision. further, the second outcome occurs when/if only the second buyer does mind that the first buyer gets the article before him, and not also vice versa. in contrast, the third outcome occurs if there are not articleinstances enough for both buyers or each buyer does mind that the other buyer gets the article before him. • the rw alternative embedded in the hb-schedule again indicates that a transaction, after having checked an article through a read, decides not to acquire the article. this may only be because the article is not the optimal choice as there is still at least one instance left. this makes it unimportant to pay attention to such patterns. if the transaction conflict had been reversed, the results would have been the same for both transactions anyway. this pattern is termed cb for checking before. actually, a tentative reversal of the corresponding transactions has a single typical result: r,(y) r2(y) w2(y) =» r2(y) w2(y) r,(y) the single outcome occurs as the first buyer's negative decision cannot be related to the second buyer's positive decision. thus, the cb-patterns should not be included in the synchronization, while the ca-patterns should. further, it is most natural to disallow an ap-pattern i.e. to have it forced into either an asor a ca-pattern. also, it is quite natural to take notice of an as-pattern i.e. to let the acquiring-sequence of article-instances matter.^ note that the caand as-patterns will be included in the synchronization by paying attention to all occurring write-read conflicts i.e. irrespectively of whether each is accompanied by corresponding read-write and write-write conflicts or not. hence, we will require full local serializability (per item or per site) but allow some global non-serializability (per system). we can now illustrate these statements with reference to fig. 1 in the "skeleton-databases" section. basically we consider having only one item per site. in the following schedule reflecting a double check-after situation, t,: r,(x)w,(x) r,(y)... t2: r2(x)r2(y)w2(y) there are two ca-patterns embedded contributing in reverse ways. the x-accesses contribute ti-»t2, and the yaccesses contribute t2-»ti. as indicated above, we can neither reverse the x-accesses nor reverse the yaccesses without possibly changing the decisions. hence, there is no way to obtain an "equivalent" serial schedule here, and we cannot allow such a schedule. in the following schedule reflecting an inconsistent retrieval situation, t,: r,(x)w,(x) r,(y)w,(y) ... t2: r2(x)r2(y) there is one ca-pattern and one cb-pattern embedded contributing in reverse ways. the x-accesses (corresponding to the ca-pattern) contribute t]— >t2, and the y-accesses (corresponding to the cb-pattern) contribute t2— >t]. as indicated above, we cannot reverse the x-accesses but we can reverse the y-accesses without possibly changing the decisions. hence, here there is a way to obtain an "equivalent" serial schedule, t,t2, and we can allow such a schedule. 8. its importance may be based on the specific buyer-identities in who-purchased or only on the number of buyer-entries in who-purchased. 60 ajis in the following schedule reflecting another inconsistent retrieval situation, h/r2 = t,: r,(x) r,(y)... t2: r2(x)w2(x)r2(y)w2(y) there is one cb-pattern and one ca-pattern embedded contributing in reverse ways. the x-accesses (corresponding to the cb-pattem) contribute ti-4t2, and the y-accesses (corresponding to the ca-pattern) contribute t2-»t,. as indicated above, we can reverse the x-accesses but we cannot reverse the y-accesses without possibly changing the decisions. hence, here there is a way to obtain an "equivalent" serial schedule, t2t,, and we can allow such a schedule. in the following schedule reflecting a double check-before situation, hdcb = t,: r,(x) ri(y)w,(y)... t2: r2(x)r2(y)w2(x) there are two cb-patterns embedded contributing in reverse ways. the x-accesses contribute ti-»t2, and the yaccesses contribute t2-»t|. as indicated above, we can reverse either the x-accesses or the y-accesses without possibly changing the decisions. hence, here there are two ways to obtain an "equivalent" serial schedule, t2t, i t,t2, and we can allow such a schedule. note that if x and y had belonged to a unique site instead of to two different sites, the situations reflected in both h/r;/h//j2 and hdcb could not have been allowed in general. this difference stems from the lack of inter-site integrity constraints but existence of intra-site integrity constraints. the ban on intra-site inconsistent retrieval and double check-before situations is the local generalization of the ban on single-item lost update situations. (both inter-site and intra-site double check-after situations are also prohibited). formal correctness criteria based on our initial discussion in the "basic concurrency control" section we get: • in the special cases where we have only one item per site or where we have several items per site but no intra-site integrity constraints, we require local serializability per item plus a global condition of nonserializability type. • in the general case where we have several items per site and some intra-site integrity constraints, we require local serializability per site plus the global condition of non-serializability type. in any case, our 2-level total requirement corresponds to the following class of schedules: c =c nc wr g wr l: wr-rw-ww to explain these designated criteria we have to introduce some extra notions. the binary relation wrd(h) contains the set of ordered pairs of transactions corresponding to the write-read conflicts in the schedule h related to items in the whole global database d. wrd(h)* represents the transitive closure of wrd(h). further, the binary relations wr-rw-wwy(h) and wr-rw-wwy(h) contain the sets of ordered pairs of transactions corresponding to the write-read, read-write and write-write conflicts in the schedule h related to items at the local site y and the single local item y respectively. wr-rw-wwy(h)* and wr-rw-wwy(h)* represent the transitive closures of wr-rw-wwy(h) and wr-rw-wwy(h) respectively. for a schedule h to be a member of class cg:wr the binary relation wrd(h)* has to be a partial order. further, for a schedule h to be a member of class cl:wr.rw-ww all the binary relations wr-rw-wwy(h)* or wr-rwwwy(h)* (corresponding to each particular site or each single item) have to be partial orders. hence, for a schedule h to be a member of our designated class of schedules cwr it has to be a member of both cg:wr and cl:wr.rw-ww, where l for local means either s for per site or i for per item. (g for global means per system). a schedule being a member of the cwr-class has one serialization which complies with the write-read conflicts for the global system plus n serializations each of which complies with the write-read, read-write and writeseptember 1995 61 write conflicts for one of the n local subsystems. all these piecewise serializations" can be separately pursued, and the 1+n resulting serial schedules^ may thus all be inconsistent with respect to their sequencing see the example schedules in the "concurrency control examples" section. the basic lack of/?/? in the local part of our class specification means that only the following reversal type ri(y)...r2(y) => r2(y)-..ri(y) is allowed in the pursuit of equivalent serial schedules for the different local subsystems. the extra lack of rw in the global part of our class specification means that also the following reversal type r,(y)...w2(y)=>w2(y)...r1(y) is allowed in the pursuit of an equivalent serial schedule for the single global system. that the ca-patterns should be included in the synchronization while the cb-patterns should not, is reflected in the global part. further, that an ap-pattern is not allowed, is reflected in the local part. effectively, while we do not allow a transaction's basis with respect to a local decision to be invalidated by one or more other transactions e.g. by not allowing any lost update situations, we do allow a transaction's bases with respect to a global evaluation to be influenced by one or more other transactions e.g. by allowing some inconsistent retrieval situations. the resulting effects of our combined local and global requirements may be termed final-state consistency + final-choice serializability. the "final-state consistency" effect stems from the local requirement's ability to preserve the integrity constraints in the local databases, and the "final-choice serializability" effect stems from the global requirement's ability to observe the overall goals of the global transactions. the basic goal of wandertransactions is to acquire articles. it is also anticipated that if a first transaction manages to acquire a specific article before a second transaction manages to check the same offer, the first transaction should not have to check another offer after the second transaction has already acquired that article. hence, the reads-from relation from bernstein (1987) and papadimitriou (1986) has to be maintained as a partial order, and the final choices of the wander-transactions will thus be serializable. this means that there will exist a corresponding serial schedule in which their final choices will be the same. we will achieve effects that may be considered a variant of so-called final-state serializability in the special cases where we have only one item per site or where we have several items per site but no intra-site integrity constraints. this takes into account the lack of inter-site value-dependencies. in contrast, we will not even achieve this variant of final-state serializability in the general case where we have several items per site and some intra-site integrity constraints. this is due to the existence of intra-site value-dependencies. for a discussion of normal final-state serializability, see papadimitriou (1986). to exemplify these statements, consider the following schedule: t,:r1(xa)w1(xa) r1(yb)w1(yb) t2: r2(xa)r2(yb) r2(yc)w2(yc) ... , 9. piecewise serialization means only paying attention to the transaction conflicts and the region indicated by the name of the corresponding binary relation. 10. note that any topological sort of the acyclic directed graph corresponding to a specific partial order gives a serial schedule which is a piecewise serialization of the original schedule. 62 ajis assume also that the first of the integrity constraints indicated in the "skeleton-databases" section applies. the h/vtts-schedule (which is a variant of the hw;-schedule from the "basic concurrency control" section) will be allowed by our concurrency rules, and the topological ordering of wrd* gives: t,t2 but the value that t2 would write to yc in this serial schedule might differ from what it will write to yc in the h/vra-schedule as it retrieves yb there before yb is updated by tv hence the mentioned variant of final-state serializability is not observed. extra priority rules wander-transactions accessing a skeleton-database correspond to a reads-before-write case. this means that a write on a specific item by a specific transaction will always be preceded by a read on the same item by the same transaction. as in the "formal correctness criteria" section the binary relations wrd(h) and rwd(h) contain the sets of ordered pairs of transactions corresponding to the write-read conflicts and the read-write conflicts respectively in the schedule h related to items in the whole global database d. further, wrd(h)* and rwd(h)* represent the transitive closures of wrd(h) and rwd(h) respectively. observe the high dominance of reads over writes and the high dominance of late writes over early writes inherent in the semantics of wander-transactions. this would for instance make a global requirement which corresponds to the binary relation rwd(any h)* being a partial order very limiting with regard to allowing possible race-situations for favourable offers. likewise, this makes the global requirement which does correspond to the binary relation wrd(any h)* being a partial order also suitable as a priority mechanism in controlling actual race-situations for favourable offers. a global wr-order (i.e. the sequencing implied by wrd(h)*) may thus be used in solving breaks with the local requirement per item/site. • when a local break is about to occur, the wr-order is consulted. if the involved wander-transactions (two or more) are ordered, any latest ordered wander-transaction among these is selected for abortion. in contrast, if they are not ordered, any of these wander-transactions may be chosen for abortion. this implies that quickly making decisions with respect to items (i.e. acquiring a specific article through a write after having checked the corresponding offer through a read) can give high priority to a wander-transaction with respect to avoiding its selection for abortion. hence wander-transactions accessing a skeleton-database experience dynamic priorities. these are not based on when transactions are started but on when actions are executed. further, they are only based on the write operations and only on specific write operations being followed by corresponding read operations. a write operation may thus give a transaction priority over some particular transactions but not over some other transactions. such a use of the binary relation wrd(any h)* as a priority mechanism also requires that it has to be maintained as a partial order. • when a global break is about to occur, the wander-transaction whose write is pan of the write-read conflict about to close a cycle in the graph corresponding to the global wr-relation is selected for abortion. hence, among the involved wander-transactions (two or more) the one issuing the latest corresponding write will normally be chosen for abortion. this means that quickly making decisions with respect to items can even give high priority to a wandertransaction with respect to it getting hold of other attractive offers. maintaining a global wr-order may thus even be used to influence the local wr-rw-ww-order i.e. the sequencing implied by wr-rw-wwy/y(h)*. the final-choice serializability mentioned in the "formal correctness criteria" section will depend on the dynamic priorities of wander-transactions, and the resulting concept may be termed priority serializability. so our designated criteria for concurrency control induce direct reactions on rule breaks. this means that when an access operation is requested which would lead to a global or local break an appropriate abortion is invoked immediately. the reasons are that a later access operation cannot fix the break which is about to occur as we employ conflict based concurrency control, and a later abort operation by a transaction itself is not normally to be expected. this corresponds to a pessimistic concurrency control. for discussions of conflict based versus view based concurrency control and pessimistic versus optimistic concurrency control, see bernstein (1987). just as a later abort operation by a transaction itself is not normally to be expected, a later commit operation by a transaction is not to be allowed totally uncontrolled. this important topic is the main theme of nygard (1994b). actually, there we have specified a set of reliability rules which may be combined in an orthogonal way. note that there is a trade-off between reliability (with regard to guaranteeing recovery properties) and september 1995 63 concurrency (with regard to allowing parallel activities). our reliability rules will be linked to the priority rules with which our designated criteria for concurrency control have been extended. finally, the result will be n-level recovery criteria in addition to our 2-level concurrency control criterion requiring full local serializability but allowing some global non-serializability. initial evaluations let us again look at the example from the "initial illustrations" section. here we get the following results: h^ { (r, ) = { (r2,7j) } hence, wr-rw-wwx(hi)*, wr-rw-wwy(ho* and wr-rw-wwz(h,)* are all partial orders, but wrd(h,)* is not a partial order. according to the criteria in the "formal correctness criteria" section we must conclude: h e c 1 wr f urther, the read of x by t2 contributing to the (ti, t2)-entry in wrd(hi) occurs before the read of y by t] contributing to the (t2, t^-entry in the same binary relation. according to the rules in the "extra priority rules" section the intermediate result is aborting t2, and this shall be enforced when ri(y) occurs. after later rescheduling t?, the final schedule is: t,:ri(x)w,(x)r,(y)r,(z)w,(y) t2: r2(z)r2(y)r2(x)w2(z) an underlying assumption is that when ri(y) is requested and thus invokes an abortion of t2, ri(y) itself is not granted until the rollback of t2 is completed. now we get the following results for this serial schedule: wr-rw-ww (hz) = [ (r. thus, wr-rw-wwx(h2)*, wr-rw-wwy(h2)*, wr-rw-wwz(h2)* and wrd(h2)* are all partial orders. according to the criteria in the "formal correctness criteria" section we may conclude: h e c 2 wr the item-variants that the two wander-transactions will acquire as executed in this schedule are ti: x + y and t2:z. observe that these are neither the same as in the isolated cases nor the same as in the hrschedule. the hr schedule reflects a double check-after situation, see the "basic concurrency control" section. 64 aj1s mechanisms and examples concurrency control mechanisms we have for our combined local and global criteria: h e c iff wr • v(yisasite)\wr-rw-wwr(myh)* is a partial order] • wr (any )* is a partial order d \ h i thus, it would be most natural and direct to employ a concurrency control mechanism based on graph-testing both locally and globally. recall that for any binary relation there is a corresponding directed graph. that the relation is a partial order corresponds to the graph being acyclic. further, for an acyclic graph there are one or more corresponding topological sorts, while for a cyclic graph there is no corresponding topological sort. let us denote the graphs corresponding to the relations wrd) wr-rw-wwy and wr-rw-wwu1' psg, csgy and csg respectively. note that csg and csgy are abbreviations for global and local conflict serialization graph respectively, while psg is an abbreviation for priority serialization graph. these graphs have been integrated in the later illustrations. each arc has labels of type a^ indicating a £c-conflict on item a. the dashed and nondashed arcs together constitute the full csg. the non-dashed arcs alone constitute the psg part. each csgy part, y is a site, appears as the projection of the full csg on all a^-arcs where a e y. let us once more look at the example from the "initial illustrations" section. from fig. 6 we have the following topological sorts: csg: none psg: none csg :tt x 1 2 csg : t t v 2 1 csc : t t : 2 i thus we must conclude: (h^csr) and (tf.e our figure clearly illustrates that there is no way to obtain an "equivalent" serial schedule here. 2rw i vwri \ xwr figure 9. psg(h,) vs. csg(h,) from fig. 7 we have the following topological sorts: csg.tt psg: 7t csg :tt x 1 2 csg :tt v 1 2 csg :tt t i 2 11. the binary relation wr-rw-wwd(h) contains the set of ordered pairs of transactions corresponding to the write-read, read-write and write-write conflicts in the schedule h related to items in the whole global database d. see the "formal correctness criteria" and "extra priority rules" sections. september 1995 65 thus we may conclude: and our figure clearly illustrates that there is no need to obtain another "equivalent" serial schedule here. xwr ywr figure 10. psg(h2) vs. csg(h2) these deductions are of course in accordance with the conclusions in the "initial evaluations" section. we will also indicate how lock-setting could be used as a concurrency control mechanism. first, we only consider serializability per item as the local criterion. before a transaction may check an item, it has to request a read-lock. and if the transaction decides not to acquire a specific item, it may release the read-lock as soon as this fact becomes apparent. this may happen immediately if the current offer is not in accordance with the access-predicate or if it is worse than an already known offer. alternatively this may not happen until later when a better offer is found. this allows breaks with the normal 2phaselock/lockuntilend-rules for all items globally. but if the transaction decides to acquire a specific item, it has to upgrade the read-lock to a write-lock. this leads to observation of the normal 2phaselock/lockuntilend-rules for each item locally. after the transaction has acquired a specific item, it may release the write-lock. but this must happen at a global locked-point or at global end. hence, the items which are both retrieved and updated need zphaselocking/lockinguntilend among themselves and "upgrade-locking" within each. further, the items which are only retrieved have to observe a "chain-locking" technique i.e. the currently best offer has to be kept locked until any better offer is found. this is a double-restricted version of long write-locks and short read-locks, see gray (1976). it is much too restrictive compared to the graph-testing mechanism. this mechanism will naturally disallow the hrschedule in the "initial illustrations" section and allow the h2schedule in the "initial evaluations" section. second, we even consider serializability per site as the local criterion. again, the write-locks have to be held until a global locked-point or until global end. further, the read-locks not upgraded to write-locks have to observe the "chain-locking" technique and have to be held until a local lockedpoint or until local end. this is once more much too restrictive. the global locked-point occurs when all the locks in the whole system are set, while a local locked-point occurs when all the locks for that site are set. the global end occurs when all the accesses in the whole system are terminated, while a local end occurs when all the accesses at that site are terminated. concurrency control examples the hrschedule in the "initial illustrations" section was an indication of what is not allowed by our concurrency control criteria, while the schedules in this section will be indications of what is allowed by our concurrency control criteria. hence, the "initial illustrations" section focused on the limitations inherent in our specifications, while this section will focus on the freedom inherent in our specifications. again we consider having only one item per site. 66 ajis let us imagine a skeleton-database containing the following two item-variants: x: y: quality = 95% price =10$ no-left = 3 who-pur. = quality = 90% price = 5 $ no-left = 3 who-pur. = we look at the following three wander-transactions: tv acquire one: lowest-price & quality > 85 % t2: acquire one: price < 15 $ & highest-quality t3: acquire one: price <, 15 $ & highest-quality + acquire one: lowest-price & quality > 85 % a possible schedule corresponding to wander-transactions ti and t2 is: t,: r,(x) r,(y) w,(y) t2: r2(x) r2(y) w2(x) this schedule reflects a double check-before situation, see the "basic concurrency control" section. from fig. 8 we have the following topological sorts: csg: none csg :tt x 1 2 psg: tt2 i 77 csg : t t y 2 1 (h^csr) but (tf thus we may conclude: our figure illustrates that there are two ways to obtain a globally equivalent serial schedule here i.e. either through y-access reversal or through x-access reversal. schedule h3 may also be allowed by the lock-setting mechanism. vrw xrw figure 11. psg(h3) vs. csg(h3) a possible schedule corresponding to wander-transactions ti and t3 is: t,: r,(x) r,(y)w,(y) t3: r3(x)r3(y)w3(x)w3(y) september 1995 67 this schedule reflects one inconsistent retrieval situation, see the "basic concurrency control" section. from fig. 9 we have the following topological sorts: csg: none csg :tt i 1 3 : r r csg : t t y 3 1 thus we may conclude: (h^csr) but our figure illustrates that there is one way to obtain a globally equivalent serial schedule here i.e. through xaccess reversal. for schedule h» even to be allowed by the lock-setting mechanism the specification of transaction ti has to be changed into for instance: t,: acquire one: price < 7.5$ & highest-quality vrw.ww figure 12. psg(h,) vs. csgch,) a possible schedule corresponding to wander-transactions tb t2 and t3 is: t,:r,(x) ri(y)w,(y) t2: r2(x)r2(y) w2(x) t3: r3(x)r3(y)w3(x) w3(y) this schedule reflects two inconsistent retrieval situations and a double check-before situation, see the "basic concurrency control" section. from fig. 10 we have the following topological sorts: csg: none csg :ttt x 1 3 2 psg:ttt ittt csg :ttt y 2 3 1 thus we may conclude: (hs e csr) but our figure illustrates that there are two ways to obtain a globally equivalent serial schedule here i.e. through yaccess reversal for t2 versus t3 and x-access reversal for ti versus t3, then through either y-access reversal for t2 versus t! or x-access reversal for t! versus t2. but schedule h5 will not be allowed by the lock-setting mechanism irrespectively of any transaction specification changes. 68 ajis / vrw.wwivwr |xrw vrw ; vrw *rw wrjxrw,ww / figure 13. psg(h5) vs. csg(h5) once more, if x and y had belonged to a unique site instead of to two different sites, the situations reflected in hj, hi and h5 could not have been allowed in general. as indicated in the "basic concurrency control" section, this is the local generalization of not allowing the situation reflected in h/ from the same section. note that if no write could ever precede a read in any transaction, the global part of the membership requirements for class cwr would be ensured for any schedule without any synchronization. hence the binary relation wrd(any h)* would be a partial order per definition. this would for instance be the effects with our criterion for a two-step model, see papadimitriou (1986). both the type of access-predicate' ̂ of a wandertransaction and the type of item-distribution ̂ in a skeleton-database may lead to a write-precedes-read situation. observe that reads-before-write refers to a single item-variant, while write-precedes-read refers to different item-variants. conclusion to sum up, knowledge about some missing integrity constraints in a distributed database opens the way for global non-serializability and knowledge about some existing overall goals of distributed transactions leads the way to the add-on to local serializability. wander-transactions accessing a skeleton-database correspond to a reads-before-write case. the specified criteria are c =c nc wk g wr l wr-rw-ww with the last part stemming from the need to preserve integrity constraints in the local databases and the first part stemming from the need to observe overall goals of the global transactions. in nygard (1993a) we have related and compared such total criteria (i.e. the combination of a global criterion per system and the local criterion per item/site) to other known and proposed criteria. this includes the material in ibaraki (1987), garcia-molina (1988), korth (1988), gray (1976), schlageter (1978), garcia-molina (1982) and du (1989). our concepts are entirely new. in nygard (1994b) we have further elaborated on recovery criteria and mechanisms for wander-transactions accessing a skeleton-database. the corresponding n-level model for partial recoverability in distributed databases is combined with the 2-level model for non-serializability introduced in nygard (1993a) and applied in this paper. see also nygard (1993b) and nygard (1994a). references bancilhon, f., kim, w. and korth, h.f. (1985) a model of cad transactions, llth int. conf. on very large data bases. been, c., bernstein, p. a. and goodman, n. (1986) a model for concurrency in nested transaction systems, technical report tr-86-03, wang institute. 12. this means the high-level goal-specification. see for example the trtransaction in the hrschedule from the "initial illustrations" section. 13. this means which types are represented at which sites. september 1995 69 bernstein, p.a., hadzilacos, v. and goodman, n. (1987) concurrency control and recovery in database systems, addison-wesley. bhargouti, n.s. and kaiser, g.e. (1991) concurrency control in advanced database applications, acm computing surveys, 23(3). breitbart, y., garcia-molina, h. and silberschatz, a. (1992) overview of multidatabase transaction management, vldb journal, 1(2). du, w. and elmagarmid, a.k. (1989) quasi serializability: a correctness criterion for global concurrency control in interbase, 15th int. conf. on very large data bases. elmagarmid, a.k. (ed.) (1992) database transaction models for advanced applications, morgan kaufmann. garcia-molina, h. and wiederhold, g. (1982) read-only transactions in a distributed database, acm trans. on database systems, 7(2). garcia-molina, h. and kogan, b. (1988) achieving high availability in distributed databases, ieee trans. on software engineering, 14(7). gray, j.n., lorie, r.a., putzolu, g.r. and traiger, i.l. (1976) "granularity of locks and degrees of consistency in a shared data base" in nijssen, g.m. (ed.) modelling in data base management systems, ifip tc-2 working conference, north-holland. ibaraki, t., kameda, t. and minoura, t. (1987) serializability with constraints, acm trans. on database systems, 12(3). korth, h.f. and speegle, g.d. (1988) formal model of correctness without serializability, acm sigmod int. conf. on management of data. nygard, m. (1993a) relaxed criteria for concurrency control in distributed databases, ifip tc6'93 int. symp. on network information processing systems, sofia bulgaria, 23 54. nygard, m. (1993b) sensible variants of non-serializability in replicated distributed databases, ifip tc6'93 int. symp. on network information processing systems, sofia bulgaria, 55 73. nygard, m. (1994a) tailor-made concurrency control with specialized distributed transactions, acis'94 5th australasian conf. on information systems, melbourne australia, 355 376. nygard, m. (1994b) partial recoverability with distributed transactions, ieee ipccc'94 1994 int. phoenix conf. on computers and communications, phoenix usa, 68 76. papadimitriou, c. (1986) the theory of database concurrency control, computer science. schlageter, g. (1978) process synchronization in database systems, acm trans. on database systems, 3(3). 70 ajis editorial this is the first issue of the australian journal of information systems (ajis). over recent years information systems has steadily grown in importance providing the much needed link between technology and the businesses and organisations which employ it. as can be seen from the articles presented in this issue, the area of information systems is an extremely diverse one impinging on almost all facets of working and social life. throughout the planning and development of this first issue a number of things became clear. the first is that there is a wealth of interest, support and expertise throughout australia in the many areas collectively known as information systems. not only has this support come from within australia but, perhaps more importantly, there is a growing respect by the international community towards the research and scholarship that is emanating from australia. the second conclusion which became apparent was the need for a forum through which this research and scholarship could be acknowledged and shared. i hope that this journal is recognised by the australian information systems community as their forum for the dissemination of knowledge. the journal should not simply act as a place for publication of material, though obviously this is important, but should provide the catalyst for the advancement of information systems both nationally and internationally. this first issue contains eight articles. the breadth of topics covered is, to say the least, challenging. it is the mission of this journal to publish quality research and scholarship in all areas of information systems and as such i hope that the diversity contained in this first issue will be the hallmark of subsequent issues. for information systems to be of importance to the community it cannot simply be the province of a small minority but must reach and support all practising professionals. i believe that the articles in this issue fulfil that aim. when launching a journal there are many who must be acknowledged. firstly i would like to thank the editorial board for its dedication to this venture. without their input this journal would not have been produced. secondly i must acknowledge the support of the consulting editors who, through their positive replies of acceptance to the position, confirmed that we are on the right track. thirdly i must thank and congratulate the eight authors in this first issue. their high quality articles have, i believe, set a precedent for the journal. i must single out for particular thanks mr ayman el-ardenli and mr john james who have been tireless in their assistance through final production of this first issue. finally my thanks goes to the information systems community for your support for this journal. rob macgregor editor ajis vol. 7, no 2 may 2000 issues of page representation and organisation in web browser's revisitation tools andy cockburn1 and saul greenberg2 'department of computer science, university of canterbury, christchurch, new zealand, andv@cosc.canterburv.ac.nz department of computer science, university of calgary, calgary, alberta, canada. saul@cpsc.ucalearv.ca abstract many commercial and research www browsers include a variety of graphical revisitation tools that let users return to previously seen pages. examples include history lists, bookmarks and site maps. in this paper, we examine two fundamental design and usability issues that all graphical tools for revisitation must address. first, how can individual pages be represented to best support page identification? we discuss the problems and prospects of various page representations: the pages themselves, image thumbnails, text labels, and abstract page properties. second, what display organisation schemes can be used to enhance the visualisation of large sets of previously visited pages? we compare temporal organisations, hub-and spoke dynamic trees, spatial layouts and site maps. keywords world-wide web navigation, history and revisitation, graphical navigation maps. introduction web browsers have rapidly become an almost essential component of the desktop-computing environment. they provide access to local and global documents and news, digital libraries, powerful search facilities, and shopping and banking. navigating the web has become such a central paradigm for human-computer interaction that microsoft has made its 'internet explorer' web browser the main interface for file management. despite this incredible popularity, the user interfaces to web browsers are far from ideal. for example, previous studies clearly show that users have a poor understanding of the back and forward buttons (cockburn & jones 1996), one of the most rudimentary navigational interface components available in a browser. yet alternatives are possible; other research has shown that back behaviour can be modified to something that better matches a user's notion of how it works (greenberg & cockburn 1999). in this paper we investigate user-interface considerations that influence the design and use of graphical revisitation tools that people can use to return to temporally distant pages (unlike back which is best used for pages seen very recently). in particular, we consider two questions that must be addressed by the designers of such systems. first, how can the tool represent pages so that people can readily distinguish previously visited pages and identify the one they want to return to? second, what display organisation schemes can be used to enhance the visualisation of a large set of previously visited pages? the relative merits of each design alternative are discussed, and related systems are used to exemplify techniques. we defer application of this discussion to our companion paper, where we describe our own webview system that implements and demonstrates alternative techniques for page representation and organisation (cockburn, greenberg etal. 1999). revisitation on the web with linear navigation in this section we summarise the nature of people's page revisitation patterns when navigating the www. we also briefly critique the back button, arguing that alternate page revisitation tools must complement back if a person is to navigate efficiently to pages seen a while ago. data on web-page revisitation tauscher and greenberg (1997) analysed patterns of web revisitation by logging the user actions of twenty-three users with a specially equipped browser. they found that on average 58% of page visits were to pages that the user had seen previously. they also found a strong recency effect, where the most recently visited pages were very likely to be visited next. to illustrate, figure 1 plots the cumulative percentage url recurrence rate as a running sum against the history distance between pages visited. for example, there is a 39% chance that the next page a person visits will be within the six most recently visited pages. the study also indicates that back is the dominant interface mechanism used for page revisitation, accounting for more than 30% of all navigational acts. other revisitation facilities were used infrequently: for example, <3% for bookmarks, and <1% for history. 120 ajis vol. 7, no 2 may 2000 tauscher and greenberg's study provides powerful evidence that page revisitation is a fundamental requirement in web navigation. it also shows that while most revisitations are back to pages that have been seen recently, a modest amount are to more distant pages. for example, figure 1 shows that there is a 19% chance that the next page the user visits will be more than six pages away from the current page (measured in terms of recency). advantages and limitations of back the web-browser's back and forward buttons can implement a surprising variety of linear navigation schemes, allowing the user to sequentially return to previously seen pages (greenberg and cockburn 1999). in general, there are two major advantages of using back. first, it is cognitively undemanding in that there is little need for decisionmaking—users need only repeatedly click back until either the desired page is displayed, or until the start of the back list is reached, or until they give up. second, it is visually compact—the two buttons consume minimal screen real estate, and for this reason it is reasonable for users to keep them on permanent display within their browsers; consequently they are always ready to hand. however, there are also several limitations to back, as follows. inefficiency in retrieving distant pages. people visit many pages, resulting in large page histories. while back is appropriate for returning to recent pages, it requires many time consuming button-clicks and page redisplays to revisit distant pages. some previously seen pages are unreachable with stack-based back schemes. the stack-based schemes supported by current commercial browsers only allow a sub-set of pages to be revisited because of the branch-pruning that occurs when pages are popped off the top of the stack. while back can be redesigned as linear rather than stack lo figure 1: cumulative url recurrence rate against history distance between pages (tauscher & greenberg 1997). traversal, this introduces other side-effects that can compromise usability (greenberg and cockburn 1999). scalability. commercial browsers are incorporating file-management capabilities into their functionality. consequently, the range of data sources accessed through 'web'-browsers is likely to increase dramatically, as will the temporal range of data elements that the user will want to revisit. this implies that many of the 'pages' a person would like to revisit will be further back on the history list than they are now. context. back gives the user a single viewpoint—the current page—into the information space through which they are navigating. the user's sense of orientation within the information space is therefore entirely dependent on their memory of previously visited pages and the contents of the current page. the reality is that most people just click back until they see their desired page or until they give up. minimalist representation. the linear navigation model underlying back represents the user's navigational actions in one-dimension. while appropriate for simple revisitation tasks, other models showing the true hypertext navigational network in two or three dimensions may be more appropriate to help a person recall a page's location. the potential of enhanced graphical tools graphical visualisations of web spaces, such as overview maps, have long been used to help users overcome the disorientation associated with navigating through hypertext (nielsen 1990). because of their representational richness, they can (and have) been used to ease the chore of revisiting pages. specifically, graphical visualisations can potentially ease or resolve the issues associated with back. they can represent many pages in a small space; this 121 ajis vol. 7, no 2 may 2000 means they are not limited to sequential page displays, and that they are scalable as they can show both recent and distant pages. selecting these distant pages is also easier, because a page can be directly accessed from its representation (provided the user can identify the page from its graphical representation!). finally, they can reveal the structure and relationship between pages, helping users orient themselves in the information space and overcoming the problems of context and minimalist representation. even if this potential could be attained, there remains a fundamental problem confounding the use of graphical tools for page revisitation. users make continual trade-offs between the screen real estate consumed by these tools and the value of the information provided. if users can successfully navigate some or most of the time without the tool, they are likely to remove, iconify, or bury its window. once the tool is no longer visible, it is no longer ready to hand: it is easily forgotten and requires overhead to redisplay it. revisitation tools must therefore maximise the value of the information they present, and do so using minimal screen real estate. the following sections address two questions that focus on this issue. first, how can each page be best represented in a graphical display? second, how can the display be organised to best reveal the relationships between pages? issues in graphical page representation the ideal page representation in a graphical display would allow immediate and unique identification of the page while consuming only minimal screen real estate. four categories of techniques for page identification are presented: the pages themselves, textual labels representing the page, thumbnail images of the rendered page, and visual cues showing abstract properties of the page. many of the techniques discussed can be combined to reinforce page identification. the pages themselves perhaps the easiest way to represent a page is to just show it within the main window. this is the technique employed by the back and forward buttons; as the buttons are pressed, the page is displayed in the browser. the result is that users just click these buttons until they recognise the page they are looking for. this simple technique can be leveraged into more powerful graphical representations by choosing appropriate metaphors. the web book, for example, can collect previously seen pages by organising them into a book; through page-turning, users can browse the book by rapidly flipping through its pages (card, roberston and york 1996). yet showing the full page is space-intensive: only one or two pages can be realistically shown at a time on the display. textual labels representing the page several mechanisms use textual labels to assist page identification. typical approaches to labelling use the page url, the <title> and <h1> text in the page's html source, samples of text from the body of the page, and explicit user annotations. while each type of label is effective in some situations, they all have serious problems that limit their universal application. urls. many revisitation systems display raw urls, either as primary labels or as tooltips. the problem is that urls are primarily machine-oriented. while some site addresses can be readily parsed and recognised by users (for example, netscape.com, microsoft.com, acm.org), not all site names are meaningful. for example, the name xoom.com gives no hint about its purpose and contents. furthermore, users thinking that they are navigating within a site may actually be navigating several different sites (either because the site links to other sites, or because the site is sub-divided). for example, links from www.microsoft.com lead to shop.microsoft.com, messenger.msn.com, msdn.microsoft.com, and www.msn.com. finally, path descriptions to specific pages within a site can be long and cryptic, and therefore of limited use. again using a page from the microsoft site (no better or worse than most other sites), it is difficult to tell what is on the page represented by the url www.microsoft.com/msdownload/iebuild/ie5 win32/en/63242.htm. 122 ajis vol. 7, no 2 may 2000 . back forxw'd reload "home 1 home page for mputer interaction 2 cpsc 861 home page 3 discover alber.. actions, and inapt. 4 reviews-: 5 business 6 internet explorer brother internet explorer sites i ^'features: . ' to welcome to toe xplorer home pagel figure 2: netscape's back menu, showing labels extracted from <title> tags. titles, extracted from the <title> tag in the page's source, can also be poor identifiers of page contents. problems with titles affect a wide range of navigation aids including the back and forward menus supported by microsoft internet explorer and netscape (figure 2), their history lists, and graphical overview maps. titling problems include the following. erroneous and missing titles. many pages provide little or no useful information about the contents of the page, even on pages that are otherwise highly professional. for example, the blank entry '7' in figure 2 resulted from an absent title in the page offering 'more info' on microsoft's internet explorer, located at www.microsoft.com/ie/info/default.asp. we have also noticed that many pages have wrong titles. we believe this is due to the poor facilities most page editors provide, where titles can be changed only by raising a pop-up dialog. this causes authors to forget to add or modify titles; the later causes mis-titling to happen easily when a previous page is copied as a template for a new page. mismatch between titles and actual page representation. the title field in a page does not always match the visually prominent text or graphics appearing at the top of the page. for example, the site pos.epson.com is titled 'epson pos products', yet the dominant graphics at the top of the display says 'welcome to epson america'. identical titles within a site. it is common to find a set of pages with identical or almost identical titles. for example, we have come across sites where all pages have the same title! sites that use html frames are particularly problematical because title information remains constant while the user navigates through pages contained within the frame. inconsistent titling. an absence of consistency in title format can also frustrate page identification. entries ' 1' and '2' in figure 2 demonstrate poor title-format consistency in undergraduate course home pages entitled 'cpsc 681 home page' and 'home page for the course cpsc481: foundations and principle of human computer interaction'. although these titles accurately describe the page contents, the lack of consistency in their presentation complicates and slows page identification. note that this example comes from one of the author's carefully crafted sites... he was very surprised (and somewhat red-faced) to find this inconsistency! long titles and truncation. many graphical mechanisms constrain the number of characters displayed. truncation is used to shorten the title length, but this can remove essential information for page identification. entries '1', '3' and '10' in figure 2 demonstrate truncation, and in entry t the major page identification cue 'cpsc 481' has been removed from the displayed title. note that some browsers truncate the middle (as in figure 2), while others truncate either the beginning or end; this means that authors cannot know where to position important words. other text sources. heuristic techniques could extract text labels from a variety of sources other than the <title> tag, such as from the major heading text for the page, or from the first few sentences of the body of the page. these techniques are also problematical as a primary label. for instance <h1> text is often intentionally omitted in pages that rely on graphical banners for page headings, and there is no easy way to extract text from these graphical banners. sometimes a summary of the first paragraph on a page is shown instead; unfortunately this often consumes too much space and consequently is usually displayed as a pop-up rather than an always-present label. explicit user annotations. explicit user-supplied annotations (such as user-crafted title added to a bookmark) are likely to be easily remembered and recognisable (for that particular user). however, it is unlikely that users will be willing to supply annotations for anything other than the most important pages. in summary, text labels are unreliable. while each technique will succeed for some pages, it will fail for others. 123 ajis vol. 7, no 2 may 2000 images of the rendered page miniaturised 'thumbnail' page images can be automatically captured as each page is rendered in the web-browser. at low scaling factors (large thumbnails), pages are readily identifiable, but image quality degrades as size is reduced. different pages suffer scaling better than others, as described below. a) calgary computer science homepage at pixel widths 12, 25, 50, 100. b) a text page at pixel widths 12, 25, 50, 100. c) four pages within the calgary computer science site at 50 pixels wide. figure 3. comparing various thumbnails. when viewed in colour, the distinctly coloured blocks of the calgary computer science homepage in figure 3a make it identifiable at small sizes, but even at large sizes the text-based page (figure 3b) provides few cues to identification. the problems are further exacerbated by consistent page-styles used within web-sites, including similarly coloured backgrounds, frames, uniform corporate banners and logos, and so on. for example, the four different images in figure 3c show the poor distinction between several calgary computer science pages with thumbnails that are 50 pixels wide. breaking consistency is, of course, a poor solution for obvious reasons. in reality, constraints on screen real-estate make icons of even this modest size unlikely to proliferate: for example, the icons used in file browsers such as microsoft's explorer are approximately 15 by 10 pixels. we can partially fix this problem of low page distinction between pages at small sizes by using zooming techniques to increase their size and distinctiveness. mosaicg (ayers & stasko 1995), and web view (cockburn, greenberg et al. 1999) support thumbnails that are magnified as the user moves the mouse over them. the data mountain adjusts the size of thumbnails as they are positioned on a 3-d inclined plane, with closer ones being larger than those further away (robertson et al. 1998). the zoomable web browser descri bed in bederson et al. (1999) takes this one step further by integrating browser functionality within each of the miniaturised page representations; the zoomed thumbnail becomes the web page. visual cues showing abstract page properties visual cues displaying abstract page properties can be used to reinforce page identification techniques. in particular, temporal information about the frequency and timing of previous visits may help users to identify pages. netscape and microsoft internet explorer, for example, both use temporal information in their history views of past visits. however, these are typically coded by text and are difficult to scan 'at a glance'. figure 4: cues showing bookmarks, temporal properties of page visitation, and visit frequency for example, figure 4 combines several graphical cues with a thumbnail to capture various abstract properties. the top left 'dog-ear' denotes how often the page has been visited, where it automatically turns progressively darker with visit frequency. the bottom-left red dog-ear is created explicitly when a person marks a page, thus becoming a type of 'bookmark'. the blue flashes on the right-hand side of the thumbnail indicate the relative time of the first and last access to the page, where the bottom of the thumbnail equates to the eldest page in the display and the top of the thumbnail equates to the current time. web view, described in cockburn and greenberg (1999) as well as an unnamed system in greenberg and cockburn (1999) implements most these concepts. issues in structural organisation of page display this section discusses issues in organising the display of many pages. four schemes are described: hub-and-spoke techniques that depict the user's navigational branching actions; spatial schemes that exploit the user's memory for 124 ajis vol. 7, no 2 may 2000 the spatial location of objects; site-maps that model the physical storage location of pages; and temporal schemes. the relative merits of each organisational scheme are discussed, and systems demonstrating each technique are noted; table 1 summarises their capabilities and relative merits. hub-and-spoke dynamic trees hub-and-spoke dynamic trees are generated in response to the user's navigational acts. each newly visited page is added to the display, with a link showing its connection with the previous page. various graph-drawing algorithms can be used to generate the display (with consideration of cyclic page links), resulting in a tree, nested or graph display (cockbum & jones 1996). hub-and-spoke dynamic trees offer several advantages. first, they recognise that a fundamental way that people revisit pages is through hub-and-spoke navigation, where people follow many 'spoke' links from a single 'hub' page, returning to the hub each time (tauscher and greenberg 1997). second, they cluster pages logically related by a user's navigation patterns, independent of a page's actual storage location. problems occur, however, when trees are deep; because each branching navigational act is treated as a new spoke of the current page, a new tree level must be created. these create long chains that are difficult to draw and visualise as a tree in a small space. to alleviate this problem, information visualisation techniques such as the hyperbolic lens (lamping el al. 1995) can be used. systems demonstrating hub-and-spoke dynamic trees include mosaicg (ayers & stasko 1995), webnet (cockburn & jones 1996), padprints (hightower el al. 1998), and webview (cockburn, greenberg el al. 1999). spatial layouts spatial organisations aim to exploit people's memory for the spatial location of objects. for this to work effectively, users explicitly position page representations (such as thumbnails) in a 2d or 3d space, and the location then serves as an additional cue to page identification. for example, robertson el al. (1998) showed that with the data mountain—a 3d spatial organisation of page thumbnails on an inclined plane—users were able to accurately find specific pages within a set of 100 pages, even after long time intervals. this result was a statistically significant improvement in page access time and error rates when compared with internet explorer's favourites utility. other systems demonstrating spatial layouts include the webforager (card el al. 1996). the primary disadvantage of spatial techniques is that users must explicitly position pages within the information space, placing both a cognitive and manipulative burden on the user. first the user must decide how to categorise the page, and consequently decide where it should be located. they must then move it to the desired location. the combination of these activities may be unacceptable to users who wish to simply glance at a page and go onto the next. although this burden could be eased by having pages assume a default location, there is a risk that the default region could become excessively cluttered, reducing the user's ability to locate pages. site maps site maps show a topology of the physical storage locations of pages. they are normally generated statically—as a batch process prior to the user's navigation to the site—and once generated, they can be made available to all site visitors. page representations within site-maps are normally active, allowing users to navigate directly to pages by clicking. the primary advantage of site-maps is their coverage. all pages at a site can be included in the site-map, aiding the user's orientation and navigation across the entire site. statically generated site-maps can support initial access to pages as well as revisitation to pages previously seen. although the wide coverage of site-maps offers advantages, it also means that the representation is likely to show many more pages than the user needs, thus unnecessarily cluttering the display. there are other disadvantages of basing the site-map topology on the physical location of pages: in particular, the physical location of pages need not reflect semantic relationships, and pages on the same topic at different sites will be excluded from the display. systems supporting site-maps include the navigational view builder (mukherjea & foley 1995), webviz (pitkow & bharat 1996). webview uses partial site-maps to ease the problem of excessive nesting that occurs with hub-andspoke organisation, where new sites become top-level hubs (cockburn, greenberg etal. 1999). temporal organisation temporal organisation schemes exploit the user's memory for the timing of their actions. for recently accessed pages, a variety of linear back and forward revisitation schemes are efficient and easy to use (greenberg & cockburn 1996). for temporally distant page access, time segments can be used. for example, microsoft internet explorer's history mechanism allows users to view the last few days, weeks, and months. many pages are likely to be present within coarse-grained temporal segments such as 'last month', and it is therefore necessary to use secondary categorisation techniques to assist page identification. 125 ajis issue when generated? coverage supports page hub-and-spoke dynamic. models user's actions. all visited pages. • vol. 7, no 2 spatial dynamic. explicitly positioned by user. all visited pages. • may 2000 site-map temporal static (or dynamic to specific link-depth). all site pages. • x (site only). dynamic. all visited pages. • revisits? display tailored to user's interest? x (models site, not actions). semantic groupings of related pages in display? • (assumes child pages are related to parent pages). • (explicit by user). •• (physical location need not model semantic relationships). x (only if user visits semantically related pages in sequence). primary advantage reveals browsing path within and across sites. location provides a cue to page identification. supports orientation within site. recency is a clue to page identification. primary disadvantage excessive display nesting with long browsing paths. explicit action necessary to position page representation. reveals a superset of 'interesting' pages. usability of temporal chunks uncertain, distant pages harder to find. table 1: summarising the capabilities and merits of display organisation techniques. summary interactive graphical tools have the potential to ease many user problems in navigating through the world wide web: they are scalable and can depict extremely large data-spaces, they can support revisitation to all previously seen pages, and they can aid the user's sense of orientation in the web-space. to realise this potential the tools must represent individual pages in a manner that aids rapid identification, and they must organise the display of large sets of pages in a manner that is meaningful to the user. this paper has raised issues in providing identifiable representations of individual pages and in organising the display of many pages. by explicitly recording these issues, we aim to promote the consideration of design alternatives. acknowledgements partial funding for this work was provided by nserc canada and microsoft research. references ayers, e. and stasko, j. 1995, using graphic history in browsing the world wide web, in 'proceedings of the fourth international www conference. 11-14 december, boston'. bederson, b., hollan, j., steward, j., rogers, d., vick, d., ring, l., grose, e. and forsythe, c. 1999, a zooming web browser, in 'human factors in web development'. card, sk., robertson, gc. and york, w. 1996, the webbook and the webforager: an information workspace for the world-wide web, in 'proceedings of chi'96', ppl 11—117. cockburn, a. and jones, s. 1996, 'which way now? analysing and easing inadequacies in www navigation', international journal of human-computer studies 45(1), 105-129. cockburn, a., greenberg, s., mckenzie, b., smith, m. and kaasten, s. (1999) 'webview: a graphical aid for revisiting web pages.' submitted to ozchi'99. greenberg, s. and cockburn, a. 1999, 'getting back to back: alternate behaviors for a web browser's back button', 5th conference on human factors and the web. hightower, r., ring, l., helfman, j., bederson, b. and hollan, j. 1998, graphical multiscale web histories: a study of padprints, in 'proceedings of hypertext'98', acm press, pp. 58-65. lamping, j., rao, r., and pirolli, p. 1995, a focus+context technique based on hyperbolic geometry for visualising large hierarchies, in 'proceedings of chi'95', acm press, pp401-408. mukherjea, s. and foley, j. 1995, visualizing the world wide web with the navigational view builder, in 'proceedings 3rd international www conference', vol. 27, pp. 1075-1087. nielsen, j. 1990, the art of navigating through hypertext', communications of the acm 33(3), 296-310. pitkow, j. and bharat, k. 1996, webviz: a tool for world-wide web access log analysis, in 'proceedings of the 1st international www conference'. 126 ajis vol. 7, no 2 may 2000 robertson, g., czerwinski, m., larson, k., robbins, d., thiel, d. and van dantzich, m. 1998, data mountain: using spatial memory for document management, in 'proceedings of uist'98', pp. 153-162. tauscher, t and greenberg, s. 1997, 'how people revisit web pages: empirical findings and implications for the design of history systems', international journal of human computer studies 47(1), 97-138. 127 ajis special edition knowledge management december 2001 from medieval philosophy to the virtual library: a descriptive framework for scientific knowledge and documentation as basis for document retrieval. frances morrissey school of information management & systems monash university frances.morrissey@sims.monash.edu.au abstract this paper examines the conceptual basis of document retrieval systems for the virtual library in science and technology. it does so through analysing some cognitive models for scientific knowledge, drawing on philosophy, sociology and linguistics. it is important to consider improvements in search/ retrieval functionalities for scientific documents because knowledge creation and transfer are integral to the functioning of scientific communities, and on a larger scale, science and technology are central to the knowledge economy. this paper proposes four new and innovative understandings. firstly, it is proposed that formal scientific communication constitutes the documentation and dissemination of concepts, and that conceptualism is a useful philosophical basis for study. second, it is proposed that the scientific document is a dyadic construct, being both the physical manifestation as an encoded medium, and also being the associated knowledge, or intangible ideation, that is carried within the document. third, it is shown that major philosophers of science divide science into three main activities, dealing with data, derived or inferred laws, and the axioms or the paradigm. fourth, it is demonstrated that the data, information and conceptual frameworks carried by a scientific document, as different levels of signification or semiotic systems, can each be characterised in ways assisting in search and retrieval functionalities for the virtual library. keywords knowledge retrieval, virtual library, semiotics, scientific paradigms, undiscovered public knowledge. introduction this paper examines the conceptual basis of document retrieval systems for the virtual library in science and technology. the knowledge community that it deals with is therefore the wider scientific and technical community, rather than dealing at organisational or enterprise level as do many important knowledge management texts (e.g. harvard 1998). creation of objective knowledge, and its transfer and use, are integral to the functioning of scientific communities, and on a larger scale, are vital to the knowledge economy as being the core activities of science and technology. libraries are memory institutions for knowledge resources, and traditionally they have been repositories of published knowledge, organised for preservation, access, and use. the development of virtual or hybrid libraries is relatively recent, and is associated with the availability of digital resources and web technologies. the virtual library may be defined as portal or interactive gateway to one or more memory institutions of situated or distributed analogue and digital knowledge resources. memory institutions, such as libraries, archives, and museums organise the intellectual and cultural record, including the scientific heritage, and libraries are focussed on in this instance, as dealing with published documents. currently, virtual libraries tend to focus on traditional curatorial values for digital resources, and must develop relevant practices to support their use and management over time, including search and retrieval protocols (dempsey 2000). however hjorland has posited that examining problems of a cognitive nature, including the meaning of documentation, is essential for the further development of information science and of document retrieval systems (hjorland 2000), and this type of analysis and associated outcomes are in fact the main objectives of this paper. the principal assumption underlying the study undertaken in this paper is that libraries have been particularly important in the infrastructure of science and technology for two main reasons. firstly, publication has long been recognised as the principal channel of formal communication in these disciplines. second, the threefold demands in conventional scientific method of empiricism (data collection and recording), performativity (reproducibility and prediction of results), and paradigm testing, have necessitated documentation and continuing reference to collective memory in each discourse community. indeed creating "literary inscriptions" has been said to be a central activity in experimental science, carried out at all stages from observation and instrumentation to analysis and ultimate publication (latour 1986). however the ways in which scientists and technologists access documented knowledge, i.e., the memory traces in a discipline, bear close analysis. hjorland (1997) has stated that the goal of information seeking in science is to identify potential knowledge, data, information, or raw material that will contribute to the theoretical or empirical development of the field or to the solution of a particular problem. further development of the methodological aspects of information seeking, he believes, can be broken down into two aspects: 80 ajis special edition knowledge management december 2001 (1) problems concerning the conceptual structuring of the objects of information seeking, and (2) problems concerning the structure and the properties of the search tools that are available to the user, neither of which has been examined thoroughly (ibid.: 146-147). this paper attempts to redress this deficiency by addressing both the conceptual basis of scientific documents and the nature of the tools that can be used to retrieve them. the world of scientific knowledge tbe proposition that there are three types of scientific writing formal communication in science generally concerns information transfer through documentation and publication. where scientists have communicated their findings by means of documents, document seeking naturally becomes the focus of attention in information seeking, and it is necessary to consider the concept of a document and the nature of a document retrieval system as metadocument (hjorland 1997: 13-19). it has been said that the information transferred in documents usually consists of either factual data or conceptual information, and that many studies have not made even this distinction (meadows 1974: 92). hjorland too has observed that information seeking tends to be routinised during normal science, concentrating on information of a factual nature, while being more to do with formulating a coherent knowledge base during theoretical transformations on the research front, thus again distinguishing only two types of information: facts and concepts (hjorland 1997: 137-138). this paper proposes that there are in fact three types of information content, in the sense of the content of documents as embodied communicative transactions, thus differing from previous studies. this paper will categorise the three types of writing that scientists do, and thus while recognising the centrality of writing, as latour does, it differs from latour's view of science as an activity in which an undifferentiated spectrum of literary inscriptions are the persuasive tools in scientific argument (latour 1986: 88). latour's range of examples, from records of laboratory data to the published graphs and figures of formal communication in a discipline, are assumed to fit into the three categories. definitions of a document in leading up to the proposition that there are three levels of meaning, first, let us consider: what is a document? in this paper, it is assumed that a scientific document, whether item (physical) or replicable event (virtual), involves an instantiation of knowledge content to observers. this is shown by considering the literal sense of "document" where "document" is derived from the latin documentum (lesson, proof, instance) and still has a sense of being a teaching, or a piece of instruction or a lesson (new shorter oxford english dictionary 1993 sense 1). in other words the information or knowledge content of the document has an independent existence and can be quoted, cited or challenged. the term "conceptus" is coined in this paper to denote that knowledge content. considering possible attributes of the conceptus leads to a set of principles that could be helpful in characterising knowledge communication and access to collective memory in science. documents in relation to popper's worlds 1,2, and 3 the philosopher karl popper divided the universe into three "worlds". these are: world 1, the physical world; world 2, the world of conscious experiences and subjective knowledge, which mediates between world 1 and world 3; and world 3, the world of objective knowledge, including documented scientific knowledge (popper 1996). world 3 has its own domain of autonomy. it is in part the by-product of language and of publication, particularly description and argument. "theories, or propositions, or statements are the most important third-world linguistic entities", and they have "objective logical content." (popper 1996: 157) thus a document belongs both to world 1 in its physical constitution, and to world 3 regarding its knowledge component. applying this interpretation to library and information science is not new: swanson (1986) used world 3 as justification for his techniques for retrieving undiscovered public knowledge. science and the three levels of signification in scientific documents what is science? this question is important if we are to understand the cognitive or conceptual basis of the scientific document. to answer this question, this paper will principally draw on the writings of popper (1979) and kuhn (1996). 81 aj1s special edition knowledge management december 2001 popper defines work in science as work directed towards the growth of objective knowledge (popper 1979: 121) and as attempts to describe and (so far as possible) explain reality (ibid.: 40). he postulates that "the method of science is the method of bold conjectures and ingenious and severe attempts to refute them."(ibid.: 81). popper describes the tasks of science as being (ibid.: 352): (1) the derivation of predictions, (2) technical application, and (3) testing of the explicans, based on universal laws and specific initial conditions. this is entirely compatible with kuhn's listing of three foci for factual scientific investigation (kuhn 1996): (1) factual determinations or increasing the accuracy and scope of factual measurements, (2) investigations of norms and predictions: comparison of facts with predictions from the paradigm theory, and (3) empirical work undertaken to articulate the paradigm theory, such as (a) determination of universal constants, (b) formulation of quantitative laws, and (c) elucidation of models where the paradigm is the overarching conceptual framework accepted by the discourse community. even einstein, known as a physicist rather than as a philosopher of science, recognised three main themes in his schematic of science (einstein 1952): (1) the system of axioms (which i identify with the paradigm), (2) deduced laws, and (3) the totality of sense experiences (with which i include the outputs of instrumentation) i would like to propose that these three foci of science correspond with three important possible facets of scientific documents, whether experimental or theoretical: the reported or implied data, the surmised information or laws and trends, and the overarching conceptual framework or paradigmatic level of scientific documentation. popper comments "all the important things we can say about an act of knowledge consist of pointing out the thirdworld objects of the act a theory or proposition and its relation to other third-world objects, such as the arguments bearing on the problem as well as the objects known" (popper 1979: 163). this makes it easy for us to equate kuhn's paradigms with popper's world 3 objects. the problem of induction and the phony story, contrasted with deduction popper makes a useful distinction between the "bucket theory of science" and the "searchlight theory of science". in the former, "...our knowledge, our experience, consists either of accumulated perceptions (naive empiricism) or else of assimilated, sorted and classified perceptions..." that is, the mind resembles a container in which perceptions and knowledge accumulate, from which objective experience or science may be distilled (popper 1979: 341). in this model, the purpose of scientific documentation is to construct a phoney narrative in which scientific hypotheses pose as inductions or as analogical reasoning. this style of writing is deplored by latour (1986: 174), because it leads to a separation of actual recorded laboratory practice and factual statements in the propounding of novel insights, and to the stripping of contingent circumstances (ibid.: 174), i.e., "scientific facts are formulated in the denial and obliteration of their own historicity" (ibid.: 277). in the alternative, the searchlight theory of science, particular hypotheses inform what kind of observations to make, and which observations deserve attention; and observations also function as tests of hypotheses. it is worth noting that berkenkotter and huckin have also noted the construction of the "phoney story" in scientific reporting, and they observe that as part of this construction, scientific genres are expected to demonstrate both novelty and intertextuality with regard to previous expositions of the paradigm (berkenkotter & huckin 1995). thus scientific documents may be structured in such a way as to either have the appearance of inductive reasoning, with concomitant citation of others' results and conclusions as well as one's own, or to be the result of deduction from the axioms, models and metaphors making up the paradigm. in either case, there are at least two levels of signification present, either explicitly or by implication. 82 ajis special edition knowledge management december 2001 hjorland on seeking knowledge objects the contemporary information scientist hjorland (1997: 147) lists four categories of concepts and tools that can be used in categorising and seeking knowledge objects: (1) the research object itself, (2) scientific theories and methods concerning the research object, (3) scientific disciplines, traditions, and paradigms that deal with the research object, (4) formal aspects of the communication process where knowledge about the research object is documented these correspond with the threesome of empiricism, rules and predictions, and the paradigm, with the documented knowledge per se in its documentary genre making up hjorland's fourth category. the threefold path in this section i have drawn on the arguments of major philosophers of science to conclude that science constitutes the pursuit of objective knowledge, with such knowledge being reported and transferred in the form of scientific documents. these philosophers seem unanimous in their opinion that science and scientific inscriptions are made up of three main threads, viz.: data, being original observations and outputs of instrumentation; trends, rules and predictions drawn from either phenomenological generalisations or theoretical derivations; and the paradigm. i extrapolate this conclusion to declare that any scientific document whether experimental or theoretical will carry or imply meaning associated with two or more of these threads. connections with conceptualism, from ockham to price conceptualism william of ockham (ca. 1285-ca. 1347) and henry habberley price (a mid-20th century philosopher) have much to say on the nature of concepts that is useful to understanding information and knowledge retrieval. taken in conjunction with the afore discussed nature of scientific knowledge, their views lead to possible new functionalities for document retrieval interfaces. what are concepts? first take the modern view (audi 1999: 169): concepts are principles of classification empowering recognition, analysis and categorisation, and they may be divided into 4 classes. these are mental representations (ideas serving classificatory purposes); brain states that serve the same function; general words and their usage; and the ability to classify correctly using one or more of these. henry price was the main proponent of this view (ibid.) second, take ockham's view: sciences, together with their premisses and objects, are collections of true propositions (kretzman et al 1982: 505), where ockham believed that mental propositions exist, and are composed of understandings or concepts (ockham 1991: 206-209). ockham stipulated that a concept is a natural sign of a thing, either in actuality or in a depossibili proposition(ibid.: 454-456). these views are important and insightful for the virtual library, because from the 17th century onward, hence entirely within the era of print-based technologies, nominalism and phenomenalism had overtaken conceptualism. that is, it was believed that experience and sense-data were the foundations of observation and generalisable truths, and that there were no universals or transcendentals that had independent existence. this runs directly counter to popper's world 3, described in the previous section. it is, however, the basis for much 20th century philosophy, such as that of willard van orman quine. quine built a philosophy of science based on linguistics and formal symbolic logic, which is understandable in terms of the sense of closure associated with print and its definitive embedding of the oral, spoken word in visual space (ong 1982). quine's view is that "words and observable behaviours are all we have to go on" (quine 1995: 6) and "sentences are the primary vehicles of meaning" (ibid.: 7). science in his view is built on sensory data and scientific posits, the latter being in the form of occasion sentences, standing sentences, and observation categoricals, and subject to truth and verdict functions (quine 1974: 3-80, 1995: 43). this view, although laudable, does not seem particularly helpful in interface design for the virtual library. the virtual library and web-based technologies have multimedia capabilities which extend beyond the semantics of written and spoken language. these capabilities can be compared with views that originated in a preliterate society, which are that concepts have an independent existence, and that these are important in the transmission of culture, including scientific culture. exploring these views in this section is built on assumptions that information seeking is primarily to do with identifying, accessing and retrieving documents that embody or exemplify particular concepts, be they to do with subject, authorship or other criteria. 83 ajis special edition knowledge management december 2001 "intentions" as signs signifying concepts ockham divided concepts into "first intentions" and "second intentions". a first intention is a natural sign of something which is not itself a sign. a second intention is a natural sign that signifies other natural signs or first intentions (audi 1999: 492). concepts and first and second intentions lead to an ability to categorise, as follows. a category has three sorts of existence: in the mind, in writing, and in speech. a category, taken significatively, is composed of concepts(ockham 1991: 475 478). if a category is taken as an ordering of things then a category may include both first intentions and second intentions. if category is taken as what is first and most common in a categorial ordering then each category is a first intention or name of first intention (ibid.: 467 471). also, in ockham's view, concepts may be distinguished as being absolute, connotative or relative(ibid.: 485-487). this has obvious relevance to modern classification schemes, which are commonly descriptive, denotative and/ or hierarchical. for example, take a recent author, gail hodge, on the details of knowledge organisation systems for digital libraries. hodge (2000) lists 3 characteristics common to knowledge organisation systems: (1) the knowledge organisation system imposes a particular view of the world on a collection and the items in it (i.e. it operates at paradigm level). (2) the same entity can be characterised in different ways depending on the knowledge organisation system used. (3) there must be sufficient degree of resemblance between the concept expressed in a knowledge organisation system and the real-world object or representation it applies to, for others to apprehend and apply the system with reasonable reliability. she lists 3 main types of knowledge organisation system: (1) term lists, (2) classifications and categories, and (3) relationship lists. as extensions of these, she gives examples of ways of linking digital library resources to related resources, and of indexing with and mapping of multiple schemes. hodges' 3 main types of kos seem to correspond directly with ockham's division of concepts into the connotative, the absolute and the relative. applications of insights from conceptualism to knowledge representation the twentieth century philosopher price said, "the occurrent manifestation of concepts is as multiform as the exercise of intelligence is. in fact it is just another name for the exercise of intelligence." (price 1953: 354). he went on to explain that intelligence is manifested in thinking, or conceptual cognition, which enables the recognition of objects, classification, .and the formulation of complex ideas through intellectual operations performed upon the basic concepts. concepts are simply recognitional capacities founded in the similitude of things, or awareness of universals (ibid.: 357). price believed that "fundamentally a concept is a recognitional capacity, whatever else it may be besides." (price 1953: 355). he stated that concepts are also manifested through: (1) sign cognition, or understanding signs through mental and psychophysical response. this includes both primary recognition and secondary recognition (where the latter can be mistaken), and induction involving the operators not, or and if, as well as statistical probability, which allows for probabilities or degrees of instantiation. (in my view this has to do with empiricism in science, and also the nature of declarative knowledge. it correlates with the first level of scientific activity after popper and kuhn, described in the previous section.) (2) production of quasi-instantiative particulars such as production and recognition of images (i.e. "image thinking", both quasi-instantiative and generic), production of replicas, and production of instantiative particulars (instantiation in action in real life). (in my view this has to do with both reproducibility and hypothesis testing in science. it correlates with the second level of scientific activity after popper and kuhn, described in the previous section.) (3) production of non-instantiative symbols (words and codes, and symbolic operations), and the ability to produce alternative verbal formulations (again, degrees of actualisation). (in my view this has to do with modelling, mathematical formalism, and formal communication in science. it correlates with the third level of scientific activity after popper and kuhn, described in the previous section.) 84 aj1s special edition knowledge management december 2001 price stressed the need for cognition of signs and symbols (price 1953: 352-353): the noise or mark, so long as i merely observe other people producing it, is not yet a symbol for me, but at most only a sign. it only becomes a symbol for me when i have learnt to use it for myself. after all, every man must understand for himself what others say or write. the others cannot understand for him. it is his concepts, not theirs, which are operative when he 'follows' what his neighbours say. his concepts, and not only theirs, must be associatively linked with the sounds or marks which he hears or sees. and that is only another way of saying that he must already be able to use these words understandingly in his own thinking. each of us must understand for himself, and no one else can understand for him. in other words, knowledge retrieval must deal not only with the text at data level (the marks or words) but the concepts as well. both must be capable of being recognised, and this has much to do with the science of semiotics, which will be discussed in the next section. this is where it will be seen in detail how much medieval scholasticism and modern conceptualism have to offer modem information science. also, conceptualism and the three levels of analysis of scientific knowledge that have been discussed (data and intentions, information such as trends, rules and predictions, and conceptus or paradigm), will be shown to feed into a new model of scientific documentation with potential retrieval points for documents, in section 5. higher order semiotics roland barthes is well known for his writing on semiotics and myth (barthes 1972). his ideas will be discussed in this section as an attempt to lay more epistemological and ontological foundations for considering document retrieval systems, answering the question "what is knowledge?" with the response "a level of meaning in a higher order semiotic system". i shall draw parallels between the theoretical and referential content of scientific communication and barthes' "myth", later to be compared with kuhn's "paradigm". barthes regards mythology as a branch of semiotics, where semiotics is concerned with signification, and analyses communication in terms of the functional relationships between the triad of signifier, signified and sign. to quote barthes (1972: 114-115; emphasis in original)., ...myth is a peculiar system, in that it is constructed from a semiological chain which existed before it: it is a second-order semiological system. that which is a sign (namely a concept and an image) in the first system, becomes a mere signifier in the second... it can be seen that in myth there are two semiological systems: a linguistic system, the language (or the modes of representation which are assimilated to it), which i shall call the language-object, because it is the language which myth gets hold of in order to build its own system; and myth itself, which i shall call metalanguage, because it is a second language in which one speaks about the first. language myth 1. signifier 2. signified 3. sign i signifier ii signified iii sign (barthes 1972: 115) figure 1: myth as second order semiotic system 85 ajis special edition knowledge management december 2001 i see the signification of scientific documents as usually involving wider or deeper meaning and inferences, signified beyond the conventional language or formulaic representations of the informationor data-content of the document as a physical item. that is, there is more than the basic signification of the words and symbols of the text. trends and paradigms may be inferred from mere data by a knowledgeable reader; and data may be predicted from axioms and laws given particular contexts and boundary conditions. there is therefore more than one level of signification, and at each level, the triadic relation of signifier-signified-sign applies. the semiotic approach to the levels of signification is described in detail in the following section. document retrieval functionalities based on characterisations of documented data, information and conceptus the following section expounds a novel analytical approach to scientific documents and documentation, based on the philosophical discussions of the previous sections. the analysis distinguishes between data, information and knowledge as different levels of meaning, and likens them to levels of semiotic systems. based on this analysis, characteristics of the documented data, information and knowledge are enumerated. it is proposed that these could be used as descriptors, identifiers or allocated or intrinsic metadata for document access and retrieval. signification at data level and associated retrieval functionalities i postulate that data consists of primary signs and significations (words, numbers and other meanings), and use of language. it includes "first intentions" and "second intentions", and representation of acts of understanding, such as the empirical content of primary scientific literature, and of informal communications. it corresponds with the first of kuhn's three foci for scientific activity. data can be identified and retrieved through specification and characterisation of the item or its data content in allocated or intrinsic metadata (where the item is the physical document, i.e. the medium and encoding existing to transfer information in the process model of communication). in semiotic terms, the triad of signifier, signified and sign at the data level may be represented in the diagram below. primary signifier code primary signified intentions, empirical results primary sign data figure 2: the semiotic triad for data signification at information level and associated retrieval functionalities i postulate that information is the meaning drawn from or associated with data or selected data, such as trends, rules and predictions. it corresponds with the second of kuhn's three foci for scientific activity. it is carried in physical 86 ajis special edition knowledge management december 2001 and virtual texts and formal and informal scientific communications, and often said to be the content of the process model of communication. the field, tenor and mode of communicated information in a discourse community are conventionalised as aspects of genre (schirato and yell 1996). information is amenable to classification through use of subject analysis. thus retrieval of information is possible through specification and characterisation of the item or its information in allocated or intrinsic metadata. these forms of metadata include bibliographic citations, metadata associated with recognised metadata schemes,and subject representation in classification schemes, indexing, or surrogate records (whether by literary warrant or user warrant). in most libraries, the indexer/ cataloguer assigned bibliographic authority files (subject, author, added entries) are important. information can also be identified and retrieved through its lexical content as item full text. in semiotic terms, the triad of signification at information level may be represented in the diagram below. it can be seen to overlay the data triad, in that data has moved from being the sign to the signifier. secondary signifier data secondary signified norms, rules, formulas, procedures i.e. declarative knowledge secondary sign information figure 3 the semiotic triad for information signification at conceptus level and associated retrieval functionalities i postulate that the "conceptus" or atomisation of overarching conceptual framework consists of knowledge, theory, and their essential contexts. these include the referential content of scientific genres, in the third of kuhn's three foci for scientific activity. the conceptus may demonstrate novelty, intertextuality and promulgation of paradigm in "normal science", or convey competing articulations and explicit discontent with the current paradigm during "extraordinary science" (see kuhn 1996, and berkenkotter & huckin 1995). the conceptus may be disseminated as a "work" or "teaching" (see the earlier discussion on derivation of "document"). the conceptus may be identified and retrieved through identifiers, through subject analysis of the paradigm (by analogy with the case of information), and through citation indexing. the aspects of imprimatur, or intellectual authority and history of the conceptus, consist of other details such as citation trails, corporate authorship, sponsorship, and publishing filters such as refereeing. these details, which acknowledge where credit is due for current or past work, may also be denoted by any details of intellectual property transfer, reformatting or reinvention which may arise, but which are not covered above. the manifest identity and uniqueness of the conceptus are associated with its ideation, as we found in discussing conceptualism, and therefore the mental states associated with a concept are as important as its lexical content. in present times, with multimedia available on the worldwide web, the ideation of the conceptus can be represented in 87 ajis special edition knowledge management december 2001 the author's, or demonstrated by the user's, sets of hypertext relations for the item. these hypertext relations may be hierarchical, associative or chaining. in semiotic terms, the triad for conceptus may be represented in the diagram below. it can be seen to overlay the triads for data and information. higher order signifier information higher order signified paradigms incltacit knowledge higher order sign ixxxjmentum/ conceptus figure 4 the semiotic triad for knowledge and conceptus undiscovered public knowledge undiscovered public knowledge is identified and retrieved through literature-based discovery from syntaxis of any or all of the concepts listed above, i.e. from conjoint retrieval of item metadata and/or any other descriptors and identifiers of the documented information or oeuvre. the retrieval of undiscovered public knowledge is based on the presumption that popper's world 3 contains implicit knowledge that has not to date been stated explicitly; and contains increasing amounts of undiscovered public knowledge as the combinations and permutations of concepts exceed the number of concepts (swanson 1986, davies 1989). davies (1989) has identified a number of categories of undiscovered public knowledge: (1) hidden refutations or qualifications of hypotheses; (2) inferences from transitive relations (partial syllogisms or inference chaining, found through cocitation analysis); (3) cumulative weak tests (large sets of documents with weak evidence); (4) unrecognised or hidden analogies; (5) hidden correlations. based on previous analysis in this paper, davies' numbers (1), (2) and (4) operate at paradigm level; (3) operates at data or information level; and (5) operates at information level. swanson has extensively tested one particular approach for finding undiscovered public knowledge. he has made it available as the "arrowsmith" front-end for processing citations from medline, at http://kiwi.uchicago.edu/index.html. the approach uses data mining on title keywords, and is described by swanson and smalheiser (1997). in brief, the methodology is statistical analysis of title keywords from "complementary literatures", assuming ajis special edition knowledge management december 2001 (1) the existence of an intermediate concept b (which may be conjecture initially) between a concept represented in a set a of articles and a concept represented in a second set c of articles (2) that this concept can be deduced from putative co-citation of keywords b in sets a and c (3) and that hypotheses may be generated and/or substantiated through examination of documents retrieved using these keywords. in terms of conceptualism, the method operates on the lexical content of the item or metadata; and in terms of davies' list, it belongs to the category of inference from transitive relations, where keywords are assumed to act as markers of partial syllogisms ab and bc, inferring ac. this digression is intended to demonstrate that the descriptive framework for scientific knowledge being presented in this paper is compatible with other studies, and, importantly, has practical applications. as another (and light-hearted) aside, some virtual library interface functionalities making use of the attributes of documents, through understanding the levels of signification, can be remembered in the maproom + x mnemonic: mp: metadata parser, roo: repositories of organised objects (i.e. physical or virtual collections using knowledge organisation systems), m: metacrawler for discovery in web-based full text resources, x: literature-based discovery through semantic and syntactic characterisation of item and conceptus synopsis of insights from semiotics the hierarchical nature of the levels of description of documented knowledge can be represented as a tree-like structure, to show figuratively that information arises from data and its relations; and that scientific knowledge at paradigm level is anchored in information and its relations. alternatively, the levels of scientific meaning or signification can be illustrated in semiotic terms. the discussion in this paper to date and the models presented of data, information and knowledge are summarised in the following diagram, which clearly shows the overlaying of the semiotic triads. primary signifier code primary signified empirical results, intentions primary sign data secondary signifier secondary signified norms, rules, formulas. procedures (declarative knowledge) secondary sign information higher order signifier higher order signified paradigms (incl. tacit conceptual knowledge) higher order sign opus, oeuvre, work, teaching, "documentum", oconceptus6 figure 5 the overlapping semiotic triads for data, information and conceptus the insights summarised above, lead to a dyadic construct for a scientific document as two entities: the physical item (the medium plus encoded data or information as first and second order semiotic systems respectively); and the metaphysical "conceptus" (the signified ideational content or knowledge component, at a third or higher order of 89 ajis special edition knowledge management december 2001 semiotics, which in science is associated with paradigms or conceptual frameworks, as per kuhn 1996). postulating recursion of this type is not new. tursman (1987: 46) states that "a sign is potentially, if not in fact, a member of an infinite series of signs". he quotes peirce's 1895 writings (eisele 1976 iv: 309): a sign stands for something to the idea which it produces, or modifies. or, it is a vehicle conveying into the mind something from without. that for which it stands is called its object; that which it conveys, its meaning; and the idea to which it gives rise, its interpretant. the object of a representation can be nothing but a representation of which the first representation is the interpretant... the meaning of a representation can be nothing but a representation... so there is an infinite regression... finally, the interpretant is nothing but another representation, to which the torch of truth is handed along; and, as representation, it has its interpretant again. [lo] another infinite series. as foreshadowed in an earlier section of this paper, documented scientific knowledge may be characterised at any of the three levels of semiotic system. for example, einstein's theory of special relativity may be characterised at conceptus level, with credit apportioned in citation trails, leading back to the original publication; or in scientific textbooks it may be treated in terms of the mathematical equations derived from it (information level, with application of laws and formularies); or it may be investigated at empirical level, with published, "grey" or informal reports of data collected in experimental studies. and in a second example, weather observations and meteorology can be dealt with at the level of data, seasonal and climatic trends, and their probable causation. i would like to conclude this section by stating that there is the potential to build on and develop further some of the attributes of documented knowledge as semiotic systems, in order to increase the effectiveness of virtual library search and retrieval interfaces. the concepts associated with the ideation of the conceptus, and its aetiology and imprimatur, are especially ripe for investigation. conclusion it has been demonstrated in this paper that conceptualism and philosophy of science from the medieval scholar william of ockham through to thomas kuhn lead to a dyadic construct for a scientific document as two entities, one physical and one metaphysical: the item (the medium plus encoded data or information as first and second order semiotic systems); and the "conceptus" (the signified ideational content or knowledge component, which in science is associated with paradigms or conceptual frameworks). it is proposed that the item at data and information level, and the conceptus as separate entities, have attributes enabling their effective characterisation and management. further, it is demonstrated that any scientific document by virtue of its subject content will carry meaning significant at three levels: data; laws, trends and predictions; and the overlying set of axioms or paradigm. these attributes can be regarded as principles informing knowledge communication and access to collective memory in science. this paper suggests that the principles just described could form the basis for a number of distinct types of functionality or functional metadata for a virtual library interface. to utilise these functionalities, metadata characterising the conceptus may have to be introduced or used more widely. the insights obtained in this paper, and the potential search/ retrieval functionalities they lead to, could increase the efficiency and effectiveness of knowledge access and transfer in scientific communities, and by extension, help reduce lead times for innovation in the knowledge economy. acknowledgements while undertaking the research covered in this paper, the author has been in receipt of a scholarship funded by the school of information management & systems, monash university; and by a smurf grant to dr frada burstein, sims (knowledge management for information communities). and an earlier version of this paper was presented at the ackmids (australian conference for knowledge management and intelligent system support) 2000 conference 4-5 december 2000, melbourne, australia. references audi, r. (ed.) (1999) the cambridge dictionary of philosophy. 2nd ed. cambridge: cambridge university press. barthes, r. (1972) mythologies., trans, by a. lavers. new york: hill and wang. berkenkotter, c. & huckin, t.n. (1995) genre knowledge in disciplinary communication: cognition/ culture/ power hillsdale nj: lawrence erlbaum. 90 ajis special edition knowledge management december 2001 brown, l. (ed.) (1993) the new shorter oxford english dictionary. oxford: clarendon press. davies, r. (1989) the creation of new knowledge by information retrieval and classification. journal of documentation 45 (4) december 1989 pp. 273-301. dempsey, l. (2000) scientific, industrial, and cultural heritage: a shared approach; a research framework for digital libraries, museums and archives. ariadne http://www.ariadne.ac.uk/issue22/dempsey.html (accessed 29 february 2000). einstein, a. (1952) letter of 7 may 1952 to maurice solovine, cited by miller, a.i. (1984) imagery in scientific thought. cambridge ma: mit press p. 45. eisele, e. (1976) the new elements of mathematics by charles s. peirce vols. 1-4. the hague: moulton, quoted bytursman(1987:46). harvard business review on knowledge management (1998) boston: harvard business school press. hjorland, b. (1997) information seeking and subject representation: an activity-theoretical approach to information science. westport ct: greenwood press. hjorland, b, (2000) documents, memory institutions and information science. journal of documentation vol. 56 no. 1 january 2000 pp. 27-41. hodge, g. (2000) systems of knowledge organization for digital libraries: beyond traditional authority files. washington dc: digital library federation, council on library and information resources. kretzman, r., kenny, a., & pinborg, j. (eds.) (1982) the cambridge history of later medieval philosophy: from the rediscovery of aristotle to the disintegration of scholasticism 1100 1600. cambridge: cambridge university press. kuhn, t.s. (1996) the structure of scientific revolutions. 3rd ed chicago: university of chicago press. (first published 1962.) latour, b. & woolgar, s. (1986) laboratory life: the construction of scientific facts. princeton nj: princeton university press. meadows, a.j. (1974) communication in science. london: butterworths. ockham, w. of (1991). quodlibetal questions, volumes 1 & 2. quodlibets 1-7. translated by freddoso, a.j. and kelley, f.e. newhaven: yale university press. ong, w.j. (1982) orality and literacy: the technologizing of the word. london: routledge. popper, k.r. (1979).objective knowledge: an evolutionary approach. rev. ed. oxford: clarendon press. (first published 1972.) price, h.h. (1953) thinking and experience. london: hutchinson. quine, w.v. (1974) the roots of reference: the paul carus lectures. la salle, il: open court. quine, w.v. (1995) from stimulus to science. cambridge ma: harvard university press. schirato, t. & yell, s. (1996) communication and cultural literacy: an introduction. sydney: allen & unwin. swanson, d.r. (1986) undiscovered public knowledge. library quarterly vol 56, april 1986, pp. 103-118. swanson, d.r. & smalheiser, n.r. (1997) an interactive system for finding complementary literatures: a stimulus to scientific discovery. artificial intelligence vol 91, 1997, pp. 183-203. available http://kiwi.uchicago.edu/webwork/alabtext.html (accessed 20 july 1999). tursman, r. (1987) peirce's theory of scientific discovery: a system of logic conceived as semiotic. bloomington in: indiana university press. 91 the australasian journal of information systems (ajis) began in 1993 and was based on the simple realisation that there was a need for an academic ‘outlet’ for is research in australia australasian journal of information systems volume 18 number 2 2014 5 ensemble artifact design for context sensitive decision support shah jahan miah information systems discipline, college of business, footscray park campus, victoria university, melbourne, australia email: shah.miah@vu.edu.au john g. gammack college of technological innovation, zayed university, abu dhabi, u.a.e, email: john.gammack@zu.ac.ae abstract although an improvement of design knowledge is an essential goal of design research, current design research predominantly focuses on knowledge concerning the it artifact (tool) design process, rather than a more holistic understanding encompassing the dynamic usage contexts of a technological artifact. conceptualising a design in context as an “ensemble artifact” (sein et al., 2011) provides the basis for a more rigorous treatment. this paper describes an is artifact design framework that has been generated from the development of several practitioner-oriented decision support systems (dss) in which contextual aspects relevant to practitioners’ decision making are considered as integral design themes. we describe five key dimensions of an ensemble artifact design and show their value in designing practitioner-oriented dss. the features are user centredness, knowledge sharing, situation-specific customisation, reduced model orientation, and practice based secondary design abilities. it is argued that this understanding can contribute to design research knowledge more effectively both to develop dynamic dss, and by its extensibility to other artifact designs. keywords: dss, ensemble view, ensemble artifact, user-centredness, is artifact, design science research. introduction currently design science knowledge in information systems (is) distinguishes two different directions that are categorised variously in terms such as ‘science of design’ and ‘design science’, ‘design research’ and ‘design science research’ (e.g. cross, 2001; hevner, march, park and ram, 2004; peffers et al., 2006; winter, 2008). winter’s (2008) designation of ‘(is) design research’ vs ‘(is) design science’ reflects the essential difference in focus between specific artifact construction and a more general reflection towards theory development. within design research also are differing emphases on the specific technical design, and on its evaluation in a user context. the influential paper by orlikowksi and iacono (2001), which restored to is a focus on the it artifact, suggested five metacategories covering its various conceptions within is (namely: tool view, proxy view, ensemble view, computational view and nominal view). of these the “ensemble view” concerns “the dynamic interactions between people and technology” (orlikowski and iacono, 2001, p.126), which includes technologies enmeshed or embedded in their wider systems of use. this category, however, constituted the smallest number of is articles in their analysis of isr (information system research) articles over 10 years, despite a strong socio-technical tradition and orlikowski and iacono’s own observation (p.131) that all it artifacts are inevitably embedded in a physical setting and (dynamic) discourse of ongoing use. they conclude that detailed practices of the use of it artifacts must be integrated into australasian journal of information systems volume 18 number 2 2014 6 theory. yet since their paper little work in design science has focussed on this aspect. using winter’s (2008) distinction, our aim in this paper is to develop a richer theoretical understanding of is artifact design development together with its usage context, which is grounded in several previous case-studies of practitioner oriented decision support systems (dss) development. arnott and pervan (2005, 2012) have suggested that there is a need to improve the quality and relevance of research into dss development and evaluation. in particular the clients and dss artifact users are poorly identified in dss scholarship (arnott and pervan, 2005), impacting relevance directly. the role of the artifact is one of the eight key issues they identify for the field (arnott and pervan, 2008) from the large sample of articles they analysed. noting decision support’s “design science heritage”, they present statistics to indicate that both a micro focus on issues of artifact design (e.g. it development), and a macro focus on (e.g organisational impact) are represented. it is unclear however, how many of the papers analysed considered both together. their later analysis, however, which focussed explicitly on design science papers in dss (arnott and pervan, 2012) found that the focus of 42.3% of papers was “the presentation and description of an artifact without any attempt to establish its worth, effectiveness or usefulness” (p.933). clearly issues of worth, effectiveness and usefulness will directly be referenced to professional practices in the field that the dss addresses, and whilst some incidental understanding may emerge from traditional requirements gathering and analysis, it is unlikely that a full appreciation of changing requirements can be captured given the dss developer is rarely an expert end user or domain practitioner. a different design approach is indicated. the analysis of design research knowledge is mainly based on 1) conceptualization of the design research theory (arnott and pervan, 2012; mckay and marshall, 2007; mckay, marshall and heath, 2008; gregor and jones, 2007); 2) theory building for enhancing our understanding of design science methodology (hevner et al., 2004; iivari, 2007;venable, 2006) and 3) adaption of design research in is (purao and storey, 2008; botts, schooley and horan, 2008; muntermann, 2009). the guidelines and methods of design research address the requirements of innovative design development, solution modelling or problem solving, and the evaluation of design (hevner et al. 2004; carlsson, 2006; mckay and marshall, 2007). many empirical studies have predominantly focused on different aspects of hevner’s et al. (2004) several guidelines, and suggested implications for better understanding of design research in is development, but the original agenda of hevner’s guidelines fundamentally focuses on the technical artifact itself. the theoretical emphasis argued as necessary e.g. by venable (2006) and by gregor and jones (2007), requires more explicit attention to the dynamic systems in which the artifact is embedded beyond the designed “solution” of a computer based is referenced to some static business requirements. as naturally embedded artifacts, dss “represents a different philosophy of support, system scale, level of investment, and potential organisational impact. they can use quite different technologies and may support different managerial constituencies” (arnott and pervan, 2005 p. 68). clark, jones and armstrong (2007) recognised the need to develop dynamic theoretical models for support systems development such as dss and related technologies such as eis (executive support systems), kms (knowledge management systems) and business intelligence (bi). their conceptual model (clark et al., 2007) aimed not only to provide core and fundamental elements such as business process, organisational members, technology and organisational outcomes but also to capture the decision making environment and the use of these systems. clearly this dynamic interaction argues for adopting an ensemble view. the ensemble view is not new, but perhaps helps frame against the other meta-categories the older socio-technical tradition developed after wwii primarily by the tavistock institute (mumford, 1983). this group identified a principle for participatory design in the context of artifact use namely “to australasian journal of information systems volume 18 number 2 2014 7 increase the ability of the individual to participate in decision taking and through this to exercise a degree of control over the immediate work environment. managers were advised to tell work groups what to do but not how to do it. the latter would come from the knowledge, experience and skill of each work group”. (mumford (n.d) see mumford and beekman (1994)). long established in the uk, scandinavia and australia, many contemporary theorists also recognise the need, when developing is applications, to include all aspects and how their processes and relationships are “entangled in practice” (e.g. crowston, 2000, orlikowski, 2009). this has led to various attempts to retheorise these concerns, particularly with respect to design science research. piramuthu and shaw (2009) noted that traditional methods using pre-programmed routines help develop static dss, however decision-making environment are not static and the fit is poor when a static tool is used in a dynamic environment. they suggest an adaptive framework with learning capabilities to incrementally update the knowledge base by monitoring for “stale” knowledge, but their perspective remains essentially a technological one. sein et al. (2011) proposed action design research (adr) as an approach to design research that assumes building the artifact is “interwoven” with organisational deployment and evaluation. they note in passing that their adr approach essentially articulates what others “serendipitously” do in practice, and their main focus is on detailing an appropriately relevant and rigorous research method for this. theoretically adopting the ensemble view of orlikowski and iacono (2001) they claim to provide “methodological guidance for is researchers who study the design of ensemble artifacts” (p.53). goldkuhl (2012, 2013) however found sein et al’s transformation from an ensemble view to an ensemble artifact problematic, in part because it reifies a special class that implies non-ensemble artifacts exist. his conceptual paper looked more closely at the theoretical utility of the putative ensemble artifact by articulating more fully characteristics of the ensemble view only implicit in orlikowski and iacono’s (2001) abstraction from particular sub-views. these include the essential embeddedness of the artifact in a dynamic social context, and the embodied social norms and structures built into the artifact’s design. his work did not attempt any methodological prescription, but argues the need for better theory in this area1. socio-technical design is essentially a complex design process that includes interaction between the technical and social systems in order to encompass the totality of the design (mumford, 1995). from a critical perspective, carlsson (2006) argued that is applications can be seen as socio-technical systems rather than just technology-centred artifact design and that design research should develop practical knowledge for the design and realisation of is initiatives on this basis. the anticipation with this type of holistic analysis is that the context and relevance of is should be more understood and explicitly accounted for. this motivates our general research question: how can we conceptualise is artifact design to effectively accommodate its contextual usage? more specifically, we focus on designs for decision support, representing a class of is that by nature essentially involves interaction between a designed artifact and a human decision maker using its outputs in a dynamic context. the structured-case approach of carroll and swatman (2000) “enables the development of deep understanding of the complex interaction of people, processes and technology within organizations” (p.236) and builds theory from examination of professional practice. importantly, this approach allows systematic and dynamic theory development through iteration and documented links to data. informed by this, to capture user’s situational realities and relevance to product, we use case studies of developed 1 in the discussion we consider a response to this work by sein’s team (purao et al., 2013). australasian journal of information systems volume 18 number 2 2014 8 dss application designs targeted to decision support problems (in particular where the design has to meet changing demands within a business context). from these we identify theoretical constructs, both from the literature and arising in practice that characterise components of a more general framework for rigorously building and evaluating dss that are relevant directly to the practitioners’ contexts. the rest of this paper is organised in the following way. the next section describes the background of our problem in this study. next, the methods used to develop the framework of artifact design are described, followed by the identified key components of our ensemble artifact. finally, a discussion and summary are presented, along with proposed potential benefit to the knowledge in the field of design research for dss development. background our aim in this study is to extend the artifact design knowledge in design research. design research “…seeks to create innovations that define the ideas, practices, technical capabilities, and products through which the analysis, design, implementation, management, and use of information systems can be effectively and efficiently accomplished” (hevner et al., 2004, p.76). winter (2008) suggested that although many contributions have been made to the justification of design, the typology of artifacts, or specific problem solutions, and rigour-related aspects are not yet sufficiently standardized to the design research community. march and smith (1995) defined differentiation of constructs, models, methods, and instantiations as artifact types that have been well-accepted in is design research (hevner et al., 2004). table 1 represents the meanings of the types of is artifact. is artifact as definitions constructs this type of artifact constitutes the language to specify problems and solutions models this type of artifact uses this language to represent problems and solutions methods this type of artifact describes processes which provide guidance on how to solve problems instantiations this types of artifact explains problem-specific aggregates of constructs, models, and methods table 1: meanings of the artifact types (adapted from march and smith, 1995) improvement of the artifact design knowledge is an essential component of design research (hevner et al., 2004; hevner, 2007). this understanding can be helpful in guiding straightforward it artifact design, if the main focus is in designing a new it solution. historically, constructs, methods and models are well covered in design research for dss development (arnott and pervan, 2012). our aim here is to examine particular instantiations of dss artifacts as it is important to explain abstractable properties of technological dss artifacts in order to provide applicable components “… for choosing desirable ends i.e., for normative actions” (winter, 2008, pp. 471) and to extend these to design knowledge for their usage context. the design science research school (what carlsson (2007, p. 213) called the “it design science research school”) has however, a technological focus that explicitly excludes numerous soft factors. for example cao et al. (2006) conceptualised design knowledge as a technology-centred design that concerns itself with technical innovation in the it artifact design. however, mckay and marshall (2007) and mckay et al. (2008) argued that this type of conceptualization excludes the surroundings of the design artifact. it shows a lack of understanding about is artifacts design in an organizational australasian journal of information systems volume 18 number 2 2014 9 context, where there are many human and social components. a design can be innovative in terms of how it can add value to decision makers/practitioner’s desires, to improve practices or to achieve goals in its target problem space, but such features lie in the organisational or social realm, not the technological provisions. further to this, iivari (2007) suggested that a design theory is not necessarily based on any scientifically validated knowledge; rather it could be based on any practitioner theory-in-use. this pragmatic concern also reinforces the generation of a collective understanding that may capture soft factors in design specially to better reflect practitioner-oriented surroundings into a design process. mckay et al. (2008) explicitly viewed professional practices as a design dimension seeing identified responsibilities to clients or designers and broader social and environmental responsibilities as a design value. this view of design emphasises the “situatedness of the designer in a real-world context involving uncertainty, ambiguity and value conflict, and inevitably links design to the personal experiences, capabilities, knowledge and intuition of the designer” (p.9). it implies that practitioneroriented surroundings should have reflection built into a contextualised design process to better conceptualize and realize artifacts intended to address the target problem situation. this can assist capture of changing requirements and situational realities beyond the traditional requirements scope snapshot. however, involving professional practices as a design dimension has not been explored so far in existing dss or design science literature. to address this, findings from several different practitioner-oriented dss development projects were revisited in this study. the development and evaluation details of a major project in rural industry (miah, kerr and gammack, 2009; miah, 2008), a second project in forestry pest management (miah, debuse, kerr and debuse, 2010) and third dss development concerned in supplier selection issues (miah, ahsan and msimangira, 2013). the previously published papers reported these dss development projects separately highlighting their contribution within the targeted problem domains and describing their technical architectures. in addition, in earlier work following a tailorable and explicitly user-centred (“human-centred”) design philosophy several dss were built in the financial sector, to assess credit risk, to identify potential card fraud, and for various insurance applications (gammack et al., 1992). whilst these latter developments were mainly proofof-concept prototypes at least one went on to commercialisation and this work also informed the present study. in this paper, we focus on generalising the implications from the holistic design and evaluation activities in these various domains in considering professional practice as a design dimension. the findings were based on representative stakeholders’ opinions solicited through focus group activities and system prototype evaluations. focus groups have been recommended as a justified method in the design research paradigm (tremblay, hevner, berndt, 2010), who described the adaptation of exploratory focus groups for artifact refinement and confirmatory focus groups for establishing the utility of the artifact in their healthcare design project. arnott and pervan (2012) also recommended their use given the endemic weakness of evaluation in dss research and the narrow methodological base currently used for this. generating such collective knowledge in the dss artifact design may support development of flexible dss applications by elevating the understanding of the end-users’ work activities and the context in which they work. this embraces a user work centredness perspective i.e.“understanding the relationship between people, technology, work requirements and organizational constraints in work settings, where people are actors in situations” (iivari and iivari, 2011, p. 131) who also considered that the extension of work centredness to activity-centredness could be a useful research challenge for future is design. as an illustration of the work-oriented activity-centred method, the dss artifact should take the system, users and the situation of use all together for an application development and their own context of use. beyond these dss design studies where an it-based solutions is the key focus, we look at the dss artifact seen from a collective innovation perspective as a socio-technical design, following carlsson’s (2006) suggestion that is artifact design can be perceived as a socioaustralasian journal of information systems volume 18 number 2 2014 10 technical system. the adopted view is similar to the view of mackrell, kerr and von hellens (2009), in which a socio-technical method was utilised to develop an agricultural dss. therefore our focus is on identifying relevant components of such an ensemble artifact using our previous research cases to enhance knowledge of socio-technical design research for designing specific dss artifact. the “ensemble artifact” concept thus extends to addressing organisational and user context of use into useful it artifact design. method gregor (2005) identifies five interrelated types of theory namely: theory for analysing; theory for explaining; theory for predicting; theory for explaining and predicting; and theory for design and action. based on this classification of theory, we find the theory for explaining is most relevant to better describe our knowledge generated from the dss development studies. it equally “could well be labelled as ‘theory for understanding’ as these theories often have an emphasis on showing others how the world may be viewed in a certain way, with the aim of bringing about an altered understanding of how things are or why they are as they are” (gregor 2005, pp. 19). we adopt this view to explain our understanding on how things are and why they are considered important to enhance our understanding on artifact design. as mentioned earlier carroll and swatman (2000) described a methodological framework called ‘structured-case’ in order to conduct is theory-building research. this approach aimed help “building theory from field research” that examines professional practices. carroll and swatman (2000) described the approach for such field research that “enables the development of deep understanding of the complex interaction of people, processes and technology within organisations” (p.236) around three key elements: conceptual framework, research cycle and literature scrutiny for targeted theory building. according to this, the conceptual framework is initially produced from the research themes and existing knowledge gathered from literature. this helps form a basis for a research cycle conducted through four stages (plan, collect data, analyse, and reflect) similar to action research approaches, finally literature scrutiny of targeted theory building compares the tentative findings with literature-established knowledge leading to review and revision of the understanding, through consolidating and examining conflicts between findings and literature (carroll and swatman, 2000). based on their three element roadmap, we describe our research activities into three phases. phase 1 the decision making problems identified for the three different problem domains are similar in nature in particular where the solution designs for the decision support had to meet changing demands within the targeted business context. the first dss design cycle was conducted in rural industries in which key decision makers were farmers and extension professionals in the context of the dairy farming business. the decision making, particularly in the dairy businesses in queensland has faced rapid changes due to the effect of climate change, government regulations and farming method changes. this change has resulted in disuse of many dss applications (mccowan, 2002; cox, 1996; and jakku et al., 2004). the responsible government department such as the department of primary industry and fisheries (dpi) was looking for suitable dss systems as well as an effective process that could improve management of the decision support application development activities (e.g. managers’ roles are important to monitor the domain experts while involving dss application development for the specific farmers group, in the contemporary dpi management context). end users such as farmers’ systems on the other hand, needed to be accommodated to current contingencies. to address the changing needs for the target stakeholders in the industry (such as farmers, extension professionals and managers), an initial solution was designed to reflect on the appropriate interaction in which farmers can achieve their situation-specific decision support obtaining expertise and knowledge from dairy extension australasian journal of information systems volume 18 number 2 2014 11 professionals. a rule based (if-then-else) decision approach which is tailorable to farmer’s rules of thumb (practice based knowledge), was used for the end user system. the solution was evaluated through focus groups. the following protocols (see table 2 below) were used: steps descriptions step 1 introduction to the project and its goals step 2 general information of the developed method given to all participants step 3 prototype method is demonstrated by running industry relevant examples step 4 participants are asked specific questions about the method and if there were areas they were unsure of step 5 a time interval is offered to the participants to use the method step 6 have discussion and capture participants views on the key functionalities step 7 participants are requested to provide detailed information about their understanding and views of the method step 8 at the end of the focus group meeting, the participants are thanked for their time and effort table 2: evaluation protocols used for the dss development studies phase 2 the process developed in phase 1 was modified to adjust with work environment of forestry pest management. in this problem space, most of the previous dss processes use knowledge from various domains in providing support for specific decision making problems. however, very few studies have identified the requirements of developing a combined dss process in which relevant practitioners can contribute and share knowledge for effective decision making (miah et al., 2010). the model-based dss such as dymex (maywald et al., 2004), a very popular dss, are unformed in terms of the involvement of practitioners in system development (miah et al. 2010). at the same time, pest management knowledge is continually evolving, and dss should reflect this by incorporating new knowledge to determine the most appropriate practice. the two key stakeholder groups, scientists and foresters, were identified within this problem domain. to address the issue here, the proposed modified dss process was mainly to enable the interaction between scientists and practitioners for better knowledge incorporation. to enhance collaboration in their work environment we invented the key system features in which end users (such as foresters) can adjust their business factors by adding/modifying/deleting into the system, under a secondary design principle2 (germonprez, hovorka and gal, 2011). users draw their particular problem scenario to find out particular scientific knowledge 2 secondary design principle can be seen as an ongoing activity for tailoring is applications by the end users so that it fits within their context. germonprez et al. (2011) described the secondary design view where “people are active, aware, and intentional participants in an ongoing process of embodied interactions involving technological and social dualities” (p.663) australasian journal of information systems volume 18 number 2 2014 12 support to improve or minimise the potential risks. scientists can add scientific knowledge and stepby-step suggestions to improve the foresters’ specific practices. this use of inputs from end users helped to implement better interactions in is solution artifact design. the domain specific details provided us to allocate features of knowledge sharing in the relevant decision support specific to situation and its possible customisation required to adjust the changes demands. our findings from the third project in a supplier selection problem in the supply chain management domain suggested similar demands as purchase officers need to have knowledge sharing with their managers in order to make decisions on which suppliers will be suitable to purchase specific products (miah et al., 2013). the selection parameters change due to the market or price/service terms and conditions changes. as a value adds to the solution design, we employed the user-centred and work-related features as value adding provisions to dss solution artifact, as part of the innovation. table 3 illustrates some comments captured during the system evaluation using same protocols. dairy industry stakeholders farmers “simple means of organising thoughts into a logical framework …. ability to modify, and suited to changing environment in addressing specific issues on a industrial farm” extension officer “the system seems overall simple and straightforward in data entry to me, however, it needs to incorporate the biological settings to improve the ability of the system which could be done by a knowledgeable user”. (research diary: 15 february 2008) dpi management “this is a nice little piece of software where we may control the decision support tools development activities which are very important from the management point of view”.( research diary: 22 august, 2007) forestry industry stakeholders forestry health scientist “because for the dymex [a popular dss in forestry] the leaf beetles will only be a proportion of the damage in some years. they’re very hard to find, let alone see the damage. so i think yeah, like mr_x_ was saying, the generic model is probably a good one to start with, and we know that we’ve got the industry support for that” table 3: example focus group data to demonstrate the practitioners’ orientation during the design research phase 3 to refine our practical understanding, we analyse our findings through dss literature to justify its value in dss development. research work by arnott and pervan (2008; 2006; 2005) consistently outlined the gap of user’s involvement as mentioned earlier. we aimed at interpretive understanding to learn about target decision making aspects of practitioners (in both cases) within their practical environment. from phases 1 and 2 we found that a socio technical design is important for industry practitioners’ dss design. to enhance tailorability, knowledge sharing and work-activity centredness, the design features were identified to address the vital issues such as practitioner context of use and knowledge sharing and collaboration issues. the findings led to identify the relevant components for a framework of dss artifact design. australasian journal of information systems volume 18 number 2 2014 13 components of the “ensemble” artifact the concept of it artifact is problematic in design science and in is generally (alter 2003). for present purposes we refer to the “ensemble artifact” characterised in the action design research approach of sein et al. (2011) to locate our work, albeit with some qualification as we discuss later. we also recognise the numerous particular formulations that offer good analysis, coherent arguments and constructive suggestions but which have perhaps hampered a standardised conceptual coherence. three examples may illustrate: the communication tool view of it artifacts (goldkuhl, 2012); the semizoic artifacts proposed by gregor and iivari (2007) which recognises the spectrum of design from the formulaic to its changing and emergent aspects, and alter’s (2003, 2013) “ work systems in organisations” vision (which makes deference to the broad predecessor sociotechnical tradition) and where the term “it artifact” is understood (preferably) as the entire it-reliant work system. whilst not identical, all can be seen as useful integrations around a similar problematic and embracing an ensemble view of design. our intention here is not to add a competing formulation, but to identify theoretical components relevant to design that explicitly follows an ensemble view and which can become operationalized in specific design contexts. we analysed relevant practical findings of dss solution developments to find out key components of dss artifact design in practical contexts of design. the findings suggest five key components that are of value in designing practitioner-oriented dss design. the components are user centredness, knowledge sharing, situation-specific customisation, reduced model orientation, and practice based secondary design abilities. our view is similar to the view of mckay et al. (2008) in that it suggests the potential use of a human centred perspective in design to expand contextual knowledge in a design artifact. based on a theoretical analysis mckay et al. (2008) identify various viewpoints of design such as design as product, design as process or action, design as intention, design as planning including modelling, representation and method, design as communication, design as user experience, design as a value, design as professional practice and design as service. our study is different in that our case analysis is focused on the design perspectives around users and their situation of use to illustrate some key components of design in particular to dss design. the following sections describe the identified components: user-centredness of artifact iivari and iivari (2011) suggested four aspects of user-centredness, namely, a focus on the system user; a focus on user work-centredness; a focus on user involvement; and a focus on system personalisation. we found that dss design requires end-user inputs to outline the support processes required for artifact design. at the same time, dss analysts need to understand end user desires and decision support requirements. the dss literature supports this perspective for example in agricultural dss development contexts (cox, 1996; mccown, 2002): mccown (2002) particularly noting a lack of fit between knowledge embodied in the system and the dynamic industry context in which business operates. contemporary design research suggested that design should focus on design by enhancing user centredness through encouraging or empowering them in the design process; this can be defined as a focus on the user work centredness (iivari and iivari, 2011), an argument long advocated by alter and others (e.g. alter, 2013). although many dimensions of user centredness such as user involvement or system personalisation can have application in dss design, work centredness appears to be most relevant to practitioner-oriented dss design, since most practitioners would expect their work environment reflected within the solution design. in alter’s work system theory (2013) he notes implementation means in the organisation, not the algorithm, and modelling provision for workarounds, bricolage and emergent change is required. australasian journal of information systems volume 18 number 2 2014 14 designing (decision) environments around work-referenced “design spaces” provides for this secondary design capability. knowledge sharedness of artifact in the scenarios of dss developments, we found that dss should have features for knowledge sharing between user groups, specially for incorporating new knowledge to determine appropriate or effective practices and for relevant decision support aids. specifically, the dairy dss includes knowledgesharing provisions for the interaction of farmers and extension professionals (miah et al., 2009; miah, 2009), whilst the forest pest example represents a process for scientists and practitioners to incorporate new scientific knowledge into practice. the practical findings show way to map both features for practitioners and domain expert (such as dss analysts). this perspective has been reflected in the dss literature, particularly in medical informatics. for example, achour et al. (2001) highlighted the importance of knowledge sharing feature in clinical dss design for different user groups such as doctors and other health professionals to determine appropriate treatments. in design research, hevner et al. (2004) suggested that design artifact must address the complex interactions among subcomponents of the problem and its solution. in our case, the relevant interaction in terms of knowledge-sharing is seen as a key component of decision support. extensible knowledge representations such as contextual graphs that allow for updating without re-engineering may be used to add dynamism to a dss (sherwell et al., 2005). situation specific customisation of artifact situation specific customisation in dss development for rural business applications has been identified within the agricultural dss literature. kerr (2004), mccown, (2002), and hayman and easdown (2002) noted that most agricultural dss are not suitable for rapidly changing situations such as industry regulations and marketing policies that impact directly on decision making, since they lack customisable features to such changes. the dss development projects demonstrated a clear requirement of situation-specific customisation. practitioner-oriented dss should have some dynamic provisions for adding or modifying decision making parameters to meet changing requirements without requiring intermediate design stages. in design research, germonprez et al. (2011) described a case of behaviour design research in that a secondary view is proposed to consider users as designers in their own right during the ongoing creation and recreation of application to adjust with problem space. this user-as designer philosophy was also more fully articulated and implemented in the dss developed by gammack et al. (1992). reduced model orientation of artifact prior dss development studies utilised simulation and survey data for dss model development (kerr and winklhofer, 2006; karmakar, lague, agnew and landry 2007). most such models become obsolete quickly due to the changes of user environment and business situations. for example, if new knowledge is to be incorporated, dss may suffer from knowledge transferring issues such as advising difficulties between farmers and extension officers (mccown, 2002; kerr, 2004; carberry et al., 2002). outcomes from model based dss also vary site to site, season to season and agreed practice to practice. in design research, the situation appears due to lack of understanding about is artifacts design in an organizational context, where many behavioural factors, involving human and social components are to be acknowledged. in other words, a design must also be innovative in terms of how it can add value to decision makers/practitioner’s practical desires within their own context. to recognise the need, when developing a dss artifact, we should have a reduced model orientation to targeted artifact design so all aspects can be addressed through how the processes and relationships should be “entangled in practice” for the targeted user. australasian journal of information systems volume 18 number 2 2014 15 practice based secondary design facilities of artifact one of the issues in previous dss development is the lack in practical knowledge for dss design, as most models are designed as a result of research or theory building or problem solving (arnott and parven, 2012; jakku et al., 2004; mccown, 2002; cox, 1996). attempts to train or encourage end users to develop dss have traditionally failed and for complex systems this approach is undesirable (power, 2010). but a technical focus on the developed dss does not consider domain-centric practical realities. we found it is important to accommodate practitioner’s practical consequences into decision-making context. for instance, in dairy dss, the study articulated practical decision making rules of thumb that are significant for dss design. in design research this reality can also be informed through the idea of secondary design germonprez et al. (2011) in that the design perspective recognizes user’s behaviours and contexts change over time and when is artifacts are engaged and customised by users in their work or problem space. an architecture that allows both end user customisation and more rigorous and informed tailoring through a secondary design facility helps address organisational dynamism, and was implemented in the systems described by gammack (2002) and miah (2009). discussion and conclusions for dss design, traditional dss development methods have several limitations in supporting businesses, including conceptual mismatches, static models and inflexibility. this has resulted in poor uptake or disuse. the study considered the contextual knowledge around (dss) artifact design in which an ‘ensemble’ artifact design view is adopted to represent entanglement of artifact, user and their situation of use. in the specific dss development context the approach is considered to address more practically the utility and ongoing relevance of a design. our approach integrates the simple it artifact design knowledge with explicit consideration of use in a practical context by identifying five key components for ensemble artifact design. the key components of framework are user centredness, knowledge sharedness, situation-specific customisation, reduced model orientation, and practice based secondary design abilities, combined in order to enhance the influence of professional practices as an explicit design dimension in artifact development. these components emerged from a structured-case approach based on a range of decision support systems (dss) development contexts, in which abstractions and reflections on key findings from potential stakeholders formed the basis of a grounded understanding. these five components capture central aspects of an artifact’s ‘interwoven’ characteristics and help identify requirements of interactions between people and technology. methodologically the action design research approach of sein et al. (2011) is naturally compatible with this, and indeed in our earlier individual studies, the iterative, evolutionary and joint application development and evaluation processes that were followed ensured continual referencing to the usage context (see miah et al. 2009, miah, 2009). the established practices in the tool focussed aspects of it artifact design, although concerned with the design prior to implementation and use and cognizant of both technical and behavioural aspects, leave concerns around users’ situations and related practicalities un-addressed in the context of their everyday or professional activities. our framework addressed this gap by enabling user-centredness as an important component of dss artifact design supported through specific user-centred principles (iivari and iivari, 2011) and making provision for secondary design. regarding knowledge sharing, a common issue across developments was the dynamism in the organisational context, where unanticipated factors required design adaptations. in the agricultural applications for example, changing market demands, new business rules and scientific findings known to the government must be shared with the (farmers) who themselves have local knowledge that requires contextual interpretation and a customisation capability. to facilitate dynamism the model orientation is reduced since over-fixing a technical solution is inflexible, obsolescent, and may become irrelevant with only a minor change in australasian journal of information systems volume 18 number 2 2014 16 organisational context. such changes can be managed though by designing for tailorability using secondary design approaches. we believe these five elements are all essentially understood, conceptualised and operationalized variously throughout the is literature, and implementing them in specific designs using the ensemble view is practicable. theoretically, however, we have largely refrained from using the term “ensemble artifact” because although we broadly agree with sein’s et al. (2011) description a deeper problem is the field’s various conceptualisations of the it artifact and whether the term is generally useful in building design science knowledge. we therefore conclude our discussion with a brief consideration of this point. alter (2003) argued that the notion of “it artifact… is too unclear to serve as a basic concept for defining the is field” (p.496) and winter (2008) suggested that the typology of artifacts to specific problem solutions are not yet sufficiently standardized to the is design research community. goldkuhl’s (2012) penetrating criticisms of (sein et al.’s) proposed “ensemble artifact” were “rebutted” by that team (purao et al., 2013) by claiming that the short hand term (derived from but not to be compared to the ensemble view) “emphasizes the key characteristics that exhort is researchers to adopt a view of all it artifacts as ensemble artifacts” (p.76). this would be theoretically vacuous if it mapped to orlikowski and iacono’s set of views, but these authors disclaim any direct lineage, arguing that their contrast is with the dominant “tool view” in design research, which is our own position. more usefully they rephrase the incisive criticism “ …“aren't all artifacts ensemble artifacts” (answer =yes) as “…“are all artifacts built as ensembles?” (answer = no)” (p.78). this does not end the argument of course, but highlights the ongoing problems in our field of settling on any standard set of basic ontological categories. our work has been on dss and we do not make a claim to greater generality here. since, however, dss exemplifies an essentially embedded, interwoven and dynamic application area, the framework suggests a generalisability to other contextualised is artifacts. our proposed framework of holistic dss artifact design is supported through the incorporation of both user-centred and secondary design principles that has not previously been dominant within the dss development context. this may lead to wider transformation in dss development processes and to relevant knowledge for design science research more widely. references achour, s.l., dojat, m., rieux, c., bierling, p. and lepage, e. (2001). a umls based knowledge acquisition tool for rule based clinical decision support system development, journal of the american medical informatics association, 8, 351-360 alter, s. (2003). sidestepping the it artifact, scrapping the is silo, and laying claim to systems in organizations. communications of the association for information systems, 12(30), 494-526. alter, s. (2013). work systems theory: overview of core concepts, extensions, and challenges for the future, journal of the association for information systems, 14 (2), 72-121 arnott, d. (2006). cognitive biases and decision support systems development: a design science approach, information systems journal, 16, 55–78 arnott, d. and pervan, g. (2008). eight key issues for the decision support systems discipline, decision support systems, 44(3), 657-672 arnott, d. and pervan, g. (2012). design science in decision support systems research: an assessment using the hevner, march, park, and ram guidelines, journal of the association for information systems, 13 (11), 923-949. australasian journal of information systems volume 18 number 2 2014 17 botts, n., schooley, b. and horan, t.a. (2008). learning object development and repository design for digital government instruction, the proceedings of the 41st hawaii international conference on system sciences, usa, 1-10. cao, j., crews, j.m. lin, m. deokar, a.v. burgoon, j.k. nunamaker jr. j. (2006). interactions between system evaluation and theory testing: a demonstration of the power of a multifaceted approach to information systems research journal of management information systems, 22 (4), 207-235. carberry, p.s., hochman, r.l., mccown, r.l., dalglish, n.p., foale, m.a., poulton, p.l., hargreaves, j.n.g., hargreaves, d.m.g., cawthray, s., hillcoat, n. and robertson, m.j. (2002) the farmscape approach to decision support: farmers’, advisers’, researchers’ monitoring, simulation, communication and performance evaluation, agricultural systems, 74 (1), 141–177 carlsson, sven a. (2006). design science research in information systems: a critical realist perspective, the proceedings of the australasian conference on information systems (acis 2006). paper 40. http://aisel.aisnet.org/acis2006/40 accessed 2/11/13 carlsson, s.a. (2007). developing knowledge through is design science research: for whom, what type of knowledge, and how, scandinavian journal of information systems, vol. 9(2), pp.75-86 carroll, j.m. and swatman, p.a. (2000). structured-case: a methodological framework for building theory in information systems research, european journal of information systems, 9, 235–242 clark, t.d., jones, m.c., and armstrong, c.p. (2007). the dynamic structure of management support systems: theory development, research focus, and direction, mis quarterly, 31 (3), 579-615 cox, p.g. (1996). some issues in the design of agricultural decision support systems, agricultural systems, 52, 355-381 cross, n. (2001). designerly ways of knowing: design discipline versus design science, design issues, 17(3), 49-55 crowston, k. (2000). processes as theory in information systems research. in r. l. baskerville, j. stage & j. i. degross (eds.), organizational and social perspectives on information technology proceedings of the ifip tc8 wg8.2 international working conference on the social and organizational perspective on research and practice in information technology, kluwer academic, june 9–11, arlborg, denmark, 149–164. gammack, j. g. (2002). constructive design environments: implementing end-user systems development. in t. barrier (ed.), human computer interaction development & management, hershey, pa: irm press, 153-173 gammack, j. g., fogarty, t. c., battle, s. a., ireson, n. s., & cui, j. (1992). human-centred decision support: the idioms system. ai & society, 6(4), 345-366. germonprez, m., hovorka, d. and gal, u. (2011).secondary design: a case of behavioral design science research, journal of the association for information systems, 12 (10), 662-683 gregor, s. (2005). the nature of the theory in information systems, mis quarterly, 30 (3), 611-642 gregor, s. and iivari, j. (2007). designing for mutability in information systems, in d. hart and s. gregor (eds.). information systems foundations: theory, representation and reality, australian national university e-press, canberra. (url: http://epress.anu.edu.au/info_systems02_citation.html) gregor, s. and jones, d. (2007). the anatomy of a design theory, journal of the association for information systems, 8, 321–335. australasian journal of information systems volume 18 number 2 2014 18 goldkuhl, g. (2012). what is an ensemble artefact?. international workshop on it artefact design & workpractice intervention, 10 june, 2012, barcelona http://liu.divaportal.org/smash/record.jsf?pid=diva2:583573 goldkuhl, g. (2013). from ensemble view to ensemble artefact – an inquiry on conceptualisations of the it artefact, systems, signs & actions , vol 7 (1), p 49-72 hayman, p.t. and easdown, w.j. (2002). an ecology of a dss: reflections on managing wheat crops in the north-eastern australian grains region with wheatman, agricultural systems, 74, 5777 hevner, a., march, s., park, j., and ram, s. (2004). design science in information systems research, mis quarterly, 28(1), 75-105 hevner, a. (2007). a three cycle view of design science research, scandinavian journal of information systems, 19 (2), 87-92. iivari, j. (2007). a paradigmatic analysis of information systems as a design science, scandinavian journal of information systems, 19 (2), 39-64 iivari, j. and iivari, n. (2011). varieties of user-centredness: an analysis of four systems development methods, information systems journal, 21, 125-153 jakku, e. thorburn, p. and gambley, c. (2004). decision support systems for farm management: a theoretical framework from the sociology of science and technology, in the proceedings of the 4th international crop science congress (url: http://www.cropscience.org.au/icsc2004/poster/4/1/1/1219_jakkues.htm retrieved on 20 january, 2012) karmakar, s., lague, c., agnew, j. and landry, h. (2007). integrated decision support system (dss) for manure management: a review and perspective, computers and electronics in agriculture, 57, 190-201 kerr, d. and winklhofer, h. (2006). the effect of rapid rural industry changes on the development of a decision support system for dairy farmers in australia, computers and electronics in agriculture, 50, 61-69 kerr, d.v. (2004). factors influencing the development and adoption of knowledge based decision support systems for small, owner operated rural businesses, artificial intelligence review, 22 (2), 127-147 maywald, g.f., sutherst, r.w. and kriticos, d.j. (2004). dymex professional: modelling natural systems, version 2.0, hearne scientific publishing, melbourne. mackrell, d., kerr, d.v. and von hellens, l. (2009). a qualitative case study of the adoption and use of an agricultural decision support system in the australian cotton industry: the socio-technical view, decision support systems, 47, 143–153. march, s.t., and smith, g. (1995). design and natural science research on information technology, decision support systems 15(4), 251-266 mccown, r.l. (2002). changing systems for supporting farmers decisions: problems, paradigms, and prospects, agricultural systems (74), 179 -220 mckay, j. and marshall, p. (2007). science, design and design science: seeking clarity to more design science forward to information systems, in the proceedings of the 18th australasian conference on information systems, sydney, 1-11 mckay, j. marshall, p. and heath, g. (2008). an exploration of the concept of design in information systems, in the proceedings of the 4th biennial anu workshop on information systems foundations, information systems workshop, australian national university, canberra, australia. australasian journal of information systems volume 18 number 2 2014 19 miah, s.j. (2008). an ontology based design environment for rural decision support, unpublished phd thesis, griffith university, australia. miah, s.j. (2009). end user as application developer for decision support, in the proceedings of the fifteenth american conference on information systems, aug 6-9 2009, san francisco, california, usa, 142 miah, s.j., kerr, d. and gammack, j. (2009). a methodology to allow rural extension professionals to build target-specific expert systems for australian rural business operators, expert systems with applications (36), 735–744. miah, s.j., debuse, j., kerr, d. and debuse, v. (2010). a practitioner-oriented decision support process for forestry pest management, in the proceedings of the 21st australasian conference on information systems, 1-3 dec 2010, brisbane, australia, pp. 1-10 miah, s.j, ahsan, k. and msimangira, k.a.b. (2013). an approach of purchasing decision support in healthcare supply chain management, operations and supply chain management: an international journal, 6 (2), 43 – 53. muntermann, j. (2009). towards ubiquitous information supply for individual investors: a decision support system design, decision support systems, 47, 82–92 mumford, e. (1995). effective requirements analysis and systems design: the ethics method, macmillan, london. mumford e. (1983). designing human systems for new technology the ethics method, manchester business school. mumford, e. and beekman, g. j. (1994). tools for change and progress: a socio-technical approach to business process re-engineering. leiden: csg quote is summary by mumford at http://www.enid.u-net.com/sociotech.htm 2/11/13 orlikowski, w. (1992). the duality of technology: rethinking the concept of technology in organisations, organisation science, 3 (3), 398–427 orlikowski, w. j. (2009). the sociomateriality of organisational life: considering technology in management research, cambridge journal of economics, 34 125-141 orlikowski w j, and iacono c. s. (2001). desperately seeking the “it” in it research – a call to theorizing the it artifact, information systems research, 12 (2), 121-134 purao, s and storey, v.c. (2008) evaluating the adoption potential of design science efforts, the case of apsara, decision support systems, 44, 369-381 purao, s., henfridsson, o., rossi, m.; and sein, m. (2013). ensemble artifacts: from viewing to designing in action design research, systems, signs & actions, an international journal on information technology, action, communication and workpractices, 7 (1), 73–81. piramuthu, s. and shaw, m.j. (2009). learning-enhanced adaptive dss: a design science perspective, information technology management, 10, 41-54 power d (2010). what are concerns about end-user dss development? available: http://dssresources.com/faq/index.php?action=artikel&id=208 steinbauer, m.j., kriticos, d.j., lukacs, z., clarke, and a.r. (2004). modelling a forest lepidopteran: phonological plasticity determines voltinism which influences population dynamics, forest ecology and management, 198, 117–131 sherwell, b. w., gonzalez, a. j. and nguyen, j. (2005). contextual implementation of human problem -solving knowledge in a real-world decision support system. proceedings of the conference on behavior representation in modeling and simulation, los angeles, ca, may. http://cc.ist.psu.edu/brims2013/archives/2005/papers/05-brims-025.pdf australasian journal of information systems volume 18 number 2 2014 20 sein m, henfridsson o, purao s, rossi m, and lindgren r. (2011). action design research, mis quarterly, 35 (1), 37-56 tremblay, m. c., hevner, a. and berndt, d. j. (2010). focus groups for artifact refinement and evaluation in design research, communications of the association for information systems, 26, 599-618. venable, j.r. (2006). the role of theory and theorising in design science research, in chatterjee, s. and hevner, a. (eds), first international conference on design science research in information systems and technology, feb 24 2006. claremont, ca: claremont graduate university. winter, r. (2008). design science research in europe, guest editorial, european journal of information systems, 17, 470–475. ajis vol. 7, no 2 may 2000 the ethics/skills interface in image manipulation jenny webber school of communication and information studies university of south australia, magill, australia email: jenny.webber@unisa.edu.au abstract image manipulation using computer technology has become a basic skill required in various graphics-dependent industries . such as advertising, and the print and electronic media, and for specialist use in institutions for in-house and on-line publishing and the creation of web pages. the 'seamless' alteration of photographs and other visual images made possible by computer technology has allowed misrepresentation with intent to deceive, and difficulty in establishing copyright of original images. the dilemma in teaching techniques of image manipulation is to create a basis for ethical practice hci in this paper refers to the ethics/skills interface in the education and work of multimedia practitioners. key words ethics, image manipulation introduction in a recent television advertisement for an australian cancer research foundation, a message is delivered by the actor steve mcqueen, who died of lung cancer some twenty years ago. he commences his appeal by saying, 'i died in 1980. there wasn't a cure for cancer then and there isn't now, but.. ." and he goes on to ask for contributions to assist cancer research. this advertisement demonstrates the technical sophistication of current, computerised, stateof-the-art imaging, but it also brings sharply into focus the question of the ethics of image manipulation. with this particular advertisement, there is no such ethical problem, for, in a recent interview (kirwan, 1999) the representative of the sydney advertising agency which produced the video clip described how this was accomplished technically, and, more importantly, how it was accomplished ethically. the story is this. the advertising agency approached and readily gained approval from the executors of steve mcqueen's estate for the use of a clip from his film, the cincinnati kid, which showed the actor speaking directly to the camera. it was intended that this moving image would be manipulated to create the impression that mcqueen was 'speaking' now, in 1999, in support of cancer research. to achieve this, an actor with a mcqueen look-alike mouth was used, together with another actor with a mcqueen sound-alike voice, and these physical features were dubbed into the old film clip, using imaging techniques similar to those used in creating the 'talking' pig and sheep, and other animals in the australian film, babe. ethically, the whole project was beyond reproach, from the worthiness of its intent, the prior gaining of permission to use mcqueen's image, the continual involvement of the executors during the advertisement's production, and their approval of the final outcome. however, such ethical behaviour is often lacking in the computer manipulation of photographic images. we are reminded of this in a recent case in australia, following the tragic happenings at port arthur when photographs of the alleged gunman appeared on the front page of some australian newspapers with the suspect's eyes 're-touched' to create a more sinister facial expression. this unacknowledged 're-touching' was made even more obvious when this re-touched photograph was compared with the same original (but untouched) photograph of the suspect which appeared in other australian newspapers. these two examples are useful in demonstrating both an ethical approach and unethical approach to the manipulation of images, and they assist the building of ethical considerations into teaching and practice. the long history of deception by image manipulation the long history of photographic manipulation dates back to the early 19th century when photography, while demonstrating its technical capability of making representations of reality, was always a 'combination of artistic, technical, theatrical and entrepreneurial achievement' (slater 1995: 218-219). in these early days the concept of photography as truth was set alongside artforms such as photomontage, where doctoring photographs was used for caricature or emphasis (evans and gohl, 1986). photographic manipulation then most often stopped short at retouching as the popular middle class portraiture brought requests for unflattering facial characteristics such as wrinkles and pock marks to be removed (lester 1991; kobre 1995). later there were less frivolous manipulations evident in historical war and political photography with obvious intent to deceive, and such deceptions were not obvious until much later. some manipulations of american civil war photographs were discovered only when similar photographs were set alongside and the comparison revealed, for example, the rearrangement of 'corpses' in different photographs of the same scene. even the popular close-up portrait of abraham lincoln was a composite of the body of southern statesman, john calhoun, with lincoln's head 89 ajis vol. 7, no 2 may 2000 sandwiched atop. (in more recent times the same photographic technique has been used to set oprah winfrey's head on the body of actress ann-margret on a tv guide magazine cover.) in making people disappear, jaubert (1986) (cited in lester, 1991:101) chronicled numerous photographic abuses, mostly by totalitarian regimes, which used retouching, cutouts and effacement techniques, to doctor historical pictures in ways which would reflect a political leader's version of the truth. similarly in more recent times, charges of manipulation cloud famous pictures. lester (1991:117) recounts the 'unrelenting ethical controversy' surrounding rosenthal's pulitzer prize classic photograph involving the us marines' flag-raising at iwo jima in world war ii. there were at least four photographs taken at the time, three by rosenthal and another by a naval photographer, which suggests that the occasion of the famous photograph was staged, and it was not the eye-witness representation of the historical event that was first believed. what is significant about this historical record of image manipulation is not so much that it happened, but that it was so easily discovered. easy detection was due to the clumsy techniques which allowed the discovery of alterations. as well, these earlier techniques took painstaking work which was expensive and time consuming and this deterred the practice from becoming more common. on both counts, such is no longer the case, for the application of computer technology now allows photographs to be modified seamlessly, that is, with little or no evidence of alteration, and with an ease and speed which encourages the pracitce. modern computerized photographic techniques now allow, as well, quick synthesis of artificial images which are not based on reality. when a photograph is taken with a digital camera, or scanned and converted to digital information, the entire image can be modified in many ways, that is, its colour, brightness and focus can be changed, and elements of the image (such as a background, or persons) can be replicated or taken out altogether. this process can be accomplished in minutes and is almost impossible to detect. sometimes these techniques are revealed intentionally, such as a science 84 cover (cited in ritchin, 1991: 32) which used three major imaging techniques to create a 'realistic' image, that is, a simulated reality. in this case, the editor did not set out to deceive, in fact the cover carried a caption 'this picture is a fake'. it is interesting to note also that at the recent wedding of prince edward and sophie rhys jones, the royal photographer 'doctored' an official photograph of the wedding party by superimposing a smiling face of prince william on to the sombre face of the young prince as it appeared in the original photograph. the royal photographer was at pains to report this image manipulation and made a statement to the press that this had been undertaken in the interests of recording the happiness of the event. there have been cases, however, when the manipulation of photographs has not been so clearly revealed. kobre (1995: 14) details a recent case where a national enquirer cover photo under the banner headline 'battered nicole: photos taken by her sister show how o.j. beat her up', purported to show nicole simpson's bloodied and swollen face. less obvious was the smaller type under the photograph, 'sister describes photos seized by cops computer re-creation'. and not much later at the time of the simpson trial, a june, 1994, cover of time magazine featured a computerretouched los angeles police department 'mug shot' of the former football star, now alleged murderer. the image was referred to on the magazine's contents page as a 'photo illustration'. when challenged by observers that the photographic manipulation as misleading, racist, and potentially legally prejudicial, the editor defended his action by arguing that the photographic image had been 'subtly smoothed and shaped into an icon of tragedy' (wheeler and gleason 1995: 8), however, under further pressure, the editor did in fact apologise to his readers for this photographic manipulation. in an effort to counter this faked information, kobre (p. 15) makes a case for a stronger photographic access to all situations, and for a wider distribution and publication of photos, so that fakes may be countered with the truth. in the case of the enquirer's faked picture, if the media had had access to the pictures of the beaten nicole simpson which were tendered during the simpson trial (which were more horrendous than the faked photograph), then the enquirer's fictional image would not have been necessary or, at least, it would have been countered by the publication of the authentic images. unfortunately for kobre's suggestion, the control of what people see in visual news information, even in democracies, increasingly rests with a powerful few and there seems less and less chance that greater photographic access will combat the misuse of adeptly altered photographs. and, while editors must consult with news writers before making significant changes to a story, photographers' images may be manipulated without negotiation, and the photographer has little chance of recourse (reaves, 1991 and 1995). editing control raises questions of copyright and ownership of the original photograph. should photographer or photographic manipulator share the reward for the new image? how might a manipulated photograph be traced back to its source? while a book may be seen as a whole, and copying of all or part may be reasonably detectable, the' easy, and mostly undetectable, manipulation of photographs and other images (including clip art) renders the ownership of this artwork difficult if not impossible to protect (mitchell, 1994: 51). these various examples depict newspaper and magazine editors using computers for image manipulation at the expense of truthful representation. these acts bring into question problems of image authentication not only in respect of photojournalism, but also in the use of photographs as evidence in legal cases, and for any other documentary purpose, such as the compilation of social and cultural histories. 90 ajis vol. 7, no 2 may 2000 such dilemmas which arise from digital imaging technologies emphasize the need for an image ethics paradigm to assist the understanding of and response to moral problems caused by the use of these techniques. philosophical foundations of image ethics computer ethics was proposed as a new branch of philosophy in the mid 1970s by walter maner of the institute of applied ethics, old dominion university, virginia, usa, who highlighted the special features of computers which could 'exacerbate the moral problem; and in a few cases ... even create novel moral problems in their own right' (see fodor and bynum, 1992, cited in bynum, 1999). maner developed a university computer ethics course modelled on applied ethics courses such as medical and business ethics, in which philosophical theories like utilitarianism and kantianism were used to analyse ethical dilemmas which were 'aggravated, transformed or created by computer technology' (maner, 1978, cited in bynum, 1999). in a similar approach, but one directly related to ethical problems in the use of images, lester (1991: 29-42), has proposed a set of six general philosophical principles which could be applied to image production and distribution, and has related these to specific ethical dilemmas such as those detailed briefly below : (1) categorical imperative in which immanuel kant emphasized the nature of an act or a decision rather than the result of such an act or decision. sherer (1986: 28, 30, cited in lester, 1991: 35) quotes a newspaper editor's application of this kantian principle. noting that a newspaper's role is to cover the news, even if some situations may raise issues of taste, judgement and the right to privacy, the principle of reporting the news is a universal rule which must not be broken, regardless of the consequences. (2) utiliarianism in which various consequences are considered and the impact of the consequences of one action is weighed in relation to the consequences of another. the principle of utilitarianism might be used to justify the publishing of gruesome accident photographs, like the pictures of victims of a boston fire escape collapse, and whereafter, in some u. s. cities, fire safety codes 'were quietly reviewed ... to be certain boston's tragedy couldn't be repeated locally' (sherer, 1986: 28, cited in lester, 1991: 37) (3) hedonism, or the principle of 'act to maximize pleasure now and not worry about the future'. aristippus, a student of socrates, who framed the ethics of pleasure was referring to pleasures of the mind intellectual pleasures, which required the exercise of good judgement and self control. his famous phrase is 'i possess, i am not possessed', however modern usage has transformed this principle to an 'eat, drink and be merry, for tomorrow we die' attitude. the publication of graphically violent images appears to reflect this modern hedonist principle, that is, publication is motivated by either the images being sensational enough to sell newspapers, or aesthetic enough to gain peer approval and thus serve personal ambition (lester, 1991: 38). (4) golden mean. aristotle's golden mean philosophy refers to finding a middle ground, that is, a compromise between two extreme viewpoints, or taking an action which fits a particular situation at a particular time. for example, a photojournalist may personally believe that all funerals should be off limits to photographers, another might judge, that, as a news event, grieving victims at a funeral should be photographed. an ethical compromise might involve a prior notification of attendance and appropriate dress and unobtrusive behaviour of the photographer (sherer, 1986: 28, cited in lester, 1991: 38). (5) veil of ignorance. john rawls (1971) outlined the veil of ignorance philosophy, that all are equal and none advantaged over others, that is, 'when seen through a veil, without noticing age, race and sex, all are reduced to their basic position in life'. a photojournalist required to take photographs of persons under stress may ask 'how would i feel if i were the person being photographed? if the answer is unacceptable, i look for a different way to tell the story in my photo.' (sherer, 1986: 28, cited in lester, 1991: 40). (6) golden rule. the golden rule philosophy teaches persons to 'love your neighbours as yourself. from the judeo-christian tradition, a photojournalist should try to protect subjects from harm inflicted by photographic coverage. sherer, 1986: 25 (cited in lister, 199: 40) notes that photojournalist jay mather, who won a 1979 pulitzer prize wrote simply 'human kindness has always been an effective and impartial editor'. these six principles, as well as philosophies related to privacy (see warren & brandeis, 1980; benn, 1988; miller, 1997; van den hoven, 1999) and intellectual property (see hughes, 1988, cited in nissenbaum, 1999) may be used to analyse ethical dilemmas which arise in the study of image ethics. however, the principles may present an ethical dilemma in themselves, for while being instrumental in challenging practitioners to reflect on decisions regarding their use of images, the individual application of these principles to image production and distribution may also demonstrate a wide diversity of professional values and loyalties. it appears more useful to establish a code of practice for the use of images in this digital age, and one for which there is some likelihood of gaining consensus and adherence. 91 ajis vol. 7, no 2 may 2000 towards a protocol for image ethics wheeler and gleason (1995) suggest an ethical protocol in respect of the digital manipulation of images which, by a system of icons, would enable a reader or consumer to ascertain whether or not an image had been modified. however, whilst a system of labelling of photographs and other artwork is commendable, there is need for a more encompassing protocol for image ethics in view of the myriad of new communications technologies developed in the 1990s, and others which are anticipated in the next millenium. these technologies include satellites, fibre optics, faxes, the internet and virtual reality, each bringing social effects and ethical issues which arise before society can assimilate these rapid changes, cooper (1998: 71). a hidden effect of the new technologies is that deception has been made easier by seamless image manipulation, as is evident not only in the accidental and deliberate substitution of illusion for reality by editors of newspapers and magazines, but also in false and persuasive advertising (messaris 1997), and infringements of the moral rights of subjects in photographs (gross et al. 1988; lester 1991, 1995 & 1996 and powell, 1998). documents and photographs can now be altered from a distance without detection, and realistic, yet artificial, images can quickly be synthesized. as cooper (1998: 74) suggests 'theoretically, a presidential candidate may now be computer generated, credentialled online, and elected without ever being publicly seen (other than via a human stand-in)'. in similar vein, powell (1994) has predicted totally synthetic news events. when invasion of privacy occurs with old technolgies such as camera with telephoto lens capturing snapshots of personalities, such invasion has very different qualities than 'cyvacy', (computer privacy invasion) (powell, 1998: 93). this new invasion of privacy is remote, undetectable and impersonal. and 'cable, satellites, and internet all amplify existing problems, (such as) obscenity and indecency by making controversial and criminal (in many cultures) images available to much larger, different, and younger audiences'. a protocol for image ethics in the next millenium must encompass the issues outlined above and be proactive rather than reactive to the ethical problems created by the unique features of the new technologies, which are exacerbated by their rapid rate of invention and implementation, and their often unpredictable effects. the human computer interface: linking ethics and skills in a personal protocol for computer imaging in higher education the teaching of ethics is mostly left to philosophers or to specially designated courses. this is a missed opportunity, for research shows that integrating ethics in discipline-appropriate ways embellishes the study of ethics (roberts 1994 & 1995; lisman 1996). this view has led to the integration of image ethics in the multimedia program of the bachelor of arts award at the university of south australia. this image ethics program relies on the model of ethical decision-making outlined by rest and narvaez (1994: 2324) which involves a continuum of four distinctive stages of ethical development, that is, moral sensitivity, moral judgement, moral motivation, and moral character. it is a particularly useful model in the context of the university's 'graduate qualities', which are a set of personal and professional qualities which are anticipated outcomes for its graduates, and underpin its educational programs. one 'quality' involves the development of the graduate as an ethical citizen and professional. naturally in a multimedia program, course subjects have a main focus on visual arts in communication which involves the construction and manipulation of images. each course subject includes an integrated image ethics component which is appropriate to the particular level of study and its technical content. the overall aim of this ethics program is that students establish a personal protocol for image ethics which they will apply in their personal and professional lives. this may be briefly summarised as follows: • an image should be identified as 'original' or 'altered' and labelled accordingly • an image should not infringe copyright or another's intellectual property • an image should not be used to deceive or persuade • an image should not infringe the moral rights of its subject(s) in relation to privacy and stereotyping • an image should not breach ethical and legal standards in relation to obscenity and decency • an image professional should anticipate that new communication technologies may carry with them unpredictable and unwanted effects for society, be prepared to research and debate these effects, and work towards society's control of technology • an image professional has an individual responsibility for vigilance and action to defend the visual truth of information. conclusion too often ethical issues in computing are studied only after serious problems have arisen. with the benefit of hindsight, it can now be anticipated that each new technology will bring both benefit and disturbance to society, and that in order for society to gain control over technology and to be able to influence its effects, evaluations of 92 ajis vol. 7, no 2 may 2000 technologies must include not only the performance of their technical properties, but their effects on organizations, societies and individuals. such is the requirement when teaching imaging techniques to future users and industry practitioners who will work in a variety of contexts. there is a vital task here for all educators to prompt awareness of the scope of ethical issues which arise from the use of this technology, and the need for moral responsibility in its use. how do we get these budding professionals to look beyond the seductive power of computer technology and the aesthetic appeal of working with images? the ethics/skills interface is all important in this task, an interface which requires that students are made aware of the ethical dimensions of working with images at the same time as they learn the skills of image manipulation. students are now fully aware of the issues of plagiarism of text in their academic work, and universities have policies in place which require students to acknowledge the intellectual property rights of others, by referencing quotations and ideas from original work. the same respect for artwork and photographs has been ignored, and a similar system of referencing which applies to a scholarly essay should be applied to the production of graphic images in a student's computing projects. but the wider issue of an ethical approach to the presentation of visual information has great importance for a society which turns increasingly to computer technologies and a reliance on pictorial communication. the challenge is to 'integrate computing technology and human values in such a way that the technology advances and protects human values, rather than doing damage to them' (bynum, 1999). and, as ritchin (1991: 37) warns: 'as readers we must remain vigilant. otherwise, what as a society, and among societies, are we going to be left with as a form of communication that can be trusted? what information will people be able to rely upon to make decisions? or most precisely, what will the role of the press in a democracy be worth? ... undoubtedly, we will all be told that what has happened is the computer's fault, and we will then be even more isolated in our own media bubbles than ever before.' references benn, s. (1998). a theory of freedom, cambridge: cambridge university press. bynum, t.w. (1999) the development of computer ethics as a philosophical field of study. australian journal of professional and applied ethics, 1: 1-29. cooper, t. (1998). new technology effects inventory: forty leading ethical issues. journal of mass media ethics, 13:71-92. evans, d. & gohl, s. (1986). photomontage: a political weapon, london, bedford: gordon fraser gallery. fodor, j.l. & bynum, t.w. (1992) what is computer ethics? (a video program). educational media resources. gross, l., katz, j.s. & ruby, j. (eds.) (1988). image ethics: the moral rights of subjects in photographs, film and television. oxford: oxford university press. hughes, j. (1988) the philosophy of intellectual property. georgetown law journal, 77: 287-365. kirwan, p. (1999.) interview, today show, channel 9, july. kobre, k. (1995). the long tradition of doctoring photos. visual communication quarterly, 2: 14-15. lester, p. (1991). photojournalism: an ethical approach. hillsdale, nj: lawrence erlbaum associates. lester, p. (1995). visual communication: images with messages. belmont, ca: wadsworth. lester, p. (1996). images that injure: pictorial stereotypes in the media. westport, ct: praeger publishers. lisman, c. d. (1996). the curricular integration of ethics: theory and practice. westport, ct: praeger. maner, w. (1978) starter kit on teaching computer ethics (self-published in 1978, republished in 1980 by helvetia press in co-operation with the national information and resource centre for teaching philosophy). messaris, p. (1997). visual persuasion: the role of images in advertising. thousand oaks ca: sage. miller, s. (1997). privacy and the internet. australian computer journal, 29: 12-15. mitchell, w.j. (1994). the reconfigured eye: visual truth in the post-photographic era, cambridge, ma.: mit press. nissenbaum, h. (1999). the puzzle of priority: devising new norms and conventions in research for the context of electronic publication. australian journal of professional and applied ethics, 1: 44-68. powell, a. (1994). on ramps to the information superhighway. media studies journal, 3: 113-121. powell, a. (1998). satellite imagery: ethics of a new technology. journal of mass media ethics. 13: 93-98. rawls, j. (1971). a theory of justice. cambridge, ma: harvard university press. reaves, s. (1991). digital alteration of photographs in consumer magazines. journal of mass media ethics, 6: 175181. reaves, s. (1995). magazines vs. newspapers: editors have different ethical standards on the digital manipulation of photographs. visual communication quarterly, 2: 4-7. rest, j. & narvaez, d. (1994). moral development in the professions: psychology and applied ethics. hillsdale, nj: lawrence erlbaum associates. ritchin, f. (1991). photojournalism in the age of computers. in c. squiers, (ed.) the critical image: essays on contemporary photography, london: bay press. 93 ajis vol. 7, no 2 may 2000 roberts, p.m. (1994 april). the place and pedagogy of teaching ethics in the computing curriculum. australian educational computing. roberts, p.m. (1995). ethics: the neglected factor in computing education. in j.d. tinsley & t.j. van weert (eds.) liberating the learner, london: chapman & hall. sherer, m. (1986): bibliography of grief. news photographer. august: 23-38. slater, d. (1995). photography and modern vision: the spectacle of 'natural magic'. in c. jenks, (ed.) visual culture, london and new york: routledge. van den hoven, j. (1999). privacy and the varieties of informational wrongdoing.. australian journal of professional and applied ethics, 1: 30-43. warren, s. & brandeis, l. (1980). the right to privacy. harvard law review, 4. wheeler, t. and gleason, t. (1995). photography or photofiction: an ethical protocol for the digital age. visual communication quarterly. 2: 8-12. 94 the australasian journal of information systems (ajis) began in 1993 and was based on the simple realisation that there was a need for an academic ‘outlet’ for is research in australia australasian journal of information systems volume 17 number 1 2011 5 introducing the otam: exploring users’ perceptions of their ongoing interaction with adopted technologies shirlee-ann knight edith cowan university s.knight@ecu.edu.au janice m. burn edith cowan university j.burn@ecu.edu.au abstract a common criticism directed at davis’ (1986; 1989) technology acceptance model relates to its failure to adequately frame the “experienced” user’s ongoing adoption and exploitation of information technologies. given the pervasive nature of technology into individual users’ ongoing, everyday communication and information interactions, along with the “new adopter” becoming an increasingly rare entity, the tam is in danger of becoming a somewhat obsolete framework for investigating user-technology interaction. presented is critical analysis of the development and current state of the tam, followed by a proposed addition to the existing perceived usefulness (pu) and perceived ease of use (peou) tam constructs. the paper contends that the inclusion of a third broad construct, perception of interaction (poi) allows researchers to develop an investigative framework which facilitates an exploration of users’ ongoing and changing perceptions of the such things as the predictability of their technology interaction processes. the paper represents one the few inductive tam investigations. introduction since its first introduction to information systems (is) research, the technology acceptance model (tam) (davis, 1986; 1989; davis et al., 1989) has been explored in literally thousands [1] of research papers. based largely on ajzen & fishbein’s (1980) theory of reasoned action (tra) – a social psychology model concerned with the determinants of consciously intended behaviours – the tam postulates that two user perceptions, namely; perceived usefulness (pu) and perceived ease of use (peou); shape users’ intended behaviours towards their adoption of specific technologies. as a model for user information system adoption, the tam has now been tested and extended by a large number of researchers for over twenty years (adams et al., 1992; agarwal & jayesh, 1999; brown et al., 2002; burton-jones & hubona, 2006; chau, 1996; dishaw & strong, 1998; lederer et al., 1998; liaw & huang, 2003; mathieson, 1991; mcfarland & hamilton, 2006; moon & kim, 2001; shih, 2004; taylor & todd, 1995; teo, lim & lai, 1999; venkatesh & davis, 2000). in that time, the model has acquired various evolving states (lee, kozar & larsen, 2003) being protracted and validated –to varying degrees – with a variety of other constructs as a way of both; (1) reckoning whether pu and peou tell the entire story of user technology adoption; and (2) to determine if other constructs might act as antecedents to pu and peou. clearly, that the model has been tested so extensively adds to its rigour, yet it also serves to highlight that researchers’ continue to feel uncomfortable with the notion that pu and peou represent an adequate description of the motivating mailto:s.knight@ecu.edu.au mailto:j.burn@ecu.edu.au australasian journal of information systems volume 17 number 1 2011 6 factors for user technology adoption, particularly in relation to the continued, or on-going, adoption of technologies. it is in the context of this growing disquiet regarding the adequacy of the tam constructs (bagozzi, 2007; benbasat & barki, 2007; straub & burton-jones, 2007) that the authors set out to; 1. theoretically examine the current constructs of the tam, in order to determine whether they are able to examine users’ ongoing adoption of technology; 2. develop an inductive analysis of a tam driven survey, designed to investigate a technologically experienced user-group’s attitudes and perceptions of web-based search engine technology. the paper discusses the theoretical inclusion of a third major construct – perception of interaction (poi) – to the existing tam, designed to measure users’ various, and evolving, perceptions of the predictability of their ongoing interactions with their adopted technologies. the evolution of the poi construct’s inclusion into the tam is discussed in the context of the design and analysis of user-data associated with the research project “user perceptions of information quality in world wide web information retrieval behaviour” (knight, 2008). the tam: a historical summary (literature review) the beginnings of the tam the main supposition of davis’ (1986, 1989) model, was that pu & peou were the primary influencing variables in individual users’ attitudes and therefore intention to engage technologies. davis et al., (1989) further stated that peou indirectly influenced behavioural intent (bi) through its effect on pu. attitude – as a construct – was, by and large, bi-passed, assumed to be “positive” (i.e.; not variable-driven) in that users intention to engage technology at that time, was largely volitional. figure 1: technology acceptance model (tam) davis et al 1989 mathieson (1991), in comparing the tam to the theory of planned behaviour (tpb) (ajzen, 1991), contended that the tam, although statistically sound, and far more simple to use that the tpb, had a tendency to over-simplify important predictors of behavioural intention, particularly in relation to users who experience initial difficulties with their systems use. mathieson (1991) concluded that tpb could be used to fill-in-the-gaps of the tam if the two models were used together in investigating users intended technology adoption. adams et al. (1992) like mathieson, advocated the inclusion of additional constructs to tam’s pu and peou, namely perceived inabilities as a negative predictor of user attitude and bi. australasian journal of information systems volume 17 number 1 2011 7 extending the tam constructs for mandated technologies taylor & todd (1995) reintroduced the tra’s subjective norm (sn) construct to the tam’s pu and peou in the context of the increasingly mandated technology-use imposed on users by business and organisations through the 1990s. the ‘early adopters’ of davis’ original tam, who did not need the added social influence of colleagues and other professionals had now been replaced by a typical user who felt somewhat pressured to adopt technologies. taylor & todd added the construct of perceived behavioural control (pbc) as a means of quantifying users’ previous experiences with technologies, which by replacement, devalued the attitude construct in the tam paradigm. constructs for the world wide web by the time lederer et al. (1998), agarwal & jayesh (1999) and teo et al. (1999) published their tam variations, the world wide web had become an established information systems environment. once again, users were seen as having a greater degree of choice in the how, why and when of technology adoption, and the principles of tam were found to apply relatively well to the prediction of web-technology use. with the increasing variables between typical user characteristics however, what davis (1989) had termed “external variables” in the original tam began to be explored with greater vigour. individual differences (idv) in users (agarwal & jayesh, 1999), and task variables (lederer et al., 1998) were investigated in conjunction with tam to examine their relationship with the pu and peou constructs. teo et al. (1999) proposed a new construct, perceived enjoyment (pe) to investigate the ‘entertainment’ attributes of the world wide web. the various extensions of the tam, including dishaw & strong’s (1998) integrated tam/tff (task technology fit) model, led to venkatesh and davis’ publication of tam ii (2000). this was an attempt by the authors to address the issues raised by multiple researchers during the 1990s. seen by its authors’ as a definitive paper, venkatesh and davis (2000) contended that many of the additional constructs introduced to the tam, including constructs like perceived enjoyment could easily be classified within the pu or peou constructs. they did, however, reinforce the inclusion of subject norm (sn) elements to explain systems usage in mandated contexts. despite the tam ii paper, the meteoric growth of web usage since the late 1990s however, continued to facilitate the stream of papers perpetually adding constructs to the tam, including;  moon & kim’s (2001) perceived playfulness (pplay)  chau’s (2001) computer attitudes (ca) – introduced specifically to test the role of prior computer experience in technology adoption; and  liaw & huang’s (2003) individual differences (idv) and system quality (sq) – introduced to investigate the growing divergence in quality standards of computer technologies available on the web. generally speaking, these construct additions were seen as “extensions” to the tam model, rather than something which would fundamentally evolve the model (straub & burton-jones, 2007). considering user task differences shih (2004) pushed liaw & huang’s ideas further, applying the tam specifically to the growing internet activity of web search and information retrieval (ir). sq was replaced by such constructs as perceived performance (pp) of the software engaged by the user (in this case internet search engines) and relevance (r) of system results to user information search/seeking behaviours. this emphasis on pp and relevance judgments introduced post-adoption constructs to the tam. that is; perceived performance becomes a predicting construct for the choice of system/task relevance and australasian journal of information systems volume 17 number 1 2011 8 information strategies the next time the user wishes to perform information retrieval. although not demonstrated in shih’s model explicitly, the model at least began to imply an iterative, cyclical process to user adoption and use of technologies. this is because successful web searching is a strategy related task, that often requires the searcher to improvise based on the search results obtained (quinn, 2003). part of that improvisation is the natural flow of thoughts associated with a searcher’s own self-perceptions of the effectiveness of their strategies. this is a highly cognitive process, involving subjective variables such as; human memory (fugmann, 1973; cox, 1999; quinn, 2003; macpherson, 2004; xu, 2007); motivation (bilal & kirby, 2002; ford, 2004); attention (vakkari,1999; woodroof & burg, 2003; benoît, 2004; anderson, 2005); concentration (spink, 2004; kuhlthau, 2005); anxiety (yee, 2004); and self-efficacy (quinn, 2003; yee, 2004). as an efficacy related process, web search is also easily compromised by feelings of self-doubt or negative perceptions regarding the system or results of a search (quinn, 2003). web search, as a task then, provided an opportunity to explore users perception and impact of interactive components of technology use. unfortunately, this aspect of user/technology interaction was never explicitly studied in the seminal tam papers of this period. considering user individual differences the proliferation of home-users with access to the web, introduced the processes of electronic informational retrieval to a population of end-users who have had little to no formal training in the use of such technologies (wang, berry & yang, 2003). this is reflected in the, at times, seemingly ineffectual use of internet search engines, despite clear help systems accompanying most search engines (barnett, 1999). the constantly returning search engine audience, regardless of the success or non-success of user/search-engine engagement, suggests that user perceptions regarding what shih called perceived performance, is tempered by users’ own self-perceptions of their information retrieval effectiveness. in regards to the task of web-based ir, it would appear users are prone to attributing perceived performance as much to their own information behaviour, as to the perceived effectiveness of the search engine system (knight, 2008). the full effects of highly individual characteristics such as anxiety and self-efficacy on the pu and peou constructs of the tam are only now beginning to be discussed within the tam literature, particularly in relation to tam and the world wide web. these are the “external variables” alluded to by davis (1989) in the original versions of the tam. considering context mcfarland & hamilton, (2006) contend that any reasonable investigation using the tam should acknowledge the context specificity of the case in question, by modeling the external variables present. pu and peou were still seen as central to the tam, but computer-efficacy (ce) and systems usage (su) were included as constructs that are mediated by the various external variables previously introduced to the tam model. there was still no explicit indication of any cyclical or feedback mechanisms in the model, which the authors’ contend remained an inherent weakness of the tam model. burton-jones & hubona (2006) also advocate the mapping of individual differences within a user-group in order to establish their impact on the tam’s pu and peou constructs. the complexity of the ‘usage’ construct, i.e., technology usage, was described in terms of both is-usage (volume) and is-usage (frequency). appendix 1 is a table outlining various historical evolutions of the tam and its constructs. australasian journal of information systems volume 17 number 1 2011 9 the problem is… “the original tam has outlived its usefulness” (benbasat & barki, 2007) the extending and flexing of the tam paradigm required to include the various individual differences of users served to highlight some of the short-falls of the model, particularly in relation to the tam’s heavy weighting towards the new use and adoption of information technologies. it was found that the model could not fully address users’ learned or habitual behaviours. subsequent computer-use research has indicated a strong causative relationship between such user attributes as habit, with technology usage (verplanken, et al., 1998; gefen, 2003; limayem, hirt & cheung, 2003). this is problematic for the tam, since habit is a condition said to bi-pass behavioural intention (bi), a central construct of the original linear tam paradigm. the implication of this, is that pu and peou may fail to capture the true complexity of the user/system interaction taking place. without a strong body of literature investigating such phenomena as; (1) how users develop pu and peou; (2) how pu and peou impact behaviour beyond influencing bi; (3) how on-going bi or behavioural outcomes might impact (in a type of feedback loop) pu and peou; or (4) how pu and peou might change and evolve with on-going systems usage; two decades of tam investigations have remained somewhat two-dimensional and, by and large, simply served to reiterate initial findings of the model; i.e., the importance of pu (benbasat & barki, 2007). some is researchers, such as benbasat & barki (2007) have recently advocated abandoning tam related research, returning instead to the roots of the tam, the tra (ajzen & fishbein, 1980) and (later) theory of planned behaviour (tpb) (ajzen, 1991); in an effort to better understand user engagement of technologies. the problem with such an approach is that both the tra and tpb (along with the tam) fall short of providing a framework to investigate users on-going and selfregulatory behaviours involved in their decision making processes (bagozzi, 2007). moreover, the pre-supposed link between each of the models’ salient belief constructs and user bi, remains central, and unquestioned, within each model. the result is three models that are unable to account for salient constructs that may bi-pass user bi. it is the authors’ contention that human behaviour is far more complex than a linear, causal relationship between users’ salient beliefs, (which) influence behavioural intention, (which) equals behaviour. this linear-relationship approach to much is-based research into user engagement with technologies does not lend itself to understanding the complex relationships between; 1. user beliefs and behaviour intentions; 2. behaviour intentions and actual behaviour – particularly when intention does not lead to expectant behaviours; 3. behaviour outcomes and evolving user beliefs; 4. behaviours that bi-pass intentions; or 5. behaviours that occur outside of user beliefs and/or intentions information system adaptations of both the tra and tpb, like the tam, adhere to what the authors consider is a flawed paradigm, which fails to provide a framework for investigating; 1. non-intended (or unconscious) behaviours; 2. how previous behaviours might cause habitual behaviour; 3. how habitual behaviours might supersede salient beliefs; and 4. how on-going usage might impact on, or evolve, salient beliefs. australasian journal of information systems volume 17 number 1 2011 10 in this context, the authors propose a shift in the tam paradigm, which will provide a more useful research framework with which to investigate the relationships between user beliefs and their decision making processes in on-going technology usage. adjusting the paradigm: introducing the otam straub & burton-jones (2007) describe the current state of tam related research as a “logjam” and advocate the need to examine the constructs of the tam more closely in order to improve the conceptual understanding of the major constructs in the model. this view is echoed by benbasat & barki (2007) who assert the restrictive nature of the tam’s pu and peou constructs provide no mechanism for the inclusion of other user perceptions or salient beliefs. the result has been almost two decades of self-justified “extensions” of the tam whenever researchers propose the inclusion of additional or alternative user belief constructs. while some authors have proposed a complete shift away from the tam (benbasat & barki, 2007; bagozzi, 2007), it is the authors’ contention that pu and peou, if examined conceptually, are still able to provide a useful tool in examining user perceptions of the technologies with which they engage. to that end, the remainder of this paper discusses an inductive analysis of the tam results of eighty users, who regularly engage web-based search engines in the retrieval of high-quality information. the process of data analysis will be presented with respect to the emergence of the “otam” (on-going technology acceptance model) as representing a more useful framework by which to investigate users’ on-going adoption of web ir technologies. the research: methodology & research framework the research project the goal of the research was to investigate users’ perceptions of information quality (iq) in the context of their world wide web information search behaviours (isb). of particular interest, was the sub-set of user behaviours typically associated with user search-engine interaction. a total of four surveys were administered to a group of 100 academics from around the world over a period of approximately six months. eighty participants completed all four surveys in the following areas: 1. demographic – largely about academic role, discipline, and experience; 2. tam-based – perceptions of: (a) web search engines; and (b) web search engine information retrieval; 3. web-based information seeking behaviours (isb) and information retrieval (ir) strategies and outcomes; 4. perceptions and value judgements in relation to the information quality (iq) of the information participants encountered during ir from the www only the user data of the eighty participants who completed all surveys was used in the study. this paper covers specifically the user data from the tam surveys. methodology: inductive analysis of tam constructs a mixed-methodological approach (described in knight & cross, 2011) was developed in order to allow a largely inductive analysis of the quantitative data associated with the tam study. for the sake of consistency, multiple choice surveys were carefully constructed for all four surveys, with a australasian journal of information systems volume 17 number 1 2011 11 total of 126 questions asked of each participant. twenty four of those questions related directly to tam constructs, the results of which were inductively analysed in the context of a number of theoretically constructed “group-cases” developed largely from users’ non-tam question results, including constructs such as gender; information task; cognitive expectancy; user-role; and selfefficacy, to name a few. importantly, as part of an inductive approach to data analysis, these groupcases were not considered as mere “partitions” of the survey data. each group-case was, in fact, chosen and analysed carefully according to existent theory of specific phenomena, in an effort to develop a rich picture of the user results. the inter-disciplinary research model proposed by wilson (1997), and adopted as a framework for the current research, implores researchers to properly investigate the theory behind any driving constructs used to partition user data. accordingly, this behoves the researcher to examine partitioned data more thoroughly, recognising that each cluster of data is not only an imposed construction, but part of a potentially rich data-set that provides multiple contexts from which to investigate the complexities of human/computer interaction (hci). essentially, this places the research analysis into an inductive, even interpretative, epistemology. although still numbering in the very small minority, qualitative analysis is not completely foreign to tam studies, with researchers – particularly of late – taking this approach (gerrard, cunningham & devlin, 2006; lin, 2006) in an effort to address the large number of tam related studies that have merely served to replicate previous studies (lee et al., 2003; benbasat & barki, 2007) without producing anything significantly new (bagozzi, 2007). the problem with tam, as an investigative framework, is that it has become a victim of its own success. so narrow in its methodological variance as to become its own paradigm (straub & burtonjones, 2007) with few researchers venturing to investigate its constructs using different research approaches (sharma, yetton & crawford, 2004). unfortunately, the result is a model shallow in its ability to actually explore what pu and peou effectively mean (bagozzi, 2007). moreover, when the same methodology is applied to users’ complex individual behavioural differences within the context of a tam study, nothing of any depth can be discovered or added to the model. bagozzi (2007) contends that almost “no research has deepened tam in the sense of explaining pu and peou” (p244), going as far as to call the current extensions of the tam as “conceptually impoverished”. the authors agree unequivocally with bagozzi, that a significant reason for this famine in deeper understanding of the driving constructs of the tam could well be the dearth of research approaches designed to investigate the “why” of the interaction aspects of the tam. the current research therefore does not take the standard statistical analysis approach usually associated with tam, but seeks to develop theoretically sound sets of data which will allow the author to more closely examine variations in users’ results. the tam surveys the overall study associated with the current research sought to investigate user perceptions of information quality (iq), and information seeking behaviour (isb) in the ‘open’ information environment of the web. the tam component was used to examine users’ attitudes towards the web technologies they engaged in order to search for and retrieve the high quality information associated with their work and research as ‘academics’ and ‘post-graduate’ level students. two tam surveys were administered, each containing twelve questions, and investigating users’ general attitudes towards; 1. information retrieval using web-based search engines (12 questions); and 2. retrieving high quality information from the world wide web (12 questions). the driving constructs at this point of the research were pu and peou, with six questions in each survey concentrating on elements, or characteristics, associated with each construct. results were australasian journal of information systems volume 17 number 1 2011 12 then placed together, so that each of the six elements of the pu and peou constructs was addressed twice, the ultimate score for which was then averaged between the two results. results were combined because the investigation sought to examine users’ perceptions and attitudes regarding the information behaviour for which they were engaging the technology, rather than simply gaining an understanding of their attitude towards web search-engines in general. within the pu construct, questions relating to such elements as search engine results, effectiveness, productivity and speed were addressed. within the peou construct, questions relating to such elements as task ease (i.e.; find-ability and locate-ability), clarity and flexibility were addressed. questions were all asked in the “positive”, and a standard seven-point multiple choice (highly likely through highly unlikely) applied. table 1 presents the questions asked in the tam surveys and the constructs those questions were designed to measure. table 1. tam (pu & peou) questions for the current research the “group-case” constructs (units of analysis) classification of the constructed data-sets were both pre-defined (from previous literature and findings) at the survey design phase of the research; and evolutional, as some group-case characteristics became apparent after results were examined and collated. this is consistent with other examples of inductive approaches to research (mcclintock et al., 1979; eisenhardt, 1989). user results discussed in this paper fall into the following partitioned constructs: 1. user (prior) experience (palmquist & kim, 2000; hyldegaard & seiden, 2004); 2. user cognitive style (palmquist & kim, 2000; workman, 2004); australasian journal of information systems volume 17 number 1 2011 13 3. user motivation to use search engines (motivated, obliged or habitual) (venkatesh, 1999; liaw, 2002 & 2005; chung & tan, 2004); 4. user pre-supposed expectancy of search engine results (zhang & von dran, 2001); 5. user self-efficacy (liaw, 2002; quinn, 2003; yee et al., 2004); 6. user task/system confidence (liaw, 2002; pace, 2004); importantly, in the context of an inductive approach to data analysis, the use of these constructed group-cases is embedded in existent theory as it is not enough to simply present the statistical variance between the clustered sub-groups of users. each “group-case” presents a context in which to investigate complex cognitive and emotive human behaviour and its possible impact on pu and peou. this type of data analysis gives the researcher the opportunity to (1) examine phenomena from multiple perspectives (stake, 1995; dooley, 2002); (2) enable a more thorough analysis of the complex constructs chosen to partition data (adam, howcroft & richardson, 2004; eisenhardt, 1989); and (3) more closely scrutinise the possible co-relationships between the constructs and/or user results (yin, 1994; dooley, 2002). ultimately, inductive analysis of quantitative measurable data, an example of which is the current research, can provide the researcher with a desirable blend between a large data-set, bound in its collection to the validity scrutiny of quantitative research, and the opportunity to build theory through investigating phenomena from multiple perspectives. the investigation: a deeper analysis mapping the measurement: the development of the poi construct by taking a more inductive approach to data analysis, the authors – safe in the knowledge that data collection had closely followed an existing, valid model – were able to move out of existing tam analysis methods, and examine the data with fresh eyes. in the first instance, the authors went back to the drawing board, so to speak, and began to scrutinise what user-driven constructs had actually been surveyed. it became apparent that elements of the pu and peou constructs contained their own ambiguity regarding which construct was being tested. amongst the pu questions, for example, was the question “using the www would make it easier for me to do my research”. this question was asked in the context of ‘easy accessibility to search engines’ being perceived by a user as a useful characteristic. so, although question clearly asks a direct question about peou, the context of the question meant the construct being measured, was pu. this ambiguity of tam’s constructs is both a weakness and strength of the model. there are many user attitude constructs that can, and do, exist in their own right, which can be explained or classified as pu and/or peou. examples of this include the social normative element, which can be classified within the pu construct, or perceived enjoyment (pe) elements, which can be classified within the peou construct. in order to determine what perceptions were actually being measured, the researcher developed a mind-map, illustrated in figure 2, of each element. pu elements became labelled as measuring a user’s perception of the “effectiveness” of an interaction, peou was labelled as measuring a user’s perception of the “easiness” of an interaction, and a third construct relating to “processes” was classified, which measured a user’s perception of the understandability and repeatability; or “predictability” of an interaction. the elements identified in figure 2 became a guide for clustering user-results into three specific constructs. it should be stated here however, that each classification still possesses a degree of australasian journal of information systems volume 17 number 1 2011 14 ambiguity in that some elements being tested could fall into more than one construct. for example; “easy (to apply)” could be included as either an “easiness” or a “processes” element, because it can refer to either the easiness or repeatability of an action. figure 2. a mind-map of the elements being measured by the tam constructs table 2 presents the tam questions in the context of the mind-map’s constructs and the elements determined to be being measured by each question. effectiveness and easiness were once again classified as pu and peou respectively, and process related elements became classified as perception of interaction (poi). the need for an ongoing/interactive construct by and large, the tam is used in deductive research to quantifiably measure and predict users’ intention to use an information system or technology, based on their attitudes towards the system (raghunathan, 1999; lin & lu, 2000; king & he, 2006; kwon choi & kim, 2006). this presents two problems for is research. firstly, its framework limits a researcher’s investigation of complex human attitude constructs such as pu and peou to a statistical analysis of their relative impact on user behaviours, without providing a framework for investigating the hither-to unquestioned relationship between behavioural intent and actual behaviour (bagozzi, 2007). moreover, it fails to provide researchers’ with a scaffold to more fully investigate what pu and peou actually are (benbasat & barki, 2007). secondly, tam was designed to test users’ early adoption of various information technologies and systems (kim & malhotra, 2005; wang, lin & luarn, 2006), which highlights a generally accepted limitation of tam (karahanna & straub, 1999; bhattacherjee, 2001; limayem et al., 2003) that it now requires a change in focus if being used to investigate on-going, or continued technology/systems usage. the user group associated with the current research, does not fit into the early adoption category, with no single user possessing less than three years experience interacting with web search engines. given the increasingly pervasive role technology plays in daily information transactions, this is most likely going to become the case for user groups associated with much contemporary research. australasian journal of information systems volume 17 number 1 2011 15 notwithstanding these issues, tam still provides a framework to begin investigating the useful concepts associated with the model, particularly if a researcher is courageous enough to develop a deeper analysis of the constructs involved. table 2: tam constructs being measured by the current research a “rich” investigation of the constructs of tam the unconventional approach taken by the authors involved seeing the pu and peou constructs as logical presuppositions of users’ adoption of technologies. of course an individual will utilise a tool if they perceive it to be useful, and of course they will engage that useful tool frequently if they find their engagement requires easily manageable amounts of cognitive effort. this assumption (supported by years of quantitative tam research) allowed the focus of the research to shift from the statistical analysis of the level of pu and peou influence on user behaviour, to one with an investigative focus of “how does this work?” the current research, therefore, represents an attempt on the part of the authors to offer to the pool of tam literature an alternative investigation of its constructs. the result is the addition of a third construct, perception of interaction (poi), designed to; australasian journal of information systems volume 17 number 1 2011 16 1. provide a construct which actively seeks a feedback mechanism into the tam, in the form of a construct which facilitates research into how (initial) resultant behaviour of pu an peou impacts on future use; 2. provide a construct which allows researchers to investigate how users’ individual differences might impact pu an peou (as well as poi); 3. provide a construct which allows an investigation of user behaviour that may bi-pass the behavioural intention (bi) construct of the current tam. 4. provide a framework which facilitates multiple avenues of exploration in relation to data analysis. introducing poi: perception of interaction the otam proposes that a third construct, perception of interaction (poi), be used in addition to pu and peou, to assess users’ ongoing, and changeable, perception of the predictability of their system interaction. this adds to tam a whole realm of user/technology and user/information engagement not facilitated by the original, or subsequent, tam frameworks. individual constructs have been used to extend tam in the twenty years since davis (1986, 1989) first proposed the model, including; user experience (adams et al., 1992; taylor & todd, 1995; liaw, 2002; gefen, 2003); subjective norm (taylor & todd, 1995); motivation (chung & tan, 2004; yi et al., 2006); self-efficacy (chau, 2001; mcfarland & hamilton, 2006); perceived playfulness (moon & kim, 2001; chung & tan, 2004); computer attitude (chau, 2001); computer enjoyment (liaw, 2002; teo et al., 1999; liaw & huang, 2003; van der heijden, 2004); computer efficacy (chau, 2001; mcfarland & hamilton, 2006); computer anxiety (hackbarth, grover & yi, 2003; schottenbauer et al., 2004); system quality (liaw, 2002); perceived performance (shih 2004); system usage (mcfarland & hamilton, 2006); it continuance usage (bhattacherjee, 2001; limayem et al., 2003); habitual behaviour (gefen, karahanna & straub, 2003; limayem et al., 2003) and trust (gefen et al., 2003). as well, individual differences, such as; age (liu et al., 2000; arning & ziefle, 2007); gender (gefen & straub, 1997; agarwal & prasad, 1999; venkatesh & morris, 2000; venkatesh et al., 2004); level of education (chuang & chuang, 2002; burton-jones & hubona, 2005); cognitive style (rapp, taylor & crane, 2003); and cognitive absorption (agarwal & karahanna 2000; saadé & bahli, 2005) have been used to look for variables in user results to pu & peou. none of these constructs or individual differences have, however, proved to be the illusive coconstruct for pu and peou, that would enable tam to become an effective high-level research framework for investigating both new and ongoing adoption of information technologies. if anything, they proved to over-complicate the model, bringing the whole body of tam literature to a somewhat confused, chaotic state (benbasat & barki, 2007). in effect, the plethora of extensions to tam have lacked the high-level generic qualities of the original pu and peou constructs, thereby making each extension too specific to examine multiple user individual differences or co-dependent sub-constructs. they have also failed to address, what benbasat and barki (2007) describe as tam’s inability as a theory “to provide a systematic means of expanding and adapting its core model ... in the constantly evolving it adoption context” (p.212). of greater concern however, is that tam, either with or without its variables and extensions, fails to really provide a great deal of value beyond confirming that users engage technologies they perceive will be “useful” (benbasat & zmud, 1999). the reality is, pu and peou as constructs themselves, require an additional construct which will allow researchers to flesh out the relationship between users’ individual differences, core salient beliefs and the central paradigms of tam. the poi construct of the otam, shares a number of common characteristics with pu and peou, not the least of which is that it is a high-level, somewhat ambiguous, construct which itself requires australasian journal of information systems volume 17 number 1 2011 17 researchers develop ways to measure it. in other words, like pu and peou, how a user perceives the predictability of their systems interaction is implied by answers to questions like: “the steps necessary to complete my assigned task would be clear enough for me to repeat” or “i would be able to find a similar website tomorrow” what poi adds to pu and peou is that it explicitly allows for a post-adoption technology investigation of user/computer interaction. it recognises that the very act of “interaction” has the capacity to impact and influence pu and peou as well as future interaction perceptions. it broadens the depth of tam to facilitate a proper examination of the complex relationships between users’ individual differences and their pu and peou of an information system. complex cognitive issues relating to phenomena such as users’ cognitive dissonance with a system, attribution of a successful (or non-successful) interaction or outcome, and general expectancy of a system interaction can be addressed in concrete, measurable terms to determine how pu and peou might influence bi. figure 3 presents a schematic illustration of the otam and it’s pu, peou and poi constructs. included are some of the relational pathways investigated as part of the current research. the “individual differences” illustrate twelve of the fourteen group-case constructs developed for the research project of which this paper was a part, six of these group-cases will now be discussed. figure 3. the otam constructs for on-going measurement of technology acceptance constructs for data analysis given the research’s assumption that users engage technologies at a cognitive level, user attitudes and perceptions, and their impact on pu, peou and poi; results were examined in relation to multiple socio-cognitive theories, including; 1. attribution theory (weiner, 1974; 1980; 1986) 2. expectancy theory (vroom, 1964); 3. cognitive dissonance theory (festinger, 1957 & 1959); 4. self-efficacy theory (bandura, 1977 & 1982). importantly, the concept of “interaction” became an increasingly significant driving concept in the investigation. previous research within multiple scientific disciplines, indicates that interaction is an important variable in users’ behavioural adoption strategies, including; acceptance (jain, ross & australasian journal of information systems volume 17 number 1 2011 18 prabhakar, 2004); learning (saadé, 2007); satisfaction (mckinney, yoon & zahedi, 2002; lindgaard & dudek, 2003); persistence of engagement (hassenplug & harnish, 1998; liaw, 2002); attitude (bhattacherjee & premkumar, 2004; porter & donthu, 2006); perceptions of quality (sage, 1991; barnes & vidgen, 2006); trust (gefen et al., 2003; varlander, 2007); and decision making (marshall, 2004). significantly, the poi context began to prove a rich context for investigating these sociocognitive theories. the results: some research findings with the goal of the research to investigate users’ perceptions of information quality (iq) in the context of their world wide web information search behaviours (isb), it was determined that the target user-group should be relatively intellectually sophisticated, possessing: (1) a relatively high degree of iq perception; (2) a work-related information need which demanded a high level of quality in information typically retrieved; and (3) the ability to make (and recount) relevant quality related value-judgments of the information they encounter on the web. an assumption was made that such a user-group could be found amongst career academics (researchers and lecturers) and postgraduate (u.s. graduate) level students, engaging masters, dba and/or phd studies. the stated criteria for inclusion in the user-group were; 1. users who were “academics”, or “postgraduate level” students – including users who fell into both these categories; 2. users who used the world wide web to retrieve information that related to their work/research as academics and/or post-graduate students. the user-group did not necessarily have to feel ‘comfortable’ retrieving work/research related information from the web, but needed to do so relatively regularly and be personally familiar with the process of using the web as an information retrieval tool for the high quality content associated with their work, research, or both. academics who engaged the web as a means of professional networking, or even entertainment were not excluded from the target user-group, however the surveys and questionnaires they completed did not relate specifically to these interactions. the user-group’s general characteristics: the user-group of eighty academics came from around the world (35% north america; 21.3% europe/africa; and 43.7% australasia), and exhibited a number of key characteristics, including; 1. technologically sophisticated; a.) highly ‘experienced’ with both web search engines and general web technologies; b.) early adopters of web technologies; with user data collection taking place between march 2006 and march 2007, over three quarters of participants had been using web technologies prior to 1998, and over a fifth using them prior to 1995. australasian journal of information systems volume 17 number 1 2011 19 2. informatically sophisticated; it should be noted, the study in no way advocates that academia holds a monopoly on intelligent individuals or high-end information users. it simply assumes that, in order to participate in post-graduate academic activities, the vast majority of users would posses above average cognitive capabilities and demand a high level of quality in their target ir. table 5 users highest completed university qualification 3. high self-efficacy & task/system confidence; an outstanding characteristic of this user-group was their high levels of self-efficacy and task/system confidence in their ability to search and retrieve their target information. when asked how often they expected to successfully find the information they were looking for, a staggering 88% of participants answered most or every time. in addition, this high-level of success was rarely attributed to the quality of the search engine used, but was seen by participants as a result of their own “good search strategies” and ability to engage the informatic environment of the web. table 6: user perceptions of their “successful” web searches australasian journal of information systems volume 17 number 1 2011 20 discussion of some of the findings the following discussion presents some of the findings of the research using the modified otam framework, with results inductively examined in the context of; (1) the entire user-group’s results; and (2) six of the fourteen clustered data-sets (called “group cases”). for the sake of clarity and conciseness, the results discussed will focus particularly on what the proposed poi construct revealed about user attitudes and perceptions of their search engine engagement. figure 4 presents the overall user-group results to the questions presented in table 2. results have been categorised into the pu, peou and poi constructs, with each construct being used to measure four elements. scores presented in the graph are out of 15, derived from using a simple numerical weighting formula (described in appendix 2). the highest score an element can attain is 15, gained if 100% of the user group select “highly likely” for that element. the average score for each of the three perception constructs is also presented. figure 4. user results (whole group) for pu, peou & poi (otam) general user results it was noted that user poi elements, which relate to the predictability or repeatability of search-engine interactive outcomes, consistently produced the lowest results. significantly, pu: expected s.e. results, which could also be seen as measuring something of the predictability of the outcome of a user’s search engine interaction, consistently scored the lowest of the pu elements. peou results were, by and large, the most consistent, with minimal variation between users results, regardless of which group-case of user results was being observed. results for six of the group-cases associated with the current paper are presented in figure 5. in keeping with the inductive analysis, results were analysed – and are discussed – in the context of previous theory. (1) user experience – fig 5a: users level of experience is presented often as a major influencing factor in user behaviour in previous literature/theory (hyldegaard & seiden, 2004; toms, dufour & hesemeier, 2004; fusilier & durlabhji, 2005; castañeda, muñoz-leiva & luque, 2007) australasian journal of information systems volume 17 number 1 2011 21 user experience: some observations and discussion overall, user-experience ranked second highest in divergence within its sub-group results, with pu and poi showing the greatest variety of responses for this group-case. this suggests that; user experience has a significant influence on users’ attitude towards the system. this is consistent with previous research findings (taylor & todd, 1995; agarwal & prasad, 1999; fusilier & durlabhji, 2005; lee, qu & kim, 2007) which contends previous exposure to a system, or relative system, is likely to have a positive influence on the constructs measured in the tam, particularly in relation to pu (chang et al., 2005), albeit at times, indirectly through increased user self-efficacy (mcfarland & hamilton, 2006). users’ level of experience has a direct effect on their perception of their potential productivity. the more experience, the greater the belief, on the part of the user, that their search engine engagement will be productive. of interest then, is the apparent contradiction between these perceptions and experienced users’ perceptions regarding the consistency and predictability of these interactions. apart from the “pu:results” scores (derived from questions relating to an anticipated improvement in research performance and quality), the elements of the poi construct represent both the lowest scores and greatest divergence in sub-group results. this is consistent with the whole user-group results for poi, which suggests that; the greatest dissonance issue facing web search engine users remains search engine unpredictability. moreover; this issue does not seem to dissipate over time, even as the user gains more experience using the system. (2) user cognitive style (query construction) – fig 5b: cognitive style has been defined as a user’s preferred (sometimes habitual) approach to both organising and representing information (frias-martinez, chen & liu, 2007). search query construction styles, such as keyword and boolean search strategies, have therefore been used previously to investigate user cognitive style (moss & hale, 1999; ford, miller & moss, 2005). in the current research, cognitive styles were classified as (a) phrase searcher {27.5% of the user-group}; (b) keywords searcher {60%}; and (c) boolean searcher {12.5%}. cognitive style: some observations and discussion the concept of users’ cognitive style has been extensively researched in various fields associated with information seeking and retrieval behaviours (moss & hale, 1999; kim, 1999; palmquist & kim, 2000; ford, miller & moss, 2002; chen et al., 2004), and has been linked with other user characteristics, such as gender (ford & miller, 1996; ong, lai & wang, 2006) and perceived systems performance (workman, 2004). the intrinsic nature of the cognitive style user characteristic (joughin, 1992), and the previously established link between system performance and perceived usefulness (liu & ma, 2006) lends itself to the finding that; cognitive style plays a significant role in search engine adoption through its impact on users’ perceptions of search engine usefulness. phrase searchers frequently returned more positive perceptions than keywords and boolean searchers, with boolean searchers usually scoring the lowest. significantly, even though boolean searchers consistently scored other perception elements lowest, their poi: clarity of interaction perception scored higher than the phrase and keywords sub-groups. this is consistent with a boolean searcher’s strategy of gaining greater personal control through manipulating search query interaction with the use of boolean tactics, supporting chen, magoulas & macredie, (2004) and (graff, 2003), who found australasian journal of information systems volume 17 number 1 2011 22 that, in terms of ‘user control’, users’ cognitive styles significantly influences user/system information interaction figure 5. user sub-group (“group case”) results for otam constructs of particular interest is cognitive style’s impact on users’ pu, which ranks second highest (behind task/system confidence) in variation between its sub-groups. in light of previous tam research findings that pu is the strongest precursor to technology adoption and use (davis et al., 1989; chau, 2001), the impact of cognitive style on user adoption attitudes towards search engines should not be understated. cognitive style, as a construct, presents a significant problem for tam researchers in that the tra (fishbein, 1967), and consequently tam, separate the “affective” and “cognitive” domains as distinct constructs of human behaviour, where it is the affective domain alone that is said to influence “attitude” (agarwal & prasad, 1999). at a conceptual level, attitude is said to underlie the tra (fishbein & ajzen, 1975) and subsequent tam, which postulate that cognition is linked not to attitude, but to belief, through that which is learned. that which is learned, according to pajares australasian journal of information systems volume 17 number 1 2011 23 (1997), occurs when a behaviour is acted or observed. if this is true, then the significant relationship between cognitive style and pu is suggestive that; perceived usefulness is notably influenced by users’ learned behaviours that have (in the past) produced a valued result. this is significant because it suggests an interactive-driven construct, such as the poi, is required as part of the tam framework to fully investigate the complex relationship between users’ specific cognitive characteristics and their pu of a system. (3) motivation to use search engines – fig 5c: classified in the current research project as the reason for which users engage search engines, a historical review of tam literature reveals that a major contributing factor to variations in user results can be traced to whether users make a conscious choice regarding their use of a particular technology (rawstorne, jayasuriya & caputi, 2000; brown et al., 2002). users were identified as: (a) motivated; (b) habitual; or (c) obliged; search engine users. motivation: some observations and discussion as expected, motivated users – those users who engage search engines because they find them to be a highly effective information retrieval tool, returned the highest results for all the tam constructs within this group-case. this result validates the constructed group-case itself, which overtly measures users general attitude towards search engines. teo et al. (1999), in fact, describe extrinsic motivation and “perceived usefulness” as the same entity. habitual users returned higher results for the pu and peou constructs than obliged users, which was also to be expected. firstly, given the previous tam research which demonstrates users who feel obliged to use a system require positive subjective norm antecedents (taylor & todd, 1995; venkatesh & davis, 2000) in order for their pu and peou results to reflect a positive attitude towards that system. and secondly, given the passive role this sub-group of users perceive they play in their choice to interact with the system. the research by verplanken et al., (1998) into the role of habit versus planned (or intended) behaviour purports that once a behaviour is habitually strong, users rely on their habit to a greater degree than conscious choice strategies. furthermore, from experiments conducted, the authors concluded that even when the process of choice was externally manipulated, it did not over-ride the effect of habit. gefen’s (2003) research supports this conclusion, finding that once “specific it behaviour become routine habit, [it] . . should become a primary predictor of use” (2003, p3). the results for the motivation group-case support the proposition that; habit has a stronger influence on users attitude toward search engine use than obligation does interestingly, the results for the motivation group-case also provides evidence that; the poi construct is measuring a different type of attitudinal interaction with the system than pu and peou, since the results for poi do not simply replicate the pu and peou results. of note, is that the poi sub-constructs tested are the only elements to return a more positive result for obliged users than habitual (also, unmotivated) users. like boolean searcher results, this could be indicative that; obliged users play a more cognitively active role in their search engine interaction than habitual users, producing slightly better perceptions of the predictability of their system interaction. australasian journal of information systems volume 17 number 1 2011 24 (4) expectations of search query results – fig 5d: much of the tam research advocates that a user’s expectation of a technology has a profound impact on how they feel about, and interact with, that technology (petersen, madsen & kjær, 2002; staples, wong & seddon, 2002; lindgaard & dudek, 2003; tesch et al., 2005). the expectancy sub-groups were classified as; (1) expectant users – who believed query results would be relevant; (2) active users – who acknowledged query results may or may not be relevant; and (3) passive users – who assumed query results may be relevant – without considering some results might be irrelevant. expectancy: some observations and discussion users who had extremely high expectations of the relevance of search engine results to their queries, demonstrated some of the most positive results of the entire study for all three tam constructs. this is consistent with desanctis (1983), who linked high levels of systems expectations with highly positive use of that system. in much of the tam research, user expectancy is often measured as “perceived performance” (bhattacherjee, 2001; d’ambra & wilson, 2004; dadayan & ferro, 2005), and ranks consistently as one of the strongest predictors of user bi (wang, wu & wang, 2008). of interest in the context of this group-case was the results for the “active users” sub-group, who were willing to entertain the possibility that some search engine query results may include results that are not relevant to their queries. this sub-group recorded higher scores than the more passive sub-group, who stated they believed results may be relevant, without demonstrating a conscious cognition that results may also not be relevant. the implication of this is that; a more cognitively “active” sub-group, while aware of the system’s flaws, or perhaps because of the system’s flaws, is more likely to recognise and embrace their own cognitive role in the process of web-based information retrieval, leading to a more positive attitude towards the system. (5) user self-efficacy – fig 5e: self-efficacy refers to the perception a user has of their own role in an outcome to a specific process or behaviour (compeau, higgins & huff, 1999), and is considered to be an important motivational construct (gist & mitchell, 1992) in the adoption of specific user attitudes and behaviours. in the current research, users were classified as having high self-efficacy if they attributed a successful web search to their own search strategies. self-efficacy: some observations and discussion except for the poi construct, the variance between the high and low self-efficacy sub-groups was relatively minimal, varying less than one point for most results. this is consistent with a number of previous studies which have generally found that users’ degree of self-efficacy has relatively little, to slightly positive, an impact on their pu or peou of the system with which they engage (igbaria & iiravi, 1995; compeau & higgins, 1995; lewis, agarwal & sambamurthy, 2003). significantly, the sub-group with lower self-efficacy returned better results for poi than the high selfefficacy group. poi is designed to measure users’ ongoing perceptions of the predictability of their interactions with search engines. this could indicate that; the higher self-efficacy sub-group have a greater recognition of the system’s flaws, and the inconsistencies of their interactions with it, resulting in a negative impact on their perceptions of the system this is consistent with previous findings regarding the negative impact of high self-efficacy and user perceptions of a system (ong et al., 2004; ceaparu et al., 2004). importantly, other researchers have also noted a higher self-efficacy’s negative impact on various elements of tam’s constructs (chau, 2001), although this seems to occur while having a positive impact on intended use. australasian journal of information systems volume 17 number 1 2011 25 the poi construct then, adds a valuable tool to the original tam framework when investigating how high self-efficacy can return lower tam results. (6) user task/system confidence – fig 5f: as a construct, confidence is recognised as being closely aligned with users’ self-efficacy (igbaria & iivari, 1995; agarwal & karahanna, 2000) however a number of authors have recognised the distinction between a user’s self-concept of their role and ability to perform a task, and their confidence that the system/technology is able to help them successfully complete that task. users’ task/system confidence was recorded by asking the user-group “how often do you expect to successfully find relevant information when utilising a web search engine?” conceptually, the question sought to bring together both a user’s pre-task (ex ante) expectation and post-task (ex post) evaluation. in this regard, it integrates some of the post-adoptive theory associated with the task technology fit (ttf) model (goodhue, 1995; goodhue & thompson, 1995; d’ambra & wilson, 2004), and implies a user’s level of “satisfaction” with the system (khalifa, 2004; tesch et al., 2005). task/system confidence: some observations and discussion the task/system confidence group-case returned the largest divergence in sub-group results across all three tam constructs. predictably, users with the lowest (classified “average”) task/system confidence returned the lowest tam results and users with “very high” task/system confidence returned the highest results. the user results confirm previous research findings regarding theory related to users’ perceived performance and confidence constructs – the higher a user’s confidence that they will be able to successfully complete the task for which they have engaged a system, the better their general attitudes towards that system will be (compeau & higgins, 1995; hong et al., 2001) cross-analysing user tam results (for task/system confidence) with their results in the non-tam surveys, the sub-group of users with very high task/system confidence demonstrated;  higher motivation (66.7%) to use search engines (16.7% group norm)  significantly higher degree of faith (50%) that search engine returns would be relevant to their query (36.2% group norm)  equal weighting of attribution to self and system (50% each) when a search was successful.  high propensity for minimal strategic change when a search query proved to be unsuccessful, with 83.3% (36.2% group norm) stating they would simply change their keywords rather than selecting a different search-query strategy these results are of particular interest because they strongly suggest that; users with very high task/system confidence, and therefore extremely high tam results exhibit a reduced tendency to personally “own” the cognitive load of their search engine interaction, with 0% using the more difficult boolean search tactics (12.5 group norm); 50% possessing high self-efficacy (15% group norm); and only 16.7% considering the possibility that search-engine query results might not be relevant (9.6% group norm). this finding is supported by evidence that; users with relatively low task/system confidence demonstrated an increased tendency to personally own the cognitive load of their search-engine interaction; with 22.2% using the more difficult boolean search tactics (22% high t/s confidence); 66.7% high self-efficacy (16.7% high t/s confidence); only 11.1% believing search engine query results would be completely relevant (38.9% high t/s confidence), 22.2% considering search engine query australasian journal of information systems volume 17 number 1 2011 26 returns may be irrelevant (5.5% high t/s confidence); and a staggering 88.9% using all the summary information associated with search engine query results to make decisions about a result’s relevancy to their information task (22.2 high t/s confidence). this result makes a significant contribution to tam literature, and demonstrate how the inclusion of the poi construct can facilitate a much richer understanding of users’ cognitive processes during systems adoption. the preliminary cross analysis of the actual information seeking behaviours (survey #3) with the very high task/system confidence sub-group implies that while their attitude is towards the technology is extremely positive, actual user/information interaction may be compromised to the extent that the attitude induces a more cognitively lazy approach to information retrieval. if this is true, then although intent to use the system is increased, effective use may not necessarily be the ultimate outcome. this conclusion is supported by the high task/system confidence sub-group’s result to the poi productivity element, which scored relatively low. contribution & areas for future research contribution the contribution of the otam to an already over-loaded tam-driven body of research is twofold. firstly, the current investigation is an example of an inductive, qualitative investigation of the tam constructs, rather than a repeat of the common quantitative approach designed to, once again, simply support davis’ (1986, 1989) hypothesis regarding the importance of perceived usefulness (benbasat & barki, 2007). secondly, for the first time, a third dynamic construct has been added to the original tam, which does not simply represent one or two user-driven “individual differences” or external variables of the user-group being applied to the investigation. perception of interaction (poi), or more pragmatically, perceived predictability of interaction, brings to the tam, a construct allowing the model to be applied to users’ ongoing acceptance and adoption of technologies. pushing research methodology there is a need in the field of is literature to develop deeper analysis of the complex relationships between human behaviour and system/technology adoption. as a relatively young discipline, theory developed within is literature is yet to experience full scrutiny from other scientific disciplines. this is however, changing, and the body of tam literature, in particular, is being examined by researchers from multiple disciplines, [2] including nursing studies (ahasan, partanen & keyoung, 2001; despont-gros et al., 2004; ammenwerth, iller & mahler, 2006; breen & zhang, 2008); medicine (aguillo, 2000); library science (kuhlthau & tama, 2001; jiao & onwuegbuzie, 2003); business studies (bruner & kumar, 2005; seyal & rahman, 2007) science & engineering (roco, 2005); marketing (gerrard et al., 2006; jelinek et al., 2006); and education (ip, jones & jacobs, 2007). it behoves the is discipline to broaden its investigative methods and more robustly define its conceptual terminologies, particular those which have been adopted and adapted from other disciplines, such as the social cognitive theory (sct) driven concepts of the tam. at a time when the is discipline has been going through what some authors’ have described as an ‘identity crisis’ (benbasat & zmud, 2003) there has never been a greater need for is researchers to consider their methodological options. benbasat & zmud contend the ever increasingly “eclectic” nature of both the practice and study of is calls for multiple paradigms and approaches when researching is. this is of particular concern in relation to the lack of diversity in research approaches within the specific is research area of technology adoption (choudrie & dwivedi, 2005). australasian journal of information systems volume 17 number 1 2011 27 significant findings and future research expectancy & motivation a cross analysis between the user motivation and expectancy sub-groups reveals some interesting correlations. motivated users are two times more likely than habitual users, and three times more likely than obliged users, to perceive a search engine will return relevant results to their queries. this is a clear confirmation of vroom’s (1964) hypothesis that expectancy is intrinsically linked to motivation. the higher the (positive) expectancy of a behaviour, the greater is the motivation to perform that behaviour (rappaport, 2004). interestingly, this same sub-group of motivated users are also some 2.5 times more likely than habitual users, and two times more likely than obliged users, to not actively consider the likelihood that a search engine may return irrelevant results to their queries, making them the most “cognitively passive” group. expectancy, as a cognitive construct, is by and large, perceived as a positive antecedent in user adoption of technologies. the current research, however, has identified that when it comes to information search and retrieval, lower expectancy produces a more “cognitively active” searcher, who is more likely to recognise and embrace their own cognitive role in the process of web-based information retrieval and search engine interaction. this cognitively active searcher warrants further investigation, to determine the role of the “self” in otam related attitudes towards web search engines. cognitive dissonance a significant finding of the current research is that; the poi construct, and specifically the ‘poi: interaction clear’ sub-construct, provide a framework by which users’ levels of cognitive dissonance can be investigated using the otam. cognitive dissonance theory (festinger, 1957) postulates that people adjust their internal attitudes (whitworth et al., 2007) to overcome stimuli that does not necessarily fit-in with what they feel should be true, and therefore limit any conflict between their expectations and experiences (brown, venkatesh & goyal, 2007). figure 6: levels of experience (group-case) results karahanna & straub (1999) contend that users’ natural need to reduce this conflict, or cognitive dissonance, is achievable if they can rationalise their usage behaviour. this paper now further suggests that this process of being over-positive is one of the elements captured by tam’s pu construct. the current research illustrates this process, both in the whole group’s results and, particularly, the user experience sub-group results. figure 6 illustrates the sub-group of users with australasian journal of information systems volume 17 number 1 2011 28 12+ years experience using search engines (see also figure 5a). this sub-group of users had the highest pu scores of the “user experience” group-case, while also recording the lowest score for the poi’s clarity of interaction. karahanna & straub’s (1999) research indicated a strong relationship between higher cognitive dissonance and more positive pu results, causing the authors to contend that users attempt to stabilise any conflicting feelings associated with system interaction, by focusing on those things about the system that are useful. the current research supports karahanna & straub’s findings by demonstrating that; even long-term users experienced a degree of cognitive dissonance concerning their search engine interaction. significantly, that this dissonance firstly does not dissipate even for users possessing 12+ years experience, and secondly does not prevent users from repeated search engine use suggests that: a result of greater experience with a system is a higher tolerance for any cognitive dissonance associated with that system. self-efficacy self-efficacy driven investigations in the context of the tam have tended to return mixed results. authors such as venkatesh (1999) and bhattacherjee (2001) align the tam (in regards to selfefficacy elements) with social cognitive theory’s two expectancy suppositions, suggesting that the pu construct is similar to sct’s “outcome expectations” and peou encompasses similar elements as “self-efficacy”. the constructs of expectancy and self-efficacy are, at times, used inter-changeably (bandura, 1977; gist & mitchell, 1992). this is particularly true in the field of is, which has found itself adopting (and adapting) various theories originally associated with the social and psychological sciences, as a bases for investigation the human component of user technology adoption and human computer interactive processes (chau,1996). in the current research however, a cross-analysis between the expectancy and self-efficacy sub-groups’ results confirms stajkovic & sommer’s (2000) finding that they are separate constructs. the former being primarily about expected outcome, the latter more about locus of control in the process of achieving an outcome. both constructs are intrinsically linked to motivation, but not necessarily always to each other. of note, is that the divergence in results within the self-efficacy sub-groups (figure 5e) for the pu and peou constructs were negligible, however, the sub-group with lower self-efficacy returned better results for the poi construct than the high self-efficacy sub-group. chau (2001) noted that selfefficacy had a negative impact on the tams constructs, but explained this impact in terms of high self-efficacy still having a positive influence on users’ intended use of a system. the results for the poi related elements sheds more light on chau’s finding, and is suggestive that; the lower results for the higher self-efficacy group is indicative of this sub-group’s awareness and recognition of the system flaws, and the inconsistencies of their interactions with the system. this awareness of the system’s flaws however is countered by their own high level of self-efficacy, or internalised locus of control, which invariably attributes the outcome of an interactive task – in this case information search and retrieval – to their own capabilities or strategies, rather than that of the system. in this regard, users can return poorer results for the poi construct, while still having an overall positive view of the system. it is this level of sensitivity offered by the poi construct which suggests it is of more value as a co-construct – as opposed to a sub-construct – of the pu and peou constructs. australasian journal of information systems volume 17 number 1 2011 29 conclusion this paper has argued for the introduction of an interactive-driven construct to be included into the original tam in order to better investigate users’ ongoing adoption of technologies and related systems. it presents the otam (ongoing technology acceptance model), which employs the original tam constructs of pu and peou, and the proposed poi, as a framework useful for investigating the complex cognitive aspects of human engagement of information technologies. these include; expectancy; motivation; self-efficacy; and cognitive dissonance. the supposition of the current research is that neither pu or peou adequately address the complexities associated with user cognitive behaviours employed during human computer interaction. the poi construct has been added as a method of analysing those user results indicative of their actual, and ongoing, interactions with a technology system. the paper has also confirmed the validity of the poi construct through the discussion of some of the research project’s results, which have not only confirmed previous findings relating to the tam, but have added additional understanding to those findings. finally, the paper has presented a combined methodological analysis of the statistical/quantitative data associated with tam research. this proved particularly useful when cross-analysing the tam statistical results with results to other surveys conducted during the course of the whole research project. this type of cross-analysis was presented in the task/system confidence observations and discussion section. whether poi goes on to become a fully accepted construct of a future version of the tam remains to be seen, as the model still requires further exploration and testing beyond the scope of this research paper. the researchers acknowledge the possibility that poi could in fact prove to become part of the existing tam model, not as a construct, but as an over-arching framework within the model, allowing for a more useful investigation of pu and peou. whichever is the case, an interaction construct is required within the tam in order to address its “inability as a theory to provide a systematic means of expanding and adapting its core model” (benbasat & barki, 2007, p.212). in addition, more examples of inductive and heuristic explorations of the tam constructs, such as the current manuscript, will add a richness to the body of tam literature. notes [1] google scholar citations search figures: based on citation search of davis’ two seminal papers (davis, 1989; davis et al., 1989); may 2005 = 1155 (771 & 444); may 2007 = 3398 (2116 & 1282); jan 2008 = 5068 (3154 & 1914); jan 2011 = 13,554 (8599 & 4955). [2] studies cited here are not just is researchers’ publications in alternative disciplines, but also represent researchers from multiple disciplines using the tam to investigate technology adoption issues within their disciplines. acknowledgements this research was made possible by the arc discovery grant “building an internet search engine for quality information retrieval” (dp0452862), a partnership between the arc, edith cowan university and the university of wollongong (2005-2008) australasian journal of information systems volume 17 number 1 2011 30 references adam, a., howcroft, d., & richardson, h. (2004). decade of neglect: reflecting on gender and is new technology, work and employment, 19(3), 222-240. adams, d. a., nelson, r. r. & todd, p. a. (1992) perceived usefulness, ease of use, and usage of information technology: a replication". misq, 16, 227 247. agarwal, r. & prasad, j. (1999) are individual differences germane to the acceptance of new information technologies? decision sciences, 30, 361-382. agarwal, r., & karahanna, e. (2000). time flies when you're having fun: cognitive absorption and beliefs about information technology usage. mis quarterly, 24(4). aguillo, i. (2000). a new generation of tools for search, recovery and quality evaluation of world wide web medical resources. on-line information review, 24(2), p138 143. ahasan, r., partanen, t., & keyoung, l. (2001). global corporate policy for financing health services in the third world: the structural adjustment crisis. international quarterly of community health education, 20(1), 3-16 ajzen, i & fishbein, m. (1980) understanding attitudes and predicting social behaviour. prenticehall, englewood cliffs, nj, 1980 ajzen, i. (1991). the theory of planned behavior. organizational behavior and human decision processes, 50, 179-211. ammenwerth, e., iller, c., & mahler, c. (2006). it-adoption and the interaction of task, technology and individuals: a fit framework and a case study. bmc medical informatics & decision making, 6(3). anderson, t. d. (2005) relevance as process: judgements in the context of scholarly research. information research, 10, 226 arning, k., & ziefle, m. (2007). understanding age differences in pda acceptance and performance. computers in human behaviour, 23(6), 2904-2927. bagozzi, r. p. (2007) the legacy of the technology acceptance model and a proposal for a paradigm shift. journal of the association for information systems, 8, 244-254. bandura, a. (1977). self-efficacy: toward a unifying theory of behavioural change. psychological review, 84, 191-215. bandura, a. (1982). self-efficacy mechanism in human agency. american psychologist, 37(2), 122147. barnett, a. (1999) a survey of internet searches and their results. reference & user services quarterly, 39, 177-182. barnes, s. j. & vidgen, r. t. (2006) data triangulation and web quality metrics: a case study in egovernment. information & management 43 (2006) 767–777. benbasat, i. & zmud, r. w. (1999). empirical research in information systems: the practice of relevance, mis quarterly, (23)1, 3-16. benbasat, i., & zmud, r. w. (2003). the identity crisis within the is discipline: defining and communicating the discipline’s core properties. mis quarterly, 29(2), 183-194. benbasat, i. & barki, h. (2007) quo vadis, tam? journal of the association of information systems, 8, 211-218. benoît, g. (2004) properties-based retrieval and user decision states: user control and behavior modeling. journal of the american society for information science and technology, 55, p.488-497. australasian journal of information systems volume 17 number 1 2011 31 bhattacherjee, a. (2001). understanding information systems continuance: an expectationconfirmation model. mis quarterly, 25(3), 351-370. bhattacherjee, a., & premkumar, g. (2004). understanding changes in belief and attitude toward information technology usage: a theoretical model and longitudinal test. mis quarterly, 28(2), 229-254. breen, g.-m., & zhang, n. j. (2008). introducing e-health to nursing homes: theoretical analysis of improving resident care journal of medical systems, 32(2), 187-192. bilal, d. & kirby, j. (2002) differences and similarities in information seeking: children and adults as web users. information processing & management, 38, p.649-670. brown, s. a., massey, a. p., montoya-weiss, m. m. & burkman, j. r. (2002) do i really have to? user acceptance of mandated technology. european journal of information systems, 11, 283295. brown, s. a., venkatesh, v., & goyal, s. (2007). expectation confirmation in technology adoption: an examination of six competing theoretical models. under review, available from: http://misrc.umn.edu/workshops/2007/spring/susan.pdf. bruner, g. c., & kumar, a. (2005). explaining consumer acceptance of handheld internet devices. journal of business research, 58(5), 553-558 burton-jones, a., & hubona, g. s. (2005). individual differences and usage behaviour: revisiting a technology acceptance model assumption acm sigmis database, 36(2), 58 77. burton-jones, a. & hubona, g. s. (2006) the mediation of external variables in the technology acceptance model. information & management, 43, 706-717. castañeda, j. a., muñoz-leiva, f., & luque, t. (2007). web acceptance model (wam): moderating effects of user experience. information & management, 44(4), 384-396. ceaparu, i., lazar, j., bessiere, k., robinson, j., & shneiderman, b. (2004). determining causes and severity of end-user frustration. international journal of human-computer interaction, 17(3), 333 356. chang, i. c., li, y.-c., hung, w.-f., & hwang, h.-g. (2005). an empirical study on the impact of quality antecedents on tax payers' acceptance of internet tax-filing systems. government information quarterly, 22(3), 389-410. chau, p. y. k. (1996) an empirical assessment of a modified technology acceptance model. journal of management information systems, 13, 185. chau, p. y. k. (2001) influence of computer attitude and self-efficacy on it usage behavior. journal of end user computing, 13, 26. chen, s. y., magoulas, g. d., & macredie, r. d. (2004). cognitive styles and users’ responses to structured information representation. international journal on digital libraries, 4(2), 93-107. choudrie, j., & dwivedi, y. k. (2005). investigating the research approaches for examining technology adoption issues. journal of research practice, 1(1), article d1. chuang, y. h., & chuang, y. w. (2002). attitudes of two-year rnbsn nursing students towards computers. the journal of health science, 5(1), 71–84. chung, j., & tan, f. b. (2004). antecedents of perceived playfulness: an exploratory study on user acceptance of general information-searching websites. information & management, 41(7), 869-881. compeau, d. r., & higgins, c. a. (1995). computer selfefficacy: development of a measure and initial test. mis quarterly, 19(2), p.189-211. australasian journal of information systems volume 17 number 1 2011 32 compeau, d., higgins, c. a., & huff, s. (1999). social cognitive theory and individual reactions to computing technology: a longitudinal study. mis quarterly, 23(2), 145. cox, r. (1999) representation construction, externalised cognition and individual differences. learning and instruction, 9, 343-363. dadayan, l., & ferro, e. (2005). when technology meets the mind: a comparative study of the technology acceptance model. in m.a. wimmer et al. (eds.): electronic government 2005, , lncs 3591(springer-verlag berlin heidelberg 2005), 137-144. d’ambra, j., & wilson, c. s. (2004). use of the world wide web for international travel: integrating the construct of uncertainty in information seeking and the task-technology fit (ttf) model. journal of the american society for information science & technology, 55(8), 731–742. davis, f.d. (1986) a technology acceptance model for empirically testing new end-user information systems: theory and results, mit sloan school of management, cambridge, ma, 1986. (doctoral dissertation) davis, f. d. (1989) perceived usefulness, perceived ease of use, and user acceptance of information technology. mis quarterly, 13, 319. davis, f. d., bagozzi, r. p. & warshaw, p. r. (1989) user acceptance of computer technology: a comparison of two theoretical models. management science, 35, 982. davis, f. d., bagozzi, r. p., & warshaw, p. r. (1992). extrinsic and intrinsic motivation to use computers in the workplace. journal of applied social psychology, 22(14), 1111-1132. desanctis, g. (1983). expectancy theory as an explanation of voluntary use of a decision support system. psychological reports, 52, 247–260. despont-gros, c., fabry, p., muller, h., geissbuhler, a., & lovis, c. (2004). user acceptance of clinical information systems: a methodological approach to identify the key dimensions allowing a reliable evaluation framework. medinfo 2004, 1038-1042. dishaw, m. t. & strong, d. m. (1998) extending the technology acceptance model with tasktechnology fit constructs. information & management, 36, p.9-21. dooley, l. m. (2002). case study research and theory building. advances in developing human resources, 4(3), 335-354. eisenhardt, e. m. (1989). building theories from case study research. academy of management review, 14(4), 532-550. festinger, l. a. (1957). theory of cognitive dissonance. london, uk, travistock publications. festinger, l. & carlsmith, j.m. (1959). cognitive consquences of forced compliance. journal of abnormal and social psychology, 58, 203-210. (available at http://psychclassics.yorku.ca/festinger/] fishbein, m. (1967). readings in attitude theory and measurement. ny, wiley. fishbein, m., & ajzen, i. (1975). belief, attitude, intention and behavior: an introduction to theory and research. addison-wesley, reading, ma. ford, n., & miller, d. (1996). gender differences in internet perceptions and use. aslib proceedings, 48, 183-192. ford, n., miller, d., & moss, n. (2002). web search strategies and retrieval effectiveness: an empirical study. journal of documentation, 58(1), 30-48. ford, n. (2004) modeling cognitive processes in information seeking: from popper to pask. journal of the american society for information science and technology, 55, p.769-782. http://psychclassics.yorku.ca/festinger/ australasian journal of information systems volume 17 number 1 2011 33 ford, n., miller, d., & moss, n. (2005). web search strategies and human individual differences: cognitive and demographic factors, internet attitudes, and approaches. journal of the american society for information science and technology, 56(7), 741-756. frias-martinez, e., chen, s. y., & liu, x. (2007). automatic cognitive style identification of digital library users for personalization. journal of the american society for information science and technology, 58(2), 237-251. fugmann, r. (1973) on the role of subjectivity in establishing, using, operating and evaluating information retrieval systems. information storage and retrieval, 9, 353-372. fusilier, m., & durlabhji, s. (2005). an exploration of student internet use in india: the technology acceptance model and the theory of planned behaviour. campus-wide information systems, 22(4), 233-246. gefen, d., & straub, d. w. (1997). gender differences in the perception and use of e-mail: an extension to the technology acceptance model. mis quarterly, 21(4), p.389-400. gefen, d. (2003) tam or just plain habit: a look at experienced online shoppers. journal of end user computing, 15, 1. gefen, d., karahanna, e. & straub, d. w. (2003). trust and tam in online shopping: an integrated model. mis quarterly, 27(1), 51-90 gerrard, p., cunningham, j. b., & devlin, j. f. (2006). why consumers are not using internet banking. a qualitative study. journal of services marketing, 20(3), 160–168. gist, m. e., & mitchell, t. r. (1992). self-efficacy: a theoretical analysis of its determinants and malleability. the academy of management review, 17(2), 183-211. goodhue, d.l. (1995) understanding user evaluations of information systems, management science 41(12), 1827–1844. goodhue, d.l. & thompson, r.l. (1995) task– technology fit and individual performance, mis quarterly 19(2), 1995, pp.213-236. graff, m. (2003). learning from web-based instructional systems and cognitive style. british journal of educational technology, 34(4), 407-418. hackbarth, g., grover, v., & yi, m. y. (2003). computer playfulness and anxiety: positive and negative mediators of the system experience effect on perceived ease of use. information & management, 40(3), 221–232. hassenplug, c.a. & harnish, d. (1998). the nature and importance of interaction in distance education credit classes at technical institutes. community college journal of research and practice, 22(6), 591-605. hong, w., thong, j. y. l., wong, w.-m., & tam, k.-y. (2001). determinants of user acceptance of digital libraries: an empirical examination of individual differences and systems characteristics. journal of management information systems, 18(3), 97. hyldegaard, j., & seiden, p. (2004). my e-journal exploring the usefulness of personalized access to scholarly articles and services. information research, 9(3). igbaria, m., & iivari, j. (1995). the effects of self-efficacy on computer usage. omega: international journal of management science, 23(6), 587-605. ip, b., jones, s., & jacobs, g. (2007). retention and application of information technology skills among nursing and midwifery students. innovations in education and teaching international, 44(2), 199-210. jain, a. k., ross, a., & prabhakar, s. (2004), an introduction to biometric recognition, ieee transactions on circuits and systems for video technology, 14(1), 4-20. australasian journal of information systems volume 17 number 1 2011 34 jelinek, r., ahearne, m., mathieu, j., & schillewaert, n. (2006). a longitudinal examination of individual, organizational, and contextual factors on sales technology adoption and job performance. the journal of marketing theory & practice, 14(1), 7 23. jiao, q. g., & onwuegbuzie, a. j. (2003). reading ability as a predictor of library anxiety. library review, 52(4), 159 169. joughin, g. (1992). cognitive style and adult learning principles. international journal of lifelong education, 11(1), 3-14. karahanna, e., & straub, d. w. (1999). the psychological origins of perceived usefulness and easeof-use. information & management, 35(4), 237-250. khalifa, m. (2004). the state of research on information system satisfaction. journal of information technology theory and application. (online: http://www.findarticles.com/p/articles/mi_qa4008/is_200401/ai_n9384759/print ). kim, k.-s. (1999). searching the web: effects of problem-solving style on information-seeking behaviour. proc of ed-media 99: world conference on educational multimedia, hypermedia & telecommunications, 1541-1542. kim, s. s., & malhotra, n. k. (2005). a longitudinal model of continued is use: an integrative view of four mechanisms underlying post-adoption phenomena. management science, 51(5), 741 755. king, w. r., & he, j. (2006). a meta-analysis of the technology acceptance model. information & management, 43(6), 740-755. knight, s. a. (2004) tam & the internet: an investigation into the changing face of the tam and its application to internet search engines. proc. 5th international we-b conference, nov 2004. knight, s. a. (2008). user perceptions of information quality in world wide web information retrieval behaviour. phd dissertation, school of management, edith cowan university. knight, s.a. & cross, d. (2011) using contextual constructs theory and model. international journal of doctoral studies (upcoming) kuhlthau, c. c., & tama, s. l. (2001). information search process of lawyers: a call for ‘just for me’ information services. journal of documentation, 57(1). kuhlthau, c. c. (2005) towards collaboration between information seeking and information retrieval. information research, 10, p225. kwon, o., choi, k., & kim, m. (2006). user acceptance of context-aware services: self-efficacy, user innovativeness and perceived sensitivity on contextual pressure. behaviour & information technology, 25(1), 1 16. lederer, a. l., maupin, d. j., sena, m. p. & zhuang, y. (1998) the role of ease of use, usefulness and attitude in the prediction of world wide web usage. proc. of the 1998 acm sigcpr conference on computer personnel research, boston, massachusetts(united states), 195 204. lee, y., kozar, k. a. & larsen, k. r. t. (2003) the technology acceptance model: past, present, and future. communications of the ais, 12, 752-780. lee, h. y., qu, h., & kim, y. s. (2007). a study of the impact of personal innovativeness on online travel shopping behavior—a case study of korean travelers. tourism management, 28(3), 886-897. lewis, w., agarwal, r., & sambamurthy, v. (2003). sources of influence on beliefs about information technology use: an empirical study of knowledge workers. mis quarterly, 27(4), p.657-678. australasian journal of information systems volume 17 number 1 2011 35 liaw, s.-s. (2002). understanding user perceptions of world wide web environments. journal of computer assisted learning,, 18(2), p.137-148. liaw, s.-s. & huang, h.-m. (2003) an investigation of user attitudes toward search engines as an information retrieval tool. computers in human behaviour, 19, 751-765. liaw, s.-s. (2005). developing a web assisted knowledge construction system based on the approach of constructivist knowledge analysis of tasks. computers in human behavior, 21(1), 29-44. limayem, m., hirt, s. g. & cheung, c. m. k. (2003) habit in the context of is continuance: theory extension and scale development. 11th european conference on information systems. italy. lin, c. c. j., & lu, h. (2000). towards an understanding of the behavioural intention to use a web site. international journal of information management, 20(3), 197-208. lin, a. (2006). the acceptance and use of a business-to-business information system. international journal of information management, 26(5), 386-400. lindgaard, g., & dudek, c. (2003). what is this evasive beast we call user satisfaction? interacting with computers, 15(3), 429-452. liu, j. e., pothiban, l., lu, z., & khamphonsiri, t. (2000). computer knowledge, attitudes, and skills of nurses in people’s hospital of beijing medical university. computers in nursing, 18(4), 197–206. liu, l., & ma, q. (2006). perceived system performance: a test of an extended technology acceptance model. acm sigmis database, 37(2-3), 51 59. macpherson, k. (2004) an information processing model of undergraduate electronic database information retrieval. journal of the american society for information science and technology, 55, p.333-347. marshall, c. c., & bly, s. (2004). sharing encountered information: digital libraries get a social life. joint conference on digital libraries, 2004 (jcdl'04), june 2004, 218-227. mathieson, k. (1991) predicting user intentions: comparing the technology acceptance model with the theory of planned behavior. information systems research, 2, p173-191. mcclintock, c. c., brannon, d., & maynard-moody, s. (1979). applying the logic of sample surveys to qualitative case studies: the case cluster method. administrative science quarterly, 24(4), 612-629. mcfarland, d. j. & hamilton, d. (2006) adding contextual specificity to the technology acceptance model. computers in human behaviour, 22, 427-447. mckinney, v., yoon, k., & zahedi, f. m. (2002). the measurement of web-customer satisfaction: an expectation and disconfirmation approach. information systems research, 13(3), 296-315. moon, j.-w. & kim, y.-g. (2001) extending the tam for a world-wide-web context. information & management, 38, 217-230. moss, n., & hale, g. (1999). cognitive style and its effect on internet searching: a quantitative investigation. european conference on educational research (ecer '99), 1189. ong, c.-s., lai, j.-y., & wang, y.-s. (2004). factors affecting engineers’ acceptance of asynchronous e-learning systems in high-tech companies. information & management, 41(6), 795-804. pace, s. (2004). the roles of challenge and skill in the flow experiences of web users. proc. of informing science and information technology education joint conference 2004(insite '04), pp.341-358. pajares, f. (1997). current directions in self-efficacy research. in m. l. maehr & p. r. pintrich (eds.) australasian journal of information systems volume 17 number 1 2011 36 palmquist, r. a., & kim, k.-s. (2000). cognitive style and on-line database search experience as predictors of web search performance. journal of the american society for information science and technology, 51(6), p.558-566. petersen, m. g., madsen, k. h., & kjær, a. (2002). the usability of everyday technology— emerging and fading opportunities. acm transactions on computer-human interaction, 9(2), pp.74–105. porter, c. e., & donthu, n. (2006). using the technology acceptance model to explain how attitudes determine internet usage: the role of perceived access barriers and demographics. journal of business research, 59(9), 999-1007. quinn, b. (2003) overcoming psychological obstacles to optimal online search performance. the electronic library, 21, 142 153. raghunathan, s. (1999). impact of information quality and decision-maker quality on decision quality: a theoretical model and simulation analysis. decision support systems, 26(4), 275286. rapp, d. n., taylor, h. a., & crane, g. r. (2003). the impact of digital libraries on cognitive processes: psychological issues of hypermedia. computers in human behaviour, 19(5), 609– 628. rappaport, j. (2004). an analysis of expectancy theory, the theorist, and the application/limitations of. info 640 theory paper, from: http://www.jrappaport.waynefire3.com/literature/info640/rappaport_expectancy_theory_final_version.pdf. rawstorne, p., jayasuriya, r., & caputi, p. (2000). issues in predicting and explaining usage behaviours with the technology acceptance model and the theory of planned behavior when usage is mandatory. proc. of the 21st international conference on information systems, brisbane (australia), p.35 44. roco, m. c. (2005). the emergence and policy implications of converging new technologies integrated from the nanoscale journal of nanoparticle research, 7(2-3), 129-143. saadé, r., & bahli, b. (2005). the impact of cognitive absorption on perceived usefulness and perceived ease of use in on-line learning: an extension of the technology acceptance model. information and management, 42(2), 317–327. saadé, r. (2007). dimensions of perceived usefulness: toward enhanced assessment. decision sciences journal of innovative education, 5(2), 289-310. sage, a.p. (1991). decision support systems engineering. new york: john wiley and sons. schottenbauer, m. a., rodriguez, b. f., glass, c. r., & arnkoff, d. b. (2004). computers, anxiety, and gender: an analysis of reactions to the y2k computer problem. computers in human behaviour, 20(1), 67–83.. seyal, a. h., & rahman, n. a. (2007). the influence of external variables on the executives’ use of the internet. business process management journal, 13(2), 263 278. sharma, r., yetton, p., & crawford, j. (2004). the relationship between perceived usefulness and use: the effect of common method bias. proc of the diffusion interest group in information technology (digit) conference, washington dc, 2004. shih, h.-p. (2004) extended technology acceptance model of internet utilization behavior. information & management, 41, 719-729. spink, a. (2004) multitasking information behavior and information task switching: an exploratory study. journal of documentation, 60, 336-351. stajkovic, a. d., & sommer, s. m. (2000). self-efficacy and causal attributions: direct and reciprocal links. journal of applied social psychology, 30(4), 707-737. http://www.jrappaport.waynefire3.com/literature/info640/rappaport_expectancy_theory_final_version.pdf australasian journal of information systems volume 17 number 1 2011 37 stake, r. (1995). the art of case study research. thousand oaks, ca: sage. staples, d. s., wong, i., & seddon, p. b. (2002). having expectations of information systems benefits that match received benefits: does it really matter? information & management, 40(2), 115131. straub, d. & burton-jones, a. (2007) veni, vidi, vici: breaking the tam logjam. journal of the association for information systems, 8, 223-229. taylor, s. & todd, p. (1995) assessing it usage: the role of prior experience. mis quarterly, 19, 561-570. teo, t. s. h., lim, v. k. g. & lai, r. y. c. (1999) intrinsic and extrinsic motivation in internet usage. omega: international journal of management science, 27, 25-37. tesch, d., miller, r., jiang, j. j., & klein, g. (2005). perception and expectation gaps of information systems provider skills: the impact on user satisfaction. information systems journal, 15(4), 343-355. toms, e. g., dufour, c., & hesemeier, s. (2004). measuring the user's experience with digital libraries. joint conference on digital libraries, 2004 (jcdl'04), 51-52. vakkari, p. (1999) task complexity, problem structure and information actions: integrating studies on information seeking and retrieval. information processing & management, 35, 819-837. van der heijden, h. (2004). user acceptance of hedonic information systems, mis quarterly, 28(4), 694-704. varlander, s. (2007). online information quality in experiential consumption: an exploratory study. journal of retailing and consumer services, 14(5), 328-338. weiner, b. (1974) achievement motivation and attribution theory. morristown, n.j.: general learning press. weiner, b. (1980). human motivation. ny: holt, rinehart & winston. weiner, b. (1986). an attributional theory of motivation and emotion. new york: springer-verla. whitworth, b., bañuls, v., sylla, c., & mahinda, e. (2007). expanding the criteria for evaluating socio-technical software. proc of ieee smc '07, 29 aug 2007. venkatesh, v. (1999). creation of favorable user perceptions: exploring the role of intrinsic motivation. mis quarterly, 23(2), p239-260. venkatesh, v. & davis, f. d. (2000) a theoretical extension of the technology acceptance model: four longitudinal field studies. management science, 46, 186 -204. venkatesh, v., & morris, m. g. (2000). why don't men ever stop to ask for directions? gender, social influence, and their role in technology acceptance and usage behavior. mis quarterly, 24(1), p115-139. venkatesh, v., morris, m. g., sykes, t. a., & ackerman, p. l. (2004). individual reactions to new technologies in the workplace: the role of gender as a psychological construct. journal of applied social psychology, 34(3), 445-467. verplanken, b., aarts, h., van knippenberg, a. & moonen, a. (1998) habit versus planned behaviour: a field experiment. british journal of social psychology, 37, 111-128. vroom, v. h. (1964). work and motivation. (new york: wiley) wang, p., berry, m. w. & yang, y. (2003) mining longitudinal web queries: trends and patterns. journal of the american society for information science and technology, 54, p.743 758. wang, y.-s., lin, h.-h., & luarn, p. (2006). predicting consumer intention to use mobile service. information systems journal, 16, 157–179. australasian journal of information systems volume 17 number 1 2011 38 wang, y.-s., wu, m.-c., & wang, h.-y. (2008). investigating the determinants and age and gender differences in the acceptance of mobile learning. british journal of educational technology, in press: (published article online: 27-feb-2008). wilson, t. d. (1997). information behavior: an interdisciplinary approach. information processing and management, 33(4), pp.551-572. woodroof, j. & burg, w. (2003) satisfaction/dissatisfaction: are users predisposed? information & management, 40, 317-324. workman, m. (2004). performance and perceived effectiveness in computer-based and computeraided education: do cognitive styles make a difference? computers in human behaviour, 20(4), 517–534. xu, y. (2007) the dynamics of interactive information retrieval behavior, part i: an activity theory perspective. journal of the american society for information science and technology, 58, 958-970. yee, p. l., hsieh-yee, i., pierce, g. r., grome, r. & schantz, l. (2004) self-evaluative intrusive thoughts impede successful searching on the internet. computers in human behaviour, 20, 85101. yi, m. y., jackson, j. d., park, j. s., & probst, j. c. (2006). understanding information technology acceptance by individual professionals: toward an integrative view. information & management, 43(3), 350-363. yin, r. (1994). case study research: design and methods (2nd ed.). beverly hills, ca: sage publishing. zhang, p., & von dran, g. m. (2000). satisfiers and dissatisfiers: a two-factor model for website design and evaluation. journal of the american society for information science & technology, 51(14), 1253–1268. australasian journal of information systems volume 17 number 1 2011 39 appendix 1: the evolution of the tam and its constructs (1989-2007) australasian journal of information systems volume 17 number 1 2011 40 australasian journal of information systems volume 17 number 1 2011 41 appendix 2: a simple comparable numerical weight for tam results the percentage results to each tam question was put through a weighting multiplication formula in order to arrive at a comparable numerical value for the result. in this case, “using the www would enable me to accomplish research related tasks more quickly” was attributed a score of 12.31 out of a possible maximum of 15, where 15 can only be achieved if 100% of users state “extremely likely” 55 information systems management: an australasian view of key issues -1996 graham pervan, school of information systems, curtin university of technology, gpo box u1987, perth, wa, 6001, australia email: pervang@cbs.curtin.edu.au abstract as part of a longitudinal study of key information systems management issues, a survey of australasia's largest 500 organisations was conducted to identify which issues were perceived by the is executives as being important, problematic and critical over the next three to five years. the most critical issues were revealed to be a mix of technology management issues (it infrastructure, communications, disaster recovery), strategic management issues (competitive advantage, is planning, aligning the is organisation), people management issues (organisational learning, educating senior management in it), systems development and data management issues (effective use of the data resource), and end-user computing. this reflects the need for a balance of business, technical, and people skills in an is executive. non-critical issues were mostly related to systems development and the individual technologies which must be integrated and managed to ensure a responsive it infrastructure. the study also reveals that some issues are much more important than problematic (disaster recovery, competitive advantage, information architecture, and is alignment) while others are much more problematic than important (end-user computing, is role and contribution, and bpr). the former reflects a growing level of knowledge in handling these issues, while the latter reflects a continuing problem with them. the differences between this study and similar studies conducted in 1988 and 1992 show that there is a growing confidence in the is executive's ability to manage the strategic issues, a continuing concern about providing a responsive it infrastructure (especially communications), and a continuing concern with educating all 'customers' in the organisation in the effective use of it, including senior management. introduction rapidly changing business environments are challenging information systems (is) executives as evidenced by the now familiar themes of downsizing, outsourcing, leveraged buyouts, strategic alliances, flexible manufacturing, just-in-time scheduling, globalisation, business process reengineering and total quality management. these environmental changes place demands on is departments to support product innovation, new production techniques, and changing organisational designs, and provide timely, high-quality information. as a result, the role of is executives has shifted from merely supporting a technical operation to a strategic role providing information for the organisation to implement strategies and achieve objectives (galliers et al. 1994). identifying and handling key issues in the management of information systems is essential for is executives to support their organisations efficiently and effectively. the investigation of these key issues by researchers serves to enhance understanding of the concerns of is executives and suggest relevant areas of investigation by is management researchers. the aims of this study are to: • determine the is management issues most important, problematic and critical to australasian is executives over the next 3-5 years; • examine trends in the status of different issues over time; • inform is professionals of current and future issues; • inform is researchers of relevant issues to investigate. a brief history of key issues studies many key issues studies have now been carried out, from ball and harris (1982) to the most recent (1995) sim (usa) survey (brancheau et al., 1996). an historical summary of 21 major studies has been provided in table 1 below: authors ball & harris martin dickson, leitheiser, wetherbe & nechis hartog & herbert brancheau & wetherbe year of study 1982 1982 1984 1986 1987 country u.s.a u.s.a u.s.a u.s.a u.s.a sample (response rate) sim members (417 / 1400) is executives (15/??) sim members (54 / ??) is managers (63 / 107) is & gen. mgrs. (68/180) total no. of issues 18 8 19 21 26 research method survey c.s.f interviews 2 rounds four round delphi survey survey and interviews three round delphi survey ajis vol 5 no 1 september 1997 56 parker & idundun rao, huff & davis davenport & buday watson moynihan caudle, gorr & newcomer niederman, brancheau & wetherbe watson & brancheau badri clark doukidis, smithson & naoum galliers, merali & spearing pervan wang & turban davis, menon, munday, thomson & young kim & sato brancheau, janz & wetherbe 1987 1987 1988 1988 1990 1991 1991 1991 1992 1992 1992 1992 1992 1994 1995 1995 1995 united kingdom singapore europe australia ireland u.s.a u.s. a international comparison gulf coop'n council u.s.a greece united kingdom australia taiwan new zealand japan u.s.a. is managers (45 / 100) is managers (19/105) is executives (75/2000) is managers (48 / 200) ceo/it mgrs. (49/??) ps mgrs. (354/1054) sim members (1. (104/241) previous studies compared is ceos (96/120) is executives (30) is managers (40) managers (incl. is) (98/??) is managers (1. (88/300) is & gen. mgrs. (297/928) various (118/392) is professionals (197/400) sim members (83/217) 23 ?? ?? 26 ?? 37 25 15 20 6 20 26 34 30 23 25 23 survey and interviews survey survey three round delphi survey structured interviews survey three round delphi survey descriptive/ interpretive survey structured interviews personal interviews survey three round delphi survey survey three round delphi survey survey three round delphi survey table 1: a history of key issues studies the most common research approach used has been the postal survey, either single-round or in threeor fourround delphi studies. the major us studies have sampled the membership of the society for information management (sim) whose membership consists of a mix of is academics and practitioners. most other studies have obtained opinions from is managers and/or other non-is managers. in most cases the organisations sampled have been quite varied in size and industry, and the sample sizes have varied significantly. the studies have generally sought opinions of the major 'key' or 'critical' information systems management issues facing these organisations. one of the most recent studies (galliers et al., 1994) considers issues as important (they have a significant impact within the organisation), problematic (they are difficult to manage), and critical (they are both important and problematic). the key issues as indicated in table 1, there are many different is management issues. while most researchers accept that is executives must handle many technology management (tm) issues, there is a growing acceptance that there are many strategic management (sm) issues which are equally, if not more, important. other categories of issues include systems development and data management (sddm), people management (pm), and end-user computing (euc). a brief discussion of the major categories and issues follows, though it should be pointed out that some of the issues may overlap some categories. these issues have changed over time, but those discussed below are based on the most recent australian (pervan, 1994), us (brancheau et al., 1996), and uk (galliers et al., 1994) studies. ajis vol 5 no 1 september 1997 57 technology management issues in the 1990s organisations are heavily committed to investment in information and communications technology. the major technology management issues may include: • building a responsive it infrastructure (to support applications in the face of rapidly changing technologies and range of applications); • measuring is effectiveness and productivity (to justify investment in new technologies); • improving data integrity and quality assurance (to overcome inconsistencies between different data sources and the lack of controls in is and user departments); • developing and managing electronic data interchange (to provide electronic communication with customers and suppliers); • planning and integrating multi-vendor open systems technologies (in the face of a variety of operating environments and unstable standards); • integrating data processing, office automation, and telecommunications; • managing data and document storage; • planning and managing communications networks (the 'lifeblood' of the 1990s organisation); • implementing and managing collaborative support systems (to help teams share information and improve their effectiveness); and • establishing effective disaster recovery capabilities (to guard against downside risks and potential loss of business). strategic management issues traditionally, the is function was seen by top management as being primarily a support activity with little impact on the profitability or nature of the business (doukidis et al., 1991), but this viewpoint is changing in the 1990s. is strategic planning can help to identify high return applications, gain competitive advantage, improve the is function, support business strategy, and provide a general framework to guide the activities of the organisation. the major strategic management issues thus may include: • improving is strategic planning (to align is plans with business strategic plans in the face of changing business and technological environments, increasing user involvement); • developing and implementing an information architecture (to identify major business categories and relationships to business processes, guide applications development, and facilitate better data integration and sharing); • aligning the is organisation within the enterprise (which may require a combination of centralised and decentralised structures); • outsourcing selected information services; • determining appropriate is funding levels; • facilitating and managing business process redesign (where it may play an increasingly important role); and • using information systems for competitive advantage (by recognising opportunities through creativity and innovation followed by rapid implementation). people management issues current and future shortages of qualified information systems personnel threaten the is department's ability to keep up with the information requirements of the organisation and greater business skills are needed to provide a better understanding of these needs. further, the members of the organisation as a whole (including senior management) need to develop appropriate knowledge and understanding in the use of information systems and technology. the major people management issues thus may include: • recruiting and developing is human resources (to overcome shortages, develop career paths, enhance business skills and skills with new technologies and methodologies); • facilitating organisational learning (in order to make appropriate use of information technologies across the entire organisation); • educating senior management in relation to it (to support resource allocation to it and enhance understanding of the strategic impact of it); and • increasing understanding of is role and contribution (by all members of the organisation). ajis vol 5 no 1 september 1997 58 systems development and data management issues in the 1980s the is discipline was mostly concerned with the problems of systems development and data management (hartog and herbert 1986). a growing number of techniques and tools have been developed to improve the productivity and quality of is development. the major systems and data issues may include: • improving the effectiveness of software development (to reduce backlogs through new development methods and platforms); • selecting and integrating packaged applications software (which may cut costs and boost productivity, but may result in integration and maintenance problems); • making effective use of the data resource (through appropriate database technologies and valuing data as a corporate asset); • managing the existing portfolio of legacy applications (which may affect integration of new technologies and migrating to new operating environments); • developing and managing distributed systems (where client-server environments provide problems of consistency of software versions and data, and challenges in project management); • improving information security and control; • planning and managing the applications portfolio (and the ever-increasing maintenance problem); and • planning and using case technology (to more effectively support the systems development process). end-user computing issues one of the most significant is challenges is the control of computing by end users who may develop and/or operate systems independently of the is department (doukidis et al. 1991). the is department must balance control with learning and innovation in order to provide support for these end users. these issues may include: • facilitating and managing end-user computing (by balancing control against the need for slack and clarifying the respective roles of is and end-users); and • facilitating/managing executive and decision support systems (to improve decision making by executives and other members of the organisation). research design in order to provide information concerning the key is management issues, a postal survey of the most senior is person in the top 500 australasian organisations was conducted in march/april, 1996. the bulletin magazine's publication of the top 500' organisations in australia and new zealand in 1995 (jacques, 1995) provided a list (and associated financial and classification data) of the 500 largest organisations by revenue. from the list of identified organisations, a mailing list of the most senior is person in each organisation was developed by telephone contact to each organisation to identify the name and exact title of the appropriate person. (personal identification and addressing of postal surveys is a recommended strategy for increasing response rates (forsgren 1989)). in the course of the development of this mailing list, some organisations were removed because they were either subsidiary to other (larger) organisations on the list which provided the it management function or because they were just unwilling to participate in such surveys. a final list of 490 'cios' (chief information officers) was produced (though some of these cios used other titles, including it manager, is manager, and director of it). the aim of this study was to identify the key is management issues for these organisations over the next 3-5 years. the approach taken was similar to the most recent uk study (galliers et al., 1994) where the respondents were asked to rate both how important and how problematic each issue was for their organisation. a list of 31 issues with an associated paragraph of explanation for each issue was produced from the 25 most critical issues from the last australian study and any new issues from the most recent us and uk studies. in addition to the brief title for each issue, a paragraph explaining the issue (its 'rationale') was included to ensure that the meaning of each issue was interpreted consistently across all respondents. these 31 issues have been briefly discussed in the previous section and a list of all issues and their rationale is provided as appendix a. the results of the study valid questionnaires were received from 105 of the 490 organisations, representing the full range of the organisations targeted (including the 1st and 497th ranked organisations). table 2 below shows that these are ajis vol 5 no 1 september 1997 59 indeed large organisations with revenue, assets and profits measured in the billions of dollars. table 3 shows the location of company headquarters and the sector to which that organisation's primary activity belongs. the response rate of 21.4% may be considered reasonable in a postal survey of this nature. variable revenue assets profit mean ($b) 1.339 1.868 0.083 st dev. ($b) 4.376 2.986 0.248 min. ($b) 0.160 0.014 -0.372 max. ($b) 18.487 30.287 1.753 table 2: respondent organisations financial data headquarters sector resources/mining retail/trading services manufacturing government total vic 3 4 8 14 5 34 nsw 3 6 7 25 3 44 sa 0 0 2 2 1 5 qld 0 2 2 1 3 8 wa 1 1 2 4 2 10 act 0 0 0 0 2 2 nt 0 0 0 0 1 1 nz 0 0 0 1 0 1 total 7 13 21 47 17 105 table 3: respondent organisations hq and sector as table 3 above indicates, responses were received from all states and territories and one from new zealand (a major beer producer). the largest sector represented in die sample (and in the target top 500 population) was manufacturing (45%), while major government instrumentalities for energy, water, and transport made up most of the 16% from the government sector. the two most populous states, new south wales and victoria, made up 74% of the sample, confirming that the majority of company headquarters of large australian organisations are in sydney and melbourne. chi-squared goodness of fit tests on location of headquarters, rank in the top 500, and industry sector showed the sample to be representative at the 5% level of significance. critical issues as indicated earlier, respondents were asked to rate both how important an issue will be for their organisation over the next 3-5 years, and how problematic the issue might be. these were both rated on a 1 to 10 scale where 1 represented a lowest priority issue and 10 represented a highest priority issue. all respondents provided a score for both on all issues. before separately examining the important issues, and similarly the problematic issues, the two ratings were combined into an overall rating of the issue. in previous studies, issues were usually assessed on how critical they were for the organisation. this study used the approach of galliers et al. (1994), where a critical issue was defined as being both important and problematic. based on this definition, how critical an issue was to the organisation was calculated as the arithmetic mean of the ratings for important and problematic. the ten most critical issues, with their ranking, mean and standard deviation of critical rating, are shown in table 4 below. a full list showing mean and standard deviation for all 31 issues is provided as appendix b. rank 1 2 3 4 5 6 7 8 8 10 issue building a responsive it infrastructure planning and managing communications networks establishing effective disaster recovery capabilities making effective use of the data resource facilitating organisational learning facilitating and managing end user computing educating senior management in relation to it using information systems for competitive advantage improving is strategic planning aligning the is organisation within the enterprise issue category tm tm tm sddm pm euc pm sm sm sm mean rating 7.19 6.93 6.88 6.87 6.86 6.80 6.73 6.72 6.72 6.71 std. dev. 1.65 1.92 1.66 1.63 1.59 1.73 1.96 1.91 1.91 2.04 table 4: the ten most critical issues ajis vol 5 no 1 september 1997 60 the relative ranking of issues within the top 10 should not be overly highlighted because their mean ratings range only by 0.48 on a 10-point scale (with a range of only 0.22 between 2nd and 10th) and the average standard deviation is 1.80. however, it is interesting to note that each of the five issue categories have one or more issues in this top group. while previous research seemed to indicate that there would be a trend away from technology management for cios, this is contradicted by the appearance of it infrastructure, communications, and disaster recovery (all technology management issues). however, this should come as no surprise in such a rapidly changing technological environment. it is clear that the fundamental role of the cio is the provision of a responsive it infrastructure for the organisation and that communications networks provide the backbone of that infrastructure in the 1990s. this infrastructure can also be assumed to play a critical role in recognising, accessing, and utilising the organisation's data resources (ranked 4th). people management issues also rank highly, with the issues of organisational learning and educating senior management in it ranked 5th and 7th, respectively, and end-user computing (which involves similar motivations and contains a significant user training/education component)) ranked 6th. with the advent of desktop computers and distributed computing, larger organisations are beginning to view the non-is people in the organisation as 'customers' rather than 'users' and to perceive is as a service function for those customers (pitt et al., 1995). further, the it education of the customers comes through very strongly in these issues, both from an organisational efficiency and effectiveness viewpoint (through increased productivity and reducing applications backlogs see brancheau and brown (1993)), and as a strategy for enhancing the acceptance of is within the organisation (moynihan, 1990). an emphasis on supporting and managing organisational learning can lead to significant increases in innovation and productivity from an organisation's it investment (henderson and lentz, 1996). as indicated in table 4, some strategic issues also ranked highly (in addition to the fact that some of the technology and people management issues have strategic aspects). strategic is planning (sisp) continues to be a major issue, being ranked 9th. it is a key element of is management and the cio must lead in the use of formal methods for sisp and encourage good practice within the organisation (earl, 1993). this can facilitate effective management of other strategic issues such as the alignment of is within the enterprise (ranked 10th) and identifying opportunities to use is for competitive advantage (ranked 8th). in summary, the most critical issues were revealed to be a mix of technology management issues (it infrastructure, communications, disaster recovery), strategic management issues (competitive advantage, is planning, aligning the is organisation), people management issues (organisational learning, educating senior management in it), systems development and data management issues (effective use of the data resource), and end-user computing. this reflects the need for a balance of business, technical, and people skills in an is executive. non-critical issues the issues which were rated and ranked the ten least critical are shown in table 5 below. overall, it may be observed from table 5 that individual technology management and systems development and data management issues are of little concern to the cio. five of the ten least critical issues relate to individual information technologies (namely, electronic data interchange, collaborative systems, data/document storage, multi-vendor integration, and case) and three relate to specific systems management issues (namely, distributed systems, applications portfolio, and legacy systems). while these systems and technologies are part of the overall infrastructure, it is their overall selection, integration, and management which is of greater concern to the cio and the organisation. rank 22 23 24 25 26 27 28 29 30 31 issue developing and managing distributed systems determining appropriate is funding levels developing and managing electronic data interchange implementing and managing collaborative systems managing data and document storage planning and managing the applications portfolio managing existing portfolio of legacy applications multivendor open systems technologies outsourcing selected information services planning and using case technology issue category sddm sm tm tm tm sddm sddm tm sm tm mean rating 6.16 5.98 5.84 5.73 5.64 5.60 5.49 5.41 5.24 4.39 std. dev. 2.35 2.09 2.01 2.17 1.91 1.85 2.07 2.27 2.41 2.21 table 5: the ten least critical issues ajis vol 5 no 1 september 1997 61 of the strategic management issues, determining is funding levels ranks quite low. this may reflect a growing recognition by the senior management of organisations of the value of it to the organisation and, as a result, less pressure on the cio in arguing the case for appropriate funding of it. the major surprise in this list of noncritical issues is it outsourcing which ranked almost last. it may be that these cios believe that, despite the hype, they have (or soon will have) solved their it outsourcing decision problems. it may also be that this issue is perceived by cios as more of a specific it infrastructure issue and whether a technology or service is provided in-house or outsourced is of little concern as long as it is provided. however, further analysis revealed that the rating of this issue was significantly higher for the 17 government organisations than for the organisations in the other sectors. political leaders continue to see outsourcing of services (including it) as a major strategy for smaller government and the cios of these organisations recognise that this is unlikely to change in the foreseeable future. important vs. problematic issues in order to clarify the distinction between what is important and what is problematic, the issues which rated, on average, in either the ten most important issues or the ten most problematic issues are summarised in table 6 below. this also helps to clarify why some issues are more critical than others (because they are important and problematic). prob. rate 6.82 6.40 6.13 6.40 6.43 6.52 6.33 6.00 6.18 6.01 6.00 6.04 6.51 6.26 prob. rank 1 5 11 5 4 2 8 18 10 17 18 14 3 9 issue building a responsive it infrastructure planning and managing communications networks establishing effective disaster recovery capabilities making effective use of the data resource facilitating organisational learning facilitating and managing end user computing educating senior management in relation to it using information systems for competitive advantage improving is strategic planning aligning the is organisation within the enterprise developing & implementing an information architecture improving information security and control increasing understanding of is role and contribution facilitating and managing business process redesign imp. rank 2 3 1 7 8 12 11 4 9 5 6 10 18 20 imp. rate 7.55 7.46 7.62 7.33 7.30 7.09 7.13 7.45 7.26 7.41 7.38 7.25 6.70 6.55 table 6: important vs. problematic issues table 6 indicates that substantial differences exist between the rankings of problematic and important issues. a rank correlation test of the first 20 important issues with the first 20 problematic issues shows a correlation of only 0.096. interestingly, the rankings of the remaining issues (the eleven least important and problematic issues) were almost identical, which implies that the non-critical issues are so because they are perceived to be neither important nor problematic. issues which are ranked as far more important than problematic include disaster recovery, competitive advantage, information architecture and is alignment. this may be because these issues are perceived as important to the organisation (i.e., they can have a significant impact on the organisation), but the cios believe that in the future they will have them under control. the growing acceptance and quality of strategic is planning supports the belief that developing an appropriate information architecture (a direct outcome of sisp), identifying is for competitive advantage, and properly aligning the is organisation will be facilitated more easily (earl, 1993). on the other hand, issues which are ranked far more problematic than important include end-user computing, is role and contribution, and business process redesign (bpr). clearly, these issues are not well managed now and the cios of these large organisations believe that they may continue to be difficult issues to manage. the literature on end-user computing (brancheau and brown, 1993) and bpr (earl, 1994; wastell et al., 1994) both support this concern. the issue concerning the development of an understanding of is role and contribution is closely associated with the other 'education' issues facilitating organisational learning and educating senior management in it (both of which also ranked more problematic than important as shown in table 6) and indicates a growing concern for developing knowledge and understanding of it throughout the organisation. ajis vol 5 no 1 september 1997 62 comparison with previous australian studies as indicated earlier in this paper, there have been many previous key issues studies but this comparison will focus on the previous australian key issues studies conducted in 1988 watson, 1989) and 1992 (pervan, 1994). table 7 below summarises the comparative rankings of the key issues from these studies and whether its status as an issue is rising, steady, or falling. for an issue to be included in the comparisons table it had to rank in the top ten issues in either 1988, 1992 or 1996, and the fifteen such issues fall neatly into three groups the five 'rising stars' (which have risen significantly over the eight years), six issues which have remained fairly steady over that period, and four 'falling stars' (which have fallen significantly over the three studies). the five 'rising stars' fall into a group of technology infrastucture management issues and a pair of educational issues. the biggest upward movers since 1988 have been it infrastructure, communications networks, and disaster recovery. in the face of the rapid pace of technological change, the management of the it infrastructure should be expected to be a key issue and the most fundamental underlying technology is the organisation's communications network which provides the critical linking of members of the organisation with each other and with the outside world. intranets and the internet can be expected to be major concerns for the cio for some time to come. in this rapidly changing technological environment it is becoming increasingly important for all levels of the organisation to receive an it education which will maintain their effective utilisation of the it resource. this is reflected in the rise of both organisational learning and it education of senior management as key issues. issue building a responsive it infrastructure planning and managing communications networks establishing effective disaster recovery capabilities making effective use of the data resource facilitating organisational learning facilitating and managing end user computing educating senior management in relation to it using is for competitive advantage improving is strategic planning aligning the is organisation within the enterprise developing & implementing an information architecture improving data integrity and quality assurance improving the effectiveness of software development increasing understanding of is role and contribution recruiting and developing is human resources 1988 ni 13 ni 9 10 8 ni 7 1 5 3 11 4 6 2 1992 2 15 10 4 12 11 ni 5 1 3 6 7 8 9 nr 1996 1 2 3 4 5 6 7 8 8 10 11 17 21 14 20 status rising rising rising steady rising steady rising steady steady steady steady falling falling falling falling table 7: historical comparison of australian key issues studies (ni = not included in the study; nr = not ranked in the top 20) examination of the 'steady' issues shows that they include the four major strategic management issues of is for competitive advantage, is strategic planning, is organisational alignment, and information architecture. these have been fairly steady (though slipping slightly in rank) and may reflect a growing confidence that these issues are being done better through the use of more formal approaches and a perhaps a better understanding of the relationship between the cio and ceo (feeny et al., 1992). facilitating and managing end-user computing has continued to be (and will probably always be) a fairly critical issue (brancheau and brown, 1993). similarly, making effective use of the data resource will continue to be fairly critical in an environment where a greater variety of data (internal and external to the organisation) on a greater variety of platforms need to be accessed, manipulated, analysed, and presented by and for the user-customers. among the 'falling stars', improving data quality and improving software development effectiveness both may reflect a better attitude to quality and effectiveness in australian organisations (or at least a perception that this is being achieved), and so a growing confidence among cios that these issues are now under control. similarly, the the fall of understanding is role and contribution may also reflect better communication between cio and the organisation's senior management (feeny et al., 1992). finally, the issue of recruiting and developing is human resources may be associated with a number of changes including greater availability of people with the necessary skills, reliance on the outsourcing of is services (provided by large and small consulting ajis vol 5 no 1 september 1997 63 organisations), and technologies which give the end-users greater capacity to develop their own systems and solve many of their own problems. conclusions and further research this paper has presented some results from a study of the key issues facing is executives in australasia's largest 500 organisations. the most critical issues are a mix of technology management issues (it infrastructure, communications, disaster recovery), strategic management issues (competitive advantage, is planning, aligning the is organisation), people management issues (organisational learning, educating senior management in it), systems development and data management issues (effective use of the data resource), and end-user computing. this reflects the need for a balance of business, technical, and people skills in a cio. non-critical issues are mostly related to systems development and the individual technologies which must be integrated and managed to ensure a responsive it infrastructure. the study reveals that some issues are much more important than problematic (disaster recovery, competitive advantage, information architecture, and is alignment), which reflects a growing level of knowledge in handling these issues. other issues, however, are much more problematic than important (end-user computing, is role and contribution, and bpr), and this reflects a continuing problem with their effective management. the differences between this 1996 study and the australian studies conducted in 1992 (pervan, 1994) and 1988 (watson, 1989) show that there is a growing confidence in the cios ability to manage the strategic issues, a continuing concern about providing a responsive it infrastructure (especially communications), and a continuing concern with educating all 'customers' in the organisation, including senior management. on the other hand, system and data quality issues, is role and contribution, and is human resources are becoming less critical. this descriptive survey has its limitations, however, and further explanatory work, which will atempt to answer some of the reasons behing the issues, is planned. a follow-up survey will revisit the issues and collect information from the cios concerning various aspects of organisational structure, management style, stage of it maturity, the external environment, and other factors with a view to identifying factors which may explain why different issues vary in how important, problematic, and critical they are in different organisations. another follow-up study will be to provide the list of issues to the chief executive officers (ceos) of these and compare their ratings with the cios. finally, a series of case studies is planned which will involve interviews of a number of the cios in the top 500 organisations to investigate the reasons behind the relative ratings of the different issues. note: an earlier (and smaller) version of this paper has been accepted for presentation at the 7th australasian conference on information systems, university of tasmania, december 1996. references badri, m.a. (1992) "critical issues in information systems management: an international perspective", international journal of information management, 12, 179-191. ball, l. and harris, r. (1982) "smis member: a membership analysis", mis quarterly, 6, 19-38. brancheau, j.c. and brown, c.v. (1993) "the management of end-user computing: status and directions", acm computing surveys, 25, 437-481. brancheau, j.c., janz, b.d. and wetherbe, j.c. (1996) "key issues in information systems management: a shift toward technology infrastructure", mis quarterly, 20,2,225-242. brancheau, j.c. and wetherbe, j.c. (1987) "key issues in information systems management", mis quarterly, 11,23-45. caudle, s.l., gorr, w.p. and newcomer, k.e. (1991) "key information systems management issues for the public sector", mis quarterly, 15, 170-188. clark, t.d., jr. (1992) "corporate systems management: an overview and research perspective", communications of the acm, 35, 61-75. davenport, t. and buday, r. (1988) "critical issues in information systems management in 1988", index group. davis, j.g., menon, r., munday, s.r., thomson, b.c. and young, l.w. (1995) "key issues in information systems management: a new zealand perspective", proceedings of the prhsm'95 international conference, maui, hawaii, january 2-3, 187-195. dickson, g.w., leitheiser, r.l., wetherbe, j.c. and nechis, m. (1984) "key information systems issues for the 1980's", mis quarterly, 8, 135-159. doukidis, g.i., smithson, s. and naoum, g. (1992) "information systems management issues in greece: issues and perceptions", journal of strategic information systems, 1,63-75. earl, m.j. (1993) experiences in strategic information systems planning, mis quarterly, 17, 1-24. ajis vol 5 no 1 september 1997 64 earl, m.j. (1994) "the new and old of business process redesign", journal of strategic information systems, 3,5-22. feeny, d.f., edwards, b.r. and simpson, k.m. (1992) "understanding the ceo/cio relationship", mis quarterly, 16,435-448. forsgen, r.a. (1989) "increasing mail survey response rates: methods for small business researchers", journal of small business management, october, 61-66. galliers, r.d., merali, y. and spearing, l. (1994) "coping with information technology? how british executives perceive the key information systems management issues in the mid-1990s", journal of information technology, 9, 223-238. hartog, c. and herbert, m. (1986) 1985 "opinion survey of mis managers: key issues", mis quarterly, 10, 351-360. henderson, j.c. and lentz, c.m.a. (1996) "learning, working, and innovation: a case study in the insurance industry", journal of management information systems, 12,43-64. jacques, b. (1995) "top 500: overview of the year", the bulletin, december 5,4-50. kim, y.g. and sato, o. (1995) "key issues in information systems management: a japanese perspective", proceedings of the priism'95 international conference, maui, hawaii, january 2-3, 167-174. martin, e.w. (1982) "critical success factors of key mis/dp executives", mis quarterly, 6, 1-9. moynihan, t. (1990) "what chief executives and senior managers want from their it departments", mis quarterly, 14, 15-26. niederman, f., brancheau, j. and wetherbe (1991) "information systems management issues for the 1990s", mis quarterly, 15,474-500. parker, t. and idundun, m. (1988) "managing information systems in 1987: the top issues for is management in the uk", journal of information technology, 3, 31-42. pervan, g.p. (1993) "results from a study of key issues in australian is management", proceedings of the 4th australian conference on information systems, brisbane, queensland, september 28-30, 113-128. pervan, g.p. (1994) "information systems management: an australian view of the key issues", australian journal of information systems, 1, 32-44. pitt, l.f., watson, r.t. and kavan, c.b. (1995) "service quality: a measure of information systems effectiveness", mis quarterly, 19, 173-187. rao, k.v., huff, p.p. and davis, g.b. (1987) "critical issues in the management of information systems: a comparison of singapore and the usa", journal of information technology, 1, 11-19. wang, p. and turban, e. (1994) "management information systems issues of the 1990s in the republic of china: an industry analysis", international journal of information management, 14, 25-38. wastell, d.g., white, p. and kawalek, p. (1994) "a methodology for business process redesign: experiences and issues", journal of strategic information systems, 3, 23-40. watson, r.t. (1988) "key issues in information systems management: an australian perspective 1988", australian computer journal, 21, 118-129. watson, r.t. and brancheau, j.c. (1991) "key issues in information systems management: an international perspective", information and management, 20, 213-223. ajis vol 5 no 1 september 1997 65 appendix a: key issues questionnaire what do you consider to be the most critical is/it issues facing your organisation over the next three to five years? you are requested to indicate the general level of importance that issue will have for your organisation and also (separately) how problematic it might be on a day to day level. please indicate your views by rating each issue on the scale 1 to 10, where 1 indicates a lowest priority issue and 10 indicates a highest priority issue. the more important/problematic the issue the higher the score. use the spaces below to assign your ratings. remember, for each issue give one importance score and one problematic score. each issue is supported by an associated rationale to clarify its meaning. lowest priority issue moderate priority issue highest priority issue 1 2 3 4 5 6 7 8 9 1 0 (please fill in your rating of how important and how problematic the issue is in the spaces provided.) important problematic key issues and their rationale 1. developing and implementing an information architecture. a corporate/global information architecture is needed to identify the major information categories used within an enterprise and their relationships to business processes. it is essential for guiding applications development and facilitating the integration and sharing of data. 2. improving information security and control as organisations increase their dependence on information systems, there is a greater risk from destruction and alteration of data, disclosure to outside sources, and disruption of information services. tight security controls and fault tolerant information delivery are becoming a necessity. 3. facilitating organisational learning organisations that prosper will need to make appropriate use of information technologies across their entire enterprise. business practices and organisational structures will need to be modified in many cases. is also must demonstrate its own ability to learn and use new technology. 4. recruiting, and developing is human resources current and future shortages of qualified information systems personnel threaten the organisation's ability to make effective use of information technology. career paths need to be clarified. more emphasis needs to be put on developing business skills such as teamwork and leadership and staying current with new technology such as object-oriented and multi-media applications. 5. measuring is effectiveness and productivity understanding how it use impacts the bottom line is crucial for justifying new investment. in addition, measuring the is organisation's performance is necessary for effective management. measurement is becoming more important as companies attempt to reduce operating expenses to meet the competition. 6. improving data integrity and quality assurance mainframe applications, timesharing extracts, and personal computer applications are all used to analyse data and prepare reports. frequently there are discrepancies among these different data sources due to lack of controls in is and user departments. too often computer output is assumed to be accurate. such assumptions can lead to faulty business decisions. 7. improving the effectiveness of software development the application development backlog remains at unacceptably high levels. traditional development methods and platforms are no longer satisfactory while new methods and platforms have not yet proven themselves. sophisticated users are getting impatient. improved effectiveness will be essential for next-generation applications. 8. facilitating/managing executive and decision support systems increasing the ability to exploit situations for competitive advantage depends on enhancing the ability of management to "experiment" with decision possibilities. many other issues also depend on this capability. decision support tools have long been viewed as a method for introducing modelling tools to executives to improve their decision making, however those efforts have met with mixed success. 9. selecting and integrating packaged application software the changing relationship among costs of hardware, packaged software, and human resources makes managing this aspect of the business critical. using purchased application software can cut costs, boost productivity, and increase user satisfaction, but it can result in poor integration of systems and serious maintenance problems. 10. aligning the is organisation within the enterprise ajis vol 5 no 1 september 1997 66 the is organisation's effectiveness in supporting the enterprise's needs is dependent on its organisational location within the enterprise. appropriate alignment may require a combination of centralised and decentralised structures. too often is is not located and structured appropriately. 11. improving is strategic planning it has always been important to align long-range is plans with strategic business plans. rapidly changing business environments, increased involvement of end-users, and accelerated technological change underscore the need to continue improving strategic planning skills. 12. making effective use of the data resource the organisation's data resource is growing in size, complexity, and value. despite this, it remains largely unrecognised, inaccessible, and underutilised. is must develop a climate within its department and throughout the organisation which values the data resources as a corporate asset. 13. developing and managing electronic data interchange electronic communication with customers and suppliers may offer competitive advantage to a company or it may be a requirement for staying in business. is executives must develop (or adapt to) standard transaction formats, keep current on technology developments, and learn to manage inter-organisational projects. 14. outsourcing selected information services the internal information systems organisation no longer has a monopoly. outside contractors may be able to provide some services more effectively. what services should be outsourced? how should contractor relationships be managed? fair and objective evaluation techniques are needed which assess both costs and benefits as well as potential risks from loss of control. 15. planning and integrating multi-vendor open systems technologies many companies are moving away from single-vendor proprietary operating environments to vendor-neutral environments based on industry and defacto standards. due to large investments in legacy systems, carefully planned migration paths are critical. the task is complicated by a still-maturing technology and unstable standards. 16. determining appropriate is funding levels there is no generally accepted way of establishing the level of is funding relative to the other funding needs of the organisation. this can put both is and general managers at a disadvantage. 17. facilitating and managing business process redesign to remain competitive, many organisations are radically changing the way they do business. it plays an increasingly important role in this change process by enabling the innovative redesign of core business processes. much has been learned about it implementation in general which can help facilitate and manage bpr projects. 18. increasing understanding of is role and contribution is is often viewed as an operational activity with little recognition of its strategic contribution to the organisation. this can result in executive management viewing is strictly as an overhead expense. funding can be cut resulting in missed opportunities for using it to solve important business problems. 19. integrating data processing, office automation, and telecommunications the capability now exists to integrate systems that are based on these diverse technologies. as organisations try to integrate their technologies, organisational and managerial problems will need to be solved. 20. managing data and document storage information stored in both electronic and hardcopy form is accumulating at a staggering pace. there is a need to provide for future data and document storage requirements. 21. planning and managing communications networks communication is the lifeblood of the organisation. using is for competitive advantage depends heavily on access to appropriate internal and external communication networks. this task is complicated by rapid advances in underlying technology and major structural changes in the communications industry. 22. managing the existing portfolio of legacy applications most organisations have a large investment in their existing applications portfolio. some "legacy" applications may need to be retired quickly. others may need to be leveraged for many years before they are replaced. integrating new technologies and migrating new operating environments can be difficult. too little is known about managing these problems. 23. using information systems for competitive advantage aj1s vol 5 no 1 september 1997 67 in many businesses, long-term survival is dependent upon using information systems to gain competitive advantage. competitive advantage results from recognition of opportunities through creativity and innovation, followed by rapid implementation. these are historical weaknesses of is. 24. developing and managing distributed systems client-server applications promise to offer a cost-effective alternative to centralised applications. unfortunately, they present many challenges including: maintaining consistent software versions; maintaining consistent data; controlling joint development projects with users; and administering large-scale distributed applications. 25. planning and managing the applications portfolio the applications portfolio is rapidly increasing in size, complexity, and maintenance cost. despite the longevity of the maintenance problem, too little is known about managing it effectively. 26. facilitating and managing end-user computing the proliferation of end-user computing through personal computers offers the promise of improved productivity but also the dangers of inadequate management control. information systems management must balance control against the need for slack. clarification of is and end-user roles is a necessity. 27. implementing and managing collaborative support systems new software is needed to support the reengineered, flat, team-based organisation of the future. appropriate it support can help teams share information, lead to faster decision making, and improved team effectiveness. such support will become even more important in a distributed ubiquitous computing environment. 28. educating senior management in relation to it the education of senior managers in an organisation will lead to more enlightened resource allocation for is, a more strategic view in is planning and an understanding of is's role in the organisation. 29. planning and using case technology significant progress has been made automating business functions within organisations; however, a vast productivity gain is possible if the automation process itself is automated. in principle, software systems can provide support for integrating the design efforts of project teams, for standardising representation methods, and for generating code. while this technology is still being refined, providing support for systems development is extraordinarily complex and will require major changes within the is function. 30. building a responsive it infrastructure building a technology infrastructure that will support existing applications while remaining responsive to change is a key to longterm enterprise productivity. this task is frustrated by the continuing rapid changes in infrastructure technology and the increasing breadth and depth of applications which need to be supported. 31. establishing effective disaster recovery capabilities down-side risks are increasing daily from the potential loss of business due to a disaster. effective recovery plans must be in place and tested regularly to ensure losses are minimised as organisational applications grow and become more integrated, the greater the risk becomes. ajis vol s no 1 september 1997 68 appendix b: critical scores for all key issues rank 1 2 3 4 5 6 7 8 8 10 11 12 13 14 15 16 17 18 18 20 21 22 23 24 25 26 27 28 29 30 31 issue building a responsive it infrastructure planning and managing communications networks establishing effective disaster recovery capabilities making effective use of the data resource facilitating organisational learning facilitating and managing end user computing educating senior management in relation to it using information systems for competitive advantage improving is strategic planning aligning the is organisation within the enterprise developing and implementing an information architecture improving information security and control measuring is effectiveness and productivity increasing understanding of is role and contribution selecting and integrating packaged applications software facilitating/managing executive and decision support systems improving data integrity and quality assurance facilitating and managing business process redesign integrating data processing, office automation, telecommunications recruiting and developing is human resources improving the effectiveness of software development developing and managing distributed systems determining appropriate is funding levels developing and managing electronic data interchange implementing and managing collaborative systems managing data and document storage planning and managing the applications portfolio managing existing portfolio of legacy applications multi-vendor open systems technologies outsourcing selected information services planning and using case technology issue category tm tm tm sddm pm euc pm sm sm sm sm sddm tm pm sddm euc tm sm tm pm sddm sddm sm tm tm tm sddm sddm tm sm tm mean rating 7.19 6.93 6.88 6.87 6.86 6.80 6.73 6.72 6.72 6.71 6.69 6.64 6.63 6.60 6.55 6.46 6.41 6.40 6.40 6.29 6.21 6.16 5.98 5.84 5.73 5.64 5.60 5.49 5.41 5.24 4.39 std. dev. 1.65 1.92 1.66 .63 .59 .73 .96 .91 .91 2.04 1.55 1.65 1.71 2.03 2.03 1.84 1.82 2.20 1.88 2.04 2.13 2.35 2.09 2.01 2.17 1.91 1.85 2.07 2.27 2.41 2.21 microsoft word ajis special 2002.doc ajis special issue december 2002 3 the meaning of tacit knowledge c.n.g. ‘kit’ dampney1, peter busch2 and debbie richards2 1 school of design, communications and information technology university of newcastle, callaghan, n.s.w. australia kit.dampney@newcastle.edu.au 2 department of computing, macquarie university, north ryde, n.s.w. australia busch@ics.mq.edu.au richards@ics.mq.edu.au abstract tacit knowledge definitions tend to be extremely varied. some argue that tacit knowledge is precisely that. others feel that only time and effort prevent all tacit knowledge from eventually becoming articulated. for the purposes of our research “tacit knowledge”, in practice at least, encompasses a medium ground, being comprised of articulable and inarticulable subsets. along the lines of weber (1997), we have formalised a meaning for this “tacit knowledge” and for comparison have completed a content analysis of the literature to determine what other researchers understand “tacit knowledge” to mean. introduction let us begin with a quote: data consists of raw facts … information is a collection of facts organised in such a way that they have additional value beyond the value of the facts themselves … knowledge is the body of rules, guidelines, and procedures used to select, organise and manipulate data to make it suitable for a specific task...(stair et. al. 1998, p.5 [italics added]). this generalisation can be extended by noting that knowledge may be partitioned further into two categories, namely tacit knowledge and articulate knowledge; the former, for example, being especially recognised by the japanese (takeuchi, 1998). it was polanyi (1959) (in greeno, 1987) who was instrumental in first … distinguish[ing] between explicit [or articulate] knowledge, “what is usually described as knowledge, as set out in written words or maps, or mathematical formulae,” and tacit knowledge, “such as we have of something we are in the act of doing” (p.12). tacit knowledge versus articulate knowledge tacit knowledge is thus that component of knowledge that is widely held by individuals but not able to be readily expressed. it is expertise, skill, and ‘know how’, as opposed to codified knowledge. alternatively: tacit knowledge is the personal knowledge resident within the mind, behavior and perceptions of individuals. tacit knowledge includes skills, experiences, insight, intuition and judgment. it is typically shared through discussion, stories, analogies and person-toperson interaction; therefore, it is difficult to capture or represent in explicit form. because individuals continually add personal knowledge, which changes behavior and perceptions, tacit knowledge is by definition uncapped (casonato and harris, 1999). articulate knowledge is typically acquired through formal education, writings, books, rule sets, legal code to name but a few of its ways and means. tacit knowledge on the other hand is often acquired through a more intimate relationship, say between a teacher and an apprentice. it is transferred more orally, more by way of example, more by sight. more generally, and this is particularly applicable in a modern organisation, tacit knowledge is acquired through shared experience in cooperative work. a working definition of tacit knowledge on the one hand, it is argued that some tacit knowledge can never actually be articulated (leonard and sensiper, 1998), or indeed all tacit knowledge (burstein, 2001). on the other hand, economists arguing in reductionist terms consider that: “only cost considerations prevent residual forms of tacit knowledge [from being] codified” (ancori, bureth and cohendet, 2000, p.281). indeed, it is often accepted “that tacit knowledge (as distinct from intangible investment more generally) is non-codified, disembodied know how that is acquired in the informal take-up of learned behaviour and procedures” (howells, 1995, p.2). tacit knowledge also has its traces in gärdenfors' conceptual spaces (gärdenfors, 2000). research by aisbett and gibbon (2001) identifies the “subconceptual layer” of gärdenfors as being representable, for example, by neural nets. this suggests that if we equate the brain's subconscious with tacit knowledge, then we have an explanation of tacit knowledge processing as subconscious pattern matching in the human mind. such pattern matching is not explicitly codified of course, until a conscious effort is made to articulate such tacit knowledge, to make it conscious, and to codify it. although the economist hayek had first discussed the presence of inarticulable knowledge (ebeling, 1999), it is polanyi (1958) who is considered the father and discoverer of tacit knowledge, with his reference to subsidiary ajis special issue december 2002 4 and focal awareness. more specifically polanyi (1968) had actually contemplated a triad of: 1) subsidiary particulars, 2) a focal target, and 3) the knower who links the particulars to the focal target. the linking highlights the dependency of tacit knowledge on the context for the particular and the target, as the knower perceives it. by focal awareness polanyi referred to our using systems of meaning to interpret what we see, hear or read; whereas our subsidiary awareness arouses within us past experiences, which guide our ability to further understand what it is we are experiencing. in short: “… tacit knowledge is manifestly present ... not only when it exceeds the powers of articulation, but even when it exactly coincides with them, as it does when we have acquired it a moment before by listening to or reading a text” (polanyi, 1968, p.92). the last part of the quotation above relates to polanyi’s concept of ‘indwelling’, or assimilating outside influences within (polanyi, 1967), so typical of the tacit acquisition process. tacit knowledge, depending on one’s interpretation, may actually be, we speculate, any pattern matching process from sensory skills such as learning to ride a bicycle, through to tricks of the trade, the latter often articulated and passed on from the senior to the apprentice. our use of tacit knowledge in this paper refers to: “those components of technology that are not codified into blueprints, manual patents and the like. in other words, tacit knowledge is intangible knowledge, such as rules of thumb, heuristics, and other “tricks of the trade'“ (arora, 1996, p.234). for the practical purposes of our information systems research, tacit knowledge could be said to be the implicit articulable it managerial knowledge that it practitioners draw upon when conducting their ‘management of themselves, others, and their careers’, as wagner and sternberg (1991a; 1991b) would say. when such tacit knowledge is shared from mutual experience and culture it gains a dimension within an organisation. it thus requires an added dimension to the theory to take into account the nature of learning that is particularly applicable to knowledge evolution. two approaches to definition according to the macquarie dictionary, definition is: [definition, n. 1. the act of defining or making definite or clear. 2. the formal statement of the meaning or signification of a word, phrase, etc. (the macquarie dictionary, 1981)] in defining tacit knowledge we have chosen to present two alternative approaches. the first approach applies formal content analysis of the literature to define tacit knowledge, based on what other authors, the research community, believe to be tacit knowledge. the second approach provides a formal framework or theory for defining tacit knowledge that is based on denotation. following weber (1977), definitions may be justified by both interpretation and representation: 1) interpretation: by agreeing amongst us that the definitions effectively describe and provide a qualitative understanding of the reality and human value systems we are dealing with. if this is satisfied we can assert that the definitions are relevant to our value systems and beliefs. note that “us” and “our” refer to the group of people entrusted with the understanding required. 2) representation: by formalisation in a mathematical theory that has sound and valid models. if representation is satisfied then from a strictly formal perspective, the definitions have a sound underlying theory. the content analysis, interpretation, and formal theory; representation, presented in the two approaches, thus complement each other. what is evident from both of these approaches is that tacit knowledge is heavily individualistic and based on self–experience, which leads ultimately to our greater understanding for situations we will confront in the future. the latter aspect of tacit knowledge ties in directly with polanyi’s epistemology, while the useful nature of tacit knowledge for improving our future understanding of situations ties in with sternberg’s epistemology where tacit knowledge is considered to be a management asset (wagner and sternberg, 1991a; 1991b). approach i: interpretation qualitative content analysis of tacit knowledge in the research literature in previous reported research, we have used formal content analysis to find, by consensus of the research community, a qualitative understanding of tacit knowledge. {busch et al, 2001) a common assertion in tacit knowledge research is that if knowledge is articulated in some way, then it is no longer tacit. while logically this may seem true at first, it is important to note that: …. in theory, tacit knowledge can be verbalised and taught (in which case we still refer to it as “tacit knowledge” even though strictly speaking it is no longer tacit) (sternberg, 1995). ajis special issue december 2002 5 in that research, an examination of 68 recently published documents, the following appeared the most widely cited as explaining what tacit knowledge entails. the terms are as follows, in descending order of groundedness. the terms given are subjectively coded ‘themes’ that have been derived from the literature, rather than direct terms, as they exist per se. we provide only those codes that have a groundedness of greater than 2 instances in the literature here: knowledge (80); individuals (50); organisational domain (46); skill (35); noncodification (28); non-verbal (27); experience (26); context specific (24); intuition (20); learned (16); know how (15); not formal (13); action (12); expertise (11); culture (10); contingency based (9); environment (9); externalisation (9); knowing (9); not easily communicated (9); practical (9); sub-consciousness (9); understanding (9); cognitive (8), internalisation (8); mental models (8); not directly taught (8); not easily transmitted (8); process (8); abilities (7); apprenticeship (7); low environmental support (7); management (7); practice (7); society (7); two dimensional (7); behaviour (6); beliefs (6); conscious (6); direct contact (6); face to face transfer (6); goal attainment (6); inferences (6); learning by doing (6); maxims (6); non-awareness (6); pattern recognition (6); perceptions (6); procedural in nature (6); routine (6); subjectivity (6); tasks (6); technology (6); values (6); common sense (5); decision making (5); embodied (5); implicit (5); implied (5); information (5); judgement (5); no idea (5); not easily codifiable (5); sharing (5); taken for granted (5); unconscious (5); everyday situations (4); interaction (4); job knowledge (4); know more than we can tell (4); not easily formalised (4); not formal instruction (4); others (4); physical control (4); riding a bicycle (4); rule (4); schema (4); time (4); touch sensitivity (4); wisdom (4); abstraction (3); access constraints (3); awareness (3); communal (3); competitive advantage (3); embedded (3); emotions (3); experientially established cognitive structures (3); focal awareness (3); groups (3); holism (3); ideals (3); importance of language (3); information retrieval (3); insight (3); learning by using (3); meaning (3); mind (3); motor skills (3); observation (3); oneself (3); particular uses/particular situations (3); performance (3); practical intelligence (3); procedures (3); resistance to revelation (3); rules of thumb (3); selective comparison (3); semantics (3); sense perception (3); transmission (3). this list is not complete, and a significant number of codes remain that contain a groundedness of 1 and 2 instances in the literature (code total 1,310), which were considered too trivial for inclusion here. note can nevertheless be made from the codes above that tacit knowledge is typically individualistic (50) (beliefs (6); oneself (3)), heavily organisationally based (46), it is directly related at least to skill (35) and it is context specific (24). furthermore it tends to be practically (9) rather than theoretically oriented in nature (practice (7); learning by doing (6); learning by using (3); practical intelligence (3)), and given the nature of human competition, it is acquired in conditions of low environmental support (7), which leads to it’s being used for competitive advantage (3). one other very important issue, often not realised with tacit knowledge is the need for understanding (9) (internalisation (8); others (4); awareness (3); meaning (3); oneself (3)) on the part of the receiver. sveiby (1997) for example, states that “knowledge cannot be described in words because it is mainly tacit … it is also dynamic and static”, and furthermore, “information and knowledge should be seen as distinctly different. information is entropic (chaotic); knowledge is nonentropic. the receiver of the information – not the sender – gives it meaning. information as such is meaningless” (pp.38, 49). in other words, tacit knowledge is not knowledge if the receiver does not understand it. this may help explain why tacit knowledge is so culturally loaded (10) (environment (9); society (7); beliefs (6); values (6); ideals (3); importance of language (3)), and why others of, for example, nesb1 people may not understand immediately what is taking place, even if they do happen to understand the syntax and semantics of english. over time the tacit knowledge component, in addition to the already acquired syntax and semantics, aids in improved communication amongst people. the content analysis has provided a means by which a balance or ‘reality check’ was able to be obtained, in addition to formalising what could be said to comprise tacit knowledge. the definitions provide a view of what many other authors have considered comprises tacit knowledge. the importance in particular of the individualistic nature of tacit knowledge serves, if nothing more, to establish the contextual nature of this knowledge and its reliance upon an individual’s weltanschauung2. the disadvantage of attempting any such form of content analysis is that authors’ definitions often tend to vary wildly and so finding any one ‘true’ definition can be difficult, if not impossible. truth, like the contextual nature of tacit knowledge is finally a subjective assessment. 1 non-english speaking background. 2 philosophy of life or world outlook. ajis special issue december 2002 6 we therefore state, that there exist articulable tacit knowledge properties, and inarticulable tacit knowledge properties. consider the articulable properties that we have selected to exemplify tacit knowledge: {a, … ,n} ⊆ atk where {a … n} is the set of the following articulable tacit knowledge constructs: {abstract high level plans, abstraction, access constraints, all purpose algorithms, analogies, aphorisms, artistic vision, assumptions, behaviour, beliefs, business knowledge, common sense, competitive advantage, complex multi-conditional rules, concepts, constructs, content, contradiction, convincing people, crafts, culture, customer's attitudes, customs, data, decision making, descriptors, discussion, everyday situations, examples can be articulated, expectations, externalisation, face to face transfer, goal attainment, grammatical rules, gut feel, habits, heuristics, hunches, ideals, imitation, impressions, information, information placed in meaningful context eg. message, innovation, interaction, job knowledge, judgement, justified true belief, know how, knowledge base that enables us to face the everyday world, knowledge of designs, logical rules, maxims, meaning, methods, negotiation, observation, perceptions, performance, perspectives, political correctness, practical know how, practice, prescriptive knowledge, principles, private knowledge, procedural in nature, procedures, process, proverbs, reproduction, riding a bicycle, ritual, routine, rule, rules of thumb, schema, script/scripted, semantics, shop lore, stories, subjectivity, swimming, task management, tasks, team coordination, technique, technology, theories, tradition, trial and error, tricks, understanding, understanding of categories, values, way things are done, wisdom} ⊆ atk in other words, the above subset forms tacit knowledge examples that are actually considered articulable, necessarily from a subjective point of view. note once again that we have taken these terms in qualitative fashion from those researchers who have sought to define tacit knowledge to date. furthermore, from our qualitative ‘database’ of tacit knowledge, we are able to identify the following as specifically constituting examples of tacit knowledge that cannot, or rather typically do not, lend themselves to being articulated, what we shall refer to here as inarticulable tacit knowledge itk: {a’ … n’} ⊆ itk where {a’ … n’} include the following inarticulable tacit constructs: {abilities, accidental, accomplishment, action, action oriented know how, action slips, ad hoc, adaptation, after the fact, analysis, application, attention, automatic, automatic knowledge, awareness, background knowledge, between the lines, body language, charisma, concentration, coordination, create and enjoy challenges, diagnostic closure, emotions, executive commitment, exists, experience, expertise, focal awareness, force/tension required, gaining promotion, gaining respect, getting one's feet wet, hands on teaching, have a feeling, here and now, hidden, high level goals, holistic in nature, how to seek out, idiosyncratic, immutable, implicit, implied, indeterminacy, inferences, inferred from actions/statements, informating, ingrained, insight, inspiration, instinctive reaction, intangibility, intimacy, intuition, involuntary, know more than we can tell, know why, knowing, knowledge possessed by itself, learning by doing, learning the ropes, lip service, management, managing relationships, managing subordinates, manual dexterity, meaning requires tacit component, mediation, mental models, meta-cognitive understanding, motivation, motor skills, networking, no idea, noiseless, non awareness, non focus on parts, orientation, out of the corner of the eye, paradigms, pattern recognition, personality, physical control, place, possessed, power, practical intelligence, practice wisdom, preciousness, presuppositions, principles, product of process, proximal knowledge, psychomotor skills, recognition, recognition of musical note, reflection in action, reflection upon reflection, relativity, residual category, rooted, second hand, second nature, see as' rather than see, selective comparison, semiconscious, sense perception, short term, skill, smell, socialisation, society, spatial awareness, spontaneity, sub-consciousness, thinking in practice, tool, touch sensitivity, unanalysed, unconscious, vision, vivid, way things ought to be, weltanschauung, wholeness} ⊆ itk these subsets are selected from busch et al's (2001) formal content analysis. they are hereby presented as identified constituents, which demonstrate the extent of tacit knowledge about any system. ajis special issue december 2002 7 approach ii formal representation of information about tacit knowledge it is apparent that there are many statements that describe what tacit knowledge is in terms of its constituents and components. the first approach above exemplifies the research community's attempts to convey information by making statements about tacit knowledge. we now take advantage of weber's work that has enabled a link to denotational semantics and thus the development of a formal framework for making statements about tacit knowledge. in what follows we use the term information structure to refer collectively to the structure in information system models and constructs. weber’s (1997) work applying bunge's (1979) ontology to information systems defines a basic set of constructs. these weber proposed as necessary and sufficient for us to construct and represent the concepts in the reality we encounter. in short, these ontological constructs provide us a framework for representing the meaning of our conceptual reality. this meaning, to emphasise, is based on and is relative to the bunge ontology as interpreted by weber and wand. we now extend the meaning of a system, as it exists in reality, by adding definitions that make explicit the underlying assumptions of our formalisation of knowledge. in essence, we add the dimension of the human mind by which information and then knowledge is realised. knowledge thus embraces intent, purpose, values and beliefs formed within the human mind by experience and whatever other mechanisms there might be. knowledge therefore is more than the concepts in an immediate encountered reality. furthermore, though we have not explored it in this paper, the underlying philosophy of knowledge should extend to an epistemology so as to embrace learning. in terms of the semiotic ladder (stamper, 1991), the extension goes beyond conventional semantics to embrace pragmatics and belief systems, the higher semiotic levels, by including notions of commitment, intelligent behaviour and wisdom, which are quintessentially of the human mind, and subjective. representation and denotation representation is at the heart of weber's work. representability is central also to the denotational semantics of programming languages. the link between representing the information systems structure of a conceptual reality and denotational semantics proves to be important. previous work by dampney (1998) showed that weber's static and dynamic models3 could be formalised using category theory constructs that turn out to be used also in programming language denotational semantics. since then jacob's (1999) monograph on categorical logic and type theory, which focuses on fibration, a means proposed by colomb, dampney and johnson (2001) for composing information structures, provides substantiating evidence linking the composition of information structures, (c.f. weber's composition model) and computation structures. representability requires that an information system σ and changes in the information system [σ σ] both be representable in a sets category. that both σ and [σ σ] be representable in the same mathematical domain requires a constraint on the changes as expressed by the operator “ ”. the constraint is that the mapping to the sets category satisfies denotation as formalized by denotation semantics (e.g. van leeuwen, 1990). thus a slightly stronger condition than representable, and than weber's formalization in sets and relations implies, is required. this constraint is satisfied if the category satisfies a property called cartesian closure. we defer further discussion of this issue in this context as we change context below to statements about knowledge systems, for which different operators apply, but which are still formalized as a category satisfying cartesian closure and the requirements of denotational semantics. this enables us to express statements about an information system in a formal theory and consequently provide mathematically sound and valid definition. an exercise in formalization we are now in a position to develop a theory within which (our understanding of) knowledge can be formally described and to take into account the dimension of the human mind. we begin formally by declaring the existence of data (d), information (i), knowledge (k), tacit knowledge (tk), and articulable tacit knowledge (atk). the term atk some may feel more comfortable labelling implicit knowledge, and for completeness we also identify inarticulable tacit knowledge (itk) as truly tacit and not able to be passed on through person-to-person interaction. this requires, from a formal denotational semantic perspective, several assumptions about how knowledge may be represented in sets so as to be subject to analysis. we say that there exist sets of whatever constitutes or characterises these various elements of knowledge and that there is a partial order, “⊆“, on the sets which 3 the representation model (static), the state-tracking model (dynamic), and the compositional model. ajis special issue december 2002 8 enables us to say that knowledge objects contain other knowledge objects. mathematically, a partial order defines a lattice (complete partial ordering (cpo)). lattices have proved to be an effective means of representation in formal concept analysis used by knowledge analysts, in computational theory (denotational semantics) and in many other areas. in a style suggested by denotational semantics, we begin by identifying the objects with which we may formulate tacit knowledge in a domain: ∃ d, ∃ k, ∃ i, ∃ tk, ∃ atk, ∃ itk together with the two combinators appropriate to describing knowledge – “∧” (conjunction also called “meet”, “intersection”), and “∨“ (disjunction also called “join”, “union”). the combinators enable us to combine any two objects, say a and b, to form new objects a∧b and a∨b. these objects are governed according to a partial ordering “⊆“ which in our case could be called “containment”. so for example we can say that a∧b ⊆ c. it would be useful if we have a means for expressing implication over sets. overloading our symbolism for the moment suppose b represents some population that satisfies some condition b, similarly for c, and that {b=>c] represents the population that satisfies the condition that if b is satisfied then c is satisfied. not necessarily all of c. we can express this as ([b=>c] ∧ b) ⊆ c it turns out that satisfying denotation requires that we introduce a new operator “=>“4, which we now identify as implication, and which satisfies ([b=>c] ∧ b) ⊆ c. this enables the domain to be cartesian closed, that is, denotable. the denotation constraint is satisfied if for every partial order (a∧b) ⊆ c there is a partial order a ⊆ [b=>c] and vice versa as illustrated by the diagram. [the arrow represents a partial order “⊆”.]. this correspondence proves useful. a b a/\b cb=>c [b=>c]/\b we can now propose definitions concerning the information structure of knowledge, which are derived from assertions made by various experts. these definitions will at least be sound and valid in the formal sense. definition: the tacit components of knowledge there is little argument in stating that articulable and inarticulable tacit knowledge are contained within tacit knowledge, thus: atk ⊆ tk and itk ⊆ tk furthermore we say that (i) atk and itk form disjoint subsets of tk, and (ii) atk and itk do not necessarily form complete subsets of tk. tacit knowledge in its entirety, necessarily includes both inarticulable and articulable knowledge, thus: (atk∨ itk) ⊆ tk furthermore, from previous argument, we define tacit knowledge as comprising a subset of knowledge, thus: tk ⊆ k definition: beyond knowledge choice and intelligent behaviour 4 often denoted by the symbol "⊃". this operator plays the same role as "→" used earlier. ajis special issue december 2002 9 we extend this representation of knowledge to include intelligent behaviour, choice, values, commitment and wisdom within our formalisation. as tuomi (1999/2000) suggests: … when the human mind uses this knowledge to choose among alternatives, behavior becomes intelligent (p.105). we can restate this assertion as knowledge (k) combined with (∧) choice (ch) is contained within (⊆) intelligent behaviour (ib) and represent it formally by (k ∧ ch) ⊆ ib (1) from the denotation constraint we have: k ⊆ [ch =>ib] (2) and, to check consistency, evaluate so that from (1) and (2) (k ∧ ch) ⊆ ([ch => ib]∧ ch) ⊆ ib we can now interpret the formal expression k ⊆ [ch => ib] as “(within our minds) knowledge (k) is contained within (⊆) the implication process [ch => ib] of choice (ch) over alternatives to gain intelligent behaviour.” definition: intelligent behaviour with values and commitment leads to wisdom. tuomi (1999/2000) further asserts: … when values [v] and commitment [c] guide intelligent behaviour [ib], behaviour may be said to be based on wisdom [w] (p. 105). restated, this says that intelligent behaviour with values and commitment is contained within wisdom. representing it formally and subscripting to designate the assertion: ib ∧ v∧ c ⊆ w(tuomi). or equivalently ib ⊆ [v∧ c => w(tuomi).] we see that {ib ∧ v∧ c} ⊆ ([v∧ c => w(tuomi).]∧ v ∧ c) ⊆ w(tuomi). in a deeper interpretation, sternberg (2000) notes that wisdom relies on altruistic principles, as in: … wisdom is defined as the application of tacit knowledge as mediated by values toward the goal of achieving a common good through a balance among multiple interests – (a) intrapersonal, (b) interpersonal, and (c) extrapersonal – in order to achieve a balance among responses to: (a) environmental contexts, (b) shaping existing environmental contexts, and (c) selecting new environmental contexts, over both (a) short – and (b) long terms … [where] … common good refers to what is good in common for all, not just for those with whom one identifies (pp.253, 254). from this interpretation we formally represent wisdom thus: v ∧ c ∧ ib ∧ al ∧ tk ⊆ w(sternberg); equivalently v ⊆ [c ∧ ib ∧ al ∧ tk => w(sternberg);] where al refers to altruism from {ib ∧ v∧ c} ⊆ w(tuomi). above we have , w(sternberg) ⊆ w(tuomi). because v ∧ c ∧ ib ∧ al ∧ tk ⊆ v∧ c ∧ ib } this brings into question whether al∧tk is superfluous, which places sternberg's interpretation in question, or whether al∧tk is required, thus making tuomi's assertion too broad. a resolution is that the notion of wisdom is different in the minds of the two authors. one would expect wisdom relative to altruism (al) and tacit knowledge (tk) to be more focused. definition: data, information and knowledge. we address the concepts of data, information and knowledge, which have been covered at length by many authors. many of these authors emphasis the lack of interconnectedness inherent in data, the contextual nature ajis special issue december 2002 10 of information and the individual belief systems that heavily bias a person’s interpretation of knowledge. zack (1999) presents a useful definition that sums up succinctly these three types of knowledge. it is for this reason that we use it here: data represent observations or facts out of context that are, therefore, not directly meaningful. information results from placing data within some meaningful context, often in the form of a message. knowledge is that which we come to believe and value on the basis of the meaningfully organised accumulation of information (messages) through experience, communication, or inference (bobrow et al. in zack, 1999). knowledge can be viewed both as a thing to be stored and manipulated and as a process of simultaneously knowing and acting that is, applying expertise (blackler, 1995; dretske, 1981; lave, 1988 in zack, 1999). if we take the above definitions as a starting point, then we may say that information is derived from data. it is the useful combination of data with context that provides us with information, the emphasis being placed on useful. the frisco report’s (falkenberg et al, 1998) definition of knowledge as “a relatively stable and sufficiently consistent set of conceptions possessed by single human actors” (p.66), whereas “the term data denotes any set of representations of knowledge, expressed in a language.” in other words, data are meaningful symbolic constructs (expressed in a language), that can be qualified as “knowledge bearing” (p.66); whereas information “is the knowledge increment brought about by a receiving action in a message transfer, i.e. it is the difference between conceptions interpreted from a received message and the knowledge of the receiving action …. an important aspect of information is how a receiver uses it” (p.68). what appears in this instance to be unique about the term tacit knowledge is that if we take the definitions of knowledge and information given above, then tacit knowledge is actually a combination of the two. it is a prerequisite of tacit knowledge that it be understood by the receiver (information) or make sense to the receiver, yet tacit knowledge comprises a set of conceptions or interpretations by human actors (knowledge) or meaning as interpreted by the receiver. we can thus formalise the concept of context (∃ co) and data (∃ d) being contained within information (i) by: [co∧d] ⊆ i context we suggest is close to or equivalent in meaning to weber's (1997) environment. now, as defined above, articulable tacit knowledge and inarticulable tacit knowledge are contained within tacit knowledge. {atk∨ itk} ⊆ tk. presuming that (in)articulable sense and (in)articulable meaning are disjoint subsets of sense and meaning respectively, we may express this concept in the following manner: ∃ ase, ∃ ame, ∃ ise, ∃ ime {ase∨ ise} ⊆ se {ame ∨ ime} ⊆ me where se = sense, me = meaning, ase = articulable sense, ame = articulable meaning, ise = inarticulable sense, ime = inarticulable meaning presuming that articulable sense and articulable meaning comprise articulable tacit knowledge and inarticulable sense and inarticulable meaning also add to the total definition of tacit knowledge, a more complete definition may be seen as the following: (i ∧ se ∧ me) ⊆ k this may be interpreted as “sense and meaning need to be combined with information to have knowledge” and as “ information is contained in knowledge”. furthermore {(ase ∧ ame), (ise ∧ ime), (ise ∧ ame), (ase ∧ ime)} ⊆ k and we now speculatively identify (ise ∧ ime) ⊆ itk; ajis special issue december 2002 11 {(ase ∧ ime) ∨(ise ∧ ame)} ⊆ atk; and (ase ∧ ame) ⊄��. to suggest that the ability to articulate both the sense and meaning of information is required for explicit knowledge. the above definitions are a consistent denotable representation in a formal theory relating the various constituents and associates of data, tacit knowledge, information, knowledge and wisdom. perhaps our major contribution has been to recognise that the elements of knowledge inter-relate within a lattice of containments, suggesting therefore that the elements of knowledge are both inter-dependent and form themselves into overlapping hierarchies. properties as constituents or characterizations of tacit knowledge in the argument presented in the formalisation, we deferred definition of the elements, if any, of what denotes, that is constitutes or characterizes, tacit knowledge. properties are a candidate as they are elemental in both the chisholm (1996) and the bunge ontologies. one may say that ‘shop talk’, ‘work experience’, ‘skills’ and so forth constitute knowledge of properties of systems. the properties may be of the various types identified by weber (1997) from general, particular, intrinsic, mutual, emergent, and hereditary. we denote tacit knowledge (tk) perceived within the human mind η of h5 as (a subset, more strictly a subobject lattice, i of) properties within of a system σ of �6. thus: denote_tk: (� →� → ℘( )) where℘is the powerset symbol and ℘( ) forms a lattice] thus the tacit knowledge tk(σ,η) by a human mind η of a system σ is about properties pj singly or associated by conjunction ∨j pj) and disjunction (∧j pj). thus tk(η,σ) maps to a sublattice i of properties. this means that some properties contain more elemental properties, and this we argue is in keeping with the way we think and reason about our reality. denotation introduces implication and the ability to express a richer set of propositions over the properties. the properties will include constraints as needed to satisfy the ontology of the domain. the properties must finally be attributed and the population of the system satisfying the attributed property determined. how well properties are attributed is a separate issue and this will inherently cause a level of uncertainty in transference of human knowledge. such issues are beyond the scope of this paper. the propositions about properties may now be regarded as statements about the system for knowledge analysis purposes. conclusion we have examined a way to formalise tacit knowledge along the lines proposed by weber (1997) for systems theory in general. as a balance to formalisation we also present some results from a content analysis of the tacit knowledge literature, which reveals themes that best interpret this special type of knowledge. tacit knowledge, at least in practice, encompasses a component that lends itself to eventual articulation. the formalization has been applied to infer a number of conclusions from more basic conjectures derived from assertions in the literature: • without sense, meaning and context properties are just recorded data. with context data becomes information. adding sense and meaning with the human mind provides the means for understanding. by unknown processes, the evidence from experience is that sense and meaning, evolves within our minds, first inarticulable and tacit, then expanding to become articulable and explicit. • knowledge is combined with other intelligence to define wisdom. wisdom involves knowledge, but within a human value system that enables intelligent behaviour and choice. the cultural context, within which tacit knowledge is acquired, therefore bears consideration, as does the competitive nature of the knowledge, which indicates why tacit knowledge is not so easily, or rather readily, transferred. whether or not these conclusions withstand closer scrutiny remains to be seen. but we now assert that the formalization has at least provided a means for closer scrutiny. 5 η of h is a human mind η from amongst all human minds. similarly for σ of σ, where σ is a system and σ is all systems. 6 see previous footnote. ajis special issue december 2002 12 finally at the core of this paper is the conjecture that information space and computational space are governed by the same essential structure. knowledge adds the human dimension that may ultimately be unfathomable, at least by us. references aisbett, j. & gibbon, g. (2001) “a general formulation of conceptual spaces as a meso-level representation” artificial intelligence, 133, pp 189-232 ancori, b., bureth, a., & cohendet, p. (2000) “the economics of knowledge: the debate about codification and tacit knowledge” industrial and corporate change 9(2), pp 255 – 287 arora, a. (1996) “contracting for tacit knowledge: the provision of technical services in technology licensing contracts” journal of development economics, 50(2), pp 233 – 256 bunge, m. (1979) treatise on basic philosophy: vol. 4: ontology ii: a world of systems, reidel, boston. burstein, f. (2001) personal communication of peter busch with prof. frada burstein, monash university busch, p., richards, d., & dampney, c. (2001) “visual mapping of articulable tacit knowledge”, information visualisation 2001: proceedings of the australian symposium on information visualisation, 9, pp 37 47 casonato, r. & harris, k. (1999) “can an enterprise really capture “tacit knowledge”?: we answer two top questions on knowledge management from the electronic workplace 1999 conference” gartner group research note select q&a 16th march chisholm, r.m. (1996) a realistic theory of categories an essay on ontology, cambridge university press. colomb, r. m, dampney, c. n. g. & johnson, m. s. j. (2001) “use of category-theoretic fibration as an abstraction mechanism in information systems” acta informatica, 38, pp1-44 dampney, c. n. g. (1998) “the event as a fundamental construct of information systems”, proceedings of the ninth australasian conference on information systems, pp 120-133 ebeling, r. (1999) the future and its enemies (review of the book by virginia postrel, new york: free press, 1998) url: http://www.fff.org/freedom/0599f.htm falkenberg, e. (1998) “frisco – “mission * impossible”?” in falkenberg, e., hesse, w., lindgreen, p., nilsson, b., oei, j., rolland, c., stamper, r., van assche, f., verrijn – stuart, & a., voss, k. (1998). a framework of information systems concepts: the frisco report (web edition) ifip (international federation for information processing) pp 165 – 170 gärdenfors, p. (2000) conceptual spaces: the geometry of thought. mit press massachusetts u.s.a. greeno, j. (1987) “instructional representations based on research about understanding” in schoenfeld, a., (ed.) cognitive science and mathematics education, lawrence erlbaum associates hillsdale new jersey u.s.a. pp 61 – 88 howells, j. (1995) “tacit knowledge and technology transfer”, working paper no. 16 esrc centre for business research and judge institute of management studies, university of cambridge u.k. september jacobs, b. (1999) categorical logic and type theory, elsevier science, studies in logic and the foundations of mathematics, vol 141 leeuwen, van j. (ed), (1990) formal models and semantics handbook of theoretical computer science volume b. the mit press massachusetts, usa. leonard, d. & sensiper, s. (1998) “the role of tacit knowledge in group innovation” california management review, berkeley, spring macquarie dictionary (1997) published by the macquarie library macquarie library macquarie university australia polanyi, m. (1958) personal knowledge: towards a post critical philosophy, routledge and kegan paul, london u.k. polanyi, m. (1967) the tacit dimension, routledge and kegan paul, london u.k. polanyi, m. (1968) “logic and psychology”, american psychologist 23(1), pp 27 – 43 stair, r. m. & reynolds, g. w. (1998) principles of information systems: a managerial approach 3rd ed, boyd and fraser, danvers mass. u.s.a. stamper, r. (1991) the semiotic framework for information systems research, in nissen, h.-e, klein, h. k. & hirschheim, r. a. (eds), information systems research: contemporary approaches and emergent traditions, north holland, amsterdam, pp515-527. sternberg, r. (1995) “theory and management of tacit knowledge as a part of practical intelligence” zeitschrift für psychologie 203(4), pp 319 – 334 sternberg, r. (2000) “wisdom as a form of giftedness”, gifted child quarterly 44(4), pp 252 – 260 sternberg, r., wagner, r., williams, w., & horvath, j. (1995) “testing common sense” american psychologist 50(11), pp 912 – 927 ajis special issue december 2002 13 sveiby, k. (1997) the new organisational wealth: managing and measuring knowledge – based assets, berrett – koehler, san francisco u.s.a. takeuchi, h. (1998) “beyond knowledge management: lessons from japan”, monash mt. eliza business review 1(1), pp 21 29 tuomi, i. (1999/2000) “data is more than knowledge: implications of the reversed knowledge hierarchy for knowledge management and organizational memory”, journal of management information systems 16(3), pp 103 – 177 (electronic) wagner, r. & sternberg, r. (1991a) tkim: the common sense manager: tacit knowledge inventory for managers: test booklet, the psychological corporation, harcourt brace jovanovich, san antonio u.s.a. wagner, r. & sternberg, r. (1991b) tkim: the common sense manager: tacit knowledge inventory for managers: user manual, the psychological corporation, harcourt brace jovanovich, san antonio u.s.a. weber, r. (1997) ontological foundations of information systems, coopers and lybrand accounting research methodology monograph no. 4, melbourne victoria australia zack, m. (1999) “managing codified knowledge”, sloan management review, 40(4), pp 45-58 using abduction to evolve inconsistent requirements specifications bashar nuseibeh alessandra russo department of computing, imperial college of science, technology and medicine 180, queen's gate, london sw7 2bz, u.k. email: (ban, ar31@doc.ic.ac.uk abstract requirements specifications are often inconsistent. inconsistencies may arise because multiple conflicting requirements are embodied in these specifications, or because the specifications themselves are in a transient stage of evolutionary development. in this paper we argue that such inconsistencies, rather than being undesirable, are actually useful drivers for changing the requirements specifications in which they arise. we present a formal technique to reason about inconsistency handling changes. our technique is an adaptation of logical abduction adapted to generate changes that address some specification inconsistencies, while leaving others. we represent our specifications in quasi-classical (qc) logic an adaptation of classical logic that allows continued reasoning in the presence of inconsistency. the paper develops a sound algorithm for automating our abductive reasoning technique and presents illustrative examples drawn from a library system case study. introduction a key problem in large software engineering projects is the management of frequently changing requirements. requirements changes have a particularly significant impact on the consistency of specifications. changes may introduce inconsistencies; and conversely, requirements changes may be necessary to handle existing inconsistencies. therefore, the ability to handle inconsistent requirements is crucial to the successful development of requirements specifications. researchers have developed a variety of techniques for analysing and managing the impact of changes on systems and their specifications (bohner & arnold 96). however, the starting point of much of this work is that a consistent artifact exists, which will be changed while preserving its consistency. the motivation is based on the intuitive assumption that inconsistencies are undesirable, given the difficulty of deriving useful information from inconsistent specifications. consequently, the techniques developed are based on rigorous consistency checking and analysis, in an attempt to eradicate inconsistencies as soon as or soon after they are detected. however, in practice, inconsistency is inevitable in real large-scale specifications (balzer 91; cugola et al. 96; schwanke & kaiser 88). living with inconsistency during evolutionary development is a fact of life. therefore, inconsistency handling mechanisms are needed to support incremental evolution of specifications. by this we mean identifying changes that address some specification inconsistencies, while leaving others (finkelstein et al. 94). in this paper, we provide a formal technique for handling inconsistencies that (a) allows such an incremental evolution of (inconsistent) requirement specifications and (b) can be implemented by exploiting existing tools for handling theory change (kakas & mancarella 90, selmen & levesque 90). requirements specifications are, in our approach, partial specifications (possibly developed by different users) related to each other by means of pre-defined "consistency rules" (nuseibeh et al. 94). each partial specification may or may not include (logical) inconsistencies. the overall specification is, in our approach, defined to be "inconsistent" when at least one of the pre-defined rules is violated. this violation can be detected by checking if the specification satisfies the negation of the rule. defining inconsistency in this way allows more flexibility to express a variety of constraints on specifications. it subsumes the notion of logical inconsistency, which is not always the most convenient way of representing the violation of constraints in real-life specifications. the inconsistency handling technique described in this paper is based on a specific type of formal reasoning, called abduction (kakas et al. 98). if a particular consistency rule is violated, our proposed abductive reasoning technique identifies (evolutionary) changes to perform on the specification, such that a particular consistency rule is no longer violated. the partial specification, as well as the consistency rules, is assumed to be expressed in quasi-classical (qc) logic (hunter & nuseibeh 97; hunter & nuseibeh 98) an adaptation of classical logic that allows reasoning in the presence of inconsistencies without trivialisation5. to provide a route towards implementing our technique, we have also developed an algorithm that translates specifications expressed in qc logic into logic programs. this enables us to deploy existing tools for abductive logic programming (kakas & mourlas 97). as shown in the paper, the results given by such tools can be translated back into the qc representation of the specifications, thus providing an automated way of generating changes to resolve inconsistencies. to summarise, this paper uses abduction to support incremental evolution of inconsistent requirements specifications. the novelty of the work is in adapting existing results on abduction for consistent specifications and applying them in the more realistic setting of inconsistent specifications (section 3). the notions of 5 trivialisation is the inference of arbitrary information from an inconsistent specification. 118 inconsistency and abduction are formally described in section 2. the abductive technique developed (section 3) is tailored to make use of existing automated tools, and the paper provides a sound translation mechanism to logic programs in order to achieve this (section 4). a complete illustrative example (section 5) and a discussion of related work and open research issues (sections 6 and 7, respectively) conclude the paper. inconsistency and abduction in this section, we provide an overview of our notions of inconsistency and abduction, which we use in the remainder of this paper as the basis of our approach. inconsistency we base our notion of inconsistency on the violation of consistency rules. we treat such consistency rules as "properties" that should be satisfied by a given requirement specification. within this context, specifications are considered to be inconsistent if they satisfy the negation of some of these rules (e.g., because of overspecification). more formally: inconsistency. given a specification s and a set sr = {r], r2, ... rn} of consistency rules, s is inconsistent if there exists at least one rule rj, for some 1 < i < n, such that s " -irj, where " denotes any underlying formal reasoning mechanism. for any violated consistency rule rj, we say that the specification s is ^-inconsistent. inconsistency handling is the process of making changes that result in particular consistency rules no longer being violated; that is, s' z -iri, where s' is the new (evolved) specification. since the changes do not guarantee the new specification to be consistent (i.e. some other consistency rules might still continue to be violated), in this paper we represent our specifications in qc logic, which allows continued non-trivial reasoning in the presence of inconsistency. abduction abduction is one of the three fundamental modes of reasoning, the others being deduction and induction. the most popular formalisation of abduction in ai defines it as the search for a set of hypotheses (an "explanation") that, combined with a given theory, achieves some given goals (an "observation") without causing contradictions (kakas et al. 98). this type of inference procedure has been shown to be suitable for addressing different kinds of problems in ai, such as diagnosis (console et al. 96), planning (esghsi 98), and database updates (console et al. 94; kakas & mancarella 90; inoue & sakama 95). recent work has also shown its application in knowledge-based software engineering (menzies 96). using abduction for theory change (e.g., database updates), the observation is a request for a particular change (update), and abduction is the process of identifying the changes to be made to a given theory so that the new theory satisfies the request (kakas & mancarella 90; inoue & sakama 95). a request can be, for example, a requirement for information to be a consequence of a given theory, or for it to be no longer inferred from a given theory. in the first case, abduction provides an explanation of the given information, whereas in the second case, it provides an anti-explanation (inoue & sakama 95). formally: abduction. let l be a given logical language and let s be a theory written in l. an abductive framework is a pair (s, ab) where ab is a set of abducibles, also written in l. let r be a request6. a pair (a+, a"), where both a* and a~ consist of ground instances of elements from ab, is an explanation of r in the abductive framework <s, ab) if • (sua + ) \a"~r • (s u a+) \ a~ is consistent where the symbol \ denotes the standard operation of subtraction between sets. a pair (a+, a~) is an anti-explanation of r in the abductive framework {s, ab) if • (sua + ) \a'xr • (s \j a+) \ a~ is consistent in the above definition, the sets a+ and a~ denote information that needs to be added and deleted, respectively, from a given theory in order to meet a request. notice that, whenever the underlying logic is monotonic7, it will always be the case that explanations (a+, a~) will have a" = 0, whereas anti-explanations will have a+= 0. 6 while our definition of abduction uses the term 'request' informally, in this paper we use it specifically to denote the request that a (consistency) rule, r, is no longer violated. 7 a reasoning mechanism is monotonic if the addition of new information to a specification preserves the set of derivable information (i.e. consequences). 119 an abductive technique for handling inconsistency a fundamental premise of standard abduction as defined in section 2 above is that we start and end with a consistent specification. in the real world, this is not always the case. to be of practical use, abductive reasoning should facilitate the identification of explanations and anti-explanations of a given request within inconsistent specifications, without leading to trivialisation. this can be achieved by deploying formal reasoning that allows continued reasoning in the presence of inconsistency. we have therefore considered qc logic an adaptation of classical logic that allows non-trivialised reasoning about inconsistent specifications and adapted classical abductive reasoning to handle inconsistent specifications expressed in this logic. in this section, we briefly review qc logic and we then describe our abductive framework for handling inconsistencies, giving some illustrative examples. an overview of qc logic qc logic adapts the proof theory of classical logic to allow the inference of literals or clauses in the presence of inconsistency. literals are formulae of the form a and -id, where a is a ground atomic formula. clauses are formulae of the form a, v cc2 v ...v a,,, with n > 2, where <xj is a literal, for each 1 < i < n. to illustrate qc reasoning, consider the following partial specification8, s, of the london ambulance system (las) (finkelstein & dowell 96): (i) accident -»medicaleemergency (ii) [call a medicalemergency a nearestambulanceavailable] —» dispatchambulance (iii) [call a -inearestambulanceavailable] -» -idispatchambulance (iv) -.hascrew -»-inearestambulanceavailable (v) -inearestambulanceavailable —> -jmocalllssue together with the following scenario: (vi) accident (vii) call (viii) -ihascrew (ix) nearestambulanceavailable it is easy to see that the scenario and specification, when combined, are inconsistent, because we can generate the two contradictory pieces of information: dispatchambulance (using (i), (ii), (vi), (vii) and (ix)) -idispatchambulance (using (iii), (iv), (vii) and (viii)) however, using qc logic, we can still continue reasoning with the above specification and generate additional non-trivial information such as: -jmocalllssue (using (iv), (v) and (viii)) so, even though the specification and scenario are inconsistent, we can still generate other useful inferences. in classical logic, it is always the case that either a piece of information or its negation is true. this link between a piece of information and its negation is decoupled in qc logic. any formula and its negation could equally be evaluated to be true within a qc model. this is due to the fact that satisfiability of clauses in a qc model is defined in terms of two separate notions of satisfiability, called strong satisfiability and weak satisfiability, depending on whether the clause is part of a given specification or is derivable from the specification, respectively. weak satisfiability reflects the notion of truth of classical disjunction (i.e. a, v a2 is weakly satisfied if either cti is true or a2 is true). strong satisfiability complements this definition with extra conditions. a clause cc] v <x2 is defined to be strongly satisfied in a model x if (a) either tti is true or oc2 is true, (b) if —>a} is true then oc2 must be true, and (c) if -i<x2 is true then ct| must be true. the extra conditions (b) and (c) allow for sound resolution steps in the qc reasoning process9. the notion of semantic entailment is also given in terms of strong and weak satisfiability. a formula is semantically entailed from a qc specification if it is weakly satisfied in all models that strongly satisfy the specification. a full account of qc logic, its semantics and proof theory is described in (hunter & nuseibeh 98). 8 for simplicity, clauses are represented in their equivalent implication form. 9 resolution is the inference of a fact p from the assumptions (ccvp) and —id. 120 abduction for qc logic we now describe how to adapt the notion of classical abduction, as defined in section 2.2, in order to develop an abductive technique for inconsistency handling. within our approach, theories are requirements specifications that are r-inconsistent for some consistency rule r. resolving an r-inconsistency means identifying the changes that need to be made on the inconsistent specification so that the negation of the rule r is no longer inferred from that specification. a request for an r-inconsistent specification is a request to delete the negation of the rule r from the set of consequences10 of the specification. our abductive technique facilitates the identification of changes (a+, a~) to make on the specification so that such request is satisfied; i.e. so that the rinconsistency is resolved. since the underlying qc logic is monotonic, these changes (anti-explanations) will only include information to be deleted from the given specification (i.e. the set a~). our approach assumes that specifications are expressed in qc logic. consistency rules are also represented in this logic and therefore can be either literals or clauses. consequently, the negation of consistency rules can only be literals, or conjunctions of literals. in this context, resolving an inconsistency means abducing antiexplanations of some of the literals that compose the negation of the violated consistency rule r. suppose that cti v ot2 is a rule violated in a given specification s. both the literals —i(xi and —icc2 can be inferred from s. to resolve this inconsistency, one of these two literals needs to be removed from the consequences of s. assume -itti is chosen. this literal can be derived from s either because it is already part of the specification, or because it has been inferred using resolution, or both. in the first case, our abductive technique identifies the literal itself as an anti-explanation; the changes needed will then be the deletion of this literal from the specification. in the second case, our abductive technique identifies, as an anti-explanation of -i<xi, literals involved in the inference process (i.e. resolution steps) that have lead to the derivation of -a\. changes will then be the deletion of (some of) these literals. in the third case, our abductive technique abduces as an antiexplanation both the literal -m\, which is in the specification, and any other literal which allows the derivation of-ict] from other assumptions in the specification. so, for the particular request of deleting a specific literal from the consequences of a specification, our qc abductive technique identifies "relevant" literals in the specifications, which, once deleted, satisfy the request. the set of abducibles is therefore given by the set of literals included in the qc specification. formally: qc abduction. abduction for qc logic is a pair (s, ab) where s is a qc specification and ab is the set of literals that appear in s. let r be a violated rule. an anti-explanation of -ir is a set a" of elements from ab, such that the new specification s' = (s \ a") z -ir. note that the condition of consistency given in the definition of "standard" abduction (section 2.2) is no longer necessary within our framework. this is because the logic itself allows for inconsistencies. if any change to a given qc specification introduces logical inconsistencies, the resulting specification would still be a valid qc specification. the abductive process for handling inconsistencies in a qc specification is essentially a backwards reasoning mechanism. suppose that a literal a needs to be removed from the consequences of a given qc specification in order to resolve an r-inconsistency. the abductive process reasons backward from all the resolution steps that have lead to that literal. if this backwards reasoning is "successful" (i.e. it reaches some relevant literals that are in the specification), then the identified literals become the abducible anti-explanation of the initial literal a. since qc logic is monotonic, the changes will then be deletion of such literals. also, since the abductive procedure is essentially used for removing literals that are proved from a given specification (as they are part of the inconsistencies), the backward reasoning will always succeed giving one or more answers. within a more general setting, for example, in which the abductive reasoning is used for performing arbitrary (evolutionary) changes to a given specification, the abductive process might not always provide an answer, for example in the case when the (evolutionary) change requests are not relevant to a given specification. example. to illustrate our abductive process, consider the following simple example. consider the qc specification s of the las, given in section 3.1, and the following consistency rule r: r: dispatchambulance v -idispatchambulance recall that: s vqc dispatchambulance and s nqc —idispatchambulance 10 consequences denote derivable information from a specification. 121 that is, the consistency rule r is violated. resolving this r-inconsistency means eliminating one of the above two literals. suppose that we want to eliminate: dispatch ambulance the backward reasoning identifies all possible ways of proving this literal from the given specification. in particular, it looks for disjunctive clauses that include this literal. for example, one possible disjunct is: -icall v -imedicalemergency v dispatchambulance the next reasoning step would then be looking for the proof of the literals call and medicalemergency since these, together with the above clause, would have inferred dispatchambulance. the backward reasoning is then applied again, but this time on either of the two literals call and medicalemergency. changes that make either of these two literals no longer inferable from the specification would equally resolve the given inconsistency. if we were interested in identifying all possibly changes, the backward reasoning would have been applied on both the two literals call and medical emergency and the resulting abduced information considered as alternative changes. in this example we choose to apply backward reasoning on the clause: medicalemergency the only clause including this literal is: -.accident v medicalemergency the next reasoning step would then be looking for the proof of the literal: accident since this, together with the clause under consideration, would have inferred medicalemergency. the backward reasoning is now applied to the literal accident. there is no clause in the specification that includes this literal, but the literal itself is part of the specification. the backward reasoning then succeeds on this literal and the abductive procedure gives the set: a" = {accident} as an answer. deleting this literal from the specification would indeed stop the inference of the initial literal dispatchambulance and therefore resolve the detected r-inconsistency. extensions of this process can be defined that would consider the abduction of disjuncts as well as literals. it is also important to note that the abductive process becomes more complex the larger the specification. the backward reasoning needs to branch on each possible way of proving a certain clause and/or literal. the final anti-explanation is, in general, given by the union of all the anti-explanations constructed in each individual successful branch". however, our approach to implementation suggests promising ways of tackling this problem. towards an implementation to implement the abductive technique described in the previous section, we have chosen the strategy of using (where possible) existing approaches and tools for abduction. in particular, many abductive procedures have been developed for logic programming (kakas el al. 98), together with associated tool support systems (kakas & mourlas 97). to deploy such techniques and tools, we propose a procedure for expressing any given qc specification, as well as any violated rules, as a logic program. existing abductive logic programming (lp) tools (kakas & mourlas 97) can then be used to identify, for each of the violated rules, anti-explanations in logic program form. these anti-explanations, mapped back into the qc framework, would then provide us with qc anti-explanations, and therefore with the changes that need to be made on our specifications to resolve rinconsistencies. our procedure for generating logic programs from a given qc specification consists of two main steps. the first step translates a given qc specification into a first-order theory, using the translation method described in section 4.2, and the second step generates a logic program from the resulting first-order theory. the intermediate translation step preserves all the semantic and proof theoretic features of qc logic. our method identifies a set of "axioms" in classical logic that correspond to the semantic notions of strong and weak satisfiability of qc logic mentioned in section 3.1. these axioms together with a first-order translation of a given qc specification derive, within standard first-order logic, the same set of consequences that are derivable from the qc specification using the qc reasoning process. the use of this intermediate step also facilitates general automated reasoning about a given (inconsistent) specification, not necessarily related to the inconsistency handling process. for this purpose, we have developed a theorem prover for qc logic, written in prolog, which takes advantage of this translation using the correspondence theorem stated in section 4.2. before describing the algorithm in detail, we briefly introduce some useful basic notions about abduction for logic programming. " alternative changes are given by the anti-explanations constructed from different branches. 122 abduction for logic programming given a first-order language l, a logic program12 p is any theory written in l composed of facts and rules. facts are ground atomic predicates and rules are formulae of the form p <ab a2, ..., a,,, where p and oj, for each 1 < i < n, are also atomic predicates. the formula p <<x|, (x2, ..., a,, is read as "p is true if cti and ... and on are true". an example of a logic program p is: medicaleemergency(person,location) 4accident(person,location). accident(john.londonroad). logic program goals can be either (ground) atomic predicates or the "negation" of (ground) atomic predicates; e.g., not medicaleemergency(person,location) (where not denotes negation by failure (kowalski 79)). informally, a logic program proof procedure is based on resolution steps between (sub)goals and rules, or between (sub)goals and facts. a goal a is satisfied if either a is in the program or there is a proof of a. analogously, a goal a is not satisfied if there is no proof of a and it is not stated in the program as a fact. abduction for lp can be used to remove information from the set of consequences (successful goals) of a given logic program. this is achieved by considering the negation of a goal under consideration and abducing changes among the facts of a given logic program, which satisfy a set consistency constraints and which would make such new negated goal provable from the program. using negation by failure, the success of such a negated goal implies that the original goal is no longer provable from the program. a little example is given here, where the logic program and request are: lp program: request: p <b. delete(p). q<bl. b. bl. the abductive procedure starts with checking if the negation of p, denoted by p* (where the asterisk (*) denotes classical negation), can be proved from the program. this is because deleting p is equivalent to having p* true. because the program does not include any rule about p*, p* is abduced as a first piece of information. for consistency checking, the procedure verifies that p is not provable. this means asking that b is not provable (because of the first rule in the program), which itself means asking that b* is provable. since there is no rule about b*, the abductive procedure abduces also b*. consistency checking on this new (abduced) assumption verifies that b is not provable. this checking succeeds since the procedure has (abduced) b*. the abductive procedure stops, giving as result the set of abduced changes a = {p*, b*}. therefore, the changes are the deletion, from the program, of the facts marked with an asterisk and the addition of the facts without an asterisk. making these changes in the above program results in p no longer being provable (as requested). implementing qc abduction using logic programming as mentioned earlier, our procedure for generating logic programs from a given qc specification consists of translating first the given qc specification into a first-order theory. this intermediate step of translation provides two main advantages: (a) it enables us to reasoning about an inconsistent specification, independently from the abductive process, and (b) it allows us to consider a simple version of logic programs without classical negation or/and disjunctive clauses. translating qc logic into first-order logic. the basic idea is to provide a set of "axioms" in classical logic that correspond to the semantic notions of strong and weak satisfiability of qc logic described in section 3.1. these axioms together with a first-order translation of a given qc specification derive, within standard firstorder logic, the same set of consequences that are derivable from the qc specification using the qc reasoning process. let lfql be a first-order language composed of a set of term symbols, given by the whole set of qc formulae, two predicates holdss and holdsw, and the set of classical connectives. the underlying semantics is the 12 note that different version of logic programs have been developed in the literature. in this paper we refer to the basic notion of logic programs without classical negation or disjunctive rules (see [kowalski 79] for further details). 123 standard classical semantics. an example of a ground atomic first-order formula is holdss(ambulanceavailable). this can be read as "the qc ground atomic formula ambulanceavailable is strongly satisfied in some qc model". analogously, the first-order formula holdsw(ambulanceavailable) can be read as "the qc ground atomic formula ambulanceavailable is weakly satisfied in some qc model". for the case of any qc literals a, the first-order formulae holdss(cc) and holdsw(cc) are equivalent. this is captured by axiom (axl) in definition 2 given in appendix, and reflects the qc logic property that strong and weak satisfiabilities are equivalent when applied to literals. any qc specification is translated into a first-order specification by defining, for each formula a in the qc specification, a ground atomic formula holdss(ex). for instance, the qc specification, s, given in section 3.1 is translated into the following first-order specification, sfol: holdss(accident —» medicaleemergency) holdss((call a medicalemergency a nearestambulanceavailable)-»dispatchambulance) holdss((call a -inearestambulanceavailable)—» -idispatchambulance) holdss(-ihascrew —> -inearestambulanceavailable) holdss(-inearestambulanceavailable —»-rnocalllssue) every qc formula a that is inferred from a given qc specification s is translated into the ground atomic formula holdsw(a). since the inference of a negated consistency rule, -.r, in qc logic is equivalent to the inference of a (conjunction of) literal(s), the first-order translation of a negated rule is also equivalent to the inference of the (conjunction of) atomic formula(e) of the form holdsw(oc), where c^ is the converse of a literal a appearing in -<r (see formal definition in the appendix). for example, the translation of the negation of the consistency rule: dispatchambulance v -idispatchambulance is given by: holdsw(-!dispatchambulance) a holdsw(dispatchambulance) since holdss and holdsw are equivalent when applied to literals, translations of negation of consistency rules can also be expressed in terms of conjunctions of holdss predicates. we will denote the first-order translation of any qc specifications s with the symbol t(s), and the first-order translation of the negation of any violated rule r with t(-.r). to simulate qc reasoning, we have defined a set of axioms, denoted by axqc, for the two predicates holdss and holdsw, that corresponds to the qc semantic definitions of strong satisfiability and weak satisfiability. a formal definition of axqc is given in the appendix. theorem 1 below shows that this set of axioms guarantees that any information is inferred from a qc specification s if and only if its first-order translation is classically inferred from the first-order translation of s. for instance, in the example described in section 3.1, medicalemergency is inferred from s together with the fact accident. using the axioms, it is easy to show that the first-order formula holdsw(medicalemergency) is also inferred from sfql together with the translated fact holdss(accident). theorem 1 (correspondence): let s be a qc specification and let t(s) be its first-order translation. for any qc formula a, a is provable from s if and only if holdsw(a) is provable from the first-order translation t(s) and the axioms axqc. this is, ssqc« <=> t(s),/urge >ol holds w(a) a proof of this theorem is outlined in the appendix, and described in detail in (russo & nuseibeh 98). we have also developed a theorem prover for qc logic to facilitate reasoning over qc specifications. this is written in prolog and takes advantage of the classical translation of qc specifications into first-order logic, using the correspondence theorem above. generating a logic program. the basic idea of this procedure is to construct from the first-order translation of a given qc specification a logic program that includes (a) the translated specification (as facts) and (b) rules, which allow the simulation of qc inferences from the specifications. the logic program resolution mechanism thus captures the resolution mechanism of qc logic. the resulting logic program also includes some integrity constraints to preserve its own consistency. suppose that the first-order translation of a given qc specification is the following set: sfol= {holdss(a,), holdss(a2), ..., holdss(an)( 124 where cq are either literals or clauses appearing in the qc specification. suppose also that the largest clause in the qc specification has m literals (i.e. size m), with m > 2. the generation of a logic program from sfol is an m-step process. each step constructs a partial logic program that extends the one generated in the immediately previous step. the construction of such a partial program is as follows: step 1 a partial program, called pi, is constructed by considering just the set (holdss(ai), holdss(cc2),..., holdss(an)}. step 2 a partial program p2 is constructed by considering each holdss(a) in p|, where a is a qc clause with size m, and adding to pj a rule of the form holdss(p)<-holdss(-iocj),holdss(<x) for each literal otj in a, for which holdss(-ioj) is in p!. the term p is the qc clause obtained from a by cancelling the literal oj. if holdss(-ioti) is not in p] for all oh in a, then holdss(ctj) is added to pi for some otj in a. step 3 a partial program p3 is constructed in a way similar to that shown in step 2, by considering all the formulae holdss(cc), where a is a qc clause of size m-1, that are in p2 or for which there is a rule in p2 with holdss(a) in its left side. step k for each k, 4 < k < m, a partial program pk is constructed in the same way as shown in step 3, by considering the clause of size (m-k)+2 that are in pk.i. the final program p is given by the program pm. for qc specifications with no clauses, but only literals, the logic program p is simply given by the set of holdss(oj), for each literal oj included in the specification. we illustrate the above algorithm with an example. example. let a qc specification be the set s = { pvqvr, -ip, -.q, tvs }. we generate a logic program for this specification in the following way. first we define its first-order translation sfolthis is given by the set: sfol = (holdss(pvqvr), holdss(-,p), holdss(-,q), holdss(tvs)} then we apply the algorithm described above to generate, from this first-order specification sfol and the classical axioms simulating qc reasoning, the associated logic program p. the construction of p is composed of 3 steps, since the largest term that appears in sfol has size 3. the final program p is given by pa below. step 1. the program pj is just the set: {holdss(pvqvr), holdss(-,p), holdss(-,q), holdss(tvs)}. step 2. the program p2 is as follows: p, u { holdss(pvr) <holdss(-,q),holdss(pvqvr). holdss(qvr) <holdss(-,p),holdss(pvqvr). } step 3. the program p3 is as follows: p2 u { holdss(r) <holdss(-,p), holdss(pvr). holdss(r) <holdss(-,q), holdss(qvr). holdss(t). } given a consistency rule r = -j, s derives the negation of this rule, (i.e. s vqc r). so, s is r-inconsistent. to resolve this inconsistency, we apply the abductive procedure for logic programs to the above program considering as request the deletion of hoidss(r)13. the procedure starts then with checking if holdss*(r) is provable from the program. since holdss*(r) cannot be proved, it is assumed as hypothetical information, i.e. aq= (holdss*(r)}. consistency for holdss*(r) is then checked. specifically, the program assumes holdss*(r) and then verifies that holdss(r) fails. with respect to the first rule in the program, holdss(r) fails if either holdss(-ip) or holdss(pvr) fails. suppose that the procedure checks if holdss(-ip) fails. this means checking if its negation, holdss*(-ip), succeeds. since holdss*(-ip) cannot be proved, it is added to the set of hypothesis, a] = (holdss*(r), holdss*(-ip)}. the procedure now uses this set of hypotheses in trying to verify that holdss(r) fails also with respect to its second rule in the program. for holdss(r) to fail, it is necessary that either holdss(-iq) or holdss(qvr) fails. suppose now that the procedure checks if holdss(qvr) fails. using its rule, this means checking that either holdss(-ip) or holdss(pvqvr) fails. since holdss*(-ip) is in the set of hypotheses, a], then holdss(-ip) fails directly. hence, the result of the abductive procedure is the set (holdss*(r), holdss*(-rp)}, which correspond to the changes: (delete holdss(r), delete hoidss(-,p)}. 13 since the request is about deleting a consequence, its translated representation should really be holdsw(r). but because weak and strong satisfiability in qc logic are, for atomic literals, equivalent, the above translation holdsw(r) is also equivalent to holdss(r), as illustrated by the axiomatisation given in appendix. 125 because holdss(r) does not belong to the set of facts in the program, only the second piece of information, holdss(-ip), is deleted. the above abductive procedure for logic programs follows a similar backwards reasoning mechanism to that outlined in section 3.2. it is, however, important to formally prove that it does indeed capture the intended result of resolving an r-inconsistency in a given qc specification. the following theorem states this, showing that our algorithm for generating a logic program from a qc specification and the above abductive procedure for logic programs, are sound with respect to our notion of abduction for qc specifications. theorem 2 (correctness): let s be a specification and let r be a violated consistency rule (i.e. s 'qc -.r). let pbe the associated program and t(-ir) the translation of the negated rule. let t(a) be the set of hypothetical changes on the program p determined by the abductive procedure on p. then: ifp\t(a)xt(.r) then s\axqc-,r. a proof of this theorem is outlined in the appendix. example we now illustrate our approach on a partial specification drawn from a library system. the requirements are that such a system should allow individuals to borrow books from the library if they need these books and if the books are available. once a book is borrowed, it is no longer available for others to borrow. a representation of this specification in qc logic is: sqc = {(bookcopy a booklnlibrary) —» bookavailable, (bookcopy a bookneeded a bookavailable) —> borrowingbook, bookcopy, booklnlibrary, bookneeded } with rqc = borrowingbook —> -.bookavailable as a consistency rule. this specification is inconsistent. the negation of the rule rqc. given by —.rgc = borrowingbook a bookavailable, can be inferred from the specification, sqc. resolving this inconsistency means eliminating one of the above two literals from the set of consequences of the specification. to eliminate the literal bookavailable, abduction for qc logic identifies two possible sets of changes: {delete bookcopy} and (delete booklnlibrary). either of these two changes, if performed on the specification, would resolve the inconsistency. next, we show how this abductive technique is performed by our implementation. first the qc specification, sqc, is translated into a first-order specification, sfol: sfol = ( holdss((bookcopy a booklnlibrary) -» bookavailable), holdss((bookcopy a bookneeded a bookavailable) -» borrowingbook), holdss(bookcopy), holdss(booklnlibrary), holdss(bookneeded) ) the translation of the negated rule (i.e. t(->rqc)) is holdss(borrowingbook) a holdss(bookavailable). applying the algorithm for generating a logic program from sfol we get: ps = { holdss((bookcopy a booklnlibrary) -»bookavailable). _ holdss((bookcopy a bookneeded a bookavailable) —> borrowingbook). holdss(bookcopy). holdss(booklnlibrary). holdss(bookneeded). _ holdss((bookneeded a bookavailable) —» borrowingbook) <— holdss(bookcopy), holdss((bookcopy a bookneeded a bookavailable) -» borrowingbook). holdss((bookcopy a bookavailable) —> borrowingbook) <— holdss(bookneeded), holdss((bookcopy a bookneeded a bookavailable) -» borrowingbook). holdss(bookinlibrary —» bookavailable) <— — holdss(bookcopy), holdss((bookcopy a booklnlibrary) -» bookavailable). holdss(bookcopy -» bookavailable) <holdss(booklnlibrary), holdss((bookcopy a booklnlibrary) -> bookavailable). holdss(bookavailable —» borrowingbook) <— holdss(bookneeded), holdss((bookneeded a bookavailable) —> borrowingbook). 126 holdss(bookavailabie -»borrowingbook) <holdss(bookcopy), holdss((bookcopy a bookavailable) —> borrowingbook). holdss(bookavailable) <hoidss(booklnlibrary), holdss(bookinlibrary -»bookavailable). holdss(bookavailable) <hoidss(bookcopy), holdss(bookcopy -> bookavailable). since holdss(bookavailable) is derivable from the above program ps, our abductive goal is to delete it. the abductive procedure considers the negated query holdss*(bookavailable), and reasons backwards performing the associated consistency checking. the procedure generates two alternative sets of (smallest) antiexplanations: a, = { holdss*(bookavailable), holdss*(bookcopy) } and a2= { holdss*(bookavailable), holdss*(bookinlibrary) } these two sets generate, respectively, two alternative sets of changes: {delete holdss(bookcopy)} and {delete holdss(booklnlibrary)} performing the change in either of these two sets on the above program, would make holdss(bookavailable) no longer provable from ps. these two sets, mapped back into qc logic, give the two sets of changes {delete bookcopy} and {delete booklnlibrary}, which were also identified by the abductive technique for qc logic. related work a number of logic-based approaches for handling inconsistency have been proposed in the literature (e.g., (ghose 99). none, however, appear to tolerate inconsistencies explicitly. zowghi and offen suggest belief revision for default theories as a formal approach for resolving inconsistencies arising during the evolution of requirements specifications (zowghi & offen 97). change actions are implicitly given by the definition of a belief revision operator, which basically changes the status of information from defeasible to non-defeasible or vice-versa, to remove the derivation of inconsistencies. similarly, ryan defines ordering relations on default information (ryan 93). conflicting defaults are resolved not by changing the specification but by considering only scenarios or models of the inconsistent specification, which satisfy as much of the preferable information as possible (in order of preference). anderson and durney on the other hand, propose "relative utility" analysis for guiding the choice among possible alternative changes in order to resolve a detected inconsistency (anderson & durney 93). changes that "enable" scenarios showing "desirable transitions" are preferred to changes that do not, and changes that "disable" scenarios showing "prohibited transitions" are also preferred to those that do not. such heuristics can also be interpreted as preferring changes that do not cause additional inconsistencies to those that do. a closely related logic-based approach is proposed by van lamsweerde et al. (van lamsweerde and letier 98; van lamsweerde et al. 98). in their work they offer a goal-driven approach to requirement engineering in which "obstacles" denote parts of a specification that lead to a negated goal, van lamsweerde's notion of goals is comparable to our notion of consistency rules, and his notion of obstacles corresponds to the abduced facts detected by our abductive technique. both approaches guarantee that changes performed on abduced information or obstacles would resolve a detected inconsistency. the main difference, which is partly due to the differing logical representations used, is that whereas van lamsweerde et al. adopt goal regression to identify possible obstacles, we use abduction. since we also use qc logic as the underlying formal reasoning, it is not necessary to resolve all inconsistency in order to continue to do meaningful reasoning. duffy et al. also propose a logic-based approach for reasoning about requirements specifications based on the construction of goal tree structures14 (duffy et al. 95). however, their approach is less systematic, in that analysis of alternative changes is carried out by investigating which goals would be satisfied and which would not, after adding or removing facts to a specification. recent work by menzies has also demonstrated the applicability of abductive reasoning in knowledge-based software engineering, using an inference procedure for "knowledge-level modeling" that can support prediction, explanation, planning, etc. (menzies 96). however, the 14 this approach is similar to logic-based model checking. however, model checkers (such as spin [holzmann 97]) differ from our tools in that they assume (and require) a (logically) consistent specification as input before they can analyse it. 127 technique is tailored for specifications represented as and-or graphs, where vertices are atomic facts and edges are causal relationships between facts. conclusions and future work the abductive technique outlined in this paper is another tool in our growing toolbox for handling inconsistency in requirements specifications. while this work provides a novel contribution to problems of managing evolving specifications, a number of critical research issues still need to be explored. to make the work more accessible, we need to extend our approach to allow (a) arbitrary qc formulae to represent requirements specifications and (b) first-order reasoning. this could be achieved by introducing automatic rewrites of any qc specification into disjunctive normal form, which would express universal and existential quantifiers, on a finite domain, in terms of conjunction and disjunction of ground formulae. this extension would facilitate an easier and more direct representation of requirements specification into qc logic. the abductive technique described can also be extended in a variety of ways. most pressing is the need to abduce not only literals, but also more complex formulae changes to a specification are typically more than just atomic changes. of course, choosing the right changes to perform, even given a simple choice of atomic changes, is a difficult task. the impact of each action must be evaluated, and actions with the most 'desirable' consequences chosen. we have started to explore ways of comparing the consequences of performing different actions on (inconsistent) specifications (nuseibeh & russo 98). however, like abducing the inconsistency handling changes described in this paper, the crucial issues of complexity and scale must also be addressed15. we are attempting to address these issues in three ways. first, using the translation technique of qc specifications into first-order logic, we are investigating the possibility of using existing theorem provers for classical logic in order to allow automated reasoning on large inconsistent specifications. second, and building on the work of menzies (menzies et al. 98), we are conducting controlled experiments to investigate the limits to (abductive) reasoning about inconsistent specifications (menzies et al. 99). these limits include attributes such as the size of the specifications and their structure. third, we are continuing case study work (russo et al. 98) to determine how to (re)structure requirements specifications using "viewpoints" (nuseibeh et al. 94) in ways that make them more amenable to the kind of reasoning we have described. the case studies are also providing valuable domain-specific knowledge and heuristics that are essential for pruning the abductive reasoning search space and focusing development actions. nevertheless, and despite the difficult issues of scalability above, we believe that using abductive reasoning as outlined in this paper provides useful guidance to the management of inconsistency in evolving specifications. the paper has laid the foundations for our approach and demonstrated it on a small, but illustrative, example. further work is still needed to explore the application contexts in which the approach is most suited, and we are currently conducting case studies with our industrial collaborators to validate the approach. acknowledgements we would like to thank steve easterbrook, tony kakas, jeff kramer, tim menzies and axel van lamsweerde for their comments on earlier drafts of this paper, and mark tan for his work on the tool support. we would also like to acknowledge the financial support of the uk epsrc for the projects mise (gr/l 55964) and voici (gr/m 38582). references anderson j.s. and durney b., "using scenario in deficiency-driven requirements engineering", ieee proceedings of international symposium on requirements engineering (re '93), 134-141, san diego, january 1993. balzer r., 'tolerating inconsistency", ieee proceedings of 13th international conference on software engineering (icse-13), austin, texas, 1991. bohner s.a. and arnold r.s., software change impact analysis, ieee cs press, 1996. console l., sapino m.l. and theseider dupre d., 'the role of abduction in database view updates", journal of intelligent systems, 1994. console l., portinale l. and theseider dupre d., "using compiled knowledge to guide and focus abductive diagnosis", ieee transaction on knowledge and data engineering, 8(5): 690-706, 1996. cugola g., di nitto e., fuggetta a. and ghezzi c, "a framework for formalising inconsistencies and deviations in human-centred systems", acm transaction on software engineering and methodology, 5(3): 19115 it is well known that abductive reasoning is np-hard [selman & levesque 90]. however, by exploiting modularity of specifications and limiting the use of abduction to particular specification structures and domains, we believe that we can still perform some useful reasoning about inconsistent specifications. 128 230, 1996. duffy d., macnish c, mcdermid j. and morris p., "a framework for requirements analysis using automated reasoning", proceedings of 7th international conference on caise'95, lncs 932, springerverlag, 68-81,1995. esghsi k., "a tractable class of abductive problems", proceedings of 5th international conference on logic programming, 1998. esghsi k., "abductive planning with the event calculus", proceedings of the international joint conference on artificial intelligence (ucai '95), 1:3-8, 1998. finkelstein a. and dowell j., "a comedy of errors: the london ambulance service case study", ieee proceedings of 8th international workshop on software specification and design (iwssd-8), 2-4, schloss velen, germany, 22-23rd march 1996. finkelstein a., gabbay d., hunter a., kramer j. and nuseibeh b., "inconsistency handling in multiperspective specifications", ieee transaction on software engineering, 20(8): 569-578, 1994. ghose a. k., "managing requirements evolution: formal support for functional and non-functional requirements", proceedings of the 1999 international workshop on principles of software evolution, fukuoka, japan, 118-124, 1999. holzmann j., "the model checker spin", ieee transaction on software engineering, 23(5): 279-295, 1997. hunter a. and nuseibeh b., "managing inconsistent specifications: reasoning, analysis and action", acm transaction on software engineering and methodology, october 1998. inoue k. and sakam c., "abductive framework for non-monotonic theory change". proceedings of the international joint conference on artificial intelligence (ucai '95), 1:204-210, 1995. kakas a.c., kowalski r.a. and toni f., 'the role of abduction in logic programming", handbook of logic in artificial intelligence and logic programming, 5:235-324, d.m. gabbay, c.j. hogger and j.a. robinson eds., oxford university press, 1998. kakas a.c. and mancarella p., "database updates through abduction", proceedings of 16th international conference on very large database (vldb '90), brisbane, australia, 1990. kakas a.c. and mourlas c., "aclp: flexible solutions to complex problems", proceedings of the international conference on logic programming and non-monotonic reasoning, 1997. kowalski r.a., logic for problem solving, elsevier, new york, 1979. menzies t., "applications of abduction: knowledge level modeling.", international journal of human computer studies, 1996. menzies t., ghose a. and waugh s., "abduction: experiments and implications", knowledge acquisition, 1998. menzies t., easterbrook s., nuseibeh b. and waugh t., "an empirical investigation of reasoning with multiple viewpoints', ieee proceedings of 4h international symposium on requirements engineering (re '99), limerick, ireland, june 1999. nuseibeh b., kramer j. and finkelstein a., "a framework for expressing the relationships between multiple views in requirements specification", ieee transaction on software engineering, 20(10): 760-773, 1994. nuseibeh b. and russo , "on the consequences of acting in the presence of inconsistency", ieee proceedings of 6th international workshop on software specification & design (iwssd-9), japan, 1998. russo a., nuseibeh b. and kramer j., "restructuring requirements specifications for managing inconsistency and change: a case study", ieee proceedings of 3rd international conference on requirements engineering (icre '98), colorado springs, usa, april 1998. russo a. and nuseibeh b., "an abductive technique for qc logic", technical report, department of computing, imperial college, 1998. ryan m., "default in specification", ieee proceedings of international symposium on requirements engineering (re'93), 266-272, san diego, california, january 1993. selman b. and levesque h.j., abductive and default reasoning: a computational core, aaai '90, 343-348, 1990. schwanke r.w. and kaiser g.e.., "living with inconsistency in large systems", proceedings of the international workshop on software version and configuration control, grassau, germany, 1988. van lamsweerde a. and letier e., "integrating obstacles in goal-driven requirements engineering", ieee proceedings of 20th international conference of software engineering (icse-20), 53-62, 1998. van lamsweerde a., darimont r. and letier e., "managing conflicts in goal-driven requirements engineering", ieee transaction on software engineering, nov. 1998. zowghi d. and offen r., "a logical framework for modelling and reasoning about the evolution of requirements", ieee proceedings of 3rd international symposium on requirements engineering (re '97), annapolis, january 1997. 129 appendix this appendix outlines proofs of theorems 1 and 2 of the paper, together with some basic definitions. for a full account of the proofs, the reader is referred to (russo & nuseibeh 98). definition 1. (focus): let at v... vet,, be a qc clause and let otj for some 1 < i < n be one of the literals in this clause. focuscctiv..^^, <xj) is the clause p,v...vpn.,, where, for each j, 1 < j < n, p j i sa literal in a|v...von different from oj. definition 2. (first-order axioms of qc specifications): the first-order axiomatisation axqcfot a qc specification is the following three axiom schemas: (axl) holdss(oc) <-> holdsw(oc). (ax2) holdss(a,v...van) <-> [ ( holdss(a,)v...v holdss(an)) a ( holdss(-iai) —> holdss(focus(aiv...van, ao)) a...a ( holdss(-,aj -» holdss(focus(a, v...v a,,, a,,))) ] (ax3) holdsw(a,v...van) <-> (holdsw(a,)v...v holdswtocj) definition 3. (translation of a violated rule): let s be a qc specification. let r be a qc violated rule. t(-ir) is a ground atomic first-order formula given by holdsw(oc) if r is the single literal a, or holdsw(aci) for some 1 < i < n, if r is the clause (xiv...v a,,. note that, because of (axl) in the above definition, holdsw(ac) and holdsw(0(cj) can equally be replaced by holdss(oc) and holdss(ocj). proof of theorem 1. the two derivability relations s qcand >ot.are sound and complete with respect to their semantic entailment relations. therefore we prove the following equivalent statement, si=qca <=> t(s), axqc \=fol holdsw(a). if half. proof by contradiction. assume that t(s)^\a:gc i=fol holdsw(a) and that sxqc oc. there exists a qc model x such that for all formulae p€ s, xl=s p and xxs a. (note that the symbol l=s denotes qc strong satisfiability.) the proof consists of constructing, from x, a classical interpretation i, such that i satisfies axqc, holdss(p) for each holdss(p) 6t(s), and -.holdsw(a). therefore, i does not satisfy holdsw(a), so contradicting the initial hypothesis. i is defined as follows. iiholdsslh = (p i xl=s p), and iiholdswii, = (p i xl=w 3). only-if half. proof by contradiction. assume that s!=qc a and that i(s)axqc xfql holdsw(a). therefore, there exists a classical model i of axqc, which satisfies holdss(p), for each holdss(p)et(s), and which does not satisfy holdsw(a). the proof consists of showing that there exists a qc model x such that xl=s p, for each pg s, and xxw a, so contradicting the initial hypothesis. (note that the symbol l=w denotes qc weak satisfiability.) x is constructed from the classical interpretation i, as follows. for each atomic ground predicate oj, of the qc language, x={+oi, i ctj e iiholdssii,m-<xi, i -,0j e iiholdssii,} x satisfies the following properties. (1) for each literal a, xl=s a if and only i satisfies holdss(a). (2) for each literal a, xl=w a if and only if i satisfies holdsw(cc). (3) for each disjunct cc| v... vo,,, xl=s <xj v... vet,, if and only if i satisfies holdss(oc,v...voj. (4) for each disjunct a,v...von, xl=w (x|v...von if and only if i satisfies holdsw(a,v...vo,,). these properties, together with the initial hypothesis, imply that, for each pg s, xi=sp and x xw a. proof of theorem 2. let s'= s \ a and let p'= p \ i(a). two main steps: (1) show that if p'xt(-,r) then t.(s'),y4;t!2cxfotx-'r). and (2) show that if t(s')^4jcccxfoli(-ir) then s'xoc-'r. the second step is already given by theorem 1. the first step is proved as follows. let tra be the new set of facts obtained by performing i(a) changes on the facts of p. by the correctness theorem in (kakas & mancarella 90), there exists a stable model, called m, of the new program p', which satisfies -.holdss(-ir) and therefore which does not satisfy holdss(--r). the rest of the proof consists of showing that there exists a classical model i of axqc that satisfies t(s') and that does not satisfy t(-ir). i is constructed as follows: for each literal a, ae iiholdsslh if and only if cce iiholdssiim. for each literal a, ae iiholdswii, if and only if ae iiholdssii,. for each clause aiv...von, if aiv...vanellholdssiim then aiv^.v^ellholdsslli; ifa|v...van6llholdssllm then, if for all ctj, l<i<n, ojg iiholdssii, then aiv^.v^e iiholdssii,; if for some o^, l<i<n o^e iiholdssil!, then <x,v... vo^e iiholdsslh if and only if for all oj, l<i<n such that o^ e hholdssii,, focus(aiv...vo,, otj) e iiholdssii,. for each clause a|v...va,,, aiv^-vctnellholdswii, if and only if for some oj, l<i<n, ojellholdssii,. 130 the australasian journal of information systems (ajis) began in 1993 and was based on the simple realisation that there was a need for an academic ‘outlet’ for is research in australia australasian journal of information systems volume 17 number 2 2012 43 a development-oriented dss evaluation approach: a case demonstration for conceptual assessment shah jahan miah school of management and information systems victoria university shah.miah@vu.edu.au justin debuse faculty of arts and business university of the sunshine coast j.debuse@usc.edu.au don kerr faculty of arts and business university of the sunshine coast d.kerr@usc.edu.au abstract as a subset of the information systems (is) discipline, decision support systems (dss) development research needs a broader conceptual lens in order to evaluate design qualities specific to dss. this paper presents a development-oriented approach for evaluating dss applications. based on a design science mental model proposed by peffers, tuunanen, rothenberger and chatterjeea (2008), our study outlines a new application area of this conceptual approach through a case demonstration of dss development through various design checkpoints. to assess the proposed developmentoriented evaluation strategy the paper also describes a qualitative investigation that has been undertaken to assess the conceptual approach with target industry dss users and developers. the findings suggest that this strategy is of use for improving the overall design qualities in every phase of dss development, and could be useful for evaluating dss design with target users and dss designers within a socio-technical design context. keywords: design science research, evaluation methods, decision support systems introduction there is a long tradition by decision support systems (dss) researchers of employing various information systems (is) methods to evaluate dss development. evidence of this can be viewed through many dss development studies (muntermann, 2009; purao and storey, 2008). the evaluation of various specialised dss applications such as es (expert systems) and kbdss (knowledge-based dss) have previously employed a validation and verification based strategy. however, it is suggested by phillips-wren, mora, forgionne and gupta (2009, p. 643) that classical is evaluation frameworks and models “lack a holistic worldview that considers jointly the organisational, designer and builder criteria of interest”. using such criteria can be problematic for evaluating dss, as these can “proactively impact the process of decision making as well as the outcome, by providing for example, real time response, distributed architectures and autonomous mailto:shah.miah@vu.edu.au mailto:j.debuse@usc.edu.au mailto:d.kerr@usc.edu.au australasian journal of information systems volume 17 number 2 2012 44 behaviour to support the decision makers”. arnott and pervan (2005) supported this argument by differentiating the key attributes of the dss from traditional is design, as it “represents a different philosophy of support, system scale, level of investment, and potential organisational impact. they can use quite different technologies and may support different managerial constituencies” (p. 68). as the development of such computer-based personalised support system is crucial in a specific problem domain, our study addresses particular boundaries of is application design and development through the differentiation of dss from is. good dss design requires having a clear concept of the nature of the target decision problem and a well-defined strategy of how to adequately support the decision process (arnott, 2006). classical dss development methods such as evolutionary prototyping provide little support to system developers on how to proceed, as they do not have checkpoints that can be used to evaluate design qualities throughout development. arnott (2006) also described another problem for a dss developer, namely how to conceptualise aspects of the decision task that need improvement during iterations of the evolutionary development process. we contend in this paper that a dss developer needs to employ approaches that are different from traditional is evaluation methods because dss development includes factors that are significantly different from the traditional is approach. in other words, the methods used for dss evaluation do not have the same goals as those in is evaluation. we aim to address this problem through a design science approach based on research conducted by peffers, tuunanen, rothenberger and chatterjeea (2008). these authors proposed a conceptual model in the design science paradigm that has six phases, namely: define problem, define objectives, design and development, demonstration, evaluation and communication. arnott and pervan (2008) have suggested that dss research has a long history of using design science research, and other studies also indicate that the rigour of dss research needs to be improved (arnott and pervan, 2005, 2008). in addition, mackrell, kerr and von hellens (2009) proposed a sociotechnical design approach for enhancing the human perspective and situational practicalities of decision making in a specific problem domain. motivated by these suggestions from arnott and pervan and the socio-technical design for is development, we attempt to address the complex requirements of dss evaluation through the mental model approach proposed by peffers et al. (2008), using the six-phase checkpoints throughout dss development. therefore, this paper describes a qualitative research study designed to investigate the applicability of peffers et al.’s model for evaluating decision support systems within a socio-technical context. in mumford’s (2000) definition, “socio-technical design is an approach that aims to give equal weight to social and technical issues when new work systems are being designed” (p. 125). within this context, this paper discusses how activities in different development phases or checkpoints can be used to assess successful design. this study assesses this comprehensive evaluation approach 1 within a decision support systems (dss) problem context. our findings suggest that the evaluation approach can be viewed not only as a basis for further knowledgeable actions in organisations, but also as a new lens with which to evaluate dss in a broader, more practical perspective. we focus on applying the design science mental model to evaluate various development stages of dss. our key argument is to illustrate how a design sciencebased approach offers a better strategy for dss evaluation, since it can take the entire development activities into consideration. within our study, we verify the appropriateness of the method-based 1 the approach was presented initially at the 5 th conference on qualitative research in it (qualit), 29-30 nov 2010, brisbane, australia. australasian journal of information systems volume 17 number 2 2012 45 mental model in a real-world dss development environment. utilising an interpretivistic approach to capture independent opinions of the target stakeholders group, we use a forestry pest management dss as a case demonstration. this type of evaluation is useful since the dss has the potential to provide rapid, simple answers to important problems within an industry. the socio-technical design science approach presented by gregor and jones (2007) defines the structural components of is design research as: “(1) purpose and scope, (2) constructs, (3) principles of form and function, (4) artefact mutability, (5) testable propositions, (6) justificatory knowledge (kernel theories), (7) principles of implementation, and (8) an expository instantiation” (p. 312). in a similar conceptual discussion, mckay, marshall and heath (2008) indicate multiple views of artefact design using aspects of the research outlined by iivari (2007) and carlsson (2006). both approaches imply that is design within the socio-technical context requires a broad approach as there are various significant components. a key component of an evaluation strategy should be the examination of the developed design within the context of the problem domain and, since most dss development problems result from poor identification of end users’ needs (arnott and pervan, 2008), end users should be able to determine whether the solution is “good or bad”. a conceptual approach such as a mental model can also give a template with checkpoints for end user and designer to help them evaluate the design under a broader context. therefore, a key question for this study is: does the design science-based evaluation approach address dss development qualities in a socio-technical context? the rest of the paper is organised as follows: the next section provides details on the background literature of the design science method and is evaluation issues. subsequently, the problem context and method used for conducting the study are described. next, the conceptual approach and its application for evaluating artefacts are defined. the following section provides details of the case evaluation and quantitative findings from our empirical investigation. finally, the conclusion summarises the key contributions and further research from this study. theoretical foundation design science methods the primary focus of conventional methods in design science research covers two areas: one based upon innovative design theory for a technical artefact, focussed upon the quality of the artefact (hevner, march, park and ram, 2004; march and smith, 1995), and the other based upon a design within technical and social phenomena, focussed on design realities such as the impact of the artefact on an organisation or society (gregor and jones, 2007; iivari, 2007; mckay and marshall, 2007). both views may have implications for guiding design practices, but in dss design most of the solution design needs to address problems in decision making that are unstructured or semistructured. the design innovation in dss covers four areas: designing methods and instruments; designing interactive computer-based systems for improved decision support; designing decisionmaker oriented process; and designing flexibility in separating data and models in dss applications (carlsson and turban, 2002). in addition, in most of the previous dss development approaches, researchers, developers and users have suffered from a lack of practical knowledge of how the intended system design fits the problem context, such as rural industries (miah kerr and gammack, 2009; miah, 2008; kerr and winklhofer, 2006; mccown, 2002; cox, 1996). the design science approach within the dss field is not a new research area, and many is researchers have described the use of design science in dss development. examples include muntermann (2009) and arnott and pervan (2008), who analyse dss development in the context of a design science methodology developed by hevner et al. (2004). they found that, although design science has been australasian journal of information systems volume 17 number 2 2012 46 widely used in dss research since the 1990s, a key shortcoming was the paucity of evaluation methodologies and supporting theory. this study addresses the identified gap by applying an evaluation methodology, based on design science theory, to the phases of development involved in producing a dss. focus groups, the primary evaluation approach suggested by arnott and pervan (2008), are used in this study. this broader use of the design science method can be considered a useful lens for evaluating dss applications as it identifies the effectiveness of each development phase through various checkpoints. issues in evaluation approaches many approaches have been used in the evaluation of is, and considerable overlap exists between them. each approach involves different features for measuring is qualities. for example, hirschheim and smithson (1988) (described by jackson and sulaksono, 1998) include the system benefits, which are examined by hitt and brynjolfsson (1996) in the areas of customer value, profitability and productivity; similarly, the technical performance of the system is measured, as performed by delone and mclean (2003). hitt and brynjolfsson’s approach does not cover perspectives such as information quality and how well the system is provided as a service, unlike delone and mclean’s (2003) model. further, existing approaches do not appear to include evaluation of the development processes used to produce the system, although jackson and sulaksono (1998) note that hirschheim and smithson (1988) include the maintainability of software to be part of its technical performance. when considering dss evaluation, forgionne (1999) suggested a model that could measure effectiveness based on the decision making process. parikh, fazlollahi and verma (2001) proposed a similar view that used four evaluation criteria based on decision quality, decision making satisfaction, decision makers’ learning and decision makers’ efficiency. however, phillips-wren, hahn and forgionne (2004) commented that these findings should include information about the outcomes in terms of decision process improvement and any gains in the users’ ability to handle decision problems. this implies that the dss evaluation strategy should focus on the process, users and their effectiveness in handling decision-making scenarios. this finding also applies to specialised dss development such as es and kbdss. as described earlier, conventional evaluation approaches for specialised dss development are based on the v & v (verification and validation) approach (mahaman,passam, sideridis and yialouris 2003; mansingh, reichgelt and bryson, 2007; juristo and morant, 1998; el-korany, rafae,baraka and eid, 2000). verification is described as “building the system right”, that is, a system appropriately implements its specification. validation is described as “building the right system” (mahaman et al., 2003). for evaluating es, mansingh et al. (2007) conducted the validation approach by comparing the system outputs with a case solution by human experts. the evaluation was based on examining the effectiveness of the user interface and the system’s recommendations. this type of evaluation is more appropriate for domain-specific system development where the system serves specific user groups. such strategies are not helpful for achieving broader objectives for dss developers, such as determining the quality of the decision or evaluating the design from the users’ point of view. problem context and methods study context to determine the applicability of our framework, we evaluate the conceptual approach in a real-world dss development environment within a rural industry. the problem context we used in the study is decision making in forestry pest management. this is a very important problem area, since pests have the potential to render plantations unviable (carnegie, stone, lawson and matsuki, 2005), and the choice of treatment regime is crucial in minimising this. the dss examined within this study allows australasian journal of information systems volume 17 number 2 2012 47 users to input a range of characteristics of a forestry site, such as the spacing of the trees, pest species identified and proportion of foliage damaged. the system then outputs a recommended action to address the identified problem, and where possible allows users to access pdf files containing further information such as photographs of symptoms and description of insecticide treatment. to capture this situation, we developed a prototype following the six activities of the proposed approach. this allowed us to evaluate the usefulness of the proposed evaluation framework from the perspective of forestry scientists; a key factor in the evaluation of usefulness was the familiarity of the scientists with the system and its application domain. methodology the aim of our interpretive approach is to capture a range of independent opinions from a target group of stakeholders. we focused on the key components of dss evaluation from a socio-technical design perspective, such as stakeholder orientation, process changes and key evaluation criteria according to the situational needs. we therefore evaluated our proposed approach using interviews. we used four individuals for the interviews and asked participants to determine how useful they perceived the evaluation checkpoints to be; these checkpoints, together with their stages, are: 1. outline the decision problem a) problem importance b) problem suitability for decision makers c) problem complexity / simplicity 2. define design objectives a) whether quantitative or qualitative measures are to be used b) appropriateness of objectives c) resources required by design objectives 3. artefact design and development a) design and development approach used b) measures used to determine the innovative features of the system 4. identify design context a) determine the context in which the system is to be used and further tested 5. measure effectiveness and efficiency a) effectiveness and efficiency of system within its application context b) whether effectiveness and/or efficiency meet target requirements 6. communication of results a) determine how the outcomes are to be presented b) determine whether the communication structure is appropriate for the target audience c) determine whether the system outputs match discipline knowledge. proposed evaluation approach we used the framework shown in figure 2 (adapted from peffers et al., 2008) to evaluate dss development because it is able to incorporate product, practice and process perspectives within a socio-technical context. the evaluation begins with problem identification; complexity is also determined to establish whether development of a dss would be infeasible or overly trivial. the extent to which the developed solution addresses the identified problem is then determined, using qualitative or quantitative measures. the approaches used to build the system, together with the quality of the knowledge embedded within it, are then evaluated. next, an assessment of how well the solution works within its problem context is made. following this, the effectiveness of the system, including for example the patchiness of data availability and its efficiency (such as how swiftly it allows decisions to be made), australasian journal of information systems volume 17 number 2 2012 48 are evaluated. we provide an option to return to the third step if performance is unsatisfactory; otherwise, an assessment is made of how well the system communicates its results, covering areas such as whether they are related to domain knowledge or their form is appropriate for their target users. activity one: outlining decision problem we evaluate problem components for modelling solution activity six: communication of results we evaluate how the outcomes of the dss studies are communicated for acquiring feedback activity two: define design objectives we evaluate user experience, intention and planning components activity five: measure effectiveness and efficiency we evaluate decision support features with respect to objectives activity three: artefact design and development we evaluate key qualities of the process and implementation features activity four: identify design context we evaluate values of design and potential impacts figure 1: conceptual evaluation approach based on the framework proposed by peffers et al. (2008) (extracted from miah et al. 2009) case evaluation and findings in the dss design research domain, arnott and pervan (2008) suggest that a design artefact can be a construct, model, method or instantiation. march and smith (1995) define a method to be “a set of steps (an algorithm or guideline) used to perform a task” (p. 257). this implies that the proposed evaluation approach may be considered a design artefact. hevner et al. (2004) proposed five distinct methods for evaluating designs and these are shown in table 1. we considered the descriptive method to be the most appropriate for our problem context, and so this was used for our study. our aim was to capture the utility of the design by constructing a detailed scenario. we focused on the application of the proposed evaluation approach in the practical context to build a convincing argument to show the utility of the proposed approach. the utility of the proposed application of the approach in relation to the existing knowledge base of scientists using the system was measured, along with its usefulness within the target problem context. australasian journal of information systems volume 17 number 2 2012 49 method description observational where designs are studied within projects analytical where qualities of the design such as its performance and complexity are measured experimental where the design is evaluated in a controlled context testing where designs are checked for errors descriptive, comprising “informed argument” where the utility of the design is measured using existing knowledge, and “scenarios”. the utility of the artefact is described within a target problem scenario table 1: the five methods for design evaluation, adapted from hevner et al. (2004) members considered scientists end users members considered plantation managers end users stage 1: problem definitions – the importance of the problem is separate from system quality, and complexity would not detract from the need for system development; it would just make it harder to do. one group member commented: “i don’t think you’d do a bad job of creating a product or a less important system just because it’s less important would you?” however, respondents considered “need for system development” in place of “system quality” – if the system is to be developed it must be important and it must be of high quality, the problem must be sufficiently complex to warrant development. this is considered a circular question, as they are in favour of system development. simpler problem solving would depend on the end user (more about context): “we’re deliberately tailoring the questions so that they can answer them”. the system is deliberately tailored to make it simpler, and remove the onus on users to determine what causes damage; instead, they can focus on the extent and type of damage. the paper version would be an alternative if simpler. problem complexity is important: “because you don’t want to turn the end user off because it’s … too complex for them”. stage 2: define design objectives – any assessment of system quality would be very useful. resources used is not a good measure of usefulness. the following comment was received: “it would be useful to be able to measure whether your system is working, qualitative or quantitatively, if that’s what the question’s asking, whether you want to be able to evaluate it once it’s produced”. this needs addressing before development. objectives need to be well defined and realistic; it is very useful to have realistic objectives: “i think it’s about having … defined objectives appropriately, but … this is one of those questions that comes in before you would develop it, isn’t it again? but certainly appropriate to have objectives – the more resources spent on the system, the higher the quality. quantitative assessments are very important. it is important to know which assessment type is used, as quantitative assessments are required and are very important: “i think this is very important because the end users want to know … with some degree of confidence that it’s recommending the right action”. this appears to be a circular argument – they would not be discussing it if the system were not wanted, unless it was a random problem: “we’re not … approaching this as a random problem”. usually the more resources spent on a system the higher its quality: “usually if you put more effort into australasian journal of information systems volume 17 number 2 2012 50 and to define them, and for them to be realistic”. as long as the system is useful at the end of it all, resources used is not a good measure. easier systems will take less time to develop: “if it’s just an easier system it takes less time to develop”. things they’re usually a lot better, though it depends on the complexity of the problem as well.” stage 3: artefact design and development – leave the design and development to the experts. would not know, and would leave this to the experts. does not matter if outcome is the same. “i’m not sure i’d know enough about the options ... as long as the outcome was the same”. prefer a working system to a flashy one: “i’d rather have a system that would work than a system that was … flashy and uploadable … as long as it worked in one format”. if the system worked then innovative features would be excellent: “if it could be transferable to … the pda system for example then … that would make it … a little bit more flexible”; “that’s something that’s going to have more impact on the grower themselves isn’t it than us”. – leave the design and development to the experts. prototyping sounds better if done by someone who can do it properly, but the approach is unimportant other than for implications such as budget or time constraints: “i think the most important thing is that it’s honed to the problem that we want and that it’s done by somebody who can develop it properly”. innovativeness is important – in forestry it is critical to use some hand-held device out in the field – if it became complex rather than simple it could be a determinant for use on a hand-held device: “from a forestry perspective … an innovation such as being mobile would be really important. anything to make the field guys’ work easier”. stage 4: identify design context – context is very important. important – specific to plantation management. if it is already developed you could have more flexibility: “in terms of developing it i think the context’s important, but if you have something that’s already been developed then you could use it in a range of contexts, so it would have flexibility”. – context is very important. context is very important – the system is of no use in other agricultural contexts: “it’s purely specific to this type of forestry”. stage 5: measure effectiveness and efficiency – both effectiveness and efficiency are very important. both effectiveness and efficiency are very important: “do effectiveness and efficiency equal accuracy? [interviewer responds that that would be part of it] … very important”. – scientists want the system to be as accurate as possible, but effectiveness is equally as important. efficiency was not considered as important unless it was problematic, such as the system crashing all the time. buggy or inaccurate systems would not be used in the field: “guys in the field, if it’s buggy and it’s always collapsing they won’t use it, they’ll just go ‘oh, i can’t be bothered with that’ … if it’s not giving accurate results they’ll do the same thing”. efficiency was not considered as important as effectiveness; if the system required a lot of power and kept crashing this would be problematic: “if they were using it on their mobile phone and it required a lot of grunt for example, and it kept crashing then … they wouldn’t be bothered with that”. scientists want the accuracy as high as possible as there is so much error in the collected input data: “from a scientist point of view i think you want as high as possible, because you know there's already so much error in the way you collected the data that goes into it”. the manager thinks differently, but this australasian journal of information systems volume 17 number 2 2012 51 table 2: participants’ comments on the proposed mental model for dss evaluation interview recordings were transcribed and the open coding method employed to break down and categorise the data; subsequently the axial coding method was employed to make the necessary connections to the different categories and to integrate the themes. the transcripts were relatively small and the themes were easy to identify; thus, comprehensive analysis using software packages was not necessary. the procedure for evaluating the approach consisted of three steps. firstly, we discussed the proposed evaluation approach through a prototype dss development in the target problem area (forest pest management). the initial discussion helped us to introduce key benefits of the checkpoints and discuss their relevance to the target dss design. this meeting consisted of three forestry scientists who develop dss applications for their use. secondly, this prototype was demonstrated. thirdly, by means of a questionnaire, we gathered opinions from scientists and end users on different evaluation checkpoints based on the activity. finally, we conducted a phone interview to acquire inside details of their comments, and recorded the conversation. four scientists were interviewed in two groups of two, focussing upon the application of each checkpoint to the prototype dss. the first interview was done through speakerphone and the second face to face. we illustrate the captured opinions in table 2 above. discussion the key argument of this study was to evaluate how a design science-based approach may offer a superior strategy for dss evaluation, since it takes the entire development activities into consideration. the study evaluated a development-oriented evaluation approach for dss design to is being answered from the scientists’ viewpoint. the system should work adequately and not crash: “it’s not so important to me either whether there were targets related to that [efficiency], i think it’s just as long as it works adequately and it’s not crashing”. stage 6: communication of results – communication of results was considered very important. very important: “i was looking at … how it looked on the screen … rather than the accuracy of the information it gives you”. “that’s what it’s all about”. “to me, the jargon can be communication structure”. a system that provides bad results would not be useful: “i don’t think there’s much use in having a product if it creates advice … that isn’t relevant”. – communication of results was considered very important – it has to be accurate. the end user just wants the answer. the less output produced the lower the system quality. this is a measure of accuracy – need directed information: “it's a kind of measure of whether those outcomes are likely to be accurate ... if it's only the pdf, then they've got to read through all that and find out whether there are any management outcomes, whereas if the thing states outright you should do this and here's a bit of extra information”. bad structure reduces system quality; for example, salinity maps have been rejected by end users: “i've seen whole projects looking at salinity where they produce these beautiful maps, and then the landholders go 'no, not using that'”. users just want the system to tell them the answer: “just tell me the answer”. very important: “[interviewer prompts: if the system says you do action a, and the literature says no, do action b] that's ... not useful”. australasian journal of information systems volume 17 number 2 2012 52 address the call by arnott and pervan (2008) to improve dss design research; peffers et al.’s approach was used as a template to demonstrate its applicability for dss evaluation. we contend that this study justifies the application of the approach in the defined theoretical and practical context of dss evaluation. we argue that the approach can evaluate whether the entire development and design cycle meets the requirements for dss development, since it allows us to evaluate properties through various checkpoints from all phases and ensures that the design meets requirements within the context as outlined by stakeholders. this is particularly useful since dss development is highly iterative, using techniques such as evolutionary prototyping. because of the need for many iterations and the high level of end-user involvement related to individual context, we consider dss evaluation to be sufficiently different to standard is evaluation approaches to warrant this alternative method. our approach also covers many of the concerns about dss development with respect to the social aspects, such as a lack of end-user uptake, rather than concentrating on verification and validation evaluation processes that concentrate on developers’ design and technical interventions (mackrell et al., 2009). the contribution of this study is to enhance the theoretical foundation of dss evaluation using design science knowledge as initially proposed by peffers et al. (2008). the proposed approach captures both the development processes and context to enable the evaluation of the artefact within an organisation. this approach therefore identifies a range of issues with development, from the problem identification through to communication of the solution. as discussed previously, we used a socio-technical lens to evaluate a design, in which perspectives such as users, process, practice and product were covered. it is hoped that this approach will also improve the adoption of dss by end users, because we consider that it will help provide practical knowledge of how the intended system design fits the problem context for both the developer and end users. we believe this will address many of the concerns about how the intended systems fit the problem context as outlined by miah et al. (2009), mccown (2002), cox (1996) and kerr and winklhofer (2006). this is because the problem identification stages are included in the evaluation, as opposed to previous dss implementations, where the problem definition was considered a given by the developers but not necessarily by end users. practically, the findings suggest that all of the six phases were important. respondents considered some aspects of the early phases, such as the problem importance and the appropriateness of its objectives, to be self-evident; the relevance of system complexity depended on its target users. however, what is self evident to some is not necessarily so for others, and previous dss solutions have suffered from a lack of context to the problem at a managerial level kerr (2004). therefore we consider an evaluation of these early phases critical, especially at the level of the end user. the respondents’ views on the types of quality measure varied, though both groups agreed that this area was important. similarly, whilst one group viewed the end product quality as more important than the resources it consumed, the second argued that the two were somewhat interrelated. obviously the expenditure of resources is very important and is critical for the viability of the dss project; however, we must infer from the response from the first group that they would consider a high quality dss with a more limited scope as a more useful product than one using more limited resources because the scope of the project was too big. the design and development approach, though important, was seen as a matter for system development experts, and this is appropriate. however, it is also very important that design and development criteria are fully understood by both parties, and that communication channels are effective. the innovation measures were considered important, particularly if they represented vital features, such as portability within the problem domain, via mobile platforms in the field. australasian journal of information systems volume 17 number 2 2012 53 the later phases were considered to be particularly important in the problem context. all respondents considered the design context to be crucial. in a similar vein, effectiveness and efficiency were considered very important, with respondents suggesting that a system that was not effective, or with efficiency issues impacting upon effectiveness, could not be used. unsurprisingly, communicating results proved to be the most important phase, in which information quantity, quality and simplicity were considered the key to solution design completion. all respondents considered it very important to ensure that the results were accurate and reflected current scientific thinking. this is an important issue that has been problematic in the past (cox, 1996), as end users often want a simplified version of the answer; the goal is to ensure that the results shown are simple enough for managerial end users to understand without being a “black box” solution as described by cox (1996). this means that the logic involved in obtaining the solution needs to be transparent to the end user. it is hoped that the approach described here will allow this to happen through the evaluation of all stages of development. conclusions and future research this research proposed a development-oriented evaluation approach to increase the rigour of dss design research, by systematically evaluating checkpoints throughout the dss development process. the results of this study show that the proposed approach has merits in assessing the quality of the entire dss development process. this represents a more rigorous approach than existing examples, since it is validated by the design science literature. practically, the approach provides superior options for the evaluation and ultimate acceptance of dss in many industries, including rural examples, where user acceptance and rigorous approaches for evaluation have been particularly problematic. the results also highlighted some issues with the study. conveying the meaning of concepts such as quality proved challenging, although examples were presented to the respondents to illustrate this. further, each group used a different definition of end user. however, this did not appear to affect the results, other than in the problem complexity checkpoint, where scientists as end users were considered to be unaffected by complexity, whereas plantation managers were viewed to be strongly affected. further limitations of this study were the number of industry participants and the use of a single dss development case. this reflects the specialised nature of the problem domain; indeed, many dss problem domains are highly specialised. acknowledgements the authors would like to thank dr simon lawson, dr helen nahrung, dr valerie debuse and dr manon griffiths for their valuable assistance with this research and in-kind support. references arnott, d. 2006. “cognitive biases and decision support systems development: a design science approach”, information systems journal (6), pp 55–78. arnott, d. and pervan, g. 2008. “an assessment of dss design science using the hevner, march, park, and ram guidelines”, the 4th biennial anu workshop on information systems foundations, information systems workshop, australian national university, canberra, australia, pp 255–284. arnott, d. and pervan, g. 2005. “a critical analysis of decision support systems research”, journal of information technology (20), pp 67–87. australasian journal of information systems volume 17 number 2 2012 54 carlsson, c. and turban, e. 2002. “dss: directions for the next decade”, decision support systems (33), pp 105–110. carlsson, s.a. 2006. “design science research in information systems: a critical realist perspective”, 17th australasian conference on information systems, acis2006, adelaide, australia. carnegie, a.j., stone, c., lawson, s.a. and matsuki, m. 2005. “can we grow certified eucalypt plantations in subtropical australia? an insect pest management perspective”, new zealand journal of forestry science (35), pp 223–245. cox, p.g. 1996. “some issues in the design of an agricultural decision support systems”, agricultural systems (52), pp 355–381. delone, w.h. and mclean, e.r. 2003. “the delone and mclean model of information systems success: a ten year update”, journal of management information systems (19:4), pp 9–30. el-korany, a., rafae, a., baraka, h., and eid, s. (2000). “a structured testing methodology for knowledge-based systems”, in: ibrahim, m., kung, j., revell, n. (eds.), dexa 2000. springer-verlag, heidelberg, pp 427–436. forgionne, g. 1999. “an ahp model of dss effectiveness”, european journal of information systems (8), pp 95–106. gregor, s. and jones, d. 2007. “the anatomy of a design theory”, journal of the association for information systems (8:5), pp 321–335. hevner, a., march, s., park, j. and ram, s. 2004. “design science in information systems research”, mis quarterly (28:1), pp 75–105. hirschheim, r. and smithson, s. (1988). "a critical analysis of information systems evaluation". in n. bjom-andersen & g. b. davis (eds.), information systems assessment: issues and challenges, noordwijkerhout, north holland. hitt, l. and brynjolfsson, e. 1996. “productivity, business profitability, and consumer surplus: three different measures of information technology value”, mis quarterly (20:2), pp 121–142. iivari, j. 2007. “a paradigmatic analysis of information systems as a design science", scandinavian journal of information systems (19:2), pp 39–64. jackson, b. and sulaksono, s. 1998. “going soft on information systems evaluation”, australasian journal of information systems (5:2), pp 41–50. juristo, n. and morant, j.l. 1998. “common framework for the evaluation process of kbs and conventional software”, knowledge based systems journal (11:2), pp 145–160. kerr d.v. (2004) factors influencing the development and adoption of knowledge based decision support systems for small, owner operated rural businesses artificial intelligence review 22(2) pp 127-147. kerr, d.v. and winklhofer, h. 2006. “the effect of rapid rural industry changes on the development of a decision support system for dairy farmers in australia”, computers and electronics in agriculture (50:1), pp 61–69. mackrell, d., kerr, d.v. and von hellens, l. 2009. “a qualitative case study of the adoption and use of an agricultural decision support system in the australian cotton industry: the socio-technical view”, decision support systems (47), pp 143–153. australasian journal of information systems volume 17 number 2 2012 55 mahaman, b.d., passam, h.c., sideridis, a.b. and yialouris, c.p. 2003. “diares-ipm: a diagnostic advisory rule-based expert system for integrated pest management in solanaceous crop systems”, agricultural systems (76), pp 1119–1135. mansingh, g., reichgelt, h. and bryson, k.o. 2007. “cpest: an expert system for the management of pests and diseases in the jamaican coffee industry”, expert systems with applications (32), pp 184–192. march, s.t. and smith, g. 1995. “design and natural science research on information technology”, decision support systems (5:4), pp 251–266. mccown, r.l. 2002. “changing systems for supporting farmers’ decisions: problems, paradigms, and prospects”, agricultural systems (74), pp 179–220. mckay, j. and marshall, p. 2007. “science, design, and design science: seeking clarity to move design science research forward in information systems”, proceedings of the 18th australasian conference on information systems, pp 604–614. mckay, j., marshall, p. and heath, g. 2008. “an exploration of the concept of design in information systems”, the 4th biennial anu workshop on information systems foundations, isf2008, s. gregor and d. hart (eds.), australian national university, canberra, australia. miah, s.j. 2008. “an ontology based design environment for rural decision support”, unpublished phd thesis, griffith university, australia. miah, s.j., kerr, d. and gammack, j. 2009. “a methodology to allow rural extension professionals to build target-specific expert systems for australian rural business operators”, expert systems with applications (36), pp 735–744. mumford, e. 2000. “a socio-technical approach to systems design”, requirement engineering (5), pp 125–133. muntermann, j. 2009. “towards ubiquitous information supply for individual investors: a decision support system design”, decision support systems (47), pp 82–92. parikh, m., fazlollahi, b. and verma, s. 2001. “the effectiveness of decisional guidance: an empirical evaluation”, decision sciences (32:2), pp 303–31. peffers, k., tuunanen, t., rothenberger, m.a. and chatterjeea, s. 2008. “design science research methodology for is research”, journal of management information systems (24:3), pp 45–77. phillips-wren, g., hahn, e. and forgionne, g. 2004. “a multiple criteria framework for the evaluation of decision support systems”, omega (32:4), pp 323–332. phillips-wren, g., mora, m., forgionne, g.a. and gupta, j.n.d. 2009. “an integrative evaluation framework for intelligent decision support systems”, european journal of operational research (195), pp 642–652. purao, s. and storey, v.c. 2008. “evaluating the adoption potential of design science efforts: the case of apsara”, decision support systems (44), pp 369–381. meta-information systems: a design study mark s. madsen, iain s. fogg & clive l.n. ruggles department of mathematics and computer science charles wilson building, university of leicester leicester lei 7rh, uk fax: +44-533-523604 abstract metadata systems are concerned with the management of data which describes other data, whether datasets, catalogues or entire database management systems. metadata systems can be used to store richly detailed forms of information, perform seamless wide-ranging searches of information distributed across networks, and to integrate information stored in disparate repositories. here we describe the design and methods of implementation derived from the experience of the leicester university metadata project (lump). our approach utilises the incorporation of semantic metadata in addition to resource metadata, resulting in a generally more powerful system than existing global directory services. a feature of the lump design is flexibility of implementation, with the ability to provide a coherent metadata system functioning above heterogeneous autonomous distributed databases. introduction to metadata systems the term metadata means, literally, data about data (lillywhite, 1991). in this paper, we use the term to denote both data which describes elements of discourse — semantic metadata — and data which describes datasets or dbmss — resource metadata. the reason for considering metadata is that a powerful solution to the management challenge posed by the large quantities of data now held globally is the development of metadata systems, whose basic function is to locate data which is relevant to a particular user, given information about the type of data required and the details of the context in which it is to be used. ideally, this context should be defined in terms of the user who requires the data: their background, interests, understandings, and preferences. the main function of a metadata system is therefore to identify data which may satisfy the requirements of the user, and to store information about its location, content, and quality relative to the interests and situation of the user — taking into account such practical considerations as accessibility, cost, and ease of use. the metadata approach to information systems provides a number of powerful advantages over other approaches: • metadata systems provide a level of abstraction between the user and the stored data. this allows the construction of specialised, custom databases as described by neelameghan (1992) which can present their data in more conventional forms and under normal cataloguing conventions, depending on the manner in which they are accessed. • such a level of abstraction can provide the conceptual space for information to be structured into knowledge as described by hsu et al (1991) and ingwersen (1992). the metadata environment can typically supply this in the form of information about the user which in turn can be used to filter the user's view of the metadata and data stored in the system. • metadata can provide an integrating framework for disparate databases so that the differences between these are transparent to the user. this can be the case even when the databases are heterogeneous and widely distributed. this issue is discussed by madsen, fogg & ruggles (1994). a complete metadata system should be capable of performing all of these functions automatically, so as to provide the user with a transparently personalised view of the information space. the overall problem set with which the system needs to be able to cope is extensive. generic problems which need to be faced are sheer volume of data managed by heterogeneous dbmss, and resilience of function against inevitable failures of metadata consistency. it is important to understand how the ideas presented here relate to existing directory/data dictionary services, such as www (world wide web), wais (wide area information service), x.500, whois-hand the gopher interconnection protocols. the overall functionality of all of these differs in a subtle but crucial manner from general metadata models. while the directory services are intended as general enquiry services which allow connection to local services via some uniform protocols, the metadata model abstracts their functions and additionally provides for the integration of knowledge modelling within the system (hsu et al, 1991). september 1994 39 existing studies of metadata systems include the use of a metadata approach to accessing protein structure databases from a knowledge-based system (eccles & saldanha, 1990). there prolog is used as a system metalanguage for the construction of database queries directly from the knowledgebase. as the authors point out, this approach amounts to using the metadata system as a rich communication channel. another study is concerned with the management of decision processes via the integration of three kinds of metadata (hsu et al, 1991). these are operating knowledge concerned with individual data or functional subsystems, control knowledge such as data transfer rules and equivalence tables for distinct data items relevant to the same logical subject, and decision knowledge for global and local processes (the terminology used is that of hsu et al, 1991). closely related with this last-mentioned study is the collaborative work system kms, described by akscyn, mccracken & yoder (1988). a specific model which incorporates some of the ideas described in this article is set out in the paper of ishikawa et al (1993). those aspects of metadata storage which are relevant to the efficiency of its entry and retrieval have been detailed by ruggles et al (1992). a conceptual study of some of the issues arising in the design of metadata systems has recently been carried out by fogg, ruggles & newman (1993). the remainder of this paper is set out as follows. the next section describes the structural aspects of the design, which are mainly concerned with the metadata model. the section after that describes the functionality of the distributed components of the system, along with the strategies adopted in the design for the everyday management of metadata. a section is then devoted to the characteristics of the metadata query language mql. in the final section, we bring together the disparate aspects of the system and describe a prototype implementation (the stile phase i system) currently being field-tested at our institution and others. limitations of space preclude a complete formal description being given, so we restrict ourselves generally to a discussion of the important features of the abstract design. we remark here that the formal system specification has been written in object-z (duke & duke, 1990, hall, 1990, cusack, 1991, and stepney, harden & cooper, 1992) and will be presented elsewhere. structural design informally, our starting point is that the world of discourse can be described in terms of concepts. davis (1986) identifies three broad categories of concept — temporal concepts, spatial concepts, and the vast remainder of pure concepts (davis simply calls these all concepts, but this terminology is too confusing for the present purpose). our approach is to encapsulate conceptual information within more general structures which we call topics. the immediate advantages of doing so are twofold: first is that higher-order metadata (data about metadata) can easily be accommodated within the system, second is that topics can in principle be extended so as to incorporate other attributes beyond resource or semantic metadata. this approach to data via the creation of a neutral database has been successfully used in the integration of commercial applications — see, for example, busby & hutchison (1992). the implementation of topics requires a representation that is amenable to human users. for this reason, textual, graphical, audio, and visual representations of topics are permitted. furthermore, it is quite natural for us to identify relationships between topics in the real world. to facilitate sophisticated browsing and querying of our metadatabase, it is also necessary for us to represent the relationships between topics in the system. the function of our metadata system can therefore be seen as providing a repository for topics, and managing relations between them. in essence, our metadatabase system is a management system for a semantic net — see quillian (1985) and sowa (1991) — although with some features peculiar to the distributed metadata environment within which it exists. this view of metadata systems is close to that expressed by hsu et al (1991). the most important attributes of a topic are the descriptive elements called nuggets, which communicate meaningful aspects of the topic to the human user, and the links from that topic to related ones. links to other topics are represented by the id of the target topic and a weight denoting the importance of that link. (although this aspect is not treated explicitly here, a requirement is that the system should provide effective learning capabilities which will allow link weights to be adjusted according to the results of usage.) the full structure of a topic is illustrated in figure 1, and is to be understood as follows: • each topic possesses at least one toplcld. • a topic may have any number of nuggets. these may consist of text, or involve graphics, audio, video or other combinations of media. each nugget is labelled by a contextld, denoting the context in which that nugget is relevant to the topic. the uses of contexts are described in more detail below. ajis a topic may have any number of links to other topics. a link consists of the topicld of the related topic and a numerical weight expressing its importance. like nuggets, each link is labelled by a context id. a topic may have a dictionary consisting of any number of topiclds representing equivalent topics: those which have been locally identified (that is, from the point of view of the user who controls that topic) as being the same as that topic containing the dictionary. note that these equivalent topics may be controlled by a different user. a topic may have any number of resource entries. each resource entry represents an external resource such as an online database, ftp server, or file viewer. the resource entries consist of scripts which contain sufficient information to facilitate online access to those external resources directly from the topic where that is possible, otherwise information which will enable the user to locate and access the resource offline. \contextid 5 link/ \7 figure 1: illustration showing the structure of a general topic and its connections to other topics within the metadata system's topic network. these ideas are summarised in the entity-relationship diagram of figure 2. it would also be possible to introduce link types to our model (croft and turtle, 1989). there are a number of reasons for not doing so at this stage of development of our metadata management model. for one thing, the benefits of link typing need to be weighed carefully against the extra complexity it implies (glushko, 1989). in our model, we would need to introduce yet another subclass of topics called link type topics, so as to avoid the danger of infinite regress. the strongest argument against link typing, however, is that even without it, our metadata model is capable of effective representation of all forms of recorded information or knowledge, and is also efficient and simple for the user to comprehend. topics with nonempty resource sets are called resource topics. they are the interface between the metadata contained in the topic net and the external data world. topics with no resource entries, on the other hand, contain what is called semantic metadata, namely descriptions of what is meant by particular terms or concepts, and which other semantic materials are relevant to specific topics. the reason for allowing topics to have multiple identifiers is so that deletion or merging of topics can proceed by removal of their contents, but without detriment to the referential integrity of the topic network. september 1994 41 nuggetld v.descriliribes}(1,1) local (o.n) .synonyi remote (0,n) topic (o.n) weight link (ofl) (0,n) nugget -q topicld -q impld (p.e) context topic resource topic access —od.n) figure 2: entity-relationship diagram for the topic structure described in the text. the access attribute of the resource topic corresponds to an external resource. an advantage of the metadata approach to information is that it allows for the possibility of incorporating knowledge management as one of the functions of metadata systems (hsu et a/, 1991). the fact that the topic model is constructed so as to facilitate the interchange of metadata relating to knowledge, rather than simply information, makes it necessary to include some scheme for classification of metadata within the capabilities of the metadatabase (ranganathan, 1967). this point is important since it is the classification and interrelation of information which transform it into knowledge as discussed by belkin (1978), lyotard (1979) and brookes (1980). such a scheme will also turn out to ease the problems of deciding how queries can be dealt with. crucial to the ability of the system to configure the metadatabase effectively so as to suit the needs of a particular user is the use of the context attribute. the purpose of this attribute, as in every form of discourse, is to establish a framework within which meaningful communication can take place. our design also gains further benefits from contexts. in response to the user specifying which context they are using, the system can simplify their view of the system. additionally, contexts are used to construct an efficient query method — see below. in our system, context is realised via the introduction of a label attached to each nugget and link of every topic within the system. this indicates within what circumstances that nugget or link is to be understood as part of the defining structure of the topic within which it is found. contexts are themselves implemented in terms of topics. the contextld mentioned above can therefore be replaced by a topicld, with the introduction of suitable invariants. in our present design, we insist that there be specific context topics. these are not allowed to be resource topics, and they are restricted to have nuggets and links whose context label is that of the global context — this is defined as the context in which all elements of all topics are relevant, and implemented via a special invariant identifier supplied as part of the initial system configuration. a modest example which clarifies some of these ideas is illustrated in figure 3. this might be represented in the system by the topic with topicld number 57, which in addition to the nugget "water" may consist of the textual nuggets "substance used for drinking and swimming", "substance found in rivers and seas", "combustion by-product", and the links ("boats",4), ("fish",8), ("whisky",!). here the ids of linked topics have been replaced by their textual names for clarity, and the number following the name is the weight of that link. the application of the context "sports", whose contextld is the topicld 92 in the example, may pick out the subtopic with the nuggets "water" and "substance used for drinking and swimming", and the link ("boats",4). although drinking is not (always) regarded as a sport, the nuggets themselves are not divisible by the actions of the system, and so the entire nugget is extracted by the sports context. it is easy to see how the topic structure allows this example to be extended to incorporate other forms of media such as sound and video. 42 ajis substance used for drinking and swimmine substance found in rivers and seas figure 3: illustration of the example discussed in the text: the representation of the concept "water" as a topic within the system. functional design the basic functional unit of the system is the imp (information management processor) as imagined in fogg, ruggles & newman (1993). the user's logical view is that each user possesses a unique imp which is responsible for the maintenance of all gathered and stored metadata, and for communication with imps belonging to other users for querying their metadata stores or responding to such received queries. as such, the imp is the user's representative, and must be able to evolve over time in such a manner as to reflect the user's own interests and background, so that the system will often be able to resolve ambiguities and uncertainties of input and queries on the basis of the stored user profile held by the imp. for example, we could use the textual objects "lager" and "beer" to describe a particularly well-known thing in the real world. the problem of course is that "lager" and "beer" are terms used by british people to denote quite different things in the real world, while to an australian or south african, they are naturally the same. so while we may use textual objects to describe topics, the topic that is being denoted by the text depends on the experience and biases of the user (see ruggles el al, 1992). the management of the metadata contained within the system, and particularly within any imf's metadata store, is clearly the central issue in the development of a meta-information system. it is, however, a nontrivial task to anticipate all possible management needs and effective policies. the solution to the management problem consists of three parts: the manner in which the topics describing metadata are communicated between the imps, the definition of the structure of a metadata query, and the strategies to be adopted in the everyday management of each imf's topic network. the approach to communications taken in this paper is that all communications between imps are expressed in terms of the system scripting language mql (see below). all lower level communications and transport protocols are expected to be supplied by the system interface (see figure 5). the second and third aspects are dealt with below. our system design requires that each imp be capable of at least the following suites of operations: september 1994 43 search is to be conducted.) this in turn requires the construction of a quantitative metadata evaluation function. this evaluation of a topic is defined relative to another topic — the similarity between topics is all that the system needs to be able to quantify, since any query requires that a set of topics be evaluated relative to that query, and returned if their evaluation is sufficiently high, according to criteria which are to be included in the structure of the query. note that the user interface is responsible for packaging all user input into the form of topics. clearly, there are many possibilities for the construction of metadata evaluation functions: our design envisions that there will be a predefined set of such functions available to each imp, so that the user can choose which is to be used in a particular query by choosing one from a menu. we remark here that, in addition to the wellknown techniques of word-counting and string-matching, a rich variety of alternative strategies for matching topics to metadata queries is provided by the nonparametric statistical methods known collectively as cluster analysis — for example, see anderberg (1973), duran & odell (1974), everitt (1974), van ryzin (1977) and references contained therein. because of the way in which contexts are used to classify stored metadata, a query must consist (at least conceptually) of two parts, the first designed to establish any appropriate contexts, the second to find appropriate response topics to the query. at the same time, a crucial concern in the design of our metadatabase system is performance. in particular, we will need effective and fast indexing to be applied to topics in order to maximise the throughput of the query processor. to achieve this the system uses a two-level indexing strategy: first the contexts are indexed and then the topics within those contexts. with all these components in place, we can describe some aspects of mql (metadata query language). schematically, a metadata query q consists of two subqueries, a context query qc and a topic query qt. the main difference between the two queries is their interpretation by the imp: the first is applied to that imf's context network, the second to the general topic network. finally, the result of the query is (at least conceptually) subjected to some form of projection. global radius threshold contextld context search criterion radius threshold topicld contextld-set attributes topic search topicld-set projections criterion tonic-set figure 4: the dynamic structure of an mql query. september 1994 45 the overall structure of a metadata query is shown diagramatically in figure 4. the interpretation of this figure is that the context query is of the syntactic form select(criterion, function, radius, threshold, global) where • criterion is the context topic against which the searched contexts are to be compared; • function is the code for the evaluation function to be used by the imp in comparing the definitions of its contexts with criterion; • radius specifies the radius of the context network that is to be returned as the result of each successful match. the radius of the subnet centred on a given context c is to be understood as a number r such that every context in the subnet can be reached by starting at c and following no more than r links. • threshold is a number specifying the minimum cumulative weight of links to be followed in constructing the context subnet. links with a cumulative weight below the threshold are ignored. • global is the contextld of the global context. in its turn, the topic query is of the same form: select(criterion, function, radius, threshold, contextldset) where the parameters are to be interpreted slightly differently: • criterion is the topic against which the searched topics are to be compared; • function is the code for the evaluation function to be used by the imp in comparing the definitions of its topics with criterion; • radius specifies the radius of the topic subnetwork that is to be returned as the result of each successful match. • threshold is a number specifying the minimum cumulative weight of links to be followed in constructing the topic subnet. links with a cumulative weight below the threshold are ignored. • contextldset is the set of ids of matched contexts returned from the context search, and is used to restrict the search space for topics matching the topic query. topics with no components bearing a contextld contained in contextldset will not be examined in the course of the search. the final projection operation is of the form project(contextldset, topiddset, attributelist) where contextldset and topiddset result from the previous select operations, and attributelist is a set specifying which attributes of the topic or topics are required. in order to allow for straightforward retrieval in the case that the relevant contextld or topicld is known to the imp initiating the query, the alternative forms select(contextid, radius, threshold, global) and select(topidd, radius, threshold, contextldset) respectively are used. a query of a remote imp is of course constructed in exactly the same way, but is passed to the remote imp for processing instead of the local one. 46 ajis system architecture the architecture of the system is shown in figure 5. it consists of the following distinct components: stored metadata the metadata may be stored using any storage engine (such as a relational, object-oriented, or flat-file database), with the provision that a translation schema must be provided to translate between the physical and logical (namely, topics and links) data models. local gateway the local gateway performs the translation between the physical and logical data models. query processor the query processor accepts queries from the user via a graphical user interface, and from remote systems via a system interface, performs appropriate optimisations, and passes the query to the local gateway for subsequent processing. the query processor is also responsible for managing remote queries. network user system interface graphical user interface buffer manager query processor update manager resource manager local gateway stored metadata figure 5: the architecture of the system at the imp level. update processor updates can be initiated only by the local user via the graphical user interface. the update processor ensures that certain constraints (for example, contexts may be linked only to other contexts) are not violated. september 1994 47 resource manager the resource manager is responsible for accessing on-line resources that have been located using the query facilities. the resource manager is compatible with a number of "standard" remote access facilities, such as www, wais, and gopher. buffer manager metadata accumulated as the result of remote queries is temporarily buffered. the user is responsible for integrating this buffered material with the local metadatabase. since the buffer is finite, the user can also specify the behaviour of the buffer manager when overflow occurs (for example, discard oldest items first, or discard newest items first). the user may also activate processes to integrate incoming metadata automatically. system interface the system interface deals with the interaction between imps, exchange of queries and metadata, detecting imps that are out-of-service, interfacing to low-level communication software, and so on. graphical user interface the graphical user interface must provide a user friendly, configurable, windowing environment for the control and display of metadata, and for the display of data through interaction with the resource manager. the lump design described above has been used in the construction of the stile (students' and teachers' integrated learning environment) system being developed at our institution and at loughborough university. the aims and agenda of this project are fully described by ruggles (1994). the stile phase i implementation is not yet fully distributed, and does not use contexts to full advantage. it does however use the topic model described above, and implements these in terms of the relational model (based on figure 2) using a bespoke database management system. figures 6 and 7 show screendumps from stile phase i illustrating both the topic search process and direct resource access. it is expected that experience with the working stile system, which is due to be updated to phase ii by late 1994, will be beneficial to the specification of the full final design of which this paper is the outline. discussion metadata systems provide an exciting and enabling new perspective on the functions and abilities of information systems. this paper shows how a relatively simple, even naive, model of the way that metadata is structured and interelated can be developed to provide a framework for the construction of extremely powerful global, distributed, heterogenous, integrating information systems. such systems have the virtues of: simplicity the underlying defining structure used for the stored metadata is simple in concept and design. unlike the metadata form used by many other proposed systems, there is no long catalogue of attributes to be learned before the user can interpret the results of retrieving information or metadata. likewise, the learning barrier which users must pass in order to be able to collect and store their own knowledge is smaller than that encountered in previous designs — for such an example, see hsu et al (1991). flexibility because of the simple structural requirements placed upon the stored metadata, our design is capable of incorporating not only new information, metadata, and resources with ease, but can also accomodate existing collections of metadata without difficulty (eccles & saldanha, 1990). 48 ajis stile : senrch ish23ilk:\guest8\arch.md81 biblio: a hoard of vrkmg silver bracelets from cushelogurt. co. biblio: a viking-age silver hoard from near raphoe, co. doneg< biblio: a lost hoard of viking age silver from magheralagen. co( image colt st ninian's isle: silver bowl image colt st ninian's isle: silver bowl image cotfc st ninian's isle: silver bowl image cotfc st ninian's isle: silver bowl image cob: st ninian's isle: silver bowl imaoe cotfc st ninian's isle: stiver bowl artefacts metafwork biblio: the geographic distribution of irish viking-age hoards biblio: reflections on hrberno-florse coinage biblio: a check-list of viking-age coin finds from ireland bibfio: money among the irish : coin hoards in viking-age irelan ; bibfio: a viking-age gold hoard from ireland bibfio: the viking -age hoards of ireland bibbo: the hibemo-norse coins in the british museum fiade.to mffael opo»«nar» figure 6: screen dump from the stile vl.o system showing the entry of a search specification. si ninian's isle bowl drawing of tbver bowl from the si ninian's isle hoard deiais of individual inaget follow •age 1: st ninian's isle bowl (page 1) location: k:\arch\mages\e_chrisl\scorl.jpg format jpeg imago copyright owner, various contributor dmo authorised br deodre o'suttvan 1: st ninian's isle bowl (page 1) figure 7: screen dump from the stile vl.o system showing the use of a resource directly from the user's imp interface. september 1994 49 extensibility although our simple design is formally complete, it contains a framework within which new types of resource and metadata can easily be incorporated into the system. for example, if it ever becomes possible to create and access online olfactory databases, then these can be straighforwardly incorporated as new resources within the topic network comprising our metadatabase design. integration the metadata approach provides the key to the integration of the functions provided by disparate heterogeneous and autonomous information systems. these systems can then be individually optimised for their own specific design function, and at the same time remain widely accessible through their metadata interface with the rest of the information world (litwin & rossopoulos, 1990). by integrating existing information systems in a flexible manner, allowing access to and use of existing catalogues and utilities, and providing a framework within which users can store and retrieve knowledge, metadata systems generate the potential for rapid and transparent access to the information stored globally both now and in the future. references akscyn, r.m., mccracken, d.l. & yoder, e.a. (1988) "kms: a distributed hypermedia system for managing knowledge in organizations", communications of the acm, vol 31, pp 820-835. anderberg, m.r. (1973) cluster analysis for applications, new york: academic press. belkin, n.j. (1978) "information concepts for information science", journal of documentation, vol 34, pp 55-85. brookes, b.c. (1980) "the foundations of information science. part i. philosophical aspects", journal of information science, vol 2, pp 125-133. busby, j.s. & hutchison, d. (1992) "the practical integration of manufacturing applications", software practice and experience, vol 22, pp 183-207. croft, w.b. & turtle, h. (1989) "a retrieval model for incorporating hypertext links", in hypertext '89 proceedings, pp 213-224, new york: the association for computing machinery. cusack, e. (1991) "inheritance in object oriented z", lecture notes in computer science, vol 512, pp 167179. davis, e. (1986) representing and acquiring geographic knowledge, new york: pitman. duke, d. & duke, r. (1990) "towards a semantics for object-z", lecture notes in computer science, vol 428, pp 244-261. duran, b.s. & odell, p.l. (1974) cluster analysis: a survey, berlin: springer-verlag. eccles, j.r. & saldanha, j.w. (1990) "metadata-based generation and management of knowledgebases from molecular biological databases", computer methods and programs in biomedicine, vol 32, pp 115-123. everitt, b. (1974) cluster analysis, london: heinemann. fogg, i.s., ruggles, c.l.n. & newman, i.a. (1993) "a distributed metadata system", research report 34, midlands regional research laboratory, university of leicester and loughborough university. glushko, r.j. (1989) "design issues for multi-document hypertexts", in hypertext '89 proceedings, pp 5160, new york: the association for computing machinery. hall, a. (1990) "using z as a specification calculus for object-oriented systems", lecture notes in computer science, vol 428, pp 290-318. hsu, c., bouziane, m., rattner, l. & yee, l. (1991) "information resources management in heterogeneous, distributed environments: a metadatabase approach", ieee transactions on software engineering, vol 17, pp 604-625. ingwersen, p. (1992) "information and information science in context", libri, vol 42, pp 99-135. ishikawa, h. et al (1993) "the model, language, and implementation of an object-oriented multimedia knowledge base management system", acm transactions-on database systems, vol 18, pp 1-50. lillywhite, j. (1991) "identifying available spatial metadata: the problem", in d.j. medyckyj-scott, la. newman, c.l.n. ruggles, and d.r.f. walker (eds), metadata in the geosciences, pp 3-12: group d publications. 50 ajis litwin, w., mark, l. & rossopoulos, n. (1990) "interoperability of multiple autonomous databases", acm computing surveys, vol 22, pp 267-293. lyotard, j.-f. (1979) the postmodern condition: a report on knowledge, manchester university press. madsen, m.s., fogg, i.s. & ruggles, c.l.n. (1994) "metadata systems: integrative information technologies", libri, vol 43, in press. neelameghan, a. (1992) "application of ranganathan's general theory of knowledge classification in designing specialized databases", libri, vol 42, pp 202-226. quillian, m.r. (1985) "word concepts: a theory and simulation of some basic semantic capabilities", in r.j. brachman and h.j. levesque, (eds), readings in knowledge representation. los altos, ca: morgan kaufman. ranganathan, s.r. (1967) prolegomena to library classification, asia publishing house. ruggles, c.l.n. (1994) "an integrated environment for open learning: the stile approach", in proceedings of the eleventh international conference on technology and education, london. ruggles, c.l.n., newman, i.a., medyckyj-scott, d. & walker, d.r.f. (1992) "structuring spatially related meta-information for effective entry and retrieval", the computer journal, vol 35, pp a475-a486. sowa, j.f. (ed) (1991) principles of semantic networks: explorations in the representation of knowledge, san mateo, ca: morgan kaufman publishers, inc. stepney, s., barden, r., & cooper, d. (1992) "a survey of object orientation in z", software engineering journal, vol 7, pp 150-160. van ryzin, j. (ed) (1977) classification and clustering, new york: academic press. september 1994 51 ajis special edition-electronic commerce november 1998 the evolution of electronic marketplaces: an exploratory study of internet-based electronic commerce within the american independent insurance agency system andrew f. burton andersen consulting chicago, illinois usa john g. mooney* the michael smurfit graduate school of business university college dublin (national university of ireland, dublin) carysfort avenue, blackrock, dublin, ireland. * correspondence should be addressed to dr. john mooney abstract evolving competitive, organisational and technology environments drive organisations to continually evaluate how information technology resources can be exploited for competitive value. the emergence of a global information infrastructure has spawned an entirely new area for competitive exploitation, electronic marketplaces. this paper first presents a review and synthesis of the conceptual and empirical literature on electronic marketplaces. drawing upon multiple theoretical perspectives, we develop a framework for analysing their potential influences on market and industry structure. this conceptual framework is then applied to an empirical assessment of the potential of the internet as an electronic marketplace for the american independent insurance agency system. "it was no accident that salesmanship had a renaissance in the merchandising of insurance, the invisible commodity. for insurance was complicated and required detailed explanation to everybody, but especially to the democratic millions who in america would have the opportunity to become insurance customers... the abstract and complex qualities of insurance together with its intimate and personal aspects, continued to keep it as a last preserve for the salesman against advertising" daniel boorstin, librarian of congress introduction coase (1937:22) defines the firm as consisting of "the system of relationships which comes into existence when the direction of resources is dependent on an entrepreneur". for over half a century, scholars have followed coase's insight into the nature of the firm by recognising firms as collections of activities that are coordinated together to produce, market, deliver, and support the firm's product or service (williamson 1975; porter 1985; mines 1993). the manner and structure through which a firm manages its relationships with its buyers and suppliers form its value system. the implementation of value systems was partly facilitated by the development of information systems that cross organisational boundaries. inter-organisational information systems, first described by barrett and konsynski (1982), electronically link firms to their suppliers and distributors. reported benefits from inter-organisational systems include increased efficiency of order processing, reduced costs from just-in-time inventory management, locking in trading partners through network participation, customisation of products and services, improved coordination at the interface between the customer and supplier, improved organisational efficiency, better information management, and lower overall co-ordination costs (bakos 1991; steinfield, kraut and plummer 1995). these electronic interactions provided the early foundation for electronic marketplaces. within the broader sphere of electronic commerce, advancements in information technology have influenced the different mechanisms that co-ordinate the flow of information and commerce. williamson (1975) proposed that firms exist to reduce the cost of negotiating, monitoring and executing transactions that are required to acquire goods and services in the open market, with the distinction between a market and a hierarchy based on the economic location of production. firms choose vertical integration to control transaction processes when transactions costs are excessive while choosing to participate in markets when transaction costs are low (nouwens and bouwman 1995). with the introduction of computing and communications technologies into the processes through which firms manage and co-ordinate their transactions, traditional hierarchical and market co-ordination mechanisms are affected, culminating in electronic hierarchies and electronic markets (malone, yates and benjamin 1989). bakos (1991) defines an electronic hierarchy as an electronic link that serves as a tool for bilateral information sharing and defines an electronic market as a market system that allows buyers and sellers to exchange information about market prices and product offerings, thus representing an investment in multilateral information sharing. as the presence of electronic hierarchies and electronic markets has increased in the commercial environment, different theories have emerged to explain their use and future direction. 20 ajis special edition electronic commerce november 1998 the electronic markets hypothesis (emh) predicts that the effect of technology on market mechanisms will increase the proportion of economic activity coordinated by markets, with a shift from hierarchies (malone, yates and benjamin 1987). the theory states that advances in computing and communications technology will facilitate an evolution from electronic hierarchies, through biased and unbiased electronic markets, to a personalised electronic market. 1 examples offered by malone et al. to support the emh include airline reservation systems, and companies such as ibm, xerox, and general electric (1987,490). the move to the middle hypothesis (mmh) offers a refinement on the emh. the mmh recognises advances in computing and communications have affected the costs of co-ordination in which value chain restructuring will be characterised by increased outsourcing and an emphasis on close, long term relationships with suppliers (clemens, reddi and row 1993). in short, the mmh predicts the movement of activity along the electronic hierarchies/markets spectrum will be towards the middle of the spectrum. thus, the reduction in transaction costs will not cause an increase in the optimal number of suppliers, rather buyers and suppliers will work closer together and develop longer term relationships. in limiting the number of suppliers, a firm can be motivated by non-contractible investments, such as quality, responsiveness, trust, information sharing and innovation (bakos and brynjolfsson 1993). it has been proposed that information technology investments are increasingly becoming non-relationship specific, and thus, will be made with fewer suppliers and for longer time periods due to economies of scale, incentives to suppliers, reduced benefits of search, return on investment, learning curve effects and other incentives (demons, reddi and row 1993). bakos and brynjolfsson (1993) found that although information technology has reduced costs and theoretically allows for more suppliers, the benefits gained by closer buyer-supplier relationships offset the possible benefits of more suppliers (1993). these relationships of few suppliers with increased co-operation and interaction between buyers and suppliers have emerged as value-adding partnerships (demons, reddi and row 1993). electronic market characteristics to understand the cost factors that buyers consider when participating in electronic marketplaces, it is necessary to consider electronic market characteristics. the above mentioned theories (emh and mmh) attempt to address the business issues organisations face when trying to develop electronic links to buyers and suppliers. along with the logistical and strategic issues, it is important to consider market conditions, specifically whether the organisation is participating in a commodity or differentiated market conditions. bakos (1991b) examined how buyers behave when they search for sellers in commodity and differentiated markets. commodity markets are characterised by identical products offered by like sellers, whereas differentiated markets are more common because they offer a variety of different product offerings to meet the wide range of buyer needs and preferences. in a commodity market, buyers will typically choose the product offered at the lowest cost (including search, transportation and other related costs). commodity markets reveal the following characteristics (bakos 1991b, 298-299): • seller prices decrease as the cost of search decreases • the amount of search increases as the cost of search decreases • the amount of search increases as seller prices become more dispersed • ceteris parabus, as seller prices become more dispersed, buyers' total costs decrease with the introduction of technology into a commodity market and the creation of an electronic marketplace, the ability of buyers to search is increased. thus, the distribution of price and product information can become more efficient and effective. in differentiated markets buyers consider the various price and product offerings available, and determine the product's ability to adequately match their needs. a number of differences exist separating a differentiated market from a commodity market, including (bakos 1991b, 300-301): • "...consumer preferences determine the location of their ideal product or product mix" • "...the heterogeneity of consumers and product offerings is central to the ability of sellers to exploit buyer search costs and thus extract monopolistic rents" • "...electronic market systems providing product and price information may generate substantial allocational efficiencies by enabling customers to locate suppliers that better match their needs" 1 in a biased market the information in the system and/or the system structure favors a particular organization or person, whereas in an unbiased system die information and system structure are unbiased towards the market participants. a personalized electronic market provides customized information tailored to the system user's needs. 21 ajis special edition electronic commerce november 1998 a buyer's search process is very complex with product offering and search cost factors interacting differently. however, with the introduction of information systems into a differentiated market to create an electronic marketplace, it is suggested substantial impacts and potential changes occur to organisational and market structures. the introduction of agency theory (i.e. the recognition that the operation of a market is not costless, as is assumed in classic economic literature, and consequently that a firm can be a substitute for the market mechanism created to reduce the costs of transactions) shows that a large amount of a firm's costs are related to the acquisition and processing of information and can therefore be reduced by application of information technologies (gurbaxani and whang 1991, 71). thus, with the introduction of organisational information systems, firms not only have the potential to reduce internal costs, but can also, with the development of customer based inter-organisational information systems, improve their ability to access a wider range of buyers due to reduced buyer search costs. as discussed previously, whether the seller is participating in a differentiated or commodity market influences the manner through which buyers search for a product or product mix. the role of an information system to distribute product/service and price information through an electronic market can serve to improve allocational efficiencies and decrease overall cost. allocational efficiencies and cost reductions are attributed to three distinct affects information technology has on organisational and market structure (malone, yates and benjamin 1987,488-489): 1. electronic communication effect information technology can allow more information to be communicated in the same amount of time while decreasing the costs of this communication dramatically. 2. electronic brokerage effect — electronic markets, by electronically connecting many different buyers and suppliers, can assist the matching of buyer preferences and supplier characteristics, thus increasing the number of alternatives that can be considered, increasing the quality of the alternative eventually selected, and decreasing the cost of the entire product selection process. 3. electronic integration effect — information technology can be used not just to speed communication but to also change the processes that create and use the information. the three electronic effects described by malone et al. contribute towards illustrating how information systems, deployed and used to create an electronic marketplace, have the potential to improve organisational and industry efficiency and reduce costs. two factors are noted to influence the utilisation of the electronic marketplace and its development as either a hierarchy or a market. "two... more specific factors that can be changed by information technology are important in determining which co-ordination structures are desirable: asset specificity and complexity of product description" (malone, yates and benjamin 1987,486).^ these factors are suggested to influence whether a product or service is biased towards participating in an electronic market or an electronic hierarchy. it has been proposed the more complex and asset specific a product or service the more likely it will be suited for an electronic hierarchy rather than an electronic market. assuming a product or service is suited for an electronic marketplace, the types of characteristics an electronic market demonstrates are important for the seller to understand and manage. bakos (1991, 1991b) identifies certain characteristics of electronic markets: • reduction in buyer search costs and increased ability to search for product or product mix information and price • promotion in price competition among sellers • reduction in supplier market power • reduction in buyer search time • reduction in seller time to communicate product and price information • the benefits of participation hi the electronic market increases as participants increase • participants are influenced by costs that could be incurred by switching from one electronic market to another • large amounts of initial capital investment required for participants • the benefits for market participants are uncertain • the electronic market offers economies of scale and scope • the timetable for change in an electronic market is uncertain 2 asset specificity includes site (whether a resource can be moved), physical (a tool or resource designed for a single purpose), human (a physical or mental skill that can not be readily applied to another purpose), and time (its value is highly dependent on a specific time period of use). complexity of product description refers to the amount of information needed to specify the attributes of a product in enough detail to allow potential buyers to make a selection. 22 ajis special edition electronic commerce november 1998 along with the market characteristics previously described, other network attributes interact to influence the promotion and use of hierarchical and market structures (steinfield, kraut and plummer 1995), including: • in addition to multiple buyers and suppliers, a need exists for some form of third party market maker to create a network to facilitate search and product/service matching • the network's ability to allow rich product descriptions for complex products • network openness to allow easy communication with new participants, public protocols widely accepted and adopted by market participants, widespread participation, and low costs for new entries onto the network • the extension of open networks to end consumers to support electronic markets • public networks reduce the constraints on connecting uncertain, low volume trading partners or potential partners the characteristics and attributes of electronic networks have been forwarded to help practitioners and researchers understand the nature an electronic market and related environmental conditions. such an understanding not only furthers the research into electronic marketplaces, but also assists in providing a strategic perspective in which to assess electronic markets. market intermediation and disintermediation it is widely argued that an inevitable effect of electronic commerce will be disintermediation (i.e., removal of the "middlemen"). however, other claims have been made that electronic commerce will not result in disintermediation, rather intermediation will continue to prosper and provide a valuable role in electronic commerce. the importance of understanding electronic markets and their effects is increased when one examines how they influence organisational functions and decision processes, also known as the decision to either "make or buy." transaction cost theory (tct) states that a firm has two options for organising its economic activities: the internal hierarchy of vertically integrated processes of the organisation, or the reliance on external organisations in the open market to perform business processes (coase 1937, williamson 1975). with the link of tct to electronic markets (malone, yates and benjamin 1987), researchers have argued that as costs are reduced by information systems, the systems can be extended to reach end consumers (benjamin and wigand 1995). thus, with the introduction of inter-organisational systems and the creation of electronic markets the manner through which industries are structured and firms relate to buyers and suppliers is questioned. researchers have attempted to understand the role intermediaries will have in inter-organisational systems as they emerge into electronic marketplaces. benjamin and wigand (1995b) and thornton (1994) explore how industry value chains and transaction patterns can be affected by electronic markets (figure 4.1). in a digital environment, producers originally limited to a distribution network of wholesalers and retailers, can bypass those members of the value chain to directly interact with members further down the chain or sell directly to end consumers. producer producer producer figure 1 value chains and transaction patterns source: adapted from benjamin and wigand (1995b): "electronic markets and virtual value chains on the information superhighway", sloan management review. winter, 62-72, and thornton (1994): "revolution in japanese retailing", fortune magazine. february, 144 some researchers have explored why and how producers will attempt to directly interact with end consumers. sarkar, butler and steinfield summarise the research best by stating, "...one fundamental question, therefore, is to what extent producing organisations would take advantage of direct electronic links with consumers, and whether in the process, intermediaries will be eliminated from the value system" (1995, 3). 23 ajis special edition electronic commerce november 1998 benjamin and wigand (1995) clearly state that market intermediaries are threatened by the national information infrastructure (i.e. an electronic market) because of market benefits, "...consumers will choose alternative forms of transactions in favour of price, high quality, selection choice, and time savings" (1995b, 70). the threatened intermediary hypothesis (tih) states that an electronic market network can be extended to directly link consumers with producers, and in doing so, lower transaction costs producers incur when marketing and selling to end consumers (sarkar, butler, and steinfield 1995). for example, in a three member value system (figure 4.2), possible transaction costs and intermediary value dictate which channel a producer uses to market and sell products for consumption. if the cost of transaction one (tl) is more than transaction two (t2) and transaction three (t3) combined, than the intermediary is valued and used. however, if the cost of tl is less than t2 and t3 and the market value of the intermediary is not needed, than the intermediary is bypassed. intermediary producer •—•• e consumer tl figure 2 value system transaction patterns source: adapted from sarkar, mirtra b., brian butler and charles steinfield (1995): "intermediaries and cybermediaries: a continuing role for mediating players hi the electronic marketplace", journal of computer mediated-communication. 3, 1-17 [url:http://jcmc.huji.ac.il/| benjamin and wigand noted the threat of disintermediation to market participants by stating, "...all the stakeholders in an industry value chain must consider whether their place in the chain is threatened and, if so, what long term strategies to experiment with" (1995b, 71). however, "...rather than the highly abstracted view of intermediaries as providing a single unified service known as 'co-ordination', the roles of intermediaries must be considered in more detail in order to understand the impact of the nil (i.e. an electronic market)," including the following multifaceted functions (sarkar, butler and steinfield 1995,7-11): • search and evaluation • needs assessment and product matching • customer risk management • product distribution • product information dissemination • purchase influence • provision of customer information • producer risk management • transaction economies of scale • integration of consumer and producer needs bakos and bailey (1997, 2-4) also addressed the issue of market value provided by market intermediaries and noted intermediaries provided the following five functions: • aggregate buyer demand or seller products to achieve economies of scale or scope and to reduce bargaining asymmetry • protect buyers and sellers from opportunistic behaviour of other participants in a market by becoming an agent of trust • provide marketing information to suppliers • facilitate the market by reducing operating costs • match buyers and suppliers with the advancement of multidimensional functions as added value intermediaries provide hi an industry value system, a more in-depth analysis and understanding is needed to understand how electronic markets can lower transaction costs and support value adding functions previously provided by market intermediaries, or the possibility that new intermediary roles and value added functions will emerge for market participants. sarkar, butler and steinfield suggested "...not only is it likely that widely available information infrastructures will reinforce the position of traditional intermediaries but the networks will also promote the growth of a new generation of intermediaries" (1995, 1). 24 ajis special edition electronic commerce november 1998 sarkar, butler and steinfield argued intermediation will continue to provide a valuable role in industry value systems, and their analysis suggested electronic markets can lead to four possible outcomes (figure 4.3): a reinforcement of existing direct producer to consumer links, the use of an electronic market by producers to bypass intermediaries and link directly to consumers, a reinforcement of existing intermediary structures, or the emergence of new electronic market based intermediaries (1995). the four outcomes model (figure 4.3) is based on transaction costs and intermediary value (figure 4.2). tkt2+t3 pre-electronic market t1>t2+t3 tkt2+t3 post-electronic market t1>t2+t3 i. electronic market supplemented direct market m. cybermediaries n. threatened intermediaries rv. electronic market supplemented intermediaries figure 3 electronic market intermediary outcomes source: adapted from sarkar, mirtra b., brian butler and charles steinfield (1995): "intermediaries and cybermediaries: a continuing role for mediating players in the electronic marketplace", journal of computer mediated-communication. 3, 1-17 [url:http://jcmc.huji.ac.il/] an interesting possible outcome was noted in which scenario iii would occur where a pre-electronic market structure consisted of direct relationships but in a post-electronic market the emergence of cybermediaries occurred to take advantage of electronic market opportunities (sarkar, butler and steinfield 1995). the ability to link directly with consumers not only has the potential to threaten intermediaries, it also has the potential to negatively impact sellers. formal support for this claim has been developed (bakes 1996, 1991), with potential impacts including: • promotion of price competition • reduction in market power of sellers • passage of system benefits from sellers to buyers • creation of more demanding customers less likely to compromise on preferred products/services academic researchers have suggested the following strategies and suggestions to anticipate and respond to the negative impacts of electronic markets (bakos 1996,1991, 1991 b; malone, yates and benjamin 1987,496-497): • seller control of the type of system introduced • development of a system that emphasises product information not price information • obtain early mover advantage to establish market leadership and obtain large market share • increase product/service differentiation to make it difficult for buyers to compare prices and product offerings (also increases buyer search costs) • sellers or developers of electronic markets should not expect indefinite competitive advantages from the system • improve underlying economies of scale by leveraging development expertise, installed networks, customer bases and proprietary databases • assess the point of customer contact to determine its ability to serve as a critical resource to determine electronic market benefits, participation and sustainability of profits with the emergence of such proposed negative impacts of electronic markets on sellers and intermediaries, researchers have questioned the motives for organisations and individuals to establish such markets. the theory of the prisoner's dilemma that sellers as a group do not benefit from an electronic market system, but as individuals the revenues that can be generated from such a system can be beneficial and outweigh the losses, 25 ajis special edition electronic commerce november 1998 assists in explaining business motivations for electronic market makers (bakos 1991). such market makers have been suggested to pursue revenue generation and increased market growth at the expense of the industry as a whole. as electronic market usage and utilisation improves by different industry participants the likelihood of electronic market participation becoming a strategic necessity and an integral part of industry infrastructure increases (bakos 1991,308). additionally, market makers often have an opportunity to create a system biased towards a given product or firm. as noted by benjamin and wigand, "...if a market maker owns or has a substantial interest in any suppliers, it can bias the market in their favour and both the consumer and other suppliers will be disadvantaged" (1995, 11). benjamin and wigand also suggest the overall market maker effect was a "...phenomenon in which the consumer does very well, producers lose their profit margins, and the market makers gain the remaining profits" (1995b, 66). finally, as noted by bailey and bakos, "...a newer paradigm of customised goods, reduced delivery time, and greater customer satisfaction may be more difficult to measure, but may be the one appropriate to use for electronic markets" (1997,13). the american independent insurance agency system the emergence of independent insurance agents in the united states began between 1795-1796 when insurance companies experimented with the appointment of exclusive agents (payan 1993, 2). mehr and cammack (1980) recognise an insurance agent as anyone authorised by an insurer to solicit, create, modify, or terminate contracts of insurance between the insurer and the insuring public, and includes the following characteristics (1980, 99101): • an agency relationship is established between principals and agents by mutual assent given in an express agreement known as an agency contract • an ability to establish a presumptive agency in which an insurer's behaviour causes a reasonable person to believe that an individual is an agent of the company the costs of agent services are covered from the premiums paid by insureds, with differences in premium levels based on differences hi agency services provided to the consumer. insurance distribution costs are reduced by shifting part of the servicing burden to consumers where they determine how much service they wish to buy and supply for themselves (mehr and cammack 1980, 554). marketing insurance company products can be done through various distribution channels, including the agency system which relies on agents to sell the insurance products, direct selling in which insurers write business directly to consumers, and mass merchandising in which selling and underwriting is conducted in a group or collective basis (mehr and cammack 1980, 565). the focus of this paper will be the american agency system where companies use independent agents to market and sell insurance products and services.-* a majority of property and liability insurers in the united states currently operate in the american independent agency system, in which: • insurers operating in the system are independent agency insurers • agents are independent from companies and can represent multiple insurers • agents issue policies, collect premiums, and retain exclusive rights to solicit renewals from customers a white-paper for the national association of insurance commissioners (naic) found the internet as a convenient way for consumers to learn about insurance products, sources, and pricing; the marketing of insurance over the internet offers insurance regulators an additional opportunity to monitor insurance market conduct; and a means for companies to use a new distribution channel which could incorporate all facets of an insurance transaction (n.a.i.c. 1997) the naic has suggested the internet serves four functions for the insurance industry (naic 1997): 1. single source sales sitein which a single insurance company markets its products and services over the internet though a www homepage; has the ability to allow consumers to purchase insurance directly from the company and provide on-line requests for quotes; and offers a medium for companies to work with agents to educate and sell to consumers 2. insurance mallsoffers a site where sellers from within one industry or across multiple industries can market and sell their insurance products and services 3 at present there are three main sources for insurance in the us: captive agents who can sell insurance for only one company, telephone insurance representatives who can offer insurance of one company, and only over the telephone, and independent insurance agents who represent multiple insurance companies and provide insurance products based on price, coverage, and service. 26 aj1s special edition electronic commerce november 1998 3. internet insurance servicingproviding full servicing capabilities to the insurance consumer during all facets of the insurance transaction 4. company agent communications to serve as a tool to enhance insurance company and agency communication and information exchange in addition to the four possible internet functions serving the industry, the naic suggests the internet provides consumers, regulators, and the industry with a number of advantages and disadvantages (table 1). table 1 internet insurance advantages and disadvantages insurance segment advantages disadvantages consumers • product and information accessibility • agent & product search • customer service • lower costs lack of personal interaction unfamiliarity with laws & regulations potential lack of market access regulators • electronic access to consumers and industry • sales and solicitation electronically monitored potential lack of market access explosive growth in use without adequate resources regulatory overview problems increased fraud industry electronic communication and transactions reduced administrative and operating costs target market opportunities improved matching of needs with products problems with regulatory compliance physical market access to emerging markets solicitation violations applicant qualification for insurance advertising laws and regulations source: adapted from national association of insurance commissioners (naic)(1997): the marketing of insurance over the internet. draft: june 4 a number of recent insurance industry movements indicate the internet is having a profound impact on how different organisations are preparing for the future, including; • the independent insurance agents of america web site connects to www homepages for more than 22,000 independent agencies on the internet (banham 1997) • insweb [www.insweb.com], a year old company that intends to offer web-based comparative shopping of insurance products and quotes and has invested at least $10 million into its web site, and quickquote, are building a presence on the internet and recruiting organisations to participate in their services (brandel 1996, halper 1997) • internet technological advances are offering insurance companies opportunities to cut their distribution margins by 15-18 percent (higgins 1996) • six insurance companies have created an alliance, know as the world insurance network, to build a virtual private network with secure internet "hooks" that will enable them to exchange e-mail and provide the foundation for electronic commerce (cole 1997) research methodology this paper aims to identify whether emergent evidence of the following electronic market attributes and effects exist within the context of internet-enabled markets in the insurance sector: • electronic communication effect • electronic brokerage effect • electronic integration effect 27 ajis special edition electronic commerce november 1998 • effect of product characteristics • impact on market structure and process • economics of market participation due to the early stage of theory building in this area, the research questions were explored through qualitative research categorised as theory building case studies in which constructs and propositions emerge inductively through the process of fieldwork (eisenhardt 1989, swamidass 1991). while inductive theory building from case studies increases the likelihood of generating novel theory, eisenhardt (1989) identifies several possible weaknesses. to offset potential methodology weaknesses a number of steps were taken: • constructs and propositions studied were obtained from previous conceptual and empirical research regarding electronic markets • these were used to develop an interview guide containing a set of exploratory and qualitative questions, supplemented by open-ended questions • initial testing of interview guides was conducted with insurance industry experts from the independent insurance agents of america, inc. • the targeted sample was obtained from participation and invitation lists from past industry events and conferences related to electronic commerce, the internet and the independent agency system • targeted respondents were senior and executive managers with .direct involvement in internet related insurance issues • organisations contacted participated in a broad spectrum of markets with national, regional and local business concerns • information gathered was collected through extensive telephone interviews and follow-up conversations • in order to reduce respondent bias and response error, information collected was not attributed or recorded as individual or organisational specific • interview results were reviewed with respondents to ensure factual consistency sample selection as previously mentioned, to assist in reducing possible weaknesses in the study the sample population was chosen from insurance companies (carriers) participating in the american independent insurance agency system. these insurance carriers use independent insurance agents to distribute, market, and sell their products were chosen due to their position in the industry value chain. the perspective of insurance carriers was deemed to be particularly valuable due to their ability to potentially change distribution channel methods should market and competitive forces change. also, the companies targeted have all created a presence on the world wide web. to identify key informants with knowledge of insurance, the independent agency distribution channel, electronic commerce and the internet, respondents were selected from relevant industry event and coalition lists. respondents were senior and executive level managers with direct involvement in internet related insurance issues from established, well-rated insurance organisations with substantial asset levels firms participating in a range of markets with national, regional and local business concerns. data collection to facilitate maximum participation in the study from the sample population numerous steps were taken. pretesting, relying on colleagues, respondent surrogates, or actual respondents to evaluate and refine the interview guide, was conducted to examine respondent interest, meaning, question transformation, continuity and flow, question sequence, skip patterns, variability, and length (cooper and emory 1995,317-321). study respondents were contacted over the telephone to confirm their ability to provide adequate target data. initial telephone contact consisted of a brief introduction, the purpose of the study, estimated time for the interview and a summary of topics to be covered during the interview. specific appointments were made with the respondents to arrange for an appropriate interview time and setting. on subsequent contact to conduct the interview, the respondents were reminded of the purpose and nature of the study, and topics to be discussed. before starting the interview the researcher asked respondents if they had any comments or questions regarding the process. during a three week time period a total of 19 respondents from 17 organisations were interviewed for one hour each. following the interviews, data gathered was reviewed with respondents to ensure factual consistency. 28 ajis special edition electronic commerce november 1998 research findings the findings clearly indicate insurance companies are integrating the internet and internet-related technologies into their business processes, and are moving towards increased reliance on the internet to serve as an electronic commerce medium. electronic communication effect the refers to the internet's ability to communicate more information in the same amount of time and decrease the costs of communication (malone, yates and benjamin 1987). after reviewing the interview data the findings can be grouped into the nature of communication, scope and structure, types and alternative means of communication. nature of communication the internet's impact on organisational communication was found to vary according to organisational perspective. the primary finding in this area, as stated by one respondent, was, "...use of the internet is definitely a revolution taking place within forward-thinking organisations, especially with the strategic development of intranet and internet technologies." the previous statement was supported by the following data: • assuming proper management methods, respondents considered that the costs of industry communication will decrease dramatically as more information is distributed through the internet (attributed to reductions in content production, distribution, postal delivery, and content maintenance costs) • communication effectiveness will increase as information is customised and tailored to individual and organisational needs • the internet has the potential to service the insurance triad (consumer, agent and company) as an open communications network with platform independence • the internet allows for more timely, convenient, and current information to be communicated for customer consumption however, with the change in the nature and medium of communication, other issues of concern emerged. respondents were concerned about long term issues such as information structure and engagement, education processes, and servicing abilities. these dimensions of the internet remain superficially explored and were noted as needing further analysis to be fully understood. another concern was the internet's current passive nature in which information providers are given little control over information consumption. currently the manner in which information is accessed and consumed is determined by internet browsers. thus, the value of communication structure was seen as becoming increasingly important. communication scope and structure the internet's national and international user base allows organisational information to be communicated to a broader audience and in greater depth. while the manner in which information is structured, integrated and tailored was identified as being very important, there was little agreement across respondents on the nature of these. for example, some respondents felt digitising hard copy brochures for internet distribution was appropriate, while others felt it to be "shovel-ware" in which information overload and ineffectiveness were end results. in short, professional, readable web sites were seen as difficult to structure to provide organisational value, with the tendency for such sites to require large amounts of organisational resources from companies. means of communication the internet's ability to facilitate an alternative means of communication with agents and consumers was found to be beneficial. for example, one respondent mentioned his company's plan to develop "...multimedia product information that can be downloaded for agency and consumer use." the internet's multimedia capabilities provide carriers with an ability to deliver multifaceted and multidimensional information for consumption. the limits of the internet were acknowledged (i.e., bandwidth, reliability, access, etc.) but the network's potential to advance and improve the current communication system was cause for optimism among respondents. however, the internet's ability to fully serve as a means of communication with consumers and agents and replace traditional methods was viewed with scepticism, especially with the current business value seen to be provided by independent agents. 29 ajis special edition electronic commerce november 1998 conclusion overall, the "electronic communication effect" of the internet was perceived as positive. respondent data indicates organisational adoption of the internet into business processes and strategic objectives. all respondents provided information as to how they plan to incorporate internet communication into their businesses though the degree of involvement varied between simple web site advertisement and in-depth agentcompany interactivity, faith and confidence in the internet's ability to fully serve as an electronic communications medium seemed to be limited at present due to technological and market barriers. the communication benefits noted lead to consideration of the internet as a medium to supplement existing communication and distribution channels used by the companies to communicate with agents and consumers. a numba of issues emerged during the interviews regarding the electronic communication effect which remained; manswered and warrant further study, including: • an adequate method to measure the value that an organisation receives from communications over the internet was not provided and remains unclear • the total reductions of organisational costs of internet communication need to be compared against the additional costs for staff internet training and support (especially related to production and maintenance of internet communication) • the degree of information engagement by consumers on the internet remains to be an issue of speculation electronic brokerage effect this section of the study attempted to explore the internet's ability to act as an electronic market connecting many different buyers and suppliers, thus increasing the number of product alternatives that can be considered, increasing the quality of the alternative selected, and decreasing the cost of the entire product selection process (malone, yates and benjamin 1987). the internet's growth and development as an electronic market has led to the emergence of new target markets. specifically, the "book-end effect" emerged as an important concept, which holds young, educated people and older, more established individuals participate in the internet's growth, but a large number of individuals are in the middle of the two groups and remain "unconnected" to the internet. the electronic brokerage effects can be grouped into the search process, consumption process and market outcomes, and market structure. search process the internet was seen as a means to facilitate the search process, especially as the insurance market evolves towards more educated and "technology savvy" individuals. such an emerging market was specifically mentioned as a target for companies to address with new resources and business strategies on the internet. the internet was seen as providing consumers with an easily accessed and potentially more efficient medium for insurance product education and selection. for example, medical savings accounts (msas) are a new product that has emerged in the us health insurance market during the last few years. msas can be actively researched, analysed and selected with the internet. the internet's ability to serve as an electronic broker was seen as a means to increase the level of consumer and agent knowledge of insurance products and servicing. the accessibility and convenience of insurance information on the internet provides consumers with a means to "...look for what they really want to know." the emergence of push technology to provide insurance consumers with information they want to receive (an example given included point cast [www.pointcast.com] tailored to insurance). additionally, the increase in the number of alternatives that can be considered was seen to improve insurance education and understanding, provide a means for comparison against captive agents, offer a means to fight for increased market share with better price and product offerings, and allow consumers to improve their ability to match their needs with an appropriate insurance product. the need for filtering mechanisms to provide balance between information available and information needed at a one-to-one mass communication level emerged as a significant issue. respondents noted the challenge the internet poses to companies interested in not overwhelming consumers with technical and non-technical insurance information, and yet differentiating themselves from the masses of potential information that exists and increases daily. an improved information delivery and filtration mechanism to facilitate buyer and seller matching and consumer understanding of insurance products was seen as important to promoting further internet market development, as one respondent said, "we need to go beyond search engines and directories to form customised technology that offers a unique point of view for customers." 30 ajis special edition electronic commerce november 1998 consumption process an area of debate was how insurance products and services on the internet would be consumed and the possible market outcomes. two possible consumption processes emerged during the study: • insurance products and services would be located and researched on the internet, after considering possible alternative insurance products, additional information and assistance would be sought through a retail outlet or personal contact, and the transaction would be continued "off-line" • insurance products and services would be located and researched on the internet, after considering possible alternative insurance products, additional information or guidance would be sought online, and the insurance transaction could be completed either on the internet or "off-line" market outcomes from these two possible consumption processes, two possible market outcomes were identified. one market outcome was seen to lead to disintermediation supported by internet development and the emergence of on-line intelligent software agents. the other market outcome was seen to result in a large number of people relying on personal interaction through traditional market mechanisms. it should be noted that these outcomes represent endpoints in a market outcome spectrum in which different market strategies could emerge. disintermediation is discussed in more detail later in the paper. respondents generally cited the following market outcomes for insurance industry members participating on the internet: • reduction in insurance market barriers resulting in increased electronic marketplace competition • customisation of products and services among solutions that have emerged, the on-line insurance mall was seen as the least promising. the need to understand consumer demographics and needs, and market on-line insurance products accordingly was seen as a promising objective. examples of such marketing techniques were the bundling of insurance information with affiliated web sites and news groups. additionally, the potential for other financial service firms to enter into the internet insurance marketplace was seen as an emerging reality. while respondents were aware of the need to differentiate their organisations from other organisations participating on the internet, the approach for doing so remained firm specific. conclusion the internet's ability to connect many different buyers and sellers of insurance was clearly recognised by respondents. the fundamental business concern that emerged regarding commerce on the internet was how to separate one's organisation and product offerings on the internet from rival organisations on the internet. the filtering mechanism respondents mentioned as necessary for effective and efficient matching of consumers with products have not yet matured, though recent developments in agent-mediated electronic commerce suggest progress is being made in this area. the respondents acknowledged the internet's ability to serve as a local, national and international interconnected marketplace matching buyers and suppliers. however, the business value and implications of participating in electronic commerce over the internet remained organisational specific and tied to strategic speculation and inference. electronic integration effect this section of the interview guide attempted to explore the internet's ability, not just to speed communication, but to also change the business processes that create and use information (malone, yates and benjamin 1987). by asking respondents for their opinions as to how the internet can change their business processes, insight was gained into how use of the internet as an electronic market changes the role information technology and information in their firms. the findings from this section were grouped into business process impacts, customer support, organisational speed, distribution and information use. business processes the development of internet and intranet technologies was seen as a tool for organisations to change their business processes to respond to new and emerging market needs. however, the theme that "...companies and agents are like dinosaurs, a lot of them aren't up to speed and have plenty of obstacles in front of them," emerged as a common view among respondents. 31 ajis special edition electronic commerce november 1998 the internet is allowing many of the participating firms to rethink the best manner for them to organise and conduct business, especially those functions which are supported by information technology. such thinking led respondents to provide the following views regarding internet technologies: • the incorporation of data warehousing, databases, and decision support systems with internet technologies increases organisational ability to improve business practices through work-flow reengineering and building new business applications. • as a technology infrastructure for business processes, the internet and related technologies are more affordable and flexible (i.e., open standards, browser technology, accessibility, etc.). • the internet facilitates the crossing of organisational boundaries and the seamless integration of vendor and agent operations. • potential business processes for change include enabling streamlining of functions, increased speed to market, and on-line claim status checking and coverage changing abilities. customer support the internet's ability to serve as a tool for companies to build customer support and service emerged as an immediate target for organisational resources. during the interviews it became clear that interconnected communications links between agents, companies and consumers could benefit carriers immensely. the targeted benefits varied between respondents, but some common areas of possible interest for improvement were: • the internet to serve as a tool to support and enable agents and consumers (example: obtaining quotes faster, more in-depth marketing information,) • increasing consumer comfort with on-line insurance servicing • increased consumer servicing and communication (especially with claim reporting, billing status, claim status, etc.) • improving agent-company relationships • building carrier reputation and brand recognition on the internet organisational responsiveness an issue that emerged during the interviews was organisational speed in meeting consumer needs and responding to market changes. the prospect of having consumers and agents interacting with organisational members instantaneously through e-mail rather than the traditional postal system has begun to encourage the organisations participating in the study to rethink how, where and when consumers interact with them. also, the ability to create a customer and agency feedback mechanism through the internet to improve organisational learning was seen as an important area for development. distribution the manner through which carrier information, products and services are distributed was provided as an example of how the internet is changing distribution. respondents saw the internet's potential to integrate publications, coverages, agents, purchasing information, relevant news, and other information into a medium of value added distribution, as having a profound impact on their businesses. however, a distinction should be made between those who had strategic, longer term outlooks on the business impacts of the internet and those who had more operational, short term oriented views. the immediate short term outlook on the internet's impact on distribution was seen as serving as an alternative or additional mechanism for agency and company use. conclusion by focusing on how organisations anticipate electronic integration of the internet into their business processes and strategic objectives, insights emerge about how insurance carriers see business processes, customer support, organisational speed, distribution and information use are being impacted by the internet. in summary, it became evident from the interviews that the internet enables not only increased speed of communication, but also can serve to provide a platform for organisations to rethink and re-engineer business processes. effects of and upon insurance product characteristics this section of the study considered the suitability of insurance products for transaction over the internet. to identify respondents views on the suitability of insurance products for an electronic market environment, we examined product characteristics and non-contractible investments. 32 ajis special edition electronic commerce november 1998 insurance product characteristics in an attempt to understand whether insurance products and services distributed through the internet would be viewed as a commodity or differentiated product, respondents were asked for their views on the nature of their products and services. from the insurer and agent standpoint, it was discovered to be advantageous to have insurance products seen as differentiated products. however, the advent of the internet as a communications and information rich medium changes how insurance is perceived in an electronic marketplace environment. personal lines insurance was seen as being closer to a commodity product, with commercial lines insurance being more differentiated. however, it was also found that a spectrum exists along a continuum between personal lines (for example personal auto insurance) as a commodity product and commercial lines (for example commercial coverage against environmental liability) as a differentiated product, with different market segments served by different product and service combinations. additionally, price and quality interact with the different product and service combinations to increase product differentiation. however, the essence of insurance content in different products remains similar due to the manner in which insurance is regulated, thereby restricting the ability of companies to truly differentiate their products from competitors. respondents recognised the potential for opportunistic companies to develop products and services tailored for internet distribution as a possibility. such an internet tailored product would be developed to be clear, simple, and concise to maximise consumer understanding. however, as assistance at the point of sale is improved through internet communications, the long term impact of product characteristics (whether a commodity or differentiated product) was seen to be minimised. also, the development of on-line comparative quoting mechanisms and improved consumer education on insurance products was seen to increase the difficulty for insurers to differentiate their products. finally, the amount of consultation and expertise a consumer requires to purchase insurance also affects how different product and service offerings would be differentiated from competitors. in addition to the whether a product is a commodity or differentiated, other product characteristics influence how the product is distributed in an electronic marketplace (malone, yates and benjamin 1987). the emerging fluidity between traditionally separate financial service organisations and their products have led to an increase in product bundling. with banking, personal finance, and insurance products becoming increasingly packaged together, combined with increased internet usage by consumers, consumer market preferences were found to be changing. the ability to connect and bundle related products on the internet through hyperlinks and web site integration creates an environment in which product marketing and fulfilment could be welded together into one interactive session for the consumer. one primary characteristic of insurance products however hinders the ability to fully integrate insurance with other products on the internet: product complexity. during the study it was found that insurance product complexity influences consumer education, customer communication, and the migration of insurance to an online environment. consumer education is required in order for a potential customer to understand their actual coverage needs, and to assist an insurer in adequately matching a product with the customer's coverage needs. with different market segments having different product needs, the ability for consumers to use the internet to purchase insurance requires a customer to adequately understand various product offerings and select the appropriate product. to improve the product selection and attainment processes on the internet, customer communication was seen as vital. providing a means for effective communication with customers on the internet is essential for carriers and agents to assist customers in understanding and reviewing their product offerings. the interviews also revealed that the nature of the on-line environment supporting complex transactions affects the suitability of different insurance products supplied through an electronic distribution channel such as the internet. to many consumers, the combination of insurance and internet complexity is potentially overwhelming. thus, in addition to simplifying the insurance products offered, the internet "experience" provided by insurance sites needs to be structured to maximise effectiveness. to prevent or at least minimise consumer confusion, the assistance of insurance agents and consumer software to support on-line insurance transaction applications was seen as a requirement. with the assistance of an agent and/or available software applications, consumers could be guided through the process of researching, buying, or adjusting their insurance coverage. an example of a consumer assistance software application could be a down-loadable program to facilitate independent analysis, financial modelling, and/or comparative rating. such applications, combined with hyperlinked, multimedia environments, were found to support companies and their agents' abilities to facilitate electronic insurance commerce. the successful migration of insurance onto the internet was found to have occurred when "...simple and complex insurance transactions were efficiently completed and professional agent consulting moved away from transaction processing towards more value-adding situations of meaningful customer contact and support" the point of customer contact was seen as the most crucial point for effective, efficient and quality service. the ability to use the internet as a tool to add as much value as possible to the point of customer contact represented a valuable opportunity. 33 ajis special edition electronic commerce november 1998 non-contractible investments the concept that organisations and individuals may limit the total number of suppliers considered in order to maximise non-contractible investments, such as quality, responsiveness, trust, information sharing and innovation (bakos and brynjolfsson 1993), was found to be relevant for internet insurance transactions. fundamentally, it became evident that the value of non-contractible investments increases as other differentiating characteristics are eliminated. efforts by carriers to add more value for agents and consumers engaging in electronic commerce were seen as valuable, directly increasing the importance of "soft issues." the emergence of instantaneous transactions and communications with agents and consumers increases the role professionalism, quality and customer service have on the internet. it was seen that the purchase of insurance, as a complex transaction, requires the process to be supplemented with professionalism, increased emphasis on credibility and protection, ease of doing business, personalised service, process simplification and streamlining, personal interaction, responsiveness, innovation and other value adding mechanisms. the challenge presented is how to provide and highlight such "soft issues" on an information and communication network such as the internet. although the fundamental business and underwriting processes do not change with increased internet usage, the delivery and service of those processes are radically altered by the internet. implied trust and credibility are provided through a well recognised and accepted brand name which assists in legitimising new business processes and servicing channels. however, fraud awareness, customer privacy, and information protection were seen to increase in importance. thus, the integration of the internet to support traditional business processes in a new distribution channel requires not only adherence to organisational values and objectives, but also a consistency with the organisation's internet business values and strategic objectives. such a duality of organisational existence between its marketplace and its marketspace was seen as a complex issue that will continue to emerge, and require the creation of new, more flexible business models. conclusion as previously mentioned, this section of the interview guide focused on the suitability and nature of insurance products to be applied to the context of the internet. different lines of products were seen to be more suitable for electronic commerce on the internet, with the characteristics of those products influencing how they are marketed, distributed and serviced. with the inability to readily distinguish one carrier's product from another, the importance of other factors, such as non-contractible investments, was seen to increase. evolving market structure and process this section of the interview guide focused on how the internet may impact market relationships in the independent insurance agency system. issues such as how members in the industry triad will communicate over the internet, possible transaction paths of commerce, and consumer switching costs were explored. in an attempt to understand what type of electronic market characteristics would emerge with the entrance of the independent insurance agency system onto the internet, the role of supplier relationships, buyer demographics, transaction pathways, network switching costs, price and product information, price competition, alliances, and regulation were explored. supplier relationships the respondents viewed the internet as relationship technology where agents and insurance companies can embrace the technology to work with customers to provide products and services according to their needs and preferences. the proprietary issues involved in agent-company relationships encourage closer alignment between the two, which encourages co-operation in introducing and utilising new information technology solutions over the internet. the internet was seen as providing easier connectivity and a tool to supplement existing relationships. buyer demographics another area of importance was recognised to be emerging markets of younger, more technologically exposed individuals, and the need to monitor and respond to their buying preferences and practices. if such an emerging market were to increase in size and attractiveness, the nature of market relationships could easily change. 34 ajis special edition electronic commerce november 1998 transaction pathways the question of whether current transaction pathways will remain the same or change as the internet is used for servicing insurance consumers remains to be seen. while respondents believed market preferences determine industry structure, they also suggested that the/7ow of information between entities will emerge as an important influence on industry structure in the future. such a concept illustrates how the independent insurance agency system is vulnerable to an electronic marketplace such as the internet where information flows freely and is accessed at the convenience of the user. the interviews also uncovered that the flow of information between consumers, agents and companies on the internet is not one-dimensional or biased towards one entity, rather the manner through which the information is used is the distinguishing feature of information based commerce. thus, the internet serves as an information tool that will supplement buyer preferences, with those preferences determining die path followed in a given transaction. for example, one respondent gave the example of an agency web site seamlessly integrated with his company in which a browser located a local independent agent's web site and used the agent's web site as the structure for securing insurance from die internet. such an example highlights the possibility that the physical distinction between agency and company becomes blurred in die virtual world, with consumer preferences determining transaction patterns. buyer loyalty and switching the increased ability and convenience of consumers to search for insurance products were seen as factors that may influence customer renewals. it was found that numerous external factors influence whether consumers renew their policies with a given company. however, it was noted that an increased ability for consumers to search and switch was not beneficial to carriers. an increased tendency to search was seen as leading towards product price having a larger role in product selection, and created a contradiction between those who saw die internet as facilitating and building closer relationships between companies/agencies and their consumers, and those who saw die internet as decreasing relationships. negative impacts of switching included increased organisational operating costs to support products and customer servicing, and potential threats to coverage renewals. product and price information availability when respondents were asked how price and product information on the internet would impact insurance distribution, the following factors where seen to interact with how product and price information are utilised on the internet: • educationconsumer education dirough die internet serves an important role in helping buyers understand insurance products and rating mechanisms, but also increases their product/service awareness and expectations • needs assessmentdie population of internet users mirrors die traditional insurance consumer population in which different buying segments have different buying preferences, including those who are price oriented, relationship oriented, and protection coverage oriented • market makerdie ability of and incentive for an organisation to create an environment of on-line, comparative insurance shopping has not been fully realised at present. although different organisations were noted as attempting to build an on-line insurance marketplace (for example ins web <www.insweb.com>) the current initiatives were seen as limited and with minimal benefits the ability of consumers to conduct rate comparisons was seen to be increased as a result of die internet. at present, the time and process to conduct rate comparisons are absorbed by independent agents or consumers. however die findings indicate me ability of a market maker to create a favourable environment on die internet for comparison shopping would be beneficial. in such an environment, product and price information would be available for consumers to assist in assessing product needs. price competition the effect the internet has on price competition between insurers was seen to be limited to me business strategy die organisations have developed. depending on whedier a carrier's competitive strategy is based on cost, service, and/or quality directly influences die internet's impact on price sensitivity. additionally, me price mechanisms are not in place to provide quick responses to customer inquires (most internet quotes are delivered though e-mail or the telephone), and the organisation and system structures have not been developed (die costs were seen as too high due to different rating mechanisms and state regulation variables). although such impediments to price competition exist, die long term perspective suggested that price would have an increased 35 ajis special edition electronic commerce november 1998 role in product selection, and that margins would decrease to meet increased customer demands on price, quality and service. one proposed effect of price competition on the industry would be increased instability as premiums were lost and carrier adherence to strict underwriting guidelines was relaxed. disintermediation the internet's impact on market intermediation and the role of agents in insurance distribution was seen by respondents as a significant business issue. as the internet's ability to fully serve as an electronic marketplace evolves, some respondents believed that market competition from other financial and non-financial organisations would increase, that cost reductions would be more aggressively sought, and that the market role of the independent insurance agent would change. evidence of the four possible market intermediation outcomes (see figure 3) proposed by sarkar, butler and steinfield (1995) is visible within the insurance industry. a direct market supported by an electronic marketplace has emerged with auto insurance. it is believed direct marketing will have a larger role with personal insurance lines in the future. the growth of direct markets through the internet is considered to threaten current intermediary structures. however, agency use of the internet to supplement their business practices (outcome iv) was seen as viable due to the emergence of business plans and strategies by insurance agents to create market niches and continually add value to the insurance transaction process. this is especially prevalent in commercial lines insurance. an interesting area to observe in the future is whether cybermediaries will emerge on the internet to serve the interests of the insurance buying public. although industry trade press and study respondents alluded to the development of cybermediaries (for example the market maker ins web <www.insweb.com>) the competitive impact and influence of their activities to date were seen as minimal, and too early to anticipate future impact. with current disintermediation targeted towards personal insurance lines, and possibly small commercial lines, the potential for adverse selection in favour of organisations aggressively utilising the internet was found to increase. some carriers and agents could potentially limit their relationships to organisations using the internet, and build relationships from a common commitment towards integrating the internet into their business processes. while some respondents believed the industry as a whole could slow insurance migration on to the internet, others believed service providers from different sectors and industries could enter the market and compete using internet technologies, negating any prolonged industry hesitation. thus, rather than impede the industry's growth on the internet, it was seen as being more viable to work with agency system members (i.e., agents and vendors) to develop the technology and understanding within the window of learning that now exists. another aspect of disintermediation was the manner in which non-internet related issues influenced the use of market intermediaries. it was previously noted that different market segments have different buying attitudes and preferences towards product quality, price, service, and other product attributes. overall, respondents noted that disintermediation was already occurring in certain markets without the internet through utilisation of directing marketing and toll-free telephone channels. the internet was seen as a new alternative to traditional direct marketing methods. the findings of the study suggest differences between traditional direct marketing opportunities and those emerging on the internet. the emerging opportunities represent a new approach to deliver insurance services to consumers. thus, disintermediation has become an increasingly viable possibility due to the internet, but the internet itself will not be the sole force pushing disintermediation. therefore, agents need to recognise changes in market demographics and consumer buying preferences to continually understand how they can provide added-value services on the internet. overall, the independent agency system's future was not seen as dire, rather it was suggested the system would merely change and adapt to new market demands. the analogy of the agency system to a triad in which insured, agents, and carriers represented the three points emerged as an interesting concept. with the emergence of the internet, information flow in the triad is multilateral and flows in different directions determined by information requirements. triad members exist to complete the insurance transaction, with their position in the triad determined by the market value provided and their individual business objectives. a networked environment of the triad was found to facilitate information exchange and seamless integration. mergers and strategic alliances the kiosk analogy to atm networks that emerged during the study also serves the purpose of illustrating opportunities for mergers, strategic partnerships, and new market entrants to offer insurance products on the internet. some respondents recognised the ability for a large retail organisation (such as a bank) with an established internet presence to include insurance products in its on-line product portfolio. with the internet it would not matter whether the insurance products originated within a business unit or from a strategic partner due to the internet's ability to seamlessly integrate products and services using hyperlinks and multimedia. again the concept of a market maker emerged, but was tailored to joint efforts of multiple organisations to create the basis 36 ajis special edition electronic commerce november 1998 for an on-line market maker. such an initiative was mentioned to potentially include a technology provider, product suppliers), and unbiased third party evaluators. state insurance regulation insurance regulation was determined to have a profound impact on how the insurance industry participates on the internet. although the united states federal government has endorsed national commerce on the internet, insurance remains under the legislative control of state legislatures and the regulatory control of state insurance commissioners, resulting in laws and regulations differing from state to state. thus, a potential area for conflict of interest has emerged in which although insurance is individually regulated by each state, the internet serves a national and international consumer market. while the legislative and regulatory issues regarding insurance and the internet are not in the scope of this study, it is essential to note the conflict of interests, and the task facing insurance regulators and the insurance industry to resolve interstate and international electronic commerce issues. conclusion the interviews revealed how market relationships, transaction paths and switching ability would be affected by the internet. the internet was widely seen to be organisationally and market neutral, with market forces dictating the structure of the independent insurance agency system on the internet. some important business issues emerged regarding how electronic market characteristics influence the entrance of the independent insurance agency system onto the internet. with the wide availability of information resources and the emerging availability of price comparison, the respondents provided insight into strategic and operational issues that need to be addressed. while the time frame for carriers and agents to fully integrate the internet and related technologies is uncertain, the strategic and business issues need further consideration and analysis to help facilitate the system's transition towards electronic commerce. economics of market participation benefits of participation the immediate benefits for agencies of participating on the internet were found to be mostly based on subjective rather than objective criteria. however, insurance carriers indicated they are attempting to build the economic justification for agency participation on the internet. the economic justification for agents and companies to participate on the internet focused on promoting the following benefits: • elimination of software distribution • reduced transaction processing costs • improved information distribution • decreased overall information technology costs • improved quality of the investment by associating it to functionality • value from e-mail communication • audit tracking • brand name and product recognition • improved consumer understanding and education • increased advertising • niche attractions through specific fact sheets and customised information • improved service beyond contract requirements general optimism surrounded carrier motivations for internet participation, but a strategic theme emerged in which the internet's ability to eventually become integrated into the industry's business processes and therefore become a strategic necessity was noted. the internet's ability to provide a flexible business platform for companies and agents to interact and serve customers was a primary reason for integrating the internet into business processes. also, as the internet's integration into fundamental insurance business practices continues, the potential business value provided to carriers by the internet was seen to increase. without active agent participation on the internet technologically facilitated adverse selection was seen as a distinct possibility. in such a situation, those intermediaries who were not participating in electronic commerce would be automatically excluded from future internet developments and advancements. additionally, agents were seen as preferred partners in internet facilitated insurance commerce, but it became clear that if consumer demand for direct interaction with companies increases or the benefits of direct interaction change, agents will not be able to block carriers from interacting with consumers on the internet. the potential to alienate agents by carrier activity on the internet was seen, but prevention of such alienation was directly targeted through 37 ajis special edition electronic commerce november 1998 continued reliance on the present agency distribution forces. however, carriers continued to find value in internet exploration and participation, especially with insurance marketing and product information. additionally, advances in internet technologies were seen as facilitating widespread adoption of internet usage into core business practices, and further promoting the eventual adoption of internet participation as a strategic necessity. network externalities at present, the benefits carriers derive from the internet were not directly linked to the number of participants on the internet. the carriers benefit from the internet indirectly by communicating with internet users about their product and service offerings. however, carriers are trying to assess long term benefits of electronic commerce and are exploring the appropriate business processes for re-engineering. overall benefit of participation was attributed to capitalising on new consumer demands by using technology solutions in delivering new business visions. investment and operational costs the initial costs and investments required for participation on the internet were justified based on carrier strategic outlook for the future. the strategic outlook for the carriers incorporated a three to four year timeframe in which the means of exchange and communication methods change to align with the "digital economy". in such an environment, it was considered that adverse selection could occur in which high performance agents participating on the internet with strategic information technology plans would increasingly benefit at the expense of low performance, non-internet-participating agencies. network switching costs organisational investments in internet technologies were seen as sound investments due to the limited switching costs faced by internet participants. the internet's design and infrastructure allow for flexibility in information technology investments (for example client server technologies, open standards, and platform independence) provide for flexible investments to address different business functions. organisational benefits from internet investment included open network standards allowing for increased agency-carrier interfacing and information exchange, as opposed to proprietary networks that can be cost intensive and investment specific, with reduced business flexibility. agency capital investment the initial capital investments for independent agents were seen as minimal. with current internet service provider costs relatively low, and the ability to incorporate internet services into agency management systems, it was seen to be in an agent's best interest to invest in internet participation. in addition to having companies and agents working together to build the technological ability to use the internet to serve customers, the initial capital investment was seen to be near zero, especially due to agency abilities to build on information technology systems already in place. in such an environment the main costs were seen to be participant education and awareness of the internet and its potential uses. to quantify the investment costs it was deemed important to look at return on investment as the ability to interact with customers, and to improve customer and carrier relationships. the tendency for agents to focus solely on sales should not be the sole objective; lead generation was suggested as a potential longer term objective as the technology evolves. a cost spectrum developed in which the cost of participation directly relates to the business value derived from the internet. for example, as the interactivity and business value increase in a web site more investment and capital outlay are required. the lower the interactivity and business value of a web site, the lower the cost. for example, for a web site to adequately support sales and customer servicing requires more investment than a simple site with advertising and product information. without specific business methodologies or plans to manage the relationship between increased cost and increased business value, the fundamental objective was seen to view investment in the internet as an investment in future business productivity and processes. finally, it was discovered to be essential to consider business and strategic issues before developing an internet presence. such consideration was seen to improve the likelihood of organisations avoiding wasted effort and resources, and maximising the benefits. 38 aj1s special edition electronic commerce november 1998 prisoner's dilemma the concept of the prisoner's dilemma in electronic marketplaces (sellers as a group do not benefit from an electronic market system, but as individuals the revenues that can be generated from such a system can benefit them and outweigh the losses) advanced by bakos (1991) was adapted to this study and used to understand insurance carrier motivations and concerns for internet participation. first mover benefits were seen to be organisational specific due to different carrier strategic and business objectives for internet development. latemover benefits were attributed more towards functional benefits derived directly from internet capabilities (for example using e-mail as an alternative communication means). although optimism and enthusiasm were associated with internet business development, concerns regarding how the industry and distribution systems would adapt, and how market competition could change remained a primary concern. the ability for financial and other non-financial institutions to leverage their organisational network capabilities, resources, and customer bases to enter the insurance market consistently also emerged as a concern. disadvantages of widespread internet adoption were seen to include rate reductions due to increased price and market competition, increased commonality of value added services through industry standard elevation, increased market threats from merged consumer processes and product bundling, and a reduction in total market participants. an interesting concern discovered was whether a global interconnected electronic market such as the internet would lead to a more globally competitive, digital insurance marketplace. from the study's findings it became evident that state regulation and oversight of insurance was seen as a substantial factor in keeping insurance localised to individual regions and states, rather than leading to increased national and international free market competition over the internet. conclusion with the wide availability of information resources and the emerging availability of price comparison, the respondents provided insight into strategic and operational issues that need to be addressed in relation to electronic market participation. in addition to understanding and justifying the benefits of participation, the role internet economics have in facilitating its adoption throughout the agency system was exposed to include switching costs, capital investments, and network externalities. while the time frame for carriers and agents to fully integrate the internet and related technologies is uncertain, the strategic and business issues need further consideration and analysis to help facilitate the industry's transition toward electronic commerce. overall conclusions the research objectives of this study were established to undertake exploratory study on how the internet, as an electronic market, is changing the american independent insurance agency system. the organisations interviewed for the study provided perspectives from the insurance producer's point of view. the intent of pursuing such a perspective in this study was to help independent insurance agents understand relevant market forces and insurance carrier motivations supporting internet participation, and to improve overall industry understanding of how electronic commerce on the internet impacts organisational and market structure. as the findings indicate, a number of issues were explored through the interview process to provide in-depth, contextual understanding of the issues involved with internet facilitated electronic commerce related to insurance. during the exploration process a number of unexpected issues were also uncovered and discussed. this combination of established and unexpected research topics for study and discussion contributes in creating practical value for those interested in understanding current complexities and concerns regarding the growth of insurance-based electronic commerce on the internet thus, the research significance of this paper is its approach to presenting and discussing electronic commerce at a general, conceptual level, the creation of a strategic framework for assessing electronic commerce by extracting core constructs and propositions from such a presentation, and applying the framework to a specific industry. from the data collected during the interviews it can be concluded that the internet is beginning to have a profound impact on the american independent agency system. the common theme throughout the study was the existence of organisational awareness of a growing internet role in the insurance industry, and strategic planning and operations for building the reliance on and use of the medium. other conclusions that can be drawn from the study include the internet's ability to offer organisations new methods and practices for conducting business, including: • enabling more information to be communicated in the same amount of time and decrease the costs of communication • providing a medium to connect many different buyers and sellers, and facilitate product offering and buyer need matching 39 ajis special edition electronic commerce november 1998 • allowing organisations to change business processes and information usage methods • changing traditional market relationships to favour greater efficiency and productivity with increased market value • encouraging sellers to differentiate their product to prevent commoditisation • increasing the possibility of disintermediation, with an overall emphasis on the value of different possible electronic market intermediary roles (for more information see figure 4.3) it is important to again note the qualitative nature of this study. to successfully achieve the research objectives of this paper qualitative research was selected, primarily due to the. early stage of theory building in the chosen research topic area. possible future research includes the development of a larger study sample with a less qualitative structure to provide a more in-depth, less narrow, empirically grounded research study. due to the lack of established academic and insurance industry theories pertaining to the internet's ability to facilitate insurance related electronic commerce in a market environment at present, such an approach was not taken. study weaknesses were minimised as much as possible, to maximise the practical and research significance of the study. additional empirical and conceptual research needs to be undertaken to explore and test the issues that emerged during this study. finally, as internet technologies develop and insurance industry adoption of electronic commerce increases, additional research should be conducted to evaluate the long term strategic outlook for the american independent agency system. references bailey, joseph p. and j. yannis bakos (1997): "an exploratory study of the emerging role of electronic intermediaries", international journal of electronic commerce, vol 1, no 3, pp. bakos, j.y. (1991): "information links and electronic marketplaces: the role of interogranizational information systems in vertical markets", journal of management information systems, x, x, pp. bakos, j.y. (1991b): "a strategic analysis of electronic marketplaces", mis quarterly, september, x, x, 295-310. bakos, j.y. (1997): "reducing buyer search costs: implications for electronic marketplaces", management science, december, x, x, pp. bakos, j.y. and brynjolfsson, e. (1993): "from vendors to partners: information technology and incomplete contracts in buyer-supplier relationships", journal of organisational computing, x, x, 301-328. banham, russ (1997): "cyber-agents", independent agent magazine, february, 16-21. barrett, s. and konsynski, b. (1982): "inter-organisation information sharing systems", mis quarterly, x, x, 93-105. benjamin, r. and wigand r. (1995): "electronic commerce: effects on electronic markets", journal of computer mediated-communication, 3,1-14. benjamin, r. and wigand r. (1995b): "electronic markets and virtual value chains on the information superhighway", sloan management review, winter, 62-72. block, michael, yves pigneur and arie segev (1996): "on the road of electronic commercea business value framework, gaining competitive advantage and some research issues," working paper: fisher center for information technology and management, university of california, berkeley, [url:http://l 30.223.171.154/yp/pub/road_ec/] brandel, mary (1996): "is the world ready for web quotes", computerworld electronic commerce journal, april 29,12. cash, james i., jr. (1996): "the point of the exercise the it treadmill may seem like more trouble than it's worth, but you'll feel the difference as the corporation firms up", information week, november 25, 128. champy, jim (1997): "how should you 'push' your pitch to consumers?", computerworld, may 26, 72. demons, e.k., reddi, s.p., and row, m. (1993): "the impact of information technology on the organisation of economic activity: the 'move to the middle' hypothesis", journal of management information systems, fall, 9-35. demons, e.k., and michael row (1993): "limits to interfirm coordination through information technology: results from a field study...", journal of management information systems, june 1,73. coase, r. h. (1937) "the nature of the firm" in williamson, o. e. and sidney g. winter (yy) the nature of the firm: origins, evolution, and development, oxford university press, new york. cole, barb (1997): "insurers pay premium for secure e-commerce", computerworld, march 17,4 cortese, amy (1997): "a census in cyberspace", business week, may 5,85. eisenhardt, k.m. (1989): "building theories from case study research", academy of management review, 14,4, 534-550. 40 ajis special edition electronic commerce november 1998 england, frederick j., jr. (1993): "the eagle has landed", in ronald w. vinson and jack payan (editors), soaring with eagles: the rise of independent insurance agents in america, independent insurance agents of america, virginia. gray, matthew (1996): internet growth summary. massachusetts institute of technology [url:http://www.mit.edu/people/mkgray/]. grover, v. (1993): "an empirically derived model for the adoption of customer-based interorganizational systems", decision sciences, may-june 1993, 603-640. gupta, sunil (1995): "hermes: a research project on the commercial uses of the world wide web," university of michigan working paper, [url:http://www.umich.edu/~sgupta/hermes/] gurbaxani, v. and whang, s. (1991): "the impact of information systems on organisations and markets", communications of the acm, january, xx, xx, 59-73. halper, mark (1997): "meet the middlemen", computerworld emmerce, april 28, 10. hess, c.m. and kemerer, c.f. (1994): "computerized loan origination systems: an industry case study of the electronic markets hypothesis", mis quarterly, september, x, x, pp. higgins, kelly jackson (1996): "are we there yet? not really. most are stuck on the slow road when it comes to ec. three industries explain why", communications week, november 4,61. hines, peter (1993): "integrated materials management: the value chain redefined", the international journal of logistics management, 4,1, pp. hoffman, donna l., thomas p. novak and patrali chatterjee (1995): "commercial scenarios for the web: opportunities and challenges", journal of computer mediated-communication, 3 ,1-21. [url:http://jcmc.huji.ac.il/] jenkins john (1997): "doing business electronically", electronic commerce association: business on the internet interest section internet briefings, 3. [url:http://www.eca.org.uk/] kambil, ajit (1995): "electronic commerce: implications of die internet for business practice and strategy", business economics, october 1, 22(7), pp. kaplan, b and j.a. maxwell (1994): "qualitative research methods for evaluating computer information systems", in j.g. andersen, c.e. aydin and s.j. jay (editors), evaluating health care information systems: methods and applications, sage publications, thousand oaks. kristula, dave (1997): "the history of the internet", a section of web site workstation [url:http://www.davesite.com/webstation/net-history/] kroll, ed and ellen hoffinan (1993): "fyi on what is the internet?", network working group request for comments, 1462, fyi:20 [url:gophen//dsl.internic.net/00/fyi/fyi20.txt] leiner, barry m., vinton g. cerf, david d. clark, roberth e. kahn, leonard kleinrock, daniel c. lynch, jon postel, larry g. roberts and stephen wolff (1997): a brief history of the internet, version 3.1, last revised february 20 [url:http://www.isoc.org/intemet-history/] malone, t., yates, j., and benjamin, r. (1987): "electronic markets and electronic hierarchies", communications of the acm, june, x, x, 484-497. mason, david and mary j. turner (1996): "timing is everything getting into ec at the right cycle is essential", communications week, september 2, s03. mehr, rober i. and emerson carnmack (1980): principles of insurance, 7th edition, homewood, richard d. irwin, inc., illinois. miller, thomas e. and peter c. clemente (1997): "the 1997 american internet user survey, realities beyond the hype", find/svp, emerging technologies research group, may 6 [url:http://etrg.findsvp.com/]. myers, michael d. (1996) "qualitative research in information systems," misq discovery, x, x. url:http://www.auckland.ac.nz/msis/isworld/] neville, john f. (1980): no violations of policy, rough notes company, inc., u.s.a. nouwens, john and harry bouwman (1995): "living apart together in electronic commerce: the use of information and communication technology to create network organisations", journal of computer mediated-communication, 3, 1-25. [url:http://jcmc.huji.ac.il/] payan, jack (1993): "gathering of eagles", in ronald w. vinson and jack payan (editors), soaring with eagles: the rise of independent insurance agents in america, independent insurance agents of america, virginia. porter, michael e. (1985): competitive advantage: creating and sustaining superior performance, free press, new york. porter, michael e. (1979): "how competitive forces shape strategy", harvard business review, marchapril, 137-45. rayport, j. and sviokla, j. (1995): "exploiting the virtual value chain", harvard business review, november-december, 75-85. 41 ajis special edition electronic commerce november 1998 sarkar, mirtra b., brian butler and charles steinfield (1995): "intermediaries and cybermediaries: a continuing role for mediating players in the electronic marketplace", journal of computer mediatedcommunication, 3,1-17. [url:http://jcmc.huji.ac.il/] steinfield, charles, robert kraut and alice plummer (1995): "the impact of electronic commerce on buyerseller relationships", journal of computer mediated-communication, 3,1-17. [url:http://jcmc.huji.ac.il/] stewart, thomas a. (1993): "boom time on the new frontier", fortune magazine, autumn, 153. swamidass, p.m. (1991): "empirical research: new frontier in operations management", academy of management review, 16,4,793-814. thornton, e. (1994): "revolution in japanese retailing", fortune, february, 144. williamson, o. (1975): markets and hierarchies: analysis and antitrust implications, free press, new york. wiseman, charles (1995): strategy and computers: information systems as competitive weapons, dow jones -irwin, new york. yin, r.k. (1994): case study research, design and methods, sage publications, newbury park. cambridge telecom report (1997): "internet commerce: ten key legal and regulatory issues facing the growth", february 17. cambridge work-group computing report (1997): "internet shopping: 'startling increase' in internet shopping", march 17. commercenet/nielson internet demographics survey (1997): spring [url:http://www.commerce.net/] commercenet/nielson internet demographics survey (1996): march-april [url:http://www.commerce.net/] consumer payment preferences in the physical world and the virtual world (1996): global concepts and bank administration institute, september [url:http://www,global-concepts.com/research.html/] consumers poised to use electronic financial services (1996), newsbytes news network, october 23, electronic commerce and the internet: a whole new world (1997), conning and company future one agency universe study (1996): independent insurance agents of american, inc. alexandria, virginia graphic, visualisation and useability center (1997): "gvu seventh annual www user survey. [url:http://www.gyu.gatech.edu/user_surveys/j ibm: study reveals details for the future insurance industry (1997), m2 presswere, april 28 national association of insurance commisioners (naic)(1997): the marketing of insurance over the internet, draft: june 4 net profits. (electronic commerce on the internet)(1994), the economist, 332, july 2, 83(3) oracle: research shows business and it at loggerheads over the take up of electronic commerce (1997), telecomworldwire, june 17 understanding cyberspace, (computer networks)(survey of technology in finance) (1996), the economist, 34 i.october 26 world insurance network: world insurance network sparks to life (1996), m2 presswire, march 18 42 ajis special edition electronic commerce november 1998 appendix interview guide section i. subject information name: position: organisation: phone number: address: city, state, zip: section ii. research objective general management question: how is the internet, as an electronic market, changing the role of independent insurance agents in the insurance industry? section iii. interview guide • electronic communication effect 1. does the internet allow for more information about insurance products and prices to be communicated in the same amount of time as traditional communication methods? how? why? 2. does the internet allow for the costs of this communication to be decreased dramatically? how? why? • electronic brokerage effect 3. does the internet allow agents and companies to connect a large amount of different buyers? sellers? how? why? 4. does the internet fulfil the function of assisting to match consumers with sellers of insurance? how? why? 5. does the internet increase the number of alternatives of insurance products and services a consumer can consider? how? why? 6. does the internet increase the quality of the alternative eventually selected? how? why? 7. does the internet have the potential to decrease the cost of the entire product selection process? how? why? • electronic integration effect 8. does the internet facilitate the ability of producers and agents to change their business processes (sales, marketing, customer support, other)? 9. does the internet facilitate the ability of producers and agents to change the manner in which they use information? • insurance market structure 10. increase or decrease in longer term relationships with consumers? 11. increase in value added partnerships? 12. is there a move to markets, the middle, or other movement in the industry? • price and product information 43 ajis special edition-electronic commerce november 1998 13. does the internet reduce buyer search costs and time for insurance products and services? 14. does the internet increase the ability of consumers to search for insurance products? • promotion of price competition 15. does the internet foster price competition in insurance products and services? • supplier market power 16. does the internet have the ability to reduce the market power of insurance companies? • network externalities 17. do the benefits of participation on the internet increases as more members join? 18. are the benefits of participation on the internet uncertain? 19. is the timetable for change in the independent agency system due to the internet uncertain? • switching costs 20. what type of switching costs are faced by agents if they change their presence and participation on the internet? • capital investment 21. does the internet require large amounts of initial capital investment? 22. does the internet offer economies of scale and scope to insurance agents? • the nature of insurance products offered by independent agents 23. commodity or differentiated market? 24. how does the complexity of insurance products influence its presence on the web? 25. do specific types of insurance limit the ability of products to be generalised to a wide market? 26. how will the following influence how insurance products are applied to the internet: (frequency of the transaction, power of buyers or suppliers, non contractible investment issues) • prisoner's dilemma? 27. do agents as a whole benefit from participating on the internet? 28. do individual agents benefit from participating on the internet? 29. will it become a strategic necessity to participate on the internet? • intermediation 30. are insurance agents threatened by the internet? why? 31. is the internet driven by consumers, agents, or companies? • what future scenario or outlook do you see for insurance and its distribution over an electronic market such as the internet? 32. generational differences? 33. long term marketing and strategic approach? 44 implementin g business process redesign: earl y lessons from the australia n experience mariann e broadbent and carey butler key centre  for technology management, melbourne business school, university of melbourne abstract business process redesign  (bpr)  is a change  management  approach aimed at achieving quantum improvements  in business performance.  industry interest  levels  in bpr are high as a direct result of  current difficulties  in  the  global  economic  climate  and  tight business  conditions.  integral  to bpr  is  the  availability  of  new  stable  technologies  which  both  stimulate and  enable  process changes.  this  paper  highlights  the  experiences  of  a  number of  australian  firms  which  have implemented  bpr. a ten step framework  for bpr is presented  together  with a series of caveats.  bpr  is a  difficult , messy  and  often  non­linear  activity  which challenges  many  of  the  ways  organisations  operate. information technology  plays a pivotal role  in bpr as both an enabler and disenabler for  change. lessons emerging  from  early  australian  experiences  with  bpr  focus  on  the  role  of  executive sponsorship,  consultants,  measurements,  education and  training, technology  and people  involved in the change process. introductio n "the '80s were a time for financial re-engineering ... the '90s are for technological re-engineering"  (freedman, 1993). business conditions  in  the early  1990s have placed  new strains on organisations  as  they strive to  survive  in  fiercely  competitive  marketplaces.  these  strains  are  intensified  amongst  i/s groups facing continuing pressure to demonstrate the value of information systems ­ often with reduced  resources.  business  process  redesign  (bpr)1  is  now  being  undertaken  by  a substantial  number  of  organisations  to  realign  their  operations  in  order  to  cope  with  these circumstances.  information  technology  (i/t)  can  be  viewed  as  both  an  enabling  force  for leveraging  these changes, and as a disenabler if i/t investments have not been  circumspect. hammer  (1990)  defines  bpr  as "the radical redesign of business processes in order to achieve dramatic improvements in their performance" -  in  terms of  quality, flexibilit y  and productivity.  bpr  analyses  the  work  flows  of  an  organisation,  discards  non­essential procedures  and may then use  the power of i/t systems and applications to  further  streamline organisational  operations.  while  the  use  of  technology  is  not  proscribed,  the  majority  of redesign  projects  undertaken  in  australia  and  elsewhere  have  invariably  included  a significant  i/t component.  the benefits passed  on  from  redesigned  processes  can  include: also commonly referenced  as business process  re­engineering 63 improved  customer responsiveness,  dramatic reductions  in  process  time,  cost  savings,  and streamlining of the organisation in terms of structure, staff and resources. this paper draws on the  literature of bpr to examine  the reported  experiences  of australian firms which have undertaken bpr.  we address a number of key questions,  through applying a framework for the implementation of bpr, and then discussing its salient  features.  these questions include: •  what are the motivations for bpr? •  what is the relationship between tqm and bpr? •  what role does the i/s function play in bpr projects? •  how  is  technology  being  used  in  redesign  projects  to  achieve  performance breakthroughs? •  what are the key drivers within australian organisations for implementing  bpr? •  what lessons have these organisations learned through their experiences  with bpr? evolutio n of bpr the  term  business process  redesign,  first  appeared  in  the  management  literature  in  mid­ 1990,  following  the the publication of seminal articles  by hammer (1990) and davenport & short  (1990).  these articles described a new change  management  philosophy: one  which transformed  organisational  processes  beyond  mere  mechanisation  of  existing  tasks,  and offered quantum performance improvements (craven, 1991). traditional  functional  management approaches  were no longer seen  as appropriate  in a time where quality, innovation and customer service  were of paramount importance  (butler cox, 1991).  there was a real need to implement processes  that would enable  a business  to meet the  ongoing  demands  of  its  marketplace  (bevilaqua  &  thornhill,  1992).  the  apparent plausibility of the approach suggested by hammer (1990) and others for attaining  these goals, has led to the rapid uptake of business process  redesign. just as bpr's effect  on an organisation is often cross­functional in nature, the approach  itself draws on tools and techniques  from a variety of well established disciplines  such as industrial engineering, quality management and systems analysis.  rigby (1993)  remarks  that many of bpr's  attributes can  be  traced  back  to  work  done  by  early  researchers  of  organisational systems and design, and that established  incremental change techniques  like activity  value analysis or core process redesign, are in many respects, earlier incarnations of bpr. there are however a number of features which distinguish  it  from  these earlier  mechanisms. these  include bpr's  radical,  largely non­incremental nature, claims  of  rapid  paybacks  for money  and  time  invested, heavy involvement by management consultancies,  and  its  strong links  with  enabling  technologies.  bpr  is  a  change  mechanism  which  if  successfully implemented, can assist organisations  to compete more effectively in a chosen market, and to realise payoffs on investment in i/t (fried,  1991; king, 1991; olson,  1992). 64 bpr and total qualit y managemen t bpr  also  differs  from  the total  quality management (tqm) approach.  while both  are  based on  the premise  that superiority  in business process  performance  is critical  to  competitiveness, they  differ  in motivation, technique, objective,  result and  the business circumstances  in which they are applied (gulden & reck, 1992).  tqm favours steady incremental gain, and may  often take  a  number  of  years  before  real  gains  are  attained.  for  firms  in  highly  competitive industries,  this  lagtime  can  allow  competitors  to  forge  ahead.  tqm  can  however,  lay  the groundwork for bpr and provide a foundation for a focus on processes: in  1991,  digital  australia  discovered  that  its  tqm  programme  would  not  give  the company "the quantum leap in improvement needed to give it a competitive advantage in the desktop computer market"  (alexander,  1993).  digital  is currently  undertaking  a bpr project which involves extensive redesign of its supply chain  process. caltex  australia embarked  on  its  pacesetter  program  in  1989  to  develop  best  practice approaches  within the  firm.  three  years  later  this  led  to  the  formation  of  a  business process  re­engineering  function  to  streamline  business  processes  and  manage  the information  resources  of caltex  in the most efficient  and effective manner. the nrma has a strong commitment to tqm and made continuous small  improvements in their processes.  the motoring organisation is now proceeding with a bpr initiative to challenge  these  processes  while  retaining  their  commitment  to  quality  management techniques (driva &  hunt,  1994). motivation s for bpr a combination of business imperatives and technology availability underpin the motivations  to commence bpr.  motivations which are not mutually exclusive include: (1)  a growing realisation  that substantial change needed  to be made  to many current business operations  if  organisations  were  to  maintain  profitability  and  improve  customer responsiveness.  these concerns stemmed directly  from: •  intensified  industry competition within domestic markets and from overseas •  a worldwide economic downturn, which forced many firms to seek seek  ways of achieving economies of scope and cost (economist,  1990;  hammer,  1990; butler cox,  1991). australian examples:  qantas airlines, bp australia, mobil australia, and caltex. (2)  the  merger  and  acquisitions  "binge"  (king,  1991)  of  the  1980s  had  left  many organisations with: •  heavy debt burdens caused by rapid expansion; and/or •  incompatible  or  totally  different  organisational  cultures,  work  practices  and information systems 65 these findings are consistent  with the australian scene.  broadbent  & others  (1993)  biennial survey on information systems issues amongst australasian share/guide2  members,  found  that respondents  regarded  business  process  redesign  as  an  i/s  management  issue  of  rapidly emerging  importance.  dun  &  bradstreet  software  services  annual  customer  poll  of  107 australasian  companies  reported  that  50% of  respondents  were  either  already  undertaking or planning  to undertake  bpr  initiatives  (mis,  1993).  a major international  research  project on the role and value of i/t infrastructure  currently underway at the melbourne business school, found  that  over  half  of  the  11  australian  firms  included in  the  study were  involved in  bpr activities.  additional  evidence  of  a  growing  interest  in  bpr  by  australian  firms  was well supported  in 1993, with around  14 bpr seminars or conferences  held round the nation. we  now  review  the  experiences  of  australian  firms  who  have  reported  implementations of bpr.  these reports are taken from a variety of sources which include: •  the  authors  detailed  case  analysis  of  two business  groups  (broadbent  &  butler,  1993; butler & broadbent,  1993) •  the  case  vignettes  of  eleven  firms  who  are  participating  in  the  melbourne  business school's  i/t infrastructure study funded by ibm consulting group •  literature reporting  australian bpr implementations •  conference  and personal reports lodged with the key centre for technology  management which  acts  as an  information source  for managers  who  have been  involved in  seminars and educational  programs  at the melbourne business school framewor k for the implementatio n of bpr the framework presented  in figure 1 has been  derived  from  the  literature and the  experiences of australian  firms.  the  framework  includes 10 steps  which appear  to provide a sound basis for  the implementation  of bpr.  in the discussion that follows, examples  from  australian firms are used  to illustrate  these  steps.  caveats  on  this framework are presented  in  later sections of this paper. 1) articulat e vision and objectives: evaluation of current business and prioritising of objectives.  the operative  word in bpr is redesign ­ to change a business process  to the extent that  it will deliver the strongest  possible  competitive advantage  (carr & others,  1992).  this is most  likely to succeed in firms which exhibit a consistency  of mission or strategic  intent. in 1991, bp australia and bp new zealand  linked their operations  to  form  the bpoaz region. a significant part of phase  1 of bpoaz's bpr project  involved determining a business vision that  would  integrate  two quite different  businesses.  the  vision  moved  bpoaz "from being volume driven to being profit driven, from being transaction processors to being knowledge workers, from being functionally managed to being process managed"  (broadbent &  butler, 1993).  digital australia's qantum  project  focused on  a  major redesign  of  its  supply process. digital  regarded  the setting of a vision "with the ability to motivate the organisation towards a membership of organisations using ibm  mainframes. 67 stretch goals and high ambitions"  as being  vital  to  the  success  of  their  project  (alexander, 1993). figure 1: a 10 step framework for bpr articulate vision and objectives 1 implement ongoing improvement strategy make new process operational ensure executive committee sponsorship develop high level process map ensure change management and support capabilities business process redesign identify process for redesign understand & measure existing process redesign the process, identifying i/t levers prototype the new process faced with increasing  accountability and the need to deliver quality services, melbourne  water has undertaken  an extensive  redesign  project  aimed  at effecting  a "quantum improvement in performance"  (lynch, 1994).  part  of its methodology  for establishing  a new business  design has involved translating the existing company vision into high level business  objectives. 2)  ensure  executive  committee  sponsorship: because of its high risk, often radical nature, bpr requires the full support and commitment from top management if it is to succeed.  hammer considers  it essential  for any re­engineering  effort  to have an high  level champion to sponsor  the project and see it through (alexander,  1993). bpoaz's  bpr  project  started  to  have  an  impact  when  executive  level  sponsorship  was evident  (broadbent & butler, 1993).  digital  australia considers  strong, visible  support  and visionary sponsorship  from senior management vital for success in bpr (alexander,  1993). 3)  develop  high level process map: develop a high level view of existing business processes.  part of melbourne water's business design study has involved defining high  level business processes critical  for meeting  business objectives.  the  10 processes identified  and the interactions between them, constitute melbourne water's business process  model  (lynch, 1994).  caltex  australia  developed  a business  process  map which acted  as  a  guide  for  its bpr  implementation. 68 bpoaz conducted a number of process  definition workshops  in order  to document how the region  currently conducted  its business in  terms  of  the  order  management  cycle  (omc)3. the  workshops  produced  a  series  of  flowcharts  which  mapped  the  stages  and  individual elements of the omc, and charted  the interactions and  levels of involvement between  various business  units  (broadbent  &  butler,  1993).  once  the  process  approach  was  accepted, charting of all major processes  was undertaken. 4) identif y process(es) for redesign: identify critical or bottleneck processes.  in 1991,  australian  transport  giant  tnt  suffered  its  first  loss  for  30  years  and  faced  strong competition  in its traditional  markets.  tnt's new change program  has involved the  company in  identifying and  reassessing  its  core  processes,  and  has  realised  a  need  to  substantially upgrade  the  existing  technology  base  to  support  these  processes  (kelly,  1993).  digital australia had  identified its omc process  as a problem area.  after  mapping  the process  and measuring its performance, digital discovered  that the process was costly and inefficient, and that the technology supporting it was cumbersome and outdated (alexander,  1993). queensland's  department  of  employment,  vocational  education,  training  and  industrial relations  (devetir)  is  the  result  of  a  recent  amalgamation  of  six  previous  state departments,  each  having  different  processes,  areas  of  focus  and  computer  environments. devetir has recently undertaken an extensive redesign project aimed at deriving a common set of processes and developing a new i/t architecture  that would support  implementation of the new processes. in  late  1992,  bp  head  office  in  london  introduced  a  new  company­wide  customer responsiveness program.  in line with this, bpoaz selected  billin g as its first major  process for  redesign.  billin g  had  strong  external  interactions  and  would  focus  the  group  on  a customer  perspective  (broadbent  &  butler,  1993).  norwich  financial  services  undertook detailed analysis of its membership application and claims processes in order  to identify  how these processes could be streamlined  (head,  1993). 5) understand and measure existing process: use it tools or other modelling techniques to document and/or understand existing processes.  after  using  the  simview computer­aided  process  modelling  package  to  evaluate  current  freighting  procedures  and prototype new alternatives, queensland rail found  ways to cut a$10 million from  the  process and  substantially  streamline  operations  (mitchell,  1993).  australian  car  manufacturers mitsubishi  and  gmh  have  also  used  simview  for  simulating  their  production  processes (mitchell,  1993). 3  the omc is a transactionally­based process which describes the work flows from lodgement  of an order by a customer to delivery of that order  to the customer.  most company operations  involve an omc element. 69 bpoaz used meta software's  idef computer­aided process modelling tool and dec's symbol­ based  top  mapping  technique  for  tracking  informational  flows  in  a  number  of  processes targeted  for  redesign  (butler,  1993).  digital  australia  used  a  variety  of  prototyping  and modelling tools extensively throughout its quantum redesign project (alexander,  1993). 6) redesign  process(es) identifyin g i/t levers: develop new process approaches including it solutions.  i/s holds a unique position in the organisation through its exposure to all areas of the business.  technology  can integrate  the organisational model by supporting the relationships between its components.  while the application of technology is a key ingredient in bpr, it is not however an end in itself. digital australia quickly realised  that technology alone would not solve its business problems, but that bpr, supported by technology,  could have a dramatic  impact on business  performance (alexander, 1993).  myer/grace bros regarded the implementation of edi/qr4  technologies as vital for maintaining  a competitive edge  in their marketplace.  they quickly realised a need for overhaul of their current systems environment and transactional processes  in order to cope with rapid increases in information capture  (butler & broadbent,  1993). a sharp drop in international  airline  travel after  the  1990 gulf war, prompted qantas airlines to  explore  avenues  for  profit  improvement.  after  developing  radical  new  plans  for  change throughout  the  business  which  included  a  number  of  technology  improvements  and  new solutions,  the  company was able  to  save us$200  million  in  operating  costs  (carr  & others, 1992).  financial  services  firms  such as  norwich and  mlc  lif e  have both used document imaging  technology to support re­engineered  claims processes.  mlc claims that productivity gains have exceeded  their expectations  (roach, 1994), while norwich claims 50­70%  of their benefits have come through the use of imaging  (head, 1993). 7) prototype and test new process(es): test new process for robustness. prototyping  can  be  applied  to  both  process  and  enabling  technologies  to  test  out  the effectiveness  of  design  with  users  before  implementation  (alexander,  1993).  after conducting  a  systems  audit  to  determine  a  suitable  i/t  architecture  design,  queensland's devetir prototyped and verified its new processes running under the new architecture at a single  site  before  moving  to  full  implementation  in  departmental  offices  throughout queensland. myer/grace  bros  opted  for  a  phased  introduction of  its  edi/qr  project.  this  entailed carefully  trialing each new electronic document with a few selected  supplier trading partners before  proceeding  to commercial activation.  this approach also allowed  the retailer  and  its suppliers  time  to  reorganise  systems  and  work  practices  in  order  to  accommodate  more complex edi documents involving quick response (butler & broadbent, 1993). electronic data interchange/quick response. 70 8) ensure change management and support capabilities: prepare people for, and involve people in the changes that the redesigned process(es) will involve.  as a process undergoes  redesign  and  prototyping, jobs  are being  reshaped.  process  redesign inevitably involves changes  in the nature of work.  some tasks are no longer done, while in other areas, the  scope  of  work  is  expanded.  case  management  approaches,  which  can  occur  when insurance  claims  processes  are  redesigned,  require  a  more  holistic approach  to  customers, with one employee  often  completing a set of tasks previously spread over several workers or even several  departments. until  1992,  brash holdings ran its business almost entirely through manual processes.  store managers  were required to complete  up  to 32  forms  each  day  for recording of events.  with the  implementation of  integrated  point­of­sale,  merchandising and  management information systems  in  1992­3,  the  nature  of  work  in  both  stores  and  head  office  changed  radically. brash holdings seconded key retail staff  to head office,  and particularly into the i/t  function, to  work on  the  most appropriate  implementation of  new processes  and  to  lead  the training program  for over 3000 staff. as  part  of  their  change  management  approach  for  redesign  of  the  billin g  process,  bpoaz held 'stakeholder briefings'.  during these meetings, bpoaz managers were introduced to the concepts of process  management.  focus  group sessions  were then held  which explored  the relationship  between  the  firm's  customer  responsiveness  program  and  the  redesign  and process management activities.  once a number of processes  had been redesigned, many staff had been  involved in redesign workshops.  this had a cumulative effect  on bpoaz's ability to speed up redesign  of subsequent processes  (broadbent & butler, 1993). the  implementation  of  edi/qr  at  myer/grace  brothers  meant  radical  change  for  both suppliers and buyers.  two approaches  were put  in place to support these changes.  a retail manager  who  had  worked  on  a  major  i/t  project  was  given  the  responsibility  of  liaising between  i/t  staff  and  retail,  buying  and  merchandising  staff.  secondly,  education  and training sessions were held.  this had the effect  of bringing together both internal buyers and external  suppliers to work through the most appropriate way to shape and implement changes in specific retail areas (butler & broadbent,  1993). 9) make new process operational: implement the changes, putting in place a process management approach.  the  implementation of  a  redesigned  process  usually requires concurrent changes  in  management approaches,  how work is performed  within  the firm,  and  how  technology  is  used.  at  bpoaz, management of  activities through business processes  has  introduced  the  concept  of  process  owners,  process  managers  and  process teams.  the  process  owner  for  billing  is  a  divisonal manager  who  has  responsibility  for monitoring  the  performance  of  the  process  across  the  various  functions  involved.  when further cross­functional enhancements are needed, the process owner sponsors these changes. daily responsibility  for the billing process  rests with  the process manager.  this is a  fulltime responsibility.  members  of  the  billing process  team  come  from  all  parts  of  the  business which are involved in that process  (broadbent & butler,  1993). 71 as myer/grace brothers has proceeded  with edi/qr introduction, changes have been made to the way the new processes  are managed.  a more holistic management approach  is now in place  with  one  manager  having  overall  responsibility  for  overseeing  the  phased implementation (butler & broadbent,  1993). 10) implement ongoing improvement strategies: put in place procedures to monitor and measure continuing improvements in the redesigned process(es).  bpoaz has evolved a firm­specific  process  management  approach  whose  final  step  involves  continuous improvement  re­assessment.  a  redesigned  process  must include  a  set  of  measures  which enable the firm to monitor and improve the performance of that process  (broadbent & butler, 1993).  just  as  the  application  of  tqm  techniques  have  provided  an  impetus  for reengineering, those same techniques are being applied  in an ongoing assessment  of process performance. nrma  considers  that  the  quality  management  approach  ­  with  its  focus  on  constant measurements aimed  at  improvements ­ combines well with bpr: "the amalgamation of these two techniques gives us a quality system that helps us examine the processes and this is what we need to do to remain competitive" (riches, quoted by driva & hunt, 1994). retrace from step 3 while ensuring that the vision, objectives and executive committee sponsorship are refreshed and in place.  at  this  point,  the  high  level  process  map  is revisited and selection of the next process  for redesign is  initiated. discussion the framework presented  above was used to highlight the steps which a number of australian firms undertaking bpr activities have experienced.  however, some caveats  are required  as bpr  is  not  necessarily  an  activity  which  can  be  universally  undertaken  in  a  step­wise, prescriptive manner.  some of the salient characteristics of bpr are now discussed. non-linear and political bpr  is a very difficult ,  messy and often  non­linear activity.  bpr challenges  the empires of existing  organisational  stakeholders  who  might  at  first  give  their  consent  and  pledge commitment.  however, when the extent of possible  changes become  evident,  that  support can  be  quickly eroded.  leading  a  bpr  team  can  be  a  risky  career  move  as  large  scale redesign  is  usually  undertaken  in  an  intensely  political  environment.  this  underlies  the importance  of  the  highest  level  of  executive  sponsorship  for  changes  which  are  cross­ functional  in scope. phases of commitment during bpr the  organisational  environment  goes  through  several  phases:  initial  commitment, excitement,  fear  of change,  scepticism,  tentative involvement and,  if  successful,  large  scale 72 commitment.  redesign activities can and do  falter  during any one of  these phases.  during implementation,  those  involved in  driving the  changes  can  expect  to  observe  these  phases occurring concurrently in different  pockets of the organisation.  restarting bpr is often  more difficul t  than gaining the initial commitment. use of consultants most of  the  bpr  implementations we have observed  have  involved  the  use  of  consultants. taking a new look at an organisation's  processes is difficul t  for those  inside the firm without some  external  inputs.  however  consultants need  to  be  part  of  a  team  headed  by  a  senior manager  in  the  firm.  once  the  first  redesigned  processes are  in  place,  subsequent  redesign can be  undertaken with a  lower level  of consultant involvement.  firms  implementing  bpr need to carefully  manage consultants to ensure the transfer of skills  from consultant  members to internal members of the team. role of technology we have identified four  different  ways where information technology  can play a pivotal  role in the redesign of processes. 1) systems development or renewal as stimulant and technology as enabler: in  some  instances,  the  redesign  was  stimulated as  part  of  a  systems  development  or renewal  activity  and  technology  is  very  much  a  driving  force  in  enabling  new processes.  examples include brash holdings, mlc and norwich financial  services. 2) technology supporting redesign: in  other  organisations,  such as  bpoaz,  the emphasis  is  on  technology  supporting  a redesigned  process.  greater value is extracted  from  the  i/t in place.  major changes  to the organisation's  i/t were not required for the level of change  implemented. 3) technology as both enabler and disenables myer/grace brothers  found  that widescale implementation of edi/qr  in  the  business could not proceed  until  renewal of their technology base was undertaken.  the current systems could not handle the data capture and processing  load generated  by integrated point­of­sale  systems  using  13 digit codes  for all merchandise.  in this  third  instance, edi  technology  is  enabling  the  extensive  change  of  processes  in  a  defined  area  of activity.  however,  this  cannot  be  extended  until  changes  are  made  to  the organisation's  the information technology infrastructure.  thus technology  can be both an enabler and a disenabler  for bpr. 4) technology discovery as stimulant and enabler: organisations  which have not previously used  i/t as major component  in  the delivery of their business might identify  a technology which radically changes  the way in which the  firm works and even  the very nature of the business.  flower and  samios, a small sydney­based  architectural  firm,  provides  an  example  of  process  change  which  was not  undertaken  as  such.  however  the  potential  of  the  application  of  cad/cam5 technology  was seen  by  the  firm's  leaders  and  implemented in  a  way which  changed computer aided design/computer aided manufacturing. 73 virtually  all  processes  and  created  a  redefined  business  (yetton,  johnston  &  craig, 1993). the role of technology  might also differ  in different  processes in  the same  firm.  the above roles are not mutually exclusive and could exist concurrently in large firms. the role of the i/s group i/s groups  are accustomed  to working with  many parts  of organisations  and  are essentially cross­functional  in  perspective.  i/s groups can  often  see what the business  needs  to do  to better manage processes.  however, this does not make  i/s managers  natural  leaders  of bpr implementations.  the  leaders  of  bpr  teams  need  to  be,  and  be  perceived  to  be,  senior managers who have a very strong and detailed grasp of the business.  sometimes  such  leaders can  come  from  i/s groups,  but  this  is  often  not  the  case  initially.  bpr  teams  need  some members  with strong  i/s backgrounds; but  if  they dominate, or are perceived  to dominate, support from key business stakeholders may be withheld if the activity is seen as just another attempt to spend more money on i/t. lessons learned organisations  facing difficul t  business conditions might see bpr as a way of bringing about radical  improvements  in  performance.  we  have  presented  reported  examples  of  bpr  in australian organisations  and mapped  these on a ten step  framework for implementing bpr. while  the  firms  quoted  have  reported  considerable  improvements  in  their  processes, extensive  examples  of widespread,  well documented,  substantial  and  sustainable  change  in the performance  of australian  firms brought about by bpr are not yet apparent.  there are however,  some  key  lessons  which emerge  from  the  early  experiences  of  australian  firms. these  include: 1)  the need for strong, visible executive management support 2)  the importance  of constant communication of the motivation and progress of process changes 3)  the use of consultants to aid lateral  thinking 4)  the need to manage consultants and ensure skills transfer 5)  the importance of measurements for both current and redesigned  processes 6)  the need for an external customer or supplier focus in the redesign of processes 7)  education and support of all stakeholders  in the process change 8)  determining an appropriate role for technology 9)  recognition  of, and preparation for, the different  phases of commitment to bpr which might be evident at different  times 10)  the  importance  of  focusing  on  the  nature  of  work  and  the  role  of  people,  not  the organisation or its technology the  changes  implemented  by  some  organisations  vary  from  simplifying  to  reinventing processes.  the extent of process change  appears to be related to a combination  of  factors: the  desperation  of  the  organisation  and  current  state  of  its  processes,  the  availability  of suitable  technology  to  enable  and  support  sustained  change,  cultural  factors  related  to organisational  readiness,  and political  considerations  concerning  the acceptability  of  radical 74 change.  we expect  that some organisations which have already had the experience of process simplification, might now take a more radical approach to some of these  issues. conclusion conceptual  and  consulting frameworks emphasise  the  inter­linkage of  strategy,  technology, jobs,  structure,  values  and beliefs  and  management  systems  (scott  morton,  1992;  hammer and champy,  1993).  each of these need  to be aligned to bring about and sustain  successful process  change. the experience  of australian firms outlined in this paper demonstrates  the complex nature of bpr.  for  many  of  these  firms,  bpr  has  been  a  change  approach  which  was politically, managerially,  and  often  technically  difficul t  to  implement.  however,  those  which  have implemented  process  change  successfully,  and  put  in  place  an  ongoing  process  change strategy, find that they are well placed to handle competitive business pressures. references: alexander,  maggie  (1993).  "digital  south pacific's quantum leap", business change and re-engineering.  1(3), pp. 26­33. baldwin,  steven  r.  (1993).  "solving  the  reengineering  paradox", frameworks for organizational change.  deloitte touche tohmatsu international. bevilacqua,  richard  j.  &  thornhill,  daniel  e.  (1992).  "process  modeling", american programmer. 5(5), may.  pp. 2­9. broadbent, marianne and butler, carey  (1993).  "initiating business process  redesign  at bp oil australia & new zealand", business change and re-engineering.  1(2), pp. 6­13. broadbent,  marianne,  butler,  carey;  hansell,  allan  &  dampney,  c.n.g.  kit  (1993). business value, quality & partnerships: australasian information systems management issues in 1992.  melbourne:  graduate  school  of  management, university of melbourne. july.  [working paper no. 5]. butler,  carey  (1993). the role of information technology in business process redesign: some observations from the literature .  melbourne:  melbourne  business  school, university of melbourne. december.  [working paper no. 18]. butler,  carey  &  broadbent,  marianne  (1993). trading on (the) line.  myer/grace  bros department  stores.  electronic  data  interchange  and  quick  response.  melbourne: melbourne case study services, melbourne business school, university of melbourne. butler  cox  foundation  (1991). the role of informatio n technology in transforming the business.  london: bcf, january.  [research report no. 79]. carr, david k.; dougherty, kevin s.; johansson, henry j.; king, robert a. & moran, david e. (1992). breakpoint: business process redesign. arlington, va: coopers  & lybrand. craven,  john  (1991).  "how  to  implement  it  through  a  strategic  plan:  creating  process change". paper  presented at: keeping your professional edge.  australian society of cpa's victorian congress  1991, melbourne. currid,  cheryl  (1992).  re­engineering?  "you  need  some  outside help", computerworld (australia) .  23 october,  p. 24. davenport,  thomas  h. & short, james e. (1990). "the  new industrial engineering: information technology  and business process  redesign", sloan management review. 31(4),  pp. 11­27. 75 driva, helen & hunt, bob  (1994).  "quality brings dividends for  top motoring  organisation", australian. 23 march. economist  (1990).  "information  technology:  the  ubiquitous  machine", economist. 315(7659),  16 june.  pp.  s5­s20. emery, james c. (1991).  "re­engineering the organization", mi s quarterly . march, pp. iii­iv . freedman, david h. (1993).  "fannie mae's fast footwork", forbes asap. june, pp.  31­32. fried, louis (1991). "a blueprint for change", computerworld (us). 25(48), pp. 91­95. gulden, gary k. & reck, robert h. (1992).  "combining quality and reengineering efforts  for process excellence", informatio n strategy. 6(3),  pp.  10­16. hammer,  michael  (1990).  "reengineering  work:  don't  automate,  obliterate", harvard business review. 68(4), pp. 104­112. hammer, michael & champy, james  (1993). reengineering the corporation: a manifesto for business revolution. new york: harper business press. head,  beverley  (1993).  "taking  the  re­engineering  plunge, but  not  automating  the  mess", financial review. 16 october,  p. 16. index group (1993). critica l issues of information systems management for 1993: the sixth annual survey of i/s management issues. boston, ma: csc index. january. kelly,  robert j.  (1993).  "the  revival  of  information  technology  at  tnt", informatio n technology and reengineering the business.  csc  index  1993  pacific  region conference. sanctuary cove, queensland, 14­16 november 1993. keen,  peter  gw  (1991). shaping the future: business design through information technology. boston, ma:harvard business school press. king, julia  (1991).  "re­engineering:  rip  it up!", computerworld  (us).  25(28),  15 july. pp. 55­57. lynch,  david  (1994).  "transforming  the  performance  of  melbourne  water", business process re-engineering '94.  aic conferences, sydney, 1­3 march 1994. mis  (1993).  "behind  the  news:  re­engineered  1  and  re­engineered  2", managing informatio n systems.  1(8), pp.  16­17 mitchell,  lisa  (1993).  "smart  software  takes  the  guesswork out  of  change modelling", the age. 31 august, p. 16. olson,  margie  (1992).  "transforming  the  organisation:  information  technology  and business  engineering".  speaker  at  a  seminar  held  by  the  australian  computer society at the graduate school of management, university of melbourne, melbourne, 5 november 1992.  pp. 22­26. rigby, darrell. (1993). "the secret history of process reengineering", planning review. 21(2), pp. 24­27. roach, jennifer  (1994). "mlc re­engineers with imaging", computerworld (australia).  11 march,  p. 6. scott morton, michael s. (1992). the corporation of the 1990s: information technology and organizational transformation. new york:oup. stadler, david a. & elliott, stephen a. (1992).  "remake  your business", inform .  6(2),  pp. 12­17. yetton, philip; johnston, kim d. & craig, jane f. (1993). "the pencil­less architect's  office: a  "deviant"  case  study  of  the  dynamics  of  strategic  change  and  information technology". proceedings of the fourteenth international conference on informatio n systems, orlando, florida, december 5­8,1993. 76 microsoft word vol 15 2 final.doc australasian journal of information systems volume 15 number 2, 2009 169 is knowledge and skills sought by employers: a content analysis of australian is early career online job advertisements mary anne kennan maryanne.kennan@unsw.edu.au dubravka cecez-kecmanovic dubravka@unsw.edu.au patricia willard p.willard@unsw.edu.au concepción s. wilson c.wilson@unsw.edu.au information systems, technology and management australian school of business the university of new south wales sydney nsw australia abstract the purpose of this paper is to develop an understanding of the knowledge, skills and competencies demanded of early career information systems (is) graduates in australia. online job advertisements from 2006 were collected and investigated using content analysis software to determine the frequencies and patterns of occurrence of specific requirements. this analysis reveals a dominant cluster of core is knowledge and competency skills that revolves around is development as the most frequently required category of knowledge (78% of ads) and is strongly associated with: business analysis, systems analysis; management; operations, maintenance & support; communication skills; personal characteristics; computer languages; data & information management; internet, intranet, web applications; and software packages. identification of the core cluster of is knowledge and skills – in demand across a wide variety of jobs – is important to better understand employers’ needs for and expectations from is graduates and the implications for education programs. much less prevalent is the second cluster that includes knowledge and skills at a more technical side of is (architecture and infrastructure, operating systems, networks, and security). issues raised include the nature of entry level positions and their role in the preparation of their incumbents for future more senior positions. the findings add an australian perspective to the literature on information systems job ads and should be of value to educators, employers, as well as current and future is professionals. australasian journal of information systems volume 15 number 2, 2009 170 introduction the discipline of information systems (is) has often been characterized by the diversity and complexity of its subject matter (e.g. avgerou, 2000; paul, 2002). the intellectual foundations of the is discipline and its body of knowledge reflect complex processes where the technical, business and social phenomena are continuously interacting and transforming each other. as the is discipline aims to produce knowledge that is relevant and applicable in practice, it tended to develop in response to the enfolding of these phenomena and changing needs in the is practice. the broad motivation for this paper arises from the need to continually investigate and understand the is discipline’s body of knowledge from the perspective of is practice. specifically in this paper we aim to examine what knowledge and skills employers require from is graduates by analysing early career is job advertisements. such a practice perspective of the is knowledge and skills is important for both is teaching and research. on the one hand, better understanding of the required body of knowledge and skills is important for the development and delivery of education programs; and on the other, it helps the is discipline fulfil its mission and produce knowledge that is relevant and applicable in practice. given that the is discipline is relatively young and that its identity in academia as well as in practice is fast changing, we need first to delimit the domain of our study. there have been many discussions on what comprises the is discipline and there is no consensus on a definition (e.g. avgerou 2000; avison & eliot 2006; avison & fitzgerald, 2002; paul 2007; paul 2002). avison and fitzgerald, for instance, define is as “the effective analysis, design, delivery and use of information for organizations and society using information technologies” (2002, p174) which delimits a broad subject domain, in an editorial for misq, lee (2001, p iii) emphasises that the is discipline “examines more than just the technological system, or just the social system, or even the two side by side; in addition it investigates the phenomena that emerge when the two interact” that we find pertinent for our study. we are targeting is jobs that involve is development, management and use in business, organizational and social contexts, implying phenomena emerging in their complex interactions. is graduates, generally, come from business school programs that variously may be named management information systems (mis), computer information systems (cis) or information technology (it) (abraham et al. 2006, p1148; avgerou 2000 p567). in australia is courses are sometimes taught in faculties with names such as information technology. in both cases the courses and therefore the graduates are generally differentiated from computer science, computer engineering and information science programs (abraham et al. 2006, p1148), thus, this study excluded job advertisements that sought candidates with degrees in computer or telecommunications engineering, computer science, or where technical qualifications (such as microsoft certified systems engineer (mcse)) obtained outside the university system were a requirement. examples of job advertisements included were those with degree titles such as bachelor of information systems or information technology, bachelor of business or commerce with an is or it major, bachelor of science with an is or it major. the australian computer society (acs) has attempted to clarify the areas of knowledge appropriate for it professionals (underwood 1997). it introduced an accreditation process thereby impacting on the development of undergraduate programs at australian universities and in turn the mix of knowledge and skills new graduates bring to the job marketplace. a number of factors provided the motivation for the study of australian is jobs reported here. student numbers in is schools at universities in australia and elsewhere are reported as dropping (lea 2006) while in contrast employers are facing difficulties in recruiting appropriate qualified australasian journal of information systems volume 15 number 2, 2009 171 graduates (alexander 2007). this quandary was discussed in a keynote presentation at the australasian conference on information systems by grant (2007) who suggested some of the causes may result from a lack of clarity about career paths and the nomenclature of the it professions. misperceptions by potential students about job opportunities and the strength of the job market are fuelled by this lack of clarity and media reports of off-shoring. there are also differences in the perceptions of educators and employers about the knowledge, skills and competencies required of is graduates the study investigates what australian employers actually state they want in job advertisements (job ads). the requirements employers express are a major means by which they indicate, more or less effectively, what they want from potential employees. the paper investigates a sub-set of the australian it job market by analysing ads for positions suitable for early career graduates, also referred to as recent graduates (defined here as new graduates and those with up to three years work experience) from university level is programs. through this analysis we aim to identify what australian employers currently require in terms of is knowledge, skills and competencies as well as any other expressed requirements such as communication skills or personal characteristics. in addition the data analysis aims to throw light on the broad structure of jobs (as represented by the ads) through the identification of groupings of the knowledge, skills and competencies in the requirements specified by employers. several issues determined the focus. primarily there are conflicting views about the australian is job market: while many employers believe that there are not enough new graduates to cater for present and future needs, falling numbers in is programs in recent years suggest that prospective students are concerned that there are not enough jobs or that the jobs which are available are not to their liking. while this research will not directly address whether there are, and will be for the foreseeable future, too many or too few graduates for the work available, it will indicate the skills and knowledge in demand to date. this has value for students in selecting courses and for is academics in planning and delivering them. a second reason for restricting the analysis to jobs appropriate for recent graduates is the belief that after several years in the workforce, experience and post graduate studies become increasingly important in decisions about the employability of candidates. the findings of other research indicate that different knowledge and skills are valued in higher level positions (zwieg et al. 2006, p104). this would be reflected in the content of the ads and lessen the usefulness of the findings of this research for establishing what knowledge, skills and competencies are sought in new graduates. while not claiming to throw light on all aspects of the employment market, job ads are probably the most easily available indicator of the short to mid-term direction of workplace demands for particular knowledge, skills and competencies. they provide a window to what is currently wanted (required as well as preferred) and hence to what employers believe they need for the organization to continue and thrive. the analysis of ads might also encourage employers to reconsider the way they express their needs in the ads. this paper briefly situates the study in the literature, describes the research method, essentially the collection and content analysis of job ads, and then presents the results. it concludes with a consideration of the issues arising, particularly the role of entry level positions in the preparation of their incumbents for future more senior positions. australasian journal of information systems volume 15 number 2, 2009 172 literature a literature review relevant to this project requires some attention being given to a ‘definition’ of the is field in australia and how this fits with related areas. the australian computer society, while recognizing the difficulties in terminology of differentiating between different parts of the it family, presents a three field model (see figure 1) which places is on a continuum between computer science and the business and management related fields in the discipline of it (underwood 1997). the acs report attempts to identify a “core body of knowledge” (cbok) for the it disciplines, which it says should be included in all it programs across the three disciplines identified in this 1992/1997 representation of the field (computer systems engineering, computer science and information systems). it recognizes that the scope of the specified cbok is very broad and could not readily be covered in an undergraduate course and therefore recommends division into two groups. first a mandatory group referred to by acs as a “generic” requirements group. in the 1997 cbok document this included interpersonal skills, ethical and social implications, professional practice; and project management and quality assurance. the 2007 cbok document suggested the “generic” requirements were ethics, project management and interpersonal communication. a second group was proposed in both documents from which courses could be selected depending on disciplinary focus and student interest as well as the accreditation requirements of the acs. figure 1: acs (underwood 1997) representation of three disciplinary groups within the it field an aim in updating the cbok has been clarification of the roles appropriate for entry level it professionals and the knowledge, skills and competencies involved (australian computer society professional standards board 2007). this review is a work in progress with the intention of completion by june 2008. both the cbok and the discussion paper refer to the profession with the discussion paper acknowledging that the terms information and communications technology (ict), it, and computing “are used more or less interchangeably in australia to refer to the australasian journal of information systems volume 15 number 2, 2009 173 profession as a whole” (p4). this lack of clear definition and distinction among the fields is seen as detrimental to disciplinary developments with several major ramifications; one of the most critical being public perception of different professions and the marketing of careers to prospective students in the various fields within it (p4). the lack of clarity almost surely also applies to employers as they seek to name and describe positions. it also does not help the developers of academic programs in the various fields who seek to give their students as wide a choice of potential employment as possible. to compound the issue of what should or should not be included in programs for is, clarke (2006) chronicles the fairly recent development of the is discipline in australia noting the “enormous breadth of topics addressed within the is discipline” (p131). the growth of topics has probably been influenced by the fact that is as an academic discipline has evolved differently around the world “with identifiably different local forms e.g. from a soft systems emphasis to a more technical focus” (gable 2006, p103). therefore we may assume that is practice and is jobs also may have evolved somewhat differently. this would seem to be justification for studying aspects of the is job market specifically in australia rather than relying on the findings of studies in other countries to guide such things as degree program development. the discussion paper suggests that a mechanism for viewing the profession can come from understandings conveyed by the word “profession” which it discusses in relation to the development of the revised cbok. it cites “professions australia” criteria of a “special knowledge and skills in a widely recognized body of learning derived from research, education and training” and the “possession of a code of ethics” as providing useful guidance. detailed studies of the professions in general indicate that a central element of each is an accepted body of knowledge over which the profession claims unrivalled expertise. abbott (1988) terms this claimed expertise the profession’s “jurisdiction”, known as “turf” in a colloquial jargon. coupled with this is a system of certifying individuals that have mastered that body of knowledge. there are other important elements which constitute a profession, which abbott sees as constituting a “system”, and which vary in significance from profession to profession. claiming, maintaining and re-claiming jurisdiction is an on-going task for the profession. macdonald (1995, pp188-189), following larson (1977), characterises this task as being a significant element in the “project” that professions engage in to maintain their identity. however, the intensity of activity in defining jurisdiction varies from time to time: most intense as the profession is established, and when it is under some challenge from rival professions. the discussion paper considers that an understanding of what the it professions entail and the nomenclature used to describe them varies widely. a stated aim in the discussion paper is the “redefining and building” of the it professions to develop a wider understanding of what it means to be an it (or is) professional and the knowledge, skills, competencies and education required. our paper aims to contribute to that debate by looking at the knowledge, skills and competencies employers state that they require in job advertisements whilst also acknowledging that any university based education needs to cover areas perhaps not explicitly acknowledged by employers, and which differentiates a university educated professional with the ability to take a critical and wider worldview than a “trade school” graduate with a simple “how to” focus (metcalfe 2006). identifying what employers want from new staff can be approached in a number of ways. commonly used approaches in investigations of this kind have been interviews with, or questionnaires of, employers and/or job seekers, or the unobtrusive method of the content analysis of, for example, newspaper or online job ads (litecky et al. 2004). a number of american studies have tracked the is job market over time to identify the direction of change and current employer requirements (lee 2005; litecky et al. 2004). an often cited study by todd et al. (1995) surveyed newspaper job ads spanning the 20 years from 1970 to 1990 and found australasian journal of information systems volume 15 number 2, 2009 174 little change in the types of requirements specified for programmers and is management positions. systems analysis positions had experienced the greatest change and they remark that contrary to expectations there had been an increase in the specification of required technical knowledge and a slight decrease in requirements for business and systems knowledge (p1). a subsequent newspaper job ads study (gallivan et al. 2004) which spanned 1988 to 2003 identified a similar trend. the authors found ongoing evidence of a recruitment gap between employers’ articulation of a desire for business knowledge and ‘soft skills’, and jobs ads concentrating on ‘hard skills’. interestingly they claim that this is due to it being easier to screen for “hard skills” (gallivan et al. 2004, p83). in seeking to explain this apparent contradiction litecky et al. (2004) propose that the hiring process is composed of two stages. they suggest that the first stage scrutinises potential employees against specified technical requirements (‘hard skills’) stated in job ads, whilst the second stage uses the job interview for the investigation of ‘soft skills’ such as communication skills for making the hiring decision. the recent interview-based it workforce investigation sponsored by the society for information management advocacy program (zwieg et al. 2006) remarks on the difficulties ahead as the use of off-shore workers for technical work may have unintended consequences. it argues that this will produce a shortage of people who have had the opportunity to develop the project and businessrelated skills required for mid-level work. these skills are often developed in entry level work work which has in the past been more often of a technical nature (zwieg et al. 2006). unlike a number of other studies which address employers’ demands as a composite, this research distinguished between the demands for entry-level and mid-level positions. the researchers also point out that entry-level positions more often seek the technically skilled with almost all the skills sought being technical ones. when asked about missing skills in recent recruits, employers identified “communication skills” and indicated that new entrants with good “communications skills” were of high value to them (p104). the researchers argue that the new it professional should have a balance of technical, business and project skills to prepare them for their careers and that in the present work situation the old model of picking up non-technical skills over time and with experience does not apply (p106). a recent american unpublished exploratory study (marion 2006) using 100 ads from two electronic job listings found once again that a large part of the specified requirements in ads were “technical skills”. marion notes that her findings support the proposition that is professionals need to understand business culture and operations to function well in their workplace stating that the term “integration” was frequently mentioned in different contexts in the ads. the importance of business and behavioural skills was stressed by lee et al. (1995) and a decade later bassellier and benbasat (2004) concluded that it professionals need a growing range of non-it skills (p691). interestingly, another study (letch and randolph 2000) indicates that while end-users in functional areas of business are more involved in is and it use and development, they do not see their roles extending to the traditional domains of is professionals such as programming, is design and systems installation and integration. it would appear therefore, that is professionals need to know more about business, and business and functional professionals need to know more about it, but the need for high level specialisation remains. north american studies explored the nature and extent of gaps between the perceptions of is academics and is practitioners about the field revealing that practitioners appear to value interpersonal skills, understanding of organisations and personal traits more than academics who in turn more highly value software tools (lee et al. 2002; trauth et al. 1993). trauth et al. (1993) identified an “expectation gap” between industry needs and academic preparation claiming that is jobs had been changing and that academic programs had to keep up with this. this finding was supported in more recent australian investigations (snoke et al 2002; snoke and underwood 2001). australasian journal of information systems volume 15 number 2, 2009 175 a delphi technique was used to elicit academic and industry expectations of the generic attributes of is graduates. the results were compared with the results of a content analysis of texts and academic program unit outlines and identified some disparity between the perceived needs of industry stakeholder and academics perceptions and curriculum requirements. industry highly rated individual motivational attributes such as learning and working independently. academics rated more highly generic interpersonal attributes such as teamwork and communication skills and also technical knowledge and competencies. many of these investigations have produced lists of skills and competencies identified as being more or less in demand. with the previous studies in mind, and the need established for educators and potential students to understand what skills and competencies employers actually require at any point in time, this study addresses the following questions: 1. what knowledge, skills, competencies and personal characteristics do employers list in current job ads for is graduates in australia? 2. are these different from those identified in previous overseas studies? if so, how are they different? the contribution of this work lies primarily in that it addresses demands in the australian workplace and so will supply information to australian educators, students and employers. as there are similarities with the design of some american papers – the focus on jobs appropriate for recent is graduates and a desire to throw light on the mix of skills which are currently in demand – this will allow for comparison. research method assumptions and limitations a fundamental assumption of this investigation is that the content of job ads is a valid representation of the labour demands of employers. it is acknowledged that some ads are likely to be more accurate in their specification than others. reasons for this may include a lack of experience in writing ads and/or a lack of clarity about what is required. there may also be non job-task factors which influence what is included and excluded such as company policy or state legislation or the desire to attract a high standard of applicant. in addition, ads vary in length and detail though the motivation to keep ads brief because of the cost associated with length is not the concern with electronic ads as it is with newspaper ads; as electronic ads are usually charged for on the basis of the duration of their listing. the study is limited to jobs suitable for recent is graduates justified on the grounds that is is a large field and focusing the investigation on a part of it is likely to produce more useful results than a study attempting to analyse all levels of positions. the concern about recent graduate perceptions stated above made this an easy decision. no study could hope to identify all available jobs at any point in time as jobs are advertised in a wide range of sources and some are filled without public advertisement, by for example word of mouth and the campus and web page recruitment advertising of large companies and employment agencies. however, analysis of ads in widely used and publicly available job listings is likely to turn up a fairly representative list of what is available and the knowledge, skills and competencies in demand. australasian journal of information systems volume 15 number 2, 2009 176 research design the unobtrusive method of job ads content analysis was chosen for this investigation in the belief that a broad sweep could be achieved with quite limited resources. content analysis is a technique through which content, for example text or images, are examined in a systematic way. content analysis begins with pre-defined categories developed from previous literature which are iteratively extended and refined through the analysis. (ezzy 2002). the analysis involves counting how often words, phrases or themes appear individually or in combinations. the results are then organised systematically for quantitative analysis of the text’s content. content analysis is a frequently used descriptive technique (neuman 2006). the identification of jobs appropriate for recent graduates involved both choosing sources for the ads and the selection of ads from them. three major australian online job lists have been used – jobserve (http://jobserve.com.au), seek (http://seek.com.au), and mycareer (http://mycareer.com.au)4. the search terms and strategies for the sources are not identical; however, all used the broad term it&t (information technology & telecommunications) and all allowed further refinement to express graduate and/or entry level. in addition searching could be done on more specific terms such as business analyst. this was done for several terms likely to be in job titles – consultant, business analyst, e-commerce, erp, and is project management and in most cases the position had already been identified in the general search. other refinements such as geographic restriction were possible but were not used. the choice of the categories under which ads are listed for example it&t, graduate/entrylevel are specified by the advertisers so misspecification by them could result in ads being missed however it probably would be very few. preliminary scanning of the job listings showed that appropriate jobs have a wide variety of titles. in addition reliance on university program names had limited usefulness as there is no uniform language and not all ads specified the qualification required though it appeared from the content that the requirement of “graduate” did not mean absolutely any degree. these two factors limited the use of quick scanning in ad identification. as our focus was the is field, we excluded those ads where a computer science or engineering degree was stipulated, and focussed on those ads which specified degree titles such as bachelors of information technology and information systems, bachelors of business or commerce with an is or it major, bachelor of science with an is or it major. where no degree was specified, we were guided by the criteria in the area indicative of “is” within the acs representation of the broader it field reproduced as figure 1 of this paper. criteria for the selection of ads were then tested in discussion with colleagues from the is and it fields before the data collection began. other criteria used in defining appropriate jobs included salary package (minimum $35,000 to a maximum of approximately $80,000 per annum), and experience requirements that would not exclude early career graduates (defined here as new graduates and those with up to three years work experience). we attempted to eliminate the duplicate ads but cannot guarantee this as the numbers 4 these sources were also used by the commonwealth department of education and workplace relations (dewr) in preparation of its monthly ict vacancy reports as they provide a broad national coverage (australian government department of employment and workplace relations 2006). australasian journal of information systems volume 15 number 2, 2009 177 of jobs scanned was quite large even after the restriction of graduate/entry level was used5. no assumptions can be made with regard to the probable percentage of entry level ads in comparison to the total number of ads. it&t included ads that covered the broad spectrum of positions and this study focuses on is positions, specifically entry level positions. in addition the online job lists not only duplicated ads between themselves, but often ads were repeated within the same online job list, even on a daily basis. thus, we selected the ads on a purposive basis excluding duplicates, according to the criteria listed above. for reasons similar to those preventing us from making assumptions about the proportion of entry level jobs we cannot make estimates about the proportion of ads excluded as a result of being deemed more suitable for other types of it graduates, such as computer scientists and engineers. to reiterate, this is not a study of the size of the job market, rather the focus is on the knowledge, skills and competencies specified in ads targeted at entry level is graduates the ads were gathered over ten weeks from 12 july to 13 september 2006. a preliminary analysis after 50 ads had been gathered showed that the approach was identifying a range of job titles and requirements. this process of job identification was found to be quite time consuming. as the period of data gathering proceeded it became clear that the same types and mix of ads were recurring. we ceased collection at ten weeks with a final sample size of 400 ads. essentially the strategy was to access one of the sources, apply the search terms and work through the entries retrieved in the several hours available for a session. the order of the ads was chronological by placement date with the most recent at the top of the list. in addition to checking the job title and brief description, the fuller description of many ads, which was available on a second screen, had to be read. two researchers worked together reaching agreement about whether the ads met the definition of an early career or entry level is graduate position defined above. once an ad was deemed appropriate it was downloaded. in addition the following data was recorded if available: the geographic location of the job, the salary range, the specification of experience or qualifications. the job titles and ad content was downloaded into simstat/wordstat (provalis research 2005) a word counting software package which enables the production of an hierarchical dictionary developed from the terms describing the desired knowledge, skills and competencies specified in the ads. the creation of the user-defined inclusion dictionary to group related words into meaningful categories is a critical step in enabling the most informative use of the software. words and phrases in the ads are examined and placed within a developing picture of the knowledge, skills and personal characteristics sought in the ads. whilst the actual words used in the ads are the basis for the dictionary, the sources used in developing major and minor categories included the knowledge of the researchers and their colleagues, various published accounts of is work such as the australian and new zealand standard classification of occupations (australian bureau of statistics and statistics new zealand 2006) and the coding schemes of a number of is work studies (marion 2006; prabhakar et al. 2005; gallivan et al. 2004; todd et al. 1995). table 1 contains the categories used including examples of dictionary terms under the 17 category labels developed. 5 dewr which scans four weeks of ads gathered over 20,000 under it&t for the four weeks to mid april 2006 and note that this included some duplication. (australian government. department of employment and workplace relations 2006). australasian journal of information systems volume 15 number 2, 2009 178 category label examples of dictionary terms architecture & infrastructure architecture, infrastructure, technology environment, itil business analysis, systems analysis ba, analyse and design, business analyst, business processes, business solutions, business and technology solutions, functional specification, monitor, system specification, system analysis, technical specification communication skills articulate, interpersonal skills, negotiation presentation, oral, written, verbal computer languages asp, c+ , c++, cobol, code, java, perl, sml, sql, uml, vb, visual basic data & information management data administration, data analysis, data cleansing, data content management, data integrity, edi, information management, relational database, warehouse enterprise resource planning enterprise application, enterprise architecture, enterprise enabling, enterprise resource planning, enterprise solutions, erp, peoplesoft, sap hardware hardware, laptop, mac, mainframe, pc, internet, intranet, web applications apache, coldfusion, cookie, css, domino, e-business, e-commerce, extranet, html, web design, xml, xhtml, xsl is development analyst programmer, applications design, applications development, development, documentation, j2ee, jee, lifecycle, .net, object oriented, quality assurance, re-engineering, testing, visual studio. is project management implementation, project management, scoping management bpo, business intelligence, business strategy, distribution chain, marketing strategy, supply chain. networks active directory, cisco, citrix, dhcp, dns, ip, lan, uucp, socket programming, vpn, wan operating systems linux, unix, windows, win, xp, nt operations, maintenance & support first level support, applications management and support, automation, bugs, change requests, desk top support, customer training, diagnose, disaster recovery, helpdesk, operational maintenance and support, training end users, upgrades personal characteristics adaptable, ambitious, attention to detail, business acumen, can do attitude, client relationship, confident, creative, desire to learn, eager to learn, energetic, enthusiastic, flexible, initiative, innovative, intelligent, leadership, organized, passionate, proactive, problem solver, quick learner, team player, relationship builder, self motivated, time management, work ethic, work independently security aci, audit, firewall, forensic, security, spam, threat software packages adobe, crystal reports, excel, lotus notes, microsoft office, photoshop, powerpoint, visio table 1: content analysis categories australasian journal of information systems volume 15 number 2, 2009 179 in determining all 17 categories, in addition to consulting the literature, academics and students with relevant knowledge and experience were consulted about what should be included. there was consensus that the skills and competencies listed under is development would be a part of any is graduate’s knowledge base, with specialization depending on electives selected and personal interests. the view was that even if graduates did not have hands-on experience in these areas, they would have an understanding of how these skills and competencies would fit in the occupational/business context. as is development includes such a wide sweep of activities a further breakdown was undertaken to establish the relative demand for various sub-categories within it. table 2 includes examples of the dictionary terms included within the sub-categories. sub-category label examples of dictionary terms is development broad terms applications design, applications development, documentation, lifecycle, software analysis object oriented object oriented, oo, oop programming programmer, programming. re-engineering reengineering, re-engineering testing testing, tester, qa, quality assurance, uat, tools, environments, technologies ado, api, dcom, j2ee, jee, .net, servlet, visual studio table 2: sub-categories of is development once the dictionary was constructed the dictionary categories were ranked to identify the frequency with which the terms within specific categories were listed in the ads. the next stage of the data analysis involved quantifying the structural characteristics of our set of job ads and looking for relationships; that is, categories which were found together in ads and categories which never or rarely occurred together. by definition, co-occurrence happens every time words in two categories appear in the same ads. wordstat offers a number of ways to measure co-occurrence. for this research we chose the jaccard’s coefficient similarity measure which is based on the occurrences of categories in the ads but does not take into account their frequency. this has the effect of compensating for large differences in counts for commonly occurring terms. the construction of the similarity matrix provides input for the multivariate technique of cluster analysis which is used to identify terms with similar co-occurrence patterns. this technique is part of the simstat/wordstat software package. using several methods to explore the data enables a more complete picture of the underlying structure. the cluster analysis is based on the pattern similarity of the 17 categories in the dictionary. this hierarchical agglomerative clustering begins by joining two terms with the most similar patterns according to the distance criteria (average linkage). following a similar linkage approach subsequent terms are joined to existing clusters until the one large cluster encompassing all 17 categories is formed. while there is no best number of clusters, the software identifies a ‘default’ cluster configuration based on the frequency of co-occurrence of terms. selection of the final cluster solution depends on researcher judgment; however, the resulting clusters should display within-cluster homogeneity and high between cluster heterogeneity. australasian journal of information systems volume 15 number 2, 2009 180 4. results almost half of the 400 job ads were collected from seek (199, 49.8%). a third came from jobserve (133, 33.3%) and the remaining 68 ads (17.0%)6 from mycareer. jobserve and seek are dedicated job listing companies whilst mycareer is a part of the fairfax publishing company which produces newspapers in sydney and melbourne and grew out of their classified job ads pages. all three sources claim national coverage. over half the ads (246, 61.5%) were for positions in the sydney metropolitan area, 65 (16.3%) were located in the melbourne metropolitan area with no other geographic area exceeding 10% of the ads. the software package was run against the content of the ads for the frequency counts and cluster analysis. table 3 ranks the dictionary categories according to the frequency of their occurrences in the ads. content analysis categories rank no. ads % ads is development 1 312 78.0 personal characteristics 2 295 73.8 communication skills 3 273 68.3 computer languages 4 204 51.0 data & information management 5 158 39.5 management 6 150 37.5 business & systems analysis 7 146 36.5 internet, intranet, web applications 8 146 36.5 operations, maintenance & support 9 139 34.8 software packages 10 87 21.8 operating systems 11 78 19.5 architecture & infrastructure 12 62 15.5 networks 13 52 13.0 is project management 14 51 12.8 hardware 15 43 10.8 enterprise resource planning 16 41 10.3 security 17 33 8.3 table 3: categories ranked by frequency of occurrence terms categorised as is development were the most frequently occurring; (in 312, 78.0% of the ads). as this was such a large and inclusive category it was further broken down into six areas. the ranked results of the breakdown are included in table 4. the more detailed analysis revealed that in addition to asking for specific software knowledge and skills, for example skills in testing and/or programming, a large number of the ads included terms which conveyed a broader scope here 6 throughout this paper, percentages may not sum to 100 due to rounding. australasian journal of information systems volume 15 number 2, 2009 181 termed is development broad terms. approximately three quarters of the 400 ads (298, 74.5%) and 95.5% of the ads providing a word match with terms under the category is development included terms in this broad sub-category. none of the terms in the more focused sub-categories occurred in more than half the ads. sub-categories of is development no. ads % is development ads % total ads is development broad terms 298 95.5 74.5 tools, environments, technologies 113 36.2 28.3 programming 89 28.5 22.3 testing 48 15.4 12.0 object oriented 25 8.0 6.3 re-engineering 5 1.6 1.3 table 4: ranking of sub-categories of the category is development the broader analysis sub-categories are not exclusive as job ads can ask for a mix of knowledge and skills which come from any number of the categories. an ad could have asked for knowledge and skills from more than one subcategory of is development (e.g. testing and programming) and from components of several other categories as well. table 2 shows that well over half the ads included terms in personal characteristics and communication skills. returning to table 2 in detail, knowledge of computer languages (computer languages), arguably one of the more technology oriented categories, was the fourth most frequently occurring category (204, 51.0%). the technologically oriented knowledge and skills (represented in these ads as the requirements under is development and computer languages) appear to form an important part of the requirements of many employers seeking recent is graduates. this technology focus in job ads is in keeping with the findings of the literature (todd et al. 1995; gallivan et al. 2004) and particularly in the content of early career job ads (abraham et al. 2006; zwieg et al. 2006). no other skills or competency category occurred in more than 50% of the ads. the two categories ranked last (enterprise resource planning and security) are probably so ranked because many of the positions requiring knowledge and skills in these areas are at a higher level than the positions targeted for investigation in this project. specifically, positions which have responsibilities for security and enterprise resource planning also have requirements for the type of business knowledge which is generally accepted to come with several years in employment in is work. personal characteristics and communications skills (ranked 2 and 3 respectively) were mentioned in nearly 75% of ads. this finding is in line with the literature which reports that employers put high value on various personal characteristics and communication skills. while a limited number of terms convey the idea of requiring communication skills (e.g. verbal, speak, write, articulate) this is not so with personal characteristics. the ads yielded over fifty terms to be included in this category (see table 1 for a selection of the terms). some of these terms were used very frequently such as “learn” which occurred in 78 cases (19.5%) in contexts such as “ability to learn”, “aptitude to learn” and “eager to learn”. the next stage of the data analysis involved looking for relationships; for example which skill categories were found together in ads and which ones never or seldom occurred together. wordstat offers cluster analysis as a method to address this. as discussed in the method section the cluster australasian journal of information systems volume 15 number 2, 2009 182 analysis is based on the pattern similarity of the 17 categories in the dictionary. using this approach the default emerged, which was a four cluster picture. these clusters are represented in the dendrogram, which is a graphical display of the clustering process (figure 2). figure 2: cluster analysis of the 17 categories as can be seen in figure 2, the dominant cluster consists of ten categories—business & systems analysis; management; operations, maintenance & support; communication skills; personal characteristics; is development; computer languages; data & information management; internet, intranet, web applications; and software packages. this cluster portrays what might be characterized as core is skills and competencies required by employers in job ads for early career is graduates. is development as the most frequently (78% of ads) required knowledge category seems to be central to this core is cluster. interestingly, the is development category is most strongly associated with personal characteristics and communication skills, thus indicating the importance of non-technical characteristics and soft skills for is jobs. the next strong is development association appears to be with computer languages and data & information management categories. given that 51% of ads require computer languages knowledge and skills, this category figures quite prominently in the core cluster. this finding is to some extent surprising as computer languages would be seen as core in more technical it jobs but not necessarily in is jobs. what employers actually mean when they request computer languages knowledge and skills in is job ads is not known. they may require general knowledge and understanding of computer languages rather then high proficiency, but this remains to be confirmed. the second cluster consisting of architecture & infrastructure; hardware; networks; operating systems; and security are all related to the more technical information technology aspects of is and would thus appear to be a rational and expected grouping. this cluster however overlaps with the software engineering and computer science body of knowledge, which requires further empirical exploration. this finding may suggest either that employers are seeking is graduates for a more technical side of the is job spectrum (but still requiring business and is knowledge) or that employers do not differentiate is from it jobs or lump them together in their ads. in addition two categories (enterprise resource planning and is project management) demonstrated little linking either with each other or with the other categories and are identified by the software as outliers. we have already speculated that the low ranking of enterprise resource planning in table 2 may be because it is more likely the preserve of more senior and experienced staff than the recent graduates australasian journal of information systems volume 15 number 2, 2009 183 on which this study focuses. this is also likely to be the case with is project management. another explanation might be that these two knowledge categories are more recent domains of early career is jobs that may expand in the future. it would be certainly interesting to monitor whether they will become in higher demand in the future. qualifications and experience in the research method section it was explained that the ad selection process aimed to restrict the selection to is positions appropriate for recent graduates. this was done through the selection of search terms (e.g. graduate/entry level), use of specific criteria (e.g. salary level) as well as investigation of the content of the ads. in regard to both salary and academic qualifications many ads made no specification. in respect of academic qualifications the omission of this would not necessarily have meant that the employer did not have an expectation that applicants would be graduates. as employers could specify retrieval terms for their ads it is probably fair to assume that if they indicated that they wished their ads to be retrieved from a search using the term ‘recent graduate’ they believed that that they had covered this point. two thirds of the ads (269, 67.3%) required academic qualification (table 5). not surprising, in light of the ad selection process, only 25 (6.3%) specified that they did not seek any qualifications. qualifications required no. total ads % total ads required 269 67.3 not specified 71 17.8 either university/tafe/microsoft 18 4.5 not required 25 6.3 either qualifications or experience 17 4.3 total 400 100.0 table 5: qualifications required analysis of the ad content for the requirement of appropriate work experience (i.e. in is work) showed that almost half of the ads (194, 48.5%) sought a person with some experience. a further 36 (9.0%) indicated a preference for some experience. while this stated preference is not concerning, the fact that almost half of the position ads retrieved on a search under terms conveying recent graduate required experience is of concern. even taking into account that some recent graduates would have had part-time work as students (which enabled them to answer the experience requirement) the number of ads requiring experience was high. there is no way of knowing what the employers who failed to specify whether experience was required (138, 34.5%) wanted. only 18 (4.5%) of the 400 ads indicated that experience was not required. australasian journal of information systems volume 15 number 2, 2009 184 experience required no. total ads % total ads required 194 48.5 not specified 138 34.5 preferred 36 9.0 not required 18 4.5 either qualifications or experience 14 3.5 total 400 100.0 table 6: experience required position titles for the job searcher the job title is the preliminary identifier as to whether a position may be suitable, hence some investigation of titles seemed a necessary step in this project. as already remarked it quickly emerged as the ads were being scanned and selected that there was a wide variety of position titles and that in some cases these were not very informative (e.g. ‘it graduate’). by contrast some positions had very specific titles (e.g. ‘j2ee developer’). almost half of the ads had the words ‘junior’ (89, 22.3%) or ‘graduate’ (92, 23.0%) as part of their title (e.g. ‘junior business analyst’, ‘graduate business analyst’). the most frequently occurring term in a job title was ‘developer’ which occurred in 97 (24.3%) of the position titles. in addition it was clear from the description of a number of other positions that developer was implied in the ad title (e.g. ‘fun with vb’ and ‘vb.net’ and ‘vb6/vb’). the next most frequently used terms were ‘consultant’ or ‘consultancy’ occurring in 56 (14.0%) of the ads. the positions under this heading varied, some dealing with technical areas and others having more of a business management focus. no other term appeared in more than 10% of job titles. programmer, which in some titles was linked with analyst and in others with specific software (e.g. ‘graduate programmer vb.net’) occurred in 32 (8.0%) of job titles and was the third most frequently occurring term. a small number of jobs had titles indicating that they were ‘help desk’ positions however there were also positions which included the term ‘support’ in their titles which appeared likely from the description to be help desk roles possibly bringing the total of this type of work to around 8%. further analysis of job titles and content of their associated ads may reveal that at least some positions were poorly titled in respect to the work involved, which has been identified as a problem (grant 2007). we speculate that some position titles might be thought to be a turn off for particular applicants, such as ‘help desk’ though whether this influences potential employers not to use them is not known. employing organizations in addition to specifying their requirements many employing organizations choose to say something about themselves. in addition to comments about aspects such as size and purpose many comments dealt with the organizational environment the new recruit would be entering. characteristics of the organizational environment were mentioned in 316 (79.0%) of the ads. terms which the researchers identified as conveying this type of message included ‘fast-paced’, ‘friendly’, ‘team’ and ‘vibrant’. it could be argued that in conveying these messages about the workplace the employer was indicating the types of characteristics a potential employee might need to fit into it. the use of team structures was often expressed in the ads as were words which conveyed a dynamic culture. of course such words do not necessarily represent what a workplace is like and few employers would australasian journal of information systems volume 15 number 2, 2009 185 advertise unattractive workplace characteristics, however they probably represent what management would see as desirable. discussion the purpose of this project was to gain a picture of what australian employers want in knowledge, skills and competencies in new entrants to the is workforce. the job ads revealed the dominant cluster of knowledge categories that might be considered as core knowledge and skill competencies for is jobs: business & systems analysis; management; operations, maintenance & support; communication skills; personal characteristics; is development; computer languages; data & information management; internet, intranet, web applications; and software packages. the competency with the highest frequency of mentions and which also appears centrally in the cluster analysis maps we categorized as is development. as this was such an inclusive category (including words representing knowledge and skills in areas such as tools, environments technologies; testing and programming) a sub-categorisation was used to illuminate demands. perhaps not unexpectedly terms categorised under is development broad terms were found in almost all the ads: in 298 (95.5%) of the 312 ads in which there was a word match with terms under the category is development and 74.5% of the total 400 ads. as is development broad terms included terms such as applications design, documentation and software analysis this is not surprising. employers would almost certainly expect recent graduates to have gained a sound foundation in these areas during their studies and be able to contribute to work in these areas even if a position was advertised as being primarily focused on for example, working with specific software. computer languages occurred in more than half of the ads and is strongly associated with data & information management and is development. the high demand for computer languages in the is entry position ads may be explained by the technical nature of these entry level positions. the high ranking of is development and computer languages is similar to findings in other studies (todd et al. 2005; gallivan et al., 2004) that found emphasis on “hard” skills in is job ads. this may confirm litecky’s contentions that ads screen potential applicants’ for technical skills, whereas interviews are more likely to screen for “softer” communications skills and personal attributes. similarly it may add weight to zwieg et al’s. (2006) proposition that early career technical jobs are seen to provide a foundation for later career positions that require more business knowledge and management skills. while no other technical or professional categories occurred in more than 40% of the ads, many are newer skills categories (such as enterprise resource planning, is project management and security). these may not currently be receiving a great deal of attention in courses, but appear to be requested by employers, and so perhaps warrants further consideration by educators. in addition, as we noted earlier, positions incorporating these skills may often be at a higher level, so graduates may need to know them for the next step in their career. personal characteristics and communications skills (ranked 2nd and 3rd in frequency respectively, and also appearing in the centre of the core cluster) were mentioned in over 70% of the ads. importantly they are closely linked with is development in the core cluster of knowledge (figure 2). these findings confirm the claims in the previous literature about the high value employers place on these skills (zwieg et al. 2006). this also strengthens the argument in the acs proposal that this set of skills remain a component of the revised cbok (australian computer society professional standards board, 2007). australasian journal of information systems volume 15 number 2, 2009 186 the area of “ethics, social implications and practice/professionalism”, comprises an important subset of “generic” skills as defined in the cbok appears only in two ads, which is surprising. however, it is likely that this particular requirement like other “soft” skills is pursued at the interview stage. (litecky et al. 2004). with regard to the cbok category “practice / professionalism”; characteristics which we have categorised under personal characteristics include those which employers label with terms such as “desire to learn”, “proactive”, “innovative”, “relationship builder”, “team player” etc. were mentioned frequently. this reinforces the need for is education programs to consider creative ways of developing these skills. the variety of jobs available was reflected in the large number of titles for positions – developers, programmers, business analysts, test analysts web designers, erp consultants to name a few. positions including the term ‘developer’ were the most frequently advertised representing almost 25% of the ads. positions for ‘programmers’ were also quite frequent though the rather vaguer terms ‘consultant’ and ‘consultancy’ occurred more often. almost half of the ads had the words ‘junior’ (89, 22.3%) or ‘graduate’ (92, 23.0%) as part of their title which would be a helpful flag for the new graduate seeking to find suitable ads. two thirds of the ads (269, 67.3%) specified a requirement for academic qualifications (perhaps not surprising in light of the ad selection process) and only 25 (6.3%) specified that they did not seek any qualifications. almost half of the ads (194, 48.5%) sought a person with some experience. a further 36 (9.0%) indicated a preference for some experience. as the search for the ads was structured to identify jobs suitable for recent graduates the high level of requirement and preference for experience is concerning. while some recent graduates would have had part-time work as students the experience requirement is still worrying. while there is no way of knowing what the employers who failed to specify whether experience was required wanted (138, 34.5%) only 18 (4.5%) of the 400 ads indicated that experience was not required. this findings regarding the high number of graduate or junior positions requiring experience provide backup to the calls made in an earlier paper for an experiential business context to be added to is programs (abraham et al. 2006). nearly 80% of employers (316) chose to say something about themselves. the terms used conveyed messages about the work environment being team based, friendly, vibrant and it seems likely that in flagging these characteristics the employer may have been suggesting that only people who wanted to work in these types of environments should apply. conclusions this investigation aimed to identify what employers wanted in new is graduates and classify required knowledge and competency skills according to their presence in the job ads. several key findings from this study make an important contribution to understanding knowledge demanded by employers of is graduates in the australian context. the first is the identification of the major cluster of knowledge and competency skills required for early career and entry level is jobs. this cluster revolves around is development, the most frequently required category (78% of ads), associated with communication skills; personal characteristics; computer languages; data & information management; business & systems analysis; management; operations, maintenance & support; internet, intranet, web applications; and software packages. identification of the core cluster of is knowledge and skills – in demand across a wide variety of jobs – is important to better understand employers’ needs for and expectations from is graduates and the implications for education programs. this cluster however is rather broad and more detailed studies are called for to australasian journal of information systems volume 15 number 2, 2009 187 examine individual categories in greater depth. it would also be useful to monitor changes in the core cluster over time. particularly interesting is the strong presence of computer languages category in the core cluster of knowledge (in 51% of ads). while the presence computer languages in the core cluster does not identify what depth of knowledge in this area is required of graduates it none the less states that they are required. establishing the balance between technical and business knowledge and skills has been “bothering” educators since the first is programs were introduced. studies in the past criticised education programs where technical knowledge dominated and emphasized the need to introduce more business knowledge and skills (farwell et al. 1995; trauth et al 1993). finding the right balance remains the challenge for educators and employers. another important finding is the second cluster consisting of knowledge and skills from a more technical side of is: architecture and infrastructure, operating systems, networks. and security. this cluster of knowledge, while present in smaller number of ads, raises questions about the nature of entry level and early career is jobs. the knowledge categories in this cluster indicate that graduates with software engineering and computer science degrees might be better suited than is graduates. on the other hand, it is conceivable that employers might be seeking is graduates but with more emphasis on technical knowledge and skills. to better understand both knowledge clusters more in-depth, longitudinal studies of is job ads, recruitment processes and is recruits would be needed. recent studies identify is job requirements as increasingly focussing on project management and business skills and less on technical requirements. while the desire for these particular skills are articulated in interviews with employers (abraham et al. 2006; zwieg et al. 2006), they are not well represented in our ads for early career is professionals. these same recent studies identify a dilemma for academia which is to determine how early career job seekers who are sought for their technical skills can be prepared to move into midand senior-level positions. we may speculate that the value of critical thinking and the broader perspective encouraged by a university education increases in value as a career progresses. it might also be that the skill encompassed by the phrase “ability to learn” identified in ads in this study and by snope et al. (2002) and snope and underwood (2000) need to be emphasised more in education, for example an explicit acknowledgement of education as a career long process, and therefore the need for skills to enable life long learning being emphasised in the curriculum. . is is a rapidly changing field and while studies done some time ago are useful and informative, it is necessary to continually gather information about education and employment. while this study looks at one aspect, job ads, there are other dimensions for analysis which could contribute to providing a rounder picture. to throw more light on is professional careers as a whole we need more research on the skills and competencies called for in job ads for more senior positions. further work, for example, an exploration of the content of courses offered in australian universities could be undertaken. nomenclature of these courses and government classifications of is work could be analysed with a view to clarification and standardisation. investigation could also be undertaken of popular perceptions of the is field and is work. field studies, including interviews with early career is professionals and their employers on their perceptions of the usefulness or otherwise of courses would add to the picture. our study provides a snapshot picture of is jobs and what knowledge and skills employers require from the is graduates in australia. the study confirms that knowledge and skills required do not form a cohesive structure. on the contrary, even the core body of knowledge identified is not monolithic and includes diverse components with breath and depth required in both the technological nature as well as the business and social nature of is development, adoption and use. australasian journal of information systems volume 15 number 2, 2009 188 and so it seems that the challenges and dilemmas of is education are not likely to be any easier in the foreseeable future. acknowledgement the authors wish to acknowledge with gratitude the support of the john metcalfe memorial fund. references abbott, a. (1988) the system of professions: an essay on the division of expert labor. university of chicago press, chicago. abraham, t., beath, c.m., bullen, c., gallagher, k., kaiser, k. simon, judith. (2006) “it workforce trends” communications of the association for information systems vol 17, pp 1147-1170. alexander, h. (2007) “short-staffed employers boost perks” the sydney morning herald. wednesday, january 10th, 2007, p 3. association for computing machinery, association for information systems, and the computer society. (2005) computing curricula 2005: the overview report, ieee computer society , los alamitos, ca, city, 2005.from http://www.acm.org/education/curric_vols/cc2005-march06final.pdf accessed 20 february 2007 australian bureau of statistics and statistics new zealand. (2006) australian and new zealand standard classification of occupations, city australian bureau of statistics and statistics new zealand, canberra, act. http://www.ausstats.abs.gov.au/ausstats/subscriber.nsf/0/b4b626deb4a0c558ca2571e60 0092d5a/$file/12200_2006.pdf accessed10 january 2007. australian computer society professional standards board (2007) redefining and building the ict profession: core body of knowledge review discussion paper. version 2.0. http://www.acs.org.au/attachments/acs_cbok_position_paper.pdf accessed 21 june, 2007. australian government department of employment and workplace relations (2006) vacancy report (including the ict vacancy index), canberra, city dewr, 2006. http://www.workplace.gov.au/nr/rdonlyres/67d843ce-81c1-43b5-b4c70bc0b4cc173e/0/december2006vacancyreport.pdf accessed 9 january, 2007 avgerou, c. (2000) “information systems: what sort of science is it?” omega: the international journal of management science vol 28, pp 567-579. avison, d. & elliot, s. (2006) “scoping the discipline of information systems” in j.l. king (ed.) information systems: the state of the field wiley and sons, hoboken nj, pp 3-18. avison, d. & fitzgerals, g. (2002) “information systems” in j. feather and p. sturges (eds.) international encyclopaedia of information and library science, routledge, london. bassellier, g. and benbasat, i. (2004) “business competence of information technology professionals: conceptual development and influence on it-business partnerships”, mis quarterly vol 28 no 4, pp 673-694. australasian journal of information systems volume 15 number 2, 2009 189 clarke, r. (2006) “key aspects of the history of the information systems discipline in australia” australasian journal of information systems vol 14 no 1, pp 123-140 http://dl.acs.org.au/index.php/ajis/article/view/12 accessed 12 september, 2007. ezzy, d. (2002) qualitative analysis: practice and innovation, allen & unwin, crows nest, nsw. gable, g. (2006) “the information systems discipline in australian universities: a contextual framework”, australasian journal of information systems vol 14 no 1, pp 103-122. http://dl.acs.org.au/index.php/ajis/article/view/11 accessed 12 september, 2007. gallivan, m. j., truex d.p. iii, & kvasny, l. (2004) “changing patterns in it skills sets 1988-2003: a content analysis of classified advertising” the database for advances in information systems vol 35 no 3, pp 64-87. grant, peter (2007) “popularity versus leadership: the impossible challenge for australian universities”, at 18th australasian conference on information systems 5th-8th december, 2007, toowoomba, university of southern queensland. http://www.acis2007.usq.edu.au/assets/slides/peter_grant_final.pdf accessed 5 february, 2008. larson, m.s. (1977) the rise of professionalism. university of california press, berkeley. lea, v. (2006) “echoes of the tech wreck fading away”, the australian, tuesday july 18, p 33. lee, a.s. (2001) “editorial” mis quarterly vol 25 no 1, pp iii-vii. lee, c. k. (2005) “analysis of skill requirements for systems analysts in fortune 500 organizations” journal of computer information systems, vol 45 no 4, pp 84-92. lee, d. m. s., trauth e.m.,& farwell, d. (1995) “critical skills and knowledge requirements of is professionals: a joint academic/industry investigation”, mis quarterly vol 19 no 3, pp 313340. lee s.; koh s.; yen d. & tang h.-l. (2002) “perception gaps between is academics and is practitioners: an exploratory study”, information & management vol 40 no 1, pp 51-61. letch, n. & randolph, c. (2000). do the skills of non-it business graduates overlap with those of it specialists?” in americas conference on information systems: amcis 2000. amcis, long beach, ca, usa. (amcis2000 proceedings, cdrom, 2000). litecky, c. r., arnett, k. p. & prabhakar, b. (2004) “the paradox of soft skills versus technical skills in is hiring”, journal of computer information systems vol 45 no 1, pp 69-76. macdonald, k.m. (1995). the sociology of the professions. sage, london. marion, l. (2006) content analysis of information systems and information technology employment ads, unpublished paper. philadelphia city, drexel university. metcalfe, m. (2006) “using contradictions to appreciate the history of is education in south australia” australasian journal of information systems, vol 14 no 1, pp 261-272. http://dl.acs.org.au/index.php/ajis/article/view/21 accessed 1 august, 2007. neuman, l. (2006) social research methods: qualitative and quantitative approaches. 6th ed. allyn and bacon, boston paul, r. (2007) “editorial: challenge to information systems: time to change” european journal of information systems vol 16, pp 193-195 australasian journal of information systems volume 15 number 2, 2009 190 paul, r. (2002) “(is)3 : is information systems an intellectual subject?” european journal of information systems vol 11, pp 174-177. prabhakar, b., litecky, c. r. & arnett, k. (2005) “it skills in a tough job market”, communications of the acm vol 48 no 10, pp 91-94. provalis research. (2005) simstat and wordstat, software. http://www.simstat.com . snoke, r. and a. underwood (2000). “generic attributes of is graduates a comparison of australian industry and academic views”, fourth pacific asia conference on information systems, hong kong university of science and technology, hong kong. snoke, r., underwood, a., & bruce, c. (2002). an australian view of generic attributes coverage in undergraduate programs of study: an information systems case study. paper presented at the 2002 annual international conference of the higher education research and development society of australasia, edith cowan university, perth. todd, p.a., mckeen, j.d. & gallupe, r.b. (1995) “the evaluation of is job skills: a content analysis of is job advertisements from 1970-1990”, mis quarterly vol. 19 no. 1, pp 1-27. trauth, e.m., farwell, d.w. & lee, d. (1993) “the is expectation gap: industry expectations versus academic preparation”, mis quarterly vol. 17 no. 3, pp 293-307 underwood, a. (1997) core body of knowledge for information technology professionals, sydney, australian computer society http://www.acs.org.au/index.cfm?action=show&conid=200509022309270170 accessed 10 january, 2007. zwieg, p., kaiser, k. m., beath, c.m., bullen, c. gallagher, k.p., goles, t., howland, j. and simon, j.c. (2006) “the information technology workforce: trends and implications” 2005-2008 mis quarterly executive vol 5 no 2, pp 101-108. ajis vol 4 no. 1 a methodology for measuring the risk associated with a software requirements specification trevor t. moores and ralph e. m. champion department of information systems city university of hong kong kowloon tong, hong kong. abstract this paper presents a six-step metrics-based methodology for assessing the risks associated with and hence the resources required to implement the requirements contained within a software requirements specification (srs). the method seeks to eliminate the use of subjective probability assessments in models of risk exposure (re) and risk reduction leverage (rrl). measurements are taken of the number of requirements and the class of risk, the number of change requests and their date of issue, and the cost of each requirement change. the class of requirements risk is tailored to a given organisation using the delphi method. the information collected is stored as an historical database for use in the analysis of subsequent srss. introduction identifying and controlling the risk associated with the development of an information system is seen as critical for the management of a software development project (alter, 1979; mcfarlan & mckenney, 1983; charette, 1989; chittister & haimes, 1993). "risk" is typically defined as 'the possibility of incurring misfortune or loss'. in software development terms, this usually implies the potential for some disruptive event to impinge upon the project. the key problem with risk is that it cannot be avoided, it can only be anticipated and its effects reduced. for a programmer this may result in the need to redo work. for a project manager this may result in a failure to deliver on-time and on-budget. for an organisation this may result in a failure to carry out its business (tate, 1988; haimes, 1991). while a number of studies have been carried out to identify the range of factors which represent risk to a software development project (e.g., anderson & narasimhan, 1979; zmud, 1980; death, 1983; charette, 1990), the traditional use of probability or uncertainty as a measure of risk has meant that a quantitative assessment of risk exposure (re) and risk reduction leverage (rrl) relies on a subjective estimate of the associated probability based on experience. it is well known, however, that humans subjects are notoriously bad at reasoning using probabilities (kahneman & tversky, 1982). this paper will develop a six-step metrics-based methodology which allows the subjective probability ratings used in risk assessment models to be replaced by objective (empirical) measurements. the focus here will be on the verification of is development (assessing the evolution of the system) rather than on validation (assessing the is solution). since the cost of reworking software requirements is more than 100 times cheaper than the cost of making changes at the operations stage (boehm, 1981), the most cost-effective point at which to apply the risk management methodology is at the requirements stage. hence, the methodology will focus on the applicability of risk analysis at this stage. the next section will discuss the re and rrl models used to assess the potential impact of risk on a software development project. the applicability of software metrics to a software requirements document will then be discussed before providing a detailed description of the techniques required to implement such a methodology within an organisation. areas of further research will then be identified, in particular, the possibility of developing a knowledge-based front-end to the methodology. assessing and managing risk given the definition of risk as the potential for an event to occur, the control of risk is a two-step process involving: the assessment of which risk factors are likely to be important; and, the management of risk by measuring and prioritising the probability of the event and it's impact on the project. according to kaplan & garrick (1981), risk assessment asks three questions for each phase of the development process: 1. what can go wrong? 2. what is the likelihood that it will go wrong? 3. what would the consequences be? these studies have typically been carried out by asking project managers or users to rate (an existing list of factors) or specify (where no prior model is in place) the importance of a list of potential risks. factor analysis allows the subsequent list to be condensed and grouped under general headings. for instance, boehm (1989, 1991) identifies four major headings of systems failure which set out what can go wrong, namely: hardware problems, software problems, organisational problems, and human problems. 55 ajis vol 4 no. 1 boehm suggests that the full list of problems (see table 1) can be used as a checklist by a project manager to identify the most likely risks. boehm defines risk exposure (re) and risk reduction leverage (rrl) as the key measures, where: re = prob(uo) * loss(uo) rrl = (rebefore-reafterxrisk reduction cost table 1. the 10 most important risk factors by category (boehm, 1989, 1991) prob(uo) loss(uo) category risk factor people resources requirements external technology 1. personnel shortfalls 2. unrealistic schedule and budgets 3. developing the wrong software functions 4. developing the wrong user interface 5. gold-plating 6. continuing stream of requirements changes 7. shortfalls in externally developed software 8. shortfalls in external personnel 9. shortfalls in real-time performance 10. straining at the bounds figure 1. risk exposure (re) contours 0.9 0.8 • • 0.7 0.6 • 0.5 • 0.4 0.3 -. 0.2 0.1 -0 0 • re=0.5c —i 1 1 1 1 1 1 1 1 1 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 risk exposure is the product of the probability of some unsatisfactory outcome, prob(uo), and the loss due to this outcome incurred by the project, loss(uo). the project manager would need to calculate re for each identified risk and decide a level of acceptable risk exposure. for factors which exceed the acceptable level of re, the project manager would then have to either reduce the probability of the event occurring, or reduce the loss associated with it (see figure 1). for instance (from boehm, 1989, p8), if the loss associated with having a particular type of interface error is estimated at us$1 million and from experience we estimate that the probability of such a fault on a scale of 0 to 1.00 is 0.30, then the risk exposure is given as: re = (0.30)*us$1000k uss300k 56 ajis vol 4 no. 1 an approach to dealing with this problem could entail buying a requirements and design interface checker for us$20k which would reduce the prob(uo) to 0.1, or, carrying out a detailed interface test for an additional us$150k of manpower which would reduce the prob(uo) to 0.05. the rrl of these two approaches (denoted as rrli and rrl2> respectively) can be calculated as follows: rrli = ([0.30]*us$1000k-[0.10]*us$1000k)/us$20k = 10 rrl2 = ([0.30]*us$1000k-[0.05]*us$1000k)/us$150k = 1.67 therefore, the interface checker approach can be seen to deliver the highest risk reduction. although boehm's approach may seem reasonable, care must be taken in interpreting these figures. firstly, although the models of re and rrl appear to be objective they are not, since although the financial impact of the risk reduction may be accurately calculated, the crucial assessment of the probability of risk is subjectively derived. in other words, the inputs to these models are best guesses by the project manager and so, the risk associated with the guess may in itself account for the difference in result between rrl] and rrl2furthermore, cheap alternatives to the above two approaches which do little to reduce the risk can appear to provide better rrl scores. for instance, employing a us$500-a-day consultant for one day may not reduce the risk a great deal, say the prob(uo) becomes 0.29, but the rrl score, rrl3, calculates as follows: rrl3 = ([0.30]*us$1000k-[0.29]*us$1000k)/us$0.50k = 20 neither can it be assumed that the consultant can reduce the risk exposure by 0.01 every day, since most errors are typically found early in the testing phase. given that rrl is simply a ratio, it is unlikely that the project manager will be happy with option 3, since the risk to the project remains at 0.29 and it is only the cheapness of the risk reduction cost that suggests option 3 is a better solution than options 1 or 2. the problem with re and rrl is the use of "probability" in the model. although risk is sometimes defined in terms of "uncertainty" rather than probabilities (e.g., barki et al, 1993), the fact remains that subjective ratings are inadequate measures of risk. principally, the calculation of a probability is based on an assessment of the number of choices available within an event. the difference between probabilistic and empirical ratings can be illustrated with the following example: suppose a coin is flipped ten times and it came up heads nine times and tails once. according to probability, the chance of the next flip producing a head, p(h)prob, remains 0.5 since it must be one of two equally likely outcomes; however, the empirical result suggests that p(h)emp=0.9 since a head has been observed on nine of ten trials. over a long period of time, it would be expected that p(h)prob^p(h)emp> but this could only be derived from empirical observation. in other words, p(h)emp is a surrogate for p(h)prob when the theoretical basis for calculating p(h)prob is not available. software development is sufficiently complex for it to be assumed that p(h)prob is not available and any models making use of probabilities such as re and rrl must be revised. rather than the use of probabilities, therefore, there is a need to apply objective measurements of the risks involved. metrics-based risk assessment a software metric is a measurement of some attribute of the software process or products that relates directly or indirectly to some property of the system that needs to be assessed or predicted. for instance, metrics have been developed to measure the attribute "size" of a software system in order to assess properties such as the effort required to implement the system (e.g., halstead, 1977; albrecht & gaffney, 1983), and predict properties such as the cost of the project as a whole (e.g., putnam, 1978; freiman & park, 1979; boehm, 1981). the general methodology for collecting metrics in order to quantify an attribute of the software development process is a six-step process, as follows: 1. identify which attribute needs to be assessed (or predicted). 2. identify those properties which are to be used as a measure of the attribute identified in t. 3. collect the measures defined in '2' until a database of sufficient size has been created. 4. quantify the relationship between the properties ' 1' and '2'. 5. apply the model on the metrics database to test its accuracy. 57 ajis vol 4 no. 1 6. if the model works sufficiently well, use the model to estimate t on a live project; otherwise, start again at step '2'. this sequence of steps sets up a project-hnetrics-hnodel->tool cycle (see figure 2), where the tool is evaluated against real world projects, and the data collected (now including the estimate) is used to update and refine the models. figure 2. the project-»metrics-»model-»tool cycle real world project 1. 2. 3. metrics database t 6. model-based tool 5. derived metrics model defining a set of measurements steps t and '2' are the key, since only with a sensible definition of the attribute to be assessed and the measurements required can useful metrics models be derived. given that software requirements specification (srss) are more structured than the initial user-defined requirements document, the methodology will be applied to the srs rather than the user document. the application of software metrics requires both a purpose and an intuitive link between the attribute to be measured and the metric used to measure the effect of the attribute (basili & rombach, 1988; fenton, 1994). in this case, the attribute to be measured is risk; the purpose is to quantify models of re and rrl; while, the property which is meant to have an intuitive link to risk is the changeability of the requirements and the cost of change. while the model of rrl remains the same, the model of re becomes: re= c°stchange in other words, the probabilistic prob(uo) factor is replaced by a quantifiable measure of the risk associated with a change to a requirement, riskchange> an^ tne cost needed to accommodate this change, costchangeboth riskchange an^ costchange can be quantified if the changeability of any given requirement and the cost of the change can be measured. 58 ajis vol 4 no. 1 figure 3. a representation of the requirements change process user change request stated requirements (in the srs document) developers 1 1 ^ ••̂ •̂ » f 1 amendment to implementation implemented •• requirement (in the operational system) the measurements defined here are based on the model given in figure 3, which puts a boundary around the process under study. as can be seen, the implementation of a requirement is represented as a transformation from its written statement in the software requirements specification (srs), to its implementation in the operational system. two parties are involved in this process: the users who decides on the requirements; and, the developers who implement each of the required functions. a change to a requirement is initiated by a change request from the user and in response, the developers make some change to its implementation. it should be noted that the change may include cancellation, and the cost of the change in this case would be the effort already expended. in order to quantify this process, the measurements required are as follows: • the number of requirements (actual number and category type); • the number of change requests (actual number with date of issue); • the cost of each change request (in the relevant currency, e.g., us$). counting the actual number of requirements depends principally on the way in which the requirements document is written. on the assumption that "simplest is best", the suggestion here is to take high-level counts of the number of requirements, such as the number of functions defined within a form-based approach (heninger, 1980). the ability to derive useful categories of requirements risk is clearly crucial. boehm (1989) derived his five categories (see again table 1) based simply on experience of developing large-scale software systems. the approach here, however, is to derive the categories more rigorously using the delphi method (delbecq et al, 1975). the delphi method is often used to generate agreement on such things as mis issues seen as being important for is executives (e.g., see niederman et al, 1991; wang, 1994). the objective of the technique is to discuss drafts of a list of issues in order to surface new issues and move participants towards a consensus. starting with a base-line questionnaire, participants are asked to rate the issues, typically on a scale of 1 to 10 with 1 being the lowest and 10 being the highest. succeeding questionnaires summarise group responses and the participants are asked to re-evaluate their opinions based on the new evidence. a three-round delphi method is common and stops after three rounds of questionnaire, although it is possible for the procedure to continue until a reasonable level of consensus is reached. 59 ajis vol 4 no. 1 table 2. reasons cited for projects failing to meet user requirements reason no. of responses poor/conflicting user participation 14 lack of understanding 12 too changeable 12 too speculative 9 insufficient time for analysis 7 uncontrolled changes 6 no agreement made 6 specification incomplete 6 inadequate testing 2 poor development standards 2 no formal review/acceptance 1 lack of it/business awareness 1 inadequate performance 1 as an example, in a survey of is managers in large uk corporations and computing companies (moores, 1993), one of the questions asked was:"what are the major reasons for projects failing to meet user requirements?" a ranked list of responses is given in table 2. clearly, this particular list would need further analysis since it contains both symptoms and causes of requirements risk. the symptoms listed include a general lack of understanding, requirements being too changeable and too speculative, while the potential causes listed include poor/conflicting user participation, insufficient time for analysis, uncontrolled changes and inadequate testing. this list can be used as the baseline delphi questionnaire, however. subsequent questionnaires are then phrased so as to focus participants towards specific causes rather than symptoms of requirements risk. the list can be tailored for a particular organisation by asking project managers to rate each of the reasons given in the baseline questionnaire or suggest their own additional reasons. once consensus has been reached, factor analysis based on the rating scores can help to identify the underlying concepts being used. the final list is then the range of categories which are to be used to classify requirements risks. for instance, data would be collected on the number of requirements changes that were identified as being due to insufficient time for analysis. the advantage of this approach over other studies is that it takes into account the idiosyncrasies of the particular organisation, a factor seen as important in developing some types of local metrics models (kitchenham, 1992). counting the number of change requests and the cost of each change depends on their being an adequate change management process in place. the count of change requests may be taken to be the number of change request forms (crfs) which were actioned, that is, actions were initiated in response to the request. crfs tend to be simple forms (e.g., see sommerville, 1992, pp555-8) documenting problems within the system. by only counting those crfs which were actioned, crfs that report the same or trivial problems can be ignored. before a crf is actioned, the change manager needs to analyse the cost of the change and it's impact on the system (pressman, 1992). within this process the cost of the change is immediately available as the cost of the rework scheduled to deal with the change request. since change requests are possible at any point during the development process, and as boehm (1981) points out the cost of any change increases down the life-cycle, both riskchange and costchange && likely to have different values depending on when the change occurs. by recording the date at which the change request is received, this issue can be taken into account building the risk models the final steps of the metrics-based methodology (steps '3' to '6') are carried out by using the measures defined above to quantify the following relationships: reqx = the total number of requirements assigned to risk category x changex = the total number of reqx that required a change costx = the total cost of changex riskchange = changex/reqx costchange = costx/changex 60 ajis vol 4 no. 1 figure 4. calculating riskchange for four categories of requirements risk requirement risk category project no. x x x x x (d x x x 0 x x x x = requirement assigned to risk category x (x) = requirement that changed with measurements taken across a number of projects, the risk exposure can be calculated for each category of requirements risk. for example, consider that measurements are taken across three projects using four broad categories of requirements risk (see figure 4). with five instances of requirements placed in category 1 and only one instance of a change, riskchange is calculated as 0.20. similarly, riskchange f°r category 3 is calculated as 0.66, while categories 2 and 4 have no recorded changes and so riskchange=0> costchange=0 and re=0. in other words, the perceived risk has not been found to entail any actual cost. costchange f°r categories 1 and 3 is the average value of changexy and this completes the re calculation. each category of risk would be derived using the method described above. the accuracy of a metrics model (step '5') is typically assessed as a function of both the general predictive ability of the model, pred, and the size of error attributable to each estimate, mmre (conte et al, 1986). a metrics model is said to be accurate if the estimate is within 25% of the actual 75% of the time, given as pred(0.25)>0.75, and the mean magnitude of the relative error is less than 0.25, given as mmre<0.25, where, mmre = -.y ( i actual j estimate ; i actual and, actuali = actual value for the i"1 data point estimatej = estimated value for the im data point this definition has been taken on board and widely used although the value of ±25% is clearly an arbitrary threshold. for instance, without formally defining pred as a relationship, boehm (1981) took it that a model could be deemed to be accurate if it produced estimates within 20% of the actual 70% of the time, that is, pred(0.20)>0.70, while other studies have required a higher level of pred at pred(0.20)>0.80 (moores & edwards, 1992). given the accuracy of risk models have not been properly assessed to-date a weaker level of accuracy is set initially, using pred(0.25) >0.70 and mmre<0.25. as more data becomes available and the model is fine-tuned, the accuracy of the models will increase over time. 61 ajis vol 4 no. 1 having completed the above steps, the risk models derived using this methodology can now be employed on live projects. the person responsible for reviewing the srs first decides which requirements are subject to some level of risk. for those requirements where a risk category applies, the risk exposure is calculated. for instance, if a requirement dealing with checking valid data entry is known to have been specified without much discussion with the user, then the requirement might bring to the project a risk under the heading of poor/conflicting user participation. if this category of risk is recorded as effecting changes in l-in-4 projects with an average cost of us$1 ok, then the re can be calculated as: re = riskchange*costchange 0.25*us$10k uss2.5k an srs with twenty similar requirements would result in re=us$50k. it is at this point that either the requirements would have to be investigated further so as to eliminate the risk, or instigate strategies of risk management (using rrl) in order to reduce the level of re to one acceptable to the project manager. conclusion this paper has sought to replace the probabilistic models of risk exposure and risk reduction leverage (re and rrl, respectively) with a 6-step metrics-based methodology which quantifies the risk of change and cost of change to requirements (riskchange and costcriange* respectively). the metrics required to quantify risk and the techniques required for their collection have been described in detail. the key to the methodology is that subjective ratings of the probability of a particular risk impacting on a project is replaced by empirically derived weights. these weights accurately reflect the number of times a class of risk has actually affected a project. the classes of requirements risk are derived for a given organisation using the delphi method. a further advantage of the metrics database is that data would be provided on the most common risks. such information would enable organisations to investigate reasons for their occurrence and remedies for the problem. as with most research, a number of issues require further investigation. firstly, the methodology described above suggests the re for a project is the sum of the re for each risk. clearly, there will be a certain amount of interdependence between risk factors. for example, when considering general project risk, boehm (1989) has noted that personnel shortfalls and unrealistic schedule and budgets are two of the most important risk factors (see again table 1). however, any calculation of schedules and budgets would take into account the efficiency of the development team, and it is only by knowing that they are of below average quality that the schedules and budgets can be judged to be "unrealistic." care must be exercised in using the delphi method and factor analysis to derive a meaningful set of risk categories, therefore, since it is this list which drives the collection of data and allows the appropriate riskchange and costchange to be assigned. finally, although the metrics-based methodology has removed one potential flaw in the use of risk models the use of subjective risk probability assessments there is still an element of skill involved in risk assessment and management. primarily, it is still up to the project manager to recognise that a requirement contains a certain class of risk. furthermore, even if the re is established there will almost certainly be a major difference in how changes to requirements are dealt with. one project manager may postpone delivery of some functionality in order to meet deadlines, whilst another may negotiate extra funding and deliver the system the customer needs. clearly, skill in terms of negotiation has an impact here. the use of skill opens up the possibility of developing a knowledge-based front-end to the methodology described in this paper. such a front-end would help to further remove subjectivity from the assessment or risk by making operational the heuristics in use. the metrics database would provide the information upon which to test the veracity of the heuristics, while the heuristics would help to clarify the data needed to assess and manage risk. the authors are currently attempting to implement such a research programme. references albrecht, a. j. and gaffney jr., j. e. (1983) software function, source lines of code, and development effort prediction: a software science validation. ieee transactions on software engineering, vol 9, no 6, pp 639-648. alter, s. (1979) implementation risk analysis. tims studies in management sciences, vol 13, no 2, pp 103119. anderson, j. and narasimhan, r. (1979) assessing implementation risk: a methodological approach. management science, vol 25, no 6, pp 512-521. 62 ajis vol 4 no. 1 barki, h., rivard, s. and talbot, j. (1993) toward an assessment of software development risk. journal of management information systems, vol 10, no 2, pp 203-225. basili, v. r. and rombach, h. d. (1988) the tame project: towards improvement-oriented environments. ieee transactions on software engineering, vol 14, no 6, pp 758-773. beath, c. m. (1983) strategies for managing mis: a transaction cost approach. in proceedings, 4th acm international conference on information systems, houston, usa, december, 1983, pp 415-427. boehm, b. w. (1981) software engineering economics. prentice-hall, englewood cliffs, nj. boehm, b. w. (1989) tutorial: software risk management. ieee computer society press, los alamitos, ca. boehm, b. w. (1991) software risk management: principles and practice. ieee software, vol 8, no 1, pp 32-41. charette, r. n. (1989) software engineering risk analysis and management. mcgraw-hill, new york, ny. charette, r. n. (1990) applications strategies for risk analysis. mcgraw-hill, new york, ny. chittister, c. and haimes, y. y. (1993) risk associated with software development: a holistic framework for assessment and management. ieee transactions on systems, man, and cybernetics, vol 23, no 3, pp 710-723. conte, s. d., dunsmore, h. e. and shen, v. y. (1986) software engineering metrics and models. the benjamin/cummings publishing co., inc., menlo park, ca. delbecq, a. l., van de ven, a. h. and gustafson, d. h. (1975) group techniques for program planning: a guide to nominal group and delphi processes. scott-foresman, glenview, il. eysenck, m. w. (1984) a handbook of cognitive psychology. lawrence erlbaum associates, hillsdale, nj. fenton, n. (1994) software measurement: a necessary scientific basis. ieee transactions on software engineering, vol 20, no 3, pp 199-206. freiman, f. r. and park, r. e. (1979) price software model version 3. an overview. in proceedings, ieeepliny workshop on quantatative software models, pp 32-41. haimes, y. y. (1991) total risk management. risk analysis, vol 11, no 2, pp 169-171. halstead, m. h. (1977) elements in software science. elsevier north-holland, new york, ny. heninger, k. l. (1980) specifying software requirements for complex systems: new techniques and their applications. ieee transactions on software engineering, vol 6, no 1, pp 2-13. humphreys, p., svenson, o. and v&i, a. (1983) analysing and aiding decision processes (eds.). north-holland publishing co., amsterdam. kahneman, d. and tversky, a. (1982) judgement under uncertainty: heuristics and biases. cambridge university press, cambridge. kaplan, s. and garrick, j. b. (1981) on the quantitative definition of risk. risk analysis, vol 1, no 1, pp 1127. kitchenham, b. a. (1992) empirical studies of assumptions that underlie software cost-estimation models. information and software technology, vol 34, no 4, pp 211-218. mcfarlan, f. w. and mckenny, j. l. (1983) corporate information systems management: the issues facing senior management. irwin, homewood, il. moores, t. t. and edwards, j. s. (1992) could large uk corporations and computing companies use software cost estimating tools? a survey. the european journal of information systems, vol 1, no 5, pp 311-319. moores, t. t. (1993) metrics and models to support the development of hybrid information systems. phd thesis, department of operations and information management, aston business school, university of aston in birmingham, uk. niederman, f., brancheau, j. c. and wetherbe, j. c. (1991) information systems management issues for the 1990s. mis quarterly, vol 15, no 4, pp 475-496. pressman, r. s. (1992) software engineering: a practitioners approach (3rd ed.). mcgraw-hill, new york, ny. putnam, l. h. (1978) a general empirical solution to the macro software and estimating problem. ieee transactions on software engineering, vol 4, no 4, pp 345-361. sommerville, i. (1992) software engineering (4th ed.). addison-wesley, wokingham. tate, p. (1988) risk! the third factor. datamation, 15 april, pp 58-64. wang, p. (1994) information systems management issues in the republic of china for the 1990s. information & management, vol 26, pp 341-352. zmud, r. w. (1980) management of large scale software development efforts. mis quarterly, vol 4, no 2, pp 45-55. 63 a.iis vnl. 3, no.2 software quality certification: identifying the real obstacles megan baker department of business management, faculty of business and economics anne rouse department of information systems, faculty of computing and information technology monash university, 900 dandenong road, caulfield east vic 3145, australia abstract a case study of software certification reveals the real difficulty of certifying quality beyond superficial assessment readers are invited to form their own conclusions. as 3563 software quality management system is the australian version of iso 9001, developed specifically for the software industry. for many australian software houses, gaining certification with as 3563 is a priority since certification has become a prerequisite to doing business with government departments and major corporations. however, the process of achieving registration with this standard is a lengthy and resource intensive process, and may have little impact on actual software quality. this case study recounts the experience of the consulting arm of one of australia's accounting firms in its quest for certification. by using a number of specific management strategies this company was able to successfully implement as 3563 in less than half the time usually taken to achieve certification a feat for which its management should be congratulated. however, because the focus of the project was on gaining certification, few internal benefits have been realised despite the successful implementation of the standard. introduction background since 1 january 1994, australian government departments and agencies have been entitled to ask information technology suppliers to provide evidence that their organisations have in place quality management systems. in 1988 the standards association of australia released for publication australian standard 3563 software quality management system (as 3563), which was specifically designed to cater for the software industry in australia. more recently, the 1991 release of as 3563 included guidelines that give practical examples of the implementation of a quality management system. as 3563 is based on iso 9001 quality system model for quality assurance and design/development, production, installation and servicing. to achieve certification with as 3563, organisations are evaluated in light of the standard's requirements, their own documented procedures, and documented evidence proving adherence to these procedures. examples of the areas which are addressed by as 3563 include contract review; design control; document control; purchasing; product identification and traceability; inspection and testing; storage, packaging and delivery; training; servicing; and control of the development environment. the government edict of 1992 galvanised the australian information technology industry into action. for many organisations, gaining the 'quality stamp of approval' has become a major priority because of the associated market implications (zampetakis, 1994). this process is usually costly and time consuming. yet by using effective implementation strategies, it is possible to reduce the timeframe quite considerably. this paper examines the implementation experience of one of australia's accounting firms which obtained certification within a relatively short timeframe. the first part of the paper reviews two particular problems encountered, and the management techniques used to overcome these difficulties. this section is followed by a discussion on the management strategies used to actively foster the rapid procurement of as 3563. the final section discusses the major challenge currently faced by the organisation maintaining on-going commitment to the quality system, despite the apparent weaknesses inherent in the standard. the paper concludes by summarising the strategies which helped the organisation achieve certification, and by making recommendations for the future evolution of as 3563. the certification experience the company on which this study was based was selected as an appropriate organisation for research because of its position as one of the leaders of the information technology industry. its success in gaining certification within the short timeframe was also an important factor. the organisation has branches in all capital cities in australia. the division within the office in which this study was based consists of approximately 180 employees. interviews were carried out with the quality manager responsible for the overall certification effort, and with a senior consultant who was involved with and affected by the certification effort. interviews were ajis vol. 3, no. 2 also held with a former manager who had been heavily involved in establishing certification, and with a representative of standards australia. questions were developed based on a model developed by du pont de nemours and company (hockman, grenville & jackson, 1994) for registration with iso 9000. interview conclusions were submitted to all interviewees, and feedback invited, to ensure validity. the decision to gain certification at the company was made when top management realised its potential for competitive advantage. consequently considerable resources were then invested in appointing a full time quality manager, hiring an external quality systems consultant, and providing training for all affected staff. internal audits were carried out to analyse existing routine work practices and written procedures. an implementation plan was created to develop new work practices where the analysis revealed the potential for improved work flows. written procedures were either created or updated to reflect these changes. individual project leaders developed quality plans for individual projects; the quality manager developed a quality manual which described the management, structure, policies and principles by which the organisation manages quality. a certifying organisation was chosen. even before the days of pre-assessment and registration, management reviews were carried out to determine the status of quality related projects, internal audit results, and adherence to quality plans to evaluate the effectiveness of the new system. further finetuning took place after the day of pre-assessment, when the organisation underwent a 'dry-run' investigation by external auditors. on the two days of certification assessment, the organisation was visited by a team of auditors from standards australia, which rigorously investigated the software quality management system in light of the requirements of the standard. some minor non-conformances were reported, but the recommendation was made that registration be allowed to proceed on the condition that a plan be developed to correct these discrepancies. the organisation gained as 3563 registration with standards australia in august 1993, and recertification with the national association of testing authorities (nata) in april 1994. the initial certification with as 3563 took just less than eight months. the average time to gain certification is eighteen months (aha, 1993). although the organisation has achieved certification, it cannot afford to rest. to maintain certification, the organisation must undergo compliance audits at regular intervals. since 10% -12% of organisations fail the compliance audits these can by no means be regarded as a fait accompli. implementation issues challenges faced to achieve certification within eight months, the organisation had to overcome two main challenges. the chief of these related to generating staff commitment to the software quality management system. the audits held on the days of pre-assessment and certification were the other major hurdle. gaining staff commitment gaining staff commitment to as 3563 was essential to the success of the process. this was a particular challenge, according to the quality manager, because of the staffs perception of the numerous weaknesses of the standard itself. these will be discussed later in the paper. yet despite the perceived weaknesses of the standard, the organisation did gain considerable commitment from both its staff and management to the certification process. senior management recognised the immediate competitive advantage, and the ongoing competitive necessity, to be gained by certification. tremendous amounts of energy were thus channelled into gaining certification. the quality manager was given access to the necessary resources to help the organisation gain certification. it was also communicated very strongly to staff that gaining certification was immensely important for the ongoing well-being of the organisation. this is reflected in the words of one interviewee who said "we recognised the need to gain certification, and were working hell for leather trying to get it". because of this strongly communicated message, staff worked very hard at meeting the established goals, even though many of the processes seemed irrelevant. managers and project leaders also contributed to the certification process by releasing their staff to attend training sessions. pre-assessment and certification careful management of the third party audits carried out on the days of pre-assessment and certification was also critically important. in order to audit a company's procedures within a few hours over a couple of days, the registers of the certifying body require the organisation to be able to lay its hand on relevant documentation almost instantaneously (effective filing and document management are part of the quality system). this is a particularly intensive activity, not only for the quality manager, but for all those responsible for maintaining ajis vol. 3, no. 2 procedural documentation. requests for proof of documentation and adherence to the standard are made at a rapid pace. at the organisation, the requests of the certifying body seemed, at times, to take on the form of a spanish inquisition, putting those at the receiving end under considerable stress. it also put staff in a defensive position from which it was very difficult to recover. the quality manager found the best way to handle this situation was to become pro-active. after the experience of the pre-assessment audit, he determined that for the actual certification audit he would take the initiative from the very first. by using this tactic he was able to control, to some extent, the direction of the audit. while the assessors still had set criteria to cover, controlling the agenda in this manner removed some of the stress from the review process. this approach was commented on favourably by the certifying body since the quality manager's fifteen minute presentation saved them considerable questioning time. another tactic used to ease the tension was having refreshments available to create a relaxed environment in which the organisation's staff could interrelate on a more personal level with the external auditors. there was also one occasion when the timely aid of a senior manager helped to negotiate the situation when the inspection had begun to resemble an inquisition. management strategies to facilitate certification gaining staff commitment through management support and a pro-active attitude were two strategies which contributed greatly to the rapid attainment of certification. other factors which were essential to the successful implementation of the quality system were the efforts of a dedicated champion and an experienced mentor; communication and training; and the use of a pilot scheme to demonstrate the quality system to the wider organisation. the use of a champion within the information technology field, a champion a person who will battle great odds in order to see his or her vision in place is often crucial to the success of an individual project. this person has a concept of the end result, and gets it implemented by obtaining funding, pushing the project over hurdles, putting his or her reputation on the line, and by generally taking on the risk of the project. at the organisation, the role of champion was assumed by the quality manager, who took on the responsibility of organising resources, communicating progress, disseminating information, and generally working on generating commitment to the certification process. up to gaining certification, this was a full time job one that frequently went beyond standard working hours, and as part of the national role, involved flying to other major cities to advise and assist quality managers from other offices. this role was particularly stressful, with deadlines constantly approaching and major hurdles having to be jumped repeatedly. despite these difficulties, the quality manager's dedication and strength of commitment were key elements in persuading employees that certification, with all its bureaucratic overtones, was indeed a worthwhile goal. one of the key techniques employed by the quality manager in his role of champion was the use of checklists and reports which provided evidence to top management of the constant progress being made towards certification, and later, enabled the organisation to pass the on-going compliance audits. after each audit, the quality manager prepared a report of activities to be completed for the next audit (including a description of the problem, the necessary action, the person to take corrective action, and a time frame) and personally followed these up. summaries of activities completed were circulated thoughout the organisation. this reporting activity served to keep the certification process in the forefront of the minds of management and staff. the use of a mentor another individual who was a key player in the successful achievement of certification was the quality systems consultant, who took on the role of mentor. the consultant had many years of experience with the implementation of standards, and so the organisation was able to capitalise on this knowledge to effectively prepare itself for the assessment days. when the consultant was first employed, he brought with him a particularly comprehensive checklist which related to the as 3563 requirements. by working through this set of questions, and ensuring that a positive response (along with an explanation) could be provided for each, the organisation was able to judge its preparedness for the external auditors. the consultant's prior experience ensured that the organisation knew the areas on which the auditing parties were likely to concentrate. a.iis vol.3, no. 2 communication and training open communication was another key element in the successful attainment of certification. the organisation already had in place a number of communication channels, since within a consulting firm, employees rely heavily on the experiences of other staff members. information on as 3563 was transmitted through face-toface meetings, the telephone and electronic mail. information was also disseminated in training courses. in addition, at the fortnightly managers' meeting which all senior consultants attend, the quality manager provided an update on initiatives and progress in regard to the quality movement. an electronic lotus notes database, accessible by all staff, was established to contain all the quality manuals and standards. the use of a pilot project one of the most effective techniques employed by the organisation to demonstrate the certification process to all staff was the use of a pilot project. a small information technology project was chosen shortly after a relevant set of methodologies had been developed. the project leader applied the methodologies to the project, established the relevant controls and produced a quality plan (all part of the requirements of as 3563). as the project itself was relatively undemanding, it enabled the project team to concentrate on carrying out the methodology to the very letter. external auditors were even brought in so that the project team could fully experience the entire standard compliance environment. at the completion of the project the project leader shared his experience with the rest of the organisation, describing the additional activities imposed by as 3563, the ramifications of doing the job with quality procedures in place, and the types of questions that could be expected from the auditors. as a learning exercise and demonstration for the rest of the firm this model project was highly successful. one interviewee reported that the pilot project significantly changed "...people's knowledge of what was involved, and what they had to do." future challenges for the organisation gaining certification is a necessary part of the as 3563 process. from this point, however, the critical issues are maintaining certification, and embedding the quality system into the everyday activities of the organisation so that maximum benefit can be gained from the investment. maintaining commitment at the organisation, while top management support was clearly evidenced by the amount of time and resources that were channelled to obtaining certification, it appears the resources were all pointed to gaining certification, and not necessarily to the underlying quality management system. while senior management "...wanted certification [and] so would do anything to make it happen", it was also pointed out that: "...the commitment [was] to getting standards national around australia". with regard to momentum, once certification was achieved "...well, it dropped off as far as top management was concerned...it was up to the quality manager to really keep driving management." it appears that once certification was achieved, many staff heaved a collective sigh of relief a fairly common reaction after the stresses of the certification process. sakovsky (1994) reports that top managers tend to believe that with certification gained, all of the work for the organization is completed; with the major hurdle passed, there is an opportunity to relax. one specific example of this at the organisation is the number of people being released to attend training courses. the quality manager reports that before certification, quality system courses would be fully booked out; once certification was gained, it became difficult to persuade managers to spare staff to attend these courses. "the partners are reluctant to send them here because it is costing them money, i mean, losing charge-out time, and the cost of the air fares for interstate staff. but they think, now that we have certification, why do we have to spend so much time and money on it?". since gaining certification, steps have been taken to counteract this attitude towards certification by incorporating the issue of quality into the annual planning process. the quality audit program and the summaries of the audits are now part of the agenda of national information technology quality board meetings. thus it appears that while the necessity of certification was recognised as a major competitive tool because of the implications for market position, the underlying issue of quality (the raison d'etre for the initial development of as 3563) was seen, for a while, to be an issue quite separate from that of certification. once certification was gained, commitment to maintaining the same level of activity in regard to as 3563 appeared to a.iis vol. .v no. 2 decrease quite rapidly before executive managers took action to place the quality issue, quite literally, back on the agenda. in australia, there is a growing recognition that to achieve high quality information systems, it is necessary to go beyond the quality assurance process (rouse & watson, 1994). the organisation has also recognised the necessity of looking beyond the requirements of as 3563 to ensure quality is part of its overall culture. obstacles in the path of on-going commitment the difficulty of generating on-going commitment to the activities required by as 3563 certification may be related to the fact that apart from the marketing advantage to be gained through certification, few benefits appear to have been realised. certification can supposedly assist in terms of software quality, and improvement in the use of software development resources through more effective and efficient work practices. an evaluation of the benefits to be gained from certification was not the focus of this research. nonetheless, some inferences will be made in the following section as to why the organisation did not achieve the benefits that are often touted in relation to certification. weaknesses of as 3563 in many ways, the standard itself was not seen to be relevant to the needs of the organisation. the following weaknesses were referred to repeatedly throughout the interviews conducted. firstly, as 3563 was designed for large projects which use traditional systems development life cycle methodologies. in many ways as 3563 is not suited to the newer development methodologies such as clientserver, fast-tracking and rapid application development (thomsett, 1993). these form some 70% of the organisation's development work. these methodologies are particularly results oriented, and concerned with getting prototypes in front of the customer so rapid feedback can be gained. they are designed so that short cuts can be taken and new development routes taken as customer requirements demand. standards, on the other hand, impose control and structure; they actually restrict developers to written company work practices secondly, the organisation also offers services in the areas of software package evaluations, outsourcing, software maintenance, and the development of strategic plans, all of which involve little development work. even though these services do not relate specifically to software development, because the department had gained certification, all its work process, rather than just the software development jobs, had to be carried out in line with the standard. thirdly, the internal audit process was viewed as onerous since considerable effort was required to maintain the auditing level. the non-conformances recorded were often regarded as trivial bureaucratic problems, with the associated follow up work being difficult to justify. examples of non-conformances which the organisation was asked to rectify by the certifying body included out-of-date lists of staff members who had attended training sessions, and incomplete records relating to work practices such as document control (smic-batanic, 1994). in many cases, quality work was carried out as "catch up" work, rather than being part of day-to-day work, thus adding to project overheads. fourthly, the preparation of quality plans for individual projects as a major overhead. this preparation was very time consuming, taking up a considerable portion of the resources allocated to projects, with the benefits being difficult to quantify. particularly for smaller projects, the associated administrative requirements were often seen as out of proportion in relation to the actual development work being performed. there was also the perception among staff that the process of certification was not directly related to the quality of the product being presented to clients. at no stage were the audits, internal or external, related to the organisation's products or services provided to its customers. thus the process of certification appeared to have little to do with the ultimate aim of as 3563 an improved software product. another problem was that the majority of procedures and methodologies had already been tried, proven and documented for many years prior to the current push for certification. in many cases, the specifications of as 3563 certification simply required staff to prove that they were already following existing procedures. the final issue is that certification is extremely costly and resource intensive. it is debatable whether the few process improvements identified warranted the costs of a full time quality manager, a consultant, support personnel, registration fees, training courses, travelling costs, and the loss of chargeable hours of those attending the courses. a.iis vol. 3, no. 2 getting the most from certification to date at the organisation, the audits, both internal and external, have related strictly to processes. to obtain long term benefits, management may therefore need to apply the auditing process not only to those procedures which relate directly to as 3563, but to extend it to those issues which are directly related to business health.. these include issues such as customer levels of satisfaction, pricing, market penetration, and product and service quality. this will increase the relevance of as 3563, and in turn generate commitment to the ongoing software quality management system. the audit results can then be used by management to develop meaningful business corrective action. this results-oriented approach was adopted by australian organisations studied by von hellens (1994). having found that the software quality management system was offering few tangible results, management within one organisation decided that "...the focal point of the quality management system implementation should move from primary software development activities to the more intangible tasks of product or service delivery" (von hellens 1994, p 485). another organisation explored new ways of describing the intangible nature of quality to make it more tangible and better appreciated by staff. this was achieved by involving development staff with service tasks so that they would become more aware of the relationship between customer needs and the quality process. quality became something "...the development team could incorporate in their working practices" (von hellens, 1994, p 485). as 3563 audits would then be treated as an improvement tool to be used not only to gain compliance with a standard, but also to improve business functions and business problems. the system would work to improve the business, and not just to satisfy as 3563 requirements. conclusion what are the lessons that can be learnt from the this organisation's experience? firstly, it is possible to achieve certification in a timeframe much less than the average of eighteen months. this is possible through focused management commitment; a dedicated individual championing and directing the certification process; the use of a knowledgeable consultant; open and pro-active communication on the subject of certification; training; and a pilot project to demonstrate the certification process to the wider organisation. secondly, the rapid procurement of as 3563 may come at a cost. a timeframe of eight months, while it allowed sufficient time for the organisation to demonstrate adherence to the requirements of the standard and gain 'the quality stamp of approval', was insufficient time to develop lasting organisational change, and put in place structures that enabled the consideration of the wider business issues beyond those procedures required by the standard. the initial emphasis on gaining certification at the expense of the underlying quality principles had serious implications for the ongoing commitment of staff to the software quality management system. one respondent commented that "the perception, held by many staff, is that the quality process generally contributes little to the inherent worth of the services provided to customers. however, it is seen as a competitive advantage, and for this reason the overheads and costs associated with the process are recognised as a necessity". the final lesson is that, if an organisation plans to put in place a software quality management system which has impact on the software product and service to customers, then resources, including time, must be put aside so that business functions can be incorporated into the audits and procedures required by the standard. without a focus on producing tangible results in regard to the final product and customer service, organisations may find it very difficult to generate on-going commitment to the software quality management system. with regard to the future of the standard itself, a number of observations can be made. it would appear that future versions of as 3563 must allow for more scope and flexibility so that the new software development methodologies, and the wide variety of work that information technology firms perform, are taken into account. it would also appear that smaller, ad hoc, jobs need to be either exempt from having to comply with the requirements of the standard, or else be allowed to come under a "blanket" procedure to avoid individual quality plans having to be developed, which creates an administrative burden out of proportion to the jobs themselves. a.iis vol. 3, no. 2 references australian information industries association (1993) quality management issues: an it industry perspective, canberra. hockman, k., grenville, r., & jackson, s. (1994) "road map to iso 9000 registration", quality progress, vol 27, no 5, pp 39-42. rouse, r., watson, d., (1994) "the role of culture in information systems quality", in shanks, g., and arnott, d., eds. proceedings of the 5th australasian conference on information systems, department of information systems, monash university, caulfield east, pp 405-415. sakovsky, s. (1994) "survival after iso 9000 registration", quality progress, vol 27, no 5, pp 57-59. srnic-batanic, b. "[the organisation'si system audit summary table for as 3563.1", quality assurance services, south melbourne. thomsett, r. (1993) people and project management, new york: yourdon press, von hellens, l. (1994) "quality management systems in australian software houses: some problems of sustaining creativity in the software process", in shanks, g., and amott, d., eds. proceedings of the 5th australasian conference on information systems, department of information systems, monash university, caulfield east, pp 479-492. zampetakis, h. (1994) "investment in quality accreditation queried", australian financial review, 31 jan. microsoft word 8 ajis 14-2.doc australasian journal of information systems volume 14 number 2 june 2007 19 a study of journal publication diversity within the australian information systems sphere carmine sellitto centre for international corporate governance research victoria university melbourne, victoria, australia. email: carmine.sellitto@vu.edu.au abstract this study reports on research that examined dest data from 14 australian universities to identify the diversity of journal outlets in the information systems (is) area. across a total of 60 years of academic publishing output, 1449 journal articles were evaluated to identify 649 different journals in which is-related articles were published. the most popular journals used by australian academics to publish isrelated articles were the lecture notes in computer science (n=94) in the computer science area, with the australasian journal of information systems (n=25) being the most popular journal in the pure and business is sphere. the study also examined publishing output against a set of 50 previously highly rated is journals and concluded that the average annual publication of articles in these highly rated journals occurred at a very low rate. the research appears to be one of the first studies to use historical dest data to report journal diversity in the australian is-sphere. introduction journal publication has evolved as the accepted manner in which the academic community disseminates knowledge, an important emphasis being placed on appropriate peer-reviewed journals (sharplin and mabry 1985; baker 2000; tenopir and king 2001). academic publication tends to not only set the foundations for furthering a discipline, but is also used as a gauge of scholarly success— where an important aspect of researcher productivity can be reflected by their scholarly publication record (ramsden 1994). an appropriate journal publication record will invariably influence university promotion and tenure, professional standing amongst peers and external industry groups, as well as impacting on research grant success (katerattanakul et al. 2005). hence, for many scholars, the decision to publish their research entails selecting appropriate outlets that provides a suitable conduit that will allow both exposure of their work and also facilitate personal recognition. in terms of publication opportunities, steele (2006) indicates that there are some 22,000 peer-reviewed journals available to academics in which to disseminate their work— many being english language based and having a northern hemisphere focus. with respect to identifying the is australasian journal of information systems volume 14 number 2 june 2007 20 journal outlets, an evolving and recognised journal compendium is the index of information systems journals (lamp 2004) that lists over 500 active journal outlets in which researchers can publish their work. furthermore, the association for information systems (ais)— the body representing the global is community— lists and rates over 100 journals as potential publishing outlets for the is scholar (http://www.isworld.org/csaunders/rankings.htm). from an australian perspective, little if any research has been undertaken on the academic productivity of australian-based academics that publish in the information systems area. recent commentary by lamp (2006) highlighted a significant increase in available information systems (is) related publication outlets (this paper uses the term is in the same context as lamp (2006)— one that is based on the broad range of journals available to academics for publishing is-related topics). moreover, lamp (2006) highlighted the importance of having journals recognised and listed by the australian government’s department of education, science and training (dest) in order for australian academics to claim reward for their publication productivity. although his practice has been recently discontinued, the remnant dest listing of journals still provides a powerful reference that identifies publications that have met certain peer-review quality attributes. furthermore, considering that the traditional funding of australian universities is directly related to a university’s academic research quantum, it is surprising that there is no centrally maintained ledger or database of past productivity output that details the different types of journals in which isrelated articles are published. indeed, the submission of data to government by universities is a mere summary of the annual publications, with the more informative data remaining with the universities. arguably, with a shifting focus within the australian university sphere onto the quality of academic output— this study takes the view that future journal output values within the is area needs to have at least a baseline point of reference that is reflected by the recently published academic works— allowing some form of future comparison. given the non-centralised availability of research quantum data, this paper reports on research that sourced and evaluated australian dest data to examine aspects of journal publication diversity for academics publishing in is related areas. is journal publication— some ranking and rating studies the number of journals publishing computing and information systems related articles has increased dramatically over the last 50 years (lamp 2006). many of these journals have evolved to focus exclusively on pure-is research, whilst other journals proportionally publish is-related scholarly works. conversely, is academics have, over the last 20 years or so, published is-related work in the journals of various allied disciplines (peffers and ya 2003). indeed, the broad sphere of is activities can be part of a number of related computer and information science areas ranging from engineering, computer science through to business-commerce related information systems (clarke 1999; clarke 2006). given this activity base, there is general recognition that is-related research can be disseminated through a broad spectrum of publishing outlets. in recent years various computing and is journal ranking systems have been published in an endeavour to highlight and evaluate appropriate publication outlets. within academic realms, journal ranking or rating has an implied notion associated with quality— a journal of higher rank being perceived as having greater quality— a factor that can shape manuscript submission behaviour and one that has been reported as influencing an academic’s standing (vokurka 1996; mylonopoulos and theoharakis 2001; katerattanakul et al. 2003; peffers and hui 2003; rainer and miller 2005; lamp 2006). various authors have used citation analysis to evaluate journal value which in turn has been used to propose various is journal rankings (holsapple et al. 1993; holsapple et al. 1994; katerattanakul and han 2002; katerattanakul et al. 2003). is journal ranking australasian journal of information systems volume 14 number 2 june 2007 21 has also been reported from a regional perspective where journal readership within geographical areas can impact on rank and hence perceived journal standing and reputation (avgerou et al. 1999; mylonopoulos and theoharakis 2001; lowry et al. 2004). other authors, such as rainer and miller (2005), after examining a series of is journal rating studies derived a composite is journal listing— a best of the best so to speak. rainer and miller also identified these is journals as relating to four areas of application— computer science, pure is/business is, management and operations. one study that captured a global publication diversity in the is sphere was by peffers and ya (2003). this study alluded to the universe of outlets for is research. the authors identified 326 journals that were segmented into several tiers of perceived quality associated with whether the journals were associated directly with information systems, an allied is area, or were professional/managerial based magazines. the identification of such a high number of journals was facilitated by a methodology that allowed respondents to name and rate journals as an extension to an initial list used in the study. furthermore, the study was noted to represent the values of academics across a spectrum of disciplines that included computer science, science and engineering, information systems, economics, management, social science and operations research. the peffers and ya (2003) study is significant for the large number of is-related publishing outlets identified, however, the authors do not indicate how academics perceived journals on the basis of country of origin or where their universities were located. with respect to determining the views and values associated with australian is journal output, no investigation or research has directly focussed on this academic area. recent work by pervans and shanks (2006) captured the views of australian heads of information systems schools across various areas of is-related activity which also included journal output performance. from their cohort of respondents the authors were able to determine that each school published on average 9.8 peer-reviewed journal articles per year. the study did not report any associated rating or value system associated with the types of journals in which articles were published. mylonopoulos and theoharakis (2001) identified a set of high ranking is journals based on the perceptions of a geographical audience that included australian academics. from a total sample of 979 respondents, some 15% (n=150) were representative of academics from the australasia region— allowing the views of asian, new zealand and australian scholars on journal standing to be captured. the study used a popularity measure to determine the journal ranking of a set of is journals with australasian academics found to have notable similarities and differences in the manner they viewed their journals when compared to their north american and european counterparts. indeed, the authors concluded that researchers from different regions tend to have diverse approaches to conducting research. furthermore, the study identified that almost one third of australasian academics perceived journal ranking lists as an important instrument associated with, or used in, promotional type activities. the study was one of the first to identify journals that were perceived to be important to australasian is academics. lowry et al (2004) conducted a journal ranking study that represented the views of 2559 academics and practitioners from 414 is departments drawn from a global audience. similar to peffers and ya (2003), the study captured the perceptions of respondents across a spectrum of disciplines that ranged from communication, engineering, mathematics, business, informatics, pure information systems and computer science. the study also reported the views of journal standing from a geographical perspective with australasia representing some 20% (~510) of the respondents. with respect to the australasian and latin american regions, lowry et al (2004) asserted that the number of is researchers from these areas will significantly increase in the future, resulting in new forms and types of is journals appearing that will be particular to these geographical regions. across the three mentioned studies, the north american journals predominate with only one regional australasian journal identifiable, that being the australian journal of information systems (ajis). australasian journal of information systems volume 14 number 2 june 2007 22 the top twenty-five rated australasia journals identified by mylonopoulos and theoharakis (2001) and lowry et al (2004), as well as the analogous corresponding set of journals from peffers and ya (2003), are detailed in table 1. table 1 summary findings for three recent is journal rating studies (key: m&t mylonopoulos & theoharakis (2001), p&y peffers & ya (2003), l lowry et al (2004) code journal name (issn) m&t p&y l cacm communications of the acm (0001-0782) 1 — 3 misq mis quarterly (0276-7783) 2 1 1 isr information systems research (1047-7047) 3 2 2 ejis european journal of information systems (0960-085x) 4 4 7 hbr harvard business review (0017-8012) 5 — — ieeetrans ieee transactions (various) 6 — 8 i&m information & management 7 =5 6 jmis journal of management information systems (0742-1222) 8 3 4 dss decision support systems (0167-9236) 9 7 12 acmtrans acm transaction (various) 10 — 9 ms management science (0025-1909) 11 — 5 smr sloan management review (1532-9194) 12 — — database data base for advances in information systems (1532-0936) 13 8 — dsi decision sciences (0011-7315) 14 — 14 jsis journal of strategic information systems (0963-8687) 15 16 15 isj information systems journal (1350-1917) 16 10 10 ijec international journal of e-commerce 17 12 17 ieeecomp ieee computer (0018-9162) 18 — — ajis australian journal of information systems* 19 25 — cais communications of the ais (1529-3181) 20 =5 — amj academy of management journal (0001-4273) 21 — — i&o information & organisation (1471-7727) ** 22 — — acs acm computing surveys (0360-0300) 23 — — amr academy of management review (0363-7425) 24 — — asq administrative science quarterly (0001-8392) 25 — — it&p information technology and people (0959-3845) — 15 18 is information systems (0306-4379) — 21 19 jit journal of information technology (0268-3962) — 40 20 jitm journal of information tech. management (1042-1319) — 27 21 jacm journal of the acm (0004-5411) — 17 22 misqe misq executive (1540-1960) — — 23 ipm information processing and management (0306-4573) — 46 24 jcomp journal on computing (1091-9856) — — 25 jais journal of the ais (1536-9323) — 9 — isf information systems frontiers (1387-3326) — 18 — jgim journal of global information management (1062-7375) — 19 — misqd misq discovery (no issn) — 20 — joeuc journal of organizational & end user comp. (1546-2234) — 22 — jgitm j. of global information tech. management (1097-198x) — 23 — infosci informing science (1521-4672) — 24 — * name changed in 2003 to australasian journal of information systems (1449-8618). ** known as accounting, management & information technologies until 2001. australasian journal of information systems volume 14 number 2 june 2007 23 australasian journal of information systems volume 14 number 2 june 2007 24 academic journal publication and the australian dest journal publication output has been alluded to as being one of the more significant indicators of research productivity for the modern day scholar (koh 2003). within the realms of australian university operations, ramsden (1994) discusses general academic research productivity amongst australian academics alluding to parameters such as quality, importance, impact and quantity— attributes that could be used to evaluate research performance. impact was deemed to be a measure of influence reflected in citation counts; quality was the perceived value of an article in the eyes of knowledgeable peers; an article’s importance required the passing of time for it to be recognised and quantity related to the number of scholarly publications produced as either counts or total pages. ramsden (1994) further suggested that the number of academic articles published was an important quantity metric in that it could be associated with the impact value of a researcher’s output. in terms of measures, quantity is the simplest to gauge and determine and is one that reflects scholarly productivity at an individual and institutional level. when capturing aspects of australian academic productivity, the onus rests with individual universities to manage, administer and record their scholarly research activities if they wish to be rewarded by government. to record and address scholarly publication output, australian universities engage in an annual higher education research data collection (herdc), that groups scholarly works into four types— books, book chapters, journal articles and conference publications. in effect the data collection reflects the scholarly publishing activities of australian academics for which their university is able to receive a compensatory return from the australian government (dest 2006). furthermore, when collecting publication data, academic works can be grouped into various domains to address the research field, educational course and/or an academic discipline (rfcd). although the allocation of an rfcd code to a publication is not a compulsory requirement for dest quantum returns, many universities collect this classifying information because it gives them a relatively accurate and representative indication of scholarly activities undertaken across specific research areas (arc 2005). one research area in the herdc process that groups publications in an information, computing and communication sciences domain is the 280000 division— a division that could be viewed as representing the general research area associated with is. this division allows authors to classify their work into specific fields associated with information systems, computation theory and mathematics, artificial intelligence, signal and image processing, computer software and data format. furthermore, two other rfcd divisions that address business aspects of the is area— electronic commerce (rfcd 350213) and business information systems (rfcd 350202)— are also relevant to this study. hence, a broad spectrum of discipline areas defines the is-sphere in this paper. furthermore, the annual herdc activity amongst the higher education sector in australia provides a significant data source in allowing the investigation of individual divisions and/or single disciplines to determine the journal publication diversity for that particular area. moreover, the compliance requirements placed on universities to maintain rigid data integrity for this type of data collected makes the data an extremely reliable and reflective resource (for an explanation of acceptable journal definitions and peer-view criteria associated with dest see lamp 2006). the pending australian research quality framework (rqf) the australian research quality framework (rqf) is scheduled for introduction in 2008. a basic tenet underlying the australian government’s introduction of the rqf is the way that research is assessed— moving from a quantitative to a qualitative focus. the new rqf aims are admirable and include the comparison of academic research with international benchmarks, as well as australasian journal of information systems volume 14 number 2 june 2007 25 endeavouring to assure that public money is achieving the appropriate research outcomes (bishop 2006). the impacts of the rqf on individual academics are numerous— however, one important issue when it comes to journal output is a notable shift from productivity to quality. in preparing for the rqf, the australian council of professors and heads of information systems (acphis) in conjunction with aais (australasian association for information systems), has proposed a list of 61 journals that can be viewed as meeting different standards of academic quality in the is sphere (aais 2006). the acphis (2006) journals are structured to reflect different levels of quality based on the their ranking status— level-a & b; non-ranked and premier professional journals. many of these journals are derived from previous journal ranking studies including the ones previously described in this paper. the level a set of journals are the definitive publication types that acphis indicates the is community should aspire to publish in. the 10 level a journals are decision sciences, decision support systems, european journal of information systems, information systems journal, information systems research, information and management, journal of management information systems, management science, mis quarterly and the journal of the ais. achphis in commenting on academic performance and the rqf suggest that it would not be expected that many scholars in australia would publish more than one or two articles in these journals. arguably, the academics that do publish in the level a journals will have a deservedly outstanding reputation as the best in their field (acphis 2006). the data collect in this study allows an opportunity to explore this acphis issue with respect to level a publication. formulating the research questions the objectives of this paper are to examine available dest research quantum data to determine the academic journal publication activity in the is-related areas. the research adopts an exploratory approach to investigating the topic and several questions are used to direct and present the research. peffers and ya (2003) identified a relatively high number (n=326) of different journals that were perceived to appropriate to publish is research. in this study the identification of the total number of publication outlets can be considered to represent the diversity of journals used by australian-based academics to publish is-related articles. thus, the first research question (rq1) posed is: what journal outlets are used by australian academics to publish is-related research? the first question can be expanded to determine the most popular journals in which academics have published. determining the popular journal publications from a historical perspective identifies the actual, rather than the perceived journal values of academics. this output could be also viewed as an indicator of preferred journal forums. hence, the second research question (rq2) posed is: which journals have been most preferred by australian academics for article publication as measured by the total number of articles published in each journal? given the previously global studies capturing the perceptions of academics with respect to journal ranking, the third question posed examines the general performance of australian scholars compare to ranked journal lists. the question (rq3) posed: to what degree have australian academics published in the highly rated journals identified in previously reported studies (mylonopoulos and theoharakis 2001; peffers and ya 2003; lowry et al. 2004)? with the pending rqf approaching, it becomes relevant to examine the performance of academics against the achpis-identified level a journals (the limitations of the study prevents an evaluation against the full set of journals). hence, the question (rq4) posed: australasian journal of information systems volume 14 number 2 june 2007 26 to what degree have australian academics published in the level a journals identified by achpis (2006)? methodology australian universities are rewarded for their research output and engage in an annual higher education research data collection (herdc) process to validate and report their scholarly works. part of the process associated with herdc is to group publications in various rfcd divisions— a process that although not compulsory, many universities use in order to gauge the range of research areas their academics engage. with respect to dest-collected data, there is no common data repository that allows the respective data sets to be united to reflect the collective publication output of all australian universities— this output is held in what could be viewed as isolated data islands by each university. hence, the individual dest collections of australian universities formed the domain from which data for this research was sourced. sourcing and procuring the data all australian universities (n=38) were approached to participate in the study by providing data associated with academic publishing output. the information systems publication data of interest was drawn from refereed journal publications (commonly coded c1) that had been grouped in the 280000 (information, computing and communication sciences), 350202 (business information systems) or 350213 (electronic commerce) rfcd divisions. the fields requested for each journal record included attributes such as issn, journal and article title, publisher and number of pages published for a five-year data-collection period (20012005). moreover, in order to minimise transcription errors and assist with easier consolidation and evaluation of data, universities were asked to supply their data in spreadsheet format. data was sourced between october and december 2006. examining the data for correctness each university’s data set was examined for anomalies, duplication or errors and involved eliminating data records that had a crucial field missing such as the journal title and removing data records data that were not part of the requested rfcd codes. it was noted that some journal titles did not have an obvious or typical affiliation with themes relevant to the research. when an atypical is journal title was encountered, the article associated with the journal was examined to confirm that it was an is-related article and that this entry was not an rfcd classification error. for example, a data record that listed a journal such as the australian family physician would have the associated article examined to see that it related to a computing theme. if the article was deemed by the researcher to be non-is in nature, it was assumed that the record had been erroneously grouped and was eliminated. once all individual university data sets were consolidated, article records were examined for duplicate listings. results fourteen of australia’s 38 universities responded to a request to provide data for the study, representing a 36.8% participation rate. the universities that participated in the study included the australasian journal of information systems volume 14 number 2 june 2007 27 following groupings (aen 2006)— group of eight (n=3), australian technology network (n=2), new generation universities (n=2), innovative research universities (n=2) and the regional/other universities (n=5). a total of 1449 journal articles were identified across the 14 universities as having published isrelated themes. these articles were published between 2000 and 2005 and represent a total of 60 years of collective academic article output across the 14 participating universities. it should be noted that because each university had a different approach to data grouping, there was variability in the annual data supplied by each university. hence, not all universities provided a complete set of journal publication records associated with all years between 2000 and 2005. indeed, many universities indicated that it had only become a recent practice to classify publications within their rfcd grouping. the total number of journal records supplied by each university was variable and reflected the relative is activities within that university. it can only be assumed that universities that had a large and well-established sphere of is activity had a greater output than those universities in which is had a secondary or low research focus. the average number of journals published across the 14 universities for the 60 years of data collection was 24.15 articles per year. however, there was a significant variation amongst individual universities in terms of the average number of journal articles published per year with the greatest number of articles publish being 61.0 and the least 6.3. table 2 summarises the journal publication data used in this study. table 2 summary of australian university publication data university code data collection period years of output (n) number of dest articles (n) average journal articles per year a 2000-2005 5 305 61.0 b 2001-2005 5 280 56.0 c 2001-2005 5 160 32.0 d 2002-2005 4 116 29.0 e 2000-2005 6 111 18.5 f 2000-2005 6 84 14.0 g 2004-2005 2 78 39.0 h 2001-2005 5 76 15.2 i 2001-2005 5 70 14.0 j 2004-2005 2 64 32.0 k 2001-2005 5 32 6.4 l 2002-2005 4 29 7.3 m 2002-2005 4 25 6.3 n 2004-2005 2 19 9.5 rq1 what journal outlets are used by australian academics to publish is-related research? an examination of the 1449 journal articles indicates that academics across the 14 universities published articles associated with is-related research in 649 different journals (the full set of identified journals and the number of articles published in each is listed in the appendix). the total number of different journals identified in this research surpasses the number (326) identified by peffers and ya (2003) in their global study of is publication outlets. the identification of a high australasian journal of information systems volume 14 number 2 june 2007 28 number of different journals represents the universe of journals that australian academics have used to publish their is-related research over the 2000-2005 period. in examining the distribution of articles published, it was found that 13 journals published 20.2% of all articles (n=292), whilst the remaining 1157 articles (79.8%) had a publication distributed across the remaining 636 journals. table 3 details the articles cumulative distribution values for the 50 most popular journals that published is-related articles. notably, 386 is-related manuscripts were published as one-off articles in different journals. a plot of each individual journal occurrence against the number of articles published in each journal is depicted in figure 1. 0 10 20 30 40 50 60 70 80 90 100 0 100 200 300 400 500 600 700 journals (n) articles (n) the long tail distribution of is-related article publication figure 1 number of articles published in different journals this high occurrence of one-off published articles, each in a different journal results in a long tail distribution effect that collectively represents the journal-selection and publishing behaviour of academics. this long tail effect could be viewed as also reflecting the multi-dimensional nature of the is area where by necessity there needs to be a broad selection of publication outlets to accommodate the varied and distinct scholarly interests. moreover, this distribution also represents the practicality of a high number of articles not being able to be published in the limited available space of the more popular journals. indeed, the nature of the is area is one of rapid development, albeit in technology, change management or business processes— which appear to require a significant diversity of journal publishing venues. significantly, the commonly held pareto phenomenon (the 80:20 rule), where it could be anticipated that 80% of articles would be associated with some 20% of available journals, does not appear to hold. in this case, some 20% of articles are associated with only 2% of identified journal outlets. rq2 which journals have been most preferred by australian academics for article publication as measured by the total number of articles published in each journal? this research question identifies the popular journals that have been used to publish is-related scholarly articles. table 3 lists the top 50 journals that published up to 5 articles by australian academics across the data collection period. some observations on the popular journals published include the following: • the top two journals relate to the computer science sphere and are affiliated with the springer publishing group. these two journals (lecture notes in computer science and australasian journal of information systems volume 14 number 2 june 2007 29 lecture notes in artificial intelligence) provide a significant outlet for computer science related articles. the modus operandi for considering articles by these two journals is to select cutting edge and pertinent works from high profile conferences allowing the latest and most up-to-date research to be quickly and broadly disseminated. • many of the popular manuscript outlets are in journals associated with the institute of electrical and electronics engineers (ieee). indeed, the high number of articles published in these types of journals is arguably an indication of the historical standing the ieee has amongst its practitioner membership as well as engineering and computer science academics. moreover, the ieee provides a broad scope of content coverage and this is reflected in the range of journals available to academics in which they can publish their work. the most popular ieee journal was transactions on signal processing and transactions on information theory. • it was observed that 6 of the 50 popular journals were found to specialise in publishing letters— where 48 articles were published in this type of output (across all 649 journals identified, 12 were letters). all the letters journals had an engineering, computer science and or electronics/data processing focus that as a general rule provide an avenue for academics to publish short but informative cutting-edge work. arguably, there appears to be a lack of this letters publication type available to academics wishing to publish in the more pure is-related area. • the most popular pure or business-related is journal was the australasian journal of information systems that published 25 articles. these articles were published by academics affiliated with 10 of the 14 universities that participated in the study. other popular pure or business-related information systems journals included the journal of issues in informing science & information technology, journal of information technology, communications of the ais and journal of the american society for information science & technology, jitta and journal of strategic information systems. • six articles were published in bioinformatics— a journal that tends to reflect the recent interest in computational science and represents the junction of medical-biology and the large data management requirements affiliated with this nascent area. • various other notable journals found to be popularly related to software engineering or development (information and software technology, journal of systems and software, international journal of software engineering & knowledge engineering), whilst scientometrics can be viewed as representing the information and library management sphere of is-related output. • like many previous studies that have identified popular journals in which articles were published, the list is dominated by the usa-north american journals with only three journals (australasian journal of information systems, australian journal of intelligent information processing systems and australasian journal of combinatorics) having an australian focus and base. australasian journal of information systems volume 14 number 2 june 2007 30 table 3 most popular 50 journals in which articles were published journal popularity journal name articles published (n) cumulative article count cumulative % (n=1449) 1 lecture notes in computer science 94 94 6.5% 2 lecture notes in artificial intelligence 52 146 10.1% 3 australasian journal of information systems 25 171 11.8% 4 ieee transactions on signal processing 24 195 13.5% 5 ieee transactions on information theory 13 208 14.4% 6 electronics letters 12 220 15.2% 7 information and software technology 11 231 15.9% 8 j of issues in informing science & information technology 11 242 16.7% 9 theoretical computer science 11 253 17.5% 10 international j of computer systems science & engineering 10 263 18.2% 11 journal of information technology 10 273 18.8% 12 journal of systems and software 10 283 19.5% 13 information processing letters 9 292 20.2% 14 journal of research & practice in information technology 9 301 20.8% 15 j of the am. society for information science & technology 9 310 21.4% 16 australian journal of intelligent information processing systems 8 318 21.9% 17 computer communications 8 326 22.5% 18 ieee communications letters 8 334 23.1% 19 ieee transactions on aerospace & electronic systems 8 342 23.6% 20 ieee transactions on communications 8 350 24.2% 21 systems & control letters 8 358 24.7% 22 ieee computer 7 365 25.2% 23 ieee transactions on automatic control 7 372 25.7% 24 wseas transactions on computers 7 379 26.2% 25 communications of the ais 6 385 26.6% 26 ieee transactions on fuzzy systems 6 391 27.0% 27 ieee transactions on neural networks 6 397 27.4% 28 ieee transactions on pattern analysis & machine intelligence 6 403 27.8% 29 information sciences 6 409 28.2% 30 international j of computational intelligence & applications 6 415 28.6% 31 int. j of software engineering & knowledge engineering 6 421 29.1% 32 jitta: an information systems journal 6 427 29.5% 33 journal of object technology 6 433 29.9% 34 machine learning 6 439 30.3% 35 neurocomputing 6 445 30.7% australasian journal of information systems volume 14 number 2 june 2007 31 36 pattern recognition letters 6 451 31.1% 37 signal processing 6 457 31.5% 38 acm transactions on information systems 5 462 31.9% 39 artificial intelligence 5 467 32.2% 40 australasian journal of combinatorics 5 472 32.6% 41 automatica 5 477 32.9% 42 bioinformatics 5 482 33.3% 43 data & knowledge engineering 5 487 33.6% 44 iee proceedings: vision image & signal processing 5 492 34.0% 45 ieee signal processing letters 5 497 34.3% 46 ieee trans. on systems, man & cybernetics part a: systems & humans 5 502 34.6% 47 international journal of robotics research 5 507 35.0% 48 journal of strategic information systems 5 512 35.3% 49 scientometrics 5 517 35.7% 50 software-practice & experience 5 522 36.0% rq3 to what degree have australian academics published in the highly rated journals identified in previously reported studies? previous studies (mylonopoulos and theoharakis 2001; peffers and ya 2003; lowry et al. 2004) identified a set of highly rated is journals (table 1). the highly rated is journals that have been used by australian academics to publish their articles in the 2000-2005 period are summarised in table 4. in effect this aspect of the research records the historical scholarly output of australian academics within the sphere of journals that have been perceived as being of relatively high standing— especially in the pure and business is areas. table 4 australian academic publishing in highly rated journals (key: m&t mylonopoulos & theoharakis (2001), p&y peffers & ya (2003), l lowry et al (2004) journal rank code journal name (issn) m& t p & y l aust. article s (n= 1449) ajis australian journal of information systems* 25 19 — 25 jit journal of information technology (0268-3962) 40 — 20 10 ieeecom p ieee computer (0018-9162) — 18 — 7 cais communications of the ais (1529-3181) 5 20 — 6 jsis journal of strategic information systems (0963-8687) 16 15 15 5 isj information systems journal (1350-1917) 10 16 10 3 ipm information processing and management (0306-4573) 46 — 24 3 isf information systems frontiers (1387-3326) 18 — — 3 cacm communications of the acm (0001-0782) — 1 3 2 ejis european journal of information systems (0960085x) 4 4 7 2 i&m information & management 5 7 6 2 australasian journal of information systems volume 14 number 2 june 2007 32 dss decision support systems (0167-9236) 7 9 12 2 databa se data base for advances in information systems (1532-0936) 8 13 — 2 ijec international journal of e-commerce 12 17 17 2 it&p information technology and people (0959-3845) 15 — 18 2 misqe misq executive — — 23 2 jgim journal of global information management (10627375) 19 — — 2 jgitm journal of global information tech. management (1097-198x) 23 — — 2 infosci informing science (1521-4672) 24 — — 2 misq mis quarterly (0276-7783) 1 2 1 1 isr information systems research (1047-7047) 2 3 2 1 jmis journal of management information systems (07421222) 3 8 4 1 is information systems (0306-4379) 21 — 19 1 joeuc journal of organizational & end user comp. (15462234) 22 — — 1 ieeetran s ieee transactions (various) — 6 8 127 (see table 5) acmtran s acm transaction (various) — 10 9 21 (see table 5) * name changed in 2003 to australasian journal of information systems (1449-8618). ** known as accounting, management & information technologies until 2001. an examination of table 4 indicates that australian academics associated with 14 universities published a total of 89 articles in 24 journals that were perceived as being highly rated. placing this finding in the context of the total number (1449) of articles evaluated— some 6% of the is-related articles published between 2000 and 2005 were in highly rated is journals. this equates to an average of 6.36 (89/14) articles per university published in highly related is journals over the 20002005 period. previous studies also identified various transaction journals affiliated with the association of computing machinery (acm) and the ieee that had a relatively high standing amongst scholars— listed as 6th and 10th in table 1. within these two transactional areas a total of 157 is-related articles were published in 46 journals (see table 5). if one assumes that the popular transaction journals identified are also of a high standing then in terms of total number of articles (1449) it can be said that some 17% (n=246) of articles were published in highly rated journals, with each university over the 2000-2005 period having an average output of 17.57 articles. australasian journal of information systems volume 14 number 2 june 2007 33 table 5 ieee and acm transactions publishing australian is-related articles ieee transactions n acm transactions n ieee tran. on signal processing 24 acm tran. on information systems 5 ieee tran. on information theory 13 acm tran. on database systems 3 ieee tran. on aerospace & electronic systems 8 acm tran. on software engineering & methodology 3 ieee tran. on communications 8 acm tran. on computational logic 2 ieee tran. on automatic control 7 acm tran. on programming languages & systems 2 ieee tran. on neural networks 6 acm tran. on asian language information retrieval 1 ieee tran. on pattern analysis & machine intelligence 6 acm tran. on computer-human interaction 1 ieee tran. on fuzzy systems 6 acm tran. on design automation of electronic systems 1 ieee tran. on systems, man & cybernetics part a: systems & humans 5 acm tran. on information & system security 1 ieee tran. on image processing 4 acm tran. on mathematical software 1 ieee tran. on knowledge & data engineering 4 ieee tran. on systems, man & cybernetics part b: cybernetics 4 ieee tran. on systems, man & cybernetics part c: applications & reviews 4 ieee tran. on biomedical engineering 3 ieee tran. on circuits & systems (part 1) 3 ieee tran. on circuits & systems ii express briefs 3 ieee tran. on geoscience & remote sensing 3 ieee tran. on software engineering 3 ieee tran. on vehicular technology 3 ieee tran. on wireless communications 3 ieee tran. on education 2 ieee tran. on speech & audio processing 2 ieee tran. on very large scale integration (vlsi) systems 2 ieee tran. on circuits & systems for video technology 1 ieee tran. on computers 1 ieee tran. on consumer electronics 1 ieee tran. on dependable & secure computing 1 ieee tran. on evolutionary computation 1 ieee tran. on industrial electronics 1 ieee tran. on multimedia 1 ieee tran. on parallel & distributed systems 1 ieee tran. on power delivery 1 ieee tran. on robotics & automation 1 ieee/acm tran. on computational biology and bioinformatics 1 australasian journal of information systems volume 14 number 2 june 2007 34 rq4 to what degree have australian academics published in the level a journals identified by achpis (2006)? twelve articles were published in 7 of the a level journals. the authors (n=23) contributing to the articles published in this set of journals were associated with 12 different discipline areas as noted by their affiliated faculty, school or area of research. seven different universities reported articles that were published in level a journals. no author published in more than one of these journals and 7 of the 12 articles had collaborating authors that were based at a non-australian university— 9 of the 23 authors (39.1%) were overseas based. table 3 summarises the characteristics associated with the level a journals. table 3 acphis level a journals: article characteristics journals articles disciplines associated with contributing authors* source (uni code) author’s academic level** decision support systems 2 computer science, information technology, management, computer engineering, business e, l european journal of information systems 2 computing, information systems, business, informatics, information management, communications b, d information systems journal 3 information systems, information technology, accounting, management b, c, l information systems research 1 information management, information systems, accounting b information and management 2 business, information management, information systems, communications, library studies a, d journal of management information systems 1 information systems, communications, library studies i mis quarterly 1 information systems, management a professor (os) 4 associate professor 3 senior lecturer 3 lecturer 3 professor 2 assistant professor (os) 2 associate professor (os) 1 director of research 1 lecturer (os) 1 phd student 1 professorial fellow (os) 1 research associate 1 no articles were published in the following acphis identified a level journals— decision sciences, management science or the journal of the ais. *list is one of identification of the different disciplines and does not reflect proportional representation of each discipline. **no distinction made between chair professor and professor. overseas (os) authors identified. this set of level a journals is consistently noted as a group of quality journals in various ranking studies (ais 2007). notwithstanding the low number of articles published in this set of journals some observations are possible: • in terms of performance and rqf standing, the 12 articles published represents just 0.83% of the total of 1449 identified articles as forums used by academics to publish their work. given, that the study reflects some 60 years of output across 14 universities, this findings indicates that many academics— not just those represented by acphis— are not publishing in this select group of journals. australasian journal of information systems volume 14 number 2 june 2007 35 • achphis may have made an assumption that the set of level a journals will implicitly publish articles from the academic constituency that it represents—the pure or business is area. from this selection of articles, there appears to be a relatively high number of allied disciplines associated with authors that have collaborated on articles. • there is a relatively high collaboration of australian based authors with overseas colleagues reflected in the articles published in this set of journals. this may suggest that australian academics in order to publish in these well known international journals may need to engage collegial collaboration with overseas scholars to facilitate this process. • there is a diversity of academic levels represented associated with the 23 different authors identified. this aspect of the study tends to suggests that the opportunities to publish articles in these top level journals is not restricted to senior academics such as professors and associate professors. with the pending rfq to be introduced in 2008, it can only be assumed that very few academics that are governed by the acphis proposal will be able to claim a quality article in the level a journals. furthermore, noting the relatively low number of articles that are annually published by each journal, it is highly commendable that some australian authors have been successful in publishing articles in this group of journals— in what is undoubtedly a competitive publishing arena. arguably, the commensurate reward for having published in these types of journals should be one of significant recognition of scholarly status by acphis and associated scholars. although achis has suggested that australian is academics should aspire to publish in these journals, in reality, it is likely that only high achieving academics will publish a small proportion of their articles in these journals across their careers. for some academics the publishing of an article at this level might well be associated with the pinnacle of their career. conclusion this study reported research that evaluated dest research quantum data from 14 australian universities for the 20002005 period. it examined journal publication productivity in rfcd areas associated with information, computing and communication sciences, business information systems and electronic commerce. the research is significant in that it is the first study that has reported australian academic journal publication output in the is sphere. moreover, the use of dest-derived data that forms the basis of the research’s investigations is an extremely reliable resource that underpins the foundations of the findings. across a composite total of 60 years of academic publishing output, 1449 journal articles were evaluated to identify 649 different journals in which is-related articles were published. the study identified a long tail distribution of journals, determining that although many journals were popular outlets for academics— a series of one-off is-related articles were published across 386 different journals. the most popular journals used by australian academics were the lecture notes in computer science (n=94) and lecture notes in artificial intelligence (n=52) in the computer science area, with the australasian journal of information systems (n=25) being the most popular journal in the pure and business is sphere. the study also examined publishing output against a set of 50 previously highly rated is journals. australian academics were found to have published some 6% of their output in these previously highly rated journals (up to 17 % if the transaction journals of the ieee and acm identified in this study are included in counts). given that the data was collected over an equivalent time period that totalled 60 academic output years, it can be concluded that the australasian journal of information systems volume 14 number 2 june 2007 36 average annual publication of articles in these highly rated journals per university occurred at a very low rate. furthermore, the study examined the performance of australian academics against the level a set of journals proposed by acphis finding that only 12 articles had been published in these journals within the 2000-2005 period. in view of the introduction of the rqf in the 2008, and the importance of the acphis proposal—it can only be assumed that very few academics will be able to claim a quality article performance at this highest acphis level. limitations and future research the main limitation of the study is the university participation rate that only reflects the is-related publishing output of 14 of australia’s 38 universities. hence, the study can only really be viewed as a snapshot of the is-related output and should be interpreted in that context. the participation of a greater number of universities in the study would have arguably resulted in an even broader diversity of journals. the study used a set of rfcd divisions to frame the scope of the study and source dest data. there may have been other rfcd categories such as librarianship (rfcd 400201) and multimedia (410303) that some academics view as belonging to is-sphere that where excluded in the study. hence, the results need to also be considered in the context of the rfcd areas from which data was sourced. the first two research questions are general in their scope and are reported as such, whilst question three, although having a focus on the pure-is area can still be viewed as exploratory and also general in coverage. hence, this broad/exploratory approach to investigating is-related journal output could be viewed as a form of limitation of the study. given the diversity of journals identified, future research will examine the identified journals for their impact value as reflected in, for example, the journals derived citation impact factor (isi®). not all journals will have an isi® value which in itself would be a notable finding. this approach using the isi® system does have its limitations, however, there is some perception of quality associated with journals that have high citation rates. the research identified a long tail distribution of journals— that represented a significant number of is-related articles being published in different individual journals. further investigation of the long tail will allow the identification of niche or specialty areas that may be emerging within the is-sphere. a cursory examination of the long tail indicates that there may be groupings of articles across different journals that identify health informatics, knowledge management and geographic systems niches. with respect to the acphis list of quality journals, an interesting and relevant extension of this research would be to use the study’s data set to evaluate academic performance against the remaining premier professional, level b and nr journals. references aen (2006). the australian education network: australian university groupings http://www.australian-universities.com/groupings-of-australian-universities.php [accessed 11/1/2007]. arc (2005). australian research council (arc): research fields, courses and disciplines classification (rfcd) codes http://www.arc.gov.au/apply_grants/rfcd_codes.htm [accessed 4/11/2006]. avgerou c., siemer j. and bjorn-andersen n. (1999). the academic field of information systems in europe. european journal of information systems, 8 (2): pp. 136-153. baker m. j. (2000). writing a literature review. the marketing review, 1 (2): 219-247. australasian journal of information systems volume 14 number 2 june 2007 37 clarke r. (1999). fundamentals of information systems http://www.anu.edu.au/people/roger.clarke/sos/isfundas.html [accessed 3/11/2006]. clarke r. (2006). key aspects of the history of the information systems discipline in australia. australasian journal of information systems, 14 (1): pp. 123-140. dest (2006). higher education research data collection: specifications for the collection of 2006 data, innovation and research systems group department of education, science and training (dest) http://www.dest.gov.au/nr/rdonlyres/830034e5-326a-4961-a02de3f34407a3f7/12781/draftherdc2007.pdf [accessed 4/11/2006]. holsapple c. w., johnson l. e., manakyan h. and tanner j. (1993). a citation analysis of business computing research journals. information and management, 25 (5): pp. 231-244. holsapple c. w., johnson l. e., manakyan h. and tanner j. (1994). business computing research journals: a normalised citation analysis. journal of management information systems, 11 (1): pp. 131-140. katerattanakul p. and han b. (2002). are european is journals under-rated? an answer based on citation analysis. european journal of information systems, 12 (4): pp. 60-71. katerattanakul p., han b. and hong s. (2003). objective quality ranking of computing journals. communications of the acm, 46 (10): pp. 111-114. katerattanakul p., razi m. a., han b. and kam h.-j. (2005). consistency and concern on is journal rankings. journal of information technology theory and aplication (jitta), 7 (2): pp. 1-20. koh c. e. (2003). is journal review process: a survey on is research practices and journal review issues. information management, 40 (8): pp. 743-756. lamp j. w. (2004). index of information systems journals, geelong: deakin university (http://lamp.infosys.deakin.edu.au/journals/). lamp j. w. (2006). recognition as a distinguishing criterion of is journals. australasian journal of information systems, 13 (2): pp. 7-16. lowry p. b., romans d. and curtis a. (2004). global journal prestige and supporting disciplines: a scientometric study of information systems journals. journal of the association for information systems, 5 (2): pp. 29-77. mylonopoulos n. and theoharakis v. (2001). on-site: global perceptions of is journals. communications of the acm, 44 (9): pp. 29-33. peffers k. and hui w. (2003). collaboration and author order: changing patterns in is research. communications of the ais, 11 (2): pp. 166-190. peffers k. and ya t. (2003). identifying and evaluating the universe of outlets for information systems research: ranking the journals. the journal of information technology theory and application, 5 (1): pp. 63-84. pervan g. and shanks g. (2006). the 2005 survey of information systems research in australia. australasian journal of information systems, 14 (1): pp. 273-280. rainer k. and miller m. (2005). examining differences across journal rankings. communications of the acm, 48 (2): pp. 91-94. australasian journal of information systems volume 14 number 2 june 2007 38 ramsden p. (1994). describing and explaining research productivity. higher education, 28: pp. 207-226. sharplin a. d. and mabry r. h. (1985). the relative importance of journals used in management research: an alternative ranking. human relations, 38 (2): pp.139-149. steele c. (2006). research with purpose. the australian higher education section. sydney, p. 42. tenopir c. and king d. w. (2001). lessons for the future of journals. nature, 413 (6857): pp. 672673. vokurka r. j. (1996). the relative importance of journals used in operations management research: a citation analysis. journal of operations management, 14 (4): pp. 345-355. appendix— listing of 649 journals publishing is-related articles acm sigmetrics—1 acm sigsoft software engineering notes—1 acm transactions on asian language information retrieval—1 acm transactions on computational logic—2 acm transactions on computer-human interaction—1 acm transactions on database systems—3 acm transactions on design automation of electronic systems—1 acm transactions on information & system security—1 acm transactions on information systems—5 acm transactions on mathematical software—1 acm transactions on programming languages & systems—2 acm transactions on software engineering & methodology—3 acoustic australia—1 acta cybernetica—1 acta informatica—1 acta polytechnica—1 advanced technology for learning—1 advances and applications in statistics—1 advances in applied probability—1 advances in complex systems —1 advances in engineering software—1 advances in environmental research—1 advances in physiology education—1 aeronautical journal—1 ai communications—1 ai magazine—1 algorithmica—2 american journal of physiology: gastrointestinal & liver physiology—1 anale (seria informatica)—2 annals of cases on information technology—3 annals of cases on information technology applications & management in organizations—1 annals of mathematics & artificial intelligence—2 annals of operations research—1 annals of probability—1 applicable algebra in engineering, communication & computing—1 applied artificial intelligence—1 applied mathematical modelling—1 applied mathematics and computation—3 applied soft computing—3 archive of applied mechanics—1 artificial intelligence—5 artificial intelligence & law—1 artificial intelligence for engineering design, analysis & manufacturing (ai edam)—1 artificial intelligence in medicine—2 asia pacific management review—1 asia pacific public relations journal—1 asian academy of management journal—1 asian journal of control—2 asian journal of information technology—3 asian review of accounting—1 asia-pacific journal of operations research—1 australasian journal of combinatorics—5 australasian journal of educational technology—1 australasian journal of information systems—25 australian & new zealand journal of public health—1 australian academic & research libraries—4 australian accounting review—3 australian computer science communications (acs)—3 australian computer science communications (ieee)—1 australian family physician—3 australian journal of communication—3 australian journal of electrical & electronics engineering—2 australian journal of emerging technologies & society—3 australian journal of experimental agriculture—1 australian journal of intelligent information processing systems—8 australian journal of public administration—1 australian library journal—1 automatica—5 autonomous agents and multi-agent systems—1 behavior research methods, instruments, & computers—3 behaviour & information technology—2 behaviour change—2 benchmarking: an international journal—1 bioinformatics—5 bit—1 bmc medical informatics and decision making—1 bmc pediatrics—1 british journal of educational technology—1 business process management journal—1 canadian journal of information & library science—1 cartographic journal—1 chaos solitons & fractals—1 chinese astronomy & astrophysics—1 chinese geograpical science—1 circuits, systems & signal processing—3 cognitive science quarterly—1 combinatorics probability & computing—1 communicating sequential processes—1 communications in algebra—2 communications of the acm—2 communications of the ais—6 complexity international—1 comptes rendus mecanique—1 computational mechanics—1 computer communication review—2 computer communications—8 computer engineering & applications—1 computer journal—1 computer law & security report—1 computer methods & programs in biomedicine—1 computer networks—2 computer physics communications—1 australasian journal of information systems volume 14 number 2 june 2007 39 computer science education—1 computers & education—1 computers & electrical engineering—1 computers & electronics in agriculture—2 computers and graphics—2 computers and industrial engineering—2 computers and mathematics with applications—1 computers and security—1 computers and the humanities—1 computers in biology and medicine—1 computers in human behavior—1 computers, environment and urban systems—1 computing and informatics—1 computing and informatics journal, special issue on grid computing—1 computing in science and engineering—1 concurrency and computation-practice & experience—4 concurrent engineering: research & applications—2 consciousness and cognition—1 constraints—3 crossroads—1 cryptologia—4 current science—1 cybernetics and systems—1 data & knowledge engineering—5 data base for advances in information systems—2 data mining and knowledge discovery—1 decision support systems—2 designs codes and cryptography—1 development southern africa—1 digital signal processing—2 discrete applied mathematics—1 discrete mathematics—3 distributed & parallel computing—1 distributed & parallel databases—4 earth planets and space—1 e-business review—1 ecological modelling—1 education and information technologies—1 education for information—1 education, communication and information—1 educational technology & society—4 ejournalist—2 electronic commerce research—1 electronic commerce research & applications—4 electronic communication law review—1 electronic journal of business research methods—1 electronic journal of information systems evaluation—2 electronic journal of information systems in developing countries—1 electronic journal of knowledge management—1 electronic journal on information systems in developing countries—1 electronic library, the—1 electronic notes in theoretical computer science—3 electronic transactions on artificial intelligence—1 electronics and telecommunications research institute journal—1 electronics letters—12 empirical software engineering—2 engineering applications of artificial intelligence—1 environment & planning b: planning & design—2 environment & planning c: government & policy—1 environmental science & technology—1 e-service journal—1 ethics and information technology—2 eurasip: journal on applied signal processing—3 eurasip:journal on wireless communications & networking—1 european journal of dental education—1 european journal of information systems—2 european journal of operational research—2 evolutionary computation—1 expert systems—1 expert systems with applications—1 far east journal of applied mathematics—2 fibonacci quarterly—1 finite fields and their applications—1 first monday—2 forensic science international—1 forum: qualitative social research—2 foundations of physics letters—1 future generation computer systems—2 geojournal—1 gests international transaction on computer science & engineering—1 global journal of flexible systems management—1 graphical models—1 graphs and combinatorics—1 group decision and negotiation—2 harvard negotiation law review—1 health informatics journal—2 health information management—1 higher-order and symbolic computation—1 human factors—1 ibm systems journal—1 iee proceedings: communications—2 iee proceedings: computers & digital techniques—3 iee proceedings: control theory & applications—1 iee proceedings: software—1 iee proceedings: vision image & signal processing—5 ieee communications letters—8 ieee communications magazine—2 ieee computational intelligence bulletin—1 ieee computer—7 ieee computer graphics & applications—1 ieee geoscience & remote sensing letters—1 ieee intelligent informatics bulletin—4 ieee intelligent sysetms—1 ieee intelligent systems—3 ieee internet computing—3 ieee journal on selected areas in communications—1 ieee multimedia—1 ieee network—2 ieee photonics technology letters—1 ieee signal processing letters—5 ieee software—2 ieee transaction on industrial informatics—1 ieee transactions on aerospace & electronic systems—8 ieee transactions on automatic control—7 ieee transactions on biomedical engineering—3 ieee transactions on circuits & systems (part 1)—3 ieee transactions on circuits & systems for video technology—1 ieee transactions on circuits & systems ii express briefs—3 ieee transactions on communications—8 ieee transactions on computers—1 ieee transactions on consumer electronics—1 ieee transactions on dependable & secure computing—1 ieee transactions on education—2 ieee transactions on evolutionary computation—1 ieee transactions on fuzzy systems—6 ieee transactions on geoscience & remote sensing—3 ieee transactions on image processing—4 ieee transactions on industrial electronics—1 ieee transactions on information theory—13 ieee transactions on knowledge & data engineering—4 ieee transactions on multimedia—1 ieee transactions on neural networks—6 ieee transactions on parallel & distributed systems—1 ieee transactions on pattern analysis & machine intelligence—6 ieee transactions on power delivery—1 ieee transactions on robotics & automation—1 ieee transactions on signal processing—24 ieee transactions on software engineering—3 ieee transactions on speech & audio processing—2 ieee transactions on systems, man & cybernetics part a: systems & humans—5 ieee transactions on systems, man & cybernetics part b: cybernetics—4 ieee transactions on systems, man & cybernetics part c: applications & reviews—4 ieee transactions on vehicular technology—3 ieee transactions on very large scale integration (vlsi) systems—2 ieee transactions on wireless communications—3 ieee xplore—1 ieee/acm transactions on computational biology and bioinformatics—1 ieej transactions on electronics, information & systems—2 ieice transactions on fundamentals of electronics, communications & computer sciences—2 ieice transactions on information & systems—1 ima journal of mathematical control & information—1 australasian journal of information systems volume 14 number 2 june 2007 40 image and vision computing—1 info—1 infor: canadian journal of operational research & information processing— 1 informatica—3 information—1 information & communications technology law—1 information & management—2 information & security—1 information age—2 information and computation—4 information and organization—1 information and software technology—11 information fusion—1 information management—1 information processing and management—3 information processing letters—9 information resources management journal—1 information retrieval—2 information sciences—6 information society, the—2 information systems—1 information systems & e-business management—3 information systems control journal—1 information systems frontiers—3 information systems journal—3 information systems management—2 information systems research—1 information technology & management—3 information technology & people—2 information technology & tourism—2 information technology, theory & application—1 information visualization—1 information, technology and disabilities—1 informing science—2 integers: the electronic journal of combinatorial number theory—1 integrated computer-aided engineering—2 intelligent systems in accounting, finance & management—1 interacting with computers—4 interdisciplinary science reviews—1 internal medicine journal—1 international education journal—1 international journal for numerical methods in engineering—2 international journal human computer studies—4 international journal of accounting information systems—4 international journal of advanced manufacturing technology—1 international journal of advanced media & communication—3 international journal of advanced robotic systems—1 international journal of algebra & computation—1 international journal of approximate reasoning—2 international journal of asian management—1 international journal of bifurcation & chaos in applied sciences & engineering—1 international journal of business information systems—2 international journal of cognition & technology—1 international journal of computational intelligence—1 international journal of computational intelligence & applications—6 international journal of computer applications in technology—1 international journal of computer mathematics—2 international journal of computer systems science & engineering—10 international journal of computers & applications—1 international journal of computers & information sciences—1 international journal of computers & their applications—2 international journal of control—3 international journal of cooperative information systems—3 international journal of data warehousing & mining—1 international journal of digital accounting research—1 international journal of digital libraries—2 international journal of distance education technologies—2 international journal of distributed sensor networks—1 international journal of e-business research—1 international journal of education & development using information & communication technology—1 international journal of electronic business—2 international journal of electronic commerce—2 international journal of engineering intelligent systems for electrical engineering & communications—4 international journal of entrepreneurship and innovation—1 international journal of expert systems with applications—1 international journal of forecasting—2 international journal of fuzzy systems—1 international journal of general systems—2 international journal of geographical information science—1 international journal of healthcare technology & management—2 international journal of human-computer interaction—1 international journal of humanoid robotics—1 international journal of image & graphics—4 international journal of informatics—1 international journal of information & communication technology education—1 international journal of information management—1 international journal of information technology—2 international journal of information technology & decision making—3 international journal of information technology & management—1 international journal of intelligent systems—2 international journal of intelligent systems technologies & applications—2 international journal of internet marketing & advertising—1 international journal of knowledge, culture & change management—2 international journal of knowledge-based & intelligent engineering systems— 4 international journal of lateral computing—2 international journal of learning & intellectual capital—1 international journal of management & decision making—1 international journal of management & enterprise development—2 international journal of medical informatics—4 international journal of modelling & simulation—2 international journal of modern physics c: physics & computers—1 international journal of network management—2 international journal of operations & production management—1 international journal of operations & quantitative management—1 international journal of pattern recognition & artificial intelligence—2 international journal of production economics—1 international journal of production research—1 international journal of remote sensing—3 international journal of robotics research—5 international journal of smart engineering system design—1 international journal of software engineering & knowledge engineering—6 international journal of systems science—2 international journal of technology & human interaction—1 international journal of tomography & statistics (ijts)—1 international journal of uncertainty, fuzziness & knowledge-based systems— 3 international journal of wavelets, multiresolution and information—1 international journal of web engineering & technology—3 international journal of web information systems—2 international journal of web service research—1 international journal of web-based communities—1 international journal of wine marketing—1 international journal of wireless information networks—1 international journal of world wide web—2 international journal on artificial intelligence tools—2 international journal on e-learning—1 international journal on very large data bases (vldb)—3 international review of law, computers & technology—3 international scientific journal of computing—1 international transactions in operational research—3 internet research—2 investigative ophthalmology & visual science—1 investment management and financial innovations—1 israel journal of mathematics—1 it professional—1 jitta: an information systems journal—6 journal of advanced nursing—1 journal of aircraft—1 journal of applied non-classical logic—1 journal of applied probability—1 journal of applied system studies—2 journal of artificial intelligence research—3 journal of automata, languages & combinatorics—2 journal of battlefield technology—1 journal of bioinformatics & computational biology—1 journal of business ethics—1 journal of business research—1 journal of business venturing—1 australasian journal of information systems volume 14 number 2 june 2007 41 journal of chemical engineering of japan—1 journal of college student retention: research, theory & practice—1 journal of college teaching and learning—1 journal of combinatorial designs—1 journal of combinatorial theory, series a—2 journal of communications & networks—1 journal of composite materials—1 journal of computer and system sciences—1 journal of computer information systems—1 journal of computer mediated communication—1 journal of computer science & technology—2 journal of computer security—2 journal of contemporary business issues—1 journal of database management—3 journal of decision systems—3 journal of discrete algorithms—1 journal of electrical & electronics engineering australia—1 journal of electronic commerce in organizations—1 journal of electronic commerce research (online edition)—2 journal of electronics & information technology—1 journal of emerging technologies in accounting—1 journal of end user computing—2 journal of environmental psychology—1 journal of experimental algorithmics—1 journal of functional programming—2 journal of geodynamics—1 journal of geographical systems—1 journal of global information management—2 journal of global information technology management—2 journal of global optimization—2 journal of global positioning systems—2 journal of immunological methods—1 journal of industrial & management optimization—1 journal of informatics education research—1 journal of information & knowledge management—3 journal of information science—4 journal of information systems education—2 journal of information technology—10 journal of information technology cases & applications—1 journal of information technology education—2 journal of information warfare—3 journal of information, communication and ethics in society—1 journal of in-service education—1 journal of integrated design & process science—1 journal of intellectual capital—1 journal of intelligent & fuzzy systems—1 journal of intelligent manufacturing—1 journal of international development—1 journal of internet technology—2 journal of issues in informing science & information technology—11 journal of knowledge management practice—1 journal of logic and computation—2 journal of machine learning research—4 journal of management development—1 journal of management information systems—1 journal of manufacturing science & engineering—1 journal of materials processing technology—1 journal of medical systems—1 journal of navigation—1 journal of network & computer applications—1 journal of network industries—2 journal of object technology—6 journal of organizational & end user computing—1 journal of parallel & distributed computing—2 journal of petroleum science & engineering—1 journal of pharmacy practice & research—1 journal of philosophical logic—2 journal of photogrammetry & remote sensing—1 journal of public procurement—1 journal of research & practice in information technology—9 journal of security engineering—3 journal of small business management—1 journal of sociology—1 journal of sound and vibration—1 journal of strategic information systems—5 journal of symbolic logic—1 journal of systems & information technology—2 journal of systems and software—10 journal of systems architecture—1 journal of systems science & complexity—1 journal of systems science & systems engineering—1 journal of technology management & entrepreneurship—1 journal of telecommunications & information technology—1 journal of telemedicine and telecare—1 journal of the acoustical society of america—1 journal of the american medical informatics association—2 journal of the american society for information science & technology—9 journal of the australian & new zealand academy of management—1 journal of the energy institute—1 journal of tourism and cultural change—1 journal of travel medicine—1 journal of universal computer science—2 journal of urban planning & development—1 journal of visual languages & computing—3 journal of web engineering—1 jurnal teknologi electronics, control, telecommunications & information technology—1 knowledge and information systems—3 knowledge engineering review—1 knowledge management research & practice—1 knowledge-based systems—2 korean journal of computational & applied mathematics—1 korean society of mechanical engineers (ksme) international journal—1 language—1 language learning & technology—1 law, probability and risk—2 learning technology—2 lecture notes in artificial intelligence—52 lecture notes in computer science—94 lecture notes on networking & mobile computing—1 library hi tech—1 library management—1 libri: international journal of libraries & information services—1 linear algebra and its applications—1 literary and linguistic computing—1 logic journal of the igpl—2 logistics information management—3 machine learning—6 machine vision and applications—1 malaysian accounting review—1 management case study journal—1 management research news—1 managerial auditing journal—1 marketing and logistics—1 match communications in mathematical & in computer chemistry—1 mathematics of computation—1 medical & biological engineering & computing—1 medical journal of australia—1 mis quarterly—1 mis quarterly executive—2 multiagent and grid systems—1 multimedia tools and applications—1 nauchno-technicheskaya informatsiya (seriya 1)—1 networks—1 neural computation—2 neural computing & applications—1 neural information processing: letters & reviews—3 neural networks—1 neural, parallel and scientific computations—1 neurocomputing—6 nonlinear analysis—1 nordic journal of computing—1 nova economia—1 numerical algorithms—1 nurse researcher—2 omega: international journal of management science—1 online information review—2 operations research letters—1 optical engineering—1 optics communications—1 or insight—1 pacific journalism review—1 parallel algorithms and applications—2 parallel computing—2 australasian journal of information systems volume 14 number 2 june 2007 42 parallel processing letters—1 pattern analysis and applications—3 pattern recognition—4 pattern recognition letters—6 performance evaluation—2 personal and ubiquitous computing—3 perspectives: policy and practice in higher education—1 philosophical psychology—1 preventive medicine—1 problems of non-linear analysis in engineering systems—1 proceedings of the ieee—3 production planning and control—1 project management journal—1 prometheus—3 publications of the astronomical society of australia—1 qualitative research journal—1 random structures and algorithms—1 real-time imaging—1 reference services review—1 research and development in higher education series—1 review of pacific basin financial markets & policies—1 revue d'intelligence artificielle—3 robotics and autonomous systems—2 robotics and computer-integrated manufacturing—1 ruanjian xuebao (journal of software)—1 russian language journal—1 scan—1 science of computer programming—2 scientometrics—5 siam journal on computing—2 siam journal on control & optimization—2 sigcse bulletin inroads—1 sigmod record—1 signal processing—6 signal processing: image communication—4 simulation—2 singapore law gazette—1 small business economics—1 soft computing—4 software process improvement and practice—2 software quality journal—2 software systems and modelling—1 software testing, verification & reliability—2 software-practice & experience—5 south african computer journal—1 southern review—1 statistica sinica—1 statistics and computing—1 strategic change—1 studia logica—4 studies in learning, evaluation, innovation & development—2 supply chain management—4 surveillance & society—1 systemic practice & action research—1 systems & control letters—8 systems engineering—1 technological forecasting & social change—1 telecommunications & networking—1 telecommunications journal of australia—4 telecommunications policy—1 telemedicine journal and e-health—2 the arbitrator and mediator—1 the british accounting review—1 the british journal for the philosophy of science—1 the grounded theory review—1 the international journal of information & security—1 the journal of supercomputing—2 the journal of the american taxation association—1 the journal of the institute of image electronics engineers of japan—2 the mathematical gazette—1 the new review of information behaviour research—1 the new zealand journal of applied computing & information technology—2 the qualitative report—1 theoretical computer science—11 theoretical informatics & applications—1 theory and practice of logic programming—3 traffic—1 trans tasman surveyor—1 transactions in gis—2 transformations: region, culture and society—1 uneac asia papers—1 universal access in the information society—1 university of ottawa law & technology journal—1 virtual reality: research, development & applications—1 web intelligence & agent systems—1 webology—1 wireless networks—1 wseas transactions on acoustics & music—1 wseas transactions on biology & biomedicine—2 wseas transactions on circuits & systems—1 wseas transactions on communications—2 wseas transactions on computers—7 wseas transactions on electronics—1 wseas transactions on systems—3 yugoslav journal of operations research—1 zidonghua xuebao (acta automatica sinica)—1 ajis vol. 5 no. 2 may 1998 towards guidelines for development and management of transnational information systems angele cavaye1, paul mantelaers2, wander van de berg3, arre zuurmond4 'dept of information systems, university of southern queensland, toowoomba, australia 2dept of computer science, delft technical university, delft, the netherlands 3dept of systems engineering, delft technical university, delft, the netherlands 4dept of public administration, erasmus university, rotterdam, the netherlands abstract this paper focuses on transnational information systems (tis) that cross organisational as well as international boundaries. such systems are being built and used in increasing numbers, but very little attention has been paid to us in the literature. this paper describes an empirical study which gathered case study data from four tis projects. the data were used to describe problems encountered during tis development and management and to discuss solutions found for those problems. an interesting finding was that organisations tend to address difficulties by lowering the level of aspiration of the tis rather than addressing difficulties head-on. keywords: transnational information systems, is development, is management transnational information systems there is an increasing use of and demand for information systems that transcend national boundaries transnational information systems (its). on the one hand developments in information and communication technology now make it possible to support many processes and tasks that cross company and national boundaries. on the other hand economic and political developments create demand for transnational information systems. this demand is clearly evident within the european union in the context of a european market for persons, goods, capital and services. similar developments can be seen on a global scale due to increased intel-nationalisation. examples of tis include international payments systems, systems linking public administration agencies, international traffic control, and logistics systems linking different countries. development and management of tis is not straightforward. the interorganisational and international nature of these systems introduces various difficulties and problems. one major reason for this may be that the participants are independent in most aspects and autonomous with regard to their own proprietary information systems. another potential cause of difficulties is the international context which introduces cultural, legal and language difficulties among participants. most individual tis to date are being developed on an ad hoc basis. practical experience and empirical research show that many problems and solutions at first glance appear to be specific to a particular development effort, but often they are not unique at all. as the needs and opportunities for tis become more prevalent, the call for generalised knowledge and guidelines to develop and manage tis increases. organisations want to know how to avoid difficulties and, if problems should occur, how to address them. instead of continuing ad hoc development there is a need for structured do's and don'ts to aid organisations that may consider using, developing, managing, or otherwise participating in tis. a search through the literature revealed a lack of studies addressing the combined transorganisational and transnational is setting. there is, however, a stream of literature concerning information systems linking parts of multinationals located in different countries. several books on this subject have appeared containing articles highlighting the main drawbacks and difficulties when developing systems that cross national boundaries (deans and kane, 1992; palvia and palvia, 1994). previous work on information systems for multi-nationals highlights issues concerning any systems that cross national boundaries: problems with language, different cultures, different abilities in terms of it, differing sophistication of ft infrastructure and telecommunications, and legal differences. apart from research about global systems there is also a substantial body of writing about information systems crossing company boundaries. much of the early discussion about interorganisational systems (ios) focused on competitive benefits of developing ios (porter and millar, 1985) and on facilitating and inhibiting factors in the development process (reich and benbasat, 1990). more recently research has taken on a more thoughtful note, pointing out that not all ios provide benefits to all participants (webster, 1995), that ios may involve risks and conflict (kumar and van dissel, 1996), and that ios involve management of relationships among the participants 13 ajis vol. 5 no. 2 may 1998 (meier, 1995). previous research suggests that major difficulties concerning systems crossing organisational boundaries include problems of responsibility, co-ordination, sharing of costs and benefits, and concerns about competitive position. this paper describes a study which attempted to take some early steps towards guidelines for the development and management of tis. in view of the fact that very little existing research is available about tis, this study focused on gathering information from an empirical setting as a basis for theory building. the paper proceeds as follows. first, the preparation of the empirical study is described. then, four cases are introduced. main tis issues are summarised and then discussed. tis research issues tis difficulties and solutions to those difficulties can be categorised in a number of ways. this study used three dimensions for highlighting and distinguishing tis issues. one way of studying tis issues is to distinguish issues in the informational, organisational and institutional domains. parsons (1960) first discussed these distinct but complementary areas of responsibility and control; he argues that they are three quite separate domains within an organisation. we adopted this concept to categorise difficulties and potential solutions during the process of development and management of tis. parsons used the term 'technical domain' to denote the technical sub-organisation; we changed the term to 'informational domain' to underline the fact that this area of attention concerns data, procedures and people as well as the technical aspects of hardware, software and communications. organisational issues refer to the way in which individual organisations need to adapt organisational structures, processes, regulations, skills, standards and ways of working to conform to the tis. institutional issues concern policy identification, political or competitive power struggles, role models as well as the formulation and development of decision making arrangements among participating organisations in order to ensure that the new order is accepted, adhered to and continues to operate effectively. a second dimension for studying tis issues concerns the various phases of the systems development process from identification of the need for a tis through to the management of an operational tis. various authors provide different names for separate development phases, divide the system development process into different numbers of phases, and describe the development process with or without iteration and feedback loops. however, authors generally agree that (1) the need for a system should be identified and requirements have to be drawn up; (2) logical and physical design and building of the system has to take place; (3) the system has to be implemented; (4) the operational system has to be managed, updated and adapted over time. we used these four distinct phases to categorise difficulties according to stages of tis development. a third way of looking at the tis area is to determine whether tis issues lie in the interorganisational domain or the international one. previous literature in global systems and in interorganisational systems presents many issues which are relevant to either the international domain or the interorganisational one. our main concern was not merely to list those issues once again, but to derive from empirical research which type of issues are most dominant. we felt it was important to determine whether tis difficulties are caused primarily by the international nature of the system or by the fact that tis requires co-operation among and co-ordination of multiple autonomous organisations. would the same or similar difficulties arise if the system were developed on a national basis? is the international component a stumbling block? research method although there is no literature specifically addressing transnational information systems, there is substantial practical experience with such systems. within the public sector in the european union alone about 30 transnational systems are currently operational or under development (kroon, 1997). this experience and knowledge tends to remain with the organisations involved unless an attempt is made to collate and generalise the lessons learnt by individual organisations. for this reason it was decided to conduct an empirical study. case research was selected as the appropriate research method. carrying out a number of case studies would provide rich data which is important in the process of theory building (yin, 1994). observation of phenomena and relationships in a variety of settings forms the basis for conceptualisation and generalisation. hence, studying the development processes of existing tis, being guided by existing constructs, keeping an open mind to uncovering new relationships, and subsequently conceptualising from findings is a sound method for formulating guidelines for tis development and management. selection of cases for this exploratory study was based on several criteria. cases had to concern tis but had to be sufficiently dissimilar to provide a rich picture of the problem area. hence, cases were selected from different sectors. tis in the cases had to support a primary process in the organisations and cases had to involve systems which were operational even if only in an early version. at least one tis was sought which involved a non14 ajis vol. 5 no. 2 may 1998 automated system; this allowed the researchers to ascertain whether tis development difficulties are related purely to technology or not. cases need not be highly successful; in fact we preferred to have cases which had encountered or were facing difficulties. an additional criterion was the requirement that organisations had to be willing and able to participate in the study. a multi-disciplinary research team was put together consisting of experienced researchers from computer science, information systems, and management. a list of definitions, a case protocol and a framework for writing up individual cases was agreed upon. researchers went into the field studying 4 tis cases by collecting documentation, tracing tis development over time, interviewing informants who were or are involved with the tis, obtaining information from it informants and also from informants primarily concerned with the business processes supported by the tis. the findings reported here are the result of this exploratory study. the four cases four cases were studied: eucaris, transit, geis and taps. eucaris is a tis that enables personnel in different countries to have access to other national car and driving licence registration systems; the primary aim of eucaris is to lower the number of stolen cars being registered in another country as a legal vehicle. transit is the tis that is being developed to speed up and facilitate custom administration concerning goods that are imported into the european union and to which a suspense regime is applied; transit should also lead to more effective fraud detection and prevention. geis is the case concerning the collection and aggregation of international greenhouse gas emissions; the aim of the system is to monitor individual country's emissions which should help to stabilise emission levels. taps is the acronym for transnational ambulatory payment systems. this is an embedded case where four different systems were studied; each system enables private customers to make non-cash payments internationally. the aim of the system is to facilitate transborder payment processing. eucaris the incidence of car theft has increased alarmingly during the past 10 years. often stolen cars are taken across european borders to be re-registered (and thus white-washed) in another country. registration authorities are obliged to register a car unless they can be prove within a short period of time that the car is registered elsewhere and therefore is a stolen car. when in doubt about the legality of a car that is being offered for registration, authorities traditionally write, phone or fax to foreign registration authorities. this process is inadequate, as it is time-consuming and difficult to manage. computerised access to registration databases in other countries would facilitate the process of checking for legal registrations. it would be much faster, more accurate, and more likely to combat re-registration of stolen cars. the dutch car registration authority conceived the idea for building a tis to enable instantaneous, electronic searching and checking of car registration data across europe; the system was called eucaris (european car registration information system). very early on it became clear that it would not be possible nor necessary to integrate the registration systems of various countries into one large system. there is at present no single european car numbering system to identify individual cars europe-wide. also, current registration systems each use different data definitions, which complicates the possible development of a single system. in addition, individual registration authorities value their autonomy and a central registration system would threaten that autonomy. hence, it was decided to develop a relatively simple system: a common shell to provide a query interface to the separate systems of participating countries. eucaris has been operational since 1995. currently six countries are on-line (the netherlands, belgium, luxembourg, uk, hungary and poland); 14 other countries are preparing to go on-line; another 10 have enquired about participation. transit goods that are brought into the european union (eu) are subject to taxes and duties. since payment is not necessarily required at the point of entry, european customs officials have to keep track of goods moving within europe and, in particular, have to be able to trace what duties and how much taxes have been paid. building a tis to computerise this international customs process was first considered in the early 1980s. a properly working tis would facilitate tracking and tracing of goods; it would enable immediate comparison of goods and duties paid, speeding up the clearance process; it would produce readable, accurate and up-to-date information at any customs office. the private sector would also benefit because of faster clearance procedures. disagreements among different eu partners withheld a us from moving beyond the 'nice idea' phase. 15 ajis vol. 5 no. 2 may 1998 the current project to build a tis, called transit, is the third attempt at developing computerised procedures. this time a serious effort has been put into tis development: fraud problems have reached alarming proportions with fraud concerning goods in transit within the eu adding up to billions of us $. computerisation of the transit procedures has now been placed high on the list of priorities of the eu. edi messages have been designed and national gateways are being set up. in order to account for diversity in local systems and to respect autonomy of individual customs agencies no attempt has been made to integrate the various systems. instead, for each country one customs application host is built which connects the central tis system with local systems in various countries. in this way local systems and procedures need not be adapted to a single european-wide system; similarly, the diversity in local systems and procedures needs not affect the central tis. at the end of 1996 the transit project is in its pilot phase; customs authorities aim to have the system fully operational as soon as possible. geis the global community has become increasingly concerned with the emission of dangerous gases into the atmosphere and the effect of those gases on the environment. it is generally recognised that combating dangerous emissions should not be a unilateral effort from individual countries. sustainable reduction in emissions can only be achieved by commitment from and co-operation among countries. in the early 1990s both the eu and the world community at large (through the united nations) agreed to address the rise in greenhouse emissions. information is a vital ingredient in any effort to combat dangerous (greenhouse gas) emissions: information is required about the state of climatic change in different countries, about technologies to combat dangerous emissions, and about projects to counter the amount of and effects of dangerous emissions. this information has to be passed from individual countries to transnational organisations which collate and monitor the data. a separate organisation (within the un) has been set up to deal with information about emissions and climatic change. the system for international information exchange is non-computerised, but involves detailed agreements about the exact data to be passed on to the transnational organisation. guidelines have been written and accepted as the standard procedures and rules according to which each nation will collect and present international greenhouse gas information. not all countries are able to collect detailed information; hence, in the cases of developing countries standard procedures are simplified. even among western countries it is difficult to ensure that all countries adhere to standard reporting rules; monitoring is therefore an important function of the transnational organisation managing the system. taps banks have had to agree on a transnational system to process and complete payments made in one country by a customer holding accounts in another country. customers travel across borders and wish to obtain cash or to pay for products in foreign countries by debiting their account in the home country. such payments made by individuals in other countries are called ambulatory payments. in europe there are currently four systems in place to support ambulatory payments. the oldest system, introduced in the 1970s, is the eurocheque an internationally recognised cheque where the customer has to enter not only the amount payable, but also the appropriate denomination for payment. the second payment system is the internationally usable europass a magnetic strip card requiring pin identification at all times. the third system is the eurocard creditcard which requires signature and pin for identification. the fourth system has been introduced in 1996: a chipcard with magnetic strip. the plan is to phase out the earlier systems and concentrate on the more modem card-based systems. these four systems, although distinct, are processed in the same way by banks. individual countries have a single channel for international transactions so that all foreign currency processing is centralised. the organisation operating the central channel maintains links to the european-wide organisation europay which operates transnational card and eurocheque transactions. europay performs international clearing and links with national clearing houses in different countries. main tis issues the cases were compared along the three basic dimensions mentioned earlier: the informational, organisational and institutional domains; issues according various phases of development; issues categorised according to the interorganisational and international domain. findings of the cross case comparison are summarised below. 16 ajis vol. 5 no. 2 may 1998 issues according to the informational/organisational/institutional domains table 1 presents a summary of the problems faced by the four tis categorised according to the informational, organisational and institutional nature of the problem. the table also shows which cases encountered which difficulties. the summary in table 1 shows that development and management of tis concerns difficulties in all three categories of problems: there are informational as well as organisational and institutional problems. some of the problems are clearly created by and because of the tis; other problems exist regardless of the tis. table 1 informational, organisational and institutional tis problems category informational organisational institutional problem technological limitations to development insufficient local sophistication in terms of ability to collect data insufficient local sophistication in terms of it infrastructure or applications technical complexity of integrated system diversity of systems to be connected unequal sharing of costs and benefits lack of a transnational organisation lack of transnational organisation to manage tis problem to be addressed by tis is seen as low priority by (some) individual participants disagreement among various local organisations about responsibility for information (local power issue) lack of consensus about degree of centralisation fear for loss of autonomy legal procedures in various countries concerning data and privacy cultural/language differences between countries eucaris x x x x x transit x x x x x x x x geis x x x x x x x x taps x (eurocheque) x (creditcard) x (chipcard) x (3 systems) x (credit and chipcard) x (3 systems) x (chipcard) x (magnetic and creditcard) informational issues need not be permanent bottlenecks. difficulties like lack of sophistication in data collection abilities or lack of it infrastructure tend to be dissolved over time as individual organisations or countries develop their systems. similarly, technological limitations tend to be overcome as time passes and new developments become commercially available. diversity of systems among partners is also overcome by increasing standardisation of products. although technical and informational problems are real and occurred in three of the cases, they are not necessarily long-term difficulties. it should be noted that solutions for informational problems are not necessarily informational but can also be organisational or institutional. similarly, organisational and institutional problems can be addressed by technical solutions. organisational issues, like inequality in costs and benefits of tis and the lack of transnational bodies to develop and manage tis, are co-ordination and co-operation issues. co-operation of individual organisations is necessary for the concept of a meaningful tis to evolve and co-ordination is necessary for the development and management of the actual tis itself. in order to gain co-operation of individual organisations it is important to ensure benefits are shared or, at least, costs are borne by those that stand to gain most of the tis. the absence of transnational organisations inhibits fast or easy tis development; a transnational body is particularly valuable for co-ordinating day-to-day tis affairs. institutional problems tend to be present but latent prior to tis development; they often come to the fore during 17 ajis vol. 5 no. 2 may 1998 early discussions about development and management of the us. such problems are not necessarily directly concerned with the tis itself and may affect tis indirectly. clearly, existing institutional problems have to be addressed if the tis is to be successful. an example of an existing problem that affects tis indirectly is 'disagreement among various local organisations'. (potential) participants of the tis can have disagreements about data, technical matters or policy with parties outside the tis and such disagreements can affect the ability of the participants to participate fully. political struggles or business competition between tis participants or with outsiders are important determining factors for the success of the tis. a system that is sound from a technical point of view may turn out to be not feasible due to institutional issues. table 2 • issues during system life cycle phases life cycle phase identification designing and building implementation ongoing management and maintenance problems need for a tis identified, but not possible to translate into tis (transit first and second attempt) possible tis identified but too many organisational problems to begin actual development (transit second attempt) need for tis acknowledged, but fear for loss of autonomy (eucaris) need for tis acknowledged, but not seen as high priority (transit; geis) tis will lead to uneven sharing of costs and benefits (transit) tis is turning into a technically complex system (transit) diversity of systems that need to be linked (eucaris, taps) time/manpower for tis development difficult to find among participants (eucaris) different abilities of participants (in terms of it) do not allow implementation of full system (geis, transit) difficulty of ensuring that standards are enforced and tis is used properly (eucaris, geis) time/manpower for co-ordination (eucaris) long term integration into existing systems is necessary for long-term success of tis (eucaris, taps) expanding the system to include more tasks/actors (eucaris, taps) solutions delay plans for development indefinitely delay indefinitely opt for technical solution that is highly decentralised delay indefinitely (transit); translate common need for tis into local issue for participant (geis) (political) pressure to accept anyway superior project management discussion with various parties participate development opt for common interface and local gateway configuration experimental development method iterative development outsourcing of development different entry levels (geis) phased implementation (geis, transit) third party management of tis itself; monitoring committee from tis general management body (geis) outsourcing of day-to-day management of tis institutionalisation of new processes by local organisations is required discussion, prototyping consensus building 18 ajis vol. 5 no. 2 may 1998 issues during the various phases of the system development every system moves through a number of recognisable phases during the development process; each of the phases may cause and may have to address different problems. the problems (and the solutions) that were found in the four cases are summarised in table 2 according to the various system life cycle phases. system identification. difficulties in the identification phase appear to be inversely related to the apparent urgency of the problem that is to be addressed by the tis. when the issue is considered important and urgent, identification of and agreement about the concept of a tis is relatively easily reached. however, when the problem that tis addresses is not considered a high priority, difficulties seem to mount up and the tis does not get beyond the 'nice idea' phase. designing and building of the tis. difficulties at this phase of development are primarily technical in nature. the technical issues are made more complicated because of the great number of participants and their different objectives. the difficulties are addressed by finding a technical solution or by improved organisation of the development process. although the problems in this development phase might seem difficult, finding solutions to the problems did not seem a major issue in any of the cases. implementation. when the tis has been properly identified and has been agreed upon by participants, implementation is a feasible matter, although likely to be laborious and costly. in both the transit and the geis case there is large diversity among the participants when it comes to informational and technical sophistication and facilities. this clearly inhibits implementation of the full tis. management and on-going maintenance. management issues include ensuring that participants see to their local systems so that they interface correctly with the tis, enforcing standards and continuing use of the tis, developing new functionality and increasing the scope of the tis. eucaris, for example, is a tis that has reached this phase and faces the new challenge of keeping the earlier momentum going. for the taps case this implies further integration inside each bank and at the national and transnational operational body level. table 3 intel-organisational vs. international issues problem insufficient local sophistication in terms of it infrastructure or applications technical complexity of integrated system diversity of systems to be connected unequal sharing of costs and benefits disagreement among various local organisations about responsibility for information (local power issue) lack of consensus about degree of centralisation fear for loss of autonomy insufficient local sophistication in terms of ability to collect data lack of a transnational organisation to develop the tis lack of transnational organisation to manage tis legal procedures in various countries concerning data and privacy cultural/language differences between countries intel-organisational x x x x x x x x x x international x x x x x 19 ajis vol. 5 no. 2 may 1998 issues derived from intel-organisational or international setting difficulties encountered during tis development or management can be caused by the interorganisational nature of the tis or by the international component of the tis. table 3 lists and categorises tis problems from tables 1 and 2 according to interorganisational or international causes. international context. the usual difficulties of operating across national boundaries were evident in all cases. although there are international differences and potential difficulties when collaborating internationally, many organisations and the people within them accept the differences and are willing to work towards overcoming such barriers. in the eucaris case where legal difficulties inhibited germany from participating fully, the german car registration authority took steps to ensure the law was adjusted and the eucaris organisation accepted germany as a partner that would join as soon as legally possible. in the transit case documents were translated into various languages to ensure that partners would all have access to documents in their own or in a very familiar language. in some cases there was no transnational organisation, but the need for and importance of such an organisation was recognised in all cases and an organisation for dealing with transnational issues was established. in all, the international context does present difficulties for tis development and management, but those difficulties are recognised, can usually be overcome and are addressed directly. interorganisational context. the table above would suggest that there are many more problems due to the interorganisational context than to the international nature oftis. some of these are technical and systems issues which might be overcome fairly easy; other issues concern autonomy and responsibility of organisations and equality among organisations. interorganisational information exchange requires common decisions, consensus and commitment. if individual organisations feel their autonomy is threatened or they are not gaining benefits on a par with other organisations, then they can withdraw participation or can become a difficult partner in the tis. in each of the cases difficulties of this kind were found and each of these problems proved to be difficult to address. tis issues which emerged during the study apart from issues which we had anticipated and had specifically looked for in the cases, issues emerged during the course of the study as empirical evidence was collated. a major issue which emerged was the level of aspiration of the tis. under rational norms we would expect organisations to aim at maximising the opportunity offered by a tis, building a system to optimise efficiency and shared benefits. in practice, though, successful tis organisations show satisficing behaviour: adopting a tis which is acceptable or 'good enough'. tis development involves difficulties and tis participants address difficulties during the development stages by lowering the level of aspiration of the tis, be it in time constraints, in functionality, or in the cost/benefit ratio. from a systems development perspective, an 'optimum* tis requires high levels of co-operation among partners and involves consensus on many aspects of the system. in the context of co-operation among independent partners it is not always feasible nor desirable to achieve high levels of consensus and commitment. there are many potential disagreements and mismatches which may mean that the 'optimum' tis is never built and that a tis at a lower level of aspiration is developed and implemented instead. participants may have agreed to a diminished level of aspiration for the tis, but the initial problems tend to resurface at a later stage. once the 'simplified' system is operational, attempts are made to improve and enhance the system by adding functionality, adding further partners, increasing the scope of the tis (e.g. in the case of eucaris). at that moment the problems which led to a reduced level of aspiration in the first place can surface again. it would seem that reducing the level of aspiration at early stages of development may postpone the moment of facing difficulties with the tis, but it does not eliminate the problems. but facing the problems headon early in development may well lead to the failure, or at least serious delay, of the tis. two other issues need to be mentioned here. we classified tis issues into three domains: the informational, the organisational, and the institutional domain. both problems and solutions regarding tis can be classified that way, but problems in one domain are not necessarily solved by solutions from the same domain. often technical solutions are used to address organisational and institutional problems. the second issue concerns the making of trade-offs. development and management of tis requires the wisdom to make the right trade-off between desirability and feasibility of solutions. in each case participants had to try and find the right balance between technical possibilities, costs, organisational competencies, power, etc. this study was exploratory in nature and the results reported here are preliminary. further research could study other cases and could investigate tis cases in settings outside the european union. research could also involve 20 ajis vol. 5 no. 2 may 1998 investigators in different countries to study individual tis from a multiple country perspective. compiling and combining findings from multiple studies should enable us to formulate guidelines and concepts to aid tis development and management. references deans, p. c. and m.j. kane (1992) international dimensions of information systems and technology, pws, kent kroon, n. (1997) "europese informatiesystemen: grensverleggend?" phd-thesis, erasmus university rotterdam (in dutch), forthcoming kumar, k. and h. van dissel (1996) "sustainable collaboration: managing conflict and co-operation in interorganisational systems", mis quarterly, sept, 20(3), pp279-300 martin, j. (1989) information engineering, book i: introduction, prentice hall, englewood cliffs, new jersey meier, j. (1995) "the importance of relationship management in establishing successful interorganisational systems", journal of strategic information systems, 4(2), pp 135-148 miles, m. b. and a.m. huberman (1984) analysing qualitative data: a source book for new methods, sage publications, beverly hills palvia, s. and p. palvia (1994) global issues of information technology management, idea group publishing, hanisburg parsons, t. (1960) structure and process in modern societies, free press of glencoe, new york porter, m. and v.e. millar (1985) "how information gives you competitive advantage", harvard business review, july-august, pp!49-160. reich, b. h. and l. benbasat (1990) "an empirical investigation of factors influencing the success of customer oriented information systems", information systems research, 1(3), pp325-347 webster, j. (1995) "networks of collaboration or conflict? electronic data interchange and power in the supply chain", journal of strategic information systems, 4(1), pp31-43 yin, r. k. (1994) case study research: design and method, 2nd edition, sage publications, londo 21 ajis vol 5 no. 2 may 1998 opportunities for knowledge discovery in spatio-temporal information systems tamas abraham and john f. roddick advanced computing research centre school of computer and information science university of south australia the levels sa 5095 email: (abraham, roddick) @cis.unisa.edu.au abstract spatial information systems and their recent temporal extensions typically store large volumes of geo-referenced information. having such size, it becomes increasingly difficult to explore their contents with current querying techniques. in this paper, we examine how data mining methods can help users in die analysis of the contents of spatial and spatio-temporal information systems. we review existing spatial applications and investigate how they can be extended to deal with time. we also look at new, alternative methods that utilise the inherent structure of spatio-temporal information as well as its rich semantics to derive rules about changes and movement keywords knowledge discovery, data mining, spatial information systems, spatio-temporal data introduction information stored in digital knowledge repositories has never been so abundant. with the rapid advance in hardware technologies and the increased affordability of computers, more organisations are turning towards storing increasing amounts of data in their information systems. spatial databases, and their temporal extensions, are examples of the technology being used to support this. the benefits of the availability of detailed information can be seen in inter alia improved decision support, database integrity enforcement and query optimisation. the exploration of these huge data sets with existing querying techniques, however, remains a burden. to alleviate this problem and offer additional tools for analysis, data mining and knowledge discovery in databases provide techniques to extract useful, implicit information from large databases, combining methods from artificial intelligence, statistics and database research (fayyad, et al. 1996b). some of these techniques have been successfully used in applications dealing with transactional and relational data. on a lesser scale, spatial and temporal applications are also in existence (fayyad, et al. 1996a, koperski, et al. 1996). the comparatively small number of data mining techniques available for spatial information systems can be explained by not only the more widespread use of other data types, but the complexity of spatial and spatiotemporal data vs. relational data. spatial information systems (sis) deal with information about our surroundings, be that in the form of maps in geographic information systems (cis), aerial or satellite images in image processing systems, land parcels in land information systems (lis), or networks such as the telephone lines or power grids in automated mapping/facilities management systems (am/fm) (abel 1996). they store spatial in addition to aspatial data, often using proprietary representational techniques, commonly based on the two best known alternatives, the location-oriented raster or the spatial entity-oriented vector structure, which are based respectively on the field and object data models. for overviews on the latest developments and advances in spatial database research, refer to (abel 1996, guenther & buchmann 1990, giiting 1994, medeiros & pires 1994, paredaens 1995). with the introduction of the temporal dimension, spatio-temporal information systems can reason effectively on a variety of new problems, ranging from environmental monitoring and impact assessment, resource management, decision support, administration, real-time navigation, and transportation scheduling to data quality and integrity enforcement. this is achieved by the incorporation of a number of new features that surpass current sis capabilities. in particular, enhanced analytical power, process visualisation and the integration of multiple data sources are available (abraham & roddick 1996, langran 1992). analysis and decision support can be further aided by the inclusion of data mining tools. the rest of this paper is organised as follows. in the next section, we revisit briefly the achievements and progress in data mining in spatial information systems and discuss problems associated with exploring spatial data. in the following section, we investigate the application of current and new data mining techniques to existing spatio-temporal systems. finally, we present some conclusions and ideas for future research directions. ajis vol. 5 no. 2 may 1998 spatial data mining data mining is an emerging field of computer science that combines elements of artificial intelligence, statistics and database theory to find implicit information in large data sets. a typical discovery process, often guided by some form of background knowledge, uses a search algorithm modified to work on large databases to extract previously unknown patterns with a high level of statistical significance, or "interestingness". common data mining tasks include deriving the general characteristics of data (e.g. via rule induction), classifying them into different groups (e.g. constructing decision trees based on database contents), extracting association rules between attributes or objects, finding relationships between individual data items (e.g. causal or functional relationships), and detecting trends and deviations (chen, et al. 1996). methods that we shall use in later discussions include the attribute oriented induction technique (cai, et al. 1991) to mine characteristic and discriminant rules, the original association rule extraction algorithm (agrawal, et al. 1993) and its later extensions, and various clustering techniques, e.g. (ng & han 1994). for further information, please refer to the monographs (fayyad, et al. 1996c, piatetsky-shapiro & frawley 1991), or some of the existing overviews (han 1996, rainsford & roddick 1996). for the more interested readers, general discussions on the future of knowledge discovery also exist (fayyad, et al. 1996b, mannila 1997). some of the existing data mining techniques have already been applied to spatial information systems. in image databases, pattern matching and analysis have been used to detect volcanos on the surface of venus (bell, et al. 1994), or recognise cyclone formation (stolorz, et al. 1995). other similar work is overviewed in (fayyad, et al. 1996a). in geographic information systems, generalisation techniques have been employed to derive the characteristics of spatial and associated attribute data (lu, et al. 1993). interactive mining of spatial association rules has also been investigated (koperski & han 1995), but most attention so far has been paid to characterising spatial information by finding their distribution patterns using clustering techniques. clustering, in contrast to generalisation methods such as attribute oriented induction, does not require a priori knowledge (commonly conceptual hierarchies, see figure 1 for examples) to guide the discovery process. traditional clustering algorithms from statistics, however, prove inefficient for large data sets. therefore, several improvements and alternatives have been proposed in the literature (ester, et al. 1995, ng 1996, ng & han 1994, zhang, et al. 1996, wang, et al. 1997). for a more detailed overview of achievements in spatial data mining, please refer to (koperski, et al. 1996). university australia t^ a commerce science education engineering nt wa chemistry physics maths addaide flinders yorke fleurieu hills city plains figure 1 : example conceptual hierarchies in spite of these attempts, few applications are currently available for general use. the aforementioned systems for volcano and cyclone recognition, and some others described in (fayyad, et al. 1996a), including earthquake detection and star categorisation are specialised systems designed to solve specific (mainly scientific) problems. this contrasts with systems that operate on relational data, where commercially oriented applications dominate. it is not yet clear whether a spatial knowledge discovery system (kds) capable of dealing with a number of spatial discovery problems on several data representational formats will be available in the future or even be a practical object to build. because of the diversity of sis, it is foreseeable that each spatial kds will have to be finetuned to an underlying sis to some degree. this dependence on the data representation techniques used by the underlying system may prevent the development of a generic spatial knowledge discovery system. another problem is that although well-known methods for characterisation and associations can be used with virtually all data types, certain discovery techniques may only apply to specific. despite these restrictions, the existence of commonly applicable techniques such as the ones mentioned above means that the development of generic spatial data mining methods remains a realistic and worthwhile aim. the geominer prototype (han, et al. 1997) is one such attempt to house a variety of tools in a single system, with spatial data stored using the sand architecture (aref & samet 1991). finally, an important issue in data mining that is especially valid for the spatial domain concerns the quality of ajis vol. 5 no. 2 may 1998 data available for knowledge discovery. this has been recognised as having comparable significance to the data mining step in the knowledge discovery process (mannila 1997). some companies are finding now that missing or erroneous data may invalidate considerable investments in data warehouses and data mining tools. noise handling remains a typical area in data mining research (holsheimer & siebes 1994, quintan 1986). consequently, sis may need to be designed to facilitate data mining as opposed to current attempts that try to find ways of extracting new information from pre-compiled, possibly noisy and incomplete spatial and attribute data. although this complicates the design process and requires some pre-conceived ideas about what will be looked for in the new information system, current data mining techniques will perform better on kd designated sis and may help the emergence of new techniques for more powerful discoveries. spatio-temporal data mining in this section, we first look at some spatio-temporal models and other issues related to the development of spatio-temporal information systems. following that, we investigate the kinds of knowledge likely to be sought after with data mining tools, and describe how existing spatial knowledge discovery techniques can be extended to deal with spatio-temporal data. we also discuss two new ways that exclusively utilise the temporal aspects of this data. spatio-temporal information systems current spatial information systems store an up-to-date, but static record of our environment. they fail to capture its dynamic nature, which considerably limits their reasoning capabilities. to remedy the situation, several attempts have been made to incorporate time into geo-referenced information over the past few decades. time, location and theme have long been recognised as the building blocks of geographic data (sinton 1978), but only recently have we seen a variety of spatio-temporal models and working system implementations. basoglu and morrison (1978) were among the pioneers in designing a file structure for historical us county boundary data, but it was armstrong (1988) who first discussed alternative models to primitive archiving scenarios in detail. among them is the snapshot model which records full database states at progressive time points. this is further improved by a proposal to only store changes from previous versions. other well-known models include the space-time composite model that fragments a base map into smaller and smaller units with their own history as it undergoes change (langran & chrisman 1988), a time based method that uses event chains to record the evolution of thematic data for a given geographic area over time (peuquet & duan 1995), and a three domain model that represents semantics, space and time as separate, interlinked domains to describe geographic processes and phenomena (yuan 1996). alternative proposals found in the literature are models based on relational databases (rafaat, et al. 1994), the object-oriented paradigm (bonfatti & monari 1993, rojas-vega & kemp 1995, wachowicz & healey 1994, worboys 1992), logical versioning (medeiros & jomier 1994) and the temporal map set concept (beller 1991). all these models implement some, but not all, of the requirements of temporal cis discussed by some authors (al-taha & barrera 1990, armstrong 1988, easterfield, et al. 1991, hazelton 1992, kemp & kowalczyk 1994, roshannejad & kainz 1995, yeh & viemont 1992). indeed, the creation of a universal spatio-temporal model may not be a realistic goal. instead, modeling solutions concentrate on capturing change and movement in order to facilitate the answering of certain types of queries. the wider range of queries a system can effectively answer, the more general purpose, and thus more desirable, it can be considered. in addition to modeling, other database issues need to be addressed before functional spatio-temporal information systems become commercially viable. abstract spatio-temporal data types and relationship operators must be combined into space efficient data structures with fast indexing methods to satisfy querying requirements. query languages with both spatial and temporal operators and the underlying mathematical formalisms must be developed, with sophisticated visualisation tools to communicate dynamic processes to users. existing systems already implement and utilise some of these features. they can be based on available gis, such as the tempest temporal gis prototype which employs the model of event chains as mentioned above (peuquet & wentz 1994), or oogeomorph, which is a set of object-oriented classes (raper 1995). both solutions access existing, conventional gis via an appropriate interface while adding temporal capabilities. fully developed scientific databases, such as gaea, an object-oriented system to study global change (hachem, et al. 1992), and quest, a prototype built to validate techniques and demonstrate the feasibility of building information systems for atmospheric and earth science databases (muntz, et al. 1995), go further by having their own spatio-temporal data types, querying and display operators. in addition, temporal extensions of existing tool sets such as the one described by story & worboys (1995) can help automate the creation of spatio-temporal applications. for more information, refer to (langran 1992), which provides a detailed discussion on temporal gis requirements and modeling techniques. a survey of papers spanning other spatio-temporal issues and new ajis vol. 5 no. 2 may 1998 spatio-temporal model proposals can also be found in (abraham & roddick 1996). spado-temporal rules when we consider mining spatio-temporal data, two questions must first be answered: what to mine and where to mine it, i.e., what kind of information we intend to extract from the spado-temporal information system and what will the format of this database be? although we emphasised that the development of useful techniques are of a high priority, we first briefly discuss the platforms currently available for implementation and testing purposes. object-oriented databases offer probably the best choice to represent spatio-temporal information: they convey its hierarchical nature and allow histories of objects to be recorded within the objects. however, efficiently querying object-oriented databases remains problematic, and they have not yet gained the general acceptance relational databases enjoy. relational databases, on the other hand, are often coupled with proprietary representations in spatial information systems, which makes the development of discovery algorithms difficult. some of the architectures preferred by research groups also have limitations. the sand architecture used to mine spatial characteristic and association rules provides operators to handle spatial data, but it does not directly handle time. temporal extensions of the public domain grass cis (neteler 1997) are available, but at present purely relational spatio-temporal models could still be the best choice for a mining platform. rafaat, et al. (1994) and beller (1991) both proposed architectures that enable the use of existing knowledge discovery algorithms with only minor modifications on vector and gridbased representations, respectively. after selecting an appropriate platform, we must determine the kind of information we intend to look for in the data. the availability of temporal information introduces the possibility of mining for new types of rules that complement existing ones. in particular, we are interested in capturing changes and movement. we shall first discuss two new types of rules that we can use for this purpose, then describe extensions to spatial data mining methods that also deal with temporal information. (a) meta-rules in our definition, meta-rules are rules derived from comparing existing rule sets that were generated at different times on the same domain. thus, spatio-temporal meta-rules describe changes in rule sets obtained from consecutive spatial snapshots of the data in the spatio-temporal information system. in this way, meta-rules can characterise the evolution or constancy of existing patterns, which is useful in predicting future trends in the modelled environment. survey data is one example for information that becomes available after regular collections. each data set can be mined for patterns to yield possibly useful rules about the environment at collection times. meta-rules can then enhance analysis by providing further insight into the changes between surveys. figure 2 illustrates how spatio-temporal data can be mined at selected time points that yield snapshot rule sets which are then used to produce meta-rules. spati^-tenlporal data set j meta-rules figure 2: spatio-temporal meta-rule mining using snapshots the typical meta-rule extraction process consists of four steps (abraham & roddick 1997). select and mine individual snapshots and collect the associated rules into separate rule sets, which are also timestamped for identification purposes. "normalise", i.e., if necessary, convert the contents of the snapshot rule sets into a consistent format to facilitate further examination. this may mean, for example, that the components of rules are put into a tabular format. a general rule reading if a then b with probability p can thus become a three-column table with separate columns for a, b and p. compare the contents of two rule sets and separate the results into four groups: new, expired, unchanged and changed rules. to determine which category a particular rule belongs to, we use user-defined difference ajis vol. 5 no. 2 may 1998 measures. these can tell us that a rule in the later rule set belongs to the new category because it did not exist in the earlier set, or that another rule in the earlier set belongs to the expired category because it no longer exists in the later set, and so on. for example, in the above if a the b with p expression, we say that the combination of a and b defines a particular rule and p is selected as the difference measure. thus, if a particular rule if a] then bj with pi exists in the first rule set and changes to if a] then bj with p2 in the second, then it will belong to the changed category. similarly, if the rule if a2 then 82 with p$ exists in the earlier rule set but not in the later one in its original or modified form, the rule will be placed in the expired category. _ conclude the meta-rule mining process by producing general descriptions for the contents of each of the four (or possibly some selected, e.g. new and expired) categories. this step constitutes the second level of abstraction described by the term meta-rule. (b) evolution rules spatio-temporal evolution rules are derived from spatio-temporal evolution descriptions that represent changes and movement of spatial objects and phenomena directly. an evolution description relates to a specific entity or group of entities, e.g. a forest fire or an animal species, and describes their spatial and attribute evolution over time. because the information being described is not generally available explicitly in temporal sis, the first step in producing evolution rules is the extraction of these primitive evolution descriptions from the data. this is in contrast to meta-rule mining, where pre-compiled rule sets are compared. investigating the relationships between evolution descriptions can reveal, for example, cause and effect relationships and associations. for example, the spread of a fire can be associated with changes in wind direction, while animal movement may be affected by temperature changes or declining or increasing levels of rainfall in habitat areas or changes in sea levels. thus, initial movement and change descriptions can be processed into rules to produce meaningful conclusions. the derivation of these rules, however, is a more complex problem than meta-rule mining and a number of subproblems must be further investigated. among them, the selection of predicates used in the initial evolution descriptions and the specification of techniques used to produce the final rules are some of the important ones and are currently being examined. (c) spatio-temporal generalisation the attribute oriented induction technique of (cai, et al. 1991) is a form of generalisation using conceptual hierarchies as background knowledge in the discovery process (see figure 1 for some example hierarchies, including a spatial one). the induction technique provides a general characterisation of a selected group of items based on the commonness of their attribute values, e.g. the description middle-aged male professors earn more than $50,000 a year can be produced from a university staff database. contrasting attribute values for similar item groups in separate collections, on the other hand, can be used to derive discriminatory information that can be used for comparisons, e.g. the salaries of professors in two separate university departments. lu, et al. (1993) provides a spatial extension to the induction technique and further develops the generalisation concept by allowing the acquisition of characteristic information in two different ways. both techniques are suited to analyse cis data. in spatial-data-dominant generalisation, first spatial regions are obtained by ascending a spatial conceptual hierarchy, then attribute information is generalised for each region. this may give us general descriptions of the temperature in certain regions of a map area, e.g. the temperature in south australia is mild. in nonspatial-data-dominant generalisation, first attribute data is generalised for all individual areas surveyed, then adjacent regions with the same characteristics are merged, yielding an overall picture of the spatial distribution of generalised attribute values. thus, for example, if several small counties in a state have a mild temperature, they could form a 'supercounty' within the state that is joined together solely because of the shared temperature characteristic. both of these approaches can be used and are considered to yield snapshot characteristics in a temporal sis. for mining characteristic rules that incorporate time, we have two options, which depend on the model employed by the system. _ if snapshots are extracted for different times as described above, meta-rules can be used to describe changes between snapshot rule sets as in figure 2. * in the spatial-data-dominant case, this means trends in attribute value changes can be observed, although sometimes these changes may not be detected if there are only small variations in the attribute values over time. this happens because the generalisation process can 'smooth out' deviations between layers by using high level concepts for descriptions. for example, a minor increase in temperatures over time may still be classified as mild in both the original and new ajis vol. 5 no. 2 may 1998 description. * in the nonspatial-data-dominant case, meta-rules can describe changes in spatial areas, such as the spread of a fire or disease. in our view, this is a useful approach in many .circumstances, because it enables the detection of environmental changes such as deforestation, increases or decreases in the levels of precipitation, or the spread of urban areas. as an example, consider an orchard sectioned according to a spatial hierarchy. if the occurrences of a certain tree disease are registered regularly over time, the affected areas for each survey can be found and meta-rules can be constructed to observe changes in those areas. the second approach is applicable only to representational techniques where full histories of individual objects are preserved. this means that temporal intervals denote the existence of object states. to explore such data, the use of temporal generalisation is an option. at present, however, very little information is available on how to effectively generalise temporal intervals (rainsford & roddick 1997), and thus only hypothetical discovery guidelines can be given here. * in the spatial-data-dominant case, for each region generated, temporal generalisation followed by attribute generalisation can be performed. this produces intervals for each region with an associated attribute concept that describes prevailing conditions for the region in a given period. * similarly, in the nonspatial-data-dominant case, temporal generalisation followed by attribute generalisation can precede the region merging process. this yields intervals for each of which attribute data is generalised and regions with similar characteristics are joined. (d) clustering clustering techniques used in spatial databases organise data into groups based on the similarity of their spatial features, without requiring any a priori knowledge, by mostly employing a distance based similarity measure. they produce a number of structures (clusters) comprising points or regions that represent spatial entities. once these are attained, another learning algorithm can be applied to establish, for example, characteristic details of these clusters. alternatively, attribute values of spatial objects can be generalised first and associated spatial clusters described by the same general attribute concepts may be found (ng & han 1994). time is another attribute in a multi-dimensional data space, possessing magnitude, and can be easily clustered on. the information attained by temporal clustering can be useful in e.g. detecting that certain spatial rules apply during different temporal intervals. for example, increases in a population can lead to an increase in the space taken by the population but only during spring. spatial clustering and attribute generalisation can thus be combined with temporal clustering, but will produce different results depending on the order in which they are applied to the data. below we give some examples (figure 3 illustrates ways the three techniques may be combined ). in a cadastral database, spatial clusters of a certain building type may be produced, followed by clustering on a temporal attribute such as ownership. the resulting information relates to periods of frequent sales of the pre-selected building type. _ if we are interested in sales patterns in general in the same cadastral database, performing a temporal clustering operation first followed by either spatial clustering or attribute generalisation would produce periods of increased sales activity whilst pinpointing the areas and/or building types involved. a further possibility is to use temporal clustering in conjunction with a "hashing function" that divides data into groups based on a selected criteria, e.g. day of month or time of day, enabling clusters to be found in derived (or a function of) data. for example, finding temporal clusters in the times of day for automatic teller machines can give us congestion periods, which can be further processed either to find whether withdrawals, deposits or transfers dominate at these times, or to determine the spatial distribution of the machines involved. clusters of spatial objects can also be used in conjunction with other temporal data mining techniques, such as temporal generalisation and meta-rules. some spatial objects, such as certain buildings, volcanos, etc., can be viewed to be spatially fixed over a long period of time. if clusters of these objects are established, it becomes possible to temporally generalise some of their attributes to characterise their behaviour over time. for other objects, the position of some spatial clusters may slowly change over time in a fashion that the identification of each cluster remains possible, such as forested areas. in this case, meta-rules may be found to describe their movement or change in size. 1 temporal clustering is not necessarily limited to spatio-temporal information. applying it to transaction databases, e.g. department store sales or bank atm (automatic teller machine) records could detect patterns of purchases/usage useful in planning and management ajis vol. 5 no. 2 may 1998 original data set tc ag ag sc ag tc sc tc tc temporal clustering sc spatial clustering ag attribute generalisation figure 3: spatio-temporal clustering (e) spatio-temporal associations this special kind of probabilistic dependency information between attribute values within records is especially useful for management purposes (agrawal, et al. 1993). formally, it is described by the formula x t> y (c%, s%), meaning that the occurrence of x in a record implies the presence of y with a confidence of c%, having at least s% of the data items supporting this claim. for example, a rule extracted from a supermarket database reading bread t? butter (90%, 15%) suggests that 90% of purchases that include bread also include butter, with 15% of all records having recorded a combined bread and butter purchase. in practice, associations with a high level of confidence and at least a moderate amount of support are searched for. support can be a relatively low percentage if the data set is large. the original rule extraction algorithm designed for the analysis of transaction records in (agrawal, et al. 1993) has since been both improved and extended to different domains, with qualitative, quantitative, negative, cyclic, fuzzy, multi-level and generalised rules being searched for (agrawal & srikant 1994, han & fu 1995, koperski & han 1995, ozden, et al. 1998, savasare, et al. 1998, srikant & agrawal 1995, srikant & agrawal). the spatial association extraction algorithm discussed by koperski & han (1995) searches for generalised associations between spatial objects, or spatial objects and attributes. rules are expressed by spatial and nonspatial predicates, e.g. schools are close_to parks. thus, they are an improvement over traditional associations with predicates now used in the antecedent and consequence. typically, conceptual hierarchies are employed in the generalisation of spatial objects and predicates. this enables the finding of multi-level associations as both the above example and one of its possible lower level versions, primary_schools are close_to parks_with_playgrounds can be a strong rule. similar to this spatial approach, we can think of purely temporal associations as rules extracted from temporal data and expressed by the use of at least one temporal predicate, e.g. car oil change precedes tune-up. thus, a spatio-temporal association rule would comprise both spatial and temporal predicates. a possible method to derive these rules is by processing evolution descriptions that may already contain such predicates, as we discussed before. this enables us to find patterns such as the convergence of events in a spatial location or the seemingly unrelated movements of spatial objects caused by some common factor. some possible examples include observing large numbers of seals near the antarctic during summer or the movements of fishing ships towards a particular reef after coral spawning. conclusions the progress in compiling spatio-temporal data sets in spatio-temporal information systems provides new opportunities in the analysis and prediction of trends in geo-referenced information. data mining offers techniques for this purpose. in this paper, we examined the extension of established spatial discovery techniques in the spatio-temporal domain and discussed new ones that exploit the temporal aspects of this data. consequently, if fully developed and integrated into sis, they offer tools for more detailed analysis capable of detecting previously unknown regularities in data. meta-rules combined with spatial data mining techniques are suited for system implementations storing layered temporal information, such as gis with survey data or a satellite image database. evolution rules can be utilised in systems with more sophisticated temporal models that are continually updated when changes occur in the environment. the development of these tools, however, are ajis vol. 5 no. 2 may 1998 dictated by the querying requirements of the users of spatio-temporal information systems. the identification of these requirements thus remains important in helping to accommodate data mining techniques within stis. our current research concentrates on the development of generic knowledge discovery tools to be used on spatiotemporal data. by incorporating these tools into various types of stis, we are aiming to provide users with enhanced analytical and data management capabilities. to this effect, we shall define a framework for mining evolution and meta-rules, and test their use in a prototype spatio-temporal knowledge discovery system. references abel, d. j. (1996) "what's special about spatial?", proceedings of 7th australian database conference, melbourne, australia, january 29-30. abraham, t. & roddick, j. f. (1997) "discovering meta-rules in mining temporal and spatio-temporal data", proceedings of the 8 international database workshop, hong kong, july 29-31. abraham, t. & roddick, j. f. (1996) survey of spatio-temporal databases, technical report cis-96-011, university of south australia, adelaide. agrawal, r., imielinski, t. & swami, a. (1993) "mining association rules between sets of items in large databases", proceedings of the 1993 international conference on management of data (sigmod 93), washington, d.c., may 26-28. agrawal, r. & srikant, r. (1994) "fast algorithms for mining association rules", proceedings of vldb 1994, santiago, chile, september 12-15. al-taha, k. k. & barrera, r. (1990) 'temporal data and gis: an overview", proceedings of gis/lis 90, anaheim, california, november 7-10. aref, w. & samet, h. (1991) "extending a dbms with spatial operations", proceedings of 2nd symposium on spatial database systems, zurich, switzerland, august 28-30. armstrong, m. p. (1988) 'temporality in spatial databases", proceedings of gis/lis 88, san antonio, texas, november 1988. basoglu, u. & morrison, j. l. (1978) "an efficient hierarchical data structure for the u.s. historical county boundary data file", harvard papers on gis, vol 4, pp 1-21. bell, d. a., anand, s. s. & shapcott, c. m. (1994) "database mining in spatial databases", proceedings of the international workshop on spatio-temporal databases. beller, a. (1991) "spatial/temporal events in a gis", proceedings of gis/lis 91, atlanta, georgia, october 28-november 1. bonfatti, f. & monari, p. d. (1993) "spatio-temporal modeling of complex geographical structures", proceedings of the d7ep tc5avg5.il working conference on computer support for environmental assessment (cseia 93), como, italy, october 6-8. cai, y., cercone, n. & han, j. (1991) "attribute-oriented induction in relational databases", g. piatetskyshapiro & w. j. frawley (eds.) knowledge discovery in databases, aaai press/mit press. chen, m.-s., han, j. & yu, p. s. (1996) "data mining: an overview from database perspective", ieee transactions on knowledge and data engineering, december, vol 8, pp 866-883. cheung, d. w., ng, v. t. & tarn, b. w. (1996) "maintenance of discovered knowledge: a case in multi-level association rules", proceedings of the second international conference on knowledge discovery & data mining, portland, oregon, august 2-4. easterfield, m., newell, r. g. & theriault, d. (1991) "modelling spatial and temporal information", proceedings of egis 91, brussels, belgium, april 2. ester, m., kriegel, h.-p. & xu, x. (1995) "knowledge discovery in large spatial databases: focusing techniques for efficient class identification", proceedings of the 4th international symposium on large spatial databases (ssd 95), portland, maine, august 6-9. fayyad, u., haussler, d. & stolorz, p. (1996a) "kdd for science data analysis: issues and examples", proceedings of the second international conference on knowledge discovery & data mining, portland, oregon, august 2-4. fayyad, u., piatetsky-shapiro, g. & smyth, p. (1996b) "knowledge discovery and data mining: towards a unifying framework", proceedings of the second internationa] conference on knowledge discovery & data mining, portland, oregon, august 2-4. fayyad, u. m., piatetsky-shapiro, g., smyth, p. & uthumsamy, r. (1996c) advances in knowledge discovery and data mining, aaai press/mit press, menlo park. guenther, o. & buchmann, a. (1990) "research issues in spatial databases", sigmod record, december, vol 19, pp 61-68. 10 ajis vol. 5 no. 2 may 1998 guting, r. h. (1994) "an introduction to spatial database systems", vldb journal, october, vol 3, pp 357399. hachem, n. i., gennert, m. a. & ward, m. 0. (1992) "a dbms architecture for global change research", proceedings of isy conference on earth and space science, pasadena, california, february. han, j. (1996) "data mining techniques", proceedings of acm-sigmod 96 conference, montreal, canada, june 4-6. han, j. & fu, y. (1995) "discovery of multiple-level association rules from large databases", proceedings of vldb 1995, zurich, switzerland, september 11-15. han, j., koperski, k. & stefanovic, n. (1997) "geominer: a system prototype for spatial data mining", proceedings of the 1997 acm-sigmod conference on the management of data, tucson, arizona, may 12-15. hazelton, n. w. j. (1992) "beyond the 2-d map: a new metaphor for multi-temporal 4-d cis.", proceedings of gis/lis 92, san jose, california, november 10-12. holsheimer, m. & siebes, a. p. j. m. (1994) data mining: the search for knowledge in databases, technical report cs-r9406, cwi, the netherlands. kemp, z. & kowalczyk, a. (1994) "incorporating the temporal dimension in a gis" m. f. worboys (eds.) innovations in gis, taylor & francis. koperski, k., adhikary, j. & han, j. (1996) "spatial data mining: progress and challenges survey paper", proceedings of 1996 acm-sigmod workshop on research issues on data mining and knowledge discovery, montreal, canada, june 1996. koperski, k. & han, j. (1995) "discovery of spatial association rules in geographic information databases", proceedings of the 4th international symposium on large spatial databases (ssd 95), portland, maine, august 6-9. langran, g. (1992) time in geographic information systems, taylor & francis, london. langran, g. & chrisman, n. r. (1988) "a framework for temporal geographic information", cartographica, vol 25, pp 1-14. lu, w., han, j. & ooi, b. c. (1993) "discovery of general knowledge in large spatial databases", proceedings of the 1993 far east workshop on gis (ffigis 93), singapore, june 21-22. mannila, h. (1997) "methods and problems in data mining", proceedings of the international conference on database theory, delphi, greece, january 8-10. medeiros, c. b. & jomier, g. (1994) "using versions in gis", proceedings of dexa 94, september 7-9. medeiros, c. b. & pires, f. (1994) "databases for gis", sigmod record, march, vol 23, pp 107-115. muntz, r., shek, e. & zaniolo, c. (1995) "using ldl-hfor spatio-temporal reasoning in atmospheric science databases" r. ramakrishnan (eds.) applications of logic databases, kluwer academic. neteler, m. (1997) introduction to grass gis software, institute of physical geography and landscape ecology, university of hannover, germany. ng, r. t. (1996) "spatial data mining: discovering knowledge of clusters from maps", proceedings of 1996 acm-sigmod workshop on research issues on data mining and knowledge discovery, montreal, canada, june 1996. ng, r. t. & han, j. (1994) "efficient and effective clustering methods for spatial data mining", proceedings of vldb 1994, santiago, chile, september 12-15. ji ozden, b., ramaswamy, s. & silberschatz, a. (1998) "cyclic association rules", proceedings of the 14 international conference on data engineering (icde "98), orlando, florida, february 23-27. paredaens, j. (1995) "spatial databases, the final frontier", g. gottlob & m. y. vardi (eds.) database theory, springer-verlag. peuquet, d. & wentz, e. (1994) "an approach for time-based analysis of spatiotemporal data", proceedings of the 6th international symposium on spatial data handling, edinburgh, scotland, september 5-9. peuquet, d. j. & duan, n. (1995) "an event-based spatiotemporal data model (estdm) for temporal analysis of geographical data", international journal of geographical information systems january, vol 9, pp 7-24. piatetsky-shapiro, g. & frawley, w. j. (1991) knowledge discovery in databases, aaai press/mit press, menlo park. quinlan, j. r. (1986) "the effect of noise on concept learning", michalski, r. s., carbonell, j. g. & mitchell, t. m. (eds.), machine learning, an artificial intelligence approach, vol 2, morgan kaufmann, san mateo, california. rafaat, h., yang, z. & gauthier, d. (1994) "relational spatial topologies for historical geographical information", international journal of geographical information systems, march, vol 8, pp 163-173. 11 ajis vol 5 no. 2 may 1998 rainsford, c. p. & roddick, j. f. (1997) "the attribute oriented induction of rules from temporal interval data", proceedings of the 8 international database workshop, hong kong, july 29-31. rainsford, c. p. & roddick, j. f. (1996) a survey of issues in data mining, technical report cis-96-006, university of south australia, adelaide. raper, j. (1995) "making cis multidimensional", proceedings of the joint european conference and exhibition on geographical information (jeg-ci95), the hague, the netherlands, march 26-31. rojas-vega, e. & kemp, z. (1995) "an object-oriented data model for spatio-temporal data", proceedings of the ninth annual symposium on geographic information systems, vancouver, canada, march 2730. roshannejad, a. a. & kainz, w. (1995) "handling identities in spatio-temporal databases", proceedings of acsm/asprs 1995 annual convention and exposition technical papers, charlotte, north carolina, savasare, a., omiecinski, e. & navathe, s. (1998) "mining for strong negative associations in a large database of customer transactions", proceedings of the 14 international conference on data engineering (icde '98), orlando, florida, february 23-27. sinton, d. f. (1978) "the inherent structure of information as a constraint to analysis: mapped thematic data as a case study", harvard papers on gis, vol 7, pp 1-17. srikant, r. & agrawal, r. (1995) "mining generalized association rules", proceedings of vldb 1995, zurich, switzerland, september 11-15. srikant, r. & agrawal, r. (1996) "mining quantitative association rules in large relational tables", proceedings of the 1996 acm-sigmod conference on the management of data, montreal, canada, june 4-6. stolorz, p., nakamura, h., mesrobian, e., muntz, r. r., shek, e. c., santos, j. r., yi, j., ng, k., chien, s.-y., mechoso, c. r. & farrara, j. d. (1995) "fast spatio-temporal data mining of large geophysical sets", proceedings of the 1st international conference on knowledge discovery and data mining, montreal, canada, august 20-21. story, p. a. & worboys, m. f. (1995) "a design support environment for spatio-temporal database applications", a. u. frank & w. kuhn (eds.) spatial information theory: a theoretical basis for gis, springer-verlag. wachowicz, m. & healey, r. g. (1994) 'towards temporality in gis", m. f. worboys (eds.) innovations in gis, taylor & francis. wang, w., yang, j. & muntz, r. (1997) "sting: a statistical information grid approach to spatial data mining", proceedings of vldb 97, athens, greece, august 26-29. worboys, m. f. (1992) "object-oriented models of spatiotemporal information", proceedings of gis/lis 92, san jose, california, november 10-12. yeh, t. s. & vie"mont, y. h. (1992) 'temporal aspects of geographical databases", proceedings of egis 92, munich, germany, march 23-26. yuan, m. (1996) "incorporating human conceptualizations into gis representation to support spatiotemporal queries", personal communication, november. zhang, t., ramakrishnan, r. & livny, m. (1996) "birch: an efficient clustering method for very large databases", proceedings of 1996 acm-sigmod workshop on research issues on data mining and knowledge discovery, montreal, canada, june 1996. 12 ajis vol. 5 no. 2 may 1998 the dynamics of establishing organizational web sites: some puzzling findings celia t. romm & jeanne wong university of wollongong department of business systems wollongong. nsw, 2522. australia telephone no: +612 4221 4043 fax no:+612 4221 4474 email: celia_romm@uow .edu.au email: jeanne_wong@uow.edu.au abstract a survey of the currently available literature on diffusion of web technologies reveals that it is at a very early stage of development. as a result, it is difficult, if not impossible, to use existing models to describe or explain the process of establishing organizational web sites. one of the few theoretical models that can be used as a starting point for research in this area is the jarvenpaa and ives (1996) model. the model consists of five propositions that describe what the authors see as the essential aspects of the diffusion of web technology. the purpose of this paper is to compare data collected in an australian university to the propositions of the jarvenpaa and ives model. the comparison demonstrates that there is a need for a more generic model which would explain the diffusion process of web technologies in diverse organizational and cultural contexts. the paper is concluded with a discussion of the political, industrial, and cultural reasons that may explain the differences between the dynamics in our case and those described in the jarvenpaa and ives model. keywords web technologies, web sites, electronic commerce, virtual communities, organizational transformation, organizational strategy, australian case introduction recent publications herald web technology as a revolutionary or transformational technology, i.e., one with the potential to dramatically change the way that organizations conduct their business (bickel 1996; kalakota & whinston 1996; rayport & sviokla 1995). some of the areas that are seen as most likely to undergo radical change as a result of the introduction of web technologies to business organizations are: (1) informational structures; (2) management strategies; (3) distribution channels; and (4) product mix. a survey of the literature on web technologies reveals that even though it is growing at a nigh rate, much of it is still dedicated to the technical aspects of the diffusion process such as how to connect to the internet (lejeune & duntemann 1995) and how to use web browsers such as mosaic, netscape and microsoft internet explorer (brown et al 1995; krol 1994; fitter & minato 1996). other areas within the growing body of research on web technologies include the study of marketing implications of the web (savola, westenbroek & heck 1995), and design issues (emery 1996; sterne 1995). most importantly for our purposes, there is only a handful of studies that actually describe the process of diffusion of organizational web technology within organizations. one of the pioneering studies in this area was recently conducted by jarvenpaa and ives (1996). in their research, the authors did not only describe in detail how two organizations in the us developed their web technology capabilities, but also made a series of propositions on how this process can be theoretically conceptualized. the objective of this paper is to use the jarvenpaa and ives (1996) study as a starting point for a more general model capable of describing the diffusion of web technologies in organizations. to achieve this goal, this paper goes through the following steps. first, based on the jarvenpaa and ives (1996) model, five propositions that have been identified by these authors as relevant to the diffusion of web technologies in organizations are outlined. second, based on data collected in australia, the degree to which the jarvenpaa and ives' propositions can be generalized across industries and cultures is assessed. third, given that there is a gap between the jarvenpaa and ives' propositions and the australian case, an alternative approach, the establishing organizational web sites (eows) model, is presented. the paper is concluded with a discussion of directions for future research and methodological implications from the eows model. literature review before we consider the jarvenpaa and ives (1996) paper, a few words about the diffusion of information technology innovation in general are in order. mcfarlan and mckenney (1982) were among the first to suggest that during the diffusion of new information technologies, the creative functions of implementation should be 60 ajis vol. 5 no. 2 may 1998 separated from the control functions. furthermore, they suggested that organizational structures should be modified to support the different aspects of the diffusion process. thus, while a more flexible, organic structure where management controls are loose and informal was recommended for the earlier, more creative stages of technology diffusion, a rigid, mechanistic organizational structures and procedures were seen as appropriate for the later, more control stages of the diffusion process. following the mcfarlan and mckenney directive, cash, mcfarlan and mckenney (1983) advocated the creation of special groups within the information technology function or appointing special "idea champions" as intermediaries between users and information technology. these individuals or special groups were expected to identify new technologies that had the potential to address users' needs, and, then, support the development and diffusion of such technologies within the organization. cash and mcleod (1985) suggested in another paper that strategic planning processes led by top management might help identify promising technologies and then provide funds for piloting them in the organization prior to an organizational-wide implementation. in contrast to these directives, other researchers, most particularly those borrowing models from the literature on organizational radical change (gersick 1991; beers, eisenstat & spector 1990) maintained that the diffusion of informational technologies that are "revolutionary" in nature, cannot be initiated by the upholders of the status quo. these theorists claimed that there are cognitive, motivational and organizational reasons why those who are in positions of power would be unable to lead changes that undermine revolutionary change, instead, they advocated the importance of "idea champions" (schon 1963; howell & higgins 1990) as vehicle of spontaneous, non-organizationally sanctioned innovation and change. a variety of issues were mentioned by researchers as reasons for the fact that radical change cannot be led by those in power. these included:(1) people's past experience and education dictate the alternatives that they consider. people in position of power are constrained by their personal experiences. given that these people are often older than the idea champion of new technologies, they might be less inclined to support them (dearborn & simon 1958). (2) revolutionary change involves a loss of what is familiar and comfortable. given the fact that radical change may influence the distribution of power within an organization, elevating the change champions at the expense of others, it may be resented by those likely to lose power, namely, those in current positions of power (markus 1981 & 1983). (3) current organizational policies and obligations constrain the behaviour of current stakeholders. another way of stating this argument is to argue that as individuals and groups become more skilled in their activities, they have a tendency to routinize their modes of behaviour and become less inclined to change them (march & simon 1958). the obvious conflict between the above arguments in that some of them advocate top management involvement as a pre-requisite to a project success, while others, claim that radical change cannot be led by top management, was the theoretical starting point for the jarvenpaa and ives (1996) study, which is one of the pioneering attempts to systematically analyze the process of web technologies diffusion in organizations. based on two in-depth case studies conducted in two computer companies in the us, jarvenpaa and ives made five propositions that reflect their understanding of what the diffusion of web technologies in organizations is about. the first proposition in the jarvenpaa and ives model is that the introduction of web technologies is likely to occur with only peripheral involvement of the information technology function. this assertion was supported by the fact that in both organizations studied by the authors members of the informational technology unit were either peripherally involved or took a traditional control orientation which centered around formalizing, and safeguarding the existing information systems resource. the second proposition in the jarvenpaa and ives model is that the introduction of web technologies is likely to occur with minimal if any involvement by top management this assertion was supported by the fact that in both organizations studied by the authors top management was very minimally involved, particularly at the early stages of the diffusion process. in both organizations web technology was diffused for months and used extensively by a large number of organizational members before members of top management (in both cases the company ceo) expressed support for the project publicly. the third proposition in the jarvenpaa and ives model is that the introduction of web technologies is likely to be accomplished by an ad-hoc, cross-functional group that has no formal organizational responsibility for promoting the technologies. this assertion was supported by the fact that in both organizations studied by the authors ad hoc groups, mostly from the r&d and marketing communication areas were established as informal champions of web technology. in both cases, these groups invested significant efforts in demonstrating the profit potential of the new technologies, giving presentations to external cio's, and fostering attention from the external press to the importance of web technologies. 61 ajis vol. 5 no. 2 may 1998 the fourth proposition in the jarvenpaa and ives model is that a performance crisis can be used to stimulate the introduction of web technologies to an organization, but that a crisis is not necessary for this change to occur. the assertion was supported by the fact that one of the two organizations studied by the authors experienced a performance crisis shortly after the web technologies project had started. it was thanks to this crisis that the ceo and other top managers began to see web technologies as part of a solution that can benefit the company. the fifth proposition in the jarvenpaa and ives model is that the introduction of web technologies to an organization can be triggered or supported by "staged events", some of which could be deliberately manufactured" by the leaders of the project. this assertion was supported by the data collected by the authors in that in both organizations, the project leaders took advantage of real or imagined deadlines to push the project forward. while in one organization the deadlines were largely "manufactured" by the leaders, in the other they were triggered by a real threat from the competition when it was revealed that it was about to launch its own web page. in the following sections the above propositions are considered as they apply to case data collected in australia. the discussion attempts to reconcile the differences between the american and the australian experience by proposing the eows model. case study methodology data for this study were collected by the authors at an australian university referred to here as uoa (the name of the university as well as the names of all characters in the case have been withheld to protect their anonymity the names that are used are pseudonyms). data was collected over a period of two years. the major source of data for this study were interviews which were conducted with twenty members of the university. an additional source of information were a variety of hard copy documents supplied to the researchers by the interviewees. during data analysis, data from all sources pertinent to a particular event were analyzed and the interpretations of all interviewees for that event compared. a decision as to the meaning of the event for the various individuals involved was reached only when it was supported by the data from all sources and when it was agreed on by the researchers. interviews: in-depth interviews with twenty members of uoa were the most important source of data for this study. during data collection, a semi-structured interview schedule, consisting of a series of open ended topics was utilized. the questions gauged interviewees' memory of the events, as well as their interpretation of the events' meaning. even though the interview schedule was semi-structured, an attempt was made to cover the same topics in all interviews. issues on which interviewees disagreed received special attention. when such issues were identified, they were included in subsequent interviews, with special attempt made to reach consensus among the interviewees over these issues. in addition to gathering personal details (such as background information, career data, and future plans), interviewees were asked to describe the quality of their work life while the case events took place, relationships with other members of the organization, and areas of responsibility. the interviews lasted on average about ninety minutes, and were all taped, transcribed and analyzed by the authors. a content analysis scheme was used for the interview data. the scheme included a categorization of major themes in the interviews, with particular emphasis on issues relating to the details of the implementation events and the interpretation of these events by the interviewees. interviews were conducted over a period of two years (1995 to 1996), which roughly corresponded to the duration of the case events. on average, two interviews were held with each interviewee, bringing the number of interviews to forty. members of three major groups were interviewed:(1) academic staff there were 10 interviewees from the academic staff; four professors (including two department chairs and one dean); three associate professors; and three assistant professors. the academic sample was derived from six different departments from all major divisions at uoa. all academics selected as interviewees had direct or indirect knowledge of the events on which the case focused. to make sure that information about the case events was as complete as possible, all members of the university campus wide information systems (cwis) committee (see following sections for more details) were interviewed. (2) administrative staff five interviews were conducted with individuals who were categorized as "administrative staff'. from top management, the provost and the personal assistant to the president were interviewed. two secretaries were also interviewed. each of the secretaries was from a different department within uoa strongly associated with the web technology project 62 ajis vol. 5 no. 2 may 1998 (3) five interviewees were members of the information technology department (itd): the head of the department, the head of user services, the secretary of the cwis committee and the past and present co-ordinators of the web project support unit were interviewed. textual analysis: a variety of documents were collected at various stages of the implementation project. these included promotional materials, training transparencies, and minutes of relevant meetings. textual analysis also included in-depth study of the organizational chart, hard and soft copy correspondence, newspaper clippings, and progress reports. a major source of data for the textual analysis was e-mail messages. over 100 e-mail messages pertaining to the project were made available to the author by the key players in the case. the e-mail messages were analyzed using a specially constructed qualitative content analysis scheme. the scheme involved a thematic categorization of the issues discussed on e-mail by the various players in the case. case data uoa is a mediumsized university, with over 1000 staff members and over 13,000 students. the university is centrally located within a densely populated, highly industrialized metropolitan area. uoa has a reputation for being one of the most technologically advanced academic institutions in the country, investing substantial resources in experimental emerging technologies. the diffusion of web technologies at uoa followed a highly successful e-mail implementation, which started in 1988 and was completed in 1992, shortly before the case events started. in the following sections we will describe the diffusion of web technologies at uoa in terms of four phases, spanning the years 1993-1996. (1993) sporadic diffusion of web technologies:following the success of the e-mail diffusion in the years prior to 1993, many staff members at uoa were using early internet technology during 1992 and 1993. the majority of users were academics. they were using a variety of search engines such mosaic, archie and gopher, to support their research and networking initiatives with colleagues in other academic institutions. in mid 1993, the head of the information technology unit (itu), mr. adam neil, was becoming aware of the need to identify one unix based, multi-platform (compatible with both mac's and pc's) to support the file servers around the university. in the search for the one technology that would be best suited for uoa, mr. neil instructed his subordinates to consider and test several options, including, gopher, wais, apple share and mandarin. despite the search effort, by the end of 1993, a decision as to the best technology to support internet servers on campus had not been made. (1994) the cwis committee:the first months of 1994 marked a dramatic change in the development of the web technologies project. the change started when professor mark lind, uoa provost for research and the chair of uoa computer planning policy committee (cppc), announced during the committee's first meeting for the year that the president of uoa had authorized him to suggest to the committee the launching of a campus wide information systems (cwis) project. a cwis steering committee was to be created as a sub-committee of the cppc. the cwis committee was to supervise a project that would result in the creation of an integrative multi-platform web based intranet for uoa. by the end of the meeting, the computer planning committee has appointed mr. anthony moore, one of mr. neil's deputies and the head of the administration information system (ais) unit, as chair of the newly established cwis committee. following the decision of the cppc, mr. neil has instructed a team of experts within itu to renew the search for a technology that could support internet servers around the campus. by this time (mid 1994), web technologies have become popular in the industry. they were seen by the members of the itu search team as much more exciting than the options that they considered in 1993. most importantly, web technologies were seen as the most appropriate solution to uoa's multi-platform problem. following this line of reasoning, the group decided to recommend the web as the anchor for the university information infrastructure. in line with this recommendation, contact was established with several web technologies vendors. within a few weeks, a contract was signed with netscape. within a few more months netscape enterprise, the company's more advanced product with more features to support an intranet was adopted as the uoa web technologies infrastructure. by the end of 1994 the newly established cwis committee held its first meeting. the meeting was attended by five individuals. three of the committee members were from itu (including mr. neil and mr. moore). the 63 ajis vol. 5 no. 2 may 1998 remaining two members included mr. jack ford, the head of uoa department of external relations, and professor jeffrey wood, chair of the department of mis. the major decision made by the committee during its first meeting was to allocate $40,000 as a salary to a person appointed as co-ordinator of the web technologies project. the money was to be allocated in equal shares by itu and the university central administration. it was agreed the person hired as co-ordinator of the web technologies project would also be the secretary of the cwis committee. within a month after the committee's meeting took place, mr. james cooper took office as the new co-ordinator of the web technologies project. mr. cooper had an extensive experience as a manager in the is industry. he has been working with web technologies for years, including several years experience as a private consultant in this area. by appointing him as co-ordinator of the project, the members of the cwis committee have clearly expressed a commitment to web technologies. (1995) launching the web project:mr. cooper's appointment marked the formal start of the web technologies project. mr. cooper interpreted his role as co-ordinator of the project in several ways. first, as secretary of the cwis committee, he decided to make the committee more representative of the university community. to achieve this goal he decided to approach three additional academics and invite them to join the committee. the new members included professor jane peel, the chair of computer science, professor susan brown, the chair of an history, and professor gill grant, chair of psychology. the addition of the three professors did not only change the committee in terms of disciplinary representation (with less representation to ft people and more to members of other units within the university), but also made the committee significantly more balanced in terms of gender. another decision taken by mr. cooper was to focus the committee's work around the establishment of a new web site. even though the university already had a one page web site, it was felt that a much more sophisticated site was needed. during the remaining of 1995, discussions over the content of the web site became the main issues of concern for the cwis committee. the debates focused on the following issues:(1) the design question: while the administrative members of staff on the committee were adamant that the first page of the web site should highlight issues relating to foreign students, who represented a major source of income for the university, the academic insisted that such a design would be discriminatory to other sub-group within the student body, i.e., blacks and females. it was finally resolved that all students would appear on the top of the list as one group. (2) the standards issue: while the representative of the it group and the external affairs group within the committee were strong advocates of one set of standards, the academic members of the committee saw the attempt to impose such standards as "stifling of academic creativity". after long debates, it was finally agreed that the various departments within the university will be allowed to use whatever design tools, color scheme, or structure they chose. the only proviso was that all departments will put the university logo at the bottom of their "official" web pages. (3) the control debate: here the committee was divided between the more technically oriented members who felt that the ultimate control of the web site should be with itu, and the academic members of the committee who felt that departments should not be pestered by the it group when to update their web pages. the final decision of the committee was that itu will make its expertise available to departments who will choose to approach it for training and advice. however, it will not police that the web site and will not impose deadlines on the update of individual web pages. by the end of 1995 the cwis committee has met four more times (bringing the total number of meetings for that year to five). toward the end of the year, the format for the university web site (the first page) was agreed on. mr. cooper was instructed to construct individual pages for the different units within the university. in particular, he was to lead the very difficult and time consuming project of getting most of central administration documents on the web a project that was expected to continue for at least another year. (1996) consolidation of the web project:the cwis committee continued to hold meetings during 1996 but not at the same frequency as during 1995. during the year only two meetings were held and these were attended by less than two thirds of the members of the committee. during 1996 several members of the committee started to be disillusioned with the project. during the cwis committee meetings, several of the academic members complained that the promise to create a "paper free organization" was not forthcoming. indeed, many of the central administration departments who were supposed 64 ajis vol. 5 no. 2 may 1998 to convert their hard copy documents to electronic databases and make them available to users on the university's web server, did not meet this goal, claiming that other responsibilities left them no time to perform this extra job. other members of the committee appeared to be concerned about issues of data security. thus, several of the academic members of the committee who were initially enthusiastic about the prospect of using the web to send thesis to external referees, started to worry that the web was not secure enough for this purpose. other members of the committee raised concerns about access to university databases by unauthorized personnel, most specifically, students. by the end of 1996, mr. cooper, the project co-ordinator, was starting to realize that many departments within the university were simply not joining the web technology bandwagon. despite pressures from itu, and continuing investment in training and promotion efforts, it was clear that the project was not going to be completed by the end of 1996 as was expected by the members of the cwis committee. this prompted mr. cooper to ask for additional resources that would allow him to establish a special development group to assist units that were unable to meet the deadline. the proposal did not win the support of the majority of the cwis committee members, many of whom claimed that the project leaders should concentrate on the "quality" of the university web site rather than on its "quantity". the unanimous decision of the committee, was, however, that if mr. cooper felt that additional resources were necessary, he should attempt to secure them through his superiors at the itu. five months later, when 1996 drew to a close, mr. cooper finally managed to secure the necessary resources to finish the project. by this time, the majority of the departments at uoa, had their own web pages and so the money was to be invested in development of web based databases to support the administration services on the web server. interestingly, and despite the technical success of the web technologies project the views of the uoa members about it were highly polarized. while many academics, (including the academic members of the cwis committee) said that they were manipulated by the it group into being a rubber stamp to it's secret agenda, the it people (including the it members of the committee) blamed the "academics" for what they saw as "the less than optimal outcome of the project". discussion what can we learn from the data in the case study? despite the fact that the case study took place in a university, an organization with a unique culture that differs significantly from other industries, the differences between our findings and those of the jarvenpaa and ives' (1996) study are striking. in contrast to the propositions of the jarvenpaa and ives model, in uoa, the web technologies project was very much led by the information technology unit, with high degree of involvement by top management. furthermore, in our case study, the web technology project was not triggered by a performance crisis, nor was it punctuated by event based crises manufactured by the project team. put differently, it appears that the australian case lacked almost all the characteristics that led jarvenpaa and ives to label the web technologies project in their sample "transformational", "radical" or "revolutionary". in our case, the project was a reflection of the continuing commitment of uoa's top management to technological innovation. it was also a natural continuation of other projects initiated by the it function prior to the case events. in view of the fact that both the it group and other members of top management were directly involved in the project right from the start, it is difficult to claim that it posed a threat to the organizational power distribution or changed it in a significant way as suggested by the label of "transformational", "radical", or revolutionary. how can we explain the disparity between the jarvenpaa and ives propositions and our case? several answers can be given to this question, each pointing at a different direction for an extension of the jarvenpaa and ives framework. the first possible explanation relates to the fact that the data for the jarvenpaa case studies was collected in organizations which represented a different industry than the australian case. the jarvenpaa and ives' case studies were based on data collected in two computer companies, while the australian case centered around the diffusion of web technologies in a university. it can be argued that the two industries differ not only in their organizational structure and work processes, but, more importantly, in their organizational culture, leading to totally different strategies of diffusing web technologies. this possibility calls for an integration of the variable of "industry" in any model that attempts to theoretically conceptualize the process of establishing organizational web sites. another possible explanation for the disparity between the australian case and the jarvenpaa and ives findings might be related to the fact that the data for the jarvenpaa and ives study was collected in the us, while the data for our case was collected in australia. prior research on national cultures (hofstede 1980, 1994a; and 1994b) tells us that national culture is a major variable determining relationships within organizations. even though the american and australian cultures are assumed by hofstede to be quite similar, some significant differences 65 ajis vol. 5 no. 2 may 1998 between them do exist. in particular, the australian culture is significantly higher than the american culture on uncertainty avoidance (australians are more risk avert). the australian culture is also significantly lower on power distance (power differentials between managers and employees are highlighted more in the us than in australia). based on these differences, it can be argued that relative to american organizations, australian organizations would value participative decision making (lower power distance score), while at the same time, discourage risk taking (higher uncertainty avoidance score). this would explain the more participative and yet highly centralized management of the web technologies project in our case relative to the two american cases studied by jarvenpaa and ives. the importance of national culture as a determinant of diffusion process of web technologies calls for an integration of the variable of "culture" in any model that attempts to theoretically conceptualize the process of establishing organizational web sites. finally, the disparity between the jarvenpaa and ives findings and our case can be explained by issues that are idiosyncratic to the organizations observed. this explanation calls for an in-depth analysis of the decision making patterns typical to organizations undertaking web technologies projects, including the motivations of the various players, the politics of the process, and the influence strategies that are deemed legitimate by the organizational culture. to conclude this discussion, we would like to present the eows model. figure 1 presents a pictorial depiction of the model. as indicated in this figure, the model perceives the process of diffusion of web technologies in organizations as a seven-step process, consisting of: need; conception; investigation; bargain; acceptance; delivery; and review. the basic model in supplemented with the variables of "industry" and "culture", which are seen as modifying its basic components. what is the meaning of the various components of the model? in the following sections, we will define and explain the content of the seven steps of the model with specific examples from web technologies projects. (1) need the need stage is the most important step of in the eows process. it involves the identification of needs to be satisfied through the web technologies project. (2) conception the focus of the conception stage is on how the need, identified in the first stage should be satisfied. at the conception stage, the www technology is introduced and the people who are exposed to this technology consider its ability to satisfy their needs. (3) investigation if the use of www technology is considered a means to satisfy the re-defined need, an investigation into the possibility of a wide scale diffusion will be carried out. depending on the aforementioned factors in the conception stage, two types of investigation can be initiated: informal investigation (not supported by top management) and formal investigation (supported and most probably resourced by top management). (4) bargain the bargain stage may take two distinct directions. first, the committee members may debate the practicability and validity of the report findings, which will eventually involve amendments to the report. second, the committee members may debate the actual strategy of the implementation process. this could involve the sharing of responsibility for the individual components of the project, the scrambling of resources to finance the various stages of the implementation, and the fight for a better promotion of the project within the organization. (5) acceptance (rejection/re-investigation) the emergence of this stage concludes the process of bargaining, even though it may not reflect a state of satisfaction for all committee members. if the www technology is officially accepted, resources will be allocated to support its implementation. if the use of www technology is rejected or re-investigation is required, this will have little or no impact on the allocation of resources. (6) delivery this stage is characterized by actualizing all the ideas which have been collected in the former stages and turning them into deliverables. these deliverables include the development of an organizational web site and the production of organizational web pages. (7) review after the official launch of the organizational web site, certain mechanisms are likely to be set up in order to encourage feedback from users internal and external to the organization. the availability of feedback mechanisms will facilitate the quality control and the inflow of new suggestions. 66 ajis vol. 5 no. 2 may 1998 figure 1: national culture and industry as the determinants of the establishing organizational web sites (eows) process independent dependent variable national culture industry process of eows need conception investigation bargain acceptance delivery review conclusion the discussion in the previous sections suggests several possible directions for future research:first, the seven steps in the basic model which are seen here as linear, may in some instances be circular, i.e., sub-processes that take the organization members "backward" or "forward" may occur. for example, it is possible dial organizations which are at the investigation stage may sometimes regress back to the conception or even need stages. it is also possible that as the process progresses new needs that have not been identified a the beginning of the process, may emerge, leading to a continual repetition of the process. second, the inclusion of "industry" and "culture" as variables that modify the basic seven steps of the model call for a variety of empirical investigations into the specific ways in which these variables do, indeed, modify the process. such investigations should include in-depth studies of the process of diffusion of web technologies in private and public sector organizations and in cultures that are not necessarily english speaking nor technologically advanced. finally, the eows model calls for an in-depth socio-political analysis of the decision making patterns typical of organizations undertaking web technologies projects. such analyses will involve a careful study of the power and politics of the web technologies project, with particular emphasis on the players, their motivations, and their influence strategies. the introduction of a web technologies project to an organization is a far more complex process than is commonly assumed. the decision process that leads to the creation of an organization web site involves aspects that go much beyond the technology that supports the site. the web site is the organization's business card. it promotes the company's image externally with potential customers, competitors and partners. if the site is used as the anchor for an organizational intranet, then, it is also the carrier of the company's internal image, with its employees. given the key role that a web site can play in an organization it is no wonder that the decisions that pertain to its creation and maintenance, as indicated in our case, are highly political, reflecting the interplay of different stakeholders within the organization. 67 ajis vol. 5 no. 2 may 1998 the politics of the decision making process leading to the creation of an organizational web server or web site is, thus, a promising area for future research. the politics of the decision making process surrounding the creation of a web site can explain why even within the same industry and even within the same national culture (the two cases in the jarvenpaa and ives study) diffusion strategy can still differ markedly. when a comparison is made between organizations in different industries and cultures (our case) an understanding of the internal politics of the project can augment the explanatory power of the other two variables. reference beers, m., eisenstat, r. a., and spector, b. (1990) "why change programs don't produce change", harvard business review, vol. 68, no: 6,158-166. bickel, r. (1996) "building intranets: internal webs give companies a new solution to an old problem", interact world, 72-76. brown, m. and associates (1995) using netscape 2, special edition, que corporation, indianapolis. cash, j. i., mcfarlan, f. w. and mckenney, j. l. (1983) corporate information systems management: the issues facing senior executives, irwin, home wood, illinois. cash, j. i. and mcleod, p. l. (1985) "managing the introduction of information systems technology in strategically dependent companies", journal of management information systems, vol: 1, no: 4, 523. dearborn, d. c. and simon, h. a. (1958) "selective perceptions: a note on the departmental identification of executives", sociometry, vol: 21, 140-144. emery, v.(1996) how to grow your business on the internet, coriolis group, inc., arizona. gersick, c. j. g. (1991) "revolutionary change theories: a multilevel exploration of the punctuated equilibrium paradigm", academy of management review, vol: 16, no: 1, 10-36. hofstede, g. (1980) culture's consequences: internationa] differences in work-related values, sage publications, california. hofstede, g. (1994) "management scientists are human", management science, vol. 40, no. 1,4-13. hofstede, g. (1994) "the business of international business is culture", international business review, vol. 3, no. 1,1-14. howell, j. m. and higgins, c. a. (1990) "champions of technological innovation", administrative science quarterly, vol. 35, no: 2,317-341. jarvenpaa, s. l. and ives, b. (1996) "introducing transformational information technologies: the case of the world wide web technology", international journal of electronic commerce, vol: 1, no: 1, 95-126. kalakota, r. and whinston, a. b. (1996) frontiers of electronic commerce, addison-wesley publishing company, inc. reading, massachusetts. krol, e. (1994) the whole internet: user's guide and catalogue, 2nd edition, o'reilly and associates. inc., california. lejeune, u. a. and duntemann, j. (1995) netscape and html explorer, coriolis group inc., arizona. march, j. g. and simon, h.(1958) organizations, john wiley and sons, new york. markus, m. l., (1981) "implementation politics top management support and is involvement", systems, objectives, solutions, 203-215. markus, m. l. (1983) "power, politics, and mis implementation", communications of the acm, vol. 26, no: 6,430-444. mcfarlan, f. w. and mckenney, j. l. (1982) "the information archipelago maps and bridges", harvard business review (september october), 109-119. pitter, k. and minato, r. (1996) every student's guide to the world wide web, mcgraw-hill companies, inc., new york. rayport, j. f. and sviokla, j. j. (1995) "exploiting the virtual value chain", harvard business review, vol. 73, no: 6, 75-85. savola, t. westenbroek, a. and heck, j. (1995) using html, special edition, que corporation, indianapolis. schon, d. a. (1963) "champions for radical new inventions", harvard business review, vol. 41, no: 2, 7786. steme, j. (1995) world wide web marketing: integrating the web into your marketing strategy, john wiley and sons, inc., new york. 68 addressing organizational issues in requirements engineering practice: lessons from action cases jim hughes and trevor wood-harper information systems institute, university ofsalford, salford. ms 4wt. uk and school of accounting and information systems, university of south australia, adelaide. south australia abstract the requirements phase of information systems development is arguably the most organizationally dependent phase and as such it could be contended that the methods used in this phase ought to explicitly address organizational issues. however the authors would maintain that there is a weakness in many of the methods that are employed in requirements engineering in that they arc not explicitly embedded within any social scientific perspective. we argue that such a perspective would enable the systems developer to address organizational contexts and to engage with users (social actors) in such a way that it may be possible to find out about their social situations and as a consequence better inform the systems development process. in this paper we present the arguments for an explicit organizational perspective in the requirements phase of information systems development. we illustrate the argument through the presentation of action case studies in which one of the authors intervened in two situations with an explicit sociological perspective and analysis methods more commonly associated with qualitative social science methods. we conclude that there is an imperative need for those involved in the requirements phase to be explicit in their assumptions about organizations and critically reflective about the methods they use if they wish to understand the domain in which they operate. introduction the authors maintain that there is a critical weakness in many of the methods employed in the requirements determination phase of information systems development since although the methods may have an underpinning philosophical basis, they are not explicitly embedded in any social scientific perspective. such a perspective would enable methods or methodologies to address the organizational contexts in which their use may be envisaged. in other words, for information systems development purposes, traditional methods have been used which are based on a philosophical paradigm that is intended to give a technological perspective. in view of the directions from which many information systems researchers have arrived (adam and fitzgerald, 1996) it perhaps is not surprising that their ideas in the realm of organizational analysis may appear to some to be epistemologically unsound. the issue here is that any analysis which aspires to reveal details of particular aspects of organizational life needs to reflect an ontological and epistemological rationale for the study of organizations as complex systems. that is to say, until it is decided what is knowable about organizations, and how such knowledge might be arrived at, the eclectic use of what may be labelled 'convenient' analysis instruments poses a threat to any serious progress. the fundamental question which must be addressed from the outset by a systems developer in any organizational or social setting, is 'what is going on?' or, more pointedly, 'how do i make sense of, and understand, what is going on?' in the traditional inquiry methods requirements engineers may bring assumptions into the organizational domain that may have little or no relevance to the context in which the questions are being asked. these assumptions may not be explicit and therefore may as a consequence never be examined. if for example requirements engineers set out to discover why people hold particular points of view, or why people in a particular part of the organization behave in a certain way then they must first be able to explain why such a discovery is important or relevant. it is against this background that the issues are addressed. the paper is structured as follows: it begins with a critique of the classical approach to requirements engineering and then proposes an alternative 'situated' approach. this is followed by two illustrative action cases. the treatment of the cases provides an explicit organizational perspective for intervention, describes the interventions and the outcomes from them. the paper concludes with a discussion of the major issues the value of situated requirements and the implications for the role of the requirements engineer. a critique of classical requirements engineering the classical approach to requirements elicitation revolves around the role of the requirements engineer and in using a term such as engineering it therefore constrains the process to be largely technical. indeed the main role of the requirements engineer is said to be to 'capture' requirements. the suggestion being that requirements may be difficult to find but they exist in a deterministically fixed form and indeed the implication is that there is a finite number of them to be found. the classical methods also tend to be restricted to questionnaire and interview but as jirotka and goguen, (1994) note "the analysis of the materials gathered is largely left to intuition" (jirotka and goguen, p.4, 1994). in practice most requirements specifications, which are considered to be the product of a requirements elicitation exercise, are written in natural language supported by diagrams that conform to some predetermined standard. examples of standards of this type are the starts handbook (ncc, 1989). mcdermid (1994) identifies the structure of typical orthodox requirements specifications and these are shown in figure 1. here the core of the specification is a set of function definitions, setting out the 64 functions to be performed by the system which are supported by a description of the structure of the data to be processed. mcdermid notes that global considerations of requirements may also be included but these tend to more problematic since they deal with properties that may be considered to be emergent rather than be decomposable. typically then if the functionality and other non-functional properties can be decomposed into modules then these may be allocated to software engineers to implement. the problem with this approach to requirements elicitation is that although the specification may be considered to be well written it may not be appropriate. as valusek and fryback (1985) explain, the process of information requirements determination is a process which incorporates obstacles within individual users, among users and between users and systems developers. in the case of obstacles among users they maintain that users' descriptions or definitions of similar functions or requirements may be quite different. typical functional specifications cover, at least normal functioning, including inputs, outputs, operation, and control abnormal functioning, exceptions etc. dependability properties eg safety and security performance eg throughput quality eg maintainability or modifiability expected changes data specifications: definition of data structure volumetrics, eg data flow rate sources and sinks of data figure 1. structure of typical orthodox requirements specification (adapted from mcdermid, p.24, 1994) furthermore, the meaning that systems developers attribute to the responses in interviews or questionnaires, that is their interpretation, may be different from the meaning held by the users. it is the obstacles within individual users that valusek and fryback claim has received little attention, that is the problems associated with the understanding of the user due to cognitive limitations on memory, recall and judgement what davis (1982) calls the constraints on humans as information processors and problem solvers. in the case where the specification is to have a level of detail to enable software engineers to code then these human centred problems are exacerbated rather than mitigated. there is also the problem in validating requirements, a problem linked to the above, because of the semantics of natural language (auramaki et al., 1988). as mcdermid (1994) notes somewhat cynically the validation process for such specifications merely documents 'what it is that the analyst thought it was the problem owner said he thought he might want, not what he'll get!'. it is possible more recently to see the requirements engineering community addressing the domain problem through the introduction of techniques such as viewpoints (easterbrook and nuseibeh, 1995) and the use case approach (regnell et al., 1993; jacobson, 1992). others address domain analysis (frakes et al., 1998a, 1998b) and elsewhere a domain theory for requirements engineering (sutcliffe and maiden, 1998). however whilst they recognise the evolutionary and conflicting nature of the requirements process and attempt to deal with the complexity of requirements they may be considered to stay true to the classical formalisation roots. further examples include the recognition by jackson (1998) that requirements engineers are vitally concerned with describing a mostly informal world in which "it is almost impossible to give a complete and formal description" (jackson 1998, p.74). however his response is to attend to the 'accuracy' of the descriptions used implying an organizational assumption that there is some given reality which can be both captured and described. we accept that there are those who are not so critically aligned to the requirements engineering tradition since some question the mechanistic view of requirements capture. dzida and freitag (1998) for example point to the use of scenarios for requirements 'construction' rather than 'capture' and potts and newsletter (1997) note the growing appreciation for naturalistic inquiry derived from the anthropological and the ethnomethodological traditions. however these views remain in a minority amongst the requirements engineering community. this is not, as it may seem, a return to the lengthy debate about 'hard' versus 'soft'. for example a method of understanding complex problem domains such as soft systems methodology (ssm) (checkland, 1981; checkland and scholes, 1990) whilst gaining popular appeal has been criticised from some quarters. brown (1992) criticises ssm on the grounds that it is flawed as a mainstream sociological research approach and there are inherent weaknesses in the first two stages of ssm the finding out stages which provides only the most impressionistic guidelines for the capture and analysis of data. as noted by dobbie and hughes (1993), this has opened the door to possible expediency in the application of inquiry methods and methodologies. that is to say that in the worst case requirements engineers, or indeed ssm practitioners, may use methods and methodologies in the organizational domain without regard to any explicit organizational perspective. specifically for requirements engineers it is argued here that what is required is an approach which deals with the explicit recognition of the organizational and social dimensions of requirements (hughes et al., 1993; 65 goguen and linde, 1993). it is very much in the spirit of potts and newsletter's (1997) assertion that naturalistic inquiry techniques incorporated into the requirements process must not be seen simply as 'a set of slick tools' but that to deliver their potential they must be 'taken with their philosophical baggage intact'. below we consider the basis for such naturalistic inquiry and in doing so we explicitly elucidate the philosophical underpinnings and the organisational perspective of the inquiry. an alternative view situated requirements an alternative view of the prevalent requirements process is that proposed by goguen (1992) in which he discusses the relationship between formal, context insensitive information which he terms 'dry' and informal, situated information which he terms 'wet'. the problem for goguen in presenting 'dry' 'wet' is the possible reconciliation between seemingly mutually exclusive states and he maintains that "formal information occurs in the syntactic representations used in computer-based systems. informal situated information arises in social interaction, for example, between users and managers, as well as their interactions with systems analysts. thus requirements engineering has a strong practical need to reconcile the dry and the wet" (goguen 1992, p. 1) the resolution is all the more difficult since the expression of context sensitive requirements is usually through the use of natural language. yet goguen notes that features of natural language such as ambiguity, and lack of explicit structure can be advantageous since these can facilitate the gradual evolution of requirements. also natural language is often the medium preferred by the individuals who represent the client. interestingly this also highlights a shift in the perspective of requirements. the move is away from requirements being considered as the means by which systems developers can proceed with systems design and implementation and towards a perspective that requirements can be expressed in order that clients (users) can readily understand them. the suggestion being that the richest way of eliciting context sensitive information is to use the words and meanings that the clients use in the settings in which the clients act and interact with each other. in sociological terms this perspective is known as ethnomethodology which is the study of everyday procedures (methodology) that people in social situations (ethno) use to create, sustain and manage a sense of their social setting. for those involved in the requirements elicitation process this may be interesting and useful for collecting high quality data about social interaction but it does not apply to the analysis of data. an attempt to overcome this analysis issue in a practical domain is discussed later in the paper. what is needed first is to understand that the essential feature of this kind of study is that it attends to social data, which is situated. that is to say that the data can only be fully understood in relation to the particular, concrete situation in which it occurred. much of the work in the area of situated action derives from studies made by lucy suchman (1987). suchman, an anthropologist, whilst not explicitly concerned with requirements determination, has made an important contribution to this area because in trying to understand fields such as cognitive science and human computer interface she constructs accounts of the relations among people including the 'culturally constituted' worlds that they inhabit. she maintains that the strategy for investigation is one in which the researcher/analyst can make sense of everyday activities and the contexts in which they arise. hence she largely draws on ethnomethodology as the sociological basis for her work. therefore in using the word 'situated' suchman is concerned with events that occur in some interaction which can only be fully understood in relation to the particular concrete situation in which they actually occurred. goguen (1994) provides six points derived from suchman's work that he considers to be the qualities of 'situatedness' and these are given in figure 2. for requirements determination, point 6 in figure 2. identifies that by considering any situation much of the practical knowledge is unarticulated and therefore for a requirements engineer the methods used for the analysis of the good quality data that ethnomethodology can produce is to determine (some of) the tacit grounded knowledge. one possible method for analysing the good quality data to determine the grounded knowledge is to use ethnomethodology principles together with grounded theory (glaser and strauss, 1967). grounded theory is a method from the social sciences used for the collection and analysis of qualitative data. using its procedures as a guideline the requirements engineer abstracts categories from interview transcripts and produces a natural language account that links the categories together. the following section briefly describes two consultancies in which grounded theory and ethnomethodology were used to elicit situated requirements (hughes and wood-harper, 1999). 66 • emergent social events can not be understood at the level of the individual, that is, in terms of individual psychology, because the members of some group through their interaction jointly construct them as social events. • local. actions and their interpretations are constructed in some particular context, including a particular place and time. • contingent. the construction and interpretation of events depends upon the current situation (potentially including the current interpretation of prior events). in particular interpretations are subject to negotiation, and relevant rules are interpreted locally, and can even be modified locally. • embodied. actions are linked to bodies that have particular physical contexts, and that the particular way that bodies are embodied in a context may be essential to the social interpretation of some events. • open. theories of social events (both those constructed by participants and by requirements engineers) cannot in general be given a final and complete form, but must remain open to revision in the light of further analyses and events. • vague. practical information is only elaborated to the degree that it is useful to do so; the rest is left grounded in tacit knowledge. figure 2. the qualities of situatedness (adapted from goguen 1994, p.168) illustrative action case studies in this section the authors present two action cases to illustrate the ideas discussed above in a practical context. the objective of the action cases was primarily to elicit situated requirements to inform a more formal requirements engineering exercise. it should also be noted that it is not our intention to suggest that requirements engineers should adopt the methods discussed. the purpose of the illustration is to suggest how an explicit organizational perspective together with appropriate methods may assist the requirements engineer to better understand and make sense of the organizational domain. we begin this section by briefly outlining the action case method and the development of the conceptual framework that informed the intervention. action case the use of the term 'action case' follows from vidgen and braa (1997) who approach action case as arising from 'soft' case study which is essentially a method for understanding in which there may also be some limited intervention, which causes change. it is used here from an alternative perspective in which intervention is planned and from which some understanding is gained about the conceptual framework in order that learning can take place. the term is also intended to convey that the learning may be achieved in a limited number of interventions. indeed a single intervention for a practitioner should be sufficient to reflect and learn about some project. in terms of action research the term action case may be characterised as 'reflective action research' as discussed by baskerville and wood-harper (1998) in which they maintain that the critical element 'is the actors' discovery of where their behaviour is unexplained by their own understanding'. they note that within reflective action research 'iteration is no longer an end in itself. this is clearly important for the practitioner since even within a single requirements project he/she may focus on the distinction between what was suggested by the theoretical framework versus what actually happened in practice. a consequence of this is the learning about the theoretical framework and the process of requirements engineering. furthermore, action case as used in the illustrative research presented here has a strong resonance with the concept of 'double loop learning' as expressed by argyris and schon (1978). they consider double loop learning to mean that following some intervention in an organization it is possible to learn about the domain they refer to this as single loop learning. however the learning may also challenge the 'norms' or framework which were the basis of the intervention. they suggest that this could cause some conflict amongst managers who established the norms. they refer to double loop learning as a resolution of the conflict. indeed as reason (1994) notes double loop learning is critical, since without the reflection on the 'governing variables', or theoretical framework, then it is possible for individuals to produce self-fulfilling systems of action which may lead to escalating errors. as mentioned earlier in this paper this is a possible consequence of a requirements engineering intervention without an explicit organizational perspective to help understand what is going on. conceptual framework the conceptual framework comprised the conjunction of ethnomethodology as a paradigm of inquiry and grounded theory as a method for data analysis. action case was the research strategy for reflection on the 67 domains. the ethnomethodological perspective and the rationale and procedures for grounded theory are described briefly below. ethnomethodological study focuses on how social actors (users in the requirements context) accomplish, manage and reproduce a 'sense' of social structure. ethnomethodologists focus on folk (common) methods and common sense reasoning. garfinkel (1967) refers to this as, 'the investigation of the rational properties of indexical expressions and other practical actions as contingent ongoing accomplishments of organized artful practices of everyday life.' since reality is produced by way of actors' interpretive procedures then the ethnomethodologists maintain that the actors' social circumstances are 'self-generating' which implies two important properties. firstly that meanings are essentially indexical, that is they depend on context and therefore it is only in the situated use of talk and interaction that objects and events become meaningful. secondly that social realities are reflexive which is to say that interpretive activities are both in and also about the social settings that they describe. thus the focus for ethnomethodological research is the treatment of talk and interaction as topics for analysis rather than merely as a means of communicating some underlying phenomena. waters (1994) proposes that for ethnomethodologists the only way in which sociologists can reveal the 'facticity' of social experience is to approach it as would an anthropologist. he maintains that the sociologist must seek to understand situations in the terms in which participants give accounts of them. furthermore sociologists must somehow induce participants to give accounts and thus to reveal the 'contextually rational properties of their social arrangements'. grounded theory or as it is more properly titled "the discovery of grounded theory' (glaser and strauss, 1967) is a method for the analysis of qualitative data. the original work had three main purposes. firstly, it was intended to offer the rationale for theory that was 'grounded', that is to say generated and developed through the inductive analysis of data collected during research projects. the use of the term 'theory' is used in the sense that it refers to local empirical models devised on the basis of data. the second aim was to suggest the procedures and the reasons for them and the third aim was to propose legitimacy for careful qualitative research. in the method conceptual properties and categories may be 'discovered' or generated from the qualitative data by following a number of guidelines and procedures. there are two critical stages of grounded theory identified by glaser and strauss. firstly that of 'constant comparative analysis', a procedure for the identification of conceptual categories and their properties which may be embedded in the data and secondly what they call 'theoretical sampling' which is both a category enriching and disconfirming procedure. these are depicted in figure 3. the procedures are intended to indicate the primacy of the actors in the social setting. it is they who provide the transcript data, usually from interviews. as soon as the first transcript is prepared coding begins. at this stage it is 'open coding' and is fairly unrestricted in the sense that each line of the transcript is scrutinised and sometimes each word of each line to produce a code which at first may seem almost to be simply repetition or a summary of the text. transcript pool memo pool ''••.. code and .-•'' .: compare :. * collect data scad (£ setting saturate categories account of orcptizotiond life legend: constant comparative analysis theoretical sampling figure 3 grounded theory procedures (hughes and howcroft, 2000) 68 the purpose is to 'open' the inquiry into the data and at this stage any part of the data may seem relevant and if it is not then later it can be discarded. as more data is collected the analyst continues the open coding but also begins to identify 'categories' rather than labels. the categories are formed from groups of words from the initial coding which pertain to the same phenomenon. categories are at a higher level and more abstract than the concepts they represent. the concepts must be repeatedly present in the data to warrant the abstraction to the level of category. the analyst may select both in vivo categories or the analyst's own categories. the categories are generated through the same analytic process as that above of making comparisons to highlight similarities and differences. for the process to be rigorous the analyst must go back through the currently available data and re-code in relation to the new categories. analysts may then use their knowledge of the domain, or their previous experience, to expand and clarify categories to sensitise themselves to ways of further exploring the data. as categories are formed it is essential that each has associated with it a memo which elucidates it and relates it back to the data. as the number of categories increase they can be related to each other and to sub-categories. this process is known as 'axial' coding in which the analyst explores the relationship amongst categories and sub-categories. this will usually necessitate the gathering of further data to elucidate perhaps quite specific points in order that a category can be saturated. saturation of a category occurs when the collection of data and its analysis can add no further to the properties of the category. categories may also be related to each other to provide an integrating framework. during axial coding the memos will now be developing the properties of categories and the relationship with sub-categories and in particular the integration of categories and the relationship between them. as the number of saturated categories grow and the relationships between categories become established the categories can be unified around a 'core' category. this usually happens towards the end of an investigation. the core category represents the central phenomenon of the study and it is important that it is arrived at although the path to selecting it may be very difficult. in the later stages of an investigation the analyst may have identified several schemes that can link the data together. what is important is that a central category is arrived at and abstraction or generalisation can achieve this. indeed the more abstract the core category then the more widely applicable it can be said to be to the organization. the 'filling in' of poorly developed categories is done at this stage, in order that all categories in the final integrative framework of categories are saturated. the memos at this stage are likely to be more elaborate and should thus be integrated, since they will form the basis of the final written account. the two analytical techniques of theoretical sampling and constant comparison that are given in the legend of figure 3 are the means by which the grounded theory proceeds. these techniques relate the coding activities to the pool of data and memos that already exist. theoretical sampling is an inductive technique that is used when the data do not exhaust the theoretical category that is being developed. the analyst must return to the domain until categories have been saturated. that is to say the analyst must sample particular categories to ascertain that its properties can not be elucidated further. specific detailed questions need to be asked of participants related to the category that has been induced. this checks the scope as well as the depth of a category. the theoretical sampling in grounded theory is theory-driven and strauss and corbin (1998) emphasise that the grounds which decide what people, events or activities need to have further data collected on them are analytic grounds based on the emerging theory. as has been described above, as each new incident or concept emerges it needs to be compared against existing concepts and categories for both similarities and differences. the purpose of this constant comparison is essentially threefold: to ensure as far as is possible the analyst's neutrality by constantly challenging him/her with new data; to help achieve greater precision in grouping together like phenomena and then to further subdivide concepts which are variations on the first; and to ensure consistency in grouping like with like. the analyst must always look for patterns and variations and identify process where it occurs. strauss and corbin (1998) stress the importance of this constant comparison by saying that concepts and their dimensions 'earn' their way into the theory. grounded theory has already had some limited exposure in the is domain as a practical method predominantly in the knowledge elicitation area. most notable contributions include oliphant and blockley (1991), pidgeon, turner et al. (1991) and galal and mcdonnell (1997). although for each of these contributions no explicit organizational perspective is adopted and the procedures are used to code and categorise qualitative data. to summarise the conceptual framework: ethnomethodology provides the organizational perspective for the collection of data and grounded theory provides reliable procedures for analysis. within an action case the outcomes are both practical situated requirements and learning through reflection for the practitioner (requirements engineer) the action cases the hvp study domain the first action case domain was a three site veterinary practice in a city in the east midlands, which we shall call hvp. one site is in the city centre and the others are situated 5 miles south and 5 miles northwest of the city centre in large village locations. the practice is a mixed practice in that it does both small animal and farm 69 animal veterinary work. the practice is a partnership with four equal share veterinary partners, one salaried partner, four full-time veterinary assistants and two part time veterinary assistants. there are thirteen practice receptionists including a reception supervisor at each site and eleven nurses including a nurse supervisor for each site. additionally the practice employs out of hours telephone receptionists who usually are the life partners of the vet on call. the general management of the practice is through the partners assisted by a full time practice manager and a practice accountant both of whom sit on the practice management team. one accounts clerk and one administrative assistant provide additional administrative support for the farm office. the hvp study intervention the nature of the intervention was to help the management team better understand its information needs prior to considering any investment in an it infrastructure. they expected an outcome of the study to be an audit or evaluation of current work practices and recommendations for change, which would accommodate the introduction of computerised information systems. during the initial talks with the management team the developer made explicit the nature of the intervention methodology and the reasons for it. it was also established and agreed that the work would also have research outcomes. the team agreed that the outcomes of the research and the outcomes that they expected from the study were not in conflict. the rationale for the study was linked to the management team's agreed strategy, which was presented through discussion rather than being formally documented. the management team in the last three years had seen an increase in the pressure on the city-centre site in terms of an increasing client base and the lack of appropriate accommodation to meet the increased demand. this led to the decision to open a new site, w, which would take some of the client numbers and also enable the practice to offer a purpose-built farm office and centre for farmers to visit to purchase veterinary products. the financial profile was such that funding the new premises was viable and the expectation was that the income from the new site would in the long term provide financial stability for the practice. at the time of the study the management team realised that the growth in business since the opening of the w site had exceeded expectations and were considering ways to stabilise their client base such that a further site would not be required in the future. the management team broadly agreed that the move to a three-site practice had put, and would put, demands on their existing systems, procedures and management and staff. the research/consultancy proceeded by establishing 'seed categories' for the interviews. although a departure from the procedures of grounded theory, miles and huberman (1994) consider this a legitimate way to give initial focus for the interview questions. the three seed categories were understanding of job roles and responsibilities, decision-making processes and communication between the three sites. the first interviews were followed up with individuals and with groups in order to fill out emerging categories until the major categories were saturated. saturation is achieved when transcript coding no longer adds to the dimensions or properties of the categories. this was followed in the analysis by the identification of the core category, which is the most abstract representation of the data. from the written accounts (known as memos in grounded theory) of each of these categories an understanding of the domain emerged which was grounded in the organization. the written accounts are given in full in hughes 1998). the hvp study the practical outcomes for the hvp management team a number of practical recommendations were made in the form of a report. the report was a distillation of the rich account produced by the use of the analysis method into a relatively thin document that largely ignored the process and context issues, which had led the consultant to make the recommendations. briefly the recommendations were: to restructure the role of the practice manager; to specify management, roles in autonomous sites; to address formal communication systems such as day book and work rotas; to introduce computerised record keeping for client records in the first instance. the hvp study the research outcomes for the researcher one of the important lessons was to consider this distillation into thin descriptions for the management to be a part of the methodology for future interventions. other research outcomes included the excessive time taken to interview, transcribe, code and categorise the use of seed categories and the adaptation of the original methods into a practical context. what emerged was a revised framework. this revised framework was part of the action cycle and informed future work. the second action case used the adapted framework and methodology. the fp uk study domain the second action case concerns a small manufacturing company, which we shall call fp uk, based in northeast wales. it is part of the fp group consisting of three private limited companies, which are financially 70 independent. the parent company fp a/s was registered in denmark in 1952 to produce iron dust cores, ferrites and ceramic capacitors. fp uk offers a manufacturing capability for the production of most types of high quality inductors and transformers. transformers are produced as custom designs and also in standard ranges. a highly flexible production facility ensures short lead times on custom designs as well as standard products. a small amount of business is with factored products (selling danish manufactured products). over the years the fp group has forged strong links with technical institutions and other companies in the uk and the eu and is currently involved in several joint research and development projects with academic and commercial concerns within the eu. fp uk specifically has a design alliance with a german manufacturer and outlets for sales in both italy and germany. fp uk employs 18 full time salaried staff who work in administrative or management roles and an additional 20 who are production line workers. the fp uk study intervention fp uk faced two major decisions with respect to the future of the organization. the first was to determine whether the future for fp uk lies either in manufacturing or in sales. the second and related decision was to determine an investment strategy in it/is linked to business objectives. the intervention was an audit to establish an is investment strategy that would enable fp uk to make the decision regarding their future direction. as with the action case given above, initial discussions included an explicit discussion of the research as well as the practical outcomes. in this case two seed categories were used since these had proved useful in the first case. these were understanding job roles and responsibilities and the impact of it on work. a change suggested by the learning from the first case was to cut the time taken for transcribing, coding and categorising. this was particularly important in the fp uk case because of the tighter deadline set by the sponsors. this was achieved by exploiting the nud • 1st software package and by not fully transcribing all interviews. using the nud»ist package it was possible to listen to the taped interviews, transcribe the relevant passage, which would range from a couple of sentences to two or three paragraphs and augment existing categories or create new ones directly and therefore avoid the full transcription step. because this seemed such a departure from the method, the taped interviews that were not fully transcribed were usually those where categories were being saturated. that is to say they were second or third interview with the same person. a core category was identified, 'demarcation' and both a full account from the coding was produced in addition to a management report. the fp uk study the practical outcomes for fp uk the practical outcomes were an identification of an existing situation that showed a lack of a common understanding in the use of a variety of information systems and their relation to each other. also a conflict between organizational goals, expressed by the managing director, and the goals of particular functional areas. the report, which was well received, recommended a single network infrastructure, internal it integration, exploitation of external it links, staff awareness issues, the appointment of an it support manager and the inclusion of is strategy as part of the business planning cycle. the fp uk study the research outcomes although the study was a departure from the original procedures for data analysis described by glaser and strauss (1967) it should be remembered that the original authors and users of the methods had no recourse to the use of such powerful software tools. in the original paper-based method great emphasis was placed on transcription since the transcribed document was also the physical medium for coding. that is to say margin notes in the form of codes were critical to the success of the method. however the introduction of the software package as the medium reduced the importance of the transcript except as evidence. (it is accepted that the tape itself is evidence but it is much less accessible than paper). hence the sequence 'tape transcribe code" becomes 'tape partial transcribe code'. the difficulty with this is that when the developer needs to compare new documents with existing documents then only the codes or categories or partial transcript are available and therefore the taped interviews have to be listened to again. a further difficulty with this practical adaptation is the auditability of the method since although categories can be traced to the source document, the document itself is not a full account of the interview. another consideration on this point is that the paradigm of inquiry for the method, ethnomethodology, is concerned with transcripts being in the interviewees' own words. whilst the adaptation is true to this in terms of the partial transcript, it clearly does not contain all of the interviewees' own words and therefore may not be in keeping with the ethnomethodological perspective. however as with the hvp action case, and as noted in the introduction, this dilemma between practical outcome within time constraints and theoretical rigour whilst of interest to the developer, must be resolved in favour of the practical outcomes. other research outcomes included the favourable use of the software package in time critical situations and the need for initial seed categories. 71 lessons from the action cases in this section we explore some of the lessons of the action cases. in particular we consider what situated requirements may add to the requirements engineering process and we also consider the implications of this for the role of the requirements engineer. situated requirements the case has been presented to suggest that ways may be available to elicit situated requirements. it may be argued that the nature of the situated requirements in the form of a lengthy, although rich, report accompanied by diagrams of hierarchical categories is almost as impenetrable as the situation itself. furthermore the result of any analysis which aims to give structure to ill-structured situations can at best be described as a snapshot and at worst described as 'pinning the butterfly'. that is to say that the use of the methods to elicit situated requirements in effect presents a description which is only a filter through which it may be claimed that some glimpse of some aspects of organizational life may be observed. the methodological questions thus remain about the use of any intervention method which necessarily affects the very area it purports to understand. in practical terms however there are more positive lessons which may be learned. situated requirement elicitation does provide a perspective on organizational life that other, more technically centred, methods ignore. since by adopting an ethnomethodological perspective the method attends to 'meaning' in constructing social situations. additionally the methods described may be said to provide a perspective through which the clients (users) better understand their own situation since it is articulated using their own words. a critical final point is that in the cases described the action case method enabled reflection on both the practical outcomes and also on the methods (process) of arriving at the situated requirements. this allowed for changes in the application of the methods and a fuller understanding for the consultant/researcher of the methods and their situated use. this has implications for the role of the requirements engineer if such perspectives are to be adopted and these are discussed next. implications for the role of the requirements engineer a new role needs to be considered for the requirements engineer which moves beyond the traditional role of expert and indeed beyond the emerging role of facilitator. it may be said that this move towards new roles has already begun in requirements engineering with the emergence of use case scenarios or with ethnographic approaches to the requirements organisational domain. in this latter case ethnography has been used in practical design settings to complement existing human centred methods of systems development based on the growing expectation that processes arising from social contexts should be informed by 'real world' analysis (hughes et al., 1995). however the arguments presented here extend this role by emphasising the thinking skills of the requirements engineer and not solely the technical skills or the 'correctness' of method. this view is expressed in the literature as 'moral agent' (walsham, 1993) or 'reflective practitioner' (schon, 1983). it may be more appropriate to consider the role of the requirements engineer as 'bricoleur' which is not simply the jack of all trades but also the inventor. it is an appreciation that the tools, techniques and methods for understanding the organisational domain are insufficient to deal with each new situation and that the requirements process depends upon the individual requirements engineer to have a deeper understanding of the limitations of the process. this deeper understanding can be attained by explicitly stating any assumptions made about the organisation and the methods used and then reflecting upon and challenging these assumptions. there are two critical elements to this understanding the first is an understanding of context (situation) aided by an explicit organisational perspective and the second is the dependence on the agent (requirements engineer) in that context and not a dependence on the methods used. the bricoleur role avoids the pitfall of treating selection of methods or methodologies as one might select tools from a toolbox since it incorporates the reflective (thinking) aspect. the reflective requirements engineer adapts methods and methodologies according to context and makes these judgements explicit. finally it may be said that this is nothing more than 'you learn by your mistakes' or 'its just experience'. it would be damaging to leave that intuitive conclusion since it is in the articulation of the learning that is a further powerful element to reflection it is the means by which one person's practical experiences become the shared learning of a community. conclusion to summarise the arguments and the relationship between the action cases and the eliciting of situated requirements the authors maintained in the introduction that there is an inherent weakness in the methods commonly associated with requirements determination. specifically this relates to the lack of an explicit organisational perspective when intervening in organisations. as potts and newsletter (1997) note this is not unexpected since 72 the requirements process is underpinned by a tacit positivistic philosophy. however if, as the paper suggests and in common with potts and newsletter (1997), we are to move to naturalistic methods which specifically address organisational perspectives then it is incumbent upon the requirements engineer to explicitly state the perspective and the underlying philosophy. therefore through the action cases in which one of the authors took the role of requirements engineer, we outline an illustrative organisational perspective, ethnomethodology and an associated method for data analysis, grounded theory, both underpinned by a constructivist philosophy. there were two outcomes from the action cases. the first was an account of organisational life which represented the situated requirements (space in this paper prohibited a full version of the account). this was used in each of the two cases to inform both the sponsor of the study and the requirements engineer with respect to future design activities. the second was an articulation of the learning from the action cases (the objective of the paper). in the learning the authors identified the richness of situated requirements particularly in bringing about a greater shared understanding of the problem domain amongst users, sponsors and the requirements engineer. also we identified a new role for the requirements engineer incorporating the role of reflective bricoleur. as with the reporting of any action-based research there may also be additional lessons for the reader that the authors have not identified. it is important to iterate that the arguments presented here are not intended to persuade requirements engineers to use the illustrative framework given in this paper. the framework presented is but one means of intervention in, and understanding of, organizational life. we would maintain that critical reflection based on some explicitly stated theoretical or conceptual framework should be the basis for any requirements engineer who intervenes in organizational settings. we would extend potts and newsletter's (1997) conclusions and suggest additionally thai in the majority of requirements projects where the underlying philosophy is positivist ihere is slill a case for greater understanding of domain being achieved where the requirements engineer explicilly states his or her assumptions and reflecls on ihe implications of ihem. although the authors would maintain lhat ihis would be more limited lhan if iheir underlying perspective was conslruclivisl. for the authors the intention is to develop this work particularly in the area of knowledge elicilalion in large organisations where the sharing of contexlual situated knowledge amongst those involved in is/it projects and processes is critical. references adam, f. and filzgerald, b. (1996) "a framework for analysing ihe evolution of the is field can is become a stable discipline?" in coelho, j., jelassi, t., konig, w., kremar, h., o'callaghan, r. and saaksjarvi, m. (eds.) proceedings of the 4th european conference on information systems. lisbon. july 2-4. pp.1732. argyris, c. and schon, d. (1978) organizational learning: a theory of action perspective. addisonwesley, reading. auramaki, e., lehtinen, e. and lyylinen, k. (1988) "a speech-acl based office modelling approach" acm transactions on office information systems. vol. 6(2), pp. 126-152. baskerville, r. and wood-harper, a.t. (1998) diversity in information systems action research methods. european journal of information systems, vol. 7(2), pp. 90-107. brown, a. (1992) "grounding soft systems research" european journal of information systems. vol. 1(6), pp. 387-395. checkland, p.b. (1981) systems thinking, systems practice. john wiley and sons ltd. chichester, uk. checkland, p.b. and scholes, s. (1990) soft systems methodology in action. john wiley and sons ltd. chichester, uk. davis, g.b. (1982) "slralegies for information requirements determination" ibm systems journal. vol. 21(1), pp. 4-30. dobbie, m. and hughes, j. (1993) "realist ethnomethodology and grounded theory: a methodology for requirements" determination in information systems analysis' proceedings of the first british computer society conference on information systems methodologies. edinburgh, pp. 311-321. dzida, w. and freitag, r. (1998) "making use of scenarios for validating analysis and design" ieee transactions on software engineering. vol. 24(12). pp.1182-1196. easterbrook, s. and nuseibeh, b. (1993) "managing inconsistencies in an evolving specification" proceedings of the second ieee international symposium on requirements engineering. york. uk. pp. 48-55. frakes, b., batory, d., biggerstaff, t., kang, k. and neighbours, j. (1998a) "linking domain analysis and domain implementation" international conference on software reuse, pp. 348-349 frakes, w., prieto-diaz, r., and fox, c. (1998b). "dare: domain analysis and reuse environment". annals of software engineering (5), pp. 125-141. galal and mcdonnell (1997) knowledge_based systems in context: a methodological approach to qualitative issues, ai & society, vol. 11, pp. 104-121. garfinkel, h. (1967) studies in ethnomethodology. prentice hall. englewood cliffs, nj. glaser, b. and strauss, a.l. (1967) the discovery of grounded theory: strategies for qualitative research. aldine. chicago. 73 goguen, j.a. (1992) "the dry and the wet" in falkenberg, e.d., rolland, c. and el-sayed nasr-el-dein el sayed (eds.) information systems concepts: improving the understanding. elsevier science publishers b.v. (north-holland) pp. 1-17. goguen, j.a. and linde, c. (1993) "techniques for requirements elicitation" ieee international symposium on requirements engineering, pp. 152-164. goguen, j.a. (1994) "requirements engineering as the reconciliation of social and technical issues" in jirotka, m. and goguen, j.a. (eds.) requirements engineering: social and technical issues. academic press. london. hughes, j and howcroft, d. (2000) grounded theory: never knowingly understood new review of information systems research (forthcoming) hughes, j., o'brien, j., rodden, t., rouncefield, m. and sommerville, i. (1995) "presenting ethnography in the requirements process" proceedings of the second ieee international symposium on requirements engineering, york, uk hughes, j. (1998) the development of the gist (grounding information systems) methodology: determining situated requirements in information systems analysis. ph.d. thesis. information systems research centre, university of salford, salford, uk. hughes, j. and wood-harper, a.t. (1999) "systems development as a research act" journal of information technology. vol. 14(1) pp. 83-94 hughes, j., o'brien, j., rodden, m., rouncefield, m. and sommerville, i. (1993) "presenting ethnography in the requirements process" proceedings of the second ieee international symposium on requirements engineering. york, uk. pp. 27-34. jackson, m. (1988) "a discipline of description" requirements engineering. springer-verlag, london. vol 3, pp 73-78 jacobson, i. et al. (1992) object-oriented software engineering, a use case driven approach. addisonwesley. jirotka, m. and goguen, j.a. (eds.) (1994) requirements engineering: social and technical issues. academic press, london. mcdermid, j.a. (1994) "requirements analysis:orthodoxy, fundamentalism and heresy" in jirotka, m. and goguen, j.a. (eds.) requirements engineering: social and technical issues. academic press. london. pp. 17-40. miles, m.b. and huberman, a.m. (1994) qualitative data analysis. second edition. sage. uk. oliphant, j. and blockjey, d.i. (1991) "knowledge-based system: advisor on the earth retaining structures" computers and structures. vol. 40(1) pp. 173-183. pidgeon n.f., turner, b.a. and blockjey, d. i. (1991) 'the use of grounded theory for conceptual analysis in knowledge elicitation" international journal of man-machine studies. vol. 35(2) pp. 151-173. potts, c. and newsletter, w.c (1997) "naturalistic inquiry and requirements engineering: reconciling their theoretical foundations" proceedings of the ieee international conference on requirements engineering, pp. 118-127. reason, p.(1994) three approaches to participative inquiry. in denzin, n.k. and lincoln, y.s. (eds.), handbook of qualitative research. sage. london, pp. 324-339. regnell, b., kimbler, k. and wesson, a. (1995) "improving the use case driven approach to requirements engineering" proceedings of the second ieee international symposium on requirements engineering. york, uk. pp. 40-47. schon, d.a. (1983) the reflective practitioner. how professionals think in action basic books, usa. strauss a and corbin c (1998) basics of qualitative research: techniques and procedures for developing grounded theory, sage, london, 2nd edition. suchman, l. (1987) plans and situated actions: the problem of human-machine communication. cambridge university press. cambridge, uk. sutcliffe, a. and maiden, n. (1998) "the domain theory for requirements engineering" ieee transactions on software engineering. vol. 24(3), pp. 174-197. valusek, j.r. and fryback, d.g. (1985) "information requirements determination: obstacles within, among and between participants" proceedings of the sigcpr acm conference on end-user computing. minneapolis, usa. vidgen, r and braa, k. (1997) balancing interpretation and intervention in information systems research: the "action case" approach. in lee, a., liebenau, j. and degross, j. (eds.), proceedings of the ifip wg8.2 international conference on information systems and qualitative research. philidelphia, usa walsham, g. (1993) "ethical issues in information systems development the analyst as moral agent" in avison, d., kendall, j.e. and degross, j.i. (eds.) human, organizational and social dimensions of information systems development. north holland. amsterdam, pp.281-294. waters, m. (1994) modern sociological theory. sage. london, uk. 74 microsoft word ajis vol 16-1.doc australasian journal of information systems volume 16 number 1 2009 149 what is an ict professional anyway? clive boughton college of engineering and computer science, australian national university email: clive.boughton@anu.edu.au abstract the intention of this paper is to instigate ongoing discussion surrounding the connected topics of ict professionalism and the ict profession. part of that discussion needs to include suggestions of ‘the way forward’ for the development and recognition of an ict professional body and the way it should govern/support/protect the professionals within. introduction the intention of this paper is to instigate ongoing discussion surrounding the connected topics of ict professionalism and the ict profession. part of that discussion needs to include suggestions of ‘the way forward’ for the development and recognition of an ict professional body and the way it should govern/support/protect the professionals within. i want to start off by describing some of the types of people that i have personally met/observed during my time in the systems/software industry and also within academia. the type classifications are mine, are not formal, and carry no essential meaning other than what i have deigned to give them. they are like a loosely typed language: they only possess meaning in a chosen context, and i can change the meaning and context whenever i like. hence the classifications to be presented are not to be interpreted as authoritative. the point in presenting these ict types is to help to identify different angles to the accepted norms on the subject of professionalism, and to use them in a discussion on the state of professionalism and indeed the current state of the ict profession. to me individual professionalism (as a norm) requires a combination of several necessary characteristics, they being: • competency, • ethicity, • knowledgeability, • learning ability, • care, • and pride. i believe that none of these characteristics alone, or as a subset, fully describes a professional. however, it is common for ict people to attach to themselves the label of ‘professional’ whilst lacking one or more of the characteristics listed above. a more formal definition of professionalism australasian journal of information systems volume 16 number 1 2009 150 will be provided later in the paper, and it will be used to help explore the subject of ict professionalism. aside from professionalism in ict, it is not clear what the ict profession is. is it a combination of two (or more) different professions? is it merely a label given to a broad range of computer, software, communication networks, electronics, and other practices and sciences? again i will draw on more formal definitions to discuss the state of play of the ict profession later in the paper. the ict (information and communications technologies) types the following descriptions are provided in the hope that readers have perhaps met people with many of the same traits and that they ponder the connection between the traits and “professionalism”. the labels given to the ‘types” are intended to summarise overall capability and social interaction. the highly broadly technical and you know it they know their stuff and make sure everyone else knows they know their stuff. they have an excellent grasp of what they know, but it’s usually the highly technical/detailed information with which they interest themselves. presumably it provides a degree of comfort/control to know a whole heap of stuff with which few others have interest. they always have something to say when discussions move to their area of expertise and they nearly always make everyone feel inadequate or irritated on such occasions. such discussions don’t last long unless there’s another “highly technical and you know it” sort of person participating but then they tend to be the last people standing, when everyone else has gone. these people are usually competent and ethical, and often perform best under pressure – they love the challenges. they typically don’t make great team players/managers because their organisational abilities only apply to themselves. when praised for their performance their attitude is typically one of: “what did you expect?” diplomacy is not one of their strong traits. the highly broadly technical and you don’t know it they are quiet/unassuming and know their stuff but generally others don’t know that they know their stuff. they have an excellent grasp of lots of things, not just their ict knowledge – but few ever know. they perform their work with a very high level of competence and due diligence, and are usually very ethical – they hate pressure though. they are typically very self-critical even when they are told that they have performed excellently. these people often squirm when praised – but in the end they do appreciate being wanted. typically they will not volunteer their expertise and they’re just as happy to occupy themselves learning new stuff if no-one bothers to include them in a team/task. these people definitely cannot manage anyone other than themselves, but at least they know it. they also know that there’s a lot that they don’t know and they often worry about this. the highly narrowly technical and you know it a little like the “highly technical and you know it” types, but they are usually newbies to the game and/or more immature. every new problem put to them is solved using the ‘one’ technique they know really, really well. these people are often seen as a pain. they usually don’t perform well even when they are given the opportunity to use their best knowledge assets – probably because australasian journal of information systems volume 16 number 1 2009 151 they are not particularly good at solving complex problems. if these people resist learning new ideas, they become increasingly incompetent. the highly narrowly technical and you don’t know it as with the “highly narrowly technical and you know it” types, every new problem put to them is solved using the ‘one’ technique they know really, really well. the trouble is others don’t necessarily know that that is what is happening during a project and if not discovered early can be disastrous. these people are a pain because they stubbornly insist that they are right and that everyone else is wrong. again, just like the “highly narrowly technical and you know it” types, they usually don’t perform well even when they are given the opportunity to use their best knowledge assets simply because they are not particularly good at solving complex problems. these people typically resist learning new ideas and so remain incompetent. the big-picture broadly technical tend to be the visionaries. they possess/understand enough/plentiful technical concepts/information to be able to provide effective plans of attack to solving large problems without being swamped with detail. these people are typically the engineers of the ict industry and they usually possess a wealth of experience. they are competent and always prepared to learn more, but not usually detailed knowledge. they can be good leaders and managers, but they are best at analysis, architecture and design. verification and validation of ideas are not their strong point, but they are very happy for others to provide that detail. these people know what they know and don’t know, and are typically prepared to inform others of the same. sometimes their ethics are questionable, especially when they produce solutions to large problems where the broader stakeholder community is not recognised, or simply ignored. the big-picture narrowly technical unfortunately the industry is riddled with these people. they try hard to emulate the “big-picture broadly technical” but usually fail because they lack the experience or they have only ever worked within a narrow field of ict, or within a single organization. these people typically only provide the same conceptually limiting solutions to all large problems to which they are presented. put them outside their safety net and they flummox. they can behave unethically when under pressure or when others have great expectations of them – but this is often caused because they don’t communicate what they don’t know. the concepts only there doesn’t seem to be too many of these people about these days. however, they typically possess very limited knowledge and understanding of the technical, process, quality, and people elements of the ict industry. the try and make themselves seem important by producing a million (mostly crazy) ideas a minute. they do try and concentrate on human usage and efficacy aspects of ict. that is not to suggest that good concepts should not be produced by people who’ve never been trained in ict, but they are usually experts within other domains. the i can do usually new graduates, very enthusiastic and keen to make their mark. often quite smart but tend to be over-confident. these are good qualities for new professionals to possess, but they do have to australasian journal of information systems volume 16 number 1 2009 152 be controlled/directed so that the individual builds on good professional behaviour. if uncontrolled all sorts of disasters can occur not only for the individual but for teams and organizations. if newbies, then expectations are typically not well-aligned with reality – this is not limited to ict. however, there is a potential to assume that ‘i can do’ ict graduates are somehow smarter and ‘righter’ than engineers, scientists, lawyers, artists, etc., and their bosses. these people can possess new ideas and new technology, but this does not mean that they should be in control. the i can’t do. perhaps these people have chosen the wrong vocation. they lack confidence. they struggle to remember/understand almost anything, and they always need training. these people are not selfmotivated and show low initiative. they don’t want to be responsible for anything or anyone. under pressure they usually fail to perform. these people probably have all sorts of other problems that override their professional responsibilities. however, given the right opportunity and environment they can change. the other professional there is a broad set of this type of people. however, from a professional viewpoint these are often the people who both criticise the ict profession and contribute to the apparent parlous state of the profession typically by undertaking ict-related work themselves (when they are not competent to do so) or making serious decisions on behalf of ict professionals without seeking any advice/input from those professionals. what have these different observed types to do with professionalism and professions? well, for a start it seems apparent that there is a diverse range of people in the ict industry, possessing a diverse (not necessarily adequate) range of knowledge, abilities and personalities. on the one hand, many of the people working in the ict industry and academia belong to no professional body, such as the acs, acm or ieee. however, many seem to work professionally and demonstrate very high ethical behaviour – often higher than any of the professional bodies might promote/expect. on the other hand, quite a number of people in the ict industry do belong to a professional society [6], and possess minimal knowledge of any part of the ict domain, display little competence, and some even occupy positions of importance/influence over others who are far more knowledgeable and competent. often, for the latter ‘professionals’ it seems that personality is the most highly valued characteristic. interestingly, whilst working as a scientific professional within the more traditional engineering and scientific domains i observed that positions of importance/influence belonged to those with appropriate knowledge and competence – their personality traits were a lesser factor in gaining such positions. of course, this situation has another set of problems. i have had about 25 years experience as a software engineer at various levels. in all that time, although i have identified many different work practices, ethical attitudes and personalities, i have not been able to establish precisely what comprises an it or ict professional. hence the question associated with this paper: “what is an ict professional anyway?” since i am not sure of the answer to this question, it seems appropriate to start with one or more definitions of a professional. australasian journal of information systems volume 16 number 1 2009 153 the professional there are numerous definitions of “professional”. to begin with, the typical dictionary (in this instance the collins english dictionary) contains definitions for the word professional such as: 1. of, or relating to, suitable for, or engaged in as a profession, 2. engaging in an activity for gain or as means of livelihood, 3. extremely competent in a job, a person who engages in an activity with great competence, 4. undertaken or performed for gain or by people who are paid, 5. a person who belongs to, or engages in one of the professions, on interpreting such definitions i can understand why prostitutes, are sometimes deemed professionals. more definitive views of professionalism include strong reference to profession, professional development and professional bodies. such a view might include the following [1]: • professionals are considered experts in their chosen vocation/field. • professionals possess a broad range of (systematic) knowledge with a theoretical base. • professionals are responsible to the public and/or community. • professionals possess a high degree of autonomy regarding their decision-making and behavior. • professionals are governed by a code of ethics, which: o is a statement of rules and values. o intends to ensure a high quality of service. o intends to guarantee competency of membership, honor and integrity. o is an expression of a professions' principles and what it expects of its members. o emphasizes no personal gain at cost to others (co-professionals, clients, community etc.). • the professional's system of rewards is primarily recognition for building up appropriate knowledge and experience, and furthering the respect of the profession. • there is a system for testing the competence of members. maister [8] provides a more humanist-oriented view of professionalism, by de-emphasising the knowledge/skill aspects. to paraphrase maister slightly, he believes that “… real professionalism has little to do with which business you are in, what role in that business you perform, or how many degrees you have. rather it implies a pride in work, a commitment to quality, a dedication to the interests of clients, and a sincere desire to help.” an important indication of the nature of professionalism made by maister is that it is “… predominantly an attitude, not a set of competencies. … skills you can teach, attitudes and character are inherent.” the key traits of a professional are shown in table 1, where they are presented as a set of criteria for professionalism. all but the last of these have been taken from the texas library service [1]. the ultimate trait has been added by myself after having read maister’s views on professionalism [8]. it is interesting to apply the criteria in table 1 to some of the ict types described earlier. these are shown in tables 2 and 3, where again the resulting summaries are drawn from my own experience and observations. australasian journal of information systems volume 16 number 1 2009 154 training there is an extensive period of training, often after a combination of formal education, training and apprenticeship; usually in a higher education environment. intellectualism the intellectual component is dominant. autonomy professionals usually have autonomy in their work. judgment professionals, because of their training, education, knowledge and experience, may use their own judgment in determining the appropriate approach to their clients or customers. independence professionals can work independently and charge fees or they can be part of an organization. service professionals possess abilities to provide a valuable service to society and operate with little or no self-interest. dedication professionals are dedicated to services and institutions. pride professionals take pride in the quality of their work. honest & trustworthy professionals can be trusted to behave honestly in all matters. table 1: criteria for professionalism from texas library service [1] with the last criterion added by the author of this paper. training usually possess a technical based degree. also do much self-education. intellectualism not usually dominant. autonomy usually. judgment whenever allowed or are asked to advise, but more likely to criticize other peoples’ judgments. independence not usually. confidence in own abilities does not usually extend to taking on the responsibilities of being independent. service usually. dedication not usually. pride usually. honest & trustworthy usually. table 2: highly broadly technical and you know it professionalism australasian journal of information systems volume 16 number 1 2009 155 training usually possess a non-technical degree. don’t do much self-education. intellectualism very little. autonomy not usually but do try. judgment whenever the opportunity arises. these people make the judgments that the ict types in table 2 criticise. independence not usually. service inconclusive observation. dedication inconclusive observation. pride sometimes. honest & trustworthy often not honest about their knowledge/capability. table 3: big picture narrowly technical professionalism the reason for choosing the two particular ict types in tables 2 and 3 is simply that they are contrasting but nonetheless represent a significant percentage of the ict industry membership. slight variations within table 2 would result when applying the criteria to the other “broadly technical” types described earlier. similarly variations within table 3 would result when applying the criteria to the other “narrowly technical” types. the “concepts only”, “i can’t do”, and “other professional” types should probably be disqualified as professionals, let alone ict professionals. one might conclude that there are no real ict professionals. i doubt that’s true. however, because there is a confusion of diverse backgrounds and training of people working in the ict industry, it seems appropriate to create an effective way to identify the various professionals the ict professionals specifically. how do we distinguish an ict professional from other professionals working within the ict industry? perhaps the answer can be found by examining the profession that supports the ict professional. the profession professions, usually exemplified in the form of an association/society/body of like-minded colleagues, are typically created to represent and support those who are accepted as accredited members (‘qualified’ professionals), and to maintain professional integrity, values and recognition. hence, real professions have responsibilities to the professionals they purport to support. for example they: australasian journal of information systems volume 16 number 1 2009 156 • create structures of subcultures for professionals. • provide legal reinforcement for the activities of professionals. • strive to provide environments of public acceptance. • promote ethical practices. • define penalties for professionals who work against the tenets & practices of the profession. the traits of a profession are summarized in table 4, which has been taken from the american management association’s handbook of project management. exclusive control – esoteric and systematic bok (body of knowledge) members have a monopoly on understanding and applying the bok. autonomy of practice members control the standards of the society. norm of altruism members act in the best interest of client. authority over clients professionals control the client/practitioner relationship. distinctive occupational culture occupation is set apart by a distinctive set of norms, values, and symbols. recognition usually legal requirement for specific training and preparation prior to practice. table 4: traits of a profession from the american management association’s handbook of project management [2]. the software engineering body of knowledge [4] provides a characterisation of an engineering profession as supporting the follow components: • an initial professional education in a curriculum validated by society through accreditation • registration of fitness to practice via voluntary certification or mandatory licensing • specialized skill development and continuing professional education • communal support via a professional society • a commitment to norms of conduct often prescribed in a code of ethics according to abbot [3] professions begin when people recognise that they are undertaking a vocation/occupation that is not covered by any existing profession. perhaps this is the current state of the ict industry. the creation of a professional association/society defines the extent of practices and necessary competencies to perform those practices – thus defining a “competence territory” (see [2]). the process, or life cycle, of professionalisation typically consists of the following steps/phase (not in strict order): • create a full time occupation/vocation. • establish professional association. • create and enforce code of ethics. • control use of the (profession’s) name and the body of knowledge (bok). • develop recognised training procedures. australasian journal of information systems volume 16 number 1 2009 157 • win political, social, and legal recognition. a profession provides comfort/security to people who fulfil its requirements and obey its tenets. however, professionalisation can often be very torturous and long drawn out simply because of other existing professions. even when the first five steps of professionalisation stated above are largely fulfilled, gaining political, social and legal recognition can be a serious problem for the associated professionals – software engineering is a good example of such a profession. the status of a profession can change significantly with time, especially when it becomes too protective of its members and doesn’t respond appropriately to serious challenges of (say) mal-practice. giving any impression to society of cover-ups and lack of interest in upholding codes of ethics can only damage a profession. charges of malpractice need to be investigated without bias or conflicts of interest by professions, and the processes of investigation need to be transparent. of course, this may be considered a simplistic stance in that there are often mitigating circumstances for what appears to be malpractice. additionally, because of the environments in which they work, many professionals are placed under significant pressure to behave unprofessionally, typically by nonor semi-professionals. with time, professions are also affected by changing political, social and legal environments, and if they are too slow to move with those types of external changes, they may suffer diminished recognition. the ama handbook on project management [2] presents the concept of semiprofessions by examining the state of the occupation of project management using the traits of a profession from table 4. table 5 represents a summary of that examination. exclusive control – esoteric and systematic bok (body of knowledge) no – boks are beginning to be recognized but still highly contested. autonomy of practice no members contribute to the standards of practice. norm of altruism not usually – societal impact of failed projects not recognised. authority over clients not usually – project managers tend to work within corporations. distinctive occupational culture possibly – certain aspects exist. recognition not yet – pm not legally recognized as a profession in any jurisdiction. table 5: the status of project management as a profession from the american management association’s handbook of project management [2]. after having (myself) attempted to elicit (without success) a specific ict bok it would seem that the state of the ict profession is similar to that of project management – it is a semi-profession because, as summarised in table 6, there are elements of the ‘required’ traits, described in table 4, that are not completely apparent. certainly ict ‘practice’ is at the stage in the professionalisation path of being a full-time occupation/vocation. splitting ict into the apparent components of it and communications. there is at least one core/common it body of knowledge (australian computer society) but there seems to be no obvious communications bok although there is a webok australasian journal of information systems volume 16 number 1 2009 158 (wireless engineering bok within the ieee communications society). as an aside, the term ‘body of knowledge’ seems relatively new and so many older professional societies may either use an alternative term or not actually refer to any specific knowledge base. the latter is probably due to an acceptance that (tertiary) educational institutions will provide the primary body of knowledge to support a society and its membership. so, identifying bodies of knowledge that might underpin a professional society may not always be straight forward. a bok is very important to a professional society, not only in defining/creating bounds to essential knowledge of members, but also in helping to define required education/training programs for basic/extra qualifications for members at different levels of the society. the bok may in fact enable the identification of several related occupations/vocations for which different certification programs might be defined. it is one of the essential elements that a profession’s membership should control, and be prepared to consider, now and then, with changing technology and societal concerns. one of the important duties of a professional society is to ensure non-ambiguity of the name of the profession. a case in point is the profession of software engineering within australia. aside from the fact that computer scientists possess a different view of what software engineering is, there is also the issue that despite engineers australia accrediting software engineering degrees, there is no apparent support to disambiguate either the typical industry or the computer scientific usage of the name. ambiguity of a profession’s name can be damaging to the professionals who have decided to be educated/trained for the profession only to find themselves competing for positions apparently advertised for them, but open to those who are either less qualified or from different professions. recognition of the profession externally becomes more difficult when recognition internal to a professional society is not obvious. a professional society’s code of ethics is intended to ensure its members maintain competency and appropriate behaviour and consideration toward other professionals, clients and the community. together the bok and the code of ethics are important in both the mounting of, and defence against charges of unprofessional behaviour. any charge of misconduct etc. against a professional can only be successful if it is proved that best practices have not been followed and that codes of ethics have been contravened. defence of charges follow the opposite view. if unprofessional behaviour is proved then appropriate penalties should be imparted. all well and good, but i have only ever heard of one person in the australian ict industry being punished for misconduct – and then completely independent of any professional body. are there no/few charges concerning professional misconduct in ict? are the associated professional bodies defending charges so successfully that very few have been actually punished for professional misconduct. is it that legal processes are more likely to ascertain charges of professional misconduct and also deliver penalties – perhaps using associated ict professions’ boks and codes of ethics to do so? don gotterbarn [7] relates stories of people working in the ict/software industry who having been fired because of unethical practices, but with no intervention by appropriate professional societies to neither condone the company that instigated the act, nor impede/prevent the perpetrators of the unethical act(s) in simply moving to similar jobs at other companies. is it any wonder that there are sceptics concerning professionalism and professions. for example, according to prof. john quiggin [5]: “professionalism is both an individual characteristic and an australasian journal of information systems volume 16 number 1 2009 159 ideological position. the primary definition is that of individual professionalism: the idea that membership of a profession carries with it a set of internalised values that will be reflected in the way in which work is carried out and the ethical standards that are adhered to”. a statement seeming to support the concept of professionalism. exclusive control – esoteric and systematic bok (body of knowledge) not fully apparent – there are contributing boks such as swebok and possibly the acs it core bok, but there doesn’t seem to be a specific bok for ict. university degrees confuse the situation because of little consistency on base knowledge/courses. autonomy of practice no members sometimes contribute to the standards of practice, but not necessarily for community interests. norm of altruism not usually – still a general lack of best fit of overall approaches and technological decisions, and many ict projects still fail on cost and achievement of deadlines. authority over clients not usually – ict professionals tend to work to other (non)professionals who control the client relationship. distinctive occupational culture possibly – certain aspects exist, but the norms, values and symbols are confused. partly because ict is a multi-profession profession. recognition seemingly – ict is assumed to be a profession, or at least it is often referred to in that way. table 6: the possible status of the ict profession this is the opinion of the author and is based on a combination of long-term observation and the seemingly confused state of an ict bok. however, quiggin goes on to say that the idea of professionalism as an individual characteristic is false given the chance to pursue their own self-interest (narrowly defined to exclude such items as the approval of professional peers) individuals will do so regardless of any attempts to instill professional values. he also quotes the words of george bernard shaw, “the professions are a conspiracy against the laity”. “that is, adherence to professional values and ethics serves to advance the collective interests of the profession at the expense of society as a whole. exactly the same critique is made of trade unions.” when it comes to the ict profession/professionals, i don’t altogether agree with quiggin. i rather think that the ict profession is nebulous in terms of bok and lacks authority or the right sort of australasian journal of information systems volume 16 number 1 2009 160 recognition to ensure professional behaviour of anyone (society members and/or non-members) practising in the ict industry. i’ll come back to this point later in the paper. personally, i believe it is appropriate for professions to set expectations of knowledge, competence and behaviour. professions and the professionals within them should all encourage continuous learning, sharing of knowledge, informing of experiences, and reflection on episodes of good (and perhaps bad) ethical behaviour and the benefits (or disbenefits) to others and themselves. establishing and maintaining a bok and a code of ethics are the easiest activities for a profession to undertake. because humans are the enablers and disablers of professionalism, there is always going to be a spectrum of acquired knowledge, ethical behaviours and personalities, ranging from high competence, high ethical behaviour, and very empathetic, earnest individual professionals down to just being competent enough, on the border line of ethical, and somewhat unpleasant individuals – who perhaps should change their profession. the most difficult activities for a professional society to undertake seem to be related to maintaining the integrity of its membership (using the criteria of table 1 and the traits of table 4 as guides). i believe, and indeed have met several people working in the ict industry who rate as professionals according to table 1. however, i don’t currently know of any ict type professional body, that maintains any but a few of the traits listed in table 4. some conclusions about the state of ict professions/professionalism so far i have concentrated on trying to establish what a professional is (independent of vocation) and what a profession is. i’ve used the definitions that i’ve found in order to establish that (really) the ict profession is very young and is slowly evolving. in answering the question: “what is an ict professional anyway?” i have to admit i’m still not sure, but i think i have a reasonable idea of the sort(s) of knowledge, competence and ethics that someone who is labelled as an ict professional, should possess. however, i do not have a definitive answer. in perusing the acs core bok for it professionals, i’m not sure that i am helped in answering the posed question. the acs bok contains the following diverse areas of knowledge: • computer organisation and architecture • conceptual modelling • database management • data communications and networks • data structures and algorithms • discrete mathematics • ethics/social implications/professional practice* • interpersonal communications* • program design and implementation • project management and quality assurance* • security • software engineering and methodologies australasian journal of information systems volume 16 number 1 2009 161 • systems analysis and design • systems software the asterisks above indicate mandatory knowledge for all professional level it courses accredited by the acs. any one of listed areas of knowledge could constitute a lifetime of professional activity/work, and indeed such professionals exist. for example, within the software engineering and methodologies area of knowledge there is another recognised bok (swebok software engineering bok [4]) and it contains the following knowledge areas: • software requirements • software design • software construction • software testing • software maintenance • software configuration management • software engineering management • software engineering process • software engineering tools and methods • software quality again, any one of these knowledge areas could occupy a lifetime of professional activity/work, and again, such professionals exist. as indicated earlier, it is appropriate to define a bok. however, it is also appropriate to define levels of practice. the swebok [4] does include descriptions of practices and processes for each of its knowledge areas, but does not define levels of practice. at this stage the acs core bok for it professionals contains little in the way of appropriate practices/processes for each of the areas of knowledge nor anything about levels of practice. this is not intended to be a criticism, the swebok took 6-7 years to define, and with the help of well over 500 professionals. however, it does show that the acs bok is not yet complete. it seems appropriate that anyone possessing significant capability surrounding any one of the knowledge areas of the swebok might be an eligible ict professional. however, they’d obviously have to possess significant understanding of other (related) knowledge areas in order to do their jobs comprehensively. so what minimum depth of overall knowledge would anyone need to reach some base level of practice enabling professional status in (say) software testing? also, what other higher levels of practice might these base level professionals aspire to, that is both defined and supported by an ict professional society? i could conclude by simply stating that i do not believe the ict profession, as it stands today, is or is even capable of providing adequate support to ict professionals and non-professionals. until that happens i doubt i or anyone else will be able to answer the question – “what is an ict professional, anyway?” however, whilst this conclusion may be correct, it neither helps in maturing the ict profession further, nor in providing less confusion for those working as professionals and non-professionals in the ict industry. australasian journal of information systems volume 16 number 1 2009 162 further work it is apparent that significant work needs to be done to fully establish an appropriate ict bok and associated practices. to that extent, it should be noted that the australian computer society (acs) is currently in the process of establishing a more comprehensive bok than is currently available. this endeavour is important for the acs because (as a professional body) it accredits tertiary it programs throughout the country. in this prosess, existing boks such as swebok, which provides a very good basis for describing base knowledge, methods and practices for each of its knowledge areas, need to be taken into account,. in this and other related bok endeavours, a need exists to ensure input to the process by a broad range of stakeholders – not (say) just academics. the mixture of stakeholders needs to include people and organizations who are not only creators of ict-related knowledge, products or services, but who are the users/receivers and/or who are impacted in some way. regarding the matter of codes of ethics, it seems that most professional societies provide these, mostly as guidelines of acceptable behaviour and also as a basis for examination of accusations of unprofessional behaviour. on the one hand, codes of ethics can be used as blunt instruments, only to be made apparent when a professional member has committed some (alleged) malpractice. on the other hand, codes of ethics should be used to actively encourage greater, unwavering commitment to dedication, integrity, honesty and trustworthiness. obviously to establish a profession containing highly professional members requires education, not just to learn the wording of codes of ethics but also learn to use the codes of ethics as a framework to clearly promote what is expected of professionals and how they should behave in any given circumstance when ethical behaviour is at risk. all this is easily stated but hard to do. nonetheless ethical behaviour needs to be imparted and governed effectively in order to raise the bar on expectations of all ict stakeholders in regard to ict activities. so, it seems that ict professional bodies currently need an injection of governance practices, especially in being proactive about supporting current members and expanding membership to help ensure wider consistency of professional behaviours, appropriate practices and projects’ outcomes. in australia there is evidence that this is happening. aside from dealing with the bok, ethical behaviour and expectations, there is the matter of determining different levels of practice/skills regarding the broad range of capabilities among people working in the ict industry. some of the ict types i described at the beginning of this paper may better fit the job description of “technician” rather than (say) “engineer” or “scientist”. all of these roles should, of course, follow professional practices and abide by ethical standards. however, the purpose in providing a range of levels of practice is to set in place a reasonable framework to establish minimum requirements for being able to practice as a particular type of ict professional, and also determine what ‘endorsing’ criteria need to be met to advance to higher levels of practice. the concept of levels of practice is certainly not new, and has the same ‘old’ advantages within other more established professions of protecting all stakeholders. for example, it should not be possible for a new graduate of the ict profession to be placed in a managerial role requiring significant technical and people management experience, to be handling significant numbers of staff (including senior staff), and to also be expected to make technical decisions on topics of which they australasian journal of information systems volume 16 number 1 2009 163 only possess no/superficial knowledge. i have seen this occur on several occasions, and the result is usually not pretty because it affects a significant body of stakeholders, including the junior professional and the organization employing them. the junior can be “scarred’ for life professionally, and the organization for which they work rarely takes the blame for the inevitable failure(s). this sort of situation is caused through the combination of lack of ethics (of both parties) and an undetermined limitation to a role in which a junior ict professional can be involved. ethically, junior professionals must be able to both admit their inability and to resist the temptation/encouragement to be involved in a role for which they do not have the requisite knowledge or skill. ethically, the organization should not be able to advise a junior professional to undertake a role for which they do not possess the appropriate capability. so, if the ict profession is to gain traction as a true profession supporting true professionals, then steps need to be made now along the lines described above. the magnitude of such a task may seem insurmountable just now, and indeed it will take some significant time to evolve. the point is that the first steps need to be undertaken as soon as possible so as the ict profession maintains/improves the recognition as a profession which it currently does not deserve. references 1. texas library service training program http://www.tsl.state.tx.us/ld/tutorials/professionalism/prof.html 2. paul dinsmore and jeanette cabanis-brewin; the ama handbook of project management (2nd edition), chapter 18, 2006. 3. andrew abbot, the system of professions; university of chicago press, 1988. 4. the software engineering body of knowledge http://www.swebok.org 5. professor john quiggin (australian research council federation fellow in economics and political science at university of queensland) – johnquiggin.com, september 2003. 6. john ridge; australian computer society, opening presentation to etgovict 2008. 7. don gotterbarn; software engineering ethics research institute, presentation to etgovict 2008. 8. david h. maister; true professionalism; touchstone, 1997. the australasian journal of information systems (ajis) began in 1993 and was based on the simple realisation that there was a need for an academic ‘outlet’ for is research in australia australasian journal of information systems volume 18 number1 2013 5 using historical methods in information systems: a primer for researchers janet toland victoria university of wellington janet.toland@vuw.ac.nz pak yoong victoria university of wellington pak.yoong@vuw.ac.nz abstract this article describes the use of historical methods in information systems research and provides a practical example of how this technique was used in a recent research project. though the information systems researcher already has a rich cornucopia of research methods to choose from, historical research has the power to offer insights over and above those provided by other techniques. the researcher is forced to step away from a narrow focus on the research question in order to examine the “big picture”. this big picture approach means that recurring patterns are identified providing a broad set of findings that are applicable in many different settings. however the flip side is that historical research can have a lack of focus and does not always offer immediate answers to specific research questions. this paper provides guidelines for the use of historical methods by information systems researchers by demonstrating how the seven step approach developed by mason, mckenney and copeland was applied to an historical research study which explored the relationship between icts and regional development in new zealand between 1985 and 2005. this research reveals the value of historical research for information systems researchers by showing the effects of long term social trends on ict use. it also highlights some of the pitfalls that potential users of historical research need to be aware of such as gaps in the data trail and the questionable credibility of some historic records. keywords: historical research, historical methods, information & communication technologies, new zealand introduction information systems have been defined as “a set of interrelated components that collect, manipulate, store and disseminate data and information and provide a feedback mechanism to meet an objective. the feedback mechanism helps organisations achieve their goals, such as increasing profits or improving customer service” (stair et al. 2011 pg 5). the discipline is relatively young, and the technologies being researched are ever changing. information systems researchers may see little value in going back to look at technologies that are now obsolete. why carry out research into cobol programming, when everyone is now using c#? however information systems is more than the study of technology alone; the interplay between technology, systems and organizations is multi-faceted and complex; the availability of a new technology on its own does not guarantee it’s successful adoption the article sets the scene by first discussing the origins of the use of historical methods in information systems and the founding work done in business history and the history of technology. it then goes on to describe a research project which used historical methods to investigate the contribution of icts to mailto:janet.toland@vuw.ac.nz mailto:pak.yoong@vuw.ac.nz australasian journal of information systems volume 18 number1 2013 6 regional development in new zealand. in particular the seven step approach developed by mason, mckenney & copeland (1997b) is discussed in detail. this is followed by a reflection on the lessons learned while using this approach. the article concludes by considering what new insights the use of the historical approach can bring to the study of information systems. the origins of historical methods in information systems historical methods consist of a collection of techniques and approaches which draw on both traditional history, and social research. the methodology was first developed in the nineteenth century by social thinkers such as marx, durkheim, and weber (neuman, 2003). there has been a resurgence of interest in historical methods in social science since the 1970s, when researchers began to recognise the limitations of methodologies such as structural functionalism and economic determinism, which take a static view of society. increasing political conflict between western nations meant that researchers became interested in exploring social change, and looked for a methodology that took into account historical and cultural contexts. historical methods provide a powerful set of tools for addressing broad, big picture questions (neuman, 2003). the most well-known example of the use of historical research in information systems is the work carried out by mason, mckenney & copeland (1997a, mason et al., 1997b). however historical comparative research has long been used by other disciplines. the two areas most closely related to information systems are business history and the history of technology. business history is concerned with understanding the interplay between economics and individuals, organisations and wider society, while the history of technology is more concerned with the technological artefact itself. business history business history is generally agreed to have begun as a discipline in the 1920s at harvard business school (hunter et al. 2006) where it grew in tandem with the use of the harvard case method as a teaching tool. each case would present issues faced by a particular organisation and ask readers to put themselves in the shoes of key decision makers. however business history has grown beyond focussing on a single organisation and the key individuals within it to encompass the broader perspectives of the industry sector alongside national and global perspectives. many researchers regard alfred chandler’s “strategy and structure” an account of emergence of the multi-divisional firm in american corporate history as the founding stone of the discipline (chandler, 1962). the early days of business history focussed on individual entrepreneurs and the organisations they founded (yates 1997). since the 1990s’ the emphasis has shifted from significant individuals to institutions and organisations. though this approach opened up new areas of research there was a concern that it could lead to institutional determinism if taken to extremes, meaning that individuals are portrayed as mere pawns at the mercy of the organisations they belong to (yates, 1997). researchers (orlikowski, 2000, yates, 2005) have used methodologies such as structuration theory to address this unease. history of technology much of business history is driven by economics, an approach which has worked well for some but has been criticised by others for having a narrow focus on the market (yates, 1997), an alternative but related field is that of the history of technology which focuses on the technological artefact. as far as information technology goes the majority of material published in this area tends to concentrate on hardware and software development. misa (2007) identifies three thematic traditions that have emerged in the history of computing in the last twenty-five years. the first phase was machine australasian journal of information systems volume 18 number1 2013 7 centred and concentrated on hardware and software, in the second phase looked at the “information age” characterised by manuel castells information age trilogy (castells, 1997). the third theme was taken up by historians who began to ask the question “how did certain institutions shape computing?” organisations which received particular attention were the us military services, the national science foundation and ibm. both business history and the history of technology have been criticised for having a technologically deterministic approach, where technology is seen as an independent variable that changes structures such as society, firms and the organisation of work (yates, 1997). in reaction to this many researchers in the history of technology field adopted the social construction of technology approach (bijker et al., 1989, hughes, 1994). the use of the social construction approach reflected three different trends, the first was a move away from a concentration on the individual entrepreneur, the second represented a move away from technological determinism, and the third trend was to study technological development as a whole, rather than making distinctions between technical, social, economic and political aspects. thomas parke hughes used a systems approach to integrate technical, social, economic and political aspects in his studies of the different ways in which electric power networks spread across western countries (hughes, 1983). historical methods in information systems the use of historical methods in information systems was pioneered by mason, mckenney and copeland in their studies of bank of america, lyons electronic office (leo) and american airlines (mason, 2004, mason et al., 1997a, mason et al., 1997b, mckenney et al., 1995, mckenney et al., 1997). their approach was also used in a study of the use of it in texaco within a forty year period (hirschheim et al. 2003; porra et al. 2005; porra et al. 2006). mason, mckenney & copeland, based their argument for the use of historical methods on the work of joseph schumpeter who saw capitalism as being characterised by “gales of creative destruction” where the economy is radically altered by innovations in products and/or processes, resulting in a fifty-five year cycle of creation, growth, and destruction, known as a kondratieff wave. schumpeter’s theories were based in turn on the work of nikolai kondratieff, who argued that the possibilities of any given generation of technologies become exhausted approximately every fifty-five years (hall, 1998). as the developments in icts form the most powerful force of creative destruction in the last fifty-five years, and can be regarded as the fifth kondratieff wave, this is of direct relevance to researchers in the field of information systems. building on schumpeter’s ideas of radical innovation, mason, mckenney & copeland advanced the concept of successful entrepreneurs who develop a “dominant design” which will change the market place. central to this approach is the concept of three characters who must be present in an organisation for successful technological change to occur, the “maestro”, the “executive” and the “supertech”. the executive is an inspirational leader with a vision for their business, the supertech has the technical skills to use it to put their vision in place, and the maestro is the bridge between the two, making the executive aware of the possibilities offered by it, and translating their vision into a form the supertech can understand. the approach includes the use of a seven step framework to carry out, analyse and present the research. it is this aspect of their research, the seven step framework, that forms the focus of this article. when introducing their approach they explain how it differs from histories of technologies, and though they acknowledge the contribution of business historians such as alfred chandler and joanne yates (chandler, 1962, yates, 1989) they do not situate themselves clearly within the field of business history or acknowledge that there might be alternative approaches to the one they have developed. their interpretation of historical methods is very dependent on the idea that it can australasian journal of information systems volume 18 number1 2013 8 produce a radical change, and though they have produced a large number of case studies to support their theory (mckenney et al., 1995) other researchers, notably joanne yates and james cortada has shown that many successful organisations have taken an incremental approach to the adoption of it (chandler and cortada, 2000, cortada, 2007, yates, 1989, yates, 2005). in any organization, the understanding of the present is facilitated by studying the past, and gaining an awareness of the long-term economic, social and political forces that shape events. the benefit of using historical methods is that deep and wide insights are obtained into the area being researched. for information systems researchers’ new perceptions can be gained by considering the long term cultural context in which their research is situated. an application of mason, mckenney & copeland’s historical methods approach was used to research the contribution that ict made to regional development in new zealand. two contrasting regions were studied over a twenty year period in order to understand the relationship between new developments in information and communications technology and the changes in social and economic development. for the study of regional development the long-time perspective, which considers the development of social capital, cultural values, and the build-up of social networks made historical methods the most suitable approach. historical methods highlighted how the two regions had changed over time and the positive and negative consequences of those changes. as a reflective methodology, historical methods enables the researcher to obtain a deeper understanding of these issues than would be obtained by using a case study approach. learning regions in new zealand the historical methods approach developed by mason, mckenney & copeland was used for research which investigated the contribution that icts made to development of “learning regions” in new zealand. the term “learning region” is widely used in the field of economic geography to identify regions that have been economically successful over a period of time, and that have successfully adapted to changed circumstances (cooke, 1996, florida, 1995, macleod, 2000, storper, 1995). such regions are characterised by the following factors: a competitive strategy based on learning; intense intra-regional linkages; capacity for innovation; creativity in both arts and sciences; efficient information flows; and regional norms and values that provide stability. these are all long term processes which can interact in a way that results in certain regions becoming consistently successful over time. icts have the potential to make an important contribution to the development of each of these factors. the term learning region was first coined by academic authors (florida 1995; morgan 1997; storper 1995) working in the fields of innovation studies and economic geography. the concept of the “learning region” is ambiguous and found in a variety of different contexts. there is no single definition of a learning region, however a common strand in the literature is that such regions have an explicit commitment to placing innovation and learning at the core of development (larsen 1999). a learning region would generally consist of a network of inter-firm relationships, supported by social capital and trust, and kept dynamic by a continuous process of interactive learning. the concept of the learning region is particularly relevant for new zealand, as a small country located at the bottom of the south pacific it faces particular problems in attempting to integrate the national economy into the global economy. primary industries dominate, and exports of meat and dairy products make a large contribution to new zealand’s economy. however industries such as forestry, horticulture, fishing, manufacturing and tourism have become increasingly significant, and over the past decades, many new industries have emerged and grown strongly, including software, biotechnology, electronics, marine, education exports, media/film and wine. new zealand’s isolation australasian journal of information systems volume 18 number1 2013 9 and physical distance from major trading partners’ means that new zealand’s predominantly small firms wanting to move into export markets face big costs. regions are affected as technological innovation and increased competition lead to business centralisation. also industry rationalisation and market deregulation have encouraged skilled people to leave rural regions for broader educational and employment opportunities in major cities (schollman et al. 2002). additionally enhancements and upgrades of physical infrastructure in rural centres have not kept pace with technological progress (due to small market sizes, lack of critical mass, and no population growth) and have led to further population out migration. this has caused problems for some of new zealand’s rural regions, and in some cases led to a vicious cycle of decline. in urban regions all new zealand cities have low-income areas and pockets of deprivation. in both urban and rural areas, mãori and pacific islanders are disproportionately represented in the disadvantaged population. the new zealand government has implemented several initiatives to help to develop a knowledge society, encourage innovation, build up regional economic development, and improve usage and access to ict. though the concept of the learning region is not explicitly stated, these initiatives are in line with the thinking that lies behind the idea of learning regions. the overarching aim is to return new zealand’s per capita income to the top half of the oecd rankings and maintain that standing. the use of ict is seen as central to all of these developments. many of these initiatives are focused at the regional level, for example project probe was a 2002 joint initiative between the ministry of education and the ministry of economic development (istart 2004). the aim was to roll-out broadband communications to schools and rural communities, with a particular emphasis on closing the disparity between rural and urban schools. in order to attempt to assess the long term success of these initiatives this research focussed on regional new zealand. two contrasting regions were investigated, one urban region, wellington and one rural region, southland. data was collected over a twenty year period, from 1985 to 2005. in the regional setting tacit or soft knowledge is more easily transferred than in a national context. this is because social interaction and exchange of information is easier and cheaper (oughton et al. 2002). these soft peoplebased social networks take time to develop, and are likely to have a significant influence on the use of the ict networks that are based within a particular region. the focus of the research was on the interplay between these soft social networks and the hard technology-based ict based networks operating within the regional setting. the central research question was: what role do information and communication technologies play in the development of learning regions? how historical methods were used in this research mason et al (1997b) have laid out clear guidelines for the researcher using historical methods, seven steps are identified which take the researcher through the stages of deciding on the research question, gathering and analysing the data, and writing up the results, these are outlined in table 1. this paper will show how this seven step model was adapted to research the topic of learning regions in new zealand. though these steps are presented in a linear fashion, when carrying out real-world research, there will always be an overlap and iteration between the steps. step one: begin with focusing questions focusing questions were arrived at by an inductive process of searching the literature, relating to the new zealand context, and using the questions asked in the study of texaco (porra et al. 2006) as a guideline. a number of questions were posed, some examples of these questions are: australasian journal of information systems volume 18 number1 2013 10  what were the significant changes in the new zealand economy between 1985 and 2005?  how have new developments in ict been adopted in regional new zealand?  what significant changes have occurred in human and social capital in regional new zealand between 1985 and 2005? step activities 1) begin with focusing questions. the questions asked are going to be about change, as history is primarily the story of change. inductive thinking is generally associated with the interpretive paradigm, and involves the researcher identifying categories, or patterns in data, that seem suitable candidates for further investigation. 2) specify the domain for the enquiry. in the studies carried out by mason et al (1997a) and (porra et al. 2006) the primary unit of analysis is an individual organisation. the researcher needs make decisions about what will be included in the domain, and what is the appropriate time span for the study. 3) gather evidence, using both primary and secondary sources. primary sources are those that came into existence during the time to which they refer, and secondary sources are those written by historians about a period in the past. primary sources can be public documents such as annual reports, statistics and academic articles, which are organised around a timeline. secondary sources can be slotted into this timeline and include less public information such as letters, budgets, and data collected from individual interviews. 4) critique the evidence. is it authentic and credible? it is common to find that evidence is contradictory, irrelevant or incomplete. many of the best storytellers favour accuracy less than they favour a gripping narrative. techniques such as counting the number of times an observation was made, determining the credibility of sources, and establishing whether there are meaningful relationships between the different parts of the evidence can be used to assist with this. 5) determine patterns using inductive reasoning. this is one of the central steps, though one of the most difficult. the task is to explain what happened, and how and why it happened. this can be done using a number of different tools; three of the most popular are conceptual frameworks, causal chain analysis, and establishing empathy with the main participants. a conceptual framework can be used to organise facts, and to concentrate attention on the essential areas to be explained. a causal chain is a type of conceptual framework that shows the sequence of events that produced the effects, results or consequences observed. conceptual frameworks and causal chains can be developed in advance independently of the phenomena to be explained, and used as an explanatory framework, or they can be used as ideal types around which historical data can be organised. a third approach is to try to achieve empathy with the characters in the study. this means imagining how events might have appeared to those who actually experienced them. 6) tell the story. this entails bringing together the results of evidence gathering, empathy, and causal chain analysis to form a narrative. 7) write the transcript. the historical method is part of the hermeneutic tradition in that it treats the world as a script. every written account takes its place in the context of a network of other written accounts that attempt to explain the relationships between living generations and their predecessors. table 1: seven step approach to historical methods (mason et al, 1997) in order to address these research questions it was decided to use the concept of the ideal type as a basis for data collection and analysis. a theoretical framework of an “ideal” learning region was built up and the two actual regions were evaluated using this framework. the framework was developed by australasian journal of information systems volume 18 number1 2013 11 reviewing 23 academic articles mainly from the economic and regional geography literature that covered the concept of the learning region in order to identify common terms and themes (christie et al. 2001; cornford 2000; florida 1995; hudson 1999; keating et al. 2002; lagendijk et al. 2000; larsen 1999; lever et al. 1999; macleod 2000; malecki 2002; maskell 1999; maskell et al. 1999; morgan 1997; oinas et al. 1999; organisation for economic co-operation & development 2001a; rio 2001; saxenian 1994a; schollman et al. 2002; sokol 2002; storper 1995; thompson 2002; wolfe 2000; wolfe 2002). twenty two common terms were identified and these were ranked according to how often they were mentioned, and then grouped into six categories. these categories are presented as the 6-i framework, shown in table 2. a more detailed explanation of this process can be found in (ref removed). the framework groups characteristics that a learning region should possess into six categories: interconnecting; informing; innovating; interacting; infrastructure and income. the “6-i” framework was used as a basis for data collection and analysis. interconnecting the existence of networks between businesses, customers, suppliers, community groups and local authorities informing features related to learning such as the presence of embedded tacit knowledge, transfer of best practice, and presence of knowledge workers innovating evidence of ability to create or adopt innovations, the presence of entrepreneurs, and a competitive culture interacting the presence of strong social capital, high levels of trust, a common regional culture, and professional networks infrastructure hard infrastructure in terms of transport and telecommunications links, plus soft infrastructure in terms of regional norms and conventions income economic features such as per capita income and employment levels table 2: the 6-i framework step two: specify the domain for the enquiry neumann (2003) distinguishes between micro-level, meso-level, and macro-level theories. with the selection of the appropriate level being based on time-span, numbers of people involved, and geographical area covered. the larger the level of the theory, the more abstract the concepts it deals with, micro-level theories would be used to explain the interactions between small numbers of individuals, whereas macro-level theories explain society-wide issues. at the meso level two contrasting regions were selected one urban region, wellington and one rural region, southland. southland is a remote rural area located at the south of the south island. farming is the mainstay of the local economy; the region also has the country’s only aluminium smelter. southland frozen meat is also a major employer. the greater wellington region (hereafter referred to as wellington) is located at the south of the north island and includes the capital city, wellington. the region includes a wide range of different socio-economic groups; there are some high income areas but also areas of deprivation. as wellington is a capital city, the public sector is of particular importance, as is the service sector. wellington is the second most important centre for the it industry in the country after auckland. as in southland, tourism is of growing importance to the region, and is often associated with events such as the rugby sevens or the arts festival. australasian journal of information systems volume 18 number1 2013 12 new zealand’s major exports are primary products and the rural sector has traditionally been the most important area of the economy. as a rural region southland provided data about how the rural economy changed over the twenty year period studied. at the same time the country was attempting to diversify its export base, and it and biotechnology were viewed as offering great potential. as an important centre for the it industry, wellington provided data about changes during the period studied. the second reason for selecting these two regions was that they both had a strong reputation throughout the country for being innovative adopters of ict networks. in 1995, wellington was one of the first cities in the world to set up a broadband network in its central business district, and in 2003 southland made a bold decision to implement a wireless broadband network throughout the region. the intention was that the scope of this research, which is located in a regional context, should be geographically located at the meso-level, which provides a link between the micro (individual) and macro (national) levels, and therefore connects the particular with the general. however in practice it proved difficult to separate the three levels. adding to this difficulty is the fact that new zealand’s small population of around four million means that regions are more inter-dependent than in more densely populated countries. the integration of micro and macro level data is traditionally a feature of historical comparative research. issues are considered at both a society wide and at an individual level (neuman 2003). for this research the impacts of macro level national policy around issues such as local government restructuring and availability of broadband had to be considered alongside the regional meso level. step three: gather evidence, using both primary and secondary sources during the first round of data collection in 2006, twelve in-depth semi-structured interviews were conducted with key figures involved in the adoption of ict networks. the interview questions addressed a common set of themes including availability of infrastructure, the extent of linkages between local organisations, regional culture, commitment to learning within the region, and the adoption of innovative ideas. the interviewees worked for a range of organisations including local and regional councils, telecommunications providers, schools, and community groups as shown in table 3. some of the interviewees were selected for their knowledge of the local situation in southland or wellington; others had a national focus. location focus role wellington national telecommunications consultant wellington national rural telecommunications expert auckland national rural telecommunications expert southland southland manager local government southland southland ict policy – local government southland southland community sector – local government southland southland telecommunications provider wellington wellington consultant – wellington regional strategy wellington wellington manager – local government wellington wellington ict policy – local government wellington wellington community sector – local government wellington wellington school – ict teacher table 3: location and role of interviewees australasian journal of information systems volume 18 number1 2013 13 the second round of data collection focused much more strongly on the regional level and also brought in the historical aspects. the aim was to build up a history of the development of ict networks in the regions of wellington and southland in the twenty years between 1985 and 2005. the strategy adopted was to carry out both macro and micro level analyses of events in both regions over a twenty year period (rooney, 1996). primary sources were used to give an overall picture of developments in each region over the whole period, and for three selected years 1985, 1995 and 2005; a more detailed micro analysis was carried out. the idea was to see how those factors that had been identified as being relevant to the development of learning regions were changing in each region during the period covered by this research. in order to obtain an even amount of material for each of the three selected years, and to cope with the problem of information overload it was decided to restrict the search to three regional newspapers (dominion/dominion post, evening post, southland times) and one national magazine (national business review). the advantage of using newspapers is that they provided a breadth of coverage that was not available from other sources. the material from the newspapers was complemented by national and regional reports produced by a range of organisations such as statistics new zealand, independent economic consultants, non-governmental organisations, professional societies and voluntary groups the initial database held 3,033 items and when coding was completed this was reduced to 2,442 items. the breakdown of the number of articles for each category and region is shown in table 4 (note that in 2005 the dominion and evening post combined to become the dominion post which partly explains the drop in numbers). what is interesting to note is that even though the two newspapers were regional, more than half of the articles selected had a national rather than a regional focus. the fact that wellington is the capital of new zealand is also significant. generally initiatives by the national government were categorised as national rather than regional even though they were located in wellington. the numbers of articles collected for each category give a broad indication of the category’s importance. though these numbers are of no hard scientific value, counting the number of times a point is mentioned is one of the techniques used in historical research to establish trustworthiness. counting is also a technique recommended by miles and huberman (1994) as a tactic for generating meaning. the same story was often reported in a number of publications, and there were often multiple articles about the same event. n– national, s – southland, wwellington table 4: newspaper statistics for 1985 to 2005 1985 1995 2005 overall total n s w tot n s w tot n s w tot interconn 97 16 27 140 44 11 43 98 41 13 5 59 297 informing 95 39 46 180 32 45 74 151 37 24 11 72 403 innovating 123 21 38 182 104 18 34 156 112 28 32 172 510 interacting 33 53 69 155 7 35 77 119 4 29 12 45 319 infrastruct 53 45 58 156 8 43 56 107 58 40 23 121 384 income 163 42 64 269 41 83 39 163 48 23 26 97 529 total 64 216 302 1082 236 235 323 794 300 157 109 566 2442 australasian journal of information systems volume 18 number1 2013 14 after the searches had been carried out the next step was to build up a picture of the situation in each region for each of the three years using the “6-i” model as a framework. this was done by combining the results of the searches ordered by category, sub-category and year for each of the two regions and the national situation. this data was used as a basis to describe developments in that region during 1985 to 2005. as with the first round of data collection further refinement took place during the writing up process, duplications and overlaps were identified and articles were reassigned to different categories as appropriate. the data collection process is summarised in table 5. stage method analysis first round interviews with significant figures working in each region. data coded using nvivo with 6-i model used as overall conceptual framework. second round regional newspapers scanned for relevant articles for years 1985, 1995 and 2005. data coded using access database with 6-i model used as overall conceptual framework. national & regional reports produced by government, economic development bodies, ngo’s and statistics new zealand collected for period from 1985 to 2005. data used to supplement and confirm data from newspaper articles. table 5: data collection rounds step four: critique the evidence. is it authentic and credible? historical source material consists of primary and secondary sources. historians select the events and people that they consider important. by doing so they don’t so much recreate the past as rediscover it, and to some extent colour it with their own set of value judgements. the historical researcher’s most important role is to choose reliable sources, in order to create reliable narratives about the past (howell et al. 2001). there needs to be a systematic approach to gathering data, as collecting only the most compelling evidence can result in material that is unrepresentative (wenger et al. 2000). utilising the authoritative source only is not a wise approach. evidence should be collected from a wide range of sources, each of which will have their own strengths and weaknesses (tosh 2000). any source material collected should be subjected to both external and internal criticism. the authenticity of the evidence is determined by external criticism, whereas credibility is established by internal criticism (shafer 1980). the use of external criticism involves establishing whether a document can be traced back to the purported originator, establishing whether it is consistent with known facts, and studying the form of the document (tosh 2000). internal criticism consists of trying to establish the author’s meaning and making a judgement as to the intentions and prejudices of the writer (tosh 2000). an overview of the two techniques is shown as figure 1. australasian journal of information systems volume 18 number1 2013 15 figure 1: internal and external criticism (from neuman, 2003, p.421) regional newspapers are an authentic primary source. the location and time of reporting are recorded, and for many of the later articles the author is also recorded. statistics from organisations such as statistics new zealand and the organisation for economic co-operation and development (oecd) can also be regarded as authentic as they have official national and international approval. they also tend to have good response rates, for example statistics new zealand surveys generally get a greater than 80% response rate. the statistics were used to cross check and confirm assertions made in the newspaper articles. as previously mentioned the large number of articles used increases authenticity, the articles can be counted, if an issue was significant it would be reported on a number of times, both within one newspaper and across different newspapers. step five: determine patterns using inductive reasoning mason et al (1997b) identify three different methods for determining patterns: conceptual frameworks; causal chain analysis and establishing empathy. with the conceptual framework and causal chain analysis generally a model or analysis would be developed before collecting the data, and then the material collected would be compared against the original model. the establishing empathy approach attempts to build an understanding of the motivations of the key historical figures in the study, and is generally carried out after data collection. mason et al.’s research (mason 2004; mason et al. 1997a; mason et al. 1997b; mckenney et al. 1995; mckenney et al. 1997) was conducted at the organisational level and used the approach of establishing empathy with individuals in those organisations. this approach was also used by hirschheim et al (2003) in explaining the history of texaco through the eyes of its chief information officers. this research was conducted at the regional level, and the establishing empathy approach was problematic due to the large number of individuals who contributed to regional development in widely different roles. the approach chosen for this research was the conceptual framework, which uses the concept of an ideal type to organise and interpret the data (mason et al. 1997a; mason et al. 1997b). the ideal type is presented as the “6-i” model, and the data collected was categorised using this model, within the context of both time and geographical location. the data collected was then evaluated against the definition of the ”ideal” for each of the six categories, and the results for each of the two regions and the national situation were compared. the empathy approach does produce a persuasive story centred on the actions of key individuals however it can be rather subjective as the researcher has to put themselves in the decision-makers shoes. a conceptual framework is more objective, makes the research more transferable and adds rigour to the research. australasian journal of information systems volume 18 number1 2013 16 step six: tell the story one of the strengths of the historical approach is the compelling story that is produced. this narrative tells the story of the economic and social development of two regions in new zealand (rural southland and urban wellington) with a particular focus on the role of hard ict-based and soft people-based networks in regional development. in order to assess the role that ict was playing in the development of learning regions, the two regions were assessed against the “6-i” framework of an “ideal” learning region shown in table 2. a discussion of the results in each of the six categories together with an overview follows. interconnecting an interconnected region will have active networks operating at international, national and regional levels. throughout the period studied new zealand developed more international connections, and ict made a major contribution to the build-up of international, national and regional networks. ict had a major effect on organisational form as businesses and public sector organisations throughout the country became more networked. research based networks linking universities, crown research institutes and large organisations were present at the national level. active networks were found in both regions, though these tended to be based in a particular sector for example the education sector. those regional networks that went across a number of sectors tended to focus on unemployment and retraining and had often been initiated by local government. though there was evidence of clusters for example the effort by a chamber of commerce to form a high technology zone in the wellington region, and the formation of the southern wood council in 2001, no such initiatives had been consistently maintained throughout the period studied. the results for this category were generally positive, new zealand did become more connected at global, national and regional levels, and icts played a major role in facilitating these connections. informing in an informed region there will be a commitment to learning and evidence of knowledge sharing between different organisations within the region. there was a strong commitment to education that came through at both the national and regional levels, and as with interconnecting, ict played a major role. at the regional level ict was used by schools to build networks and to share resources; at the national level the education sector was viewed as the leader for new developments, such as the rollout of broadband into rural areas. in the education sector, knowledge sharing was enhanced by the use of collaboration software such as videoconferencing and interactive whiteboards. the use of collaboration software demonstrated that tacit as well as explicit knowledge was being exchanged. however, many interviewees made it clear that technologies such as videoconferencing could complement but would never replace face-to-face contact, indicating that there were limitations on the use of ict networks to exchange tacit knowledge. another factor that should be found in a learning region is evidence of a bottom -up approach to knowledge sharing and transfer of best practice. this was definitely evident in the education sector in both regions studied. in the wellington region, primary and intermediate schools under the direction of the ministry of education had formed an ict cluster in order to share ideas and resources. in southland a survey carried out prior to broadband adoption showed a strong desire on the part of local schools for collaboration. commitment to learning at individual, organisational and regional level is another feature of a learning region. the commitment to learning was found most strongly at the regional level australasian journal of information systems volume 18 number1 2013 17 particularly in southland, where the regional economic development body, venture southland demonstrated a strong commitment to improving the educational level of regional residents. the one weak area was skills shortages, at both regional and national levels. out-migration of skilled workers in sectors such as health and it was a major issue. this migration was strongly influenced by economic conditions. when times were good people stayed, but in lean times they looked for better opportunities offshore. though the problem of staff shortages was felt across the country, it was clear that the problem was much worse for the rural region of southland. however southland worked hard to address this issue with a number of initiatives, one of which was the introduction of a zero fees policy at the regional higher educational institute in 2001, these efforts paid off and by 2005 the population of southland had stabilised. in summary, the results for this category were generally positive. throughout the period skill shortages decreased and investment in education grew. there was evidence of ict enabled regional knowledge sharing in both regions. innovating in an innovative region there will be evidence of new ideas, in terms of both products and processes, and the local culture will encourage competition. there should be evidence of entrepreneurial activity and a strong commitment to research and development. a broad definition of innovation was used, which included adopting innovative ideas from other regions or other countries; this was felt to be most appropriate for the regional context. though there was some evidence of innovation, especially in the it sector, and certainly strong evidence that new zealanders were world leaders in terms of adopting new ict technology, neither region demonstrated the density of innovative activity that would be expected in a learning region. the success of individual it companies was not built on and developed. this was not helped by the low investment in research and development at the national level. although the government’s financial commitment to research and development could be questioned, there was strong evidence of entrepreneurial activity in new zealand, though it was noted that there did seem to be a slowdown in the rate of patent applications between 2000 and 2005, despite a growth in the number of researchers. it is difficult to get an accurate measure of innovation levels as many software companies don’t bother to apply for patents. in pure economic terms individuals and businesses in the more populated urban regions of new zealand were much more likely to be adopters of new ideas and technologies such as broadband than those in the less populated rural regions, even taking account of population size. however, social capital also made an impact. if a strong social network was in place with an active local champion, a new idea was much more likely to take off. in both regions studied there was some evidence of innovation, and also evidence of the adoption of innovations from outside the region. in the opinions of interviewees the two regions were innovative, and both regions had won strong reputations throughout the country for certain projects e.g. southland for a wireless broadband scheme. however there was a lack of hard data to back this up. the evidence does seem to indicate that the capacity for innovation is directly related to population size, with the most populated regions being the most innovative. the findings for this category were mixed, there was evidence of innovation nationally and in both regions, but it was not present at the high levels that would be expected in a classical learning region. australasian journal of information systems volume 18 number1 2013 18 interacting in an interactive region individuals within the region will share a common culture, social capital will be high and crime rates will be low. there will also be evidence of active social networks through work, sport, voluntary groups and similar. there was strong and consistent evidence of high social capital in new zealand at both the national and the regional levels. there was a strong cultural identity in southland; in wellington cultural loyalties tended to be to the local authority area rather than to the region. by the end of the period community groups had become well aware of the contribution ict could make to regional development. in 1985 and 1995 ict networks were mainly used by the government and private business, but by 2005, ict was widely used in the voluntary and community sectors. this trend was observed in both regions and was reinforced at a national level by the publication of the government’s digital strategy (ministry of economic development 2005). community groups were using hard ict networks to complement and reinforce existing soft networks. by 2005 information technology was being increasingly used by the not-for-profit sector and mãori groups. however, though ict networks were identified as playing a role in building interaction within a region they were seen as a complementary to rather than as a replacement for face-to-face contact. the results for this category were very strongly positive, at both the national and regional levels there was plenty of evidence of good social capital, and active citizen involvement in civic life. infrastructure the ideal learning region will have a well-developed telecommunications and transport infrastructure, together with institutional thickness, as demonstrated by lively interactions between different organisations in the region. in new zealand the issue of infrastructure seems to be of more importance than in other countries with more developed infrastructure, such as europe and the usa. a well -developed infrastructure tends to be taken for granted; it is when there are gaps that it becomes a more pressing issue. at both national and regional levels there was significant investment in telecommunications infrastructure. despite complaints that new zealand was not keeping up in global terms, it was clear that successive governments were committed to developing telecommunications and believed it would strengthen the economy. in both regions there were frictions between local and regional government, which worked against the development of institutional thickness. there was evidence of interaction between the different organisations within a region. however, frequent changes caused by local politics meant it was difficult for these networks to develop and grow. the results for this category were also mixed. though telecommunications were recognised as crucial to the economy, new zealand’s low population means that it is always going to be more expensive to build up infrastructure than in more densely populated countries. the country is always going to struggle to improve its position in the oecd rankings when it comes to features such as broadband adoption. there was also a tension between the different levels of government that at times seemed to inhibit progress. income the ideal learning region is consistently economically successful, with low unemployment rates. though the economy of both regions improved over the period studied, alongside reduced levels of australasian journal of information systems volume 18 number1 2013 19 unemployment, neither region exhibited the exceptional economic performance that would be expected of a learning region. though the findings for this section are a little disappointing, it should be noted that as a small country, located at some distance from major world markets new zealand faces a more difficult task than many other countries in developing its economy. the fact that the country has managed to make progress despite these challenges needs to be recognised. overview of findings the positive areas were interconnecting, informing and interacting and ict was found to be making a contribution in all three areas. between 1985 and 2005, organisations became much more interlinked in terms of their ict networks, and information technology opened up access to the rest of the world. ict was used to increase interconnection at the regional level, particularly in the dairy farming, education and community sectors. these interconnections opened up new opportunities for regional learning and innovation. both regions were successful in setting up high quality ict networks, most notably in the education sector in southland and the community sector in wellington. however, though ict contributed to positive developments in these areas, it could not operate in a vacuum. the existence of good social networks and strong local champions were critical to regional development. ict could complement these social networks but was no replacement for them. therefore there was no direct cause and effect relationship between ict and regional economic development. though many examples were found of positive initiatives in both regions it was difficult for initiatives to gain momentum and achieve lasting change. at various points throughout the twenty year period, initiatives were set up around establishing clusters, developing a regional strategy, setting up high technology zones or developing partnerships between education and business, but there was no evidence that such initiatives built steadily over the years. proposed changes at a regional level seemed to be met with infighting and local resistance, which inhibited any steady long-term development. so though the soft networks formed by clusters, joint ventures and networks were present, no clear pattern of development could be observed. in terms of infrastructure the general picture that emerged is of a clear linear progression in terms of the development of hard networks, but a more attenuated pattern in terms of soft networks where the same issues were revisited a number of times over the years. though there was evidence of a relationship between the soft networks that existed at the regional level and the utilisation of hard ict networks within a region, it was difficult to quantify. a learning region is typically characterised by high levels of innovation, which in turn lead to economic success. though new zealanders have a reputation for being innovative, and examples were found of successful individual companies, neither region managed to develop anything close to a regional innovation system. the issues previously discussed are part of the reason. the findings of the research show that hard and soft networks evolve differently over time and that the relationship between the two is nuanced. though good social capital existed in both regions, especially in rural southland, it was located in different interest groups and was not easy to bring together. this lack of co-ordination meant that the possibilities opened up by ict infrastructure in terms of increasing innovation were not fully realised. both regions did demonstrate a strong commitment to learning, but this learning had yet to be translated into economic success. australasian journal of information systems volume 18 number1 2013 20 step seven: write the transcript the transcript produced needs to be placed within the context of previous work. this research builds on the work on historical methods in information systems carried out by mason et al (mason et al. 1997a; mason et al. 1997b) by applying it at a regional rather than at an organisational level. it also provides an example of the use of a conceptual framework, the “6-i” model for data analysis, as well as using regional newspapers as a source of data. historical research is often incomplete and provisional, it provides a rich thick description of events, that is particular and descriptive rather than being analytical and general (neuman 2003). a major goal of historical research is organising and giving new meaning to evidence rather than providing an authoritative account. this research fits in with this tradition, by providing a detailed examination of the use of ict in two regions of new zealand over a fairly limited period of time. the findings demonstrate the important role that soft social networks play in the successful utilisation of ict networks within a regional setting. this was found to hold true whether the technology being considered was videotex, the internet or ultra-fast broadband. lessons learned the following section reflects on the lessons learned at each stage of the research. step one: begin with focusing questions this research asked a broad big picture question and historical methods was chosen as the best approach to address it because of the ability to provide deep and wide insights. learning regions take time to grow and the development of social capital; cultural values and the build-up of networks are most meaningfully examined using a long time perspective. however, big picture type questions need to be made manageable, according to the resources available to the researcher. in this case the research question was broken down by using a framework developed from the literature review. this imposed some order on the research process, by facilitating the selection and ordering of relevant data. step two: specify the domain for the enquiry the issue of deciding on the appropriate scope for the research is critical. most historians would consider looking back only twenty years as barely touching the tip of the iceberg. to some extent this can be justified by the fact that ict is a fairly recent phenomenon. however in terms of social networks it would have been useful to dig further back into the history of each region. one of the most difficult aspects of using historical methods for an it researcher is setting an end date, the rapid rate of new developments in the it field means that it requires immense self-discipline to put them to one side while concentrating on the past. in the case of this research there seemed to be an almost constant stream of new initiatives around the issue of broadband, which were very difficult to ignore. alongside this is the concern that the findings of the research will be dismissed as out-ofdate and irrelevant. step three: gather evidence, using both primary and secondary source when using historical methods the availability of data is a key issue, if there is no data, there is no story. at an early stage the researcher needs to establish if there is enough information available to answer the research question. one frustration with this research was the difficulty of finding accurate australasian journal of information systems volume 18 number1 2013 21 data at the regional level; though statistics new zealand now collect regional statistics they weren’t available for the earlier parts of the period studied. another issue when using historical methods is the large quantity of data that is collected, this is not only time consuming it also creates the challenge of ordering and categorising the data in order to make it meaningful. details and individual incidents may be significant but overall findings have to be reported in a concise fashion. the technique used in this research to organise the data was the use of a conceptual framework, in this case a model of an “ideal” learning region, the “6-i” framework, was used to organise the data into categories. another strategy was to only collect detailed data for three key years, 1985, 1995 and 2005, during the twenty year period studied. this meant there were ten year gaps between each collection point during which a lot of information was missed, meaning that the data collected can only be regarded as a snapshot in time. strictly this means the research could be categorised as a longitudinal study rather than true historical research (bannister 2002). this was ameliorated to some extent by the use of other materials such as statistical reports, but it is still a limitation of the research. step four: critique the evidence. is it authentic and credible? the use of newspapers for historical research raises questions about whether such materials are a good source for historical truth, as reporting can be biased and inaccurate. some steps were taken to address this, such as cross checking events across a range of publications, and using reports produced by independent bodies, but it does need to be acknowledged that newspapers can be fallible. contradictions were found. different articles on the same topic often contained conflicting facts and figures; claims made by politicians weren’t supported by the statistical evidence. every effort was made to try and resolve these contradictions by cross-checking data from a number of sources, but in many cases this was not possible and data was presented as found. the trustworthiness of qualitative research is always open to question; newspapers have an advantage over data collected by techniques such as interviews or focus groups, in that they are in the public eye. newspapers can face libel if they publish inaccurate information therefore journalists take some steps to check their facts, and readers have a feedback mechanism in the form of the letters page. one of the techniques of historical research is to listen for "silences", in other words to work out what is missing from the data. the regional newspapers did not provide good coverage of the industries in their regions, and initiatives such as the formation of business clusters tended to be under reported. this issue relates back to step 3, in this case newspapers were used extensively as little other data was found at the regional level, so a researcher not only needs to consider if there is data available to answer the research question, but also to assess the quality of that data before proceeding. step five: determine patterns using inductive reasoning mason, mckenney & copeland have outlined three approaches that can be used for this: conceptual frameworks, causal chain analysis and establishing empathy with the main participants. it is important for the researcher to decide at an early stage which approach they are going the use, as this will affect both the research question and the approach taken to data gathering. establishing empathy is the most common approach used to date, and is suitable for a study of one organisation, it also has the advantage of producing a compelling story. this research has demonstrated the use of a conceptual framework, causal chain analysis is potentially the most rigorous approach, but also the most challenging. australasian journal of information systems volume 18 number1 2013 22 step six: tell the story the main goal of historical research is to produce a narrative. however due to the extensive data collection, that use of historical methods usually involves, that story is often rather long and very detailed. this creates issues for researchers who are under pressure to get their work published. currently in information systems, publishing in journals is given more weight than writing a book, but it is often difficult to compress the findings of historical research into the word limits set by journals. step seven: write the transcript the researcher needs an understanding of where there work fits in with previous studies, they should be aware of previous research in this area, and what contribution will be made by their study. conclusion this article gives an overview of the use of historical research in information systems and provides an example of how historical methods was used in a research project. in particular, the seven stage method of mason et al. (1997b) was applied to explore the role of ict in facilitating the development of two learning regions in new zealand. ict is a maturing discipline, even though new zealand was a relatively late adopter of ict, with the first mainframe computer, an ibm 650 for the treasury, not arriving in the country until 1960 (newman, 2008), computers have still been around for over fifty years. even before this the precursors to computers in the form of tabulating machines (yates, 2005) and totalisers (doran, 20062007) were in widespread use. this gives researchers a long enough period to study the use of icts within the broader social, cultural and economic context. the reasons for the success of ict in one setting and its failure within another become clearer. for example the rapid adoption of the internet in new zealand in 1995 can be seen in the context of a strong desire by citizens of a remote country to improve their connections with the rest of the world, this longing for fast and affordable international communications can even be traced back to henniker heaton’s campaign for a penny post between australia and the rest of the commonwealth (de garis, 1972). overtime historical research reveals underlying patterns which enable cause and effect to be established, this provides researchers with greater insights into the reasons behind the differing fortunes of ict systems in different contexts. while the scope of this research was too limited to definitively establish cause and effect, underlying patterns were revealed, in particular the impact of power struggles between different groups within the two regions which often worked against the long term success of new initiatives. the strength of historical research is that it takes the big picture approach which considers developments in information systems within the context of wider changes at the organisational, regional or national level. however the big picture approach can also prove a weakness as well as a strength; tackling a large scale problem often means that the contribution of any one piece of research is rather limited. historical research should be regarded as a transcript which needs to be placed within the context of previous work (mason et al., 1997b). it is often incomplete and provisional, providing a rich, thick description of events that is particular and descriptive rather than analytical and general (neuman, 2003). a major goal of historical research is organising and giving new meaning to evidence rather than providing an authoritative account. new historical research in information systems should be regarded as providing only part of the big picture and should not be judged in isolation but evaluated on the contribution it makes towards building up that authoritative account. australasian journal of information systems volume 18 number1 2013 23 information systems is a discipline that prides itself on being forward looking, there is a tendency for researchers to focus on the latest trends and out-of –date technology is often dismissed as irrelevant. however people change more slowly than technology and patterns of behaviour tend to repeat themselves. there are many lessons to be learned from the past, and as the information systems discipline matures historical methods will form a useful addition to the information systems researchers’ toolkit. references bannister, f. "the dimension of time: historiography in information systems research," electronic journal of business research methods (1:1) 2002, pp 1-10. christie, i., and hepworth, m. "towards the sustainable e-region," in: digital futures: living in a dot com world, j. wilsdon (ed.), earthscan publications ltd., london, 2001. cornford, j. "the evolution of the information society & regional development in europe," norwegian regional development ministry, oslo, norway. florida, r. "toward the learning region," futures (25:5) 1995, pp 527-536. hirschheim, r., porra, j., and parks, m. s. "the evolution of the corporate it function and the role of the cio at texaco how do perceptions of it's performance get formed," the data base for advances in information systems (34:4) 2003, pp 8-27. howell, m., and prevenier, w. from reliable sources: an introduction to historical methods cornell university press, ithaca, london, 2001. hudson, r. "the learning economy, the learning firm, and the learning region: a sympathetic critique of the limits to learning," european urban & regional development studies (6:1) 1999, pp 59-72. hunter, i., and morrow, d. (eds.) city of enterprise: perspectives on auckland business history. auckland university press, auckland, 2006. istart "broadband for all project probe explained," in: istart, 2004. keating, j., badenhorst, a., and szlachetko, t. "victoria as a learning region: background report," oecd, melbourne victoria, pp. 1-154. lagendijk, a., and cornford, j. "regional institutions and knowledge tracking new forms of regional development policy," geoforum (31:2) 2000, pp 209-218. larsen, k. "learning cities: the new recipe in regional development," organisation for economic coooperation and development:, the oecd observer (summer 1999) 1999, pp 73-76. lever, w. f., and turok, i. "competitive cities: introduction to the review," urban studies (36:5-6), may 1999 1999, pp 791-793. macleod, g. "the learning region in an age of austerity: capitalizing on knowledge, entrepreneurialism, and reflexive capitalism," geoforum (31) 2000, pp 219-236. malecki, e. j. "hard and soft networks for urban competitiveness," urban studies (39:5-6) 2002, pp 929-945. maskell, p. "globalisation and industrial competitiveness: the process and consequences of ubiquitification," in: making connections: technological learning and regional economic change, e.j. malecki and p. oinas (eds.), ashgate publishing ltd, 1999, pp. 35-59. maskell, p., and tornqvist, t. building a cross-border learning region: emergence of the north european oresund region copenhagen business school press, 1999. australasian journal of information systems volume 18 number1 2013 24 mason, r. o. "the legacy of leo: lessons learned from an english tea and cake company's pioneering efforts in information systems," journal of the association for information systems (5:5) 2004, pp 183-219. mason, r. o., mckenney, j. l., and copeland, d. g. "developing an historical tradition in mis research," mis quarterly (21:3), sept 1997 1997a, pp 257-278. mason, r. o., mckenney, j. l., and copeland, d. g. "an historical method for mis research: steps and assumption," mis quarterly (21:3), sept 1997 1997b, pp 307-320. mckenney, j. l., copeland, d. c., and mason, r. o. waves of change: business evolution through information technology harvard business school press, cambridge, massachusetts, 1995. mckenney, j. l., mason, r. o., and copeland, d. g. "bank of america: the crest and trough of technological leadership," mis quarterly (21:3) 1997, pp 321-353. miles, m., and huberman, a. qualitative data analysis: an expanded sourcebook, (2nd ed.) sage, thousand oaks, 1994. ministry of economic development "the digital strategy: creating our digital future," wellington. morgan, k. "the learning region: institutions, innovation and regional renewal," regional studies (31:5) 1997, pp 491-503. neuman, w. l. social research methods: qualitative and quantitative approaches, (5th ed.) allyn & bacon, boston, 2003. oinas, p., and malecki, e. j. "spatial innovation systems," in: making connections: technological learning and regional economic change, e.j. malecki and p. oinas (eds.), ashgate publishing ltd, 1999. organisation for economic co-operation & development cities and regions in the new learning economy organisation for economic co-operation & development, paris, 2001a. oughton, c., landabaso, m., and morgan, k. "the regional innovation paradox: innovation policy & industrial policy," the journal of technology transfer (27:2) 2002, pp 97-110. porra, j., hirschheim, r., and park, m. s. "the history of texaco's corporate information technology function: a general systems theoretical interpretation," mis quarterly (29:4) 2005, pp 721-746. porra, j., hirschheim, r., and parks, m. s. "forty years of the corporate information technology function at texaco inc. a history," information and organization (16:1) 2006, pp 82-107. rio, c. r. d. "learning to innovate: learning regions," organisation for economic co-operation and development /institute for regional development, andalucia. saxenian, a. regional advantage: culture and competition in silicon valley and route 128 harvard university press, cambridge, m a., 1994a. schollman, a., o'neill, d., doczi, m., and kelly, f. "regional innovation, learning & governance rediscovering new zealand's regions: a central government perspective," learning cities and regions conference, oecd, melbourne, 2002, pp. 1-24. shafer, r. j. a guide to historical method, (3rd ed.) the dorsey press, 1980. sokol, m. "regional dimensions of the knowledge economy: implications for the "new europe"," in: centre for urban and regional development studies, university of newcastle upon tyne, newcastle upon tyne, 2002. stair, r., moisiadis, f., genrich, r., and reynolds, g. principles of information systems, (2nd ed.) cengage learning, melbourne australia, 2011. australasian journal of information systems volume 18 number1 2013 25 storper, m. "the resurgence of regional economics, ten years later: the region as a nexus of untraded interdependencies," european urban & regional development studies (2:3) 1995, pp 191221. thompson, h. "creating and sustaining online communities replicable web-based services meeting the diverse needs of regional and rural australia," learning cities and regions, melbourne, australia, 2002. tosh, j. the pursuit of history: aims, methods and new directions in the study of modern history, (3rd ed.) pearson education limited, harlow, 2000. wenger, e. c., and snyder, w. m. "communities of practice: the organisational frontier," harvard business review), january-february 2000 2000, pp 139-145. wolfe, d. a. "globalization, information and communication technologies and local and regional systems of innovation," in: transition to the knowledge society, k. robenson and h. schuetze (eds.), vancouver institute for european studies, university of british columbia, vancouver, 2000, pp. 147-162. wolfe, d. a. "social capital and cluster development in learning regions," in: knowledge, clusters and learning regions: economic development in canada, j.a. holbrook and d.a. wolfe (eds.), school of policy studies, queen's university, kingston, 2002, pp. 11-38. yates, j. "using giddens' structuration theory to inform business history," business and economic history (26:1) 1997, pp 159-183. strategic systems? only when they work! ernest jordan david musson macquarie graduate school of management macquarie university email: emesljordan@mq.edu.au abstract this paper presents the results of a sample survey of australian organisations' approaches to business and information technology (it) contingency planning. the survey was undertaken because there was a perception that coping with disaster is a much neglected aspect of management in australia. it is an exploratory study that seeks to establish the current levels of awareness and preparedness. the findings reveal that most organisations are inadequately prepared and fail to take the issue seriously. business continuity is not rated as a high priority. keywords ek08 disaster plans, ek is security, el08 is risk management introduction the late 1980s saw a dramatic rise in the perceived importance of strategic planning for is and ft. several surveys of north american mis managers (such as dickson et a/., 1984; hartog and herbert, 1986; lederer and mendelow, 1986; brancheau and wetherbe, 1987) showed strategic planning high on the list of key issues, having risen rapidly to that prominence. in part it was fuelled by the necessity for many organisations to think in terms of strategic applications because their customers, competitors, suppliers and others in the world around them were doing the same (earl, 1989). it became a competitive necessity for an organisation to carry out some form of planning if its competitors were themselves searching for opportunities for competitive advantage. the external stimulus provided a focus for the changes that were finding their way into the organisation. security and control however a falling star during this era was security and control. brancheau and wetherbe (1987) show 'security and control' as only ranked 18th in the list of key issues. perhaps 'everyone knew' what they had to do in order to keep systems secure, and to maintain control over the it environment. by the time of the most recent (1994-5) survey in the series (brancheau, janz and wetherbe, 1996), security and control has disappeared completely. interestingly, the beguiling 'competitive advantage' has sunk to 18th place so we can anticipate that it too will disappear from future surveys. however the top item in the 1994-5 survey is 'building a responsive it infrastructure' which reflects the demanding challenge of constructing a reliable and responsive framework for connectivity and applications. we would argue that reliability is an integral feature and that 'security and control' is implied. if it applications are indeed competitive or strategic or critical to business needs, then one would anticipate that establishing a high level of operational reliability for such systems is imperative. part of this challenge concerns what happens in an emergency or disaster, a topic usually known as it contingency planning or disaster recovery planning. one way of demonstrating both the real level of importance of it to an organisation, including its strategic or competitive potential, and the gap between the 'headline' it and the operational reality, is to consider the cinderella issue of security and control. if it truly is of critical importance to an organisation, if systems are 'strategic', then it certainly needs to be safeguarded; perhaps the systems simply aren't critical or strategic. on the other hand, it could be a matter of a difference between what is said and what is done, what argyris called a difference between 'theory espoused' and 'theory in use'. perhaps it is critical or strategic, but we just don't do what we know we should, mimicking the individual's responses to home insurance and the like. security is certainly not glamorous; cios would not allocate this issue to their 'best and brightest'. it is, however, a challenge for the organisation. given that disasters do happen, we wished to examine the preparedness of organisations to deal with them. particular issues include the level of preparedness and the personnel responsible for it. we have also found links in the literature between it contingency planning and business contingency planning. while it may be a key operational enabler, the business itself unquestionably would need to be able to continue its very existence should be safeguarded. in this context we wanted to find out to what extent it and business planning were coordinated, whether the same people were involved and which of these was more widespread. research questions the research addresses three key areas: the nature of experienced contingencies, the extent of planning, and the operationalisation of the plan. the overall aim is to find out whether australian businesses are taking the risks to their businesses and it seriously. from this we can establish the level of awareness and preparedness, and plan interventions such as education, workshops and self-assessment schedules. experiences there is a wide range of contingencies that may be experienced, some of them common (such as power failure and communication line failure) and others extremely rare (such as extensive fires, major accidents, or bombs). we wished to establish benchmarks for the frequencies of the most common events. the level of occurrence can be expected to be related to the extent of computerisation so this needed to be monitored. plan formulation the simplest, clearest and most widely agreed strategy to deal with contingencies is to have a plan. there are many standards published in australia that cover aspects of storage, protection and documentation of it assets, but the umbrella document is the australian standard as4444:1996 information security management (standards australia, 1996). we were concerned to find out whether organisations had created plans and what those plans contained. in particular, to determine the extent to which the plans covered items mentioned in the standard. plan operationalisation putting a plan into operation does not require a disaster to happen. testing and rehearsing aspects of the plan, educating and training those with special responsibilities, such activities are to be expected in a prepared organisation. minor contingencies need to be dealt with frequently and these can reveal shortcomings in the plan. the study examines the evidence that the plans are operational. hypotheses since this investigation is attempting to determine current levels of preparedness against conventional advice, such as the australian standard, it was not considered necessary to develop comprehensive a priori hypotheses. it is the aim of the research to enable hypotheses to be formed for future work and to establish a reference point for future studies. methodology given that the overall aim is to establish an indication of the australia-wide level of preparedness, it was decided that a survey was necessary. furthermore, the majority of the concepts and issues related to contingency planning are well understood and may reasonably be answered in a questionnaire format. the study questionnaire was addressed to the ceo of the selected organisations, to be forwarded to the 'person responsible'. a reply coupon from ceos specified the contact person; most participants requested a copy of the final report. in defining the population to be the target of the survey, it was decided that australian state and federal government departments and agencies would be excluded. there were two main reasons for this: it was anticipated that the overall level of preparedness would be high because of mandatory controls, and secondly, we expected some reluctance to respond from such organisations. we further restricted ourselves to australian organisations with significant computer configurations (more than 20 users). sample selection the mailing list was based upon the mis 4000 database of australian and new zealand computer users (strategic publishing, 1996). this database lists 4169 computer users. 19% (or 792) of these were in new zealand, 16.4% (or 684) of these were in the public service and 5.7% (or 238) were it suppliers. excluding these gave a total of 2455 users. 31 other organisations were also excluded; mainly holding companies whose operating companies were also listed. this left a total of 2424 organisations. a stratified random sample of organisations was selected, representing some 10% of the larger industry groups and some 20% of the smaller industry groups. industry mining & agriculture services electricity & gas transport personal services & tourism table 1: sampling population and sample sizes industry manufacturing wholesale retail construction finance total size sample 822 82 265 27 121 202 393 23 19 41 size 138 318 63 70 32 2424 sample 28 33 11 13 6 283 questionnaire development the questions in the questionnaire can be divided into several groups. these are: introductory questions: organisation size, and computer configuration. service interruption questions: effects of loss of systems, frequency of interruptions, duration and cause of failures, and tolerable out-of-service times. backup questions: intended to build up a picture of the backup procedures and the experience in taking and restoring backups. contingency planning questions: seeking to establish the degree of planning in place, staff responsibilities and policies. initial tabulations detailed tabulations of all questions as well as the questionnaire are published elsewhere (musson and jordan, 1997). the response rates were not significantly different across the industry sectors. the overall response rate was just over 25% — acceptable for such a demanding and invasive questionnaire. the responses received to a selection of the questions in the survey are set out below. no. of employees 1-49 50-199 200 499 500 + no answer total number of employees count 9 22 17 22 1 71 percentage 12.7 31.0 23.9 31.0 1.4 100.0 table 2: responses by number of employees table 2 shows that the responses demonstrate a reasonable coverage of the range of organisational sizes. there is no excessive emphasis on any size category. type mainframe midrange client/server lan/networked pcs standalone pcs number 17 47 51 69 49 % of responses 23.9 66.2 71.8 97.2 69.0 table 3: it holdings of respondents multiple responses were anticipated and received, following questions. this causes some anomalies to counts in some of the no answer critical very disruptive disruptive minor inconvenience no inconvenience total m/frame count 12 3 2 17 loss % 70.6 17.6 11.8 100.0 midrange count 31 13 2 1 47 loss % 66.0 27.7 4.3 2.1 100.0 server count 1 24 14 l_ 13 1 53 loss % 1.9 45.3 26.4 24.5 1.9 100.0 network count 12 28 19 10 69 loss % 17.4 40.6 27.5 14.5 100.0 pc count 1 1 3 20 23 2 50 loss % 2.0 2.0 6.0 40.0 46.0 4.0 100.0 table 4: effect of loss of computer services table 4 shows that the effect of loss of mainframe, midrange and server systems is typically regarded as very disruptive or critical. network failure is not quite as critical and the failure of standalone pcs are not generally regarded as more than 'disruptive'. no yes don't know total experienced interruptions count 18 51 2 71 % 25.4 71.8 2.8 100.0 length of interruption less than 1 hour 1 to 4 hours 4 to 24 hours 1 to 3 days 4 days to 1 week count 483 161 58 9 1 table 5: critical service interruptions table 6: length of interruptions table 5 indicates that interruptions to critical services are not unusual, with the majority of organisations experiencing this during a period of two years. the detail provided in table 6 is substantial, with each organisation experiencing and reporting some ten interruptions on average. type hardware failures software failures power failures accidents other causes count 115 153 61 22 41 % 29.3 39.0 15.6 5.6 10.5 table 7: cause of service interruptions no answer < 8 hours 8 24 hours 1 2 days 2 7 days > 7 days total count 1 28 21 16 4 1 71 % 1.4 39.4 29.6 22.5 5.6 1.4 100.0 table 8: longest tolerable "out-of-service" time for most critical applications ajis vol. 6 no. 1 september 1998 table 7 shows that hardware and software failures together comprise the majority of failures experienced. "disasters" are indeed rare. table 8 shows that some 40% of organisations regard up to 8 hours as the largest tolerable out of service time. this gives an absolute size to the qualitative judgements such as 'critical' or 'disruptive' shown in table 4 and elsewhere. yes, time acceptable yes, time unacceptable no estimate of time could not recreate data don't know total count 34 8 23 2 3 70 % 47.9 11.3 32.4 2.8 4.2 98.6 table 9 estimate of time to recreate data if backup failed^ more than 50% of organisations are exposing themselves to risks in this area. an estimate of the time to re-create data from scratch is needed in any comprehensive plan. complete business operations including it it only neither business nor it total company has plans for: count 14 23 34 71 % 19.7 32.4 47.9 100.0 table 10: holding of contingency plans by australian companies table 10 shows that only one fifth of organisations have established plans for both the business and for it. almost half of the organisations have no contingency plans at all. over 30% of the organisations only have contingency plans for it. < 1 year 1 -2 years 3 -5 years > 5 years no plan being developed total business plan duration count 4 7 2 1 53 4 71 % 5.6 9.9 2.8 1.4 74.6 5.6 100.0 it plan duration count 6 11 13 4 34 2 70 % 8.5 15.5 18.3 5.6 47.9 2.8 98.6 table 11: length of time the plans had existed only three organisations have had business contingency plans for more than two years. it plans are much better established. this question confirms the answers given in previous questions about the general absence of plans. 5 in several tables multiple responses and missing or qualified answers cause different totals 31 ajis vol. 6 no. 1 september 1998 reason no answer necessary but low priority not considered necessary not considered insufficient resources don't know being developed but not ready total business plan missing count 4 24 4 4 11 7 4 58 % 6.9 41.4 6.9 6.9 19.0 12.1 6.9 100.0 it plan missing count 1 20 1 1 11 2 2 38 % 2.6 52.6 2.6 2.6 28.9 5.3 5.3 100.0 table 12: reasons for not having produced plans overwhelmingly the reason for not having contingency plans is their perceived low priority, together with the resource requirement needed to produce plans. no answer during the last year 1-2 years ago over 2 years ago not tested don't know no plans total last test of business plans count 3 2 2 8 49 7 71 % 4.2 2.8 2.8 11.3 69.0 9.9 100.0 last test of it plan count 1 25 5 5 12 22 1 71 % 1.4 35.2 7.0 7.0 16.9 31.0 1.4 100.0 table 13: time of last testing of contingency plans testing of business contingency plans seems to be a patchy business, with most organisations unable to answer. it plans were recently tested in roughly one third of organisations. md / chief executive main board member it manager line manager staff member no plans total day-to-day responsibility implementation responsibility count 0 3 44 12 2 13 74 % 0.0 4.0 58.7 16.0 2.7 17.3 100.0 count 11 7 37 8 5 10 78 % 14.1 9.0 47.4 10.3 6.4 12.8 100.0 table 14: responsibility for plan implementation in an emergency table 14 shows that more than half of organisations give responsibility to the it manager for the plans, both business and it. this then leaves open the question, hopefully to be investigated in the future, of why this is the case. the implementation of plans in an emergency is also frequently the responsibility of the it manager, although more senior managers are also cited. 32 ajis vol. 6 no. 1 september 1998 no answer yes, completely confident yes, believe they will they probably will unsure that they will qualified answer total count 2 11 33 11 13 1 71 % 2.8 15.5 46.5 15.5 18.3 1.4 100.0 table 15: level of confidence that plans will ensure survival of the business it is our belief that only one of the answers in table 15 is acceptable, yet only 15% of organisations gave it. discussion highly significant extracts from the survey show that: just 20% of australian organisations have disaster recovery plans that cover their entire business operations, and 50% have no plans at all. 38% of organisations that have a maximum of 24 hours out-of-service time for their computer systems and have no plans to deal with a longer stoppage. in most cases, the it manager is responsible for the company disaster recovery plans. these headline figures show that planning for recovery from disaster is still a neglected subject in australian boardrooms. whilst australian business has been relatively free from attack by terrorists and other malevolent people, it is still vulnerable to damage from fire, flood and earthquake, and to accidental damage from a wide variety of sources. failure to plan for unscheduled interruption to the business is thus a serious failure in corporate governance. "very few board chairmen, presidents or general managers would run a business without insurance. for some reason, they have yet to look at disaster recovery planning in that light" (rothstein, 1988). contingency planning before beginning an analysis of the survey results, it is perhaps appropriate to clarify the term "contingency planning". hardy (1992) proposed that all events that occur at an organisation could be described in the dimensions of predictable / unpredictable and controllable / uncontrollable. the federal emergency management agency in the us uses the term "emergency" to describe these unpredictable, uncontrollable events. it defines emergency as: "any unplanned event that can cause death or significant injuries to employees, customers or the public; or that can shut down your business, disrupt operations, cause physical or environmental damage, or threaten the facility's financial standing or public image." (fema, 1996) the same document makes the point that the term "disaster" is not used in the document because "it lends itself to a pre-conceived notion of a large-scale event, usually a 'natural disaster'. what might constitute a nuisance to a large industrial facility could be a 'disaster' to a small business" (fema, 1996). in our survey, we were looking at the degree of planning against emergencies in australian organisations. the issues two management issues are clearly evident in the results of the survey. these are: the number of companies without contingency plans and other key policies; and the question of who is in charge of plans, both day-to-day and in an emergency. these are dealt with in turn below. plans and policies question 7 of the survey asked what was the respondents' longest tolerable "out-of-service" time for critical applications and/or services; the answers were noted in table 8, above. sixty-eight percent of respondents cannot 33 ajis vol. 6 no. 1 september 1998 tolerate breaks in service of a day or less, and over 90% could not tolerate breaks of 2 days or less. these figures would certainly support the case for contingency planning amongst the majority of australian organisations; it is inconceivable that any but the smallest organisation could completely re-create their it systems within 24 hours without a plan. it would be even less likely that they could restart their complete business in that time without a plan. holding of plans the main findings were: • nearly half of all organisations have no formal plans for disaster recovery of their business and their it. • less than 20% of organisations have full business contingency plans; of these plans, 57% have never been tested. • 53% of organisations have it plans. two thirds of these it plans had been tested in the last 12 months, but a significant number had not been tested and several were last tested over two years ago. the detailed results are shown in tables 10 and 13, above. note that there are fewer it plans reported in table 10 than there are tested plans in table 13. this suggests that there are informal plans for it disaster recovery held in organisations. the low level of testing is as worrying as the low level of plan holding. clearly a plan that has not been tested is not a reliable plan; the act of testing the plan may well show up omissions in the plan or, for example, faulty safety equipment. one person interviewed during the course of this study instanced the testing of an emergency siren that was found to be not working it had been installed for five years and never previously tested. other surveys of plans holdings other surveys in australia and the uk help to put these figures in context. the coopers & lybrand survey (coopers & lybrand, 1992) reported that 51% of australian organisations had computer contingency plans, although their figures included public sector organisations who might be more likely to institute contingency plans. figures from the uk appear to show that companies there are better prepared for disaster. in 1994, 59% of uk companies polled by the national computing centre in 1994 had formal it plans and this figure rose to 63% when the poll was repeated in 1996 (ncc, 1994; 1996). the securicor (1996) data security report surveyed 1,741 medium and large uk organisations, and found that 60% of them had it disaster recovery plans. the ernst & young (1996) information security survey for 1996 found that almost 75% of organisations polled (in the us) had an it disaster recovery plan but 42% of these had not been tested. in general, it is clear that it management is well aware of the potential for damage to business operation posed by a failing computer system, and makes formal or informal plans for recovery from system failures. there have been fewer surveys of business contingency planning. coopers & lybrand (1992) reported that 39% of australian organisations had a corporation-wide contingency plan for critical services, although thenfigures again included public sector organisations. the uk 1996 ibm/cranfield study (ibm, 1996) found that 35% of private sector organisations had business disaster recovery plans, again suggesting that the uk is rather better prepared, although the figure is still low. further analyses of the survey findings on planning detailed analysis of our australian survey figures throws more light on to the state of contingency planning for it failure. comparing planning arrangements to the respondents ranking of the effects of loss of their computer services produces the results in tables 16, 17 and 18. if we restrict analysis to those respondents who regarded their systems as critical to their business operations, 24% of these mainframe respondents, some 32% of mid-range respondents and some 30% of server respondents had no plans of any kind. 34 ajis vol. 6 no. 1 september 1998 company critical very disruptive disruptive total has complete plans 3 3 has only it plans 5 5 has no plans 4 3 2 9 table 16: planning arrangements against effects of loss of mainframe computer system company critical very disruptive disruptive minor inconvenience total has complete plans 4 2 1 7 has only it plans 12 5 1 18 has no plans 15 6 2 22 table 17: planning arrangements against effects of loss of midrange computer system company critical very disruptive disruptive minor inconvenience total has complete plans 6 4 10 has only it plans 3 8 6 17 has no plans 15 6 3 1 26 table 18: planning arrangements against effects of loss of server system a breakdown of the companies planning arrangements by their maximum tolerable "out-of-service" time produces the results shown in table 19, below. nineteen responses, or 27% of the total, had maximum times of less than 8 hours but no business contingency plans; 26 responses or 37% had maximum times of less than 24 hours, but no plans. given that a relatively minor fire or explosion could put an organisation out of business for several days, then 37% of australian organisations are putting their entire business at considerable risk. company less than 8 hrs 8 24 hours 1 2 days 2 -7 days more than 7 days total has complete plans 5 6 2 1 14 has only it plans 4 8 7 3 22 has no plans 19 7 7 1 34 table 19: planning arrangements against maximum tolerable out-of-service time confidence in planning arrangements question 20 asked whether respondents were confident that their current contingency plans would ensure the survival of their business after a disaster. responses were given in table 15, above. less than 20% were unsure that their plans were sufficient but only 16% were completely confident that they were in fact sufficient. two further analyses show that many businesses have boundless confidence in their ability to survive a disaster without any form of planning. table 20, below, shows a comparison between the responses to the question on confidence and the answers on the existence of contingency plans. almost 60% of the organisations with neither business nor it plans had some confidence that they could survive a disaster. 35 ajis vol. 6 no. 1 september 1998 company completely confident believe sufficient probably ok unsure sufficient total has complete plans 3 10 1 14 has only it plans 6 13 2 1 22 has no plans 2 10 8 12 32 table 20: planning arrangements against confidence of survival table 21, below, shows confidence against the arrangements for temporary computing facilities. eleven respondents (over 15%) had no alternative arrangements. ten respondents intended to buy new or used replacements. this is not likely to be an effective strategy for most companies, because of the time lag between the disaster and getting the systems back in service, due to the time taken in sourcing, purchasing, installing and commissioning the replacements. confidence: in-house manuf./ hot site 3rd party stand-by 3rt party hot site 3rd party cold site buy new/used reciprocal arr. no arrangement total complete 7 3 1 11 strong 11 7 2 3 2 6 2 33 probable 2 1 1 1 2 2 2 11 unsure 3 1 1 1 7 13 table 21: temporary it arrangements against confidence of survival who is in charge? the change in emphasis in contingency planning from ensuring an uninterrupted computing service to ensuring the survival of the business has meant that, overseas, the responsibility for planning and management of these plans has moved from the it manager to senior line management. this is reflected in the survive! survey conducted in the uk between april and july 1996 (survive! 1996). the responsibility for business continuity was held by the it organisation in only 19% of the 450 respondent organisations, compared to 45% only 18 months before. our survey showed the it manager is responsible for day-to-day management of plans in almost 60% of australian organisations (see table 14, above) and is responsible for implementation of those plans in an emergency in nearly half of australian organisations. table 22 give valuable insights into the placement of responsibility. for those companies with comprehensive plans, some 29% gave their it manager responsibility for implementing plans in an emergency but over 64% held the it manager responsible for the management of the plans. for those companies with only it plans, 70% of them gave the it manager responsibility in an emergency. this suggests that it is still seen as the owner of the company's disaster recovery plans in australia. it is noticeable that those companies with comprehensive plans mostly (57%) appointed a director to take charge in an emergency. 36 ajis vol. 6 no. 1 september 1998 company has md/ chief executive main board director it manager line manager staff member no plans total day-to-day responsibility complete plans 1 9 3 1 14 only it plans 1 19 2 22 no plans 1 16 3 1 13 34 implementation responsibility complete plans 6 2 4 1 1 14 only it plans 2 3 16 1 1 23 no plans 2 2 15 1 4 9 34 table 22: planning arrangements against responsibility, day-to-day and implementation conclusions there are clear conclusions that may be drawn from this survey, even though the sample size is small. the statistics overwhelmingly describe a situation where senior management have abdicated responsibility for business contingency planning to the it manager. furthermore, in just too many cases, it is apparent that planning has not been done, or else not tested and revised. the risks that are taken on are very high. almost routinely there are minor interruptions and disruptions to it services. major incidents have the potential to cripple or destroy organisations. while the routine may be dealt with by most it professionals with or without a plan major incidents are not best 'muddled through'. this investigation had raised a considerable number of research questions. prime among these are questions concerning the plans or their absence: do the espoused reasons of low priority and insufficient resources stand up to detailed examination? why are it managers given the planning responsibility? a second set of hypotheses concerns the quality and potential effectiveness of the plans. why is testing so rare and infrequent? why are standards and legislation ignored? are the standards themselves adequate? this research program has many issues to face before australian business disaster recovery planning is at a sufficient standard. references brancheau, j. c. and wetherbe, j.c. (1987) 'key issues in information systems management', mis quarterly, 11, i.march 1987,23-45 brancheau, j.c., janz, b.d. and wetherbe, j.c. (1996) 'key issues in information systems management: 1994-5 sim delphi results', mis quarterly, 20, 2, 225-242 coopers & lybrand. (1992) corporate security and contingency planning, sydney dickson, g. w., leitheiser, r. l., wetherbe, j. c. and niechis, m. (1984) 'key information system issues for the 1980s1, mis quarterly, 8,4, september 1984, 257-266 earl, m. j. (1989) management strategies for it, prentice hall, hemel hempstead, uk ernst & young (1996) information security survey 1996, october 1996, new york fema (1996) emergency management guide for business and industry, federal emergency management agency, september 1996, washington hardy, k. (1992) 'contingency planning', business quarterly, spring 1992, 56,4,26-28 hartog, c. and herbert, m. (1986) '1985 opinion survey of mis managers: key issues', mis quarterly, 10, 4, december 1986, 351-361 ibm (1996) a risk too far, ibm, london, with cranfield university, cranfield, uk lederer, a. l. and mendelow, a. l. (1986) 'issues in information systems planning', information & management, 11, 245-254 musson, d. and jordan, e. (1997) 'business and computer contingency planning in australia,' macquarie graduate school of management working paper 97/40. ncc (1994) information security breaches survey 1994, national computing centre, uk ncc (1996) information security breaches survey 1996, national computing centre, uk rothstein, p.-j. (1988) 'up and running' datamation, 15 october, p87 37 ajis vol. 6 no. 1 september 1998 securicor (1996) 'data security report' information mgmt & computer security, 4, 2, 14-17 standards australia (1996) information security management, as/nzs 4444:1996, sydney strategic publishing (1996) the mis 4000, strategic publishing group, sydney survive! (1996) survive! survey of disaster recovery, survive!, london acknowledgment the authors wish to thank data security services, a division of ausdoc, for their sponsorship and support of this project. 38 the australasian journal of information systems (ajis) began in 1993 and was based on the simple realisation that there was a need for an academic ‘outlet’ for is research in australia australasian journal of information systems volume 16 number 2 2010 5 software appropriation over time: from adoption to stabilization and beyond antonette mendoza department of computer science and software engineering the university of melbourne melbourne, victoria email: mendozaa@csse.unimelb.edu.au jennie carroll department of property, construction and project management rmit university melbourne, victoria email: jennie.carroll@rmit.edu.au linda stern department of computer science and software engineering the university of melbourne melbourne, victoria email: linda@csse.unimelb.edu.au abstract technologies are introduced into an organization with the aim of improving productivity. however, persistent use of an information system is required to enhance productivity and user satisfaction. in this paper, we examine the process of technology appropriation and the influences that encourage and discourage use of an information system over time. in a longitudinal field study in an educational setting, we identify changing expectations and influences that encourage persistent use of a technology above and beyond adoption. our work indicates that stabilization of a technology may only be temporary. further appropriation may occur beyond initial stabilization as expectations and influences encourage new uses of a technology. we have found that the driving forces in encouraging productive use of a technology are the ability for users to experience the technology as useful and to have easy access to ongoing training. introduction organizations invest millions of dollars on information systems to achieve productive outcomes for individuals and for the organization as a whole. however, the expected productivity and organizational benefits cannot be realized unless information systems are used over the longer-term, well after the initial adoption. there has been much research into the early period of use in information systems, evident in investigation of the influences that affect adoption and acceptance of a technology (brown, massey, montoya-weiss and burkman 2002; davis 1989; karahanna, straub australasian journal of information systems volume 16 number 2 2010 6 and chervany 1999; rogers 1995; taylor and todd 1995; venkatesh, morris, davis and davis 2003). the influences on longer-term use have received much less attention. observations that users adapt their practices over time to accommodate a new technology as well as adapting the technology to suit their needs, suggest that studying longer-term use is an important topic to investigate (chu and robey 2008; majchrzak, rice, malhotra and king 2000; ciborra 1996; desanctis and poole 1994; trigg and bodker 1994; tyre and orlikowski 1994, 1992; leonard-barton 1988). there has been little exploration to date of the journey from initial adoption of a technology through to its stabilization in a particular context, raising questions such as: what are the influences that encourage longer-term use? how long does it take before a technology is stabilized? this paper investigates the influences that encourage long term use through the lens of appropriation. appropriation means to “take possession of” (turner 1988); technology appropriation is: "the way that users evaluate and adopt, adapt and integrate a technology into their everyday practices" (carroll, howard, peck and murphy 2002). implicit in these definitions is the idea that appropriation is a process that may take different forms, depending on the individuals involved, the technology, and the particular social or organisational context in which it is introduced (orlikowski 1992). the definition of technology appropriation applied in this study captures both adoption and adaptation over time. adoption of a technology refers to a state when users have made a decision to use an innovation when faced with it (adapted from rogers 1995). adaptation refers to adjustments and changes following the implementation of a new information technology. in this paper, two aspects of user adaptation are applied. firstly, users may adapt their practices to suit the technology. secondly, users may adapt the technology. that is, they customise and adjust the way in which the technology and its features can be used to suit their needs; this may include applying the technology for new or unintended purposes. in this research the processes of appropriation of technology are studied in an educational setting. the research question addressed by this paper is: what are the influences on the way in which users appropriate a technology over time? a longitudinal approach was taken whereby users were observed at a number of points of time over a 20 week period. the findings were analysed and interpreted using the model of technology appropriation (carroll et al. 2002). our study identifies influences that encourage persistent use of a software application in an educational setting from first encounters through to long-term use. 1 we begin the paper with the theories relating to the process of technology adoption and use, followed by the research methodology. the research findings are presented and their implications are discussed. the paper concludes by presenting some avenues for future research to build on the findings of this work. theoretical background many information systems studies have focussed on technology acceptance and use (brown et al. 2002; davis 1989; davis, bagozzi and warshaw 1989; moore and benbasat 1991; thompson, 1 an earlier version of this work was published at the australian conference on information systems in 2005 (mendoza et al. 2005). australasian journal of information systems volume 16 number 2 2010 7 higgins and howell 1991; taylor and todd 1995; venkatesh and davis 2000). researchers have been interested in understanding individual perceptions, to understand what motivates them to accept or use a technology. different approaches have been taken to examine technology acceptance and use. some researchers have used a variance approach to identify and test the effects of causal factors on use, focusing on antecedents of adoption and usage of new technologies (davis 1989; venkatesh 2003; agarwal and prasad 1999; compeau, meister and higgins 2007). according to the variance approach, levels of outcomes are predicted from predictor variables, at a specific point in time. a cause is necessary and sufficient for an outcome which invariably occurs when necessary and sufficient conditions are present (markus and robey 1988; mohr 1982). the technology acceptance model (tam) has been one of the most prominent variance models. according to this model, adoption and usage of a technology are predicted by intentions to use the technology, which, in turn are influenced by perceptions and attitudes about the technology. perceived usefulness and perceived ease of use are considered to be the main determinants of user acceptance (davis 1989) which act as the antecedents of attitude. perceived usefulness along with user attitudes determines a user’s behavioural intention. perceived ease of use also has a significant effect on perceived usefulness (taylor and todd 1995; venkatesh and davis 2000; szajna 1996; davis 1989) and behavioural intentions (igbaria et al. 1997; adams, nelson and todd 1992). however, some other studies have suggested that these effects are not significant (chau and hu 2002a; szajna 1996). even with a recent revision of the tam (venkatesh et al. 2003) that includes social influences and key moderators, the lack of further explanation of the relationships between social influences and the key moderators, has prompted some researchers to question the generalizability of tam (straub, keil, and brenner 1997; van der heijden 2004; sun and zhang 2006). another prominent theory grounded in empirical work that includes observations is the diffusion of innovation theory (rogers 1995). it examines the process of adoption and implementation of innovations. according to the diffusion of innovation theory, diffusion is a process by which an innovation is communicated through certain channels over time among members of a social system. the innovation decision process takes place in sequential stages. it starts from a user gaining knowledge about the innovation, to forming an attitude towards it, to deciding to adopt or reject the innovation, to implementing the new idea and finally to confirming the decision. an innovation perceived by individuals as having greater relative advantage, compatibility, trialability, observability and less complexity will be adopted more rapidly than other innovations. the diffusion of innovation theory has made a major contribution to the understanding of the influences on adoption of an innovation. however, the diffusion of innovation theory has depicted the stages as sequential in nature. the need for process research, involving descriptions and analysis of sequences of activities, may provide a dynamic view of the innovation process and has been advocated by roger (1991) and others (for example, gallivan 2001). to date, little is known about the ongoing process from a user’s initial encounters with a new technology through to long term use or rejection. some studies have examined continued use or “continuance” (bhattercherjee 2001) of an information system in the context of “implementation” and “routinization” (cooper and zmud 1990) or even as the “confirmation” stage in the five-stage adoption decision process according to the innovation diffusion theory (rogers 1995). according to these theories, users initially accept a new information system. at the post-acceptance stage, the innovation becomes a part of users’ everyday routine and then they re-evaluate their earlier decision during the “confirmation” stage, deciding whether to continue or discontinue using the innovation. these studies have used the same set of variables to understand acceptance and continued use of a technology, assuming that once a technology is accepted initially, an individual will continue using it as a part of the adoption decision process. also australasian journal of information systems volume 16 number 2 2010 8 implicit in such variance approaches is an assumption that two of the key concepts, use and technology, are unchanging over time. other researchers have focused on a process approach to understand adaptation as a part of technology use (orlikowski 1996; tyre and olikowski 1994; desanctis and poole 1994; majchrzak et al. 2000). the studies have shown how users change their skills, beliefs, attitudes, expectations (tyre and orlikowski 1994), modify their work practices (leonard-barton 1988; desanctis and poole 1994) and sometimes re-invent the technology in un-anticipated ways (griffith 1999; leonard-barton 1988). little is known however about why users accept and adopt a technology initially but reject it over time. less attention is paid to changes in adopters’ practices or to adaptations of the innovation itself, that have long been observed by is researchers (ciborra 1996; leonard-barton 1988; orlikwoski 1992). some studies have focused on adaptation as a process of appropriation. tyre and orlikowski 1994 argue that most adaptation takes place following initial implementation, in what they term a window of opportunity. the technology use stabilizes and the window of opportunity may then close. adaptations may not be gradual and may be highly discontinuous in nature. these adaptations may be initially steep with brief windows of opportunity in which technologies could be altered. in some other studies, the technology adaptation process is viewed as continuous, with cycles of misalignments or discrepant events that gradually reduce over time. eventually, an alignment of the delivery system, the technology and the performance criteria take place (leonard-barton 1988). other researchers have used structuration theory (giddens 1984) to examine the micro-processes through which humans and technology are mutually constituted (orlikowski 1992; desanctis and poole 1994). the structuration theory suggests that use of a new technology is stochastic in nature. users adopt and then adapt a technology, reshaping their technologies to suit their needs. despite all these theories, the lack of an increased understanding of how people engage with various technologies in their everyday practices and the evolving dynamics of changes in technology use and user expectations is a theoretical concern (orlikowski 2000). more recently, one study has revisited the concepts of structural features and spirit focussing on the technology related concepts of structural features and systems spirit for use in research based on the behavioural and social effects of it use (markus and silver 2008). lyytinen and gaskin (2010) explored factors influencing individual appropriation of technology and concepts related to psychological ownership. yet, there has been little research, into the influences that encourage or discourage appropriation of a wide range of technologies used by different cohort of users, over time. carroll et al. (2002) conducted field research into the use of mobile technologies and induced a model of technology appropriation (mta) to build theory about appropriation. according to the mta, a technology that is introduced in a work place (technology as designed) is changed over time. the technology shapes the users' practice and in turn, is shaped by users' actions. this changed technology (technology in use) is an outcome of the process of appropriation, which involves users’ trialling, evaluating and adapting the technology to suit their personal needs based on their perceptions and various other influences. the model as shown in figure 1 represents three levels of evaluation of an information communication technology, with influences affecting these levels: level 1: users are introduced to the technology and they are faced with the decision of whether to adopt the technology or not during these initial encounters with the ict. this decision is based on various influences that have been well-examined in the literature, such as perceptions of the technology, the user, the task and the context of use (davis 1989; rogers 1995). level 2: once users decide to adopt the technology, they enter into a deeper level of use, the process of appropriation, where they trial, evaluate and adapt the technology to suit their needs. at this level, users may explore and experiment with the technology. consequently, users may adapt the australasian journal of information systems volume 16 number 2 2010 9 technology to suit their practices or may adapt their practices to suit the technology. at any time during this exploration, users may reject the technology because, for example, it is difficult to use or does not meet their needs. technology in use level 3 technology as designed possibilities non-adoption explore evaluate adapt disappropriation appropriation level 2 level 1 adoption appropriation process figure 1: the model of technology appropriation (adapted from carroll et al. 2002) level 3: use of the technology is persistent. in this level users have integrated the technology into their practices and the technology is considered to be stabilized. persistent use is “reinforced” by influences and changes in these influences may lead to re-evaluation of the technology or even its rejection. thus the mta represents the process of appropriation from adoption through to long-term use. it suggests that the influences on users’ actions are not static but may change over time and it includes changes both in the technology (from technology as designed to technology in use) as well as in users’ actions. the mta was developed by examining young people using mobile technologies. we were interested in understanding the influences on the actions of users throughout the period from initial encounters through to long-term use of a software application. we selected the mta as a practice lens in designing the research and interpreting the results in the study because it acknowledges that the status of a technology can change with time and because it describes the process of adoption and actual use of a technology. it was not initially clear how well the mta would fit such a different domain to mobile technologies. research methodology our objective was to understand the process of technology adoption through to long term use. therefore, a longitudinal study was deemed most appropriate. we examined users of a software application called endnote. it is a bibliographic software package that allows users to search online bibliographic databases, organize their references and images, and create bibliographies in documents. we selected endnote for the study because our goal was to gain deeper understanding on adoption and longer-term use of an application software and we were able to access participants from their first encounter with it to later periods of use. australasian journal of information systems volume 16 number 2 2010 10 with permission from the facilitator of the endnote training courses conducted by the school of graduate studies at the university of melbourne, one of the researchers attended 5 training courses and invited participants for the study. 14 participants (9 female and 5 male) agreed to participate. twelve out of the 14 participants had no prior experience of using the software. participants were studied from their initial encounter with endnote after training (1-2 weeks) through to 20 weeks of use. data were collected at the initial encounter with the technology and 3 subsequent times over the following 20 weeks, as shown in table 1. the research design used similar methods to that of the original mta research, which included interviews, focus groups, scrap books and participant observations (carroll et al. 2002). this approach allowed us to capture and triangulate (lee 1991; yin 2002) participants' perceptions and expectations during their initial encounter with the technology after attending a training program. it also enabled us to track their likes, dislikes and actual experiences with the technology during continued use of the technology. however, it should be noted that not all of the 14 participants were interviewed and observed at every time point but attempts were made to follow up all participants. some of the participants were either overseas or busy at critical dates but all claimed that they were using endnote. only subsets of the 14 participants were interviewed and observed at critical dates (explaining the drop in the number of participants sampled in table 1). time-line no. of participants sampled techniques 1 – 2 weeks 14 interview 3 – 4 weeks 6 focus group + scrap book 7 – 8 weeks 8 participant observation + scrap book 16 – 20 weeks 7 follow-up interview table1: data collection timelines, number of participants and techniques all 14 participants were interviewed between 1 and 2 weeks after initial training. some demographic information about them, their research, their prior knowledge about endnote and the department or faculty they belong to, was noted. post hoc recollection of how they came to know of endnote and why they attended the training program were also collected. issues such as their attitude and expectations during their initial encounter with the technology were explored a time when they were faced with the decision to adopt endnote. scrap books were given to all participants during the first interview with the aim of capturing and tracking participants’ expectations, likes and dislikes of endnote when the researcher was unable to be present. the scrap book was used to validate post hoc recollections made by participants (carroll et al. 2002). a focus group of 6 participants was conducted at weeks 3 and 4, in which users shared their experiences and expectations with each other (vaughn, schumm and sinagub 1996). the use of focus groups in the collection of data encouraged interaction and greater openness among users as they shared their experiences, expectation, likes and dislikes about the technology, with each other. eight participants (5 from the focus group) were observed in their work settings between 7 and 8 weeks. they were actively probed about their actions while using the technology. use of this method facilitated collection of data about the role the technology played in users' research practices, their experience with the look-and-feel of the interface, the features they used and the reasons for selecting specific features. australasian journal of information systems volume 16 number 2 2010 11 follow up interviews were conducted with 7 participants between 16 and 20 weeks after the initial training course. the interview questions were related to how they used the technology in their research practice and their likes and dislikes about the technology and its features. they were also asked why they continued using the technology and what influences them to continue using the technology. data were collected using audio tapes and field notes. they were transcribed and descriptive codes were used to identify general and specific themes. a time ordered matrix was also used to display and analyse the themes (miles and hubermann 1994) from the data collected during different times in the research. results in this section we describe how participants used endnote to support their research practices. we also describe the influences that encouraged and discouraged their use at different times. the data are organized according to the time after the initial training course, reflecting the duration of use. decision to adopt endnote all participants stated that they attended the endnote training program because they perceived it might be useful to them in their research. after the initial encounter with endnote at the training program, they were faced with the decision whether or not to adopt this technology. influences observed to encourage the decision to adopt included the subjective norm, ease of access, expected usefulness and relative advantage, as shown in table 2. technology use during weeks 1 and 2 during weeks 1 and 2, participants were trialling endnote, not in the training environment but in their everyday study environment. they used the instruction sheet provided to them during training and their personal notes to assist them during this period. positive influences positive influences noted by participants using endnote in weeks 1 and 2 were integration, usefulness, adaptability and the ability to contact trainers. integration of endnote with other software such as microsoft word in participants’ research practice and the ability to link a word document with endnote gave the participants the perception that endnote was compatible with their needs and existing expertise. comments such as "i like that it actually ties in with microsoft word so when you are doing citing, it inserts there, it's well integrated" and "i find it very helpful, it is windows based and everything is just on the top, it’s there and i can click it" highlight this fact. this is similar to compatibility, defined by rogers (1995) as the degree to which an innovation is perceived as being consistent with the existing values, needs and past experiences. participants perceived that endnote integrated with their research practices and the existing technology used by them. usefulness, that is, user’s perceptions of how beneficial a technology is in their research practice, was a positive influence noted. australasian journal of information systems volume 16 number 2 2010 12 influences description and relevance user comments subjective no norm a person's perception that most people that are important to him/her think that he/she should or should not perform the behaviour in question (davis 1989; ventatesh et al. 2003). in this study most participants (13 out of 14) decided to attend the training because their supervisor and peers suggested they use endnote for their research program. "it was my supervisor primarily but other students in the department were also using it and found it useful and they recommended that i should certainly go for it". ease of access access to the software was free and easy. participants could obtain a copy of endnote from the library and download it on to their computers. this ease of access to endnote was one of the positive influences that led some participant’s decision to adopt the technology. "…for procite i would have to go and buy one on my own, but to use endnote i had access via the library" "i realized that it was available to everyone". expected usefulness the degree to which a person expects that using a particular system would enhance his or her job performance (adapted from davis 1989). in the study, after attending the training program, all participants stated that they expected endnote to be useful to them in their research practice. "i expect it to organize my stuff…and if i want to find something make it easier for me... help me with the citation and referencing and everything". "… the thing i really like about it is you can put all your notes in and do a key word search and bring it all together… consistency and cross referencing i suppose … i can see that it is going to be useful”. relative advantage the degree to which adopting or using the information technology is perceived as being better than using the practice it supersedes (adapted from rogers 1995). in the study, some participants perceived that using endnote would be better than their prior research methods, such as collecting references or articles manually from libraries and filing them. "if i don’t use endnote i’ll probably be sifting through lots of papers and get frustrated….endnote is supposed to collect all my information". "i can see referencing, i can see now, when, in my old research during my undergraduate studies, it was all over the place". table 2: influences that encourage decision to adopt endnote participants commented that the technology was useful in terms of the features provided by it. this is reflected in comments such as: "being able to search and find through keywords, that's very useful for me i think" and "the cite while you write feature which i found very useful". adaptability, that is, the extent to which users can personalize or customize the technology to fit their work practices, was a positive influence observed by participants. participants were in the process of trialling, exploring and learning to change their existing research practices to suit endnote: "…i have been entering whatever i have done, all my literature papers into endnote". the ability to contact the trainer and attend further training programs helped participants resolve some of the problems at this early stage. three participants contacted the trainer or attended more than one training session to further explore the technology. for example, one participant commented: "…i think for me this was the time to identify those problems and ring up whenever i need assistance…this australasian journal of information systems volume 16 number 2 2010 13 is what i have been doing…identifying the grey areas, identifying areas that i’m not too familiar with, and seeking assistance". negative influences several negative influences were noted during weeks 1 and 2 of endnote use. the difficulty in adapting and learning to use the technology were prominent negative influences. the inability to adapt individual practices due to shortcomings of the technology was observed as a negative influence. during the process of adapting practices, some participants ran into problems. for example, one participant commented, "… i was filling a research application and when i tried to add the research protocol and the reference list, the reference list went to the back of the application form, not within the same cell. i think i manually did it in the end". the participant ended up working around the problems instead of finding a solution. lack of ease of learning to use endnote was also observed as a negative influence among participants as they were in the process of learning to use it. this is reflected in comments such as: "i’m still learning it (endnote), i have needed the manual to guide me through". some participants found the technology to be unintuitive and not easy to learn to use: "at the moment you feel like you have to look up and follow it step by step because it does not speak to you from the screen, it does not suggest where to go next". the help feature on the software did not help them because they were not familiar with the terminology used: "…when i looked at the help to find a field, i don’t know what they called it. i would call it field but i think they call it something else... it’s the jargon that makes it difficult". participants expressed frustration when the instruction sheet given to them did not help them trouble shoot when they faced obstacles: "… they provide you with the manual, but sometimes manuals do not help you trouble shoot, there are certain areas where the manuals do not address". technology use during weeks 3 and 4 most participants’ expectations of endnote changed as they used the technology over 3 to 4 weeks. this is reflected in comments such as, "it’s been hard in the sense, you expect a lot and now i’m learning that i can’t get all that i want from it. it feels like as though i don’t have any expectations from it now". frustrations were building up among participants and expectations were lowered. this is reflected in comments such as: "just the little annoying things, they really add up…i’m not saying they should revolutionize it or something, i was not expecting that, but yeah, it’s not performing” or "one of my expectations of endnote was that i could just search around the database and find everything and download it into my computer, but could not…i was put off by it…i didn’t do anything about it…”. they worked around some of the problems, “…what i have used so far is just to manually find what ever i’m reading and type it in". positive influences positive influences that continued to be noted by participants in weeks 3 and 4 of using endnote were usefulness and the ability to contact trainers. usefulness continued to encourage participants to use the technology, even though expectations were lowered about what the technology could do for them: "it’s a way for me to organize my stuff, which i like… little pieces of references that i know is safely in one place and its easy to search and call up the subject list, or print out a bibliography or things i want to look up". for some, the technology continued to be useful because of some of the features it possessed. this is reflected in comments such as, "it has also helped because i have been doing a bit of writing as well, the cite while you write feature i have been using that quite well" or "i like the insertion of notes that helps when writing up...we won’t have to worry about the style". despite problems, participants continued using australasian journal of information systems volume 16 number 2 2010 14 the technology: "for me it’s the best bibliographic software there is at the moment…so that is motivation enough to continue to use endnote... it is useful, am i having problems with it? yes”. the availability of trainers and other on-line tutorials continued to help participants fix some of their problems. for example one participant had problems downloading information using endnote, "i got only the first reference from the ones i marked. that was a problem”, and contacted the trainer for help, “the [trainer] said 'down load the additional filter from the university web site" . another participant used on-line tutorial to learn to use features, "i found myself running back again to those on-line tutorials". negative influences the difficulty in adapting, integrating and learning to use the technology were prominent negative influences during weeks 3 and 4. lack of adaptability continued to be observed as a negative influence. the inability to adapt and customize some of the features led to frustration among participants: "there are many fields that i don’t use, i should be able to set it up so that i can put in the fields i would like to use" or "you should be able to customize something quite easily…there seems to be a lot of fields that you don’t need it". having too many features did not help participants adapt the technology. for example, one participant said, "there are too many fields some of which i don’t know…some of the fields… you don’t need them…also missing features that should be there". lack of easy integration of endnote with other databases was another negative influence that was evident from participant’s comments. while participants liked the ability of endnote to tie up with microsoft word, they complained about the lack of information given to them on updates of filters that allow references to be accessed from other databases. one participant said, "obviously it has problems. if they have additional downloads, then they could just email the user, something to say, or automatically update". lack of ease of learning continued as a negative influence with time. participants found it difficult to remember the number of steps they had to go through in order to get a particular task done. the inability of the technology to guide the user through the steps frustrated participants, "if you find out eventually what to do and you repeat it enough of times, then you can learn to do anything really. it takes so long, i find myself consciously trying to remember the steps". if a technology is not easy to learn then it can influence users to start comparing other technologies "nothing seems intuitive…and i have found myself trying the help all the time… i can’t recall any other software that i have ever pressed help as many times as this one and even then it didn’t quite help me". technology use during weeks 7 and 8 in weeks 7 and 8, it was noted that little or no change was observed in the way participants used endnote. all participants used endnote as a part of their everyday research practice. routinized and stable activities with the technology were observed. for example one of the participants said, "i’m not experimenting with it. i don’t want to spend more time. i know the basic functions of endnote, i know i need one and its there".. participants had adapted to the technology. this is reflected in comments such as: "i think this whole technology is tuning me to work around it…it is dictating the manner i should work which is ok with me" or "…it also helped me standardize all my references in the bibliography so i don’t have to worry". they also adapted the technology to suit their needs by selecting some of the features and opted not to use other features that the technology provided. this is reflected in one such comment, "i have not used all the columns, some of them i think are irrelevant, may be not irrelevant but just that i don’t have the data available to me... basically i use certain features all the time such as title, author, year, journal, volume, issue, keywords... i hardly use the australasian journal of information systems volume 16 number 2 2010 15 short title... it may be necessary later but at this point i don’t use". thus, stabilization of the technology was attained as a result of mutual adaptation participants adapted the technology and adapted to the technology to suit their needs. positive influences usefulness continued to influence longer term use. usefulness was expressed in terms of the technology as a whole, "it provides a platform to organize my references…" and the useful features provided by it, "that’s what i wanted to do, cite while you write, have a database with references and to be able to put them into your word document and standardize them. i really like that you can change it if you are writing an article and want a particular style you can change it and you don’t have to manually go and change it". negative influences however, even after months of use, some negative influences on participants’ use, such as lack of integration with other databases and lack of ease of learning to use features provided by the technology, continued to persist over time. lack of integration with other databases was an issue that continued to frustrate some participants. participants were unable to either transfer information from one software to endnote or connect to databases related to the specific area of research. one participant said: "i can’t search from outside because i can’t connect to those libraries. so i have to manually search outside using my music journals, they have their own databases. i will slowly switch between the endnote and the databases and enter it myself. it’s tiring though". lack of ease of learning continued to be a problem among participants during further exploration and adaptation of endnote. participants commented that they found it difficult to remember the various steps they had to click before getting to what they wanted. for example one participant said, "…that’s not the way to go, because it’s not in the screen, i have to memorize how to do all that, and if i go away for a few days and didn’t remember the right button to push, i’m back again to the manual, because it’s not like as its easily remembered". the lack of ease of learning to use the technology due to usability issues may be a hurdle even if easy access to the manual or the trainer be made available when problems need to be resolved. this is reflected in one such comment, "some of the features could be more detailed, like if you get stuck it should be able to tell you then and there, how to go about it… i think that was frustrating for me because you don’t want to be calling the instructor all the time…i was reading the manual itself several times, its quite detailed but if you don’t know what the problem is, then it does not make sense". not all features provided by the technology, were used by participants, due to the lack of knowledge or further exploration of the features supported by the technology. for example, one participant said, "when i key in, there are too many entries in the bottom… alternate title, translator etc… they are not relevant to me. i would like to get rid of them in my course. whether i can customize that i don’t know, it’s not obvious to me". although participants faced problems while using the technology, stabilization of the technology was attained as participants worked around problems they encountered during the process of appropriation. the negative influences did not stop any of the participants from using the technology. technology use after 16 to 20 weeks in weeks 16 to 20 weeks, some users of endnote expressed the need to resolve some of their existing problems which they had faced while using the technology earlier-on, "…a few niggling things were bothering me, so i went for this course”. a few others expressed the need to further explore and adapt new features provided by endnote, “…i thought i’d try to make my own style”. australasian journal of information systems volume 16 number 2 2010 16 positive influences influences such as usefulness and easy access to ongoing training continued to encourage use of the technology. usefulness was expressed in terms of saving time while using endnote in their research practice, “there was a time when i thought it was creating more work, now it’s actually going to work from me. it’s saving time". in addition, usefulness was also expressed in terms of the ability to re-use and share information with others. some participants realized that the technology could be a common platform where large amounts of information can be collected, organized and later shared among peers working in the same area of research. for example one of the participants said, "…other people working in other parts or areas can also be able to look at this…that’s the reason i thought better pursue it if it’s in a commercial format then other people can then use it for other purpose… that’s why i ultimately persevered using it". easy access to ongoing training helped participants resolve existing problems that arose while adapting the technology to suit their new needs, "things like these the importing and exporting…with the training it made it easy. this filter thing is not something that i would have known if i had not gone for the training”. the training session also helped users further explore and use the technology. this is reflected in comments such as: "… i went for this course and asked her [trainer] a few things and then i clicked this one thing and it changed from lower case to upper case" or "…i enrolled into the course and found out that i could just have adjusted or modified one of the other styles". the on-going training also emphasized the capabilities of the technology to participants: "… i think i know now what the program limitations and my limitations are. before, it was big frustration…but now i know what it can and can’t do and i know i can’t push it after more than it can give”. negative influences negative influences on participants’ use, such as lack of adaptability and lack of ease of learning to use features provided by the technology, continued to frustrate users of endnote, even after 16 to 20 weeks. lack of adaptability of the technology continued to be a negative influence for some participants as new needs arose. as they encountered new tasks or activities in their research practice, over time, they expressed frustration in adapting endnote to suit those immediate needs. for example one of the participants commented that adapting the style feature to suit her research practice, was not easy, "i did not know how to change the existing style…and the way i do it now is, i have to start from scratch and make my own style… there must be some easy way to do it". lack of ease of learning continued to negatively influence further exploring and adapting the technology, "they [references] were in upper case or lower case and then when i print…it would all be in lower case. it was really annoying me". summary of findings the positive and negative influences that were noted at different time points are summarized in table 3. as seen in table 3, the most prominent positive influences that encouraged users of the technology in weeks 1 and 2 were integration, usefulness expressed in terms of useful features, adaptability and the ability to contact trainers and attend training courses. the lack of ease of learning to use the technology and the lack of adaptability in their everyday research practices were prominent negative influences observed in weeks 1 and 2. australasian journal of information systems volume 16 number 2 2010 17 over time, as users gained experience in using the technology, usefulness continued to be a strong influence on use of endnote. usefulness was expressed in terms of the technology as a whole, its features, the ability to re-use and share information and the ability to save time while using it. in addition, the ability to contact trainers and attend on-going training courses was an on-going influence on continued use of the technology. however, negative influences such as lack of ease of learning and the lack of adaptability were prominent negative influences over the 20 weeks. discussion by investigating how and why users adopt and then adapt a technology to suit their needs, over time, our study highlights the varying influences that encourage and discourage longer-term use and the uncertain length of time before a technology stabilizes. in identifying the influences that encourage and discourage appropriation and long term use of a technology in an educational environment, we view our findings through the practice lens of the mta, which has three levels (figure 1). at the initial encounter, when the decision to adopt a technology is made (mta level 1), we found that perceived usefulness, subjective norm and relative advantage were important influences that supported users’ decision to adopt. these findings are consistent with previous reports by others about subjective norm (davis 1989; venkatesh et al. 2003) and relative advantage (moore and benbasat 1991; rogers 1995; fidock, and carroll 2006). we also found that easy and free access to a technology was a positive influence. the importance of cost has been observed in the mta related studies on mobile technologies. these studies have identified cost, in terms of purchase and expected usage, as an influence that encouraged the decision to adopt (herszfeld, carroll and howard 2003; carroll et. al 2003). we therefore suggest that technologies with low or no purchase cost and easy availability may influence the decision to adopt along with other perceptions such as usefulness. perceived ease of use has been a strong influence in some studies on technology adoption (davis 1989; venkatesh et al. 2003) and appropriation of mobile and sms-related environments (herszfeld et al. 2003; carroll et al. 2003). in this study however, we did not note perceive ease of use as an influence in decision to adopt endnote. we suggest that perceived ease of use could sometimes be less important, for instance when users perceive that adopting the technology may improve their everyday work practices or simply the inexperience with the technology could hinder users’ judgement on whether a technology is easy to use or not. strong influences such as expected usefulness, subjective norm, relative advantage and easy and free access to the technology may be stronger driving forces in enabling early decision-making with perceived ease of use taking a backseat in the decision to adopt. in addition, the formation of perceptions, attitudes and high expectations, an outcome of effective training, may have contributed in the decision to adopt the technology. in this study, we found that some of the positive influences that encouraged adoption such as subjective norm, relative advantage and easy access to the technology, did not appear to persist over time to encourage actual use. therefore, this study suggests that influences on users’ initial decision to adopt a technology may not be sufficient to encourage longer-term use of a technology (also see mendoza et al. 2008). as users interact with the technology, explore its capabilities, adapt to the technology and adapt the technology to suit their needs (mta level 2), their expectations and perceptions change. new influences emerge as driving forces in technology appropriation, when users adapt to the technology and adapt the technology to suit their needs in their everyday practices. australasian journal of information systems volume 16 number 2 2010 18 influence s decision to adopt 1 2 wks 3 4 wks 7 8 wks 16 20 wks positive  expected usefulness  usefulness (useful features)  usefulness (useful features)  usefulness (useful features)  usefulness (useful features) (ability to reuse & share information) (ability to save time)  integration  adaptability  ability to contact trainer/attend training  ability to contact trainers/acces s to online tutorial  easy access to ongoing training  relative advantage  easy access  subjective norm negative none observed  lack of ease of learning  lack of ease of learning  lack of ease of learning  lack of ease of learning  lack of adaptability  lack of adaptability  lack of adaptability  lack of integration  lack of integration table 3: positive and negative influences at different time points our study reveals that, with time, as users actually use a technology, ease of use may or may not emerge as an influence that encourages or discourages actual use of a technology. different attributes of ease of use may emerge as users explore and adapt the technology in their everyday practices. in our study, we found that users expressed ease of use in terms of ease of learning to use a technology. users of endnote found it difficult to learn to use features of the technology during the entire period (observed from weeks 1-2 until 20). it is therefore important to realize that the determinant ease of use (davis 1989; venkatesh et al. 2003) may posses varying attributes over time. it is therefore necessary to make a clear distinction between ease of use and ease of learning to use while discussing influences that encourage and discourage appropriation and long-term use of a technology. this study also reveals that users’ perception of expected usefulness may shift to usefulness as users actually manipulate and evaluate the technology to suit their work practices. in this study we found that usefulness was noted as a strong influence among users of endnote throughout the study supporting previous findings (karahanna et. al. 1999; mendoza et al. 2005; fidock et al. 2006). new australasian journal of information systems volume 16 number 2 2010 19 attributes of usefulness emerged with time. usefulness was initially expressed in terms of the useful features and was later expressed in terms of the ability to re-use and share information. our study suggests that users re-evaluate and continue changing their expectations over time. it is therefore important that information systems researchers gain deeper understanding of changing influences on the process that leads to longer-term use in respect to different technologies by different cohorts, in order to gain in-depth understanding of when, how and why expectations and perceptions change over time. our study also suggests that lowered expectations and negative influences that persist with time may not discourage use completely if users see continued benefit from using a technology (also see mendoza et al. 2008): that is, if the value of the benefits overrides the negative influences. we found that, for all endnote users, the positive influence of usefulness outweighed the negative influences, such as the lack of ease of learning, integration and adaptability. it should be noted that a class-room based training session during adoption or early use may not be sufficient in encouraging longer term use. in this study, it was noted that training did not seem to overcome lack of ease of learning among users. the key to encouraging continued use may be easy access to varied support mechanisms. contacting trainers and attending advanced training programs could be a way of encouraging users to overcome negative influences and see further benefits from using the technology. in our context, the ability to contact trainers or attend on-going training courses was a positive influence whenever users were adapting and exploring the technology. with respect to stable use of the technology (mta level 3), we observed that use of the technology stabilized in weeks 7-8. our findings suggest that users established ways of using the technology to gain value from it by working around the problems they faced earlier or by lowering their expectations (tyre and orlikowski 1994; mendoza et al. 2005). users chose to employ some features to suit their research practices and temporarily rejected some or ignored other features. adaptation took place in different forms: changing the physical configuration of the technology and substantially altering their existing research practices to suit the technology which may also involve using a suite of other technologies (also see mendoza et al. 2007). usefulness continued to positively influence longer-term term use of the technology. our study suggests that even after a technology is stable in use, further appropriation can still take place. that is, adaptation continues after an initial stabilisation of technology-in-use. deeper evaluation of longer-term use in our study (weeks 16-20) revealed that stabilization of a technology may only be a temporary plateau. a stable technology can re-enter the process of appropriation. this re-entry has been noted in some previous findings (carroll et al. 2002; tyre and orlikowski 1994). the need to explore and further adapt the technology is based on emerging events and on the need to resolve some of the problems that may have been experienced previously. these factors encouraged some users to further explore and adapt the technology even after a period of stabilisation. in addition, we found that strong negative influences such as the lack of adapting the technology to new needs and the lack of ease of learning to use the technology did not stop longer-term term use. it appears that the positive reinforcers are stronger than the negative influences. in this study we found that the ability to re-use and share information, a new aspect of usefulness, emerged as a strong positive influence, well after the plateau. we also found that the availability of an ongoing training program helped overcome the negative influences and acted as a facilitating condition (thompson et al. 1991) that influences further appropriation. therefore, our findings reveal that there may be multiple windows of opportunity for appropriation (also see mendoza et al. 2007), unlike holstrom (1999) and tyre and orlikowski (1994) who observed a certain "window of opportunity" when adaptation is most likely to occur. australasian journal of information systems volume 16 number 2 2010 20 our work suggests that one way of encouraging further appropriation and avoiding stagnation of technology use is to provide ongoing access to support, such as advice from experts or further training. such support was valuable for our participants through an initial dip in expectations, observed between weeks 1 to 4, and frustrations in resolving issues throughout the 20 weeks of this study. some users in this case study attended a voluntary advanced training session. the training session helped them resolve pre-existing issues and further adapt the technology to suit ongoing activities in their research practices. from our experience, we argue that designers, trainers and managers need to be aware that providing training for users during the implementation stage alone may not be sufficient to support and improve persistent and longer-term term use of the technology. access to a variety of support mechanisms, throughout the periods of exploration and stabilisations, may be a key factor in encouraging productivity and user satisfaction. conclusion three main implications have emerged from our study. first, influences on users’ appropriation activities change with time and user needs. second, as a consequence of changing influences, the influences that encourage adoption of a technology may not be sufficient to sustain longer-term term use. as users move from the initial adoption to longer-termer term use, a shift from perceptions of expected usefulness to usefulness may be seen as users actually manipulate and evaluate the technology to suit their work practices. over time, new aspects of usefulness can emerge. third, stabilization of a technology may only be temporary, where users’ knowledge and expertise in using the technology are at a plateau. changing needs, curiosity about possible features or changing evaluations of the technology encourage users to re-enter the process of appropriation – to explore and construct new practices from a stable use of the technology. training supports this re-entry. our study therefore draws attention to the crucial role played by ongoing training services in encouraging the process of appropriation leading to persistent and productive use of a technology. the access to training programs and trainers may encourage users to overcome negative influences and see benefits from using a technology. evidence of plateaus of stabilisation and re-entry into the process of appropriation between 16 and 20 weeks suggests that both researchers and practitioners have under-estimated the length of time required for some more enduring stabilisation (if achieved). the findings in this research have been based on an in-depth study of how and why 14 participants appropriated endnote over 20 weeks. theoretical issues on the appropriation process including stabilization and further appropriation have been addressed. further research will need to focus on understanding appropriation by different cohorts over a longer-term time period. therefore, we plan to investigate longer-term use of different technologies in an educational environment and to extend the period of research beyond 20 weeks. references adams, d.a, nelson, r.r and todd, p.a. (1992) 'perceived usefulness, ease of use and usage of information technology: a replication', mis quarterly, vol 16 no 2, pp 227-47. brown, s.a., massey, a.p., montoya-weiss, m.m. and burkman, j.r. (2002) 'do i really have to? user acceptance of mandated technology', european journal of information systems, vol 11, pp 283-295. bhattacherjee, a. (2001) 'understanding information systems continuance: an expectationconfirmation model', mis quarterly, vol 25 no 3, pp 351-70. australasian journal of information systems volume 16 number 2 2010 21 carroll, j., howard, s., peck, j. and murphy, j. (2002) 'a field study of perceptions and use of mobile telephones by 16 to 22 year olds', journal of information technology theory and application (jitta), vol 4 no 2, pp 49-62. carroll, j., howard, s., peck, j. and murphy, j. (2003) 'from adoption to use: the process of appropriating a mobile phone', australian journal of information systems (ajis), vol 10 no 2, pp 38-48. chau, p.y.k., and hu, p.j. (2002a) 'examining a model of information technology acceptance by individual professionals: an exploratory study', journal of management information systems, vol 18 no 4, pp 191-229. chu, t. and robey, d. (2008) 'explaining changes in learning and work practice following the adoption of online learning: a human agency perspective', european journal of information systems, vol 17, pp 79-98. ciborra, c.u. (1996), groupware and teamwork, john wiley & sons, new york. compeau, dr., meister, d.b. and higgins, c.a. (2007) 'from prediction to explanation: reconceptualizing and extending the perceived characteristics of innovating', journal of the association for information systems (jais), vol 8 no 8, pp 409 39. cooper, r.b., and zmud, r.w. (1990) 'information technology implementation research: a technological diffusion approach', management science, vol 36, pp 123-39. davenport, t., jarvenpaa, s., and beers, m. (1996) 'improving knowledge work processes', sloan management review, vol 37 no summer, pp 53-65. davis, f.d. (1989) 'perceived usefulness, perceived ease of use and user acceptance of information technology', mis quarterly, vol 13 no 3, pp 319-339. davis, f.d., bagozzi, r.p., and warshaw, p.r. (1989) 'user acceptance of computer-technology a comparison of two theoretical models', management science, vol 35 no 8, pp 982-1003. desanctis, g. and poole, m.s. (1994) 'capturing the complexity in advanced technology use: adaptive structuration theory', organization science, vol 5 no 2, pp 121-147. fidock, j., and carroll, j. (2006) 'the model of technology appropriation: a lens for understanding systems integration in a defence context', proceedings of the 17th australasian conference on information systems (acis 2006), adelaide, australia. gallivan, m.j. (2001) 'organizational adoption and assimilation of complex technological innovations: development and application of a new framework', the data base for advances in information systems, vol 32 no 3, pp 51-85. giddens, a (1984) university of california press, berkeley, ca. griffith, t.l. (1999) 'technology features as triggers for sense making', academy of management review, vol 24 no 3, pp 472-88. herszfeld, s., carroll, j., and howard, s. (2003) 'job allocation by sms: technology appropriation in the construction industry', proceedings of the 14th australasian conference on information systems (acis 2003), perth, australia. holstrom, j. (1999) '"it’s as if somebody slammed the door" on the limits of organizational change around information technology', proceedings of the 7th european conference on information systems, vol 1, pp 320-329. igbaria, m., zinatelli, n., cragg, p., and cavaye, a.l.m. (1997) 'personal computing acceptance factors in small forms: a structural equation model', mis quarterly, vol 21 no 3, pp 279305. australasian journal of information systems volume 16 number 2 2010 22 jasperson, j., carter, p.e and zmud, r.w (2005) 'a comprehensive conceptualization of postadoptive behaviours associated with information technology enabled work systems', mis quarterly, vol 29 no 3, pp 525-57. karahanna, e., straub, d.w. and chervany, n.l. (1999) 'information technology adoption across time: a cross-sectional comparison of pre-adoption and post-adoption beliefs', mis quarterly, vol 23 no 2, pp 183-213. lee, a. s. (1991) 'integrating positivist and interpretive approaches to organizational research', organization science, vol 2, pp 342-365. leonard-barton, d. (1988) 'implementation as mutual adaptation of technology and organization', research policy, vol 17 no 5, pp 251-267. lyytinen, k. and j. gaskin (2010) 'psychological ownership and the individual appropriation of technology', proceedings from the 43rd hawaii international conference on system sciences, koloa, kauai, hawaii. majchrzak, a., rice, r., malhotra, a. and king, n. (2000) 'technology adaptation: the case of a computer-supported inter-organizational virtual team', mis quarterly, vol 24 no 4, pp 569600. markus, l.m., and robey, d. (1988) 'information technology and organizational change: causal structure in theory and research', management science, vol 34 no 5, pp 583-98. markus, m. l and silver, s. m (special issue 2008) 'a foundation for the study of it effects: a new look at desanctis and poole’s concepts of structural features and spirit ' journal of the association for information systems, vol 9, no 10/11, pp. 609-632. mendoza, a., carroll, j. and stern, l. (2005) 'adoption, adaptation, stabilization and stagnation: software appropriation over time', proceedings of the 16 th australasian conference on information systems (acis 2005), sydney, australia. mendoza, a., carroll, j. and stern, l. (2007) 'plateaus in long-term appropriation in an information system', proceedings of the 18 th australasian conference on information systems (acis 2007), toowoomba, australia. mendoza, a., carroll, j. and stern, l. (2008) 'influences on continued use of an information system: a longitudinal study', proceedings of the 16th european conference on information systems (ecis 2008), galway, ireland. miles, b.m. and hubermann, a.m. (1994) qualitative data analysis, 2nd ed., sage publications. moore, c.g. and benbasat, i. (1991) 'development of an instrument to measure the perception of adopting an information technology innovation', information systems research, vol 2 no 3, pp 192-222. orlikowski, w.j. (2000) 'using technology and constituting structures: a practice lens for studying technology in organisations', organisation science, vol 11 no 4, pp 404-28. orlikowski, w.j. (1996) 'improvising organizational transformation over time: a situated change perspective', information systems research, vol 7 no 1, pp 63-92. orlikowski, w.j. (1992) 'the duality of technology rethinking the concept of technology in organizations', organization science, vol 3 no 3, pp 398-427. raymond, l. (1990) 'end-user computing in the small business context: foundations and directions for research', database, vol 20 no 4, pp 20-26. rogers, e.m. (1995) diffusion of innovations, 4th ed., free press, new york. straub, d., keil, m., and brenner, w. (1997) 'testing the technology acceptance model across cultures: a three country study', information and management, vol 33 no 1, pp 1-11. australasian journal of information systems volume 16 number 2 2010 23 sun, h. and zhang, p. (2006) 'applying markus and robey's causal structure to examine user technology acceptance research: a new approach', journal of information technology theory and application (jitta), vol 8 no 2, pp 69-81. szajna, b. (1996) 'empirical evaluation of the revised technology acceptance model', management science, vol 42, no 1, pp 85-92. taylor, s. and todd, p.a. (1995) 'understanding information technology usage a test of competing models', information systems research, vol 6 no 2, pp 144-176 . thompson, r.l., higgins, c.a. and howell, j.m. (1991) 'personal computing: towards a conceptual model of utilization', mis quarterly, vol 15 no 1, pp 124-143. trigg, r. and bodker, j. (1994) 'from implementation to design: tailoring and the emergence of systematization in cscw', acm conference computer-supported cooperative work cscw 94, pp 45-54. turner, b. (ed.) (1988) 'connoisseurship in the study of organizational cultures' in doing research in organizations ', london: routledge. tyre, m.j. and orlikowski, w.j. (1994) 'windows of opportunity: temporal patterns of adaptation in organizations', organization science, vol 5 no 1, pp 98-118. van de ven, a.h. and poole, m.s. (1995) 'explaining development and change in organizations', academy of management review, vol 20 no 3, pp 510-40. van der heijden, h. (2004) 'user acceptance of hedonic information systems', mis quarterly, vol 28 no 4, pp 695-704. vaughn, s., schumm, j.s. and sinagub, j.m. (1996) focus group interviews in education and psychology, sage publications, thousand oaks. venkatesh, v. (1999) 'creation of favourable user perceptions: exploring the role of intrinsic motivation', mis quarterly, vol 23 no 2, pp 239-260. venkatesh, v. and davis, f.d. (2000) 'a theoretical extension of the technology acceptance model: four longitudinal field studies', management science, vol 45 no 2, pp 186-204. venkatesh, v., morris, g., davis, g.b. and davis, f.d. (2003) 'user acceptance of information technology: towards a unified view', mis quarterly, vol 27 no 3, pp 425-454. xia, w. and lee, g. (2000) 'the influence of persuasion, training, and experience on user perceptions and acceptance of it innovation', proceedings of the twenty first international conference on information systems (icis), pp 371-384. yin, r. k. (2002) case study research, design and methods, 3rd ed., sage publications, newbury park. ajis special edition electronic commerce november 1998 inter-organisational systems, strategy and structure: the case for planning environments pat finnegan«, robert galliers** and philip powelw •executive systems research centre, university college, cork, ireland e-mail: p.finnegan@ucc.ie ••information systems research unit, warwick business school, university of warwick, coventry, cv4 7al, uk ft depl of maths and computing sciences, goldsmiths college, university of london, new cross, london, uk e-mail: orspp@razor.wbs.warwick.ac.uk abstract electronic trade and commerce have brought inter-organisational co-operation to the forefront of organisational strategies. however, inter-organisational systems (1os) planning lacks appropriate methodological guidelines, and tends to be practised in an insular manner. this paper examines the development of systems in three interorganisational networks in ireland in order to ascertain the planning that went into them. the networks studied are representative of three types of structural interdependencies pooled, reciprocal and sequential. although it is difficult to identify an activity that can be clearly labelled ios planning, planning is integral to the development of all networks. the interdependencies influence planning processes and outcomes. for the pooled and reciprocal networks, the planning process gives a focus. for the sequential network, the main planning is done by the hub organisation, and extended to the spokes. planning interactions across all networks illustrate similarities that reveal that existence of planning environments within which ios planning takes place. these planning environments are proposed as being an appropriate basis for the development of ios planning guidelines. introduction a number of researchers have argued that, in order to cope with its environment and reduce environmental uncertainty, an organisation should link with others (e.g. van de ven, 1976; provan, 1982; borman, 1994). such links encourage inter-organisational communication and co-ordination, and thus reduce environmental uncertainty (eisenberg et al., 1985). while consideration of the potential of inter-organisational co-operation has been advocated since the 1960s (kaufman, 1966), it is the recent possibility of electronic trade and commerce (kuula, 1995) that has brought inter-organisational co-operation to the forefront of organisational strategies. yet, inter-organisational systems (ios) planning lacks guidelines, and tends to be practised in an insular manner (finnegan et al., 1998). this paper examines the development of systems in three inter-organisational networks, in order to examine the basis of the planning underpinning them. the networks represent three types of structural interdependencies which influence planning processes and outcomes. however, planning interactions across the networks illustrate similarities that reveal that existence of archetype planning environments within which inter-organisational systems planning takes place. these environments may form the basis for the development of ios planning guidelines. the paper begins by considering the role of inter-organisational systems in the development of electronic trade and electronic commerce. this highlights the need for planning. next the research framework is set out the research findings are then analysed in terms of strategy, planning, infrastructure, organisational activity and personnel. this leads to the identification and discussion of planning environments. electronic trade, commerce and inter-organisational systems the terms electronic trade, electronic trading and electronic commerce are essentially synonymous, and refer to trading transactions that are conducted using electronic media (kuula, 1995). byles (1993) defines electronic commerce as 'the use of electronic information and network systems for the transmission and recording of the many kinds of communications in a commercial transaction'. cunningham and tynan (1993) refer to electronic trading as 'any trading relationship which relies upon the use of computer technology for inter-organisational communications, normally involving telecommunication links'. these definitions agree on the use of telecommunications to conduct commercial transactions, but differ according to the type of market to which they 45 ajis special edition electronic commerce november 1998 refer. electronic trading is business-to-business transactions in an industrial context, while electronic commerce refers to retailing and the consumer sector as well as mass marketing (kuula, 1995). electronic trading systems exploit information technology to improve the efficiency of communications and/or to alter the nature of inter-organisational transactions. electronic trading systems and extra-organisational systems are inter-organisational systems designed to operationalise electronic business opportunities as shown in figure 1. electi business *• * ••> » ' . " • ' _ , ' ' efectrdn/c commerce dimensions: mass, retail and consumer markets , systems: extra organisational systems u ' 1 .-, > a:,; \f/ectwfif$ trade .•;"•" " ' '/ [dimensions: business to business [systems: inter organisational systems 1 ;" electronic trading systems /-* figure 1: electronic business and ios an element common to both electronic trading and electronic commerce is the use of electronic marketing channels. this refers to the electronic applications that provide the mechanism for purchasing and selling products and services. such channels are also referred to as electronic linkages or electronic connections, which are, in general, associated with inter-organisational systems (kuula, 1995). inter-organisational systems can positively affect inter-organisational transactions by reducing costs and improving efficiency (swatman and swatman, 1992; cox and ghoneim, 1994). in addition, they reduce environmental uncertainty by facilitating communication and providing information (henderson, 1990; bergeron and raymond, 1992; scala and mcgrath, 1993). finally, they can also affect the competitive positioning of organisations by encouraging closer relations with suppliers and customers (cash and konsynski, 1985; swatman and swatman, 1992; west, 1994). this diversity of inter-organisational functions is reflected in the variety of ios available. based on the work of thompson (1967) and robey and sales (1994), kumar and van dissel (1996) propose a topology of inter-organisational systems. in this topology, ios are categorised according to the level of organisational interdependency they facilitate. interdependency means conditions where organisations rely on, or are influenced by, others in relation to their operations or financial welfare. kumar and van dissel's categorisation includes pooled interdependency, sequential interdependency, and reciprocal interdependency as shown in table 1.2. pooled ios involve the sharing of is/it resources, value/supply-chain ios facilitate customer-supplier relationships along the value chain, and networked ios operationalise reciprocal inter-dependencies such as joint ventures and strategic alliances. these interdependences range along a continuum from pooled to reciprocal, with those further along the scale inheriting the characteristics of the earlier ones. further, as organisations may be involved in more than one type of inter-organisational relationship, a particular ios may exhibit characteristics of more than one type of interdependency (kumar and van dissel, 1996). the importance of this topology is that kumar and van dissel (1996) recognise that inter-organisational structures, and the design, implementation and operation of ios are inter-dependent. this means that organisational structures affect systems choices rather than the traditional view that technology affects structures only. inter-organisational systems development is consequently constrained by more factors than just technology. this is also demonstrated by research on ios success. organisational factors are consistently found to be more important than technical factors in determining ios success (clarke and jenkins, 1993; holland et al., 1994; finnegan and golden, 1996). of these, planning is regarded as the most important in influencing the success of inter-organisational systems (finnegan et al., 1998b). 46 ajis special edition electronic commerce november 1998 type of interdependence >• , . " . ' . ...a,: configuration * , ^ »*• co-ordination mechanisms •• technologies t *j -v structurabihty y potential for conflict type of ios examples of ->• implementation technologies and applications a jpooled sequential reciprocal interdependence interdependence interdependence q^ ^c ct "c standards and rules * mediating < ' * * ' high j :; " i | o • • • i '<^> *c~*} *(~^1 (^~^u \ fc/^^' iv^v n.̂ ^^-^j f^-fs \ > « v j . ^. ,'' ; i8^.-: ' standards, rales, standards, rules, p schedules and schedules, plans & . s ^ plans mutual adjustment ^ long-linked ^ intensive .medium low f j.' /' : • ,. . ' • . • low .» medium high •e i . ' . • ' • ' • • • pooled mformanon value/supply-chain networked ios resource ios ^ ios shared~databases oedi apphcauons cad/gasedata ;., networks _ voice mail niterchange apphcations facsimile central repositories . electronic markets desk-top sharing : a^ideo^ionferencing ' * -v .. ' • • . • : • : 4 . " • • " . ' • , ' • ''<••'. \ • .-•• ' ''•'.••• • table 1: an ios topology (from kumar and van dissel, 1996) the need for planning the advent of inter-organisational systems and the growth of electronic commerce presents new challenges for is planning processes. the few existing ios planning guidelines are fragmented and focus on individual ios technologies such as electronic data interchange (edi) and group technology (holland and lockett, 1990; opper, 1994). these technological guidelines assume that ios planning is accommodated within existing is planning processes as just another technology, and neglect the fundamental non-technical aspects of ios planning and its organisational base. while emerging technologies such as edi and the internet may be accommodated within existing planning structures, the need to involve external parties, and the associated issues of co-operation and control are outside the frame of reference of existing planning efforts. in addition, the interorganisational structure (kumar and van dissel, 1996) which provides the context for planning is significantly different especially in terms of its complexity. while some is planning methods consider analysing the external environment as an integral part of the planning process, they do not deal with issues of designing and developing information systems that transcend organisational boundaries. although some systems have been successfully developed within a single organisation and utilised across organisational boundaries (e.g. american hospital supplies), these systems have 47 ajis special edition electronic commerce november 1998 tended to arise in a somewhat serendipitous manner and, by and large, control has remained with the developing organisation. in addition, recent research has indicated that the real benefits of inter-organisational systems go hand in hand with high levels of business and technological change. this means that organisations need to make substantial changes to their activities, structures and systems in order to take full advantage of ios. it is unlikely that such high level changes will not occur by a one organisation imposing systems onto others, though there are examples of this e.g. webster (1995). given the above and the limited exploratory empirical research on ios planning, research that examines the planning and deployment of inter-organisational systems in order to explain the nature of ios planning appears to be timely. the research framework it was decided to adopt a case study research method given the exploratory nature of the research and the need to obtain rich data in the complex inter-organisational contexts under examination. 'a case study examines a phenomenon in its natural setting, employing multiple methods of data collection to gather information from one or a few entities (people, groups, or organisations). the boundaries of the phenomenon are not clearly evident at the outset of the research and no experimental control or manipulation is used' (benbasat et al., 1987). yin (1984) proposes that case studies are most appropriate when the research objective involves studying contemporary events, without the need to control variables or subject behaviour. the strength of case studies is their ability to capture reality in greater detail, and to analyse more variables than is possible with any other method (galliers, 1991). pettigrew (1992) proposes that context and action are interwoven in the study of strategy and it is important to consider the past and present when looking to the future. this further justifies the choice of case methods, as it emphasises contextual understanding and emphatic research objectives (gable, 1994). within the case method, a multiple case study research design was devised so as to facilitate the collection of context data on ios planning within a natural setting. eisenhardt (1989) suggests that fewer cases are more suitable when there are mini cases within the larger ones, as here. three inter-organisational networks were selected on the basis of type and structure of network, type of planning experience and technology and systems used. the study includes examination of the planning processes of partner / participant firms in the interorganisational network. the case studies were chosen to give diversity in the nature of inter-organisational networks rather than any preconceived notion of best practice. a combination of data collection methods was used. the primary methods were personal interviews and document analysis. the choice of personnel was based on willingness to co-operate, involvement in ios implementation, nature of knowledge, and seniority. each interviewee was asked to indicate the most appropriate personnel to speak to regarding the development of the network. a small number of people were involved in each case due mainly to the personal nature of the planning activity. the third network contained a larger number of participants, but even here there were a small number of key decision makers. all those identified as key personnel were interviewed where possible. those interviewed had both technical and general business backgrounds, and included senior and middle management. the networks are classified as examples of pooled dependency, sequential dependency, and reciprocal dependency as described by kumar and van dissel (1996). as shown in table 2, these systems can also be categorised as being examples of inter-organisational systems (ios) and extra-organisational systems (eos) in their support of electronic trade and commerce. for confidentiality reasons, the organisations are referred to by pseudonyms. network 1 operates in the pharmaceutical industry and is an example of pooled interdependency. the network uses an open standard inter-organisational system, co-operatively developed by wholesalers and pharmacies during the mid-1980's in order to share information on product pricing and availability, and to act as an electronic market for ordering. network 2, an example of sequential dependency, uses edi based inter-organisational systems operating between a retail supermarket and its suppliers operating in the grocery retail sector in ireland. the main systems used relate to invoicing, price catalogues, credit notes, and dispatch advice. network 3 is a group of 15 independent organisations from different european states, operating as a virtual organisation providing and promoting technologies for business process transformation. this network delivers on-line products and services over the internet to european organisations engaged in business process redesign. 48 ajis special edition electronic commerce november 1998 -.•'••• /. :-: --network.-:' ' ; : ; . r ;netwofkl . 3 :̂.a •;•-'•; : (pharmaceutical fe&isfiy).' . :,'. • '•:*.;f-']w~:*.~"c'-:'-.j';'fpv?rf''':. '• ' . tfetwpiki .,: v^:--1 • . . ' ,• .'>, '(rffbfif sector) ' .• ' ' " '•'<i.x:--•;'• : : ' '"/*?& •/••'. '••.r,^. •vr^;^ !%$f$^; nettvprksyv £'y '. 'v '•'. (internet baied virtual , • organisation) : -y . •'. ;v'&£'!w^-:-'-. "' '.'.' •.''.*, ;»;£?• «v":. ;;v;.' '•; ; i^y*.-/ -.',• . *v '.rv"t^r ".'' *.''i ^ , • '• i'-v' • ' ' • • .. •: :.'•'""• ~& '. ''•• •• business electronic trade electronic trade electronic trade electronic commerce : '-••''. systems .":" -| intei--wganisati6nal ;• database systeniifo" . , sb^eprbduct. information. v , edi-basedlntety v organisational ] tradmgysysteins for . ! ;, price cataloguing and invoicing. „ : : losb'efireen . , : i partners for . developing a virtual • organisation and i providing services. ^ ! eos with customers j to deliver services. table 2: inter-organisational networks studied analysis examination of ios planning in the networks reveals that, while little thought was given to planning processes in any of the organisations studied, planning was nevertheless perceived to be an important aspect of the developments. such planning tends to have similar features in all three networks, despite a lack of formal debate on the form that the planning should take. in many ways the planning in these networks displays many characteristics of the organisational is planning approach described by earl (1993), where planning is a continuous decision making activity shared by the business and is. the results show that planning has a more important role than just identifying and prioritising systems projects in that it contributes to the development of inter-organisational systems by helping to establish co-operation as well as negotiating systems and other details within the confines of inter-organisational influence and objectives. in addition, planning is, in many ways, a negotiation tool used by ios participants to establish inter-organisational arrangements, in addition to delineating systems products. analysis of the findings shows that the interaction of the individual organisational planning approaches with the reality of an inter-organisational operating environment produces a series of planning environments. ios planning is seen to take place with these environments, which influence ios planning processes and outcomes. inter-organisational activity in relation to systems and organisational activity illustrate these planning environments, as discussed in the following sections. these planning environments are proposed as being the basis for developing ios planning guidelines. introducing inter-organisational systems: strategic issues while the methods used illustrate an informal approach to planning, all systems were introduced as participant organisations deemed them an important aspect of their business strategy. the inter-organisational systems are diverse in their relationships with the business strategies. individual organisations and networks use business strategy in different ways, but all use it as a tool for implementing inter-organisational systems. the necessity of making a business case for ios within individual organisations is recognised, but ios champions tend to use business strategy in an opportunistic manner rather than as a guiding set of objectives. the evidence suggests that ios planners must ensure that the business strategy is conducive to ios rather than having to drive ios. the business strategy angle is useful in two ways: first as a selling tool, both internally as well as between organisations; and second, as a tool for reaching a co-operative decision by negotiation. all organisations take account of the expectations of their business strategy, yet it was clear that a formal association between ios planning and business planning did not usually exist. a wholesaler in the pharmaceutical industry was the only firm studied that stated that its interest in ios was the result of a top-down business-driven strategy. according to the it manager 'the push for this system came from the top'. the ceo visited the us in the early 1980s and examined electronic ordering systems being implemented by american wholesalers. on return he asked the it department to investigate the technical possibilities and employed a person to market the concept to pharmacists. the company was in the process of developing their own systems when die pharmacist representatives approached them with a view to developing a co-operative system. this can be contrasted with the two other networks where the linkage was less formal. one is manager in the retail 49 ajis special edition electronic commerce november 1998 sector, who had been involved in developing a business strategy two years previously, stated 7 have not seen the [business strategy] document since, and am not really sure what the current strategy is...electronic trade has become part of the business strategy, rather than business strategy driving electronic trade'. despite the lack of a formal network strategy, those responsible for ios in individual organisations know what their organisation expected from them, and how ios could help realise these expectations. for some partners within the third network, the internet-based systems were simply an extension of services that they currently provided manually. although in some cases they were unsure of the role that ios would eventually play in their business, it was clear that it was important to be involved. according to one partner 'we have to be involved as this project directly affects our business. it would be unwise to let ourselves become a competitor rather than a partner'. a knowledge of business expectations is useful in recognising opportunities for applying technical solutions, but it people realise that agreement between senior business managers in the organisations concerned is necessary to introduce ios; this is perceived as beyond the remit of is personnel. the is manager in one of the retail sector organisations believes that 'the real decisions are made between the is people in trading organisations; senior executives are brought in to formalise the process'. a business case has to be made, but this is done after the is manager has decided what should be done, and how it should be done. therefore, the relationship between business strategy and ios can best be described as fragmented. ios is based more on the strategies of individual organisations rather than on a network strategy. it is these individual strategies that led organisations to work together within the network, and, in many cases, the network strategy emerged from the collective strategies of network participants rather than being formulated in advance. however, it is notable that, where one organisation had a more powerful position in the market than others, their strategy became the default network strategy. nevertheless, network strategies are more likely to be visions rather than formulated objectives from which more detailed, lower level plans can be developed. lower level decision making and planning appears, therefore, based very much on negotiation among participants based on their own goals. this is where the realities of business strategy are incorporated. planning fundamentals planning within the three networks studied takes place within a business framework that tries to take advantage of the emerging technology. it has both strategic and operational aspects. the strategic element provides direction within a business rationale, and the operational aspects concentrate on implementation of the technical infrastructure. yet, ios planning is more emergent than formal. this is especially true in relation to the role that the systems would play in the networks, and the contribution such systems would make to participants' businesses. technical details at an operational level received more detailed planning attention. a conscious decision was not made on a network-wide basis to engage in ios planning. the realities of interconnecting systems demanded precise technical details and operational procedures, while the emerging nature of the available technology and the uncertainty surrounding its co-operative potential resulted in short planning horizons and conservative expectations among many ios participants. the retail sector manager hi the leading value added network (van) for edi was sure that planning took place. she believed that it was generally the hub (central) company that did the planning, 'the hub sets its objectives, decides on how to move forward and then draws up a list of suppliers'. this scenario is typical of edi in the retail sector. it is necessary to have a strong proponent, both within an organisation and in the network. however, it does not have to be the hub company. one of the spoke organisations in the retail sector, because of its size, dependence on edi and commitment to developing edi-based systems, was very involved in planning. however, when the is manager in this organisation was questioned on planning, her response was 'what planning?' she did not see that planning took place. however, when questioned further, it became evident that she knew exactly what direction the inter-organisational systems would take. her policy was to decide on what to do and then convince customers to request the systems, 'a request from an important customer is strategic enough to get the board's approval'. this personal approach to ios planning was also evident in the other network studied. in each, someone provided direction for the development of systems within the network. even with a strong proponent leading ios strategy, ios planning requires lower level decision making. an examination of this reveals that ios planning decisions are taken in one of two ways. one is that the issues are debated among participants to reach a solution. alternatively, one member made the decision and sold it to the others. many of these decisions concerned details relating to technology issues or process requirements. power and influence of key players within the network are an important element of how these decisions were made, regardless of whether the decision making process was by negotiation or by decree. the hub company and the van in the retail sector made the technical decisions. decision making in the other two networks was more cooperative in nature. however, partners with technical expertise were very influential hi this process. nevertheless, the nominal involvement of all interest parties in the decision appears to have made implementation of the technical decisions easier. 50 ajis special edition electronic commerce november 1998 systems infrastructure planning begins as a learning process where organisations attempt to find out about ios technologies and what they can provide in specific circumstances. this helps create a business argument that is used to conceptualise the nature of the systems to be implemented. this process is inter-woven with issues of the nature of data to be exchanged. finally, an implementation process is planned. a dominant partner who imposes or sells these decisions to trading partners can undertake all these processes. alternatively, the processes can be more cooperative. nevertheless, knowledge of the technology or experience of its use elsewhere proves an advantage in convincing trading partners to support the ideas. learning a key part of the ios planning processes in the networks studied is learning about the technology. this learning is double faceted in that it considers technical functionality and systems capabilities. in addition to providing information on available technology, this learning can be important in achieving co-operative momentum or as a selling aspect to bring other organisations on board. 'achieving co-operation is the hardest part of the process' according to a pharmacy owner, 'however, once you get co-operation on one issue, you create a precedence which helps the next time'. this co-operation is important in getting agreement when influence is equally distributed among participants. it also helps to build trust, which is an essential element in benefiting from ios. knowledge of the success of a particular technology in similar applications elsewhere is an important starting point in creating a vision for using that technology. this vision can then be used as a selling point, both within and without the organisation, for bringing others on board. the ceo from one drug wholesaler went to the us to study electronic ordering systems. pharmacists were reading about ios elsewhere and were anxious that a situation such as the american hospital supplies, where wholesalers installed the systems, would not happen to them. the owner of a chain of pharmacies acknowledged that 'an agreement is more likely to be accepted if someone can support their argument with evidence from elsewhere'. representatives from the retail sector on the anai (article numbering association of ireland) sub committee on edi studied the influence of edi technology elsewhere, while the spread of internet-based electronic commerce initiatives was central to creating the vision in the third network. consequently, the business justification of adopting such technology is based on creating a systems-based argument that illustrates how it can help meet business objectives. technology and systems planning the inter-organisational systems infrastructure is planned around ios technology. the choice of technology is a natural output from the learning process discussed above. in one case, an open system was developed because pharmacists did not want to be tied into one standard. they considered tradacomms but rejected it, according to one it manager, as '// was a sledge hammer to crack a nut'. inter-organisational systems in the retail sector are edi-based because of the commitment of it personnel in this sector to introducing the technology. internet-based systems are used in the third network because it is central to the electronic commerce aspirations of its founders. participants perceive that detailed nature of ios technology planning is a necessity because of the decision making required in relation to standards and protocols used to operate systems between organisations. these decisions are an important part of the planning process because they can limit the size and nature of the interorganisational network. however, the systems infrastructure is specifically planned around what the technology can provide to meet the business requirements. these business objectives then get translated into systems concepts. planning the systems infrastructure has both internal and external considerations. technical planning at the network level focuses on major approaches and technologies. for example, the decision to adopt edi was made by representatives of the grocery industry. such decision making can either be the result of a co-operative process or the acceptance by trading partners of the decision of one partner. following these decisions, an internal planning process within individual companies must deliver their individual responses. the contrast between internal and network objectives can be seen from the efforts by a drugs wholesaler to develop a standard that would meet their internal requirements and still be acceptable as a network solution. according to the it manager, 'such planning would be much easier if you could control both ends. things are more difficult when it is necessary to co-ordinate independent parties'. overall, technology and systems planning on an inter-organisational basis appears to be a case of marrying the objectives and the capabilities of participants. this is perhaps more co-operative than some other activities, but knowledge and influence are still leveraged as part of the decision process. technical constraints are a limiting factor which need to be examined as part of the planning process in order to facilitate inter-organisational interaction. often these technical constraints focus on the transmission of data between organisations. systems planning must necessarily consider data to be transmitted. arguments for the development of ios often 51 ajis special edition electronic commerce november 1998 come from using the data to meet a business need. for example, organisations that choose to implement an invoicing system will have already considered the broad outline of the data it will contain. structured data systems such as those in the retail sector require detailed consideration of data sources, fields, presentation forms etc. the issue of availability of data is central to ios planning by the hub company in the retail sector. according to the edi manager, 'the ability to send data in the correct form at the right time is crucial to the success of the system. much of the planning for new installations is, therefore, concerned with getting this data in an appropriate form'. however, this form is not decided in conjunction with the supplier. edi standards adopted by the hub dictate what must occur. the planning of systems implementation is a natural follow-up to planning and designing the systems and data infrastructures. issues considered during this process relate to timing, sequence, technical capabilities etc. implementation planning in a situation where one organisation controls the process is less complicated than where it is necessary to get co-operative agreement. in a co-operative situation, participants are more likely to try to get the implementation plan to suit their own goals. ios and organisational activity the concern by technical people for ios technology and the involvement of senior managers in the strategic elements of ios cover two main elements in introducing ios. inter-organisational systems also affect the way in which individual organisations, as well as organisational networks, conduct their activities and cover issues of process, procedural and structural changes. those introducing ios, despite claims that they constitute an important element of ios benefits, rarely consider such changes. the degree to which such changes are considered appear to be related more to the objectives of individual organisations rather than to a network strategy. however, it is clear in the short term, that certain network cultures are more likely to produce activity changes than others. in terms of affecting organisational processes, ios starts as a technical issue that tends to spread into other facets of the organisation. a pattern starts to emerge whereby the it people champion the systems and technology, and convince senior management and functional areas to adopt it. when the systems become stabilised, they become the responsibility of the functional area. process changes occur at this level rather than as a result of top-down planning. however, some procedures may be developed as part of the process. the situation that led to the development of the pharmaceutical systems was different. inter-organisational systems were seen as central to process improvement. wholesalers wanted electronic ordering to reduce the overhead associated with teleordering, while pharmacists were 'keen to reduce the time spent on administrative duties such as ordering'. however, such process issues were internal considerations, and did not form a large part of the interorganisational planning process. nevertheless, these issues were key to the system's development. procedures for using inter-organisational systems can be manual or computerised. they are, therefore, a necessary element of ios planning. these procedures can be categorised as those that operate across the network and those that apply to a particular organisation because of their adoption of the ios. procedures that relate to network activities require the involvement of other parties, or at least a consideration of their needs. a dominant partner such as a hub organisation or systems expert can heavily influence such procedures. in the third network, procedures were prototyped by the technical partner and commented upon by others. according to one partner,, 'generally, procedures relating to the technology were accepted, but adjustments were made to data related procedures where other partners had ideas on how certain things should be done. these were generally from within that partner's area of expertise or directly affected their activities'. it is clear that the effects of inter-organisational systems on structure are not a consideration for those implementing ios. the potential of such systems to affect organisational structure is realised, but it is most certainly not planned for. the benefits of ios can be felt in many departments, but structural alterations are not to the forefront of the minds of those planning them. however, according to one partner to the third network, 'as a virtual organisation, a completely new organisational structure was being created using ios'. however, while at a high level this structure was planned, roles and their consequential structural issues emerged as opposed to being formulated, 'the structures being created were dependent on the organisations involved. these structures were centred around areas of expertise, and evolved more than were planned'. in fact, when the planning process started to consider explicit roles and tasks within the context of delivering a commercial product, it based much of its decision making on the roles that participants already fulfilled. ios and personnel this section investigates planning aspects of the effects of ios on staff. staff can feel such effects to two ways. first, there is the issue of skills required to implement the systems, and second, there are changes to roles, responsibilities and activities of personnel whose function is affected. the study finds that both these aspects of electronic business are considered secondary to the technology issue, and were approached to a bottom-up manner. it skills received more attention than organisational skills as they directly affect the daily operations of 52 ajis special edition electronic commerce november 1998 the it department. this is often seen as a once-off training issue, whereas the organisational skills issue is more on-going and developing. the effects on it staff were considered in the it department in the hub company in the retail sector. according to the edi manager 'the selling of the edi concept by vendors necessitated a consideration of haw edi would affect the it department'. systems development was undertaken internally with the advice of the edi van. existing personnel who also deal with internal systems undertook this development. the technology is not seen as radically different and external interests are considered where necessary, rather than involving them in the planning process. considerations of changing organisational roles are limited in the context of ios planning. this is especially true when the ios is instigated by a dominant partner, and where minimal changes are made. more co-operative inter-organisational situations are more likely to change organisational structures as the negotiation process involved in their establishment is more complex and is likely to examine structural change. changes in organisational roles have occurred for sales people within one firm in the retail sector. fewer sales people are required than before. therefore, some consideration of the effects on the sales department had to be undertaken. this was done by the is manager in conjunction with the sales director. the nature of the sales activity changes when customers commit to implementing edi in that it not only becomes more stable, but also places different challenges on the sales people. this requires some retraining, but more important is the role of the surplus sales people. key here is that issues of changes in organisational roles are worked out between the heads of the functional areas concerned. this was not seen as central to the ios planning process, however. discussion and conclusions the above analysis shows that ios exist in many different contexts, even within the same network. such diversity is evident in the ios planning approaches adopted by the participants. although it can be difficult to identify an activity that can be clearly labelled ios planning, planning was integral to the development of all networks. for the pooled and reciprocal networks, the planning process gave a focus to the activities and helped the projects gain momentum. for the sequential network, the main planning was done by the hub organisation, and was extended to the spokes in order to bring them on board. this stand-alone approach to planning was possible because of the power of the hub organisation. this power helped negate the need to gain co-operation by engaging in joint planning. instead of having to use planning as a process for gaining consensus, it could focus on transactions! details that needed to be applied in order to implement the inter-organisational system. however, even within these networks, it is clear that influence and co-operation played an important role in planning activities. business and is planning processes are important for setting objectives and formulating plans to meet the targets. these activities are based on organisational requirements for coordinated activity. within the ios domain there is also a need for setting objectives and targets. however, ios planning has a far more important role than just giving direction. this role is in achieving co-operation, and is related to the setting in which the interorganisational system is being developed. this co-operative role is most evident in the networks characterised as pooled and reciprocal interdependency. however, certain inter-organisational activities across all networks are based on co-operation, while others are based on the power and influence of an individual or organisation. the result is a complex inter-play of planning environments that not only influence the planning process, but also the resulting system. in an attempt to understand ios planning environments, it may be useful to describe them as lying along a continuum, characterised as ranging from monarchist to club. the characteristics of the extremes are shown in table 3. basis |.l •;::£*;;;«;. • " • . i ' ' \t? '•' t . ; , ' • • ' itype . i focus ; ! process ... : monarchist leverage power ^directive : _j transactional '•'-".< formulatied ri.. }:~&£ club:'. :'-•:•• ;co-operative momentum ; participate : v^leaming . ,0; c7emer«ent"': /••-"•ir;/-. • • • -:! i i j 1 table 3: monarchist and club ios planning environments the monarchist environment is based on a strong organisation that holds some power over others in the network. for example, a large retailer has power over a series of small suppliers. planning in this case is traditional in that 53 ajis special edition electronic commerce november 1998 the dominant organisation can set the objectives and design a process where others in the network must meet objectives. planning is directive and focuses on transactional issues such as technical standards, budgets, and timescales. a traditionally formulated ios planning process is often appropriate in this case, as the issues are more clear-cut. at the other end of the scale, the club environment is based on the premise that no single organisation dominates. the planning process is more participative as those involved need to accommodate each other's needs and expectations. this is more a learning environment where strategies evolve. transactional issues become relevant here in two ways. first, as part of the learning process where a co-operative network strategy emerges, and second, as the implementation issues after co-operation has been achieved. the planning process here is often emergent rather than formulated, although a more formulated process may be used for the transactional elements. it is clear that many different planning environments operated within the networks studied. however, a broad classification would suggest that network two (retail sector) was mostly monarchist and network three (virtual organisation) was mostly club, with network one (pharmaceutical) operating in between these two. however, even within these classifications others are evident. for example, technical issues with network three were mostly approached in a monarchist manner. the type of planning required, therefore, falls into two categories. one is a co-operative learning activity, the other a discrete, formulated process that focuses on tasks, budgets and technical standards. low level issues such as standards and budgets are required in all networks, while the higher level learning activity is required where there is a need for a co-operative approach to ios implementation. in arguing that ios planning processes should be based on inter-organisational relations rather than an extension of internal systems planning, the basis for developing ios planning guidelines starts to emerge. however, it is clear that these guidelines must be based around the realities of inter-organisational relations and systems requirements. the importance of these guidelines is emphasised by the 'soft' nature of the existing organisational approach to planning, which is recognised by earl (1993) as a weakness of such approaches. planning frameworks are required at both the level of the individual organisation and at the network level. this is necessary in order to provide participants with a mechanism to consider both internal organisational issues and issues that affect the whole network. the development of these frameworks needs to result from an analysis of the role played by each participant. once planning frameworks have been established, it is necessary to determine which roles each participant organisation need to play, and who within the organisation is most suited for the roles. this is not necessarily an issue of psychology, but rather the ios planning process must provide an opportunity for such issues to be considered. ios need to be the result of an objective that contributes to business strategy or an organisational structure/activity need. consequently, planning efforts must consider these issues in a thorough and complete manner. this is especially important as inter-organisational systems become more widespread and affect more of the value chain. finally, ios planners must deal with systems, data and technology issues. while ios planners may borrow from internal approaches in this regard, it is necessary to realise that even these issues are complicated by their existence within an inter-organisational domain. references benbasat, i, goldstein, d. k. mead, m. (1987) "the case study research strategy in studies of information systems", mis quarterly, 11 (3) 369-386. bergeron, f. and raymond, l. (1992) "the advantages of electronic data interchange", data base 23 (4) 19-30. borman, m. (1994) "common knowledge, interorganisational networks and the future for the organisation of production", journal of information technology, 9 203-212. byles, t (1993) "electronic commerce: the new foundation for trade", edi forum: the journal of electronic data interchange, 6 (3) 12-17. cash, j.i. and konsynski, b.r. (1985) "is redraws competitive boundaries", harvard business review, (2) 134-142. clarke r. and jenkins m. (1993). "the strategic intent of on-line trading systems: a case study in national livestock marketing", journal of strategic information systems 2 (1) 57-76. cox, b. and ghoneim, s. (1994) "benefits and barriers to adopting edi in the uk: a sector survey of british industries", in baets, w.r.j. (ed.) proceedings of the second european conference on information systems, nijenrode university, the netherlands, may 30-31. cunningham, c. and tynan, c. (1993) "electronic trading, inter-organisational systems and the nature of buyerseller relationships: the need for a network perspective", international journal of information management, 13 3-28. earl, m.j. (1993) "experiences in strategic information systems planning", mis quarterly, 17 (1) 1-24. eisenberg, e.m., farace, r.v., monge, p.r., bettinghaus, e.p., kurchner-hawkins, r., miller, k.i. and rothman, l. (1985) "communication linkages in interorganisational systems: review and synthesis" in dervin, b. and voigt, m.j. (eds) progress in communication sciences vol. 6, ablex publishing corporation, norwood, n.j. 54 ajis special edition electronic commerce november 1998 finnegan, p. and golden, w. (1996) "key success factors for late adopters of inter-organisational systems: a non-technical perspectives from irish organisations", proceeding of the 29th hawaii international conference on system sciences, 377-386. finnegan, p. galliers, r. d. and powell, p. (1998) inter-organisational systems planning: learning from current practices, international journal of technology management, forthcoming. finnegan, p., golden, w. and murphy, d. (1998b) "implementing electronic data interchange: a non-technical perspective", international journal of electronic commerce, 2 (4). gable, g. (1994) "the case study in is research", proceedings of the searcc conference on it, kuala lumpar, malysia, august 11-14. galliers, r.d. (1991) "choosing appropriate information systems research approaches: a revised taxonomy", in nissen, h.e., klein, h.k. and hirschheim, r. (eds.) in information systems research: contemporary approaches and emergent traditions, proceeding of the ifip tc8/wg 8.2 working conference, copenhagen, denmark. henderson, j.c. (1990) "plugging into strategic partnerships: the critical is connection", sloan management review, spring, 7-18. holland c.p., lockett g., richard j.m. and blackman i. (1994). "the evolution of a global cash management system", sloan management review (fall), 37-47. holland, c.p. and lockett, a.g. (1990) "an implementation model for electronic data interchange", proceedings of the international conference on information management, budapest, 675-685. kaufman, f. (1966) "data systems that cross company boundaries", harvard business review, jan-feb, 141. kumar, k. and van dissel, h.g. (1996) "sustainable collaboration: managing conflict and co-operation in interorganisational systems", mis quarterly, 20 (3) 279-300. kuula, j. (1995) inter-organisational information systems as a media for supporting the internationalisation of business, ph.d. thesis, university of jyvaskyla, finland. opper, s. (1994) "choosing and implementing a groupware system", datapro. pettigrew, a.m. (1992) the character and significance of strategy process research. strategic management journal 13 5-16. provan, k.g. (1982) "interorganisational linkages and influence over decision making", academy of management journal, 25 (2) 443-451. roby, d. and sales, c.a. (1994) designing organisations, irwin, homewood il. scala, s. and mcgrath, r. (1993) "advantages and disadvantages of electronic data interchange: an industry perspective", information and management, 25 85-91. swatman, p.m.c. and swatman, p.a. (1992) "edi system integration: a definition and literature survey", the information society, 8, 169-205. thompson, j. (1967) organisations in action, mcgraw-hill, new york, ny. van de ven, a.h. (1976) "on the nature, formation and maintenance of relations among organisations", academy of management review, october, 24-36. webster, j., (1995) networks of collaboration or conflict? electronic data interchange and power in the supply chain. journal of strategic information systems, 4 (1) 31-42. west, l.j. (1994) "breaking down the barriers to edi implementation", tma journal 14 (1) 10-15. yin, r.k. (1989) case study research: design and method, sage publications, ca. 55 microsoft word ajis vol 16-1.doc australasian journal of information systems volume 16 number 1 2009 165 ict governance and what to do about the toothless tiger(s): professional organizations and codes of ethics don gotterbarn, professor emeritus east tennessee state university abstract ict has had numerous unfortunate incidents in planning, development, and delivery. a typical response to these incidents is to both complain about the toothless tiger of unenforced and unenforceable technical and professional standards and to also advocate the development and implementation of strong government regulations – licensing and legislation. these regulations constitute one form of what has been called “ict governance”. unfortunately, there are significant limitations to both the constraining regulations approach and the vague toothless tiger approach to ict governance. there is an approach to ict governance which takes advantage of and strengthens some roles of professional organisations and avoids the dangers of giving government malformed regulatory teeth which limit the positive potentials of ict and introduce overt harms. the purpose of this paper is to define strategies for professional organisations to meet their responsibilities to the ict profession and ict professional; a strategy which moves toward regulation without curtailing ict potential with ineffective sanctions. professional organisations need a strategy for reducing negative incidents and improving professional responsibility without simply introducing sanctions on a narrow range of practitioners who happen to be members of that organisation. there are ways in which the toothless tiger(s) can have a significant positive influence. introduction information and communication technology (ict) has been with us for many years and in the past ten years there has been a growing interest in something called “ict governance” as a means of reducing information system disasters. there have been national organizations formed, professional organizations have organized sub-committees to address ict governance and it has even been called a “discipline”. the unwrapping of this concept, like many concepts, has several inconsistent interpretations and ineffective implementations in industry. in some cases the concept has been modified to meet a particular sectors needs. this broadening of concepts to fit individual needs is not new and sometimes is quite useful. there is however a fundamental mistake in the narrowness of most interpretations of ict governance which make it less likely that it will achieve its ultimate goals. i believe this mistake can and should be addressed by professional computing organizations. australasian journal of information systems volume 16 number 1 2009 166 in what follows i will examine the various approaches to ict governance, the difficulty it tries to address and i will argue for what i consider its critical limitations. i will then show how professional organizations can address the weakness of ict governance using tools they already have at hand. the problem from the software perspective in the early days of computing 1960’s people worried about “the software crisis”; worried about the failure of software systems. software workers addressed these problems by developing models for building well engineered software. the focus of the computing community was primarily internal; focusing on how to develop and test a program. this was the period of mathematical modeling of software development in books on software science (halstead, 1977) and the discipline of programming (dijkstra 1976). the focus was on making computing into a reliable engineering-like discipline. the impacts and concerns addressed were local to the particular system being developed. in 1968 the term “software crisis” was coined to describe problems with developing correct and reliable software programs. the responses to these software problems generated and continues to generate many single mode solutions; do this one thing in software development and the problems will be solved. these approaches included an emulation of an engineering approach to development of software, then a structured approach to program design, then a formal proof of software system requirements, then object-oriented software development, a focus on individual programmers counting the number of errors they make (personal software process) and now an agile or extreme programming approach to software development. these single mode methods were regularly interspersed with approaches which emphasized measuring the software’s size, reliability, and space and time efficiency. the indication of the lack of success of these approaches is the rise of ict governance to address the negative impacts of the software crisis on industry. i believe there were three major difficulties with these approaches by the software community. first the software crises is a complex problem and single mode approaches over emphasize one piece of the problem and tend to ignore other pieces of the problem. second, computing technology and applications are constantly changing, and changing at a very rapid rate. this means the domain of the software crisis is also constantly changing; advances like computerized robotic surgery was not an issue in the 1970’s. e-commerce had no meaning 15 years ago. the software crisis is still a problem with software development but the software being developed has expanded applications and the relevant stakeholder communities have correspondingly increased. the third problem is that the solution to a problem with software interaction with business and society was addressed only from the software side of the problem. these single mode approaches were focused on what software developers do. since 1968 the answer has been the same in different clothesand the mistake is the same. if i am an honest person and work hard following a good process then the problems will go away. unfortunately this has consistently not worked. there are still significant systems failures which lead to major corporate failures. the effect of programmed trading on the stock market in october 1987 took years to recover from. there are numerous standards organizations such as the american national standards organization (ansi), the ieee, the international standards organization (iso), and standards australia which have worked on developing rules and standards to control and monitor software developments. australasian journal of information systems volume 16 number 1 2009 167 the diverse and developing nature of ict makes the application of these rules to software development difficult. • since no common education is required many ict workers do not know the standards/rules • since there are few sanctions, oversight, or threat to losing an ict license obeying the rules is voluntary and it needs to be made clear what is gained by following them/ • since the environment is changing so rapidly rules either no longer apply by the time they are finally written or they are inconsistent because they address different sides of an issue. who enforces fraud legislation on international internet sales? the focus to solving the software crisis was internal on the technology and on how to be a good technician; the software developers were going to fix it. another manifestation of these single mode internally focused approaches was a method of software development called ‘over the wall development’. at its worst such development had the following scenario. the skilled software technician would gather requirements for a business software project and then develop a complete system without any further consultation with the client. the system developed (as the software person misunderstood it) would be presented to the customer (thrown over the wall). the assumption being that the technician had some how understood all of the important business issues and addressed them in the software. this systemic disconnection of developer and customer led to working systems being delivered which had nothing to do with the customers needs. since corporate executives did not fully understand how it worked, when a corporate manager complained about the style or function of the delivered system itc personnel could simply declare that what they delivered was the only way the computer could do the requested task. this problem is not unique to software development but occurs where there is discrete development elements and no communications between those working on different parts of a project. there is a disconnection between the world of the computer astute developer and the customer who is skilled in their own domain. the two parties speak different languages. the programmer who understands the syntax of a programming language is overwhelmed when requested to write a system to model genetic mutations which calculates the sequential effects of genetic drift, inbreeding, selection, gene flow, and mutation upon the proportion of a population’s gene pool comprised of the second of two alleles. to think that the difficulty of developing this project could be resolved by focusing on software technology is a mistake. this programmer optimism is repeated i numerous projects. history repeats itself in the development of ict governance, i think history is repeating itself. i think a similar set of mistakes is being made in ict governance’s attempt to address issues with ict systems. many areas in ict governance are taking a single mode approach which focuses primarily on its side of the problem, on the business element, minimizing the ict elements and it does not take into consideration the rapid change of the domains of ict and business. ict governance in general ict governance emerged as an attempt of business to deal with the impacts of major software system failures on business. although it was primarily a part of corporate governance which focused on information technology, many ict professional organizations contributed to the process of developing standards. these organizations advocated that their members adhere to these standards. australasian journal of information systems volume 16 number 1 2009 168 in 2003 australia published several other standards for corporate governance including: good governance principles (as8000) ,fraud and corruption control (as8001), organizational codes of conduct (as8002), corporate social responsibility (as8003), whistle blower protection programs (as8004) so the pattern was set for a document from the corporate side addressing ict governance. standards were developed such as as8015 – (2005). this australian standard for corporate governance of ict to deal with in-house development issues and the fact that outsourcing to vendor’s interests was not always consistent with those of a user organization. this is similar to one of the problems of over the wall software development. the interest in itc governance is international and quality development from the corporate side is defined in the information technology infrastructure library http://www.itil.co.uk/; a detailed framework with hands-on information on how to achieve a successful governance of it. as in the attempt to deal with the software crisis described above, there is an iso standard to deal with corporate governance. “because inadequate information technology (it) systems can hinder the performance and competitiveness of organizations or expose them to the risk of not complying with legislation, the new iso/iec 38500 standard provides broad guidance on the role of top management in relation to the corporate governance of it.” http://www.iso.org/iso/pressrelease.htm?refid=ref1135 . the material on ict governance is not consistent in describing the level where ict governance comes into play. one should not be misled to think that there is a single standard of ict governance. there are compliance standards like sarbanes –oxley (section 404) that focus on configuration management. on the other hand, weil (2005) in a study of 300 enterprises around the world “did not identify a single best formula for governing ict.” they say that “ict governance specifies accountabilities for ictrelated business governance and helps companies align their ict investments with their business priorities.” ict governance performance for them involves the corporation’s “cost effective use of ict, ict for growth, ict for asset utilization and, ict for business flexibility.” they argue that “ict governance is the decision rights and accountability framework for encouraging desirable behavior in the use of ict.” this is very different from viewing ict governance as managing the interface between ict development and corporate management. sense of ‘control’ but if we follow exactly the australian definition of ict governance; it uses evaluation and control but it takes many forms; both in what is controlled – ict systems development and the current and future corporate use of ict. this same diversity exists in the standards being developed by the international standards organization. there is also diversity in what is meant by “control”. control is characterized both as a rigorous highly delineated control and a general directional guidance and principles to help the professional make judgments. single mode as in software development, it looks like ict governance takes a single mode solution approach and that there are a variety of single modes. there are ‘structuralists” who think everything is handled by structure and that the primary goals for information technology governance are to (1) assure that ict generate business “value” and reduce the risks that are associated with ict by implementing a carefully defined organizational structure. this is sometimes connected with who is in charge or has ownership of the system; an over the wall problem. australasian journal of information systems volume 16 number 1 2009 169 over the wallit is mine! the definition of ict governance is tied to corporate governance and relates the business focus of an organization and ict management. it mandates that ict decisions are owned by the corporate board rather than by ict managers. this has the same problem as over the wall development. there are indeed limitations to what ict can do and the ways it does things will have different effects. balanced ict governance needs the ict side in their systems guided by icters. limited view of stakeholders several of the problems associated with software development are recurring in ict governance. there is an additional problem which is common to the each of the issues of software development. there is a limited view of who constitutes the stakeholders in a project. the limited view of project stakeholders is constantthe developer and the customer. the effect of an ict system on a business or on extended stakeholders is not considered. the current concept of ict governance is modeled on the traditional business ethics concepts of who needs to be considered as the relevant stakeholders, namely those who have some financial interest in the business ( agle 1999) the current concept of ict governance stakeholders is “ict governance implies a system in which all stakeholders, including the board, internal customers, and in particular departments such as finance, have the necessary input into the decision making process.” this limited view of the system context and the stakeholders in that context is also supported by iso standard which characterizes ict governance as the management system used by directors. “ict governance is about the stewardship of ict resources on behalf of the stakeholders who expect a return from their investment.” we shall return to how this problem can be addressed by professional organizations. in a recent survey of ethical issues conducted by cappe (lucas and weckert, 2008) the limited understanding of stakeholders is clearly indicated when most of them say they do not consider their work to be related to larger segments of society. the concept of ict governance is closely tied to some concept of “control.” the use of words like ‘control’ and ‘govern’ imply some type of enforcement or sanctions for failure to follow the rules, but in ict professional societies these rules seem like toothless tigers. there are no real sanctions for failing to follow the rules. professional societies can have a significant roar without any associated bite. we see the attitudes of many computer practitioners toward attempts to promulgate rules of behavior clearly in lucas (2008). a member of the australian computer society (acs) complained about the rules because they wanted them to apply to everyone practicing ict. “the acs can discipline its members for breaches of its ethical code but that is no barrier to employment and it has no effect on the vast majority of workers in the industry who are not acs members.” (lucas and weckert, 2008) this failure of universal application of principles and regulations is a common complaint of honest hard working computer practitioners. it should be noted that these are not complaints about the importance of following such rules. the complaints have other motivations. there is the problem of fairness that they are held accountable as members of the acs. “why should i follow the rules if those outside the acs are not bound by them? another motivation for the complaint is that the practitioners realize that following these standards will reduce some of the harms caused by software and improve the lot of humanity and as such everyone ought to follow the rules. the same need to ‘enforce’ compliance is held in ict governance. the primary motivator for following the australasian journal of information systems volume 16 number 1 2009 170 rules is the addition of teeth – sanctionsfor not following the rules. as we have seen this is done with some compliance standards like sabanes-oxley and iso standards. the limitation of the ict governance approach it seems to me that ict governance is repeating some of the same moves made by software developers in their attempts to address the software crisis and i believe ict governance is heading for a similar set of problems as those faced by software developers. the development of systems software has to develop an interface between the technology of computing and the nature of the enterprise. the nature of this interface must be guided by a consideration of the impact of the system on a broad range of stakeholders. software developers focus on the nature of the software and how to reduce errors in the programs. they tend to focus on a limited set of stakeholders in the system: developer, customer, sponsor, and vendor and tend to pay limited attention to those who will be impacted by the deployment of the software. ict governance, likewise, seems to have an internal focus on the business side of the software system and a narrow a view of the stakeholders as those with a financial interest in the system (agle 1999, weil 2005). we can see some of the consequences of this narrow approach by looking at the information technology treatment of ‘software risk” the narrow stakeholder focus in software risk although the need for high quality software is obvious to all and despite efforts to achieve such quality, information systems are frequently plagued by problems (ravichandran 2000). a narrow approach to risk analysis and understanding the scope of a software project and information systems has contributed to significant software failures. informaticians have been evolving and refining techniques to mediate risks of developing software products that meet the needs of their clients. the risks focused on include: missed schedule, over budget, and failing to meet the system’s specified requirements. in spite of this attention to risks, a high percentage of software systems are being delivered late, over budget, and not meeting all requirements, leading to software development being characterized as a “software crisis” and a mistrust of software systems. risk management generally consists of an iterative series of steps like the ones shown in figure 1. the context referred to in the top box—the context in which the project is being developed— includes the organizational structure, and its competitive and political position as well as its risk management structure. the risk identification process identifies potential negative impact on the project and its stakeholders. as/nzs lists potential negative areas of impact such as “asset and resource base of the organization, revenue and entitlements, costs, performance, timing and schedule of activities, and organisational behaviour” [as/nzs, 1999, p. 39] the risk analysis process divides the identified risks by their severity and the likelihood that they will occur, producing a given level of risk. this level of risk is generally determined using statistical analysis or calculations with fault trees and event trees. a typical calculation is ‘risk exposure’, a metric derived by multiplying the anticipated costs by the probability of the event occurring. two forms of exposure are commonly calculated. the first method using quantitative risk analysis provides quantitatively expressed assessment of the negative consequences of an event as the outcome of an event e.g., “a delay of one day will cost $3000 in sales.” the second, qualitative risk analysis is often used to address risks which are not readily quantifiable, other than by australasian journal of information systems volume 16 number 1 2009 171 describing the degree of risk e.g. “the delay will upset our distributors causing significant loss of goodwill”. generally “qualitative analysis is often used first to obtain a general indication of the level of risk… or where the level of risk does not justify the time and effort for a quantitative analysis…” [as/nzs, 1999, p. 14]. its role primarily is to characterize and identify the impact of a risk generally asserted in terms of dollars. surprisingly, in standard risk methodologies the qualitative risk approach typically looks at quantifiable data which can be easily prioritized and facilitates analysis. these standard methods of risk identification and risk characterization have been used extensively in software projects but are incomplete. figure 1. risk management [as/nzs, 1999 p.16] the association for information systems defines “system quality” in terms of currency, response time, turnaround time, data accuracy, reliability, completeness, system flexibility and ease of use [ais 2005]. even after using these generic models of risk analysis, information systems have been produced which have significant negative social and ethical impacts. the risks of these impacts are not traditionally included in the tripartite concept of software failure-over budget, late, or not meeting stated functions. the extended stakeholders in developed software are all those who are affected by it even though they are not directly related to the use or financing of a system. the political candidate who is not elected because of a difficult voting machine interface is a stakeholder in the development of that voting machine the person who suffers identity theft because of a flaw in the security for an information system is a stakeholder in that information system. the developer’s obligations to these stakeholders are not included in the generic concept of software failure. these systems may have been a success in terms of being developed on schedule, within budget, and delivered on schedule but were a failure because they failed to take into account the conditions in which they were used. the user interface which met specifications had a significant impact on the australasian journal of information systems volume 16 number 1 2009 172 lives of others. the system used to record dosages of pediatric medicine correctly handles negative interactions of dosages but was awkward to use in emergency situations resulting in 3 medication errors out of every 100. (pediatrics 2006) contributing factors two interrelated factors related to system stakeholders contribute to missing these professional and ethical failures. first, limiting the consideration of system stakeholders to just the customer/client, software developer and those who have a financial stake in the system ignores the needs of other relevant stakeholders. some have realized that the focus on technical risks is too narrow; but unfortunately the risk focus only expands to other internal issues related to the development of the system. for example ravichandran (2000) says “research in software quality has focused largely on the technical aspects of quality improvement, while limited attention has been paid to the organizational and sociobehavioral aspects of quality management.” this is similar to the position maintained in the australasian risk analysis model above. a second factor is limiting the scope of software risk analysis just to technical and cost issues. a complete software development process requires 1) the identification of all relevant stakeholders and 2) enlarging risk analysis to include social, political, and ethical issues. a complete risk analysis requires a process to help identify the relevant stakeholders and broaden the scope of risks anticipated. to meet the goal of quality software, developers focus on particular risks including: project and schedule slips, cost increases, technical and quality risks, the timeliness of the product, risks that the final product will not fit the business for which it was designed. nevertheless, developers use risk exposure to help them focus on the most critical risks. the use of easy to read fonts or an easy to use back up system may be ignored in an effort to get a product out in time or at lower cost. the risks that are addressed are those with the highest risk exposure. all consequences are given dollar values. even qualitative risks are turned into a numerical hierarchy. the resulting risk of the september 11th disaster was calculated in terms of the number of deaths that occurred on that day or lifetime dollar earnings potential of those who died. the negative effects that need to be addressed in risk analysis include both overt harm and the denial or reduction of goods. an automated surgical system which randomly moved inches instead of centimetres hurting patients would have a negative effect; as a pay phone system which disabled all usage, including 911, without an approved credit card would also have a negative effect preventing the report of an accident. these stakeholders, patient and someone hurt in a fire, are not normally considered. the scope of a project needs to be identified in terms of its real stakeholders. this extended domain of stakeholders includes: users of the system, families of the users, social institutions which may be radically altered by the introduction of the software, the natural environment, social communities, informatics professionals, employees of the development organization and the development organization itself. the design of many of the usa’s voting machines correctly counted votes but made it difficult for people to enter their votes. modifying the approach to stakeholders. in the software developers response to the ict crises the focus was internal on the technology and how to be a good technician. analogously ict governanceis going through a similar, though much shorter, life cycle. as initially developed, ict governance focuses on the governance within an australasian journal of information systems volume 16 number 1 2009 173 organization; “evaluating and directing the plans for the use of ict to support the organization and monitoring this use to achieve plans. it includes the strategy and policies for using ict within an organization." (as8015, 2005). the focus is clearly internal with statements like “to better understand their obligations and work more effectively to maximise the return and minimise the risks to the organisation from ict” the internal focus suffers from the same problems as ‘over the wall’ software development. the role of professional organizations professional organizations have been involved in the development of these ict governance standards. for example the acs was involved in the development of as8015 and its narrow approach. both ict and the business sector have contributed what they view as solutions from their own sectors – object-oriented software design and financial flexibility of corporate systems—which model single mode solutions. they have difficulty enforcing these solutions and worry about system ownership. they are each concerned that their side of the wall has control of the system, and of course they also differ as to which stakeholders primarily need to be considered when developing these systems. the professional organizations have the ability to resolve this. one of the roles a professional organization needs to play in resolving these problems is to broaden the range of stakeholders considered in the current concept of ict governance. ict professional organizations have codes of ethics and codes of practice which actually address each of the ict governance problems discussed above. their codes of ethics require the it professional to consider a broader range of stakeholders, to consider a range of stakeholder which includes all those whose lives are affected by the it project and the way it was implemented. the code of ethics is very useful as a model of governance but is ignored because these codes are viewed as less enforceable than other types of regulations. they are considered worthless because they are like toothless tigers. professional computing codes of ethics can address the ict governance problem to see how codes can be useful we need to look at some of the general functions of codes and how they are managed in professional organizations. functions of codes codes of ethics are developed by professional organizations for a variety of reasons. codes of ethics have served a variety of functions and are frequently directed at several audiences. at the simplest level, a code of ethics is a statement of the obligations of individual computing professionals in the conduct of their profession. the code will generally embody a moral commitment of service to the public. sometimes they are used to clarify expectations and appropriate behavior of professionals. more positive functions of codes of ethics include: making a statement to members or aspiring members of the profession about shared commitments and agreed upon rules, sensitizing members to new issues, and provide guidance to individual members when they are confronted with ethical situations. australasian journal of information systems volume 16 number 1 2009 174 sometimes codes are used to win public confidence and stave off external regulation. some codes or parts thereof are disciplinary in order to convey a sense of self-regulation by the profession. they also help set expectations for employers and clients about dealing with members of profession and socialize novices in field. codes are sometimes used to express and strengthen the community orientation of the group. some codes are very general and are therefore hard for the individual practitioner to apply in concrete cases. some confusion about codes of ethics arises from a failure to distinguish between closely related concepts about codes which direct the behavior of practicing professions. moving from less restrictive to more restrictive codes, we first mention codes of ethics which are primarily aspirational giving a mission statement for the profession. there are also codes of conduct which describe professional attitudes and some professional behavior. codes of practice are very specific and tied closely to the practice of the profession. they are the easiest to use as a basis for legal action. because practicing professionals deal with human affairs, the underlying ethical principles are the same across professions. studies have shown that most codes are a hybrid of these three types of code (berleur, 1994) several authors have discussed the various goals of a code of ethics (martin). these goals of codes of ethics could be ranked from the most benefit to society to most benefit to the individual member of the profession. these goals include: 1. inspiration and guidance for ethical conduct. 2. support those seeking to act ethically by appealing to the public system of ethics established in a code. 3. education and shared understanding (by the professional and the public) of standards of practice. 4. deterrence and discipline for specific actions by sanctioning code violations. 5. protect the status quo by stifling dissent and state minimal level of ethical conduct. 6. promote business interests by forbidding competitive bidding. one way to evaluate codes of ethics is to examine which of these functions assumes prominence in the code. codes also indirectly educate the public at large about what professionals consider to be a minimally acceptable ethical practice in that field, even as practiced by nonprofessionals. the swiss computer code suggests that the responsibility of a national and/or professional society is to be in charge of making the public aware of the society’s guidelines it also advocates regularly publishing information about code violations as a means of sensitizing the public about what is expected of a computing professional. as we shall see later some codes also include responsibilities of the ict profession itself as well as about the responsibilities of the individual practitioner. two common problems for codes in computing are that they need to be able to address a rapidly changing environment and there are difficulties in enforcing them. turning a code into law makes it static and eliminates some of the other important functions of codes of ethics. recent codes have become more specific about the ict workers responsibility to society and responsibility to a broad range of stakeholders (gotterbarn 1996). the canadian information processing society (cips) code of ethics and standards of conduct addresses a diverse audience. the cips code has imperatives for six audiences: colleagues, clients, students, the public, myself, and the employer and management. by separating the client and the employer it avoids the possibility that the interests of the client and the employer may not be identical. this code and australasian journal of information systems volume 16 number 1 2009 175 others are based on the belief that a set of ethical obligations--professional ethics— based on responsibility to the client is based on the societal impact of the occupation. because of the nature and impact of computing, a higher level of care is required. consistent with this, many codes advocate the avoidance of negative consequences of professional activities. in the association of computing machinery (acm) code there are general statements dealing with responsibilities in the event of negative consequences. for example section 1.2 dealing with the responsibility for negative consequences says that a person is obligated to undo or mitigate negative consequences as much as possible. this is clearly a shift from earlier codes. the earlier codes were designed to protect the computing professional. this and numerous other sections are designed to protect society, almost in the sense of strict liability. it does not say that the member is obliged to undo the negative consequence only if they were due to a failure or malpractice of that member. even if the negative consequence were the fault of the customer’s incorrect use of a product, the member is still responsible. the code, at this and numerous other places, first protects society, and then the professional. the development of a computer system requires a consideration of all stakeholder’s rights. for example, section 3.5 states “articulate and support policies that protect the dignity of users and others affected by a computing system. designing or implementing systems that deliberately or inadvertently demean individuals or groups is ethically unacceptable. computer professionals who are in decision making positions should verify that systems are designed and implemented to protect personal privacy and enhance human dignity.” the acm and the ieee have adopted a software engineering code of ethics and professional practice (secepp). this code has been adopted by numerous ict professional organizations and like the other codes above it clearly points to the developer’s responsibility to a range of stakeholders which goes beyond the software developer and the client. “these obligations are founded in the software engineer's humanity, in special care owed to people affected by the work of software engineers, and in the unique elements of the practice of software engineering. … in all these judgments concern for the health, safety and welfare of the public is primary; that is, the "public interest" is central to this code.” but as we have seen simply mentioning a broad range of ethical obligations does not seen to satisfy those who want to see the codes enforced with severe sanctions. lockheed martin enforced its ethics regulations and fired 28 software engineers for violating the ethics policy. the enforcement of the policy struck many as toothless since the fired software engineers will easily get other jobs and lockheed martin and their staff suffered because of a staff shortage. it sometime looks as if professional organizations actually do not want to enforce the code. there is a problem that some codes seem vague and are neither easily enforceable nor useful in making decisions. there is the already mentioned problem of jurisdiction only over an organizations membership. how can you enforce a code on people who are not members of your organization and have not agreed to be bound by the code? professional organizations can leave their codes as hollow statements or do something to help them have an impact. two major difficulties with codes for professional ethics committees. we will trace the development of the acm’s code of ethics, a development which matches the most code’s development and also maps a difficulty with ict development. the two major themes running through the code’s development are the questions concerning enforcementby whom and howand currency-how address changing technology. (berleur, 2008) australasian journal of information systems volume 16 number 1 2009 176 the early years: ethical standards in search of identity on november 11, 1966 the association of computing machinery (acm) adopted an ethics standard. the acm council adopted a set of guidelines called “professional conduct in information processing” (parker 1968). the pattern of concerns and development of this ethics standard is similar to the patterns of many such developments. the 1966 discussion revolved around questions of: whether information processing was really a discipline, whether it was a single discipline which could have a single standard; what types of effective enforcement it could have, was it meaningful to merely expel miscreants from membership in the acm, who would determine when to enforce the standard, and a narrowly us-centric concern that the enforcement of a professional standard by the acm might alter the acm’s tax status as a scientific society. at least two trends from this early approach continue through the acm’s development of ethical standards: there is recognition that the rapid and unanticipated changes in the profession will require modification of the ethical standards at some level and there is a recognition that agreement on enforcement is difficult to manage. the 1966 standard handled both the enforcement and the change issue in the same way. the recognition of change was addressed by calling the first ethics document a “guideline.” the label was also intended to address the enforcement issue – “the acm council has wisely adopted ethical rules as a guide to members rather than a code to be enforced”. as a result there is no enforcement function directly related to the code/guidelines. approaches to the issue of change have been constant while in the acm there has been a significant change in the approaches to enforcement. the society’s means of addressing these issues of enforcement and technological change dictate the role of the acm’s committees related to ethics. 1970-1992 from guidelines to standards and the need for enforcement just 4 years later there was a change to article 3 section 4 of the acm constitution that stated ‘demonstrating a lack of integrity’ was a reason for being ‘admonished, suspended, or expelled’ and gave the authority to acm council to impose these sanctions. the amendment also mandated the development of a code of professional ethics. this led to the development of a code with detailed ethical statements which are more amenable to enforcement, and whose violation is easier to determine. there was an obvious flaw in the implementation of this approach. a fundamental concept was overlooked. in order to obey the rules you need to know what they are. this is education problem is not unique to the acm. according to lucas and weckert (2008), more than 30% of those questioned are unaware of the acs code. this is an education problem. perhaps education needs to be emphasized more than enforcement. there is also a conceptual problem with the approach. there is a basic problem with a precise checklist approach to ethics especially when it applies to a technical field. the computing field changes and advances very rapidly. what was considered best practice and mandated 20 years ago may be dangerous now. early medical practices are now considered very dangerous. a precise ethics checklist, which is easy to enforce, is out of date almost as soon as it is off the press and what it advocates may be inconsistent with current best practice. this code was adopted in 1974; nevertheless an enforcement procedure was not approved until 1978 (smoot, 1981) reflecting the continuing uneasiness about enforcement (perspective). during this time the acm did have a committee on ‘professional standards and practices’ whose role included services to individual acm members who face ethical problems such as whistle-blowing, product reliability and safety issues, and employment problems. australasian journal of information systems volume 16 number 1 2009 177 the adoption of the 1974 code and later adoption of a policy designating the enforcement method as the sole responsibility of acm council was a significant change from the acm’s original ethics guideline. addressing and enforcing ethical issues related to acm members was now the sole responsibility of the governing organization of the acm the acm council. according to the acm constitution article 6 section 8 “a member may be admonished, suspended or expelled for demonstrating lack of integrity” by a three fourths vote of council after a hearing”. this still left open the nature of the hearing. 1990 the emphasis on guidelines returns in 1990, adopting the insights from 1966 that ethical guidelines for computing need to change to address unanticipated changes in the profession, ron anderson proposed that the current acm code which had a structure that listed possible ethical violations needs to be revised. he further argued that the “acm needs a revised organizational structure for an ongoing review, reformulation, interpretation, and application of its code of ethics and professional conduct.”(anderson, 1990) acm council supported this request and two years later, on 16 october 1992, a code of ethics and a suggestion for a revised review standard was presented. this 1992 code was developed over a two-year period, which included multiple drafts and reviews by acm members. the acm approved a new code of ethics which de-emphasized enforcement and emphasized education of members, of prospective members and of the public. the code’s use as an aid to decision making was also emphasized. the code, which is still in use today, has a two level structure. it consists of 24 ethical imperatives each of which has an associated guideline illustrating the imperatives application in computing. the imperatives are divided into four sections. the first section gives a set of general moral considerations, the second identifies additional ethical principles applies to computing professionals, the third section pertains to organizational leaders, and the final section deals with issues of general compliance with the code. it was envisioned that the high level imperatives would be constant and that the lower level clauses would require updating when technology and practices changed. in addition to the approval of the code, a committee on professional ethics (cope) was also established to meet the need for revision and nurturing of the code. the charge for the cope committee was to 1) promote ethical conduct among computing professionals by publicizing the code of ethics and by offering suggested interpretations of the code; 2) plan and review activities to educate the membership in ethical decision making on issues of professional conduct; and 3) review and recommend updates, as necessary, to the code of ethics and professional conduct and its guidelines. these changes should have addressed the concerns of 1966 about enforcement and code revision. the nature of the code, emphasizing voluntary compliance and consisting of aspirational and normative imperatives rather than disciplinary imperatives helped reduce the concern about sanctions and enforcement. the structure of the code with fairly constant imperatives and flexible guidelines helped address the state of flux of the computing profession. cope was a committee with a charge but without a structure. many of the items described above were not addressed in the establishment of cope. the president of the acm appointed the chair of cope. the chair of cope determines its structure and guides its activities. the outreach functions of cope are fairly straightforward and modelled to some extent on the work when the code was first passed. cope members present papers and participate in computer ethics workshops. they write articles that offer interpretations of the code. (miller, 2003) other professional societies, the australasian journal of information systems volume 16 number 1 2009 178 acs and german gesellschaft für informatik for example, have used the original case studies developed when the 1992 acm code was passed.(anderson, 1993) cope is currently working on a specific set of examples related to internet issues because the world wide web only achieved prominence after the general code was approved in 1992. the role of cope extended in 1999 when the acm and the institute of electrical and electronics engineers, inc. (ieee)-computer society jointly developed and adopted the software engineering code of ethics and professional practice as a standard for a sub-specialization of computing. cope’s domain now includes both of these codes. cope has helped with translations of the acm’s codes by professional organizations that want to adopt these codes. the software engineering code so far has been translated into 9 languages. (secepp). other computing organizations have adopted secepp. for example, in september 2006, the association of software testing resolved to adopt the acm code of ethics as a series of principles to guide and govern practice among its membership.(software testing) the australian computer society (acs) adopted the software engineering code of ethics. in meeting its charge cope also gets involved in design of posters of the code of ethics distributed to member organizations, works on the design of web pages, and includes a commitment to the acm as a separate item on membership renewal forms. cope has primarily limited its education function to the membership of the acm and has only reached out in terms of getting the code included in appropriate textbooks and conducing workshops a computing conferences. the original charge to cope is merely a starting point. the absence of a fixed structure, including a regular schedule of meetings, has led cope to function in response to external requests. the committee is one of the ethical focal points within the acm. cope members are asked to review many of the ethics articles submitted to the communications of the acm. it also responds to ethics complaints forwarded to it by acm headquarters. these complaints vary from the trivial to very significant such as the development of a plagiarism policy that is consistent with the acm’s codes of ethics. in many cases committee members are not knowledgeable in the domain of the ethics problem and need to bring in other committees who have a better understanding of the situation. the absence of a charge which involves the ethics committee in all such issues means that on occasion some very significant ethical issues do not have their ethical component addressed adequately. for example there was a significant issue raised by one acm member – regarding religious discrimination – that was never brought to the attention of cope. (complaint) one of the reasons for the absence of such a charge is that cope only serves in an advisory role. without a clearly defined structure it is sometimes difficult for cope to achieve its goals. another problem is the separation of primarily proactive and advisory functions given to cope, and enforcement functions given to the acm council. often this structure may contribute to a perception that the role of cope is less important. on one hand, the lack of a clearly defined structure makes it difficult at times to achieve its goals. on the other, the absence of a defined structure has the virtue that when an unanticipated issue arises, such as the relation of the code to a plagiarism policy, cope can be involved in those situations without having to wait for a formal meeting. every professional organization ought to have an ethics committee which promotes the code of ethics. there are several things required for such a committee to function effectively: 1. when an ethics committee is established both its charges and structure should be specified. 2. all ethics issues should be passed through the ethics committee. australasian journal of information systems volume 16 number 1 2009 179 3. the method of updating a code of ethics needs to be clearly defined by either the national/professional society or by its ethics committee. this method should be as rigorous and cautiously entered into as the original creation of the code. 4. the structure of the ethics committee should not limit the issues it can address. 5. ethics committees should have a regular venue in the society’s publications to help promote a proactive approach to ethics. this should be accompanied by an annual ethics award included in a national/professional societies repertoire of awards. the ethics committee and ict governance and the software crisis an ethics committee promoting the code of ethics amongst the ict community is a help in addressing the three common difficulties identified in ict governance and in addressing the software crisis. a major role of the ethics committees is public education. the codes of ethics all focus on the professional’s responsibility to a broad range of stakeholders. this responsibility to other stakeholders is paramount in the codes. this lessens the significance of the controlling influence of the discussion of ‘who owns the system” and raises to the forefront the discussion of what are the consequences of the system for all concerned. the breadth of concern makes clear that single mode solutions are inadequate for systems with broad stakeholder responsibility. a code advocates a quality of action and not a particular technology. a constant reminder of the social and ethical impacts of ict systems makes clear that simply focusing on inline documentation in a software program is an irresponsible approach to building an electronic voting system. but codes have no teeth even though the concerns about the constant flux of the computing profession and enforcement of a code’s imperatives have been addressed by modifications to the structures of the codes of ethics there remains the underlying concern of how to get everyone to follow the code. codes are not the teeth of an organization. they do not contain the due process and sanctions within an organization but they do describe the conscience of a profession. within organizations there is generally limited enforcement of a code of ethics. codes get some teeth when they are used by a professional organization to make decisions. the acm code of ethics was used in developing the acm policy and procedures on plagiarism based on “respecting intellectual property rights is a foundational principle of the acm's codes of ethics”. the acm publications board then clearly defined the procedure for reporting alleged plagiarism, the procedure for investigation and how confidentiality during the investigation will be managed. if the paper has already been published the “acm will post a notice of plagiarism based on the investigation on the acm digital library's citation page of the plagiarizing paper and will remove access to the full text.” this give the plagiarism charge some teeth. codes of ethics do not have teeth. they do not define the discipline for violating the code. codes are not self-referential, organizations have bylaws and the code is a bylaw. the due process and sanctions for violating the code is defined outside the code. codes do not have teeth, they are the mind and conscience of a profession. the profession is what gives the rules nurture and teeth. the original concerns of the acs member about those outside the acs not being bound by the standards remain. in a recent case major sections of a student’s master thesis were copied and submitted to a conference and happened to be passed to a reviewer who was the original author’s thesis supervisor. the conference was not an acm event, the plagiarizer was not an acm members australasian journal of information systems volume 16 number 1 2009 180 so the processes defined by the acm did not apply. but that does not prevent the ethics committee from informing the reviewer and the conference about how the acm understands professional standards and the action they would take with members. code of ethics impose accountability on the professional organization a code of ethics is like a swiss army knife, serving many important and useful functions. it is a statement to members about the ethical stand of an organization and profession, a conscience of the profession, an announcement to non-members what the profession standards for (although most often stated in terms of the actions of individuals), it imposes functions on an ethics committee to educate the membership, and it imposes responsibilities on the professional organization itself. one of the functions of an ethics committee is to help its own professional organization understand the importance of the role played by ethical standards. sometimes professional organizations lose sight of these responsibilities or get distracted. in 1972 the ieee set up an ethics task force in response to the bart disaster and because the committee was not publicized, they had no cases until 1978 when it was publicized and they had 11 cases to address. in 1990 they set up hotlines and sent copies of the code out with membership renewals. with each renewal for the acm members sign that they will adhere to the acm code. the ieee hotline provided a direct channel for ieee members to get help when they were faced with painful professional dilemmas. for a variety of reasons the hotline was discontinued. in 2003 the acs established the committee on computer ethics (cce) to promote the development of computer ethics policies (in australia). they are charged to work with acs sigs, help develop policies for government, and promote the importance of computer ethics in the ict community. micromacro ethics confusion the distinction between macro and micro ethics is important for these committees and for the use of codes of ethics in addressing the ict governance/software crisis problem. the lack of attention to this distinction is a problem that pervades many codes of ethics, ethics committees and professional organizations. generally they all focus on the responsibility of the individual computing professional. the general view is that codes and regulations are about the behavior of the individual member and not about the organization nor of the profession as a whole. micro focused codes of ethics talk about ‘you’ and not specifically about the professional organization or about the profession. the complaints about an absence of sanctions view the code as being primarily about the ict person’s behavior. 23% of public sector workers in australia did not see their work as related to a larger whole (lucas, 2008). it is the ict individual who is asked to be ethical. this narrow understanding of the scope of a code of ethics and professional standards is a problem. this view of the scope of a code of ethics affects their work. they “view themselves as not part of other systems but as separate ict folk” (lucas, 2008, p 23). if micro-ethics is the focus then enforcement australasian journal of information systems volume 16 number 1 2009 181 is the primary consideration and the professional losses sight of the contribution ‘doing it right’ makes to the quality of life. unfortunately few codes have sections dealing with macro ethics. sometimes it is not clear in an ethics committee’s terms of reference. the terms of reference for the acs committee on computer ethics has it right when it says they should promote “the value and importance of computer ethics within the wider australian, as well as regional and international, ict community”. the standards are not just for acs members but for the ict community at large. they also recognize the responsibility to society as a whole in the charge “to advise the society and the ict community on ‘best practice’ in relation to computer ethics;” this awareness and promoting of the professional organization’s responsibility to the whole profession, not just to acs members addresses the whole software development standard. it shows a major function of professional ethical standards beyond the punishment of miscreants. this macro understanding justifies an approach to encourage adherence to the code. if the standards become common knowledge, the public knows what ought to be done then those who do not follow the standards will receive less business, provided the failure to follow the standards is made known as in the swiss model. for example, there was some concern about the lead content in the paint of some children’s toys sold in the usa. citizens were made aware a) of the danger of the levels of lead and b) that toys exceed these levels of lead. in spite of a lack of government testing on all toys, when the knowledge spread that x toy was below this safety standard the toys are no longer purchased. the ethics committees need to clarify and formally state principles important to the profession as a whole. this need was supported by the comments in lucas and weckert (2008) which indicate the importance of training, but the commenters see ethics solely as an individual responsibility: • any other commentsethicspunitive measures [when asked about them] are also useful but you can't take action unless you communicate your expectations. the message is: communicate your expectations, police them, and then maintain them. if there are no consequences people will not be motivated to behave ethically. so you need to put the sign post up (i.e. 80 km/h), communicate it and police it. • needs to be incentives, and the value of ethics for the business to be recognized. • organizations need to communicate their ethical expectations from staff and then ensure that they are aware of them. management should encourage staff to take their ethics training so that they do not have an excuse when it comes to expectations. • needs to be more promotion of ethics. frequently overlooked. essentially it is up to individual, but training helps. conclusion codes of ethics and regulations can have teeth. if professional organizations are committed to elevating professional practice and truly committed to the standards in their codes then they should publish a list of expelled miscreants who violated the standards. maintaining silence in the light of such violations is inconsistent with the content of the codes of ethics. the acs takes a different view. acs reprobates are not named but “describe[ed] in general terms (ie, to protect the privacy of australasian journal of information systems volume 16 number 1 2009 182 those involved) the breaches that have occurred and the actions that have been taken by the acs in respect of those breaches.” the important stakeholder is not the reprobate. the important stakeholders are everyone affected by that persons actions and the profession which maintains silence about unprofessional actions. professional committee need to be aware of: 1. a focus on micro ethics contributes to the single mode solutions and who owns the system problems. it also is used to justify ignoring a sense of responsibility for the whole project, for the whole profession. 2. the worst model of leadership is the whip. the committees must educate about the what and the why of regulations. 3. the rules are guidelines with a purpose. codes and ict governance are not checklists which will be out of date shortly after they are written. 4. there needs to be as much if not more emphasis and financial support for education than on enforcement. enforcement/encouragement can take a variety of forms. 5. take the codes seriously-publish the list of those ejected for violation of code of ethics. be the better business bureau for software quality and list if there are open complaints. codes of ethics and regulations may be toothless tigers, but they can still be heard. their message is important to address the problems at the interface of ict governance. references acm policy and procedures on plagiarism http://www.acm.org/pubs/plagiarism%20policy.html acs see http://www.acs.org.au/news/060404.htm acs procedure http://www.acs.org.au/index.cfm?action=show&conid=memberdisciplining agle, b.r., & mitchell, r.k., sonnenfeld, j.a. (1999). “academy of who matters to ceos? an investigation of stakeholder attributes and salience, corporate performance, and ceo values.” management journal: 42(5) october, 1999: 507-525. ais (2005) “definition of system quality” – /http://business.clemson.edu/ise/html/system_quality.html 2005 anderson, r. ( 1990) “a rationale for the proposed revision of the association for computing machinery’s code of professional conduct,”http://www.southernct.edu/organizations/rccs/resources/research/comp_and_priv/a nderson/background.html anderson, r. (1995) “the acm code of ethics: history, process, and implications,” social issues in computing, mcgraw-hill, new york, , pp. 48–72. anderson, r; johnson, d.; gotterbarn, d.; perrolle, j; ( 1993) “using the new acm code of ethics in decision making”, communications of the acm 3, 2 february 1993 as/nzs (1999) “risk management, standard 4360-1999”, standards australia, standards new zealand australasian journal of information systems volume 16 number 1 2009 183 as8015-2005 “corporate governance of information and communication technology.”, standards australia committee it-030, ict governance and management in 2005 berleur, j (2008) ethics of computing committees: suggestions for functions, form and structure, to promote discussion inside the ifip national societies, on behalf of ifipsig9.2.2: jacques berleur, oliver burmeister, penny duquenoy, don gotterbarn, philippe goujon, kari kaipainen, kai kimppa, benjamin six, debora weber-wulff, diane whitehouse, eds., ifip press, laxenburg-austria, 2008, 32 p., isbn 978 3 901882 24 3. the account of the acm code of ethics development is based on my report to this group. berleur, j. & d’udekem-gevers, (1994) “codes of ethics, or of conduct, within ifip and in other computer societies”, 13th world computer congress 1994 pages 340-348. bowern, m. (2006) “ethics” information age 14/02/2006 13:21:16 http://www.infoage.idg.com.au/index.php/id;1260028245;fp;131072;fpid;0 the british computer society documentation on its code of conduct and disciplinary procedures can be found from a keyword search at : www.bcs.org complaint, “my complaint against the acm a leading technological society condones employment discrimination against some of its own members. copyright © 2001-2003 by david e. ross, http://www.rossde.com/acm.html cockburn, a. (2004) crystal clear, addison-wesley, dijkstra, edsger w. (1976), a discipline of programming, prentice-hall, englewood cliffs, n.j gotterbarn, d. (1996) “software engineering: the new professionalism,” the professional software engineer, c. myer, ed., springer-verlag, new york. gotterbarn, d., k. miller and s. rogerson, (1998) “software engineering code of ethics”, [online: available, http://www.acm.org/serving/se/code.htm accessed 1/27/2007] communications of the acm. halstead, maurice h. (1977) elements of software science, operating, and programming systems series volume 7. new york, ny: elsevier. highsmith, j. (2002) agile software development ecosystems boston: addison-wesley. lucas, r. and weckert, j. (2008) “etgovict 2008 conference, briefing paper for presenters” centre for applied philosophy and public ethics, australian national university ©2008 martin, m. w. et al., ethics in engineering, 2nd ed., mcgraw-hill, new york, 1989. miller, k. gotterbarn, d., “computer ethics in the undergraduate curriculum: case studies and the joint software engineer's code.” small college computing conference journal 2003 parker, d. (1968) “professional conduct in information processing,” communications of the acm, (11) 3 march paulk, mark c. (1995) the capability maturity model: guidelines for improving the software process. reading, ma: addison-wesley publishing company, pediatrics (2006) accessed 1/25/2007 http://pediatrics.aappublications.org/cgi/content/abstract/118/5/1872, perspectives on the professions, centre for the study of ethics in the professions at illinois institute of technology, http://ethics.iit.edu/perspective/pers1_1mar81_3.html australasian journal of information systems volume 16 number 1 2009 184 ravichandran, t. (2000) “total quality management in information systems development: key constructs and relationships” journal of management information systems, 1999/ 2000, 16 (3) ,p119,37p secepp http://seeri.etsu.edu/codes/thesecode.htm1 smoot, http://ethics.iit.edu/perspective/pers1_1mar81_3.html software testing, read resolution on adoption of the acm code of ethics http://www.associationforsoftwaretesting.org/ethics.html weill, p & ross, j.w. (2005) “how effective is your it governance,” http://mitsl soan.mit.edu/cisr/pdf/weill-ross%20-%20effective%20it%20governance-final.pdf microsoft word 8 ajis 14-2.doc australasian journal of information systems volume 14 number 2 june 2007 81 the development of a b2g online authentication standard: a design perspective of the policy consultation process john campbell school of information sciences and engineering university of canberra canberra, australia john.campbell@canberra.edu.au abstract the primary focus in design science research is the development of innovative and useful system artifacts. apart from it-centric artifacts such as software and hardware, design research outputs can also include constructs, models, methods and better theories. however, there is very little published research concerned with these alternative artifact genres. the research reported in this paper focuses on one of these alternative design outputs that are of particular interest to information systems; the development of innovative technology standards. in this paper it is argued that much can be learnt from using a design science approach to analyze these types of information systems artifacts. a design science theory of punctuated action is presented and used to briefly explore the public consultation process in the development of a b2g online authentication standard for the australian federal government. keywords: design science, method artifact, standards development, online authentication introduction design science is concerned with the creation and evaluation of innovative artifacts aimed at achieving human-defined goals and is founded on two fundamental development activities – build and evaluate (simon 1981). while at first glance these two concepts appear intuitively straightforward, their operationalization can be quite complex as often there are sometimes any number of viable design solutions available. also, the functionality of each of these design alternatives will be highly reliant on implementation context. consequently, a failure to understand australasian journal of information systems volume 14 number 2 june 2007 82 implementation context can result in artifacts that are inappropriately designed or that have undesirable side effects (march and smith 1995). design science research has not had the same recognition as behavioral science in the information systems literature. however, the approach is gaining greater appreciation and recognition as a core element in information systems research (see burstein and gregor 1999, gregg et al. 2001, hevner and march 2003, hevner et al. 2004, march and smith 1995, markus et al. 2002, nunamaker et al. 1991). in an information systems context, design science research involves the study of innovative design artifacts for the purpose of understanding, explaining, and improving the performance of information systems. a designed artifact can take many forms beyond computer-based systems and can include instantiations of organizational structure, work processes, information systems, strategic plans, standards, and public policy (hevner et al. 2004). the primary goal of this paper is to demonstrate how design science principles can be used as a means of understanding the development of inter-organizational standards by examining the public consultation process in the development of a b2g online authentication framework. in the following sections, design science is explained in the context of it artifacts and information systems research. a theory of punctuated action (simon 1977, boland 2002) is advanced to better understand the implications that flow from distinct intelligence, choice and design configurations. the design implications for the public policy consultation process are considered and i conclude with a discussion on other research work that might provide additional insight for using design science in information systems research. design science in information systems recently, hevner in collaboration with others (hevner and march 2003, hevner et al. 2004) have developed a framework of information systems research that brings together the behavioral science and design science paradigms. figure 1 describes this framework in which the building and evaluating of theories and artifacts define information systems research. although hevner et al. (2004) acknowledge that policies and work practices are valid examples of designed artefacts their model does not show how these objects are accommodated within their framework. figure 1 contains minor amendments to the original model so that it includes scope for documented design artefacts such as work practices, procedures and information systems policy australasian journal of information systems volume 14 number 2 june 2007 83 figure 1. information systems research framework (adapted from hevner et al. 2004) research activity is moderated by environment and by the knowledge base that can be applied. the environment is the problem space in which the phenomena of interest are located, and the knowledge base as a composition of information systems research foundations and methodologies. information systems research is located at the centre of this framework where appropriate methodologies are chosen from a discipline knowledge base and applied according to business needs. hevner et al. (2004) argue that better information systems research comes from combining and integrating behavioral science and design science. from a behavioral science perspective, information systems research is viewed as a branch of the social sciences (hevner et al. 2003, 2004). this view has developed from the traditional natural science perspective where the primary goal of research is to test and validate theories about the design of information systems. the evaluation and validation activities of research methods in the behavioral science paradigm include the familiar means of case studies, experiments, field studies and surveys. these are the traditional research methods used by information systems researchers to is research relevance rigor knowledge environment application implementation additions to the knowledge base foundations • theories • frameworks • models • methods • etc. methodologies • data analysis • measures • formalisms • validation • etc. people • roles • capabilities • characteristics organisations • strategy • structure • culture • process technology • infrastructure • architecture • applications assess refine justify/evaluate • case studies • experiments • field studies • simulate/model • etc. develop/build • theories • designed artifacts • applications • policies • procedures australasian journal of information systems volume 14 number 2 june 2007 84 intelligence design choice identify and test theories. in contrast, the design science perspective in information systems research seeks to evaluate the utility or quality of the system artifact. system utility and quality must be clearly defined in order to undertake this evaluation. for example usability, functionality, completeness, consistency, accuracy, reliability, and performance are all potential system quality attributes (hevner et al. 2004). however the overall goal of the evaluation is to not only assess system usability but also to provide guidelines for incremental improvement of the system artifact. relevance is determined by how well information systems research addresses business needs while rigor is achieved through the appropriate application of foundation principles and methods. while software and hardware are regarded as the core "working" artifacts in information systems (see orlkowski and lacono 2001, weber 2003), there are other information systems artifacts that are also important components in the creation of innovative information technology systems (sen 2006). these artifacts are constructs, models, methods (march and smith 1995, hevner et al. 2004) and better theories (rossi and sein 2003). constructs define the conceptual vocabulary of a domain, models contain an expression of how constructs are related, methods provide a description on how to perform some task, and better theories are derived from experimental like proofs of concept or method during the design construction phase. the focus in this paper is on the application of design science in the development of an online b2g authentication framework – a method artifact. the following section establishes the theory of punctuated action based on distinct decision-making configurations of the intelligence, design and choice activities articulated in herbert simon’s decision-making theory (simon 1977, 1981). design, intelligence and choice in designed artefacts simon (1977) described three interrelated decision-making activities in his book the new science of management decision: intelligence, design and choice. intelligence is a critical function directed towards identifying a problem or some deficit relating to an earlier design choice, or absence thereof, which is less than satisfactory. intelligence alerts a decision-maker to the need for action in response to some new threat, opportunity, or to the need for change if a required objective is not being achieved. design, is the combination of alternative actions that can be brought to bear to change an existing situation to better achieve desired objectives. design includes an understanding of the medium, methods and processes available to the decision-maker as well as the potential implications and trade-offs between different design choices. choice is the act of selecting a design alternative that will achieve desired objectives. choice includes an appreciation for how competing alternatives will fit with other design choices and includes an appreciation of how the current conditions are likely to affect design alternatives. these three interdependent decision making activities are illustrated in figure 2. figure 2: interdependence of intelligence, design and choice in management australasian journal of information systems volume 14 number 2 june 2007 85 while intelligence, design and choice are interdependent elements in simon’s model, decisionmakers experience each element as a separate form of action. in practice, each of the three elements is engaged in a way that brings that particular activity to the fore while the other two activities are forced into the background. boland (2002: p. 3) observes that, " ... we attend to any one aspect from (or in light of) the others. while one aspect is in focal attention, the others are held in a subsidiary, or tacit, way. so that while the three are thoroughly interdependent, they are not simultaneously in focal attention, and we can therefore think of the three aspects as taking place separately and sequentially in management action." each activity, in itself, is a complex decisionmaking process with each element containing its own intelligence, design and choice activities – phenomena herbert simon referred to as being "wheels within wheels." boland (2002) draws on karl weick’s ideas of sense making to illuminate how individuals might punctuate decision making into sequential patterns of actions to produce plausible and coherent understandings of a given situation. in so doing, boland is able to disentangle the circular pattern of influence depicted in figure 2 to produce six distinctive ways that a decision maker can punctuate decision making action into specific sequences of intelligence, design and choice. in each of these punctuations one action happens first which then serves as the beginning of a narrative episode of acting in the world. each action sequence reflects a sense of moving forward, making sense of a new situation or existing context, and then some form of adaptation as a consequence. these punctuated action sequences of intelligence, design, and choice are described below. 1. rational goal seeking (intelligence design choice) this punctuated sequence corresponds with the classic view of rational goal seeking behavior and relies on the application of judicious intelligence to inform and guide organizational action. this is an instantiation of herbert simon’s basic decision-making model where intelligence recognizes the need for intervention, design provides alternatives for consideration, and choice selects the best option (or one that satisfices). although this punctuated action sequence does provide us with logical explanations for particular decision outcomes, it can also lead to ever finer levels of reductionism and detail that eventually become irrelevant to the human condition. 2. happenstance rationalization (design choice intelligence) this activity punctuation begins with an action or a given environmental state that constrains, at least to some extent, the possibilities for future action. this initial action or state is accepted in whole or in part as intelligence is applied to refine what has occurred, or to justify the initial choice action. this sequence of punctuated action can lead to the premature acceptance of choices that, in turn, leaves intelligence action to reinforce this happenstance through some process of rationalization. for example, a manager may have difficulty in ignoring the sunk costs associated with some pre-existing organizational strategy or investment when considering a future course of action. 3. existential introversion (choice intelligence design) in this punctuation, action begins with existential choice. it is then followed by an analysis of how to achieve this choice and then the design alternatives. this action punctuation might be characterized by the tinkering manager who fixes things that are not broken or has a solution to some yet to be discovered problem. boland (2002) contends that this action punctuation provides for poor results, as the decision-maker tends to operate in isolation from the situation or context within which the action takes place. while poor outcomes might be the general rule, there are examples where existential introversion has won out. the rise and rise of the microsoft organization is one example that readily comes to mind. australasian journal of information systems volume 14 number 2 june 2007 86 4. deferred design (intelligence choice design) this form of punctuation begins with the decision-maker sensing a situation in the environment and making choices for action. subsequently, the decision-maker acquires knowledge or understanding of other courses of action that could have been followed. this action sequence is characterized by a delayed use of design. this punctuated action sequence is a failed version of the rational goalseeking action described above. in this case, the decision-maker rushes to a decision only to later find that alternative designs might have been viable. 5. situational sensemaking (design intelligence choice) in the situational sensemaking punctuation, a design enactment comes to the actor first as raw action from the environment. the design enactment causes the decision maker to engage in a sensemaking process in which intelligence is applied to selecting those elements of the initiating action that are meaningful and acceptable. this intelligence activity precedes the choice of which meanings and structures to carry forward in subsequent instantiations. boland (2002) describes this action punctuation as having parallels to weick’s sensemaking model of variation, selection, and retention (see weick 1979: pp. 122-126). interestingly, simon (1977: p. 43) proposed that this action punctuation was highly appropriate where novel technologies were being considered. 6. existential heroism (choice design intelligence) in this form of punctuated sequence, existential choice is the primary driver of action as it is for the existential introversion type punctuation described above. however, the actor first chooses who or what they are, develop alternative designs, and then perfect these designs based on some selfdefined criteria. like the existential introvert, the existential hero makes no reference to the environment for feedback on goals. the environment is essentially ignored in favor of the existential choice of the individual. the six punctuated action sequences described above will be used to briefly analyze the public consultation process in the development of a b2g online authentication framework currently being proposed by the australian federal government. australia can be described as a liberal democratic country where citizens and stakeholder interest groups are implicitly included in many government policy decision-making processes (päivärinta and sæbø 2006). the following section provides background information, details on the policy development process, and a description of the main thrust of the proposed authentication framework design. e-government and the online authentication framework the australian government authentication framework (agaf) is aimed at providing a whole-ofgovernment approach to electronic authentication in all online b2g transactions and was released as a draft exposure document on 21 may 2004 (agimo 2004). the specific content and practical implications of the agaf draft proposal are not central to the analysis presented in this paper. this is because the focus of analysis is the structure of the decision activity within the standards development process and not directly on the content of the policy itself. nevertheless, the authentication framework is briefly discussed below along with relevant background information so as to define the context within which the online authentication standard is developed, and by which events the research is framed. australasian journal of information systems volume 14 number 2 june 2007 87 although electronic government initiatives have lagged commercial applications of internet technology, this position is rapidly changing with many national administrations aggressively pursuing online programs (west 2005, nzssc 2004). here in australia, online business transactions with government provide significant reductions in b2g transaction costs and improvements in decision-making (noie 2003). the australian bureau of statistics recently reported that seventy-seven percent of australian businesses utilized the internet in 2004-2005 with internet-based business income rising to $40 billion during 2004-2005 up from $11 billion in 20012002 (abs 2006). not surprisingly, a rapid increase in the volume of b2g online transactions has accompanied these trends. for example, in 2002-2003 seventy-one percent of businesses using the internet visited a government web site to access a range of online services including: the lodgment of taxation forms online (twenty-one percent of businesses) online payments (twenty-eight percent of businesses) information or services related to taxation (forty-two percent of businesses) information on regulations (thirty-five percent of businesses) information on employment (twenty-six percent of businesses) the australian federal government is endeavoring to build on the progress in e-government to date and move forward towards the vision of becoming a connected and responsive government by supporting activities in four core service-oriented areas (agimo 2006): meeting the needs of citizens consistent and unified points of access achieving value for money enhancing public sector capability. with this increasing business and government interaction, there has emerged the need for the development of an authentication framework capable of supporting a trusted online environment where businesses can interact safely and securely with all federal government departments. consequently, the australian government information management office (agimo) released the australian government electronic authentication framework exposure draft on 21 may 2004. the importance of a secure b2g transacting environment is highlighted by the following extract: when transacting online, there will be occasions where businesses and the government agency will need to be completely assured of each other’s identity and the legitimacy of the assertions that are being made. these assertions may relate to a range of attributes such as identity, professional qualification, or that a person is authorized to conduct a specific transaction. the need for assurance is particularly important when funds or sensitive information is involved. the process of establishing the legitimacy of assertions, be they identity or other attributes, is the key element of business authentication.” agimo (2004: p2) currently government agencies use a range of authentication technologies including passwords, digital certificates and public key infrastructure. the agaf draft document proposes a whole-ofgovernment approach to user authentication in online b2g environments (agimo 2004). the stated objective of the framework is to ensure that australian government agencies apply a consistent approach when deciding the level of authentication required for different types of online b2g transactions. consumer to government transactions appear to be outside the terms of reference of the proposal at this stage but would seem to be a conceivable extension to the framework at some future point in time. the proposal explicitly recognizes that different authentication techniques will be required for different types of transactions based on some form of risk assessment. figure 3 shows that low-risk transactions will require only low-level authentication while high-risk transactions will require high-level authentication mechanisms such as public key infrastructure. australasian journal of information systems volume 14 number 2 june 2007 88 figure 3: authentication techniques and risk levels described within the agaf draft proposal document this section has provided a very brief introduction to the agaf proposal and its implications for australian businesses. further policy details can be obtained directly from the draft document (agimo 2004). also, a number of organizations have produced written responses to agaf and some of these documents are publicly available (see for example acci 2004). the following section looks specifically at the agaf public consultation process using the theory of punctuated action discussed earlier to structure the analysis. agaf policy as design artefact the overall objective of this section is to use the punctuated action theory to interpret the structure of the agaf standard public consultation process. the general objective when applying this model is to test the usefulness of design science for framing the development of standards as a designed method artifact. the suitability of using design science as a lens for analyzing information systems standards development is addressed by asking the following: what insight can design science provide to help better understand, predict and evaluate the australian government authentication framework development process? while it might be difficult to provide absolute predictions about agaf policy development outcomes, we can nevertheless abstract system properties that are likely to arise out of a particular mode of system organisation. low level of authentication high l o w l e v e l o f r is k h ig h minimal high low moderate no authentication required (eg, information on government services) password or pin-based system (eg, business entry point) pki/digital signature (eg, lodgment of business activity statements) shared information (eg, application for an australian business number) australasian journal of information systems volume 14 number 2 june 2007 89 for this study, we consider only the development of information systems policy in the context of design science as it relates to the public consultation process for the agaf draft exposure document. a design science analysis can be applied to the whole of policy development process from conception through to implementation, and arguably beyond. examining only the consultation process allows for deeper understanding of a core component of standards development at a critical point in time. another advantage is that processes during this stage are readily observable due to the emphasis on public interaction and feedback. in contrast, other stages of the policy development process are undertaken internally by agimo, which make it very difficult to gain access to policy data, decision-making processes and outcomes. this is particularly so where a policy is likely to have significant implications in the wider business community. design science implications for the consultation process in agaf policy development “there is nothing a government hates more than to be well-informed; for it makes the process of arriving at decisions much more complicated and difficult” john maynard keynes (skidelsky 1992, p. 630). despite this populist view expressed by keynes, participation has become a core principle of public sector governance in australia and is seen as being achieved in practice by the engagement of experts, interested parties and stakeholder groups during various stages of public policy development and implementation (catt and murphy 2003). the participation process can take a number of forms, but most variations seek to ensure that communities and other affected groups have satisfactory input into the regulatory activities and practices of government agencies. while the responsibility for policy formulation and the final decision-making unquestionably rests with government, it is argued that better policies and greater community commitment can be realized from engaging the public in policy-making processes (oecd 2001). for the purposes of this study, the public consultation phase in the development of agaf makes for an interesting juncture for analysis using design science. the theory of punctuated action will be used to assess the agaf consultation process paying particular attention to how the process is constituted and to how this process might influence and shape policy outcomes. after many years in development, the agaf policy has reached a stage where an initial draft exposure document has been produced and released for public comment (for an historical perspective on the evolution of the framework see noie 1998, 2002 and agimo 2004). various individuals and key stakeholder groups acting alone or in concert with others are then able to respond to the draft exposure by submitting a formal submission to agimo. after the due date for formal submissions, key contributors are then invited to participate in a business impact consultative group face-to-face session. this meeting is held in confidence with participants contributing to a general discussion on the strengths and weaknesses of the draft proposal. it is anticipated that policy is further adapted in the light of the feedback received before being prepared for ratification by the relevant departmental head or minister. applying punctuated action theory, the policy development process can be illustrated as in figure 4. the internal environment within which the policy is formulated and communicated to the public is provided by the host government agency – in this case agimo. here it is assumed that the policy development process initially follows simon’s rational goal seeking punctuation where intelligence precedes design and choice (i.e., intelligence design choice). australasian journal of information systems volume 14 number 2 june 2007 90 figure 4: the public consultation process in agaf standards development. boland (2002) identifies this action narrative with many public policy developments. this view assumes that that there is some shared national goal and that a range of alternatives can be developed and that the approach that maximizes the national interest is selected. at first glance, the commonsensical nature of the rational goal seeking punctuation appears fundamentally attractive. while the strength of this approach is that it focuses attention on developing policy aimed at satisfying existing needs, its weakness is that it may not be as useful in emergent environments where policy development must be more forward-looking. the structure of this punctuated action narrative also has implications for how much influence public consultation will have on the design of a standard. on the one hand, a draft exposure document that was determined on the basis of a rational goal seeking approach would, in theory, provide a solid design base upon which stakeholders could focus their comments and concerns (as implied in phase a in figure 4). on the other hand, the standards foci and scope will have been "hard-wired" to a great extent into the initial draft exposure document thereby restricting debate and legitimizing only incremental variations on the original design. this "design tautology" could be further exaggerated if elements of the policy are made more explicit in an attempt to preempt the specific concerns that might emerge from the public consultation process. standards development: consultation process public submissions i d c d c i d c i d c i d i c public consultation d i c phase b phase a draft exposure d c i d i c ? australasian journal of information systems volume 14 number 2 june 2007 91 discussion a very interesting pattern emerges from the public consultation phase with regard to stakeholder action narratives available with reference to punctuated action theory. as the design action is a given, a respondent can adopt only one of two distinct action narratives: (1) happenstance rationalization (design choice intelligence), or (2) situational sensemaking (design intelligence choice). those submissions involving stakeholders acting collectively will most likely display elements of both punctuations. while happenstance rationalization may not be desirable, situational sensemaking behavior is likely to generate useful feedback and should therefore be encouraged. as mentioned already, there is some risk that the policy development cycle that resumes within the government agency (as depicted in phase b) will not be entirely responsive to the unusual or extreme suggestions that might emerge from the public consultation process. this is particularly so if happenstance rationalization (design choice intelligence) activity dominates this stage of the process. nevertheless, restricting scope in this way may not be detrimental to the design process and could in fact have desirable outcomes. for example, restricting agaf focus to b2g transactions will cause controversial suggestions such as extending the framework to include c2g transactions to fall outside the scope of public consultation. the discussion presented in this paper provides only a brief demonstration of how the theory of punctuated action can be used to analyze the public consultation process in the design of information systems policy. some comments are required regarding the assumptions used in this study. these include: the design, intelligence, choice actions can be broken up into discrete activity segments; the design, intelligence, choice actions follow a punctuated linear progression; the punctuated action process doesn’t loop back onto itself (e.g., d i d c etc.) any particular decision-making action might not occur within a three-part decision sequence (e.g., could be d c or c d) these assumptions and other limitations require further research. ideally, the preliminary analysis presented in this paper should be extended to fully map the evolution of the authentication framework into a formal standards artifact. conclusion the question, "how do we get good policy?" inexorably leads us to a more pressing concern, "how do we get good policy design?" carlsson (2002) argues that good policy design comes from removing obstacles to creativity and innovation through policy facilitation. policy developers and organizational decision-makers alike will always be bound by the limits of rationality and also by the constraints of power, politics and social context. design science may lead to better policy design by facilitating an appreciation of how different design mechanisms address desired and undesired outcomes from particular policy development processes. however little is really known about which theoretical models and frameworks are most appropriate for these types of design artifacts. herbert simon spent a lifetime analyzing and describing human artifacts. his ideas on design science are particularly illuminating and justify their pre-eminence in design science research. australasian journal of information systems volume 14 number 2 june 2007 92 while simon’s ideas are compelling, there are of course alternative frameworks that may also prove useful. for example, the work of stafford beer (1968) appears well suited to design science problems yet his name hardly bears mention in mainstream design science literature. while simon focuses on exploiting uncertainty through contingency, beer’s approach to designing the effective organization is aimed at reducing uncertainty through explicit control mechanisms. despite these fundamental differences, both simon and beer share a common desire to further the development of self-sustaining and ethically responsible systems. karl weick’s (1979) widely acclaimed view of sensemaking in adaptive systems also appears to be highly applicable to design science. as discussed earlier, the situational sensemaking punctuation (design intelligence choice) described in this study is analogous to weick’s sensemaking model of variation, selection, and retention. however weick’s appreciation of consensual validation and sociocultural evolution may support a higher level of granularity for analyzing the design of policy and other artifacts. the appropriateness of these frameworks for understanding designed artifacts requires further research. particularly where the designed artifact has social implications or is negotiated collectively as is generally the case in information systems standards development. references abs, australian bureau of statistics, business use of information technology 2004-2005, 8129.0, 17 march 2006. acci, australian chamber of commerce and industry, acci submission to the australian government information management office, 31 may 2004, http://www.acci.asn.au/text_files/submissions /2004/australian%20government%20electronic%20authentication%20framework.pdf, (accessed 17 june 2006). agimo, australian government information management office, australian government electronic authentication framework exposure draft, 21 may 2004. agimo, australian government information management office, 2006 e-government strategy: responsive government: a new service agenda, march 2006. beer, s. management science: the business use of operations research, doubleday & company, garden city, ny, 1968. boland, r.j. "design in the punctuation of management action," workshop on managing as designing, weatherhead school of management, case western reserve university, cleveland, ohio, june 14-15 2002, http://design.case.edu/2002workshop/index.html, (accessed 17 june 2006). burstein, f. and gregor, s. "the systems development or engineering approach to research in information systems: an action research perspective," proceedings of the 10th australasian conference on information systems, victoria university of wellington, new zealand, 1999, pp. 122-134. catt, h. and murphy, m. "what voice for the people? categorising methods of public consultation," australian journal of political science 38:3, 2003, pp. 407-421. carlsson, b. "public policy as a form of design," workshop on managing as designing, weatherhead school of management, case western reserve university, cleveland, ohio, australasian journal of information systems volume 14 number 2 june 2007 93 june 14-15, 2002, http://design.case.edu/2002workshop/index.html, (accessed 17 june 2006). gregg, d., kulkarni, u. and vinze, a. "understanding the philosophical underpinnings of software engineering research in information systems," information systems frontiers, 3:2, 2001, pp. 169-183. hevner, a.r., and march, s.t. "the information systems research cycle,” ieee computer, 36:11, 2003, pp. 111-113. hevner, a.r., march, s.t., park, j., and ram, s. "design science in information systems research," mis quarterly, 28:1, 2004, pp. 75-106. march, s. and smith, g. "design and natural science research on information technology," decision support systems 15, 1995, pp. 251 266. markus, m. l., majchrzak, a., and gasser, l. “a design theory for systems that support emergent knowledge processes,” mis quarterly (26:3), september 2002, pp. 179-212. noie, national office for the information economy, establishment of a national authentication authority: a discussion paper, 19 august 1998, http://www.agimo.gov.au/publications/1998/08/naa, (accessed 17 june 2006). noie, national office for the information economy, online authentication a guide for government managers, 2002, http://www.agimo.gov.au/__data/assets/file/21171/onlineguidefinal.pdf, (accessed 17 june 2006). noie, national office for the information economy, e-government benefits study, 2003, april, http://www.agimo.gov.au/__data/assets/file/16032/benefits.pdf, (accessed 12 july 2006) nunamaker, j.f., chen, m., and purdin, t.d.m. "systems development in information systems research," journal of management information systems 7:3, 1991, 89-106. oecd, organization for economic cooperation & development, citizens as partners: information, consultation and public participation in policy making, 2001, http://www1.oecd.org/publications/e-book/4201131e.pdf, (accessed 17 june 2006). orlkowski, w.j. and lacono, c.s. "research commentary: desperately seeking the 'it' in it research-a call to theorizing the it artifact," information systems research 12:2, june 2001, 145-165. päivärinta, t. and sæbø, ø. "models of e-democracy," communications of the ais 17:37, may 2006, 1-42. rossi, m. and sein, m. "design research workshop: a proactive research approach," 26th information systems research seminar in scandinavia, iris 26, august 9 – 12 2003, http://tiesrv.hkkk.fi/iris26/presentation/workshop_designres.pdf, (accessed 6 july 2006). sen, r. “a qualitative analysis of the role of users, vendors, and governments in the standards development process," communications of the ais 17, 2006, pp. 57-70. skidelsky, r. john maynard keynes: a biography, volume 2: the economist as saviour, 19201937, macmillan, london, 1992. simon, h.a. the new science of management decision, revised edition, prentice hall, englewood cliffs, nj, 1977. australasian journal of information systems volume 14 number 2 june 2007 94 simon, h.a. the sciences of the artificial, 2nd edition, the mit press, cambridge, ma, 1981. nzscc, new zealand state service commission, authentication for e-government: best practice for authentication, april 2004, http://www.e.govt.nz/services/authentication/authenticationbpf/bpf.pdf, (accessed 6 july 2006). weber, r. editor’s comments: still desperately seeking the it artifact, mis quarterly, 27:2, 2003, iii-xi. weick, k.e. the social psychology of organizing, 2nd edition, addison-wesley publishing, reading, mt, 1979. west, d.m. global egovernment 2005, http://www.insidepolitics.org/egovt05int.pdf, 2005, (accessed 6 july 2006). microsoft word ajis vol 15-1 m.doc australasian journal of information systems volume 15 number 1 december 2007 95 sharing a vision: comparing business and is managers’ perceptions of strategic alignment issues jenny leonard university of sydney, nsw, australia email: j.leonard@econ.usyd.edu.au abstract this paper compares the perceptions of business and is managers regarding the issues known to affect the achievement of alignment of is with the business. previous work has identified information systems (is) managers’ organisational position, is business partnerships, and is managers’ understanding of business respectively, as being critical to attaining is alignment. on these issues, the findings of this research are encouraging, and suggest that progress has been made in the past fifteen years. the paper’s main contribution is in identifying a new focus for alignment research – the exploration of three, interrelated issues: the extent to which business and is managers share a vision of the alignment profile of information systems, the extent to which they have a common understanding of the time lags required to achieve such alignment; and the need to educate business managers regarding the strategic potential of is. introduction this paper compares the perceptions of business and is managers regarding the issues known to affect the achievement of alignment of is with the business, and then frames new research questions in the context of the results of the research. alignment has been shown to be critical to maintaining business value. (henderson and venkatraman 1992; henderson and venkatraman 1993, 1999; henderson and venkatraman 1994; chan, huff et al. 1997; hirschheim and sabherwal 2001; sabherwal and chan 2001; sabherwal, hirschheim et al. 2001; chan 2002; ward and peppard 2002; peppard and breu 2003) it has also been shown to be difficult, both to conceptualise (ciborra 1997; scott 2005; silvius 2007) and to achieve (chan 2002; scott 2005). several models of alignment have been discussed in the literature. early models took a “top down” approach, where is strategy was developed from business strategy, australasian journal of information systems volume 15 number 1 december 2007 96 using it strategic planning methodologies (peppard and breu 2003; silvius 2007). a much cited development was that of an alignment model which suggested that the four domains of business strategy, it strategy, is infrastructure and processes, and organisational infrastructure and processes, should all be kept in alignment (broadbent and weill 1993; henderson and venkatraman 1993, 1999). several other studies have further developed this approach – see for example (chan, huff et al. 1997; sabherwal and chan 2001; chan 2002) more recently, two different approaches to alignment have been suggested. the first approach develops the four domain model into an alignment maturity model which can be used to assess an organisation’s current ability to attain alignment, and to indicate how it could maximise alignment enablers and minimise alignment inhibitors (luftman 2000). the second approach suggests that “one size does not fit all” and develops a contingency model with three types of alignment profile: technical resource, business enabler or strategic weapon (weiss, thorogood et al. 2006) the first approach, the alignment maturity model identifies six maturity criteria. three of these; namely governance, partnership, and communications, are directly related to is and business managers’ roles and status, respective knowledge, and working partnership. this emphasis on is and business managers comes from the original empirical study of enablers and inhibitors on which the alignment maturity model is based (luftman and brier 1999). many other empirical studies support this emphasis. see for example (stephens, ledbetter et al. 1992; gottschalk 1999; ross and feeny 2000; enns, huff et al. 2001; enns, huff et al. 2003; preston 2004) on status, (taylorcummings 1998; peppard and ward 1999; preston 2004) on relationships, and (pervan 1998; armstrong and sambamurthy 1999; ross and feeny 2000; basselier, benbasat et al. 2003; broadbent and kitzis 2005) on knowledge. in addition (ross and feeny 2000; broadbent and kitzis 2005) suggest that there have been shifts in all these areas over the last fifteen years. the first contribution of this paper, therefore, is to provide current benchmarks regarding is and business managers influences on the maturity of alignment within their organisation. the first research question, therefore, is as follows: 1. how do is and business managers perceive their own and each others’ contribution to the alignment maturity of their organisation. the second approach, the alignment contingency model suggests that an organisation’s alignment profile will affect the way in which is and business managers need to influence alignment. an organisation with a “technical resource” profile requires little in terms of relationships between is and business managers, and in terms of senior management involvement in is. an organization with a “business enabler” profile requires partnership between the most senior is manager and the ceo; and a ceo who is aware of and supports it. an organisation with a profile that uses is as a “strategic weapon” requires the ceo to take personal responsibility for exploiting is, and requires a cio to be a valuable member of the executive team. this leads to an interesting, new area of investigation. if an organisation’s requirements of its is and business managers can vary so radically depending on its alignment profile, then it is important that is and business managers ensure that they are working together towards the same goal. this leads to the second research question: 2. do is and business managers agree on the alignment profile of their organisation? the paper describes a quantitative and qualitative study designed to address the two research questions. it then uses the results to identify new areas of research in three, linked areas; the differences in perceptions of is and business managers respectively regarding the alignment profile of their organisation; differences in perceptions of the time that it takes to implement new systems australasian journal of information systems volume 15 number 1 december 2007 97 to improve that alignment, and the need for more education of business managers informed by the framework of different alignment profiles. literature review the business impact of alignment a recent, comprehensive australian study named the alignment of is with the business as one of four factors affecting returns on investment in organisations (gregor, fernandez et al. 2004). the reasons why alignment is important are explained by ward and peppard (2002: p47) who describe the problems that occur without it, including systems investments which do not support business objectives; a lack of systems integration resulting in a lack of coherence in the information resource; and systems with a shorter than expected business life requiring considerable is spend to redevelop more frequently than should be necessary. several studies of alignment have reinforced its importance to the success of an organisation (sabherwal and chan 2001; chan 2002). reflecting its importance, a number of surveys of chief information officers (cios) have shown alignment to be one of their top areas of concern (watson 1990; hollands 2005; luftman 2005). theories of alignment models of alignment were first put forward in the early 1980s, when both strategic alignment – the alignment of business and is strategy, and structural alignment – the alignment of the is structure and business structure – were defined. at that time, the view was that is strategy was a response to business strategy, and that the is structure should mirror the business structure. the implementation of this alignment model was by means of methodologies, and focussed on large bespoke information systems (silvius 2007). for a review of these early models, see ward and peppard (2002), and peppard and breu (2003). an extensively cited model, and the one which still underpins much work on alignment today, is that developed by henderson and venkatraman (1993, 1999). they modelled four domains: business strategy, information technology strategy, information systems infrastructure and processes, and organizational infrastructure and processes, and defined alignment as being a dynamic, interactive process between all four. broadbent and weill (1993) suggested a similar, four domain model. luftman (1996) expanded on the four domains to describe twelve dimensions in more detail. chan, basing her analysis on a model similar to that of henderson and venkatraman, states that there are aspects of alignment that are still not well understood, particularly some of the informal organisational aspects. (chan 2002). the internal dynamics of alignment, again using a four domain model, has been set within the context of rapid change in the external business environments, represented as a punctuated equilibrium model (sabherwal, hirschheim et al. 2001). the effect on alignment of rapid change to the external environment has been modelled as a coevolutionary process (peppard and breu 2003; benbya and mckelvey 2006a). based on chaos and evolutionary theory, this work is still in early stages of development. other approaches include that of scott, who urges us to think in terms of the multiple business planning profiles which affect the approaches taken to, and problems and issues with, it planning. (scott 2005). current thinking on alignment can be categorised into two approaches. the first approach is that taken by luftman, who develops an alignment maturity model (luftman 2000). by assessing its australasian journal of information systems volume 15 number 1 december 2007 98 alignment maturity, an organisation can establish whether it is at the level of maturity it requires to reach its organisational goals, and can identify any enablers that it could improve, or inhibitors that it could minimise. the alignment maturity model is developed using earlier work on the four domains and twelve components of alignment (luftman 1996), and on its enablers and inhibitors (luftman and brier 1999). luftman develops six strategic alignment criteria, and a five level maturity process model which depends on the level of maturity of each of the six criteria. this is illustrated in table 1 below (luftman 2000). 1. intial / ad-hoc 2.committed 3.established/ focussed 4. improved/ managed 5. optimised communications increasing levels and pervasiveness of business /it understanding competency/ value increasingly comprehensive measures of efficiency, effectiveness and value governance increasingly formal and pervasive governance processes partnership relationships between is and business increasingly integrated scope and architecture increasingly integrated systems across the organisation and with partners skills increasingly shared risk and reward taking for is/it table 1: the alignment maturity model (luftman 2000) the second approach to alignment is that described by weiss, thorogood et al (2006), who have developed an alignment contingency model. this model draws on the work of porter, and of earl and feeny (earl and feeny 2000; porter 2001), and suggests that businesses vary with regard to their requirements for is alignment. if a business is merely using is as a technical resource, then its alignment requirements are minimial. where is is a business enabler, significant alignment is required, and where it is a strategic weapon their needs to be very high levels of integration. table 2 provides a summary of this approach: business objective dimension technical resource business enabler strategic weapon internal business – it integration low medium high external market it integration low medium high table 2 the alignment contingency model (weiss, thorogood et al. 2006) there are links between the two approaches: an organisation using is as a technical resource will be at a significantly lower level of alignment maturity than an organisation which is using is as a strategic weapon. implications of alignment for is and business managers. this paper does not seek to evaluate models of alignment. it takes the viewpoint that both the alignment maturity model and the alignment contingency model can help organisations in the complex and difficult task of defining and achieving alignment. australasian journal of information systems volume 15 number 1 december 2007 99 with regard to the first research question, namely how do is and business managers perceive their own and each others’ contribution to the alignment maturity of their organisation, table 3 indicates the areas within the alignment maturity model where is and business managers have a direct impact, and associates it with the requirements for each of the three alignment profiles in the alignment contigency model. table 3 is used as the basis to define the exact areas of enquiry for research question 1. they are as follows: • governance: position of is managers in organizational and governance structures • partnership: nature of the partnership between is and business managers • communication: levels of understanding of is by the business, and of business by is managers table 3 also indicates that there are major differences in requirements of is and business managers, depending on the alignment profile of their organisation. this emphasises the importance of the second research question 2. do is and business managers agree on the alignment profile of their organisation? this was explored by asking two specific questions: • level of shared understanding of the role of information systems • nature of understanding of the role of information systems alignment maturity model criteria implications for is and business managers effect of a contingency approach on the nature of the requirement governance reporting levels steering committees reporting levels of cio cio involvement with senior executive technical resource is manager does not need to report at a senior level business enabler cio partners with ceo strategic weapon cio is a valuable member of the senior management team partnership relationship/trust style business perception of it value nature of the partnership between is and business managers technical resource: no partnership required business enabler cio partners with ceo strategic weapon cio is a valuable member of the management team. communication: understanding of business by it understanding of it by business levels of understanding of business and is by is managers and business managers should be appropriate to organisational need technical resource: is and business need minimal awareness of each other’s activities business enabler senior managers need to be aware of and support is. knowledge transfer required between business and is. strategic weapon ceo needs to exploit it as a first order thinker. table 3 relationship between the benchmark questions for is managers and business managers, and the alignment maturity and alignment contingency models australasian journal of information systems volume 15 number 1 december 2007 100 in the section below, previous empirical research on each of the areas of enquiry is discussed. organisational and governance position of is managers improved business alignment is likely to occur in situations where the chief information officer (cio) works directly with the senior executive (preston 2004). based on a summary of the technological forces, executive attitudes and applications portfolio affecting the cio, it has been postulated that external factors have led to the evolution of the role from functional head, who must deliver on promises, to strategic partner, aligning it with the business, to business visionary, who drives strategy (ross and feeny 2000). empirical studies lend weight to the assertion that in some organisations the cio is a senior executive, and part of the senior management team. (stephens, ledbetter et al. 1992; gottschalk 1999; luftman 2000; enns, huff et al. 2001; enns, huff et al. 2003) is business partnerships several studies have discussed the importance of is and business managers’ relationships in understanding alignment (reich and benbasat 1996; rockart, earl et al. 1996; armstrong and sambamurthy 1999; reich and benbasat 2000; sabherwal and chan 2001; preston 2004; preston, karahanna et al. 2006)there has been some discussion regarding problems with the is manager – business manager relationship. taylor-cummings refers to a “deeply ingrained belief” that there is a culture gap between is professionals and other executives, and asserts that it is constantly referred to as a gap in understanding: is managers do not understand the business, and business managers do not understand technology (taylor-cummings 1998). other work suggests that culture is “an excuse for rather than a cause of ineffective application of it for business benefits and value” (peppard and ward 1999 p 29). assertions that the gap may be closing are based on circumstantial evidence such as job tenure rates, and backgrounds of cios (broadbent and kitzis 2005). luftman (2000) found that out of 25 fortune 500 companies, none viewed the relationship as either a valued partnership or valued service provider, with most seeing the relationship as primarily transactional, and several seeing it as a minimal or conflict ridden relationship. business and is managers understanding of is and business alignment is also dependent on the levels of knowledge of is and business managers. businesses achieve better outcomes from their is investments when business managers have a good understanding of information systems issues (armstrong and sambamurthy 1999; basselier, benbasat et al. 2003; broadbent and kitzis 2005). working with “it-savvy” business managers, the cio also needs to have good business knowledge in order to be able to put forward a visionary agenda (ross and feeny 2000; broadbent and kitzis 2005) despite this, pervan (1998) found that cios “see a greater need for the it education of senior management than do the senior managers themselves” (ibid p 101). luftman (2000) also found that the understanding of business by it ranked more highly than the understanding of it by business. is and business managers views on levels of alignment the question whether is and business managers agree on the level of alignment they have obtained has not been directly, quantitatively addressed in other studies. australasian journal of information systems volume 15 number 1 december 2007 101 is and business managers shared visions on the alignment profile in a study conducted ten years ago, pervan (1998) indicated that, while senior executives viewed information systems as important to their organisation, their views on the importance of key issues differ from those of is managers. chief executive officers were “greatly concerned with ‘the bottom line’” (ibid p 102), and were concerned that their data resources and software development were effective, and seen to be so. both ceos and is managers considered competitive advantage to be important to the organisation. is managers put greater emphasis on educating senior management, increasing the understanding of the is role, and aligning the is organization with the enterprise. previous research, then, emphasizes the key role that is and business managers have in achieving alignment, and indicates that there may be some important differences in perceptions between the two groups which may be affecting the ability of organisations to achieve alignment. method the research questions were explored both quantitatively and qualitatively. the quantitative data was constructed from previously validated questions used with the authors’ permission (preston 2004). these questions were incorporated in a postal survey consisting of two parallel questionnaires: one for the top is managers (cios) in an organisation, and a similar but shorter one for senior executives (ses). the question areas and their relationship to the research questions are summaries in table 4 below. due to the personal nature of some of the questions being asked, strict anonymity was preserved. however, it is possible to match the response of any individual cio with the response of the senior executive in their organisation. this is because the whole mailout package for each organisation was sent to the most senior is manager. the package consisted of a cio questionnaire, and a senior executive questionnaire contained in an unsealed envelope. there was a message on the envelope inviting the cio to pass it to a member of their senior executive, and ask them to return the survey sealed in the envelope. the cio combined this with his/her own completed survey and sent the whole package back to the researchers. despite the anonymity, some cross-checking of responses was possible, because respondents were also able to request copies of results, and indicate an interest in further study. while these requests could not be linked to individual survey results, they did allow some confirmation that the survey respondents were, indeed the cios within their organisations. in addition, respondents were asked for their job title, meaning that a visual scanning of the data also provided confirmation of the role of the respondents. the survey used a fairfax business research database (fairfax 2004), which contained the mailing details of cios for all firms with more than 1,000 employees, and for any particularly is intensive firm, within australia. cios had been defined as the most senior is executive in the organisation, and fairfax had checked the status of each entry. there were 5,386 cios so defined, and the survey was sent to all of them. australasian journal of information systems volume 15 number 1 december 2007 102 research area survey question areas cio survey se survey research question 1: how do is and business managers perceive their own and each others’ contribution to the alignment maturity of their organisation. reporting levels of the cio x involvement with the senior executive x frequency and type of communication with the top management team x cio demographics x 1. position of is managers in organisational and governance structures senior executive demographics (for comparison) x cio’s communication skills, credibility and track record x x 2. nature of the partnership between is and business managers cio’s communication and education techniques x cio’s knowledge of business and is x x 3. business and is managers levels of understanding of is and business respectively? senior executive levels of knowledge of applications of is x research question 2: do is and business managers agree on the alignment profile of their organisation? 4. is and business managers views on levels of alignment strategic alignment of systems with the business x x level of shared understanding of the role of information systems x x 5. is managers and business managers vision of the alignment profile in their organisation nature of shared understanding of the role of information systems x x table 4: quantitative survey: question areas and their relationship to the research questions in addition, qualitative data was obtained by interviews with survey respondents who indicated that they would be willing to take part in further investigations. four organisations were involved in the qualitative survey. the first (org 1) is based in australia and new zealand. it distributes plumbing and electrical supplies. its operations are divided into four business lines. interviews were conducted with the chief information officer and the chief financial officer. the second organisation (org 2) is global, highly dispersed and organised as a matrix. it operates on approximately a thousand sites, and is divided into ten regions which are spread through europe, africa, america, asia and australia. its headquarters are in geneva, switzerland. it specialises in inspection, verification, testing and certification and its business is divided into ten lines. the cio for the asia pacific region was interviewed. the third organisation (org 3) is the distribution arm of an australian publishing company. interviews were conducted with the it manager, the marketing manager, and the general manager for a subsidiary company specialising in merchandising. the fourth (org 4) was an australian public sector organisation, where an interview was conducted with australasian journal of information systems volume 15 number 1 december 2007 103 a senior director of operations. the purpose of the interviews was to deepen understanding of the findings of the quantitative survey, and to provide new insights into the area under investigation. all respondents were therefore asked to trace the “story” of one or more strategic activities which they considered significant, with minimal prompts from the interviewer. in addition, they were asked to comment on the relationship between themselves and their counterpart (is managers were asked about their relationships with business managers, and vice versa). quantitative results 365 cios responded to the survey – a 6.77% response rate. this is within the expected response rate for such senior executives (chan, huff et al. 1997; pervan 1998; enns, huff et al. 2003) of those 365 cios, 94 returned a survey from a matched senior executive. there was no significant difference (at a level of 0.05) between the responses of the 271 cios who returned a single survey, and the 94 who also returned a senior executive response, for any of the survey question areas requiring a response from both parties. appendix 1 gives detailed results of the quantitative survey. the findings for each of the research questions are given below: organisational and governance position of is managers the majority of cio respondents report at one level below the ceo. however, 33% report directly to the ceo, and only 8 % report at two levels or more below the ceo. it was possible to show that this represents a significant increase over time of cios reporting to the ceo. by comparing reporting levels with those in a 1989 survey, also conducted in australia, (watson 1990). in 1989 only 14% of top is managers reported to the ceo. 46% of is managers identified themselves as members of the senior executive, with a further 28% stating that they were frequently involved with the senior executive team. 20% had occasional involvement. only 4.5% said their involvement with the senior executive team was rare, and 2% said they had no involvement. 90% of cios had informal contact with the senior executive at least weekly, and of these 44% were in daily contact. formal communication was also frequent, with weekly or more frequent face to face meetings for 71% of respondents, telephone communication for 64% of respondents, and email for 84%. interestingly, the figures for socialising with the senior executive were low: only 11% of cios socialise on a weekly or more frequent basis with the senior executive. is business partnerships this was obliquely tested by asking senior executives about their cios’ interpersonal skills, credibility and track record, and asking cios the same questions about themselves. statistics regarding the cios’ communication skills were encouraging. most senior executives agree that their cio can read contentious situations accurately, act with tact, be an effective communicator, be articulate with the senior executive. cios were also regarded as having a good track record. on a scale from strongly agree (5) to strongly disagree (1) the median result was agree (4), and the mean was above 4 in all cases. notably, results were better than the cio self evaluations, which returned a mean of slightly below 4. matched pairs of scores (from a cio and senior executive in the same organisation) did not show a significant difference, when compared using the wilcoxon signed rank test. the cios were asked some additional questions regarding their communication and education techniques with the senior executive. they largely agreed that they provide insight into australasian journal of information systems volume 15 number 1 december 2007 104 emerging it, assist with computer literacy, and manager senior executive expectations regarding is. they do not tend to agree that they share common interests with the senior executive. business and is managers understanding of is and business a surprising finding from this survey was that senior executives rated their cio’s knowledge of the business of the firm, and of the industry’s practices generally, at a similar level to their cio’s knowledge of how to use is infrastructure to suit the business. on a scale of extremely well informed (5) to not very well informed (1) they returned medians of 4 (well informed) and means of 3.7 and 3.8 for both business and is knowledge. by contrast, cios believed they were better informed about is than about the business. however, the general level of ratings in all these areas was high. however, perhaps reflecting pervan’s findings that cios believe senior executives should have a higher level of understanding of is than senior executives believe they need, (pervan 1998), and corroborating luftman’s findings in this area (luftman 2000), the ratings of cios regarding the is knowledge of their senior executive were relatively low – between “somewhat informed” (2) and “well informed” (3). is and business managers views on levels of alignment the response to the questions on strategic alignment was remarkable both because they were so positive, and because there was such close agreement between the cio and senior executive. on a scale of strongly agree (5) – strongly disagree (1) the general response was between “agree” and “strongly agree”. there was no significant difference between the matched pair response of cios and senior executives. is and business managers shared visions on the alignment profile results indicate that there are some differences in the way in which senior executives and the cio understand alignment profiles. as the results of the wilcoxon signed rank tests in appendix 1 show, this is the one area where cios and “matched” senior executives show significantly different views. the research therefore explored this further by performing cross tabulations and regression analysis on the responses to the following three questions: 3. the cio/senior executive and i have a shared understanding of the role of is in our organisation 4. the cio/ senior executive and i have a shared understanding of how is can be used to increase productivity in our organisation’s operations. 5. the cio/ senior executive and i have a shared view of the role of is as a competitive weapon in our organisation the results were as follows: for senior executives: a = 1.06 + 0.43*b + 0.24*c; r2 = 0.43 for cios: a = 0.69 + 0.42*b + 0.39*c; r2 = 0.54 (a – role; b – productivity; c competitiveness) this analysis indicates that cios regard a shared understanding of the role of is to be as much about its potential to enhance productivity (reflecting the technical resource and/or business enabler profiles of alignment) as about its potential as a competitive weapon (reflecting the strategic weapon profile). senior executives, by contrast, see shared understanding of productivity as more significant than their shared understanding of is as a competitive weapon, thus indicating a vision australasian journal of information systems volume 15 number 1 december 2007 105 of is which is more focussed on is as a technical resource and/or business enabler than as a strategic weapon. qualitative results the qualitative data reinforced the findings of the quantitative data on the first four research questions, and produced different but possibly complementary findings to the fifth research question. findings are discussed question by question below. table 5 summarises the interviews and organisations, for ease of reference all is/it managers had senior roles, and overall responsibility for is within their organisations. while their duties and responsibilities varied, they were specifically asked to focus on strategic projects relevant to the whole organisation. organisation industry interviews conducted strategic initiative discussed org 1 pipelines, plumbing and electrical supplies (australia and nz) chief information officer (cio) chief financial officer (cfo) consolidation of erp implementation org 2 inspection, verification and testing (global) cio asia pacific implementation of a global wan. introduction of project management methodologies org 3 distribution arm of an australian publishing company it manager marketing manager merchandising manager web based marketing materials org 4 tertiary education senior operations manager erp implementation table 5: qualitative interviews organisational and governance position of is managers the cios of org 1 and org 2 had high status in the organisation, and specifically discussed new initiatives for it governance and the way in which this would further empower them. . org 1’s cio, reported, as is typical at “one below” the ceo, but also had a “very strong dotted line through to the ceo”, and had recently got promoted to the newly formed executive management team. org 2’s cio was a regional is manager in a highly dispersed organisation. he sat on the global it council, which had recently been given more powers to mandate project management methodologies. org 3’s it manager described himself as part of the executive team. org 4’s cio was also part of the senior executive team. is business partnerships interviews showed evidence of close working relationships between is and business managers. indeed, interviewees were often at pains to stress how good this working relationship was: “we see australasian journal of information systems volume 15 number 1 december 2007 106 very much eye to eye on just about everything” (org 1 cio of org 1 cfo). the operations manager of org 4 spoke of her cio as “a great mentor”. business and is managers understanding of is and business all is managers contextualised their discussions in terms of the business without prompting. none of the business managers interviewed mentioned problems with the is managers’ understanding of the business. indeed, they specifically referred to the business knowledge of is managers in some instances: “[the cio] could understand the business, listen to what the business managers were saying, but also understand the system requirements”. (org 1 cfo of org 1 cio). is managers, by contrast made clear that they needed business managers to have a good understanding of is issues, and that this was sometimes a problem. for example org 1’s cio talked of being able to work well with both the cfo and ceo of the organisation, because of their “it savviness”. however, for some other managers this was problematic. he linked this particularly with whether managers had grown up with is as part of the business; where they did not have this background they tended to have a limited view of his job as “keeping some hardware running”. is and business managers views on levels of alignment there were no major discrepancies between business managers and is managers views of whether their information systems were aligned with the business. managers agreed on the problem areas within their respective organisations, and also agreed on where things were working well. is and business managers shared visions on the alignment profile the most striking difference of opinion between business managers and is managers was on the time it would take to implement strategic initiatives. indeed, interviewees in three of the four organisations stress it was the major point of contention between the business and is: “he’s generally supportive but … he’s impatient for the pace at which we can do things. i don’t think we’ve ever had a major disagreement to be honest, but where we certainly have some disagreements is around the pace of change” (org 1 cio describing relationship with ceo) “he wants to fix a bunch of things and he’s frustrated with the pace of change because i’ve got the classic cio dilemma of i’ve got unlimited demand limited supply of resource and cost. the view is these guys are doing a great job but it’s just too slow” (org 1 cio describing relationship with line manager) “because of (the it manager’s) insistence, we actually prepared a very, very good document. however, the downside is that it’s taken a long time to do it” (org 3’s marketing manager) “she helped enormously by being able to understand that the business was the prime reason for having a system, rather than the system being the prime reason for having the business. ….now there were conflicts between her and me…and that is where something was happening at the business interface which was creating strife and causing all sorts of negative feedback. my requirement of those people was to fix it. ..if there was a bad problem which they said would only take six weeks to fix and i australasian journal of information systems volume 15 number 1 december 2007 107 said it had to be done in two days” (org 4’s operations manager describing relationship with is manager). conclusion regarding the first research question, “how do is and business managers perceive their own and each others’ contribution to the alignment maturity of their organisation”, results were encouraging. is managers’ organisational status has improved over time, most of them are heavily involved in governance mechanisms, and is managers and business managers also believe that their partnerships with each other work well. in addition, business managers perceive their is managers to have good levels of business knowledge. with regard to the second research question, “do is and business managers agree on the alignment profile of their organisation?” the findings were interesting, and more mixed. is and business managers agreed regarding the level of alignment in their business. however, three, interrelated problems have been identified. 1. business managers tend to view their organisation’s alignment profile as a “technical resource” or “business enabler”, improving productivity. unlike their cios, they do not necessarily envision the potential of is as a competitive weapon 2. business managers do not have the level of understanding of is that is managers believe they should have 3. business managers disagree with is managers regarding the time required to obtain advantages from strategic initiatives. interviewees indicated some connections between issues 1 and 2: managers who had little background experience of is, and therefore a poor understanding of its requirements were more likely to see is purely as a means of cost-cutting – productivity – rather than as a means of gaining more comprehensive competitive advantage. the literature on competitive advantage discusses the role of time lags in competitive advantage – an organisation implementing an it based strategic initiative sustains a competitive advantage during the time lag before other organisations implement the same initiative. see piccoli and ives for a review of this literature (piccoli and ives 2005). business managers who do not fully understand the competitive advantage that information systems can bring may also be unaware of the time lags involved. it is suggested that further investigation of is and business managers’ perceptions of the advantages to be obtained from is based strategic initiatives would be beneficial. the results could then be used to design a specific programme of is education for business managers, focussed specifically on a framework of the three different alignment profiles of technical resource, business enabler and strategic weapon. such an education programme would have the potential to improve alignment within organisations. australasian journal of information systems volume 15 number 1 december 2007 108 references armstrong, c. p. and v. sambamurthy (1999). "information technology assimilation in firms: the influence of senior leadership on it infrastructures." information systems research 10(4): 304 327. basselier, g., i. benbasat, et al. (2003). "the influence of business managers' it competence on championing it." information systems research 14(4): 317 336. benbya, h. and b. mckelvey (2006a). "using coevolutionary and complexity theories to improve is alignment: a multi-level approach." journal of information technology 21: 284-298. broadbent, m. and e. kitzis (2005). the new cio leader: setting the agenda and delivering the results, harvard business school press. broadbent, m. and p. weill (1993). "improving business and information strategy alignment: learning from the banking industry." ibm systems journal 32(1): 162-179. chan, y. e. (2002). "why haven't we mastered alignment? the importance of the informal organization structure." mis quarterly executive 1(2): 97-112. chan, y. e., s. l. huff, et al. (1997). "business strategic orientation, information systems strategic orientation, and strategic alignment." information systems research 8(2): 125150. ciborra, c. (1997). "de profundis? deconstructing the concept of strategic alignment." scandinavian journal of information systems 9(1): 67-82. earl, m. j. and d. f. feeny (2000). "how to be a ceo for the information age." sloan management review 41(2): 11-23. enns, h. g., s. l. huff, et al. (2001). "how cios obtain commitment to strategic is proposals: barriers and facilitators." journal of strategic information systems 10: 3 14. enns, h. g., s. l. huff, et al. (2003). "cio lateral influence behaviours: gaining peers' commitment to strategic information systems." mis quarterly 27(1): 155 176. fairfax (2004). fairfax business research marketbase, fairfax business media pty ltd. gottschalk, p. (1999). "strategic management of is/it functions: the role of the cio in norwegian organisations." international journal of information management 19: 389-399. gregor, s., w. fernandez, et al. (2004). achieving value from ict: key management strategies. henderson, j. and n. venkatraman (1992). strategic alignment: a model for organizational tranformation through information technology. transforming organisations. t. a. kochan and m. e. useem. oxford, oxford university press: 97-117. henderson, j. and n. venkatraman (1993, 1999). "strategic alignment: leveraging information technology for transforming organisations." ibm systems journal 32 (1993); 38 (1999)(1 (1993); 2&3 (1999)). henderson, j. and n. venkatraman (1994). strategic alignment: a model for organizational transformation via information technology. information technology and the corporation of the 1990s: research studies. t. j. allen and m. s. scott morton. australasian journal of information systems volume 15 number 1 december 2007 109 hirschheim, r. and r. sabherwal (2001). "detours in the path toward strategic information systems alignment." california management review 44(1): 87-108. hollands, m. (2005). delivering it's contribution: the 2005 cio agenda, a gartner group presentation to the australian computer society, australia. luftman, j. (1996). competing in the information age: practical applications of the strategic alignment model. new york, oxford university press. luftman, j. (2000). "assessing business-it alignment maturity." communications of the association for information systems 4. luftman, j. (2005). "key issues for it executives 2004." mis quarterly executive 4(2): 269-285. luftman, j. and t. brier (1999). "achieving and sustaining business-it alignment." california management review 42(1): 109-122. peppard, j. and k. breu (2003). beyond alignment: a coevolutionary view of the information systems strategy process. twenty-fourth international conference on information systems. peppard, j. and j. ward (1999). "'mind the gap': diagnosing the relationship between the it organisation and the rest of the business." journal of strategic information systems 8: 29-60. pervan, g. (1998). "how chief executive officers in large organizations view the management of their information systems." journal of information technology 13: 95-109. piccoli, g. and b. ives (2005). "review: it-dependent strategic initiatives and sustained competitive advantage: a review and synthesis of the literature." mis quarterly 29(4): 747776. porter, m. e. (2001). "strategy and the internet." harvard business review: 62-78. preston, d. (2004). shared mental models between the chief information officer and top management team: towards information systems strategic alignment, university of georgia. phd: 178. preston, d., e. karahanna, et al. (2006). "development of shared understanding between the chief information officer and top management team in us and french organizations: a crosscultural comparison." ieee transactions on engineering management 53(2): 191 206. reich, b. h. and i. benbasat (1996). "measuring the linkage between business and information technology objectives" mis quarterly, 20:1, pp. 55-81 reich, b. h. and i. benbasat (2000). "factors that influence the social dimension of alignment between business and information technology objectives." mis quarterly 24(1): 81-113. rockart, j. f., m. j. earl, et al. (1996). "eight imperatives for the new it organisation." sloan management review. ross, j. w. and d. f. feeny (2000). the evolving role of the cio. framing the domains of it management: projecting the future through the past. r. w. zmud and m. f. price. cincinatti, ohio, pinnaflex educational resources: 385-402. sabherwal, r. and y. e. chan (2001). "alignment between business and is strategies: a study of prospectors, analyzers, and defenders." information systems research 12(1): 11 33. sabherwal, r., r. hirschheim, et al. (2001). "the dynamics of alignment: insights from a punctuated equilibrium model." organization science 12(2): 179-197. australasian journal of information systems volume 15 number 1 december 2007 110 scott, g. m. (2005). "still not solved: the pertistent problem of it strategic planning." communications of the association for information systems 16: 904-936. silvius, g. a. j. (2007). business and it alignment in theory and practice. proceedings of the 40th hawaii international conference on systems sciences 2007. hawaii. stephens, c. s., w. n. ledbetter, et al. (1992). "executive or functional manager? the nature of the cio's job. ." mis quarterly: 449 467. taylor-cummings, a. (1998). "bridging the user-is gap: a study of major information systems projects." journal of information technology 13: 29-54. ward, j. and j. peppard (2002). strategic planning for information systems: third edition. chichester, john wiley and sons ltd. watson, r. t. (1990). "influences on the is manager's perceptions of key issues: information scanning and the relationship with the ceo." mis quarterly: 217-231. weiss, j. w., a. thorogood, et al. (2006). "three it-business alignment profiles: technical resource, business enabler, and strategic weapon." communications of the association for information systems 18: 676-691. australasian journal of information systems volume 15 number 1 december 2007 111 appendix 1: statistical details from survey min median max mean sd z asymp sig cios communication skills, credibility, strongly agree(5) – strongly disagree (1) cio 2 4 5 4.0 0.6 can accurately read contentious situations se 2 4 5 4.0 0.8 -0.425 0.671 cio 2 4 5 3.9 0.7 can act with tact se 2 4 5 4.0 0.9 -0.019 0.985 cio 3 4 5 4.2 0.5 can develop good rapport se 2 4 5 4.1 0.8 -1.042 0.298 cio 2 4 5 4.0 0.6 is an effective communicator se 1 4 5 4.0 1.0 -0.009 0.992 cio 1 4 5 4.1 0.6 is articulate with the senior executive se 2 4 5 4.2 0.7 -1.691 0.091 cio 1 4 5 4.0 0.7 has high credibility with the senior executive se 2 4 5 4.1 0.8 -1.643 0.100 cio 1 4 5 4.2 0.7 has a good track record se 2 4 5 4.3 0.7 -1.173 0.241 cios communication techniques with se, strongly agree(5) – strongly disagree (1) shares interest with se cio 1 3 5 2.9 1.0 provides insights on new it cio 1 4 5 3.9 0.7 assists computer literacy cio 1 4 5 3.7 0.8 educates on is cio 1 4 5 3.9 0.7 manages expectations cio 1 4 5 4.1 0.7 gives realistic expectations cio 1 4 5 4.3 0.7 not applicable cios knowledge of business and is extremely well informed(5) – not well informed (1) cio 2 4 5 3.6 1.0 firm’s products se 1 4 5 3.8 0.9 -2.043 0.041 cio 2 4 5 3.6 0.9 industry’s practices se 2 4 5 3.7 0.9 -1.144 0.253 cio 1 3 5 3.3 1.0 firm’s competitors se 1 3 5 3.2 1.0 -1.112 0.266 cio 1 3 5 3.0 1.1 competitors application of is se 1 3 5 3.0 1.1 -0.467 0.641 cio 1 4 5 4.0 0.9 how to utilise is infrastructure to suit the business need se 2 4 5 3.8 1.0 -1.072 0.284 cio 1 4 5 3.6 0.8 how to identify emerging is for firm se 1 4 5 3.6 1.0 -0.153 0.878 cio 1 4 5 3.6 1.0 how to guide the firm’s decisions on timing and level of investment se 1 4 5 3.6 1.0 -0.699 0.485 australasian journal of information systems volume 15 number 1 december 2007 112 senior executive levels of knowledge of is extremely well informed(5) – not well informed (1) is limitations cio 1 3 5 2.8 0.9 next generation of is cio 1 2 5 2.0 0.7 competitors’ is cio 1 2 5 2.3 0.8 level of shared understanding of the role of is, strongly agree(5) – strongly disagree (1) cio 1 4 5 3.5 1.0 shared understanding of role of is se 2 4 5 4.3 0.7 -6.539 0.000 cio 1 3 5 3.3 1.0 shared view of is as a competitive weapon se 3 4 5 4.0 0.7 -5.540 0.000 cio 1 4 5 3.6 0.9 shared understanding of is contribution to productivity se 2 4 5 4.1 0.7 -4.309 0.000 cio 1 4 5 3.5 0.9 common view of prioritisation se 2 4 5 3.8 0.8 -2.763 0.006 cio 1 3 5 3.3 0.9 common language se 2 4 5 4 0.8 -4.997 0.000 cio 1 4 5 3.9 0.9 cio uses business terminology se 2 4 5 3.9 0.9 -0.577 -1.528 cio 1 4 5 4.1 0.8 cio avoids jargon se 2 4 5 3.9 0.7 -1.528 0.564 strategic alignment of systems with the business strongly agree(5) – strongly disagree (1) cio 1 4 5 4.0 1.0 is strategy congruent with business strategy se 2 4 5 4.0 0.8 -0.501 0.616 cio 1 4 5 3.8 1.0 is planning decisions tightly linked to organisation’s strategic plan se 2 4 5 3.8 0.9 -0.187 0.852 cio 1 4 5 3.7 1.0 business and is strategy closely aligned se 2 4 5 3.8 0.9 -0.374 0.708 requirements evolution and reuse using the systems engineering process activities (sepa) k. suzanne barber, thomas j. graser, paul s. grisham, stephen r. jemigan the laboratory for intelligent processes and systems the university of texas at austin austin, tx 78712 voice: 512-471-6152 fax: 512-471-3316 barber@mail.utexas.edu abstract as more organizations attempt to reuse previous development efforts and incorporate legacy systems, typical software development activities have transitioned from unique ground-up coding efforts to the integration of new code, legacy code, and cots implementations. this transition has brought on a whole new set of development issues, including resolving mismatches between integrated components and tracing legacy and cots components to requirements. this paper presents the systems engineering process activities (sepa) methodology, developed to address these and other issues in current software development practices. sepa aids the reuse and integration process by focusing on requirements integration and evolution, while maintaining traceability to requirements gathered from domain experts and end users. the sepa methodology supports the development process by promoting requirements analysis prior to design, separation of domain-based and application-based (i.e. implementation-specific) requirements, and evaluating system component suitability in terms of domain and application requirements. the paper also presents an example illustrating the application of sepa in the emergency incident response domain to facilitate requirements management and foster requirements reuse.2 introduction despite the introduction of software development methodologies and case tools to help alleviate the "software crisis," the development and maintenance of software is still too expensive and error prone. effective reuse from previous software development efforts has the potential for increasing software productivity and quality an order of magnitude (caldieri, gianluigi, & basili, 1991). however, current methodologies do not emphasize the reuse of artifacts other than code (analysis or design) from previous implementations. while code reuse typically occurs only at lower-level system design artifacts, analysis and design reuse often results in whole collections of related artifacts being reused (department of defense, 1996). methodologies can further increase odds for reuse by identifying potential reuse at all system levels (e.g. system, component, code function) and among all participating elements (e.g. new development, legacy systems, and commercial off the shelf (cots) components). reuse of requirements provides a number of benefits, including the following: 1. motivation for selection of components: requirements guide the selection of optimal components for reuse. when requirements are transferred between development efforts, the rationale behind the original component selection decision is made available to the system designer. 2. context for reuse decisions: requirements trace back to information gathered from domain experts and system users. requirement-based reuse decisions are set in the context of domain processes or specific implementation needs. 3. parametric constraints: requirements come in many forms, including parametric constraints (i.e. the system delivered must run at speed x) as well as general guidelines (e.g. the system's interface should be user friendly) and domain tasks and processes. parametric constraints allow a static evaluation to narrow the field of available components. the leveraging of legacy systems that were not designed with reuse in mind comprises a large part of the reuse motivation. however, the reusability of old systems is often difficult to evaluate by direct inspection. perhaps the rework effort can be determined through code evaluation and system documentation. however, it can be difficult to determine the relative adherence of the old implementation to newly gathered requirements. when originally under development, requirements were likely gathered and analyzed for satisfaction by the old implementation. given that a requirements effort has already occurred, an alternative is to determine which of the old requirements can transfer to satisfy new requirements rather than take on a fresh evaluation of the old system. 2 this research was supported in part by the texas higher education coordinating board advanced technology program (atp #003658452) and the defense advanced research projects agency (darpa). 75 section 0 provides a model of requirements evolution and a list of features which are strongly desired in a methodology intending to support requirements evolution and reuse. section 0 introduces the system engineering process activities (sepa) methodology and positions its distinguishing characteristics among these features. section 0 provides an example illustrating the application of sepa in the incident response domain.3 section 0 concludes with a discussion highlighting selected sepa contributions to requirements evolution and reuse evident in the example. requirements evolution and necessary methodological features in this section, we discuss some essential features of good software engineering methodologies to support requirements evolution and reuse, with an emphasis on activities during requirements gathering and analysis. figure 6 depicts the requirements evolution scenario and the traceability to implementations under a comprehensive development methodology. old requirements captured, analyzed, and verified are eventually linked to components in the implementation, old implementation. as new requirements are captured, analyzed, and verified, they are mapped to old requirements, thereby indirectly referencing components from the old implementation. the mapping from new requirements to old requirements may not cover the complete set of new requirements and some of the old requirements may not be relevant in the new requirements. as the old implementation may only partially address the new requirements, there is a need for evolving (modifying) the old implementation. the remaining new requirements spawn development of new components (new implementation), which when integrated with components from the old implementation, result in a delivered product. requirements mapping old requirements new requirements o; 0°old \^f ' • implementation traceability links bnew implementation integrated components delivered product figure 6: requirements evolution and traceability the difficulty in implementing the model in figure 6 lies in matching requirements to implementations. to effectively render this mapping, implementations must be expressed in a meta-specification composed of the same language as the requirements themselves. furthermore, this meta-specification should operate at an abstraction level that allows it to be independent of implementation specifics (e.g. component a requires an integer value while component b provides a float value), yet be expressive enough to capture a broad range of requirements (e.g. component must perform domain task x on operating system y with throughput z). defining this meta-specification at the appropriate level of abstraction along with careful management of traceability links4 from requirements to implementations will encourage the reuse of whole collections of related artifacts from analysis and design. given the requirements evolution process outlined above, the development cycle becomes an issue of integration on two levels: 3 detailed discussions of the operations of sepa tools are beyond the scope of this paper. 4 traceability links provide a two way path between artifacts and symbolize the refinement or translation of information. for instance, a requirement would be connected to design elements that help to fulfill the requirement. the two way path permits navigation (i) to other artifacts from which a particular artifact is derived and (ii) to other artifacts derived from a particular artifact. 76 1. mapping, evolving, and integrating requirements previously gathered, analyzed, and verified into new requirements and 2. integrating technologies inferred by the traceability links from integrated requirements. figure 7 illustrates this two-level integration problem. requirements are derived from an array of sources: a previous development effort, available media (e.g., system documentation), multiple domain experts, and multiple system users. requirements elicited from multiple sources often overlap and possibly contradict. yet understanding requirements involves incorporating multiple viewpoints, involving the capture, analysis, and resolution of many ideas, perspectives, and relationships at varying levels of detail (kotonya & sommerville, 1997). once integrated, requirements guide the selection of appropriate cots components, legacy systems, and newly developed implementations. multiple system configurations may satisfy the same set of requirements, requiring further tradeoff studies for evaluation. resulting selections are then integrated into a cohesive solution two-level integration problem previous development efforts v requirements h 4 fc 4 k •* k.w ^ f ^ f -^ pvarious media used different different for requirements domain system capture experts users y ^ techno w .j »> r ^ logy •* ^ ' i -* > across multiple sources integrated requirements to technology matches r various legacy various cots infrastructures implementations developers across multiple technology solutions figure 7: two-level integration problem opportunities for successful reuse, as suggested by the model in figure 6, are increased through the use of a methodology that supports the following six features. • traceability: throughout the requirements lifecycle, emphasis should be placed on maintaining accurate traceability. in (gotel & finkelstein, 1994), gotel reported the results of a survey of more than 100 software engineering practitioners that indicated "most of the problems attributed to poor requirements traceability were found to be due to the lack of (or inadequate) pre-requirements specification (pre-rs) traceability." contrary to the motivation of these results, they noted that existing tools mainly supported post-rs traceability only. moreover, palmer notes that current tools provide little support for identifying and maintaining traceability information (palmer, 1997). methodologies must provide the emphasis for the development of tools that support the identification and evolution of traceability information from requirements elicitation to requirements maintenance. • support for multiple viewpoints: the recognition of 'viewpoints' in software engineering is first credited to the core methodology (mullery, 1979) in the late 70s. since that time, it has become widely accepted that a complete requirements gathering effort must seek out the individual needs of the various stakeholders that will interact with the system to be developed (sommerville, sawyer, & viller, 1998). often the requirements gathered from these 'viewpoints' will overlap and possible conflict. compromises are made in every development process to arrive at a single set of requirements for the project. these compromises may be implicit, un-documented, and performed without the input from the original stakeholders. methodologies that hope to enable reuse of requirements and derivative artifacts (e.g., code) must accurately identify the source of these requirements (see traceability above). this need requires that the methodology acknowledge the existence of multiple viewpoints and guide the viewpoint reconciliation process. • computational requirements representation: typical requirements management approaches rely on tracing text fragments to design and development artifacts (chipware, 1999; qss, 1998; technologies, 1999). this natural language representation supports only text string searches and provides little ability for structured requirements types and complex relationships among various requirements types (palmer, 1997). the end result is a greater difficulty associating applications features to originally stated requirements. 77 • separation between domain and application requirements: to improve chances for reuse, software methodologies often suggest that "what" a system must do should be modeled independently of "how" the system should be implemented. the presumption is that "what" is done in a domain changes far less often than "how" it is accomplished, especially given frequent changes in technology (tracz, 1991; tracz, 1993; tracz, 1995). despite this advice, other requirements management tools do not explicitly recognize this separation. • comprehensive process support for requirements refinement and management: numerous requirements management tools are available for requirements analysis and refinement, taking different approaches to requirements representation (chipware, 1999; qss, 1998; technologies, 1999). even more tools support object-oriented design after requirements have already been elicited, validated, and merged (ses, 1999; verilog, 1997). some tool suites attempt to bridge requirements identification with subsequent object-oriented analysis and design (rational, 1998). however, no current tools adequately support the requirements refinement process with automated support for traceability maintenance from elicitation through maintenance (gotel & finkelstein, 1994; palmer, 1997). without this integration, the transition from requirements capture, to requirements synthesis, to class derivation, to system design becomes excessively difficult. in the absence of a comprehensive process support for requirements refinement and management, some requirements may be ignored while others may not be maintained in the long-term after implementation or as the domain scope changes. • support for requirements volatility: requirements change even during the initial development of a system. changes may be promoted by more accurate requirements elicitation, changing user needs, changes in the relative importance of various user's requirements, and technology changes. dobson et. al. (dobson, blyth, chudge, & sterns, 1992) note that requirements can also be driven by the downstream development process. failure of the methodology to support changing requirements will result in requirements that are "incomplete, inconsistent with the new situation, and potentially unusable" (christel & kang, 1992). the prospect that requirements might be used (reused) beyond the current development effort underscores the need for the methodology to recognize and support requirements volatility throughout the project's lifecycle. in this section, a model for requirements evolution was presented and six necessary methodological features were identified. in the following section, a methodology is proposed that emphasizes these features so that the requirement evolution and reuse model can be realized. the sepa methodology the systems engineering process activities (sepa) methodology builds upon the dssa approach (tracz, 1991) and popular object-oriented methodologies by distinguishing between domain functional requirements and implementation-specific requirements, defining a domain architecture, identifying applicable technology solutions, and finally utilizing the domain architecture and technology solution specifications to assist reuse evaluations and design trade-off decisions. the domain architecture represents an object-oriented partitioning of domain functional and data requirements without designating a particular implementation. to realize an actual system implementation, the system designer must select appropriate technology solutions that satisfy both domain requirements and application implementation-specific requirements. in this section, activities that comprise the sepa methodology are discussed. sepa activities are described by the sepa funnel structure as shown in figure 8, emphasizing the continuous gathering, narrowing, and refinement of user requirements into a component-based system design specification. requirements require refinement for a number of reasons, including the need to: (i) merge input from multiple sources, (ii) discard irrelevant information, and (hi) distinguish between inputs relating to system implementation requirements and those relating to general domain knowledge. the sepa methodology encourages commitment to requirements gathering and subsequent analysis, emphasizing: (i) requirements analysis prior to design, (ii) the separation of domain and application requirements, and (iii) requirements analysis for component-based development. although the funnel illustration suggests a one-directional flow of knowledge, sepa is a highly iterative design process, where new knowledge is continuously added to the top of the funnel through knowledge acquisition from multiple viewpoints: end-users, domain experts, system integrators, developers, etc. 78 knowlege acquisition unified knowledge model *^_ ^ application infrastructure requirements ^ model (airm) ^ . • • • . ' . ' infrastructure ref̂ ch. classes (iracs) v ; tfr~ • • registered technology solutions system design specifications knowledge acquisition knowledge modal creation & synthesis paration current technology, solutions appli reauire reqmts derivation system configuration evaluation knowledge acquisition transcripts domain model technology solutions repository domain reference architecture classes (drags) figure 8: the sepa process funnel the sepa methodology encourages gathering requirements from a diverse group of stakeholders (end-users, domain experts, developers, etc.) that are representative of all the contributing perspectives. the first step in the knowledge acquisition effort is to identify the relevant viewpoints from which domain knowledge will be elicited. to support the knowledge synthesis and resolution process, the domain viewpoints should be organized hierarchically. next, domain experts must be selected and associated with a domain viewpoint. ideally, each knowledge acquisition (ka) session will be conducted with a single stakeholder, and the artifacts generated as a result of that session will reflect knowledge from a single domain viewpoint. the preferred approach to knowledge acquisition is to elicit information in the context of scenarios of operation (mcgraw & harbison, 1997), whether the scenario relates to the entire domain, current project, or past projects. in generating the session transcripts, the knowledge engineer should not worry about interpreting or filtering the acquired knowledge. requirements modeling and validation will occur at a later phase, and the session transcripts will serve as an anchor for all subsequent requirements traceability. knowledge model creation and viewpoint resolution following the ka sessions, the knowledge that was elicited is represented in graphical knowledge models (kms), including process traces, collaboration diagrams, task decompositions, etc. during knowledge modeling, the ke is often required to make interpretive decisions, such as the relevance of certain details, the proper levels of abstraction, and what to do about inconsistent facts. the knowledge models created from a given knowledge acquisition session are returned to the stakeholder who was interviewed for validation. once the knowledge models from a particular session have been validated by the stakeholders, the models must be integrated with other validated domain knowledge. these models reflect the viewpoint, level of abstraction, and terminology of the stakeholder from whom the information was acquired. to be useful to developers, the viewpoints of the individual stakeholders need to be resolved together to form a homogeneous set of system requirements. the primary deliverable of the knowledge synthesis activity is the unified km (ukm) synthesized from the viewpoints of multiple stakeholders. 79 the representation used for the synthesized model must be expressive enough to contain all of the km types. while uml is a popular meta-language appropriate for certain task and data modeling, additional flexibility can be gained through the use of more abstract data representations (e.g., semantic nets or first-order logic). elements and assertions from the individual kms must be mapped into the common target representation of the synthesized model (ukm) and conflicts resolved. the difficulty of this conflict detection and resolution process is reduced if mapping proceeds through three phases. first, the main concepts from all kms are mapped into the synthesized model so that the model contains the equivalent of a set of domain dictionary entries. matching synonyms and disambiguating homonyms provides a more stable foundation on which later mappings can be made. the next phase assures that assertions regarding concept types are consistent. for example, a concept may be used as a "task" in one knowledge model but refer to a "state" in another model. since the sepa methodology asserts that the "task" and "state" concept types conflict, the knowledge engineer must either choose a single type or divide the original concept. during the final phase, relations between concepts are merged into the developing model. between each phase, the ukm's syntactic rules help to identify conflicts, and the resolution of these conflicts is recorded to ensure traceability. knowledge models are combined in a stepwise fashion up the viewpoint hierarchy. first models relating to a specific session or stakeholder are merged. next, all models related to a specific viewpoint in the domain viewpoint hierarchy are merged. at the end, the synthesized ukm represents the compromises between the disparate viewpoints in the domain. while model synthesis's primary deliverable is the ukm, the intermediate models are also of value. for instance, if an intermediate model is the result of all the knowledge acquisition sessions conducted with a particular stakeholder viewpoint (e.g. physician, nurse, etc.), then that model represents the domain from that viewpoint. these models, a byproduct of the synthesis process, represent new, viewpoint-based models. viewpoint models are especially useful when developing components, such as guis, that are specific to a single type of domain user. separation of requirements although the goal of a particular ka session may be to focus on the elicitation of one type of knowledge over another, a typical ka session may gather a broad spectrum of information. the information gathered may contain domain knowledge, specific application requirements, and/or information about specific technologies currently in use (e.g., legacy systems). in an ideal situation, the ke would conduct separate ka sessions with a domain expert or end-user for each of these types of information. however, domain experts and end-users typically do not have this abstracted view of their work and may find it difficult to provide such information. thus the information captured in the ukm contains both domain-specific and application-specific information. the sepa methodology emphasizes the separation of requirements associated with a particular application from the requirements applicable to the domain in general. this emphasis on separation was made in an effort to facilitate reuse of domain analysis and design artifacts. the domain requirements are the set of functional or data requirements shared by all or some of the respective viewpoints within a domain. the basic rule for identifying domain requirements is to locate tasks and data identified by the stakeholders. this information is gathered together into the domain model (dm). application requirements reflect the details of a system implementation or the implementation-specific requirements of a particular system client. in general, application requirements will be performance constraints on domain entities (tasks, roles, resources, etc.) or constraints on implementation choices. these requirements are gathered together into the application infrastructure requirements model (airm). performance constraints are quantitative requirements on the domain entities irrespective of implementation strategy, e.g., speed constraints, numerical accuracy, etc. implementation or delivery constraints specify user requirements on the implementation of the system. for instance, a delivery constraint might be that the payroll system must be developed for a specific hardware platform and the system must interact with a particular legacy application. user interface requirements are also an example of implementation constraints. for instance, a particular application may be required to support federal accessibility guidelines, or adhere to the interface defined in a prototype model. during knowledge acquisition, knowledge engineers are often given information about specific technology solutions, such as cots tools or legacy applications. a technology specification template is available to the knowledge engineer to assist in gathering application information during this phase. this structure of the software specification template is based on the ieee 830-1993 recommended practice for software requirements specification and our extensive interaction with software developers and integrators on prior/current development efforts. domain requirements class derivation the domain reference architecture (dra), is comprised of a collection of domain-based component classes derived from information in the domain model. the domain ra provides a key link between analysis and implementation, reflecting domain information found in the domain model and providing developers with a 80 template for identifying and developing new technologies for particular implementations. the domain ra must be completely domain-specific and highly flexible for building similar systems in the future. this flexibility is achieved by describing domain reference architecture classes (dracs) in terms of their functional requirements. in the process of creating the ra, the functional, procedural domain model is transitioned to an object-oriented reference architecture composed of technology independent dracs applicable to a family of applications in the domain. while it is feasible to partition dracs along either functional or object-oriented boundaries, an object-oriented approach promotes qualities such as reusability, faster development, flexibility, scalability, maintainability, and better correspondence to the domain being modeled (graham, 1995). more importantly, by following an object-oriented approach, the delineation of dracs based on what services must be performed allows for flexibility and accommodates changes in how a task is performed when dracs are instantiated in a particular system design. a drag is assigned responsibilities based on domain tasks and represented by three categories of information (graser, 1996). the declarative model (d-m) defines the attributes and services contained within and offered by a drag. the behavioral model (b-m) defines the states of a drag, the transitions between those states, and the events which affect transitions. the integration model (i-m) defines the constraints and dependencies between dracs, capturing integration requirements, potential subsystem compositions, and user integration preferences. ra derivation is an iterative process, where successive iterations represent increasing coverage of domain information and greater refinement of the ra based on architectural goals. typical architecture goals include extensibility, comprehensibility, and maintainability, often reflecting the benefits associated with object-oriented approaches. rare guides the architect towards these goals through the application of architecture heuristics, architecture metrics, and heuristic strategies. these are described as follows: • architecture heuristic: a "rule of thumb" compiled from expert experience on past projects which assists the architect in making rational decisions in defining dracs. one well-known object-oriented heuristic recommends reducing coupling among dracs to encourage reuse (riel, 1997). • architecture metric: a measurement of a particular characteristic of an ra which provides an indication whether the architect adhered to a given heuristic. continuing with the previous example, the drag inheritance hierarchy and/or number of messages passed among dracs would provide some evidence as to the degree of coupling in the ra. • heuristic strategy: a step-by-step procedure (sequence of actions) used to apply a given heuristic. following the "reduce coupling" example, a strategy might explicitly state, "move service si from drag dl to drag d2" to eliminate the need to exchange data between dracs dl and d2. the resulting collection of domain reference architecture classes (dracs) comprise a domain-based, object-oriented reference architecture which is traceable to stakeholder requirements through the functional requirements defined in the dm, provides a blueprint for developers, aids in domain understanding, and identifies rules of composition among architecture classes. 81 technology component 1 internet software software 1 system software 1 application software middleware | hardware 1 computing platform peripheral people 1 adapter cards 1 11 1 database servers application server web servers system mgmnts/w internet tools operating system device drivers utility programs assembler debugger compiler runtime environment runtime library end-user cots — 1 word processiny spreadsheet database graphics desktop publishing presentation communication network or messaging middleware database middleware copy management/ replication middleware static display dynamic display modem _ storage device dynamic physical, graphical and sound i/o figure 9: subset of irac hierarchy application requirements classes the infrastructure reference architecture (ira) models classifications of technology solutions and associated classifications of infrastructure required to support the different kinds of technology solutions. these technology solution classifications span from software, to hardware equipment, to people. most software systems require such basic service support as operating systems, database management systems, and middleware (braun, 1999). the services provided by those kinds of components e.g., resource/data management, communication, coordination, and user-interface comprise the system infrastructure and can be constrained by the system stakeholders. for instance, a software system deployment site may require that the system communicate via a wireless network. the system will require specific types of networking hardware, and the networking middleware will have to be configured for broadcast, rather than point-to-point organization. since these types of requirements refer to and constrain specific implementations, these requirements are not captured in the domain reference architecture. system delivery constraints specifying implementation and installation constraints were previously identified and represented in the application infrastructure requirements model (airm). maintaining traceability links to the original source, delivery constraints are mapped to instances of infrastructure reference architecture classes (iracs). iracs model technology solutions as domain-neutral classes of implementations. when technology solutions are registered, their implementation details are specified in terms of iracs and are evaluated against the delivery constraints. a high-level view of the iracs is presented in figure 9. in addition to the classes themselves, the ontology contains relationships between the classes. for instance, a class of applications identified as interpreted programs, require the services of a runtime environment to execute. networking middleware requires a network adaptor to connect components on different computing platforms. the technology registration process handles instantiating the specifications of technology solutions with regard to the iracs and modeling the technology implementation and infrastructure constraints. component technology registration component technology registration is the process of modeling technology solutions (e.g. legacy systems, cots applications, application systems or components under development, or planned systems or components) 82 by mapping the technology's coverage of functional requirements and describing how the technology is implemented. since a wide array of technology exists (namely, technology an end-user recognizes as delivering some functional services and infrastructure-related technology), sepa distinguishes the two during the registration process. an "end-user application" is a specific technology solution which can be directly mapped to the domain requirements it provides. registration of an end-user application begins by identifying a drag which approximates the basic domain functionality of the application. ideally, the functionality of the end-user application will align very closely with the structure of the drag in the domain ra, since this will allow systems to take advantage of the object-oriented features of the reference architecture design. an application is registered to a drag if it offers any service or manages any data or events in the specification of that drag. however, registration can be handled on a per-service or per-data-element basis as well. after the domain requirements have been covered, it is necessary to acquire information about the way in which the application has been constructed. an end-user application is composed of one or more infrastructure technologies. knowledge about the infrastructure requirements of the end-user application are represented as instances of respective iracs. an end-user application might be composed of an executable software application, a database, an operating system, and a computing platform with a large hard disk. each of these entities would be represented as infrastructure application technology which must be configured to implement the end-user application. system configuration evaluation as the system designers and integrators attempt to determine which application technology or systems can be deployed in a particular system configuration for a specific user installation site, they must determine the ability of applications to satisfy domain-related functional and data requirements specified in the domain ra, the state of the applications (legacy custom system, cots application, under development, planned), and the ability of the applications to meet delivery expectations with regard to implementation. a system design specification (sds) is constructed for a site by first selecting the subset of the domain ra (dracs or selected services within dracs) which must be implemented for that site. for convenience, this subset is labeled as a "version" of the dra. as defined in sepa, a "site" is a group of physical computing platforms or locations which collectively must provide the services in that version. once dra services have been selected, end-user applications registered to those services can be identified. knowledge about application infrastructure and relationships to dra component classes guide decisions in selecting how a domain service in the dra can be satisfied by candidate technology solutions. for example, dra services are related based on the events and data they exchange; these service dependencies are represented in respective drag integration models. the selection of a particular service for an installation may infer additional required services based on service dependencies. an application architecture is created for the site which covers the desired subset of the domain requirements in the domain ra version. depending on the alignment of the selected end-user applications to their corresponding drag boundaries, the application architecture may or may not demonstrate the beneficial design characteristics of the domain reference architecture. if the application architecture is satisfactory, then the underlying system infrastructure can be configured from infrastructure applications to create a site implementation architecture. several dimensions of evaluation are possible. first, the basic installation requirements of the end-user applications can be checked for satisfaction. second, a preliminary integration report can be generated, driven by the topology of the application architecture and utilizing the infrastructure details in the implementation architecture. the integration reports can identify potential integration problems and suggest areas for further testing. finally, the implementation architecture can be evaluated to determine if application and site requirements instantiated in the iracs have been satisfied. the solutions may be chosen based on any number of design trade-off concerns (e.g. cost, availability, ease of implementation, etc.). the resulting system design specification contains a set of application component solutions, system infrastructure components (e.g. cots tools, middleware, and hardware), and the integration dependencies between respective end-user applications and between end-user applications and the infrastructure. the user may identify multiple system specifications that adequately satisfy requirements. requirements management and reuse example using the sepa process and tool suite sepa is being applied in a number of domains, including cancer treatment protocol development, emergency incident response, and joint military forces command and control. among them is a defense advance research project agency (darpa) initiative to develop information systems supporting first responders (fire, police, ems) during emergency situations (e.g., fire, natural disaster, chemical/biological [chem/bio] attacks) that may occur at large public events (e.g. rock concerts, political gatherings). in today's emergency response 83 environment, most incident types are initially managed by first respondcrs. the process by which the incident is managed is locally defined and evolves during the course of the incident, based on pre-defined standard operating procedures and local resource availability. the system under development will have functionality including responder task assignment and location tracking, casualty assessment, chem/bio agent analysis, and post-incident reporting and analysis. a key mandate of the resulting system is that it be flexible enough to support a wide variety of incident response facilities and organizations. this mandate will demand a significant amount f • • '"• ' i'.;]^: : _ ~ of application reuse and system customizability. the l: a^-**' requirements of each facility must be evaluated in the context of the system and incident response domain and satisfied by associated applications, making reuse at the requirements level the clear approach to multi-site implementation. on large, complex projects such as the darpa emergency response initiative, the requirements management processes meter suggested by the sepa methodology are impractical without adequate tool support. sepa process research is < ^ rivt complemented by a suite of tools to support each activity in the sepa process. figure 10 illustrates how the tool suite covers figure 10: sepa tool suite the activities associated with the sepa funnel presented in figure 8. a brief description of each tool follows. 1. knowledge acquisition manager (kam): provides project and document management functions for the knowledge acquisition (ka) process through a web interface. kam allows the knowledge engineer to (i) document ka plans, (ii) specify and maintain participant (e.g. domain experts, end users, knowledge engineers) contact info and background profiles, (iii) document intended ka session objectives, (iv) document elicitation scenarios, and (v) upload and search session reports which document the knowledge acquired during a session. 2. hybrid domain representation archive (hydra): provides intelligent reasoning functions that guide the user during the creation of knowledge models (kms) and unification of the kms. hydra's objectives are to: (i) aid the knowledge engineer (ke) by providing tool support for ka and modeling; (ii) automate the transition from unstructured, incomplete requirements to formal, complete, and consistent requirements; and (iii) maintain traceability through the necessary merging of requirements from varying viewpoints into the unified knowledge model (ukm), representing the combined viewpoints of all domain experts. 3. separator: assists the knowledge engineer in navigating the ukm and separating domain requirements from application requirements, and identifying specifications of existing, current technology solutions. 4. reference architecture representation environment (rare): semi-automatically guides in the transition from the domain model (dm) to a domain reference architecture (dra) by systematically applying object-oriented (oo) heuristics, software architecture heuristics, and quality metrics. the derivation process is guided by goals prioritized by the system architect, including extensibility, reusability, comprehensibility, and maintainability. 5. modeling environment for technology requirements (meter): assists the developer in modeling application requirements from the ukm in the interest of domain-independent infrastructure reference architecture classes (iracs) to capture (i) delivery requirements imposed by client demands; (ii) installation constraints imposed by site requirements; (iii) application installation requirements; and (iv) integration issues imposed by the selection of cooperating applications. 6. application specification environment and registration tool (asert): provides system developers with an interface to specify applications in terms of data and functional requirements in the (dra) and application installation and integration requirements in the (ira). 7. requirements and integration verification tool (rivt): assists stakeholders in system configuration and interrogating requirements in the dra and ira from various perspectives (e.g. end-user questions, developer questions, system integrator questions). rivt's querying capabilities encourage reuse by (i) aiding the identification and evaluation of applications against domain, application, and system infrastructure requirements and (ii) facilitating impact analysis as requirements and application technology change and/or evolve. using the darpa emergency response project for illustration, the following subsections guide the reader through the activities depicted by the sepa funnel, introducing the above tools as they become relevant. v •f* sepa activity: knowledge acquisition supporting tool: knowledge acquisition manager (kam) 84 all users hazmat specialist •/-fire'' '-.' police emt hazmat incident commander fire incident commander f fire station ^ administrator figure 11: viewpoint hierarchy the first step in gathering requirements is to determine the stakeholder viewpoints or perspectives that must be considered when designing the system (sommerville & sawyer, 1997). sommerville suggests that these viewpoints be gathered into a stakeholder viewpoint hierarchy. sepa's knowledge acquisition manager (kam) tool actually uses two orthogonal hierarchies. the first hierarchy is used to describe the domain role viewpoints (with leaf nodes such as hazmat incident commander or fire administrator). the second hierarchy is used to describe the organizational viewpoints (with leaf nodes such as city y fire department or city x police department). while the kam tool uses both viewpoint hierarchies simultaneously, this example will only use the shaded portion of the simple domain role viewpoint hierarchy shown in figure 11. domain experts for this project were selected so that a variety of expert viewpoints were represented, which for our example includes fire personnel and hazardous material specialists. the kam tool was used to maintain contact information on project participants and allowed these domain experts to be assigned to leaf nodes of two hierarchies based on their domain role viewpoint and organization viewpoint. kam was also used to schedule several ka sessions with these experts and can record session information such as session goals, times, dates, locations, participants, and ka approaches. although kes used a variety of approaches to elicit domain and system requirements, scenario analysis was the primary approach used to acquire domain task, performer, and timing information (harbison, 1997). other ka sessions focused on acquiring specific implementation requirements through prototype review, yielding information about preferred look-and-feel as well as installation specifics such as required operating system. the sepa example which follows is rooted in the information acquired from the sessions listed in table 1. table 1; ka sessions for incident response example session 1 2 3 expert with viewpoints jim hendrix hazardous materials specialist and hazmat incident commander • sam cook fireman, fire incident commander, and fire station administrator bob smith fireman and fire incident commander location :ity x ]ity response center )ity y 'ire station 1 :ity y 'ire station 2 ka approach scenario analysis scenario analysis prototype review information gathered tasks, performers, and resources for chem/bio response tasks, performers, and resources for fire response user interface (ui) requirements and specific installation needs for fire response to document each session, knowledge engineers (kes) created one or more ka session reports, accompanied by supporting diagrams, videos, and supplemental documents. ka session reports represent knowledge from domain role and organizational viewpoints associated with the domain experts involved in each session. knowledge engineers use kam to (1) identify domain experts, (2) define domain and organization viewpoints, (3) define an overall ka session plan, and (4) store the products from each session. project participants can perform content-based searches with filters based on both domain role and organization viewpoints. these session reports, diagrams, videos, and supplemental documents provide the foundation for requirements traceability in the sepa methodology. sepa activity: knowledge modeling and knowledge model synthesis supporting tools: hybrid domain representation archive (hydra) and third party modeling tools information found in ka session reports and related documents is not structured, making effective reasoning by a computer difficult. to transition ka artifacts to a computational representation, the ke interprets the artifacts 85 and creates structured graphical and textual knowledge models (kms) in hydra and other third party modeling tools (e.g., oracle designer (oracle, 1999)). currently, the ke has the choice of a number of models, including data flow diagrams, task decomposition diagrams, task templates, venn diagrams, entity-relationship diagrams, and concept maps. appropriate knowledge models are selected based on the type of knowledge acquired. for example, task decomposition diagrams provide an overall view of domain tasks and subtasks, while a task template contains specifics about the data, timing, and performance requirements for a specific task. to facilitate the validation of ka artifacts with domain experts, each ka session yields a standalone collection of knowledge models called a model space (ms). the resulting knowledge models generated from the sessions in table 1 are summarized in table 2. corresponding knowledge models are shown in figure 7figure 12. table 2; knowledge models from incident response example knowledge model venn diagram (figure 7error! reference source not found.) task decomposition (figure serror! reference source not found.) task templates (figure 9error! reference source not found.) task decomposition (figure loerror! reference source not found.) task templates (not shown) task performance constraint template (figure 1 1 error! reference source not found.) system constraint template (figure 12error! reference source not found.) source ka session 1 1 1 2 2 3 3 data represented attributes composing the "weather" concept. task decomposition for chem/bio incident response. selected task details for chem/bio incident response tasks, including pre/post conditions, performer, and input/output data. task decomposition for small fire incident response. selected task details for small fire incident response tasks, including pre/post conditions, performer, and input/output data. prototype presented to expert is associated with respective domain tasks as suggested implementation approach. overriding system implementation constraints (e.g. operating system). 86 pssvenn editor fie help diagram j mf0 | -rrace j history | status j o huid?*267-wmd direction hud #264-temperalure select tool ^—, j ~ , .... . ,.., -a. ,-,..,av.^ select as view point object figure 7: venn diagram from ka session 1 manage responders during incident 1.0 assess impact of toxic agent release 2.0 manage sector location 1.1 create after action reports 3.0 replay incident 3.1 create incident spill report 3.2 capture weather information 2.1 determine agent dispersion and possible casualties 2.2 gather agent characteristics 2.2.1 determine population distribution 2.2.3 gather weather and env. parameters 2.2.2 estimate casualties 2.2.5 create dispersion plume 2.2.4 figure 8: summary of task decomposition from ka session 1 87 task: manage res ponders during incident precondltion: during incident postcondition: not specified performer: incident commander data input: not specified data output: not specified task: gather weather and env parameters precondition: not specified postcondition: not specified performer: not specified data input: weather data output: not specified figure 9: summary of task templates from ka session 1 manage responds rs during incident 1.0 establish env. for local installation 2.0 assign crew to sector 1.1 create after action reports 3.0 maintain units and shifts 2.1 manage sector location 1.2 replay incident 3.1 create incident report 3.2 figure 10: summary of task decomposition from ka session 2 to achieve a single, unified picture of requirements, hydra assists the ke in synthesizing requirements gathered from multiple experts, following a hierarchy of viewpoints defined for a domain. in this example, all models from "hazardous materials specialist" experts would be merged into a "hazardous materials specialist" model space (step 1 in figure 13error! reference source not found.), and all models from "incident commander" experts would be merged into an "incident commander" model space (step 2 in figure 13). these models space would subsequently be merged into a unified model space, the unified knowledge model (ukm) (step 3 in figure 13). implementation id: city y fire dept. constraints: operating system: windows nt memory: 128mb fixed disk: 8 gb lan: 10 mbps ethernet figure 11: summary of system constraint template from ka session 2 task: manage responders during incident interface prototype: . respondei tiiinkc hhd prototype description: map-based approach for assigning responders to various incident hotspots and tracking their location as an incident progresses. figure 12: summary of task performance constraint from ka session 2 kmsin ka session 1 model space kmsin ka session 2 model space kmsin ka session 3 model space, r kmsin hazmat specialist model space kmsin incident commander model space kmsin unified knowledge model figure 13: knowledge synthesis process as synthesis proceeds, hydra detects conflicts between the knowledge models and presents the ke with possible resolutions. for example, when the ordering of two tasks vary between two models, hydra prompts the ke to choose which ordering to retain or to mark the tasks unordered. as each decision is made, hydra ensures the ke's rationale is captured. traceability is retained to record how model elements were changed or merged to produce the resulting element. details regarding the synthesis operation and representations can be found in (barber & jernigan, 1999). conflicts detected in this example along with their selected resolutions are: 1. task with almost identical children: task 3.0 in figure 8 and task 3.0 in figure loerror! reference source not found, have the same name and share a common subtask. however, the other subtask (3.2) is different. during the merge, hydra detects this situation and asks the ke if it should (i) keep all three subtasks, (ii) ignore one of the dissimilar subtasks, or (iii) merge the two dissimilar subtasks into one task. in this case, the ke recognized that these are the same task and asked hydra to merge the two dissimilar tasks into one with the name "create incident report." alternatively, the difference between the subtasks under task 1.0, "manage responders during incident," of the two task decompositions will not create a problem if the task decomposition from the first ka session (the session with only one subtask), is marked with an open world semantics or incomplete flag. 2. type usage conflict: hydra holds an ontology that specifies types of model elements, classifications of those model elements, and the relationships between those model types. the venn diagram for "weather" indicated the concepts in the diagram were "states" (not shown in panel exposed in figure 7). however, the "weather" concept was used in a task template knowledge model as a "resource" (data input in figure 9). hydra detects this usage as being inconsistent with the type provided by the venn diagram (figure 7). since, the knowledge models were validated by the domain experts before the merge, hydra cannot change them and preserve the validation. however, it can ask if the user would like to ignore the conflict, ignore the usage of "weather" in the task template, or ignore the type information from the venn diagram. finally, the user has the choice of changing a source knowledge model, revalidating it with the domain expert, and re-performing the merge process. in the example, the ke chose to change the venn diagram so that it indicated that the concepts were to be classified as type "information," a non-volitional, intangible, and consumable kind of resource. 3. "weather" not created: as part of the last steps of a merge, hydra attempts to perform some completeness checks. due to the decisions resulting from conflict 2 above, "weather" is now treated as a consumable resource model element type. as a consumable resource, it must have been produced by some task before it can be used by the "gather weather and env parameters" task (described in figure 9). since no task produces "weather," a conflict arises. to resolve this conflict the ke adds "weather" as a data output on the "capture weather information" task and re-merges. 89 establish env. for local installation 1.0 manage responders during incident 2.0 maintain units and shifts 1.1 assign crew to sector 2.1 manage sector location 2.2 create after action reports 3.0 assess impact of toxic agent release 4.0 replay incident 3.2 create incident report 3.1 capture weather information 4.1 determine agent dispersion and possible casualties 4.2 determine population distribution 4.2.3 gather weather and env. parameters 4.2.2 estimate casualties 4.2.5 create dispersion plume 4.2.4 figure 14: merged task decomposition from ukm following the synthesis operations depicted in figure 13, the incident response ukm should include concepts originating from the combined set of knowledge models. the merged task decomposition information in the ukm is illustrated in figure 14 for clarity. sepa activity: separation of domain and application requirements supporting tool: separator the ukm artifact provides a single, consistent model that becomes the basis for all further requirements analysis in the sepa methodology. before a domain reference architecture can be derived, domain requirements must be separated from application requirements in the ukm. separator aids the process of identifying domain content in the ukm, originally represented as concept models (such as the venn diagram in figure 7) and task-based models. the resulting extraction of domain requirements is captured as the domain model (dm). in contrast to concept and task-based models, task performance constraint and system constraint models (figure 11 and figure 12) are used to characterize currently available or desired applications and sites. this information is subsequently instantiated in the infrastructure reference architecture (ira), and becomes particularly relevant during technology registration and system design. in this example only domain requirements were elicited from ka sessions 1 and 2. application requirements were elicited from ka session 3. ft ^ ^^sepa activity: reference architecture derivation and application registration \ \~^~\ supporting tools: reference architecture representation environment (rare) \j and application specification environment and registration tool (asert) the domain requirements represented in the domain model (dm) reflect the synthesis of domain tasks and data expressed by stakeholders and compiled by kes. this knowledge is captured with respect to scenarios of operation. stakeholders describe their jobs, the data they exchange, and the technology they use in the context of those scenarios. given this elicitation approach, the dm follows a functional, rather than object-oriented, decomposition. however, to encourage the reuse of requirements and applications in multiple system configurations, sepa mandates that domain requirements be organized into object-oriented classes to which applications are registered. the rare tool assists in the translation from the function-based domain model (dm) to an object-oriented domain reference architecture (dra). this translation involves the creation of domain reference 90 architecture classes (dracs) and the allocation of tasks and data represented in the dm to specific dracs. the dra serves as a blueprint for developers, where each drag specifies domain data owned, domain services (tasks) provided, and data and event dependencies with other dracs. these dependencies arise from data and events exchanged between services in different dracs. domain tasks, data, and events can be allocated among classes in a number of possible combinations. as with traditional object-oriented analysis, the identification of classes is not an exact science but strongly depends on the priorities and skills of the analyst. furthermore, an entire object-oriented model is rarely defined in a single iteration; the model is typically refined over multiple passes. rare supports the iterative nature of objectoriented analysis and applies proven object-oriented heuristics based on priorities established by the analyst. to document the analyst's decisions and the evolution of the dra, the analyst's rationale is captured and traceability to dm tasks and data is established. to drive the allocation process, rare follows a set of high-level goals prioritized by the architect. four primary goals were emphasized in the darpa emergency response project: 1. align dracs with task performers in the domain: many domain tasks will remain un-automated, thus the collections of tasks assigned to a drag should closely follow the tasks assigned to the respective domain performer (e.g. incident commander). 2. align dracs with existing cots applications: to maximize reuse of existing cots applications capable of performing domain tasks, tasks should be grouped based on those tasks associated with existing applications. 3. create classes that increase installation customizability: to increase customizability for each installation, it is recommended to reduce the number of services offered by each drag and thus increase the total number of dracs. in general, a greater number of smaller dracs can be combined in more arrangements to support specific installation requirements. 4. allocate tasks among classes based on when tasks are typically performed during an incident response: incident responders only concern themselves with a certain set of tasks during any particular time period of an incident. for example, tasks such as assigning crews to shifts and defining resources (e.g. fire trucks) are done outside of any particular incident. tasks in different time periods naturally have fewer coupling constraints than tasks in the same time period. therefore these large grained time periods provide guidance for the assignment of tasks to dracs so that inter-drac coupling and the number of dracs the responder must interact with during any given period are reduced. a subset of the dra produced for the darpa emergency response project is depicted in figure 15 below. the "incident manager" drag was created based on goal 1, while the creation of the "weather manager" drag was strongly influenced by goal 3. drag: incident manager services: • assign crew to sector » manage sector location drag: weather manager services: • capture weather information contributing goal: align dracs with task performers in the domain. rationale: the domain role of incident commander is significant. contributing goal: create classes that increase installation customizability rationale: weather data should be independently managed from other types of data so data handlers can be combined in different ways for different installations. figurels: subset of domain reference architecture once the dra is defined, the process of associating a new, existing, or proposed (notional) end-user application with drag data and functionality is referred to as "registration." the asert tool allows the developer to identify drag data and services provided by both cots applications and customized applications. in this example, the applications under consideration and the domain services they provide include the following: • fdmanager v2.0 a cots application that provides a complete set of services to support small and midrange incidents. these services include resource allocation and responder assignment and tracking. fdmanager v2.0 provides drag services: • "assign crew to sector" in drag incident manager • "manage sector location" in drag incident manager 91 responderlocator vl.o a newly developed map-based application used to assign tasks and locations to responders and monitor their location. responderlocator vl.o provides drag services: "assign crew to sector" in drag incident manager "manage sector location" in drag incident manager weathermonitor vo.7a newly developed application that provides an interface for collecting weather data and disseminating this data to multiple responders. weathermonitor vo.7 provides drag service "capture weather information" in drag incident manager. ^-—-j?^---sepa activity: application requirements modeling | ff supporting tool: modeling environment for technology requirements (meter) associating an application to respective drag services is only part of the registration picture. during system design, the system integrator performs a brokering activity based on "how" an application performs its functions as well as "what" domain tasks the application provides. thus, an application must also be characterized by "application requirements," including specific implementation features and infrastructure needs. the asert tool allows an application to be registered against both dracs and iracs. figure 16 depicts both the "what" and "how" registration mappings for the fdmanager v2.0 and responderlocator vl.o applications listed above. registering against a drag specifies "what" an application does; while registering against an irac specifies "how" the application does it. as described in section 0, sepa's irac ontology is used to represent domain-independent knowledge about application requirements. following the separation process aided by the separator, the meter tool assists system stakeholders in representing application requirements, originally from the ukm, using the irac ontology (e.g. task performance constraint and system constraint requirements from ka session 3). the irac ontology is used to represent both the application requirements expressed by stakeholders and the specifications of an application being registered through asert. drag: incident manager a end-user application: fdmanager 2.0 assign crew to sector manage sector location end-user application: respond er locator 1.0 assign crew to sector irac: software application name: fd manager 2.0 executable 0> al $ irac: operating system type: microsoft windows 32-bit os irac: computing platform processor type. intel pentium class processor speed > 90 mhz main memory. > 32 mb disk storage: 2 5 mb display device: 640x480x256 irac: software application name: responder locator 1.0 executable <d "83s irac: runtime library name: xml library 1.1.1 irac: middleware data handler name: incident middleware 0.8 irac: runtime environment type: java2 runtime environment figure 16: application registration example with an understanding of the domain and application requirements for a particular installation, the system integrator performs the activity of system design by selecting specific applications that together comply with the client's installation requirements. as evident from the task decompositions generated from ka sessions 1 and 2, the sepa process must be capable of specifying two incident management configurations capable of satisfying two very different installations: city x and city y. to accommodate the requirements of each installation from a single set of 92 registered applications, sepa's separate domain and application requirements representations are used to aid the designer in identifying potential system designs and candidates for reuse. beginning with a blank slate, system design typically starts with a questions such as "what domain tasks should be automated at this installation?" sepa tools help the designer to answer this question independently of implementation concerns by perusing the domain model and/or reference architecture to identify candidate tasks. browsing the reference architecture is often preferred since services are likely organized into dracs that correspond to familiar domain roles (e.g. incident commander role, hazmat specialist role). for city x, two required domain functions documented in ka session 1 are: • manage sector location • capture weather information once the respective set of dracs has been identified, the requirements integration and verification tool (rivt) can be used to determine registered applications capable of performing the selected domain tasks. based on the domain services selected for city x, candidate applications suggested are: • manage sector location fdmanager \>2.0 or responderlocator vl.o • capture weather information weathermonitor 0.7 for city x, neither task constraints nor system constraints have been specified. thus the designer may select either fdmanager v2.0 or responderlocator vl.o to provide the "manage sector location" service. the designer narrows the set of candidate applications by imposing implementation requirements. these requirements (constraints) are expressed in the same language as the application registered infrastructure, the irac ontology. once the designer has successfully satisfied the domain and application requirements associated with each application, the brokering process continues by evaluating (i) the compatibility of the application collection against the system environment and (ii) the mutual compatibility of the applications. specific irac information relevant to this example includes the following: • fdmanager v2.0 • processor type: intel • processor speed: > 90 mhz • disk-space: > 5 mb • memory: > 32 mb • display size: 640x480x256 • requires-os: { windows nt workstation v4.0 sp4 i windows 95 i windows 98 } • responderlocator vl.o • requires-os: { windows nt workstation v4.0 sp4 i windows 95 i windows 98 } • depends-on-application-framework: incidentmiddleware vo.8 • uses-system-library: xmllibrary v 1.1.1 • executes-in-runtime-environment: javaruntimeenvironment vl.2 • depends-on-application-framework: incidentmiddleware vo.8 • weathermonitor 0.7 • requires-os: { windows nt workstation v4.0 sp4 i windows 95 i windows 98 } • executes-in-runtime-environment: javaruntimeenvironment vl.2 in addition to highlighting possible incompatibilities among applications, the integration process also highlights the need for secondary supporting applications, such as databases, word processors, and web servers. these related infrastructure applications may, in turn, have further infrastructure needs. for example, the infrastructure application incidentmiddleware vo.8 required by responder locator vl.o, requires the storage repository incidentrepository v2.0. • in contrast to the city x installation, options for the city y installation are constrained by the requested map-based user interface and the required operating system (windows nt) based on ka session 3. applications used in the city x installation are reused for the city y installation based on their registration to required tasks, providing they satisfy the stated application requirements. these requirements help narrow the set of feasible application, resulting in selection of responder locator vl.o to provide the "assign crew to sector" task, since the task is constrained by the requirement for map-based application. ^-—-sepa activity: requirements analysis supporting tool: requirements and integration verification tool (rivt) in addition to supporting the system configuration process, rivt also provides a query facility to interrogate the information contained in ira, dra, and registration representations to support impact and reuse analysis from a variety of stakeholder perspectives. 93 impact analysis for new application development in a domain as complex as incident management, the domain scope modeled will likely need to grow to satisfy a larger customer base. through ka, new domain tasks are added in context with existing functionality. relationships to existing tasks are determined during knowledge model merging and filtered down to the ra as changes in data and event exchange between drag services. when new applications are under consideration, an initial analysis of the reference architecture and currently registered applications can provide information regarding the degree to which a proposed application will affect or be affected by other applications. functionality to be provided by the new application is identified in the ra. if the domain task has not yet been represented in the ra, additional ka yields an expanded dm and results in new ra services. likely interaction among applications is evident through data and event exchange between the dracs that provide the functionality under consideration. these interactions can identify (i) other applications already registered which may require modification to correctly interface with the new application and (ii) functionality which has not yet been automated but must now be automated if the functionality under consideration is to be included in the system design. requirements analysis to support reuse of existing applications the sepa process and tools enable reuse of requirements through requirements modeled in a computational representation. application implementation and domain requirements represented in sepa can be used in combination to determine the likelihood of reuse. as in the brokering examples described in the prior section, the first step in satisfying the needs of an installation site is determining which domain tasks are to be supported. through application registration against the ra, these domain tasks reference applications that become candidates for reuse. to identify domain functionality of interest, the rivt user can take a number of approaches in posing questions to the ra. users focused on particular domain data elements: data elements represented in the domain model are "owned" by dracs in the ra. rivt can be used to determine what drag services utilize those data elements and what applications are registered to those services. users familiar with high-level domain tasks that are to be automated: high-level tasks in the dm identify lower level tasks that are satisfied by drag services in the ra. rivt can be used to determine what applications are registered to those tasks/services. users intending to automate the tasks for a particular domain role (performer): tasks performed by a domain role (e.g. incident commander) are identified in the dm; those tasks are satisfied by services in one or more dracs. furthermore, dracs often closely align with performer roles, thus collecting the services associated with a role. rivt can be used to determine what applications are registered to the service associated with dracs. once a selection of candidate applications is identified for reuse, infrastructure requirements associated with registered applications provide additional information for determining level of reuse. questions that can be answered using the sepa irac representation include: • what types of resources does the application require (e.g. hard disk, memory, peripherals)? • does the application run in the chosen runtime environment (e.g. java, visual basic for applications)? • does the application run under the chosen operating system? • will an application conflict with other candidate applications if integrated in a single installation (e.g. the sum of memory required by two applications exceeds that offered by the destination server)? • considering the pool of available application developers, is the application developed using skills developers possess and technologies they are familiar with (e.g. java, visual basic)? figure 112 depicts the result screen for a selected query in rivt designed to answer these types of questions. this screen provides the developer, end-user, or integrator with a high-level picture of the relationships between ra services (domain tasks), dracs, registered applications, and installation sites. in the example shown, the service "manage sector location" has been selected. the drag offering this service is highlighted, incident manager. in the third column, applications registered to the incident manager are highlighted, fdmanager v2.0 and responderlocator vl.0. the installation sites requiring this service are highlighted, including both city x and city y. 94 j requiienients integration and verification tool rivt ., h r̂̂ ,/-̂ .̂-status evaluation >; ,v4.a-:.̂ .1̂ -̂ . v -...4-^.1: -; ^,..«_^,^.^^i^ •i,^.-^.... 17'™' .-^_^figure 112: example rivt query interface for requirements reuse analysis conclusion effective requirements management is a significant ingredient in ensuring successful software projects and the long term reuse of code, design, and modeling artifacts. however, the requirements process becomes increasingly complex based on project size, the number of expert viewpoints, the variety of requirements classifications (e.g. functional, non-functional, project), and the different demands placed on software artifacts by system stakeholders. a breakdown in this process can both hinder current development efforts and limit possibilities for future reuse. the systems engineering processing activities (sepa), under research at the university of texas at austin in the laboratory for intelligent processes and systems (lips), seeks to improve the systems engineering process by providing a comprehensive methodology and a suite of supporting tools that emphasizes requirements analysis prior to implementation. this paper highlighted several salient issues regarding effective requirements management and refinement. additionally, the sepa methodology and tool suite is presented in the context of a project in which it is currently being applied. the following issues (indicated in bold text) summarize how the sepa process and supporting tools address these issues: traceability: attempting code reuse through requirements is exceptionally difficult without an accurate traceability mechanism from knowledge sources, through interim models, to system architecture, to system design. by emphasizing traceability throughout the requirements analysis and refinement process, sepa is capable of supporting queries against artifacts (models) and discovering relationships among artifacts. in addition to the standard benefits gained by accurate traceability to knowledge sources, the ability to navigate the sepa traceability links improves the opportunities for stakeholders to discover information about one model in terms of another. for example, a domain expert may be more familiar with high-level domain tasks than with specific applications. to discover applications associated with domain tasks, the tasks residing in sepa's unified knowledge model (ukm) can be traced to domain reference architecture services, which can subsequently be traced to related applications. support for multiple viewpoints: obtaining the viewpoints provided by multiple experts is critical to acquire sufficient breadth for domain modeling and system implementation requirements. however, developers must refer to a common picture of the domain to produce a software system with correctly cooperating application components that fulfill designated functions in the domain. contrasting developer needs, other stakeholders (e.g., end-users, domain experts) representing individual domain viewpoints want to be assured either that their interests are being reflected in the resulting system or that there is adequate justification for why their interests have not been taken into account. the knowledge synthesis process supported by sepa's hydra tool balances these demands by guiding the analyst during the synthesis process yet retaining and linking to knowledge originally stated by the stakeholders. hydra captures the original results from each ka session in a separate model space (a collection of knowledge models). as artifacts from ka sessions are synthesized into a unified knowledge model (ukm), traceability is retained and the rationale for each ke decision during model synthesis is recorded. one benefit afforded by this approach is that even after the unified model is created, an expert can continue to browse domain tasks in their own terminology and referencing unified tasks via traceability links. furthermore, the hydra synthesis process follows a "viewpoint hierarchy," first unifying requirements from stakeholders holding the same viewpoint. this retains separate model spaces representing the functional and data requirements associated with each viewpoint. computational requirements representation: sepa's ability to reason about requirements throughout the lifecycle is grounded in a series of related computational requirements representations. beginning with individual graphical and textual knowledge models (kms) transcribed by knowledge engineers (kes) from ka sessions, sepa aids the analyst in transitioning from one representation (e.g., unified knowledge model) to the next (e.g., domain model, domain reference architecture) as refinement proceeds, helping to ensure 95 consistency and completeness in the models at each stage. among other benefits, a computational representation increases the ability to answer questions posed by system stakeholders, including determining if a resulting software system meets expectations. without a computational representation, it is significantly more difficult to associate application technology features to originally stated requirements. for example, suppose a text requirement stated "the application supporting the billing process shall run on a windows nt machine with no more than 128mb memory." breaking this statement into a flat, "text fragment" representation leads to some difficulty for tool supported automated reasoning: (i) the statement actually suggests two different requirements: "windows nt" and "less than 128 mb." (ii) to determine if "128 mb memory" is satisfied, a numeric representation is required for proper comparison, (iii) both these requirements act as constraints on the "billing process" task, and any application registered to the "billing process" task should satisfy both the "windows nt" and "less than 128 mb" requirements. separation between domain and application requirements: to improve opportunities for long-term reuse and clarify the analysis process, software process experts often recommend separating concerns during analysis, focusing on "what" a system must do independently of "how" the system should be implemented. sepa explicitly recognizes this division and guides the analyst in making the separation. the resulting two models, the domain reference architecture and the infrastructure reference architecture, allow independent specification and analysis of "what" and "how" requirements. while the domain ra contains functions and data defined in the context of a single domain, the infrastructure ra is supported by a domain-independent ontology that provides consistency in representation across projects, domains, and implementations. this approach to "separate concerns" is particularly effective when configuring multiple systems to support the same domain. applications are "registered" to both domain ra functions and infrastructure ra specifications. thus, an particular application may be selected for a configuration based on its ability to perform domain tasks independently of its specific implementation features or required infrastructure. in many instances, multiple applications are registered as capable of providing the same domain ra function. the consistent infrastructure representation allows applications to be selected based on a common implementation feature, independent of domain functions (e.g. find all applications that run on windows nt). once applications have been selected, the representation also simplifies design trade-off analysis between system configurations. the designer can compare and contrast configurations of application technologies according to their ability to (i) meet domain functionality/data requirements, (ii) satisfy site implementation-specific requirements, and (iii) integrate and install as a coherent system. comprehensive process support for requirements refinement and management: software analysis and design methodologies typically suggest multiple phases during the requirements analysis process (e.g. gathering, modeling, refinement). however, the practical transition between such phases poses a significant challenge. in a practical setting, an analyst may apply different tools and/or customized approaches at each phase. as a result, the analyst's productivity is hindered by the difficulty in bridging processes and tools. the sepa methodology and tool suite is designed to guide each phase of requirements management and refinement and to transition seamlessly between phases. beginning with a series of knowledge models (kms) that capture the requirements expressed by multiple stakeholders, sepa aids the synthesis into a single unified knowledge model (ukm). the rare tool then guides the derivation of a domain reference architecture from functional and data information in the ukm based on a set of goals prioritized by the architect. the meter tool helps the analyst model application requirements, identified in the ka and modeled in the ukm, using a domain-independent infrastructure ontology. these models are then considered in combination by the rivt tool during system configuration and requirements interrogation. complementing the support for transition between artifacts, sepa's common traceability representation between artifacts provides the benefits previously mentioned. support for requirements volatility: the processes of requirements gathering, modeling, and refinement are inherently iterative. to maintain integrity between representations, traceability links and respective model elements require maintenance as requirements evolve and information is added, changed, and deleted. thus, a requirements methodology and supporting tool suite must be accompanied by procedures for reanalyzing existing modeled information in light of new contributions and modifications. in particular, changes to an artifact must be appropriately reflected in artifacts preceding and following the respective artifact. current sepa research is focusing on forward propagation. while back-propagation has been recognized as an important feature, this research has just begun in earnest. sepa is currently being applied to complex projects involving large numbers of stakeholders holding many different perspectives. focusing on one of those projects, the example in this paper illustrated how sepa is contributing to code and artifact reuse through requirements and provided evidence regarding sepa's support for the issues outlined above. as research and tool development proceed, additional experiments are planned to evaluate sepa's ability to support requirements management as domain knowledge and stakeholder demands evolve over an extended period. 96 references barber, k. s., & jernigan, s. r. (1999, june 28-july 1). changes in the model creation process to ensure traceability and reuse. paper presented at the international conference on artificial intelligence, las vegas, nevada. braun, c. l. (1999). a lifecycle process for the effective reuse of commercial off-the-shelf (cots) software. paper presented at the symposium on software reusability (ssr99), los angeles, ca. caldieri, gianluigi, & basili, v. r. (1991, february 1991). identifying and qualifying reusable software components. ieee. chipware, inc. (1999). icconcept rtmrequirements & traceability management tool, [web site]. marconi systems technology, inc. available: http://www.mstus.com/. christel, m. g., & kang, k. c. (1992). issues in requirements elicitation (technical cmu/sei-92-tr-12). pittsburg, pennsylvania: carnegie mellon university. department of defense, u. s. (1996). guidelines for successful acquisition and management of softwareintensive systems : department of the air force, software technology support center. dobson, j. e., blyth, a. j. c., chudge, j., & sterns, m. r. (1992, september). the ordit approach to requirements identification. paper presented at the sixteenth annual international computer software and applications conference. gotel, o. c. z., & finkelstein, a. c. w. (1994). an analysis of the requirements traceability problem. paper presented at the first international conference on requirements engineering, colorado springs, co. graham, i. (1995). migrating to object technology. wokingham, england ; reading, mass.: addison-wesley pub. co. graser, t. j. (1996). reference architecture representation environment (rare) a reference architecture archive promoting component reuse and model interaction. unpublished masters, the university of texas at austin, austin. harbison, k. (1997). scenario-based engineering process . http://caesar.uta.edu/caesar/process.html: center for advanced engineering systems and automated research, the university of texas at arlington. kotonya, g., & sommerville, i. (1997). requirements engineering with viewpoints. in r. h. thayer & m. dorfman (eds.), software requirements engineering (pp. 150-163). los alamitos, ca: ieee computer society press. mcgraw, k., & harbison, k. (1997). user-centered requirements. mahwah: lawrence erlbaum associates, publishers. mullery, g. (1979). core a method for controlled requirements specification. paper presented at the fourth international conference on software engineering, munich, germany. oracle. (1999). oracle designer . http://www.oracle.com/tools/designer/: oracle corporation. palmer, j. d. (1997). traceability. in r. h. thayer & m. dorfman (eds.), software requirements engineering (pp. 364-374). los alamitos, ca: ieee computer society press. qss. (1998). doors 4.0 . http://www.qss.co.uk/doors4/: quality systems and software. rational. (1998). software development, component-based, programming tools: rational. http://www.rational.com/: rational. riel, a. j. (1997). object-oriented design heuristics. reading, ma: addison-wesley. ses. (1999). ses/objectbench: object-oriented analysis, simulation and code-generation.. sommerville, i., & sawyer, p. (1997). requirements engineering : a good practice guide. new york: wiley. sommerville, i., sawyer, p., & viller, s. (1998, april 6-10, 1998). viewpoints for requirements elicitation: a practical approach. paper presented at the third international conference on requirements engineering, colorado springs, colorado. technologies, t. (1999). slate require. available: http://www.tdtech.com/. tracz, w. (1991, november 18-20). an outline for a domain-specific software architecture engineering process. paper presented at the fourth annual workshop on software reuse, reston, va. tracz, w., coglianese, l., young, p. (1993). a domain specific software engineering process outline. acm sigsoft software engineering notes, 18(2), 40-49. tracz, w. (1995). dssa (domain specific software architecture) pedagogical example. acm sigsoft software engineering notes, 20(3), 49-62. verilog. (1997). verilog objectgeode . http://www.verilogusa.com/home.htm. 97 a.iis vol. .1, no. 2 facing up to the plurality of goals, methods, needs, and resources in hci stephen w. draper gist (glasgow interactive systems centre) university of glasgow glasgow g128qq u.k. email: steve@psy.gla.ac.uk www url: http://www.psy.gla.ac.uk/-sicve keywords affortlance, activity theory, plurality, tasks, gucssahility, leamahilily, experienced user performance, learning by exploration. abstract most analyses of how humans use artifacts, and interactive software in particular, have a strong tendency lo assign 1:1 correspondences between goals and methods: to see software as supporting one task, users is having one way of executing a task, one thing to learn when learning a command, and one source for discovering the information. in fact this is a rare case, and multiplicity of goals, methods, information needs, and information resources is the rule even in simple software. how this causes problems for the design and testing of user interfaces can be illustrated by examples from a wide range of domains and levels of design, including studies on learning by exploration, the effect of machine delays on user strategies, the icamability of icon sets, evaluation studies of computer assisted learning, and an analysis of the concept of affordancc. such plurality can he a source of robustness for the performance of interfaces: it is a problem mainly for analysis and hci research, which struggle to account for the frequent case of high average performance levels mixed with a few residua 1 problems. to address this plurality, we must extend our analyses to cover sets of alternative methods for tasks rather than single user procedures, and perhaps draw on concepts such as activity theory to address users' mental organisation of such plurality. introduction we can detect and correct the main bugs in user interfaces by observing users having problems and employing an iterative design and testing cycle, just as in practice we find the main bugs in software by the initial informal testing of "trying it out". what we cannot do is approach a guarantee of bug free user interfaces through comprehensive testing, as we would for non-interactive software. proving correctness through exhaustive testing of all possible inputs (black box testing) is usually not possible because of the combinatorial explosion of possible inputs, and this applies also to hci since the "inputs" here would include all possible prior states of knowledge of the user. software engineers therefore often use "white box testing", where they use their knowledge of the internal structure of the code to exercise every branch of it. in hci we have to test the combined system of user and machine, so white box testing would require knowing what "procedures" or "methods" the user is executing. although task analysis purports to do this, a growing range of observations show that users are much more variable in their methods than analysts assume, and indeed more variable then they are aware of themselves. the rational way to design software is usually to organise the code so that each function needed as part of the design is implemented once and once only. on the one hand functions are not duplicated wastefully (providing two ways to do one job), but on the other hand each function usually does only one job so that if modification is needed for one task, others are not disrupted and need not be considered during the redesign. this approach of having a 1:1 correspondence between jobs and methods saves code (comparable to a person economising on learning), maximises re-use, and is thought to raise reliability over the program's life. most analyses of how humans use artifacts, and interactive software in particular, have a strong tendency to follow this approach and assign 1:1 correspondences between goals and methods: to see software as supporting one task, users as having one way of executing a task, one thing to learn when learning a command, and one source for discovering the information. however it seems humans are not like that. they do not usually all follow a single method, nor memorise the same things about an interface, nor follow the same goals. multiplicity of goals, methods, information needs, and information resources is the rule even in simple software. how this causes problems for studies of user behaviour can be illustrated by a wide range of examples. 31 a.iis vol. 3, no. 2 tasks in many cases, interactive software and its user interface has to support multiple tasks with different requirements. sorgaard (1988) discusses designs for a seat reservation system for railways. he criticises one based on traditional database queries which only supports requests for seats in terms of a few salient, explicitly named attributes e.g. window or aisle, facing forward or back, and proposes one that shows the seating plan for the whole train in a diagram that both customer and clerk can see and point to. this design allows customers to point to the seats they prefer, and allows them to take many possible attributes and relationships into account e.g. distance from the dining car, distance from the door, etc. sorgaard's design finesses any laborious attempt to discover all the "tasks" customers are trying to perform, and yet will support a far greater range of seat specification types including those that might be very rare in the user population as a whole. it seems clearly superior exactly because it will support tasks the designer could not have anticipated. the general point therefore seem to be that designers cannot foresee the tasks (top level goals) that users will in fact bring to artifacts. another example commonly quoted is that of spreadsheets, which are said to have been designed originally to support accounting (calculating totals of money flows), but are now widely, even predominantly, used to support financial planning in which the bottom line is entered as a given and possible component values calculated backwards from that. the way in which a new artifact often generates new, unforeseen, applications is also often commented upon, particularly by task-artifact cycle theorists (e.g. carroll et al 1991), who have developed a well elaborated account of this phenomenon. sorgaard's design suggests that there may be ways for designers to cope with this, but not by relying on task analysis in any usual form. instead, it is a case where the task analysis needed is not of specific individual tasks (which it may not be practicable to ennumerate), but of sets of tasks. it is not an accident that the solution offered is in the form of a particular visual representation to be displayed, and indeed to form the centre of the user interface. most really successful interfaces seem to be based around such a representation, whose virtue is to serve multiple implicit needs well. however the need to support multiple tasks with different requirements also occurs at a lower level, and in almost every program. a study (draper & barton 1993) of how users approach learning a new application program (macpaint) showed that roughly half their time was spent in learning by exploration (lbe) not in searching for a command to achieve a concrete task but in using a command experimentally to learn its effect. few if any task analyses have learning as a task, and so cannot capture this important and frequent user behaviour. furthermore the requirements for supporting lbe are different from those for supporting the material tasks associated with the same command. (the former requires action -> task mappings to be perceptible after the event, whereas material goals require task -> action mappings to be made by the user before the event.) whether subjects are instructed to explore or to do concrete tasks, both kinds of activity are seen: instruction seems to reverse the approximately 60:40 ratio of those activities, rather than abolishing either. in a few cases in some interfaces, even experienced users never learn to distinguish some commands and persist in rediscovering them by exploration as a permanent alternative to making the mnemonic effort. nevertheless, lbe is more important to first time users, and is a reflection of how the tasks as well as the methods users employ shift as they learn an interface. hence users' needs in part depend on their experience with the interface, and can be divided into guessability (first time use), learnability, and eup (experienced user performance). an interface must usually be designed (and tested) to satisfy all of these different needs. another general issue is the methods a user must discover for recovering from errors. in principle.there is one of these for every possible state of themachine. they may be infrequently executed, but are part of the set of user tasks a design must support, even though often forgotten in analyses. methods users exhibit plurality in methods (i.e. action sequences or procedures) as well as goals. at the highest level of tasks, software is just one means to some external end, as appears in the problem of evaluating the effectiveness of cal (computer assisted learning) programs [4]. here success at the task (i.e. learning) depends on the whole set of learning resources not just the software so that changing the software may have little measurable effect. for instance, if the cal is defective on some point, students motivated to pass exams will simply switch to an alternative source such as the textbook. consequently, little change in test results can be expected in real classroom settings as the quality of the cal varies over a wide range. artificial experiments can isolate the effect, but these are grossly unrealistic as they mean cutting students off from the multiple sources normally available e.g. help from teachers, consulting books. thus when computers are considered in their work settings, they have to be 32 a.iis vol. 3, no. 2 considered as part of an ensemble of resources and methods. in this context lack of use is not necessarily a failure, any more than a trouser belt starts to "fail" when the wearer lies down and it ceases to be the factor holding the trousers in position. at a lower level of task, that of learning basic command functions in software, users also normally employ diverse methods. even in the first few hours of organised instruction, a group of students can be seen to develop diverse methods for basic tasks. for instance, some students will use the save command frequently (to back up the current version to disk), but others may wait to the end of their job and then use the quit command which then leads them via a prompt into doing a save. logfiles might suggest that some students never learned to save as they never use the save command, but in fact they have just learned an alternative method for that task. in fact in a lot of software it is hard to find any function that can only be done in one way (e.g. to change the font of some text you can either type the text, select it, use a font command; or else select an insert position, use the font command, and type in the text); and this can lead to numbers of commands being apparently unused by some users. in a study centering on the task of copy typing three digit numbers over periods long enough for users to reach "expert" level, three user procedures are seen. they differ in whether the users bother to look at the screen for the prompt, simply type ahead, or pause until they estimate the machine is ready again. an early study (teal & rudnicky 1992) argued that the machine response delay determined which was adopted, but our followup studies (o'donnell & draper 1995) showed more complex results, indictating that we cannot yet predict what method users will select, even when well practiced at the task. the order of training, the need to recover from errors, or just lapses in concentration may all cause switches in procedure. just as users exhibit diverse methods of action, so they exhibit diverse methods of recognising icons. icons could be recognised by various features e.g. shape, position, and the question is which are actually used (moyes 1995). while some generalisations hold, users may in fact use different features of each icon in a set e.g. position for one, colour for another. there is evidence that the features remembered and used change with experience, although this cannot be detected by performance measures of time and errors: an example of a silent shift of cognitive method that both user and designer may be unaware of. cf. (kaptelinin 1993) information needs and sources the changes in icon feature used in recognition are related to a user's changing needs as they learn. another set of needs can be grouped under the heading of "affordance" (draper & barton 1993). in general there are three different pieces of knowledge a user learns or otherwise gains access to for each command, connecting it to action knowledge at its own level and at the levels above and below. consider for example the command "paste" in macintosh applications, a) it copies text from a hidden buffer into the current document (what the command does at its own level). users have to learn that the name "paste" refers to this action, and conversely that when they require that effect they should search for that command name, b) it is typically used as part of a standard two step plan of cut then paste to move text (how it fits into action plans and tasks at the next level up). this is almost impossible for users to discover for themselves (searching for a "move" command will not help), and they probably learn this from other users talking about "cut and paste", c) to execute it the user must first use the mouse to select the insertion point and then use the "edit" menu (the action level below). these three kinds of affordance information compete for expression in the interface design e.g. in the choice of command name. for instance type (c) how-to information is usually supplied by using a single consistent visual appearance for buttons and menus, so the user will try to operate any item of that kind; whereas if menus are concealed then users may not discover them at all. for instance, in think pascal on the macintosh, the window's title is really a pull down menu if you hold down the command key, but users never discover this without being told. of course, the title bar could have had an instruction ("click on me"). to express type (b) information about higher level tasks, a designer can offer a menu command named for the task that then leads the user through the necessary sequence of actions: "print" is of this kind, and similarly a "move" command could be offered like that. because of this competition for expression, many items of information remain unexpressed: an example of the plurality of user information needs lying behind the design of command presentation. designers can seldom even enumerate these needs, much less adequately serve them all. on the other hand, there is also a plurality of information sources. for instance users can learn either from the interface directly e.g. through self-explanatory command names, or by exploration. as long as one source works, the others may fail with little deterioration of performance. hence an important issue in usabilty testing is how to decide if a momentary problem is in need of fixing. when "failures" in fact lead soon to a user trying an alternative action and succeeding this can be a successful example of lbe, which is a legitimate and important technique for communicating information to users. conversely, however, the fact that users can 33 a.i1s vol. 3, no. 2 often find alternative methods may conceal failures that perhaps should be fixed. the fact that students can learn from a textbook does not mean that ineffective cal material should be left in use, and the fact that users can rediscover a command's meaning by exploration does not mean that an unlearnable or misleading command name or icon should be left in place. analysing plurality thus the pervasiveness of plurality significantly complicates rational testing, although it explains why interfaces with many imperfections can nevertheless produce good user performances on average (i.e. ignoring the minority of users and occasions where all methods fail together) thus reducing the urgency of perfecting them. we need to practice analysing, not the single way a user is expected to do something, but the complete set of ways they might try to do it. we need to begin to formulate desirable properties of these sets e.g. rapid convergence on task achievement, alternative methods that do not require the use of the eyes, or one hand, or prior expert knowledge, etc. and we need to formulate the need for a smooth learning path between one method and another (e.g. see [2] for how better to support the learning of keyboard accelerators for menu commands.) one theory that may help us to understand at least some aspects of this pluralism underlying human behaviour and knowledge is activity theory (at), not least because of its linking of consciousness to levels of behaviour. according to at, our consciousness is focussed on the central level of actions an analysis consistent with ideas of direct manipulation as engaging users by sustaining the feeling of doing things "directly". it is obvious that more automatic acts (e.g. positioning finger tips over keys) are largely beneath the level we are attending to and so unconscious or at least unattended to, but we should note that at also implies that the level of activities above conscious actions is also in effect largely outside our awareness. we remain engaged with moment to moment actions rather than reflecting how these are or are not contributing to our larger goals. this is consistent with how people can lose track of time while working at direct manipulation tasks, and with how people are bad at giving an accurate account of how they do their work, as suchman has argued for a long time (suchman 1983, 1995). the frequent failure in hci to attend to the need for lbe and for error recovery may be an effect of this: if you observe users you see them doing both, but if you ask them what they do when they use a piece of software they are unlikely to mention them. thus partly because our attention both as users and as designers is usually concentrated on only one level, human behaviour and the properties of successful user interface designs have more complexity than we think. references carroll j.m., kellogg, w.a. & rosson, m.b. (1991) "the task-artifact cycle" in j.m.carroll (ed.) designing interaction: psychology at thehuman-computer interface pp.74-102 (cambridge, uk.: cambridge university press). dix,a. (1995) "accelerators and toolbars: learning from the menu" hci'95 adjunct proceedings (eds.) g.allen, j.wilkinson, p.wright (school of computing & mathematics, university of huddersfield). pp.138-143 draper, s.w., & barton, s.b. (1993) "learning by exploration, and affordance bugs" interchi'93 adjunct proceedings (eds.) s.ashlund, k.mullet, a.henderson, e.hollnagel, t.white (acm) pp.75-76 draper,s.w., brown,m.i., edgerton.e., henderson,f.p., mcateer,e., smith,e.d., & watt.h.d. (1994) observing and measuring the performance of educational technology tilt project, robert clark centre, university of glasgow kaptelinin, v. (1993) "item recognition in menu selection: the effect of practice" interchi'93 adjunct proceedings (eds.) s.ashlund, k.mullet, a.henderson, e.hollnagel, t.white (acm) pp. 183-184 moyesj. (1995) putting icons into context: the influence of contextual cues on the usability of icons phd dissertation, dept. of computing science, university of glasgow. o'donnell,p.j. & draper.s.w. (1995) "how machine delays change user strategies" hci'95 adjunct proceedings (eds.) g.allen, j.wilkinson, p.wright (school of computing & mathematics, university of huddersfield). pp.111-117. sorgaard, pi (1988) a discussion of computer supported cooperative work ph.d. thesis, aarhus university.denmark suchman, l.a. (1983) "office procedures as practical action: models of work and system design" acm transactions on office information systems vol.1 pp.320-328. suchman, l.a. (1995) "making work visible" cacm vol.38 no. 1 pp.56-64. 34 a.iis vol. 3, no. 2 teal, s.l. & rudnicky. a.i. (1992) "a performance model of system delay and user strategy selection" proc. chi '92 pp.295-305 35 quality management systems in australian software houses: some problems of sustaining creativity in the sofware process liisa a. von hellens school of computing and information technology, griffith university nathan campus, kessels road, brisbane, 4111 qld, australia l.hellens @cit.gu .edu .au abstract software houses are taking steps towards the implementation of quality management systems (qms) and achieving certification to international quality standards. there is an increasing tendency to require quality certificates from system suppliers before business can be even considered. the qms is seen as a way of avoiding personnel risk if product and market knowledge remains in the possession of individuals. it is also felt that quality procedures in place will improve the company's image, attract new staff and help to keep the staff turnover low. however, the application of standards to the less structured tasks, eg. pre-sales or research and development tasks, was seen to stifle the creativity needed to complete them successfully. introduction background — the national initiative to improve software quality as barriers of free trade are gradually disappearing the opportunities for co-operation and world wide trade increase also in the it industry. the new trade order requires adoption of adequate standards, otherwise the offerings of individual nations will not be trusted and the proper development of the market will not become possible. for some markets, it is essential to have quality certification for software products. an essential step in achieving software quality is the construction of a quality management system (qms). this involves the identification and formalisation of all business processes that can affect delivered quality. the collective efforts of staff can then be recognised by seeking conformance certification of the qms to international standards by impartial certification companies operating under agreed arrangements. this paper provides an insight into some of the problems related to the quality accreditation process in systems houses. although the research was carried out in australia and the results therefore reflect local industry structure, we feel that software industry in other countries is very much the same, with only minor differences in emphasis. our plan is in the near future to compare our findings with the situation in some european countries. the australian software industry is a major growth industry* playing a vital role in enabling the manufacturing and service industries to become more efficient, productive and competitive. despite its vital role several factors are threatening the software industry's ability to survive and flourish in the increasingly competitive international market.^ there is a lack of information available for senior management on the internal and external value of quality management systems for software developers and on more general quality issues. there should be more options for software developers to apply quality methods, and infrastructure support for companies wishing to implement quality. software pilot case studies the software pilot case studies project which is considered in this paper is part of the australian commonwealth government response to requests from the software industry, to provide an overview of the current status of software quality management systems in medium sized systems houses and to provide a basis for recommendations to improve the benefits of such as system. in this paper we discuss the process of implementation of quality management systems in the systems houses included in the pilot investigation. the according to the department of industry technology and regional development, the software and information industries are forecast to grow by 16 % per annum. twelve of the top 50 companies were listed as the fastest growing in australia were software developers. in a recent survey it was also pointed out that exports have grown strongly, but there is significant potential for further growth; see smilie (1993). the discussion paper by professor geoff dromey (1990), the software quality research institute at griffith university, points out several factors are causing problems in software industry: 1) outdated management methods, 2) indifference to quality throughout the industry, 3) lack of coordination in relation to quality, standards, accreditation, marketing, education and training, 4) lack of international software market information and access. 14 ajis paper emphases issues, that from the software developers' point of view, seem to be reinforcing or inhibiting the creativity of the more intangible and less structured activities of the sales cycle and product development. by creativity we mean tasks that are highly unstructured and for which the means to complete them are not well specified. creativity is also characterised by originality of thought. the six systems houses included in this study (named here as companies a, b, c, d, e and f) have total staff ranging from 4 to 80 people of which the number of software staff ranges from 3 to 60. the turnover varies from a$ 100,000 to a$ 5 m. the companies are all fairly young, the oldest has been operating in its present form 1973; the others were established in the mid 1980s. every company currently offers a software package, the applications vary from text retrieval systems and industry-specific financial packages to scientific software for marine navigation. five companies started as a package supplier, but only two have the majority of sales still coming from the package sale (eg. license fees); others moved to the computing service provision and are currently developing and implementing systems to fit individual customer needs. development tasks include the definition of the parameters of the software package to make the software fit the users' information systems needs. all companies have started to export their software products, the target markets being asia, north america and europe, especially the u.k. we have to point out that certain factors have had a significant impact on the research and should be considered in mind when evaluating this report. first, the data had to be collected to complete the assignment for the commonwealth government. the goal was to present a variety company experiences to illustrate the individual nature of quality management systems. a systematic investigation of specific aspects of the implementation process, eg. of the impact on creativity was secondary at this stage. secondly, the australian software industry is new to the researcher whose previous expertise is from the uk and scandinavia. the vantage point of someone from outside having a fresh view may have helped to pinpoint factors in the qms implementation that otherwise perhaps would not have not been identified. research methodology the central problem discussed in this paper concerns the effect the qms is perceived to have upon the creativity of software development. we define creativity as a process of a "fruitful combining which reveals to us unsuspected kinship between facts, long known but wrongly believed to be strangers to one other" (couger, et al. (1993))in software development new ways of structuring the program often leads to more efficient system, not only by combining long known facts but also by incorporating some of the latest features of the operating systems. it is felt that a certain level of freedom in application development facilitates innovative solutions on the one hand, but uncontrolled software process can also lead to unprofitable systems that are difficult to maintain. our objective is to find out what are the characteristics of creativity that are important to the innovative development process but at the same time allow the control of it to protect corporate knowledge or to assure the solutions are cost effective and easy to maintain providing the software house with a sustainable competitive advantage. given the broad objectives of the case studies, our research into creativity at this stage is exploratory, rather than rigorously testing existing hypotheses, trying first to identify general problems of the qms implementation. secondly, we aim to identify the features of the qms implementation that seem to contradict the organisation's objectives of product development in terms of creativity. we suggest that such features inhibit the success of the qms implementation as they are likely to cause conflict between the quality controls required for the fulfilment of iso standards and any existing mechanisms that the company already has to control the quality of its product development and service delivery. we used the following questions to explore the qms implementation in the six case companies: (i) what is the motivation for qms implementation in the small software development organisation? in terms of the reason behind the desire to become accredited to iso quality standards what patterns are evident? (ii) how do small software development organisations perceive the link between the quality control and quality assurance activities and the software process ? does a variation emerge in the pattern which helps to explain the success of the qms implementation? (iii) are there activities in the product or service delivery process that are not covered by the qms; and if so, should the company-specific quality management system be adjusted to cover them too? (iv) how does the qms relate to total quality management (tqm) ? september 1995 15 the data was collected by interviewing a selection of company executives in each company by the author and a research officer from the department of industry, technology and regional development. we spent 1 2 days in each company, interviewing the management and staff and studying the forms and documents used in the software process. for the interviews we had an agenda with twenty open questions and ten more specific semistructured questions relating to the points (i) to (iv) above. our plan was to interview general management, the sales and marketing staff, and the technical staff involved in product development. the managing director was interviewed in all cases and in five companies we also managed to talk to the sales and marketing directors. in two companies we were not able to talk to the technical staff, but in these cases the top executives, whom we did interview, were heavily involved in the product development and that enabled us to consider the company's qms from the programmer's point of view too. in three organisations we interviewed the quality manager; others had not yet appointed a quality manager. the technical staff interviews provided us with information to evaluate whether the technical people's perceptions of the benefits of company-specific quality manual were in line with those of top management and marketing staff. all interviews were recorded, and we used the transcripts to describe in detail the expressed areas of concerns about the company-specific quality management system. the interview agenda focused on the procedures which facilitate the creation and monitoring of quality covering the following areas: • corporate strategy issues — to find out what are the long term and short term objectives of the qms and the benefits expected through its implementation. • implementation of the quality management systems — to find out the differences between expected and perceived benefits, and the reasons for the differences between them. we also wanted to know whether the quality management systems implementation was similar to situations where new technologies have been adopted or process innovations have taken place. this, we felt, would advise how to manage successfully the implementation of the qms. • controls placed to fulfil the requirements for standards — to find out their impact on quality and whether they were totally new procedures, or modified from the existing controls. we asked the staff members to express their opinions about the level of impact of the control procedures on the quality of the product or service delivered to the client (product quality), on the customer satisfaction (customer value), on the cost of producing these satisfactions (development cost), and on the organisation as a whole considering quality as a general philosophy pervading the whole organisation (quality culture). • the role of the qms in the software house's product and service delivery process — to find out activities addressed by it, and to find out why others aren't. in three of the companies (a, e, and c) the interviews were completed by running a workshop with the general management, technical and marketing staff to draft the company's value chain (porter (1985)) to identify the current business processes. the workshop material enabled us to examine if staff members representing different functions (general management vs. sales and marketing vs. technical staff) agreed with the description and the flow of activities. it also enabled us to discuss with the staff about the benefits of including more activities in the qms. as stated earlier, our research is exploratory and builds hypotheses rather than rigorously testing them through case study interviews. we formulated working hypotheses for the research questions (a) (d) which then helped us in analysing the interview material by providing a framework to look for arguments to support or reject them. first, we proposed that the reason to become accredited was the desire to improve the company's image and to cope with emerging market imperatives. we also wanted to know whether export oriented companies differ from those operating solely in the domestic markets. especially we were interested to find out what exactly are the driving forces for the qms implementation knowing that the customers in asia — a popular target for australian software exporters — do not require formal quality certificates as the potential customers in most european countries do. secondly, certain organisational factors may affect the qms implementation. previous research has showed that small software houses in most cases have started as one product companies and are technically oriented focusing on product development rather than fulfilling the clients' information systems needs (von hellens (1991)) assuming that the companies studied here are also either technically or market oriented depending on how well customer needs and wants are taken into account in their product or service delivery process, we suggested that in a truly market oriented company the staff sees the quality controls, even though they require boring paperwork, as something which actually complements the business rather than stifles its less structured, highly creative activities. our case companies are all relatively small organisations and thus the technical staff is expected to contribute to customer support and service. because of this, we assumed that the staff does not regard the implementation of standards as being forced to do something totally different from their everyday business. instead, we suggest that there are practical problems in converting the existing customer focus and 16 ajis quality of the service to something the development team (technical staff) can incorporate in their working practices. in addition we wish to find out what the companies are doing to overcome such problems. our third proposition states that because some of the r & d and sales cycle activities are intangible in nature and the quality of these activities being highly dependent on the appropriate skills of the individuals involved, there is no single way to complete them. to document these activities to embrace individual variations seems to be a fruitless effort. as a result they are not addressed by the qms either. we suggest that the main problem is that these activities place more emphasise on the results, ie. a new product or a contract, than on the delivery process. consequently, the performance indicators that are in use, evaluate the product, not the process. therefore it is difficult to see the relationship between quality of the process (controlled by the qms) and quality of the product, which depends on the skills (and creativity) of the staff member involved. in our research we wanted to find out if software houses agreed with this, and if so, do they have any desire or plans to change the situation. finally, we assumed that the software houses see the qms as a means to develop the organisation's total quality management (tqm) where the management seeks to remove the cause of faults rather than merely detect them when they have occurred. quality management system and maturity of software development organisations quality software process leads to quality software products ? quality refers to the totality of features and characteristics of a product or service that bear on its ability to satisfy stated or implied needs.3 quality assurance is all those planned and systematic actions necessary to provide adequate confidence that a product or service will satisfy given requirements for quality. quality control consists of all operational techniques and activities that are used to fulfi l requirements for quality. quality management is the aspect of overall management function that determines and implements the quality policy. quality system is the organisational structure, responsibilities, procedures, processes, and resources for implementing quality management. the most widely accepted quality management system is the iso standard 9001 (edelstein, et al. (1991)). as engineering standards have concentrated on the specification of quality product requirements, the software engineering standards emphasise almost solely the development process. the recent work by international standards committees have focused on developing quality standards for software products.(standards australia (1993)) quality management systems seek conformance to the process oriented standards. we assume that because software development still possesses craft elements and is very much dependent on an individual's programming skills, the relationship between a 'quality process' and a 'quality product' that fulfil client needs is not clear amongst staff in system houses. furthermore, standards are seen as stifling the otherwise very creative systems development process and attempts to implement standards may lead to conflicts within software development organisations. cmm — a challenging ladder to optimised software development the capability maturity model (cmm) by humphrey (humphrey (1988)) of the software engineering institute at carnegie mellon provide a useful framework to consider the role of a quality management system in the progress towards the higher levels of maturity of software development organisation. the software processes at the first level (initial process) are ad hoc type and the organisation operates without formalised procedures. organisations lack of ability to cope with crises and as a result the software processes capability is unpredictable. in the second level the organisation has established basic management controls. organisations have learned from their prior experience on similar projects, hence the name — repeatable process — for the second level. although organisations generally may have learned to manage costs and schedules and the process is repeatable, they face major risks when new challenges arise, such as changes in tools and methods or new types of products. organisational changes also cause problems, eg. when key persons leave the organisation in the middle of the assignment. to overcome the risks the process has to be defined. the third level of maturity — defined process — provides a basis for continuous progress. the process definitions are qualitative descriptions not necessarily involving any exact data of the effectiveness of processes. however, even the qualitative definitions can be used to improve the capability of the development organisation and the product 3 definitions of quality and quality assurance are from the iso 8402 [iso 8402| see eg. ould (1990), p.98 104; price waterhouse (1988), p.98. september 1995 17 tqm level 5: optimizing optimized software processes level 4: managed qms used alongside management control mechanisms accredited qms f level 3: defined software processes are measured qms level 2: repeatable software processes are defined level 1: initial basic management controls in place figure 1: the qms and maturity of software development organisation. to overcome the practical problems in convening the very strong existing customer focus and quality of the service to something the development team (technical staff) can incorporate in their working practices, the company e explored new ways of describing the intangible nature of quality to make it more tangible and better appreciated by the staff. as the general manager stated: "we now redefine the content of the quality management process itself and the sort of things which the company wants to capture aiming at success in the market". their quality manual was defined to provide a more comprehensive understanding of the intangible elements of quality and the business success of the company. in the company e they are also considering ways to re-organise the projects to increase quality appreciation amongst staff. for example, it was noticed that when development staff was dealing with the service tasks they became aware of the direct relationship between customer needs/focus and the process quality and the development teams were brought closer to the customers. the development team was seen to benefit by finding it easier, after being involved in support activities, to define what quality is, and to convert it to something the development team can incorporate in their working practices. it was observed that the opinions on what exactly are the activities of the product life cycle in systems houses differed significantly amongst staff. in the company b, application development, to construct a well defined system to fit a certain hardware platform and programming environment is seen to be different from a true software project of constructing a system to fulfil the specified information system needs of the client. for an application development project neither the existing software development method could be applied nor the process oriented qms. the major criticism of the qms shared by the management of the systems houses in this study, is that the perspective it represents is too narrow, leaving out important business issues, because the objectives of the qms is to assure that the client's requirements are taken into account during the course of development without considering the cost effectiveness of the project. similarly, the bidding process is also seen as being separate from the planning of the project. these criticisms are strongly supported by the sales staff which was involved in the negotiations and dealt with the delicate aspects of initiating new businesses. however, the technical staff involved in the process do not always share these views. in contrast, they felt that a major part of bidding (also called as tender process) was highly structured, could be well defined and could be included in the qms. bidding activities were regarded by technical staff as preliminary project planning tasks which support significantly the actual project planning. technical staff members also pointed out the risks of losing valuable 'corporate information' if a staff member leaves after a successful bid but before project has september 1995 19 started. the company b which stressed this point had lost business due to such instances, after which not enough customer information was left in the company and, subsequently, the competitive advantage they initially possessed, had vanished. alongside the definition of the intangible sales cycle activities appropriate measurements need to be developed to provide information of the quality of these activities. they are highly unstructured utilising versatile product knowledge accumulated in the systems house. in order to define and control such a process, various types of qualitative information should be gathered and tools able to make use of qualitative information must be designed. such tools would help to identify factors that have significant impact on a tender's success. by controlling the factors not only the quality of the processes could be assured but also the success rate of tenders could be raised. in any business organisation a variety of performance indicators to evaluate the sales cycle activities (eg. sales results and customer sales potential) are commonly used. performance indicators provide a useful basis to define quality measurements for intangible sales cycle activities. unexpected changes of requirements during systems development a qualitative evaluation of the client's maturity would support project planning, providing information on the likelihood of previously unidentified, major system requirements emerging during the course of the development, which will need to be taken into account in the product or service production to assure the highest possible level of customer satisfaction. previously unidentified customer requirements may cause unexpected changes during the development, and according to one systems house in this study the bigger the system is the more of them there are. unexpected changes to the systems originating from the client are something the software development organisation cannot easily accommodate, except by keeping constantly in touch with the client during the development and having procedures in place to handle new information about the requirements, as was the case with one of the companies interviewed (company c). the procedures involve advice on how to deal with different types of change requests. they vary from "quick fix", to new developments which will result in a "new development project" during the initial systems development life cycle. in both cases the estimated budget and time to complete the initial assignment has changed. the change may have been caused by something external to the client or is simply something the client did not feel as being important for the system structure in the first place, eg. in a financial package a type of invoice, which happens regularly but scarcely, for example once a year, and is processed differently causing major changes in the system structure if taken into account. whatever caused the change (from "they did not know" to "they did not tell") it will be dealt with carefully bearing in mind the primary goal behind the product or service design: "total customer satisfaction, both immediate and ongoing, which will result in a sound and profitable business." according to the company this has been applied successfully ("in our opinion we produce a better product for our client than our competitors"). however, it may turn out to be difficult to document the process in a concise form for a quality management system without losing the existing creativity in restoring the customer focus, that the company c clearly shows, in handling the ever changing customer requests. company c has recently started to place formalised and documented quality controls on their product/service production process and plans to have the qms accredited sometime during the next 12 months. from what we have seen so far the time estimate is perhaps too optimistic. client maturity evaluation the unexpected changes represent a major risk factor for the project. in order to minimise the impact there should be procedures in place to evaluate during the tender process or contract negotiations how well and comprehensively the client able to present the business processes the supplied system is meant to support. if the customers are mainly organisations that are more mature in the sense that their business processes are well defined, the risk of unexpected changes is minimal and there is no need to define quality controls for handling unexpected changes. some systems houses do evaluate the client relationship ('intelligence feedback', 'distance to project location') during the tender process. company b uses 'a bid approval request'-form to provide information to support the go/no go decision making during the bidding. potential projects are ranked on the basis of ten variables describing the client relationship and the suppliers experience in handling the type of application as well as competition which affects the success of getting the business. each variable is rated from 'poor' (= 0 to 1 points) to 'superior' (= 9 to 10 points) and the prospect with a total of 60 points will be recommended as worth pursuing. the risk evaluation procedure of the company b does expose some situational factors that may inhibit or reinforce the project's success. however, the procedure does not actually say anything about the prospect's 20 ajis maturity or capability to define the application and system needs. we feel that this is a reflection on the fact that the bidding process is so dominated by the competition that it hides factors that will .certainly affect the development. for example, extra implementation costs may occur when new development tasks are needed to add system features which were not previously specified. the staff members admit that there is a more or less total separation between pre-sales activities (tender processing, bidding) and contract negotiations during which the system is specified in detail, and thus many of the documents used in tender processing do not provide accurate information for the project planning. according to the quality manager in the company e they concentrate of getting the business ("we try to convince the prospect that we are the best") rather than getting the details right ("we worry about it later") in the tender process. the company felt, however, a genuine need to expand the quality management system to the tender processing, in particular if prototypes were used to support the bidding. depending on the nature of the business the clients are different, eg. the insurance industry in australia is for historical reasons well developed. insurance companies as clients very seldom cause significant changes in systems logic during development (although the systems may possess other risk factors, such as new technology, new software tools), because they have a very definite picture of the business processes and actions that need to be performed to provide services that fulfil their clients' needs. however, not all client groups are as homogeneous as those of the systems house f providing computer-based systems for insurance firms. other industries are much less developed in this respect. client maturity is a kind of a market imperative, appreciated by sales staff in the companies studied, but rarely understood in the sense of the implications for systems development and quality control. if information on such incidents is not gathered to aid the qms development, there is no way to guarantee that the quality management system is really helping to improve the overall effectiveness because it will merely capture the errors where the enter into the life cycle, but will be unable to advise about a client related risk factor that will badly influence the development process later. internal and external reasons for seeking quality accreditation quality accreditation to protect corporate knowledge the quality management systems of a software development organisation can be assessed by a first, a second or a third party. in practice the two latter are the most valuable because they provide an independent evaluation of the organisation and are thus more reliable in the potential clients' eyes than an internal assessment by the organisation itself although it may be based on extensive internal reviews, audits, and client satisfaction surveys. a second party assessment is when an organisation and its products are assessed by a purchaser against product or quality system standards selected by the purchaser organisation. such an assessment may be relatively narrow in scope, covering only the elements relating to the product or services purchased by the assessing second party, but it can provide a sharp focus on that particular area. the second party assessment may be a valuable step towards third party accreditation, as it was for the company a in this study. they gained second party certification to australian standard as 3563 from their major customer queensland government in 1990. it became clear that this process eased the way towards the third party certification awarded by bureau veritas, an international quality auditor, in june 1993. software development organisations may have external or internal pressures for seeking accreditation. an example of the external pressure is the market imperative, such as demands by powerful buyers who require that their suppliers have certain types of standards in place before business can be decided. product and market knowledge in the sole possession of individual staff members are risks which need to be controlled. this is an example of internal pressures to become certified. internal pressures also stem from the need to secure skilled work force in the future. as business is becoming more service oriented the staff required have to possess not only technical skills but also a variety of interpersonal skills. due to increased competition, obtaining multi-skilled people rather than purely technical specialists is difficult ("no longer can i say that we have the elite of the industry" as the managing director of the company a stated, and continued: "we have start taking people at different levels"). the accreditation of the qms is seen to attract more people and provide more alternatives in staff selection, because the company will be seen as an advanced and challenging workplace. product and market knowledge represents corporate knowledge, expertise accumulated during the pre-sales activities and during the r & d tasks of software production. these less structured tasks are more difficult to define and it was strongly felt in the organisations studied that any formal structure may stifle the essential creativity needed for the successful completion of them. at the same time it was felt that the qms could benefit both areas and help to avoid some of the problems, such as the too low success rate of tenders and freely evolved innovative product structure which causes problems for those who have to maintain the system. september 1995 21 the dilemma between the r & d and maintenance requirements was illustrated by the managing director of the company d who stated: "we want to keep the internal r & d as informal as possible and the specs [system structure] should be flexible enough. at the same time we do have organisational problems as someone else needs to be able to maintain the system. so far we don't know how to define the qms for us." "history of complaints" triggers qms implementation extensive negative feedback from the market often triggers the implementation of a quality management system. complaints from the customers make suppliers know that they have to improve the quality of their products. most companies included in this study had a 'history of complaints' which became impossible to handle effectively unless radical actions to improve the systems were taken. for example, in the company a an incident happened in mid 1980s when a large number of a certain release of a text retrieval system were sold before it was discovered that the system could not handle large data volumes. the management still recall the event as a major disaster that must never happen again. the need for further testing and controls became clear and as a result the quality management system implementation and continuous improvement has full support from the top management. for the company a, the visible management support is essential, as the staff resistance is significant especially from the more experienced analyst and programmers. company a has also the largest number of staff, 80 people, some of them working continuously at the clients' sites, which makes it difficult to impose quality practices and control their adoption. other 'histories' were not as closely related to certain releases and to certain types of errors, but to a generally low technical quality of a piece of software. a relatively large number of unsatisfied customers occurred when the product license was bought by staff members establishing a new company. the combination of dealing carefully with the angry clients' complaints and at the same time rewriting parts of the software in order to fix the errors behind the complaints made the management and new staff adopt the theme of "total customer satisfaction" for the software process. in both cases (company a and c) to improve quality became a matter of survival for the company. a different story was company d whose product is an application of seamless charting for marine navigation and mine mapping. the development team has extensive experience in electronic cartography application use and development. the software product is not a mass market product like a financial package to fit any small or medium sized business, but is highly sensitive to the quality of hardware peripherals attached to it, both of which strongly demand that the software is error free. due to problems in the relationship with the joint venture partner (which went into receivership) the company d was left alone for several months with the initial concept (given by the joint venture partner) of navigation software. as the business virtually stopped times were rough for the company d. because of the circumstances the company d was provided with a more or less welcome break from sales and marketing which allowed the small software development team to thoroughly test the software. after a some six months period they had a flawless piece of software ready to be marketed through a marine equipment retailer. the resulting application is a navigation software system built around the charting system, which is unique in the world and ideal for export. the export markets indonesia and canada which company d had chosen do not require formal quality certifications, but to be able to show such a certificate is almost the only way to convince export partners that the software is of high quality. the target is to establish export activity in one new market annually. if the quality management system enables the achievement of this target its implementation becomes justified. however, at the same time the management of the company d is very concerned that the system will stifle the creativity of the research and development activity which is at the core of their products success. the company is currently appointing new high quality computing experts to join the research and development team and one of the main concerns is how to keep their activities separate from the software processes covered by the quality management system. conclusions: next step is to implement measurements balancing between improved company image and stifled creativity it became obvious that the main reason for taking the steps towards the implementation of quality management systems and, subsequently, to become accredited, was to improve the company's image, eg. to be accepted as a quality supplier amongst potential clients and to assure a quality work force in the future. in the long run the quality management system was also seen to be representing an initial step towards total quality management and improved effectiveness of the company. in most cases the basis for the company-specific quality management systems was laid during the time of peaceful software testing or during vigorous debugging in order to keep the customers happy. 22 ajis however, the implementation of standards possessed a threat as it was seen to stifle the creativity needed for product innovations. this was causing some concerns among management as well as staff resistance. the limited nature of current quality management systems was also highlighted as it was observed that some of the business processes, eg. pre-sales activities and tender processing, all of which are important for the competitiveness in the long run and for commercial success in the short run were not covered. measurement is the key to continuous improvement the quality management systems should be seen as a step towards total quality management. tqm is about continuous improvement for which three components are needed: management commitment for improvement, motivation for improvement, and measurement for improvement (alter (1993)). if any of the three components is missing the organisation may not be delivering competitive quality. this study showed that any measurement for improvement is considered very difficult in the systems houses even though the companies do collect a variety of customer and development trouble reports. partly because of the qualitative nature of the information in such reports and partly because the main objective is to give enough information to the development team to help them fix the system according to the customer's complaints. although trouble reports are collated and kept in files they cannot easily be used for other purposes, eg. for collecting statistics of the nature of complaints and system modifications, without developing, a way to analyse and classify the complaints. we feel that a content analysis type of study of the trouble report files would provide ideas of an appropriate classification which would suit the company and support the management to control and assure the quality of systems development. analysis of value activities and their relationships the management understands too well that the money is not just magic, the tasks require a balanced combination of interpersonal skills, technical skills and a quality system. to define the optimal level of quality for each task, multiple perspectives are needed to appreciate the relative nature of quality. quality procedures in place enable the organisational change, as the quality manager of the company a put it: "quality culture is an evasive thing that will just develop over time, we put signs everywhere". two interrelated issues seem to inhibit the rapid expansion of quality improvement programmes. first, the market was not seen to be mature enough as the customers are not willing to pay for the increased quality which is something intangible and not influencing their day to day use of the system being supplied. secondly, the management's opinions about linkages between the quality management systems and business development vary from no relationship at all to vaguely supportive. we feel that sales cycle activities should be explored more to discover elements that have major impact on both the success rates of tenders and to provide hints about how customer requirements will be changing during the development process. generally the management agreed that pre-sales and research and development tasks should be incorporated in the quality management system, but they did not really have any suggestions on how to go about it. the main problem was: how to control the quality of these activities and how to measure the impact of those activities on customer satisfaction and on revenue development. furthermore, because these activities are less structured and involve hidden knowledge it is very difficult to describe them explicitly and in a way that they could be included in the company's qms. to make the qms support highly unstructured tasks such as sales and pre-sale activities or ongoing research and development, an analysis of the internal competencies of the systems houses should be carried out. the analysis should include a detailed evaluation of the relationship between business development and quality software processes. the value chain analysis-* which examines the flow of primary and supporting activities that lead to successful product and service provision would help the managers to direct their effort into areas that improve the competitiveness of the systems house. we feel that the resulting overall picture of the company's activities will smooth the way to total quality management. acknowledgment the author wishes to acknowledge the grateful support of all those executives from the six software houses in australia participating in this research and thanks the department of industry, technology and regional development for initiating this research and supporting it in many ways. 5 see partridge & perren (1993), which discusses how porter's (porter, competitive advantage, 1985) value chain analysis can be applied to service industries. september 1995 23 references alter, steven (1993), 'tqm in the is function," tutorial at the icis'14, orlando, florida, december 1993. couger, j. daniel & higgins, lexis f. & mclntyre, scott c (1993), "(un)structured creativity in information systems organisation", mis quarterly, december 1993, pp. 375-397. douglas, d. p. & walsh, l. (1993), 'tqm in is: defining user requirements", i/s analyzer, may 1993, vol. 31, no. 5. dromey, geoff (1990), "a national software industry initiative", 1990. edestein, v. & fuiji, r. & guerdat, g. & sull, p. (1992), "internationalizing software engineering standards", computer, march 1992, pp. 74-75. von hellens, l. a. (1991), conditions for the success in the design and implementation of packaged software. a study of accounting packages for small companies in the united kingdom, d.phil thesis, templeton college, university of oxford, 1991. humphrey, w. s. (1988), a method for assessing the software engineering capability of contractors, carnegie mellon university, software engineering institute, february 1988. humphrey, w. s. & kitson, d. h. & gale, j. (1991), "a comparison of us and japanese software process maturity", 13th icse austin tx, may 1991, pp. 38-49. ould, martyn a. (1990), strategies for software engineering, the management of risk and quality, john wiley & sons, 1990. partridge, mike & perren, lew (1993), "achieving competitive advantage", management accounting, november 1993, pp. 37-38. porter, michael (1985), competitive advantage, free press, 1985. price waterhouse (1988), "software quality standards: the costs and benefits. a review for the department of trade and industry", april 1988. smilie, t. (1993), "the software quality target", informatics, april 1993, pp. 18-22. standards australia (1993), "voting advice on iso/eec dis 12119, information technology software packages quality requirements", october 1993. verner, j. & daneshgar, f. (1992), "quality of software development process: australia needs to take notice", quality australia, vol. 9 no. 4. pp. 8-10, 1992, pp. 8-10. 24 ajis ajis vol. 4 no. 2 may 1997 reassessing function points g.r. finnic', g.e. wittig' and j-m. deshamais" * school of information technology bond university, gold coast queensland 4229, australia " software engineering laboratory in applied metrics 7415 rue beaubien est, suite 509 anjou, quebec, canada him 3r5 abstract accurate estimation of the size and development effort for software projects requires estimation models which can be used early enough in the development life cycle to be of practical value. function point analysis (fpa) has become possibly the most widely used estimation technique in practice. however the technique was developed in the data processing environment of the 1970's and, despite undergoing considerable reassessment and formalisation, still attracts criticism for the weighting scoring it employs and for the way in which the function point score is adapted for specific system characteristics. this paper reviews the validity of the weighting scheme and the value of adjusting for system characteristics by studying their effect in a sample of 299 software developments. in general the value adjustment scheme does not appear to cater for differences in productivity. the weighting scheme used to adjust system components in terms of being simple, average or complex also appears suspect and should be redesigned to provide a more realistic estimate of system functionality. introduction estimating the size of a software project, and hence the project cost and development effort, remains a difficult problem. considerable research and practical effort has gone into developing models and methodologies to assist in estimation, for example cocomo, slim, estimacs, function point analysis, (albrecht, 1979; albrecht & gaffney, 1983), spans, checkpoint, (jones, 1991) and costar, (ferens & gurner, 1992). of these, function point analysis (fpa) has possibly the widest use in practice (dam & langbroek, 1992; dreger, 1989; jones, 1991). suitable estimation models enable a realistic assessment of size early in the systems development life cycle (sdlc). metrics such as lines of code (log) are biased by implementation details and are difficult to assess accurately early in the sdlc. fpa operates by trying to measure the somewhat loosely defined concept of functionality. in fpa functionality is measured by determining the number of inputs, outputs, inquiries, internal files and external files in the target system. each of these is scored for complexity and weighted accordingly. the total score gives the number of unadjusted function points (ufp) which is then adjusted by a factor composed of 14 general system characteristics (gsc's) which attempts to compensate for differences in complexity between software projects. fpa was developed in the data processing environment of the 1970s, (albrecht, 1979; albrecht & gaffney, 1983), and has become more formalised over time. groups such as ifpug (ifpug cpm, 1994), have developed extensive guidelines for computing function points (fp's). however the underlying basis of weighted scoring (and weight values) and the use of specific gsc's (albrecht & gaffney, 1983), remains unchanged. the technique has attracted criticism on these grounds (abran & robillard, 1994; kemerer,1987), and attempts have been made to provide alternative functionality based metrics, for example the symons mkll function points method (symons, 1991). this paper describes a critical assessment of the weighting scheme used in fpa and the role and value of the gsc adjustment, and follows on some earlier work done by desharnais (1988). several analyses were performed on a sample of 299 software developments from 17 organisations. some prior research on fp metrics a function point (fp) count of a system measures two components. these are firstly the information processing size, expressed in unadjusted ufp's, and secondly the technical complexity measure, expressed as the gsc's. the information processing size is determined by categorising a system into 5 function types. these are external inputs, outputs, inquiries, internal logical files and external interface files. each of these is further classified as either low, average, or high functional complexity, depending on the number of data element types, and other factors. each classification for each function type is allocated a number of points or weights, and the sum for all components is expressed as the number of unadjusted function points. table 1 shows the matrix of the points allocation for each classification. 39 ajis vol. 4 no. 2 may 1997 table 1 function point allocation description low external input 3 external output 4 external inquiry 3 external interface file 5 internal file 7 average 4 5 4 7 10 high 6 7 6 10 15 as an example, a system with 6 simple inputs, 4 average outputs, 3 complex outputs, 2 average internal files and a complex external interface file would have a total ufp count of: 6x3 + 4x5 + 3x7 + 2x10 + 1x10 = 89 ufp the treatment of complexity is somewhat subjective, but its determination, using the 14 gsc's is supported by guidelines for interpretation (ifpug cpm, 1994). the value adjustment factor (vaf) is derived by the following equation: vaf = (tdi x 0.01) + 0.65 where tdi is the total degree of influence as determined by the gsc's. the degree of influence is determined by 14 complexity factors. each of these is evaluated on a scale of 1 to 5, depending on the degree to which the factor is present in the system. a 0 would be allocated if the factor was absent, while a 5 would be allocated if the factor exerted a strong influence throughout the system. the other values fall between these two extremes (decimal values are permitted), depending on their degree of influence. the sum of all 14 factors determined in this way is termed the total degree of influence (tdi). symons (1991) in his effort to overcome some difficulties associated with fpa has proposed his mkll function point model. the information processing size is expressed in unadjusted function points, but is now the sum of the weighted number of input data element types, the weighted number of entity-type references, and the weighted number of output data element types. symons scaled the mkll weightings so that for the 8 systems he examined, the average size of unadjusted function points were in the same range as albrecht's fpa model (albrecht, 1979). in addition symons expanded the number of factors affecting the technical complexity from 14 to 20. symons also questioned the weight of each degree of influence, and has suggested that this should vary with technology. research has not proved conclusively that symons' mkll model is a better size metric enabling more accurate estimates to be made than when albrecht's fpa model is used. results of research done by ratcliffe and rollo (1990) are inconclusive. to improve on the original function point counting technique various other adaptations have also been suggested (jones, 1991; reifer, 1990). in a research study macdonell (1994) identified and evaluated nine function-based assessment and estimation methods. data used this database is the result of a compilation of 299 projects from 17 different organisations (desharnais et al, 1990). the standard deviation and the skewness of the data suggests the possible presence of outliers, but none of these were excluded from the analysis. table 2 summary of project data effort ufp fp duration ufp/hour description effort in hours unadjusted fp adjusted fp duration in months productivity mean 7086 298 267 14 0.071 min. 247 48 40 1 0.008 max. 86478 1257 1182 67 0.696 skew. 4.75 1.81 1.86 2.17 3.99 # cases 299 299 299 201 299 the total development effort in hours is somewhat skewed towards smaller projects but still covers a range typical of commercial developments. the productivity variation from 0.008 ufp/hr. to 0.696 ufp/hr. (87 times) is also typical of the type of variation possible in systems development and is why software effort estimation remains such a difficult problem. 40 ajis vol. 4 no. 2 may 1997 analysis several different types of analysis were performed to determine how effective the use of gsc's and the weighting scheme was. regression analysis was used to assess the effectiveness of fpa for predicting development effort i.e. how accurate different regression models were in estimating total development time in effort hours. anova and t-tcsts were used to assess whether the gsc's were performing effective adjustments to cater for differences in productivity between projects i.e. were there any significant differences in productivity between system developments with high or low ratings in specific gsc's. logically it could be expected that projects requiring extensive use of e.g. data communications might have lower productivity than projects without this complexity. factor analysis was also performed to determine the covariance of the gsc's i.e. how separable are the general system characteristics as defined by fpa. t-tests were used to determine whether there were any significant differences in productivity between projects with different proportions of inputs, outputs, inquiries, internal files and external files. if the fpa model is valid it should make no difference in terms of productivity if a project has a relatively high proportion of e.g. inputs as opposed to one with a low number of inputs. regression analysis the sample of 299 was divided randomly into 249 training cases and 50 test cases. regression was performed on the 249 training cases to extract models to estimate development effort based on unadjusted function points, function points (adjusted), log-linear transformations of these and development effort as well as two multiple regression models based on ufp with the 14 gsc's as input as well as log-linear ufp with the 14 gsc's. (since the gsc's are used to compute fp's from ufp's there is no need to perform multiple regression with fp and the gsc's). log-linear models were investigated to assess whether these compensated for the effect of system size as productivity is generally known to decrease with increasing system size. these models were then used to estimate development effort in the other 50 cases. the quality of the estimation is based on two measures, the mean absolute relative error (mare) and the proportion of the estimate within 25% and within 50% of the actual development effort. the results are summarised in table 3 and discussed in the results section below. table 3 regression analysis results ufp fp ufp log-linear fp log-linear ufp + gsc's log ufp+gsc's mare 1.002 0.903 0.790 0.733 1.234 0.623 <25% 34% 30% 40% 40% 24% 36% >25% <50% 28% 30% 22% 28% 26% 36% >50% 38% 40% 38% 32% 50% 28% the predictive accuracy of fpa with regression models clearly shows considerable error and even in the best case only provides 40% (20 out of 50 cases) of the test cases with an effort prediction within 25% of the actual. the best model overall (with an mare of 0.623) is a log-linear multiple regression model using unadjusted function points and the 14 gsc's. other methods such as the use of artificial neural nets (anns) and case based reasoning (cbr) have been investigated by the authors (finnic & wittig, 1996), and could provide more accurate models. however the underlying questions concerning the validity of the construction of fp's remain a serious factor (abran & robillard, 1994). using general system characteristics gsc's are intended to adjust the function point count to compensate for differences in project complexity e.g. a project which makes extensive use of data communications could be expected to require more development effort for the same function point count as a project which requires little data communication capability. they have been criticized on the grounds that they do not adequately cover the spectrum of issues which influence complexity and because the effect of any single complexity factor is no more than 5% of the total estimated effort. since gsc's adjust for complexity there should be no difference in productivity (measured in fp's per hour) between those projects which rank high on a particular complexity factor as opposed to those 41 ajis vol. 4 no. 2 may 1997 which have a low score. two analyses were performed. firstly the full sample of 299 cases was partitioned for each gsc into those with a low score on the factor (between 0 and 2) and those with a high score (3-5). t-tests were used to determine whether there was any significant differences in productivity (as measured in both ufp/hr and fp/hr) for each factor. the results are given in table 4 and are discussed in more detail in the results section. however it would appear that in general the gsc weights are insufficient to compensate for differences in productivity which might be due to the specific gsc. for example, in the "multiple sites" gsc the differences in productivity between the ufp and fp cases remains significant i.e. use of the complexity gsc did not provide sufficient weight to override the effect of complexity. on the other hand several gsc's did not appear to be a significant factor in productivity (e.g. heavy use) and their value in adjusting the function point count is questionable. table 4 differences gsc data communications distributed system performance heavy use transaction rate on-line data entry end-user efficiency on-line update complexity reuse installation ease operational ease multiple sites facilitate change in productivity ufp t-value -0.99 1.33 3.59 1.00 1.62 -2.165 1.88 1.00 3.50 1.17 1.10 1.37 4.42 4.27 by gsc (t-test) ufp probability 0.16 0.09 .0002 .159 0.05 0.02 0.03 0.16 0.0003 0.12 0.14 0.09 0.00001 0.00001 fp t-value -2.91 0.69 1.78 -0.68 -0.29 -4.04 -0.17 -1.10 2.36 0.002 0.02 0.05 2.97 2.56 fp probability 0.002 0.25 0.04 0.25 0.39 0.00004 0.43 0.14 0.009 0.50 0.49 0.48 0.002 0.005 table 5 differences in productivity by gsc (anova) gsc data communications distributed system performance heavy use transaction rate on-line data entry end-user efficiency on-line update complexity reuse installation ease operational ease multiple sites facilitate change ufp f-value 0.54 0.72 2.51 1.93 1.65 1.07 2.52 1.05 6.57 0.67 2.85 3.61 2.27 4.55 ufp probability 0.75 0.61 0.03 0.09 0.15 0.38 0.03 0.39 0.0001 0.65 0.002 0.003 0.05 0.001 fp f-value 1.14 0.70 0.60 1.27 0.86 2.34 0.66 0.23 4.35 0.56 1.25 1.12 1.89 2.73 fp probability 0.34 0.63 0.70 0.28 0.51 0.05 0.65 0.95 0.001 0.73 0.28 0.35 0.09 0.02 in addition anova was used to determine whether any significant differences existed between the groups for each gsc. the sample was partitioned into 6 groups for each gsc based on the score for the specific factor i.e. from 0 to 5. the analysis was performed by controlling for the effect of differences in the number of inputs, outputs, inquiries, internal files and external interface files. this analysis was not as useful as the first as the variable sample sizes and within group variance tended to reduce the prospects of determining any significant differences. the results are given in table 5 and discussed in the results section. these results again indicate that the value of gsc's in adjusting for productivity differences is low. factor analysis of gsc's the 14 gsc's have been criticised for ambiguity and incompleteness. factor analysis was used to assess how strongly the various factors belonged together i.e. could be interpreted as being part of one component or factor. spss factor analysis with varimax rotation was used to extract solutions for two, three and four factors. of these the two factor solution appears to be the best as the others tend to share 42 ajis vol. 4 no. 2 may 1997 a number of variables. the two factor weights are given in table 6. for this data set it appears that there is considerable covariance and that a number of the factors are difficult to separate. table 6 : factor analysis of gsc's factors data communications distributed system performance heavy use transaction rate on-line data entry end-user efficiency on-line update complexity reuse installation ease operational ease multiple sites facilitate change eigenvalue percent of variance factor 1 0.11 0.13 0.77 0.53 0.71 0.11 0.57 0.37 0.70 0.26 0.60 0.59 0.01 0.54 4.95 35.3% factor 2 0.84 0.31 0.19 0.40 0.33 0.81 0.57 0.73 -0.29 0.26 0.24 0.06 0.52 0.27 1.67 12.0% analysis of factor weights the unadjusted function point total is computed from the weighted score of simple, average and complex inputs, outputs, inquiries, internal and external files. the scoring scheme is given in table 1 and was determined by albrecht on the basis of "trial and debate". if the weighted scoring scheme adequately compensates for the differences in functionality (and hence development effort) between inputs, outputs, etc. and between simple, average and complex variants of these, there should be no difference in productivity (in ufp/hr) between projects with differing proportions of each component. for example, a project with a high ratio of input functionality to the total unadjusted function point score should (subject to gsc differences) have no difference in productivity to one with low input ratio and a high external interface file ratio. the ratio of the weighted input score to the total ufp count was computed. the median of this ratio was determined for the sample and the total partitioned on the median. a simple t-test was performed to see if any significant differences in productivity were present. this was repeated for the outputs, inquiries, internal files and external interface files, as well as for the total of input, outputs and inquiries. in addition the ratio of inputs, outputs and inquiries to external and internal files was also computed. the results are given in table 7 and clearly indicate that the relative proportion of different components in a system development is a very significant factor in productivity. for example, systems with a high proportion of inquiries in the system had far lower productivity than those with a low proportion of inquiries. it is apparent that the scoring system in no way compensates for this effect and that the weighting scheme used in function point analysis is in need of some reassessment. table 7 analysis of factor weight results ratios loi/files input/total output/total inquiries/total internal files/total external files/total loi/total tstatistic -3.18 -2.33 1.51 -4.93 1.72 2.17 -3.50 probability 0.0008 0.01 0.06 8.8*e-07 0.04 0.02 0.0003 results in general the results support the concerns about the value of gsc's and the weights used in computing unadjusted fp's. using multiple regression with fp's (table 3) was not particularly successful in accurately predicting development effort. the results are similar to those obtained by other researchers (jeffery, 1987). 43 ajis vol. 4 no. 2 may 1997 study of table 4 suggests that the gsc's adjustments are not adequate to compensate for the effect of each factor on productivity. in the unadjusted function point case only seven of the gsc groups show significant differences in productivity. for the other seven it is possible that within group variance has exceeded between groups variance so that the t-test is inconclusive or it may indicate that there is in fact no significant difference in productivity for the other groups. in five of the seven cases the use of the adjusting factors (as indicated by the significant differences for the adjusted function point column) did not negate the effect of productivity differences sufficiently to remove any significant differences between the groups. in two cases the use of the technical complexity adjustment in fact worsened the situation i.e. for on-line data entry and for data communications. table 5 is not very conclusive as the within group variance (for 6 groups) has probably exceeded the between group variance and makes it difficult to sensibly interpret the results. only 6 factors show significant differences in productivity for unadjusted fp's. of these, three remain significantly different while on-line data entry becomes a significant difference. the results suggest again that the gsc's reduce the productivity differences but do not eliminate them. the factor analysis show one strong factor which probably reflects general complexity (factors such as performance, heavy use, transaction rates, end-user efficiency, complexity, installation ease, operational ease and facilitate change) and a second factor which appears to relate to on-line distributed system use (factors such as data communications, heavy use, on-line data entry, end-user efficiency, on-line update and multiple sites). the fairly strong relationship between some of the components of each factor again suggests that the gsc's are not easy to separate in practice. table 7 shows very significant differences in productivity between groups with high and low ratios of a specific ufp component e.g. the difference in productivity between projects with a low ratio of inputs to the total unadjusted function point score and those with a high ratio is significant at the p <0.01 level. with the exception of outputs all other components show significant differences (outputs has t = 1.51 and p=0.06). this strongly suggests that the weighting scheme applied, at least in this sample, is not compensating for the differences in productivity between developing, for example, an average input and a complex inquiry. conclusions for the large data set analysed the results support the view that the use of fp's to estimate project size and hence development effort has a number of problems. the vaf appears to be inadequate and different methods of adjusting an estimate to account for complexity need to be devised, particularly in the light of new development methodologies and environments. the fundamental inputs to the fp count i.e. the inputs, outputs, etc. do not reflect the differences in development effort for different types and levels of components. abran and robillard (1994) found that fp's do not derive from a well-defined and proven theory, and they are entirely empirically based on expert opinion. they identified the existence of implicit transformations and implicit models without which the measurement process would be invalid. the results from the study in this paper would appear to indicate that both the type of component and the weighting scheme used should be reassessed, in addition to clarifying some of the implicit fp mappings. references abran, a., & robillard, p.n. (1994) function points: a study of their measurement processes and scale transformations, journal of systems and software, vol. 25, pp. 171-184. albrecht, a.j. (1979) measuring development productivity, proceedings of the joint share/guide/ibm application development symposium, pp. 83-92. albrecht, a.j., & gaffney, j.e. (1983) software function, source lines of code, and development effort prediction: a software science validation, ieee transactions on software engineering, vol. 9, no. 6, pp. 639-648. dam, j.v.y. & langbroek p.l. (1992) gebruik van functiepuntanalyse vraagt on beleid, informatie, vol. 34, no. 6, pp. 323-333. desharnais, j.m. (1988) analyse statistique de la productivity des projets informatiques a partir de la technique des point de fonction, master thesis, universite du quebec a montreal. desharnais, j.m., et al. (1990) adjustment model for function points scope factors a statistical study, ifpug spring conference, florida. dreger, j.b. (1989)function point analysis, prentice hall, englewood cliffs. ferens, d.v., & gurner, r.b. (1992) an evaluation of three function point models for estimation of software effort, ieee national aerospace and electronics conference naecon92, vol. 2, pp. 625-642. 44 ajis vol. 4 no. 2 may 1997 finnic, g.r., & wittig, g.e. (1996) ai tools for software development effort estimation, proceedings of the conference on software engineering : education and practice, university of otago. function point counting practices manual, release 4.0 (1994) international function point users group, blendonview office park, 5008-28 pine creek drive, westerville, oh 43081-4899, usa. jeffery, d.r. (1987) time sensitive cost models in the commercial mis environment, ieee transactions on software engineering, vol. 13, no. 7, pp. 852-859. jones, c. (1991) applied software measurement, mcgraw-hill, new york. kemerer, c.f. (1987) an empirical validation of software cost estimation models, communications of the acm, vol. 30, no 5, pp. 416-429. macdonell, s.g. (1994) a comparative review of functional complexity assessment methods for effort estimation, discussion paper series, university of otago, no. 94/8, issn 1172-6024. ratcliff, b., & rollo, a.l. (1990) adapting function point analysis to jackson system development software engineering journal, pp. 79-84. reifer, d.j. (1990) asset-r: a function point sizing tool for scientific and real-time systems, journal of systems software, vol. 11, pp. 159-171. symons, c.r. (1991) software sizing and estimating mkll fpa, john wiley & sons, chichester. 45 the australasian journal of information systems (ajis) began in 1993 and was based on the simple realisation that there was a need for an academic ‘outlet’ for is research in australia australasian journal of information systems volume 18 number1 2013 5 barriers and motivations affecting information systems usage by hajj–umrah religious tourism operators in saudi arabia hani brdesee rmit university haniid@hotmail.com brian corbitt rmit university brian.corbitt@rmit.edu.au siddhi pittayachawan rmit university siddhi.pittayachawan@rmit.edu.au abstract hajj & umrah religious tourism accounts for seven million visitors each year in saudi arabia. the government has recently taken initiatives to promote the use of information systems (is) in the religious tourism industry, encouraging firms to adopt is innovations like e-commerce and enforcing the use of the makha’a information system in umrah for external pilgrims and the yosr information system in hajj for internal pilgrims. this study outlines the motivations and challenges that affect the utilisation of various is services in the saudi religious tourism industry through a qualitative analysis of the views and perceptions of senior executives and owners of travel firms. the findings suggest that while government initiatives and industry competitiveness were two positive factors promoting is use, there are some major barriers preventing private firms from fully utilising the advantages of information systems. these include external factors such as lack of support from the it industry and access to it resources as well as internal factors within an organisation such as lack of commitment or the need for professional it expertise. the study finds that relative advantage is a critical contributor to is utilisation which depends on information systems characteristics. keywords: hajj umrah, religious tourism, information systems use, technology adoption, saudi arabia introduction this research investigates the usage of information systems (is), particularly e-business through ecommerce and e-management systems, in the religious tourism industry in saudi arabia. with the growing tourism potential of the country and the annual hajj–umrah pilgrimage that draws in millions of visitors, e-business has come to be seen as a valuable medium for boosting efficiency and profitability in the tourism industry. efficiencies in is applications in marketing practice can help the tourism industry achieve cost reduction and greater productivity, (ep-moh 2012). the aim of this research is to identify the motivations and barriers that influence the use of is by religious tourism operators in saudi arabia. this research will clarify the reasons behind the low level of is utilisation in tourism firms and help policymakers and industry leaders to undertake measures that can overcome the existing barriers and take further steps to amplify the motivating factors that promote is use. mailto:haniid@hotmail.com mailto:brian.corbitt@rmit.edu.au mailto:siddhi.pittayachawan@rmit.edu.au australasian journal of information systems volume 18 number1 2013 6 the paper also adds to the understanding of technology adoption in arab countries by drawing upon generic theories of technology change and using their insights to study the issue of is utilisation in the specific cultural locale of saudi arabia. although, there is now a greater diversity of literature available on is research and online marketing, researchers still tend to derive their theories and models from the work in the united states, united kingdom and europe. a west-centric bias has tended to dominate the assumptions regarding the mores of international trade and theories on different aspects of business research in academic literature. this study examines a global phenomenon of trade and technology from a standpoint of a non-western context in its own traditions. while using a generic theory of technology adoption derived from west-based academic literature, this localised focus on saudi arabia will also highlight some of the context related dimensions. research background with the red sea to the west and the arabian gulf to the east, the kingdom of saudi arabia occupies the greater part of the arabian peninsula. the country’s population is estimated to be 26.534 million in july 2012 including the expatriate population in the country which ranges from 5.5 to 6.5 million (world factbook 2012). arabic is the native language and english is the lingua franca for social occasions and business purposes. the country is the birthplace of islam and all saudi nationals are muslims and those wishing to seek citizenship must convert to islam. islam is the basis of the saudi society and its constitution, legal system, government and commercial transactions. on a social level, a devout islamic lifestyle together with arab customs and traditions forms the foundation of saudi business culture. the bulk of tourist traffic comes to saudi arabia for the two main religious pilgrimages, hajj and umrah. hajj is an annual pilgrimage that occurs from the 8 th to 12 th day of dhu al-hijjah, the 12 th and last month of the islamic calendar. it commemorates the exodus of the prophet mohammed ( محمد صلى ,and his followers from the holy city makkah to the holy city medinah in the year 622 ce (هللا عليه وسلم marking the beginning of the hijra or the islamic calendar. it is obligatory upon every physically and financially capable muslim, male and female, to make the hajj pilgrimage at least once during their lifetime. in the last few decades, up to four million people converge every year on makkah from around the world to attend the hajj. there is a subsidiary pilgrimage called umrah, which may be taken anytime during the year. this pilgrimage is not dedicated to any sacred event or historical date but covers a general tour of holy sites in makkah and madinah as well as all other places of historical antiquity and religious importance in this vast area. pilgrims visit the holy mosques in makkah and madinah, complete the devotional act of seven rounds called sai, and other seven circumambulations called tawaf around the kabah (shrine), (ministry of hajj 2012). while some people automatically include the umrah in their hajj schedule, others consider umrah as an independent ritual that may be performed separately. information systems use and the religious tourism industry in the preceding decades, public or private sector organisations frequently lacked access to it services. however, recently, the government has taken steps to liberalise the it market and involve the private sector in this industry to raise the level of internet use and computer penetration rate to 30% by 2013 (bmi 2009). the saudi private sector is responding to the it challenge by vigorously adopting new technology and investing in the it sector, (aldogily 2009; alzhrani 2009). with its fast-growing population, rising wealth index and growing industry, analysts consider saudi arabia a australasian journal of information systems volume 18 number1 2013 7 market with great potential for it, (bmi industry report 2012). despite the 2008–2009 economic slowdown, the kingdom continued to grow as a market for technology products and services as it continued to invest in upgrading its it infrastructure, and deploying high-speed internet or mobile transmission facilities (bmi 2009). the saudi arabia it report by business monitor international states that annual per capita expenditure on it reached $us330, accounting for around 40% of it spending in the gulf region. saudi arabia has the largest it market in the gulf region, with a forecast value of us$3.5bn in 2009 expected to rise to us$4.8bn by 2013. in terms of is utilisation for business, the arab advisors group, in its 2009 report, found that saudi arabia is leading the region in e-business, registering a record sar12 billion transactions in the kingdom in 2008 (tradearabia 2009). the report further shows that 3.5 million internet users, or one-eighth of the saudi population, were engaged in e-commerce transactions. in the last few decades, information and communications technology (ict) have evolved in saudi arabia from wired to wireless hardware, passive to active software, delivering competitive edge to businesses from diverse sectors of the industry, (bmi industry report 2012). now, information systems have become the backbone of many businesses, emerging as the primary vehicle for communicational needs ranging from information storage, human interaction to data transfer. apart from all these general utilities, e-business and online management of marketing and business operations, is an is service that holds great promise for the commercial sector in increasing productivity, spreading customer-base and easing logistical difficulties of conducting transactions, (turban et al. 2010). the hajj–umrah religious tourism industry in saudi arabia could gain considerable benefits from ebusiness and is technologies, (masoud 2005; ep-moh 2012; brdesee et al. 2011; brdesee et al. 2012). the religious travel and pilgrimage industry in the country is a complex system built up over centuries, but integrating it efficiencies into its business model could help improve efficiencies and better serve their global clientele of 21st century muslim pilgrims. the industry can use is/e-business portals to provide prospective pilgrims reliable information on the internet and enable customers to make bookings and complete their transactions swiftly for all travel needs from transport to accommodation. complexity in travel information, logistical difficulties in communications with the firm or means of payment will simply push potential clients to search for different providers, (pavic et al. 2007; hooshang et al. 2006). the organisations can resolve this impasse and market their products to a global or local market of potential clients who are long used to selecting travel services and purchasing them online. as religious travel in saudi arabia involves millions of visitors from all over the world, it is controlled by the government for security and logistics reasons. the government controls and monitors the entry of visitors into the country for pilgrimage, (moh 2012). effective is utilisation in the sector can not only help the government provide an efficient marketing/booking channel but act as an effective monitoring system for all pilgrims in the country. to this end, the government has been motivating providers to use information systems hajj–umrah needs, which are then sold by several it providers, (ep-moh 2012). to explain further, the hajj–umrah industry operates in four basic modes, (moh 2012):  hajj for local muslims: the ministry of hajj in saudi arabia offers licenses to about 40 companies for local hajj pilgrims, which include saudis and muslim expatriates living in saudi arabia.  hajj for external muslims: the ministry of hajj in saudi arabia fully controls the arrangements offered by local providers like (twafaa establishments, مؤسسات الطوافة) in makkah, (w’okala office, مكتب الوكالء الموحد) in jeddah. australasian journal of information systems volume 18 number1 2013 8  umrah for local muslims: this group of umrah packages are meant for saudis and muslims expatriates living in the country and here any pilgrimages can retail packages on its own the whole year round.  umrah for external muslims: the ministry of hajj in saudi arabia licenses about 20 companies as local agents to arrange umrah for external pilgrims through international agents. for the purpose of this research, the authors intend to focus on travel firms that conduct two of the above services, namely, hajj for local muslims and umrah for external muslims. these two packages involve local travel agents who must make use of the current is and e-business portals to deliver services. the other two groups are excluded because they do not involve local firms or mandatory use of is systems. in the case of hajj for external pilgrims, packages are directly administered by the government and providers such as twafaa establishments. for non-muslim countries, there is special arrangement between travel agents in these countries and the ministry of hajj in saudi arabia, for example, australian travel agents can provide regular escorted umrah travel and hajj packages (ourworld 2011). yosr is a common booking system used for arranging hajj for local pilgrims. it allows these organisations to register pilgrims, cater for their accommodation and transportation needs, and finalise payments (yosr 2012). the procedure for umrah for external pilgrims is much more complex. there is a main is portal called the (makha’a, ُمخاع) which links all the stakeholders, internal licensed umrah agents, and external travel agents with the ministry of hajj, and the ministry of the interior in one information system, (ep-moh 2012). there are five makha’a internet providers and about 40 umrah agents licensed by the ministry of hajj. the government requires designated umrah pilgrimage providers to work online with their international dealers because local travel agents are not permitted to offer umrah services directly to the external pilgrims, (ep-moh 2012). the umrah operator begins by creating an umrah package at a fixed price. umrah agents are required to administer to the needs of all pilgrims from the time of their arrival at jeddah airport until the end of their pilgrimage, (ep-moh 2012). the package offered must address all travel needs, including accommodation, transportation, catering, and additional services such as medical insurance, (ep-moh 2012). this is then sent to the ministry of hajj for approval through makha’a. when the moh approves the package, the umrah operator offers the package to its external agents outside saudi to start retailing it to pilgrims. when the external agent has acquired customers, it sends their information to makha’a for further processing through the umrah operator. the makha’a will then send this information to the moh to check the data of individual passengers and give each passenger a ministry of foreign affairs (mofa) approval number, which will be used to issue a visa from the saudi embassy. the external agent will then be notified through makha’a to collect the visas from their local embassy for the accepted passengers of the group (bab umrah 2012). literature review to thrive in any market, most organisations seek competitive advantage through superior products and services, cost controls and competitive pricing, and innovation, (pavic et al. 2007; hooshang et al. 2006). information system solutions have emerged as a primary means for solving a whole range of operational needs from communication to data storage, and in particular, e-business portals have emerged as a cost-effective medium for sales and marketing. molla and licker (2005, p.90) note that “organizations that perceive market forces as ready for e-commerce, as an example for information systems, are likely to adopt e-commerce or embark on more sophisticated e-commerce australasian journal of information systems volume 18 number1 2013 9 implementation from fear of competitive disadvantage or perceived e-commerce benefit”. kshetri (2007) notes, however, that organisations from emerging economies tend to find economic, sociopolitical and cognitive issues with implementing e-business. these include issues relating to legal and financial security and cognitive barriers that relate to knowledge, skill and confidence in using is. hence the issue of technology adoption has become a subject of extensive research in marketing and business literature in the last few decades, (fishbein and ajzen 1975; davis et al. 1989; tornatzky and fleischer 1990; davis & venkatesh 1996; molla and licker 2005, selim 2008; suebsin and gerdsri 2009). theoretical modeling is used to explore the acceptance rate for new technologies and many factors affecting ict adoption can be extracted from these tested models. one such model innovation diffusion theory (idt) was postulated by rogers (1962) to explore the factors affecting the adoption or rejection of innovative techniques. the theory proposes that five major factors impact the diffusion of an innovation: relative advantage, compatibility, complexity, trialability and observability. another model called the theory of reasoned action (tra), posited by fishbein & ajzen (1975), focuses on the subject’s intention to act by measuring his propensity to adopt or reject a particular technology. tra posits that the beliefs and evaluations of the subject determine the user attitudes toward the matter under review (ajzen and fishbein 1980). tam or technology adoption model is another important theory in technology adoption developed by davis et al. (1989). it posits perceived ease of use and perceived usefulness as the two important factors in explaining system usage where perceived ease of use is defined as the extent to which a person believes that using a given technology will save effort and labour and perceived usefulness is the extent to which a person believes that the relevant technology will enhance performance of a task. the adoption of technology within an organisation occurs at two levels: primary adoption within the organisation as a whole, and at the secondary level as it is taken up on an individual basis (suebsin and gerdsri 2009). some scholars find that technology adoption theories such as technology acceptance model (tam) should focus on individuals’ behaviour, not the behaviour of an entire organisation (davis et al. 1989). but when considering the adoption of is in an industry as a whole and the manner in which technology emerges as a driver for change within an organisation, there is need to encompass systemic and structural reasons along with individual opinions. the technology, organization, and environment context (toe) framework developed in 1990 by tornatzky and fleischer (1990) takes this broader perspective by examining the overall contextual basis for the adoption of a technology in a particular location. it identifies three aspects of an enterprise's context that can influence the process by which it adopts and implements a technological innovation: technological context, organizational context, and environmental context. the toe framework as originally presented and later adapted in it adoption studies, presents an approach that shows greater cognizance of the effect of context on technology adoption. it provides a useful analytical framework for studying the adoption and assimilation of different types of it innovation in a specific locale. this study also holds the primacy of contextual factors over individual predilection of executives in matters of technology adoption. the most immediate and compelling reasons for adopting technologies are initiated within a company’s own market positioning, strategic vision and operational needs. business models, objectives and operations vary from one company to another, even in the same industry and this will affect plans regarding is adoption as well. organisation’s characteristics, such as size, vision and planning (ke and wei 2008) as well as subjective factors such as the level of commitment, customer service and leadership by managers affect the use of is (nusair et al 2011; singh et al. 2009). a good business plan in information system use facilitates online strategies and allows the firm to assess and measure its progress and adjust to a new technology or a changing business environment. researchers have also emphasised the role that certain factors external to the business can exercise in promoting the adoption of a technology. governments can australasian journal of information systems volume 18 number1 2013 10 play a crucial role in providing policy, regulations and incentives to encourage private sector to respond to technological change (molla and licker 2005). others emphasise the importance of a spectrum of industry-based motivations, such as strategic positioning, supply chain logistics and information-sharing, in motivating organisations to respond to technology change (piris et al. 2004; claycomb et al. 2005). furthermore, access to sufficient human and technology resources in the national market and their cost-effectiveness has an impact on technology adoption by businesses and enterprises (ko et al. 2008). as is apparent, researchers tend to cluster these larger structural factors of technology change into variables external to the organisation (industry, government) and internal variables (organisational culture, resources, staff motivation); (molla and licker 2005). recent research in saudi arabia many scholars have studied the challenges in is utilisation as well as the motivations that have promoted their adoption in several areas like education, business and management in saudi arabia (alkahtani, 1998; al-gahtani 2003, al-gahtani et al. 2007; agourram 2009; al-zharani 2011). algahtani (2003) conducted an extensive study on behavioural attitude to is use of about 1200 knowledge workers in 56 public and private medium and large organizations across saudi arabia. using fishbein & ajzen’s theory of reasoned action they examined how attributes of relative advantage, compatibility, complexity, trialability and observability affect is use and found that these attributes explain up to 87% of the innovation rate of adoption. in a more focussed small-scale study, agourram (2009) explored how is success is defined and perceived by a group of managers in a university in saudi arabia using hofstede’s (1980, 2001) cultural dimensions. it found that culture does influence perceptions of is success which poses problems particularly to organizations that decide to implement enterprise resource planning systems (erp). al-gahtani et al. (2007) conducted a comparative study of it acceptance in north america and saudi arabia using a modified model of tam called the unified theory of acceptance and use of technology (utaut). utaut examined the influences of select antecedent social influence and cognitive instrumental constructs on perceived usefulness and usage intentions (venkatesh, morris, davis & davis 2003). hofstede (1980, 2001) cultural dimensions were also used to show the degree of difference between north american and saudi arabian adoption of it. the authors found that some aspects of behavioural intention and deterrents to usage had real impact on it adoption in saudi arabia. a study by al-zharani (2011) examined the perceptions and difficulties of is implementers at both government and private sectors. lack of coordination within and between saudi organisations as well as issues associated with technical, behavioural and structural factors were found to impede is implementation. as apparent from this review, several studies have been conducted on different aspects of is use in different organisational contexts in saudi arabia. the millions of pilgrims who enter the country for hajj and umrah pose severe logistical challenges as well as business opportunities which could be better addressed by sophisticated is and e-commerce portals. furthermore, the religious tourism industry of hajj and umrah in saudi arabia is sufficiently large and mature to attract research for directing its growth. this has motivated researchers to study and find solutions from is for a host of problems like crowd management and tracking. for example, khozium (2012) conducted research to design an information system that could manage crowds during the (nafra, النفرة) ritual at hajj when about 3 million of pilgrimages move from arafat to muzdalifa on one holy night. using thermal cameras as sensors at strategic points and roads, a computer interface fuzzy logic, operations research and decision support was developed to produce alternate decisions to the system controllers enabling them to control the movement of the huge crowd. as the movement of vehicles and mass transit during hajj is a major challenge facing urban planners and designers in makkah, koshak (2006) used web-based geographic information system australasian journal of information systems volume 18 number1 2013 11 (gis) to provide broader and easier distribution of the traffic plan that could facilitate easier understanding and adherence to the traffic plan. in another study, amro and nijem (2012) proposed a communication and information system that could assist tourist guides (mutawwif) in organising the movement of pilgrims in their group and add new capabilities for incidental difficulties such as lost pilgrims, and predict and avoid possible lost ones. naser et al. (2010) studied the usability and the risks and limitations of radio frequency identification (rfid) technology in managing security issues, organizing pilgrims' movement, residential management, healthcare management, luggage tracking and e-passport. while these studies are concerned with the implementation and limitation of specific is instruments, masoud (2005) focused on the vital role played by it in supporting the development of saudi tourism industry and in particular the assessment of the efficiency and capability of the economic performance. the study showed a positive impact of is-based portals on the quality of tourism services, the ability of controlling mechanism to regulate demand and supply as well as the possibility of increasing the regular income of tourism operators. research goal as this review shows, there are quite a few studies on is use in the tourism industry in saudi arabia, but the specific focus adopted in these studies are also accompanied with some limitations. as pointed out earlier, one limitation is that most of these studies apart from masoud (2005) focus on specific technical applications of some is instrument and not the wider impact of is use for the tourism industry in general. secondly, even masoud’s study is more concerned with enumerating the positive benefits of is use rather than problematising the use and adoption of is for the tourism industry. also, most of these studies have focused on the end-user not organisational usage, but as it was argued earlier systemic factors in the environment are critical to the adoption and use of various information systems within an organisation or an industry as a whole. in light of these limitations of the existing research on is use in the saudi religious tourism industry, the present study makes a substantial intervention insofar as it takes a macro-perspective on the issue of is use in the religious tourism industry and accounts for internal and external factors constituting organisational as well as cultural motivations that impact is use. methodology this paper used a interviews to discover the beliefs and attitudes of executives of saudi religious tourism organisations and identify the factors influencing the adoption of various is. jeddah was selected as the site for the research and ten firms in the hajj and umrah service or both, were approached for interviews. the majority of participating firms in this study were registered tourism organisations licensed by the ministry of hajj and/or members of the jeddah chamber of commerce and industry and listed on the chamber’s websites. jeddah is the gateway to the two holy cities at makkah and al-madinah. as the entry point for pilgrims on hajj or umrah, pilgrims may also spend time around jeddah, visiting its other attractions, historical monuments, art and culture. potential respondents were invited by telephone or email to participate in the study, and received letters explaining the study, approximate appointment times and dates with individual consent forms for signature. seven firms were interested in participating in the study, of which only five were finally able to participate. once interview times were mutually finalised, the interviews were conducted at each firm’s premises. the data was collected through semi-structured interviews and audio-taped with the respondent’s consent. open-ended interview questions were used to minimise influence on the participants’ responses (bryman and bell 2007). australasian journal of information systems volume 18 number1 2013 12 participant qualification age experience industry sector p.1 masters (bus. admin) 25+ 7 years religious tourism (hajj) p.2 phd (strategic mgmt) 50s 32 years religious tourism (hajj) p.3 masters (bus. admin) 40s 22 years accommodation & religious tourism (umrah) p.4 diploma (commerce) 60s 40 years religious tourism (hajj & umrah) p.5 not advised 40s 23 years travel agency (umrah) table1. interviewee profiles the interviews recorded on the audiotapes were transcribed and the feedback from the respondents were arranged and coded for common themes to facilitate analysis and comparison with the scholarly literature. the interpretation of the collected data plays an equally important role in the actual analysis as the actual collection of data (kval and brinkmann 2009). interpretation is not merely a simple reading of the text but a process of actively creating meaningful ideas and concepts from the text, (creswell 2009). a variety of approaches can be used for interpreting texts, the most important of which is the hermeneutical approach. originating from german philosophical tradition, the hermeneutical approach developed out of a dilemma about how meaning is constituted in human speech and actions (dilthey 1988). hermeneutics is a means of textual interpretation or finding meaning in the written word through a process of “structuring of the manifest of what is said to deeper and more critical interpretation of the text” (kvale and brinkmann 2009, p207). as an art and science of interpreting meanings, hermeneutics also emphasises the socio-cultural and historic influences on the way in which a certain phenomenon is perceived or interpreted. the reliability and validity of the data collected in any qualitative research is of prime importance (creswell 2009). many steps were taken to ensure reliability of the collected data. first, the authors checked transcripts to make sure that they did not contain any mistakes made during the transcription. second, they checked the data to ensure that the meanings of the codes were free from shift or drift in definition. third, a coordinated communication among the coders took place for sharing codes and analysis and cross-checking of results. the authors also used rich and detailed descriptions to convey the findings of the research. rich and detailed description, providing cohesive and information about all facets of the collected data, make it more realistic and coherent (creswell 2009). also, the main author spent extensive time in the field, studying the hajj & umrah industry and conducting field research for about three months in jeddah city. creswell (2009) stresses the importance of spending time in the field as it helps convey details about the site and people under research that lend credibility and accuracy to its findings. furthermore, the authors did not edit negative or odd findings that run counter to the theme under discussion and presented them without bias. such different or contrarian perspectives to an issue can actually enhance the credibility of the research as honest acceptance and presentation of contradictory evidence lends realism to the research conducted (creswell 2009). australasian journal of information systems volume 18 number1 2013 13 analysis & discussion this section considers the results from the analysis categorised on the basis of the key factors motivating is use. these include internal factors of market condition, customer influence, organisational commitment and industry competiveness, while government initiatives and supporting ict industry are classified separately as external factors motivating is utilisation. each of these themes is also accompanied by a selective reporting of some noteworthy responses from the participants 1 . external support government initiative the majority of participants in this study accepted that the use of is had enhanced their productivity, particularly in improving work processes. this shift, however, had occurred not entirely of the firms’ own volition but because of government intervention, as the saudi government required all businesses involved in umrah religious travel to operate online and encourages e-business in the hajj & umrah industry. this finding echoes some studies which find that economies subject to government regulation in ict adopt technology more than others who are not so affected by government control (molla and licker 2005). one participant, p2, the motawef (a representative of a twafa establishment) for external hajj pilgrims, said that he valued the ministry of hajj’s information system, used by all motawef offices to record and process hajj activities, and issue visas. another participant, p1, noted the minor positive effects of supplementary information from the ministries’ websites. “information on the ministry of hajj and ministry of islamic affairs, ‘da`wah, guidance and endowments’ website assisted us in creating a well-informed online system. we uploaded and re-formatted details to offer an easier understanding of hajj. pilgrims appreciate this additional information and it gives them a positive impression of our services” [p.1]. a manager from hotel chain and external umrah agency, p3, stated that the government’s compulsory national information system (makha’a, مخاع) is the main reason for the successful implementation of is in this industry. “the reason we use makha’a is because the ministry of hajj dictates that its use is a condition of an umrah licence. we have to renew our contract annually with a makha’a’s it provider but we have the choice to select which provider’s makha’a system we prefer” [p3]. in the same context, participant p4 mentioned the example of online banking, arguing that saudis only used online banking because most public and government sectors, like electricity bills, telecommunications and other governmental services like passport renewal, have encouraged their clients for transactions to be conducted online. this respondent said that people would not have shifted to booking online without this type of enforcement by the government. 1 note that the participants are signified by the letter [p. participant number]. australasian journal of information systems volume 18 number1 2013 14 “let’s take electronic banking services as an example. people did not use these services until the government made it compulsory to proceed most services online with payments and changed physical transactions, for example passports and other public sector accounts” [p.4]. the majority of participants argued that there was insufficient government action in the form of regulations, infrastructure and programs to support local business. the absence of these government initiatives was a major barrier to the productivity of fully integrated is adoption in all kinds of services in the hajj & umrah industry. participant p4 explained: “generally, people responsible for building the infrastructure only consider current needs and probable usage when designing the equipment. they do not consider future needs as they are uncertain of returns or high costs in such speculative action” [p.4]. however, surprisingly, participant p4 thought that in spite of these benefits of is use and the existing lack of is implementation in the tourism industry, the current legislation was sufficient. “i don’t believe that there is a need for more legislation for e-business, because the courts for commercial conflicts can also consider those that occur online” [p.4]. on the whole, the participants acknowledged the critical importance of government intervention and believed that the government could further extend its administrative powers to make the ict channels even more useful and efficient. several participants suggested that government could encourage the uptake of is by making it more difficult for businesses to file paper forms for licenses, or by improving ministerial portals for e-government. supporting ict industry access to inexpensive, efficient and advanced ict resources and infrastructure was a critical issue for many respondents. if the ict service and infrastructure in the market is deficient, no businesses would consider investing in such services. participants expressed concern about the quality of the internet connections, saying that lack of speed and reliability in internet connection could completely jeopardise e-business in the industry. some participants noted that the recent innovation in online licensing and marketing had improved the access of travel agents to it resources and expertise for conducting their business. p1’s organisation used an online portal (yosr, يسر), which assists many hajj providers for internal hajj pilgrims to organise their client travel data from registration to the end of hajj rituals. the annual cost of the portal is reasonable as development costs are shared by all hajj travel providers and this reduced the cost of groups purchasing ict services. “we had an erp [enterprise resource planning] information system specified for our company. it was fine for our needs at that time, but the annual cost was prohibitive. however, when we moved to yosr, technological advances were absorbed by many firms at reasonable prices because it is available for many hajj travel providers” [p.1]. another participant, an umrah agent, praised the utility of the makha’a system in these words: “the conditions of our contract with the ministry of interior include permission to approve inclusion of the pilgrims to umrah and the numbers of pilgrims are also referred to the ministry of the hajj. we could not productively administer this trade without the online system” [p.3]. australasian journal of information systems volume 18 number1 2013 15 the participants were aware that there are certain deficiencies in the local it providers and they talked at length about the absence of qualified is providers. one could observe that there was reluctance among participants from the larger and more sophisticated travel firms to employ local ict support firms due to a perceived lack of competence. “for market advantage, we would have no problem in establishing global computer systems immediately. if you know a firm that can perform, please let me talk to them. we usually seek out new technologies. for example, we contracted a global provider for a customer relations system, but we cannot find local it companies with sufficient professional expertise” [p.5]. some participants also commented on the cost of services provided by ict providers. while participants realised that higher quality would cost more, they said that they often found these costs to be prohibitive. the size of their business and profitability determined their ability to justify the outlay for elaborate information system and online services. small and medium sized organisations were not able to afford high cost information system and e-business solutions. “local it companies inflate the costs in their quotations for websites with online sales, i do not know why ... even testing a trial version is so expensive ... i searched for some other services as well and i found much better solutions than what we have but they would cost us 10 times more . . . you should keep the cost in mind, cost is a critical factor” [p.3]. while most participants appreciate the creation of the yosr and makha’a online booking systems, their comments and suggestions reveal the need for further improvements of ict providers’ availability and capability, and is affordability. in addition, the large data transfers and information storage for the travel industry are required via excellent technology, appropriate networking and skilled operators. internal capability human resources the access to requisite it infrastructure also depends on the degree of sophistication of the it expertise in the country, (brdesee et al. 2011). the web developers, programmers and designers who work for the local firms are expatriates. as the majority of these workers are not fluent in arabic, the travel operators have difficulty in communicating their needs and the resultant e-business portals are not sufficiently attractive or compatible to induce customers to buy their services. “we employed several it staff. most of them do little through lack of competency. they do not have the capacity to develop an is throughout the organisation. i believe that if the it department was better qualified that the firm could do better also. i think this is a reason we have not moved to e-business by ourselves” [p1]. to tackle this issue, a participant said that his corporation had developed its online booking website in canada by people of arabic background. business model participants were of the opinion that the extent of investment and adoption of it depended on that internal business plan and strategic vision, (brdesee et al. 2012a, brdesee et al. 2012b). the chief executive officer, or the owner, often made these crucial decisions about technology adoption. australasian journal of information systems volume 18 number1 2013 16 e-readiness in a business model encompasses themes such as the organisation’s understanding of the online environment, its support of automated trading, strategic requirement and commitment to systems change. therefore, lack of commitment contributes considerably to absence of an online business. ict awareness and competency of the manager also affects the level of technology adoption. participant p4 explained his business plan and model in relation to is adoption: “we do not have any current plans to invest further in information technology, because our information systems perform well. but we are focusing on expanding our bus fleet and building or buying new hotels. yet, this does not mean we are not going to adopt new technology if it proves to be necessary” [p.4] participant p5 indicated that his firm was undertaking ict development but this was not necessarily directed at is implementation for marketing and e-business. “we focus on internal development more than planning electronic business. we are restructuring our organisation with new organisational services and profit centres. we recently moved from paper to online records and our workers are being trained in the new systems. it is an expensive program for staff-training in 15 locations and we have to ensure that all records are changed over successfully as well” [p.5] if the manager does not understand the nature of the technology, or is unable to evaluate the available technology to meet the firm’s needs, then there is disconnect between need and procurement, and uncertainty about successful implementation prevails. relative advantage market condition while is may have its benefits for religious tourism industry, the demand of customers for such services is what actually determines the level and extent to which firms implement such services. some participants were sceptical about the popularity of e-commerce and the number of customers who would book online. participant p3, a representative of a national hotel and umrah agency, argued that the size of the potential market was a critical factor in the adoption of e-commerce. “the whole issue [of online booking system] has been on my mind for a while, but a visitor from egypt, for example, is not going to be able to book online due to saudi visa restrictions. in this case i can only offer online bookings to citizens of the arab gulf countries, expatriates living in saudi, and local saudi nationals” [p.3]. on the other hand, a hajj agent said that there was no sense in investing in an is ticketing or marketing system for local customers since they would prefer to come directly to the shop. since the government dealt directly with international operators for international hajj pilgrims, he could only have the business of local customers and in the domestic market, “e-commerce was fairly ineffective”. these examples show that varied access to different market types for religious travel influences firms’ business decisions in adopting is. similarly, p2, a hajj provider to non-saudi pilgrims (a motawef, and a twafaa senior officer explained how their service to international pilgrims only begins (مطوف with their arrival in the country. “hajj pilgrims from any muslim country can apply to governments or hajj organisers in their countries. we do not have any input at this stage. our australasian journal of information systems volume 18 number1 2013 17 mission starts when we receive the pilgrim in saudi arabia, we cannot undertake any marketing in those countries” [p.2] non-business travellers to saudi arabia are restricted by severe visa constraints. due to security concerns and the logistics of conducting the annual hajj, saudi visas for umrah and hajj pilgrims are only available through national and international agents who book services with designated saudi providers. this restricts the share of international leisure and travel markets in the saudi tourism industry. in support of p3’s view, two other providers noted that given the limited number of external pilgrims for hajj and umrah each year, agents do not see a need for online booking catering to international tourists as their allocations are easily sold to local customers through their shopfront stores. p1 and p4 noted: “i think only 2% of our clients would book online for hajj. thus, the investment in an e-booking system would be costly, and the potential number of customers very low, that is way i’m not motivated to go online. also, each hajj agent is assessed and graded by the government and this grade affects the pilgrim package allocation. for example, we are grade a, so we are allocated sales for a thousand pilgrims. i can sell this number of vacancies without an e-booking system” [p.1]. “we get a number of vacancies each season from the ministry of hajj. we depend highly on referrals to market those vacancies. most of our customers use word of mouth in their selection of hajj agent. i don’t think there is any need for e-commerce in our case” [p.4]. these tourism executives were of the opinion that saudis would perhaps not purchase travel services online. they were satisfied with their business within the local market and did not want to extend their customer base globally though e-commerce channels. customer service different types of sales channels are exploited by firms, whether it be a shopfront, direct contact through telemarketing, or online marketing, but people’s perceptions of the security and efficiency of a particular system may differ. here, participants stated that older people may not trust the internet for payment. other comments were made regarding customer trust, security issues, issues with prototypes on e-shopping, and payment logistics were other reasons cited by some participants. p3 was convinced that fear of fraud and robbery were uppermost on the minds of potential customers. some interviewees also believed that online sales could not engage the customer more directly and build rapport and trust between the travel firm and a potentially long-term customer. participants p1 and p4 said: “it is an aim of my company to go online, but we will not be able to have personalised customer relations in this medium. from experience, most of our customers make their hajj arrangements through our company because they trust our staff. this advantage will not be available on the internet … i also do not believe other technologies can duplicate the process of meeting customers and talking through their travel plans” [p.1]. “arabs naturally prefer the opportunity to see what they are buying, not just to hear about it … you can see how busy travel agents in shops are. customers go to the agents to meet the agent, talk about their plans, build a trusting relationship and negotiate prices” [p.4]. australasian journal of information systems volume 18 number1 2013 18 these comments highlighted the importance of customer service and personal interaction especially in a market like religious tourism in saudi arabia which can be heavily influenced by traditional business practices. the use yosr and makha’a information systems are two good examples of customer relation management information systems. however, in e-business, the principle of face-to-face rapport, trustbuilding and negotiation may be declining as these firms reach out to a clientele of global muslims dispersed across the world, and here e-commerce portals can come in to provide a direct interface for conducting such transactions. competitive advantage the level of is implementation and its penetration in a single firm is subject to the prevailing standards in the industry as a whole. this study found that participants’ firms display competitive behaviour and adopt technological change only when the rest of the industry had adopted those technologies like ict. two participants p3 and p1 gave examples of competitive advantage that could affect their responses. “this is the nature of people. yes, if my competitor was successful with an initiative, i would definitely follow, but i need to be sure that when i invest one riyal in e-commerce, i get back ten. i can give you an example on the impact of competition. in the nineties, mercedes and bmw dominated the global luxury car business. then the japanese introduced the lexus with higher standards and cheaper prices. what happened? the two brands improved their standards to retain their positions in the market” [p.3]. but competitive innovation also requires justification and p1, a licensed hajj provider for local pilgrims, said that a competitor in the saudi religious tourism does not adopt innovative advantage unless a real threat to the business is detected. “my competitors could use an information systems to offer a more convenient managing and marketing channel. i won’t respond immediately, but will wait and see what happens. if i feel certain that this is working for them and is negatively affecting my business, then i will plan a change” [p.1]. the majority of participants accepted that is portals can provide competitive advantage, but they said they would only adopt those services once it was established in their industry and they were convinced that it was financially feasible. participants commented on competitive advantage about seventeen times during the interviews, and this reflected their attention to the importance of this issue. summary of results the analysis of the participants’ responses has confirmed some of the organisational factors as well as the external factors identified in the literature on is adoption (rogers 1962; rogers 2003; tornatzky and fleischer 1990; molla and licker 2005; al-gahtani et al. 2007; suebsin and gerdsri 2009) as discussed in the literature review. the analysis shows that the advantages of all information systems discussed is perceived, but the adoption level is dependent on the characteristics of the information system. for example, e-commerce and has some security, trust and perceived relative advantage by religious tourism limitations that hinder its adoption. however, the management information systems like yosr and makha’a have some motivational characteristics that promote their adoption like government enforcement and affordability. australasian journal of information systems volume 18 number1 2013 19 the following factors affect information systems usage by hajj–umrah religious tourism operators in saudi arabia: external factors (government initiative and supporting ict industry): government initiative is one of the critical factors behind the use of is in the industry. the industry in general, is appreciative of the steps taken by the government, especially in encouraging centralised is makha’a systems. however, the participating executives are seeking further solutions to overcome existing legal and infrastructural obstacles for further is development. on the other hand, participants were critical of the ict support industry and said that the relative backwardness of the local ict industry had discouraged some operators from adopting is systems like e-commerce. there was also a prevailing perception that the quality of service of local providers was inferior to international ict support firms. it was also pointed out that while local ict providers, particularly website developers, were substandard, their costs were significantly higher than those prevailing in the global market for a particular service. internal capability (human resources and business model): apart from broader systemic factors in the saudi industry, the individual business model of the company and its e-readiness was also critical to is adoption. the organisations also need staff that can handle and maintain is systems. also, skilled staff with well qualifications and visions are required to take their organisations to a further step toward is utilisation. organisational commitment to adapt to is or automated systems can also play a major role. some organisations give the is usage and implementation high priority. these organisations are more likely to overcome any further obstacles. the individual business model of the company and its e-readiness also determines is adoption. relative advantage: the analysis shows that relative advantage of is is an important factor. relative advantage of is is predicated on the advantages it provides in terms of cost benefit, productivity, time-saving, profit generation and customer service. it must, however, be noted that as the majority of firms did not have an online strategy, cost effectiveness and profit generation were only mentioned by the minority whose business model included or depended on online sales. conclusion from this analysis of data from a series of interviews with saudi religious tourism operators, this study has examined the perceptions of managers and owners to identify factors motivating is use. government initiative is one of the critical issues influencing is use in the industry, and government intervention through the creation of mandatory e-business portal, makha’a and yosr, is in the major reason behind the existing practice of is use by hajj/ umrah operators. competitiveness in the travel industry as a whole could also prove to be a potential factor motivating individual firms to adopt is. if is use was found to provide relative advantage to a company, competitors would soon follow suit. however, problems in accessing ict resources and the relative backwardness of the local ict industry have discouraged some operators from adopting is. in addition, their perceptions about customer service also negatively impacted is use as most participants generally felt sceptical about ebusiness, preferring a shopfront approach rather than the onerous work of instituting is without reaping nay benefits in terms of increased sales. these research findings can be further analysed to develop recommendations for both government policymakers and industry leaders in the field. the study contends that the endings are also valuable for practitioners from western cultures in applying computer-based information systems solutions to developing countries. it can also be used for future research to theorise about the motivations and barriers to is use in arab or islamic contexts. australasian journal of information systems volume 18 number1 2013 20 limitations and future research limitations of this research resemble those that are inherent in any qualitative research concerning a single geographic location with a restricted composition of study participants. the researchers has no experience in the field of tourism and marketing, however the long time the main author has spent in saudi arabia investigating the tourism market would overcome this limitation. future research should be conducted in other organisational settings in saudi arabia to determine if similar factors influence is adoption and test the generaliseability of the findings of this study at least within the saudi market. a study with a quantitative approach using statistical analysis may be conducted to test the validity and interpretability of the findings of this qualitative study. it is recommended that the suggested research model in figure 1 should be considered for future work with further rigorous analysis. the model needs to be tested with appropriate factor analysis tools to understand the interrelation between the constructs and evaluate the validity of the construct of its factors. external factors motivated firms cost benefit intention to utilise is supporting ict industry internal capability human resources business model relative advantage competitive advantage customer service government initiative figure 1: the suggested research model for future work, motivations affecting information systems usage further, a longitudinal research may be conducted to examine the change in perceptions of is use among managers/owners in the tourism industry over time. in fact, a study focussed on the point of view by the is industry executives may be undertaken to highlight any dichotomies between the management and employees in the tourism industry in engaging is. acknowledgements the main researcher's work is sponsored by jeddah community college at king abdulaziz university, jeddah, saudi arabia. thus, he would like thank them for their support. however, the authors would like to thank the participants of the study for their insights on the issue and willingness to participate in the study. australasian journal of information systems volume 18 number1 2013 21 references agourram h (2009) the quest for information systems success in saudi arabia. a case study. bishop’s university, lennoxville , canada. ajzen i, fishbein m (1980) understanding attitudes and predicting social behaviour. prentice-hall, englewood cliffs, nj. aldogily m (2009) saudi arabia at first in e-commerce expansion. alriyadh news, no.14999. http://www.alriyadh.com/2009/07/18/article445570.html. accessed 10 november 2012 al-gahtani s (2003) computer technology adoption in saudi arabia: correlates of perceived innovation attributes. information technology for development 10:57–69. doi:10.1002/itdj.1590100106 al-gahtani s, hubona g, wang j (2007) information technology (it) in saudi arabia: culture and the acceptance and use of it. information and management 44(8):681-691. doi:org/10.1016/j.im.2007.09.002 alkahtani mufleh m (1998) computer-based information systems and their use by managers in saudi arabia and the uk. mufleh m. alkahtani. http://dspace.lboro.ac.uk/dspacejspui/handle/2134/7759. accessed 26 june 2011 al-zharani s (2011) an empirical investigation of the information technology implementation in saudi arabia. journal of information & systems management 1(1):37-45. alzhrani s (2009) alrajhi banks of secure payment system for e-commerce transactions. okaz news, no. 2974. http://www.okaz.com.sa/new/issues/20090808/con20090808297179.htm. accessed 3 november 2009 amro a, nijem q (2012) pilgrims "hajj" tracking system (e-mutawwif).contemporary engineering sciences 5(9):437-446. babumrah (2012) eumra industry. bab umrah. http://www.babalumra.com/en/aboutus.asp. accessed 27 june 2012 bmi (business monitor international) (2009) saudi arabia information technology report q4 2009. business monitor international, london, england bmi industry report (2012) saudi arabia information technology report q2 2012. business monitor international. london, united kingdom, london. brdesee, h, corbitt, b, & pittayachawan, s (2011): technology for business: supporting industries as a main driving force for e-commerce adoption in saudi arabia, in: proceeding of the 11th international conference on e-business (pp. 175–180), spain: seville. brdesee, h, corbitt, b, pittayachawan, s, & alsaggaf, w (2012a): organisational culture and adoption of electronic commerce: a study of the saudi arabian tourism market, in: proceeding of the 7th international conference on computer science & education (pp. 857–862), australia: melbourne. brdesee, h, corbitt, b, pittayachawan, s, (2012b): lessons to be learnt: cultural means impeding ecommerce adoption in a saudi industry, international journal of e-education, e-business, emanagement and e-learning, 2( 6): 488–492. cabrera a, cabrera f, barajasc s (2001) the key role of organisational culture in a multi-system view of technology-driven change. international journal of information management 21(2): 245–26. claycomb c, iyer k, germain r (2005) predicting the level of b2b e-commerce in industrial organisations. industrial marketing management 34(3):21–234. http://www.alriyadh.com/2009/07/18/article445570.html http://dx.doi.org/10.1016/j.im.2007.09.002 https://dspace.lboro.ac.uk/dspace-jspui/handle/2134/7759 https://dspace.lboro.ac.uk/dspace-jspui/handle/2134/7759 http://www.okaz.com.sa/new/issues/20090808/con20090808297179.htm http://www.babalumra.com/en/aboutus.asp australasian journal of information systems volume 18 number1 2013 22 creswell j (2009) research design qualitative, quantitative and mixed methods approaches. 3rd edn. sage. ca: thousand oaks. davis fd (1989) perceived usefulness perceived ease of use and user acceptance of information technology. mis quarterly 13(3):319-340. dilthey w (1988) introduction to the human sciences: an attempt to lay a foundation for the study of society and history. (r. j. betzanos, trans.). detroit. mi: wayne state university. (original work published in 1883) ep-moh (electtronic poortal of ministry of hajj) (2012) process of umrah for external pligrmiages. http://www.haj.gov.sa/pages/umrahoutprocedures.aspx. accessed 25 november 2012 fishbein m, ajzen i (1975) belief, attitude, intention and behavior: an introduction to theory and research. addison-wesley, reading, ma. hofstede g (1980) cultural consequences: international differences in work related values. sage, beverly hills, ca. hofstede g j (2000) you must have been at a different meeting: enacting culture clash in the international office of the future. journal of global information technology management 3(2):42-58. hooshang mb, esmail salehi-sangari, engstrom a (2006) competitive advantage with e-business: a survey of large american and swedish firms. competitiveness review 16(2): 150-157 ke w, wei k k (2008) organizational culture and leadership in erp implementation. decision support systems 45(2):208–218. khozium m (2012) a hybrid intelligent information system for the administration of massive mass of hajjis. life science journal, 9(4):171-180 ko e, kim s h, kim m, woo j y (2008) organisational characteristics and the crm adoption process. journal of business research 61(1):65–74. koshak n (2006) developing a web-based gis for hajj traffic plan (hajjgis.net). journal of urban planning research, cairo university 6(6): 1-13 kshetri n (2007) barriers to e-commerce and competitive business models in developing countries: a case study. electronic commerce research and applications 6(4): 443-452. kvale s, brinkmann s (2009) inter views: learning the craft of qualitative research interviewing. sage publications inc, usa masoud r (2005) tourism information system and its impact on economic performance indicators in saudi arabia: an empirical study on jeddah district. unpublished dissertation. http://islamiccenter.kau.edu.sa/arabic/maktabah/rasael/r656.pdf ministry of hajj (2012) notes on different types of tawaf. http://www.hajinformation.com/main/f35.htm. accessed 25 november 2012 moh (2012) responsibilities of the ministry of hajj. http://www.hajinformation.com/main/m10.htm. accessed 10 november 2012 molla a, licker p s (2005) e-commerce adoption in developing countries: a model and instrument. information & management 42(6):877-899. naser m, rafie m, budiarto r, alsalihy w (2010) security considerations in embedding rfid in 'hajj' system. european journal of scientific research 42(1):133-138 nusair k, parsa n g, cobanoglu c (2011) building a model of commitment for generation y: an empirical study on e-travel retailers. tourism management 32(4): 833-843. http://www.haj.gov.sa/pages/umrahoutprocedures.aspx http://www.hajinformation.com/main/f35.htm http://www.hajinformation.com/main/m10.htm australasian journal of information systems volume 18 number1 2013 23 osterwalder a, pigneur y (2002) e-reality: constructing the e-economy: an e-business model ontology for modeling e-business. in: proceeding of the 15th bled electronic commerce conference, bled: slovenia, 2002 ourworld travel (2011) hajj 2012/1433h packages. ourworld travel, wollongong, nsw. http://www.duniana.com/hajj-travel.php#details. accessed 20 may 2012 pavic s, scl koh, simpson m, padmore j (2007) could e-business create a competitive advantage in uk smes?. benchmarking 14(3): 320-351 piris l, fitzgerald g, serrano a (2004) strategic motivators and expected benefits from, e-commerce in traditional organisations. international journal of information management 24(6):489–506 roger e m (1962) diffusion of innovation. the free press, 1st edn, new york, ny. singh m p, chopra a k, desai n (2009) commitment-based service-oriented architecture. computer 42(11): 72-79 soundvision (2012) umrah and hajj. soundvision. http://soundvision.com/info/hajj/hajj05.asp. accessed 27 june 2012 tornatzky l, fleischer m (1990) the process of technology innovation. lexington, ma, lexington books. tradearabia (2009) saudi tops in e-commerce transactions. http://www.highbeam.com/doc/1g1203635495.html. accessed 21 january 2011 vanderstoep s, johnston d (2009) research methods for everyday life: blending quantitative and quantitative approaches. john wiley. san francisco: ca. wagner ja, hollenbeck jr (2010) organizational behaviour: seeking competitive advantage. routledge, new york: ny. world factbook (2012) country data population 2012. world factbook. https://www.cia.gov/library/publications/the-world-factbook/geos/sa.html. accessed 19 november 2012 yosr (2012) yosr functions. yosr for hajj. http://www.yosr4haj.com/. accessed 27 june 2012 http://www.duniana.com/hajj-travel.php#details http://soundvision.com/info/hajj/hajj05.asp http://www.highbeam.com/doc/1g1-203635495.html http://www.highbeam.com/doc/1g1-203635495.html https://www.cia.gov/library/publications/the-world-factbook/geos/sa.html http://www.yosr4haj.com/ ajis special issue requirements engineering 1999 editorial a crucial first step in software development is the generation of a description of the artefacts that we seek to build. among other things, this description must include the requirements that the system must satisfy. it is widely recognised now that these descriptions might be the critical determinants of software quality, given empirical studies showing that errors in generating these descriptions are the most numerous in the software life-cycle and the most expensive and timeconsuming to correct. the field of requirements engineering considers the problem of engineering such descriptions. requirements engineering is a relatively mature field of inquiry, but the growing complexity of present-day systems (and their organisational contexts), has focused considerable attention on the area, both in academia and in industry. this special issue is a carefully compiled collection of state-of-the-art papers that seeks to provide readers with a broad perspective on new and interesting developments in the area. this issue consists of two invited contributions and seven refereed submissions. the two invited contributions are representative of two ends of the research spectrum in requirements engineering: object-oriented approaches and formal methods. the invited position paper by froehlich, hoover and sorenson makes the case for the use of hooks in the context of architectures that apply to suites of related products or to entire application domains. it shows how the use of hooks to encapsulate requirements can help localise the effects of requirement changes and focus attention on elements of these generic architectures that require modification for the specific product being developed. the invited paper by tsai and li provides an approach to the verification and validation of formal (and executable) requirements specifications via parallel execution. the underlying premise is that executing and reasoning with formal specifications enables us to resolve conflicts amongst the various system stakeholders and verify that certain domain-independent properties (such as liveness or consistency) hold, but is typically too slow to be of practical use. the authors suggest a novel parallel execution model for formal specifications that combines elements of both top-down and bottom-up evaluation strategies to address this problem. the seven refereed submissions were selected after careful vetting (by at least two reviewers) from a total of twenty submissions and represent the diversity of research themes within the area. two of these papers present comprehensive requirements engineering methodologies and supporting tools. barber, graser, grisham and jernigan present the software engineering process activities (sepa) methodology that supports the integration new implementations with legacy systems and cots products in the context of component-based development. like froehlich, hoover and sorenson, they argue for the separation of requirements that apply across domains or product-lines from those that are application-specific. they describe a set of tools that implement this methodology with additional support for traceability, evolution and re-use. requirements refinement and re-use is also the focus of the reuse-assisted requirements engineering (rare) methodology presented by cybulski and reed. their approach involves the use of faceted classification schemes for classifying requirements and design artefacts, similarity-based retrieval to support artefact re-use and the use of thesauri for mapping problem domain terms (in which requirements are expressed) to solution domain terms (in which design artefacts are described) to support the process of refining requirements specifications to designs. the difficult problem of requirements evolution is the focus of two separate papers in this issue. a key question in requirements evolution is deciding whether to proceed with a change, given that most changes involve significant cost. impact analysis provides a means for determining the extent, complexity and cost of proposed changes to a system in order to inform the decision on whether to accept or reject a proposed change. lock and kotonya propose a set of requirementsajis special issue requirements engineering 1999 level techniques to identify how changes to certain artefacts may impact others and how the consequences of these changes propagate through the entire specification. nuseibeh and russo address questions that arise after a decision to accept a proposed change has been taken. they propose an ambitious scheme that uses techniques derived from abductive logic programming to identify means for resolving inconsistencies generated by changes to requirements specifications. the formal language in which they propose to represent specifications is in itself quite novel quasi-classical (qc) logic is a variant of classical logic that permits reasoning with portions of formal specifications that are not directly affected by inconsistencies, in the event that such inconsistencies appear in specifications (typically as a consequence of requirements change). formal methods, such as those discussed in the papers by tsai and li, and nuseibeh and russo, are often criticised for being too inaccessible to practitioners and expensive to implement. droschl presents a case study that suggests otherwise. his case study involves the application of a semi-automatic theorem prover (pvs) to the analysis of requirements for an access control system, and contains important lessons for future similar efforts. extra-functional (or non-functional) requirements are often ignored in requirements engineering frameworks. the paper by hochmuller makes a strong case for explicitly accounting for extrafunctional requirements in requirements process models. hughes and wood-harper make a case for providing more tangible and explicit support for another facet of the requirements engineering process that is typically ignored the organisational perspective. they support their argument through two case studies that suggest that an explicit sociological/organisational perspective can indeed better inform the exercise of requirements elicitation and analysis. the problems confronting the requirements engineering community are challenging ones, requiring a complex toolkit of approaches. the nine papers contained in this special issue represent the proverbial tip of the iceberg. this issue, hopefully, provides pointers to the interesting research terrain that remains to be explored. i would like to specially thank jacob cybulski, frank moisiadis, yusuf pisan, debbie richards, alessandra russo and didar zowghi for their help with reviewing submissions to the special issue. aditya ghose department of information systems university of wollongong ajis vol. 7, no 2 may 2000 tribes of users and systems developers sharon dingley(l), hanifa shah (2) and paul colder (3) (1) school of computing, university of central england, perry barr, birmingham, b42 2su, uk. email: sharon.dingley@uce.ac.uk (2) school of computing, staffordshire university, beaconside, stafford, uk. email: h.shah@soc.staffs.ac.uk (3) department of computer science and applied mathematics, aston university, aston, birmingham, uk. email: paul.golder@aston.ac.uk abstract effective communication is essential for information systems development crossing functional, organisational and national boundaries. as organisations attempt to overcome cultural barriers to communication across the world, communication with colleagues across the corridor remains problematic; cultural barriers between departments remain unchallenged. this paper introduces the concepts of 'culture' and 'tribe' into the discussion of the relationship between business users and information systems developers. previous research has focused on identifying specific barriers to user-systems developer communication and on ways of eliminating these barriers. in contrast, this paper suggests that much can be learnt through the recognition of cultural differences inherent to the differing roles of user and systems developer. maintenance of cultural identity is essential to the individual if he/she is to function effectively as a member of his/her tribe, whether it is the 'tribe' of developers or the 'tribe' of users. communication problems within the systems development process may be addressed by a mutual understanding of cultural differences between the 'tribes' of users and systems developers. this degree of understanding cannot be achieved by attempting to change, persuade or convert the other tribe. the problems of user-systems developer communication need to be addressed through effective communication which acknowledges the differing cultures. introduction the global marketplace provides cultural challenges to corporate competition and alliance. it is recognised that cultural differences affect all business functions and strategies are being developed to address these differences. information systems and information systems development cross functional, organisational and national boundaries. recent studies (tan & gable, 1997; hunter & beck, 1996) have investigated the international cross-cultural impact of information systems development. however, as organisations attempt to overcome cultural barriers to international communication, cultural barriers to internal communication remain unchallenged. effective communication between business users and systems development staff is essential for the development of information systems which satisfy business needs. despite much investigation in this area, the seemingly insurmountable division imposed by 'invisible' walls between 'them' and 'us' proves to be stronger than physical departmental walls. this paper presents a cultural analysis of the communication barriers between the information systems and business functions during information systems development. it is proposed that communication problems can be explained in term of inherent cultural differences between departmental communities and that an understanding and appreciation of cultural behaviour can help improve interdepartmental relations. becher (1989) describes academic disciplines as tribes with their own territory (building on campus), language (specialist terms) and artifacts (e.g. computer manuals). a key problem of interdisciplinary inquiry is the interpretation of terminology specific to the native discipline. however, ackerman (1965) suggests that interdisciplinary work does not require encyclopaedic knowledge of the foreign area. he rejects the view that a complete understanding of terms, as used by experts in a discipline, is a necessary pre-requisite, proposing that an elementary introduction to principles in a discipline can stimulate invention in another. it is this view which provides the basis for this paper. this paper is written from an information systems perspective and it recognises that a complete understanding of terms used in other academic disciplines is not attainable. the paper provides an elementary introduction to the principles of cultural analysis to stimulate discussion concerning their application and relevance to information systems development. the following section briefly outlines previous investigations of the user-systems developer relationship. the international cross-cultural impact of information system development is then reviewed, introducing levels of cultural analysis. a cross-cultural analysis of communication barriers between users and systems developers is presented. the paper concludes by considering the practical implications of the concepts discussed. previous investigation of the user-systems developer relationship the importance of effective user-systems developer communication during the analysis and design of information systems has been widely recognised. problems resulting from poor communication and common theories for these problems arising are outlined in shah et a/.(1994). various aspects of the user-developer relationship which have been investigated by information systems researchers are summarised by joshi (1992). different approaches to analysing this relationship have included: 1. investigating user involvement in information systems development and the affect of this on the resultant information systems (baroudi etal., 1986). 20 ajis vol. 7, no 2 may 2000 2. examining the user-systems developer relationship in terms of their relative power (markus & bjornanderson, 1987; saunders & scamell, 1986; lucas, 1984; oliver & langford, 1984). 3. exploring conflict in this relationship (robey & farrow, 1982). 4. employing a communications perspective to examine the interaction between users and systems developers (bostrom, 1989; guinan & bostrom, 1986; valusek & fryback, 1985; martin & fuerst, 1984). 5. exploration of users' lack of understanding of computer systems and illogical problem solving approaches (kaiser & king, 1982). 6. examining analysts' technical nature and their lack of appreciation of managerial realities (werner, 1979). 7. examining analysts' techno-economic orientation rather than organisational-behavioural orientation (kumar & welke, 1984). 8. exploring differences in perception (gingras & mclean, 1982) and personality (kaiser & bostrom, 1982; benbasate/a/.. 1980). although the nature of information systems, the development of information systems and the roles of user and developer have changed, rourke & elliot (1996) report that problems between users and developers still prevent the full business benefits of information systems and information technology being derived. as information systems have evolved from automation to business process redesign, the autonomy of business units has been threatened. this has led to the development of approaches to examine the wider context of information systems (cavaye & christiansen, 1996). rourke & elliot (1996) identify a number of strategies to resolve or manage differences between users and systems developers. these include education and training of users, the appointment of hybrid managers, outsourcing it facilities, decentralising it to users and end user involvement in the development process. however, kaulio & karlsson (1998) focus on user orientation rather than user involvement; user orientation emphasises a positive attitude to users and their contribution to information systems development. joshi (1992) identifies a number of 'persuasion' strategies relevant to user-systems developer interaction. these include: 1. friendliness attempting to make a favourable impression by being pleasant and congenial, so that the target person will be more likely to do what is requested. 2. bargaining involving offers to exchange some benefits or favours based upon social norms of obligation and reciprocity. 3. reason attempting to convince others through the use of facts and logical argument. 4. assertiveness attempting to influence others by means of forceful manner. 5. higher authority involving the threat of appeal to higher authority or reference to directives from a higher authority. 6. coalition mobilising other people in the organisation to assist in persuading the target person. underlying these strategies is the basic assumption that a particular strategy must be applied to user-systems developer communication and that the communication process can be improved by the identification of the most appropriate strategy. this view can be enhanced by an understanding of the inherent cultural differences of the two roles. this paper proposes that a cultural analysis of communication barriers can improve the effectiveness of usersystems developer interaction, adopting kaulio & karlsson's (1998) focus of user orientation. cross-cultural information systems development culture can be observed at many different levels, for example, national culture, regional culture, culture of a particular generation, social culture and corporate culture (hofstede, 1991); at times, these levels of cultural values may conflict. differences in national and corporate culture are illustrated in a case study (parkinson, 1993) which investigates the issues which arise when two work groups from different companies and different countries work together on a single project. one group was from a large multinational company based in the united kingdom, the other group was from a small software house in the united states of america. differences in national culture were found to place a significant strain on communication between the two groups. although both groups spoke the same language (english), communication problems arose because of preconceived ideas held by the groups. the english widely perceive americans to be brash, over-zealous people and americans often perceive the english to be 'stuck up' and old fashioned. these cultural stereotypes imposed communication barriers between the groups. team members had to carefully phrase communication to ensure that the other work group would not be insulted or offended. this is comparable with the user-developer relationship; although both groups appear to 'speak' the same language, preconceptions hinder effective communication. similar findings are reported in a study of two work groups in germany and the united states of america (meyer, 1993) which documents differing characteristics of conversational style between the german and american work groups. the abrupt, blunt conversational style of the germans, appeared to be rude and impolite to the american group. in contrast, the informal american style of conversation for establishing social relationships was seen as superficial by the german group. 21 ajis vol. 7, no 2 may 2000 in parkinson's study (1993), corporate cultures differed in a number of ways which affected the views and opinions of the employees. firstly, the two organisations varied significantly in size; the large organisation shielded employees from external difficulties, such as market forces, to a large extent but in contrast, employees in the smaller organisation were much more exposed to the impact of external factors. this resulted in additional pressures on the team members which affected, for example, the cognitive views of a situation. secondly, corporate goals differed; the larger organisation had established quantified strategic goals to improve product quality over the next decade. this clashed with the much shorter term objectives of the smaller organisation which were to expand rapidly and become financially more stable. such goal differences place strain on the intergroup relationship and as time progresses, such differences can cause the groups to diverge or converge, depending on the current strategies of the groups involved. if the dependency relationship is high then both companies will strive to ensure the intergroup relationship is maintained, conversely, if the intergroup dependency is low then it must be accepted that the relationship will inevitably be under constant strain. national culture and corporate culture, illustrated in these two case studies, have been examined in the context of cross-cultural joint ventures in the strategic management literature. for example, barkema et al. (1996) provide evidence to demonstrate that cross-cultural joint ventures are most susceptible to cultural distance than wholly owned subsidiaries because national and corporate culture form a 'double-layered' acculturation. park & ungson (1997) investigate the popular belief that cross-cultural joint ventures are unstable which is based on the premise that the ownership of similar cultural values reduces misunderstanding. they examined 186 joint ventures in the electronics industry and revealed no evidence to support the claim that joint ventures were more likely to dissolve between culturally distant countries. in addition, in contrast to parkinson's study (1993), similarity of scope, size and age of organisations did not improve the 'fit' of the venture. park & ungson (1997) conclude that the suitability of joint ventures may depend on the participants ability to understand cultural differences. as cultural differences can severely impede intergroup working, they need to be effectively addressed. the intergroup relationship can be improved if each group understands the culture, thought process and environment of the other. this will enable both groups to appreciate the intergroup relationship so that cultural sensitivities are considered. one way of overcoming this situation would be to use a video camera to record team building exercises which could then be viewed by the other group. in a previous study (dingley, 1996) video material was used in the preparation for an interview between a british, female, information systems researcher and the american, male, 1993 world individual motorcycle speedway champion. the purpose of the interview was to review the information and information systems requirements of the sport. videos of the rider at work at the stadium, being interviewed by journalists and participating in working parties to improve the sport were reviewed. the videos provided an appreciation of the champion's environment, pressures and conversational style which contributed to the preparation of the interview. for example, it was noted that the rider refused to talk of problems or discuss mistakes, speaking only of opportunities. rather than discussing information systems solutions, the interview was therefore framed by opportunities to improve the provision of information. the videos helped the researcher to learn aspects of the national and 'organisational' culture in preparation for the interview. team building activities could be extended to the user-systems developer relationship, if for example, analysts met their prospective users outside the office environment, perhaps they would accept them as people first and users second. this may help to remove the preconceived barriers erected by contextual stereotyping. hofstede (1991) recognises that one reason why solutions do not work or cannot be implemented in organisations is because differences in thinking have been ignored. hofstede suggests that understanding differences in thinking is at least as important as understanding technical factors. a range of tools with which to identify and understand cultural differences are included in berger (1996). the members of the project teams in the case studies (parkinson, 1993; meyer, 1993) were entering each others communities and problems of intergroup communication were caused by a number of cultural differences between the two groups. differences in the use of language, goals, cognitive views, frames of reference and organisational pressures all contributed to the communication difficulties experienced. we suggest that similar issues arise when two work groups from the same company and in the same country work together on a single project. cultural analysis of users and systems development groups culture is a holistic principle (leach, 1965) which is learned and shared by members of an environment (bird, 1979), providing a guide to acceptable behaviour (smith, 1990). a person can be examined at three levels of abstraction (wallerstein, 1990; hofstede, 1991): 1. universal inherited characteristics of human nature. 2. sets of learned characteristics that define a person as a member of a group. 3. the person's idiosyncratic characteristics. it is the second level of learned characteristics that wallerstein (1990) labels traits, behaviours, values and beliefs that are neither universal nor idiosyncratic but cultural. the term 'culture' denotes the 'way of life' of a group of people (johnson, 1979). this suggests that groups have culture, implying that it is only possible to speak of cultures and not just of culture (smith, 1990). an individual is a 22 ajis vol. 7, no 2 may 2000 member of many groups which each has its specific 'culture'. a person therefore participates in many cultures (wallerstein, 1990). a group is more than a collection of individual behaviours; interpersonal interaction and mutual influence; the objective of the group unites members (zander, 1977) fulfilling the basic human need to belong. individuals relate to a 'we group' as opposed to 'them', that is, those people who are excluded from the 'we group' (hofstede, 1991) as a source of identity and protection (segall et al., 1990). wallerstein (1990) proposes that culture summarises the distinction between groups, representing what is shared within the group that excludes those outside the group. leach (1965) observes that all social systems are necessarily subdivided into mutually exclusive segments, which may be actively hostile. nanda (1991) defines a tribe as a 'culturally distinct population'; members of the tribe think of themselves as being part of the 'same people'. members recognise a relationship with other members of the tribe based on ethnic origin, language or pattern of interaction (pfeiffer, 1977). we suggest that the concept of a tribe can be aptly applied to a group of business users and systems developers. table 1 applies tribal characteristics to organisational departments. a tribe: possesses culture. retains self-awareness. maintains a sense of boundaries. enforces shared pattern of socialisation to reinforce values. consists of a complex social structure. shares problems. engages in rituals for coming together. establishes traditions which sustain cohesive relationships. influences politics closely involved with the rest of the social organisation. settles disputes by the next higher order. a group of employees in a department: possesses a distinct culture which may have differing values and beliefs compared with other departments or that of the organisation as a whole. retains a self-awareness of their existence and purpose. each member identifies with the group. is aware of the boundaries which separate the group from the other areas of the organisation that which separates 'us' from 'them'. engages in social relationships within the group which encourage adherence to group values, necessary to sustain membership. adopts a complex social structure based on management hierarchy, subareas, project teams and seating plans. shares departmental problems. attends departmental meetings and team briefings which are rituals to bring the members together. establishes traditions such as the manner in which birthdays are celebrated and the 'office christmas party' to sustain cohesive relationships. is influenced by organisational politics. settles disputes by the next higher order, such as the next level of management. a tribe: may operate local units independently. a tribe may be divided into smaller segments such as villages and regional groups which may be further divided into clans and households. unites in response to external conditions such as the threat of attack or the opportunity to attack others. depends upon informal means for controlling deviant behaviour and settling conflicts. adopts a common language. shares a common goal and distinct practices. possess territory and will actively defend it against intruders. a group of employees in a department: may operate local units independently, for example, a sales team may be divided into regions which operate independently. unite in response to external conditions such as the threat of perceived attack by the information systems department. depends upon peer pressure to control deviant behaviour. adopts a common language of technical terms, codes and abbreviations. shares the common goal of the department. possesses territory in terms of the boundaries of the department or team within the department. table 1: application of tribal characteristics to organisational departments markarian (1977) defines three points of view which a culturological frame of reference needs to consider. these are the: 23 ajis vol. 7, no 2 may 2000 1. subjects of the activity. 2. fields of activity, showing the object and goals of the human activity. 3. mode of activity, describing how the human activity is implemented. in information systems development, attention has been focused on the field and mode of activity (the 'what' and the 'how'). denison et al. (1996) propose a three domain model to measure the effectiveness of cross-functional teams which also focuses on the fields and modes of activity. in contrast, the following sections focus on the subjects of the activity and how this focus can improve communication in information systems development. table 1 demonstrates that as tribes, users and systems developers have distinct cultures. the richness of culture is reflected in the range of elements that have been identified to explore it. for example, hofstede (1991, 1998) identifies symbols, heroes, rituals and at the deepest level, values as manifestations of culture; johnson (1992) proposes a cultural web as a device for undertaking a cultural audit of an organisation which examines: symbols, power structures, organisational structures, controls, rituals and stories. figure 1 identifies manifestations of culture which affect communication in information systems development. each of the cultural communication barriers identified in figure 1 are discussed, starting with the most external barrier. tribe of users values cognition language terminology behaviour behaviour terminology language cognition values tribe of systems developers figure 1: cultural communication barriers in information systems development terminology every area of human activity has its own specialist terminology; the computer industry is no exception. language is an element of culture which assists in maintaining the differentiation between cultural units. levin (1965) reports that specialist language makes communication difficult even between specialists in different branches of the same field. the increasing familiarity of computers is perhaps accompanied by the wider acceptance of computer terminology beyond the specialist area. anderson (1993) suggests that if this were true, there would be a convergence of users' jargon with information systems developers' jargon. anderson examined professional publications from these different groups for their use of each others jargon and reports a movement of each community away from its own jargon, but not towards the other's. this implies the possible existence of an emerging common jargon, a public language for communication between tribes. although computer specialists should refrain from using jargon during communication with business departments, the problem is not limited to computing. engineering, finance, personnel and sales departments all use specialist terminology which is not widely acknowledged outside the particular area of expertise. all departments must 24 ajis vol. 7, no 2 may 2000 recognise that the language they use may not be understood outside their department and such terms should be explained or avoided. use of specialist terminology is a barrier to effective communication but an inherent aspect of cultural foundation. the adoption of specialist language within a community fulfils part of the human need to belong but also forms a barrier to exclude others from the group. language the relationship between language and culture is most clearly shown in the vocabulary of a language which reflects the cultural emphasis and the ways in which the culture categorises its environment (nanda, 1991). different communities and individuals within the communities may attribute different meanings to words. shekelton (1991) recognises that words help us to position ourselves to interact in the world and that although words are not the most accurate tool with which to analyse reality, they are only challenged when a problem of communication arises. misunderstandings are a natural occurrence in daily life, however, the implications are more serious when these ambiguities occur during specification of the business requirements which a computer system is required to satisfy. meaningful communication is dependent upon the prior existence of an agreed set of semantic and syntactic rules which the recipient must use in order to interpret the communication received (defined by the helsinki principle, crockett el al, 1991). models in information systems development attempt to overcome the ambiguity of natural language. however, mittermeir et al. (1982) point out that although notation is an important element for efficient communication, both the communicator and the recipient must understand the notation used. proponents of systems development models hold the view that the notation adopted is intuitive, requiring little explanation. for example, avison et al. (1992) report of a particular commercial situation where data flow diagrams were considered to form an appropriate basis for user communication. however, for effective communication to take place, the basic concepts behind the notation must first be understood. when entering another community, it is necessary to consider the use of language from a different point of view. shekelton (1991) suggests that we should be more cautious of our use of words, considering the possible interpretation which may be applied to the words used. cognition cognition may be defined as the way in which humans perceive, understand and organise their response to the environment (nanda, 1991). leach (1965) proposes that to understand the relationship between a society and its environment it is necessary to understand the verbal categories which explain how the society organises its environment. hofstede (1991) states that words are the vehicles of cultural transfer as the categories of words available in the cultural language limit the cognitive visions. the cultural background of a person therefore frames their perception of the world (leach, 1976). the environment is meaningless if the code which gives it order is not known, for example, a farmer may be 'lost' in a city because a different code is needed to interact with the city culture. reality is infinite; individuals view reality through a 'cognitive filter' developed through the cultural values and 'norms' of the adopted community. it is necessary to be conscious of the fact that everyone may have a different, valid view of the world. taggart (1982) and oliver & hall (1984) suggest that the barrier between users and systems developers results from a fundamental difference in thinking. they present empirical evidence to support the proposal that systems developers convey objective, reductionist views which are opposed to the subjective, holistic views of the user environment. a systems developer enters a community with the aim of ascertaining an objective view of the situation. hirschheim & neumann (1988) suggest that systems developers are portrayed as being fascinated with the technical aspects of systems and that the culture of systems developers inculcates a belief in their mission to civilise organisations. the members of the community entered have a subjective view of their environment and consider the holistic impact of changes, they are only interested in technology to the extent of its impact on their work. mittermeir et al. (1982) describe this distinction between views as users and systems developers holding different frames of reference. the users perceive a holistic system to adopt in the environment in which they are submerged while analysts decompose the system into objects, data, processes, volumes and time constraints. values weinberg (1971) suggests there are personality differences between computer and non-computer specialists. this is further supported by cronan (1984) who lists different areas in which business users and systems developers feel more strongly. however, ferratt & short (1988) conclude that there are no specific differences, other than those which emerge from the type of work undertaken. for example: a business user may move to a position as a computer specialist and a computer specialist may move to a position in the user community. similar barriers are experienced between the 'user' and systems developer when a computer specialist is the 'user' for a system being developed by another computer specialist as when developing systems for non-computing specialists. even if 25 ajis vol. 7, no 2 may 2000 initially the systems developer and the prospective user belonged to the same community, different roles imposed by the development can divide the community. we suggest that differences between business users and systems developers are cultural differences of values linked to cultural roles at level two of wallerstein's (1990) abstraction, rather than personality differences at level three. people are required to adopt a number of different roles during their daily life but cannot be definitively categorised as other species. humans are not born computer specialists or business users or parents or salesmen, these are roles circumstances lead them to adopt. rather than attributing differences in thinking to different types of people, it can be more adequately presented as differences resulting from adopted roles. contemporary society is complicated as an individual can belong to different groups simultaneously, each of which uses its own linguistic and behavioural codes (leach, 1965; sengupta, 1977; hofstede, 1991). recognition of one group's culture implies the coexistence of further cultures. recognition of business users as a 'tribal unit', implies that the systems development community exists as a foreign tribe which is potentially in opposition. cultural differences occur between the two communities when their behaviour is directed towards different goals (tennekes, 1971). cultural analysis of users and systems development interaction user-systems developer interaction is often synonymous with conflict. hofstede (1991) remarks that an external enemy has always been one of the most effective ways to maintain internal cohesion. this results from the basic belief in many cultures that what is different is dangerous. however, co-operation between rival tribes is sometimes necessary in order to address common problems (hofstede, 1991). we suggest that cultural differences are at the foundation of conflict between users and systems development staff in an organisation. the systems developer enters the tribe of users to analyse the business needs and information requirements. the role of the systems developer has been described as 'archaeologist and scribe' (yourdon, 1989, p 56) whose purpose is to uncover details which may exist only as 'tribal folklore', passed down generations of users. beynon-davies (1990) suggests that there is similarity between the activities of the anthropologist investigating and documenting the life of a social group, and the activities undertaken to analyse information systems requirements. table 2 summarises the tasks of the anthropologist identified by nanda (1991), these tasks are also undertaken by an information systems developer. tasks of an anthropologist observe, learn and ask questions. rely on informants for information and to establish a network of social relationships. establish trust and co-operation in these relationships. participate to understand the difference between what people action they actually take. say they do and the adopt the role of a stranger in another society. table 2: tasks of an anthropologist (based on nanda, 1991) the traditional approach to systems development adopts the ontological assumption that reality exists independently of any observer. objectivity, observing behaviour without judgement, was also considered to be a necessary trait of the anthropologist (tennekes, 1971). however, objectivity has been dismissed on the basis that as an anthropologist is a human being studying other human beings, a completely objective study of another culture is impossible (robb, 1992; nanda, 1991; ackerman, 1965; leach, 1965). anthropology aims to balance both an "insider's" view of a culture from a native point of view, with an external view of an "outsider". both the systems developer and the anthropologist are required to balance participation to appreciate the natives' views with objectivity to perceive patterns and relationships of which a native may not be consciously aware (nanda, 1991). objectivity may initiate further communication problems between tribes. objectivity can be equated with remoteness which alienates both the researcher from his subjects and the subjects from the researcher. an objective detached approach may raise suspicion which further hinders effective communication. when entering a foreign community, it is the task of both the anthropologist and the systems developer to overcome the suspicion of a stranger. however, deliberate efforts to be accepted by the community can have an adverse effect. strategies which exhibit overt friendliness, as presented by joshi (1992) for example, can strengthen rather than diminish the walls of suspicion. suspicion between tribes needs to be transformed into trust. rather than being a constructive approach, a stranger entering a community with the intention of obtaining an objective view of the situation may result in an adverse effect. the members of the community have developed a subjective view of their world and a stranger questioning that view is likely to be run out of town. this is because by challenging a person's view of the world, causes them 26 ajis vol. 7, no 2 may 2000 to doubt the principles on which they have based their very existence. zander (1977) suggests that members of an organisation may use secrecy to protect their culture and prevent the occurrence of undesirable conditions. a more conciliatory relationship may be developed by a more cautious approach which appreciates cultural differences. marcolin & higgins (1992) attribute communication problems between users and systems developers, to systems staff not being customer oriented. systems developers provide a service to business departments and therefore business users are the clients of the systems department. a history of poor communication between the departments with the delivery of late and inadequate computer systems, has strengthened the barriers and tarnished opinions held by both communities. oliver & langford (1984) discuss common myths systems developers have concerning users which illustrate lack of respect to their clients. the myths include 'users keep changing their minds' (p 120), 'users are stupid' (p 122) and 'users react emotionally and illogically' (p 121). these myths reflect the lack of appreciation and understanding of systems developers to their client's situation. the business environment encapsulates many dynamic forces of which the development of a computer system is just one component in the environment of the client. the systems analyst aims to obtain an objective view of the situation at a level of abstraction. the forces impacting the view of the analyst are relatively stable and changes to the business requirements may therefore seem illogical to the analyst. system developers need to appreciate the user's position in a dynamic environment. marcolin & higgins (1992) conclude that the development of a cooperative environment is essential for effective information systems development and can be achieved through mutual respect. the following section discusses the practical implications of applying cultural understanding to improve user-developer communication. practical implications of cultural understanding we have proposed that the difficulties experienced between users and systems developers are inherent to the cultural differences between the two social groups. we believe that an appreciation of these cultural differences can improve communication and interaction between the two groups. this view is supported by hofstede (1991) who considers that negotiations are more likely to succeed if the parties understand the reasons behind their differing views. there is no defined procedure to achieve this level of understanding, however, we suggest key elements to improve all communication include: 1. recognise that the community entered will have different cultural values. 2. appreciate that the community comprises of individuals not stereotypes. 3. develop a conducive context within which to work effectively with a different community. guidelines for improving communication between users and information systems developers are presented in shah etal. (1994). hofstede (1991) suggests that there are three phases to the acquisition of intercultural communication abilities. these are: 1. awareness: recognition that others have different mental software. 2. knowledge: interaction with cultures, leads to knowledge (symbols, heroes and rituals) to understand where values differ. 3. skills: recognition and application of symbols of other culture, experience satisfaction of getting along in the new environment, resolving simpler and more complicated problems of life among others. intercultural contact does not automatically breed mutual understanding, it only confirms each group has its own identity. members of the other group are not perceived as individuals but as stereotypes. wynn (1982, p 349) argues that while user are represented 'two-dimensionally, stereotypically, adversarially' user participation in systems design will be restricted. cultural appreciation in systems development figure 2 summarises recommendations to improve communication in information systems development. explain terminology all departments must recognise that the language they use may not be understood outside the confines of their specialist area. the manipulation of language to reinforce departmental barriers hinders effective communication. there is no standard for considering one group as inferior or superior to another (hofstede, 1991). a business user who does not share the same specialism as a systems developer, is no less inferior to the developer, than the systems developer is to the business specialist for whom the system is being developed. careful use of specialist terminology portrays greater professionalism than irresponsible administration of jargon. 27 ajis vol. 7, no 2 may 2000 tribe of users values ~l cognition language terminology behaviour behaviour terminology language consider use of i 1 language ' explain i i terminology i i respect values i appreciate , i i1 cognitive views i cognition values tribe of systems developers figure 2: improving communication in information systems development consider use of language the development of effective and efficient information systems requires the resolution of communication problems experienced between users and systems developers (christensen, 1991). language is the resource of communication but it is ambiguous; misunderstandings occur within, among and between groups (valusek & fryback, 1985). as shekelton (1991) suggested, both clients and systems developer need to be cautious and consider the use of language from a different perspective. appreciate different cognitive views in addition to possessing, universal characteristics and the characteristics of a particular group, a person also possesses idiosyncratic characteristics (wallerstein, 1990). this combination of characteristics constructs a unique individual with a different, valid view of the world. it is therefore necessary to be aware that different frames of reference exist. respect values an individual is a member of many groups each of which has a different culture and within each group, the individual adopts particular roles. in the user-systems developer relationship, the individuals involved adopt specific roles which represent their values. an appreciation of the different roles in a relationship is necessary to improve communication and interaction. summary of approach the approach of cultural appreciation advocated in this paper is summarised by segall el al. (1990, p 316) who propose that 'to avoid impending ecological catastrophes....we ought to identify with the whole human race as enthusiastically as we presently do with our own tribes'. although lack of cultural appreciation will not result in 'ecological catastrophe', the results may prove just as catastrophic to the organisation concerned. 28 ajis vol. 7, no 2 may 2000 conclusion information systems development requires effective interaction between systems development and other departments. we suggest basic cultural differences are the origin of both the cause and resolution of interdepartmental communication problems. organisations, functions, departments and offices form specialisations which exhibit distinct 'sub-cultures' in the anthropological sense (bell, 1979). a cultural system emerges within a department which may be distinct from that of a neighbouring department or the corporate culture. communication barriers are erected between these communities to retain and protect individual departmental cultures. cultural systems are a necessary part of human activity, fulfilling the human need to identify with a social group, dictating how an individual perceives their environment which is manifested in, for example, their use of language. 'outsiders' to the culture are treated with suspicion and 'insiders' unite to defend their community against the perceived threat of attack. we suggest that both effective interdepartmental and international communication cannot be achieved by attempting to change, persuade or convert the other tribe but through the appreciation and understanding of different cultures. references ackerman, j.s. (1965) "on scientia", in: halton, g. (ed.), science and culture: a study of cohesive and disjunctive forces, boston: beacon press, pp 14-23.anderson, j.m. (1993) "jargon convergence as a measure of alignment in understanding between management and information systems professionals", ibs computing quarterly, winter, pp 8-15.avison, d.e., shah, h.u., powell, r.s. & uppal, p.s. (1992) "applying methodologies for information systems development", journal of information technology, vol 7 no 3, pp 127-140.barkema, h., bell, j. & pennings, j. (1996) "foreign entry, cultural barriers and learning", strategic management journal, vol 17, pp 151-166.baroudi, j.j., olson, m.h. & ives, b. (1986) "an empirical study of the impact of user involvement on systems usage and information satisfaction", communications of the acm, vol 29 no 3, pp 232-238.becher, t. (1989) academic tribes and territories: intellectual enquiry and the cultures of disciplines, milton keynes: open university press. bell, d. (1979) 'the disjunction of culture and social structure: some notes on the meaning of social reality", in: barret, m., corrigan, p., kuhn, a. & wolff, j. (eds.), ideology and cultural production, great britain: croom helm ltd, pp 236-250.benbasat, i., dexter, a.s. & mantha, r.w. (1980) "impact of organisational maturity on information system skill needs", mis quarterly, vol 4 no 1, pp 21-34. berger, m. (1996) cross-cultural team building: guidelines for more effective communication and negotiation, maidenhead: mcgraw-hill. beynon-davies, p. (1990) "the behaviour of systems analysts", computer bulletin, march, pp 21-23. bird, e. (1979) "aesthetic neutrality and the sociology of art", in: barret, m., corrigan, p., kuhn, a. & wolff, j. (eds.), ideology and cultural production, great britain: croom helm ltd, pp 25-48,. bostrom, r.p. (1989) "successful application of communication techniques to improve the systems development process", information and management, vol 16 no 5, pp 279-295. cavaye, a.l.m. & christiansen, j.k. (1996) "understanding is implementation by estimating power of subunits", european journal of information systems, vol 5 no 4, pp 222-232. christensen, d.m. (1991) "the gap between systems developers and users", information systems management, vol 8 no 4, pp 73-75. crockett, h.d., guynes, j. & slinkman, c.w. (1991) "framework for development of conceptual data modelling techniques", information and software technology, vol 22 no 2, pp 134-141. cronan, t.p. & means, t.l. (1984) "systems development: an empirical study of user communication", database, vol 15 no 3, pp 25-33. denison, d.r., hart, s.l. & kahn, j.a. (1996) "from chimneys to cross-functional teams: developing and validating a diagnostic model", academy of management journal, vol 39 no 4, pp 005-1023. dingley, s. (1996) "a composite framework for the strategic alignment of information systems development", phd thesis, department of computer science and applied mathematics, aston university in birmingham, uk. ferratt, t.w. & short, l.e. (1988) "are information systems people different ? an investigation of how they are and should be managed", mis quarterly, vol 12 no 3, pp 427-443. gingras, l. & mclean, r.l. (1982) "designers and users of information systems: a study of differing profiles", proceedings of the 3rd international conference on information systems, ann arbor, michigan, pp 169181. guinan, p.j. & bostrom, r.p. (1986) "development of computer-based information systems: a communications framework", data base, vol 17 no 3, pp 3-16. hirschheim, r. & newman, m. (1988) "information systems and user resistance: theory and practice", the computer journal, vol 31 no 5, pp 398-408. hofstede, g. (1991) cultures and organisations: software of the mind: intercultural cooperation and its importance for survival, maidenhead: mcgraw-hill. 29 ajis vol. 7, no 2 may 2000 hofstede, g. (1998) "identifying organizational subcultures: an empirical approach", journal of management studies, vol 35 no 1, pp 1-12. hunter, m.g. & beck, j.e. (1996) "a cross-cultural comparison of "excellent" systems analysts", information systems journal, vol 6 no 4, pp 261-281. johnson, g. (1992) "managing strategic change strategy, culture and action", long range planning, vol 25 no l,pp 28-36. johnson, r. (1979) "histories of culture / theories of ideology: notes of an impasse", in: barret, m., corrigan, p., kuhn, a. & wolff, j. (eds.), ideology and cultural production, great britain: croom helm ltd., pp 49-77. joshi, k. (1992) "interpersonal skills for cooperative user-analyst relationships: some research issues", data base, vol 23 no l.pp 23-25. kaiser, k.m. & bostrom, r.p. (1982) "personality characteristics of mis project teams: an empirical study and action-research design", mis quarterly, vol 6 no 4, pp 43-60. kaiser, k.m. & king, w.r. (1982) "the manager analyst interface in systems development", mis quarterly, vol6nol,pp49-59. kaulio, m.a. & karlsson, m. (1998) "triangulation strategies in user requirements investigations: a case study on the development of an it-mediated service", behaviour and information technology, vol 17 no 2, pp 103-112. kumar, k. & welke, r.j. (1984) "implementation failure and system developer values: assumption, truisms, and empirical evidence", proceedings of the icis conference, tuscon, arizona, pp 1-17. leach, e.r. (1965) "culture and social cohesion", in: halton, g., (ed.), science and culture: a study of cohesive and disjunctive forces, boston : beacon press, pp 24-38. leach, e. (1976) culture and communication: the logic by which symbols are connected: an introduction to the use of structuralist analysis in social anthropology, cambridge: cambridge university press. levin, h. (1965) "semantics of culture", in: halton, g., (ed.), science and culture: a study of cohesive and disjunctive forces, beacon press, boston, pp 1-13. lucas, h.c. jr. (1984) "organisational power and the information services department", communications of the acm, vol 27 no 1, pp 58-65. marcolin, b.l. & higgins, c.a. (1992) "effectively dealing with is user complaints", journal of systems management, vol 44 no 12, pp 29-36. markarian, e.s. (1977) "the concept of culture in the system of modern science", in: bernadi, b. (ed.), the concepts and dynamics of culture, netherlands: mouton publisher, pp 103-118. markus, m.l. & bjorn-anderson, n. (1987) "power over users: its exercise by systems professionals", communications of the acm, vol 30 no 6, pp 498-504. martin, m.p. & fuerst, w.l. (1984) "communications framework for systems design", journal of systems management, vol 35 no 3, pp 18-25. meyer, h-d. (1993) "the cultural gap in long-term international work groups: a german-american case study", european management journal, vol 11 no 1, pp 93-101. mittermeir, r.t., hsia, p. & yen, r.t. (1982) "alternatives to overcome communication problem of formal requirements analysis", in: gallicrs, r. (ed.), information analysis selected readings, wokingham: addison-wesley, pp 153-165. nanda, s. (1991) cultural anthropology, belmont: wadsworth publishing company. oliver, i. & hall, a. (1984) "a repertory grid study of the systems analyst client interface", international journal of man-machine studies. oliver, i. & langford, h. (1984) "myths of demons and users: evidence and analysis of negative perceptions of users", galliers, r. (ed.), information analysis selected readings, wokingham: addison-wesley, pp 113123. park, s.h. & ungson, g. r. (1997) "the effect of national culture, organizational complementarity, and economic motivation on joint venture dissolution", academy of management journal, vol 40 no 2, pp 279-307. parkinson, r.n. (1993) "international group working", internal report, department of computer science and applied mathematics, aston university in birmingham, uk. robb, f.r. (1992) "organisations, institutions and management", systemist, vol 14 no 1 pp 15-25. robey, d. & farrow, d. (1982) "user involvement in information systems development: a conflict model and empirical test", management science, vol 28 no 1, pp 73-85. rourke, g. & elliot, g. (1996) "strategic management of the culture gap", proceedings of the bit'96 conference, business information systems uncertain future, vol 2, manchester metropolitan university, manchester, uk, pp 209-219. saunders, c.s. & scamell, r.w. (1986) "organisational power and the information services department: a reexamination", communications of the acm, vol 29 no 2, pp 142-147. segall, m.h., dasen, p.r., berry, j.w. & poortinga, y.h. (1990) human behaviour in global perspective: an introduction to cross-cultural psychology, oxford: pergamon press. 30 ajis vol. 7, no 2 may 2000 sengupta, d. (1977) "a unified theory of society field aspect", in: bernadi, b. (ed.), the concepts and dynamics of culture, netherlands: mouton publisher, pp 119-140. shah, h.u., dingley, s. & colder, p. (1994) "bridging the culture gap between users and developers", journal of systems management, vol 45 no 7, pp 18-21. shekelton, j.f. (1991) "aristotle, the entity model and neurosis", boxes and arrows, vol 92, pp 11-14. smith, a.d. (1990) "towards a global culture?", theory, culture and society, vol 7 no 2-3, pp 171-191. taggart, jr. w.m. (1982) "the other half of the systems development potential: are we half-brained systems professionals?", computer personnel, vol 9 no 1, pp 17-22. tan, w-g. & gable, g.g. (1997) "maintaining centralised application systems: a cross-country, cross-sector, cross-platform comparison", european journal of information systems, vol 6 no 4, pp 193-207. tennekes, j. (1971) anthropology relativism and method: an inquiry into the methodological principles of a science of culture, netherlands: van gorcum and company. valusek, j.r. & fryback, d.g. (1985) "information requirements: obstacles within, among and between participants", in: galliers, r. (ed.), information analysis selected readings, wokingham: addison-wesley, pp 139-151. wallerstein, i. (1990) "culture as the ideological battleground of the modern world-system", theory, culture and society, vol 7 no 2-3, pp 31-55. walmsley, j. (1986) brit-think ameri-think: a transatlantic survival guide, london: harrap books. weinberg, g.m. (1971) the psychology of computer programming, . van new york: nostrand reinhold. werner, r. (1979) "the business systems professional", mis quarterly, vol 3 no 1, pp 65-66. wynn, e.h. (1982) "the user as a representation issue in the us", systems design for, with, and by, the users, proceedings of the ifip wg 9.1 working conference on systems design for, with, and by the users (briefs u, ciborra c and schneider l, eds), netherlands: north-holland publishing company, pp 349-358. yourdon, e. (1989) modern structured analysis, new york: prentice hall. zander, a. (1977) groups at work, united states of america: jossey-bass publishers. 31 microsoft word vol 15 2 final.doc australasian journal of information systems volume 15 number 2, 2009 5 addendum july 2008 era replaces rqf julie fisher caulfield school of information technology, monash university, victoria, australia email: julie.fisher@infotech.monash.edu.au g shanks department of information systems, faculty of science, melbourne university, victoria, australia email: gshanks@unimelb.edu.au j lamp school of information systems, faculty of business and law, deakin university, victoria, australia email: john.lamp@deakin.edu.au introduction since this paper was written and published (fisher, shanks, & lamp, 2007) further work has been undertaken on the development of the australian information systems journal and conference rankings list. more details became available late in 2007 on the dest requirements relating to journal and conference rankings for the rqf. a newly installed federal government resulted in the abandonment of the rqf, replaced with era (excellence in research for australia). it was therefore appropriate to provide an update of the revised ranking list and the process undertaken to arrive at the updated list. although the rqf in its current form has been abandoned, era also requires a ranked journal list. dest requirements during 2007, more information relating to the impending rqf was released. information systems was accepted as a separate discipline for the purposes of the rqf and this continues with era (field of research 0806). the australian information systems community was required in 2007 to provide dest with a journal and conference rankings list for the discipline. furthermore, dest provided guidelines for the ranking scheme, suggesting that 5% of journals should be ranked a+, 15% should be ranked a, 30% should be ranked b and the rest should be ranked c. although no hard guidelines were articulated for era, arc officials have since stated that these percentages will stand. australasian journal of information systems volume 15 number 2, 2009 6 2007 ranking activities in order to achieve a reasonable number of a+ journals the overall list needed to be expanded from the original 30. once the new dest requirements were known, additional journals were identified for the list. this was done by asking academics in the australian information systems community to nominate journals they thought should be included. journals from the index of information systems journals (http://lamp.infosys.deakin.edu.au/journals/) were identified and added, and journals previously excluded were reconsidered. the new list comprised 182 journals. information on rankings for these journals was compiled and a new ranking developed. prior to the september 2007 workshop of acphis, the ranking list was circulated to all acphis members for comment and changes to the list were made. acphis workshop adelaide 2007 at the acphis 2007 workshop further discussion of the ranking list took place. in november 2007 the ais published what they described as the "senior scholars" list which comprised eight journals (http://home.aisnet.org/displaycommon.cfm?an=1&subarticlenbr=378). the group agreed that seven of these journals would be included in our a+ journals, journal of strategic information systems was left out and decision support systems and information and management included. the new list was reviewed and any re-ranking as a result of this review took place by the acphis members, representing the most senior and eminent scholars in the australian information systems community. the revised version of the list was circulated again to all members of acphis for comment. further adjustments to the list were made based on this feedback. the list in this addendum is the one that was provided to dest in november 2007, although for conciseness only journals ranked a+, a and b journals are included. for the purposes of era the current list was accepted in 2008 without any adjustment. the full list can be found at www.acphis.org.au. any changes made as a result of the era ranking process will also be reflected on the website. the list will be regularly reviewed and updated and made available through the acphis website. fisher, j., shanks, g., & lamp, j. (2007). a ranking list for information systems journals. australasian journal of information systems, 14(1), 5-18. ajis vol. 4 no. 2 may 1997 standardising the information technology environment at telstra corporation: a case study carey butler and peter weill key centre for technology management, melbourne business school, university of melbourne email: c.butler@mbs.unimelb.edu.au and p.weill@mbs.unimelb.edu.au abstract telstra is australia's leading telecommunications carrier and largest single user of information technology (it). in november 1990, the australian federal government announced wide-ranging reforms and the introduction of competition within the telecommunications sector. realising that it was essential for the corporation to leverage its use of technology on a firm-wide basis in order to remain a strong and flexible competitor, in 1991, telstra embarked upon a major overhaul of its existing it infrastructure. central to this was the establishment of standardised operating environments (soe) for all computing resources used at telstra. this paper traces the development of telstra's data network and desktop soes. the paper examines the strategic drivers for the soes, the business benefits, and issues related to telstra's computing environment prior to, and after the introduction of network and desktop soes. introduction "our industry is in transition. rapidly developing new technologies, new markets, fierce competition and higher customer expectations are combining to generate change on a scale never experienced in the australian telecommunications industry" frank blount, ceo, telstra annual report 1994. in late 1994, brian lovelock, telstra's cio, sat in his office at telstra's corporate headquarters in melbourne, australia and contemplated the future. over the last three years the australian telecommunications marketplace had changed remarkably since the introduction of a competitor to telstra's traditionally monopolistic domain. significant change had also occurred within telstra itself, as it rose to the challenge of competition by improving customer responsiveness, and striving for service excellence. since 1991, telstra had also undertaken a major overhaul and rationalisation of its internal communications environment. this change had standardised the company's voice and data platforms, and greatly improved access to information resources across the firm. more change was necessary' lovelock realised, as his eyes scanned the melbourne cityscape far below. full unrestricted competition in the telecommunications sector was planned for 1997. lovelock wondered what this would mean for telstra, and whether the information technology initiatives implemented so far would be sufficient to support future business needs as the marketplace underwent yet another major transformation. background telecommunications environment in australia for many years, the provision of telecommunications services in australia was the sole domain of three government-owned instrumentalities the domestic carrier9 telecom, the international carrier otc10, and the national satellite carrier aussat. in 1987, the commonwealth government of australia undertook a fundamental review of the australian telecommunications industry. in november 1990, this resulted in the announcement of wide-ranging reforms to the basic structural and regulatory framework of the telecommunications sector. reforms focused on the introduction of broad-based competition and a restructuring of the three monopoly carriers. in 1991/92, this resulted in the establishment of a competitive duopoly, based on a single government-owned carrier (aotc)11 and a private sector carrier (optus communications)12. both telcos offer domestic and international services, with optus currently holding around 10% marketshare. 9 a term for a telephone company. carriers are also known as telcos. otc overseas telecommunications corporation. the australian and overseas telecommunications corporation was formed by the merger of the existing domestic and international carriers telecom and otc. the formation of a second carrier was conditional upon the privatisation of aussat. 25 ajis vol. 4 no. 2 may 1997 cessation of the duopoly is planned for july 1, 1997 when the australian telecommunications marketplace will be opened up to full, unrestricted competition. telecommunications is one of the most rapidly expanding industry sectors in australia. the last decade has seen the introduction of consumer choice in terms of a wide range of different products, equipment and services. strong growth has occurred in long distance and international telephone call rates. cellular mobile calls have grown at an astounding rate increasing from 3 million in 1987, to 319 million in 1992 (s&p 1993a), with one in every fifteen australians now owning a mobile phone. the formation of telstra corporation in 1991, telecom and otc merged their operations to become aotc13. in 1993, the name aotc was replaced by telstra corporation. the company provides a full range of telecommunication services and products to domestic and international customers. although government-owned, telstra operates as a separate commercial enterprise, employing some 60,000 staff. telstra is headquartered in melbourne, and owns around 10,000 sites14 throughout australia. it also has business offices or joint ventures in 20 countries around the world. the high growth asia/pacific region is a particular area of current interest and investment. internationally, telstra owns around 2% of telephone lines worldwide, is the third largest owner of submarine cables in the world, and has one of the largest digitised terrestrial networks in the world with over 1,000,000 km of fibre optic cable in active service. within australia, telstra currently handles around 10 million telephone calls per day. the company's customer base is the largest of any australian enterprise, with more than 95.5% of australian households connected to its domestic telephone network. organisational structure: the merger of telecom and otc prompted the need for wide-ranging realignment and restructuring within the new company in order to eliminate duplication of functions and other inefficiencies. "the changes we have made deal with the very structure of our organisation and with all of our systems: management, financial, operating and product/service development", said telstra's chief executive officer (ceo), frank blount (aotc 1992a). change was driven by the broad-based needs of the new competitive market and by specific customer needs. the corporation reorganised itself into a number of semi-autonomous business divisions15 (see figure 1), and a corporate centre function. each division has its own customers, its own assets, its own team of employees, and its own culture within the context of the overall organisational culture. the corporate centre provides support to the various business divisions, and is responsible for planning and decision-making related to the long term strategic direction of the firm. in 1994, there were six major businesses, each with commercial accountability and a central focus on servicing the needs of a particular customer group. "for example, corporate consumers are handled separately to residential consumers", said michael bennett, former national manager for telstra desktop solutions. "we are aware that our customers have a choice in service providers, and that we have to do all we can to win those customers each and every day." at the same time the divisions were established, shared resource units (srus) for activities such as accounting, supply, property management and a corporate level information technology group (itg) were also set up to provide services across the customer divisions. "the srus transfer price their costs to the divisions and are required to be competitive with service supply external to telstra", said bennett. the sheer size and scope of telstra's operations, means that single business units are often dispersed over multiple geographic locations. commercial and consumer (see figure 1) is a case in point; with over 32,000 staff, it is the firm's largest business unit. the corporation has established an organisation characterised by individual, multi-disciplined teams of people, focused on unique markets. this arrangement enables the various businesses to quickly respond to the changing needs and demands of customers, while still working within an 'envelope' of corporate-wide expertise. of the 2 entities, telecom was about 5.5 times larger than otc in terms of revenue. includes business offices, line depots, major telephone exchanges, and radio terminal stations. this followed on from work done with mckinsey consulting in 1987 which had organised telecom along business unit lines. 26 ajis vol. 4 no. 2 may 1997 figure 1: business divisions at telstra finance and administration is part of the corporate centre, and has responsibility for corporate and strategic functions such as corporate finance, treasury, audit, business planning and regulatory affairs, corporate affairs, and legal matters. corporate, international and enterprises corporate provides services to around 3,000 of australia's largest enterprises from the corporate and government sectors. services include: customised voice and data transfer, valueadded services, facilities management and systems integration. international is structured around the core of the former otc. this unit provides overseas telecommunication services and products. customers are typically australian organisations with significant offshore operations, and foreign multinationals enterprises operates two lines of business mobile communications services, and directory products and services. each line is responsible for its own operating profit, controls its own assets, and manages its own product development, sales and marketing teams. commercial & consumer commercial provides products and services to 840,000 small and medium-sized australian businesses. this represents more than 80% of telephone services. consumer delivers telecommunication and information services to around 6 million residential customers. network and technology owns telstra's core network, and oversees network planning, development, construction and operations. the unit is also responsible for developing network products, and for the wholesale marketing of these to telstra's own retailing business units and to other carriers. business strategy: the merger of telecom in 1991 with the smaller, but highly profitable otc, resulted in an organisation with a strong presence16 across all market segments. the new corporation has sought to protect its revenue and marketshare by: • introducing new products and services which focus on customer needs; • aggressively marketing its services; • improving the range and quality of its services through network modernisation. in 1994, telstra announced a a$3.3b network modernisation program for its australian network; • introducing new operational systems; and • re-orienting itself to customer focused performance targets (s&p 1993b). these initiatives have been effective in retarding revenue loss as competition increases in the marketplace, and are clearly reflected in telstra's corporate plan which has a strong customer focus, and is geared towards "the aggressive pursuit of business success within both the newly competitive australian market and offshore." (aotc 1992a). key business imperatives include: • achieving excellence in customer service and value; • striving for world's best practice financial performance, including competitive pricing and management of costs, capital efficiency and systems support; and • driving business growth, through the development of new markets in australia and offshore. in 1994, telstra led in all markets segments except mobile communications. 27 ajis vol. 4 no. 2 may 1997 despite competition and the inevitable erosion of some market share, telstra is in a sound position. to date, it has increased profit and revenue, expanded its markets both domestically and offshore improved customer service, and contained spending in most areas. the corporation has clear goals for the future which include making it the first choice among customers with telecommunications needs, becoming a leading player in the asia-pacific region, and ultimately, one of the most effective, most efficient and most innovative telecommunications companies in the world. telstra it environment (1994) telstra is australia's largest single user of information technology (it), with an annual expenditure in this area of many millions of dollars. more than half of the 60,000 staff employed by telstra use it applications as part of their every day work. many of these applications are telstra specific, mainframe-based systems, but other user applications such as wordprocessing, spreadsheets and email are microcomputer-based. telstra is the largest microsoft mail site in the world. conceptual framework for it: a conceptual framework for the development and delivery of all major information systems for telstra operations is provided by the firm's overall systems architecture (osa). osa is in turn based on a number of business drivers which are a direct extension of the firm's key business imperatives. osa provides the architecture to integrate telstra's business processes, its systems and its network. osa specifies a number of discrete processing layers17, and focuses attention on system "building blocks" groupings of business functionality and data. osa imperatives include ensuring: • provision of the highest quality customer services; • provision of timely, accurate and relevant management information so that business managers can better direct, manage and control the business; • development of business systems that meet telstra's current and future business needs; • alignment of systems with business processes; • core applications are adequately integrated, and major hardware platforms are compatible; • it maximum value for money is obtained from expenditure on it; • capabilities are fully exploited to assist the introduction of new business activities and new work practices; and • reduction of costs through more efficient and effective use of network resources. a summary of osa's key principles, major drivers, and its linkages with telstra's main business functions and processes is shown in figure 2. current it platform: in australia, telstra has two major data centres accommodating a wide variety of hardware, and a network size of 1,500 locations supporting around 46,000 terminals. 87% of these terminals are personal computers (abm 1994). telstra's current hardware platform includes eight amdahl mainframes, two ibm es/9000 mainframes, seven bull mainframes, and a number of hp, pyramid and sun midrange processors. systems running on the bull mainframes are gradually being migrated over to the ibm machines. telstra uses unix and a number of proprietary operating systems. database management systems include db2, adabas and oracle. sap's mainframe-based r/2 is currently being implemented to manage back office operations. core application areas include: billing, customer management, service delivery, trouble management", product management and network plant assignment. telstra has a history of developing its it solutions inhouse. "for large scale developments that approach is still true", said john mitchell, telstra's manager for network rationalisation. "we have a strong ethos of inhouse support and development, partly because of the control that this approach confers, but mainly because many of our big systems are very specific to telstra." which are aligned (o established or emerging international communications standards. 28 ^ c cd _cd cd cd 2 x: •5 g ob cd w iii o 2« $> £e !c "s-2 j7 1̂ s— xj co (d = w to i y j=*^ at s ys te m s m us t b e de si gn ed to s er ve ob je ct iv es o f t he b us in es s pr oc es se : • c '3 £ s th e im po rta nc e of p ro je ct s no t c on ti to th e ob je ct iv es o f a p ro ce ss m us t t qu es tio ne d • 6 5 cd 0) in fo rm at io n ca pt ur e m us t s up po rt th i en d pr oc es s . 0) i & ys te m s an d n et w or k te ch no lo gy i ite gr at ed a nd a lig ne d w ith b us in e: ro ce ss es in o rd er to a ch ie ve re al us in es s br ea kt hr ou gh s co 5 0a co •o cd rt he se th re e el em en ts m us t b e pl an r de si gn ed to ge th er • cd xi •o £ s 3 xi st in g s ys te m s m us t be ra tio n a l!: th e ex is tin g 40 0+ n et w or k sy st em s re du ce d in n um be r to a bo ut 8 0 uj . co co § co at a q u a lit y m us t be im pr ov ed c or e da ta ba se s w ill p ro vi de s ha re d to k ey b us in es s in fo rm at io n q . co 13 d .2 eu b us in es s pr oc es se s an d sy st em s m en su re t ha t f in an ci al a nd n on -fi na nc ar e ca pt ur ed a nd m ai nt ai ne d to ge th i . «^ 'ils>!2 th e us ef ul ne ss o f da ta m us t be re cc be yo nd th e ar ea im m ed ia te ly r es po r its c ap tu re , so th at it is n ot lo st • 1 4> +* 0) fle xi b le s ys te m s in fr a st ru ct u re is i th e p ro vi si o n o f t e ls tr a 's s er vi ce < s c 3 o ^ fii n ew s ys te m s w ill p ro vi de a fo un da ti w hi ch n ew s er vi ce s ca n be a dd ed w m aj or m od ifi ca tio ns • cd xi s2 "c<g m id dl ew ar e w ill b rid ge d iff er en t t ec h pl at fo rm s an d al lo w fu nc tio n an d da l sh ar ed b et w ee n ap pl ic at io ns • ^\ covj %*• li! ® »^ o 8^ « s3cd ^o »j2 c j2 nf 0) ic^ .2 o 0 i co i cd o 'w (0 o cd « i 1 1 ~oc ,*d c g 13 c "q ,0 a v is io n ap pl ic at io ns s ho ul d b e b ui lt, a nd 03 x: "5 co r> r vx: ^* cd k. 13 {j q> "1 ^c 2 o 1 q. < 1 qj o) 0 ^ co 0} o co •s co 13 c co io £• w •d c c o *-*•jj £ 1 j= s a t ec hn w or ki ng o f t he b us in es s u ni ts ^ 1 cd 1 cd tr n n 3 co o c co q. co c g 13co "c 11 < co s q. 1 1 ll ii •ii fl s u o -c -c — "3 l» it 12 it? fe. i 5! -c •2?-5 o co i •5 i q o^ a d riv er s th e b us in es s q ii _._ ~* cd o js to m er s er vi ce . a ll in d fu nc tio na lit y re qi im er n ee ds m us t b e av ai la bl e at th e. cu st • "o ne s to p" c i in fo rm at io n a se rv ic e cu st c im m ed ia te ly se rv ic e po in t co cd (d -* 0 co soco n ic es se s an d w or k pr si ng r een gi ne er ed t i ex ib le a nd e ffi ci en t • b us in es s pr o th es e ar e be th em m or e fli •dc >< v '5cr i •a2 a. cd c • in tro du ct io n i ea si ly _> 3= cd ' co st s th ro ug h m or e us e of t ec hn ol og y • r ed uc tio n of an d ef fic ie nt cd 0 u .§• co d" to e o> x: cd • p ro vi si on o f 1 i2 cd n ab ilit y to m ee t c us ti aj or th em e of o s a se rv ic e. t he ne ed s is a m c co cd £ .c "o in :im el y, a cc ur at e an d t i nf or m at io n in o rd e na ge rs c an b et te r di co nt ro l t he b us in es s • p ro vi si on o f 1 m an ag em en i bu si ne ss m a m an ag e an d ajis vol. 4 no. 2 may 1997 the need for a standard operating environment (soe) host systems connectivity: telstra's use of it is longstanding, and during the 1970s and 1980s was based around large, independent mainframe systems using single dumb terminal workstations to service one application. "as we progressed through the 1980s", said mitchell "it -became increasingly apparent that this approach was insufficient for business needs. the number of applications grew, mainframe usage grew dramatically, and there was a need for terminals to be multifunctional. because of system incompatibilities, a situation developed where some people had three workstations on their desks. each of these was used for a different application or for accessing a different system. clearly this was not sensible, scalable or sustainable." bob james, telstra's group general manager for strategic development agreed: "when people had three terminals on their desks, it was easy to visualise five or even ten. we realised we needed single terminal access, but dumb terminals that could handle multiple protocols didn't exist. around 1987/88, lans started to become commercially viable, minicomputers appeared, and pcs were becoming serious business support tools." in 1988, telstra partially solved the problem of host connectivity by moving to a pc lan (refer figure 3) environment, and developing drift. drift is an inhouse protocol emulation software toolkit that transparently links to several different mainframes irrespective of platform, thereby enabling corporate data to be accessed from the desktop. drift allows a single pc to do the job of up to nine dumb terminals, and has saved telstra the considerable expense of rewriting many of its mainframe applications. "all this was done under d/s/vef18", said bennett, "an early network that really pushed the move to lans. another driver was that pcs with emulation cards were now actually cheaper than dumb terminals. drift resided on a pc connected to a lan file server, which connected to a communications server, which in turn connected to austpac. any user connected to the file server could get access to host system applications." workgroup connectivity: for many years telstra had taken a decentralised approach towards it, in allowing its various businesses to conceive and implement it solutions tailored to their individual needs. while this method was generally effective at a local level, corporate-wide pooling and interchange of information, was difficult, time consuming and frequently impossible. telstra has thousands of buildings, so workgroups are often spread over a number of geographic sites. by the late 1980s, telstra's technology base had evolved into a complex series of numerous, largely isolated networks based on many different vendor offerings. electronic mail could be sent within individual businesses but not necessarily firmwide because of platform and/or email software differences. network management was also an issue. "on one of the larger networks, there were problems like "broadcast storms", where traffic would just avalanche through the network and bring it to crash point", said james. cost and duplication of effort were other issues. "many of the networks almost paralleled each other", said james. "the corporate customer division had individual melbourne to sydney links, so did business services and so did the residential division, and yet they were all part of the same company. the cost of communications to telstra was immense, particularly when you consider that any one group in the company is as large as the australian federal government in terms of spend. there was a lot of inconsistency and incompatibility. telstra's purchasing leverage was not being maximised." responsibility for it decisions in terms of systems development, equipment and software choice was largely localised. "what we ended up with", recalled james, "was a situation where every organisation within telstra had different data networks and voice facilities, had its own it group, and its own ideas about which pc environment (dos or macintosh), email package or office applications should be purchased." for staff moving to jobs in different areas of telstra, this often entailed a substantial learning curve with every new job move. "another problem", commented brian lovelock, "is that organisational entities within telstra tend to have a half-life of about 18 months. so if pan of business services had moved into the corporate customer division they would have had to change their network !" by the late 1980s, telstra was operating twenty six large and separate networks, with little ability to intercommunicate. "that's when things came to a head for us", recalled james "and we realised the need for a solution not only at the network level, but down to the desktop as well." but it was the business drivers that were really the key motivators. 18 distributed information systems network. 19 australia's principal public packet switched network which uses x.25 protocols. 31 ajis vol. 4 no. 2 may 1997 "telstra's move in the late 1980s to a business driven, customer focus", said bennett "underlined an urgent need to move away from the isolated systems of the pre-competitive era, towards developing integrated information systems that could be used across the organisation. our ceo had identified a number of key business imperatives. these were used as the basis for development of our overall systems architecture principles which further expanded them in the areas where it played a decisive role. you only have to glance at these to realise that the focus is on satisfying the customer and on more efficient use of our internal communications networks." maximising the effect on customer focus will be a lot easier with a communications environment that exhibits synergy between systems, and an ongoing flexibility to grow apace with changing company and customer needs. we are aiming for a level of integration where all the information and the functionality required to service any customer need is immediately available from the one service point the "one-stop" shop approach. and this is regardless of where the information is stored or on which system. a return to how we were in the late 1980s is now unthinkable." development of a standard operating environment in 1990, a corporate-level information technology group (itg) was formed with a specific mandate to devise and implement a technology framework for integrating telstra's internal communication systems, reducing associated costs, and improving customer access across the firm. "this was a first for telstra", said bennett. "prior to this, no such group had ever existed." itg has responsibility for it policies and architectures, strategic planning, management of core projects and provision of it services. the businesses are responsible for tactical planning, expenditure, definition of requirements and management of non-core projects. "itg has been entrusted with forming the needed balance between building a centralised it infrastructure and working with the bus to develop the systems they require", said bennett. a key initiative for achieving these objectives was the development of a consistent environment for voice and data networks that would bridge current technology islands, and provide a fast, flexible, reliable platform that was independent of geographical or organisational boundaries. bob james was working as general manager of itg networks at this time: "in 1991, we came up with the idea of a standard operating environment (soe) which was a concept we borrowed from at&t1' who used it for software specification. we took it much further in applying it to hardware as well, and by producing a product guide." the product guide21 specified which software, hardware, networking and telecommunications products were to be used within telstra right down to the particular model number of a printer or version number of operating system software. documents which broadly defined the soe for the desktop, data networks and voice networks were also developed at this time. telstra soes specify the general technical environment for a broad base of users across all areas of the organisation. soes are the actionable aspects of telstra's overall systems architecture. the soes also address the more specific development and operating environments which are required by specialised areas such as development teams and network specialists. soe content is dynamic, and changes according to business needs and/or ongoing technological advancements. soe categories include: desktop specifies the parameters for all desktop-related hardware and software network applies to the purchasing and installation of all lan and wan network products, and to development of applications in the corporate network environment processing addresses the operational environment for core applications that support the main business functions within telstra at&t is a major american telco. there are currently 24 hardware and software suppliers listed in the product guide, arranged in 4 categories: desktop, networking, enterprise and mid-range solutions (mis (australia). winter 1994. p. 100). 32 6 35 jo <»j 2 o u z j 9 cd co co ajis vol. 4 no. 2 may 1997 "we had a family of concepts and supporting documents", said james "and each of these was a separate standardisation exercise. having set the path for change, the next task was convincing people that they needed to move to soe. we chose the desktop as the first target for introduction ofsoe; it was people-intensive, and it was also an area where we reckoned the businesses could make significant savings in the purchase of hardware and software. we hoped this would provide sufficient incentive to adopt the soe." desktop soe: standardisation of the desktop started in 1990. "by mid-1990, it became obvious that pcs and the windows environment had come of age", said bennett. "at the time telstra had around 20,000 pcs and 2,000 macintoshs, which is roughly the user ratio worldwide. mac users were ferociously loyal, but pcs were already connecting to mainframes systems via drift22, and there were connectivity difficulties with the mac. as integration of applications was the central gameplan, choosing a microcomputer2* standard was really relatively simple." "we established centralised contracts with absolutely world beating prices for desktop products", said james. "we used to buy hardware and software on a per pc or per software package basis. during the late 1980s, we changed that buying model completely by going out to tender for 10,000 pcs at a time. the net result was that we ended up getting at machines for the price of an xt. for the desktop soe, we further improved the buying model by setting up approximate quantity contracts. this meant the business units could deal directly with the end vendor and still get the contract price. we also managed to reduce the supply time down from many weeks to several days. we currently have three vendors who compete for supply of our hardware." these arrangements provided a big incentive for the businesses to move to soe-compliant hardware products. "the soe specifies minimum standards and the most economic approach for the purchasing of new equipment", said bennett "and in 1994/95, '486 machines were the standard. however back in 1991, the original soe for desktop machines was for '386s, and the majority of telstra's machines are still '386s. these are upgraded to '486 level on a business needs basis." telstra also used the leverage of its buying power when it came to purchasing software for the desktop. "in 1991, we did the world's largest deal for windows products", recalled james. "the initial microsoft agreement was a multi-million dollar contract, which covered the operating system, word processing, spreadsheet, graphics and project management application packages." in 1992/1993, telstra established a corporate level agreement with microsoft for the supply of software. "at the time, you could only buy software a box at a time", said james, "and a quantity contract usually just meant a discount per box." the new agreement with microsoft moved away from box licences to user licences, and then progressed to blanket fee corporate licences. this enabled telstra to greatly reduce the annual cost of software from several thousand dollars per user to several hundred. further cost and time reductions have been made by confining application installations to telstra's 1,300 lan file servers rather than individual machines. "it means we only need one copy of each application on the file server," said bennett who became national manager for telstra desktop solutions in 1992. "manageability is a big issue; it's a lot more efficient to upgrade applications at the file server level than locally upgrading over 40,000 hard disks." under telstra's software arrangements, soe-compliant products are effectively free to users. this has provided many of the businesses with a big incentive to adopt the new standards. "we could never force people to use the microsoft products", said bennett "but when the choice was spending $600 for a single copy of wordperfect and facing a lot of interworking problems, or seamless integration with word for windows that was ostensibly free, there was really no contest." databases have not been included in the desktop soe specification as telstra does not believe pcs are an appropriate place to store corporate-level data of this nature. "core database systems such as oracle and db2 run on the mainframes and provide shared access to key business information", said bennett. "all pcs have mainframe access. if people want to compile local databases they can use excel as aflat file to do this." electronic mail is another component of telstra's desktop soe, and was one of the key drivers. the establishment of corporate-wide email would enable staff to send and exchange messages, documents and files at an inter-business level, and encourage greater communication within the organisation. "up until the early 1990s, there was no corporate email at telstra", said bennett. "we had 9,000 email users on 12 different systems. these were a legacy from the days when individual business groups 22 telstra has subsequently developed a drift for windows product. 23 telstra still has around 1,000 macintosh machines. these are not viewed as being part of telstra's new it infrastructure. 34 ajis vol. 4 no. 2 may 1997 made their own decisions on it needs. we developed a service called officelink which used the mainframe-based softswitch product to integrate all the disparate mail systems then in use. we were able to use this as a migration tool for moving towards a single email system based on microsoft's ms mail." migration to the new mail system started in june 1991. by june 1994, telstra was the largest microsoft mail site in the world with 37,500 users across australia, and end-to-end delivery anywhere on its corporate network within 10 minutes. bennett believes implementation of the desktop soe has been much more rapid than originally planned. "we thought it would take four years to convert our disparate users to the soe standards, but three years later the vast majority of our users are now following the standards. while the corporate licence with microsoft has undoubtedly provided some of the impetus for this, the main reason seems to be that people are realising the benefits of standardisation and integration much earlier than we expected. implementation of our corporate email system also greatly assisted in communicating the benefits to be had from a desktop soe." provision of a common infrastructure for the desktop has greatly reduced previous software and hardware incompatibilities and all associated costs, simplified purchasing arrangements, cut training and servicing costs, and markedly improved communications across the firm. "it's completely changed the way we work", said bennett. "the business benefits are not just cost-related. the frustration levels are down, and people enjoy using the gui computing approach. it's easy to learn, and doesn't require you to be a technocrat to obtain results." corporate data network (cdn) soe: in 1988, telecom had trialed the use of fibre optic cable in the melbourne cbd area. "that project interconnected lan file servers in a few major buildings using either baseband or broadband backbones, and x.25 as the communications protocol over the austpac network", said bennett. "// formed the basis for corpnet, telecom's first wide area network." corpnet was a bridged network and principally used by the corporate and government24 business unit. the network grew rapidly, and 2mb/sec high speed links were established between melbourne and other australian capital cities. other business units established networks along the same lines, and by the end of 1991, telstra had thirteen large and autonomous wans in operation. corpnet was the largest of these networks. "corpnet was an early effort at lan interconnect", said john mitchell, telstra's manager for network rationalisation "and was partly used as a driver to progress to bigger and better things." corpnet managed data traffic around australia, but suffered from poor response times, was often unreliable, and was expensive to run. "bridging was fine", said mitchell "but we didn't want to be sending large files over austpac. we also wanted something more scalable, so we started to architect a network that would sit alongside corpnet and perform different functions." telstra's information technology group (itg) had been given the task of integrating the company's internal communications environment. itg realised that significant benefits could be derived both in operational efficiency and reduction in telecommunications costs, by rationalising each of telstra's many separate networks onto a common, corporate data network (cdn) platform. its vision was to implement a reliable, cost-effective, multi-protocol, high speed network that would be accessible to all of its business units, and capable of carrying high volume data traffic. the new network would allow connectivity from any lan within telstra to any corporate computing resource. connectivity was seen as a necessary precondition for any move to integrated systems. a network soe was developed for the cdn based around ibm mainframes and an sna architecture. the new network would be router-based rather than bridged, and would use a fibre distributed data interface (fddi)25 as its underlying transport medium. the cdn would initially comprise two elements: cdn(s) (via x.25 protocols) a public packet switched network service for low and medium volume traffic, based on austpac cdn(b) (via tcp/ip protocols) a broadband, non-switched, router-based backbone network used for high volume and time-critical data transfer this choice of network services would enable application developers to choose the appropriate network service based on business needs, economic and performance criteria. it also satisfied osa requirements (refer figure 2) which specified that all computing equipment must be capable of connecting to the network. a short technical note on the two elements of telstra's cdn is provided in figure 4. now known as corporate, international and enterprises (refer also figure 1). fddi is capable of carrying high data volumes, and allows transmission of many disparate network protocols. 35 ajis vol. 4 no. 2 may 1997 construction of cdn(b) commenced in july 1991, and underwent a phased implementation. cdn(b) has since expanded to become one of the largest, most effective and efficient enterprise-wide networks. thousands of users access corporate computing resources and other lans from more than 46,000 pcs and terminals, around 500 unix workstations, and 1,200 servers (see also figure 3). figure 4: telstra's network soe cdn(s) the switched element of the cdn is based on austpac using x.25 protocols to provide a range of services to the client lan environment. services include terminal access to corporate systems, small file transfer capabilities, and file sharing on remote lans. cdn(s) is reliable, low cost and flexible, has good response times, and is distance independent. cdn(b) the cdn(b) is designed to meet telstra's need for high volume, high-speed data communications, and large file transfer capabilities. lans in each state are linked through high-speed router nodes that comprise a web of data links across australia. the backbone extends to the two corporate data processing centres, proving high-bandwidth connection to telstra's ibm mainframebased systems. any workstation on a lan connected to the backbone can access any other lan on the network as well as any connected mainframe system. the backbone network supports a wide range of standards-based interfaces for different media, host connections and protocols. network reliability on the cdn is ensured by diverse routing between cities. in 1994/95, sites were linked by high speed communications lines capable of supporting speeds from 2mbps to 34mbps. proprietary server without native x.25 or tcp/ip support x.25 & tcp/ip gateways server with native x.25 and tcp/ip support non native x.25 environments non native tcp/ip environments native x.25 entity native tcp/ip entity 26 telstra is responsible for approximately 50% of all traffic carried by the austpac network. 36 ajis vol. 4 no. 2 may 1997 telstra's cdn offers a fully end-to-end managed service, is highly scalable, and is designed to allow the company to grow the backbone network as internal traffic dictates (telecommunications 1993). the network spans the australian continent, and provides a wide range of applications essential to competitive customer service. data carried by the cdn includes high volumes of electronic mail and transactional traffic, file transfer, and remote access to mainframe-based systems via drift. examples include core customer facing systems used primarily by the commercial and consumer division such as flexcab (customer billing) and axis (service delivery), and internally focused systems such as famad (financial accounting), raps (payroll), perkins (personnel), and fleetmis (fleet management). the network also carries high bandwidth applications involving newer technologies such as imaging and video. in 1993, a gateway was established giving telstra access to the global resources of the internet. by mid-1994, the firm had a well developed site on the internet's world wide web. telstra estimates the new network has saved the company approximately us$35 million in annual running costs (telecommunications 1993). in addition, the adoption of network standards has allowed the flexibility to interconnect best-of-breed technologies to all technical platforms as required. cdn(b) continues to grow and absorb existing non-standard and autonomous wans. in 1996, the two separate elements of the cdn were merged into one fully integrated network. conclusion the introduction of a standardised operating environment has provided telstra for the first time with a consistent and effective, corporate-wide approach to it. the soe is still evolving. with more than 46,000 pcs and terminals in use, the introduction of computing standards has produced significant cost savings, and a major increase in telstra's business effectiveness and skill levels. benefits have included reduced training costs, greater buying power, simplified information sharing, and smoother network integration. "what we've ended up with," said lovelock "is an amazing corporate asset. we have the most standard corporate desktop environment in the world in terms of user numbers, probably the third or fourth largest electronic mail network in the world, and a very, very large network taking over from the twenty or thirty competing wide area networks that had built up over the years." telstra views its technological infrastructure as strategically important and providing a foundation for the future growth of the company. "this story in many ways parallels the story of telstra itself, mused james. "we've moved very quickly from being a state-based, somewhat insular organisation to being a nationally run firm. we need the infrastructure we have to continue to run the company the way we do. i can send email to anyone, anywhere in the company now, quickly, easily and reliably. i can sit at my pc and watch our ceo deliver a speech from another location. with drift, i can transparently view up to nine sessions simultaneously from different mainframes all on the one screen. our previous infrastructure only allowed you to run the company the way it was wired." of necessity, telstra's move to an standard operating environment has involved a phased approach which has taken place over a six year period. the sheer size of the firm, and the complexity and diversity of its installed it base, ensured that this project would never be rapid in its implementation. telstra's positive experiences with the development and deployment of its desktop and network soes highlights some key issues for enhancing project success: • the need for strong top management support to enable and sustain change of this magnitude. this was clearly provided by telstra's ceo, frank blount. • the project was driven by clearly defined, high level key business imperatives. from the outset, the project was viewed as essential for the business, rather than just another it project. • the key business imperatives in turn provided the foundation for the development of a detailed, overall systems architecture (osa) document, which specified how information technology would be deployed within telstra to deliver maximum value to the business and the customer bottom line. this further strengthened linkages between the business and it via integration with business processes. • the osa in turn provided the high level blueprint for specification and implementation of soes governing the desktop and network. each soe was carefully documented and circulated to all business units. in addition, all soe developments were clearly communicated to telstra staff. • the need to provide compelling incentives in order to reduce resistance to change. business units within telstra were offered soe-compliant software and hardware at little or no cost. software and hardware purchased outside desktop soe specifications was not supported. 37 ajis vol. 4 no. 2 may 1997 the establishment of a standardised desktop has greatly reduced previous software and hardware incompatibilities, simplified acquisition arrangements, and significantly reduced purchase and training costs. the rationalisation of telstra's many networks into a common, corporate data network platform has substantially improved operational efficiencies, allowed simplified access to mainframebased applications via a single, consistent windows-based interface, enabled the introduction of corporate-wide email thereby greatly improving communications within the firm, and greatly reduced telecommunication costs incurred by the firm. driven by the firm's key business imperatives and governed by the overall systems architecture, telstra's soes ensure that the firm has a flexible blueprint which is capable of changing according to business needs and/or ongoing technological advancements. references abm (1994). australian business monthly, 14(12), october 1994. p.75. aotc (1992a).australian and overseas telecommunications corporation: annual report 1992. p.6. aotc (1992b). australian and overseas telecommunications corporation: annual report 1992. p.ll. s&p (1993a). standard & poor's industry profiles telecommunications, 23/06/93. p. 10. s&p (1993b). standard & poor's industry profiles: telecommunications, 23/06/93. p. 19. telecommunications (1993). telecommunications, october 1993. p.89. 38 what happens when designers don't play by the rules: towards a model of opportunistic behaviour in design a. khushalani , r. smith1'2 and s. howard1 centre for systems methodologies, department of computer science, swinburne university of technology, hawthorn, victoria 3122, australia; j •'department of information systems, monash university, caulfield, victoria 3145, australia. abstract it is being recognised increasingly in the scientific literature that real world design problem solving is noo hierarchical, being characterised by a strategy, termed 'opportunism', whereby designers discover and/or adapt their problem solving goals and activities, in response to die state of the problem and the environment in which that problem exists. this paper explores the concept of opportunism more formally than has hitherto been attempted. we adopt an approach (which we term "theorybuilder") which is a hybrid of the theory testing and theory building approaches. the theorybuilder approach taken adapts the lancaster soft systems methodology to generate an initial model of opportunistic behaviour, to be refined by subsequent iterative comparison with the existing literature and empirical studies. the utility of the initial model is demonstrated by constructing a list of the major cognitive elements for all activities appearing in this initial model, and then using it to analyse a sample verbal protocol collected from an empirical study observed by the authors. initial results are very promising, supporting the need for more comprehensive examination and refinement of the model possible future research directions are discussed. introduction whilst impressive developments in computer support tools (eg, case and cad) have enabled designers in various fields to generate final products of apparently improved quality, it may be significant that only limited contributions have been made to the provision of support to designers undertaking what might be considered the more innovative, up-stream design tasks (eg, problem decomposition and solution searching). one interpretation might attribute this to the relatively impoverished state of our present understanding of the strategies designers employ when they seek to understand and solve design problems. research in recent years into design problem solving strengthens the conviction that the use of many design methodologies is 'opportunistic', being characterised by frequent discovery and/or adaptation of goals and activities, in response to changing circumstances. such is hardly surprising given the obstacles which exist to rigorous application of top-down design problem solving methodologies (eg, incomplete and vague requirements, real world complexity, the inability of designers to comprehend and track all details, and the preconceived ideas of designers (pamas & clements, 1986)). the current literature, however, provides a confused picture of opportunism. in the present paper, we seek to address this shortcoming through development of a model of opportunistic behaviour. against a background discussion of the notion of design problem solving, and a substantial review of the present literature on opportunistic behaviour, we synthesise a definition of opportunism, which we use subsequently to stimulate development of an initial model of opportunistic behaviour, generated using an adapted form of the lancaster soft system methodology. some initial attempts to use the model to structure the analysis of sample verbal protocols collected from design studies observed by the authors, are reported. finally, we speculate on the ongoing research process required to iteratively refine the model. design problem solving in the context of this paper, design is to be viewed as the process of problem understanding and problem solving with the aim of producing an artefact. to establish the context in which we study opportunistic behaviour we first examine the present literature which addresses, in turn, the notion of a design problem, the processes undertaken by design problem solvers, and the strategies they adopt 13 ajis design problems problems have been conveniently classified on a continuum between the extremes of 'hard' and 'soft' (checkland, 1981). a 'hard' problem is one which can be formulated as a search for an efficient means of achieving a defined end (eg, how should an engineer construct a bridge to convey specified vehicles across a specified gorge?), whereas a 'soft' problem is one in which ends, goals and purposes are themselves problematic (eg, how should the australian government address the problem of unemployed youth on our streets?), and so it is not meaningful to talk about means-ends relationships. this terminology has not generally been used in the design literature, where instead problems are characterised as either well-structured (equivalent to 'hard'), or ill-structured (essentially equivalent to 'soft'). simon (1973) has suggested however that 'the boundary between well-structured and illstructured problems is vague, fluid and not susceptible to formalisation', consistent with checkland's notion of a continuum. a considerable literature now suggests that real-world design problems (eg, organisational design, mechanical engineering design, software design) are difficult to express comprehensively, as many components of the problem will not be visible until some attempt is made at generating solutions (lawson, 1980). further, it is observed (bisseret et al., 1988; carroll et al., 1979; davies, 1989; guindon, 1990a; malhotra et al., 1980; visser, 1991b) that: generation of solutions will lead to emerging implications, causing objectives and priorities to change; designers perceive problems differently and are likely to formulate different solutions to the same design problem; in principle there is an infinite number of solutions to the same design problem; and optimal solutions do not necessarily exist. in short, design problems in the real world generally h'e towards the ill-structured or 'soft' end of the continuum. it seems clear then, that any complete and systematic understanding of design problem solving will need to account for the effects of different problem classes. furthermore, any understanding that can be applied to the real world will need to be grounded in ill-structured domains. the design problem solving process numerous studies present a variety of views of the design problem solving process. lawson (1980) characterises it as never-ending in principle, as a consequence of the observation that typically design problems defy comprehensive description and possess numerous possible solutions. the end of the process is solely a judgement of the designer. alternative views have it that design problem solving is a series of cycles (carroll et al., 1979), iterative stages (malhotra, 1980) or plans built upon a kernel idea (ie., designers pursue a single conceptual design very early and adapt it accordingly to accomplish a solution state (guindon, 1990b; kant, 1985; kant & newell, 1984; ullman et al., 1988)). all agree that design problem solving deals, with different emphases, with the following set of issues: problem identification (in the context of a pre-existing unsatisfactory situation); problem definition; problem exploration; analysis (resolving the problem into simpler elements); generation of alternative solutions; evaluation of alternatives; selection of a solution; sequencing of tasks to achieve the selected solution; implementation of the solution; and assessment of whether the chosen solution has corrected/improved the identified problem, which has in turn addressed the pre-existing unsatisfactory situation (eunson, 1988; huber, 1980; visser, 1990c). in summary, the design problem solving process, whether viewed as cycles, stages or plans built around a kernel idea, is dynamic and evolutionary, requiring continuous adaptation by designers to deal with a broad spectrum of issues. design strategies a major topic of present debate concerns both the strategy (ie., the blueprint for structuring the taking of decisions by designers during design problem solving, whether it be top-down, bottom-up, depthfirst, breadth-first, simulation, analogy, reuse, ...) employed by designers during the design problem solving process, and how the choice of strategies influences solution development. several design studies (adelson & soloway, 1985; davies, 1991; jeffries et al., 1981; kant & newell, 1984) support the existence of a top-down strategy whereby design problems are progressively decomposed into smaller, manageable sub-problems, however this research was typically conducted under restricted conditions, generally in the software development domain involving novice programmers working on simplified problems, thereby raising questions as to whether the strategies identified could legitimately be mapped to those used by experts working on real-world problems (visser, 199 ib; visser & hoc, 1990a). may 1994 14 a number of studies conducted in a variety of domains over the past few years have questioned such a top-down view of the design problem solving strategy. it is beyond the scope of the present paper to report a complete review of this rapidly growing literature, however, the interested reader might consult: eg, guindon, 1990a, 1990b; hayes-roth, 1980; hayes-roth & hayes-roth, 1978, 1979; nii, 1986; sumiga, 1992; visser, 1990c, 1992a, 1992b. as a result of careful consideration of this literature by the present authors, it is concluded that whilst it does not present a single, coherent definition of the behaviours being observed (generally termed 'opportunism'), the literature tends to characterise designers as employing strategies whereby they might discover new, and/or adapt their previously intended problem solving goals and activities, in response to the state of the problem and the environment in which that problem exists, at a particular point in time (ie., implying event-driven reasoning (sumiga, 1992)). a rationale for the present research the current research has evolved out of concern at the current conflict in the literature regarding design strategy, coupled with a perceived need to give a more formal treatment of opportunism. if indeed opportunistic behaviour is an essential component of the strategies practiced by real world designers, benefits which should accrue from such research include: • provision of a framework against which to analyse some of the possibly ill-founded assumptions routinely expressed concerning design problem solving; • development of a model which reflects the actual process, as opposed to a model which reflects a desired process; and • identification of possible problems with current methodological approaches. a similar sentiment has been expressed in the research agenda proposed by broadbent (1993). a methodology for generating a model of opportunism methodological background in modelling problem solving behaviour there are two related approaches that might conceivably be pursued (smith et al., 1994): theory testing: for example, an extensive literature review could be conducted and a comprehensive model could be pieced together to represent the totality of the behaviour being reported. hypotheses could be generated and tested through experimentation in order to refine and develop the model (see for example the work of norcio, 1982); or theory building: for example, empirical studies of the behaviour of real-world design problem solvers could be conducted and a model developed, over time, to represent the behaviour being observed (see for example the work of shneiderman, 1986). as discussed earlier, the current literature on opportunism provides a confused picture, mitigating against a theory testing approach. on the other hand, empirical studies provide very rich behavioural data but can be very difficult to conduct in unexplored areas, requiring a rudimentary model to guide observation. thus theory building would be difficult without an appropriate framework. in the present research we adopt an approach (which we term "theorybuilder") which is a hybrid of the two, in that it circumvents the above problems by using an adapted form of the lancaster soft systems methodology (ssm) (checkland, 1981; checkland & scholes, 1990) to propose an initial framework which would support both a coherent examination of the literature, and which will simultaneously provide the rudimentary model required to guide observation. a substantial discussion of this approach is available in smith et al. (1994). the theorybuilder approach is particularly suited to our purposes because it supports the development of systems-based models which can be usefully compared with the unstructured 'messes' which are to be found in the real-world (the present state of knowledge of opportunism could be considered one such unstructured mess). further, it uses the language of the human activity system, the notion of purposeful human activity (ie., deliberate, decided, willed action by an individual or group) to build models (one might postulate that opportunistic behaviour can be viewed as a form of purposeful human activity), and uses the models so developed to provide a language which will support coherent debate. theorybuilder should be viewed as a variant of ssm. we find it convenient to introduce this term which distinguishes both the context (ie., behavioural modelling based upon verbal protocol analysis) and additions to ssm technique (ie., use of restricted verb and noun set in the building of cognitive 15 ajis conceptual models (see stage 3 below), and the introduction of specialised comparison techniques (see stage 4 below)). approach it is beyond the scope of the present paper to describe in any depth the procedures and techniques of the theorybuilder approach. readers are referred to the paper by smith et al. (1994), which adapts ideas and modelling techniques from ssm (checkland, 1981; checkland & scholes, 1990; davies & ledington, 1991; wilson, 1990), for a comprehensive description. in terms of the five stage representation of theorybuilder (smith et al., 1994), the present study can be viewed as follows: stage 1 (find out): finding out involves displaying the perceived problematic situation such that a range of possible and hopefully relevant cognitive activity systems can be revealed. a cognitive activity system is similar to the human activity system described by checkland (1980), but describes purposeful human activity at a cognitive level (ie., in terms of thought and consequent behaviour). in the present case, an extensive investigation of existing literature (see the discussion of design strategy above) has been conducted to establish a base of observational material on opportunism in a variety of design domains. stage 2 (specify relevant systems): at this point we considered a range of possible input-to-outputtransformations relevant to an understanding of opportunism (eg, need to change goals and/or activities in response to events => changed goals and/or activities which represent a response to events), the possible perspectives which express the broader purpose of the transformation (eg, such changes can be deemed desirable to both process and product by designers), and environmental factors (eg, the study constraints; the existing problem situation; an established knowledge base spanning both the intended design problem solving goals/activities and the designers' wider collective knowledge and experience). in the present case, one such notional system which seemed relevant to producing insight was selected, and a root definition (a concise, tightly constructed description which makes explicit the input to output transformation, perspective taken, and environment) of the relevant cognitive activity system has been proposed. stage 3 (build cognitive conceptual models): a "cognitive conceptual model" is a systemic account of a cognitive activity system built on the basis of that system's root definition. it takes the form of a structured set of verbs in the imperative mode, these verbs representing the minimum necessary cognitive activities for the system to be the one named in the root definition. thus, a cognitive conceptual model is a sub-set of the ssm type conceptual model, in that the construction of cognitive conceptual models is constrained by restricted use of verbs (eg, assimilate, understand,...) and nouns (eg, trigger, variation, criteria, ...) related to the cognitive activity of designers. on the other hand, ssm type conceptual models can be constructed using an unrestricted set of verbs and nouns. in summary, a cognitive conceptual model is to be viewed as a model of the acquisition and manipulation of knowledge. it is important to note that the model so created does not purport to describe the real world; only to represent activities which must exist for the system to be that named in the root definition. the verbs are ordered and displayed with unking arrows according to logical contingency (by which is meant that a given activity can only commence if activities at the tails of all entering arrows have been completed). this modelling from root definition to cognitive conceptual model is the closest thing to a deterministic technique in the theorybuilder approach, and adopts the methods discussed extensively by authors such as checkland & scholes (1990) and davies & ledington (1991). in the present case, the root definition relevant to an understanding of opportunistic behaviour was mapped to a cognitive conceptual model. stage 4 (compare): the cognitive conceptual model activities have then been overlaid with the real world, as previously established at stage 1 above, complemented by our own observational studies of design problem solving. as discussed by smith et al. (1994), a technique found useful is to create a list of cognitive elements (terms to which meaning must have been ascribed by any actors undertaking the activities in the cognitive conceptual model for that activity to take place). for example, if a cognitive conceptual model has an activity concerned with "identifying and evaluating a possibility to vary from intended actions in response to events which act as triggers", one might identify as cognitive elements: "identified trigger", "evaluated trigger", "possibility of variation",... (see smith et al. (1994) for further discussion). having identified the cognitive elements for a given cognitive conceptual model, one can probe both the existing literature, and one's own observational studies of design problem solving, for reference to such categories. the existence of references to such cognitive elements is prima facie evidence of the existence of the associated cognitive activity. may 1994 16 tabular and graphical methods designed to display the outcome of such a comparison of the cognitive conceptual model with both existing literature, and with verbal protocols collected during observational studies, have been reported by smith et al. (1994). in the present study, we have established a list of cognitive elements for the cognitive conceptual model developed to understand opportunistic behaviour, and we have adopted a tabular format (see table 2 later) to display the comparison of observed behaviours with the model. such comparison has been pursued on two bases: can evidence be found in the literature for model activities? and, can empirical evidence, drawn from our own observational studies, be found for model activities? also noted, was evidence of real world activities which could not be accommodated in the model. stage 5 (refine the model): in order to establish a comprehensive and fully validated model of opportunistic behaviour, there is a need to develop the initial model to accommodate the totality of the literature and empirical material. this can be conceptualised as shown in figure 1, as an iterative process of comparison and modification of the model to generate a closer match of the refined model to the real world. tentative definition of opportunism empirical stuies what is actually happening? generation of initial model literature what is being reported? comparison of the model t modification of the model refined, validated model of opportunism iterate figure 1. model development process model of opportunism naming of a relevant system the making of opportunistic variations to intended design problem solving goals and activities can be viewed as a cognitive activity system, and using the root definition-cognitive conceptual model mechanisms outlined above, its minimum necessary activities can be identified. the following root definition, structured in the style recommended by smith et al. (1994), based upon that of checkland (1981), provides the foundation for modelling: 17 ajis a system to make opportunistic variations to intended design problem solving goals and activities: a designers' owned and operated system to make opportunistic variations to intended design problem solving goals and activities such that these variations could be deemed desirable both to process and product by the designers, in an environment characterised by the constraints of the study itself, the problem situation, and an established knowledge base spanning both the intended design problem solving goals and activities, and designers' wider collective knowledge and experience. * opportunism is to be interpreted, in accord with the discussion of the existing literature above, as: a strategy whereby design problem solvers discover and/or adapt their problem solving goals and activities, in response to the state of the problem and the environment in which that problem exists (ie., implying event-driven reasoning). it should be noted that the discovery and/or adaptation by designers often results in a changed ordering of activities, omission of activities and/or inclusion of new activities in the previously intended path. human activity system based model of opportunism the minimum necessary activities demanded by the above root definition are shown in the cognitive conceptual model of figure 2. this model suggests that designers, in response to an identified possibility to vary from intended design problem solving goals and activities, will identify a set of possible variations, define the criteria for determining the desirability of the identified variations, evaluate according to those criteria and finally proceed to interface the variations deemed desirable and apply them. of the fifteen activities illustrated, activities 5 to 9 are amenable to relatively straightforward expansion to a higher level of detail, through a process of further root definition-cognitive conceptual model mapping. these elaborations of the model are displayed in figures 3 to 7. as a means of further clarifying the intent of each activity, we present, in appendix a, an interpretation in terms of this model of a scenario involving opportunistic behaviour presented by bimbaum (1985). the cognitive conceptual model of figure 3 suggests that designers, on perceiving a trigger or triggers, evaluate those triggers based on certain factors drawn from their individual or collective knowledge and experience (eg, a difficulty/simplicity perception strategy) thereby recognising the possibility for a variation(s) suggested by the perceived trigger(s). the term 'trigger' is here taken to mean an essential act, word, or symbol which unlocks a pattern in the receiver, thereby resulting in a flow of ideas. the cognitive conceptual model of figure 4 suggests that after designers have identified the possibility for variation(s) they attempt to articulate the possibility as appropriate variation(s). the notion of possible variation space introduced here is taken to mean the set of all the variations of which the designers could conceive, and its use here is intended to convey an interest by designers in exploring variation options rather than taking a single variation as granted. the cognitive conceptual model presented in figure 5 proposes that designers define criteria for determining the desirability of variations through a synthesis of criteria which reflect the ease with which one can conceptualise the variations, implement the variations, and predict the product benefits to be gained through application of the variations. the cognitive conceptual model presented in figure 6 proposes that designers, based on the desirability criteria developed in figure 5, determine a list of variations deemed desirable. to apply those criteria, the designers must conceptualise the variations, determine the processes involved in implementing the variations, and determine the effect on product outcome of the variations. those variations deemed desirable are then time ordered (based upon issues such as priority and contingency) and submitted to be actioned. finally, the cognitive conceptual model in figure 7 suggests that designers, after determining the extent to which variations affect the previously intended design problem solving goals and activities, interface those variations to the previously intended goals and activities if necessary. variations are then applied, together with those aspects of the previously intended design problem solving goals and activities which survive. we emphasise again that the model in figures 2 to 7 is an ideal type creation which expresses, in accord with the tenets of theorybuilder and ssm, those activities which must exist for the system to achieve the purpose expressed in the root definition. the model provides a framework for subsequent theory testing and building only, and prior to that refinement process has no status beyond providing a may 1994 18 language in terms of which one can begin to probe and express a growing understanding of the existing literature and observational studies. establishment of cognitive elements using the approach of smith et al. (1994), an initial list of the object-related cognitive elements (noun, noun phrases and adjectival noun phrases) and action-related cognitive elements (verbs and verb plus adverbial phrases) used in each cognitive conceptual model was created. this initial list was then rationalised, with an aim of removing duplicates and synonyms. the list of nouns and verbs remaining constituted the set of cognitive elements (terms to which meaning must have been ascribed by any actors undertaking the activities in the cognitive conceptual model for these activities to take place). a list of the major cognitive elements for the activities appearing in figures 3 to 7, so developed is presented in table 1, and has been used, as discussed below, as a means of comparing the model with our observational studies of problem solving behaviour. cognitive elements lessen the burden of conducting the comparison with protocol data by lending themselves easily to identification of areas of the protocol transcription which are potentially associated with model activities, by identification of utterances which reflect that the designers are attempting to deal with concepts which could be mapped to cognitive elements. define measures of performance: efficacy an efficiency 13 define measures of performance: effectiveness figure 2. a system to make opportunistic variations to intended design problem solving goals and activities 19 ajis from 1-4,6,8 and 9 from 2 i 5.1 perceive triggers) 5.3 recognise the possibilty for variation(s) figure 3. activity 5 identify possibility for variation(s) from 5 from 1-4 explore possible variation space 6.2 articulate variation(s) to 5 to 8 figure 4. activity 6 identify possible variation(s) which might be brought to bear on the intended design problem solving goals and activities may 1994 20 from 1-4 define criteria relevant to the judgement: ability of variations) to yield an improved product define criteria relevant to the judgement: ability to conceptualise the variations) define criteria relevant to the judgement: ability to undertake/complete the variation(s) determine relative weightings to be given to criteria components: conceptualisation implementation improved product express set of criteria relevant to the present situation to 8 figure 5. activity 7 define criteria for variation(s) to be desirable towards refinement of the model of opportunism as discussed above, the initial model reported must now be subjected to a process of iterative comparison with the existing literature and with empirical data, and must be modified to better reflect opportunistic behaviour in the real world. it is beyond the capacity of the present paper to describe the totality of this work. to indicate the nature of research in progress however, we present below an application of the model to the interpretation of a sample verbal protocol. an empirical study using simultaneous verbalisation during a period of seven weeks, comprehensive observation of a three member team involved in developing a strategic information system plan for the office of a university department head was undertaken, with a view to providing a basis for improved support of office functions. the stated intended activity sequence of the design team was to pursue the preparation of a strategic information system plan in accord with the seven stage process of ssm (checkland, 1981), adopting a comparison phase which utilised the maltese cross technique of wilson (1990). the three designers were asked to verbalise their thoughts during their design problem solving activity, without intervention (ie., concurrent verbal protocol as per ericsson & simon, 1980; also used by guindon et al., 1987; guindon & curtis, 1988; visser, 1987, 1988, 1990b, 1990c, 1991a, 1991c, 1992a). 21 ajis from 6 1 from 7 8.1 conceptualise the varialion(s) 8.4 define measures of performance: efficacy an efficiency determine processes involved is/in implementing the variation(s) determine the effect on product outcome of the variations) consolidate to list of vanation(s) deemed desirable understand issues which might affect the time ordering express the time ordered sequence of possible/desirable varialion(s) figure 6. activity 8 determine which variation(s), if any, are desirable and time order from 8 determine the extent to which the variation(s) affects the intended design problem solving goals and activities 9.3 apply variation(s) interface vanation(s) to the intended design problem solving goals andd activities apply the remaining aspects of the intended design problem solving goats and activities from 1-4 figure 7. activity 9 apply intended design problem solving goals and activities with opportunistic variation(s) may 1994 22 table 1. cognitive elements (a system to make opportunistic variations to intended design problem solving goals and activities) activity no 5.1 5.2 5.3 6.1 6.2 7.1 7.2 7.3 7.4 7.5 8.1 8.2 8.3 8.4 8.5 8.6 8.7 9.1 9.2 9.3 9.4 elements a perceived trigger an evaluation of a trigger an acknowledgment of the possibility for variation an appreciation of the scope of possible variation(s) list of possible variations set of criteria for assessing the extent to which a variation can be conceptualised set of criteria for assessing the extent to which a variation can be implemented set of criteria for assessing the extent to which a variation can yield an improved product a position on the relative importance of conceptualisation, implementation and product improvement in deciding whether to proceed with a variation list of criteria relevant to assessing the desirability of variations in the present situation a conceptualisation of the nature of a variation list of processes involved in implementing a variation list of effects on product outcome of a variation a determination, for each possible variation, of its desirability list of variations deemed desirable an understanding of issues which might affect the time ordering of variations time-ordered list of variations deemed desirable an understanding of the extent to which variations affect the intended design problem solving goals and activities an interface of variations to the intended design problem solving goals and activities variations which have been implemented a resumption of the remaining aspects of the intended design problem solving goals and activities the designers the three designers were enrolled in a master's level course in information technology. they were rated highly competent, as assessed by academic record prior to joining the course, and subject performance throughout the course. designer di had more than ten years professional experience in information systems analysis along with a degree in electrical engineering. designer d2 had worked as a lecturer in information a. systems with the university for more than ten years. designer d3 had nearly ten years expertise in software development with a government organisation. data collection concurrent verbal protocols were collected by videotaping designers during their design problem solving activities. all documents (such as interview notes with clients) and diagrams (such as rich picture and conceptual models) produced by designers were collected, and incorporated into the transcribed verbal protocols. procedure protocol analysis the procedure adopted to analyse the recorded protocols was as follows: a. the intended design problem solving goals and activities of the team (strategic information systems planning according to the seven stage process of ssm) were matched to utterances in the transcription; b. potential shifts in design problem solving goals and activities were highlighted by identification of utterances in the transcribed verbal protocols which reflected that the team was investigating concepts which could be mapped to the cognitive elements listed in table 1; 23 ajis c. the transcription in the vicinity of each potential shift so identified was examined in detail, in the light of the intentions determined at step (a) above, to identify whether it constituted a variation from previously intended design problem solving goals and activities; d. the sources giving rise to identified variations, and the subsequent activities as they evolved, were matched to activities 5 to 9 of figures 3 to 7; and e. activities not compatible with figures 3 to 7 were noted for subsequent consideration. data analysis • validation one excerpt from the empirical study is presented in appendix b, and its interpretation is discussed below, to give some indication of both the utility of the model, and the iterative refinement process which has commenced. table 2 presents a comparison of the protocol (from appendix b) with the model in a simple tabular form, analogous to the 'formal questioning' technique of checkland & scholes (1990). as shown, a table of model defined questions was used, probing the extent to which the activities in the model (ie., 5.1,5.2,5.3,... 9.4) were carried out in the situation under study (ie., in this case the protocol excerpt illustrated in appendix b). for example, activity 5.1 ("perceive trigger(s)"), with associated cognitive element "a perceived trigger", can be mapped in the protocol excerpt (appendix b) to designer di noticing service provision problems being faced by part time evening students. activity 5.2 ("evaluate triggers)"), with associated cognitive element "an evaluation of a trigger", can be mapped to designer dl's elaborate description of elements of the noticed problematic service. similarly, activities 5.3 to 9.4 were mapped, giving rise to the interpretation illustrated in table 2. more recent work (smith et al., 1994) discusses alternate graphical techniques for displaying this comparison, which are closer to checkland's notion of 'model overlay1 (checkland & scholes, 1990). discussion the excerpt analysed in table 2 is but one, selected as representative of the extensive analysis of collected protocols which is in progress. some tentative conclusions, drawn from analysis to date, include: • there is no explicit evidence of designers exploring more than one variation option at any instant. the status of activity 6.1, and the subsequent activities involved with sequencing variations (activities 8.5, 8.6 and 8.7) is therefore in question. designers in the studies analysed to date have responded to a triggering event by exploring a single possible variation in depth, to the point where it has been either accepted and actioned, or rejected. evidence from the design literature tends to suggest that such a depth-first strategy is characteristic of novice designers, whilst experts tend to adopt a breadth-first strategy. the studies we have analysed have involved designers of substantial experience in information systems analysis and design, although they have had only limited experience in the chosen problem solving methodology (ssm). further studies, involving more experienced ssm practitioners, are planned to establish whether the above behaviour is generalisable or is simply an artefact of the limited specific methodological experience base of the subjects; • there is only weak evidence of designers determining explicitly their criteria for assessing whether a variation of goal or activity will be desirable, and there is no explicit evidence that designers weight the various criteria, prior to commencing the variation (activities 7.1-5). more often such criteria and weightings as we have been able to recognise have been identified by reference to the talk-aloud protocol collected as designers implement their proposed variation. this finding is consistent with the studies of hayes-roth (1980), which concluded that criteria used by designers when taking decisions are often not stated explicitly, but instead are implicit, and become apparent only as designers undertake activities which follow from the taking of those decisions. one possible implication of such a finding would be that environments which support the up-stream phases of design might encourage the explicit expression of criteria, and weights given by designers to those criteria, prior to the taking of decisions to vary design goals and/or activities; may 1994 24 • there is little evidence of designers considering explicitly the extent to which a possible variation will affect their previously intended goals and activities (activities 9.1-2). such 'interfacing' of variations to the previous intentions of the designers seems to be handled implicitly within the subsequent processes of applying the variation and then resuming problem solving according to previously intended goals and activities. if confirmed, this finding has implications for the provision of tool support to the upstream phases of design. specifically, one could envision support which enables designers to explore the effects of deviating from a current intended path (eg, by simulating the impact of a variation scenario). consolidation of these conclusions, and the modification of the model presented in figures 2 to 7 which must consequently be made, awaits completion of a program of extensive literature comparison and empirical study which is in progress. table 2. model based analysis of opportunistic behaviour (see excerpt in appendix b) activity 5.1 -perceive trigger(s) 5.2 evaluate trigger(s) 5.3 recognise the possibility for variation(s) 6.1 explore possible variation space 6.2 articulate variation(s) 7.1 define criteria: ability to conceptualise the variation(s) 7.2 define criteria: ability to undertake and complete the variation(s) 7.3 define criteria: ability of variation(s) to yield an improved product 7.4 determine relative weightings 7.5 express set of criteria relevant to the present situation 8.1 conceptualise the variation(s) observation di noticed circumstances surrounding service provision to part time evening students. di described the noticed problem in detail, elaborating elements of the problematic service to part time students. di, based on the above evaluation, recognised the possibility to incorporate handling of the noticed problem as a recommendation in the study. implicit in the subsequent dialogue is the potential for the team to attempt to formulate a recommendation now (an ad hoc recommendation), rather than allow recommendations to emerge as a matter of course later in the study as an outcome of ssm based analysis. only one possible variation (ie., formulate an ad hoc recommendation) was explored. the possibility of formulating an ad hoc recommendation was articulated only implicitly, as di and d3 began together to explore a possible recommendation (that each staff member of the office should answer student queries on a rotational basis throughout the week). this excerpt offers no support for this activity. the designers had no apparent problem in appreciating the notion of formulating an ad hoc recommendation and so the issue of criteria did not arise. we believe that there will be evidence of this activity only when designers are faced with a large cognitively taxing variation. this excerpt offers no support for this activity. the designers had no apparent problem in understanding the mechanism of formulating an ad hoc recommendation and so the issue of criteria did not arise. we believe that there will be evidence of this activity only when designers are faced with a large cognitively taxing variation. explicitly stated in the last sentence of the excerpt are criteria related to product, against which the quality of any recommendation, ad hoc or as a result of detailed analysis, will be assessed (ie., a recommendation must lead to resolution of student queries and productivity improvement). in addition, d3 found the notion of answering queries on a rotational basis to be constrained by the different job functions of each staff member (eg, the secretary could not be expected to answer all queries relating to the computer science department as well as information technology). designer d3 had indirectly defined a further product-based criteria (ie., acceptable routing of student queries). there is no evidence of the designers explicitly or implicitly weighting any of the above various product based criteria more highly than others. there is no explicit statement of criteria prior to discussing possible ad hoc recommendations, however the product-based criteria discussed at activity 7 above were cited by the designers as they attempted to formulate a recommendation. this excerpt offers no support for this activity. the designers had no apparent problem in appreciating the notion of formulating an ad hoc recommendation. 25 ajis 8.2 determine processes involved in implementing the variation(s) the designers entered debate on a possible ad hoc variation with no apparent consideration of how their discussions might proceed. 8.3 determine the effect on product outcome of the variation(s) it was determined that implementing the ad hoc recommendation which was emerging would lead to productivity improvement and resolution of student queries, as evident from the last sentence in the excerpt. the productivity improvement which designer d3 referred to, arises from allocating a day to each staff member on a rotational basis, which effectively ensures that on a day when it is the secretary's turn to answer queries, the other two staff members are going to have an uninterrupted working day. the resolution of student queries can be achieved by altering the roster time of staff members, for example, asking the staff member who's turn it is to answer student queries to start at 9:00 and finish at 6:00 or start at 10:00 and finish at 6:00, as mentioned in the excerpt having applied the product-based criteria, the designers accepted that formulating an ad hoc recommendation and writing it directly into the study outcomes was desirable. 8.4 apply criteria to determine whether the variation(s) is/are desirable 8.5 consolidate to a list of variation(s) deemed desirable only one variation (ad hoc recommendation formulation) was considered. 8.6 understand issues which might affect the time ordering as there was only one variation, time ordering was not an issue. 8.7 express the time ordered sequence of possible/desirable variation(s) as there was only one variation, sequencing was not an issue. 9.1 determine the extent to which the variation(s) affects the intended design problem solving goals and activities there was no explicit consideration of the effect that formulating an ad hoc recommendation to cover this issue might have on the remainder of the study, although implicitly it led to a re-scoping of work (see below). 9.2 interface variation(s) to the intended design problem solving goals and activities an implicit re-scoping of the study did take place, in that there was no further discussion of the issue of handling part-time student enquires in the transcripts of the remaining ssm study, effectively removing the development of ssm conceptual models etc. relevant to this issue from the intended study activities. 9.3 apply variation(s) the designers proceeded to formulate an ad hoc recommendation, without reference to the modelling stages of ssm. the final recommendation read: "organise a roster for an office member to start work at 10:00am and finish at 6:30pm so evening students can access the office. additionally, a system of routing the query to the right person must be instituted". 9.4 apply the remaining aspects of the intended design problem solving goals and activities the designers resumed their rich picture building (stage 2 of ssm), which they had suspended while processing the above variation (ie., formulation of an ad hoc recommendation to cover a specific issue). research reflections theorybuilder (smith et al., 1994) has supported the process of cognitive model building, from initial model conception to model testing against empirical data. theorybuilder circumvented many of the problems germane to theory generation using orthodox psychological techniques, eg, verbal protocol analysis and experimental method, where problems arise in part due to a lack of support for data analysis and lack of a theory from which to generate testable hypotheses, respectively. may 1994 26 in sympathy with theorybuilder's origins in ssm, the process of model development was discretionary, and hence the resultant model reflects only a partial interpretation of the data on opportunism. with respect to the model development process, work is ongoing in exploring opportunism from more than one methodological perspective. theorybuilder is facilitating current research in exploring cognitive conceptual models through processes characterised by the development of multiple transformations, multiple root definitions and distinct cognitive conceptual models. with respect to the model per se, work is ongoing in further decomposing model activities and validating the activities against suitable empirical data. furthermore, some research is focusing on the generalisability of the model to other domains and design tasks and the implications, for design tools, of opportunistic behaviour. summary and outlook in this paper, we have expressed the need to focus attention on the concept of opportunism, which is rapidly gaining recognition in a variety of independent problem domains. numerous studies of opportunism have been reviewed and a tentative definition of the phenomenon has been provided. based upon that definition, and using the modelling principles which underpin the lancaster soft systems methodology, an initial model of opportunistic behaviour has been proposed. the model provides a framework for subsequent theory testing and building only, and prior to that refinement process has no status beyond providing a language in terms of which one can begin to probe and express a growing understanding of the existing literature and observational studies. the utility of some activities of the initial model has been demonstrated through its use to interpret an empirical study conducted by the authors. results are very promising. further research in progress seeks to: • refine the model by additional empirical study; • establish the extent to which this model subsumes models developed by other authors; and • focus on select model activities with a view to generating useful taxonomies of opportunistic behaviours, and investigating the consequences for support of up-stream design problem solving. acknowledgment the first author would like to acknowledge support gained for the above research through an australian postgraduate research award. the second and third authors acknowledge support from the australian research council. references adelson, b. and soloway, e. (1985) "the role of domain experience in software design", ieee transactions on software engineering, vol se-11, no 11, pp 1351-1360. bimbaum, l. (1985) "a short note on opportunistic planning and memory in arguments", in proceedings of the ninth international joint conference on artificial intelligence, vol 1, pp 281-283. bisseret, a., figeac-letang, c. and falzon, p. (1988) "modelling opportunistic reasonings: the cognitive activity of traffic signal setting technicians", research report no 898, rocquencourt: inria. broadbent, d. (1993) "planning and opportunism", the psychologist: bulletin of the british psychological society, issue 6, pp 54-60. carroll, j. m., thomas, j. c. and malhotra, a. (1979) "clinical-experimental analysis of design problem solving", design studies, vol 1, pp 84-92. checkland, p. (1981) systems thinking, systems practice, wiley, chichester. checkland, p. and scholes, j. (1990) soft systems methodology in action, wiley, chichester. davies, l. j. (1989) "designing from ill-defined problems", international journal of information management, vol 9, pp 199-208. davies, l. and ledington, p. (1991) information in action, soft systems methodology, macmillan information systems series. 27 ajis davies, s. p. (1991) "characterising the program design activity: neither strictly top-down nor globally opportunistic", behaviour and information technology, vol 10, pp 173-190. ericsson, k. a. and simon, h. a. (1980) "verbal reports as data", psychological review, issue 87, pp215251. eunson, b. (1988) behaving: managing yourself and others, mcgraw-hill book company, australia. guindon, r. (1990a) "designing the design process: exploiting opportunistic thoughts", humancomputer interaction, vol 5, pp 305-344. guindon, r. (1990b) "knowledge exploited by experts during software system design", international journal of man-machine studies, vol 33, pp 279-304. guindon, r. and curtis, b. (1988) "control of cognitive processes during design: what tools would support software designers ?", in proceedings of cht88, washington, d.c., acm press, addison-wesley publishing, pp 263-268. guindon, r., krasner, h. and curtis, b. (1987) "breakdowns and processes during the early activities of software design by professionals", in olson, g. m., sheppard, c. and soloway, e. (eds.), empirical studies of programmers: second workshop, norwood, nj: ablex, pp 65-82. hayes-roth, b. (1980) "human planning processes", technical report no r-2670-onr, rand corporation, santa monica, california. hayes-roth, b. and hayes-roth, f. (1978) "cognitive processes in planning", technical report no r-2366onr, rand corporation, santa monica, california. hayes-roth, b. and hayes-roth, f. (1979) "a cognitive model of planning", cognitive science, vol 3, pp 275-310. huber, g. p. (1980) managerial decision making, glenview. jeffries, r., turner, a.a., poison, p. and atwood, m. e. (1981) "the processes involved in designing software", incog anderson, j. r. (ed.), cognitive skills and their acquisition, hillsdale, nj.: erlbaum, pp 225-283. kant, e. (1985) "understanding and automating algorithm design", ieee transactions on software engineering, se-ll,pp 1361-1374. kant, e. and newell, a. (1984) "problem solving techniques for the design of algorithms", information processing and management, vol 20, no 1-2, pp 91-118. lawson, b. (1980) how designers think, the architectural press ltd: london, eastview editions, inc: westfield, new jersey. malhotra, a., thomas, j. c., carroll, j. m. and miller, l. a. (1980) "cognitive processes in design", international journal of man-machine studies, vol 12, pp 119-140. nii, h. p. (1986) "blackboard systems: blackboard application systems, blackboard systems from a knowledge engineering perspective", the ai magazine, part 1, summer 1986, pp 38-53, and part 2, august 1986, pp 82-106. norcio, a. f. (1982) "indentation, documentation and programmer comprehension", in proceedings of the conference on human factors in computer systems, gaithersburg, md. parnas, d. l. and clements, p. c. (1986) "a rational design process: how and why to fake it", ieee transactions on software engineering, vol 12, pp 251-257. shneiderman, b. (1986) "empirical studies of programmers: the territory, paths and destinations", in empirical studies of programmers, ablex. simon, h. a. (1973) "the structure of ill-structured problems", artificial intelligence, vol 4, pp 181-201 smith, r., howard, s., sutherland, t., and khushalani, a. (1994), "theorybuilden a behavioural perspective on modelling and improving systems development", in proceedings of the 1st australian seminar on modelling and improving system development (ozmisd'94), melbourne australia, feb 3-4 1994. sumiga, j. h. (1992) "programming and design", technical report no sit-cs-1/92, swinburne university of technology, melbourne, australia. ullman, d. g., dietterich, t. g. and stauffer, l. a. (1988) "a model of the mechanical design process based on empirical data", ai edam, vol 2, pp 23-52. visser, w. (1987) "strategies in programming programmable controllers: a field study on a professional programmer", in olson, g., sheppard, s. and soloway, e. (eds.), empirical studies of programmers: second workshop, norwood, nj: ablex, pp 217-230. may 1994 28 visser, w. (1988) "towards modelling the activity of design: an observational study on a specification stage", in proceedings of the ifac/ifip/iea/ifors conference on manmachine systems, analysis, design and evaluation, oulu, finland, pp 249-254. visser, w. and hoc, j. m. (1990a) "expert software design strategies", in hoc, j. m., green, t., samurcay, r. and gilmore, d. (eds.), psychology of programming, london: academic press, pp 235-249. visser, w. (1990b) "evocation and elaboration of solutions: different types of problem-solving actions. an empirical study on the design of an aerospace artefact", in kohonen, t. and fogelman-soulie, f. (eds.), cognitiva 90, at the crossroads of artificial intelligence, cognitive science and neuroscience, proceedings of the third cognitiva symposium, amsterdam: elsevier, pp 689-696. visser, w. (1990c) "more or less following a plan during design: opportunistic deviations in specification", international journal of man-machine studies, special issue: what programmers know, vol 33, pp 247-278. visser, w. (199la) "designers' activities examined at three levels: organisation, strategies and problem-solving processes", knowledge based systems, vol 5, no 1, pp 92-104. visser, w. (1991b) "design activities through the research in a cognitive psychology team", paper presented at the edcaad-workshop, edinburgh. visser, w. (1991c) "the cognitive psychology viewpoint on design: examples from empirical studies", in gero, j. (ed), artificial intelligence in design, oxford: butterworth-heinemann, pp 505524. visser, w. (1992a) "planning and organisation in expert design activities", (paper submitted for publication in jan 1992). visser, w. (1992b) "design organisation: there is more to expert knowledge than is dreamed of in the planner's philosophy", research report no 1765, rocquencourt: inria. wilson, b. (1990) systems: concepts, methodologies and applications, wiley, chichester. appendix a: model based analysis of literature excerpt the following scenario has been proposed by bimbaum (1985). it is interpreted in table ai in terms of the activities included in the model displayed as figures 3 to 7. "suppose a planner decides to go to the store to buy milk. in the course of executing that plan, while at the store, he notices that eggs are on sale. realising that he will need eggs in the near future, he checks to see whether he has sufficient funds, and if he does, he buys some." table ai. interpretation of literature excerpt activity 5.1 -perceive triggers) 5.2 evaluate triggers) 5.3 recognise the possibility for variation(s) 6.1 explore possible variation space 6.2 articulate variation(s) 7.1 -define criteria: ability to conceptualise the variation(s) 7.2 define criteria: ability to undertake and complete the variation(s) interpretation the planner notices eggs are on sale. the planner responds to having noticed eggs on sale, by realising his need for eggs in the near future. the planner, based on the above evaluation, recognises the possibility to incorporate the purchasing of eggs within his intended plan (which originally was to purchase milk only). only one possible variation (ie., purchase eggs now) is explored. one could speculate that in an expanded scenario, he might explore a number of possibilities (eg, buy eggs at another shop, borrow some from a neighbour on the way home). the possible variation in articulated form is "purchase eggs now". the purchasing of eggs is an easy variation to conceptualise and so the issue of criteria does not arise. the ability to undertake the variation (ie., purchasing eggs now), in the excerpt above is to be judged according to the criterion that the purchase of milk must not be compromised (by the purchase of eggs exhausting his funds). 29 ajis 7.3 define criteria: ability of variation(s) to yield an improved product 7.4 determine relative weightings 7.5 express set of criteria relevant to the present situation 8.1 conceptualise the variation(s) 8.2 determine processes involved in implementing the variation(s) 8.3 determine the effect on product outcome of the variation(s) 8.4 apply criteria to determine whether the variation(s) is/are desirable 8.5 consolidate to a list of variation(s) deemed desirable 8.6 • understand issues which might affect the time ordering 8.7 express the time ordered sequence of possible/desirable variation(s) 9.1 determine the extent to which the variation(s) affects the intended design problem solving goals and activities 9.2 interface variation(s) to the intended design problem solving goals and activities 9.3 apply variation(s) 9.4 apply the remaining aspects of the intended design problem solving goals and activities implicit in the excerpt is the product-based criteria that there should be financial savings (ie., eggs are on sale). it could also be speculated that since the planner has realised the need for eggs in the near future, another product-based criteria might be time saving (ie., benefit would result from avoiding a trip to the store in the near future to purchase eggs). although not explicitly acknowledged, the scenario seems to suggest that the issue of financial savings is strongly weighted in the decision to proceed. there is no explicit statement of criteria. the planner has no apparent problem in appreciating the notion of purchasing eggs now. the planner determines that he must check available funds as a component of purchasing eggs now. the planner determines that purchasing eggs now would lead to financial savings (as eggs are on sale). we could speculate that he also determines that he will save himself an extra trip to the store to purchase eggs in the near future. the planner decides that purchasing eggs now is in accord with his criteria (financial savings, and will still allow him to purchase milk), and determines to purchase eggs now. only one variation (ie., purchase eggs now) is considered. as there is only one variation, time ordering is not an issue. as there is only one variation, sequencing is not an issue. the planner determines that purchasing eggs now will not affect his capacity to also purchase milk. there are no adaptation issues evident in this excerpt. no interfacing is needed. the planner purchases the eggs. the planner purchases the milk as per his intended plan and leaves the store with milk and eggs. may 1994 30 appendix b: protocol excerpt di : another thing that i noticed as the basis of our recommendations. most part timers, evening students specially, come in at 4:30 at the earliest, often 5:00, and they cannot find anyone to relate to or hand in assignments, ask questions, do anything of that sort, especially in the early days when they don't know whether they are coming or going. they don't know who to speak to, where to get things organised and there is nobody to help them at all. so whilst they are the clients of the office of the computer science department they can never use it because whenever they are on the sceneit is closed. d3 : actually, one of our recommendations is interruptions; somebody answers student queries on monday, somebody else on tuesday, somebody else's turn on wednesday. now, i think one of the reasons that precludes that is everyone has a different set of job functions and if it is the secretary's turn to answer student queries for information technology students, she is not going to tap information technology. di : not the change. if she needs to do that, she may do it over the first couple of months until; look because queries are of two different nature, either they are of general nature or a specific nature. if they are of general nature, after a while, even if she is not able to answer them right away in the beginning, after a couple of months she will learn the responses. if they are of specific nature, then there are two ways of handling them. either she liaises with whoever is responsible, acting as a buffer, which means sure enough his time will be taken. he will be interrupted but that is part of his job, but there will at least be a buffer there or if it is not of an urgent nature, she can tell the student or whoever to come back at a particular time when he is on the job. either of the ways, i mean that's for them to decide. the recommendation however should be to introduce something like that because it will improve productivity tenfold, and the person who is on duty on a given day, rather than starting at 8:00, starts at 9:00 but finishes at 6:00, or starts at 10:00 and stays till 6:00. 31 ajis microsoft word 599-965-2-ce[1].doc australasian journal of information systems volume 16 number 1 2009 147 the governance of information and communication technology* john weckert centre for applied philosophy and public ethics charles sturt university richard lucas centre for applied philosophy and public ethics australian national university information and communication technologies (ict) are ubiquitous but governance in the industry is piecemeal at best. government regulations and various standards apply to parts of the industry, and professional bodies, the australian computer society (acs) for example, have some influence on individuals, on professional education and on policies. however, unlike medicine, law, accountancy, or plumbing, there are no regulations concerning who can be employed in ict. while the acs and similar professional bodies in other countries can and do have an effect on the industry, given that there is no accreditation or licensing scheme in place for those in ict, their role is not the same as similar professional organisations in many other professions. the acs can discipline its members for breaches of its ethical code but that is no barrier to employment and it has no effect on the vast majority of workers in the industry who are not acs members. it does not follow from this that these professional bodies have no important role to play in the governance of the industry, but it is less formal than in many other professions. as already mentioned, they have input to professional education, to government policy-making, and to industry standards, for software development for example. additionally they provide codes of ethics and conduct for those in the industry, as well as other information through their publications. these and other activities are designed to improve professionalism in ict. what this suggests is that much of the governance in ict is informal. activities and strategies to improve professionalism, and ethical behaviour in general, are therefore of utmost importance. the three papers in this section all contribute to the discussion of aiding professionalism and ethical conduct. in the first of these papers “what is an ict professional anyway?” clive boughton examines the concepts of a professional and of a profession; concepts easy to use but difficult to define. he takes the original approach of describing various types of people working in the industry, based on his experience, and then considers what it means to be a professional in general. he concludes with a number of suggestions regarding what needs to be done in order for the ict “profession” to become truly professional. in the second paper don gotterbarn addresses the role of codes of ethics in a profession in detail in “ict governance and what to do about the toothless tiger(s): professional organizations and codes of ethics”, as part of his examination of ict governance more generally. his argument is that there is a middle way between a strict regulation approach to governance on the one hand and the relative powerlessness of ict professional organisations, what he calls “the vague toothless tiger” approach, on the other. according to gotterbarn, professional bodies and codes of ethics can have teeth. australasian journal of information systems volume 16 number 1 2009 148 in the final paper in this section, “the decision disconnect”, cecilia ridgley points out that there are various relationships between ethics and governance and, in particular, explores “the centrality of value systems in the practice of governance in the enterprise”. she argues that there is commonly a disconnect between those governing and the governed. ridgley develops, explains, and argues for a strategy to overcome this disconnect. these three papers will, we hope, promote further discussion of some of the important topics within the governance of ict. the conference on which this special issue is based was part of a research project supported under australian research council's linkage projects funding scheme (project number lp0560659), with the australian computer society as the industry partner. 45 a pilot study of small business's perception of vendor provided services: are these associated with small business it educational requirements? r.c. macgregor' d.j. bunker1 j.k. pierson2 k.a. forcht2 'department of business systems university of wollongong northfields avenue, wollongong, australia 2522 2department of information and decision sciences james madison university harrisonburg virginia 22807 usa abstract despite the extensive use of microcomputers in the small business environment, little research has been applied to the training needs of those small businesses. this study examines the computer related training needs of 131 small businesses and the factors which appear to impinge upon those needs. in particular the paper examines whether satisfaction with vendor provided services (both pre and post sales) are associated specific educational requirements in small business. the results suggest that there a need for the development of a university/college it training program directed specifically towards small business. these programs should avoid the more technical aspects of it and concentrate on areas which involve the analysis and use of software in the small business. the results further show that a number of vendor provided services appear to be associated with the rating of importance of potential curricular inclusions in it training programs. introduction since the microcomputer became readily available in the early 1980's, more and more small firms have adopted them in their day-to-day operations. initially, most small businesses acquired computer technology for the purposes of accounting (baker 1987, heikkila, saarinan & saaksjarvi 1991). more recently, computer technology has been acquired for a variety of uses (raymond & pare 1992, raymond 1990, raymond & bergeron 1992). with increase in sophistication of information systems used in small business, there has been a growing need to provide adequate training for personnel (cragg & zinatelli 1995). originally training was minimal, usually being provided by the vendor from whom the technology was purchased. today, however, more and more training is provided by service organisations and, in some cases, universities and colleges. a number of reasons have been suggested. raymond et al (1993) suggest that the use of technology by small business tends to affect the structure of the business, often resulting in a more complex organisation, with the increase of technology use. chan & huff (1993) suggest that technology allows the organisation to focus on strategic decisions rather than simply maintaining the operational status quo. with this change of emphasis, there is a requirement to examine training curricula which addressed the needs of the small business client. when considering curriculum design, nelson (1991) suggests that a curriculum which merely addresses the technical issues only provides half the requisite knowledge and skills necessary to adequately adopt and maintain information technology in an organisation. this is supported in the literature by the many studies which point to the problem incurred when organisational issues are not adequately considered at the inception of technology (turner & karasek 1984, macgregor & clarke 1988, raymond 1988, sharp & lewis 1992, kahn & robertson 1992, williams 1992, bergeron et al 1992, hedberg & harper 1992). a number of studies (wattenberger & scaggs 1979, rislov 1979, hansen 1985, el-khawas 1985, dawkins 1988, beeson et al 1992) have suggested that in order to include organisational issues in curriculum design, it is necessary to involve employer groups in the development of university/college it training programs. while many of these studies focus on larger businesses, recent studies (neergaard 1992, holzinger & hotch 1993, macgregor & cocks 1994, delvecchio 1994) have suggested that there is no less a need to involve employer groups at the small business level. a number of factors have been found to impinge upon both the use of computer technology as well as the training requirements deemed necessary in the small business environment. some of these factors may be termed 'intrinsic', ie. they are part of the structure and function of the firm. these would include the size of the firm, the level of management involvement, the level of skill of the workforce (cragg & zinatelli 1995, macgregor & bunker 1995). other factors such as complexity of the software acquired, changes to structure of the workplace after the implementation of it and criteria for determination of success are more 'extrinsic' factors which impinge upon use of has well as training requirements. this paper begins by briefly considering the findings of several studies examining the acquisition and subsequent use of computer technology in the medical profession which provides the research background for the study. the nature of small business is briefly considered. the paper then examines small business it curricula needs. finally, it is hypothesised that, in line with the background study findings, the level of satisfaction with preajis vol 5 no 1 september 1997 46 installation vendor service will affect those curricular inclusions deemed most important by the small business community, while the level of satisfaction with vendor post-installation services will affect those curricular inclusions less important to the small business community. research background several studies were carried out in australia examining the various branches of the medical profession and their use and training need following the acquisition of computer technology (macgregor & cocks 1994, macgregor et al 1997). these studies found that amongst other factors associated with use and training requirements of it was the level of satisfaction with vendor services. the study found that the level of satisfaction / dissatisfaction with those vendor services normally provided prior to purchase and installation was strongly associated with the rating of curricular inclusions deemed important inclusions in training programs. these pre-installation services included delivery of equipment, vendor supplied information concerning the suitability of equipment and software to the business as well as overall equipment suitability. by comparison, the study found that no such association was evidenced when post-installation services were examined. these post installation services included training, vendor supplied manuals and documentation as well as vendor after sales service. indeed, the study found that the level of satisfaction / dissatisfaction with vendor post-installation service was only associated with curricular inclusions deemed less important in a training program. based on the finding within the medical profession, this study aims to determine whether these associations (satisfaction with vendor services & rating of importance of potential curricular inclusions in an it training program) are unique to the medical profession or whether they apply to small business in general. the nature of small business there have been many studies, both in terms of research initiatives as well as governmental studies on the nature of small business. many of these studies have opted for the approach which identifies the differences of small to large business and it is this approach which best summarises the nature of small business. studies by brigham & smith (1967), walker (1975), delone (1988) found that small businesses are more risky than their larger counterparts. rotch (1987) found the amount of record keeping is minimal compared to larger businesses. reynolds et al (1994), in an extensive study, suggest that, among other factors, small businesses tended to be centralised with informal and inadequate planning and control, concentrated more on product than customers and used limited process or product technology. this is accentuated when computer technology is introduced into the small business environment. studies (cragg & king 1993, holzinger & hotch 1993, chen 1993, delvecchio 1994) showed that small businesses lacked the relevant technical expertise to identify and acquire computer technology, very often relying on vendor/consultant groups to perform these tasks. indeed, one of the more pertinent studies examining this phenomenon was carried out in singapore (see yap et al 1992). among other findings, the study showed that where larger businesses used a resident ft department, smaller businesses viewed the vendor/consultant as a surrogate of that it department. the study showed that the level and type of service provided by the vendor very often permeated all facets of continued use with the technology including upgrading decisions, company structuring around the newly acquired technology, level of use of the technology. this has been supported in subsequent studies (cragg & zinatelli 1995, macgregor & bunker 1995). in all these studies the vendor was a small business consisting of a sales force with some technical support. most were retailers of technology rather than branches of manufacturers. background to the research the definition of what constitutes a small business varies both across research initiatives as well as governmental guidelines. a number of researchers (delone 1988, chen 1993) have avoided these questions by using governmental guidelines and employing mailing lists prepared by these government agencies. the current study has adopted a similar approach by developing a mailing list through the small business component of the niawarra chamber of commerce directory, known as the illawarra business directory. the geographic area covered by this directory includes southern suburbs of sydney, as well as the cities of nowra and wollongong (a population of approximately half a million). as with the definition of small business, decisions concerning inclusions in it training programs offered through university/colleges vary widely. two internationally regarded models have been provided by the acm and the dpma groups. lo (1991) suggested that in order to fully examine all possible curricular inclusions a composite subset of those provided by acm and dpma needed to be established. lo's study arrived at 51 subjects. aj1s vol 5 no 1 september 1997 47 the current study began with the set suggested by lo. these were examined in the light of small business needs. a number were discarded, being considered more relevant to computing professionals than small business. these included areas such as data communications, advanced programming and algorithm development, expert systems, human computer interaction. those that remained were further analysed for content to determine whether they could be 'collapsed' into a smaller group. this approach is in line with the views expressed by seeborg & ma (1989) who state that when non-computing groups were considered, the potential offerings should be significantly reduced. as such, lo's original 51 subjects were reduced to 15 (see table 1). subjects structure and function of computer hardware programming database/spreadsheets business analysis information analysis office automation business accounting systems computer evaluation techniques accountancy finance marketing business law statistics management principles interpersonal skills table 1 curriculum subject groups brief description study of the major components of the microcomputer problem solving and program development the use of databases and spreadsheets in common business problems analysis of the major functions and data in the business data modeling techniques integration of the microcomputer into the office design and use of accounting software techniques for testing and comparing potential hardware and software purchase options introductory principles of accounting introduction to corporate valuations and financial markets market segments, buyer behaviour etc. partnerships, liabilities, contracts etc. descriptive and inferential statistics goal determination, implementation etc. verbal, written, formal and informal communication in line with the studies carried out by macgregor & cocks (1994) two hypotheses were tested. for clarity these will be phrased in the affirmative, rather than the null. hi the level of satisfaction with vendor pre-installation service (delivery of equipment, vendor supplied information, equipment suitability) is positively associated with the perception of importance of those auricular inclusions deemed important. in other words, high levels of satisfaction with vendor services will be accompanied by a higher rating of those curricular inclusions deemed most important. ajis vol 5 no 1 september 1997 48 h2 the level of satisfaction with vendor post-installation service (manuals, training, after sales service) is positively associated with the perception of importance of those curricular inclusions deemed less important. in other words, high level of satisfaction with vendor post-installation service will not be accompanied with a higher rating of those curricular inclusions deemed most important but will be accompanied with a higher rating of those curricular inclusions deemed less important. it should be noted that the determination of which curricular inclusions are most important and which are less important are based on the responses to the questionnaire. in line with previous studies (seeborg & ma 1989, lo 1991, ang & lo 1991, ang 1992), only the top six rated curricular inclusions will be considered most important. all others will be considered less important. method as already noted, a mailing list was developed from a government supplied list known as the illawarra business directory. the definition used within this document for small business was that the company had 50 or less employees and was not a subsidiary of any other firm. as the study was examining the association between vendor supplied services and the rating of curricular inclusions in an it training program, vendor companies fitting the profile of small business were removed from the mailing list. a questionnaire was developed to be sent to small business managers. respondents were asked to rate the curricular inclusion areas (see table 1) in terms of their importance (1 not very important 5 very important) as an inclusion in a college/university provided it training program directed towards small business. additionally respondents were asked to rate their perception of vendor services (1 very poor 5 very good) in the following: • delivery and installation of computer equipment • availability of information concerning computer technology • satisfaction with equipment and software • after sales service • computer systems training • computer systems documentation analysis of results a total of 600 questionnaires were distributed. responses were obtained from 131 businesses representing a 21.8% response. this is substantially higher than the 10% and 7% responses achieved by macgregor & cocks in their examination of the medical profession. 58% of respondents indicated that they had previously undertaken some form of college/university training in their particular field. less than 20% had undertaken any college/university based it training. of particular interest was the fact that all respondents indicated the regular use of it in their day-to-day business, with only 7% of respondents indicating they had been using computer technology for less than a year. prior to examining the hypotheses, it was necessary to gain an overall opinion as to the general interest in college/university based it program directed to small business. as such, the following approach was adopted. if any one of the 15 curricular inclusion areas (see table 1) was rated 5 (very important) by a respondent, the respondent was categorised as seeing the development of a college/university it training program for small business as very important. if no curricular inclusions was rated at 5 but at least one was rated at 4 (important), the respondent was categorised as seeing some importance to the development of such a program. all other respondents were categorised as seeing no importance in such a program. table 2 provides the responses. table 2 the importance of developing a college/university based it training program for small business very important (at least one curricular inclusion rated 5) 109 important (at least one curricular inclusion rated 4) 12 not important 10 the results in table 2 suggest that there is a perception by the small business managers who responded to the survey of the importance of a university/college based ft training program specifically directed to small business. 83% of the respondents considered the development of a training program to be important, with a further 9% giving some support to such a development. while only 58% of the respondents indicated some form ajis vol 5 no 1 september 1997 49 of university/college education, the results are almost identical to those carried out on the medical profession (macgregor & cocks 1994). in those studies, 84.5% of respondents indicated their support for a university/college based it training course directed towards their specific needs, with a further 10.4% seeing some importance to the development. respondents were asked to rate vendor performance (on a scale 1 very poor to 5 very good) across six categories. these were: • delivery and installation of computer equipment • availability of information concerning computer technology • satisfaction with equipment and software • after sales service • computer systems training « computer systems documentation table 3 indicates the overall responses. table 3 overall rating of vendor provided services by small business managers service rating 1 2 3 4 5 very poor very good • delivery and installation of 9 16 31 50 25 computer equipment • availability of information 6 25 44 32 24 concerning computer technology • satisfaction with equipment 7 11 34 51 28 and software • after sales service 13 22 49 28 19 • computer systems training 28 26 51 17 9 • computer systems documentation 26 24 42 24 15 a study carried out by macgregor & cocks (1994) on the computer usage of the australian veterinary profession showed that by far the greatest dissatisfaction with vendor services appeared to be in the areas of vendor documentation and manuals, vendor after sales problem rectification and vendor training. an examination of table 4 would suggest that this trend is not isolated to the area of veterinary software but appears to affect all areas of small business. while 63% of small business respondents rated the equipment as above average, only 19.6% rated the training in the same manner. an examination of the figures pertaining to manuals and documentation, as well as after sales problem rectification, show that vendor after sales service is, for the most part not rated highly. only 28% of the respondents considered the manuals and documentation above average, while 37% rated problem rectification by the vendor as above average. indeed, it would seem that in all areas of after sales service (changes to system, training, documentation and problem rectification) vendor support and services were significantly less than those areas which might be designated 'pre-sales support'. respondents were asked to rate the 15 curricular inclusions in terms of their importance for a university/college it training course for small business. table 4 shows the means and ranking of responses. aj1s vol 5 no 1 september 1997 50 table 4 relative importance of each of the curricular inclusions topic mean rank struct/funct of hardware 3.16 10.5 programming 3.09 13 database/s'sheet 4.18 2 business analysis 4.04 4 info. analysis 4.08 3 office autom. 3.75 6 bus/acc't systs 4.28 1 eval. techniques 3.39 8 accounting 4.01 5 finance 3.65 7 marketing 3.10 12 business law 2.66 15 statistics 2.99 14 mgt principles 3.16 10.5 interpersonal skills 3.19 9 the results in table 4 show that the most preferred inclusions (deemed to be the top six) in a university/college it training programs for small business are: business and accounting systems, database and spreadsheets, information analysis, business analysis, accounting, and office automation. it is interesting to note that a previous study of the medical profession (macgregor & cocks 1994) showed that five of the six were also considered very important. indeed, the only difference between the two respondent groups was that the medical profession indicated a need for marketing in preference to office automation. the results in table 4 highlight several other important issues. firstly, there appears to be a perception by small business respondents of the need to fully understand the nature of their business and the data that business utilises. this is borne out in the high ratings of business analysis and information analysis. secondly, there is a perception that knowledge of hardware and software is of less importance to the business than the understanding of the proper use of the software. finally, there appears a perception that in order to use database, spreadsheets and accounting software correctly, there is a need to complete some study in the area of accounting. a series of chi-square tests were applied to the data to determine whether the ratings of the curricular inclusions were associated with the ratings of the six vendor services. table 5 indicates those curricular inclusions whose rating was associated with the rating of vendor services. to simplify the data, those respondents who rated vendor services as 1 or 2 ( very poor, poor) were considered dissatisfied. all others were considered satisfied. table 5 provides a number of results. firstly, satisfaction in terms of vendor delivery of equipment appears to be associated with the rating of several curricular inclusions database and spreadsheets, information analysis, business accounting systems. in all cases, dissatisfaction with vendor provided services is associated with a lower rating of these curricular inclusions in terms of their importance. satisfaction with vendor provided information was examined to determine if it was associated with the rating of any of the 15 curricular inclusions. table 5 shows that the rating of two curricular inclusions business accounting systems, management principles were associated with the level of satisfaction of the respondents. those who were satisfied with vendor provided information rated the curricular inclusion business accounting systems higher than those who expressed dissatisfaction. this was reversed for the curricular inclusion management principles, with those who were dissatisfied rating it higher than those who expressed satisfaction. the results in table 5 show that the level of satisfaction with the equipment is associated with two curricular inclusions database and spreadsheets, information analysis. in both cases those who were satisfied rated the curricular inclusions higher than those who were dissatisfied. satisfaction with after sales service is associated with the rating of one curricular inclusion statistics. those who were satisfied with the after sales service rated this inclusion higher than those who were not. aj1s vol 5 no 1 september 1997 51 table 5 the rating of potential curricular inclusions in a college/university it training course for small business which were associated with the ratings of vendor services vendor service subject rating of service rating of importance of curricular inclusions 1 2 3 4 5 not impt very impt. delivery of equipment database/ satisfied spreadsheet dissatisfied significant (p<0.05) information satisfied analysis dissatisfied significant (p<0.05) business acct satisfied systems dissatisfied significant (p<0.005) vendor provided info. equipment after sales service training manuals business acct. satisfied systems dissatisfied significant (p<0.05) management satisfied principles dissatisfied significant (p<0.05) database/ satisfied spreadsheet dissatisfied significant (p<0.005) information satisfied analysis dissatisfied significant (p<0.05) statistics satisfied dissatisfied significant (p<0.05) marketing satisfied dissatisfied significant (p<0.05) statistics satisfied dissatisfied significant (p<0.05) 0 2 0 2 0 3 2 1 7 0 0 2 1 2 6 6 11 3 6 6 0 2 5 7 6 0 5 0 14 7 7 6 16 6 7 1 20 5 11 1 32 6 8 0 24 1 26 15 25 19 21 21 26 6 28 9 38 7 29 13 38 13 28 4 29 33 4 27 10 29 7 70 14 55 59 11 58 14 18 5 71 12 53 7 17 3 7 16 9 10 the results in table 5 show that the level of satisfaction with vendor provided training is associated with the curricular inclusion marketing. unlike the rating of after sales service, those who rated the training as satisfactory tended to rate the inclusion marketing lower than those who were dissatisfied. finally satisfaction in terms of vendor provided manuals was compared to the 15 curricular inclusions. the results in table 5 show that satisfaction with vendor provided manuals is associated with the rating of the inclusion statistics. while those who were dissatisfied with the vendor provided service could see some merit in the inclusion, those who were satisfied, at best, tended to be equivocal concerning its importance. ajis vol 5 no 1 september 1997 52 discussion before discussing the issues raised by the data and examining the hypotheses posited, the limitations of the study need to be emphasised. the first obvious limitation of this pilot study is the geographic spread of the respondent small businesses. all respondents were located in one urban region of australia. all businesses are within easy commuting distance of a large university campus and a high percentage of managers have had some university experience. the second limitation of this study centres around the concepts of satisfaction and perception of importance. the literature suggests that very often these concepts are viewed comparatively rather than in their own right. for example, it may be argued that dissatisfaction with vendor provided training may be a measure of it being less effective than vendor supplied after sales service. thus while training may have, of itself, been successful, it was judged comparatively with other measures. finally, the comparison of auricular inclusions, rated across a five point scale and satisfaction rated across a two point scale does not necessarily provide a linear basis for discussion. the reduction, however, of satisfaction to a two point scale was deliberate such that comparative issues were reduced. despite the limitations, a number of insights are provided by the data. firstly, there appears to be an overwhelming desire by the respondents for some form of university/college it training program specifically directed towards small business. the study confirmed earlier findings by cragg & zinatelli (1995) and chen (1993), that computer technology in the small business arena is still primarily aimed at accounting type functions. this is highlighted in the data with accounting centred curricular inclusions being rated higher than other less accounting centred ones. indeed, the data suggests that technical inclusions structure and function of hardware and software and programming are of little interest to the small business manager. of particular interest is the rating by respondents of vendor supplied services. it would seem that vendor companies are primarily sales directed and provide far better pre-sales service than their after sales. vendor provided training and manuals appear to be inadequate to the need of the small business. two hypotheses were posited. the first suggested that satisfaction with vendor provided pre-sales service would be associated with a higher rating of those curricular inclusions deemed important. the results showed that three of the top inclusions database and spreadsheets, information analysis, business accounting systems were rated higher by those who expressed satisfaction as compared to those who expressed dissatisfaction. interestingly, one inclusion not within the top six, management principles, was also associated with the level of satisfaction of vendor provided pre-sales service and actually showed a reversal of rating association, with those who were dissatisfied rating it higher than those who expressed satisfaction. as such, the first hypothesis is rejected. a number of reasons for the associations are possible. it may be argued that those who rate inclusions such as database and spreadsheets, information analysis, business accounting systems were better planned in their acquisition of it. in such cases more attention was paid to vendor provided information, delivery schedules and actual equipment. conversely it may be argued that satisfaction with the vendor has allowed the small business more scope to investigate training and training requirements than a less satisfied customer. the second hypothesis suggested that higher levels of satisfaction with vendor provided post-sales service would be associated with higher ratings of those inclusions deemed less important. unlike the studies carried out by macgregor & cocks (1994), this study does not support the hypothesis. while it is true to say that post-sales satisfaction is only associated with the rating of less important inclusions, the nature of the association varies both with the particular vendor service as well as the inclusion. conclusion this pilot study of 131 small businesses in australia has examined both the perception of vendor services as well as the perception of training needs. the results suggest that there is an overwhelming need for the development of a university/college it training program directed specifically towards small business. these programs should avoid the more technical aspects of it and concentrate on areas which invlove the analysis and use of software in the small business. the results further show that there is a perception by small business that vendor post-sales service, particularly in the areas of training and manuals are not adequate. finally the study has shown that a number of vendor provided services appear to be associated with the rating of importance of potential curricular inclusions in it training programs. although the results have important implications, additional research is required in a number of areas. firstly, the reasons why certain measures of satisfaction only appear to be associated with certain curricular inclusions. in order to pursue this, these factors need to be refined and followed up by extensive interviewing. perhaps more importantly, a more intensive examination needs to be carried out to examine how small businesses communicate with vendors, such that the effect of various measures of satisfaction may be better explained. ajis vol 5 no 1 september 1997 53 references ang a.y. (1992) information systems curricula: a southeast asian perspective journal of computer information systems pp. 7 -15 ang a.y. & lo b.w.n. (1991) changing emphasis in information systems curriculum: an australian industrial perception proceedings of the second conference on information systems and database special interest group pp. 339 355 baker w.h. (1987) status of information management in small businesses journal of systems management vol38,no.4,pplo-15 beeson g.w., stokes d.m. & symmonds h.c. (1992) an innovative higher education course to meet industry's needs journal of higher education research and development vol 11, no. 1, pp 21 38 bergeron f., raymond l. & rivard s. (1992) organisational benefits of electronic data interchange proceedings of the third australian conference on information systems wollongong, australia pp 563 578 brigham e.f. & smith k.v. (1967) the cost of capital to the small firm the engineering economist vol. 13, no. 1, pp. 1 26 chan y.e. & huff s.l. (1993) investigating information systems strategic alignment proceeding of the 14th annual conference on information systems orlando, fl, pp 345 363 chen j.c. (1993) the impact of microcomputers on small businesses: england 10 years later journal of small business management vol. 31, no. 3, pp. 96 -102 cragg p.b. & zinatelli n. (1995) the evolution of information systems in small firms information & management vol 29, pp 18 cragg p.b. & king m. (1993) small firm computing: motivators and inhibitors mis quarterly vol. 17, no. 1, pp. 47-60 dawkins j.s. (1988) higher education: a policy statement canberra, australian government printing office delone w.h. (1988) determinants for success for computer usage in small business mis quarterly pp. 51 61 delvecchio m. (1994) retooling the staff along with the system bests review vol. 94, no. 11, pp. 82 83 el-khawas e. (1985) campuses weld the corporate link educational record spring pp 37 39 hansen l.a. (1985) a more or less happy relationship: industry university co-operation in denmark european journal of education 10, pp 197 208 hedberg j & harper b. (1992) information systems strategy in the small education organisation proceedings of the third australian conference on information systems wollongong, australia heikkila j., saarinen t. & saaksjarvi m. (1991) success of software packages in small business: an exploratory study european journal of information systems vol 1, no. 3, pp 159 169 holzinger a.g. & hotch r. (1993) small firms usage patterns nations business vol. 81, no. 8, pp. 39 42 kahn h. & robertson i.t. (1992) training and experience as predictors of job satisfaction and work motivation when using computers journal of behaviour and information technology vol. 11, pp 53-60 lo b.w.n. (1991) australian information systems curricula the journal of computer information systems vol. 31, pp. 20 33 macgregor r.c. & clarke r.j. (1988) the loss of the 'informal' in systems design in bullinger h.j., protonotarios e.n., bouwhuis d. & relm f. (eds.) proceedings of eurinfo '88 athens, greece macgregor r.c. & cocks r.s. (1994) computer usage and satisfaction in the australian veterinary industry australian veterinary practitioner vol 25 no 1 pp 43 48 macgregor r.c. & bunker dj. (1997) the effect of criteria used in the acquisition of computer technology on the ongoing success with information technology in small business, forthcoming macgregor r.c., bunker d.j., cocks r.s., pierson j.k. & forcht k.a. (1997) a comparison of two studies examining computer education requirements in small business, forthcoming meredith g.g. (1994) small business management in australia mcgraw hill, 4th edition neergaard p. (1992) microcomputers in small and medium sized companies: benefits achieved and problems encountered proceedings of the third australian conference on information systems wollongong, australia nelson r.r. (1991) education needs as perceived by is and end-user personnel: a survey of knowledge and skill requirements mis quarterly december pp 503 521 ajis vol 5 no 1 september 1997 54 raymond l. (1988) the impact of computer training on attitudes and usage behaviour of small business managers journal of small business management vol. 26, pp. 8 13 raymond l. (1990) end user computing in the small business context: foundations and directions for research data base vol 20, no. 3, pp 20 28 raymond l. & pare g. (1992) measurement of information technology sophistication in small manufacturing businesses information resource management journal vol 5, no. 2, pp 113 raymond l. & bergeron f. (1992) personal dss success in small enterprises information & management vol 22, no. 5, pp 301 308 raymond l., pare g. & bergeron f. (1993) information technology and organisational structure revisited: implications for performance proceeding of the 14th annual conference on information systems orlando, fl, pp 129 143 reynolds w., savage w. & williams a. (1994) your own business: a practical guide to success itp rislov s. (1979) speaking with employers in cohen a.m. (ed.) shaping the curriculum josey bass, san francisco, pp 51 54 rotch w. (1967) management of small enterprises: cases and readings university of virginia press seeborg i.s. & ma c. (1989) mis program meets reality: a survey of alumni from an undergraduate program interface vol. 10, pp. 51 60 sharp c. & lewis n. (1992) corporate memory and management information systems proceedings of the third australian conference on information systems wollongong, pp 149 160 turner j.a. & karasek r.a. jr (1984) software ergonomics: effects of computer application design parameters on operator task performance and health ergonomics vol. 27, no. 6, pp 663 690 walker e.w. (1975) investment and capital structure decision making in small business in walker e.w. (ed.) the dynamic small firm: selected readings austin press, texas wattenberger j.i & scaggs (1979) curriculum revision and the process of change in cohen a.m. (ed.) shaping the curriculum josey bass, san francisco, pp 1 10 williams t.a. (1992) information technology and interorganisation change proceedings of the third australian conference on information systems wollongong, australia pp 295 308 yap c.s., soh c.p.p. & raman k.s. (1992) information systems success factors in small business international journal of management science 20, pp. 597 609 ajis vol 5 no 1 september 1997 ajis vol. 7 no. 1 sept 1999 using qfd to tame wicked information system's requirements steve howard swinburne chi laboratory, school of information technology, swinburne university, po box 218. hawthorn, 3122. australia. showard@swin.edu.au abstract this paper argues that information system's requirements can be in tension with both themselves and the context to which they apply. the tensions act as limiters on our ability to deliver effective information systems. in the paper we describe examples of such limiters. the paper then presents a response to this problem in the form of quality function deployment and calls for a more systems oriented view of information systems development, which though not capable of removing the ceiling, is capable, in principle, of engineering best compromises. a case study is used to illustrate the application of qfd to the specification of information system's requirements. keywords requirements, qfd, house of quality, trade-off analysis information systems, compatibility and wicked problems ensuring compatibility between the various elements of an information system, eg. users and their tasks, organisational structures and culture, and information technology, has reigned as the primary goal of information systems for the past two decades (avison and fitzgerald, 1988). even assuming that we can design for compatibility, what do we do when two (or more) different 'compatibilities' are in tension and we, as information system designers, cannot ensure both? information system compatibility, when seen from this perspective, is a 'wicked' problem (rittle and webber, 1973 and vicente et al, 1997) characterised by: • the lack of a definitive formulation. different stakeholders have different perspectives on what is and what is not the most important information systems issue; • solutions to information systems design problems tend to be more or less satisfactory, not 'right' or 'wrong'; • the solution space to any information systems design problem is not exhaustively identifiable. often candidate solutions are overlooked, less satisfactory solutions chosen etc. the traditional response to the problem of information systems requirements analysis is essentially reductionist and based on principles of decomposition or step-wise refinement. though suitable for complex and well defined problems (where the means-ends relationships are stable and understood) such an approach is problematic when we consider the characterisation of wicked problems presented above. this paper builds of many traditions: churchman's (1971, '78) work on inquiring systems; stowell and west's (1989, '90) work on client-led design; checkland's impressive work on the soft systems methodology (checkland, 1991); and work by the quality function deployment community (cohen, 1995, akao, 1990) on capturing and using the 'voice of the customer' in product development. we present a discussion of the problematic of wicked requirements problems and a practical tutorial to a technique that can be used to analyse such complex requirements situations. the paper will be of most interest to those unfamiliar with the techniques of quality function deployment and those tasked with the analysis of wicked requirement situations. consider an example from another domain for illustration. the australian iron man' competition is a triathlon composed of run, swim and board (essentially a sea kayak) legs. in the early 90s, trevor hendy (a previous australian iron man champion) attempted to become a member of the australian olympic kayak team. to do this, hendy preferred board training over run and swim activities. during the early part of 1996 hendy returned to iron man competition and struggled in the swim and run legs of the event due to the combined effect of underdeveloped lower body strength and increased upper body muscle bulk. there would seem to be a tension then between the three individual sports and therefore a conflict when sports are combined within one individual. iron men must compromise their performance in the individual sports in order to optimise their performance overall. it follows then that it is not necessarily the case that a given system (in this case a triathlete) can be designed with maximum compatibility between all of its subsystem components (i.e. maximum performance across all three sports). it may be appropriate to sacrifice sub-system compatibility (e.g. board, swim or run leg performance) for the sake of overall system goals (e.g. triathlon performance). do such tensions exist in information systems design? figure 1 illustrates an information system as a set of 'compatibility relations'. it should be stressed that the diagram is not intended to be complete, on the product 19 ajis vol. 7 no. 1 sept 1999 side for example one could reasonably add the user's task or, in the case of a cscw system, other users. it should also be stated that the compatibility relations illustrated are those seen from a particular perspective, and are not independent of that perspective. the technique we go on to present below can be used equally well in a relativist mode where the final expression of the requirements depends on the discourse between various stakeholder groups (and so 'optimum conpatibility' would be that defined by a particular group) as in a positivist mode where the requirements are a function of the implied system's ontology. in figure 1 the design problem is seen as one of 'information system design1that is, designing the most appropriate fit between all of the elements in the information system (including user, organisation, technology etc) rather than simply software or hardware design. the user can be seen as consisting of a number of subsystems (e.g. visual, motor, cognitive...), and likewise the machine can be viewed in terms of its elements (input dialogue, output dialogue, input and output devices etc). in turn, the subsystems themselves can be broken down into ever-smaller units, for example a mouse consists of various mechanical and electrical components. an information systems designer might be concerned with the compatibility between screen layout and the task; between the amount of domain knowledge possessed by the user and that required by the computer; between the computer system and the organisation within which the users work etc. unpacking the diagram further, the designer also has to consider the environment within which the information system will be situated and the impact that the physical and social context will have over the information system. these seem like obvious points, but the implications, developed below, are profound. figure 1 compatibility relations and information systems the product of designinformation system element compatibility organisational and social environment of use user sub-system machin sub-system further sub-systems machine subsystem machine subsystem machine subsystem further sub-systems compatibility relation environment of design designer „! sub-systeny • — . j >further sub-systems ^ ---, / \ design sub-system / v^ design sub-system the process of design 20 ajis vol. 7 no. 1 sept 1999 potentially then, all information system subsystems can be in a compatibility relation with all other subsystemsthe number of possible compatibility relations being very large indeed for most real systems. there are three possible logical relations between information system subsystems: irrelevant, compatible and incompatible, as shown in figure 2. irrelevant relations exist when two information system subsystems are not connected (ie. do not communicate and are not interdependent), as in the case of elements b and c in figure 2 above. for example, a designer may believe that a mouse is an appropriate input device for the design task in hand, and that an information system should model domain knowledge. these two subsystems (input device and domain model) are not connected and are therefore irrelevant. compatible relations exist when acting on all alternatives is possible, as is the case where a designer decides to implement both a menu and a command based interface and allow information system users to select between the two (interestingly, the users then would be forced to make the selection). figure 2 three compatibility relations incompatible | element b] irrelevant incompatible relations exist when a clear choice must be made between the alternatives, when for example a menu or command based interface must be selected. in this case, the alternatives are clearly in tension. the next section will consider examples of incompatible relations drawn from ongoing information system research. irrelevant relations are significant because they can become compatible or incompatible relations. if the designer above decided to implement both menu and command interfaces at a risk to the project deadline the two irrelevant alternatives are made incompatible. any given information system can therefore be seen in terms of a network of interrelated compatibility relationsrelations between and within the user, the technology and the organisational and social environmental a number of levels of abstraction. the implications of such decomposition however challenge the very notion of being able to 'design for compatibility' for as we have seen, there are in fact many 'sub-compatibilities' and when we have many of anything there is the possibility that they might interrelate and compete. put simplythe various compatibility relations within any information system, that must be adequately engineered if we are to have successful information systems, can be in tension with each other. further, figure 1 alludes to the fact that the information system product can also be in tension with the design processwhen for example the designer does not have appropriate resources (tools, knowledge, time) to deliver a desired information system. any real design problem will contain many such compatibility relations and, given their interrelated nature, design might be better conceived as being concerned with achieving the most appropriate balance, rather than striving for an abstract ideal information system. in part this is another way of saying that the design of information systems is a wicked problem. one way to reason about the information systems, and their composition from interrelated compatibility relations, is to distinguish between harmony and compatibility. harmony is a global compatibility, and describes the degree of dissonance between all subsystem elements, including users, tasks, organisational contexts etc. in this formulation, systems can vary between maximum harmony, which represents the presence of complete compatibility between subsystems, and minimum harmony, which represents the situation of complete incompatibility between these elements. all information systems can, in principle, be placed somewhere on this continuum. the goal of information systems development is to create a system with maximal harmony, though in practice, due to the tensions discussed above and below, satisfactory harmony only is possible. in order to identify the degree of harmony embodied in a system, it is necessary to consider the various compatibility relations, ie. sub-system elements and their interrelationships. the measure of harmony then becomes the total accumulation, or aggregate, of system compatibility relations. as illustrated above however, compatibility relations can take three forms (irrelevant, compatible and incompatible) and this will affect the derivation of harmony. the next section illustrates situations in which compatibility relations are not simply additive, but competitive, in their contribution to compatibility. 21 ajis vol. 7 no. 1 sept 1999 information system's requirements ceilings the examples below illustrate different types of tension in various information systems. of course, by judicious design, it may be possible to resolve the tension and "find a way out1. we do not mean to imply that the tensions are insurmountable, only that they are present and, given appropriate tools (see 'house of quality' below), perhaps resolvable. the tensions fall into three classestensions between information system subsystems, tensions between information system subsystems and their context and tensions between development processes and the information system. each tension can be seen as the tension between two or more compatibility relations in their contribution to harmony. between different subsystems • individuals versus groups and time-shared computers time-sharing on main-frame computers satisfies the requirements of a group sharing the same resource with a minimum of encumbrance, yet does not satisfy the requirements of an individual programmer in terms of compile time, response time etc. designers of such systems have made a choice to sacrifice the needs of the individual programmer, so as to optimise the availability of resources for the group. how do we represent such choices and allow information systems design teams to reason about the consequences of those choices? • real time versus batch and order entry users taking transient orders (e.g. over the phone) often require different functionality of computing support than users inputting orders from paperwork due, in part, to differences in the temporal aspects of their tasks. how should designers respond to this challenge when both groups of users are to use a single system? • experts versus novices and the user interface users with differing computer expertise may have different requirements of computing support e.g. command input versus menu selection, access to advanced features, access to on-line tutorials. information systems designers can respond to this challenge in a number of ways, e.g. ignore the variation and produce a single interface, produce user-centred configurable interfaces, produce machine-centred adaptive interfaces. how do we support the designers and users in exploring and prioritising the options? • consistency versus compatibility and sub-optimal style guides many information system designers are required to follow standards and guidelines. however, many standards do not embody optimal advice (teasley and scholtz, 1997). here we see a tension between adhering to the style guide to ensure consistency across information systems and ignoring the style guide so as to ensure compatibility between the user and the information system. is it better to be consistently incompatible or inconsistently compatible? • abstract versus concrete and corporate style guides ongoing work in the implementation of corporate style guides aims to provide designers with concrete instantiations, or exemplars, of the advice contained in the style guide. however, any particular abstract style rule can be instantiated as any number of concrete exemplars. furthermore, concrete exemplars may arbitrarily and unwisely constrain the solution space. providing exemplars is 'good' (they make concrete the rules of design), but providing exemplars is also 'bad' (they may restrict the cognitive solution space to that bounded by the example). do we use exemplars or not? in the above examples, the conflict can be seen as a 'subsystem ceiling', where information system subsystems themselves present a ceiling to their optimisation. improving one compatibility relation necessarily negates another, unless the design problem can be reconceptualised. if a subsystem tension cannot be designed out of the system then negotiations to achieve compromise will be necessary between the relevant subsystem owners. a subsystem tension then is generally resolvable at the level of the subsystems concerned. 22 ajis vol. 7 no. 1 sept 1999 between subsystems and their context • message versus context and headphone use current research (e.g. petrie et al, 1997) in navigation aids for visually impaired people has struggled with the tension between providing users with a clear auditory signal (by requiring, for example, users to wear headphones) whilst not isolating users from the surrounding, and potentially dangerous or otherwise significant, physical and social context. the challenge here is that both a clear auditory message and access to environmental cues are important components of the information system and should be optimised, however they are in tension. to conceive of the problem as one of simple information system compatibility is useless when the very compatibilities themselves are in tension. here the tensions can be seen as context ceilings, where the context presents a ceiling to the optimisation of subsystem compatibility. if context ceilings cannot be side-stepped through elegant redesign then they imply high level changes may be required involving many stakeholders. identifying the type of tension is important in understanding the nature of the appropriate resolution. between product and process • usability versus cost usability is often cited, sometimes unfairly, as a cost factor in the development process (bias and mayhew, 1994). *we could produce more user-friendly systems, but we just don't have the money or the time' is the cry often heard. whether real or apparent, usability (and other information system product qualities) and development costs frequently seem in tension. a designer capable of resolving this tension needs to know about development budgets and usability factors, and needs to understand the knock-on effects that each has on the other. here the conflict can be seen as a process ceiling, where the development process itself presents a ceiling to the optimisation of usability. process ceilings are common in information systems development; the system cannot be optimised because the budget has been spent, or the time has expired, or the team do not have the relevant expertise, or access to the right sort of stakeholders could not be obtained. if process ceilings cannot be avoided through appropriate design then negotiations between the customer and the developers will be necessary, aimed at changing either the product requirements, or the project constraints, or both. summary the three compatibility tension types illustrated above are not necessarily mutually exclusive. complex design problems will often contain many tensions. further, there are semantic difficulties in identifying types of tension, in particular, whether a problem is seen as a subsystem tension or a context ceiling depends in part on the grain of analysis used, in that other subsystems themselves occupy the environment of any particular subsystem. in each case above, if the tension cannot be removed by reconceptualising the design problem then an appropriate balance must be struck involving negotiations between the stakeholders and developers cited. what then can we offer the identification and satisfactory resolution of such tensions? the house of quality quality function deployment (qfd) is a systematic planning process designed to explicitly incorporate the 'voice of the customer' into product design. used in the manufacturing industries (akao, 1990, and cohen, 1995) to prioritise customer needs and map these needs onto technical constraints, applications are appearing in the software domain (shindo, 1991 and zultner, 1992). qfd's primary components include a structured planning process which draws input from cross-functional teams working within a concurrent engineering paradigm. ongoing decision making and communication is supported through the use of a graphical notation termed the 'house of quality'. although aspects of the planning process are hardware specific (cohen, 1995), the house of quality notation appears to work well for software development (shindo, 1991 and zultner, 1992, ala-siuru, 1993 and lundell and williams, 1993). this section considers the use of the house of quality to articulate and reason about the types of tensions discussed above. 23 ajis vol. 7 no. 1 sept 1999 planning and gathering the voice of the user the first step in developing the house of quality (hoq) involves collecting the voice of the customer'. within qfd both qualitative and quantitative data are collected using a variety of situated and non-situated techniques including direct observation, one-on-one interviews, focus groups and contextual inquiry. the results of this data collection are displayed using various graphical techniques including affinity diagrams (see figure 3), tree diagrams and prioritisation matrices (cohen, 1995). the affinity diagram shown in figure 3 represents discussions with teachers and pupils regarding their needs for a particular information system, a pc based 'soft' keyboard. the diagram captures both verbatim statements of their desires, and also structures them along hierarchical lines, where compatibility relations and harmony can be represented, as shown in the diagram. these data collection techniques and graphical representations appear to be compatible with the early, analytic stages of information systems development and can be conducted in parallel, or alongside. figure 3 affinity diagram for keyboard users harmony teachers needs o"play notes quick! on t want to pre wrong noteonlylimited funds £*needto play tune rignt*n play keyboard wi mouse ys easy to find and payout to conform exdectations compatibility relations building and using the house of quality figure 4 illustrates the most basic version of the hoq. many different extensions to the hoq exist according to specific needs and contexts (see cohen, 1995 and akao, 1990 for a detailed review of the variations). the output of the previous step is typically a hierarchical representation of the customer's needs, similar to figure 3. the first step in building the hoq is typically to list the customer's needs and benefits. this is then mapped against the technical response to those needs and benefits. typically, the customer's needs and benefits are represented in the 'voice of the customer' (verbatims taken from interview notes or video taped observations) whilst the technical response is the engineer's or development organisation's response to the customer's needs, and is written in appropriately technical language. the technical response can be made at a number of different levels of abstraction, e.g. acceptance criteria, requirements or product features. if formal measurements are available for either the customer needs (from previous information system benchmarking for example) or the technical response, then these can be included in the hoq along side their narrative descriptions. the hoq then can be seen as a graphical tool for evaluating the relationship between the concerns of the customer and those of the developer. the next step in completing the hoq is to examine the relationships between these concerns. for each cell in the matrix, the cross-functional team (importantly, consisting of both customers and developers) enters a 'value' that captures the extent to which the engineering variables represented in the technical response contribute to 24 ajis vol. 7 no. 1 sept 1999 overall customer satisfaction. this is perhaps the most important outcome of the hoq analysis as it allows information system developers to prioritise their activities in line with customer needs. the final step in completing the hoq shown in figure 4 is to complete the technical correlations matrix. this matrix captures the way in which different technical variables support or compete with each other. this information serves two main purposes. firstly, by identifying which technical variables are interrelated it allows the engineers to identify key design bottlenecks. secondly, if two technical variables are owned by different members of the cross-functional design teams then the correlations suggest communication pathways within the teams. sometimes this is indicated in the hoq by an arrow pointing from one technical variable to another that will be impacted if problems arise during its development. figure 4 the house of quality v vt i *of — omer ,§ customer needs/benefits . . . . ^ •• • s .̂ — / technical coirelations voice of the engineer relationships information systems and the house of quality how would the use of the hoq have helped in the examples discussed earlier where information systems ceilings were encountered? figure 5 illustrates the tension between providing blind people with headphones, thus improving the quality of auditory feedback they receive whilst isolating them from their surroundings. clearly, a full hoq analysis of this problem would contain many information system's requirements and technical responses, and many consequent correlations. for the purposes of illustration, only the tension introduced above has been represented. figure 5 illustrates the user's concern (i.e. to be able to hear both the taped messages and the environment) and the engineer's response (the engineering options considered are to provide either headphones or an external speaker). from a technical perspective, the two engineering options are not seen as related (hence no symbol appears in the technical correlations 'roof of the hoq). later in the analysis it may be that costs force these two options to become exclusive (and hence 'negatively linked') in the sense that both options cannot be provided within the budget for the product. the relationships matrix indicates that, whilst providing headphones is strongly positively linked to the need for a clear audio signal, it is also negatively linked to the need to hear the surrounding audio environment. a compromise solution (from those enumerated) would appear to be the speaker that is moderately positively linked to both user needs. the hoq has been extended from that presented earlier to include an 'information system's requirements correlations' lattice. just as technical correlations captured situations where technical factors were in tension, so the information system's requirements correlations lattice captures the ceilings, or tensions, introduced earlier. 25 ajis vol. 7 no. 1 sept 1999 information system's requirements correlations inform the design team that the user has competing needs. either compromises will be required on the part of the user, or a radical rethink of the design is needed. in this case it may be that the design team needs to consider providing blind people with a single headphone. we have found that it is sufficient to simply note that an information system's requirements correlation exists (by placing a '+' or '-' symbol in the appropriate cell) though clearly distinctions between different types of information system's requirements correlation are possible at this early stage, e.g. whether the requirements in tension are 'mandatory', 'nice-to-have1 or 'safety critical'. figure 5 the house of quality and information systems ceilings key + negatively correlated positively correlated x negatively linked o moderately positively linked © strongly positively linked 41 a. £ s is requirements h "i need to hear the taped messages" "i need to hear noise around me" vls£ £ & £ ® x o •3j £ o 0 is requirements correlations an additional benefit of capturing information system's requirements correlations comes when the design is being conducted for a diverse stakeholder population. in this circumstance, information system's requirements correlations can span different stakeholder communities and their presence suggests the necessity for these communities to communicate with respect to their interrelated needs. this was the case with the expert/novice example discussed above. the information system's requirements correlations for this example would have highlighted that there was a tension between providing both groups with appropriate dialogue, feature access etc and that, if the tension could not be removed through design, the groups would need to be brought together and compromises would be required. the hoq then has different uses for different participants in information systems development. developers can use it to summarise basic technical data and highlight the interrelations between technical variables. if the software being developed is to be a commercial product, then for marketing personnel the hoq summarises the user's priorities whereas for management new product opportunities can be uncovered. fundamentally, the hoq encourages all of the stakeholder groups to understand and debate each other's priorities (hauser and clausing, 1988). using the house of quality figure 6 shows an expanded hoq modelling a problem in more detail. the design team were at the initial stages of designing a musical keyboard to be shipped with a simple animation package to be used by school children and information technology teachers alike. the keyboard was to appear on a computer vdu and notes were to be played, in sequence, using a mouse. chords were not required. the animation package had a 'record and play' function that allowed strings of notes played on the keyboard to be recorded and played back at defined points in an animation, thus allowing students and teachers to add simple soundtracks to cartoon like animations. 26 ajis vol. 7 no. 1 sept 1999 the main challenge faced by the designers was that of satisfying both the teachers and the children with one design; an explicit constraint placed on the project by the customer. the hoq in figure 6 has been framed specifically around resolving this tension and so is rather focussed in what it models. as shown in figure 6, the teachers' primary needs were for a keyboard that allowed them to record the correct string of notes quickly, and that the keyboard should be of a conventional design. according to pitt's law (card, moran and newell, 1983), time to point to an object (using a mouse for example) is a function of the size of the object and the distance to that object and hence pointing time can, in part, be reduced by designing musical keys positioned closely together. pointing time can be further reduced by maximising the control/display gain. however, both the pupil's concerns (to play strings of notes accurately) and characteristics (novice mouse users) suggested that accuracy be the preferred design criterion and so the designers should aim for large keys positioned far apart. this can be read in the technical response section of figure 6. the hoq, as shown in figure 6, was used as the basis of an heuristic walkthrough with both the users and designers present. the walkthrough was conducted on the design teams first prototype, the linear keyboard illustrated in figure 7. it was clear from developing the hoq that the two user groups had competing needs. examining the technical correlations roof of figure 6 indicated that the technical responses made to the teachers needs emphasised speed of selection whereas the technical response to the pupil's needs emphasised accuracy of selection. furthermore, the teacher's need for a conservative keyboard layout conflicted with the teacher's need to play notes quickly; a conventional linear keyboard would result in some keys being far apart in space and consequently longer selection times. the linear keyboard of figure 7 then appeared to suit neither the teacher's need for speed nor the pupil's need for accuracy of key selection. it did however satisfy the teacher's concern for a keyboard that conformed to expectations. the second prototype produced the pie design in figure 7. here the designers produced a keyboard on which pupils could have large keys placed far apart (by playing around the periphery of the keyboard) and teachers could have small keys positioned close together (by playing in the central region of the keyboard). however, the pie keyboard clearly violated the teacher's requirement that the keyboard "conform to expectations". referring to the relative importance 'funnel' of figure 6, previously the teachers had indicated that the ir requirement "conform to expectations" was more important than being able to "play notes quickly". the design team therefore decided to relax the design feature that minimised speed of selection (placing keys close together) in favour of a more conventional keyboard design. the resulting design can be seen as the hybrid prototype in figure 7. this design allows the pupils to play around the upper region of the keyboard and the teachers to play around the more compact lower region. further, the design adequately conforms to expectations (in the view of the users!). the dimensions of the house of quality the hoq is the product of the assembly and interaction of the users, information systems, developers, development processes, organisational contexts etc. when using the hoq to model information system requirements three aspects of both the information system requirements and the technical response become relevant: scope the scope of the information captured in the hoq relates to the number of elements (i.e. instances of user, machine, context) represented. ideally, the scope in the hoq should be broad enough to include all significant elements in the intended information system, but not so broad as to be unfocused. additionally, as the subsystem/cost tension illustrated, the hoq may also include aspects of the development process itself, for example, the designer's characteristics, knowledge of development budget and project timeline. in principle, the scope may be very limited, and consist of an interaction between only two elements, or very extensive, and consist of many elements and interactions. we have found it best however to focus the development of the hoq on a specific prioritisation problem, rather than attempt to model the entire design. 27 ajis vol. 7 no. 1 sept 1999 figure 6 the house of quality and resolving the ceiling key + positively correlated negatively *• correlated x negatively linked d moderately positively linked §> strongly g""! | q £ l•e ss» ] xxxxxxxyv\xx/v\a/s, *> vk o o l * positively linked ^n>v "a ^^ =^sx^ -5 is requirements ^^ v vt 1 •9 <u s vt &s "need to play notes quickly" "layout to conform to expectations" "need to play tune right" "don't want to press the wrong note" "keys easy to find and press" h 3 5 5 3 5 /nx-xxx/x/x/xxx/x/ x speed fcj 1 of wh s<j & jc © x x £] 1 j»f u © x x accuracy £ 1̂ 5 i x © © fli* f x © © other 1o *.a;m 1 x © © figure 7 prototype musical keyboards linear pie hybrid granularity the granularity of the elements relates to the level at which the elements in the hoq are specified or understood. the level of granularity is likely to be affected by many things, e.g. sophistication of the knowledge held by the 28 ajis vol. 7 no. 1 sept 1999 agent of the elements concerned. for example, when a designer views all users as falling into the abstract class 'user' the designer is drawing on an abstract, or high-grain, view of the user. we have found it best to tailor granularity to need. the affinity diagrams produced prior to building the hoq represent information system requirements in a hierarchical form, thereby potentially distinguishing between harmony and compatibility, and facilitate hoq's being built at different levels of abstraction. balance the degree of balance relates to the extent to which each element in the hoq is able to contribute to the functioning of the overall system. ideally, the balance in the hoq should be such that priority is given to critical elements. hence, to reiterate, achieving overall harmony may necessitate the sacrificing of certain compatibility relations. the aim of the hoq is to make explicit the prioritisation process, and to record the sacrifices being made. extensions to the house of quality deriving the dimensions a variety of techniques are available for eliciting and structuring information system requirements and technical responses. qfd's recommendations include (cohen, 1995) one-on-one interviews, contextual inquiry and focus groups with data from these sources being structured using affinity diagrams, tree diagrams and prioritisation matrices. additionally, practitioners can employ task analysis to identify user problems and concerns. alternatively, the rich picture technique from the soft systems methodology (checkland, 1981) can be used to elicit issues that are of concern to user groups, these can then be structured using affinity diagrams. metrication the symbology used to represent the mappings and correlations between the requirements and the responses can be expanded in a variety of ways. when available, benchmarks and criteria can be used to metricate the information system requirements. work in software quality measurement (see akao, 1990, chapt 12) is working towards metrics useful for formalising the technical response. a large number of different symbol systems is used for indicating mappings and correlations, ranging from the graphical notation presented above, through to statistical and mathematical treatments. this will be an ongoing area of work as qfd and hoq are used in information systems development. we have found the graphical notation presented in figure 6 to be satisfactory for supporting the resolution of information systems tensions. modelling actual-present and desired-future akao (1990) presents a hoq expanded to model temporal aspects of the development process. essentially, a hoq is produced that models the actual-present and desired-future situations. in this way ongoing design is given a firm reference point of current problems and concerns. further extensions include calender-dates that relate the future products to present problems and development processes. the next step in manufacturing applications of qfd a series of hoq's are produced (named 'streets'), each feeding into the next. it is possible to conceive of steps in the information systems development process being framed by the drawing of a hoq, and thereby encourage the flow of customer priorities right through the development process. as illustrated in figure 5 above, the first hoq would map information system requirements (the rows or 'whats') onto high level technical concerns (the columns or 'hows'). next the high level technical concerns (e.g. 'headphone') become 'whats' to the new 'hows' of low level solutions (i.e. detailed technical specifications of specific headphone sets). eventually an attempt could be made to map the final product back onto the original requirements. these linked houses would therefore convey the user's voice through the information systems development process. this process would facilitate product evaluation and traceability. additionally, in representing the temporal aspects of the design process the hoq's produced capture the unfolding design rationale. streets are therefore one way of overcoming the limitation on the complexity of what is modeled that is imposed by the simple 2 dimensional matrix (particularly limitations in the scope of the requirements and the granularity of their representationsee above). streets can be constructed that stretch across the development life cycle (as 29 ajis vol. 7 no. 1 sept 1999 discussed) or hierarchically arranged and thus supporting the refinement of requirements at lower and lower levels of abstraction. conclusion this paper has argued that information systems design needs to concern itself not just with 'how' to engineer compatibility between all the elements of the information system, but also, given that relations often compete, 'which1 compatibilities to give priority to. the hoq was introduced as a mechanism that cross-functional design teams could use to represent competing information system requirements and explore the relationships between the voice of the user' and the voice of the developer'. we use the hoq as both an investigation tool and also an analysis approach. we have found that completing the matrix imposes a rigor on the discussion of requirements that adds value to the kinds of collaborative cross-functional sessions we are conducting. we are using the notation throughout the lifecycle and constantly refining our understanding of the user's needs in the light of our growing enlightenment about what is and is not possible, what is and is not wise. we have used it as both a stand-alone technique and in conjunction with scenario based design and the soft systems methodology (howard and dearden, 1998, dearden and howard, 1998). in each case it acts to clarify the relations between the concerns of multiple stakeholder groups and assists us in prioritizing the information systems requirements. in order to manage the complexity of data that can result, software tools are being developed by the author and others (see for example qfd2000 and qfdcapture). the hoq adds to the growing body of participative design techniques being used in information systems (eg. checkland, 1981, avison and wood-harper, 1990). extensions specific to the use of the hoq for modelling information system tensions were discussed. how does the hoq respond to three aspects of wicked problems introduced earlier? • the lack of a definitive formulation: the hoq is used by multi-function design teams to come to agreement on both the relationships between their concerns (e.g, users and designers) and the prioritisation of those concerns. in that sense the hoq can be seen as the team's operationalisation of the design qualities under discussion, including information system requirements. although arriving at a definitive definition of the requirements is beyond the notation, the hoq does at least allow the current working definition of the information system requirements to be articulated and debated. • solutions to information system problems tend to be more or less satisfactory: the hoq directly supports the team in prioritising the customer attributes and technical response. in that sense, both the nature and the degree of the solution's 'satisfactoriness' is articulated and recorded. • the solution space to any information system problem is not exhaustively identifiable: in recording the nformation systems requirements and the technical dimensions of the associated response, the hoq captures the limits of the search conducted during the development of the information system. as with design rationale techniques, the house both encourages a more considered search and captures the extent of the searched space (and by implication, options not considered). references akao, r. 1990, quality function deployment. productivity press, cambridge, mass. al-siuru, p. 1993, using qfd techniques in user interface specification: first experiences. in proceedings of the fifth international conference on human computer interaction. pub: elsevier science. avison, d. and fitzgerald, g. 1988 information systems development: methodologies, techniques and tools. oxford: blackwell scientific publishers. avison, d. and wood-harper, t. 1990 multiview methodology. oxford: blackwell scientific publishers. bias, r.g. and mayhew.d.j. 1994, cost-justifying your usability. boston, academic press. card, s., moran, t. and newell, a. 1983, the psychology of human computer interaction. pub: elsevier science. checkland, p. 1981, systems thinking, systems practice. chichester: john wiley and sons. churchman, c.w. 1978, philosophical speculations on systems design, handbook of operational research lmoder, s. elmaghraby (eds), van nostrand reinhold churchman, c.w., 1971, the design of inquiring systems: basic concepts of systems and organization, basic books inc publishers. cohen, l. 1995, quality function deployment: how to make qfd work for you. addison wesley. dearden, a. and howard, s. 1998, capturing user requirements and priorities for innovative interactive systems. in proceedings of ozcht98, adelaide, nov 1998. 30 ajis vol. 7 no. 1 sept 1999 howard, s. and dearden, a., 1998, innovating usable software: integrating qfd with the soft systems methodology. published in the proceedings of world innovation and strategy conference (wisc'98). 2-5 august, sydney, australia. hauser, j.r. and clausing, d. 1988, the house of quality. harvard business review, may-june, 1988. lundell, j. and williams, d. 1993, integrating qfd into software development. in proceedings of the fifth international conference on human computer interaction. pub: elsevier science. petrie, h., johnson, v., strothotte, t., michel, r., raab, a and reichert, l. 1997, user centred design in the development of a navigational aid for blind travellers. in s.howard, j.hammond and g.lindgaard (eds) human computer interaction, interact97. chapman and hall. qfdcapture, 1999, www http.7/qfdcapture.com/webpage/ (accessed june 8, 1999). qfd2000, 1999, www http://www.users.globalnet.co.uk/-tqs/ (accessed june 8, 1999) rittle, h.w.j. and webber, m.m. 1973, dilemmas in a general theory of planning. in policy sciences, 4, 155169. shindo, h. 1991, quality function deployment applied to software at digital equipment corporation. thayer school of engineering, dartmouth college, may 1991. stowell f. a. and west d., 1989, expert systems: ramifications for the knowledge engineer. in systems analysis, modeling, simulation 16(9). stowell f. a. and west d., 1990, the contribution of systems ideas during the process of knowledge elicitation. in r.l.flood, m.c.jackson and p.keys (eds.) systems prospects the next ten years of systems research, teasley, b. and scholtz, j. 1997, user-centred gui standards design: a case study. in s.howard, j.hammond and g.lindgaard (eds) human computer interaction, interact'97. chapman and hall. vicente, k.j., burns, c.m. and pawlak, w.s. 1997, muddling through wicked design problems. in ergonomics in design, january, 1997. zultner, r. 1992, quality function deployment for software. american programmer magazine, february, 1992. 31 the australasian journal of information systems (ajis) began in 1993 and was based on the simple realisation that there was a need for an academic ‘outlet’ for is research in australia australasian journal of information systems volume 18 number 2 2014 133 exploring the perceived measures of privacy: rfid in public applications mohammad alamgir hossain school of business, north south university dhaka, bangladesh email: mahripon@yahoo.com abstract the purpose of this study is to explore the measures that may protect privacy of the users in the context of rfid use in public applications. more specifically, this study investigates what the users perceive to have securing their privacy, particularly for the rfid applications in public uses. qualitative research approach has been utilised for this study. the author conducted two focus-group discussion sessions and eight in-depth interviews in two countries: one from australasia region (australia) and the other from asia (bangladesh), assuming that the status, and the perceptions and tolerance of the citizens on privacy issues are different in the stated regions. the explored factors have been analysed from privacy perspectives. the findings show that, in developed and developing countries, the basic perceptions of the users on privacy protection are complimentary; however, privacy is a more serious concern in australia than in bangladesh. data analysis proposed some attributes that may improve users’ privacy perceptions when rfid is used in public applications. this study is the single initiative that focuses on privacy of rfid users from national-use context. as practical implication, the proposed attributes can be exercised by the deploying agencies that implement rfid technology for citizens’ use. keywords: rfid; national id; smart-id; privacy; e-passport; qualitative study introduction radio frequency identification (rfid) is an automated data-capturing and data-storing technology. the captured data can be used to identify an object uniquely (rfid journal 2005). in industrial and supply chain applications, rfid technology can trace a product through its entire lifecycle from the production line to all the way to the recycling centre (lin 2009). many countries have developed and implemented rfid-based human identification system that uses the data obtained from this system for various national administrative purposes and/or to provide specific services to the citizens. national identity (id) cards and electronic passports (e-passport) are the main applications that governments are more interested-in because in recent times governments are more serious to combat potential terrorism activities and crimes. in national-level public applications of rfid, the captured data about the citizens is accessed, handled and shared by many departments or authorities. for instance, the proposed (but failed) national identification system of australia “intended that thirteen government agencies would use the australian card” (jordan 2010). consequently, the issue of citizens’ privacy comes as very prominent and thus demands special attention from the deploying authorities to keep the data confidential and inaccessible to any unauthorized use (kelly & erickson 2005; vaudenay 2006). unlike the use of rfid in retail stores where a proper and practical implementation of rfid system does not affect individual customers’ privacy (murray 2003), securing the privacy of the citizens is more sensitive and complex. citizens’ privacy can be abused if the data is accessible to any unauthorized or unlawful person(s) australasian journal of information systems volume 18 number 2 2014 134 and/or if the data is supplied to any unauthorized third party. in fact, several privacy leaking through data-abuse incidents such as supplying the citizens’ information to marketing companies (particularly with malaysia’s mykad) have raised and/or strengthened public concern and perceptions protecting privacy with highest priority. privacy international specifically suggested that china and malaysia need to be serious about privacy; both countries are using rfid in public applications extensively while their privacy ranking is worst in the world (privacy international 2006). although most rfid researchers are obsessed to develop techniques for privacy protection, the actual problem (and hence the potential solution) lies somewhere else (hossain & prybutok 2008). it is generally assumed that in united kingdom “at any one time, one percent of staff will be willing to sell or trade confidential information for personal gain.” (davies 1996). therefore, not surprisingly, there have been instances of forgery and counterfeiting of identity cards, not due to a lack of security features but due to the assistance of corrupted public-officials holding positions of trust in government (thomas 2004). such privacy abuse could harm more especially after the implementation of anti-terrorism laws in several countries. hence, “effective action is needed” so that citizens “can trust that the various applications of rfid are privacyfriendly” (langheinrich 2009). the current study explores the perceptions of the actual users which may protect and/or enhance their privacy stipulation in national applications. this study, therefore, tries to contribute to fill up this research gap by exploring the privacy catalysts of rfid use in national applications from a behavioural study. in order to get a comparative picture, this study explores the perceptions of the users from a developed country (australia) and a developing country (bangladesh). in general, people in australia are more concerned and serious about their privacy. for instance, in 1985 the hawke government proposed for a national system of identification, which was rejected in the 1988 referendum (saunders 2008). again in 2006 (although claimed not as the national identity card), howard government proposed a ‘smart card’ “that would fight welfare cheats, terrorism” but “the scheme failed so quickly” and could not get that much success (saunders 2008). in both the occasions, the main concern was the ‘privacy’. on the contrary, most of the asian countries including bangladesh, where people are believed to be more resilient on privacy, have been using national identity card for generations. this research will explore the factors that the users in these two countries (australia and bangladesh) perceive as important on privacy issue particularly when they are tagged with rfid technology in the forms of public applications. background rfid in brief rfid is one type of automatic identification technology that uses radio wave as the way of communication (angeles 2005). an rfid system consists of tags (transponders), readers (transceiver /interrogators) and a network system (middleware) (finkenzeller, 1999; wu et al. 2006). the tags store data. the readers scan and communicate with tags via electromagnetic wave and transmit the information on the tags to a database or data processing network through the middleware (wu et al. 2006). the middleware manages the data collected from the tags, processes the data, and provides the data in real-time software systems (e.g. erp systems) or to the internet (finkenzeller 2006; wu et al. 2006). rfid tags come in different types according to their functionality: active or passive. active tags have an active radio frequency (rf) transmitter and a built-in battery to power the logic chip and to communicate with readers. on the other hand, passive tags do not have any in-built power source; therefore, they need to get power from readers in order to run the digital logic on the chip and to issue a response to the reader (plos et al. 2012). in this study, we focus only on passive tags due to their substantial usage in public applications and hence possess highest impact on users’ information privacy. australasian journal of information systems volume 18 number 2 2014 135 rfid in public applications an rfid card enables the system to automatically identify the bearer of the card by the means of rf. it is worthwhile to note that ‘automatically’ means the card does not need to be inserted or swiped but is read by an rfid system only when the card comes in a readable distance of a reader. it is not the case that the reader can read the card from nowhere; actually the operating frequency is kept low to restrict reading of the cards from far away. most of the public applications of rfid technology use the proximity cards that would not be read if the tag and reader are far than 10 centimetre. national identity card (hereinafter, smartid4) is the main application of rfid technology among public. malaysia is the first country to introduce rfid in its national identity card (mykad) in 2001 (tedjasaputra 2006; thomas 2004). several other regions including hong kong, estonia, finland, belgium, portugal, spain and recently china and albania issued smart-id to its citizen (abi research 2008; wikipedia 2009). the next major application of rfid in national use is the electronic passport (e-passport); rfid chip is integrated in a passport which stores data such as name, date of birth and address, as well as biometric data like facial recognition (image), fingerprint recognition and iris recognition of the bearer (juels et al. 2005). many countries have already implemented and mandated rfid-based electronic passports while many other countries are in the process. for example, since january 1, 2011 no nigerian without an e-passport is allowed to travel into or out of nigeria (all africa 2010); in 2004, canada mandated its passport issuing authority (passport canada) to adopt e-passport; trinidad and tobago motivates its citizens to have their e-passport by 2017 so that they will “be allowed to enter a foreign country … with the just swipe of a card” (kowlessar 2012). there is a continuous global and regional pressure on the adoption of rfid in passports and identity cards. the international civil aviation organization (icao) has mandated for every traveller with rfid-enabled electronic passport (e-passport) by april 2014 (icao 2009). however, the deadline has been extended to 2017 (kowlessar 2012). most of the countries hence already started their projects for the implementation of e-passport facilities. till march 2012, “approximately 95 countries issued epassports, including all ... g8 nations” (baird 2012, p8). similarly, as a regional pressure, european union (eu) is on its way to implement a globally-unique cross identification process by the means of electronic identification system with the intention of sharing the ids with allied countries for the purposes like interpol investigations or visa-free-entry (e-finland 2004). privacy review and rfid technology ‘privacy’ is considered as a fundamental requirement to any modern democracy. a survey conducted by capgemini (2005) revealed that privacy is perceived as the top concern by the rfid users. rfid (information) privacy issues is well explored in literature; however, the researchers are concerned and concentrate mainly on the technical and technological issues of item-level tagging and privacy issues 4 we should not be confused with rfid and smart card. though both technologies use radio frequency and contactless features, there are technological and operation differences between rfid and smart card. in general, the read range of smart card is up to10 cm which is from 10 cm to 10 m for rfid cards. in this paper, smart-id is defined as the identification by means of rfid technology. for detail, readers are encouraged to read papers including http://www.frost.com/sublib/displaymarket-insight-top.do?id=83467478 australasian journal of information systems volume 18 number 2 2014 136 of the customers in retail stores (e.g., chong & chan 2012; juels 2006; kelly & erickson 2005; peslak 2005). there is scant behavioural literature on the privacy issues of rfid in the context of national id cards, commuting cards, etc. the nature of captured data, data use, and data exposure through an rfid system in a retail chain is very different from that in a public use; in fact, the latter case is more serious as it is vulnerable to civil rights violations (hossain & prybutok 2008; peslak 2005). moreover, customers may reject shopping from an rfid-enabled shop but may not refuse rfid on public use because of government law. for instance, wal-mart “cancelled” its rfid-based ‘smart shelf’ trial (gilbert & shim 2003) “while there have been some complaints about the privacy implications” (masnick 2003), and many customers showed reluctance to shop. a number of privacy controversy in retail sector have been reported by thiesse (2007). on the contrary, it is not reported yet if a citizen refuses rfid-enabled e-passport. privacy is a ‘natural right’ or ‘right to be left alone’. the treaty of the european union states that everyone’s privacy right “shall be no interference by a public authority with the excercise of this right except such as is in accordance with the law and is necessary in a democratic society in the interests of national security, public safety or the economic wellbeing of the country, for the prevention of disorder or crime and freedoms of others’’(european communities 2004). almost similar provisions are made in the u.s. and other countries (us congress1995). therefore, collecting and using users’ data through rfid-use in state applications is lawfully and ethically justified. however, this provision cannot be applied to retailers to dissiminate customers’ personal data. hence, consumer advocacy groups are currently lobbying for privacy legislation regarding use of rfid (whiting 2003). however, they do not object the government use of rfid for national interest (caspian 2003-04), but urge and demand that under no circumstance users’ personal and sensitive information should be misused. in general, personal data means any data (a) relating directly or indirectly to a living individual; (b) from which it is practicable for the identity of the individual to be directly or indirectly ascertained; and (c) in a form in which access to or processing of the data is practicable (hong kong government 2012). and sensitive information means personal data that reveals “racial and ethnic origin, political opinions, religious, philosophical or moral beliefs, labour union membership, and information concerning health conditions or sexual habits or behaviour” (argentine government 2008). hence, the information that is stored by the government agencies through the smart-ids is personal and sensitive. therefore, even for national interest, citizens’ information has to be authentically accessed by the right personnel and lawfully be used for national interest. on the contrary, rfid in a retail chain does not keep too personal data; it finds pattern on customer’s taste, purchase and so on (cazier 2008). therefore, rfid and citizens’ privacy is more serious and requires attention, which is yet well addressed in the literature compared to that of the retail sector. for a detail review on privacy literature, see bélanger & crossler (2011) and pavlou (2011); a brief is presented in table 1. australasian journal of information systems volume 18 number 2 2014 137 reference area brief findings sutanto et al. (2013) smart-phone users provision and then personalization of privacy features are (perceived as) effective securing users’ privacy bélanger & crossler (2011) review on information privacy identified 23 privacy concern including unauthorized secondary access and use, risk, vulnerability pavlou (2011) review on information privacy pointed on the privacy paradox in literature: “an individual’s concerns about information privacy do not necessarily map with the individual’s intentions to share personal information” garfinkel et al. (2005) rfid-related privacy concerns different privacy threat to persons: action threat, association threat, location threat, preference threat, transaction threat, breadcrumb threat smith et al. (1996) information privacy privacy concern in terms of: collection of personal information, improper/unauthorized access, unauthorized secondary use (by the organization itself that collects it, and by the other organizations that collect information from the collector organization), combining data into larger database (mosaic effect), and errors in data table 1 a brief examination on existing privacy studies and major findings irrespective of developing or developed countries there have always been some debates and lack of trust in using a strong technology like rfid which has the capability of tracking a person in real-time. generally, privacy is a big concern especially in developed countries whereas in developing countries it is bit flexible for government-use of citizens’ information. the constitutional bodies (e.g. election commission) in many developing countries (including bangladesh), unlike the developed countries (like australia), are not beyond the control of the government. hence, a corrupted political government may use citizens’ data for their own benefits such as political harassments to opposition parties or to manipulate election result. moreover, discrimination on the basis of religion, past criminal record or medical history is also very prominent in smart-id privacy abuse cases (thomas 2004). furthermore, a citizen/consumer with lower personal tolerance places higher importance and sacrifices less on personal privacy (ohkubo et al. 2005). especially in the developed countries, many of the rfid projects had been cancelled because of the strong protests from the consumer advocacy groups. along with the wal-mart example given earlier, metro ag, the giant retailer in europe “discontinued” its trial with rfid-tagged ‘customer pay-back card’ (metro ag 2004). still we see lots of potential privacy abuse cases from developed countries; for example, the state of illinois receives around $10 million from record selling (kurtz 1998) – the question is: do they include citizens’ personal information? the answer is crucial and remains unanswered. research design for this study, the qualitative method is considered as most appropriate because of the exploratory nature of the research. scholars argue that, understanding a phenomenon from the point of the participants is difficult to achieve when textual data are quantified (kaplan & maxwell 1994). therefore, a ‘pseudo case study’ that involved a qualitative study of a small number of participants would meet the objectives of this study. as such, field study approach has been adopted as the research method (patton 1999; zikmund 2000). moreover, qualitative methods permit the evaluator to study selected issues in depth and in detail. in order to ensure the positivist stand of this research the fieldstudy was performed without being constrained by predetermined outcomes rather relying on openness, australasian journal of information systems volume 18 number 2 2014 138 and detail of qualitative inquiry (patton 1999). correspondingly, eisenhardt (1989) argued that qualitative study is “particularly well suited to new research areas or research areas in which existing theory seems insufficient” (p. 548-549). rfid is considered as the world’s “oldest new technology” (poirier & mccollum 2006, p3); not many studies came up with the exploration of (behaviorial) privacy measures, except a handful research from the technological perspective. therefore, this study used multiple case based field-study approach which is considered as an appropriate research design when the purpose of the research is descriptive, theory building, and practice-based, and where the profound thoughts and experiences of the subjects are important (benbasat et al. 1987). sampling and data collection this study obtained qualitative data from two focus group discussion (fgd) sessions and eight in-depth interviews conducted with rfid users in perth (western australia) and dhaka (bangladesh).fgd in perth and dhaka involved six snd seven discussants respectively. participants were ranged in the age between 18 to 61 years and had almost equal participation from both the genders. each session was conducted by one moderator, which lasted about 70 minutes while the average interview time was around half-an-hour. in addition, eight direct interviews (four in perth and four in dhaka) were conducted to explore users’ insights on this current research agenda. regarding the number of cases to investigate, opinion varies among the researchers. some researchers suggest an open-ended number of cases while others recommend a restricted range while the most appropriate range falls between four and eight (eisenhardt 1989). however, considering the importance of the issue, this study approached to the individuals until reaching the saturation. in both cases, participants were recruited using convenient sampling in order to ensure productive findings. the interviewees have been using at least one or more rfid applications provided by the state. the respondents from perth use smartrider whereas the respondents from dhaka use spass – both are the ticketing cards for public transport commuting service supplied by the western australian public transport authority and bangladesh road transport authority, respectively. at the beginning of each interview the focus groups were given a brief outline of the rfid technology and the research purposes. the participants were informed that they could quit the interview at any time without any prejudice. also they were informed about the usage-policy of captured data from the discussions. in both cases a semi-structured open-ended questionnaire has been used. the respondents were allowed to discuss on the privacy issues related to rfid technology, and were probed when required. to start the discussion, the following questions were asked: a. what is your perception on privacy, related to rfid technology use? b. what features you perceive as useful to maintain privacy on rfid data? reliability was ensured by using the same interview-protocol for each session. the moderator and the interviewer inserted the questions into the discussion/conversation and prompted when needed. with the permission of the participants, the discussions/interviews were recorded. following seidman’s (2005, p. 64) advice, while conducting the interview the interviewer of this study did not take detail written-notes, but only short notes. these ‘working notes’ helped the interviewer to concentrate on participant’s comments. this also helped the interviewer to note a prompt question to be asked at a later stage without interrupting the participants. data analysis to examine and analyse the qualitative data, this study employed ‘content analysis technique’ (siltaoja 2006) and developed relationship among relevant concepts. among “various ways” of content analysis (siltaoja 2006), inductive and deductive analyses were carried out (berg 1989). in the inductive phase; themes, sub-themes, and concepts explaining variables, factors, and, sometimes, measurement scales australasian journal of information systems volume 18 number 2 2014 139 have been explored. in a later stage of inductive phase, the explored factors and variables have been ‘induced’ into a single framework. at this stage, this study followed both ‘theoretical replication’ as well as ‘literal replication’ (chan & ngai, 2007). theoretical replication was made by contrasting cases among respondents, while literal replication was obtained from their similarities. nvivo software from qsr international has been used in this study to analyse data. as claimed by bazeley (2007, p.2), “nvivo has been developed by researchers, with extensive researcher feedback, and is designed to support researchers in the varied ways they work with [qualitative] data.” moreover, nvivo “supports analysis of qualitative data” by managing data, managing idea, permitting data query, developing graphical models, and reporting from data (bazeley 2007; welsh 2002). by coding the discussions/interviews word-by-word using nvivo8, a number of ‘free nodes’ have been developed by naming each segment of data with a label. each ‘free node’ summarized and accounted for each concept about the data. later, ‘tree nodes’ were developed from the free nodes. each tree node became a prospective construct which consists a set of relevant free nodes with similar concept. findings perceived by the respondents, for the success of the rfid, an efficient and flexible privacy mechanism needs to be taken by the respective agencies. the following strategic issues were discussed by the interviewees: explicit consent “the treatment of personal data is unlawful when the data owner has not given his or her express consent which must be given in writing, or through any other similar means, depending on the circumstances” (argentine government 2008). respondents both from australia and bangladesh perceive that explicit consent from the citizen is essential for securing privacy; the clear consent must would state that the “data would not be used in a manner other than it mean to be” (fcd 1&2), and the data owner-should sign the consent form or check the box in online form. they unanimously agreed that once the explicit consent is obtained by an agency from a citizen, the relevant stakeholders must use the data without further consent. for example, permission for accessing the contacts of the residents of an electorate by a politician through the election commission is implicitly given by the citizens. however, the respondents differ with the degree of datasharing among the government agencies. bangladeshi respondents are more liberal and believe that, in order to avail a service, the extent of rights can be relaxed with explicit (or even implicit) consent; that means, every government agency may share data although data may have been obtained by one agency. however, australian respondents are more serious: “it is not automatic to waive the privacy for every agency” when only one agency is granted the permission (respondent 3: australia); data collected from one public application might be used for other public applications too, but the respondents urge that the data-owner should be informed and asked every time whether the authorities can do share. they suspect that government agencies abuse the opt-out model of data acquisition-and-use where the citizens’ information may be distributed till they refuse – “this is a catch”. proving the seriousness of australians to privacy, they demand a quick shift from opt-out to opt-in model that requires citizens’ consent to share their data. the first proposition, hence, is developed as follows: proposition 1. explicit consent from the government agencies will increase the privacy of the citizens; yet, citizens from developed countries perceive that opt-in model is better securing citizens’ privacy whereas citizens from developing countries still are comfortable with opt-out model of privacy. australasian journal of information systems volume 18 number 2 2014 140 detail privacy statement the respondents asserted that the agencies should publish and provide a detail privacy statement while collecting personal information. ideally, the statement should include a clear, conspicuous and detail listing of methods of collecting and using data. when asked about what the things the respondents expect to see on the consent form, at least six items came up as significant which are not exhaustive: (i) why the data will be collected, (ii) how the data will be collected, (iii) how long the data will be kept, (iv) how the data will be used, (v) how the security of the personal data is preserved, and (vi) who do have the access to the data. one of the respondents overtly mentioned about some acts from international arena: the acts under federal privacy act 1983 in canada ensures detail explanation on how data will be collected and how such information can be used, which is further administered and watched-over by an independent commissioner or ombudsperson, with the authority to investigate complaints (checked and clarified from canadian government 2009). moreover, the statement would explain the appropriate security-standards for personal information including establish security systems, conduct security audits and risk assessment, employ ‘watch-dog’ authority and manage security policies properly (floerkemeier et al. 2005). the respondents from bangladesh mentioned that such detail statement is very rare and may develop ambiguity and lack of confidence on government’s use of citizens’ data. in the context of developed countries, detail privacy statement is provided to the citizens. however, respondents from both the countries claimed that detail privacy statements would increase the confidence of the citizens toward securing their privacy. therefore, the proposition becomes: proposition 2. detail privacy statement in government forms will increase the privacy of the citizens which is available in the developed countries but not in the developing countries. legislative protection “a record-keeper who has possession or control of a record that contains personal information shall ensure that the record is protected, by such security safeguards as it is reasonable in the circumstances to take, against loss, against unauthorized access, use, modification or disclosure, and against other misuse” (australian government 2008) the respondents in this study re-established that every country should have legislations against unauthorized access and/or use of personal data, harvested by government agencies. for instance, privacy act 1983 in canada protects the personal information collected by government institutions. “under no circumstance the state should tolerate any information abuse” collected from an rfid system (respondent 2, australia); “lack of legislation is one of the main reasons for privacy abuse with rfid in malaysia” (respondent 1: australia5) . they mentioned that, although some existing privacy laws cover the use of data collected by electronic systems, more direct laws are to be considered dealing with the issues particular to rfid that would secure public data from any source of unrestricted publicaccess. 5 after a long waiting, finally in 2007, a bill was prepared, called as the bill of personal data protection, in reducing the privacy deficiency was finalized to regulate the collection, processing and use of personal data in malaysia (khaw 2002; kettha 2007) australasian journal of information systems volume 18 number 2 2014 141 both fgds revealed that the overall application of privacy law is better in australia than most of the asian countries, which is supported by mcdonagh (2002). the australian respondents believe that legislation against privacy abuse can secure their privacy; in the worst case, they can go to the court and ask for compensation. on the contrary, the bangladeshi respondents claimed that, there is no such legislation in bangladesh which can protect privacy of the citizens while the movement just has been initiated (farjana 2012). even so, the respondents are sceptical about the effectiveness of such law because, in general, the practice of laws is very insignificant (fcd: bangladesh). however, most of the respondents from bangladesh believe that, the citizens should be protected by law regardless. one of the respondents working in academic research recommended that governments should behave “smartly” with handling “a smart technology” like rfid. he appreciated and recommended contemporary laws such as the e-government act 2002 of united sates of america which provides a framework for the agencies to follow assessing the impact on privacy when implementing rfid-like technologies in particular (us congress 2002). hence, it is proposed that: proposition 3. legislative protection, specific to electronic data and data obtained from electronic systems, will increase the privacy of the citizens; yet, legislation is effective and better practiced in developed countries than in developing countries. data-owners’ accessibility the respondents believe that, to their personal information, citizens must have control over the amount of access to the data that the government agencies possess. they believe that citizens, once they have duly evidenced their identity, should have the right to obtain data and request to change the information on their personal data-field. to support their recommendation, it is found that many countries (including argentina and canada) ensure that citizens can access information collected about them, can challenge the accuracy of the information and can request to edit their personal information, held by federal government organizations (canadian government 2009; argentine government 2008). the bangladeshi respondents claimed that, unfortunately, people do not have sufficient access to their data; they cannot upgrade the information on their profile “involves unnecessary hassle – both financially as well as mentally….if you want to take the initiative to let the country know about your current status, it is simply stupidity they [the agencies] will abuse you[r responsible behaviour] (fgd 2)”. therefore, the bangladeshi respondents are cynical about providing their personal information to government – they rather feel comfortable with private agencies. on the contrary, the australian respondents believe that the access on their personal data is more secure and easy to access and modify. yet, they demand that, instead of updating data by different agencies, a replicated distributed database system would be more effective. therefore, the fourth proposition is developed as follows: proposition 4. data owners’ accessibility will increase the privacy of the citizens; yet, citizens from developed countries perceive that they have more control over data than that in developing countries. data authenticity to ensure privacy, it is a fundamental requirement that data should be safeguarded properly – this is more critical for a technological innovation like rfid. it is commonly observed especially in developing countries that data is not technologically secured enough, and hence is a soft target by hackers. more often, citizens’ data are sold to marketing companies and hence violating privacy of the citizens. therefore, the respondents urge that the systems require government-owned and governmentmanaged central cryptographically-secured database, without sharing the information to third-party. respective agencies must take technical and organizational measures to guarantee the security and confidentiality of personal data in order to avoid their alteration, loss, and unauthorized consultation or treatment. moreover, the respondents emphasized that, more importantly, as techniques evolve every australasian journal of information systems volume 18 number 2 2014 142 now and then the agencies should upgrade data authenticity with contemporary measures as well, not just relying on the obsolete techniques. hence, the next proposition can be stated as follows: proposition 5. data authenticity will increase the privacy of the citizens. developing countries have less effective data-authenticity mechanism than in developed countries; yet, agencies need to adopt contemporary mechanisms that are effective to combat hackers. communication channels finally, it is found from the analysis that, the role of communication channels is very important to secure the privacy of the citizens. as a representation of the collective citizens, different advocacy groups can exercise pressures to the agencies as well as conduct privacy awareness programs which would ultimately secure the privacy indirectly. “it is not always possible to raise my [own] voice against privacy because i do not have a platform and [i] might be treated as a member of opponent party … the [representatives of the] civil society should take a leadership role and work as a watch-dog [protesting a privacy violation]” (respondent 4: bangladesh). similar suggestions were proposed by the australian discussants too. they further suggested that the government (agencies) must take initiative to improve the level of public knowledge and understanding about potential privacy issues related to rfid as, in general, there is a negative ‘hype’ about rfid confronting privacy. alternatively, such type of publicity and public-awareness programs would help to the success of this technology as it removes ambiguity among the citizens. moreover, opinion leaders can be engaged for public dialogue in the mass media. finally, technology promoting agencies can contribute to the process. therefore, the roles of the communication channels are twofold: exercising pressure to the agencies to ensuring privacy; and disseminating rfid-knowledge among the citizens. the final proposition, hence, is developed as follows: proposition 6. communication channels may increase the privacy awareness as well as privacy status of the citizens. figure 1 presents the dimensions of privacy in public use of rfid technology, with the propositions developed from the field study. explicit consent detail statement legislative protection data owner’s access data authenticity communication channel rfid privacy figure 1 the proposed model for privacy dimensions in public use of rfid technology table 2 presents the important dimensions of perceived privacy with their respective weights. the weight is presented at percent. in total, 21 respondents were involved in this research that contributed australasian journal of information systems volume 18 number 2 2014 143 to 100%. it is observed that, for instance, 85% of the respondents urge for government to pass laws on citizens’ data privacy and ensure its application . antecedents of privacy % of respondents explicit consent 23 detail privacy statement 57 legislative protection 85 data owners’ accessibility 62 data authenticity 28 communication channels 38 table 2 the variables extracted from the qualitative analysis and respective weight discussion summary of findings one general research question drove this research – what the actual users perceive as the key strategies that can protect and enhance their privacy while they use a technology (i.e., rfid technology) especially which can track and trace-back their movement and so on. this study conducted in-field indepth interviews and focus group discussion sessions in a developing and in a developed country; in general, the results show that the general perceptions of the citizens on data/information privacy are not contradictory rather complimentary. the only difference exists on perception regarding the role and activities of the government regarding privacy protection in respective countries. for instance, bangladeshi people expressed their frustration that government is the supreme authority that could ensure their privacy, but the government itself abuse the citizens’ data frequently. in this case, overall, the citizens are cynical to provide their personal information to the government agencies, and simultaneously are hopeful that different pressure groups would play a significantly important role in protecting individual’s data from government’s abuse. on the contrary, most government agencies in australia are autonomous and are not necessarily under the government control; therefore, data abuse by public agencies is less frequent there. in case of a violation of privacy, there is a good possibility that a citizen may receive justice (e.g. compensation) in austalia which is not guaranteed in bangladesh. briefly, the respondents recommended some fundamental actions that need to be taken while tagging citizens with rfid technology in the form of some public applications including identity card and commuting ticketing system. the very first step to privacy protection, they recommended, is the opt-in approach. in contrast to the (australian) senate committee’s report on information technology of november 2000 (popularly known as cookiemonster? privacy in the information society) (mcdonagh 2002) where the committee recommended for an implicit consent and opt-out approach of data/information collection, the current study argues for explicit and opt-in approach. the opt-out approach of informed consent permits the collection of personal information until the consumer specifically requests that the data not to be collected (laudon & laudon 2012). on the contrary, the opt-in approach prohibits a business from collecting any personal information unless the consumer specifically takes action to approve information collection (laudon & laudon 2012). moreover, while collecting personal data, a detail statement regarding the use and discloser of data should be provided. to be more specific, the privacy commission of australia recommends that “the privacy statement be on the same page as the form or prominently linked to it” (mcdonagh 2002, p335). later on, the data owners should get the unconditional access to control the data (e.g., change, add, delete, modify). recent study by sutanto et al. (2013) finds that the provision and then personalization of privacy australasian journal of information systems volume 18 number 2 2014 144 features increases smart phone adoption; therefore, while collecting data, citizens’ should be offered with provision of privacy features (i.e. detail statement), and the right to personalizing data (i.e. access). to enhance citizen’s privacy, the government can also play an important role by incepting electronic technology use-related privacy laws and also by monitoring privacy parameters. furthermore, data should be secured by implementing contemporary techniques and technologies. finally, opinion leaders and privacy advocacy groups can enhance privacy awareness and exercise pressure on government to enhance privacy probations, and against privacy breaches. similarly, government can use various communication channels to enhance public awareness on privacy issues. implications the effect of perceived privacy on the adoption of rfid technology is well researched, especially in the context of retail customers (e.g., hossain & prybutok 2008). most of these studies are mostly interested in the effect of the construct ‘privacy’ (mostly on adoption) but failed to explore its antecedents. also, the behavioural solutions protecting privacy is comparatively less studied. more glaringly, privacy study in public use of rfid is even least studied whereas rfid has increasingly been adopted in public applications including national identification, passports, or commuting cards. the current study is the only initiative that explored the dimensions for securing privacy in the context of rfid-use in public applications. this is the first of its kind to explore the strategies to protect privacy – suggested by the actual users. consequently, this study developed six contributing factors that enhance the privacy of the users. hence, this study believes to have significant contribution to the existing body of knowledge in privacy and rfid studies. in the context of privacy protection, this study demonstrated a systematic guideline for the government agencies that implement rfid technology in public use. it underscores that, in order to enhance the perceptions of the citizens on privacy, the respective agency must consider the privacy issues of the users seriously. more importantly, this study presents the components of privacy measures so that the agency can check their status and adjust their action. moreover, it suggests that, privacy advocates should monitor the privacy status and exercise pressure on specific concern. similarly, technology manufacturers and vendors may use different communication channels to enhance the privacy perception of the users. limitations and future research directions the main objective of this research was exploratory. in future, the dimensions can be validated with empirical data and may compare the differences in perceptions and their respective effects in developed and developing countries. moreover, this study involved the respondents who use only a single rfidapplication (e.g., ticketing in public commuting service). however, future research would consider simultaneous use of more applications which will provide a more detail picture. furthermore, adapting the traditional behavioural theories and models such as technology acceptance model (tam), future study may develop an adoption-diffusion model investigating the use of rfid in public use; along with other relevant constructs, the direct effect of perceived privacy and the indirect effect of the six dimensions explored in the current study on adoption and perceived trust, for instance, may be investigated. conclusion in many countries, government is the major driving force for rfid adoption and diffusion. simultaneously, the government is supposed to be the largest body to secure the privacy of the rfid users. therefore, a holistic approach to privacy management is necessary for rfid’s success. from a field study, conducted in bangladesh and australia, this paper proposed six dimensions that might australasian journal of information systems volume 18 number 2 2014 145 capture the privacy perceptions of the citizens. the detail nature of the privacy concern explored from the respondents of two different countries and from various backgrounds is the main strength of this study that also provides theoretical and practical implications. references abi research (2008) rfid moving beyond chinese national id program, abi research.http://www.ccidreport.com/market/article/content/3377/200803/178552.htmlaccesse d 04 march 2013. all africa (2010) nigeria: e-passport deadline. http://allafrica.com/stories/201004190825.html accessed 04 march 2013. angeles, r. (2005) rfid technologies: supply chain applications and implementation issues, information systems management 22(1): 51-66. argentine government (2008) argentina: changes to the data protection act. world data protection report. australian government (2008) information privacy principles. http://www.privacy.gov.au/materials/types/infosheets/view/6541#d accessed 04 march 2013. baird, j. (2012) passport canada’s fee-for-service proposal to parliament, ministry of foreign affairs, canada. http://www.pptc.gc.ca/publications/consultations/proposition-eng.pdf accessed 04 march 2013. bazeley, p. (2007) qualitative data analysis with nvivo. sage publications, thousand oaks, ca. bélanger, f. and crossler, r.e. (2011) privacy in the digital age: a review of information privacy research in information systems. mis quarterly 35(4): 1017-1042. benbasat, i., goldstein, d.k. and mead, m. (1987) the case research strategy in studies of information systems. mis quarterly 11(3): 369-386. berg, b. (1989) qualitative research method for the social science. allyn and bacon, boston, usa. chan, s.c.h. and ngai, e.w.t. (2007) a qualitative study of information technology adoption: how ten organizations adopted web-based training. information systems journal 17(3): 289-315. canadian government (2009) privacy legislation in canada. http://www.priv.gc.ca/fsfi/02_05_d_15_e.cfm#contenttop accessed 04 march 2013. capgemini (2005) rfid and consumers. what european consumers think about radio frequency identification and the implications for business. www.capgemini.com/news/2005/capgemini_european_rfid_report.pdf caspian (2003-04) position statement on the use of rfid on consumer products. http://www.spychips.com/jointrfid_position_paper.html accessed 04 march 2013. cazier, j. a., jensen, a. s. and dave, d. s. (2008) the impact of consumer perceptions of information privacy and security risks on the adoption of residual rfid technologies. communications of the association for information systems, 23(14): 235-256. chan, s.c.h. and ngai, e.w.t. (2007) a qualitative study of information technology adoption: how ten organizations adopted web-based training, information systems journal 17(3): 289-315. chong, a. y-l and chan, f. s. (2012). understanding the acceptance of rfid in the healthcare industry: extending the tam model, in decision-making for supply chain integration: decision engineering, volume 1, springer, pp. 105-122. khaw, l.t. (2002) towards a personal data protection regime in malaysia, journal of malaysian and comparative law (11), http://www.commonlii.org/my/journals/jmcl/2002/11.html. australasian journal of information systems volume 18 number 2 2014 146 davies, s. g. (1996) big brother: britain’s web of surveillance and the new technological order, london: pan books. e-finland (2004) pan-european electronic identity being developed, in wide cooperation. egovernment articles. eisenhardt, k.m. (1989) building theories from case study research, the academy of management review 14(4): 532-550. european communities (2004) european convention for the protection of human rights and fundamental freedoms, article 8. farjana (2012) speakers demanded privacy and data protection law in the national convention. http://www.voicebd.org/node/361 accessed 04 march 2013. finkenzeller, k. (1999) rfid handlook: radio-frequency identification fundamentals and applications. john wiley & son, chippenham. floerkemeier, c., schneider, r. and langheinrich, m. (2005) scanning with a purpose – supporting the fair information principles in rfid protocols, in ubiquitous computing systems, berlin: springer / heidelberg, pp. 214-231 garfinkel, s.l., juels, a. and pappu, r. (2005) rfid privacy: an overview of problems and proposed solutions. security & privacy, ieee 3: 34-43. gilbert, a. and shim, r. (2003) wal-mart cancels ‘smart shelf’ trial, in cnet news. http://news.cnet.com/2100-1017_3-1023934.html accessed 04 march 2013. hong kong government (2012) the hong kong personal data (privacy) ordinance 2012. office of the privacy commissioner for personal data. http://www.gld.gov.hk/egazette/pdf/20121627/es12012162718.pdf accessed 04 march 2013. hossain, m. and prybutok, v. (2008) consumer acceptance of rfid technology: an exploratory study, ieee transactions on engineering management 55(2): 316-328. icao (2009) international civil aviation organization. http://www.icao.int/cgi/statesdb4.pl?en accessed 04 march 2013. jordan, r. (2010) identity cards and the access card.parliament of australia. http://www.aph.gov.au/about_parliament/parliamentary_departments/parliamentary_library/ publications_archive/archive/identitycards accessed 04 march 2013. juels, a. (2006) rfid security and privacy: a research survey, ieee journal on selected areas in communications, 24(2):1-19. juels, a., molnar, d. and wagner, d. (2005) security and privacy issues in e-passports, first international conference on security and privacy for emerging areas in communications networks, pp. 74-88: http://www.library.ca.gov/crb/rfidap/docs/juelsetallsecurityandprivacyofe-passports.pdf kaplan, b. and j.a. maxwell. (1994) qualitative research methods for evaluating computer information systems, in evaluating health care information systems: methods and applications, j.g. anderson, c.e. aydin and s.j. jay (eds.), sage, thousand oaks, ca, pp. 45-68. kelly, e. p. and g. s. erickson (2005) rfid tags: commercial applications vs. privacy rights, industrial management + data systems, 105(5/6): 703-713. kettha (2007) ministry finalising draft of personal data protection bill, ministry of energy, kl. kowlessar, g. (2012) warner: e-passports by 2017. http://guardian.co.tt/news/2012-12-06/warner-epassports-2017 accessed 04 march 2013. kurtz, l.a. (1998) the invisible becomes manifest: information privacy in a digital age. washburn law journal 38: 151-174. australasian journal of information systems volume 18 number 2 2014 147 langheinrich, m. (2009) a survey of rfid privacy approaches, personal and ubiquitous computing, 13(6): 413-421. laudon, k. and laudon, j. (2012) management information systems, 12th edition. prentice hall. lin, c.-y., 2009. an empirical study on organizational determinants of rfid adoption in the logistics industry. journal of technology management & innovation 4, 1-7. masnick, m. (2003) wal-mart cancels rfid smart-shelf trial, in techdirt. http://www.techdirt.com/articles/20030709/1138246.shtml accessed 04 march 2013. mcdonagh, m. (2002) e-government in australia: the challenges to privacy of personal information, international journal of law and information technology, 10:(3): 327-343. metro ag (2004) the use of rfid in the future store in rheinberg', http://www.futurestore.org/servlet/pb/menu/1002376_l2/index.html accessed 04 march 2013. moroz (2004) understanding radio frequency identification (rfid). http://www.rfidcanada.com/rfid.html, accessed 04 march 2013. murray, c. j. (2003) privacy concerns mount over retail use of rfid technology, electronic engineering times. http://eetimes.com/electronics-news/4046620/privacy-concerns-mountover-retail-use-of-rfid-technology, accessed 04 march 2013. ohkubo, m., suzuki, k. and kinoshita, s. (2005) rfid privacy issues and technical challenges, acm, 48(9): 66-71. pavlou, p.a. (2011) state of the information privacy literature: where are we now and where should we go, mis quarterly 35(4): 977-988. patton, m. q. (1999) enhancing the quality and credibility of qualitative analysis, health services research 34(5): 1189-1208. peslak, a. r. (2005) an ethical exploration of privacy and radio frequency identification, journal of business ethics 59(4): 327-345. plos, t. aigner, m., baier, t., hutter, m., plos, t. and wenger, e. (2012) semi-passive rfid development platform for implementing and attacking security tags, international journal of rfid security and cryptography (ijrfidsc): (1:1/2): 16-24. poirier, c.c. and mccollum, d. (2006) rfid strategic implementation and roi: a practical roadmap to success, j. ross publishing. privacy international (2006) 2006 international privacy survey: ranking by country, privacy international, http://www.privacyinternational.org/survey/phr2005/aboutphrtable.pdf accessed 04 march 2013. rfid journal (2005) what is rfid? rfid journal, http://www.rfidjournal.com/articles/view?1339 saunders, l. (2008) id cards for australian?,the drum opinion. abc news. http://www.abc.net.au/unleashed/31898.html accessed 04 march 2013. seidman, i. (2005) interviewing as qualitative research, third ed. teachers college press. siltaoja, m.e. (2006) value priorities as combining core factors between csr and reputation – a qualitative study, journal of business ethics 68(1): 91–111. smith, h.j., milberg, s.j. and burke, s.j. (1996) information privacy: measuring individuals' concerns about organizational practices, mis quarterly, 20(2): 167-196. sutanto, j., palme, e., tan, c.-h. and phang, c.w. (2013) addressing the personalization-privacy paradox: an empirical assessment from a field experiment on smartphone users, mis quarterly, 37(4): 1141-1164. tedjasaputra, a. (2006) mykad: technology for whom? http://www.rfid-asia.info/2006/07/mykadtechnology-for-whom.htm accessed 04 march 2013. australasian journal of information systems volume 18 number 2 2014 148 thiesse, f. (2007) rfid, privacy and the perception of risk: a strategic framework. the journal of strategic information systems 16, 214-232. thomas, m. (2004) is malaysia's mykad -the one card to rule them all-the urgent need to develop a proper legal framework for the protection of personal information in malaysia, melbourne university law review, melbourne university. u.s. congress (1995) directive 95/46/ec of the european parliament and of the council, official journal of the european communities. u.s. congress (2002) one hundred seventh congress of the united states of america, edited by u.s congress. washington. vaudenay, s. (2006) rfid privacy based on public-key cryptography, in information security and cryptology – icisc 2006, lecture notes in computer science; springer, 4296: 1-6. violino, b. (2005) what is rfid?, rfid journal. http://www.rfidjournal.com/articles/view?1339 accessed 04 march 2013. welsh, e. (2002) dealing with data: using nvivo in the qualitative data analysis process. forum: qualitative social research 3. whiting, r. (2003) rfid backers, privacy advocates seek common ground, information week (11:17). http://www.informationweek.com/rfid-backers-privacy-advocates-seek-comm/16100902 accessed 04 march 2013. wikipedia (2009) identity document.http://en.wikipedia.org/wiki/identity_document accessed 04 march 2013. wu, n., nystrom, m., lin, t. and yu, h. (2006) challenges to global rfid adoption. technovation 26(12): 1317-1323. zikmund, w. g., babin, b., carr, j. c. and griffin, m. (2010) business research methods. eighth edition. cengage learning: canada. a review of object oriented database concepts and their implementation. narciso cerpa school of computer science and engineering faculty of engineering university of new south wales p.o. box 1, kensington sydney, nsw australia roy dean school of information systems faculty of commerce and economics abstract object oriented design and databases has attracted a great deal of attention in recent years. this article outlines and discusses the semantic data principles used inter alia in understanding object oriented concepts. to illustrate and lend substance to this discussion a list is presented of oodbms implementations. their weaknesses and strengths are analysed. and their suitability for specific applications is assessed. finally we offer some conclusions about research in this area and the directions in which further development should proceed. introduction there is a twofold purpose to this paper. firstly to present an insight into object-oriented database technology in both research and commercial arena. secondly, to provide a view of object-oriented database management systems (oodbmss). the range and breadth of industrial and commercial needs today are beyond the technological grasp of the more traditional data base management systems. most of these older systems were developed for commercial applications, traditionally in accounting, administration, resource exploration and financial support systems. today's applications are wider in scope. these newer applications include computer aided design (cad), computer aided software engineering (case) and hypertext applications. they have specific characteristics and requirements and they require a different dbms with capabilities honed to their needs. there have been a number of approaches to this problem. for example stonebraker, et al (1990) in their third generation manifesto of dbms attributes, suggest a darwinian view. they opine that the contemporary evolution of relational technology provides equally efficient solutions. consequently they propose a contemporary view of relational technology as an alternative to the "newer" generation of dbms. others refute this proposition. khoshafian et al (1986), jackson (1991) and garvey et al (1989) all argue that even though some extensions have been made to the relational model,(to overcome some of its limitations) more flexibility is still needed. they propose the object-oriented dbms approach as an alternative to contemporary relational technology. because object-oriented database technology is based on the object-oriented programming languages, some of the language and its terminology, is presented within this paper. this paper also highlights the importance of oodbms components, analyses some of the oodbmss currently available and presents an overview of their implementation. the expectations of oodbmss in the commercial marketplace, highlighting the areas which require further research, are discussed. in addition some major problems in oodbms technology are presented. finally, some conclusions on the results from the last decade of research on oodbmss are given, outlining the achievements and presenting directions for the future. september 1993 13 the object-oriented approach the newer applications require: • a flexible abstract data typing approach • the capability to encapsulate data and • the operations on this data by using the message metaphor. a combination of the object-oriented capabilities with the traditional storage management functions, will result in a system which keeps the capabilities of the traditional approach, but includes an extensible data typing facility. this facility enables the storing of information usually not suited for normalised relations (maier et al, (1986)). in object-oriented systems, all conceptual entities are modelled as objects. a simple integer or string is as much an object as is a complex assembly of parts. an object has value and/or attributes. more complex objects contain instance variables. the behaviour of an object is encapsulated (caught) in methods (processes). (banerjee, et al, 1987a). according some researchers and practitioners, (for example khoshafian et al (1986), jackson (1991) and garvey et al (1989)) an oodbms should fulfil all the functions which are expected of data base systems, plus others features which are typical in object-orientism. these can be described as follows: object identity: for an object to exist in its own right it requires the characteristic of uniqueness. in oodbms terms uniqueness is called identity. khoshafian et al (1986), jackson (1991) and garvey et al (1989) all say that identity is that property of an object which distinguishes each object from all others. when real world objects are modelled, it is usually with some particular purpose in mind. a subset of that object's description relevant to the purposes of the model is included in the model this subset is not always complete. to reflect the object's uniqueness sometimes artificial identifier is required. the relational model introduces the notion of user-defined identifier keys to represent the identity of an object. these researchers suggested implementing the object identity by a substitution method called surrogates. in this way the unique identifiers are generated. each system has its own set of surrogates. surrogates are completely independent of any physical location. each object of any type is associated with a surrogate at the moment it is instantiated. this association will internally represent the identity of its object throughout the lifetime of the objecifrom creation to deletion. inheritance:according to paton, et al (1991) inheritance can be supported in three ways. the first kind of inheritance support, occurs when the structure of a superclass is inherited by its subclasses. the second kind occurs when the behaviour is inherited. the third kind is a combination of both. in object-oriented data bases, inheritance is possible by the modelling of is_a relationships which are implemented by the oodbms. (for a more extensive coverage of semantic data modelling see elmasri and navathe (1989), hull and king (1987)). inheritance reduces the need to specify redundant information, and simplifies updating. (garvey, et al (1989)) information about many object instances can be changed in a single update action. abstract data types are defined by garvey, et al (1989) as data types which allow users to define new data structures. the object-oriented approach enables objects to possess attributes which are themselves objects (jackson (1991)). this is achieved by using the adt facility and building objects from existing data structures. adts provide a set of methods which may be used to manipulate the attributes of the object, but hiding the physical storage representation of the information. method and message: a message is defined as the way in which objects communicate and perform all their computations (garvey, et al (1989)). a message can be divided into the object, the method and the list of arguments it represents the interaction between objects, message can also be a request for an object to carry out one of its operations. message sending supports data abstraction. the calling program does not make any assumptions about the implementation and data type used. 14 ajis overloading: occurs when the same operator can be used for the same operations on different data types. (garvey, et al (1989)) so, distinct methods can be given the same name for two different classes of objects. overloading is also related to the term polymorphism described above. late binding: is a run-time interpretation of message passing, also called dynamic binding (thomas (1989)). the language is interpreted, instead of compiled. (garvey et al, (1989)) this allows features to be added to a method without affecting existing programs. the "how to do it" question, is resolved at run-time. encapsulation an object consists of an encapsulated representation or state and a set of messages which can be applied to that object (thomas (1989)). this concept is also called data abstraction or modular programming. the data is packaged together with the procedures for accessing the data. this data can only be accessed by using the methods or messages provided by the implementer (zdonik, et al (1990)). complex/composite objects: are built by applying constructors to basic objects (jackson (1991)). oxborrow, et al (1991) state that composite objects represent one kind of complex object whereas kim, et al (1987) define a composite object as an object with a hierarchy of exclusive component objects. the instances which constitute a composite object belong to classes that are also organised in a hierarchy. this hierarchical collection of classes is called a composite object schema. composite objects augment the semantic integrity of the objectoriented data base model through the notion of dependent objects. to implement a complex object, requires clustering and locking support. object oriented summary a definition of oodbmss was presented by atkinson et al (1990) at the first international conference on deductive and object-oriented data bases. this definition included the features mentioned by researchers and practitioners as discussed above. to this definition atkinson et al (1990) add the concept that the data manipulation language of the data base should allow the user to express any computable function. this is in sharp contrast with the current sql approach in relational data bases where computational characteristics are very limited. other functions, which are described by some researchers include: • the graphical interface for supporting the interaction with complex objects structures, • and a version schema feature. technical functionalities for oodbms implementations there are technical functionalities which are critical to the success of all dbmss including oodbms. these functionalities include: persistence: persistent data is described by khoshafian (1990) as data which is stored outside a transaction context and so survives transaction updates. there are two principal strategies to determine whether objects should become persistent. these are persistence extensions and persistence through accessibility. the former incorporates the notion of a class extension to make the instances of a class persistent. the latter enables one or more persistent data base roots and every object, reachable from these roots, persistent. transaction management and concurrency control: garza et al (1988) present a good analysis of transaction management. they show how the transaction subsystem provides a concurrency control mechanism to allow interleaved execution of multiple concurrent transactions (serializability). the two-phase locking protocol is used. the granules for locking are the data base, the index, the class, and the instance levels. composite object locking is used for defining and manipulating a set of objects as a single entity. it also delivers semantic integrity. to achieve class lattice (network), and composite september 1993 is object locking, new lock modes have been added to the conventional compatibility matrix for granularity locking. the aim is to increase efficiency of storage and retrieval functions. recovery: garza et al (1988) describe a recovery process which supports recovery only from soft crashes (leaving the content of the disk intact). when this occurs user-initiated transactions abort. this does not support recovery from disk head crashes. the implemented approach uses the undo from log procedure. this procedure requires that pages containing updated objects are forced to disk at the end of a transaction. security: is both mandatory and vital in oodbmss. (thuraisingham (1989)) it is largely based on subjects (processes) and entities (objects) with assigned security levels. the following rules apply: . a subject has read access to any entity if the subject's security level dominates the security level of the entity. . a subject has write access to an entity, if the subject's security level equals the security level of the entity. . a subject can execute a method if the subject's security level dominates both the security level of the method and the type on which the method is defined. . a method executes at the security level of the subject who initiated the execution. . during the execution of a method ml, if another method m2, is executed, then m2 can execute only if the execution level of ml dominates both the security level of m2 and the security level of the type on which m2 is defined. . if a new object is created as a result of executing a method, the object is created at the security level of the subject who initiated the execution of the method. this security system is the multi-level security system on which the subjects are assigned a level of security (e.g. unclassified < confidentiak secret < topsecret) and the security is enforced based on the level of security of the subject and the object as created or accessed. data access performance effective clustering can have a significant impact on performance particularly when accessing data. chang et al (1989) discuss effective clustering in an oodbms based on a prototype object-oriented data manager developed by the uc-berkeley cad group. they propose using inheritance and structure semantics for achieving such improved performance, in the same way that the relational query optimiser uses the table cardinality and indexing information for producing efficient access plans. they also propose a run-time clustering algorithm, which, under high structure density and large read/write ratio conditions, can greatly improve system response time. a simulation study conducted by cheng et al (1991) showed an application where a high read/write ratio justified a fully dynamic reclustering strategy. conversely, when the ratio is not high enough, the reorganisation overhead degrades the overall performance. performance enhancement by using data replication has been proposed by shekita, et al (1989). here replicated data eliminates the use of functional joins that would otherwise be required during query processing. this technique is also called, field replication, and this should only be specified on reference paths which are frequently accessed although infrequently updated. two strategies are proposed. they are: • in-place replication, where the replicated values are stored directly in the objects that cause replication to take place, and • separate replication when replicated values are stored in different objects that are shared within the set. one of the issues is the propagation of updates for both strategies (in the same way as replication in the relational model). kemper, et al (1990) propose creating redundant, and separate structures (access 16 ajis support relations) to store frequently traversed object references in data base queries. this access support relations technique is combined with a query optimiser to transform queries into expressions. this addition is needed to take full advantage of access support relations. query facility a proposed query model (kim 1989) restricts the query target to a single class or a class hierarchy rooted at that class. this model excludes operations comparable to relational joins and set operations. it reflects the semantics of the class hierarchy. in an object-oriented data base, the operand and the result of the set operation may be a heterogeneous set of instances. developments in object-oriented dbmss there are a number of implementations of oodbms with both research and commercial applications. each implementation differs. this is mainly due to the differences between the ool which forms their genesis. a list of oodbmss follows with a short discussion outlining their weaknesses and strengths. the list is largely but not exclusively chronological in development. languages and oodbmss named in this list are proprietary. the characteristics of each item on the list correspond to the date of the research conducted by the cited researchers. gemstone this data base system resulted from a project where the objective was to merge object-oriented language concepts with data base systems concepts. (described in maier, et al (1986)). gemstone was mainly based on the smalltalk-80 language, and its data definition and manipulation language, called opal, was also based on smalltalk-80 capabilities and features. gemstone attempts to permit variations in structured objects, arbitrary data items as values and modification of data base schemes without physical data base restructuring. gemstone also provides a mechanism for accessing an entity as a unit. gemstone provides most of the object-oriented features, but not totally. the implementation of opal as the data definition language for gemstone, had as an objective to implement a computationally complete data manipulation language. all data base access and operations use only opal. according to dittrich (1986) there are three levels of object-orientism definition, namely structural, operational, and behavioural. gemstone supports complex objects, and allows the definition of abstract data types. therefore it is labelled as having behavioural object-orientism. however it does not supply multiple inheritance. the gemstone architecture, presented by maier, et al (1986), provides most of the features of data base systems, such as secondary storage management, concurrency control, authorisation, transactions, recovery, and support for associative access. concurrency is supported by providing each user with a workspace area. this area contains a shadow copy of the object derived from the most recently committed object table. the concurrency control schema used, is the optimistic schema, in which conflicts are checked at commit time, rather than prevented from occurring through locking. this schema ensures that read-only transactions never conflict with other transactions. the recovery process is based on the use of the shadow and shared copy, not making use of logging files, but the important part of recovery is the garbage collection, when removing detritus of the transactions which had not committed before the crash. the authorisation access and ownership are based on segments, which are the areas assigned to users for the creation of new objects. a user can grant read or write permission on a segment to other users, being the grantor (i.e. the original owner). postgres postgres is described and defined by rowe, et al (1987), as a relational model extended to include features such as abstract data types, relation attribute of type procedure, and attribute and procedure inheritance. these features can be used to simulate object-oriented modelling constructs such as aggregation and generalisation, complex objects with shared subobjects, and attributes which are able to reference tuples in other relations. postgres has a relational database, taking the best features and september 1993 17 concepts from the relational approach (in this specific case, from ingres) it includes new objectives to add semantic capabilities to the relational model. postgres provides better support for complex objects, extensibility for data type, operators and access methods. it also gives facilities for active data bases and inferencing, providing support for forward and backward chaining.this data base system provides a collection of atomic and structured types, where all atomic data types are defined to the system as adts. these adts are defined by specifying their names, lengths, input and output procedures, as well as their default values. the input and output procedures are written in a conventional programming language such as c. although this adt mechanism is provided by postgres, it is limited in comparison with some implementation of abstract data types in some object-oriented programming languages. there is no inheritance mechanism for adts in postgres. based on dittrich (1986) object orientism definition, postgres has some elements of both structural and operational object-orientism; particularly in the way that it implements adts and complex objects. however these characteristics are not combined. its features agree with the third generation manifesto of stonebraker, et al (1990). the postgres architecture supports transaction management, but does not use the conventional write ahead log. (stonebraker (1987)). the main characteristics of the approach used, are a collection of modules which provide transaction management. no recovery code is used. all updates are turned into insertions, and take advantage of specialised hardware. it is assumed that non-volatile main memory exists in some reasonable quantity concurrency control is implemented by the conventional two-phase locking protocol and using a main memory lock table. postgres also keeps archival records on an archival medium by having an archival storage system compatible with worm (write once read many) devices. postgres provides housekeeping to maintain the archive system free of invalid records. while postgres is not classified as an oodbms by its creators. however it is in this analysis for its capabilities in simulating some semantic and object-oriented features. all the features of postgres agree more with the third generation manifesto, than with the object-oriented approach. but its variety of features could classify it as a very interesting multi-featured data base system. iris iris is a research prototype. its main purpose is to meet the needs of current data base applications (e.g. case tools, cad, etc) (fishman, et al (1987)). iris provides a different facility to other oodbmss. it was designed to be accessible from any number of programming languages. two lexically oriented interfaces are also supported. they are osql, an object-oriented extension to sql, and inspector, an extension of a lisp structure browser. the iris object manager provides the support for schema definition, data manipulation, and query processing. the data model is mainly based on constructs such as objects, types, and operations, providing support for inheritance, constraints, complex objects or normalised data, user defined operations, version management, inference and extensible data types. iris architecture is built over conventional relational storage manager, supporting transactions, concurrency control, logging and recovery, archiving, indexing and buffer management. the transaction management is based on the conventional two-phase locking protocol. research is being conducted to improve concurrency, to provide prolonged access and to manipulate data base objects. vbase vbase is a commercially available data base system. it was built with a schema definition language concept where objects are defined as data types. these data types have attributes also defined as being of a specific type (andrews et al (1987)). the main purposes of vbase are to combine a procedural language with support for persistent objects. it also has the strong typing inherent in object systems for both language and data base. vbase has the most important influences from clu, a programming language developed at mtt. vbase is based on the abstract data typing paradigm, rather than the object/message paradigm. in vbase object behaviour is required by a combination of properties representing static behaviour, and operations representing dynamic behaviour. the vbase architecture is based on a four layer approach, namely language layer, abstraction layer, representation layer, and storage layer. the language layer consists of the compilers for defining and implementing the behaviour of objects. the abstraction layer provides support for inheritance, 18 ajis operation, dispatching, method combination, and property manipulation. the representation layer is the basis for the reference semantics. the storage layer is responsible for the management of objects and their persistence. vbase provides features such as clustering objects on disk and in memory, triggers implementation, customised types, but very little time is given in this paper by the authors, to data base features, such as transaction management, concurrency, recovery, and others. orion orion is a prototype object-oriented data base system, used for supporting the data management needs of proteus (an expert system). there are two versions of orion, a single-user and multi-task system called orion-1, and a multi-user and multi-task system called orion-ls, in which a single server provides persistent object management on behalf of several workstations. orion supports functions such as version and change notification, composite objects, dynamic schema evolution, transaction management, associative queries, and multimedia data management. orion has been implemented in common lisp, to produce the integration of a programming language with a data base system. the orion architecture consists of: . a message handler receiving all messages sent to orion objects, . an object subsystem providing high-level functions, schema evolution, version control, query optimisation, and multimedia information management, . a transaction subsystem providing concurrency control and recovery mechanism to protect data base integrity, while allowing the concurrent execution of multiple transactions. the conventional two-phase locking protocol is used for concurrency control. a logging mechanism is used for recovery from system crashes and aborts. . a storage subsystem providing access to objects on permanent storage (disks). it also finds and places objects on pages and also manages the movement of pages to and from permanent storage. encore encore is described by hornick (1987) as a data base system in which all objects are considered instances of some type which describe the behaviour of its instances. types, operations, and properties are all objects in their own right and as such have a type that describe their behaviour. types can be related to each other by means of the property is_a, which induces an inheritance relationship between types. encore supports multiple inheritance. operations are active objects which are supported by programming code, and operation types correspond to a procedure definition, while the instances of operation types correspond to procedure activations. properties are objects used to relate other objects. these properties are called association in semantic terms, and can also be used for expressing part-of relationships when modelling composite objects. encore also allows dealing with changes, by providing a version control mechanism. other implementations o2 is based on the framework of a set-and-tuple data model. o2 was designed with the purpose of integrating data base technology with the object-oriented approach, all in one system (lecluse, et al, 1988). this language is also data model where objects may have a tuple or set structure, or be atomic. these objects form a directed graph with potential cycles. consistent sets of objects are used to interpret domains for type structures and method signatures. types consist of a type structure and set of methods. cactis was built from scratch. the focus is on deriving values from both attributes within the object itself and in other objects (heintz (1991)). relationships are defined to allow accessing of information external to the object. cactis does not support abstract data typing, except when an object refers to others through a relationship link. this system supports the specification of constraints and class/subclass hierarchies, also given capabilities for rolling back and recovering the effect of a change. september 1993 19 development summary most of the implementations of oodbmss discussed in this section, try to meet in some way the object-orientism definition of dittrich (1986). they do this by implementing different aspects of the object-oriented approach. varietal approaches are heavily influenced by a specific language and/or the base framework taken for implementation. there are major differences in the physical implementation of each model as well as in the number of features implemented. this varietal approach is further compounded by major differences adopted by different researchers and practitioners. some emphasise the abstract data type paradigm, while others emphasise the message and method paradigm. at this time there is no clear path for researcher or practitioner. the expectations of object-oriented dbmss from this discussion we can draw expectations of the oodbmss. the aim is to make them more powerful than the current relational model, for certain specific applications. the quest is not for a universal superiority. nevertheless oodbms need a strongly developed foundation if they are to succeed in the commercial arena. from the implementations described in these preceding pages, we believe that some of the issues which require solutions are: . rationalisation of object-query languages and programming languages . development of theory to support oodbmss . include techniques from deductive and /or semantic data base technology . standardisation of implementation approaches. . keep or improve features of current data base technology it is also expected that oodbmss will be a good alternative to applications which require management of unformatted data such as image, picture, and voice, as well as alphanumeric data. in these applications, the implementation of the relation model is not feasible. object-oriented databases can satisfy the needs of users whose applications have requirements such as a variety of data types and type constructors, modelling accuracy, derived data, set value attributes, and the ability to model actions. current researchers in knowledge data base management systems and intelligent databases often include oodbms as a subset. these kinds of databases will represent a new technology for information systems management. they are the result of the integration of traditional approaches to data base systems, with new developments, such as object-oriented concepts, expert systems, hypermedia and information retrieval. although some research has been done on query model, languages, and optimisation, there are still problems which must be resolved before oodbmss can be commercially competitive. the current problems in this area, are related to the complexity of the data structure in object-oriented data bases, and to the lack of an equivalent of the simple and concise relational algebra used in the relational model. the interfaces used by dbmss in general lack friendliness and quality. most of the work on user interfaces has been related to specific implementations, or specific kinds of systems such as decision support systems, window-based systems, etc, but the user interface topic, deserves to be treated as a separate technology. in the case of oodbmss the user interface plays an important role when representing complex objects, by using graphical capabilities or hierarchical structures. other problems such as the representation of multimedia objects, manipulation of large objects and handling of active objects are also a concern in oodbmss user interfaces. we believe that most methodologies for designing data bases cannot cope with object-oriented data base design. there are some object-oriented software design methodologies extant, but they include very little about object-oriented data base design. computer aided software engineering tools in the same way as they exist for current data base models is a reasonable expectation. the class hierarchy and class composition hierarchies can make the logical and physical data base design very difficult. the three level ansi/sparc architecture presented in the iso standards, describes the view mechanism as a powerful tool in data base design (ansi/x3/sparc (1978)). the current implementation of oodbmss does not support a mechanism for views, because of the lack of a complete query language, and the difficulty presented by the object identity concept. views should have new object identifiers, but it is not clear if they should be persistent or transient. there 20 ajis are other problems with views, even in current data base systems implementations (relational), but the powerful use of them made in the relational model indicate that they are necessary and that some research in this area is required. most of the current implementations of an oodbms, are based on specific object-oriented programming languages (these languages in some cases are not compatible). this is very important, because a standardisation of the programming languages, could provide interoperability. this requires a common notion of object-oriented semantics, in aspects such as type, computation, query, identity and inheritance. the performance issue has been a problem with the transition from one data base technology to the next, because making the programmer's job easier has resulted in performance cost this problem was created with the transition to the second generation of data base systems (relational), because the use of declarative queries implies the need for some sort of system or expert system, to optimise the access path to the data base. object-oriented data base research will also have to keep up to date with new researches in technology for relational data bases. this will include technology used by distributed databases, tools for data base design, etc. these new findings will require adoption to the object-oriented data base approach. based on the literature review, as well as from practical experience in current data base environments (hierarchical, network, and relational), the issues shown above are the main areas for research in object-oriented databases. the basic building blocks are in place but there are still some important areas which require improvement. conclusions object-oriented data bases has become a major area of research. early research results pointed the way to serious scholarship and experimentation. from these activities the attention of practitioners and researchers has become more focused. data base technology has carried techniques from previous data base generations, such as concurrency control, recovery techniques, storage management, distributed data, and query optimisation. these techniques have been reused and improved, with the purpose of implementation in the object-oriented model. the development of new technology to support object-oriented databases, is also an important outcome of the research in this field. technologies, such as indexing techniques, clustering techniques, schema modification, performance metrics, client/server architecture, etc., are in part, results of some research in objectoriented databases. there are several commercial implementations, research prototypes, which were described in this review, and which were built with the purpose of experimentation and for evaluating the performance and functionalities of these object-oriented data base implementations, as well as creating an alternative to conventional dbmss. important definitions, such as the object-orientism by dittrich (1986) , and the manifesto for object-oriented data bases by atkinson, et al (1989) have set up touchstones. greater formalisation of these standards at the level of international committees such as iso task groups, will be required for making oodbmss the next generation of data base management systems. from the implementations described in this paper, gemstone, ontos (successor of vbase), and orion are oodbmss which have received better reception from practitioners. these products have reached a level of maturity which make them very tempting in the marketplace. another issue is the need for migration paths from current data base systems to object-oriented data bases. this will be essential for the success of oodbmss in the commercial market. there is also a need for oodbmss to participate in heterogeneous distributed data base systems. this could imply that organisations, use different kind of dbmss for different purposes (e.g. relational for current or standard applications, object-oriented for design databases, etc). thus the migration path becomes easier and more reliable. we draw the conclusion that research on object-oriented databases, has been highly successful, contributing new technology for information systems, and improving current technology. our understanding of systems development and data storage is constantly undergoing change. oodbms are contributing to that change. these newer contributions are driven by commercial needs and research findings. this heady mixture fuels the ongoing debate and fires a high level of expectations for the next generation of dbms s. september 1993 21 references andrews t. & harris c., (1987) combining language and database advances in an object-oriented development environment. proceedings of international conference on object-oriented programming systems, languages, and applications, orlando, florida, 1987. sigplan notices. 1987, vol 22, no 12,430-440. ansi/x3/sparc study group data base management systems, "framework report on data base management systems" afips press, montvale nj, 1978 atkinson m., bancilhon f. & de witt d. (1990) the object oriented database system manifesto. proceedings first conference deductive and object-oriented databases. kyoto, japan. chang e.e. & katz r.h. (1989) exploiting inheritance and structure semantics for effective clustering and buffering in an object-oriented dbms. proceedings of international conference on management of data, sigmod record., vol 18, no 2, june, 348-357. cheng j r. & hurson a r. (1991) effective clustering of complex objects in object-oriented databases. proceedings international conference on management of data. sigmod record.vol 20, no 2, june, 22-31. dittrich k. (1986) object-oriented database systems: the notions and the issues. proceedings of international workshop on object-oriented database systems ieee, asilomar, ca, elmasri r. & navathe s. (1989) fundamentals of database systems. sydney : the benjamin/cummings publishing company, inc. fishman d., beech d., gate h p, chow e c, connors t., davis j w., derret n., hoch c g., kent w., lyngbaek p., mahbod b., neimat m a., ryan t a., and shan m c. (1987) iris: an object-oriented database management system. acm transactions on office information systems.vol 5, no 1, january, 48-69. garvey m a and jackson m s, (1989) introduction to object-oriented databases. information and software technology. vol 31, no 10, december, 521-527. garza j f. and kim w., (1988) transaction management in an object-oriented database system. proceedings international conference on management of data. sigmod record, 37-45. heintz t j, (1991) object-oriented databases and their impact on future business applications. information & management. vol 20,95-103. hornick m f. and zdonik s b., (1987) a shared, segmented memory system for an object-oriented database. acm transactions office information systems. vol 5, no 1, january, 70-95. hull r. and king r., (1987) semantic database modeling: survey, applications, and research issues. acm computer surveys, vol 19, no 3, september, 201-255. jackson m s. (1991) tutorial on object-oriented databases. information and software technology. vol 33, no 1, january, 4-12. kemper a. and moerkotte g., (1990) access support in object bases. proceedings international conference of management data. sigmod record vol 19, no 2, june, 364-374. khoshafian s., (1990) insight into object-oriented databases. information and software technology. vol 32, no 4, may, 274-289. khoshafian s. and copeland g., (1986) object identity. proceedings of international conference on object-oriented programming systems, languages, and applications. portland, september. sigplan notices. 1986, vol 21, no 11, november, 406^16. kim w., (1989) a model of queries for object-oriented databases. proceedings of the fifteenth international conference on very large data bases, amsterdam, the netherlands, 423432. kim w., banerjee j., chou h., garza j f., and woelk d., (1987) composite object support in an object-oriented database system. proceedings of international conference on objectoriented programming systems, languages, and applications. orlando, florida, sigplan notices. vol 22, no 12, december, 118125. lecluse c., richard p., and velez f., (1988) 02, an object-oriented data model. proceedings of international conference on management of data. sigmod record. 424-433. maier d., stein j., otis a., and purdy a., (1986) development of an object-oriented dbms. proceedings of object-oriented programming systems, languages, and applications, portland, sigplan notices. vol 21, no 11, november, 472-482. 22 ajis oxborrow e., davy m., kemp z., linington p., & thearle r., (1991) object-oriented data management in specialised environments. information and software technology. vol 33, no 1, january, 22-30. paton n.w., and diaz o., (1991) object-oriented databases and frame-based systems : comparison. information and software technology. vol 33, no 5, june, 357-365. rowe l. and stonebraker m. (1987) the postgres data model. proceedings of the thirteenth conference on very large data bases, brighton, england, 83-95. shekita e. j. & carey m. j., (1989) performance enhancement through replication in an objectoriented dbms. proceedings of international conference on the management of data. sigmod record. vol 18, no 2, june, 325-336. stonebraker m. r. (1987) the design of the postgres storage system. proceedings of thirteenth conference on very large data bases. brighton, england. 289-300. stonebraker m. rowe l. a., lindsay b., gray j., carey m., brodie m., bernstein p., and beech d. (1990) third-generation database system manifesto. sigmod record vol 19, no 3, september, 31-44. thomas, d., (1989) what's in an object. byte. march, 231-240. thuraisingham m. b., (1989) mandatory and discretionary security issues in object oriented database systems. proceedings of international conference on object-oriented programming systems, languages, and applications, new orleans. 203-210. zdonik s. and maier d., (1990) fundamentals of object-oriented databases. readings in object oriented database systems. morgan-kaufman, san mateo, ca september 1993 23 forecasting demand using survival modeling : an application to us prisons joanna r. baker department of information and decision sciences james madison university harrisonburg, virginia 22807 usa pamela k. lattimore* national institute of justice us department of justice washington dc 20531 usa abstract a systems approach to modeling demand which incorporates survival modeling is applied to the problem of prison population projection. the approach models the flow of inmates through the prison system and differs from earlier approaches by exploiting the differences in the incarceration hazard rates of individuals in the general population and those who have previously been incarcerated and explicitly considering the impact of constrained prison capacity on release policy and future admissions. the methodology capitalizes on the impact of recursion in the prison population and reduces the amount and complexity of data required for long-term forecasts.. first-time arrivals to prison are modeled as a poisson process arising from the general population; recidivist arrivals are modeled using a failure model, where the reincarceradon hazard rate is a function of age and race. the model is demonstrated for the state of north carolina located in the southeastern region of the united states. the effect of limited prison capacity on the mean of the time-served distribution is shown. the results demonstrate that an early release policy will generate an increase in prison admissions through the return to prison of former inmates. further, the results show that a systems approach to modeling of prison demand which includes the non-linear effect of recidivism, i.e., survival modeling, has a significant impact on the accuracy of forecasts. introduction prison crowding is one of the most serious domestic issues currently being faced by the united states. between 1980 and 1988, the number of inmates in state and federal prisons increased more than 90 percent, from 329,821 to 627,402 (us department of justice, bureau of justice statistics, 1989). at the end of 1992,43 state jurisdictions and the federal prison system were operating at 100 percent or more of their prison capacities (us department of justice, bureau of statistics, 1993, p. 6). recently, "get tough" sentencing policies—including "three strikes and you're out" provisions have been enacted in several states. these policies imply that if an offender is found guilt of a violent crime three times, they will receive a mandatory sentence of life imprisonment. such policies, although popular with a voters frustrated by the escalating nature of crime in the us, will only exacerbate the current crowding situation. for example, the state of california's prison population was 115,534 on june 30, 1993 (us department of justice, bureau of justice statistics, press release, october 3,1993) and the california department of corrections estimates that recently enacted "three strikes" legislation will increase incarceration by 81,628 prisoners by the turn of the century (california department of corrections, 1994). the current crowding conditions in us prisons are due, in part, to a failure to predict the long-term demand for prison capacity. while adequate one-year-hence forecasts are possible with simple tools such as moving averages or other linear models, long-term forecasts are difficult to obtain. this is due, in part, to the nature of the prison population. specifically, a recent survey showed that recidivists, those returning to prison, comprise about half of all prisoners in this country (beck, etal, 1993, p.l 1). the effect of this subgroup on the flow of inmates through prison and on capacity is synergistic. that is, recidivists are more likely to receive longer sentences and thus increase demand for prison beds. the effect of insufficient capacity also has a synergistic effect with recidivists (see lattimore and baker, 1992, for a discussion of this point.). for example, if capacity is limited, then one of the few options open to prison administrators (assuming a lack of concern about prison conditions) is to release inmates earlier, hence recidivists are free sooner and "available" to commit a crime and be reincarcerated-the "revolving door" of the criminal justice system. thus, the non-linearity in the long-term input-output process which characterizes the prison system has made traditional modeling techniques and traditional * points of view are those of the author and do not necessarily represent the official position of the us department of justice or the national institute of justice. ajis linear approaches inadequate for accurate forecasting of demand. as succinctly stated by macdonald (1989,), "reality has had a habit of outstripping forecasts." determining the future capacity needs for institutions in which demand is characterized by input and output processes that are stochastic in nature, driven, wholly or in part, by a population demographics, and subject to policy intervention is a difficult problem for planners. in addition to prisons, other systems which share some or all of these characteristics include hospitals and, to a lesser extent, school systems. traditional approaches to predicting future demand for these types of systems include time series, econometric models, simulation studies and stochastic techniques (e.g., see rhodes, 1990). generally, these approaches require accurate estimates of the input population, disaggregated on the basis of historical data or a priori assumptions concerning changes in population demographics and trends. for example, a prison population at time t may be represented as a mapping of the general population at time t i onto the prison population at time t. thus, the demand-generating population is exogenous to the model, making predictions dependent on accurate characterization of the input process. these models ignore the endogenous effect of the process on the future intake population, and the effect of policy initiatives on future capacity. we develop an approach to predicting demand for capacity which explicitly considers the feedback effect of the output process on the input process. we apply the methodology to prison population projection, extending some earlier work of blumstein, cohen and miller (1980) and barnett (1987). our approach explicitly includes recidivists, incorporating the techniques of survival modeling into the stochastic modeling approaches of blumstein, cohen and miller (bcm) and bamett. the prison population projection model developed herein assumes that the prison intake population is generated from two distinct populations-the general population and the population of former inmates. thus, intake is comprised of those coming to prison for the first time ("first-timers") and those returning to prison (recidivists). secondly, our model explicitly considers the impact of limited capacity on service rate, current prison population and future prison population. although our model is not limited to prison application, indeed the generalization to demand for hospital beds, for example, is conceptually straightforward, the application allows us to call upon the theoretical work of bamett and bcm and address a problem which is of increasing public concern, namely the problem of prison crowding. the state of north carolina prison system will be used to demonstrate the model. the system selected has particular relevance to the modeling approach as in 1986 the state of north carolina instituted a capacity ceiling in response to prison overcrowding and thus provides an opportunity for a model such as ours, which considers the impact of constrained capacity on the population forecast, to be validated. the next section briefly examines the problem of prison overcrowding and reviews research in prison population project subsequent sections present the proposed projection model assumptions and formulation, and the data and estimation technique. demonstration of the model for the state of north carolina under assumptions of unconstrained and constrained capacity is presented. prison population projection methods used to predict prison population include a variety of models, ranging from simple linear projections to microsimulation models (us general accounting office, 1984; see also rhodes, 1990). one of the first probabilistic models for predicting inmate (and parolee) population was developed by stollmack (1973) who derived a model from deterministic differential equations representative of the correctional system input and output (i.e., admissions and releases). the system of differential equations was shown to yield a model identical to an infinite-server queuing system with exponential service times. he developed the model for the case of a general service-time distribution (m/gl/°°) and projected estimates for the district of columbia jails. the model included a constant intake or commitment rate.y, an exponential service distribution with service rate, u,, and associated service time s. prison population at time t, p, was estimated recursively, by the following summation of retention and intake: p^p^-^+y-s-o-<?-") , (1) where the first term on the right-hand side is the number of prisoners at time t -1 who remain in prison at time t and the second term is the number of individuals committed to prison in the period (t -i, t] who remain in prison at time t. bcm (1980) extended the approach of stollmack by disaggregating the inputs by age (a), sex (s), race (r), and offense (<?) type. information on criminal justice system processing rates (arrest, indictment, conviction, and incarceration) was used to develop demographic-and-offense-specific arrival rates (jano). the service rate, uo^ , associated with the agerace-sex-offense-specific mean time served s,̂ were also estimated. this information was then used to estimate equation (1) for each arso group. summing over these yielded the prison population at time t; prison population forecasts for the state of pennsylvania for the years 1970 through 2000 were made. the bcm work was subsequently extended by barnett (1987), who allowed more flexibility toward the modeling of policy changes. arrivals in barnett's model were from the "chronic offender population," a subset of the general population assumed to begin crime at age c and retire at age r. by modeling the processes governing retirement from september 1994 crime and imprisonment, barnett developed a status-quo model (i.e., assuming no changes in incarceration policy) of the following form: p, = ) p,(a) • pr(a)da (2) c where pxa) is the age-specific density of chronic offenders and pr(a) is the probability that offenders of age a will be incarcerated at time t. barnett assumed one homogenous, chronic-offender class characterized by the same starting age, retirement and incarceration processes, and sentencing distribution. chronic offenders were assumed to be a known proportion of the general population and the proportion of offenders of various ages, races, etc., were assumed to remain constant over time. barnett used data from the bcm study to arrive at his own projections state of pennsylvania prison population. he also made status-quo projections for other states in the us, including massachusetts, florida, and utah, and projections which assumed that policy changes would result in increased (mean) time served. finally, barnett used his model to examine the impact of changes in the assumed proportion of individuals in the chronic offender population, and sentencing policy. the forecasting approaches of bcm and barnett take advantage of information that can be known with some confidence (i.e., general population forecasts) to improve prison population projections over simpler methods which, for example, project next year's prison count on the number of individuals in prison this year. two shortcomings of their approaches are 1) the impact of recidivism on the intake population cannot be addressed endogenously, and 2) the impact of capacity on time served and therefore its effect on the return of recidivists cannot be ascertained. by separately modeling the return to prison of those previously incarcerated (about 50 percent of all prison admissions) two benefits are derived. firstly, the non-constant (with respect to time) hazard rate of repeat offenders can be used to more accurately model the "career criminals" discussed by barnett (1987). secondly, the impact of capacity constraints and changes in sentencing policy on future admissions can be evaluated. model a recidivism-inclusive-population-projection model is developed as an input-output model. inputs are assumed to be stochastic in nature, following a well-defined probability distribution. similarly, outputs are generated as a function of a well-defined distribution over time served. the "stocks" at any time f, include the general population (n,), the prison population (?„), and the population of released offenders (r,). "flows" through the system in the interval (t 1, t] include first-timer commitments from the general population (c,c, , )• recidivist commitments from the former inmate n population ( ct _ | t )» and releases ( /?,_,, ). this process is illustrated in figure 1 . the (j)^ and <))* are probability density functions that describe the probability that an individual will be incarcerated at time t for the first time from the general population, and reincarcerated from the released population, respectively. these distributions represent the joint probability of arrest, indictment, conviction, and incarceration for each group. for the moment, we assume homogeneity of the two arrival-generating populations, ^v. and rf . thus, the components of the model can be defined as follows: nt = the number of individuals in the general population at time f, r = the number of former inmates free at time f, p f = the total number of inmates at time f who are incarcerated for the first time; k = the total number of inmates at time t wh f r pk = the total number of inmates at time t who have at least one prior incarceration; p = the total prison population at time t = pf + p cf_, number of first-timer commitments in the period (/-/,/]; c* = number of recidivist commitments in the period (t l,t] ; ty^ = the probability of arriving to prison from the general population; <j)f (l) = the probability of arriving to prison from the released population r. i < r, t f = the service rate for the first-timer prison population (t -/,/]; and if = the service rate for the recidivist prison population in period (t /,/]. first-time commitments are assumed to arrive from the general population, n, via a poisson distribution with constant mean tyf . total commitments from this population in time(f /,f] are: ajis cd c 3 o q. o o q) q. t] o </> o 7) •ssi.i: o september 1994 commitments from the released population, r, arrive via a split-lognormal failure model. the lognormal failure function was chosen because the hazard rate of this distribution, which increases and then decreases, is consistent with observed recidivism patterns (see, for example, schmidt and witte, 1988). as the hazard rate associated with the lognormal distribution is not constant, the probability of returning to prison at time t is conditional on the time since release. specifically, the probability of returning to prison in period (/ l,t], given release in period (i-1, i], i < t, is: where (5) and 5, \i, and a are maximum likelihood estimates of the split-lognormal survival model. the splitting parameter, 5, "splits" the released population into two groups, those who will eventually recidivate and those who will not. inclusion of this parameter adapts the more familiar failure model to the situation in which not all individuals ultimately fail and allows explicit consideration of the career criminal or chronic offender paradigm since desistance (or "retirement") is accommodated by this parameter. thus, commitments in period (t l,t] from the released population are: <)>, ( i ) -r,_u (6) 1=1 as can be seen, the number of commitments from the released population is a function not only of the total number of released individuals at risk in the period (t -1, t], but also of the time since their releases. releases from prison are assumed to follow a negative exponential distribution, where sentence length (service time) is dependent upon whether the inmate is a first-timer or a recidivist. these distributions have means 1( = (5*. ) for first-timers, and tf =(5^ ) for recidivists, where sf and 5"* are estimates of the mean time served for first-timers and recidivists, respectively. the general model for predicting prison population at time t is thus: p, = /£ •*-'' + /;* -e* +c,vsf-(i-*the number of individuals in prison at time / is the sum of those admitted prior to time t who have not been released by t, plus those admitted and not released between t 1 and t. iteration of this model produces subsequent population projections. it should be noted that the prison admission rate is not necessarily constant over time, but instead is a function of the number of released individuals and the timing of their release. note that a constant overall admission rate implies , (0 ' *;-u ' *,« =*"'**, + 5>* , (/) ' /u / n, (8) i=i 1=1 there is no reason to assume that this equality will hold, particularly in times of large demographic changes in the populations. the model presented in equation (7) assumes only two homogeneous populations from which prison admissions are generated. as propensity to crime varies by age, race and sex, improvements in projections can be achieved by estimating the model for specific demographically homogeneous classes, as suggested by stollmack (1973) and incorporated into the models of bcm (1980) and barnett (1987). thus, equation (7) will be estimated for age/race/sex specific classes and the total p will be found by summing over the classes. when different age groups are considered, a small complication arises with respect to "aging" the recidivist commitments. specifically, the probability of returning to prison is a function of age at release from the last admission to prison. by equation (6), the number of recidivist commitments of age class a in period (t 1, t] is: 6 ajis (9) where <j>* (/ , a) is the probability that a member of age class a at the previous admission returns to prison in period (ti, t] given release in period (i /, i] and r. (a) is the number of releases who were in age class a when admitted prior to release in period (i /, /]. however, some of the r. . ( a) individuals will have aged into an older class prior to this new commitment and some members of younger classes will have aged into class a. thus, aging of the commitment population is necessary. finally, the model presented in equation (7) assumes that pt < pf°p for all /. rather than population, this model actually predicts demand or what stollmack (1973) referred to as "population pressure." imposition of a capacity constraint, e.g. p^af ', can be operationalized by assuming that the s will be reduced as prisoners are released early to meet the capacity constraint under a status-quo assumption (meaning that prisoners incarcerated at time f = 0 will serve the same sentence as those imprisoned in t = t + n) and given that p and c are known, the number of releases (from each prison population) required in the interval (t 1, 1] is: *,-„ = p,-, • (i o + c,-,, • (i t' • (i o) the number of releases required, however, is cr, =*,.,+ ̂ t" given that r^ap is known, the following equation identifies t* and the time served distribution required to satisfy the capacity constraint: a search routine (bisection method) was used to identify t'. when p ap > p, status-quo prison population projections are obtained from equation (7), thus providing an upper bound on expected demand. when p ap < p, and p ap is known, the new service times needed to meet available capacity can be obtained. data and estimation methods data were obtained for the state of north carolina. population estimates by age and race groups were provided by the north carolina office of budget and management for 1980 through the year 2000. the north carolina department of correction provided information on the probability of first-time incarceration by age and race group, average annual prison population (1979 through 1988), and the number of releasees in 1979. data to estimate the recidivism models were obtained from the inter-university consortium of political and social research. these data comprise the 1980 north carolina release cohort data set described in schmidt and witte (1988). this data set (henceforth referred to as the s&w sample) contains recidivism information for 9,549 prisoners released from north carolina prisons between july 1, 1979, and june 30, 1980. projections will be made for "classes" of individuals. as male prisoners comprise about % percent of the prison population in north carolina, forecasts will be made only for males. specifically, first-timer and recidivists commitments and prison populations will be estimated by race (white and non-white) and age. seven age categories will be used: [15,20), [20, 25), ..., [40,45), and [45+) years. thus, we have 14 classes for each of our two populations. the model was initialized using data for the year 1979 (t = 0) and estimates of commitments, releases, and prison populations were generated for the years 1980 through 2000. the average 1979 male prison population (/^) was 13,489. to properly initialize the model, the population was disaggregated by age, race and previous incarceration status (first-timerversus-recidivist) as shown in table 1 . the number of males released from prison in 1979 was estimated to be 8,919. this total was disaggregated into age/race categories using the age/race distribution of the s&w sample. this distribution is also included in table 1 . as this release cohort represents only a fraction of the total number of previously incarcerated individuals "on the street" in north carolina in 1980, it was necessary to estimate a larger, more accurate /? . the number of male prisoners released for 1959 through 1978 were developed using data from volumes of the sourcebook of criminal justice statistics ( us department of justice, 1975, '976, 1977, 1978, 1979, 1980). the number who were still on the street at t = 0 were estimated as r,./ , times one minus the cdf (at t 2 1 , 20, .... 1 years) of the estimated split-lognormal failure model (see september 1994 table 1: initial prison and releasee age/race distribution (%) age [15, 20) [20, 25) [25, 30) [30, 35) [35,40) [40, 45) [45+) total first-timers white 9.38 9.38 3.84 2.32 1.37 0.92 1.50 28.71 non-white 7.38 8.15 4.80 2.33 1.04 0.8 0.95 25.48 recidivists white 2.12 5.20 3.87 3.46 2.63 1.92 3.40 22.60 nonwhite 1.77 5.32 5.45 3.96 2.41 1.48 2.81 23.20 releasees white 14.04 13.70 7.20 5.35 3.66 2.84 4.5 51.31 nonwhite 11.65 13.49 9.39 5.36 3.31 2.05 3.44 48.69 lattimore and baker, 1992). this iterative process generated an /? of 114,012. recall that commitments in each period (/ /, /] derive from two populations, the general population, ^v(, and the releasee population, r . commitments from n were assumed to be generated by the distribution §f, the probability distribution of first-time incarceration. <j)f(a,r) was calculated from information provided by the north carolina department of corrections (see table 2). as can be seen, the probability of a first-time incarceration increases from the youngest to the next youngest class and then decreases with age. this distribution over age is due to two factors. firstly, younger adult males are more likely to commit crimes than older males; secondly, those admitted to prison at older ages are more likely to be recidivists. commitments from the general population, ct ,, were obtained by multiplying <j)f ( a, r ) by estimates of the general population by age/race for the years 1980 through 2000. to estimate recidivist commitments, it was necessary to develop parameter estimates for the split-lognormal recidivism models (see equations 5-7). the s&w data were used to generate maximum-likelihood parameter estimates for each of the fourteen age/race groups. the actual and estimated numbers of failures for 15-to-20 year old white males are shown in figure 2. the results for the other classes are similar suggesting that the models provide good fit to the data. the parameter estimates are shown in table 3. the estimates imply a range of mean time to failure from 22.2 months for the youngest white group to 124.5 months for the [35,40) non-white group. table 2: probability of first-time incarceration by age and race, * (°'r) age group 15-to-20 years 20-to-25 years 25-to-30 years 30-to-35 years 35-to-40 years 40-to-45 years 45+ years white 0.0039 0.0040 0.0028 0.0016 0.0014 0.0010 0.0003 non-white 0.0129 0.0143 0.0141 0.0107 0.0058 0.0040 0.0008 as previously noted, prison releases are assumed to follow a negative exponential distribution. the mean of this distribution is the inverse of the mean time served, s. these data were acquired from the 1980 release cohort data set s was found to be significantly different for each age group; further, recidivists served longer sentences than first timers. race was not a factor in the average time served. s by age and first-timer versus recidivist status are shown in table 4. ajis (q c cd to • ^ o ?t ® 9 3to 5 oq. -< q) cd 3 fi) q. is m y xa*o 5 «§ff 2 cd sl 5' cd o 0) fi) cd fi) 5' 3 to # reincarcerations -»• -̂ n3 isj co w o c n o c n o c n o t n q)o_ tt. _̂x en en september 1994 9 table 3: failure model survival estimates race white non-white age [15, 20) [20, 25) [25, 30) [30, 35) [35, 40) [40,45) [45+) [15,20) [20, 25) [25, 30) [30, 35) [35,40) [40, 45) [45+) number 1204 1195 628 481 331 246 427 999 1203 839 494 309 190 320 failures 529 370 197 151 100 84 105 525 525 327 157 99 65 92 8 0.481 0.390 0.480 0.462 0.356 0.557 0.289 0.631 0.551 0.478 0.533 0.564 0.411 0.335 n 2.68 3.04 3.42 3.25 2.84 3.47 2.55 2.88 3.03 2.96 3.56 3.68 2.88 2.71 a 0.927 .090 .318 .479 .072 .643 .338 .095 .114 .093 .523 .513 .083 .136 note: values are maximum-likelihood parameter estimates for split-lognormal failure models estimated using 1980 north carolina data. table 4: mean time served by age and status (years) age 15-to-20 years 20-to-25 years 25-to-30 years 30-to-35 years 35-to-40 years 40-to-45 years 45+ years first-timers .5 .4 .2 .1 .0 .0 .0 recidivists 2.6 2.4 2.0 1.8 1.8 1.3 1.3 results for north carolina we initially estimate north carolina prison populations for the years 1980 through 2000 under the following status-quo assumptions: 1 ) the probability of a first-time incarceration (by age and race), (|) f , is constant over the forecast period. 2) the probability distribution of a return to prison (by age and race) is constant over the forecast period and over the period in which the initial released population was generated. 3) s (by age and prior-incarceration status) is constant over the forecast horizon. 4) the distribution of intra-class age-at-admission, in other words the proportion of each class of each age, is constant over the forecast horizon. 5) these assumptions imply that there will be no changes in criminal justice system efficiency or policy nor changes in the propensity of individuals to commit crime and enter prison. the next subsection presents results under the assumptions presented above; subsequently, results are presented when assumptions 1, 3 and 5 are relaxed. status quo projections figure 3 shows the projected admissions and prison populations for the years 1980 through 2000. as can be seen, both prison admissions and prison population are projected to increase until about 1995 after which they stabilize. the male prison population is projected to increase by about 39 percent, from 15,177 in 1980 to 21,117 in the year 2000. admissions are projected to increase from 10,400 to 13,800. for the forecast period, the general population in north carolina is also projected to grow, but the general population growth is less than that of the projected prison population growth. 10 ajis the growth in admissions and prison population is largely attributable to increasing numbers of recidivists entering prison (and staying for longer periods). first-time admissions remain between about 7,000 and 7,500 throughout the forecast period. recidivist admissions increase, however, from about 3,500 in 1980 to more than 6,200 in the year 2000. the increase in recidivist admissions is, of course, attributable to increases in releases over the planning horizon. as the prison population grows, the number of releases also grows, generating a larger population of potential recidivists. indeed, over the 21-year forecast horizon, annual releases increase 43 percent (from 9,613 in 1980 to 13,741 in the year 2000). as was noted in the previous section, the admission rate by age/race group may change over the forecast horizon. the data show that there is considerable growth in the admission rate for the older age groups over the forecast period, suggesting an "aging" of the prison population over time. table 5 shows the projected changes in admission rates (per 100,000 population) between 1980 and 2000. the admission rates for the youngest (15-to-20 years) groups are relatively stable primarily due to the high proportion of first-time admissions in this class (88 percent in 1980; 85 percent in 2000) and the assumption that <|)f (a,r) is constant over t. the prison admission rates for older groups increase substantially over time—for example 22 percent for the 40-to-45-year-old white group and 47 percent for the 35-to-40year-old nonwhite group. table 5. percentage change in prison admission rate, 1980 2000 age group 15-to-20 years 20-to-25 years 25-to-30 years 30-to-35 years 35-to-40 years 40-to-45 years 45+ years percentage change white -0.02 -0.03 0.05 0.18 0.22 0.15 0.2 nonwhite 0.04 0.13 0.27 0.39 0.47 0.42 0.75 the estimations in this section assume that capacity is unlimited and "f and t* remain the same as that served by the 1980 release cohort. in the next section, we modify our model to allow for the impact of constrained capacity on prison population, mean time served, releases, and admissions. subsequently, we consider the effects of potential changes in sentencing policy in a constrained and unconstrained environment. capacity constrained projections after a brief turndown in 1982 through 1984, north carolina prison population grew rapidly until 1986 after which it held steady at just under 17,000. there is a simple explanation for the relatively constant number of north carolina prisoners over the last three years. in 1986, the north carolina state legislature imposed a three-year capacity constraint of 18,000 (male and female) inmates on north carolina's prison population. the north carolina department of correction administers this constraint by the early release of inmates when the prison population reaches 17,460. about 94.3 percent of north carolina's prison capacity is available for male inmates implying a maximum allowable male prison population of 16,465. this legislated constraint of course mirrored available capacity. if the prison population at time t exceeds the capacity, pt cap, additional releases are required. these "early releases" reduce s for the aggregate population. we initially assume that these additional releases are distributed proportionally across the 28 age/race/prior-incarceration classes. we then examine how the capacity ceiling of 16,465 affects mean time served and, subsequently, the distributions of returns to prison. the effect of the capacity ceiling on mean time served was modeled. results are shown in figures 4 and 5. figure 4 shows, for the youngest age groups, the change in the time served means required to achieve the capacity constraint these and the results for the other groups suggest-other things equal-that a "steady state" is achieved fairly quickly under the constrained status quo conditions. the status quo and "steady state" means are shown in figure 5. these results suggest that a decrease of about 25 percent in the means of the time-served distributions would be required. although strictly comparable figures are not available from north carolina, between 1983 and 1989, the average time served by released felons dropped by 12.8 percent (from 1.95 years to 1.7 years), while the average sentence length of those admitted has increased by 15.3 percent (from 6.0 to 7.2 years). the average time served by misdemeanants dropped by more than 40 percent during the same period. 12 ajis a decreasing mean service time has the effect of increasing the releasee pool r thereby increasing the number of admissions over the status quo model (assuming criminal behavior, as measured by the parameters of the survival model, does not change). the model predicts an additional 3,530 commitments due to the early release policy engendered by the capacity constraint. as the model is structured, these additional commitments are due entirely to the younger ages of release of the offenders. a second factor which could affect the rate of recidivist returns is a reduction in the deterrent effect that could result from reduced punishment. conclusions a model which accurately predicts future demand for prison beds is important however, most models which have attempted this task have offered projections which ignore the fact that there is not enough space to house inmates. thus, these models are not only unrealistic, but limited in that they fail to take into account the synergistic effect of early release policies (instituted in response to overcrowding) and the intake process. recall that close to fifty percent of the inmate population are recidivists. and a large percentage of these individuals are in crime-prone ages. early release policies therefore tend to place highly active individuals on the street earlier allowing them more time to recidivate. in other words, the "revolving door" spins faster. underlying any discassion of capacity needs is the issue of what constitutes "optimal" capacity. fiscal and humanitarian issues not withstanding, the question of how many beds are needed must consider the impact of current and future capacity on demand. population projection models which view the system as being incapacitated provide an upper bound of demand. practically speaking, no jurisdiction is likely to build enough prisons to house all projected inmates. because capacity will never meet demand, the problem becomes one of determining what level of capacity is acceptable. results here suggest that this question cannot be answered completely unless the effect of reduced capacity on future demands is addressed. also, if there is a deterrent effect and it is inversely related to time served (ceteris paribus), one could also see changes in the recidivism model. for example, if serving 18 months is more of a deterrent to future criminality than serving six months, then capacity constraints and resulting early release policies may have an even more dramatic effect on future admissions. in this paper, we developed a stochastic model for an input/output process which exploits the effect of outputs on the input process. the model is the first to model the prison system from the vantage point of interrelated system in which the outputs reenter as inputs. this approach is a dramatic change from linear forecast models which view inputs and outputs to be linear in nature, to one in which the control of the system requires modeling of the non-linear properties of the population flow. results reported suggest that a prison population projection model which includes the effect of recidivism on future demand provides a good forecast and, more importantly, allows for consideration of the effect of policy changes on the input/output process. september 1994 13 (q c 3 3 tj ft) o i-* li > •go os 3 j5. 3 o cd fi) 3 h (d 0) 0 0 o cd c (d cn 3 •o q> o o q) t3 q) o o o 3 (/) r-hs 3' r+ o 3 <d co o (d q. 0) d -n (d (d o > (d o (d (q (d o o c o t 10 j 01 w ^ o v1 o ? *? 5° september 1994 references barnett,a. (1987). "prison populations: a projection model", operations research, vol 35, pp 18-34. beck, a., gilliard, d., greenfield, l., harlow, c., heater, t., jankowski, l., snell, t., stephan, j. & morton, d. (1993) survey of state prison inmates, 1991. washington, dc: us department of justice, bureau of justice statistics. blumstein, a., cohen, j. & gooding, w. (1983) "the influence of capacity on prison population: a critical review of some recent evidence", crime and delinquency, vol 29, pp 1-51. blumstein, a., cohen, j. & miller, h.d. (1980) "demographically disaggregated projections of prison populations", journal of criminal justice, vol 8, pp 1-26. california department of corrections (1994) "population and fiscal estimates of the "three strikes" initiative", sacramento, california, february 28. lattimore, p. k. & baker, j.r. (1992) "the impact of recidivism and capacity on prison populations", journal of quantitative criminology, vol 8, pp 189-215. rhodes, w. (1990) .models of the criminal justice system: a review of existing impact models, us department of justice, bureau of justice statistics, washington, dc. schmidt p-and witte, a.d. (1988) predicting recidivism using survival models. new york: springer-verlag. stollmack, s. (1973) "predicting inmate populations from arrest, court disposition, and recidivism rates", journal of research in crime and delinquency, vol 10, pp 141-162. us department of justice, us law enforcement assistance administration. (1975) sourcebook of criminal justice statistics--1974. washington, dc: us department of justice. us department of justice, us law enforcement assistance administration. (1976) sourcebook of criminal justice statistics-1975. washington, dc: us department of justice. us department of justice, us law enforcement assistance administration. (1977) sourcebook of criminal justice statistics-1976. washington, dc: us department of justice. us department of justice, us law enforcement assistance administration. (1978) sourcebook of criminal justice statistics-1977. washington, dc: us department of justice. us department of justice, us law enforcement assistance administration. (1979) sourcebook of criminal justice statistics-1978. washington, dc: us department of justice. us department of justice, us law enforcement assistance administration. (1980) sourcebook of criminal justice statistics-1979. washington, dc: us department of justice us department of justice, bureau of justice statistics. (1981) sourcebook of criminal justice statistics-1980. washington, dc: us department of justice. us department of justice, bureau of justice statistics. (1982) sourcebook of criminal justice statistics— 1981, washington, dc: us department of justice, us department of justice, bureau of justice statistics. (1993) "prisoners in 1992", washington, dc, may. us general accounting office. (1984) federal, district of columbia, and states future prison and correctional institute populations and capacities: report to the honorable arlen specter united states senate. gao/ggd-84-56, february, washington, dc: us general accounting office. 16 ajis microsoft word 8 ajis 14-2.doc australasian journal of information systems volume 14 number 2 june 2007 141 power relationships that lead to the development of feral systems d. v. kerr griffith university, queensland, australia email: d.kerr@griffith.edu.au luke houghton griffith university, queensland, australia kevin burgess queensland rail abstract this research identifies factors affecting the operation of a supply chain in a large, asset rich transport utility, and how a recent enterprise resource planning system (erp) implementation was perceived with respect to its usability for the task. a lack of trust in the erp, ineffective training methods and complexity in extracting the data from the erp were identified as a problem which lead to the development of “feral systems” (systems outside the accepted erp or corporation condoned information systems – sometimes called skunkworks). this research uses an interpretative case study approach to gain insights into the human sense-making within the study organisation. the research argues that power relationships between operational managers and financial managers and processual power relationships between operational managers led to the development of these systems. key words:enterprise planning systems, power, feral systems introduction the implementation of enterprise resource planning systems (erp) provides many advantages for the companies involved. these include the integration of information technology and all the competitive advantage possibilities associated with improved data access (heizer and render 2003). however, these advantages are not always easily achievable with many erp implementations not producing the results expected and in some cases implementation projects are classed as failures (umble, haft and umble 2003). this paper investigates end-user perceptions and power relationships within the supply chain of a large, asset rich transport utility in an effort to identify australasian journal of information systems volume 14 number 2 june 2007 142 some of the possible reasons for these failures; in particular, this research investigates the development and use of information systems for use outside the organisationally condoned erp. to date there has been little research into the impact of erp implementations on employee’s motivation to develop their own information systems to supplement or replace the corporately endorsed system. this research looks into this aspect using a single case study approach. the research seeks to more fully understand the methods employees use to overcome erp implementation problems. for example, previous research has highlighted many cases of end-user development of information systems and the resultant problems invariably associated with this approach (kreie, cronan, pendley and renwick 2000) and (mcgill and klobas 2005). this area of research was initiated when it became apparent that end-user development was a significant factor in the way operational managers approached their work. this end-user phenomenon became a dominate theme throughout all stages of a series of interviews conducted within the case study organization (described later in this paper). because of the unique features of these end-user developed systems, they have been described as “feral systems” by houghton and kerr (2006). a feral system is defined as “an information system [computerised] that is developed by individuals or groups of employees to help them with their work, but is not condoned by management nor is part of the corporation’s accepted information technology infrastructure. its development is designed to circumvent existing organisational information systems” (houghton and kerr 2006, p 137). we contend that the feral system example is indicative of a general unease with the corporate approved information system by employees and that these are developed because of power tensions between financial and operational managers. in addition it is our contention that processual power relationships between operational managers lead to the formation of coalitions to help overcome these power tensions and to circumvent the existing erp in an effort to improve work processes at the operational level of the organisation. previous research indicates that power relationships play a major role in erp implementations, for example; nandhakumar, rossi and talvinen (2005) have suggested that n “imbalance in power relations between two divisional heads might restrict the progress of it projects [and that] a strong hierarchical culture may prevent users from interacting directly with the designers of the technology” (page 225). the research outlined in this paper expands on the power imbalance suggestion by nandhakumar, rossi and talvinen and contends that there are power relationships within the case study utility and that this power is having a major influence on the way that employees view the organisation and how they interact with the implemented erp. one power play that is apparent is processual power. jasperson, carte, saunders, butler, croes and zheng (2002, page 401) describe processual power as “part of the decision making sphere and micropolitics of organisational life. decisions and priorities involved in negotiations are emergent phenomena” they also refer to (fincham 1992 p 743) who suggests that power lies "not in concrete resources but in strategies like coalition-formation and the manipulation of information that protagonists employ in the power game". the first proposition in this paper is that processual power is a driving force in the development of feral systems and this proposition is tested using the following case study. the consideration of power in the analysis of this data is consistent with foucault’s assertion that “every relationship is a power relationship” (avgerou and mcgrath 2007). in this paper, the we assert that processual power is one of the driving forces in the development of feral systems as they are usually developed through the coalition of operational managers and their manipulation of information in parallel to or instead of the existing erp system. as jasperson et al (2002) indicate, processual power involves coalition formation and information manipulation and we contend that australasian journal of information systems volume 14 number 2 june 2007 143 these are the basic ingredients in feral systems development. boonstra_and de vries (2005) also cite fincham (1992) and suggest that the power concept when related to inter-organizational systems can be viewed from the following perspectives: 1. processual power 2. institutional power and 3. organizational power. boonstra and de vries go on further to say that “processual power is in the socialinteraction between interest groups” and “institutional structures of inequality form the external bases from which power is mandated to organizations. organizational power is embedded in the internal structure of organizations” (page 489). other authors (waddell, devine, jones and george 2007) have described power in terms of the different types of power namely: 1. legitimate power where the power is legitimized through the position the person holds within the organization 2. reward power where the manager can give or deny rewards such as promotion or pay rises 3. coercive power where the manager can punish others 4. expert power where the power is based on special knowledge and skills possessed by the manager or others. 5. referent power where the power comes from the personal respect and admiration given by subordinates. 6. information power where the power is based on controlling information needed by others. it is proposed in this paper that another reason for the development of feral systems is associated with an employee backlash against the other types of power being exerted by financial managers. these could include legitimate, reward, coercive, expert and information power that leads to the decision to implement a system that is not considered ideal by operational managers. this second proposition is also tested in the following case study. the case study the organisation used for this study is a large, networked transport utility. it is an essential utility service within australia and has been operating since the 1860’s. utilcom (a pseudonym) has a global reputation for providing innovative and performance driven services, for example utilcom employees advise and consult similar utilities at an international level on a regular basis. the corporation has an annual turnover of over 2 billion dollars (aud) and over 13,000 employees. utilcom recently implemented the erp, sap r/3. sap (systems applications and products in data processing) is a provider of business software solutions designed to integrate information technology resources for companies. the website describes their company as “serving more than 36,200 customers worldwide, sap is the world’s largest business software company and the world’s third-largest independent software company overall. today sap employs more than 38,400 people in more than 50 countries. our professionals are dedicated to providing the highest level of australasian journal of information systems volume 14 number 2 june 2007 144 customer service and support” (sap 2006, about us section). a full description of the sap r/3 erp is also shown on their website (sap 2006). the implementation was designed to improve reporting and other functions. the process involved conversion from an earlier version namely, sap r2 to sap r3 for 6,000 users. the modules involved in the implementation were: financial; material management; logistics; forecasting and planning; materials resources planning (mrp); human resources; information systems including executive information systems; project management; and office integration. the research project reported in this paper was conducted as two stages. the first stage was designed to develop an understanding of the basic social, technical and political workings of the supply chain and how each of these factors interacted with each other. this first stage involved 31 semi-structured interviews with key stakeholders (operational managers, financial managers and other experts) along a supply chain associated with utilcom. interviews were conducted with a range of people from human resource personnel to operational staff and included the supply chain partners. the interviews were conducted to determine important elements in each individual’s job and how they related to the supply chain. this resulted in a substantial database of 157,000 words in 230 pages of transcript. the supply chain in question was an integral part of infrastructure building and maintenance for utilcom and involved two other organisations, one was the manufacturer of the product and the other transported the product to the primary distribution point. this first stage of the research pointed towards concerns about the future usefulness of the yet to be implemented sap r/3. for the purposes of this paper we will refer to the first stage of the project as stage one and the second as stage two. in stage two, 16 semi-structured interviews were conducted with previously identified experts in the utilcom supply chain. these experts were identified as key personnel within the supply chain by the research team and in consultation with the general manager of the supply division of utilcom. the object was to gain a better understanding of the important social factors associated with the erp implementation. the expectations of the new implementation were manifold and the supply division of utilcom considered sap r/3 to be a useful system for determining inventory levels and transactions across the whole supply chain. however, the results from interviews conducted during stage 1 of the research showed that some concern was expressed about understanding the day-to-day operations of the supply chain. method this research is an interpretative case study approach (klein and myers 1999, walsham, 1993) that attempts to gain insights into the human sense-making within utilcom. the approach is to find an understanding of the situation as described by klein and myers (1999) “through social constructions such as language, consciousness, shared meanings, documents, tools and other artifacts (page 69).” this interpretative approach lends itself to a case study research method (yin 1994, stake 1995) using a combination of evidence sources as also advocated by klein and myers (1999). the sources used in this inquiry are described in table 1. australasian journal of information systems volume 14 number 2 june 2007 145 table 1 – data used for this case study source sources used in this case study documentation intranet postings, company annual reports and reports by the information technology section of utilcom participant observation one of the authors worked at utilcom for three days a week over a three month period, during this time he participated in training and conversed with personnel within utilcom. interviews the semi-structured interview approach was used three months was spent at the headquarters of utilcom and this time enabled one of the authors to observe the implementation process of sap r/3. this author was able to work within utilcom as a contractor and was able to observe work processes as well as do the same training in erp use as utilcom employees. this period of study allowed the research team to more fully understand the power relationships and the underlying history of the sap r/3 implementation. for example it was established through informal discussions with operational managers that sap replaced a more operationally effective software package and that it was perceived by these managers that sap was introduced to handle the financial aspects of the organisation rather than specific operationally based aspects. these managers felt that the previous software developed by mincom was superior to the newly implemented sap r/3 system. the relationships developed and observations made over the three month period at utilcom enabled the authors to iterate through the hermeneutic circle with close attention being paid to the contextualization of the gathered data, interaction between themselves and the subjects and awareness of multiple interpretations and suspicion (klein and myers 1999). this allowed the authors to gain an appreciation of the broader context of the erp implementation through the insights of the one author who observed the process over the three month period. this one author’s involvement enabled a degree of relationship building with the actors within utilcom and allowed for a credible and relevant interpretation of their knowledge and meanings with respect to the interview transcripts from the second stage of the research. documentation the documentation at utilcom portrayed a positive (technologically determinist see houghton, kerr and burgess, 2004) slant to the erp implementation and none of the publicly available documentation mentioned the use of relevant modules of sap r/3 for decision making or forecasting. intranet postings were available to all employees at utilcom as soon as they logged on to their computer and in many instances these postings provided updates of the sap r/3 implementation. questions like the validity of the technology and it’s use in social systems is a question rarely asked in the literature and it’s not surprising that industry practitioners and consultants would hold similar positions. interviews interviews were semi-structured and open-ended in nature with questions expanding on emerging themes asked as required. the initial 31 interviews (the first stage of the research) were conducted before the implementation of the sap r/3 and the remaining 16 were conducted a year after australasian journal of information systems volume 14 number 2 june 2007 146 implementation. coding of the relevant theme was completed by the open coding method (nuemann 2003). this involved organizing transcript passages into concepts and emerging themes by looking for a consistent pattern by reading the printed transcripts and highlighting identified themes and key words with different coloured highlighter pens. this coding method provides a platform for examining the interviews for both observable themes and allows a comparison with abstract concepts found from informal discussions, observations and follow-up questions asked during the interviews themselves. insights from this first series of 31 interviews led to the following research question; how effective is the current erp approach in providing a method for employees to obtain data relevant you their own job requirements? the stage one interviews indicated that many of the staff at utilcom was not fully convinced of the usefulness of the erp implementation. this was emphasised in many occasions during both the first and second stage interviews and led to many “workarounds” by staff. this identification of “workarounds” led the authors to the concept of “feral systems”; reported later in this paper. it should be noted that a feral system is different to a workaround in that workarounds are designed to bypass of a recognized problem in a system and this is done usually as a temporary fix. feral systems on the other hand are deliberate attempts to permanently circumvent a fully functional and operationally sound system. researcher bias the first named author is an experienced researcher with a systems thinking approach within the positivist school. he has some knowledge and appreciation of the qualitative approaches undertaken within the information systems discipline. he was and still is a university employee and was on academic study leave while undertaking the participant observation during this case study. he had some knowledge of the problems of erp implementations at utilcom through his previous association with a general manager at utilcom (the third author in this paper). this author was able to closely involve himself in the daily operations at the supply division of utilcom and attended meetings and training sessions as well as having informal discussions with utilcom staff. results the results for this study are divided into the main problems discovered from the three approaches namely, observations from the documentation provided to employees, participant observation of utilcom and analysis of the interview transcripts. feral systems there was an extensive use of end-user developed systems referred to as “feral systems”. these computer systems were developed by individuals outside the accepted sap environment. for example, statements by middle management (an engineer) in the early stages of the research (stage 1 interviews) provided illustrations to support this assertion, for example; … i rely heavily on we’ve got a diary that tracks all material usage on a daily basis so it will have on there how many [of the product] we unloaded today so bruce will come in write in the diary in what section, how many lengths of [product] he does. that diary then goes into a database internally within here and onto a spreadsheet … i do the inventory audits probably once every 3 to 6 months … sort of thing. (construction engineer) australasian journal of information systems volume 14 number 2 june 2007 147 this indicates a high level of internal information systems (feral systems) are developed and used to either supplement the sap system or replace it. the use of feral systems was further confirmed in stage 2 interviews when the researcher mentioned the feral systems concept, respondents recorded statements such as; yes that’s a very good point and the people that make these feral systems will have no end of argument about why they are needed and yes there may be justifications but at the end of the day if everyone goes down their own little track with their own systems we are going to be in the same position as we were before we implemented r3 (upper level manager) other respondents indicated that feral systems may not be as much of a problem as indicated and that they could be confused with other reporting tools, for example; certainly we've pulled data out of sap and put it into access databases or excel spreadsheets to do various manipulations so to that extent yes we are doing that but i don't see them as feral systems. i see those as different reporting tools(upper level manager commodities) on the other hand the same respondent indicated that he could see problems with feral systems in certain cases where there was little or no visibility for the organisation, for example: at the other end of the equation there are a number of customers who are doing their planning outside of sap so there is no visibility for the organization because their planning exists in excel spreadsheets or access databases and isn't rolling up into any corporate plan so that does create a problem (upper level manager commodities) and (interviewer) the information from [supervisors], how does that information come? is it stored in [enterprise] system? (interviewee) [no] stored in email and excel. but can be referred back to. (interviewer) is that shared with others or not. (interviewee) no not really.[we use] emails and telephones. with [place a] and infrastructure, once a week, usually thursday. thursday’s loading day at [place a]. so therefore we know exactly what, thursday and friday mornings is loading day, and it leaves [place a] friday afternoon. so by thursday, we know exactly what’s going out, hopefully if there’s going to be a problem we, we’ve addressed it before then. and that’s basically all there is. [place a] faxes the information to us, to here, allen has a copy, i take a copy, i then inform infrastructure on what we’re doing, see if there’s any problems. infrastructure will then tell me if there is a change in the deliveries, and it’s a step back down the line, back to [place a]. (contracts administrator) notice in these selected quotes that there is no perceived need to get this crucial information back into the main erp system for either forecasting or knowledge management purposes. again, this highlights the use of ‘feral’ systems and a local conceptualisation of a supply chain. some other quotes: you have to have good information storage, albeit in a social rather than technical system. (production manager) oh yeah, very much with others. couldn’t do it alone. in terms of working with others, i don’t need to worry about what they are doing as such, rather i am suggesting i take almost a helicopter view in this, sort of checking where things are at and therefore needing a lot of information on how they are going … that’s right. what i also feel works well is our systems accuracy. although, i guess, a little bit of question about what australasian journal of information systems volume 14 number 2 june 2007 148 doesn’t work well, i might talk about limitations in the system but what we do have is very accurate. we never have occurrences where in our documentation there is any discrepancies in the volumes and product we send. or what we send. i don’t have detailed knowledge of the systems but from … it seems to me that they are very robust.(sales) (interviewer) and where do you get this [information] from? people or systems? (interviewee) a lot of it is from the systems and a lot of it is communication between people. cause usually if mike gore has got some orders coming up we get some enquiry, he’ll contact me by phone or email and we will discuss it on how best we can do it. put it into practice. (despatch) here are two quotes that show how two parts of the supply chain relate to each other by bypassing established systems. the people that send out the goods (despatch) rely on email and telephones to keep track of orders that come from the sales people. with such a large organisation this is surprising, given that sales occur in the millions on a regular basis. recent problems at the organisation have indicated that such reporting has led to ‘missing’ stock and may not be as efficient as claimed above. more examples of this kind of thinking are shown below: if they are doing the ordering system right then i know about it. the only trouble is sometimes they don’t do the ordering system right and they might not order it off the right plan and i don’t know about it for two weeks. generally people ring up so communication is a lot of to-ing and fro-ing, backwards and forwards which is a bit ad hoc, unfortunately. but because if somebody wants brand new [product] they probably need it soon and it’s all this other stuff that takes time. you know there is a project happening, there is a project manager that’s also in the background who may be saying to us, ‘six months ago i need [product] for such and such.’ he is not the person who ordered it so, yeah, we go off the formal thing so we know where to get the information from. (project officer) i think we would work pretty hard to maintain that, because it is very, you know we’ve got websites and stuff like that, but they can’t see it. but from my point of view, i can see, and it is not to see to be critical, it is to see to help to sometimes, you know what can we do? like the example of what’s happening while it’s pouring rain, … they can send emails, and they can quote information, and we can look at information and share information, it’s not top secret, it’s stuff that we both need to know and make sound decisions about what are we going to do here. do we stop, do we keep going here, or whatever. (transport manager) (interviewer) how do you get the wagons, how do you put that process in place? (interviewee) via email the wagons are there. if they are not at [place a], the rotations of the wagons is predictable. just goes round in circles like a hornsby set. goes on sunday, will come back monday night. has a process, it could run itself really. (interviewer) are you able to track it? (interviewee) yes. (logistics officer) the following quote demonstrates a more explicit connection between the erp system and the operations of firm a: (interviewer) why not just use sap for your records? (interviewee) sap isn’t … doesn’t have the functionality. with our three-way checking system the order is raised on sap but it’s only when the boys at [the station] acknowledge receipt for payment that it then goes into the sap inventory system. that’s australasian journal of information systems volume 14 number 2 june 2007 149 too … i need information before then and with more function to tell if something is going wrong. the only time sap would tell you if something is wrong is when the invoice doesn’t match the money stated in the order. there is a hell of a lot more happens than that so i need far more information to be able to plan, tell customers what is happening and fix problems when things go wrong. (interviewer) can you expand on what you do with the faxes and the emails from firm a? (interviewee) like i say, it’s cross-check. the emails are really only there if something goes wrong. i can then work it out. but it is the faxes that really count coz they are raised when things are actually dispatched. the emails are from a different source and more around what was produced. i keep track of all the faxes in the excel spreadsheet i have set. i need the system to do my work with the accuracy for supplying [product]. i would struggle to function without it.(commodity analyst), in these sampled quotes, ‘feral systems’ conceptualisations are clear. they reflect a wide range of thinking about how crucial information and communication technology is used. in some cases the functionality of sap, which it is known for, was argued not to exist. this is more than likely a lack of knowledge on behalf of the person who said it, but it also highlights the lack of conception of the meaning and purpose of the system and the underlying conflict that is evident. discussion the interpretative research principle of the hermeneutic circle, multiple interpretations, contextualisation and abstraction and generalisation (klien and myers 1999) were applied in this study and iteration between individual transcripts and how they related to the whole were made throughout the analysis. the now long standing relationships between the researcher and the staff at utilcom and the three month period of working within the utility has made a deep understanding of the interview transcripts possible. this is due to the transcripts being able to be viewed in a holistic manner and the researcher being able to relate them to the corporation’s cultural and historical context (stake 1995). this study tries to identify the organisational cultural factors and reward systems that drive the employee perceived need to develop feral systems where end users develop their own systems presumably because the sap system does not provide the information they want. it is our contention that these feral systems are developed due to processual power factors whereby operation staff want control over their own work and work processes. there are indications that operational managers are concerned about the highly financial nature of sap and the fact that the sap r/3 software replaced a more operationally useful software package developed by a local software firm, mincom. thus employees developed feral systems to gain back some control. it is also the case that expert and informational power was exerted by the financial managers through their insistence that the sap r/3 erp be implemented and that it replace the more operationally effective, mincom system. it is this exercise of power that the authors consider to be a major factor in the decision of operational managers to develop feral systems. if the more operationally effective software is being introduced, then employees needed to develop their own systems to help in operational matters. on the other hand, it could be the case that these feral systems were developed some time ago (before the erp implementation) and could be classed as legacy systems. kwasi (2007) suggests that “users of these legacy systems typically have vested interests, valuable experience and knowaustralasian journal of information systems volume 14 number 2 june 2007 150 how in those systems. replacing legacy systems means that people have to ‘‘relearn’’ new skills and their unwillingness to do so might lead them to perceive the erps as being difficult to use.” however, the proposition that processual power is a reason for the development of feral systems is a more likely reason because the vast majority of operational managers considered the previous mincom software package to be a very good fit to their work needs. if this is the case at utilcom, it could answer some of the questions asked about the reasons for this phenomenon and could be another reason for the apparent lack of commitment to the erp implementation by some employees. the acceptance of feral systems would depend on corporate policies with respect to management’s tolerance of such systems. in utilcom, tolerance of feral systems seems to be reasonably high and further research would have to be conducted to see if is applies to other utilities. the research conducted here indicates that employees at utilcom are aware of the advantages that erps provide under ideal conditions with respect to integrated it resources and better decision making through better access to data. however, in practice these ideal situations do not eventuate, especially under a networked asset rich utility environment. this appears to be due to social and other factors such as a lack of strategic and operational communication, a lack of alignment with other reporting systems, the use of feral systems instead of sap and a general mistrust of the erp itself. it is suggested that this could be due to utilcom’s environment where computer literacy levels could be lower in employees who have more secure job tenure and the motivation to make the erp work is not as great because the organisation does not have an incentive to reduce head counts. in other words the employee’s job is not “on the line” as may be the case in a commercial enterprise. other aspects were identified in this study but were too numerous to report in this paper, however a summary of these factors are provided in this paragraph as they relate to employee perceptions of the processes used in the implementation of sap and to the reasons why these power tensions occurred. the integrity of the data used in sap and the links to meta-data was discussed and deemed to be problematic by some staff members. other issues associated with utilcom and of relevance to other erp implementations, were end user information literacy, poor computer literacy levels and the ways organisations can make people want to engage with the erp. the apparent lower computer literacy levels could be due to the job security levels associated with the utility. employee turnover is less in the case study environment and the incentive to re-skill may be less, resulting in a workforce with lower computer literacy levels. this lack of desire to re-skill was evident in the period the author was working at utilcom with managers lamenting about the low number of graduates employed by the corporation. in addition, procurement and invoicing practices differed in the various areas of the state and it became obvious that country based employees had different approaches to procurement because as quoted by one respondent “they have to live in the district and rapid payment to suppliers was a big priority for them”. we contend that the additional factors reported in the previous paragraph all have some bearing on the reasons why there are power imbalances within utilcom and hence the reason for the development of feral systems. for example the low computer literacy levels can have a direct bearing on perceptions of expert and informational power and can lead to more faith in systems developed by operational managers with limited but still significantly higher computer skills than fellow co-workers. australasian journal of information systems volume 14 number 2 june 2007 151 conclusions this study has uncovered problems with the use of the implemented erp system within utilcom. despite a number of relevant modules within sap, many employees were not aware their existence, in addition, organisational cultural and social aspects such as inconsistent procurement policies makes the effective use of the erp very difficult. some of these problems were related directly to utilcom, while others were more generic in nature. the authors believe that utilcom employees have developed feral systems because of the power relations between operational managers and financial managers and that these power relationships work both in a top down manner (expert, informational and coercive power) and in a bottom up manner (processual power). these power relationships need to be investigated at a much deeper level as they indicate a real and possibly universal concern with respect to erp implementations across a wide range of industries. this study identified many aspects associated with the erp implementation and many have not been reported in this paper, however we suggest that they have underlying causal connections to the reasons why there are power imbalances at utilcom. the only way a firm connection between these factors and the power imbalances can be determined is through a whole of system approach to the research problem, however under the present guidelines to publishing in information systems literature this is deemed to be too ambitious a project. perhaps the information systems discipline needs to consider ways of reporting research projects that have a more holistic view of business problems. references boonstra a._and de vries j.(2005) analyzing inter-organizational systems from a power and interest perspective, international journal of information management 25 485–501 heizer, j., & render, b. (2003). operations management—international edition (7th ed). upper saddle river, nj: pearson education inc. houghton l and kerr d.v. (2006) a study into the creation of feral information systems as a response to an erp implementation within the supply chain of a large government-owned corporation int. j. internet and enterprise management, vol. 4, no. 2, pp 137-142. houghton l, kerr d.v. and burgess k (2004) sap is it systematic research bias, which is to blame for such post implementation disappointment? 15th australian conference on information systems, hobart, tasmania, (1-3 december 2004) klein h. k. and myers m.d. (1999) a set of principles for conducting and evaluating interpretive field studies in information systems, mis quarterly vol 23 (1) pp 67-94. kreie, j., cronan t.p., pendley j. and renwick j.s. (2000) applications development by end-users: can quality be improved? decision support systems 29 pages 143–152 kwasi a.g (2007) perceived usefulness, user involvement and behavioral intention: an empirical study of erp implementation computers in human behavior vol 23 pp.1232–1248 mcgill t. j. and klobas j.e. (2005) the role of spreadsheet knowledge in user-developed application success decision support systems 39 pages 355– 369. australasian journal of information systems volume 14 number 2 june 2007 152 nandhakumar j, rossi m. and talvinen j (2005) the dynamics of contextual forces of erp implementation journal of strategic information systems 14 pages 221-242 nuemann l (2003) social research methods, 5th edition, allyn-bacon/pearson, usa. sap (2006) sap forecasting and replenishment (2006) http://www.sap.com/industries/retail/pdf/bwp_forecasting.pdf accessed february 6 2006. stake, r. (1995). the art of case research. thousand oaks, ca: sage publications. umble e.j., haft r.r. and umblene m.m. (2003) enterprise resource planning: implementation procedures and critical success factors, european journal of operational research 146 pp 241–257. waddell d, devine j., jones g. r. and george j. m. (2007) contemporary management, mcgrawhill irwin, australia walsham g., (1993) interpreting information systems in organizations, john wiley, chichester. yin r. k. (1994) case study research: design and methods sage publications london. appendix major questions for stage 2 of the research 1. what does your job entail? a. roles b. responsibilities c. is your job understood by others? 2. if you had a magic wand, what would you do to make your job better? a. better for whom? i. customers ii. yourself iii. others 3. what information could be provided to help you with your work? 4. what information is provided already that helps you with your work? 5. what would you need to help you make better decisions? 6. of all the suggestions you have given, what are the most important? paper title (paper title style) australasian journal of information systems cosic, shanks & maynard 2015, vol. 19, pp. s5-s19 business analytics capability 1 a business analytics capability framework ranko cosic university of melbourne rankocosic@hotmail.com graeme shanks university of melbourne gshanks@unimelb.edu.au sean maynard university of melbourne seanbm@unimelb.edu.au abstract business analytics (ba) capabilities can potentially provide value and lead to better organisational performance. this paper develops a holistic, theoretically-grounded and practically relevant business analytics capability framework (bacf) that specifies, defines and ranks the capabilities that constitute an organisational ba initiative. the bacf was developed in two phases. first, an a priori conceptual framework was developed based on the resourcebased view theory of the firm and a thematic content analysis of the ba literature. second, the conceptual framework was further developed and refined using a three round delphi study involving 16 ba experts. changes from the delphi study resulted in a refined and confirmed framework including detailed capability definitions, together with a ranking of the capabilities based on importance. the bacf will help academic researchers and industry practitioners to better understand the capabilities that constitute an organisational ba initiative and their relative importance. in future work, the capabilities in the bacf will be operationalised to measure their as-is status, thus enabling organisations to identify key areas of strength and weakness and prioritise future capability improvement efforts. keywords: business analytics; resource-based view; thematic content analysis; business analytics capability framework; delphi study. 1 introduction business analytics (ba) systems provide benefits to organisations by enabling improvements to business processes, firm performance and creating competitive advantage (davenport and harris, 2007). ba systems include the people, processes and technologies involved in the gathering, analysis and transformation of data used to support organisational decision-making (negash 2004). ba technologies include: data warehousing, reporting, visualisation, online analytical processing (olap), forecasting, predictive modelling and statistical analysis (watson 2010). although empirical evidence shows that ba systems bring benefits to organisations (kohavi et al. 2002; allmendinger and lombreglia 2005; piccoli and watson 2008), few studies provide a sound theoretical basis for understanding how and why these benefits are achieved. the resource-based view theory of the firm provides this theoretical base. hence, in this study, we draw upon the resource-based view theory to identify and define a set of capabilities that constitute an organisational ba initiative. this research is important for three reasons. first, ba continues to be the top investment priority of chief information officers (gartner 2015). second, many organisations are making large strategic investments in ba. this trend forecast to continue with global spending expected to reach $143.3 billion dollars by 2016 (computerweekly.com 2013). third, there is a strong need for a holistic, theoretically grounded and practically relevant ba capability framework (bacf) that can help explain how organisations achieve benefits with ba systems. the bacf is developed in two phases. first, an a priori conceptual framework is developed based on the resource-based view and a thematic content analysis of the ba literature. second, australasian journal of information systems cosic, shanks & maynard 2015, vol. 19, pp. s5-s19 business analytics capability 2 a delphi study is used to refine and validate the framework structure, components and detailed capability definitions. furthermore, the delphi study is used to rank the ba capabilities based on importance. participants in the delphi study include both academic and practitioner experts, resulting in a bacf that is both theoretically grounded and practically relevant. the paper is organised as follows. first, we discuss relevant background literature. next, we describe and justify the thematic content analysis and delphi study research approaches used to develop and refine the bacf. following that, we present the revised bacf and explain the revisions and clarifications that resulted from each round of the delphi study. finally, we conclude the paper with some implications of the work for academic researchers and industry practitioners, and suggestions for future research. 2 background in this section, three areas of relevant literature are reviewed. these include: (i) the resourcebased view and dynamic capabilities; (ii) use of the resource-based view in information systems (is) research; and (iii) ba capabilities and frameworks. 2.1 resource based view and dynamic capabilities the resource-based view is the dominant theory in the field of strategic management research (barney 1991). it conceptualises organisations as bundles of capabilities and resources. in the resource-based view a capability is the ability to utilise resources in order to perform a certain task (wernerfelt 1984). these resources may be tangible (e.g. technology and people) or intangible (e.g. data, processes, skills and knowledge) (barney 2001). to be a source of sustainable competitive advantage (sca), capabilities and resources must be valuable, rare, inimitable and non-substitutable (vrin). • valuable – enable an organisation to implement a value-creating strategy; • rare – in scarce supply; • inimitable – cannot be copied by rivals; • non-substitutable – cannot be countered by a rival with a substitute. a limitation of the resource-based view is that it is relatively static, and does not take into account the rapidly changing nature of dynamic business environments (teece and pisano 1994). dynamic capabilities were conceptualised in order to overcome this limitation (barreto 2010). dynamic capabilities enable organisations to continuously renew and reconfigure their resource base in order to keep pace with rapid changes in the business environment. dynamic capabilities comprise three organisational routines: sensing, seizing and reconfiguration (teece et al. 1997). sensing and seizing involve identifying and prioritising ba-enabled business opportunities. reconfiguration involves implementing selected ba-enabled business opportunities by creating new resource configurations (helfat et al. 2007; sharma and shanks 2011). 2.2 the resource based view in information systems research in is research, the resource-based view has been used to explain how is capabilities and information technology (it) resources lead to benefits including organisational value (raber et al. 2012) and sca (barney 2001). the resource-based view in is literature delineates the distinction between it resources and other organisational resources. for example, it resources include computer networks, hardware, software and data whilst other organisational resources encompass people, processes and routines (wade and hulland 2004; arial and wiell 2007; nevo and wade 2010). three key findings that emerge from this literature stream are as follows. first, it is highly unusual to find a vrin it resource per se (wade and hulland 2004). this is because it resources have become increasingly commoditised (gartner 2009) and can easily be duplicated by competitors (porter 2001). australasian journal of information systems cosic, shanks & maynard 2015, vol. 19, pp. s5-s19 business analytics capability 3 second, non-vrin it resources can be combined with other non-vrin organisational resources to produce it-enabled resources. an example, of an it-enabled resource is a geographically dispersed team of people supported by a videoconferencing tool (nevo and wade 2010). third, vrin is capabilities are developed from interactions that occur between non-vrin it resources and other non-vrin organisational resources. for example, a geographically dispersed team’s communication capability is developed through opportunities to apply a videoconferencing tool in making team-based decisions (nevo and wade 2010). the accumulated culture, history, learning and experience from the team’s interaction with the tool cannot be duplicated, thus giving the communication capability its vrin properties (teece and pisano 1994). 2.3 business analytics capabilities and frameworks taken as a whole, the preceding findings demonstrate that is capabilities are more likely to yield competitive advantage than it resources. furthermore, the extant is literature provides extensive support for the link between is capabilities and organisational value and competitive advantage (bhatt, grover and grover 2005; johnston and carrico 1998; saraf, langdon and gosain 2007). this is particularly important since ba capabilities are also is capabilities. there are a number of ba and business intelligence (bi) capability maturity frameworks, from both the academic and practitioners literature (see for example davenport and harris 2001; watson et al. 2001; eckerson 2009; yeoh and koronios 2010 and raber et al. 2012). none of this work is theoretically grounded in the resource-based view and refined and validated empirically. this gap in knowledge will be addressed through the development of a bacf. capability frameworks are important instruments for information management (rozemeijer 2007) and ba is an important application field for these frameworks. from the extant literature, we identified two important ba capability frameworks, both in the form of maturity models (davenport and harris 2007; raber et al. 2012). the earliest of these was the 5 stages of analytical competition model proposed by davenport and harris (2007). in this model, ba capabilities are structured hierarchically. for example, davenport and harris (2007) envisaged the overall ba capability of an organisation to be comprised of three lowerlevel ba capabilities: human, technological and organisational. this two-level hierarchical structure is also consistent with other conceptualisations of is capabilities (e.g. barreto 2010; pavlou and el sawy 2006). davenport and harris’s (2007) model was derived from extensive industry experience and case studies of world leading analytics organisations. for this reason, the model has a high degree of practical relevance but lacks a theoretical foundation. in contrast, the ba capability maturity model proposed by raber et al. (2012) does have a theoretical foundation. for example, raber et al. (2012) draw on is success models and their underlying theoretical foundations as the basis for their model. the model consists of 58 ba capabilities grouped into five distinct areas that represent is concepts including: (i) strategy, (ii) social system, (iii) technical system, (iv) quality and (v) use/impact. like davenport and harris’s (2007) model, raber et al.’s (2012) model also has practical relevance. this is because ba practitioners, specialists and executives played a key role in the development and evaluation of the model. a limitation of the aforementioned models is neither incorporates dynamic capabilities. becker et al. (2010) argue that in an era of rapidly changing business environments the resource-based view, particularly dynamic capabilities, should be used as the theoretical foundation for ba maturity models. in view of this, this study aims to develop a holistic, theoretically-grounded and practically relevant bacf that is based on the resource-based view and includes dynamic capabilities. australasian journal of information systems cosic, shanks & maynard 2015, vol. 19, pp. s5-s19 business analytics capability 4 3 research approach in this section, we discuss the research approach and methodologies used to produce the bacf. 3.1 thematic content analysis the capabilities that constituted the initial version of the bacf were obtained from a thematic content analysis of the extant ba literature. first, the term ba capability was qualified. this was done by extrapolating resource-based view capability conceptualisations to the ba context. for example, as previously mentioned, in the resource-based view a capability is conceptualised as the ability to utilise resources in order to perform a certain task, and is produced from the interaction between it assets and other organisational resources. in line with this, a ba capability was defined as: ‘the ability to utilise resources to perform a ba task, based on the interaction between it assets and other firm resources’. next, a sample of 51 publications focused on ba was sourced from the extant literature. this sample included major is journals/conferences (e.g. senior scholars’ basket of 8, european conference on is (ecis) and international conference on is (icis)), as well as the most highly cited publications between 1991 and 2014 from all other outlets. the 51 publications were then analysed thoroughly in order to identify any keywords, themes or phrases that corresponded to the previously mentioned definition of a ba capability. all such items were assigned a specific definition that was in turn used to identify other instances of that capability. three cycles of analysis and interpretation of the publications produced the 16 ba capabilities (brief ba capability definitions are provided in appendix 1). the ba capabilities were grouped into four capability areas based on similarities. these are: • governance a mechanism for managing the use of ba resources and the assignment of decisionrights and accountabilities to align ba initiatives with organisational objectives (weill and ross 2004). • culture tacit and explicit organisational norms, values and behavioural patterns that form over time and lead to systematic ways of gathering, analysing and disseminating data (leidner and kayworth 2006). • people individuals who use ba as part of their job function (davenport et al. 2007). • technology development and use of hardware, software and data within ba activities (negash 2004). figure 1 shows the initial bacf with the 16 ba capabilities grouped within the four ba capability areas that comprise the overall ba capability. higher quality ba capabilities will lead to improved value and competitive advantage. australasian journal of information systems cosic, shanks & maynard 2015, vol. 19, pp. s5-s19 business analytics capability 5 figure 1: initial business analytics capability framework. 4 delphi study in is research, there are two fundamental research quality criteria that need to be taken into account: (i) theoretical and methodological rigor and (ii) practical relevance (moody 2000). a delphi study was used to further develop and refine the bacf. the use of a delphi study further enhances the methodological rigor of the bacf. however, importantly, it also serves to enhance the overall practical relevance and utility of the bacf through direct input from experienced ba experts. a delphi study is a qualitative research methodology, which involves iterative cycles of questionnaire preparation, collation and analysis of responses, concept refinement and the provision of feedback to participants, ideally culminating in a consensus of opinion (skulmoski et al. 2007). satisfaction measures are used to determine whether a consensus has been reached (worrell et al. 2013). a delphi study is used for two main reasons. first, it is well suited to the development of conceptual frameworks such as the bacf (okoli and pawlowski 2004). second, it offers several important advantages over a focus group including; participant anonymity, no need for faceto-face meetings, it is relatively free from the influence of social pressure and dominant personalities, participants can be sourced from a wider geographical area and they can respond at their convenience (stewart and shamdasani 1990). three key design considerations in a delphi study are: the number of participants; selection of participants; and the number of rounds (okoli and pawlowski 2004). we aimed to have 15 participants as this provides a broad range of opinions and having more should not make a significant difference to the results (dalkey 1969). however, 20 participants will be invited in order to allow for the possibility of attrition. participants included leading australian academic and practitioner ba experts to strengthen the generalisability of the outcomes. each participant required a minimum of 5 years experience in ba. studies have shown that three rounds are usually sufficient to achieve an expert consensus. studies have also shown that after three rounds participant interest declines and response fatigue sets in (skulmoski et al. 2007). in view of this, in this study, the decision whether to conduct a subsequent round will be based on whether a consensus has been reached, or if the australasian journal of information systems cosic, shanks & maynard 2015, vol. 19, pp. s5-s19 business analytics capability 6 number of participants falls below a certain level. similar to indulska et al. (2009), consensus is correlated with a minimum satisfaction/importance measure of 8.0 and a maximum standard deviation of 2.0. in line with dalkey (1969), the minimum number of participants required for a round is 15. 5 delphi study discussion this section first discusses details of the delphi study participants, response rates for each round and the data analysis approach used. then, for each round, changes to the bacf are discussed. 5.1 participants, response rates and data analysis in all, 20 participants including 8 from academia and 12 from industry were invited to participate. the academics included 4 professors, 2 lecturers, a senior lecturer and a research fellow. the practitioners included 10 managers, a senior consultant and technical specialist from a range of industries including banking, finance, telecommunications, retail and software. sixteen of the participants were from melbourne, 3 were from sydney and 1 was from switzerland. participants had on average 14 years of ba experience. the response rate in the first round was 80% (4 academics and 12 practitioners). the response rates for the remaining rounds were 75% (4 academics and 11 practitioners) for the second round and 80% (4 academics and 12 practitioners) for the third round. importantly, the response rates all exceed 70%, which is the recommended minimum requirement for rigor (mullen 2003). the delphi study produced a mixture of qualitative and quantitative data. the qualitative data was used to refine the ba capability definitions and identify key concepts and themes. the quantitative data was summarised using simple descriptive statistics and used to assess the level of consensus among the participants with respect to the ba capability definitions, and to rank the ba capabilities in descending order of importance within each capability area within the bacf. 5.2 outcomes of round 1 in the first round, participants were provided with a table containing very brief definitions of the capabilities. participants were asked to add up to three more key aspects, which had been overlooked in the existing definitions. in addition to this, participants were also asked to suggest up to four more capabilities that they thought should be added to the table. two types of change are discussed below: changes to the capabilities and changes to the capability descriptions. the criteria for accepting recommended changes were agreement between two or more participants or a change that was judged to yield a new or important insight by the researchers. 5.2.1 changes to the capabilities the capability ‘decision rights’ was renamed to ‘decision rights and responsibilities’ for two reasons. first, rights were considered different from responsibilities and should therefore be combined in the definition. second, responsibilities were considered an important facet of this capability. the capability ‘change management’ was renamed to ‘impact and change management’ to reflect better the scope of the capability. a capability labelled ‘flexibility and agility’ was removed from the framework as 25% of participants agreed that it overlapped with the ‘impact and change management’ capability. several additional capabilities were suggested, however only one, ’communication’, was supported by more than one participant. it was added to the framework. several suggestions for changes involved capabilities that were synonyms for existing capabilities. for example, ‘the ability to champion ba’ is already in the framework as ‘executive leadership and support’. nearly 70% of the participants did not suggest any additional australasian journal of information systems cosic, shanks & maynard 2015, vol. 19, pp. s5-s19 business analytics capability 7 capabilities. this suggests that the ba capabilities identified in this study were robust and holistic. 5.2.2 changes to the definitions of capabilities a total of 62 changes were made to capability definitions, of which 38 were changes recommended by more than one participant and 24 were changes deemed to yield a new and important insight. 30 of the changes were suggested by practitioner participants, 9 by academic participants and the remainder by both practitioner and academic participants. this suggests strong participation by both types of participant. two examples of the many changes concern the definitions of ‘impact and change management’ and ‘data management’. for ‘impact and change management’, participants added user involvement, rewards and the value of ba. for ‘data management’, participants added data quality, single version of the truth (practitioners only), master data and meta-data. 5.2.3 areas of disagreement two capabilities where considerable disagreement occurred were ’dynamic ba capabilities’ and ‘management skills and knowledge’. there were many suggestions for ‘dynamic ba capabilities’ but none were in agreement or considered to yield a new or important insight. this may be due to the more abstract nature of dynamic capabilities, which is essentially a firstorder concept, intended to renew existing ba capabilities (pavlov and el sawy 2006). several participants considered project management to be an important facet of ‘management skills and knowledge’, while an equal number considered project management to be a general or normal capability not directly related to ba. it was retained in the definition as it is considered important by davenport and harris (2007). 5.3 outcomes of round 2 in the second round, participants were provided with a table containing detailed definitions of the capabilities. participants were asked to provide comments about each capability definition, to rate the importance of each capability on a scale from 1 (not important) to 10 (very important), and to rate their satisfaction with each capability on a scale from 1 (not satisfied) to 10 (very satisfied). see table 1 on page 10 for satisfaction and importance ratings for this round. 5.3.1 changes due to comments in the second round, there were no direct disagreements between the participants or further changes to the names of the capabilities; however a large number of minor changes were made to all of the capability definitions. in summary, 11 participants suggested an average of 5 changes each, while the remaining 4 participants did not suggest any changes. of the changes made, 9 were suggested by more than 1 participant and 43 were considered to yield a new or important insight by the researchers. specific details of the most significant changes are outlined as follows. interestingly, 5 participants made comments that suggested broader capability areas, including ‘technology’ and ‘culture’, with details that closely match the capability area definitions provided in section 4 of this paper. for example, when discussing c11 (visualisation ba technology), one participant suggested that “a general technology capability is needed”. also, when discussing c16 (entrepreneurship and innovation), another participant noted they were “key aspects of their ability to innovate is organisational culture”. these, and other similar changes, clarify the ba capabilities that constitute the technology and cultural aspects of an organisational ba initiative and enhance the work of davenport and harris (2007). only two out of the 53 changes in this round were deletions and both were for the same capability, c3 (dynamic ba capabilities). the term ‘renewal’ was removed from the description as 2 participants were ‘not sure what it meant’ and noted that it ‘did not resonate with their logic’. also, the sentence about intellectual property retainment was deleted as australasian journal of information systems cosic, shanks & maynard 2015, vol. 19, pp. s5-s19 business analytics capability 8 participants found it placed too much importance on the retention of intellectual property within dynamic capabilities. many minor additions were made to the capability definitions. a noteworthy change was for c14 (business skills and knowledge) where it was noted that people in ba-related management roles “perform an intermediary role: translating and communicating the potential value of ba (insights) to the senior executive team”. this is important because much of the literature related to this capability discusses the importance of executive managers understanding the value of ba (pfeffer and sutton 2006; shanks et al. 2012). however, much less is said about the ways in which this can be achieved. c14 (business skills and knowledge) was further refined in order to make it less ambiguous and more parsimonious, following comments from participants that the definition was “too long” and “too broad”. c9 (data management) and c16 (entrepreneurship and innovation) were extended in order to make certain points clearer and less ambiguous. the phrase “single version of the truth” was replaced with “common usage and understanding of the data” in c9 and the phrase was replaced with “think outside the square” was replaced with “ability to rationally assess risks and benefits”. finally, the definitions of c13 (technology skills and knowledge) and c14 (business skills and knowledge) were modified to make it clearer that they refer to the combined skills and knowledge found ‘throughout the organisation’ rather than the skills and knowledge of a particular individual. these changes were based on comments from participants, for example “finding people with all of the above skills at a deep level would be extremely rare (i don’t know anyone like this)” 5.3.2 importance ratings average importance ratings for 13 of the 16 ba capabilities were above the required threshold of 8.0 with maximum standard deviation of 2.0, indicating a high level of consensus after the second round of the delphi study. the three ba capabilities that did not meet the threshold were c6 (embeddedness) (7.8: 2.0), c10 (systems integration) (7.6: 2.1) and c12 (discovery ba technology) (8.1: 2.2). 5.3.3 satisfaction ratings the participant’s satisfaction ratings were averaged, and only 6 of the capabilities were above the threshold of 8.0 with maximum standard deviation of 2.0. 5 other capabilities were very close to the threshold and the remaining 5 capabilities were well below the threshold. the high variation was mainly due to 1 participant who assigned ratings of 1 out of 10 for several capabilities. 5.4 outcomes of round 3 in the third round, as in the second, participants were provided with a table containing detailed definitions of the capabilities. participants were asked to provide comments about each capability definition, to rate the importance of each capability on a scale from 1 (not important) to 10 (very important), and to rate their satisfaction with each capability on a scale from 1 (not satisfied) to 10 (very satisfied). see table 1 on page 10 for satisfaction and importance ratings for this round. 5.4.1 changes due to comments in the third round, there were no direct disagreements between the participants or further changes to the names of the capabilities. in summary, 9 participants suggested a total of 34 changes, of which 14 were deemed important enough to modify the capability definitions. minor changes were made to the definitions for 7 capabilities (c1, c2, c6, c9, c12, c15 and c16). of these changes, only one was suggested by more than 1 participant. for example, in relation to c1 (decision rights and responsibilities) two participants shared the view that “people responsible for decisions must be held accountable”. another example of a minor change was to c12 (discovery ba technology). one participant suggested that “text mining as australasian journal of information systems cosic, shanks & maynard 2015, vol. 19, pp. s5-s19 business analytics capability 9 one of the analysis tools” be included in the description, as c12 involved the analysis of unstructured data. 5.4.2 importance ratings there was strong consensus among the participants with respect to the average importance ratings for the capabilities. please note c1 (decision rights and responsibilities) and c3 (dynamic ba capabilities) had the same rating and standard deviation and therefore were listed in alphabetical order. c10 (systems integration) and c12 (discovery ba technology) also had the same rating but c10 had a lower standard deviation (1.5) than c12 (1.6) and therefore was ranked higher. the minimum average importance rating in this round was 8.0 (c10 (systems integration)) as required and the maximum standard deviation was 1.6 (c10 (systems integration) and c12 (dynamic ba capabilities), which was well below the required 2.0. most of the importance ratings and all of the standard deviations decreased between rounds 2 and 3. 5.4.3 satisfaction ratings a strong consensus was achieved among the participants with respect to the average satisfaction ratings for the capabilities. for example, the minimum average satisfaction rating was 8.1 for c6 (embeddedness), which was above the required 8.0 and the maximum standard deviation was 1.5 for c6 and c13 (technology knowledge and skills), which was well below the required 2.0. all of the average satisfaction ratings increased and the standard deviations decreased between rounds 2 and 3. the ratings for 6 capabilities (c1, c2, c3, c4, c5, c16) increased by more than a point and the standard deviations for 8 capabilities (c1, c3, c4, c5, c11, c12, c14, c16) decreased by more than a point. this shows that changes to the definitions from round 2 led to a significant increase in participant satisfaction. table 1: ba capability satisfaction and importance measures australasian journal of information systems cosic, shanks & maynard 2015, vol. 19, pp. s5-s19 business analytics capability 10 the shaded areas in table 1 show the measures that did not meet the criteria for consensus e.g. minimum rating of 8.0 and maximum standard deviation of 2.0 6 summary delphi study key findings there were four key findings from the delphi study which are reflected in the revised bacf (figure 2). first, participants initially had diverse expectations of what constitutes an organisational ba capability. this was not surprising given the absence of an explicit definition for the term ba capability within the extant literature. for example, both davenport and harris (2007) and raber et al. (2012) produced ba capability maturity models but did not qualify, nor explicitly define what is meant by the term ba capability. figure 2: revised bacf based on delphi study. second, a consensus amongst participants was achieved after three rounds, which is expected in a delphi study (skulmoski et al. 2007). the response rates for each of the three rounds were above the recommended minimum of 70% required for rigor. however, the response rates between the two groups of experts differed widely. for example, only half of the academics participated in each round. by contrast, all practitioners participated in the first and third rounds and only one missed the second round. these high practitioner response rates served to enhance the overall practical relevance of the results and the bacf itself. third, there were minor changes to the overall structure (e.g. replacement of ‘flexibility and agility’ with ‘communication’ capability) and significant changes to all of the capability definitions. the inclusion of ‘communication’ capability was critical as it was rated the most important capability within the ‘culture’ capability area (see table 1), and is consistent with davenport and harris (2007) and laursen and thorlund (2010), who strongly emphasise the importance of communication in ba initiatives. finally, this study found direct support for the ‘technology’ and ‘culture’ capability areas. together with ‘governance’ and ‘people’ these capability areas are common themes in the is literature (de bruin 2009). however, the unique contribution to knowledge in this study is a rigorous and relevant set of clearly defined ba capabilities. full definitions and literature sources for the ba capabilities in the revised bacf are shown in appendix 2. australasian journal of information systems cosic, shanks & maynard 2015, vol. 19, pp. s5-s19 business analytics capability 11 7 conclusion in this paper, we have described the development of a bacf that is comprised of 16 ba capabilities that are evenly distributed across four main capability areas. the initial conceptual framework was developed based on the resource-based view theory of the firm and a rigorous thematic content analysis of the ba literature. this framework was revised and confirmed in a delphi study. the delphi study saw significant changes made to capability descriptions in the framework, minor but important changes to the overall structure, and a high level of consensus achieved after three rounds. the contributions of this paper are relevant to both researchers and practitioners. for researchers, the bacf will provide a rigorous and relevant set of clearly defined ba capabilities. for practitioners, it will provide a systematic means of understanding the range of ba capabilities that are required for ba initiatives. the importance rankings will help in prioritising the development of ba capabilities, and the detailed descriptions will help in the planning and development of comprehensive ba capabilities. furthermore, a framework for identifying given capabilities in ba is valuable for companies as it provides benefits by enabling improvements in business processes, firm performance and creating competitive advantages. in future work, these capabilities will be operationalised to measure their as-is status, thus enabling organisations to identify key areas of strength and weakness and better prioritise future capability improvement efforts. developing measures for each ba capability enables the bacf to form the basis for a ba capability maturity model. future work should examine how the ba capabilities develop over time and their relationship to organisational strategy, performance, value and competitive advantage. references anderson-lehman, r., watson, h.j., wixom, b.h., and hoffer, j.a. 2004. continental airlines flies high with real-time business intelligence, misq executive, 3(4), 163-176. allmendinger, g. and lombreglia, r. 2005. four strategies for the age of smart services. harvard business review, 83(10), 131–145. aral, s. and weill, p. 2007. it assets, organisational capabilities and firm performance: how resource allocations and organisational differences explain performance variation, organisation science, 18(5), 1-18. barney, j. 1991. “firm resources and sustained competitive advantage,” journal of management(17:1), pp 99-120. barney, j.b. 2001. is the resource-based "view" a useful perspective for strategic management research? yes, academy of management review, 26(1), 41-56. barreto, i. 2010. dynamic capabilities: a review of past research and an agenda for the future, journal of management, 36(1), 256-280. becker, joerg; niehaves, bjoern; poeppelbuss, jens; and simons, alexander, 2010. "maturity models in is research" ecis 2010 proceedings. paper 42. bhatt, g. d., grover, v., & grover, v. 2005. types of information technology capabilities and their role in competitive advantage: an empirical study. journal of management information systems, 22(2), 253-277. cosic, r; shanks, g; maynard, s (2012) "towards a business analytics capability maturity model". acis 2012 : location, location, location : proceedings of the 23rd australasian conference on information systems 2012, australasian conference on information systems (23rd : 2012 : geelong, victoria), pp 1-11. computerweekly.com 2013. global business intelligence spending to double in four years. accessed 12th august 2014, from australasian journal of information systems cosic, shanks & maynard 2015, vol. 19, pp. s5-s19 business analytics capability 12 http://www.computerweekly.com/news/2240175647/global-business-intelligencespending-to-double-in-four-years dalkey, n.c. 1969. an experimental study of group opinion, futures, 1(5), 408-426. davenport, t.h. and harris, j.g. 2007. competing on analytics: the new science of winning. harvard business school press, boston. de bruin, t. 2009. business process management : theory on progression and maturity. phd thesis, queensland university of technology. eckerson, w. 2009 tdwi’s business intelligence maturity model, chatsworth. gartner. 2015. flipping to digital leadership: insights from the 2015 cio agenda report, gartner group. helfat, c.e., finkelstein, s., mitchell, w., peteraf, m.a., singh, h., teece, d.j., and winter, s.g. 2007. dynamic capabilities: understanding strategic change in organisations. blackwell, carlton. indulska, m., recker, j., rosemann, m., & green, p. 2009. business process modeling: current issues and future challenges. in advanced information systems engineering, 501-514. springer berlin/heidelberg. johnston, h. r., & carrico, s. r. 1988. developing capabilities to use information strategically. mis quarterly, 37-48. kohavi, r., rothleder, n. and simoudis, e. 2002. emerging trends in business analytics, communications of the acm, 45(8), 45-48. lahmann, g., 2010. business intelligence maturity models: an overview. in d’atri, ferrara and spagnoletti (eds.) proceedings of the vii conference of the italian chapter of ais, naples, italy. laursen, g.h.n., and thorlund, j. 2010. business analytics for managers: take business intelligence beyond reporting. john wiley and sons, hoboken, nj. leidner, d. e., and kayworth, t. 2006. a review of culture in information systems research: toward a theory of information technology cultural conflict, mis quarterly, 30:2, 357399. moody, daniel l. 2000, "building links between is research and professional practice: improving the relevance and impact of is research." proceedings of the twenty first international conference on information systems. association for information systems. mullen, p. m. 2003. delphi: myths and reality. journal of health organization and management, 17(1), 37-52. negash, s. 2004. business intelligence,” communications of the association for information systems, 13, 177-195. nevo, s. and wade, m. 2010. the formation and value of it-enabled resources: antecedents and. consequences of synergistic relationships, mis quarterly, 34(1), 163-183. okoli, c. and pawlowski, s.d. 2004. the delphi method as a research tool: an example, design considerations and applications, information & management, 42, 15-29. pavlou, p.a. and el sawy. o.a. 2006. from it competence to competitive advantage in turbulent environments: the case of new product development, information systems research 17(3), 198-227. pfeffer, j. and sutton, r. 2006. evidence-based management. harvard business review, 84(1), 62-68. australasian journal of information systems cosic, shanks & maynard 2015, vol. 19, pp. s5-s19 business analytics capability 13 piccoli, g. and watson, r. 2008. profit from customer data by identifying strategic opportunities and adopting the “born digital” approach, mis quarterly executive, 7(3), 113-122. raber, d., winter, r. and wortmann, f. 2012. using quantitative analyses to construct a capability maturity model for business intelligence. in proceedings of the 45th hawaii international conference on system science (hicss), 4219-4228. rozemeijer, e. 2007. frameworks for it management. van haren publishing. saraf, n., langdon, c. s., & gosain, s. 2007. is application capabilities and relational value in interfirm partnerships. information systems research, 18(3), 320-339. shanks, g., bekmamedova, n., frederic, a., and daly, m. 2012. “embedding business intelligence systems within organisations,” in a. respicio, f. adam and g. phillipswren (eds) fusing decision support systems into the fabric of the context, amsterdam, nl: ios press,113-124. sharma, r., reynolds, p., scheepers, r., seddon, p., and shanks, g. 2010. business analytics and competitive advantage: a review and research agenda. in a. respicio, f. adam and g. phillips-wren (eds.) bridging the socio-technical gap in dss challenges for the next decade. ios press, amsterdam. sharma, r. and shanks, g. 2011. the role of dynamic capabilities in creating business value from is assets, amcis 2011 proceedings all submissions, paper 135. skulmoski, g. j., hartman, f.t. and krahn, j. 2007. the delphi method for graduate research. journal of information technology education, 6, 1-21. stewart, d.w. and shamdasani, p.n. 1990. focus groups: theory and practice. sage, newbury park, ca. teece d.j. and pisano g. 1994. the dynamic capabilities of enterprises: an introduction. industrial and corporate change, 3(3), 537-556. wade, m. and hulland, j. 2004. review: the resource-based view and information systems research: review, extension, and suggestions for future research, mis quarterly, 28(1), 107-142. watson, h., t. ariyachandra and r. matyska 2001 data warehousing stages of growth, information systems management, 18(3), 42-50. watson, h.j. and wixom, b.h. 2007. the current state of business intelligence. computer, 40(9), 96-99. watson, h.j. 2010. bi-based organizations. business intelligence journal, 15(2), 4. watson, h.j. 2011. business analytics insight: hype or here to stay?, business intelligence journal, 16(1), 4-8. weill, p. and ross, j.w. 2004. it governance: how top performers manage it decision rights for superior results. harvard business school publishing, boston, ma. wernerfelt, b. 1984. a resource-based view of the firm, strategic management journal, 5(2), 171-180. williams, s., and williams, n. 2007. the profit impact of business intelligence. morgan kaufmann publishers, san francisco, ca worrell, j., gangi, p. and bush, a. 2013. exploring the use of the delphi method in accounting information systems research, international journal of accounting information systems, 14(3), 193-208 yeoh, w. and a. koronios 2010. critical success factors for business intelligence systems, journal of computer information systems, 50(3), 23-32 australasian journal of information systems cosic, shanks & maynard 2015, vol. 19, pp. s5-s19 business analytics capability 14 appendix 1 ba capability definition decision-rights determining those responsible for making each kind of decision to ensure the right person makes the right decision at the right time (weill and ross 2004). strategic alignment aligning ba initiatives with organisational goals (williams and williams 2007). dynamic ba capabilities continuous renewal of organisation’s ba resource-base and capabilities to respond to changes in dynamic business environments (shanks et al. 2011). change management to manage resistance from, and provide training to, people impacted by ba initiatives (anderson-lehman et al. 2004). evidence-based management culture where authority, reputation, intuition and ad-hoc decision-making are superseded by decisions based on data and quantitative analysis (pfeffer and sutton 2006). embeddedness extent to which ba has permeated the organisation’s social fabric and become ingrained into people’s values and daily work habits (shanks et al. 2012). executive leadership and support ability of senior managers to infuse a passion for ba and data-driven decision-making throughout the organisation (laursen and thorlund 2010). flexibility and agility receptivity of non-managerial ba personnel to changes in business environment (cheese 2005). data management mechanism for ensuring data used in ba initiatives is fit for purpose and meets the information requirements of the organisation (watson and wixom 2007) systems integration seamless integration of ba and operational systems to exploit the capabilities of both (myerson 2002). reporting and visualisation ba technology development and utilisation of reports, dashboards, scorecards, olap and data visualisation technologies to display output information in a format readily understood by its users e.g. managers and other key decision-makers (ramaamurthy et al. 2008). discovery ba technology development and utilisation of sophisticated statistical and data mining software to explore data and identify useful correlations, patterns and trends and extrapolate them to forecast what is likely to occur in the future (negash 2004). technology skills and knowledge skills and knowledge of ba technology specialists including data management, application development and it in general (davenport and harris 2007). business skills and knowledge skills and knowledge of ba business specialists e.g. accounting and finance (davenport and harris 2007). management skills and knowledge skills and knowledge of management specialists responsible for enterprisewide ba initiatives or projects in local business units (davenport et al. 2010). entrepreneurship and innovation skills and knowledge of managers to use ba technologies to develop innovative and more effective processes and products that lead to better organisational performance and sca (sharma et al. 2010). australasian journal of information systems cosic, shanks & maynard 2015, vol. 19, pp. s5-s19 business analytics capability 15 appendix 2 ba capability definition decision rights and responsibilities (c1) assignment of decision rights and responsibilities by determining; (i) those responsible for making certain decisions in relation to the planning, implementation and applications of ba, (ii) where appropriate, those who will provide the input for such decisions, and (iii) those who will be held accountable for the resulting actions and outcomes of these decisions. it is important that a person responsible for making a certain decision is held accountable for the resulting actions and outcomes (weill and ross 2004). strategic alignment (c2) alignment of an organisation’s ba initiatives with its business strategy. it is a two-way relationship in the sense that ba initiatives can help measure and enforce a business strategy, whilst business strategy necessarily shapes ba initiatives as they evolve. this requires a clearly defined business strategy that is enunciated to all staff and translated into a set of measurable outcomes. it also requires a genuine commitment to the strategy demonstrated by the decisions and actions of senior people (williams and williams 2007). dynamic ba capabilities (c3) ability to reconfigure and leverage an organisation’s ba resources and capabilities in order to respond to changes in the business environment in a timely and efficient manner. such responsiveness requires the ability to identify potential ba opportunities (search), prioritise those opportunities based on business need, risk and technology maturity (select) and then funding and implementing the opportunities (asset orchestration) resulting in new and unique resource configurations (sharma and shanks 2011). impact and change management (c4) ability to manage human, technological and process impacts across the organisation arising from ba initiatives. this involves managing changes to the systems environment and the provision of training and rewards in order to; (i) demonstrate the value and utility of ba, (ii) encourage the adoption of new ba technologies and work practices, (iii) mitigate potential resistance, and (iv) manage expectations. furthermore, it is important that all types of ba users, from managers to operational staff, are involved in the initial planning of a ba initiative (negash 2004). evidence-based management (c5) a culture where (i) formal authority, reputation, intuition and ad-hoc decision-making are preceded by decisions based on data, (ii) ba users, including power users, are encouraged to actively participate in the development of a data-driven environment, (iii) there is trust in data and the ba tools used to analyse data, (iv) whenever possible, assertions are substantiated with data, and (v) although the emphasis is on fact-based decision making, there is still some room for intuition and ad-hoc decision-making, particularly when the required data is not available (pfeffer and sutton 2006). australasian journal of information systems cosic, shanks & maynard 2015, vol. 19, pp. s5-s19 business analytics capability 16 embeddedness (c6) extent to which ba has permeated the fabric of an organisation e.g. business processes and values (e.g. appreciation for ba analysis tools and data-driven insights). it is reflected in the extent to which people routinely use data and ba tools to solve problems and make decisions. it is facilitated by sharing metadata and the use of a collaboration portal. the portal enables work to be shared and intellectual property to be spread throughout the organisation. where appropriate, models are used to make decisions on an ongoing and pervasive basis (shanks et al. 2012). executive leadership and support (c7) ability of senior managers and executives to advocate the use of ba systems and data-driven decision-making throughout the organisation. this requires (i) a clear vision, (ii) first-hand experience and understanding of the benefits and successes of ba and (iii) the promotion of this vision and understanding throughout the organisation, and (iv) the provision of financial and material support for ba initiatives (laursen and thorlund 2010). communication (c8) ba personnel across the organisation foster a culture of open communication and trust between themselves and other business users. this involves listening carefully to the needs of business users and translating ba concepts into every-day business language. it is facilitated by close and frequent contact via a variety of different communication channels (davenport and harris 2007). data management (c9) mechanism for (i) sourcing data for ba initiatives from multiple channels, including operational/transactional systems and third-party sources, (ii) ensuring its quality e.g. consistency, accessibility, flexibility, integrity, timeliness and availability and (iii) integrating it with existing data in a central repository e.g. enterprise data warehouse. it also includes master data management and metadata management to ensure data definitions are consistent across organisational units to encourage common usage and understanding of the data (watson and wixom 2007). systems integration (c10) seamless integration of ba systems with operational/transactional systems at the process, technology and data levels in order to exploit the capabilities of both. systems integration is important for leveraging value from ba and is facilitated by the flexible design of technology infrastructure and systems architecture. it also introduces a degree of complexity and therefore should be done with care and careful consideration of the need (sharma and shanks 2011). reporting and visualisation ba technology (c11) ability to develop and utilise self-service analysis applications e.g. reports, dashboards, scorecards, online analytical processing (olap) and data visualisation technologies, which display output in a userfriendly format that is readily understood by non-technical users. these applications are particularly useful for addressing structured problems and facilitate the visual manipulation and exploration of data (watson and wixom 2007). australasian journal of information systems cosic, shanks & maynard 2015, vol. 19, pp. s5-s19 business analytics capability 17 discovery ba technology (c12) ability to develop and utilise quantitative and qualitative analysis tools (e.g. statistical analysis, data mining, text mining and predictive analysis) to facilitate the semi-automated analysis of numerical, semistructured and unstructured data to; (i) discover new actionable insights from patterns in the data, and (ii) extrapolate patterns found in the data to predict what is likely to occur in the future. these tools are particularly useful for addressing less structured problems (negash 2004). technology skills and knowledge (c13) combined skills and knowledge of ba technology specialists across the organisation including; programming, optimisation software, algorithms, database/file management, etl (extraction, transformation and loading), data warehousing, software development methodologies and high level architectures. some level of business domain and industry knowledge is necessary to apply these skill sets. furthermore, teams should consist of specialists whose skills are complementary to other team members (davenport and harris 2007). business skills and knowledge (c14) combined skills and knowledge of people throughout the organisation that are involved in the business side of ba initiatives including; (i) fundamental business principles, and (ii) depth of domain knowledge of the organisation’s key products, services, processes, value chain and industry in general. it also includes the ability to; (i) network, (ii) seek out opportunities and threats, and (iii) develop and drive an agenda. some level of technical expertise is necessary to understand the data available to them and communicate with ba technical specialists (anderson-lehman et al. 2004). management skills and knowledge (c15) combined skills and knowledge of people in ba related management roles throughout the organisation to (i) prioritise and manage ba projects, (ii) redesign business processes as a result of implementing ba, and (iii) translate, communicate and sell the potential values and benefits of ba to senior executives (e.g. senior executives and general managers). some level of technical expertise is necessary to understand the data available to them and communicate with ba technical specialists (davenport and harris 2007). entrepreneurship and innovation (c16) combined skills and knowledge of ba managers and other ba users throughout the organisation to (i) continually challenge the status quo, (ii) manage new innovation as a separate activity to continuous improvement, (iii) create and promote a technical innovation team, as well as (iv) an innovation forum made up of innovation teams from other business units. it is characterised by an entrepreneurial mindset and vision and the ability to rationally assess risks and benefits. it is enhanced through the provision of some authoritative autonomy and financial independence, which provides ba managers with a degree of freedom to pursue value-creating actions. (sharma et al. 2010). copyright: © 2015 cosic, shanks & maynard. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems cosic, shanks & maynard 2015, vol. 19, pp. s5-s19 business analytics capability 18 which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. ajis vol. 4 no. 2 may 1997 conceptual data modelling: an empirical study of expert and novice data modellers graeme shanks department of information systems monash university melbourne, australia email: graeme.shanks@is.monash.edu.au abstract this study explores the differences between conceptual data models designed by expert and novice data modelling practitioners. the data models are evaluated using a number of quality factors synthesised from previous empirical studies and frameworks for quality in conceptual modelling. this study extends previous studies by using practitioners as participants and using a number of different quality factors in the evaluation. the study found that data models produced by expert data modellers are more correct, complete, innovative and flexible than those produced by novices. the results suggest that further research into the aspects of expertise that lead to such differences and how training courses can narrow the gap between expert and novice performance is required. introduction conceptual data models provide a "precise and unambiguous representation of organisational information requirements" (kim and march 1995, p!03). they also provide a means of communication amongst information systems professionals and business users when discovering and modelling information requirements. entity-relationship (er) models are a widely used form of conceptual data model (chen 1976). the development of a conceptual data model is a design activity influenced by various kinds of knowledge including both application domain knowledge and data modelling representation and process knowledge. when designing a conceptual data model, expert data modellers develop a holistic understanding of a problem, categorise problem descriptions into standard abstractions and reuse generic data models from their previous experience (batra and davis 1992, chaiyasut and shanks 1994). novice data modellers follow simple guidelines such as "make a list of all the nouns ..." (veryard 1984, pi5). many previous empirical studies of conceptual data modelling have focused on comparing representation formalisms or have traced the process of the modelling task. this study explores the differences between conceptual data models designed by expert and novice data modelling practitioners using the entity relationship modelling formalism. the data models are evaluated using a number of quality factors synthesised from previous empirical studies and frameworks for quality in conceptual modelling. this study extends previous studies by using practitioners as participants and using a number of different quality factors in the evaluation. the results suggest that further research into the aspects of expertise that lead to such differences and how training courses can narrow the gap between expert and novice performance is required. the paper is structured as follows. the next section of the paper reviews previous empirical studies in conceptual data modelling. the third section describes the research approach adopted in the study. the following section discusses the results of the study and the final section of the paper presents a number of implications of the study and suggests areas for further research. previous empirical studies in conceptual data modelling there have been a number of previous empirical studies of conceptual data modelling. these studies have investigated the effects of different data modelling formalisms (juhn and naumann 1985, jarvenpaa and machesky 1986, shoval and even-chaime 1987, batra et al. 1990, kim and march 1995) and expert and novice performance in conceptual data modelling in terms of both product and process (batra and davis 1992, maiden and sutcliffe 1992, shanks et al. 1993, chaiyasut and shanks 1994). these studies vary in the participants involved, the data modelling formalisms used, the kinds of task undertaken and the focus of the study. a comparative summary of these studies is presented in table 1 (adapted and extended from kim and march 1995, pi04). it can be seen that many of the participants involved in the studies were students, the most commonly used data modelling formalism was entity relationship modelling, the most common task was to build a conceptual data model from a small case description and the focus of most studies was on the quality of the conceptual data model designed. 63 ajis vol. 4 no. 2 may 1997 table 1 summary of previous empirical studies in conceptual data modelling study juhn and naumann (1985) jarvenpaa and machesky (1986) shoval and evenchaime (1987 mantha (1987) eatraetal. (1990) batra and davis (1992) maiden and sutcliffe (1992) shanks et al. (1993) chaiyasut and shanks (1994) kim and march (1995) participants mis mba students (novices) is undergrad. students (analysts) is grad. students (analysts) practitioners (analysts) is undergrad. students (novices) students and practitioners, (expert/novice) grad. is students (novices) grad. is students and is practitioners (expert/novice) is practitioners (expert/novice) grad. is students and is practitioners (analysts) data modelling formalisms er (and lds) relational (and dad) lds relational object-role relational er er, relational er dfd er er eer, object-role task comprehension, modelling modelling modelling modelling modelling modelling modelling modelling modelling modelling, model validation focus quality of data model quality of data model quality of data model, modelling time quality of data model quality of data model cognitive process cognitive process, quality of data model quality of data model cognitive process comprehension, quality of data model those studies which investigated the effects of different data modelling formalisms found that semantic data models (such as er) lead to the design of better quality data models than the formalisms based on storage structures (such as relational). other studies traced the cognitive process of expert and novice data modellers in the conceptual data modelling task. these studies showed that expert data modellers were better able to conceptualise and understand the case description than novices. this enabled them to partition the case description into sub-problems, reuse patterns from their previous experience, spend more time evaluating their solutions and move their solutions to higher levels of abstraction. both maiden and sutcliffe (1992) and shanks et al. (1993) evaluated the quality of conceptual data models designed by expert and novice data models. in each case the models developed by experts were found to be of higher quality than those of novices. this study extends these previous studies in several ways: • all participants in the study are information systems practitioners rather than students. this should increase the external validity of the results (hoffer 1982). • the evaluation of the models was more comprehensive and based on several quality factors synthesised from a number of quality frameworks. • the er formalism used was the martin (1987) standard, which is widely used in practice, rather than the chen standard commonly used in other studies (hitchman (1995) found that only 5% of practitioners used the chen notation). 64 ajis vol. 4 no. 2 may 1997 research approach the research approach used is a "differential study" (graziano and raulin 1989). a differential study is in many ways similar to an experiment although in a true experiment participants are randomly assigned to groups. in a differential study groups are formed from values of pre-existing variables. research model the research model for this study is shown in figure 1 and depicts the relationships between the participants, the task and the quality of the data models. participants expert novice case example representation task focus of study data model correctness completeness innovation flexibility understandability overall quality figure 1 research model the task each participant in the study was required to read a case study description and prepare a conceptual data model. the case study problem was based on a small electronics trading and repair company. the description of the case study problem was provided by the manager of the company using a tape recorder and later transcribed into written form. participants worked entirely from the narrative; there was no "user" present to answer questions. because the case example described a common, familiar type of company there was no need to have any specialised domain knowledge to understand its operations. each participant was also required to complete a brief questionnaire after the data modelling task containing details of their industry experience and training. completion of the task and questionnaire took between one and two hours. the participants independent variable the participants in this study were all practising information systems professionals. each of the participants had attended a three day course in entity relationship modelling. expertise in data modelling is defined by the number of years of experience as a specialist data modeller together with the number of data models to which the participant contributed. participants with at least four years experience as specialist data modellers and who had contributed to the development of at least ten conceptual data models were categorised as expert. other participants were categorised as novice. it can be seen that the definition of an expert data modeller is very stringent. data model quality dependent variable quality is defined as "...an emergent property measuring the extent to which the [model] is fit for use and meets the customer's expectations" (kaposi and kitchenham 1987, p3). moody and shanks (1994) note that there are two aspects to understanding quality in conceptual data modelling; process quality 65 ajis vol. 4 no. 2 may 1997 and product quality. process quality relates to understanding and improving the process of building a conceptual data model. product quality refers to understanding and evaluating the data model which is built. this study focuses on product quality. a number of frameworks have been proposed which can be used to understand and evaluate quality in conceptual data models (see for example shanks et al. 1993, lindland et.al. 1994, krogstie et al. 1995, moody and shanks 1994 and simsion 1994). in addition a number of previous studies have developed criteria for comparing conceptual data models using different notations (see for example batra et al. 1990, mantra 1987, shoval and even-chaime 1987, kim and march 1995, hoffer 1982). the quality factors used in this study are synthesised from these previous studies and include correctness, completeness, innovation, flexibility and understandability. in addition, an "overall quality" rating is obtained for each model. this is to ensure that "transcendental" qualities which cannot be factored out are considered (kaposi and kitchenham 1987). each of these quality factors and its evaluation is defined below. a combination of objective measures and subjective ratings is necessary when measuring quality within the software domain (kaposi and kitchener (1987). subjective quality ratings can be strengthened in three ways (moody and shanks 1994): • by providing clear and precise definitions of the quality criteria • by using many reviewers and applying inter-rater reliability measures • by getting expert and experienced reviewers to carry out the assessment. a definition for each quality criteria used in this study and a discussion of its measurement follows. (a) correctness the correctness of a conceptual data model concerns its syntactic representation (lindland et al. 1994). correctness is defined as the degree to which the data model conforms to the syntax rules of the particular notation used. all participants were trained in the variant of the entity relationship notation defined in martin (1987). the evaluation instrument used for correctness is based on those developed by ridjanovic (1986), batra et al. (1990) and kim and march (1995). evaluation of correctness is based on the number of major and minor errors found in the data models. major errors include no name for entities or relationships, no cardinality constraint on relationships and no optional/mandatory symbol on relationships. minor errors include duplicate entity or relationship names, non-noun entity names and incorrect symbols. a major syntactic error is assigned a 0.5 penalty and a minor syntactic error is assigned a 0.25 penalty (kim and march 1995). the algorithm for correctness (expressed as a percentage) is: correctness = n o.se 0.25e x 100 where n is the number of instances of constructs (entities and relationships) in the model, ei is the number of major errors for each construct and £2 is the number of minor errors for each construct. the overall correctness is obtained by averaging the participant's performances for each modelling construct. because all participants have had equivalent training and the entity relationship notation has a relatively simple syntax, it is expected that there should not be any significant differences in data model correctness between the two groups. hence the following hypothesis is posited: hi there will be no difference in the correctness of data models built by expert data modellers from those built by novice data modellers. (b) completeness the completeness of a conceptual data model concerns its perceived semantic content (krogstie et al. 1995). completeness is defined as the extent to which the data model supports user information requirements. the evaluation instrument used for completeness is based on those developed by ridjanovic (1986), batra et al. (1990) and kim and march (1995). evaluation of completeness is based on a comparison of each of the data models with a "correct" solution developed by the author and another data modelling academic. the "correct" solution determines the features which should be present in each of the data models. kim and march (1995 p!09) note that "... a required construct is considered to be present if there is any semantically equivalent construct in the participant's data model. for example, a concept represented by an entity in the "correct" model could be represented as ... a relationship in the [participants] model" 66 ajis vol. 4 no. 2 may 1997 evaluation of completeness is based on the number of major and minor errors found in the data models. major errors include missing entities or relationships. minor errors include incorrect extra or redundant entities or relationships, incorrect cardinality and incorrect degree. a major syntactic error is assigned a 1.0 penalty and a minor syntactic error is assigned a 0.3 penalty (kim and march 1995). the algorithm for completeness (expressed as a percentage) is: completeness = n e ) 0.3e2 x 100 where n is the number of constructs (entities and relationships) in the "correct" model, e\ is the number of major errors for each construct and £2 is the number of minor errors for each construct. the overall completeness is obtained by averaging the participant's performances for each modelling construct. it is expected that the data models designed by experts should be more complete than the data models developed by novices. hence the following hypothesis is posited: h2 expert data modellers will design data models of higher completeness than novice data modellers. (c) innovation the level of innovation in a conceptual data model is defined as the extent to which "new concepts" are introduced into the model (shanks et al. 1993). these "new concepts" must be relevant and valid for the model. a data model is said to be "literal" if it only includes concepts directly mentioned by the user. in this study a data model which included only entity types which matched nouns in the case description was deemed to exhibit no innovation. those data models which included entity types which were not explicitly included in the case description were considered innovative. in order to evaluate innovation, a list of nouns was developed for the case example narrative. a count was made of the number of entity types in each of the data models which did not correspond to nouns in the noun list. those entity types which were clearly only resolving many-to-many relationships and did not add any new concepts to the models were not counted. the ratio of "innovative" entity types to the total number of entity types in each data model was used as a measure of the level of innovation in the data models. the algorithm for innovation (expressed as a percentage) is: innovation = ni_ x 100 nt~ where ni is the number of "innovative" entity types and nt is the total number of entity types. previous empirical studies have shown that experienced data modellers recognise similar problem categories and are able to reuse data model patterns (batra and davis 1992, chaiyasut and shanks 1994). these patterns may include entity types not included in the case example narrative and thus will be seen as innovations. experienced data modellers frequently use the aggregation and generalisation abstractions to introduce new concepts to data models (chaiyasut and shanks 1994, simsion 1994). hence the following hypothesis is posited: h3 expert data modellers will design data models with higher innovation than novice data modellers. (d) flexibility flexibility is defined as the ease with which the data model can reflect changes in requirements without changing the data model itself. other terms for flexibility include extensibility, stability and evolvability (moody and shanks 1994). flexibility is one of the most important characteristics of a data model (avison and fitzgerald 1995, leviten and redman 1995) and has a major effect on the maintenance costs of a system (moody and shanks 1994). the use of generic concepts and the representation of business rules as structures in the data model will lead to a more flexible data model (barker 1989, simsion 1994). there is no objective measure for the flexibility of a data model. flexibility is given subjective expert ratings using a seven point likert scale. the following hypothesis is posited: h4 expert data modellers will design data models with higher flexibility than novice data modellers. 67 ajis vol. 4 no. 2 may 1997 (e) understandability understandability relates to the ease with which concepts and structures in the data model can be understood by stakeholders in the data modelling process. each stakeholder will interpret and understand the model in different ways (krogstie et al. 1995). the ratings reflect the ability of the reviewers to understand the data models and were given using a seven point likert scale. because expert data modellers are expected to produce more correct and complete data models, it is expected that the data models designed by experts should be more understandable than the data models developed by novices. hence the following hypothesis is posited: h5 expert data modellers will design data models that are more understandable to the reviewers than the models of novice data modellers. (f) overall quality overall quality is a measure of the raters' subjective evaluation of the model in total. this measure is included to account for the "transcendental" properties of the data models which are not included in the other quality factors (kaposi and kitchenham 1987). alexander (1979) refers to these properties as "the quality without a name". it is expected that the data models designed by experts should have an overall quality greater than the data models developed by novices. hence the following hypothesis is posited: h6 expert data modellers will design data models of higher overall quality than novice data modellers. data analysis and selection of statistical tests each of the three reviewers of the models was an academic specialising in the area of conceptual data modelling who also had previous industry experience in conceptual data modelling. however, none of the reviewers would have met the criteria for inclusion in the expert data modeller category. to enhance the validity of the evaluation all data models were transcribed using the same template so that the evaluation would focus on only the content of the data models rather than the presentation. to avoid sequencing bias each of the raters evaluated the models in a different sequence. inter-rater reliability (cronbach alpha co-efficient) was determined for all subjective ratings and for any scoring which involved expert judgement. the data collected from the evaluation is mostly interval data (correctness, completeness and innovation). likert scale ratings for understandability and flexibility are ordinal, however the treatment of ordinal data as interval data is not uncommon in information systems and social science research (periasamy 1994, pervan and klass 1992). the seven point likert scales had literal descriptions of only the mid-point and end-points thus providing no descriptive influence on the distances between the points on the scale. accordingly, mean, standard deviation and t-test were used for statistical inference and correlation was used for exploratory data analysis. the statview statistical software tool was used for all statistical analysis. results participant characteristics a summary of participant characteristics is shown in table 2. it can be seen that the categorisation of participants as expert or novice (using the criteria defined in section 3.3) agrees well with their self assessment. 68 ajis vol. 4 no. 2 may 1997 table 2 summary of participant characteristics years data modelling experience (mean/stdev) number of data models practical expertise (self assessment) theoretical expertise (self assessment) expert: 18 participants 6.41 (2.85) greater than ten three to ten one or two none expert novice expert novice 18 0 0 0 13 5 11 7 novice: 21 participants 1.52 (0.93) greater than ten 0 three to ten one or two none expert novice expert novice 21 21 3 12 6 0 0 hypothesis testing results for each of the six hypotheses tested are summarised in table 3. for all hypotheses, tests for significant differences between the two groups were conducted. in addition, for understandability and flexibility the mean value was tested for significant difference from a normally distributed population with a mean value of 4.0 (mid-point on likert scale). the first hypothesis was rejected (p=0.0137, significant with alpha = 0.05) with high inter-rater reliability. expert data modellers produced data models which were more correct than data models produced by novice data modellers. this result could be explained by the more frequent use of the data modelling notation by the expert data modellers despite both groups receiving the same recent three day training course. considering the simple syntax of the entity relationship model, the average scores for both experts and novices were quite low. in order to explore whether any particular facet of the notation contributed most to the difference a second scoring of the data models was conducted (see table 4). only 53% of expert data modellers and 29% of novice data modellers used relationship names. this is of great concern as relationship names are essential in documenting the business rule which relationships represent. a possible explanation is that the notation used in this study uses lines to represent relationships. representing relationships as diamonds may increase their prominence in the model and encourage naming. table 3 summary of hypothesis testing results hypothesis correctness (%) completeness (%) innovation (%) flexibility (7 point likert) understandability (7 point likert) overall quality (%) expert (mean/stdev) 84.33 (17.24) 72.89 (14.04) 17.00(16.45) 4.18 (0.81) 3.96 (1.03) 58.28 (13.97) novice (mean/stdev) 71.52 (17.47) 56.00 (13.52) 6.00 (8.34) 3.32 (0.75) 3.53 (0.77) 47.33 (1.177) inter-rater reliability (alpha) 0.97 0.92 0.93 0.82 0.73 0.87 tvalue t(37)=2.29 t(37)=3.88 t(37)=2.69 t(37)=3.44 t(37)=1.50 t(37)=2.66 p-value 0.0137 0.0002 0.0053 0.0007 0.0705 0.0058 hypothesis two was strongly supported (p=0.0002, significant with alpha = 0.01) with good inter-rater reliability. as expected, data models built by expert data modellers are more complete than data models built by novice data modellers. the average score of the novice group of 56% was quite low and indicates the importance of experience in the data modelling task. the third hypothesis was supported (p=0.0053, significant with alpha = 0.01) with good inter-rater reliability. data models built by expert data modellers have higher innovation than data models built by novice data modellers. the innovative concepts introduced into data models by experts may be explained by previous studies of data modelling process which showed that expert data modellers are able to categorise problem descriptions into standard abstractions and reuse generic data models from their previous experience (batra and davis 1992, chaiyasut and shanks 1994). this is in contrast to novice data modellers who developed "literal" data models from first principles. 69 ajis vol. 4 no. 2 table 4 summary of correctness results may 1997 facet entity name (%) relationship name (%) cardinality (%) mandatory / optional (%) expert (mean/stdev) 100.00 (0.0) 53.89 (50.08) 100.00 (0.0) 86.67 (20.29) novice (mean/stdev) 100.00 (0.0) 29.52 (19.55) 80.95 (9.09) 64.29 (45.56) t-value t(37)=1.61 t(37)=2.68 t(37)=1.92 p-value 0.0575 0.0055 0.0310 hypothesis four was strongly supported (p=0.0007, significant with alpha = 0.01) with adequate interrater reliability. it can be seen however that the flexibility rating for experts was only just above the mid-point of the likert scale and that for novices was significantly lower than the mean (p=0.0002, significant with alpha = 0.01) indicating that data models produced by novices have very low flexibility. flexibility in data models is indicated by use of generalised concepts and business rules stored as data values in the data model. both these data modelling strategies lead to the data model becoming more abstract. this result may be explained by process tracing studies which showed that expert data modellers move their solutions to higher levels of abstraction whereas novice data modellers do not (chaiyasut and shanks 1994). hypothesis five was supported (p=0.0705, significant with alpha = 0.1) with adequate inter-rater reliability. it can be seen that the understandability rating for both experts and novices was below the mid-point of the likert scale. the novice rating in particular was significantly lower than the mean (p=0.0052, significant with alpha = 0.01) indicating that data models built by novices have very low understandability. the low level of understandability of both expert and novice data models is consistent with empirical studies which question the useability of conceptual data models (goldstein and storey 1990 and hitchman 1995). a possible explanation for the low understandability of data models built by expert data modellers is the innovative and abstract concepts they contain together with the absence of many relationship names. the poor understandability of data models built by novice data modellers may be explained by their low completeness rating and poor use of relationship names. the sixth hypothesis was supported (p=0.0058, significant with alpha = 0.01) with good inter-rater reliability. as expected data models built by expert data modellers are of higher overall quality than data models built by novice data modellers. interactions between quality factors a correlation analysis was conducted to determine any interrelationships between the quality factors using pearson product moment correlation. the results are shown in table 5 below. the observed correlations are substantively less than 1 suggesting that the quality factors represent separate but correlated measures of quality (gable 1996, p258). there were strong positive correlations between overall quality and understandability (.757, p<.001), completeness (.643, p<.001), and correctness (.711, p<.001). these three quality factors are also the three important goals in the theory-based quality framework of krogstie et al. (1995). other interesting correlations were found between understandability and correctness (.657, p<0.001) and understandability and completeness (.442, p<.001). krogstie et al. (1995) note that for a conceptual model to be readily understood it should be correct and complete. the correlation between flexibility and innovation (.628, p<.001) indicates that many of the innovative concepts introduced into data models be specialist data modellers may be due to the use of abstraction to make the data models more flexible. an example is the introduction of the concept invoicable item by one expert data modeller as a generalisation of the entity types sale and repair. 70 ajis vol. 4 no. 2 may 1997 table 5 results of correlation analysis correct complete understand innovation flexible complex overall correct 1.000 .643 **** .657 **** .395 ** .650 **** .095 .711 **** complete 1.000 .442 *** .563 **** .690 **** .491 *** 729 **** understan d 1.000 -.146 .305* -.191 .757 **** innovatio n 1.000 .628 **** .383* .121 flexible 1.000 .303* .532 *** complex 1.000 .167 overall 1.000 note: **** p<.001 p<.01 p < 0.05 p<0.10 a small negative correlation was found between understandability and innovation (-0.146). this indicates that the introduction of new, innovative concepts into the data model may mean that the model becomes more difficult to understand. this is an important result for specialist data modellers who use the generalisation abstraction to make their data models more flexible but at the cost of understandability. the other negative correlation, between understandability and complexity, indicates that simpler data models may be easier to understand. however, neither of these negative correlations are statistically significant. conclusion this study found that data models built by expert data modellers are more correct, complete, innovative, flexible and better understood than those built by novices. the greatest differences between the models were found in completeness, innovation, flexibility, and in the naming of relationships. these results were related to previous empirical studies of data modelling process which offer insights into why these differences may occur. limitations empirical studies of this type have inherent limitations (mantra 1987). the validity and generality of the results should be considered in the light of a number of factors. the participants were a group of thirty nine data modelling practitioners who had each attended a three day data modelling course and were prepared to participate in the study. the extent to which they are representative of the population of data modellers needs to be verified by further similar studies. the data modelling task was undertaken with the data modellers unable to discuss requirements with a client. however, since all participants were subjected to the same limitation the effect of this limitation is minimised. some participants may have lacked knowledge about the domain of the case problem, although the particular problem used was selected to minimise this limitation. further research additional research into how aspects of process and expertise lead to the differences in quality of data models is required. outcomes from this research could be used in the design of training courses which narrow the gap between expert and novice performance. further research is also required in how to improve understanding of conceptual data models. krogstie el al. (1995) suggest a number of means to improve understanding of conceptual models, including explanation. two approaches to explanation which may help are the use of scenarios (carrol 1995) and argumentation-based design rationale (buckingham shum and hammond 1994). the author is currently developing a methodology for designing and representing conceptual data models which synthesises potts et al. (1994) inquiry based life cycle model for capturing scenarios and design rationale with the viewpoint development and integration process model of darke and shanks (1996). further research will involve empirical studies of the use of the methodology in designing and representing conceptual data models. references alexander, c. (1979) the timeless way of building, oxford university press avison, d. and fitzgerald, g. (1995) information systems development: methodologies, techniques and tools (2nd edition), oxford, blackwell scientific barker, r. (1989) case*method: entity relationship modelling, addison-wesley 71 ajis vol. 4 no. 2 may 1997 batra, d., hoffer, j.a. and bostrom, r.p. (1990) comparing representations with relational and eer representations, communications of the acm (feb.) 33:2, p!26-139 batra, d. and davis, j.g. (1992) conceptual data modelling in database design: similarities and differences between novice and expert designers, int. journal man-machine studies, 37, p83101 brodie, m.l., mylopoulos, j. and schmidt. j.w. (eds.) (1984) on conceptual modelling, springerverlag, new york buckingham shum, s. and hammond, n. (1994) argumentation-based design rationale: what use at what cost, int. journal human-computer studies, 40, p603-652 carrol, j.m. (ed.) (1995) scenario-based design, john wiley and sons, new york chaiyasut, p. and shanks, g. (1994) conceptual data modelling process: a study of novice and expert data modellers in t. halpin and r. meersman (eds.) proc. 1st intl. conf. on objectrole modelling, univ. of queensland darke, p. and shanks, g. (1996) stakeholder viewpoints in requirements definition: a framework for understanding viewpoint development approaches, requirements engineering journal, 1:2, p88-105 goldstein, r.c. and storey, v.c. (1990) some findings on the intuitiveness of entity-relationship constructs in f.h. lochovsky (ed.) entity-relationship approach to database design and querying, elsevier science, amsterdam graziano, a. m. and raulin, m. l. (1989) research methods: a process of enquiry , harper and row hitchman, s. (1995) practitioner perceptions on the use of some concepts in the entityrelationship model, european journal of information systems, 4, 31-40 hoffer, j.a. (1982) an empirical investigation into individual differences in database models in proc. 3rd intl. conf. on information systems, (december) p!53-168 juhn, s. and naumann, j. (1985) the effectiveness of data representation on user validation in proc. 6th intl. conf. on information systems, indianapolis, p212-226 jarvenpaa and machesky (1989) data analysis and learning: an experimental study of data modelling tools, intl. journal man-machine studies, 31, 367-391 kaposi, a. and kitchenham, b. (1987) the architecture of system quality, software engineering journal, 2:8 kim, y-g. and march, s.t. (1995) comparing data modelling formalisms, communications of the acm, (june), 38:6, p!03-l 13 konsynski, b.r. (1979) database driven systems, university of arizona krogstie, j., lindland, o.i. and sindre, g. (1995) towards a deeper understanding of quality in requirements engineering proc. caise leviten, a. and redman t. (1995) quality dimensions of a conceptual view, information processing and management, 31:1, p81-88 lindland, o., sindre, g. and solvberg, a. (1994) understanding quality in conceptual modelling, eeee software, (march) 42-49 maiden, n. and sutcliffe, a. (1992) analysing the novice analyst: cognitive models in software engineering, intl. journal man-machine studies, 367, pp 719-740 mantra, r.w. (1987) data flow and data structure modelling for database requirements determination: a comparative review, mis quarterly, (december) p531-545 martin, j. (1987) recommended diagramming standards for analysts and programmers, prentice-hall, new jersey moody, d. and shanks, g. (1994) what makes a good data model? evaluating the quality of entity relationship models in p. loucopolis (eds.) proc. thirteenth international entity relationship conference, manchester, england moody, d., simsion, g., shanks, g., venable, j. and olson, n. (1995) stakeholder viewpoints in conceptual data modelling, proc. 6th australasian conference on information systems, curtin university of technology periasamy, k.p. (1994) development and usage of information architecture, phd dissertation, university of oxford pervan, g.p. and klass, d.j. (1992) the use and misuse of statistical methods in information systems research in r.d. galliers (ed.) information systems research: issues, methods and practical guidelines, blackwell, oxford p208-229 potts, c., takahashi, k. and anton, a.i. (1994) inquiry-based requirements analysis, ieee software, (march) p21-32 72 ajis vol. 4 no. 2 may 1997 ridjanovic, d. (1986) comparing quality of data representations produced by non-experts using logical data structures and relational data models, phd dissertation, university of minneapolis shanks, g., simsion, g. and rembach, m. (1993) the role of experience in conceptual data modelling, proc. 4th australian conf. on information systems, university of queensland shanks, g. (1997) the challenges of strategic data planning in practice: an interpretive case study, the journal of strategic information systems, forthcoming shoval, p. and even(thaime, m.. (1987) database schema design: an experimental comparison between normalisation and information analysis, database, 18:3 (spring) p30-39 simsion, g.c. (1994) data modelling essentials, von nostrand reinhold van fliet, j.c. (1993) software engineering: principals and practice, john wiley and sons, chichester, england 73 ajis vol. 7, no 2 may 2000 understanding organisational learning in military headquarters: findings from a pilot study leoni warne joint systems branch information technology division defence science and technology organisation c3 research centre fernhill park bruce act 2617 e-mail: leoni.warne@dsto.defence.gov.au abstract this paper reports on the findings of a pilot study for a longer term research project investigating organisational learning in command and control organisations in the australian defence force. one of the long-term objectives of the research project is to facilitate the development of future information systems that will enable organisational learning. the aim of the pilot study was to determine the feasibility of observing and documenting social learning processes and to trial the use of ethnographic techniques for this purpose. introduction understanding and enabling organisational learning in australia's military organisations is one of the objectives of the enterprise social learning architectures (esla) research project being conducted by the defence science and technology organisation (dsto). esla is a three year research study investigating social learning within the australian defence force (adf). the study is designed to enhance understanding of social learning within the command and control structures of the adf and ultimately, to develop frameworks, or architectures, of the most effective social learning practices within the enterprise. these architectures will then be used to guide the development of systems that will closely support the way the enterprise learns, giving the adf a strategic advantage in organisational learning and more tools to help achieve and maintain crucial knowledge management practices. social learning is a term generally used to describe learning that is done in or by a group, or an organisation, or any other cultural cluster; and to distinguish it from learning that is purely individual. for the purposes of the study, social learning is defined to include: the procedures by which knowledge and practice are transmitted across posting cycles, across different work situations and across time; and the procedures that facilitate generative learning learning that enhances the enterprise's ability to adjust to dynamic and unexpected situations and to react creatively to them (senge 1992). a pilot study for the esla research study was conducted over a six month period in 1998. the objectives of the pilot were: to see if it was feasible to observe, understand and document social learning processes, particularly in military command and control situations; and to trial the use of ethnographic techniques for this purpose. the research team's findings on the ethnographic methodology are briefly discussed in this paper. the multi-layered social learning findings derived from the pilot study range from observations of the detailed structured and unstructured learning processes used to propagate the communities of practice in the pilot setting; through the models and disciplines that facilitate organisational learning; to the broader social issues of power and discipline that impinge on the effectiveness of social learning in the organisation. this paper, however, reports only on the organisational learning findings. theoretical foundations and research objectives in the past decade, there has been an increasing emphasis on investigating the social and organisational factors that may help the information systems industry build more successful systems. investigation of these issues necessitates a sound understanding of organisational culture, human social interactions, communication and relationships. this reflects an increasing awareness of the importance of the social aspects of socio-technical design, particularly when it comes to understanding users' requirements. consequently, there has been a revival of interest in socio-technical techniques, with an increasing emphasis on understanding the social aspects of work practices, and thereby giving individuals closer control over their work environment and the systems that are designed to support this work. the focus is 'user-driven' design and this is also the impetus for the enterprise social learning architectures research, which is designed to gain a better understanding of the social aspects of learning in the adf, in order to guide the development of technical systems that will support it (warne 2000). the enterprise social learning architectures research although the latest computer technology is freely available to adf personnel, and large supporting command systems are frequently specified and built, the major individual use made of information system technology in the adf appears to be for messaging and word processing. the high end laptop machines used by commanders and their supporting staff often tend to be used largely for preparing documents and for some self-developed spreadsheet or database applications. large command systems that are specified and then delivered several years later are not 78 ajis vol. 7, no 2 may 2000 always useful to the clients for whom they were built, because the way the clients work has changed since the systems were specified (warne 2000). the enterprise social learning architectures research is an attempt to circumvent this problem by understanding and, eventually, modelling the successful social learning patterns that underpin collaborative command and control work. the esla study is a long term project which aims to understand how personnel involved in this work learn to do their jobs, the way they use and pass on information, they way the work they are doing changes over time, the way they adapt to these changes, and how they react creatively in new situations. this is an attempt to move away from specifying systems from the point of view of information processes, or even task analysis, back to a basic understanding of how human beings work and how they learn. it is envisioned that once the components, patterns and inter-relationships of this form of social learning are identified, understood, and documented, the resulting architecture can drive the delivery of computer systems that support the way the users work. these systems could be based on intelligent software agents or on a newer form of software yet to be developed. rather than attempting to mould the users' work habits to fit the system, it will be possible to configure the system to suit the users' most productive work habits (warne 2000). a related outcome of the study is hoped to be the facilitation of learning organisations within the adf. learning organisations arc those in which people can continually expand their ability to create positive organisational outcomes, where innovation is nurtured, and where learning is a continuous, collaborative, supportive and supported process (senge 1992). if the adf can learn faster and better then its enemies, it will have a powerful and sustainable advantage over them. social learning while the term social learning has been used occasionally by the education and psychology disciplines, the term has been adapted and redefined for this research. the definition used is based on theories presented by jean lave, etienne wenger and the organisational learning literature. communities of practice lave and wenger (1991) highlight the fact that learning in the workplace is learning in a social world, that "learners inevitably participate in communities of practitioners and that the mastery of knowledge and skill requires newcomers to move towards full participation in the socio-cultural practices of a community" (lave & wenger 1991:29). the learning processes involved in newcomers efficiently acquiring the necessary knowledge to be effective in the workplace are an important part of this research, because of the adf's posting cycle. however, learning is an on-going process as work place practice develops and changes. wenger (1998) characterises the need for members of a community of practice to adjust and adapt to changes over time and in new situations as another manifestation of learning in a social world. organisational learning and learning organisations the organisational learning literature also deals with learning in the social world of the work place, and it is this aspect of the pilot study that is described in this paper. in the current, rapidly changing business and information environment, an organisation's ability to learn and adapt to change is highly valued. according to courtney, croasdell and paradice: "when members of an organization share associations, cognitive systems and memories, organizational learning is taking place. learning by organizations relies on the people and groups as agents for the transferal of knowledge. over time, what is learned is built into the structure, culture and memory of the organization" (courtney et al 1998:4) in this way, knowledge is assimilated into the organisation and is not lost when individuals are posted out or when situations change. this relates directly to the effectiveness of knowledge management in an organisation that, in turn, has implications for the development of systems that manage organisational knowledge effectively. bolk et al (1996) believe that the concept of the intelligent, learning enterprise has direct implications for the type of software and systems required to support it and they argue that this support should come from group decision support systems; intelligent software agents and team based system architectures. in his seminal work on organisational learning, senge (1992) states that in order for organisations to maintain a competitive edge, they must be capable of continuous learning and he outlines the five models, or disciplines, that will enable this. there is a potential synergy between senge's model of organisational learning and computer systems that facilitate and support that model. senge distinguishes between adaptive (or 'survival' learning) and generative learning, which he defines as learning that enables organisations to adjust to changing situations and enhances their capacity to create. this concept of generative learning is particularly relevant to command and control situations in the adf and has been included in the definition of social learning for the esla research. 79 ajis vol. 7, no 2 may 2000 the concept of a 'learning organisation' has been defined and described in a variety of ways by different scholars. garvin states that learning organisations are organisations that are skilled at five primary activities: "systemic problem solving, experimentation with new approaches, learning from the experiences and best practices of others, and transferring knowledge quickly and efficiently throughout the organisation" (garvin 1993:81). according to hoffman and withers (1995), learning is an autonomous, largely uncontrollable function of being human, however, the culture in which this learning occurs is the compelling determinant of the quality of that learning. in other words, the organisational culture either supports and steers learning, or it does not; and accordingly, an organisation is either a learning organisation, or it is not. furthermore, hoffman and withers believe that successful learning organisations must operate in an environment of trust and the reduction of fear; teamwork and sharing; leaders as champions of people and their ideas; and the encouragement of constant change. charles handy (1995) characterises learning organisations as being built upon an assumption of competence supported by four other essential qualities: curiosity, forgiveness, trust and togetherness. the many definitions of learning organisations are, surprisingly, not contradictory. rather, they seem to be variations on the same theme. the common factor in most of these definitions and characterisations is the belief that the key to an organisation's success and longevity is in its ability to effectively adapt to its changing environment and to efficiently nurture the growth, sharing and sustenance of the corporation's historical and dynamic knowledge. these factors are present in senge's 1992 discourse on the disciplines of learning organisations. therefore, senge's seminal work was selected as one of the theoretical foundations against which the pilot study findings were evaluated, in an attempt to determine the extent to which these disciplines occur in the pilot study setting. research methodology studies which identify the need to consider the socially constructed context in research tend to support qualitative techniques where the key analytic task is to expose the multiple layers of meaning in the social process. since context is crucial to qualitative observations and analyses, techniques that effectively explore contextual complexities of meaning are important. the main body of techniques that fit this criteria fall under the domain of ethnographic approaches (harvey & myers 1995). given the exploratory nature of the pilot study research, the importance of the context arid the need to understand the social process of learning, ethnography appeared to be the most logical and viable form of methodology to adopt. ethnography is not a new research methodology; it has been standard practice in anthropology for decades. however, it is now becoming more widely and successfully used in information systems research (myers 1999). ethnographers try to immerse themselves in the situation they are researching, in an attempt to obtain a deeply contextual view. from this immersion, the ethnographer is able to gradually see and understand the key concepts that influence the setting being studied. ethnographers take on a learning and interpretive role, rather than attempting to test hypotheses. the primary analytic task is to uncover and explicate the way in which people manage their day-today situations (miles & huberman 1994). the setting for the pilot study the setting chosen for the study was a wing command of the royal australian air force strike reconnaissance group (srg). five field trips were made over a six month period: four to the wing headquarters at the strike reconnaissance group at amberley air base in queensland, and one to darwin. in darwin, personnel from the amberley wing headquarters were joined by members of other wing headquarters, to form 95wing hq, the headquarters formed to take the role of 'the enemy' in a military exercise (warne 2000). in this way, the research team was able to observe the organisation under both peace and 'war' conditions. strike reconnaissance group much of the fieldwork was conducted at srg's normal peacetime environment at amberley air force base. work at amberley focuses on training, and maintaining capability. the primary mission of srg is to conduct strike missions. strike missions are conducted by aircrew flying f-l 11 aircraft. the strike aircrew consists of pilots and navigators, since flying an f-l 11 requires one pilot and one navigator. members of the aircrew world may be in flying positions or ground jobs. all aircrew take on secondary ground jobs. the strike aircrew community is refreshed by new members every six months (o'neill 1998). darwin military exercise the 95wing headquarters, in darwin, was a particularly interesting setting, as it was possible to observe how work was conducted during a war exercise, when stress levels were higher and errors and exceptions became more obvious. 80 ajis vol. 7, no 2 may 2000 the officer commanding (oc) 95wing, his executive officer (xo) and his support staff all came from the wing hq at amberley, and all had worked together as a team previously. the settings for most of the observations made of this 'war' were the operations room and the briefing room at 95wing hq. the operations room was immediately outside the oc 95wing's office, and, after the daily air tasking directive (atd) was received from hqac (headquarters air command), in sydney, the operations room was where the air tasking order (ato) was largely constructed. the atd reflected hqac's daily battle directives for the military exercise, the ato transformed these into objectives for flying missions, the mission directors planned the missions that would meet these objectives, and the mission commanders implemented them. there was a table in the centre of the operations room, which was a focal point for producing the atos. at this table, laptops were available for the creation of documents, folders contained the latest paper versions of different types of supporting information, and a number of telephones were available for communicating with people in other areas on the base or in the 'war zone'. the table itself was used for checking on requirements, and conducting muted conversations and phone calls. in front of the table, on two free standing noticeboards set at a slight angle, was a paper map of the 'war zone', and notices of the latest information required to plan missions and score targets. the maps were annotated with plastic overlays, so that missions and target runs could be planned visually and changes could be immediately visible. several people were tasked with keeping this map, and the related notices, up-to-date with the latest information at all times. this map/notice board became the focal point for information transfer during the exercise. small groups would form around the map, break apart and then re-form as people gathered around the maps sorting out the information required to plan the atos and the missions (agostino et al 1999). this arrangement was extraordinarily successful, as the table and map became a focus for both formal learning and informal social learning about the missions and the war. the briefing room was used in a number of different ways. it was the venue for a series of formal pre-mission briefs and post-mission debriefs. the debriefings proceeded much as they had for peacetime missions at amberley, with tapes, photos and mudmaps showing engagements at different times, and a special emphasis on lessons learned. there was considerable emphasis on the learning opportunities provided by the exercise in darwin, and lessons learned were both formal and informal points of discussion after every mission cycle the briefing room was also an arena for semi-formal planning, and it provided a meeting place for people to discuss things informally (especially before and after briefs). the work of 95wing hq involved aligning activity systems to perform an afternoon and evening mission every day. to drive this alignment, a tight timeline was used to establish a rhythm of work to which all players needed to conform in order for missions to be successfully conducted. the timeline defined a strict 48-hour mission cycle which dictated planning, briefing and flying times for both daily missions (agostino et al 1999). the importance of adhering to the timeline was stressed by the executive staff. the timeline dictated the rhythm of the work and ensured the quality and completeness of the ato. it became clear that personnel who had been on previous, similar exercises managed to fit into the routine and rhythm of the work far more quickly then those who had not had some previous experience with this type of military exercise. some problems occurred early in the exercise when there was a mismatch in expectations between the operations room and another area which were tasked with keeping the operations room up-to-date. there was, initially, no shared understanding of the importance of the timeline as a driving force for the work. however, as all sections of 95wing aligned themselves with the timeline, the missions began to run more smoothly, and the stress level in the operations room began to decrease. discussion of the ethnographic methodology used field trips were organised so that the research team could spend several days at a time observing the work taking place in different settings within srg, using directed questioning to clarify any issues that were confusing or unclear. great care was taken to remain as unobtrusive as possible during the field trips. observations were written up in field notes, individually. these field notes were later circulated to all team members and discussed at team meetings. the preferred method for conducting observations was to pair ethnographers in each setting. the ethnographic study began with a team of four researchers: three computer scientists and one social scientist. it ended with a team of four researchers: two computer scientists, one social scientist and one researcher from an information systems background. the field notes of different ethnographers revealed interesting contrasts in what each had observed. some ethnographers were information focused, identifying what was articulated in conversations, what artefacts were used, and the contents of these artefacts. other ethnographers focused on the interactions that occurred between people, the body language, the relationships, and how these relationships affected the discourse. the ideal pairing was between an ethnographer who was information-oriented and an ethnographer who was interactionoriented. this also seemed to be a gendered process where, on the whole, the women on the team were more interaction-oriented and the men more information focused (agostino et al 1999). without attempting to make any conclusions, it is also interesting to note that the information-oriented ethnographers were from a computer science or engineering background, while the interaction-oriented ethnographers were from a social sciences or information systems background. 81 ajis vol. 7, no 2 may 2000 the research team would meet daily and discuss observations and any issues arising from the day's fieldwork. team workshops were also held after each field trip to identify and consolidate the findings as the work unfolded and to identify key social learning issues across the field notes. before each team workshop, each individual researcher's field notes would be distributed to all team members. these team discussions, or workshops, provided a forum for negotiating across the different intellectual cultures of the team. these negotiations bridged the different backgrounds of each team member, their different experiences in the setting, the different meanings people assigned to what they saw, and the different issues that they perceived to have arisen from the work. individual team members viewed the work in different ways depending on their own experience, discipline, and theoretical base. the different perspectives of the team members enriched the data and the minutes of the team meetings became the objective record of the data analysis and the study findings (agostino et al 1999). constraints and limitations there were a number of constraints placed on the ethnographic aspects of the pilot study. over the period the study took place the composition of the team changed significantly with all but one team member leaving the team either permanently or for a period of time. limited funding for travel and fieldwork also curbed opportunities to observe the settings in greater depth. according to harvey and myers (1995), there are three major limitations to ethnographic research: the difficulty of generalising findings; the time factor; and, finally, the difficulty in presenting findings. furthermore, the research team found a further limitation arose because much of what was being investigated was often invisible until processes failed, or an exception occurred. the first limitation of ethnographic research identified by harvey and myers is that it leads to in-depth knowledge only of particular contexts and situations. while this is certainly true, harvey and myers also state that it is only true until a large body of knowledge of many situations is developed. this esla research study aims to collect a significant body of knowledge by going into a number of settings over a number of years. however, before conclusions are reached, in terms of generalised models, it may be necessary to employ supplementary methodologies to test the wider applicability of the findings. the second major limitation of ethnography identified by harvey and myers is the time commitment required to gather data and carry out many levels of interpretive analysis. however, as harvey and myers themselves conclude, despite the difficult and time-consuming nature of ethnographic research, it is a very 'productive' research method considering the amount and substance of the research findings. this is also the conclusion reached by the ethnographers involved in the pilot study. it is difficult to conceive of any other research methodology that would have resulted in the same multi-layered, rich pictures derived in the pilot. in the exploratory stages of this sort of new research, the time commitment appears to be well worth the results received. harvey and myers third, and final limitation is identified as dealing with the publication of results. with ethnographic research, they explain, the approach and the resultant nature of the knowledge is holistic, making the knowledge generated recursive rather than cumulative. this distinction is often not appreciated by those who operate through more traditional approaches. this limitation is yet to be tested by the esla research team. making the invisible visible an interesting limitation of the methodology identified by the research team was the fact that, for routine work, social learning was sometimes invisible. this essentially left the team with two options. either wait for exceptional conditions where the routine fails, exposing some of the invisible structures, or identify anomalies in the observations and follow up with direct questioning. both of these approaches proved successful although it was more reliable to use direct questioning followed by further ethnographic observations to facilitate understanding and to reveal the otherwise 'invisible' structures. the ethnographic approach to investigation is said to be most appropriate where research is of an exploratory nature, the context is of particular importance and there is a need to understand social processes all of which applied to the pilot study research. inevitably, there was an epistemological learning curve for the research team, which involved the team's own social learning. despite the limitations, the research team found ethnography a suitable and successful methodology for the purpose of this study. findings on learning organisation disciplines senge (1992) suggests that five disciplines will enable a learning organisation: personal mastery, mental models (or assumptions), shared vision, team learning, and systems thinking. 82 ajis vol. 7, no 2 may 2000 personal mastery senge acknowledges that while individual learning does not guarantee organisational learning, without it, organisational learning cannot occur. responsibility for learning must be accepted by each individual if the organisation is to create a learning organisation environment (hoffman & withers 1995). ideally, organisations should provide multiple avenues for individual and collaborative learning. the importance of individual learning is well understood in the srg environment, as the consequences of not doing a job properly can mean the difference between life and death. it appears that, in general, every member of the community knows they have a role to play and they understand the vital significance of their role, whether that role involves flying or maintaining an f-l 11, providing intelligence for a flying mission, planning a flying mission, or being responsible for the pastoral care of the aircrew who will fly a mission. however, senge's discipline of personal mastery includes more than a commitment to individual learning or expertise. personal mastery also involves an individual in "continually clarifying what is important" to them, and "continually learning how to see current reality more clearly" (senge 1992:141). this, according to sengc, means having an unconditional commitment to the truth; a realistic understanding of the gap between current reality and personal vision; and a clear understanding of the inter-relationships and interdependencies of the systems within which an individual operates. given this additional dimension, personal mastery is a discipline the research team observed within the executive world and the aircrew world of srg, particularly during the military exercise. to the extent that it is possible to generalise from the observations made, it appears that both the executive staff and the aircrew were very much committed to continual clarification of what was important and the need to see current reality more clearly. they were also very much aware of the interdependencies of the systems within which the exercise was being conducted. however, personal mastery was also observed in the breach. during the early stages of the military exercise, there was an obvious disconnection between one of the sections supplying information and the section planning missions. individuals within the information-supplying area did not, initially, have a clear understanding of the interrelationships and interdependences within the 95wing hq, nor a clear understanding of the gap between their current reality and personal vision. however, given the fact that this problem was quickly recognised and rectified, it is possible to say that, during the pilot study, the wing hq was observed to display a significant degree of a personal mastery discipline. managing mental models mental models are the entrenched assumptions or "deeply internal images of how the world works, images that limit us to familiar ways of thinking and acting" (senge 1992:174). mental models can be simple generalisations like 'people are basically dishonest' or quite complex theories about how a business process works. these models, or assumptions, dictate and often constrain actions and interactions. to overcome this potential limitation, it is necessary to manage these models; to confront, test, reflect on, and improve internal assumptions of how the world works. weintraub (1995) believes that changing mental models can be the most difficult kind of learning but mental models will change, as people come to understand and experience success with the behaviour that arises from new models and as this behaviour become more habitual and tacit. because of the tacit nature of mental models and the limited duration of the pilot study, it was not possible for the research team to make definitive observations about individual or organisational mental models, or how they may have been managed or changed over time. this will be overcome with the use of supplementary methodologies in future research. however, the research team observed some tenets of the deeply entrenched organisational culture at srg, and some of the ways in which this culture was sustained and perpetuated. it seems likely that an organisation that invests so much in nurturing its culture, must, by definition, be managing mental models. this culture is discussed in the next section. building a shared vision a shared vision is a vision that most people in an organisation are truly committed to because it reflects their own personal vision and it creates a sense of commonality and shared purpose. however, a shared vision is about more than purpose or objectives. a shared vision creates a common identity, purpose, understanding, image, and a set of governing values for an organisation (senge 1992). building a shared vision can be seen as an ongoing process, actively practised by organisational leaders; incorporating the changing personal visions emanating from all levels of the organisation. governing values are an important part of this shared vision as they form the basis of the culture of an organisation: "a strong, consistent culture grown from shared values provides all of the control necessary for directed learning" (hoffman & withers 1995:474). the key to an organisation's cultural strength is consistency, and the key to consistency is having everyone sharing in the understanding, sustenance and responsibility for that culture. any military organisation is steeped in a deep, historical tradition that forms the basis of its culture. this is certainly the case of the raaf personnel at srg. furthermore, in an organisation like the wing hq the sense of 83 ajis vol. 7, no 2 may 2000 shared purpose and objective is very much heightened, as the ultimate price can be paid for divergence from that vision. the safety of the aircrew flying strike missions is a paramount and pervasive imperative and every person in the organisation seems to understand they have a role to play in ensuring that safety. this shared responsibility also leads inevitably to an environment necessarily built on trust. this was also observable at the early stages of the military exercise, where 95wing brought together many different aircraft and aircrew that had not flown together before. the trust within the members of the amberley-based wing hq had to be extended to the other pilots and navigators involved in the missions and this acceptance seemed to be negotiated and nurtured through the familiarity of the mission briefings, and informal discussions around the operations room table. during the first week of the military exercise, when a united states air force fa-18 pilot (on a different, unrelated military exercise) crashed in the exercise target zone, the 95wing missions for that day had to be speedily re-planned. this need to divert from the normal, planning timeline resulted in an uneasiness among mission commanders and aircrew that, one can speculate, could only have been exacerbated by this tragic reminder of the possible consequences of error. the governing values of trust, the minimisation of fear, and forgiveness, are all factors that foster innovation and provide a fertile environment for learning; freeing people to speak honestly, share information and offer suggestions without fear of ridicule or retribution (handy 1995, hoffman &withers 1995). "for a learning organisation to take root and grow, it must stop holding people accountable for mistake-free performance, and begin holding them accountable for learning from their mistakes" (hoffman & withers 1995:470). the learning organisation approach accepts not only the making of mistakes, but also encourages the sharing of them for the benefit of the whole organisation. this focus on lessons learned, rather then mistakes made, is very much part of the culture of srg where protocols for meetings and briefings are applied specifically to avoid confrontation and fault-finding. this was also the overriding theme at the military exercise. in darwin, the executive staff was constantly emphasising that the exercise was an opportunity for learning. this was re-enforced at the mission debriefings where any 'mistakes' or 'misjudgments' made during missions were discussed openly as a problem to be discussed and solved, and no attempt was made to assign blame. the predominant atmosphere in these debriefings was inquiring and supportive and, therefore, conducive to learning. this emphasis on the learning outcomes of the military exercise was also evident in the post-exercise debriefing given to the research team on their last visit to srg. the executive officer informed us he was planning to burn a cd-rom of the atos developed, the tools used and the lessons learned at the 1998 exercise, to add to a database of previous military exercises. according to hoffman and withers (1995), every lasting culture is involved in the ongoing creation and evolution of cultural signals, or rituals, which are a means of group identification, communication, education and reassurance. they allow group members to feel comfortable within their culture and contribute to interpersonal understanding and inter-generational continuity. the research team saw many instances of the creation and nurturing of these cultural signals and rituals at srg and at the military exercise. this included the usual raaf rituals and also symbols specific to srg: treasured drawings on staff room walls; special caps issued for military exercises, past and present; and the 'war' stories that emerged from previous military exercises. all of these symbols add to the strong group identification making srg more a cohesive community, than a mere organisation. clearly, the wing hq and srg have been very successful in nurturing and sustaining a shared vision. team learning sharing lessons learned from mistakes requires not only trust, but also a strong sense of teamwork, and a belief that individuals benefit when their colleagues continue to learn and develop (hoffman & withers 1995). certainly, empowered, self-directed teams are considered a potent force in organisations today; acknowledged as an effective way of conducting business and advancing learning (dilworth 1995). the srg personnel operate as a team on a number of levels: from the wing level; through the squadron level; down to the vital inter-dependence of the pilot and navigator at the cockpit level. beyond this, each member of these teams has a role as a team member in other teams, through secondary duties and formations like 95wing, thereby inculcating the skills and practices of team learning more broadly. furthermore, cross-peer learning is practised extensively at srg, both informally through mutual engagement and more formally through peer review and exchange programs with other national airforces. according to dilworth, cross-peer learning is an inherent consequence of effective team learning and "cross-peer tutoring becomes a natural occurrence in a learning organisation "(dilworth 1995:251) the discipline of team learning, senge contends, also involves mastering the skills of dialogue and discussion. dialogue is grounded in reflection and inquiry, and involves the "free and creative exploration of complex and subtle issues"; while, in discussion, "different views are presented and defended and there is a search for the best view to support decisions that must be made" (senge 1992:237). dialogue, then, involves the suspending of team assumptions (building on the individual's ability to confront their own mental models); and being able to see each other as colleagues and equals. although the research team observed instances of both dialogue and discussion during briefings and debriefings, there was not enough opportunity to observe different team interactions in enough detail to conclude that these techniques are so pervasive and entrenched as to be called a discipline within the wing hq being observed. however, it is possible to say that, during the pilot study, aspects of dialogue and discussion 84 ajis vol. 7, no 2 may 2000 were observed, as were the briefing protocols, courteous communication skills, strong relationships, shared vision and strong cultural identity that would facilitate the skills of effective team dialogue and discussion. systems thinking systems thinking is often seen as an analytical tool for dealing with complexity, but systems thinking is also a powerful means for generating shared meaning and insight. senge sees system thinking as the fifth discipline: "the conceptual cornerstone that underlies all of the five learning disciplines ... the cornerstone of how learning organisations think about their world" (senge 1992:69). systems thinking is both the incentive and the means for integrating all the disciplines of organisational learning: "all are concerned with a shift of mind from seeing parts to seeing wholes, from seeing people as helpless reactors to seeing them as active participants in shaping their reality, from reacting to the present to creating the future" (senge 1992:69). systems thinking involves seeing interrelationships, and balancing and leveraging processes through managing systemic delays and feedback, rather than by focussing on linear cause and effect. the research team observed, and heard about, many minor instance of systems thinking at srg, in terms of personnel seeing the whole, rather than the parts, in problem solving and conducting missions. this is a discipline that seems to be particularly re-enforced by the culture of the organisation a nurturing culture that values common identification and a shared vision that is very much about creating the future. since the research team was not able to observe the second week of the military exercise, they were keen to find out how the disconnection between the information-providing area and the operations room had been resolved. consequently, this became one of the topics raised during interviews at the last field trip to srg. presumably, the linear approach to solving the problem would have been merely to instruct the officer-in-charge of the non-performing area to fix the problem. in fact, the research team was interested to hear that the executive staff of 95wing had taken a systems approach to solving the problem. the xo stated that, clearly, insufficient training and instruction had been given to the non-performing information-providing area in preparation for the exercise. consequently, in the second week of the exercise, the staff from the area were given clear instructions about how, when and who would up-date the two sets of information required. furthermore, a lot of effort was put into involving senior staff from the non-performing area more directly with the mission commander groups and in ensuring these officers reported back to the officer-incharge of this area, who was himself more actively aligned with the ato development. organisational learning and military organisations while there are occasional references to military organisations, most of the theories on organisational learning use business based organisations as examples or case studies. military organisations differ from business based organisations in a number of ways that may affect their ability to be learning organisations. some of these differences are: members of military organisations generally choose to become members rather than happen to become members; new members of the organisation are generally expected to make a long-term commitment to the organisation; members of military organisations take on a full-time commitment and a way of life, not just a job; members of military organisation are generally extensively trained by the organisations; military organisations have a historical tradition and culture that pre-dates all current members of the organisation; members of a military organisation are expected to conform in ways that other organisations do not (uniforms, discipline, hours of work);military organisations are hierarchical and have a tradition of autocratic leadership styles; the goals and long term directions of military organisations are usually dictated by government ministers, and not market forces or the personal vision of military personnel. on face value, several of these differences may appear to work against the disciplines of a learning organisation, as espoused by some scholars. in particular, the notion that: "the barriers generated by hierarchy, and the autocratic leadership styles, create an atmosphere of distrust, fear, blocked communications, fragmentation of work effort, and stultification of organisational learning (dilworth 1995:245). however, the observations made by the research team contradict this view. while srg is certainly hierarchical, like any military organisation, and some leadership styles may seem somewhat autocratic, the over-riding perception is one of concerned, committed and strong leadership. this form of leadership seems to engender organisational learning, rather than stultify it. furthermore, the majority of differences listed above appear to positively facilitate the personal mastery, shared vision, team learning and systems thinking disciplines that are seen to be so important in enabling organisational learning. in fact, it may be true that the nature of military organisations, particularly at the tactical level, provides an environment much more conducive to organisational learning than the environment of most business organisations. certainly, it is the opinion of the research team involved in the pilot study, that srg and the wing headquarters studied are practising, to some degree, most of the five disciplines of a learning organisation. conclusion the esla pilot study had two primary objectives: to see if it was feasible to observe, understand and document social learning processes, particularly in command and control situations; and to trial the use of ethnographic 85 ajis vol. 7, no 2 may 2000 techniques for this purpose. the pilot study clearly showed that it is feasible to research social learning processes in command and control situations, and a number of social learning mechanisms, contexts, and influences were identified. furthermore, the pilot study also validated the use of ethnographic techniques for this purpose. the numerous theories underpinning this research led to the identification of social learning constructs at various levels of abstraction, resulting in a set of complex, synergistic findings. some of the processes and strategies that engender organisational learning in the pilot study setting were found to include: peer review and cross-peer learning; building a common identity; the judicious use of protocols at meetings and briefings; the encouragement of dialogue based on inquiry and reflection; and effective formal information flows. similarly, some of the disciplines, or contexts, that were found to facilitate and nurture organisational learning were found to be based on: a strong common cultural identity with a shared purpose, objective and vision; the encouragement of individual expertise and mastery; the willingness to change the assumptions upon which the organisation is built; a clear understanding of the inter-relationships and interdependencies of the systems within which individuals in the organisation operate; an organisational focus on sharing lessons learned, rather than mistakes-made and faultfinding; and an emphasis on systems-thinking approaches to problem solving. furthermore, the findings verify that organisational learning will flourish in an environment of trust, forgiveness and sharing. the pilot study has been seen to be successful from the srg perspective, dsto's perspective, and the research team's perspective. the study validated the utility of ethnography and the feasibility of observing, understanding and documenting social learning processes. as a result the research will continue in a new setting later this year. while it is too early to make conclusions about the final shape of a social learning architecture, some of the likely constructs are already becoming visible. further research will add more understanding of appropriate architectures for systems that will support and facilitate social learning. concurrently, it is envisaged that the esla task will lead to an enhanced understanding of the social structures that most successfully support people working, learning, and collaborating in command and control in the adf. acknowledgements the author wishes to acknowledge the vital contribution made to this work by dr john o'neill, the originator and first task manager of this research study, and by dr jennie clothier whose vision made it possible. thanks are also due to team members katerina agostino, ronnie gori, paul prekop and malathi carthigaser, and to all the raaf personnel who participated in the study. copyright commonwealth of australia (c) 1999. leoni warne, for, and on behalf of, the commonwealth of australia assigns a non-exclusive licence to use this document for personal use and in courses of instruction provided that the article is used in full and this copyright statement is reproduced. references agostino, k. gori, r. and warne, l. (1999) the enterprise social learning architectures task: context, and conclusion from the pilot study, (dsto-rr-0160), canberra: dsto, dept. of defence. bolk, h., van elswijk, p., melis, t., and van praag, a.m. (1996) "only people learn: organisations can adapt", computers in industry, vol 33, 2, pp 209-216. courtney, j.f, croasdell, d.t, and paradice, d.b. (1998) "inquiring organizations" http://www.cba.uh.edu/~parks/fis/inqorg.htm (also published in australian journal of information systems, vol 6,1.) dil worth, r. (1995) 'the dna of the learning organization" in learning organisations: developing cultures for tomorrow's workplace, portland, oregan: productivity. garvin, d.a. (1993) "building a learning organisation", harvard business review, july-august 1993, pp 78-91. handy, c. (1995) "managing the dream", in learning organisations: developing cultures for tomorrow's workplace, portland, oregan: productivity. harvey, l. and myers, m.d. (1995) "scholarship and practice: the contribution of ethnographic research methods to bridging the gap", information technology & people, vol 8, 3, pp 13-27. hoffmann, f., and withers, b. (1995) "shared values: nutrients for learning" in learning organisations: developing cultures for tomorrow's workplace, portland, oregan: productivity. lave, j. and wenger, e. (1991) situated learning: legitimate peripheral participation, cambridge: cambridge university press. miles, m.b., and huberman, a.m. (1994) qualitative data analysis: an expanded sourcebook, 2nd ed., london: sage. myers, m.d. (1999) "qualitative research in information systems", isworld net, http://www.auckland.ac.nz/msis/isworld/index.html 86 ajis vol. 7, no 2 may 2000 o'neill, j. (1998) social learning: preliminary findings of a pilot study (dsto-cr-0098), canberra: dsto, department of defence. senge, p.m. (1992) the fifth discipline: the art and practice of the learning organization, sydney: random house. warne, l. (2000) "a socio-technical approach to social learning analysis in the australian defence force", in the new sociotech: graffiti on the long wall, e. coakes, r. lloyd-jones and d. willis (editors), london: springer-verlag for the british computer society. weintraub, r. (1995) 'transforming mental models through formal and informal learning; a guide for workplace educators" in learning organisations: developing cultures for tomorrow's workplace, portland, oregan: productivity. wenger, e. (1998) communities of practice: learning, meaning and identity, cambridge: cambridge university press. 87 ajis vol4 no. 1 a content specification for business process models akhilesh bajaj & sudha ram dept. of m.i.s., university of arizona address: email: abajaj@misvms.bpa.arizona.edu mailing address: akhilesh bajaj dept. of m.i.s. karl eller graduate school of management, mclelland hall university of arizona, tucson, az 85721, u.s.a. phone: (520) 621 2748 fax: (520) 621 2433 abstract business process modeling is an essential prerequisite to business process reengineering (bpr), and workflow management (wfm). process models have been traditionally used to model software processes, and many business process models are adaptations of these process models. using these process models to represent business processes results in two problems. first, since these process models usually represent different perspectives of systems (or in this case, businesses) the user needs to integrate multiple existing models to completely represent the business processes. this reduces the ease of use, and leads to a lower acceptance by users. second, business processes contain concepts not found in software processes (e.g., physical objects, roles, etc.). traditional process models cannot represent these new concepts, and hence traditional process models model business processes inadequately. these two problems can be easily solved if a comprehensive business process model exists, that models all perspectives of a business process, and that allows representation of these new concepts. as a first step towards this goal, we propose a content specification that would need to be satisfied by such a business process model. the primary contribution of this work is a comprehensive content specification for a business process model that will solve the two problems listed above. this content specification also serves as a framework to analyze process models in detail, and to compare them based on their content (i.e., what concepts they model) and the degree to which they model each aspect of a business process (i.e., how much of a business process they model). introduction process models have traditionally been used to model software processes [curtis (1992)]. recently, attention has turned to modeling and reengineering business processes, especially in the emerging area of workflow management [attie (1993), georgakopoulos (1994), georgakopoulos (1995), joosten (1994), joosten (1995)]. most work in this area has entailed extending existing process models, which were used earlier primarily for software processes. while a large number of process models (over a hundred) exist [olle (1986)], adapting any of these to model business processes poses two problems. the first is that different models model different facets of a system [curtis (1992)]. thus, [booch (1994)] advocate the use of class diagrams, object diagrams, state transition diagrams, physical diagrams to model different aspects of a system. [barker, (1992)] also advocate different models (dependency charts, functional hierarchy charts, data flow diagrams (dfds), etc.) for modeling different aspects of a system. [martin (1987)] recommend decomposition diagrams, dependency diagrams, entity-relationship (e-r) diagrams, state transition diagrams, and dfds to model different aspects of business processes. while many of these models sufficed for software processes, none of these models are considered sufficient by themselves to model a business process comprehensively. many commercial tools based on one or more of these models exist for modeling business processes ([georgakopoulos (1995)] provide an extensive listing of these). however, many of these tools either also use different models to model different aspects of business processes (e.g., [booch (1994)]), or alternately, arbitrarily extend these models in their software, without any attempt at formality. many of these models were formulated initially in the late 1970's and early 1980's, and researchers have recently tried to add formal extensions to these models [kung (1991), tao (1991), ward (1986)]. however, once again, none of these extensions result in a model that comprehensively captures all aspects of a business process. one of the requirements of a model is that it should be easy to use [curtis (1992), kramer (1991)]. the fact that a model can only model some perspectives of a business process leads to a situation where many (often complicated) instances of different models have to be integrated to form a complete view of business processes. this has resulted in low ease of use for users, and low acceptance of models by industry. the second problem is that business processes require some new concepts that were not needed for modeling software processes. e.g., physical objects, humans, location, time, etc. need to be modeled in business processes [georgakopoulos (1995), joosten (1995)]. in order to solve these 2 problems, a single business process model is needed, that can comprehensively model all aspects of a business process. in this work we take a first step towards this goal, by proposing a contents specification for business process models. each dimension of this framework represents a need that should be fulfilled, in order to model a business process. this specification makes two contributions. first, it serves as a 22 ajis vol 4 no. 1 framework to analyze and compare existing models, based on their content and which perspectives of a business process they model. second, it is a useful guideline for developing a comprehensive business process model that will solve the two problems mentioned above. the rest of this paper is organized as follows. in the next section, we examine two taxonomies that have been used to study existing process models. we also examine some requirements for process models that have been suggested earlier. after that, we specify the content specification for a comprehensive business process model. we then demonstrate the use of this specification as a framework for detailed analysis and comparison of two existing process models. finally, we discuss how this content specification can be used for further research. existing frameworks and requirements for process models existing frameworks to study process modeling many taxonomies have been proposed to classify existing process models [15-19 in tolvanen (1994)]. we consider two recently proposed frameworks here. the first classifies models based on the size of the system they try to model, as well as the contents and functionality of the model. the second is based purely on the content of the models. [tolvanen (1994)] propose a two-dimensional framework for analyzing business process models. the first dimension is divided along the size of the information system (is) being modeled, and has 5 values: methods for modeling: specific internal iss, internally integrated iss, organizational interface iss, interorganizational relationships and iss for business networks (many organizations). the second dimension is based on what information the model captures. it consists of four sub-dimensions, each of which represents one functionality dimension. the sub-dimensions are: scope of analysis: what aspects of the real-world are modeled. e.g., dfds have processes and dataflows. system structuring: how these elements are put into an orderly whole. e.g., the principles of decomposition that govern a model. heuristics for candidate designs: the rationale behind generating designs automatically. e.g., a shortest path algorithm to generate the shortest path in a graph. standards of performance evaluation: how to assess and select the candidate designs or solutions. [falkenberg (1991)] propose viewing each model as containing objects, with a set of intuitive axioms that govern how the designer can use these objects, as well as a set of rules, derivable from the axioms and the rules of the language used to create the model. thus, dfds have processes and dataflows as objects, with a set of axioms. e.g., an axiom is: each process must have at least one dataflow connected to it. if the dfd model is specified using predicate logic, then rules can always be derived using the rules of predicate logic, and the axioms of the model. in a sense, this taxonomy corresponds to the "scope of analysis" and "system structuring" sub-dimensions proposed by [tolvanen (1994)]. these two frameworks reflect the fact that most taxonomies of process models are based on: the content and functionality of the model, the aspects of a process that are captured, and on the size and complexity of the system that the model can represent. however, the dimensions used in most existing taxonomies are broad, and do not allow an extensive analysis of the models. as part of this work, we propose a detailed content specification for business process models. the specification represents different aspects of a business process that must be modeled in order to represent the process. any process model will meet these requirements to varying degrees. this specification improves on existing frameworks for analyzing process models because it allows a much more detailed analysis of process models, based on their content and what aspect of a business process are modeled. it also serves as a good design prescription when developing a business process model. previously identified requirements for process models several requirements for process models have been identified in past research. [opdahl (1994)] describe the concepts of transportation (through space), transformation (of state) and storage (transportation through time) as basic aspects of processes. in addition, they distinguish between simple input/output relationships between processes, and the actual flow of data and matter between processes. [rubin (1992)] discuss the importance of time ordering in dynamic modeling. object states, events, and the sequence in which these events occur are all stated to be important. sequencing is classified as concurrent, repetitive, selective or optional. they construct the lifecycle of objects as a series of states, with events causing changes to these states. they also specify other desirable attributes of a process model: it should be analyzable for syntactic correctness, consistency, completeness. in addition, it should support more advanced analyses of concepts such as reachability, deadlocks, race conditions and behavioral ambiguities. 23 ajis vol 4 no. 1 they describe four commonly represented perspectives that a process model should represent: functional (what activities are being performed, what is being produced, what is being consumed, etc.), behavioral (when activities are performed, as well as how (feedback loops, iteration, entry criteria, exit criteria, etc.)), organizational (where and by whom in the organization are activities performed, where artifacts are stored, and the physical communication mechanisms by which they are performed), and finally, informational (which informational entities are produced, manipulated or consumed by a process). [curtis (1992)] describe process modeling to include the modeling of phenomena that are enacted by humans as well. they enumerate five basic uses of process models: to facilitate human understanding, to automate process descriptions, to set a standard for actual process execution in the organization, to provide a framework for analyzing processes, and to automate actual processes. other information that people need from process models is "what is going to be done, who is going to do it, when and where will it be done, how and why will it be done, and who is dependent on its being done." [curtis (1992)] [kramer (1991)] list adequacy, readability and ease of use, hierarchical decomposability and amenability to formal analysis and reasoning as typical requirements for process models. in the workflow literature, [georgakopoulos (1995)] define a workflow as a collection of tasks organized to accomplish some business process. they state that workflow models must describe processes, subprocesses, dependencies between these, and required roles that can fulfill these tasks. they also list capturing process objectives (such as customer satisfaction) in the workflow model. [attie (1993)] specify certain dependencies that can exist between various tasks, and propose a finite automata model to enforce these. some of the dependencies they list are: existence dependencies (if task a occurs, then b must also occur), conditional existence dependency (if a and b occur, then c must occur), temporal dependencies (if a and b both occur, then a must precede b), etc. [joosten (1994)] describe how triggers can be a part of a workflow model. in addition, there has been extensive work on modeling dependencies between workflow transactions operating on a database (e.g., [georgakopoulos (1994)]). we do not review this literature here, since it is specific to transaction processing, and hence at a lower level than the process models we are trying to classify here. the requirements listed above justify the dimensions in our specification, which is described next. a content specification for business process models we divide our specification into broad dimensions such as system, state, space, time, etc. these dimensions are chosen based on previous requirements (described in the previous section). within each dimension are listed the actual sub-dimensions pertinent to that dimension. system • the model must define what is an entity in the system, and how it is recognized. e.g., an entity could be defined as a logical object that is different from other objects. • the model must be able to clearly distinguish between entities in the system, and those outside the system. e.g., dfds use external entities to model entities outside the system. • entities must be able to take on one of many different roles. the set of possible roles must be predefined in the system. examples of roles are: agent, resource, responsible agent, customer, performer, etc. • entities must be distinguishable into artifacts (non-human physical objects), information entities (non-human, non-physical objects), or humans. state • the model must define what it means by statejype. e.g., the "attributes" of a set of entities may make up a state_type, and values of attributes of all these entities might constitute a state_instance. the definitions should be verifiable, and unambiguous (i.e., the analyst should be able to tell what statejype the system is in, and what state_instance it is in). space • the model should be able to capture the spatial location of entities in the system. • it should support a measure of spatial distance, and be able to model spatial constraints based on this measure. e.g., it should be able to model the fact that a milling machine can never be more than 50 meters from a lathe. 24 ajis vol 4 no. 1 time • the model should be able to capture the temporal location of statejnstances in the system. thus, a statejnstance will occur at a particular time, which will be its location on the temporal dimension. • it should support a measure of temporal distance, and be able to model temporal constraints based on this measure. e.g., state jnstance si must not lag behind statejnstance s2 by more than 6 hours. transformation / transportation / storage • the model should support the notion of three activity_types: an activity-type that transforms the value of a state_type, an activity jype that transports entities through space and an activity<_type that transports statejtypes through time (storage). this is based on the idea proposed by [opdahl (1994)], where processes, flows and stores respectively support the 3 aspects of transformation, transportation and storage. e.g., activity_type a transforms statejype s. an instance al of a may then transform instance 57 of s to instance s2 of 5. sequencing and control flow • the model should support sequencing of a set of activityjtypes. • it should support the atomic execution of a set of activitity_types. e.g., if an instance of activity_type a occurs, then an instance of activity_type b must also occur. • it should support concurrent execution of a set of activityjtypes. • it should support either / or execution between two or more sets of activity jypes, based on a predicate (similar to the "switch" statement in programming languages). • it should support while and repeat-until flow on a set of activityjypes, also based on a predicate. note that the above specifications subsume the concept of triggers (execution of an activity based on a predicate). decomposition • entities: the decomposition of entities must be supported. e.g., a document must be decomposable into its sections, which are themselves entities. the rules for this decomposition must be clearly stated. e.g., in the object-oriented (oo) paradigm, super-classes may be decomposed into sub-classes, following a strict inheritance. • states: the model must support the decomposition of statejtypes. descriptive substates: if an overall statejype is used to describe a part of the system, it should be possible to decompose this overall statejype along descriptive substate jypes. e.g., an automobile assembly line can have an overall statejype, say, "state_of_assembly_line" and this overall statejtype may be further described by the descriptive substate_types: "cars_output_rate", "defects_output_rate" and "worker_morale". thus, statejtypes are decomposed in this dimension on a purely descriptive basis. spatial substates: the model should support the decomposition of statejtypes along the space dimension. e.g., it should be able to represent the statejype of the whole office, as well as the substatejype of different cubicles spatially located in the office. statejtypes are decomposed in this dimension on a purely spatial basis. temporal substates: the model should support the decomposition of statejtypes along the time dimension. e.g., it should be able to represent the statejtype of the assembly line across a week (maybe by using an average measure for the week), and this statejtype can be decomposed into substatejtypes that represent the state of the assembly line for a day. these substate jypes can be further decomposed into substate_types that represent the state of the assembly line for an hour, etc. statejtypes are decomposed in this dimension on a purely temporal basis. note that this is different from transporting a statejype across time, as described earlier. in the latter case, no decomposition of statejtypes takes place. • activityjtypes: the model must support the decomposition of activity jypes. 25 ajis vol 4 no. 1 —decomposition of activityjypes that transform state jypes: subactivitv types acting on descriptive substate types: this would be a decomposition of activityjtypes on a descriptive dimension. e.g., an overall activity_type might transform the statejtype "state_of_assembly_line" and its descriptive subactivityjypes will transform the descriptive substate jypes "cars_output_rate", "defects_output_rate" and "worker_morale". subactivitv types acting on spatial substate types: this is a decomposition of activity jypes on a spatial dimension. thus, if an activityjype a transformed a statejype s, the spatial subactivityjypes of a would transform the spatial substatejypes of s. subactivitv types acting on temporal substate types: this is a decomposition of activity jypes on a temporal dimension. thus, if an activityjype a transformed a state jype s, its temporal subactivityjypes would transform the temporal substatejypes of s. note that the decomposition of activity jypes that transform state jypes corresponds to the decomposition of state jypes described earlier. in addition to activityjypes that transform state jypes, we have two more activity jypes, whose decomposition is described below. -decomposition of activity jypes that transport entities through space: if an activityjype a transports a set of entities {£/...£„} through space distance l, then its subactivityjypes may transport a subset of these entities through l. another type of decomposition would be subactivityjypes of a that transport the entire entity set {£/...£„} through distances l! ... ln, such that li + l2 +....+ lm = l. -decomposition of activity jypes that transport state jypes through time: if an activityjype a transports a state jype s through a time distance t, then its subactivityjypes transport s through time distances t(...tn such that t] + t2 ... + tn = t. note that a can also be decomposed into subactivityjypes that act on substatejypes of s, but this has already been described earlier. • the model should have atomic constructs that signify the end of a decomposition. e.g., primitive activityjypes might signify the end of activity jypes decomposition for a system. constraints and axioms the model should support the following constraints: • availability of resources; • real-time constraints for activityjypes and state jypes (e.g., a state jnstance can exist only for a certain time period); • spatial constraints, as described earlier; • temporal constraints, as defined earlier; and • state jype constraints. e.g., the state jype: "working_state" for a researcher can never have a "vacation" state jnstance. • the model must explicitly use axioms that make any instance of the model unambiguous, and that show if the instance of the model is correct. • the model must be specified in a language with well defined rules, that allow the derivation of new rules from axioms. e.g., the model may be specified using a language based on predicate logic. advanced analyses the model should support analyses that show: • if a state jype is reachable; • whether deadlock is possible; • whether or not a particular model specification is optimal, based on pre-defined criteria (e.g., does the model specification consume the minimum resources to achieve the final state?); • whether a process will ever go into an infinite loop; and • whether a certain sequence of activities will ever cause a race condition. the dimensions and sub-dimensions for the content specification are summarized in figure 1. 26 ajis vol 4 no. 1 transportation/ transformation/ sequencing/ decompconstra nts/ syst 1 1 entiti" rol :m state sp ice time i i i i state.type 1 dtftin 1 spat™1 guisha'ble statejnstlocation ance itjmpora location 1 cin.tr'"duainu spattal storage ttansp. ortationnporal tance trans . i °temper 1 constraints mat control flow osition axioms on stor i aton or * °~ ition concurrent execution either/or er title: stacs resources real-time spatial temporal state activities while repeat advanced analyses reachability deadlock optimality infinite loop race condition anguage axioms fig. 1. dimensions and subdimensions of the content specification for business process models using the content specification to classify existing models we now show how our content specification can be used to analyze and compare existing process models. dfds are considered intuitively appealing and are popular [opdahl (1994)]. there are also a lot of commercial models available based on the idefo model [laamanen (1994)]. we examine both these process models. the intention is not to criticize either of these models, but to demonstrate the usage of our specification as a framework for analysis and comparison. data flow diagrams in their basic form, dfds [demarco (1978), gane (1982)] consist of activators (processes, external entities and data-stores) and dataflows. a short, and by no means complete list of axioms that are used to ensure consistency in the dfd model are shown in table 1. 27 ajis vol 4 no. 1 e a c h d a t a f l o w m u s t h a v e one source and one d e s t i n a t i o n . dfds may h a v e cycles bu t no po in t cycles. d f d s c a n n o t h a v e isola ted ac t iva to rs , w h i c h h a v e no f l o w s connec t ed . da t a stores and ex te rna l en t i t i e s have to be m u t u a l l y d i s t i n c t (as do processes). each d a t a f l o w m u s t h a v e a process at at least one end of i t any a c t i v a t o r connected w i t h o n l y one f low has to be an e x t e r n a l e n t i t y processes can be h ie ra rch ica l ly decomposed , but it has to be a s t r ic t tree h i e r a r c h y (i.e. par t ia l order + o n l y one pa ren t ) if a d a t a f l o w exists at d i f f e r e n t l eve l s , it m u s t fo l low the h i e r a r chy , i.e. the d a t a f l o w at a lower leve l can occur be tween two ac t iva to r s i ff i t occurs at a h i g h e r level be tween ances tors of the ac t iva to rs . table 1. some axioms in the dfd process model let us see how our framework can be applied to this model: system: we see that dfds support the notion of entities outside the system (external entities) and inside the system (dataflows). there is no concept of roles. they do not distinguish between entities (all internal entities are considered to be information entities). state: the state is represented by dataflows. thus, a dataflow plays the part of both, an internal entity and the statejtype of the entity. e.g., a dataflow called "unsigned_purchase_order" may go to a process called "purchaser", who may sign it and a dataflow called "signed_purchase_order" (the same entity in a new state) will emanate from the "purchaser". space and time: none of the sub-dimensions of the space or time dimensions are explicitly supported by dfds. however, they could be informally mentioned in the descriptions of the dataflows. e.g., "unsigned_order_in_purchase_dept." captures the spatial location of the purchase order. transformation/transportation/storage: in dfds, processes transform flows. they could also potentially be used to transport flows through space. e.g., a process called "move_information" may transport a flow called "customer_data" across a computer network. data-stores are used to transport flows through time. note that it is not possible for a flow to be simultaneously transported through time and be transformed or transported through space (since data-stores are not processes). it is, however, possible for a flow to be transported through space, and transformed at the same time (since both these functionalities are served by processes). sequencing / control flow: dfds only support the notion of sequencing. the sub-dimensions of atomic execution, repetition, either/or, while/repeat and concurrent execution are not supported. decomposition: traditionally dfds allow only the decomposition of processes. since dataflows model entities and states, the decomposition of these is not explicitly supported. however, the decomposition of activity jypes (processes) is very well supported. the only activity_type whose decomposition is not well supported is one that transports state_types through time (since these are modeled using data-stores). thus, processes can be decomposed in ways that support all the decomposition requirements of activities described earlier, except for the decomposition of activity_types that transport state_types through time. e.g., an overall process may transform a dataflow "insurance claim". its decompositions may transform the dataflows: "accident_description" and "repair_cost_estimate". thus, in a sense, the fact that the decomposition of most activity jtypes is supported and strictly hierarchical, implies that the decomposition of state_types is implicitly supported and is also strictly hierarchical. finally, dfds do support the notion of primitive processes, that signify the end of a decomposition. constraints and axioms: dfds do not explicitly support constraints modeling resource scarcity, real-time constraints, spatial constraints, temporal constraints or state_type constraints. the axioms in dfds do make instances of dfds unambiguous. however, traditionally dfds were not formulated in a language that allowed derivation of new rules from axioms. 28 ajis vol 4 no. 1 advanced analyses: dfds do support reachability (by modeling a dfd instance as a directed graph, where every dataflow is a node, and every process an arc, it is possible to determine reachability of a dataflow). they do not support analyses for deadlock, optimality, infinite loops or race conditions. we now examine some extensions and enhancements to the dfd model, and see how they fit in our framework. [falkenberg (1991)] proposed a formal specification of dfds in a language based on predicate logic. this clearly meets the previously unfulfilled requirement that a model must be specified in a language that is formal and allows derivation of rules from axioms. [kung (1991)] proposed a formal decomposition for processes in dfds, again using predicate logic as the base language. [tao (1991)] also proposed a formal specification for dfds, based on set theory. [ward (1986)] proposed extensions to dfds that met requirements in the sequencing i control flow and the time dimensions of our framework. our examination of dfds has shown that the dataflow concept (as well as the process concept, to a lesser degree) is overloaded. [opdahl (1994)] describe extensions to dfds that reduce the overloading of the dataflow concept somewhat, in that dataflows and predicate conditions are modeled separately. the idefo model the idefo model [laamanen (1994)] consists of boxes, that represent the system's functions and arrows that represent one of: inputs, outputs, controls or mechanisms, to support the function. a short, and incomplete, list of axioms that are used to ensure consistency in the idefo model is shown in table 2. input, ou tput and control arrows can only have one direction ( input and control arrows go into the box, while output arrows come out of the box. there can be mult iple arrows of any one type for a func t i on . mechanism arrows that come into the box ident i fy the means by wh ich a f u n c t i o n performs inpu t s to outputs . outward arrows (call arrows) s ignify that the decomposit ion at the present level of abstraction is complete, but tha t fur ther details can be found in lower levels. all child diagrams must have at least 3 and at most 6 sub func t ions . boxes ( funct ions) can be decomposed, fo l lowing a strict tree hierarchy. arrows are viewed as condui ts containing other arrows. at any point , the appropriate detail can be shown, by decomposing or grouping or both. once an arrow belongs to a f u n c t i o n , it can only belong to the f u n c t i o n , a child or an ancestor. arrows can belong to mult iple funct ions (e.g. input to one func t ion can also be i npu t to another func t ion) . arrows can play multiple roles, e.g. ou tput from a func t ion can be inpu t to another f u n c t i o n . table 2. some axioms in the idefo model. let us see how our framework can be applied to this model: system: entities in idefo are modeled using arrows. they can take on one of 4 roles: inputs, outputs, controls and mechanisms. entities external to the system are not modeled. entities may be artifacts, information entities or human artifacts, as long as they play one of the 4 roles. state: the state is represented by arrows. thus, arrows represent both the entity and the state_type of the entity in idefo. e.g., "unresolved_operational_issues" may be an input to a box, and the output may be "resolved_operational_issues", which represents a new statejnstance of the implicit statejtype "operational_issues_status". space and time: the sub-dimensions in the space and time dimensions are not explicitly modeled in idefo. however, as in dfds, it is possible to capture some aspects of space and time in idefo. e.g., the input to a box may be "document_at_location_x", the box may be: "move_document_to_y", and the output may be "document at location y". 29 ajis vol 4 no. 1 transformation i transportation / storage: in idefo, boxes can be used to transform input arrows. they can also be used to transport input arrows through space (if the input arrows represent entities), or through time (if the input arrows represent states of entities). sequencing and control flow: idefo models sequencing of functions. however, atomic execution, repetition, concurrent, either / or and while/repeat execution are not supported. decomposition: in idefo, decomposition is primarily done along the box (function) dimension. this means that activity_type decomposition is the primary decomposition. in addition, decomposition and aggregation of arrows (in the form of "forking") is supported, although "it is still unclear as to the proper implementation of a fork arrow" [laamanen (1994)]. thus, the decomposition of entities and states is not well supported (since arrows represent these). we do not mention examples here, but it should be clear that, since boxes can represent all the activity_types in our framework, and since their decomposition is well supported, activity jtype decomposition is supported along all the sub-dimensions in that dimension, in our framework. also, as we saw for dfds, it is possible to model statejype decomposition implicitly, by decomposing activity_types. the idefo model does model the end of decomposition (boxes that do not have downward arrows emanating from them are not further decomposed). constraints and axioms: idefo does not explicitly support constraints modeling resource scarcity, real-time constraints, spatial constraints, temporal constraints or statejtype constraints. the axioms in idefo make an instance of idefo unambiguous. however, like dfds, idefo is not formulated in a language that allow derivation of rules from axioms. advanced analyses: idefo does support reachability (by modeling an idefo instance as a directed graph, where every input and output is a node, and every box an arc it is possible to determine reachability of an input or output). it does not support analyses for deadlock, optimality, infinite loops or race conditions. differentiating between the 2 models system: idefo supports roles and different types of entities. dfds support external entities. state: we find that dataflows in dfds and arrows in idefo are overloaded to model both entities and states. space and time: this is not formally supported in either model. transformation / transportation / storage: in dfds, processes are used to transform and transport, while datastores are used for storage. in idefo, boxes are used for all 3 aspects. sequencing / control flow: this is similar in both models. decomposition: both models support activityjtype decomposition. neither supports the decomposition of entities or states. constraints and axioms: both models have axioms that signify correctness, and unambiguity of an instance of the model. advanced analyses: both models allow analyses for reachability of state, but none of the other analyses are supported. conclusion the content specification proposed here can be used as a framework for a detailed analysis of business process models, based on the content and the aspects of a business process they model. two widely used process models: dfds and idefo were analyzed as a demonstration of the use of this framework. the content specification defined here is part of a larger research project whose overall goal is to provide support to workflow management using a database management system. we plan to use this content specification to formally define a business process model that will meet all the requirements listed here. we anticipate the new model will contribute to solving the two problems identified earlier: the problem of difficulty of use amongst end-users and the problem of capturing all the concepts required by a business process. references attie, p.c., singh, m.p. sheth, a. & rusinkiewicz, m. specifying and enforcing intertask dependencies. proceedings of the 19th vldb, dublin, ireland, pp. 134-145,1993. barker, r. & longman, c. case*method: function and process modelling. oracle corp., uk ltd. booch, g. object oriented design with applications. benjamin-cummings, 1994. curtis, w., kellner, m.i. & over, j. process modeling. communcns. of the acm. vol. 35, pp. 75-90, 1992. demarco, t. structured analysis and system specification. yourdon, inc., 1978. 30 ajis vol 4 no. 1 falkenberg, e.d., van der pols, r. & van der weide, th. p. understanding process structure diagrams. information systems, vol. 16, no. 4, pp. 417-428 (1991). gane, c. and sarson, t. structured system analysis. mcdonnell douglas, 1982. georgakopoulos, d. & hornick, m. a framework for enforceable specification of extended transaction models and transactional workflows. international journal of intelligent and cooperative information systems, vol. 3, pp. 225-253,1994. georgakopoulos, d., hornick, m. & sheth, a. an overview of workflow management: from process modeling to workflow automation infrastructure. distributed and parallel databases, vol. 3, pp. 119-153,1995. joosten, s. trigger modeling for workflow analysis. proc. con 1994: workflow management, pp. 236247,1994. joosten, s. conceptual theory for workflow managament support systems. tech. report. center for telematics and information technology, univ. of twente, p.o. box 217, 7500 ae enschede, the netherlands. kramer, b. and luqi. towards formal models of software engineering processes. j. systems and software, vol. 15, pp. 63-74, 1991. kung, c. process interface modeling and consistency checking. j. systems and software, vol. 15, pp. 185 191,1991. laamanen, m.t. the idef standards. methods and associated tools for the information systems life cycle ifip (a-55) a.a. verrjin-stuart and t.w. olle (ed.). elsevier science b.v. (north-holland), 1994. martin, j. recommended diagramming standards for analysts and programmers: a basis for automation. prentice-hall, inc. (1985). olle, t.w. et al. (eds.) proceeding of the ifip wg 8.1 working conference on comparative review of isd methodologies: improving the practice, borth-holland, the netherlands, 1986. opdahl, a.l. & sindre, g. a taxonomy for real-world modelling concepts. information systems, vol. 19, no. 3, pp. 229-241, 1994. rubin, k.s. & goldberg, a. object behavior analysis. commncns. of the acm. vol. 35, pp. 48-62, 1992. tao, y and kung, c. formal definition and verification of data flow diagrams. j. systems and software, vol. 16, pp. 29-36,1991. tolvanen, j. & lyytinen, k. modeling information systems in business development: alternative perspectives on business process re-engineering. ifip, (a-54) b.c. glasson, et al. (ed.). elsevier science b.v. (north-holland), 1994. ward, p.t. the transformation schema: an extension of the data flow diagram to represent control and tuning. ieee trans. software engg. vol. se-12, pp. 198-210, feb. 1986. 31 the australasian journal of information systems (ajis) began in 1993 and was based on the simple realisation that there was a need for an academic ‘outlet’ for is research in australia australasian journal of information systems volume 18 number1 2013 5 data, information, knowledge, wisdom (dikw): a semiotic theoretical and empirical exploration of the hierarchy and its quality dimension saša baškarada university of south australia baskarada@gmail.com andy koronios university of south australia andy.koronios@unisa.edu.au abstract what exactly is the difference between data and information? what is the difference between data quality and information quality; is there any difference between the two? and, what are knowledge and wisdom? are there such things as knowledge quality and wisdom quality? as these primitives are the most basic axioms of information systems research, it is somewhat surprising that consensus on exact definitions seems to be lacking. this paper presents a theoretical and empirical exploration of the sometimes directly quoted, and often implied data, information, knowledge, wisdom (dikw) hierarchy and its quality dimension. we first review relevant literature from a range of perspectives and develop and contextualise a theoretical dikw framework through semiotics. the literature review identifies definitional commonalities and divergences from a scholarly perspective; the theoretical discussion contextualises the terms and their relationships within a semiotic framework and proposes relevant definitions grounded in that framework. next, rooted in wittgenstein’s ordinary language philosophy, we analyse 20 online news articles for their uses of the terms and present the results of an online focus group discussion comprising 16 information systems experts. the empirical exploration identifies a range of definitional ambiguities from a practical perspective. keywords: data, information, knowledge, wisdom, dikw hierarchy, quality, semiotics, wittgenstein, ordinary language philosophy introduction there is a growing concern … about the present situation, where too many fuzzy or ill-defined concepts are used in the information system area. scientific as well as practice-related communication is severely distorted and hampered, due to this fuzziness and due to the frequent situation that different communication partners associate different meanings with one and the same term. there is no commonly accepted conceptual reference and terminology, to be applied for defining or explaining existing or new concepts for information systems (falkenberg et al., 1998, p. 2). whilst the above quotation dates back to the previous century, its message is still as relevant as ever (weinberger, 2010). the highly-abstract language currently used to describe some of the key information systems constructs can be difficult to understand and is potentially at risk of becoming meaningless (shields, 2001). accordingly, it has been argued that the information systems mailto:baskarada@gmail.com mailto:andy.koronios@unisa.edu.au australasian journal of information systems volume 18 number1 2013 6 community “has to uncover, through hermeneutic interpretation and analysis, the fundamental nature of its discipline” (o'donovan and roode, 2002, p. 34). information systems are socio-technical communication systems that use artefacts to represent and transmit data (beynon-davies, 2009a, allen, 2000, claver et al., 2001). they have existed for thousands of years, long before the advent of modern information and communications technologies. for instance, the inca civilisation operated an effective information system without utilising a written language; they recorded complex messages on coloured, knotted, cotton cords (beynon-davies, 2007). the data, information, knowledge, wisdom (dikw) hierarchy (see figure 1), also referred to as the knowledge hierarchy, the information hierarchy, and the knowledge pyramid (rowley, 2007, p. 163) forms the foundation of information systems research. while the hierarchy itself is often quoted, and explicitly or implicitly applied in the information systems literature (information is usually defined in terms of data, knowledge in terms of information, and wisdom in terms of knowledge) there has been limited direct discussion of the content and the structure of the hierarchy (zins, 2011). the concept of wisdom in particular has received very limited discussion in the literature (rowley, 2007). since only limited consensus regarding definitions has been reached, the terms as well as their relationships remain ambiguous (chen et al., 2009, aamodt and nygård, 1995, stenmark, 2002, shields, 2001, hasan, 2002). for instance, the terms data and information are often used synonymously; e.g. data management and information management (chen et al., 2009). similarly, in the context of data mining, knowledge has been equated with useful information, and described as pattern that exceeds some threshold determined by the user (fayyad et al., 1996). on the other hand, it has also been argued that, data is not information and that information is not knowledge (chen et al., 2009, kallinikos, 2006, kamhawi, 2010). given the definitional challenges involved, it has even been suggested that it is “beyond the scope of computer science as a discipline to provide a general definition of these terms” (aamodt and nygård, 1995, p. 193). some have gone even further, arguing that the hierarchy is based on flawed assumptions and that it should be abandoned altogether (frické, 2009). however, given that most information systems literature relies on these concepts, abandoning the hierarchy does not seem feasible. as a result, developing clear, consistent, and unambiguous definitions of the terms, their relationships, and their quality dimensions is imperative. figure 1: the dikw hierarchy developed from (rowley, 2007, p. 163) theoretical exploration this section presents a theoretical exploration of the dikw hierarchy and its quality dimension. relevant terms and their relationships are contextualised within a semiotic framework. the dikw hierarchy rowley (2007) identified a range of ambiguous and/or conflicting definitions, a subset of which is shown in table 1, by examining a number of popular information systems and knowledge australasian journal of information systems volume 18 number1 2013 7 management textbooks. she attributed some of the earliest definitions to ackoff (1989) and zeleny (1987), paraphrasing ackoff as follows: wisdom is the ability to increase effectiveness. intelligence is the ability to increase efficiency. knowledge is know-how, and is what makes possible the transformation of information into instructions. information provides answers to who, what, where and when questions. data are defined as symbols that represent properties of objects, events and their environment. they are the products of observation (rowley, 2007, p. 166). w is do m wisdom is accumulated knowledge, which allows you to understand how to apply concepts from one domain to new situations or problems (jessup and valacich, 2003). wisdom is the highest level of abstraction, with vision foresight and the ability to see beyond the horizon (awad and ghaziri, 2004, p. 40). wisdom is the ability to act critically or practically in any given situation. it is based on ethical judgement related to an individual’s belief system (jashapara, 2005, pp. 17-18). k no w le dg e knowledge is the combination of data and information, to which is added expert opinion, skills, and experience, to result in a valuable asset which can be used to aid decision making (chaffey and wood, 2005, p. 223). knowledge is data and/or information that have been organised and processed to convey understanding, experience, accumulated learning, and expertise as they apply to a current problem or activity (turban et al., 2005, p. 38). knowledge builds on information that is extracted from data ... while data is a property of things, knowledge is a property of people that predisposes them to act in a particular way (boddy et al., 2005, p. 9). in fo rm at io n information is data which adds value to the understanding of a subject (chaffey and wood, 2005, p. 233). information is data that have been shaped into a form that is meaningful and useful to human beings (laudon and laudon, 2006, p. 13). information is an aggregation of data that makes decision making easier (awad and ghaziri, 2004, p. 36). d at a data has no meaning or value because it is without context and interpretation (jessup and valacich, 2003, bocij et al., 2003, groff and jones, 2003). data are discrete, objective facts or observations, which are unorganised and unprocessed, and do not convey any specific meaning (awad and ghaziri, 2004, chaffey and wood, 2005, pearlson and saunders, 2004, bocij et al., 2003). data items are an elementary and recorded description of things, events, activities and transactions (laudon and laudon, 2006, turban et al., 2005, boddy et al., 2005). table 1: defining data, information, knowledge, wisdom: ambiguous and/or conflicting definitions developed from (rowley, 2007, pp. 170-174) data and information in line with information theory (shannon, 1948), data has been defined in terms of thermodynamic (physical) entropy, information in terms of shannon (symbol) entropy, and knowledge in terms of cognitive (context) entropy (boisot and canals, 2004). as such, data can be considered as patterns with no meaning, information as interpreted data (i.e. data with meaning), and knowledge as “information incorporated in an agent's reasoning resources“ (aamodt and nygård, 1995, p. 196). whilst some of the above definitions seem to imply a linear relationship from data to wisdom, it has also been argued that data emerges from information and that information emerges from knowledge australasian journal of information systems volume 18 number1 2013 8 (tuomi, 1999). in other words, data can be viewed as an external/objective (zins, 2011) medium for communication and storage that is encoded from information by a sender and interpreted into information by a receiver (kock et al., 1997). for instance, in the context of cryptography, encrypted information can be considered as data (singh, 1999). but, more broadly, any binary representations that are stored on or communicated via a digital medium are referred to as data (chen et al., 2009). on the other hand, information, which reduces uncertainty at the receiver (shannon, 1948), can be viewed as a subjective/internal phenomenon (zins, 2011). others have argued that the concept of information is difficult to define due to its multidimensional nature (beynon-davies, 2009b). knowledge from an epistemological perspective, philosophers and scientists have been struggling with the definition of knowledge for thousands of years (alavi and leidner, 2001). similarly, it has been argued that knowledge management, the term that originated in us think tanks and management schools (shields, 2001), is not clearly defined (zhao et al., 2012) or distinguishable from related research areas (gray and meister, 2003). nevertheless, literature explicates a number of dimensions, including that knowledge is a subjective/internal phenomenon (zins, 2011), is context dependent, tacit, embedded, and socially constructed (gosain, 2003, burstein and linger, 2003, østerlie et al., 2012, shields, 2001). whilst some authors discriminate between book and practical knowledge (hara and hew, 2007), others differentiate between knowledge of truths and knowledge of things (russell, 1912). according to russell, knowledge of truths (unlike knowledge of things) has an opposite (i.e. error), and all knowledge of truths has some degree of doubt (1912). plato’s original definition, “justified true belief“ (plato), while sometimes challenged (gettier, 1963, nozick, 1981, popper, 1979), is still largely accepted in the philosophical and scientific communities (chisholm, 1982). thus, it has been argued that “one can say ‘he believes it, but it isn't so’, but not ‘he knows it, but it isn't so’“ (wittgenstein et al., 1975). however, it needs to be noted that from the pragmatist perspective the adjective ‘true’ is the word applied to the beliefs which have been justified, and that justification is relative to an audience and to a range of truth candidates (rorty, 1982). on the other hand, in line with the constructivist theory of knowledge (piaget, 1977), it has also been argued that knowledge is subjective and constructed from personal interpretations (martin et al., 2009). as such, a common view is that knowledge is the product of a synthesis in the mind of the knowing person (leonardi and treem, 2012), and exists only in his or her mind (zins, 2011, galliers and newell, 2001, linger and warne, 2001). given that knowledge always involves a knowing person (mcdermott, 1999), it could be argued that “all knowledge is tacit, and what can be articulated and made tangible outside the human mind is merely information“ (stenmark, 2002, p. 928). for instance, a bookcase or a computer may contain a lot of information, but no knowledge (mcdermott, 1999). wisdom whilst the concept of wisdom has received very limited discussion in the information systems literature, it is generally defined in terms of value driven (ethical) intelligent behaviour (dampney et al., 2002, tuomi, 1999, sternberg, 2000, ackoff, 1989, williams, 2001, bierly et al., 2000, rooney et al., 2008). awareness of one’s own cognitive limitations has also been described as a key characteristic of wisdom (takahashi and overton). in addition, psychologists have argued that wisdom is a personal (and not a group/social) quality, is a uniquely human trait, is a form of advanced cognitive and emotional development, is a rare quality, is experience driven (rather than innate), can be learned, increases with age, and can be measured (jeste et al., 2010). others have disputed the claim that it increases with age (sternberg, 2005), have attempted to measure it (jason et al., 2001), and have attempted to explain it in terms of neurobiological models (meeks and jeste, 2009). australasian journal of information systems volume 18 number1 2013 9 the quality dimension literature provides several definitions of quality (hardie and walsh, 1994), including “the capacity to satisfy wants“ (edwards, 1968, p. 37), and “a subjective term for which each person or sector has its own definition“ (asq, 2002, p. 56). while shewhart (1931) and crosby (1979) argued that quality is represented by a difference between the preferred state and the actual state (thus, mainly defining quality as conformance to requirements), proposed dimensions of quality nowadays include: performance, features, reliability, conformance, durability, serviceability, aesthetics, and the like (garvin, 1987). on the other hand, feigenbaum (1986) and juran (1974) defined quality in terms of customer satisfaction, where customer expectations can be seen as the preferred state and the concrete product or service received as the actual state. in terms of the dikw hierarchy, many researchers and practitioners use the terms data quality and information quality, which have been defined as “fitness for use“ (wang and strong, 1996, p. 6), interchangeably (madnick et al., 2009). more specifically, wang and strong (1996) defined data/information quality from users’ perspective through a framework comprising four categories and 15 dimensions; these have now become generally accepted in literature (madnick et al., 2009). additionally, several other researchers have developed a range of data/information quality metrics (pipino et al., 2002, pipino et al., 2005, redman, 1996) and assessment instruments (lee et al., 2002, price et al., 2008, baškarada, 2010, alkhattabi et al., 2011), which have been used to assess different aspects of data/information quality (baškarada, 2011a). regarding knowledge quality, it has been argued that knowledge can be tested by analysing factual data and comparing the result with relevant theoretical predictions (stumpf and mcdonnell, 2003). in other words, the quality of knowledge is usually tested via the scientific method (kosso, 2011). whilst the term wisdom quality does not appear in literature, it has been argued that “crucial to achieving wisdom is the capacity for reflexivity, which is the ability to understand the epistemic and ethical basis of our practice within its discursive domain“ (rooney et al., 2008, p. 38). semiotics semiotics is a field of study that deals with the relationships between representations, intended meanings, and interpretations of signs and symbols (baškarada, 2011b). modern semiotics studies the construction of meanings with respect to communication as well as to the construction and maintenance of reality (chandler, 2007, beynon-davies, 2009b). semiotics is based on the supposition that the whole of human experience is an interpretive activity mediated and sustained by signs (deely, 1990). as such, social activities are seen as being underpinned by sign-systems (organised collections of signs); e.g. spoken and body language (beynon-davies, 2007). in the context of semiotics, a datum is a “symbol or a set of symbols used to represent something“ (beynondavies, 2009c, p. 173), and information is defined in terms of this concept of a symbol or sign (beynon-davies, 2009b). swiss linguist ferdinand de saussure (1983) and american logician and philosopher charles sanders peirce (1998) are considered as the founders of semiotics (chandler, 2007). saussure defined a sign as an object with a meaning, comprising a signifier (signifiant) and a signified (signifié). many semioticians, including eco (1979), nowadays refer to the signifier and signified as sign-vehicle and meaning, respectively. the signifier carries the meaning and refers to the form that the sign takes. the signified refers to the concept the signifier represents; a mental activity of receiving a signifier represents the actual meaning that is carried. thus, both the signifier and the signified were initially considered as psychological (abstract) concepts, which did not necessarily have to be physical. however, the signifier is nowadays commonly interpreted as the physical form of the sign (chandler, 2007). saussure also argued that signs only make sense in relation to other signs, and that there is no australasian journal of information systems volume 18 number1 2013 10 inherent relationship between the signifier and the signified (saussure, 1983). as such, a sign has no absolute value; the value only emerges in relation to other signs. peirce defined semiosis as comprising three basic elements (peirce, 1998). a sign (representamen) stands to somebody for something in some respect or capacity. an object is that referred to by the sign. the interpretant is an individual’s comprehension of, and reaction to, the sign-object association. comparable to saussure’s model, peirce’s semiosis elements do not refer to human subjects or physical objects, but to abstract entities (eco, 1979). since symbols are interpreted according to rules or habitual connections (peirce, 1998), any interpretation is based on previous interpretations and will generate new interpretations (eco, 1979). thus, it may be impossible to determine the final and absolute meaning of signs. the semiotic ladder the american philosopher charles w. morris extended peirce’s semiotics by focusing on the relationships between signs and other signs, signs and objects, and signs and interpretants (morris, 1938). as a result he proposed three complementary areas of study. syntactics (or syntax) is the study of the formal or structural relations between signs—i.e. logic and grammar of sign systems. it deals with the physical form, rather than the content/meaning of signs (beynon-davies, 2007). semantics is the study of the relations of signs to objects which they stand for. as such, semantics deals with rules that state the conditions under which signs apply to objects. pragmatics is the study of the relation of signs to interpreters—for instance, pragmatics deals with rules which state the conditions in the interpreters under which the sign vehicle is interpreted as a sign (fiordo, 1977). as such, pragmatics “links the issue of signs with that of intention“ (beynon-davies, 2007, p. 316). however, according to zemanek (1966), syntactics, semantics, and pragmatics are difficult, if not impossible, to consider in isolation from each other. three additional levels (physical, empirical, and social) were latter added by stamper (1973) to arrive at the semiotic ladder shown in table 2. the physical level utilises the methods of physics and engineering to model signals, their sources, destinations, and routes over which they are transmitted. empirics deals with coding and signal-to-error rates. and, the social level deals with values and action (stamper et al., 2000). level aspects of representation social beliefs, expectations, commitments, contracts, social laws, culture, ... pragmatic intentions, communication, conversations, negotiations, speech acts, ... semantic meanings, propositions, validity, truth, signification, denotations, ... syntactic formal structure, language, logic, data, records, deduction, software, files, ... empirical pattern, variety, noise, entropy, channel capacity, codes, efficiency,, ... physical signals, physical distinctions, hardware, speeds, economics, laws of nature,... table 2: the semiotic ladder adapted from (falkenberg et al., 1998, p. 54) semiotic dikw framework figure 2 groups the semiotic ladder levels into three layers. the forma layer deals with the physical characteristics of signs. the informa layer deals with the content or meaning of signs. and, the performa layer deals with communication within social action. data are generated in the performa layer, reside in the forma layer, and are cognitively interpreted into information in the informa layer. for instance, due to his or her education and more extensive background knowledge, a doctor looking at a patient’s blood test data should be able to infer more information compared with the patient looking at the same data. endsley provides the following example: “[a]n experienced pilot will be able to classify observed aircraft into exact models (e.g., f18c vs. f-18d). … a novice may not be able to make this level of classification and would australasian journal of information systems volume 18 number1 2013 11 consequently have less information from the same data input” (endsley, 1995, p. 42). therefore, we propose the following definitions: definition 1: data are physical signs. they have no meaning because they reside outside of a human mind. definition 2: information (or meaning) emerges through cognitive processing of data. fo rm ac tio ns in te nt io ns m ea ni ng st ru ct ur e physical semantic syntactic empirical social pragmatic fo rm a in fo rm a pe rf or m a data information knowledge wisdom figure 2: semiotic dikw framework adapted from (beynon-davies, 2011) as such, words in a sentence (in the forma layer) may constitute high quality data (i.e. correct letters, spelling, grammar, etc.), but result in poor quality information when interpreted via the informa layer (e.g. a non-native speaker may not be able to understand a sentence as well, or not at all, compared to a native speaker). if we accept that data quality depends on the forma layer, then it chiefly depends on the encoding, storage, and communication protocols and technologies. from that point of view, data quality problems are primarily caused by engineering (software and hardware) flaws as well as by relevant human factors issues. for instance, buggy software may use incorrect data types, which may result in a loss of precision, or worse—e.g. a couple of integers may inadvertently be stored as strings and, due to operator overloading, subsequent addition may lead to string concatenation and a nonsensical result/output. similarly, data may be incorrectly encoded by individuals in the performa layer—e.g. through introduction of stochastic or systematic spelling mistakes. hence, data quality should be assessed against predefined specifications for encoding, storage, and communication of data. such protocols are developed by relevant communicative and engineering communities—i.e. by information system users and developers. therefore, we propose the following definition: definition 3: data quality is the gap between physical characteristics of signs and their specifications (the smaller the gap, the higher the quality). australasian journal of information systems volume 18 number1 2013 12 given that information is cognitively generated and fundamentally relates to communication, this suggests that information quality needs to be evaluated in terms of a mapping (or a gap-analysis) between two corresponding cognitive states (i.e. sender’s and receiver’s). however, given the difficulties involved in comparing complex cognitive states, behaviouristic approaches may be preferred (skinner, 2011). such behaviouristic approaches to information quality assessment may involve evaluations of actions of the two persons (sender and receiver), which are principally reliant on the communicated information element(s). variations in the resulting behaviour may indicate information quality problems. as deming observed: “[t]he only communicable meaning of any word, specification, instruction, proclamation, or regulation is not what the writer thereof had in mind, but is instead, the result of application. how does the instruction work in practice? what happens?” (deming, 2000, p. 70). for instance, poorly communicated information describing a location for a meeting (e.g. building and room number) may result in the receiver going to a different location compared to the sender; this divergence in behaviour implies an information quality problem. on the other hand, the quality of information that never has any impact on any behaviour naturally cannot be assessed using such approaches. however, from a behaviouristic perspective, it may be argued that such information has no observable effects and is therefore strictly speaking meaningless (harré and tissaw, 2005). as information quality depends on the intended use of the information, it only makes sense when contrasting multiple information elements/sets (i.e. better than, worse than). alternatively, one would need to contrast a concrete information element/set with a hypothetical (ideal) element/set. the preceding discussion suggests that, similar to information, information quality is also subjective. as such, what is considered as high quality information by one person may be considered as low quality information by another. for instance, if someone’s purpose is to deceive an audience then non-factual (inaccurate) information may be preferred by that person and considered as high quality. therefore, we propose the following definition: definition 4: information quality is information’s fitness for purpose. adopting rorty’s (1982) argument that knowledge is relative to an audience and to a range of truth candidates, we may argue that interpreted data (i.e. information) may be validated via the performa layer into knowledge; knowledge also depends on the believing subject and social validation via the performa layer. in other words, if one does not believe some information, then it is not knowledge. similarly, if one believes some information that is not recognised as truth by a relevant audience, then it is not knowledge either. thus, knowledge only makes sense in relation to communities which have the conceptual tools to provide such validation. the difference between information and knowledge is that information does not require justification and social validation as it only relates to cognitive processing of data. in other words, information may be fictitious whereas knowledge has to be factual. as such, personal knowledge is a subset of personal information. given that knowledge is relative to an audience (and considering that there may be multiple audiences with competing and perhaps incompatible views), similar to information quality, knowledge quality also only makes sense in relation to its purpose. however, information quality has a broader scope than knowledge quality as information’s purpose could be to entertain (e.g. reading fiction or watching movies), or to influence the behaviour of others (e.g. propaganda). the question of truth has no relation to information or information quality; it is only relevant to knowledge. therefore, we propose the following definitions: definition 5: knowledge constitutes a person’s beliefs which have been socially judged to be true. definition 6: knowledge quality is knowledge’s fitness for purpose. finally, if we accept the proposition that wisdom is also relative to an audience and a range of good decision (i.e. ethical truth) candidates, then we may argue that it also depends on the performa layer for validation. as such, wisdom depends on knowledge as well as social validation via the performa australasian journal of information systems volume 18 number1 2013 13 layer. and, given that there may be multiple audiences with competing and perhaps incompatible views, similar to knowledge quality, wisdom quality also only makes sense in relation to its purpose. therefore, we propose the following definitions: definition 7: wisdom constitutes a person’s normative judgements which have been socially judged to be desirable. definition 8: wisdom quality is wisdom’s fitness for purpose. proposed definitions drawing on the preceding theoretical discussion, we summarise the proposed definitions in tables 3 and 4. term definition example data data are physical signs. they have no meaning because they reside outside of a human mind. characters in a book. bits in computer memory. street signs. information information (or meaning) emerges through cognitive processing of data. reading a book. watching a movie. knowledge knowledge constitutes a person’s beliefs which have been socially judged to be true. the sun is at the centre of our solar system. wisdom wisdom constitutes a person’s normative judgements which have been socially judged to be desirable. we should reduce carbon emissions. table 3: proposed definitions of data, information, knowledge, and wisdom term definition example data quality data quality is the gap between physical characteristics of signs and their specifications (the smaller the gap, the higher the quality). low data quality: poor handwriting, scratched cd/dvd, corrupt computer file, etc. information quality information quality is information’s fitness for purpose. if, ceteris paribus, following recipe a is more effective at producing a tasty meal than following recipe b, then, recipe a constitutes higher quality information. knowledge quality knowledge quality is knowledge’s fitness for purpose. if, ceteris paribus, cognitive behavioural therapy (cbt) is more effective at reducing phobias than psychoanalytic therapy, then, cbt constitutes higher quality psychotherapeutic knowledge. wisdom quality wisdom quality is wisdom’s fitness for purpose. if, ceteris paribus, increasing public spending on education results in a superior society than increasing public spending on law enforcement, then, increasing public spending on education constitutes higher quality wisdom. table 4: proposed definitions of data quality, information quality, knowledge quality, and wisdom quality australasian journal of information systems volume 18 number1 2013 14 empirical exploration this section presents an empirical exploration of the dikw hierarchy and the associated quality terms. methodology following wittgenstein’s ordinary language philosophy (wittgenstein, 2001), we reviewed 20 online news articles for their uses of the terms data, information, knowledge, and wisdom (five articles for each term). ordinary language philosophy considers traditional philosophical problems (in our case the dikw hierarchy) as being caused by language misunderstandings, and argues that focusing on the uses of the terms in ordinary language may alleviate such misunderstandings (boland et al., 2010). it “focuses on language not as an abstract system but in the actual context in which it functions in our daily life and activities” (weinzweig, 1977, p. 118). as such, recent online news articles containing relevant terms were identified using google news; each of the terms was searched for individually, and the top five articles were reviewed. direct quotes containing relevant terms were extracted, and are presented in table 5. terms data quality, information quality, knowledge quality, and wisdom quality returned a very limited number of search results in google news (256, 18, 0, and 0). as it appeared that data quality and information quality were used synonymously in those news articles, the quality related terms were excluded from this part of the empirical analysis. at the same time, we also initiated and facilitated an online focus group discussion comprising 16 information systems experts (practitioners and academics), who were identified through the international association for information and data quality (iaidq). the participants were asked to define and discuss each term in the dikw hierarchy as well as the associated quality terms. overall, the focus group discussion resulted in 28 sequential responses totalling approximately 5,000 words. rooted in wittgenstein’s ordinary language philosophy, the data were subsequently analysed using keywords-in-context (kwic) analysis, which is an appropriate analysis method when “there are specific words that are of interest to the researcher” (leech and onwuegbuzie, 2007, p. 566). as the name implies, kwic involves searching for a particular keyword in a text and analysing its local meaning in relation to a fixed number of words immediately preceding and following it. results direct quotes from the online articles, representing ordinary language uses of each of the terms, are shown in table 5. starting with data, one aspect that immediately stands out is the fact that all quotes seem to be using data in its plural form, thus, referring to it in terms of a set of data and not in terms of an individual datum. considering information, it is difficult to identify any significant usage differences compared with the term data. as such, in the context of ordinary language, the results indicate that the terms data and information may potentially be interchangeable in most situation. with reference to knowledge, it is clear that all quotes refer to it in terms of a subject (i.e. a knowing person). the quotes refer to the knowledge of institutions, history, events, and things. as such, there appears to be a clear difference between knowledge on one hand and data and information on the other. considering wisdom, there seems to be some overlap with knowledge. however, some of the quotes seem to imply that wisdom includes value judgments—e.g. political and religious wisdom as well as investment related wisdom. whilst the focus group discussion did not reach significant consensus, it did produce a rich data set representing a wide range of views from expert practitioners and academics. however, at least one participant was not convinced that reaching consensus was even possible, writing: australasian journal of information systems volume 18 number1 2013 15 w is do m conventional wisdom says that president obama is on the road to victory for re-election in 2012 (dunn, 2011). as laurie brlas, chief financial officer of cliffs natural resources inc. (clf), prepared to announce a deal to buy a rival mining company this year, she was confident investors would eventually see the wisdom of her company’s plan (mider and mccracken, 2011). don't ignore ancient wisdom in predicting monsoon, drought (gupta, 2011). jfk's wisdom on religion and politics (lat, 2011). this time we review warren buffett’s 1987 annual shareholder letter for his accumulated investing nuggets of wisdom (gf, 2011). k no w le dg e knowledge on hst makes all the difference (nichol, 2011). to mark this event and to enhance the knowledge of the students of various courses of father muller charitable institutions, a new building with state-of-the-art facilities was built (margaret, 2011). know your boston bruins history? brush up on some stanley cup knowledge, beantown style (dmp, 2011). good knowledge of history is the mark of an educated and well-rounded person who is likely to be a good steward of the society where he or she lives and works (tsp, 2011). crowds keen for food knowledge at cockermouth's fell gather (aap, 2011). in fo rm at io n a former expert-network consultant provided "to the decimal point" confidential financial information about technology companies to her hedge-fund clients in a "brazen insider-trading scheme," prosecutors said as her trial wound down on thursday (wsj, 2011). vivek kundra, the white house’s first chief information officer, will leave in august after two and a half years in the post (southall, 2011). google seeks to put users back in control of their online information with a service called me on the web that monitors your reputation (claburn, 2011). the government must understand that montreal’s residents and off-island commuters need specific, detailed and daily information on the status of roads and bridges and what measures have been take to alleviate the congestion and help people get around (mg, 2011). latest traffic information from transport nsw (margaret, 2011). d at a overland conceded that the force had failed to properly qualify the data for assaults in melbourne's cbd, but he said there was no basis for a finding that the error of judgment was a case of ‘conscious manipulation’ (ferguson, 2011). the early estimate of a 0.9 per cent rise in the implicit price deflator for goods imports, a measure of price changes also noted with the march trade data, was confirmed by the abs on wednesday (fairfaxdigital, 2011). australian stockmarket opens higher after positive us data (aap, 2011). mr nanterme said the world of data was now more volatile, more complex and expanding at speed. … ‘this plays in favour of having stronger analytics,’ he said. ‘plus, there's just so much more data that you need to understand (dearne, 2011). this is encouraging data and a good first indication that the human papilloma virus (hpv) vaccine is effective at preventing cervical cell abnormalities (bt, 2011). table 5: ordinary language usages australasian journal of information systems volume 18 number1 2013 16 there is no answer but that doesn't mean that the contemplation isn't worthwhile. only let's not contemplate with the idea that we're going to discover the answer. … most of what is being put forward as definitions really amounts to opinions. —data quality/governance professional other participants attempted to define the terms through asymmetric explanations; in other words, to define/explain information in terms of data, knowledge in terms of information, and wisdom in terms of knowledge. for instance: “information” and “data” from an academic perspective are different, as information is seen as using data in a given context. —data quality and master data professional it all begins with data—it is the seed for information, wisdom and knowledge. —business information manager data is seen to be a raw diamond from the mine and once polished or placed on a ring it's information. and knowledge and wisdom are defined from information. —data manager information is a collection of bits of data organized in a way that makes sense. … wisdom is very much an intangible thing. it's the collection of experience and knowledge that a person builds up over time which allows them to make good decisions. —data governance architect i … would say that knowledge is the collection of a body of data or information. wisdom is the ability to apply thought processes to make sense of available data, information and/or knowledge. —business intelligence professional as can be seen, many definitions/explanations relied on analogies and metaphors like “data is seen to be a raw diamond” and “data … is the seed for information”. as such, they were not very specific and thus open to interpretation. even “information is seen as using data in a given context” does not clearly specify the nature of the context and its specific relation to the user, data, and information. defining data also proved challenging. some proposed definitions included the following: data, as we speak of it, is a more-or-less direct representation of circumstance. —data quality analyst [data is] a representation of reality. —crm and mdm professional data has been defined as "raw facts" and information is "data in context". —enterprise information management professional data is just a collection of instantaneous observations. i weigh 170 lbs. the car i see is red. there are 10 people standing in line at starbucks at 2pm on a tuesday. that is data. data by itself is not meaningful. it is the fundamental building block for everything above it. —data governance architect the quote above states that data by itself is not meaningful, yet it provides examples like “the car i see is red” and “there are 10 people standing in line at starbucks at 2pm on a tuesday”. are these instantaneous observations not meaningful? it turned out the discussion of the quality related terms provided additional context and shed more light on the basic terms and their relationships. nevertheless, some participants argued that there was no real difference between data and information quality. for instance: australasian journal of information systems volume 18 number1 2013 17 the terms “data quality” and “information quality” are indeed often used synonymously. … experienced from my daily work with people from business and it staff i don’t think people in general care about the difference between “data quality” and “information quality”. —data quality and master data professional on the other hand, others argued: i think that "data quality" and information quality" are related, but not really interchangeable. —enterprise information management professional information quality depends highly on data quality. —data governance architect one participant explained the difference in terms of an internal versus external view: i don't think there is much dispute that we can distinguish between data and information … i approach it from an internal versus external view. i see data quality as the internal view in an enterprise focused on whether the data … held in the information systems meets the specifications that have been defined for it. in another sense dq is the engineering view of "does the data meet the specifications?". on the other hand i see information quality as more of an external view of users of the information products produced by the system. —professor of information science an example provided by another participant further clarified this potential distinction: now … we can … conclude that the data comprising the content of this thread is high quality (i.e. the characters making up the words, sentences and paragraphs) but the information (our thoughts constructed of the data) is low quality… —customer relationship management and master data management professional another participant elaborated on the relationship between data quality and information by arguing: i've come to the conclusion that information is high quality data. … if the customer wants to be informed (wants information), and high quality data should do just that, then they are one and the same. put another way, if data is not high quality then it is not information. is "high quality use of information" the same as knowledge or wisdom? —data strategy and standards professional however, another participant argued: quality is the combination of the objective and subjective … we spend way too much time on the objective, declining to consider the subjective as too squishy or unmeasurable. —customer relationship management and master data management professional defining knowledge quality and wisdom quality proved much more difficult. one of the participants explained: if i used “knowledge quality” or “wisdom quality” i guess people will look at me as pretentious. —data quality and master data professional compared with the news articles, which did not seem to differentiate between data and information, the focus group discussion did attempt to draw a distinction between these two terms. this may be explained through the fact that the focus group comprised information systems specialists; nevertheless, they did not manage to reach significant consensus. while even less consensus regarding the terms knowledge and wisdom was reached, it was noted that they depend on data and information (e.g. knowledge is the collection of a body of data or information), and that wisdom should help people make good decisions. australasian journal of information systems volume 18 number1 2013 18 concluding remarks in this paper, we aimed to comprehensively review and provide a high-level synthesis of the literature on the dikw hierarchy, its constituent elements, and its quality dimension. furthermore, we attempted to ground the hierarchy in a semiotic framework with a view of providing clear and consistent definitions (see tables 3 and 4). whilst several researchers have already critically reviewed the dikw hierarchy and its constituent elements (rowley, 2007, frické, 2009, zins, 2011), building on falkenberg et al. (1998) this paper makes a theoretical contribution by explicitly grounding the hierarchy in the literature on semiotics (beynon-davies, 2007, 2009c, b, a, 2011, morris, 1938, peirce, 1998, saussure, 1983), and by exploring the relationship between the hierarchy and its quality dimension. the paper also offers an empirical contribution by providing further evidence for the lack of consistency in how the relevant terms are used in every-day language as well as by information systems experts. acknowledgement the authors would like to thank two anonymous reviewers for their constructive comments provided on earlier drafts of this paper. references aamodt, a. & nygård, m. 1995. different roles and mutual dependencies of data, information, and knowledge an ai perspective on their integration. data and knowledge engineering, 16, 191-222. aap. 2011. australian stockmarket opens higher after positive us data [online]. available: http://www.theaustralian.com.au/business/markets/australian-stockmarket-opens-higher-afterpositive-us-data/story-e6frg916-1226076957825 [accessed 17 june 2011]. ackoff, r. l. 1989. from data to wisdom. journal of applied systems analysis, 16, 3-9. alavi, m. & leidner, d. e. 2001. review: knowledge management and knowledge management systems: conceptual foundations and research issues. mis q., 25, 107-136. alkhattabi, m., neagu, d. & cullen, a. 2011. assessing information quality of e-learning systems: a web mining approach. comput. hum. behav., 27, 862-873. allen, j. p. 2000. information systems as technological innovation. information technology & people, 13, 210-221. asq 2002. quality glossary. quality progress, 35. awad, e. m. & ghaziri, h. m. 2004. knowledge management, upper saddle river, nj, pearson education international. baškarada, s. 2010. information quality management capability maturity model, wiesbaden, vieweg+teubner. baškarada, s. 2011a. how spreadsheet applications affect information quality. journal of computer information systems, 51, 77-84. baškarada, s. 2011b. towards a semiotic information position framework for network centric warfare. 16th international command and control research and technology symposia (iccrts). québec city, canada. beynon-davies, p. 2007. informatics and the inca. international journal of information management, 27, 306-318. http://www.theaustralian.com.au/business/markets/australian-stockmarket-opens-higher-after-positive-us-data/story-e6frg916-1226076957825 http://www.theaustralian.com.au/business/markets/australian-stockmarket-opens-higher-after-positive-us-data/story-e6frg916-1226076957825 australasian journal of information systems volume 18 number1 2013 19 beynon-davies, p. 2009a. the ‘language’ of informatics: the nature of information systems. international journal of information management, 29, 92-103. beynon-davies, p. 2009b. neolithic informatics: the nature of information. international journal of information management, 29, 3-14. beynon-davies, p. 2009c. significant threads: the nature of data. international journal of information management, 29, 170-188. beynon-davies, p. 2011. in-formation on the prairie: signs, patterns, systems and prairie dogs. international journal of information management, 31, 307-316. bierly, p. e., kessler, e. h. & christensen, e. w. 2000. organizational learning, knowledge and wisdom. journal of organizational change management, 13, 595-618. bocij, p., chaffey, d., greasley, a. & hickie, s. 2003. business information systems: technology, development and management for the e-business, harlow, ft prentice hall. boddy, d., boonstra, a. & kennedy, g. 2005. managing information systems: an organizational perspective, harlow, ft prentice hall. boisot, m. & canals, a. 2004. data, information and knowledge: have we got it right? journal of evolutionary economics, 14, 43-67. boland, r. j., newman, m. & pentland, b. t. 2010. hermeneutical exegesis in information systems design and use. information and organization, 20, 1-20. available: http://www.sciencedirect.com/science/article/pii/s1471772709000323. bt. 2011. cancer vaccine data 'encouraging' [online]. available: http://www.belfasttelegraph.co.uk/news/local-national/uk/cancer-vaccine-data-encouraging16012890.html [accessed 17 june 2011]. burstein, f. & linger, h. 2003. supporting post-fordist work practices: a knowledge management framework for supporting knowledge work. information technology & people, 16, 289-305. chaffey, d. & wood, s. 2005. business information management: improving performance using information systems, harlow, ft prentice hall. chandler, d. 2007. semiotics: the basics, routledge. chen, m., ebert, d., hagen, h., laramee, r. s., liere, r. v., ma, k.-l., ribarsky, w., scheuermann, g. & silver, d. 2009. data, information, and knowledge in visualization. ieee computer graphics and applications, 29, 12-19. chisholm, r. m. 1982. the foundations of knowing, minneapolis, university of minnesota press. claburn, t. 2011. google suggests googling yourself [online]. available: http://www.informationweek.com/news/internet/google/230800083 [accessed 17 june 2011]. claver, e., llopis, j., gonzález, m. r. & gascó, j. l. 2001. the performance of information systems through organizational culture. information technology & people, 14, 247-260. crosby, p. 1979. quality is free, new york, mcgraw-hill. dampney, k., busch, p. & richards, d. 2002. the meaning of tacit knowledge. australasian journal of information systems, 10, 3-13. available: http://dl.acs.org.au/index.php/ajis/article/view/438 [accessed jan 2012]. dearne, k. 2011. sas, accenture in data analytics alliance [online]. available: http://www.theaustralian.com.au/australian-it/sas-accenture-in-data-analytics-alliance/storye6frgakx-1226076874647 [accessed 17 june 2011]. deely, j. 1990. basics of semiotics, bloomington, in, indiana university press. deming, w. e. 2000. out of the crisis, mit press. http://www.sciencedirect.com/science/article/pii/s1471772709000323 http://www.belfasttelegraph.co.uk/news/local-national/uk/cancer-vaccine-data-encouraging-16012890.html http://www.belfasttelegraph.co.uk/news/local-national/uk/cancer-vaccine-data-encouraging-16012890.html http://www.informationweek.com/news/internet/google/230800083 http://dl.acs.org.au/index.php/ajis/article/view/438 http://www.theaustralian.com.au/australian-it/sas-accenture-in-data-analytics-alliance/story-e6frgakx-1226076874647 http://www.theaustralian.com.au/australian-it/sas-accenture-in-data-analytics-alliance/story-e6frgakx-1226076874647 australasian journal of information systems volume 18 number1 2013 20 dmp. 2011. know your boston bruins history? brush up on some stanley cup knowledge, beantown style [online]. available: http://middletownpress.com/articles/2011/06/16/sports/doc4dfaaf7cc4a9b828770607.txt [accessed 17 june 2011]. dunn, c. w. 2011. dunn: conventional wisdom that says nobama [online]. available: http://www.washingtontimes.com/news/2011/jun/16/conventional-wisdom-that-says-nobama/ [accessed 17 june 2011]. eco, u. 1979. a theory of semiotics, indiana university press. edwards, c. d. 1968. the meaning of quality. quality progress. endsley, m. r. 1995. toward a theory of situation awareness in dynamic systems. human factors: the journal of the human factors and ergonomics society, 37, 32-64. available: http://hfs.sagepub.com/content/37/1/32.abstract. fairfaxdigital. 2011. preview of next week's economic data [online]. available: http://www.tradingroom.com.au/apps/view_breaking_news_article.ac?page=/data/news_resear ch/published/2011/6/168/catf_110617_141200_5632.html [accessed 17 june 2011]. falkenberg, e. d., hesse, w., lindgreen, p., nilsson, b. e., oei, j. l. h., rolland, c., stamper, r. k., assche, f. j. m. v., verrijn-stuart, a. a. & voss, k. 1998. a framework of information system concepts: the frisco report, the netherlands, department of computer science, university of leiden. fayyad, u. m., piatetsky-shapiro, g. & smyth, p. 1996. from data mining to knowledge discovery: an overview. ai magazine, 17, 37-54. feigenbaum, a. v. 1986. total quality control, new york, ny, mcgraw-hill. ferguson, j. 2011. 'widespread flaws' in crime data collection [online]. available: http://www.theaustralian.com.au/national-affairs/widespread-flaws-in-crime-datacollection/story-fn59niix-1226076689823 [accessed 17 june 2011]. fiordo, r. a. 1977. charles morris and the criticism of discourse, john benjamins publishing company. frické, m. 2009. the knowledge pyramid: a critique of the dikw hierarchy. journal of information science, 35, 131–142. galliers, r. & newell, s. 2001. back to the future: from knowledge management to data management. proceedings of ecis. bled, slovenia. garvin, d. 1987. competing on the eight dimensions of quality. harvard business review, 65, 101109. gettier, e. 1963. is justified true belief knowledge? analysis, 23, 121–23. gf. 2011. in search of investment wisdom — a review of berkshire hathaway’s 1987 annual shareholder letter [online]. available: http://www.gurufocus.com/news/136560/in-search-ofinvestment-wisdom--a-review-of-berkshire-hathaways-1987-annual-shareholder-letter [accessed 17 june 2011]. gosain, s. 2003. issues in designing personal knowledge exchanges: first movers analyzed. information technology & people, 16, 306-325. gray, p. h. & meister, d. b. 2003. introduction: fragmentation and integration in knowledge management research. information technology & people, 16, 259-265. groff, t. r. & jones, t. p. 2003. introduction to knowledge management: km in business, amsterdam, butterworth heinemann. http://middletownpress.com/articles/2011/06/16/sports/doc4dfaaf7cc4a9b828770607.txt http://www.washingtontimes.com/news/2011/jun/16/conventional-wisdom-that-says-nobama/ http://hfs.sagepub.com/content/37/1/32.abstract http://www.tradingroom.com.au/apps/view_breaking_news_article.ac?page=/data/news_research/published/2011/6/168/catf_110617_141200_5632.html http://www.tradingroom.com.au/apps/view_breaking_news_article.ac?page=/data/news_research/published/2011/6/168/catf_110617_141200_5632.html http://www.theaustralian.com.au/national-affairs/widespread-flaws-in-crime-data-collection/story-fn59niix-1226076689823 http://www.theaustralian.com.au/national-affairs/widespread-flaws-in-crime-data-collection/story-fn59niix-1226076689823 http://www.gurufocus.com/news/136560/in-search-of-investment-wisdom--a-review-of-berkshire-hathaways-1987-annual-shareholder-letter http://www.gurufocus.com/news/136560/in-search-of-investment-wisdom--a-review-of-berkshire-hathaways-1987-annual-shareholder-letter australasian journal of information systems volume 18 number1 2013 21 gupta, s. 2011. don't ignore ancient wisdom in predicting monsoon, drought [online]. available: http://articles.economictimes.indiatimes.com/2011-06-17/news/29670079_1_rainfallforecasts-monsoon [accessed 17 june 2011]. hara, n. & hew, k. f. 2007. knowledge-sharing in an online community of health-care professionals. information technology & people, 20, 235-261. hardie, n. & walsh, p. 1994. toward a better understanding of quality. international journal of quality & reliability management, 11, 53-63. harré, r. & tissaw, m. a. 2005. wittgenstein and psychology: a practical guide, ashgate publishing company. hasan, h. 2002. information systems as dialectic, tool mediated activity. australasian journal of information systems, 10, 30-39. available: http://dl.acs.org.au/index.php/ajis/article/view/441 [accessed jan 2012]. jashapara, a. 2005. knowledge management: an integrated approach, harlow, ft prentice hall. jason, l. a., reichler, a., king, c., madsen, d., camacho, j. & marchese, w. 2001. the measurement of wisdom: a preliminary effort. journal of community psychology, 29, 585– 598. jessup, l. m. & valacich, j. s. 2003. information systems today, upper saddle river, nj, prentice hall. jeste, d., ardelt, m., blazer, d. & meeks, t. 2010. expert consensus on characteristics of wisdom: a delphi method study. gerontologist, 50, 668–680. juran, j. m. 1974. quality control handbook, new york, ny,, mcgraw-hill. kallinikos, j. 2006. information out of information: on the self-referential dynamics of information growth. information technology & people, 19, 98-115. kamhawi, e. m. 2010. the three tiers architecture of knowledge flow and management activities. information and organization, 20, 169-186. available: http://www.sciencedirect.com/science/article/pii/s1471772710000321 [accessed 2010/10//]. kock, n. f., mcqueen, r. j. & corner, j. l. 1997. the nature of data, information and knowledge exchanges in business processes: implications for process improvement and organizational learning. learning organization, 4, 70-80. kosso, p. 2011. a summary of scientific method, springer. lat. 2011. jfk's wisdom on religion and politics; secure communities in california; the rights and protections of marriage [online]. available: http://www.latimes.com/news/opinion/letters/lale-0617-friday-20110617,0,3661883.story [accessed 17 june 2011]. laudon, k. c. & laudon, j. p. 2006. management information systems: managing the digital firm, upper saddle river, nj, pearson prentice hall. lee, y., strong, d., kahn, b. & wang, r. 2002. aimq: a methodology for information quality assessment. information management, 40, 133-146. leech, n. & onwuegbuzie, a. 2007. an array of qualitative data analysis tools: a call for data analysis triangulation. school psychology quarterly, 22, 557-584. leonardi, p. m. & treem, j. w. 2012. knowledge management technology as a stage for strategic self-presentation: implications for knowledge sharing in organizations. information and organization, 22, 37-59. available: http://www.sciencedirect.com/science/article/pii/s1471772711000510. http://articles.economictimes.indiatimes.com/2011-06-17/news/29670079_1_rainfall-forecasts-monsoon http://articles.economictimes.indiatimes.com/2011-06-17/news/29670079_1_rainfall-forecasts-monsoon http://dl.acs.org.au/index.php/ajis/article/view/441 http://www.sciencedirect.com/science/article/pii/s1471772710000321 http://www.latimes.com/news/opinion/letters/la-le-0617-friday-20110617,0,3661883.story http://www.latimes.com/news/opinion/letters/la-le-0617-friday-20110617,0,3661883.story http://www.sciencedirect.com/science/article/pii/s1471772711000510 australasian journal of information systems volume 18 number1 2013 22 linger, h. & warne, l. 2001. making the invisible visible: modelling social learning in a knowledge management context. australasian journal of information systems, 9, 56-66. available: http://dl.acs.org.au/index.php/ajis/article/view/214/186 [accessed jan 2012]. madnick, s. e., wang, r. y., lee, y. w. & zhu, h. 2009. overview and framework for data and information quality research. acm journal of data and information quality, 1. margaret, j. 2011. latest traffic information from transport nsw [online]. available: http://blogs.abc.net.au/nsw/2011/06/latest-traffic-information-fromtransport-nsw.html [accessed 17 june 2011]. martin, c., philp, w. & hall, w. 2009. temporal convergence for knowledge management. australasian journal of information systems, 15, 133-148. available: http://dl.acs.org.au/index.php/ajis/article/view/207/480 [accessed jan 2012]. mcdermott, r. 1999. why information technology inspired but cannot deliver knowledge managennent. california management journal, 41, 103-117. meeks, t. & jeste, d. 2009. neurobiology of wisdom. arch gen psychiatry, 66, 355-365. mg. 2011. the gazette's view: on roads and bridges, we need fixes – and information [online]. available: http://www.montrealgazette.com/gazette+view+roads+bridges+need+fixes+information/4960 712/story.html [accessed 17 june 2011]. mider, z. r. & mccracken, j. 2011. record stock gains for acquirers defy wall street selloff wisdom [online]. available: http://www.bloomberg.com/news/2011-06-16/record-stockgains-for-buyers-defy-wall-street-wisdom-on-takeover-selloffs.html [accessed 17 june 2011]. morris, c. w. 1938. foundations of the theory of signs, chicago, chicago university press. nichol, a. 2011. knowledge on hst makes all the difference [online]. available: http://www.coastreporter.net/article/20110617/sechelt0302/306179970/1/sechelt/knowledge-on-hst-makes-all-the-difference [accessed 17 june 2011]. nozick, r. 1981. philosophical explanations, harvard university press. o'donovan, b. & roode, d. 2002. a framework for understanding the emerging discipline of information systems. information technology & people, 15, 26-41. østerlie, t., almklov, p. g. & hepsø, v. 2012. dual materiality and knowing in petroleum production. information and organization, 22, 85-105. available: http://www.sciencedirect.com/science/article/pii/s1471772712000024. pearlson, k. e. & saunders, c. s. 2004. managing and using information systems: a strategic approach, new york, wiley. peirce, c. s. 1998. the essential peirce, volume 2: selected philosophical writings, 1893-1913, indiana university press. piaget, j. 1977. the development of thought: equilibrium of cognitive structures, ny, viking press. pipino, l., lee, y. & wang, r. 2002. data quality assessment. communications of the acm, 45, 211-218. pipino, l., wang, r., kopcso, d. & rybolt, w. 2005. developing measurement scales for dataquality dimensions. in: wang, r. y., pierce, e. m., madnick, s. e. & fisher, c. w. (eds.) information quality. armonk, ny: m.e. sharpe. plato theaetetus popper, k. r. 1979. objective knowledge: an evolutionary approach, clarendon press. http://dl.acs.org.au/index.php/ajis/article/view/214/186 http://blogs.abc.net.au/nsw/2011/06/latest-traffic-information-fromtransport-nsw.html http://dl.acs.org.au/index.php/ajis/article/view/207/480 http://www.montrealgazette.com/gazette+view+roads+bridges+need+fixes+information/4960712/story.html http://www.montrealgazette.com/gazette+view+roads+bridges+need+fixes+information/4960712/story.html http://www.bloomberg.com/news/2011-06-16/record-stock-gains-for-buyers-defy-wall-street-wisdom-on-takeover-selloffs.html http://www.bloomberg.com/news/2011-06-16/record-stock-gains-for-buyers-defy-wall-street-wisdom-on-takeover-selloffs.html http://www.coastreporter.net/article/20110617/sechelt0302/306179970/-1/sechelt/knowledge-on-hst-makes-all-the-difference http://www.coastreporter.net/article/20110617/sechelt0302/306179970/-1/sechelt/knowledge-on-hst-makes-all-the-difference http://www.sciencedirect.com/science/article/pii/s1471772712000024 australasian journal of information systems volume 18 number1 2013 23 price, r., neiger, d. & shanks, g. 2008. developing a measurement instrument for subjective aspects of information quality. communications of the association for information systems, 22, 49-74. redman, t. c. 1996. data quality for the information age, boston, artech house. rooney, d., hearn, g. & ninan, a. 2008. handbook on the knowledge economy, edward elgar. rorty, r. 1982. consequences of pragmatism: essays 1972-1980, university of minnesota press. rowley, j. 2007. the wisdom hierarchy: representations of the dikw hierarchy. journal of information science, 33, 163–180. russell, b. 1912. the problems of philosophy, feedbooks. saussure, f. d. 1983. course in general linguistics, london, duckworth. shannon, c. 1948. a mathematical theory of communication. bell system technical journal, 27, 379–423 & 623–656. shewhart, w. a. 1931. economic control of quality of manufactured product, new york, ny, van nostrand. shields, r. 2001. the new 'knowledge speak': the implications of contested definitions of knowledge and information. australasian journal of information systems, 8, 142-149. available: http://dl.acs.org.au/index.php/ajis/article/view/250/221 [accessed jan 2012]. singh, s. 1999. the code book: the science of secrecy from ancient egypt to quantum cryptography, newyork, ny, anchor books. skinner, b. f. 2011. about behaviorism, knopf doubleday publishing group. southall, a. 2011. information officer is leaving [online]. available: http://www.nytimes.com/2011/06/17/us/politics/17brfs-kundra.html?_r=2 [accessed 17 june 2011]. stamper, r., liu, k., hafkamp, m. & ades, y. 2000. understanding the roles of signs and norms in organisations. journal of behaviour & information technology, 19, 15-27. stamper, r. k. 1973. information in business and administrative systems, london, batsford. stenmark, d. 2002. information vs. knowledge: the role of intranets in knowledge management. proceedings of the 35th hawaii international conference on system sciences. hawaii, usa. sternberg, r. 2000. wisdom as a form of giftedness. gifted child quarterly, 44, 252 – 260. sternberg, r. 2005. older but not wiser? the relationship between age and wisdom. ageing international, 30, 5-26. available: http://dx.doi.org/10.1007/bf02681005. stumpf, s. & mcdonnell, j. 2003. data, information and knowledge quality in retail security decision making. 3rd international conference on knowledge management (iknow’03). graz. takahashi, m. & overton, w. cultural foundations of wisdom: an integrated developmental approach. in: sternberg, r. j. & jordan, j. (eds.) a handbook of wisdom: psychological perspectives. tsp. 2011. our view: disturbing trend: students with poor knowledge of history [online]. available: http://www.thestarpress.com/article/20110616/opinion01/106160318 [accessed 17 june 2011]. tuomi, i. 1999. data is more than knowledge: implications of the reversed knowledge hierarchy for knowledge management and organizational memory. in: hicss-32. proceedings of the 32nd annual hawaii international conference on system sciences, 5-8 jan maui, hi , usa. http://dl.acs.org.au/index.php/ajis/article/view/250/221 http://www.nytimes.com/2011/06/17/us/politics/17brfs-kundra.html?_r=2 http://dx.doi.org/10.1007/bf02681005 http://www.thestarpress.com/article/20110616/opinion01/106160318 australasian journal of information systems volume 18 number1 2013 24 turban, e., rainer, r. k. & potter, r. e. 2005. introduction to information technology, new york, wiley. wang, r. y. & strong, d. m. 1996. beyond accuracy: what data quality means to data consumers. journal of management information systems, 12, 5-34. weinberger, d. 2010. the problem with the data-information-knowledge-wisdom hierarchy. hbr blog network [online]. available: http://blogs.hbr.org/cs/2010/02/data_is_to_info_as_info_is_not.html [accessed 18 feb 2013]. weinzweig, m. 1977. phenomenology and ordinary language philosophy. metaphilosophy, 8, 116-146. available: http://dx.doi.org/10.1111/j.14679973.1977.tb00267.x. williams, m. c. 2001. ethics in research. australasian journal of information systems, 8, 68-74. available: http://dl.acs.org.au/index.php/ajis/article/viewfile/244/214 [accessed jan 2012]. wittgenstein, l. 2001. philosophical investigations, carlton, vic, australia, wiley-blackwell. wittgenstein, l., anscombe, g. e. m. & wright, g. h. 1975. on certainty, blackwell. wsj. 2011. first expert-network trial winds down [online]. available: http://online.wsj.com/article/sb10001424052702303499204576389873708709788.html [accessed 17 june 2011]. zeleny, m. 1987. management support systems: towards integrated knowledge management. human systems management, 7, 59-70. zemanek, h. 1966. semiotics and programming languages. communications of the acm, 9, 139143. zhao, j., de pablos, p. o. & qi, z. 2012. enterprise knowledge management model based on china’s practice and case study. computers in human behavior, 28, 324-330. available: http://www.sciencedirect.com/science/article/pii/s0747563211002093. zins, c. 2011. conceptual approaches for defining data, information, and knowledge. journal of the american society for information science and technology, 58, 479-493. http://blogs.hbr.org/cs/2010/02/data_is_to_info_as_info_is_not.html http://dx.doi.org/10.1111/j.1467-9973.1977.tb00267.x http://dx.doi.org/10.1111/j.1467-9973.1977.tb00267.x http://dl.acs.org.au/index.php/ajis/article/viewfile/244/214 http://online.wsj.com/article/sb10001424052702303499204576389873708709788.html http://www.sciencedirect.com/science/article/pii/s0747563211002093 data, information, knowledge, wisdom (dikw): a semiotic theoretical and empirical exploration of the hierarchy and its quality dimension abstract introduction theoretical exploration the dikw hierarchy data and information knowledge wisdom the quality dimension semiotics the semiotic ladder semiotic dikw framework proposed definitions empirical exploration methodology results concluding remarks acknowledgement references ajis vol. 7, no 2 may 2000 editorial this is the second issue of volume 7 of the australian journal of information systems. as editor i am pleased to report the ongoing interest in the journal both in australia as well as overseas. our first paper efficient mobile client caching supporting transaction semantics is presented by ii young chung & chong sun hwang, department of computer science and engineering, korea university. this paper argues that in mobile client-server database systems, caching of frequently accessed data is an important technique that reduces the contention on narrow bandwidth channels. the paper proposes a caching policy to maintain cache consistency for mobile computers. the proposed protocol adopts asynchronous (non-periodic) broadcasting as the cache invalidation scheme, and supports transaction semantics in mobile environments. peter clutterbuck, faculty of business, university of the sunshine coast, provides our next paper increased availability for networked software services. the availability of networked software services is an increasingly important quality of service issue. this paper develops from a user perspective a quality of service framework for the availability of networked software services. the paper utilises this quality of service framework to suggest the functional characteristics of a new transport layer software specification that would facilitate the increased availability of networked software services. our third paper is tribes of users and systems developers sharon dingley, school of computing, university of central england, hanifa shah, school of computing, staffordshire university & paul golder, department of computer science and applied mathematics, aston university. this paper introduces the concepts of 'culture' and 'tribe' into the discussion of the relationship between business users and information systems developers. previous research has focused on identifying specific barriers to user-systems developer communication and on ways of eliminating these barriers. in contrast, this paper suggests that much can be learnt through the recognition of cultural differences inherent to the differing roles of user and systems developer. bruce lo, school of multimedia & information technology, southern cross university & cede sri darma, pendidikan national university, provide our next paper employee perception of the impact of information technology investment in organizations: a survey of the hotel industry. this paper examines how employees view the impact and value of it, specifically from the perspectives of it usage, satisfaction, changes in performance of employees and changes in organizational performance. our fifth paper arguing for phd coursework is provided by mike metcalfe, information systems doctoral school, university of south australia & margaret kiley, advisory centre for university education, adelaide university. this paper argues for the establishment of semi-structured programs for phd students, but importantly, one that has an appreciation that research is an argumentative process, trying to develop knowledgeable audiences, and thus is a social activity. our sixth paper testing hammer and stanton's reengineering-success diagnostic is provided by felicity murphy, peter seddon, department of information systems, university of melbourne & sandy staples queens university, kingston ontario. this paper looks at the following: (1) investigating the organisational context prior to the embarkation on projects of major change in australia, (2) testing the validity of a diagnostic proposed by hammer and stanton (1995) to be used when assessing the ability of organisations to succeed at reengineering, and (3) examining the impact of the organisational environment prior to change on the outcome of reengineering projects. gregory shailer, department of commerce, australian national university, kenneth stevens, school of information systems, technology and management, university of new south wales & hargo utomo, department of commerce, australian national university provide our next paper eftpos impacts on branch banking: an extraorganisational analysis. this paper examines the impact of eftpos on branch banking activities using internal and extraorganisational criteria. impacts are identified in terms of efficiency, effectiveness and competitiveness. leoni warne, joint systems branch, defence science and technology organisation provides our next paper understanding organisational learning in military headquarters: findings from a pilot study. this paper reports on the findings of a pilot study for a longer term research project investigating organisational learning in command and control organisations in the australian defence force. the aim of the pilot study was to determine the feasibility of observing and documenting social learning processes and to trial the use of ethnographic techniques for this purpose. our last group of papers are the best papers from the recent ozchi99 conference. our next paper the ethics/skills interface in image manipulation jenny webber, school of communication and information studies, university of south australia — looks at image manipulation and its use in advertising, electronic media etc. the paper examines the question of ethics and copyright of such images. our next paper the integrated decision model in emergency dispatch management and its implications for design william wong, department of information science, university of otago, examines a number of decision making models and integrates these into a single model which is validated in emergency dispatch management situations. saul greenberg, department of computer science, university of calgary, geraldine fitzpatrick, crc for distributed systems technology, university of queensland, carl gutwin, department of computer science, ajis vol. 7, no 2 may 2000 university of saskatchewan & simon kaplan, department of computer science and electrical engineering provide our next paper adapting the locales framework for heuristic evaluation of groupware. this paper extends the single user heuristic evaluation by taking locales frameworks and restating them as heuristics capable of evaluating groupware. our next paper is ecological interface design for anaesthesia monitoring marcus watson, swinburne computer-human interaction laboratory, john russell, department of anaesthesia and intensive care, royal adelaide hospital & penelope sanderson swinburne computer-human interaction laboratory. this paper looks at the requirements for an information environment for anaesthetists. james phillips & thomas triggs, psychology department, monash university provide our next paper entitled cursor control device characteristics. this paper looks at improvement in cursor control by addressing the efficiency devices such as the mouse, trackball etc. our final paper issues of page representation and organisation in web browser's revisitation tools andy cockburn, department of computer science, university of canterbury, new zealand & saul greenberg, department of computer science, university of calgary looks at graphical revisitation tools that let users return to previously seen pages. a number of alternative designs are discussed. once again i would like to pass my thanks on to the authors and consulting editors for the high quality of material in theis issue. i would like to thank lily soh for her continued assistance with the journal and diniz da rocha for the maintenance of the home page. rob macgregor editor http://www.uow.edu.au/ajis/ajis.html paper title (paper title style) australasian journal of information systems someh & shanks 2016, vol 20, selected papers from the isf conference emergence of business value emergence of business value from complementary interactions between informational and transactional it systems ida asadi someh department of computing and information systems, the university of melbourne ida.asadi@unimelb.edu.au graeme shanks department of computing and information systems, the university of melbourne gshanks@unimelb.edu.au abstract different it asset classes generate business value consistent with the strategic goal of that asset class. while transactional it systems contribute to process efficiencies, informational it systems generate insights and contribute to informational benefits. we argue that the complementary interactions between these two classes of it systems can support both informed and efficient processes and lead to greater business value. we use systems theory to develop a research model, which includes the complementary interactions between transactional and informational it systems and the emergence of higher-order it-enabled business systems. the two it systems interact and augment other business systems to create higher-order emergent it-enabled business systems. emergent it-enabled business systems lead to transactional, informational and strategic benefits. the paper concludes by calling for empirical studies that focus on the complementary relations between transactional and informational it systems and their impact on business value. keywords: informational it systems; transactional it systems; systems theory; emergence; business value 1 introduction information systems researchers have sought to understand the business value of information technology (bvit) resources for more than two decades. this research has resulted in a diverse literature in terms of it characterisation, theories applied and the analytical and methodological approaches employed for empirical assessment. despite advancements in bvit research, particularly in conceptualising it resources and business value, further research on the bvit resources and their complementary interactions with other business resources is strongly encouraged (kohli 2008). in this paper, we address two important gaps in the bvit literature. first, current research on the relationship between it and business value examines the business impacts of one class of it (aral and weill 2007), in isolation from other classes of it. researchers have explored the business value of it infrastructure (e.g. computers, email and networks) (broadbent et al. 1999), transactional it resources (e.g. automated systems) (weill 1992), and informational resources (e.g. business intelligence and information management systems) (elbashir et al. 2008; mithas et al. 2011). however, there is insufficient understanding of the complementary interactions between different classes of it resources in generating it-based business value. particularly, the interactions between transactional and informational it resources still remain unexplored. transactional it resources standardise and automate organisational processes to cut costs and increase efficiencies (aral and weill 2007). informational it resources process organisational data and provide business processes and decision-makers with required information and insights (davenport and harris 2007). we 1 australasian journal of information systems someh & shanks 2016, vol 20, selected papers from the isf conference emergence of business value focus on these two classes of it resources and seek to understand how they can interact to develop higher-order it-enabled business resources. second, informational it resources (aral and weill 2007) such as business analytics (ba) systems and business intelligence (bi) systems have recently topped the list of technology priorities for cio’s (hagerty et al. 2012). informational it resources collect, store and analyse organisational data and provide organisational decision-makers with meaningful information and insights at multiple levels of an organisation (davenport and harris 2007; lavalle and lesser 2013). although several industry-based reports and academic research have provided evidence for the business value of informational it systems (davenport and harris 2007; davenport et al. 2010; isik et al. 2011; shanks and bekmamedova 2012; sharma et al. 2010; wixom and watson 2001; wixom et al. 2013), there is still little theoretical understanding of how these systems create business value and improve firm performance. therefore, we also seek to develop a theory that explains the complementary interactions between informational and transactional it resources as a mechanism for business value generation. based on these two research gaps, we ask the following research question: how do informational and transactional it resources complement each other in generating it-based business value? to answer this research question, we use systems theory and the concept of emergence to develop a research model that explains how informational and transactional it systems interact with each other to augment other business systems and develop emergent it-enabled business systems. the emergent it-enabled business systems include complementary interactions (or emergent properties) between transactional and informational it systems. the emergent properties include competitive actions of business people based on analytical insights. the competitive actions are both informed and effective and efficient. the emergent it-enabled business systems are capable of creating significant business value. we explore the research question using examples from customer relationship management (crm). crm involves managing customer processes and interactions in sales, marketing and services (buttle 2004). it assets and capabilities have a significant impact on the quality of interactions with customers (ray et al. 2005). particularly, informational it systems have been widely used in crm and have provided crm people and processes with insights from customer value analysis, customer behavior analysis and customer segmentation (goodhue et al. 2002). transactional it systems such as sales force systems have also standardised and automated many customer-facing processes (buttle 2004). therefore, crm is a suitable context to study the complementary interactions between informational and transactional it systems and the emergent it-enabled crm systems. our motivations for this paper are two-fold. first, research on bvit has shown mixed results for the relationship between it investments and business value (chae et al. 2014). despite these mixed results concerning business value, it has become an integral part of every business, and organisations are investing more on advanced information systems to operate, innovate and gain competitive advantage over their rivals (mithas et al. 2012). therefore, there is a clear need for further examination of the business value of it. second, industry reports and case studies have highlighted the significance of informational it systems such as ba systems to managers (chen et al. 2012; davenport and harris 2007). ba systems were identified as one of the four major technology trends in 2012 (ibm 2013). developing a useful theory that explains the interactions of informational it systems with other it systems such as transactional it systems in generating business value is an important direction of research. the paper is organised as follows. first, we discuss the theoretical background to the research and highlight important research themes that emerge from the bvit literature. second, we introduce systems theory and emergence as the main theoretical underpinning for understanding the complementary interactions between transactional and informational it systems and emergence of it-enabled business systems. third, we discuss emergent itenabled business systems including the complementary interactions between transactional and informational it systems. following that we present the research model, and provide 2 australasian journal of information systems someh & shanks 2016, vol 20, selected papers from the isf conference emergence of business value definitions for the constructs and hypotheses in the model. we then discuss the contributions to knowledge of the study, its implications for researchers and practitioners, limitations and future research and conclude the paper. 2 business value of it understanding the relationship between it and business value has long been of interest to information system researchers. early research was unable to demonstrate that investments in it provided business value, with some arguing that it is a commodity and not associated with strategic value (brynjolfsson 1993; carr 2003; lucas 1999). since then researchers have used rbv theory to examine the business value of it using the constructs of it investment, resources and capabilities. the rbv theory argues that sustainable competitive advantage is the result of organisational resources that are valuable, rare, difficult to imitate and nonsubstitutable (vrin properties) (barney 1991; penrose 1995). although rbv theory has been successful in linking it resources to business value, studies in this area still show mixed results for the direct relationship between it and business value. bharadwaj (2000) and santhanam & hartono (2003) reported a significant superior performance for organisations with betterquality overall it capabilities. however, chae et al. (2014) replicated the study and did not find support for the same hypotheses in following years, concluding the relationship may not exist. another body of literature examined the impact of overall it investment on firm performance. while aral and weill (2007) found no significant variations in firm performance attributed to overall it investment, a recent study reported a significant positive relationship between total it investment and firm profitability (mithas et al. 2012). we argue that establishing a direct link between overall it investments, capabilities and resources and business value has not provided convincing insights into how it influences organisational processes and consequently organisational performance. therefore, we identify four themes in the literature that help understand the mechanisms of it-based business value generation. 2.1 research themes from bvit literature four research themes that are important in conceptualising the it construct and examining it-based business value are: (1) it capabilities complement it assets in creating it-based business value; (2) it asset classes generate value consistent with their strategic goals; (3) it resources indirectly influence business value (via it-enabled business resources); and (4) business value from it resources is contingent on other organisational variables. the four themes are discussed below. research theme 1: it capabilities complement it assets in creating it business value it assets are the technologies that organisations invest in to enable their it based capability. they include hardware, software and data. it capabilities are interlocking systems of competencies (including it skills and it management) and practices (including it processes and routines and the culture of it use) (aral and weill 2007). it assets and capabilities together are associated with greater business value than conceptualising it as assets or capabilities separately (aral and weill 2007). it assets and capabilities together comprise the it resources in an organisation (aral and weill 2007). this is because organisational it capabilities complement and reinforce the impact of it assets on firm performance (aral and weill 2007). since investing in it and acquiring it assets has become a competitive necessity for organisations, it assets and investment are not likely to explain the variance in business value (ray et al. 2005). however, firm-specific and unique it capabilities developed over time and embedded in organisational processes, reinforce the impact of it assets on business value (bharadwaj 2000b). for example, human it resources complement technological it resources in creating business value (melville et al. 2004). the increased complementarity of combined it assets and capabilities is associated with increased business value per dollar investment (aral & weill 2007; melville et al. 2004). research stream 2: it asset classes generate value consistent with their strategic goal 3 australasian journal of information systems someh & shanks 2016, vol 20, selected papers from the isf conference emergence of business value different it asset classes generate business value consistent with the strategic goal of the asset (aral and weill 2007). it asset classes include infrastructure, transactional, informational and strategic (weill 1992). infrastructure assets are the shared it resources utilised across organisations such as servers, networks, laptops and databases. transactional it assets standardise and automate organisational operations to cut costs or increase efficiency for the same cost. examples of transactional it resources include operational crm resources and supply chain management (scm) resources. informational it assets provide information for purposes such as accounting, sales, compliance and communication. examples include decision support, bi and ba systems. these systems support effective decision-making and enable profitable operations. strategic it resources help organisations achieve competitive advantage by supporting entry into new markets or by helping to develop new products, services or business processes (such as mobile banking in the finance sector). each of these asset classes will contribute to business value consistent with the strategic goal of that class. for instance, transactional it investments contribute to reduced costs while investments in informational it assets influence profitability (aral and weill 2007). research stream 3: it resources indirectly influence business value the third research theme suggests that it resources positively and indirectly influence business value (bharadwaj 2000a; elbashir et al. 2008; melville et al. 2004; mithas et al. 2011; pang et al. 2014; pavlou and el sawy 2006; tanriverdi 2005). the indirect influence is through the intermediate contribution of it resources to other business resources. it resources are deployed to enhance the capabilities and business processes. it and other business capabilities together may be conceptualised as higher-order it-enabled business capabilities, which in turn influence firm performance (bharadwaj 2000). however, the first-order effect of it on business capabilities is also important. research stream 4: business value from it resources is contingent on other organisational variables the fourth research theme provides evidence that achieving value from it investments is contingent on other organisational variables (melville et al. 2004; ray et al. 2005; weill 1992). this theme is based on contingency theory which attempts to understand the interrelationships within and among organisational subsystems as well as between the organisation and its environment under varying conditions (kast and rosenzweig 1981; weill and olson 1989). the contingency approach suggests that a number of variables mediate or moderate the relationship between it and business value. for example, shared knowledge between it managers and line managers moderates the relationship between it and customer service performance (ray et al. 2005). the complementarity of other organisational capabilities (such as management quality and the culture of it use) to it investments is another contingency variable that will moderate the relationship between it and business value (melville et al. 2004; aral and weill 2007). nevo and wade (2011) argue that the business value of it assets is contingent upon their ability to form synergistic interactions with other organisational resources. we draw on these four research themes to conceptualise our understanding of the value of transactional and informational it resources implemented in customer processes. 3 systems theory and emergence systems theory deals with systems taken as a whole, rather than individual parts (ackoff 1971). a system is a composite thing comprising a number of parts (subsystems), which interact to accomplish a set of goals. the interactions among parts constitute a whole, which is greater than the sum of the individual parts. the whole system, derived from the synergistic interaction of the parts equals the sum of its parts plus their complementary interactions (ackoff 1971). however, it is very complex to model exactly how subsystems interact to constitute a whole. rules and laws can never be used to explain the evolution of a system, as there are a large number of options and associated decisions at each point in the system’s 4 australasian journal of information systems someh & shanks 2016, vol 20, selected papers from the isf conference emergence of business value evolution (corning 2002). however, the properties, patterns and structure that emerge from the interactions are an effective way of studying the whole. the new properties derived from the subsystem’s interactions are called emergent properties. the emergent properties of a collective system are the things that can be perceived and measured (corning 2002). emergent properties can be positive, neutral or negative according to organisational goals. positive emergent properties are called synergy (nevo and wade 2010). synergy between it and other organisational resources is realised when it and other organisational resources are compatible and integrated (nevo and wade 2010). compatibility refers to the degree to which systems fit with each other and is achieved when systems are able to seamlessly work together. compatibility between transactional and informational systems ensures that transactional systems (people and processes) are able to work with informational tools, functionalities and insights. conversely, the informational tools should match the decision needs of the transactional system, as different organisational resources may have different information needs (isik et al. 2011). integration effort refers to the effort of management to bring transactional and informational resources together and direct and support their interaction congruent with organizational goals (nevo and wade 2010). management is responsible for managing the changes and properly accommodating informational resources in relationships with transactional resources. an organisation, with respect to systems theory, can be conceptualised as a set of interconnected systems (kast and rosenzweig 1981). this contrasts with the rbv, which views an organisation as a bundle of resources (barney 1991). the use of systems theory helps to model the interaction among resources, which is not possible using rbv theory. a system from the systems theory perspective corresponds to a bundle of resources from rbv theory. therefore, a system in our study refers to combination of assets and capabilities (aral and weill 2007). 4 emergence of it-enabled business systems our argument is that transactional and informational it systems directly interact with other business systems and transform them to higher-order it-enabled business systems with emergent properties. while transactional it systems contribute to process efficiencies, informational it systems generate insights and contribute to informational benefits within the processes of other business systems. more importantly, the two it systems interact and complement each other when developing the higher-order it-enabled business systems. complementarity is the main source of synergy and the emergence of new properties within the business system (nevo and wade 2010; tanriverdi 2006). the emergent it-enabled business system supports informed and intelligent, as well as effective and efficient processes. figure 1 represents the properties of informational and transactional systems and emergent properties arising from their complementary interactions. transactional and informational it systems complement each other in several ways. the use of transactional it systems enables new it-based business opportunities for organisations including the implementation of informational it systems. the huge amount of data generated from transactional it systems has led to the emergence of advanced informational it systems to take advantage of the organisational data and provide insights for organisational processes. organisations collect extensive amounts of data using their transactional it systems about their customers, internal processes and suppliers. this data is managed, analysed and used for reporting purposes. although transactional it systems support limited analytics and reporting using operational data, advanced analysis of organisational data and the generation of insights require a different set of systems and skills. informational it systems offer sophisticated analysis, reporting and visualisation of organisational data. the advanced informational it systems would not exist without the transactional it systems and the data collected from organisational processes using these systems. 5 australasian journal of information systems someh & shanks 2016, vol 20, selected papers from the isf conference emergence of business value figure 1. complementary interactions between transactional and informational it systems informational it systems analyse organisational data and enhance organisational processes and their transactional systems by supplying meaningful information and insights to decisionmaking processes. the enhanced transactional systems enable organisations to take competitive actions leveraging insights from the informational system and promote fact-based decisions. transactional it systems reinforce the impact of informational systems by automating and embedding insights into their operational processes. insights from informational it systems would not be exploited and frequently used unless they were automated and embedded within transactional systems. transactional it systems help reengineer business processes and develop efficient and also intelligent processes. the new insights embedded in transactional systems will subsequently generate a new source of data for informational systems to analyse, provide feedback and propose new initiatives. hence, transactional and informational it systems are interdependent and complement and reinforce each other in practice. these complementary interactions between it systems can be a significant source of it-based business value. historically, transactional it systems were implemented in crm processes to reduce the costs of customer-facing processes in sales, marketing and service areas (buttle 2004). this was particularly evident in the trend towards call-centre and contact-centre operations where sales and service processes were re-engineered. process reengineering based on crm workflows meant that managers had better reporting and control (buttle 2004). transactional it systems generated large amounts of customer-related data. as companies began to build more complete customer-related databases, based on customer purchases, service enquiries and so informational it system • factual decisions • actions based on facts • real-time decisions • single version of the truth emergent it-enabled business system • effective, efficient and intelligent processes • embedded insights in processes • competitive actions based on facts transactional it system • process effectiveness • process efficiency • standardisation and automation • data collection 6 australasian journal of information systems someh & shanks 2016, vol 20, selected papers from the isf conference emergence of business value on, it became possible to overlay informational it systems. transactional it systems generated much of the data that analytical systems began to exploit for cross selling, up selling and churn prediction in particular (buttle 2004). analytical crm is enabled by the data generated by transactional it systems, together with other external data obtained from third parties. the insights from informational it systems improve decision-making in operational processes. aral and weill (2007) explain this complementary interaction of informational and transactional it systems using the 7-eleven japanese case study. 7-eleven used transactional it to process 35 million sales transactions and 5 million order transactions per day and extracted the relevant data for its informational systems. the data were integrated, analysed and injected back into operational processes. store workers leveraged the insights to take actions and increase sales and profitability. these actions were supported by both transactional and informational it systems and their complementary interactions. 5 proposed research model the research model, presented in figure 2, explains how the quality of transactional and informational it systems and their complementary interactions can generate business value. the locus of it value generation is a business system such as a crm system. a business system comprises business assets and capabilities. for example, a crm system refers to the combination of business people (e.g. marketers and sales people) and processes (e.g lead generation and campaign management). the goal of the transactional and informational it systems is to extend the capabilities of the business system. we use a systems approach (burton-jones et al. 2014) to develop the research model, proposing that the two it systems interact and complement each other to augment other business systems and develop the emergent it-enabled business system. informational and transactional it systems are associated with transactional and informational benefits, respectively. however, the emergent it-enabled business system is a higher-order business system with new properties and will influence strategic measures of business value. the definitions of constructs in the model are provided below and summarised in table 1. the hypotheses follow. figure 2. research model 5.1 construct definitions detailed definitions of the constructs in the research model are provided below. business value h1 h2 h3 emergent it-enabled business system quality transactional it system quality informational it system quality transactional informational strategic 7 australasian journal of information systems someh & shanks 2016, vol 20, selected papers from the isf conference emergence of business value 5.1.1 transactional it system quality transactional it system quality refers to the degree to which transactional it systems are successful in standardising and automating organisational processes (weill 1992). the organisational goal of transactional it systems is to reduce the cost of transacting with customers through achieving efficiency and effectiveness (aral and weill 2007). construct definition reference transactional it system quality the degree to which transactional it systems are effective in standardising and automating organisational processes. (buttle 2004; iriana and buttle 2006) informational it system quality the degree to which informational it systems are effective in storing, analysing, reporting and generating insights from organisational data. (davenport and harris 2007) emergent itenabled business system quality the degree to which a business people can effectively leverage analytical insights provided by informational it systems and embedded in the transactional it systems (buttle 2004; iriana and buttle 2006) business value business value that is generated from the use of emergent it-enabled business systems. business value has three dimensions including, transactional, informational and strategic benefits. transactional benefits include process efficiencies, effectiveness and cost reduction. informational benefits include fact-based decision making, real-time decisions, single version of the truth and actions based on facts. strategic benefits include time to market, increased revenue and superior customer experience. (aral and weill 2007; coltman 2007; wixom et al. 2013) table 1. definition of constructs in the research model transactional it systems include transactional assets and capabilities. transactional assets include standard workflow and automation tools, software packages from it vendors such as sap and in-house software technologies to design and develop automated systems. the transactional capabilities include the human it skills in business analysis, developing it applications and integrating them with existing applications. they include the ability of business analysts to identify inefficiencies and propose optimised solutions for organisational processes. they also include the ability of the transactional it staff to customise off-the-shelf vendor applications to address business needs. transactional assets and capabilities interact to standardise and automate customer-facing processes in marketing, sales and service (buttle 2004; iriana and buttle 2006). 5.1.2 informational it system quality informational it system quality refers to the degree to which it systems are successful in storing, analysing, reporting and interpreting high quality data (davenport and harris 2007). the organisational goal of informational it systems (aral and weill 2007) is to provide the right information to the right people at the right time (davenport and harris 2007). informational it systems are composed of assets and capabilities. informational assets are the technologies that organisations invest in to enable their information-based analytical capability. informational assets include data warehouses, extract transform & load (etl) tools, analytical software packages from it vendors such as ibm cognos, oracle’s obiee, data mining tools, online analytical processing (olap), digital dashboards, and high-quality data (shanks and bekmamedova 2012). informational capabilities include analytical human skills to analyse and generate insights from data (analytical professionals and data scientists), management quality in planning, implementing and measuring initiatives, analytical 8 australasian journal of information systems someh & shanks 2016, vol 20, selected papers from the isf conference emergence of business value processes and routines and analytical culture in the organisation (shanks and bekmamedova 2012). 5.1.3 emergent it-enabled business system quality emergent it-enabled business system quality refers to the degree to which the it-enabled business system is able to leverage analytical insights provided by the informational system and embedded in the transactional system. in the case of unstructured decisions, emergent itenabled business systems may directly use the insights from informational systems (without using transactional systems). the emergent it-enabled business system comprises informational it and transactional it systems and its quality depends on the quality of the emergent properties arising from the interaction between its components. 5.1.4 business value business value is generated from the use of emergent it-enabled business systems. emergent it-enabled business systems include informational it and transactional it systems and their complementary interactions. emergent it-enabled business systems will generate transactional, informational and strategic benefits. transactional benefits include process efficiencies, effectiveness and cost reduction (aral and weill 2007) and are generated as a result of using transactional it systems within emergent it-enabled business systems. informational it systems create informational benefits. informational benefits include factbased decision making, real-time decisions, single version of the truth and actions based on facts (wixom et al. 2013). strategic benefits include time to market, increased revenue and superior customer experience (coltman 2007; wixom et al. 2013). the strategic benefits are created from the emergent properties arising from the interaction between transactional it and informational it systems within other business systems. the business value is perceived at the process-level (melville et al. 2004). 5.2 hypotheses development (hypotheses 1-3) the hypotheses associated with the research model are explained below. transactional it systems standardise and automate organisational processes to achieve effectiveness and efficiency and reduce the cost of organisational processes (aral and weill 2007). based on the second research stream discussed in section 2.1 above, transactional it systems will generate benefits consistent with their organisational goals. thus, it is hypothesised that: hypothesis 1: transactional it system quality has a positive effect on the transactional dimension of business value. based on the second research stream discussed in section 2.1 above, informational it systems (aral and weill 2007), will create informational benefits (wixom et al. 2013) consistent with their organisational goals. therefore, it is hypothesised that: hypothesis 2: informational it system quality has a positive effect on the informational dimension of business value. the variance in business value from the use of transactional and informational systems is also explained by the quality of the emergent it-enabled business system. our third hypothesis is based on systems theory and the complementarity between systems as the main driver of business value from it. based on systems theory, the interaction between informational and transactional it implemented within a business system will give rise to emergent it-enabled business systems over time with emergent properties such as competitive actions based on analytical insights that are also embedded within processes. we explain this hypothesis using an example from a crm business system. the interaction of transactional and informational it systems within a crm business system will give rise to an emergent it-enable crm systems with emergent properties. the emergent properties are new and extended capabilities of crm people to take competitive actions based on insights provided by the informational system and embedded using the transactional it system. the 9 australasian journal of information systems someh & shanks 2016, vol 20, selected papers from the isf conference emergence of business value competitive actions include cross selling, up selling and targeted marketing based on the insights from customer value analysis, customer behaviour analysis and customer segmentation (goodhue et al. 2002). the emergent it-enabled crm system uses the transactional it system to collect customer data over time from many different touch points. the informational system analyses the data with different analytical tools and proposes a variety of tailored services, which can be automated using the transactional it system and then frequently used. this use of it systems has transformed the role of crm from an operational area to a strategic area for organisations to innovate, compete and gain competitive advantage (goodhue et al. 2002). examining the value of transactional and informational it systems in isolation does not capture the complementary interactions or the emergent properties. the emergent it-enabled business system includes the complementary interactions between its components. therefore, it is capable of generating business value in addition to transactional and informational value. as a result, it is hypothesised that: hypothesis 3: emergent it-enabled business system quality has a positive effect on the strategic dimensions of business value. 6 discussion the theoretical model developed in this paper explains how transactional and informational it systems interact to develop higher-order emergent it-enabled business systems. emergent it-enabled business systems are enabled by the two it systems and capture the complementary interactions between transactional it and informational it. 6.1 contributions to research this conceptualisation has six implications for the bvit literature. first, organisations invest in different classes of it system to achieve different goals. a successful implementation of one class of it system encourages and enables organisations to invest in other classes of it systems. furthermore, technological advancements and competitive market conditions force managers to invest more in it, rather than lagging behind the competition. this leads to the implementation of multiple it asset classes in a single business unit. the bvit literature to date has tended to examine the business value generated by one class of it system. this underestimates the business value created from the interaction between different classes of it system. in this paper, we conceptualise the interactions and complementary relations between two different classes of it system and propose that the two systems together can create greater business value. this is consistent with the current competitive business environment, in which organisations invest in multiple it systems. second, the relationship between it and business value is indirect, through the mediating role of organisational processes. transactional and informational it systems generate business value indirectly by enabling and augmenting business unit capabilities and processes. while transactional systems are implemented in a business system to automate the processes, informational systems are implemented to informate the same processes (zuboff, 1988). transactional and informational systems together support more intelligent and efficient business processes. we conceptualise the indirect path using a systems approach (burtonjones et al. 2014). we argue that transactional, informational and business systems together comprise a new higher-level system, in which they will complement and reinforce each other in accomplishing organisational tasks. the business unit capabilities enabled by these two interacting systems will influence the strategic dimensions of business value. the systems approach extends the literature on testing complementary relations. the current literature on complementarity measures the concepts in isolation and brings them together as a statistical interaction effect (aral and weill 2007; titah and barki 2009). in contrast, using the systems approach, we conceptualise the interactions among the systems and their emergent properties, which may not be captured using statistical approaches. this extends the rbv and complementarity (aral and weill 2007) literature as well as enriching the fit concept (venkatraman 1989) and the theory of task-technology fit (goodhue and thompson 1995). 10 australasian journal of information systems someh & shanks 2016, vol 20, selected papers from the isf conference emergence of business value third, we argue that multiple it systems interact with other business systems and higher-level emergent it-enabled business systems emerge from their interaction. based on systems theory, the evolution of systems and the emergence of new systems is a socially complex process. therefore, emergent it-enabled business systems satisfy the vrin properties of rbv theory in terms of having path dependencies, causal ambiguity and a complex social structure (bharadwaj 2000b). therefore, emergent it-enabled business systems are capable of creating significant business value and contributing to competitive advantage. on the other hand, each system in our research model includes assets as well as capabilities. this signifies that technological assets alone are not a source of value. however, complementarity and synergy between it capabilities and it assets are important in creating value. fourth, multilevel approaches to theory and research seek causal explanations for emergence in organisational contexts. we use complementarity as the underlying phenomenon that explains the emergence of higher-order it-enabled systems. the positive emergent properties are a source of significant benefits for organisations. fifth, our research model also helps to theorise about the business value generated from informational it systems (such as ba systems). in recent years, several theoretical models have been proposed to explain how value is created from ba systems (davenport and harris 2007; davenport et al. 2010; elbashir et al. 2008; isik et al. 2011; shanks and bekmamedova 2012; sharma et al. 2010; wixom and watson 2001; wixom et al. 2013). none of these models considers the interaction of ba systems and other organizational systems. our study describes how informational it systems can interact with other organizational systems to generate business value. this is important because informational it systems are strategic investments for many organisations. sixth, within rbv, organisations are conceptualised as collections of resources (barney 1991). although rbv theory has successfully helped researchers to establish a link between it resources and business value, it does not model the interactions among resources. hence, the interactions among resources have been addressed in the bvit literature only to a limited extent. the use of other theoretical lenses such as systems theory in addition to rbv theory can help mitigate this problem. 6.2 contributions to practice for practitioners, the research model provides a means of understanding the importance of complementary relations between different it systems and the emergence of new it-enabled business systems. this understanding will effect manager's it investment decisions in choosing systems that will complement their current it systems. it also signifies the importance of bringing the systems together and facilitating complementary interactions among them to embed it in business processes, generate business value and gain competitive advantage. the research model helps to conceptualise the business value of informational it systems, which is important in justifying the huge investments in these systems. 6.3 future research directions we have proposed a model that explains how two classes of it system can together create itbased business value for organisations. our conceptualisation implies that having two separate high-quality systems is associated with limited benefits and is not necessarily a driver of business value. however, how the two systems work together, complement each other and have synergy, is critical in leveraging the it systems and creating it-based business value (asadi someh and shanks 2013a, 2013b). understanding the underlying mechanisms that realise synergy between it and other business systems is an important research direction. also we have proposed three testable research hypotheses for benefits from transactional it and informational it systems in isolation and together within emergent it-enabled business systems. empirical confirmation and refinement of the research model is an important research direction to follow. we will design and conduct a survey to collect data in the crm area to test the three hypotheses. since there is a time lag between implementing it systems 11 australasian journal of information systems someh & shanks 2016, vol 20, selected papers from the isf conference emergence of business value and achieving the benefits at firm level, a longitudinal study is another avenue for future research. 7 conclusion the objective of our study was to examine how two classes of it systems, namely transactional and informational systems, interact to augment business processes and create business value. we used systems theory and the concept of emergence to propose a new research model. the model argues that transactional and informational it systems individually create transactional and informational benefits. emergent it-enabled business systems capturing the complementary interactions between transactional it and informational it will create strategic benefits in addition to the benefits created by the individual systems. further empirical assessment of the research model is encouraged for future research. references ackoff, r. l. 1971. “towards a system of systems concepts,” management science (17:11), pp. 661–671. aral, s., and weill, p. 2007. “it assets, organizational capabilities, and firm performance: how resource allocations and organizational differences explain performance variation,” organization science (18:5), pp. 763–780. asadi someh, i., and shanks, g. 2013a. “the role of synergy in achieving value from business analytics systems,” in international conference on information systems (icis), milan, italy. asadi someh, i., and shanks, g. 2013b. “realising synergy in business analytics enabled systems,” in 24th australasian conference on information systems, melbourne, p. paper 251. barney, j. 1991. “firm resources and sustained competitive advantage,” journal of management (17:1), p. 99. bharadwaj, a. 2000a. “a resource-based perspective on information technology capability and firm performance: an empirical investigation,” mis quarterly (24:1), pp. 169–196. bharadwaj, a. 2000b. “a resource-based perspective on information technology capability and firm performance: an empirical investigation,” mis quarterly (24:1), p. 169 (doi: 10.2307/3250983). broadbent, m., weill, p., and neo, b. 1999. “strategic context and patterns of it infrastructure capability,” the journal of strategic information … (8 december 1996), pp. 157–187. brynjolfsson, e. 1993. “the productivity paradox of information technology,” communications of the acm (36:12), pp. 66–77. burton-jones, a., mclean, e. r., and monod, e. 2014. “theoretical perspectives in is research : from variance and process to conceptual latitude and conceptual fit,” european journal of information systems, pp. 1–16. buttle, f. 2004. customer relationship management : concepts and tools, oxford: elsevier butterworth-heinemann. carr, n. g. 2003. “it doesn’t matter,” harvard business review (81:5), new york, pp. 41–49, 128. chae, h., koh, c., and prybutok, v. 2014. “information technology capability and firm performance: contradictory findings and their possible causes,” mis quarterly (38:1), pp. 305–326. chen, h., chiang, r., and storey, v. 2012. “business intelligence and analytics: from big data to big impact,” mis quarterly (36:4), pp. 1165–1188. 12 australasian journal of information systems someh & shanks 2016, vol 20, selected papers from the isf conference emergence of business value coltman, t. 2007. “why build a customer relationship management capability?,” the journal of strategic information systems (16:3), pp. 301–320. corning, p. a. 2002. “the re-emergence of emergence: a venerable concept in search of a theory,” complexity (7:6), pp. 18–30. davenport, t. h., and harris, j. g. 2007. competing on analytics: the new science of winning, harvard business school press. davenport, t. h., harris, j. g., and morison, r. 2010. analytics at work: smarter decisions, better results, cambridge, ma: harvard business school press. elbashir, m. z., collier, p. a., and davern, m. j. 2008. “measuring the effects of business intelligence systems: the relationship between business process and organizational performance,” international journal of accounting information systems (9:3), pp. 135–153. goodhue, d. l., and thompson, r. l. 1995. “task-technology fit and individual performance,” mis quarterly (19:2), p. 213 (doi: 10.2307/249689). goodhue, d., wixom, b., and watson, h. 2002. “realizing business benefits through crm: hitting the right target in the right way,” mis quarterly executive (1:2). hagerty, j., sallam, r. l., and richardson, j. 2012. “magic quadrant for business intelligence platforms,” gartner document. ibm. 2013. “the 2012 ibm tech trends report,” (available at https://www.ibm.com/developerworks/community/blogs/techtrends/entry/byindustr y?lang=en). iriana, r., and buttle, f. 2006. “strategic, operational, and analytical customer relationship management: attributes and measures,” journal of relationship marketing (5:4). isik, o., jones, m., and sidorova, a. 2011. “business intelligence (bi) success and the role of bi capabilities,” intelligent systems in accounting, finance and management (176), pp. 161–176. kast, f. e., and rosenzweig, j. e. 1981. “general systems theory: applications for organization and management,” academy of management journal (11:7), pp. 32–41. kohli, r. 2008. “business value of it: an essay on expanding research directions to keep up with the times,” journal of the association for information (9:1), pp. 23–39. lavalle, s., and lesser, e. 2013. “big data, analytics and the path from insights to value,” mit sloan management review. lucas, h. 1999. information technology and the productivity paradox: assessing the value of investing in it (google ebook), oxford university press. melville, n., kraemer, k., and gurbaxani, v. 2004. “review: information technology and organizational performance: an integrative model of it business value,” mis quarterly (28:2), pp. 283–322. mithas, s., ramasubbu, n., and sambamurthy, v. 2011. “how information management capability influences firm performance,” mis quarterly (35:1), pp. 237–256. mithas, s., tafti, a., bardhan, i., and goh, j. m. 2012. “information technology and firm profitability: mechanisms and empirical evidence,” mis quarterly (36:1), pp. 205–224. nevo, s., and wade, m. 2010. “the formation and value of it-enabled resources: antecedents and consequences of synergistic relationships,” mis quarterly (34:1), pp. 163–183. nevo, s., and wade, m. 2011. “firm-level benefits of it-enabled resources: a conceptual extension and an empirical assessment,” the journal of strategic information systems (20:4), pp. 403–418. 13 australasian journal of information systems someh & shanks 2016, vol 20, selected papers from the isf conference emergence of business value pang, m.-s., lee, g., and delone, w. h. 2014. “in public sector organisations: a public-value management perspective,” journal of information technology (29:3), pp. 187–205. pavlou, p. a., and el sawy, o. a. 2006. “from it leveraging competence to competitive advantage in turbulent environments: the case of new product development,” information systems research (17:3), pp. 198–227. penrose, e. t. 1995. the theory of the growth of the firm, new york: oxford university press. ray, g., muhanna, w., and barney, j. 2005. “information technology and the performance of the customer service process: a resource-based analysis,” mis quarterly (29:4), pp. 625–652 (available at http://www.jstor.org/stable/25148703). santhanam, r., and hartono, e. 2003. “issues in linking information technology capability to firm performance,” mis quarterly (27:1), pp. 125–153 (available at http://www.jstor.org/stable/30036521). shanks, g., and bekmamedova, n. 2012. “achieving benefits with business analytics systems: an evolutionary process perspective,” journal of decision systems (3:november), pp. 37–41. sharma, r., reynolds, p., scheepers, r., seddon, p. b., and shanks, g. 2010. “business analytics and competitive advantage: a review and a research agenda,” in bridging the socio-technical gap in decision support systems: challenges for the next decade, , august 1, pp. 187–198. tanriverdi, h. 2005. “information technology relatedness, knowledge management capability, and performance of multibusiness firms,” mis quarterly (29:2), pp. 311– 334. tanriverdi, h. 2006. “performance effects of information technology synergies in multibusiness firms,” mis quarterly (30:1), pp. 57–77. titah, r., and barki, h. 2009. “nonlinearities between attitude and subjective norms in information technology acceptance: a negative synergy?,” mis quarterly (33:4), pp. 827–844. venkatraman, n. 1989. “the concept of fit in strategy research: toward verbal and statistical correspondence,” academy of management review (14:3), pp. 423–444. weill, p. 1992. “the relationship between investment in information technology and firm performance: a study of the valve manufacturing sector,” information systems research (239). weill, p., and olson, m. 1989. “an assessment of the contingency theory of management information systems,” journal of management information systems (6:1), pp. 59–85. wixom, b., and watson, h. 2001. “an empirical investigation of the factors affecting data warehousing success,” mis quarterly (25:1), pp. 17–41. wixom, b., yen, b., and relich, m. 2013. “maximizing value from business analytics,” mis quarterly executive (2013: june), pp. 61–71. zuboff s. 1988. in the age of the smart machine: the future of work and power. basic books, new york. copyright: © 2016 asadi someh, shanks. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. 14 http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems someh & shanks 2016, vol 20, selected papers from the isf conference emergence of business value 15 1 introduction 2 business value of it 2.1 research themes from bvit literature 3 systems theory and emergence 4 emergence of it-enabled business systems 5 proposed research model 5.1 construct definitions 5.1.1 transactional it system quality 5.1.2 informational it system quality 5.1.3 emergent it-enabled business system quality 5.1.4 business value 5.2 hypotheses development (hypotheses 1-3) 6 discussion 6.1 contributions to research 6.2 contributions to practice 6.3 future research directions 7 conclusion microsoft word ajis vol 19.docx australasian journal of information systems burmeister 2015, vol. 19, pp. 55-70 professional doctorate completion rates 55 improving professional it doctorate completion rates oliver k. burmeister school of computing and mathematics charles sturt university, bathurst, australia oburmeister@csu.edu.au abstract professional doctorates in information technology (it) have been a relatively recent phenomenon, giving it professionals career management choices not previously available to them. however, successful completion rates are the lowest of all disciplines. completed doctorates rate in quality equivalent to phds, and retention has been identified as a major obstacle to completion. this qualitative study, involving 44 semi-structured interviews with students, supervisors and institutional support personnel, investigated the obstacles. amongst the strategies discovered to improve completion rates were retention, student engagement with supervisors, feedback on progress, student engagement in the course, and student involvement in institutional communities of practice. keywords: career management, community of practice, postgraduate education, retention. 1 introduction it is disappointing that information technology (it) doctorates have the worst completion rates of all professional doctorates. that makes the choice to invest significant money and time into career advancement through postgraduate study an even more difficult choice for professionals in it, than it does for those in education, business and other disciplines. although there exists an extensive literature on the professional doctorate and postgraduate education relating to doctorates, it is predominantly in education and on full-time research. comparatively little research has been conducted into part-time professional doctorates in it. the present study aimed to improve completion rates in it doctorates. it examined multiple contributing factors, including support structures, curriculum design, supervisory capacity and student perspectives, by drawing on 44 interpretive interviews with students, supervisors and support staff. it makes recommendations about better student recruitment, more integrated preparation for the research component, ways to better resource students and supervisory teams, and draws on lessons in other disciplines to recommend how to reduce attrition and improve successful completion. career management literature, including career construction theory (guan et al., 2014) and subjective career success (ng & feldman, 2014), have demonstrated that in the modern world people are readily changing jobs to achieve career aspirations and many are willing to pursue further training, including advanced tertiary education. the latter is because it is seen as an “entry into many high-paying and high-status occupations” (ng & feldman, 2014, p. 172). recent literature on it doctorates has called for the development of doctorates that are not focused on pure research (phd), but which address the needs of practicing professionals, for example, one that argued that no such program existed in the state of georgia and proposed such a program (hoganson & amoroso, 2011). however, perhaps because of the recency of such it doctorates, unlike other disciplines, where curriculum design is discussed, the approach is rarely evaluated (hoganson & amoroso, 2011; merritt et al., 2004). a further contribution of this article is that it describes the curriculum design and evaluates the efficacy of the program. there are different types of doctoral programs, including not only the doctor of philosophy (phd), but also various full-time and part-time professional doctorates, such as the doctor of business administration, and the doctor of information technology (dit). a dit for this study is a part-time doctorate typically undertaken by mid-career professionals from various it backgrounds, including computer science and information systems, who seek to move into australasian journal of information systems burmeister 2015, vol. 19, pp. 55-70 professional doctorate completion rates 56 more managerial roles within their respective organisations. the article begins with a review of professional doctorates. then the methodology for the study is detailed, including the course design. that is followed by a discussion of the results, in which implications are drawn in relation to policy, managing doctoral it courses, vocational choices, supervisory training, and resourcing issues. 2 doctoral education doctoral studies focus on the phd (metcalfe & kiley, 2000), and on supervision practices, as opposed to student perspectives (fink, 2006; lee & mckenzie, 2011), although professional doctorates have been the subject of research for the past two decades (fink, 2006; lee, brennan, & green, 2009). as one might expect, the self-reflective nature of educational research is seen in that the dominant research into doctorates, both phd and professional, has taken place in education-related disciplines (green, 2012; lee et al., 2009; lee, green, & brennan, 2000). it doctorates are poorly represented in the literature (hirschheim & klein, 2012), often subsumed under science doctorates. numerous initiatives have taken place, particularly in europe, to define what constitutes a doctorate. tracing european developments, lee (2009) referred to the 1999 bologna declaration, and examined the 2004 lisbon agenda, the 2005 bergen communiqué, and other doctoral reviews, which the european union saw as a way of streamlining the lengthy phd process to “last 3-4 years and [because] they wanted programmes to meet the wider needs of the employment market” (lee, 2009, p.2). 2.1 completion rates and attrition risks in doctoral programs completion rates are closely associated with attrition risks. a concerted effort at the graduate school of arts and sciences, university of texas, lifted doctoral completion rates from 34% to 70% over a 10 year period to 2004 (smallwood, 2004). a twelve-year study of completion rates, involving 30 institutions, revealed that doctoral programs in mathematics and physical sciences (including computer science) had the second lowest completion rates, of 55.5%. of all the discipline-specific categories investigated, computer science had the lowest completion rate, of 41% (sowell, 2008). australian data shows that at monash university across all disciplines, completion rates were approximately 40% in 1994-1996, but that by focusing on improving this, their faculty of arts improved completion to above 70% by 2008 (cuthbert, 2008). poor completion rates in it are of particular concern for developing nations, because the development of it infrastructure is often seen as vital in the effort to keep up with more developed nations. for example, one south african study revealed that from 2000-2007 there were 1,039 doctoral graduates, with 26 phd graduates per million people (kritzinger & loock, 2012, p. 2). in 2007 “7% were from engineering and technology” (kritzinger & loock, 2012, p. 3). focusing on the university of south africa’s (unisa) school of computing, they found that “few students graduate” (kritzinger & loock, 2012, p. 11), with no graduates in 2007 and 2010, and that for those who did, the average time to completion was 4.2 years. 2010 data revealed that in information systems the doctoral dropout rate was 50% in the first year and 66.7% for computer science, averaging 55%. they concluded that “the dropout rates of master’s and doctoral students is currently the most vulnerable aspect contributing to the low postgraduate success rate.” (kritzinger & loock, 2012, p. 15) that finding is mirrored by an earlier study in the usa, which experienced a 70% dropout rate, but for the students who graduated, the quality of their work was of a high standard (merritt et al., 2004). this suggests that more effort needs to focus on retaining doctoral it students within a program, that is, that in order to increase successful completion, efforts need to focus on retention. the role of supervisors in student progression has been extensively explored, particularly in relation to phds and within that on supervisory capacity (engebretson et al., 2008; mccallin & nayar, 2012). however, literature which addresses student progression in doctoral programs is less common (cumming & kiley, 2011, p. 14). literature on supervisory capacity includes australasian journal of information systems burmeister 2015, vol. 19, pp. 55-70 professional doctorate completion rates 57 recommendations to reduce supervisory workload requirements through doctoral coursework, which is the dominant model for professional doctorates, but less common for phds (alpert & kamins, 2004; kiley, 2012); to increase training in supervision (amundsena & mcalpine, 2009; kiley, 2012; mcalpine & amundsen, 2012), particularly for people new to academia (blass, jasman, & levy, 2012; christine halse, 2011; c. halse & malfroy, 2010), which is the case for industry co-supervisors; to better cater to the growing demand for off-campus study and therefore for distance supervision (andrew, 2012; crossouard, 2008; crossouard & pryor, 2009; erichsen, bolliger, & halupa, 2012); and, group supervision (berteau & villeneuve, 2006; fenge, 2012). a topic insufficiently addressed in the literature is the workload demands and capacity to supervise of industry supervisors. these are typically senior people in industry who volunteer to become, typically unpaid, secondary supervisors (pavlova & silbereisen, 2014). volunteering can show leadership in one’s discipline and is also a way of giving back, of making a contribution to the discipline and to the next generation of professionals therein. 2.2 communities of practice traditional pedagogical research concerning workplace learning is focused on training undergraduate students for the workplace. however, professional doctorates are typically undertaken by people in mid-career and therefore the nature of the education-workplace mixture does not fit with those traditional pedagogies. the closest conceptual framework is the concept of communities of practice (cop). cop is a synthesis of pedagogies and epistemological foundations which facilitates the encompassing of research education into professional practice (kimmerle et al., 2013; loftus, 2010). that is, whereas workplace learning, practicebased education and related traditional formulations have sought to involve relatively ‘uninitiated’ students in professional practice during their education, professional doctorates seek to bring research related education into the professional practice of experienced practitioners. for instance, one study reported that for professional doctorates “entering students’ careers extend from five to 33 years, with an average among students of 13 years” (grossman, tappert, bergin, & merritt, 2011, p. 135). thus professional doctorates are typically undertaken by mature students, frequently in middle to senior roles, who work demanding jobs, whilst engaged in their studies. this distinction is important and has long been recognised, as illustrated by schön’s (1987) claim that beginners in any profession cannot fully appreciate what the work involves until they have been completely immersed in the work routines for some time. that is, there exists a fundamental difference in the nature of professional practice, when engaged in by a student on work placement who has little professional experience, and a person who is engaged in professional development activities of an educational nature, who has extensive industry experience. following his review of the cop literature, loftus defined it as follows (loftus, 2010, pp. 41-42): communities of practice are groups of people who share an interest in something that they do. a key insight is that people learn how to do their practice better as they interact regularly. as one recent study reported, the types of skill training required in information systems at an undergraduate level is more technically oriented, whereas at a postgraduate level, soft skills are required (richards & marrone, 2014). in another study barnett and coate (2005) argued that the aim of educational experiences was to assist students to become professional practitioners. however professional doctorates are undertaken by people who are already professional practitioners and but who are not yet researchers. in summary, doctorates in it have the lowest completion rates, and little research has been published on professional, part-time it doctorates. unlike current conceptual frameworks which focus on inexperienced graduates, cop offers a pedagogical framework that better addresses the workplace experiences of mid-career professionals undertaking doctoral studies. 3 the dit this section puts the doctor of information technology (dit) into its broader context. it begins with a review of it doctorates. from there the australian context of similar courses is sketched. australasian journal of information systems burmeister 2015, vol. 19, pp. 55-70 professional doctorate completion rates 58 the dit has evolved since its inception less than two decades ago. one innovative design is a doctor of science, which involved an “instruction-delivery distance learning approach” (hoganson & amoroso, 2011, p. 311). another example is that of pace university, whose course design is based on six agile principles (grossman et al., 2011, p. 139). still another program requires weekend face-to-face meetings, and has distance interaction between such weekends, following a more traditional doctoral education model of full-time, on-campus students, but allowing for the fact that these students are professionals who work full-time (merritt et al., 2004). the latter course is one of the earliest in it, having begun in 1999, and is one of the few where there have been attempts to evaluate its efficacy. the academic quality of the dissertations was evaluated by objective reviewers who were tasked with evaluating “the meaningfulness of the problem, the methodological strength of the investigation, and the readability of the presentation” (merritt et al., 2004, p. 44). that study reported results for the 2001 cohort, who graduated in 2004. although their stated aim was 100% retention, of the 20 students who started, only six graduated. the objective reviewers, on a rating scale to a maximum of 4.0, rated those six as 3.4 for significance of the problem, 3.3 for strength of the investigative methodology and 3.5 for readability (merritt et al., 2004, p. 45). that study was conducted 5 years into their new program, with a later study conducted in the 11th year revealing that over the period of those 11 years completion rates had averaged 52% (grossman et al., 2011). charles sturt university (csu), which for several years has been the national leader in parttime, post-graduate it education, saw the dit as a way to build on that leadership. for example, the deewr ucube statistics (http://highereducationstatistics.education.gov.au/) revealed that of the 3,821 part-time, post-graduate it 2013 enrolments across australia, not only was csu the leader with 566 new enrolments, but its closest rival, the university of technology, sydney, had 372, demonstrating that csu is not just leading by a small margin. of the 39 australian universities, only four others offer a professional doctorate in it: queensland university of technology, university of new south wales (unsw canberra at adfa), and edith cowan university, and murdoch university. three others have been discontinued: monash university, university of south australia, and federation university australia. the unsw course is only offered to full-time domestic students; the other three universities require international students to be full-time on campus, but allow domestic students a part-time option. murdoch university does not offer all their coursework subjects in distance mode, further demonstrating the university preference for full-time, on-campus study. 4 the context of this study: the dit structure at csu although the csu dit has existed for over a decade, it recently underwent a significant redevelopment. the fees, course structure, coursework to research components and all other aspects of the dit have been considered and redesigned, to make it competitive when compared to other offerings within australia. wide industry consultation took place, to ensure the relevance of the applied nature of the course. as illustrated in table 1, the new post-masters dit is not a single award. as with the course at pace university (grossman et al., 2011, p. 139) mentioned above, the dit also follows an agile design philosophy, with four of those principles reflected in table 1. firstly, early and continuous delivery of valuable product, reflected in that students can exit in three stages. also within each of the first two years of coursework students are guided to develop an article and submit it to a conference or journal. secondly, welcoming changed requirements, even late in development, reflected in that few students complete the course in the precise area of study that drew them to enroll in the first place. instead, as the course progresses, their understanding and interests change, as do available resources, access to data and workplace and life circumstances. thirdly, delivering working product frequently, which is reflected in the paper development each year. each paper spans the coursework subjects of that year, with progressive iterations of the paper, helping students through writing to explore research and their chosen topic. finally, measuring progress primarily through useful deliverables, is also australasian journal of information systems burmeister 2015, vol. 19, pp. 55-70 professional doctorate completion rates 59 seen in the coursework component, not only through the papers, but in that a credit average is required to progress from first to second year and from second year to the dissertation stage. therefore, aside from the student choosing to take out a lesser award than the doctorate, if they successfully complete a year but fail to achieve a credit average, then they have to exit with the lesser award. subjects award study load itc711 research issues in it graduate certificate in computing research year 1 itc712 conducting a research project in it itc713 research project plan in it master of computing research year 2 itc714 research project development in it itc715 thesis (full-time) or itc716 thesis (parttime) doctor of information technology years 3 to 6 table 1: structure of the doctor of information technology course (2012 onwards) of the 36 applications received for the intake in the first session of 2012, 28 were domestic and 8 international. 28 offers were made (24 domestic, 4 international), resulting in 22 acceptances (20 domestic, 2 international), with 4 of those deferring. of the 18 students who started in the first session of 2012, 5 students withdrew and 2 failed the first subject; both students who failed chose not to re-enroll, but withdrew from the course. that left 11 students who progressed to the second subject, itc712. three more withdrew before the end of the first year, and two exited (with lesser awards) before entering the dissertation phase. in 2014 the remaining six all entered their thesis component; one of those six transferred into a phd, due to changing career aspirations during the coursework stages, which resulted in a desire to become an academic. it should be noted that although all dit students are part-time, it is possible to do the thesis component in full-time mode; for instance, one student works for a university (not csu) and will be full-time for six months during their thesis component, whilst on sabbatical from that university. candidate selection is a critical factor to successful completion (cantwell, scevak, bourke, & holbrook, 2012; kiley, 2011). an assessment has to be made about the ability of the student to successfully complete. the above figures show that of the 36 initial applicants, 28 offers were made and 10 failed to complete the first year, signaling that the initial screening process was inadequate. screening of applicants has improved significantly from that first intake. from the second 2012 intake to the second intake 2014, the enrolments have been 8, 5, 3,14,7, and the attrition (withdrawal or failure) has been 2,1,0,4, and 1 respectively. however, attrition and successful completion need to be distinguished. that is, few students now leave in the first year. however, because they can exit at 2 points prior to achieving their doctorates, those that do, have not attained the doctorate, but neither do they fit in traditional notions of ‘attrition’, in that they leave with a lesser award. thus attrition in the csu dit only relates to those who leave in the first year of their course. in other words, when comparing the dit to a course such as the unisa or pace university ones cited above, although our attrition rates are significantly lower than theirs, it does not necessarily mean that our success at doctoral completions will be better. each dit student is appointed an industry co-supervisor. this serves two functions: firstly it provides a strong industry focus, by someone who has extensive, practical experience in the discipline area; secondly, it is a strategy to increase supervisory capacity. by restricting principal supervision to academics and co-supervision to industry experts, who have mentoring and supervisory expertise, but are not required to have a phd, the strong industry link sought by students is provided, whilst still ensuring that the research aims of the award are achieved. in addition to the immediate supervisors, the wider institutional supports effectively extend the supervisory team, and help to achieve a cop that combines industry and research in an effective strategy. australasian journal of information systems burmeister 2015, vol. 19, pp. 55-70 professional doctorate completion rates 60 5 methodology the project followed an interpretivist, constructivist approach which sought to improve completion rates in the dit specifically, as well as in it doctoral research generally. the research questions were: how can successful student progression be facilitated? which strategies and resources are required for successful student progression? the project took place in the second half of 2012, when the focus was solely on the dit, and in the first half of 2013, when the focus widened (as part of the original design of the study) to include other it doctoral students and wider institutional doctoral support personnel. csu ethics approval was sought and granted for each of the two phases of the research. 5.1 sampling interviewees were chosen on the basis of criterion sampling (patton, 2002), which employs a particular set of criteria to guide participant selection. for example, supervisors included people with experience as principal supervisors, co-supervisors, supervisors who were supervising for the first time, people who had not yet had a student successfully complete their doctorate under that person’s supervision, and people who had supervised at least one student to successful completion of their doctorate; all supervisors needed to be the principal or cosupervisor of at least one dit student. the criterion for dit students included students enrolled under the old dit structure, those enrolled under the new 2012 structure, graduates of the dit, students who had discontinued under the old structure, students who had discontinued under the new structure, and students who had deferred their dit studies. the 44 semi-structured interviews included 13 supervisors, eight dit and six it phd students, and 17 support staff, representing every aspect of institutional support personnel for doctoral students, including disability support, library support, student services (such as for students from non-english speaking backgrounds), scholarship support, sub-deans for graduate studies, research office (ro) staff and dit administrative officers (such as for enrolment and deferral procedures). each interview was from 45 to 120 minutes duration, and used the same set of questions, although a few relating to supervisory experiences were adapted for interviews with support staff who did not have experience of supervision. 5.2 data analysis all the interviews were tape recorded and transcribed verbatim. the transcribed interviews were analysed using thematic (qualitative) analyses. the data analysis from both phases followed closely a process described by morse (2008) who advocated that the researcher should start by identifying categories from the data, with themes developed later to tie the identified categories together. data analysis was completed with the help of qsr nvivo 10, a software package for managing qualitative data. the unit of analysis was each individual interview document. a total of 32 categories in five themes emerged. the following section draws on these findings to discuss the results, drawing on exemplary quotations to show supporting evidence from the data. 6 results and discussion improvements and impediments to completion rates can be seen in the following key areas. the first of these concerns retention. 6.1 retention as seen in the literature, students who graduate typically do well, but it is retention that is one of the largest contributors to poor completion rates (kritzinger & loock, 2012; merritt et al., 2004). two retention issues were seen as significant in this study. firstly, students need help in achieving a work-life-study balance. dit students are typically high achievers in the workplace, used to juggling multiple projects. they come into the dit expecting it to just be australasian journal of information systems burmeister 2015, vol. 19, pp. 55-70 professional doctorate completion rates 61 another project to manage. however the demands of the course and the nature of research, as well as the length of involvement in the dit, make it very different from the typical projects that they are used to dealing with. first-year dit students voiced concerns about inadequate ability to fit the requirements of the dit into their busy work and family situations. one student with two young children said that she got into the habit of rising at 4 am, to have time to focus on the dit. another student stated that he has 4 hours per week of study leave from his employer, which on the one hand is wholly inadequate to do what the dit demanded of him, and on the other hand was frequently infringed upon by his employer, as work deadlines required all employees to contribute/be present. related to this is the inadequacy of formal session deadlines in the coursework structure. a flexible approach might permit the extension of assessment commitments, to better accommodate the busy and pressured working lives of these students. one student withdrew from a subject in 2012 because unexpected work demanded his attention. had he been able to delay submission of assessment tasks, he could have continued. thus one area of focus for retention needs to address the work-life-study balance. the agile course design, first seen in the pace university course (grossman et al., 2011), addresses another area of retention, namely the need to teach research skills by engaging students through early and continuous delivery of valuable products. supervisors believed that it is important for student progression that they publish articles based upon their work to date. for instance, a co-supervisor who had supervised one dit to completion and was supervising a second dit student: the supervisor relationship had ended and i was just producing a few papers for a couple of journals, actually conferences. so i just feel that hasn’t been exploited, it’s the logical outcome and that needs to be, yeah just taught better. a principal supervisor, who had supervised three phd students to completion, was supervising six more, plus one dit student, stated: the publication of papers in my mind is a way of vetting it before you put it in. so you get some feedback from external people so you are on the right track. the advantages of publication are numerous. institutionally it helps lift the research quantum output. for supervisors co-publishing with their students, it lifted not only their publication rates, but because their students cited their supervisors, it also lifted the citation rates of the supervisors involved. for the students, having peer-reviewed publications during their dit progression meant that thesis examiners could readily see that the research had already stood up to scrutiny in the wider research community, and thus increased the chances of a successful examination process. one student who dropped the dit during the second year of coursework claimed that, had he been encouraged to publish his work from the first year (as is the case now with the new dit structure), it would have shown him that his work was valued, it would have meant that he could in-part fulfill the purpose he had for doing the dit which he claimed was to ‘give back’ to the ict community, and he expressed the view that it would have helped him to build a portfolio that he could use to promote himself in the industry marketplace. one cautionary note from a research support person was that supervisors have to beware not to take away the voice of their student, as they guide them to write better. finally, by involving students and supervisors together in a cop that combines discipline-specific interests with a focus on writing, the university can increase its publication rate, whilst also building positive relationships between students and supervisors, as well as thus encouraging greater networking between students with each other and within cops at the university. 6.2 student engagement with supervisors as was true for the wider literature on doctoral supervision, the busy workloads of academics frequently result in inadequate contact with part-time distance students. three supervisors stated that they met weekly with all their students, be they distance or full-time on-campus. the other supervisors had varying policies regarding contact with part-time students, ranging australasian journal of information systems burmeister 2015, vol. 19, pp. 55-70 professional doctorate completion rates 62 from every 2-3 weeks, to every two months, to whenever the student made contact with them. examples include the following: on average probably about every couple of weeks, usually by email and sometimes it's a flurry of email and sometimes one or other of us is off on their work commitments or whatever else. so it's probably on average, yeah about once every 10 days, once a fortnight there abouts. … i haven’t been at all uncomfortable or unhappy with the degree of contact. (mid-career male dit student) the first couple of months … on the phone. he called me on the phone and then it turned out to be skype calls. … every three weeks or so, i would say. (overseas, midcareer female dit student) all supervisors received the same workload allocation for supervision, although those with weekly contact with students, which typically involved reviewing progress on a weekly basis and pre-reading of the student’s latest work prior to the meeting, had a heavier workload. one situation was reported where no contact took place for six months, which was how long it took for the student to contact the supervisor. intuitively one could imagine that student progression and engagement would be significantly better for students who met weekly or fortnightly with their supervisors (typically using skype), than for students who met every two months or less often. thus a strategy of supervisor-to-student communication needs attention, possibly in the form of a policy directive by the school. the master-apprentice model derived from traditional phd supervisory practices is reflected in the dit. there were only two exceptions: in both cases these were overseas-trained academics, who brought with them to csu what they valued from their overseas experience, namely team supervision, in which the co-supervisors (1 or 2 co-supervisors) are an integral part of the process and are expected to meet with the student every time the principal supervisor meets with them. interestingly, however, one of the students interviewed was a student of one of these two overseas-trained supervisors, and that student expressed the view that they dealt mostly with the principal supervisor and rarely with the co-supervisor. thus it seems that contradictory information is being reported by the supervisor and one of their students. nonetheless, a strategy of collaborative supervision should be investigated further. there is evidence in the literature that collaborative supervision is increasingly being employed elsewhere to improve student progression (berteau & villeneuve, 2006; fenge, 2012), including inter-university collaborative supervision (hammond, ryland, tennant, & boud, 2010; malfroy, 2011). it also seems appropriate to the new dit structure, which relies on the involvement of industry co-supervisors, who may have little experience in the supervision of research and thus would benefit by being part of a team approach to the supervision of their students. another area that students highlighted was the relationship between them and their supervisors. in several cases the supervisory team had undergone numerous changes. one graduate of the dit stated that they had worked with six supervisors during the period of the degree. three changes were beyond the control of the institution, in that one supervisor retired, another shifted to a different faculty, and another took a redundancy package. furthermore, the student experienced a further delay when the final principal supervisor became severely ill and took seven months to recuperate, during which this student was left without adequate supervision, because the then co-supervisor, who did not have a phd, was unable to advise on the final write-up process, which was the stage that the student was at. with each change of supervisor came a change in focus, change in literature to review, and with the last, a change from quantitative to qualitative research. that student felt that the result was that the degree took two years longer than it would have otherwise. although this may be an extreme situation, the frequent changes of supervision and the inadequacy of supervision during the coursework components of the course, were recurrent themes. for instance, one student dropped the dit after experiencing a situation where his first supervisor left csu after six months and the next supervisor retired within two months of taking over as the principal supervisor. although csu could not have foreseen that the initially appointed supervisor would leave the institution, csu australasian journal of information systems burmeister 2015, vol. 19, pp. 55-70 professional doctorate completion rates 63 management would have been aware of the pending retirement of the second principal supervisor at the time of that person’s appointment as this student’s supervisor. 6.3 supervision appointment an inadequacy in preparing people for supervision was also identified. the strategies employed across the university sector for teaching academics how to supervise vary significantly. one person trained at a research intensive university stated that she had to be co-supervisor to an experienced principal supervisor for two years, sitting in on every consultation that person had with their students, before she was permitted to become a principal supervisor herself. that compares with the csu policy of a and b registers. a person new to supervision attends a threehour seminar about the supervisory process at csu and is then placed on the b register for 12 months, during which they can only be a co-supervisor. there is no standard for what happens during those 12 months and, as noted above, many principal supervisors refuse to involve cosupervisors at all, following the traditional master-apprentice model. then after 12 months the academic can apply to be listed on the a register, and through the simple expedient of that application becomes eligible to be a principal supervisor. intuitively one would have to assume that the depth of preparation differs significantly between the csu approach and that of the csu academic trained at a research-intensive university. this suggests that an institutional strategy needs to be investigated to improve preparation for principal supervision at csu. although there exists a nine point list of supervisor responsibilities and the ro makes supervisory training available throughout the year, using webinars, there is no compulsion on supervisors to participate in the training. in this researcher’s own experience, no one has ever held me to account for how my supervision stacks up against the nine requirements. related to the last point is the strategy of supervisor appointment for the dit. under the old structure supervisors were appointed to students, without the student’s input. in two cases students were told by their supervisors not to contact them, because the work that the students wanted to research had nothing of interest to these supervisors. in other cases, as indicated above, students experienced multiple changes of supervision, until either they adapted their research to suit the supervisor’s interests, or they found a supervisor with an interest in the student’s area of interest. a better approach would be to enable a mutual choosing. to this end, and similar to the pace university example of symposiums as a deliberate strategy of course design (grossman et al., 2011), the new dit structure includes an annual doctoral symposium which provides staff and students an opportunity to meet. 6.4 feedback on progress there is a need for policies around feedback and guidance. subject coordinators are understandably focused on the assessment requirements in their respective subjects. their feedback and guidance relates specifically to their subject. but two of the students who dropped the dit and one who graduated from it stated that the big picture was missing. they needed someone to guide them through the process, to help them see how what was being done in individual subjects was also building their skill sets for the eventual thesis. one such strategy of feedback and guidance is that of mentoring. under the old dit structure supervisors were appointed from day one, but effectively had little or no involvement with students, and it appeared to students that supervisors were not interested in them until they had completed the coursework components and were ready to begin the thesis work. under the new structure of the dit, students are similarly claiming that they are having to re-tell their story, their goals and ambitions in research to each subject coordinator, and that there is no single person who seems to know their personal aspirations and is willing to provide feedback and guidance. but the appointment of mentors to small groups of students in the first year of the dit would solve this. such a mentor would remain a student’s mentor until a supervisor is appointed mid-way through the second year. again this notion of mentoring is supported by the conceptual cop framework, in that it helps to build a small community of practice, involving 4-6 students with similar interests, methodologically or discipline-based, with at least one staff member who is an expert in that area. australasian journal of information systems burmeister 2015, vol. 19, pp. 55-70 professional doctorate completion rates 64 other strategies to improve feedback and guidance relate to adequate provision of subjectbased assessment criteria and resources to complete tasks. for the research proposal subject under the old dit structure, no exemplars were given and one student dropped the course when receiving a poor grade for that assessment task, despite having approached numerous sources within csu for help, including the subject coordinator. the student stated that the assessment of his submission clearly showed that certain things had been expected, yet those expectations had not been communicated. in undergraduate subjects csu policy for assessment criteria requires clarity of assessment expectations as part of the subject outline, which students receive upon enrolment, yet that does not appear to be the case for doctoral coursework. 6.5 student engagement in the course student engagement is required from the start. the present dit design starts with itc711, which introduces students to research, but in a general and generic fashion. two students who completed the first year of the new dit commented that itc711 was their first taste of the dit and that the subject was a turn-off. they would have preferred to see right from the start how their new research studies could be relevant to their workplace, rather than have to take it on faith that the eventual thesis work would have an applied and workplace relevant focus. 6.6 contribution the following is typical of dit students, this from a man in his late 50s: (with his supervisor) that was a brain storming at its very best and she was great to bounce ideas off and talk about things – that really pressed my buttons. … as mature age students which i suspect most of the dit students would be – there is no age difference and therefore the if you like a level playing field in terms of the interaction with staff is really empowering – not to be treated as students but to be treated as peers who have good valuable contributions to make. that’s why most folk i would think do it because they feel they have something to contribute and the dit is a way of achieving that outcome. the concept of contribution requires further investigation. all the dit students interviewed envision their doctorate as a way of ‘giving back’. they felt as though they had something to contribute. however, that desire on the part of these mature-age, industry-savvy students is an untapped resource. that desire suggests that these students are a rich source of ideas, research possibilities and other opportunities. 6.7 student involvement in institutional cops csu has various institutional support mechanisms that are geared to doctoral research including, but not specifically aimed at, professional doctorates. as mentioned in the introduction above, there are numerous expressions of professional doctorates. the faculty of education had one, although it has been discontinued. the faculty of science has several, particularly in areas related to medical and agricultural research. thus even though the new dit program structure is unique within csu, many of the doctoral support mechanisms are helpful to the dit. examples of support include: yesterday i did one, an induction yesterday for 17 phd students ... give them directional information particular to them you don’t have time in 20 minutes, whereas an induction that we do with the one-on-one might go for an hour, an hour and a half where you actually go into it in a bit more detail, point out things that are relevant to them, rather than general stuff. (library research support staff) i’m enjoying what i’m doing but keeping up with the workload in conjunction with a seriously demanding job is tough, so without – i think i would find it much harder without the support that i’m getting. (dit student) (csu has) been good at developing its support mechanisms. and the support that we have through the structural things like sub deans of research and research australasian journal of information systems burmeister 2015, vol. 19, pp. 55-70 professional doctorate completion rates 65 management committees and all of those sort of things. but right down to being, you know communicative to the students and making them aware, sort of all of the things that are here to support you, it's actually quite large. (dit and phd supervisor) university research supports include the ro, library staff whose function is supporting research, and the learning skills unit. faculty supports are predominantly available through sub-deans for graduate studies, their administrative support personnel and the resources they make available. school supports are provided through doctoral course coordinators, which in the case of the dit involves the dit course coordinator. thus from an institutional perspective, what has been described in the literature as strategies of collaborative supervision (borrell-damian et al., 2010) and what can be seen as a cop is in evidence. in cop terms, the ro primarily see their training role as encouraging a research culture amongst staff, that is, of supporting a community of practice for research active staff. students are welcome to attend research training events, but those events are designed to benefit research-active staff, not students. aside from their own training, they also facilitate access to research training provided through a consortium of universities. at present it appears that industry and other non-csu supervisors are not included in this training. however, given that such training is open to doctoral students if they express an interest in participating, it should also be possible to include industry supervisors, should they wish to participate. thus the ro focus can been seen in terms of what in the literature is described as strategies for training in supervision (amundsena & mcalpine, 2009; kiley, 2012; mcalpine & amundsen, 2012). that body of literature also focuses attention on the need to particularly provide training for early career researchers and for people new to academia (blass et al., 2012; christine halse, 2011; c. halse & malfroy, 2010). however, as seen above, there is a disconnect in professional development. that disconnect is between initial training for supervision, to rapidly build supervisory capacity, and large supervisory workloads that make accessing ongoing training difficult to attend, even virtually. for instance, two people new to supervision stated: … there’s no real follow-up training, like we don’t have a mandatory ongoing system of pd or, you know, professional development throughout the year (supervisor of 2 phd and 3 dba and 1 dit) there is some workshops or supervision, but i think that is not very helpful. (supervisor of 2 phd and 5 dit) it’s should never be just the research students, they should have an opportunity to get to interact certainly … but i’m actually thinking of the research students in the research community and that includes supervisors and others, because supervisors get very little training. … one of the reasons why csu has gone the way it has, is simply it has, it’s just not – it doesn’t have enough supervisors. it wants to get more and more students in; it doesn’t seem to be wanting to put the investment into really developing quality supervisors, so a lot of people get thrown into the deep end. in some cases, and i think that whole idea of experts and novices and different communities of practice …experienced and non experienced supervisors working together in expressly in communities of practice. (research support staff member; former doctoral supervisor) just as the ro is promoting a cop for research staff, so too an initiative of the fob sub-dean graduate studies to provide intranet-based resources to supervisors has proved popular. interviews with supervisors indicated that the majority appreciated this as a significant boost to helping them to become better supervisors. the resources made available through that site have been helpful in terms of understanding policies, overcoming challenges, and feeling part of a cop, that is, of being part of something bigger and not being alone in dealing with supervisory issues that arise. the ro have in the past held doctoral workshops for students, but found that they were poorly attended. one example was a generic online webinar, which was only attended by 20 doctoral students from across the university, and not a single professional doctorate student was among australasian journal of information systems burmeister 2015, vol. 19, pp. 55-70 professional doctorate completion rates 66 them. similarly, attempts by the ro to involve doctoral students in online forums have resulted in poor student engagement. this has consequently been devolved to the faculty level, on the assumption that this way there will be a greater discipline-specific appeal to students. this is not unique to csu. for example, the largest doctoral study yet completed, which involved over 17,000 survey respondents, reported that over 70% did not collaborate online, with 58% not having posted online contributions to institutional initiatives aimed at helping students to network (anonymous, 2012, p. 37). in that report the authors went on to recommend that institutional supports be discipline-based, rather than attempt to bring all doctoral students together. retention, student engagement and institutional support are interdependent. as one research student support officer said, students need to learn to manage their supervisors, not just wait to be managed by them. the dit is delivered within the faculty of business (fob), which has 130 students in various doctoral programs including phd, dba and dit. thirty of these were contacted by the sub-dean graduate studies. those students were predominantly working full-time and expressed the desire not to be in contact with other doctoral students. that has implications for faculties wanting to create a cop involving their students. again this is not unique to csu. the survey of over 17,000 doctoral students, mentioned earlier, found that whilst students expressed in-principle support for sharing research between students, when it came to making their own research available to other students, they were reluctant to do so, preferring to work alone. a strategy of developing collaborative research teams amongst doctoral students did not appeal to over 90% of respondents (anonymous, 2012, p. 41). area aids to progression barriers to progression policy implications further research retention agile course design writing papers along the doctoral journey flexible delivery to improve work-lifestudy balance student engagement with supervisors weekly or fortnightly contact contact less frequent than 3 weeks staff workload dependence on a sole supervisor change of principal supervisor frequency of contact supervisor (mentor) guidance during the coursework component collaborative supervision impact of frequency of contact on student engagement and progression supervision appointment opportunities for students and potential supervisors to meet and get to know each other supervisor has little interest in topic of student supervision training transition for cosupervisor to principal supervisor mutual choosing of student and supervisors how compliance with supervision policies can be improved feedback on progress coursework assessment expectations need to be clearly communicated feedback and guidance mentor during coursework student engagement in the course early engagement is important to retention address the disconnect between initial coursework research subject and their (often technical) work contribution making a contribution is motivating to these mature are students writing papers is a way of ‘giving back’ how to tap this resource student involvement in institutional cops post induction, oneon-one sessions between research library staff and doctoral students heavy staff supervisory workloads demanding workplaces of dit students supervisor workload management how to encourage part-time students to engage in cops table 2: factors for improving retention and progression in computing doctorates australasian journal of information systems burmeister 2015, vol. 19, pp. 55-70 professional doctorate completion rates 67 as seen above, seven areas have been identified as requiring focus in order to improve completion rates in professional it doctorates. they are summarised in table 2, which highlights key aids and barriers to progression, implications for institutional policy and areas for further research. 7 conclusions improving the completion rates of students undertaking a professional doctorate in it has multiple benefits, and not only for the students. the principal attraction of doctoral students for a university is not the money that they bring in, but research outputs. in addition to the contribution that this makes to the wider society and to the industry, such outputs lead to increased reputation, which in turn attracts more undergraduate students. doctoral studies have focused on the phd, and on supervision practices, as opposed to student perspectives, and discipline-based doctoral studies have rarely included it. this is despite the wide acceptance that doctoral students are a key part of the publishing strategy of a university. although the literature shows that in professional it doctorates the completion rates are poor, particularly for part-time students, lessons learned from interviews have resulted in the identification of key strategies aimed at mitigating particularly the risk of attrition, because the literature supports the notion that if students complete, then the quality of outcomes are comparable to conventional research-oriented doctorates, which in turn supports the contention that the professional doctorate is a ‘research doctorate’. acknowledgment this research was supported by two flexible learning institute fellowships from the csu deputy vice-chancellor (academic), which bought out half my teaching for two semesters. references alpert, f., & kamins, m. a. (2004). doctoral coursework is needed in australasia. australasian marketing journal, 12(1), 66-72. amundsena, c., & mcalpine, l. (2009). 'learning supervision': trial by fire. innovations in education and teaching international, 46(3), 331-242. andrew, m. (2012). supervising doctorates at a distance: three trans-tasman stories. quality assurance in education, 20(1), 42-53. anonymous. (2012). researchers of tomorrow: the research behaviour of generation y doctoral students. london. barnett, r., & coate, k. (2005). engaging the curriculum in higher education. maidenhead: mcgraw-hill. berteau, g., & villeneuve, l. (2006). integration of the learning process and the group development process in group supervision. groupwork an interdisciplinary journal for working with groups, 16(2), 43-60. blass, e., jasman, a., & levy, r. (2012). supervisor reflections on developing doctoralness in practice-based doctoral students. quality assurance in education, 20(1), 31-41. borrell-damian, l., brown, t., dearing, a., font, j., hagen, s., metcalfe, j., & smith, j. (2010). collaborative doctoral education: university-industry partnerships for enhancing knowledge exchange. higher education policy, 23(4), 493-514. cantwell, r. h., scevak, j. j., bourke, s., & holbrook, a. (2012). identifying individual differences among doctoral candidates: a framework for understanding problematic candidature. international journal of educational research, 53(0), 68-79. doi: 10.1016/j.ijer.2012.02.001 australasian journal of information systems burmeister 2015, vol. 19, pp. 55-70 professional doctorate completion rates 68 crossouard, b. (2008). developing alternative models of doctoral supervision with online formative assessment. studies in continuing education, 30(1), 51-67. doi: 10.1080/01580370701841549 crossouard, b., & pryor, j. (2009). using email for formative assessment with professional doctorate students. assessment & evaluation in higher education, 34(4), 377-388. doi: 10.1080/02602930801956091 cumming, j., & kiley, m. (2011). research graduate skills project. in t. gale & k. parker (eds.), good practice report student transition into higher education: department of education, employment and workplace relations. cuthbert, d. (2008). beyond the doom and gloom. campus review. engebretson, k., smith, k., mclaughlin, d., seibold, c., terrett, g., & ryan, e. (2008). the changing reality of research education in australia and implications for supervision: a review of the literature. teaching in higher education, 13(1), 1-15. doi: 10.1080/13562510701792112 erichsen, e. a., bolliger, d. u., & halupa, c. (2012). student satisfaction with graduate supervision in doctoral programs primarily delivered in distance education settings. studies in higher education, 1-18. doi: 10.1080/03075079.2012.709496 fenge, l. a. (2012). enhancing the doctoral journey: the role of group supervision in supporting collaborative learning and creativity. studies in higher education, 37(4), 401-414. fink, d. (2006). the professional doctorate: it's relativity to the phd and relevance for the knowledge economy. international journal of doctoral studies, 1, 35-44. green, b. (2012). addressing the curriculum problem in doctoral education. australian universities' review, 54(1), 10-18. grossman, f., tappert, c., bergin, j. o. e., & merritt, s. m. (2011). a research doctorate for computing professionals. communications of the acm, 54(4), 133-141. doi: 10.1145/1924421.1924450 guan, y., guo, y., bond, m. h., cai, z., zhou, x., xu, j., ye, l. (2014). new job market entrants' future work self, career adaptability and job search outcomes: examining mediating and moderating models. journal of vocational behavior, 85(1), 136-145. doi: http://dx.doi.org/10.1016/j.jvb.2014.05.003 halse, c. (2011). ‘becoming a supervisor’: the impact of doctoral supervision on supervisors' learning. studies in higher education, 36(5), 557-570. doi: 10.1080/03075079.2011.594593 halse, c., & malfroy, j. (2010). retheorizing doctoral supervision as professional work. studies in higher education, 35(1), 79-92. hammond, j., ryland, k., tennant, m., & boud, d. (2010). building research supervision and training across australian universities. strawberry hills, nsw: australian learning and teaching council. hirschheim, r., & klein, h. k. (2012). a glorious and not-so-short history of the information systems field. journal of the association for information systems, 13(4), 188-235. hoganson, k., & amoroso, d. (2011). inter-disciplinary doctor of science with computer science and information systems. kiley, m. (2011). where are our doctoral candidates coming from and why? : altc. kiley, m. (2012). altc:coursework in australian doctoral education: what's happening, why, and future directions? , from http://www.olt.gov.au/project-coursework-australiandoctoral-education-whataposs-happening-why-and-future-directions-2012 australasian journal of information systems burmeister 2015, vol. 19, pp. 55-70 professional doctorate completion rates 69 kimmerle, j., thiel, a., gerbing, k., bientzle, m., halatchliyski, i., & cress, u. (2013). knowledge construction in an outsider community: extending the communities of practice concept. computers in human behavior, 29, 1078-1090. kritzinger, e., & loock, m. (2012). a critical investigation into the current shortage of information technology postgraduates produced by unisa. paper presented at the odl12 conference. lee, a. (2009). some implications of european initiatives for doctoral supervision eua bologna handbook. making bologna work. european university association. lee, a., brennan, m., & green, b. (2009). re-imagining doctoral education: professional doctorates and beyond. higher education research & development, 28(3), 275-287. lee, a., green, b., & brennan, m. (2000). organisational knowledge, professional practice and the professional doctorate at work. in j. garrick & c. c. rhodes (eds.), research and knowledge at work: perspectives, case-studies and innovative strategies (pp. 117136). london: routledge. lee, a., & mckenzie, j. (2011). evaluating doctoral supervision: tensions in eliciting students’ perspectives. innovations in education and teaching international, 48(1), 69–78. loftus, s. (2010). exploring communities of practice. in j. higgs, d. fish, i. goulter, s. loftus, j. reid & f. trede (eds.), education for future practice. rotterdam, the netherlands: sense publishers. malfroy, j. (2011). the impact of university-industry research on doctoral programs and practices. studies in higher education, 36(5), 571-584. mcalpine, l., & amundsen, c. (2012). challenging the taken-for-granted: how research analysis might inform pedagogical practices and institutional policies related to doctoral education. studies in higher education, 37(6), 683-694. mccallin, a., & nayar, s. (2012). postgraduate research supervision: a critical review of current practice. teaching in higher education, 17(1), 63-74. merritt, s. m., stix, a., sullivan, j. e., grossrnan, f., tappert, c. g., & sachs, d. a. (2004). developing a professional doctorate in computing: a fifth-year assessment. paper presented at the iticse 2004 conference, 28-30 july 2004, leeds. metcalfe, m., & kiley, m. (2000). arguing for phd coursework. australasian journal of information systems, 7(2), 52-59. morse, j. m. (2008). confusing categories and themes. qualitative health research, 18(6), 727-728. ng, t. w. h., & feldman, d. c. (2014). subjective career success: a meta-analytic review. journal of vocational behavior, 85(2), 169-179. doi: http://dx.doi.org/10.1016/j.jvb.2014.06.001 pavlova, m. k., & silbereisen, r. k. (2014). coping with occupational uncertainty and formal volunteering across the life span. journal of vocational behavior, 85(1), 93-105. doi: http://dx.doi.org/10.1016/j.jvb.2014.05.005 richards, d., & marrone, m. (2014). identifying the education needs of the business analyst: an australian study. australasian journal of information systems, 18(2), 165-186. schön, d. a. (1987). educating the reflective practitioner: towards a new design for teaching and learning in the professions. san francisco: jossey-bass. smallwood, s. (2004). doctor dropout. chronicle of higher education. sowell, r. (2008). ph.d. completion and attrition: analysis of baseline data. columbia: australasian journal of information systems burmeister 2015, vol. 19, pp. 55-70 professional doctorate completion rates 70 copyright: © 2015 burmeister. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. editorial this is the first issue of volume 3 of the australian journal of information systems. as editor, i am pleased to report over the past six months there has been on-going interest in the journal, both from authors and subscribers, and recently we have developed a home page on the world wide web (see below). in this issue we have four papers from australia, together with an article from the usa and norway. our first paper 'evaluating the factors that facilitate a deep understanding of data analysis' by oliver burmeister. centre for information systems research, swinburne university of technology explores the important issue of the need for deep understanding, to successfully carry out the task of data analysis. the paper shows that despite the apparent decline of deep learning in tertiary course, deep learning can be enhanced through an increase in intrinsic motivation, assessment which seeks integrated, developed yet comprehensive understanding of the analytical concepts, and knowledge of student learning preferences. the next paper 'quality management systems in australian software houses: some problems sustaining creativity in the software process' by liisa a. von hellens, school of computing and information technology, griffith university, examines the effect of implementing quality management systems and certification to international quality standards. the paper suggests that the introduction of standards to less structured tasks such as pre-sales or research appears to be stifling the creativity needed to complete these tasks successfully. daniel l. moody and graeme c. simsion. simsion bowles and associates, contribute a paper entitled 'justifying investment in information resource management'. the paper suggests that while many organisations have introduced information resource management functions, few have developed formal measurements of their effectiveness. the paper identifies key criteria for the measurement program and proposes some candidate metrics. the next paper is 'software quality management and organizational fit' by s.h. nielsen, school of computing and information technology, griffith university. this paper suggests that after an initial period of staff acceptance of prescribed quality management procedures, certain features of organisational culture, power and structure have an effect on continued conformance. the paper suggests that by using a pluralist approach to organisational analysis, the nature and extent of changes to the quality management system can be revealed. sarah m. north, max m. north and nazir a. warsi, human computer interaction group and virtual display laboratory, computer and information science department, clark atlanta university contribute a paper entitled 'visual navigation of complex information spaces'. this paper lay the foundation for the introduction of visual navigation aids to assist computer users in direct manipulation of complex information spaces. the authors introduce a visualisation interface called visual-x which incorporates icons, dynamic slider and bindable attributes. the final paper in this issue is tailor made concurrency control distributed transactions as a case' by mads nygard, the database technology section, sintef delab, the norwegian institute of technology, trondheim. this paper applies a model for distributed databases and transactions with a distinction between global and local correctness criteria. the paper suggests that commonly used criteria for concurrency control tend to be very strict. a more relaxed set of criteria (allowing more true parallel behaviour) is demonstrated. once again i would like to pass on my thanks to the authors and consulting editors for the high quality of material in this issue. i would like to especially thank mr ted gould who took the role of editor for the previous edition of the journal and mr david dodds, without whose help, this journal would not have been produced. r.macgregor editor home page: http://www.uow.edu.au/public/faculties/commerce/business/ajis/ajis.html september 1995 microsoft word ajis special 2002.doc ajis special issue december 2002 59 information systems foundations karl popper's third world craig mcdonald university of canberra craig.mcdonald@canberra.edu.au abstract the various information professions have matured separately over the years, developing different bodies of theory and practice to meet their evolving purposes and needs. a problem arises however, when different information professions address the same knowledge domain and there is no explicit correspondence between the conceptual structures embedded independently in each. in this situation, a knowledge worker involved in the domain is faced with a range of possibly incompatible structures presented in different forms by a range of information professions. this is a common problem that is being exacerbated by the explosion in information production and the widening access to information distribution technology, notably the world wide web. information systems now need to combine the best of what the information professions in a domain have to offer the domain’s knowledge workers. this paper examines the problem by exploring one of the foundations of the information disciplines karl popper’s 3 worlds theory, applying it to a case study and suggesting that the information systems discipline alone has a sufficiently broad agenda to integrate the various informatics themes needed to support today’s knowledge workers. popper’s three worlds the idea that the universe of human experience can be analysed in terms of three interacting worlds is an old one. versions of the idea date back at least to plato’s cave analogy and to aristotle8, but the elaboration by karl popper is an interesting contemporary formulation. in popper’s theory, reality is divided into three parts: … first, the world of physical objects or of physical states; secondly the world of states of consciousness, or of mental states, or perhaps of behavioural dispositions to act; and thirdly, the world of objective contents of thought. (popper, 1972, p.106). popper distinguished thought, in the sense of the content of statements, and thought, in the sense of thought processes, as belonging to two entirely different worlds: if we call the world of ‘things’ – of physical objects – the first world, and the world of subjective experiences (such as thought processes) the second world, we may call the world of statements in themselves the third world. (popper, 1976, pp.180-1). the three worlds theory is diagrammatically represented in figure 1. objects & events mental processes statements 3: information world2: cognitive world 1: physical world figure 1: the three worlds turning first to the relationship between worlds 1 and 2, consider a human, conscious mind (so in world 2) that perceives objects in the physical world (world 1). what it perceives is constrained first by the nature of the sense organs and lower brain functions, which exclude much potential stimulus, and, secondly, by the nervous system and brain which organise the stimulus that has been detected. perception is the change of state in the mind as a result of paying attention to objects in world 1. humans do not only perceive world 1, they also conceive it. that is, their sensing of the world is tempered and interpreted in terms of what they have learned and what they understand to be the significance of what they perceive. so when the mind in our scenario sees a full moon it is perceiving the light as a disc and conceiving of what the full moon might mean; perhaps romance, or danger. 8 all translations of classical works follow conventions for identifying paragraphs. the relevant references are plato's republic 514-516 and aristotle's on interpretation 16a4. ajis special issue december 2002 60 there is, however, a two-way interaction between worlds 1 and 2. the one described above is perception, the other is action. when the person makes a deliberate act, there is an effect on world 1 as a result of the world 2 state. making a statement about the world is a special kind of human act. in its physical form, it is a part of world 1, speech sending out sound vibrations, for example. however, as a statement it is part of world 3. statements are symbolic representations of concepts held by the speaker. these representations take the form of words and linguistic structures. examples of objective knowledge are theories published in journals and books and stored in libraries; discussions of such theories; difficulties and problems pointed out in connection with such theories; and so on; “... we can call ... the world of the logical contents of books, libraries, computer memories, and suchlike ‘world 3’” (popper 1972 pp.73-4). world 2 acts on world 3 by writing, and inversely, by reading. clearly words have physical properties and as such are in world 1. they are perceived through the senses, but what sets them worlds apart is that the concepts they evoke in the mind of the perceiver are not those of black marks on a page, or letters of the alphabet or words of the language, but what those words denote. a link in the mind is being made between the word, or symbol, and the world 1 reality to which it refers (shown by the dotted arrow). figure 2 shows the interaction between the three worlds. perception 3: information world2: cognitive world 1: physical world action reading writing denotation figure 2: interaction between the three worlds the focus of this paper is on world 3, the information world, not with what it stands for in world 1, nor with who is creating and using it in world 2. this focus contrasts with research approaches such as sense making (dervin, 1991) or discourse analysis (beghtol, 1995; olsson, 1998), which respectively take a cognitive or communicative approach in which the content of world 3 is largely incidental. the three worlds model isolates information from its human or physical context and allows it to be examined per se. popper and common themes in informatics there are several major themes and terms that permeate the disciplines that study of world 3. these themes are notoriously broad and are used to denote what are really many different ideas. popper’s ideas as described above provide the basis of an integrated informatics theory within which the various themes can be comfortably accommodated. information to popper, a thought once “formulated in language so clearly that i can look at it critically from various sides ... is the thought in the objective sense”. he continued “the decisive thing seems to me that we can put objective thoughts – that is, theories – before us in such a way that we can criticise them and argue about them. to do so, we must formulate them in some more or less permanent (especially linguistic) form” (popper, 1976 p.182). the world 3 products of thought about the world take linguistic form; they are what this paper calls statements. this is not to say that all linguistic forms are in world 3; speech acts like greetings, curses or commands, for example, are not statements in popper’s sense. popper said that he peopled his world 3 with inmates including “statements or theories, also problems and arguments, especially critical arguments” (popper, 1976 p.182). but he had earlier alluded to the idea that all abstractions are part of world 3 including “poetic thoughts and works of art” (popper, 1972 p.106) and identified a wider class of inmates in world 3: … we may regard the world of problems, theories and critical arguments as a special case, as a world 3 in the narrow sense, or the logical or intellectual province of world 3; and we may include in world 3 in a more ajis special issue december 2002 61 general sense all the products of the human mind, such as tools, institutions, and works of art (popper, 1976 p.187). this wider scope for world 3 seems unavoidable in popper’s theory as, clearly, any abstraction that has been written down cannot be part of either world 1 or 2. however, popper was a philosopher of science and his basic conception of world 3 seems to have been concerned with statements of a scientific kind. this paper is interested in popper’s special case of statements that are intellectual; statements of a scientific kind. scientific statements may refer directly to world 1, say as a theory or data about a physical phenomenon, or indirectly, as in a criticism of such a theory. as such criticism and argument themselves comprise statements, with their referents already in world 3, a statement may refer to either worlds 1 or 3. to popper “the third world, the world of objective knowledge ... is man-made. but it is to be stressed that this world exists to a large extent autonomously; that it generates its own problems, especially those connected with methods of growth; and that its impact on any one of us, even on the most original of creative thinkers, vastly exceeds the impact which any of us can make upon it” (popper, 1972, p 147). this autonomy and separation of world 3 from either the creator or user of objective knowledge is important because the idea of information that is emerging is that it takes the form of statements that have been contributed to a domain of discourse and that refer either directly or indirectly to a set of world 1 phenomena. this definition of information is quite different from those that are based in the function of information. the dictionary of information science and technology, for example, defines information as the meaning assigned to data within some context for the use of that data (watters 1992). to be operationalised, watters’ definition requires a particular cognitive context in order to tell if a particular statement is to be counted as information or not. it is important to separate the structural and functional aspects of information because information systems are increasingly open the characteristics and purposes of the knowledge workers they support are much less predictable than in the past. concepts and cognition popper distinguishes between … subjective knowledge (which should better be called organismic knowledge, since it consists of the dispositions of organisms) and objective knowledge, or knowledge in the objective sense, which consists of the logical content of our theories, conjectures, guesses...” (popper, 1972 p.73) here he is drawing the distinction between tacit and explicit knowledge explained by polanyi (1958). the term concept has been used to refer to a unit of knowledge and is reified in common usage to refer to units in both world 2 and world 3. the distinction is between one's having a concept (world 2) and there being a concept (world 3). the difference is that the first is a state of mind, of cognitive action, while the second is a statement. to popper, meaning and understanding are world 2 processes. what a world 3 statement carries is the potential to create in its reader concepts similar to those held by its author. the statements do not mean anything on their own meaning is an attribute of a single mind in action, not an attribute of a statement. this view has two implications. firstly, given this idea, how can social groups display apparently collective, or shared, meaning (eg. “everyone understands what a red traffic light means”)? the answer comes from the socialisation of people; the education, legal and law enforcement systems; and from other ways of aligning human behaviour, not from the proposition that a statement has ‘objective’ meaning. secondly, the computer takes on a genuinely world 2 character as it takes a world 3 document (a computer program comprising statements) and executes them, generally producing new world 3 documents. information systems embody concepts. there is increasing need for systems to have world 3 schema that can be defined in a sharable way, just as language can define ideas in a sharable way between members of a society. representation popper saw world 2 being a mediator linking the two other worlds. the three worlds are so related that the first two can interact, and the last two can interact. thus the second world, the world of subjective or personal experiences, interacts with each of the other two worlds. the first world and the third world cannot interact, save through the intervention of the second world, the world of subjective or personal experiences (popper, 1972 p.155). popper’s description of world 2 as a mediator between worlds 1 and 3 entails that a world 3 statement is a representation of a world 2 state. this raises the problem of the theory-ladenness of personal observation; all observers hold, in the subconscious, a myriad of preconceptions and biases about the way the world operates that affect their perception to make observations (whitelaw et al, 1992). semiology, post-structuralism and postmodernism have given very different accounts of the nature of language and the relationships between people, language and the world (potter 1996). understood from these accounts, language does not describe reality; it actually constitutes it (cunningham & turner, 1993). ajis special issue december 2002 62 popper’s principle of falsification provides a response to the issue. a statement, he argues, cannot be verified (i.e. proved to be true) but can be falsified by observing a counter example. in conjectures and refutations, popper (1969) argues that to be admitted to world 3, a statement, as a conjecture, must be refutable; that is testable at some point against world 1, not just world 2. for information systems the issue of representation is core as systems embody concepts and data that needs to be valid and effective in their domains. systems analysis is about the questioning and verification of concepts that systems hold and use. communication it was harold d. laswell (1948) who seems to have first described a general model of communication identifying a flow of information involving a source, message, medium, receiver and effect. he suggested that this might be a useful way to structure communication research with separate research agendas for each component of the model – control studies, content analysis, media analysis, audience analysis and effect analysis. this communication theme concentrates on either the production of a message or the change that a message engendered in a receiver. because the focus of communication is on the expression of the sender's mental state or the mental state of the receiver rather than the message itself, it is a world 2-oriented approach. popper included messages in world 3 several times in his writings (eg.popper, 1982, p.115; 1972, p.157) but it seems that he thought of a message as having a more perennial nature than the modern day usage of the term. to popper “.... books and journals can be regarded as typical world 3 objects” (popper, 1976, p.182) rather than a telephone message or an e-mail message, but increasingly document management technology is redefining what it is that constitutes a ‘document’ and information systems are storing a variety of ‘grey’ information. integrating informatics themes the informatics themes discussed above can be integrated in the popperian framework shown in figure 3. the subject of a document, the referent, is shown in popper’s world 1 or 3. the generator and user of a document are in world 2, the world of states of consciousness, or of mental states. the documents containing statements make up world 3, the world of poppers “objective contents of thought”. this theoretical framework accommodates all the informatics themes. referent statements (documents) world 3world 2 world 1 generator user referent figure 3: integrated informatics themes in a popperian framework debates about epistemology, consciousness and the relationship between mind and the world have gone on for thousands of years. the advent of the computer has made some major changes to these debates as it can represent and process symbols, functions previously thought to be characteristics of mind not machine (borgmann, 1999; hobart & schiffman, 1998). in the face of these changes, popper’s views remain forceful as an informatics theory. the theoretical base presented above can now be used to develop an operational framework for knowledge domains and to integrate the various information professions that are active in domain analysis and formulation, and in the service of domain knowledge workers. knowledge domains and information professions world 3 has been presented so far as a fairly amorphous collection of statements, but it is actually structured, complex and dynamic. this section gives some form and structure to the notion of world 3, showing that the body of information recorded about some phenomenon has an underlying ontological structure that this is presented in artefacts used by the information professions. ajis special issue december 2002 63 domain has been used to refer to very broad areas, for example, the ‘domain of the humanities’, and to very small areas, as, for example, in mathematics, where domain refers to the set of elements to which a mathematical or logical variable is limited and on which a function is defined. in the information systems discipline, an application domain is the specification of the objects, relationships and behaviours that exist within a particular system. domain in this paper refers to the body of information that is recorded about some phenomena. domain scope hirschfeld & gelman (1994) argue that while “some domains appear to carve the world at its joints” (p.25), the degree to which a domain is dependent on the world is complex and variable and that “a principled way of defining what a domain is continues to elude us” (p.25). just as the common use of the term concept tends to confuse its world 2 and world 3 meanings, so a common sense use of domain tends to confuse its world 1 and world 3 meanings. as a world 1 idea, domain would refer to an interrelated set of phenomena. however, several different world 3 domains might address the same phenomena. a child’s behaviour, for example, can be seen from the psychological, sociological, physical or educational viewpoints, each supported by a different body of world 3 theory and information. as a world 3 idea, domain is simply a set or cluster of interrelated statements. however, the above definition does not give any way of isolating one domain from others in world 3. world 3 is a passive place, a store of statements. the statement of what constitutes a domain is just like any other statement in world 3. a domain is a concept like any other, created in world 2 and reported explicitly in world 3. for example, a document describing the concepts in a university course in plate tectonics isolates the domain of plate tectonics, at least as seen by the professor who authored it. other documents from other sources that seek to give an account of plate tectonics may well see the domain differently. forms of domain content the physical nature of documents used to hold information is largely text-based, traditionally on paper in books and journals, more frequently now in full text electronic form and in hypertext. there are other, less obvious, documents that hold world 3 statements. a knowledge-based computer system, for example, comprises statements in a formal logic language, a computer simulation system comprises statements in mathematical language and a database system comprises statements in a data definition language. all these computing systems can be seen as documents holding statements in world 3. while all statements are information, the form of computer-based statements makes them a special kind of information. the terms data and knowledge are controversial in informatics, but popper’s three worlds model allows a clear definition of them to be made. data is a kind of statement with a specific structure. it is a statement that identifies the value of an attribute of a phenomenon. for example the statement “the author of the book, dilemmas in the study of information is s. d. neill” is data because it identifies that a phenomenon (the particular book) has an attribute (the author), which has a value (his name). the components of data statements are parts of a domain theory, in this case a theory in which books and authors play some role. as data is symbolic it is in world 3 but it has a very close relationship to the world 1 phenomena it represents – the representation is concrete. figure 4 shows the location of data statements in world 3. information is generally contributed to world 3 via world 2, but data can be electronically or mechanically detected and recorded directly into world 3. knowledge has many definitions, most of which view it as a capacity or an ability of some agent to process information. as such, knowledge belongs in the cognitive world, not the information world. however, statements that represent knowledge are a different matter. they are statements that assert relationships between concepts that can be used in reasoning, once they have been learned or programmed into an informationprocessing agent (debenham 1989). represented knowledge is in world 3, but when it is activated, in a human mind or a computer, it is in world 2. figure 4 shows the location of represented knowledge in world 3. world 3world 2 world 1 knowledge represented data active knowledge figure 4: data and knowledge as special kinds of information in world 3 ajis special issue december 2002 64 world 3 then, comprises information in the form of statements. two special kinds of statements are data and represented knowledge. information systems accept and store information and make it available to world 2 agents knowledge workers of the human and computer kinds. for such systems to be possible an ontology underlying world 3 is necessary. ontology as a philosophical term, ontology concerns the branch of metaphysics that investigates the nature of being or existing (the other branch is cosmology, the theory describing the origin and structure of the universe). “if one asks whether there are numbers and other abstract objects, or whether there are properties, one is asking ontological questions” (guttenplan, 1994, p.452). koepsell (1999) argues that ontology is distinguished from meta-physics by being concerned with categories rather than with “being qua being ... ontology may include such objects as angels, apples and bosnia without addressing the typically metaphysical questions of whether angels exist or how many can fit on the head of a pin” (p.217). a distinction can be made between what might be called pure ontology, being qua being, and applied ontology, which concerns the exposing of the conceptual underpinning of a problem or position (smith 1999). the scope of an ontology is either global, attempting to organise and categorise everything that is possible in the universe (aristotle; ranganathan, 1967; poli, 1996), or local to a domain (van der vet & mars, 1998; guber, 1993). lehmann (1993) produced a draft ‘concept systems catalogue’ with more than two hundred ontological schemes in the collection. ontologies have numerous uses. smith (1999), from a computer science perspective, identifies roles for an ontology in knowledge engineering, information retrieval and extraction, natural language translation, database design, conceptual modelling, information systems design and enterprise integration. guarino (1998) says that its importance is being recognised in information modelling and object-oriented analysis, knowledge management and agent-based systems design. vickery (1997), and sowa (1984) describe how ontology is increasingly being used to refer to a class of artefacts that identify and describe the conceptual structure of a domain. unfortunately, the information professions have over time selected and developed domain scopes, user typing, ontologies and ways of expressing ontologies in isolation from each other. the information professions the information professions are those groups of practitioners that take as their subject matter the organisation and mobilisation of information. machlup & mansfield (1983) identified over thirty information professions. each profession can be viewed as a particular kind of mediator between information in world 3 and a particular kind of user in world 2. for example, the teaching profession might be primarily concerned with building cognitive ability in students, however in achieving this end it needs to organise the concepts in the domain being taught. the same domain may well be being organised quite separately by another profession, say librarianship for its own reasons. the professions both have a defined and common set of phenomena to address – the information domain; it is in a common form – language; and they may well have a common clientele. each profession has created a range of methods and tools that it uses to specify the conceptual structure, the ontology, of the various knowledge domains that it addresses. examples of these methods include the representation techniques for specifying data and knowledge structures in the information systems profession, the methods of cataloguing and classification of books in the library profession, and curriculum and subject matter design in the teaching profession. the context and aims of each profession have influenced the development of the methods and tools it uses. the library profession a library’s “fundamental purpose is to acquire, organise, disseminate, or otherwise provide access to the vast bodies of knowledge already produced” (rubin, 1998, p 171). there has been a shift of emphasis from the collection and storage of books that characterized the library of thirty years ago to the provision of access to information in physical or electronic forms today. the evolution of virtual or digital libraries (vlib 2000) on the world wide web is increasingly providing a complement to the physical library. the ontological artefacts that library professionals use to describe and organise documents are a classification scheme and controlled vocabularies, including the thesauri and subject heading lists. document descriptions are stored in catalogues that can be accessed through information retrieval systems using the descriptors. information retrieval by keywords from a catalogue, full text database or the world wide web can be supported by additional artefacts such as dictionaries and encyclopaedias. the information systems profession the information systems profession is essentially concerned with the development of computer-based data management systems in a corporate setting (whitten et.al., 1989). the systems collect, store and process data at ajis special issue december 2002 65 the operational, tactical and strategic levels of an organisation. the information systems (is) professional is concerned with issues of corporate data analysis and with the design of systems using information technology and addressing the human and organisational aspects necessary to make a data management system function effectively. specialisations have developed to address the data management problems of specific industries, such as health informatics (hersch 1995), of particular types of data management problems, such as bibliographic data and of particular technologies, such as database management systems or systems development methodologies. the information systems specifications are ontological artefacts produced by a systems analyst/designer after a conceptual analysis of the application domain. this analysis defines the information and data types, constraints and processing rules, or algorithms, necessary to manage domain data. systems are constructed from the specification using information technology such as database management systems and computer programming languages. mcdonald (1996) shows how the various is ontological artefacts (ie. specifications in the form of diagrams, etc) can be expressed in a common coherent language – sowa’s conceptual graphs. information systems are often concerned with data and processing policies that are private to an organisation. the exponential growth in information systems applications has created a need for inter-operation or integration of systems for efficiency and consistency. this trend has exposed a considerable problem because the conceptual structures of individual systems are proving to be incompatible when they are merged. this integration problem is an example of the wider problem being addressed in this paper – detecting differences in domain structures. the teaching profession while teaching is concerned with building cognitive ability in students, it is an information profession in so far as it organises and presents domain information to its users. there is a wide range of learning theories and educational concepts that underlie teaching and learning practice, but most modern theory is centered on cognitive psychology (that students’ learning is changing their mental models) and constructivism, that proposes: … learning is an active process in which learners construct new ideas or concepts based upon their current knowledge. the learner selects and transforms information, constructs hypotheses, and makes decisions, relying on a cognitive structure to do so. cognitive structure (i.e. schema, mental models) provides meaning and organisation to experiences and allows the individual to ‘go beyond the information given’. (oltc 1996) here, again, part of the information world is being organised into a domain to suit the needs of a particular user, the student. from a world 3 perspective, texts used by the educator comprise the domain. these texts include the materials the educator produces for teaching, such as lecture notes and study guides and the textbooks they prescribe and refer to. the ontological artefacts that are used to represent the organisation of the domain can be found in curriculum specifications and in the conceptual organisation of lecture notes, study guides and textbooks teachers prescribe and use. the view of a domain by the professions this section has reviewed knowledge domains and three information professions and the kinds of ontological artefacts that they employ. figure 5 shows a hypothetical domain with a set of documents and artefacts being used by three professions. ajis special issue december 2002 66 information professions library information systems teaching domain document artefact figure 5: information professions addressing different documents and artefacts in a domain some documents are not used at all, perhaps being inaccessible to all information professions in a particular local domain; some are shared by two professions – textbooks for example, that are part of the library collection and used in teaching; but generally documents are the preserve of one or other of the professions. some artefacts are associated with a document, a back-of-the-book index for example; and others are not. a case study the argument so far has separated out the main components of the information world and demonstrated the linkages between domain participants, documents, ontological artefacts, real-world phenomena, and information professions. a problem arises however, when different information professions address the same knowledge domain and there is no explicit correspondence between the conceptual structures embedded independently in each. in this situation, a knowledge worker involved in the domain is faced with a range of possibly incompatible structures presented in different forms by a range of information professions. mcdonald (2000) provides an example. the growing of grapes, viticulture, is an ancient agricultural industry that has, during the twentieth century, increasingly become the focus of scientific research, mainly because of the value of the crop for wine making. there are specific research institutes for viticulture, the literature has been expanding rapidly and there are various courses in viticulture with different curricula. figure 6 shows the relationships between the information professions, the user and the viticultural domain at charles sturt university, australia (csu). the viticulture domain comprises the collection of documents held by the academic library, the teachers’ course notes and other teaching materials, and the specifications of a significant viticultural information system (ausvit dss). ajis special issue december 2002 67 (viticulture course) information system world 3 library teaching (csu) (ausvit dss) user (viticultural knowledge worker) user's vineyard world 2 world 1 knowledge generator figure 6: full operational framework using the viticulture domain at csu as an example a method for researching this situation, domain meta-analysis, was described by mcdonald (2000). the method allows an analysis the concepts in domain organising artefacts and compares them to show the kinds of differences the professions have in their views of the structure of the viticulture domain. figure 7 is a venn diagram showing the number of concepts contributed to the domain from each profession (lp for the librarianship profession, tp for the teaching profession and is for the information systems profession). the teaching profession, for example, contributes 1730 concepts that are not used by either of the other professions, it shares 49 concepts only with the librarianship profession, it shares 6 concepts with both the librarianship and information systems professions and lastly it shares 44 concepts only with information systems. 446 1 49 223 1730 148 is tp lp figure 7: venn diagram of the number of concepts by profession in the viticulture domain (lp librarianship; tp teaching; is information systems ) the total number of concepts in the viticulture domain is 2201, the sum of the above segments. the numbers used here represent unique concepts. that is, a concept may be used many times in different professions but it is counted only once in the appropriate cell of the diagram. so acid-soil conditioners is unique to the library profession and is one of the 223 concepts in that sector, australia occurs in both librarianship and teaching so is part of the 49 concepts shared by those professions, and diseases is used by all three professions so is part of the 6 in the central part of the diagram. the problem for the user is that the different information professions address the same knowledge domain yet there is no explicit correspondence between the conceptual structures embedded independently in each. the user is faced with a range of possibly incompatible structures presented in different forms by a range of information professions. ajis special issue december 2002 68 consequences of the argument a truly integrated information system in viticulture would address the above problem and, through a range of technologies, allow a knowledge worker to move seamlessly between the different conceptualisations of the viticulture domain by the different information professions. as importantly, it would allow the professions to compare and discuss their conceptualisations. mcdonald et al (1997) present a knowledge management system that would facilitate this kind of activity. this paper has examined a problem faced by knowledge workers in a domain and at the same time exposed one of the foundations of information systems discipline karl popper's 3 worlds theory. there are other foundations systems science, organisational behaviour, cognitive science and so on, but without a firm world 3 theory there is no way to integrate and systematise them. the information systems profession is the one that is supposed to see the world systematically and holistically in terms of emergent properties, interacting parts, multiple views and so on, but it has restricted itself too much. that it has not moved much from its roots in the organisational data processing of thirty years ago has opened the door to new “fields” like bpr, ecommerce and knowledge management that are essentially just forms of information systems. the information systems discipline is the only informatics discipline that has the potential to accommodate the various forms of world 3, the technologies that store it and the knowledge workers who produce and use it. the future demands much more of the information systems discipline than it is currently delivering and if it does not start to produce information systems professionals with a much broader capacity, then some other discipline will arise that does. references beghtol, c. (1995) “domain analysis, literary warrant, and consensus: the case of fiction studies”, journal of the american society for information science 46(1), pp 30-44. borgmann, a. (1999) holding on to reality; the nature of information at the turn of the millenium, university of chicago press: chicago. cunningham, s. & turner, s. (eds.) (1993) the media in australia allen and unwin: sydney. debenham, j. (1989) knowledge systems design, prentice hall: englewood cliffs, n.j. dervin, b. (1991) “information as non-sense; information as sense: the communication technology connection”, in bouwman, h., nelissen, p. & voojis, m. (eds.), tussen vrag end aanbod, pp 4-59. guarino, n. (1998) “formal ontology and information systems” http://www.ladseb.pd.cnr.it/infor/ontology/papers/fois98.pdf guber, t. (1993) “a translational approach to portable ontologies”, knowledge acquisition, 5(2), pp 199229. guttenplan, s. (ed.) (1994) a companion to the philosophy of mind, blackwell: oxford. hersh, w. r. (1995) information retrieval: a health care perspective, springer: new york. hirschfeld, l. a. & gelman, s. a. (1994) mapping the mind, cambridge university press: cambridge. hobart, m. e. & schiffman, z. s. (1998) information ages: literacy, numeracy and the computer revolution, the johns hopkins university press: baltimore. koepsell, david r. (1999) “introduction to applied ontology: the philosophical analyses of everyday objects”, american journal of economics and sociology 58(2), pp 217-220. lasswell, h. d. (1948). “the structure and function of communication in society”, in bryson, l. (ed.) the communication of ideas, harper and row, pp 37-51. lehmann, f. (1993) http://briet.berkeley.edu/phil/categories.html machlup, f. & mansfield, u. (1983) the study of information: interdisciplinary messages wiley: new york. mcdonald, c. (1996) “information systems modelling and conceptual graphs”, proceedings of 7th australian conference on information systems, hobart, december, pp 461-473. mcdonald, c., pun, w. k. & weckert, j. (1997) “knowledge management for the applied sciences” proceedings of the american association for artificial intelligence spring symposium series artificial intelligence in knowledge management, stanford university, march, pp 110-114. mcdonald, c. (2000) analysing and coordinating the conceptual structures of the information professions (a domain oriented approach), unpublished phd thesis, university of new south wales. olsson, m. (1999) “discourse: a new theoretical framework sor examining information behaviour in its social context”, in wilson, t. d. & allen, d. k. exploring the contexts of information behaviour, taylor graham, pp 136-149. oltc (1996) open learning technology corporation, http://www.oltc.edu.au/ polanyi, m. (1958) personal knowledge, routledge paul: london. poli r. (1996) “ontology for knowledge organisation”, in green, r. (ed.), knowledge organization and change, indeks, frankfurt, pp. 313-319 ajis special issue december 2002 69 popper, k. r. (1969) conjectures and refutations: the growth of scientific knowledge, 3rd ed, routledge paul, london. popper, k. r. (1972) objective knowledge, an evolutionary approach oxford, clarendon press, london. popper, k. r. (1976) unended quest: an intellectual autobiography fontana, london. potter, j. (1996) representing reality, discourse, rhetoric and social construction sage, london. ranganathan, s. r. (1967) prolegomena to library classification, 3rd ed, asia publishing house, london. rubin, r. e. (1998) foundations of library and information science, neal-schumann, new york. smith, b. (1999) “ontology: philosophical and computational”, http://wings.buffalo.edu/philosophy/faculty/smith/articles/ontologies.htm sowa, j. f. (1984) conceptual structures: information processing in mind and machine, addison wesley, reading, ma. van de vet, p. e. & mars, n. j. i. (1999) “condorcet query engine: a query engine for coordinated index terms”, journal of the american society for information science 50(6), pp 485-492. vickery, b .c. (1997) “ontologies”, journal of information science 23(4), pp 277-286. vlib (2000) http://vlib.org/aboutvl.html watters, c. (1992) dictionary of information science and technology, academic press, boston. whitelaw, m., weckert, j. & mcdonald, c. (1992) “is information technology shaping human cognition?” proceedings of anzaas convention, brisbane. whitten, j. l., bentley, l. d. & barlow, v. m. (1989) systems analysis and design methods, 2nd ed, irwin, homewood, il. paper title (paper title style) australasian journal of information systems latimore, nolan, simons & khan 2017, vol 21, research on indigenous ict reassembling the indigenous public sphere 1 reassembling the indigenous public sphere jack latimore university of melbourne latimore.j@unimelb.edu.au david nolan university of melbourne margaret simons university of melbourne elyas khan university of melbourne abstract this paper seeks to provide an initial theoretical grounding to assess a practical project: a new software application that attempts to be a beneficial resource in the field of indigenous representation. as a starting point, we are concerned to provide a theoretical ground for considering the inherited and shifting spaces of indigenous media representation. to this end, this paper reconsiders the strengths and weaknesses of debates surrounding the ‘indigenous public sphere’. this is used as grounds for critically understanding the relations that constitute this field. following this, we consider how a more materialist approach to publics might enable a productive reconceptualization, and in particular how digital media initiatives and shifting news markets may be contributing to change. finally, drawing on this model, we outline both the ‘wakul app’ project, and how this framework might inform an assessment of its impact. keywords: indigenous media; public sphere; activism; amplification; social media; representation 1 introduction critical work that has focussed on news media representations of indigenous people and issues has focused on a number of profound shortcomings. across numerous studies, it has been found that indigenous people are positioned as a ‘problem population’. issues facing indigenous communities have relied on a narrow range of frames and voices, and discourses of deficit and blame. while such findings should not be dismissed, there is a risk that critical perspectives might, albeit inadvertently, tend to position issues facing indigenous people as unchanging and intractable, by positioning the field of representation as overly static (cottle 2000: 9). in recent years digital transformations have precipitated ‘crises’ within, and reconfigurations of, mass media markets that have resulted in both new players and altered relations with audience-users (simons 2013; lewis 2012). at the same time, the rise of networked icts and social media platforms have provided new opportunities for both ‘mass self-communication’ (castells 2009) through peer-to-peer networks, and for collective, networked media initiatives that leverage off the popularity and affordances of platforms such as twitter and facebook. as numerous studies have noted, indigenous australians have been strong adopters of new digital media communication technologies, leading to a proliferation of indigenous voices online (carlson and frazer, 2015; mcnair ingenuity research 2014; waller et al 2015, sweet et al 2013; lumby 2011). more indigenous people now choose to be informed about the issues and events that affect them via digital media channels and sources, and may bypass more traditional media outlets altogether. such adoption, as we discuss below, has also contributed to a proliferation of indigenous news outlets that are now covering issues and events that previously were not reported at all by the mainstream media. in light of such transformation, it is timely to reconsider extant conceptualisations of the space of indigenous media representation. to this end, we begin this article by reconsidering the concept of the ‘indigenous public sphere’. as we go on to discuss, this has previously been australasian journal of information systems latimore, nolan, simons & khan 2017, vol 21, research on indigenous ict reassembling the indigenous public sphere 2 elaborated in two distinct and contrasting ways. on one hand, the indigenous public sphere is positioned as a space of indigenous media production that is separate from, and provides a significant critical counterpoint to, mass media produced by non-indigenous individuals and organisations. in this conception, the indigenous public sphere acts as a ‘counter-public’ that is distinct from and significantly contests the ways in which indigenous people and issues are reported and discussed in ‘mainstream media’. on the other hand, the indigenous public sphere has also been positioned as a wider sphere of representation, where patterned but heterogeneous ideas and images of indigeneity are produced and circulated across a broader (usually national) space by both indigenous and non-indigenous actors. by contrasting the work of hartley and mckee (2000) and meadows (2005), we begin this article by critically comparing and contrasting these traditions. in doing so, we not only consider their respective strengths and possible weaknesses, but also how both share a tendency to conceptualise the public sphere as a space of dialogue and speech. while this usefully foregrounds issues of voice and representation, we argue that it can tend to problematically underplay more prosaic, material processes and practices through which publics are ‘made’ (latour, 2005). by adopting a materialist focus that draws on the tenets of actor network theory (ant), we move to consider the potential contribution of a particular media intervention, the wakul app, that seeks to use digital media affordances to improve indigenous media access and representation. 2 reconsidering the indigenous public sphere in an overview of research on representations of indigenous people and issues, mccallum and waller (2017, forthcoming) note that the majority of research tended to mount (or provide evidence to support) a critique of ‘media racism’. such work has focused critically and analytically on how media coverage and reporting practices serve to sustain racist ideologies, and has repeatedly found that routine journalism practices result in portrayals of indigenous people and issues as a source of conflict, blame and deficit (due and riggs, 2011; meadows, 2001; brough, 1999; ewart, 1997; jakubowicz, 1994; cunneen, 1992). such representations contribute to an environment that produces negative social and health outcomes for indigenous people, both directly and in their contribution to the process of developing and implementing policy (stoneham et al 2014, mccallum 2011, paradies et al 2008). in summarising the findings of a wide range of relevant studies, mccallum and waller conclude: …news reporting overwhelmingly represents indigenous australians as a source of societal risk and as problematic for the mainstream; indigenous policy is generally only of interest when it meets a narrow range of news values – most importantly conflict and proximity to political elites; and the media construct indigeneity as deviance and privilege news values of conflict and otherness (mccallum and waller 2017, forthcoming). while the concept of an ‘indigenous public sphere’ does not necessarily exclude such critical perspectives, it nevertheless stands in partial contrast to them. work that focuses on media racism tends to critique the role of mainstream media as perpetuating power relations underpinned by racial inequality. the model of the public sphere, by contrast, tends to be mobilized as a (implicitly or explicitly) normative concept, and a (actual or potential) space of inclusive, democratic debate. in his celebrated (and much critiqued) account of the bourgeois public sphere and its subsequent structural transformation, habermas (1989) presented the public sphere as both a basis for describing actual spaces of public debate and simultaneously as a normative critical ideal. this conflation has been strongly criticised for its tendency to result in an overly idealised account, that underplays the degree to which mechanisms of exclusion, as well as principles of inclusion, structured to bourgeois public sphere (see thompson 1995; fraser 1992; eley 1992). nevertheless, the indigenous public sphere inherits a tendency to position the public sphere simultaneously as an actual space of communication (the actually existing public sphere) and as something that refers to an ideal or aspirational model of democratic communication. australasian journal of information systems latimore, nolan, simons & khan 2017, vol 21, research on indigenous ict reassembling the indigenous public sphere 3 the use of the public sphere approach as an alternative to critiques of media racism is most strongly articulated by hartley and mckee (2000), who announce that their project is ‘an attempt to re-theorize the field’ that stands in contrast to ‘previous work [that] has often concentrated on finding examples of racist or ‘negative’ coverage’ (2000: 1). in contrast to such ‘negative’ findings, hartley and mckee present a more optimistic perspective, in which the indigenous public sphere is presented as moving increasingly towards a more ‘dialogic’ model. in this model, the public sphere is conceived as a space that does not simply reflect ‘the public’ as an externally existing population. rather, by enabling the formation and exchange of ‘public opinion’ and ‘public imagery’, it is through the public sphere of media representations that conceptions of collective identity, and the role played by indigeneity within it, are negotiated by indigenous and non-indigenous australia. the ‘indigenous public sphere’, in this definition, does not refer solely to the space of news, but rather to the entire semiotic space in which images and conceptions of indigeneity are produced and circulated through various (non-indigenous and indigenous) media. the indigenous public sphere is thus also not entirely, or even primarily, the preserve of indigenous participants. indeed, hartley and mckee comment that “[t]hus far, the indigenous public sphere has hardly been under the control of indigenous people” (2000: 6). nevertheless, they strongly emphasise that indigenous people are “far from passive recipients of media representations”, for they “are themselves media producers, and are active participants in the processes of media production, dissemination, regulation, reception and innovation” (2000: 6). this framework draws on and reproduces aspects of hartley’s earlier theorisation of the feminist public sphere (hartley 1996), in which he emphasised the degree to which feminist publics not only create their own ‘counter-public’ spaces of dialogue, but have successfully sought to both challenge and gain access to the wider public sphere, such that the latter both represents and contributes to the negotiation of a feminist identity politics. hartley explicitly notes that a feminist public sphere and a broader public sphere are, at least in part, mutually constitutive (hartley 1996: 69). building on this, hartley and mckee (2000) point out that indigenous people are not mere passive recipients of representation, but active and effective participants in media production, and that this contributes to the public contestation of notions of indigeneity. this is an important point, and one that is increasingly relevant in light of the dramatic and effective take-up and mobilisation of digital platforms by indigenous media activists. nevertheless, the utility of hartley and mckee’s model is compromised by an overly idealised set of claims regarding the ‘dialogic’ space of the public sphere. in conceiving this space as dialogic, hartley and mckee draw on anthropologist fred myers’s analysis of pintupi meetings as being geared primarily toward the achievement of consensus, and assert that media increasingly play a similar role, as a means via which understanding of indigeneity is increasingly developed and contributory to collective indigenous and non-indigenous identity (2000: 83). here, the conception of ‘dialogue’ is underpinned by a claim that media are increasingly moving towards a progressive and inclusive public sphere: an ‘evolutionary’ process they view as inherent to a ‘logic of democratic equivalence’. hartley has argued elsewhere this logic is inherent to modernity, and progressively fulfilled in the shift toward a ‘postmodern’ commercial media environment (hartley 1996, 1999). this also forms the foundation of hartley and mckee’s theorisation and reading of the ‘indigenous public sphere’: political institutions are evolving in which commercial organizations are taking responsibility for democratic practice. public participation is much higher and more enthusiastic in 'commercial democracy' than in the formal mechanics of representative politics (hartley and mckee 2000, p4). this reading has several problems. firstly, as mccallum and waller argue, it is not supported by hartley and mckee’s own analysis of media materials, which provides ‘as much evidence for the dominance in media reporting of discourses of ‘correction and protection’ of indigenous people as it does for the celebratory outlook for indigenous self-expression’ (2017, in press). such a contradiction can be resolved, in circular fashion, by simply referring back to the premise of evolutionary modernity, such that the evidence is presented as a remnant product australasian journal of information systems latimore, nolan, simons & khan 2017, vol 21, research on indigenous ict reassembling the indigenous public sphere 4 of a prior modernist regime of news that has yet to be fully expunged (hartley and mckee 2000: 207). secondly, hartley and mckee’s approach discounts critical consideration of the legacies of colonial racism and its contemporary manifestations, as such legacies are apparently on the way to being overcome by modernity’s inherent teleo-logic. it also discounts consideration of how political and (particularly) commercial influences might contribute to the reproduction of discourses of indigeneity and indigenous issues as inherently and inevitably problematic and intractable, in ways that limit rather than enable engagement with indigenous voices (fforde et al 2013, brough 1999). thirdly, the ‘evolutionary’ model proposed not only overrides the need to support arguments by reference to evidence, but also undermines the need for intervention, or political struggle. the indigenous public sphere is already established, and insofar as it is currently inadequate, such issues will be resolved both through a ‘logic of democratic equivalence’. ultimately, what is presented as an historical process is, in reality, a rather ahistorical (and entirely theoretical) set of claims that provide a ground for an asserted narrative of progress toward a dialogic, egalitarian ideal. for michael meadows (2005), by contrast, the indigenous public sphere is conceived as a space that derives from a lack of adequate representation in mainstream media, such that “many indigenous people are turning their backs on mainstream media and engaging in their own forms of cultural production” (2005: 37). in contrast to hartley and mckee’s claims regarding an increasingly representative indigenous public sphere that emerges through ‘dialogue’ between indigenous and non-indigenous australia, meadows proposes a conception of indigenous public spheres as cultural sites for specifically indigenous identities and voices to engage in dialogues that support the maintenance and expression of specific identities, and provide vehicles through which problematic representation in mainstream media can be contested: [r]ather than adopting the idea of a single, all-encompassing public sphere, we should think in terms of a series, of the existence of parallel and overlapping public spheres – spaces where participants with similar cultural backgrounds engage in activities of importance to them…although they develop in close proximity to – and with a great deal of influence from – mainstream society, they should be seen as discrete formations that exist in a unique context as the product of contestation with the mainstream public sphere’ (meadows 2005: 37-38; emphasis added) while meadows’s definition of the indigenous public sphere does not absolutely contradict that of hartley and mckee, it is nevertheless markedly different. as noted above, hartley and mckee refer to definitions of indigeneity being produced and circulated between indigenous and non-indigenous media spaces, particularly as the latter facilitate spaces for inclusive and egalitarian ‘dialogue’. meadows, on the other hand, conceives the indigenous public sphere as, vitally, both distinct from and a product of contest with the mainstream. it is conceived as a space of identity and position formation that allows participants ‘to formulate their own discursive styles and formulate their own positions that are then brought to the wider public sphere’ (2005: 38). for meadows, indigenous public spheres exist as resistant responses to the inadequacies of mainstream media representation that he has elsewhere documented (meadows 2001), as well as mechanisms for the persistence of indigenous self-determination through communicative practices. in contrast to hartley and mckee, meadows’s model is more attentive to the ways in which representations of indigeneity are structured by historical and contemporary disparities of power. arguably, however, meadows’s claim that indigenous public spheres can exist as ‘discrete formations’ despite the influence of mainstream society nevertheless presents an overly idealised model for understanding cultural ‘resistance’. folker hanusch has suggested that ‘indigenous media are almost always a response to the dominant culture’s media treatment of indigenous people’ (2013, 953). if this statement exaggerates the point, it nevertheless calls for recognition of how discussions of indigenous identities and issues are often structured by, as well as resistant to, the limiting terms of non-indigenous discourses, debates and representations. this is to say that, in both their direct exposure to mainstream media discourses and through the indirect influence these exert on wider public debates and australasian journal of information systems latimore, nolan, simons & khan 2017, vol 21, research on indigenous ict reassembling the indigenous public sphere 5 public policy, indigenous people are subject to (as well as being subjects who produce) various cultural discourses and texts that provide (variable) resources for understanding indigenous identities, issues and concerns. in this respect, although meadows’s emphasis on the agency of indigenous people and the degree to which indigenous media practices embody cultural selfdetermination is important, the characterisation of indigenous public spheres as ‘discrete formations’ may be overly idealised. arguably, too, both accounts are ‘idealised’ in another sense, insofar as the adoption of the public sphere discourse itself tends, by implication, to position the problem of representation through an image of speech and dialogue, centring on the indigenous and non-indigenous participants themselves (although, it should be acknowledged, meadows’s wider focus on community media as resources that facilitate speech goes beyond this). one consequence of this is a continued tendency to approach ‘publics’ as either a body of people and/or as a range of representations, without further consideration of the broader relations in which both people and representations are implicated. this, it may be argued, is in part a legacy of the adoption of the habermasian concept of the public sphere, in which the ‘space’ of media tends to be metaphorically positioned as an expanded version of eighteenth century coffee houses and salons. to counter this tendency, in the next section we focus on the more materialist approach to public representation suggested by bruno latour’s conception of ‘making things public’ (2005), and how this has been suggestively taken up by recent media scholarship that focuses on public representation as produced by, and implicated in, a broader field of relationships. 3 public assemblages recent work concerned to develop an expanded horizon for understanding and analysing publics has been informed by the take-up of actor network theory (ant) (hawkins 2013, anderson 2013, anderson and kreiss 2013). rather than focus simply on human actors and what they say (or, alternatively, political-economic structures and what they do), ant analyses draw on a wider consideration of how publics are formed and facilitated through the dynamic relations between a wide array of material ‘things’. this approach is broader than seeing ‘media’ as mere secondary conduits for ‘speech’ or economic and political ‘power’. it focuses on how media provide a heading for a complex and dynamic assemblage of human and nonhuman things. this is to say that media, as they have historically existed and transformed, exist as a pluralistic network of institutions, practitioners, technologies, discourses, practices, formats and heterogeneous relationships (with media owners, advertisers, state funders, regulators, sources and users) that collectively influence its various forms at any given time, and over its history. as anderson and kreiss argue, understanding the nature of public life involves taking into account the ways in which such assemblages have been historically constructed and transformed through the interactions (to draw on latour’s terminology) of the various actors and actants that constitute them: [t]o understand power and reform social institutions, and even uproot them, requires attention not just to theories of participation, deliberation and the public sphere, but the socio-technical engineering of democratic publics and the cultural presuppositions that guide it. (anderson and kreiss 2013: 380) this theoretical approach to understanding publics invites a potentially wide array of analytical approaches, ranging from a macro-level focus on the various relations that contribute to public-making, to micro-level consideration of the impacts of particular developments within this field of relations – such as, for example, shifts in legal and regulatory frameworks, or the impact of a particular media platform and its affordances. the important contribution it makes, however, is its reconnection of considerations of democratic communication (and the democratic shortcomings and inequalities that structure the field) with the material relations that structure it. returning to public debates surrounding public representation of indigenous people and issues, we can now consider the intersections and divergences of various cultural and material influences. these include the continued impact of news values that tend to focus on ‘race’ as a site of social conflict (bell 1997); focus on and construct ‘events’; and tend to australasian journal of information systems latimore, nolan, simons & khan 2017, vol 21, research on indigenous ict reassembling the indigenous public sphere 6 preference the perspective of social authorities (particularly government sources) (mccallum et al 2012). it is also important to consider the particular influence of institutional goals and the degree to which outlets are structured by different forms of market appeal that result in distinct editorial positions and practices. in this respect, market dynamics, and the degree to which these facilitate and constrain the market strategies of particular actors and institutions, serve as forces that materially enable and constrain public life, alongside state initiatives such as investment in media content initiatives (public and community media) and content regulation. alongside such influences, the discursive practices of governmental and non-governmental sources, and the particular ways in which these work to create media agendas to further public and political agendas by appealing to media predilections, also play a constitutive role in shaping public agendas and representations. the representation of indigenous australia is also a matter of great interest to private actors that use public relations and sponsorship to further their interests and manage their reputations, as well as exert influence over the nature of policies relating to indigenous australia such as those on land use and land rights. in addition to these political, economic and media influences, a range of broader factors serves as material influences on representation. what technologies exist and are used to produce and circulate representations, and what are their affordances and constraints? to what extent have powerful and influential figures (such as journalists) acquired a cultural competence in reporting on indigenous australia? (mccallum, waller and meadows, 2012; waller 2012). what understanding do they bring to that practice? what training have they undertaken that directly relates to such reporting, if any? to what extent do both journalists and audience-users rely on, and reproduce, inherited cultural tropes and narratives that serve to inform the stories they tell and how they are told? how far do such tropes and stories diverge and/or converge to provide a coherent and shared, or contested and multi-vocal range of representations? such questions take us beyond theoretical debates surrounding representation and the degree to which media support a ‘genuine’ public sphere, which is the point. ultimately, a consideration of public discourse must consider how speech, representation and dialogue are facilitated and constrained by the relations that make “public(s)” and how they are constituted through them. in his case study analysis of the changing nature of news production and news products in the philadelphia region, anderson (2013) has referred to the relationship between ‘assemblages’ of material relationships, and the shifting ‘ecologies’ of news institutions and practices. consideration of how such assemblages and ecologies contribute to ‘public-making’ does not negate a consideration of questions such as ‘who speaks?’ and ‘who is heard?’. rather, it is to focus on how such issues are imbricated in a particular, historical and mutable set of relationships. having provided this brief overview of this field of relationships as it relates to indigenous representation, we turn now to consider some recent work that has considered the role that the emergence of digital platforms has contributed to it. 4 media ecologies and the problem of listening in the context of indigenous representation in australia, the problem of understanding the implications of networked digital media platforms for voice and listening has been taken up by the recent work of dreher, mccallum and waller (2016), which focuses on how the shifting media landscape provides opportunities for indigenous people to gain a voice in policy debates. they also look at whether such opportunities enable actual political participation, and what potential constraints might limit opportunities for such voices to be heard (2016: 25). a number of key observations, concepts and prior findings inform this work. observationally, they begin by noting the considerable proliferation of indigenous outlets, practices and voices that have emerged in recent years. many of these have been facilitated by the networked, participatory affordances of social media platforms, and their capacities of connection and networking content across local spheres, thereby creating new possibilities of connecting indigenous voices, perspectives and debates. such initiatives include blogs such as celeste liddle’s ‘rantings of an aboriginal feminist’; the deadly bloggers site australasian journal of information systems latimore, nolan, simons & khan 2017, vol 21, research on indigenous ict reassembling the indigenous public sphere 7 (www.deadlybloggers.com), which is a network/site that provides access to a range of indigenous-authored blogs; and @indigenousx, a twitter-based initiative run by indigenous culture advocate luke pearson, which operates as an outlet for debate and dissemination of diverse perspectives through a rotation of indigenous commentators and topics on a weekly basis (sweet et al 2013). @indigenousx has also developed a strong relationship with the mainstream media outlet guardian australia, enabling both increased publicity and, at key moments, opportunities to reach a considerably wider audience. the @indigenousx relationship with guardian australia is also an example of shifts in the ecology of australian mainstream news. launched in 2013, guardian australia is a web-only local edition of the well-established british-based news organisation. it is a distinct new editorial presence in the highly concentrated australian media marketplace, previously dominated by news corporation and fairfax media. notably, guardian australia has sought to challenge the australian as the predominant site through which indigenous-related events and issues are covered. such developments are positioned by dreher, mccallum and waller as significant additions to what they characterise as australia’s media ecology: indigenous participatory media in australia belong to a media ecology that incorporates mainstream media reporting about indigenous people and issues; a government-funded and supported indigenous communication sector, including the national indigenous television network nitv; community newspapers, radio and online platforms, organizational and advocacy publications, as well as social media hubs, networks and citizen media producers. (2016: 28) transformations in this media ecology have, undoubtedly, been shaped in part by opportunities afforded by networked digital media technologies and social media platforms which have expanded the potential for indigenous voices to gain a degree of media participation. however, dreher, mccallum and waller are concerned to investigate the degree to which increased media participation translates into increased political participation, an issue that brings us to the conceptual aspect of their work. an important issue is whether the emergence of digital platforms and practices of indigenous participatory media have enabled such voices to contribute to shaping policy agendas. to consider this, dreher, mccallum and waller draw on the work of carpentier (2011), who makes a distinction between a ‘minimalist’ definition of participation centred on access to infrastructures of media production and dissemination, and a ‘maximalist’ definition which refers to participation in processes of political decision-making. previous work focused on indigenous policy found that policymaking practices were, to a large extent, ‘mediatised’: that is, were increasingly targeted towards, and responsive to, the agendas, logics and routines of mainstream media outlets (mccallum et al 2012, waller 2013). one consequence of this, following carpentier’s model, is that whether indigenous participation is ‘minimalist’ or ‘maximalist’ depends to a large degree on how far a wide range of indigenous voices gain access to mainstream media. this, as dreher, mccallum and waller note, is consistent with a conceptual shift in recent years from focusing on a politics of voice and who has opportunities to speak, to the problem of ‘listening’, which centres on the equally, if not more crucial, problem of which voices are heard in mediatised policy debates. this, they argue, ‘shifts some of the focus and responsibility for democratic outcomes from marginalized voices and onto the conventions, institutions and practices that shape who and what can be heard in an increasingly mediatized policy-making environment’ (dreher, mccallum and waller 2016: 28). in addressing the problem of how far maximalist political participation (policy influence) has been enhanced by the emergence of participatory media, their findings are mixed. if such influence is to be achieved, it relies on how far mainstream media serve to ‘amplify’ a range of indigenous perspectives and concerns, as opposed to a narrow range of established, dominant indigenous voices in discussions of events and issues, and in the establishment of media agendas. to a large extent, it remains the case that mainstream media act as a constraint on the capacity for indigenous voices to be heard, as they tend to continue to focus on a narrow range of australasian journal of information systems latimore, nolan, simons & khan 2017, vol 21, research on indigenous ict reassembling the indigenous public sphere 8 favoured and well-resourced voices that dominate media coverage and news agenda-setting. nevertheless, they also found that on occasions, particular examples of activist campaigning were able to shift agendas in ways that – at least temporarily – enabled new voices and perspectives to enter into the dominant public sphere. a striking example of this was provided by @indigenousx’s intervention in the mediatised public debate over constitutional recognition for australia’s first nations people. the debate had largely been dominated by the well-funded ‘recognise’ campaign, a campaign that had received large-scale political and business support, and had been bolstered by the predominance of select indigenous political figures such as noel pearson and warren mundine. despite this, a range of indigenous voices had raised concerns that the campaign for constitutional recognition served as a basis for narrowing agendas surrounding indigenous rights to land and genuine treaty negotiations. after recognise released a poll  which suggested 75 per cent of all australians and 87 per cent of indigenous australians supported recognition, @indigenousx conducted an alternative poll, which found that a much smaller proportion of indigenous people supported the constitutional reform agenda. @indigenousx in collaboration with indigenous blogger celeste liddle then reported on the findings of its poll on the indigenousx website and distributed the story via the @indigenousx twitter account. the question of which poll was more accurately representative was less significant than the effect of the intervention by @indigenousx. because these disparate polls highlighted newsworthy aspects of conflict and controversy, and appealed to the established media interest in reporting on quantitative measures of public opinion, this poll soon became a news story that was amplified by a range of traditional and non-traditional outlets, including independent news website new matilda, the guardian australia and the more mainstream abc radio. in so doing, the intervention opened a space for constitutional recognition to be rendered as a matter of public controversy and debate, thus temporarily creating media space for a wider range of voices and perspectives to gain representation. what is particularly useful about dreher, mccallum and waller’s framework is that it enables consideration of how both enabling potentials and constraints on participation are linked to an abiding, but also changing, media ecology. this media ecology shapes, and is shaped by, shifting technological, institutional and political relationships. these relationships represent an inherited historical and contemporary assemblage that gives rise to forms of patterned representation, persistent inequalities of access, and presents continued structural constraints on the capacity for a wide range of indigenous voices to be amplified and heard. at the same time, transformations in these relationships, with the emergence of new players and infrastructures, may serve to disrupt and potentially to transform these relationships. this may, on particular occasions and/or over time, work to reconfigure these relationships in ways that may enable such constraints to be, at least partially, overcome. 5 wakul app:  facilitating  political participation through amplification  as outlined above, the problem of ‘voice’, ‘listening’ and ‘participation’ relates not only to a willingness for media outlets to host indigenous voices and to listen to and amplify existing perspectives, but to how relatively limited or enhanced infrastructures of voice and listening are produced by technologically, institutionally and politically structured relationships through which processes of ‘public representation’ occur. over the last decade, there have been two major developments that, while not entirely restructuring the field, have nevertheless contributed to a certain restructuring of those relationships. the first of these is the emergence of new institutional actors that have served to challenge and, at least partially, transform the space of indigenous representation. these have included new social media platforms and businesses, such as facebook and twitter, and new media players such as guardian australia. social media platforms have enabled an augmentation of the space of community media representation, through the provision of new infrastructures for the hosting and dissemination of voice, while new media players have sought to include distinctive forms of indigenous representation as part of their editorial and market strategies. the second development, australasian journal of information systems latimore, nolan, simons & khan 2017, vol 21, research on indigenous ict reassembling the indigenous public sphere 9 related to the first, is a proliferation of indigenous media initiatives that have taken up and used new platforms to produce representations, have used digital platforms to network indigenous voices (both through social media initiatives such as @indigenousx and digital broadcast media such as nitv), and have used such platforms to gain increased exposure in mainstream media. in both cases, the emergence of digital networked communication tools and infrastructures has been vital to these developments. as we have seen, however, how far such developments produce ‘maximal’ levels of participation is dependent on how they interrelate with other aspects of an existing network of relationships. the simple existence of digital tools and platforms and new media initiatives does not, for example, result in a wholesale transformation in ‘news values’ that structure mainstream news. nor does it automatically follow that the potential of digital communication platforms to provide mainstream journalists with access to a wider range of sources means that they will access them, or even know about them. journalists working to deadlines are usually time poor, which contributes to a tendency to rely on and reproduce already-existing story frames and established, easily accessed sources. such factors both condition and delimit the potential for new developments to exert a transformative influence. bearing this in mind, the centre for advancing journalism (caj) at the university of melbourne has developed a piece of infrastructure that seeks to make a modest intervention into this network of relationships. the genesis of this intervention began with the collation of a preliminary dataset of 150 indigenous news sources with a view to amplifying their output to infiltrate australia’s dominant media systems, following the successful example of @indigenousx. this dataset includes well-established legacy print media and broadcast entities, and also new digitallyenabled media, including social media channels that are breaking news and distributing original content across diverse geographic locations and demographics. this collation was part of a larger civic impact of journalism project which aimed  to  provide empirical evidence on the impact of  journalism  in  australia, and how this is changing (simons, et al 2015). in this context, the construction of the database was preliminary work for a case-study based examination of the degree to which these developments in the field were allowing new forms of indigenous media participation, and the implications of this for civic impact, particularly on policy processes. the methodology for the broader civic impact campaign conceives of ‘impact’ as a generic term, incorporating both reach and engagement, but going  beyond  those ideas to encompass the reactions of institutional players, including governments and policy makers, and also how users are applying news and information in their personal and civic lives (simons et al 2015, clark 2010). the preliminary dataset, and the amount of indigenous media participation that quickly became apparent, exposed the need for a more advanced methodology to collecting data for such a diverse range of news sources, and the potential to create a valuable public resource.   as noted above, for the potential impact of indigenous use of new media to be maximized, amplification is key (mccallum and waller 2015).  yet, as with the challenges faced by news media more broadly, indigenous outlets are adversely affected by audience fragmentation which makes it harder to gather a large audience for any one piece of media content (simons 2010, simons 2013, given 2003). the challenge faced by indigenous news media is to find a ‘centralised’ or ‘aggregated’ audience, and also ‘bridge’ indigenous perspectives and stories into mainstream news media more directly – in order to achieve the  amplification  of its intervention, on which maximalist participation in terms of impact and influence relies.  with this challenge in mind, in 2016 caj began to develop a prototype system designed to facilitate the amplification of first nations voices and improve representation both within the connected counter-public of indigenous public spheres and the broader mainstream public sphere. the wakul  app (meaning ‘one’ in traditional gathang language) is an advanced piece of software infrastructure principally tailored to the needs of connecting indigenous communities through the technology they are using. the design of wakul  app has been implemented using a spiral development process, which enables us to keep indigenous advisors engaged in the system design throughout the development period. this consultative approach will continue in future stages of development and has informed our decision to release the source code under an open australasian journal of information systems latimore, nolan, simons & khan 2017, vol 21, research on indigenous ict reassembling the indigenous public sphere 10 source licence. in 2018 the github repositories containing the code will be made public to promote code re-use and collaboration and adaptation from the public.   there are two key components to  the wakul  app: a data layer and the mobile application. the data  layer stores rich and fine-grained data of indigenous news outlets, news content and social media content and uses a combination of different  database management systems to cater for different user needs.  the result is a document-oriented and heterogeneous archive of news outlets. each  news source is attributed with unique geospatial  data, such as: broadcast reach of radio stations; circulation area of print media; and  analyses  of location-enabled social networks. this geospatial data can then be combined with aggregated census statistics enabling, for example, the ability to determine which specific issues are affecting searchrelevant indigenous communities and how far such issues are receiving attention in local, regional or national media.  in addition, through a process known as ‘twitter harvesting’, the wakul app can provide data for social network source and sentiment analysis.   wakul  app allows users, communities, researchers, the media and the broader public to meaningfully interact with the collected data. the app contains a rich feature set including news feeds, a first nations interactive map and a news source directory. the news feed updates with local stories as the user seeks information on geographic locations. the news feed also highlights trending topics within indigenous networks, determined via real-time analysis of news articles and social media activity. the emphasis of the algorithm is placed on discovering what issues matter to indigenous users and communities rather than what is being distributed by mainstream media outlets. a directory of the indigenous news outlets is also accessible, enabling users to retrieve the details and contact information, promoting direct communication between indigenous users and the wider media. caj is currently in discussion with media organisations about ways in which the wakul app can be used within the news industry to amplify indigenous perspectives and improve representation. these might include allowing mainstream journalists to “spot” issues being discussed by indigenous australians, assisting editorial executives to identify and recruit indigenous citizen-activists and promising journalists, and/or allowing the production of an aggregated page of indigenous news and views, to be hosted on a mainstream news media website. in developing the wakul app, our concerns are twofold. firstly, and most directly, the project is informed by the caj mission of ‘advancing journalism’ by addressing an area of identified need. as discussed above, a common finding of work addressing indigenous media representation is that a relatively narrow range of stories is linked to the dominance of selected, established voices in mainstream media. in recent years, however, transformations in media technologies and markets have facilitated new initiatives within the actor networks through which indigenous public representation occurs. in light of this, the wakul app provides a tool via which a trend toward amplifying voice via digital networks might be furthered through the networked tracking and aggregation of indigenous debates. it will also, importantly, provide a tool through which non-indigenous users, including professional journalists, can gain a closer understanding of indigenous concerns and debates, as well as identify and potentially access actors that are articulating such concerns. secondly, we are concerned to understand the impact of this initiative, by understanding how the wakul app interfaces with, and intervenes in, the field of relationships in which it is taken up and used. in approaching this problem, however, we are both concerned to avoid a simplistic technological determinism, and mindful of the problems of ‘effects’ approaches. furthermore, we do not assume that diagnoses of a failure to ‘listen’ in mainstream media are likely to be resolved by the development of an app. for example, while the wakul app may assist time-pressed journalists in efficiently tracking indigenous debates and in identifying a wider range of voices, the tendency among journalists to rely on a narrow range of sources does not stem only from a lack of access to other voices. rather, as this paper has suggested, to consider the agential power exerted by the wakul app as one thing within a wider network, it is necessary to consider this in relation to an existing and shifting field of relations, and the wider range of agencies that constitute the field of indigenous representation. australasian journal of information systems latimore, nolan, simons & khan 2017, vol 21, research on indigenous ict reassembling the indigenous public sphere 11 6 conclusion this article has sought to provide a reconsideration, and ultimately a reconceptualization, of framings of indigenous media representation. to this end, it identified two distinct, and largely contrasting, approaches. on one hand, hartley and mckee’s model of the indigenous public sphere as a product of all representations (indigenous and non-indigenous) was considered. a strength of this model is that it enables consideration of not only how indigenous discussions and debates were shaped by a wider public sphere, but also the significant role played by indigenous voices in this broader space. a weakness, however, was that a framing of the public sphere as a progressively inclusive ‘dialogue’ provided little basis for engaging with both how and why indigenous voices continue to be largely marginalized in mainstream media debate, and representations of indigenous people and issues continue to position both as sites of deficit and intractable problems. by contrast, meadows approach defines the indigenous public sphere as multiple, discrete sites of self-determining communication, through which indigenous people exchange perspectives and maintain voice. these alternative spaces are seen as existing as resistant responses to the perceived inadequacies and injustices of mainstream media representation. while a particular strength of this model was its emphasis on self-determination and resistance, a potential shortcoming was that its model of discrete formations may overemphasise the degree to which sites and practices of indigenous communication are entirely discrete – either from each other, or from the influence of wider processes of public representation. in addition, in both cases it was noted that an effect of the habermasian model tended to reproduce a model of public representation as conversations writ large, a framing that tends by implication to exclude a wider consideration of the broader relations through which public representation occurs, and publics are thus ‘made’ on an ongoing basis. an alternative approach, underpinned by ant, focuses instead on the network of material (political) relations through which forms of public representation occur, and the effects for both representations of indigenous australia and the degree to which indigenous voices are amplified. here a review of recent work highlights how the digital initiatives both offer potentialities for increased amplification of indigenous voices through their disrupting the settled nature of relations, but such potentials are also subject to constraint. ‘listening’, we argue, is not simply a matter of altering the consciousness of journalists or of white australia, but rather is a longer game that involves effecting transformations in an assemblage of relations that has been historically structured through inequalities of power, the violence of settlement, as well as indigenous resistance and self-determination. while such a complex undertaking is unlikely to be radically affected by any ‘magic bullet’, our discussion of the wakul app, as an initiative aiming to make a modest contribution to re-assemblage, is informed by our understandings that the existing relations that support public representation are products of history, albeit one that is deeply embedded in this public assemblage. nevertheless, historically formed relations are subject to the effects of historical change, and we can therefore both expect and hope for their transformation. references anderson, a. a., brossard, d, scheufele, d. a., xenos, m. a. and ladwig, p. (2014). “the ‘nasty effect’: online incivility and risk perceptions of emerging technologies.” journal of computer-mediated communication 19: 373–387. doi: 10.1111/jcc4.12009 anderson, c.w. (2013) rebuilding the news: metropolitan journalism in the digital age. philadelphia: temple university press. anderson, c.w. & kreiss, d. (2013) “black boxes as capacities for and constraints on action: electoral politics, journalism, and devices of representation”. qual sociol 36:365– 382 bell, p. (1997), “news values, race and ‘the hanson debate’ in australian media”, asia pacific media educator. 2:38–47. australasian journal of information systems latimore, nolan, simons & khan 2017, vol 21, research on indigenous ict reassembling the indigenous public sphere 12 broersma, m. & graham, t. (2012) “social media as beat: tweets as a news source during the 2010 british and dutch elections”, journalism practice, 6(3), pp.403-419. brough, m. (1999), “a lost cause? representations of aboriginal and torres strait islander health in australian newspapers”, australian journal of communication, 26(2):89– 98. bruns, a. (2008) “life beyond the public sphere: towards a networked model for political deliberation.” information polity 13(1-2):65-79. bruns, a. (2008) “the active audience: transforming journalism from gatekeeping to gatewatching”, making online news: the ethnography of new media production, paterson, c. and domingo, d. (eds.) 171-184. new york: peter lang. carlson, b. & frazer, r. (2015) “it’s like going to a cemetery and lighting a candle”, alternative: an international journal of indigenous peoples, 11(3):211–224. castells, m. (2009) communication power. oxford; new york: oxford university press. carpentier, n. (2011) “the concept of participation: if they have access and interact do they really participate?” cm-časopis za upravljanje komuniciranjem, 6(21):13–36. cottle, s. (2000) “introduction media research and ethnic minorities: mapping the field” in s. cottle (ed) ethnic minorities and the media.1-30. maidenhead; philadelphia: open university press. cunneen, c. (1992) ‘racial stereotypes kill’, in k. eggerking and d. plater (eds), signposts: a guide to reporting aboriginal, torres strait islander and ethnic affairs, sydney: australian centre for independent journalism, pp. 24–26. davis, m. & g. williams. (2015) everything you need to know about the referendum to recognise indigenous australians, new south publishing: university of nsw, sydney. dreher, t. mccallum, k & waller, l. (2016) “indigenous voices and mediatized policy-making in the digital age”, information, communication & society. 19(1):23-39 due, c. & riggs, d. (2011) representations of indigenous australians in the mainstream news media, brisbane: post pressed. dunlop, t. (2013) the new front page: new media and the rise of the audience. scribe publications. dutton, w. h. (2008) “the fifth estate emerging through the network of networks.” prometheus. 27(1):1-15. eley, g. (1992) “nations, publics, and political cultures: placing habermas in the nineteenth century”, habermas and the public sphere. calhoun. c (ed.) 289-339. cambridge, mass and london, england: mit press. ewart, j. (1997) “the scabsuckers: regional journalists’ representation of indigenous australians”, asia pacific media educator, 3:108–17. fforde, c. bamblett, l. lovett, r. gorringe, s. & fogarty, b. (2013) “discourse, deficit and identity: aboriginality, the race paradigm and the language of representation in contemporary australia”, media international australia, 149:162–73. flew, t. (2009) “democracy, participation and convergent media: case studies in contemporary online news journalism in australia.” communication, politics & culture 42(2):87-109. fraser, n. (1992) “rethinking the public sphere: a contribution to the critique of actually existing democracy”, habermas and the public sphere. 109-42. cambridge, mass and london, england: mit press. australasian journal of information systems latimore, nolan, simons & khan 2017, vol 21, research on indigenous ict reassembling the indigenous public sphere 13 gans, h.j. (1999) “deciding what’s news”, news: a reader, tumbler, h. (ed.) 235-246. new york: oxford university press. gitlin, t. (1998) “public sphere or public sphericules?” , media, ritual and identity, liebes,t. curren j, katz, e, (eds.) 168-174. new york: routledge. given, j (2003) turning off the television: broadcasting’s uncertain future, unsw press. gladwell, m. & shirky, c. (2011) “from innovation to revolution: do social media make protests possible?”, foreign affairs 90(2):153. golding, p. philip, r. & courtney, e. (1979) making the news. longman publishing group. graham, t, & wright, s. (2015) “a tale of two stories from ‘below the line’ comment fields at the guardian.” the international journal of press/politics. 20(3): 317-338. habermas, j. (1989) the structural transformation of the public sphere: an inquiry into a category of bourgeois society. trans. burger,t & lawrence, f. cambridge, mass.: mit press. hanusch, f. (2013) “charting a theoretical framework for examining indigenous journalism culture”, media international australia. 149:82–91. hartley, j. (1996) popular reality: journalism, modernity, popular culture. london: arnold. hartley, j. & mckee, a. (2000) the indigenous public sphere: the reporting and reception of aboriginal issues in the australian media. oxford: oxford university press. hawkins, g. (2013). “enacting public value on the abc’s q & a.” media international australia, 146:82-92. jakobowicz, a. goodall, h. martin, j.a. mitchell, t. randall, l. & seneviratne, k. (1994) racism, ethnicity and the media, sydney: allen & unwin. jericho, g. (2012) the rise of the fifth estate: social media and blogging in australian politics. scribe publications. latour, b. (2005) “from realpolitik to dingpolitik, or how to make things public.” making things public: atmospheres of democracy. latour, b. & weibel, p. (eds.) cambridge, mass.: mit press. lewis, s.c. (2012) “the tension between professional control and open participation: journalism and its boundaries.” information, communication & society 15(6):836-866. lumby, b. (2011) “cyber-indigeneity: urban indigenous identity on facebook”, australian journal of indigenous education, 39 (supplement):68–75. meadows, m. (2001) voices in the wilderness: images of aboriginal people in the australian media, westport, ct: greenwood press. meadows, m. (2005) “journalism and indigenous public spheres”, pacific journalism review, 11(1): 36–41. mccallum, k. (2011) “journalism and indigenous health policy”, australian aboriginal studies 2: 21–31. mccallum, k. waller, l. & meadows, m. (2012) “raising the volume: indigenous voices in news media and policy”, media international australia, 142:101–11. mccallum, k. waller, l. meadows, m. dunne-breen, m. & reid, h. (2012) the media and indigenous policy: how news media reporting and mediatized practice impact on indigenous policy, canberra: university of canberra. mccallum, k. &waller, l. (2013) “the intervention of media power in indigenous policymaking”, media international australia 149:139-149. australasian journal of information systems latimore, nolan, simons & khan 2017, vol 21, research on indigenous ict reassembling the indigenous public sphere 14 mccallum, k. & waller, l. (in press, 2017) the dynamics of news media and indigenous policy in australia. bristol: intellect. mcnair ingenuity research (2014) media usage amongst aboriginal and torres strait islander people. available at: http://mcnair.com.au/wp-content/uploads/mediausage-amongst-aboriginal-and-torres-strait-islander-people-mcnai....pdf [accessed 20 may 2016]. newman, n. dutton, w.h. & blank, g. (2012) “social media in the changing ecology of news: the fourth and fifth estates in britain”, international journal of internet science, 7(1):6-22. paradies, y. harris, r. & anderson, i. (2008) the impact of racism on indigenous health in australia and aotearoa: towards a research agenda, discussion paper no. 4, darwin: cooperative research centre for aboriginal health. shirky, c. (2011) "the political power of social media: technology, the public sphere, and political change.” foreign affairs jan/feb simons, m. (2010) the content makers – understanding the media in australia. penguin, melbourne. simons, m. (2013) journalism at the crossroads. scribe, melbourne. simons, m. tiffen, r., hendrie, d. carson, a. sullivan, h. muller, d. & mcnair, b. (2016) “understanding the civic impact of journalism: a realistic evaluation perspective”, journalism studies. 1-15. shoemaker, p.j. vos, t.p. & reese, s.d., (2009) “journalists as gatekeepers”, the handbook of journalism studies wahl-jorgensen, k. & hanitach, t. (eds.) 73-88 routledge: new york & london. stoneham, m.j. goodman, j. & daube, m. (2014) “the portrayal of indigenous health in selected australian media”, the international indigenous policy journal, 5(1), available at http://ir.lib.uwo.ca/iipj/vol5/iss1/5 [accessed 14 september 2016]. sweet, m. pearson, l. & dudgeon, p. (2013) “@indigenousx: a case study of community-led innovation in digital media”, media international australia. 149:104–111. thompson, j. b. (1995) the media and modernity: a social theory of media. cambridge: polity press, 1995. waller, l. (2012) “journalists, ‘remote’ indigenous australia and cultural competence”, the media and indigenous policy: how news media reporting and mediatized practice impact on indigenous policy mccallum, k. meadows, m. waller, l. dunne-breen, m. & reid, h (eds), 51-59, canberra: university of canberra. waller, l. dreher, t. & mccallum, k. (2015) “the listening key: unlocking the democratic potential of indigenous participatory media”, media international australia, 154: 57– 66. waters, r. tindall n. & morton, t.s. (2010) “media catching and the journalist–public relations practitioner relationship: how social media are changing the practice of media relations.” journal of public relations research 22(3):241-264. wright, s. & street, j. (2007) “democracy, deliberation and design: the case of online discussion forums”, new media & society 9(5):849-869. wright, s. (2012) “politics as usual? revolution, normalization and a new agenda for online deliberation”, new media & society, 14(2):244-261. copyright: © 2017 latimore, nolan, simons & khan. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems latimore, nolan, simons & khan 2017, vol 21, research on indigenous ict reassembling the indigenous public sphere 15 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems lane, tiwari & alam 2016, vol 20, research on rural and remote ict case study of broadband in rural australia 1 the supply and use of broadband in rural australia: an explanatory case study of the western downs region michael lane school of management and enterprise university of southern queensland michael.lane@usq.edu.au sanjib tiwari school of management and enterprise university of southern queensland khorshed alam school of commerce university of southern queensland abstract this research seeks to provide an understanding of broadband infrastructure supply and its relationship with household broadband use and satisfaction in rural australia through an explanatory case study of the western downs region (wdr). the broadband ecosystem provided a comprehensive framework for examining broadband infrastructure supply and household use and satisfaction with broadband services. publicly available data was used to map the coverage of broadband access technologies in wdr. a large scale survey assessed household use and satisfaction with broadband services available in wdr. our findings indicate that rural regions such as wdr are highly reliant on wireless broadband which is more variable in reliability and less affordable comparative to wired broadband. our findings also indicate that household dissatisfaction with wireless broadband services is particularly evident in remote and outer regional areas. this suggests that the lack of reliable and affordable wireless broadband services with adequate data quotas are a real barrier to rural communities such as the wdr actively participating in a digital future. this study provided a number of important contributions. the broadband ecosystem provided a comprehensive framework for understanding the complex problem of broadband access in rural australia by analysing two units of analysis, broadband infrastructure (supply) and broadband use and satisfaction (demand). broadband infrastructure is an important and under-researched area in information systems. future government policy needs to ensure that access to reliable and high speed broadband services is part of its universal service obligation so that the current shortcomings in broadband infrastructure in rural australia will be prioritised and addressed. we believe this will require a commitment from future governments to facilitate both public and private investment in broadband infrastructure in rural australia. keywords: broadband infrastructure; broadband services; rural australia; technology use and satisfaction 1 introduction the main objective of this paper is to provide a detailed understanding of the supply, use, and satisfaction with broadband services in rural australia through an in-depth case study of the western downs region. furthermore, this paper seeks to show that the supply of broadband infrastructure in terms of availability, speed and reliability in rural australia influences household use and satisfaction with broadband services in rural australia. broadband access has become increasingly critical for day-to-day life and business activities. broadband is now widely considered to be one of the key enablers and drivers for social and economic growth (budde 2014; hudson 2013; touré 2014). time-series analyses of australian bureau of statistics (abs) data shows that internet subscriptions in australia overall have increased substantially from 60% of households in 2005-6 to 86% of households in 2014-15, australasian journal of information systems lane, tiwari & alam 2016, vol 20, research on rural and remote ict case study of broadband in rural australia 2 although household adoption and use of the internet by rural australia is still lagging behind urban australia (abs 2016). rural australia could be directly affected if many rural communities are excluded from the potential benefits of accessing and utilising broadband infrastructure as a universal service. inadequate broadband infrastructure and lower levels of services in rural areas are ongoing problems (alam & imran 2015; bandias & ram vemuri 2005; park et al. 2015). according to the 2014–15 regional telecommunications review (rtr), individuals and businesses in rural communities have limited options in choosing a broadband provider; they typically have lower internet connection speeds, smaller data quotas, limited mobile broadband availability and generally higher prices compared to their urban counterparts (rtirc 2015). this ultimately impacts on effective use of digital services in rural australia. while numerous previous studies have been conducted on adoption and use of broadband these have tended to focus on impacts at the macro level or national level such as socio-economic development, developing broadband diffusion strategies and policy, and providing some technical solutions for rural broadband (islam, selvadurai & town 2008; middleton & chang 2008; stern et al. 2004). there has been a lack of in-depth empirical studies aimed specifically at rural australia that have examined the broadband ecosystem (kelly & rossotto 2012) by investigating supply, demand/use and satisfaction at the micro level (hill, burgan & troshani 2011; hollifield & donnermeyer 2003). although the rtr study shows that there is a lack of broadband infrastructure in australian rural communities, a significant gap in the literature exists in terms of a detailed study of the relationship between supply of broadband infrastructure and household use and satisfaction with broadband services in rural australia (whitacre, strover & gallardo 2015). in general, it is argued there has been a lack of empirical studies on digital infrastructure, including broadband infrastructure. tilson, lyytinen and sørensen (2010) refer to digital infrastructure research as the orphan of information systems research and an important but under-researched area. therefore, the aim of this research is to investigate the problems associated with broadband access by focusing on supply of broadband infrastructure and household use and satisfaction with broadband services in rural australia in order to provide a more complete picture and understanding of the problem. the use of broadband is based on its availability and capability to meet requirements of the users—in other words, highly dependent upon both the supply of, and the demand for, that technology (middleton & chang 2008). in this paper some of the key results of an in-depth case study of the broadband infrastructure (supply) and household use and satisfaction with broadband services in rural australia—specifically the western downs region—are presented. the paper is structured as follows. first, we present the theoretical and conceptual framework—the broadband ecosystem—which provides the context and, subsequently, guided the data collection and analysis in this study. then we provide a background to adoption and use of broadband in australia and the digital divide that exists between urban and rural australia due to the lack of reliable and affordable broadband infrastructure in rural australia. the methodological approach used in this study is described and justified. thereafter, we provide a snapshot of the current broadband infrastructure (supply-side) available in the western downs region. this helps to more fully explain some of the key findings regarding household use (demand-side) and satisfaction with broadband services in the western downs region. the results of the data analysis presented in this paper sought to address following research questions: rq1: what broadband access technologies are available and in what locations in the wdr? rq2: what is the extent of use of broadband services by households in the wdr? rq3: what is the extent of satisfaction of households with the broadband services available in the wdr? subsequently, some conclusions and implications regarding theory and practice are provided, drawing on the key findings reported in this paper. australasian journal of information systems lane, tiwari & alam 2016, vol 20, research on rural and remote ict case study of broadband in rural australia 3 2 background literature for study 2.1 broadband ecosystem model the world bank ecosystem framework for deployment and adoption of broadband provided the theoretical and conceptual framework for understanding the supply of broadband infrastructure, demand (adoption) and use of broadband (cohen & levintbal 1990; world bank 2012) (see figure 1 broadband ecosystem adapted from world bank). the broadband ecosystem includes the broadband networks, broadband-compatible devices, content and applications (fcc 2010). the broadband ecosystem is characterised by wired and wireless broadband network services and computing devices for accessing broadband and a key range of stakeholders, namely consumers, business and government which, in turn, influence the adoption and use of applications and content. in this research we focus on the relationship between the broadband infrastructure — networks that deliver broadband as a service — and the household use and satisfaction with broadband in rural australia. in other words, the supply and use elements of the broadband ecosystem. the broadband ecosystem framework has been adapted to the context of this study by including satisfaction with broadband services as a variable that influences use. the overall impact of the broadband ecosystem is framed within rural communities. figure 1: broadband ecosystem (source: adapted from world bank broadband tool kit 2012; (kelly & rossotto 2012)) broadband internet can be represented as an ecosystem consisting of supply and demand components which are equally important to the successful expansion and delivery of broadband infrastructure and services (kelly & rossotto 2012). the supply side is concerned with the push—the building and delivering broadband network infrastructure which encourages increased demand for broadband services. the demand side is concerned with australasian journal of information systems lane, tiwari & alam 2016, vol 20, research on rural and remote ict case study of broadband in rural australia 4 using the available infrastructure to access digital services which ultimately encourages the push of latest broadband access technologies. there is evidence that providing faster, cheaper broadband internet services with increased data quotas increases customer satisfaction and the likelihood of more engagement with broadband access technologies. higher infrastructure quality is expected to lead to better broadband internet performances and higher user satisfaction and use, leading to positive benefits for individuals and rural communities. this should result in greater use of broadband and ensuing benefits in various sectors, including education, health, communication, work and other digital services; and also drive demand for higher speed broadband services (kerpez et al. 2014; seymour & naidoo 2013). the use of broadband has been demonstrated to have a positive impact on regional, state and national gdp in countries which have embraced broadband and have the digital literacy to utilise broadband to drive technology led innovation (gruber, hätönen & koutroumpis 2014; whitacre, gallardo & strover 2014). when supply, demand and use are co-ordinated, the benefits of broadband in rural communities will be significant—with improved satisfaction with broadband services resulting in significant realised benefits in terms of economic, human and social capital. 2.2 availability of broadband in rural australia the supply of broadband in rural areas of developed countries has been much lower compared to urban areas (abs 2013a; dobson, jackson & gengatharen 2013; dwivedi, alsudairi & irani 2010; grubesic 2012; kawade & nekovee 2012; prieger 2013; rajabiun & middleton 2013). a combination of poor telecommunications infrastructure coverage, low population density, inadequate regulation, and a focus by telecommunications companies on high-cost technologies designed for urban markets makes internet connectivity in many rural areas a complex and costly proposition (arai, naganuma & satake 2012; grubesic 2012; hill, troshani & burgan 2014; kawade & nekovee 2012). due to inadequate information and communication infrastructure and facilities in rural and remote areas, the digital divide between rural communities and urban communities is not improving and could be widening (park et al. 2015; willis & tranter 2006). table 1 highlights that wireless broadband access technology is a finite shared broadband service which is highly susceptible to interference in contrast to wired broadband, in particular, fibre optic broadband which has abundant capacity and is highly reliable (lehr & chapin 2009). this is particularly pertinent for rural australia, where many smaller towns have limited or no access to wired broadband such as adsl/adsl2+ and are highly reliant on wireless broadband, in particular, mobile broadband. network capability wired broadband networks wireless broadband networks capacity abundant scarce topology point-to-point broadcast reliability reliable unreliable mobility fixed mobile layering effective inefficient legend: wired broadband (fibre optic, adsl), wireless broadband (3g/4g mobile networks, satellite broadband) table 1: key persistent differences in wired broadband versus wireless broadband (source: adapted from lehr & chaplin 2009) typically, rural australia lags behind urban australia in terms of broadband infrastructure which is often highly variable in terms of speed, reliability, and is often under serviced or not serviced in large parts of rural australia in comparison to urban australia (see table 2) (dobson, jackson & gengatharen 2013; hill, burgan & troshani 2011; hill, troshani & burgan 2014; middleton & park 2014; rtirc 2015). australasian journal of information systems lane, tiwari & alam 2016, vol 20, research on rural and remote ict case study of broadband in rural australia 5 broadband access technology download speed upload speed availability inner regional outer regional remote adsl1 8 mbps 384 kbps available available limited availability adsl2+ 20 mbps 820 kbps available limited availability not available cable (hsd) 100 mbps 2 mbps limited availability not available not available nbn fibre 100 mbps 40 mbps limited availability not available not available 3g 7.2 mbps 2 mbps available available available 3g hspa+ 42 mbps 22 mbps available limited availability limited available 4glte 150 mbps 50 mbps limited availability limited availability not available 4glte advanced 300 mbps 150 mbps limited availability limited availability not available nbn fixed wireless 50 mbps 20 mbps limited availability limited availability not available nbn satellite wireless 25 mbps 1 mbps limited availability limited availability available table 2: download and upload speeds for broadband access technologies in australia (source: this research) table 3 suggests that the growth in mobile broadband presents a dilemma for rural australia as the exponential growth in demand for data is outstripping the capacity of the current mobile network infrastructure available in much of rural australia. this further exacerbates problems faced in rural communities because of the availability of broadband access technologies. in particular, fixed line broadband access technologies—which provide users with much larger and cheaper data quotas comparatively than mobile broadband technologies such as 3g/4g and satellite broadband—are limited to the larger towns in rural australia (as evidenced in table 2). further investigation of the www.rfnsa.com.au and www.adsl2exchanges.com.au websites revealed a similar pattern of wired and wireless broadband coverage for a number of other rural regions in australia (see our detailed analysis presented in the results and discussion section). the australian government and key policymakers are well aware that rural communities have a demand and requirement for high quality broadband and telecommunication services (middleton & park 2014). the australian federal government has also held a number of consultations with businesses and the general public and received many recommendations to improve broadband availability and access. as a result, many policies have been developed to invest in more broadband infrastructure in rural communities. however, to date, none of these policies have been fully implemented or have lagged in the roll out—the nbn being a good case in point. hence, major efforts to overcome the digital divide that exists between urban and rural communities have so far been unsuccessful (given 2008; middleton & park 2014). more recently, starting from a very low base in 2006, we have seen an exponential increase in the adoption of mobile broadband services which contributed significantly to the overall adoption of broadband services across australia. figure 2 shows how mobile broadband services have overtaken fixed line adsl services since 2010; while dial-up narrowband access has become almost negligible after being accountable for almost 50% of internet access technologies back in 2006 (abs 2013b, 2015). figure 2: number of internet subscribers by internet access connection technology (source: (abs 2015) australasian journal of information systems lane, tiwari & alam 2016, vol 20, research on rural and remote ict case study of broadband in rural australia 6 figure 2: number of internet subscribers by internet access connection technology (source: (abs 2015) jun-12 dec-12 jun-13 dec-13 jun-14 dec-14 jun-15 tb tb tb tb tb tb tb broadband fixed line (b) 389 130 526 472 629 964 823,421 963,429 1,112,379 1,349,975 wireless (c) 25 301 28 196 27 232 37,426 32,731 34,339 38,673 total broadband 414 431 554 668 657 196 860,847 996,160 1,146,718 1,388,648 total volume of data downloaded (d) 414 537 554 771 657 262 (a) data reported for the three month period prior to the reference date. (b) fixed line includes dsl, cable, fibre and other fixed line broadband. (c) wireless includes satellite, fixed wireless, mobile wireless via a datacard, dongle, usb modem or tablet sim card and other wireless broadband. excludes data downloaded via mobile handsets which is reported in the mobile handset chapter. (d) dial-up volume is not shown separately, but is included in the total. (abs 2015) table 3: volume of data downloaded by type of access connection (a), isps more than 1,000 subscribers (source: abs 2015) however, this rapid adoption of mobile broadband services does not tell the true story of the relationship between the data explosion and use of different broadband access technologies (see table 3). currently as a broadband access technology, mobile broadband is prohibitive in terms of speed and cost of data usage comparative to wired broadband access technologies such adsl1/adsl2+ and fibre to the home or node (chaleyer-kynaston 2012; wallsten & mallahan 2010). a comparison of mobile broadband and adsl services based on cost and data quotas is provided in appendix 1. it also must be noted with reference to table 3 that mobile phone subscriptions, which are usually individual subscriptions, cannot be equally compared to adsl broadband subscriptions—which are usually household subscriptions. moreover, data quotas are currently quite restrictive for mobile phones compared to data quotas available for adsl broadband services. hence, growth in data usage has predominately been with fixed line broadband services, although this is likely to change over time as mobile data services decrease in cost while at the same time increasing substantially in terms of data download/upload speeds with the roll out of 4g mobile networks. 0 10 20 30 40 50 60 sep-06 dec-07 dec-08 dec-09 dec-10 dec-11 dec-12 dec-13 dec-14 jun-15 % in te rn e t co n n e ct io n proportion of internet subscribers by connection type, for isps with more than 1,000 subscribers dial-up cable, fibre, satellite and other dsl mobile and fixed wireless australasian journal of information systems lane, tiwari & alam 2016, vol 20, research on rural and remote ict case study of broadband in rural australia 7 table 3 further highlights some critical differences between wired broadband and wireless broadband in terms of data usage over the period from june 2012 through to june 2015 referencing abs national data. the rapid improvements in wireless broadband access technology cannot alone meet exponential growth in demand for data that will occur in the future. despite the rapid rise of mobile broadband access technologies this has tended to disadvantage rural australia. the rural australia has a high reliance on mobile broadband networks for broadband services particular in remote and outer regional areas and the inherent shortcomings of mobile broadband as a wireless broadband access technology are notable as highlighted in table 1. there is also a reluctance of mobile phone operators to invest in upgrading mobile broadband infrastructure in rural australia given the lower return on investment because of low population densities. in the next section we provide an overview of the case study location— the western downs region—where we examined the broadband supply (infrastructure) and the household demand (adoption), use and satisfaction with broadband services. 2.3 case study site: western downs region (wdr) the wdr is located in south-east queensland (se qld) 200 kilometres north west of brisbane, the capital city of queensland, australia. the economy of the wdr has continued to grow at an extensive pace in the last five years to 2014/15, where the annual growth rate of 6.3% is significantly higher than the annual overall growth rate for the state of queensland (2.5%) over the same period of time (wdrc 2015). western downs regional council (wdrc) is the 20th largest council in terms of area in queensland. the western downs has six main towns, namely, dalby, chinchilla, jandowae, miles, tara and wandoan—with another 14 smaller towns and villages scattered throughout the region. figure 3: showing study area western downs region, queensland, australia (source: advance western downs 2015, davis n.d., western downs region wikipedia 2015) the wdr is rich in coal and natural gas reserves which underpin the power generation industry for se qld. the region is also well-known as a rich agricultural area for broad acre and intensive cropping, dairy products and meat production (wdr 2015). however, the region still experiences poor telecommunication services and infrastructure (especially mobile) in many locations, with many black spots and poor coverage for both voice and data (internet) communications. the australian standard geographical classification (asgc) defines the relative remoteness of rural communities in australia (abs 2014). since this study is examining a rural region of australia, we used the asgc classification to classify rural communities (towns) in the wdr australasian journal of information systems lane, tiwari & alam 2016, vol 20, research on rural and remote ict case study of broadband in rural australia 8 as one of three: (1) inner regional australia, (2) outer regional australia and (3) remote australia (see table 5). 3 research method and approach this research used an explanatory case study approach to evaluate the supply, demand and use of broadband internet in rural communities in the western downs region (yin 2014; stake 1995). an explanatory case study guided by the broadband ecosystem framework (kelly & rossotto 2012) allowed us to gain an in-depth of understanding of the phenomena of interest, the relationship of broadband infrastructure supply with household use and satisfaction with broadband services in rural australia. the two units of analysis in this study are (1) broadband infrastructure supply in rural communities, and (2) household use of broadband services in rural communities of wdr. multiple sources of quantitative data obtained for these two units of analysis allowed the researchers to capture a rich picture and deep understanding of the phenomena of interest in the explanatory case study of wdr. to determine and assess broadband infrastructure available in the wdr (to answer research question 1) archival data from the radio frequency national site archive (rfnsa) (www.rfnsa.com.au) (rfnsa 2016) and publicly contributed data from the social media website www.adsl2exchanges.com.au (adsl2-exchanges 2015) were used. this allowed us to construct table 5 which provides an overview of the wired and wireless broadband infrastructure availability across the 20 towns in the wdr. we also constructed similar tables for rural regions in other parts of rural australia to provide a comparison of the available broadband infrastructure. this study also conducted mobile broadband speed tests to check the reliability and availability of services using the opensignal mobile app (www.opensignal.com) (opensignal 2014) for the three mobile network services (telstra, optus and vodafone) providers in the wdr. this research also conducted a large scale survey of households in the wdr during 2014 to collect quantitative data about household adoption and use of broadband internet. the survey instrument was developed from a number of previous studies on broadband technology adoption and use which have used instruments proven to be valid and reliable (straub, boudreau & gefen 2004). a number of key definitions were provided in the survey, including the definition of broadband provided by oecd and nbn (nbn 2014; oecd 2014) so that survey participants had a clear idea what was meant by the term ‘broadband’. the survey instrument was pre tested with a number of academics with extensive experience in survey research design. the survey was then piloted with a number of households before the main survey data collection was undertaken. the survey instrument is available on request from the authors. the survey package was delivered in person to randomly selected households using a stratified sampling method to ensure that 1500 surveys were distributed to a sample representative of the population distribution of the towns and localities in the wdr. a total of 301 usable survey responses were obtained (see table 4). a response rate of 20% was achieved, which is comparable to similar previous studies of broadband adoption and use by households (brown & venkatesh 2005; dwivedi, alsudairi & irani 2010; dwivedi, choudrie & brinkman 2006). table 4 shows the geographical distribution of household survey responses across the wdr. australasian journal of information systems lane, tiwari & alam 2016, vol 20, research on rural and remote ict case study of broadband in rural australia 9 location in wdr household responses percentage combined district percentage live in dalby 125 42% dalby district combined 49% nearest to dalby 21 7% live in chinchilla 29 10% chinchilla district combined 13% nearest to chinchilla 10 3% live in tara 10 3% tara district combined 7% nearest to tara 12 4% live in miles 13 4% miles district combined 8% nearest to miles 10 3% live in wandoan 14 5% wandoan district combined 6% nearest to wandoan 4 1% live in jandowae 15 5% jandowae district combined 7% nearest to jandowae 7 2% other place (please specify) 31 10% other places in wdr 10% 301 100% table 4: geographical distribution of respondent households in wdr survey proportionately, the respondent households are reasonably representative of the population distribution across the towns and surrounding districts in the study area of the wdr. the survey data was analysed using the statistical data analysis software spss. the interpretation of the results of the survey data in relation to research questions 2 and 3 was also informed by the understanding of the available broadband infrastructure gained from the audit of broadband infrastructure in the wdr which was addressed by research question 1. 4 results and discussion first we present our findings for rq1: an assessment of the supply of broadband services in the wdr. then we present some key findings from the household broadband adoption and use survey which addressed the main aims of rq2: the use of broadband by households. the main aim of rq3 was to determine the level of household satisfaction with the broadband services available to them in the wdr. 4.1 access and availability of broadband infrastructure table 5 provides an overview of broadband infrastructure availability in the wdr. australasian journal of information systems lane, tiwari & alam 2016, vol 20, research on rural and remote ict case study of broadband in rural australia 10 pop pstn adsl1 adsl2+ fibre optic backbone 3g 4g asgc dalby 16000 y y (* t) y(*±) y (c) y (all) y (all) inner regional chinchilla 7200 y y (t) y (t) y (c) y (all) y (all) outer regional miles 1450 y y (t) y (t) y (c) y (all) y (all) outer regional tara 1100 y y (t) y (±t) n y (t) n outer regional jandowae 1000 y y (t) y (t) n y (to) y (to) outer regional wandoan 400 y y (t) y (t) n y (to) y (t) remote bell 300 y y (t) proposed (t) n y (to) y(o) outer regional meandarra 250 y y (t) y (t) n y(t) y (t) remote condamine 135 y y (t) y (t) n y (t) n outer regional dulacca 120 y n (±t) proposed (t) n y (o) y(o) remote warra 120 y n (±t) proposed (t) n y (to) y(to) outer regional kaimkillenbun 100 y n (±t) proposed (t) n n^ n outer regional drillham 70 y n n n y(t) y(t) outer regional brigalow 40 y n (±t) proposed (t) n n^ n outer regional kogan 40 y n (±t) proposed (t) n y(t) y(t) outer regional moonie 40 y n (±t) proposed (t) n n^ n remote jimbour 40 y n (±t) proposed (t) n n^ n outer regional glenmorgan 20 y n (±t) proposed (t) n n^ n remote macalister 15 y n (±t) proposed (t) n n^ n outer regional gulugaba 10 y n (±t) proposed (t) n n^ n remote legend y = available; n = not available; n^ = rely on signal from mobile phone tower in neighbouring town; c = commerical use only; t= telstra; o= optus; v=vodafone y(all)= telstra, optus & vodafone; to= telstra & optus; tv= telstra & vodafone, * = agile (internode) and chime (iinet); ± = telstra no ports available. [notes: satellite broadband coverage of entire region but only available to households and businesses in remote areas with no access to alternative broadband services radio networks proprietary radio networks cover the entire region but are not available to general public and are beyond the scope of this study table 5: mapping of digital infrastructure coverage in wdr (source: (adsl2-exchanges 2015; rfnsa 2015) public switched telephone network (pstn) coverage of the towns in the wdr is complete. pstn provides the underlying infrastructure for broadband adsl enabled telephone australasian journal of information systems lane, tiwari & alam 2016, vol 20, research on rural and remote ict case study of broadband in rural australia 11 exchanges. almost half of the larger towns (9 out of 20) in the wdr have adsl enabled exchanges and eight of those towns have access to adsl2+ broadband services. all adsl and adsl2+ broadband infrastructure in wdr is provided by telstra. two fibre optic backbone networks are provided by nextgen group (www.nextgengroup.com.au) and nexium telecommunications (www.nexium.net.au) a subsidiary of ergon, a state government electricity utility run through the three major towns in wdr. however these fibre optic backbone networks currently provide wholesale fibre network services which are affordable and accessible only to government entities, isps and businesses. the telstra 3g mobile network has coverage across the majority of towns towns in wdr, except for glenmorgan, gulugaba, kaimkillenbun and macalister. optus provides 3g mobile network services in eight towns, dalby, chinchilla, miles, jandowae, bell, wandoan, dulacca and warra. vodafone provides 3g mobile network services in only three towns, dalby, chinchilla and jandowae. telstra provides 4g services in 9 towns, dalby, chinchilla, miles, jandowae, warra, drilham, kogan, wandoan and meandarra. optus provides 4g services in 7 towns, dalby, chinchilla, miles, jandowae, bell, dulacca and warra. vodafone provides 4g services in the three largest towns, dalby, chinchilla and miles. when we look at the towns across the wdr by agsc classification for remoteness we can clearly see that the more remote towns and surrounding localities have poor broadband infrastructure coverage. some of the remote towns also do not have adsl enabled telephone exchanges and/or have limited or no 3g mobile network coverage (e.g. glenmorgan, gulugaba, kaimkillenbun and macalister). an analysis of the broadband infrastructure in other rural regions of queensland, tasmania and new south wales using the same methodology revealed a similar picture in terms of the availability of broadband infrastructure in rural australia. 4.2 wired and wireless broadband availability assessment selected rural regions in australia 4.2.1 queensland pop pstn adsl adsl2+ fibre optic backbone 3g 4g nbn wireless 4g asgc warwick 16,000 y y y n y (all) y (all) n inner regional stanthorpe 5,500 y y y n y (all) y (all) y outer regional allora 1,276 y y y n y(t) y(t) n inner regional killarney 911 y y y n y (all) y (all) y outer regional the falls 197 y n (±t) proposed (t) n n n y outer regional table 6: southern downs regional council (source: (adsl2-exchanges 2015; rfnsa 2015)) pop pstn adsl adsl2+ fibre optic backbone 3g 4g nbn wireless 4g asgc goondiwindi 6,397 y y y n y (all) y (all) proposed outer regional inglewood 1,069 y y y n y (all) y (all) n outer regional texas 1,159 y y y n y(to) y(to) proposed remote yelarbon 493 y y proposed (t) n y(to) proposed (to) proposed outer regional toobeah 218 y n (±t) proposed (t) n proposed proposed n remote bungunya 189 y n n n n n n remote table 7: goondiwindi regional council (source: (adsl2-exchanges 2015; rfnsa 2015)) australasian journal of information systems lane, tiwari & alam 2016, vol 20, research on rural and remote ict case study of broadband in rural australia 12 4.2.2 tasmania pop pstn adsl adsl2+ fibre optic backbone 3g 4g nbn wireless 4g asgc bagdad 1,266 y y y n y (all) y (all) proposed outer regional mangalore 521 y n n n y (t) y (t) proposed outer regional kempton 353 y y y n y(t) proposed (t) proposed outer regional table 8: southern midlands council (source: (adsl2-exchanges 2015; rfnsa 2015)) pop pstn adsl adsl2+ fibre optic backbone 3g 4g nbn wireless 4g asgc queenstown 1975 y y y n y (to) y (o) n remote strahan 824 y y y n y (t) y (t) proposed remote rosebery 922 y y y n y (to) y (o) n remote new south wales table 9: west coast council (source: (adsl2-exchanges 2015; rfnsa 2015)) 4.2.3 new south wales pop pstn adsl adsl2+ fibre optic backbone 3g 4g nbn wireless 4g asgc gilgandra 2,700 y y y n y (all) y (all) y outer regional tooraweenah 371 y y n n y (tv) y(v) n outer regional armatree 284 y n (±t) proposed (t) n y (t) n n outer regional table 10: gilgandra shire (source: (adsl2-exchanges 2015; rfnsa 2015)) pop pstn adsl adsl2+ fibre optic backbone 3g 4g nbn wireless 4g asgc narrabri 7,392 y y y n y (all) y (all) n outer regional wee waa 2,089 y y y n y(to) y(t) n outer regional baan baa 525 y n (±t) propose d (t) n y(to) y(t) n outer regional table 11: narrabri shire (source: (adsl2-exchanges 2015; rfnsa 2015)) legend y = available; n = not available; t= telstra; o= optus; v=vodafone y (all) = telstra, optus & vodafone; to= telstra & optus; ±t = telstra no ports [note: these data about broadband availability were captured during the study and may differ that a later time] 4.3 results of testing mobile network services (telstra, optus, vodafone) in wdr the selective testing of the mobile network services provided by telstra, optus and vodafone using the mobile app open signal www.opensignal.com on 26-27th february and 10th and 14th march 2014 (see table 12) showed that the three largest towns in wdr (dalby, chinchilla, miles) all experience good mobile network speed and reliability of service. in contrast, selective testing in smaller towns in the wdr (condamine, kogan, moonie, drilham) of mobile network services provided by telstra, optus and vodafone showed that speed and reliability of service varied from good, to poor, to non-existent depending on the coverage of mobile phone carrier and the time of day. australasian journal of information systems lane, tiwari & alam 2016, vol 20, research on rural and remote ict case study of broadband in rural australia 13 location date time service provider download upload network type dalby 26/02/14 7:15 am telstra 13.24 mbps 3.48mbps 4g vodafone 7.15 mbps 2.67 mbps 3g optus 1.82 mbps 484 kbps 3g 4:13 pm telstra 16.01 mbps 4.07 mbps 4g 27/02/14 1:47pm telstra 17.33 mbps 3.30 mbps 4g chinchilla 26/02/14 11:10 am telstra 28.1 mbps 2.9 mbps 4g vodafone 2.4 mbps 2 mbps 3g optus 677 kbps 409 kbps 3g miles 14/01/14 01:52pm optus 1.54 mbps 410 kbps 3g 14/03/14 08:22am telstra 19.52 mbps 17.7 mbps lte-4g vodafone 9.14 mbps 2.22 mbps hspa optus 1.52 mbps 420 kbps 3g condamine 26/02/14 12:12 pm telstra 9.46 mbps 769 kbps hspa kogan 26/02/14 3:45 pm telstra 5.18 mbps 627 kbps hspa 10/03/14 3:51pm telstra 9.89 mbps 1.419 mbps hspa moonie 10/03/14 8:39 am telstra 3.5 mbps 99 kbps 3g/ hspa (with signal booster) telstra n/a 105 kbps hspa (without signal booster) drillham 14/03/14 10:29 am telstra 2.53 mbps 476 kbps 3g optus 470 kbps 80 kbps 3g [note: these download and upload speeds were captured during the study which may be differ at a later point in time] table 12: selective testing using opensignal mobile app three carriers mobile networks in the wdr in summary, from the analysis of the data sources from the rfnsa website and the results of the mobile network services testing with the open signal mobile app we were able to determine availability and reliability of mobile broadband services in the wdr. the proprietary radio networks which have extensive coverage in the wdr are not listed in table 5 because these networks are dependent on the capability of specific organisations and individuals to set up and maintain the infrastructure to these radio networks. 4.4 discussion of household use and satisfaction with broadband services in wdr in this section we present and discuss the results of the household survey in the wdr in terms of the demographics of the respondent households and their use and satisfaction with the available broadband services. note, all of the respondent households in this survey were using the internet. 4.4.1 household use of broadband services to understand household use of broadband services in the wdr, survey respondents were asked the following questions. 1. does your household have access to the internet? (filter question) 2. how long has your household been using the internet? 3. what kind of internet connection is used in your household? 4. how often each week does your household use internet services? 5. how many members of your household use the internet? australasian journal of information systems lane, tiwari & alam 2016, vol 20, research on rural and remote ict case study of broadband in rural australia 14 6. where do members of your household access the internet? 7. which of these devices are used by households to access the internet? 8. what activities does your household use the internet for? now we discuss the survey results for each of these questions in turn below. three quarters of households have been using the internet for six or more years. interestingly, almost 40% of households started using the internet more than 16 years ago when the primary means of accessing the internet was via narrowband and a dialup modem. moreover, 60% of households started using the internet since adsl broadband internet services became readily available across australia from 2005 when telstra started to roll out adsl enabled telephone exchanges en masse across the country (fletcher 2009; islam, selvadurai & town 2008). more recently, the roll out of mobile broadband has also contributed to an increase in the adoption and use of the internet by households. the most widely used internet connection technologies in the wdr are 3g/4g mobile broadband (60%), adsl/adsl2+ broadband (48%) and satellite broadband (10%). the majority of households (76%) are accessing the internet on a daily basis ninety-eight percent of households are accessing the internet from their home, while almost 56% of households are accessing the internet from work; 27% from school, tafe or university; 19% from public places; and 10% from a library. these findings are consistent with acma (2015) survey data which shows that home remains the main location where household members go online. however, it may be that this will change over time as mobile broadband becomes increasingly available and persuasive. at 80%, laptops and notebooks are the most common devices used by households to access the internet. at 64%, smart phones are the second most common device used by households to access the internet. at 57%, desktop computers are the third most common device used by households to access the internet; while at 55%, tablets are the fourth most common device used by households to access the internet. to a lesser extent—at 15% and 13% respectively— gaming consoles and internet tv are the fifth and sixth most common ways to access the internet. here, internet tv is defined as an internet enabled tv which could connect through a home network to access the internet. these results are also somewhat aligned with the national level aggregated data (abs 2016) where most households accessed the internet through a desktop or laptop computer (94%), followed by mobile or smart phones (86%) and tablets (62%). the top ten internet based digital services used by respondent households in the wdr were as follows. email is most widely used (97%); and, not surprisingly, online banking and social media (facebook) also figure prominently. e-government services come in at number 10 (47%) with almost half of the respondent households using government internet based digital services. overall, the survey results from wdr households demonstrate that the use of internet services is similar to the results of a recent national level study of households conducted by abs (2016) where the most popular activities are banking, social networking, services, online shopping and entertainment. 4.4.2 user satisfaction with broadband services household satisfaction with their broadband service(s) was measured in terms of affordability and overall satisfaction with their current broadband service(s). figure 4 shows the respondent household beliefs regarding affordability of their broadband service(s). australasian journal of information systems lane, tiwari & alam 2016, vol 20, research on rural and remote ict case study of broadband in rural australia 15 figure 4: respondent household beliefs regarding affordability of broadband internet in the wdr n = 301 (source: this research) for the survey question: “my annual household income is enough to afford subscribing to broadband internet” seventy two percent of respondent households agree that their broadband internet service is affordable; while 14% of households disagree that their broadband internet service is affordable; and 15% of households are undecided. hence, a small but significant proportion of the sample population in this study felt that broadband services were expensive and this is most likely to be outer regional and remote locations in the wdr where only mobile broadband services are available. it should be noted that the wdr has lower socio-economic status than major urban areas in queensland such as toowoomba and brisbane (queensland government 2014). table 13 breaks down household perception of broadband affordability by the type of broadband service. here we see some interesting trends emerge. s tr o n g ly d is a g r e e d is a g r e e s o m e w h a t d is a g r e e n e it h e r a g r e e n o r d is a g r e e s o m e w h a t a g r e e a g r e e s tr o n g ly a g r e e t o ta l p e r c e n ta g e o f h o u s e h o ld s 3g/4g (mobile) 4% 6% 8% 11% 19% 35% 18% 181 60% adsl/adsl2+ 1% 1% 4% 11% 18% 40% 25% 145 48% satellite 3% 24% 10% 38% 24% 29 10% dial-up 40% 40% 20% 5 2% table 13: affordability for each type of broadband service (source: this research) broadband services are perceived to be affordable by the majority of the respondent households (3g/4g 72%; adsl/adsl2+ 83%; satellite 72%); whereas only a small number of households indicated that affordability of broadband services was an issue (3g/4g 18%; adsl/ads2+ 6%; satellite 3%). interestingly, there were a number of households that were undecided about the affordability of broadband services (3g/4g 11%; adsl/ads2+ 11%; satellite 24%). the number of households still using narrowband dial-up internet service is very low so it is difficult to comment on this other than to note that the use of this service has become almost non-existent and many isps no longer offer this type of service. further table 14 breaks down household perception of broadband affordability by the type of broadband service across remoteness classifications for wdr. 8 (3%) 12 (4%) 21 (7%) 44 (15%) 56 (19%) 103 (34%) 57 (19%) 0 20 40 60 80 100 120 strongly disagree disagree somewhat disagree neither agree nor disagree somewhat agree agree strongly agree australasian journal of information systems lane, tiwari & alam 2016, vol 20, research on rural and remote ict case study of broadband in rural australia 16 t y p e o f in te r n e t c o n n e c ti o n r e m o te n e s s c la s s if ic a ti o n s tr o n g ly d is a g r e e d is a g r e e s o m e w h a t d is a g r e e n e it h e r a g r e e n o r d is a g r e e s o m e w h a t a g r e e a g r e e s tr o n g ly a g r e e p e r c e n ta g e o f h o u s e h o ld s 3g/4g (mobile) inner regional 5.7% 5.7% 12.9% 14.3% 35.7% 25.7% 60% outer regional 6.7% 4.4% 7.8% 8.9% 24.4% 32.2% 15.6% remote 4.8% 9.5% 14.3% 14.3% 14.3% 42.9% adsl/adsl2+ inner regional 1.1% 4.5% 11.4% 21.6% 38.6% 22.7% 48% outer regional 2.0% 2.0% 10.2% 14.3% 40.8% 30.6% remote 12.5% 12.5% 12.5% 50.0% 12.5% satellite inner regional 100.0% 10% outer regional 12.5% 18.8% 43.8% 25.0% remote 9.1% 27.3% 36.4% 27.3% dial-up inner regional 50.0% 50.0% 2% outer regional 66.7% 33.3% remote table 14: affordability for each type of broadband service across remoteness classifications in wdr table 14 shows that remote households are more concerned about the affordability of broadband services compared to outer and inner regional households. households were asked whether they were satisfied with their current broadband service and plan. figure 5 shows the level of satisfaction with the current broadband service and plan used by the respondent households figure 5: household satisfaction with broadband services n = 301 (source: this research) interestingly, 32% of respondent households are dissatisfied with their current broadband service and plan, while 52% of the respondent households are satisfied with their broadband services and 15% of respondent households are non-committal about their satisfaction with their broadband services. table 15 breaks down the level of household satisfaction with broadband services by the type of broadband service. here we see some interesting trends emerge. australasian journal of information systems lane, tiwari & alam 2016, vol 20, research on rural and remote ict case study of broadband in rural australia 17 s tr o n g ly d is a g r e e d is a g r e e s o m e w h a t d is a g r e e n e it h e r a g r e e n o r d is a g r e e s o m e w h a t a g r e e a g r e e s tr o n g ly a g r e e t o ta l p e r c e n ta g e o f h o u s e h o ld s 3g/4g (mobile) 15% 8% 13% 17% 12% 22% 12% 181 60% adsl/adsl2+ 4% 6% 10% 17% 19% 27% 17% 145 48% satellite 21% 10% 17% 10% 17% 17% 7% 29 10% dial-up 20% 20% 60% 5 2% table 15: level of satisfaction with type of broadband service (source: this research) the level of satisfaction with adsl broadband services is much higher (63%) compared to 3g/4g mobile broadband services (46%) and satellite broadband services (41%). similarly, the level of dissatisfaction with adsl broadband services is much less (20%) compared to 3g/4g mobile broadband services (36%) and satellite broadband services (48%). this is not surprising given that adsl/adsl2+ broadband services are faster and cheaper when compared to mobile broadband services. this supports our previous findings in relation to the supply of broadband infrastructure that 3g/4g mobile broadband services and satellite broadband services are much more variable in terms of coverage, speed and reliability of service than fixed line adsl broadband services, hence, the lower levels of satisfaction. dialup narrowband services have such a low usage these days that we consider its impact in the wdr to be negligible. further table 16 breaks down the level of household satisfaction with broadband services by the type of broadband service across remoteness in wdr. t y p e o f in te r n e t c o n n e c ti o n r e m o te n e s s c la s s if ic a ti o n s tr o n g ly d is a g r e e d is a g r e e s o m e w h a t d is a g r e e n e it h e r a g r e e n o r d is a g r e e s o m e w h a t a g r e e a g r e e s tr o n g ly a g r e e p e r c e n ta g e o f h o u s e h o ld s 3g/4g (mobile) inner regional 12.9% 4.3% 7.1% 20.0% 14.3% 28.6% 12.9% 60% outer regional 13.3% 10.0% 18.9% 15.6% 11.1% 16.7% 14.4% remote 33.3% 9.5% 9.5% 14.3% 9.5% 23.8% adsl/adsl2+ inner regional 3.4% 6.8% 5.7% 18.2% 18.2% 29.5% 18.2% 48% outer regional 4.1% 2.0% 20.4% 16.3% 20.4% 20.4% 16.3% remote 12.5% 12.5% 25.0% 37.5% 12.5% satellite inner regional 100.0% 10% outer regional 25.0% 6.3% 18.8% 12.5% 25.0% 6.3% 6.3% remote 18.2% 18.2% 18.2% 9.1% 9.1% 18.2% 9.1% dial-up inner regional 100.0% 2% outer regional 33.3% 33.3% 33.3% remote table 16: level of satisfaction with type of broadband service across remoteness in wdr (source: this research) table 16 shows that remote households and to a lesser degree outer regional households have higher levels of dissatisfaction with broadband services than inner regional households. we acknowledge the satisfaction of households with their broadband service is based on the perceptions of the major decision maker—the household survey respondent in terms of the type of broadband service that is available to them—and that is a limitation of this study. however, we believe that most rural households, even if they do not have access to a fast and reliable broadband service, are well aware—through their network of relatives, friends and work colleagues—that much better wired and wireless broadband services are available in australasian journal of information systems lane, tiwari & alam 2016, vol 20, research on rural and remote ict case study of broadband in rural australia 18 larger population centres in the wdr and in major urban centres throughout queensland and australia. this may be the reason for users’ dissatisfaction with their current internet services. the cost of broadband services does not appear to be a major cause of household dissatisfaction with broadband services in wdr. however in the remote and outer regions of wdr in many instances only mobile broadband is available which is much more expensive in comparison to wired broadband services such as adsl1/adsl2 (see comparison of mobile broadband services versus adsl1/adsl2+ on price and data quotas in appendix 1). 5 conclusion and implications this research has made a number of significant theoretical and practical contributions to existing knowledge. historically, information systems research has missed opportunities to make a significant theoretical and practical contribution by adopting a limited perspective, improving performance of the organisation as a unit of analysis while ignoring the important challenges facing society (clarke 2015). in our adapted broadband ecosystem model we examined the important relationship between supply of broadband infrastructure and household use and satisfaction with broadband services which is likely to influence the perceived benefits of broadband services in rural australia. this research also addressed another gap in information systems research—the lack of empirical research on broadband infrastructure. tilson, lyytinen and sørensen (2010) referred to digital infrastructure research as the orphan of information systems research and an under researched and important area of research. this research has made a number of significant practical contributions which can inform government policy by identifying that availability, reliability and affordability shortcomings of broadband infrastructure in outer regional and remote regions is impacting household use and satisfaction with broadband services in rural australia. this paper presented some key findings regarding the relationship between the supply of broadband access technologies and household use and satisfaction with broadband services in wdr. mobile broadband is highly variable in terms of speed and reliability across the region and is much more restrictive in terms of data quotas than adsl—which also makes it a much more expensive broadband service. adsl broadband is only available in 9 of the 20 towns in the wdr. our findings suggest that the variability and cost of wireless (mobile, satellite) broadband in wdr would appear to be contributing to household dissatisfaction with broadband services particular in remote and outer regional locations. our findings identified the limitations in the supply of broadband infrastructure in the remote and outer regions, and that household dissatisfaction with current mobile broadband services is clearly evident in these locations. to lesser extent given the reliance of remote and outer regions on mobile broadband, affordability of broadband is also an issue for households given the lower socio-economic status of much of rural australia including wdr. hence, there would also appear to be a digital divide particularly between the remote and outer regions of wdr and inner regions and urban areas of australian. overall our analysis of broadband infrastructure indicates that a lack of reliable and affordable broadband services due to rural australia’s reliance on wireless broadband could be a limiting factor for rural regions such as wdr being able to successfully transition to the digital economy. we believe that this current situation also exists in much of rural australia which suffers from inadequate provision of reliable broadband infrastructure as evidenced by the analysis of the supply of broadband infrastructure presented in this paper. this situation is also detrimental to the short term and long term growth of economic, human and social capital in rural regions such as the wdr. the key findings of this paper, we believe, have significant implications for national, state and local government in terms of broadband policy. we argue that government and policy makers should consider including high speed broadband access as part of the universal service obligation (uso) to ensure that rural and regional australia is not disadvantaged in a digital future. clearly in rural australia, wireless broadband networks (mobile broadband, broadband satellite) are the dominant broadband access technology on which most rural communities are highly reliant. this study identified poor network coverage australasian journal of information systems lane, tiwari & alam 2016, vol 20, research on rural and remote ict case study of broadband in rural australia 19 and highly variable quality of service for these wireless broadband networks in rural australia, suggesting that a digital divide still exists between much of rural australia and urban australia. future government policy needs to address the lack of government investment in the backbone broadband infrastructure in rural australia which is needed to underpin high speed wireless broadband in rural australia. finally, future thought and consideration needs to be given to correcting the current failure of telecommunication markets to address the shortcomings of wireless broadband networks in rural australia. we believe this market failure could be addressed in part through subsidies, including access to broadband as part of the universal service obligation which currently only considers access to a fixed telephone service. acknowledgements we would like to acknowledge the assistance and support that we received from craig tunley, the senior economic development officer at western downs regional council and the western downs regional council to conduct this research. we would also like to acknowledge the support and guidance we received from professor julie cotter, the director of the australian centre for sustainable business and development to conduct this research. finally, we wish to acknowledge the academic staff at university of southern queensland who assisted us with the pre-test of the survey instrument developed for this research. references abs 2013a, 8153.0 internet activity, australia, june 2013, 08 oct 2013, abs, canberra, 1 december 2014, 8135, <http://www.abs.gov.au/ausstats/abs@.nsf/lookup/8153.0main+features1june% 202013?opendocument>. abs 2013b, '8153.0 internet activity, australia', viewed 20th september 2013, <http://www.abs.gov.au/ausstats/abs@.nsf/detailspage/8153.0june%202013?ope ndocument>. abs 2014, australian statistical geography standard (asgs), australian bureau of statistics, viewed 15 december 2015, <http://www.abs.gov.au/websitedbs/d3310114.nsf/home/australian+statistical+geog raphy+standard+%28asgs%29>. abs 2015, type of access connection, australian bureau of statistics, viewed 20 october 2015, <http://www.abs.gov.au/ausstats/abs@.nsf/lookup/8153.0main+features1june% 202015?opendocument>. abs 2016, household use of information technology, australia, 2014-15, australian bureau of statistics, canberra, <http://www.abs.gov.au/ausstats/abs@.nsf/mf/8146.0>. acma 2015, communications report 2013–14 series report 1—australians’ digital lives australian communications and media authority, viewed 20 may 2015, <http://www.acma.gov.au/~/media/research%20and%20analysis/research/pdf/aus tralians%20digital%20livesfinal%20pdf.pdf>. adsl2-exchanges 2015, adsl2 exchanges, viewed 25 october 2015, <http://www.adsl2exchanges.com.au/>. alam, k & imran, s 2015, 'the digital divide and social inclusion among refugee migrants: a case in regional australia', information technology & people, vol. 28, no. 2, pp. 344-65. arai, y, naganuma, s & satake, y 2012, 'local government broadband policies for areas with limited internet access. an analysis based on survey data from japan', networks and communication studies, vol. 26, no. 3-4, pp. 251-74. awd 2015, our region, advance western downs, viewed 15 december 2015, <http://www.advancewesterndowns.com.au/our-region/>. australasian journal of information systems lane, tiwari & alam 2016, vol 20, research on rural and remote ict case study of broadband in rural australia 20 bandias, s & ram vemuri, s 2005, 'telecommunications infrastructure facilitating sustainable development of rural and remote communities in northern australia', telecommunications policy, vol. 29, no. 2, pp. 237-49. brown, sa & venkatesh, v 2005, 'model of adoption of technology in households: a baseline model test and extension incorporating household life cycle', mis quarterly, vol. 29, no. 3, pp. 399-426. budde, p 2014, 'global buddecomm’s international broadband and trans-sector activities', viewed 20th october 2014, <http://www.budde.com.au/>. chaleyer-kynaston, m 2012, wi-fi vs. wireless broadband, abc open news, viewed 5 march 2016, <https://open.abc.net.au/explore/10378>. clarke, r 2015, 'not only horses wear blinkers: the missing perspectives in is research', keynote paper in proceedings of the australasian conference on information systems adelaide, australia, 2 – 4 december 2015. cohen, wm & levintbal, da 1990, 'absorptive capacity: a new perspective on learning and innovation', administrate science quarterly, vol. 35, no. 1, pp. 128-52. davis, t n.d., queensland australia map queensland map australia, viewed 10 december 2015, <http://australiaresorts.us/cat13/queensland-australia-map.html> dobson, p, jackson, p & gengatharen, d 2013, 'explaining broadband adoption in rural australia: modes of reflexivity and the morphogenetic approach', mis quarterly, vol. 37, no. 3, pp. 965-91. dwivedi, yk, choudrie, j & brinkman, w-p 2006, 'development of a survey instrument to examine consumer adoption of broadband', industrial management & data systems, vol. 106, no. 5, pp. 700-18. dwivedi, yk, alsudairi, ma & irani, z 2010, 'explaining factors influencing the consumer adoption of broadband', international journal of business information systems, vol. 5, no. 4, pp. 393-417. federal communications commission 2010, connecting america: the national broadband plan, federal communications commission, washington dc, <https://www.fcc.gov/general/national-broadband-plan>. fletcher, p 2009, wired brown land: telstra's battle for broadband, unsw press. given, j 2008, 'australia's broadband: how big is the problem?', media international australia, vol. 127, no. 1, pp. 6-10. gruber, h, hätönen, j & koutroumpis, p 2014, 'broadband access in the eu: an assessment of future economic benefits', telecommunications policy, vol. 38, no. 11, pp. 1046-58. grubesic, th 2012, 'the us national broadband map: data limitations and implications', telecommunications policy, vol. 36, no. 2, pp. 113-26. hill, sr, burgan, b & troshani, i 2011, 'understanding broadband adoption in rural australia', industrial management & data systems, vol. 111, no. 7, pp. 1087-104. hill, sr, troshani, i & burgan, b 2014, 'broadband adoption in regional and urban households', journal of computer information systems, vol. 54, no. 3, pp. 57-66. hollifield, c & donnermeyer, jf 2003, 'creating demand: influencing information technology diffusion in rural communities', government information quarterly, vol. 20, no. 2, pp. 135-50. hudson, he 2013, 'beyond infrastructure: broadband for development in remote and indigenous regions', journal of rural and community development, vol. 8, no. 2, pp. 44-61. australasian journal of information systems lane, tiwari & alam 2016, vol 20, research on rural and remote ict case study of broadband in rural australia 21 islam, ar, selvadurai, n & town, g 2008, 'wireless broadband technologies for regional and rural australia: a last-mile perspective', telecommunications journal of australia, vol. 58, no. 2-3, pp. 28.1-.18. kawade, s & nekovee, m 2012, 'is wireless broadband provision to rural communities in tv whitespaces viable? a uk case study and analysis', in proceedings of the ieee international symposium on dynamic spectrum access networks (dyspan), ieee, bellevue, wa, usa, pp. 461-6, 16-19 oct. 2012 kelly, t & rossotto, cm 2012, broadband strategies handbook, world bank publications, washington, d.c. kerpez, k, cioffi, j, ginis, g, goldburg, m, galli, s & silverman, p 2014, 'software-defined access networks', communications magazine, ieee, vol. 52, no. 9, pp. 152-9. lehr, w & chapin, j 2009, 'rethinking wireless broadband platforms', in proceedings of the wireless technologies: enabling innovation and growth georgetown center for business and public policy, washington dc, april 2009. middleton, c & chang, s 2008, 'the adoption of broadband internet in australia and canada', in yk dwivedi, et al. (eds), handbook of research on global diffusion of broadband data transmission, igi global, london, vol. 1, pp. 820-42. middleton, c & park, s 2014, 'waiting for the national broadband network: challenges of connectivity in regional australia', in proceedings of the 20th its biennial conference econstor, rio de janeiro, brazil, 30 nov. 03 dec. 2014. national broadband network 2014, broadband, nbn co, viewed 01 february 2014, <http://www.nbnco.com.au/glossary.html>. oecd 2014, broadband access, organisation for economic co-operation and development, viewed 01 february 2014, <https://data.oecd.org/broadband/wireless-mobilebroadband-subscriptions.htm>. opensignal 2014, open signal, viewed 10 december 2013, <http://opensignal.com/>. park, s, freeman, j, middleton, c, allen, m, eckermann, r & everson, r 2015, 'the multilayers of digital exclusion in rural australia', in proceedings of the 48th hawaii international conference on system sciences (hicss) ieee, hawaii pp. 3631-40, january 5th 8th, 2015. prieger, je 2013, 'the broadband digital divide and the economic benefits of mobile broadband for rural areas', telecommunications policy, vol. 37, no. 6, pp. 483-502. queensland government 2014, queensland regional profiles, queensland government, viewed 20 january 2016, <http://statistics.qgso.qld.gov.au/qld-regional-profiles>. rajabiun, r & middleton, ca 2013, 'multilevel governance and broadband infrastructure development: evidence from canada', telecommunications policy, vol. 37, no. 9, pp. 702-14. rfnsa 2016, radio frequency national site archive, viewed 25 march 2016, <http://www.rfnsa.com.au/nsa/index.cgi>. rtirc 2015, regional telecommunications review 2015 , australian government, viewed 15 december 2015, <http://www.rtirc.gov.au/wpcontent/uploads/sites/2/2015/10/rtirc-independent-committee-review-2015final-low-res-version-for-website.pdf>. seymour, lf & naidoo, m 2013, 'the usage and impact of broadband: a south african household analysis', electronic journal of information systems evaluation, vol. 16, no. 2, pp. 134-47. australasian journal of information systems lane, tiwari & alam 2016, vol 20, research on rural and remote ict case study of broadband in rural australia 22 stern, se, gregor, s, martin, ma, goode, s & rolfe, j 2004, 'a classification tree analysis of broadband adoption in australian households', in proceedings of the 6th international conference on electronic commerce acm, delft, the netherlands, pp. 451-6, october 25-27, 2004. straub, d, boudreau, m-c & gefen, d 2004, 'validation guidelines for is positivist research', communications of the association for information systems, vol. 13, no. 24, pp. 380427. tilson, d, lyytinen, k & sørensen, c 2010, 'research commentary-digital infrastructures: the missing is research agenda', information systems research, vol. 21, no. 4, pp. 748-59. touré, dhiis-g 2014, commiting to connecting the world: introduction, viewed 2nd january 2014, <http://www.itu.int/en/action/broadband/pages/intro.aspx>. wallsten, s & mallahan, c 2010, residential broadband competition in the united states, available at ssrn: http://dx.doi.org/10.2139/ssrn.1684236 wdr 2015, the western downs at a glance, viewed november 15 2015, <http://www.westerndownsregion.com.au/html/profile.asp>. wdrc 2015, western downs economic annual 2015 january , advance western downs viewed 27 may 2015, <http://www.advancewesterndowns.com.au/?s=western+downs+economic+annual+ 2015+january>. whitacre, b, gallardo, r & strover, s 2014, 'broadband׳ s contribution to economic growth in rural areas: moving towards a causal relationship', telecommunications policy, vol. 38, no. 11, pp. 1011-23. whitacre, b, strover, s & gallardo, r 2015, 'how much does broadband infrastructure matter? decomposing the metro–non-metro adoption gap with the help of the national broadband map', government information quarterly, vol. 32, no. 3, pp. 261–9. wikipedia 2015, western downs region, wikipedia, viewed 10 december 2015, <https://en.wikipedia.org/wiki/western_downs_region> willis, s & tranter, b 2006, 'beyond the ‘digital divide’internet diffusion and inequality in australia', journal of sociology, vol. 42, no. 1, pp. 43-59. world bank 2012, broadband strategies handbook: chapter 1 building broadband, world bank, <http://broadbandtoolkit.org/custom/core/documents/broadband%2 australasian journal of information systems lane, tiwari & alam 2016, vol 20, research on rural and remote ict case study of broadband in rural australia 23 appendix 1 comparison of wired broadband and wireless broadband services available by cost and data quotas provider data cost contract terms adsl 2+ telstra 100gb $75 per month24 month contract telstra 500gb $95 per month24 month contract telstra 1000gb $115 per month24 month contract optus unlimited $110 per month24 month contract tpg -off net 50gb $49.99 per month-6 month contract tpg -off net 200gb $59.99 per month-6 month contract tpg -off net 300gb $79.99 per month-6 month contract tpg -off net 500gb $99.99 per month-6 month contract tpg 50gb $29.90 per month-6 month contract tpg 150gb $39.90 per month-6 month contract tpg 500gb $49.99 per month-6 month contract tpg unlimited $59.99 per month-6 month contract dodo 10gb $49.80 per month24 month contract dodo 50gb $54.90 per month24 month contract dodo unlimited $59.90 per month24 month contract iprimus 40gb $59.95 per month24 month contract iprimus 100gb $69.95 per month24 month contract iprimus 300gb $89.95 per month24 month contract iprimus unlimited $129.95 per month24 month contract iinetoff net 100gb $79.90 per month24 month contract iinetoff net 300gb $89.90 per month24 month contract iinetoff net 500gb $119.90 per month24 month contract iinetoff net 1000gb $149.90 per month24 month contract iinet 250gb $59.90 per month24 month contract iinet 1000gb $79.90 per month24 month contract mobile – post-paid plan telstra 500mb $35 per month 12 month contract telstra 5gb $50 per month 12 month contract telstra 10gb $60 per month 12 month contract optus 1.5gb $10 per month 24 month contract optus 2.5gb $20 per month 24 month contract optus 4gb $30 per month 24 month contract optus 8gb $45 per month 24 month contract optus 12gb $60 per month 24 month contract optus 25gb $90 per month 24 month contract optus fixed wireless 50gb $70 per month24 month contract vodafone 4gb $20 per month 24 month contract vodafone 9gb $30 per month 24 month contract vodafone 13gb $45 per month 24 month contract vodafone 19gb $60 per month 24 month contract vodafone 26gb $90 per month 24 month contract vodafone 51gb $140 per month 24 month contract australasian journal of information systems lane, tiwari & alam 2016, vol 20, research on rural and remote ict case study of broadband in rural australia 24 iprimus 1gb $15 per month 24 month contract iprimus 2gb $25 per month 24 month contract iprimus 6gb $39 per month 24 month contract mobile – pre-paid telstra 3gb $30 per monthno contract telstra 5gb $50 per yearno contract telstra 16gb $140 2 yearno contract optus 4gb $30 per month no contract optus 7gb $50 per yearno contract optus 22gb $130 per yearno contract tpg 1.5gb $19.99 per monthno contract tpg 2gb $29.99 per monthno contract tpg 5gb $39.99 per monthno contract dodo 2gb $17.90 per monthno contract dodo 3gb $29.90 per monthno contract dodo 6gb $39.90 per monthno contract dodo 9gb $49.90 per monthno contract dodo 12gb $85 per monthno contract iinet 2gb $19.99 per monthno contract iinet 7gb $39.99 per monthno contract iinet 9gb $49.99 per monthno contract [note: this figures were captured during the time period 30-05-2016 which may be different other time] copyright: © 2016 lane, tiwari & alam. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ a.i is vol. 3, no. 2 philosophical and conceptual perspectives on the design of group support systems karma sherif and james f. courtney department of business analysis & research texas a&m university college station, texas usa 77843-4217 (409) 845-9541 kss6354@ tamvm i .tamu.edu j-courtney@tamu.edu abstract: design of information systems (is) has been one of the most elusive tasks of the system analyst, requiring technical expertise, creative inspiration and deep understanding of the problem under examination. perhaps even more than is the ease with other is. group support systems (gss) design is challenging because it involves groups who vary in behavior, process and culture. the system designer's worldview will shape its features. in an attempt to shed new light on gss design, this paper examines the problem from four philosophical perspectives or paradigms: functionalism, social relativism. radical structuralism and ncohumanism. the generic requirements identified from the analysis include support for multitasking, multiple languages, customization, multiple communication modes, interoperability, rational discourse and knowledge validation. while these appear to be valid requirements to support computer-mediated decision making, it may be diff icul t if not impossible to satisfy them in one software system. introduction design of information systems has been one of the most elusive tasks of the system analyst, requiring technical expertise, creative inspiration and deep understanding of the problem under examination. it is the core of the whole endeavor of developing systems. the process of determining requirements; be it user requirements or system specifications, is very challenging. this vital role of design manifests itself everyday, especially after system implementation when it becomes evident that good design has its encouraging rewards, and that bad design is painfully expensive. thus an incisive understanding of the problem, the social context surrounding it, and of forces interacting within, is crucial for pinpointing requirements , setting goals and envisioning change. this directed change can be predetermined by the designer and wished for in response to organizational needs or even out of social responsibility. users can sometime require change to adapt to their environment. whoever or whatever is driving the change, it is through design that we try to modify our environment to a better and a more purposeful one. therefore it remains the responsibility of the designer to foresee and test the impact of the intervention on the targeted community. the worldview of the designer affects the properties of the system being designed (churchman, 1971; mandviwalla, 1994). and the properties of the system determine how useful it will be across various user groups and cultures. the broader the design perspective, the more likely it is that the resulting system will be suitable for heterogeneous social contexts. thus the objective of this paper is to provide insight into how group support systems (gss) software, which is now being more widely deployed in global markets, can be made more suitable for a wider variety of users in various social and cultural contexts. to identify generic design requirements that would make gss more pluralistic, we examine gss design from broad perspectives. since much of the anticipated outcomes of gss use are driven by the designer's worldview, the role of the designer is examined in some detail within the context of four philosophical paradigms: functionalism, social relativism, radical structuralism and neohumanism (hirschheim, et al., 1996). the paper begins by reviewing the results of studies of gss use, and their effects on group decision outcomes and group behavior. next gss design is considered from the four perspectives mentioned previously. the paper ends with a proposed set of challenging generic requirements for gss design. group support systems (gss) gss are systems designed to improve group meetings through communication and by providing tools for structuring decision analysis (desanctis & gallupe 1987) . different features have been supported, ranging from mere communication to more complex systems allowing audio and video communication along with decision support tools ( ayre & gottesman 1994). interest in the technology stems from the belief that 57 a.iis vol. 3, no. 2 collaborative work is capable of producing performance superior to an individual's efforts due to the synergy of the group (george, easton, nunamaker 1990). however certain characteristics of groups restrict the realization of their full potential or even matching the performance of its best members (shaw 1978). lack of coordination, diffusion of responsibility, conflict and lack of motivation (known collectively as process losses) account for limited group performance. accordingly gss are designed, to help increase group effectiveness with regard to decision making by diminishing the effect of process losses or positively directing the characteristics to better serve the group decision outcomes. the gss designer's goal is to build optimal systems that will improve behavior and instruct users on techniques to better structure their problem solving effort (feldman & march, 1981). in accomplishing their goals. designers and developers have offered a wide variety of functions, the most common ones being : • structured steps for problem solving including : problem definition examination of alternatives and generation of new ideas decision support tools for analysis building consensus (poole, holmes &desanctis 1991). • open communication between members. • providing a role for a facilitator or leader to better direct the group decision process. effects of gss researchers have been preoccupied by the effects that technology has on groups. reported outcomes have been inconsistent across research efforts in this subject area due to variance in group size, group structure, tasks and technology. four main outcome areas, though, have been identified (see table 1). these relate to processes, behavior, decision quality and organizational structure. process outcomes: the deterministic intervention of some gss in the form of structured steps taken for problem solving activities identified participation, satisfaction and time to decision as process outcomes ( easton et al 1990). gss were found by many researchers to enhance participation among all members rather than allowing dominance by one or more participants (hiltz, johnson & turoff 1986; nunamaker, applegate & konsynski 1987; desanctis & gallupe 1987). however beauclair (1987), jarvenpaa, rao & huber (1988) and easton (1988) reported no effect on participation by gss. inconsistent findings were also reported on time to decision. time to decision was either found to be longer in computer mediated communication (easton et al 1990) or unaffected (beauclair 1987, easton 1988). findings on satisfaction with the process were more inclined towards increased satisfaction for gss groups. while a number of researcher found no effect, gallupe, desanctis & dickson (1988) and watson, desanctis & poole (1988) noted that satisfaction with the process and outcome was reduced by gss. behavior outcome: regarding behavior, positive outcomes such as effective conflict management and uninhibited behavior have been identified (poole, holmes & desanctis 1991). anonymity played an invisible role in bringing about these changes, encouraging free expression, depersonalization of discussion, rambling conflict and causing members to concentrate on issues rather than "personal antagonism" (poole et al 1991 p934). table 1 summary of gss effects on groups variables process outcome behavior outcome decision outcome organizational design & structure definition and findings participation, satisfaction with the process and the decision and time to decision. findings were inclined to increase participation by group members and increased satisfaction with both the process and the decision. time to decision was reported longer in gss meetings than in face to face. conflict management and uninhibited behavior. gss empirical research showed that gss sessions reduced rambling conflict and encouraged open unrestrained discussions. decision quality measured in terms of correctness, creativity and uniqueness of alternatives. experiments proved gss sessions to result in better decision quality. some decentralized organizations have centralized. some centralized organizations have decentralized. 58 a.iis vul..). no. 2 decision outcome: reported impacts on decision outcome include better decision quali ty, as measured in terms of correctness, creativity or uniqueness of alternatives (steeb & johnson 1981: lewis 1982: turoff & hiltz 1982; gallupe, desanctis & dickson 1988; zigurs. poole & desanctis 1988). results have been relatively consistent in this area. organizational design and structure: apart from the effects on group behavior and decision outcome, gss is thought to have considerable impact on organizational design and structure (huber 1990). the findings regarding change in design and structure have also been inconsistent due to the nature of the organization and the surrounding environment. computer-assisted communication and group decision making have led to decentralization in highly centralized organizations, and to centralization in highly decentralized organizations. organizational structure has also undergone an ongoing change after the introduction of gss. through the extensive use of such systems, organizations have realized benefits of the technology and adapted their structure to better exploit these privileges. all of these contradicting results and emerging changes pose a great challenge to gss design and makes it complicated for the designer to generalize. the role of the designer before proceeding to the discussion of gss design we need to comprehend the exact role that the designer plays. is it the designer who is driving these changes in group behavior and organization structure through the implementation of the design? or are organizations as social institutions the driving force? desanctis' (1993) "individualistic" and "collectivism" assumptions serve as a rich, but simple dichotomy for theories underlying gss design (see table 2). the "individualistic" approach, or as spacaman & oskamp (1990) call it "determinism", assumes that technology is capable of changing behavior. technology is viewed as "an exogenous force which determines or strongly constrains the behavior of individuals and organizations." (markus & robey. 1988, p. 585). technology has the power to enhance individual skills and overcome deficiencies inherent in groups. accordingly, the effectiveness of design is measured in terms of the ability of the system to help groups reach quality decisions, increase participation, reach consensus and increase satisfaction. deterministic changes are designed to empower the individual with skills to better serve the whole group (desanctis 1993). such design is built on individuals as the unit of analysis. the challenge to the designer with individualistic assumptions is how to aggregate individual scores to achieve group measures. with single-user applications, design is more forward and the user has the luxury of changing the tool to match individual needs and preferences. in multi-user applications, users collaboratively use the system and are deprived of custom tailoring (mandviwalla 1994). table 2 assumptions underlying gss design assumption individualistic collectivism role of technology deterministic emergent implications to design systems should enhance decision quality. the challenge is to achieve group measures for group requirements and decide on the spirit the system should adopt. systems should convey society's rules and values. the challenge is how to code social values and rules into the system. the process of translating group requirements into a system's functionali ty gets complicated due to the plural i ty of personalities and tasks, and the variety in a group context. the question then arises as to which requirements should the system serve? what worldview should the designer inst i l l upon the system? the worldview being the inner beliefs that one holds about the world and that unconsciously influences what one does and the way one chooses to do it (whiteside & wixon 1988). accordingly designers rely on explicit or implicit worldviews to guide their design and development effort. understanding the worldview of a designer is crucial to understanding the system's belief structure and verifying its technical features. the worldview of a system is one aspect of the technological structure viewed as the objectives in terms of goals and attitudes that the technology wishes to accomplish. poole & desanctis (1990) have clearly distinguished between the "spirit" and the "structural features" of a system in their proposed "adaptive structuration theory." the "spirit" is defined as the worldview and the "structural features" as the translated design issues that serve the "spirit." thus the "structural features" are 59 a.iis vol. 3, no. 2 functionally dependent on the system's "spirit." should the system's "spirit" then reflect the worldview of the designer? that is, what the designer regards appropriate and socially acceptable. or should it convey the beliefs of a group leader and serve the motives of the elite? can the designer be objective and detach his or her own views from the design process to support the worldviews of the group and satisfy their requirements? can a "utopian" system attending equally to the needs of all groups and all members be built? how can the designer accommodate diversities in race, gender and culture in gss design? collectivism views organizations as social structure, and technology as an outcome of the social culture (desanctis 1993). such an assumption might help explain the contradictory findings regarding the impact that technology brings on groups and organizations. since culture changes, technology will have different meanings, different roles and different purposes across organizations and groups and along time. it is these social complexities that affect technology development. the designer's goal is seen as a way to reinforce cultural rules and values, to encode cultural norms into gss, and "to bring gradual improvement in the social order of the organization or community" (desanctis 1993 p.98). according to collectivism the world view of the system would convey the rules and values of the organization. the concern is more about the goals society is trying to achieve through the technology rather than the goals of the system. the impact gss is thought to have on groups and organizations is considered to be a function of the surrounding social context. designers must be socially aware of users' needs and cultural values instead of being consumed by the technology. such social concern will help minimize wasted resources deployed on the development of fancy eye-catching tools that are not fulfilling their social obligation. the challenge posed to the designer is how to code social values and rules into the system? how can the system reflect society's values and norms to assertively alter the community in a productive way? in trying to determine answers for these inquiries a leading step would be to examine the different philosophical paradigms underlying the design of current gss. a subsequent step would be to explore group requirements and the challenges they proclaim to the designer. paradigmatic foundations of is and their application to gss hirschheim, lyytinen & klein (1996) classify major theoretical assumptions underlying is design under four philosophical paradigms: functionalism, social relativism, radical structuralism and neohumanism. these are summarized in table 3. functionalism is pursuing the design of systems that have specific and clear objectives. it should serve the decision maker by providing timely and meaningful information to help solve organizational problems. according to this approach management is the best candidate for providing the specific and clear goals of a system. the designer acts as a technology expert interpreting system objectives into a structured formalized set of steps and procedures. this approach assumes that there is only one face to reality which is modeled in a purely technical fashion. change is deterministic in nature and predetermined by management and the designer. the designers comprehension (possibly affected by his or her own views) of management's needs are reflected in the system's features and its "spirit." aspects of functionalism do apply to gss design. most of the current gss systems offer tools for the formalization and structuring of problem solving procedures. as mentioned earlier, positive outcomes were reported regarding decision quality that were believed to be the results of imposing structuration. most of these systems were buil t on scientific management theories. it does however, consider social conventions that, to a great extent, determine the outcome. table 3 iso paradigms and their application to gss paradigm functionalism social relativism radical structuralism neohumanism emphasis on technical rationality and legitimate management needs sense making and mutual understanding the interest of the working class emancipation gss features that support the paradigm structuring and formalization of problem solving techniques encouragement of participation and building consensus anonymous communication not yet applied 60 a.i is vol. .v no. 2 but functionalism fails to explain why conflicting results have been reported . it can be inferred that gss systems that are not targeted to a specific group tend to support functionalistic views, anticipating similar outcomes across groups and assuming that rules and patterns govern the reality that we perceive. gss systems that concentrate on diminishing the effect of process losses through structuring apply functionalism. technology is believed to cope successfully with problems associated with group work regardless of human behavior (mandviwalla 1994). the central flaw of this approach is inadequate understanding of meaning in the construction of reality. functionalism forgets that "the link between designs and the effects they produce is the meaning attributed to them" (hirschheim, et al. 1996. p.108). social relativism, the second paradigm, concentrates on building systems that encourage sensemaking and mutual understanding. there is no one accepted reality for what we discern or what we anticipate. perceptions is what we have and perceptions vary depending on the environment, previous experiences and culture. the designer's role is to ease the transition from one perception to the other through continuous interaction. gss systems that promote information exchange among group members apply social relativism. through interaction, reality is socially constructed. technology is viewed as a tool for addressing problems through participation. bui lding consensus via group acceptance is of major concern. right or wrong is relative and what counts is what the group agrees on and perceives as emotionally and morally right. functionalists might argue that social relativism would result in emotion-based, effectively questionable decisions. however, research in cognitive science has indicated that nonrational "socioemotional" decisions often turn to be highly functional (etzioni 1988). radical structuralism is mainly concerned with building systems that promote the interests of the working class, rather than l imit ing the design focus to management. this paradigm strongly urges the designer to advocate the labor side and to adjust the balance of power between management and labor. to some extent, this insight is captured in gss design and supported by anonymous communication. subordinates are given the right to freely express their views and to strongly defend their beliefs against management. though gss systems encourage participation, the power bestowed on workers is still limited to their competency and their ability of persuasion. the problem with these systems is their deliberation of conflict as a vital sign for group discussions, thereby adding to the complexity of the decision process. the neohumanist approach has taken from all three aforementioned paradigms and added the notion of "emancipation." emancipation is defined as the ability of information systems to "facilitate the widest possible debate of organizational problems such that truly shared objectives could be agreed upon as well as policies for achieving them" (hirschheim, et al., 1996, p.87). the gss system designer should try to balance between technical intervention, shared understanding and "continued emancipation." the focus should be on freeing groups and organizations from distorting influences and other constraints to "rational discourse." neohumanism is considered a sound theory, however, no work has been reported that fully supports it. its idealistic nature assumes that "rational discourse" can be attained without suggesting how. this has led to the reluctance of designers to apply the concept. it is the application of neohumanism to gss design that forms the general challenge since it considers all aspects be it technical, social or emancipatory. implications for gss design gss software presently reflects a design based primarily on functionalistic assumptions. by being cognizant of these assumptions, and reflecting upon the implications of the other paradigms, hopefully will result in a set of gss design requirements that are more neohumanistic in nature. of course, no design can satisfy all the requirements for all groups for all time. nevertheless, by starting from a broader perspective, it seems reasonable to expect to produce a more pluralistic set of generic requirements than has been the case with previous efforts (see figure 1). such a set of requirements is described in table 4. 61 a.iis vol. 3. no. 2 group needs functional • multitask •different work methods • interoperability social • imiltilanguage • adjustable groupwan different interaction methods emancipatory •rational discourse •information validation •active participation in idea generation figure 1. group requirements first, within a single group, understanding the dynamics of group characteristics, group process and decision outcomes is multidisciplinary. we need to take a wider view of group work studying "organizational theory, management science, social psychology, decision theory and communication" (mandviwalla 1994 p.249). there are multiple tasks involved in the decision making process and a variety of work methods for each task. groups differ in the tasks they choose and the way they perform them. table 4 summary ofgss requirements gss feature multitasking leadership style multilinguistic multcommunication customization interoperability rational discourse knowledge validation description support for multiple tasks in the decision-making process. support worldviews reflecting different leadership styles and participation modes. user support for multiple languages. support for text, audio and video communications modes. tailorable to the needs of the group. seamless integration with other popular software packages. support for robert's rules of order and structured debates. provide means to validate and manage information and knowledge brought into the decision group process. second the conflict on the system worldview must be recognized and accommodated. the choice of a worldview should be carefully and objectively rendered to fit with the organizational culture and to accord with the social meaning. group culture must also be considered. though aiken, hwang, de magalhaes & martin (1993) had reported no significant differences between cross cultural groups in terms of ease of communication, fear of peer criticism and overall meeting satisfaction, special consideration should be given to 62 a.i is vol. 3. so. 2 issues such as race, gender and beliefs. for example leadership and participation is viewed differently across culture. it might be true that groups in us. organizations value participation and appreciate the point view of each and every member. it is to be expected then, that these groups would oppose deterministic intervention of a leader with the decision process. their acceptability of products with limited selections would be low. however, it is highly questionable that the same will hold with middle eastern groups. the leader in a country such as egypt is the catalyst who promotes productivity by managing people and the decision process. a more authoritarian approach may be acceptable in such cultural contexts. as organizations become more global in nature, gss are being used in wider and more varied l inguis t ic domains. i t is important to consider language in future gss. as mono-linguistic groups may no longer be considered the sole users of such systems. religion may also be decisive factor in group acceptance of a gss in some cultures. in cultures where religion plays a major social and political role, it would be naive to ignore its implications on the group's implicit needs or approval of a product. a considerable sector of egyptians, for instance, are highly swayed by religion, and systems that are compatible with religious beliefs would easily appeal to their persona]ities. there is only one reality beyond any shadow of doubt; that defined by the holy book. this fact was confirmed when islamic investments companies were launched in 198()'s and were a huge success signaling an alarm to government of a massive takeover. the designer's understanding of the relat ivi ty of "universal truths" and the uniqueness of groups should be reflected in the design (mandviwalla & olmen 1994). in order to ensure group satisfaction and a high ut i l iza t ion rate, the system should support mult i -group tasks through different system modules (mandviwalla & olmen 1994, huber 1984). the broadening of the system's functionality certainly will defy the designer unless access to various other software products from different vendors are incorporated into the design. besides multiple tasks, systems should support multiple work methods for each task to serve the needs of different groups (hackman 1990). the freedom of choice is more cherished than a predetermined fixed set of functions (malone & lai 1992). for example, dykstra & carasik (1991) present a "conversation-based" design that offers the group the freedom to establish its structure. groups also require multiple interaction methods which they use interchangeably to accommodate to changes in the criticality of the job. a gss should provide both face to face communication through audio and video media. the whole concept is creating adjustable groupware that can be easily customized by the group to better serve its needs. accordingly, designers need to consider possibilities of the system migration across applications modelessly. groups often require information saved on different software or they might occasionally need to conduct professional analysis provided by a separate tool. in certain contexts, such as formal legislative bodies and the like, it may be useful for the system to impose restrictions on group communication, thereby leading to a form of "rational discourse." as desanctis and gallupe(1987) have suggested, it might be helpful to enforce robert's rules of order. or. hodges' dialectical engine (1991) might be incorporated into negotiation systems to enforce rules of structured debate (rescher, 1977). a final and highly neglected aspect of collaborative work that designers should consider is the abili ty of the system to validate input brought into by group members. according to hirschheim. lyytinen and klein (1996) is is defined as "technically mediated social interaction systems aimed at creating, sharing and interpreting a wide variety of meaning." in that respect gss falls short of creating meaning and is limited in interpreting it. validation of information is vital because of the influential power that information exerts on individuals. a gss system should be able to endorse or falsify ideas and proposals initiated by group members. a gss (hat can extrapolate and fill in unspecified gaps can be greatly helpful. for example, liou & nunamaker (1993) propose a gss that facilitates knowledge acquisition from multiple experts to build an information center help-service expert system. the electronic documentation of experts' knowledge, the structured process, the parallel knowledge extraction and the collaborative atmosphere enriched the domain of expertise. a definite challenge would be to take this research a step forward and incorporate the expert system knowledge base within gss to help groups generate ideas and facilitate discussion. earlier discussions on particular problems can be searched and the effectiveness of the decision outcome can be evaluated against the results. qualified decisions can be incorporated into the knowledge base as a learning experience. the design of an active participating gss is definitely difficult. it is hard to agree on standardized labels for everything around us. the need to standardize our description of specific experiences and agree on common names for simple labels is a compelling prerequisite for the design of a "lockean system" (churchman 1971). another challenge is the community agreement for what is true or false and a willingness to accept the 63 a.iis vol. 3, no. 2 worldviews of professional community members. the problem however, is the system's presumption that the future will follow the lead of the present in an "isomorphic sense" or else will be unreliable. conclusion design is the pillar for the development of highly operable and productive systems. the goal of a gss is improving group decision processes and outcomes by providing communication and techniques for supporting group endeavors. the design of a gss is complicated because it involves groups whose behavior and decision process are hard to predict. therefore it is crucial to understand the group before attempting to determine its requirements and choosing a system worldview. the major assumptions and paradigms underlying gss design were examined as a starting point in determining group requirements. a generic set of requirements for gss systems groups were proposed . though not all of the requirements can be modeled they certainly present valid needs for groups to effectively and efficiently proceed with computer mediated decision making. references aiken, m.,c. hwang, r. de magalhaes,& j. martin (1993). a comparison of malaysian and american groups using a group decision support system. journal of information science principles & practice. 19(6)489-491. ayre, r., & gottesman, b.z. (1994). workgroup computing group enabled. pc magazine 13(11), 171189. beauclair, r. a. (1987). an experimental study of the effects of group decision support system process control applications on small group decision making, unpublished ph. d. dissertation, indiana university, bloomington, in, 1987. churchman, c.w. (1971) the design of inquiring systems : basic concepts of systems and organization basic books, inc., ny. desanctis, g. (1993) group support systems new perspectives shifting foundations in group support system research. macmillan ,ny. desanctis, g., & gallupe, r. b. (1987). a foundation for the study of group decision support systems. management science, 33(5), 589-609. dykstra, e. & carasik, r. 1991. structure and support in cooperative environments: the amsterdam conversation environment. int. j. man-mach. stud. 34, 419-434. easton, g. k., (1988). group decision support systems vs. face-to-face communication for collaborative group work: an experimental investigation. doctoral dissertation, university of arizona. etzioni, a. (1988). the moral dimension: toward a new economics. new york: the free press. feldman, m. s., & march, j. g. (1981). information in organizations as signal and symbol. administrative science quarterly, 26(2), 171-186. gallupe, r. b., g. desanctis, & g.w. dickson, (1988). computer based support for group problemrinding: an experimental investigation, mis quarterly., 12(2), 277-296. george, j. f., easton, g. k., nunamaker, j. f., jr. northcraft, g. b. (1990). a study of collaborative group work with and without computer-based support. information systems research, 1(4), 394415. hackman, j. r. (1990) groups that work and those that don't. jossey-bass, san francisco. hiltz, s. r., turoff, m., & johnson, k. (1989). experiments in group decision making,3: disinhibition, deindividuation, and group process in pen name and real name computer conferences. decision support systems, 5(2). 217-232. hodges, w. s. (1991). dialectron: a dialectical engine for the support of strategic planning. unpublished ph. d. dissertation, texas a&m university. hirschheim, r., lyytinen, k. & klein, h. (1996) information system development and data modeling : conceptual & philosophical foundations, cambridge university press. huber, g. p. (1984) issues in the design of group decision support systems. mis quart. 3, 195-204. huber, g. p. (1990). a theory of the effects of advanced information technologies on organizational design, intelligence, and decision making. academy of management review, 15(1), 47-71. jarvenpaa, s. l., v. s. rao, & huber g. p., (1988). computer support for meetings of group working on unstructured problems: a field experiment, mis quart., 12(4) ,645-666. lewis, f. l.. (1982). facilitator: a micro computer decision support systems for small groups. doctoral dissertation, university of louisville, 1982 64 a.iis vol. 3, no. 2 liou, y. i. & nunamaker, j. f. jr. (1993) an investigation into knowledge acquisition using a group decision support system. information & management. 24(3) 121-132. malone, t. and lai, k (1992) toward intelligent tool for information sharing and collaboration. in computer augmented teamwork, r. bostrom, r. watson and s. kinney, eds. van nostrand reinhold, new york, 86-107. mandviwalla, m. & olfman, lome (1994). acm transactions on computer-human interaction, 1(3), 245-268. mandviwalla, m., (1994). the world view of collaborative tools. the arachnet electronic journal on virtual culture, 2(2) 1994. markus, m. l., & robey, d. information technology and organizational change: casual structure in theory and research. management science, 1988, 34(5), 583-598. nunamaker, j. f., applegate, l. m., & konsynski, b. r. (1988). computer-aided deliberation: model management and group decision support. operations research, special issue on decision support systems, 36(6),826-848. poole, m. s., holmes, m., & desanctis, g. (1991). conflict management in a computer-supported meeting environment. management science, 37(8), 926-953. rescher, n. (1977). dialectics: a controversy-oriented approach to the theory of knowledge, state university of new york press, new york. steeb, r. & s. c. johnson, (1981) a computer-based interactive system for group decisionmaking. ieee transactions on systems, man and cybernetics, smc-11,8 , 544-552 turoff, m. & s. r. hiltz. (1982). computer support for group versus individual decisions. ieee transactions on communications, 30(1), 82-91. watson, r. t., g. desanctis, & m. s. poole, (1988). using a gdss to facilitate group consensus: some intended and unintended consequences, mis quart., 12(3), 463-478. whiteside, j & wixon, d. (1988) contextualism as a world view for the reformation of meetings. in cscw proceedings. acm, new york, 369-376. zigurs, i., m. s. poole, & g. desanctis, a study of influence in computer-mediated group decision making, mis quart., 12(4), 625-644. 65 ajis vol. 7, no 2 may 2000 ecological interface design for anaesthesia monitoring marcus watson1, w. john russell2, and penelope sanderson' 'swinburne computer-human interaction laboratory school of information technology swinburne university of technology john street. hawthorn, victoria 3122 http://www.it.swin.cdu.au/schil 2dcpartment of anaesthesia and intensive care royal adelaide hospital north terrace adelaide, sa 5000 abstract the operating theatre is a noisy place with many uninformative and redundant alarms. using data from a recent observational study, we demonstrate that anaesthetists actively respond to only 3.4% of all audible alarms. we outline a range of possible solutions to the alarm problem. ecological interface design (eid) helps to outline the requirements for an information environment for anaesthetists and to indicate the possible benefits of continuous auditory signals. our observational data are then "reworked" to give an indication of possible benefits of a continuous auditory display. finally we indicate steps we are taking to test these ideas empirically key words interface design, patient monitoring systems, alarm systems, cognitive engineering, ecological interface design introduction much of the literature on human interaction with medical equipment—such as information systems and devices in the operating room (or)—has focused on critiques of existing devices (cook & woods, 1996; bogner, 1994). less often have suggestions been made about the redesign of equipment and prototypes developed. however in the last few years, under the stimulus of ecological interface design, concrete design suggestions and prototypes have emerged (effken, kim & shaw, 1997; lin, isla, doniz, harkness, vicente & doyle, 1995; sharp & helmicki, 1998). in this paper we outline a long-standing problem with unwanted and uninformative alarms in the or and take the first step towards a new kind of solution. when looking at how to design effective alarms in critical care environments, many researchers have applied findings from alarm studies in aviation and nuclear power plant environments (cook & woods, 1996). however the anaesthesia and critical care environments pose distinct problems of their own. new technologies are continually being introduced in a piecemeal fashion rather than part of an integrated system. moreover, the most important part of the sensed environment—the patient—is not a fixed engineered system, but instead has unique physiological characteristics that should be taken into account when determining what is normal and abnormal. finally, alarms reflect the state of sensor technologies that have their own failure modes independent of patient status. in light of these factors we undertook an initial observational study to determine the problems and needs in or alarm systems. background we studied anaesthetists' use of alarms in a series of observations at royal adelaide hospital (rah). observations were over a two-week period and involved 42 operations varying in length, type and monitoring equipment used by anaesthetists. the focus was on detecting whether auditory alarms successfully directed an anaesthetist's attention to a problem with the patient or with the anaesthetic machine. we recorded the number of times an alarm limit was breached and the number of times auditory signals sounded. furthermore any manipulation of the auditory alarm systems by the anaesthetists to monitor the patient (such as changing upper or lower limits, or using alarms as reminders to visually check trends) were recorded. alarms were classified under several headings: physiological parameter indicated by the alarm (eg blood pressure, heart rate); type of alarm (lower or higher limit breached, noise in signal, equipment disconnection or warning); severity of the alarm; number of times the alarm sounded due to the incident (total number of auditory warnings for each alarm); phase of the operation in which the alarm occurred; and time. in addition the anaesthetist's response to each alarm signal was observed and coded. as figure 1 shows, 35.4% of the alarm sounds reflected the initiation or first sounding of a specific alarm and the remaining 64.6% were "run-on" alarms, simply re-signalling the initial alarm. only 3.4% of all alarm sounds caused the anaesthetist to actively make adjustments to change patient state—only 5.3% of the initial alarms and 1.9% of the run-on alarm sounds led to responses. of the 125 alarm limits breached during induction, 20 cases (16%) led to a response; of the 232 alarm limits breached during maintenance, 29 (12.5%) led to a response; and of the 124 alarm limits breached during emergence, only one led to a response (0.8%). anaesthetists are well aware of the failure of alarms to indicate useful information. typical comments made by the anaesthetists in our sample include the following: 109 ajis vol. 7, no 2 may 2000 • "i know the nibp alarm [blood pressure high] is going to go off again; [however] it takes a little while for these drugs to take effect and the pressure is already dropping" • "not much i can do about that alarm [ecg noise alarm]. every time the surgeon uses the diathermy it'll go off. if i silence it i might miss an alarm that is important" • "when the disconnect alarms are going, you normally have your hands full with the patient coming around. you know they [alarms] are unimportant, so you have to ignore them". as woods (1995) has noted, alarms often go off in parallel or in cascades, compounding difficult situations rather than helping them. the rah data showed this both in critical incidents and in routine activity. the former is best illustrated in one operation where the patient's heart rate, blood pressure, oxygen saturation and respiration all started to decline rapidly. this was accompanied by alarms from spo2, nibp and the gas monitoring system. the anaesthetist was searching the monitor for information before the alarms sounded, having already noticed the downward trend. the anaesthetist's finger was ready to silence the alarms before they even went off. after this the focus was on the patient and several visual alarms flashed for other monitoring devices that had reached alarm limits, but since they were now silenced they were missed by the anaesthetist as he dealt with the patient. there were further frequent cases of disconnect warnings for two or more of spo2, ecg, nibp and etco2 that all started in the same minute and lasted for 10 to 20 repeats while the anaesthetic staff attended to the patient. *** total number of alarm limits breached 481 (35.4%) >^t existing alarm^. total number of ' auditory signals 1358 (100%) 64.6 % of total alarms 1 / ŝ. alarms responded \ a" alarms to on the first responded to auditory signal 46 <3-4%> 26(1.9%) figure 1. alarms breached, auditory events, and anaesthetists' responses design goals our goal is to contribute to the design of a more effective information environment about patient state. the quantitative and qualitative data reviewed indicate just some of the problems with current alarm systems in the operating theatre. clearly, a more effective alarm management environment would have to achieve the following: • decrease confusion about which alarm went off that currently forces anaesthetist to check on monitor • decrease the number of uninformative alarms that sound, decreasing noise stress • decrease the density of alarms during critical events • increase the proportion of clinically useful alarms over "nuisance" alarms • increase anaesthetist's situational awareness and monitoring of higher-order patient state variables. table 1 provides a range of possible solutions that have been suggested at various points by researchers and clinicians interested in alarms in operating theatre environments. the solutions include suppressing "run on" alarms, suppressing some alarms at specific phases (such as ecg disconnect alarms at emergence), using earcons, using a continuous auditory display, developing intelligent alarm management, and using a heads-up display. as table 1 makes clear, all approaches have advantages and disadvantages. as will be seen, the solution we explore in this paper is the idea of using a continuous auditory display for key parameters. this approach has been in place for some years since the advent of pulse oximetry. the pulse oximeter is a peg-like device that clips to a patient's finger, ear or toe and measures oxygen saturation in arterial blood as well as heart rate. the result is displayed continuously as a series of beeps where the rate indicates heart rate and the pitch of each beep indicates oxygen saturation, or spo2. we are exploring the possibility of extending the concept of continuous auditory displays to other continuously sensed parameters. ecological interface design is the conceptual framework we have chosen to guide this work. 110 ajis vol. 7, no 2 may 2000 eid approach eid is an approach to interface design that is based in rasmussen's cognitive work analysis (rasmussen, pejtersen, & goodstein, 1994; vicente & rasmussen, 1992; reising & sanderson, 1998). the goal of eid is to engineer an information "ecology" around the human operator that is as easy to understand and as simple to control as the natural ecology in which we have evolved. the steps to achieve this are as follows: 1. analyse the work domain in its first principles. 2. analyse the control tasks that must be performed for the work domain to achieve its functional purpose. 3. map critical properties of the work domain onto visual or auditory displays in a way that best supports the execution of control tasks (often termed "semantic mapping"). suggestion suppress "runon" alarms suppress selected alarms at certain phases use "earcons" use redundant signals for single measure use continuous auditory displays intelligent alarm management head-up displays advantages reduction of noise in or (64.6% fewer alarms in rah data) annoying alarms removed. type and origin of alarm discernible. artifact-based alarms reduced. reduces the # of alarms needed. keeps the anaesthetist "in-the-loop" hypothesises higher order state. reduces low-level nuisance alarms. reduces the # of alarms needed. keeps the anaesthetist "in-the-loop" disadvantages anaesthetist may forget to check if preoccupied. design challenge to tell when "run-on" alarm finished and new incident starts. danger of false negatives. still the same amount of noise. information still "separated". not available for all parameters. anaesthetist unsure of accuracy of measurement due to the variation between different devices. continuous noise. acoustic design challenge. requires a reevaluation with alarm noise. only limited use possible. degree of human trust is an issue. reduction of human "in-the-loop" an issue. visual cues may be inadequate to attract anaesthetist attention. discomfort of head mounting for some.. table 1. alarm management strategies: advantages and disadvantages a key component of the anaesthesia work domain is the human body in normal, compromised, and deranged states. figure 2 shows a work domain analysis (wda) of some anatomical structures and physiological functions of the human body that are most relevant for anaesthesia. for any node in the hierarchy, nodes above it say "why" the structure or function is there, and nodes below it say "how" the function is achieved. compromised or even deranged states emerge from abnormalities in the elements, processes, and functions noted in the analysis. superimposed on the analysis are circles indicating the kind of information that is picked up by key sensors in the operating theatre. the control tasks are all the activities that must be carried out for safe, effective anaesthesia. the anaesthetist's role is to support cell perfusion (oxygenation via maintaining airway, breathing, and circulation) while inducing effective anaesthesia (an appropriate balance of pain relief, muscle relaxation, and sleep). this must be achieved with a safe induction (often including intubating the patient and running invasive lines) and a safe emergence (often including extubation and return to unassisted ventilation). normal physiological functions are maintained, compromised functioning compensated for, and deranged states managed with the tools and drugs at the anaesthetist's disposal. semantic mapping involves revealing the properties, constraints, and boundaries of the work domain in a way that makes a direct connection with the control tasks to be performed. the "affordances"—or possibilities for action—in the work domain should be shown. the human operator should not have to laboriously infer what is happening from low-level data, but should instead be able to maintain a constant level of situational awareness about the patient's physiological state. hitherto, most discussion about semantic mapping has dealt with visual displays (buttigieg & sanderson, 1991; bennett, nagy, & flach, 1997; hansen, 1995) and relatively little with auditory displays (gaver, 1993; gaver, smith, & o'shea, 1991). it is notable that the most comprehensive treatment of the ecological approach to human-machine systems (flach, hancock, caird & vicente, 1995) does not include a chapter on the auditory environment. for human-machine systems designers, auditory displays are almost exclusively auditory alarm systems. although there is a substantial tradition of work into the discriminability and management of alarms (stanton, 1994) there is very little on the continuous display of system state through the auditory modality. however, woods (1995) presents an alternative view of the possibilities for the auditory environment in continuous process environments. he argues that an alarm system should be seen as an "agent" that attempts to direct the attention of the human operator. an auditory signal should help maintain a continual level of awareness in the 111 ajis vol. 7, no 2 may 2000 operator, at what woods calls a "pre-attentive" level. the display should be capable of reorienting attention towards a changed or changing parameter before an alarm sounds so that the alarm is unnecessary or when the alarm does sound, the operator already knows what is happening with that parameter. functional purpose balance being maintained general function processes affecting anatomy and physology anatomical elements figure 2. wda of cardiovascular and respiratory parameters in the domain of anaesthesia, the auditory pulse oximetry display is a paradigmatic case: if the pitch changes (indicating dropping oxygen saturation) or the auditory event rate changes (indicating a change in heart rate), anaesthetists often turn to check the digital readout on the monitor before an alarm limit is reached. pulse oximetry has been shown to be highly successful at preventing incidents. runciman, webb, barker, & currie (1993) have reported that of the 1256 incidents in the australian incident monitoring study (aims), pulse oximetry detected the largest proportion of evolving incidents. figure 2 positions the two parameters picked up by pulse oximetry on the work domain analysis of the human body. oxygenation of the blood is a general function inferred from saturation at one anatomical point. heart rate is a physical process picked up by the device. together they provide important information about physiological balances being maintained. in our work we are exploring the feasibility of extending continuous auditory display to other systems—especially the respiratory system. the importance of continuously monitoring cardiac and respiratory rhythm has been advocated for over 80 years (runciman et al, 1993). the value of such monitoring is strongly corroborated in the aims study. as noted, of the 1258 monitor-related incidents, 27% incidents were first picked up by pulse oximetry (device most often first to detect an incident) and 24% by capnography, or co2 monitoring (device next most often to detect an incident). figure 2 positions three key respiratory parameters: end tidal (or expired) co2 as picked up by capnography, respiration rate, and end tidal volume on the work domain analysis. the three parameters are sensed as processes rather than as functions. however, the wda makes it clear how they combine to indicate general physiological functions and balances. rework of rah data continuous auditory display of key parameters should, in principle, remove the anaesthetist's dependence on limitbased alarms for information about those parameters if the anaesthetist happens not to be watching trends on the monitor. we conducted a "what if analysis with the rah data to see how many alarm sounds would have been eliminated if limit alarms were removed and a continuous auditory display used. we first looked at the reduction in alarms if limit alarms in pulse oximetry were suppressed and then if both pulse oximetry and respiratory alarms were suppressed. we hasten to point out that, at this stage, we do not recommend that limit alarms should be suppressed. continuous auditory displays may, however, allow limit alarms to indicate only more extreme deviations from the norm than is currently the case. moreover, the analysis that follows does not necessarily reflect what would happen if continuous auditory displays were introduced and limit alarms suppressed, since such an arrangement would have a reactive effect and would possibly lead to a quite different monitoring strategy by the anaesthetist. the right hand figure in figure 3 shows the expected reduction in auditory alarms across the 42 operations when pulse oximetry and respiratory systems have their limit alarms removed. the total number of alarm sounds drops to 53% of the original value and the number of alarm incidents drops to 37% of the original value. it is important to 112 ajis vol. 7, no 2 may 2000 note that seven alarms coded as "useful" that the anaesthetist responded to and three further "useful" alarms that the anaesthetist did not respond to would not have sounded under this policy. however there would still have been a continuous auditory signal carrying the information. it is possible that a continuous (ongoing) auditory signal with no limit alarm may be safer than the 10 episodes (out of 42 operations) in which the main monitor alarms were completely silenced for long tracts of the operation. the general reduction in alarms particularly the confusing and "cry wolf alarms, can be expected to lead to greater attention by the anaesthetist to the alarms that remain (bliss et al., 1995). propsed auditory alarms total number of alarm limits breached 481 (35.4%) /*c alarms responded to on the first auditory signal 26(1.9%) existing alarms. total number of * auditory signals 1358(100%) 64.6 % of total alarms ^^vall alarms responded to 46 (3.4%) ^-± /~^ total number of ^r"""" alarm limits breached 180(25.3%) l / ^-^ alarms responded all alarms to on the first responded to auditory signal 28 (3.9%) 19(2.7%) ,_ ^^v^ total number of auditory signals 710(100%) figure 3. auditory events with suppressed co2 and respiratory alarms figure 4 shows the rate per minute of alarm limits breached and all alarm sounds (1) as currently observed in the rah data, (2) if there were no pulse oximetry limit alarms and (3) if there were neither pulse oximetry nor respiration limit alarms. pulse oximetry and respiration alarms contribute equally to the rate at which alarm limits re breached and the rate per minute of all alarm sounds. the persistence of larger alarm rates during emergence is due to the very high number of ecg and nibp disconnect signals that occurred during emergence in most operations. 3 '•« 08 0.6 0.4 ncurrent abrrre no spcc alarms no respiration or spce to spcc alarms no respiration or spcc alarm limits breached all alarm sounds figure 4. reduction in auditory events over operative phases overall, the above "what if analysis establishes that there may be some value in downplaying limit alarms in favour of continuous auditory monitoring. respiratory parameters are clearly pivotal, as the wda in figure 2 shows as well as the aims study data. the challenge for an ecological approach will be designing the continuous auditory display for end tidal co2, respiration rate and possibly also end tidal volume so that it is easy for the anaesthetist to extract the higher order-properties of patient state, as shown in figure 2. for this task we seek a configural auditory display (buttigieg & sanderson, 1991; bennett et al., 1997). moreover, the sound signal chosen for a continuous auditory display for respiratory model must coexist successfully—in terms of psychoacoustics and auditory aesthetics—with pulse oximetry sounds. combining the insights of woods (1995) with an important eid goal of making boundaries of safe operation very evident, continuous auditory display of physiological parameters should become more potentially attention-attracting as boundaries of safe regions are approached and bypassed. 113 ajis vol. 7, no 2 may 2000 current and future work we are developing continuous auditory alarms in a small-scale but versatile anaesthesia simulation environment— the body™ anesthesia simulator from advanced simulation corporation. body provides us with a testbed for research and development into alarm management systems. body contains a dynamic physiological and pharmacological model, on top of which we are constructing a fully end-user configurable research environment (body+) that allows us to develop and to empirically evaluate alternative alarm management systems. any alarm management system that performs well in laboratory tests must be evaluated in a full-scale anaesthesia simulators for there to be a basic level of confidence in its potential effectiveness for the or. acknowledgements this work was supported by an arc small grant to the third author. we thank royal adelaide hospital for permission to conduct the study and rah anaesthetists for their participation. we also thank tim purtill and matthew trentini for programming support and malcolm boyd for advice about the body anesthesia simulation. references bennett, k., nagy, a. & flach, j. (1997). visual displays. in g. salvendy (ed.), handbook of human factors and ergonomics (2nd edition). new york: wiley. bliss, j. p., gilson, r. d., & deaton, j. e. (1995). human probability matching behavior in response to alarms of varying reliability. ergonomics, 38, 2300-2313. bogner, m. s. (1994). human error in medicine. hillsdale, nj: lea. buttigieg, m. a., & sanderson, p. m. (1991). emergent features in visual display design for two types of failure detection tasks. human factors, 33, 631-651. cook, r. i. & woods, d. d. (1996). adapting to new technology in the operating room. human factors, 38, 593613. effken, j. a., kim, n.-g., & shaw, r. e. (1997). making the constraints visible: testing the ecological approach to interface design. ergonomics, 40, 1-27. flach, j., hancock, p., caird, j., & vicente, k. (1995). global perspectives on the ecology of human-machine systems (volume 1). hillsdale, nj: lea. gaver, w. (1993). how in the world do we hear? explorations in ecological acoustics. ecological psychology, 5, 283-313. gaver, w., smith, r., & o'shea, t. (1991). effecrive sounds in complex systems: the arkola simulation sound. proceedings of the acm chi'91 conference on human factors in computing systems. association for computing machinery, pp. 85-90. hansen, j. p. (1995). representation of system invariants by optical invariants in configural displays for process control. in p. hancock, j. flach, j. caird, & k. vicente (eds.), local applications of the ecological approach to human-machine systems (volume 2). hillsdale, nj: lea lin, l., isla, r., doniz, k., harkness, h., vicente, k., & doyle, j. (1995). analysis, redesign, and evaluation of a patient-controlled analgesia machine interface. proceedings of the human factors and ergonomics society 39* annual meeting. san diego, ca. october 9-13. rasmussen, j., pejtersen, a. & goodstein, l. p. (1994). cognitive systems engineering. new york: wiley interscience. reising, d. v., & sanderson, p. m. (1998). designing displays under ecological interface design: towards operationalizing semantic mapping proceedings of the 42nd annual meeting of the human factors and ergonomics society. chicago, il. 4-7 october. runciman, w. b., webb, r. k., barker, l., & currie, m. (1993). the pulse oximeter: applications and limitations—an analysis of 2000 incident reports. anesthesia and intensive care, 21, 543-550. sharp, t., & helmicki, a. (1998). the application of the ecological interface design approach to neonatal intensive care medicine. proceedings of the human factors and ergonomics society 42nd annual meeting. chicago, il. october 5-9. santa monica, ca: hfes. stanton, n. a. (1994). human factors in alarm design. london: taylor & francis. vicente, k., & rasmussen, j. (1992). ecological interface design: theoretical foundations. ieee transactions on systems, man, and cybernetics, smc-22, 589-606. woods, d. d. (1995). the alarm problem and directed attention in dynamic fault management. ergonomics, 38, 2371-2394. 114 the australasian journal of information systems (ajis) began in 1993 and was based on the simple realisation that there was a need for an academic ‘outlet’ for is research in australia australasian journal of information systems volume 17 number 1 2010 127 examining the contribution of green it to the objectives of it departments: empirical evidence from german enterprises nils-holger schmidt chair of information management, university of goettingen, germany email: nschmid@uni-goettingen.de koray erek chair of information and communication management, berlin institute of technology, germany email: koray.erek@tu-berlin.de lutz m. kolbe chair of information management, university of goettingen, germany email: lkolbe@uni-goettingen.de ruediger zarnekow chair of information and communication management, berlin institute of technology, germany email: ruediger.zarnekow@tu-berlin.de abstract the article examines the contribution of green it activities to the objectives of it departments by analysing empirical data from 116 companies using exploratory factor analysis. the outcomes indicate that green it contributes to the objectives of efficient internal operations, reputational management, and market competitiveness. in particular, reputational management plays a major role for green it engagement. these findings provide cios, it managers, and environmental officers with new insights and enable a more systematic application of green it measures. introduction enterprises are increasingly challenged by the changing demands of their stakeholders in the scope of eco-awareness and social consciousness (hewlett et al. 2009). dealing proactively with social and environmental issues is much more than just cost-driven; it can be a source of opportunity, innovation, and competitive advantage (porter & kramer 2006). this general movement has reached information technology (it) departments. there, the environmental impact of it is being discussed under the term of green it. green it has been driven primarily by business, and is seen as one of the major concerns for chief information officers (cios) (kurp 2008; molla et al. 2008). australasian journal of information systems volume 17 number 1 2010 128 scientific research and industry literature outline the importance of green it by referring to its multifaceted benefits (mines & davis 2007; molla 2008; erek et al. 2009; fujitsu australia 2009; gadatsch & juszczak 2009). kuo and dick assume that green it implementation is a response to pressures from competition, legitimation and social responsibility (kuo & dick 2010). in this research, we consider green it as an instrument of the it departments to pursue certain objectives. to explore these objectives, we investigate the underlying factors of green it benefits. describing this interdependency will provide researchers and practitioners alike with a better understanding of the topic. the article aims to answer the following three research questions.  what do it departments want to achieve with green it?  how do the objectives relate to the domains and processes of it departments?  which recommendations can be derived for researchers, cios, it managers, and environmental officers? in the next step, the research questions will be answered with the results from a large cross-sectoral survey of 116 german companies. we apply factor and correlation analysis to discover the underlying objectives. these findings contribute to existing knowledge on green it and provide practical recommendations for cios, it managers, and environmental officers, as well as new opportunities for further research in this field. theoretical background the principle of corporate sustainability sustainability has been extensively discussed within corporate management under the synonyms of corporate social responsibility (csr), greening the business, eco-efficiency or eco-advantage. although many studies concerning sustainable management have been introduced, sustainability in information systems (is) research has not been evaluated until now. global development and challenges as well as the general need to align is strategy to corporate strategy, form the need for an integrated concept of sustainability in is. in its primary sense sustainability can be described as a survival assurance meaning that an economical, environmental or social system should be preserved for future generations so that, necessary resources should only be exploited to a degree where it is possible to restore them within a regeneration cycle. the most common definition, from the brundtland commission defines sustainability as a “development that meets the needs of the present without compromising the ability of future generations to meet their own needs” (wced, 1987). all definitions of sustainability have the preservation of the economical, environmental and social system for the benefit of future generations in common. these dimensions represent the three main pillars of sustainability and are known as the “triple-bottom-line” concept (elkington, 1997). the “triplebottom-line” concept provides a framework for companies to measure and report their performance and organisational success in relation to these pillars. thus, the primary objective of a corporate sustainability program is to account for the triple bottom line. corporate sustainability is about minimizing a business‟ negative impacts on people, societies and the environment while maintaining or enhancing value for customers, business partners and shareholders. especially at the business level, sustainability is mainly equated with the economical or financial sustainability (dyllick & hockerts, 2002). however, integrated corporate sustainability is achieved by recognizing the interdependence of the three dimensions over time and keeping an optimal balance between them. the value chain of it departments in order to define the field of research, it is necessary to identify the relevant scope of is management and to outline the key activities of it departments. the it business consists of internal (in-house) and australasian journal of information systems volume 17 number 1 2010 129 external organisations that provide products and services, such as hardware, software and services that can be assigned to it departments. these types of departments generally follow the processes source, make, deliver, and return through which the value creation takes place. the management of these processes defines the scope of it management. the foundation for this process-oriented concept originated from the supply chain operations reference (scor) model (supply chain council, 2006), a well-known value chain concept in industrial management, which makes it applicable for it hardware providers. the transfer of the scor model to it software and it service providers has been done by zarnekow et al. by developing the integrated information management (iim) model (zarnekow, brenner & pilgram, 2006). the iim model focuses on the whole it value chain including customer and supplier relationship, while traditional it management concepts are focusing on the management of applications (hochstein, brenner & uebernickel, 2006) figure 1 illustrates the value chain of it business, including a return process and the stakeholders‟ interests. deliver management of customer relationship make management of service production source management of service supplier relationships source govern (organisational issues) it department it service provision (internal) customer it service acceptance m a r k e t (in te rn a l/e x te rn a l) stakeholder interests return deliver make data centre office environment internal external figure 1. value chain of the it department (source: erek et al. 2009) the govern function encompasses strategic procedures and measures which ensure that allocated it products and services contribute to the business objectives. in particular, it governance determines the supervisory functions, organisational structures, and processes. the source process covers all tasks within the supplier relationship management. usually, it departments purchase hardware components, software solutions, personnel or other technological resources. these resources are used in the production phase and are transformed to marketable it services. the make process comprises all tasks for the management of it service production. this paper focuses on the two main locations of it service production: the data centre and the office environment. attention is therefore drawn to the efficient planning, development and production of it services in these two areas. the delivery process is responsible for the management of internal and external relationships. the main objective is to meet all types of internal and external demands in an adequate manner. this not only comprises it services, but also demands for compliance or transparency. the delivery process can be seen as a mediator function between the internal make and the customers‟ source process. based on the original scor model we included a return process into the it value chain. the return phase depicts the processes of recycling, preserving and reusing tangible and/or intangible resources. it ensures a lifecycle oriented view on it services, including waste management and reutilisation of products in the value chain. australasian journal of information systems volume 17 number 1 2010 130 corporate social responsibility green it benefits on the corporate level benefits on the it department level c a te g o ri es o f m a in o b je ct iv es ( w ag n er 2 0 0 7 ) in te rn a l o p er a ti o n s  asset utilisation (figge 2005)  optimized production processes (schaltegger & synnestvedt 2002)  cost savings (schaltegger & synnestvedt 2002)   cost efficiency (erek et al. 2009; gadatsch & juszczak 2009; mines & davis 2007)  increased flexibility (erek et al. 2009)  energy efficiency (gadatsch & juszczak 2009; fujitsu australia 2009)  improved business efficiency (molla 2008)  reduced carbon emissions (molla 2008) m a rk et c o m p et it iv en es s  customer satisfaction (luo & c.b bhattacharya 2006)  buying intention (sen & c. b. bhattacharya 2001; klein & dawar 2004)  attitude towards products and services (berens et al. 2005)   influence the mindset of customers (molla 2008)  better quality of products and services (erek et al. 2009)  strategic differentiator and competitive advantage (molla 2008; fujitsu australia 2009)  facilitating innovations (gadatsch & juszczak 2009)  enabler of other green initiatives (molla et al. 2008)  increased customers (fujitsu australia 2009) r ep u ta ti o n m a n a g em en t  corporate image (yoon et al. 2006)  attractiveness for investors and business partners (tsoutsoura 2004)  workplace attractiveness (turban & greening 1997)  reputational improvement (fombrun 2005; tsoutsoura 2004; schwaiger 2004; verschoor & murphy 2002)  stronger brand (sen & c. b. bhattacharya 2001; brown & dacin 1997)   risk reduction (erek et al. 2009)  influence mindset of investors (molla 2008)  reputational improvement (erek et al. 2009)  increased staff morale (fujitsu australia 2009)  positive brand image (fujitsu australia 2009; molla 2008; gadatsch & juszczak 2009)  regulatory compliance (mines & davis 2007; molla 2008; fujitsu australia 2009) table 1: comparison of csr and green it value drivers and their possible categorization to frame the value chain of it departments, stakeholders’ interests are taken into account. the reason for doing so is that the diversity of stakeholders like shareholders, policy makers, suppliers, labour unions, customers or others, can have a major impact on corporate in this case it management performance. summing up, the model cuts the value chain into four core processes, which have to be considered simultaneously for implementing sustainability in it management. australasian journal of information systems volume 17 number 1 2010 131 green it measures can be structured regarding the strategic, operational and technical levels of an it department (schmidt et al. 2009). they also can be assigned to the value chain of it management; more specific to the source, make, deliver and return processes (molla 2008; schmidt et al. 2009). comparison of benefits from csr and green it for our green it survey we draw on research from the scope of csr. a literature review shows a broad variety of potential benefits from csr for the corporate level (table 1). wagner (2007) categorized these benefits in an empirical investigation into the three categories of objectives: internal operations, market competitiveness and reputation management. literature review on green it reveals benefits, which are comparable to those of csr. due to the similarity of the benefits, we assumethat the discovered categories of wagner (2007) are also applicable to green it. methodology research model the proposed research model as shown in figure 2 is based on the findings from the literature review. green it measures are implemented in one of the processes (govern, source, make, deliver, return) of the value chain of it departments. these measures generate certain benefits for the it department or the entire enterprise. value chain benefits objectivesgreen it im p le m en te d in g en er at es co n tr ib u te s to figure 2. the connection between green it, value chain, benefits and objectives all benefits are related to some type of overall objective. through these mediators, green it contributes to the strategic objectives of the it department. understanding the relationships helps to select adequate green it measures under the given objectives of the it department. questionnaire and statistical analysis a cross-sectional survey was conducted to evaluate the proposed research model. a paper-based questionnaire was sent out via postal mailing in september 2009. additionally, each letter included a covering letter, a self-addressed envelope and the basic definition of green it. the addressees were cios, it managers, environmental managers and staff responsibles for green it from 619 major companies listed in german stock indexes. to increase the response rate respondents were offered a report of the results. over the time of three weeks, we received 116 anonymous replies, amounting to a response rate of 18.7%. this response rate is consistent with rates in similar surveys in is research (mani et al. 2010; poppo & zenger 2002). the questionnaire was compiled based on a previous survey from the scope of csr (wagner 2007). additional questions were developed based on findings from fifteen case studies conducted on green it (erek et al. 2009). further questions were added from the survey results of fujitsu australia (2009). general questions, regarding the enterprise and the perception of green it, were integrated. a panel of five academic experts reviewed the questions to confirm that the constructs were adequately described by the item wording. the statistical analysis was done using the software spss statistics australasian journal of information systems volume 17 number 1 2010 132 17.0. all questions for the factor analysis had a 5-point likert scale ranging from “agree strongly” to “disagree strongly” (hague 2002). for the survey, a practical oriented definition of green it was provided to the participants, which tried to comprise the above-mentioned aspects under the given limitations of space and perceivability. we defined green it as follows: “green it comprises the management of all activities and measures of the it department, which are aimed to reduce the resource consumption by it, e.g. in terms of energy, material or paper. furthermore, it includes instruments to control, steer, and communicate the success”. this is a broad view of green it, which also includes management aspects. the environmental impact, e.g. in form of carbon dioxide emissions, was not pointed out explicitly since we see this as the consequence of reduced resource consumption. we apply exploratory factor analysis, which can be used to reveal underlying variables (field 2009). exploratory factor analysis is a widely utilised and broadly applied statistical method in the social sciences to discover structures in large variable sets (costello & osborne 2005). factor analysis aims to reduce complexity by looking at which variables seem to cluster together in a meaningful way (field 2009). all correlation coefficients are calculated using spearman's rho (rs) for non-parametric data. sample profile all findings are based on the sample profile shown in table 2. the participating enterprises belong to a variety of industries, such as manufacturing (35%), trade and commerce (19%), information and communication technologies (16%), and others (30%), which are representative for germany. annual turnover 2008 in millions of euros employees of the enterprise employees of the it department percent frequency percent frequency percent frequency 1-4 6% 7 1-99 9% 10 1-4 10% 12 5-9 3% 4 100-499 16% 19 5-9 8% 9 10-49 5% 6 500-999 9% 11 10-49 29% 34 50-499 25% 29 1,0004,999 30% 35 50-499 18% 21 500+ 53% 62 5,000+ 35% 41 500+ 35% 40 missing values 7% 8 n=116 n=116 n=116 table 2: turnover and employees of the responding companies the annual turnover and the number of employees indicate that large enterprises dominate the sample. the total number of employees significantly corresponds to the number of employees in the it department, rs = .74, p < .001. following zarnekow et al. (2006), the participants were asked regarding the target market of their it department. the results in table 3 show that the internal market is most relevant to it departments of the respondents. employees in the it department target market of the it department australasian journal of information systems volume 17 number 1 2010 133 percent frequency percent frequency 1 4 10% 12 internal market 76% 88 5 9 8% 9 internal and external market 16% 19 10 49 29% 34 external market 3% 3 50 499 18% 21 missing values 5% 6 500 + 34% 40 n = 116 n = 116 table 3: size and target markets of the it departments results from the empirical analysis importance and implementation for 56% of all participants, green it is an important or very important topic. the importance of green it is significantly related to the overall environmental engagement of the company, rs = 0.542, p (one-tailed) < 0.001. this illustrates that green it is seen, in the environmental context, of the enterprise. green it also correlates significantly with the number of employees in the enterprise, rs = 0.245, p (one-tailed) < 0.001, and the number of employees in the it department, rs = 0.374, p (onetailed) < 0.001. this indicates that larger companies tend to be more active in the scope of green it. green it is primarily implemented in the make process of the it value chain. there, the data centre is the most considered domain for green it (table 4). in which domains did you implement green it? (multiple answers allowed) # item value chain domain percent frequency q10.2 govern in-house training 14% 16 q10.4 remuneration of employees 2% 2 q10.9 source procurement 46% 53 q10.6 make data centre 82% 95 q10.3 office environment 48% 56 q10.5 deliver internal and external communication 32% 37 q10.1 distribution 13% 15 q10.7 marketing 12% 14 q10.8 return disposal 53% 61 n = 116 table 4: implementation and domains of green it australasian journal of information systems volume 17 number 1 2010 134 benefits and objectives to determine green it‟s contribution to the objectives of it departments an exploratory factor analysis was conducted. for the factor analysis, questions evaluating the benefits related to green it were included. the distributions of measured variables should be examined prior to conducting factor analysis (fabrigar et al. 1999). therefore, the distributions of these items were tested whether they deviate from a normal distribution. for the test of normality, the kolomogorov-smirnov test and shapiro-wilk test were applied (field 2009). with p < 0.001 for both tests, the distributions of measured variables were significantly non-normal. if the assumption of normality is violated the principal factor analysis method or principal axis factoring (paf) (in spss) is recommended (costello & osborne 2005). all variables were examined regarding their usability for factor analysis. the reliability of factor analysis and individual variables can be determined using the kaiser-meyer-olkin measure of sampling adequacy (kmo) (kaiser 1970). kmo values greater than 0.5 are barely acceptable (kaiser 1974). kmo values between 0.5 and 0.7 are mediocre, values between 0.7 and 0.8 are good, values between 0.8 and 0.9 are great and values above 0.9 are superb (field 2009). the anti-image correlation matrix provided kaiser-meyer-olkin values for each individual variable. scores below 0.5 are unacceptable and should be removed from the factor analysis (hutcheson & sofroniou 1999). therefore, one item with kmo = 0.484 was excluded. there is a debate over the criterion used to decide whether a factor should be retained for analysis (field 2009). kaiser recommended retaining all factors with eigenvalues greater than 1 (kaiser 1960). the scree plot graphs each eigenvalue (y-axis) against the factor with which it is associated (x-axis). the cut-off point for selecting factors should be at the point of inflexion of this curve. only factors left of the point of inflexion should be retained (cattell 1966). an initial analysis was run to determine eigenvalues for each factor in the sample. four factors had eigenvalues over kaiser‟s criterion of 1, with the fourth factor being just above it with an eigenvalue of 1.033. the scree plot was ambiguous and showed inflexions that would justify retaining three factors. the best choice for researchers to determine the number of factors is the scree test (costello & osborne 2005). furthermore, three factors were proposed in the research model from section 4. therefore, the three factors, efficient internal operations, market competitiveness and reputation management were retained in the final analysis. the rerun of the examination with three factors showed an equally low crossloading of one item on two factors. hence, it was dropped from the analysis as suggested (costello & osborne 2005). a principal factors analysis (paf) was conducted on the remaining 18 items with orthogonal rotation of varimax (table 5). the kaiser-meyer-olkin measure verified the sampling adequacy for the analysis with kmo = 0.828 as „great‟. all kmo values for individual items were > 0.71, which is above the acceptable limit of 0.5 (field 2009). bartlett‟s test of sphericity should be significant (p < .001) (field 2009). bartlett‟s test of sphericity χ 2 (153) = 989.295, p < 0.001, indicated that correlations between items were sufficiently large for paf. in the preliminary analysis the sample was checked for the problem of multicollinearity (field 2009). the pearson correlation coefficient matrix reported a determinant of 0.0000306 which is slightly greater than the necessary value of 0.00001, indicating that multicollinearity was not a problem. the three factors explain 52.93% of the variance. table 5 shows the factor loadings after rotation. the items that cluster on the same factors suggest that factor 1 represents the objective of efficient internal operations, factor 2 the objective of market competitiveness and factor 3 the objective of relationship management. all factors have more than three strongly loading items with 0.50 or better. therefore all factors are considered to be solid (costello & osborne 2005). australasian journal of information systems volume 17 number 1 2010 135 objectives of green it factor 1 factor 2 factor 3 #item benefit internal operations market competitiveness reputation management q32 postpone investments .742 .031 .050 q34 predict future investments .720 .195 .109 q33 extend life-span of equipment .715 -.003 .067 q23 shorter processes .669 .313 .183 q30 better forecast of costs .630 .246 .314 q21 lower capital investments .625 .108 .098 q22 better utilisation of equipment .593 .015 .201 q29 improved process quality .582 .246 .292 q24 reduced share of fixed costs .540 .209 .172 q28 lower process costs .520 .258 .267 q16 above average prices for current it products / services .083 .865 .155 q18 more sales of current it products / services .118 .747 .318 q17 above average prices for future it products / services .192 .722 .253 q20 gaining competitive advantage .368 .522 .153 q27 decision criterion for the buyer of it products / services .178 .337 .718 q26 quality indicator for selecting contract partners .214 .210 .687 q25 competence signal of the it department .212 .148 .665 q36 improved reputation of the it department .115 .136 .651 eigenvalues 4.400 2.677 2.453 % of variance 24.426 14.874 13.635 cronbach‟s α .889 .847 .816 most important factor for respondents in % (n = 103) 27.2 35.0 37.9 note: factor loadings over .40 appear in bold table 5: benefits of green it and related objectives australasian journal of information systems volume 17 number 1 2010 136 when using factor analysis the consistency of the questionnaire should be checked using cronbach‟s α, which should be around 0.8 (field 2009). the factors representing efficient internal operation, market competitiveness, and relationship management have high reliabilities with cronbach‟s α = 0.889 / 0.847 / 0.816. the objective of efficient internal operations is achieved by the following green it benefits: to postpone investments, better planning of future investments and costs, extended life-spans of it equipment, shortened processes, better utilisation, improved quality and lower process costs (table 5). market competitiveness is achieved by to the opportunity to ask for higher prices and sell more products or services (table 5). green it serves also as a quality indicator and a competence signal which increases the reputation of the it department (table 5). even though the factor of efficient internal operation explains the greatest variance (24.4%) the majority of respondents (37.9%) perceive the contribution of green it to the objective of reputation management as most important, followed by market competitiveness (35.0%) and efficient internal operation (27.2%). domains and objectives table 6 highlights the correlations of the revealed objectives with the domains in which green it is applied. each objective is significantly positively correlated with a compatible domain. efficient internal operation is related to green it in the data centre, rs = 0.276, p (one-tailed) < 0.001, but not to the office environment. on the other hand, reputation management is linked to the office environment, rs = 0.250, p (one-tailed) < 0.01. reputation management is also correlated to communication, rs = 0.321, p (one-tailed) < 0.001, and the overall number of domains where green it is applied, rs = 0.310, p (one-tailed) < 0.01. market competitiveness shows a significant link to the domain of distribution, rs = 0.241, p (one-tailed) < 0.01. factor 1 factor 2 factor 3 # item value chain domain internal operations market competetiveness reputation management q10.2 govern in-house training .084 -.002 .15 q10.4 remuneration of employees -.021 .213* -.066 q10.9 source procurement .030 .098 .165* q10.6 make data centre .276** -.031 .212* q10.3 office environment .000 .226* .250** q10.5 deliver internal and external communication .191* .188* .321*** q10.1 distribution .000 .241** .119 q10.7 marketing .172* .132 .082 q10.8 return disposal -.067 -006 .029 number of green it domains .136 .175* .310** australasian journal of information systems volume 17 number 1 2010 137 note: spearman‟s correlation coefficients. significance of correlation coefficients, * p < .05 (1tailed), ** p < .01 (1-tailed), *** p < .001 (1-tailed) table 6: correlations between green it domains and objectives of the it department implications from these results in section 4.2 it can be concluded, with certain limitations, that green it measures and activities contribute to the objectives of an it department, which are efficient internal operations, market competitiveness, and reputation management. from the revealed objectives in section 4.2, reputational management (37.9%) is the most important, followed by market competitiveness (35.0%), and efficient internal operation (27.2%). as these outcomes indicate, doing green it is not just necessarily aimed at achieving efficient internal operations, for reducing costs or emissions. doing green it is rather used as a tool to signal own competence, quality or reputation to important relationship collaborates, such as suppliers and employees. considering the factor of market competitiveness, it is also aimed at increasing the sales and prices of products or services. the objectives are all correlated to the application of green it in specific domains (section 4.3). efficient internal operation is related to green it in the data centre but not to the office environment. this could be done because benefits might be easier to realize with greater impact in the data centre. on the other hand, reputational management is linked to the office environment, communication and the overall number of domains where green it is applied. an explanation for this might be that green it measures in the office environment are aimed to create involvement, positively influence the employees‟ opinion about the idea of environmental protection and to demonstrate own competence. market competitiveness is correlating with green it in the domain of distribution, which seems useful when products or services are advertised. the results provide cios, it managers, and environmental officers with recommendations on how to efficiently implement green it into an it strategy and an overall business strategy. considering the business and it strategy, cios and it managers should decide which of the objectives efficient internal operations, reputation management, and/or market competitiveness are most relevant for doing the business. based on this analysis they should implement green it in the adequate domains of the it department value chain. limitations and conclusion there are a number of clear limitations in these preliminary findings. foremost is the exploratory nature of the research and small sample size that limits generalisability of these findings. the sample selection and the applied statistical methods impose specific limitations to the results. due to the sample selection the results of this study are only representative for german it departments. the perception of green it might vary from region to region (molla et al. 2008). limitations also derive from the application of principal factor analysis. conclusions are restricted to the sample collected and generalizations of the results can be achieved only if analysis using different sample reveals the same factor structure (field 2009). furthermore, it has to be noted that showed correlations do not imply causality between two variables because there may be other measured or unmeasured variables affecting the results (field 2009). to test for non-response bias, the responses were split into two groups based on their chronological return. there is a difference between early and late respondents when cross-tabulated with, “extent of green it planning and implementation”: 61% of the early respondents had “planned” or “implemented and planned” green it, but this proportion decreases to australasian journal of information systems volume 17 number 1 2010 138 29% of the late respondent group. the chi-square statistic was significant at p < .05. an explanation for this could be that respondents who are currently planning green it activities are more likely to answer the questionnaire. it can be assumed that non-response bias has not caused any significant problems with the study. despite the given limitations, these results provide insights and guidelines for cios, it managers, and environmental officers for applying green it into their daily business. in this paper we have shown the contribution of green it to the objectives of it departments. we provided theoretical foundation to the topic and suggested practical recommendations for implementing green it along the value chain of it departments. this enables cios, it managers, and environmental officers to get further insights about green it and to take the appropriate actions to link green it with the it and business strategy. the results need further investigation and validation. therefore, future research should replicate the survey in other countries to determine if the results are transferable. the outcomes should be discussed by conducting case studies and expert interviews. green it is able to contribute to the it as well as to the business strategy. the development of a procedural model defining tasks, roles and responsibilities in the scope of green it for the entire enterprise seems useful. it departments have to be aware that environmental topics need to be addressed in order to remain competitive in the future. given the rising prices for energy and other resources and the increasing awareness of all stakeholders to environmental issues, the relevance of green it is destined to gain even more importance in the future. hence, the concept of environmentally oriented it has an increased relevance for policy makers, practitioners, and researchers. references berens, g., van riel, c.b.m. & van bruggen, g.h., 2005. corporate associations and consumer product responses: the moderating role of corporate brand dominance. journal of marketing, 69(3), 35-18. brown, t.j. & dacin, p.a., 1997. the company and the product: corporate associations and consumer product responses. journal of marketing, 61(1), 68-84. cattell, r., 1966. the scree test for the number of factors. multivariate behavioral research, 1(2), 245-276. costello, a.b. & osborne, j.w., 2005. best practices in exploratory factor analysis: four recommendations for getting the most from your analysis. practical assessment, research & evaluation, vol. 10(7). available at: http://pareonline.net/pdf/v10n7.pdf. dyllick, t. & hockerts, k., (2002) beyond the business case for corporate sustainability, business strategy and the environment, 11, 130-141. elkington, j., 1997. cannibals with forks: the triple bottom line of 21st century business, capstone, oxford. erek, k., schmidt, n.-h., zarnekow, r. & kolbe, l. m., 2009. sustainability in information systems: assortment of current practices in is organizations. amcis 2009 proceedings. available at: http://aisel.aisnet.org/amcis2009/123 [accessed november 4, 2009]. fabrigar, l.r. et al., 1999. evaluating the use of exploratory factor analysis in psychological research. psychological methods, 4(3), 272-299. field, a., 2009. discovering statistics using spss third edition., sage publications. australasian journal of information systems volume 17 number 1 2010 139 figge, f., 2005. value-based environmental management. from environmental shareholder value to environmental option value. corporate social responsibility and environmental management, 12(1), 19-30. fombrun, c.j., 2005. building corporate reputation through csr initiatives: evolving standards. corporate reputation review, 8(1), 7-11. fujitsu australia, 2009. green it: the convenient truth. available at: https://wwws.fujitsu.com/au/whitepapers/convenient_truth.html. gadatsch, a. & juszczak, j. eds., 2009. ergebnisse der kurzumfrage zum stand von green it im deutschsprachigen raum, sankt augustin. available at: http://www.fb01.hbrs.de/wirtschaftsanktaugustinmedia/downloads/schriftenreihe/bd24web.pdf. hague, p.n., 2002. market research, kogan page publishers. hewlett, s.a., sherbin, l. & sumberg, k., 2009. how gen y & boomers will reshape your agenda. harvard business review, 87(7/8), 71-76. hochstein, a., brenner, w. & uebernickel, f., 2006. operations management and is: using the scor-model to source, make, and deliver is services, in: proceedings of the twelfth americas conference on information systems, august 04 – 06, acapulco, méxico, indiana university bloomington. hutcheson, g. & sofroniou, n., 1999. the multivariate social scientist, sage. kaiser, h.f., 1970. a second generation little jiffy. psychometrika, 35(4), 401-415. kaiser, h.f., 1974. an index of factorial simplicity. psychometrika, 39(1), 31-36. kaiser, h.f., 1960. the application of electronic computers to factor analysis. educational and psychological measurement, 20(1), 141-151. klein, j. & dawar, n., 2004. corporate social responsibility and consumers' attributions and brand evaluations in a product-harm crisis. international journal of research in marketing, 21(3), 203-217. kuo, b.n. & dick, g.n., 2010. the greening of organisational it: what makes a difference? australasian journal of information systems, 16(2), 81-92. kurp, p., 2008. green computing. communications of the acm, 51(10), 11. luo, x. & bhattacharya, c., 2006. corporate social responsibility, customer satisfaction, and market value. journal of marketing, 70(4), 1-18. mani, d., barua, a. & whinston, a., 2010. an empirical analysis of the impact of information capabilities design on business process outsourcing performance. mis quarterly, 34(1), 3962. mines, c. & davis, e., 2007. topic overview: green it. available at: http://www.forrester.com/research/document/excerpt/0,7211,43494,00.html [accessed november 5, 2009]. molla, a., 2008. gitam: a model for the adoption of green it. australasian conference on information systems 2008, 658-668. molla, a. et al., 2008. e-readiness to g-readiness: developing a green information technology readiness framework. australasian conference on information systems 2008, 669-678. poppo, l. & zenger, t., 2002. do formal contracts and relational governance function as substitutes or complements? strategic management journal, 23(8), 707-725. porter, m.e. & kramer, m.r., 2006. strategy & society: the link between competitive advantage and corporate social responsibility. harvard business review, 84(12), 78-92. australasian journal of information systems volume 17 number 1 2010 140 schaltegger, s. & synnestvedt, t., 2002. the link between 'green' and economic success: environmental management as the crucial trigger between environmental and economic performance. journal of environmental management, 65(4), 339-346. schmidt, n.-h., erek, k., kolbe, l. m. & zarnekow, r., 2009. towards a procedural model for sustainable information systems management. in hawaii international conference on system sciences. pp. 1-10. schwaiger, m., 2004. components and parameters of corporate reputation an empirical study. schmalenbach business review (sbr), 56(1), 46-71. sen, s. & bhattacharya, c.b., 2001. does doing good always lead to doing better? consumer reactions to corporate social responsibility. journal of marketing research (jmr), 38(2), 225-243. supply chain council, 2006. supply-chain operations reference-model (scor): overview version 8.0., supply-chain council inc., pittsburgh, washington dc. tsoutsoura, m., 2004. corporate social responsibility and financial performance. available at: http://escholarship.org/uc/item/111799p2. turban, d.b. & greening, d.w., 1997. corporate social performance and organizational attractiveness to prospective employees. the academy of management journal, 40(3), 658672. verschoor, c. & murphy, e., 2002. the financial performance of large u.s. firms and those with global prominence: how do the best corporate citizens rate? business & society review (00453609), 107(3), 371-380. wagner, m., 2007. der sustainable economic value von sozialer nachhaltigkeit und umweltmanagement: konzept und empirische anwendung. in m. müller & s. schaltegger, eds. corporate social responsibility: trend oder modeerscheinung? oekom, pp. 229-245. yoon, y., gürhan-canli, z. & schwarz, n., 2006. the effect of corporate social responsibility (csr) activities on companies with bad reputations. journal of consumer psychology (lawrence erlbaum associates), 16(4), 377-390. wced world commission on environment and development, 1987. our common future, oxford university press, uk and new york. zarnekow, r., pilgram, u. & brenner, w., 2006. integrated information management: applying successful industrial concepts in it, 1st ed., springer, berlin. editorial this issue of the ajis contains papers from australia and chile and is unique, in that it not only contains contributed papers, but three papers judged to be amongst the best ten from the recent 5th australian conference on information systems (acis) held in melbourne last september. they were chosen on the basis of maximum points awarded to them by the referees. due to space restrictions, only three of the papers are published in this edition, those by darke & shanks, broadbent & butler and hasan & gould. the others will be included in the september edition (vol 3 no 1). the first paper by alan colman and ying leung from swinburne university of technology is from the field of human computer interaction and proposes a modified version of johnson's knowledge analysis of tasks. as well as applying the modified form to intelligent distributed control systems they also show how it can be useful in object-oriented analysis. paper two is from jose pino and jose montanana in chile and deals with a software system designed to facilitate interactive communication between negotiating teams. this is particularly relevant in light of the improved teleconferencing facilities promised by the information superhighway. john maltby from southern cross university presents an interesting and unusual analysis of the windows environment in order to quantify operational complexity thus enabling interface dialogue to be described as markov process. the fourth paper by peta darke and graeme shanks from the acis conference focuses on the niam conceptual design process. they propose that viewpoint development, one approach to the socio-organisational dimension of information systems development, be incorporated into the niam procedure. paper five by marianne broadbent and carey butler, also from the acis conference, highlights the experiences of a number of australian firms which have implemented business process redesign. ten lessons for would be implementers of this process emerged from this study. these are listed and explained. fiona rohde's paper gives an ontological evaluation of the jackson system development methodology in terms of the wand and weber model so that its strengths and weaknesses can be evaluated. the last paper also from the acis conference by hasan and gould describes a longitudinal study, conducted over a period of four years, of a company as it went through the stages of implementing an information system for use by managers. i would like to thank all the authors and consulting editors as well as the convenors of the acis conference for their help in producing this journal. their assistance has been particularly valuable while the regular editor, rob macgregor, is on study leave. ted gould acting editor microsoft word vol 15 2 final.doc australasian journal of information systems volume 15 number 2, 2009 109 a survey of current e-business (e-government) development practices in australia don kerr university of the sunshine coast e-mail:dkerr@usc.edu.au kay bryant griffith university e-mail: k.bryant@griffith.edu.au abstract the internet is a dynamic part of the business scene and there have been many examples of success and failures of e-commerce and e-business ventures. this research looks at the attitudes of top level executives (notably the chief information officer) to ecommerce and the level of adoption of the various aspects of ecommerce by firms in australia. to achieve this aim, a survey was administered asking questions about industries understanding of current e-business (e-government) practices. the object of the survey was to provide a benchmark of current practice across a wide range of industries. the results of this survey could have a major impact on academic curricula. the survey was sent to 671 australian businesses and was addressed to the chief information officer. only twenty four usable questionnaires were returned and while it is accepted that this is a limited sample, disturbing trends have appeared with respect to the lack of strategic planning for information technology use particularly in medium sized businesses in australia. in addition, relationships between key factors associated with the business objectives for their ict infrastructure and the benefits of the organisation’s extranet, intranet and internet initiatives were identified. introduction in the last two decades of the 20th century, affordable and powerful computer hardware and software along with the advanced telecommunication technology have laid a solid foundation for the emergence and popularity of the internet. individuals as well as organisations have quickly adopted the internet as a communication tool for data transmission and the provision of information and services. the internet is fast, simple, and convenient without the limitation of distance and time. it is a dynamic part of the business scene and there have been many examples of success and failures of e-commerce and e-business ventures. in the past, the us has been leading the world in the use of the internet for business activities. as such, much of the research conducted into ecommerce and e-business has been centred in the us, but it is timely for studies to be done in other parts of the world. this research looks at the attitudes of top level executives (the survey was addressed to the chief information officer) to ecommerce and the level of adoption of the various australasian journal of information systems volume 15 number 2, 2009 110 aspects of ecommerce by firms in australia. the approach was to administer a survey asking questions about current e-business (e-government) practices. the object of the survey was to provide a benchmark of current practice by providing an overview of attitudes to the overall organisational goals of e-business and the methods and hardware infrastructure used by australian firms across a range of industries. this paper begins with an overview of the world’s e-readiness and how organisations have adopted the internet for business transactions. next, the research that was undertaken is described. findings are presented in the third section, where the extent and manner of organisational adoption of ebusiness is reported. the final section provides a discussion of these results. internet usage substantial proportions of individuals and organisations in developed countries are online or at least have internet access. current research conducted by the economist intelligence unit on the ereadiness of the world’s largest economies is published in its white paper (eiu, 2007). the definition of e-readiness used by the eiu is “that each country possesses an interconnected set of infrastructural, political, commercial, legal and social attributes that, when combined effectively, help the economy grow and government and society improve” (pg 2). in this report, eiu has made some changes to how it assesses e-readiness, with some consequential movements among the rankings. the eiu research shows the us holding its second place behind denmark; a place it now shares with sweden. australia was ranked ninth out of 69 countries; one place down from the previous year but still one place ahead of the 2005 rankings. nine of the top ten countries from the previous year still remain so although the rankings have changed. the net affect is that while ereadiness is progressing around the world, its achievement is becoming more complex. the eiu research has developed a score that relates to a country’s e-commerce readiness. this score is out of 10 and is derived from six factors measuring social, political, economic and technological development. nonetheless, the differential between the top 20 countries, with an average score of 8.4, and the bottom 20 countries, whose score is 4.1, is diminishing. overall, the world’s ereadiness is improving as many developing countries are committing significant resources to developing their digital economy. the department of communications, information technology and the arts present, in their 2005 report (dcita, 2005), the situation in australia. dcita reports that 62% of households and 74% of all businesses had internet access. further, 25% of businesses had web sites. of those businesses that are online, 50% still used dial-up connections while 41% used broadband and the remaining 9% used isdn. with respect to broadband, the vast majority of business used either dsl (67%) or cable (28%) technologies. the entry level costs for broadband had increased marginally (less than 1%) from the previous year, while the entry level access for cable decreased by slightly more than 7%. the report went on to say that australia remains one of the countries with the lowest entry level prices for broadband and cable access. dcita also reported on the online activities of all businesses within australia. the majority of online businesses used the internet to place orders for goods and services (42%); and lodge electronic payments (28%) and taxation forms (26%). most small to medium enterprises (smes) are active in e-commerce/e-business activities. while the majority of these activities such as paying for (62%); receiving payments for (50%), and taking orders for (41%) goods and/or services increased from the previous year, placing orders for goods and/or services decreased slightly (53%). internet banking remaining the most used activity (70%). the report contained an estimate for business internet income of $33 billion, which was $8.7 billion up from the previous year. this total business income figure was equivalent to 2% of total business income for the surveyed firms australasian journal of information systems volume 15 number 2, 2009 111 and 4% of gross domestic product. these figures can be considered as a moderate achievement, however, they are well below the conservative forecasts of 15% of the late 1990s (markham, 1998). neither the eiu report nor the dcita report provides information on how e-business has been incorporated into organisations. the most significant challenge organisations are currently facing is how to incorporate information and internet technologies into their business activities so that an overall improvement in effectiveness and efficiency is achieved, and so influence the strategic positioning of the organisation. this paper seeks to identify how organisations are incorporating ebusiness into their business activities and identify the resources used to achieve this. further, this paper identifies the objectives behind the e-business initiatives and the benefits that are realised. the literature mckay and marshall (2004) suggest that at a strategic level, a shared vision of purpose, goals and objectives is imperative, and that a strategic plan should be an ongoing activity that is essential to ensure effective ict3/business alignment. this ict/business alignment is essential for companies implementing any information system and this lack of alignment is a well recognised management problem within many companies (roepke, agarwal and ferratt 2000). the lack of alignment is also a major criticism of ict and one of the major aspects that nicolas carr considered in relation to the failure of ict to provide competitive advantage (carr 2003). carr defines ict as an infrastructural technology in that it needs to be shared and cannot be used in isolation. in the early stages infrastructural technology is restricted and therefore provides the user some competitive advantage, however as the previously scarce resource becomes more available, it loses its competitive advantage and becomes ubiquitous and eventually a commodity that everyone has and hence offers little if any competitive advantage (much like electricity is provided as a service). another dimension to the business application of ecommerce is the level of risk and adaptation to change a company is willing to undertake. hirschheim and sabberwal (2001) suggest that organisations can be placed in one of three business stances. these stances are (1) prospector, where the company takes a somewhat aggressive, entrepreneurial stance to their business dealings; or (2) defender where the company establishes itself in a niche market and is usually resistant to change; or (3) analyser, that is midway between the two having a cautious approach to risk but with a more entrepreneurial approach to business dealings than the defender stance. these different stances are applicable to this study and are used to categorise the survey results in the discussion section of this paper. with respect to this study and implicit in the survey questions (appendix one) is porter’s competitive strategy five forces model (porter 1985). this five forces model has been adapted by gendron (2006) to look at the potential impact on web enablement. the five forces are: [1] threat of new entrants [2] industry competition and rivalry among existing companies [3] suppliers and bargaining power [4] buyers and bargaining power [5] substitute products 3 as is convention ict (information and communication technology) and it (information technology) are used interchangeably. australasian journal of information systems volume 15 number 2, 2009 112 the survey instrument contained questions relating to all five forces and how these relate to responses is discussed in the discussion section of this paper. other studies teo, lin and lia (2008) report on the e-procurement adoption levels in singapore. they used a similar approach as this study, namely a survey questionnaire to determine the factors associated with adoption. their findings revealed that the size of the firm, top management support, indirect benefits and the influence of business partners were significantly associated with adoption in a positive way. in another study, al-qirim (2007) uses a regression approach similar to that used in this study to analyse adoption levels of ecommerce in new zealand. this study showed that the innovativeness of the chief executive officer had a major influence on many ecommerce technology adoptions within small enterprises (sme’s) in new zealand, while relative advantage and competition were significantly associated with more outward focused technologies such as extranets. the study also found that pressure applied by suppliers was the only determinate for electronic data interchange (edi) adoption amongst these sme’s. both these studies looked at factors affecting adoption levels of ecommerce in their respective countries, they did not look at the existing use of the technology and the attitudes of top executives to the technology itself as is the case in this study. methods an extensive eight-page questionnaire was developed to survey organisations about their e-business (and e-government) initiatives and development practices. the survey had ethics approval and respondents were assured that all information would be made anonymous and that only de-identified summary statistics would be reported. the survey was conducted in june and july 2007. the object of the survey was to provide a benchmark of current practice. the survey instrument was developed by california state university and contained five components (tsai and ching 2003). the first component sought information about organisational characteristics, how they classified their e-business goals and what objectives they wanted to achieve with their initiatives. the second component sought information on the key decision makers and other participants to the decisionmaking process. the ict infrastructure was the focus of the third component of the instrument. the physical components that make up the ict infrastructure were sought along with tools used for data mining and communications; the information management functions performed within the organisation and functions that were outsourced. the component also required the identification of areas impacted by the implementation of e-business initiatives. the fourth component focused on network usage, particularly the use of the internet, extranets and intranets by the organisation. information on the benefits realised from these initiatives and how performance or outcomes were measured were also sought. the final component asked for name and address details on the organisation as well as the name and title of the person completing the questionnaire. the final part of the component provided the respondent with a space where they could write any comments they believed were relevant to the study. questionnaires were sent to the top level information management staff, both in the private and public sectors, throughout the australia. the list of organisations was obtained from a commercial company specializing in mailing lists. ethical clearance was requested and obtained from the university before the questionnaires were sent out. the target population for the survey was the australasian journal of information systems volume 15 number 2, 2009 113 chief information officer of state government offices, local government offices, and corporations through out australia. a total of 671 questionnaires were sent out to organisations via the australian postal service. data analyses were conducted on the data related to business objectives for ict infrastructure and benefits of the organisation’s extranet, intranet and internet initiatives. the analyses involved descriptive statistics including correlations and regression analysis. results the survey was divided into five components, namely, organisational characteristics, the personnel involved in decision making, the information infrastructure, network usage and contact details of participating organisations. this section in the paper provides details of responses to questions for all but the last component of the survey. details on the participating organisations are not provided to maintain anonymity of the respondents. unfortunately, only twenty-four valid responses were returned, resulting in a response rate of 3.58%. organisational details the first four questions asked about organisational characteristics including, gross revenue, number of employees and whether the employees were involved in ict or e-business development. the australian bureau of statistics provides a classification scheme for organisations based on the number of employees: small where the number of employees are below 20, medium were the number of employees are between 21 and 200; and large where the number of employees is above 200 (abs, 2005, pg 11). participating organisations were classified accordingly; however, only two categories were evident – medium-sized and large-sized. table one shows the average of these variables across the two categories as well as the range of values. the gross revenue of the companies involved varied from $3 million to $500 million and the number of staff ranged from 30 to 2600. annual gross revenue was not reported by two respondents and this may be due to concerns about privacy. characteristic medium-sized average range large-sized average range annual gross revenue (millions) 8.2 3 18 303.0 40 1500 total employees 50.8 0 83 1059.0 215 2600 it employees (percentage of total employees) 13.4 30 87 11.0 .01 70 e-business employees (percentage of it employees) 6.9 0 40 9.5 0 30 number of participating organisations 9 15 table one characteristics of the participating organisations ($aud) descriptive statistics responding organisations were split with respect to their business activities; half were involved in government at the state or local level; while the remaining organisations were involved in private australasian journal of information systems volume 15 number 2, 2009 114 enterprise. the range of activities for those organisations involved in private enterprise included: health, energy retailing, manufacturing, printing, real estate, import and/or distribution, design, consulting, plant maintenance, and timber milling. the respondents were split equally based on whether they had a written three-year strategic technology plan or not. when the respondents were asked to categorize their e-business goals as described by ward and peppard (2002, pg 42). they categorised systems in terms of their role in the organisation as shown below: • strategic – applications that are critical to achieving future business strategy • key operational – applications that the organisation currently depends on for success • high potential – applications that might be important in achieving future business success • support – applications that are valuable but not critical to business success the majority of respondents classified their systems as strategic (46%). other classifications were high potential (25%), key operational (25%) and support systems (33%). respondents were able to select more than one category. in classifying their e-business goals, seven of the large-sized organisations reported their application portfolios as being strategic, one as a combination of strategic and high potential, three as high potential only, three as support systems and two as key operation systems. the medium-sized organisations classified their e-business goals as strategic and key operational (four each), five as support systems and two as high potential. of these, two respondents used more than one category to describe their e-business goals; one combined strategic with key operational and the other used all four categories. medium-sized organisations tended to classify their e-business goals as being either key operational or support systems more often than large-sized organisations that used strategic and high potential categories more often. the final question in the organisational section related to the objectives for the ict infrastructure. these are shown in table two. a likert-type scale was used for the rating; from 1 not important to 5 very important. the final column in the table provides a comparative ranking of the objectives, from least (1) to most (9) important. the most important aspects were reliability, integrity, security, availability and user friendliness, while instantaneous response and portability were ranked least important. objectives average rating relative importance portability 2.9 1 instantaneous response 3.5 2 adaptability 3.7 3 flexibility 3.7 3 scalability 3.7 3 manageability 3.8 4 manageability 3.8 4 maintainability 3.9 5 functionality 4.3 6 availability 4.4 7 user-friendly 4.4 7 security 4.6 8 integrity 4.6 8 reliability 4.7 9 table two – objectives for ict infrastructure based on a 5 point likert score (1 being least important to 5 most important). australasian journal of information systems volume 15 number 2, 2009 115 personnel the second component of the questionnaire contained two questions designed to identify which personnel were involved in their organisation’s e-business strategic planning. respondents could select more than one option for both questions. the first question focused on the key management decision makers. respondents named the chief executive officer as the primary decision maker (75.0%), followed by the chief financial officer (45.8%), and the chief information officer (12.5%). the second question focused on the key participants in the design and implementation aspects of their organisation’s e-business strategic planning not including traditional system development personnel. the respondents named external ict specialists (58.3%) and external business consultants (41.7%) as the main participants, followed by information security officers (29.2%) and legal consultants (16.7%). ict infrastructure the third component of the questionnaire focused on the information technology infrastructure. again, respondents could select more than one option in each question. the first question related to the physical components of the e-business infrastructure. there were three aspects to this question: hardware, software and database structure. one respondent did not provide any information on the hardware question and three others did not provide a response to the software or database structure elements. as these were technical questions using some industry jargon, the non ict specialist respondents had difficulty in answering them and chose to leave them blank. the vast majority of hardware components were used by almost all respondents. applications servers and database servers were used by all who responded, and backup servers and firewalls were used by 87.5% of respondents. email servers (83.3%), web servers (79.2%) and routers/switches (75%) were the next most used hardware components. storage servers and printer servers were used by just over half of the respondents. server support applications (62.5%) were the primary software components followed by web server support applications (50%) and middleware (16.7%). the relational database structure was selected by a majority of the respondents (91.7%) with the network database (66.7%) the second most common structure used. the object-oriented structure was used by 16.7% while the hierarchical and other structures were each employed by 12.5% of the respondents. the next two questions related to the use of data mining tools for e-business and the primary method of communication with their online customers. most respondents employed data mining tools for combined customer/product/service purposes (62.5%). email (58.3%) and web pages (58.3%) were the primary forms of communications with online customers. telephone (29.2%) and call centres (8.3%) were the least selected options for communications with online customers. two questions focused on the types of information management functions undertaken within the organisation and which functions were outsourced. responses to the questions are shown in table three. one respondent did not provide information on the types of information management functions undertaken. this particular survey response was completed by the general manager who indicated that “this survey used jargon not fully understood by someone not involved daily with it”, thus indicating that he may have had some confusion about the functions listed. while four respondents, two at the ceo level, one at the cio level and another being the director of sales and marketing did not provide a response to the functions outsourced question. as all these respondents were top level executives and they were able to fill in all the other questions accurately, we consider this non response to indicate that none of the functions mentioned on the survey question were outsourced by those respective firms. australasian journal of information systems volume 15 number 2, 2009 116 content management, contingency management and security management were the three functions selected most often. emerging technology management was selected by only three of the respondents. outsourcing in one form or another was undertaken by most of the participating organisations. traditional systems maintenance, system integration, and database management were outsourced most, while customer relationship management was not selected at all. e-business development was outsourced by 37.5% of respondents, while e-business maintenance was outsourced by only 25.0% of respondents. information management function performed percentage of respondents relative importance content management 91.7 1 contingency management 70.8 2 security management 62.5 3 customer relations 58.3 4 inventory management 58.3 4 documentation management 58.3 4 vendor management 54.2 5 performance management 50.0 6 knowledge management 45.8 7 development management 29.2 8 integration management 20.8 9 emerging technology management 12.5 10 business functions outsourced percentage of respondents relative importance traditional systems maintenance 62.5 1 system integration 58.3 2 database management 41.7 3 e-business development 37.5 4 off-the-shelf program evaluation 37.5 4 systems evaluation 33.3 5 network management 29.2 6 e-business maintenance 25.0 7 data warehouse/data mining 20.8 8 e-mail system management 20.8 8 contractor solicitations 16.7 9 internal end user support 12.5 10 call centre 8.3 11 contractor management 8.3 11 customer relationship management 0.0 table 3: – the percentage who responded to each factor and their relative importance (rank) for information management functions performed within the organisation and business functions that have been outsourced. the most important having a relative importance ranking of 1. the final question in this component of the survey questionnaire identified which areas were impacted by the implementation of e-business systems within the organisation. the main areas impacted were ict infrastructure (75%), business processes (70.8%) and resources (70.8%). australasian journal of information systems volume 15 number 2, 2009 117 operation processes and strategic planning were selected by 54.2% and 50% of the respondents respectively, while management decisions (37.5%) was least affected by e-business initiatives. network usage the fourth component of the questionnaire focused on network usage. there were five questions in this component. respondents could select more than one option for the first three questions and the last one; while the fourth question required the respondent to rate the benefits of its e-business initiatives. the first question asked about the organisation’s reasons for using the internet. the responses are shown in table four. internet usage percentage of respondents relative importance e-mail 91.7 1 web page presence 87.5 2 business-to-consumer 65.5 3 marketing and advertising 58.3 4 business-to-business 41.7 5 e-government 37.5 6 e-education 25.0 7 management driven 16.7 8 customer-to-customer 12.5 9 table four – the percentage who responded to each factor and their relative importance (rank) for the factors associated with the reasons for internet usage within organisations. the most important having a relative importance ranking of 1. the goals associated with the organisation’s use of extranets were the focus of the next question. table five shows the responses. the most commonly selected goal by far was customer service and support, whereas production and inventory control between business partners was the least selected goal. extranet usage within the organisation percentage of respondents relative importance customer service and support 75.0 1 post electronic forms 45.8 2 products/services catalogue information 41.7 3 electronic funds transfer 37.5 4 electronic data interchange 29.2 5 portal 25.0 6 marketing and advertising 25.0 6 collaboration between business partners 16.7 7 sales 16.7 7 publish information among business partners 12.5 8 production and inventory control between business partners 0.0 table five – the percentage who responded to each factor and their relative importance (rank) for the factors of goals of the organisation’s use of extranets. the most important having a relative importance ranking of 1. australasian journal of information systems volume 15 number 2, 2009 118 the next question related to the objectives associated with the organisation’s use of intranets. table six shows the responses. the intranet was primarily used to broadcast information. software distribution and management and inbound logistic systems were the least selected objectives of the intranet. intranet usage within the organisation percentage of respondents relative importance broadcast information 79.2 1 post electronic forms 62.5 2 education and training 54.2 3 document management 54.2 3 team collaboration 41.7 4 inbound logistic system 20.8 5 software distribution and management 8.3 6 table six – the percentage who responded to each factor and their relative importance (rank) for the factors of objectives of the organisation’s use of intranets. the most important having a relative importance ranking of 1. the fourth question required the respondents to rate the importance of the benefits realised from their extranet, intranet and internet initiatives. a likert-type scale was used for to rate the benefits; from 1 not important to 5 very important. the final column in the table provides a comparative raking of the benefits, from least (1) to most (11) important. table seven provides the results. the most important benefits were increased productivity, efficiency, enhanced communications and increased sales and marketing efforts. the least important benefits were supply chain coordination, competitive advantage and industry leadership. objectives average rating relative importance supply chain coordination 2.22 1 competitive advantage 2.72 2 industry leader 2.91 3 customer retention/loyalty 2.95 4 increased profitability 3.18 5 cost reduction 3.32 6 better decision making abilities 3.55 7 customer satisfaction 4.00 8 enhanced communication 4.09 9 increased sales and marketing efforts 4.09 9 efficiency 4.13 10 increased productivity 4.22 11 table seven– the percentage who responded to each factor and their relative importance (rank) for the factors of benefits realised by the organisation’s use of extranets, intranets or the internet australasian journal of information systems volume 15 number 2, 2009 119 the final question focused on how the organisations measured the performance or outcomes of their e-business initiatives. productivity was selected by the majority of respondents (66.7%), while profit and cost was selected least; 20.5% and 25 % respectively. one respondent indicated that this could not be measured by them. further analysis – business objectives for ict infrastructure correlations and regression analysis were undertaken with respect to the business objectives of the information technology infrastructure. the correlation matrix is shown in table eight. strong correlations are evident for adaptability against flexibility and scalability, indicating the obvious conclusion that more scalable, flexible systems are considered more adaptable. other strong correlations were for manageability against functionality and maintainability, again indicating that systems with better functionality as well as being able to be maintained more easily were more manageable. a strong negative correlation was identified between systems integrity and portability. the more portable a system was, the more concern was expressed about the overall integrity of the system. there was also a strong positive correlation between integrity and reliability with more reliable systems being judged as more sound. another significant correlation was between user friendliness and security with a strong positive correlation indicating that secure systems were thought to be more user friendly (table eight). perhaps this is not so surprising when we consider that the survey was completed by cio’s who would have a vested interest in having secure systems. the final significant correlation was a negative one between instant response and maintainability, indicating that the process of systems maintenance may be a contributing factor to slower response times. business objectives adapt ability avail ability flexib ility function ality manage ability maintain ability instant response adaptability 1 availability .308 1 flexibility .827** .240 1 functionality .280 -.117 .171 1 manageability .156 .120 .115 .441* 1 maintainability .211 -.080 .405* .372 .546** 1 instant response .011 .049 -.249 .199 -.171 -.449* 1 integrity .031 .291 .038 -.041 .012 -.073 -.075 portability .263 -.039 .110 .275 .119 -.013 -.028 reliability -.004 .388 -.005 .141 -.055 .171 -.296 scalability .444* .108 .383 .423* .115 .106 .276 security -.288 -.169 -.348 .133 -.354 -.273 .328 user-friendly -.023 -.089 -.103 .031 -.372 -.091 .056 num it emp -.139 .216 -.166 -.224 -.271 -.249 .272 gross rev $m -.056 .265 -.023 -.129 .095 -.041 .224 ** correlation is significant at the 0.01 level (2-tailed). * correlation is significant at the 0.05 level (2-tailed). table eight – correlation matrix for business objectives for ict infrastructure australasian journal of information systems volume 15 number 2, 2009 120 business objectives integrity portability reliability scalability security user friendly num it emp gross rev $m adaptability availability flexibility functionality manageability maintainability instant response integrity 1 portability -.482* 1 reliability .549** -.241 1 scalability -.163 .214 -.005 1 security .159 -.260 .046 -.273 1 user-friendly -.061 -.297 .319 -.103 .541** 1 num it emp .082 -.218 .171 .364 .031 -.061 1 gross rev $m .002 -.119 -.097 .058 -.347 -.159 .032 1 ** correlation is significant at the 0.01 level (2-tailed). * correlation is significant at the 0.05 level (2-tailed). table eight – correlation matrix for business objectives for ict infrastructure cont. regression analyses were conducted on these data in an attempt to determine causal relationships. a total of six statistically significant regression equations (p<0.05) were obtained from the dependent variables adaptability, flexibility, functionality, manageability, maintainability, integrity and scalability (table nine). these regression equations indicated the following aspects. regression equation one indicates that around 70 percent of the variation associated with adaptability within ict infrastructure can be explained by flexibility and scalability. regression model number two in the same table has the highest coefficient of determination (r2) and suggests that nearly 74 percent of the variation associated with flexibility can be explained by adaptability and maintainability. regression equation three indicates that around 33 percent of the variation associated with the ict infrastructure business objective functionality is explained by manageability and scalability. similarly with regression model four in table nine, functionality and maintainability can explain around 36 percent of the variation associated with being able to manage the ict infrastructure (manageability). maintainability was strongly correlated with three variables, namely flexibility, manageability and instant response. however, flexibility was dropped from the regression model as it did not show a significant relationship at the 95% level. in regression model five, around 43 percent of the variability associated with maintainability can be explained by manageability and instant response. the sixth regression equation in table nine indicates that just over 43 percent of the variation associated with integrity can be explained by portability and reliability. finally, regression model seven indicates that about 29 percent of the variability associated with scalability can be explained by adaptability and functionality. australasian journal of information systems volume 15 number 2, 2009 121 regression std coefficients sig. r-square .f sig. model for dependent variable: adaptability (regression model one) (constant) ..532 .703 25.805 .000 flexibility .770 .000 scalability .149 .259 model for dependent variable: flexibility (regression model two) (constant) .416 .739 29.722 .000 adaptability .776 .000 maintainability .241 .047 model for dependent variable: functionality (regression model three) (constant) .054 .335 5.298 .014 manageability .398 .037 scalability .377 .047 model for dependent variable: manageability (regression model four) (constant) .459 .364 6.001 .009 functionality .277 .155 maintainability .443 .028 model for dependent variable: maintainability (regression model five) (constant) .002 .428 7.870 .003 manageability .483 .009 instant response -.367 .040 model for dependent variable: integrity (regression model six) (constant) .032 .431 7.960 .003 portability -.371 .040 reliability .459 .113 model for dependent variable: scalability (regression model seven) (constant) .244 .294 4.370 .026 adaptability .353 .079 functionality .324 .105 table nine– regressions on business objectives for ict infrastructure these results suggest that the most important aspects associated with the business objectives for ict infrastructure are considered to be adaptability, flexibility, functionality, maintainability, manageability, integrity and scalability. it is interesting to note that security is not considered a major concern with respect to any of the profit or productivity factors but was significantly correlated with user friendliness. this indicates (at least with this sample) that secure systems have to be user friendly and that a lack in this area should be of concern to management. further analysis – benefits of the organisation’s extranet, intranet and internet initiatives correlations and regression analysis were undertaken with respect to benefits of the organisation’s extranet, intranet and internet initiatives. the correlation matrix is shown in table ten. significant correlations were again strongly positive with increased productivity being associated with efficiency and increased sales and marketing efforts. increased profitability was strongly australasian journal of information systems volume 15 number 2, 2009 122 correlated with competitive advantage, industry leadership, supply chain coordination, customer retention and loyalty and increased sales and marketing efforts (table ten). benefits increased productivity increased profitability efficiency competitive advantage industry leader supply chain coordination cust. satisfaction increased productivity 1 increased profitability .136 1 efficiency .465* -.068 1 competitive advantage .200 .869** .182 1 industry leader .085 .434* .383 .644** 1 supply chain coordination .169 .695** .219 .688** .596** 1 customer satisfaction .279 .122 .656** .224 .311 .111 1 customer retention/loyalty .110 .538** .473* .646** .468* .561** .423* cost reduction -.062 .404 .329 .416* .504* .467* .135 enhanced communication .294 -.082 .627** -.110 .039 .117 .470* better decision making .305 .127 .547** .424* .425* .193 .545** increased sales & mkt efforts .489* .731** .224 .754** .302 .591** .300 num it emp -.068 -.280 .289 -.177 .057 -.272 .179 gross rev $m -.225 .017 .133 .109 .087 .160 -.112 benefits customer retention /loyalty cost reduction enhanced communication better decision making increased sales & mkt efforts num it emp gross rev $m increased productivity increased profitability efficiency competitive advantage industry leader supply chain coordination customer satisfaction customer retention/loyalty 1 cost reduction .403 1 enhanced communication .170 .187 1 better decision making .427* .063 .214 1 increased sales & mkt efforts .697** .086 .065 .394 1 num it emp -.016 .147 .217 .170 -.354 1 gross rev $m .365 .050 -.070 .032 .075 .032 1 ** correlation is significant at the 0.01 level (2-tailed). * correlation is significant at the 0.05 level (2-tailed). table ten – correlation matrix for benefits realised from extranet, intranet or internet initiatives cont. australasian journal of information systems volume 15 number 2, 2009 123 regression analyses were conducted on these data in the same way as in the previous section. while there were several variables correlated with the dependent variables, many did not have statistically significant partial regression coefficients (p<0.05) and as such were dropped from the regression model. a total of ten statistically significant regression equations (p<0.05) were obtained. these were from the dependent variables of increased productivity; increased profitability; efficiency; competitive advantage, industry leader, supply chain coordination, customer retention / loyalty; and increased sales and marketing efforts. while a number of variables were significantly correlated with each of the dependent variables, only those producing a significant relationship in the regression model are shown in table eleven. the regression equations indicated the following aspects. regression equation one indicates that around 37 percent of the variation associated with productivity can be explained by efficiency and increased sales and marketing efforts. regression equation two indicates that around 82 percent of the variation associated with increased profitability is explained by competitive advantage, industry leadership and supply chain coordination. increased profitability is also highly correlated with increased sales and marketing efforts. with regression model three in table eleven around 60 percent of the variability associated with efficiency can be explained by customer loyalty, enhanced communications and increased productivity. independent of this regression model, efficiency is also highly correlated with customer satisfaction. the dependent variable competitive advantage has two distinct regression equations associated with it. the first regression equation, model four a in table eleven, shows the highest coefficient of determination and indicates that 89 percent of the variability associated with competitive advantage can be explained by increased profitability, industry leadership and better decision making. in the second equation (model four b), just over 57 percent of the variability associated with competitive advantage is explained by supply chain coordination and customer retention and loyalty. regression model number five has a coefficient of determination of 56 percent indicating that 56 percent of the variability associated with industry leadership is associated with increased profitability, competitive advantage and cost reductions. the sixth regression equation shows that 59 percent of the variation in supply chain coordination is explained by increased profitability and industry leader. supply chain coordination also has a significant relationship with competitive advantage distinct from the regression model. customer retention/loyalty has two separate regression equations. the first (model seven a), shows that over 59 percent of its variation is explained by efficiency and increased sales and marketing efforts. the second equation (model seven b), indicates that nearly 50 percent of its variability is explained by competitive advantage and customer satisfaction. finally, regression model six eight indicates that about 80 percent of the variability associated with increased sales and marketing efforts can be explained by increased productivity, increased profitability and customer retention/loyalty. in addition to this regression model, increased sales and marketing efforts is also significantly correlated with competitive advantage. these results suggest that the most important aspects associated with the benefits associated with the organisation’s use of extranets, intranets and the internet are considered to be increased productivity; increased profitability; efficiency; competitive advantage; industry leader; supply chain coordination; customer retention and loyalty; and increased sales and marketing efforts. no significant regression equations were found for customer satisfaction, cost reduction, enhanced communications, and better decision making. australasian journal of information systems volume 15 number 2, 2009 124 regression std coefficients sig. r-square .f sig. model for dependent variable: increased productivity (regression model one) (constant) .024 .373 6.244 .007 efficiency .375 .047 increased sales & marketing efforts .406 .033 model for dependent variable: increased profitability (regression model two) (constant) .002 .818 29.874 .000 competitive advantage .870 .000 industry leader -.285 .040 supply chain coordination .266 .066 model for dependent variable: efficiency (regression model three) (constant) .271 .605 10.199 .000 increased productivity .284 .068 customer retention / loyalty .360 .020 enhanced communication .482 .004 model for dependent variable: competitive advantage (regression model four a) (constant) .001 .889 53.283 .000 increased profitability .743 .000 industry leader .222 .024 better decision making .235 .010 model for dependent variable: competitive advantage (regression model four b) (constant) .919 .571 13.994 .000 supply chain coordination .475 .012 customer retention / loyalty .379 .040 model for dependent variable: industry leader (regression model five) (constant) .004 .565 8.646 .001 increased profitability -.681 .040 competitive advantage .950 .006 cost reduction .415 .063 model for dependent variable: supply chain coordination (regression model six) (constant) .842 .590 15.082 .001 increased profitability .537 .002 industry leader .363 .029 model for dependent variable: customer retention / loyalty (regression model seven a) (constant) .280 .591 15.200 .000 efficiency .334 .030 increased sales & marketing efforts .622 .000 model for dependent variable: customer retention / loyalty (regression model seven b) (constant) .803 .498 10.434 .001 competitive advantage .580 .001 customer satisfaction .293 .078 model for dependent variable: increased sales and marketing efforts (regression model eight) (constant) .002 .807 27.963 .000 increased productivity .382 .001 increased profitability .460 .001 customer retention / loyalty .407 .002 table eleven – regressions on benefits realised from extranet, intranet or internet initiatives australasian journal of information systems volume 15 number 2, 2009 125 discussion this section provides a discussion of the results presented in the previous section. of the responding organisations, 15 were classified as large and 9 as medium. all of the large-sized organisations had some dedicated ict staff and all but one had staff working on their e-business development. eight of the medium-sized organisations had ict staff and four of these had ebusiness development staff. having written three-year strategic technology plans were reported for 66.7 percent (n=10) of the large-sized organisations and 22.2 percent (n=2) of the medium-sized organisations. it is surprising that while all of the large-sized organisations employ ict staff and have staff working on their e-business development (with one exception) that they do not all have strategic technology plans. further, while medium-sized organisations all but one have dedicated ict staff, only two have a written strategic technology plan. in addition, when the dataset is divided into organisations with less than 300 employees, only two out of 13 have a written strategic ict plan. this appears inconsistent with organisations with over 300 employees that typically had written ict plans. there was one exception here; the organisation with the highest number of employees did not report having a written strategic ict plan. as indicated by mckay and marshall (2004), a strategic plan should be an ongoing activity that is essential for it/business alignment. the importance of strategic ict plans needs to be stressed to australian companies and it appears from this research that many medium to large organisations need to think about planning the implementation of their ict resources at a strategic level. the literature is specific in the recommendation that strategic plans are essential for effective management of ict resources and it appears that further education and research is needed to encourage australian businesses to seriously consider how their ict resources should be allocated. the research appears to confirm carr’s contention that ict is treated as an infrastructural technology (at least by many australian companies) and that in many cases business is guilty of assuming that the opportunities for gaining competitive advantage will be available indefinitely and without the necessary planning. the fundamental concern with carr’s thesis is that if ict is treated as an infrastructural technology, the promises of competitive advantage will not flow through; in other words, just because a company has the technology, it doesn’t mean that it will benefit from it unless it has a well thought out plan that will ensure ict is used to its maximum competitive advantage and not just as a commodity. it is our contention that a strategic ict plan should be completed if for no other reason than to ensure that ict is not treated as an infrastructural technology that provides nothing more than a communication technology or service with no real scope for developing competitive advantage. the authors believe that ict is more than simply an infrastructural technology, unlike the railway and electricity examples outlined by carr (2003); ict doesn’t just work by plugging an appliance in or running a locomotive along a railway line. other important human factors need to be considered and while these same factors were of concern with the introduction of railways and electricity, the technology was far less complex. for example, the improvements in decision making that accrue from the use of decision support systems are far more abstract and intangible than the use of an electrical appliance, facilitated through plugging into an electrical circuit. the development of an ict strategic plan is an essential aspect in ensuring that the technology can deliver the promised strategic and competitive advantage that has been promised by much of the information systems literature. if australian businesses wish to remain competitive in world markets, they need to adopt world’s best practice with respect to e-business strategies. this research has highlighted various australasian journal of information systems volume 15 number 2, 2009 126 gaps with some companies not undertaking very basic planning activities such as developing a strategic ict plan. in the area of application portfolios, the majority of large organisations were classed as strategic, inferring that the applications were critical to achieving future business success. the mix of application portfolios can allow some categorisation of business approaches with this research indicating that the majority of large companies have a prospector stance (hirschheim and sabberwal 2001) to their business dealings (n=7). with only a few (n=3) having a conservative, defender stance as indicated by their categorisation of their portfolio as support. the remainder (n=5) appear to have the analyser stance to their business dealings. smaller companies appear to be more conservative with the majority appearing to sway towards the defender stance with the majority (n=5) categorising their application portfolios as supporting (valuable but not critical to business success). these categories of business stances are interesting in that it appears that larger firms are prepared to take greater risks than smaller companies. this stands to reason as larger firms could have deeper pockets thereby being able to take more risks. decision making tended to be undertaken by the chief executive officer, the chief financial officer and the chief information officer. there was no evident pattern across the two categories of organisations and this appears to be consistent with the cavalier approach to ict resources outlined above; that is, a lack of a strategic ict plan and a prospector stance to business dealings. the respondent companies appeared to be more concerned with the technical aspects of implementation with low ratings given to management decisions with respect to areas impacted by implementing ebusiness within the organisation. this technical slant is consistent with the authors’ experience with ict departments in organisations; that is, ict departments in many companies have a great deal of say in the way e-business initiatives are handled, while management has less of a say, mainly because they are not fully aware of the capabilities of the technology and do not have a clear understanding of what is possible. nonetheless, there is recognition that there is a very strong relationship with competitive advantage being associated with increased profitability, being an industry leader and better decision making as described by gendron (2006). the results shown in this study suggest that in relation to porter five forces model, while australian businesses do relate to industry competition and rivalry among existing companies (industry leadership) and the threat of new entrants, suppliers and buyers and bargaining power (increased profitability), they also consider better decision making as an important aspect. perhaps better decision making relates to all five strategies, in any case the relationship between these factors is quite strong and is indicative of genuine concern about competitive advantage and profitability, leadership and better decision making in australia. the strong relationships between adaptability and flexibility, scalability and maintainability may seem somewhat obvious; however, it is good that they have been confirmed through this study. the other strong relationship relating to management is increased profitability being explained by competitive advantage, industry leadership and supply chain coordination. competitive advantage and industry leadership are obvious, but supply chain coordination is noteworthy in that it suggests these organisations place a great deal of importance on effective supply chain management. the connection between competitive advantage and effective supply chain management is well established (mason-jones and towill 1997) and it appears that managers in these australian organisations are fully aware of the implications and value of supply chain coordination and competitive advantage. australasian journal of information systems volume 15 number 2, 2009 127 the singapore study (lin and lia 2008) had a similar result to this study with respect to firm size with larger firms significantly associated with a greater uptake of ecommerce activities; however no mention was made of the need for a strategic technology plan. it was mentioned that top management support was a vital aspect of ecommerce adoption and it may be implicit that a strategic plan is considered part of the process of gaining top management support. another interesting aspect of the factors influencing the decision to adopt ecommerce technology was the influence of business partners. this indicates a very reactive approach to adoption and is indicative of a defender stance with respect to application portfolios. this is similar to the 5 smaller companies in our study. the new zealand research (al-qirim 2007) used a regression approach and this is similar to the approach used in this study. the main finding of the new zealand study was the strong role that the ceo has in the adoption of ecommerce technologies. however a significant factor in the adoption of the technology was again the influence of suppliers (business partners). it should be noted that the new zealand study only looked at small to medium enterprises. conclusion it is hoped that this research can provide a benchmark for future investigations into e-business in australia; however an obvious limitation of this research is the small response rate and subsequent small sample size. future studies will need to obtain a significantly larger portion of australian businesses by being much better resourced and have strategies that allow for follow up letters to be sent to respondents. the apparent cavalier approach to ict planning in the organisations that did respond to the survey is of concern to australia’s future with respect to an effective and world class e-business environment. this research suggests that australian companies need to understand the need and value in developing strategic ict plans, especially in medium-sized firms. the ict departments still appear to have a great deal of say in e-business implementations and there still appears to be a lack of it/business alignment in this cross-section of australian organisations. despite this, it appears that managers are aware of the importance of competitive advantage and can link it with the profitability of the business, industry leadership and supply chain coordination. this indicates a degree of awareness of the requirement to be competitive in a global marketplace. it appears that more research should be done to help australian businesses link these expressed needs with the requirements for a strategic plan for ict infrastructure development. the implications of this study on university curricula are manifold with the need to provide students with skills to enable them to implement relevant business principles when analysing ecommerce requirements. the fact that very few companies had a business and technology plan and that many seemed to think that the benefits will come if they only implement the technology indicates a very naïve approach to the use of ecommerce and the strategic benefits it may bring. a stronger focus on the business benefits of a strategic analysis of ecommerce and the teaching of the tools needed for this endeavour will result in students that are highly sought after by employers. australasian journal of information systems volume 15 number 2, 2009 128 references al-qirim n. (2007) the adoption of ecommerce communications and applications technologies in small businesses in new zealand electronic commerce research and applications 6 pp. 462–473. australian bureau of statistics, (2005), information paper: a statistical view of counts of businesses in australia. australian bureau of statistics, commonwealth of australia. dcita, (2005) “current state of play november, 2005”, department of communications, information technology and the arts, commonwealth of australia. date accessed: 13/05/2007. url: http://www.dcita.gov.au/__data/assets/pdf_file/33120/current_state_of_ play_-_november_2005.pdf. carr, n., (2003) it doesn’t matter, harvard business review 81:2; 41-49. eiu, (2007) white paper, “the 2007 e-readiness rankings: raising the bar”, economist intelligence unit, london. date accessed: 23/05/2007. url: http://a330.g.akamai.net/7/330/25828/20070420205432/graphics.eiu.com/files/ad_pdfs/2007 ereadiness_ranking_wp.pdf. gendron (2006) creating the new e-business company: innovative strategies for real-world applications thomson higher education, ohio usa. hirschheim, r., and sabberwal, r. (2001) “detours in the path toward strategic information systems alignment”, california management review, 44:1; 87-108. markham, j.e. (1998) the future of shopping: traditional patterns and net effects. london, england: macmillian publishing. mason-jones, r. and towill, d.r. information enrichment: designing the supply chain for competitive advantage supply chain management: an international journal volume 2 number 4 pp. 137-148 mckay, j., and marshall, p. (2004) strategic management of e-business. milton, qld: john wiley & sons australia, ltd. porter, m.e. (1985) "competitive advantage", the free press, new york, 1985. roepke r., agarwal, r. and ferratt t.w. (2000) aligning the it human resource with business vision: the leadership initiative at 3m, mis quarterly, vol. 24, no. 2. pp. 327-353 teoa tsh, lina s and laib k (2009) adopters and non-adopters of e-procurement in singapore: an empirical study omega 37 pp.972 987 tsai, n. and r. ching (2003) “current information technology infrastructure for e-business,” proceeding of 2003 international conference on electronic business, december, 2003. ward j and peppard j. (2002) strategic planning for information systems. 3rd edition. chichester, uk: john wiley & sons. australasian journal of information systems volume 15 number 2, 2009 129 appendix – the survey instrument a survey of current e-business (e-government) development practices description. the objective of this survey is to gain a better understanding of current e-business (or egovernment) practices. this survey will allow us to see what changes have occurred in this information age. the results may have a major impact on academic curricula throughout the country. confidentiality. your responses will be kept confidential, and your identity recorded anonymously. i. organisation what is the approximate annual gross revenue of your organisation? _________________ [1-1] approximately how many employees work in your organisation? _________________ [1-2] of those employees, approximately what percentage work in information technology (it)? ______% [1-3] of those in it, approximately what percentage work on e-business (or e-government) development? ______% 1-4] indicate the primary business activity of your company. ____________________________________[1-5] does your company have a written three-year strategic technology plan? yes [1-6-1] no [1-6-2] from these categories, how would you classify your e-business (or e-government) goals? high potential system [1-7-1] strategic system [1-7-2] key operational system [1-7-3] support system [1-7-4] on a scale of 1 to 5, with 5 being the most important and 1 being the least important, please rate the following e-business (or e-government) objectives for your it infrastructure. objectives not important very important adaptability [1-8-1] 1 2 3 4 5 availability [1-8-2] 1 2 3 4 5 flexibility [1-8-3] 1 2 3 4 5 functionality [1-8-4] 1 2 3 4 5 manageability [1-8-5] 1 2 3 4 5 maintainability [1-8-6] 1 2 3 4 5 instantaneous response [1-8-7] 1 2 3 4 5 integrity [1-8-8] 1 2 3 4 5 portability [1-8-9] 1 2 3 4 5 reliability [1-8-10] 1 2 3 4 5 scalability [1-8-11] 1 2 3 4 5 security [1-8-12] 1 2 3 4 5 user-friendly [1-8-13] 1 2 3 4 5 ii. personnel for the strategic planning of your organisation’s e-business (or e-government) processes, who are the key management decision makers? (please check all that apply.) chief executive officer [2-1-1] chief financial officer [2-1-2] chief security officer [2-1-3] information security officer [2-1-4] privacy officer [2-1-5] other (please specify) [2-1-6] ____________________________________________ australasian journal of information systems volume 15 number 2, 2009 130 for the design and implementation of your organisation’s e-business (or e-government) processes, other than traditional system development personnel, who are the key participants? (please check all that apply.) legal consultant [2-2-1] information security officer [2-2-2] sociologist [2-2-3] psychologist [2-2-4] external it specialist [2-2-5] external business consultants [2-2-6] iii. it infrastructure for your e-business (or e-government) initiatives, what are the physical components that make up your it infrastructure? (please check all that apply.) hardware application server(s) [3-1-1] database server(s) [3-1-2] e-mail server(s) [3-1-3] backup server(s) [3-1-4] printer server(s) [3-1-5] storage server(s) [3-1-6] firewall(s) [3-1-7] routers/switches [3-1-8] web server(s) [3-1-9] software server support applications [3-2-1] middleware [3-2-2] web server support applications [3-2-3] database structure relational [3-3-1] network [3-3-2] hierarchical [3-3-3] object-oriented [3-3-4] other [3-3-5] what is the purpose of utilizing data mining tools for e-business (or e-government)? (please check all that apply.) customer focus [3-4-1] product focus [3-4-2] service oriented [3-4-3] customer/product/service [3-4-4] none [3-4-5] what is the main form of communication with your online customers? call centre [3-5-1] e-mail [3-5-2] telephone [3-5-3] web page [3-5-4] none [3-5-5] does your organisation perform the following information management functions? (please check all that apply.) content management [3-6-1] contingency management [3-6-2] customer relations [3-6-3] knowledge management [3-6-4] development management [3-6-5] documentation management [3-6-6] emerging technology management [3-6-7] australasian journal of information systems volume 15 number 2, 2009 131 inventory management [3-6-8] integration management [3-6-9] performance management [3-6-10] security management [3-6-11] vendor management [3-6-12] does your organisation outsource the following functions? (please check all that apply.) traditional systems development traditional systems maintenance [3-7-1] off-the-shelf program evaluation [3-7-2] systems evaluation [3-7-3] system integration [3-7-4] contractor solicitations [3-7-5] contractor management [3-7-6] call centre [3-7-7] customer relationship management [3-7-8] e-business development [3-7-9] e-business maintenance [3-7-10] e-mail system management [3-7-11] database management [3-7-12] data warehouse/data mining [3-7-13] network management [3-7-14] internal end user support [3-7-15] of the following areas, which ones are impacted by implementing e-business (or e-government) on your organisation? (please check all that apply.) strategic planning [3-8-1] management decisions [3-8-2] it infrastructure [3-8-3] operation processes [3-8-4] business processes [3-8-5] resources [3-8-6] iv. network usage please indicate the reason(s) for internet usage within your organisation. (please check all that apply.) web page presence [4-1-1] marketing and advertising [4-1-2] business-to-business [4-1-3] business-to-consumer [4-1-4] customer-to-customer [4-1-5] e-education [4-1-6] e-government [4-1-7] e-mail [4-1-8] management driven [4-1-9] please indicate the goal(s) of the organisation’s utilization of an extranet. (please check all that apply.) customer service and support [4-2-1] products/services catalog information [4-2-2] publish information among business partners [4-2-3] post electronic forms [4-2-4] collaboration between business partners [4-2-5] production and inventory control between business partners [4-2-6] electronic data interchange [4-2-7] electronic funds transfer [4-2-8] portal [4-2-9] sales [4-2-10] australasian journal of information systems volume 15 number 2, 2009 132 marketing and advertising [4-2-11] please indicate the objectives(s) of the organisation’s utilization of an intranet. (please check all that apply.) inbound logistic system [4-3-1] education and training [4-3-2] broadcast information [4-3-3] post electronic forms [4-3-4] team collaboration [4-3-5] software distribution and management [4-3-6] document management [4-3-7] on a scale of 1 to 5, with 5 being the most important and 1 being of least importance, please rate the following benefits your organisation realizes from its extranet, intranet, or internet initiatives. objectives not important very important increased productivity [4-4-1] 1 2 3 4 5 increased profitability [4-4-2] 1 2 3 4 5 efficiency [4-4-3] 1 2 3 4 5 competitive advantage [4-4-4] 1 2 3 4 5 industry leader [4-4-5] 1 2 3 4 5 supply chain coordination [4-4-6] 1 2 3 4 5 customer satisfaction [4-4-7] 1 2 3 4 5 customer retention/loyalty [4-4-8] 1 2 3 4 5 cost reduction [4-4-9] 1 2 3 4 5 enhanced communication [4-4-10] 1 2 3 4 5 better decision making abilities [4-4-11] 1 2 3 4 5 increased sales and marketing efforts [4-4-12] 1 2 3 4 5 how does your organisation measure the performance or outcome of its e-business (or e-government) initiatives? profit [4-5-1] cost [4-5-2] productivity [4-5-3] v. participant name of organisation: __________________________________________________________________ [5-1] name of department: __________________________________________________________________ [5-2] address: __________________________ ____________________________________________ [5-3] name of person completing the survey: _______________________________________________________ [5-4] title of person completing the survey:_____________________________________________________ [5-5] check here if you would like a copy of the preliminary results. thank you for your participation! your comments are greatly appreciated. [5-6] ajis vol. 4 no. 2 may 1997 evolution of oo methods: the unified case matti rossi university of jyvsskyla department of computer science and information systems p.o. box 35 sf-40351 jyvaskyla finland internet: mor@jyu.fi fax: 358-41-603011 abstract this paper takes an evaluative look into oo methods and especially the evolution of the new unified method from its ancestors, omt and ooda. the paper tries to classify the components of the earlier methods and identify the parts that have been taken into the unified method. the research applies the method metrics approach. for the sake of compactness we limit ourselves to the class diagram technique of all methods. we make observations about the number of concepts in each variation and show how the metrics can be used to analyse the changes in the techniques. introduction recent years have witnessed the appearance of new systems development paradigms and methods. examples of these are object-oriented analysis and design methods (eg see booch (1991), rumbaugh, blaha, premerlani, eddy and lorensen (1991), booch and rumbaugh (1995) etc.). however, we feel that there is need for improvement in the analysis of these methods and in understanding their use and functionality. although some attempts have been made to compare existing methods champeaux (1991), livari (1994), olle, sol and verrijn-stuart (1982) and numerous others, the studies lack rigour and a sound conceptual foundation, and are mostly based on ad hoc feature analysis techniques. some recent attempts have proposed more systematic approaches, based on a common formal metamodelling language to describe methods (e.g. see hong, goor and brinkkemper (1992), song and osterweil (1992). in this paper we investigate a few "generations" of oo methods and identify their differences and similarities. the aim of the study is to highlight the changes in the methods and discuss the features of the evolved methods. especially we investigate the changes in the definitions of the core method concepts by analysing the metamodels of the methods. we are interested in the properties of the methods and the representation of different methods within a common formalism. through the modelling of methods we hope to gain more understanding about their nature. in general several metamodel based comparison strategies can be used. here we distinguish three ways, namely comparing, combining and statistical measuring. first one, the comparison strategy, is most obvious way to utilise metamodel in which we use two or several metamodels and compare them directly to each other (i.e. two by two or all together). this allows us to find commonalties and obscurities in methods concept structures and seek for different connectivity approaches between modelling languages. second, the combination approach applies several metamodels in order to build a method that include some (best) parts from each method. combination can be done either by building up a super method (collecting all of aspects or elements defined) as in hong, goor and brinkkemper (1992), selecting a minimum set of common aspects, or through a hybrid approach using both of the previous ways. third, the measurement approach in rossi and brinkkemper (1995) collects knowledge from all the metamodels and tries to find for example regular or divergent structures. in this paper we apply the last approach by making some observations with metrics on metamodels. we try to measure the complexity of learning and applying the methods. we restrict the complexity of techniques and methods to two aspects. on the one hand we try to measure the complexity of learning and understanding the technique, which is related to the number of different concepts and constructs (say, object types and relationship types) used in the technique. on the other hand it is desirable to get insight in the complexity of the internal structure of the models resulting from applying the technique. this complexity is dependent on the number of describing properties of the technique's objects and relationships. this paper is organised as follows. in the next section we present the metamodeling method used to describe the knowledge of methods. in section 3 we present the resulting metamodels of the three selected methods. these metamodels capture both the static concept structure part of the method and the dynamic part of the method use process. in section 4 we compare these methods by obtained results of metrics. finally, section 5 summarizes research outcomes and presents future directions for this kind of evaluation. 55 ajis vol. 4 no. 2 may 1997 the method description language techniques of interest here consist of traditional graphical formalisms. these techniques describe the object systems by objects and their relationships. in many cases the relationships and objects can have attributes or properties. the techniques usually describe only one aspect of an object system (such as data flows or state changes etc.). to be able to compare and analyse techniques, we describe their structure using one common language to define the metamodels of the methods. we use here the oprr (object, property, relationship, role) methodmodelling language, as presented by smolander (1990) to model the techniques and methods. the use of one methodmodelling language gives us a neutral basis to compare the properties of techniques, and it provides a common background for the formulation of metrics. it is important to notice, however, that there are a few factors that can bias the results. first, the precision of the method description is dependent on the quality of the technique's description in the textbook. some authors present detailed formal descriptions of their techniques, and others define their techniques more vaguely. secondly, the experience and personal preferences of the method modeller affect the model. for example, a given concept might be metamodelled either as a property or as a relationship. in this particular case all of the techniques have been modelled by one experienced method engineer, and we can thus expect that they have been modelled with a consistent style. the acronym oprr comes from the words object, property, role, and relationship which are the meta-types in oprr. welke (1988) defines the meta-types in the following way: • object is a "thing" which exists on its own. examples of objects are process, flow, store, source, module, etc. properties are the describing or qualifying characteristics associated with the other meta-types. typical properties include name, description, definition, etc. • relationship is an association between two or more objects. for example, there may be a relationship between a source and a process meaning that the process uses the source. • role is the name given to the link between an object and its connection with a relationship. from the example above, the process would be the user and the source would be the origin of the data. in metaedit's came environment we have extended oprr by defining explicit mappings from these concepts onto their representations. we have used the following notation: an object type is represented by a rectangle, a property type by an ellipse, a role type by a circle and a relationship type by a diamond. the name of each object, property, role or relationship type is written inside its symbol. an example of such a graphical oprr model is in figure 1. techniques compared all the methods use a number of individual techniques. as most of them apply quite similar state diagrams and the main techniques are the class diagrams, that describe the class structure of the system under development, we restrict ourselves to the cds. in this chapter we describe the three techniques used for the comparison. for the sake of compactness we use only the first published version of ooda and omt class diagrams and the unified version 0.9. for each of the techniques we describe its objects and relationships and show its graphical oprr model. the models can be found from rossi and tolvanen (1994), except for unified cd, which was modelled for this project. technique: class diagram this is the (1991) version of quite popular booch method for object-oriented systems development booch (1991). it describes a system by its class structure with special emphasis to groupings of the classes. technique's concepts class, class category, class utility relationships of the technique class category visibility, inherits (compatible type), inherits (new type), instantiates (compatible type), instantiates (new type), metaclass, undefined, uses (for implementation), uses (for interface) 56 ajis vol. 4 no. 2 may 1997 name j] i visibility 1 i persistence glass cardinality (concu descendant (new type) i ^class utility i documentation---—j class from (class category) scendant (corrtpatjtxe type)class category visibility inhwjts (compatible>type) user (loifimplementation) used tor implementation) instantiates (new type) uses for implementation) undefined > instantiates (compatine type) < metaclass figure 1. oprr model of ooda cd technique: omt class diagram the omt class diagram describes the static structure of the objects in a system and their relationships rumbaugh, blaha, premerlani, eddy and lorensen (1991). technique's concepts class, disjoint divider, divider, object, subclass group relationships of the technique aggregation 1 to 1, aggregation 1 to m, association (optional to 1), association (optional to many), association (optional to optional), association 1 to 1, association 1 to many, association many to many, generalisation, generalisationspecialisation, instantiation, qualified association 1 to 1, qualified association 1 to many, qualified association many to many, specialisation 57 ajis vol. 4 no. 2 may 1997 figure 2. oprr model of omt cd technique: unified class diagram the unified method can be seen as a direct descendant of the two above mentioned methods and especially the class diagram technique seems to be an amalgamation of their concepts. the unified modeling language, or uml, is claimed to be a third-generation object-oriented modeling language. at present uml is being pushed to the object management group in the hope that it will become a standard modeling language for oo development. the model here is based on the preliminary 0.8 version of the technique booch and rumbaugh (1995). technique's concepts class, object relationships of the technique aggregation, association, inheritance, instantiation, qualified association remarks about the techniques there are noticeable similarities between the concepts of the techniques. omt and unified distinguish the oo notions of class and instance (object), but ooda uses classes and their categories. ooda uses a different technique for modelling the instances. the relationships in the methods differ noticeably. we can argue, that the ooda relationship types come from the ada programming language, whereas the omt cd relationships come from semantic modelling camp. the unified cd's have mainly adopted, but simplified, the omt relationships. the properties of the concepts are mostly identical. 58 ajis vol. 4 no. 2 may 1997 qualified association from ff qualifier figure 3. oprr model of unified cd analysis of the methods in this chapter we present the analysis of the techniques based on method metrics and concept classification. we then discuss shortly the findings. the metrics used here are described in greater detail in rossi and brinkkemper (1995). we use only a few of the metrics of the suite presented there to show how they can be applied when analysing the evolution of techniques. independent measures 1. n(0t) the count of object types per technique. this metric shows the number of individual object types used to specify object systems. 2. n(rt) the count of relationship types per technique. this is the number of concepts, which are used for describing connections between objects. 3. n(pt> the number of property types per technique. the reader should notice, that this metric measures the total number of property types in the whole technique, whereas the following metrics count properties of individual object types or relationship types. aggregate metrics the independent metrics above described the individual characteristics of techniques. in this section we propose some aggregate metrics, that can be used to measure the overall complexity of the technique. 4. c(mt.o) = j0(mt.0) pr(mt , e) ' 5. c(mt) = the quotient (formula 4) shows the division of work in this technique, i.e. are things described by their internal properties, or by external connections. the quotient will get higher values if there are many 59 ajis vol. 4 no. 2 may 1997 properties and a few relationship types with a few properties. formula 5 gives the average for the whole technique. 6. c'(mt) = a/n(0t)2+n(rt)2-»-n(pt)2 the total conceptual complexity of a technique is the complexity vector in a three;-dimensional coordinate system. the vector can be compared with other techniques. the idea of using the complexity vector is, that one can see the complexity of the technique by looking at how long the vector is and atm the same time one can see the "style" of the technique by looking at in which direction the vector goes. comments on metrics the values obtained from the different class diagrams are shown in table 1. we can shortly notice, that all techniques use a relatively small number of object types, but interestingly the new unified class diagram uses the smallest number of them. the same applies to relationships as well, however it is important to notice, that the huge number of relationships in omt cd comes from the representational variations of the association cardinalities. the number of properties is, interestingly enough, such that the unified falls between its two ancestors. the formula 5 shows, that there is a clear tendency to move from the relationship orientation to more property oriented style in unified. we claim that this tendency comes from the difficulties to apply the relationships in practise. technique ooda class diagram class diagram unified class diagram 1 3,00 5,00 2,00 2 9,00 15,00 5,00 3 10,00 14,00 13,00 5 ,70 ,91 2,65 6 13,78 21,12 14,07 table 1. metric values obtained from the techniques in the figure below we present the boxplots of the first three formulas, which is a five number summary of smallest observed value that isn't outlier, lower quartile, median, higher quartile and largest observed value that isn't outlier. notice that extreme values (values which are more than 1.5 times the difference between the lower and upper quartile outside of the quartiles) are plotted separately. omt cd omt cd o omt cd o o unified cd 2 4 6 8 1 0 1 2 1 4 figure 4. boxplots of formulas 1, 2 and 3 16 60 ajis vol. 4 no. 2 may 1997 it is interesting to see, that omt cd is an outlier in all of the formulas. ooda has values, that are near the maximum line, but not above it. unified cds use a "normal" number of relationships and roles, but a big number of properties. 12 properties 10 relationships figure 5. technique cube the technique cube in figure 5 is used to show the complexity vectors of the techniques. the class diagram technique is the most complex by this measure as it uses properties and relationships extensively, but it contains an average number of objects. unified cd's differ in style from the others, as the have a small number of objects and relationships, but a lot of properties. to summarise our findings, we can say, that it is interesting to notice, that by a metrical analysis the techniques evolution doesn't seem to necessarily lead all the time to more complex techniques. conclusions in this paper we have compared generations of separate techniques (omt and ooda class diagrams) and their merger, unified class diagram. our goal was to show how the method metrics can be used to analyse different techniques. to obtain a thorough understanding, one needs to use these metrics together with other comparison aids such as livari's classification hierarchies livari and kerola (1983) and tabular feature comparisons hong, goor and brinkkemper (1992). the proposed metrics are relatively simple to understand and easily implemented in a tool. the comparison of the techniques showed, that the authors have merged their previous techniques in such a way, that the total complexity has actually gone down. in the future we shall expand this research to contain all techniques of the methods and we shall also use qualitative analysis of the individual components to show how their definitions have evolved. references booch, g. (1991), object-oriented design with applications, benjamincummings, redwood city, california. booch, g., j. rumbaugh (1995), unified method for object-oriented develoment, rational software corporation, santa clara, us. champeaux, d. de (1991), a comparative study of object oriented analysis method, research report, hp laboratories. hong, shuguang, geert van den goor and sjaak brinkkemper (1992), a formal approach to the comparison of object-oriented analysis and design methodologies, pp. 689-698 in 61 ajis vol. 4 no. 2 may 1997 proceedings of the 26th hawaii international conference on systems science, j. nunamaker, r. sprague and s. hong (ed.), ieee computer society press, usa. livari, j., p. kerola (1983), a sociocybernetic framework for the feature analysis of information systems development methodologies, pp. 87--139 in information systems methodologies: a feature analysis, t. w. olle, h. g. sol and c. j. tully (ed.), north-hollland, amsterdam, livari, juhani (1994), object-oriented information systems analysis: a comparison of six objectoriented analysis methods, pp. 85-110 in methods and associated tools for the information systems life cycle (a-55), a. a. verrijn-stuart and t. w. olle (ed.), elsevier science b.v. (north-holland). olle, t. w., h. g. sol and a. a. verrijn-stuart (1982), proceedings of the ifip wg 8.1 working conference on comparative review of information systems design methodologies, north-holland, amsterdam. rossi, m., j.-p. tolvanen (1994), metamodeling approach to method comparison: a survey of a set of isd methods, working paper, university of jy vaskyla, jy vaskyla. rossi, m., s. brinkkemper (1995), metrics in method engineering, pp. 200-216 in advanced information systems engineering, proceedings of the 7th international conference caise'95, j. livari, k. lyytinen and m. rossi (ed.) no. 932, springer-verlag, berlin. rumbaugh, j., m. blaha, w. premerlani, f. eddy and w. lorensen (1991), object-oriented modeling and design, prentice-hall, englewood cliffs, nj, usa. smolander, k. (1990), oprr • a model for modeling systems development methods, proceedings of the second workshop on the next generation of case tools, university of jy vaskyla. song, x., l. osterweil (1992), towards objective and systematic comparisons of software design methodologies, ieee software 18(5) pp.43-53. welke, r. j. (1988), metabase a platform for the next generation of meta systems products, proceedings of case studies conference, meta systems, ann arbor, michigan. 62 ajis vol. 7, no 2 may 2000 testing hammer and stanton's reengineering-success diagnostic felicity murphy peter seddon sandy staples* department of information systems the university of melbourne melbourne, australia e-mail: fmumh\@studentiiiail.dis.unimelb.edu.au t>.seddon@dis.unimelb.edu.au *queens school of business queens university kingston, ontario, canada e-mail: sstaples @ business, queensii. ca abstract business process reengineering (bpr) is no longer referenced in academic literature with the regularity that it was in the heady days of the mid-nineties but it is still in use in australian organisations. we are also changing processes radically as new software is implemented or new electronic markets are embraced and the lessons of reengineering can be applied to these process changes. this study makes three main contributions by: (1) investigating the organisational context prior to the embarkation on projects of major change in australia, (2) testing the validity of a diagnostic proposed by hammer and stanton (1995) to be used when assessing the ability of organisations to succeed at reengineering, and (3) examining the impact of the organisational environment prior to change on the outcome of reengineering projects. keywords: business process reengineering (dd0402) introduction business process reengineering (bpr) has been an important phenomenon. brancheau, et al. (1996) found that in 1994/5 the second most important issue faced by top managers was "facilitating and managing business process redesign". although reengineering has ceased to be a fashionable topic for research, the fundamental lessons it taught us about embarking on programs of radical change hold true and can provide guidance when undertaking future rounds of major organizational reconstruction. the idea of making major changes to business processes, as advocated by reengineering, has become commonly accepted. in much of the literature on enterprise systems (also known as erp systems) it is often recommended that business processes should be modified to fit the "best-practice" processes embedded in the system (davenport, 1999). organisations are also changing their processes in response to the electronic commerce imperative. james champy (1998) recognised the ongoing need to reengineer processes in response to the increase in "digital commerce" places firms under pressure to "restructure the entire organisation". given the ease with which organisations are now persuaded to change business processes the lessons to be learned from reengineering should not be overlooked. since 1990 when the seminal reengineering work by michael hammer (1990) appeared in the harvard business review there has been much academic research done on the critical success factors for implementing reengineering projects. this work has, however, overlooked the importance of the organisational context prior to embarkation on a reengineering project. it has focussed, instead, on conduct during implementation (e.g., grover et al.,1995, clemens et al., 1995, hall et al., 1993, miles et al., 1995, broadbent & weill 1995, broadbent et al. 1996, butler 1993 & 1997). this suggests that understanding of how the organisational context prior to reengineering (or other major organizational change projects) impacts the outcome of the project is limited. with this in mind we undertook a study in australia that examined reengineering practices and the context that existed at the time of the decision to reengineer but prior to the beginning of the reengineering project. in particular, we wanted to investigate the validity of a diagnostic instrument proposed by hammer and stanton (1995) that purported to help managers decide if their organization was ready to embark on a reengineering project. in chapter 6 of their book, hammer and stanton offer a list of 20 questions, grouped as three factors or themes, and suggest that certain minimum scores should be attained for each factor and the overall score before reengineering is attempted. thus the research question addressed in this paper is as follows: does the diagnostic proposed by hammer and stanton really predict whether organisations are well positioned for reengineering success? 60 ajis vol. 7, no 2 may 2000 business process reengineering was defined at the outset of our study as: a deliberate (planned) change, typically enabled by information technologies in an attempt to redesign a business process to achieve performance breakthroughs in measures such as quality, speed, customer service, and cost. (groveretal., 1995). the remainder of this paper is organised as follows. the next section summarises the literature about successful reengineering projects. the third section describes the methodology used in our study. the fourth section presents the results. in the final section, the results are discussed and implications for practice and research are offered. a review of the reengineering literature in an extensive review of the reengineering literature grover et al. (1995) found that there are six main factors which effect the implementation of projects. these factors are shown in the left-hand column of table 1. coming from the practitioner literature, hammer and stanton (1995) emphasised that management support, planning and scoping of reengineering projects, and change management could begin long before an organisation embarked on implementation. this recognition was embodied in a diagnostic they proposed requiring practitioners to consider three factors before making the decision to start reengineering. these three factors are shown in the right-hand column of table 1. clearly there is some overlap in these factors. in this paper, we focus on the three factors that hammer and stanton (1995) believe are important prior to embarkation on a reengineering project. each of these is now discussed in turn. groveretal. (1995) factors affecting reengineering success hammer and stanton (1995) themes (pages 86-90) change management technological competence process delineation _ project management project planning ^engineering leadership organisational readiness style of implementation management support ^ , table 1: reengineering success factors during the project (grover et al., 1995) and before embarking on the project (hammer and stanton, 1995). reengineering leadership hammer and stanton's (1995) first pre-embarkation factor for reengineering success is reengineering leadership. the first six questions in their self-assessment diagnostic assess aspects of the leader's vision and commitment to the project. elsewhere in the literature, "management support and leadership" in reengineering efforts has also been identified as perhaps the most significant contributing factor to the success of an initiative. keen (1995) suggests that effective managers understand how to lead an organisation through radical changes by combining a tenacious pursuit of the performance objectives with the flair for building organisational consensus at all levels. top management support is necessary to develop faith throughout the organisation in the radical changes proposed and a respect for the project team (hall et al. 1993, hoopes 1995, grover et al., 1995). hall (1993) states strongly that reengineering "will fail without the full commitment of senior executives ... supplying reengineering with all resources required: the best people, investment in it and skills training." hammer in an interview with moad (1993) stated that "reengineering requires two things to succeed, ... one of which is that it requires process orientation. ... that's something that can only be driven by the guy who is above all these functional barons, because none of them wants to give up any autonomy. the other requirement is clout. the person who makes reengineering happen requires a lot of clout because a lot of people need clouting." hammer's comments reinforce his belief that management support is highly influential on the outcome of reengineering initiatives and that a leader can engender support for a project through the sheer force of their passion or belief. organisational readiness hammer and stanton's (1995) second pre-embarkation factor for reengineering success is organizational readiness. nine questions in their self-assessment diagnostic assess aspects of organizational readiness such as the existence in the organisation of an open attitude toward reengineering, the capability to make the changes needed and the resources required to support the process. reengineering entails radical change and as such "a fair amount of anxiety usually surfaces when a reengineering project is announced" (manganelli and raspa, 1995). much of the reengineering literature focuses on the need to manage change and foster a positive attitude toward the change throughout the organisation. one of the methods suggested for mitigating the resistance to change is the communication of the vision behind the impending change. it is suggested that the reasons and expectations for the change should be communicated organisation-wide as a way of motivating people and enticing them to commit themselves to the project (davenport 61 ajis vol. 7, no 2 may 2000 1993, grint et al. 1996, hall et al. 1993). involving the entire organisation in the change process is a good way to overcome lethargy and resistance within the organisation (romney 1996). in the grover et al. (1995) study, change management issues emerged as the set of problems with the most significant negative relationship with project outcome. reengineering is no different from other major projects and as such planning is imperative. grover et al. (1995) focus on the importance of project planning on the outcome of reengineering projects and focus on the reengineering team and the resources and authority that are placed at its disposal. the quality of resources and the amount of authority given to the project team indicate to the rest of the organisation the level of importance and management support the project is receiving. the resources that reengineering teams require include human (i.e., having the most highly skilled and knowledgeable people); financial (e.g., money for new ft or other expenses); training for team members; and, in some cases, a consultant to provide expert advice (hammer & champy 1994, hammer & stanton 1995, keen 1995, miles et al. 1995). style of implementation hammer and stanton's (1995) third pre-embarkation factor for reengineering success is style of implementation. the last five questions in their self-assessment diagnostic assess aspects of style of implementation, such as the need for the reengineering project to be scoped correctly, for project participants to be empowered to make changes and management systems to be put into place before the implementation of reengineering projects. the literature reinforces the need to aim redesign efforts at key processes with hall (1993) noting that often processes for reengineering are "too narrowly defined and therefore have little discernible impact on overall performance". the scope of the process chosen for redesign must be broadly defined in terms of cost improvements or increases in customer value for it to improve performance across the organisation. redesign projects can also be defined too radically with changes being too wide for organisations to cope with (davenport 1993, martinez 1995). process delineation should be undertaken carefully after having assessed the organisations resources and it should involve the identification of performance improvement goals for the redesigned process (demons 1995, miles et al. 1995). grover et al. (1995) argued that while process delineation problems may not be difficult to address they have a significant impact on project outcomes. management systems such as incentive and reward structures, new skills training, and other human resource policies require alteration in response to the organisational restructuring. it is important that these changes are made before the organisation is restructured if management wish to cultivate the required values towards the reengineering project. hammer and stanton (1995) present their book the reengineering revolution as a handbook for bpr implementation. the diagnostic consisting of a twenty-question checklist, along the three dimensions discussed above, is meant to indicate "the minimum numbers we believe an organisation should score before tackling reengineeringthat is, prior to launching the effort" (hammer and stanton, 1995). they argue that high scores on these three categories of questions prior to embarking on a reengineering project "characterise an organisation that is well positioned for successful reengineering" (hammer and stanton, 1995). the results of this review suggest that much of the work toward smoothly implementing reengineering projects could be effected before they are begun. to test the propositions of hammer and stanton (1995) and the effect of the organisational context prior to implementation on eventual outcome of projects, data were gathered using a mail questionnaire. the instrument development, its recipients and distribution are discussed in the next section. methodology a quantitative research design was chosen to examine the importance of the organisational context prior to reengineering on the perceived success of projects. a questionnaire was used in order to facilitate collection of information from a large and geographically disperse sample. the questionnaire was pre-tested on a group of colleagues who understood the hypotheses to be tested and the proposed audience as suggested by dillman (1978). this section describes the measures used, sampling method, and analysis methods employed. measurement hammer and stanton (1995) make it clear that their diagnostic is designed to assess organisational readiness for reengineering, but it is not clear who should complete the diagnostic. so we decided to survey both senior managers and project leaders. senior managers were asked to give opinions based on their experience of reengineering exercises in the organisation as a whole. project leaders were asked to consider a single reengineering project they had worked on. since we wished to compare organizational readiness before the engineering project to success after, we asked respondents who had completed reengineering projects to try to recall the state of organizational readiness before the project commenced. many of the 20 questions from hammer and stanton (1995) contained compound questions covering very different concepts. for example question 1 asks whether: 'the leader of reengineering is a senior executive who is strongly committed to reengineering and who possesses the title and authority necessary to institute fundamental change". for our questionnaire, we split this question in two to become q.i: "the reengineering leader possessed the title and authority necessary to institute fundamental change", and q.31: 'the leader of reengineering was a senior executive who 62 ajis vol. 7, no 2 may 2000 was strongly committed to reengineering" on our questionnaire, (see appendix a). respondents were instructed to "indicate the extent to which the conditions prevailed in the organisation prior to the implementation of reengineering" on a five point likert scale. the likert scale was anchored at 1 by "strongly agree", at 3 by "neither agree nor disagree", and at 5 by "strongly agree". the primary dependent variable adopted for our study was perceived success. perceived success, measured by one fivepoint scale question, is according to delone and mclean (1992) the most widely used measure of mis success. a cost reduction success measure was also used for estimating the success of the individual projects considered by the project leader. sample a nation-wide mail survey of australian organisations was conducted. the questionnaire was distributed to senior managers of the 1000 top australian companies (chosen on the basis of revenue) in the private or public sectors via a mailing list purchased from drake list management services. the questionnaire package contained a cover letter addressed to the senior manager asking him/her to pass the enclosed questionnaire on to a project leader of a recent reengineering project. both the senior manager and the project leader were offered a report of the findings of the study in an attempt to motivate them to respond. to increase the response rate, a reminder letter was sent to senior managers of non-responding organisations about three weeks after mailing the questionnaire package. of the 1000 surveys sent, 196 people indicated that they had not undertaken reengineering (an option we provided in the covering letter) and 155 organisations returned a completed questionnaire. therefore, our response rate was 35.1 percent. we had responses from 88 pairs of respondents from the same organisation, 49 responses from senior managers in an organisation only, and 18 responses from project leaders only. the potential for non-response bias was addressed by profiling both early and late responders and comparing the two groups according to techniques described by fowler (1993). this analysis yielded no significant differences between the groups and supports the position that non-response bias did not appear to be a problem. respondents were from both the public and private sector and came from a mixture of industries. 16.8% responded that they were employed in financial services, 14.6% in manufacturing, 8.7% in government, 7.3% in utilities, 5.1% in mining, 4.4% in it (including professional services) and 4.3% in health. the average annual revenue of the reporting organisations was $ 1.58 million and the average number of people employed was 6568. responding senior managers reported their organisations had initiated an average of 10.4 reengineering projects. cost reduction was the most commonly cited goal with 69 percent of respondents considering this as a very important factor in the decision to reengineer. worker productivity increase was the second most commonly cited trigger (46 percent), customer satisfaction level increase was third (45 percent), fourth was cycle time reductions (36 percent) and fifth was defects reduction (24 percent). perceived success of reengineering projects, from the senior managers' point of view is shown in figure l(a). success rates were surprisingly high, given the frequency with which reengineering project failures have been reported in the literature. responding project leaders had been employed with the current organisation for 10.2 years and reported having an average of 5 years and 4.37 projects worth of experience with reengineering. reengineering training had been received by only 31 percent of the project leaders. the business areas that were the focus of reengineering consisted of a single business division in 29.5 percent of cases, two or more divisions in 33.3 percent of cases, and organisation-wide in 37.2 percent of cases. reengineering teams consisted of an average of 16 members and consultants were used in 63.5 percent of the reported projects. the consultant input into the reengineering exercise was generally regarded as valuable although it did not correlate with the perceived success of projects. perceived success of reengineering projects from the project leaders' point of view is shown in figure l(b). again these projects were remarkably successful. senior manager: perceived success of reengineering projects(n=133) mean: 1.9 s.dev:0.71 number of firms scoring 1 or 9 106 80 70 60 so 40 x 20 10 0 1 j —r-^— . 1 1 . 2 3 4 5 project leader: perceived success of reengineering projects (n=95) mean: 2.00 s.dev: 0.87 number of firms scoring 1 or 2 79 60 50 40 x x 10 0 1 1n i— i 2 3 4 5 figure 1: perceived success of reengineering projects, from (a) senior managers and (b) project leaders. (1 = success, 3=neither a success or failure, 5 = failure. no projects were classified as failures.) for both senior managers and project leaders the questions about reengineering leadership seemed to elicit the most responses in strong agreement to indicate that a strong, positive leadership existed before reengineering was embarked upon. organisational readiness and style of implementation questions were more often answered with neither agreement nor disagreement. 63 ajis vol. 7, no 2 may 2000 results recall that our research question is as follows: does the diagnostic proposed by hammer and stanton really predict whether organisations are well positioned for reengineering success? we answered this question by (a) computing scores as suggested by hammer and stanton and comparing them to perceived success, (b) examining correlations between hammer and stanton's three factors and perceived success, and (c) using linear regression to examine relationships between hammer and stanton's factors and perceived success. we also examined the factor structure of hammer and stanton's instrument to see if the factor structure predicted by hammer and stanton could be confirmed empirically. first, scores for the twenty questions proposed by hammer and stanton (1995) were reconstructed by summing and averaging the answers to the 31 component questions asked in our questionnaire (see appendix a). scores for the first six questions were then summed to calculate a score for reengineering leadership, the next nine questions for the score on organizational readiness, and the last five questions for the score on style of implementation. according to hammer and stanton (1995), theme scores should exceed 24, 28, and 18, respectively, with the total exceeding 75 if the firm is ready to proceed with reengineering. table 2 shows percentages of senior manager and project leader scores for the three "themes" and overall. the last row of table 2 shows the proportion of projects classified as successes. variable reengineering leadership organizational readiness style of implementation overall score perceived success of subsequent reengineering project(s) maximum possible 6*5=30 9*5=45 5*5=25 20*5=100 recommended threshold score before proceeding 24 28 18 75 percentage of senior managers scoring their organization above min. 39.7 75.7 71.3 55.1 106 of 133 cases (80%) percentage of project leaders scoring their organization above min.. 42.2 53.9 48.0 25.5 79 of 95 cases (83%) table 2: percentages of factor and overall scores exceeding for the hammer and stanton (1995: 86-90) thresholds, plus perceived project success using overall scores as the criterion, it would seem that for the projects our respondents classified as reengineering, hammer and stanton have set their threshold scores too high. only 55.1% of senior manager evaluations passed hammer and stanton's threshold score, yet about 80% of projects were later judged to be successes. (alternative explanations are (a) that the projects our respondents described as reeengineering were not really reengineering, i.e., perhaps they involved less radical change, or (b) that despite being insufficiently prepared, the organization took steps later to overcome its problems, and so succeeded.) the corresponding figures for project leaders were 25.5% and 83%, respectively. second, even if hammer and stanton have set their threshold scores too high, it seems likely that higher readiness scores would lead to more successful projects. pearson correlations between the four independent variables (above the solid line in table 2) and perceived success (below the line) are shown in table 3. correlations with perceived success were much lower than correlations between the readiness indicators. for senior managers, correlations between the three readiness factors and eventual project success were significantly correlated (p<0.001). for project managers, the only significant correlation was between organizational readiness and project success (p<0.001). in addition, for the 88 cases where we had responses from senior managers and project managers in the same organization, we ran correlations between their scores for the three theme variables. as shown in the left of figure 2 (the curved arcs), all three pairs of variables were significantly correlated. 64 ajis vol. 7, no 2 may 2000 leadership org. readiness style overall score perceived success cost reduction senior managers leadership 1.000 .665 .829 .901 -.283 p=0.001 org. readiness .665 1.000 .725 .900 -.279 p=0.001 style .829 .725 1.000 .923 -.339 p=0.001 overall score .901 .900 .923 1.000 -.328 p=0.001 project leaders leadership 1.000 .543 .754 .874 -.076 n.s. .030 n.s. org. readiness .543 1.000 .580 .853 -.235 p=0.022 .097 n.s. style .754 .580 1.000 .864 -.109 n.s. -.016 n.s. overall score .874 .853 .864 1.000 -.173 p=0.094 .057 n.s. perceived success see below -.212 p=0.066 all correlations between the four readiness variables significant at the 0.001 level (2-tailed). n.s. means not significant, n.a. means not available. correlations with success are negative because perceived success was scored 1 for greatest success. table 3: correlation between the three themes, overall readiness score, and perceived success third, we used linear regression to see how much variance in the perceived success variable was explained by the three theme measures after controlling for the effect of the other factors. path coefficients (standardized beta) are also shown in figure 2. coefficients of determination (adjusted r-square) for these regressions were poor: 0.097 (n=132) for senior managers, and 0.028 (n=95) for project leaders. evidently, hammer and stanton's diagnostic variables, although significantly correlated with perceived success, are not good predictors of reengineering project success. finally, we factor analysed the 31 questions used in the questionnaire (appendix a) to see if hammer and stanton's intuitive grouping of questions could be replicated empirically. for meaningful factor analysis it is desirable to have some hundreds of responses and at least five cases for each variable (tabachnick and fidell, 1989). since senior managers and project leaders had both answered the same 31 questions, we were able to combine responses from the two datasets to give a sample size of 234 respondents (88 pairs from the same organization). finally, results from principal components analysis with varimax rotation are shown in appendix b. although six eigenvalues were greater than one, the scree plot suggested no more than four distinct factors, so we report the fourfactor solution here (see appendix b). possible names for the first three factors are, in order, reengineering leadership, management commitment, and organizational readiness. in other words, two of hammer and stanton's expected three factors emerged from the factor analysis, but questions that measured hammer and stanton's organizational readiness factor split into two factors that we called management commitment and organizational readiness. their style of implementation factor did not emerge clearly from the analysis. correlation and regression analysis showed that the new factors were not any better correlated with perceived success than hammer and stanton's factors. limitations of the study as with all research, our work has a number of limitations. in our questionnaire, respondents were required to cast their minds back in time in order to answer many of the questions. the ability of the respondents to recall past events could potentially lead to errors. our study also relied on respondent perceptions. using a survey and perceptual measures is problematic because it introduces the potential for bias; however, for this research a mail survey was the most effective way to reach a large number of geographically dispersed respondents. a second weakness is that with the organisational readiness questions the respondent was required to answer on behalf of the organisation. the potential for bias exists because a single individual is required to make assumptions on behalf of the organisation. we considered that the combination of both senior managers and project leaders recollection and opinion would help overcome these potential weaknesses. as the opinions of senior managers and project leaders were significantly correlated we hope believe bias is not too strong. a third limitation of this research is the lack of a sample of organisations that consider their reengineering exercises unsuccessful. we asked respondents to consider any reengineering project their organisation had undertaken and it seems there could be a degree of self-selection bias as no projects were reported as unsuccessful. 65 ajis vol. 7, no 2 may 2000 figure 2: regression analysis of theme and success variables for both senior managers and project leaders, with correlations between the 88 pairs of senior managers and project leaders from the same organisation. conclusions the reengineering literature stresses the importance of change management in projects and hammer and stanton (1995) contend that organisations should be made ready for change prior to embarking on a reengineering project. this contention is supported by the results of our study, although not strongly. variance in the three independent readiness variables explained little variance in the dependent variable, perceived success (regression analysis). in addition, three of the eight correlations with perceived success were not significant. this investigation was done in conjunction with research into critical success factors for implementation of reengineering projects based on the work of grover et al. (1995) (see table 1). the same project leaders who responded in the hammer and stanton (1995) study were asked to consider the severity of certain problems encountered during implementation of the reengineering project. the results of the grover et al. replication stressed the importance of change management, management support, project management, and planning, during the implementation of reengineering projects, with these factors correlating significantly with perceived success (murphy and staples, 1998). many of the questions asked and factors measured by murphy and staples were similar in content to those asked by the hammer and stanton (1995). this seems to indicate that the impact of the environment prior to reengineering, although important, is less so than the environment and conduct during implementation. hammer and stanton (1995) acknowledge that this is the case with their statement about the diagnostic: "these minimum scores, as we said are what you need before you start. once implementation is under way, however, the required minimum scores go up sometimes way up.... as implementation progresses, intensity increases meaning that leadership, resources and focus, already strong must get stronger. so don't just take this test once and forget it, use it again and again." in order to understand why hammer and stanton's (1995) factors did not correlate strongly with success we compared their factors with those used in the murphy and staples (1998) replication of grover et al. (1995). it was interesting to note the difference between hammer and stanton's (1995) reengineering leadership construct and grover et al.'s (1995) management support construct. the grover et al. (1995) construct focuses on a management team leading the reengineering project, in contrast the hammer and stanton (1995) construct considers a single "reengineering leader". this observation leads us to question the idea, popularised in practitioner literature, of a strong leader single-handedly wielding his or her influence over an organisation in the throes of change. hammer and stanton's (1995) organisational readiness and style of implementation constructs range across the grover et al. (1995) factors of project planning, process delineation and change management rather than being two distinct constructs as defined in the diagnostic. although we consider the diagnostic an important tool for raising awareness in practitioners of issues surrounding implementation of change programs, our analysis raises some question about its rigor. the compound nature of questions and the lack of evidence that they measure the three underlying factors expected by hammer and stanton (1995) lead us to question its use as a prescriptive diagnostic. we suggest that practitioners use it as a guide or 66 ajis vol. 7, no 2 may 2000 descriptive indicator of issues to be managed rather than as a universal yardstick of readiness. finally, although the hammer and stanton (1995) diagnostic focuses on reengineering we suspect its lessons are equally valid for other major programs of change. the diagnostic, while not completely validated by this study, can be used to heighten management's appreciation of the important factors to consider when embarking on any major program of change. references brancheau, j. c, janz, b. d. and wetherbe, j. c. (1996) key issues in information systems management 1994-95 sim delphi results. mis quarterly, 20:2,225-242. broadbent, m. and weill, p. (1995). busting up the business: different approaches to business process redesign and it infrastructure investments. mis (australia), 42-48. broadbent, m., weill, p. and st. clair, d. (1996). the implications of it infrastructure for business process redesign. melbourne business school, the university of melbourne: melbourne. butler, c. (1993). the role of information technology in business process redesign: observations from the literature. melbourne business school, university of melbourne: melbourne. butler, c. (1997). business process redesign: the australian experience. melbourne business school, university of melbourne: melbourne. champy, j. (1998). get ready for more reengineering. sales and marketing management, 150:3, 26-27. demons, e. k. (1995) using scenario analysis to manage the strategic risks of reengineering. sloan management review, 36:4,61-71. demons, e. k., thatcher, m. e. and row, m. c. (1995) identifying sources of reengineering failures a study of the behavioral factors contributing to reengineering risks. journal of management information systems, 12:2,9-36. davenport, t. h. (1993) process innovation : reengineering work through information technology. harvard business school press,: boston, mass.:. davenport, t.h. (1998) putting the enterprise into the enterprise system. harvard business review, july august, 121-131. dillman, d. a. (1978) mail and telephone surveys : the total design method. wiley: new york. de lone, w. and mclean, e. r. (1992) information systems success: the quest for the dependent variable. information systems research, 3:1, 60-95. fowler, f. j. j. (1993) survey research methods. sage publications: newbury park. grint, k., case, p. and willcocks, l. (1996) business process reengineering reappraised: the politics and technology of forgetting. in orlikowski, w. j., (ed.) information technology and changes in organizational work: proceedings of the ifip wg8.2 working conference on information technology and changes in organizational work, december 1995, . chapman & hall on behalf of the international federation for information processing (imp); london. grover, v., jeong, s. r., kettinger, w. j. and teng, j. t. c. (1995) the implementation of business process reengineering. journal of management information systems, 12:1, 109-144. hall, g., rosenthal, j. and wade, j. (1993) how to make reengineering really work. harvard business review, 71:6, 119-131. hammer, m. (1990) reengineering work: don't automate obliterate. harvard business review, 68:4, 104-112. hammer, m. and stanton, s. a. (1994) the reengineering revolution: a handbook harper business: new york. hammer, m. and champy, j. (1993) reengineering the corporation : a manifesto for business revolution. nicholas brealey publishing, allen & unwin: london: st. leonards, n.s.w. hoopes, j. (1995) western civilisation versus the flat organisation. in burke, g. and peppard, j., (eds.); examining business process re-engineering: current perspectives and research directions, kogan page, london. keen, m. (1995) bpr: managing the change process or the process of change managing bpr? in burke, g. and peppard, j., (eds.); examining business process re-engineering: current perspectives and research directions, kogan page, london. manganelli, r. l. and raspa, s. p. (1995) why reengineering has failed. management review, 84:7, 39-43. martinez, e. v. (1995) successful reengineering demands is/business partnerships. sloan management review, 36:4, 51-60. miles, r. e., coleman, h. j., jr. and creed, w. e. d. (1995) keys to success in corporate redesign. california management review, 37:3,128-145. moad, j. (1993). does reengineering really work? datamation. murphy f.k. and staples, d.s. (1998) reengineering in australia: factors affecting success. australian journal of information systems, 6:1. pedhazur, e. j. and schmelkm, l. p. (1991) measurement, design, and analysis : an integrated approach. lawrence erlbaum associates: hillsdale, n.j. romney, m. (1996) has reengineering left you financially stronger? cpa journal, 66:1. tabachnick, b.c. and l.s. fidell, using multivariate statistics, 2nd ed. harper collins, 1989. 67 ajis vol. 7, no 2 may 2000 appendix a qu i. 2. 3. 4. 5. 6. 7. 8. 9. 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 questionnaire statement the reengineering leader possessed the title and authority necessary to institute fundamental change the organisation placed a high value on serving customers the organisations experience with total quality management (tqm) had created an environment that was receptive to reengineering managers were motivated to assure that the processes were successfully reengineered the organisation believed that the commitment of the senior management team would be long-lasting key staff organisations human resources, finance, and information systems were positive about the prospect of reengineering the organisation believed that the reengineering leader was truly committed to reengineering the organisation had none of the complacency and arrogance that often follow a sustained period of success the organisation believed that the senior management team were truly committed to reengineering the reengineering leader had a vision of the kind of organisation he or she wished to create the organisation understood the nature of reengineering, including the fact that it results in multidimensional change that impacts processes, jobs, organisational structure, management responsibilities, etc. the reengineering effort was directed at key business processes rather than organisational units the organisation believed that the reengineering leaders commitment would be long-lasting the reengineering leader truly understood the nature of reengineering the organisation had a solid understanding of customer needs the organisation had the human resources needed to implement reengineering the reengineering leader was able to express the vision he or she had for the organisation clearly and simply in operational terms the organisation as a whole recognised the need for reengineering and fundamental change the organisation was comfortable with the way in which reengineering proceeds, through risk taking, learning, and ambiguity the reengineering leader was ready and able to exercise leadership through communications, personal behaviour, and systems of measurement and reward in order to make reengineering succeed the entire senior management team shared the leaders enthusiasm for reengineering measurement systems and performance goals were established to chart the progress of reengineering key staff organisations human resources, finance, and information systems were capable of innovative responses to reengineering's demands the organisation was free of the scepticism, mistrust, and ambivalence that often follows a period of downsizing or restructuring the reengineering leader was prepared to commit the organisational resources that reengineering required the organisation had the financial resources needed to implement reengineering the members of reengineering teams felt empowered to 'break the rules' and to challenge long-standing assumptions the reengineering leader was prepared to commit the personal attention that reengineering required the reengineering leader understood the magnitude of the change reengineering entailed managers were given end-to-end responsibility for the processes to be reengineered the leader of reengineering was a senior executive who was strongly committed to reengineering h&s theme & qu. rl1 or 15 or 14 si 19 or 9 or 13 or 9 or 10 or 9 rl3 or 8 si 18 or 9 rl2 or 15 or 12 rl3 or 7 si 16 rl4 rl6 si 20 or 13 or 11 rl5 or 12 si 17 rl5 rl2 si 19 rl1 68 ajis vol. 7, no 2 may 2000 appendix b rotated component matrix for 31-question factor analysis (n = 234) nn or nn?o oiios nn?n nn n nim nmn nnr? nu in nn u on 7 mi?? nn n mi 11 nn?fi nn q mi 4 niifi mi s nn i nn? nn 0/1 nn?t mi ifi mi 10 nn ix nn ti nn is mi i? nm mi r i an 7s? 7vs 7?q t>f> 700 fi«s fi7fi fisn sfi4 <no 401 48? 4£1 ? <;in 7fi9 7^8 7^4 7m <s1^ sni /1 1a 4?? ^ a<;i fi->1 <w fil? sn /iai 4?fi 4 s18 408 470 fisfi extraction method: principal component analysis. rotation method: varimax with kaiser normalization, a rotation converged in 8 iterations. factor 1: reengineering leadership factor 2: management commitment factor 3: organizational readiness factor 4: unnamed felicity murphy, peter seddon, and sandy staples © 1999. 69 microsoft word 8 ajis 14-2.doc australasian journal of information systems volume 14 number 2 june 2007 95 information systems outsourcing relationship model richard fleming school of information systems, technology and management the university of new south wales, sydney, australia graham low school of information systems, technology and management the university of new south wales, sydney, australia email: g.low@unsw.edu.au abstract increasing attention is being paid to what determines the success of an information systems outsourcing arrangement. the current research aims to provide an improved understanding of the factors influencing the outcome of an information systems outsourcing relationship and to provide a preliminary validation of an extended outsourcing relationship model by interviews with information systems outsourcing professionals in both the client and vendor of a major australian outsourcing relationship. it also investigates whether the client and the vendor perceive the relationship differently and if so, how they perceive it differently and whether the two perspectives are interrelated. keywords: outsourcing models, outsourcing relationships, outsourcing factors introduction the relationship between client and vendor is a vital component of a successful outsourcing agreement (kern, 1997, kern and willcocks, 2000, koh et al., 2004, kishore et al., 2003, mcfarlan and nolan, 1995). social exchange theory sees the relationship as involving teamwork between the parties to achieve a win-win situation (kern, 1997, kern and willcocks, 2000) while contract theory sees it as a formal agreement between the parties for the exchange of goods and services (fitzgerald and willcocks, 1994, kern, 1997, kern and willcocks, 2000). the approach taken by outsourcing partners in establishing a relationship affects its longterm management. if the relationship is developed from a social exchange theory perspective (soft-based management), it is established to respond dynamically to new requirements from either partner (kern, 1997). if the relationship is developed and managed from a contract theory perspective (hard-based management), it remains simple and easily australasian journal of information systems volume 14 number 2 june 2007 96 controlled. the negative to this approach is that it can be too rigid and unyielding for the partners to manage successfully (kern, 1997). a successful relationship requires a combination of soft-based and hard-based relationship management. hard-based relationship management allows a relationship to be established within a strong contractual framework to govern the ensuing soft-based relationship (fitzgerald and willcocks, 1994, kern, 1997, kern and willcocks, 2000, koh et al., 2004). a relationship that combines the legal contract with mutual trust allows for a much stronger bond between the client and vendor parties (klepper, 1994, lee and kim, 2005, sabherwal, 1999). the latter forms an effective partnership and plays a major role in creating a successful outsourcing agreement (lee and kim, 2005). dibbern et al. (2004) indicated the need for further research into how the client and vendor interact within a relationship in their review of the information systems outsourcing literature. the current research aims to provide an improved understanding of the factors influencing the outcome of an information systems outsourcing relationship and to provide a preliminary validation of an extended information systems outsourcing relationship model within an australian context. it also investigates whether the client and the vendor perceive the relationship differently and if so, how they perceive it differently and whether the two perspectives are interrelated. the paper is organised as follows: we review the relevant literature and then outline the research approach. this is followed by the development of the extended outsourcing relationship model, followed by a preliminary validation and conclusions. previous outsourcing relationship models information systems outsourcing relationship models have been designed to measure and provide an understanding of which aspects of a soft-based outsourcing relationship lead to outsourcing success for both the client and the vendor. within these models the determinant variables of outsourcing success can be categorised as behavioural partnership variables and psychological partnership variables. the former are the key factors that affect the operational performance of the relationship while the latter are the degree to which the parties believe that the partnership will be sustained over time (henderson, 1990, lee and kim, 2005). we review five outsourcing relationship models (henderson (1990), lee and kim (1999, , 2005), kim and chung (2003) and kim and park (2003)) prior to proposing an extended outsourcing relationship model. henderson’s (1990) model distinguished between performance (behavioural) and psychological variables (figure 1). the behavioural variables were referred to as “partnership in action” while the psychological variables were referred to as “partnership in context”. he developed the model through interviews with senior executives together with a follow-up workshop. australasian journal of information systems volume 14 number 2 june 2007 97 partnership in action partnership partnership in context organisational linkage distinctive competency and resources shared knowledgepredispositioncommitmentmutual benefits figure 3 – henderson (1990) partnership model lee and kim (1999) distinguished between “determinants of partnership quality” and “partnership quality” (figure 2). the former is a behavioural dimension that directly influences the psychological dimension (“partnership quality” between the client and vendor). partnership quality (psychological dimension) dynamic factors participation joint action communication quality coordination information sharing static factors age of relationship mutual dependency contextual factors culture similarity top management support trust business understanding benefit and risk share conflict commitment business perspective user perspective determinants of partnership quality (behavioural dimension) figure 2 – lee and kim (1999) research model for partnership quality although the lee and kim (1999) model was statistically significant, lee (2005) recognised it “followed intuition rather than any theories” as the model “could not suggest a strong theoretical background for a 3-stage model (antecedents-process-outcome)”. this led lee and kim (2005) to develop and test three different theoretically supported models based on similar variables and dimensions. the three models were direct modifications of the henderson (1990) model. • model based on behavioural-attitudinal theory – • figure 3 (2-dimensional model with the set of behavioural variables influencing psychological variables). australasian journal of information systems volume 14 number 2 june 2007 98 • simple direct path model (1-dimensional model with all variables directly influencing outsourcing success). • model based on a theory of reasoned action (2-dimensional model with the set of psychological variables influencing behavioural variables). the model based on behavioural-attitudinal theory ( figure 3) was found to be the most effective at modelling an information systems outsourcing relationship using questionnaire data obtained from 225 senior korean information systems executives. it was developed using behavioural-attitudinal theory proposed by kappelman and mclean (1991, 1992). perception of commitment perception of trust perception of mutual benefits shared knowledge organisational linkage mutual dependency outsourcing success behavioural variables psychological variables figure 3 –lee and kim (2005) model based on behavioural-attitudinal theory kim and chung (2003) and kim and park (2003) also developed cross-sectional single-dimensional information systems outsourcing relationship models. kim and chung (2003) proposed a model ( figure 4) where the determinant factors were derived from relational exchange theory and transaction cost theory. they evaluated it using data obtained from a survey mailed to 2200 computer executives. only solidarity, flexibility and monitoring of the vendor were found to be positively related to the success of information systems outsourcing. surprisingly, they found that role integrity and asset specificity were negatively related to success. relational exchange characteristics solidarity continuity expectation role integrity flexibility monitoring of the vendor task characteristics uncertainty asset specificity success in is outsourcing satisfaction percieved benefits figure 4 – kim and chung model (2003) kim and park’s (2003) model considered outsourcing satisfaction for three client groups within an information systems outsourcing relationship: outsourcing project director, users and operators australasian journal of information systems volume 14 number 2 june 2007 99 (figure 5). the project director is responsible for accepting the system provided by the vendor. the users are employees who use the information systems system while the operators are a customer group that represents the maintenance technologists within the client company. as expected, they found that the factors that influenced each of the groups within the client company differed. the project director’s satisfaction was influenced by the transaction relationship and partnership constructs while output performance had a positive influence on the satisfaction of the users and operators, reflecting the fact that short-term consequences are important for these groups (kim and park, 2003). is outsourcing customer satisfaction project director users operators transaction relationship personnel participation past experience transaction period cultural affinity partnership task understanding risk sharing technical level responsiveness output performance task-related is-related figure 5 – kim and park (2003) model research approach this research aims to propose an improved outsourcing relationship model in an australian context. to achieve this aim the following research approach has been adopted. step 1: develop an extended relationship outsourcing model based on the literature discussed in the previous section. step 2: perform a preliminary validation of the factors and structure of the extended outsourcing relationship model. step 3: refine the extended model based on the results of the preliminary validation. australasian journal of information systems volume 14 number 2 june 2007 100 development of extended model in this section we discuss the development of the extended outsourcing relationship model and then consider how the constructs are measured. lee and kim (2005) demonstrated that the model structure developed from behavioural-attitudinal theory, a model structure similar to the lee and kim (1999) model, to be the most effective structure to model an outsourcing relationship. as such it is used as the basis of the proposed information systems outsourcing relationship model (figure 6). this structure has an intervening psychological dimension between the behavioural dimension and the overall outsourcing success of the relationship. two of the behavioural factors within the lee and kim (2005) model, shared knowledge and organisational linkage, proved to be effective positive determinants of the psychological dimension. mutual dependency had a substantial negative influence on the psychological variables, but it is tested in the australian context to determine whether their results “reflect korea’s unique outsourcing situation and environment” (lee and kim, 2005) or whether similar findings will occur in other countries. the psychological dimension in lee and kim’s (2005) study comprised perception of mutual benefits, perception of commitment and perception of trust (predisposition). these factors are included in the proposed model as they demonstrated a significant influence on outsourcing success. the lee and kim (1999) model is similar in structure to the lee and kim (2005) behaviouralattitudinal model. however it includes additional factors to those tested in the later model: participation, communication quality, coordination, age of the relationship, cultural similarity, and top management support as behavioural factors; and business understanding and conflict as psychological factors. of these factors, participation, coordination, communication quality and top management support are included in the behavioural dimension of the extended model as there was an influential relationship between these factors and the psychological dimension. business understanding is included into the psychological dimension since it was found to have a positive relationship with outsourcing success. australasian journal of information systems volume 14 number 2 june 2007 101 perception of commitment shared knowledge organisational linkage client persp. of outsourcing success behavioural dimension psychological dimension perception of business understanding participation top management support perception of commitment perception of business understanding shared knowledge organisational linkage vendor persp. of outsourcing success behavioural dimensionpsychological dimension perception of trust participation top management support perception of mutual benefits perception of trust perception of mutual benefits client perspective vendor perspective mutual dependency commuication quality mutual dependency commuication quality coordinationcoordination figure 6 –extended outsourcing relationship model kim and park (2003) and kim and chung (2003) also developed models that endeavoured to determine the factors within an information systems outsourcing relationship. the majority of factors within these models overlapped with factors that have been extracted from the lee and kim (1999, , 2005) models. of the factors that did not overlap, none were found to have a significant effect on outsourcing success and therefore were not included in the extended model. support for inclusion of the factors in the extended model is also found in the general outsourcing literature. • organisational linkage: (henderson, 1990, mohr and spekman, 1994) • shared knowledge: (henderson, 1990, lacity and hirschheim, 1993, mohr and spekman, 1994) • mutual dependency: (henderson, 1990, kern and willcocks, 2000, klepper, 1994, lacity and hirschheim, 1993, mohr and spekman, 1994) • participation: (henderson, 1990, kim and park, 2003) • coordination: (mohr and spekman, 1994) • communication quality: (mohr and spekman, 1994) • top management support: (henderson, 1990, lee and kim, 1997) • trust: (henderson, 1990, kern, 1997, kern and willcocks, 2000, mohr and spekman, 1994, sabherwal, 1999) • business understanding: (kim and chung, 2003, kim and park, 2003) australasian journal of information systems volume 14 number 2 june 2007 102 • mutual benefits: (ang and straub, 1998, henderson, 1990, kim and park, 2003, koh et al., 1999, koh et al., 2004, mohr and spekman, 1994) • commitment: (kern, 1997, kern and willcocks, 2000, kim and chung, 2003, mohr and spekman, 1994) constructs the proposed extended outsourcing relationship model (figure 6) uses the constructs developed by lee and kim (1999, , 2005) for the behavioural and psychological factors. the behavioural factors are: • communication quality – the timeliness, accuracy, completeness and credibility of communication. lee and kim (1999) regard effective communications as essential for the partners to reach their intended objectives since it assists them in being better informed. • coordination – how well the partners work together to understand each other’s needs and solve complex tasks. coordinated actions between partners are judged to lead to mutual benefits and successful partnerships. • mutual dependency – the extent to which the partners can influence each other as well as the extent to which they depend on each other. it develops through both partners perceiving mutual benefits from their interaction (lee and kim, 2005). • organisational linkage – the level of cooperation and joint effort that exists between the partners. as organisational linkage increases, organisational boundaries become blurred with the integration of activities such as long-range planning, value analysis and training (lee and kim, 2005). • participation – the level of involvement of both partners in the outsourcing relationship. active participation of the partnership members plays a major part in enhancing the longterm sustainability of the partnership (henderson, 1990). • shared knowledge – the extent to which critical information is communicated between the partners. closer relationships should result in more frequent and more relevant information exchanges (lee and kim, 2005, lee et al., 2000). • top management support – how well executives from each partner understand and support the partnership. the psychological dimension variables are: • perception of business understanding – the degree to which the partners understand and comprehend each other’s goals, policies and behaviours. • perception of commitment – the degree of pledge of relationship continuity between the partners. the presence of commitment allows the partners to focus on the long-term benefits rather than short-term benefits. the contractual side of a relationship is an important influence on the relationship commitment of the partners, due to the reassurance that the contract provides. the contract is seen as the first step of commitment; after this stage, the partners become proactive beyond what the contract suggests (lee and kim, 2005, henderson, 1990). • perception of mutual benefits – the degree to which each partner shares the risks and benefits as well as mutual incentives. this construct is only influential if the relationship is australasian journal of information systems volume 14 number 2 june 2007 103 a partnership-based relationship rather than the transactional type where understanding mutual benefits is less significant (lee and kim, 2005). • perception of trust – the degree of confidence and willingness between the partners (lee and kim, 1999). a lack of trust can lead to the client being unwilling to delegate responsibility to the vendor (sabherwal, 1999). within the majority of the information systems outsourcing literature, the client perspective is used to measure the outcome of an information systems outsourcing relationship. the client perspective of outsourcing success is broken down into three measures. • generic satisfaction (grover et al., 1996, heckman et al., 1994, kim and chung, 2003, kim and park, 2003, lee and kim, 1999, lee and kim, 2005, poppo and zenger, 1998). • realisation of client expectations (lacity et al., 1996, lacity and hirschheim, 1993, reponen, 1993). • overall performance of the outsourced system (sabherwal, 1999). since the extended model is intended to apply to both the client and vendor perspectives of an outsourcing relationship, the generic satisfaction measure is the more applicable measure. generic satisfaction may measure different aspects of satisfaction depending on the context: overall satisfaction, business satisfaction, and user satisfaction. the proposed research model employs each of these categories to determine the success of an outsourcing relationship from both a client and vendor perspective. the overall satisfaction measure adopted is adapted from poppo and zenger’s (1998) study that delineates the satisfaction measure into three basic measures of overall cost, quality of service and the responsiveness to problems and enquiries. the business satisfaction measure is derived from the grover et al. (1996) study and has been used along with the user satisfaction to measure information systems outsourcing success in both the lee and kim (1999, , 2005) studies. they measured user satisfaction as: reliability, relevance, timeliness, accuracy, currency, and completeness of the information provided to the users in the outsourcing relationship. preliminary validation of extended model a case study of the outsourcing relationship between a client (client x) and vendor (vendor a) was conducted. the relationship between the two parties is still relatively young with respect to the full length of the contract. however it is sufficiently developed for any initial relationship and contractual issues to be resolved. previous to this information systems outsourcing agreement, vendor a has been involved in a smaller-scale outsourcing relationship with client x which was not regarded by either side as successful. when client x looked to outsource more of its information systems infrastructure, it renegotiated its current outsourced desktop infrastructure relationship to include its network and voice infrastructure as well as development. the renegotiated contract was consciously reestablished to be a more beneficial relationship for both parties. as this is a preliminary validation aimed at providing support for the initial model and any possible modifications, we conducted a limited number of interviews. three interviews were conducted with information systems outsourcing professionals from client x (c1, c2 and c3) who had at least 3 years management experience with relationships from a client perspective and four interviews with australasian journal of information systems volume 14 number 2 june 2007 104 information systems professionals from vendor a (v1, v2, v3 and v4) who have at least 5 years experience with outsourcing relationships from a vendor perspective. each interview lasted approximately an hour. the first phase of the interview process followed an investigatory interview structure similar to that adopted by henderson (1990). • determine the interviewee’s experience within is outsourcing relationships, including the number of years involved and the roles held. • obtain an understanding of the is outsourcing relationships with which the interviewee had been involved (both with the current employer and elsewhere). • determine what factors (with examples) the interviewee believes affect the success of the current outsourcing relationship. • determine the factors of an outsourcing relationship that the interviewee believes contribute to the relationship’s effective execution on a day-to-day basis? • determine the factors of an outsourcing relationship that lead the interviewee to believe that a relationship will be sustained over time? the second phase of the interview considered the proposed extended outsourcing relationship model. the extended model was presented to each of the interviewees and he/she was asked to respond to the following questions. • the perception of the overall model structure and relevance to how the interviewee believes an is outsourcing relationship functions. • the perception of each of the variables within the model and the validity of each of the variables as a determinant of is outsourcing success. • discussion of the feedback effect (e.g. does a vendor’s perception of a satisfied client influence a vendor’s satisfaction.) interview analysis and results this section discusses the results of the interviews with both client and vendor information systems outsourcing professionals. table 1 summarises the client and vendor perspectives of each of the factors. the client and vendor perspectives of an outsourcing relationship and the responsibilities that come with these perspectives are notably different. this differentiation leads to differences in the importance each partner places on the various factors. in the first phase of the interviews, the interviewees discussed their experiences and views of what influences the outcome of an information systems relationship. the client interviewees found trust to be the most important factor with c2 commenting: trust has to be the most important factor that will tend to ensure that the relationship will be last a long period of time. trust doesn’t exist between client x and vendor a; it exists between me and my counterparts at vendor a. you can’t measure trust; you can’t have a kpi on trust. you either trust someone or you don’t and you can’t put together a process that causes someone to trust you, it’s just the way that you behave. c3 had a similar impression: australasian journal of information systems volume 14 number 2 june 2007 105 there can obviously be some major things that undermine the value of the relationship, trust is one of them but we try to differentiate between the individual and the organisation. you can deal with an individual trust problem… but if you have an organisation trust problem it can leave the relationship undermined. the vendor interviewees also found trust to be the most essential factor. v2 conveyed the vendor’s overall perspective: at the end of the day client x has to trust me to make the technical decisions, as that is what they has empowered me to do. communication quality is an important factor for both parties, but the client interviewees considered it more essential as the client had outsourced its systems and infrastructure and required clear communication as to their status. c2 indicated that “after trust the most important factor within a relationship is to communicate and set expectations realistically with your partner”. he also recognised that the existence of “communication is an indication of ‘one team’ benefits.” the client was more influenced by mutual benefits than the vendor as the client placed greater emphasis on getting value for money for the systems and infrastructure it had outsourced. for instance c1 noted that the is outsourcing relationship is “not a zero sum game – that every dollar that vendor a gains is not a dollar that client x loses.” c3 remarked that client a’s objective was to “work with vendor a to see if we could solve things mutually, how they increase their revenue and we increase our services.” the vendor was less influenced by sharing mutual benefits as the contract already stipulates the vendor’s desired benefit and income. v1 remarked that for mutual dependency to exist within an is outsourcing relationship the deal has to be fundamentally sound on both sides, if not both sides will be unsatisfied. the previous deal between client x and vendor a, had some commercial challenges that led to ongoing commercial issues through the life of that contract, which is the reason why we signed a new contract. the vendor’s requirement for the client’s business knowledge was evident in the importance that the vendor interviewees placed on business understanding. v1 commented shared knowledge is critical to the overall success of the whole arrangement. we try to have an open book on all aspects of the engagement other than confidential commercial issues but basically anything that we know, we allow our customer to share that knowledge as well. obviously, the greater the vendor’s business understanding the easier it is to provide a quality service to the client. it is not surprising that the vendor placed an equal emphasis on the shared knowledge behavioural factor. the vendor requires the client to share its strategic and business knowledge to enable it to develop the necessary business understanding. v4 commented “sharing developed understanding is an integral part of making sure that both parties are focused on the common goal” the interviews were able to provide substantial information in regards to the structure, perspective and factors within the extended outsourcing relationship model presented to the interviewees. the behaviour-attitudinal structure defined within the model is consistent with the interviewees’ belief that a partner’s perception of the relationship is more influential on the overall success of the relationship than a partner’s behaviour in the relationship. for instance v2 said he: agreed with psychological dimension, as partners within a relationship [develop] a mental model of how things should happen. this is impacted by good experiences. australasian journal of information systems volume 14 number 2 june 2007 106 there was unanimous agreement that a client perceives an outsourcing relationship differently to how a vendor perceives the same relationship. v4 concurred with other interviewees indicating that it would be interesting to measure and compare the relationship from both perspectives as the vendor is primarily concerned about the requirements within the relationship, whereas the client is more interested with their expectations being met, only when these meet can a similar measurement be used. the interviews found that all of the factors proposed in the model were applicable to both perspectives. although the common consensus of the interviewees was that some of the factors may be more influential on one perspective and less influential on the other (see also table 1). australasian journal of information systems volume 14 number 2 june 2007 107 influence on client perspective influence on vendor perspective difference between perspectives structure of model perspectives within relationships perception of commitment the vendor is more committed, sometimes overcommitted. perception of mutual benefits client has greater concept of the mutual benefits achievable within the relationship. perception of trust ps yc ho lo gi ca l f ac to rs perception of business understanding both recognised the vendor’s requirement to know the client’s business, but vendor realises their limitations caused by their lack of business understanding. shared knowledge client restrict vendors access to their knowledge, vendor goes out of their way to share all knowledge. mutual dependency participation ? ? organisational linkage client shows more initiative to create strong linkages – ‘one team’. top management support b eh av io ur al f ac to rs communication client believes that it is the second most important factor as it sets the expectations of the relationship. output performance output performance was more influential for the client as it inflicted them more – vendor was able to recognise this. cultural similarity all vendor interviewees made reference to the need for cultural similarity as it is an important factor. contract failure establishment of contract client believed that it is an extremely important phases within the relationship a dd iti on al f ac to rs partner maturity feedback effect vendor a judge their successfulness off how satisfied they are with the relationship = rejected factor ? = unverified factor = validated factor = important factor = essential factor table 1 – comparison of how factors were perceived differently by the client and vendor perspectives the interviewees also commented that users with different perspectives within the organisations will perceive the outsourcing relationship differently, signifying that they will be satisfied differently in an outsourcing relationship and that their satisfaction will be influenced by different factors. kim and park (2003) also found differences in the effect that some factors had on outsourcing customer satisfaction for each of the three client groups studied: project director, user and operator. australasian journal of information systems volume 14 number 2 june 2007 108 the interviews were able to confirm the model from both the client and vendor perspectives. the factors within the model were considered appropriate although, interestingly, none of the interviewees directly indicated whether participation was a decisive factor in the model. the requirement for participation was indirectly ascertained from the interviewees’ behaviour and extraneous comments. modification to the extended model although the interviewees confirmed the structure and factors included in the model, they also discussed other factors that they also deemed important in an outsourcing relationship: cultural similarity, output performance, outsourcing experience, perception of partner satisfaction (feedback effect), and contract establishment. the first four factors are included in the revised model (figure 7). contract establishment is not included as its effects should be accommodated by the factors already included within the behavioural and psychological dimensions of the model. figure 7– revised extended outsourcing relationship model cultural similarity both partners noted the omission of cultural similarity and highlighted the influence that similar organisational cultures have on organisations being able to establish a strong relationship. for perception of partner satisfaction perception of partner satisfaction perception of commitment client persp. of outsourcing success perception of business understanding perception of commitment perception of business understanding vendor persp. of outsourcing success perception of trust participation perception of mutual benefits perception of trust perception of mutual benefits shared knowledge mutual dependency organisational linkage coordination top management support communication quality shared knowledge mutual dependency organisational linkage top management support output performance output performance cultural similarity cultural similarity participation communication quality coordination outsourcing experience outsourcing experience vendor perspective client perspective psychological dimension psychological dimension behavioural dimension behavioural dimension australasian journal of information systems volume 14 number 2 june 2007 109 instance v3 commented “cultural similarity plays an important role within the relationship as both partners’ culture allow for an easier relationship to develop.” although cultural similarity was not shown to be an influencing factor within the lee and kim (1999) and kim and park (2003) models, the interviews suggest it should be included in the revised extended model. output performance output performance was not originally included in the model as it had a mixed influence on different groups in the kim and park study (2003). the factor had a negative influence on the project director’s satisfaction levels and a positive influence on the users’ and operators’ satisfaction levels. however v3 commented that “part of keeping client x happy is to deliver satisfactory services and meet our agreements with them.” the influence of performance was also evident from a client perspective with c3 stating that there was a “remarkable improvement in quality and timeliness of delivered systems from vendor a over the last two years.” output performance is an influential factor in regards to determining how an outsourcing relationship is perceived. for this reason output performance is included in the model as a factor within the behavioural dimension. outsourcing experience the interviewees believed that “the customer’s lack of experience causes problems with the outsourcing relationship.” client x and vendor a are both rather immature in regards to their outsourcing experience and find that this immaturity can act as a hindrance to the relationship. however c2 commented: “vendor a’s naivety and lack of experience made it easier for me to manage the outsourcing relationship.” there is a parallel between outsourcing experience and age of relationship which was unsuccessfully tested as a behavioural factor within the lee and kim (1999) model. the factor was employed to represent the belief that longer relationships are more likely to continue than short-term and younger relationships, as the longer timeframe would have allowed the partners to develop the relationship over time. contract establishment the importance of the method used to establish an outsourcing contract was a point that was touched on by many of the interviewees, in particular from the client perspective. they commented that the bid stage was often conducted in a highly confrontational manner. when it is conducted in such a way, it creates a problem when the relationship has to start once the contract been signed. the relationship has not had time to recover from the confrontational bid. the other problem with the contract negotiation stage is that the contract sets the direction for the relationship. if the contract is insufficient in explaining what the client needs or what the vendor can supply, it leads to problems that need to be solved by the newly established relationship. australasian journal of information systems volume 14 number 2 june 2007 110 the feedback effect there was an overall agreement that a feedback effect is an existent part of an outsourcing relationship. c2 commented on the feedback effect being existent from the client perspective as “keeping the vendor happy is just as important as keeping the client happy.” he then went on to say that “a good relationship leads to a good account to work on, which leads to the best employees to work on it. it is in our interests to be a good customer.” however the vendor respondents generally recognised the feedback effect to a greater extent than the client respondents with v1 commenting it’s (feedback) a real effect; if i think that they are happy, i make my team happy, the customer sees a happier team, their team is happier. if there is a major level of dissatisfaction across the customer environment that impacts all levels of the relationship and our internal perception of that does alter the attitudes and behaviours within my organisation. the interviews have shown that the feedback effect is existent in outsourcing relationships. this factor is included as a psychological factor, perception of partner satisfaction, in the revised model. conclusion the overall objective of this research was to provide an improved understanding of the factors affecting the success of an outsourcing agreement. the proposed model was based on previous models described in the literature. interviews with outsourcing professionals associated with one client/vendor outsourcing relationship were conducted. the interviews found a considerable difference between the factors that a client recognises to be essential and the factors that a vendor believes are important. agreement was achieved on the structure of the proposed model along with its factors. however four additional factors were identified and are included in the revised extended model (figure 7): cultural similarity, output performance, outsourcing experience, and perceived partner satisfaction (feedback effect). the current research is useful to management in that it provides an improved understanding of the factors that affect outsourcing success in a major australian outsourcing relationship. it also highlights the importance of the feedback effect with the vendor respondents generally recognising its importance to a greater extent than the client respondents. the validation is limited to a single outsourcing relationship between two organisations. although it supports the factors and the form of the proposed model, it is also limited since it is based on qualitative information from the one outsourcing relationship. to better determine the structural fit of the proposed model and improve its generalisability a quantitative study using questionnaire data across a number of outsourcing relationships is planned. this will be analysed using pls (chin, 1998). references ang, s. & straub, d. w. (1998) "production and transaction economies and is outsourcing: a study of the u.s. banking industry", mis quarterly, vol 22 no 4: pp 535-552. australasian journal of information systems volume 14 number 2 june 2007 111 chin, w. w. (1998) "(1) issues and opinion on structural equation modeling", mis quarterly, vol 22 no 1 (march): pp vii-xvi. dibbern, j., t. goles, r. hirschheim, b. jayatilaka (2004) "information systems outsourcing: a survey and analysis of the literature", database for advances in information systems, vol 35 no 4: pp 6-102. fitzgerald, g. & willcocks, l. p. (1994) contracts and partnerships in the outsourcing of it. fifteenth international conference on information systems, vancouver: canada. grover, v., cheon, m. j. & teng, j. t. c. (1996) "the effect of service quality and partnership on the outsourcing of information systems functions", journal of management information systems: jmis, vol 12 no 4 (spring): pp 89-116. heckman, r., king, w. r. & beachboard, j. (1994) behavioral consequences of customer satisfaction with outsourcing services: discretionary collaboration, relationship commitment, and coflict. fifteenth international conference on information systems, vancouver. henderson, j. c. (1990) "plugging into strategic partnerships: the critical is connection", sloan management review, vol 3 no: pp 7-18. kappelman, l. & mclean, e. (1991) the respective roles of user participation and user involvement in information system implementation success. proceedings of the 12th international conference on information systems. kappelman, l. a. & mclean, e. r. (1992) "promoting information system success: the respective roles of user participation and user involvement", journal of information technology management, vol 3 no 1: pp 1-12. kern, t. (1997) the gestalt of an information technology outsourcing relationships: an explanatory analysis. eighteenth international conference on information systems. kern, t. & willcocks, l. p. (2000) "exploring information technology outsourcing relationships: theory and practice", journal of strategic information systems, vol 9 no: pp 321-350. kim, b. & park, k. (2003) "satisfying different customer groups for is outsourcing: a korean is company's experience", asia pacific journal of marketing and logistics, vol 15 no 3: pp 48-69. kim, s. & chung, y. s. (2003) "critical success factors for is outsourcing implementation form an interorganisational relationship perspective", journal of computer information systems, vol 43 no 4: pp 81-90. kishore, r., roa, h., nam, k., rajagopalan, s. & chandhury, a. (2003) "a relationship perspective on it outsourcing", communications of the acm, vol 46 no 12: pp 86-92. klepper, r. (1994) "outsourcing relationships", in khosrowpour, m. (ed.) managing information technology investments with outsourcing. harrisburg, pa: idea group publishing, 218-243 p. koh, c., ang, s. & straub, d. w. (2004) "it outsourcing success: a psychological contract perspective", information systems research vol 15 no 4: pp 356-373. koh, c., tay, c. & ang, s. (1999) managing vendor-client expectations in it outsourcing: a psychological contract perspective. 20th international conference on information systems. australasian journal of information systems volume 14 number 2 june 2007 112 lacity, m. c. & hirschheim, r. (1993) information systems outsourcing: myths, metaphors, and reality, new york: john wiley and sons. lacity, m. c., willcocks, l. p. & feeny, d. f. (1996) "the value of selective it sourcing", sloan managment review, vol 37 no 3: pp 13-25. lee, j.-n., huynh, m. q., chi-wai, k. r. & pi, s.-m. (2000) the evolution of outsourcing research: what is the next issue? 33rd hawaii international conference on systems sciences. lee, j.-n. & kim, y.-g. (1997) "information systems outsourcing strategies for affiliated firms of the korean conglomerate groups", journal of strategic information systems, vol 6 no 3: pp 203-229. lee, j.-n. & kim, y.-g. (1999) "effect of partnership quality on is outsourcing success: conceptual framework and empirical validation." journal of management information systems, vol 15 no 4: pp 29-61. lee, j. n. (2005) explanation of differences between 1999 model and 2005 model. in fleming, r. (ed.) sydney. lee, j. n. & kim, y. g. (2005) "understanding outsourcing partnership: a comparison of three theoretical perspectives", ieee transactions on engineering management, vol 52 no 1: pp 43-58. mcfarlan, f. w. & nolan, r. l. (1995) "how to manage an it outsourcing alliance", sloan management review, vol no winter: pp 9-23. mohr, j. & spekman, r. (1994) "characteristics of partnership success: partnership attributes, communication, behavior, and conflict resolution techniques", strategic management journal, vol 15 no: pp 135-152. poppo, l. & zenger, t. (1998) "testing alternative theories of the firm: transaction cost, knowledgebased, and measurement explanations for make-orbuy decisions in information services", strategic management journal, vol 19 no: pp 853-877. reponen, t. (1993) outsourcing or insourcing. the 14th international conference on information systems. sabherwal, r. (1999) "the role of trust in outsourced is development projects", communications of acm, vol vol. 42 no no.2: pp 80-86. the australasian journal of information systems (ajis) began in 1993 and was based on the simple realisation that there was a need for an academic ‘outlet’ for is research in australia australasian journal of information systems volume 18 number1 2013 5 decision-making model for business process outsourcing of enterprise content management zhuojun yi sun yat-sen university email: xiaobuzi@hotmail.com dongming xu university of queensland email: d.xu@business.uq.edu.au abstract business process outsourcing (bpo) of enterprise content management (ecm) is a growing though immature market. bpo of ecm focuses on pursuing market transactions in the process of managing all types of content being used in organizations. however, inferior sourcing decisions lead to organizationally sensitive content exposure, high transaction cost, poor outsourcer performance, and low flexibility. ecm bpo in general is rarely discussed in academic literature with any discussion being found on decision-making strategies in ecm bpo. in this paper, we present a decisionmaking model for ecm bpo that will fill the knowledge gap and thus guide industry practitioners with ecm sourcing decision-making strategies. our proposed decisionmaking model includes two parts. part 1 involves an ecm functional framework that shows what functionality component or functionality combinations can be outsourced. part 2 involves a decision-making model that provides guidance for decision making in ecm bpo. we apply the model in two case studies, the results of which indicate that the model can successfully guide the sourcing decision-making process for organizations, and determine the factors when considering sourcing alternatives in ecm. keywords: enterprise content management (ecm); business process outsourcing (bpo); decision-making model introduction enterprise content management (ecm) focuses on managing all types of content being used in organizations. it is a convergence of previous approaches that focus on managing only particular types of content (grahlmann, et al., 2012). however, the field of ecm is still in its infancy (smith & mckeen, 2003) with scientific literature on ecm being limited (grahlmann, et al., 2012), and ecm business process outsourcing (bpo) still a virgin field. the limited literature gives little insight into how enterprises make strategy, make decisions, and manage ecm business process outsourcing. bpo is an important strategy for organization management that aims to avoid unnecessary organization expansion, reduce cost of human resources, maximize profits, focus on core competitiveness, and acquire resources/capabilities not available within the organization. inadequate attention to business processes being outsourced is a major factor contributing to bpo (aron & singh, 2005). companies spend insufficient time evaluating processes for outsourcing so that what assessment is done is performed too narrowly (mcivor, et al., 2010). mailto:xiaobuzi@hotmail.com mailto:d.xu@business.uq.edu.au australasian journal of information systems volume 18 number1 2013 6 bpo of ecm has also been developing. prism (the global trade association for information management companies) had 586 members at may 2013, 455 of whom were located in north and south america. although many of the outsourcing companies are of small to middle size, large international examples exist. using information and communication technologies, more and more functionalities in content management can be supplied from the market. the ecm market, though immature, continues to grow, involving a number of content management vendors (content technology vendor map, 2013). as the market grows, bpo of ecm becomes a considerable option for organizations. nevertheless, ecm outsourcing is different from other outsourcing processes. besides cost saving and focusing on core business, managers need to look more broadly at ecm. the objectives of ecm in an enterprise have been identified by päivärinta & munkvold (2005) to include: improved internal and external collaboration through digital content in and among enterprises with commonly enacted practices; value-added or new customer services and products; reliability and quality of information content resulting in fewer errors; the modern and professional image of the enterprise; efficiency, effectiveness and flexibility of knowledge work and business processes; meaningful knowledge work involving easier and less tedious human routines; organizational memory recording an enterprise’s practice, history and transactions; satisfying external regulations and standards; and platforms and capabilities to develop and maintain targeted applications quickly. the resulting impact of ecm bpo must accommodate, but not be limited to the above objectives. moreover, risks deriving from outsourcing are one of the critical considerations in bpo of ecm, owing to the information characteristics of enterprise content, such as confidentiality, sensitivity, security and management control requirements, and the transaction cost in the is field. on the other hand, ecm is related to information resource management, electronic document management, knowledge management, it technology and infrastructure. specialized capabilities attainment is one of the important strategies. many enterprises seeking bpo of ecm expect better ecm performances through strategic introduction of professional ecm solutions from intermediate market. therefore, the decision about assessing ecm bpo needs guidance. however, the literature on ecm bpo is still a blank, according to our findings. this means that how a company decides to conduct ecm bpo or not is critical. as a result, the existing knowledge in bpo is insufficient to guide ecm bpo. this paper develops a decision-making model for ecm bpo, which includes the determinants of the decision-making process for organizations facing the dilemma. we continue through improving the functional ecm framework by grahlmann, et al. (2012) to explain what can be outsourced in ecm within an organization. based on the extended functional framework, an organization can employ our decision-making model to determine whether or not to outsource a specific functionality or combination of functionalities. the remainder of this paper is organized as follows. in the next section, background knowledge is provided, including ecm, ecm functional framework, decision making in strategic business process outsourcing, and transaction-cost theory. the extended ecm functional framework is presented in the subsequent section. we then provide the ecm bpo decision-making model, followed by two case studies which demonstrate the first evaluations of the model. the conclusion highlights the contributions and limitations, and provides recommendations for future research. australasian journal of information systems volume 18 number1 2013 7 background overview of enterprise content management grahlmann, et al. (2012) discovered that there is no consensus on the definition of ecm. the definitions found were from different perspectives on content (information, user, system), technology, enterprise, and process. a consistent and comprehensive definition of ecm is that it comprises the strategies, processes, methods, systems, and technologies that are necessary for capturing, creating, managing, using, publishing, storing, preserving, and disposing of content within and between organizations (grahlmann, et al., 2012). furthermore, o’callaghan & smits (2005) states in an ecm strategy framework, ecm needs specification, value assessment, cost assessment, content objects portfolio decisions; existing infrastructure (cm staffing included). an ecm strategy framework view is partially consistent with strategic outsourcing decisions, such as value assessment, existing resource, and cost assessment. however, the ecm strategy framework does not take the market into consideration when making strategy. enterprise content management functional framework figure 1: functional framework (grahlmann, et al. 2012) grahlmann, et al. (2012) summarized the functionalities that an ecm can provide. the access layer refers to functionalities for users and information systems to interface with the ecms in order to retrieve content and/or to use/invoke other functionalities. functionalities related to control and coordination refer to the process layer, while those related to capturing, manipulating, using and publishing content refer to the services layer. the fourth repository layer includes the functionalities related to the storage and preservation of content. according to the guidelines of grahlmann, et al. (2012), the framework is comprehensible, usable, complete, generic enough to be suitable for making comparisons, distinguished enough for capturing differences, and future-proof. this original functional framework indicates directions of analysis units’ segmentation. according to this australasian journal of information systems volume 18 number1 2013 8 framework, enterprises can use an ecm function or a functionalities combination as the analysis units for making evaluations. overview of decision making in strategic business process outsourcing since analysis units have been discussed in the last section, the next question is which ecm unit can be outsourced. therefore, in this section, we explore the theoretical background for ecm outsourcing and sourcing decision. in strategic bpo, outsourcing results and the core competency issue as the driving force are discussed in the literature. the resource-based view is also widely discussed in the bpo literature. achieving sustained competitive advantage is discussed as another driving force and important issue to outsourcing other than focusing on core competency and cost saving. in the final section, the literature discussed decision making in sourcing. holcomb and hitt (2007) define strategic outsourcing as the organizing arrangement that emerges when firms rely on intermediate markets to provide specialized capabilities that supplement existing capabilities deployed along a firm’s value chain. holcomb and hitt (2007) also provide three possible economic motives behind strategic outsourcing: reducing bureaucratic complexity, improving production economies when firms fail to achieve sufficient production scale to overcome cost disadvantages, and financial advantage. furthermore, quinn and hilmer (1994) suggest that one of the concentrates for strategic outsourcing is a firm’s set of core competencies where it can achieve definable pre-eminence; the other of the concentrates is for outsourcing other activities which are neither critical nor something in which the company has special capability. gilley and rasheed (2000) identify substitution-based and abstention-based strategic outsourcing. as a result, strategic outsourcing creates cost economies, also creates economic value, especially when production involves the use of potentially more valuable specialized capabilities (fine and whitney, 1999; mowery, et al., 1996). the theory of firm resources and sustained competitive advantage (barney, 1991) has been the foundation theory for strategic business-process outsourcing. because of firm-resource homogeneity and immobility, not all firm resources hold the potential of sustained competitive advantage. therefore, a resource-based view in strategic outsourcing concerns the role of specialized capabilities obtained through intermediate markets. holcomb and hitt (2007) address those four different conditions affecting the value of capabilities sourcing from intermediate market: complementarity of capabilities, strategic relatedness consisting of goal congruence and commonality of knowledgesharing routines, relational capability-building mechanisms, and cooperative experience. sourcing decisions have their origins in make/buy alternatives (tayles & drury, 2001). tayles & drury (2010) develop decision-making logic for manufacturing companies, ensuring wider issues are considered in a logical manner based on rbv theory. their sourcing-decision model not only applies to transparent and operational activities, but also to non-manufacturing activities such as support services. the starting point of tayles & drury’s (2010) model is whether the process or component is prima facie strategic, according to a core competencies focus. the next logical question involves the cost associated with in-house build or out-sourcing, which is supported by cost economies theory. the model considered strategic review in the sourcing decision-making process. if the process or component is not deemed strategic, the decision would go to whether the company possessed the capability. if it does not, and a supplier exists that can satisfy the company’s needs regarding quantity, quality, delivery and price, then the company should buy, which means pursuing outsourcing. if the capability exists in-house, resources are available, and their use is shown to be economical, then the company should retain them in-house. transaction cost is an important aspect when conducting strategic outsourcing also. australasian journal of information systems volume 18 number1 2013 9 overview of transaction cost theory in it outsourcing transaction cost is a crucial consideration in is outsourcing (alvarez-suescun, 2010). aubert & weber (2001) said decision-makers must weigh up the production and transaction costs associated with executing a transaction within their firms (insourcing) versus the production and transaction costs associated with executing the transaction in the market (outsourcing). it’s argued that environmental factorsasset specificity, uncertainty, and small numbers trading affect the type of governance decision for a transaction (williamson, 1985). williamson (1985) is unequivocal about the primacy of asset specificity. aubert, et al (2004) states that assets can be machinery required to manufacture a product, knowledge needed to perform a service, or even appropriate location convenient for dealing with the other party. aubert & weber (2001) argue that the essence of asset specificity is that lock-in effects occur, which potentially lead to hold-up problems. when asset specificity is high, regardless of investments in physical or human capital, the investor needs an insurance contract or mechanism to protect itself. increased uncertainty in a transaction will lead to increased use of internal governance. two types of uncertainties has been addressed (aubert, et al., 2004). one type is demand uncertainty, referring to parties who do not know ex ante the volume that will be required or ignore the form the service will take. another type is related to measurement. evaluating the service or product, in terms of quality, quantity, and timeliness is of great importance when parties step forward to the contract negotiating stage. the uncertainty is also related to contingency. there may be too many contingencies for an exhaustive list to be drawn, and they may be too complex to estimate the probabilities. these kinds of uncertainty also increase transaction cost. lacity and willcocks (1995) specify a transaction as either occasional or recurrent, to measure frequency. aubert, et al. (1996) argue that low frequency transactions are likely to be organized through market interactions or with the help of any general governance mechanism available in the community. rather than create new governance or expanding an existing one for low frequency transactions, firms are more likely to bear the risk. and internalization of transactions by the firms is only efficient for recurrent transactions (aubert, et al., 1996). the transaction cost framework can explain sourcing behavior, such as the choice of sourcing and the contract between a firm and its outsourcer. according to the framework of aubert, et al. (1996), for the transactions with high-asset specificity, internal governance is expected to apply to those, of which high uncertainty combines with high frequency. when transactions have high asset specificity, low frequency, but high uncertainty, relational governance can emerge. in this framework, a complete contract for outsourcing applies to transactions of high-asset specificity and low uncertainty. figure 2: transaction cost framework (aubert, et al, 1996) australasian journal of information systems volume 18 number1 2013 10 development of bpo decision-making model of ecm ecm modelling an ecm model may divide ecm into transactions or amounts of specific functionalities, which are able to show which part of the ecm transactions can be analyzed, evaluated, and even used to conduct market transactions or outsourcing. in the first sections, we need to re-examine the functionalities of ecm, and thus generate an ecm functional framework. what follows determines the factors in decision making in ecm bpo, leading to the next section, where we develop our decision-making model. paper-based content ecm in the information system field mainly focuses on managing digital-based content or strategies, processes, technologies, and methods for managing digital-based content, which does not accord with ‘all types of content’. in the functional framework by grahlmann, et al. (2012), all functionalities in the access layer are digital related or information communication technology based, involving clientapplication integration, desktop application, eai interface, intranet, extranet, and the organization’s website. functionalities like physical records and localization have been mentioned in the service layer, but they can only be accessed digitally. furthermore, other functions based on paper-based content are not included in the repository layer, such as physical records which can be evidenced in legal affairs. therefore, a paper-based content view should also be adapted in the functional framework. the scarce ecm literature mentioning paper-based content led us to look at the records management and archive management literature in general as a supplementary knowledge basis for our functional framework. according to american archival theory, current records are looked upon as future archives to provide advice on how they can best be handled. management of semi-current records and current records led to important thought and action in the establishment of records management programs (pinkett, 1981). the life span of a record from creation to final disposition is called the record cycle. it begins at creation, followed by classification, storage, and retrieval. after a determined period of time, records are transferred to other places — perhaps within the organization to a less expensive storage site in terms of overhead costs. appraisal is the function of determining whether to retain forever or a stated number of years based on the value of records. at the end of the stated number of years, the record is destroyed or put into the archives, a permanent storage place (zhaoyu, 2008). in the book “modern archives”, theodore r. schellenberg (1903-1970), puts forward the ideas of the primary and secondary value of records. primary value is based on a record’s administrative justification, while secondary value is rooted in the historical and evidential value. schellenberg divides the secondary value into evidential and information value. records with evidential value document the organization and functioning of the creating agency (stapleton, 1983). reference service is another important functional area. it is based on the view that public records or records in a public repository should be available for use to the maximum extent consistent with the public interest. a reference service may increase the availability and use of archives. meanwhile, maximum accessibility must be balanced with a concern for individual rights, legal safeguards for personal privacy, public security, and intellectual property. in archives management, records survey and statistics of collection and use of records are routines and the basis for the whole management, which provide measurements for paper-based records transactions, such as the volume, and records services quantity and quality. when the organization australasian journal of information systems volume 18 number1 2013 11 considers intermediate storage, records center, destruction services, and related services from market sourcing, statistics from records survey provides a means to decrease the uncertainty of organizational demand and requirement. ecm functional framework according to the “all types of content” definition of ecm, a paper-based view should be added to ecm functions. we called it the extended functional model based on the functional framework by grahlmann, et al. (2012). in developing countries, paper-based content, also called records, documents or archives, is the dominant type of content within organizations. resources devoted to information systems may be considered less competitive than those devoted to capital assets, equipment, facilities, etc. enterprises in developing countries are less likely to manage and retain all content in information systems or data centers. most firms implement some types of content management software or systems; but the measures adopted tend not to be reliable, safe or capable of storing and maintaining easy access to digital content, nor to have enough technology and resource support. ‘paper-based’ is still the choice for most firms in developing countries, since paper-based records can support the basic content management requirements within organizations. we extract five specific management functions from both records and archive management to supplement paper-based content functionality in the ecm model. the five added functions do not overlap the existing functions in the original framework by grahlmann, et al. (2012). in the access layer, the function of the records management department is listed as the entrance for paper-based content. the records management department may use aids such as a catalogue, a general inventory, or an index list. the department may also provide reference services to direct access the content. in the repository layer most like archive management functionalities, we add two more uses for the content function/value: reference and evidence values. the archive management sub-category is developed in the same layer, including appraisal and records survey as the last two adding functions. the ecm functional model is demonstrated in figure 3. some of the functions are identical, overlapped, or similar to both digital-based and paper-based content management. accordingly, we do not add records or archive management terms of these functions in the model. for example, content aggregation delivers similar function of records collecting or acquisition. imaging is the technique of transforming paper-based content to raster objects. information retrieval can apply to digital-based and paper-based content. localization refers both to where to keep the digital-based or paper-based content, and where to transfer them. physical records refer to the management and use of paper-based content or content in other physical carriers. printing is the method of transforming digital-based content to paper-based. content storage is needed in the repository layer for both digitaland paper-based content. archival description contains the metadata function such as the data set name and data fields as well as similar information. the taxonomy shown in figure 3 is a system of classification for digitaland paper-based content. version management not only refers to versions of content management software and different digital formats transforming, but also refers to the historical versions of specific records as well as the original copy and duplicate/backup copy for digitaland paper-based content. overall, both digitaland paper-based content can support auditing function in ecm. in our case study section, we apply the ecm functional model to exemplify the content management components in enterprise. based on the framework shown in figure 3, organizations may go forward to the sourcing decision-making process. australasian journal of information systems volume 18 number1 2013 12 factors in decision making for bpo of ecm according to the ecm functional framework, an enterprise can use an ecm function or a functionalities combination as the analysis units, then make outsourcing evaluations. there are three groups of factors that drive ecm bpo decision. one group of factors is estimated sourcing decision results while others involve factors to consider in the ecm bpo model shown in figure 4 and figure 5. the sourcing decision in ecm is identified in the first sub-section ; the important issue of core competencies relatedness is discussed in the second sub-section, along with several other informationfeatured considerations; and finally, in the third sub-section we discuss other considerations in the model. sourcing decisions there are four outcomes for the model, which are the same for the two decision-making figures we discussed. “make” refers to adapting internal management strategy without additional resource. “invest/acquire resource” also adapts internal management, but through internal sourcing to acquire necessary resources. “partial outsourcing” means a relational management style, the enterprise, and the outsourcer developing a cooperative relationship. “outsourcing” stands for substitution-based outsourcing, pursuing external management, discontinuing internal management, and replacing existing activities or processes with capabilities provided by an intermediate market. contract type is related to asset specificity. when the outsourcer invests in an asset to support a transaction, the asset may have little or no value apart from that transaction. if the asset specificity is low or acceptable, ordinary contracts may be used for short-term situations. if an asset is highly specific to the transaction, the investor may use an insurance contract to minimize cost. one type of insurance contract is long-term, where an adequate period of time is locked in and the cost of the investment is more likely to be recouped. australasian journal of information systems volume 18 number1 2013 13 figure 3: ecm functional framework core competency relatedness the logical starting point for our model is the question as to whether ecm functionality relates to core competencies or not. ecm functionality relates to core process/key source/strategic component usually existing within the enterprise in particular forms, like activity monitoring, business intelligence, work-flow management and project management, owing to the supportive and monitoring functions of ecm. the process or functionality in ecm is highly revealing of the content or information. firstly, information confidentiality requirements would be extremely high for ecm outsourcing related to core competencies. sensitive or confident content should be shared within a specific range of people and departments in the organization. an ecm function revealing confidential or sensitive content is more likely to be kept under internal management. however, there are other means to protect confidentiality such as limiting exposure of confidential content by segregating confidential information to keep it internal. if this is a practical option, then the next consideration in the decision-making process can be addressed. if not, it is better to return to in-house management. another way to protect secrecy is by outsourcing in partnership (partial outsourcing), by which the enterprise outsources the equipment, technology etc. to obtain the necessary resources and capability, but does not outsource the actual management process or functionality. secondly, there would be no tolerance for information insecurity for ecm outsourcing related to core competencies, which should apply higher information security standards and provide the resources required to maintain security standards. if security standards can not be met, the decison should return to internal management or relational management. however, for ecm functionality not related to core/process/key resource/strategic component, there is an extended tolerance for information insecurity. thirdly, management control must be maintained within the organization for ecm outsourcing related to core competencies. yang, et al. (2006) noted that the most prominent risks in outsourcing are to information security and loss of management control because an enterprise needs to maintain management control for ecm functionality related to core competencies. in the condition of lost management control, process and spec of the specific ecm functionality should be changed. fourthly, there is little scope for flexibility for ecm outsourcing related to core competencies. flexibility in management is valued above maintaining control in non-core competencies related circumstance. yang, et al. (2006) argue that outsourcing non-core activities also increase flexibility through the better use of internal resources, enables quick responsiveness to customer needs, and decrease financial risks by reducing capital investments. when the flexibility evaluation is low, the process or services being evaluated should be modified. other considerations firstly, the cost of the ecm functional process conducted in-house and outsourced must always be compared. if the cost to maintain the process in-house is cheaper than outsourcing, the next question is whether the capability is available within the organization. internal management by make is the decision when resource and capability exist in the enterprise. when the answer is no, if the cost of make (incurring the necessary investments to internalize the process) is cheaper than the buy resource or capability from the intermediate market, the next question is whether the resource can be acquired. if it can be bought, the decision goes to internal management through investing or acquiring resources. if buy is cheaper than make, or the resource cannot be acquired by the enterprise itself, partial outsourcing is conducted to obtain requisite resource or capability. secondly, when transaction cost evaluation is necessary for the enterprise to evaluate asset specificity, uncertainty and frequency separately, we apply the transaction cost framework by aubert, et al. (1996) to determine the evaluation. whenever the asset specificity is low, market transactions should australasian journal of information systems volume 18 number1 2013 14 be considered with priority. if asset specificity is evaluated as high with low uncertainty, outsource under a long-term contract is likely to emerge. when high uncertainty combines with high frequency, it is better to maintain internal government or management. if high uncertainty combines with low frequency, relational management should be conducted. therefore, if market transactions and outsourcing contracting will occur in an organization, then transaction cost evaluation is true and a decision will next take supplier availability into consideration; otherwise, if the relational government and internal governance will be employed and then the decision will take resource/capability availability into consideration. thirdly, supplier availability in the subject conditions cannot be ignored. in addition, the average level of suppliers’ services or production quality is an important concern. if the market does support such ecm functional transactions with acceptable quality in the mass, the decision can step further to the confidentiality question. model development the logic for ecm sourcing decisions is developed to link all considerations and decisions’ results. in the light of tayles & drury’s (2001) decision-making logic for manufacturing companies, our sourcing decision-making model develops the logic for ecm sourcing to ensure that all issues are considered in a logical manner. we develop two figures to demonstrate the sourcing decision-making processes for ecm functionality related to core competencies and those not related to core competencies respectively, since the core competency issue is the starting point of our model. the model is developed into sequential consideration logic. simply by going through the former consideration and getting the appropriate evaluation result, the decision can lead to the next consideration. if an evaluation gets an inappropriate result, the decision would take the other direction shown in figure 4 and figure 5, or need to change the process or spec and start again. for example, if the ecm functionality is not related to core competencies and also not existing within the organization, it is an innovation to pursue these ecm functions from market sourcing. supplier availability and transaction cost evaluation are the next two logical questions. then the cost acceptance is taken into account whether the new added functionality in ecm can be financially justified. if the answer is yes, information insecurity tolerance should then be evaluated. with high insecurity tolerance and high flexibility, the new ecm functionality can be outsourced. if any one of (i) a supplier is unavailable, (ii) the transaction cost evaluation has no result, (iii) the cost is unacceptable, (iv) there is low information tolerance, or (v) there is low flexibility is the case, the enterprise should change the process or spec and go through the decision-making steps again. australasian journal of information systems volume 18 number1 2013 15 figure 4: decision making model（ecm functionality related to core competencies) figure 5: decision making model（ecm functionality not related to core competencies) australasian journal of information systems volume 18 number1 2013 16 case study methodology roethlisberger & lombard (1977) suggest that case research is particularly appropriate for certain types of problems: those in which research and theory are at their early, formative stages. bonoma (1983) adds sticky, practice based problems where the experiences of the actors are important and the context of action is critical. in this paper, case research enables research in a natural setting. rather than participation, the observation and investigation in the case study can test our model inside a natural setting. moreover, the implementation of the decision guided by our model is a complex process involving multiple actors and is influenced by unexpected events. benbasat, et al.(1987) argue that a case study methodology is well-suited to identifying key events and actors and to linking them in a causal chain. as a result, we adopt a case study with a two-case design. the two cases cover two types of enterprise contents: digital-based and paper-based content, and two corporations, a realty development company and an electric power plant. they will yield more general research results, and indicate how the model may be extended. as to the selection of the organizations, the case study can only be conducted if the organization has experienced the ecm outsourcing process or is confronted with the outsourcing decision. generally, those organizations paying close attention to internal information management intend to outsource or partially outsource ecm, because they recognize the asset value of the content. so the organization selection would focus on those emphasized on content or information management. evaluation criteria kung & cheng (2004) contributed to the evaluation criteria that affect the performance of outsourcing activities. five categories of performance evaluation criteria have been extracted, which are efficiency, quality, innovation, customer responsiveness, and integration capability. these criteria are based on firm level. but in the ecm outsourcing field, we looked further than production, research and development, and innovation might not be a proper factor to evaluate outsourcing performance, choosing rather to consider factors at both process and firm level. as a result, we turn to literature focusing on information system outsourcing. wang, et al. (2008) stated that the expected benefits for it outsourcing include cost reduction, improved cost control, improved technology or technical service, focusing business on core competencies, access to new technologies and technical talents, and improved flexibility. the outcome measures that adapt both the processand firm-level view comprise expectations realization, the satisfaction with technological, strategic and economic benefits and market value of the client firms. we found that integration capability is the factor included by both firmand process-level evaluation. besides, we chose seven factors among the above for our evaluation of ecm outsourcing performance, which are efficiency, quality, customer responsiveness, cost saving, and focusing on core competencies. we applied our models in two case studies in which we derived the impacts of the sourcing decision in ecm based on the above criteria. real estate sales system in this case, the realty industry enterprise is located in southern china and has been developing different kinds of real estate for more than 12 years. it is a middle size, single-billion renminbi corporation, with 238 employees. the enterprise operates several construction sites in different cities, and sells them when projects are finished. construction projects managers manage individual sites, but the sales management and control remain at headquarters. the sales volume reached 4.2 hundred millions rmb in 2010, 4.5 hundred millions rmb in 2011, and is scheduled to double in the near australasian journal of information systems volume 18 number1 2013 17 future. the sales department is directly under the ceo of the enterprise, and is responsible for establishing overall sales policies and strategies including competitive positioning, business collaborations and policies, and strategies for sourcing decisions. in keeping with its sales management purposes of centralization and distinctiveness, the sales department decided to strip most sales operation processes from the office automatic system, and transfer them to a new sales system. in the meanwhile, the ecm functions related to sales are crucial to sales management and the whole capital chain in the enterprise. it became a question for the heads of the sales and information department as to how to source the ecm functionality in sales. the ecm functionality needed in this case is embedded in the whole sale process/component, and is illustrated in our ecm model in a grey background in figure 6. most of them are digital-based content functions. although some of the functions exist in the sales system, as desktop application, project management, business intelligence, information retrieval, and version management, more is needed, such as the eai interface with the existing financial system and office automatic system, case handling support, collaborative editing, monitoring, content aggregation, electronic records and taxonomy etc. the decision-making process that we derived through semi-structure interviews is illustrated in figure 7 in a grey background. the ecm functionality related to sales management, which is definitely the core process and strategic component to the realty development enterprise. first of all, they stated that the human cost of designing ecm functionality in systems within the organization was impossible. whether the enterprise developed these ecm functions in the sales systems or relied on the old office automatic systems to link up to the new sales system, the costs were unacceptable if its sales department conducted internal sourcing. so the decision went to market transactions. secondly, interviewees considered questions about transaction cost evaluation. we found that the design of the ecm functions in the system is occasional, while the use of these functions is recurrent. furthermore, the outcome of the functions’ application design is not uncertain as long as they can control the contract with the outsourcer. they found out that sales-related ecm functions existed in other companies in the same industry. the functions illustrated in figure 6 were in the low level in asset specificity. undoubtedly, the content relating to sales was sensitive and confidential, which cannot be avoided in the management, use and storage of them. after the above considerations, the enterprise decided to outsource the design and programming of the ecm functions they needed. as a result, they started to negotiate with the supplier of the new sales system on whether they could achieve these ecm functions in the new system. according to the existing ecm functions in the original sale system, the supplier added the absented functions in figure 6. the contract between them was modified, limiting the technological support and consultancy by the supplier after the system was put into effect. the realty company and the sales system vendor developed a cooperative relationship in the implementing of and training in the new sales system and ecm functionality related to sales. australasian journal of information systems volume 18 number1 2013 18 figure 6: the needed ecm functionality in real estate sales system the practical application of the decision-making model during the case study indicated that the model can generally guide decision making. the realty development enterprise followed the logical process in the model and went to the right sourcing decision. this led to benefits derived from enhanced it performance leading to enhanced efficiency of sales management; faster structured data aggregation, case handling, and collaborative editing contributing to the efficiency and the customer-response factor; and new price policy and discount approval decisions available faster to sales or real estate management advisors so that deals can be made faster. in addition, sales performance can be analyzed daily or weekly rather than monthly, which is the former expectation to the modification flexibility. after the taxonomy adjustment, the storage of electronic records and different version sales policies can serve the needs for authorized internal consultancy, the quality of which can support auditing and reference. as the major purposes of outsourcing, cost saving and focusing on core competency, the relational management decision contributes most. the modified contract developed the links among each of the sales, financial, and office automatic systems, included test, secondary development and following training, with the 78% price of the basic sales system and being half the price of the office automatic system. most important of all these evaluation factors, the cooperative-relationship decision let the enterprise focus on carrying forward constructions and promoting its sales. australasian journal of information systems volume 18 number1 2013 19 figure 7: the ecm sourcing decision for the real estate sales system power plant extension construction archives project the power plant is a state owned corporation located in the suburbs of a second-line city in southern china where it generates electricity and feeds it into the power grid managed by china southern power grid. because of the skyrocketing energy needs in and around the city, the power plant was going to implement the strategy of extension, and increase production progressively. the addition of new generators was the strategic component in the extension, while a new office building and other facilities were also needed. the power plant established a committee to take charge of the whole extension project. to fulfill the office building and its related facility requirements, after the committee finished public bidding for construction units and construction supervision unit, it consulted the city construction archives to find that it had full responsibility in the construction archives projects and its final acceptance. as a result, the committee started to ascertain the function scope in the construction archives project and found that the content related to the new office building and facilities was not related to core process or key resource. the ecm functionality in the construction archives project is illustrated in our ecm model with grey background in figure 8. these functions are paper-based and digital-based mixed. according to the regulation of the city construction archives, the power plant needed to manage the whole archives project, including monitoring the construction recordings in the standard for guangdong province construction project completion acceptance technical documentation. the committee needed to ensure that the recording was consistent with the construction by aggregating recordings from construction units and supervision unite, arranging and storing all physical records with classification, and transferring back-up copy to the city construction archives in the final stage. furthermore, the construction archives project is coincident with the construction project. australasian journal of information systems volume 18 number1 2013 20 the committee divided the project into two components. component one is the power plant extension construction archives project, component two is the content storage for the construction archives. the decision-making process for component one is illustrated in our decision-making model with grey backgrounds in figure 9. although the archives management and records management process existed within the organization, there were gaps between the existed capability and the capability needed in the project. since the construction archives project is coincident with the construction project, the process and schedule can be influenced by many factors like capital chain, waiting for architectural planning approval, etc. herein, human resource use is not intensive but lasts a long period. the cost of internal management was not worthy in the judgment of the committee. afterwards, they started to think about outsource cost. in this particular example, the construction archives project is highly specific to the extension transaction, but the uncertainty of it is relatively low due to regulations by city construction archives. even if the transaction continues for a long period, it is occasional and unlikely to be repeated in at least five to eight years, so the transaction cost evaluation was agreed to by the committee. furthermore, a local outsourcer with good relationships with some state-owned enterprises and some departments of city government was available. in addition, the content revealed through these construction transactions was not confidential, because the building artificial intelligence system and security system development would come after the construction complement and acceptance. except for some important licenses and articles of the construction project received from the government, the archives project had high tolerance for information insecurity. the committee negotiated with the outsourcer about the flexibility. as a result, the outsourcer was to send one full-time staff to the construction site during the whole period, while professional data processors could be called on when needed during the construction stage. furthermore, professional consultants could also be reached by the committee. owing to the highasset specificity of the archives project, the outsourcer and the committee agreed on an insurance contract, in which the cooperative relationship lasted for the whole basic construction period, and the payment schedule was also laid out and agreed to by both parties. besides, the committee recognized the importance of the component tow content storage, and its high-asset specificity to the localization and storage of the whole power plant extension project. component two involves the expansion of the storage capability in the archive office to make room for the construction archives and records related to new generators’ purchase, test, operation and maintenance. the decision-making process for component two is presented in figure 10. the committee of the power plant admitted that the decision-making model could guide the sourcing decision-making process for organizations. the decisions let the committee focus on core components and key resources including the purchase, logistic, assemblage and test of the new generators. the cost saving was presented in such a way to reduce long-term human resource cost. the power plant was able to get in professional advisors when they needed. the on-site staff from the outsourcer liaised between the power plant and the outsourcer, monitored on behalf of the owner, and even arranged the records for the enterprise. by getting in such professionals, the quality of the construction archive project could be ensured to a certain extent and thus assist the power plant to gain acceptance from the city construction archives. right up until this paper was written, the construction archives project was moving to the final stage, with both the committee and the outsourcer looking confidently to final acceptance within half a year. also it is an important way to integrate professional construction archives capability and professional storage capability. through the outsourcing contract, the power plant develops a widen horizon and better understands the content storage function and other ecm functionality. the internal reference and evidence functions are clear for the future goals of the power plant ecm functionality. australasian journal of information systems volume 18 number1 2013 21 figure 8: the needed ecm functionality in power plant extension construction archives project in general, we found that the ecm functional model is a helpful tool in describing and segregating ecm functionality. after the addition functions, the model can be applied to both digital-based and paper-based content management. with the analysis support of the ecm functional model, the decision-making model that divides into both functionality related to core competencies and functionality not so related, provides general guidelines for the decision-making process and ensures that issues that may arise are considered in a logical manner. australasian journal of information systems volume 18 number1 2013 22 figure 9: the ecm sourcing decision for the power plant extension construction archives project figure 10: the content storage sourcing decision for the power plant extension australasian journal of information systems volume 18 number1 2013 23 discussion and conclusion discussion during the case studies, we found the ecm functional framework can cover most of the ecm functions within organization. it is a useful to both segment and combination with ecm. the framework has been shown to be comprehensiveness in determining ecm outsourcing analysis and evaluation units. all the links in the decision-making model are linear. during the case studies, we recognized that circumstance may arise where the decision maker begins at the starting point but stays in specific consideration, and may finally reach no sourcing result. therefore, we add the change process or spec component in the decision-making model by creating loops in figures both related to core competencies and not related to core competencies. the component of modification of services, process, etc is also put into the model not related to core competencies. these loops enable decision makers to modify or separate the analysis unit being evaluated and start the process again. after the case studies, the decision-making model for bpo of ecm has also shown appropriate guidance in the decision-making process. the core competency issue and the related considerations for information confidentiality, security, and management control and flexibility have shown high appropriateness in the decision-making process. other considerations like process/functionality inhouse or not, costs comparison, and supplier availability are also helpful steps in the process. also, the sequential order has been exemplified in our two case studies in light of the decision makers. however, the lack of supplier solution evaluation, difficulty in cost estimation before the outsourcing transactions, and the self interest of the outsourcers cause limitations to the model. we observed from the feedback in these two cases that the model showed appropriateness to the ecm bpo decision process, but did not produce a successful or satisfying ecm bpo decision. while the availability of eligible suppliers would not be a problem in the evolving intermediate market of ecm, how to assess solutions the suppliers provided is a critical step in a successful ecm bpo. the solutions provided from the suppliers need to be evaluated against ecm objects of the enterprise, in order to make an outsourcer decision. because a lack of supplier-solution evaluation in our model is a major defect, it needs future study. the enterprise may defer the bpo of ecm decision until suitable and acceptable solutions come under their consideration. therefore, supplier-solution evaluation determines outsourcer choice, and when and how to conduct bpo of ecm. costs comparison exists in the model twice, but it is difficult to estimate or calculate cost in advance. the real estate corporation had not targeted all their ecm requirements before the contract drafting. a supplementary contract was made with the outsourcer for more specific needs after the test run of the ecm functions in the sales system. as a result, a set of unanticipated costs existed in this ecm bpo transaction. therefore, requirement targeting should be undertaken before the cost comparison, to make the cost estimation more accurate. moreover, the self interest of the outsourcer has been a continued problem in outsourcing transactions. although the enterprise intended to obtain new knowledge or new applications in ecm from the introduction of external resources, it is not clear about whether or how the new solution or technology fits the enterprise model. the outsourcer may insist on their original solutions and not investigate attentively on each specific enterprise model. this response will affect the compatibility of the ecm bpo outcomes. therefore, further research on bpo is needed for those ecm processes/functionalities not existing in-house. australasian journal of information systems volume 18 number1 2013 24 conclusion the purpose of this study is to ascertain how to make a sourcing decision in ecm based on the complex and dynamic environment that an enterprise faces. this study sought to address the concerns: whether it is correct to conduct a specific ecm bpo; whether or not an ecm transaction should be outsourced; what allowances should be made for different ecm outsourcing decision strategies and different sourcing types. as a result, we focused on the development of a decisionmaking model for bpo in ecm. the first part on the ecm functional framework showed what functionality components or functionality combinations can be outsourced, and presented an important analysis tool for ecm. the second part on the decision-making model provided guidance for decision-making processes, and determined the factors to be considered in ecm bpo. this paper’s theoretical contributions are two-fold: first, the ecm function model is extended by including paper-based content based on the theory from records management and archives management. it extends the ecm research and leads to the integration of knowledge from related subjects from both electronic and non-electronic perspective; and second, our decision-making model for bpo of ecm adds new knowledge into the ecm field. the decision-making model fills the gap of outsourcing decision strategies in content management. for industry, the decision-making model provides guidance for practitioners on how to make decisions for bpo of ecm, and suggests factors and procedures. the sequential logic in our model is tested to be appropriate in ecm bpo decision making. it is an important guiding tool for those who make or are responsible for sourcing decisions in ecm, such as the cios. one of the main limitations in this paper is the generalization of the decision-making model, concerning representativeness and transferability of cases (goeken & börner, 2012). the two cases studied are both from southern china and relate to the construction industry. it means the ecm bpo decision still needs to be modified or studied in countries other than china, and in industries other than construction, such as production and pharmacy. furthermore, each of the two cases faced different circumstances, but two different decision-making paths did not cover all steps in the model. as well, the combined functional scope of each case did not cover all functionalities in the ecm model. all in all, future research is required to focus on extending the generalizability of the decisionmaking model. another main limitation was the lack of supplier solution evaluation steps and the lack of solution assessment guidelines or criteria. supplier solution evaluation and assessment can determine whether a bpo of ecm is successful and satisfying or not, in regards to cost estimation, outsourcer choice, and when and how to conduct an ecm bpo transaction. this domain will direct future is ecm research, focusing on targeting the enterprise’s ecm requirements, how to meet the ecm requirements and fulfill the objects of ecm, and what are the fundamental principles or basic elements in ecm solutions. references alvarez-suescun, e. (2010) “combining transaction cost and resource-based insights to explain it implementation outsourcing”, information systems frontiers, vol 12, pp 631–645. aron, r. & j.v. singh (2005) “getting off shoring right”, harvard business review, vol 83, pp 135– 43. aubert, b.a., rivard, suzanne. & party, michel. (1996) “a transaction cost approach to outsourcing behavior: some empirical evidence”, information & management, vol 30, pp 51-64. aubert, b. a. & weber, r. (2001) “transaction cost theory, the resource-based view, and information technology sourcing decisions: a re-examination of lacity etal.’s find”. http://www2.hec.ca/gresi/documents/cahier0108.pdf accessed 2 july 2012. http://www2.hec.ca/gresi/documents/cahier0108.pdf australasian journal of information systems volume 18 number1 2013 25 aubert, b. a., rivard, s. & patry m. (2004) “a transaction cost model of it outsourcing”, information & management, vol 41, pp 921–932. barney, j. (1991) “firm resources and sustained competitive advantage”, journal of management, vol 17, pp 99-120. barney, j., wright, m., ketchen, d. j. & jr. (2001) “the resource-based view of the firm: ten years after 1991”, journal of management, vol 27, pp 625-641. benbasat, i., goldstein, d.k, & mead, m. (1987) “the case research strategy in studies of information systems”, mis quarterly, vol 11 no 3, pp 369-386. bonoma, t.v. & wong, k.b. (1985) “a case study in case research: marketing implementation”. distributed by hbs case service, harvard business school. fine, c.h. & whitney, d.e. (1996) “is the make-buy decision a core competence?”. http://dspace.mit.edu/bitstream/handle/1721.1/1626/make_buy.pdf?sequence=2 accessed on 6 february 2013. gilley, k.m. & rasheed, a. (2000) “making more by doing less: an analysis of outsourcing and its effects on firm performance”, journal of management, vol 26 no 4, pp 763–790. grahlmann, k.r., helms, r.w., hilhorst, c., brinkkemper, s. & amerongen, s.v. (2012) “reviewing enterprise content management: a functional framework”, european journal of information systems, vol 21, pp 268–286. grant, r.m. (1996) “toward a knowledge-based theory of the firm”, strategic management journal, vol 17 (winter special issue), pp 109–122. goeken m. & börner r. (2012) “generalization in qualitative is research approaches and their application to a case study on soa”, australiasian journal of information systems, vol 17, no.2, pp79-108. harrison, j.s., hitt, m.a., hoskisson, r.e. & ireland, r.d. (2001) “resource complementarity in business combinations: extending the logic to organizational alliances”, journal of management, vol 27 no 6, pp 679–690. holcomb, t.r., holmes, r.m. & hitt, m.a. (2006) “diversification to achieve scale and scope: the strategic implications of resource management for value creation”. in advances in strategic management: ecology and strategy, jai-elsevier science, uk. holcomb, t.r. and hitt, m.a. (2007) “toward a model of strategic outsourcing”, journal of operations management, vol 25, pp 464–481. kung, chaang-yung, & cheng, chao-ran. (2004) “grey assessing the performance of enterprise outsourcing management”, the journal of grey system, vol 1, pp 63-72. lacity, m.c. & willcocks, l.p. (1995) “interpreting information technology sourcing decisions from a transaction cost perspective: findings and critique”, accounting, management & information technology, vol 5 no 3-4, pp 203-244. mcivor, r., humphreys, p. and mckittrick, a. (2010) “integrating the critical success factor method into the business process outsourcing decision”, technology analysis & strategic management, vol 22 no 3, pp 339–360. morrow jr, j.l., sirmon, d.g., hitt, m.a. & holcomb, t.r. (2005) “creating value in the face of declining performance: firm strategies and organizational recovery”, strategic management journal, vol 28 no 3, pp 271-283. mowery, d.c., oxley, j.e. & silverman, b.s. (1996) “strategic alliances and interfirm knowledge transfer”. strategic management journal, vol 17, pp 77–91. http://dspace.mit.edu/bitstream/handle/1721.1/1626/make_buy.pdf?sequence=2 australasian journal of information systems volume 18 number1 2013 26 nordheim, s. & päivärinta, t. (2006) “implementing enterprise content management: from evolution through strategy to contradictions out-of-the-box”, european journal of information systems, vol 15, pp 648–662. o’callaghan, r. & smits, m. (2005) “a strategy development process for enterprise content management”. in proceedings of the 13th european conference on information systems, may, pp 26–28. päivärinta, t. and munkvold, b.e. (2005) “enterprise content management: an integrated perspective on information managemet”. in the proceddings fo the 38th hawaii international conference on system sciences. parkhe, a. (1993) “strategic alliance structuring: a game theoretic and transaction cost examination of interfirm cooperation”. academy of management journal, vol 36 no 4, pp 794–829. pinkett, h.t. (1981) “american archival theory: the state of the art”, the american archivist, vol 44 no 3, pp 217-222. quinn, j.b. & hilmer, f.g. (1994) “strategic outsourcing”, sloan management review, vol 35 no 4, pp 43–55. real story group (2013) “content technology vendor map”. http://www.realstorygroup.com/research/vendors accessed on 19 feburary 2013. roethlisberger, f.j. & lombard, g.f.f. (1977) “the elusive phenomena: an autobiographical account of my work in the field of organizational behavior at the harvard business school.” harvard business school. schellenberg, t.r. (1965) “the appraisal of modern public recordr”, bulletin the national archives, vol 8, pp 8. shy, o. & stenbacka, r. (2003) “strategic outsourcing”, journal of economic behavior & organization, vol 50, pp 203–224. smith, h.a. & mckeen, j.d. (2003) “developments in practice viii: enterprise content management.” communications of association for information systems, vol 11, pp 1–26. spender, j. c. (1996) “making knowledge the basis of a dynamic theory of the firm”, strategic management journal, vol 17, pp 45-62. stapleton, richard. (1983) “jenkinson and schellenberg: a comparison”, archivaria, vol 17, pp 7585. tayles, m. & drury, colin. (2001) “moving from make/buy to strategic sourcing: the outsource decision process”, long range planning, vol 34, pp 605–622. tyrväinen, p., päivärinta, t., salminen, a. & iivari, j. (2006) “characterizing the evolving research on enterprise content management”, european journal of information systems, vol 15, pp 627–634. usman, m., muzaffar, a.w & rauf, a. (2009) “enterprise content management (ecm): needs, challenges and recommendations”, computer science and information technology, iccsit, 2nd ieee international conference, pp 283-289. wang. li, gwebu, k.l, wang, j, & zhu, david.x. (2008) “the aftermath of information technology outsourcing: an empirical study of firm performance following outsourcing decisions”, journal of information systems, vol 22 no 1, pp 125–159. williamson, o.e. (1985) “the economic institutions of capitalism”. free press, new york. yang, dong-hoon,, kim, s., nam, c. & min, ja-won (2007) “developing a decision model for business process outsourcing”, computers & operations research, vol 34, pp 3769 – 3778. zhao yu zhang (2008). english reading on archives studies, qunyan press, beijing. http://www.realstorygroup.com/research/vendors microsoft word ajis vol 15-1 m.doc australasian journal of information systems volume 15 number 1 december 2007 3 ajis editorial ajis publishes high quality contributions to theory and practice in the global information systems discipline. while research is global, ajis contributions should have an emphasis on the australasian context of the work. the journal welcomes papers on research and conceptual development based in a very wide range of inquiry methods, ways of thinking and modes of expression. over half of the submissions to ajis in the last 6 months were declined as they do not explicitly contribute to the development of the discipline. ajis is an academic journal and submissions need to make scholarly and/or research contribution. the australasian heads and professors of information systems (acphis) holds an annual workshop to review is education and research, and to discuss the state of the is discipline (see http://www.acphis.org.au/). at its adelaide meeting in august 2007 the scope of ajis was discussed. in particular, whether there should be: 1. an 'ajis intern' section of the journal for papers from phd and masters students. these papers would meet the normal standards for publication but would make a more modest contribution to theory. the main advantage would be to help research candidates to start publishing. there are practical issues however around whether the section would be seen as second rate when in fact the contribution made may well be as significant as papers in the rest of the journal. 2. an is education section. 3. an is industry section. the general response from acphis members was that it may be better not to complicate the journal and that ajis featured themes could absorb special cases. if you have any thoughts on these matters please contact the editor. discussions about ajis continued at 18th australasian conference on information systems at the university of southern queensland (see http://www.acis2007.usq.edu.au/) among members of the australian association for information systems (aais see http://www.aaisnet.org/). aais is the owner of ajis. resulting from those discussions a number of associate editors have been appointed whose task it is to see a paper through the reviewing process. this additional resource should speed up the process and will allow for more personal advice on the development of papers. thanks to julie fisher, jon heales, john gammack and david wilson who are working on submissions. all back issues of ajis were loaded into the online system progressively over the first 6 months of 2007. by the end of the year there had been over 140,000 downloads of ajis papers (just google ajis). yes, 140,000. this exceeded the editorial board's expectations by orders of magnitude. my thanks to julie fisher for her work in editing the featured theme and to the paper authors. thanks also to john campbell and jon heales for editorial help, and to the faculty of information sciences and engineering, university of canberra, for its significant ongoing support and, in particular, marina materne. craig mcdonald craig.mcdonald@canberra.edu.au december 2007 paper title (paper title style) australasian journal of information systems chutikulrungsee 2016, vol 20, post publication review security & privacy for sme cloud adoption post publication review senarathna i., yeoh w., warren m., salzman s. (2016). security and privacy concerns for australian smes cloud adoption: empirical study of metropolitan vs regional smes. australasian journal of information systems, 20. doi: http://dx.doi.org/10.3127/ajis.v20i0.1193 review the above paper by senarathna, yeoh, warren, and salzman (2016), is interesting because it demonstrates that security and privacy are not major factors for smes, even though the literature states that they should be. much research suggests that cloud computing would significantly contribute to smes; however, as senarathna et al point out, according to the literature, security and privacy are major barriers to adoption. studies have focused on barriers to cloud computing adoption by smes worldwide, yet little is known about cloud computing adoption by smes in australia. this paper identifies and fills this gap in literature by focusing on the correlation of security, privacy, and physical locations with cloud computing adoption by smes, particularly in metropolitan and regional areas. furthermore, this paper is a great exemplar of a clearly structured and well-organised manuscript from start to finish. the authors provide definitions of relevant terms, introduce common elements of cloud computing (nbn as a gateway, advantages, and challenges), and discuss cloud computing adoption by smes before reviewing literature along with proposing hypotheses. structuring the paper this way strengthens and lends support to the proposed hypotheses. given that h1 and consequently much of the paper is focused on cloud privacy, the privacy aspects of the paper are worth exploring in more depth. a great deal of literature has been published in this area, not just in relation to cloud and sme, but more generally in areas of online media, professionalism, health and more (al-saggaf & chutikulrungsee, 2015; bowern, burmeister, gotterbarn, & weckert, 2006; burmeister, 2000, 2015; burmeister, islam, dayhew, & crichton, 2015; carlson, farrelly, frazer, & borthwick, 2015; chutikulrungsee, burmeister, al-saggaf, & bhattacharya, 2016; cockcroft, 2006; scott, richards, & adhikari, 2015; simpson, nevile, & burmeister, 2003; teipel et al., 2016). as the authors point out (p2), the literature shows that “cloud computing poses an inherent challenge to privacy because cloud computing services usually present data in an unencrypted form on a machine owned and operated by a different organisation than the data owner”, that is, the cross-border data transfer and storage by a third party are of concern, which raises jurisdictional issues. this is particularly pertinent given the changes to the commonwealth privacy act which were brought into effect mid-2014 and whereby there are now far greater requirements for transparency in the handling of personal information. the authors also point out the inherent challenges of ict technologies, given the open nature of the internet (p5). yet their research appears to indicate that such well documented concerns overseas, are not evident in australia, which is curious. what remains unclear is why the findings are contrary to those of existing literature on cloud computing adoption. this paper provides explanation about these contradictions that are owing to smes’ lack of awareness in security and privacy on cloud computing or smes’ tight budget (p12). this is an opportunity for future research to extend this study in depth by conducting qualitative study using an in-depth interviews to examine the reasons behind these implications. this approach should consider other aspects such as size of organisations, data types (general, sensitive, personal, personal identifiable information), types of cloud deployment models, as well as environments of adoption, in conjunction with security and privacy. the reasons for these considerations are as following. first, whether smes should adopt cloud computing or not is a question of trade-offs; hence, weighting security and privacy against those factors will likely make a difference to the results. for instance, security and privacy may not be a significant influence on cloud adoption for smes of 15 employees, in contrast with smes of 100 employees. in addition, level of security and privacy may be 1 australasian journal of information systems chutikulrungsee 2016, vol 20, post publication review security & privacy for sme cloud adoption different, depending on cloud computing deployment models (public cloud, private cloud, hybrid cloud, and community cloud). in such contexts security and privacy may not significantly influence sme decision making on adopting private cloud, as opposed to smes considering adopting public cloud. future research can also carry out multiple case studies to explore security and privacy regarding cloud computing adoption in-depth. additionally, a comparison of security and privacy between ‘before’ and ‘after’ adoption would be useful in making decisions about whether smes should adopt cloud computing or not in the future. thus this important contribution to the literature by senarathna et al provides a good basis upon which to extend the exploration of sme cloud security and privacy. tharntip tawnie chutikulrungsee charles sturt university tchutikulrungsee@csu.edu.au author response we thank the author for their post-publication review of our newly published article in ajis. the review mainly considered privacy aspect of cloud computing while just mentioning cloud security. it also highlighted opportunities for future researches to extend our study. we are happy to see these future research avenues as an extensions of our research study. ishan senarathna department of information systems and business analytics deakin business school william yeoh department of information systems and business analytics deakin business school matthew warren department of information systems and business analytics deakin business school scott salzman department of information systems and business analytics deakin business school references al-saggaf, y., & chutikulrungsee, t. (2015). twitter usage in australia and saudi arabia and influence of culture: an exploratory cross-country comparison. paper presented at the rethinking communication, space and identity, queenstown, nz. bowern, m., burmeister, o. k., gotterbarn, d., & weckert, j. (2006). ict integrity: bringing the acs code of ethics up to date. australasian journal of information systems, 13(2), 168-181. burmeister, o. k. (2000). applying the acs code of ethics. journal of research and practice in information technology, 32(2), 107-120. burmeister, o. k. (2015). improving professional it doctorate completion rates. australasian journal of information systems, 19, 55-70. doi: 10.3127/ajis.v19i0.1073 burmeister, o. k., islam, m. z., dayhew, m., & crichton, m. (2015). enhancing client welfare through better communication of private mental health data between rural service providers. australasian journal of information systems, 19, 1-14. doi: http://dx.doi.org/10.3127/ajis.v19i0.1206 carlson, b. l., farrelly, t., frazer, r., & borthwick, f. (2015). mediating tragedy: facebook, aboriginal peoples and suicide. australasian journal of information systems, 19. doi: 10.3127/ajis.v19i0.1174 2 australasian journal of information systems chutikulrungsee 2016, vol 20, post publication review security & privacy for sme cloud adoption chutikulrungsee, t., burmeister, o. k., al-saggaf, y., & bhattacharya, m. (2016, september). denial of choice: group level disclosure of private information. paper presented at the 12th ifip tc9 human choice and computers conference, salford, uk. cockcroft, s. (2006). information privacy: culture, legislation and user attitudes. australasian journal of information systems, 14(1). doi: 10.3127/ajis.v14i1.7 scott, k., richards, d., & adhikari, r. (2015). a review and comparative analysis of security risks and safety measures of mobile health apps. australasian journal of information systems, 19. doi: 10.3127/ajis.v19i0.1210 senarathna, i., yeoh, w., warren, m., & salzman, s. (2016). security and privacy concerns for australian smes cloud adoption: empirical study of metropolitan vs regional smes. australasian journal of information systems, 20. doi: 10.3127/ajis.v20i0.1193 simpson, c., nevile, l., & burmeister, o. k. (2003). doing ethics: a universal technique in an accessibility context. australasian journal of information systems, 10(2). teipel, s., babiloni, c., hoey, j., kaye, j., kirste, t., & burmeister, o. k. (2016). information and communication technology solutions for outdoor navigation in dementia. alzheimer's & dementia: the journal of the alzheimer's association, 1-13. doi: 10.1016/j.jalz.2015.11.003 copyright: © 2016 chutikulrungsee. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. 3 http://creativecommons.org/licenses/by-nc/3.0/au/ a comparison of five alternative approaches to information systems development rudy hirschheim1 juhani livari2 heinz k. klein3 1 university of houston 2 university of oulu 3 suny binghamton introduction the field of information systems (is) has grown dramatically over the past three decades. recent trends have transformed the is landscape. these trends include: the evolution of implementation technology from centralized mainframe environments towards distributed client-server architectures, embracing the internet and intranets; changes in user interface technology from character-based to graphical user interfaces, multimedia, and the world wide web; changes in applications from transaction processing systems towards systems supporting collaborative work; and the use of information technology as an enabler of business process reengineering and redesign. these technology changes coupled with changes in organizations and their operating environment, such as the growth of the network and virtual organization, internationalization and globalization of many organizations, intensified global competition, changes in values such as customer orientation (service quality) and quality of working life, have imposed new demands on the development of information systems. these changes have led to an increasing discussion about information systems development (iso), and in particular, the various methods, tools, methodologies, and approaches for isd. we believe such discussion has opened the door for new, alternative is development approaches and methodologies. our paper takes up this theme by describing five alternative isd approaches, namely the interactionist approach, the speech act-based approach, soft systems methodology, the trade unionist approach, and the professional work practices approach. despite the fact that most of these approaches have a history of over 15 years, their relevance to is development is not well recognized in the mainstream of is practice and research, nor is their institutional status comparable to traditional approaches such as structured analysis and design methods. therefore we characterize the five approaches as 'alternative' in the sense of alternative to the orthodoxy. the selection of the five approaches is essentially based on the finding that research on isd approaches and methodologies has been dominated by a single set of philosophical assumptions regarding the nature of the phenomena studied and what constitutes valid knowledge about those phenomena (hirschheim and klein, 1989; orlikowski and baroudi, 1991; and livari, 1991). the idea behind the selection of the five isd approaches has been to include approaches which challenge the dominant assumptions. these alternative approaches typically build upon radically different conceptions of the goals, meaning, function and processes of isd. part of the rationale for our paper is to meet the need of a concise yet penetrating way of introducing alternative ways of system development to a wider audience. the way in which the approaches are introduced, highlights their underlying principles and features. this naturally leads to a critical examination of their strengths and weaknesses. from this angle the paper adds more detail to the earlier work on mapping the terrain of the complex literature on is development (cf. episkopou and wood-harper, 1986; hirschheim and klein, 1989; livari, 1991; orlikowski and baroudi, 1991; baskerville, etal. 1992; avison et al. 1992; avgerou and cornford, 1993; fitzgerald, 1994; hirschheim, klein and lyytinen 1995; avison and fitzgerald, 1995; jayartna and fitzgerald, 1996; wynekoop and russo, 1997; livari, hirschheim and klein 1997). the paper can be expected to be of interest to the is community in three respects. firstly, the five alternative approaches are likely not to be as widely known as they deserve to be. the following meets the need of a concise introduction to them. secondly, the paper continues our earlier work on mapping the terrain of the complex literature on is development (hirschheim and klein, 1989; livari, 1991; hirschheim and klein, 1992; hirschheim, klein and lyytinen 1995, 1996; livari, hirschheim and klein, 1997). thirdly, it is our contention that the five alternative approaches point the direction which some important is research will likely take in the future to strengthen the interpretive and critical traditions (orlikowski and baroudi, 1991; hirschheim and klein, 1994) within the field. review of the relevant literature the interest in isd approaches in the current discussion about isd it is possible to detect two countervailing trends and this paper can be ajis vol 5 no 1 september 1997 viewed as a contribution to both. the first trend reflects the belief by some that the interest in methodologies has been waning for reasons of their supposed deficient practicality. even though there is little empirical research on the actual use of iso methodologies, existing evidence (e.g. hardy et al. 1995, chatzoglou and macaulay, 1996) suggests that their use is limited in practice, and as far as they are used, they are not literally applied (cf. westrup 1993). chatzoglue and maculay (1996), for example, report that nearly half of the projects (47%) did not use a methodology in their survey of 72 projects within the uk, while another british survey, hardy et al. (1995), suggest an encouraging figure (18%) for the non-use of methodologies. hardy et al. (1995) report that 38% of methodologies used were in-house developed and were customized in 88% of cases. wynekoop and russo (1993), report the findings of a survey of over 100 organizations which indicated that 65% of organizations had developed their own methodology in-house and 89% of the respondents believed that formal methodologies should be adapted on a project-by-project basis. mathiassen (1988) went so far as to question the practicality of methodologies in guiding the work of seasoned practitioners, suggesting that methodologies are primarily intended for beginners as the primary vehicle by which they are initiated into the field (cf. andersen, et al. 1990). more moderately, unhelkar and mamdapur (1995) propose a metaphor of a "road map" for isd methodologies, suggesting that a methodology may not be able to recognize all situational factors (e.g. roadblocks) and is more useful for a "foreigner" (beginner) than for a "seasoned" practitioner (henderson-sellers, 1995). at the same time, there is a second trend of resurgence in methodology interest for several reasons. firstly, the recent interest in object-orientation has created a new wave of isd methodologies. since the late 1980's dozens of object-oriented methodologies has been published leading to an increasingly complex methodology landscape (monarchi and puhr, 1991; hong, et al. 1993; livari, 1994). secondly, computer aided systems/software environments/engineering (case) with their companion methodologies (vessey, et al. 1992) have also brought isd methodologies to the forefront. most current case tools support a specific, standard methodology, such as modern structured analysis or some object-oriented methodology. the tight coupling between methodologies and case tools is suspected to stifle the evolution of is methodologies (bubenko, 1988). the latest case research has focused on more flexible case environments leading to the idea of method engineering as an engineering discipline to design, construct and adapt isd methodologies and methods (kumar and welke, 1992; harmsen, et al. 1994). thirdly, software quality (e.g. iso 9000-3) has been brought to the forefront in software engineering (computer, 1996), the common view being that the quality must be built in the software right from the beginning. this obviously requires some sort of methodology. finally, there has been considerable research and industry interest in software process maturity and improvement since the publication of the capability maturity model (cmm) by the software engineering institute sponsored by the dod (humphrey, 1989). cmm identifies five maturity levels which are based on an increasingly disciplined, standardized and predictable software process (paulk, et al. 1995). untangling the methodology jungle avison and fitzgerald (1988) coined the phrase "methodology jungle" to describe the status of isd methodologies as a whole, as an unorganized collection of numerous methodologies which are more or less similar to each other. bubenko (1986) estimated a decade ago that there existed hundreds of isd methodologies, and more recently jayaratna (1994) estimated the number to be of order of 1000. in such a state, it is clear that the highest priority should not lie in developing new methodologies but in better understanding the existing stock and the collective methodology knowledge embedded in them. and as far as new methodologies are concerned, one should be careful that a new methodology differs in some significant way from existing ones, livari, hirschheim, and klein (1997) suggest that categorizing isd methodologies into classes of similar methodologies, begins to address the problem of the methodology jungle. they define an isd approach as a set of fundamental goals, guiding principles, concepts, and principles for the isd process for is development, which drive interpretations and actions in information systems development. the goal specifies the general purpose of the approach. guiding principles form the common "philosophy" (cf. avison and fitzgerald, 1995) of the approach which ensures that its methodology instances form coherent wholes (cf. hirschheim, klein and lyytinen, 1995). fundamental concepts define the nature of an is implicit in the approach as well as the focus and unit of analysis in isd. principles of the isd process express essential aspects of the isd process in the approach. the methodology instances of an isd approach are stipulated to share these fundamentals. an isd methodology, on the other hand, can be interpreted as "an organized collection of concepts, methods1, beliefs, values and normative principles supported by material resources" (hirschheim, klein and lyytinen, 1995). more because the word "method" has two other meanings the meaning of methodology which is increasingly adopted in europe, and the meaning of operation or service in the object-oriented vocabulary the term "technique" is preferred in the following. ajis vol 5 no 1 september 1997 specifically, an iso methodology is codified into a set of goal-oriented procedures which guide the work and cooperation of the various parties (stakeholders) involved in the building of an information systems application. these procedures are usually supported by a set of preferred techniques and tools and activities (cf. hirschheim, klein and lyytinen, 1996). a technique, in this context, consists of a well-defined sequence of elementary operations which more or less guarantee achievement of certain outcomes if executed correctly. emerging from the concept of isd approach, is the need for methodology customization (or more formally "method engineering", cf. kumar and welke, 1992). one should note, however, that customization is neither easy nor necessarily being successfully performed (hardy, et al. 1995). livari, hirschheim and klein (1997) suggest the concept of an isd approach as a class template of its specific isd methodologies would be a useful intellectual tool in the customization process, allowing customization as an instantiation of a specific approach. the concept of approach leads to focusing on the fundamental goals, guiding principles, concepts, and principles for the isd process in the customization process. to conclude, this paper takes the position that isd methodologies are still important. at the same time one should recognize additional roles of isd methodologies which are complementary to the traditional idea that methodologies are rules systems to be applied literally. these roles include to serve as convenient knowledge representation forms, ideals of a "rational" design process (parnas and clements, 1986), vehicles of learning (checkland, 1981), and to provide concepts and metaphors for expert communications (madsen, 1989). hence methodologies deserve continued attention even though the focus of this attention may be shifting to new aspects. however, the paper goes beyond specific methodologies to consider isd approaches, and in particular, the paper goes beyond the prevailing "functionalist" approaches when introducing and discussing five alternative isd approaches. selection of approaches there has been considerable interest in the underlying assumptions of is research covering both their broad philosophical (paradigmatic) assumptions (e.g. klein and welke,1982; mumford, et al. 1985; banville and landry, 1989; hirschheim and klein, 1989; livari, 1991; orlikowski and baroudi, 1991; nissen, et al. 1991; hirschheim and klein, 1992; hirschhein, klein and lyytinen 1995, 1996; livari and hirschheim, 1996) as well as more specific aspects of is development, such as the values held by is designers, and the views of is users (e.g. hedberg and mumford, 1975; bostrom and heinen, 1977; dagwell and weber, 1983; kumar and welke, 1984; kumar and bjorn-andersen, 1990). hirschheim and klein (1989) applied the paradigmatic framework of burrell and morgan (1979) to categorize is development approaches. their literature analysis suggested that there exists an identifiable "orthodoxy" which ignores those based on fundamentally different assumptions. orlikowski and baroudi's (1991) analysis of 155 empirical research articles published from 1983 to 1988 provides confirmatory evidence. they report that existing is research exhibits a single set of philosophical assumptions regarding the nature of the phenomena studied and what constitutes valid knowledge about those phenomena. they also introduced two additional research philosophies for consideration the interpretive and the critical illustrating each through an example. livari's (1991) paper based on the analysis of fourteen books representing seven contemporary traditions or "schools of thought" of is development software engineering, database management, management information systems, decision support systems, implementation research, the sociotechnical approach and the scandinavian infological approach also confirms the existence of an "orthodoxy".2 if we accept the notion of a prevailing "orthodoxy" to isd, it is natural to seek out its counterparts and this has provided the motivation and criterion for selecting "alternative" approaches for analysis. it is our preconception that the following five approaches represent philosophical assumptions which make them significantly different from the dominant textbook "orthodoxy": 1. the interactionist approach, 2. the speech act (sa)-based approach, livari (1991) found that all the seven traditions have quite similar assumptions. in his words, they were dominated by the view of information/data as descriptive facts and an information system as a technical artifact with social implications with the exception of the sociotechnical and infological approaches which perceive an information system to be a social system as well. technology is seen as a matter of human choice, and organizations as stable structures. the preferred epistemology is positivist with a means-end-oriented view of is science. the values of is research mostly reflect organizational and economic goals, but most of the seven traditions also pay some attention to user-oriented criteria. in the case of human nature and research methods the analysis did not identify clear dominant views. these assumptions are largely consistent with each other and could be characterized as a functionalist "orthodoxy" in the sense of hirschheim and klein (1989) or orlikowski and baroudi (1991). ajis vol 5 no 1 september 1997 3. the soft systems methodology (ssm), 4. the scandinavian trade unionist approach, and 5. the professional work practice (pwp) approach. the selection of the five approach as a contrast to the prevailing "orthodoxy" is argued in more detail in livari, hirschheim and klein (1997). as noted in the introduction, these five approaches are called 'alternative' because their relevance to is development is not widely recognized as the seven contemporary traditions or "schools of thought" of iso analyzed in livari (1991), nor do they have an institutional status comparable to them. synopses of five alternative isd approaches the survey of the five approaches is based on the published literature, much of which we might add was not readily (easily) available. before proceeding to a detailed review of the five approaches, a brief description and listing of the key literature references of each of the approaches is provided. the interactionist approach refers to the body of research conducted at the university of california at irvine by kling as his colleagues (e.g. kling and scacchi, 1980, 1982; kling and lacono, 1984; kling, 1987; lacono and kling, 1988; and kling and lacono, 1989). the speech act-based (sa) approach to is development draws its basic concepts from the linguistic philosophy of searle (searle, 1969, 1979; searle and vanderveken, 1985). it has stimulated considerable debate in the is community over a number of years (e.g. flores and ludlow, 1980; goldkuhl and lyytinen, 1982, 1984; lehtinen and lyytinen, 1986; winograd and flores, 1986; auramaki, el al. 1988; flores, et al. 1988; dewitz and lee, 1989; dietz and widdershoven, 1991; dobson, et al. 1991; donaldson dewitz, 1991; kensing and winograd, 1991; auramaki, et al. 1992a, 1992b). soft systems methodology (ssm) is a general systems approach developed by checkland and his colleagues at the university of lancaster (checkland, 1981; wilson, 1984; checkland and scholes, 1990). although ssm was originally a general systems approach, without any specific orientation towards information systems, its advocates are increasingly perceiving it to be well suited to is development (e.g. checkland and scholes, 1990). within the is community ssm has been incorporated as an integral part in such approaches as multtview (avison and wood-harper, 1990) and faor (schafer, et al. 1988). the trade unionist approach is a tradition which has evolved mainly in scandinavia as a trade union response to the challenge of scandinavian co-determination arrangements and laws enacted in the mid-1970's which ensured the employees and unions the right to participate in the design of and decision-making concerning computer systems (cf. ehn and kyng, 1987). this approach was initially based on a strong 'class polities' perspective to organizations (kling and scacchi, 1982). this marxist ideology has, however, been weakening recently and the approach is in a transition towards "cooperative design" (greenbaum and kyng, 1991a) that has raised considerable interest lately (kuhn and muller, 1993; bansler and kraft, 1994). the professional work practices (pwp) approach has been developed during the 1980's in denmark by mathiassen and his colleagues (lanzara and mathiassen, 1985; gronbaek, 1989; jepsen, et al. 1989; nielsen, 1989; andersen, etal. 1990). the interactionist approach the principal strength of the interactionist approach is that it can shed light on such organizational issues of isd as user resistance, work around, and/or the failure to realize the full technical potential of is. this approach refers to the body of research conducted at the university of california at irvine by kling and his colleagues (kling and scacchi, 1980, 1982; kling and lacono, 1984; kling, 1987; lacono and kling, 1988; and kling and lacono, 1989). the adjective "interactionist" is selected to characterize the most distinctive aspect of the approach — its view of organizations. the adjective has not directly been used lately by the representatives of this tradition to characterize their approach, but it is used in kling's (1980) "the social analyses of computing", and to denote one of the six perspectives for analyzing computing in kling and scacchi (1980). in this work kling and scacchi distinguish six perspectives, i.e. the rational, structural, human relations, interactionist, organizational politics and class politics perspective. they then assert that "the interactionist and organizational politics perspectives go farther in providing appropriate languages and conceptions of social dynamics to help explain attractions and dilemmas of computer use in organizations than do their competitors" (p. 259 in kling and scacchi (1980)). in a later paper, kling and scacchi (1982) further suggest that the formalrational, structural, interactionist and (organization) political perspectives have dominated the literature. we grouped the formal-rational and structural perspectives together and called them simply "structural". similarly, we grouped the interactionist and political perspectives together and called them simply "interactionist". one can identify two phases in the conceptualization of isd within the interactionist approach: the first phase is based on the distinction between the discrete-entity model and the web-model; the second phase on the distinction between the tool and institutional viewpoint of information systems. ajis vol 5 no 1 september 1997 the distinction between the discrete-entity model and the web-model has been applied as an analytical device to analyze different aspects of is as are indicated in table 1. these aspects were suggested in kling and scacchi (1982) and are important for interpreting the earlier research done within the interactionist approach. "the basic unit of analysis of the discrete-entity model is a computing resource" (kling and scacchi, 1982, p. 9), the use of which is isolated "from the actual work practices and organization of labor within which automated data systems are typically developed and used" (p. 3). in distinction to this, "web models make explicit the salient connections between a focal technology and its social and political contexts" (p. 3). the assumptions of these two models are summarized in table 1 and is borrowed from kling (1987). kling and scacchi (1982) claim that the discrete-entity model is fairly flexible, because it employs not only ideas from the formal-rational perspective, but also from the structural and political. in contrast, the web model cannot be framed in the formal-rational perspective, but can apply any of the other three theoretical perspectives (p. 24), i.e. the structural, political and interactionist. the distinction between the tool and institutional views has been used in a similar way as the discrete entity and web model to analyze social and organizational aspects of is (cf. table 2). it was proposed by kling and lacono in later papers (lacono and kling, 1988; kling and lacono, 1989). unfortunately, the authors do not explain the relationship between this later distinction and the earlier one of discrete-entity model vs. web-model, but we conclude that the distinction between the tool view and the institutional view is an evolutionary step towards a more structural view of organizations. lacono and kling (1988) contend that computer-based information systems (cbis), developed from complex, interdependent social and technical choices are better conceptualized as institutions than as tools. they suggest (p. 104) that when "analysts emphasize the information-processing capabilities of a computer-based technology, they are foregrounding its 'toolness' or instrumental value of is for particular social units". on the other hand, when "analysts emphasize the social and political choices that organizational actors have made over time, they are foregrounding its institutional character". kling and lacono's analysis of the social, political and technical determinants of the two views is summarized in table 2. their viewpoint suggests that "the image of a cbis as a tool is associated with tremendous personal freedom", "there is an underlying assumption that computer-based technologies have no inherent politics: they are consistent with any social order" and "attention is focused on a future of technological perfection". in contrast, "institutional analyses emphasize the social use of cbis and social control over computing arrangements", "politics play an important role" and the focus lies on "the developmental trajectories of cbis. institutions develop a character based on the interest they have served in the past, their organizing ideologies, and the world views which bind their participants together" (pp. 104-105). kling and lacono propose that characterizing cbis as institutions is important for three reasons (lacono and kling, 1988; kling and lacono, 1989). first, the usability of cbis is more critical than the abstract informationprocessing capabilities of the underlying technology. second, cbis that are well-used and have stable social structures are more difficult to replace than those with less developed social structures and fewer participants. third, cbis vary from one social setting to another according to the ways in which they are organized and embedded in organized social systems. while the interactionist approach's focus on the broader organizational issues is valuable, it unfortunately has not proposed any clear methods or methodologies for is development which are based on the web model or the institutional view of is. ajis vol 5 no 1 september 1997 table 1. the discrete-entity vs. the web model dixcreu'-cnlitv model dl. a computing resource is best conceptualized as a particular piece of equipment, application which provides specifiable information processing capabilities. a. each computing resource has costs and skill requirements which are largely identifiable, b. computer-based technologies are tools, and socially neutral. d2. role of infrastructure a. the infrastructure for supporting the focal computing resource and the organizational procedures by which it is organized and sustained are critical elements. b. each computer-based service is provided through s set of structured computing resources and organized infrastructure. deploying, managing, and setting procedures for these infrastructural resources is separable from the deployment of the focal computer-based technology. infrastructure, either technical or administrative, is a neutral resource. d3. control over infrastructure: a. organizations have ample resources to support all of their computing developments and uses simultaneously. elements of infrastructure are necessary for making equipment or technique available to developers or users, and they can be counted on to be of adequate quality and available when needed. d4. the focal computing resource and any element of infrastructure can be analyzed independently of: a. its interactions with other computing resources; b. the social or organizational arrangements within which computer-based services are developed and provided (infrastructure and macrostructures). d5. social action: a. organizational behaviour is best described by the formal goals, procedures, and administrative arrangements of the acting units. b. the use of a computing resource is best described by its formal purposes and features. weh model wl. a computer system is best conceptualized as an ensemble of equipment, applications, and techniques with identifiable information processing capabilities. a. each computing resource has costs and skill requirements which are only partially identifiable. b. in addition to its functional capabilities as an information processing tool, computer -based technologies are also social objects. w2. role of infrastructure a. the infrastructure for supporting the focal computing resource and the organizational procedures by which it is organized and sustained are critical elements. b. each computer-based service is provided through s set of structured computing resources and organized infrastructure. this organization of essential resources makes computer-based systems into a form of social organization. like any organization or institution, it is not necessarily neutral. w3. control over infrastructure: a. organizations have limited resources. not all necessary infrastructural resources are available (in adequate quality) as needed. b. computer-using organizations rarely have complete administrative or political control over all their requisite infrastructure. infrastructural resources may be spread across several organizational units or nominally independent organizations. w4. the information processing leverage provided by a focal computing resource, and its other costs and benefits are contingent upon: a. its interactions with other computing resources; b. the social or organizational arrangements within which computer-based services are developed and provided (infrastructure and macrostructures). w5. social action: a. political interests, structural constraints, and participants' definition of their situations often influence organizational action. an organizational process model or a negotiated order model of social activities is used to analyze social relations. ajis vol 5 no 1 september 1997 table 2. tool view vs. institutional view social local and simple negotiating context political local control and self interest historical freedom of the present gives hope for the future institution complex and overlapping negotiating context shared control and interest groups commitments of the past constrain the future the speech act-based approach the principal strength of the speech act (sa)-based approach is its focus on understanding the rich meanings which are exchanged in ordinary conversation. this language focus makes it particularly suitable for the determination of requirements of groupware or other forms of computer mediated social communication. the theoretical roots of the sa-based approach to information analysis are in searle's philosophy of language (searle, 1969, 1979; searle and vanderveken, 1985). as is evident from the growing number of references, it has aroused considerable interest in the is community.3 since the vocabulary of speech act theory is quite complex, this synopsis introduces only the main ideas and concepts. more detailed summaries can be found in lehtinen and lyytinen (1986), winograd and flores (1986), auramaki, et al. (1988) and dietz and widdershoven (1991). speech acts are basic units of communication expressing a human intent, such as making a promise or asserting a claim. to understand the meaning of a speech act the social situation in which it occurs (its context) must be taken into account. this typically consisted of the speaker, the hearer, time, and place of the communication. the context also includes the set of possible worlds covering all features of speakers, hearers, times, and other aspects. speech is very complex in that people often use the same phrase to mean different things. for example a question can actually be a request (can you close the door?) or express doubt (did you say it was raining?). similarly, assertions are often questions. to complicate things further, in irony or sarcasm a sentence typically means the opposite of what it says. hence speech is a complex form of human action. in order to sort out what people mean with their words, speech act theory distinguishes utterance acts, prepositional acts, perlocutionary acts and illocutionary acts as aspects or "subacts" of speech acts: a speech act is performed by uttering an expression (an utterance act), and when successfully performed it will have effects on the hearer, i.e. the speaker performs a perlocutionary act (e.g. if the question was meant as a request, and this was actually understood, then the hearer will actually close the door otherwise she might say:"no i cannot". if the question was understood as an order the denial has to take the form, "no, i don't want to" or "no, i will not"). performing a speech act also includes a subsidiary act of expressing the prepositional content of the speech act (i.e. the prepositional act; in the example this is expressing that the door is currently open and implying that it is possible to close it, i.e. it is not a locked automatic door as in a bus). the core aspect of a speech act is its illocutionary act, which has a prepositional content and an illocutionary force. the prepositional content refers to propositions expressed in an illocutionary act. the illocutionary force is the principal component of a speech act, including among other things the illocutionary point4. searle claims that speech acts can be classified by their illocutionary point into the following five categories: assertives telling how the world is or will be (statements of fact or predictions), commissives committing the speaker to doing something (e.g. to promise or to agree), directives trying to get the hearer to do things (e.g. to order or to request), declaratives changing the world by saying so, and expressives expressing the speaker' feelings and attitudes. directives and declaratives can be regrouped into "imperativa" because they both imply that there is a source of power which can command that the world needs to be changed to fit the state described in the imperative statement.5 the principal references are: flores and ludlow (1980), goldkuhl and lyytinen (1982, 1984), lehtinen and lyytinen (1986), winograd and flores (1986), auramaki et al. (1988), flores, et al. (1988), dewitz and lee (1989), dietz and widdershoven (1991), dobson, et al (1991), donaldson dewitz (1991), kensing and winograd (1991), auramaki, et al (1992a, 1992b), dietz (1992), janson and woo (1992), truex (1993), klein and truex (1995). 4 more exactly, an illocutionary force has seven components: illocutionary point, mode of achievement, degree of strength, prepositional content conditions, preparatory conditions, sincerity conditions, and degree of strength of sincerity conditions. dietz and widdershoven (1991) point out that habermas has criticised this taxonomy in his recent theory of communicative action (habermas, j., theorie des konununikativen handelns, erster band, suhrkamp verlag, frankfurt am main, 1981 and habermas, j., bemerkungen zu j. searle's 'meaning, communication and representation', s ajis vol 5 no 1 september 1997 10 in the context of classical is development, assertives take a prominent place in that the contents of files and data bases are interpreted as "statements of fact" (see hirschheim, klein, lyytinen 1995 for a detailed review of the literature). in distinction to this, some authors of the speech act-based approach (e.g. winograd and flores, 1986) paid special attention to directives and commissives as being of particular importance for management from the five categories of illocutionary points. while the direction of fit in the case of assertives is from wordto-world aimed at getting the word to match the independently existing state of affairs of the world, the direction in the case of directives and commissives is world-to-word, i.e. the illuctionary point is a commitment to alter world to match the word (winograd and flores 1986 see the obtaining of such commitments as the essence of management). in the former case adjectives such as "true" and "false" characterize how well the matching has taken place while in the latter case adjectives such as "fulfilled" and "unfulfilled" can be used to characterize how well the matching has taken place. speech acts form larger wholes, e.g. networks of recurrent conversations (winograd and flores, 1986) or conversations and discourses (auramaki, el al. 1988; klein and truex, 1995).6 a simple example is the case of a question that must typically be followed by an answer or a request for clarification, a refusal to answer or a counter-question. this example is simplified, because the question might have been a rhetorical figure for expressing doubt in which case the subsequent moves are more involved. hence in order to identify the meanings which are exchanged in larger wholes or speech act patterns, it is necessary to analyze how sequences of speech acts interlink like the "moves" in a chess game. the analysis of such larger patterns is currently being undertaken (klein and truex, 1995). even though the pioneering works on the sa-based approach in north america and europe have had a common theoretical root, they have differed somewhat in their application. winograd and flores et al. (1986, 1987/1988) and flores, et al. (1988), for example, have focused on discourses as ordered sequences of speech acts, especially those related to directives and commissives, whereas lyytinen et al. (e.g. lehtinen and lyytinen, 1986) have covered a wider variety of illocutionary points and have also focused on the prepositional content of speech acts. since the early 1990's, speech act theory has aroused considerable interest in the is community, leading to a number of methodologies or methodology fragments as is evident from the growing number of references. this brief synopsis indicates that sa theory consists of a rich set of concepts for the analysis of intricate social communications, but its weakness is that it easily leads to a very complex approach. efforts to adopt some simplified subset of sa theory has led to a number of is development approaches that makes the current state sa-based approach quite fragmented (cf. auramaki, et al. 1988). soft systems methodology the strength and distinguishing feature of soft systems methodology (ssm) is its explicit focus on problem formulation by helping to identify the "relevant" systems from the perceptions of possibly disagreeing stakeholders. although ssm is a general systems approach without any specific orientation towards information systems, its developers have increasingly advocated its application to is development for which it is well suited (e.g. checkland and scholes, 1990, p. 53). it also has aroused considerable interest within the larger is community especially in europe and australasia. for example, soft systems thinking is an integral part of such methodologies as multiview (wood-harper et al. 1985; avison and wood-harper, 1990) and faor (schafer, et al. 1988). it also has an extensive case history documentation. in order to keep our discussion as close as possible to what checkland intended, the following synopsis of ssm is primarily based on the original texts of ssm, in particular checkland and scholes (1990).7 nachmetaphysisches denken, suhrkamp verlag, frankfurt am main, 1988) suggesting a taxonomy of four categories based on four kinds of dominant claims: claims to truth, claims to power, claims to justice and claims to sincerity. the four speech categories are imperativa implying a claim to power as do searle's directives (including questions which habermas interprets as directives to provide an answer and which can be refused like any other directive); constativa making claims to truth and correspond to assertives in searle's classification; regulativa including those directives, commissives and declaratives in searle's classification which are based on claims to justice; and expressiva covering those commissives and expressives which are based on claims to sincerity. 6 the is literature often uses the terms 'conversation' and 'discourse' interchangeably despite their fundamental differences (see ljungberg and holm, 1996). in view of the need to model larger units of communication as well-formed sequences of speech acts, the term 'discourse' would seem to be more appropriate. 7 ssm has been evolving since the early 1970's and was developed by checkland and his colleagues at the university of lancaster. its principal versions are published in three books (checkland, 1981; wilson, 1984; checkland and scholes, 1990). ajis vol 5 no 1 september 1997 11 the overall purpose of ssm is well summarized by von bulow (1989): "ssm is a methodology that aims to bring about the improvement in areas of social concern by activitating in the people involved in the situation a learning cycle which is ideally never-ending. the learning takes place through the iterative process of using systems concepts to reflect upon and debate perceptions of the real world, taking action in the real world, and again reflecting on the happenings using systems concepts. the reflection and debate is structured by a number of systemic models. these are conceived as holistic ideal types of certain aspects of the problem situation rather than accounts of it. it is taken as given that no objective and complete account of a problem situation can be perceived." the development of ssm has very heavily relied on action research. in distinction to other views of action research, checkland and scholes underline the crucial importance of intellectual frameworks (in their case ssm) as a precondition for effective learning in action research (checkland and scholes, 1990, p. 16). originally, the inquiry process of ssm consisted of seven stages (checkland, 1981; checkland and scholes, 1990): (1) the identification of the problem situation which is considered to be problematic, (2) the expression of the problem situation, (3) the formulation of root definitions of relevant systems, (4) the formulation of conceptual models of the relevant systems, (5) the comparison of the models and the real world, (6) description of systemically desirable and culturally feasible changes, and (7) action to improve the problem situation. stages 1-2 and 5-7 are 'real-world' activities whereas stages 3 and 4 are 'systems thinking1 activities. checkland and scholes (1990) propose a revised version of ssm which is described in figure 1. the most essential change in the new version is the incorporation of cultural analysis as a parallel stream to logic-based analysis. the logic-based stream starts with selecting relevant systems. their choice may be primary task-based or issue-based. a primary task refers to purposeful action (official task) of some institutional arrangement (an organization or a department) in the real world. an issue-based relevant system does not necessarily have an institutionalized counterpart in the real world it is a nominal system which may be created from scratch. the naming of the relevant systems is based on root definitions which express the core purpose of a purposeful activity system. the core purpose is always expressed as a transformation process in which some entity is transformed into some new form of that same entity in order to achieve some longer term aim. ssm proposes the catwoe mnemonic to help with formulating complete root definitions: c = customers (the victims or beneficiaries of t), a = actors (those who would do the t), t = transformation process (the conversion of input to output), w = weltanschaung (the worldview which makes this t meaningful in context), o = owners (those who could stop t), and e environmental constraints (elements outside the system which it takes as given). modeling relevant systems takes place in terms of human activity systems. a human activity system is a notional system model ("holon") which expresses some purposeful human activity, which could in principle be found in the real world (checkland and scholes, 1990). they are notional in the sense that they are not descriptions of the actual real-world action but are intellectual constructs, ideal types for use in debate about the possible changes which might be introduced into a real-world problem situation (checkland, 1981).8 one of the essential features of ssm is its emphasis on the necessity to create several models of human activity systems. the emphasis on this feature follows from the recognition of multiples stakeholders. as each of these has human ability to interpret the world in different ways, there will never be only one relevant human activity system when examining real-world situations characterized by purposeful action (checkland and scholes, 1990). checkland and scholes also suggest five 'e's' as criteria for judging the successfulness of the transformation t: efficacy, i.e. whether t or the means chosen work in producing the desired output, efficiency, i.e. the amount of output divided by amount of resources used, effectiveness, i.e. whether t meets the longer term aim, ethicality, i.e. whether the transformation is morally right, and elegance, i.e. whether it is aesthetically pleasing. the logic-based analysis stream also requires the analyst to compare the new system models with the actual reality as perceived. ssm suggests four ways of doing the comparisons: informal discussion, formal questioning, scenario writing based on operating the models, and trying to model the real world in the same structure as the conceptual models. 8 we shall adhere to checkland's interpretation of human activity system which differs slightly from that of wilson. wilson (1984) defines a human activity system as a system of activities plus social system (p. 25) whereas checkland (1981) defines a social system as a human activity system plus natural system (p. 121). checkland and scholes (1990) do not use this interpretation of a 'social system'. ajis vol 5 no 1 september 1997 12 would-be improvers of the problem situation problem situation in the real world relevant models real situation the situation as a culture 1. analysis of the intervention 2. 'social system' analysis 3. 'political system' analysis 0— oi differences between models and the real world stream of cultural analysis changes: systematically desirable culturally feasible action to improve the situation logic-based stream of analysis the stream of cultural inquiry consists of three types of analyses: analysis of the intervention, social system analysis and political system analysis. the first, includes a role analysis of the intervention focusing on three roles: the 'client' who caused the study to take place, the 'would-be problem solver' who wishes to do something about the situation in question and the 'problem owner'. for social system analysis checkland and scholes propose a model of three interacting elements: roles, norms, and values. a 'role' refers to a social position that is recognized as significant by people in the problem situation. 'norms' are expected behaviors characterizing a role, and 'values' are the standards used to judge the performance in a role. in the context of social system analysis, the authors point out that the nature of the social system is not likely to emerge from direct questions because the answers are merely likely to reiterate the official myths. instead, after each conversation, interview, or perusal of documents, etc., one should review the experience in order to form 'best judgments' about the meanings of roles, norms and values in question. political system analysis treats politics as a process by which differing interests reach accommodation, with which different parties decide to "go along" even though the conflicts endemic to human affairs continue to exist. checkland and scholes recognize the sensitivity of political analysis to public exposure: politics is ultimately concerned with power and its disposition, and the results of the political analysis will reveal overt and covert power relationships. if these are made public (along with potential vested interests), they can easily become a potent means of power in the real politics of the situation (p. 51). therefore users of ssm should be circumspect about the use of political analysis. the purpose of the logical and cultural streams of inquiry and action is to encourage a structured debate about the changes that could help with the problematic situation at hand. ssm proposes that the changes should be 'systemically desirable' and 'culturally feasible'. one problem with ssm in this context is that it does not give clear guidelines on how to proceed from the debate on the relevant human activity systems to the implementation of "systematically desirable and culturally feasible" changes. the likely reason for this is the strict conceptual separation between human activity systems and the real world. human activity systems by definition aj1s vol 5 no 1 september 1997 13 are not domain descriptions of the real-world, they are supposedly purely "notional" systems. if so, this precludes their use as constitutive models for the desirable and feasible changes in the real-world action, because in that case they would describe possible real-worlds and if implemented, would then describe the real-world (the model of the activity system could serve as the documentation for the new system). even if this point is waved as a mere conceptual difficulty, ssm does not advocate that the activity system should become the new system hence it is not clear how the transition from the status quo to an improved, feasible system is to be made (of course, structured methods do not solve this problem either, i.e. how to get from the functional model of the existing system to a new and improved functional model). in conclusion, we can say that ssm has a number of strengths in the area of problem formulation, but its major weakness from the viewpoint of is development is that its role in is development is not particularly well articulated. checkland and scholes (1990) suggest an approach to is development which is essentially based on recognizing rather than avoiding conflicts. it does this by defining multiple human activity systems to stimulate debate about their relevance and about the legitimacy of alternative root definitions (world views) behind them. once a 'truly relevant' system has been agreed upon, one can analyze information flows, data and processing needs. for each activity identified in the relevant human activity, one can ask what information would have to be available to enable someone to do this activity and what information would be generated by doing it (pp. 5657). from there one can proceed to using more conventional is development methods (p. 313).9 both multtvffiw (avison and wood-harper, 1990) and faor (schafer, et al. 1988) apply the "old" version of ssm that is confined to the logic-based stream of analysis only. multtvtew includes a specific approach for is development that consists of five stages (avison and wood-harper, 1990): analysis of human activity, information analysis, analysis and design of socio-technical aspects, design of the computer interface, and design of technical aspects. at this general level of five stages, the postponement of the socio-technical analysis until the information analysis has been completed (which means that the information system's functions and entities are defined) seems problematic. the sequence gives priority to the information system as a technical subsystem rather than giving equal emphasis to the social and technical options. faor (schafer, et al. 1988) applies ssm mainly as a source of general problem-solving and learning heuristics. these are applied in each of the main stages of faor: exploration, method tailoring, analysis and evaluation. compared to the other five approaches, ssm is a relative complete, well documented methodology the viability of which is well illustrated in many published cases. it is somewhat surprising that it has not received more widespread attention. the trade unionist approach the strength of the trade unionist approach is its focus on political-institutional power issues. it therefore appears to be best suited to organize the cooperation between management and collective bargaining units or trade unions. it suggests that the improvement of systems development is an issue of industrial democracy. this carries on the tradition that has evolved since the early 1960's mainly in three scandinavian countries, denmark, norway and sweden. the scandinavian background of the approach can largely be explained by the high unionization of the scandinavian labor force leading to strong national labor federations and by large social democratic parties with strong links to the national federations (ehn and kyng, 1987). this has created a favorable situation for the co-determination arrangements and laws in the mid-1970's which ensured employees and unions the right to participate in the design of and decision-making on computer systems. however, the opportunity to participate in isd posed a considerable challenge to trade unions for the following reason. trade unions had traditionally dealt with distribution issues such as pay, working hours, and terms of employment. with the participation in systems development, they were confronted with design and manufacturing issues which are often more unstructured. there were no clearly formulated union objectives and demands. the unions' and employee's previous experience were of limited value (sandberg, 1985). the recognition of this challenge was an important motivation for the formulation of trade union strategies for the development of computer-based systems. one can identify three major generations in the evolution of the approach (ehn and kyng, 1987).10 the first covers the first three major projects (i.e. njmf, demos and due, as referenced in the footnote. it proposed a 9 the more specific is development methodology of wilson is essentially based on similar ideas (wilson, 1984 pp. 245247). 10 the review of the trade unionist approach below and its analysis is based on the following material: nygaard and bergo (1974), carlsson, et al. (1978), ehn, et al. (1983), sandberg (1983), sandberg (1985), ehn and kyng (1987), bodker, et al. (1987), ehn (1988), kyng (1989), bodker, et al. (1991), bodker and gronbaek (1991a), bodker and gronbaek (1991b), ehn and kyng (1991), ehn and sjogren (1991). the preferred research strategy of the trade union approach has been action research. the most important action research projects were the njmf project in norway (nygaard and bergo, 1974), the ajis vol 5 no 1 september 1997 14 model for systems development the essence of which is to have a union-led "shadow organization" to challenge the proposals developed by the management-led project organization and to propose alternatives to it (ehn and sandberg, 1983). the second generation, the so-called "collective resource approach" based on the utopia project, has advocated the notion of craftsmanship as an ideal of work and a tool perspective as its natural ingredient. these two generations seem to reflect a gradual softening of the original, quite ideological marxist tones and goals (nygaard and bergo, 1974), to a theoretical synthesis of marxist, heideggerian and wittgensteinian elements (ehn, 1988). the most recent publications reflect the third phase of the trade unionist approach, called "cooperative design" (b0dker, el al. 1991; b0dker and gr0nbaek, 1991a; b0dker and gr0nbaek, 1991b; ehn and kyng, 1991; ehn and sjogren, 1991), which is theoretically based more on the philosophies of heidegger and wittgenstein rather than on marx." a summary of the approach up to the late 1980's can be found in ehn and kyng (1987) and in ehn (1988). the theses derived from ehn and kyng (1987) are summarized and presented in table 3. table 3. major results (theses) of the trade unionist approach rationalization of labor processes 1. systems design is a fundamental division of the labor process. 2. changes of labor processes must be applied to the totality as well as to the separate parts, to the division of labor between different groups of workers within the labor process of systems design, to the division of labor between systems design and the labor process, and to division of labor within the use labor process. 3. capital accumulation or generation of profits is the basic driving force in changes of labor processes. 4. intensification of work and use of new technology are two basic strategies for capital accumulation. 5. direct control and responsible autonomy are complementary strategies for capital accumulation. 6. class struggle is an important aspect of actual changes in labor processes. not only of the use process designed, but also of the systems design process and of possible integration in the future. trade union "design" activities 7. in democratization of design and use of new technology in scandinavia, trade unions especially on local level should play an active role. 8. design and use if new technology requires new trade union activities. 9. a participative approach to the design process is not sufficient. 10. the most important prerequisite for trade union participation in management's design process is a parallel and independent process of accumulation of knowledge on the part of the union. 11. the union investigation and negotiation strategy is a democratic and workable complement to traditional design strategies. but it is very resource consuming for the local unions. 12. local unions need external resources and support in their design activities. 13. the practical application of the local union strategy has to depend on the particular prerequisites in each case. 14. a local trade union strategy has to be based on solidarity between the different groups of workers involved, a solidarity which goes beyond the traditional division of labor in the labor process and the traditional jurisdictions between the unions involved. 15. a clear distinction based on negotiations between union and management roles in the design process is not in opposition to, but a prerequisite for the democratization of co-operation and decision-making in the work organization. 16. local union design efforts have to be supplemented by central union design activities. 17. the existing technology in many cases restricts the possibilities to locally reach trade union objectives, especially with respect to skill, but also with respect to work organization. 18. central trade unions must influence the process of research and development of new technology to change the supply of technological and organizational solutions. 19. there are possibilities to design new technology based on social criteria such as skill and democracy at work. 20. equally important is a trade union strategy to influence the demand for these technological and organizational alternatives. demos project in sweden (carlsson, el al. 1978; ehn and sandberg , 1983; sandberg, 1983), the due project in denmark (kyng and mathiassen, 1982) and the utopia project in denmark and sweden (bodker, el al. 1987; ehn, 1988). 11 b0dker et al. (199la) nevertheless, express their sympathy for users as though they are members of "resource weak groups" (as opposed to management or executives), and refer to the trade union-centered approaches for organizing development projects for such resource weak groups (pp. 146-147). ajis vol 5 no 1 september 1997 15 21. central trade unions must provide training with a trade union perspective on new technology and work organization, and influence the supply of professional training for skilled work. design for democracy and skill 22. tools and techniques for local trade union "design" work have to reflect the specific conditions of the work such as a trade union perspective and the limited resources available. 23. design of computer support is design of (conditions for) labor processes. 24. labor processes cannot be reduced to information processes. 25. design use models. 26. hardware should be considered early in the design, in parallel with software, not after. 27. important aspects of labor processes in relation to design of computer support cannot be formally described. 28. professional experience with and knowledge of the labor process for which computer support is being designed are important in the design process. 29. professional experience with and knowledge of computers are important when designing computer support for a labor process. 30. design should be done with users, neither for nor by them. 31. mutual learning should be an important part of the work in a design group. 32. design by doing. 33. designers should restrict their activities to a few domains of application, and they should spend at least a year or two getting acquainted with a new area before doing actual design this third phase emphasizes "cooperative design". as mentioned above, it theoretically leans more on the philosophies of heidegger and wittgenstein rather than on marx. bodker et al. (1991a) nevertheless, express their sympathy for users if they are members of "resource weak groups" (as opposed to management or executives), and refer to the trade union-centered approaches for organizing development projects for such resource weak groups (pp. 146-147). at the same the trade-unionist approach has lost most of its it critical edge, and has quite ironically approached the sociotechnical approach in many respects. bjerknes and bratteteig (1995) remark that despite the intellectual conflict between the sociotechnical approach and trade unionist approach (see ehn, 1988), it is difficult to see a difference between the two in practice. this is more so in the case of the "cooperative design" in which the "design process is viewed as a (rather harmonious) dialogue between a designer and a user about the design of a particular computer application", the assumption being that the cooperative process leads to improved work situation and that organizations are willing to introduce the proposed changes (bjerknes and bratteteig, 1995).12 the "cooperative design" approach in particular has recently aroused considerable interest (kuhn and muller, 1993). it has focused on the technical prerequisites of user participation in the actual building of is. to facilitate the greatest possible role for users, it has aimed at developing various tools, techniques and principles to support effective user participation. in research, these tools are explored through experimenting with prototypes. this makes it difficult to evaluate how effective they would be in practice, because little research has been done to evaluate how effective these tools are in developing successful systems, in particular when these cover larger organizational units. the experience with cooperative prototyping tools is mainly based on action research projects with considerable research intervention and funding from researchers (clement and van den besselaar, 1993). clement and van den besselaar also report considerable difficulty with sustaining the continued use of the participatory design approaches after the research intervention when the novelty wanes and other work priorities reassert themselves. the attempts to transfer the ideas of "cooperative design" to north america have also aroused considerable debate in scandinavia (kraft and bansler, 1994; kyng, 1994). even though the proponents of the trade union tradition have used various names to characterize their approach, the label "trade unionist" emphasizes its most distinctive features. these are its origin, the scandinavian industrial relations as its social background, the role of trade unions as sponsors and partners of its major research projects, and its belief in trade unions as legitimate representatives of resource weak groups. the professional work practice (pwp) approach the strength of the pwp approach is its emphasis on studying the actual work practices of systems professionals before attempting to improve them. its advocates criticize methodologies for not paying sufficient attention to what system practitioners actually do and if what they do is deficient, to take a realistic look how it could be 12 this comes quite close to mumford's (1983) argumentation that "people should be able to influence the design of their own work situations and that if this kind of intervention is encouraged then there are likely to be both job satisfaction and efficiency gains" (p. 1) ajis vol 5 no 1 september 1997 16 changed for the better. in this sense the focus of the pwp approach has been on the relationship between is development methods and practice. from this perspective, new methodologies are to be considered as one of the options for improving the working practices, but by no means the only one. the empirical base for the strategy of the pwp approach is derived from investigating the conditions which determine the actual working practices of systems developers. the empirical base of the pwp approach appears to be stronger than its theoretical underpinnings. its theoretical base is inspired by organizational learning theories, in particular, the "reflective practitioner" (schon, 1983).13 the pwp approach's practical starting point is best explained by its historical link to the mars projects.14 a small research group which included mathiassen initiated the mars projects in denmark during the early 1980's. they investigated how systems development was actually carried out in practice (e.g. lanzara and mathiassen, 1984). they also experimented with different ways of changing working practices (andersen, et al. 1990, p. xii). these action research projects challenged the popular belief that new is development approaches (methods, techniques and tools) would improve the effectiveness of is practitioners by displacing poor approaches. they found that the more experienced the analysts were, the less they followed documented methodologies. this applied even if the organization had introduced a specific method of its own as a development standard. methods were at best crutches for beginners to be tossed aside after a period of apprenticeship. more important for understanding how systems are in fact developed are the actual working practices of the systems professionals in the organization which the beginners joined. the basic organizational action learning perspective of the pwp approach is summarized in figure 2 (ibid, p. 43). it is based on three dichotomies: product-oriented vs. process-oriented view, reflection vs. action, and vision vs. present reality. "performance" in figure 2 refers to the actual systems development comprising the reflective functions (analysis and design) and innovative action function (realization). "management" on the other hand corresponds to project management. the approach especially underlines experience, reflection and learning in the development of working practices towards increased professionalism. its development is heavily based on empirical experience from practical systems development projects, without any strong theoretical preconceptions. this may explain why the underlying theoretical assumptions of the approach are weakly documented. in addition to the dissertation of mathiassen (1981), the ideas of argyris and schon (e.g. argyris and schon, 1978; schon, 1983) seem to form the only consistent source of inspiration. andersen et al. (1990, pp. 46-59) condense their practical recommendations for is development into a number of principles which are summarized in table 4. table 4 indicates that the pwp approach attempts to integrate ideas from different sources into the unifying theoretical framework of figure 2. from this framework more specific tools have been derived to support professional learning and reflection. the types of contributions of the pwp approach include: maps for diagnosing problematic situations (diagnostic, ecological and virtual maps, lanzara and mathiassen, 1985; andersen, et al. 1990); the use of metaphors (madsen, 1989) in generating visions (kensing, 1987; kensing and madsen, 1991); and the systematic maintenance of professional diaries to document how decisions were made through the development process (cf. lanzara and mathiassen 1984; jepsen, etal. 1989). 13 the pwp is described in a number of papers lanzara and mathiassen (1985), kensing (1987), gronbaek (1989), jepsen, et al. (1989), madsen (1989), nielsen (1989), kensing and madsen (1991), and in a book (andersen, et al. 1990). the following review is mostly influenced by the last reference, i.e. andersen et al. (1990). 14 we emphasize the differences to the trade union approach here rather then the commonalities. the professional work practice' (pwp) approach has been developed during the 1980's in denmark by mathiassen and his colleagues. while it followed the mars project, it also has some of its roots in the trade unionist approach. the influences from trade union strategy follow from mathiassen's association and some of his colleagues with the utopia project (e.g. kyng and mathiassen, 1982). however, fundamental disagreements about the practicality of the utopia strategy as the sole basis for professional education prompted mathiassen to pursue a new strategy which differs substantially from those pursued by ehn and kyng. ajis vol 5 no 1 september 1997 17 visions present reality visions system development performance design • •̂ analysis *realization management planning •^ evaluation f regulation figure 2. the basic framework of the pwp approach table 4. principles of is development performance principles pi analysis and design are mutually dependent, and should therefore be performed concurrently in order to support each other. p2 product-oriented reflection (analysis and design) and realization affect each other, and should therefore be performed concurrently in order to support each other. p3 it is not possible to perform qualified analysis and design strictly according to given guidelines. the activities require experience, intuition, imagination, and reflection. p4 the project group's relation to the users is of decisive importance to the quality of the analysis. if direct cooperation cannot be established between developers and users, precise requirements specifications and knowledge about the application situation are needed. p5 good design is a question of flying high and keeping both feet firmly on the ground. new visions and unconventional solutions must be created. it must at the same time be possible to introduce them into the organization. p6 technically oriented analysis and design may result in perfect solutions to wrong problems. qualified analysis and design require technical, organizational and social competence. p7 qualified analysis and design require that different perspectives are applied. the choice of techniques and tools, and the choice of how to organize the process depend upon the situation. p8 neither the analysis nor the design activities can move unidirectionally from totality to detail. knowledge of details and the concrete conditions are prerequisites for obtaining a comprehensive view of the situation. p9 tests do not solve any problems. plan with time for repairs, thus preventing permanent firefighting. p10 qualified realization requires thorough planning of the conversion. the overall design should include the conversion plan. management principles ml evaluation and planning are mutually dependent, and should therefore be performed concurrently in order to support each other. m2 process-oriented reflection (planning and evaluation) and regulation affect each other, and should therefore be performed concurrently in order to support each other. ajis vol 5 no 1 september 1997 18 m3 systems development is characterized by a high degree of uncertainty. the most important prerequisite for qualified management is therefore transparency in both processes and product. m4 it pays to establish the project systematically. m5 baselines and checkpoints are better than traditional phases. traditional division into phases confuses time and content, thus making dynamic regulation of the project difficult. m6 project plans must facilitate evaluations. they must be in writing and contain evaluation criteria and procedures. m7 only the systems developers know enough to make realistic plans and to evaluate the status of the project. ms it is important that all participants in a project understand and accept the plan. m9 it is necessary to apply several estimation techniques. the plan should be based on a problem estimate, and express the degree of uncertainty in the estimate. m10 it is necessary to plan with management activities. management typically constitutes 15% to 25% of the effort. principles of the relationship between management and performance pm1 a systems development project should be organized in a way which ensures direct and close interaction between performance and management activities. pm2 the most important intermediate products are the plan and the overall design. in summary, the basic research question of the pwp approach is to ask how experience, reflection and learning can be marshaled in the development of good working practices towards increased professionalism. the fundamental answer suggested in andersen et al. (1990, p. 60) is that better working practices can be learned by a combination of three strategies. the first is theoretical study (i.e. learning of new methods and tools or frameworks of analysis). the theoretical learning strategy must always be combined with the second strategy, seeking a broad spectrum of practical experience (i.e. by working with accomplished masters of the art). these two strategies should help new practitioners to design and maintain their own ways of learning. an important avenue to achieve professionalism is to help practitioners improve their capabilities to reflect upon their experiences and then change their behaviors accordingly. in this way both successes and failures become important assets for improving the practice. professional diary keeping is to help track the experience record of success stories and failures what worked and what did not. the third strategy for improving learning is building open-minded professional attitudes which encourage practitioners to actively seek out pertinent information by interacting with the professional community and through the study of the literature. in order to support good professional attitude formation, anderson et al. also state that an environment and tradition for professional systems development should contain elements such as the following (p. 9): active system developers, sufficient resources, exchange and evaluation of experience and study of relevant literature. almost ironically, the pwp approach has failed to see the importance of methods in this process of reflection and learning, not as methods to be followed obediently, but as intellectual frameworks with which the experience can be compared and reflected upon (cf. ssm above). summary of the five isd approaches whereas the five approaches cannot be considered "full-scale" is development methodologies, they nevertheless represent significant streams of is research and have the potential of becoming complete methodologies. although their current contribution is mainly cognitive in nature, i.e. broadened awareness of the phenomena related to is development, they also provide a fair amount of practical support for is development at least in the limited areas that are discussed in the above synopses. table 5 summarizes the five isd approaches in terms of their goals, guiding principles, fundamental concepts, and principles of the isd process (see section 2 of the paper). the characterization of the interactionist approach in table 5 is based on the web and institutional views rather than the discrete-entity and tool views. this follows from kjing and scacchi's (1982) preference for providing a rich set of ideas to characterize the web and institutional views. table 5 is able to summarize only some of their major points. however, the interactionist approach is very embryonic as an isd approach, because it has not been elaborated into a concrete methodology for isd. nevertheless, we propose to treat it as an approach to exemplify a case of an isd approach without any methodology instances. furthermore, kling and scacchi (1982) clearly recognize the possibility of elaborating the web model to the level of a more concrete isd methodology (p. 71). the sa-approach is an example of an approach with a number of methodology instances. its interpretation in table 5 is not entirely based on the abstraction of common features of existing methodologies, but also on its theoretical background. to illustrate this, table 5 identifies discourse/conversation analysis and prepositional content analysis (information modeling in lehtinen and lyytinen, 1986) as a fundamental aspect of the sa-based approach. this is based on our interpretation of speech act theory (see also ljungberg and holm, 1995). as ajis vol 5 no 1 september 1997 19 noted in section 4.2 however, winograd et al. (1986, 1987/1988, 1988) do not address the prepositional content of the subject of content analysis. consequently, we interpret the sa-based methodology of winograd et al. as partial in this respect. it is beyond the scope of this paper to go into details of these specific methodologies. we only wish to point out with this example that each methodology instance of the sa-based approach may use slightly different activities, techniques and tools for discourse and prepositional content analysis. ssm is an example of an isd approach with a number of methodology instances. for example, ssm (according to checkland, 1981 and checkland and scholes, 1990), ssm (wilson's 1984 version), multiview (avison and wood-harper, 1990) and faor (schafer, et al, 1988) are all methodology instances. ssm can be considered as both an approach and a methodology in parallel. we interpret the extension of the ssm to include the stream of cultural analysis (checkland and scholes, 1990) in addition to the stream of logic-based analysis as an evolution of ssm as an approach. at the same time, checkland and scholes cautiously distance themselves from the seven stage model (checkland, 1981) of the stream of logic-based analysis. this can be interpreted as an evolution of ssm as a methodology from checkland's (1981) original version. even though techniques like rich pictures and catwoe are intimately associated with ssm, they are techniques which do not define ssm as an approach. although the trade unionist approach has evolved through a number of generations, these have implied certain changes in emphasis and focus which we interpret as a continued expansion and enrichment of the approach. the development of the trade unionist approach has mainly taken place at the level of an approach. as a methodology the trade unionist approach is still somewhat embryonic, proposing some specific techniques (such the union-led shadow project organization, mock-ups, prototypes) but needing further development of tools, techniques and activities. pwp has also mainly been developed as an isd approach. it includes a limited repertoire of specific techniques (such as diagnostic mapping techniques, metaphorical design, future workshops and use of diaries). as an isd approach, it has been particularly explicit in stating a number of principles for the process of isd. these principles revolve around the dualities listed under the entry 'concepts' in table 5 and should be addressed concurrently. ajis vol 5 no 1 september 1997 1ua 0 l. aa a ft) a_o cy>o •o m 1 0 cm o </> (1j m ar ii e 3 cn «k cu zs h •5* * p ro fe ss io na l w oi p ra ct ic e ap pr oa c tr ad e u ni on is t ap pr oa ch * ^ *• * k*) & sa -b as ed ap pr oa ch .h in te ra ct io ni ap pr oa ch j= u 2 q. q. < •s ™ to p ro m ot e in cr ea s pr of es si on al is m o f 1 de si gn er s o c <*' o (xs *0 0. "5§ s & s to d ev el op c on di ti ef fe ct iv e w or ke r p a pa tio n in o rd er to s de m oc ra cy a t w or k qu al ity o f w or k e -a y 8. s .s a. 0 e s s s s 2 s lu l l~ — c cj > •§ ° oiii! (2 l i d s g> ii . 4 £ 1 § s a 6-g o e .2 « m e to p ro vi de a m et h fo r m od el lin g c om ac tio n in o rg an iz al es pe ci al ly s pe ec h • < of c ha ng es : cr ea tir ta in in g, re po rt in g, .£ ^ « § ic •s § « § e _ h -c '— u § = ° 2 a c« °s! c >» »i|ji "0 .2 s 0« .8 « -1 .! !s •§ a -o i f2 g s § e 1 o co n '£ co bo an d te rm in at in g or co tio na l c om m it m en i to 3 e » t« ^2s.s° «y e os « .y•a » 3 g to « 1 g °-£• «u t sl c 5£ a -s ka l^_ l>" d es ig n of c om pu te su pp or t i s de si gn o : co nd it io ns f or w or t o , e = i 4) co 3 to p —*!s l i i § e £<«-. j£ m 11! .2 tj e -fu|s1 § g c c "b. -2 1 e 1 s, >>l-si .s '5 <= c 8 -g< co £ i« > i-'! « §£l•s -5vs ca o c s 2 | 1 8.!.5 to co c * § < .2 s m tt 1! f v1 o _ so' _ " o. 3 ° c ,_ cla « p i i-iti c c u « t > ' o . u o c a 2 ' s s i l s l s i i l i a .s'l 8 s g»|-8l s-si s'l'i.-sl 2 2 « o | | l ^ s § § s 5 j « s | gs u — « a > c — t u d . f c c ao a g '5 ^ b -t: 5> g e s g § q s s £ g o t r o s . e s j 2 i i s a b s . § 4)•o .. edd tft) u *!j!i ililt 111 iiu 2 < & § a 1 « x 5 "3. .2 •= 1& e ^ c u •tu « '> 2 .£ e a •£ -s 1 s>^ s>i a^ c >> c 2 c ~ j « .2 8: g « .y -2 « s 3 5f 8 i sf £ c ^1 e i e;, o c .e tu > u £ op «9 c s j> ts g § 2 < £ 2 & •e x 1 i e 1 2 "3 § u c« o ofls; >> « s c>•* .c 2 w s 1 8 8 aii hiiiin il'si *̂* to n— ̂ mm <*— e to r3 co * ^0. 4) eo -j q'o s s « a s ^ e s § c 0 co .5 — — b co -s a s « jb -5 "o 1 s -s. s o a •so. c o s f . o !il.|» i* i»« £ — ' c 3 0 « > . . s c 5 ' s 1 1 g>-a 2 2 s 11 > = «£ s'5 1 i § 1 s^ -s.ees a.2c3 1 s.5 82 ^ fft pe rf or m an ce v s. m an ag em en t; r ef le ct io n vs . a ct io i t»~ c om pu te rs a s to ol s (u nd er th e co nt ro l c ea ch w or ke r) co o to .. c/3 00 >, .. c i1 c w cl ta ns ch au u h um an a ct iv i r oo t de fm iti oi « g sp ee ch a ct s; il lo cu tio na ry p oi nt pr op os it io na l co nt i u, °a "i ° £ s, .5 in fo rm at io n s in st it ut io ns ; so ci al u se o f 2 ĉ ii « e fi i >> "« si . v is io ns v s. p re se nt pr od uc tor ie nt ed v s e r el ev an t sy st e co co « co d is co ur se s/ co nv er o. co 1o sy st em s; c om pl ex a nd pr oc es s or ie nt ed ; a na ly si s vs . d es ig n; ^t o !ss u im c o 0 >s pi ng n eg ot ia ti n on -n eu tr al it co pl an ni ng v s. e va lu al re so ur ce s u co -o co c u id a ll th e ab ov e du al it m ut ua lly d ep en de nt s bfiu e•o •=e « pa ra lle l a nd in de pe pr oc es s of a cc um ul co 'so >. 1 "2~ u co v) •s& 3 0 st re am o f c ul t st re am o f lo gi co « d is co ur se /c on ve rs an al ys is ; < z ^cm pr in ci pl e th e is o is1 § cao. ,_ u . _ o o •s 00 "w— c <u lip— "\ jb « » 5 e sllll •a c u u o 5 — '*" "z "z•§•& i s o a •n discussion the synopses and comparison of the approaches not only aimed at introducing the five approaches but also allows us to recognize their strengths and weaknesses. additionally, we can look how the approaches complement each other. each of these is explored in the next sections. strengths and weaknesses of the iso approaches the strength of the interactionist approach lies in its rich and articulated view of organizations. in interactionism, information systems are clearly considered to be first of all, social systems and only secondarily, technical systems. even though it focuses on organizational symbolism and organizational politics, it also recognizes the potential value of the other views, i.e. the formal-rational, structural, human relations and class politics perspectives of is. currently, the interactionist approach is an analytical framework for analyzing, understanding and interpreting is development situations without concrete methods on how to actually develop systems. the sa-based approach clearly has the most rigorous theoretical background sa theory. it is also the most concrete in many respects as an is development approach. it has led to a number of technical implementations. the sa-based approach is quite means-end-oriented without any consideration of the actual consequences of is development and even less critical of the consequences. the conceptual complexity of the sa theory has led to a fragmented situation in which there are a great number of specific is development methods applying some subset of the theory. they have not yet coalesced into a systematic, cooperative, approach covering all aspects of isd. ssm as an is development approach also has a firm theoretical background. the obvious strength of ssm lies in problematic situations in which there is considerable controversy on the very nature of the human activity system to be supported by an information system. as suggested by faor (schafer, et al. (1988)) ssm may also be useful as a general learning approach in less problematic situations. a clear weakness of ssm is that it is weakly integrated with is development. it has been subject to evolution during the 1980's as a general systems approach and been updated in checkland and scholes (1990). furthermore, even though checkland (1981) suggests that the lack of a critical perspective in ssm is a matter of practice rather than principle, there are several aspects in ssm that reflect the values of those in power, i.e. management. the most distinctive aspect of the scandinavian trade unionist approach has been its critical view of is and the primacy given to workers' interests. as pointed out above this aspect has been weakening in the recent evolution of the approach. on the other hand, this less radical ethical stance may explain the increased interest in the trade unionist approach. its latest two stages 'collective resource' approach and 'cooperative design' approach have also advocated a tool perspective of computer artifacts. even though the tool view can be considered as an ideal, it has led to a quite technical view of computer artifacts (information systems), a view that has lost sight of the possible role of information systems as a means of control and domination. it also explains the artifact-oriented design and development of work in which work is not subjected to any explicit analysis, design and evaluation. the strength of the professional work practice approach is that it is firmly grounded in is development practice. the approach is practical and seemingly quite clear, but lacks clarity and structure at a more theoretical level. this is likely so because it lacks a clear, unifying theoretical background. the pwp approach especially focuses on experience, reflection and learning in the development of working practices towards increased professionalism of is developers, but it does not recognize the potential value of articulated methods as intellectual frameworks facilitating effective learning. the pwp approach as articulated in andersen et al. (1990) is eclectic. it adopts a collection of concepts, ideas and methods the underlying assumptions of which make it look "orthodox" in some respects. 24 astley, w.g., (1985), "administrative science as socially constructed truth", administrative science quarterly, vol. 30, pp. 473-513 auramaki. e, hirschheim, r. and lyytinen, k. (1992a), "modeling offices through discourse analysis: the sampo approach", the computer journal, vol. 35, no. 4, 342-352 auramaki. e, hirschheim, r. and lyytinen, k. (1992b), "modeling offices through discourse analysis: a comparison and evaluation of sampo with ossad and icn", the computer journal, vol. 35, no. 5, pp. 492-500 auramaki, e., lehtinen, e. and lyytinen, k. (1988), "a speech-act-based office modeling approach", acm transactions on office information systems, vol. 6, no. 2, pp. 126-152 avgerou, c. and cornford, a. (1993), "a review of the methodologies movement", journal of information technology, vol.8,, pp.277-286. avison, d.e. and fitzgerald, g. (1988), information systems development methodologies: techniques and tools, blackwell, oxford avison, o.e. and fitzgerald, g., (1995), information systems development: methodologies, techniques and tools, second edition, mcgraw-hill, ny avison, d., shah, h., powell, p., and uppal, p. (1992), "applying methodologies for information systems development", journal of information technology, vol.7, pp. 127-140. avison, d.e. and wood-harper, a.t., (1990), multiview, an exploration in information systems development, blackwell scientific publications, oxford banville, c. and landry, m., (1989), "can the field of mis disciplined?", communications of the acm, vol. 32, no. 1, pp. 48-60 baskerville, r., travis, j. and truex, d. (1992), "systems without methods: the impact of new technologies on information systems development projects", in kendall, k.e., lyytinen, k. and degross, j.i. (eds.), the impact of computer supported technologies on information systems development, north-holland, amsterdam, pp. 241-269 bjerknes, g. and bratteteig, t., (1995), "user participation and democracy: a discussion of scandinavian research on systems development", scandinavian journal of information systems, vol. 7, no. 2, april, pp. 73-97 bostrom, r.b. and heinen, j.s., (1977), "mis problems and failures. a socio-technical perspective: part i: the causes", mis quarterly, vol. 1, no. 3 bubenko, j. (1986), "information systems methodologies a research view", in olle, t.w., sol, h.g. and verrijn-stuart, a.a. (eds.), information systems design methodologies: improving the practice, northholland, amsterdam bubenko, j. (1988), selecting a strategy for computer-aided software engineering (case), syslab report no. 59, university of stockholm, sweden burrell, g. and morgan, g., (1979), sociological paradigms and organizational analysis, heineman, london bodker, s. et al., (1987), "a utopian experience, on design of powerful computer-based tools for skilled graphic workers", in bjerkenes, g., ehn, p. and kyng, m. (eds.): computers and democracy, avebury, aldershot, pp. 251-278 bodker, s., greenbaum, j. and kyng, m., (1991), "setting the stage for design as action", in greenbaum, j. and kyng, m. (eds.), cooperative design of computer systems, lawrence erlbaum associates, hillsdale, new jersey, pp. 139-154 bodker, s. and gronbaek, k., (199la), "design in action: from prototyping by demonstration to cooperative prototyping", in greenbaum, j. and kyng, m. (eds.), cooperative design of computer systems, lawrence erlbaum associates, hillsdale, new jersey, pp. 197-218 bodker, s. and gronbaek, k., (1991b), "cooperative prototyping studies users and designers envision a dental case record system", in bowers, j.m. and benford, s.d. (eds.), studies in computer supported cooperative work, elsevier science publishers b.v. (north-holland), amsterdam, pp. 315-332 carlsson, j. et al., (1978), "planning and control from the perspective of labor: a short presentation of the demos project", accounting, organizations and society, vol. 3, no. 3/4, pp. 249-260 chatzoglou, p.d. and macaulay, l.a., (1996) "requirements capture and is methodologies", information systems journal, vol. 6, no.2, pp. 209-225 checkland, p., (1981), systems thinking, systems practice, john wiley & sons, chichester checkland, p. and scholes, j., (1990), soft systems methodology in action, john wiley & sons, chichester churchman, c.w. (1978), the systems approach and its enemies, basic books, new york dagwell, r. and weber, r., (1983), "system designers' user models: a comparative study and methodological critique", communications of the acm, vol. 26, no. 11, pp. 987-997 ajis vol 5 no 1 september 1997 25 dewitz, s.k. and lee, r.m., (1989), "legal procedures as formal conversations: contracting on a performative network", in degross, j.i., henderson, j.c. and konsynski, b.r. (eds.), proceedings of the tenth international conference on information systems, boston, massachusetts, pp. 53-65 dietz, j.l.g., (1992), "subject-oriented modeling of open active systems", in falkenberg, e.d., holland, c. and el-sayed e.n. (eds.), information systems concepts: improving the understanding, ifip transactions a-4, north-holland, amsterdam, pp. 227-238 dietz, j.l.g. and widdershoven, g.a.m, (1991), "speech acts or communication action", in bannon, l., robinson, m. and schmidt, k. (eds.), proceedings of the second european conference on computersupported cooperative work, kluwer academic publishers, dordrecht, pp. 235-248 dobson, j.e., martin, m.j., olphert, c.w. and powrie, s.e., (1991), "determining requirements for cscw: the ordit approach", in stamper, r.k., kerola, p. and lyytinen, k. (eds.), collaborative work, social communications and information systems, elsevier science publishers b.v. (north-holland), amsterdam, pp. 333-353 donaldson dewitz, s., (1991), "contracting on a performative network: using information technology as a legal intermediary", in stamper, r.k., kerola, p. and lyytinen, k. (eds.), collaborative work, social communications and information systems, elsevier science publishers b.v. (north-holland), amsterdam, pp. 271-293 ehn, p., (1988), work-oriented design of computer artifacts, arbetslivscentrum, stockholm ehn, p. and kyng, m.,(1987), "the collective resource approach to systems design", in bjerkenes, g., ehn, p. and kyng, m. (eds.): computers and democracy, avebury, aldershot, pp. 17-57 ehn, p. and kyng, m., (1991), "cardboard computers: mocking-it-up and hand-on the future", in greenbaum, j. and kyng, m. (eds.), cooperative design of computer systems, lawrence erlbaum associates, hillsdale, new jersey, 169-195 ehn, p., kyng, m., sundblad, y. et al, (1983), "the utopia project", in briefs, u., ciborra, c. and schneider, l. (eds.): systems design for, with, and by the users, north-holland, amsterdam, pp. 439-449 ehn, p. and sandberg, a., (1983), "local union influence on technology and work organization", in briefs, u., ciborra, c. and schneider, l. (eds.): systems design for, with, and by the users, north-holland, amsterdam, pp. 427-437 ehn, p. and sjogren, d., (1991), "from system descriptions to scripts for action", in greenbaum, j. and kyng, m. (eds.), cooperative design of computer systems, lawrence erlbaum associates, hillsdale, new jersey, pp. 241-268 emblay, d.w., jackson, r.b. and woodfield, s.n., (1995), oo-systems analysis: is it or isn't it, ieee software, vol. 12, no. 4,1995, pp. 19-33 episkopou, d. and wood-harper, a.t. (1986), 'towards a framework to choose appropriate is approaches", the computer journal, vol.29, no.3, pp.222-228. fitzgerald, b. (1994), "the systems development dilemma: whether to adopt formalized systems development methodologies or not?", proceedings of the second european conference on information systems, w. baets (ed.), nijenrode university, pp.691-706. flores, f., graves, m., hartfield, b. and winograd, t. (1988), "computer systems and the design of organizational interaction", acm transactions on office information systems, vol. 6, no. 2, pp. 153-172 flores, f. and ludlow, j.j., (1980), "doing and speaking in the office", in pick, g. and sprague, r.h. jr. (eds.): decision support systems: issues and challenges, pergamon press, elmsford, ny, pp. 95-118 goldkuhl, g. and lyytinen, k., (1982), "a language view of information systems", in ross, c. and ginz-berg, m. (eds.): proceedings of the third international conference on information, ann arbor, pp. 13-29 goldkuhl, g. and lyytinen, k., (1984), "information system specification as rule reconstruction", in bemelmans, th.m.a. (ed.): beyond productivity, information systems development for organizational effectiveness, north-holland, amsterdam, pp. 79-94 gr0nbaek, k., (1989), "rapid prototyping with fourth generation systems an empirical study", office: technology and people, vol. 5. no. 2, pp. 105-125 gr0nbaek k., (1991), prototyping and active user involvement in systems development: towards a cooperative prototyping approach, phd thesis, aarhus university greenbaum, j. and kyng, m. (eds.) (199la), cooperative design of computer systems, lawrence erlbaum associates, hillsdale, new jersey greenbaum, j. and kyng, m., (1991b), "introduction: situated design", in greenbaum, j. and kyng, m. (eds.), cooperative design of computer systems, lawrence erlbaum associates, hillsdale, new jersey, pp. 1-24 hammer, m. and champy, j., (1993), reengineering the corporation: a manifesto for business revolution, nicholas brealey, london, 1993 ajis vol 5 no 1 september 1997 26 hardy, c.j., thompson, j.b. and edwards, h.m. (1995), "the use, limitations and customization of structured systems development methods in the united kingdom", information and software technology, vol. 37, no. 9, pp. 467-477 harmsen, f., brinkkemper, s. and oei, h. (1994), situational method engineering for information system project approaches, in verrijn-stuart, a.a. and olle, t.w. (eds.), methods and associated tools for the information systems life cycle, ifip transactions a-55, north-holland, amsterdam, pp. 169-194 hedberg, b. and mumford, e., (1975), "the design of computer systems: man's vision of man as an integral part of the systems design process", in mumford, e. and sackrnan, h. (eds.): human choice and computers, american elsevier publishing company, new york, pp. 31-59 henderson-sellers, b. (1995), "who need an object-oriented methodology anyway", journal of object oriented programming, vol. 8, no. 6, pp. 6-8 hirschheim, r. and klein h.k., (1989), "four paradigms of information systems development", communications of the acm, vol. 32, no. 10, pp. 1199-1216 hirschheim, r. and klein h.k., (1992), "paradigmatic influences on information systems development methodologies: evolution and conceptual advances", in yovits, m. (ed.). advances in computers, vol. 34, pp. 293-392 hirschheim, r. and klein, h.k., (1994), "realizing emancipatory principles in information systems development: the case for ethics", mis quarterly, vol. 18, no. 1, march, pp. 83-109 hirschheim, r. and klein h.k. and lyytinen, k., (1995), information systems development and data modeling: conceptual and philosophical foundations, cambridge university press, cambridge, uk hirschheim, r. and klein h.k. and lyytinen, k., (1996), "exploring the intellectual foundations of information systems", accounting, management and information technologies. vol. 6, no. 1 / 2, pp. 1-64 humphrey, w.s. (1989), managing the software process, addison-wesley, reading, ma hong, s., van den goor, g. and brinkkemper, s., (1993), "a formal approach to the comparison of objectoriented analysis and design methodologies", proceedings of the twenty sixth annual hawaii international conference on systems sciences, vol. x, ieee computer society press, pp. 689-698 lacono, s. and kling, r., (1988), "computer systems as institutions: social dimensions of computing in organizations", in degross, j.i. and olson, m.h. (eds.): proceedings of the ninth international conference on information systems, minneapolis, minnesota, pp. 101-110 livari, j., (1991), "a paradigmatic analysis of contemporary schools of is development", european journal of information systems, vol. 1, no. 4, pp. 249-272 livari, j., (1994), "object-oriented information systems analysis: a comparison of six object-oriented analysis methods", in verrijn-stuart, a.a. and olle, t.w. (eds.), methods and associated tools for the information systems life cycle, ifip transactions a-55, north-holland, amsterdam, pp. 85-110 livari, j. and hirschheim, r., (1996), "analyzing information systems development: a comparison and analysis of eight is development approaches", information systems, vol. 21, no. 7, pp. 551-575 livari, j., hirschheim, r. and klein, h., (1997) "a paradigmatic analysis contrasting information systems development approaches and methodologies", accepted for publication information systems research iso 9000-3 (1991), guidelines for the application of iso 9001 to the development, supply, and maintenance of software, international organization for standardization, geneva janson m.a. and woo, c.c., (1992), "investigating information and knowledge gathering methods: a speech act lexicon perspective", in falkenberg, e.d., rolland, c. and el-sayed e.n. (eds.), information systems concepts: improving the understanding, ifip transactions a-4, north-holland, amsterdam, pp. 239-257 jayaratna, n., (1994), understanding and evaluating methodologies, nisad: a systematic framework, mcgraw-hill, maidenhead jayartna, n. and fitzgerald, b. (1996), (eds.) lessons learned from the use of methodologies: fourth conference on information systems methodologies jepsen, l., mathiassen, l. and nielsen, p., (1989), "back to thinking mode: diaries for the management of information systems development projects", behavior and information technology, vol. 8, no. 3, pp. 207-217 johannesson, p., (1995), "representation and communication a speech act based approach to information systems design", information systems, vol. 20, no. 4, pp. 291-303 kensing, f., (1987), "generation of visions in systems development", in docherty, p., fuchs-kittowski k., kolm, p. and mathiassen, l. (eds.), systems design for human and productivity participation and beyond, north-holland, amsterdam, 1987, pp. 285-301 kensing, f. and madsen, k.h., (1991), "generating visions: future workshops and metaphorical design", in greenbaum, j. and kyng, m. (eds.), cooperative design of computer systems, lawrence erlbaum associates, hillsdale, new jersey, pp. 155-168 ajis vol 5 no 1 september 1997 27 kensing, f. and winograd, r., (1991), "the language/action approach to design of computer-support for cooperative work: a preliminary study in work mapping", in stamper, r.k., kerola, p. and lyytinen, k. (eds.), collaborative work, social communications and information systems, elsevier science publishers b.v. (north-holland), amsterdam, pp. 311-331 klein, h.k. and truex, d., (1995), "discourse analysis: a semiotic approach to the investigation of organizational emergence", in p.b. andersen and b. holmqvist, the semiotics of the workplace, walter degruyter. klein, h.k. and welke, r.j., (1982), "information systems", in huff, s. (ed.): asac, vol. 3, part 4, pp. 106116 kling, r., (1980) "the social analyses of computing", acm computing surveys, vol. 12, no. 1, pp. 61-110 kling, r., (1987), "defining the boundaries of computing across complex organizations", in boland, rj. and hirschheim, r.a. (eds.): critical issues in information systems research, john wiley & sons, chichester, pp. 307-362 kling, r. and lacono, s., (1984), "the control of information systems developments after implementation", communications of the acm, vol. 27, no. 12, pp. 1218-1226 kling, r. and lacono, s., (1989), "the institutional character of computerized information systems", office: technology and people, vol. 5, no. 1, pp. 7-28 kling, r and scacchi, w., (1980), "computing as social action: the social dynamics of computing in complex organizations", in yovits, m.c. (ed.): advances in computers, vol. 19, pp. 249-327 kling, r and scacchi, w., (1982), "the web of computing: computer technology as social organization", in yovits, m.c. (ed.): advances in computers, vol. 21, pp. 1-90 kraft, p., (1977), programmers and managers, springer, new york kraft, p. and bansler, j.p., (1994), "the collective resource approach: the scandinavian experience", scandinavian journal of information systems, vol. 6, no. 1, pp. 71-84 kuhn, s. and muller, m.j., (1993), "participatory design", communications of the acm, vol. 36, no. 4, pp. 24-28 kumar, k. and bjorn-andersen, n., (1990), "a cross cultural comparison of is designer values", communications of the acm, vol. 33, no. 5, pp. 528-538 kumar, k. and welke, r., (1984), "implementation failure and system developer values: assumptions, truisms and empirical evidence", in maggi, l., king, j. and kraemer, k. (eds.): proceedings of the fifth internationa] conference on information systems, tucson, pp. 1-13 kyng, m., (1989), "designing for a dollar a day", office: technology and people, vol. 4, no. 2, pp. 157-170 kyng, m., (1991), "designing for cooperation: cooperating in design", communications of the acm, vol. 34, no. 12, pp. 65-73 kyng, m., (1994), "collective resources meets puritanism", scandinavian journal of information systems, vol. 6, no. 1, pp. 85-96 kyng, m. and mathiassen, l., (1982), "systems development and trade union activities", in bjorn-andersen, n. (ed.), information society, for richer, for poorer, north-holland, amsterdam lanzara, g. and mathiassen, l., (1985), "mapping situations within a systems development project", information & management, vol. 8, pp. 3-20 lehtinen, e. and lyytinen, k., (1986), "action based model of information system", information systems, vol. 11, no. 4, pp. 299-317 ljungberg, j. and holm, p. (1996), "speech acts on trial", scandinavian journal of information systems, vol. 8, no. 1, pp. 29-52 lyytinen, k. and hirschheim, r., (1988), "information systems as rational discourse: an application of habermas' theory of communicative action", scandinavian journal of management, vol. 4, no. 1/2, pp. 19-30 lyytinen, k. and klein, h., (1985), "the critical theory of jurgen habermas as a basis for a theory of information systems", in mumford, e., hirschheim, r., fitzgerald, g. and wood-harper, a.t. (eds.): research methods in information systems, elsevier science publishers b.v. (north-holland), amsterdam, pp. 219236 lyytinen, k., (1986), information systems development as social action: framework and critical implications, jyvaskyla studies in computer science, economics and statistics 8, jyvaskyla lyytinen, k., (1987), 'two views of information modeling", information & management, vol. 12, no. 1, pp. 9-19 madsen, k.h., (1989), "breakthrough by breakdown: metaphors and structured domains", in klein, h.k. and kumar, k. (eds.), systems development for human progress, north-holland, amsterdam, 1989, pp. 4153 ajis vol 5 no 1 september 1997 28 mathiassen, l. (1988) oral presentation in the panel on "information systems development methodology evaluation" in the ninth international conference on information systems, minneapolis, minnesota monarchi, d.e. and puhr, g.i., (1992), "a research typology for object-oriented analysis and design", communications of the acm, vol. 35, no. 9, pp. 35-47 mumford, e., (1983), designing human systems for new technology: the ethics method, manchester business school, manchester mumford, e., hirschheim, r., fitzgerald, g. and wood-harper, a.t. (eds.), (1985), research methods in information systems, north-holland, amsterdam nielsen, p.a., (1989), "reflections on development methods for information systems: a set of distinctions between methods", office: technology and people, vol. 5, no. 2, pp. 81-104 nielsen, p. (1995) "action research into participative is development", working paper, aalborg university, department of computer science, denmark nissen, h.-e., klein, h.k. and hirschheim, r. (eds.), (1991), information systems research: contemporary approaches & emergent traditions, north-holland, amsterdam nygaard, k. and bergo, o.t., (1974), "the trade unions new users of research", personnel review, vol. 4, no. 2, pp. 5-10 orlikowski w.j. and baroudi, j.j., (1991), "studying information technology in organizations: research approaches and assumptions", information systems research, vol. 2, no. 1, pp. 1-28 parnas, d.l. and clements, p.c., (1986), "a rational design process: how and why to fake it?", in ehring, h. et ah (eds.), formal methods and software development, springer-verlag, berlin paulk, m.c., curtis, b., chrissis, m.b. and weber, c.v. (1995), "capability maturity model, version 1.1", ieee software, july, pp. 18-27 sandberg, a.. (1983), "trade union-oriented research for democratization of planning in work life problems and potentials". journal of occupational behavior, vol. 4, no. 1, pp. 59-71 sandberg, a., (1985), "socio-technical design, trade union strategies and action research", in mumford, e., hirschheim, r.a., fitzgerald, g. and wood-harper, a.t. (eds.): research methods in information systems, north-holland, amsterdam, pp. 79-92 schon, d., (1983) the reflective practitioner: how professionals think in action, temple smith, london schafer, g. et al., (1988), functional analysis of office requirements: a multiperspective approach, john wiley & sons, chichester searle, j.r., (1969), speech acts, an essay in the philosophy of language, cambridge university press, cambridge searle, j.r., (1979), expression and meaning, studies in the theory of speech acts, cambridge university press, cambridge searle, j.r. and vanderveken, d., (1985), foundations of elocutionary logic, cambridge university press, cambridge unhelkas, b. and mandapur, g. (1995), "practical aspects of using a methodology: a road map approach", report of object analysis & design, vol. 2, no. 2, pp. 34-36, 54 vessey, i., jarvenpaa, s. and tractinsky, n. (1992), "evaluation of vendor products: case tools as methodology companions", communications of the acm, vol. 35, no. 4, 1992, pp. 90-105 von bulow, i., (1989), "the bounding of a problem situation and the concept of a system's boundary in soft systems methodology", journal of applied systems analysis, vol. 16, pp. 35-41 westrup, c. (1993), "information systems methodologies in use", journal of information technology, vol.8, pp. 267-275 whitley, r. (1984), the intellectual and social organization of the sciences, claredon press, oxford, uk whitley, r. (1985), the transformation of business finance into financial economics: the roles of academic expansion and changes in u.s. capital markets, working paper, manchester business school, manchester wilson, b., (1984), systems: concepts, methodologies, and applications, john wiley & sons, chichester winograd, t., (1987/1988), "a language/action perspective on the design of computer supported cooperative work", human-computer interaction, vol. 3, no. 1, pp. 3-30 winograd, t. and flores, f., (1986), understanding computers and cognition: a new foundation for design, ablex, norwood, nj wood-harper, a.t., antill, l. and avison, d.e., (1985), information systems definition: the multiview approach, black well scientific publications, oxford wynekoop, j.l. and russo, n. (1993), "systems development methodologies: unanswered questions and the research-practice gap", in degross, j.i., bostrom, r. and robey, d. (eds.), proceedings of the fourteenth international conference on information systems, orlando, fl, pp. 181-190 ajis vol 5 no 1 september 1997 29 wynekoop, j.l. and russo, n. (1997), "studying systems development methodologies: an examination of research methods", information systems journal, vol.7, no.l, january, pp.47-65. ajis vol 5 no 1 september 1997 the australasian journal of information systems (ajis) began in 1993 and was based on the simple realisation that there was a need for an academic ‘outlet’ for is research in australia australasian journal of information systems volume 18 number 1 2013 volume 18 number 1 2013 australasian journal of information systems ajis issn 1449-8618 australasian journal of information systems volume 18 number1 2013 1 australasian journal of information systems (issn 1449-8618) published by australasian chapter of the association for information systems (aais) website hosted by the australian computer society (acs) management board dr deborah bunker, the university of sydney (acphis president) dr don kerr, university of the sunshine coast (aais president) dr mark toleman, university of southern queensland (acphis past president) editor-in-chief dr kathy lynch, formerly of the university for the sunshine coast senior editors 18(1)) dr john campbell, university of canberra dr jon heales, university of queensland dr don kerr, university of the sunshine coast dr kathy lynch, university of the sunshine coast dr bruce rowlands, griffith university reviewers 18(1) mustafa appas ally, peter busch, john campbell, julie fisher, gerald goh, kavitha gurusamy, jon heales, don kerr, john lamp, kathy lynch, claire matthews, eric pardede, charles palmer, graham purvan, sherre roy, jim underwood previous editor-in-chief dr rob macgregor, university of wollongong (founding editor: 1993 2006) dr craig mcdonald, university of queensland (2006-2009) dr jon heales, university of queensland (2009-2012) submissions and subscriptions submissions need to make scholarly and/or research contributions. instructions to authors can be found on the journal’s website: http://dl.acs.org.au/index.php/ajis/about/submissions#authorguidelines. the journal is free to access online, however, please contact the assistant to the editor-in-chief for information regarding purchase of hard copies of the journal. enquiries and comments editorial enquires should be directed to ajis editor-in-chief (ajis.eic@gmail.com) subscription and general enquiries should be directed to dr sherre roy, assistant to the editor-in-chief, university of the sunshine coast, queensland. sroy2@usc.edu.au paper title (paper title style) australasian journal of information systems motahar, mukhtar, safie, ma’arif & mostafavi 2018, vol 22, research on educational technologies towards a product independent erp training model 1 towards a product independent erp training model: an insight from a literature review seyed mohammad motahar national university of malaysia (ukm) united nations university international institute for global health (unu-iigh) inventiontime@gmail.com muriati mukhtar national university of malaysia (ukm) nurhizam safie united nations university international institute for global health (unu-iigh) muhamad yusnorizam ma’arif national university of malaysia (ukm) shima mostafavi north metropolitan tafe, western australia abstract erp can be called as one of the most significant contributions of computer science and particularly information systems for the business environment which can support all complicated business process of an organisation including sale, purchase, warehouse and manufacturing into a single database. for such complex system like erp, implementation involves much collaboration which raises the concern of the success of the project. among most significant success and failure factors of erp implementation cited in the literature are training and consultation. for many years the majority of erp vendors promote their product specific training models as long-term and expensive allowing the organisation to gain knowledge of the erp system and the way it handles the business process. however, recent studies showed that erp training can be conducted in the short term and with minimum investment and yet to be effective. findings triggered a signal and opened a new door for research to investigate vendor-independent training models that require limited investment in training and consultancy. this paper aims to provide an overview of the literature; highlighting the research methods and theories used by authors towards building erp training models and methods. keywords: erp, erp training, erp teaching, erp learning 1 introduction among most important success and failure factors of erp implementation cited by the literature are training and consultation (noudoostbeni, yasin, & jenatabadi, 2009a) which allows the organisation to gain knowledge of the erp system and how it handles the business process. this information is especially needed since during implementation, there need to be a mapping and alignment of business process with features of erp systems which need proper understanding of erp systems. therefore training is among the most critical success factor identified by researchers and is a major phase in implementation procedure of erp systems (kale & kale, 2000) (sumner, 1999). despite the critical role of erp training, there are limited australasian journal of information systems motahar, mukhtar, safie, ma’arif & mostafavi 2018, vol 22, research on educational technologies towards a product independent erp training model 2 studies attempted to go further by conducting experiments to investigate erp training models and methods that are independent of the vendor and their product. results of our preliminary study showed that majority of the research articles in the area of erp systems training are product specific and top vendors like sap®, oracle® and microsoft® are promoting their erp certification programs following their erp training models and methods in the market. it is evident that an adequate erp training should not only focus on theoretical aspects. it should be supported by hands-on exercise to make sure participants have gained practical skills to use a product. however, such training programs may not be seen as a viable option for many companies due to two simple reasons, cost and time. majority of erp vendors suggest a long-term continuous training programs for their erp packages (s. koh, a. gunasekaran, & j. cooper, 2009). an example is sap that suggests “continuous process of lifelong learning” (noudoostbeni, ismail, jenatabadi, & yasin, 2010) and past research in the area of erp systems and erp training seem to accept that as a fact. majority of the research was done following such long-term training models and methods recommended by vendors. in addition to the time, the cost is another factor since such training classes provided by vendors are expensive (bradford, vijayaraman, & chandra, 2003). erp just like any other enterprise systems are costly, and it is estimated that 50 75% of enterprise systems implementation failed to justify the huge investment in the project. literature reported that training related factors are the main contributors to the failure of the project. (c. arasanmi, wang, singh, & ekundayo, 2011). therefore training is reported to be one of the expensive elements of the erp implementations and vendors recommended allocating extra budget to cover the expensive cost of training (josé manuel esteves, carvalho, & santos, 2001). there is no doubt that erp training models and methods provided by top vendors follow their business strategy. this strategy pushes learners toward certification programs that are expensive and is becoming one of their main revenue streams. for example, a wide range of courses and e-learning materials are available and provided by sap, and it is unsurprising that about 11% of the company income is the revenue coming from the training programs (aldrich, 2000; montandon & zentriegen, 2003). such expensive and “continues the process of lifelong learning” is subject to criticisms and concern from experienced professionals where training models supported by certifications may not be appropriate and necessary for all candidates. hitchcock cited critics from a top rank forum contributor who criticised such certifications to be expensive for three reasons: “to ensure that one invests so much they will never recommend any other product; to enable certification vendors to claim large numbers of certified professionals supporting their products; and the money it brings” (hitchcock, 2005). despite the fact that many research articles in the erp area are addressing a comprehensive and ongoing training cycle for erp adoption, findings from a study conducted by (s. koh et al., 2009) on a training model devised by an erp vendor reveal that erp systems can be implemented by end users by attending a very short course instead of the external consultancies. the training model of the case company was not discussed in koh’s article (2009) and the main objective of koh’s work was to check the validity of the claim. australasian journal of information systems motahar, mukhtar, safie, ma’arif & mostafavi 2018, vol 22, research on educational technologies towards a product independent erp training model 3 therefore, our study aims to continue this work by reviewing the literature in the area of erp training and to represent the current status of knowledge as well as to investigate the research methods and theories used by authors towards a ‘product independent’ erp training model. the following sections of this paper is organized as follows: research methodology section introduces the approaches used to collect data from the sources, the findings section, provides a detailed analysis of the articles included in the process of the systematic literature review and provides the result in a data table. the discussion provides a detailed description of the articles collected and their contributions toward the topic. last but not least, the conclusion section summarises the gaps in the literature that needs more attention from researchers. 2 research methodology in our research, we follow systematic review approach, an effective method of conducting a literature review (bryman, 2012). this method involves a review of the literature using online databases in step by step procedure. we started by using “erp teaching” and “erp training” as the initial keywords and applied a search filter to show results containing the keywords in the title of the paper. the industry makes most of the contributions in the area of erp training. therefore, the initial search was done on google scholar® with the aim of capturing academic papers contributed by industry. also, keywords and filters were applied in two major academic databases, scopus and web of science (wos) to capture top-tier papers in journals and conferences. the authors carefully reviewed all the results by scanning the abstracts and excluded irrelevant results from above three sources. in addition to that, to have better coverage in the area of study, references for selected papers have been analysed to identify possible relevant sources to add to the repository. as a result, authors convinced that majority of the papers in scopus and google generated scholar are industry relevant and in most cases they just examined the training models and methods that top erp vendors provided in their training channels. since the objective of our work is to focus on vendor-independent training models and methods, we decided to exclude google scholar and scopus from our literature review. this is done in order to ensure that an academic foundation and high quality works can be captured within a proper academic method. therefore, the optimised keywords used to search the isi web of knowledge®, and at the end, the repository was analysed, and findings of the results are discussed in the next section. figure 1 demonstrates the process of extracting journals from the source. australasian journal of information systems motahar, mukhtar, safie, ma’arif & mostafavi 2018, vol 22, research on educational technologies towards a product independent erp training model 4 figure 1 the process of extracting journals from the sources 3 findings the papers were analysed, and their full text had been extracted from isi web of knowledge®. there were few papers excluded from the discussion like (shea & khalil, 2004) which we could not find the abstract from the publisher’s website. there was also a paper that has been found to be in violation of ieee's publication principles (jing, hua, & ieee, 2008) and therefore its text was not analysed in our review process. table 1 provides a detailed analysis of the articles included in the process of the systematic literature review. in general, as shown in figure 2 six different themes emerged from the review that includes erp training models, importance of erp training, usability of erp training, erp training and user satisfaction, erp training evaluations. australasian journal of information systems motahar, mukhtar, safie, ma’arif & mostafavi 2018, vol 22, research on educational technologies towards a product independent erp training model 5 figure 2 erp training research themes erp training erp training models importance of erp trainig usablity of erp training erp training and user satisfaction erp training evaluations australasian journal of information systems motahar, mukhtar & safie 2018, vol 22, research on educational technologies towards a product independent erp training model 6 table 1 data table of the articles reviewed by categories australasian journal of information systems motahar, mukhtar & safie 2018, vol 22, research on educational technologies towards a product independent erp training model 7 4 discussions in this section, a detailed description of the articles collected is provided. the objective is to provide a summary of the findings of the papers in regards to erp training and their contribution towards the topic. in a recent study conducted in taiwan, shih studied the effects of innovation management (im) and vocational training on the corporate operating performance of smes in taiwan after implementing erp systems. the findings showed the fulfilment of operation performance during erp implementation stages, and employees addressed the importance of implementation. (lin et al., 2013). wang made another attempt in china (wang et al., 2012), with the objective of strengthening the training course for "erp talents" and by viewing the system from a staff perspective. an erp simulation system has been designed in the study. the simulation system that has been designed by the author has two parts, operational part for students and administration part for managers. the objective is to separate the business modules from management related modules. so the students can run simulated operations using sub-modules within the operational part and managers can use the sub-modules of the administration part to run management operations. the system has been designed using advanced java technologies, including structs2 and spring and hibernate for data access layer which allows the system to be scalable. despite the fact that many research papers in the erp area are addressing a comprehensive and ongoing training cycle for erp adoption, findings from a study conducted by koh on a training model devised by an erp vendor revealed that end users can implement erp systems for smes by attending a very short course. this short training course provided by an erp vendor in this study was only five days and consists of three days technical and two days administration courses. sme-specific erp vendor studied as a case in this research claimed that their erp system could be implemented and run in production by users with a very limited investment in the training course. as koh mentioned in his work, this area of research is novel, and the result of such research will be very useful for many smes seeking to minimise investment on training and consultancy (s. c. l. koh et al., 2009). koh’s paper (s. c. l. koh et al., 2009) would have been much more useful if he included the main elements of the model itself. the training model of the case company is not discussed (it may be due to the confidentiality requested by the vendor), and the main objective of koh’s work was to check the validity of the claim. another objective of koh’s work was to identify key factors that influence the success of the model. he conducted in-depth interview and analysis feedbacks and experience of the clients of the case company to form a theoretical framework. koh’s finding suggests that simplified system design is one of the key factors that brings ease of use and allows adoption of the system with minimum effort of business process engineering (bpr). users who have prior experience of working with erp systems is another key factor that could help end users to understand the process better and work with the system. however, highly experienced users who are responsible for implementing the software in the customer environment are the major key element that influenced the success of the model. australasian journal of information systems motahar, mukhtar & safie 2018, vol 22, research on educational technologies towards a product independent erp training model 8 it seems that the support environment was another contributor to the success of the training model. by studying in the case company, koh identified three major tools that were used by the vendor to support clients including a test database and a training compact disc (cd) supplied, an extensive release notes document as well as support via telephone. koh's work triggered a signal and subsequently opened a new door for research to investigate training models that are short-term yet effective. the research also aims to allow power users to do the implementation themselves with minimum investment in training and consultancy. in malaysia, a research study was done to investigate the different training methods of erp in sme sectors in malaysia (noudoostbeni et al., 2009b). table 2 shows the summary of training methods identified by him. by studying malaysian smes, he concluded that training is referred to as the most important critical success factor in erp implementation. the study then investigates further by evaluating the effectiveness of different training methods from the end user’s perspective. his findings show that lecturer and on the job training and computer based training are among the most used training methods in malaysian smes. figure 3 training methods of erp in sme sectors in malaysia (noudoostbeni et al., 2009b) in another study done by mozaffar, she refers to the findings from gartner and other sources that indicate the importance of the training in erp implementation (mozaffar & mozaffar, 2009). then by introducing the impact of business process and its role in new techniques like 6sigma, lean manufacturing, jit, she argues that new enterprise systems (i.e. erp, crm, scm, etc.) are evolving to respond to this change yet user training needs to be changed from a taskoriented approach to a process-oriented approach to allow end user to make use of the system in a more effective way that can contribute in 2010, albadri published a research paper in which they described the adoption of a new training model called the end-users characterization and competency building approach (euccb) (albadri & abdallah, 2009). it was designed based on the existing theories of australasian journal of information systems motahar, mukhtar & safie 2018, vol 22, research on educational technologies towards a product independent erp training model 9 “training management cycle” and “iceberg competency model”. the objective of the model was to improve the healthy level of erp implementation, and the model was designed to be pluggable to the erp implementation life cycle. the focus of the training model was to solve the erp end-users’ system acceptance in the post implementation phase of the erp project to ensure the usage effectiveness of the erp during the life cycle of the system. this is reported by the author to be around 10-50 years. albadri & abdallah described the model as a series of relevant guidelines and procedures organised around four main components. figure.4 shows the main elements of the euccb model. figure.4 main elements of the euccb model using the case study’s research method, the described approach was adopted during the erp implementation project in one of the major companies in the field of oil and gas in the united arab emirates. the result of the survey reveals that there is an increased end-user acceptance rate. it was reported that 60% of end users participated in the survey were satisfied and found the training useful. furthermore, during the knowledge test conducted by researchers, a group of end users who attended the training programs conducted as part of the (euccb) achieved a higher result. aristomenis has reported the first study of adopting the semantic web concept into the development of erp training material in 2009 (macris, 2009). in the paper, the author proposed a model to develop training materials for the erp systems. by means of the case study method the usefulness of the new prototype model, a training scenario was chosen to develop an erp training material for end users has been illustrated. the training scenario is focused on the meaning of the general ledger (gl) in the microsoft dynamics erp package. figure 5 demonstrates general ledger transactions scenario training aid based on semantic web principles. in the case study of the scenario, two main components are created: 1. transaction scenario for gl account 2. user interface of the scenario gl account training & change management strategy training cycle processes: identification, design, delivery, application and evaluation training programming: planning & management training & evaluation tools & methods australasian journal of information systems motahar, mukhtar & safie 2018, vol 22, research on educational technologies towards a product independent erp training model 10 the author argues that the method which is based on the semantic web is focusing on capturing the existing knowledge. this method is used to represent it as ontology-based knowledge network which can provide additional benefits such as: 1. semantic search 2. knowledge navigation, and 3. knowledge dissemination the author claims that the model is designed to be used within existing methodologies to enhance the user satisfaction. aristomenis claims that the method is not only applicable in the erp area, but also in a wider range of training. "this method is also highly transferable to the development of training materials for any kind of systems and users and can produce similar gains in learning and understanding in any domain of knowledge (macris, 2009)" figure 5 general ledger (gl) transactions scenario training aid based on semantic web principles due to the novelty of the method, it may require further research to validate with more erp process scenarios including more advanced workflows and different erp implementation packages. another study was done to investigate the application of general system theory (gst) by mahadevan (mahadevan, agbinya, braun, & ieee, 2006). in this extensive research, a new strategic framework has been suggested, and the authors highlighted the role of gst in erp training. a logistics process which is among the challenging operational areas in any erp implementation has been used, and the three different usability aspects of the erp systems have been demonstrated in developing a business logistics management process. the authors conclude that “by using ahp method there will be an opportunity for a tc business system with formal usability trade-offs”. figure 6 shows three different aspects of erp usability used by mahadevan australasian journal of information systems motahar, mukhtar & safie 2018, vol 22, research on educational technologies towards a product independent erp training model 11 figure 6 three different aspects of erp usability used by mahadevan (mahadevan et al., 2006) another study was done by lee to investigate the impact of training satisfaction on the perceptions of ease of use in erp implementation (lee & bradley, 2004). qualitative data collection was done in the implementation of erp in a higher education context. results from the research identified the fact that there is a relationship between the job type and gender of trainees with the satisfaction of the training. with the objective to overcome the lack of erp training in erp implementations, dr ip has applied the system dynamic theory in the training process of erp systems (ip et al., 2004). in the case study conducted by ip, an erp training model has been proposed to simplify the training process in the manufacturing context. figure 7 shows this erp training model (lee & bradley, 2004) figure 7 ip training model (lee & bradley, 2004) the model consists of three main elements that were incorporated into a website. generic education provides a basic understanding of erp and application training focusing on erp system itself and how the system can allow performing functions. within the constructive australasian journal of information systems motahar, mukhtar & safie 2018, vol 22, research on educational technologies towards a product independent erp training model 12 problem-solving part of the training is a series of problems that are in fact a day to day problems in the operations. the model has been integrated with the web-based training system, and the result has been used as a framework for training in the case study company, namely a lamp manufacturer. the main objective of the system dynamic is to help scientists to understand objects in a system interacting with each other and not as an isolated object. the theory has been applied to solve many problems in different context ranging from a steam engine to a bank account or even a basketball team (forrester, 1995) (nelson & somers, 2001). results from the study revealed benefits and usefulness of the model in the erp context that can contribute to the success of erp implementation. the effect of computer self-efficacy and training on intention to use erp has been investigated by fan (fan et al., 2004). the study is based on the technology acceptance model, and it examined the driving factors for end users to work with the erp system. using a survey instrument, a total of 202 responses were received from end users working with ten different companies. the result then analysed and author concluded that conducting an effective erp training has a positive impact on perceived ease of use of erp system for end user. in addition to that effective erp training has an indirect positive impact on perceived usefulness, attitude, and intention to use erp system for the end user (fan et al., 2004). shtub in 2001 proposed a model of teaching and training of erp systems (shtub, 2001). having a good understanding of the erp systems for the managers and increasing inefficiency of training courses were the main objectives to design a new erp training and teaching model. shtub argue that erp products are focusing on automating a business process and a complete process of sales can just go beyond sales module or a warehouse stock taking process can ultimately post into journals for automatic stock valuation and therefore the actual process within organization is supported by more than one module whereas erp training course tend to focus on modules without considering the complete process. shtub’s new erp development framework consist of a build in interactive tool and a methodology to cover three main areas: 1. the traditional management of operations in a functional organisation. 2. the integrated and dynamic approach to operations 3. the use of advanced information systems (of the erp type) to support the management of operations (shtub, 2001) figure 8 demonstrates the operation trainer (ot )method as a teaching aid that integrates the case study and the modelling. australasian journal of information systems motahar, mukhtar & safie 2018, vol 22, research on educational technologies towards a product independent erp training model 13 figure 8 ot is a teaching aid that integrates the case study and the modelling (shtub, 2001) the new ot model was not only used in the industry training but also in university settings. the framework was applied to teach erp implementation to managers. results from the case studies demonstrate the effectiveness of the new erp training model and its potential to be applied to teach complex process within erp systems (shtub, 2001). in a project funded by united nations university (unu) motahar proposed a new applied approach to teach erp implementation using open source erp framework. (motahar, safie, mukhtar, aljunid, & mostafavi, 2013). motahar compiled a live erp system and compiled as part of the course materials in pen drives and distributed to audience. participants were asked to configure their own erp package in every stage of the training by referring to the live and pre-configured version as a working example. at the end of the training course, the results from the training shows that end users can understand the complex business process and apply their knowledge by configuring a test instance and compare it along the way with a live and working version to validate their configuration. 5 conclusion as a summary, findings from the literature shows that there are various methods and models examined by researchers for erp training that are successful and promising. the results of applying those methods reveal that training can still be successful without following vendors training models that recommend “continues life long learning” which is very expensive and not affordable especially for small and midsized enterprise (sme) and findings triggered a signal and opened a new door for research to investigate vendor-independent training models that require limited investment in training and consultancy. australasian journal of information systems motahar, mukhtar & safie 2018, vol 22, research on educational technologies towards a product independent erp training model 14 this article aimed to provide an insight from the literature review to highlight the research methods and theories that can be used towards building erp training models and methods that are product independent. based on our findings and discussion of the literature review of erp training, we identified the following gaps in the literature that need more attention from researchers. although many papers presented training models and explored training methods, training methods and approaches suffer from a limitation regarding the targeted audience. there is a group of participants for erp training. they are consultants and implementers. they are neither students who are lacking the knowledge of business process, nor end users who are working on a day to day basis with erp package. there are limited training model or method found in the literature to investigate and address their needs. in addition, although many papers presented training models for end users, most papers assumed that implementation is done alongside the training from the vendor. there is limited research to address a training approach that can allow end user not only to use the system but to be able to implement and maintain it. a training model that can cover the implementation procedure for “power users” needs more attention and requires further research and investigation. open source erp systems are becoming popular these days, especially for smes which have limited budgets to spend on expensive proprietary systems. there is a lack of research in the area of training for open source erp systems. none of the above training models used any open source erp systems and most of them are following the materials and methods supported by property erp providers through their academic alliance programs (e.g. sap academic alliance ®) in the industry and to conduct training, there are two important groups, particularly end users and power users. companies may prefer to have the training to be conducted for power users also regarded as key users in the organisation who have a good understanding of the whole business process in departments and an overall understanding of the functionality of the erp system that has been implemented. the mechanism is to focus on the key user to transfer the knowledge and then the key user will conduct the internal training for the end users. this important fact which is a common practice in small and many mid-sized companies has not been investigated by researchers yet, and there is no model to address training methods to plan for such group. there is only one research stating that there is a possibility to conduct an erp training to the end users so that they can implement the system without consulting. they validated the statement. however, they did not discuss the elements of the model. it is recommended that researchers investigate this area further for future outlook. erp training is not only an important area of research in the erp systems for academicians, but it also plays an important role in the erp implementation and is a critical success factor of million-dollar investments in the erp system. likewise its result has great potential to be applied in industry. therefore, the present paper provides an initial literature review of erp training to shed the lights on the past and present. it is suggested for researchers to investigate this topic further to address the identified research gaps and to contribute to the future direction of erp training. australasian journal of information systems motahar, mukhtar & safie 2018, vol 22, research on educational technologies towards a product independent erp training model 15 references albadri, f. a., & abdallah, s. (2009). erp training and evaluation: erp life-cycle approach to end-users' characterization and competency building in the context of an oil & gas company. norristown: int business information management assoc-ibima. aldrich, c. (2000). customer-focused e-learning: the drivers. training & development, 54(8), 34-34. ali, a. h. m. (2014). ict facts and figures. retrieved from www.mis.iran.ir arasanmi, chris n.; wang, william y.c.; singh, harminder; and ekundayo, samuel, "what motivate end-users to transfer gained knowledge from enterprise systems training?" (2011). pacis 2011 proceedings. 17. https://aisel.aisnet.org/pacis2011/17 arasanmi, c. n., wang, w. y. c., & singh, h. (2013). the influence of system interface, training content and it trust on erp learning utilization: a research proposal. in l. uden, f. herrera, j. b. perez, & j. m. c. rodriguea (eds.), 7th international conference on knowledge management in organizations: service and cloud computing (vol. 172, pp. 153161). berlin: springer-verlag berlin. bradford, m., vijayaraman, b., & chandra, a. (2003). the status of erp integration in business school curricula: results of a survey of business schools. communications of the association for information systems, 12(1), 26. bryman, a. (2012). social research methods: oxford university press. dalveren, y. (2014). using e-learning in enterprise resource planning (erp) training: a case study to assist curriculum designers in turkey. 5th world conference on educational sciences, 116, 1353-1357. doi:10.1016/j.sbspro.2014.01.396 esteves, j. m. (2014). an empirical identification and categorisation of training best practices for erp implementation projects. enterprise information systems, 8(6), 665-683. doi:10.1080/17517575.2013.771411 esteves, j. m., carvalho, j., & santos, a. a. (2002). towards an erp life-cycle costs model. in n. fiona fui-hoon (ed.), enterprise resource planning: solutions and management (pp. 5162). hershey, pa, usa: igi global. fan, j. c. j., guh, w. y., chou, c. h., ku, w. l., & sher, p. j. (2004). the effect of computer selfefficacy and training on intention to use erp. hangzhou: zhejiang univ press. forrester, j. w. (1995). the beginning of system dynamics. mckinsey quarterly, 4-17. hitchcock, l. (2005). industry certification: value, validity, and a place for sodis®. acm sigcse bulletin, 37(4), 59-63. huang, w. f., yu, z. h., & liu, y. c. (2013). analysis on the integrated teaching mode of erp course based on cdio. newark: information engineering research inst, usa. ip, w. h., lai, c. l., & lau, c. w. (2004). a web-based training model of enterprise resources planning for the manufacturing industry. international journal of engineering education, 20(5), 733-741. jing, y., hua, j., & ieee. (2008). research on model of teaching about erp platform simulation in mis education. new york: ieee. australasian journal of information systems motahar, mukhtar & safie 2018, vol 22, research on educational technologies towards a product independent erp training model 16 kale, v., & kale, v. (2000). implementing sap r/3: the guide for business and technology managers: sams. koh, s., gunasekaran, a., & cooper, j. (2009). the demand for training and consultancy investment in sme-specific erp systems implementation and operation. international journal of production economics, 122(1), 241-254. koh, s. c. l., gunasekaran, a., & cooper, j. r. (2009). the demand for training and consultancy investment in sme-specific erp systems implementation and operation. international journal of production economics, 122(1), 241-254. doi:10.1016/j.ijpe.2009.05.017 lee, c. c., & bradley, j. (2004). erp systems training at a university: a case study. hershey: idea group publishing. lin, s. h., lin, h. y., & kao, p. l. (2013). study on effects of innovation and vocational training on operation performance by implementation of erp. in j. zhang, z. j. wang, s. r. zhu, & x. m. meng (eds.), information technology applications in industry, pts 1-4 (vol. 263-266, pp. 3260-3264). stafa-zurich: trans tech publications ltd. linkedin. (2015). mustafa rangwala | linkedin . retrieved from https://in.linkedin.com/in/mustufa-rangwala-0a46777 macris, a. m. (2009). enterprise resource planning: an applications' training aid based on semantic web principles. in m. d. lytras, j. m. carroll, d. avison, a. dale, j. sipior, e. damiani, r. d. tennyson, a. naeve, p. lefrere, f. tan, & g. vossen (eds.), visioning and engineering the knowledge society: a web science perspective, proceedings (vol. 5736, pp. 366-375). berlin: springer-verlag berlin. mahadevan, v., agbinya, j., braun, r., & ieee. (2006). analyzing usability alternatives in multicriteria decision making during erp training. new york: ieee. mahapatra, r. k., & lai, v. s. (1998). intranet-based training facilitates erp system implementation: a case study. atlanta: association information systems. montandon, c., & zentriegen, m. (2003). applications of customer focused e-learning. informing science, 1227-1237. motahar, s. m., safie, n., mukhtar, m., aljunid, s., & mostafavi, s. (2013). an applied approach to teach hospital information systems development using an open source erp framework. procedia technology, 11, 1259-1265. mozaffar, h., & mozaffar, f. (2009). process-oriented user training for enterprise resource planning systems. los alamitos: ieee computer soc. nelson, k., & somers, t. (2001). exploring erp success from an end-user perspective. amcis 2001 proceedings, 206. noudoostbeni, a., ismail, n. a., jenatabadi, h. s., & yasin, n. m. (2010). an effective enduser knowledge concern training method in enterprise resource planning (erp) based on critical factors (cfs) in malaysian smes. international journal of business & management, 5(7). noudoostbeni, a., yasin, n. m., & jenatabadi, h. s. (2009a). a mixed method for training erp systems based on knowledge sharing in malaysian small and medium enterprise australasian journal of information systems motahar, mukhtar & safie 2018, vol 22, research on educational technologies towards a product independent erp training model 17 (smes). paper presented at the international conference on information management and engineering, 2009. icime'09. noudoostbeni, a., yasin, n. m., & jenatabadi, h. s. (2009b). to investigate the success and failure factors of erp implementation within malaysian small and medium enterprises. paper presented at the international conference on information management and engineering, 2009. icime'09. pantelic, o., pajic, a., & vujaklija, d. b. (2012). performance evaluation of erp systems training program. inted2012: international technology, education and development conference, 6068-6074. scott, j. e., & walczak, s. (2009). cognitive engagement with a multimedia erp training tool: assessing computer self-efficacy and technology acceptance. information & management, 46(4), 221-232. shea, t., & khalil, o. (2004). erp training effectiveness: a field experience from a higher education institution. hershey: idea group publishing. shtub, a. (2001). a framework for teaching and training in the enterprise resource planning (erp) era. international journal of production research, 39(3), 567-576. sumner, m. (1999). critical success factors in enterprise wide information management systems projects. paper presented at the proceedings of the 1999 acm sigcpr conference on computer personnel research. wang, y., li, m., & yang, x. j. (2012). the design and realization of erp system for staff training. in w. z. chen, x. p. xu, p. q. dai, & y. l. chen (eds.), advanced manufacturing technology, pts 1-4 (vol. 472-475, pp. 3425-3429). stafa-zurich: trans tech publications ltd. zhang, c. s., li, z. x., & chen, h. y. (2011). teaching reform of erp sand table for enhancing the entrepreneurial ability of college students. hong kong: etp-engineering technology press. copyright: © 2018 motahar, mukhtar, safie, ma’arif & mostafavi. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems motahar, mukhtar & safie 2018, vol 22, research on educational technologies towards a product independent erp training model 18 1 introduction 2 research methodology 3 findings 4 discussions 5 conclusion ajis vol. 5 no. 2 may 1998 a fresh look at integrated business applications neither package nor custom-made! andrea engelhard! bereich wirtschaftsinformatik i (department for information systems i) friedrich-alexander universitat, erlangen-nlimberg erlangen-niirnberg, germany email: engelhardta@acm.org abstract since integrated business applications demonstrate weaknesses, intermediate solutions positioned in between packages and custom-made software need to be explored. one suggestion for such an adaptable architecture uses a functionally stable core with peripheral applications. this architecture accounts for software market dynamics and interests of different parties involved. the ideas developed stem from a case study in the german market for small business software. the case describes the problems within this business configuration and presents a technical solution that implements a core application approach. the concept could turn out to be useful for other business networks, as first hypotheses hint. keywords software (cb), organizational characteristics (da), organizational functions (db), is management (e), information systems (h) current problems with integrated business applications businesses have been replacing in-house developed business applications with packaged software for the last three decades. at present, it is fair to state that average german application portfolios are composed of 55% custom-made software2 and 45% packages, with a continuing trend towards using more packages (bmft 1994). both types of business software hold some disadvantages, i.e.: • especially large integrated enterprise packages3 are often overloaded with functionality. an individual business might on the one hand only need certain parts of a package, but is nevertheless forced to buy it completely. on the other hand, the software might very well not fulfil special requirements. an abundance of functionality also contributes to a high degree of complexity, which then necessitates expensive expert advice to customise the system. the inflexibility of systems may cause the organisation to adapt to the package instead of the software adapting to the organisation. • individually developed software is expensive simply because there are no economies of scale, both during development as well as during maintenance. it has also been difficult to achieve a balanced skills portfolio within the it department, covering a broad range of fast evolving technologies. packaged and custom-made software form the boundaries of a research field (figure 1): how can the information systems discipline systemise intermediate solutions in between these two extremes in order to facilitate organisations achieving and maintaining a desired level of application integration? integration has been a keyword in information systems research. at its core integration means the unification of actions towards organisational goals. it is the role of business information systems to facilitate this. two aspects, which are instrumental in defining intermediate solutions, have been neglected in the integration discussion so far: (1) the fact that there is a dynamic software market. integration needs to be achieved over time using software from different vendors with changing product releases. (2) the fact that there are business networks whose is-strategies need to be supported. integration within the traditional "model enterprise" (becker/schutte 1996, mertens 1997, scheer 1994) seems to be a too limited perspective, especially since business form tends to change one enterprise might be absorbed by another or dispersed into smaller units. it is the objective of this research to provide a framework for integrated business information systems that accounts for software market dynamics and the requirements of business networks through a concept unifying the advantages of packaged and custom-made software. 2 this figure includes software embedded in products ("embedded systems"). 3 business software packages from vendors such as baan, jd edwards, peoplesoft, sap, etc. 22 ajis vol. 5 no. 2 may 1998 on 6 "intermediate solutions" custom-made business software "enterprise" packages \ (for any industry) j organisational fit figure 1: the research field between packaged and custom-made business software (adapted from (meyer 1993)) approach at first the research field displayed in figure 1 is dissected using three characteristics business size, industry, and integration technology. • differentiating by size leads to three segments large, medium-sized and small businesses. • trying to contrast the software needs of different industries introduces the idea of horizontal and vertical software markets. applications vary in their degree of industry neutrality. it is possible to identify core business systems, applicable in all industries; the so-called horizontal applications. vertical systems subsume applications that are typically used in a certain industry, such as manufacturing, retail, etc. • recent software technologies facilitate application architectures displaying better integration quality than possible with traditional technologies. secondly, an in-depth case study describes one specific business network, which centres on software requirements for small businesses being served by various suppliers. experiments with the various parties involved will assist in identifying strengths and weaknesses of the proposed solution. finally the concept is extended to other business networks sharing some of the features described in the case study. case: software for the german small business market germany's small business count reflects the typical picture in advanced economies most businesses are small and medium-sized. according to the official statistics4 (bundesamt 1987) there are some 2.1 million businesses in germany, 99.8 % of which employ less than 500 people and 94.7 % less than 20 people. abstracting from the diversity of small business, a few industry verticals are conceptualised. these verticals differ from those used for large companies, which ibm for example uses in order to focus its organisation (ibm 1995). a suggestion for seven small business verticals differentiates in construction, retail, hotel and food & beverage, craft, professional service, commercial service and health care. the representation in figure 2 implies that there are more similarities of software requirements inside a vertical than across. some business functions, however, cut across the vertical divisions. most notably, every business encompasses an accounting function. personnel administration is another example of a business function that can usually be found in different industry lines. mostly large software vendors offer solutions for these business functions, which can also be thought of as core systems. further business functions, such as inventory management or production planning and control, could also offer a large enough software market volume to allow for core system developers to prosper, even if their recent publications show quite different estimates concerning the total number of businesses. the last reliable count dates from 1987, from which one has to extrapolate and also gauge the effects of german reunification. 23 ajis vol. 5 no. 2 may 1998 core systems would not be useful in each industry vertical. financial accounting personnel administration inventory management production planning and control figure 2: small business verticals and some horizontal core business functions the term "core" is to imply lack of completeness. the core needs to be augmented by other applications, in order to fulfil all of the software requirements of a business within an industry vertical. the following case study presents a loose partnership between a large core system supplier and many small vendors complementing a core application to offer integrated solutions in a variety of industries. the role of the tax accountant as the provider of a core application: financials the majority of small german businesses have outsourced the accounting function to their tax accountant the tax accountant industry is strictly regulated by law and professional institutions much like other professions such as medicine, law or architecture. differing from most international practices, it is prohibited to run a tax accountancy as a commercial enterprise. the professional needs to apply his or her own expertise to every piece of work that leaves the office, so there is little room for delegation. this provision represents the main reason why tax accounting has been a small business industry in germany and why there are no large german accounting fums, e.g. as they have developed in the uk or the usa. the restrictive effects of regulation are offset by some favourable rules. the law stipulates a monopoly for certain types of work that solely a licensed tax accountant can perform. most importantly this monopoly includes bookkeeping. the german lawmaker reasons that the task of bookkeeping frames a tax declaration. therefore, in order to further the common good, this task needs to be performed by a trusted professional. this provision leaves small business with the choice of keeping their books themselves or outsourcing it to a tax accountant. for many small businesses, accounting does not merit employing someone full-time. thus a high level of outsourcing can be observed. the role of the tax accountants' data processing finn datev at the end of the sixties, tax accountants were faced with the challenge to process the monthly declarations of large number of clients for the newly introduced value-added tax. it was time to give the profession access to data procession capabilities. one tax accountant implemented the idea of a data processing centre commonly owned by the members of the profession. he founded datev, a company exclusively serving german tax accountants, in the legal form of a co-operative. datev has grown to be germany's largest provider5 of financial accounting solutions, handling the accounts of some two million business entities, which comes close to almost every small business. products range from entrylevel to full-fledged pc-accounting packages complementing traditional mainframe-based data centre services. laws restricting accountants' activities extend to datev. this means that datev is bound to which business 1996 sales of almost dm 1 billion, some 4.700 employees. 24 ajis vol. 5 no. 2 may 1998 functions its software offerings may cover. unless there is a dramatic change in the law, datev's solutions centre on accounting, personnel administration and taxes. this restriction to the accounting function is what makes the datev case especially interesting. other vendors specialising in small business software might rather opt for increasing the scope of their packages to include functionality needed in other business functions. the most obvious example might be a function for invoicing. however, concentration on a specific function may very well be a matter of strategic choice. integration technologies facilitate multi-vendor business software and allow software suppliers to develop and price competencies in a functional area. an example for this kind of strategic positioning is american software vendor coda (coda 1997). the datev case allows studying vendor-concentration on one business function without having to address the complexities of packages in the "big" business sector. datev has to face an increasingly competitive software market that caters for the need of the small business sector. some of the forces are the growing international presence of software vendors, offering personal finance packages that appeal to small business owners, as well as an increasing willingness of these businesses to try and use information technology. datev has reacted to these trends by offering pc-based solutions that accountants can provide to their respective clients. knowing that an accounting solution would only cover a portion of the potential software requirements, datev needs to offer seamless integration between its packages and the ones a client might use for other business functions, such as invoicing or inventory control. software solutions with datev financials at its core should ideally offer an equivalent quality as integrated packages sold by a single source. the role of small software vendors the software industry mirrors the structure of overall german business. there are some large, few medium-sized and many small software companies. some of the smaller software shops deliver business applications to their clients. their limited resources make it difficult for them to maintain and update their own solutions, while at the same time tending to their customers' needs. small software firms feel the competitive pressures even more than a large business entity such as datev. they are looking for strategies to offer complete solutions while keeping their advantage of flexibly reflecting individual customer wishes in their offerings. traditional file interfaces that datev published to enable third-party software to link up with datev's accounting solution, were widely accepted within the small business software market. a datev initiative to implement rich interfaces to their pc-based software packages based on newly available technology could experience a similar market reception and develop into a quasi-standard, since it would allow to small developers to quickly integrate an accounting part into their applications. the role of the client: the small business as software user the small business sector has gained attention, especially, since the studies of birch (birch 1987) in the us found that this segment contributes more to job growth than large business. small business is thought to adapt faster and to know their niche markets better. even if there are critical voices doubting the foundation of claims made by small business researchers (el-namaki 1990) special government programs geared towards these businesses are wide-spread world-wide6. current initiatives encourage more software use in general, and access to the internet in particular. the software market for the small business clientele is expected to grow. the generation change is one favourable influence, because the future leaders are thought to appreciate information technology more than their predecessors. another factor is the changing cost structure for information systems decreasing hardware prices allow for spending more on software and services. prototype business network configuration put together in one picture (figure 3), the four parties need to align their interest in order to achieve integrated solutions for small businesses. some features of this network can be found in other configurations. solutions suggested for this configuration might be useful for other configurations (discussed later in this paper). for an example refer to error! reference source not found. (02.08.1997), 5th framework programme for research, technological development and demonstration of the european union, section on "innovation and participation of smes". 25 ajis vol. 5 no. 2 may 1998 tax accountant datev small business software house figure 3: a prototype business network: configuration with tax accountant, datev, software house, and small business a semantically rich wrapper of a core application concept there are other technological approaches to reach intermediate solutions between packages and custom-made software. two examples are the template-approach (hofmann/rockart 1993), which uses case-tool-based software designs to adjust an application to specific requirements or "frame technology" (bassett 1996), where software consists of adaptable components. the ideas put forward in these approaches are helpful in rethinking the software "moulding" process. however, they address the problems of large, complex systems running within one organisation. the initiative of the open applications group (oag), a not for profit organisation of large software vendors, also addresses the problems of large, mostly globally operating companies. the oag has set out to deliver a standard that facilitates to link the business applications of various large producers. the oagis (oag integration specification) defines integration points where applications exchange business object documents, which execute program functions in the receiver application (oag 1995). an example of an implementation of the oagis are the sap bapis (business application programming interface) (sap 1997). the solution described here differs in that it addresses smaller applications bought by smaller clients from a number of vendors in regional market. the proposed technical solution caters for the requirements of a software market. coming from the case study just described the following "application architecture"7 is proposed. increasingly specific vt application wrapper end-user contribution figure 4: peripheral applications integrated with a core application having a functionally stable wrapper a functionally stable core application allows access to its inner workings by exposing a "wrapper". the wrapper's semantics stay stable over time, even if there are new releases of the application inside the wrapper, which then provides a frame of reference for independent software developers linking up with the core application. in this case a functionally stable wrapper is designed for an accounting package taking up the role of the core 7 software development is often likened to architecture. "software" or "application architecture" could thus refer to the design of one application or to a model for a whole business. both meanings of the word "architecture" use the analogy of designing a single structure; in this context the author would like to suggest to instead think of the work of a city planner, who would integrate several buildings. 26 ajis vol. 5 no. 2 may 1998 application. the provider of this core application has to decide which elements to expose through the wrapper. independent software developers supply "peripheral" applications. in this context "peripheral" does not mean less important, but rather expresses a higher degree of individualisation and adaptation to the requirements of a vertical or a specific client. next to supplying peripheral components, these independent software developers take up the role of component integrators. at this point in time it can only be speculated if there will be a "division of labour", where some software houses concentrate on supplying components and others specialise in integrating solutions built on them. the application architecture is further enriched, as soon as end-users develop enough skills to exploit the possibilities of modern office packages and complement their application portfolio with highly individualised elements. implementation since the users in the presented case study are small enterprises, the proposed solution builds on pc-software platforms. recently available technologies for application component integration open possibilities for effectively linking the strategies of a package vendor like datev with those of the developers of less widely distributed business applications. they allow exposing selected parts of an application package for outside access. these selected parts constitute the wrapper for the application within. ole-automation lends itself as an integration mechanism. the part of the accounting application that is to be accessible to independent developers is compiled as an ole-server, exposing its objects in a type library. an ole-controller can then display the information in an object catalogue, supporting the developer in his integration work. the interface description of the accounting application takes up the form of an ole-object model. seen from a developer's perspective, an object catalogue is available within the programming environment, which allows access to the elements of the object model in their application.8 for a core application as described above, it is important to expose a semantically stable object model. this means in particular that the naming of the classes as well as their methods and properties has to be consistent and easily understandable. it also needs to capture the functional essence of the core, which remains the same even if the implementation of the application might change, such as the move to a different data base system. for the core application in the case described above, this leads to defining a reference model for the accounting function (figure 5). a reference model can then serve as a standard should market participants accept it. strong market presence of the core application supplier helps to exert de facto standardisation power. bookkeeping-application 1 businesspartner businesspartners i businessevent businessevents application | company companies 1 ( 1 1 account accounts bankstatement reports bankstatements 1 cash cashlist bank banks payable payables receivable 1 receivables 1 inventory inventories figure 5: a reference model for the accounting function in a small business represented as ole object hierarchy testing the suitability of the approach has undergone a few testing phases. • internal development of a prototype 1. prototype peripheral application integrated with an entry-level accounting product for an in-depth discussion of ole (brockschmidt 1995). 27 ajis vol. 5 no. 2 may 1998 2. transferral of the prototype to demonstrate integration with larger products in the accounting product line 3. consolidation of the experiences into one accounting object model • external testing with selected partners 4. independent software houses 5. students the first step was to show the technical feasibility of the approach. this was accomplished by integrating a bookkeeping application at that time under development in datev, with some interfaces a third-party product might include. at first customer data of a prototype retailing application was exchanged with the bookkeeping application. secondly, the prototype sent invoices to the bookkeeping product. these two examples address the same issue as the business object documents of the oagis "synccustomer" and "loadreceivable" (oag 1997). both examples used the ole-automation interface of the bookkeeping application. this step led to some suggestions on how to change the design of the bookkeeping product in order to supply a better object model in order to facilitate programming work of external software producers. the graphical representation of the object model (similar to the one in figure 5) was used in negotiations with selected software houses supplying peripheral applications for certain industries. this phase is ongoing, and will ideally result in a few prototypes that expose the strengths and weaknesses of the approach, as well as a preliminary description of the co-operation process. this phase will be complemented by experiments in the university, where students take up the role of independent software suppliers integrating their "solutions" with the datev core application. these tests will help to establish the semantic stability of the wrapper, i.e. what should be exposed of an accounting application. integration within other business networks if one looks at the business network constituted by small businesses, small software houses, tax accountants, and datev, the question arises whether lessons learnt can be extended to other business configurations involving small businesses. the following paragraphs outline scenarios of how one could transfer the concept of a standardised, well-defined software core supplemented by peripheral applications. setting up case studies in order to research these scenarios will deepen the understanding what business functions could be supported by a standardised core packaged by a wrapper. cases to study this issue could be found in the following three configurations. two configurations are presented describing a market situation, since they involve fairly independent business units. then the concept is carried over to the hierarchy of a larger business, emulating a small business environment. configuration within markets two different industries are selected in order to demonstrate how the core application concept could be extended. automobile industry: car retailing there are different business models of how car manufacturers implement their retailing function. one configuration used in practice consists of four parties the car manufacturer, a retailing unit (owning a certain percentage of each dealership), the car dealers, and software companies, offering solutions (figure 6). the manufacturer is interested in receiving timely, accurate, and comparable data from their retail outlets, as well as in ensuring a similar quality of customer service. this is the reason, why this party tries to influence is used in dealerships. the retailing unit is interested in keeping the cost of is low. also they want a solution that fits the different characteristics of each car retailer. and retailers want to decide themselves what is to use as well as when to introduce and change it. software suppliers for these types of solutions tend to be larger and often provide systems to the manufacturer's it department. the competitiveness of offered solutions suffers from die fact that the circle of bidding software vendors is rather small. 28 ajis vol5no.2 may 1998 retailer holdii company car manufacturer car dealer jl software house figure 6: configuration within the car industry the involved parties could agree on a functionally lean core system that is complemented by independent vendors: the core in this case could contain next to a financial accounting solution cost accounting and inventory management functionality and a customer service component. the inventory part would allow the car dealer to download part information from the car manufacturer. the support for reordering would help to keep inventory to a minimum in the whole supply chain. a common cost accounting component permits bench marking of car dealers. the customer service component ensures comparable treatment of clients regardless where they bought their car. car dealers might opt to individualised peripheral components for invoicing, the administration of their repair activities and their payroll system. insurance industry: insurance agencies application architectures for the insurance industry have been under development for quite some time. 1aa (insurance application architecture) by ibm or insurance reference models by kpmg can be quoted. these architectures tend to concentrate on the insurance enterprise and deliver blue prints for the corporate is department how to proceed in constructing their specific is architecture. small agencies represent the point-of-sale of many insurers. theninformation systems need to be strengthened in order to exploit the advantages of an agency-based business model in the insurance industry, which is facing more competition e.g. from banks selling insurance products or direct insurers renouncing on an expensive agent network. the parties involved in this configuration (figure 7) are the insurance company, the is department of the insurance, the agencies and independent software suppliers. again a core system provided by central is used in all insurance agencies could ensure consistency and maintainability. the agencies achieve individualisation by enriching the raw solution by hiring smaller software houses. insurance company central is agency software house figure 7: configuration within the insurance industry in this case the core would contain an accounting solution, a contract management component, damage administration, as well as a terms and conditions application. these are parts of the application portfolio of agencies that have links to the insurance company and that need to be functionally stable as well as up-to-date in each agency. individualised parts could be personnel administration, which only agencies with several producers would need or an extended customer database with detailed information about the client-agency relationship. configuration within a hierarchy large businesses have been emulating the advantages attributed to operating in a "small business" mode by decentralising (frese 1993). for these types of businesses a similar problem arises to what has been described in the cases above (figure 8). there are small units that need to be co-ordinated to some extent by central is within the whole corporation. there is a need for some standardisation. also, central is cannot forecast the different requirements arising in the various units and its modus operandi is geared towards large systems. there is a need for flexibility and for practices emerging in the decentralised units. 29 ajis vol. 5 no. 2 may 1998 in particular, within a business using a profit centre organisation the individual units will want to decide themselves what type of software will allow them to perform and achieve set goals. a core application approach might help to achieve the standardisation needed to achieve is efficiency within the organisation as a whole and allow for the emergence of decentralised is-strategies helping to reach the effectiveness pursued with decentralisation9. large buslne88e={> profit centre software central is k^f house figure 8: configuration within a large business the determination of what should make up the standardised core system and what parts of the application portfolio could be left to the individual business units is a reasoning process that would come up with differing set-ups depending on the purposes pursued in the configuration. organisational goals will shift the lines between core system and its peripherals. next steps and preliminary conclusion the lines between business functions and the border between what should be standardised and what should be left to the individual business unit are shifting as the business environment and strategies change. by extending the core application concept to other business networks, the process of defining what belongs to a core application and what constitutes peripherals can be more deeply discussed. software technology needs to support these shifting domain definitions. and business administration needs to provide an understanding for the processes that lead to a sensible division between core and peripheral, which may happen in a dialogue involving interested parties or within a standardisation process. this way, application architectures can integrate the advantages of packages and custom-made software. one example of how this can be achieved from a business perspective as well as from a technical point of view was discussed for a loose business network configuration. switching standpoint to the segment of small business can yield fresh insights for information systems research and deliver frameworks for other business configurations. references bassett, p. (1996) framing software reuse: lessons from the real world, upper saddle river. becker, j. and schiitte, r. (1996) handelsinfonnationssysteme (wholesale trade and retailing information systems), landsberg/lech. birch, d.l. (1987) job creation in america: how our smallest companies put the most people to work, new york. bmft (1994) bundesministerium fur forschung und technologic (federal ministry for research and technology) initiative zur forderung der software-technologie in wirtschaft, wissenschaft und verwaltung (initiative for the advancement of software technology in commerce, science and administration), bonn. brockschmidt, k. (1995) inside ole, 2* ed., redmond. bundesamt fur statistik (1994) (federal bureau of statistics) unternehmen und beschaftigte am 25.05.1987 (businesses and employees on 25.05.1987), statistisches jahrbuch 1994, wiesbaden, 137. coda (1997) why coda? error! reference source not found., 02.08.1997. ei-namaki, m.s.s. (1990) small business the myths and the reality, long range planning, (23) 4,78-87. frese, e. (1993) geschaftssegmentierung als organisatorisches konzept zur leitbildfunktion mittelstandischer strukturen fur grobuntemehmen (business segmentation as organisational concept the lead role of medium-sized structures for large businesses), zfbf (zeitschrift fur betriebswirtschaftliche forschnng), (45) 12,999-1024. the balance between deliberate vs. emerging strategies (mintzberg 1989) needs to addressed for the isfunction as for all other business strategies. 30 ajis vol. 5 no. 2 may 1998 hofmann, d. and rockart, j.f. (1994) application templates: faster, better, and cheaper systems, sloan management review, (36) 3,49-60. ibm (1995) die ibm geht neue wege zum markt (ibm takes new roads to the market), ibm nachrichten, (45) 321,70-75. mertens, p. (1997) integrierte informationsverarbeitung i (integrated information processing), 11th ed., wiesbaden. meyer, j.-a. (1993) semiindividuelle software (semi-individual software), dbw (die betriebswirtschaft), (53) 1, 129-130. mintzberg, h. (1989) mintzberg on management, new york. oag (1995) open applications group, white paper, chicago. oag (1997) open applications group, open applications group integration specification, release 5, document number 970630oag, chicago. sap (1997) the bapi catalog, error! reference source not found., 02.08.1997. scheer, a.-w. (1994) wirtschaftsinformatik referenzmodelle fur industrielle geschaftsprozesse (information systems research reference models for industrial business processes), 4* ed., berlin. acknowledgements this research has been funded by datev eg, niirnberg, germany (http://www.datev.de). 31 paper title (paper title style) australasian journal of information systems ram & zhang 2018, vol 22, research article an investigation into failure of internet firms 1 an investigation into failure of internet firms: towards development of a conceptual model jiwat ram shandong university jiwat.ram@gmail.com zeyang zhang xi’an jiaotong – liverpool university abstract the last two decades have witnessed an exponential growth in internet and social media based commerce in china, resulting in a number of foreign internet firms launching their businesses to capitalize on the market opportunities. surprisingly though, having been successful globally, these firms were not able to remain competitive in china with majority of them suffering losses from their failed ventures and ceasing their operations. despite this ongoing problem, little or no research exists that might explain what is causing these problems. addressing this gap in knowledge, we build literature-based insights and through our analysis, we: (1) provide a structured understanding of some of the major issues causing failures, (2) identify and categorize factors/sources of failures into internal versus external driven, and (3) grounded in theory and supplemented by literature evidence, develop hypotheses and a corresponding conceptual model explaining the relationships among these factors/sources and the failure of foreign internet firms. the proposed model serves as a means by which information systems managers / chief information officers (cios) / technology and business managers can understand the sources of failures and conduct an introspective exercise within the firm to plug some gaps before launching a business in a foreign country. academically, the study has developed a theoretically-grounded comprehensive model to advance knowledge in a scantly researched area of challenges faced by the foreign internet firms and to help in the development of strategies to mitigate these problems. the proposed model also adds to the current knowledge on information systems socio-technical theory and the comparative theory of competitive advantage. keywords: foreign internet firms, causes and factors of failure, patriotic tendencies, perceived globalness, sustainable business model, network effect 1 introduction the last two decades have witnessed the rise of oriental giant ‘china’ in the internet industry. china is now one of the largest internet markets, with an internet population of an estimated 772 million, 753 million mobile internet users and a promising future; the internet penetration rate doubled from 2006’s 25.5% to 55.8% by december 2017 (cnnic, 2018; chinadaily, 2018). this phenomenal growth, despite an increasing number of government regulations and a worldwide economic recession, could be partly attributed to favourable government policies on the sharing economy. the “internet plus” policy introduced in 2014 seemingly offers unprecedented opportunities to operate in the chinese internet market, albeit with some extended procedures at the local government level (cheng & liu, 2016). australasian journal of information systems ram & zhang 2018, vol 22, research article an investigation into failure of internet firms 2 given these policies and the potential of profits and opportunities, foreign internet firms have turned their focus on the chinese market. notably, two internet giants from the united states, uber and airbnb, launched their businesses in china in 2014 and 2015 respectively. yet two years later uber china surrendered to its local rival didi chuxing. this has come at a cost, as uber china is reported to have paid approximately $1 billion per year in subsidies and captured less than 15% of market share (cheng & liu, 2016; dou & chu, 2016). adding to this saga is the fact that uber’s failure in china is not the first time that a foreign internet firm could not survive competition against a local rival. ebay, the first foreign internet firm entering china in the c2c segment, suffered a dramatic market share fall from 85% to 36% in 2005 as a result of taobao’s entry into the china market. tencent qq “monopolized internet message market” with a share of 84.4%, effectively wiping out msn who only managed a market share of 13.9% in 2006. google is another example with 16.6% market share compared to baidu with 76.9%; it had to leave china in 2010 due to regulatory issues (oh & zhang, 2010). in fact, nearly half (48%) of foreign firms have failed and rolled back their chinese operations within 2 years (china bridge, 2013; marquis & yang, 2014) of starting their business. airbnb, the only surviving foreign internet giant is also facing stiff competition from three chinese competitors: xiaozhu (short rental), tujia (travel rental), and ctrip (integrated travelling services). the problem is compounded by the fact that china is witnessing an explosive growth in online business activities with online retail transactions reaching an estimated rmb7.175 trillion in 2017, registering a year-on-year growth of 32.2 percent (cnnic, 2018; deloitte, 2016). while there seem to be many opportunities for internet firms, it also raises questions about why some firms are failing in the chinese market, despite efforts by government to accentuate growth through favourable policies. the situation underlies a significant ongoing problem that merits investigation. yet limited research exists on the reasons, challenges and issues contributing to the failure of foreign internet firms in china. motivated by this gap in knowledge, we study the following research question: what causes internet firms to fail when they operate in a foreign country such as china? the main idea of this paper is to examine and identify some of the possible sources or factors driving failure of internet firms in general, by learning from experience of china and build fundamental knowledge on the challenges of operating in a foreign internet market. drawing on the existing literature and using a theoretical foundation, we then develop a conceptual model to show relationships among the possible contributing factors and the outcome i.e, the failure of foreign internet firms. through this study, we expect to facilitate the development of concrete strategies to avoid or handle this problem more effectively. the study contributes significantly in multiple ways: 1. it builds a structured understanding of issues and challenges faced by foreign internet firms operating in china (section 2); 2. it identifies a number of factors/sources of failures, categorizes them as internally or externally driven, and then builds some theoretical knowledge of them (section 4); 3. grounded in theory and supplemented by literature evidence, it develops hypotheses explaining the relationships between the factors or sources and the failure of foreign australasian journal of information systems ram & zhang 2018, vol 22, research article an investigation into failure of internet firms 3 internet firms. this opens up opportunities for further modelling and examination of relevant issues (section 4), and 4. it creates a conceptual model advancing a holistic understanding of the issues related to failure of foreign internet firms, which adds to existing knowledge on information systems socio-technical theory (stt) (section 4). 2 a critical perspective of relevant literature 2.1 motivation for foreign internet firms to operate in china driven by profits and pursuing a globalization strategy, foreign internet firms are actively extending their reach and penetration into overseas markets. for the purpose of this study, we adopt a definition of internet firms from prior studies as “the firms that offer goods and services mainly using the internet technology and infrastructure” (my, kim & sayim, 2015). being the world’s largest internet market with 772 million users, 7.175 trillion cny trading volume per year and a nearly 55.8% internet penetration rate, china is naturally a hotbed of potential and opportunities in the internet market (cnnic, 2018). adding to that, certain favourable government policies have enticed foreign firms to seek opportunities in the bullish chinese internet market. however, despite all this optimism, the reality is far different. a number of foreign internet firms such as ebay, msn, google and uber have failed to capitalize on the potential and promise of the chinese internet market and have lost ground to their respective local competitors (also see section 1). the only exception, so far, is airbnb who entered into china in 2015 and despite many struggles has survived the challenges from its three local competitive “snipers” xiaozhu, tujia, and ctrip (carew, 2015; han, 2015). this situation paints a grim picture for foreign internet firms and necessitates examination of the reasons, challenges, and issues that are causing failure. based on a review of literature, the reasons for foreign internet firms’ failure in china can be broadly classified and considered from three perspectives: (1) cultural, linguistic and psychological differences (section 2.2), (2) differing strategic approaches (section 2.3), and (3) political, legal and social aspects (section 2.4). we discuss these in detail comparing the differences in approaches taken by foreign internet firms as against their local rivals. 2.2 reasons for failure: cultural, linguistic, and psychological differences cultural differences, as the most intuitive reason, have been identified by various researchers. chinese users have significantly distinct expectations. wang (2012) points to these differences and compared asian against western countries’ cultures, i.e. “collectivistic” versus “individualistic” on the notion of “uncertainty avoidance” – i.e. the level of accepting uncertainty and ambiguity. they found that chinese, as well as most asians, are risk-averse and less trusting, and score significantly lower on these two criteria compared to westerners and even to the global average. these cultural, linguistic and psychological tendencies translate into distinct preferences and perceptions requiring adjustments of business environments as well as product / service offerings. we discuss some of the issues identified in the literature below. australasian journal of information systems ram & zhang 2018, vol 22, research article an investigation into failure of internet firms 4 2.2.1 ease of communication with the sellers help generate trust given the cultural context, secure online payment systems and communication with sellers are crucial to chinese consumers’ positive experience for doing online shopping. this can be seen from a comparison of taobao and ebay payment systems. ou and davison (2009) offer an example. aliwangwang, a chatting application for buyers to contact with sellers is embedded in the taobao (chinese c2c portal) website. compared to skype or email used by ebay for communication between sellers and buyers, aliwangwang is much more convenient, since users do not need to download and register to use this application and its associated features. additionally, the application has an innovative feature that a blue or grey icon of aliwangwang appears under every item selling at taobao, indicating whether the seller is online and available to chat; this helps users in seeking more information and making wise buying decisions. with chinese consumers having a “strong preference” to contact the seller, such functionality plays a critical role in the overall buying experience and completion of the sales process (ou & davison, 2009). marquis and yang (2014) suggest that not having an effective, easy-to-use communication tool is the key reason for ebay’s failure. although limited literature exists that has recognized cultural differences and preferences with an in-depth analysis (ackerman & tellis, 2001), not much work has been done in an internet business context. similarly, some studies have analysed the impact of price or quality as the deterministic factors influencing chinese buyers’ purchasing decisions (law, to & goh, 2008). others have found an integrated website layout as one of the factors that drives users’ acceptance and reduces the chance of failure (zhou, lu & wang, 2013). overall, there is little knowledge about these issues in an internet market context that includes foreign internet firms as well. hence, the need for more research in this context. 2.2.2 ease and modes of payments alipay is the online payment system launched one year after taobao was launched. alipay used by taobao users for shopping provides multiple methods of payments including online credit or debit card, cash on delivery, and allows adding funds to an online account at over 66,000 post offices or bank branches (greeven, 2012). compared to ebay this is much more flexible, as ebay only accepts online credit / debit card payments. chinese prefer cash and bankbooks rather than cards, especially in the early 2000s when credit / debit cards were not generally accepted by many businesses. this difference is one of the possible causes of ebay’s failure in china (wang, 2012). 2.2.3 risk avoidance a customer’s preference for avoiding risks when shopping online is facilitated by the seller’s reputation. ye et al., (2013) reported that there is a positive correlation between sales volume and customers’ comments about a seller’s reputation on taobao. it supports the view that chinese customers tend to choose a less risky commodity and care less about the price, whereas ebay’s markets are price-based. this may explain why ebay is less popular in china compared to taobao. however, since the research was conducted with data from taobao china and ebay us, further research focused on the chinese market will add value. 2.2.4 localization of shopping environment ebay, as the first c2c platform in china, applied an auction market model, which is “alien” to chinese customers (china bridge, 2013). in contrast, taobao has a bargaining market model australasian journal of information systems ram & zhang 2018, vol 22, research article an investigation into failure of internet firms 5 “designed by and for chinese users” (ou & davison, 2009), which allows customers to bargain with sellers, similar to a real-world business and which fits the habits of chinese customers. a similar problem exists in the way uber operates, whose user interface was designed for american users and then copied into a chinese version. didi (uber’s local chinese competitor) has a more localized interface. specifically, customers have to send emails to uber if they have any questions, while didi has a button on main menu to call a customer service centre. considering that email penetration rate in china is only 36.8% (cnnic, 2016), this could be a hurdle to use of uber’s services. chinese customers typically use a phone to hail a taxi online. it highlights that “a copy-and-paste” approach in china is not effective and does not contribute to success (ke & sun, 2014). 2.2.5 perceived globalness foreign firms also prefer a standardized service with global consistency. it leads to linguistic and cultural problems. failing to localize with special features that cater to the different preferences of local customers do not help in engaging with them (oliver, 2015). however, oh and zhang (2010) provided contrasting evidence, suggesting that standardized user environment, which they call “perceived globalness” is an advantage. they argue that consumers may prefer the international brand as a symbol of social status. they conducted the research by “sociotechnical preference model”, illustrating eight dimensions of the services provided by domestic and foreign firms, including “perceived globalness”. the linear regression results proved that “perceived globalness” is a significant factor that could influence consumers’ choice. however, given that the questions asked in the questionnaire aimed to determine whether the interviewees agree with the concept of “perceived globalness” and not whether this concept has influenced their behaviour, the results should be treated with caution. while one can argue that such a perception, in particular for high-end goods exists, more work is needed to understand the influence of this concept on the choices made by consumers, and how significant this is. 2.2.6 patriotic tendencies when it comes to preference for foreign brands, another stream of thought suggests that some customers prefer domestic products for patriotic reasons (oh & zhang, 2010). terming it as “consumer ethnocentrism” the researchers tested chinese consumers’ attitude towards domestic and foreign products. the results show a negative inclination towards buying products of foreign firms, though not significantly. whilst various researchers (e.g., dong & yun-tang, 2012) assert that it should be an important factor explaining why foreign corporations are failing in china, it is not limited to the internet industry alone. 2.3 reasons for failure: strategic and operational approaches a lack of understanding of customer preferences may also affect operating strategies and performance. when providing the same or similar types of services, firms in competition may distinguish themselves by adopting unique operating strategies (ke & sun, 2014). we discuss some of these issues in the literature below. 2.3.1 sustainable business model it is argued that a tailored long-term sustainable business model is crucial to success, whilst a “one size fits all” strategy leads to failure (china bridge, 2013). australasian journal of information systems ram & zhang 2018, vol 22, research article an investigation into failure of internet firms 6 the first major sino-foreign internet competition between taobao and ebay points to the distinct business models they use. taobao applied a “fee-less business model” to attract both sellers and customers, whereas ebay copied its global strategy of generating profits by charging fees for providing a trading platform (greeven, 2012). it seems that taobao’s long-term strategic approach suited the local context better than ebay’s well-established model based on its global marketing strategy. additionally, the bargaining model used by taobao versus the auction model of ebay seems to be one of the contributing factors leading to failure of ebay in china. providing free services to enlarge market share and then profiting from large sales volume is a business model that has been utilized successfully ever since taobao drove ebay out of the chinese market in 2006 (jin, 2016). another example of sustainability of business model, similar to taobao’s, is baidu’s advertising business. although repeatedly questioned and blamed on recent social media traffic for several advertisement-related incidents, baidu is generating significant amount of income from its advertisement activities (cnnic, 2016). such a situation is contributed by baidu’s dominating market share as well as the loyalty from more than 60% of its customers, whose decisions are vastly influenced by baidu’s search results (wang, 2012). in contrast, google failed in china before it could find and launch a sustainable profit-generating model. airbnb, like its local rivals tujia and xiaozhu, has not been able to construct a sustainable model yet. suggesting that “growing our business is a higher priority now than profitability”, the co-founder of tujia acknowledged that tujia has not yet reached a break-even point (carew, 2015). along with tujia and xiaozhu, airbnb as a late-entrant into the market and is trying to expand by enlisting accommodations. though it is difficult to get more accommodations listed, the growing number of orders and abundant cash flowing into the short-rental industry have shown promising prospects and a sign of maturity (han, 2015). but one asks whether the existing model is feasible and sustainable. and what kind of business model is most suitable? is the business model significant to airbnb’s future in china? further analysis is required. uber and didi applied and developed their business model further by offering subsidies. founded in 2012, didi initially focused on licensed taxi-hailing business, providing a free platform benefitting both taxi drivers and passengers. having acquired 99% of taxi-hailing market shares, didi started offering subsidies to both taxi drivers and passengers, after entry of other local rivals in the taxi business segment. with the entry of uber in chinese market in 2014, didi acquired or partnered with other local competitors to keep its hold on the market intact. unlike didi, uber started its business with private cars, copying the us model. didi on the other hand, continued broadening its business by developing a network of not only the licensed taxis but also enlisting private cars. the white-hot “subsidies war” became fiercer as both firms were striving for maximum market share. however, uber was not able to sustain itself in the market for long and after spending more than $1 billion per year to achieve less than 15% market share, it declared its failure in china (dou & chu, 2016). uber’s business model could be considered as less effective and problematic; it was no surprise that it didn’t work for uber (dou & chu, 2016). ou and davison (2009) suggest that the model based on offering subsidies is not a profit-oriented model. however, it has worked for didi, australasian journal of information systems ram & zhang 2018, vol 22, research article an investigation into failure of internet firms 7 as it is being used in a large number of cities in china, benefiting the public and drivers alike (cheng & liu, 2016). it remains unclear, however, whether the failure of uber is caused by its ineffective business model. there could be other elements of their operating strategy that may have caused uber’s failure, e.g., price differences as a result of use of different fee formula and subsidies, and differing consumer preferences, to name a few. therefore, it will be worth examining other factors that may have had a role in uber’s failure in china. 2.3.2 operating strategies researchers have also identified several specific differences among the operating strategy of various service providers (dou & chu, 2016; han 2015). first-mover advantage has been argued to be a possible factor. dou and chu, (2016) han (2015), and ou and davison (2009) argued that a first-mover advantage is vital and played a role in the failure of google, uber, and airbnb in china. wang, (2012) however didn’t find support for the first-mover advantage. for instance, ebay had the first-mover advantages as it started its operations in china before the birth of taobao, yet ebay still could not survive against its local rival. hence, ebay’s failure is a stark reminder of how operating strategies of the service providers affect their chances in the market. contrastingly as uber and airbnb are late entrants, hence their operating strategies warrant further examination. 2.3.3 network effect greeven, (2012) introduced the concept of “network effect,” which essentially means an enterprise expanding its business into several interlinked complementary fields by forming a network of partners. for instance, alibaba’s network “ecosystem” consists of “centre” taobao for shopping, alipay for online payment, alisoft for software service to sellers, and alimama for posting specialized advertisements (greeven, 2012). another example is didi’s “enterprise lines” which includes: taxi, private car, high class private car, ride-sharing, designated drivers, shuttle bus, test driving, and car rental (cheng & liu, 2016). baidu is following a similar trend and expanding its business to group purchase, take-out delivery and financial services, adding to its previous business including encyclopaedia, input tools, bbs, music, video, digital map and so on and so forth (baidu, 2017). in contrast, foreign firms have failed to “weave” a similar network of partners. nonetheless, there is no clear evidence indicating that the network effect is significant to users or contributing to failure if not present. to form a successful “network”, choosing right local partners and investors with local information and staff is vital (custer, 2015). for instance, didi established a partnership with tencent, whose products qq and wechat are the most popular instant messaging mobile applications in china (cnnic, 2016; dou & chu, 2016). in exchange for sharing didi’s profit; tencent not only provides money but also big data analysis support, digital map services and wechat advertising thus allowing wechat users to push one button to use didi’s service without downloading and registering any specific app (cheng & liu, 2016). as a rival to didi, uber made the accusation a number of times that their articles and messages were frequently blocked on wechat (dou & chu, 2016). it seems that partnership with domestic firms and investors is significant to a foreign firm’s success. 2.4 political, legal, and social reasons government regulations, political and legal aspects, also play a role in failure or success of a business in any country. unlike in developed countries where markets are driven by the australasian journal of information systems ram & zhang 2018, vol 22, research article an investigation into failure of internet firms 8 demand and supply equation, china’s market economy system emphasizes the role of regulatory bodies in facilitating market operations, especially in case of direct foreign investment (lee, 2010). several authors (stevenson-yang, 2007; financial times, 2005) have argued that local internet firms are far better off in meeting the regulatory requirements, in comparison to foreign firms; this often leads to the foreign firms’ failure. it is believed that tightening of regulatory policies contributed to the retreat of google (lee, 2010; marquis & yang, 2014). similarly, both uber and airbnb have had to deal with a range of regulatory requirements. yet a clear policy on regulation concerning short-renting services is not yet available. stevenson-yang, (2007) citing “2006 china search engine survey report” highlighted that based on chinese users’ views, google’s search results are more relevant than baidu’s. in addition, wang (2012) points that google uses fewer advertisements and offers unique services such as input tools, translators, notebook facilities and google mars. yet google was not able to operate successfully in china. however, the literature is split on the reasons for its failure. there has been a view that google’s servers in china were much slower and more liable to breakdown (lee, 2010). moreover, the market share of google at 16.6% compares with baidu’s 76.9% in 2009. the increasing gap between them points to other problems that google may have had in the market (oh & zhang, 2010). so it is possible that even if the regulatory matters had been resolved, google may still not have been able to remain competitive due to its minority share in the market. interestingly, just like with google, though the regulations applied to uber and didi equally, yet uber was not able to survive. it is also noteworthy that uber sold its china operation to didi chuxing on aug 01, 2016, just a few days after an encouraging and soft policy was announced by the government during last week of july 2016 (abkowitz & carew, 2016; cheng & liu, 2016). airbnb, along with local rivals tujia, xiaozhu, and ctrip, are operating with an even more complicated business arrangement (xinhua insight, 2015). short renting, as a new form of service in expanding sharing-services industry, is compliant with china’s “internet plus” and sharing economy strategies, but government regulations need to evolve on that. examples from other countries show that, even in western countries airbnb’s business face several challenges on legal and market fronts (somerville & bellon, 2016). however, this might be an advantage over local rivals since they have experiences negotiating with governments (han, 2015). it could be that the success or failure of airbnb may be decided by its relationships with the local governments. custer, (2015) concludes that it is logical that foreign firms have to abide by china’s government regulations, otherwise, there is only a “one-way ticket home.” 2.5 summary of the literature review the above review suggests that failure of foreign internet firms in china stems from multiple reasons including: cultural differences, linguistic disadvantages, dis-aligned business models, network impact, choice of local partner(s), and regulatory requirements, among others. one aspect clearly arising from this discussion is the comparative advantage that local internet firms had over their foreign counterparts which has resulted in driving out the competition in the majority of the cases. it seems that local internet firms leveraged their local knowledge, networks, capabilities and resources more competently to develop and deploy strategies to capture a greater market share, leading to failure of foreign internet firms. australasian journal of information systems ram & zhang 2018, vol 22, research article an investigation into failure of internet firms 9 3 theoretical underpinning drawing upon the review of the literature, we use the “comparative theory of competitive advantage” to build a conceptual model for reasons and challenges of entering internet market in china and contextually similar countries / cultures. the comparative theory of competitive advantage rejects the neo-classical assumption of the “theory of perfect competition” and posits that firms operate in a dynamic environment (hunt & morgan, 1995). the key tenets of the theory are (hunt & morgan, 1995): 1. consumer demand is heterogeneous and dynamic with taste and preference of consumers changing; 2. consumers have imperfect information about the products / services “that might match their tastes and preferences”; 3. humans, while serving dual role as consumers and managers of their firms, are constrained by self-interest. this points to the opportunistic nature of human beings and the fact that their morality, ethical behaviour, actions and deeds in life are subject to the significance of opportunity; 4. fourth and fifth is that a firm strives to seek superior financial performance under the condition of imperfect information about customers and competitors. the use of notion of ‘superior’ instead of ‘maximum’ is “both because of the well-documented fact that they (the firms) lack the information to do so (i.e. they operate under bounded rationality [simon 1979]), and because morality considerations at times constrain them (or some of them) from doing so.” the measure and relative importance of financial performance varies from firm to firm and industry to industry; 5. the sixth tenet is that the firms make use of available resources, both tangible and intangible, to produce efficiently and create offerings that are valuable to a segment of market or customer base; 6. seventh, the resources available to a firm are “both significantly heterogeneous across firms and imperfectly mobile.” the make-up and structure of resources available to a firm are unique and not “commonly, easily, or readily bought and sold in the marketplace.” firms in competition can attempt to copy the resources of the successful competitors, but heterogeneity can still persist for a length of time. firms with unique mix of resources if able to use those resources in an effective way can achieve a comparative advantage by developing offerings that are seen by the customers as value added and lower costs. the distinctiveness of offering is essentially dependent on the distinctiveness of inputs; 7. the theory also defines the role of management that is to “recognize and understand current strategies, create new strategies, select preferred strategies, implement or manage those selected, and modify them through time”; 8. ninth, in contrast to neoclassical theory which posits that the “structure of its (the firm’s) industry, strictly determines its conduct (or strategy) and performance (profits),” the comparative advantage theory suggests that the structure only influences that strategy and performance. the firms competing in a certain market need to develop their own australasian journal of information systems ram & zhang 2018, vol 22, research article an investigation into failure of internet firms 10 strategy based on uniqueness of their resources and inputs thereby targeting specific set of customers, markets and competitors as deemed necessary. 4 development of a conceptual model drawing upon the critical review of the literature presented in section 2, we identify a number of possible factors or sources contributing to failure of foreign internet firms (table 1). these factors are then used to drive hypotheses resulting in development of a conceptual model that shows the relationships between these factors and the outcome i.e. failure of internet firms. the conceptual model is presented in table 1. factors internally vs externally oriented factor drawn from discussion in section(s) reference 1. strategic misalignment internal 2.2.2 grewal, chandrashekaran & dwyer, (2008) 2. lack of internal preparedness internal 2.2.2 chen, (2013); walcott, (2014) 3. lack of environmental awareness internal 2.2.1, 2.2.2, 2.2.3 shkurti & leka, (2010) 4. foreign organizational culture internal 2.2.2 shin, (2004); selmer & de leon, (2002) 5. foreign management style internal 2.2.2 chen & myasin, (2008); wendt, (2015) 6. lack of business model adjustment internal 2.2.2 byhlin & holm, (2012); sleuwaegen, (2013) 7. lack of adjustment to dynamics of business eco-system internal 2.2.2, 2.2.3 kwak, lee & chung, (2012); rangan & drummond, (2004) 8. competitors’ strategic leverage external 2.2.1, 2.2.2, 2.2.3 sofka & zimmermann, (2005); jurse & jager, (2014) 9. regulatory constraints external 2.2.3 li, yang & yue, (2007); sofka & zimmermann, (2005) 10. socio-cultural influences external 2.2.3 hoffman & preble, (2004) 11. language preferences external 2.2.1 piekkari, vaara, tienari & säntti, (2005) 12. patriotic tendencies external 2.2.1 akin, (2016) table 1. factors / sources of failure of foreign internet firms 4.1 internally driven sources 4.1.1 strategic misalignment strategic alignment is defined here as “the alignment of organizational strategies to the needs and context of the business eco-system in which the organization operates.” drawing upon the tenets of the comparative theory of competitive advantage, a firm must use its resources in such a way that these resources create value for the targeted set of customers. organizations do not have unlimited resources at their disposal and their use of resources is often a strategic decision. therefore, if the use of organizational resources is strategically australasian journal of information systems ram & zhang 2018, vol 22, research article an investigation into failure of internet firms 11 misaligned, the resources could be mis-utilized and do not end up creating value for the target set of customers (grewal, chandrashekaran & dwyer, 2008). strategic misalignment has remained one of the issues when multinational companies establish strategies for their foreign subsidiaries in line with their global strategies which may not fit to the local context (grewal, chandrashekaran & dwyer, 2008). strategic alignments therefore, requires that foreign internet firms establish long term goals as well as the corresponding strategies and tactics that suit the local context. these include setting up an organizational structure that has an understanding of: local business environment, cultural norms, people’s mindsets, regulatory environment; and has a leadership team made up of people that are locally connected or born, with an office infrastructure that can help facilitate operations effectively. also needed are a good branding strategy, appropriate marketing and sales strategy (pricing, promotion etc.), understanding of local distribution channels, financial markets, customer buying capacities, and local market trends (greeven, 2012). however, a number of foreign firms have failed to strategically align their business operations to the chinese context and were not able to capitalize on the available market opportunities and operate sustainably (marquis & yang, 2014). given the above, we propose: h1: a strategic misalignment to the needs of local market is significantly associated with the failure of foreign internet firms. 4.1.2 lack of internal preparedness an internal preparedness to enter into a foreign market is defined here as “the state of being prepared with resources, information, appropriate network of contacts, and human resource capabilities to start, operate and sustain a successful business operation.” an internally well prepared foreign firm will do an internal assessment of whether their human resources are capable to start, operate, and sustain a business on a foreign land, both remotely and on ground. the firm will also look at the internal processes and see if they are tailorable and scalable to the needs and context of foreign operations. having an assessment of the financial and physical resources needed and availability and appropriateness of the resources to the local context is a key indicator of internal preparedness. according to the tenets of the comparative theory of competitive advantage, firms must identify a set of unique resources to be sustainable in the market. the internal prepared assessment, therefore, will help firms understand whether they have the right balance of resources to compete and sustain in a foreign market. the evidence shows that uber and ebay were not well prepared when they entered chinese market, as their processes and business operations were not entirely compatible to local customer preferences and hence these foreign firms faced challenges and subsequently failed (ke & sun, 2014). drawing upon the above discussion, we propose the following hypothesis: h2: a lack of internal preparedness for entering and operating into a foreign internet market is significantly associated with the failure of foreign internet firms. 4.1.3 lack of environmental awareness the understanding of the local market eco-system, including customers, suppliers, distribution channels, regulations, tax regime, culture, values, resources availability, access to australasian journal of information systems ram & zhang 2018, vol 22, research article an investigation into failure of internet firms 12 utilities and infrastructure (and the service providers thereof) and financial and banking channels is expected to facilitate starting and continuing operations in china. a lack of environmental awareness has caused failure of foreign firms such as ebay when they used their ‘auction model’ in china leading to their failure (china bridge, 2013). the above argument also corresponds to information systems socio-technical theory (stt) which emphasizes the fit between technical and social context (bostrom & heinen, 1977). if the firm lacks an awareness of its eco-system, it will not be able to seek and deploy the resources that facilitate its sustainable operations. this is consistent with the tenets of the comparative theory of competitive advantage, which calls for firms to know and use their resources in a way that create a unique mix of the resources. we therefore propose: h3: a lack of environmental awareness for entering and operating into a foreign internet market is significantly associated with the failure of foreign internet firms. 4.1.4 foreign organizational culture the culture of an organization has significant bearing on the strategic and operational functioning of an organization (degravel, wilkin & zhou, 2016). organization culture not only affects the processes, policies, and ways of working but also influences the norms and behaviours of people working in the organization. therefore, imposing a foreign culture that is not aligned or compatible to a particular context can cause challenges and affect the operations of a firm (shin, 2004). organization culture is also considered a resource because of its impact on the sustainability of an organization. therefore, when firms start their operations in a foreign country, they often impose the parent organization culture which is counterproductive as we discussed in the context of chinese experience of foreign firms. drawing upon the tenets of the comparative theory of competitive advantage which advocates the wise use of resources to create a comparative advantage, we believe that imposing a foreign organizational culture may create distortion in the development of unique assets. thus, we propose: h4: a foreign organizational culture is significantly associated with the failure of foreign internet firms. 4.1.5 foreign management style management style is potentially one of the most influential drivers of motivation, inspiration and productivity within an organization (femi, 2015). management style is generally contextual and needs to be tailored to environment to achieve the results. dealing with human beings is a dynamic process and imposing a management style that does not fit the norms, behaviours and cultural context can lead to failure (chen & myasin, 2008). the failure of foreign internet firms operating in china might be attributable to their management style, as it did not take into account the local conditions properly. the failure of ebay and uber are stark examples (section 2). according to the comparative theory of competitive advantage, firms choose a management style and strategies that give them comparative advantage. these firms modify their strategies as needed to achieve the advantage. from the discussion in section 2, we can argue that foreign internet firms in china were not able to modify their management style effectively, and this contributed to the challenges they faced. australasian journal of information systems ram & zhang 2018, vol 22, research article an investigation into failure of internet firms 13 therefore, drawing from the theory and given the above discussion, we hypothesize: h5: a foreign management style is significantly associated with the failure of foreign internet firms. 4.1.6 lack of business model adjustment we define business model adjustment as the “ability of firms to fine-tune or modify their business model to operational needs of the changed context.” firms operating in foreign countries, therefore, would need to understand the types of business model that suits the needs of operating in that country (byhlin & holm, 2012; sleuwaegen, 2013). a lack of business model adjustment has led foreign internet firms in china to face various challenges as discussed in section 2. the notion of business model adjustment also confirms to the tenets of the comparative theory of competitive advantage, which states that a firm’s structure must not be an impediment to its strategies. hence firms wanting to achieve advantage over competitors need to adjust their structure, in this case adjustment of business model, to be able to execute strategy that suits the market and the needs of target customers. following the above discussion, we propose: h6: a lack of business model adjustment to the local needs is significantly associated with the failure of foreign internet firms. 4.1.7 lack of adjustment to dynamics of business eco-system we define adjustments to dynamics of business eco-system as “the ability of an organization’s structure, systems and processes to respond flexibly and adjust to the needs of changing business eco-system.” to be able to adjust to the dynamics of business eco-system, a firm needs to establish a scalable and flexible organizational structure, systems and processes that can be refined, modified and adjusted as needed in wake of changing dynamics of the business ecosystem (kwak, lee & chung, 2012). building managerial and organization-wide human resource capabilities is critical in implementing the actual adjustments (kwak, lee & chung, 2012; rangan & drummond, 2004). the failure of foreign internet firms such as uber, ebay could be attributed to their inability to adjust to the dynamics of business eco-system after launching their operations in china (section 2). the adjustment to the dynamics of business eco-system also confirms to the tenets of the comparative theory of competitive advantage, which proposes that managers should adjust the strategies through time. adjustment to dynamics of business eco-system could mean focusing both inwardly and outwardly. these adjustments could be repositioning of products and services, re-branding, adjusting marketing and sales strategies, building new enhancements in products and services, adjusting organizational culture, and restructuring of internal hierarchical and staff structures, to name a few. drawing upon the above discussion, we hypothesize: h7: a lack of adjustment to dynamics of business eco-system is significantly associated with the failure of foreign internet firms. australasian journal of information systems ram & zhang 2018, vol 22, research article an investigation into failure of internet firms 14 4.2 externally imposed sources 4.2.1 competitors’ strategic leverage we define competitors’ strategic leverage as “the ability of competitors to strategically manoeuvre their position and available resources to achieve maximum advantage in the market.” competitors with local knowledge of markets, customers, cultural and business norms will be expected to have the ability to strategically manoeuvre their position (jurse & jager, 2014). firms starting a business in a foreign market may, therefore, face challenges from their local competitors. because local firms are expected to have better market intelligence and understanding; they will be better embedded in the local population, have stronger networks with suppliers and distributors, and know the psyche of customers better. such a position will help local firms competing against foreign counterparts to deal with market dynamics and uncertainties effectively and proactively (sofka & zimmermann, 2005). the literature evidence shows that one of the reasons foreign internet firms in china faced challenges and were not able to operate sustainably was because their local competitors outmanoeuvred them by offering products, services and strategizing in a way that better fits the needs of chinese customers (section 2). the position is consistent with the tenets of the comparative theory of competitive advantage which proposes that under the conditions of imperfect information about customers and competitors, the firms that use their situation better achieves superior outcomes. thus, we propose: h8: competitors’ strategic leverage is significantly associated with the failure of foreign internet firms. 4.2.2 regulatory constraints regulatory constraints have been identified in the literature as one of the key reasons for failure of foreign internet firms in china (section 2). these constraints include policies, procedural complications and legal disposition which could be very different from those applied in the foreign firms’ world headquarters (li, yang & yue, 2007; sofka & zimmermann, 2005). interpretation of policies and laws could also be an impediment in terms of understanding and implementing the policies according to the local requirements (li, yang & yue, 2007). this situation gives a comparative advantage to the local firms as they will be better positioned to understand, implement and comply with the policies and traverse through the legal landscape (section 2; li, yang & yue, 2007; sofka & zimmermann, 2005). this is also consistent with the argument put forward by the comparative theory of competitive advantage. therefore, we test the following hypothesis. h9: regulatory constraints imposed by the host country are significantly associated with the failure of foreign internet firms. 4.2.3 socio-cultural influences we define socio-cultural influences as “the social and cultural parameters ingrained in a society’s ways and means of dealing with day-to-day life and needs that are often not codified or explicitly expressed.” australasian journal of information systems ram & zhang 2018, vol 22, research article an investigation into failure of internet firms 15 socio-cultural influences are often dynamic in nature and depend on human interactions. there is always a level of unpredictability which makes it challenging to codify these influences and develop concrete strategies (hoffman & preble, 2004). however, it is expected that the local rivals of foreign firms will have a distinct advantage in understanding and responding to effects of socio-cultural influences. the literature evidence shows that uber, ebay and other foreign internet firms in china faced great challenges in dealing with the sociocultural tendencies (section 2). consistent with the comparative theory of competitive advantage argument that seeks firms to establish strategies that are compatible for target customers and markets, we hypothesize: h10: socio-cultural influences are significantly associated with the failure of foreign internet firms. 4.2.4 language preferences language preferences have been found to be one of the common challenges for firms when they operate in a foreign market (piekkari, vaara, tienari & säntti, 2005). the literature shows that foreign internet firms faced considerable challenges due to language preferences of chinese consumers (section 2). while having website and local staff can help in reducing the barriers, it is also important that product information, support and after-sales services and managerial staff should be localized to reduce possibilities of failure. having local language compatibilities across various spheres of operation will help in building the required rapport and impetus for sustainable operations. such a proposition is also consistent with the tenants of the comparative theory of competitive advantage. we therefore hypothesize as follows: h11: language preferences of customers are significantly associated with the failure of foreign internet firms. 4.2.5 patriotic tendencies we define patriotic tendencies as “the preference of customers to buy and use locally made products and services as a matter of allegiance and respect to the country and nation.” patriotic tendencies have been found to be one of the factors that contributed to challenges faced by the foreign internet firms in china (section 2). firms operating in foreign market need to be aware of possibilities of such tendencies and should design measures and strategies to reduce the impact of such tendencies (akin, 2016). developing tailored strategies to mitigate or lessen the impacts of patriotic tendencies is in line with the tenets of the comparative theory of competitive advantage as well which underscores the need for tailored strategies keeping in view the target customers. we, therefore, hypothesize: h12: patriotic tendencies of customers are significantly associated with the failure of foreign internet firms. australasian journal of information systems ram & zhang 2018, vol 22, research article an investigation into failure of internet firms 16 figure 1. the conceptual model 5 discussion and conclusion a number of foreign internet firms operating in china have failed to sustain their position in the market and have lost out to their local rivals. the list includes big names such as ebay, google and more recent cases like uber and airbnb. the cost of failure runs in millions of dollars and stakes have been high. this has raised questions about what is causing such failures. compounding the problem is a lack of studies that have investigated the sources and issues driving these failures. with the exception of a few studies (e.g., cheng & liu, 2016; dou & chu, 2016), a comprehensive understanding is still lacking. motivated by this gap in knowledge, the study described in this paper lays the foundation for a systematic understanding of sources of failure of foreign internet firms in china by developing a theory-driven conceptual model. in doing so, we have identified a number of possible factors or sources of failure, and have framed them in the model to advance our understanding of relationships among these factors and the outcomes. we build upon the existing knowledge and literature evidence and propose that these factors are both internally and externally oriented. the internally driven factors comprise both strategic and operational types. some of possible strategic sources of failure are strategic misalignment, and lack of business model adjustments. internally driven operational sources of failure are lack of internal preparedness, lack of environmental awareness, and lack of adjustment to dynamics of eco-system. the extent to which some of these factors cause failure could be understood by testing the hypothetical relationships as shown in the conceptual model. we also posit that model should australasian journal of information systems ram & zhang 2018, vol 22, research article an investigation into failure of internet firms 17 take into account the moderating effects of the size of firms, the number of foreign staff, and the location of foreign firm’s world headquarters as well as their location in china, to build a more comprehensive understanding of the reasons for failure of foreign internet firms in china. 6 implications, future directions and limitations 6.1 implications for theory and practice the study makes several significant contributions and has implications for theory and practice. academically, the study has developed a theory-driven comprehensive conceptual model for advancing understanding of factors and sources of failures and the possible interrelationships. secondly, by classifying the factors and sources into internally and externally oriented sources, the model proposed by this study is a significant advancement towards looking at the issue of foreign internet firms’ failure more comprehensively. thirdly, the factors identified cover a broad range including strategic, operational, socio-cultural, and human preference issues to build an in-depth knowledge. finally, the conceptual model extends earlier work on information systems socio-technical theory (stt) which emphasizes a correspondence and fit between technology and social context for achieving the best outcome and avoiding failures of technology driven business ventures such as those examined by this study. such an understanding is also in line with the study published in mis quarterly by bostrom and heinen, (1977). managerially, the model is a good starting point for firms to understand what might be causing the failure of foreign internet firms in china. the model could serve as an introspective device for managers to plug some gaps before launching a business in a foreign country. the model can also serve as a foundation for managers through which to devise appropriate strategies by learning from the experience of others as has been discussed in this study. 6.2 future research work the study has opened up a number of opportunities for further research. firstly, the model can be extended by inclusion of more factors and sources of failure to build a comprehensive understanding of the issues. secondly, while a number of internal and external factors could be causing the failure, the role of leadership and management is critical in all cases. therefore, further in-depth studies are required to investigate whether leadership and management is the trigger behind other sources of failure. thirdly, more work is needed to identify moderating and mediating factors such as: organizational size, location of headquarters, location in china or a foreign country of operation, management configuration, size of foreign staff, and investments in dollars, that may have a role in influencing the relationships between factors and the outcomes. finally, developing a model that is well-grounded in theory provides an opportunity for future work of undertaking either a mixed-method or a purely quantitative study to test the proposed hypotheses and extend knowledge on information systems socio-technical theory and the comparative theory of competitive advantage. 6.3 limitations the study has some limitations. while efforts were made to take into account as many factors as possible to build a conceptual model explaining the relationships among these factors and the failure of foreign internet firms, the model does not present an exhaustible list of factors australasian journal of information systems ram & zhang 2018, vol 22, research article an investigation into failure of internet firms 18 and the corresponding relationships. another limitation of the study comes from the lack of depth of academic literature on the subject, which could hinder creation of a more comprehensive model of factors driving the failure of foreign internet firms. also due to the fast-paced developments of internet in china, the consumer behaviors and habits are changing frequently which could also require changes to the configuration of relationships and updating of the model. finally, while this study has significantly advanced theoretical knowledge by developing a comprehensive model, however, it remains to be tested in the field. acknowledgements the authors would like to thank the editor and the anonymous reviewer for their constructive comments and insightful suggestions on an earlier version of the paper, and roger tagg for his proof editing of the article. references akin, m. (2016). ‘the relationship between ethnocentric tendencies of turkish consumers and socio-psychological factors’, int. review of management and business research, 5(4), 1394. ackerman, d., & tellis, g. (2001). ‘can culture affect prices? a cross-cultural study of shopping and retail prices’, journal of retailing, 77(1), 57-82. abkowitz, a. & carew, r. (2016). ‘uber sells china operations to didi chuxing’, wall street journal, https://www.wsj.com/articles/china-s-didi-chuxing-to-acquire-rival-uber-schinese-operations-1470024403 (viewed: 31-01-2018) baidu (2017). ‘baidu milestones and investor relations’ http://ir.baidu.com/phoenix.zhtml?c=188488&p=irol-milestones (viewed: 31-01-2018) bostrom, r.p., & heinen, j.s. (1977). ‘mis problems and failures: a socio-technical perspective, part ii: the application of socio-technical theory’. mis quarterly, 11-28. byhlin, h., & holm, e. (2012). ‘entering the chinese e-merging market: a single case study of business model adjustment’, thesis: umeå school of business and economics. carew, r. (2015). ‘investors flock to chinese travel site’, wall street journal eastern edition, 266(28), c1-c3. chen, w. (2013). ‘get your feet wet-a study of entry strategy of dutch architecture firms in china’, thesis submitted to technical university of delft chen, w., & myasin, m. (2008). ‘working with chinese culture in luxury hotels in shanghai: an expatriate manager's perspective on management challenges’, international journal of chinese culture and management, 1(2), 199-218. cheng, w. & liu q. (2016). ‘didi (sharing economy changes china) (chinese edition)’, posts & telecom press, isbn: 978-7-115-42503-4. china bridge. (2013). ‘foreign giants failing to strike gold in china’, china bridge, http://cbichinabridge.com/foreign-giants-failing-strike-gold-china (viewed: 31-01-2018) australasian journal of information systems ram & zhang 2018, vol 22, research article an investigation into failure of internet firms 19 chinadaily (2018). 'nearly 69% of chinese netizens use online payment: report,' http://www.chinadaily.com.cn/a/201801/31/ws5a71aeaaa3106e7dcc13a038.html (viewed: 17-02-2018) cnnic (2016). ‘38th statistical report on internet development in china’, https://cnnic.com.cn/idr/reportdownloads/201611/p020161114573409551742.pdf cnnic (2018). ‘41st statistical report on internet development in china’, http://www.cnnic.net.cn/hlwfzyj/hlwxzbg/hlwtjbg/201801/p020180131509544165973.pd f (viewed: 17-02-2018) custer, c. (2015). ‘the 3 biggest reasons foreign companies fail in china’, techinasia. https://www.techinasia.com/3-biggest-reasons-foreign-companies-fail-china degravel, d., wilkin, c.l., & zhou, x. (2016). ‘the fluid paradigm': the impact of national culture on small business management’, journal for international business and entrepreneurship development, 9(1), 62-87. dong, l., & yun-tang, z. (2012). ‘recessive, dominant and direct situation of home products consciousness based on the theory of social approval’ in management science and engineering (icmse), 2012 international conference on (pp. 739-744). ieee. dou, e. & chu, k. (2016). ‘uber's efforts to build chinese business ultimately fail against homegrown rival’, wall street journal, https://www.wsj.com/articles/ubers-efforts-tobuild-chinese-business-ultimately-fail-against-homegrown-rival-1470056431 femi, a.f. (2015). ‘influence of management style on workers’ productivity in nigeria’ machine tools, osogbo, osun state, nigeria. international journal of innovative research and development, issn 2278–0211, 4(4). grewal, r., chandrashekaran, m., & dwyer, f. r. (2008). ‘navigating local environments with global strategies: a contingency model of multinational subsidiary performance’, marketing science, 27(5), 886-902. greeven, m. (2012). ‘the case study: how taobao bested ebay in china’, the financial times, march 13, 2012, 10. han, y. (2015). ‘three challenges facing airbnb in china’, business insider, http://www.businessinsider.com/three-challenges-facing-airbnb-in-china-2015-8 hoffman, r.c., & preble, j.f. (2004). ‘global franchising: current status and future challenges’, journal of services marketing, 18(2), 101-113. hunt, s.d., & morgan, rm. (1995). ’the comparative advantage theory of competition’, the journal of marketing, 1-15. jin, c. (2016) ‘bao pin zhan lue’, beijing: beijing united publishing co., ltd. jurse, m., & jager, j. (2014). ‘complexities of the firm marketing channel strategy management in international markets’, in an enterprise odyssey. international conference proceedings (p. 1208). university of zagreb, faculty of economics and business ke, x. & sun, q. (2014). ‘study of csf in china’s b2c e-commerce based on expert evaluation’, journal of electronic commerce in organizations, 12(2), 31-45. australasian journal of information systems ram & zhang 2018, vol 22, research article an investigation into failure of internet firms 20 kwak, j., lee, h., & chung, d.b. (2012). ‘the evolution of alliance structure in china’s mobile telecommunication industry and implications for international standardization’, telecommunications policy, 36(10), 966-976. law, r., to, t., & goh, c. (2008). ‘how do mainland chinese travelers choose restaurants in hong kong?: an exploratory study of individual visit scheme travelers and packaged travelers’, international journal of hospitality management, 27(3), 346-354. lee, m. (2010). ‘revisiting the 'google in china' question from a political economic perspective’, china media research, 6(2), 15-24. li, j., yang, j.y., & yue, d.r. (2007). ‘identity, community, and audience: how wholly owned foreign subsidiaries gain legitimacy in china’, academy of management journal, 50(1), 175190. marquis, c. & yang, z. (2014). ‘learning the hard way: why foreign companies that fail in china haven’t really failed’, china policy review, 10, 80-81. https://ecommons.cornell.edu/bitstream/handle/1813/40209/failures_cpr.pdf;sequence =4 my, n.n.q., kim, p.s., & sayim, m. (2015). ‘initial performance of ipos and the bankruptcy risk: a comparison of internet firms and the traditional firms’, research in business and management, 2(2), 34-51 oh, l. & zhang, y. (2010). ‘understanding chinese users’ preference for domestic over foreign internet services’, journal of international consumer marketing, 22(3), 227–243. oliver. (2015). ‘why do international companies fail in china?’, http://marketingtochina.com/why-do-international-companies-fail-in-china ou, c.x. & davison, r.m. (2009) ‘why ebay lost to taobao in china: the global advantage’, communications of the acm, 52(1), 145-148. piekkari, r., vaara, e., tienari, j., & säntti, r. (2005). ‘integration or disintegration? human resource implications of a common corporate language decision in a cross-border merger’, the international journal of human resource management, 16(3), 330-344. rangan, s., & drummond, a. (2004). explaining outcomes in competition among foreign multinationals in a focal host market. strategic management journal, 25(3), 285-293. selmer, j., & de leon, c.t. (2002). ‘parent cultural control of foreign subsidiaries through organizational acculturation: a longitudinal study’, international journal of human resource management, 13(8), 1147-1165. shin, m. (2004). ‘convergence and divergence of work values among chinese, indonesian, and korean employees’, in management and international review (pp. 105-129). shkurti, r., & leka, b. (2010). ‘development of the accounting profession in albania in the new era of market economy’, studies in business & economics, 5(3). sleuwaegen, l. (2013). ‘scanning for profitable (international) growth’, journal of strategy and management, 6(1), 96-110. sofka, w., & zimmermann, j. (2005). ‘there's no place like home: a strategic framework to overcome liability of foreignness in the german car market’, zew discussion papers, 05-84 australasian journal of information systems ram & zhang 2018, vol 22, research article an investigation into failure of internet firms 21 somerville, h. & bellon, t. (2016). ‘airbnb’s legal challenges get real’, time.com, http://time.com/money/4541529/airbnb-legal-challenges/ (viewed: 31-01-2018) stevenson-yang, a. (2007). ‘the curse of the chinese internet’, far eastern economic review, 170(3), 41-44. walcott, s. (2014). ‘capitalist china comes to the southeastern united states: localizing foreign direct investment in the carolinas and georgia’, southeastern geographer, 54(3), 291-307. wang, x. (2012). ‘foreign direct investment and innovation in china’s e-commerce sector’, journal of asian economics, 23(3), 288–301. wendt, k. (2015). how does political, economic and social transformation in china challenge the market for mncs, and to what extent does this affect the strategic focus of mncs. master’s thesis copenhagen business school. xinhua insight (2015). ‘xinhua insight: sharing economy shaking up business in china’, http://news.xinhuanet.com/english/2015-08/25/c_134554335.htm (viewed: 31-01-2018) ye, q., xu, m., kiang, m., wu, w., sun, f. (2013) ‘in-depth analysis of the seller reputation and price premium relationship: a comparison between ebay us and taobao china’, journal of electronic commerce research, 14(1), 1-10. zhou, t., lu, y., & wang, b. (2013). ‘a comparative analysis of chinese consumers’ increased vs. decreased online purchases’, in e-commerce for organizational development and competitive advantage (pp. 40-57). igi global. copyright: © 2018 ram & zhang. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 a critical perspective of relevant literature 2.1 motivation for foreign internet firms to operate in china 2.2 reasons for failure: cultural, linguistic, and psychological differences 2.2.1 ease of communication with the sellers help generate trust 2.2.2 ease and modes of payments 2.2.3 risk avoidance 2.2.4 localization of shopping environment 2.2.5 perceived globalness 2.2.6 patriotic tendencies 2.3 reasons for failure: strategic and operational approaches 2.3.1 sustainable business model 2.3.2 operating strategies 2.3.3 network effect 2.4 political, legal, and social reasons 2.5 summary of the literature review 3 theoretical underpinning 4 development of a conceptual model 4.1 internally driven sources 4.1.1 strategic misalignment 4.1.2 lack of internal preparedness 4.1.3 lack of environmental awareness 4.1.4 foreign organizational culture 4.1.5 foreign management style 4.1.6 lack of business model adjustment 4.1.7 lack of adjustment to dynamics of business eco-system 4.2 externally imposed sources 4.2.1 competitors’ strategic leverage 4.2.2 regulatory constraints 4.2.3 socio-cultural influences 4.2.4 language preferences 4.2.5 patriotic tendencies 5 discussion and conclusion 6 implications, future directions and limitations 6.1 implications for theory and practice 6.2 future research work 6.3 limitations justifying investment in information resource management daniel l. moody & graeme c. simsion simsion bowles and associates, i collins st, melbourne, 3000. telephone: (03) 650 5100 abstract information resource management (irm) is the management of information as a corporate resource rather than as the property of individual systems or users. while many organisations have established irm functions to address this goal, there has been little formal measurement of their effectiveness. a recent survey of irm groups in the u.s. showed that only 13% of groups have implemented procedures for measuring their performance. such measures are essential for justifying the existence of the group, maintaining management support and continuously improving irm practices. this paper discusses the need to measure irm effectiveness, identifies key criteria for a measurement program, and proposes some candidate metrics. further research is in progress to refine these metrics and empirically validate them with practitioners. introduction information resource management the goal of information resource management (irm) is to manage information as a corporate resource, in the same way that other organisational resources such as finance, personnel and property are managed (henderson, 1987). in the absence of such explicit management, different parts of an organisation are likely to collect their own copies of information and store them in different formats. the costs of an uncoordinated approach include duplication of capture, storage and update effort, and difficulty in consolidating information for reporting. the terms data resource management (english, 1992), data administration (hufford, 1991) and data management (goodhue et al, 1992) are also used for both the philosophy of irm and the group responsible for implementing it. irm was ranked as the number two information systems management issue in a survey of fortune 500 is executives in the u.s. (neiderman et al, 1991). despite the conceptual appeal of irm, it has had a chequered history in practice. many organisations that have attempted to implement irm have either failed or experienced major difficulties (kahn, 1983; tillman, 1987; goodhue et al, 1988, 1992; lederer and sethi, 1991). one explanation for this failure is that it fails to deliver tangible benefits to the organisation and quickly loses management support. according to simsion (1991), irm objectives are often unclear, activities unfocussed or poorly aligned with objectives, and survival of the irm group is more often a result of "selling" irm concepts than of demonstrated benefits. state of measurement in practice while there is considerable literature on irm techniques and their potential benefits (eg mckenney and mcfarlan, 1982; martin, 1989; katz, 1990; kerr, 1991; hancock, 1992; hufford, 1993; english, 1993; love, 1993), there has been little serious attempt at quantification of these benefits. there are few published cases in the literature of successful irm efforts, and very few, if any, with benefits quantified in dollar terms. "success stories" which are regularly presented at industry forums such as ibm's share-guide (eg hancock, 1992) or the data administration and management association (dama), seldom report quantitative benefits, and usually represent the only the perspective of the irm group, which has a vested interest in being perceived as effective. a recent survey of irm practitioners in the u.s. (figure 1) showed that only 13% of organisations had implemented procedures for measuring irm performance (english and green, 1991). this was less than the percentage found in a similar survey carried out in 1987. 76% of the respondents had no performance measurement programme in place nor any intentions of establishing one in the near future. september 1995 25 currently measuring _ results no measurement 76% ~ intend to establish measurement programme within the next year 11% figure 1. measurement of irm benefits (source: english and green, 1991) english and green concluded: "although these responses could be interpreted as indicating that irm is not producing the desired results, or that it is too soon to determine what the impact is, the fact that so little performance measurement has been implemented suggests that the effects of irm, desired or undesired, actually are not known." in the absence of any formal measurement of the impact of irm in practice, the assumption that irm adds value to the business remains essentially an unproven one. as strassman (1985) says, "you cannot tell whether you have improved something if you have not measured its performance". aside from purely anecdotal evidence, there is little to say whether irm has had a net positive or negative effect in organisations. reasons for lack of measurement there are a number of problems involved in measuring irm benefits that may explain the lack of measurement in practice. • as with any infrastructure activity, irm aims to add value to other activities rather than delivering benefits directly. measurement requires that the contribution of irm be isolated from other contributions. this is particularly difficult in the situation where irm staff and project staff are working closely together, and the measurement program requires that only one of the groups be credited with an achievement. • another problem is that the benefits of irm accrue to the organisation as a whole, and are not always visible as improvements in the short term performance of individual business units. many organisations find it difficult to justify expenditure that cannot be related to specific projects or business areas and which will not produce benefits until some time in the future (butler cox, 1991). • a final problem is that many of the benefits claimed for irm are intangible or poorly defined. terms such as "integration", "flexibility", "data availability" and "re-use" require firm definition before they can be measured. the study by english and green (1991) found that the benefits of irm cited most frequently were also the least tangible: system quality, system flexibility and user satisfaction. more measurable benefits, such as reduction in development costs, or reduction in end user data entry were rarely reported by the survey respondents. another reason for the lack of measurement may be that irm practitioners do not think that it is important. a gartner group survey showed that only 12% of irm professionals surveyed saw benefit identification as a major challenge to implementing irm in an organisation (gartner, 1992). this could be because most irm managers are from an it background rather than a business background. brancheau and wetherbe (1987) found that business executives consistently considered measurement to be more important than is executives. irm practitioners often place far more emphasis on logical arguments rather than economic justification. the lack of measurement of benefits in the irm area is indicative of a general problem in it. it is generally agreed that current is measurement practices are inadequate and that further development of measures of is performance are badly needed (saunders and jones, 1992). as merlyn (1992), says: "in general, the state of measurement and quantification in mis is woefully inadequate. most justification today requires an abundance of faith." why measuring the effectiveness of irm is important measurement provides a means for particular functions within an organisation to quantify and improve their value to the organisation. effective measures serve as a framework for defining shared goals and for 26 ajis communicating how the goals of each function support organisational goals (walrad and moss, 1993). measurement of irm effectiveness is important both from the perspective of individual organisations, who need to assess the return on their investment in irm, and from the broader perspective of establishing sound industry practices. justifying initial investment in irm in the past, it may have been sufficient to demonstrate the apparent logic of properly managing information, or to cite the results of past mis-management, in order to convince an organisation to invest in irm. there is a major emphasis both in the literature and in practice on "selling" the benefits of irm. as one irm manager said, "... the marketing of irm is a continual necessity, particularly in the light of the difficulty in quantifying its benefits." (green, 1989b). such approaches are becoming less effective in the absence of well-documented success stories, and the requirement for sound quantified business cases to support all it investment. a study of 31 organisations implementing irm carried out by goodhue et al (1988) showed that the successful companies in the sample justified irm not by conceptual or technical arguments but by compelling business needs. survival of the irm group in today's economic climate, organisations are looking to maximise return on investment in all areas of the business, and to cut back in areas that are not directly contributing to profitability. usually "infrastructure" groups, such as irm, are the first victims of cost cutting exercises (simsion and drummond, 1990). because they do not contribute directly to the task of delivering systems, they are seen as expendable in times of crisis. the existence of formal measurement of results acts as an effective insurance policy against such a demise. management support lack of management support is consistently cited by irm practitioners as the major reason why irm does not succeed and as the major obstacle to achieving ongoing success (english and green, 1991; martin, 1989). the exercise is more likely to fail from a lack of management commitment than from any technical reasons. perhaps management are right in withdrawing support for a group which cannot justify its value to the organisation. management will not continue to support a concept that cannot demonstrate measurable benefits to the firm. clarifying goals irm groups are often established with a rather vague charter, for example "to manage data as a corporate asset", or "to promote integration and sharing of data throughout the organisation". however it is very difficult to translate slogans into action. as a result, in spite of large amounts of effort and good intentions, they end up achieving very little. without clearly defined goals and ways of measuring progress against these goals, irm groups often lack direction and focus. just the process of defining goals and measures will help irm groups to define what their charter really means, and clarify the role and function of irm in the organisation. focussing irm investment measurement of results is fundamental to ensuring that activities are concentrated in areas of most benefit (darcy, 1994). an irm group may take a variety of initiatives all aimed at improving the management of information: eg establishing standards for data formats, reviewing database designs prior to implementation, implementing a repository. it is important to be able to measure the effect of each initiative in order to focus resources where they are providing greatest benefit, and to identify problem areas. for example, effort expended on integration should be proportional to the need for integration: in some areas, overheads may be higher than the benefits (goodhue et al, 1988, 1992). ensuring conformance one of the major obstacles to the success of irm in practice is ensuring that development projects conform to the enterprise data model. by itself, an enterprise data model is nothing more than an idealised picture of how an organisation's data should be structured. it becomes valuable only if it is used in practice. many enterprise data models are ignored after they are completed and the expense put into developing them is wasted (butler cox, 1988; ibm, 1992). this is despite conformity to the model being mandated by standards in many september 1995 27 organisations. project teams seeking exemption from the standards usually do so on the basis that compliance will add extra cost or time to the project. the irm manager needs to be able to counter this argument with the corresponding costs of failure to comply. the best way to ensure compliance is to promote the benefits of conformance to the enterprise data model rather than by prohibition or policing. in most cases, business managers and systems developers are prepared to make compromises, provided they have a full understanding of the benefits (butler cox, 1991). improvement of irm practices measurement is essential for the improvement of processes, whether incremental (eg total quality management approaches) or radical (eg business process re-engineering). simsion (1991) suggests that many irm groups do little to review or improve their processes after initial establishment, possibly because of a lack of feedback through measurement. as a result, irm groups operate based largely on "what ought to work" rather than what has been demonstrated to work. because methods for implementing irm are still in their infancy and there are no generally agreed standards or textbooks on the subject, irm groups should adopt an adaptive approach based on experience in practice. irm groups should review their progress every 6 months or so, and modify their approach if necessary. measures are necessary to determine whether the approach is working or not and to identify areas for potential improvement. requirements for irm performance measures there are a number of key requirements which must be satisfied by any metrics proposed for measuring irm effectiveness. need for externally relevant measures measures of irm effectiveness need to be meaningful to key stakeholders outside the irm group. measures such as the number of objects stored in the repository, average number of standards violations to repository objects, the number of subject databases implemented and number of accesses to subject databases per month are often proposed as measures of irm effectiveness (green, 1989a). while such measures may assist in the internal management of irm, they do not assist in determining the value of irm in meeting it or business goals. in developing measures, it is important to ask: who are you measuring for? who will ultimately decide whether the irm group has been effective or not? creating measures which cannot be understood by people outside the irm group defeats the purpose of measurement, which is to justify the value of irm to the organisation at large. one of the best ways to ensure the relevance of measures developed is to involve key stakeholders in the process of establishing measures. need for quantifiable benefits for maximum effectiveness in getting management support, the benefits of irm should be expressed in financial terms. this is not always possible, but they should be quantified as far as is reasonable. a common complaint by irm practitioners is that benefits are intangible and cannot be quantified (green, 1989a). while there is no doubt that quantification of irm benefits is difficult, so is justifying expenditure on an advertising campaign, or on research and development, but with some effort it can be done. in most cases establishing measures for intangible benefits is easier than anticipated (rockart, 1982). while "hard" quantitative data should form the major part of the measurement process, "soft" measures such as user satisfaction ratings should not be ignored. it is important not to manage on visible figures alone (zultner, 1992). it may, in some cases, be more appropriate to express some measures in terms of an organisation's key performance indicators, particularly in the case of non-profit organisations whose primary goals may be nonfinancial. need to consider both it and business benefits irm is unlikely to succeed unless it is seen by top management as critical to the strategic goals of the organisation. this means that irm must be justified in terms of its relevance to business goals, or support will evaporate when the true costs become apparent, a major project is delayed or when some other business issue diverts their attention. it takes more than recognition by an it group that integration of data across the organisation is important (goodhue et al, 1992). 28 ajis theoretical model, but in determining appropriate indicators and standards. it is clear from the literature that irm practitioners have some notion of what they are trying to achieve and therefore what constitutes effective performance. however these notions are very difficult to operationalise. this paper draws together a large number of research efforts in specific areas (eg. reuse, quality assurance) in order to produce a set of measures which are justified in the literature. this set of metrics is not intended to be complete, but as a starting point, to be refined and extended by further research. goal #1: information value as the irm group is the custodian of the organisation's information resource, the ultimate measure of their success should be the "value" of this resource to the organisation in terms of increased revenue or reduced costs. the result of all their efforts should be to increase the value of the information resource by expanding it, enriching it and finding new ways of exploiting it for business advantage. one of the familiar catch phrases of irm is that "information is an organisation's most valuable asset and should be managed in the same way as other assets". however unlike other assets, there has been little attempt in most organisations to quantify the value of information. according to glazer (1993), establishing a procedure for doing so is the primary mechanism by which a firm can integrate it strategies with business strategies, and successfully use it for competitive advantage. proposed metrics: 1. current value of information ($). glazer (1992) describes a method for measuring the value of information, using managerial judgement to estimate the increased revenues or reduced costs to the organisation as a result of having the information (as compared to not having it). a decision calculus methodology is used to assist the managers in arriving at this estimate through an iterative series of questions and responses. 2. potential value of information ($). glazer (1992) defines a method for estimating the potential value to the organisation of using information (which already exists in the firm or could be obtained) in ways that have not yet been realised in practice. 3. information gap ($). this represents the difference between the current actual value of information and the potential value of information. the firm should direct its it efforts to closing this information gap. goal #2: information quality information quality is defined as the accuracy and timeliness of information produced by information systems. information quality was ranked third in a study of measures of overall it performance (saunders and jones, 1992). increasing information quality reduces the amount of time spent resolving data discrepancies, and improves the value of information as a decision making resource (hufford, 1993). decisions made based on incomplete, incorrect or out of date information can be very costly to the organisation (love, 1994). in practice, many important decisions end up being made based on "guesstimates" rather than facts, because information is not available in time to act on it. proposed metrics: 1. end user surveys of data quality (saunders and jones, 1993). 2. customer/client surveys of data quality (saunders and jones, 1993). these are individuals external to the organisation who receive information outputs from the organisation eg. statements, invoices. 3. percentage/size of errors based on statistical sampling of information in databases (simsion and drummond, 1992; morgan, 1994). this metric is directly useful to the business, by giving them an indication of the margin for error they should allow for in their decision making. in most cases, decision makers do not know how reliable the information is and therefore cannot give a confidence level in their decisions. 4. estimated business cost of errors ($). these costs can be quantified in terms of operational costs (eg. cost of service call to incorrect address), loss of business (eg. as a result of incorrect billing), or the cost of inaccurate decisions (eg. pricing margins). 30 ajis goal #3: information accessibility one of the goals of the irm group should be to increase the accessibility of information across the organisation. unlike other organisational resources, information is not appropriable, is not depletable (though is often perishable) and generally increases in value the more it is used (glazer, 1992). in most organisations, there is an enormous amount of existing computer based information that could be used to advantage by the business. research on competitive edge applications by butler cox (1991) shows that the majority of such applications arose from new ways of using information stored in existing databases. however the opportunities are not realised because few people are aware of what is available. accessibility to information is enabled by the irm group through the provision of an information catalogue or "data inventory", which maps business definitions of data contained in the enterprise model to databases and files (devlin and murphy, 1988; hufford, 1993; butler cox, 1988). this information can be used by both systems developers and users and is one of the ways that an irm group can provide immediate and tangible benefits to the organisation. proposed metrics: 1. average time taken by end users to locate relevant data. use of an information catalogue can reduce the time taken by users to locate relevant data from weeks to hours (butler cox, 1988). 2. average time taken by developers to locate and extract source data for mis and eis. one of the major problems in building mis is finding out how to access and interpret "legacy" data in existing systems (giordano, 1993). the information catalogue can reduce time to locate data as well as providing the basis for conversion of data to a common format. 3. estimates of time savings for both users and developers as a result of using the catalogue. 4. percentage of existing databases/files mapped to the enterprise data model (simsion and drummond, 1992). this should include pc applications, software packages and applications developed by end users. this gives an indication of the level of completeness of the information catalogue, and is also a measure of the irm group's progress in documenting the organisation's existing data. goal #4: organisational integration organisational integration is defined as the ability of an enterprise to coordinate operational activities and to make organisation wide responses to business problems. porter and miller (1985) define an organisation's value chain as a system of interdependent activities which are connected by linkages. careful management of these linkages is a powerful source of competitive advantage because of the difficulty competitors have in perceiving it. according to stalk (1988), gaining competitive advantage almost always requires integration of business activities. improving downstream linkages with suppliers and upstream linkages with customers also provides opportunities for competitive advantage (porter and miller, 1985). a prerequisite for improved organisational coordination is an enhanced ability to communicate within or among organisational units (goodhue et al, 1988). the irm group can assist in this by ensuring that data have the same meaning and use across time and across users. this effectively provides a common language for communicating about business events, which is essential for coordinating diverse and far-flung units of the organisation (goodhue et al, 1992). sharing of information between different business areas leads to improved coordination and communication across organisational functions. this increases the opportunity for synergistic cooperation to streamline business processes and cut costs in ways that increase business effectiveness (hufford, 1993). proposed metrics: 1. information float. this is the time delay from when one area of the organisation captures a fact of information until others in the information value chain are able to access it (english, 1993). usually this time delay is the result of having to re-enter data into different systems or for interface programs to copy or translate the data from one system to another. at each stage in this process integrity problems may be introduced. 2. turnaround time to customers and suppliers. this is the average time it takes for customers or suppliers to receive a response to a query or a transaction. 3. cost impact of turnaround time ($). a method for quantifying the costs of organisational response time is to conduct a statistical survey of customers to determine the extent to which this feature might affect their decision to purchase. with this information, it is possible to project the impact on sales (cougar, 1987). september 1995 31 goal #5: data model quality as part of the process of ensuring conformance to the enterprise model, one of the roles of the irm group is to conduct quality reviews of all project data models. quality assurance of data models decreases development costs because the sooner an error is found and corrected the less costly it will be (moody and shanks, 1994). research shows that the cost of rectifying defects post implementation is between 100 and 10,000 times as much as fixing the problem that caused them during analysis (index foundation, 1993). proposed metrics: 1. requirements error distribution per development phase ($): this measures the occurrence and cost of requirements errors in each phase of development. this can be monitored over time and against industry averages. studies have shown that more than half the errors which occur during development are the result of inaccurately defined requirements (martin, 1989). 2. requirements change distribution per development phase ($): this measures the occurrence and impact of requirements changes in each phase of development. the cost of adding to or changing requirements increases exponentially over the development lifecycle (brooks, 1978; butler cox, 1987). the number of changes required after analysis reflects how well requirements were defined to start with. 3. estimated cost savings as a result of early detection of errors ($): this is calculated by looking at the total cost of requirements errors and changes compared to pre-irm figures or industry averages. research shows that defect detection during the analysis phase is up to 33 times more cost effective than testing at the end of development (russell, 1991). 4. cost of maintenance changes ($). the irm group should ensure that data is organised in a way that reduces the need for future restructuring as a result of requirements changes or new and unanticipated processing requirements (simsion and drummond, 1992). experience shows that the most expensive changes of all are those which require changes to the database structure, because each such change has a "ripple effect" on all the programs that use it (simsion, 1994). comparison of maintenance costs between systems developed with irm involvement and without over time will show the impact of irm practices on maintenance. empirical studies have shown reductions of 80% in maintenance costs of systems developed in an irm environment (hufford, 1993). the measures proposed here may be used to monitor improvements in development productivity over time. however these measures are even more useful for assessing the effectiveness of irm if they are collected separately for systems developed with irm involvement and those developed without (eg. prior to the irm function being established). this allows the impact of irm on development productivity to be more precisely determined by effectively providing an experimental "control group". goal #6: reuse of data a major role of the irm group in the development process is to identify opportunities for data reuse. reuse is one of the most direct, powerful and quantifiable measures of irm benefits, and represents one of the major opportunities for organisations to increase their application development productivity (index foundation, 1993). empirical research shows order of magnitude increases in development productivity (up to 300%) as a result of a strategy of systematic reuse across an organisation (banker and kaufmann, 1991). proposed metrics: 1. reuse percentage (%): average percentage of data reused per project. once there is a "critical mass" of data in the enterprise model, new projects should require minimal new development and will tend to just reuse and refine existing data and processes. studies in practice have shown that after only two years, projects require only 25% new objects on average (banker and kaufmann, 1991). this translates into a reuse percentage of 75%. 2. potential reuse savings ($): this can be estimated by converting the data and processes reused to function point form and then multiplying this by the average cost per function point for the organisation. in practice, the amount of effort required should be less than this as a result of the scale diseconomies inherent in systems development (banker and kaufman found that 1% reuse generally leads to 1.9% reduction in effort). 3. actual reuse savings ($): this is the difference between the estimate of project costs based on total function points less the actual project costs. there are a number of reasons why the actual savings will differ from potential savings. one is the margin for error in all estimates of effort. the other is that the project may decide to build their own version of the common data rather than reusing it at the physical level. 32 ajis goal #7: management of redundancy redundancy refers to the duplication of the same information in different databases or files. the goal of the irm group is to minimise redundancy of data through data sharing. a second goal is to minimise the impact of redundancy on the user by ensuring that different copies of data are synchronised. a certain amount of data redundancy is unavoidable in most organisational contexts. the enterprise model, when mapped to existing systems, provides the mechanism for keeping track of all copies of a piece of business data, and ensuring that appropriate data movement methods exist to keep copies consistent (butler cox, 1988). controlled redundancy is where the synchronisation of different copies is handled automatically by the dbms or application programs. uncontrolled redundancy is where information must be manually updated in different locations (date, 1989). controlled redundancy represents an overhead to the organisation in terms of storage costs and synchronisation costs. uncontrolled redundancy represents a threat to data quality (as a result of copies of data becoming inconsistent) as well as requiring the same data to be entered separately into different systems. proposed metrics: 1. average number of copies of corporate data. this is determined by mapping objects in the enterprise data model to files and databases. this should include pc applications, software package and user developed systems (eg. spreadsheets). it is doubtful whether many organisations know the true extent of data duplication that exists. it is likely to be much higher than anyone imagines but is rarely quantified in practice. love (1994) reports a study carried out in one organisation where it was found that there were, on average, 25 copies of every piece of corporate data. clearly, the cost to the organisation of storing this information and keeping it up to date would be enormous. 2. cost of storage of redundant data ($). this can be calculated based on the total amount of redundancy multiplied by the unit cost of physical storage. 3. cost of redundant data entry ($). this can be estimated using standard data entry costs. multiple data entry represents an ongoing cost to business areas and a major source of irritation to users. one insurance company found that it was storing a single fact of data 43 times, and that 43 different people within the enterprise were creating and maintaining the same data independently (english, 1993). this measure applies to uncontrolled redundancy only. 4. synchronisation costs ($). this is the cost of keeping different copies up to date in the case of controlled redundancy. this can be estimated using a standard transaction cost multipled by the update frequency (simsion and drummond, 1992). 5. overall cost of redundancy ($): this is the total cost of redundancy to the organisation, calculated by adding together the three previous measures. calculating the overall cost of redundancy to the organisation provides an excellent business case for irm. however very few, if any irm groups attempt to quantify the cost, despite its obvious utility as a means for their justification. it is an expensive and time consuming exercise to try to do this across all the organisation's data stores, but sampling techniques can be used to estimate the overall cost. 6. level of inconsistency (%). this represents the variance between different copies of the same data using statistical sampling. this will depend on whether the redundancy is controlled or uncontrolled, volatility of the data, and frequency of synchronisation. these measures should be used to assess the current cost of redundancy to the organisation and then to measure improvements to the situation as a result of irm initiatives. goal #8: systems integration systems integration is defined as the ability of different systems to communicate with each other without the need for explicit translation or conversion of data. incompatibility of data definitions between systems means that it is difficult to access or consolidate information across application boundaries. the result of this is that it is often quicker to rekey information produced by one system so that it can be used by another (butler cox, 1988). irm can help to improve integration of systems by ensuring common data naming, definitions and formats across different information systems. this simplifies communication between systems and removes the need to build explicit interfaces between them. proposed metrics: 1. interface costs ($). irm should greatly reduce costs of building interfaces between systems. empirical research shows that these savings are of the order of 85% (hufford, 1993). the cost of developing, september 1995 33 operating and maintaining these interfaces can be included in this metric (simsion and drummond, 1992). 2. manual costs of consolidating data from different systems ($). in a non-integrated environment, data from different applications can only be merged and consolidated with significant manual intervention, translation and discrepancy resolution. this is a hidden cost of non-integration which is borne by business users. a division of a fortune 500 company found that 46% of its total resources were dedicated to deriving information from data (hufford, 1993). by standardising data definitions and formats, irm activities can reduce this work effort required. 3. data translation processing costs ($). studies by ibm have shown that upwards of 70% of processing time in large systems is spent translating data formats and structures between applications (ibm, 1993). by providing common definitions of data across systems, the translation effort required should be minimised through irm practices. 4. cost of extracting data for mis and eis ($). irm can help to simplify the collection, comparison and aggregation of data for management reporting as a result of common data definitions across operational systems. the major overhead in building mis and eis is in building interfaces to extract data from operational systems and convert it to a common format. it is estimated that this represents 90% of the cost of such systems (inmon, 1992). application in practice selection of metrics it is not expected that an irm group would use all of these metrics in implementing a measurement programme. because each irm group has unique circumstances, finding a single, comprehensive set of measures that will suffice for the evaluation of the performance of all groups is unlikely (green, 1989a). the metrics proposed here are intended to be used by organisations as a starting point for developing their own measures, by selecting those which are most appropriate and devising new measures if necessary. selection of appropriate measures to use should take place based on perceptions of the value of the measure, as well as the difficulty of obtaining the information. many of the measures defined are dependent on particular information being collected as part of the development process (eg. number of defects, cost of defects). if such recording is not part of is procedures, alternative measures will need to be used. a team approach is highly recommended to determine the metrics to use. involvement of all stakeholders (application developers, users, senior management) in the definition of appropriate measures will result in better organisational understanding of the aims of the irm group, and a cooperative approach to solving the information management problems of the organisation. implementing an irm measurement programme the collection of performance measures may seem like a major operational overhead to the average irm manager, which may compete for already scarce resources that could be used to solve problems of a more immediate nature. however for the long term survival of irm in an organisation, the question that needs to be asked is: can you afford not to? too often irm groups defer establishing the necessary information collection systems until senior management requests justification for the existence of the group or funding for a particular irm project. by the time such a request is made, it is often too late to gather the information that will prove the value of the group (green, 1989a). it is never too early to begin measuring results. however the task of collecting measures across the entire collection of organisational data sources is too large to tackle at once, and should be attempted in stages, based on organisational information priorities. manageable subsets of data may be defined so that the task can be approached in an incremental manner. for example, customer data may be measured first, then product data, account data and so on. given the value of such information for gaining and maintaining support, it is reasonable that an irm manager devote a significant portion of his resources (eg. 10 percent) to the establishment of measurement procedures and evaluation of results (green, 1989a). 34 ajis summary of metrics the table below summarises i goal the goals and metrics proposed in this paper: metric(s) information value information quality information accessibility organisationa 1 integration data model quality reuse of data management of redundancy system integration current actual value of information ($) potential value of information ($) information gap ($) end user surveys customer/client surveys percentage/size of errors based on statistical sampling estimate of business cost of errors ($) average time taken by end users to locate relevant data average time taken by developers to locate and extract source data estimated time savings for end users and developers ($) percentage of databases and files mapped against enterprise model information float turnaround time to customers and suppliers cost impact of turnaround time ($) requirements error distribution per development phase ($) requirements change distribution per development phase ($) estimated cost savings as a result of early detection of errors ($) cost of maintenance changes ($) reuse percentage potential reuse savings ($) actual reuse savings ($) average number of copies of each piece of corporate data cost of storage of redundant data ($) cost of redundant data entry ($) synchronisation costs ($) overall cost of redundancy ($) level of inconsistency (%) interface costs ($) manual costs of consolidating data from different systems ($) data translation processing costs ($) cost of extracting data for mis/eis ($) figure 3. table of irm goals and metrics conclusion and further research implications for irm practice use of the metrics proposed in this paper will help irm practitioners in a number of ways. firstly, it will help irm groups to justify their value to the organisation in a tangible way, and to develop and maintain ongoing management support. it will also help to ensure the success of the irm effort through communication of the benefits rather than through prohibition or policing. it can also help the irm group to identify those areas in the organisation where efforts may prove most useful, and provide the basis for continuously improving and adapting their strategies. irm measurement can also help to develop better measures of the overall effectiveness of the it function. current notions of it effectiveness measurement concentrate on operational efficiency and immediate problems hampering these operations. the irm measures defined here focus on the information itself rather than the technology, which is the key to successfully aligning it with business strategies (glazer, 1992). september 1995 35 further research research is currently in progress to empirically validate the goals and metrics defined with expert practitioners. the set of goals and metrics defined in this paper will be used as a starting point for developing a "consensus" set of irm goals and metrics, agreed across a wide range of experts, including application developers, irm practitioners and business users. it is also the subject of further research to extend the framework by exploring strategies which may be used to achieve irm goals. for example, building subject databases (martin, 1989) or reference tables (simsion, 1994) may be used to reduce redundancy of data. references apte, u., sankar, c.s., thakur, m., and turner, i.e. (1990) "reusability based strategy for development of information systems: implementation experience of a bank", mis quarterly, december. banker, r.d., and kauffman, r.j. (1991) "reuse and productivity in integrated computer aided software engineering: an empirical study", mis quarterly, september. basili, v.r. and rombach, h.d. (1987) "tailoring the software process to project goals and environments", proceedings of 9th international conference on software engineering. brancheau, j. and wetherbe, j. (1987) "key issues in information systems management", mis quarterly, january. brooks, p.p. (1975) the mythical man-month: essays on software engineering, addison-wesley, new york. butler cox foundation (1988) managing the evolution of corporate databasesresearch report 64, butler cox and partners limited, london. butler cox foundation (1991) managing technical architecture, research report 83, butler cox and partners limited, london. cameron, k.s., (1986) "effectiveness as a paradox: consensus and conflict in conceptions of organisational effectiveness" , management science, 32,5. cougar, j.d., (1987) 'techniques for estimating system benefits" in galliers, r., information analysis: selected readings, addison-wesley. darcy, p., (1994) "establishing best practice for it", australian computer society annual conference (victorian branch), lome, march 25-27. date, c.j., (1989) introduction to database systems, 4th edition, addison wesley. devlin, b.a. and murphy, p.t. (1988) "an architecture for a business and information system", ibm systems journal, vol. 27, no. 1. english, l.p. (1993) "drm today, drm tomorrow", database programming and design, november. english, l.p. and green, c.w. (1991) "results of the 1991 advanced irm survey part i", database newsletter, vol. 19, no. 6, november/december. gartner group (1993) "key challenges in implementing architectures", gartner group research note, march. giordano, r. (1993) "the information 'where?' house", database programming and design, september. glazer, r. (1991) "marketing in an information intensive environment: strategic implications of knowledge as an asset", journal of marketing, 55, no. 4,. glazer, r. (1993) "measuring the value of information: the information intensive organisation", ibm systems journal, vol 32, no 1. goodhue, d.l., kirsch, l.j., and wybo, m.d. (1992) "the impact of data integration on the costs and benefits of information systems", mis quarterly, september. goodhue, d.l., kirsch, l.j., quillard, j.a. and rockart, j.f. (1988) "managing the data resource: a contingency perspective", mis quarterly, september. green, c.w. (1989a) "the importance of measuring results", database newsletter, vol. 17, no. 1, january/february. green, c.w. (1989b) "management commitment to data administration: a report from the front lines", database newsletter, vol. 17, no. 2, march/april. hancock, j.k. (1992) "information management: necessity and mission", international share-guide conference, san francisco, march. henderson, m.m. (1987) "the importance of data administration in information management", information management review, spring,. hufford, d. (1993) "data administration support for business process improvement", database newsletter, september/october. 36 ajis ibm (1992) the information framework: financial services data model description, emea banking solution centre, dublin. index foundation (1993) how to succeed in reducing systems development costs, index foundation special report, csc index, london. inmon, w.h. (1992) "eis and the data warehouse", database programming and design, november. kahn, b.k. (1983) "some realities of data administration", communications of the acm, october. kerr, j. (1991) the information resource management imperative, john wiley and sons, new york. klir, g.j. (1985) architecture of systems problem solving, plenum press. lederer, a.l. and sethi, v. (1988) "the implementation of strategic information systems planning methodologies", mis quarterly, vol. 12, no. 3, september. lientz, b.p. and swanson, e.b. (1980) software maintenance management: a study of the maintenance of computer application software in 487 data processing organisations, addison-wesley, reading. love, b. (1994) enterprise information technologies, van nostrum reinhold, new york. martin, j. (1989) strategic data planning methodologies, prentice hall. mckenney, j.l. and mcfarlan, f.w. (1982) "the information archipelago maps and bridges", harvard business review, september-october. merlin, v. (1992) "case implementation: an industry research report", database newsletter, march/april. moody, d.l. and shanks, g.g. (1994) "what makes a good data model? evaluating the quality of entity relationship models", proceedings of the thirteenth international conference on the entity relationship approach, manchester, december 14-17. morgan, j. (1994) "making data quality stick!", australian computer society annual conference (victorian branch), lome, march 25-27. neiderman, f., brancheau, j.c. and wetherbe, j.c. (1991) "information systems management issues for the 1990's", mis quarterly, december. porter, m.e. and millar, v.e. (1985) "how information gives you competitive advantage", harvard business review, july-august. rockart, j.f. (1979) "chief executives define their own data needs", harvard business review, march/april. russell, g.w. (1991) "experience with inspection in ultra-large scale development", ieee software, january. saunders, c.s., and jones, j.w. (1992) "measuring performance of the information systems function", journal of management information systems, vol. 8, no. 4. simsion, g.c. and drummond, h. (1992) "what are we measuring? beyond project based metrics", australian computer society annual conference (victorian branch), march 25-27. simsion, g.c. (1994) data modelling essentials: analysis, design and innovation, van nostrum reinhold, new york. strassman, p. (1985) information payoff: the transformation of work in the electronic age, the free press, new york. tillman, g.d. (1987) "why data administration fails", computerworld, september 7. van genuchten, m. (1992) brethouwer, g., van den boomen, t. and heemstra, f, "empirical study of software maintenance", information and software technology, vol. 34, no. 8, august. walrad, c. and moss, e. (1993) "measurement: the key to application development quality", ibm systems journal, vol. 32, no. 3. zultner, r.e. (1992) "qfd for software: satisfying customers", american programmer, february. september 1995 37 paper title (paper title style) australasian journal of information systems otenyo 2017, vol 21, research on indigenous ict being left behind amidst africa’s rising imagery 1 being left behind amidst africa’s rising imagery: the maasai in the world of information and communication technologies (icts) eric e. otenyo northern arizona university eric.otenyo@nau.edu abstract today the media is replete with stories about africa rising and enjoying middle –income status. those promoting the narrative of africa rising include the world bank and western –trained elites. the narrative of success assumes that prosperity trickles down to indigenous communities, who are pictured consuming mobile phones and tablets. at the same time, hollywood and western media outlets continue to present images of indigenous maasai as a cultural export to be consumed. with emphasis on social media narratives, this paper interrogates the continued marginalisation of the maasai people amidst the myth of a rising africa. in sum, the africa rising myth and penetration of icts in rural areas masks the dispossession of maasai means of livelihood, and therefore worsening the groups’ conditions of living. keywords: maasai; social media; icts; africa rising; indigenous people 1 introduction this paper explores aspects of the critical issues that affect the everyday experiences of the maasai community. scholars in the area of development studies regard inequities in resources between indigenous and non-indigenous communities to be apparent and exacerbated through economic systems that hold back the disadvantaged communities. a hallmark of the new economy is the increased use in information and communication technologies (icts), which include mobile phones, now available among maasai (santos, 2010). although these devices are gaining popularity and have contributed to economic growth, there is evidence that problems exist in the manner in which growth benefits are shared in emerging economies. efforts to improve the living conditions of historically marginalised communities have failed to serve all the disadvantaged groups. one key reason is the structure of the international political economy. by promoting the use of icts in development, policy makers have not sufficiently addressed the global implications of the two opposed forces of, (a) integrating indigenous communities in the increasingly globalised new economies and (b) resolving the long-standing exploitation of wealth belonging to indigenous groups. without striking a balance of meeting the demands of indigenous communities, and encouraging them to be a part of the ict-led emerging economy, it is difficult to see how economic growth and rising incomes in developing countries of africa serve the interest of historically oppressed and neglected groups. the global faith in the ability of icts to communicate the success stories of marginalised communities has to be interrogated along the equally compelling discourse of the desire to promote justice for historical wrongs meted against indigenous groups, including the maasai of east africa. this line of thinking also allows for reflection on the roles of social media and other icts in human rights activism and mobilisation for political and economic development in marginalised communities. the predicament to be discussed revolves around the uses of icts for service to indigenous tribes against the possibility that the benefits from the emerging economic miracles have not done much to change their depravity. this study is abased on accumulated data about how maasai fit in the now established but also contested narrative that africa is rising within a globalised and networked world system. australasian journal of information systems otenyo 2017, vol 21, research on indigenous ict being left behind amidst africa’s rising imagery 2 1.1 methods the study area included kajiado, ngong, and the environs of nairobi. a few maasai people living in western kenya, specifically vihiga county were also interviewed, as a focus group and asked to share their stories concerning the uses of icts and reasons for moving away from their homelands. the discussions followed research protocols and none of the interviews was coerced or bribed to share their stories. in addition, documentary work touching on maasai dispossession was identified and utilised. the paper also draws on historical and cultural data and personal observations by the author, who spent summer 2015 in kenya. major themes embodied in cultural studies include the relationship between identities and the material and social conditions within which maasai live. this research makes meaning of the social realities of maasai people, especially as is shaped by the growing proliferation of icts. also, much work on maasai is now founding documentary films, youtube, instagram, facebook, whatsapp, twitter, linkedin, youtube, snapchat, google +, blogs, wikis, and other social media platforms. granted that internet as marshall and rossman (2016, p.30), ethnography is now a legitimate social science and since there is increased use of mobile phones among indigenous people, such resources advance our understanding of their everyday experiences. therefore, the research made use of internet resources including researcherseeking clarifications (through e-mail, and computer –mediated sms tools available on mobile phones) on material facts covered on online daily newspapers, magazines, blogs, and as a tool for gathering supplementary data. in addition, the context of the internet as an area though which to analyse the structure and negotiated meaning of what constitutes indigenous maasai rights informs much of the discussion about the intersection between icts, political activism and development in maasai land. this is not surprising, as markham (2004) suggests, the internet is both a tool and a discursive milieu that facilitates the researchers’ ability to understand the construction of social structures, and i add, especially, those in which identity politics is of central importance. 1.2 the setting the maasai number and are considered as africa’s most visible group. the maasai community is frequently mentioned in the same sentence with tourism in kenya and tanzania. indeed, maasai land is home to some of the most popular tourist attractions in the region including the maasai mara, serengeti, ngorongoro, amboseli and tanrangire game parks. according to the maasai association (2015), the group’s population stands at over 1 million, spread through kenya and tanzania. although the bulk of maasai live in kenya, census data on numbers in tanzania are hard to come by because the government there does not ethicise census data collection. it is estimated that maasai lands cover over 160,000 square kilometers. the maasai society is comprised of sixteen chapters (iloshon): listed on the maasai association’s website as follows: “ildamat, ilpurko, ilkeekonyokie, iloitai, ilkaputiei, ilkankere, isiria, ilmoitanik, iloodokilani, iloitokitoki, ilarusa, ilmatatapato, ilwuasinkishu, kore, parakuyu, and ilkisonko, also known as isikirari (of tanzania). debates over which african groups qualify to be included among the world’s indigenous people never questioned the legitimate inclusion of the maasai. and in east africa, events marking the un declaration of decade of the world’s indigenous people (1995-2004), acknowledged the groups political and economic marginalisation. most observers acknowledged that the maasai way of life was a factor and modes of production in the state’s discriminatory practices. importantly, their land rights were denied and openly brushed aside from broader questions of human rights in both kenya and tanzania. 1.3 the literature and media narratives on a rising africa the proliferation of icts has helped nurture a growing myth of africa rising from its long stupor in dark ages. yet, the mythology being built around africa’s entry into the global markets of consumption of high-end goods and services are not always interrogated within the australasian journal of information systems otenyo 2017, vol 21, research on indigenous ict being left behind amidst africa’s rising imagery 3 scholarly community. it is frequently an act of wishful thinking on the part of the ruling elite, whose ideas mirror those of global owners of capital and international finance. thus, the promise of good life for a majority of africans must be deconstructed in light of empirical realities. no doubt, the myth of the rising africa has roots in the global search for markets and control of resources. the reality that icts spread to the interior of kenya and ingrained in its social and economic life is applauded. there are at least two streams of literature that shape the discourse on development in indigenous african communities. in simple form these are the same arguments about afro pessimism and optimism as projected through increased consumption of mobile phone and other computer–mediated icts. observations on afro–optimism, and the view that africa’s image is changing exist in popular media and scholarly literature (nothias, 2014). negative stereotypes on lingering violence and continuing poverty have been replaced by positive images of a more hopeful african future (sieff, 2008; taylor, 2014). according to the economist (2011), africa had the fastest growing economy and had real chance to follow in the footsteps of asia. africa’s middle class is said to provide customers for manufactured goods. a similar narrative suggests international companies are looking to africa as the best market for investments (walt, 2014). those studying the factors that have led to increases in gdp are quick to point at the boom in technology use in the continent (perry, 2012). others like clark (2013) remark that africa’s renaissance exists but it deserves the support from developed countries. the world bank adds fuel to this narrative by suggesting poverty reduction in africa s taking place (devarajan, 2012; world bank, 2014). and there are also suggestions that the prosperity excludes girls, women, and the youth (drummond, thakoor and yu, 2014; lagarde, 2014; saba, 2014; yahya, 2015). still, the most significant doubts about the rise of africa has also come from observers in the business sector who consider the poor infrastructure, corruption and weak political institutions as real barriers to africa rising. sieff (2014), mohammed (2015) and johnson (2015) call into question the veracity of the claims of africa rising. they, for various reasons, dismiss the emerging unrealistic assumption of improved economic growth serving as a catalyst to improved lives of africans. most of the claims of prosperity in africa are naively associated with the rise of icts, especially the ease at which mobile phones facilitate communications in previously unreached areas. and we also know that increased presence of indigenous communities on social media has been mistaken for social development without digging deeper into the underlying assumptions of the claims. oxfam (2015) offers the most significant insights in raising the question about who benefits from the increased economic outputs? oxfam contends that the growth benefits foreign businesses and not the well being of most african indigenous groups. from a theoretical perspective, hong (2015) notes that simply acknowledging the rise of africa misses out the point that millions of the poorest africans have not benefitted from the growth in gdp. assuming that both afro–pessimists and those optimistic about africa rising are looking at different places, a partial understanding of the plight of indigenous communities paints a better picture of the theoretical contributions for assertions that africa’s rise is a myth. work done on indigenous communities shows the limits to the africa rising narrative and the reinforcement of these communities as cogs in the wheel of a skewed globalisation machine that creates preordained winners in the name of ruling elites and their overseers in the corporate world. one is tempted to say that the basis for using information and communication technologies to bring together indigenous communities into the development narrative is to minimise the fact that these devices do not change the power relations among different communities and nations. and there is little evidence to support the logic of spreading human development among all communities. the structures that promote images of maasai and other indigenous groups utilising icts are also the same structures that promote their conditions of depravity and marginalisation. in the next section, i make observations and report the core finding about the predicaments of the maasai as they navigate through state and corporate controlled new economies of east africa. australasian journal of information systems otenyo 2017, vol 21, research on indigenous ict being left behind amidst africa’s rising imagery 4 2 observations and results for indigenous people, the administrative state traditionally lacks the will to support human development in these communities. state apparatus also do not place values on indigenous rights to exist in whichever ways that uplift the marginalised. if anything, state apparatus undermines the survival of indigenous communities and, are in fact, using icts to further exclude the maasai and other indigenous communities. the theoretical argument is that icts reinforce existing power structures and do not change the economic position of marginalised groups. as a result, the maasai, compared to other groups are still among the largest offline groups. the systematic deprivation of maasai is accentuated through online apparatus. this is not hard to visualise. state and corporate governance structures exclude the indigenous leaving them out of the decision-making channels. when much of the decision making over indigenous resources is conducted over digital devices, access to decision channels by those affected becomes necessary. yet current corporate practices present barriers in terms of relevance of the contents conveyed through the digital devices. not to mention the charges for data usage and move toward cashless economy. for instance, maasai cattle traders are captured moving their trade items but lacking the sophistication to utilise modern applications of the cashless economy. in addition, the state sells some of the data about the maasai and their immediate environs to tourist and travel agencies. however, the state does not share the foreign exchange earnings with maasai communities in any meaningful ways. further, the state has no laws to protect maasai from exploitation of their identity. its claim to uphold human rights and constitutionalism falls flat considering that trust between the government and maasai communities is far fetched. the maasai are not allowed to kill lions that maraud their cattle, and if they do so, it is front-page news and a matter of popular culture entertainments. compensation for wildlife destruction of maasai savanah, is never treated as a matter of urgency. compensation for destruction of flora and fauna in maasai neighborhoods is not budgeted for in tourism budgets. despite state failure to compensate maasai for the destruction of their means to livelihood, the blogospheres tout virtues of digitalisation of the maasai economy. instead, the narrative is that maasai need apps to help them track down their cattle. but the big picture is lost. and just how have these icts been a part of the everyday realities in maasai lands? 3 social media and identity politics the effects of social media on indigenous cultures are a fairly new area of academic interest and is gaining momentum, for example pioneering scholarship carlson (2013) and rice, haynes, royce, & thompson (2016). carlson’s exemplary work posits that “social media is a new frontier for indigenous people to represent and identify themselves and to others in the emerging global amphitheater” (2013, p.148). focusing on australian aboriginal people, she articulates the view that facebook and other social media offers space and platforms for confirmation and demonstration of one’s identity and public recognition. to effectively use social media platforms, indigenous communities are working with other groups to seize the moment and take advantage of the strengths of the new applications. as tools for communication, social media platforms enable discussions to occur around goals. in the case of maasai, they function to inform the policymaking publics, including international allies wishing to fight for the inclusion of indigenous people’s in national prosperity and cultural survival. in fact, ackerman and krupp (2016) described an organisational behaviour model of using social media that includes sharing ideas and encouraging international awareness and learning. from a political perspective, additional goals have recently included activism, especially to the empowerment of women, whose well-being is adversely impacted by discrimination through the denial of educational opportunities and cultural practices like australasian journal of information systems otenyo 2017, vol 21, research on indigenous ict being left behind amidst africa’s rising imagery 5 female genital mutilation (fgm), practiced among maasai tribes. and there are several ways through which social media highlights the condition of maasai. in describing ways in which social media has been applied, one must recognise that governments also use social media, to present a different reality of the indigenous lives. for starters, success stories of prominent “maasai” are widely circulated on social media. for instance, professor george saitoli, was vice president and mathematician phd at university of warwick, likewise, the experiences of kenya’s second vice–president joseph murumbi and former speaker of the house of representative, francis ole kaparo and freedom fighter and successful member of parliament, prominent land owners john keen and william ole ntimama were frequently presented as maasai icons. more recently, former permanent secretary for education james ole kipiapi who ran for election president of the republic of kenya in 2013 were presented in the media as role models. the reality is that these individuals are a small minority of the ruling national cosmopolitan elite. these individuals do not represent the middle within the maasai moran traditional culture. because kenya’s political system is heavily ethnicised, there is the temptation to present the success of a few wealthy individuals as a symbolic mirror of the larger impoverished tribal base. in some ways, the assumption here is that notion of an individual’s success tells the story of his or her community and that there are no boundaries between community and its leaders. thus, for example, an ethnic maasai tribesman being named minister for education is a big deal for the thousands of starving and marginalised maasai. evidence suggests that the co-opted rich maasai are not human rights activists advocating for protection of the interests of the poor tribesmen. perhaps, this is a classic case of the elite reconstructing their identity, as accounted for in the literature discussing groups modernising and leaving behind their old ways (e.g. harris, 2013, p. 14). as a result of the neglect of rights of the less fortunate, the material conditions in vast areas of indigenous counties are worsening rather than improving. as stated above, kenya’s political landscape is paved on ethnicity. these clashes have mostly been about land. in 2004 and 2005, for example, maasai villagers invaded private ranches in laikipia district and fought against kikuyu farmers in mai mahiu. at least 40 people died in the 2005 ethnic clashes. the government rejected the maasai claims to their lands. periodic skirmishes have continued in the rift valley regions. and after the ethnic clashes which followed the 2007 presidential election, the national cohesion and integration commission (ncic) was formed. the commission aimed at promoting cohesion among the different ethnicities. the commission targeted hate speech and mobilisation, especially, that presented on social media. the commission chairman, francis ole kaparo, a maasai acknowledged the role of elite mobilisation through social media outlets. following escalation of conflicts in the post-2007 era, some of which targeted the maasai and samburu cattle owners (gitonga, 2015). but there was little evidence that maasai leaders in kenya used social media to articulate cases of land deprivations as was done in tanzania. interpretation over maasai land agreements with british settlers in areas such as laikipia are an issue that remains unresolved. 3.1 popular culture and stereotypes: maasai as warrior narrative another action that deserves mention is the misrepresentation of the maasai men as brave defenders of the nation or tribe. the popular culture continues to present maasai men as warriors, killing lions to impress women suitors. hollywood moviemakers have taken notice of this illusionary depiction of an entire tribe and captured it in movies like lion king, and in out of africa, adopted from sal dinesa’s 1972 book. the corporate mentality was to update the stereotypes of the tribe as cultural artifacts to be enjoyed. this predates contemporary hollywood caricatures. in 1908 and 1949, a documentary depicted maasai hunting lions. the video is shown on the serengeti’s channel of youtube (youtube, 2011). later, the movie bomba, the jungle boy presented in a comical way a maasai warrior killing a male lion (see, youtube, 2007). other clips followed, e.g. lion vs. maasai, amboseli national park, safari kenya, (youtube, 2015). australasian journal of information systems otenyo 2017, vol 21, research on indigenous ict being left behind amidst africa’s rising imagery 6 other online, youtube media presentations continue to depict the maasai as entertainment objects. for instance, “maasai attacked by lioness” is entertaining for large audiences. the description in a video uploaded on may 11, 2010 states: an unlucky maasai warrior gets chased and then pounced by a ferocious lioness (who was the culprit for killing their livestock, hence why they were hunting her), whom he and his fellow warriors cornered on some kopjes. he becomes seriously injured, and eventually dies from his injuries (youtube, 2010). as of august 2016, more than 2.9 million people viewed the video. for some, maasai on youtube and social media signifies an emerging information revolution in sub saharan africa (bhan, 2013). bhan cautions that presenting maasai images on social media should not promote “stale imagery” of the past. other youtube images of the maasai advance a similar narrative, for example, the movie the white maasai trailer based on corinne hofman’s autobiographical internet video work, the maasai: the rain warriors directed by pascal plisson, film essays of maasai life, (creative cowboy film), and the maasai: from sand to snow, a documentary movie are among those uploaded from dvds. for one observer, john kariuki (2003), the maasai were the new ‘red indians’ of hollywood. this intensity at which maasai people are featured in movies, reifies the commercialisation of their cultures based on stereotypical presentations as morans (warriors). the marketisation of indigenous cultures is reinforced by sophisticated branding techniques, which makes maasai images, captured in digital devices a commodity. 3.2 brand maasai there are several examples that shed light on this phenomenon. journalist cordelia hebblethwaite (2013) reported that one isaac ole tialolo, a maasai elder and chair of the maasai intellectual property initiative proposed that it was time the thousands of maasai products like beads and dress be protected as a brand. ole tialolo argued that: we all know that we have been exploited by people who just come around, take pictures and benefit from it.” we have been exploited by so many things you cannot imagine.” tourists take photos of maasai people, in many instances against the will of those being photographed. in ole tiaolo’s words, the maasai equate illegal photographing of their people to taking their blood. …we believed that if somebody takes your photograph, he has already taken your blood (hebblethwaite 2013). further, the abuse of maasai culture is perpetuated in thousands of online travel magazines without any direct benefits for the community. the problem is compounded by the fact that some of the images and video clips embedded in these online magazines are redistributed in multiple social media outlets. according to light years ip, a london based non-profit organisation, which specialises in protecting intellectual property rights; about 80 companies around the world use the maasai image. it has formulated a policy known as intellectual property value capture to change rules on global trade involving people in developing countries (light year, 2014). by many estimates, maasai is the biggest cultural brand in the world. new zealander, ron layton, founder of light years ip, estimated that if maasai “brand” “were owned by a corporation, it would be worth more than $10 million a year” (hebblethwaite, 2013). among the corporations that have utilised maasai images include accessories branded maasai made for british automobile company, jaguar land rover and maasai barefoot technology that produces specialty trainers. also, the high–end fashion house louis vuitton manufactures a maasai line, that includes hats, scarves, duffle bags, and beach towels (hebblethwaite, 2013). maasai shukas (toga) inspired the designers of the scarves. other fashion companies that have been inspired by maasai artifacts include bedding shirts by calvin klein, and trousers by ralph lauren. furthermore, in 2003, italian pen maker, delta introduced as part of its indigenous people luxury line, a red and brown luxury fountain pen that retailed at $600 (faris, 2013). these are just a few examples of the commercial value of brand maasai that corporations exploit at the expense of the dignity of an indigenous people. australasian journal of information systems otenyo 2017, vol 21, research on indigenous ict being left behind amidst africa’s rising imagery 7 so, if brand maasai can make billions of dollars for international corporations, just how much do the maasai people make? probably nothing. the argument presented by the corporate world is that there is no unified body of maasai body for corporations to seek permission from. the maasai brand was also manifest in local tourist circuits. for instance, the maasai mara is one of the best-known game lodges in the world. through numerous travel magazines and the use of social media platforms like facebook, it is promoted as one of the greatest places from which to experience the wildebeest migration, an event considered as one of the world’s most fascinating natural wonders, an african safari mecca of sorts. the maasai mara is home to the big cats like lions, cheetahs, leopards, and other exotic mammals. it is also rich in flora and fauna and visitors may view the game on hot air balloons. through increased human activity that includes deforestation, grazing, farming and land development, the ecosystem in the region is now threatened. yet, the narok county, where the maasai mara lodge is situated collects millions of dollars, most of which is not reinvested in the region. in many instances, tourists have to pay in american dollars or other hard currencies signifying the externalisation of the maasai mara economy. moreover, for several years, the education system ignored elements of sustainability that would have incorporated maasai cultural lifestyles and indigenous knowledge. and, still part of the tourist attraction is the continued reification of the narrative that the maasai are a people to be “experienced.” even famous conservationists like the karen blixen camp promote their activities in a language, which commoditises the maasai life style. the karen blixen camp’s website notes: experience how the maasai people live in the middle of the bush with zebras grazing and lions roaring just outside their homes and get to know their culture and customs on a visit to the traditional maasai village, the local primary school or the busty market day (karen blixen, 2015). karen blixen states that it supports the maasai community by encouraging its guests to visit the villages where maasai live. guests of karen blixen are required to obtain tickets before visiting maasai villages. the fees collected are forwarded to the maasai villagers for purposes of providing basic needs. besides the conservative educational initiatives, there are other efforts to use smartphone technologies to promote environmental sustainability education. these efforts are meant to reengage the maasai on issues of sustainability through modern pedagogical approaches (dogbey et al., 2014). using icts, for example, smartphones expand, visually the magnitude of the challenges of pastoralism but not succeed without massive support from all stakeholders. the government of kenya, typically, controls the curriculum and leverage icts and experiential education systems only if such efforts contribute immediately toward foreign exchange earnings. although efforts by voluntary societies are on the increases, the economic gains, thus far, do not reflect the high visibility of the indigenous community. in other words, maasai per capita incomes are at the bottom among kenyan communities. more important, the perpetuations of practices that reduce opportunities for advancement of women remain a huge challenge among development practitioners and scholars. instead, the commodification of maasai women has been reinvented through a unique set of circumstances, which brought kenya on hollywood’s radar. a kenyan-mexican, lupita nyongo won an oscar academy award for best supporting actress in her role as patsey in steve mcqueen’s drama 12 years a slave (2013). because of her strong showing in popular media, she inadvertently contributed to the promotion of african dress and adornments. inevitably, maasai women were depicted in lupita nyongos’s attire in many of her hollywood-related media appearances. her unintended consequence was to show the world that maasai culture but at the same time lend credence to the marketability of maasai as an artifact culture. blogs and fashion magazines all over the world gave glowing tributes to her dress, necklace and other outfits. in spite of the glamour, there was little interrogation of the australasian journal of information systems otenyo 2017, vol 21, research on indigenous ict being left behind amidst africa’s rising imagery 8 material conditions that shaped the maasai artistry or even the copyrights and rights of the maasai women who have for generations mastered such fine works of arts. 3.3 making good use of e-activism beyond entertainment and branding, contrary to expectations and an unintended consequence of government promotion of icts, the use of these platforms for political mobilisation and engagement appears to be benefiting the maasai. among the reported success stories has been the use of blogs such as cultural survival, (www.culturalsurvival.org,) which advocates for indigenous people’s rights on a global scale. since 1972, this non-profit organisation has presented evidence of political marginalisation of indigenous groups across the nations. the platform includes blog posts from maasai groups like the maasai education discovery (med), which has campaigned against female circumcision within the tribe. when embedded in social media like facebook and tweeter, posts from med receive much needed publicity and to a small degree has influenced policy debates leading to greater state intervention in girl education within maasai society. perhaps, a more pronounced social media –driven advocacy and form of e-activism was reported in the case of tanzania. the guardian global development, an online magazine, which is supported by bill and melinda gates foundation, has observed that social media actually helped the maasai living in loliondo, tanzania to mobilise for a worthy cause-to save their land. the government in tanzania had allocated land around the serengeti national park to a foreign hunting company in the early 1990s. in 1992, the government licensed a company owned by an official in the united arab emirates government, the ortelo business corporation (obc) to organise hunting expeditions on the region. previously, in 1959 the colonial administration had designated the landmass a “game controlled area.” as a game area, rich in wildlife like lions, leopards, cheetahs, wildebeest, antelopes and other animals, it provided a perfect resource for sport and trophy hunters. what was ignored was the fact that for centuries, the loliondo territory was home to pastoral maasai. the action to sell the land to foreigners ignored the calls from maasai, whose community rights were not considered in the government decision. the minister for natural resources and tourism allocated 150,000 hectares, which was 40 percent of the loliondo, ngorongoro district area to the hunting company, thereby attempting to evict over 20,000 people (ngoitiko and nelson, 2013). sale of land would have been inconceivable in the early 1960s, when president nyerere, through his ujamaa (community) doctrine would have never allowed land appropriation at the expense of maasai community. tolu ogunlesi (2014), a nigerian journalist quoting mwalimu nyerere (1962) book, stated: in a country such as this, where, generally speaking, the african are poor and the foreigners are rich, it is quite possible that, within 80 or 100 years, if the poor african were allowed to sell his land, all the land in tanzania would belong to wealthy immigrants, and the local people would be tenants. in 2009, the government of tanzania evicted several hundreds of maasai from their traditional homelands. this action was captured in local and international online and print media. avaaz, an online activist organization posted information on this struggle for land rights and attracted more than 2.3 million worldwide supporters who signed the petition to the president. avaaz posted a petition to tanzania’s president jakaya kikwete: as citizens from around the world, we call on you to oppose any attempt to evict maasai from their traditional land or require them to relocate to make way for foreign hunters. we are counting on you to be a champion for your people and stop any attempt to change their land rights against their will (avaaz.org, 2023). 4 discussion the moral question that needs to be answered is whether maasai are still being left behind amidst the narrative of a growing african middle class. civil society groups framed as a australasian journal of information systems otenyo 2017, vol 21, research on indigenous ict being left behind amidst africa’s rising imagery 9 question of human rights and mobilised accordingly. the global media coverage of the displacement of maasai from their homelands forced the government to reverse its decision. there have also been other interventions geared toward transforming maasai lifestyles and promotion of modernisation. in particular, international development agencies like fao and undp attempts to push maasai to live sedentary life styles have been praised by western– trained elite. yet, there is evidence sedentarise maasai projects have failed. instead additional indigenous land has been taken away. in tanzania, the government evicted the maasai from simanjiro plains (mccarthy, 2013). and even after the high court of tanzania ruled in favor of maasai, the government did little to compensate the groups from years of exploitation. the economic conditions remain intact and resources to provide sustainable development have dwindled. efforts by maasai for economic empowerment are not given the weight they deserve by way of helping market maasai products. maasai enterprises, with help from shgw, a dutch ngo, produce yoghurt, butter and ghee but these are not well promoted in local marketing avenues. with the maasai now at the heart of the organization of indigenous people’s of africa (oipa), their informal economic settings are beginning to receive attention but their informal status exclude them from the market, upon which the narrative of africa rising is rooted. perhaps, this explains why the ict mobile phone company, airtel tanzania, which is active in the maasai regions of tanzania offered to join unesco funded projects to construct boma housing units for the maasai (maduekwe, 2013). the goal of airtel and other ict corporations believe that formal housing units would provide incentives for women to set up small businesses. having home implies better organisation for producing items such as handicrafts, raising chicken, selling milk and joining cooperatives (maduekwe, 2013). with the maasai traditional homes taken away from them in the name of serengeti national park, tsavo national park, and mkomazi game reserve, where is the reality of maasai living? outsiders, including large–scale hunting companies have been lured to maasai life styles and encroached on the indigenous people’s wealth. if maasai were a part of africa rising why would a maasai elder martin saning’o kariongi from northern tanzania, make it his life mission to save his people’s way of life? maasai are not a part of the emerging narrative of the rising africa when they have to be saving themselves. the logic of the governments in kenya and tanzania is to continue to encourage commercialisation of an indigenous people for purposes of gaining foreign exchange earnings. these governments do so without significant investments in the repair of the ecosystem. conservation then has become more of the mission of western research funds, the world wildlife fund (wwf) and volunteer non-profit conservatories like the karen blixen camp. the karen blixen camp, for example offers training to maasai on sustainable ways to harvest honey. investments in maasai education are insufficient and not commensurate with tourist revenues from the community. the karen blixen camp, is explicit on its website and provides the example of the maasai predicament. the aitong primary school is a kenyan government school and provides free education for the maasai children. however, governmental funding has not been sufficient and well wishers as well as parents are contributing with financial support for teachers’ salaries, school fees, construction of classrooms and basic learning aids (karen blixen camp, 2015). the case of aitong is typical of all schools in the maasai region. without adequate funding, maasai competitiveness and goals for navigating through the new economy driven by icts will not be achieved. beyond appropriate funding for schools, the maasai continue to be affected by acute shortages of water (archambault, 2014). the maasai are a community on the margins of survival, without land, water, and wireless, for the most part. 4.1 saving the maasai through digitisation of culture since 2008, efforts to digitise aspects of maasai culture have been launched, albeit in a pilot scheme. through the world intellectual property organization (wipo), maasai can document australasian journal of information systems otenyo 2017, vol 21, research on indigenous ict being left behind amidst africa’s rising imagery 10 their own values and archive key elements of their culture for future generations (oppenneer, 2008). this collaborative approach included the center for documentary studies (cds) at duke university, library congress, and wipo. while archiving traditions is seen as “empowering” the tradition bearers, it does not transcend the ideas put forward in the reinforcements theories of the roles icts play in modern society. the question becomes, who controls the narrative about the maasai? while the initiatives involve working with a few native maasai, the cultural documentation project assumes the maasai culture is under threat. the new economy has not altered the power structures and to confine maasai to archives, which undergirds the creative heritage project, of which the wipo is a part of. the basis for digitisation seems to be an acknowledgement that the maasai are a disappearing culture, yet the economic forces causing the situation are not interrogated in the process toward archiving the maasai culture. the same argument can be made with regard to formatting maasai economies for further globalisation. this is in the form of ifad taking action to manage maasai adaptation of information and communication technologies through mapping their settlements with geographic information systems (cordone, 2010). ifad has established plans to not only use gis to map the maasai lands but also collect observations on herding and weather. on the surface, these icts appear as innocent ideas about reinforcing maasai traditions and to assist the group in navigating through climate changes. without adapting icts from below, there is the danger that resource management systems may not be sustainable. by involving elders from the maasai community, the ifad project seeks to help maasai people to manage their rangeland in sustainable manner. yet, the fundamental issues of education and ability the tanzanian government to absorb and coordinate donor funded initiatives remains weak. while the ifad grant seems reasonable, the government itself lacked the capacity to engage all the stakeholders in a sustainable way. providing matching funds and successor programs, to be managed by the maasai beneficiaries themselves seems a far shot. a visit to maasai land leads to discovery that the vast majority do not actually belong to the newfound lower middle class country status that kenya is now a part of. consider the education fraud boarding schools built for samburu and maasai – pastoralists were taken over buy other ethnic groups. if education is said to be a path for inclusion and tool for narrowing the digital divide, then how it has been provided for among the maasai that resources must be channeled. this includes maasai mara university, to enable it be a catalyst in providing leadership on sustainable use of resources, including facilitation of ict education among the indigenous pastoralist communities. above all, is a deliberate policy shift to stop the alienation of maasai lands and to commercialisation of their culture? maasai land is increasingly being alienated at alarming rates. it is predicted that the maasai of kajiado will become landless within a few years due to expansion of settlements and real estate developments associated with nairobi’s expansion (koross, 2015). and when sold, it is often at below market prices. witnesses, rich landowners, including ruling elite have built lodges in the mara. and chinese investors have also added to the stock of lodges in the mara area. in fact, many of the land deals in the region are not legal. adding to the mix are private “conservancies,” funded directly by tourism interests including ventures by richard branson. it is reported that some of the land was leased for a throw–away amount of $27 per hectare, per year (hatcher, 2013). even government has underpaid maasai for land appropriated for purposes of power station installations, railway and road construction. already, protests of local maasai over compensation have received much press coverage (githaiga, 2015). maasai herders have also protested against the kenya agriculture and livestock research and organization (kalro), which they claimed appropriated native land. the maasai lost in an ensuing court case against the research center (kipngenoh, 2015). in addition, maasai land is experiencing massive desertification through climate change and human caused destruction of biodiversity. still, the destruction of the maasai’s natural resources through poaching of wildlife hunting, misappropriation of tourist dollars from the australasian journal of information systems otenyo 2017, vol 21, research on indigenous ict being left behind amidst africa’s rising imagery 11 maasai lodges build within government national parks all deny the community the dollars to empower the communities. further, maasai lifestyle is a source of revenue for the kenya tourist board, which actively promotes maasai cultural artifacts and lifestyles as a commodity to be enjoyed by western and eastern tourists. but in the collective minds of ruling elites, the maasai are the undeserving poor. they need not join the middle class because they own cattle and are culturally comfortable in their environments. the myth of africa rising does not include empowerment of those without western education and those refusing to be part of the “logical western markets.” the origins of the exclusion of the indigenous communities can be traced back to the early british settlements in kenya. not to forget to mention, that the maasai homelands are in both kenya and tanzania, then, a german colony. the splitting of families across two different countries took its toll on cohesion and resourcefulness. the larger point, is, that the british planted the seeds for marginalisation of the community. the diaries and anthropological studies of the british colonisers are replete with comments that are derogatory of maasai. during the lancaster house talks leading to independence, the issue of maasai lands was raised. everyone at the conference knew that maasai land was communal and yet, the british and their african allies accepted to impose the market philosophy of “willing buyer–willing seller” against the wish of maasai elders. as a result, illegal transaction involving a colonially instituted lands board processed transactions that gave huge chunks of land to british settlers in places such as, elbourgon, nyandarua, and laikipia. before independence, the maasai leadership, through the maasai united front had termed the kikuyu settlers as squatters. the maasai also lost land to those building the kenya –uganda railway line, magadi soda company and other mining corporations. as mwaruvie (2009, p.45) sates, the corporations received favorable concessions at the expense of the maasai community. and after 1963, a good deal of the lands was dished out to president jomo kenyatta’s henchmen. president kenyatta gave away land extra constitutionally. kenyatta assumed that his role as president was functionally equivalent to that of the queen of england and he was only answerable to himself. after the british left, an emerging neo-colonial african elite continued along the same path, marginalising the maasai and alienating them from their land. witness, nairobi itself means place of cold waters in maasai language. now accounting for over 60 percent of the country’s gnp, nairobi is hardly maasai in character. instead, it is modeled along the lines of the big cities of the west. indeed, mwalimu julius nyerere used to comment, tongue in cheek, that one does not have to go to london anymore, nairobi would take care of the business. a good number of maasai in nairobi serve as night watchmen, sleeping on people’s verandah’s, out in the cold. later, government policy reiterated the idea of economic modernisation in maasai land through extending individual property rights protections. and scholars joined in by offering explanations about the merits of indigenous management of their commons. for example, mwangi, (2007) argued that maasai pastoralists in kajiado supported subdivision of group ranches, in the 1980s, as a strategy to their own rights to land. bottom-line, maasai wanted to be in charge of their own resources, especially land. although the political support to maasai rights was expressed through political rhetoric, the fundamental principles, based on marketisation of community lands held back the maasai. in other words, the fundamentals of the economy did not match the rhetoric of development. 5 conclusion in conclusion, this study has described the nexus between icts supported by government and corporate power and the continued marginalisation of the maasai. there is evidence that social media serves interests of all stakeholders in different ways. while on the one hand, icts help project the myth of africa rising, paradoxically, new technologies help marginalised communities find voice to press forward with struggles for human rights. bottom line, increased investments in icts must not be an end in themselves. perhaps the exploitation of australasian journal of information systems otenyo 2017, vol 21, research on indigenous ict being left behind amidst africa’s rising imagery 12 maasai resources has to be rejected outright in view of african nationalism original objectives, which was to end all manner of discrimination and shared prosperity for all people. one is tempted to revisit julius nyerere’s time tested observation that an increase in numbers of millionaires in a society doesn’t amount to much if the system perpetuates the marginalisation of a huge segment of its people. according to nyerere, the appearance of millionaires in any society is no proof of its affluence; they can be produced by very poor countries like tanzania just as well as by rich countries like the united states of america. for it is not efficiency of production, nor the amount of wealth in a country, which makes millionaires; it’s the uneven distribution of what is produced (nyerere 1962). therefore, the narrative of africa rising, as perpetuated by the world bank and local ruling elites is a disruption that masks the true motive of global capitalist goal, which is eradicate poverty through social engineering practices that end indigenous cultures. and that is to drive them to the digitalised archives of history. the digitisation of indigenous expressions, especially increased presence on social media, with the goal of promoting and passing along cultural heritage to succeeding generations recognises the utility of new technologies, but it also indirectly affirms that forces of destruction of these communities are real. references ackerman, a.s., & krupp, m.l. (2016) “puzzled by social media? make connections.” hr news, the magazine of the international public management association for human resources (ipma-hr). (september): 12-13. avaaz.org. (2013) stand with the maasai. avaaz.org: the world in actin. available at: http://www.avaaz.org [accessed 8 april, 2016]. bhan, n. (2012) mobile phones, social media and the maasai: time to refresh the image. blog. available at: http://www.nitibhan.com [accessed 24 may 2016]. carlson, b. (2013). the ‘new frontier’: emergent indigenous identities and social media. in m. harris, m. nakata & b. carlson (eds.), the politics of identity: emerging indigeneity (147-168). sydney: university of technology sydney e-press clark, h. (2013) africa’s renaissance deserves continued support. united nations development programme. may 24. available at: http://www.undp.org/content/undp/en/home/ourperspective/ourperspectivearticles/ 2013/05/24/african-renaissance-and-the-rising-south.html [accessed 16 august 2016]. cordone, a. (2010) indigenous maasai cultural center. reinforcing traditional maasai adaptation practices with information and communication technology in the united republic of tanzania. international fund for agricultural development, ifad: rome. devarajan, s. (2012) africa is rising is poverty falling? the world bank. march 1. available at: http://blogs.worldbank.org/africacan/africa-is-rising-is-poverty-falling [accessed 12 march 2016]. dinesen, i. (1972) out of africa, vintage books, new york. (first published in 1937). dogbey, j., quigley, c., che, m., & hallo, j. (2014) “using smartphone technology in environmental sustainability education: the case of the maasai mara region in kenya.” igi global. 6 (1): doi:10.4018/ijmbi2014010101. drummond, p., thakoor, v. & yu, s. (2014) africa rising: harnessing the demographic dividend. imf working paper, wp/14/143. available at: https://www.imf.org/external/pubs/ft/wp/2014/wp14143.pdf economist. (2011) “after decades of slow growth, africa has a real chance to follow in the footsteps of asia.” (december 3), available at: http://www.economist.com/node/21541015 [accessed 3 june 2014]. australasian journal of information systems otenyo 2017, vol 21, research on indigenous ict being left behind amidst africa’s rising imagery 13 faris, s. (2013) can a tribe sue for copyright? the maasai want royalties for use of their name. october 24. bloomberg news. available at: http://www.bloomberg.com/bw/articles/2013-10-24/africas-maasai-tribe-seekroyalties-for-commercial-use-of-their-name [accessed 2 july 2016]. githaiga, p. (2015) “locals protest compensation.” standard. july 15:28. gitonga, m. (2015) professionals fanning conflict in baragoi through social media, ncic says. daily nation june 23. available at: http://www.nation.co.ke [accessed 24 june 2015]. hebblethwaite, c. (2013) brand maasai: why nomads might trademark their name. bbc news. may 28. pri’s the world. available at: http://www.bbc.com [accessed 12 july 2016]. hong, h. (2015) “africa rising”: a hopeless falsity or a hopeful reality? undercurrent, 11(1): 16-22. harris, m. (2013) “emergent indigenous identities: rejecting the need for purity,” in harris, m., nakata, m., & carlson, b. (eds.). the politics of identity: emerging indigeneity. uts publishing, sydney. 10-25. hatcher, j. (2013) “the maasai mara: it will not be long before it’s gone.” the guardian. available at: http://www.theguardian.com/travel/2013/aug/23/masai-mara-tourismpolitics [accessed 13 august 2016]. johnson, s. (2015, october 27) slowdown calls ‘africa rising’ narrative into question. financial times. available at: http://www.ft.com/cms/s/3/93d5c572-7bf6-11e5-a1fe567b37f80b64.html [accessed 27 october 2015]. karen blixen camp, (2015) community visits. available at: http://www.karenblixencamp.com [accessed 21 june 2016]. kariuki, j. (2003) maasai: the new red indians of hollywood. the east african, independent weekly, nairobi. march 3. world press review, 50 (5). available at: http://www.worldpress.org/africa/1063.cfm [accessed 3 august, 2016]. kipngenoh, b. (2015) court restrains maasai herders from land after failed agreement. standard, august 13. available at: http.www.standardmedia.co.ke [accessed august 6, 2016]. koross, k. (2015) how your new plot of land could be turning maasai into squatters. standard, august 14. available at: http://www.standardmedia.co.ke [accessed 23 june 2016]. lagarde, c. (2014) africa rising—building to the future (keynote address). international monetary fund. may 29. available at: https://www.imf.org/external/np/speeches/2014/052914.htm [accessed 29 may 2014]. light years (2014) about us. available at: http://www.lightyearsip.net/news/ [accessed 23 june 2016]. louw-vaudran, l. (2015) africa: no need to panic africa is still rising... slowly. allafrica. october 27. available at: http://allafrica.com/stories/201510281438.html [accessed27 october 2015]. maasai association, (2015) “preserving and celebrating maasai cultural heritage. “available at: http://www.maasai-association .org/maasai.html [accessed 3 august 2016]. maduekwe, a. (2013) “a little technology can go a long way.” horizons, unesco publication. a world of science, 11 (2), april –june, available at: http://www.unesco.org [accessed 10 june 2015]. markham, a.n. (2004) “internet research, “in silverman, d. (ed.), qualitative research (3rd edition), sage, london. 111-128 australasian journal of information systems otenyo 2017, vol 21, research on indigenous ict being left behind amidst africa’s rising imagery 14 marshall, c & rossnman, g.b. (2016) designing qualitative research (6th ed.), sage, los angeles. mccarthy, d. (2013) tribal elder modernizing the maasai to avoid extinction cnn july 26. available at: http://www.cnn.com [accessed 3 august 2016]. mohammed, o. (2015) not everyone is buying into the “africa rising” narrative. quartz. march 6. available at: http://qz.com/397757/some-investors-are-not-fully-buyinginto-the-africa-rising-narrative/ [accessed 6 march 2015]. mwangi, e. (2007) socio economic change and land use in africa: the transformation of property rights in maasai land, palgrave macmillan, new york. mwaruvie, j. m. (2009) “political economy of railway development in kenya,” in amutabi, m. n. (eds.). studies in the economic history of kenya: land, water, railways, education, and entrepreneurship, edwin mellen press, lewiston. 2976. ngoitiko, m. & nelson, f. (2013) what africa can learn from tanzania’s remarkable maasai lands rights victory. guardian global development, october 8. available at: http://theguardian.com [accessed 16 june 2016]. nothias, t. (2014) ‘rising’, ‘hopeful’, ‘new’: visualizing africa in the age of globalization. visual communication, 13(3), 323-339. doi:10.1177/1470357214530063. nyerere, k. j. (1962) ujamaa: the basis of african socialism. available at: http://www.nathanielturner.com [accessed 23 june 2016]. oguniesi, t. (2014) tanzania’s maasai prefer death to eviction. opinion. november 23. al jazeera. available at: http://www.aljazeera.com [accessed 10 august 2016]. oppenneer, m. (2008) digitizing indigenous culture: the maasai of laikipya. wipo press release, geneva, ethnos project, may 20, 2008. pr/2008/553. available at: http://www.ethnosproject.org [accessed 12 june 2016]. oxfam. (2015) africa: rising for the few. june 2. available at: https://www.oxfam.org/sites/www.oxfam.org/files/world_economic_forum_wef.afric a_rising_for_the_few.pdf. perry, a. (2012) africa rising. time, 180 (23): 48-52. rice, e.s., haynes, e., royce, s.c., & thompson, s.c. (2016). social media and digital technology use among indigenous people in australia: a literature review. international journal for equity in health. 15:81. doi:10.1186/s12939-016-0366-0 saba, m. (2014) is africa rising with women leading the way?, ic publications ltd, london. santos, r. (2010) “the maasai tribe goes mobile,” july 24. 2010 global marketing. available at: http://www.2010global marketing.worldpress.com [accessed 6 january 2016]. sieff, m. (2008) africa: many hills to climb. world policy journal, 25(3):185-195. doi:10.1162/wopj.2008.25.3.185 taylor, i. (2014) is africa rising? brown journal of world affairs, 21(1): 143-161. walt, v. (2014) is africa’s rise for real this time? fortune. september 18. available at: http://fortune.com/2014/09/18/africas-rise-for-real-this-time/ [accessed 18 september 2014]. world bank. (2014) africa rising: a tale of growth, inequality and great promise. april 14. available at: http://www.worldbank.org/en/news/feature/2014/04/14/africa-rising-atale-of-growth-inequality-and-great-promise [accessed 28 july 2016]. yahya, m. (2015) a sidelined youth: the soft underbelly of ‘africa rising’. united nations development programme. july 15. available at: australasian journal of information systems otenyo 2017, vol 21, research on indigenous ict being left behind amidst africa’s rising imagery 15 http://www.undp.org/content/undp/en/home/blog/2015/7/16/a-marginalizedyouth-the-soft-underbelly-of-africa-rising-.html [accessed 3 may 2016]. youtube, (2015) “lion vs. maasai, amboseli national park, safari kenya,” february 23. available at: https://www.youtube.com/watch?v=3djqlepsl8k [accessed 12 june 2016]. youtube, (2007) “maasai warriors kill lion.” available at: https://www.youtube.com/watch?v=xe3sywtsmgs [accessed 6 july 2016]. youtube, (2007) january 26; “maasai warriors kill male lion, bomba, the jungle boy 1947,” available at: https://www.youtube.com/watch?v=xe3sywtsmgs [accessed 16 july 2016]. youtube, (2010) “lion attack! get wild.” available at: https://www.youtube.com/watch?v=6unamrilgqg [accessed 16 july 2016]. copyright: © 2017 otenyo. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ editorial this is the first issue of volume 6 of the australian journal of information systems. as editor i am pleased to report the ongoing interest in the journal both in australia and overseas. our first paper inquiring organisations james courtney, david croasdell & david paradice, business analysis and research department, texas a & m university examines how contemporary businesses can use models of inquiring systems to become more efficient. the paper explores the relationship between inquiry and learning in organisations and how information technology can be used to support the process of knowledge creation in the context of inquiring systems. our second paper evaluation of database modeling methods for geographic information systems thanasis hadzilacos, computer technology institute, university of patras & nectaria tryfonas, department of computer science, aalborg university evaluates different modeling techniques for the design of geographic information systems. a set of exemplary problems in spatial systems on which the models can be tested is discussed. ernest jordan & david musson, macquarie graduate school of management provide our next paper strategic systems? only when they work! this paper presents the results of a sample survey of australian organisations' approaches to business and information technology (it) contingency planning. the findings reveal that most organisations are inadequately prepared and fail to take the issue seriously. business continuity is not rated as a high priority. our next paper is evaluating method engineer performance: an error classification and preliminary empirical study steven kelly & matti rossi, department of computer science and information systems, university of jyvaskyla. this paper describes an approach to empirically test the use of metacase environments to model methods. the results from this pilot study confirm the usefulness of the classification, and show some interesting differences between the paradigms. graham mcleod, university of capetown provides our next paper method points: a metric for method complexity. this paper presents a metric for method complexity which aids in choosing between competing methods. the paper goes on to recommend way to improve the practical adoption of new methods. our next paper is reengineering in australia: factors affecting success felicity murphy, simsion bowles and associates & sandy staples, department of information systems, university of melbourne reports on a study replicating a us study in the australian context. the compares factors leading to success in the us with those found in australia. paul o'brien, information systems group, university of queensland provides our next paper electronic commerce, the internet and travel cybermediaries. this paper investigates the likely disintermediation effects of electronic commerce on retail travel agents. it provides the basis for the development of a strategic framework for intermediaries that can be used to enhance business performance. our next paper is an investigation into the feasibility of tasmanian stonefruit growers employing electronic commerce stewart pedersen, school of information systems, university of tasmania. the paper presents an assessment of the feasibility of tasmanian stonefruit growers adopting electronic commerce. the paper suggests that there is a need for these issues to be understood prior to adopting electronic commerce. victor van reijswoud, department of information systems, delft university of technology & hans mulder, essential action engineers, present our next paper speech act based communication and information modelling with demo. the field of speech act based modelling of information systems has attracted increasing attention in recent years. this paper provides several case studies using demo. michael rosemann & michael zur muehlen, department of information systems, university of muenster, provide our next paper evaluation of workflow management systems a meta model approach. this paper provides an organisational reference meta model. the paper shows that this model helps users specifying their requirements for workflow management systems. our next paper is reengineering a software review process with the function based process analysis (fpa) method claude strieker & jintae lee, department of decision sciences, university of hawaii. this paper looks at how www technology can improve existing processes in the context of the software review process. our final paper is tool based psychology as a philosophy of technology irina verenikina, russian academy of sciences & edward gould, department of business systems, university of wollongong. this paper examines the influence of psychology in the area of human computer interaction. as a possible alternative to the cognitive approach the vygotskian psychological framework of activity theory introduced. once again i would like to pass on my thanks to the authors and consulting editors for the high quality of material in this issue. i would also like to thank keith curie for his book review and david dodds for his assistance in producing final copy of the journal and the maintenance of the home page. r. macgregor editor paper title (paper title style) australasian journal of information systems dainow 2017, vol 21, research on applied ethics and ict threats to autonomy from emerging icts 1 threats to autonomy from emerging icts brandt owen dainow maynooth university ireland bd@thinkmetrics.com abstract this paper examines threats to autonomy created by significant emerging ict’s. emerging ict’s cover a wide range of technologies, from intelligent environments to neuroelectronics, yet human autonomy is potentially threatened by all of them in some way. however, there is no single agreed definition of autonomy. this paper therefore considers the ways in which different versions of autonomy are impacted by different systems. from this range of threats we will derive some properties which any ict must exhibit in order to threaten human autonomy. finally, we will show how the range of definitions of autonomy creates problems for customary approaches to vale-sensitive design, and indicates a need for greater flexibility when attempting to improve the ethical status of emerging ict’s. keywords: ict; ethics; autonomy; foresight studies; futures research; etica; value-sensitive design 1 introduction this paper examines ways in which human autonomy can be threatened by emerging ict’s. emerging ict’s embrace a wide range of technologies, including autonomous systems, intelligent environments, bio-electronic implants, robotics, and artificial intelligence. we will explore the different ways in which each ict threatens autonomy to show that the nature of the threat, if any, depends on the version of autonomy being used. from this survey of threats we will derive some properties which any ict must exhibit if it threatens human autonomy, no matter how it is defined. finally, we will argue the range of definitions of autonomy demonstrates the need for greater flexibility when attempting to improve the ethical status of emerging ict’s. in order to discuss the complete range of emerging ict’s it is necessary to have a system by which to conceptualise and organise them. we shall use for these purposes the output of the etica project (stahl 2011), which attempted an examination of the complete range of emerging ict’s, developed an ordered taxonomy (ikonen et al. 2010), and researched the ethical concerns associated with each (heersmink, van den hoven, and timmermans 2010). 2 methodologies for evaluation of emergent technology to count as “emerging” (as opposed to “possible”) an ict must be sufficiently developed that we can understand its general features, but which has not yet achieved maturity in all aspects. since the 1960’s the predominant model for such analysis has thus been to treat technologies as “socio-technical systems” (trist 1981; lamb and kling 2003). this position contrasts with the other major treatment of technology, technological determinism, which treats technology as affecting society, but as developing independent of social, cultural, political and other “intangible” causes (smith and marx 1994). an emerging ict will have little market penetration (it may still be in prototype), such that its final place and role in the market is yet to become evident; its full and final set of features are unlikely to be completely determined; the business models under which it will function are likely to be nascent; users may still be working out patterns of usage; and the regulatory framework is most probably undeveloped. autonomous cars provide a simple example of an emerging ict which illustrate these uncertainties. google’s autonomous cars are still in prototype (solveforx 2016), while tesla vehicles are just beginning to penetrate the market (statistica 2016). in both cases, the systems australasian journal of information systems dainow 2017, vol 21, research on applied ethics and ict threats to autonomy from emerging icts 2 are still in development such that the final set of functions is indeterminate (solveforx 2016; musk 2016). the regulatory framework for autonomous vehicles has seen some development in a few places, such as california, but most countries haven’t even begun to think about the technology. in the usa, for example, as of march 2017, only eleven states had any regulations regarding this technology, while the majority of attempts to introduce legislation in the usa since 2012 have failed (national conference of state legislatures 2017). business models for the sale and use of this technology are still undetermined. for example, tesla recently proposed models in which people would be able to send their tesla car out as an automated taxi when they’re not using it (musk 2016). in the meantime, users are still learning how to use the systems, often in conflict with how the designers intend them to be used. the most dramatic example of this disparity between designer’s intent and user behaviour is the frequent driving of tesla vehicles without human supervision, despite the tesla’s frequent statements that the user should pay attention and keep their hands on the steering wheel at all times (consumer reports 2016). any attempt to understand the ethical impact of autonomous cars must therefore commence with a vision of the state of affairs when the technology has matured – with standardised features, customary patterns of usage, stable regulatory regimes and (reasonably) settled business models. any attempt to assess the ethical implications of autonomous cars thus becomes an exercise in anticipating what the future will look like. this brings us into the remit of foresight studies. 2.1 introducing foresight studies ‘foresight studies’ is one of the terms used to describe the discipline of considering the future in a methodical manner. foresight studies can be used in any discipline and so the objectives in such research vary widely (glenn 2009; godet 2009; veikko, kanerva, and kouri 2009). this presence within many disciplines has led to a variety of terms being used to describe the activity. common alternative terms for foresight studies are “future studies” (most commonly used in the usa), “futures research” (europe), “futurology” (australasia), “prospective studies” (france), “futures field” (europe) and “prognostics” (russia) (veikko, kanerva, and kouri 2009). a wide variety of foresight methodologies have evolved and most foresight research projects combine several methodologies. the different methodologies used within foresight studies can be divided between quantitative and qualitative in technique and between normative and exploratory in purpose, though some methodologies bridge these divides (veikko, kanerva, and kouri 2009; haegeman et al. 2013; gordon and glenn 2004). quantitative methods are used mainly in predictive forecasting and seek to produce probabilities (veikko, kanerva, and kouri 2009; haegeman et al. 2013). quantitative methods are also used in complex modelling, for example, climate modelling. such methods are based on the assumption that future development is a continuation of past trends. while they provide a continuity from past to future, they cannot allow for disruptive technologies or unexpected developments. some argue for the need to support quantitative methods with qualitative ones (veikko, kanerva, and kouri 2009; haegeman et al. 2013) as they are better suited for the anticipation of abrupt or unexpected changes (ogilvy 2002; veikko, kanerva, and kouri 2009). foresight research also distinguishes between methodologies on the basis of whether they are normative or exploratory. normative forecasting is concerned with whether a particular future is desirable and the ethical status of decisions which would lead to it. by contrast, exploratory forecasting explores what is possible, irrespective of its desirability (veikko, kanerva, and kouri 2009). ethical assessment of the future is rarely a formal component of foresight research project (bell 1996; poli 2011; brey 2012; clarke 2005; grunwald 1999; harris et al. 2010). key foresight research figures such as wendell bell have attempted to determine what ethical values are universally accepted by all peoples in all cultures (bell 1996). however, bell has since been criticised for accepting as objective and universal what are merely western cultural values (poli 2011; dator 2011; rubin 2011). a less controversial approach has been to incorporate ethical values as research data. here distinct sets of exploratory and normative data are gathered. for example, the functionality of future technologies can be drawn the visions of those creating them, after australasian journal of information systems dainow 2017, vol 21, research on applied ethics and ict threats to autonomy from emerging icts 3 which philosophical sources can be mined for ethical treatments of these functions. this was the approach taken by the etica project. 2.2 etica the most comprehensive foresight research project focused on ethical assessment of emerging ict’s so far has been the eu’s etica (ethical issues of emerging ict applications) project, which ran from 2009 2011. the project’s aims were to identify “significant emerging ict’s” (stahl 2011, 1), the ethical issues these gave rise to, and to make recommendations for eu policy makers. a key output from the etica project was a taxonomy of emerging ict’s. here etica developed formal structures for technological descriptions and scenario construction (ikonen et al. 2010). it then used bibliometric analysis to assess published discourse regarding the ethical issues associated with these ict’s (heersmink, van den hoven, and timmermans 2010). the project validated these findings with expert focus groups and surveys (heersmink et al. 2010). finally, etica developed a series of recommendations (rainey and goujon 2011), together with supporting philosophical and methodological expositions (veikko, kanerva, and kouri 2009; rader et al. 2010; rainey and goujon 2011). etica identified 107 different technologies as significant and emerging, which it aggregated into eleven groups. it then identified approximately 400 ethical concerns associated with these technology groups. the aim was to collate the ethical concerns, not explore them in depth, much as an 18th century naval cartographer might map the location and outlines of a chain of islands, but not explore their interior. it is not our intention to review all of these ethical concerns, but to concentrate on issues concerning autonomy. we shall first catalogue how human autonomy can be challenged by each technology group, then elucidate the common characteristics shared by all. however, the meaning of the term ‘autonomy’ is much contested, so what one considers a threat to autonomy depends on what one means by the term. it is therefore necessary to briefly explore the concept of autonomy. our aim is not to produce a definition of the term ‘autonomy’, but to better understand the different ways it may be used when discussing ethical threats. 3 autonomy emmanuel kant was the first to apply the concept of autonomy to human beings (schneewind 2007; dryden 2015; paul, miller, and paul 2003; dworkin 1988). before kant the term ‘autonomy’ was a purely political one. a state was said to be “autonomous” if its laws were drafted within that state, as opposed to being drafted by a distant imperial court or some similar extra-national body (dworkin 2015; schneewind 2007). kant applied the concept to the individual in groundwork of the metaphysics of morals (kant 1785). his aim was to justify the right of each individual to make their own judgements regarding morality. for the previous 100 years, enlightenment thinkers had been seeking arguments to combat the politically dominant understanding of ethics, in which ordinary people were seen as too weak-willed to act morally without threats of punishment and promises of reward. under this view, society was dependant on the guidance of those few exceptional people whom god had enabled to understand and teach his moral laws. the essence of morality for everyone else was to do what they were told (schneewind 2007). groundwork of the metaphysics of morals set out to prove that each person drafted their own rules for their own conduct, that these rules constituted each person’s moral code, and that no education was required for this ability it was universally present in all humans. kant’s labelled the innate capacity of all humans to determine what was morally correct as ‘autonomy.’ he then used this capacity for autonomy as the basis for human dignity, which then became the basis for human rights. thus kant’s concept of autonomy created an interlinking of the concepts of individuality, freedom, morality, autonomy, dignity and politics (schneewind 2007; bittner 2014; dryden 2015; paul, miller, and paul 2003). since kant introduced the concept, a number of differing definitions of autonomy have arisen. the range of such accounts, as well as the lack of progress towards any consensus, has led to the concept of “regimes” of autonomy (anderson 2014; dworkin 2015) – clusters of similar, but not identical, theories, such that “about the only features held constant from one author to australasian journal of information systems dainow 2017, vol 21, research on applied ethics and ict threats to autonomy from emerging icts 4 another are that autonomy is a feature of persons and that it is a desirable quality to have.” (dworkin 2015, 8). kant’s criteria for autonomy were, by later standards, rigid and limited. under kant’s account, people were required to determine what was right or wrong without reference to the consequences of their actions. furthermore, under kant, rules of conduct only counted as a moral if the person believed everyone should act the same way. there was no room for moral pluralism in kant, or tolerance of other people acting differently (kant 1998; guyer 2003; johnson 2014). this type of definition of autonomy has come to be known as a “substantive procedural” account. a “procedural” account of autonomy takes the position that autonomy is a specific form of thinking process (ie: follows a specific procedure). a subset of procedural accounts are substantive, adding the requirement that autonomy also involves thinking about certain things. by contrast, “content-neutral” procedural accounts try to define autonomy without reference to any particular concern or aim. content-neutral accounts of autonomy form the majority of modern accounts (dryden 2015) and are, perhaps, more suited to multi-cultural or pluralistic societies in that they allow individuals to determine for themselves what sources and values to consider when making ethical decisions. the value given to autonomy varies according to the importance given to competing claims, most frequently those of paternalism and those forms of communitarianism which devalue individual choice in favour of community needs (bell 2014). kant’s original concept was developed in an effort to remove external authority as a source of morality. kant specifically identifies autonomy as oppositional to outside influence. however, we are all influenced by external forces, including family, cultural and religious influences. consequently, outside influence and the point at which it reduces autonomy is a problem which must be dealt with by every definition of autonomy. one response has been to argue that the concept of autonomy as self-governance isolates the individual from their society and morally devalues family, community, culture and tradition (christman 2014; mackenzie and stoljar 2000; donchin 2000; buss and zalta 2015). these positions have given rise to relational and social conceptions of autonomy based on the necessity of human sociality. it has also been argued autonomy, if conceived of as only certain forms of thinking, does not account for the full range of human emotional and physical experience, such as trained muscle action (e.g.: playing sport or music) or healthy reflexes, which can also express personal autonomy (meyers 2005, 1989). these issues collide with particular urgency where ict’s become concerned with care for the ill or aged (agich 2003; burmeister 2016; dworkin 1988), but also wherever ict’s come to mediate what someone considers to be core elements of their life. kant applied autonomy strictly to the moral realm, though it had political implications. since then western philosophy has come to term the capacity to determine one’s own moral codes as ‘moral autonomy.’ to moral autonomy have been added the concepts of “personal autonomy” (the capacity to determine one’s own actions) and “political autonomy” (the capacity to make one’s own political decisions and have them heeded) (dryden 2015; anderson and christman 2005). what these three definitions share is the concept of “self-governance.” ict’s threaten human autonomy whenever they interfere with this self-governance. this occurs whenever ict’s make decisions for people, especially when an ict imposes on the user a way of doing something which is antithetical to the manner in which the user would have chosen. some threats to autonomy are universal under all definitions of autonomy, but most depend on the particular form of autonomy being used. reducing autonomy may be justified on other grounds but such a justification, like the threat, will be based on the version of autonomy being used. it will often be the case that a justification works under one concept of autonomy, but not under others. 4 etica’s technology groups and their threats to autonomy identification of ethical issues of significant emerging ict’s is dependent upon first having identified the ict’s which are both significant and emerging. etica defined “significant” technologies as those which were likely to have an important influence on society and australasian journal of information systems dainow 2017, vol 21, research on applied ethics and ict threats to autonomy from emerging icts 5 conceptions of the self. it defined “emerging” technologies as those on a stable development path and likely to be in common use by 2030 (stahl 2011). 107 technologies meeting these criteria were identified, including 3-d printing, sensor networks, augmented reality, locationbased services, lab-on-a-chip, molecular electronics, nanobots, mobile payment systems, robotics, social network analysis, speech recognition, internet of things, and wireless power. these were then organised into eleven groups on the basis of shared characteristics, such as type of user, operational context, functionality, and relation to other technologies. for example, both wireless sensor networks and wearable computing were placed into the category of ambient intelligence. the project identified the following eleven technology groups: 1. affective computing 2. ambient intelligence 3. artificial intelligence 4. bioelectronics 5. cloud computing 6. future internet 7. human/machine symbiosis 8. neuroelectronics 9. quantum computing 10. robotics 11. virtual and augmented reality (ikonen et al. 2010, 42; stahl 2011, 5) not all technology groups were seen by etica as generating ethical threats in and of themselves. etica assessed quantum computing as unlikely to have any impact on society in the next 10-20 years, while cloud computing and future internet were seen as enablers of other technologies and not as raising unique ethical issues themselves. we shall now briefly examine the remaining technology groups and discuss some of the ways in which each has the potential to reduce human autonomy. our aim is not to explore the details of the individual threats, but simply to see how autonomy can be threatened by each technology. while a few technologies threaten autonomy no matter how it is defined, the central point of this survey is to demonstrate that most threats only appear under specific definitions of autonomy. 4.1 affective computing affective computing involves treating human emotion as input and generating output which either emulates human signals of emotion or which seeks to manipulate human emotions. here ethical concerns derive from the importance of human emotion in communication and the risks of manipulation and misunderstanding. how affective computing threatens autonomy depends on whether affective computing works or not. unlike some other technologies, there is no guarantee it will ever be possible to build systems which can accurately read human emotions (barrett, gendron, and huang 2009; picard 2003). the greatest danger with affective computing lies in the possibility that people will think it is working when it is not. we can usually tell when most technologies operate incorrectly. however, there is no simple way of checking to see if an affective computing system’s assessment of a person’s emotion is accurate. incorrect assessment of someone’s emotions threaten their autonomy when that data is used to make decisions which affect them. if ict’s can successfully understand human emotions, affective computing threatens privacy through the ability to glean information about people which they do not wish revealed. wherever such information is used to make decisions affecting the person, it may constitute a restriction of their autonomy. here it depends on the form of autonomy one subscribes to. some “second-order” procedural accounts of autonomy hold that autonomy is preserved provided one would have agreed with the decision if they had been given the chance (dworkin 1988, 2015). however, accounts of autonomy which do not allow for such “second-order” assessments hold that any decision denied the person is a restriction on their autonomy. if australasian journal of information systems dainow 2017, vol 21, research on applied ethics and ict threats to autonomy from emerging icts 6 those decisions were made in an effort to benefit the person, they may be justifiable on the grounds of form of paternalism. otherwise they simply constitute a form of oppression. affective computing therefore constitutes a threat to autonomy whenever affective data is used to make decisions about someone, unless one is using a second-order procedural definition of autonomy and the person would have agreed with the decision if they had been given the chance. emulation of emotion makes possible threats to autonomy in that it offers the chance to overwhelm rationality with emotion. any manipulation of a person’s emotions which prompts them to make a decision they would not have made otherwise constitutes a restriction of autonomy, irrespective of how autonomy is conceived. the concern is therefore that introducing affective output changes the power balance between system and user. if the ability to accurately detect emotions is combined with the ability to emulate emotional expression, the power of such systems to manipulate and persuade becomes greater than any previous human invention. in this sense, affective systems become human manipulation technology. 4.2 ambient intelligence ambient intelligence refers to ic technology which is embedded into the environment. its defining characteristic is the invisibility of the devices, often associated with automated input (for example, a change in room temperature) and less-than obvious responses by the ambient system (for example, changing the air conditioning settings). ambient intelligence offers the capability of personalising the environment to the individual user. where personalisation is under control of the user, it represents an extension of their autonomy. where personalisation is outside the user’s control, it represents a reduction in autonomy under most, but not all, definitions. second-order accounts allow for the preservation of autonomy if the user would have agreed to the personalisation had they been given the chance. only under such accounts is autonomy preserved by automated environmental personalisation. if the personalisation is intended to improve the user’s quality of life, reductions in autonomy may be justifiable on the grounds of paternalism. however, that is a problematic debate which cannot be resolved in terms of universal principles, but will ride on the specifics of each case; the type of personalisation, the individuals involved and their personal values. ambient intelligence shares several ethical issues with affective computing. autonomy is threatened by poor quality personalization services, either as a result of an inadequate understanding of human nature, or by the skewing of personalisation protocols resulting from commercial interest or developer ignorance (or bias). inference of user needs from their behaviour is particularly problematic, especially with regard to children. there is the danger that users will be forced to change their behaviour to get the best out of ambient systems, and so end up being “trained” by their environment’s designers (soraker and brey 2007). ambient intelligence creates the risk of the personal environment acquiring power over people. since ambient intelligence makes the personal environment controllable by third parties, the personal environment becomes a contestable zone, open to commercial and state interests, both in terms of gathering information and seeking control. some contestation of the personal environment has always occurred. for example, disputes over noise can be traced back to the earliest cities (goldsmith 2012). however, ambient intelligence represents such a significant increase in power that contestation for the personal environment becomes a new type of issue – the power to (potentially) totally control someone else’s personal space. 4.3 artificial intelligence etica examined both “hard” ai (emulation of human thought) and “soft ai” (e.g.: expert systems and software agents). ethical concerns focused on the use of soft ai as an enabler for other emerging technologies, such as ambient intelligence and affective computing. while issues associated with hard ai were considered, etica’s judged that hard ai was unlikely to do more than appear in earliest prototypes by 2030 and so was outside the scope of the project. australasian journal of information systems dainow 2017, vol 21, research on applied ethics and ict threats to autonomy from emerging icts 7 our discussions of ambient intelligence and affective computing have both shown how human autonomy can be threatened by the power of these systems over people. ai significantly enhances this power, and so can be an enabler of threats to autonomy wherever it is deployed within ict systems. particularly problematic is the possibility that ai may enable systems to learn things about people not anticipated by designer or user. implementation of informed consent becomes extremely difficult when one does not know what information may be acquired. it has been suggested the solution is to build ethical reasoning into ai systems (dennis et al. 2013). however, ethical systems are abstract principles of value, not problemsolving algorithms (kraut 2016; graham 2004). ethical dilemmas only exist where multiple ethical values conflict because it is these conflicting values themselves which create the dilemma. ai developers are not, therefore, able to reach for an existing corpus of ethical problem-solving algorithms as if the matter were a simple sort task. any implementation of ethical processing will therefore impose someone’s particular ethical values on the system. should such an ethical value set not accord with the user’s set of ethical values, the user’s autonomy will be compromised by the actions of their ai-enabled system. the only way to counter such a threat would be to allow users to customise their ai-enabled systems with their own ethical values and reasoning. 4.4 bioelectronics etica defined bioelectronics as ict systems which interact directly with the human body. the primary ethical considerations were focused on use in health care, but those who see bioelectronics as a path to artificial human bodies were also considered. here bioelectronics represents the potential for technological modification of the human form. it therefore directly confronts the essence of what it is to be human and impacts core human values, such as autonomy, freedom and dignity. bioelectronics can be used to change people’s internal states by delivering medicines and intervening in bodily processes and thus has the potential to reduce autonomy. however, this concern is dependent on the concept of autonomy used. if one accepts a second-order procedural concept of autonomy (dworkin 1988), autonomy may be preserved post-hoc by agreeing with the earlier bioelectronic intervention. however, conceptions which situate autonomy at the moment of decision must see any bioelectronic intervention as a reduction of autonomy. whether this loss of autonomy can be justified on paternalistic grounds depends on the intent behind the intervention. however, it may be possible to reframe the issue such that autonomy is not threatened. discussions of bioelectronics tend to treat intervention as morally equivalent to an intentional act. under this perspective, each individual bioelectronic intervention is treated as a discrete act to which assent may be given or withheld. this raises problems for the preservation of autonomy because of the requirement for ongoing consent (agich 2003). however, where this intervention is frequent and automated, such as is the case with drug delivery, it may be more productive to think of it as a form of reflex. some definitions of autonomy incorporate reflex into their schema as non-cognitive expressions of the self (meyers 2005, 1989). treating bioelectronic interventions as artificial reflexes removes any threat to autonomy under such definitions. the nature of bioelectronic intervention means it is difficult to resist. this places a great deal of potential power in the hands of those controlling the devices; granting them the ability to control someone to a degree not possible by other means. the capability of bioelectronics to change internal bodily states and processes is the same as that of drugs. accordingly, bioelectronics shares many of the same issues pertaining to autonomy as do drugs. there are today debates about the advisability of treating some mental states as illnesses, concerns about inappropriate use of drugs to control behaviour in elderly and children, and other issues relating to appropriate boundaries to medical intervention (smith 2012). these same issues apply to interventions by bioelectronic devices. threats to autonomy apply here in two ways; firstly, “authentic” conceptions of autonomy involve the concept of authenticity (that there is some given essential nature to each individual), such that autonomy requires being in conformance to that essential self (kühler and jelinek 2013). these accounts hold that there is australasian journal of information systems dainow 2017, vol 21, research on applied ethics and ict threats to autonomy from emerging icts 8 something innate about humans which cannot be changed without harming their authenticity, such as particular forms of cognition (frankfurt 1971). theological accounts of autonomy may also depend upon a divinely-ordained authentic human nature (wilson 1978; niebuhr 2004). meanwhile, “coherentist” accounts base autonomy on continuity with personal history, such that changes or decisions which radically depart from someone’s previous patterns or tastes are inauthentic, and thus reductions in autonomy (miller 1981; ekstrom 1993). bioelectronic interventions which take a person away from an essential self, however that self is conceived, are reductions to these forms of autonomy. finally, where bioelectronic interventions are unwelcome and forced on the individual, they constitute a clear reduction in autonomy under all definitions of autonomy. 4.5 human-machine symbiosis etica characterised human-machine symbiosis as the pairing of innate human capabilities with ict. this definition covers an extremely wide range of systems, including haptic interfaces, decision-support systems, computer-assisted surgery, augmented reality and direct neural interfaces. there is considerable overlap with other technologies, especially bioand neuroelectronics and artificial intelligence. attempts to enhance human beings through the addition of ict components may constitute reductions in autonomy under the authenticist views just outlined. therapeutic devices, designed to replace lost human capabilities rather than enhance them, may be autonomypreserving, but other accounts can hold that even these constitute a loss of autonomy (bublitz and merkel 2009). this is particularly the case where those devices require monitoring or control by others. a device may therefore reduce autonomy when it is operated by someone else, but enhance autonomy where it is controlled by the user (sharon 2017). such concerns are not limited to devices implanted into people’s bodies. use of external devices, for example, as aids to memory, may constitute reductions to autonomy, especially under accounts based on the concept of an “extended self” (olson 2011; rachlin and jones 2010), in which parts of our personal identity are embedded in external objects, such as clothing or mobile phones (ahuvia 2005; belk 1988; turkle 2011). for someone who embeds part of their identity in their phone, even something as simple as an enforced software update may constitute a reduction in their autonomy. 4.6 neuroelectronics etica defined neuroelectronics as technology interfacing between the human nervous system and electronic devices. while neuroelectronics clearly overlaps with bioelectronics, etica felt the intimate connection between the person and their brain meant neuroelectronics could not be adequately examined within a general treatment of bioelectronics. as with bioelectronics, autonomy is not threatened when neuroelectronics is used to gather data, but only when used to induce changes. since changes cannot be made without knowledge of internal states, gathering data is, however, a privacy concern as an enabler of reductions to autonomy. any neuroelectronic system which produces changes within the person constitutes a threat under most accounts of autonomy. while this is clearly the case with conceptions of autonomy which involve an authentic nature or coherent life-history, it may also be the case under second-order procedural accounts (which allow for autonomy preservation if someone agrees to a change afterwards). this is because a person’s agreement to a previous procedure may be merely the result of the changes they have undergone. where these devices are installed to deal with cognitive impairment, such as dementia, informed consent is impossible and thus autonomy must be reduced (though this may be justifiable on paternalistic grounds). as with bioelectronics and human-machine symbiosis, any accounts of autonomy which involve a divinely-derived authentic human nature will view all neuroelectronic changes as reductions in autonomy. australasian journal of information systems dainow 2017, vol 21, research on applied ethics and ict threats to autonomy from emerging icts 9 4.7 robotics etica defined robots as “machines with motor function that are able to perceive their environment and operate autonomously” (ikonen et al. 2010, 114). etica’s focused on new developments which increase the mobility and intelligence of robotic devices, permitting their deployment into wider areas of society, such as the home and healthcare, and focused on robots built for specific roles within these contexts. etica did not consider general-purpose, fully mobile devices controlled by strong artificial intelligence seeking to make war on humanity. the degree to which robotics can threaten autonomy depends on the definition of autonomy used and the use made of the robot. as has been noted earlier, traditional procedural accounts of autonomy as self-determination have been criticised for unrealistically portraying people as isolated individuals and leaving no space in the account for communitarian elements such as the influence of family or culture (mackenzie and stoljar 2000; stoljar 2015). on this basis some have gone so far as to argue autonomy is an unattainable ideal (strawson 1994). a more common response has been to develop a treatment of autonomy which includes space for the influence of others, such as the concept of a “social self” (meyers 2005, 44). the central premise in relational accounts of autonomy is the necessary role of other people in the development of one’s values and the centrality of interpersonal relationships to human existence; that it is not possible to function autonomously without the influence of others (mackenzie and stoljar 2000; stoljar 2015; donchin 2000). such conceptions of autonomy are threatened when robots replace humans in roles which have social externalities, such as health care and education. while robots are able to undertake the central tasks, the loss of the human contact constitutes a reduction in relational autonomy. the consequence of this is that there may be some tasks for which robots are not ethically suitable, for no other reason than that they are robots. if the social externalities of a work role are essential for the maintenance of the recipient’s autonomy, then that role must be reserved for humans. were we to replace nurses or teachers with robots, we could thus see people argue they have an inalienable human right to refuse to be served by a robot and to demand human service. 4.8 virtual/augmented reality etica’s union of virtual reality with augmented reality into one technology group was based on the common feature they share the imposition of digital output onto the human sensory field. etica defined virtual reality as occurring where digital output completely replaced sensory data. where it did not completely replace external sensory data, etica used the term ‘augmented reality’ (heersmink, van den hoven, and timmermans 2010, 114). etica’s concerns were founded on the fact that such systems mediate or replace interaction with the physical environment. autonomy is threatened when virtual or augmented realities depict objects, people and places in the real world and thus become involved with the many ethical issues associated with depiction, cultural bias and its influence the development of attitudes and taste (zimbardo and leippe 1991). the imposition of foreign cultural models in virtual realities would constitute a threat to relational and social conceptions of autonomy (tomlinson 1991), while lack of alternatives and lack of configuration options threatens all conceptions of autonomy. 4.9 summary many of the technologies examined provide personalised services tailored to the individual. these technologies threaten human autonomy when they fail to personalise effectively. this may result from misreading the user, insufficient granularity within the personalisation, lack of user control, inadequate modelling of the user or by imposing on the user ways of living which are contrary to their values. a number of the technologies change power balances within society, granting for the first time, or greatly increasing, the actant power of the environment over the individual. this power threatens autonomy if used to impose on the user or to enable autonomy-limiting personalisation. some conceptions of autonomy contain elements which can be disrupted by some ict’s. conceptions of autonomy which include inter-personal relationships are challenged by robotics, while conceptions of autonomy which make reference australasian journal of information systems dainow 2017, vol 21, research on applied ethics and ict threats to autonomy from emerging icts 10 to authenticity, coherent life-history, or some form of given human nature, are challenged by bioand neuro-electronics and may also be challenged by human-machine symbiosis. irrespective of the nature of autonomy used, our survey of etica’s research suggests there are certain characteristics any ict is likely to have if it threatens autonomy: 1. surveillance: surveillance consists of the obtaining of information pertaining to an individual from their behaviour or communication, followed by the use of that data for purposes, either unknown to that individual or against their wishes (lyon, ball, and haggerty 2012). while it can be used in a paternalistic fashion to enhance the individual’s autonomy, our concern is when it is used in a manner which limits the individual's autonomy. the presence of a surveillance system is a necessary precondition for the use of personal information by others and for personalisation services because it constitutes the means by which information supporting personalisation is gathered. control of the outflowing data by the individual is therefore an effective way in which to retain autonomy. thus control of privacy is an enabling right to autonomy. 2. disparity of control: disparity of control occurs when a third party has a greater control over the use of one’s personal data and the autonomy-limiting processes than oneself. this lack of control can be accomplished through lack of knowledge on the part of the individual as to what is occurring, through “take it or leave it” terms of use combined with a lack of alternatives, and through the lack (or concealment) of user configuration capabilities. in some cases, such as data profiling, not just functions, but entire industries may be concealed (federal trade commission 2014; turow 2011; dainow 2015). 3. insufficient configurability: autonomy can be threatened by lack of variability in configuration options suitable to reflect the variations in the user’s desires, style of operation or range of outcomes. this lack of variation can stem from:  a lack of recognition by the ict developers of the need for, or even existence of, such variations.  insufficient consideration of need for user variability during the design process.  insufficient granularity of configuration options.  systemic biases within the delivery mechanisms, including business models, market competition, regulatory framework or any other factors regarding the operational delivery of services. for example, a system might be highly configurable, but delivered to users with a standardised configuration which cannot be changed in an effort to reduce support costs. 4. insufficient variation in operational models: many new ict’s are owned and operated by a very limited number of providers (noam et al. 2003; hillis, jarrett, and petit 2013). often a single provider dominates the market to near-monopoly levels. this limits choice of service provision, and consequently the model under which it is provided. in many cases the business model is identical across all providers within an ict sector, such that change of provider does not change the circumstances under which the user accesses the service. for example, social networking is only available under a capitalist for-profit model in which service provision is exchanged for personal data which is then commoditised (or monetised). non-profit and privacy-preserving social network systems do not exist to the degree that a real choice is available. in addition, ip protection promotes walled gardens and suppresses interoperability, further limiting options for choice by users because it is not possible to interoperate between social network providers. australasian journal of information systems dainow 2017, vol 21, research on applied ethics and ict threats to autonomy from emerging icts 11 5 conclusions the common feature to all conceptions of autonomy is self-determination. what one selfdecides, how, on what basis, according to what procedure, and subject to which influences, varies from account to account. yet efforts to preserve autonomy are central to ict ethics (spiekermann 2015). efforts to reduce the negative ethical impact of ict’s on people through processes such as value-sensitive design need to be cognisant of the various conceptions of autonomy relevant to the intended functionality of the system. this is especially the case with technologies which enter into spheres of life which have previously been purely human or which reduce the power people have over their personal lifeworld. given the wide range of definitions of autonomy, no single definition can possibly cover all cases. because autonomy is frequently central to the ethical status of any technology, it is rarely possible to categorise any ict as threatening or reducing autonomy without reference to the form of autonomy used. methodologies to improve the ethical sensitivity of ict technology, such as value-sensitive design, make frequent reference to the need to specify a particular brand of ethics, such as deontology or utilitarianism, from which to draw values (spiekermann 2015; nissenbaum 2001; winfield, blum, and liu 2014), but the application of such values still depends on the concept of autonomy being used. it is generally assumed that there is a single, fixed meaning to autonomy, at least in any given context, and that preservation of autonomy simply requires, at most, identification of the correct version (manders-huits 2011). some have noted the meaning of autonomy can vary with different contexts (burmeister 2016), but it is possible that it only appears the meaning has changed because the difficulties perceived apply to one version of autonomy and not others, thus making the problematic version of autonomy obvious. our review of etica’s survey of threats to autonomy has revealed that many versions of autonomy may be applicable within the one context, some of which may have problems while others do not. thus, attempts to find the “correct” version of autonomy for any context may be moot. on this basis, it seems more appropriate to develop ict functionality which does not depend on a single definition of autonomy, but adapts to the user’s own definition. this implies a loose pairing of system behaviour to coding, such that output can be fine-tuned after deployment, either by a user, their technical support staff or through the use of soft ai components focused on adapting the system to user feedback. all significant emerging icts can threaten autonomy, but there is no simple way of avoiding this. efforts such as value-sensitive design stand more chance of success if they focus less on incorporating a single set of values and focus instead on ways by which users may adapt ict’s to reflect their own individual values. references agich, george j. 2003. dependence and autonomy in old age: an ethical framework for long-term care. 2nd ed., rev. cambridge, uk ; new york: cambridge university press. ahuvia, aaron c. 2005. ‘beyond the extended self: loved objects and consumers’ identity narratives’. journal of consumer research 32 (1): 171–184. anderson, joel. 2014. ‘regimes of autonomy’. ethical theory and moral practice 17 (3): 355– 68. doi:10.1007/s10677-013-9448-x. anderson, joel, and john christman. 2005. ‘introduction’. in autonomy and the challenges of liberalism: new essays, 1–23. cambridge, uk ; new york: cambridge university press. barrett, lisa feldman, maria gendron, and yang-ming huang. 2009. ‘do discrete emotions exist?’ philosophical psychology 22 (4): 427–37. doi:10.1080/09515080903153634. belk, russell w. 1988. ‘possessions and the extended self’. journal of consumer research 15 (2): 139–168. australasian journal of information systems dainow 2017, vol 21, research on applied ethics and ict threats to autonomy from emerging icts 12 bell, d. 2014. ‘communitarianism’. in the stanford encyclopaedia of philosophy , edited by edward n. zalta, summer 2014. http://plato.stanford.edu/archives/sum2014/entries/communitarianism/. bell, wendell. 1996. foundations of future studies. new jersey: transaction publishers. bittner, rüdiger. 2014. ‘autonomy modest’. erkenntnis 79 (7): 1329–39. brey, philip a. e. 2012. ‘anticipatory ethics for emerging technologies’. nanoethics 6 (1): 1– 13. doi:10.1007/s11569-012-0141-7. bublitz, jan christoph, and reinhard merkel. 2009. ‘autonomy and authenticity of enhanced personality traits’. bioethics 23 (6): 360–374. burmeister, oliver k. 2016. ‘the development of assistive dementia technology that accounts for the values of those affected by its use’. ethics and information technology, 1–14. doi:10.1007/s10676-016-9404-2. buss, sarah, and edward n. zalta. 2015. ‘personal autonomy’. stanford encyclopedia of philosophy. http://plato.stanford.edu/entries/personal-autonomy/. christman, john. 2014. ‘relational autonomy and the social dynamics of paternalism’. ethical theory and moral practice 17 (3): 369–82. doi:10.1007/s10677-013-9449-9. clarke, steve. 2005. ‘future technologies, dystopic futures and the precautionary principle’. ethics and information technology 7 (3): 121–126. consumer reports. 2016. ‘tesla’s autopilot: too much autonomy too soon’. consumer reports. http://www.consumerreports.org/tesla/tesla-autopilot-too-much-autonomytoo-soon/. dainow, b. 2015. ‘digital alienation as the foundation of online privacy concerns’. computers & society ethicomp special issue: 109–17. doi:10.1145/2874239.2874255. dator, jim. 2011. ‘wendell bell: the futurist who would put my grandmother in prison’. futures 43 (6): 578–582. dennis, louise, michael fisher, marija slavkovik, and matt webster. 2013. ‘ethical choice in unforeseen circumstances’. in towards autonomous robotic systems, 433–445. springer. donchin, anne. 2000. ‘autonomy and interdependence: quandaries in genetic decisionmaking’. in relational autonomy: feminist perspectives on autonomy, agency, and the social self, edited by catriona mackenzie and natalie stoljar, 236–58. new york: oxford university press. dryden, jane. 2015. ‘autonomy’. internet encyclopedia of philosophy . http://www.iep.utm.edu/autonomy/. dworkin, gerald. 1988. the theory and practice of autonomy. cambridge studies in philosophy. cambridge ; new york: cambridge university press. ———. 2015. ‘the nature of autonomy’. nordic journal of studies in educational policy 1 (0). doi:10.3402/nstep.v1.28479. ekstrom, laura waddell. 1993. ‘a coherence theory of autonomy’. philosophy and phenomenological research 53 (3): 599–616. federal trade commission. 2014. ‘data brokers: a call for transparency’. federal trade commission. https://www.ftc.gov/system/files/documents/reports/data-brokers-calltransparency-accountability-report-federal-trade-commission-may2014/140527databrokerreport.pdf. frankfurt, harry g. 1971. ‘freedom of the will and the concept of a person’. the journal of philosophy 68 (1): 5–20. australasian journal of information systems dainow 2017, vol 21, research on applied ethics and ict threats to autonomy from emerging icts 13 glenn, jerome c. 2009. ‘introduction to futures research methods’. in futures research methods, edited by jerome c glenn and theodore j gordon, 3rd ed. washington, dc: the millenium project. godet, michel. 2009. ‘11. structural analysis’. in futures research methods, 3rd ed. washington, dc: the millenium project. goldsmith, mike. 2012. discord: the story of noise. oxford: oxford university press. gordon, theodore j, and jerome c glenn. 2004. ‘‘integration, comparisons, and frontiers of futures research methods’. in eu-us seminar: new technology foresight, forecasting & assessment methods. vol. 1314. graham, gordon. 2004. eight theories of ethics. london; new york: routledge/taylor and francis group. grunwald, armin. 1999. ‘technology assessment or ethics of technology? reflections on technology development between social sciences and philosophy.’ ethical perspect 6 (2): 171 – 182. guyer, paul. 2003. ‘kant on the theory and practice of autonomy’. in autonomy, edited by ellen frankel paul, fred dycus miller, and jeffrey paul, 70–98. cambridge: cambridge university press. haegeman, karel, elisabetta marinelli, fabiana scapolo, andrea ricci, and alexander sokolov. 2013. ‘quantitative and qualitative approaches in future-oriented technology analysis (fta): from combination to integration?’ technological forecasting and social change 80 (3): 386–397. harris, ian, penny duquenoy, richard jennings, david pullinger, and simon rogerson. 2010. ‘ethical assessment of new technologies: a meta-methodology’. journal of information, communication and ethics in society 9 (1): 49–64. heersmink, richard, jeroen van den hoven, and job timmermans. 2010. ‘d.2.2 normative issues report’. d.2.2. etica project. heersmink, richard, job timmermans, jeroen van den hoven, and kutoma wakunuma. 2010. ‘d.2.3 normative issues workshop report’. d.2.3. etica project. hillis, ken, kylie jarrett, and michael petit. 2013. google and the culture of search. new york: routledge. ikonen, veikko, minni kanerva, panu kouri, bernd stahl, and kutoma wakunuma. 2010. ‘d.1.2. emerging technologies report’. d.1.2. etica project. j o h n s o n , r o b e r t . 2 0 1 4 . ‘ k a n t ’ s m o r a l p h i l o s o p h y ’ . e d i t e d b y e d w a r d n . z a l t a . t h e s t a n f o r d e n c y c l o p e d i a o f p h i l o s o p h y . http://plato.stanford.edu/archives/sum2014/entries/kant-moral/. kant, immanuel. 1785. grundlegung zur metaphysik der sitten. riga: johann friedrich harttnoch. ———. 1998. groundwork of the metaphysics of morals. translated by mary j. gregor. cambridge texts in the history of philosophy. new york: cambridge university press. kraut, richard. 2016. ‘aristotle’s ethics ’. in the stanford encyclopedia of philosophy , edited by edward n. zalta, spring 2016. http://plato.stanford.edu/archives/spr2016/entries/aristotle-ethics/. kühler, michael, and nadja jelinek, eds. 2013. autonomy and the self. dordrecht: springer netherlands. http://link.springer.com/10.1007/978-94-007-4789-0. lamb, roberta, and rob kling. 2003. ‘reconceptualizing users as social actors in information systems research’. mis quarterly, 197–236. australasian journal of information systems dainow 2017, vol 21, research on applied ethics and ict threats to autonomy from emerging icts 14 lyon, david, kirstie ball, and kevin d haggerty. 2012. routledge handbook of surveillance studies. abingdon, oxon; new york, ny: routledge. mackenzie, catriona, and natalie stoljar, eds. 2000. relational autonomy: feminist perspectives on autonomy, agency, and the social self. new york: oxford university press. manders-huits, noëmi. 2011. ‘what values in design? the challenge of incorporating moral values into design’. science and engineering ethics 17 (2): 271–287. doi:10.1007/s11948-010-9198-2. meyers, diana t. 1989. self, society, and personal choice. new york: columbia university press. ———. 2005. ‘decentralizing autonomy: five faces of selfhood’. in autonomy and the challenges to liberalism , edited by john christman and joel anderson, 27–55. cambridge: cambridge university press. http://ebooks.cambridge.org/ref/id/cbo9780511610325a011. miller, bruce l. 1981. ‘autonomy & the refusal of lifesaving treatment’. hastings center report 11 (4): 22–28. musk, elon. 2016. ‘tesla master plan, part deux’. personal blog. july 20. https://www.tesla.com/blog/master-plan-part-deux. national conference of state legislatures. 2017. ‘autonomous | self -driving vehicles legislation’. national conference of state legislatures. http://www.ncsl.org/research/transportation/autonomous-vehicles-legislation.aspx. niebuhr, r. 2004. the nature and destiny of man: a christian interpretation : human nature. library of theological ethics. westminster, john knox press. nissenbaum, h. 2001. ‘how computer systems embody values’. computer 34 (3): 118–20. doi:10.1109/2.910905. noam, eli m, donald hay, michael r baye, and john morgan. 2003. ‘the internet: still wide open and competitive?’ oii internet issue brief, no. 1. ogilvy, james. 2002. ‘futures studies and the human sciences: the case for normative scenarios’. in new thinking for a new millennium: the knowledge base of futures studies, edited by richard slaughter. london: routledge. olson, eric t. 2011. ‘the extended self’. minds and machines 21 (4): 481–495. paul, ellen frankel, fred dycus miller, and jeffrey paul, eds. 2003. autonomy. cambridge: cambridge university press. picard, rosalind w. 2003. ‘affective computing: challenges’. international journal of human-computer studies 59 (1): 55–64. poli, roberto. 2011. ‘ethics and future studies’. int. j. management concepts and philosophy 5 (4): 403–10. rachlin, howard, and bryan a. jones. 2010. ‘the extended self.’ in impulsivity: the behavioral and neurological science of discounting., edited by gregory j. madden and warren k. bickel, 411–31. washington: american psychological association. http://content.apa.org/books/12069-015. rader, michael, a. antener, rafael capurro, michael nagenborg, and l. stengel. 2010. ‘d.3.2 evaluation report’. etica project. rainey, stephen, and philippe goujon. 2011. ‘d.4.2 governance recommendations’. d.4.2. etica project. http://www.academia.edu/download/30848560/deliverable4.2.pdf. australasian journal of information systems dainow 2017, vol 21, research on applied ethics and ict threats to autonomy from emerging icts 15 rubin, anita. 2011. ‘living in the age of emotional rationality: wendell bell, social media and the challenges of value change’. futures 43 (6): 583–589. schneewind, jerome b. 2007. the invention of autonomy: a history of modern moral philosophy. 7th ed. cambridge: cambridge univ. press. sharon, tamar. 2017. ‘self-tracking for health and the quantified self: re-articulating autonomy, solidarity, and authenticity in an age of personalized healthcare’. philosophy & technology 30 (1): 93–121. doi:10.1007/s13347-016-0215-5. smith, brendan l. 2012. ‘inappropriate prescribing’. monitor on psychology 43 (6): 36. smith, merritt roe, and leo marx. 1994. does technology drive history?: the dilemma of technological determinism. cambridge, mass.: mit press. solveforx. 2016. ‘google self-driving car project website’. google self-driving car project. https://www.google.com/selfdrivingcar. soraker, johnny hartz, and philip brey. 2007. ‘ambient intelligence and problems with inferring desires from behaviour’. international review of information ethics 8: 7–12. spiekermann, sarah. 2015. ethical it innovation: a value-based system design approach. boca raton, fl.: taylor & francis. stahl, bernd carsten. 2011. ‘project final report’. etica project. statistica. 2016. ‘statistics and facts about tesla’. the statistics portal. http://www.statista.com/topics/2086/tesla/. stoljar, natalie. 2015. ‘feminist perspectives on autonomy’. in the stanford encyclopedia of philosophy , edited by edward n. zalta, fall 2015. http://plato.stanford.edu/archives/fall2015/entries/feminism-autonomy/. strawson, galen. 1994. ‘the impossibility of moral responsibility’. philosophical studies: an international journal for philosophy in the analytic tradition 75 (1/2): 5–24. tomlinson, john. 1991. cultural imperialism: a critical introduction. baltimore, md.: johns hopkins university press. trist, eric l. 1981. on socio-technical systems. ontario: ontario ministry of labour. turkle, sherry. 2011. alone together: why we expect more from technology and less from each other . new york: basic books. http://public.eblib.com/choice/publicfullrecord.aspx?p=684281. turow, joseph. 2011. the daily you: how the new advertising industry is defining your identity and your world. new haven: yale university press. veikko, ikonen, minni kanerva, and panu kouri. 2009. ‘d.1.1 heuristics & methodology report final’. d.1.1. etica project. wilson, edward o. 1978. on human nature. cambridge, mass.: harvard university press. winfield, alan ft, christian blum, and wenguo liu. 2014. ‘towards an ethical robot: internal models, consequences and ethical action selection’. in advances in autonomous robotics systems, 85–96. springer. zimbardo, philip g, and michael r leippe. 1991. the psychology of attitude change and social influence. philadelphia: temple university press. copyright: © 2017 dainow. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits nonhttp://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems dainow 2017, vol 21, research on applied ethics and ict threats to autonomy from emerging icts 16 commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. paper title (paper title style) australasian journal of information systems plumb & kautz 2015, vol. 19, selected papers from acis innovation determinants and barriers innovation determinants and barriers: a tri-perspective analysis of it appropriation within an early childhood education and care organisation melinda plumb faculty of business university of wollongong map016@uowmail.edu.au karlheinz kautz faculty of business university of wollongong kautz@uow.edu.au abstract empirical studies on information technology (it) in early childhood education and care organisations are scant, despite an increasing number of these organisations choosing to innovate with it. this paper presents a framework to understand the appropriation of it as an innovation within such an organisation. the framework consists of three perspectives on innovation: an individualist, a structuralist and an interactive process perspective. while the first focuses on concepts such as leadership, it champions, previous it exposure, the second focuses on organisation size, parents as stakeholders, competitors, government compliance and regulatory requirements. the third perspective views the innovation as a dynamic phenomenon of change, produced by the continuous interaction of the innovation content, its context, and the appropriation process as related in an interactive process. we demonstrate the framework’s applicability and determine that the three perspectives supplement each other and together provide a deeper understanding of the it appropriation process in terms of innovation determinants and barriers. keywords: information technology, appropriation, early childhood organisation, educators 1 introduction the number of early childhood education and care (ecec) organisations who are innovating with information technology (it) is increasing, with interest and support for it to be integrated into policy, curriculum and practice (bolstad 2004; barron et al. 2011; palfrey and gasser 2008, preston and mowbray 2008, spears 2009 cited in bourbour et al. 2014). to date there have been few empirical studies on it in ecec organisations. plumb et al. (2013) found that the majority of existing research involves descriptive studies of use by the educators with the children and pedagogical benefits of the use of the it as a teaching and learning tool with young children, interspersed with a few studies examining the acceptance of the it by children and/or educators. the diversity of research in terms of theory and methodology is limited, as these studies have mostly relied on traditional individual-level adoption theories such as the technology acceptance model and diffusion of innovations (see for example al-qirim 2011, 2012) and primarily focus on individual factors, although some studies make additional mention of organisational structure, environmental and contextual influences (e.g. clark and luckin 2013; crichton et al. 2011). the studies employ these theories in a quantitative manner and provide useful information on factors and their contribution to the outcome of technology adoption, but these studies of correlates of variables neglect the “often messy process through which teachers struggle to negotiate a foreign and potentially disruptive innovation into their familiar environment” (zhao et al. 2002 p.483). as schroeder et al. (1986) note, many studies of innovation focus on the facilitators and inhibitors to, or outcomes of, innovation in a given setting, but few examine how “innovations emerge, develop, grow or terminate over time” (p.501-502). 1 australasian journal of information systems plumb & kautz 2015, vol. 19, selected papers from acis innovation determinants and barriers this research aims to move beyond simply examining the adoption factors of it; instead we undertake to understand the process of it appropriation, “the way that users evaluate and adopt, adapt and integrate a technology into their everyday practices” (mendoza 2010 p.6) as an innovation within an ecec organisation. as part of understanding the process of it appropriation, we seek to understand not only what innovation determinants and barriers are present, but additionally how they interact over time to influence the appropriation process. 2 theoretical background the word innovation is used to describe an object, idea or process that is new, such as a new it device; however it is maintained that it is not whether an object or process is new to an environment, rather it is the perception that it is new by the adopting unit (cooper and zmud 1990; rogers and shoemaker 1971). research on innovation has been carried out at a number of different levels of analysis such as individual, group, organisational and societal (schroeder et al. 1986; slappendel 1996). with our interest in ecec organisations, we focus on the organisational level. slappendel (1996) conducted an extensive review of the existing literature on innovation in organisations and developed a framework to classify the body of literature based on the assumptions of who and what causes innovation within an organisation. the framework affords three perspectives: • the individualist perspective explains innovation determinants in terms of the actions and personality traits of the organisational participants. the perspective views individuals as self-directed agents who are rational beings, unconstrained by external factors, and make decisions which are guided by the goals that they set. individual characteristics such as age, gender, educational level, and personality are of interest, in addition to concepts such as change agents, leaders and innovation champions. • the structuralist perspective assumes that organisational characteristics such as size, task structure, and centralisation of power are influential in determining innovation. it presumes that organisations have goals, the most basic of which includes survival. this perspective highlights the relationship between the organisation and its environment, which is actualised by including stakeholders, competitors and government policy as structural elements that influence the innovation. • the interactive process perspective views innovation as a dynamic, continuous phenomenon of change, produced by the continuous interaction of the actions of individuals and the structural influences over time. this view of innovation is in contrast to the previous two perspectives that view innovation as either being caused by individual actions, or by objective structures (slappendel 1996). as kautz and nielsen (2004) and saren (1987 cited in slappendel 1996) note, the actions of innovative individuals cannot be divorced from the activities of other individuals nor from the organisational structures within which they operate. unlike the previous two perspectives that perceive the innovation to be static and objectively defined, the interactive process perspective views the innovation as being subjectively perceived and subject to reinvention and reconfiguration. this tri-perspective framework has been extended and tested in other information system (is) related organisational change studies by kautz and nielsen (2004), madsen et al. (2006) and alaranta and kautz (2012) and allows the identification and examination of both individual/human and structural/organisational elements influencing the innovation process. in particular, through the third perspective, the framework additionally affords the ability to understand how these elements interact with each other over time to influence the innovation process. we draw on these three perspectives but further develop and refine their elements by combining and integrating existing contributions in the fields of innovation and it 2 australasian journal of information systems plumb & kautz 2015, vol. 19, selected papers from acis innovation determinants and barriers appropriation. due to the paucity of literature on it in ecec organisations (plumb et al. 2013) we look to the literature examining innovation (it-based and non-it-based) in other educational sectors (namely schools and universities) for elements of our framework, and also from the general body of organisational innovation literature, particularly those studies which included reviews of the literature (such as crossan and apaydin 2010; frambach and schillewaert 2002; slappendel 1996; wolfe 1994). in order to provide a more comprehensive understanding of the innovation process, we extend the framework to include not only innovation determinants but also potential barriers or constraints which we have identified from the small body of literature on barriers to it integration in ecec organisations. these studies identify a number of barriers related to an individual’s internal characteristics and traits, and to organisational characteristics and elements of the environment surrounding an ecec organisation. however the existing research does not investigate from a process perspective which barriers and constraints are present and influential at certain stages of the appropriation process which ertmer (1999) suggests is worthy of investigation as “different barriers are likely to appear at different points” (p.53) of the process and barriers may never be eliminated completely, suggesting instead that they will “continue to ebb and flow” (p.52). 2.1 individualist perspective although baldridge and burnham (1975) suggest that individual characteristics are poor predicators of adoption of innovations, our study of the it appropriation process looks not just at adoption but also the adaptation and integration of it, therefore we still consider it useful to examine a number of individual-level antecedents. the attitudinal state of an organisation’s members is considered influential in organisational innovation (pierce and delbecq 1977), with studies on educational it implementation demonstrating the dependency on the attitudes of educators (bullock 2004, kersaint et al. 2003, woodrow 1992 cited in albirini 2006). the existence of champions and their role in facilitating successful technological innovation is wellrecognised in the innovation literature (howell and higgins 1990). a champion is defined as a person who makes “a decisive contribution to the innovation by actively and enthusiastically promoting its progress through the critical [organisational] stages” (achilladelis, jervis, and robertson 1971 p.14 cited in howell and higgins 1990 p.317). in previous innovation studies within schools (daft 1978; grunberg and summers 1992; sharma 2001), leadership was considered influential in the success of innovation. in their examination of ipad adoption and use in the tertiary educational sector, murphy (2011) found previous technology exposure promoted uptake of the innovative it devices. as the individualist perspective focuses on an individual’s internal characteristics and traits, the literature on barriers to it integration identified negative educator beliefs and attitudes (fenty and mckendry anderson 2014; ihmeideh 2009; joshi et al. 2010; li 2006; lindahl and folkesson 2012a; tsitouridou and vryzas 2004; wood et al. 2008), a lack of knowledge and skills (edwards 2005; fenty and mckendry anderson 2014; ihmeideh 2010; leung 2003; li 2006; nikleia and despo 2005; parette et al. 2013; plowman and stephen 2005; tsitouridou and vryzas 2004; wood et al. 2008), the age of the educator (parette et al. 2013; ihmeideh 2010) and lack of confidence (blackwell et al. 2014; fenty and mckendry anderson 2014; joshi et al. 2010; li 2006; nikolopoulou and gialamas 2013; plowman and stephen 2005; tsitouridou and vryzas 2004) as barriers to the it appropriation process. 2.2 structuralist perspective organisational size and complexity have been found to be significant in previous innovation studies in educational organisations (baldridge and burnham 1975; corwin 1975; daft 1978). the level of centralisation of decision making in an organisation and the formalisation, or the extent of the use of rules and formal procedures, has also been found to influence innovation (hameed et al. 2012). within the environment of schools and ecec organisations, parents of children/students are considered influential stakeholders and play a role in influencing the organisation’s innovativeness (bidwell 1965 and sieber 1968 cited in baldridge and burnham 1975; burden et al. 2012 cited in clark and luckin 2013). as larner and phillips (1994) posit, “the traditional image of parents as relatively passive partners in programmes that care for 3 australasian journal of information systems plumb & kautz 2015, vol. 19, selected papers from acis innovation determinants and barriers children has been joined by a new image of parents as consumers seeking to maximise their purchasing power in the childcare marketplace” (p.47). other environmental elements include government compliance and regulatory requirements (clark and luckin 2013) and existing infrastructure (clark and luckin 2013). competing organisations (crocombe et al. 1991 cited in slappendel 1996) have been noted in the innovation literature and may be influential in our study as the early childhood educational sector is comprised of organisations competing for the business of providing child education and care services to parents. a lack of equipment and resources (fenty and mckendry anderson 2014; ihmeideh 2009, 2010; joshi et al. 2010; leung 2003; liu and pange 2014; nikleia and despo 2005; nikolopoulou and gialamas 2013; wood et al. 2008), support (blackwell et al. 2013; fenty and mckendry anderson 2014; li 2006; liu and pange 2014; nikleia and despo 2005; nikolopoulou and gialamas 2013; plowman and stephen 2005; wood et al. 2008), training (blackwell et al. 2013; fenty and mckendry anderson 2014; ihmeideh 2009; li 2006; nikleia and despo 2005; parette et al. 2013; plowman and stephen 2005; wood et al. 2008), time (fenty and mckendry anderson 2014; ihmeideh 2009, 2010; li 2006; wood et al. 2008), funding (ihmeideh 2009; li 2006; nikolopoulou and gialamas 2013; parette et al. 2013; plowman and stephen 2005; wood et al. 2008) and it technical problems (blackwell 2013; edwards 2005; fenty and anderson 2014; ihmeideh 2009; li 2006; nikolopoulou and gialamas 2013) have also been identified as structuralist barriers or constraints to it appropriation in the literature. 2.3 interactive process perspective the interactive process perspective is of particular interest as it permits the study of the organisational innovation process: the temporal sequence of events that occur as people interact with others and the structural elements of the organisation to appropriate the innovation within the organisational context. events are instances when changes occur in the innovation ideas, people, transactions, contexts, or outcomes while an innovation develops over time (van de ven et al. 1989). according to schroeder et al. (1986), the innovation process begins with a ‘shock’, something that stimulates efforts by organisational members to begin work on an innovation. although the environment as a context is under examination as part of the structural perspective, walsham (1993) notes that it is important to see organisational change as “linked to both intraorganisational and broader contexts, and not to try to understand projects as episodes divorced from the historical, organisational or economic circumstances from which they emerge” (p.53). we therefore look to studies of innovation as a process to enrich our interactive process perspective, and draw on pettigrew’s ‘triangle’ of context, content and process (1987) from his work on studying strategic change, and on schroeder et al.’s observations from the minnesota innovation research program (1986). we combine concepts from these studies into the following three elements of our interactive process perspective: • the content of an innovation (the ‘what’) be it a product or a process, is perceived subjectively and is subject to ongoing reinvention and reconfiguration. • the context of an innovation (the ‘why’) is subdivided into inner context: the structure, corporate culture, and political context within the organisation; and outer context: the social, economic, political, and competitive environment. • the process of innovation (the ‘how’) refers to the actions, reactions and interactions from the various interested parties as they seek to move the organisation from its present to its future state. these three perspectives summarised in table 1 below form a comprehensive and coherent analytical framework that we will utilise to organise, describe and analyse our findings. 4 australasian journal of information systems plumb & kautz 2015, vol. 19, selected papers from acis innovation determinants and barriers perspectives individualist structuralist interactive process concept source concept source concept source innovation determinants attitude towards it it champions leaders previous it exposure bullock 2004, kersaint et al. 2003, woodrow 1992 cited in albirini 2006; pierce and delbecq 1977 howell and higgins 1990 daft 1978; grunberg and summers 1992; sharma 2001 murphy 2011 size complexity centralisation formalisation parents as stakeholders government compliance and regulatory requirements existing infrastructure competitors baldridge and burnham 1975; daft 1978 baldridge and burnham 1975; daft 1978 hameed et al. 2012 hameed et al. 2012 bidwell 1965 and sieber 1968 cited in baldridge and burnham 1975; burden et al. 2012 cited in clark and luckin 2013 clark and luckin 2013 clark and luckin 2013 crocombe et al. 1991 cited in slappendel 1996 shocks context (outer and inner) content process schroeder et al. 1986 pettigrew 1987; schroeder et al. 1986 pettigrew 1987; schroeder et al. 1986 pettigrew 1987; schroeder et al. 1986 barriers and constraints negative educator beliefs and attitudes lack of knowledge and skills age of the educator lack of confidence fenty and mckendry anderson 2014; ihmeideh 2009; joshi et al. 2010; li 2006; lindahl and folkesson 2012a; tsitouridou and vryzas 2004; wood et al. 2008 edwards 2005; fenty and mckendry anderson 2014; ihmeideh 2010; leung 2003; li 2006; nikleia and despo 2005; parette et al. 2013; plowman and stephen 2005; tsitouridou and vryzas 2004; wood et al. 2008 parette et al. 2013; ihmeideh 2010 blackwell et al. 2014; fenty and mckendry anderson 2014; joshi et al. 2010; li 2006; nikolopoulou and gialamas 2013; plowman and stephen 2005; tsitouridou and vryzas 2004 lack of equipment and resources lack of support lack of training lack of time lack of funding it technical problems fenty and mckendry anderson 2014; ihmeideh 2009, 2010; joshi et al. 2010; leung 2003; liu and pange 2014; nikleia and despo 2005; nikolopoulou and gialamas 2013; wood et al. 2008 blackwell et al. 2013; fenty and mckendry anderson 2014; li 2006; liu and pange 2014; nikleia and despo 2005; nikolopoulou and gialamas 2013; plowman and stephen 2005; wood et al. 2008 blackwell et al. 2013; fenty and mckendry anderson 2014; ihmeideh 2009; li 2006; nikleia and despo 2005; parette et al. 2013; plowman and stephen 2005; wood et al. 2008 fenty and mckendry anderson 2014; ihmeideh 2009, 2010; li 2006; wood et al. 2008 ihmeideh 2009; li 2006; nikolopoulou and gialamas 2013; parette et al. 2013; plowman and stephen 2005; wood et al. 2008 blackwell 2013; edwards 2005; fenty and anderson 2014; ihmeideh 2009; li 2006; nikolopoulou and gialamas 2013 temporality and influence of barriers to be identified by this study table 1. our tri-perspective analytical framework 5 australasian journal of information systems plumb & kautz 2015, vol. 19, selected papers from acis innovation determinants and barriers 3 case setting this research involves an exploratory, interpretive case study within an ecec organisation in metropolitan new south wales, australia (herein referred to as bfs). bfs is responsible for 26 early childhood centres within the region, providing education and care services for children from birth to five years old. the innovation under study is a software ‘app’ called kinderloop that runs on tablets and mobile devices, in particular on apple ipad tablets, but is also accessible on pcs via a web portal. it is promoted as a safe, secure and private way for early childhood educators to communicate with parents and families of children attending an early childhood centre, while also providing the functionality of documenting information on child activity and development. this combination of technologies will be herein referred to as ipadkinderloop. the kinderloop app began development in 2012 in response to the founder’s concern about not having appropriate times and opportunities to communicate with the educators at his children’s early childhood centre in regards to being informed about his child’s activity through the day. ipadkinderloop aims to enhance early childhood centre-parent communications through the following process: 1. an early childhood centre installs the app onto their tablets or mobile devices, which are then made available to the educators during the day; 2. at appropriate times, the educator opens the app on the device, takes a photo and writes a short description about what is occurring; the educator can link to learning outcomes, practices and principles, centre philosophy, national quality standards, policies and procedures, educational visions etc.; 3. the child/ren are ‘tagged’ in the photo; 4. the photo and annotation are then uploaded to the centre’s private kinderloop instance; 5. kinderloop automatically and securely posts update notifications to the tagged child/ren’s parents; 6. parents can then login to the centre’s private kinderloop instance using their own device with the app installed, or navigate to the online web portal using any internet-accessible computer and see all of their child's updates and can ‘like’ or comment on the posts that are visible to them. 4 research approach this research aims at obtaining a deeper understanding of organisational innovation through the process of appropriating it into an ecec organisation. we also aim to assess the applicability of the presented analytical framework, in particular the suitability of the interactive process perspective as a lens into the it appropriation process, and as such we have used the framework as background for our data collection, the coding of the data and the data analysis. data collection occurred at eight bfs centres that were appropriating ipadkinderloop between november 2013 and march 2015. not all centres were at the same ‘stage’ or level of appropriation due to differences in the timing of the roll-out; although the bfs head office mandated the use of ipadkinderloop, it was left to centre directors to decide when they would start using it. the empirical data was collected via semi-structured interviews with two or three educators at each centre, each centre director, and the chief executive officer (ceo) of the bfs organisation, resulting in a total of 24 interviews. each of the interviews lasted between 30 and 50 minutes and were recorded and transcribed verbatim to prepare the data for coding and analysis. the interview data was complimented by a collection of 12 short videos provided by the kinderloop software founder which were comprised of short testimonials from current kinderloop users, including educators, centre directors and parents/family members. these videos are publicly available on the vimeo website (http://vimeo.com/kinderloop). complimentary data (eisenhardt 1989; yin 2009) was also obtained via researcher notes of observations of current practices and the examination of secondary documents used by early childhood centres in australia including the early years learning framework and national quality framework. 6 australasian journal of information systems plumb & kautz 2015, vol. 19, selected papers from acis innovation determinants and barriers the transcriptions of the 24 interviews were coded and analysed utilising the concepts from both the analytical framework and the data, in a constant comparative method (lincoln and guba 1985). first the transcripts were coded according to the elements of the individualist and structuralist perspectives, including both innovation determinants and barriers. in regards to barriers, we checked for synonyms such as ‘constraints’, ‘problems’ and ‘issues’, to allow for newly identified barriers to emerge. second, we coded the transcriptions according to the elements of the interactive process perspective, enfolding the individualist and structuralist elements within a processural analysis of the data. the 12 short videos were first viewed by the first author who made notes on the vision, and these notes were subsequently coded and analysed in a similar manner to the interview transcripts. in the following we use pseudonyms for our interview participants when quoting original data. 5 findings 5.1 practices prior to ipadkinderloop before we examine the innovation process of appropriating ipadkinderloop and the resultant changes, it is useful to understand the practices of communicating with families, documentation processes and usage pattern of ipads within the bfs centres prior to ipadkinderloop. 5.1.1 prior methods of communicating centre director rochelle outlines how centres communicated with families of children attending their centre prior to the introduction of ipadkinderloop: “in the old days, we used to put stuff in parent pockets, and parents would never check pockets. we put notes up on the door, parents wouldn’t read them, and we were really frustrated that the communication wasn’t getting through”. 5.1.2 prior methods of documenting children’s learning and development documentation of children’s development is a critical aspect of the role of an early childhood educator, and the use of paper-based documentation occurs extensively within the early childhood sector (piper et al. 2013). within the curriculum for australian early childhood education and care providers the process of documentation is noted as part of the assessment for learning and intentional teaching aspects of the role of an early childhood educator (agdeewr 2009). educators are responsible for making formal observations of children’s activities and documenting the learning that is occurring. these observations were traditionally conducted an in ad hoc manner, with educators required to carry around a notepad and pen throughout the day to document their observations. during scheduled ‘release from face-to-face duties’ time, educators would type up their hand-written notes into a word-processed document and perform analysis of the documented learning, using it to program future planned educational experiences for the children. there were two key documents produced within the centres by educators for the benefit of parents: the day book, also known as a diary or reflection book, and child portfolios. the day book was observed as a physical book which was placed at the entry to the centre and provided parents with the opportunity to see an overview of what their child and their peers had experienced during the day. it was comprised of printed photos and annotations (either handwritten or typed) that illustrated and described activities that the children had participated in during the day. child portfolios were comprehensive hard-copy documents provided to parents at the end of the year which included photos, annotations and examples of their children’s art or other artefacts demonstrating the developmental and learning progress of the child. portfolios were historically costly, hand-written documents with commercially-developed photos glued on the paper where required, but with more centres providing pcs for educators, the presentation of the portfolios changed to word-processed printed documents which included printouts of photos taken with digital cameras. 7 australasian journal of information systems plumb & kautz 2015, vol. 19, selected papers from acis innovation determinants and barriers 5.1.3 prior usage of ipads centres had begun to appropriate ipads before the arrival and establishment of ipadkinderloop. they were used both by individual children and in groups to play games, watch videos, and to look up items of interest by the children on the internet. educators also spoke of the usefulness of the ipad as a tool to help settle children who were experiencing separation anxiety when their parent dropped them off at the centre, and as a particularly useful tool for children with special needs. 5.2 the individualist perspective 5.2.1 educator attitude towards ipadkinderloop and confidence the majority of educators and centre directors spoke positively about ipadkinderloop, describing it as “exciting” (cindy), “amazing” (rochelle) and “something i am interested in” (sharon). as rochelle noted, “everyone here was pretty keen to do it, everyone was pretty motivated”. chris spoke of how using ipadkinderloop helps his teaching practices, particularly in saving time: “it means less time off the floor mucking around with paper and typing it on computers, because i can do it all on the go and then because of that it means i get to spend more time with the children, and ideally that’s what i want, and that’s what the families want as well”. the enthusiasm was however not across the board, with one educator in particular speaking of the constraint of being overwhelmed and lacking confidence in using it, calling it “a very big learning curve”, but acknowledging that she was building confidence. centre director emma stated that “not all educators are doing it [using the tagging functionality on kinderloop], it’s a confidence thing”. 5.2.2 it champions rochelle and judy, both centre directors, exhibited traits of being it champions; rochelle explained how four years ago, she and judy had the idea of starting a blog for her centre, in order to get families “more involved in what they were actually doing at the service”, and because traditional forms of communication with the parents such as those mentioned above were not entirely successful. when describing how centres were chosen to be pilots for the kinderloop app, judy recounted suggesting rochelle and her centre as a pilot site, describing rochelle as “very innovative” and being “totally open to it”. judy described how she and rochelle had been looking for innovative ways to communicate with families “for years and years…and then we found kinderloop!” rochelle and judy are also considered ‘superloopers’ by the kinderloop founder, promoted as ‘key ambassadors’ for the app. as part of this role they were responsible for visiting other bfs centres and providing advice to directors on how to begin appropriating ipadkinderloop. 5.2.3 leaders the direct influence of the ceo as a leader on the appropriation of ipadkinderloop was evident. after he had been introduced to the kinderloop founder at an industry conference in march 2012, he recounted how in his next meeting with the kinderloop founder “in an hour he sold me kinderloop hook, line and sinker” and that he “made the decision that we would roll out kinderloop to all of our centres because we saw great value in it”. he viewed kinderloop as “new and innovative” and he wanted it to be a part of the value-add experience that his organisation provides in their early childhood services. the ceo was not only directly influential in the ipadkinderloop appropriation, but also indirectly; he was described by judy as “passionate about the industry”. she described him as really supportive of innovative activities, and that he was “passionate about it [kinderloop] and driving it, because he’s all about families and communities”. 5.2.4 previous it exposure and skill set as noted earlier, the eight bfs centres examined had previously appropriated ipads, and were already familiar with the device which forms the platform for ipadkinderloop. the majority of 8 australasian journal of information systems plumb & kautz 2015, vol. 19, selected papers from acis innovation determinants and barriers educators spoke of using the devices in their personal lives, which meant that they were already familiar with them and had acquired the skills to use these devices, although centre director sharon noted that she was “a bit unsure of the ipad because i don’t have an ipad, i have a tablet [at home]”. the existing skill set of educators was determined primarily through prior experiences and not through training in regards to their qualifications, as none of the educators reported acquiring specific it skills during the course of obtaining their early childhood educator qualification. thus the potential barrier of ‘lack of skills’ was avoided through the prior experiences of educators with it in their working and personal lives, despite the lack of it skill development during their studies to obtain a qualification. 5.3 the structuralist perspective 5.3.1 size, complexity, centralisation, and formalisation as mentioned in the case setting bfs is responsible for 26 early childhood centres within the region, providing education and care services for children from birth to five years old. bfs is governed by a board of directors who are responsible for determining policy, strategic direction and operation of the organisation. it has over 500 staff with 268 employed within the 26 early childhood centres. it has a flat organisational structure with very few hierarchy levels: each centre has a director who reports to a group of area managers. the area managers have responsibilities including staffing and budgetary performance and developmental responsibilities such as staff and centre development and the development of effective family and community relationships. they report to the general manager, people and operations, who in turn reports to the organisation’s ceo. bfs utilises a combination of centralised and decentralised decision-making when it comes to it. the decision to appropriate ipadkinderloop was made by the ceo, and after approval by the board of directors and a meeting with centre directors, all centres began the appropriation of ipadkinderloop in august 2012. the appropriation was made mandatory, but the ceo explained: “i didn't compel a hard and fast deadline. the primary motivator for local action was periodic contact from the area managers and head of marketing on progress, as in my experience the best motivator for action is the compulsion to report back”. it was up to centre directors to decide how and when they would begin appropriating ipadkinderloop. centres had a degree of autonomy in deciding to purchase it, although as the ceo explained it is a “standard inclusion” for any new bfs centres. the ceo also stated that the centres acquired their it in different ways: “some centres purchased them [it devices] with the assistance of their parents and citizens groups; others put them on their capital request bids, and big fat smile hq has arranged it for them. and the third source is the early start initiative1 at the university”. these different channels of funding available to bfs centres ensured that there were no funding-related barriers to the ipadkinderloop appropriation process. for example, when centre director judy was preparing to pilot the kinderloop app at her centre, she recounted how she wanted to keep the existing ipads for use with the children, and buy some more for use with kinderloop: “i just went and got them [the ipads] out of my budget, and i just rang someone at head office and just said ‘look i’m just buying two more ipads, i’ve got four rooms in the centre i need two more ipads, i’m going to do this kinderloop thing, i’m just buying them’”. this is supported through the statement by the ceo that the organisation “will work on the understanding that it [the purchase of it] is a carefully considered decision locally and we place trust in the director that that is an appropriate purchase for that centre”. no formalised procedures had to be followed in regards to the appropriation of ipadkinderloop, nor was there a specific implementation plan; however this did not appear to cause problems in the appropriation process. rochelle described how in conjunction with her staff, a set of guidelines or ‘recommendations’ were developed, although as she recalled “we [also] had discussions with the area managers and head office and they came up with a few 1 the early start initiative is a project aiming to providing opportunities for local, national and international collaboration with the goal of enhancing social capital and addressing disadvantage in vulnerable communities (https://earlystart.uow.edu.au/overview/index.html). 9 australasian journal of information systems plumb & kautz 2015, vol. 19, selected papers from acis innovation determinants and barriers guidelines as well”. these guidelines were practical in nature and intended to be used to guide how educators utilised ipadkinderloop, such as a “three sentence maximum for the individual posts”, “no personal posts”, and “processes in place so that we’re checking each other’s posts”. 5.3.2 parents as stakeholders participants spoke of the importance of communication between centres and parents, as the ceo stated that “when people are paying for the services we provide you find ways in which the connections can be stronger...there’s an onus on us, as a provider, to ensure that the parents have as much information as possible, so they can feel good about their purchase decision!”. the ability of ipadkinderloop to provide a way to communicate directly with parents was considered “really important” and the ceo described its role as a communications tool which “helps us overcome the pressures and tensions of those short contact points [between parents and educators] each day”. when speaking about the trial of ipadkinderloop at her centre, rochelle commented: “we started off with just a small focus group of families, so probably about 20 families, and we chose families that were tech savvy, that probably wouldn’t mind if we made mistakes as well…and then we started adding more and more people on. and now all the families are on, and yeah they love it”. the support of parents and citizens groups at centres was also influential in obtaining the ipads prior to the establishment of ipadkinderloop, as centre director sharon recounted: “they [the parent committee] had a substantial amount of money sitting in their kitty. so we just said to them that we would like to purchase them [ipads] to use with the kids to broaden their capacity with technology. so we talked about that and they were really easy going. they were like ‘oh, yeah, if you think that's a really important thing then we'll get them’”. from a negative perspective, centre directors at bfs centres situated in low-socioeconomic areas reported that parents with low levels of access to it constrained the ipadkinderloop appropriation, as although it did not stop the centres from appropriating ipadkinderloop, it prevented parents from being able to access the centre’s kinderloop both at sign-up time and in an ongoing manner. 5.3.3 government compliance and regulatory requirements the national quality framework (nqf) was established in 2012 and applies to most preschool/kindergarten and outside schools hours care services in australia (acecqa 2014), including the bfs centres. the national quality standard (nqs), a key aspect of the nqf, sets a national benchmark against which every early childhood education and care centre in australia is assessed. when a centre is assessed against the nqs, they receive a rating for seven areas (educational program and practice; children’s health and safety; physical environment; staffing arrangements; relationships with children; collaborative partnerships with families and communities; and leadership and service management) and an overall rating, which is then published for public viewing on the australian government’s ‘mychild’ website (acecqa 2014b). the nqs is also linked to the national early years learning framework (eylf) which describes the principles, practices and outcomes that support and enhance young children’s learning from birth to five years, as well as their transition to school (agdeewr 2009). the ceo confirmed that these government compliance and regulatory requirements were a significant consideration in the ipadkinderloop appropriation, as kinderloop provided “the ability for our educators in our centres to far more readily, and cost-effectively i might add…deliver on their obligations, the documentation and reporting, and relationships with families is one major part of one of the seven assessment criteria against which we are all being rated”. the success of this was demonstrated by two bfs centres involved in this study achieving perfect scores on all 58 elements of the rating for the national quality standard where kinderloop was, according to the bfs annual report, “a huge plus for the preschools that went through the national assessment and rating process, with assessors impressed by the ease with which records and learning outcomes were documented and recalled” (bfs 2014 p.22). under the nqf ecec organisations are subject to national law and regulations and are required to adhere to specific educator-to-child ratios according to the age of the children. this ratio impacts on what extra activities the educators can complete whilst supervising children, including using ipadkinderloop. as anita explained: “we only have one staff [educator] doing it at a time, ‘cause who would watch the children otherwise”. 10 australasian journal of information systems plumb & kautz 2015, vol. 19, selected papers from acis innovation determinants and barriers 5.3.4 existing infrastructure and resources the four bfs centres in our study had previously appropriated ipads, therefore the establishment of ipadkinderloop simply required the installation of the kinderloop app onto the devices. as rochelle recalls: “we had the ipads, we were pretty much ready to go, we just needed wi-fi installed and we had to download one app”. rochelle’s quote highlights the other infrastructure required for ipadkinderloop: a wi-fi connection, as the kinderloop app requires access to the internet. some centres had wi-fi in place; others had to have it installed. however numerous educators spoke of becoming frustrated with the issues that unreliable wifi access presented during the appropriation process. additionally, several bfs centres spoke of having to deal with no internet access at times, which presented as a significant barrier and resulted in a cessation of practices such as communicating with families in real-time via kinderloop. participants at two bfs centres described how their appropriation of ipadkinderloop was constrained by an inadequate number of ipads in their centres. as educator cindy describes, at her centre: “if the children are using the ipad we can’t document anything, unless you can get kinderloop on our phones…but taking photos on your phone and stuff like that, it’s a bit of a privacy issue…so we don’t like to do that”. this resulted in interruptions to the performance of their communication and documentation practices at times when they would like to use kinderloop but the children are using the ipads. similarly in centre director felicity’s centre, they did not have “enough ipads to allow two groups to document at the same time”. this resulted in an accommodation where a roster was created to ensure that each group had access to the ipad during the day, in addition to looking into purchasing another ipad for staff to use kinderloop on. 5.3.5 competitors the early childhood education and care industry is competitive, as judy states: “we’ve got 17 services that i’m in direct competition in and ... just two, that are community-based not for profit. and i’ve had a little bit of a look around at some of them, and they are run by people that just want to make money”. bfs is a not-for-profit organisation who according to the ceo focuses on a “very deliberate differentiation on the high quality side of things...all of those little value-adds, whether they’re cultural, sporting, convenience, however they might be perceived, are tied up in what we’re presenting as our brand value proposition…[and] kinderloop is yet one-more value-added”. 5.3.6 support and training although previous studies such as blackwell et al. (2013), fenty and mckendry anderson (2014), plowman and stephen (2005) and wood et al. (2008) have identified a lack of support and training as structuralist barriers to the integration of it, they were not found to be present in this case study. in regards to support, the ceo stated that that “if there is an enthusiasm for the technology or delivering something in a program that is enhanced by the technology, we will step in and support that”. in regards to training, educators spoke of not requiring training for ipadkinderloop due to their familiarity with the ipad, and concerning the kinderloop app itself, educator jackie suggests that “it’s [kinderloop] very easy to use, so we didn’t have any professional development for it…and i think if you’re familiar with facebook then yeah, it’s pretty simple”. 5.4 the interactive process perspective the motivation behind the development of the kinderloop app was an inherently social one: as a parent, the founder felt that communication between parents and educators at the centre his child attended needed improving; parents are always rushed when picking up their children and they do not have time to stop and talk to the educators about how their child was through the day; and they may feel guilty or anxious about leaving the child at a centre while going to work, wondering if they are okay. 11 australasian journal of information systems plumb & kautz 2015, vol. 19, selected papers from acis innovation determinants and barriers the bfs ceo was first exposed to kinderloop at an industry conference. the kinderloop founder had been invited to attend this conference by a bfs board of directors member who was a head teacher of child studies at the local tafe (a vocational tertiary education provider) and had become aware of kinderloop through her work. the ceo “saw great value” in kinderloop and evaluated it against its expected affordances and against the existing practices and norms of the bfs early childhood centres. from a social perspective, the ceo reviewed the affordances of kinderloop within the context of a number of social and cultural contextual concepts, including parental guilt and anxiety over leaving children at centres and not knowing what they were doing through the day; timepoor parents; and the “need to provide as much information as possible to parents” and the “importance of strengthening family-centre communications”. as part of this initial evaluation by the ceo, a barrier was identified as his perception of parental attitudes towards the features of the kinderloop app; however, as he recounts, this barrier was overcome by educating parents about the current practices of how the bfs educators document children’s learning and development: “so i’ve got to say, upfront people go ‘hold on, taking photos of children? that can only lead to negative horrible things’, and when you sit down with people, explain to them the closed network, and the fact that taking photos happens routinely as part of mapping their development, we’re just finding another convenient way to get you the information as we go”. the decision by the ceo to introduce kinderloop to the organisation was also shaped by the fact that bfs exists within in a competitive market of early childhood education and care service providers and is striving to differentiate themselves by providing high quality services with added values, of which he considered kinderloop to be one such added value. a decision was made to trial ipadkinderloop in two centres managed by directors with technology champion traits and who had previously considered different ways to better communicate with families by digital means. one of these directors, judy, recounted how she was initially cautious about ipadkinderloop, which could be considered a potential barrier to the ipadkinderloop appropriation as the director was reluctant to take on anything that took away time from establishing relationships with families and settling children at her newly-built centre. however once she had begun to appropriate it, she describes how her attitude had changed: “when i actually had a look at it, i was like ‘oh my god what am i doing, this is going to help me with my families, and relationships!’”. once the pilot at the two centres had been deemed successful, the decision was made at bfs head office to make kinderloop mandatory across all centres. however no timeframe was given, only that centre directors needed to report their progress to their area manager every two months, which the ceo viewed as a “stronger incentive for centres to roll out ipadkinderloop than enforcing a deadline”. during the introduction stage, despite some accounts of lack of confidence by educators, in general confidence was not identified as a major issue in the ipadkinderloop appropriation process, as the bfs ceo commented: “it came to my notice that every employee at [a particular centre] was using it [ipadkinderloop], including some people who were known to be less than enthusiastic, a bit frightened of technology, having a go, getting on board, and realising it wasn’t this big frightening thing, it’s quite simple to use”. as there were no formalised procedures, centre directors worked collaboratively with their staff to develop guidelines for its use. centre director rochelle described how these resultant guidelines were practical in nature and intended to be used to guide how educators utilised ipadkinderloop, such as a “three sentence maximum for the individual posts”, “no personal posts”, and “processes in place so that we’re checking each other’s posts” to ensure a certain level of quality. educator chris described an informally negotiated norm between himself and the other educator who teaches in his room, where they mutually negotiated to make “about 30 posts a day, we try our best to cover each child at least once”. we found evidence that the content of the innovation of ipadkinderloop differed in a number of centres. at two centres in particular it was evident that the way ipadkinderloop was being 12 australasian journal of information systems plumb & kautz 2015, vol. 19, selected papers from acis innovation determinants and barriers used was directly influenced by the understandings that the directors had of its affordances and their evaluation of it as a tool amongst the existing practices. centre director rochelle had evaluated the affordances and determined its suitability as a communication tool for her centre, but with a distinct focus on documenting learning that is happening, which is then useful for educators to ‘cut-and-paste’ when programming2 to save time: “we use it mainly as a communication tool, but we also try to show the learning that’s actually happening as well…when we’re programming, take bits and pieces off kinderloop as well that we’ve seen, like little observations and we use it as part of the children’s individual plans”. in contrast, centre director judy had developed strong views on not using it as a developmental documentation tool but more as a simple event-recording tool: “we’re not using it as a massive developmental tool for analysis of the learning that’s occurring, because i don’t think i’d like it to be used that way...i think it’s far more beneficial as a communication tool for families”. centre directors and educators spoke of how ipadkinderloop had transformed their work practices. in regards to the practice of documenting children’s learning and development, in some cases the day book and portfolios were discontinued and replaced by ipadkinderloop. as educator chris recounted: “in terms of programming, we don’t have to do daily reflections anymore, which is good because kinderloop puts out all the pictures we do, it lets people know what we’re doing throughout the day”. there was also evidence that the practice of communicating information to parents had changed substantially, not only in how the information was transmitted, but also in the responses from parents which indicated increased engagement, as rochelle recounts: “we’ve put a lot of things [on kinderloop], like last year we did like a pet interest session, and normally even if i were to email parents, we might get one or two photos of kids’ pets…last year we put photos on a pet board, we talked about the pets, people brought pets in, and we had so much more engagement from families”. there was also evidence of a content change of ipadkinderloop as a result of the structuralist educator-to-child ratio constraint during the appropriation process, where anita explained that this ratio was “one of the reasons why we’ve made them [the posts] casual now, so that we can just do them in a couple of minutes, because they were getting a bit detailed and it would just take too long, and that’s just taking us away from the children”. centre director felicity spoke about structuring group time activities to accommodate the ratio, with one educator leading the group and the other supporting person could be “doing your quick blurb…but at those times you would be making sure it was very quick. it was just your pictures for your parents, and things like that. the more detailed information would have to be when you're off the floor in your documentation time or things like that”. the kinderloop app as part of the innovation content of ipadkinderloop was noted as evolving and continued to evolve throughout the appropriation process; rochelle recounted the very first time they met with the kinderloop founder: “it wasn’t even a proper app when they were showing us, it was just like a pdf kind of thing to actually show us how it all worked”. then when the app was first piloted, the directors of the pilot centres worked with the kinderloop founder to adjust software features to suit them, as rochelle describes: “it’s been a really interesting process for us to go through because it was a very basic app to begin with, like there was no tagging, you could only put one photo in, and then, working with the [kinderloop] guys they were like ‘oh so you want to put more than one photo in?’ and [we said] ‘well yeah…we want to show the progression of what a child’s doing’, so then they added more photos…so there were steps that we went through with them, to help develop it”. rochelle also mentions that they continue to work closely with the kinderloop founder to suggest more features to be added to the software, such as exporting details and including video footage in posts. the structuralist barrier of it technical issues was found to be present numerous times and influenced the presence of the individualist barrier of a negative attitude, with the result that both barriers constrained the process of the ipadkinderloop appropriation. numerous 2 programming here refers to the educators’ activity of documenting an experience and activity sequence before and after observing the children within the early childhood centre. 13 australasian journal of information systems plumb & kautz 2015, vol. 19, selected papers from acis innovation determinants and barriers educators at one particular centre spoke of being frustrated with the issues that unreliable wifi access presented during the appropriation process, with simone describing the wi-fi problem at her centre as a “massive challenge” and that it discouraged her from using ipadkinderloop, stating it “just all became too hard”. however it did not prevent the appropriation process from continuing; rather the educators simone and cindy spoke of making accommodations in their documentation and communication practices, taking photos using the ipads when required, but then uploading the actual photos and posts to the centre’s kinderloop at a later stage when they could physically stand in a place with a strong enough wi-fi signal to obtain internet connectivity. other accommodations were identified in regards to centres experiencing it technical issues, as centre director felicity described how her centre adapted to such a problem: “we had no internet. we had no wi-fi. we had nothing. and so, that was a big, ‘oh, what are we going to do?’ so, we did take a lot of photos, just back on the camera or on the ipad, and downloaded them onto the computer. we couldn't post [onto kinderloop], but we just did a bit of a camera roll [on the interactive whiteboard] for the parents during that week. and i think because we do still have a clipboard, with just a weekly paper review…we were still able to write down what was happening during the program. we had the photos to back us up. but once all our problems were solved, we backdated our [kinderloop] posts”. 5.4.1 the impact of parents as key stakeholders on the ipadkinderloop appropriation process parents as key stakeholders were found to be influential at different stages of the process, both directly and indirectly. initially during the pilot, a small group of families were specifically chosen to be a part of the activities. then after the pilot had been completed, other families became involved. with the initial introduction of ipadkinderloop into centres, there was evidence that some educators held particular perceptions of parental concerns in regards to their use of ipadkinderloop, believing that parents may have an adverse reaction to seeing the educators using ipadkinderloop. however, as centre director felicity explained, this barrier did not end up impacting the appropriation process: “if parents walk in and see staff member having an ipad and typing, i express to my staff, ‘it's no different to parents walking in with you and a clipboard and your head's down and you're writing. are my staff gonna spend too much time on an ipad trying to do kinderloop and stuff like that? and that was my argument and i said, ‘it's no different to them having their notebooks or their clipboards with their obs[ervations] on it and writing stuff down then’. to me, it's more in the moment”. the bfs centres in areas classified as ‘low-socioeconomic’ presented particular barriers to the process of appropriating ipadkinderloop through the low-levels of parent access to it at different stages of the ipadkinderloop appropriation process. at the beginning of the appropriation process one centre encountered issues where parents did not have email addresses, which are required for parents to sign-up for and access the centre’s kinderloop. this was described by centre director anna as “a little bit of a stumbling block”. as a result, the centre educators were required to maintain their existing paper forms of communication for the foreseeable future, although the centre director described how she and her staff would be “happy to sign them [the parents] up to a gmail address” in order to provide them with access to kinderloop. a similar finding was revealed at another centre in a low-socioeconomic area, where centre director emma stated that “a third of children attending” came from families without access to it. although other centres had planned (and in some cases had already begun) to move to completely online communication with families by utilising kinderloop, at emma’s centre although the process of the appropriation of ipadkinderloop continued it was constrained in an ongoing manner, with the centre director and educators having to complete two forms of communication (both the traditional paper communications as well as electronically through kinderloop) which increased the educators’ workloads. bfs centres where children in attendance are in foster care or other government-organised care arrangements experienced barriers to the educators’ appropriation of ipadkinderloop and resulted in accommodations as the educators attempted to mitigate these barriers. due to the nature of the living arrangements of these children and the requirement and/or preference to 14 australasian journal of information systems plumb & kautz 2015, vol. 19, selected papers from acis innovation determinants and barriers keep a certain level of confidentiality in regards to the child’s identity, their care-givers did not permit the child’s participation in kinderloop. additionally the often fluid nature of the living arrangements of such children meant that these associated barriers could occur at any time during the appropriation process, as children begin to attend and then leave the centre. as centre director emma recounts: “we have a lot of children here who are in the care of the department of family and community services or a foster care situation, short term/long term care…so you will find that there's a lot of people that go, okay, no, i don't want kinderloop…they need to keep that child's identity confidential.” this resulted in educators at emma’s centre being constrained in their use of kinderloop, where care had to be taken with taking photos of children and uploading them to the centre’s kinderloop to ensure that children who were not to be included in kinderloop posts were not present in individual or group photos. privacy as a potential barrier to the ipadkinderloop appropriation also manifested itself as a combination of both individual educator and structural environmental elements at another centre where a child’s parents had requested particular privacy conditions where the child was not to be in group photos which were shared to other parents on kinderloop. centre director felicity described how they dealt with this potential barrier: “when we were doing photos, staff were not to include that child. when we posted to that family, it was just their child. we would still post group times to that family, so they still knew what that child was doing, but he would not be included in the photos…i took that back to a staff meeting, explained it all to the staff...‘he is not to be photographed, 'cause this is the rule,’ so all staff were aware.” this privacy issue arose a number of times throughout the appropriation process in different forms at different centres. at one of the pilot centres where ipadkinderloop had been in use for quite a while, educator simone recounted how the educators had set up tags relating to the two rooms (pre-school and early learning) that housed children according to their age. this meant that when a photo was tagged with a particular room tag, it would go to a particular set of parents who had children in that room. a situation occurred where an educator had inadvertently used the wrong tags when posting photos of children to the centre’s kinderloop which resulted in those children’s photos being viewable by unintended parents and a “parent rang up and made a complaint and wasn’t happy about it”. another privacy issue occurred after the use of ipadkinderloop had been well-established in regards to a lack of cultural understanding at a centre that had a high proportion of families from countries other than australia. centre director felicity described a situation where an educator uploaded a post and photos of a family with a child for their birthday at the centre. however she noted “the dad came in and he said, ‘can you please take it down? my wife's not allowed to be photographed. arabic women aren't allowed to have their photos done publicly’”. many educators and centre directors spoke of positive feedback from families; as rochelle stated, “they love it”. in addition to this positive feedback, educator anita recalls how parental feedback on the posts that educators were making to kinderloop informed changes to the content of the photo annotations: “it used to be a formal observation of what the child was doing and how it links to the eylf; we still do link the outcomes to the photos, but we’ll just put ‘lo 4.1’ so that it means nothing to the parents, they can still see that but it’s just for our use. so what we used to do is we would write something like ‘bella is using her right hand to draw a picture and from this we can see we she’s got good fine motor skills’, using that technical language whereas now we’d write ‘bella is having a great time drawing a picture for mum’, it’s really casual and more informal”. this comment also illustrates how the government regulatory requirements influenced the way the educators were utilising ipadkinderloop, as the educators use their kinderloop posts to demonstrate how they are meeting the required outcomes from the prescribed curriculum. 6 discussion according to the individualist perspective, individuals have traits or characteristics which predispose them to innovative behaviour, and innovation is caused by individual actions. although not all of the previously identified individualist barriers to it appropriation from the 15 australasian journal of information systems plumb & kautz 2015, vol. 19, selected papers from acis innovation determinants and barriers literature were present, we did observe the presence of ‘negative educator attitudes’ and ‘lack of confidence’ as barriers. however we found that they did not halt the ipadkinderloop appropriation process, but rather constrained the extent to which ipadkinderloop was being appropriated in some centres. the presence of two centre directors with personal characteristics such as a positive attitude towards it in early childhood and the personal initiative to consider digital forms of communication with parents such as blogs long before the arrival of ipadkinderloop positively influenced the successful ipadkinderloop appropriation, through their support of the pilot and also their role as key ambassadors for ipadkinderloop. this supports the studies on innovation that have established that innovative activity is promoted by technological champions (howell and higgins 1990). the ceo’s role in the initiation of the appropriation process within the bfs organisation cannot be underestimated; his initial exposure to the app via the founder at the industry conference which had been instigated by a board member, and his subsequent acceptance and drive for the appropriation has clearly influenced the appropriation process. through the ceo’s passion for delivering ‘value-add’ to the services his organisation provides, and his enthusiasm for the families and their involvement in his centres (as testified by his staff), the benefits that ipadkinderloop affords have been realised. we concur with daft (1978) who notes that “leaders are active in the innovation process” (p.193). however as saren (1987 cited in slappendel 1996) argues, the actions of innovative individuals cannot be divorced from either the activities of other individuals or from the organisation structure within which they operate. therefore although applying the individualist perspective provides useful insights, it is limited in only providing partial explanations. the structuralist perspective contributes to our understanding of the ipadkinderloop appropriation by examining organisational characteristics and elements from the organisation’s environment. the previously reported structuralist barrier of ‘it technical problems’ was observed to be a constraint to the ipadkinderloop appropriation rather than a strict barrier, as educators mitigated the impact of by making accommodations to their workplace practices. other structuralist barriers such as ‘lack of training’ and ‘lack of funding’ were not identified in our study. the relatively flat hierarchy of the bfs organisation seems to have contributed to the smooth ipadkinderloop appropriation, despite the absence of a formalisation implementation plan for the roll-out. this finding is in contrast to other studies on innovations in educational institutions who found that a “large, complex organization with a heterogeneous environment is more likely to adopt innovations than a small, simple organization with a relatively stable, homogeneous environment” (baldridge and burnham 1975 p.175). the fact that a strict deadline was not imposed on centres for the appropriation, coupled with the lack of guidelines meant that although the appropriation was mandated in a ‘top-down’ fashion, centres had some degree of autonomy in deciding when and how the appropriation would unfold. the environmental elements of parents as stakeholders, government compliance and regulatory requirements, existing infrastructure and competitors were certainly influential in the appropriation process. as leaner and phillips (1994 p.43) state, “few would disagree that parents are a key childcare stakeholder group” and the desire of the organisation as recounted by the ceo and directors and educators to strengthen parentcentre communications was a significant driver in the ipadkinderloop appropriation. we have revealed the existence of previously unidentified barriers related to parents as stakeholders, including barriers related to low-levels of parent it access, children in specialised care arrangements, and privacy issues, reaffirming existing findings of the importance of parents at this level of education (bidwell 1965 and sieber 1968 cited in baldridge and burnham 1975; burden et al. 2012 cited in clark and luckin 2013). however as with other barriers identified in this study, we observed that their impact was mitigated by the educators’ accommodations in workplace practices. along with ipadkinderloop’s ability to transform the practices of centre communication with parents, the recent government compliance and regulatory requirements of the nqf, nqs and eylf were influential in how ipadkinderloop was appropriated, as its features allowed educators to replace traditional forms of documenting children’s learning and development and facilitated, in the words of the ceo, “our educators in our centres to far more readily, and cost-effectively, deliver on their obligations, the documentation and reporting”. 16 australasian journal of information systems plumb & kautz 2015, vol. 19, selected papers from acis innovation determinants and barriers however, as with the individualist perspective, the structuralist perspective only provides partial explanations, and so we look to interactive process perspective to take into account the relationship between the individualist and structuralist elements, and by focusing on the complex interplay between these various factors, permits a more detailed analysis. the interactive process perspective allows us to understand how the interplay of the individual and structural elements influenced the innovation process over time. the appropriation of ipadkinderloop can be traced to the ‘shock’ of the kinderloop founder being introduced to the bfs ceo at the industry conference. as schroeder et al. (1986) state, a shock does not need to be viewed as a negative, but rather as something that stimulates efforts by people to begin work on an innovation, which is what happened in our case study, as the ceo took an interest in kinderloop and began to evaluate its potential within the context of the bfs ecec organisation. the outer context concepts of the social and competitive environment were particularly influential at all stages of the process; at the beginning, the evaluated affordances of kinderloop to strengthen communications between centres and parents and to deliver value-add to the services provided by bfs were evident. the social context of the importance of communicating with parents, and the nature of parents as stakeholders was particularly evident throughout different stages of the appropriation process as both positive and negative influences: from the beginning when parents were involved in the pilot and where the structuralist barrier of ‘low levels of access to it’ caused difficulties for the introduction of kinderloop in a centre where parents did not have email addresses with which to sign up to kinderloop; through to ipadkinderloop in fluent every-day use and the positive feedback provided to educators by parents, in addition to the continuing constraint of both the ‘low levels of access to it’ and ‘privacy issues’ at a particular bfs centre which caused the educators to have to complete ‘double lots’ of work in regards to both paper and electronic communications. in a social context, from the perspective of the centres, ipadkinderloop had transformed the way educators communicated with families, providing a deeper level of engagement with them. from the perspective of parents, their appropriation of the app on their devices enhanced a series of processes that are notably social in nature, for example allowing them to be reassured that their child is doing well while they are in attendance at the centre, or promoting engagement with their children at home, as parent megan described “[kinderloop is] a really great conversation starter in the evening, because most young children can’t remember what they did”. the content of the innovation was shown to evolve in different ways in different centres, as a result of the individual perspectives of the centre directors in regards to their evaluated affordances of the technology, combined with the structural influence that although ipadkinderloop had been mandated by the ceo, centre directors had a degree of autonomy in the decision-making that shaped how ipadkinderloop was appropriated in their own particular centres. the content of ipadkinderloop continued to change throughout the appropriation process: through influences such as parent feedback and the mitigation of the structuralist barrier related to the educator-to-child ratio regulatory requirement, both of which resulted in changes to the annotations of posts; and educator feedback to the developers, such as the introduction of new functionality of tagging in posts and uploading of videos. the work practices of educators, in particular communication with parents, and documentation of children’s learning and development, were transformed by ipadkinderloop appropriation. as the work practices of the educators are considered part of the content of the innovation of ipadkinderloop, the barriers and constraints which arose at different stages of the appropriation process resulted in accommodations and adjustments by educators to their work practices in an ongoing manner, thus modifying the content of the innovation over time. for example, the structuralist-categorised barrier of ‘it technical problems’ (such as centres who experienced wi-fi problems) appeared at numerous times during the appropriation process, and educators did their best to mitigate the problem when it appeared by implementing strategies such as moving to a place in the centre with stronger wi-fi, or delaying the creation of kinderloop posts. this finding concurs with that by becker (1993 in ertmer 1999) who stated that barriers “may never be eliminated completely but rather that they will continue to ebb and flow throughout the evolutionary integration process” (p.52). 17 australasian journal of information systems plumb & kautz 2015, vol. 19, selected papers from acis innovation determinants and barriers although other studies of innovation have found that as an innovation develops, the ‘old’ and the ‘new’ ways of doing things exist concurrently and over time are linked together (schroeder et al. 1986), we observed that this was not always the case with our study; for example the ‘old’ way of communicating the day’s activities through the production of a day book, and the ‘old’ way of documenting a child’s learning and development in a portfolio were both discontinued and replaced by the artefacts produced by ipadkinderloop at a number of the centres in our study. 7 conclusion this study addresses shortcomings in the limited body of literature on innovation and it appropriation in ecec organisations. we contribute to the is literature a detailed, triperspective account of organisational innovation and the process of it appropriation, and demonstrate that neither an individualist nor a structuralist perspective alone provides a deep understanding of the process. we confirm that “organisation change and its implementation is viewed as a complex, messy process inseparable from its intra-organisational and broader contexts” (walsham 1993 p.53) and we have demonstrated that the process of it appropriation occurs through a complex interaction between individual action and structural influences and thus is better understood through a tri-perspective framework which examines both innovation determinants and barriers. as we have provided a rich case study of an it appropriation, we contribute to is practice by exposing the multi-faceted influences on it appropriation which provides a basis for managers to plan and prepare for it appropriation. we acknowledge that there are other implications to the appropriation of such technology including workplace privacy, employee performance monitoring, the inadvertent recording of child misbehaviour and/or injury, and digital inclusion issues, which have not been addressed in this study. thus further research is required to derive more detailed information to guide managers in facilitating the appropriation of it. references al-qirim, n. 2011. "determinants of interactive white board success in teaching in higher education institutions", computers & education (56:3), pp. 827-838. al-qirim, n. 2012. "adoption vs. usage of interactive white board technology by teachers in higher education institutions", acis 2012, geelong, victoria, australia. alaranta, m., and kautz, k. 2012. "a framework for understanding post-merger information systems integration", journal of information technology theory and application (13:1), pp. 5-30. albirini, a. 2006. "teachers’ attitudes toward information and communication technologies: the case of syrian efl teachers", computers & education (47:4), pp. 373-398. australian children’s education and care quality authority. 2014a. "introducing the national quality framework" retrieved 27 january, 2014, from http://www.acecqa.gov.au australian children’s education and care quality authority. 2014b. "the national quality standard" retrieved 27 january 2014, from http://www.acecqa.gov.au australian government department of education employment and workplace relations. 2009. "belonging, being and becoming: the early years learning framework for australia" retrieved 27 january 2014, from http://docs.education.gov.au baldridge, j.v., and burnham, r.a. 1975. "organizational innovation: individual, organizational, and environmental impacts", administrative science quarterly (20:3), pp. 165-176. barron, b., cayton-hodges, g., bofferding, l., copple, c., darling-hammond, l., levine, m. 2011. “take a giant step: a blueprint for teaching children in a digital age”. the joan ganz cooney centre at sesame workshop, new york. 18 australasian journal of information systems plumb & kautz 2015, vol. 19, selected papers from acis innovation determinants and barriers big fat smile. 2014. “big fat smile annual report for 2014” retrieved 02 july 2015, from http://www.bigfatsmile.com.au/about-us bolstad, r. 2004. "the role and potential of ict in early childhood education a review of new zealand and international literature" retrieved 15 january 2014, from http://www.nzcer.org.nz bourbour, m., vigmo, s., and pramling samuelsson, i. 2014. “integration of interactive whiteboard in swedish preschool practices”, early child development and care, pp. 121. blackwell, c. 2013. “teacher practices with mobile technology integrating tablet computers into the early childhood classroom,” journal of education research (7:4), pp. 231-255. blackwell, c.k., lauricella, a.r., and wartella, e. 2014. “factors influencing digital technology use in early childhood education,” computers and education (77:0), pp. 82-90. blackwell, c.k., lauricella, a.r., wartella, e., robb, m., and schomburg, r. 2013. “adoption and use of technology in early education: the interplay of extrinsic barriers and teacher attitudes,” computers and education (69:0), pp. 310-319. british educational communications and technology agency. 2004. “a review of the research literature on barriers to the uptake of ict by teachers”. clark, w., and luckin, r. 2013. "what the research says: ipads in the classroom". london knowledge lab, institute of education, university of london. cooper, r.b., and zmud, r.w. 1990. "information technology implementation research: a technological diffusion approach", management science (36:2), pp. 123-139. corwin, r.g. 1975. "innovation in organizations: the case of schools", sociology of education (48:1), pp. 1-37. crichton, s., pegler, k., and white, d. 2011. "personal devices in public settings: lessons learned from an ipod touch / ipad project", in: 6th international conference on elearning. okanagan kelowna, british columbia, canada. crossan, m.m., and apaydin, m. 2010. "a multi‐dimensional framework of organizational innovation: a systematic review of the literature", journal of management studies (47:6), pp. 1154-1191. daft, r.l. 1978. "a dual-core model of organizational innovation", academy of management journal (21:2), pp. 193-210. edwards, s. 2005. “identifying the factors that influence computer use in the early childhood classroom,” australasian journal of educational technology (21:2), pp. 192-210. eisenhardt, k. 1989. “building theories from case study research”, the academy of management review (14:4), pp. 532-550. frambach, r.t., and schillewaert, n. 2002. "organizational innovation adoption: a multi-level framework of determinants and opportunities for future research", journal of business research (55:2), pp. 163-176. fenty, n.s., and mckendry anderson, e.m. 2014. “examining educators’ knowledge, beliefs, and practices about using technology with young children,” journal of early childhood teacher education (35:2), pp. 114-134. grunberg, j., and summers, m. 1992. "computer innovation in schools: a review of selected research literature", journal of information technology for teacher education (1:2), pp. 255-276. 19 australasian journal of information systems plumb & kautz 2015, vol. 19, selected papers from acis innovation determinants and barriers hameed, m.a., counsell, s., and swift, s. 2012. "a conceptual model for the process of it innovation adoption in organizations", journal of engineering and technology management (29:3), pp. 358-390. hew, k., and brush, t. 2007. “integrating technology into k-12 teaching and learning: current knowledge gaps and recommendations for future research,” educational technology research and development (55:3), pp. 223-252. howell, j.m., and higgins, c.a. 1990. "champions of technological innovation", administrative science quarterly, pp. 317-341. ihmeideh, f.m. 2009. “barriers to the use of technology in jordanian pre-school settings,” technology, pedagogy and education (18:3), pp. 325-341. ihmeideh, f.m. 2010. “the role of computer technology in teaching reading and writing: preschool teachers' beliefs and practices,” journal of research in childhood education (24:1), pp. 60-79. joshi, a., pan, a., murakami, m., and narayanan, s. 2010. “role of computers in educating young children: u.s. and japanese teachers’ perspectives,” computers in the schools (27:1), pp. 5-19. kautz, k., and nielsen, p.a. 2004. "understanding the implementation of software process improvement innovations in software organizations", information systems journal (14:1), pp. 3-22. larner, m., and phillips, d. 1994. "defining and valuing quality as a parent", in valuing quality in early childhood services: new approaches to defining quality, p. moss and a. pence (eds.). thousand oaks, california: sage publications inc. leung, w.m. 2003. “the shift from a traditional to a digital classroom hong kong kindergartens,” childhood education (80:1), pp. 12-17. li, h. 2006. “integrating information and communication technologies into the early childhood curriculum: chinese principals’ views of the challenges and opportunities,” early education and development (17:3), pp. 467-487. lincoln, y.s., and guba, e.g. 1985. naturalistic inquiry. beverly hills, ca: sage publications, inc. lindahl, m.g., and folkesson, a.-m. 2012a. “can we let computers change practice? educators’ interpretations of preschool tradition,” computers in human behavior (28:5), pp. 1728-1737. lindahl, m.g., and folkesson, a.-m. 2012b. “ict in preschool: friend or foe? the significance of norms in a changing practice,” international journal of early years education (20:4), pp. 422-436. liu, x., and pange, j. 2014. “early childhood teachers’ perceived barriers to ict integration in teaching: a survey study in mainland china,” journal of computers in education, pp. 1-15. ljung-djärf, a. 2008. “to play or not to play that is the question: computer use within three swedish preschools,” early education and development (19:2), pp. 330-339. madsen, s., kautz, k., and vidgen, r. 2006. "a framework for understanding how a unique and local is development method emerges in practice", european journal of information systems (15:2), pp. 225-238. mendoza, a., carroll, j., and stern, l. 2010. "software appropriation over time: from adoption to stabilization and beyond", australasian journal of information systems (16:2), pp. 5-23. 20 australasian journal of information systems plumb & kautz 2015, vol. 19, selected papers from acis innovation determinants and barriers murphy, g.d. 2011. "post-pc devices: a summary of early ipad technology adoption in tertiary environments", e-journal of business education & scholarship of teaching (5:1), pp. 18-32. nikleia, e., and despo, k. 2005. “computer based early childhood learning,” the international conference on computer as a tool (eurocon 2005), belgrade: ieee, pp. 1032-1035. nikolopoulou, k., and gialamas, v. 2013. “barriers to the integration of computers in early childhood settings: teachers’ perceptions,” education and information technologies, pp. 1-17. parette, h.p., blum, c., and quesenberry, a.c. 2013. “the role of technology for young children in the 21st century,” in instructional technology in early childhood, h.p. parette and c. blum (eds.). brookes publishing, pp. 1-28. pettigrew, a.m. 1987. "context and action in the transformation of the firm", journal of management studies (24:6), pp. 649-670. pierce, j., l., and delbecq, a.l. 1977. "organization structure, individual attitudes and innovation", the academy of management review (2:1), pp. 27-37. piper, a.m., d'angelo, s., and hollan, j.d. 2013. "going digital: understanding paper and photo documentation practices in early childhood education", in: 16th acm conference on computer-supported cooperative work (cscw). san antonio, united states: pp. 1319-1328. plowman, l., and stephen, c. 2003. “a ‘benign addition’? research on ict and pre-school children,” journal of computer assisted learning (19:2), pp. 149-164. plowman, l., and stephen, c. 2005. “children, play, and computers in pre-school education,” british journal of educational technology (36:2), pp. 145-157. plumb, m., kautz, k., and tootell, h. 2013. "touch screen technology adoption and utilisation by educators in early childhood educational institutions: a review of the literature", in: 24th australasian conference on information systems (acis). melbourne, australia. reid, p. 2014. “categories for barriers to adoption of instructional technologies,” education and information technologies (19:2), pp. 383-407. rogers, e.m., and shoemaker, f.f. 1971. communication of innovations; a cross-cultural approach. new york: free press. schroeder, r., van de ven, a.h., scudder, g.d., and polley, d. 1986. "managing innovation and change processes: findings from the minnesota innovation research program", agribusiness (2:4), pp. 501-523. sharma, r. 2001. "innovation in schools: identifying a framework for initiating, sustaining and managing them", in: annual meeting of the american educational research association. seattle, washington, usa. slappendel, c. 1996. "perspectives on innovation in organizations", organization studies (17:1), pp. 107-129. tsitouridou, m., and vryzas, k. 2004. “the prospect of integrating ict into the education of young children: the views of greek early childhood teachers,” european journal of teacher education (27:1), pp. 29-45. van de ven, a.h., angle, h.l., and poole, m.s. 1989. research on the management of innovation: the minnesota studies. new york: ballinger/harper & row. walsham, g. 1993. interpreting information systems in organizations. chichester, uk: john wiley & sons ltd. 21 australasian journal of information systems plumb & kautz 2015, vol. 19, selected papers from acis innovation determinants and barriers wolfe, r.a. 1994. "organizational innovation: review, critique and suggested research directions", journal of management studies (31:3), pp. 405-431. wood, e., specht, j., willoughby, t., and mueller, j. 2008. “integrating computer technology in early childhood education environments: issues raised by early childhood educators,” alberta journal of educational research (54:2), pp. 210-226. yin, r. k. 2009. case study research: design and methods, thousand oaks, california: sage publications inc. zhao, y., pugh, k., sheldon, s., and byers, j. 2002. "conditions for classroom technology innovations", the teachers college record (104:3), pp. 482-515. an earlier version of this paper was presented at the australasian conference on information systems (acis) 2014 in auckland, new zealand. copyright: © 2015 plumb & kautz. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. 22 http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 theoretical background 2.1 individualist perspective 2.2 structuralist perspective 2.3 interactive process perspective 3 case setting 4 research approach 5 findings 5.1 practices prior to ipadkinderloop 5.1.1 prior methods of communicating 5.1.2 prior methods of documenting children’s learning and development 5.1.3 prior usage of ipads 5.2 the individualist perspective 5.2.1 educator attitude towards ipadkinderloop and confidence 5.2.2 it champions 5.2.3 leaders 5.2.4 previous it exposure and skill set 5.3 the structuralist perspective 5.3.1 size, complexity, centralisation, and formalisation 5.3.2 parents as stakeholders 5.3.3 government compliance and regulatory requirements 5.4.1 the impact of parents as key stakeholders on the ipadkinderloop appropriation process 6 discussion 7 conclusion \l aj1s special edition electronic commerce november 1998 towards new web application development practices angelita poulymenakou' william drakos2, anastasia papazafeiropoulou3, georgios douktois4 heltrun hellenic electronic trading research unit athens univeisity of economics and business, 76 patission st 104 34 athens greece http://heltrun.aueb.gr e-mails: 'akp® aueb.gr, 2 asd@ aueb.gr. 3agp@ aueb.gr, 4gjd@ aueb.gr abstract electronic commerce over the internet, aims to become a global conveyor belt of business transactions. web applications of increasing sophistication emerge in almost every business sector, reflecting a variety of technical and technological approaches. in this paper we argue that system developers need to reconsider their professional practices in the context of these new technologies by taking advantage of opportunities like short response cycles and easy diffusion of systems results, while they recognise the limitations of traditional practice. we discuss a framework of is development issues for internet based applications and propose guidelines towards new development practices. introduction the case of ec application development, presents for the is profession new and unfamiliar challenges. web based applications characteristics appear to challenge the appropriateness and relevance of methods and techniques followed for conventional systems. moreover, the transition from a developed application to a system in use (and hence an information system) presents user organisations and technology providers alike with unfamiliar complexities. for example in the analysis phase, developers face the pressure of user requirements in the absence of real users and within a very wide, open-ended solution space. the users of the internet vary considerably demographically, making the analysis of their behaviour an extremely difficult task. additionally technological capabilities such as hyperlinking within internet applications makes their design complicated and difficult to standardise. finally, testing and users feedback, have certain advantages in the new medium including speed and accuracy of user response. therefore the body of knowledge concerning system development practice in the domain literature does not reflect the reality of such applications and does not address the requirements of ec projects. the need emerges for a reappraisal of system development theory and method in the light of practical, hands-on experience accumulated in the industry through real life ec application development. in this paper we demonstrate the merits of such an approach by abstracting from our practical project-based experience in ec development21. we discuss systematically the impact that specific characteristics of ec technology on systems development practice. in our approach we stress the transition from the traditional life cycle based conception of systems projects to the new reality created by electronic commerce. standard action guidelines despite criticisms on the relevance usefulness and effectiveness of is development methodologies (bransler & bodker 1993, wynenkoop & russo 1995) these still feature prominently as a basis upon which developers should rely on within an is project methodological guiding principles are summarised by livari (1991) along the following dimensions: • is planning, which refers to the development of the organisation-wide information system plan • is design, which is used to denote systems analysis and design activities involved in the development of individual application information systems • is implementation, is used in the sense of organisational implementation • is use, refers to the end-user's activities related to services provided by the system • is evolution, refers to enhancement type maintenance of the system in this paper we adhere broadly to this terminology with some semantic changes regarding these guidelines. the is planning for example, is examined not only within one organisation's limits, but includes also interorganisational systems. additionally, the is design will be separated in two phases: is analysis and is 21 among the heltrun projects are the following:tappe telematics for administrations: public procurement in europe. an eu telematics for administration project (1997-1998) [17]; newspaper online an internet based multimedia information retrieval system for a greek newspaper (1997); [19,20]; coin classifieds over internet, an interactive business to consumer web based application (19971998); active advertising and commerce through internet in the context of virtual enterprise. an eu esprit project for the establishment of a multi-agent electronic retail marketplace (1998-) [18]; syn an intelligent electronic catalog system for the athens chamber of commerce and industry 107 ajis special edition-electronic commerce november 1998 between chain stores and customers is a meaningful preparatory strategy. analysis problem identification. the analysis phase in an is project is concerned with determination of user requirements accompanied by all the other necessary negotiation processes that will enable the development of an application model (data, process, real time behaviour). a major difference between conventional and web based is is the importance of market characteristics in defining application requirements. while in conventional is, requirements are usually influenced by the employees who will be the direct users of the application, in web applications it is far more difficult to approach consumers-at-large who will be the ultimate end-users. consumers usually present large discrepancies in demographic characteristics, market search patterns and buying behaviour. moreover, the manner in which the proposed application will be integrated with existing systems needs to be addressed explicitly at two levels: the level of business processes and the level of internal information systems. for example orders received through the internet need to be forwarded to the central processing system, while 'customer support' needs to be restructured to cater for internet borned customers. this situation places emphasis on business modelling as part of systems analysis. new conditions specification. under the conditions discussed above analysis can only be effective as an interactive process driven by prototype development; this process is usually initially driven by technological capabilities of the application environment. obviously a lot depends on the choice of technological platform and application tools used for the first prototype. however the real opportunity of web applications is the speed with which end user response concerning the initial prototype can be obtained. subsequent rounds of prototype refinement can and should be informed by consumer related data, gathered through user monitoring and profiling. proposal for action. analysis for web based development needs to draw on a variety of data resources: market data is needed to reveal structural characteristics that need to be reflected in the application functionality, and consumer-and product-related data that should serve as the basis for developing the application development model. traditional data modelling methods such as entity relationship modelling have still a role to play in this stage. however it should be stressed that "analysis" for web applications can no longer be seen as an is lifecycle stage; it is rather an iterative and interactive negotiation activity informed by user feedback obtained over the internet. design problem identification. there are two main areas of concern for designers of web applications (balasubramanian et al 1995, isakowitz et al 1995), the design of the application interface and the design of the underlying information management structure. internet applications require a new conception of the role and functionality of the user interface. there are four areas of issues here. firstly, the end user-consumer should have the feeling of contacting the physical entity (store, shopping mall, service provider, sales person) corresponding to the transaction being carried out. secondly, the interface should reflect and exploit the necessary search and retrieval processes (concerning product characteristics etc) of the physical commercial environment. moreover there is a need to promote the additional capabilities offer by the electronic medium (value added services such as customised services). thirdly, the interface should offer clear guidance and support for the entire purchasing process; the particular problem here is that this process is made up by heterogeneous activities (searching through product features versus making an electronic payment). lastly, the interface should offer capabilities to support related business process such as product promotion and advertisement. as far as the data design and management are concerned the requirements of web applications do not pose significant new challenges to designers and database design principles (cutts 1991) can be used. new conditions specification. internet technology offers technical opportunities to address the above issues, in particular hyperlinking. this allows the easy navigation through the application's features and linking to other related commercial sites. systems development methodologies (balasubramanian et al 1995, isakowitz et al 1995) specifically targeted to internet applications offer concrete support for the design stage (e.g. automatic creation of web interface). data design becomes a particularly important activity for internet applications for two reasons. firstly, because internet applications are data driven in the sense that they are meant to reflect products, services, customers as well as the firm itself. secondly, because through a solid data design the application developer has a greater grasp over the entire application as well as the capability to enhance and extend it in the future. finally, integrated electronic commerce transactions require the seamless co-operation of a multitude of heterogeneous services (payment, shipping, dispatching). in many cases third party components are applicable, which offers opportunities for lowering the cost of application design and development but systems designers 110 ajis special edition electronic commerce november 1998 still need to create the links to the main application. proposal for action. database design skills are a sine-qua-non within the context of web based applications. there is an opportunity here for systems professionals to capitalise on existing experience in totally new and innovative application fields. in essence the design of internet applications sets off with the design of the database and processes with the embedding and the integration of new technologies such as web authoring tools or java. implementation problem identification. conventionally the main challenges of systems implementation (i.e. "going live") have been the familiarisation of the users with a new environment and the migration from the old to the new system. both these issues have very different profiles for internet based applications. as far as users are concerned the onus lies on the system itself in capturing the attention, supporting the interaction and helping the user in cases of difficulty. with respect to migrating to a new way of work, business to consumer applications are usually complementary to traditional commerce operations. there are significant issues to be tackled in the restructuring of internal business processes (e.g. order processing, inventory and distribution management) to cater for the business environment. the integration of business processes creates new stakeholders for internet applications, these are employees in other business functions who, although not directly involved with the application need to deal with business issues resulting from each operation. new conditions specification. in addition to internet applications under development there is the need to set up the transaction management environment for this application. powerful middleware is available (tennenbaum et al 1997, evans & rogers 1997) but the important issue here is parameterisation. this process needs to take advantage of the potential of the technology while at the same time there is a need to provide also opportunities for improvement of existing business operations. for example, the monitoring of systems use and the collection of consumer related data might lead to the development of new marketing strategies. the monitoring of system use provides also data of the volume of users as well as of transactions created through the application, that are important for subsequent extension and exploitation of the application. proposal for action. given the strpng influence of technological capabilities offered within electronic commerce platforms, as well as by application development tools, a systems project in this area usually begins by the acquisition and installation of such a technological infrastructure and proceeds with the tailoring of each application component. use problem identification. business to consumer internet applications have a user community which is potentially the world-at-large. the very nature of these applications is significantly affected by the emerging mode of their use by consumers at large. successful applications are those which incorporate feedback from end-users in their amended versions. this is an open-ended process, which requires the allocation of development effort on a continuous basis. new conditions specification. the decision to use a business to consumer internet application lies entirely on end-users. however, there are several factors that impact on that with respect to the technological side of the system die necessary conditions for achieving successful system use can be summarised under the headings of user friendliness, simplicity, understadability, consistency and integrated support. however there are product/service as well as firm related characteristics that have a direct impact on system use in this case. products or services offered through the internet need to compete in terms of quality and cost on a global basis. business processes of the firm need to be efficient and reliable in accordance to the "promises made" in the new system. in many cases system use is the first step in a process of sweeping changes across the organisation. proposal for action. a critical point in securing successful use of such applications is their careful promotion over electronic (e.g. advertisement in related sites) as well as over traditional means. in many cases such applications need to be based on a close collaboration between the providers of the products or services with the providers of technological capabilities, for example internet providers as well as with business partners to whom particular processes are outsourced (e.g. distribution). in such cases system use may trigger a new round of system development activities aiming at the integration of systems across partners in the emerging virtual chain. i l l ajis special edition electronic commerce november 1998 systems development as systems evolution business to consumer internet applications are being developed against the background of rapidly changing markets increasing sophistication of consumer behaviour, rapid technological change, significant legal and regulatory constrains and the current limited penetration of commercial internet use in households (cockburn and wilson 1996). the terrain is changing fast. many applications of today will not survive tomorrow without substantial rethinking and restructuring. in many cases current business to consumer applications are more significant as proof-of-concept vehicles (to explore the viability of internet as a commerce medium) rather than as business entities in their own right. in this sense systems development does not end with a successful launching of the application over the internet; but rather this is the point at which it takes off. conclusions: implications for research and practice internet based applications are a prime example of systems projects that may be seen as beginning at the implementation stage. in a sense, the discussion on planning, analysis and design provided above, should be seen as the framework of issues to be considered at the outset of an internet driven systems project. given the strong influence of technological capabilities offered within electronic commerce, a systems project in this area usually begins by the acquisition and installation of such a technological infrastructure and proceeds with the tailoring and parameterisation of each component. this situation is not dissimilar to what takes place in other areas of systems development (e.g. integrated management support systems). systems professionals and users alike need to adapt to the notion of projects evolving outside the systems lifecycle mentality. such projects necessitate flexibility in time and resource allocation, emphasis on close collaboration between developers and clients, and ample contingencies. the apparent challenges that this presents for the management of systems projects are mitigated by the capabilities that technology offers to developers for quick project turnover. the bottom line here is: negotiation, integration, feedback and evolutionary development the adoption of an evolutionary mentality concerning the development of such systems is necessary for maintaining the momentum, preserving the experience gained by previous efforts and exploiting the impact of electronic presence in consumer markets. in creating a new body of knowledge for the information systems profession which will reflect the emerging conditions of development we suggest the following: (i) the development of case study based templates of exemplar practices and the dissemination of such material over electronic means. the five dimensions of systems development reviewed in this paper may serve as a framework for structuring such material, (ii) the fostering of closer collaboration across systems developers communities working on ec projects where ever such communities may be located: in academic and research institutions, in technology providers, in user organisations. a means for achieving this through the set up of networks of excellence using the model already practised in other areas of collaborative research, with special emphasis on industry specific ec requirements. (iii) the development of closer links with existing fora already focused on awareness creation (iv) the updating of academic and technical education curricula through the linking of traditional systems development competencies highlighted in this paper with specific requirements of this new type of systems applications. references j. p. bansler, k. bodker, "a reappraisal of structured analysis: design in an organisational context", acm transactions on information systems, vol. 11, no 2, april 1993, pp. 165-193 v. balasubramanian, b. ma min, j. yoo, "a systematic approach to designing a www application ", communications of the acm, august 1995, vol. 38, no 8, pp. 47-48 r. benjamin, and r. wigand, "electronic markets and virtual value chains on the information superhighway", sloan management review, winter 1995, pp. 62-72 c. cockburn, t. wilson, "business use of the world wide web", international journal of information management, 1996, vol. 16, no 2, pp. 83-102 cutts g., "structured analysis and design methodology", alfred waller ltd. publishers, 1991 t. davenport, "process innovation", harvard business press, 1994 w. drakos, g. doukidis, (1997), "electronic publishing over internet: the newspaper on line". 7th mini euro conference on electronic commerce, bruges, belgium, march 1997. w. drakos, g. doukidis, n. papazafeiropoulou, (1997), "the newspaper on-line service: from a business to consumer application perspective". european multimedia, microprocessor systems and electronic commerce conference and exposition (emmsec), florence, italy november 1997. 112 ajis special edition electronic commerce november 1998 european workshop for open systems (ewos) 'technical guide on electronic commerce" ewos etc 066, sep. 1996 e. evans, d. rogers, "using java applets and core a for multi-user distributed applications", ieee internet computing, may-june 1997, vol. 1, no 3, pp 43-55 d.l.hoffman, t..p. novak, p.chatterjee "commercial scenarios for the web:opportunities and challenges" frontiers of electronic commerce reading, ma:addison wesley 1996 j. livari, "a paradigmatic analysis of contemporary schools of is development", european journal of information systems, vol. 1, no. 4, pp. 249-272, 1991 t. isakowitz, e. stohr, p. balasubramanian, "rmm: a methodology for structured hypermedia design", communications of the acm, august 1995, vol. 38, no 8, pp. 34-44 kalakota r. and whinston a., " frontiers of electronic commerce", addison-wesley publishing company, inc 1996 c. molenaar, "interactive marketing", gower 1996 b. pergioudakis, g. doukidis and j. pappas, "defining an architecture for electronic public procurement in europe". 10th international bled electronic commerce conference, bled, slovenia, june 1997. j. m. tennenbaum , t. s. chowdhry, k. hughes, "eco system: an internet commerce architecture", ieee computer, may 1997, vol. 30, no 5,48-55 g.terpsidis, a.moukas, b.pergioudakis, g.doukidis, p.maes, "the potential of electronic commerce in reeneineering consumer-retailer relationships through intelligent agents". european multimedia, microprocessor systems and electronic commerce conference and exposition (emmsec), florence, italy november 1997. s. vinoski, "corba: integrating diverse applications within distributed heterogeneous environments", ieee communications magazine, february 1997, pp. 46-55 t. j. l. wynenkoop, n. l. russo, "systems development methodologies: unanswered questions", journal of information technology (10), 1995, pp. 65-73 113 paper title (paper title style) australasian journal of information systems marrone, taylor & hammerle 2018, vol 21, research on educational technologies international students & active learning 1 do international students appreciate active learning in lectures? mauricio marrone macquarie university mauricio.marrone@mq.edu.au murray taylor macquarie university mara hammerle macquarie university abstract active learning has been linked with increased student motivation, engagement and understanding of course material. it promotes deep learning, helping to develop critical thinking and writing skills in students. less well understood, however, are the responses of international students to active learning. using social constructivist theory, the purpose of this study is to examine domestic and international student perceptions of active learning introduced into large undergraduate accounting information systems lectures. several active learning strategies were implemented over one semester and examined through the use of semi-structured interviews as well as preand postimplementation surveys. our results suggest broad improvements for international students in student engagement and understanding of unit material when implementing active learning strategies. other key implications include international student preference for active learning compared with passive learning styles, and that international students may receive greater benefits from active learning strategies than domestic students due to social factors. based on these findings this paper proposes that educators should seek to implement active learning to better assist and integrate students of diverse backgrounds. keywords: active learning, lectures, international students, information systems, peer interaction, student engagement, collaborative learning 1 introduction the global demand for higher education is resulting in a rapid increase in the number of students desiring an international experience (douglass and edelstein 2015). measured as the proportion of international students enrolled at tertiary education providers, australia is among the countries with the highest levels of international students as a percentage of total tertiary enrolments, with approximately 17.5 percent in 2013 (oecd 2015). the higher education sector received 43.4 percent of all full-fee paying international students in australia in 2015, with enrolments increasing by 9.1 percent and commencements increasing by 7.1 percent compared to 2014 (australian government department of education and training 2015a). in 2015, education activity from international students living and studying in australia contributed 18.2 billion australian dollars in exports, further contributing to the sustainability of its economy and universities (australian government department of education and training 2015b). faced with unfamiliar cultural and educational concepts, international university students often lack social capital in their host country (neri and ville 2008). the unfamiliarity with teaching strategies common in western cultures is considered to be a learning shock for international students and presents an educational barrier (andrade 2006; huang 2012). the language barrier further limits international students from participating fully in the classroom learning environment (kim 2011). these difficulties alongside the benefits of fostering international education experiences suggest that particular emphasis is placed on ensuring australasian journal of information systems marrone, taylor & hammerle 2018, vol 21, research on educational technologies international students & active learning 2 that international students are provided with the tools necessary for a successful integration into their new context. universities are considering different strategies to deal with the difficulties faced by diverse students in western classrooms and are increasingly moving towards more active learning strategies (beetham and sharpe 2013). a significant issue yet to be explored is how active learning impacts international students and their ability to learn in the host country. international students have been found to be challenged by foreign environments due to language and cultural differences (andrade 2006; huang 2012; elliott and reynolds 2014). international students often experience difficulties integrating into the new environment (guo and chase 2011), are challenged with new teaching styles (rienties et al. 2012) and encounter problems with loneliness and isolation (sawir et al. 2008). research suggests that international students prefer to remain in familiar cultural and language groups rather than interacting or participating with domestic students (eisenchlas and trevaskes 2007; dunne 2009; montgomery 2009). given these challenges, a call for research to help understand the impact of active learning on international students is warranted. such research would assist in understanding the experiences of international students to ensure their needs are being met (taylor et al. 2015) and present a way to deal with frustrations of students and staff concerning difficulties with actively engaging and encouraging international students to attend class (ellis, cummings, and turner 2009). this research aims to answer the research question, “how does active learning impact international students?” given that active learning caters for a diverse range of learning styles, our study aims to understand how active learning strategies in large lectures impact international students. the paper’s distinct contribution is a discussion of how international students benefit from a variety of active learning methods that emphasise group learning. the aims of this paper are to understand and compare the impact of active learning on international and domestic students (rq1), to investigate how international and domestic students perceive active learning compared to more traditional passive lecturing styles (rq2) and to understand if and how international students are impacted by an active learning strategy (rq3). we discuss student perceptions ascertained from preand postimplementation surveys and indepth focus group interviews. we elaborate on the issue of the effectiveness of active learning regarding its ability to integrate international students. the following section discusses related work and our methodology. we then examine the findings from surveys and interviews, before we conclude the paper and provide an overview of future research. 2 literature review motivated by a desire to move away from the prevalent passive teaching style and involve students in learning, a number of studies assess the integration of active learning in classroom environments (córdoba and piki 2012; wu and sankar 2013; tan, satin, and lubwama 2013). active learning is described as incorporating strategies that involve students in their learning processes to keep them engaged (prince 2004). active learning not only involves engaging with activities, students must also reflect, analyse, evaluate and communicate information (fink 2013). in comparison to “passive learning” methods whereby the teacher is responsible for instruction, active learning is student-centred and inductive (smith and cardaciotto 2011). active learning has been found to assist in the creation of learning environments that engage, inspire and motivate business students to learn both the unit material and the soft skills required for their future careers (auster and wylie 2006; richards and marrone 2014). several active learning studies have also focused on the benefits of collaborative learning. cooperation may develop interpersonal relationships, strengthen social supports and improve self-esteem (prince 2004). the social constructivist theory has been used by studies to underpin collaborative learning experiences (cooperstein and kocevar-weidinger 2004; woo and reeves 2007; schreiber and valle 2013). this theory places the emphasis on the construction of knowledge by the learner, arguing that an individual’s knowledge is produced through the process of negotiating meanings with others (savin-baden and major 2004). au (1998) argues australasian journal of information systems marrone, taylor & hammerle 2018, vol 21, research on educational technologies international students & active learning 3 that a learner’s cognitive development further depends on social interactions with more advanced and knowledgeable individuals. while engaging with peers and experts, learners generate ideas, negotiate, synthesise individual and group thinking processes, and refine tasks based on sharing insights and perspectives (woo and reeves 2007). collaborative learning activities based on social constructivism offer learners opportunities to experience perspectives of students from diverse backgrounds and to develop critical thinking skills through assessing different viewpoints (stacey 2007). in contrast to students working individually, social interaction enables increased discussion, enthusiasm, participation and experimentation. students are resultantly more relaxed, confident and likely to share opinions (cooperstein and kocevar-weidinger 2004). in response to the limitations of passive lecture formats, instructors have employed the use of multiple active learning strategies. studies have examined collaborative learning (opdecam and everaert 2012; brickner and etter 2008; cavanagh 2011) various games and simulations (fowler 2006; bentley, brewer, and eaton 2009) and mobile learning (murphy et al. 2014). research has found that social interaction is important for an individual’s cognitive ability development (carroll and ryan 2007). peer interaction may also provide learners with heightened belonging, which can positively impact student retention rates and learning outcomes (eames and stewart 2008; huijser, kimmins, and evans 2008). the possibility for students to discuss their interpretations with their peers exposes them to a wider diversity of perspectives than their teacher can provide (liu and carless 2006), which is particularly important in large classrooms where timely feedback from instructors is difficult (rubin 2006). other studies have examined the use of interactive student response systems (carnaghan and webb 2007; edmonds and edmonds 2008; blasco-arcas et al. 2013). this strategy offers a solution that is logistically practical for large lectures, offering a method of increasing students’ metacognitive awareness and engagement levels (hall et al. 2005; pye et al. 2015) but it prevents increased social interaction among students, a common issue faced by traditional lectures (pohl, gehlen‐baum, and bry 2011). in business education, a review of 85 articles on active learning literature has shown that the most frequently occurring outcomes of introducing active learning were understanding of course material, student engagement and real-world understanding with the majority of the research carried out at the undergraduate level (marrone et al. 2016). previous research in the field of information systems have indicated the positive impacts of active learning in a faceto-face environment on students, including an improvement in student satisfaction (wu and sankar, 2013), motivation and engagement (córdoba and piki, 2012), social interactions (abrahams and singh, 2013), and on student academic performance (halttunen, 2005; riley and ward, 2015). together, these studies highlight the positive impact that active learning has on students. however, the literature tells us little about the impact of active learning on international students, an increasingly predominant sector of universities worldwide (kelly and moogan 2012). international students face a range of issues in adapting to a new lifestyle and culture including differences in learning styles, culture shock, homesickness and social difficulties (sawir 2005). problems with english language proficiency have also been found to be a persistent challenge for international students in new learning environments (andrade 2006; kim 2011). studies have found that the language barrier limits the abilities of international students to convey their thoughts in english and to complete classroom tasks (sawir 2005; kim 2011). alongside a perceived preference to remain in familiar cultural and language groups (eisenchlas and trevaskes 2007), limited english language ability creates obstacles for interaction with domestic students (arkoudis et al. 2013). assessment tasks common in western countries, including group discussion and classroom presentations, may additionally be unfamiliar to international students, further limiting their ability to become fully engaged with their peers (kim 2011). thus, the silence typically associated with international students in classroom environments may also be linked to their perceptions of being useless in group discussions and beliefs that domestic students perceive them to be deficient in some way (hsieh 2007). australasian journal of information systems marrone, taylor & hammerle 2018, vol 21, research on educational technologies international students & active learning 4 3 case setting haar et al. (2002) argue that the ability to teach to the needs of learners with different learning styles has become increasingly important as student populations have diversified. as such, a number of active learning strategies were incorporated into a second-year undergraduate accounting information systems unit at a australian metropolitan university over the session 1 2015 teaching period. consistent with the social constructivist theory, the integration of weekly in-class discussions and collaborative activities focused on simulating real-world concepts with direct links to the unit material sought to increase student participation and interaction. student engagement and motivation were also addressed by collecting the names of those students who asked and answered questions during large group discussions in anticipation of a raffle held at the end with a prize relevant to that week’s topic. as an example, the prize offered in the lecture dedicated to the revenue cycle was a package of chocolate coins. rewards are widely viewed as effective, motivating students to expend effort and to encourage their intrinsic interest in an activity (brophy 2013). socrative, a student response system, was also used intermittently to enable the lecturer to test critical thinking questions, formal assessment checkpoints and understanding of the content material (blood et al. 2008). before answering the questions in socrative, students were asked to pair up and discuss their answers. these activities stressed the importance of collaborative learning in both small and large groups. in tutorials, a problem-based learning approach was followed. 4 research approach through a mixed-method approach, incorporating both preand postsurveys and semistructured interviews, student perceptions concerning the implementation of these active learning strategies were examined. student responses from both session 2 2014, before the implementation of the learning strategies, and session 1 2015, in which the learning strategies were integrated, were recorded with the aim of better understanding changes in student perceptions. in accordance with yin (2013), a case study design based on a mixed method approach was adopted, allowing for an increased depth of data to be obtained. through complementing quantitative methods with qualitative, case study-based research, we enhance the validation of the research findings (modell 2005). the questions posed in the likert surveys were based on a question bank provided by the university. the interviews aimed to further develop the findings from the surveys. 4.1 surveys survey research offers many advantages, including versatility, simplicity and speed and when performed correctly, can provide an accurate and efficient overview of a target population (johnson, bristow, and schneider 2011). the survey was distributed to lectures of the unit in session 1 2014 and session 2 2015. social constructivist theory suggests that students can gain a deepened understanding of the material by peer learning and being exposed to a wide variety of perceptions (vygotsky 1980). collaborative learning may be linked to such learning outcomes as increased student engagement, opportunities to assist learning (chad 2012), a more positive course experience and developed teamwork skills (reinig, horowitz, and whittenburg 2011). learners become active in the process of knowledge acquisition as they participate in discussions and co-create knowledge through exchanging perspectives with other students (brindley, blaschke, and walti 2009). hence, we chose questions from the end of session learning and teaching survey that most closely aligned with these factors. in addition to demographical aspects, questions included: [crt01] this teacher communicated well with the class. [crt02] this teacher seemed enthusiastic about teaching the class. [crt03] this teacher created a good environment for learning. australasian journal of information systems marrone, taylor & hammerle 2018, vol 21, research on educational technologies international students & active learning 5 [crt04] this teacher used methods and techniques that were effective in helping me learn. [crt05] this teacher showed genuine interest in assisting students’ learning. [ent01] this teacher motivated me to learn. [ent02] this teacher used a style of presentation that held my interest. [ent03] this teacher encouraged students to learn more about the ideas presented. [ent04] this teacher encouraged students to participate in class. [gcu01] this unit enhanced my ability to think creatively. each question was coded for easier analysis. two hundred and twenty-two students participated in the presurvey whereas 191 students responded to the postsurvey, forming a total of 413 students. a high proportion of students involved in the surveys were international students and spoke a language other than english as their first language. they were also required to take the unit as a compulsory component of their degree program. there were slightly more males involved in the presurvey as compared to females in the postsurvey however this difference was not large. although the second cohort was not comprised of the same students, their demographic similarity and a large number of students surveyed enabled comparisons to be made. table 1 highlights the breakdown of students according to gender and student status. gender nationality male female n/a international domestic n/a pre 92 98 32 105 94 19 post 71 107 13 101 60 29 pre (%) 41% 44% 14% 48% 43% 9% post (%) 37% 56% 7% 54% 31% 15% table 1: demographic details of survey participants 4.2 interviews by securing rich descriptions and capturing the individual’s point of view, qualitative research is distinct from quantitative research methodologies (yin, 2013). semi-structured interviews are accessible, flexible and capable of uncovering important and often undisclosed aspects of human behaviour that surveys are not able to reveal (qu and dumay 2011). in the final weeks of session 1 2015, semi-structured interviews were conducted with student volunteers enrolled in the unit, contributing to the data obtained from the surveys. students were recruited by a research assistant and received gift cards for their participation. to minimise bias, the research assistant was not involved in teaching the unit. the interviews were held in focus groups. focus groups enabled participants to respond to questions and prompts by the interviewer and other participants. as the active learning approach was collaborative and often required the group interaction of students, the use of focus groups is justified and considered extremely useful for studying the impact of individual’s interactions (belanger, 2012). the interviews sought to address how students perceived the learning environment in the active learning classroom as compared to their other units and whether any challenges were faced with the aim of improving future offerings of the unit. each focus group had a maximum of four students and involved the posing of open-ended questions. open-ended questions were used to gain participants attitudes and opinions while simultaneously avoiding biasing respondents towards specific issues that the researchers deemed more important (belanger, 2012). the final number of participants interviewed was australasian journal of information systems marrone, taylor & hammerle 2018, vol 21, research on educational technologies international students & active learning 6 12 with four focus groups being conducted. examination of a higher number of participants or focus groups was not necessary as the four focus groups offered rich data and provided opportunities for cross-case analysis. strauss and corbin (1990, p.188) state that researchers should collect data until “no new or relevant data seem to emerge”. after the final focus group increasing interview data was deemed unnecessary as the twelve interview participants across four focus groups represented the point of theoretical saturation. of the twelve participants, nine were female and three were male, three were classified as international students while the remaining nine were domestic students. the students were split into four focus groups. the participants in each focus group were divided to create a more balanced sample however one focus group was comprised of two female students while another focus group had only domestic students. 4.3 analysis the data collected in the surveys was number-coded using a likert scale with 1 being the lowest possible score and 5 being the highest possible score. likert surveys are well-known, can be used to obtain data from a large number of respondents and enable statistical generalisation (eyvindson et al. 2014). the mean for each outcome was calculated separately before a oneway anova test was used to identify statistical significance. one-way anovas have previously been used by scholars to compare group differences in such aspects as learner satisfaction, learning achievements and participation (jung et al. 2002), perceived cognitive learning (rovai and barnum 2007) and memorisation (cherney 2008). the interviews were transcribed and analysed using both manual and software analysis to help avoid possible interpretation errors (flick, 2008). based on over four hours of interview data more than 24,000 words were transcribed. initially, each interview transcript was manually coded by the researchers. as qualitative data is textual, unstructured and non-numerical, this process allowed for the emerging phenomena to be coherently compressed (basit 2003). this helped the researchers gain insight into the data as well as control over the analytical process. by examining the interview data, the researchers were able to conceptualise key themes and commonalities both within and between focus groups. this helped extract overall impressions and tentative themes. to help further sort, organise, store and manage the large quantities of data involved in the study nvivo software was adopted. although the use of the specialised qualitative software was implemented, it does not replace the researcher or their role in the analysis or interpretation of data (morse et al., 2008). to categorise themes more easily, the research questions were referred to as part of the coding process. nvivo was found to be a useful tool for linking the data with emerging concepts and themes. to aid in the transparency of findings, illustrative quotes from focus group participants are used to emphasise key themes or issues that emerged from manual and software analysis (pratt 2008, 2009). for each illustrative quote, the participant is provided a pseudonym so that they may remain anonymous. 5 findings this section commences by seeking to understand how active learning is perceived by both international and domestic students. first, we present the survey results, comparing the impact on these student cohorts before and after the active learning strategies were implemented (rq1). following this, the interview data from focus groups is presented to highlight how international and domestic students perceive active and passive learning strategies (rq2). finally, the paper will seek to address the impact of active learning on international students (rq3). 5.1 overall findings from surveys rq1: the impact of active learning on international and domestic students to assess the impact of active learning on both international and domestic students (rq1), one-way anovas and an analysis of means were used to analyse the preand postaustralasian journal of information systems marrone, taylor & hammerle 2018, vol 21, research on educational technologies international students & active learning 7 implementation survey data. the overall means are presented in figure 1 and highlights an increase in mean across all aspects assessed. additionally, figure 2 shows that the standard deviations for all outcomes decreased in the second survey, indicating that the active learning strategies lead to overall higher mean scores with lower variability. figure 1: means for survey outcomes figure 2: standard deviation (sd) for survey outcomes as highlighted by table 2, there was a statistically significant difference between preand postimplementation groups as determined by one-way anovas for all ten outcomes examined, suggesting that the introduction of active learning was associated with improvements in all aspects investigated. the size of the effect (r) was calculated for all appropriate analyses. 4. 44 4. 51 4. 38 4. 13 4. 31 4. 07 4. 17 4. 16 4. 29 3. 98 4. 8 4. 85 4. 74 4. 62 4. 69 4. 49 4. 63 4. 57 4. 76 4. 48 c r t 0 1 c r t 0 2 c r t 0 3 c r t 0 4 c r t 0 5 e n t 0 1 e n t 0 2 e n t 0 3 e n t 0 4 g c u 0 1 pre-implementation post implementation 0. 84 0. 83 0. 88 0. 95 0. 93 0. 93 0. 97 0. 95 0. 84 0. 98 0. 47 0. 44 0. 53 0. 63 0. 54 0. 66 0. 57 0. 61 0. 51 0. 71 c r t 0 1 c r t 0 2 c r t 0 3 c r t 0 4 c r t 0 5 e n t 0 1 e n t 0 2 e n t 0 3 e n t 0 4 g c u 0 1 sd pre-implementation sd post-implementation australasian journal of information systems marrone, taylor & hammerle 2018, vol 21, research on educational technologies international students & active learning 8 outcome f p r [crt01] this teacher communicated well with the class f (1,378) = 24.937 0.001 0.25 [crt02] this teacher seemed enthusiastic about teaching the class. f (1,375) = 22.077 0.001 0.24 [crt03] this teacher created a good environment for learning f (1,382) = 22.333 0.001 0.24 [crt04] this teacher used methods and techniques that were effective in helping me learn f (1,381) = 32.06 0.001 0.28 [crt05] this teacher showed genuine interest in assisting students’ learning f (1,373) = 22.241 0.001 0.24 [ent01] this teacher motivated me to learn. f (1,378) = 23.979 0.001 0.24 [ent02] this teacher used a style of presentation that held my interest. f (1,367) = 28.211 0.001 0.27 [ent03] this teacher encouraged students to learn more about the ideas presented. f (1,378) = 23.658 0.001 0.24 [ent04] this teacher encouraged students to participate in class. f (1,380) = 39.386 0.001 0.25 [gcu01] this unit enhanced my ability to think creatively f (1,371) = 30.191 0.001 0.27 table 2: statistical significance of improvements in survey outcomes the results suggest that active learning strategies can positively affect student perceptions of the lecture. specifically, our results suggest that when active learning is introduced into a large lecture hall, students may perceive improvements in the lecturer's communication skills, enthusiasm and interest in assisting student’s learning and will benefit from the creation of a good learning environment and the use of effective learning techniques. the students reported higher motivation to learn, participate, think creatively and remain interested in the unit material. 5.2 survey perceptions of international students concerning active learning in the following, we compare how international and domestic students perceived the active learning introduced into the accounting information systems lectures (rq1). a comparison of means is provided in figure 3. similarly, for international students, the means for all aspects assessed increased, suggesting that improvements in all areas examined by the surveys were noted. in comparison, the differences between means (pre and post) for international students tended to be higher than for domestic students in both surveys, indicating that international students may perceive a greater positive impact on active learning than domestic students. australasian journal of information systems marrone, taylor & hammerle 2018, vol 21, research on educational technologies international students & active learning 9 figure 3: comparison of the means for international and domestic students for survey outcomes a one-way anova was calculated of international students' perception of their learning experience before and after the introduction of active learning. the analysis presented in table 3 highlights that the improvements in all ten variables analysed were statistically significant. 4.56 4.65 4.47 4.15 4.36 4.08 4.25 4.21 4.38 4.02 4.89 4.88 4.72 4.70 4.77 4.46 4.59 4.58 4.81 4.45 4.31 4.35 4.27 4.12 4.25 4.06 4.08 4.11 4.20 3.94 4.74 4.82 4.75 4.56 4.63 4.51 4.65 4.57 4.71 4.50 3.7 3.9 4.1 4.3 4.5 4.7 4.9 crt01 crt02 crt03 crt04 crt06 ent02 ent03 ent06 ent07 gcu03 domestic pre domestic post international pre international post australasian journal of information systems marrone, taylor & hammerle 2018, vol 21, research on educational technologies international students & active learning 10 f p r [crt01] this teacher communicated well with the class. f (1,193) = 20.484 0.001 0.31 [crt02] this teacher seemed enthusiastic about teaching the class. f (1,193) = 24.492 0.001 0.34 [crt03] this teacher created a good environment for learning. f (1,200) = 22.597 0.001 0.32 [crt04] this teacher used methods and techniques that were effective in helping me learn. f (1,196) = 15.179 0.001 0.27 [crt05] this teacher showed genuine interest in assisting students’ learning. f (1,196) = 13.991 0.001 0.26 [ent01] this teacher motivated me to learn. f (1,195) = 16.140 0.001 0.28 [ent02] this teacher used a style of presentation that held my interest. f (1,190) = 29.386 0.001 0.30 [ent03] this teacher encouraged students to learn more about the ideas presented. f (1,196) = 18.813 0.001 0.30 [ent04] this teacher encouraged students to participate in class. f (1,197) = 27.771 0.001 0.35 [gcu01] this unit enhanced my ability to think creatively. f (1,193) = 22.479 0.001 0.32 table 3: statistical significance of improvements in survey outcomes for international students our results suggest that incorporating active learning into the classroom has a greater impact on international students than domestic students. international students may appreciate being immersed in an active learning environment to a greater extent than domestic students. in the interviews, international students tended to report outcomes that could be closely linked to the statements “this teacher encouraged students to participate in class” and “this teacher created a good environment for learning.” as can be observed from table 3, these outcomes were amongst those that experienced the most significant improvements. 5.3 focus group perceptions from international and domestic interviewees comparing active and passive learning rq2: how international and domestic students perceive active learning compared with passive lecturing styles our findings suggest that both international and domestic students had similar perceptions of learning styles, with participants highlighting the benefits of active learning over a more passive approach. a key outcome of the interviews was the emphasis on increased levels of student engagement due to active learning. both domestic and international students found that the introduction of activities into the lecture improved student engagement, comparing this with passive lectures in which students are commonly disengaged. according to elias: “it was actually very engaging because usually in other lectures, there’s just a lot of passive, just sitting down, just listening. but this one was good because you engaged more with the lecture.” students reinforced the notion that engaging and active lectures better-supported student learning. passive lectures do not encourage student engagement whereas active lectures provide students with greater opportunity to respond, encouraging students to discuss the unit material and reprocess what has been taught. another student, ruby, echoes this sentiment, suggesting that the teaching materials in the active learning lectures help students to understand more. australasian journal of information systems marrone, taylor & hammerle 2018, vol 21, research on educational technologies international students & active learning 11 “like in my lecture, she [the lecturer] just gets the slides and finish and go earlier. but that [active learning] lecturer, he keeps more activities and slides, so we can actually know what he’s trying to explain.” another significant comparison between active and passive lectures mentioned by both domestic and international students was the possibility to make friends and feel a sense of connection. according to both groups, there is little possibility to interact with the students seated nearby in passive lectures due to the focus placed on listening to the teaching staff. in comparison, the students of this study felt they were able to chat and get to know other students in the active lectures due to the integration of collaborative activities. the integration of group work in applying theory to practice exposed students to a wide variety of different perspectives, questioning their notion of the truth. according to mark: “there was an activity where we had to describe the methods that we use to make tea. that was to demonstrate that not everyone develops the system in the same way… i was able to see that some people developed tea without using milk [laughter], some people think that it should be used with milk like me [laughter]. so people have different cultural backgrounds and different perspectives of the way we do things.” the benefits associated with the active learning strategies resulted in both domestic and international students feeling more motivated to attend lectures. a number of students reported that they feel unmotivated to attend lectures in which there is little opportunity to ask questions or interact. selina perceives studying at home to be more beneficial when she is unable to understand the content of passive lectures. in comparison, the active lectures helped her to better understand greater amounts of content in a shorter amount of time: “sometimes i feel that with the other lectures… if you don’t understand it, you don’t feel motivated to come. it just seems like i can study at home and understand it better. but these lectures i feel, really do help me understand the work, and i don't have to put so much effort when i go home”. 5.4 focus group perceptions from international interviewees discussing benefits of active learning rq3: how are international students impacted by an active learning strategy in this section, we aim to explore why active learning was particularly effective for international students. we focus on key themes that emerged from the interview data. overall international student perceptions of active learning approaches during lectures are positive suggesting that international students perceive active learning to be highly beneficial. a key theme that emerged from focus groups was that international students felt active learning strategies assisted them in understanding the course content. students reported that the activities helped to make intangible concepts more comprehensible and easy to understand. by focusing on examples after introducing theory, active learning can improve students’ ability to apply difficult theoretical concepts. for ruby, the tea activity, in which students were instructed to write a procedure for brewing tea, helped her understand how the same task can be achieved in a number of equally beneficial ways. active learning activities such as the tea activity help students to understand the different processes that organisations have, thereby helping a topic that was intangible and difficult to understand become comprehensible. this highlights the importance of using examples that students are familiar with to guide their comprehension. as ruby highlights: “so that shows a difference and then we know that different people will have different processes to do one thing. so different organisations will have [different] process for what they’re doing.” the active learning strategies for the international student interviewees provided exposure to a wide variety of different perspectives through collaborative learning. several interviewees placed an emphasis on the benefits of student interaction, stressing their satisfaction in being australasian journal of information systems marrone, taylor & hammerle 2018, vol 21, research on educational technologies international students & active learning 12 able to understand a concept from another student’s point of view. student interaction was a critical aspect of this as it forced students to engage with others and resulted in a broader understanding of the different ways in which a topic can be approached and understood. according to laurice: “when you interact with people, you always get good ideas. and they also help you more to understand that topic.” [sic] through the collaborative learning activities, students are encouraged to interact with a number of students. this contrasts with the traditional lecture where there is limited opportunity for students to discuss the topic and form their own meanings and interpretations. this aspect was particularly prominent for wonder: “someone’s sitting next to you in a lecture, because everyone’s listening to the lecturer and that’s all, after two hours, “okay, we’re done, just leave.” but he asks us to talk in pairs and discuss the questions.” these factors helped to ensure an environment in which international students felt safe to participate in class discussions. for laurice, the emphasis placed by the lecturer on asking questions and providing encouragement helped her to feel more comfortable to interact. this sentiment was also echoed by other interviewees who felt that the active learning strategies helped to promote more class participation than passive learning lectures. students can often find it difficult to express themselves in front of large groups of people due to fear of being laughed at. the active learning environment ensured that they felt comfortable in discussions. as laurice emphases; “but like more than 30 students, i’m still feeling nervous. but i think in the lecture, it’s okay. even if i give the wrong answer, no one will laugh at you or something. it’s just a very fair discussion, like a huge group discussion.” another key theme that emerged was the number and quality of peer interactions that international students experienced because of an active learning approach. this helped them achieve a sense of belonging and friendship. the ability to make friends is a beneficial aspect of the active learning strategies implemented. a recurrent theme in the interviews was a desire amongst interviewees to be involved and be a part of the community nurtured in the active learning classroom. the collaborative learning activities can be particularly helpful in ensuring belonging due to the potential for making friends through interactions enforced through the activities. as ruby discusses; “i think i have a sense of belonging because we have to discuss one question with the one sitting next to me all the time… we can make friends every lecture, like the one sitting next to me. i think that’s good.” 6 discussion the present study was designed to investigate how and why international and domestic students perceive and are impacted by active learning strategies within lectures. the first research question in this study sought to determine the impact of active learning on both international and domestic students. our results indicate that both international and domestic participants perceived that the active learning strategies implemented increased their understanding, satisfaction with the unit and desire to participate, thus strengthening their ability to progress through the course. our results show the benefits of active learning in addressing a wide range of concerns, ensuring that students feel comfortable to express themselves and enhancing their learning opportunities. in a context in which lectures can be described as depersonalised experiences (smith and cardaciotto 2011), these contributions are particularly noteworthy. the study also set out to investigate how international and domestic students perceive active learning to differ from more traditional passive lecturing styles. our findings highlight that international and domestic students were found to appreciate the opportunity to explore new australasian journal of information systems marrone, taylor & hammerle 2018, vol 21, research on educational technologies international students & active learning 13 ways of learning, enabling them to more readily express their viewpoints and participate in class discussions. in the case of international students, previous studies indicate that typically asian students participate less in group discussions or debate in class and are less keen to ask or answer questions (duanmu, li, and chen 2009). international students may face difficulties in adjusting to a new learning style (jackson 2003) given the emphasis placed on alternate learning styles in different educational paradigms (hawthorne, minas, and singh 2004; wong 2004). our findings for international students, however, indicate an increased motivation to participate and motivation to learn. increases in engagement and participation were indicated by both the surveys and interviews. international students indicated that a good environment for learning was created and that they were able to benefit from the focus on active learning and collaborative activities. similar to the findings of chalmers and volet (1997), these results challenge university educators to reconsider their views of international students being surface learners who are passive and non-participatory in class. finally, we explore why international students are impacted by an active learning strategy. results show that the exposure of students to different perspectives helped them to form a deeper understanding of the unit material as their perceptions were shaped by the knowledge and experiences of their peers. the comments made by the interviewees resemble the positions articulated by the social constructivist theory; that knowledge is co-constructed through working cooperatively with peers. through peer interaction, beliefs were challenged, and the effects of cultural backgrounds on the students’ initial perceptions were made apparent. in this process, students gained an increased sense of belonging and inclusion. these findings contrast those of other studies, which have found that international students tend to feel alienated and isolated, resulting in undermined confidence, reduced motivation, decreased participation and withdrawal from learning opportunities (treloar et al. 2000; hawthorne, minas, and singh 2004). instead, our study highlights that the exposure of a multicultural cohort to active learning may benefit students in a number of ways, including by enhancing opportunities for peer interaction and belonging. this understanding is particularly important as positive changes in self-perceptions including scholastic competence, self-worth, and likelihood to complete college degrees are linked with improved social networks (pittman and richmond 2008; wilson and gore 2013). campbell and li’s (2007) research found that group work could provide international students with opportunities to improve their english language skills, enhance their cultural understandings and develop their teamwork, interpersonal communication and negotiation skills. this study builds on this and highlights that when collaborative learning tasks require different skills and approaches to complex problems, placing students in culturally heterogeneous groups can result in more advanced learning outcomes. overall, the study’s findings highlight the need for educators to design the learning approach such that students can participate and express their viewpoints. we conclude that breaking large student cohorts into small groups may be especially beneficial for international students as they feel more comfortable and have a greater motivation to learn. as indicated by our study, students can brainstorm ideas in a smaller, safer social environment and are more confident in expressing their views to a larger audience. taken together, these findings emphasise the importance of transitioning from passive learning modes to a more active and cohesive approach. the findings of this study highlight that the introduction of active learning may assist in improving the tertiary learning experience providing important insights for other university educators. results of this study are inherently limited. the preliminary results of this study represent only a small sample of the student cohort enrolled in this unit over the two semesters. thus, the results from the self-selected small sample of students included in the interviews may not be reflective of the broader population. furthermore, while the survey and focus group questions focused on the lecture, tasks completed in tutorials may also affect the perception of students. further qualitative research examining larger groups of international students would be beneficial. this study also focused solely on an australian context. as results may differ between contexts, similar studies in other countries with different cohorts are needed to australasian journal of information systems marrone, taylor & hammerle 2018, vol 21, research on educational technologies international students & active learning 14 support our findings. nevertheless, our study reports findings from an important yet underresearched study area. we provide support for the use of active learning in large lectures with diverse student cohorts, using a wider range of strategies to support student learning than in previous studies in this area. we place greater emphasis on differentiating between the perceptions of international and domestic students and are the first that we know of to report on the reasons why international students may benefit from active learning. importantly our findings show that international students may benefit from active learning strategies to a greater extent than domestic students. this research represents a step forward in our understanding of the benefits of active learning strategies for both domestic and international students. however, future research should continue to seek to understand which obstacles active learning can assist in overcoming. a large body of literature suggests that international students must cope with a number of different obstacles when compared with domestic students. common obstacles mentioned in the literature include difficulties to socially integrate into the host country (rienties et al. 2012), academic perceptions of international students’ english language ability (arkoudis et al. 2013) and feelings of homesickness (baba and hosoda 2014). future research might also seek to address the academic culture shock experienced by international students and how active learning might help in this area. the opinions of lecturers and teaching staff on how they perceive active learning rather than passive teaching approaches to benefit international students would also be a prudent topic of research. this would aid in the understanding of teaching strategies for international students and inform future policy so that the demands and needs of teaching staff are also met. 7 conclusion this research aims at obtaining survey and interview data concerning the perceptions of international and domestic students towards active learning in lectures in an undergraduate accounting information systems course. the findings of this research clearly highlight that active learning is perceived positively by both student groups, improving student attitudes towards the course and teaching staff, engagement and participation. international students, in particular, benefited from this learning strategy with its emphasis on integration and participation in weekly lectures. active learning provided students with an environment in which they felt safe, supporting peer interaction and a heightened ability to apply and understand theoretical concepts. furthermore, our findings indicate that social constructivist theory assists with better understanding the complexities inherent in collaborative learning. students were exposed to a wide variety of perspectives rather than merely the lecturer’s view, helping them to shape and reshape their thoughts and perspectives. students learnt from each other and were involved in the co-construction of knowledge. this study can aid in the development of institutional planning by providing key insights into the ways that active learning may benefit both domestic and international students. references abrahams, alan s., and tirna singh. (2013). “expeditionary learning in information systems: definition, implementation, and assessment.” decision sciences journal of innovative education 11 (1): 47–75. doi:10.1111/j.1540-4609.2012.00369.x. andrade, maureen s. (2006). “international students in english-speaking universities.” journal of research in international education 5 (2): 131–54. doi:10.1177/1475240906065589. arkoudis, sophie, kim watty, chi baik, xin yu, helen borland, shanton chang, ian lang, josephine lang, and amanda pearce. (2013). “finding common ground: enhancing interaction between domestic and international students in higher education.” teaching in higher education 18 (3): 222–35. doi:10.1080/13562517.2012.719156. australasian journal of information systems marrone, taylor & hammerle 2018, vol 21, research on educational technologies international students & active learning 15 au, kathryn h. (1998). “social constructivism and the school literacy learning of students of diverse backgrounds.” journal of literacy research 30 (2): 297–319. doi:10.1080/10862969809548000. auster, ellen r., and krista k. wylie. (2006). “creating active learning in the classroom: a systematic approach.” journal of management education 30 (2): 333–53. doi:10.1177/1052562905283346. authors, 2016. australian government department of education and training. (2015a). “export income to australia from international education activity in 2014-15.” https://internationaleducation.gov.au/research/researchsnapshots/documents/export%20income%20fy2014-5.pdf. ———. 2015b. “monthly summary on international student enrolment dataaustraliaytd october (2015).” https://internationaleducation.gov.au/research/internationalstudentdata/documents/monthly%20summaries%20of%20international%20student%20enr olment%20data%202015/10_october_2015_monthlysummary.pdf. baba, yoko, and megumi hosoda. (2014). “home away home: better understanding of the role of social support in predicting cross-cultural adjustment among international students.” college student journal 48 (1): 1. basit, tehmina. (2003). “manual or electronic? the role of coding in qualitative data analysis.” educational research 45 (2): 143–54. doi:10.1080/0013188032000133548. beetham, helen, and rhona sharpe. (2013). rethinking pedagogy for a digital age: designing for 21st century learning. routledge. belanger, france. (2012). "theorizing in information systems research using focus groups." australasian journal of information systems 17 (2). bentley, kathleen a., peter c. brewer, and tim v. eaton. (2009). “motivating students to prepare for class and engage in discussion using the hot seat.” journal of accounting education 27 (3): 155–67. doi:10.1016/j.jaccedu.2010.07.001. blasco-arcas, lorena, isabel buil, blanca hernández-ortega, and f. javier sese. (2013). “using clickers in class. the role of interactivity, active collaborative learning and engagement in learning performance.” computers & education 62 (march): 102–10. doi:10.1016/j.compedu.2012.10.019. blood, erika and richard neel. (2008). “using student response systems in lecture-based instruction: does it change student engagement and learning?” journal of technology and teacher education 16 (3): 375–83. brickner, daniel r., and edwin r. etter. (2008). “strategies for promoting active learning in a principles of accounting course.” academy of educational leadership journal 12 (2): 87. brindley, jane, lisa marie blaschke, and christine walti. (2009). “creating effective collaborative learning groups in an online environment.” the international review of research in open and distributed learning 10 (3). http://www.irrodl.org/index.php/irrodl/article/view/675. brophy, jere e. (2013). motivating students to learn. routledge. carnaghan, carla, and alan webb. (2007). “investigating the effects of group response systems on student satisfaction, learning, and engagement in accounting education.” issues in accounting education 22 (3): 391–409. doi:10.2308/iace.2007.22.3.391. carroll, judith, and janette ryan. (2007). teaching international students: improving learning for all. routledge. australasian journal of information systems marrone, taylor & hammerle 2018, vol 21, research on educational technologies international students & active learning 16 cavanagh, michael. (2011). “students’ experiences of active engagement through cooperative learning activities in lectures.” active learning in higher education 12 (1): 23–33. doi:10.1177/1469787410387724. chad, paul. (2012). “the use of team-based learning as an approach to increased engagement and learning for marketing students: a case study.” journal of marketing education, july, 0273475312450388. doi:10.1177/0273475312450388. chalmers, denise, and simone volet. (1997). “common misconceptions about students from south‐east asia studying in australia.” higher education research & development 16 (1): 87–99. doi:10.1080/0729436970160107. cherney, isabelle d. (2008). “the effects of active learning on students’ memories for course content.” active learning in higher education 9 (2): 152–71. doi:10.1177/1469787408090841. chu, ling, and theresa libby. (2010). “writing mini-cases: an active learning assignment.” issues in accounting education 25 (2): 245–65. doi:10.2308/iace.2010.25.2.245. cooperstein, susan e., and elizabeth kocevar‐weidinger. (2004). “beyond active learning: a constructivist approach to learning.” reference services review 32 (2): 141–48. doi:10.1108/00907320410537658. córdoba, josé-rodrigo, and andriani piki. (2012). “facilitating project management education through groups as systems.” international journal of project management 30 (1): 83–93. doi:10.1016/j.ijproman.2011.02.011. doubleday, alison, blase brown, philip patston, pamela jurgens-toepke, meaghan strotman, anne koerber, colin haley, charlotte briggs, and g. knight. (2015). “social constructivism and case-writing for an integrated curriculum.” interdisciplinary journal of problem-based learning 9 (1). doi:10.7771/1541-5015.1502. douglass, john, and richard edelstein. (2015). “the global market for international students: american perspectives.” international higher education 0 (59). https://ejournals.bc.edu/ojs/index.php/ihe/article/view/8482. duanmu, jing-lin, gang li, and wei chen. (2009). “determinants of international students’ academic performance: a comparison between chinese and other international students.” journal of studies in international education, may. doi:10.1177/1028315309331490. dunne, ciarán. (2009). “host students’ perspectives of intercultural contact in an irish university.” journal of studies in international education, january. doi:10.1177/1028315308329787. eames, chris, and kevin stewart. (2008). “personal and relationship dimensions of higher education science and engineering learning communities.” research in science & technological education 26 (3): 311–21. doi:10.1080/02635140802276686. edmonds, christopher t., and thomas p. edmonds. (2008). “an empirical investigation of the effects of srs technology on introductory managerial accounting students.” issues in accounting education 23 (3): 421–34. eisenchlas, susana, and sue trevaskes. (2007). “developing intercultural communication skills through intergroup interaction.” intercultural education 18 (5): 413–25. doi:10.1080/14675980701685271. elliott, carole jane, and michael reynolds. (2014). “participative pedagogies, group work and the international classroom: an account of students’ and tutors’ experiences.” studies in higher education 39 (2): 307–20. doi:10.1080/03075079.2012.709492. australasian journal of information systems marrone, taylor & hammerle 2018, vol 21, research on educational technologies international students & active learning 17 ellis, l., e. a. cummings, and p. turner. (2009). “engaging international students through pbl.” in proceedings of the 2nd international research symposium on pbl ’09, 1:ej. http://ecite.utas.edu.au/61489. eyvindson, kyle, annika kangas, teppo hujala, and pekka leskinen. (2014). “likert versus qapproaches in survey methodologies: discrepancies in results with same respondents.” quality & quantity 49 (2): 509–22. doi:10.1007/s11135-014-0006-y. fink, l. dee. (2013). creating significant learning experiences: an integrated approach to designing college courses. john wiley & sons. flick, uwe. (2008). designing qualitative research. sage. fowler, lou. (2006). “active learning: an empirical study of the use of simulation games in the introductory financial accounting class.” academy of educational leadership journal 10 (3): 93. guo, shibao, and mackie chase. (2011). “internationalisation of higher education: integrating international students into canadian academic environment.” teaching in higher education 16 (3): 305–18. doi:10.1080/13562517.2010.546524. haar, jean, gretchen hall, paul schoepp, and david h. smith. (2002). “how teachers teach to students with different learning styles.” the clearing house: a journal of educational strategies, issues and ideas 75 (3): 142–45. doi:10.1080/00098650209599254. hall, richard h, harvest l collier, marcie l thomas, and michael g hilgers. (2005). “a student response system for increasing engagement, motivation, and learning in high enrollment lectures viewcontent.cgi.” in amcis 2005 proceedings. http://aisel.aisnet.org.simsrad.net.ocs.mq.edu.au/cgi/viewcontent.cgi?article=1787&co ntext=amcis2005. halttunen, kai, and kalervo järvelin. (2005). “assessing learning outcomes in two information retrieval learning environments.” information processing & management 41 (4): 949–72. doi:10.1016/j.ipm.2004.02.004. hawthorne, lesleyanne, i. harry minas, and bruce singh. (2004). “a case study in the globalization of medical education: assisting overseas-born students at the university of melbourne.” medical teacher 26 (2): 150–59. doi:10.1080/0142159032000150539. hsieh, min-hua. (2007). “challenges for international students in higher education: one student’s narrated story of invisibility and struggle.” college student journal 41 (2): 379–91. huang, ying. (2012). “transitioning challenges faced by chinese graduate students.” adult learning 23 (3): 138–47. doi:10.1177/1045159512452861. huijser, h, l kimmins, and p evans. (2008). “peer assisted learning in fleximode: developing an online learning community viewcontent.cgi.” journal of peer learning 1 (7): 51–60. jackson, jane. (2003). “case-based learning and reticence in a bilingual context: perceptions of business students in hong kong.” system 31: 457–69. johnson, jill m., dennis n. bristow, and kenneth c. schneider. (2011). “did you not understand the question or not? an investigation of negatively worded questions in survey research.” journal of applied business research (jabr) 20 (1). http://cluteinstitute.com/ojs/index.php/jabr/article/view/2197. jung, insung, seonghee choi, cheolil lim, and junghoon leem. (2002). “effects of different types of interaction on learning achievement, satisfaction and participation in webbased instruction.” innovations in education and teaching international 39 (2): 153– 62. doi:10.1080/14703290252934603. australasian journal of information systems marrone, taylor & hammerle 2018, vol 21, research on educational technologies international students & active learning 18 kelly, philip, and yvonne moogan. (2012). "culture shock and higher education performance: implications for teaching." higher education quarterly 66 (1) 24-46. kim, hye yeong. (2011). “international graduate students’ difficulties: graduate classes as a community of practices.” teaching in higher education 16 (3): 281–92. doi:10.1080/13562517.2010.524922. lincoln, douglas j. (2008). “teaching with clickers in the large-size principles of marketing class.” marketing education review 18 (1): 39–45. marrone, mauricio, taylor, murray and hammerle, mara. (2016). "who is active in active learning: a systematic literature review of business literature.” academy of taiwan business management review, 12 (2), 63-73 modell, sven. (2005). “triangulation between case study and survey methods in management accounting research: an assessment of validity implications.” management accounting research 16 (2): 231–54. doi:10.1016/j.mar.2005.03.001. montgomery, catherine. (2009). “a decade of internationalisation: has it influenced students’ views of cross-cultural group work at university?” journal of studies in international education, january. doi:10.1177/1028315308329790. murphy, a., farley, h., lane, m., hafeez-baig, a. and carter, b. (2014). "mobile learning anytime, anywhere: what are our students doing?" australasian journal of information systems, 18(3). neri, frank, and simon ville. (2008). “social capital renewal and the academic performance of international students in australia.” the journal of socio-economics 37 (4): 1515– 38. doi:10.1016/j.socec.2007.03.010. oecd. 2015. “education at a glance (2015): oecd indicators.” oecd publishing. http://dx.doi.org/10.1787/eag-2015-en. oliver, gary r., and rodney coyte. (2011). “engendering learning engagement in a diverse cohort: a reflection.” accounting research journal 24 (2): 195–204. doi:10.1108/10309611111163727. opdecam, evelien, and patricia everaert. (2012). “improving student satisfaction in a firstyear undergraduate accounting course by team learning.” issues in accounting education 27 (1): 53–82. doi:10.2308/iace-10217. pohl, alexander, vera gehlen‐baum, and françois bry. (2011). “introducing backstage – a digital backchannel for large class lecturesnull.” interactive technology and smart education 8 (3): 186–200. doi:10.1108/17415651111165410. pratt, michael g. (2008). "fitting oval pegs into round holes tensions in evaluating and publishing qualitative research in top-tier north american journals." organizational research methods 11 (3): 481-509. ———. (2009). "from the editors: for the lack of a boilerplate: tips on writing up (and reviewing) qualitative research." academy of management journal 52 (5): 856-862. prince, michael. (2004). “does active learning work? a review of the research.” http://www.rlillo.educsalud.cl/capac_docente_becadosaps/metodologias/aprendiza je%20activo%20prince_2004.pdf. pye, g., holt, d., salzman, s., bellucci, e. and lombardi, l. (2015). "engaging diverse student audiences in contemporary blended learning environments in australian higher business education: implications for design and practice." australasian journal of information systems, 19. reinig, bruce a., ira horowitz, and gerald e. whittenburg. (2011). “the effect of team-based learning on student attitudes and satisfaction.” decision sciences journal of innovative education 9 (1): 27–47. doi:10.1111/j.1540-4609.2010.00289.x. australasian journal of information systems marrone, taylor & hammerle 2018, vol 21, research on educational technologies international students & active learning 19 richards, deborah, and mauricio marrone. (2014). "identifying the education needs of the business analyst: an australian study." australasian journal of information systems 18.2. rienties, bart, núria hernández nanclares, divya jindal-snape, and peter alcott. (2012). “the role of cultural background and team divisions in developing social learning relations in the classroom.” journal of studies in international education, november, 1028315312463826. doi:10.1177/1028315312463826. riley, j. and ward, k., (2015). passive learning methods in an accounting information systems course. issues in accounting education. rovai, alfred p., and kirk t. barnum. (2007). “on-line course effectiveness: an analysis of student interactions and perceptions of learning.” international journal of e-learning & distance education 18 (1): 57–73. rubin, robert s. (2006). “the academic journal review process as a framework for student developmental peer feedback.” journal of management education 30 (2): 378–98. doi:10.1177/1052562905277185. sandy q. qu, and john dumay. (2011). “the qualitative research interviewnull.” qualitative research in accounting & management 8 (3): 238–64. doi:10.1108/11766091111162070. savin-baden, maggi, and claire howell major. (2004). foundations of problem-based learning. mcgraw-hill education (uk). sawir, erlenawati. (2005). “language difficulties of international students in australia: the effects of prior learning experience.” international education journal 6 (5): 567–80. sawir, erlenawati, simon marginson, ana deumert, chris nyland, and gaby ramia. (2008). “loneliness and international students: an australian study.” journal of studies in international education 12 (2): 148–80. doi:10.1177/1028315307299699. schreiber, lisa m., and brielle elise valle. (2013). “social constructivist teaching strategies in the small group classroom.” small group research, may, 1046496413488422. doi:10.1177/1046496413488422. smith, c. veronica, and leeann cardaciotto. (2011). “is active learning like broccoli? student perceptions of active learning in large lecture classes.” journal of the scholarship of teaching & learning 11 (1): 53–61. stacey, elizabeth. (2007). “collaborative learning in an online environment.” international journal of e-learning & distance education 14 (2): 14–33. strauss, anselm, and juliet corbin. (1990). basics of qualitative research. vol. 15. newbury park, ca: sage. tan, john, diane c. satin, and christopher w. k. lubwama. (2013). “a real-world business approach to teaching m.b.a. managerial accounting: motivation, design, and implementation.” issues in accounting education 28 (2): 375–402. doi:10.2308/iace50393. taylor, jennyfer lawrence, jessica tsimeris, xuanying zhu, duncan stevenson, and tom gedeon. (2015). “observations from teaching hci to chinese students in australia.” in proceedings of the asean chi symposium’15, 31–35. asean chi symposium’15. new york, ny, usa: acm. doi:10.1145/2776888.2780366. treloar, carla, natalie mccall, isobel rolfe, sallie-anne pearson, gail garvey, and andrew heathcote. (2000). “factors affecting progress of australian and international students in a problem-based learning medical course.” medical education 34 (9): 708–15. doi:10.1046/j.1365-2923.2000.00625.x. australasian journal of information systems marrone, taylor & hammerle 2018, vol 21, research on educational technologies international students & active learning 20 vygotsky, l. s. (1980). mind in society: the development of higher psychological processes. harvard university press. welsh, elaine. (2002). “dealing with data: using nvivo in the qualitative data analysis process.” forum qualitative sozialforschung / forum: qualitative social research 3 (2). http://www.qualitative-research.net/index.php/fqs/article/view/865. wilson, steffen, and jonathan gore. (2013). “an attachment model of university connectedness.” the journal of experimental education 81 (2): 178–98. doi:10.1080/00220973.2012.699902. wong, joseph kee-kuok. (2004). “are the learning styles of asian international students culturally or contextually based?” international education journal 4 (4): 154–66. woo, younghee, and thomas c. reeves. (2007). “meaningful interaction in web-based learning: a social constructivist interpretation.” the internet and higher education, special section of the aera education and world wide web special interest group (edweb/sig), 10 (1): 15–25. doi:10.1016/j.iheduc.2006.10.005. wu, yun, and chetan s. sankar. (2013). “impact of hands-on research experience on students’ learning in an introductory management information system course.” decision sciences journal of innovative education 11 (4): 335–58. doi:10.1111/dsji.12017. yin, robert k. (2013). case study research: design and methods: design and methods. 5th ed. thousand oaks, california: sage publications. copyright: © 2017 marrone, taylor & hammerle. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 literature review 3 case setting 4 research approach 4.1 surveys 4.2 interviews 4.3 analysis 5 findings 5.1 overall findings from surveys 5.2 survey perceptions of international students concerning active learning 5.3 focus group perceptions from international and domestic interviewees comparing active and passive learning 5.4 focus group perceptions from international interviewees discussing benefits of active learning 6 discussion 7 conclusion ajis vol 4 no. 1 information systems success in small and medium enterprises: an evaluation by australian public accountants dieter fink school of management information systems edith cowan university pearson street churchlandswa6018 australia abstract traditionally, the success of information systems (is) has been studied in the context of large organisations with a focus on systems that have been developed inhouse. most businesses, however, are small and medium enterprises (smes), and they have increasingly adopted packaged application software. to evaluate their success we examined two research variables, namely user practices and design features. the evaluation was done by public accountants who deal with smes as clients and are most likely to have well-developed opinions on the use of packaged software. in general, they rated small/medium accounting software packages as satisfactory with some notable exceptions. the installation phase of packages is not very well managed. packages appear to lack a number of important control features which, considering the lack of internal is expertise in smes, could result in malfunctions of their accounting systems. the study identifies better documentation and user education as ways to overcome perceived deficiencies. introduction traditionally, the success of information systems (is) has been studied in the context of large organisations with a focus on systems that have been developed inhouse. most businesses, however, are small and medium enterprises (smes), and they have increasingly adopted packaged application software to meet their information processing requirements. furthermore, due to the small number of employees, smes depend on outside support for the success of their is activities. studies in the small/medium packaged systems domain are now emerging. for example, gray (1991) investigated the incongruences between accounting professionals when assisting small organisations in the selection of accounting information systems (ais). ponemon and nagoda (1990) investigated the perceptions of team members during packaged ais implementation. our study extends this research by investigating user practices and design features of installed small/medium packaged accounting software. at the commencement of our research we believed, as did nazem (1990), "that some sense of maturity with respect to the use of computers is emerging in small businesses" (p. 100) and that it is time to evaluate the use and design of packages that have been operating for a while. we chose public accountants acting as advisors to smes to perform this evaluation. the study's primary aim was to provide feedback of contemporary adequate and inadequate user practices and design features to small/medium accounting package users and developers. background literature for the present study, three areas of is research are particularly relevant. they are information systems success, the use and design of small/medium accounting packages, and public accountants involvement with packaged software. the notion of information systems success interest in what constitutes the "success" of information systems has mainly focused on large computer systems. typically, factor research is carried out which "deals with the influence of static organisational abstractions and technical system quality upon successful implementation." (cerveny and sanders, 1986, p. 192) during the 1980s research focused on the criteria of system quality, information quality, use, user satisfaction, individual impact and organisational impact (delone and mclean, 1992). in recent years, is success variables have included system impact on strategic direction, the integration of the is plan with the corporate plan, the quality of information outputs, contribution to organisational performance, and the identification and integration of new technologies (saunders and jones, 1992). studies in the smaller systems area have found factors determining is success to include consultant effectiveness, vendor support, is experience, sufficiency of financial resources, ceo support, and user participation (yap et al, 1992). furthermore, there is evidence that information technology (it) growth in small-firm computing is strongly motivated by the owner's enthusiasm towards computing and inhibited by lack 32 ajis vol 4 no. 1 of is knowledge, lack of managerial time, poor support, and limited financial resources (cragg and king, 1993). it is generally acknowledged that the is success notion is difficult to operationalise. the key reason for this is that over time measures have changed to reflect first, changes in it and second, differing expectations of the role of is within organisations. formal measurement frameworks that have emerged in recent years include the is success model (delone and mclean, 1992), the is function performance evaluation model (saunders and jones, 1992) and the goal/system resource model (sethi et al, 1993). the use of different measurement frameworks make comparisons between studies difficult, if not impossible, and render findings ambiguous and confounding. sethi et al (1993) concluded that the literature has not yet build a critical mass of research in the area. further developments and refinements of measures of is performance are needed according to saunders and jones (1992) before is success can be measured precisely. our study adopted the model of successful is implementation provided by cerveny and sanders (1986). this model identifies factors that lead to the improved quality of implementation which contributes to successful implementation outcomes. we examined the left side of the model, namely designer characteristics (in the form of package design features) and user characteristics (in the form of package user practices). user practices and design features of accounting packages whilst computer hardware is becoming reliable and mature, the software industry remains volatile. this is a potential concern to smes who rely heavily on off-the-shelf rather than inhouse developed software, particularly for basic accounting applications (nazeem, 1990). the opportunity for modification is usually limited in extent and to the time of implementation since in order to protect future sales the vendor rarely sells the source code for the program. "therefore, with less opportunity to modify acquired software, the potential for failures may be heightened for smaller organizations." (gray, 1991, p. 19) once prewritten application software is operational, smes may have to be satisfied with the design features of packages provided for them. yet, nazeem (1990) found that off-the-shelf software users in small businesses were satisfied with the flexibility of their systems and their ability to expand it. opinions regarding performance and reliability of application software were also favourable. it appears therefore that packaged software can be flexible and, mostly through the setting of parameters, is capable of being configured to more closely suit the needs of users. whereas design features are largely given, user practices are determined by the enterprise's internal practices and preferences. nazeem (1990) found that users of packaged software in small business were familiar with their systems and achieving competence, and appeared to be satisfied with the security offered by packages. those that were concerned with security tended to develop their own inhouse is. it appears that in small business particular importance is placed on the software attributes of performance and audit trail (gray, 1991). potential support offered by public accountants the 1980s provided smes with the opportunity to introduce low-cost computer hardware and packaged systems. the following years yielded "a picture of growth and stagnation in small-firm computing. while many firms had experienced growth in the number and type of it applications, there had been little change with respect to the management of it in small firms." (cragg and king, 1993, p. 57) the lack of internal expertise points to the need to use external support from vendors and consultants, including public accountants, to ensure the continued effectiveness of is. package suppliers appear to have recognised this by providing interface functions within packages that allow third parties to develop specialised modules to meet particular functional and industry requirements. in yap et al's (1992) study, 53% of the small singaporean businesses in the sample engaged consultants to assist in implementation and their support significantly affected information systems success. the contribution of consultants is dependent on the quality of the consultant. clients in yap et al's (1992) study described an ineffective consultant as lacking specific experience, not considering future expansion, seeming unaware of actual user requirements, and ignoring specific aspects of the business. when consultants are involved in implementation they need to have agreement among themselves since it has been found that a strong relationship exists between incongruences within the implementation team and is failures (ponemon and nagoda, 1990). in gray's (1991) study a large number of accounting professionals had both user and advisor experiences. this led to the conclusion that in the small-organisation domain, "because of the widespread distribution of microcomputers, advisors may tend to be users of their own ais. this may indicate that small ais advisors will be more sensitive to the users' needs than large ais advisors." (p 32) however, there are indications of a 33 ajis vol 4 no. 1 negative attitude by small firms towards consultants. a recent study found that small firms appear to have poor relationships with it specialists and also show little desire for the situation to change (cragg and king, 1993). the study our research took the approach that public accountants who deal with smes as clients in the capacity of auditors, accountants, advisors or consultants are most likely to have well-developed opinions on the use of small/medium pre-written packaged accounting software. we were influenced in this approach by the finding that the majority of small firms rely on external support to meet their it requirements (yap et al, 1992) and that it management in small firms is not sophisticated (cragg and king, 1993). furthermore, it appears that many advisors have experience in small-business computing themselves (gray, 1991). we selected two variables of small/medium accounting packages, namely user practices and design features, for our research. the variables play an important part in cervany and sanders' (1986) model of successful is implementation discussed in an earlier section. below follows a discussion of the research process and associated validity and reliability considerations. research design and conduct we found that the selection of the attributes against which the two research variables could be evaluated provided many choices. as stated by gray (1991) "selecting the appropriate attribute subset becomes a significant decision in itself (p. 21). zeleny (1982), quoted in gray, 1991) defines the requirement of attributes as: attributes-descriptors of objective reality. they may be actual objectives traits, or they may be subjectively assigned traits, but they must be perceived as characteristics of the object in the "outside" world. we decided to maintain symmetry between the two main research attributes, user practices and design features, and not to make the questionnaire so lengthy that it would cause a lack of response. consequently, the user practices section contained six subsets and design features section seven subsets. within each of these subsets (or categories) are three items except for two subsets with four items. altogether user practices consisted of 20 items and design features of 21 items. details of categories and items compiled by us are shown in tables 3 and 4. the construction of the questionnaire was validated externally as discussed later in the article under validity and reliability testing. the research was supported by the institute of chartered accountants in australia (icaa). together with the institute's officials we selected the members to be invited to participate in the study on the basis of their fields of interest, as contained in the membership data base of the icaa. the interest fields selected were auditinformation systems & control; auditexternal; business services; computer services; information technologyconsulting; and small business. the questionnaire was sent to chartered accountants in public practice with the position of manager and one of the above fields of interest. the importance of the project was conveyed in the accompanying covering letter which was written by the icaa. the covering letter stated the desirability of the research, namely that it would assist to establish and maintain the icaa's "leadership position on the application of it to business". we selected managers as subjects because they typically are required to maintain a "hands-on" relationship with clients whilst maintaining an overall perspective of the engagement. they were requested to pass the survey to an appropriate person within their firm should they be unable to complete the survey themselves. the member data base of the icaa identified 470 managers with the relevant fields of interest. of those, 53 were overseas, leaving 417 recipients of the questionnaire. altogether 135 questionnaires (32.1%) were returned. of those 23 questionnaires were unsatisfactorily completed and rejected for further processing. this left 112 useable responses yielding a response rate of 26.8%. a breakdown of the responses by job function identified four major groupings: management consultants, business advisors, auditors, and accounting services staff. they accounted for 103 of the 112 responses available for processing. the remaining two groupings, taxation advisory and business recovery, contained only three responses each, whilst the job functions of a further three responses were not specified and are therefore unknown. table 1 provides demographic data for both the total responses (n=112) and for the four major groupings (n=103). 34 ajis vol 4 no. 1 table 1: demographic data of respondents n % n % job functions management consulting business advisory taxation advisory auditing accounting services business recovery unknown total positions held partner director manager senior junior other total age distribution 20-25 26-35 36-45 46 55 total state new south wales victoria queensland western australia south australia tasmania northern territory overseas total 21 33 3 35 14 3 3 112 22 9 72 6 1 2 112 7 74 25 6 112 44 25 17 12 8 3 1 2 112 18.7 29.5 2.7 31.2 12.5 2.7 2.7 100.0 19.6 8.0 64.3 5.4 .9 1.8 100.0 6.3 66.0 22.3 5.4 100.0 39.3 22.3 15.2 10.7 7.1 2.7 .9 1.8 100.0 21 33 35 14 103 19 8 67 6 1 2 103 6 69 22 6 103 42 22 16 11 6 3 1 2 103 20.4 32.0 34.0 13.6 100.0 18.4 7.8 65.0 ' 5.8 1.0 2.0 100.0 5.8 67.0 21.4 5.8 100.0 40.8 21.3 15.5 10.7 5.8 2.9 1.0 2.0 100.0 as seen from the above table, most participants in the study described themselves at the manager level, they were most often in the age bracket 26-35 years, and new south wales was the most common location of respondents. validity and reliability we sought construct validity of our questionnaire by requesting members of the it chapter of the icaa to review of our draft questionnaire. based on the comments received, the questionnaire was revised in two main areas. first, greater clarification of the objectives of the study and the reason why public accountants were surveyed was made in the introductory section. second, the terminology in various sections was improved. the final version of the questionnaire stated the purpose of the research, the support provided by the icaa, collected information on subjects' backgrounds, and captured opinions on the user practices and design features with regard to small/medium accounting packages. the items in the questionnaire were mostly phrased "positively", i.e. expressing desirable user practices and design features. public accountants were asked to indicate whether they agreed or disagreed that these existed among their clients of small/medium package users. the exceptions were two questions stated in the negative form but reversed for statistical analysis. respondents expressed their opinion on the propositions on a scale of 1 (disagree) to 7 (agree). internal consistency of the questionnaire responses, i.e. the extent to which all of the subparts of an instrument or scale measure the same characteristics, was established through the cronbach-alpha technique. table 2 35 aj1s vol 4 no. 1 shows the alpha value for each category in the questionnaire. some concern is reflected in the internal groupings with alpha values below 0.80, the generally regarded satisfactory level (sekaran, 1984). as seen, the category "accounting principles" showed an unsatisfactory alpha value. as a further measure of internal consistency, the inter-item correlation coefficients of the seven multiple-item categories were calculated. all the correlation coefficients with the exception of the "accounting principles" category were positive and significant at the 0.001 level. table 2: internal consistency tests category user practices acquisition processing & reporting security reconciliation installation understanding category total design features accounting principles processing & reporting audit trail access control data control documentation installation category total survey total number items 4 4 3 3 3 3 20 3 3 3 3 3 3 3 21 41 of alpha value 0.64 0.58 0.60 0.80 0.58 0.36 -0.18 0.72 0.70 0.69 0.68 0.67 0.47 correlation coefficient 0.31* 0.26* 0.35* 0.58* 0.30* 0.16* -0.06 0.48* 0.43* 0.44* 0.41 * 0.40* 0.23* * (p<.001) since four job functions, namely management consulting, business advisory, auditing, accounting services were the main contributors to the study (see table 1), it was decided to carry out statistical analysis on the responses received from these groupings. furthermore, concern over the internal consistency of the category "accounting principles" (see table 2) caused us to exclude this category from further consideration. below follow the results and discussion of our data analysis. results and discussion to recap, in our study we asked public accountants to indicate whether they agreed or disagreed that certain user practices and design features existed among those of their clients that were small/medium package users. respondents expressed their opinions on propositions on a scale of 1 (disagree) to 7 (agree). user practices table 3 shows the mean and standard deviation (sd) for each item and each category of user practices. categories and items within categories are ranked from the highest to the lowest means. 36 ajis vol 4 no. 1 table 3: means and standard deviations for user practices mean sd installation outside qualified support is required for the installation of packages and software updates. system parameters are properly set up. working papers show that conversion from the old to the new system was properly carried out. reconciliations ledgers, sub ledgers and accounts are regularly reconciled. a formal procedure (e.g. checklists, reports) exists for period-end processing. adequate documentation is retained as evidence that reconciliations have been done. security data and programs are secured through regular and systematic backups. access to the system is controlled, for example through the use of passwords. master files are regularly checked for anomalies, for example account balances are printed and verified. processing and reporting transactions entered into the system are supported by adequate 4.49 5.92 4.30 3.26 4.48 4.82 4.38 4.23 4.38 4.91 4.61 3.63 4.29 4.72 1.34 1.02 1.42 1.59 1.47 1.51 1.43 1.48 1.61 1.40 1.76 1.69 1.50 1.36 documentation. reports are produced to support management decision-making. 4.67 1.44 input editing programs are used to ensure only valid data enter the system. 3.97 1.58 the nature of input and usage of data is defined in a data dictionary or 3.84 1.63 procedure manual. acquisition 3.92 1.53 packages are acquired through an evaluation and selection process (e.g. 4.24 1.46 request for proposal). users depend on modifications and customisation to packages to ensure a 4.05 1.63 satisfactory fit. users are involved in establishing package requirements. 4.04 1.61 users have the knowledge and skills to participate in the evaluation and 3.33 1.45 selection of packages. understanding 3.51 1.43 users have ongoing access to training to overcome lack of package 3.71 1.46 understanding. users are able to manage packages' operations and reporting because of 3.50 1.42 package understanding. users receive adequate training before packages are installed. 3.32 1.41 of the six user practice categories, the means of four categories were above the scale midpoint of 4 and the means of two categories fell below, indicating that user practices were generally regarded satisfactory. the highest rated category was the installation of packages. however, when examining the items within the category, concern arises as to the capability of smes to install packages themselves. two items attract attention; they were rated the highest and the lowest of all items respectively. the items are: • outside qualified support is required for the installation of packages and software updates (highest mean of 5.92). • working papers show that conversion from the old to the new system was properly carried out (lowest mean of 3.26). 37 ajis vol 4 no. 1 three user practice categories, namely reconciliation, security, and processing and reporting, were judged to be carried out satisfactorily. there is however one concern among security practices, indicated by a low mean rating, namely • master files are regularly checked for anomalies, for example account balances are printed and verified (mean 3.63). the remaining user practice categories are rated below 4, the scale midpoint. they are acquisition and understanding. within the former, particular concern exists for the following item: • users have the knowledge and skills to participate in the evaluation and selection of packages (mean 3.33). this lack of knowledge and skill is reflected in the category of understanding where two items are lowly rated: • users are able to manage packages' operations and reporting because of package understanding (mean 3.50). • users receive adequate training before packages are installed (mean 3.32). from analysis of responses it can be concluded that the installation phase of packages could pose problems within smes and a major reason for this appears to be the lack of user understanding. the lack of understanding manifests itself in inadequate training prior to the acquisition of packages which may impede smes' participation in the evaluation and selection of packages and subsequently not understanding packages' operations and reporting functions. the findings confirmed pervious studies, reviewed in an earlier section, that smes lack internal is expertise and depend on outside support in this respect. design features table 4 provides means and standard deviations (sd) for design feature responses; categories and items are ranked in descending order of means. three of the six design feature categories were rated above the scale midpoint of 4. the features offered for processing and reporting, access control and installation appear to exist in a satisfactory manner among smes with the exception of two items, namely • packages are not complex enough to require the support of vendors during installation (mean 2.90). • a trail is kept of system users, their identity and activities (mean 3.36). the first item relates to the installation of packages and may be one of the major reasons why smes rely on outside support as reported earlier. the remaining design feature categories, audit trail, documentation and data control, were all rated below the scale midpoint. the items with the highest concerns indicate that packages appear to lack adequate control features. control appears to be deficient in the following areas: • error messages provide explanations of errors and corrective actions required (mean 3.53). • the system forces the printout or archiving of transactions before they are purged from the system (mean 3.53). • inbuilt controls alert users to reconcile sub ledgers with control accounts (mean 3.55). • packages do not permit period-end procedures to be completed until all required reports are produced (mean 3.56). from the analysis of responses it can be concluded that smes find packages complex, at least in the opinion of public accountants, and hence smes require outside support. furthermore, packages appear to lack a number of important control features which, considering the lack of internal is/it expertise in smes, could result in malfunctions of their accounting information processing systems. 38 ajis vol 4 no. 1 table 4: means and standard deviations for design features mean sd processing and reporting data input provides space to store transaction descriptions and crossreferences. details stored in ledgers and sub ledgers enable transactions to be traced. reports produced by packages support management decision-making. access control packages contain access control mechanisms such as password control. access can be controlled to enable different persons varying degrees of access to data and functions. a trail is kept of system users, their identity and activities. installation packages can be installed in modules at different intervals. package modules can easily be integrated with modules installed earlier. packages are not complex enough to require the support of vendors during installation. audit trail changes to master files are recorded in audit trails. audit trail details are printed to allow reference to supporting documentation. packages do not permit period-end procedures to be completed until all required reports are produced. documentation documentation (e.g. on conversion) support the installation of packages. the provided documentation support the effective use of packages. error messages provide explanations of errors and corrective actions required. data control warnings are provided before importing master or transaction files over existing ones. inbuilt controls alert users to reconcile sub ledgers with control accounts. the system forces the printout or archiving of transactions before they are purged from the system. 4.88 5.07 5.05 4.53 4.52 5.27 4.96 3.36 4.17 5.15 4.55 2.90 3.90 4.14 3.99 3.56 3.89 4.08 4.05 3.53 3.83 4.44 3.55 3.53 1.31 1.26 1.21 1.47 1.43 1.19 1.43 1.67 7.35 1.23 1.46 1.38 7.52 1.49 1.53 1.54 1.39 1.41 1.35 1.43 1.59 1.56 1.69 1.54 relationship within and between user practices and design features to establish the relationship of the six categories with each other, correlation coefficients were computed in three areas; they are shown in tables 5, 6 and 7. table 5: correlations within user practice categories (1) (2) (3) (4) (5) (6) (1) acquisition (2) processing&reporting (3) security (4) reconciliation (5) installation (6) understanding ***p<.001 **p<.01 1 .34** .33** .42*** .32** .34*** * p < .05 1 .47*** .53*** .56*** .34*** 1 .48*** .48*** .24* 1 .61*** .35*** 1 .36*** 1 39 ajis vol 4 no. 1 table 6: correlations within design feature categories (7) (8) (9) (10) (11) (12) (7) processing&reporting (8) audit trail (9) access control (10) data control (11) documentation (12) installation 1 .39*** .32** .24* .47*** .06 1 .39*** .54*** .28** .10 1 .41*** .25* .06 1 .44*** .30** 1 .33** 1 p<.001 **p<.01 *p<.05 table 7: correlations between user practice and design feature categories user practices. /design features (7) processing&reporting (8) audit trail (9) access control (10) data control (1 1 documentation (12) installation acquisi tion(l) .14 .12 .09 .22* .20* -.04 proc. & rep. (2) .41*** .21* .25* -.01 .51*** .08 security (3) .33** .37*** .39*** .20* .36*** -.07 reconcilia tion(4) .43*** .21* .17 .17 .42*** -.05 installatio n(5) .46*** .30** .35*** .24* .53*** .20* understand ing(6) .13 .12 .21* .22* .38*** .16 ***p<.001 **p<.01 *p<.05 user practice categories are all significantly correlated with each other (table 5); strong correlations also existed among design feature categories with the exception of installation which only correlated with data control and documentation (table 6). when correlating user practices and design features many strong relationships can be observed (table 7). figure 1 shows the correlations that were found between user practices and design features at the 0.01 confidence level. the correlation coefficients between user practice and design feature categories shown on figure 1 are all positive, i.e. they vary directly with one another. most of the user practice categories are strongly correlated with the documentation of packages. the exception is the user practice of acquisition which does not appear on the diagram. also missing from the diagram are the design feature categories of data control and installation as they do not correlate strongly with any of the user practice categories. implications public accountants are in a good position to form judgements of the use and design of packages used by their sme clients and as such are able to provide valuable input into the what can be expected from these systems. from their opinions flow a number of implications for the management of smes and the developers of accounting packages. implications for sme management in general, public accountants rated small/medium accounting software packages as satisfactory in the majority of user practices and design features. the findings indicate that one of the key activities of small business computing, namely processing and reporting (see gray, 1991), appears to be well handled. similarly, reconciliations appear to be carried out satisfactorily within smes. another pleasing finding is that security is exercised by users; they are supported in this respect by package features that control access to the system and to a lesser extent by features that provide control over data. by drawing the attention, however, to those areas where concerns exist that sme management can implement action to improve the success of their packaged is. first, smes need to recognise, if they have not already done so, that at present they do not appear to possess the internal expertise to install packaged software. public accountants perceive a level of complexity within packages beyond their capabilities. this indicates, as did previous studies in the small/medium systems domain, that support from vendors is required. in addition, better working papers showing details of the conversion from the old to the new system need to be kept. 40 ajis vol 4 no. 1 figure 1: significant correlations between user practice and design feature categories design features user practices design features .41 processing & reporting access control processing & reporting .51 reconciliation security installation understanding documentation audit trail second, sme management needs to increase users' understanding of packages. the study indicates that users should receive knowledge and skills that would enable them to participate in the evaluation and selection of packages. furthermore, training before the packages are installed would increase their understanding and thereby their utilisation of the processing and reporting functions available in packages. education of users would lessen the reliance placed on outside support. the third area of concern is the lack of control features that were identified in packages. these related to inadequate explanations of errors and corrective actions required, obtaining a printout of transactions to be purged, alerting users when ledgers are not reconciled, and ensuring completion of period-end procedures. these apparent shortcomings should be addressed in conjunction with the package providers as discussed below. implications for the developers of packaged accounting systems it was not the intention of this study to unearth specific deficiencies with specific packages but to gain a general understanding of the perceived strengths and weakness of packaged accounting systems. package developers should therefore review their products and, if required, aim to improve the shortcomings of their particular product. first, a number of package control features appear to be deficient. these were identified in the preceding section and should be overcome in conjunction with the management of smes. second, packages should offer design features that give greater support to users. the documentation of packages appears to play an important role in this regard since documentation is strongly correlated with most user practices. public accountants did not rate the documentation that currently exist very high and scope therefore exists to make improvements in this area. particular concern related to documentation that supports the effective use of packages, viz. documentation (e.g. on conversion) to support the installation of packages, to provide error messages that explain errors and corrective actions required, and to produce a trail of system users. 41 ajis vol 4 no. 1 third, smes apparently regard accounting packages to be complex. vendors could reduce complexity by offering training thereby increasing their customers' understanding of their products. this should commence prior to the acquisition of the package to enable users to participate in the evaluation and selection processes and be ongoing to enable them to maximise packages' processing and reporting functions. conclusion research to determine the success of information systems is still agonising over the dimensions of success and as yet there is no instrument which can satisfy the requirements of all studies. furthermore, research into the success of small business is has only recently emerged. this study used two research variables, namely user practices and design features. they form part of cerveny and sanders' (1986) model to measure is success and were selected because of their relevance to packaged accounting software which is the predominant form of is in smes. rather than canvass the opinions of smes or package vendors, we approached public accountants because of their independence and experience with smes. whilst packages overall appear to be used and designed satisfactorily, a number of concerns were established. management and package developers should take notice of current shortcomings in order to increase the success of information systems within smes. acknowledgment the study was financially supported by the institute of chartered accountants in australia and jointly carried out with ken stevens of the australian national university. references cerveny, r.p. & sanders, g.l. (1986) implementation and structural variable, information & management, vol 11, pp 191-198. cragg, p.b. & king, m. (1993) small-firm computing: motivators and inhibitors, mis quarterly, march, pp 47-60. delone, w.h. & mclean, e.r. (1992) information systems success: the quest for the dependent variable, information systems research, vol 3, iss 1, pp 60-95. gray, g.l. (1991) accounting information system selection in small organizations: incongruences between accounting professionals, journal of information systems, spring, pp 17-35. nazem, s.m. (1990) sources of software and levels of satisfaction for small business computer applications, information & management, vol 19, pp 95-100. ponemon, l.a. & nagoda ii, r. j. (1990) perceptual variation and the implementation of accounting information systems: an empirical investigation, journal of information systems, spring, pp 1-14. saunders, c.s. & jones, j.w. (1992) measuring performance of the information systems function, journal of management information systems, vol 8, iss 4, pp 63-82. sekaran, u. (1984) research methods for managers a skill-building approach, wiley. sethi, v., hwang, k.t. & pegels, c. (1993) information technology and organizational performance, information & management, vol 25, pp 193-205. yap, c.s., soh, c.p.p. & raman, k.s. (1992) information systems success factors in small business, omega, vol 20, iss 5/6, pp 597-609. zeleny, m. (1982), multiple criteria decision making, new york, mcgraw-hill. 42 microsoft word ajis_18(3)_v10.docx australasian journal of information systems volume 18 number 3 2014 303 the perceived impact of the agile development and project management method scrum on information systems and software development productivity karlheinz kautz faculty of business, university of wollongong, wollongong nsw 2522, australia kautz@uow.edu.au thomas heide johansen progressive as, dk-2730 herlev, denmark thj@progressive.dk andreas uldahl ernst & young denmark, dk-2860 søborg, denmark andreas.uldahl@dk.ey.com abstract this research contributes to the body of knowledge in information systems development (isd) with an empirical investigation in form of a case study that demonstrates the positive impact of the agile development and project management method scrum on information systems and software development productivity and it provides a useful operationalization of the concept through seven identified indicators for productivity. despite the fact that the case unit had challenges with the use of scrum, the indicators identified the areas where the company had managed to exploit the potential of scrum and its practices with regard to increasing productivity. the research results are discussed both with regard to the existing scrum literature as well as to complex adaptive systems (cas) as a foundation for isd and agile development. introduction over the last decade agile information systems and software development has received much attention from researchers and practitioners as an approach for dealing with change and the unpredictable and hardly controllable elements of isd in a dynamic environment. while numerous publications claim a positive impact of agile development and in particular scrum on information systems and software development, only little empirical work exists to verify these claims. the literature review, which was part of the study reported here uncovered some notable exceptions. to further contribute to this body of knowledge we set out to answer the following two research questions: what impact has the introduction of the agile development and project management method scrum on information systems and software development? what is the effect of any deviations from the guidelines for scrum? the results we present in the following are part of a larger project where we developed a framework for investigating the impact of scrum (see johansen & uldahl 2012). in this paper however we concentrate on one of these concepts, namely scrum’s impact on productivity in information systems and software development. in the remainder of the paper we first briefly introduce scrum, and then we describe our theoretical background and the research setting and method. subsequently we present and discuss our findings against the existing literature on scrum and relate them to complex adaptive systems (cas) theory, a theory which is considered to provide a theoretical foundation for isd (kautz 2012) and in australasian journal of information systems volume 18 number 3 2014 304 particular agile development (highsmith 2002). we finish with some conclusions and an outlook to future research. scrum – an agile development and project management method scrum is an agile information systems and software development method with a strong focus on project management, which was formalized and tested by schwaber and sutherland in the mid 1990ties (schwaber & beedle 2002, schwaber 2004). scrum focuses on an iterative and nimble development process, on transparency, visibility and on cooperation in and between the development team and the customers. in scrum the development team is called the scrum team. unlike traditional development projects where analysts, developers and testers are typically separated, scrum teams are built on an interdisciplinary basis and comprise all these roles in one team preferably in one physical location. this structure, as well as scrum’s focus on self-organization aims at creating team dynamics and a better understanding of the tasks to be performed jointly. in this context the role of the product owner has the responsibility to represent the project and product externally to other stakeholders and customers and to handle and manage the tasks that appear in the product and release backlogs (see below) (schwaber & beedle 2002). internally, the role of the scrum master will provide leadership, motivate and facilitate the team in line with the scrum values, practices and development process. a scrum development process is structured through a product backlog, which is a prioritized list of required business and technical functions of the envisioned product. it might change in line with the customer's new needs. a release backlog is a prioritized subset of the total product backlog and defines the functions to be included in a release. a scrum, performed in so-called sprints, is a set of development tasks and processes which a scrum team carries out to achieve a given sprint goal. the length of a sprint is predefined. it typically lasts between 5 and 30 calendar days (schwaber & beedle 2002). what needs to be done during a sprint is determined by a prioritized sprint backlog, which is determined together with a sprint goal before the start of each sprint by the team and scrum master and others, if necessary, at a planning meeting. throughout a project a burn-down chart shows the amount of work left to do versus time over a given period (schwaber 2004). in short daily scrum meetings project members briefly present what they have done during the preceding day, which tasks they take on that day, as well as any challenges and obstacles that might have prevented them from carrying out their work without any solution being discussed. scrums of scrums are additional short meetings by the scrum masters of projects, which consist of several scrum teams. at the end of a sprint a sprint review meeting takes place where the scrum team, the product owner, other management, and one or more representatives from the customer (schwaber & beedle 2002) assess the team's development process and progress in relation to the predefined sprint goal. finally the scrum team, the scrum master and possibly the product owner hold a meeting, called a retrospective, to secure learning and further improvement in the team where both the process and the product are assessed and discussed by each individual team member. literature review and theoretical background in our study we were interested in the impact of a specific method, namely scrum on information systems and software development. our literature review was therefore focused on that particular approach and not in general on project management methods’ or agile methods’ impact on information systems and software development. this limited our sources to writings which take their starting point in agile software development. we combined a concept-centric with an author-based approach (webster & watson 2002) and applied backward referencing of sources. our original search with keywords such as ’impact of scrum’, ’effect of scrum’, ’impact of scrum implementation’, and ’effect of scrum implementation’ primarily in google, google scholar and ieee sources lead to about 90 publications of which 8 dealt more precisely with our research problem. an additional 8 articles were australasian journal of information systems volume 18 number 3 2014 305 identified through the other mechanisms. from that literature we derived a number of concepts and for these concepts indicators for the impact of scrum on information systems and software development processes and projects. the resulting framework consisted of the identified, interrelated concepts productivity, quality, role of leadership, transparency, customer satisfaction, as well as employee satisfaction and a total of 38 indicators, which defined the concepts on a more detailed level. here due to page limitations we are focusing on scrum’s impact on the first one. productivity in the agile literature is an expression of the development team productivity (moe & dingsøyr 2008). there are a number of interrelated indicators that are linked to different areas that may impact on productivity. dybå & dingsøyr (2008) describe the results of a comparative case study where productivity was measured in projects driven by traditional and agile development methods based on the number of lines of code (loc) per hour, month or employee. guang-yong (2011) describes the measurement of productivity in the number of lines of code, and demonstrates how productivity increases gradually as a team becomes more self-organized and manages to review its development processes to avoid the repetition of mistakes. appelo (2010) has a different view how productivity can be measured. he highlights the increased functionality to the final product as a direct indicator of improved productivity. the way he measures the functionality is the number of story points that have been completed within a given period. a story point is a number that reflects the severity of a given task. mahnic and vrana (2007) and mahnic and zabkar (2008) define the assessment of productivity as a ratio of the added value versus the associated financial costs as well as costs associated with bug fixes. we use these sources to investigate the indicators employee performance, the time associated with fixing bugs, and repetition of the same mistakes. sutherland and altman (2010) use the term "perfect hours" as a label for a project participant’s undisturbed and uninterrupted work. they emphasize that a project’s progress and productivity should be measured by taking perfect hours combined with other indicators into account. the number of interruptions and the number of uninterrupted development hours were the two indicators we descended from these authors. moore et al. (2007) argue that increased productivity through the use of scrum is grounded on its focus on delivering functional software in short time intervals with fixed deadlines where developers do not end up in endless development cycles in an attempt to provide perfect solutions with a product that can handle everything at one time. the avoidance of continuous development cycles and compliance with deadlines are the last two indicators we derived from these authors. research setting and method we chose a case study approach to research the impact of scrum on information systems and software development processes and projects. the chosen case organization has approximately 40 years of experience in solving complex it tasks. some years ago it changed from being publically owned to private company. it has about 3,000 employees, who are involved in the development of administrative and statutory software solutions. the investigated case department falls into the latter category and has 45 employees. its sole product is a case management system for municipal job centers, which gives administrators the opportunity to work across different platforms. for the development of the case management system, the department previously followed the traditional waterfall model. in 2011 it launched the implementation of scrum as the preferred development model. at the time of our investigation, the department had completed three full releases with the use of scrum. as such the department had the profile of the unit of analysis we were looking for, an organization that had recently, within the past year, chosen to implement scrum, and that had previously used the traditional waterfall model. with the former model still in their minds we expected the employees to make candid assessments of the impact of scrum as compared to the past. as we were not able to make direct measurements, such as number of interruptions, errors, uninterrupted development hours, etc., we chose to directly ask respondents about their perceptions of australasian journal of information systems volume 18 number 3 2014 306 the given concepts. the indicators, which we had derived from the literature review, were therefore transformed into direct questions for our interviews, which we validated with 2 employees in a small pilot study before putting them to the 11 interview partners, who were available for the study. we developed 3 largely overlapping interview guides for the three stakeholder groups, with 6 developers as respondents, 4 respondents in leadership roles such as scrum master, product owner or unit managers and one representative from the service department, which is responsible for customer liaisons. all interviews were recorded, transcribed and handed over to the respondents for approval. the results of our analysis were also presented to the participants of this study and the case organization at large. the data collection with standardized interviews allowed both collections of qualitative and quantitative data. we first asked the respondents to numerically assess, on a scale from -5 to + 5, for each indicator its individual change, improvement or decline, as compared to the situation before the implementation of scrum and then to evaluate its impact on the concept in question, here productivity. after that quantitative judgment we asked into the reasons for these assessments, which provided rich qualitative data.this combination of data allowed for data and method triangulation to improve the validity of our findings (andersen 2006). the subsequent analysis was based on mean values for the quantitative data within each indicator; these were interpreted on the basis of the qualitative opinions. the results were then compared and discussed with regard to published scrum guidelines, findings from the literature, and cas theory. it is worth pointing out that the numerical element of the collected data should be considered secondary. the interviews were intended as the primary source to collect qualitative data with a statistical element and not vice versa. the quantitative data was exclusively used to create an indication and an overview over any specific area. results – scrum’s impact on productivity table1 summarizes the respondents’ assessment of scrum’s impact on productivity. despite some individual variations the respondents’ mostly positive scores indicate their favourable assessment and an improvement in productivity after the implementation of scrum. improve ment impact on productivity range of score in both dimensions no of interruptions 1.4 2.0 0 4 endless development cycles 2.8 2.8 1 5 repetition of mistakes 1.1 1.4 -1 4 compliance with deadlines 2.9 2.1 0 5 bug fixing time 0.5 1.7 -2 3 no of uninterrupted development hours 0.8 1.3 0 3 employee performance 3.5 4 3 4 table 1: mean values of scores for scrum’s impact on productivity number of interruptions the respondents largely agreed that the number of interruptions had been reduced, and that this had affected productivity in a positive direction. at the high end of the scale one developer argued as follows: "well, mainly because i refer to my scrum master now, go through her, i say. i'll do it, but it must come from the right path. if it comes from the right direction, then it is not so much an interruption, it's just a part of scrum." australasian journal of information systems volume 18 number 3 2014 307 this respondent also scored the impact on productivity high. he underlined the way scrum worked and emphasized that the number of interruptions had not necessarily decreased, but as the process had changed, he perceived the interruptions no longer as disturbances. all other respondents signalled a moderate improvement. there was broad consensus that they still were upset in their everyday routines when interruptions occurred, however, they felt that the number of interruptions was less than before. the following quote supported this; respondent a said: "yes i [a developer] would say +2, and that’s based on that there are disruptions; although we try to avoid them, they are there nevertheless. but when i think back to the previous projects i have been involved in i think there have been even more disturbances (...)." in slight disagreement another developer answered: "so, i would say 0, i think that i have neither got more nor less interruptions (...)." both respondents still decided to give the impact on productivity the same score they had given to the first dimension. the general higher score for impact was due to the individual respondents’ opinion that the benefits of fewer disturbances had lifted productivity significantly. endless development cycles the respondents generally agreed that the introduction of scrum had resulted in that they were not to the same degree than before in endless development cycles. one respondent had an interesting position on this: "yes, because i think what, among others, happened in the our tradition approach was that we developed for two months and then we found out that it was too large and complex; we did not recognize that we had bitten off more than we could chew. so there was too much work in it, we had to squash it and that happened many times, the consequence being, it had to be unpacked six months later again, or when we had the necessary resources, and then we had to start again from the very beginning with much of what had been done earlier." this statement provided evidence that the case unit had previously experienced projects that had been developed for a number of months, only to be shelved and then started all over again. this was further supported by another developer: "(...) before we got scrum, it happened many times that an entire delivery was actually thrown away (...) now we think much about that it [a sprint or sprint goal] should be clear in itself, and that the next one should also be clear in itself (...)." this respondent rated the improvement and impact as significant. both accounts point to that scrum has minimized the risk that the case unit did unnecessary development work. this had happened because, according to the respondents, the focus on delivery of functional software had increased and because the development process now was iterative and took place in short cycles. another developer was however slightly less positive, because, while he thought that there had been an improvement, he argued that there had not been any designated endless development cycles before the implementation of scrum. he said: "no, not really, but that has something to do with the fact that we have always been under a bit of time pressure. so, this endlessness where you sit and fiddle with things, it has been non-existent. we have a little history that we may have focused on providing 80% solutions. because it was more important that we came out with new functionality than that it was 100% correct. so in that way we did not produce these endless tracks." this perception, however, was unique among the respondents, but pointed to another issue, namely the delivery of less complete solutions. repetition of mistakes with regard to this indicator there was a noticeable difference in how respondents with greater responsibilities replied compared to the other respondents. we also asked respondents directly to decide australasian journal of information systems volume 18 number 3 2014 308 whether the indictor's influence was due to the scrum processes’ focus on self-organization and retrospectives. one manager, who had rated both dimensions rather high, explained why he felt that the tendency to repeat the same mistake had been reduced after the introduction of scrum: "yes. i think so. well, i think due to the fact that each individual has more responsibility one just does not make the same mistakes. that’s what i think. it has nothing to do with retrospectives, the assessments one makes, but more with the fact that the responsibility lies with the individual. i think more that's what makes that a person will remember next time, here, we have to maybe do something else." according to this respondent, the improvement had solely come about through the increased personal responsibility, i.e. through self-organization and not through learning in retrospectives. an explanation how the case unit used retrospectives was given by a developer, who saw no improvement nor any change in impact: "there is no difference because we do not use retrospectives to talk about what has been developed. de facto we use them to discuss the method, what has been good and whether there have been more or less disruptions or, if yes, whether we had a problem with that (...)." a developer, who saw a slightly negative impact on productivity although he meant that the repetitions of mistakes had been reduced, explained his choice with this argument: "well, it's because before we sat in professional groups. now a .net developer, a main framer, a tester and an analyst sit together. and we have a good team, that's not it. but there’s another dynamic, there is." overall, the result of this indicator shows that the respondents felt an improvement, which had a positive impact. the examination of this indicator also identified that the case unit only discussed the scrum method in their retrospective meetings, but not the developed product. compliance with deadlines our respondents generally agreed that there had been a clear improvement in this particular area. one of the main reasons that respondents were so positive was that scrum broke the development process down into smaller iterations and deliveries. several respondents highlighted this. for example, one developer stated: "yes, it is because it is a shorter and more manageable process, it is like that. well i would say it has become easier to meet the deadlines for when we should deliver." another developer added: "because you have a well-defined task, you have your notes, you follow up on. so that’s completely mastered. you have this 14-day sprint to do it (...)." another argument for why the impact of deadlines had increased came from a manager, who underlined that it not just had become easier to meet deadlines due to the decomposition and prioritization of tasks, which had had a great effect on productivity. he attributed the increase also to the fact that it had become easier to handle those elements and components that could be cut off at the end of the development process without much loss of functionality. this became possible because scrum prioritizes and structures its processes different than the waterfall model did. he said: "well, if you take the components of a car in the right order. if you wait to mount the wheels last, you fail to prioritize. it's the antenna, which should be mounted at the end, which we can leave out if we do not reach it (...)." it was, however, not everyone, who shared the positive majority attitude; one developer saw no change, although he felt that the work had become easier to handle; he explained: "(...) now, it's hard in a different way, because you have a sprint, so you have a deadline of 14 days instead of a deadline of 3 months. plus, now you make minor things, where before you after all sometimes built these big chunks. now it's more manageable. so on the whole i think it's the same. it's just so hard to achieve it, as it was before. " this person stood alone with his opinion, thus it did not substantially affect the overall positive picture. australasian journal of information systems volume 18 number 3 2014 309 bug fixing time the respondents were unanimous in assessing this indicator, with the exception of one developer, who chose to evaluate the first dimension quite negative (-2), but its impact as very positive (3). he argued although more time was spent on bug fixes than in the past an observation, which he interestingly enough did not refer to himself it now happened at an earlier time in the process, which had a positive effect: "(...) it may not be me personally, who spends so much more time, but i've seen developers spent more time on it, and it's probably a -2 ... well, we spend much time on bug fixes, but we do it at a better point in time now than we did, because, i feel, we used to spend much time on bugs that were registered quite late by customers; we now simply find more errors as a consequence of our scrum processes (...)." there was a broad agreement that the restructuring of the bug fixing and error correction process with short iterations and testers directly on the team had improved and that this had a positive impact on productivity. one developer gave an interesting explanation for this; he felt that productivity had increased, because the individual developers were now spared of being repeatedly and often with a delay reminded of their own errors. "(...) that way one can say that it [the new process] also helps to protect ourselves more. that we can say. the team is protected from mistakes they might have made themselves earlier in the development process." several respondents stated that it had made a big difference that there was now dedicated time specifically for bug fixes, and that it happened earlier in the process. moreover, it was also mentioned that the interaction between testers and developers within the small teams did that both parties were given an insight into the work of the other resulting in fewer errors, and ultimately less time spent on bug fixes. number of uninterrupted development hours there was much agreement about this indicator across all respondents. the slight improvement was explained with that there were more meetings now, but that the meetings had become better to support the teams in handling challenges. these led to higher scores for the indicator’s impact on productivity. the respondents stated that they were now working more efficient because problems were solved within the teams and because they had better knowledge about the tasks. this was best described by two developers, who both stressed the positive side of performing tasks in a team: "this is because many of the things we now do in the team, so there are not so many of those big formal review meetings. and earlier it took a long time, both the test case writing by the developers and analysts, but this now is done inside and across the team." "simply across the table, a ping pong that helps us to avoid endless, stupid time-wasting analysis and dead ends, so we quickly can resolve any issues (...)." a third developer emphasised the increase of knowledge, which he related to the fact that there now were new tasks beyond coding, which were considered as development. although this was novel and a challenge for him, as he had a prior focus on programming, he perceived this change not as necessarily negative and an interruption of his development work: "no, i do not think that, sitting down and analysing actually has the positive effect that one gets to know the tasks much better." still he was very conservative in his assessment of the achieved improvement and its impact as he scored both dimensions with 0, interestingly indicating that he did not perceive a significant change of this indicator. australasian journal of information systems volume 18 number 3 2014 310 employee performance while all other indicators of productivity were primarily assessed from the developers’ perspective this one was based on the opinion of the four respondents in leadership roles. they provided a very positive assessment. one of them explained this by referring to their negative experiences and what had been bad before the implementation of scrum and the prevailing perception about those involved in the development process: "(...) the waterfall model also represents a kind of hierarchy. so, those on the top [at the very beginning of a project] are the best and the coolest, and at the bottom we find the testers, those, who are low skilled, have little education or are not educated at all. so it was not just the waterfall model, which allowed the analyst to possibly spend too much time, and then they delivered too late down to (...) the developers, who passed it on even later to the testers. (...)" the respondent clearly felt the waterfall model put a disadvantage on those, who worked towards the end of a development process. all time overruns happened at the expense of the testers, who thus could not test sufficiently, which eventually led to rather erroneous software put into production. given that this no longer happened as scrum made sure that there was knowledge and understanding across the different professional disciplines, the respondent believed that the overall performance had increased. another manager fully agreed that it had increased, but had slightly different reasons: "(...) because now everyone has to make visible what he or she has been doing every day, that makes that one cannot so much hide behind a task (...)." this manager argued that the daily scrum meetings raised the individual employees' performance, since there was greater pressure to either deliver results or to demonstrate what challenges one had run into. such a result-oriented environment could well have had a negative effect on the unit’s culture and its ambience, but this was neither stated by any of the leaders nor by the developers; in contrast a recently performed survey in the organization confirmed these results. summary of results and discussion of findings as mentioned earlier the investigation of scrum’s impact on productivity in information systems and software development was part of a larger study, which both developed and applied a comprehensive framework consisting of seven related concepts. although a presentation of the overall result would give a much more comprehensive portrait of the method’s impact we have here focused on one of the keystone concepts mostly due to page limitations. this still provides some valuable insights and where necessary we will relate to the other concepts. as a starting point for our subsequent discussion we summarize the results of our analysis concerning scrum’s impact on productivity in the case unit as follows: we found that the decrease in the number of interruptions was limited, but it had led to a significant, perceived impact on productivity. in addition, the changed process with interruptions now coming from an authorized person and thus being less perceived as disturbances was appreciated. prioritizing new functionality higher than error-free deliveries had been the organization’s strategy to avoid endless development cycles. nevertheless these were experienced by the majority of the respondents. the increased focus on delivering functional software in defined, short iterations has prevented endless development and has resulted in more productivity, however not on the expense of product quality. scrum’s impact on quality and the interrelation of productivity and quality in the case organization are beyond the scope of this paper, but are documented in johansen & uldahl (2012). with regard to the repetition of mistakes there was also positive development. primarily the respondents' explained this progress with scrum’s focus on self-organization and not that much with the practice of retrospectives, which are the method’s explicit mechanism for the identification of weaknesses and subsequent process and product improvements. australasian journal of information systems volume 18 number 3 2014 311 the respondents felt that scrum’s decomposition and prioritization of tasks had positively changed the compliance to deadlines and had had a positive impact on productivity in general. bug fixing time was the area with the least perceived improvement compared to the other indicators. despite low average ratings, respondents expressed that although the actual time spent had not decreased, bug fixing now happened at a much better and appropriate time in the process. thus, its impact on productivity was assessed significantly higher. the perception of the number of uninterrupted, continuous development hours had also only seen a very modest increase. the respondents reasoned that the use of scrum had led to more meetings than in the past, which led to interruptions in the continuity of their work. the frequent meetings resulted, however, in a better understanding of the tasks. this was appreciated by the respondents as having a positive impact on their productivity as they thought they now both worked more efficiently and tackled unforeseen challenges much better. the managers among the respondents assessed that the employees' performance had increased significantly. they provided two different arguments for this. first scrum’s emphasis on visibility and transparency, which we had identified as a separate concept for investigating scrum’s impact on information systems and software development, made it compulsory for developers to publically present their work and take a position with regard to any challenges they had encountered. as a consequence they put more focus on the execution of their tasks. the other reason was related to the avoidance of project overruns. in the past overruns had always been passed through the chain of development tasks, with the results that the developers or even more so the testers became timepressured and could not do their job properly. the shorter iterations carried out by a multidisciplinary team avoided this effect and resulted in overall better performance. these favorable results are in line with the results for the other concepts and their indicators, which with the exception of customer satisfaction were all very positive (johansen & uldahl 2012). as with all qualitative studies of this kind we of course have to take the danger of positive bias and a respondents’ tendency of reporting future expectations rather than stating actual perceptions into account. on this background, we now compare our empirical data first with the literature on agile information systems and software development and in particular the identified writings about scrum. according to these sources, there are a number of areas that impact on productivity, these being: sprints, a focus on functional software, retrospectives, self-organization, the product backlog and the daily scrum meetings. the introduction of an iterative sprint development process (schwaber & beedle 2002) plays a central role in the use of scrum. in the case unit, sprints had made it easier to comply to deadlines as tasks were now decomposed in smaller manageable items with clear definitions, which allowed for their easier handling and execution. these results are confirmed in empirical work reported by among others augstine et al. (2005), vidgen and wang (2006) as well as wang and vidgen (2007). scrum’s increased focus on iterative delivery of functional software should according to the literature increase productivity while avoiding falling into endless development cycles in an attempt to develop the ‘perfect piece’ of software (moore et al. 2007). this was the effect scrum had on the case unit, which thus was an area where the method lived up to the expectation. retrospectives are intended to increase the productivity among others as a result of the project participants’ learning from their own and others' mistakes, so that errors and faults are not repeated in the next sprint or iteration. retrospectives should address both, the overall application of the method, its processes and practices, but also the more specific experience in the daily development work and its relation to the resulting product (schwaber 2004). the latter turned out to be an area the case unit did australasian journal of information systems volume 18 number 3 2014 312 not focus on and thus did not benefit from in their daily work. this prioritization of topics discussed during retrospectives can be explained with the case unit’s early stage of utilizing scrum and their lack of experience with regular retrospectives. in the case organization this area should therefore get further attention with an increased focus on scrum’s practices to support learning. self-organization in a scrum team has among others the objective to protect and relieve individual team members from certain tasks and create an environment where they are not constantly disturbed in their work. in a successfully self-organized team, everyone has insights into the other team members' tasks, while at the same time a scrum master is clearly identified and appointed (schwaber & beedle 2002). this means that when there is a need for input from a specific team member, the other team members are not unnecessarily disturbed, as the tasks have been clearly defined, broken down and distributed. if in doubt, the scrum master is available to facilitate or solve the problem. at the case unit this had not yet been fully achieved, which meant that employees were still interrupted and disturbed in their work and further efforts will be needed to progress. however, one of the benefits of the scrum master role had been achieved already, since the respondents expressed that the interruptions now came from the right person. in the literature the avoidance of repeating errors is ascribed to retrospectives. as discussed above in the case unit retrospectives had not yet been applied to their full potential, yet the perception of the respondents had been that the repetition of errors had drastically decreased. this was attributed to the influence that self-organization had. as a consequence of the increased individual developer’s responsibility now, team members had become more mindful not to repeat the same mistakes. individual and collective mindfulness have been reported as characteristics of agile development independently of a particular method or agile practice (matook & kautz 2008). this supports that the lack of exploiting retrospectives in the case organization has been compensated by self-organization and mindfulness to lead to a positive outcome with regard to avoiding the repetition of mistakes. in the case organization the introduction of a product backlog had primarily an effect on compliance with deadlines. as the work was now broken down to single items, there were ongoing opportunities to check whether the agreed schedule was met. additionally, there was now the possibility to prioritize and plan the order of executing the items in an appropriate manner, which according to the literature (see e.g. schwaber & beedle 2002) further increases the overall productivity. the introduction of a product backlog at the case unit had affected both of these areas positively, and thus the overall productivity. product backlogs can also be used to plan a specific test, debugging and error correction period in form of a dedicated item for these tasks (schwaber 2004). in the case unit this did not lead directly to a reduction of the time spent on bug fixing, but it had resulted in bug fixing happening at a more appropriate point in time, which ultimately had had an impact on productivity. finally, daily scrum meetings, have among others the objective to create visibility in a scrum team. this helps that everyone in a team gains insight into what the others are working on and at the same time it makes it difficult for employees to conceal modest work efforts, since they publically have to communicate and document their results (schwaber & beedle 2002). the latter had a substantial impact on employee performance in the case unit and as such affected productivity positively as openly explaining why as task took longer than expected, had the psychological effect that it deprived the employees of the opportunity to hide behind a task longer than necessary. in addition, the meetings had both a positive and negative impact on the number of uninterrupted development hours. the increased number of meetings had reduced the amount of uninterrupted development hours. this was outweighed, however, by the fact that the meetings created better visibility, oversight and knowledge. this allowed employees to tackle unforeseen challenges better, which had a positive effect on productivity, since waste time was avoided. australasian journal of information systems volume 18 number 3 2014 313 our overall positive assessment of scrum on the productivity of information systems and software development confirms empirically the expectations and claims, which are made in many of the conceptual and non-academic writings we had identified in our literature review. it also fills a gap in the area of empirical studies of agile software development (dybå & dingsøyr 2008). in the absence of quantitative data and with no possibility to make direct measurements and collect such data throughout the project it is however built on subjective perceptions. nonetheless, on a more theoretical level our study can be related to complex adaptive systems (cas) theory to find support for the increase of productivity as one outcome of scrum. cas theory underpins agile information systems and software development methods (highsmith 2000) such as scrum and the case unit appears to be rather successful after its transition to scrum. on this background the above results can be linked to cas concepts and principles. if isd, in our case agile development supported by scrum, is understood as cas, certain characteristics of the process are recognized to facilitate good performance and thus productivity, while others inhibit it (meso & jain 2006; kautz 2012). a number of concepts are frequently used when discussing cas. all these core concepts are intertwined and mutually reinforcing. within the area of isd vidgen and wang (2009) as well as kautz (2012) have summarized and put them forward as follows: interconnected autonomous agents are able to independently determine what action to take, given their perception of their environment; yet, they collectively or individually are responsive to change around them, but not overwhelmed by the generated information flow. self-organization is the capacity of these agents to evolve into an optimal organized form, which results from their interaction in a disciplined manner within locally defined and followed rules. co-evolution relates to the fact that a complex adaptive system and/or its parts alter their structures and behaviours in response to their internal interactions and to the interaction with other cas where adaptation by one system affects the other systems, which leads to reciprocal change where the systems evolve individually, but concertedly. time pacing indicates that a complex adaptive system creates an internal rhythm that drives the momentum of change, which is triggered by the passage of time rather than the occurrence of events; this stops them from changing too often or too quickly. poise at the edge of time conceptualizes a complex adaptive system’s attribute of simultaneously being rooted in the present, yet being aware of the future and its balance of engaging exploitation of existing resources and capabilities to ensure current viability with engagement of enough exploration of new opportunities to ensure future viability. poise at the edge of chaos describes the ability of a complex adaptive system to be at the same time stable and unstable; this is the place not only for experimentation and novelty to appear, but also for sufficient structures to avoid disintegration; cas that are driven to the edge of chaos out-compete those that are not. the above analysis has provided examples of interacting interconnected autonomous agents, such as the involved developers and testers, their selforganization as individuals and as project teams, their co-evolution through knowledge sharing and learning from each other, as well as for time pacing in the short iterative development cycles, and for poise at the edge of time and chaos, for instance with regard to compliance to deadlines, bug fixing time and uninterrupted development hours, which thus empirically and theoretically lend support to the identified perceived positive impact of scrum on performance and productivity of information systems and software development in our case setting. conclusion while the usual disclaimers for the shortcomings of qualitative research also apply for our study, our work contributes to the body of knowledge in information systems development with an empirical investigation that demonstrates the positive impact of the agile development and project management method scrum on information systems and software development productivity and it provides a useful operationalization of the concept through seven indicators. despite the fact that the case unit had challenges with the use of scrum, the indicators identified the areas where the company had managed australasian journal of information systems volume 18 number 3 2014 314 to exploit the potential of scrum and its practices with regard to increasing productivity. through the analysis we found an interesting area where the case unit differed from the scrum literature’s recommendations. the case unit’s handling of retrospective meetings only reflected the actual process and method, but not the developed product. this put the unit at the risk of missing out on any knowledge, which could contribute positively to the future iterations and development projects. therefore future research should further investigate the relationship between team learning and interaction of autonomous interconnected team members in retrospectives and how productivity supported through scrum stems from learning. although several authors underline the importance of an open organizational culture for agile development (cockburn 2001; highsmith 2002; nerur et al. 2005; robinson & sharp 2005; kautz et al. 2009) and argue that an innovative and open organizational culture is necessary to develop software according to agile principles we decided to disregard the concept as such as we assumed that the culture, its elements, the basic assumptions held by all members of that culture, their values and beliefs, and their artefacts and creations (schein 2004) and the cultural changes as a result of an implementation of scrum would have an impact and become visible through the indicators. in other words, for culture as a broad concept we thought it would make more sense to be implicitly investigated through the productivity indicators. in hindsight the relationship between culture and productivity in the use of agile methods such as scrum does however also merit a thorough investigation through future research on its own. references andersen, i. 2006. the apparent reality (in danish). “samfundslitteratur” publisher. frederiksberg, denmark. augustine, s., payne, b., sencindiver, f., and woodcock, s. 2005. “agile project management: steering from the edges,” communications of the acm (48:12), pp. 85-89. appelo, j. 2010. management 3.0 leading agile developers, developing agile leaders. addisonwesley. crawfordsville, indiana, usa. cockburn, a. 2001. agile software development. addison-wesley. boston, ma, usa. dybå, t., and dingsøyr, t. 2008. “empirical studies of agile software development: a systematic review,” information and software technology (50:9-10), pp 833-859. guang-yong, h., 2011. “study and practice of import scrum agile software development,” in proceedings of the 3rd international ieee conference on communication software and networks (iccsn), nanjing, kina. highsmith, j. 2000. adaptive software development: a collaborative approach to managing complex systems. dorset house publishing. new york, usa. highsmith, j. 2002. agile software development ecosystems. addison-wesley. boston, ma, usa. johansen, t., and uldahl, a. 2012. measuring the impact of the implementation of the project management method scrum (in danish). msc thesis. copenhagen business school, copenhagen, denmark. kautz, pedersen, c. f., and monrad, o. 2009. “cultures of agility – agile software development in practice,” in proceedings of the 20th australasian conference on information systems. melbourne, australia. kautz, k. 2012. “beyond simple classifications: contemporary information systems development projects as complex adaptive systems”, in proceedings of 33rd international conference on information systems. orlando, fl, usa. australasian journal of information systems volume 18 number 3 2014 315 mahnic,v., and zabkar, n. 2008. “using cobit indicators for measuring scrum-based software development,” wseas transactions on computers (7:10), pp. 1605-1617. mahnic, v., and vrana, i. 2007. “using stakeholder-driven process performance measurement for monitoring the performance of a scrum-based software development process,” electrotechnical review (74:5), ljubljana, slovenia, pp. 241-247. matook, s., and kautz, k. 2008. “mindfulness and agile software development,” in proceedings of the 19th australasian conference on information systems. christchurch, nz, pp. 638-647. meso, p. and jain, r. 2006. “agile software development: adaptive systems principles and best practices,” information systems management (23:3), pp.19-30. moe, n. & dingsøyr, t., 2008. “scrum and team effectiveness: theory and practice,” in proceeding of the conference on agile processes in software engineering and extreme programming. springer. berlin, germany, pp.11-20 moore, r., reff, k., graham, j. and hackerson, b. 2007. “scrum at a fortune 500 manufacturing company”, in proceedings of the agile conference (agile). st. paul, minnesota, usa. nerur, s., mahapatra, r., and mangalaraj, g. 2005. “challenges of migrating to agile methodologies,” communications of the acm (58:5), pp73-78. robinson, h., and sharp, h. 2005. “organizational culture and xp: three case studies,” agile development conference, ieee computer society. denver, co, usa. pp 49-58. schein, e. 2004. organizational culture and leadership. 3rd edition. wiley & sons. san francisco, ca. usa. schwaber, k., 2004. agile project management with scrum. microsoft press. redmond, washington, usa schwaber, k., and beedle, m., 2002. agile software development with scrum. prentice hall. upper saddle river, usa. sutherland, j., and altman, i. 2010. “organizational transformation with scrum: how a venture capital group gets twice as much done with half the work”, in proceedings of the 43rd hawaii international conference on system sciences. kauai, hawaii usa. vidgen, r., and wang, x. 2006. “organizing for agility: a complex adaptive systems perspective on agile software development process,” in proceedings of the 14th european conference on information systems. göteborg, sweden. ljunberg, j., and andersson, m. (eds.), pp. 1316-1327. vidgen, r., and wang, x. 2009. “coevolving systems and the organization of agile software development,” information systems research (20:3), pp. 355-376. wang, x., and vidgen, r. 2007. “order and chaos in software development: a comparison of two software development teams in a major it company,” in proceedings of the 16th european conference on information systems. winter, r., et al. (eds.). st. gallen, switzerland, pp. 807818. webster, j., and watson, r.t. 2002. “analyzing the past to prepare for the future: writing a literature review,” mis quarterly (26:2), pp 13-23. an earlier version of this paper was presented at the australasian conference on information systems (acis) 2013 in melbourne, australia. ajis vol. 6 no. 2 may 1999 database issues in knowledge discovery and data mining chris p. rainsford defence science and technology organisation information technology division dsto c3 research centre femhill park, canberra 2600 chris.rainsford@dsto.defence.gov.au john f. roddick school of computer and information science university of south australia mawson lakes, adelaide. south australia 5095 roddick@cis.unisa.edu.au abstract in recent years both the number and the size of organisational databases have increased rapidly. however, although available processing power has also grown, the increase in stored data has not necessarily led to a corresponding increase in useful information and knowledge. this has led to a growing interest in the development of tools capable of harnessing the increased processing power available to better utilise the potential of stored data. the terms "knowledge discovery in databases" and "data mining" have been adopted for a field of research dealing with the automatic discovery of knowledge impb'cit within databases. data mining is useful in situations where the volume of data is either too large or too complicated for manual processing or, to a lesser extent, where human experts are unavailable to provide knowledge. the success already attained by a wide range of data mining applications has continued to prompt further investigation into alternative data mining techniques and the extension of data mining to new domains. this paper surveys, from the standpoint of the database systems community, current issues in data mining research by examining the architectural and process models adopted by knowledge discovery systems, the different types of discovered knowledge, the way knowledge discovery systems operate on different data types, various techniques for knowledge discovery and the ways in which discovered knowledge is used. introduction in recent years the continued growth in the size of databases has led to an increased interest in the automatic extraction of knowledge from data. it is therefore not surprising that many leading database researchers have identified this as an area worthy of significant investigation (silberschatz, stonebraker and ullman 1996; stonebraker, et al. 1993). the term data mining, or knowledge discovery in databases (kdd), has been adopted for a field of research dealing with the discovery of information or knowledge from data held in more or less structured databases (fayyad, et al. 1996; piatetsky-shapiro and frawley 1991). although these two terms have been used interchangeably in the past, leading researchers in the field have only recently distinguished between them (fayyad, piatetsky-shapiro and smyth 1996). following their distinction, knowledge discovery in databases can be seen as the overall process of extracting useful and interesting information from databases. this process includes the selection and preparation of data and the manipulation and analysis of results. by comparison data mining can be viewed as the application of knowledge discovery algorithms without the other stages of the knowledge discovery process, and is therefore a subset of kdd. kdd is typically most useful in situations where the volume of data is either very large or too complicated for traditional methods, or where human experts are unavailable to extract knowledge. as would have been expected, kdd has borrowed heavily from traditional machine learning and database theory. learning can be defined as knowledge acquisition in the absence of explicit programming (valiant 1984). machine learning aims to automate the learning process, so that knowledge can be acquired with minimal dependency upon human input (michalski, carbonell and mitchell 1984). machine learning has traditionally focussed on learning from sets of specifically and, in many cases, artificially generated data. data mining aims to adapt these machine learning paradigms to learn from databases containing real world data. learning from within databases has some advantages (roddick and rice 1998): • the data is stored in a more or less structured manner. for example, in a relational database data is typically normalised into relations that eliminate redundancy and can be joined in various ways to retrieve required data sets from the database. in other database paradigms, either more or less structure is available. nevertheless any apriori known structure can be utilised. • some domain knowledge is already encoded implicitly within the database. for example the existence of a participation constraint may be flagged by a not null constraint. similarly the cardinality of relationships is also often explicit within database structure and constraints. 101 ajis vol. 6 no. 2 may 1999 • high performance query, data manipulation and transaction tools are already available. this would include the database management system, associated query language, specialised hardware and other database tools. it therefore makes some sense to use these tools to interrogate the database where appropriate. • the number of databases with data applicable to mining techniques is large and growing. the effort expended in developing such tools is thus economically viable. similarly, the use of data from databases imposes a number of characteristics and constraints: • the volume of data is typically very large. for example, the skicat system has been developed to process three terabytes of graphic images resulting from a sky survey (fayyad, weir and djorgovski 1993). therefore, any data mining tool must perform satisfactorily on large volumes of data. • the data may contain noise. data mining tools must provide adequate mechanisms for finding sufficiently accurate results from noisy data. • the database may contain incomplete information. not all information useful for the discovery of knowledge may actually be stored within the database. likewise much redundant and useless data may also be present. therefore data mining tools must facilitate both the selection of relevant data, and learning with incomplete knowledge. • the data has not generally been collected for the purpose of knowledge discovery. as well as leading to some of the above problems this means that data may be formatted inappropriately. knowledge discovery tools must therefore be able to access data stored in various formats. one overall goal of knowledge discovery and data mining research is to utilise the advantages of learning from databases, while accommodating the constraints imposed. this paper provides a survey of current areas of research within data mining. the next section describes a process model of a data mining system. the major types of discovered knowledge are then discussed and following that, the data types that are targeted for knowledge discovery. some of the major applications of discovered knowledge are then described and areas for future research are identified and discussed in the final section. a data mining model figure 1 represents one possible model of the data mining process (adapted from (rainsford and roddick 1996)). for any database, the number of possible rules that can be extracted is far greater than the number of tuples in the database. knowledge discovery can thus be viewed as the multi-stage process of selecting interesting rules from the total rule-space that exists within a database. there is therefore a process of progressively reducing the initial infinite rule space down to a small subset of useful rules. the model outlined is based on the nature of the refinement as illustrated in many current research tools that of a reduction process performed using a selection of filters that reduce the target rule space on the basis of source data, rule pattern, statistics and semantics. there are many special cases where a phase of the filtering process does not exist or is not used within a given research tool, and in these cases the filter effectively allows the rule space to pass unreduced. for example, semantic filtering may not be used if rules are to be used for query optimisation. each of the filtering stages may consist of zero or more filters specified by the user or discovery system. the target rule set may be passed back and forth between the filters for reprocessing. a central controller coordinates the operation of the filters. as noted on the diagram the final rule set can be integrated into the existing knowledge base. as indicated, both the knowledge base and the user may interact with each step of the rule-space reduction process. note that the filtering processes may have a profound effect on the outcome of the data mining process. thus, the outcome of the data mining process is quasi-nondeterministic. data filtering the initial stage involves the selection of the data of interest for knowledge discovery. the user may direct the kdd system to areas of interest using templates, visualisation tools and by specifying sampling strategies. the result of this phase is a reduced set of test data and a correspondingly reduced rule-space. pattern filtering the second phase is the pattern filter, where a particular rule type to be searched for is specified. this may be fixed by the limitations of the system or specified to a greater or lesser extent via the use of templates or rule type selection. the type of pattern discovered is typically restricted by the kdd system itself because most systems can only discover a limited number of rule types. the pattern can be further restricted by the user requiring the presence of particular attributes on one or other side of a discovered rule, or by restricting the number of possible conjunctions it may contain. the effect of the pattern filter is therefore to reduce the relevant rule space to only rules of a particular type, eg. association rules with a given term in the consequent. the 102 ajis vol. 6 no. 2 may 1999 specification of a pattern to be searched for can be made intuitively in the form of a template-based request such as: find all association rules with butter in the consequent. applications expert systems semantic query optimisation intelligent interfaces decision support systems integrity enforcement database design a a a f v v -'database. i knowledge, : ' . ' : base . •. augments final rule set y > data filters 1 t 4 * c~ ~ frules > j > t domain/ prior knowledge ' > pattern *• filters l 4 * ^ ^•rules > j r statistical ^•filters 1 ^ t ^" ^ •rules -; / > r k semantic *• filters 1 control control control control templates. visualisation and selection tools t rule type specification and templates \ t parameters and thresholds h } browsers and filters i / k figure 1. a model of the data mining process. in this simplistic example of a template, the key words in italics may be replaced to form different requests. this can be implemented in an interface using tools such as pick lists, which simplify the users interaction with the system. most kdd systems implement some form of template for pattern specification. one example is the nielsen opportunity explorer™, a knowledge discovery tool utilising discovery templates as described by anand and kahn (1993). within this system, knowledge discovery templates allow the user to also specify both the analytic techniques to be applied (statistical filtering), and the data that they are to be applied to (data filtering). likewise the explora system utilises statement types that act as templates, corresponding to specific types of rules to be discovered (hoschka and klosgen 1991). 103 ajis vol. 6 no. 2 may 1999 statistical filtering the rule space is further refined through a third phase of statistical filtering. at this stage the space of rules that satisfy the pattern is further reduced by the application of statistical tests. this process eliminates rules that fit the specified pattern, but are deemed statistically unsatisfactory or insignificant. the user may interact with this phase by setting statistical parameters or turning statistical tests on or off. as an example consider the following simplistic template: find all association rules with butter in the consequent having a minimum support of 0.002 and a minimum confidence of 0.85 in this case the template specifies two statistical measures support and confidence, with a required minimum value. although the statistical values a tool associates with a rule vary, they typically describe attributes such as the confidence of the rule or the amount of supporting evidence for the rule in the database. whilst traditional statistical techniques form a foundation for statistical evaluation, in many cases customised measures have been developed to meet the specific requirements of kdd applications. as an example of a statistical measure developed for kdd, agrawal et a/.(1993) describe two statistical functions that can be used to describe an association rule, support and confidence. support is a measure of the probability that a transaction in the database will contain the given pattern. given the presence of the antecedent in a tuple, confidence is the probability that the consequent will also be present. another approach is adopted by explora version 1.1 as described by klosgen (1995b), which uses measures of evidence and affinity to help eliminate redundant findings. evidence is a statistical measure of the quality of an individual rule, which may be calculated in several ways. affinity is an asymmetric measure of the similarity between two findings. these two measures can be combined to filter out a finding that is similar, but weaker than another finding and hence reduce redundancy. anand et al. (1995b) advocate evidential theory as a more general-purpose statistical technique for use within knowledge discovery (guan and bell 1991, 1992). evidential theory, which employs a generalisation of the bayesian model for uncertainty, has two major advantages of over the widely used bayesian model. firstly, it allows a belief value to be associated with ignorance that can be used to handle missing database values. secondly, it allows evidence at various levels of coarseness to be combined. this system is inherently parallel and therefore leads to parallel systems for knowledge discovery. the application of evidential theory to parallel knowledge discovery is described by anand et al. (1995a); likewise the application of evidential theory to knowledge discovery from images in a spatial context is described by bell et al. (1994). semantic filtering the final, and arguably the most difficult, phase of rule space reduction is semantic filtering. at this stage the interestingness and utility of discovered rules is assessed. some rules that satisfy all other requirements may simply be uninteresting or redundant, and hence are removed from the rule space. this phase typically involves heavy interaction with the user via browsers and rule visualisation tools. in addition the knowledge base may be consulted to place the discovered rule set in the context of what is already known. the output of this process is the final set of discovered rules that satisfy the user requirements. for the purposes of semantic evaluation it would be desirable to allow users to browse a set of discovered rules. interesting rules could then be identified for further investigation or presentation. a good example of support for interactive semantic selection can be seen in the explora system. the explora system which finds rules and examples corresponding to user selected statement types facilitates interactive browsing of its discovered results (hoschka and klosgen 1991). the results of analysis are presented as messages linked by appropriate relationships and the user may browse the resulting message space. a report of the analysis can be interactively composed with the assistance of three available activity spaces. the analysis space contains a window for each statement type underlying the search and related findings such as supporting and contradictory examples can be browsed. an outlining space facilitates the creation of the final report from selected analysis results. an argumentation space can then be used to organise the selected findings into argumentative structures. this set of browsing tools thereby provides a user interactive process to create valuable reports from raw analytical results. the utility of kdd tools can be improved if the semantic evaluation of rules is automated, removing the requirement for human interaction at this stage. however the definition of what makes a rule interesting remains subjective and composed of too many aspects to be easily definable and the problem has become widely acknowledged in the kdd field ay. piatetsky-shapiro (1994). klosgen however identifies several aspects that capture the nature of interestingness in discovered knowledge (klosgen 1995b). the properties suggested by klosgen are evidence, redundancy, usefulness, novelty, simplicity and generality. a similar set of adjectives 104 ajis vol. 6 no. 2 may 1999 describing intereslingness is provided by asa and mangano: performance, simplicity, novelty and significance (asa and mangano 1995). types of discovered knowledge the type of knowledge that is discovered from databases and its corresponding representational form varies widely depending on both the application area and database type. the specification of the type of knowledge to be discovered directs the pattern filtering process. knowledge learned from large sets of data can take many forms including classification knowledge, characteristic rules, association rules, functional relationships, functional dependencies and causal rules. this section will describe each of these categories of knowledge and discuss example systems that learn each type. in table 1 the types of knowledge which are explicitly supported by a selection of current data mining tools are indicated. many of these tools are subject to ongoing development and therefore this represents a summary at the present time. moreover, the purpose of this survey is to demonstrate the broad diversity of a cross section of data mining tools and not to form the basis of any tool comparison or evaluation. classification knowledge classification knowledge can be used to categorise new examples into classes on the basis of known properties. such information can, for example, be used by lending institutions to classify the credit risk of prospective borrowers and could be constructed from records of past loans. following the formalism of agrawal el al. (1992) inferring classification functions from examples can be described as follows: let g be a set of m group labels {gi, g2,..., gm}. let a be a set of n attributes (features) {a], a2,..., an}. let dom(aj) refer to the set of possible values for attribute a,. we are given a large database of objects d in which each object is an n-tuple of the form < vj, v2,..., vn > where v, € dom(a,) and g is not one of a,. in other words, the group labels of objects in d are not known. we are also given a set of example objects e in which each object is a (n+l)-tuple of the form < yi, v2,..., vn, gp> where v, e dom(a,) and g* e g. in other words, the objects in e have the same attributes as the objects in d, and additionally have group labels associated with them. the problem is to obtain m classification functions, one for each group g,, using the information in e, with the classification function f, for group g; being f): aixa2x...an g; fory = i,..., m. we also refer to the examples set e as the training set and the database d as the test data set. system / system extension clementine dbminer(han, etal. 1996) emerald explore (klosgen 1993; klosgen 1995b) ; mine rule (meo, psaila and ceri 1996) mineset 1.1 posch (long, irani and slagle 1 99 1 ) quest (agrawal, et al. 1996) rx project (blum 1982) savnik & flach (savnik and flach 1993) o £t vt u> 3) o 5 3 ' ' s ' ' ' q"oo 3. vt 0 ^ ^ ^ s < 0 r> ' si o • ' ^ ^ ^ g. £ § 5 c 3 § co ^ 1? 3 s q. 2§ 1' o q3 $ ' 0 e (a ^ ^ h f? 1e. ' ^ ^ ^ • ^ 0 c ^^s. do ^ ^ ^ ^ ^ ^ table 1 knowledge types supported by selected research tools as noted by rubinstein and hastie (1997) classification can be approached in two different ways. the first approach is informative learning where the focus is upon finding a good description for each class. classification then proceeds by comparing the instance to be classified with each class to find the best match. this approach is also useful when the emphasis is on determining the characteristics that are associated with a 105 ajis vol. 6 no. 2 may 1999 particular class of instances. the second approach is discriminative learning where the emphasis is on defining the boundaries between classes. a model for categorising examples into each class is developed rather than a model describing the characteristics of each class. this approach is applicable to problems where the emphasis is on classifying instances of unknown class. one of the most popular models for representing discriminative classification knowledge is the decision tree. the nodes of the tree represent attributes to be tested, and the branches correspond to the possible values of those attributes. the leaves of the tree represent classes into which examples are classified. therefore, starting at the root node on the tree, an example can be tested at each node it encounters and follow the resulting branches to be classified in its appropriate leaf. the induction of decision trees has been widely investigated and an important algorithm for the induction of decision trees, ids, is described by quintan (1986). this algorithm has been built on extensively and proposals to accommodate extensions, such as the accommodation of approximate data, have been numerous. decision trees can be equally represented as rules. one rule can be constructed for each leaf node, where the leafs classification class is the consequent of the rule. the antecedent is a conjunction of the attribute value pairs encountered at each branch of the tree. agrawal el al. (1992) describe an interval classifier 1c that produces kary classification trees. importantly, non-categorical attributes are not divided into binary sub-trees as in id3, but are instead split into k-ary sub-trees. in comparison with id3 the resulting trees can be created with greater efficiency and display favourable classification accuracy. the improved speed of classification represents a significant advantage within knowledge discovery systems where the data is both dynamic and of large volume. in addition, speed of classification is important for the support of ad-hoc queries. one informative approach to classification is to identify target classes and then find a description that uniquely identifies members of each class. following this approach cai et al. describe a classification rule learning algorithm lclr (cai, cercone and han 1990). their approach utilises conceptual hierarchies as a tool for induction. classes to be classified are firstly specified. all tuples in the learning data set that describe a specific class are then generalised stepwise until a description consisting of a specified number of generalised tuples is reached. generalisation takes place via concept ascension of attribute level conceptual hierarchies. as descriptions describing more than one class are eliminated, areas of overlapping classes in the decision space remain undescribed by the resulting clauses. this approach produces a classification rule in the form of a generalised relation. plants (seed producing) {non seed producing) i i spermatophyta pteridophyta i i ovule structure environment (ovules borne naked ) {ovule enclosed) (water-based) (rooted in soil / rocks / trees) i i i i gymnospermae angicspermae group 2 group 1 i i i leaf structure rooting arrangement of spores (finale) (simple) (aquatic / subaquatic) (terrestrial) (sporangia fused) (sporangia free) i i i i i i zamiceae ...furthersubdivision ...farther subdivision sdeginellaceae psilotaceae -.further subdivision figure 2. an example decision tree, plant classification (after (beadle, evans and carotin 1986)) another informative approach to classification is to classify attribute values individually and assign appropriate classification(s) that can then be combined to classify an object displaying these values. this approach is adopted by chan and wong (1991) who describe a statistically based technique for the classification of objects from existing databases. their approach assigns a weight of evidence to attribute values indicating how the value of an attribute effects an object's potential membership of a given class. the total evidence for and against 106 ajis vol. 6 no. 2 may 1999 membership of a particular class can then be determined for each object to be classified, based upon its attribute values, and the best classification determined. this approach can be applied to noisy data with missing values, which may be encountered in real world databases. in addition, objects to be classified are tested for possible classification against all classes. because weightings are used, it can be determined how good an example of a class a particular object is. membership of more than one class is also possible, where the membership weighting is within acceptable limits. this feature is particularly useful in domains where membership of more than one class are valid. characteristic rules a characteristic rule can be defined as an assertion that characterises the concept satisfied by all of the relevant data in the database (han, cai and cercone 1993). characteristic rules are essentially rules that describe the characteristics of a concept, via an abstraction based upon the data in the database. the need to provide generalisations to describe concepts means that a path of generalisation, such as a conceptual hierarchy, is typically required as domain knowledge. characteristic rules make no explicit reference to relationships between entities, or means of classifying them. however, characteristic knowledge is useful for providing a summary or abstraction of the data it describes, which may be used for applications such as query optimisation, integrity enforcement or the automatic discovery of dependencies. han et al. (1993) describe an approach to learning characterisation rules that is based on a process of concept ascension. within this model generalisation occurs at the attribute level, following a generalisation hierarchy provided as domain knowledge. this approach assumes that such a hierarchy is available. both a qualitative and quantitative algorithm has been developed. the quantitative algorithm facilitates statistical methods, whereas the qualitative method can be used for purposes such as complete induction of the entire data set. lu et al, describe similar techniques to induce characteristic rules from spatial data (lu, han and ooi 1993). association rules the discovery of association rules in large databases was first described by agrawal et al. (1993). the initial motivation for association rules was to aid in the analysis of large transaction databases, such as those collected by supermarkets. the discovery of associations between the purchase of various line items can potentially aid decision making within retail organisations. transaction databases are therefore the primary domain targeted for association rule discovery. using the formalism provided by agrawal etal, association rules can be defined as follows. let i = i j , \2,---, im be a set of binary attributes or items and t be a database of tuples. association rules were first proposed for use within transaction databases, where each transaction t is recorded with a corresponding tuple. hence attributes represented items and were limited to a binary domain where t(k) = 1 indicated that the item 1̂ had been purchased as part of the transaction, and t(k) = 0 indicated that it had not. association rules in this form can however be generalised to any attribute domain in which one set of domain values corresponds to 1 and another set to 0. for example within a university database domain values corresponding to science subjects {chemistry, maths, physics} may be assigned the value 1, and humanities subjects {literature, modern art, environmental design} assigned the value 0. therefore t may be any tuple with binary domain attributes, which need not represent a transaction. let x be a set of some attributes in i. we say that a transaction t satisfies x if for all attributes 1̂ in x, t(k) =1. by an association rule, we mean an implication of the form x ̂ y where x, the antecedent, is a set of attributes in i and y, the consequent, is a set of attributes in i that is not present in x. the rule x => y is satisfied in the set of transactions t with the confidence factor 0 < c < 1 iff at least c% of transactions in t that satisfy x also satisfy y. we will use the notation x => yl c to specify that the rule x =* y has a confidence factor of c. association rules are particularly applicable to commercial data mining applications. for instance, in a database containing details of product sales they allow the user to request knowledge such as: find any rules that have diet lemonade as the consequent such a request enables market analysts to find the factors affecting the sale of diet lemonade. conversely learning requests regarding the antecedent can be asked, such as: find any rules that have taco shells in the antecedent requests such as this that involve finding rules with an item in the antecedent can help analysts in determining how the particular item may be associated with the purchase of other items or more generally, indicate to researchers possible correlations that may be worth further investigation. as a simplistic example, it may be 107 ajis vol. 6 no. 2 may 1999 induced that the decision to stop selling taco shells may lead to a significant fall in the sale of related products such as taco sauce. the strength of any such associations could be tested using an association rule such as the one above. this approach may help identify previously unknown or overlooked associations, and test the strength of associations believed to exist. meo et al. describe an sql-like operator for mining association rules mine rule (meo, psaila and ceri 1996). this operator may be utilised by systems users and analysts to direct searches such as those described above. an itemset can be defined as a set of line items purchased as part of a single transaction. the initial algorithm, a/5, proposed by agrawal et al, operates by looking for itemsets with the required support within the database, called large itemsets. trying to find all of the large itemsets in a single pass over the data is inefficient because most of the itemsets that would be measured are not large. the a/5 technique reduces this inefficiency by making several passes over the data and using the results of the current pass to produce candidate large itemsets to evaluate in the next pass. this process of discovering large itemsets continues until no more large itemsets are found. this approach however has proven to produce too many small itemsets as candidates for large itemsets. this is because it produces candidate itemsets by extending the existing large itemsets using itemsets that are not large. mannila et al. (1993,1994) describe an algorithm that is faster than the agrawal et al. algorithm, and has demonstrated a performance improvement of a factor of five in testing examples. the efficiency gain in their algorithm is based upon the observation that subsets of large sets must also be large. the number of candidate sets can therefore be dramatically reduced by using only large itemsets in the construction of other candidate large itemsets. they also point out the viability of sampling as an efficient method for finding large itemsets. however, as expected, sampling techniques result in a trade-off in accuracy. as a result of the same observation as mannila et al, agrawal and srikant propose two alternative algorithms, apriori and aprioritid, both of which are more efficient than a/5 (agrawal and srikant 1994). the efficiency gain is obtained by only considering previously discovered large itemsets in the construction of candidate large itemsets. the underlying principle is the same as that observed by mannila et al\ namely that subsets of large itemsets must also be large. the improvement in performance increases with the size of the database being input, ranging from a threefold improvement of performance to more than an order of magnitude improvement on larger sets of data. the aprioritid algorithm differs from the apriori algorithm, in that it does not need to test the candidate itemsets for support against the main database, but can check against the existing large itemsets themselves. this approach can offer significant performance improvement particularly when a large number of attributes are present in the candidate itemset. however this performance improvement is contingent upon aprioritid being able to store the large itemsets in main memory and where this is not the case, apriori starts outperforming aprioritid. with this trade-off in mind, agrawal and srikant describe apriorihybrid, a hybrid of apriori and aprioritid. apriorihybrid begins like the apriori algorithm scanning the database to generate each generation of candidate large itemsets, until it estimates the large itemsets will fit into main memory. at this point, the algorithm then generates candidates from existing large itemsets like aprioritid. a further improvement in performance is achieved by park et al. (1995) who describe dhp, a hash-based algorithm for the discovery of association rules. their approach is based upon the observation that the most costly process in the discovery of association rules is the determination of large itemsets in the early iterations. importantly, large performance improvements can be gained by reducing the cost of finding large itemsets with two items. this reduction is achieved by using a hash table constructed in initial passes over the data to select fewer redundant candidate large itemsets. because fewer candidate large itemsets are generated, the processing costs are significantly reduced. subsequent performance improvements can also be achieved by reducing the size of the transaction database by removing redundant items and itemsets. testing has shown that after the initial overhead required to produce the hash table, the performance of this algorithm is significantly better than algorithms such as apriori. another approach to reducing the cost of learning association rules is described by savasere et al. (1995) who describe an algorithm called partition for the efficient discovery of association rules. this technique reduces disk i/o by limiting the number of passes over the database to two. this is achieved by firstly finding all candidate large itemsets in the first pass. the actual support for these itemsets can then be determined in the second pass. in the initial pass the database is divided into multiple partitions small enough to be placed in main memory and all large itemsets within each partition are found. these large itemsets are then combined to produce a set of all itemsets that are candidates to be large within the entire database. this set contains all large itemsets because any itemset that is large within the database must be large in at least one partition. han and fu, describe a set of algorithms for learning association rules with the use of conceptual hierarchies (han and fu 1995). this approach is powerful because it widens the scope of rules that can be learnt. in addition, if the hierarchies represent real-world relationships, then it allows users to exploit an intuitive and powerful notation for selecting sets. itemsets of different size and attributes at different levels of generalisation are both considered in the search for multiple level association rules. the order in which the search considers different sized itemsets and different levels of generalisation effects the efficiency of the algorithm. the 108 ajis vol. 6 no. 2 may 1999 different algorithms proposed by han and fu vary in their approach to searching efficiently, which largely results from reducing the search space as quickly as possible. however the utility of individual algorithms is dependent upon the data being used. a method for selecting an appropriate method based upon the nature of the data would therefore be useful. srikant and agrawal also address the issue of mining association rules at different levels of generalisation (srikant and agrawal 1995). similar to han and fu, they describe multiple algorithms for mining association rules. their optimisation techniques include a sampling technique using a subset of the database to help determine candidate large itemsets. in addition, they describe a measure of interestingness specific to the discovery of multiple level association rules. this measure is based upon the assertion that if the support for an association is consistent with the support for its more generalised form, then this more specific association is of little interest and therefore is a candidate for pruning. the potential exists to extend association rules to encompass both different types of associations, and different types of data. whilst the initial focus has been upon transaction databases, association rules can also be extracted from other organisational databases. relational databases within many organisations store quantitative and categorical attributes. quantitative association rules learnt from quantitative and categorical data are described by srikant and agrawal (1996). likewise, temporal or spatial relationships can, for example, form the basis of an association. spatial association rules are discussed by koperski and han (1995), and an algorithm for their derivation is provided. functional relationships functional relationships within data describe the value of one or more attributes as a function of other attributes. for example, an attribute y may be described as a function of two other attributes x, and z as y = -j2x * 7z. such relationships are important within the scientific domain where the functional relationship between two attributes within the data may reflect relationships in the underlying domain. however whilst it may be expected that such a relationship exists within a data set, the exact relationship may be unknown. because the data sets in question may be very large and complex, manual extraction of relationships from the data may be impractical. therefore the automatic discovery of functional relationships, using ai techniques is a useful application domain for data mining. a major limitation amongst function finding tools is highlighted by schaffer (1991) and, in earlier work, referred to in the survey by angluin and smith (1983); for any given data of example values, an infinite number of describing functions can be derived. the selection of the most appropriate function is therefore integral to the operation of function finding tools. the characteristics of the most appropriate function depend upon both domain knowledge, and the proposed use of the discovered function. as these parameters will vary between applications, no single universal solution can be provided. because of this, in many cases the utility of function finding tools will be determined in part by the ability of domain experts to specify the characteristics of desirable functions. typically, a set of heuristics must be used to moderate the search including the expected accuracy and simplicity of the candidate functions. for example, if a quadratic and a cubic equation give similar accuracy on the data, then it is likely that the quadratic would be preferred for simplicity. functional dependencies adopting the notation of elmasri and navathe (1989), a functional dependency can be described as follows. given two sets of attributes x and y existing in the database, a functional dependency (denoted as x—»y) states that for any two given tuples t j and \^ if tj(x) = t2(x), then tj(y) = t2(y) must also be true. an example of a functional dependency may be: course_code —» course_name that is, if we know a subject's course code, we can determine its name. the implication of this within a relational database is that we need only store each course_name once with its corresponding course_code. because of this property, functional dependencies are used to design the structure of a relational database, helping to eliminate redundant data storage, via normalisation. if previously undetected induced dependencies are found to exist within a database, restructuring of the database may take place, via a process of schema evolution, see (roddick 1995) for a survey of publications on schema evolution. given the importance of functional relationships within the relational model, it is not surprising that the induction of functional dependencies within databases has been widely investigated. roddick et al, loosely defines induced functional dependencies as follows: functional dependencies that are consistent with the data currently held in the database but which have not been defined within the database schema, (roddick, craske and richards 1996). induced dependencies only hold true for the current data in the database, ie. there is no 109 ajis vol. 6 no. 2 may 1999 guarantee that valid data, which contradicts them, may later be entered into the database. restructuring the database on the basis of induced dependencies may therefore be unwise, however they may be used for purposes such as semantic query optimisation, or integrity enforcement and error detection. top-down approaches to the discovery of induced dependencies begin by suggesting general induced dependencies, and then refining these to more specific dependencies as they are made invalid by contradicting examples within the data. however such an approach is impractical, as it is slow within the realm of very large databases. this is a major limitation, given that the discovery of induced dependencies is most promising within larger databases. savnik and flach propose an improvement by initially adopting a bottom-up inductive approach (savnik and flach 1993). their technique begins by defining the cover for invalid dependencies, via bottom-up induction. this is stored in a tree-like data structure, and can be quickly accessed to assess the validity of any proposed functional dependencies. therefore the efficiency for the top-down assessment of proposed dependencies can be dramatically improved. a totally bottom-up approach to the induction of functional dependencies may also be adopted, for example using the algorithms of han et al. (1993).the utility of functional dependencies is not restricted to traditional relational databases. where other data models are used, functional dependencies, which incorporate the associated semantics, can potentially be found. within temporal databases, some functional dependencies may be dependent upon temporal relationships, for example, the time and value of a change in one set of attributes may be inferable from a change in another set. in addition, functional dependencies can be constrained to be valid only at specific times. roddick et al, (1996) address this issue, extending functional dependencies to the temporal database domain by defining temporal induced dependencies. within spatial databases, spatial semantics may likewise be incorporated into functional dependencies. causal rules causal rules describe relationships where changes in one part of the modelled reality cause subsequent changes in other parts of the domain. blum provides the following operational definition of causality: a is said to cause b if over repeated observations (1) a generally precedes b, (2) the intensity of a is correlated with the intensity ofb, and (3) there is no known third variable c, responsible for the correlation, (blum 1982). the discovery of causal relationships is important within many areas of scientific investigation and especially medicine. the search for causal rules within databases also offers potential for uncovering knowledge useful in the understanding of organisational operation. the work of roddick et al, mentioned in the previous section also has relevance to causal relationships, as temporal dependencies can be the result of underlying causal relationships. however causal relationships are not implied by temporal dependencies and their existence is not investigated. causal relationships are common targets of scientific investigation within the medical domain, where the search for factors that may cause particular medical conditions is a fundamental objective. therefore, it is not surprising that much of the investigation into causal rule discovery to date has been within the medical domain. most notably, the rx project (blum 1982) is well known as a tool for the discovery of causal relationships within patient databases. another example of a tool for the discovery of causal relationships within medical data is the program on the surgical control of the hyperlipidemias (posch) al project described by long et al. (1991). unlike the rx project, which operates upon data not collected for the purposes of knowledge discovery, the posch al project operates upon controlled data derived from a clinical test. therefore whilst the rx project is capable of operating upon existing data, it must accommodate greater amounts of noise and errors than the posch al project. as a result, less significance can be assigned to the findings of the rx project because the objectivity and accuracy of the data is unknown. causal relationships typically require a significant statistical proof, and therefore, once detected via knowledge discovery, may require additional investigation. despite this, kdd tools are useful for uncovering potential causal relationships in the first instance. expert guidance may also be utilised at a high level in suggesting possible relationships to be investigated, and scrutinising the results. once detected, detailed experiments can be set up to undertake a more thorough investigation of suspected causal relationships. temporal knowledge a key characteristic of kdd and data mining is the presence of a dynamic domain where data is typically updated on a regular basis. therefore it is often useful to examine the way that data and the knowledge derived from it are changing over time. trends, cycles and patterns may occur and their detection can be useful in analysing historic data and predicting future behaviour. importantly, these patterns can exist in both discovered knowledge and the underlying data. while temporal knowledge can describe a wide range of different types of 110 ajis vol. 6 no. 2 may 1999 rules derived from different types of data, the common component is the consideration of the temporal dimension and its influence on the behaviour of entities within the modelled domain. a common form of temporal knowledge is the existence of changes in derived rule sets over time. the detection of patterns within time series data has received significant attention. the types of data considered are typically numeric, continuous and use complex algorithms to detect patterns within a time series. as noted by keogh and smyth most approaches to solving this kind of problem require three fundamental components: (1) a technique to represent abstract shapes, (2) a distance measure for comparing two sequences, and (3) a mechanism for finding matching sequences within large time series databases (keogh and smyth 1997). a similar problem is the discovery of patterns of sequences in categorical data. because the data is discrete and with a typically limited number of values this problem can be somewhat less computationally expensive to solve. shapes can be represented as a sequence of domain values, the distance between sequences can be determined by comparing categorical values and the quality of the match between two sequences is determined accordingly. this is particularly interesting given that knowledge discovery has extended the application of machine learning techniques to everyday organisational databases in which categorical data is often found. padmanabhan and tuzhilin describe the use of temporal logic to discover pattern occurring in categorical data (padmanabhan and tuzhilin 1996). agrawal et al. propose a technique for comparing sequences and describe fast techniques for finding matching sequences. clustering knowledge clustering is a technique concerned with identifying clusters of instances within the domain space. clustering is a form of unsupervised learning that partitions observations into classes or clusters (collectively called a clustering) (fisher 1995). as noted by fisher, clustering approaches can be defined in terms of the way they evaluate clustering quality (objective function) and the way that they search the space of clusterings (control strategy). the unsupervised nature of clustering makes it applicable to applications where the user has limited domain knowledge. an example application is the clustering of web-search results. although several approaches to clustering exist and several different distance measures can be employed it has been demonstrated by bouguettaya et al. that in some circumstances many of these methods have similar performance (bouguettaya, viet and colea 1997). for the purposes of data mining, where large volumes of data are present it may therefore be more appropriate to choose clustering techniques based upon their efficiency. in addition finding an appropriate control strategy and objective function there is a need to find methods to determine the optimal number of clusters within the data. objects may be merged with nearest neighbours to form clusters, finding a stopping point for this process requires a technique that can estimate the optimal number of clusters in the data. smyth addresses this issue and introduces a new technique based upon monte carlo cross-validation for determining the optimal number of clusters (smyth 1996). smyth compares his proposed algorithm with several existing approaches, concluding that the monte carlo cross-validation method offers an alternative to other methods. zamir et al. (1997) describe the application of clustering has been applied to web document retrieval. ketterlin (1997) describes a bottom-up approach to clustering sequences of complex objects. this approach uses a hierarchical control strategy to find the least generalised covering cluster of the component objects in a sequence. clustering is widely applied to spatial databases where clustering can be used to group items of close proximity in physical space. some examples of spatial clustering are described in section 4.5. target data types most research into kdd has focused on the discovery of knowledge within the context of traditional database paradigms such as the relational model. however techniques designed for relational databases are likely to fully exploit only the relational types of knowledge implicit within other database models. likewise the type of data being mined may be multi-dimensional, textual or graphical. this section describes knowledge discovery from eight different data sources that are distinguished by the nature of the data itself and the characteristics of its storage. whilst we have limited our examination of data and its storage to eight major areas, kdd techniques are also applicable to other forms of data, for example czyzewski (1996) describes the application of data mining techniques to the removal of noise from audio data. in this example the audio signal is broken down into bands which are then masked out or left unchanged based upon their noise content. a rough set technique is used to derive rules determining which segments of the signal should be masked from a set of sample data. these rules are then applied to the entire audio signal. this approach has yielded positive results and indicates the potential utility of applying kdd techniques to audio processing applications. i l l ajis vol. 6 no. 2 may 1999 relational data relational databases are in widespread use throughout many organisations. within the relational model data are normalised into relations to eliminate redundancy. as a result, data required for the purpose of data mining may be stored in several relations. to retrieve this data, relations must be joined, and required attributes projected out. for the purpose of data mining it is typical to assume that the data being processed is stored in a single relation created through the appropriate operations. the data is therefore presented in a simple tabular form. most kdd tools developed to date are designed to operate on such tabular data sets and are therefore highly applicable to relational databases. importantly however the task of generating a single relation for knowledge discovery may involve a large amount of data cleaning and manipulation. object oriented data object oriented databases (oodb), have the structure to model complex objects found in application areas such as computer aided design (cad), software engineering and geographic information systems. this data typically contains hierarchies of objects and include concepts such as classes, inheritance, encapsulation and polymorphism, qv. nahouraii and petry (1991). in addition, objects have associated methods and communicate via messages. whilst the oodb model allows for flexibility in modelling entities, the resulting lack of uniformity may hinder the kdd process. importantly, oodb's such as gemstone, associate types with individual values, not the attribute fields they are stored in, (maier and stein 1990). therefore a field or slot in an object may potentially contain data of any type and structure. whilst stronger restrictions may be refined in the design process, the trade-off between uniformity and flexibility remains at the heart of the oodb paradigm. in general, where the data being stored is comparable to the data stored in relational databases, existing relational database techniques are applicable. however when more complex data exists, potential approaches to kdd become unclear. the complexity of object-oriented databases makes the application of kdd techniques a challenging problem. as part of the data filtering process existing knowledge discovery systems normally begin by selecting data from the target database in the form of one or more tables. therefore the major focus is upon learning from tabular data. whilst tabular data can be extracted from object-oriented databases for kdd purposes, methods for exploiting the more complex semantics associated with them remains a major challenge. to date little investigation has been undertaken in this area. however nishio etal. (1993) describe the 'first step' towards knowledge discovery in object-oriented databases. their work focuses on the extension of attributeoriented induction techniques to object-oriented databases. by utilising an attribute-oriented induction technique, the primary challenge to be overcome is the creation of generalisation hierarchies for the complex data types typically found within object oriented databases. the existence of set based attributes poses a problem for the construction of conceptual hierarchies as several approaches to generalisation are available. each item in the set can be generalised, and redundancy in the resulting set removed. alternatively a generalisation may be a description of the overall set, such as the number of items, or the mean value of the items. in addition, an object may inherit attributes from one or more parent objects, provided that these attributes can be retrieved. it should be noted that data mining is likely to be of most use in large datasets and while oodbs have high structural complexity, their data volumes have, at least to date, been comparatively low. transaction data much of the research associated with the generation of association rules assumes the availability of transaction data structured to list items association with a single commercial or other form of transaction. in the commonly used example, that of market basket analysis, each data record consists of those items that are purchased at the same time, together with optional additional information which may be of use. this is transformed into the data required. textual data the application of data mining techniques to textual documents to extract knowledge has become an increasingly interesting field with the advent of large textual databases. in recent years many models for the management of textual data have been proposed and developed, qv. loeffen (1994). the lack of any standard model can be attributed to the complex issue of storing and retrieving textual data with its associated semantics. different applications will inevitably require different semantics to be associated with textual data. for example, books may be structured into chapters, while a dictionary or thesaurus may be divided according to keywords. this lack of any uniform approach to the storage of textual data means that the development of general-purpose 112 ajis vol. 6 no. 2 may 1999 textual knowledge discovery tools is currently problematic and may be unrealistic. whilst tools that can learn from raw, unstructured text are the most versatile, the utility of tools that cannot exploit the full semantics available is reduced. this is a serious limitation given the importance of structure in organising text by flagging keywords or similar attributes. most documents are not created in a format where structure is explicit. however structures exist in many types of documents and structural components can be identified by consistent types of formatting. therefore these components can be identified within stored documents. ahonen et al. describe a technique for finding a small description of a documents structure, in the form of grammars once its components have been identified (ahonen, mannila and nikunen 1993). their approach begins by constructing weighted finite-state automata, which are then generalised. the automata are generalised in relation to each other and finally transformed into regular expressions. therefore if the components of a document can be identified, a grammar describing its structure can be derived. this may then be useful in indicating the type of knowledge associated with a document or conducting queries upon it. a major application of knowledge discovery within textual databases is the development of classification rules to automatically classify documents. the primary objective behind this is to allow the automatic selection of documents of interest within a specific domain. apte" et al. (1993) describe an approach to document classification, which begins by extracting relevant keywords within documents as attributes. these attributes, along with an appropriate document classification, are then used to induce a set of classification rules with an associated measure of classification accuracy. the extraction of classification knowledge is greatly aided by the availability of large numbers of examples. however large numbers of example documents can pose a processing problem that may necessitate the use of a random sampling technique. in addition, the number of classifications to be learnt is also typically high. the most promising approach where pre-classified examples exist is to develop a classification model for a single class of documents at one time, with all other documents being used as negative examples. a third problem is created by the size of the dictionary of attributes or keywords. apte" et al. advocate a simple elimination of less significant attributes based upon frequency of occurrence, where the most commonly occurring attributes are used. this rule-based approach has been applied to several large document databases with promising results. he"brail and marsais (1992) describe experiments on the analysis of research project descriptions. the experiments have been conducted within electricit6 de france, where more than 1,500 research projects are undertaken each year. this large number of projects has made summarising the overall research being undertaken within the organisation a complex task. the application described analyses textual project reports to provide management with an overview of its research activities. their approach utilises a custom thesaurus as a source of domain knowledge. the thesaurus contains over 13,000 keywords that are classified into almost 300 separate subject areas. keywords are also linked via a semantic net, which models synonymy, genericity, specificity and related topic relations. this custom thesaurus is an integral part of the data analysis system, and it is likely that the creation of such a resource would be a major undertaking, however once completed it would be of general use in other areas such as document retrieval. where keywords are associated with documents they can be treated as data and associations between the keywords that commonly occur together can be investigated. feldman and dagan (1995) describe the kdt system for knowledge discovery in text. this system operates upon texts marked with associated keywords. keywords are organised into conceptual hierarchies describing the associated domain. by examining concept distributions knowledge about the texts can be derived. for example, in economic newswire data it may be found that crop producing regions of the world feature disproportionately in articles discussing agricultural concepts. by measuring deviations between expected distributions of concepts and actual observations interesting relationships can be found and changes in distributions can be tracked over time. similarly feldman and hirsh (1996) describe the fact system for discovering associations from text documents in the presence of background knowledge. given a collection of documents with associated keywords, relevant background knowledge and a user-specified query the fact system finds all of the appropriate associations between the keywords. the learning process is similar to the association rule learning algorithm of agrawal et al. (1993). the fact system employs a user interface that allows queries for associations to be easily defined. importantly this system utilises background knowledge to restrict the search space and hence the incorporation of background knowledge improves the learning efficiency of the tool. the fact system has been successfully applied to newswire data. temporal data traditional databases store only the current state of the data, so that when new values become valid, old values are overwritten. this approach cannot model the way the entities represented in the database change over time. temporal databases overcome this limitation by not overwriting attribute values, but instead storing valid time 113 ajis vol. 6 no. 2 may 1999 ranges with them, which can be used to determine their validity at particular times, including the present23. this functionality is further extended by the fact that future values can be entered proactively in preparation for their impending validity, ie. before they become current. in traditional databases, reasoning about temporal information is restricted to comparisons between temporal valued attributes. temporal information in such systems is treated in the same way as other attributes. for example, lee et al. (1985) describe the application of temporal inference to administrative databases. for a survey of temporal semantics in information systems see roddick and patrick (1992). whilst much progress has been made in the development of temporal databases (tansel, et al. 1993), little progress, has been made towards the development of general-purpose temporal data mining systems. a discussion of some issues involved with temporal knowledge discovery is provided by rainsford and roddick (1996). likewise a theoretical framework for temporal knowledge discovery described by al-naemi (1994). within the context of bitemporal databases roddick describes a formal method of defining induced temporal relationships termed temporal induced dependencies in (roddick 1994), see also (roddick, craske and richards 1996). temporal induced dependencies are induced functional dependencies that are weakened by temporal conditions. these dependencies may only be valid at particular times, or may specify a temporal relationship between part of the functional dependency. temporal dependencies can be induced using techniques such as the characteristic rule learning algorithm described by han et al, (han, cai and cercone 1993). however this is only possible if a framework for generalising temporal intervals is provided. rainsford and roddick present a simple framework for facilitating the generalisation of temporal intervals in the context of attribute-oriented algorithms (rainsford and roddick 1997). a temporal database is not essential for temporal knowledge discovery. for example hoschka and klosgen (1991), describe the potential for limited temporal reasoning within the explora system. the suggested temporal reasoning is added by storing separate snapshots of the rule set over time. these rule sets can then be compared to draw conclusions regarding the change in data over time. this technique could be applied to any nontemporal database to allow some temporal reasoning. however because data is not stored within a temporal database, rules describing the change in the data over time can only be derived indirectly from changes in the stored rule set. because the snapshots are derived without any knowledge of temporal patterns existing within the data, many interesting temporal patterns may be lost. moreover the fact that only a limited number of views of the rule set are available restricts the reasoning capability of such a system. for example temporal behaviour such as cycles may not be detected if the time between rule sets is too great. in addition, the ability to find rules describing the change in data over time is dependent upon, and restricted by, the information held in the stored rule sets. for these reasons, the use of temporal databases in situations where temporal semantics are meaningful is arguably a better approach. in order to describe relationships between temporal events a taxonomy of temporal relationships is required. a widely used taxonomy of relationships between intervals is described by allen (1983). as an example of the role temporal relationships play consider the rule: the blackouts occurred after the peak period the above rule uses the temporal relationship after as an expression of the relationship between two intervals. the taxonomy of allen is generalised by freksa (1992), who describes an algebra based upon semi-intervals. this approach supports reasoning with partial knowledge, where only one endpoint is known. in addition, freksa's taxonomy allows the coarseness of temporal reasoning to be adjusted via neighbourhood relations to accommodate the information available. in addition to relationships between temporal events, there has been significant investigation into tools for finding pre-specified patterns within temporal data. for example, wade et al. (1994) describe a set-based approach to detecting temporal patterns in patient drug usage data. drug misuse can occur unwittingly, when a patient is prescribed two or more interacting drugs independently for usage within temporal proximity to each other. drugs that interact undesirably are recorded along with the time frame in the form a pattern that can be looked for within patient records. rules that describe such instances of drug misuse are then successfully induced based on medical administrative records. whilst wade et al. focus on the detection of patterns within tabular data, the detection of patterns within continuous data is more applicable to many domains. berndt and clifford describe the detection of patterns in time series data (berndt and clifford 1995). they adopt a dynamic time warping technique utilised in natural language processing. as stated by the authors, the discovery of patterns within time series data is a challenging 23 the three types of time that can be associated with an attribute in a database are: valid time; the time at which events actually take place within the modelled reality; transaction time; the time at which data is actually entered into the database, ie. the time transactions actually take place; and user-defined time; any attribute within the database which records time, for example, date fields etc. 114 ajis vol. 6 no. 2 may 1999 problem, and the development of general-purpose tools poses several problems. for example, the number of possible patterns a linear series can follow is potentially very large. therefore in order to develop tools to look for such patterns within data, a comprehensible and manageable set of patterns must first be defined. a shape definition language sdl is defined by agrawal el al. to describe patterns or shapes occurring in historical data (agrawal, et al. 1995). based upon sdl a query language for defining time series patterns and trends is described by agrawal and psaila (1995). one obvious limitation of such a query language is that linear patterns are most intuitively described visually, and typically textual descriptions involve the use of informal language. agrawal and psaila allow the user to create their own language with complex patterns being defined in terms of primitives such as up or down. there are two significant problems that are associated with pattern detection from time-series data, scale and proximity. the problem of scale is posed because searching for patterns at different resolutions will yield different results, and the choice of resolution is largely domain dependent. patterns that may be significant in one set of time series data may only be considered noise within a different domain. therefore the resolution at which to search for temporal patterns remains a parameter which must be specified by users. however patterns existing over different time scales may all be interesting and therefore methods of considering such issues must be investigated if automated tools are to be developed. the proximity of events in time determines if any significant relationship between them can be drawn. for example the fact that an engine failed after a power surge may not be interesting if the power surge occurred thirty years before the engine failed. if however the events occurred within moments of each other then the relationship may be significant. many application domains may involve delayed reactions and therefore determining an appropriate proximity is essential. windowing techniques are commonly employed to tackle the problem of proximity. a time window is an interval within which data is examined only in the context of the other data in the window. the rx project (blum 1982), utilises several sub-modules to discover causal relationships from temporal data within a medical domain. rx utilises non protocol and non randomised data or in other words, no special data acquisition techniques are assumed. the data used is from an operating database where data input is typically on a day to day basis. other notable contributions to temporal knowledge discovery include the work of mannila et al. who describe an algorithm for the discovery of frequently occurring episodes in sequences (mannila, toivonen and verkamo 1995). an extension of this work employing temporal logic is described by padmanabhon and tuzhilin (1996). spatial data spatial databases model multi-dimensional space and are typically found within geographical information systems (cis) (abraham and roddick 1998, 1999). the complexity of spatial data necessitates the development of special purpose data mining tools. it is important to note that spatial and non-spatial information will typically need to be integrated into any learning system. therefore, whilst existing tools and techniques remain partially valid, further extensions to accommodate spatial reasoning are required. although the application of kdd tools to spatial databases is relatively new, some work has already been undertaken. from this initial research, two main approaches to knowledge discovery within spatial databases have emerged. the first approach is based directly upon spatial operations, where properties such as distance and proximity are used directly within discovery, in conjunction with techniques such as clustering. clustering can be defined as the process of grouping physical or abstract objects into classes of similar objects (chen, han and yu 1996). the second approach is based heavily upon the relational model, where spatial attributes are converted via spatial operations into corresponding attributes and these attributes are processed using largely conventional techniques with extensions to include spatial operators. bell et al. (1994) describe their experience of data mining within spatial databases. their technique applies the dempster-shafer theory of evidence to the location of volcanoes on venus. volcanoes are located from images captured by the magellan-venus space probe and evidential reasoning is used to combine the spatial evidence from multiple images. images differ in quality due to the angle at which they were taken and their resolution. with evidential theory, the evidence provided by each of the images can be assigned a weighting to reflect the image quality. this approach deals with images representing spatial data, however other techniques may be required for knowledge discovery within spatial databases where the spatial data is represented in other ways. rule discovery mechanisms used for non-spatial data can potentially be extended to include spatial information. for example, koperski and han describe spatial association rules (koperski and han 1995). one of the important points highlighted in their work is that existing spatial processing functions should be exploited where possible and existing techniques for relational data are often useful within spatial knowledge discovery. they define spatial association rules and provide an algorithm for their top-down induction within relational based gis systems. spatial association rules can be loosely described as conventional association rules as defined by 115 ajis vol. 6 no. 2 may 1999 agrawal et al. (1993), with at least one spatial attribute and the potential for spatial predicates such as, close-to, within or next-to describing the association within the rule. therefore a simple example may be: is_a(x, city) a within(x, australia) -» close_to(x, coast) spatial relationships need to be defined by domain experts, and may have various interpretations at different levels of abstraction, which also need to be defined. for example, a country may be defined as being close-to a city if it is 50 km away, but a backstreet may not be defined as being close-to a school if it is also 50 km away. therefore depending upon the level of abstraction being discussed, spatial terms need to be re-interpreted. whilst this approach provides support for natural language semantics, it creates the potential for ambiguity and a requirement for an increased level of expert input. the koperski and han algorithm exploits pre-defined conceptual hierarchies to conduct top-down induction. initially the spatial data of interest is extracted and processed to find support for any associations fitting the userspecified pattern. strongly supported rules at this high level of abstraction are then identified. more specific rules can then be derived from the high level set by descending the conceptual hierarchy and these rules tested for adequate strength within the database. the required minimum support threshold can be lowered for more specific rules, where less data is available. by continuing down the conceptual hierarchy, seeking more specific rules from the previous rule set, association rules at all levels can be discovered. this approach enables the discovery of rules at various levels of abstraction. it also reduces processing costs, by eliminating large sections of uninteresting data at a high level of abstraction, and avoiding futile searches for strong rules. however this approach relies upon detailed domain knowledge in the form of conceptual hierarchies and the definition of relationships at multiple levels. lu et al. also employ conceptual hierarchies in the induction of characteristic rules from spatial data (lu, han and ooi 1993). they describe basic algorithms for attribute oriented induction of spatial and associated nonspatial data. the induced rules characterise the non-spatial properties and relationships of spatial objects, and the learning process is initiated by a user learning request that may focus on attributes of interest. spatial generalisation may be performed with the use of existing conceptual hierarchies, clustering techniques or even spatial indexing structures. generalisation is performed on spatial and non-spatial data, and the order in which this is done effects the resulting generalisation rules. this leads to the specification of two basic algorithms, the non-spatial-data-dominated algorithm that generalises non-spatial attributes first, and the spatial-data-dominated, which generalises spatial attributes first. the possibility of algorithms for interleaving spatial and non-spatial generalisation is also discussed. an alternative technique for knowledge discovery in spatial databases is the application of clustering techniques. ng and han (1994) describe clarans, a clustering technique designed for spatial data mining. the effectiveness of clarans, and the techniques it is based upon, rests strongly in its ability to determine the central representative object or mediod for each cluster. this process involves testing candidate mediods against their neighbouring objects. once this first step has been performed other objects can be assigned to the appropriate cluster based on their proximity to the mediod. however an exhaustive search for the best mediod within the dataset is impractical within very large datasets. rather than look for the mediods within sample sets of the data, clarans works with the complete dataset, but only tests a mediod against samples of neighbouring objects. both a spatial dominant version sd(clarans), and a non-spatial dominant version, nsd(clarans) are described. the spatial dominant version performs clustering of spatial attributes before applying the dblearn system (han, cai and cercone 1993), to the non-spatial attributes associated with each cluster. this approach provides a non-spatial description of the spatial clusterings. the non-spatial dominant approach applies dblearn algorithms to the non-spatial data first, before applying clarans to the spatial attributes associated with each of the generalised tuples. this approach reveals spatial clusterings existing within groupings of non-spatial items. clarans is further investigated by ester et al. who describe the efficient application of this clustering algorithm to spatial databases, (ester, kriegel and xu 1995). they describe performance enhancement techniques showing a slightly reduced effectiveness for a large gain in efficiency. such a trade-off is likely to be highly desirable in applications involving very large volumes of spatial data. they apply their technique to a large protein database, to aid in the identification of similar protein surfaces. in addition, ester et al. point out that clustering techniques are not dependent upon domain knowledge. combinatorial data combinatorial data contains complex objects such as trees, graphs and sequences that often exist in databases describing sophisticated domains such as circuit layouts, molecular structures or computer code. one application for knowledge discovery tools in combinatorial domains is to discover generalisations of data, by finding substructures which occur repeatedly and can be substituted and hence reduce the volume of the encoded 116 ajis vol. 6 no. 2 may 1999 data. finding similarities between complex objects is often too difficult to perform by inspection and this can also be performed using knowledge discovery techniques. the emphasis in combinatorial data mining is upon identifying patterns in the underlying structure. djoko et al. describe subdue a tool for the use of domain knowledge in the discovery of substructure within combinatorial data (djoko, cook and holder 1995). the input data can be substituted with a pointer to the appropriate substructure. statistical measures of the goodness of fit between the substructure and the input data determine if the fit is sufficient to support the substitution of the input data with the pre-defined substructure. once discovered, these substructures can be used to form a compressed representation of the original data or employed for the purposes of providing an overview of what structures exist in the data. substructures are modelled in a hierarchy and occur at multiple levels. therefore they can be generalised and decomposed into other substructures within the hierarchy. the discovery of similar substrings in protein sequences is discussed by wang et al. (1994). their approach is based upon the initial selection of appropriate patterns from a sample of the database. these patterns are then compared against the rest of the database. their technique can find non-consecutive patterns separated by arbitrary lengths without prior knowledge of their structure or occurrence. the diversity of structure than can be potentially found in combinatorial data is limited only by the application domain. therefore techniques for combinatorial knowledge discovery may require application specific approaches. data from the internet the exact manner in which knowledge discovery can be applied to the internet is still an area open to investigation. the standards that exist within the internet are largely associated with data presentation and navigation, while issues related to storing large volumes of data are comparatively overlooked. the enormous volume of data resources makes the internet a useful target and source of input for knowledge discovery. however the lack of standardised storage formats, the wide diversity of data types and the wide distribution of data across the internet hinders general-purpose knowledge discovery over the internet. one possible solution is to construct multi-level databases upon the raw data; an approach described by han and fu (1994). by proving a generalised summary of the data at a high level of abstraction, tools could locate information of interest quickly and then perform knowledge discovery upon the low-level data. however the extent to which such an approach will be universally adopted is debatable. another approach could be the use of intelligent agents capable of learning from diverse sources of data and then collaborating with each other in the summation of knowledge and presentation of results. little investigation has been undertaken into the application of intelligent agents to knowledge discovery but davies et al. examine an intelligent agent approach towards data mining using first order logic (davies and edwards 1995b). one application of data mining to the world wide web is the analysis of usage logs to characterise user behaviour patterns. once specific user groups have been identified, websites can adapt to offer them customised information and hyperlink pathways through the data. chundi and dayal (1997) describe a technique for providing a list of links for web users based upon previous access patterns. the approach uses clustering techniques to classify website clients into various user groups based upon their access behaviour. once identified as belonging to a specific group the tool can offer further links based upon links most likely to be useful. the approach contains an adaptive component that can adjust the links presented over time as new behaviour patterns are observed. although this approach is limited to presenting prospective links, it could be extended to provide alternative information, details and customised advertising offers, based upon the user profile. data from data warehouses in a traditional database environment the processing to answer queries and support analysis is conducted as the user requests arise. this may involve the integration of data from multiple databases and information sources. this data may subsequently require reformatting and adjustment to create a single integrated source of data from which user analysis can then be conducted. as both the volume of data and the value of information has grown the importance of analysing and exploiting organisation data has increased. the desire to integrate and hence analyse data more efficiently has led to the emergence of data warehousing technology. defined simply a data warehouse is an analytical database that is designed for large volumes of read-only data, providing intuitive access to information that will be useful,in making decisions (fong and zeng 1997). data warehouses store integrated data in predefined formats specifically selected to support user queries and analysis. this data is typically gathered from multiple databases and information sources and may contain historical data as well as metadata. the rapid adoption of data warehouse technology by organisations means that much knowledge discovery is now likely to be undertaken in the context of data warehouses. 117 ajis vol. 6 no. 2 may 1999 in an effort to support data analysis data warehouses store information that can be utilised by knowledge discovery tools. inmon identifies four types of data characteristic of a data warehouse that may be useful for knowledge discovery (inmon 1996). firstly, integrated data in a data warehouse reduces the need to combine, clean and reconstitute data that may originate from multiple sources and hence allow speed up the knowledge discovery process. secondly, detailed and summarised data allows data mining tools the option of analysing data at various levels of abstraction without needing to perform any generalisation. thirdly, historical data supports longitudinal analysis such as the detection of trends and cycles within the modelled domain. fourthly, metadata provides useful contextual information that can aid in the knowledge discovery process. all of these data types may be derived independently however their presence in a data warehouse would avoid the need to create them hence saving time in the preparation of data for the knowledge discovery process. in the same way that existing kdd tools can exploit the database management system, and query languages of existing databases, kdd tools can also exploit the technologies associated with data warehouses such as olap (online analytic processing) and data cube technologies. kamber et al. describe the use of data cubes for metarule-guided mining of association rules (kamber, han and chiang 1997). the metarule in this context refers to a template that acts as a pattern filter, restricting the search space. when a data cube exists the large one item itemsets can be found by examining the 1-d aggregation layer of the cube. based upon these large itemsets candidate two item large itemsets can be constructed and tested by examining the 2-d aggregation layer. this process continues until itemsets with the same number of items as appearing in the metarule have been discovered. from here the large itemsets can be evaluated as rules. this approach avoids multiple scans over the entire database by utilising the summary information held in the data cube. overall it can be seen that data warehouse technology provides an improved environment for knowledge discovery processes and it is likely that many kdd tools will be adapted to exploit this. the utilisation of discovered knowledge both data mining and kdd are usually associated with decision support and knowledge base creation. however other opportunities exist for the exploitation of discovered knowledge. for example, knowledge can be used for the detection of inconsistencies and integrity enforcement. other applications include semantic query optimisation and the discovery of hidden structures or dependencies within the data, which may lead to database restructuring. the choice of application largely dictates the nature of discovered knowledge and hence the discovery processes. this section will discuss four main applications of discovered knowledge and provide examples of each. detection of inconsistencies and enforcement of integrity within databases the detection of inconsistencies and enforcement of integrity within databases can be partially automated with the use of inductive techniques. semantic rules can be induced from the target database and any violation of these rules can then be flagged as an exceptional occurrence, and appropriate warnings can be activated. as pointed out by schlimmer, a major strength of this approach to integrity enforcement is that it can be applied to domains where no domain expert is available (schlimmer, mitchell and mcdermott 1991). furthermore, the integrity rules can be automatically updated periodically to reflect changes in the database over time. schlimmer (1993) describes carper, an inductive learning tool used to maintain integrity within databases. this tool combines both learnt and given knowledge to construct attribute range/value models. these models can be used to detect inconsistencies in database entries. in addition existing database entries can be checked for validity against the rest of the database. carper constructs a single decision path for the entry being checked. the adaptation includes the generation of multiple trees where two attributes are equally appropriate for branching. this reduces the number of false alarms also reduces the number of actual violations detected but to a lesser degree. therefore the choice of using a single tree or multiple tree in such situations, should be determined by the nature of the application. kamel describes the use of an expert system shell as a front end to a database to test the integrity of updates (kamel 1995). this simplistic approach allows rules in the expert system to enforce integrity upon data entered into the database. rules for such a system could be automatically generated using a rule induction technique. however for such a system to be practical the rules would need to be maintained consistently over time to ensure they facilitate changes in the database over time. it is worth noting that active databases could be employed usefully for tasks such as the detection of inconsistencies and integrity enforcement. active databases can respond automatically to the entry of unusual data by notifying the system user of the unusual nature of the data. the entry of some quantity of new data can be seen as an event. if any inconsistencies are found between this data and the rules derived previously via data mining then this can activate a trigger. the action taken by the trigger could be to notify the user of the 118 ajis vol. 6 no. 2 may 1999 exception nature of the data in question. an application of active databases in knowledge discovery is described by agrawal and psaila (1995) who describe the application of triggers to detect trends in rules describing the application domain. important changes in the application domain can be automatically flagged for users attention. there is no reason however why a similar approach cannot be adopted for integrity enforcement. the major advantage of this approach is that users are not required to manually conduct regular analysis to detect interesting behaviour in the rule set and underlying data. semantic query optimisation semantic knowledge about the contents of a database can be useful in the optimisation of queries. rules that are known to hold in the database can be used to transform queries to improve performance, a process known as semantic query optimisation. semantic query optimisation can be seen as a two-phase process (siegal, sciore and salveter 1991). firstly possible transformations of the original query to produce different, semantically equivalent queries must be found. these transformations are performed based upon available rules or semantic knowledge and it is this knowledge that might be derived by knowledge discovery. the optimisation process must then determine which query will has the lowest execution cost as an example consider the following database relation noting that there is an index on the department attribute: staff (id, name, department, building, room no) consider a simple query that requests a list of all lecturing staff in the mitchell building. select staff.name where staff.building = 'mitchell' if we have a rule that states that all staff in the mitchell building are from the computing department, we may then transform the query into the following query: select staff.name where staff.department = 'computing' and staff.building = 'mitchell' in this transformation of the original query we then may utilise the index structure to speed up the query by quickly selecting only the computing staff. as noted by siegel el al, no methodology for the specification of useful semantic rules by experts has been developed (siegal, sciore and salveter 1991). therefore it cannot be guaranteed that the rules specified by experts would be optimal for semantic query optimisation. moreover the maintenance of semantic rules may place an impractical burden on domain experts as databases are constantly changing. the automatic generation of useful semantic rules is therefore a promising application of knowledge discovery techniques. anand et al. (1994) describe state-aware query optimisation. this approach aims to use discovered semantic knowledge to reformulate queries to better utilise available resources. the system is described as state-aware, because it reformulates queries to reflect the hardware resources currently available on the system. the strip algorithm (anand, et al. 1995a) is used for the data mining component. siegel et al. (siegal, sciore and salveter 1991; siegel, sciore and salveter 1992) describe the automatic derivation of rules for semantic query optimisation. this process first describes the characteristics of desirable rules. a search of the database is then undertaken to see if such rules can be derived. the explicit definition of the type of rule being searched for confines the search space and this consequently improves the efficiency of the search process. as noted by the authors the search for such rules is favoured by the availability of a definition of the rule type required and a means of evaluating the utility of the result. yu and sun (1989) describe an approach to semantic query optimisation that makes use of previously executed queries as a source of knowledge. they define the difference between static integrity constraints and dynamic integrity constraints. static integrity constraints are those constraints known to exist permanently within the current operation of an organisation. dynamic integrity constraints are constraints that currently hold true for the data, but may not remain true in the future. given that learning and storing rules for semantic query optimisation uses system resources it would be beneficial to concentrate resources upon learning semantics that are useful in answering queries. one approach to this is to have a query driven approach that learns semantic rules based upon user queries. hsu and knoblock describe a system for semantic query optimisation that uses user query patterns to direct the learning of semantic optimisation rules. expensive queries trigger the learning process and hence the search for optimisation rules. therefore the knowledge available for semantic query optimisation will match user query patterns. importantly this approach will allow complex joins across multiple relations to be used for semantic query optimisation. because the learning process is triggered by user queries then superfluous joins will not be investigated. testing 119 ajis vol. 6 no. 2 may 1999 has shown that this technique is effective however a mechanism for updating outdated semantic optimisation constraints is still being investigated. knowledge base creation for expert systems discovered knowledge can be used to construct knowledge bases that can then be exploited to create expert systems. an expert system can be described as a computing system capable of representing and reasoning about some knowledge-rich domain ... with a view to solving problems and giving advice (jackson 1986). the automatic creation of knowledge bases is attractive for several reasons. in situations where human experts are unavailable knowledge bases can be constructed from data sets. automatically discovered rules can also be used to verify rules proposed by human experts. the use of data to construct knowledge bases can also overcome problems encountered when trying to extract information from human experts such as efficiency and objectivity. moreover, because databases may contain vast numbers of previous records they can be seen as a valuable source of data from which expert system algorithms can learn. a distinction should be made between the two extremes of learning from sets of specially selected training data and learning from databases. many researchers have successfully derived expert systems from prepared training data. for example, carter and catlett describe the use of machine learning to derive various decision trees to assess credit card applications, (carter and catlett 1987). the experimental decision trees were created using id3 and c4, and displayed better accuracy than the existing organisational categorisation technique. by comparison, extracting expert knowledge directly from databases poses greater problems because of the need to deal with comparatively unprocessed data, and has received less attention to date. one example of a system that can be used for deriving expert system rules from a database is the rulearn® system described by koch and fehsenfeld (1995). taking a large set of data, this system characterises the data into a small number of rules, describing the dependencies existing in the database. in addition to the creation of new knowledge bases it is also possible to use kdd techniques to maintain and update existing knowledge bases. for example schlimmer et al. describe cobble, a system to aid in the development and refinement of knowledge bases (schlimmer, mitchell and mcdermott 1991). cobble is used to test the conditional aspect, or left hand side of rules, determining if they are too specific. the cobble system uses both the knowledge base being scrutinised and appropriate supplementary or background information such as domain terms or task-independent facts. within cobble, knowledge base rules are tested by comparing them for consistency with background knowledge. generalised conditions for a rule are derived and these are compared with the existing conditional component of the rule. any unnecessarily restrictive conditions are then removed or replaced with more general ones. the use of discovered rules for decision support a final application for the use of discovered knowledge is as a resource for decision support. decision support typically involves ad-hoc analysis of data that is guided by the user in an exploratory way. the facilitation and utilisation of user guidance is therefore a significant factor for decision support tools. many existing kdd tools are well suited to decision support in this respect because they allow users to direct the search for knowledge. conventional decision support tools commonly allow decision-makers to query data directly and conduct what-if analysis. kdd tools complement the power of decision support systems by allowing the search for more complex patterns and semantic relationships to be undertaken. the selection of an appropriate kdd tool depends upon the nature of the database to be utilised and the nature of the desired knowledge. association rule learning systems such as ais (agrawal, imielinski and swami 1993) are likely to be appropriate for the analysis of supermarket transaction databases. similarly the classification rule learning algorithm of cai et al. (1990) may be applicable to the analysis of a database of insurance claimants. highly autonomous kdd tools may potentially pre-empt decision making by identifying opportunities from relevant organisational data. with this in mind, another application of discovered knowledge might concern the decisions made by database administrators. knowledge concerning hidden structures or dependencies within the database can potentially support or pre-empt decisions concerning database re-structuring. for example the identification of previously unknown functional dependencies can lead to a restructuring of the database to reduce redundancy and ensure integrity. security implications of knowledge discovery as the reasoning power of induction within databases has become widely acknowledged, the potential misuse of inductive learning tools to induce restricted information has become an area of concern. organisations may unwittingly provide classified information implicitly within non classified data. at the same time security of 120 ajis vol. 6 no. 2 may 1999 data within databases is increasingly important as sensitive information is stored within them. in a progress report on kdd, piatetsky-shapiro (1994) includes privacy of data and related ethical and legal issues in a list of difficulties to be overcome within kdd. with the worldwide proliferation of electronic information systems and databases, the disclosure of data has received increasing examination both legally and ethically. however issues surrounding the social and legal implications of emerging kdd technology remain largely unresolved. o'leary points out that whilst traditional approaches to database security address the unauthorised access of data, they do not address the unauthorised acquisition of knowledge from data (o'leary 1991). as a consequence authorised users may use the data that they have access to in attaining knowledge restricted from them. he also warns that confidential information relating to decision making could potentially be induced from examples of past decisions. such information could help criminals avoid detection, allow competitors to preempt decisions or expose confidential decision making criteria. given these concerns o'leary argues that the threat posed by kdd technology to organisational security warrants specific counter-measures. traditional techniques to avoid intrusion remain useful as induction can be performed upon data obtained with or without permission. such techniques could however be extended where required to detect suspicious activities such as the dumping of large numbers of records, which may be used for inductive techniques. in addition o'leary also points out that approaches which exploit the weaknesses and limitations of existing inductive tools should be investigated. application area astronomy database engineering finance geology insurance law marketing and sales medical physics sport world wide web example applications sky survey analysis (fayyad, weir and djorgovski 1993). semantic query optimisation (siegal, sciore and salveter 1991), integrity enforcement (schlimmer 1993). discovering missing semantics (li, huang and chen 1997). scheme discovery (miura and shioya 1997). automotive quality control (wirth and reinartz 1996). semiconductor fault diagnosis (saxena 1993). credit assessment (feelders, le loux and van't zand 1995). stock market analysis (ziarko, golan and edwards 1993). earthquake detection and measurement (stolorz and dean 1996). premium setting, workflow analysis (keats and loo 1997). behaviour patterns in health insurance (viveros, nearhos and rothman 1996). fraud detection (shortland and scarfe 1994). quick market intelligence (alexander, bonissone and rau 1993)'. market survey analysis (ciesielski and palstra 1996). insurance marketing analysis (keats and loo 1997). rna study and analysis (hofacker, et al. 1996; wang, el al 1996) drug side effect detection (wade, etal. 1994). discovery of causal relationships (blum 1982). diagnosis of acute abdominal pain (provan and singh 1996). diagnosis of headache and facial pain (tsumoto and tanaka 1996). spectral data analysis (buntine and patel 1995). basketball game analysis (bhandari, et al. 1997). analysis of usage patterns and hyperlink customisation (chen, park and yu 1996; chundi and dayal 1997). table 2. kdd application areas the threat to security imposed by inductive techniques need not imply the use of kdd tools. miller describes the way that data within a statistical database can be compromised when supplementary knowledge exists (miller 1991). a statistical database typically contains knowledge made public on the condition that individual subjects from whom the information has been derived are not identified with the data. this form of anonymity is considered essential when dealing with detailed surveys such as census data however supplementary knowledge can compromise this. supplementary knowledge can be loosely defined as background knowledge known by the user. miller argues that supplementary knowledge is a major threat to database security. this argument is 121 ajis vol. 6 no. 2 may 1999 supported by actual examples of database compromises, reached through the application of supplementary knowledge to statistical databases. that is to say that where a system user has background information about a subject they may use this to isolate that subject and thereby removing their anonymity. she defines a classification system for various types of supplementary knowledge and compromises. what data is used for knowledge discovery and how the results of knowledge discovery are used are the two main security issues. the source of data used for knowledge discovery and the consent of its use by the individuals concerned are significant factors in determining its sensitivity. klb'sgen distinguishes between primary data collected explicitly and secondary data collected as a by-product of other transactions (klosgen 1995a). the application of knowledge discovery techniques to secondary data collected for purposes other than knowledge discovery is a particular concern. klosgen also highlights the relative lack of restrictions placed upon the handling of sensitive data by private organisations. this is in contrast to government agencies that often have high levels of self-regulation imposed upon data collected for official purposes. klosgen proposes an architecture for knowledge discovery systems where the data source is isolated from the analytical components of the kdd tool by a data management component. this data manager enforces appropriate security restrictions on the data passed onto the analysing components of the knowledge discovery system. application domains knowledge discovery in databases has been applied in many application areas where the volume of data is large. we provide a table of application areas updated from the table provided by frawley et al. (1991) see table 2. while this list of application areas is impressive, the potential for the application of data mining technology to other areas still remains. as databases continue to grow in size and new databases are created, the motivation to automatically extract implicit knowledge will continue to grow. it is also likely that in some application areas automatic discovery will become necessary as data volumes make manual analysis infeasible. future research much work has already been undertaken in the development of data mining systems. however challenges remain, four of which are discussed below. interestingness the problem of specifying the interestingness desirable in required rules. a system that constantly reports useless facts is unlikely to be popular amongst users. it should be noted however that for some applications, such as semantic query optimisation, the notion of what is interesting is complicated as rules which exist in the data, meaningful or not, can often be useful for optimising operations restricted to that same set of data. whilst the search for interesting rules is clearly the objective, the problem of differentiating between interesting and non-interesting rules remains largely unsolved. nevertheless, some suggestions have been made (piatetsky-shapiro 1994), and the issue of interestingness in knowledge discovery is discussed by silberschatz and tuzhilin (1996). they distinguish between objective and subjective measures of pattern interestingness. objective measures relate to the structure and statistical strength of the pattern while subjective measures also consider the user viewing the patterns. this distinction is important because while objective measures are easily applied to a set of patterns, subjective measures are not easily defined. silberschatz and tuzhilin propose unexpectedness and actionability as fundamental characteristics that make patterns interesting to users. unexpectedness is a measure of how the discovered pattern differs from the user's beliefs and hence how surprising it is. actionability measures the ability of an interesting piece of knowledge to be acted upon by the user. they also argue that actionability and unexpectedness typically occur together in patterns. based upon this assumption, it is sufficient to evaluate patterns on the basis of unexpectedness and avoid difficult measures of actionability. modelling the users current beliefs considers both hard beliefs that are fixed and soft beliefs that may change over time. by measuring the deviation of newly discovered knowledge from the users current beliefs a relative measure of interestingness can then be derived (srikant and agrawal 1995). user dependence a major limitation of existing systems is their reliance upon user input and direction. this dependence of data mining systems upon the user comes in two forms. firstly, there is still a heavy dependence of data mining tools upon available domain knowledge. such information often needs to be specified explicitly by domain experts. 122 ajis vol. 6 no. 2 may 1999 the development of techniques to automatically acquire domain knowledge, or facilitate its acquisition may offer one partial solution to this problem. secondly, kdd tools require substantial run-time support from the user. ideally, kdd tools could operate more or less autonomously in the background, presenting interesting results as they are found, and applying this new knowledge where appropriate. one major reason why more fully automated tools have not been developed is the question of defining interestingness mentioned above. the use of intelligent agents to discover and report interesting findings to users is an area of significant interest. davies and edwards have investigated this area and developed an initial framework for the operation of agent based data mining (davies and edwards 1995a, 1995b). their approach focuses on distributed agents co-operating to find interesting knowledge. other application domains apart from overcoming existing limitations, the potential to extend kdd tools to more diverse application domains, is a major focus of the current effort. some issues related to this are listed below. • the diversity of data being stored necessitates the development of specific techniques optimised to particular data types and data storage models. the development of tools capable of fully exploiting the associated semantics remains a challenge in areas such as temporal, spatial and object oriented databases. • there has been little investigation of hybrid or co-operative systems employing multiple paradigms for machine learning. because different learning paradigms have different strengths and weaknesses, it is likely that a system employing more than one paradigm may be more effective than a single paradigm approach. • whilst many applications for discovered knowledge exist, most kdd systems only incorporate support for one or two. systems that can support many applications, making efficient use of discovered knowledge by using it in multiple ways, are still to be fully investigated. in summary, both opportunities and challenges remain within the broad field of kdd. as might be expected, many of these issues are currently the focus of current research efforts. it is likely that as the trend towards larger and more semantic rich databases continues, the opportunities for kdd research and application and prototype development will grow. in the longer term, it is highly likely that the development and use of appropriate kdd techniques will become a commercial necessity for competitive performance. references abraham, t. and roddick, j. f. (1998). "opportunities for knowledge discovery in spatio-temporal information systems". australian journal of information systems. vol. 5, no. 2, pp. 3-12. abraham, t. and roddick, j. f. (1999). "survey of spatio-temporal databases". geoinfonnatica. vol. 3, no. 1, in press. agrawal, r.. et al. (1992). "an interval classifier for database mining applications". eighteenth international conference on very large data bases, vancouver, canada, pp. 560-573. agrawal, r., et al. (1993). "mining association rules between sets of items in large databases". acm sigmod international conference on management of data, washington dc, usa, acm press. vol. 22. pp. 207-216. agrawal, r., et al. (1996). "the quest data mining system". second international conference on knowledge discovery and data mining, portland, oregon, aaai press. agrawal, r. and psaila, g. (1995). "active data mining". first international conference on knowledge discovery and data mining (kdd-95), montreal, quebec, canada, aaai press, menlo park, california. pp. 3-8. agrawal, r.. et al. (1995). "querying shapes of histories". twenty-first international conference on very large databases (vldb '95), zurich, switzerland, morgan kaufmann publishers, inc. san francisco, usa. pp. 490-501. agrawal, r. and srikant, r. (1994). "fast algorithms for mining association rules". twentieth international conference on very large data bases, santiago, chile, pp. 487-499. ahonen, h., et al. (1993). "forming grammars for structured documents". knowledge discovery in databases: 1993 aaai workshop, washington, d.c., aaai press. al-naemi, s. (1994). "a theoretical framework for temporal knowledge discovery". international workshop on spatio-temporal databases, benicassim, spain, pp. 23-33. alexander, w. p., et al. (1993). "preliminary investigations into knowledge discovery for quick market intelligence". aaai-93 workshop on knowledge discovery in databases (kdd '93), washington d.c., aaai press, menlo park, california, pp. 52-60. allen, j. f. (1983). "maintaining knowledge about temporal intervals". communications of the acm. vol. 26, no. 11, pp. 832-843. 123 ajis vol. 6 no. 2 may 1999 anand, s. s., et al. (1994). "database mining in the architecture of a semantic pre-processor for state-aware query optimization". aaai-94 workshop on knowledge discovery in databases (kdd '94), seattle, anand, s. s., etal. (1995a). "data mining in parallel". wotug-18, anand, s. s., et al. (1995b). "evidence based discovery of knowledge in databases". iee colloquium on knowledge discovery in databases, anand, t. and kahn, g. (1993). "opportunity explorer: navigating large databases using knowledge discovery templates". aaai-93 workshop on knowledge discovery in databases, washington d.c., aaai press, pp. 45-51. angluin, d. and smith, c. h. (1983). "inductive inference: theory and methods". acm computing surveys. vol. 15, no. 3, pp. 237-269. apte", c., et al. (1993). "knowledge discovery for document classification". aaai-93 workshop on knowledge discovery in databases, washington d.c., aaai press, pp. 326-336. asa, j. a. m. and mangano, j. j. (1995). "selecting among rules induced from a hurricane database". journal of intelligent information systems. vol. 4, no. 1, pp. 39-52. beadle, n. c. w., et al. (1986). flora of the sydney region. new south wales, reed books. bell, d. a., et al. (1994). "data mining in spatial databases". international workshop on spatio-temporal databases, benicassim, spain, berndt, d. j. and clifford, j. (1995). "finding patterns in time series: a dynamic programming approach", in advances in knowledge discovery and data mining. aaai press/ mit press, pp. 229-248. bhandari, i., et al. (1997). "advanced scout: data mining and knowledge discovery in nba data". data mining and knowledge discovery. vol. l,no. l,pp. 121-125. blum, r. l. (1982). "discovery, confirmation, and incorporation of causal relationships from a large timeoriented clinical database: the rx project", in computers and biomedical research. vol. 15. pp. 164187. bouguettaya, a., et al. (1997). "a comparison of group-based and object-based data clustering techniques". eighth international database workshop, data mining, data warehousing and client/server databases., hong kong, springer-verlag singapore, pp. 119-136. buntine, w. l. and patel, t. (1995). "intelligent instruments: discovering how to turn spectral data into information". first international conference on knowledge discovery and data mining (kdd-95), montreal, quebec, canada, aaai press, menlo park, california, pp. 33-38. cai, y., et al. (1990). "an attribute-oriented approach for learning classification rules from relational databases". sixth ieee international conference on data engineering, los angeles, ca, ieee computer science press, pp. 281-288. carter, c. and catlett, j. (1987). "assessing credit card applications using machine learning". ieee expert no. fall, pp. 71-79. chan, k. c. c. and wong, a. k. c. (1991). "a statistical technique for extracting classificatory knowledge from databases", in knowledge discovery in databases. cambridge, ma, aaai press/mit press, pp. 107-123, (ch. 6). chen, m.-s., et al. (1996). "data mining: an overview from database perspective". ieee transactions on knowledge and data engineering. vol. 8, no. 6. chen, m. s., et al. (1996). "data mining for path traversal patterns in a web environment". sixteenth international conference on distributed computing systems, pp. 385-392. chundi, p. and dayal, u. (1997). "an application of adaptive data mining: facilitating web information access". 1997 sigmod workshop on research issues on data mining and knowledge discovery, arizona, usa, pp. 31-37. ciesielski, v. and palstra, g. (1996). "using a hybrid neural/expert system for data base mining in market survey data". second international conference on knowledge discovery and data mining (kdd96), portland, oregon, aaai press, menlo park, california, pp. 38-43. czyzewski, a. (1996). "mining knowledge in noisy audio data". second international conference on knowledge discovery and data mining (kdd96), portland, oregon, aaai press, menlo park, california. pp. 220-225. davies, w. and edwards, p. (1995a). "distributed learning: an agent-based approach to data-mining". ml95 workshop on agents that learn from other agents, davies, w. h. e. and edwards, p. (1995b). "agent-based knowledge discovery". aaai spring symposium on information gathering from heterogeneous, distributed environments., aaai press, menlo park, ca, u.s.a. pp. 34-37. djoko, s., et al. (1995). "analyzing the benefits of domain knowledge in substructure discovery". first international conference on knowledge discovery and data mining (kdd-95), montreal, quebec, canada, aaai press, menlo park, california, pp. 75-80. 124 ajis vol. 6 no. 2 may 1999 elmasri, r. and navathe, s. (1989). fundamentals of database systems. redwood city, ca, benjamin/cummings. ester, m., el al. (1995). "knowledge discovery in large spatial databases: focusing techniques for efficient class identification". fourth international symposium on large spatial databases, maine, fayyad, u., m., et al. (1996a). "from data mining to knowledge discovery: an overview", in advances in knowledge discovery and data mining. aaai press/ mit press, pp. 1-34. fayyad, u., m., et al., ed. (1996b). advances in knowledge discovery and data mining. menlo park, california, aaai press. fayyad, u. m., et al. (1993). "automated analysis of a large-scale sky survey: the skicat system". aaai93 workshop on knowledge discovery in databases (kdd '93), washington d.c., aaai press, menlo park, california, pp. 1-13. feelders, a. j., et al. (1995). "data mining for loan evaluation at abn amro: a case study". first international conference on knowledge discovery and data mining (kdd-95), montreal, quebec, canada, aaai press, menlo park, california, pp. 106-111. feldman, r. and dagan, i. (1995). "knowledge discovery in textual databases (kdt)". first international conference on knowledge discovery and data mining (kdd-95), montreal, quebec, canada, aaai press, menlo park, california, pp. 112-117. feldman, r. and hirsh, h. (1996). "mining associations in text in the presence of background knowledge". second international conference on knowledge discovery and data mining (kdd96), portland, oregon, aaai press, menlo park, california, pp. 343-346. fisher, d. (1995). "optimization and simplification of hierarchical clusterings". first international conference on knowledge discovery and data mining (kdd-95), montreal, quebec, canada, aaai press, menlo park, california, pp. 118-123. fong, j. and zeng, x. (1997). "data warehouse for decision support". eighth international database workshop, data mining, data warehousing and client/server databases., hong kong, springer-verlag singapore, pp. 195-207. frawley, w. j., et al. (1991). "knowledge discovery in databases: an overview", in knowledge discovery in databases. menlo park, california, aaai press, pp. 1-27. freksa, c. (1992). "temporal reasoning based on semi-intervals". artificial intelligence. vol. 54, pp. 199-227. guan, j. and bell, d. (1991). evidence theory and its applications. north-holland. guan, j. and bell, d. (1992). evidence theory and its applications. north-holland. han, j., et al. (1993). "data-driven discovery of quantitative rules in relational databases". ieee transactions on knowledge and data engineering. vol. 5, no. 1, pp. 29-40. han, j. and fu, y. (1994). "dynamic generation and refinement of concept hierarchies for knowledge discovery in databases". aaai'94 workshop on knowledge discovery in databases, seattle, wa, pp. 157-168. han, j. and fu, y. (1995). discovery of multiple-level association rules from large databases. cmpt tr 95-05. simon fraser university. han, j., et al. (1996). "dbminer: a system for mining knowledge in large relational databases". second international conference on knowledge discovery and data mining, portland, oregon., pp. 250-255. hdbrail, g. and marsais, j. (1992). "experiments of textual data analysis at electricite de france". international federation of classification societies conference, hofacker, i. l., et al. (1996). "knowledge discovery in rna sequence families of htv using scalable computers". second international conference on knowledge discovery and data mining (kdd96), portland, oregon, aaai press, menlo park, california, pp. 20-25. hoschka, p. and klosgen, w. (1991). "a support system for interpreting statistical data", in knowledge discovery in databases. cambridge, ma, aaai press/mit press, pp. 325-345, (ch. 19). inmon, w. h. (1996). "the data warehouse and data mining". communications of the acm. vol. 39, no. 11, pp. 49-50. jackson, s. (1986). introduction to expert systems. addison-wesley publishers limited. kamber, m., et al. (1997). using data cubes for metarule-guided mining of multi-dimensional association rules. technical report report cmpt-tr-97-10. school of computing science, simon fraser university. kamel, m. n. (1995). "a prototype rule-based front end expert system for integrity enforcement in relational data bases: an application to the naval aircraft flight records data base". expert systems with applications. vol. 8, no. 1, pp. 47-58. 125 ajis vol. 6 no. 2 may 1999 keats, g. and loo, s. (1997). "an intelligent business workbench for the insurance industry: using data mining to improve decision making and performance.". eighth international database workshop, data mining, data warehousing and client/server databases., hong kong, springer-verlag singapore, pp. 256-274. keogh, e. and smyth, p. (1997). "a probabilistic approach to fast pattern matching in time series databases". third international conference on knowledge discovery and data mining, newport beach, california, aaai press, menlo park, california, pp. 24-30. ketterlin, a. (1997). "clustering sequences of complex objects". third international conference on knowledge discovery and data mining, newport beach, california, aaai press, menlo park, california. pp. 215-218. klosgen, w. (1993). "some implementation aspects of a discovery system". aaai-93 workshop on knowledge discovery in databases, washington b.c., aaai press, pp. 212-226. klosgen, w. (1995a). "anonymization techniques for knowledge discovery in databases". first international conference on knowledge discovery and data mining (kdd-95), montreal, quebec, canada, aaai press, menlo park, california, pp. 186-191. klosgen, w. (1995b). "efficient discovery of interesting statements in databases". journal of intelligent information systems. no. 4, pp. 53-69. koch, t. and fehsenfeld, b. (1995). "discovering expert system rules in data sets". expert systems with applications. vol. 8, no. 2. koperski, k. and han, j. (1995). "discovery of spatial association rules in geographic information databases". fourth international symposium on large spatial databases, maine, pp. 47-66. lee, r. m., et al. (1985). "temporal inferencing on administrative databases". information systems. vol. 10, no. 2, pp. 197-206. li, s.-h., et al. (1997). "discovering missing semantics from existing relational databases". eighth international database workshop, data mining, data warehousing and client/server databases., hong kong, springer-verlag singapore, pp. 275-286. loeffen, a. (1994). "text databases: a survey of text models and systems". sigmod record. vol. 23, no. 1. long, j. m., et al. (1991). "automating the discovery of causal relationships in a medical records database". in knowledge discovery in databases. aaai press/mit press, pp. 465-476. lu, w., etal. (1993). "discovery of general knowledge in large spatial databases". 1993 far east workshop on cis (ffigis 93), singapore, pp. 275-289. maier, d. and stein, j. (1990). "development and implementation of an object-oriented dbms", in readings in object-oriented database systems. morgan kaufmann publishers inc. mannila, h., et al. (1994). "efficient algorithms for discovering association rules". aaai workshop on knowledge discovery in databases, seattle, washington, pp. 181-192. mannila, h., et al' (1993). improved methods for finding association rules. technical report report c1993-65. university of helsinki, finland. mannila, h., et al. (1995). "discovering frequent episodes in sequences". first international conference on knowledge discovery and data mining (kdd-95), montreal, quebec, canada, aaai press, menlo park, california, pp. 210-215. meo, r., et al. (1996). "a new sql-like operator for mining association rules". twenty-second international conference on very large data bases (vldb '96), mumbai, india, morgan kaufmann publishers, inc. san francisco, usa. pp. 122-133. michalski, r. s., et al., ed. (1984). machine learning an artificial intelligence approach. berlin, springerverlag. miller, m. (1991). "a model of statistical database compromise incorporating supplementary knowledge". the second australian database-information systems conference, sydney, australia, world scientific. miura, t. and shioya, i. (1997). "differentiation for scheme discovery". eighth international database workshop, data mining, data warehousing and client/server databases., hong kong, springer-verlag singapore, pp. 62-77. nahouraii, e. and petty, f., ed. (1991). object-oriented databases. los alamitos, california, ieee computer society press. ng, r. t. and han, j. (1994). "efficient and effective clustering methods for spatial data mining". twentieth international conference on very large data bases, santiago, chile, nishio, s., et al. (1993). "knowledge discovery in object-oriented databases: the first step". aaai-93 workshop on knowledge discovery in databases, washington d.c., aaai press, pp. 299-313. o*leary, d. e. (1991). "knowledge discovery as a threat to database security", in knowledge discovery in databases. aaai press/mit press, pp. 507-516. 126 ajis vol. 6 no. 2 may 1999 padmanabhan, b. and tuzhilin, a. (1996). "pattern discovery in temporal databases: a temporal logic approach". second international conference on knowledge discovery and data mining, portland, oregon, aaai press. park, j. s., et al. (1995). "an effective hash-based algorithm for mining association rules". acm sigmod conference on the management of data, san jose, ca, acm press. vol. 24. pp. 175-186. piatetsky-shapiro, g. (1994). "knowledge discovery in databases: progress report". the knowledge engineering review. vol. 9, no. 1. piatetsky-shapiro, g. and frawley, w. j., ed. (1991). knowledge discovery in databases. menlo park, ca, aaai press/mit press. provan, g. m. and singh, m. (1996). "data mining and model simplicity: a case study in diagnosis". second international conference on knowledge discovery and data mining (kdd96), portland, oregon, aaai press, menlo park, california. quinlan, j. r. (1986). "induction of decision trees". machine learning. vol. 1, pp. 81-106. rainsford, c. p. and roddick, j. f. (1996). "temporal data mining in information systems: a model". seventh australasian conference on information systems, hobart, tasmania, vol. 2. pp. 545-553. rainsford, c. p. and roddick, j. f. (1997). "an attribute-oriented induction of rules from temporal interval data". eighth international database workshop, data mining, data warehousing and client/server databases (idw'97), hong kong, springer verlag. pp. 108-118. roddick, j. f. (1994). a model for temporal inductive inference and schema evolution in relational database systems. ph.d. thesis. department of computer science and computer engineering, la trobe university. roddick, j. f. (1995). "a survey of schema versioning issues for database systems". information and software technology. vol. 37, no. 7, pp. 383-393. roddick, j. f., et al. (1996). "handling discovered structure in database systems". ieee transactions on knowledge and data engineering. vol. 8, no. 2 (april), pp. 227-240. roddick, j. f. and patrick, j. d. (1992). "temporal semantics in information systems a survey". information systems. vol. 17, no. 3, pp. 249-267. roddick, j. f. and rice, s. (1998). "towards induction in databases". ninth australasian information systems conference, university of nsw, vol. 2. pp. 534-542. roddick, j. f. and spiliopoulou, m. (1999). "a bibliography of temporal, spatial and spatio-temporal data mining research". sigkdd discoveries. vol. 1, no. 1, in press. rubinstein, y. d. and hastie, t. (1997). "discriminative vs informative learning". third international conference on knowledge discovery and data mining, newport beach, california, aaai press, menlo park, california, pp. 49-53. savasere, a., et al. (1995). "an efficient algorithm for mining association rules in large databases". twentyfirst international conference on very large data bases, zurich, switzerland, pp. 432-444. savnik, i. and flach, p. a. (1993). "bottom-up induction of functional dependencies from relations". aaai93 workshop on knowledge discovery in databases, washington d.c., aaai press, pp. 174-185. saxena, s. (1993). "fault isolation during semiconductor manufacturing using automated discovery from wafer tracking databases". aaai-93 workshop on knowledge discovery in databases (kdd '93), washington d.c., aaai press, menlo park, california, pp. 81-88. schaffer, c. (1991). "on evaluation of domain-independent scientific function-finding systems", in knowledge discovery in databases. cambridge, ma, aaai press/mit press, pp. 93-104, (ch. 5). schlimmer, j. c. (1993). "self modelling databases : learning and applying partial integrity constraints to detect database errors". ieee expert no. april. schlimmer, j. c., et al. (1991). "justification-based refinement of expert knowledge", in knowledge discovery in databases. aaai press/mit press, pp. 397-410. shortland, r. and scarfe, r. (1994). "data mining applications in bt". bt technology journal. vol. 12, no. 4, pp. 17-22. siegal, m. b., et al. (1991). "rule discovery for query optimization", in knowledge discovery in databases. cambridge, ma, aaai press/mit press, pp. 411-427, (ch. 24). siegel, m., et al. (1992). "a method for automatic rule derivation to support semantic query optimization". acm transactions on database systems. vol. 17, no. 4, pp. 563-600. silberschatz, a., et al. (1996). "database achievements and opportunities into the 21st century". sigmod record. vol. 25, no. 1, pp. 52-63. silberschatz, a. and tuzhilin, a. (1996). "what makes patterns interesting in knowledge discovery systems". eeee transactions on knowledge and data engineering. vol. 8, no. 6, pp. 970-974. 127 ajis vol. 6 no. 2 may 1999 smyth, p. (1996). "clustering using monte carlo cross-validation". second international conference on knowledge discovery and data mining (kdd96), portland, oregon, aaai press, menlo park, california. pp. 126-133. srikant, r. and agrawal, r. (1995). "mining generalized association rules". twenty-first international conference on very large databases, zurich, switzerland, srikant, r. and agrawal, r. (1996). "mining quantitative association rules in large relational tables". acm sigmod conference on the management of data, montreal, canada, stolorz, p. and dean, c. (1996). "quakefmder: a scalable data mining system for detecting earthquakes from space". second international conference on knowledge discovery and data mining (kdd96), portland, oregon, aaai press, menlo park, california, pp. 208-213. stonebraker, m., et al. (1993). "dbms research at the crossroads: the vienna update". nineteenth international conference on very large databases, dublin, ireland, morgan kaufmann, palo alto, ca. pp. 688-692. tansel, a. u., et al. (1993). temporal databases: theory, design and implementation. redwood city, ca, benjamin cummings. tsumoto, s. and tanaka, h. (1996). "automated discovery of medical expert system rules from clinical databases based on rough sets". second international conference on knowledge discovery and data mining (kdd96), portland, oregon, aaai press, menlo park, california, pp. 63-69. valiant, l.g. (1984). "a theory of the learnable". communications of the acm. vol. 27, no. 11, pp. 11341142. viveros, m. s., et al. (1996). "applying data mining techniques to a health insurance information system". twenty-second international conference on very large data bases (vldb '96), mumbai, india, morgan kaufmann publishers, inc. san francisco, usa. pp. 286-293. wade, t. d., et al. (1994). "finding temporal patterns a set based approach". artificial intelligence in medicine. no. 6, pp. 263-271. wang, j. t.-l., et al. (1994). "combinatorial pattern discovery for scientific data: some preliminary results". acm sigmod international conference on management of data, minneapolis, minnesota, pp. 115125. wang, j. t. l., et al. (1996). "automated discovery of active motifs in multiple rna secondary structures". second international conference on knowledge discovery and data mining (kdd96), portland, oregon, aaai press, menlo park, california, pp. 70-75. wirth, r. and reinartz, t. p. (1996). "detecting early indicator cars in an automotive database: a multistrategy approach". second international conference on knowledge discovery and data mining (kdd96), portland, oregon, aaai press, menlo park, california, pp. 76-81. yu, c. t. and sun, w. (1989). "automatic knowledge acquisition and maintenance for semantics query optimisation". ieee transactions on knowledge and data engineering. vol. 1, no. 3, pp. 362-375. zamir, 0., et al. (1997). "fast and intuitive clustering of web documents". third international conference on knowledge discovery and data mining, newport beach, california, aaai press, menlo park, california, pp. 287-290. ziarko, w.( et al. (1993). "an application of datalogic/r knowledge discovery tool to identify strong predictive rules in stock market data". aaai-93 workshop on knowledge discovery in databases (kdd '93), washington d.c., aaai press, menlo park, california, pp. 89-101. 128 paper title (paper title style) australasian journal of information systems zhang, gable & rai 2016, vol 20, selected papers from the isf conference principles of construct clarity toward principles of construct clarity: exploring the usefulness of facet theory in guiding conceptualization meng zhang queensland university of technology m.zhang@qut.edu.au guy gable queensland university of technology g.gable@qut.edu.au arun rai georgia state university arun.rai@eci.gsu.edu abstract conceptualization in theory development has received limited consideration despite its frequently stressed importance in information systems research. this paper focuses on the role of construct clarity in conceptualization, arguing that construct clarity should be considered an essential criterion for evaluating conceptualization and that a focus on construct clarity can advance conceptualization methodology. drawing from facet theory literature, we formulate a set of principles for assessing construct clarity, particularly regarding a construct’s relationships to its extant related constructs. conscious and targeted attention to this criterion can promote a research ecosystem more supportive of knowledge accumulation. keywords: conceptualization; construct clarity; concept clarity; concept formation; construct development; meta-theoretical principle; theoretical framework; facet theory; it value; is success. 1 introduction constructs are invented by researchers to designate conceptual abstractions of phenomena (kaplan 1964). they are the basic building blocks of theory (weber 2012). the conceptual meaning of a construct is specified through conceptualization (schwab 1980). given the frequently stressed importance of constructs, appropriate conceptualization is vital. this paper discusses conceptualization in theory development. conceptualization is differentiated from operationalization wherein relevant empirical measures of a construct are identified. our discussion focuses primarily on conceptualization. though conceptualization and operationalization must be consistent, they are usually characterized as separate phases according to their temporal occurrence (e.g., mackenzie et al. 2011). most researchers would agree that conceptualization should precede operationalization; that is, the meaning of a construct is determined before associating measures with it1,2. note that, although the terms “construct” and “concept” can be differentiated, for simplicity we treat them as synonymous 1 alternatively, one might advocate for defining the meaning of a construct according to its empirical measures, the underpinning rationale of which is often called operationalism or operationism. operationalism, irrespective of its associated advantages, has historically received much criticism, particularly regarding limited generalizability of empirically defined constructs (cook and campbell 1979). 2 the meaning of a construct can also be modified subsequent to its operationalization (schwab 1980). given that modification of meaning will create a new construct that differs from the original one, there is a need to operationalize the modified construct again. hence, the logical order from conceptualization to operationalization remains unchanged even when a construct’s meaning is modified after operationalization. 1 australasian journal of information systems zhang, gable & rai 2016, vol 20, selected papers from the isf conference principles of construct clarity throughout this paper; we use both terms because they have been used interchangeably in the literature. given the criticality of conceptualization in theory development, how conceptualization can be effectively carried out remains an important research question. generally, researchers consider two categories of quality criteria relevant to conceptualization – (i) construct clarity and (ii) construct validity. schwab (1980, pp. 5-6) defined construct validity as “representing the correspondence between a construct (conceptual definition of a variable) and the operational procedure to measure or manipulate that construct.” there are widely accepted procedures for establishing construct validity (e.g., mackenzie et al. 2011; petter et al. 2007; straub et al. 2004). moreover, construct validity pertains mainly to operationalization (suddaby 2010). this paper focuses discussion on the first category of criteria – construct clarity – that has received relatively less attention in the literature. in particular, we are interested in: how can conceptualization achieve greater construct clarity? the remainder of this paper is organized as follows. in the subsequent section, we use the example of information technology value to illustrate the importance of construct clarity in conceptualization. we then engage more deeply with a discussion about achieving various aspects of construct clarity in conceptualization. we focus on one critical aspect – clarity in terms of a construct’s relationships to extant related constructs. by drawing from facet theory literature, we formulate a set of principles and explore how this set of principles might be used to guide conceptualization toward greater construct clarity. we discuss how future research might be informed by these ideas. this paper concludes with a summary and a consideration of limitations. 2 a case from the it value research field information technology (it) value often implicitly serves as an “umbrella” notion (hirsch and lewin 1999), broadly referring to the various consequences of using it or information systems (is), and subsuming similar terms such as it business value (kohli and grover 2008; melville et al. 2004), is success (delone and mclean 1992), and is effectiveness (grover et al. 1996). few would deny that it value is among the most important and widely employed concepts in the is field. research concerned with the implementation and use of it often employs it value as a dependent variable (shang and seddon 2002; burton-jones and straub 2006). in practice, senior managers need to know the expected and realized value from it investments (hitt et al. 2002; murphy and simon 2002). in the research field of it value, a plethora of studies addressing related issues have been published over the decades (schryen 2012; delone and mclean 1992). despite periodic naysayers questioning whether it generates value (e.g., carr 2003), most studies report positive (direct or indirect) contribution from it investment to firms’ economic and business performance (e.g., dedrick et al. 2003; im et al. 2001; rai et al. 1997). extensive and continuing interest in it value has generated diverse theoretical and empirical approaches to its study (c.f. kohli and grover 2008; dedrick et al. 2003; delone and mclean 1992; 2003). in particular, it value is investigated at different levels of analysis (e.g., individual level versus organizational level [chan 2000; petter et al. 2008]), with various intents (e.g., predicting empirical relationships [devaraj and kohli 2003] versus explaining theoretical phenomena [rai et al. 2002]), or using various data collection methods (e.g., panel data [hitt et al. 2002] versus perceptual data [gable et al. 2008]). concomitant with this diversity is growing concern with possible incomparability across research findings (e.g., grover et al. 1996; seddon et al. 1999); such incomparability can be a potential hindrance to knowledge accumulation3 (melville et al. 2004; wade and hulland 3 though several frameworks are proposed to improve comparability across diverse it value studies through identifying their differentiating aspects such as the type of it investigated and the level of 2 australasian journal of information systems zhang, gable & rai 2016, vol 20, selected papers from the isf conference principles of construct clarity 2004). one factor that can frustrate comparability is inadequate clarity in it value conceptualization. it value can be selectively emphasized. as argued by petter et al. (2012), the focus of it value research can change over time. in the early days of computing when the technology was used primarily as a calculator for computing-intensive tasks, the accuracy of calculation was an important concern for defining it value. as it functionality expanded and human-computer interaction became increasingly widespread, other aspects of it value such as usefulness and user satisfaction came to the fore. another variable is the way in which it is used. waves of technology innovation can alter how users interact with it and with each other. for instance, the increasing accessibility of mobile devices can move the virtual workplace from the sole use of an organizational pc to the simultaneous use of multiple platforms; this change can reshape the value dimension of it (e.g., the capability of synchronizing digital content across multiple platforms becomes more important than ever). moreover, a collaborative effort by multiple firms to leverage it can similarly redefine the meaning of it value (e.g., relational or collaborative benefit in the longer term must be considered) (kohli and grover 2008). yet, extant it value dimensions rarely disappear in conjunction with the emergence of new aspects of it value; thus conceptions tend to accumulate. although new value dimensions for mobile devices need to be captured, decades-old aspects such as system quality and information quality (delone and mclean 1992) may still remain fundamental and relevant for conceptualizing it value. when new aspects of it value arise from changing it or it use contexts, the onus is on the researcher to consider the continuing relevance of extant dimensions. conceptualization is further confused and complicated, as few studies are explicit and precise in articulating their conceptualization of it value. for example, it value is labelled variously as “benefits”, “perceived value”, “impact”, and “business value” deriving from it or is (delone and mclean 2003), while important distinctions such as “who perceives the value”, “value for whom”, and “what kinds of value are business value” remain unclear (cf. grover et al. 1996; seddon et al. 1999). the example of it value research suggests merit in paying greater attention to concept clarity in conceptualization. among possible related issues, this paper centres on general conceptualization “goals”. conceptualization can be better guided and more effective with explicit awareness of goals or criteria for good conceptualization. to avoid misleading reader expectations, brief clarification of the research scope is necessary. rather than a comprehensive solution to the methodology of conceptualization, this paper offers exploratory considerations with regard to the qualities of a clear conceptualization. in particular, we set aside how to conceptualize clearly and instead move up to a metatheoretical level, investigating what a clear conceptualization is. addressing this metatheoretical question may inform targeted and mindful practice of conceptualization. in other words, a deeper understanding of what strong conceptualization is may help us understand how such conceptualization can be achieved. this paper addresses the first link in this chain of understanding: what are the qualities of a clear conceptualization? 3 a focus on construct clarity in conceptualization “the need for developing precise concepts […] cannot be overstated” (osigweh 1989, p. 579). clear concepts or constructs can be defined as “robust categories that distil phenomena into sharp distinctions that are comprehensible to a community of researchers” (suddaby 2010, p. 346). van de ven (2007, p. 116) noted “what makes definitions of terms significant is that they analysis chosen (e.g., grover et al. 1996; seddon et al. 1999), these attempts have achieved limited success. 3 australasian journal of information systems zhang, gable & rai 2016, vol 20, selected papers from the isf conference principles of construct clarity classify the universe into ways that are critical to a theory; or as plato said, they ‘carve at the joints.’” in a 2010 editorial in academy of management review, suddaby (2010) noted that lack of construct clarity is a common reason that reviewers and editors reject manuscripts. his view on the importance of construct clarity is echoed and amplified in several recent articles (e.g., klein and delery 2012; locke 2012; skilton 2011; yaniv 2011). in the is discipline, markus and saunders (2007, p. v) emphasized construct clarity when outlining the characteristics required in theory and review papers submitted to mis quarterly. these characteristics include: • clear conceptualizations of concepts unique to information systems research (e.g., the “it artefact”) • clear conceptualizations of concepts relevant to information systems practice (i.e., “action levers” or interventions such as prototyping, training, etc.). furthermore, weber (2012) noted that a clearly defined construct is able to delineate the boundary conditions of what phenomena are included and what are not included in a theory. constructs should be precisely defined, otherwise it is not possible to distinguish to which empirical phenomena the theory can apply and to which it cannot (weber 2012). an unclear construct with ambiguous meaning will jeopardize construct validity (schwab 1980). suddaby (2010) offered relatively more structured analysis for thinking about construct clarity. he outlined four essential components of construct clarity – (1) definitions, (2) scope conditions, (3) relationships between constructs, and (4) coherence. he suggested a good definition of a construct should achieve three things – it should “effectively capture the essential properties and characteristics of the concept or phenomenon under consideration”, “should avoid tautology or circularity”, and “should be parsimonious” (suddaby 2010, p. 347). the scope conditions of a construct need to make clear the contextual conditions under which the construct will or will not apply, specifically with regards to the three general categories of space, time, and values. for relationships between constructs, suddaby (2010) posited that two kinds of relationships should be clarified: (i) a construct’s relationships to other constructs in a nomological network and (ii) a construct’s relationships to extant related constructs that share conceptual meaning. the fourth component of construct clarity is coherence; coherence requires that the previous three criteria of construct clarity – definitions, scope conditions, and relationships between constructs – need to be organized in a logically consistent way (suddaby 2010). the first two components of construct clarity suggested by suddaby (2010) – definitions and scope conditions are relatively well attended to in the methodological literature (e.g., mackenzie et al. 2011; nunnally and bernstein 1994; schwab 1980; weber 2012). the last component of coherence, though important, demands a degree of intuitiveness and usually relies on one’s capabilities (suddaby 2010); thus its practice may be less amenable to normative prescriptions. we focus our attention on addressing the third component of construct clarity – clarity in the relationships between constructs, the difficulty of which may be a reason it has not been well addressed. suddaby (2010) suggested that clarity in the relationships between constructs requires two separate considerations. first, a construct’s relationships to other constructs in a nomological network must be clear. this consideration is widely appreciated. according to cronbach and meehl (1955), a nomological network causally relates theoretical constructs to each other, or theoretical constructs to observables. if constructs differ within a nomological network, they are said to be qualitatively distinct. for example, “usefulness”, “ease of use”, and “intention to use” are distinct because their nomological network has been evidenced by good empirical support (davis 1989). yet, the use of a nomological network cannot address clarity in the second type of relationship: a construct’s relationships to extant, related constructs that have developed historically along with the focal construct (suddaby 2010). this kind of relationship demands a consideration separate from the nomological network, because the meanings of these constructs may, and 4 australasian journal of information systems zhang, gable & rai 2016, vol 20, selected papers from the isf conference principles of construct clarity often do, “overlap” or “measure the same thing” to some extent. to demonstrate cumulative knowledge progression, strong conceptualization should specify the relationships of a focal construct with extant constructs that may share conceptual meaning (suddaby 2010). because it is unproductive to label a construct differently and assume it to be new, it is critical to ensure a new construct is indeed conceptually distinct from extant related constructs. aside from suddaby (2010), other scholars4 similarly noted the importance and difficulty of clarifying the relationships between constructs in achieving construct clarity. drawing from osigweh (1989), van de ven (2007) summarized two general approaches for defining the meaning of a construct, termed semantic and constitutive definitions. he submitted (van de ven 2007, p. 115), a semantic definition describes the meaning of a term by its similarities and differences with other terms. reference to synonyms and antonyms, as well as metaphors and analogies are useful heuristics for developing semantic definitions. […] a constitutive definition describes a term with reference to its component parts. […] while semantic definitions specify the meaning of a concept by extension (i.e., how it is similar to and different from other concepts at the same level of abstraction), constitutive definitions locate the meaning of a concept by intention (i.e., what component terms comprise the concept at lower levels of abstraction, and what more aggregate terms the concept is a member of at high levels of abstraction). this comparing and contrasting strategy in defining a construct is also suggested by several other scholars. mackenzie et al. (2011) recommended that the definition of a construct should specify attributes or characteristics that are common to the focal construct and other similar constructs. nunnally and bernstein (1994) argued that defining a construct not only requires the explication of its conceptual domain but also adequately addressing how the construct is different from other constructs. previous researches that recommend comparison and contrast for achieving construct clarity do not explicitly offer a systematic approach. toward such a systematic approach, we consider a set of principles for assessing clarity in a construct’s relationships to extant related constructs. we hope these principles may help to increase clarity in conceptualization. beyond what is discussed, it is also necessary to pay fuller attention to other criteria sets suggested by suddaby (2010) and others. 4 principles for guiding conceptualization drawing from facet theory literature, we offer a set of principles for exploring construct clarity in conceptualization. these principles are adapted from mcgrath (1968, pp. 192 – 197); they describe the relationships between a focal construct and other similar or potentially overlapping constructs. in what follows, we briefly summarize facet theory and identify its knowledge contribution to various disciplines and areas, critically assessing its strengths and shortcomings. next, we direct attention to the imagery of facet theory and the logic derived from the imagery. 4.1 facet theory in the discipline of psychology, or more precisely, its sub-discipline psychometrics, guttman (1954a; 1954b) proposed facet theory (ft) in response to concerns arising from the arbitrariness of definition. guttman (1971, p. 329) noted, definitions are of course arbitrary. […] one can make words mean what one wishes [… hence,] all that is formally required of a definition is that it be clear: that it enable reliable use of the concept concerned. a more informal, heuristic, desideratum is that it actually influence[s] theorists and researchers to progress in their work. 4 we thank the special issue editor for pointing us to this relevant body of literature. 5 australasian journal of information systems zhang, gable & rai 2016, vol 20, selected papers from the isf conference principles of construct clarity the solution for “reliable use” of a concept offered by guttman is to put the focal concept together with concepts that share meaning in a definitional framework “to make explicit what the focal concept has in common with the others and how it differs from them” (guttman 1971, p. 329). the entirety of related concepts is presumed to form a content universe within a domain of investigation. a core activity in facet theory is the construction of a definitional framework (guttman 1954a; 1954b). to understand what a definitional framework looks like, the notion of a facet remains centrally relevant. according to guttman and greenbaum (1998, p. 17, emphasis added), a facet is a set of attributes (variables) that together represent underlying conceptual and semantic components within a content universe. shye and elizur (1994, p. 23) define the facet as a "set that plays the role of a component set of a cartesian set." in mathematical terminology the word "set" refers to a collection of items or objects. within a set, objects are called "elements." facets are proposed by the investigator and are comprised of elements that define the different values logically describing the variations within a facet (brown, 1985, p. 22). for example, consider three concepts related to satisfaction: user satisfaction (wixom and todd 2005), user information satisfaction (ives et al. 1983), and end-user computing satisfaction (doll and torkzadeh 1988). user satisfaction refers to “the attitude that a user has toward an information system” (wixom and todd 2005, p. 87), whereas user information satisfaction is defined as “the extent to which users believe the information system available to them meets their information requirements” (ives et al. 1983, p. 785). end-user computing satisfaction refers to “the affective attitude towards a specific computer application by someone who interacts with the application directly” (doll and torkzadeh 1988, p. 261). if the three concepts related to satisfaction were the only ones under investigation, they would comprise the content universe. further, the three concepts share a common skeleton: they all refer to an evaluative response toward some evaluative target (melone 1990; muylle et al. 2004). hence, two facets can be accordingly considered: evaluative response and evaluative target (see table 1). evaluative response has two elements or values: attitude and belief. evaluative target also has two elements: information system and computer application. the three concepts, user satisfaction, user information satisfaction, and end-user computing satisfaction can be contrasted according to the two facets of evaluative response and evaluative target. notably, there can be other facets not included (e.g., evaluative stakeholder with elements of user, senior manager, developer, vendor, consultant, and so on). we simplified the example for illustration. facet user satisfaction user information satisfaction end-user computing satisfaction a: evaluative response a[1]: attitude a[2]: belief a[1]: attitude b: evaluative target b[1]: information system b[1]: information system b[2]: computer application table 1. an example of facets based on the above facets and elements of facets, user satisfaction can be characterized uniquely by a combination of elements, denoted as “a[1]b[1]”. similarly, user information satisfaction and end-user computing satisfaction can be characterized as “a[2]b[1]” and “a[1]b[2]” respectively. guttman (1954a; 1954b) does not explicitly define what a definitional framework is. a definitional framework or more specifically a facet definitional framework must include a full collection of facets and elements of the facets that are used to describe the relationships among concepts in a content universe. further, when a facet definitional framework is used to describe a content universe, every concept in the content universe should effectively correspond to a unique sequence of elements (e.g., in the prior example “user satisfaction” identified as “a[1]b[1]”) (guttman and greenbaum 1998). 6 australasian journal of information systems zhang, gable & rai 2016, vol 20, selected papers from the isf conference principles of construct clarity in addition to facets and elements of the facets, a definitional framework contains mapping sentences. argued by guttman and greenbaum (1998, p. 16, emphasis added), a mapping sentence (shye, 1978, p. 413), is "a verbal statement of the domain and of the range of a mapping, including verbal connectives between facets as in ordinary language." it always consists of two main parts: a formal part made up of the facets and a less formal part comprising the phrases linking the facets together (shye, 1978, pp. 179-180; hans, bernstein, & marcus, 1985; levy, 1990). an example of a mapping sentence is depicted in figure 1 (adapted from guttman and breenbaum [1998, p. 16]). this mapping sentence summarizes the various possible definitions for intelligence. by selecting from any combination of values in the three facets (e.g., a1, b2, and c1), a specific definition for intelligence can be created (e.g., one concept of intelligence can be defined as “the performance of examinee through oral expression on an item presented orally by the tester with the aid of numerical language, and requiring inference”.). there are a total of 27 (3x3x3) possible definitions for intelligence. figure 1. an example of a mapping sentence (adapted from guttman and breenbaum [1998, p. 16]) 4.2 facet theory methodology the original intent of facet theory was to provide “a systematic approach to facilitating theory construction, research design, and data analysis for complex studies, that is particularly appropriate to the behavioural and social sciences” (guttman and greenbaum 1998, p. 13). according to guttman and greenbaum (1998, p. 16), guttman’s (1954a) “theory” refers to an hypothesis of a correspondence between a definitional system for a universe of observations, together with a rationale for such an hypothesis. in other words, “theory” as the term is employed by guttman, consists primarily of several parts: a definitional framework for a content universe, a set of empirical observations, a hypothetical correspondence between the definitional framework and the set of empirical observations, as well as a rationale underlying the hypothetical correspondence. this definition of “theory” differs from many contemporary views. for example, sutton and staw (1995) argued that theory must explain “why variables or constructs come about or why they are connected” (p. 375); for them, a list of variables or a construct alone is not theory (sutton and staw 1995). typically, the process of guttman’s (1954a) “theory” construction starts with articulating a definitional framework for a content universe and ends with examining the correspondence between the definitional framework and empirical data. in particular, once a mapping sentence is created, each alternative definition in a mapping sentence can act as a survey scale to measure the corresponding concept. after all the concepts in a content universe are used to 7 australasian journal of information systems zhang, gable & rai 2016, vol 20, selected papers from the isf conference principles of construct clarity assess an empirical sample, the ratings of the scales are expected to exhibit a similar pattern of closeness as the concepts in the content universe (guttman and greenbaum 1998). if two concepts differ only in one facet out of many, the meanings of the two concepts can be regarded as relatively close (in other words they overlap to a greater extent); in principle, the ratings of the two concepts in an empirical sample would also be close. facet analysis is a process in which the researcher examines the correspondence between empirical data and a facet definitional framework, whereas the research design for constructing guttman’s “theory” is called a facet design (shye et al. 1994). notably, guttman’s use of the term “facet theory” appears to be unique; facet theory is neither theory by the definition of guttman nor by the definitions of others (e.g., sutton and staw 1995). we believe that guttman’s “facet theory” is better understood as a methodology for developing guttman’s “theory”. in contemporary language, facet theory can be characterized as a methodology for construct definition, operationalization, and validation. to avoid confusion, we use the term facet theory methodology rather than facet theory in the following discussion. 4.3 impact of facet theory methodology guttman’s (1954a; 1954b) creative work on facet theory methodology (ftm) influenced various disciplines and areas. a main contribution of ftm is to the development of measurement theory in psychometrics or the methodology of measurement (guttman 1971); in particular, ftm is sometimes considered to be the forerunner of factor analysis (e.g., lange 2008; mcgrath 1984; loehlin 1998). in addition, some techniques of ftm (e.g., facet analysis techniques) are being used for studying work commitment (carmeli et al. 2007; morrow 1983), work values (elizur 1984), and interpersonal trust (paul and mcdaniel 2004), or in combination with other methods for measurement scale development (chin et al. 2008) and data analysis (loehlin 1998). ftm can also provide meaningful interpretation for procedures taken in statistical techniques, such as multidimensional scaling (mds) (borg and groenen 2005). according to ftm, the procedures of partitioning mds space can be interpreted as ensuring each region being representative of a facet element (borg and groenen 2005, p. 89). moreover, ftm can assist with the development of typologies (lange 2008; shapira and zevulun 1979). in particular, types of a typology differ by salient dimensions or facets. ftm gives explicit attention to the most important task in typology development: identifying salient facets for types in a domain of interest and appropriately organizing the facets (mcgrath 1968; 1984). such explicit awareness can facilitate more careful comparison of otherwise identical types (mcgrath 1968). the entirety of ftm is rarely employed and less so more recently. ftm, though claiming strength from comprehensiveness, is criticized for its inattention to a fuller research cycle. the “selection” of related concepts and facets, for example, relies partly on intuition, observation, or “general understanding” of a research domain (shapira and zevulun 1979). there is little systematic guidance for this initial selection process, which can, and often does, entail substantial effort and uncertainty (mcgrath 1968). furthermore, we suspect that the use of ftm is infrequent because of its inattention to links with existing knowledge. scientific knowledge rarely exists in isolation; it must be a part of the larger scientific enterprise. it appears difficult to position ftm relative to other streams of thoughts. some procedures in ftm are similar to those in other methods; notions in ftm too sometimes overlap with other contemporary notions (e.g., construct, item, measure, operationalization, etc.). however, there has been little effort to bridge the gap between ftm and contemporary notions and methods. it thus becomes difficult, if not impossible, to evaluate studies that use ftm or identify the core strengths of ftm. regardless of these concerns, we argue that the implied logic of ftm may assist in clarifying overlapping concepts. we next discuss what the logic is and explore how we can use it for conceptual clarification. 8 australasian journal of information systems zhang, gable & rai 2016, vol 20, selected papers from the isf conference principles of construct clarity 4.4 a deeper look at the logic of ftm the measurement and empirical analysis techniques of ftm have been a primary focus in previous literature (mcgrath 1968; 1984; lange 2008). we instead focus on the implied logic of ftm in which we believe the central value lies. to clarify the implied logic of ftm, a distinction between the “imagery” of ftm and that of other methods and approaches must be made (shye et al. 1994). ftm conceives concepts, a priori, as closely related to other concepts from the same domain of investigation; a collection of these related concepts constitutes a content universe. ftm assumes that, ontologically, concepts can be likened to “continuous rather than discrete physical bodies” (shye et al. 1994, p. 40). this ontological view can be contrasted with a relatively “mechanical view” of concepts, which regards concepts as discrete entities; according to this mechanical view, concepts can be independently defined and subsequently be used to investigate causal relationships among them (shye et al. 1994). ftm imagery points to the imperative to delineate precisely the conceptual overlap among related concepts, prior to examining causal links to and from them. further, an implicit logic based on the imagery can be used for clarification (mcgrath 1968); this logic formally describes the structure and the state of a classification system that ought to clarify related concepts to the extent possible. mcgrath (1968, p. 192 197) succinctly summarized the implied logic with a set of principles. we thus adapted mcgrath’s (1968) principles to explicate the logic of ftm. note that there are alternative terms used in the literature. for instance, mcgrath (1968) and others employ “property” and “dimension” as synonymous with “facet” and refer to concepts as “objects” and elements of facets as “values” and “categories”. we replaced mcgrath’s (1968) terms with the terms that are more familiar to the general is community. the adapted principles are outlined as follows (mcgrath 1968, pp. 192 – 197). p1: concepts in a content universe should be specified in terms of all relevant facets. p2: the facets, collectively, should be logically exhaustive of the content universe. p3: the logical relationships among facets should be specified; independence among facets is preferred. p4: each facet should be analysed into a set of collectively exhaustive elements. p5: each facet should be analysed into a set of mutually exclusive elements. p6: the logical relationships among elements of a facet should be specified. p7: the relationships among the concepts defined according to the facets and the elements of facets should correspond to the focal phenomenon. this set of principles centres on clarifying overlapping concepts, the practice of which can be contrasted with a nomological view that emphasizes causal relationships among constructs. by recognizing a continuum of the similarity in the meanings of constructs, the principles encourage conceptual clarification without relying on causal links. the principles suggest any construct ought to be conceptualized based on the meanings of other related constructs. if the relations between a focal construct and other related constructs fulfil the requirements set by the principles, the conceptualization of the focal construct is considered to be clear. positivists and constructivists might have distinctive expectations about construct clarity. for instance, positivist researchers might insist on a construct’s role in predicting empirical observables, and therefore might support the view that clear constructs must also enable easy operationalization across various contexts. in contrast, constructivist researchers might develop constructs merely to label particular phenomena within a context (i.e., not requiring that the constructs can be easily operationalized within other contexts) and hence, might not hold the same view on construct clarity as positivist researchers. consistent with suddaby (2010), we hold a relatively neutral philosophical stance, avoiding aligning with either view. 9 australasian journal of information systems zhang, gable & rai 2016, vol 20, selected papers from the isf conference principles of construct clarity we believe our ideas presented may have pragmatic value for both (the relevance of the neutral stance is further amplified in the last principle illustrated in the subsequent section). 4.5 illustration of the principles drawing from our previous example of satisfaction (table 1), we illustrate in detail the set of principles. principle 1: concepts in a content universe should be specified in terms of all relevant facets. it is important to ensure that every concept within a content universe is specified by all relevant facets. in the illustrative example, each of user satisfaction, user information satisfaction, and end-user computing satisfaction must be specified by all the two facets of evaluative response and evaluative target. in practice, this step is often done through systematic search and analysis of the literature. for example, li and belkin (2008, pp. 1833 – 1835) carefully searched all the relevant facets for the concepts of task; they begin with an initial identification of facets used explicitly or implicitly by extant literature and proceed further with categorizing facets and merging redundant facets. principle 2: the facets, collectively, should be logically exhaustive of the content universe. the exhaustiveness criterion for facets is met if and only if including an additional facet will not achieve any further distinction of concepts in the content universe. this means the criterion of exhaustiveness can be a stopping point for searching for new facets. assume we did not define evaluative target facet in the illustrative example; user satisfaction and end-user computing satisfaction are characterized as the same by existing facets. by defining the additional facet of evaluative target, user satisfaction and end-user computing satisfaction can be further distinguished. we can therefore conclude that the set of facets that contains only evaluative response is not exhaustive for the content universe consisting of these three concepts. in contrast, both facets of evaluative response and evaluative target are exhaustive, because all three concepts in this content universe are already distinguished from each other; any attempt to include more facets would not achieve further distinction. principle 3: the logical relationships among facets should be specified; independence among facets is preferred. relationships among facets must be clearly specified. it is preferred that facets are logically independent of each other; such a group of facets is most efficient (mcgrath 1968). independence means the determination of an element in any facet will not affect the determination of an element in another facet. if the independence criterion were violated, there would exist so-called “null cells” (in a cross-reference of all facets in multidimensional space) that would never be referenced (mcgrath 1968). again, using the illustrative example, assume we have defined an additional facet, called theoretical level, characterizing the level of theory to which the generalization of the concept is manifested (kozlowski and klein 2000). we further assume this facet has two elements, individual level and organizational level. this facet is not independent of the evaluative response facet, because when attitude is used to characterize concepts, by implication the individual level will always be used. in this example, the combination of attitude and organizational level will thus never be used to characterize any existing or potential concepts (here, we assume a collective’s attitude is no longer an attitude, but another type of evaluative response). the inclusion of theoretical level as a new facet will thus result in possible null cells. null cells indicate inefficient use of facets and are undesirable. parsimony or simplicity can be gained from efficient usage of facets. principle 4: each facet should be analysed into a set of collectively exhaustive elements. elements within a facet should also be exhaustive. this exhaustiveness criterion for elements is met if and only if each concept in the content universe can be characterized by at least one element in each facet. consider another variation of the illustrative example, where the evaluative response facet has only the one element, attitude. in such a case, the set of elements 10 australasian journal of information systems zhang, gable & rai 2016, vol 20, selected papers from the isf conference principles of construct clarity for evaluative response facet is not exhaustive, because user information satisfaction cannot be characterized by any element in the evaluative response facet. in contrast, it is exhaustive only when evaluative response facet includes both elements of attitude and belief. principle 5: each facet should be analysed into a set of mutually exclusive elements. in addition to exhaustiveness, the elements in any facet must be mutually exclusive, such that each concept in a content universe can be characterized by only one element of any given facet. for example, all three concepts of user satisfaction, user information satisfaction, and enduser computing satisfaction, can be exclusively characterized by either attitude or belief of evaluative response facet (never both). the elements in evaluative response facet are thus mutually exclusive. principle 6: the logical relationships among elements of a facet should be specified. relationships among elements of a facet must be clearly specified. there are several possibilities. for the evaluative target facet in the illustrative example, a hierarchical order of elements could exist, such as computer application and information system ranked from lower to higher level of analysis. alternatively, the relationship between these two elements could be specified as inclusive – information system consists of computer application and other parts. all others being equal, any selection for the relationship among elements must be justifiable to the extent the purpose of the research is satisfied. principle 7: the relationships among the concepts defined according to the facets and the elements of facets should correspond to the focal phenomenon. how can we know or assess the usefulness of facets and elements used? mcgrath (1968) suggested “the principle of concordance or contiguity” for evaluating the usefulness of facets and elements. he (mcgrath 1968, p. 197) argued, regardless of the purpose of the system or the nature of the objects to be classified, though, it seems clear that the major aim of any a priori classification ought to be to order the objects in terms of their logical properties in such a way as to be predictive of their ordering on (meaningful) empirical properties. concepts that are theorized to be alike should also be empirically alike (mcgrath 1968). in the illustrative example, user satisfaction and end-user computing satisfaction are different on the evaluative target facet, whereas user information satisfaction and end-user computing satisfaction are different on both the evaluative response and the evaluative target facets. enduser computing satisfaction can be interpreted as more “like” user satisfaction than user information satisfaction (by assuming equal contribution of every facet to “likeness”). the relationships among concepts or the “likeness” should be expected from empirical observations. note that there might be other ways to operationalize “likeness”. we adapted mcgrath’s (1968) “principle of concordance or contiguity” to accommodate diverse views. in particular, we believe that it is unnecessary to constrain the target concepts to those emphasizing empirical prediction; instead, for concepts that do not emphasize strong empirical prediction or “mirroring reality” (e.g., metaphysical or linguistic concepts such as “dilemma”, “paradox”, “system”, “adaptation”, and “punctuated equilibrium”), the researcher can establish the correspondence between a definitional framework and an (shared or not shared) interpreted or constructed phenomenon. the usefulness of facets and elements can be verified through the process in which the concepts are continuously used by a group of researchers. 5 discussion the logic of ftm offers guidance on clarifying concepts. usually, first and foremost in clarification, is the need to explore and specify relevant criteria: to what extent can concepts be compared and contrasted? on what bases should comparison and contrast be achieved? in 11 australasian journal of information systems zhang, gable & rai 2016, vol 20, selected papers from the isf conference principles of construct clarity response, the logic visualizes how concepts ought to be related and encourages continuous efforts toward the fulfilment of the vision. inquiries into construct clarity must rely on a fundamental belief system. kaplan (1964) observed an infinite regression that pervades scientific investigation. answering one question can stimulate further inquiries into the assumptions taken in addressing the first question; it is crucial that we, at some point, stop asking “why” but instead believe certain assumptions. when clarifying one focal concept, we certainly could delve into more than one metatheoretical level, through, for example, treating facets and elements of the facets as unknown concepts and clarify their meanings before applying them to the focal concept. such regressive clarification can go through several iterations. to avoid endless clarification, we must, however, at some point, regard the meanings of some concepts (or meta-theoretical concepts) as self-evident (e.g., the meanings of evaluative target and response in our prior illustrative example). from a more pragmatic perspective, van de ven (2007, p. 117) commented, thus, even though the objective of semantic and constitutive definitions is to clearly specify the meaning and usage of terms, they always remain vague in some respects. lines are and must be drawn for the pragmatic purpose of being sufficient to address the problem. […] tolerance of ambiguity is important for scientific inquiry. a caveat should be noted. the prior illustrative example of satisfaction might unwittingly give the impression that only linguistic interpretation of a concept is emphasized. we note this impression is far from our intent. specifically, it is unnecessarily restrictive in ftm to require that facets and elements must come from a linguistic or semantic definition of a concept, or, conversely, that a definition of a concept must include all facets and elements of the facets. put differently, the proposed principles are not limited to a concept’s linguistic meaning, but also apply to its underlying, implied properties; facets and elements can be and often are absent in a definition (e.g., level of analysis). 6 limitation and future research this research is exploratory in nature and entails limitations. this research analysed construct clarity mostly by focusing on a construct’s relationships with extant related constructs. the complications of isolating the issue may need to be considered. given the existence of other aspects of construct clarity, it remains less clear how the whole, rather than the sum of the differing aspects, of construct clarity can be achieved and to what extent these different aspects of construct clarity can be separately achieved. furthermore, this research emphasized metatheoretical principles. future research may consider specific strategies for using the metatheoretical principles in conceptualization. 7 conclusion this paper was initially motivated by the central importance of conceptualization and associated issues with research comparability and cumulative contribution to knowledge. our reference to the it value literature is used to illustrate the relevance of construct clarity and to enliven discussion of the issue within a broader is community. we put forward the view that construct clarity is essential in conceptualization, and offered a meta-theoretical account of conceptualization that might help achieve construct clarity. this account explores what a clearly conceptualized construct ought to be. we hope these considerations can guide conceptualization practice and advance methodological theory of conceptualization. a related aim is to introduce facet theory to the is research community and encourage its exploitation. we believe that the perspective taken in facet theory is largely absent from is research and that attention to facet theory could offer much unexplored potential. references borg, i., and groenen, p. j. 2005. “mds and facet theory,” in modern multidimensional scaling: theory and applications, springer, pp. 87–109. 12 australasian journal of information systems zhang, gable & rai 2016, vol 20, selected papers from the isf conference principles of construct clarity burton-jones, a., and straub, d. w. 2006. “reconceptualizing system usage: an approach and empirical test,” information systems research (17:3), pp. 228–246. carmeli, a., elizur, d., and yaniv, e. 2007. “the theory of work commitment: a facet analysis,” personnel review (36:4), pp. 638–649. carr, n. g. 2003. “it doesn't matter,” harvard business review (81:5), pp. 41–49. chan, y. e. 2000. “it value: the great divide between qualitative and quantitative and individual and organizational measures,” journal of management information systems (16:4), pp. 225–261. chin, w. w., johnson, n., and schwarz, a. 2008. “a fast form approach to measuring technology acceptance and other constructs,” mis quarterly (32:4), pp. 687–703. cook, t. d., and campbell, d. t. 1979. quasi-experimentation: design and analysis issues for field settings, boston: houghton mifflin. cronbach, l. j., and meehl, p. e. 1955. “construct validity in psychological tests,” psychological bulletin (52:4), pp. 281–302. davis, f. d. 1989. “perceived usefulness, perceived ease of use, and user acceptance of information technology,” mis quarterly (13:3), pp. 319–340. dedrick, j., gurbaxani, v., and kraemer, k. l. 2003. “information technology and economic performance: a critical review of the empirical evidence,” acm computing surveys (35:1), pp. 1–28. delone, w. h., and mclean, e. r. 1992. “information systems success: the quest for the dependent variable,” information systems research (3:1), pp. 60–95. delone, w. h., and mclean, e. r. 2003. “the delone and mclean model of information systems success: a ten-year update,” journal of management information systems (19:4), pp. 9–30. doll, w. j., and torkzadeh, g. 1988. “the measurement of end-user computing satisfaction,” mis quarterly (12:2), pp. 259–274. elizur, d. 1984. “facets of work values: a structural analysis of work outcomes,” journal of applied psychology (69:3), pp. 379–389. gable, g. g., sedera, d., and chan, t. 2008. “re-conceptualizing information system success: the is-impact measurement model,” journal of the association for information systems (9:7), pp. 377–408. grover, v., jeong, s. r., and segars, a. h. 1996. “information systems effectiveness: the construct space and patters of application,” information & management (31:4), pp. 177–191. guttman, l. 1954a. “an outline of some new methodology for social research,” public opinion quarterly (18:4), pp. 395–404. guttman, l. 1954b. “a new approach to factor analysis: the radex,” in mathematical thinking in the social sciences p. f. lazarsfeld (ed.), new york, ny, us: free press, pp. 258–348. guttman, l. 1971. “measurement as structural theory,” psychometrika (36:4), pp. 329–347. guttman, r., and greenbaum, c. w. 1998. “facet theory: its development and current status,” european psychologist (3:1), pp. 13–36. hirsch, p. m., and levin, d. z. 1999. “umbrella advocates versus validity police: a life-cycle model,” organization science (10:2), pp. 199–212. 13 australasian journal of information systems zhang, gable & rai 2016, vol 20, selected papers from the isf conference principles of construct clarity hitt, l. m., wu, d. j., and zhou, x. 2002. “investment in enterprise resource planning: business impact and productivity measures,” journal of management information systems (19:1), pp. 71–98. im, k. s., dow, k. e., and grover, v. 2001. “research report: a reexamination of it investment and the market value of the firm—an event study methodology,” information systems research (12:1), pp. 103–117. ives, b., olson, m. h., and baroudi, j. j. 1983. “the measurement of user information satisfaction,” communications of the acm (26:10), pp. 785–793. klein, h. j., and delery, j. e. 2012. “construct clarity in human resource management research: introduction to the special issue,” human resource management review (22:2, si), pp. 57–61. kohli, r., and devaraj, s. 2003. “measuring information technology payoff: a meta-analysis of structural variables in firm-level empirical research,” information systems research (14:2), pp. 127–145. kohli, r., and grover, v. 2008. “business value of it: an essay on expanding research directions to keep up with the times,” journal of the association for information systems (9:1), pp. 23–39. kozlowski, s. w. j., and klein, k. j. 2000. “a multilevel approach to theory and research in organizations: contextual, temporal, and emergent processes,” in multilevel theory, research, and methods in organizationsk. j. klein and s. w. j. kozlowski (eds.), san francisco: jossey-bass inc., pp. 3–90. lange, d. 2008. “a multidimensional conceptualization of organizational corruption control,” academy of management review (33:3), pp. 710–729. li, y., and belkin, n. j. 2008. “a faceted approach to conceptualizing tasks in information seeking,” information processing & management (44:6), pp. 1822–1837. locke, e. a. 2012. “construct validity vs. concept validity,” human resource management review (22:2), pp. 146–148. loehlin, j. c. 1998. latent variable models: an introduction to factor, path, and structural analysis, lawrence erlbaum associates publishers. mackenzie, s. b., podsakoff, p. m., and podsakoff, n. p. 2011. “construct measurement and validation procedures in mis and behavioral research: integrating new and existing techniques,” mis quarterly (35:2), pp. 293–334. markus, m. l., and saunders, c. 2007. “editor's comments: looking for a few good concepts…and theories…for the information systems field,” mis quarterly (31:1), pp. iii–vi. mcgrath, j. e. 1968. “a multi-facet approach to classification of individual group and organizational concepts,” in people, groups, and organizations, new york: teachers college press, pp. 191–215. mcgrath, j. e. 1984. groups: interaction and performance, prentice-hall englewood cliffs, nj. melone, n. p. 1990. “a theoretical assessment of the user-satisfaction construct in information systems research,” management science (36:1), pp. 76–91. melville, n., kraemer, k., and gurbaxani, v. 2004. “review: information technology and organizational performance: an integrative model of it business value,” mis quarterly (28:2), pp. 283–322. morrow, p. c. 1983. “concept redundancy in organizational research: the case of work commitment,” academy of management review (8:3), pp. 486–500. 14 australasian journal of information systems zhang, gable & rai 2016, vol 20, selected papers from the isf conference principles of construct clarity murphy, k. e., and simon, s. j. 2002. “intangible benefits valuation in erp projects,” information systems journal (12:4), pp. 301–320. muylle, s., moenaert, r., and despontin, m. 2004. “the conceptualization and empirical validation of web site user satisfaction,” information & management (41:5), pp. 543– 560. nunnally, j. c., and bernstein, i. h. 1994. psychological theory, new york: mcgraw-hill. orlikowski, w., and iacono, c. s. 2001. “desperately seeking the ‘it’ in it research—a call to theorizing the it artifact,” information systems research (12:2), pp. 121–134. osigweh yg, c. a. b. 1989. “concept fallibility in organizational science,” academy of management review (14:4), pp. 579–594. paul, d. l., and mcdaniel, r. r., jr. 2004. “a field study of the effect of interpersonal trust on virtual collaborative relationship performance,” mis quarterly (28:2), pp. 183–227. petter, s., delone, w., and mclean, e. 2008. “measuring information systems success: models, dimensions, measures, and interrelationships,” european journal of information systems (17:3), pp. 236–263. petter, s., delone, w., and mclean, e. r. 2012. “the past, present, and future of ‘is success’,” journal of the association for information systems (13:5), pp. 341–362. petter, s., straub, d., and rai, a. 2007. “specifying formative constructs in information systems research,” mis quarterly (31:4), pp. 623–656. rai, a., lang, s. s., and welker, r. b. 2002. “assessing the validity of is success models: an empirical test and theoretical analysis,” information systems research (13:1), pp. 50– 69. rai, a., patnayakuni, r., and patnayakuni, n. 1997. “technology investment and business performance,” communications of the acm (40:7), pp. 89–97. schryen, g. 2012. “revisiting is business value research: what we already know, what we still need to know, and how we can get there,” european journal of information systems (22:2), pp. 139–169. schwab, d. p. 1980. “construct validity in organizational behavior,” research in organizational behavior (b. m. staw and l. l. cummings, eds.) (2), pp. 3–43. seddon, p. b., staples, s., patnayakuni, r., and bowtell, m. 1999. “dimensions of information systems success,” communications of the association for information systems (2:20), pp. 1–61. shang, s., and seddon, p. b. 2002. “assessing and managing the benefits of enterprise systems: the business manager's perspective,” information systems journal (12:4), pp. 271–299. shapira, z., and zevulun, e. 1979. “on the use of facet analysis in organizational behavior research: some conceptual considerations and an example,” organizational behavior and human performance (23:3), pp. 411–428. shye, s., elizur, d., and hoffman, m. 1994. introduction to facet theory: content design and intrinsic data analysis in behavioral research (s. shye, d. elizur, and m. hoffman, eds.), thousand oaks, ca, us: sage publications, inc. skilton, p. f. 2011. “getting the reader to ‘i get it!’: clarification, differentiation and illustration,” journal of supply chain management (47:2), pp. 22–28. straub, d., boudreau, m.-c., and gefen, d. 2004. “validation guidelines for is positivist research,” communications of the association for information systems (13:1), p. 24. 15 australasian journal of information systems zhang, gable & rai 2016, vol 20, selected papers from the isf conference principles of construct clarity suddaby, r. 2010. “editor's comments: construct clarity in theories of management and organization,” academy of management review (35:3), pp. 346–357. sutton, r. i., and staw, b. m. 1995. “what theory is not,” administrative science quarterly (40:3), pp. 371–384. van de ven, a. h. 2007. “building a theory,” in engaged scholarship a guide for organizational and social research a. h. van de ven (ed.), oxford: oxford university press. wade, m., and hulland, j. 2004. “review: the resource-based view and information systems research: review, extension, and suggestions for future research,” mis quarterly (28:1), pp. 107–142. weber, r. 2012. “evaluating and developing theories in the information systems discipline,” journal of the association for information systems (13:1), pp. 1–30. wixom, b. h., and todd, p. a. 2005. “a theoretical integration of user satisfaction and technology acceptance,” information systems research (16:1), pp. 85–102. yaniv, e. 2011. “construct clarity in theories of management and organization,” academy of management review (36:3), pp. 590–592. copyright: © 2016 zhang, gable, rai. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. 16 http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 a case from the it value research field 3 a focus on construct clarity in conceptualization 4 principles for guiding conceptualization 4.1 facet theory 4.2 facet theory methodology 4.3 impact of facet theory methodology 4.4 a deeper look at the logic of ftm 4.5 illustration of the principles 5 discussion 6 limitation and future research 7 conclusion paper title (paper title style) australasian journal of information systems mclean, wilson & lee 2017, vol 21, research on indigenous ict the whiteness of redmen 1 the whiteness of redmen: indigenous mascots, social media and an antiracist intervention sheelah mclean university of saskatchewan sheelah.mclean@usask.ca alex wilson university of saskatchewan erica lee university of saskatchewan abstract resistance to the use of indigenous themed mascots in north america has taken a variety of forms over the past several decades. this paper describes and analyses how a new vehicle for resistance, social media, can be integral to dismantling and eradicating racist images of indigenous peoples. specifically, this paper focusses on one campaign that questioned a high school sports mascot and team named the “redmen”. by using examples from social media, the authors demonstrate how white settlers came to rely on the mascot imagery as a way to position themselves as superior and to regulate representations of indigeneity. the authors’ analysis posits that the mascot is in itself a form of racialised colonial violence and they discuss how the name and mascot were protected by and through white settler surveillance and control. to intervene in this discourse of superiority and regulation, the paper describes how an anti-racist approach was used to design a social media campaign that built mass critical consciousness and a network of support within the community. the social media campaign coincided with and rallied support from the grassroots indigenous movement, idle no more. the larger joint effort strategically and effectively redirected the public and critical focus to how the “redmen” name and logo and other racist indigenous mascots become normalised. increased knowledge via social media catalysed a shift in public opinion which ultimately leads to retirement of the team name, logo and mascot. keywords: social media; indigenous mascots; indigenous resistance; white supremacy; anti-racist education. 1 introduction the problematics of indigenous mascots have been a topic of academic analysis and a focus for resistive practices in north america for decades. although there have been varied approaches to understanding and positioning resistance, scholars such as francis (1992) and king (2004) use critical race theory and anti-racist theory to explain the production, regulation and control of images of indigenous peoples in a white settler context. that context is produced through a history of colonial policies and practices that exalt white settler identities and institutions, while denying indigenous peoples’ self-determination. this paper will focus on a social media campaign, informed by anti-racist theory that sought to remove a high school sports team’s name–the “redmen”—and the associated mascot. analysis of the #changeredmen campaign, and the subsequent resistance to it, reveals how colonial discourses are enacted by individuals and communities on (but not limited to) the canadian prairies. in the end, the “redmen” mascot was retired. the #changeredmen campaign’s use of social media, coinciding and in conjunction with the power of the social media driven grassroots movement idle no more, was integral to a successful intervention to dismantle and eradicate the use of the team name, mascot, and logo. indigenous mascots in and of themselves, can be understood as a form of racialised colonial violence that is created and protected by white settler surveillance and control. staurowsky (2007) contends that indigenous mascots are a white invention defended by institutions that australasian journal of information systems mclean, wilson & lee 2017, vol 21, research on indigenous ict the whiteness of redmen 2 are predominantly white. the indigenous mascot becomes a symbol or is a text that can be read and understood as embodying numerous codes of white settler colonialism both past and present (farnell 2004). these codes draw upon discourses such as the ‘indian warrior’ (fryberg et al. 2008) and the ‘noble savage’ (ncai 2013), thereby reinforcing stereotypes of indigenous men as violent and threatening. mascots signify stock tropes of indigeneity and masculinity that freeze indigenous people in the colonial era (leavitt et al. 2015). they collapse hundreds of diverse indigenous nations into one caricature with characteristics such as clothing and feathers that belong only to some plains nations (farne 2004). these reductive and often paradoxical representations are used by institutions such as schools and the mainstream media to regulate indigeneity in a white settler state. the #changeredmen social media campaign provided a space for indigenous and antiracist pedagogy to be enacted while being lead by indigenous women and two-spirit people. antiracist theory, drawing from critical theory and feminist theory, contends that our identities are constructed through discursive formations that position us in particular ways. dei (1995, p.14) describes integrative antiracism as an examination of the interrelated components of gender, sexual orientation, class and race that are “mediated in people’s historical and everyday lived experiences” and contends that understanding these relationships are necessary for progressive learning and social change. with relations of power at the center of analysis the focus shifts to how identity-making practices are produced through discourses, the differential ways people are positioned by these discourses, and how these formations are reified in particular contexts (mclean 2016). in the colonial context, indigenous peoples have been marginalised by white institutions, such as schools and the mass media. social media in its increased accessibility to indigenous peoples provides a public space where people can openly challenge stereotypical representations thereby disrupting and circumventing institutional regulation and control. as indigenous scholar bronwyn carlson (2016) states, “[s]ocial media has changed the nature of communication with sites like facebook and twitter becoming the primary sources of communication where indigenous people can find out what is happening in our networks and in the world.” recent research indicates that indigenous people are “using social media at rates 20% higher than non-indigenous people” (carlson & frazer 2016, p.218). through social media indigenous peoples can narrate their own stories, produce their own symbols of representation, and challenge racialising practices. 2 the #changeredmen campaign although the use of indigenous mascots has decreased over the past decade (ncai 2013) the topic of “mascotting” has become increasingly visible across north america. the term “mascotting” was introduced by cultural critic dave zirin in his 2015 interview with erica lee and suzan shown harjo (zirin 2015). as king (2004, p.194) suggests, “individuals and organizations, from high-school students and teachers to [indigenous rights movements], passionately and aggressively have contested mascots, forcing public debates and policy changes”. various educational institutions have revised their use of the imagery, while others have retired their mascots. as the national collegiate athletic association (ncai) (2013, p.6) reports, “over 2,000 ‘indian’ references in sports have been eliminated during the past 35 years”. the most notable instances at the post-secondary level are the retirement of the stanford university indian and the dartmouth college indian (king 2015; usa today 2013). at the secondary school level, the minnesota board of education (education week 1998) and the los angeles unified school district (los angeles times 2013) have both enacted policies to ban or change racist mascots. several canadian junior and high school teams have recently opted to change their names following public concern. in 2013, the nepean “redskins” youth football team in ottawa changed its name to the nepean eagles after a human rights complaint (schnurr 2013). shortly after the western canada high school in calgary dropped its “redmen” name and logo after consulting with school board elders (cbc news january 12, 2014). australasian journal of information systems mclean, wilson & lee 2017, vol 21, research on indigenous ict the whiteness of redmen 3 the authors of this article worked on the #changeredmen campaign from 2010 until its successful completion in 2014. two of the authors, sheelah mclean and erica lee first met when lee was a student in several of mclean’s classes in her senior year of high school. they began to work together on social justice campaigns and speaking engagements both inside and outside of school. mclean had just completed her master’s degreein antiracist education and chose to share excerpts from daniel francis’ (1992) book, the imaginary indian: the image of the indian in canadian culture. the purpose of assigning the francis chapters was to invite students to problematise and critique the issue of racist school mascots. although some of the students exhibited discomfort with the book, the readings provided important discussion points and insights and resonated strongly with lee. mclean invited cree scholar alex wilson to work with them on the campaign where wilson provided mentorship and guidance in antiracist theory and technical support for the social media campaign. as an editor of the school newspaper, erica lee decided to publish an article issuing a challenge to change the mascot and team name from the “redmen” to the “spidermen”, noting that the colour red would remain. although she wrote the piece in a humorous tone, she recalls the backlash from students and teachers who told her they were discomforted by the article. lee created the first facebook page for the #changeredmen campaign a few years after she graduated from the high school. she continues to discuss the significance of removing indigenous mascots, in her blog moontime warrior: like racist costumes and stereotypes, far from being just a name, the bedford road “redmen,” washington “redskins,” moose jaw “warriors,” and chicago “blackhawks” are products of white supremacy: stories told about us while excluding our voices. while it seems like a small issue on the surface, challenging inaccurate depictions of native people is a step toward a better life (lee 2014). state-sanctioned racist stereotypes such as indigenous mascots in schools justify and normalise inequality. racist symbols reify notions of biological difference and determinism which in turn entrench a racial hierarchy (gould 1981). these discourses of racial difference, and the idea that white settlers are dominant because they are genetically predisposed to intelligence, civility, hard work ethics and other attributes mask the ways in which access to social and political power are determined through racial significations. groups that are othered are constructed as biologically or culturally different and deficient, when injustices are committed daily. this cycle normalises oppression in order to maintain systems of dominance (dei et al. 2005). strong (2004, p.83) states that the practice of appropriating indigenous culture is part of the normalisation that serves to “exclude contemporary native americans from full citizenship by treating them as signs rather than as speakers, as caricatures rather than as players and consumers, as commodities rather than citizens.” indigenous people become singular objects rather than agents of their own social and political lives. sports fans and students who ‘play indian’ by adorning their bodies with paint and feathers become part of the text or performance of the mascot, further normalising a racial hierarchy, dominance and oppression. as staurowsky argues: one of the harms of native american sport team imagery is the day-to-day, easily accessible instruction it provides in how to inappropriately claim symbolic american indian identity without thought to the individual and collective consequences of doing so. as a tool of social control, non-american indians become adept at misappropriating without conscience or consideration while creating mass confusion as to who is an indian (2007, p.66). this appropriation is integral to white settler’s erasure of indigenous peoples’ legitimate reclamations of land and sovereignty. these performances are produced through the white imaginary and desire for legitimacy and belonging on lands that are stolen. research suggests that indigenous mascots are harmful to society and studies have examined how indigenous mascots create a racially hostile environment (baca 2004; farnell 2004; king 2004; staurowsky 2007). as a result of research and increased pressure from indigenous peoples and human rights organisations, several organisations in the united states have australasian journal of information systems mclean, wilson & lee 2017, vol 21, research on indigenous ict the whiteness of redmen 4 passed resolutions calling for a ban of indigenous mascots and related symbols by schools and sport teams (american psychological association 2005; inter-tribal council of the five civilized tribes 2001; u.s. commission on civil rights 2001). the american psychological association (apa) (n.d.), one such organisation, made a public statement declaring the harmful impacts of sexist and racist mascots and logos on indigenous peoples, racialised minorities, and women and called for “the immediate retirement of all american indian mascots, symbols, images, and personalities by schools, colleges, universities, athletic teams, and organizations.” apa’s position was important because not only did it demand the retirement of mascotting, it also made the direct link to violence and degradation of american indians and a connection to the internalisation of that violence to self-esteem (apa 2005). apa’s position is supported through policy in other organisations. in august of the same year, the national collegiate athletic association (ncaa) in the united states announced a policy that would require colleges and universities with native american mascots and imagery to refrain from displaying those during ncaa-sponsored events (ncaa 2005). this policy enacted sanctions on schools that did not comply with the change stating that institutions with this imagery would be ineligible to host ncaa championships starting in 2006 (staurowsky 2007). in spite of the extensive and thorough research that was provided showing the racism endemic in the creation and use of indigenous mascots, each of these associations received backlash from the public and institutions that challenged their recommendations. farnell (2004) suggests that this resistance is a way to maintain schools and sports institutions as white domain. figure 1: andre bear at the brit tournament. photo credit star phoenix 3 ‘the creepy white gaze’: settler pleasure and control in a colonial context, white settler identity is co-constructed with indigeneity, so that the images of indigenous peoples come to represent everything that white settlers are not (francis 1992; said 1979). dei et al. (2005, p. 93) suggests that, “[w]hat originally grew out of the colonial gaze has become cemented in the construction of the other and our firm knowledge of what is and what is not. what we are and what we are not”. whiteness is constructed on discourses of civility, domestication, superior work ethic, and ingenuity (thobani 2011). white settlers imagine that state rights are acquired because of their own intrinsic goodness, rather than colonial practices of domination. in fact, state rights were accorded on the basis of how the national public was produced in relation to gender, race, sexuality, class, and other identity australasian journal of information systems mclean, wilson & lee 2017, vol 21, research on indigenous ict the whiteness of redmen 5 markers (mclean 2016). these narratives of white superiority define both the canadian national identity and determine the policies and practices which shape canada’s social and political landscape. in stark contrast to white settlers, indigenous peoples have been predominantly represented as primitive, savage, and wild. often indigenous peoples are constructed as part of nature and wilderness, which is in keeping with the alternate use of animal imagery as school and team mascots (francis 1992). the ongoing racialisation and dehumanisation of indigenous peoples through texts such as mascots is necessary for normalising white supremacy. white settlers come to rely on indigenous mascots as a way to both understand themselves as superior and regulate representations of indigeneity. in this way, white settlers maintain control over representations of indigeneity as a technique for continuous self-constitution. drawing from foucault’s (1995) concept of surveillance, sterzuk and mulholland (2011) use the term ‘creepy white gaze’ to interrogate the relationship between white identity and power. foucault (1995) contends that power lies at the root of the gaze, which is produced through disciplinary and regulative acts that exercise control and dominance over the bodies of those who are othered. as foucault (1995, p220) contends: …to substitute for a power that is manifested through the brilliance of those who exercise it, a power that insidiously objectifies those on whom it is applied; to form a body of knowledge about these individuals, rather than to deploy the ostentatious signs of sovereignty. in terms of what the gaze accomplishes, warren (2005) suggests that by virtue of the repetitive act of looking, the gaze constitutes race as a category of difference. white settlers derive pleasure from indigenous mascots because they are (re)constituted as both human and superior through the public rituals that celebrate and reproduce this racial distinction. the power whites wield over representational texts such as indigenous mascots provides a playground for mastery over racialised objects (sterzuk & mulholland 2011). white settlers socialised into dominance will resist retiring the mascot because it is a blow to who they think they are, and their desire to control indigeneity. indigenous people continue to resist by exerting sovereignty and self-determination in all aspects of their lives, including representation. as lee (2014) states: refusing to put up with racist sports team names, mascots, and logos is a declaration that we are more than disposable stereotypes with tomahawks in john wayne stories where the injuns always lose. we are teachers, students, lawyers, writers, artists, musicians, workers, and activists. we are capable of doing well in schools and universities while still holding on to our traditional knowledge—recognizing that the two are not incompatible, and that indigenous knowledge ought to be (and already is) present in western academic institutions. we are members of unique communities and we are individuals. the #changeredmen social media campaign was intentional. we sought to ‘shift the gaze’ (hooks 1991; mulvey 1973; razack 1998), in other words, shine a very public and critical lens on white settler (and implicitly patriarchal) discourses that normalise the use of racist indigenous mascots. the backlash our campaign initially received on social media and in the wider community was often framed with emotional rhetoric such as “honor,” “school pride,” “tradition,” and a personal connection to school memories that the mascot somehow embodied. these iterations are code words that make the violence seem more palatable and maintain the innocence of white settlers. the emotional investment that so many white settlers had in the racist mascot is a protection of their whiteness and white supremacy as a colonial system of power. the silencing the student-author, lee, experienced over her letter in the newspaper was a microcosm of the eventual wave of violence she and others involved in the campaign would experience online once we launched the social media campaign. australasian journal of information systems mclean, wilson & lee 2017, vol 21, research on indigenous ict the whiteness of redmen 6 4 the dynamics of online antiracism campaigning: shifting the gaze in march 2011, the campaign started a facebook page called “bedford road ‘redmen,’ it’s time for change”. within a few days, the page had received local and even national media attention. members of the campaign had ongoing discussions regarding strategies for using online education as a mode of intervention. effective strategies include: 1. focus on brief, clear messages as part of the intervention. one such message was “we are people, not mascots”. many of these messages were also hashtags (#peoplenotmascots) that allowed us to track usage and gain momentum as people started using them. the hashtags were portable between facebook, twitter, instagram and multiple other social media. 2. avoid the “oppression olympics”. we decided to avoid comparisons to other racialised groups as mascots. although king (2004) suggests that these comparisons may have been a successful strategy in the past, they can be problematic. comparisons between racialised groups are sometimes called the “oppression olympics,” (origin unknown) whereby oppressed groups are forced into competing positions rather than emphasising how dominance and oppression are structured in a way that produces differential effects. comparing the indigenous mascot to other racialised groups also replicates more images of racist mascots (defeating our goal), and displaces the broader colonial frameworks and racial formations that specifically impact indigenous peoples. 3. draw from antiracist theory and pedagogy in order to maintain a consistent message in the campaign. we held local teach-ins to supplement our social media campaign in the form of a panel at the local university and a class at the peoples free university to further educate the public. 4. have several spokespeople represent the campaign on social media. to disrupt the individualisation of the campaign onto particular organisers, as well as to highlight the reality that this was a large movement of people we ensured that a variety of voices and faces were visible. and, 5. connect to emerging and existing similar campaigns and/or solidarity movements. each of these decisions was made in order to build the campaign, educate the public, build a supportive network and to redirect and dissipate some of the violent backlash. the issue of racist indigenous mascots became a national conversation in canada as the idle no more (inm) movement and the human rights case against the nepean “redskins” in ontario gained traction in the media. the idle no more movement expertly used social media and non-violent direct action to raise critical consciousness regarding indigenous selfdetermination and land protections. idle no more organisers created a web of networks which included over four hundred facebook pages and event pages, including a causes page with over 150,000 members (crowe & wilson 2015; mcadam, wilson, & case 2016). we also created an idle no more website and several twitter pages, where we could highlight pictures of the events and actions such as protests, gatherings, ceremonies and round dances happening across the globe. this strategy connected people to the movement, and provided a platform for organising with others in the movement. idle no more quickly exploded into a global movement and is considered the largest indigenous rights movement in north america (kino-nda-niimi collective 2014). the #changeredmen campaign in the small prairie city of saskatoon now had a very public spotlight and support from across canada and beyond. the scope of a public social media campaign is important in pressuring organisations to make necessary changes rather than maintaining the status quo. australasian journal of information systems mclean, wilson & lee 2017, vol 21, research on indigenous ict the whiteness of redmen 7 5 backlash, whiplash, and the idle no more movement in 1991 susan faludi published the first edition of backlash: the undeclared war against american women. fauldi (1991, p.10) details through an in-depth presentation and analysis of then-current research a powerful counterassault on women's rights, a backlash, an attempt to retract the handful of small and hard-won victories that the feminist movement did manage to win for women…[that] stands the truth boldly on its head and proclaims that the very steps that have elevated women's position have actually led to their downfall.” she carefully links that backlash to a “closed system that starts and ends in the media (fauldi 1991, p.6). the work was significant because it not only provided a much-needed analysis, but it also documented historical trends indicating that when women make positive sociopolitical and cultural advancements toward dismantling systemic sexism, there is inevitably a misogynistic backlash. in the second edition, fifteen years after the first printing, faludi (2006) introduces the book by asking herself the question “is there still a backlash”? her answer is that no there isn’t, there is something worse. drawing from faludi’s concept of backlash, co-author and cree scholar alex wilson (2010) describes her understanding of what that “something worse” might be. wilson analyses and discusses how continued efforts to undermine feminism have a particular and specific effect on the lives and bodies of indigenous women and two-spirit people. wilson purports that the sexualisation and normalisation of violence in north america leads to “whiplash” or a negative reaction to positive advancements in indigenous, queer, and environmental rights. that is, indigenous women and two-spirit people feel the brunt of the force of an intersectional whiplash. there is precision, targeting and physical violence of a whip cracking (wilson 2010). this is something that organisers of idle no more and other recent social media campaigns know all too well. as the #changeredmen campaign gained more visibility, the attacks via social media on individual women and two-spirit people who were part of the campaign increased (e. lee, personal communication june 2013; s. mclean, personal communication june 2013; wilson 2015). the topic of indigenous feminisms itself became a target both of mainstream whiplashers and even within indigenous communities on facebook, on twitter, on online commentary and in real life. within the first few months of idle no more, it was clear that not only had the movement reached global proportions (full duplex 2013), but there were continued efforts to undermine the tremendous advancements that inm was able to accomplish in such a short period of time. the success of the #changeredmen campaign without a doubt was linked to the increased visibility of indigenous women and two-spirit people on social media and the power and momentum that inm was catalysing. the people who did the first work on both the #changeredmen campaign and inm were women and two spirit people who skillfully used social media. with them came their networks which included more women and two spirit people. the whiplash effect reinforces settler-colonial domination and perpetuates supremacy of white, straight male and cisgendered bodies (student teachers anti-racism society 2011; wilson 2010). and because of this, the existence and vocal presence of indigenous women and two-spirit people is a constant threat to these systems of domination. an analysis of the first six months of idle no more by full duplex (2013) focused on the social media aspects of the movement and provided analysis and commentary on the power of the then-young movement: the significance of inm cannot be downplayed. inm is a grassroots movement that leveraged the power of the participatory web and “open source” ideology. it is an energy source, which has united people of different communities. tweets, facebook pages, podcasts, blog posts, forums, images, and videos built new bridges and obstructed traffic on others. competition for attention is fierce and inm managed to do more than simply get noticed (full duplex 2013, p.1). australasian journal of information systems mclean, wilson & lee 2017, vol 21, research on indigenous ict the whiteness of redmen 8 full duplex provides graphs highlighting the correlation between increases in tweets with the initiation of specific inm campaigns. for example, january 2013 was categorised as a period of high engagement with well over 500 tweets per day mentioning the phrase idle no more. during this month, there were a number of national and international campaigns such as the national day of action (known as #j11) with 265 inm events that took place around the globe and over 60,000 tweets on that day, on the january global day of action (#j28), and other coordinated days of action. the idle no more communications team has similarly documented that peaks in social media (twitter and facebook mentions) correlate with specific campaigns (mcadam, wilson, & case 2016). #changeredmen was one such campaign. the full duplex (2013) report provides in-depth discussion of the gender breakdown of social media. the general trend was that women were supporting and posting positive messages about the movement while men comprised a majority of the negative social media commentary, or were predominantly driving the whiplash. significantly, however, for the first time in history a worldwide social movement was being informed and lead by indigenous women (full duplex 2013). 6 the gendering of the campaign the [#changeredmen] campaign may have gone differently if there was a man out front. a lot of the dissenting comments we received were about the group being “too sensitive” and “too emotionally attached to the issue.” in fact, the very first news article published about our campaign set the tone, with the unfortunate headline “redmen team name irks saskatoon high school graduate,” something i never lived down as my friends joked “watch out, erica is irked!” poking fun at how the school, the school board, and some media tried to downplay our anti-mascot campaign as nothing more than the boredom of one emotionally-driven young girl (lee 2015). this campaign took place in saskatoon—treaty 6 territory, neyinoway askik (cree territory), and métis homeland—a colonial space much like the rest of the canadian state where heteronormativity and patriarchy are naturalised through organisational, familial, and institutional discursive practices (st. denis 2007; thobani 2011). furthering the whiplash (wilson 2010), the individualisation and feminisation of the campaign as “emotional” by the mainstream media marked it as inferior and not to be taken seriously. this occurred frequently as articles regarding the campaign focused on erica lee describing her as “irked” and noting her tears during interviews. these tropes of female/emotional/subordinate were juxtaposed with the construction of the mascot as a male/fierce/dominant embodying particular forms of masculinity and reproducing a gender binary. the “redmen” mascot is the face of an indigenous man—a red, feathered caricature of a man’s head. this representation of indigenous masculinity is made to be recognisable by the broader society—an image used in sports culture because it evokes discourses of stoicism, strength and an opponent to be feared. the feminisation of the #changeredmen campaign as “emotional” worked to govern subjects that were invested in this performance of racialised masculinity, and bring shame or emasculating those working against it. the mascot social media campaign was one form of intervention into the white supremacy reproduced within the school and community. many individuals, particularly white males in positions of authority, were resistant to the idea that indigenous people should have control over how they are represented, particularly in the media (full duplex 2013). in a white supremacist society, white settlers are socialised into practices that maintain state control through both individual and institutional mechanisms. these discursive practices naturalise white heteropatriarchy and are used to discipline bodies into subjugation. control over racist texts such as the “redmen” mascot is fiercely protected by those who benefit most from the maintenance of inequitable power relations. from the onset, the indigenous women on the campaign team experienced both racialised and sexualised violence from individuals who enacted the protection of the mascot by attacking the women leading the campaign. as lee (2015), who was one of the administrators of the #changeredmen facebook page, explains: australasian journal of information systems mclean, wilson & lee 2017, vol 21, research on indigenous ict the whiteness of redmen 9 there were comments with gendered, racialized slurs left on our group’s page with regularity, directed at the women involved, sometimes using our names and including personal information, like our locations of work and school. i’m only now starting to think about the implications of notions of masculinity conjured by sports mascots and what it meant for our little group of primarily indigenous & nonindigenous women to challenge those notions. what does it do to challenge that silently stoic cartoon “redman” face and replace it with an image of real-life indigenous women saying no? an example of the violent backlash against the campaign is provided below in a screenshot from the “bedford road ‘redmen,’ it’s time for a change” facebook page: figure 2: comment on the it’s time for change facebook page. in this excerpt, the writer employs racist and sexist language as a means to regulate and silence those involved in the campaign against the mascot. discourses such as “attention whore pussy bitch” and other gendered/sexualised terms are derogatory threats that might protect the mascot as a symbol that upholds white heteropatriarchy. each of these discourses is constructed as abject subjects—terms that historically have been used to mark bodies as degenerate and inferior. this individual’s use of the term “attention whore” also shows how the attention the #changeredmen campaign was receiving on social media was a threat to the life and longevity of the indigenous mascot. although many of the violent responses like this one were from teenagers (high school age), these youth were already socialised into whiteness and performing violence to protect the social order. 7 institutional mechanisms of control the individuals who campaigned to keep the mascot performed multiple moves to settler innocence: in particular, the upholding and weaponising of a small number of indigenous voices that explicitly conformed to their wishes to keep the mascot and the attempted delegitimisation of anyone who supported the #changeredmen campaign. the high percentage of indigenous people who spoke out against the mascot affected the dynamics of the mascot debate in this space, because indigenous people were not invisibilised. one of the strategies of school systems and athletic clubs is to frame the mascot issue by focusing on the tensions between indigenous groups that disagree. the most common tactic is to label and divide indigenous people into ‘academic indians vs pow wow indians,’ with the presumption that academics are anti-mascot, while ‘traditional’ people support it. these discourses, false binaries, legitimised anyone upholding the logo as speaking from a position of authority and authenticity, while suggesting that those against the logo were influenced by the colonial system of academe. these colonial tropes suggest that indigenous people who support the logo are ‘real indians,’ while marking all indigenous people who engage in contemporary resistance as illegitimate (farnell 2004; st. denis 2004). these tactics of divide and conquer exasperated tensions in the indigenous community regarding the mascot. it must be noted that the historical and ongoing erasure of indigenous people from positions of power in colonial institutions and positive representations in texts such as mass media, education, and government often leads to a desire from some indigenous people to maintain indigenous mascots. this hinges on the belief that some visibility and representation (in the guise of an honour) is better than none at all. our campaign insisted that indigenous people australasian journal of information systems mclean, wilson & lee 2017, vol 21, research on indigenous ict the whiteness of redmen 10 can and will decide how they are represented to the public, in particular moving away from the violence of stereotypical and objectified symbols and signs. the actions to uphold the discourses that constructed the mascot as an honour created a hostile environment for everyone in the school and community that supported the campaign. as farnell (2004, p49) contends: this racializing discourse [of honouring] empowers whites, not only to claim a superior right to determine the representations but also to silence native voices, insisting with neocolonial impunity that whites will determine what indians will mean for this community. these indirect indexes reveal the institutionalized racism at work here. these tensions were heightened during the whiplash to our social media campaign, which caused many people to feel uncomfortable about supporting our anti-mascot campaign. we encouraged school boards and community organisations to engage in critical antiracist education with their members. we believe that education is key to understanding how indigenous mascots normalise racism and sexism in our society. in fact, one of the school board members that voted to retire the mascot publicly admitted that reading the research on indigenous mascots influenced his decision to ban the mascot. there were many public letters of support shared on the campaign’s social media from various individuals and institutions throughout the province, such as a letter from the department of educational foundations at the university of saskatchewan, which called for the mascot to be banned and antiracist education for the school and community: figure 3: post by department of educational foundations at the university of saskatchewan this growing solidarity with the campaign helped to educate the broader public and put pressure on the school board to ban the racist mascot. 8 non-violent direct action in 2013, a group of people who were part of the campaign, including indigenous drummers, gathered to protest the “redmen” mascot at an invitational basketball tournament, a prominent annual high school sporting event. two of the indigenous women who were a highly visible part of the campaign tried to buy tickets for the event, but were turned away while the three white allies that were with them were allowed entrance. outside the school, there were multiple police cars and officers patrolling. one of the white allies shared the following account of the night on facebook: we were treated like terrorists in possession of lethal weapons (our posters, our message). i can’t believe i was told by a guy who spoke highly of his participation in idle no more rallies in ottawa that “you’ll never win” if we are trying to change his high school’s mascot. it was sad to know he was the same person that turned away all our australasian journal of information systems mclean, wilson & lee 2017, vol 21, research on indigenous ict the whiteness of redmen 11 drummers. however, this was all fuel to the cause and i am glad we made our presence known. this is about resistance, coming together, and overcoming this kind of racist oppression. props to everyone who came out and also to those who were there in spirit. figure 4: youth representing the #changeredmen campaign the ways in which the school board used surveillance and control at the basketball tournament to deter the right to publicly protest shows the powerful impact of non-violent direct actions by a small number of people. the plan that the youth representing the #changeredman campaign has was simply to hold up signs—what could be so threatening about this act? the simple act of holding a sign can at once turn the gaze and denaturalise the racist indigenous mascot that is central to an event celebrated predominantly by white settlers. as lee (2015) explains: the hardest part of this work is listening to the stories of aboriginal youth who dare to speak out against stereotypes and faced ostracism as a result. in january, a student from a local high school, stood up at bedford road’s basketball tournament with a sign that said, “we are people, not mascots.” he was booed by hundreds of basketball fans before being escorted out of the school, demonstrating that there is a certain type of “red man” that is acceptable (a cartoon) and one that is not (a living person with a voice). to me, that is a real warrior. non-violent direct actions such as these often have detrimental effects on both the resistors involved who face colonial violence, as well as those people that are witnessing the intervention who may want to speak up. the lesson is this: you will face public anger and shame if you speak out against racism in your school and community. the significance of using both social media and non-violent direct actions to disrupt the dominant discourses surrounding the text of the mascot cannot be underestimated. 9 the retirement: white tears in a vote of 8 to 2 on march 4, 2014 the saskatoon public school board voted to respectfully retire the racist “redmen” mascot (saskatoon public schools 2014). in an interview with ctv news, the school administration said that retiring the mascot was “like losing a member of the family” (young 2014). the tensions between groups that advocate for the mascot, and those who campaign against them are often high. this is because of the emotional investment that white settlers have in racial significations such as an indigenous mascot. individuals who fight to maintain mascots often exhibit a range of emotion such as anger, fear, frustration and sadness. when other mascots are replaced with an updated logo, there is no outcry. indigenous mascots are an emotional investment for white settlers because the mascot represents white identity, not indigeneity. whiteness is co-constructed with indigeneity, and one of the many ways this construction is reified is through the stereotypically racist images of indigenous people. in this way, the text of the indigenous mascot comes to represent white supremacy. australasian journal of information systems mclean, wilson & lee 2017, vol 21, research on indigenous ict the whiteness of redmen 12 the spectacle of indigenous mascots represents the colonisers fantasy of racial superiority. as king states: accounts of the history and significance of such metaphoric recyclings rightly underscore the articulations of power, representation, and racialization, unpacking the ways whites have used (enacted, really) signs of difference to formulate identities, craft community, author national narratives, and consolidate racial hegemony (2004, p.193). there is still much educational work to do in order to build a public consciousness that includes a critical analysis of white settler colonialism, and how racist symbols such as indigenous mascots are dehumanising and violent. although the “redmen” mascot was finally retired, white settlers continue to resist any commitment to build safer spaces in schools for indigenous students through antiracist education. we suggest that this is how white supremacy is reproduced in the face of a successful campaign to dismantle and eradicate racist images of indigenous peoples. 10 conclusion this paper examines how social media created a powerful space for indigenous peoples’ refusal through a campaign that sought the retirement of a racist indigenous mascot. the primary intervention of our campaign was the use of antiracist education on social media in order to build mass critical consciousness and a network of support. the significance of our social media campaign as a space for indigenous and antiracist pedagogy specifically lead by indigenous women and two-spirit people cannot be underestimated. in a colonial context, indigenous peoples have been marginalised by white institutions such as the mass media, and social media provides a public space that directly challenges this regulation and control, as it is increasingly accessible to indigenous peoples worldwide. we suggest that the use of social media created a campaign that was powerful enough to lead to the ban of a racist mascot after unsuccessful attempts by many other groups. acknowledgements the authors would like to acknowledge the many people who contributed to retiring the mascot, before and during the #changeredmen campaign. we would also like to acknowledge and thank anna lee popham smith, jaylene murray and janet sarson for providing feedback on drafts of this paper. funding partial funding to support this writing was provided by sshrc grant 435-2014-1488. references american psychological association. (2005, october 18) “apa resolution recommending the immediate retirement of american indian mascots, symbols, images, and personalities by schools, colleges, universities, athletic teams, and organizations”. available at, https://www.apa.org/about/policy/mascots.pdf [accessed 31 january 2017]. american psychological association. (n.d.) “summary of the apa resolution recommending retirement of american indian mascots”. available at, http://www.apa.org/pi/oema/resources/indian-mascots.aspx [accessed 31 january 2017]. baca, l.r. (2004) “native images in schools and the racially hostile environment”, journal of sport & social issues, 28(71). doi: dx.doi.org/10.1177/0193732503261149 baym, n.k. & boyd, d. (2012) “socially mediated publicness: an introduction”, journal of broadcasting & electronic media, 56(3):320-329. doi:dx.doi.org/10.1080/08838151.2012.705200 australasian journal of information systems mclean, wilson & lee 2017, vol 21, research on indigenous ict the whiteness of redmen 13 carlson, b. (2016) “for aboriginal and torres strait islander people, what are the health impacts of social media”, croakey. available at, https://croakey.org/for-aboriginaland-torres-strait-islander-people-what-are-the-health-impacts-of-social-media/ [accessed 31 january 2017]. carlson, b. & frazer, r. (2016) “indigenous activism and social media: the global response to #sosblakaustraia”. in mccosker, a., vivienne, s., & johns, a. (eds.), rethinking digital citizenship: control, contest and culture, pp. 115-130, rowman and littlefield international, england. cbc news. (2014, january 12) “redskins name scrapped by ottawa minor football club” available at, http://www.cbc.ca/news/canada/ottawa/redskins-name-scrapped-byottawa-minor-football-club-1.2493697 [accessed 31 january 2017]. crowe, r. & wilson, a. (2015) “idle no more round dance revolution at the hemispheric institute of performance: a discussion with roewan crowe and alex wilson”, caribbean rasanblaj, 12(1). available at, http://hemisphericinstitute.org/hemi/en/emisferica121-caribbean-rasanblaj/crowe [accessed 5 may 2017]. davis-delano, l.r. (2007) “eliminating native american mascots: ingredients for success”, journal of sport & social issues, 31(4):340-373. doi:dx.doi.org/10.1177.0193723507308251 dei, g., karumanchery, l., & karumanchery, n. (2005) playing the race card: exposing white power and privilege, peter lang, new york, ny. dei, g. (1995) “integrative antiracism: intersections of race, class and gender”, race, gender & class: perspective on canadian antiracism, 2(3): 11–30. education week. (1988) “minnesota districts asked to drop indian team names”. available at, http://www.edweek.org/ew/articles/1988/06/01/x36state.h07.html [accessed 5 may 2017]. faludi, s. (1991) backlash: the undeclared war against american women, crown publishing group, danvers, ma. farnell, b. (2004) “the fancy dance of racializing discourse”, journal of sport and social issues, 28(1):30-53. foucault, m. (1995) discipline and punish: the birth of the prison (a. sheridon, trans), vintage books, new york, ny. francis, d. (1992) the imaginary indian: the image of the indian in canadian culture, arsenal pulp press, vancouver, bc. fryberg, s.a., markus, h.r., oyserman, d., & stone, j.m. (2008) “of warrior chiefs and indian princesses: the psychological consequences of american indian mascots”, basic and applied social psychology, 30(3):208-218. doi:dx.doi.org/10.1080/01973530802375003 full duplex. (2013, june) “idle no more at six months: an analysis of the first six months of the idle no more movement”. available at, http://digitalpublicaffairs.com/wpcontent/uploads/2013/06/idlenomoreatsixmonths.pdf [accessed 31 january 2017]. gould, s. (1981) the mismeasure of man. w.w. norton and company, boston, ma. hooks, b. (1992). “the oppositional gaze: the black female spectator” black looks, race and representation, pp. 115-131, south end press, boston. inter-tribal council of the five civilized tribes. (2001) “the five civilized tribes intertribal council mascot resolution”. available at, http://aistm.org/2001.civilized.tribes.htm [accessed 31 january 2017]. australasian journal of information systems mclean, wilson & lee 2017, vol 21, research on indigenous ict the whiteness of redmen 14 king, c.r. (2004) “this is not an indian: situating claims about indianness in sporting worlds”, journal of sport and social issues 28:3-10. doi:dx.doi.org/10.1177/0193732503261147 king, c.r. (2015) “american indian and nickname changes: a chronology”. in c.r. king (ed.), the native american mascot controversy: a handbook, pp.259-264, lanham, ma, rowman & littlefield publishing group, inc. kino-nda-niimi collective. (2014) the winter we danced: voices from the past, the future, and the idle no more movement, arp books, winnipeg, mn. leavitt, p.a., covarrubias, r., perez, y.a., & fryberg, s.a. (2015) “’frozen in time’: the impact of native american media representations on identity and self-understanding”, journal of social issues, 71(1)39-53. doi:dx.doi.org/10.1111.josi.12095 lee, e.v. (2014, may 10) “change(d) the name: challenging “savage” stereotypes and native sports mascots” [blog post], moontime warrior. available at, https://moontimewarrior.com/2014/05/10/changed-the-name-challenging-savagestereotypes-and-native-sports-mascots/ [accessed 31 january 2017]. lee, e.v. (2015, march 8) “reflections on ‘redmen’ and indigenous women’s organizing” [blog post], moontime warrior. available at, https://moontimewarrior.com/2015/03/08/reflections-on-redmen-and-indigenouswomens-organizing/ [accessed 31 january 2017]. los angeles times. (2013, december 13) “change makers”. available at: https:www.pressreader.com/usa/los-angeles-times/20131215/282346857623007 [accessed 5 may 2017]. mcadam, s., wilson, a., & case, p. (2016) “idle no more: current projects and future visions”. presentation at the native american indigenous studies conference, honolulu, hi. mclean, s. (2016) disturbing praxis: a foucauldian analysis of student subjectivities and classroom pedagogies in public schools. doctoral dissertation. university of saskatchewan, saskatoon, sk. mulvey, l. (1974). “visual pleasure and narrative cinema”, screen (16)3, 6-18. national collegiate athletic association (ncaa). (2005) “ncaa executive committee issues guidelines for use of native american mascots at championship events”. available at, http://fs.ncaa.org/docs/pressarchive/2005/announcements/ncaa%2bexecutive%2 bcommittee%2bissues%2bguidelines%2bfor%2buse%2bof%2bnative%2bamerican %2bmascots%2bat%2bchampionship%2bevents.html [accessed 31 january 2017]. razack, s. (1998) looking white people in the eye: gender, race and culture in courtrooms and classrooms, university of toronto press, toronto, on. said, e.w. (1979) orientalism. vintage books, new york, ny. saskatoon public schools. (2014) meeting minutes of the board of education of the saskatoon school division no. 13 of march 4, saskatchewan. available at, http://www.saskatoonpublicschools.ca/board/meetings/minutes/mar414%20regular. pdf [accessed 3 may 2017]. schnurr, j. (2013, september 3) “ottawa man files human rights complaint against nepean redskins” ctv ottawa. available at: htto://ottawa.ctvnews.ca/ottawa-man-fileshuman-rights-complaint-against-nepean-redskins-1.143852 [accessed 5 may 2017]. st. denis, v. (2004). “real indians: cultural revitalization and fundamentalism in aboriginal education”. in c. schick, j. jaffe, & a.m, watkinson (eds.), contesting fundamentalisms, pp. 35-47, fernwood, halifax, ns. australasian journal of information systems mclean, wilson & lee 2017, vol 21, research on indigenous ict the whiteness of redmen 15 st. denis, v. (2007) “feminism is for everybody: aboriginal women, feminism and diversity”. in j. green (ed.), making space for indigenous feminisms, pp. 33-52, fernwood, halifax, ns. staurowsky, e.j. (2007) “you know, we are all indian: exploring white power and privilege in reactions to the ncaa native american mascot policy”, journal of sport and social issues 31(1):61-76. sterzuk, a. & mulholland, v. (2011, spring) “creepy white gaze: rethinking the diorama as a pedagogical activity”, alberta journal of educational research 57(1):16-27. strong, p.t. (2004) “the mascot slot: cultural citizenship, political correctness, and pseudoindian sports symbols”, journal of sport and social issues 28:79-87. student teachers anti-racism society. (2011, may) “student teachers anti-racism society (stars) educational resource”. available at, http://starsusask.blogspot.ca/2011/05/interconnecting-isims-race-gender-and.html [accessed 31 january 2017]. thobani, s. (2011) exalted subjects: studies in the making of race and nation in canada, university of toronto press, toronto, on. usa today. (2013, september 24) “list of schools that changed native american nicknames”. available at, https://www.usatoday.com/story/sports/2013/09/12/native-americanmascot-changes-ncaa/2804337/ [accessed 5 may 2017]. u.s. commission on civil rights. (2001, april 13) “statement of u.s. commission on civil rights on the use of native american images and nicknames as sports mascots”. available at, http://www.aics.org/mascot/civilrights.html [accessed 31 january 2017]. warren, j. (2005) “bodily excess and the desire for absence: whiteness and the making of (raced) educational subjectivies”. in b. alexander, g. anderson, & b. gallegos (eds.), performance theories in education: power, pedagogy, and the politics of identity, pp. 83-106, erlbaum, mahwah, nj. wilson, a. (2010, september) “whiplash: interconnecting isms, race, gender and homophobia”. presentation for stars (student teachers anti-racism society), college of education, university of saskatchewan, saskatoon, sk. wilson, a. (2015) “a steadily beating heart”. in e. coburns (ed.) more will sing their way to freedom, pp.272-276, fernwood, winnipeg, mb. young, m. (2014, march 6) “overwhelming positive vote on team”, ctv saskatoon. available at, http://saskatoon.ctvnews.ca/video?clipid=301773&binid=1.1165954&playlistpagenum =1 [accessed 31 january 2017]. zirin, d. (2015, july 2) “the battle to change the mascots”. socialism 2015 interview with erica violet lee and suzan shown harjo. available at, http://wearemany.org/a/2015/07/battle-to-change-mascots [accessed 5 may 2017]. copyright: © 2017 mclean, wilson & lee. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems mclean, wilson & lee 2017, vol 21, research on indigenous ict the whiteness of redmen 16 evaluation of database modeling methods for geographic information systems thanasis hadzilacos nectaha tryfona computer technology institute department of computer science university of patras fredrik bajers vej 7e box 1122 aalborg university or-26110, patras, greece dk-9220, aalborg, denmark hh@cti.gr tryfona@cs.auc.dk tel: +30(61)274496 tel: +45(96)358898 fax: +30(61)222086 fax: +45(98)159889 abstract we present a systematic evaluation of different modeling techniques for the design of geographic information systems as we experienced them through theoretical research and real world applications. a set of exemplary problems for spatial systems on which the suitability of models can be tested is discussed. we analyse the use of a specific database design methodology including the phases of conceptual, logical and physical modeling. by employing, at each phase, representative models of classical and object-oriented approaches we assess their efficiency in spatial data handling. at the conceptual phase, we show how the entity-relationship, efo and omt models deal with the geographic needs; at the logical phase we argue why the relational model is good to serve as a basis to accommodate these requirements, but not good enough as a stand alone solution. keywords geographic information systems, spatial requirements, spatial database modeling, conceptual geographic models, logical geographic models, entity-relationship model, object modeling technique. introduction geographic information systems (gis) is but one of a large number of data-intensive application areas -often referred to as "non-standard"including among others, architectural and vlsi design, robotics, image and voice processing, artificial intelligence, multimedia and knowledge-based systems. modeling gis has gained much attention and popularity over the last years firstly due to their increased use and, secondly due to the special requirements for their design and use. peculiarities of gis steam from the main difference between geographic and classical objects: their "position" in space. two major different approaches have been adopted from the scientific community to capture the spatial dimension: (a) the employment of the already existing data models; for example, (oxborrow and kemp, 1992) and (worboys, et. al., 1990) use the entity-relationship (er) and object-oriented models respectively, to represent spatial objects and, (b) the extension of these models for the efficient handling of geoinformation; characteristic examples are (benoit et. al., 1993) and (scholl and voisard, 1991) for the extension of o2, (pelagatti et. al., 1991) for the extension of er, (ogis, 1994) for the establishment of a common spatial data model based on objectoriented aspects for the communication among heterogeneous spatial systems. despite the significant research efforts, the result is not the expected one: gis still lack portability, maintainability, scalability and sometimes even correctness. the reason is that they are built without any consideration to a particular methodology -usually they are developed in an ad-hoc way. our position is that gis are special cases of information systems (is) and spatial databases an indispensable part of them. by using well-known modeling techniques such as standard application design methodologies for is, the resulting systems are -at leasteasy to be extended and maintained. on the other hand, such methodologies should be extended to accommodate the particularities of spatial information. from this perspective, for the analysis and design of spatial database design for geographic application we adapted a well-known methodology, which includes the phases of conceptual, logical, and physical organisation of data. for the conceptual and logical phases we employed representative -already existingmodels to capture * this work was completed after the second author had joined the university of aalborg. her research was supported in part by the chorochronos project, funded by the european commission dg xii science, research and development, as a networks activity of the training and mobility of researchers programme, contract no. fmrx-ct96-0056. the spatial aspects. whenever it was desirable, we extended the provided modeling techniques for that purpose. we tested this approach in real world applications, such as network utility (utilnets, 1994) (tsironis, 1992), cadastral, and forest management systems. in this paper we present an excerpt of the evaluation and comparison of the models we used during this research effort. the contribution of this work lies at exactly this point: by providing a comparative study about the modeling techniques that can be used for geographic data modeling, we help designers to choose the most appropriate models for this purpose, based on specific application requirements. the models we compare are: (a) er (chen, 1976), as a representative of the conceptual models family, (b) ifo (abiteboul and hull, 1987), as a strong mathematical conceptual model, (c) the object model of the object modeling technique (rumbaugh, et. al., 1991), as a typical object-oriented model and, (d) the powerful and mathematically sound relational model. we focus on the more interesting and general points, leaving out modeling aspects concerning particular geographic environments (for example, special network modeling needs). the rest of the paper is organised as following: in section 2 we give a set of exemplary problems for spatial systems on which we test the suitability of each model. in section 3 we discuss the criteria of the evaluation for a good spatial database model. section 4 shows the evaluation of models: in §4.1 we discuss the er, ifo and omt models for the conceptual design of gis, while in §4.2 we argue why the relational model is a good basis to serve spatial peculiarities at the logical level. we conclude in section 5 with the comparative results. desiderata for the geographic database design of geographic applications in this section we give a set of exemplary problems for spatial systems on which the suitability of a model will be tested. practical experience (utilnets, 1994), (tsironis, 1992) as well as theoretical research (tryfona and hadzilacos, 1995a), (tryfona, 1994) reveal the following issues as the most important and critical peculiarities of geographic databases: • object's position in space. in the real world, objects have a position, which is the object's link with space. in information systems we are only interested in position for some objects: these are the geographic or spatial objects of the application. the position of an object includes its location (centroid), shape, size and orientation. position has a fixed meaning: it is a function on all and only on geographic objects and returns for each geographic object a part of space. there is also the need to capture space in order to locate objects in it. space is a set. the elements of space are called points. any set will do for space. a very important intuition and interesting theories come-up from nonstandard spaces; however for practical reasons, in current spatial applications, space is modelled as a subset of r2 orr3. • views of geographic objects. geographic objects may have more than one views in space, e.g., a land parcel can be seen as a point or as a region, depending on the current scale of the application. the need for modeling different views of the same excerpt of the real world does not only steam from the fact that scale changes (multi-resolution representations), but also from the requirements of the spatial database, e.g., the user wants to be able to see and refer to a land parcel either as a point or as a region or both. apart from issues dealing with assessing consistency among these different perspectives, we are also invited to integrate all different views in one single conceptual schema as well as to preserve uniformity of successive results when dealing with them; for example, how is defined the distance between two land parcels when these are captured (a) both as points (b) one as a line and the other as a region? • space-depending attributes. a fundamental peculiarity of spatial information systems is that some properties of interest do not properly belong to any particular object. for example, soiljtype in a cadastral application. although one application view may regard the soiljtype of the land parcels as an attribute of the parcel, it is clear that: (a) it is defined wh"--.-.er or not a land parcel exists at that position in space, and (b) when a land parcel is moved, it will not keep the -:. ,ue of "its" attribute; rather it inherits new values from the new position. these attributes are called spatial or sp :cedepending attributes. informally, space-depending attributes are properties of space which indirectly become properties of objects situated at some position in space. overlapping objects share the same values for these attributes. the value of a space-depending attribute depends on position only, and not on the object itself. formally, a space-varying attribute is a function whose domain is space and range is any set. under this perspective, soiljtype needs to be modelled as a function from space to the set (sand, clay, ...}. space-depending attributes can be found vthe literature also as thematic maps or layers. • spatial relationships. dealing with geographic objects means dealing with relationships among them, e.g., the umjjorden traverses the city ofaalborg. relationships among geographic objects are actually conditions on object's positions and are called spatial or geographic relationships. spatial relationships are translated into spatial integrity constraints of the database. conceptual geographic models should lead to straightforward solutions for explicitly storing topology in the logical and physical levels -a common practice despite topology being derivable from object positions (hadzilacos and tryfona, 1992). • complex geographic objects. constructing complex geographic objects from "simpler" ones involves their position; that "a network is an (ordered) set of network segments" differs from "a class is a set of students" in that the former grouping has a spatial dimension as well: the position of the network is the geometric union of the positions of its constituent segments -whereas nothing of this sort holds in the second case. • retrieval of geographic aspects. there is the need to access geographic objects, their attributes as well as thematic maps or layers. for example two land parcels may be merged or the soil_type of a land parcel changes. methods are the only means to access these "components" of the database. three types of basic methods on geographic databases exist: (a) those which manipulate objects and act only on descriptive attributes; for example, change the name of a river, (b) those which manipulate objects and act on objects' position; for example, retrieve the distance between two buildings, (c) those which manipulate layers (space-depending attributes); there are four types of this category (delis, et. al., 1995): attribute derivation, spatial computation, overlay, and reclassification. for example, overlay the soiljype and the erosion map. methods of types (b) and (c) are called geometric or geographic methods or operations. the database must keep track of these changes which constitute its dynamic aspect. criteria for a "good" geographic database model for the evaluation of geographic data models, we used five criteria as guidelines to test their suitability. some of these criteria can be applied to conceptual models, while some others to logical ones. all of them are "general", in the sense that they can be used for the evaluation of any "classical" modeling technique. but when they are applied to spatial systems they gain additional meaning as they "measure" the models' ability to deal with the spatial peculiarities described in section 2. the five criteria are: (a) expressiveness. the more expressive a model is, the more close to the real world application will be, the more semantics will capture. semantics play an important role in geographical systems as it is common in spatial applications to find different objects with the same semantics as well as same objects with different semantics. (b) power of abstraction. (for the conceptual models). one of the criteria to evaluate conceptual (or semantic) models is their ability to represent real world in a highly abstract way. being able to understand and attribute objects' structure without including details allow us to come closer to objects' semantics and their role in the application. (c) complexity. this appears to be one of the most crucial issues in selecting a database model. the usual trade-off between expressiveness and complexity exists. the more expressive a model is, the more complex appears to be. (d) friendliness. friendliness and ease of adaptation/use of a model relates to its complexity. it is considered important, as a powerful -in terms of expressivenessmodel may result in a useless model in terms of usage. model's friendliness is connected to both user and designer: at the phase of conceptual modeling the user -who is considered as the person who has the domain knowledge of the applicationchecks the conceptual schema and verifies the correctness of the application design, while at the phase of logical modeling the designer translates the conceptual schema into the logical one. in both cases, it is critical to deal with an easy model. (e) extendibility. another important issue is how easily a model can be extended; the general trend for new application areas modeling -such as multimedia, image and voice processing, geographic systemsis to build methods and techniques on top of well-known data models in order to make use of the already existing knowledge from the classical areas. using different models for the design of geographic databases through theoretical research and real world applications, we experienced the use of models and tools for the design and development of geographic applications at: (a) different levels, and (b) different approaches in the same level, (a) at the phase of conceptual geographic database modeling we used: • new approaches, such as the object-oriented one, with representatives the object modeling technique (omt) (rumbaugh, et. al., 1991), and kappa system (intellicorp, 1993), 17 • classical approaches, with the use of the entity-relationship (er) (chen, 1976), the ifo model (abiteboul and hull, 1987), the semantic data model (sdm) (hull and king, 1987), and the general semantic model (gsm) (abiteboul et. al., 1995), • new (specialised or extended) models within certain approaches. in order to capture the peculiarities of gis we augmented models such as the er, ifo, omt, and kappa with the spatial dimension, resulting in the geoer (hadzilacos and tryfona, 1996), geoifo (tryfona and hadzilacos, 1995a), geoomt (tryfona et. al., 1997). in this work we show the evaluation of er, ifo and omt: er, as it is undoubtedly the most popular and easy to use semantic model; ifo, because of its complete and sound mathematical background; and omt as a representative of the object-oriented family. we present the way we handle the six geographic peculiarities of section 2 by using (extending or specialising) each one of these three models^. (b) at the phase of logical geographic database modeling we extended (hadzilacos and tryfona, 1994) the mathematically sound relational model with the aspects of geometry and time. here, we argue why the relational model is good to serve as a basis for an extended model to accommodate spatial needs, but not good enough as a stand alone solution. evaluation of conceptual data models for geographic applications the conceptual modeling phase refers to that level of abstraction which (a) employs no computer metaphors, (b) is understandable to the user who has the domain knowledge of the application, and (c) is formal and complete, so that it can unambiguously be transformed into the logical data model without additional user input. the models er, ifo and omt (a) er (chen, 1976) is arguably the first conceptual model that appeared in the literature. its main advantages are the ease of use that provides and the minimum set of supported constructs. its basic elements are the: (a) entity sets, which represent autonomous ontologies (objects), (b) attributes of entity sets, which capture their properties; properties associate a value from a domain of values for that attribute with each entity in an entity set, and (c) relationships among entity sets, defined as an ordered list of these sets. relationships can be 1:1 (one to one), 1 :m (one to many), n:m (many to many). isa is a special kind of relationship which is used to model an entity set as a subset on another one; the subset inherits all the properties of its ancestor. (b) ifo (abiteboul and hull, 1987) has become a standard conceptual data model due to its precise mathematical specification rather than its widespread use in practice. an ifo schema is a directed graph with vertices and edges. the vertices represent object classes, which may be atomic (further divided into abstract, printable and free) or complex (recursively built from atomic through aggregation and grouping). an abstract atomic data type models an object class whose underlying structure is irrelevant to the application and can not be printed. a printable atomic type corresponds to simple predefined objects that can serve as input or output. a free atomic type represents objects obtained via isa relationships. edges represent functional dependencies. aggregation refers to a form of abstraction that formally composes a new object class from previously defined ones by forming a cartesian product. grouping denotes the collecting of elements of an already existing (atomic) class to form a set. isa relationships in are distinguished in generalisation and specialisation. (c) the object model technique (rumbaugh et. al., 1991) is based on a software development tool for objectoriented analysis and design. it became very popular the last years as it is both powerful and affordable with many of the features of a case (computer aided software engineering) tool. the object model of the omt contains object diagrams whose nodes are object classes and whose arcs are relationships among classes. its basic elements are the: (a) object classes, which represent a set of autonomous ontologies (objects) and show their internal structure. the symbol representing a class has three areas: the upper area contains the class name, the middle area its attributes, and the lower area its operations (methods), (b) attributes of object classes, which capture their properties, (c) associations among object classes, which are used to model relationships, (d) generalisation hierarchy, which is a "is-a" association. attributes and associations of the supertype are inherited by the subtypes, (e) aggregation, which is a "part-of' association and is related to the construction of complex object classes. 2 for a thorough description of this work the reader should refer to the corresponding citations. handling spatial needs at the conceptual level next, we show how each one of the above described models handles the spatial needs discussed in section 2. for reasons of convenience and better presentation, we combine logically these spatial peculiarities, without following the order in §2. • geographic objects' position in space and different (multiple) views of objects. for that purpose, we need to represent space, object's position, connect position to space (position is a part of space) and show different representations of position. we introduced -in all three models, er, ifo, omtspecial object classes (entity sets)-' and functions: (i) a special object class space, (ii) the special object classes positions, "size", "shape", "location" and "orientation", which represent object's position in space; positions is determined fully and non-redundantly as an aggregation of "size", "shape", "location" and "orientation" (tryfona and hadzilacos, 1995b), (iii) a special function "is_located_at" (1:m), which connects the geographic object to its position (one object may have more than one positions in different spaces), (iv) a special function "belongs_to" (m:l) between positions and space, (v) the special object classes "0-dimensional", "1-dimensional" and "2-dimensional", which are "shape"(s) of geographic objects; whenever these classes appear simultaneous they represent different views of the same object. (a) er 0-djraensioml|ll-diroensional i legend: entity-set attribute > relationship 3 for the rest of the paper, we shall use only the term "object class" as equivalent to the term "entity set". furthermore, in the following figures, for reasons of simplicity, we use the term "object" instead of "object class". (b)ifo geographic object positions space -^^^^^^^^ i 0-dimensional ^^ shape 2-dimensional legend: abstract aggregation functional dependency ] printable free isa (c) omt geographic object class is located at 1+ | 1 1 size sha legend: 1±j positions i1* belongs-to 0-dimensional ^ [ 1-dimensional \ | 2-dimensional \ object class attributes methods lrr£ generalization o aggregatbn ?n discussion * as ifo and omt provide (mathematically) the construct of aggregation, we model positions as an aggregation of the four components ("shape", "size", "location" and "orientation"); none of them can be null in order to fully define positions; er does not provide this option, » in er, we connect "shape", "size", "location" and "orientation" to positions via the "part-of' relationship. although this helps us to represent the constructs of positions, the four component-classes don't have any attributes and the design decision seems redundant and not elegant enough, * in ifo, the four components of positions can be shown as printable objects, i.e., they can be represented in a computer system. although this is a logical organisation aspect (and conceptual modeling is independent of implementation issues) it is important for the designer to be early aware of what kind of information he has to implement and how easily this can be done, * in ifo, the object classes "0-dimensional", "1-dimensional", and "2-dimensional" are specialisations of "shape" and not just isa hierarchies. this adds more semantic information as "shape" pre-exists and the three classes represent different roles of it, * in all three cases it was easy for us (the designers) to specialise the models towards the spatial aspects and for the user to follow it. * space-depending attributes and methods on them • space-depending attributes are functional dependencies from space to specific domains. in that way we assign to each part of space a specific value of the attribute. we modelled them either as: (i) attributes or entity-sets connected to space via the relationship "has space-depending attribute" (in er); the two approaches are equivalent as the first results from the second, (ii) functional dependencies (in ifo), or (iii) object classes (in omt). by connecting geographic object classes to positions (via the special relationship "is_located_at", see previous figures) and positions to space (via "belongs_to") objects are related to the space-depending attributes. (a)er space-depending attribute (b)ifo spa space i ^j space-depending attij (c) omt space 1+ space-dep. attr selection at t rib. derivation spatial computat. reclassification h spatial attr. memb attribute name geometry: geomet. data typ. discussion * as ifo is function-oriented, it gave us more flexibility in representing space-depending attributes the way they are: as dependencies (spa) from space to specific domains, * with omt we had even more freedom and modeling flexibility: we created an object class "space-depending attribute" and connected it to space with the function spa. this object class shows actually a layer (coverage) which represent the property. the third part of this class has all the operations (methods) we can apply on this layer: "select an area with a specific value on this attribute", "reclassify based on this attribute", etc. a layer is a "group of parts of space with specific attribute values. for example, a layer representing soil type includes subareas with clay, sandy, etc. soil type. for that purpose we represent "space-depending attribute" as a grouping (*) of "spatial_attr_member". each subarea ("spatial_attr_member") is distinguished by its specific geometric type (point, line, region, or a combination thereof) which is captured by the attribute geometry^, * omt appears difficult to follow it due to the complex constructs it provides. * spatial relationships in all the three models we represent spatial relationships as functions or relationships among object classes. in all cases the resulting schema appears easy to follow it. (a) er spatial relationship (b)ifo geo obj graphic ect 1 <:natial r ilationship geographic object 2 (c) omt geographic object 1 spatial relationship geographic object 2 4 a detailed description of this approach is described in [tryfona, et. al., 1997]. • complex geographic objects extension with "spatial member of and "spatial part of constructs we extended all three models with two new constructs; for ifo and omt we extended the constructs of aggregation and grouping, while for er the "part_of and "member_of" relationships. each one of these mechanisms has specific properties (hadzilacos and tryfona, 1996): (i) the components of "spatial_part_of' are not necessary all spatial; the position of the composed object is the union of the positions of its geographic components, while (ii) the components of "spatial_member_of' are all spatial and have the same geometric type. in the following figures the extensions (new constructs) are highlighted. (a)er complex geographic object complex geographic object spatial_member_of spatial member (b)ifo complex geographic object complex geographic object spatial_part_of spatial_member_of (c) omt spatial_part_of spatial_member_of complex geographic object complex geographic object i spatial_partl spatial member • methods on geographic objects and attributes properties of databases are categorised in: (i) static, which characterise the objects of the database such as "the identification number (id) of a land parcel" and (ii) dynamic, which access on static ones, like "change of landparcel's id". both static and dynamic properties form objects' behaviour. er and efo capture only the static properties. omt captures the dynamic ones also. in the graphic representation of objects by using omt, the symbol has three areas: the upper area contains the class name, the middle area its attributes and the lower area its operations (dynamic properties). in the following example, the geographic object has a name and an id and we can change the id and move object's position. omt evaluation of the relational model for geographic applications the modeling phase that follows the conceptual in the adapted design methodology us the logical one. here, the conceptual schema -which is the graphical representation of the applicationis transformed into a formal, system independent, yet system implementable description. handling spatial needs at the logical level the standard data model for logical database design is the relational. it can be used for geographic applications and has extensively been used so, even at the low level of describing polygons and other geometric features (burrough, 1986). the question is why is the relational model not good enough for spatial applications -while it is still a good basis from which to build one. the power and elegance of the relational model stems from the fact that it uses a single construct: the relation. five fundamental closed operations are defined on relations (union, difference, selection, projection and cartesian product, all others being simply notational shorthands, see (ullman 1988). for spatial applications, however, the resulting representation is inadequate. for example, if layers (maps representing space-depending attributes) are represented with plain relations, operations such as overlaying and reclassification cannot be derived from the fundamental relational ones (delis et. al., 1994). what usually happens when the relational model is used for logical modeling of geographic applications is that these operations are hidden in the physical level. as a result important information is lost and the system is tied to some specific implementation. thus relations are inadequate as the sole modeling construct for geographic applications. a fundamental requirement for spatial database design is the ability to model space-depending attributes (section 2). spatial applications deal with two, orthogonal, generalisations of spatial properties. one is associations of the whole of space with one attribute, and the other is associations of sets of attributes with a geometric feature (point (0-dimensional), line (1-dimensional), region (2-dimensional) or combination there of). the former is modelled with concepts like layers whereas the latter is modelled with concepts oriented towards objects. (couclelis, 1992) offers an insightful view on the orthogonallity of the two approaches. the georelational data model (hadzilacos and tryfona, 1994) is an extension of the relational. it provides a language for the definition of relations (used for non-spatial entities and relationships); of layers (to represent space-dependent attributes); of object classes (to represent geographical entities that have characteristics from more than one layer); and of constraints among objects or layers (used for topological and other spatial relationships). an excerpt of its data definition language follows: define relation rel_name (attr_namei, domain^, <key>),...,(attr_name}c, domain^, <key>) <time_point attrjiamep define layer n <layer_name> <temporal> attr (attr_namein,domainin,<umque>),...,(attrjnamemn n, domainmn n. <unique>) geometric type geometric_type positioning coordinate_system> <constraint topological_condition > 9.4 define object class obj_class_name <geometric type geometric_type> <subtype of sup_obj_class> <on layers layer_idi,...,layer_idk> <with attributes attr_name1(...,attr_namem> <constraint composite_constraint_name> define constraint constraint_name on (object_class_name i layer_id( among(obj_classi,...,obj_classk i layer_idi,...,layer_idjc} as condiiion_specification discussion results we present an evaluation of classical and object-oriented modeling approaches for geographic database design based on spatial peculiarities handling. the whole research effort is under the perspective of the use of a specific methodology for the analysis and design of spatial applications; it is also governed by the idea that gis are special information systems. for the conceptual geographic database modeling we test the er, jjfo and omt tools. all of them need to be firstly specialised and then extended to handle efficiently spatial needs. • er is easy to be extended, but the result is not expressive enough for the spatial semantics. sometimes it appears to be too naive, with no elegant solutions. it can be easily followed by the user. • efo is easy to be extended, the resulting schema appears to be more complex but still can be followed by the users. it provides sophisticated and semantically rich mechanisms to deal with space. • omt provides high level of expressiveness, is more understandable than bfo and can be easily extended. the result is semantically rich enough to represent space. the ability to capture dynamic properties of geographic objects is one of its major advantages, as spatial databases are mainly characterised by changes of objects' behaviour. the following table summarizes the results. er ifo omt expressiveness low medium high complexity low medium medium friendliness high medium medium extendibility high high high at the logical level, the relational model provides a good basis for adding the aspects of geometry and time for gis handling. next step in our research plans is to incorporate temporal aspects in the phase of conceptual modeling of spatial applications. the extendibility and friendliness of the selected models will play an important role as time-varying spatial information is a complex issue in terms of semantics and representation. acknowledgements. the second author wishes to thank the computer technology institute for the sponsorship of the presentation of an earlier version of this work at the international workshop on evaluation of information systems (dtp 8.1), barcelona, catalonia, june 1997. references abiteboul, s., and hull, r. ifo: a formal semantic database model, acm tods, vol. 4, (1987) pp. 525565. abiteboul, s., hull, r., and viannu, v, complex values and object databases chapter 20, addison wesley (1995). benoit, b., laurent, r., and guylaine, s. ge02why objects in a geographical dbms?, in d. abel, b.c.ooi (eds.), advances in spatial databases, symposium on spatial databases (1993) berlin. burrough, p. principles of geographical information systems for land resources assessment, oxford university press (1985). chen, p.s. the entity-relationship model: toward a unified view of data, acm tods, vol. 1, (1976) pp. 936. couclelis, h. people manipulate objects (but cultivate fields): beyond the raster-vector debate, lecture notes in computer science vol 639 berlin: springer-verlag (1992). delis, v., hadzilacos, th., and tryfona, n. an introduction to layer algebra, proceedings of the 6th international symposium on spatial data handling, edinbrough, uk, taylor and francis (1995). hadzilacos, th., and tryfona, n. a model for expressing topological integrity constraints in geographic databases. lecture notes in computer science, vol. 639, pp. 252-268, berlin: springer-verlag (1992). hadzilacos, th., and tryfona, n. logical data modeling for geographic databases, international journal of geographical information systems vol. 10, no 2. (1994). hadzilacos, th., and tryfona, n. extending the entity-relational model to capture spatial semantics, sigmod record (sept. 1997) (to appear). hull, r., and king, r. semantic data modeling: survey, applications and research issues, acm computing surveys, vol. 19, no. 3. (1987). intellicorp inc, kappa manual set (1993). ogis, project document 94-025r1, the open geodata interoperability specification (1994). oxborrow, e., and kemp, z. an object model for distributed multimedia geographic data, proceedings of the 3rd european conference on geographic information systems, munich, germany, 23-26 march, 2, pp. 1294-1303 (1992). pelagatti, g., belussi, a., negri, m., rossi, m., and sbatella, l. design and implementation of a geographic information system: the case of regione lombardia, proceedings of the 2nd european conference on geographic information systems, brussels, belgium, 2-5 april, 2, pp. 856-865 (1991). rumbaugh, j., blaha, m., premerlani, w., eddy, f., and lorensen, w. object-oriented modeling and design. englewood clifs, nj:prentice-hall (1991). scholl, m., and voisard, a. object-oriented database systems for geographic applications: an experience with o2, international workshop on geographical databases, capri, italy, may, 16-17 (1991). tryfona, n. a structured database design methodology for geographic applications, ph.d. thesis, department of computer engineering and informatics, university of patras (1994). tryfona n., and hadzilacos, th. geographic applications development: models and tools at the conceptual level, 3rd acm workshop in geographic information systems, dec. 1-2, baltimore, md. (1995a). tryfona n., and hadzilacos, th. conceptual data modeling for geographic applications: desiderata and solutions, cti technical report cti-95.03. (1995b). tryfona, n., pfoser, d., and hadzilacos, th. modeling behaviour of geographic objects: an experience with the object modeling technique. proceedings of the 9th international conference on advanced information systems and engineering (caise'97), lecture notes in computer science vol. 1250, berlin: springer-verlag (1997). tsironis, g. a network utility management system, diploma thesis, department of computer engineering and informatics, university of patras (1992). ullman, j. principles of database and knowledge-base systems, vol. 1, rockville, md: computer science press (1991). utilnets, work program of brite-euram project 7720, dg xii, european union, brussels, belgium (1994). worboys, m., hearnshaw, h., and maguire, d. object-oriented data modeling for spatial databases, international journal of geographic information systems, vol. 4, no 4 (1990). 7.6 paper title (paper title style) australasian journal of information systems ong, islam and zhao 2017, vol 21, selected papers from ausdm preface 1 preface to the special section of selected papers from the australasian conference on data mining (ausdm) 2016 kok-leong ong la trobe business school la trobe university kok-leong.ong@latrobe.edu.au zahid islam school of computing and mathematics charles sturt university zislam@csu.edu.au yanchang zhao data61 csiro yanchang.zhao@csiro.au this special section is a compilation of selected papers invited from australasian data mining conference (ausdm) 2016. the ausdm series began in 2002 as a workshop and has grown each year since. the conference is devoted to the art and science of data analytics and has in the recent years, heavily promoted the use of analytics in various business applications through an industry track reflecting the common goals and focus shared by the australasian journal of information systems. it is therefore a privilege to have selected papers from ausdm 2016 to be included for publication at the australasian journal of information systems. papers selected and invited from ausdm 2016 underwent a rigorous review process in line with the requirements of the journal. these papers include either new findings since the conference publication, or include additional content from the conference paper. the group of reviewers that have contributed to the reviewing of these publications are acknowledged below and as editors, we are deeply appreciative of their contributions to the process. of course, we wish to specially acknowledge the on-going support and patience of the editor in chief, associate professor john lamp, whom we sincerely thank for accepting this special section proposal and the on-going support to make this happen. • lianhua chi, ibm australia • selvaraaju murugesan, act government • wei peng, la trobe university • wenyuan li, university of southern california • rana ahmed, alrafidain university college • yee-ling boo, rmit university • jie chen, university of south australia • leminh kieu, data61, csiro • khondker jahid reza, charles sturt university • mouhib alnoukari, syrian virtual university • ickjai lee, james cook university australasian journal of information systems ong, islam and zhao 2017, vol 21, selected papers from ausdm preface 2 as we mentioned in the previous 2015 special section preface, the “need for organisations to make sense of their data has never changed”. if anything, the ways in which we can now analyse data for various business applications and problems have evolved. and in 2017, this represents another maturing view of business analytics as well as a shift in focus within the research community in the last few years to concentrate more on how analytical outcomes can be used, evaluated and effectively communicated. in some ways and on reflection, these questions might have been dismissed by hardcore computer scientist as problems in the realm of information system researchers. within the emerging discipline of business analytics however, we are starting to see hardcore analytical methods and research increasingly overlapping with traditional problems or questions that were once considered by information systems researchers. this crossing of two disciplines is an exciting evolution that we are seeing and most likely, to take a bigger leap forward from 2018 and beyond. the selected papers from the 2016 ausdm conference have reflected this with many developing solutions that draw and synthesize from multiple disciplines. we believe you will find the collection an interesting read and we hope you enjoy this special section. sincerely, ong, islam and zhao special section editors copyright: © 2017 ong, islam and zhao. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ a case study on the application of pvs to requirements analysis georg droschl droschl @ist.tu-graz.ac.at austrian research center seibersdorf and institute for software technology, technical university of graz miinzgrabenstr. 11/2, 8010 graz, austria, europe abstract this paper presents the results of a formal methods case study in which the prototype verification system (pvs) has been used for requirements analysis of one module of a physical access control system. pvs is a tool for writing formal specifications and constructing proofs. previously, the same requirements have been analyzed by means of testing supported by the ifad toolbox for vdm-sl. the capabilities of the two formal methods are compared. introduction formal methods informal requirements are inappropriate to serve as a basis for software development. still, most requirements documents are expressed in natural language. "formal methods are one of a number of techniques that when applied correctly have been demonstrated to result in systems of the highest integrity" (bowen and hinchey, 1995). formal methods exploit the power discrete mathematics. their mathematical basis includes the disciplines of set theory, abstract algebras, the lambdacalculus, petri nets, etc. the term 'formal methods' embraces formal specification and verified design. the two main approaches to formal specification are called model-oriented, and property oriented. model-oriented specification models the behavior of the system by choosing specific data domains, and by defining functions and operations on the data domains. model-oriented specification languages include ccs, csp, b, vdm-sl, and z. the basic idea of property oriented specification is to describe data structures by only giving names of data structures and basic functions, and by defining characteristic properties among the functions. the main asset of the property oriented specification style is to encourage underspecification specifying less rather than more, and doing so as abstractly as possible thereby avoiding the tendency to focus on how a concept is realized rather than what is required of it. property oriented approaches include clear, larch, lotos, and obj. a good introduction to formal methods is given by (clarke and wing, 1996). this paper includes an overview of notable methods and industrial applications. requirements analysis css is a comprehensive security system which has been developed at the austrian research center in seibersdorf (arcs). css includes features like digital video recording and automatic door control. in a series of case studies, the ssd module is used as a practical example for the investigation of the benefits of formal methods (droschl, 1999a; droschl, 1999c). prior to the present case study, ssd's requirements have been analyzed using ifad's vdmtools (elmstrom et al., 1994). by animating the specification using the ifad toolbox, a simple test case has been found, revealing the presence of inconsistencies. however, we failed to establish a better understanding of the nature of the contractions. second, several minor ambiguities had to be resolved. the results of the analysis using the ifad toolbox are summarized in fig. 1. the toolbox includes a wide range of useful features: it can animate a subset of vdm-sl constructs by applying a test suite. in an interactive mode, functions and operations may be invoked. test coverage is a powerful feature which enables the user to determine to which extent the specification is covered by the test suite. due to an automatic c++ code generator, the toolbox can also be used in projects where code has to be delivered. 146 assets supports animation / testing statement / expression based test coverage automatic c++ code generation liabilities no test case generator readily available exhaustive test difficult test ,,low level" figure 1: experiences with ifad's toolbox for vdm-sl for requirements analysis. in order to guarantee a systematic approach to testing the specification, a test suite has to be selected in a thorough manner (poston, 1996). however, the toolbox does not have a test case generator. this and the huge size of the state space of ssd makes an exhaustive test difficult. according to our experience, testing supported by the ifad toolbox is a "low level" activity. it would be desirable to generate test cases based on general properties. in order to overcome this problem, parts of the logic of ssd were used for the generation of test cases (droschl, 1999b). tool support is known to be one of the success factors in formal specification based analysis. essentially, there are rewriting based methods (including theorem proving) and state enumeration based methods (including the application of test cases like in the ifad toolbox, and model checking (holzmann, 1997)). for an overview to formal specification based analysis see for example (clarke and wing, 1996; nasa, 1997). in enumeration based techniques, the size of the state space is known to be the major limiting factor. in the present case study, for the following reasons a theorem prover has been selected: first, ssd has a huge state space. as mentioned above, this makes achieving complete test coverage a difficult task. second, we were interested in evaluating an approach which is opposed to the one employed in the first case study. the list of theorem provers provided by paulson17 includes isabelle, pvs, hol, otter, coq and z/eves. despite its potential bugs18, the prototype verification system (pvs) (owre et al., 1992; rushby and stringercalvert, 1995) is one of the most popular tools. thus, it is very well supported. pvs is a tool for writing specifications and constructing proofs. it comprises a specification language, a number of predefined theories, a theorem prover, and various utilities. the specification language of pvs supports property oriented specifications. the prototype verification system combines an expressive logic with an interactive proof checker that supports top-down proof exploration and construction. in addition to its proof checker, the pvs system includes a parser, prettyprinter, and typechecker. pvs has been applied to many applications including real-time (kellomaki, 1997; van de pol et al., 1998), protocols (rajan and fujita, 1997), avionics (dutertre and stavridou, 1997), space applications (lutz and ampo, 1994), and digital circuits (miner and johnson, 1996; shostak, 1983). pvs has been used for requirements analysis before (butler, 1996; di vito and roberts, 1996; dutertre and stavridou, 1997; easterbrook et al., 1998; heimdahl and czerny, 1996). the issue of integrating vdm and pvs for ssd is discussed by (agerholm, 1996) and (droschl, 1999c). structure of this paper this paper is structured as follows: section 2 gives a general presentation of ssd, followed by some advanced features. that section is concluded by a discussion of ..switches", which are the subject of this analysis. in sec.3 a formal pvs specification for switches will be developed. in the second half of that section, the specification will be used for proving requirements properties. section 4 discusses the insights gained in the previous section, as well as a comparison to previous work. 17paulson's list of theorem provers http: / /www. el. cam. ac. uk/users / icp/ 18pvs frequently asked questions and bug tracking database. sri computer science laboratory, menlo park, ca, usa. http://pvs.csl.sri.com/bugs.html 147 an access control system in this section, some of ssd's features will be presented, including the notions of guard, round, station, target, and in particular switches. basic features ssd is one module of an access control system called css. ssd provides an interface between the guards on duty, devices like intrusion-circuits, and the operator. the basic principle is as follows: a number of guards supervise a physical plant by night. by means of rounds, the plant is divided in areas. each round consists of a list of stations. the guards are required to visit (and "hit") stations one-by-one. this principle is illustrated by fig. 2. factory, bank, ...: by night figure 2: a site to be supervised by a guard. stations are physical devices (terminals) which are spread over the site. stations are shown as terminals over triangles. the area to be supervised is defined by a series of stations, called a round. stations which shall be visited by the guard are marked in black. the most obvious task of a guard is to visit stations. typically, in one round, there is one guard. the system and the entirety of guards are supervised by an operator. the operator interface includes features to select, interrupt, and terminate rounds. he or she may also take stations out of order temporarily (deactivate them), or enable the guard to continue the round at any station if the round has been interrupted. advanced features ssd supports more advanced features, including the following: a guard may be on his own, and two guards may form a team. rounds may run in parallel, and share stations. under certain circumstances, a guard may hit a station of another guard's round. the guard's identity has to be checked whenever a station has been hit. however, there are stations which do not return the identity of the guard. if the guard is threatened by an intruder, the guard can make use of a feature called "silent operator notification". there is a range of causes which cause a round to be interrupted. then, as part of a recovery procedure, the operator must clear the interrupt. to catch up after a (possible) delay that has occurred after an interrupt, the operator may enable the guard to skip any of the following stations. one of ssd's advanced features is the switching mechanism. 148 switches css is a system which may be used for night-time supervision of physical plants. at night, doors are usually locked, lights are off and intrusion circuits are activated. all of these items represent obstacles for the guards who need to circulate. a switching mechanism is concerned with unblocking and blocking of these obstacles whenever it is necessary. essentially, the guard's interface to the system is restricted to a location mechanism: when a guard has hit a station, the system is aware of the position of the guard in the site. unblocking (and blocking) must take place automatically. since such a switch triggers the submission of data to one (or many) obstacles, they are also called switch targets or simply targets. each target may be in one of two states: blocked or unblocked. when a station gets hit by a guard, switches may cause the state of the target to change. the action of unblocking a target is called an entry-switch, and blocking the target is referred to as an exit-switch. this principle is shown in fig. 3. /\ e n t r y ! [iden t i fy ! swi t ch \ / a station n stat ion n+1 figure 3: part of an example round: the guard first visits some station n which causes an entry switch to the given target. the guard then proceeds to station n + 1, which will send an exit switch. unfortunately, the switches principle becomes (unnecessarily) more complicated. first, parts of the requirements are given on an operational level: rather than stating the properties a solution satisfying the requirements shall have, an explicit algorithm is proposed. second, there are an important number of special cases, which all need to be treated separately. also, some of these cases may occur simultaneously. motivation for this analysis we will now show the test case that has indicated that there are contradictions in the requirements. on the way through the round, the guard has to visit stations one after another. however, the operator may temporarily deactivate single stations while leaving the round functioning. as pointed out above, for a certain scenario the proposed requirements are known to fail: in a round with 5 stations, number 2 and 4 inactive, the guard may be unable to finish the round, because a target is impassable. a detailed explanation of why this is so goes beyond the scope of this paper. clearly this given scenario should not require the operator to interfere. thus, it is unacceptable. system size the present paper reports on one of a series of case studies. these case studies and the original development are carried out in parallel. they are all based on a collection of 60 informal requirements rules, given on 10 a4 pages. the vdm specification consists of 60 a4 pages, and the pvs specification covering only the switches aspect is about 1000 lines long. 149 formal specification and analysis the previous section has briefly introduced some of ssd's principles. in the present section, the discussion will get a little more technical. first, a formal specification will be developed. it will then be used in the proof of safety and liveness properties. formal specification we will begin by formalizing some of the concepts which are related to switches. the full pvs specification consists of about 1000 lines, including empty lines and comments. for brevity, only a few examples can be given. terminology pvs supports identifiers composed of uppercase and lowercase letters as well as a wide range of special characters. in order to make the specification more easily readable, we have introduced a few restrictions. names of types will be written in uppercase_letl brs, functions and constants in lowercasejetters. boolean_valued_functions? have a question mark at the end of their name. pvs functions are total. thus, undefined values can be handled by means of supertypes and subtypes. in the specification, there are plenty of type declaration pairs where one is a subtype of the other. in these cases, the supertype has an additional question mark at the end, as, for example, in t_st and t_st?. some more information on superand subtypes will be given below. a glimpse on the specification a target is a physical device which can be in one of the two states blocked and nonblocked. in state blocked, the target is impassable for the guard. examples for targets are light switches, doors, and intrusion circuits. target identifier. t_id? and t_id are datatypes. they both contain all physically existing (,,valid") target id's. in t_id? there is an extra value called null_tid. predicate t_id_notnull? holds for all valid targets. as pointed out above, t_id is a subtype of t_id?. note the "+" in the declaration of t_id? which makes sure that the type is non-empty. t_id? : type+ nuluid : t_id? t_id_notnull? ( t id : t_id? ) : bool = tid /= null.tid t_id : type = (t_id_notnull?) target state. t_st indicates the present state of the target: blocked or nonblocked. t_st contains all values of t_st? but t_st_null. again, t_st is a subtype of t_st?. t_st? : type = { nonblocked, blocked, t_st_null } t_st_notnull? ( t s t : t_st? ): bool = 1st /= t.st.null t_st : type = (t_st_notnull?) round identifier. essentially, a round consists of a list of stations. stations are devices which are visited and hit by the guard on his or her way through the site. examples for stations are card readers, terminals, or ordinary push buttons. similar to the target id, each round has a unique identifier of type r_id. r_id? : type+ null_rid : r_id? r_id_notnull? (rid : r_id? ) : bool = rid /= null_rid r_id : type = (r_id_notnull?) target in the pvs specification, a target is modeled by its state and a reference to the round that has last modified that state. each of these fields may hold null values: first, if tjastmod? is undefined, then the model is unaware of a previous state change. however, in such a case the target may be either blocked or nonblocked. second, a null value of the state is used to indicate that it is a non-existing target null_t. 150 t? : type= [# t_state? : t_st?, llastmod? : r_id? #] t_notnull? (t: t?): bool = t_st_notnull?(t_state?(t)) t : type = (lnotnull?) null_t: t? = (# t_state?:=t_st_null, t_lastmod?:=null_rid #) t_st_mod_yet? (t: t) : bool = r_id_notnull?(t_lastmod?(t)) target database. the collection of known targets is modeled as a mapping from target id's to targets. t? is used as the domain of this mapping. since in pvs mappings (they are functions) are total, some id's may map to a null target. it should now become clear why there has to be a null value for the target type t?. t_db : type = [ t_id -> t? ] system state. according to the pvs model, the system state consists of the target database defined above, and a round database which is specified elsewhere using the same principles. sys_st: type =[# rdb:r_db, tdb:t_db #] event. an event of type ev is a subtype of uu_ev. essentially, it models the transition from one system state to another, triggered by some switch. uu_ev : type = [# sys.before : sys_st, trig : tr, sys_after: sys_st #] scenario. a scenario of type scenario is a subtype of u_scenario. a scenario is the set of all possible sequences of events. according to the pvs model, these are defined by means of axioms. this is in contrast to (droschl, 1999b) and (droschl, 1999c), where ssd's scenario is essentially defined as a relation between all pairs of events. u_scenario : type = list[ev] formal proof of requirements in the first part of this section, parts of ssd's formal pvs specification have been shown. now, this specification will be used in a discussion of formal proof of requirements. which kind of properties can be proved in pvs ? in general, axioms and theorems. we defined a scenario as the set of all possible sequences of events. let p(scenario) be a certain predicate over some given scenario. the following axiom formally states that property p should hold for all possible scenarios. example_property: axiom forall (scenario:scenario): p(scenario) safety and liveness an example rule taken from the requirements is given in fig. 4. the main problem of requirements validation is to find something to validate the system against (sommerville and sawyer, 1997). this is particularly true for 151 this present problem, simply because the switches' requirements lack a clear definition of which high-level properties shall hold. instead, most of the requirements are given on a low operational level. rule 17: an exit switch may only be sent, if the target is in state nonblocked, set by the same round. figure 4: one rule from ssd's requirements. it is meant to serve as an example showing that many of the requirements focus on a low operational level rather than stating which high-level properties should hold. however, from re-reading the requirements and interviewing domain experts, the issues shown in fig. 5 have emerged. these will serve us as a starting point for further analysis. a missing link in the requirements in an attempt to formalize the safety requirement given in fig. 5, it has turned out that there is a missing link between a switch and the station that needs to be hit in order to trigger that switch. this is particularly surprising, because the operator needs to be aware of such a concept in round setup. liveness: is it possible that a guard may not finish a round, for example, because a door remains locked ? safety: is the state of the target after the guard hit equal to its initial state ? figure 5: two liveness and safety properties. a more restricted property there are two main consequences: first, an assumption19 has to be added to the requirements in order to avoid possible misunderstandings between developers and users of the system. second, even though such an assumption could be made prior to further formal analysis, we need to find a more relaxed liveness property for ssd. otherwise, a conclusion meaningless for the existing implementation of ssd could possibly be drawn from a too restrictive assumption. the following axiom tst_do_not_change states that for all scenarios sc the initial target state is equal to the final one. tst_do_not_change: axiom forall (sc:scenario): ( length(sc)?=l and forall (ev:ev), (rid:r.id): member(ev,sc) and rid=tr_rid(trig(ev)) ) implies tdb(sys_before(nth(sc,0))) = 19please recall that there are entry and exit switches. an entry switch unblocks some target/obstacle, whereas and exit switch blocks it. an assumption could be of the following form: if some target ti is assigned to some station si in terms of an entry switch, the guard passes that target on the way from station si to station si,i. if a target t2 is assigned to some station sj in terms of an exit switch, the guard passes it between station sj.i and station s^ (it is assumed that stations si, s i4 l and sj-i , sj are not deactivated). 152 tdb(sys_after(nth(sc,length(sc)length(sc)?=l restricts the consideration to those scenarios consisting of one event or more. member(ev,sc) makes sure that event ev is indeed part of scenario sc. round id rid is bound to the round related to the trigger by rid=tr_rid(trig(ev)). tdb(sys_before(nth(sc,i))) extracts the states of the targets before the ith event. according to our understanding, this property should be provable. however, we did not succeed. is it impossible to show, based on the given specification ? for several reasons, this is a difficult question to answer: first, pvs has a great variety of prover commands with plenty of options. which ones are suitable ? second, the very structure of the specification is known to have a major impact on the success of a proof. third, it is not straightforward to track down the relationship between the description of pvs logic in the manuals and its actual behavior. finally, pvs is known to have possible sources of errors. the following two axioms can be shown to hold: tst_may_change states that there exists a scenario where the equality does not hold. in fact, it is the inverse of the previous axiom. tst_do_not_change_restrict can also be proved. it states that for all scenarios, where there is no interrupt, the initial target state is equal to the final one. tst_may_change: axiom exists (sc:scenario) : ( length(sc)?=l and forall (ev:ev), (rid:r.id) : member(ev,sc) and rid=tr_rid(trig(ev)) ) implies tdb(sys_before(nth(sc,0))) /= tdb(sys_after(nth(sc,length(sc)1 ))) tst_do_not_change_restricted : axiom forall (sc:scenario) : ( forall (ev:ev), (rid:r.id) : length(sc)?=l and r_state(rdb(sys_before(ev))(rid)) /= interrupted and member(ev,sc) and rid=tr_rid(trig(ev)) ) implies tdb(sys_before(nth(sc,0))) = tdb(sys_after(nth(sc,length(sc)1 ))) discussion this case study had two main objectives: first, to analyze ssd's requirements. the results will be given in sec. 4. 1 . second, to investigate the benefits of formal proof on an industrial application. prior to this case study, the requirements have been analyzed by means of testing supported by the if ad toolbox. a comparison between these alternative approaches will be given in sec. 4.2. application specific result formal properties. in this case study, a more thorough understanding of the requirements has been achieved, including a few properties which have been expressed formally by means of axioms. in a sense that their level of generality can be chosen by the developer, axioms have shown to be more flexible than single test cases. complementary models. in the first case study, all of ssd's requirements have been formally specified. the ifad toolbox for vdm-sl has been selected, because it supports both analysis and code development. ssd's vdm specification may be used with little modifications to automatically generate c++ code. the pvs specification developed in the present case study is targeted at analysis only. thus, the two models are complementary. they not only cover different parts of the requirements (the pvs specification is only concerned with the switches part), they are also given in a different manner: in the vdm specification, events are modeled by means of executable functions. the pvs specification makes use of axioms. those interested in the details of the vdm specification are asked to refer to (droschl, 1999d; droschl, 1999b). the (non-)duality of the vdm and pvs specifications is investigated more closely in (droschl, 1999c). missing link in requirements. this work has revealed that the requirements lack a ,,link" between switches and stations. there is no statement like the following one: when the guard has previously visited station i, and an entry switch is required such that the guard may reach station i + 1, then that switch shall be assigned to station i. vice versa for exit switches. 153 clearly, using such an assumption in this analysis could have lead to a conclusion meaningless in practice. in consequence, a more relaxed property has been used. proof vs. testing the if ad toolbox can help to analyze specifications by means of animation and testing. one possible scenario of using testing for requirements analysis is the following: first, identify the properties to be checked. then, select a suitable collection of test cases for which that property should be checked. run the test cases and evaluate the properties. in these two case studies, testing has turned out to be much easier to handle than formal proof. however, this has been the first project where the author of this paper was involved in testing and formal proof of an application of this size. what seems to make testing a challenge is the need to cover all relevant test cases. in theory, even if just a single test case is left unconsidered, the entire result may turn out to be false (droschl, 1999b). as already pointed out in the first part of this section, the vdm and the pvs models are complementary: for example, in the vdm specification events are modeled by means of executable functions. in pvs, the emphasis has been on analysis aspects. thus, non-executable concepts could be used. initially, even the vdm specification has made use of non-executable concepts: some of the functions have been defined by means of preand post conditions. however, these implicit functions have subsequently been complemented by their explicit counterparts. in general, it is easier to develop an executable model from a collection of implicit function definitions, than from a specification with lots of axioms. in a situation, where a major body of knowledge exists in vdm (the pure vdm specification without comments is about 30 a4 pages long), there are two main arguments for performing an analysis in the ifad framework: first, there is no need for developing a new specification in an alternative environment. this could result in a major effort, partly due to differences in the logics. even though there are efforts to extending the ifad toolbox with a prover (see for example (agerholm and frost, 1997)), at present, there is no such feature. second, a developer is believed to be able to get started testing a specification in a reasonable time. even though the application of single test cases may be straightforward and insightful, their choice remains the critical step. in the first case study, a test case has been identified indicating the presence of contradictions in the requirements. in order to achieve a more thorough understanding of these contradictions, an alternative analysis approach had to be found. parts of the requirements have been re-formalized in the pvs environment. even though it has been shown how a vdm-sl specification can be turned into its pvs counterpart (agerholm, 1996), the original requirements have been used as the basis for formalization: first, only parts of the requirements had to be formalized in pvs. second, the structure of the vdm specification is targeted at making the specification executable. this was in contrast to the aims of the pvs specification. third, the construction of a "fresh" specification out of the original requirements was believed to lead to good structure. there are a variety of formal methods tools. most of them have their own specification languages, incompatible with other languages. at best, there are syntactic differences only. however, this is rarely the case. current attempts of integrating formalisms (ambriola et al., 1997; kramer et al., 1996) indicate that the transition from one formalism to another is a key element to the efficient application of formal methods. this has also been observed as part of our work (droschl, 1999c). in principle, using pvs for requirements analysis seems to be a promising approach. theorems can be used to express arbitrarily general/abstract properties. in contrast, single test cases cover some very restricted aspect only. however, the (potential) user of pvs may face substantial obstacles. for example, pvs has shown to require a significantly longer start up time than testing. also, proof attempts can get quite challenging from a technical perspective. on the one hand, pvs supports a variety of different types of proofs. on the other, this results in a many prover commands and a confusing user interface. the need for a practical user interface in formal methods has been recognized before (heitmeyer, 1998). 154 figure 1 summarized the main findings of the vdm analysis. in fig. 6 the results of this analysis using pvs are given. assets formal proof has shown to lead to a better understanding of the requirements than testing. proof is flexible level of generality of axioms can be chosen by the user. the act of thinking about and of writing down properties may yield important insights. liabilities proof is more difficult than testing. concrete form of specification may have a major impact on the success of proof. the pvs user interface suffers from a large variety of commands. re-formalization may result in a significant effort. figure 6: experiences with pvs for requirements analysis. further work model checking can become feasible once a suitable abstraction is found. this approach has not been pursued yet, because it was unclear which aspects to abstract away from. however, it would be interesting to see if our understanding of ssd's requirements could further benefit from model checking, and how these techniques relate to previous research. an important issue is the construction and maintenance of formal specifications. due the results of requirements analysis, the informal requirements may now be changed. obviously, this will result in modifications of the vdm and the pvs specifications. even though the informal requirements were thoroughly formalized, the modifications may result in a major effort. one approach to overcome this problem is given in (droschl, 1999e). however, only model oriented specifications have been considered. conclusion in this paper, we have presented the experiences gained in a formal methods case study in which the prototype verification system has been used for requirements analysis. the experiences with this formal method have been compared with previous work where the if ad toolbox has been used for a similar purpose. the conclusion is that both approaches have their weeknesses and strengths the if ad toolbox can be used as part of larger projects where c++ code has to be developed. however, it lacks a test case generator which can make testing a painstaking task, depending on the application. the kind of analysis that is supported by pvs frees the user mostly from having to bother with low level details. however, a certain amount of technical knowledge is required for using this tool it effectively. also, the variety of commands at the interface may be considered difficult to overlook. finally, the concrete form of specification may have a major impact on the success of proof. acknowledgments the author of this paper would like to thank wolfgang herzner, walter kuhn, michael thuswald, and erwin schoitsch (of arcs) for their interest in applying formal methods to an industrial application. arcs grants the author a phd scholarship. peter lucas (software tech., tu graz) has provided invaluable guidance throughout this project. pascal fenkam, ralf huuck, christian luidolt, walther neuper, christian schinagl, and rudi schlatte have read earlier versions of this paper. 155 references agerholm, s. (1996). translating specifications in vdm-sl to pvs. in j. von wright, j. and j. harisson, editors, proceeding of the 9th international conference on theorem proving in higher order logics, volume 1125 of lecture notes in computer science, pages 1-16, turku, finland. springer verlag. agerholm, s. and frost, j. (1997). an isabelle-based theorem prover for vdm-sl. in gunter, e. and felty, a., editors. theorem proving in higher order logics: 10th international conference, tphols '97, volume 1275 of lecture notes in computer science, pages 1 -?? springer-verlag. ambriola, v., cignoni, g., and fernstroem, c. (1997). current issues on integration. in schafer, w., editor, proceeding of software process technology: 4th european workshop ewspt '95, noordwijkerhout, the netherlands, pages 197-?? springer-verlag. lecture notes in computer science no. 913. bovven, j. p. and hinchey, m. g. (1995). ten commandments of formal methods. computer, 28(4):56-63. butler, r. w. (1996). an introduction to requirements capture using pvs: specification of a simple autopilot. nasa technical memorandum 110255, nasa langley research center, hampton, va. clarke, e. m. and wing, j. m. (1996). formal methods: state of the art and future directions. technical report cmu-cs-96-178, carnegie mellon university. davis, a. m. (1993). software requirements: objects, functions and states. prentice hall, englewood cliffs. davis, a. m. et al. (1993). identifying and measuring quality on software requirments specification. in proc. software metrics symp., pages 141-152. ieee cs press. di vito, b. l. and roberts, l. w. (1996). using formal methods to assist in the requirements analysis of the space shuttle gps change request. nasa contractor report 4752, nasa langley research center. droschl, g. (1999a). analyzing the requirements of an access control using vdmtools and pvs (abstract). in wing, j. m., woodcock, j., and davies, j., editors, proceedings of fm'99 formal methods, world congress on formal methods in the development of computing systems, toulouse, france, number 1709 in lecture notes in computer science. springer verlag. droschl, g. (1999b). design and application of a test case generator for vdm-sl (extended abstract). in fitzgerald, j. and larson, p. g., editors, workshop materials: vdm in practice!, part of the fm'99 world congress on formal methods, toulouse. springer-verlag. droschl, g. (1999c). events and scenarios in vdm and pvs. in 3rd irish workshop in formal methods, galway, electronic workshops in computing. springer-verlag. droschl, g. (1999d). formal specification and analysis of an access control using ifad's vdmtools. technical report ist-tec-99-06, institute for software technology, tu-graz, austria. droschl, g. (1999e). interactive development and maintenance of model-oriented formal specifications. technical report ist-tec-99-19, institute for software technology, technical university of graz, austria. submitted for publication. dutertre, b. and stavridou, v. (1997). formal requirements analysis of an avionics control system, volume 23 of ieee transactions on software engineering, pages 267-278. easterbrook, s., lutz, r., covington, r., kelly, j., ampo, y., and hamilton, d. (1998). experiences using lightweight formal methods for requirements modeling, volume 24 of ieee transactions on software engineering, pages 4-14. elmstrom, r., larsen, p. g., and lassen, p. b. (1994). the ifad vdm-sl toolbox: a practical approach to formal specifications, volume 29 of acm sigplan notices, pages 77-80. fitzgerald, j. and larsen, p. g. (1998). modelling systems practical tools and techniques in software development. cambridge university press, the edinburgh building, cambridge cb2 2ru, uk. heimdahl, m. p. e. and czerny, b. j. (1996). using pvs to analyze hierarchical state-based requirements for completeness and consistency. in ieee high-assurance systems engineering workshop (hase '96), niagara on the lake, canada, pages 252-262. heitmeyer, c. (1998). on the need for "practical" formal methods. in ravn, a. p. and rischel, h., editors, proceedings of the fifth international symposium on formal techniques in realtime and faulttolerant systems (ftrtft'98), lyngby, denmark, volume 1486 of lecture notes in computer science, pages 18-?? springer-verlag. hoare, c. a. r. (1996). how did software get so reliable without proof ? in gaudel, m.-c. and woodcock, j., editors, fme '96: industrial benefit and advances in formal methods, volume 1051 of lecture notes in computer science, pages 1-17. springer-verlag. holzmann, g. (1997). the model checker spin. ieee trans, on software engineering, 23(5):279-295. special issue on formal methods in software practice. kellomaki, p. (1997). verification of reactive systems using disco and pvs. in fitzgerald, j., jones, c. b., and lucas, p., editors, fme'97: industrial applications and strengthened foundations of formal methods (proc. 4th intl. symposium of formal methods europe, graz, austria, september 1997), volume 1313 of lecture notes in computer science, pages 589-604. springerverlag. isbn 3-540-63533-5. 156 kramer, j., finkelstein, a., and nuseibeh, b. (1996). method integration and support for distributed software development: an overview, volume 1078 of lecture notes in computer science, pages 115-??, baltimore, md. springer-verlag. lutz, r. and ampo, y. (1994). experience report: using formal methods for requirements analysis of critical spacecraft software. in proceedings of the 19th annual software engineering workshop, pages 231-248, greenbelt, md. nasa goddard space flight center. miner, p. s. and johnson, s. d. (1996). verification of an optimized fault-tolerant clock synchronization circuit: a case study exploring the boundary between formal reasoning systems. in sheeran, m. and singh, s., editors, designing correct circuits, bastad, sweden. springerverlag electronic workshops in computing. nasa (1997). formal methods, specification and verification guidebook for verification of software and computer systems. vol 2: a practitioner's companion. nasa, washington, dc, usa. owre, s., rushby, j. m., and shankar, n. (1992). pvs: a prototype verification system. in kapur, d., editor, llth international conference on automated deduction (cade), volume 607 of lecture notes in artificial intelligence, pages 748-752, saratoga, ny. springer-verlag. plat, n. and larsen, p. g. (1992). an overview of the iso/vdm-sl standard, volume 27 of sigplan notices, pages 76-82. poston, r. m. (1996). automating specification-based software testing. ieee computer society, los alamitos. rajan, s. p. and fujita, m. (1997). atm switch design: parametric high-level modeling and formal verification. in johnson, m., editor, algebraic methodology and software technology, amast'97, volume 1349 of lecture notes in computer science, pages 437-450, sydney, australia. springer-verlag. rushby, j. and stringer-calvert, d. w. j. (1995). a less elementary tutorial for the pvs specification and verification system. technical report sri-csl-95-10, computer science laboratory, sri international, menlo park, ca. revised, july 1996. available, with specification files, at http://www.csl.sri.com/csl-9510.html. shostak, r. e. (1983). formal verification of circuit designs. in uehara, t. and barbacci, m., editors, computer hardware description languages, pages 13-29. north-holland. sommerville, i. and sawyer, p. (1997). requirements engineering. wiley, chichester. van de pol, j., hooman, j., and de jong, e. (1998). formal requirements specification for command and control systems. in engineering of computer based systems (ecbs), pages 37-44, jerusalem, israel. ieee computer society. 157 the australasian journal of information systems (ajis) began in 1993 and was based on the simple realisation that there was a need for an academic ‘outlet’ for is research in australia australasian journal of information systems volume 17 number 2 2012 137 collaborative ethnography for information systems research: studying knowledge work practices and designing supportive information systems ronald maier university of innsbruck ronald.maier@uibk.ac.at stefan thalmann university of innsbruck stefan.thalmann@uibk.ac.at abstract understanding knowledge work and supporting it with information systems (iss) are challenging tasks. although there have been numerous studies on knowledge work in varying contexts, many of them are not detailed enough or outdated, particularly with respect to is support. knowledge work underwent dramatic changes over recent years, and empirical research studies into how knowledge work is currently performed are scarce. ethnography would appear to be the most suitable qualitative research method for studying knowledge work, were it not too time-consuming, costly and unfocused for the fast-changing is domain. moreover, if we want to ensure the take-up of results from qualitative studies, these need to be transformed into artefacts for the design and engineering of is requirements. this paper proposes a procedure for the study of knowledge work practices that was adapted from collaborative ethnography, and can be used to inform requirements gathering for, and the design of, is. the procedure is illustrated with an example of a collaborative ethnographic study which involved seven organisations in four european countries who took part in a large-scale international is research and development project. the paper also critically discusses the procedure’s implementation and its limitations. introduction increasing the productivity of knowledge work is a key priority for businesses and organisations in the light of the increase in the share of highly paid knowledge work and of its outputs, with profound impacts on current and future business performance (drucker 1994; wolff 2005). knowledge work takes place in complex social settings. understanding it and turning the insights gained into artefacts informing is design are challenging tasks for requirements engineers and designers of iss. conversely, not understanding its processes and the (social) conditions of its performance can constitute a major obstacle to successful is development. many projects that focused only on technical issues were unsuccessful, since they did not sufficiently take into account the human and organisational dimensions, or the social context of work environments (al-karaghouli et al. 2005; hughes et al. 1994). yet, interpretive research is capable of providing deep insights into human thought and action in social and organisational contexts (walsham 2006). applied to the study of is phenomena, and using social constructs (klein et al. 1999), such an approach would offer a distinct advantage, particularly when the insights can be transformed into the design of it artefacts (hevner et al. 2004). thus, we mailto:ronald.maier@uibk.ac.at mailto:stefan.thalmann@uibk.ac.at file:///c:/users/prasad3/appdata/local/microsoft/thalmann/appdata/local/microsoft/windows/temporary%20internet%20files/content.outlook/kxhxcakj/ajis%20(2nd%20revision)%20rm2.docx%23_enref_57%23_enref_57 file:///c:/users/prasad3/appdata/local/microsoft/thalmann/appdata/local/microsoft/windows/temporary%20internet%20files/content.outlook/kxhxcakj/ajis%20(2nd%20revision)%20rm2.docx%23_enref_48%23_enref_48 australasian journal of information systems volume 17 number 2 2012 138 argue that there is a high potential for applying interpretive research methods to design science activities. this can be undertaken by is academics or by is practitioners, and is particularly suited to research and development projects which aim to develop is in complex social settings. ethnography would appear to be the most appropriate qualitative research methodology for studying knowledge work. however, it is too time-consuming, costly and unfocused for the fast changing is domain. moreover, to ensure the take-up of qualitative research results, these need to be transformed into artefacts for the design and engineering of is requirements. developing concepts and theories in a behavioural science-based research paradigm differs from developing design artefacts in a design science-based paradigm and insights from one site are not easily transferable to the other. this paper explores the suitability of a modified ethnography in a large-scale collaborative setting for is design. the research aimed to answer the question, “how can a modified approach to ethnography be applied in a large-scale collaborative setting to inform is design?” the paper’s distinct contribution is a procedure for the coordination of a collaborative team of ethnographers investigating different social settings, with a common overarching design and research purpose. the aims of this paper are, (1) to present a procedure, based on collaborative ethnography, tailored to the study of knowledge work practices and with a view to enabling requirements gathering and design of is, and (2) to illustrate the procedure with a collaborative ethnographic study performed in a large-scale is research and development project. we discuss a large number of recommendations from the literature. we elaborate on the issue of coordination mechanisms that aim to combine the flexibility of an ethnographic approach with a more structured and focused approach necessary for obtaining the type of results that can be transformed into design artefacts. the following section discusses related work and the theoretical background. we then present the procedure step-by-step, including the experiences gained from its application, reflect on the procedure, its implementation and limitations, before we conclude the paper and provide an outlook to further research. literature review this section positions our approach from a methodological perspective as bridging interpretive, empirical research with design science research. it then addresses knowledge work practices as objects of investigation, ethnography as an approach to the study of knowledge work practices, and finally argues for a new approach, which combines rapid and collaborative applications of ethnography. bridging interpretive and design science research there are complex relationships between developing concepts and theories in a behavioural sciencebased research paradigm and developing design artefacts in a design science-based research paradigm. interpretive research has become more important (walsham 2006) due to its perceived advantages with respect to providing deep insights into human thought and action in social and organisational contexts. this applies especially to the study of is phenomena through social constructions such as language, consciousness, shared meanings, documents, tools, and other artefacts (klein et al. 1999), where meanings are at the centre of the enquiry. the relevance of is research is directly related to its applicability in designing it artefacts (hevner et al. 2004). empirically-based is concepts and theories should be easy to implement, able to synthesise an existing body of research or stimulate critical thinking among is practitioners (benbasat et al. 1999). thus, we argue that there is high potential in applying interpretive research methods in design science activities, by is academics as well as is practitioners, and particularly in cases when research and development projects aim to develop is to be used in complex social settings. australasian journal of information systems volume 17 number 2 2012 139 due to its inherently innovative nature, it is constantly evolving and it artefacts are used in emergent application areas where there have not been any empirically-tested concepts and theories (markus et al. 2002). here, it artefacts are first created and used before theories are developed. the theories focus on the impact of applying these it artefacts to human problem solving and organisational capabilities (hevner et al. 2004). it artefacts comprise not only software and hardware, but also concepts and models that represent the problem domain as-is or as-intended. methods such as simulations, experiments or field studies help in the practice of design. contributions of design science research comprise design artefacts for solving an identified business need, evaluated constructs, models, methods and instantiations (hevner et al. 2004) adding to the knowledge base as well as to the methodology. concretely, by capturing current work practices of people performing knowledge work we have developed design artefacts that contribute, respectively, to the design of it support for knowledge work, and to the knowledge base of the is community. however, in our view, our most important contribution is to methodology, i.e. proposing a procedure for is design employing modified collaborative ethnography. this methodology is intended to support collaborative research and development projects targeting so-called “wicked problems” (rittel et al. 1973). these are characterised by unstable requirements and constraints, complex interactions among subcomponents of problem and solution as well as goals that only emerge during design. research and development projects thus need flexibility in the design processes and artefacts as well as broadening the inherent engineering focus in design science research to a constructivist, interpretive stance. knowledge work practices the concept of knowledge work(er), coined by peter drucker, as well as the concept of penrose rents, developed by edith penrose in her analysis of economic benefits that organisations can get due to their superior idiosyncratic positions have been around for more than 50 years (drucker 1959; penrose 1959). knowledge work in organisations refers to, among other things, creatively solving illstructured problems in complex domains with a high degree of variety and exceptions. it requires the creation, acquisition, application and distribution of knowledge. inputs and outputs of knowledge work consist primarily of data and information and it concerns mainly organisations with a high share of highly skilled employees. in some industry sectors, knowledge work organisations are characterised by a high number of patents, or by the centrality of customer knowledge, of communications and a high degree of information needs (alvesson 2004; hayes 2001; kelloway et al. 2000; schultze 2003; starbuck 1992). the term knowledge work is used in a variety of ways in the literature and can refer to (1) professions, i.e. occupations or job positions that are classified into “knowledge workers” and “non-knowledge workers”, the latter being routine or manual workers, (2) group characteristics, i.e. referring to experts’ work and thus defining a group of individuals who share certain characteristics appertaining to the nature of their work, and (3) activities or behaviour, i.e. knowledge work is not restricted to a certain class or group of employees, but allows a focus on commonalities concerning work practices and places across professions and positions (kelloway et al. 2000). we refer to knowledge work here in the latter, activity-oriented sense of knowledge work practices. people performing knowledge work are often highly competent, committed and selforganised. yet, in order to be productive and effective, they require an organisational environment facilitated by iss enabling them to jointly apply their competencies in a way that, in penrose’s terms, moves organisations as a whole into superior idiosyncratic positions or retains them there. knowledge work is prevalent in high-tech industries and expert-driven organisations such as in professional services, engineering or pharmaceutical sectors (graham et al. 2004). however, we argue that knowledge work is sufficiently similar across occupations and industries to allow designing generic instruments to foster knowledge work. according to our results, people performing knowledge work typically rely mostly on standard is tools, such as office, communication and australasian journal of information systems volume 17 number 2 2012 140 collaboration systems, and the development of is for knowledge work is mostly delegated to external is companies which aim at offering general purpose solutions targeting a diverse user base. aided by the widespread use of advanced collaborative is, knowledge work has changed substantially in recent years. whilst knowledge management has seen an abundance of concepts, models, methods, tools and systems, mostly in the 1990s (alavi et al. 2001; blackler 1995; brown et al. 1998; davenport 1998; nonaka 1991; wiig 1993; zack 1999), many of which have failed to achieve their goals (bishop et al. 2008), studies on how knowledge work is currently performed are scarce, not detailed enough or outdated particularly with respect to is support. there were quite a few studies in the 80s and 90s on how computers impacted on the behaviour of groups (dennis 1996; suchman 1987). since then, there has been a general shortage of workplace studies identified for quite some time in organisation science (barley et al. 2001), with particular needs highlighted in relation to studies on the support of knowledge work through is. one important factor for the scarcity of research in this area lies in the difficulty of studying knowledge due to its metaphysical nature (schultze 2000). blackler (1993), therefore, recommends that knowledge work be studied by focusing on work practices. this strong empirical focus on what people do, rather than on what they know, and, one might add, why and how they do what they do, calls for a much more thorough investigation. ethnography ethnographic research was developed to investigate new cultures and social settings and is well established in anthropology and social sciences as the “art and science of describing a group or culture” (fetterman 1999). the first ethnographers described exotic cultures in remote islands, presented in form of monographs (malinowski 1922). even to this day, ethnography seeks immersion or active participation in social settings to understand why and how things happen (jordan 1996). an ethnographer aims to become a member of the community by working with people in their natural environments, typically for long periods of time (fetterman 1999), whereas a conventional researcher never becomes a member in the same way (emmerson et al. 1995). motivations and intentions of participated actions can be observed by the ethnographer (hammersley et al. 2007). a key success factor is to be open to all kinds of situations rather than restricting one’s focus by excluding situations (fetterman 1999). theories are developed inductively from the data gathered, rather than proposed and tested. ethnographers should be aware that it is difficult to ignore prior knowledge and ethnographers are guided by hypotheses and ideas (atkinson et al. 2007; orlikowski 1993). recently, ethnography has become more popular in disciplines other than anthropology and sociology (ball et al. 2000; harper 2000). this is especially true for design activities which focus on users and their thoughts (sharrock et al. 2004). there have also been a number of studies in the is field (harper 2000; myers et al. 1997; orlikowski 1991; suchman 1995). these studies were aimed at developing a thorough understanding of current work practices as a basis for the design of is support (simonsen et al. 1997). while traditional software engineering methods often fail to consider crucial aspects of the users’ environment, such as the social context of work and organisation, ethnographic research explicitly focuses on the social settings in which the intended system should be used (hughes et al. 1994; myers 1999). however, an ethnographic approach tends to be much more in-depth and time consuming than traditional software engineering methods. rapid and collaborative ethnography traditional ethnographic studies require between six months to 2 years’ fieldwork, ideally studying anything that happens without preoccupations (fetterman 1999). modified versions, called rapid australasian journal of information systems volume 17 number 2 2012 141 ethnography (millen 2000), ‘quick and dirty’ ethnography (hughes et al. 1994) or applied ethnography (ball et al. 2000; maginn 2007) have been proposed as less time-consuming and more appropriate research approaches for informing is design (harper 2000). the reduced period of fieldwork constitutes the biggest difference with traditional ethnographic approaches (hughes et al. 1994). this can be achieved by keeping the focus rigorously narrowed down to the specific study’s goals (millen 2000), including by explicitly considering concepts and theories without excluding the openness for the unforeseen (ball et al. 2000). thus, the ethnographer does not aim at a complete and detailed understanding of the whole social setting, but rather at an understanding of aspects deemed relevant to the study (hughes et al. 1994). here, applying the ‘viewpoint analysis’ technique enables the definition and selection of such viewpoints (viller et al. 2000). furthermore, time may be saved by using a range of different observation techniques and tools (millen 2000), such as photography, audio and video recordings, to enhance the data collection (crabtree 2003; pink 2006); and by working collaboratively and, hence, parallelizing tasks (millen 2000). however, it should be noted that parallelizing is limited as a certain depth of immersion is required. the most distinguishing feature of ethnographic studies is the use of ethnographic observations (warne et al. 2005) in the field. researchers acquire rich data representing the context by immersing themselves in the researched situations. thereby, work procedures can be observed as they take place in different settings, and questions can be asked if clarification is needed (warne et al. 2005). modified ethnographic studies are usually realised in the case of highly situated work practices and a need for specific support (plowman et al. 1995). such studies have been used for a variety of purposes, e.g., to perform complete requirements analyses of engineering processes, (ball et al. 2000; weng et al. 2006), to provide additional information to traditional requirements engineering methods, (bentley et al. 1992; simonsen et al. 1997; viller et al. 2000), to investigate the impact or the quality of existing software, (robinson et al. 2007), or to study real-world phenomena for theory-building, (maginn 2007; millen 2000; tuula et al. 2006). by modifying the characteristics of traditional ethnography, the boundaries between different qualitative research strategies become blurred. ethnography and case study can be seen as two independent research strategies on the same level (creswell 2003; darke et al. 2002; visconti 2010; white et al. 2009). however, there are several similarities between case study and ethnography (klein et al. 1999). the main difference between case studies and ethnographies is the extent to which the researcher immerses into the research setting (myers 1999). immersion in this context is the hallmark of ethnography and constitutes an “attempt to understand the way others construe, conceptualise, and understand events, concepts, and categories, in part because these are assumed to influence individuals’ behaviour” (kaplan et al. 1988). in case studies, the primary source of data is the interview, in which respondents answer from their individual perspective. in contrast, ethnographers collect data primarily through observation and participation in natural social settings over a prolonged period of time (creswell 2003) is essential for ethnography. ethnography is seen as inward-looking, aiming to uncover the tacit knowledge of a culture or a setting, while case study is seen as outwardlooking, aiming to describe the nature of phenomena through a detailed investigation of individual cases and their contexts (cohen et al. 2003). thus, even though some characteristics of ethnography may be modified, such as spending less time in the field, which needs to be acquainted for, e.g., by enhancing data collection or applying a focus on certain aspects to be studied, the primary differentiating characteristic of ethnography needs to be retained, i.e. allowing enough time and openness for the researcher to immerse into the studied social setting. compared with surveys, ethnographic studies are usually performed by a single researcher and limited to the practices, social settings and culture of only one organisation, or in the case of medium to large organisations, confined to the study of a small section of the organisation. this is commonly australasian journal of information systems volume 17 number 2 2012 142 criticised (myers 1999) and limits applicability of the results for design activities that aim at supporting users across departments, organisations or even industry sectors. this calls for a collaborative approach among ethnographers to address this limitation. furthermore, collaboration between ethnographers can increase the quality of a study and of its results by bringing in different perspectives (buford-may et al. 2001). collaborative ethnographic studies deliberately and explicitly emphasise collaboration in all its aspects, from conceptualisation, to fieldwork, analysis and discussion (lassiter 2005). however, the goals, concepts and theories held by the participating ethnographers need to be coordinated in order to get comparable results. collaboration during is design is not new, however, and a variety of target groups have been investigated, including the collaboration between teams of software engineers (whitehead 2007), between stakeholders and designers (luna-reyes et al. 2008), user groups and designers (wu et al. 2003) or between software users and designers (kendall et al. 1986). furthermore, methods of data collection imported from social sciences have been applied in is research, such as interviews (walsham 1995), observations (wu et al. 2003) or folklore technique (kendall et al. 1986). the distinct contribution this paper makes is by proposing a procedure for the coordination of a collaborative team of ethnographers investigating different social settings, with a common overarching design and research aim. procedure this section describes the procedure we propose for modified collaborative ethnography based on related work that helped in its elaboration, step by step. each step is illustrated with a case described as follows, that also allows us to reflect on the lessons learned. case description in our large-scale research and development project mature, funded by the european commission under its framework programme 7 (www.mature-ip.eu), a collaborative ethnographic study was realised by 18 ethnographers who participated in the daily work lives of 31 employees predominantly performing knowledge work. the study comprised seven instances, i.e. seven organisations in four european countries. the sample was designed to include a wide range of organisations representing a mix of contexts and characteristics deemed to influence knowledge work, i.e. size, sector, it intensity and country (see table 1). organisation size (employees) sector it intensity country i large (20,000) telecommunication high switzerland ii large (4,000) health care medium germany iii large (1,000) professional services medium united kingdom iv large (400) it services high germany v large (300) professional services medium united kingdom vi medium (130) it franchising high germany vii small(30) consulting services medium spain table 1: characterisation of the sample the authors took a lead role in the study, by undertaking its design, running a pilot, coordinating study instances and leading the data collection, analysis, reflection and writing-up of results. australasian journal of information systems volume 17 number 2 2012 143 furthermore, the authors ensured the take-up in the project’s software development activities which included requirements analysis, design activities and evaluation. the time frame for the study was limited, partly because some organisations were not willing to accept researchers participating in daily activities for more than two weeks. allowing researchers to participate in and investigate employees’ daily work practices requires a substantial amount of trust. hence, we relied on the voluntary participation of organizations with whom the researchers had some kind of prior relationship, whether through personal contacts within the organisation or jointly performed project work. furthermore, the researcher’s existing knowledge of the organisations made it easier to assess and select the appropriate organisational units to be visited and their suitability for the study. additionally, their knowledge and their existing personal relationships sped up the immersion into the organisational contexts. immersion was further aided by the fact that ethnographers and subjects had similar professional and cultural backgrounds. ethnographers were familiar with the aspects of knowledge work to be studied from their own professional experience as knowledge workers. however, in order to mitigate a potential bias in this selection, we included two organisations with whom we had no such prior relationships. within the organisational settings, we studied knowledgeintensive processes, such as product development or customer relationship management and employees who were primarily engaged in knowledge work, such as it consultants or project managers. phase 1: definition first of all, the study team needs to decide whether the design problem it targets with its research and development project is indeed “wicked” (rittel et al. 1973), i.e. characterised by unstable requirements and constraints as well as complex interactions among subcomponents of problem and solution among others. if this is the case, the procedure will be beneficial, otherwise more traditional procedures and methods for requirements engineering and design would probably be more costeffective. fundamental to our procedure for collaborative ethnography is that several researchers use the same approach to answer a joint research question (lassiter 2005). however, they perform fieldwork individually in one unique social setting with the collective arrangement in mind. each individual study performed by any one ethnographer can be seen as constituting a distinct ethnographic study in its own right, as well as one part of a collaborative study. researchers bring their own personal perspectives to the study, reflecting their different professional backgrounds, and thus describe and highlight different details and aspects. thus, collaborative ethnography needs to provide a process for combining these perspectives, into a broader view on the setting (buford-may et al. 2001), particularly if study instances are performed in different social settings. ethnographic studies cannot be sped up without modifying the corresponding procedure (crabtree 2003). in order to ensure the effective use of limited time, the ethnographer needs to narrow down the study’s focus to objectives relevant to design activities (ball et al. 2000; hughes et al. 1994; millen 2000) and only capture the main characteristics of the setting, i.e. aspects related to concept development and prospective design tasks. focusing parts of the setting requires the coordination of ethnographers. guidelines describing general procedures in terms of do’s and don’ts are useful (hammersley et al. 2007). due to the uniqueness of each social setting and the situational character of ethnography, it is impractical to develop detailed guidelines ex ante. evaluating informant-expressed needs and considering them when developing guidelines can be one way of coordination before performing study instances (lassiter 2005). due to the restricted focus and the shortened time, modified ethnography also benefits from a strong theoretical framework guiding the fieldwork (ball et al. 2000) or it risks leaving out important aspects. guidelines for fieldwork therefore should contain descriptions of the study’s goals and areas of focus that are based on established concepts and theories. australasian journal of information systems volume 17 number 2 2012 144 in collaborative ethnographic studies, it is necessary to agree on ways to share experiences and data (gordon et al. 2006).ethnographers should communicate about and agree on goals and focus areas to ensure common understanding. this turned out as very effective for a distributed group of ethnographers having different professional backgrounds. goals and areas of focus need to be balanced, i.e. on the one hand sufficiently general to allow elaborating on interesting individual aspects in study instances and, on the other hand, sufficiently detailed to guide fieldwork effectively. more heterogeneous professional backgrounds and interests of ethnographers, and more diverse studied social settings, require more generic guidelines. thus, coordination is aided by shared resources (malone et al. 1994). these shared resources can be guidelines and later on design artefacts used as boundary objects (leigh star et al. 1989). an experienced study coordinator (designated to the group) will be able to facilitate a dispersed team, to broker experiences between team members and will have the authority to resolve any issues or disagreements. the archetype of a study coordinator is adopted from medical studies, in which coordinators commonly accompany whole studies, develop forms and guidelines for coordination and serve as the main contact person (davis et al. 2002). becoming familiar with the social setting can take considerable time during which the ethnographer´s capacity for absorption is reduced (millen 2000). in order to minimise this time, ethnographers ideally are already familiar with the domain in which studied individuals act in order to make it easier to understand their activities (ball et al. 2000). studying documented information about the setting ahead of the actual study also sped up the familiarisation process in our study. knowing organizations, workplaces or social settings from previous interactions significantly reduces familiarisation with the study milieu, as social relationships are already established. meetings preceding the study can be useful to become familiar with individuals and their context early-on (ball et al. 2000). whilst prior knowledge of organisations speeds up time needed for immersion, potential bias in the procedure should be balanced. some authors argue that it is generally very difficult or even impossible to perform an unbiased procedure and to ignore common known knowledge, no matter whether previous relationships exist or not (glaser et al. 1979; gläser et al. 2006; walsham 2006). the researcher can hardly avoid to be guided by hypotheses and ideas, which motivate his fieldwork and the selection of the social setting to be studied. instead of dismissing the theoretical background and existing knowledge about the social context, the ethnographer should be aware of them (atkinson et al. 2007; fetterman 2010): the willingness to reflect on and revise initial assumptions and theories is considered crucial in this regard (walsham 2006). joint reflection of ethnographers can here be exploited as one of the advantages of collaborative ethnography, particularly if they investigate related settings, e.g., within one organisation. taking the previous deliberations into account, the following procedure is suggested:  base the definition of goals and focus areas on a strong theoretical framework.  define the role of a study coordinator and assign it to one ethnographer.  analyse artefacts which provide information about the research setting.  meet prospective participants before the start of the study.  define conditions, such as the ethnographers and the social settings.  communicate the study outline to all ethnographers, and ensure that they share the same understanding.  establish acceptance of planned activities by the organisations studied. applying this procedure, our study had two closely related aims. the first was to investigate knowledge workplaces using practices of knowledge work as a guiding theme. the second aim was to gather requirements and support the design of an is supporting chained practices of knowledge work aimed at developing collective knowledge in organisations. australasian journal of information systems volume 17 number 2 2012 145 we negotiated the organizational framework within the project consortium. the first and most crucial point was to define the goals and the focus areas of the study. the authors made a proposal detailing and agreeing the project goals for the study. five overarching focus areas could be developed and refined in a collaborative procedure involving the other project partners. all ideas gathered within this procedure were consolidated and agreed at one face-to-face meeting with all ethnographers. the first focus area was knowledge maturing and the stages in this process. the second was the knowledge routines and knowledge processes (davenport et al. 1996) and their ict support. the third focus was the usage of knowledge in specific situations or knowledge-oriented actions (ellis 1993; knights et al. 1993; schultze 2000). the fourth focus was the motivation of knowledge workers during interaction with knowledge. the last focus was the structure of knowledge in organisations. thereby, formal structures such as folder hierarchies were deemed particularly important. between 2 and 6 more detailed questions, such as: “in which situations do people give up their privacy?” or “are there any policies or rules for structuring knowledge?” were used to describe each focus point in more detail. building on these focus points, the authors proposed a study design. the proposal was discussed once more, slightly adapted and finally approved in another face-to-face meeting with all ethnographers. in the same meeting, organizations which were considered suitable for the study were identified and discussed after each ethnographer team had proposed potential organisations by providing a short presentation on the organisation, its units and processes as well as the intended time schedule. finally, the project consortium agreed on the organisations and processes within those organisations to be studied on the one hand as well as teams of ethnographers on the other hand. phase 2: pilot study the main aim of this phase is to familiarise researchers with phenomenon and context. using a single case as a pilot study is beneficial if the research is exploratory (benbasat et al. 1987). in a collaborative setting, the study coordinator should participate in one study instance, ideally in the pilot study. experiences gathered in the pilot study can be written up as “good practice” (o'dell et al. 1998) for other ethnographers which successfully sped up the fieldwork in our case. when time is of the essence, the additional effort might outweigh the advantage of lessons learned to be already taken up in the study’s further instances, particularly if participating ethnographers already have extensive prior experiences in the study’s domain and social settings. the following procedure is suggested:  perform a pilot study.  transfer lessons learned to ethnographers.  identify “good practices”, refine goals, focus areas and guidelines.  communicate the refined guidelines to ethnographers, and ensure that they agree with them. the pilot study was performed by the authors. due to the success of the pilot study, the procedure of the study was applied to the other study instances. the pilot study took five weeks in total, of which two weeks were used for fieldwork, i.e. ethnographic observations and immersion (see phase 3 fieldwork for details of the procedure). only some minor changes were made to the focus areas and guidelines while the goals stayed the same. thus, the pilot study is fully comparable with the other study instances. our familiarity with the domain and the trusted relationship with the participants proved to be highly beneficial for an effective use of the scarce time and sped up the immersion into the work context. the trusted relationship developed through a previous positive experience of a joint project resulted in strong management support for our study and an unobstructed, open work atmosphere with participants. the pilot study allowed the authors to transfer experiences collected on the procedure early on to the other ethnographers in a workshop-style meeting. the authors reported their experiences of the study australasian journal of information systems volume 17 number 2 2012 146 and elaborated on their experiences in the form of anecdotes and stories. the procedure of the pilot study and experiences from the fieldwork provided the focus for this meeting, not (preliminary) results and reflections on them. the anecdotes and stories were presented in a sequence of distinct phases of the fieldwork. in the workshop, they were jointly reflected as lessons learned, taking on board previous experiences of all participating ethnographers. examples of questions that were targeted here and for which general guidelines have been agreed on were: how to address management, how to conduct the first meeting with the study participants, how to raise awareness for the study’s focus areas, how to conduct the first day of the study, how to handle breaks, when to write field notes, how to determine the appropriate level of detail of field notes, how to split the time spent with the individuals studied and spent on isolated reflection, how to handle the uninterrupted observation of work practices, how to discuss self-reporting with studied individuals ex ante and ex post, how to keep contact during the period between site visits or how to perform the post-hoc reflection. the aim was not to provide a rigid set of rules, but to jointly go through the handling of circumstances that each ethnographer was likely to encounter and to reduce the cognitive load required for decisions that need to be taken in the field. in this way, we substantially reduced the distraction from fieldwork. phase 3: fieldwork reducing time for fieldwork can lower sensitivity to longitudinal aspects about practices of knowledge work. the period of fieldwork can be split in order to give participants more time to critically reflect on aspects of knowledge work. communication channels, such as chat, e-mail or videoconference, can be used to bridge the time in between. this form of communication cannot replace “staying in the field”, but it can help ethnographers to stay in contact with participants and to be informed about ongoing activities (hine 2000). participants can use templates to report on events related to the study, which requires their awareness for the study’s focus areas. ethnographers need to help create awareness of participants during the initial period of fieldwork. the reported events can then be taken up in the second field visit and discussed with the ethnographers returning to the site. technology can be used to collect more data in less time, e.g., photography, audio and video recordings (millen 2000; pink 2006). however, recordings require participants’ consent, which may considerably impact on their behaviour, and recording devices can only record what is observable and in their focus (dufon 2002). data gathered in this way are additional sources which need to be interpreted and linked to the study’s context (pink 2006). ethnographic observations are the defining element of the modified approach to ethnography. the researchers immerse themselves into the social setting, take part in a variety of situations and ask questions if needed (warne et al. 2005). during fieldwork, practitioner-researchers usually reflect on occurrences and activities in the field (schön 1983). this reflection-in-action should be understood as an iterative process in which ethnographers are guided by an ”overarching theory“ (dickover 1994) and construct new theory of the unique case during their practice (schön 1983). the aim should be to learn more about occurrences in the field and to become aware of new aspects or to ask other questions. it would seem useful for the study coordinator to attend the start of every study instance in order to gain a direct impression of the instance’s setting and to coach ethnographers during initiation. the role should not be misunderstood as aiming to control ethnographers, but to get an impression aiding the later synthesis of results from combined study instances. too much involvement could also constrain ethnographers and hamper the study instance’s initiation. the following procedure is suggested:  plan how to involve the study coordinator in instance studies.  perform ethnographic fieldwork and create awareness for the study’s focus. australasian journal of information systems volume 17 number 2 2012 147  support participants in self-reporting and stay in contact with them.  perform ethnographic fieldwork and discuss self-reporting. due to the time constraints given by some of the participant organisations, time for fieldwork within the study was generally limited to two weeks. the time was divided into two periods of one week each, bridged by a period of two to three weeks for self-reporting. this approach allowed ethnographers to investigate medium and long running work practices at different stages and turned out to be very beneficial. due to the immersion into work situations and the rich spectrum of gathered data, our data collection approach goes far beyond interviewing. participating in all aspects of the work, including work process and social life, casual meetings, breaks and also some joint leisure time activities, allowed a very quick and thorough immersion. furthermore, using directed questioning combined with observation and also self-reporting of work activities lead to a deeper understanding and contextually rich data. we created a template for the self-description period based on the focus areas and established concepts we had previously agreed, in our case the concept of knowledge stance (maier 2005). a knowledge stance is a trigger for a knowledge-oriented action in which knowledge is created or reconstructed. it comprises steps which can be supported by it services and are framed by a defined start occasion and end state used to structure and describe knowledge work situations. the main viewpoints were briefly explained and one short example was given. during the first week of fieldwork, participants were instructed in the use of this template. awareness and sensitivity for situations in which participants handle knowledge in a non-routine way were created during the first week of fieldwork as the main focus of self-reporting. raising awareness of participants about the study’s focus areas was beneficial so that they could share their reflection-inaction with ethnographers providing further background information. thus, we could discuss data collected over a five weeks’ period during which participants were engaged in different tasks and project phases. time spent with participants versus isolated reflection was split so that time with participants was maximized while ethnographers had the time to retreat as and when required. this enabled the researchers to spend most of the working day with participants, including lunch and coffee breaks. some teams of ethnographers had a separate room at their disposal in their studied organisation where they could meet for isolated or joint reflection which helped reflection-in-action as close to the action as possible, both physically and time-wise. one of the authors acted as study coordinator and thus attended at least the first day of each study instance in order to accompany participating researchers. the main aim was to provide support for ethnographers, to ensure comparability between instances and to get an overall impression of the settings. the study coordinator accompanied ethnographers without interrupting them and tried to stay in the background. during breaks and at the end of each day, he facilitated a joint reflection on the fieldwork. participating researchers frequently asked for suggestions on how to handle emerging problems and circumstances. background knowledge and personal impressions gained during participation proved very useful for coordinating the collaborative data analysis. phase 4: data analysis field notes are the main artefact for analysing ethnographic data (ball et al. 2000). due to its highly contextualised nature, the data can only be analysed effectively by those researchers who performed the fieldwork (emmerson et al. 1995). hence, it is not advisable to centralise data analysis in a collaborative study. field notes should be analysed locally and they can be coordinated by templates (lewis et al. 1996). this procedure is strongly related to reflection-on-action whereby ethnographers think back on situations in the field, on actions by participants of the study and also on their reflection-in-action (schön 1983). in a collaborative setting, this reflection can be supported by australasian journal of information systems volume 17 number 2 2012 148 formal “after action reviews” in which groups of researchers jointly reflect on occurrences in the field (baird et al. 1999). however, reflection-on-action is not performed arbitrarily, but employs a method for qualitative data analysis. this typically involves coding field notes, i.e. the categorization of field notes into concepts suggested by the field data (agar 1980). in traditional ethnography, codes are ideally developed in a purely inductive way without previous deliberations. however, even in case of only one ethnographer and an extended period of time, it is considered difficult to ignore common knowledge and prior deliberations (glaser et al. 1967). one approach bridging both perspectives is the axial coding in which inductively developed categories are linked to theories in order to select and refine them (strauss et al. 1990). due to its undirected nature, the purely inductive procedure can be considerably time-consuming which seems inefficient in a collaborative setting. hence, initial codes or at least categories, should be developed based on precisely defined concepts and the focus areas agreed between researchers (lewis et al. 1996). openness requested in inductive approaches is ensured by adjusting and extending initial categories flexibly if deemed necessary in a study instance (patton 2002). results from study instances need to be consolidated. this can be realised centrally, collaboratively or in a combination of both. in any case, a coordinating role seems to be effective, and personal experiences about settings of individual study instances gathered by the study coordinator invaluable. there is an important issue here about when ethnographers should let go of their private data and findings. at a certain point in time results need to be transferred to designers who should benefit from the broadened perspective of the collaborative study without having to deal with all the specifics of each study instance. we suggest a gradual approach beginning with individual data analysis, sharing results with other ethnographers early on, negotiating how to document findings, making the context as explicit as necessary for ethnographers to compare their findings and interpreting these collaboratively. this leads to a consolidated set of results rather than to a collection of isolated results. several authors propose principles or criteria for validating qualitative research (horsburgh 2003; klein et al. 1999; lincoln 1995; patton 2002; seale 1999). these criteria cannot be applied in a mechanistic way and need to be adapted to each study individually (klein et al. 1999). however, these principles can be seen as basic rules of research, forming the philosophical base of interpretative research and can hence also be applied to the study design of collaborative ethnography. the following procedure is suggested:  define templates to guide joint analysis of field notes.  develop initial codes or categories.  share results early on and make context explicit.  consolidate individual findings into a collaborative set of findings.  use one set of principles for evaluating interpretative field studies. a workshop for all ethnographers was set up to share experiences from fieldwork as well as to agree on the procedure for data analysis. each team of ethnographers presented their study to get an overview of all study instances. while the other teams of ethnographers performed their fieldwork, the authors already started with the data analysis and developed a proposal for a collaborative procedure. templates were developed that contained general conditions of the intended description of results without restricting specific results. these templates were refined in a multi-step process in order to take on board specific experiences by study instances that might be of interest to all instances and guided data analysis. researchers were requested to identify further relevant aspects that should be considered in all study instances during this multistep negotiation process. furthermore, a timeline coordinating the integration of the results of the individual data analyses was proposed, discussed and approved. this illustrates overlapping phases, an approach we took to further improve timeefficiency. the study coordinator continuously stayed one step ahead concerning the study instance in australasian journal of information systems volume 17 number 2 2012 149 which he participated, preparing the next phase while still engaged in coordinating the previous phase. coding was supported by a structural model of situations covering the six dimensions when, who, why, where, what and how. characteristics of the former four generic dimensions were developed inductively, while characteristics of the latter two specific dimensions were developed with an informed inductive procedure (patton 2002). this resulted in a set of sequences of knowledge routines with specific demands regarding ict support, a list of indicators for assessing their performance and a typology of knowledge work situations and knowledge elements. together with identified barriers and enablers, these results were used to detail our “wicked problem”(rittel et al. 1973), especially in the area of ict support.the principles proposed by (klein et al. 1999) for evaluating interpretative research were used to ensure the quality of our study. the set of principles turned out to be useful for reflecting on the procedures used for data collection and for data analysis. the results were turned into proposals how to develop, structure, describe and visualise implications for design activities in the subsequent phase. guidelines for reflections in local teams as well as the agenda for the entire project meeting were developed. the design principles and the implications for the study on the local team of ethnographers as well as on the entire project group are depicted in table 2. principle according to (klein et al. 1999) measures taken in the individual social setting and the study instance measures taken in the collaborative study as a whole principle of the hermeneutic circle local social phenomena were reflected in regard to the entire organisation consolidated interpretations were reflected regarding the study´s aims principle of contextualisation historical background with the social setting was disclosed and discussed research settings were presented and their suitability and limitations discussed principle of interaction socializing to the views of the participants was discussed short summary on the interaction was provided and jointly reflected upon principle of abstraction and generalisation design artefacts were created, discussed and refined initial coding scheme was based on concepts known from the literature principle of dialogical reasoning the coding scheme was reviewed and revised after the first coding the coding scheme was jointly reviewed after individual coding principle of multiple interpretations individual interpretations were jointly reflected during fieldwork, after coding and after artefact creation consolidated interpretations per study instance were jointly reflected after fieldwork, after coding and after artefact creation principle of suspicion individual interpretations were challenged and participants were called back consolidated interpretations per study instance were challenged by people not engaged in the fieldwork table 2: discussion of evaluation principles australasian journal of information systems volume 17 number 2 2012 150 phase 5: artefact creation the take-up of results in design activities raises issues of how to communicate findings from ethnographic studies to design teams (hughes et al. 1994) and to transform results into parts digestible by designers (crabtree 2003; viller et al. 2000). codes and categories are too abstract and intangible. they need to be translated into design artefacts useful for designers, e.g., requirements, use cases or process models, guided by the purpose of design (lewis et al. 1996). as there are farreaching decisions to be taken, intermediate steps might help to reduce the complexity of moving between levels of abstraction and also to provide boundary objects that ethnographers and designers can both relate to (chang et al. 2008). artefacts known from object-oriented analysis, such as case diagrams or sequence diagrams are suitable if existing systems are in the focus of research (viller et al. 2000). however, richly described artefacts, such as personas, i.e. fictitious, concrete representations of target users which describe what they want to accomplish in regard to a design task (pruitt et al. 2006), are beneficial in case of designing a tool for work practices that so far have not been supported. modified versions of ethnography are goal-oriented. fieldwork and data analyses are directed towards creating such artefacts, hence artefacts can be expected to inform is design. ethnographers act as communicative agents between users and designers (crabtree 2003). study results can be used for scoping and prioritising design activities, for designing is that target critical social settings and for evaluation (lewis et al. 1996). as a process iterating with data analysis, artefacts should also be created collaboratively (lewis et al. 1996). furthermore, the study coordinator should guide the process to ensure the comparability of artefacts that can be understood by designers. the appropriateness of design artefacts must be rigorously shown via well-executed evaluation methods (hevner et al. 2004). the evaluation of the created artefacts provides new insights which can be used to improve the quality of the product as well as the design process, i.e. the study design (hevner et al. 2004). the second point demands a reflection of the data analysis and its results in particular. the fit of the is created on the basis of the design artefacts to the tasks of the employees can be evaluated, e.g., by assessing the task technology fit (goodhue et al. 1995), the effectiveness of the is in general (hamilton et al. 1981) or the is success (delone et al. 2003). ultimately, the is needs to be adopted, i.e. an organization decides to use the is (rogers 1995), it needs to be accepted, i.e. a user decides voluntarily to use the is (dillon et al. 1996) and assimilated, i.e. the is-based solution diffuses into organizational work processes and corresponding daily activities (chatterjee et al. 2002). the following procedure is suggested:  create artefacts suitable for designers.  ensure findings are used to inform design and to evaluate design artefacts.  apply suitable evaluation methods to check the appropriateness of artefacts. results and insights gained from the study were shared in numerous meetings and teleconferences with designers to jointly refine them into design artefacts. the procedure was initiated by a modelling workshop. the authors prepared design artefacts such as personas or process models in advance and presented them to the consortium. the suitability and ideas for revision were discussed and a time schedule for the artefact creation was agreed. participating researchers were requested to search for additional design artefacts and encouraged to create proposed artefacts out of their field data. this initiation of a process for creating different kinds of design artefacts was very fruitful and resulted in a large number of design artefacts. these reflect the agreed consolidated results of the data analysis as well as the individual experiences of ethnographers. hence, artefacts needed to be consolidated. due to the fact that the artefacts should be used for the design activities, a joint meeting involving designers and ethnographers seemed useful. design artefacts of the same type were categorized and grouped as part of preparing for this workshop. this initial grouping provided a australasian journal of information systems volume 17 number 2 2012 151 beneficial starting point for the break-out group discussions in the meeting. the goal of each breakout group was to select and consolidate the most suitable design artefacts out of a given set. thus, the design artefacts could be reduced to a manageable number. furthermore, a hand-over of design artefacts from ethnographers to designers took place. the necessary knowledge transfer was aided by the fact that both groups collaboratively discussed the design artefacts and by the ethnographers and designers constituting overlapping groups. finally, chains of knowledge practices were described with the help of process models and enriched by further model types and elements, i.e. goals, indicators, personas, knowledge elements, services and barriers. personas formed important communication media in the project and helped consolidate findings and guide decisions about iss. templates had been used during data analysis, such as a table of characteristics of personas or criteria for identifying processes to be supported by the intended system. evaluation scenarios were developed from the gathered requirements and used in design, prototyping and later on in testing. software demonstrators targeting the support of the identified knowledge processes were created and evaluated. the evaluation of these demonstrators was performed by researchers who had already participated in the fieldwork, and the results of the evaluation were used to refine the design artefacts and to justify the applicability of concepts built into the data analysis, e.g., instantiations of indicators for assessing services supporting knowledge work. a second stream of evaluation was performed in regard to the theoretical results of the study. several concepts and models that had been created or refined in this collaborative ethnographic study were empirically tested with the help of semi-structured interviews with a much larger sample of organisations in a follow-up study. examples are the importance, state-of-support and success of performing the identified knowledge practices, the importance of barriers and ways how to overcome them, supporting it services and the ways in which organisations have successfully appropriated technology in support of knowledge practices. discussion and limitations figure 1 summarises the five phases of our procedure which are similar to the common research procedure for qualitative research (patton 2002) and also describes major inputs, outputs and activities of the study coordinator in each phase. in the following, we discuss the specifics of the proposed procedure. phases are performed in sequence but overlapping with each other. the output of the preceding phase is used as input for the next phase. phases 1 and 2 as well as phases 4 and 5 are performed iteratively so that the results of the pilot study are used to reflect the definition of the study design, and the artefact creation influences the data analysis. the latter materializes an important specific characteristic of this procedure, namely that the study’s goals are explicitly targeted at the design of an (it) solution for the problems identified in the study in order to improve effectiveness or efficiency of knowledge practices. australasian journal of information systems volume 17 number 2 2012 152 phase 1: definition design task research question theoretical framework study´s goals, conditions and guidelines agreements with ethnographers and participating organisations phase 2: pilot study lessons learned revised conditions and guidelines phase 3: fieldwork field notes and self descriptions experiences from reflection-in-action phase 4: data analysis experiences from reflection-on-action categorisation linked to theories phase 5: artefact creation artefacts for is design and evaluation concepts for theory building study coordinator moderate perform support guide consolidate figure 1: phases of the procedure our approach targets “wicked problems” and supports their analysis and solution with the help of design artefacts (rittel et al. 1973). specifically, we use ethnography for investigating ill-defined environmental contexts in order to clearly identify requirements and constraints. our approach explicitly considers multiple views of participating researchers and their goal-driven consolidation. the conceptual artefacts, such as personas and knowledge processes, created with concepts, theories and subsequent design in mind help to reduce the cognitive load on the designers. collaboration is strongly encouraged to understand the problem domain in its social environment and renders design a collaborative activity. together with involving designers in collaborative ethnography, this enhances their social abilities so that resulting solutions are potentially more effective across the investigated social settings. we identified a critical issue concerning the timing for when the data analysis should be handed over from individual ethnographers to the team of ethnographers. it was difficult for ethnographers to let go of the privilege and burden to be the only ones to understand and interpret the specific context of a setting. coordinating this procedure is necessary, so that the creation of design artefacts benefits from the broader context of several study instances interpreted simultaneously. overlapping phases were highly effective with the authors always being the first to perform a step of the study, collect results, present them to ethnographers, agree on the procedure, send out templates, collect filled templates, consolidate them, agree on results and then convey them to ethnographers including designers. concerning validation of results, strong agreement among ethnographers and strong take-up of artefacts can be seen as internal validation of study results as well as first external validation by designers as customers. evidence for a strong take-up of artefacts is given by their reuse in richly describing use cases with the help of personas and knowledge practices, reuse and further refinement of the models in the project’s model-driven software architecture, and finally, by taking up concepts such as knowledge indicators in the project’s evaluation activities. the study follows a different layout compared to traditional ethnography. the main difference is that time spent for fieldwork was limited to at least four weeks per study instance, with two weeks of ethnographic fieldwork on site. while a longer time frame would have been desirable, this was not feasible due to limited resources on the part of participant organisations which had to agree to open up to ethnographers. most study instances built on established relationships so that time required for immersion could be shortened substantially. however, this could also have caused another limitation, namely that ethnographers become socialised to the views of participants and lose the benefit of gaining a fresh australasian journal of information systems volume 17 number 2 2012 153 perspective of the situation (walsham 2006). in order to account for this, two organisations were chosen without prior relationships to ethnographers, however, their results were in line with other study instances. our efforts in the study design, guidelines and coordination were important to overcome this potential bias. this also helped to compensate for the fact that individual researchers generally have their own objectives (hughes et al. 1994). the coordination of a large and diverse team of ethnographers is a major challenge, which we were able to handle with the help of extensive communication, boundary objects and the institutionalised role of a study coordinator. we also realized that flexibility in applying guidelines and templates is critical. guidelines and templates should be sufficiently specific to properly guide ethnographers and should allow individual freedom and openness for unknown phenomena. this flexibility demands a multi-perspective procedure in which ethnographers reflect on aspects added by others. flexibility and number of joint reflection meetings demand time and resources, enhance sensitivity for new aspects and have to be defined case by case. summing up, our proposed procedure based on collaborative ethnography offers four advantages over other qualitative methods in general and traditional ethnography in particular, as follows: 1. the context is broadened to include several study instances which are conducted and analysed in a coordinated manner. the variety of multiple social settings as well as the backgrounds and perspectives of many ethnographers could be incorporated (18 in our case). however, this means ethnographers need to collaborate which requires coordination. our procedure gives a number of recommendations on how to address this non-trivial challenge. compared to traditional ethnography, advantages of our approach are the broadening of focus and the inclusion of multiple views. 2. the procedure’s advantageous time constraints allow investigating organisations not willing to open up for extended periods of time. compared with interviews or observations, ethnographers can still immerse themselves much more deeply into the social settings and hence are able to gain a greater insight into motivations and intentions. however, the following core of ethnography must be ensured to apply the procedure successfully. immersion into the field needs to be ensured. taking our experiences into account, the time spent on site should be divided into at least two one week periods working together with one group of people. factors reducing the time needed for immersion include previous trusted relationships with subjects, familiarity with the domain and the aspects to be studied as well as similar professional and cultural background as the subjects. researchers need to participate in the subjects’ natural environments to get a real understanding of the happenings. working together with people in constructed situations or in predefined tasks contradicts the nature of ethnography and limits the ability to understand the happenings. furthermore, taking part in social interactions needs to be achieved to emphasize the added value compared to observations. 3. the procedure addresses “wicked” problems and informs requirements engineering and is design. our procedure enables more time-efficient investigations compared to traditional ethnographies without losing the openness to unforeseen phenomena. hence, it can be integrated more easily into research and development projects. however, compared with traditional requirements elicitation approaches, e.g. brainstorming, document analysis or interviews (hossenlopp et al. 2008), more time is needed for the investigation. the larger variety of investigated social settings allows for a comparison across study instances which leads to a heightened level of justification for following design activities. compared to interviews or observations, the greater variety and richness of study results enhances opportunities for an indepth description of “wicked” problems that can then be addressed by design activities. the increased complexity is influenced by the clarity of the problem, the level of change and the australasian journal of information systems volume 17 number 2 2012 154 number of stakeholders involved (hossenlopp et al. 2008). considering that our approach is able to target such highly complex settings, the greater investment in time and effort seems justified. for design tasks that do not address “wicked“ problems or focus on incremental improvement, such as the minor further development of existing systems, less time-consuming methods seem more appropriate. 4. our procedure motivates for involving designers in the study. the early and direct involvement of designers and implementers fosters the take up of study results for the ensuing design activities. it seems beneficial to bring both groups together as early as possible as it helps to align the study’s goals with the needs of design. knowing the designers’ and the developers’ ideas helps to define the focus areas and can be taken into account by ethnographers during fieldwork. collaboration between designers and ethnographers also makes it easier to transform study results into design artefacts. this is considered an advantage as study results are usually only handed over from researchers to designers who then create design artefacts independently. reflecting study results in meetings with ethnographers and designers turned out very beneficial to address the evaluation criteria, e.g. the principle of suspicion (klein et al. 1999) and transfer first-hand experiences of social settings to designers which can be used as forms of reality check for their ideas and to which they can always turn back. conclusion traditional ethnography is of limited help in is design for little researched, complex, dynamic and innovative knowledge work. this paper discussed modified collaborative ethnography for investigating socio-technical organisational settings aimed at designing is that are useful in these settings. collaboration broadens the focus, integrates different perspectives and multiplies the time spent on ethnographic fieldwork for investigating the “wicked” problem in case of limited accessibility of organisations. however, it requires coordination mechanisms that balance openness to relevant facets required by ethnography with the focus, structure and results required by is design. the proposed procedure for modified collaborative ethnography is thus able to provide a much richer picture compared to traditional requirements engineering methods. this can be achieved by considering the social context, problems and solutions that people performing knowledge work can neither easily capture in interviews, nor can they be straightforwardly recognised in observations. this modified approach is faster than traditional ethnographic studies, but requires collaboration, focus and coordination and thus, more time than traditional requirements engineering methods. from a business perspective, the approach should therefore only be applied for innovative, challenging and complex design tasks that are worth the additional effort. coordination was aided by a variety of boundary objects and the institutionalised role of a study coordinator. boundary objects are essential for building new concepts and theories on studied phenomena and for communicating results to designers. the study coordinator drives the procedure, moderates, supports, guides and applies instruments for consolidation and yet must not interfere with the creative processes taking place when ethnographers collect and interpret their rich data. the paper also reports experiences from a large-scale is research and development project for which the procedure was operationalised and critically discusses its application and limitations. our approach, we argue, is especially suited for more innovative, large and diverse study settings, projects, designers, intended is, and target groups. artefacts such as personas and processes can reduce the effort for end-users and help keep a consequent focus on how users perform practices of knowledge work. although our approach requires much less time compared to traditional ethnography, the effort of carefully preparing and coordinating a collaborative study should not be underestimated and needs still more time than traditional requirements elicitation methods. australasian journal of information systems volume 17 number 2 2012 155 avenues for future work are to integrate more closely our procedure with approaches to user-centred design. in extension of users bringing on board their explicit knowledge limited to what they are aware of and deem important for the design problem at hand, ethnography taps into the potential of understanding what users actually do, how they do it and why they do it in a much more profound way. another area worth further research is the analysis of factors which determine the right balance between the added effort for modified collaborative ethnography in is design and the superiority of the resulting it-supported knowledge work practices. finally and from an academic perspective, it seems promising to continue the refinement of research methods that help build a bridge between (behavioural) computer science and qualitative-interpretive as well as design science-oriented is research communities. acknowledgements this work was co-funded by the european commission under the information and communication technologies theme of the 7 th framework programme, integrating project mature (contract no. 216356, http://mature-ip.eu). we thank the reviewers for the insightful and constructive comments that helped improve the paper substantially. references agar, m. h. the professional stranger: an informal introduction to ethnography, academic press, new york, ny, 1980. al-karaghouli, w., alshawi, s., and fitzgerald, g. "promoting requirement identification quality enhancing the human interaction dimension," journal of enterprise information management (18:2) 2005, pp 256-267. alavi, m., and leidner, d. e. "review: knowledge management and knowledge management systems: conceptual foundations and research issues," management information systems quarterly (25:1) 2001, pp 107-136. alvesson, m. knowledge work and knowledge-intensive firms, oxford, 2004. atkinson, p., and hammersley, m. ethnography principles in practice, (3rd ed.) routledge, new york, 2007. baird , l., holland, p., and deacon, s. "learning from action: imbedding more learning into the performance fast enough to make a difference," organizational dynamics (27:4) 1999, pp 19-32. ball, l. j., and ormerod, t. c. "applying ethnography in the analysis and support of expertise in engineering design," design studies (21:4) 2000, pp 403-421. barley, s. r., and kunda, g. "bringing work back in," organization science (12:1) 2001, pp 76-95. benbasat, i., goldstein, d. k., and mead, m. "the case research strategy in studies of information systems," management information systems quarterly (11:1) 1987, pp 369-386. benbasat, i., and zmud, r. w. "empirical research in information systems: the practice of relevance," management information systems quarterly (23:1) 1999, pp 3-16. australasian journal of information systems volume 17 number 2 2012 156 bentley, r., hughes, j. a., randall, d., rodden, t., sawyer, p., shapiro, d., and sommerville, i. "ethnographically-informed systems design for air traffic control," acm conference on computer-supported cooperative work, acm, toronto, canada 1992, pp. 123-129. bishop, j., bouchlaghem, d., glass, j., and matsumoto, i. "ensuring the effectiveness of a knowledge management initiative," journal of knowledge management (12:4) 2008, pp 1629. blackler, f. "knowledge, knowledge work and organizations: an overview and interpretation," organization studies (16:6) 1995, pp 1021-1046. blackler, f., reed, m., and whitaker, a. "epilogue an agenda for research," journal of management studies (30:6) 1993, pp 1017-1020. brown, j. s., and duguid, p. "organizing knowledge," california management review (40:3) 1998, pp 90-111. buford-may, r. a. b., and pattillo-mccoy, m. "do you see what i see? examining a collaborative ethnography," qualitative inquiry (6:1) 2001, pp 65-87. chang, y., lim, y., and stolterman, e. "personas: from theory to practices," proceedings of the 5 th nordic conference on human-computer interaction: building bridges, acm, lund, sweden, 2008, pp. 439-442. chatterjee, d., grewal, r., and sambamurthy, v. "shaping up for e-commerce: institutional enablers of the organizational assimilation of web technologies," management information systems quarterly (26:2) 2002, pp 65-89. cohen, a., and court, d. "ethnography and case study: a comparative analysis," academic exchange quarterly 2003. crabtree, a. designing collaborative systems a practical guide to ethnography, springer, london, 2003. creswell, j. w. research design: qualitative, quantitative, and mixed method approaches, (2nd ed.) sage, thousand oaks, 2003. darke, p., shanks, g., and broadbent, m. "successfully completing case study research: combining rigour, relevance and pragmatism," information systems journal (8:4) 2002, pp 273-289. davenport, t. h. "putting the enterprise into the enterprise system," harvard business review (76:4) 1998, pp 121-131. davenport, t. h., jarvenpaa, s. l., and beers, m. c. "improving knowledge work processes," sloan management review (37:4) 1996, pp 53-65. davis, a. m., hull, s. c., grady, c., wilfond, b. s., and henderson, g. e. "the invisible hand in clinical research: the study coordinator’s critical role in human subjects protection," journal of law, medicine & ethics (30:3) 2002, pp 411–419. delone, w. h., and mclean, e. r. "the delone and mclean model of information systems success: a ten-year update," journal of management information systems (19:4) 2003, pp 9-30. dennis, a. r. "information exchange and use in group decision making: you can lead a group to information, but you can’t make it think.," management information systems quarterly (20:4) 1996, pp 433-455. australasian journal of information systems volume 17 number 2 2012 157 dickover, n. "reflection-in-action: modeling a specific organization through the viable systems model," systems practice (7:1) 1994, pp 43-62. dillon, a., and morris, m. "user acceptance of information technology: theories and models," annual review of information science and technology (31) 1996. drucker, p. f. landmarks of tomorrow, harper, new york, 1959. drucker, p. f. "the age of social transformation," the atlantic monthly (274:5) 1994, pp 53-80. dufon, m. a. "video recording in ethnographic sla research: some issues of validity in data collection," language learning & technology (6:1) 2002, pp 40-59. ellis, d. "modeling the information-seeking patterns of academic researchers a grounded theory approach," library quarterly (63:4) 1993, pp 469-486. emmerson, r. m., fretz, r. i., and shaw, l. l. writing ethnographic field notes, university of chicago press, chicago, 1995. fetterman, d. m. ethnography step by step, (2nd ed.) sage, london, 1999. fetterman, d. m. ethnography step-by-step, (3rd ed.) sage, thousand oaks, 2010. glaser, b., and strauss, a. the discovery of grounded theory: strategies for qualitative research, aldine, chicago, 1967. glaser, b. g., and strauss, a. l. "die entdeckung gegenstandsbezogener theorie: eine grundstrategie qualitativer sozialforschung," hopf, c. and weingarten, e.h. (eds.), qualitative sozialforschung, klett, stuttgart, 1979, pp. 91-111. gläser, j., and laudel, g. experteninterviews und qualitative inhaltsanalyse, (2nd ed.) verlag für sozialwissenschaften wiesbaden, 2006. goodhue, d. l., and thompson, r. l. "task-technology fit and individual performance.," management information systems quarterly (19:2), pp 213-236. gordon, t., hynninen, p., lahelma, e., metso, t., palmu, t., and tolonen, t. "collective ethnography, joint experiences and individual pathways," nordisk pedagogik (26:1) 2006, pp 3–15. graham, p., bowerman, c., and bokma, a. "adaptive navigation for mobile devices," . attewell, j. and savill-smith, c. (eds.), learning with mobile devices research and development, learning and skills development agency, london, 2004, pp. 61-68. hamilton, s., and chervany, n. l. "evaluating information system effectiveness part ii: comparing evaluator viewpoints," management information systems quarterly (5:4) 1981, pp 79-86. hammersley, m., and atkinson, p. ethnography principles in practice, (3rd ed.) routledge, 2007. harper, r. h. r. "the organisation in ethnography: a discussion of ethnographic fieldwork programs in cscw," computer supported cooperative work (9:2) 2000, pp 239–264. hayes, n. "boundless and bounded interactions in the knowledge work process: the role of groupware technologies," information and organization (11:2) 2001, pp 79-101. hevner, a. r., march, s. t., park, j., and ram, s. "design science in information systems research," management information systems quarterly (28:1) 2004, pp 75-105. hine, c. virtual ethnography, sage, london, 2000. australasian journal of information systems volume 17 number 2 2012 158 horsburgh, d. "evaluation of qualitative research," journal of clinical nursing (12:2) 2003, pp 307312. hossenlopp, r., and hass, k. b. unearthing business requirements: elicitation tools and techniques, management concepts, vienna, va, usa, 2008. hughes, j. a., king, v., rodden, t., and andersen, h. "moving out of the control room: ethnography in systems design.," acm conference on computer supported cooperative work, acm, chapel hill, usa 1994, pp. 429 439. jordan, b. "ethnographic workplace studies and cscw," shapiro, d., tauber, m., and traunmüller, r. (eds.) the design of computer supported cooperative work and groupware systems, elsevier, amsterdam, netherlands, 1996, pp. 17-42. kaplan, b., and duchon, d. "combining qualitative and quantitative methods in information systems research: a case study," management information systems quarterly (12:4) 1988, pp 571-586. kelloway, e. k., and barling, j. "knowledge work as organizational behavior," international journal of management reviews (2:3) 2000, pp 287-304. kendall, k. e., and losee, r. d. "information system folklore: a new technique for system documentation," information & management (10:2) 1986, pp 103-111. klein, h., and myers, m. d. "a set of principles for conducting and evaluating interpretive field studies in information systems," management information systems quarterly (23:1) 1999, pp 67-94. knights, d., murray, f., and willmott, h. "networking as knowledge work: a study of strategic interorganizational development in the financial services industry," journal of management studies (30:6) 1993, pp 975-995. lassiter, l. e. "collaborative ethnography and public anthropology," current anthropology (46:1) 2005, pp 83-105. leigh star, s., and griesemer, j. r. "'translations' and boundary objects: amateurs and professionals in berkeley's museum of vertebrate zoology," social studies of science (19:3) 1989, pp 387-420. lewis, s., mateas, m., palmiter, s., and lynch, g. "ethnographic data for product development: a collaborative process," interactions (3:6) 1996, pp 52-69. lincoln, y. s. "emerging criteria for quality in qualitative and interpretive research," qualitative inquiry (1:3) 1995, pp 275-289. luna-reyes, l. f., black, l. j., cresswell, a. m., and pardo, t. a. "knowledge sharing and trust in collaborative requirements analysis," system dynamics review (24:3) 2008, pp 265-297. maginn, p. j. "towards more effective community participation in urban regeneration: the potential of collaborative planning and applied ethnography," qualitative research (7:1) 2007, pp 25-43. maier, r. "modeling knowledge work for the design of knowledge infrastructures," journal of universal computer science (11:4) 2005, pp 429-451. malinowski, b. argonauts of the western pacific, dutton & co. inc., new york, 1922. australasian journal of information systems volume 17 number 2 2012 159 malone, t. w., and crowston, k. "the interdisciplinary study of coordination," acm computing surveys (26:1) 1994, pp 87-119. markus, m. l., majchrzak, a., and gasser, l. "a design theory for systems that support emergent knowledge processes," management information systems quarterly (26:3) 2002, pp 179-212. millen, d. r. "rapid ethnography: time deepening strategies for hci field research," conference on designing interactive systems, acm press, new york, usa, 2000, pp. 280-286. myers, m. "investigating information systems with ethnographic research," communications of the association for information systems (2:4) 1999, pp 1-22. myers, m. d., and young, l. w. "hidden agendas, power, and managerial assumptions in information systems development: an ethnographic study," information technology & people (10:3) 1997, pp 224-240. nonaka, i. "the knowledge-creating company," harvard business review (69:6) 1991, pp 96-104. o'dell, c., and grayson, c. j. "if we only knew what we know: identification and transfer of internal best practices," california management review (40:3) 1998, pp 154-174. orlikowski, w. j. "integrated information environment or matrix of control? the contradictory implications of information technology," accounting, management and information technologies (1:1) 1991, pp 9-42. orlikowski, w. j. "case tools as organizational change: investigating incremental and radical changes in systems development," management information systems quarterly (17:3) 1993, pp 309-340. patton, m. q. qualitative research & evaluation methods, (3rd ed.) sage, thousand oaks, 2002. penrose, e. the theory of the growth of the firm, john wiley and sons, new york, 1959. pink, s. doing visual ethnography: images, media and representation in research, (2nd ed.) sage, london, 2006. plowman, l., rogers, y., and ramage, m. "what are workplace studies for?," 4th european conference on computer-supported cooperative work, stockholm, sweden, 1995, pp. 309 324. pruitt, j., and adlin, t. the persona lifecycle, keeping people in mind throughout product design, kaufmann publisher, san francisco, 2006. rittel, h. w. j., and webber, m. m. "dilemmas in a general theory of planning," policy sciences (4:2) 1973, pp 155-169. robinson, h., segal, j., and sharp, h. "ethnographically-informed empirical studies of software practice," information and software technology (49:6) 2007, pp 540-551. rogers, e. diffusion of innovations, free press, new york, 1995. schön, d. a. the reflective practitioner, basic books, 1983. schultze, u. "a confessional account of an ethnography about knowledge work," management information systems quarterly (24:1) 2000, pp 3-41. schultze, u. "on knowledge work," holsapple, c.w. (ed.), handbook on knowledge management, springer, berlin, 2003, pp. 43-58. seale, c. "quality in qualitative research," qualitative inquiry (5:4) 1999, pp 465-478. australasian journal of information systems volume 17 number 2 2012 160 sharrock, w., and randall, d. "ethnography, ethnomethodology and the problem of generalisation in design," european journal of information systems (13:3) 2004, pp 186-194. simonsen, j., and kensing, f. "using ethnography in contextual design," communications of the association for information systems (40:7) 1997, pp 82-88. starbuck, w. h. "learning by knowledge-intensive firms," journal of management studies (29:6) 1992, pp 713-740. strauss, a., and corbin, j. basics of qualitative research: grounded theory, procedures, and techniques, sage, newbury park, ca, usa, 1990. suchman, l. plans and situated actions: the problem of human-machine communication, cambridge university press, new york, 1987. suchman, l. "making work visible," communications of the acm (38:9) 1995, pp 56-64. tuula, g., hynninen, p., lahelma, e., metso, t., palmu, t., and tolonen, t. "collective ethnography, joint experiences and individual pathways," nordisk pedagogik (26:1) 2006, pp 3-15. viller, s., and sommerville, i. "ethnographically informed analysis for software engineers," international journal of human-computer studies (53:1) 2000, pp 169-196. visconti, l. m. "ethnographic case study (ecs): abductive modeling of ethnography and improving the relevance in business marketing research," industrial marketing management (39:1) 2010, pp 25-39. walsham, g. "interpretive case studies in is research: nature and method," european journal of information systems (4:2) 1995, pp 74-81. walsham, g. "doing interpretive research," european journal of information systems (15:3) 2006, pp 320-330. warne, l., pascoe, c., and ali, i. "painting with and without numbers: the use of qualitative and quantitative methods to study social learning," australasian journal of information systems (12:2) 2005, pp 112-120. weng, c., mcdonald, d. w., sparks, d., mccoy, j., and gennari, j. h. "participatory design of a collaborative clinical trial protocol writing system," international journal of medical informatics (76:1) 2006, pp s245-s251. white, j., drew, s., and hay, t. "ethnography versus case study," qualitative research journal (9:1) 2009, pp 18-27. whitehead, j. "collaboration in software engineering: a roadmap " briand, l. c., wolf, a. l. (eds.) future of software engineering 2007, minneapolis, minnesota, pp 214-225. wiig, k. m. knowledge management foundations. thinking about thinking. how people and organizations create, represent, and use knowledge, schema press, arlington (tx, usa), 1993. wolff, e. n. "the growth of information workers," communications of the acm (48:10) 2005, pp 37-42. wu, j., graham, t. c. n., and smith, p. w. "a study of collaboration in software design," isese '03 international symposium on empirical software engineering ieee, rome, italy, 2003. zack, m. h. "managing codified knowledge.," sloan management review (40:4) 1999, pp 45-58. paper title (paper title style) australasian journal of information systems luck 2019, vol 23, research on applied ethics (cybersecurity) entrapment behind the firewall 1 entrapment behind the firewall: the ethics of internal cyber-stings morgan luck charles sturt university moluck@csu.edu.au abstract internal cyber-attacks (cyber-attacks which occur from within an organization) pose a serious threat to an organization’s security. one tool that organizations can employ to help them detect such threats is the internal cyber-sting. an internal cyber-sting involves an organization enticing its members into performing a (controlled) internal cyber-attack in order to apprehend them. however, there is (rightly) considerable moral consternation about employing such a tool; for it is deceitful and undermines trust. the aim of this paper is to present four separate actions that might be taken by organizations to strengthen their moral reason for employing internal cyber-stings. keywords: entrapment; cyber-security; stings; internal threats; ethics 1 introduction internal cyber-attacks (cyber-attacks against an organization from within the organization) pose a serious threat to an organization’s security. ibm’s cyber security intelligence index (2016) claims that 60% of all cyber-attacks are internal. verizon’s data breach investigations report (2016) claims that 77% of cyber-attacks that result in a breach involve an insider (p.36). these attacks are not only frequent, but also very damaging; with the estimated cost to organizations for each successful internal attack being between $100,000 and $500,000 (usd) (ca tech., p.15, 2018). in response, organizations are justifiably “shifting their focus on detection of insider threats” (p.4). one tool that organizations can employ to help them detect these internal threats is an internal cyber-sting. an internal cyber-sting involves an organization enticing its members into performing a (controlled) internal cyber-attack in order to apprehend them. however, there is (rightly) considerable moral consternation about employing such a tool; for (amongst other concerns) it is deceitful and undermines trust. the aim of this paper is to present four separate actions that might be taken by organizations to strengthen (but perhaps not justify) their moral reason for employing internal cyber-stings. to this end, this paper is broken into several sections. in the next section, section 2, i describe internal cyber-stings. in section 3, i relate these stings to entrapment (arguing that they are instances of neutral entrapment). in section 4, i explain what is meant by strengthening an organization’s moral reason for the use of stings. in the subsequent four sections, 5 to 8, i will introduce each of the four actions in turn, and the principles which motivate them. we proceed now to a description of internal cyber-stings. 2 what are internal cyber-stings? in order to introduce internal cyber-stings, consider the following paradigm case: a banking security manager decides to employ a hacker to help them expose weaknesses in their online banking system. however, the manager has a concern. the australasian journal of information systems luck 2019, vol 23, research on applied ethics (cybersecurity) entrapment behind the firewall 2 hacker’s work requires that she has access to important internal systems. the manager worries that the hacker, given her abilities, could exploit this access and secretly transfer large sums of money to untraceable accounts. in order to help protect the bank’s interests the manager creates a financial account in a vulnerable location that the hacker will surely notice. however, unbeknownst to the hacker, if she attempts to improperly transfer funds out of this account, security will be instantly informed and they will be detained. the hacker then proceeds to transfer the funds out of the account. this hacking case has at least three salient features. first, it involves a type of sting – that is, an agent (in this case the manager) is enacting a plan that uses deceit to ensnare a target (the hacker). second, the sting is internal – that is, the target and the agent are both part of the same organization (the bank), and the sting is performed in its interests. third, the sting largely unfolds in a cyber-environment (the bank’s it systems). these are the three main features of internal cyber-stings. these types of stings are strongly related to entrapment. consequently, we can build upon the reasons given for the moral impermissibility of entrapment to help us judge the moral status of internal cyber-stings. to this end, we shall proceed to relate this type of sting to entrapment. 3 how do internal cyber-stings relate to entrapment? there are important similarities between internal cyber-stings and entrapment which, if made clear, should help us to consider the moral permissibility of these stings. in this section, i introduce some of these similarities (and potential differences). entrapment is most commonly understood as a legal defence to criminal charges. it is used when, roughly speaking, an officer of the law (the agent) performs a sting upon a target, which induces them to perform some (otherwise criminal) wrongdoing that they would not have been likely to perform if not for the sting. if this defence is successful (that is, if the sting is determined to be an instance of entrapment), the target is found to be not guilty of the criminal charge. i shall refer to this understanding of entrapment as legal entrapment. legal entrapment is understood negatively. that is, it is something which law officers should not do. not simply because, if proven, no conviction can be forthcoming; but because it is morally wrong. reasons for this vary. one reason is that not only do such stings fail to prevent crimes (as it is unlikely the target would have committed this type of crime had they not been entrapped). a second reason is that they cause people to act wrongly, and as result makes the world worse. a third reason is that legal entrapment “violates the autonomy of those subject to it, it undermines an essential condition of moral agency and criminal liability” (hughes, 2007, p.45). a forth reason is that “such activity will only lead to public disillusionment with our police and criminal justice system and a decrease in respect for the law" (carlon, 2007, p.1101). a fifth reason is “because it subverts the moral capacities of entrapped persons” (howard, 2016, p.25). these reasons are far from exhaustive. however, not all conceptions of entrapment are negative. hill, mcleod and tanyi (2017) promote a neutral, and broader, understanding of entrapment. it is neutral in the sense of being morally neutral (i.e. stings are not necessarily morally permissible, nor are they necessarily impermissible). and it is broader in that (a) the agents australasian journal of information systems luck 2019, vol 23, research on applied ethics (cybersecurity) entrapment behind the firewall 3 involved need not be law officers (that is, it includes instances of private entrapment),1 and (b) the actions of the target need not be otherwise criminal. hill et al. argue that the entrapment occurs whenever: 1. an agent plans that the target commit an act; 2. the planned act is of a type that is criminal, immoral, embarrassing, or socially frowned upon (measurable in part by the extent to which the target would probably not like the act to be exposed to colleagues, an employer, friends, family, or the public); 3. the agent procures the act (by solicitation, persuasion, or incitement); 4. the agent intends that the target’s act should, in principle, be traceable to the target either by being detectable (by a party other than the target) or via testimony (including the target’s confession), that is, by evidence that would link the target to the act; 5. in procuring the act, the agent intends to be enabled, or intends that a third party should be enabled, to prosecute or to expose the target for having committed the act. (pp.13 – 14) to illustrate this account, let us see how it applies to the hacking case. first, the bank’s security manager (the agent) plans that the hacker (the target) commit an act (that of transferring the funds). second, the transfer of the funds is the type of act that is clearly immoral/criminal. third, the manager procures the transfer (by placing the vulnerable account in a place the hacker would surely notice). forth, the manager intends that the hacker’s transfer be traceable to the hacker. fifth, in procuring the transfer, the manager intends to expose the hacker. so, according to hill et al. this is a case of entrapment, in the neutral, broad sense. i shall refer to this understanding of entrapment as neutral entrapment. although the paradigm case of an internal cyber-sting (the hacking case) is also an instance of neutral entrapment, it does not follow that all cases of internal cyber-stings are. it may be that the four actions i will go on to establish (for strengthening an organization’s moral reason to undertake a sting) are applicable to all cases of internal cyber-stings; yet, for the sake of simplicity we focus solely on internal cyber-stings that are also instances of neutral entrapment. so, with this sub-class of internal cyber-stings identified, and their relation to entrapment made clear, we are now well placed to consider what actions might strengthen an organization’s moral reason to enact such stings. however, before outlining these actions, let’s first be clear about what is meant by “strengthening moral reason”. 4 what does it mean to strengthen moral reason? the aim of this paper is to detail four actions which organizations might take to strengthen their moral reason for employing internal cyber-stings. but what exactly is a moral reason and what does it mean to strengthen one’s moral reason (or to have stronger moral reason)? a 1 see yaffe (2005) for more on the notion private entrapment. australasian journal of information systems luck 2019, vol 23, research on applied ethics (cybersecurity) entrapment behind the firewall 4 moral reason is a reason for the moral permissibility of an action (a reason which has some degree of moral weight). for example, a moral reason for a surgeon to cut their patient is that it is required remove a malignant tumour. moral reason for an action is strengthened when we have more reason for its moral permissibility. for example, a surgeon’s moral reason for cutting a patient is strengthened when, in addition to needing to cut out the malignant tumour, they also have the patient’s consent to being cut. although this may sound straightforward, it is worth noting that: (1) strengthening moral reason for an action doesn’t necessarily make it morally permissible; (2) it is possible to strengthen the moral reason for an action that is already permissible; and (3) that the notion of stronger moral reason can operate under multiple ethical theories. the reminder of this section will focus on these three points. the first point to note is that, strengthening the moral reason for an action doesn’t necessarily make it morally permissible. to illustrate this point, consider again the surgeon who discovers that their patient has a malignant tumour that needs to be cut out. this discovery strengths the surgeon’s moral reason to cut their patient. however, this doesn’t mean its permissible. why? because, the surgeon also requires the consent of the patient for the procedure to be permissibly. so, there is a marked difference between having more reason for an act’s permissibility, and the act being permissible; or as norcross (2006) puts it, the “fact that there is a moral reason to perform some action, even that there is more moral reason to perform it than any other action, doesn't mean that one ought to perform it” (p.48). this point is worth stressing, for the arguments presented here are not for the moral permissibility of internal cyber-stings; they merely aim to provide more moral reason for their permissibility. the second point worth noting is that, it is possible to strengthen the moral reason for an action that is already permissible. to illustrate this point, consider again the surgeon who discovers that their patient has a malignant tumour that needs to be cut out. let us assume the surgeon has fulfilled all the conditions required for the operation to be morally permissible (e.g. gained the patient’s consent, etc.). however, it may be still be possible to strengthen the surgeon’s moral reason for the operation. for example, imagine that the surgeon is informed that the patient is a scientist who is on the brink of curing cancer; and without the operation they will be unable to complete the cure. if this is an additional reason for the permissibility of removing the tumour, then the surgeon’s moral reason has been strengthened. however, removing the tumour would have been permissible regardless of this stronger moral reason. this point is worth stressing, as although the arguments presented here aim to strengthen the moral reason for internal cyber-stings, there is no presumption that such stings are impermissible without such strengthening. the third and last point worth noting is that the notion of stronger moral reason can operate under multiple ethical theories. to illustrate this let us consider an example of stronger moral reason operating under consequentialism, and another under deontological ethics. consequentialism is the ethical theory “that whether an act is morally right depends only on the consequences of that act” (sinnott-armstrong, 2015). broadly put, consequentialism holds that the act that results in the most moral value is the act you should perform. to illustrate how one’s moral reason might be strengthened under consequentialism, consider the following case, a surgeon in an understaffed er ward who must choose between operating on patient a or patient b (where the patient left untreated will die). australasian journal of information systems luck 2019, vol 23, research on applied ethics (cybersecurity) entrapment behind the firewall 5 for the sake of simplicity let us proceed on the assumption that the lives at stake in this example are the only things of moral value, and every life is equally valued. as things stand, saving either patient seems morally permissible to the surgeon, as in either case she saves the most lives she can (i.e. one life). however, upon hearing that patient a is a scientist who is on the brink of curing cancer, their moral reason for saving patient a has now been strengthened. this is because, more lives will be saved if patient a survives. this an example of moral reason being strengthened under consequentialism. let us now consider an example of stronger moral reason under deontological ethics. according to deontological ethics, “what makes a choice right is its conformity with a moral norm” (alexander, 2016). a moral norm can be thought of as a rule that must be followed (regardless of the consequences) to act morally. to illustrate how one’s moral reason might be strengthened under deontological ethics consider following example, you have the option to either do nothing and let an empty runaway trolley careen down track a killing three innocent people stuck there, or instead redirect the trolley to track b where it will safely run off the tracks, over a cliff and into the sea, or instead redirect the trolley to track c where it will safely come to a halt upon the track. let us imagine that the following two moral norms are in place: “save lives” and “do not be wasteful”. and for the sake of simplicity, let us assume that these are the only relevant moral norms. in this case one might consider that there is stronger moral reason for redirecting the train to track c, rather than track b. why? because by redirecting the train to track c one upholds both moral norms (i.e. three lives are saved, and the trolley is not wasted). this an example of stronger moral reason under deontological ethics. that moral reason can be strengthened (or can be stronger) under multiple ethics theories is worth stressing. for the arguments presented (which aim to strengthen the moral reason for internal cyber-stings) do no assume the correctness a particular ethical theory, or that they will operate equally well under multiple ethical theories. so, given this understanding of what is means to have stronger moral reasons to perform an action, let us now proceed to outline each of the four actions; each of which i argue strengthens an organization’s moral reason to employ an internal cyber-sting. 5 action 1: obtain the target’s consent in this section, i shall argue that there is stronger moral reason for an internal cyber-sting if the target properly consents to it. so, gaining the target’s consent is an action an organization can perform to strength its moral reason for stinging. to this end, i will draw an analogy between internal stings and internal monitoring.2 consider the following example of internal monitoring: a manager wishes to determine how hard her employees are working. without notifying her employees of the possibility of this type of monitoring, she starts secretly reading their emails in order to determine to what extent they are business-related. 2 although not all internal monitoring occurs in the workplace (for example monitoring one’s own family) since the most frequently considered type of internal monitoring is workplace monitoring, i will focus chiefly on this sub-class. australasian journal of information systems luck 2019, vol 23, research on applied ethics (cybersecurity) entrapment behind the firewall 6 in this email-monitoring case, the manager’s covert monitoring of her employees is generally considered to be impermissible. reasons for this vary. some (miller & weckert, 2000) argue that such a practice constitutes a breach of an individual’s right to privacy (and in turn precipitates the breakdown of trust). some (clarke, 2005) argue that this practice fails to uphold the principle of autonomy. these reasons are likely to collapse into each other if, as wall (2011) maintains, the moral right to privacy is built upon the moral right to autonomy (p.69). we will proceed on this assumption. if covert internal monitoring is impermissible because it ultimately represents a breach of privacy (and so a breach of autonomy), a key question to consider is: would the manager’s actions be less objectionable were she to gain her employees’ consent to being monitored? let us consider some reasons for why it would be less objectional. first, if covert internal monitoring is impermissible because it constitutes a breach of an individual’s right to privacy, then provided an employee provides proper consent (i.e. they are not improperly coerced, and they are sufficiently informed, etc.)3 the employer may be able to avoid breaching this right. why think this? because, as mccloskey (1980) states, it is evident that we can consent to forgo privacy, properly, as in marriage, friendship, and certain other social relationships, putting certain areas of our lives outside the sphere of privacy for certain other persons. (p.37) to illustrate this point, consider the right to privacy i have in my own home. i can consent to forgo some degree of privacy by choosing to live-stream my home-life antics on the web (provided i have not been coerced into this, and i am fully informed of the intrusion [i.e. where the cameras are etc.]). so how is it that by gaining someone’s consent, we avoid breaching their right to privacy? westin & ruebhausen’s seminal work on privacy provides an indication. westin & ruebhausen define privacy as “the claim of individuals, groups or institutions to determine for themselves when, how and to what extent information about them is communicated to others” (westin & ruebhausen, p. 7). and how do agents govern this extent. wall argues that is via consent, casting privacy as the “moral right to consent to access by others to one’s personal information” (p.69). so, by gaining someone’s consent to monitor them, we uphold their moral right to consent, and so in turn uphold their right to privacy, and so avoid the breach.4 note that i am not suggesting that gaining proper consent is sufficient for internal monitoring to be permissible (although it may well be). rather i am suggesting that gaining proper consent (as opposed to not) provides stronger moral reason for workplace monitoring. one might think that this is obviously the case, on the grounds that if someone consents to some manner of treatment then it is permissible to treat them in this manner. however, this principle seems too strong, for, as miller (2005) points out “even if people do consent to some 3 see palm (2009) for a discussion on the necessary conditions for morally acceptable consent (in the context of limiting one’s right to privacy in the workplace). 4 we can run a similar line if covert internal monitoring is impermissible because it fails uphold the moral right to autonomy (but not because of issues of privacy). for provided an employee properly consents, then one’s ability “to determine the course of one's own life free of the interference of others” (clarke, p.237), is diminished less by internal monitoring if one has freely chosen a course of life involving such monitoring. australasian journal of information systems luck 2019, vol 23, research on applied ethics (cybersecurity) entrapment behind the firewall 7 sort of treatment, it doesn’t follow that it is moral to treat them in that manner” (p.276). for some manners of treating people, for example treating people as slaves, may be impermissible regardless of the fact they have properly consented to such treatment. yet what seems more defensible is the following principle: there is stronger moral reason for treating someone in a manner they have properly consented to, than otherwise. for example, although treating someone as a slave may always be morally impermissible, there is more moral reason to treat them as such if they have consented to it, than had they not. to support this point, imagine you were forced between treating someone as a slave who has properly consented to this (i.e. not coerced economically or otherwise [a submissive,5 for example]), or treating someone as a slave who has not. this does not seem like a choice one should make by flipping a coin. the addition of consent does seem to have some moral weight.6 if this principle holds, then we can clearly see how it also applies to workplace stings. for, if the target properly consents to the possibility of a sting (as opposed to not), there will be more moral reason for its permissibility. so, in the hacking case, if the hacker properly consented to the possibility of a sting (perhaps as a standard condition of employment), there will be more reason for the sting’s permissibility. however, even if this principle fails an argument can be given for the same conclusion by drawing an analogy between internal monitoring and internal stings. if internal monitoring without consent is wrong because it breaches an employee’s moral right to autonomy, then the question to consider now is why an internal sting without consent is wrong. to answer this question it is worth noting, as dworkin (1988) does in relation to the stings used in cases of legal entrapment, that what “these techniques have in common is the use of deception to produce the performance of a criminal act under circumstances in which it can be observed by law enforcement officials” (p.130). that these stings involve deceit (by which the agent procures the wrongdoing) is the relevant point here; as deceit, it is sometimes argued, threatens autonomy. why? one reason is that a deceptive environment is an environment where it is difficult make informed choices. and one needs to be able to make such choices in order to properly determine the course of one's own life. so, internal stings may be wrong because they involve deception, which threatens our autonomy. if both internal stings without consent, and internal monitoring without consent, are wrong because they both breach the employee’s autonomy, then perhaps, like internal monitoring, there can be more moral reason for internal stings when employers gain proper consent. in other words, perhaps the following argument can be mounted: 1. internal monitoring and internal stings are relevantly similar. 2. there is stronger moral reason for internal monitoring when proper consent is gained (as opposed to when it isn’t) 5 a person who desires (at least for limited period of time) to submit their will to another. 6 note i am not suggesting that slavery might be permissible if consent is given (slavery may always be impermissible), only that consent has some moral weight. i am also not suggesting the permissibility of slavery is relative to the people being enslaved. as slavery may always be impermissible regardless of whether consent is given (despite it having some moral weight). australasian journal of information systems luck 2019, vol 23, research on applied ethics (cybersecurity) entrapment behind the firewall 8 so, 3. there is stronger moral reason for internal stings when proper consent is gained (as opposed to when it isn’t) however, if this argument also fails to persuade, there is a further argument for the same conclusion we shall also now consider. rather than drawing an analogy between monitoring and stings, perhaps it might be more fruitful to think about how gaining proper consent reduces the element of deceit. to this end, let us consider a related issue in medical ethics: the use of placebos. a placebo is “a substance that the physician believes has no known specific pharmacological activity against the condition being treated” (bostick et all., 2008, p.58). physicians sometimes administer placebos to patients without the patient knowing a particular treatment is a placebo, as this can have therapeutic benefits (known as the placebo effect). some argue that it is impermissible to administer placebo treatments as they involve deception, and so “essentially run counter to the ideal of informed consent, which is central to the ideal of patient autonomy” (groll, 2011, p.198). however, others argue that in those cases where it is possible to gain a patient’s general consent, the treatment may be permissible, for example, a physician could explain to a patient that a more certain diagnosis or better understanding of his or her condition could be achieved by evaluating the effects of different types of medication, including one that is not pharmacologically active, namely, a placebo. by obtaining the patient’s cooperation in this manner, the physician need neither identify which medication is the placebo nor seek specific consent immediately before its administration. this example of shared decision making demonstrates an approach that respects the autonomy of patients and fosters trust within the patient-physician relationship. moreover, the authorized use of placebos is not expected to significantly diminish their clinical effectiveness as research suggests that little variation in clinical outcomes is observed between patients who are informed that they are to be treated with placebos and patients who are administered placebos in a deceptive manner. (bostick et all., p.59) the relevant point here is that the wrongness of a deceptive practice (a practice in tension with the patient’s autonomy) is arguably lessened (or perhaps even alleviated) when proper consent to the practice is obtained. by the same token, there is reason to think that the wrongness of the deceptive practice involved in an internal cyber-sting is lessened when proper consent to the practice is obtained in a relevantly similar manner. which in turn supports the conclusion that there is stronger moral reason for an internal cyber-sting if the target properly consents to it. so, getting an employee to consent to the possibility of a sting is an action that organizations can take to strengthen the moral reason for their use. (the other additional benefit worth briefly mentioning is that informing employees of possible stings may also act as a deterrent.) however, some organizations may not wish to gain consent on the grounds that informing a target of the prospect of a sting may increase the possibility of them evading it. this is an understandable concern. however, even if the possibility of stinging a target is reduced in this fashion, it might not be reduced so much as to make stings ineffective. for example, although criminals outside of the workplace are aware of the possibility of a sting by law enforcement agencies, it doesn’t follow that such stings fail to ensnare their targets. so, although gaining australasian journal of information systems luck 2019, vol 23, research on applied ethics (cybersecurity) entrapment behind the firewall 9 consent may lower the chances of a successful sting, it may still be worth it given it strengths an organization’s moral reason for the practice. 6 action 2: ensure the stakes are high in this section, i shall argue that the more that is morally at stake, the stronger moral reason for the internal cyber-sting. so, ensuring that the stakes are high is something an organization can do to (proportionally) strengthen its moral reason for stinging. to help make this point, we begin with an illustration of a simpler principle. consider the following case: while discussing beloved films with a new acquaintance you are asked about your favourite film. you are about to tell her that it is bladerunner when you remember that this is the password to your world of warcraft account (a popular online game that you are mildly amused by). in order to lessen the chances of this acquaintance gaining access to your game account, you lie and tell her your favourite movie is akira instead. some may argue that lying in this password case represents a breach of trust and an act of deceit, and so is to some extent (that is, pro tanto) morally wrong. whether the breach might also be all-things-considered morally wrong, and so morally impermissible, is less clear. perhaps, because what was at stake was so trivial it was impermissible to lie; or perhaps because what you were lying about was so trivial it was permissible. however, what does seem clear is that there is more reason to lie if “bladerunner” were the password to your bank account (filled with your life’s savings). the principle i am supporting here is: the greater the cost, the more reason there is to lower the risk.7 if this principle holds, then the more you stand to lose from your new acquaintance gaining access to your bank account, the more reason there is to lie. however, it does not follow that you have more moral reason to lie (or that it is morally permissible to lie). whether there is more moral reason will depend on whether what is at stake is of moral import. in this password case, if there isn’t anything morally at stake then there may be no moral reason to lie to your friend. but if there were something morally at stake then there would be moral reason to lie. this idea can be captured by the following principle (which is a variant of the previous simpler one): the greater the moral cost, the more moral reason there is to lower the risk. if this principle holds then there will be stronger moral reason for a sting the more there is morally at stake (given the sting is a means to lower the risk). so, for an organization’s moral reason to be strengthened by this consideration significantly, they should ensure that there is something of significant moral value at stake. sometimes, it will be easy to determine whether there is something of significant moral value at stake. for example, a business that merely wishes to lower the amount of toilet paper their employees are using at work does not have anything of significant moral value at stake; so, it has little moral reason (in terms of the stakes) to enact a sting to discover culprits using generous amounts to toilet paper. however, agencies that protect sensitive security data (such as the cia, mi5 or asio) clearly do have something 7 where cost is simply understood as something bad, and risk is understood as the chance of something bad. australasian journal of information systems luck 2019, vol 23, research on applied ethics (cybersecurity) entrapment behind the firewall 10 of significant moral value at stake; so, they have much more moral reason to enact a sting in order to discover culprits who might seek to divulge this data to hostile groups. let us now return to internal cyber-stings and examine how this principle might be applied to the hacking case. before enacting a sting upon the hacker, the banking security manager should consider whether there is anything morally at stake. at first it might seem that there is little at stake here morally, as the bank only stands to lose money, which (unlike a person’s life) doesn’t obviously have moral value. however, this assessment is far too quick. the bank might have a moral obligation to its employees and/or shareholders to protect its own interests; and so, since both its financial and reputational interests might be damaged by the hacker’s cyber-theft, the bank might have a moral reason to guard against it. and it is easy to see that the more money that is at stake the more moral reason there is to protect it, which in turn strengthens the moral reason for a cyber-sting that offers such protection. the bank might also have a moral obligation to protect their customers’ interests; for example, the private information kept by the bank on its customers (such as addresses, dates of birth, and transaction information) might also be things the bank has moral reason to protect (reason complementary to, but independent from, its own interests). the bank might also have a moral obligation to act against cyber-theft regardless of what they stand to lose. why think this? because organizations may have a moral reason, as we all arguably do, to take reasonable steps to act against wrongdoing. note that i am not suggesting that they have a moral obligation to stop wrongdoings (although presumably they sometimes do). only that, they have some moral reason to. for example, if you pass someone about to wrongfully litter, then, even if you do not have a moral obligation to stop them, there still exists a moral reason for you to do so. and the bigger the wrongdoing the more moral reason there is for its prevention. weckert and miller make a similar point regarding the permissibility of internal monitoring, stating that “theft and fraud in the workplace are still theft and fraud, so some surveillance can be justified in order to apprehend culprits” (p.260). or put another way, if failure to prevent cyber-theft comes at a moral cost, there is some moral reason to prevent it. consequently, if this principle holds (that the greater the moral cost, the more moral reason there is to lower its risk) then the more that is morally at stake, the stronger the moral reason for taking steps to lower the risk; steps which may include enacting an internal cyber-sting. note that i am not suggesting here that organizations should not enact an internal cyber-sting unless the stakes are high (although this may be the case). only that by ensuring that the stakes are high, organizations can be ensured that there is stronger moral reason for a sting (as opposed to if the stakes are low). 7 action 3: ensure you have evidence of the target’s predisposition to act wrongly in this section, we shall argue that the more evidence an agent has for the target’s predisposition to act wrongly (in a particular way), the more moral reason the agent has to sting the target (by activating this predisposition). so, ensuring that there is considerable evidence for the predisposition is something organizations can do to (proportionally) strengthen its moral reason for stinging. to help establish this principle, consider the following case: australasian journal of information systems luck 2019, vol 23, research on applied ethics (cybersecurity) entrapment behind the firewall 11 you have no reason to think that your partner is cheating on you, however you are determined to put them to the test anyway. to this end you use your extensive knowledge of your partner to create an enticing online profile of their perfect match. you then contact your partner online under the guise of this perfect match in order to see if you can catfish8 them. some may argue that your actions in this catfish case are morally impermissible. this is because (in addition to the use of deceit and the violation of trust) you are also attempting make your partner do something immoral (i.e. to intend to cheat on you). unless one is a moral saint, we are all vulnerable such traps. that is, even those of us who only have a small disposition (rather than a predisposition) to act wrongly, will act wrongly if this disposition is sufficiently encouraged. given this, we shouldn’t encourage people to act wrongly unless we have good reason; otherwise we are making the world worse. as dworkin (1988) states in respect to the techniques lead to legal entrapment: the central moral concern with pro-active law enforcement techniques is that they manufacture or create crime in order that offenders be prosecuted and punished. they do not discover criminal activity; they create it. (p.136) or, as carlon (2007) puts it, it is critical that “we are prosecuting existing crimes, rather than crimes that we have created for the purpose of punishment” (p.1124). so, there is a moral risk to encouraging wrongdoings via such stings, as there is a chance one is acting impermissibly by creating wrongdoings that would not have occurred otherwise. now let us contrast this catfishing case to one where we have some evidence that our partner has a predisposition to cheat (although this predisposition is not so strong that it mitigates any blame if acted upon; for example, he is not a sex addict). let us also stipulate that, although we have some evidence that they cheat, we do not have sufficient evidence to justifiably end the relationship (and so save ourselves from being cheated on longer than is necessary). although it still might prove impermissible to catfish our partner in this case, it does now seem less wrong. or put another way, there seems to be more moral reason for the sting in this case. so why think this? although in both cases the sting causes the partner’s disposition/predisposition to manifest, in the case of the partner with the small disposition to cheat, had they not been stung the chances of them intending to cheat would be low. whereas in the case of the partner with the predisposition to cheat, had they not been stung, the chances of them intending to cheat would remain high. what does this tell us? it tells us it is likely that the first sting makes the world worse, whereas the second doesn’t (in fact the second sting provides you with the evidence required to end the relationship and move on). this difference is moral one. that is, given the possibility of making the world worse, the more evidence we have that our target would have performed the wrongdoing in the absence of the sting, the more moral reason there is for the sting. accordingly, we can mount the following principle on the back of this difference: the more evidence an agent has for the target’s predisposition to act wrongly (in a particular way), the more moral reason the agent has to sting the target (by activating this predisposition). this principle is also supported by one of the arguments for entrapment defence: the subjective 8 that is, trick them into making a romantic advance with the fake online persona. australasian journal of information systems luck 2019, vol 23, research on applied ethics (cybersecurity) entrapment behind the firewall 12 approach. the subjective approach holds that the entrapment defence is successful (resulting in the defendant being found not guilty) when the evidence fails to demonstrate that the defendant has a predisposition to act wrongly. and inversely, the “entrapment defence fails if it can be shown that the defendant was “predisposed” to commit the crime” (yaffe, p.7). why? because, as sinnott-armstrong (1999) states, those “with weak predispositions to crime are unwary innocents, whereas those with strong predispositions to crime are the very people who need to be found guilty and punished to prevent their crimes” (p.99). so, for example, although someone might fall for a sting by unwarily buying illegal drugs from an uncover police officer, if the evidence gleamed from this sting is insufficient to demonstrate a criminal predisposition to buy such drugs, they may be found not guilty, and the moral permissibility of a sting questioned. on the other hand, if one has prior evidence (perhaps a criminal record, or testimonial evidence, of the defendant buying illegal drugs) (evidence that motivated the police stinging this individual), then if the sum of the evidence is sufficient to demonstrate the predisposition to buy the drugs, they may be found guilty, and the moral permissibility of a sting left unquestioned. how does the subjective approach support this principle? the subjective approach holds that if one has sufficient evidence to demonstrate a target’s predisposition to act wrongly then the sting is morally permissible. it follows from this that the more evidence one has of this predisposition, the more moral reason there is for the sting’s permissibility. this principle is of particular relevance to the hacking case (and perhaps the cybersecurity sector more generally). this is because there is a real pragmatic advantage to organizations employing hackers with actual cyber-crime experience (that is, current or former black and grey hat hackers) to help strengthen their security; for, as the adage attests, set a thief to catch a thief. however, with this pragmatic advantage also comes a risk, as the hacking case illustrates. but if one has evidence that the hacker once performed such wrongdoings, one also has, to some degree, evidence of a predisposition. why is this important? because if this principle holds, then the more evidence an organization has, the stronger the moral reason to target the hacker with a sting. note that i am not suggesting that organizations should not enact an internal cyber-sting unless they have sufficient evidence of this predisposition (although this may be the case). only that by ensuring that they have such evidence, organizations can be ensured that there is stronger moral reason for stinging. 8 action 4: ensure the sting’s inducement is low in this section, we shall argue that the lower the sting’s inducement to act wrongly, the stronger the moral reason for the sting. so, ensuring that the inducement is low is something organizations can do to (proportionally) strengthen its moral reason for stinging. to help establish this principle, consider the following case: thomas anderson, a struggling unemployed software developer, is attending a job interview at a prestigious tech company. before facing the interview panel, the hr officer assigns thomas to a computer to undertake a standard english language aptitude test. thomas discovers that the previous computer’s user is still signed into their email account. although thomas knows that the right thing to do is sign out of this account on their behalf, he is enticed by the subject heading of the most recent australasian journal of information systems luck 2019, vol 23, research on applied ethics (cybersecurity) entrapment behind the firewall 13 email: “questions to ask thomas anderson during the interview today”. despite his better judgment thomas opens and reads the email. what he doesn’t realize is that he has been stung. the hr officer engineered this situation to expose (to the interview panel) each candidate’s willingness to read someone else’s emails. some might argue that the hr officer acted wrongly in the interview case. one reason for this is that the inducement to read the email was too high; that the chance for a struggling unemployed job-candidate to gain an advantage in an interview for a prestigious job is just too tempting. let us grant that even if thomas only had a very small disposition to violate someone’s privacy, he would still be all but compelled to read this email. so why is it that baiting a sting with such a strong inducement might be impermissible? it is for the same reasons as given in the previous section; that is, it is because, as yaffe states (in regards to legal entrapment), these stings may be causing wrongdoings without preventing any. …by engaging in behavior that risks ensnaring the unpredisposed…the government engages in behavior that is likely to cause criminal conduct without preventing any, since the unpredisposed are not likely to have acted criminally without temptation from the government. (yaffe, p16) let us take as read that causing such wrongdoings is wrong. we discussed (in the previous section) that one way to lower the chances of causing such a wrongdoing is by gathering evidence of the target’s predisposition to act wrongly prior to establishing the sting. in this section, we focus on establishing a particular kind of sting with a low inducement, in order to lower the chances of causing the same type of wrongdoing (i.e. one that would not have otherwise occurred). to see why it is that the lower the sting’s inducement, the lower the chances of causing a wrongdoing that would not have otherwise occurred, we need to consider the relationship between predispositions and inducements. this relationship, as hall et al explains, is one of inverse proportion: other things being equal, the greater the degree of inducement or enticement has to be before the person succumbs to temptation and commits the offence, the less the person can properly be held to have been predisposed to commit the offence. (p.15) given this relation, as carlon states, “the success of a slight inducement indicates great predisposition; the resort to an extremely powerful inducement militates for a finding of less predisposition” (p.1093). so, if we have more moral reason to sting those with higher predispositions to act wrongly, then we have moral reason to lower the sting’s inducement (without rendering it impotent). let us return to the interview case to apply this principle. what this principle tells us is that, if the lure of subject heading “questions to ask thomas anderson during the interview today” constitutes an inducement so high that thomas was all but compelled to take a peek, then it offers little indication of him having a predisposition to breach someone’s privacy. but it does more strongly indicate that the hr officer has done something wrong, by causing a wrongdoing where none would had occurred otherwise. yet, were we to adjust the sting somewhat, so the subject heading was now something much less enticing, such as “how to get into the executive bathroom”, and thomas now took a peek, then this sting offers more of australasian journal of information systems luck 2019, vol 23, research on applied ethics (cybersecurity) entrapment behind the firewall 14 an indication that he has a predisposition to breach someone’s privacy, and in turn less strongly indicates that the hr officer has done something wrong. so, let us apply this to the internal cyber-security hacking case. when setting up the sting, the bank’s security manager has (according to this principle) moral reason to lower the amount of money in the account that the hacker might be tempted to steal. for the more money in the account, the more chance there is that, even if the hacker only had a small disposition to steal, he would be all but compelled to do so, and so the greater the chance the manager would have done something wrong in causing this wrongdoing. consequently, if the principle holds (that the lower the sting’s inducement to act wrongly the stronger the moral reason for the sting) there is moral reason to setup the cyber-sting with a small (but still potent) inducement. note that i am not suggesting by this that organizations should not enact an internal cyber-sting unless the inducement is small (perhaps if the stakes are high enough, for example, it might still be warranted). only that by ensuring that the inducement is low, organizations can be ensured that there is stronger moral reason for the sting. 9 conclusion the aim of this paper was to present four actions (each underpinned by a separate principle) that can be taken by organizations to strengthen their moral reason for employing internal cyber-stings, and so increase their internal cybersecurity. these actions were: obtain the target’s consent; ensure the stakes are high; ensure you have evidence of the target’s predisposition to act wrongly; and ensure the sting’s inducement is low. it is worth emphasising in closing that taking these actions is no guarantee that any subsequent sting is morally permissible (perhaps they never are). however, in the absence of an established account which clearly lays out the necessary and sufficient conditions for permissibility of neutral entrapment, it seems prudent to consider these recommended actions before employing an internal cyber-sting. references alexander, l. (2016). “deontological ethics”, in zalta, e. (ed.), stanford encyclopedia of philosophy. https://plato.stanford.edu/entries/ethics-deontological/ bostick, n., sade, r., levine, m. & stewart, d. (2008). “placebo use in clinical practice: report of the american medical association council on ethical and judicial affairs”, the journal of clinical ethics, 1:19. ca tech. (2018) insider threat report. ca technologies. carlon, a. (2007). “entrapment, punishment, and the sadistic state”, virginia law review, 93:1081. clarke, s. (2005). “informed consent and electronic monitoring in the workplace”, in weckhert, j. (ed.) electronic monitoring in the workplace: controversies and solutions, idea group publishing. dworkin, g. (1988). the theory and practice of autonomy. cambridge university press. groll, d. (2011). “what you don't know can help you: the ethics of placebo treatment”, journal of applied philosophy, 2:28. australasian journal of information systems luck 2019, vol 23, research on applied ethics (cybersecurity) entrapment behind the firewall 15 hill, d.j., mcleod, s.k. & tanyi, a. (2017). “the concept of entrapment”, criminal law and philosophy (2017). https://doi.org/10.1007/s11572-017-9436-7 hughes, p.m. (2004). “what is wrong with entrapment” the southern journal of philosophy, 1:42 ibm. (2016). 2016 cyber security intelligence index. ibm x-force research. mccloskey, h. j. (1980). “privacy and the right to privacy”, philosophy 55: 211. miller, s. (2005). “guarding the guards: the right to privacy, and workplace surveillance and monitoring in policing”, in weckhert, j. (ed.) electronic monitoring in the workplace: controversies and solutions, idea group publishing. miller, s. & weckert, j. (2000). “privacy, the workplace and the internet”, journal of business ethics, 3:28. norcross, a. (2006). “reasons without demands: rethinking rightness”, in dreier, j. (ed.), contemporary debates in moral theory. blackwell. palm, e. (2009). “securing privacy at work: the importance of contextualized consent”, ethics and information technology, 4:11. sinnott-armstrong, s. (1999). “entrapment in the net?”, ethics and information technology, 2:1. sinnott-armstrong, s. (2015). “consequentialism”, in zalta, e. (ed.), stanford encyclopedia of philosophy. https://plato.stanford.edu/entries/consequentialism/ verizon. (2016). data breach investigations report. verizon enterprises. wall, e. (2011). “privacy and the moral right to personal autonomy”, international journal of applied philosophy 1:25. westin, a. f., & ruebhausen, o. m. (1967). privacy and freedom. new york: atheneum. yaffe, g. (2005). “‘the government beguiled me’: the entrapment defense and the problem of private entrapment”, journal of ethics and social philosophy, 5:2. copyright: © 2019 luck. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ knowledge analysis of tasks as a basis for interface design of complex developmental systems alan w. colman and ying k. leung, swinburne computer-human interaction laboratory, school of computer science & software engineering, swinburne university of technology, hawthorn campus, victoria 3122. e-mail: {awc,ying}@jupiter.cs.swin.oz.au abstract complex development systems are interactive software systems used for the manipulation, design or development in complex problem domains. this paper highlights some of the limitations of johnson's knowledge analysis of tasks (kat) (johnson, 1989, 1992) and proposes a modified version of kat where task knowledge can be analysed and grouped in a way that will make it useful in the derivation of interfaces in complex developmental systems. this modified form has been applied to the domain of intelligent distributed control systems in an attempt to develop interface concepts for the development, installation and documentation of such systems. the paper further shows how this extended version of kat may prove a useful input to object-oriented analysis. introduction developing methods of task analysis has been a major endeavour in the field of humancomputer interaction (hci). the aim of task analysis is to ground the analysis and design of systems in the study of users interaction with such systems. diaper (1989) gives an overview of the various and diverse methods of task analysis as they have developed from the study of human factors and ergonomics. some of these methods include hierarchical task analysis (hta), task-action grammar (tag) (payne and green 1989); and task analysis for knowledge descriptions (takd) and knowledge analysis of tasks (kat) both developed by johnson (1989). all the methods developed recognise the importance of the cognitive aspects of tasks in studying hci, although there are differences in approach as to what extent task analysis has to be based on the systematic observation of people performing tasks (diaper 1989 pll). the transition from task analysis to system design has also been problematic. this paper is an attempt to apply johnson's task analysis method kat to what we have characterised as "complex developmental systems", in a way that will prove useful for the design of software interfaces. complex developmental systems are interactive software systems used for the manipulation, design or development of complex problem domains. examples of such systems are case tools, simulation software, control network development systems, etc. these systems are complex in the sense that the problem domain is structurally complex and multi-faceted. information on the state of facets of the problem domain needs to be presented to the user, in order for the user to manipulate the problem domain or its representation effectively. structurally complex domains can be represented semantically as sets of relationships between different types of objects. for example, plain text in this sense is not structurally complex (although it can be structurally large); so whilst a word processor is a complex piece of interactive software, it is not representing a structurally complex domain1. complex developmental systems are developmental in the sense that they are used in the process of design, and thus the representation of the structure of the problem domain changes over time. a real time monitoring system may be interactive and structurally complex, but is not developmental. complex developmental systems exhibit a number of features that impact on the way we analyse tasks performed on such systems in order to provide an input to interface design. these can be summarised as follows: • the development tool can be separated from the problem domain; • the development process is opportunistic; • the problem domain is structurally complex and the user must maintain a sophisticated mental model of the problem domain in order to interact with it. to represent this domain the system must present multiple system-images. knowledge analysis of task (kat) is a method for identifying a user's task knowledge structures proposed by johnson (1989). johnson identifies three distinct parts to the kat method: (1) identifying knowledge the identification and collection of knowledge that people use in the performing of tasks. (2) analysis of knowledge the identification of the representativeness and centrality of a particular task knowledge component and establishing generic task knowledge. (3) construction of a task knowledge structure (tks) a tks is knowledge of tasks "acquired through learning and performing a given and associated task" (johnson 1989, pi67). johnson categorises these knowledge structures into three substructures: goal substructure knowledge about goals and subgoals. this substructure includes enabling and conditional states that must prevail if a goal or subgoal is to be achieved. taxonomic substructure knowledge about the properties of task knowledge and their associated actions. the taxonomic substructure "identifies the object properties and attributes, including class membership, the procedures in which it is commonly used, [and] its relation, to other objects and actions..." procedural substructure knowledge about achieving a goal. procedural substructures are developed for "well-practised" tasks and are "different from goal substructures in so much as they are executable in a single unit". 1 text, of course, has a grammatical structure, but a writer is primarily concerned with the meaning of the words, not the structure of the grammar. a word processor is unstructured in that it cannot differentiate the semantic structure of the text. text can have semantic structure in terms of section headings, chapters etc and some modern word processors such as outliners and folding editors understand such structures. johnson is concerned with the learnability of user interfaces. the theory of tks assumes: "as people learn and perform tasks, they acquire (or more correcdy develop) knowledge structures that are established from previous task experiences and applied to future task performance in a dynamic process.... usability and learnability are direcdy related to the amount of existing knowledge that a user is able to transfer from the existing tasks to the revised form of those tasks as they should optimally be performed using the designed software" (johnson 1989 p. 162). the analysis of task knowledge involves identifying the "representativeness" and "centrality" of objects and actions, in order that the task knowledge structures can be made more general, and thus transferable (learnable). this paper suggests some extensions to the kat method which can be applied to complex developmental systems. these modifications arose from an attempt to apply kat to analysis of a development system for intelligent distributed control networks, the aim being to use kat as input to the design of an interface for such a development system. complex developmental systems the separation of development tool from problem domain in a complex developmental system we need to distinguish knowledge of two domains: (1) the tools for manipulating and developing the problem domain, and (2) the problem domain itself2. in computer systems, tools can vary from very simple to complex with the complexity the task differing accordingly; for example, from the bold button on a wordprocessing tool-bar to a network traffic analyser for a distributed control system. in any description of tasks, the description of the action-object pairs will differ depending on whether we are describing interface objects as tools (e.g. "with a mouse, move the cursor to the button and click the left button"), or whether the objects we are describing are problem domain objects (e.g "define the network variables for the alarm node"). an analogy with an architect's task is apposite. if we want to describe the knowledge required to design a house, we need to be concerned with knowledge of design principles, building regulations and the current state of the design, rather than just a knowledge of drafting skills. the knowledge of the tools is secondary (knowledge of how to draft a house design may change depending on whether you are using a pencil or computer) to the knowledge of the problem domain (knowledge of how to design a house and knowledge of the state of the design). traditionally, much human-computer interaction (hci) research into tasks has focused on describing task execution at the tool level e.g. keystroke level model (card et al. 1979). this is, in part, due to the fact that the tool is intrinsic to the interface; the interface is seen as separate from the "real" underlying application. it is also easier to quantify performance of task execution at the tool level, rather than to quantify task acquisition, or execution, at the 2 this distinction between tool and problem domain is similar to the distinction between the lexical/syntactic and semantic layers of an interface in the 'seeheim model' see johnson, drake .wilson (1990). carroll and olson (1988) make a similar distinction between "task-knowing" and "system interface knowing" in mental models, but adds knowledge of how the system works "system-architecture knowing" problem-domain level. understanding task acquisition or execution at the problem-domain level relies on concepts such the user's "mental model" which is difficult to substantiate.3 to interact with a system the user requires problem-domain knowledge of two types: task acquisition requires the user to have an understanding of the current state of the particular problem-domain development. the system interface needs to provide feedback to the user on this current state. to develop an interface we need to develop system-images to represent the problem domain. in complex domains these images are more easily understood if the problem domain is amenable to representation in graphical form (larkin & simon, 1987). task execution requires problem-domain knowledge on how to manipulate the problem domain. such knowledge is often held by the user rather than by the system. systems that incorporate such knowledge include rule-based expert systems. the incorporation of such knowledge into the system is outside our definition of a complex developmental system.4 the interface for a developmental system must, therefore, represent both the problem domain over time, and provide a tool for manipulating the problem domain. figure 1 illustrates the inherent interaction between the user and the problem domain. developer's mental model and tks representation of problem domain tool interface development tools problem domain tks/kat focus user level system level real world interaction extended tks/kat focus figure 1 interaction/knowledge transfer in a complex developmental system when analysing the user's knowledge of a system, to what extent should we be analysing knowledge of the problem domain, compared with analysing knowledge of the tools? it can be argued the most effective performance of tasks occurs when the user feels as if he or she is 3 see norman's (1986) theory of action' fora framework for discussing task "execution" and "evaluation". 4 non-expert systems, such as case tools and the lon development system (see example below), can often check a design for consistency with "how to" knowledge on the syntactic level. manipulating the problem domain directly. the tool should appear transparent to the user. this transparency can be a result of thorough familiarity with the tool so that the knowledge in the procedural substructure is executed automatically as a block. this transparency can also be a result of an appropriate, well-learned metaphor or the result of a direct manipulation interface style. such a style tries to minimise the number of conceptual levels operating between the user and the problem domain. to produce the tool-transparency of a direct manipulation style interface, we therefore need to give primacy to task analysis from the perspective of the problem domain. that is, in any given interaction, we are more concerned with describing the object in the problem domain that is being viewed or manipulated, than with describing the tools being used to manipulate the object. there are also design methodology imperatives for focussing on the problem domain rather than the tools; tools change, whereas the problem domain tends to be more stable. software built on stable objects is more extensible (meyer, 1988). the knowledge of tasks, as johnson describes it, is a knowledge of the application of tools to a problem domain. it does not take into account the knowledge a user may have of the problem domain that is independent of the user's previous tool-level interaction with that domain. such knowledge a mental model the user develops of the problem domain can be represented in similar terms to a tks taxonomy, but needs to be described in more complex terms than a simple response to a task-action. to johnson, a mental model (e.g. a desktop metaphor) is a mechanism for transferring knowledge of tasks (johnson 1989, p. 179), rather than informing the user about the state of the system (e.g. the state of the file system) so he/she can acquire and execute the task5. how will the process of task analysis differ when the focus is on analysing the user interaction with the problem domain rather than the user's interaction with the tools? the analysis of tasks (object-action pairs) will be an analysis of the actions on problem domain objects, rather than a description of user's manipulation of the tool. if the source of the data is the detailed observation of the user's interaction with a system, the analyst will need to abstract the task knowledge of the problem domain from the user's interaction with the tool. more accessible sources for problem domain task knowledge are likely to be problem-domain descriptions, documentation, existing system-images and so on. in the latter case, the domain objects are likely to be described in a way that is already generified: in johnson's terms the objects/actions will already be central and representative. in this case, the role of observational task analysis of subject interaction would be limited to the verification that the knowledge (as represented by the various kat substructures) is the same in the documentation/system-image as it is in the subject's head. even if the knowledge representations differ, it is problematic whether this difference is the result of the inadequate learning of the knowledge by the subject, or inadequacies in the documentation/systemimage representation of that knowledge. in summary, in a structurally complex domain the interface needs to provide a representation of the problem domain. the taxonomic 5 johnson makes no mention of the concept of a mental model other than metaphorical models. he points out that "metaphorical relations do not, themselves, identify general or any knowledge in detail .. [and] should be thought of as a mechanism for facilitating transfer rather than as a means of identifying knowledge that could be transferred." (johnson, 1989, p!76) 6 substructure will be derived from the abstractions of the problem domain in a form that is already generic. opportunism following johnson's approach, we wish to describe the knowledge required for the execution of tasks. in particular, we wish to describe the tasks related to software system development. researchers have identified the highly opportunistic nature of the software design process (guindon krasner & curtis, 1987; visser, 1991; khushalani, smith & howard, 1994) wherein the designer bounces back and forth across different levels of abstraction in developing the design. top-down balanced development only occurs when the designer fully understands the problem. johnson's goal substructure cannot, in this case, be taken as a high level procedural description, although obviously the user/designer uses some such structure to keep track of the progress of the development. in attempting to describe the tasks involved in the development of a system, and in providing a suitable interface for the carrying out of these tasks, it is not possible to describe a single sequence of procedures. we can, however, describe a set of goals and sub-goals that heeds to be achieved in order to complete the task. while the process of development as a whole may, not be a linear sequence, there are set sequences of tasks that must be followed in a particular development stage (e.g. the function of a node must be determined before one can link it into a network). these prerequisites, or 'enabling states' as johnson refers to them, can provide us with the basis of a generalised goal substructure. the procedural substructure, as johnson defines it, is a description of tasks executed as a single block. tasks described at the problem-domain level do not usually exhibit such automatic sequences, but need to be continually re-evaluated in terms of the goals. procedural substructures are therefore more relevant to a task description at the tool level. for example, a trip in a car can be described at the tool level as a series of actions manipulating interface objects such as the steering wheel, gear lever, brake pedal etc. from the user's point of view, such actions, after a while, become automatic with no conscious thought required. they are also easily transferable from one car to the next, because of the similarity of car-user interfaces. on the other hand, from the problem-domain level, the description of the trip is a description of the route from a to b, with the turnings at road intersections continually needing to be evaluated in terms of getting to the goal b, given problem-domain knowledge such as shortest route, traffic conditions, etc.. the driver needs feedback on the current state of the problem domain (he/she needs to know where they are on the road), and this understanding of the status of the problem domain is fundamental to applying their goal knowledge. the dynamics of an interface for a complex developmental system needs to support the opportunistic nature of the interaction, thus the description of the dynamics of the interface at problem-domain level needs to be goal, rather than procedurally, oriented. complexity context and multiple system-images in a complex system like a development system the user/designer needs a complex and theoretical mental model of the problem domain in order to interact with it, because the problem domain is complex and theoretical. such a model may be independent from the learning or performing of tasks. an interface needs to represent this model; the information on the state of the problem domain the interface gives back to the engineer is critical to the usability and learnability of the interface. knowledge of the state of the problem domain is knowledge that gives context to the task being undertaken. as johnson points out (1989, p.165): "a task can be more precisely identified and defined by taking into account the context in which it is found to occur". the context of a task can be defined from any number of perspectives. johnson (1989, p!68) mentions role, temporal, experiential contexts, but makes no mention of the context provided by a mental model of the problem domain. in a complex system, the inter-object relations that are specific to the problem domain (that make up a context) need to be modelled in both the user's head and the interface (figure 1). if there are a number of system-images provided by an interface, then these images should also be contextualised within the system as a whole. in other words the links between the various system-images should be "visible" from within a particular system-image. if a system develops over time, as in a developmental system, then a temporal or goal context also needs to be provided. the representations of the problem domain, as it develops over time, can be thought of as a series of system-images. a system-image is a representation of a set of objects, with relationships linking those objects. a system-image needs to provide context for the user and to comprehensively describe one aspect of the system. system-images will vary depending on the nature of the problem domain, but in the case of the example studied (see below) these vary depending on: • the stage of development; • the type of model (e.g. logical or physical); • the level of detail. system-images of different model types should describe a mutually exclusive set of relationships. on the other hand, system-images at different stages of development, or different levels of detail, should contain the same set of objects and relationships. as our ultimate aim is to develop a (set of) system representations from these objects and their relationships (an object model) we have derived from the taxonomic substructure, we need to classify objects in terms of the sort of representation to which they belong. difficulties arise in defining the relationship between different system-images, because the objects in the system-image, and the notation with which we refer to those objects, may differ depending on which system-image we are using. if these difficulties are to be overcome, system-images need to provide the following: • context a particular system-image should be able to be contextualised within the system as a whole. in other words, not only should the context of interface objects be contextualised within a system-image, but the links between various system-images should be "visible" from within a particular system-image. • stability there should be a core of objects whose user representation is stable over the development process and common to various images of the system. for shneiderman (1987), the continuous representation of the object of interest is a key property of direct manipulation interfaces. object-oriented analysis suggests a solution to this problem of linking system-images. by first defining the objects in a system, we create greater stability of our system representation than can be achieved through approaches such as functional decomposition. objects that are common to the various system-images should be used as the means of linking the various system-images. ideally, system-images of a network should be like different perspectives of the same network (figure 2). these perspectives would give different views of the same objects, depending on the stage in the development process, the model used, and the level of detail described. system image 3 system image 2 q object relationship figure 2 /* system image 1 different system-image perspectives of common objects in a system relationships between objects will differ depending on system-image. the stability of the objects across the different system-images provides the link between those system-images. the process of interface analysis and design becomes a process of elaboration of the same objects and the relationship between those objects from different perspectives. this linking provides the context of a system-image in relation to other systemimages. johnson (1992, pi82) suggests the identification of representative, central and generic properties of tasks, so that task knowledge is generalised, and thus transferable (learnable) between applications. as stated above, in complex systems that change during the development process, we also need to be concerned with knowledge transfer from the system to the user, so that the user can maintain his or her representation of the problem domain. the identification of task-objects that are visible in different system-images, can provide a basis for the identification of interface objects. analysis and design guidelines in view of the above characterisation of a complex developmental system we can identify the following broad analysis and design guidelines for the interfaces of such systems: (1) the focus of the task analysis and design should be the problem domain rather than the development tools. development tools should be as transparent as possible. (2) problem domain objects used in tasks, should be the initial focus of analysis, rather than functionality or procedures. the identification of interface objects should be on the basis of representativeness, centrality and stability across system-images. the description of the dynamics of the interface should initially be goal, rather than procedurally, oriented 6 meyer (1988) makes this points with regard to the software development process. the same insight seems applicable to the design and implementation in complex domains. (3) the representation of problem domain objects should be: • encapsulated in interface objects. the interface should be a "direct manipulation" so that, as far as possible, the objects identified in the taxonomic knowledge substructure are represented by interface objects. • contextualised in relation to other objects. the relationships between problemdomain objects should be explicitly represented by the interface as objects. • notationally stable over different stages of the development process and across different models of the system. where an object is present in more than one systemimage, such a stable representation will provide the link between different systemimages. a revised version of knowledge analysis given the nature of complex development systems discussed above, we can summarise the modifications to the knowledge analysis phase of kat, in table i (below). problem domain design purpose of task analysis focus of analysis analysis criteria kat simple (no mental model required only knowledge of tool level tasks) unstructured static transfer of knowledge user/tool interface representativeness centrality proposed method complex (mental model required) structured/generified developmental feedback on state of problem domain user/problem domain interface visibility in various system-images stability table i a summary of the modifications to the knowledge analysis phase of kat the revised method can be summarised as follows: (1) identification of task knowledge • identify objects and actions • identify goal sub-structure (2) analysis of knowledge • different representations of the system have to be identified. • objects and actions are analysed for centrality and representativeness, knowledge structures are already likely to be generic. such 10 • analyse objects for visibility in, and stability across, different representations of the system (e.g. control and physical attributes) and at different stages of the development. • analyse the visibility of the attributes of various objects at different stages of the system. (3) develop a tks of taxonomic and goal knowledge, based on the stable objects in the system. linking kat with an object-oriented methodology we have seen how an extended kat can be used as an input for the derivation of interface representations. can the above revised method of knowledge analysis provide input to an object-oriented analysis? there is some morphological similarity between the kat model of a user's knowledge, and the system representation of the object modelling technique (omt) of rumbaughet al (1991). • johnson's definition of the taxonomic substructure as identifying "the object properties and attributes, including class membership, the procedures in which it is commonly used, [and] its relation to other objects and actions..." (johnson 1989, p.168) could be a definition of the object model if applied to problem domain or system representations. omt's object model diagrams provide a useful tool for representing the complexity of objects and relationships in a taxonomic substructure (see figure 4 below). • the procedural substructure has some similarity with the dynamic model in omt (for "learnt procedure for a well-practised task" read "program control"). the major difference is that in kat, procedures are small single executable units (an attempt to maintain psychological validity for the model), whereas a computer system has no such limits on the size of memory chunk thus its executable units can be much larger. in this sense there is some overlap between the dynamic model in omt and the goal substructure in kat. • both the goal substructure in kat and functional model in omt are logical-level descriptions of what is to done. the goal substructure describes the intentions of the user as established by kat (either articulated or observed behaviour), and the functional model describes how the requirements of the system are met, in terms of the transformation of input to output given the constraints7. in the case of task analysis at the problem-domain level, the granularity of the analysis means that we do not deal with the procedural substructure (tasks executed in a single unit). the mapping process for the input of kat into object modelling techniques (omt) can be represented by the following block diagram (figure 3). it should be noted that task analysis is undertaken at a problem-domain and a tool level, producing two different tks knowledge representations. the tks at the problem-domain level is more fundamental and provides the semantic structure to the interface. the tks at the tool level would describe the user knowledge of interaction styles used in the interface. 7 the goal substructure is more important to interface design, than the omt's functional model has proved to be in practice to system design. see graham (1994, pp238-9) ii the more the style matches existing standards, with which the user may be familiar (e.g. windows), the easier the interface will be to learn. the omt's object model notation provides a rich notation for describing the objects in the taxonomic substructure: their properties, attributes, class membership, procedures and relationships to other objects and actions. by using the same notation for both the taxonomic substructure and the object model, what is in the user's head can have a more direct mapping to the system representation. kat user representation omt system representation at ul level tks af problem domainlevel / taynnnmir; n / knowledge n / i \ goal / v x \^ /-tks at tool levef"x j / taynnnmir "\ ,_\ y goal i / \ procedural / 1 object model i functional l model / dynamic model v\ / ,\ / .\i ^ \ h 1 f \ / v interface j interface> objects interface dynamics at problem domain level interface dynamics at tool level figure 3 mapping between the elements of kat and omt while the distinction between tool and problem-domain task analysis may be useful analytically, in the design of the interface artefact these aspects of the interface need to be integrated. further work needs to be done to examine the relationship between the dynamic model of a user interface, and the user's procedural and goal substructures. an example the complex development system on which the task analysis was originally undertaken for t*m this paper was the lonbuilder system developed by echelon corporation for the design and construction of lon intelligent distributed control networks. lonbuilder provides an integrated environment for a set of tools used for node and network software design and implementation. the following example is a simplified version of the analysis done on the lon development system. the example illustrates the results of the main stages of the revised kat task analysis as outlined above. identification of problem-domain objects and goal substructure analysis of the existing system was undertaken to identify objects in the problem domain, and a high level goal knowledge. the problem-domain objects were derived from the 12 existing system and its documentation, and thus were already generified as 'representative' and 'central'. these descriptions have not been reproduced here, but the high-level goal description provided the basis for the identification of different developmental stages, and thus a classification for the different system-images. these images consisted of a physical model and control (logical) model of the system, as well as a series of stages (goals) for a node developer, and a series of stages (goals) for a someone using the system to install and configure a network. classification of objects into visibility in system-images and the identification of stability of objects across system-images once the problem-domain objects and system-images had been identified, the objects were analysed for visibility in, and stability across, system-images using the grid (table ii). object aggregation/ attribute intra nodal objects application node neuron chip i/o circuitry application image actuator sensor router node nw manage node transceiver name self-doc text string physical location function logical address target hardware eeprom ram application buffers network buffers rom (optional) clock and timers application i/o block scheduler watch dog timer inter-nodal objects media network connection lontalk protocol network image network variable explicit message system-image model con trol ^ s s s s s s s s s s s s s s s ph ysl cal / ^ ^ / / s s s s s s s s s s s s s s s node develo 1 ^ ^ ^ / ^ 2 s ^ ^ ^ ^ s 3 s s s s s s iment stage 4 ^ v s s s s s s s s s s 5 s s s s •s s s s s s s s s s s s s s s s s s s s s 6 ^ ^ ^ ^ s s s s s •/ s s s s s s s s s s s s s installation stage 1 s s s s s •/ v s 2 ^ s s s s s s s s •/ s s s s s 3 s s s s s s s 4 s s s s s s s s s 5-7 s s s s s s s s s s s v s s table ii object visibility and stability grid for various system-images 13 it will be noted for the above grid that the application node is the only problem-domain object present in all system-images. if we wish to build an interface where notationally stable objects provide the context between different system-images, in this case the application node needs to have a consistent representation through all system-images. the different system-images would represent an elaboration the relationships between the application nodes, with the level of detail of each image indicated by the visibility, or otherwise, of other objects and attributes. taxonomic substructure / object model an example of the object model notation used to described a lon system is illustrated below. such a diagram could be the product of kat task analysis or object-oriented analysis of the problem domain. the diagram has been simplified to show the objects only (not their attributes or methods). in a complete diagram, the boxes representing objects would also contain the attributes of the object, and the actions (tasks) performed on the object, as shown in the key to the diagram. such a diagrammatic notation allows a much richer description of the objects than a simple listing of object-action pairs. [ lon i network i node loaded into i network i ( i image i ^ 1 connection <> connects to" " e network | [ router | (application | ranagement node node ™de j i ) i ) f\ y f *. v i circuitry in | application | i image l/\ 9 contrafe [ logical | | physical j connection i i media i i i i r ^ 1 network i 1 variable 1 1 1 message actuator /sensor objects object name attributes actions key to symbols relationships rs is a (generic) o has a (aggregate) —• many +1 one or more figure 4 top level taxonomy of problem domain using simplified omt notation 14 each of these objects can then be given a detailed taxonomic description depicting the object's attributes, the actions performed on the object, and the context of the object (superordinate categories, membership of system-images, relationships and so on). deriving interface objects as with the objects themselves, the attributes, actions and relationships of an object can be analysed using a visibility grid similar to the one above. from such a grid we can then derive the characteristics of a representation of the object must have in a particular system-image the appearance of the object. in other words, we can derive the problem-domain-level attributes and behaviours of an interface object. if the process is followed, we should have interface objects that match the user's taxonomic knowledge of the problem domain, and provide a image of the object appropriate to the particular sub-goal of the user (or stage of development from a system view-point). to develop a complete system-image we need to analyse the appearance of all objects, and relationships between those objects, visible at a particular sub-goal stage. the description of the interface given by such a process can be thought of as a series of snapshots of system states each relating to user sub-goal. what we know about the interface at that stage is which objects and relationships are visible, the appearance of those objects in terms of problem-domain attributes, and the problem-domain-level actions we can perform on the objects and relationships. it is important to note that while the analysis may provide an input into the problem-domain content and structure of the interface, we have derived a description of the interface at the problem-domain level rather than a graphical representation of the objects, or their behaviour at a tool level. concluding remarks kat type analysis of knowledge can provide an input to the object-oriented analysis and design process, and the construction of the interface. by developing a taxonomic substructure based on stable objects, and describing it in a notation consistent with objectoriented analysis, we can provide an input to the object model of the system and thus to the design of the interface objects and system-images. it may also provide an input to the dynamic model of the interface. the identification-of-knowledge stage of kat needs to take account of the degree of generification or abstraction already present in the problem domain. it should also differentiate between knowledge of tools and knowledge of the problem domain. to provide a design input with complex developmental systems, the analysis of knowledge stage of kat needs to be extended to take account of • the provision of feedback to the user's mental model; • different subsets of that mental model as represented by system-images; • the stability of objects across those system-images. there is strong compatibility and complementarity between kat and omt. while the focus of these techniques is different, they have the potential to develop a mutually beneficial relationship if combined in the development process. kat can offer ooa more rigour in 15 ensuring user-centredness. omt can offer a framework and notation through which the task analysis using kat can be used in the development of software systems. further development of the revised kat methodology is currently under consideration, involving its application to a particular design. specific work to be considered includes the following: • validating the efficacy of the revised kat methodology by applying it to a real-world interface such as lonbuilder; • further exploring the role of task knowledge in creating the dynamic model of an interface; • exploring the applicability of the method to other problem domains. references card s., moran t., & newell a.(1979) the keystroke level model for user performance time with interactive systems. report ssl-79-1 (march) xerox parc. carroll j. & olson j. (1988) "mental models in human-computer interaction", in helander, m. (ed.), handbook of human-computer interaction, amsterdam: elsevier science publishers pp. 45-65. diaper, d. (1989) task analysis for human-computer interaction, chichester: ellis horwood. graham, i. (1994) object oriented methods, second edition, wokingham: addison wesley. guindon, r., krasner, h, & curtis, b. (1987) "breakdowns and processes during the early activities of software design by professionals", in olson, g.m., sheppard, c. and soloway, e. (eds) empirical studies of programmers, second workshop, norwood, n.j.: ablex, pp 65-82. johnson, p. (1989) "supporting system design by analysing current task knowledge", in diaper, d. (ed.) task analysis for human-computer interaction, chichester: ellis horwood, pp. 160184. johnson, p (1992) human computer interaction psychology, task analysis and software engineering, london: mcgraw-hill book company. khushalani, a., smith, r. & howard, s. (1994) "what happens when designers don't play the rules: towards a model of opportunistic behaviour in design", australian journal of information systems, vol 1, no 2, pp. 13-31. larkin, j. & simon, h. (1987) "why a diagram is (sometimes) worth ten thousand words", cognitive science, vol ll,pp.65-99. meyer, b. (1988), object-oriented software construction, eaglewood cliffs, n.j.: prentice-hall. norman, d.a. (1986) "cognitive engineering" in norman and draper (eds.), user centered design, hillsdale, n.j.: lawrence erlbaum associates. payne s. and green, t. (1989) "task-action grammar: the model and its developments" in diaper, d. (ed.) task analysis for human-computer interaction, chichester: ellis horwood, pp.75105. rumbaugh j., blaha m., pemerlani w., eddy f., & lorensen w. (1991) object-oriented modelling and design, eaglewood cliffs, n.j: prentice-hall. shneiderman, b. (1987) designing the user interface, addison wesley. visser, w. (1991) "the cognitive psychology viewpoint on design: examples from empirical studies", in gero, j. (ed.), artificial intelligence in design, oxford: butterworth-heinemann, pp.505-524. 16 microsoft word ajis_18(3)_v10.docx australasian journal of information systems volume 18 number 3 2014 29 standard business reporting in australia: past, present, and future nena lim curtin university n.lim@curtin.edu.au brian perrin curtin university b.perrin@curtin.edu.au abstract standard business reporting (sbr) is an australian government initiative aimed at enhancing business productivity by reducing compliance costs. this initiative commenced in 2010 and is based on an international reporting language, extensible business reporting language (xbrl). to date, general awareness or knowledge of the program is limited and the adoption rate by businesses is far below the government’s expectations. the objective of this paper is twofold. first, it describes in detail the different components of sbr and provides a clear overview of how the components work together. second, it examines the issue of sbr adoption in australia using the diffusion of innovation (doi) model and the technology, organization, and environment (toe) framework. results show that the most relevant factor in the doi model is relative advantage. compatibility does not appear to be a problem to businesses but complexity potentially is an issue. moreover, enhancing trialability and observability can help promote sbr. in the context of the technological perspective in the toe framework, perceived costs is an obstacle to sbr adoption as insufficient information is available. the quality of sbr reports has hardly been mentioned or discussed but it is an important factor for the long term success of the sbr. the most salient issue in the organizational perspective is the lack of awareness and expertise on sbr and xbrl among business managers and accountants. in the environmental perspective, there is an increasing pressure for sbr adoption but the external support to help businesses appears to be insufficient. the discussions in this paper provide useful information to managers and accountants on how the sbr components work together. the findings also provide specific recommendations to the government agency in charge of sbr to help the further development and adoption of the program. keywords: standard business reporting, sbr, extensible business reporting language, xbrl, australia introduction standard business reporting (sbr) is a deregulated reform under the 2008 council of australian governments (coag) seamless national economic reform (productivity commission, 2012a). based on an international reporting language, extensible business reporting language (xbrl), sbr is an australian government initiative aimed at reducing the business-to-government (b2g) reporting burden for businesses by simplifying the reporting process. based on the principle of ‘recorded once, reported to many’, the sbr initiative aims to achieve savings by reducing overlapping or redundant information in the government reporting process (australian government, 2012). sbr helps businesses and intermediaries such as accountants spend less time in the reporting process as well as minimize errors and delays in submitting government forms. the sbr project was officially begun in july, 2010. it was australasian journal of information systems volume 18 number 3 2014 30 estimated that sbr has 1.5 million prospective users. however, only 1,000 businesses have submitted 12,000 reports to sbr by 2011 (productivity commission, 2012a). even though the number of reports lodged has increased to 148,000 by june 2013, the adoption rate still is far below the target (australian government, 2013). given the huge investments involved and the low adoption rate, department of treasury, the leader of the sbr program has conducted official reviews of the program (power, 2013; productivity commission, 2012a). as a result of the official reports, the australian government currently is contemplating to make sbr mandatory. a consultative document on australian securities and investments commission (asic) submission was issued in late 2012 inviting sbr stakeholders in particular accounting associations to comment on the proposal (australian government, 2012). to date a few studies have examined the reasons behind the low adoption rate of xbrl in australia (azam & taylor, 2012; 2013; doolin & troshani, 2007; troshani & doolin, 2007; troshani & rao, 2007). unfortunately, data in these studies were not current as they were gathered prior to the official start of the sbr program. moreover, as senior management in australia do not seem to have much knowledge of sbr (aig, 2011), and the response rates in two survey studies were only about 10 percent of the top 500 listed companies (azam & taylor, 2012; 2013), it is arguable how useful the results of these studies are for understanding the reasons behind the non-adoption of sbr. because of the financial significance and the potential impacts of this new kind of reporting method on businesses, we believe a comprehensive analysis of sbr development in australia beyond the mandatory or voluntary adoption argument is necessary to highlight important issues that have been overlooked by the existing literature. moreover, we believe one of the major obstacles of sbr adoption is not just a low level of awareness of sbr (productivity commission, 2012a; 2012b) but a lack of comprehensive and easy-to-understand information to help the general public understand how the program works and its potential. hence, using secondary data gathered from the literature and government departments, this paper aims to achieve two objectives. first, it describes in detail the different components of sbr and provides a clear overview to managers and accountants on how the components work together. second, based on the diffusion of innovation (doi) model and the technology, organization, and environment (toe) framework, it examines the sbr from a broad perspective and provides specific recommendations to the government agency in charge of sbr to help the further development and adoption of the program. the remainder of this paper is organized as follows. the next section describes briefly the concept of xbrl. it is followed by a description of the analytical framework and the research methodology employed in this study. the next section then describes how different components of sbr work together. it is followed by an analysis of the sbr adoption in australia. the final section summarizes the findings and recommendations. extensible business reporting language (xbrl) as sbr in australia is based on extensible business reporting language (xbrl), a basic understanding of xbrl is essential if one wants to know how sbr works. being a markup language originated from extensible markup language (xml), the xbrl is a programming language that is independent of any software or platform, and it facilitates the preparation and exchange of business information. the xbrl comprises two basic components: taxonomies and instance documents. an xbrl taxonomy is like a data dictionary and provides a complete list of data elements called tags that describe financial information. without xbrl tags, receivers of information would not be able to understand australasian journal of information systems volume 18 number 3 2014 31 the meaning of data. for example, the following pair of tags in a balance sheet statement means that 5.5 million represents the current assets value: <current_assets>5,500,000</current_assets> both xml and xbrl are extensible because the tags can be increased when necessary. in contrast to the hypertext markup language (html), xbrl focuses on the meaning rather than the presentation of data. similar to html, xbrl documents are simple text files which can be read by any computer. however, one should note that, despite its text format, xbrl documents “are generally only machinereadable” (australian government, 2012 p. 9) because the convoluted codes look awfully complicated to human eyes. a way to solve the problem is to adopt a technology called inline xbrl (ixbrl) that turns xbrl documents into what you see is what you get (wysiwyg) format and makes the documents human-readable. u.k.’s mandatory tax reporting is an example of the use of ixbrl. apart from being the dictionary of all elements, a taxonomy includes other interrelated xml files such as label linkbase, calculation linkbase, reference linkbase, presentation linkbase, and definition linkbase (plumlee & plumlee, 2008). because of the differences in local legislations and accounting standards, specific taxonomies are required for different countries. the second component of xbrl is instance documents. instance documents are xbrl-formatted documents. they are financial reports coded with the taxonomy tags. a balance sheet formatted in xbrl format is an example of an instance document. the syntax and semantics of taxonomies and instance documents are stated in the xbrl specification issued by xbrl.org, an international organization that was established in 1999 to promote the language (xbrl international, n.d.). analytical framework a considerable amount of research has focused on the adoption and implementation of new information technology. the different theoretical adoption models that researchers have used include diffusion of innovation (doi) model (rogers, 1995), technology acceptance model (tam) (davis, 1989), technology, organization, and environment (toe) framework (tornatzky & fleischer, 1990), theory of planned behavior (tpb) (ajzen, 1991), and unified theory of acceptance and use of technology (utaut) (venkatesh et al., 2003). these it adoption or diffusion models can be classified to either individual level or firm level (oliveira & martins, 2011). for example, the toe framework and the doi model have been extensively used to analyze it innovation and adoption problems at the firm level. on the other hand, tam, tpb and utaut models focus on the individual level (jeyaraj et al., 2006). in this research we combine the doi model (rogers, 1995) and the toe framework (tornatzky & fleischer, 1990) to examine and discuss the factors influencing sbr adoption in australia. according to the doi model, five factors significantly affect the rate of adoption of an innovation such as new technologies. those five factors are relative advantage, compatibility, complexity, trialability, and observability. relative advantage refers to the relative benefits of the innovation in relation to the previous practice or technology. compatibility refers to how much the innovation is being seen as consistent with existing value and practice. complexity refers to how easy or difficult the innovation is for one to use. trialability refers to the extent one can experiment with the innovation. observerability refers to the extent one can view the results of the innovation. the toe framework is based on the assumption that three contexts (technological, organizational, and environmental) influence an organization’s decision to adopt and implement a technological innovation. first, the technological context can be broadly described as ‘both the internal and external technologies relevant to the firm’ (tornatzky & fleischer, 1990, p153). this study examines two australasian journal of information systems volume 18 number 3 2014 32 technological characteristics of sbr: perceived costs, and quality. second, the organizational context refers to specific internal characteristics of an organization. this study examines three major organizational characteristics: financial resources, expertise, and top management support. third, the environmental context refers to the external pressure faced and external support received by businesses. this study also examines the role of software developers as a part of the environment of sbr. the toe framework and the doi model are chosen for this study because they are based on the firm level view and are strongly grounded in the literature. for example, the toe framework has been used to examine the adoption of xbrl (troshani & rao, 2007), enterprise resource planning systems (pan & jang, 2008), electronic data interchange (kuan & chau, 2001), e-business (oliveira & martins, 2011), and e-commerce (teo et al., 2008). the doi model also has been used to examine the adoption of personal information systems (kim & ammeter, 2014), enterprise systems (ramdani et al., 2009), and e-commerce (kendall et al., 2001). methodology the adoption of it innovations is a complex issue and is vital to the long term success of both the economy and individual organizations (oliveira & martins, 2011). therefore, it is essential that businesses are responsive to new it innovations and are fully aware of the factors that can impede or facilitate the adoption and diffusion of the technology. yet the level of understanding of sbr is low and information about sbr is sketchy. we believe a meta-analysis of the existing literature would benefit sbr stakeholders by providing a comprehensive overview of the status quo of the program. meta-analysis is a technique where findings from independent studies and secondary data from other sources such as government websites are combined and analyzed (crombie & davies, 2010). according to church (2001), analysis of secondary data is an acceptable methodology when primary data such as survey results and interview transcripts are unavailable. using academic databases and the internet in general, we gathered data from prior studies, official government websites and reports, and so on. data were then analyzed and examined based on the doi model and the toe framework. standard business reporting components although sbr has been described repeatedly in several government documents, those descriptions are rather clumsy and difficult to understand (madden, 2010b; 2011; miller, 2013). as a picture is worth a thousand words, we have summarized the relations among major components of sbr in figure 1. currently, more than 10 government agencies such as the australian taxation office (ato), australian securities and investments commission (asic), eight state and territory revenue offices (sros) are ready to accept sbr-enabled forms (sbr, 2010). to simplify the description, only two participating government agencies are shown in the diagram. before we describe each component of the sbr system, one should note that we have classified the sbr system into two parts: front end and back end because it is important for readers to distinguish which sbr components are directly relevant to system users and which components are operated without users’ direct involvement. australasian journal of information systems volume 18 number 3 2014 33 figure 1: sbr components according to madden (2011), the sbr system has four components: auskey, sbr taxonomy, sbr core services, and various government information systems. all these components are developed and maintained by government agencies. we have added sbr-enabled software as the fifth component even though software development is outside the responsibility of the australian government agencies because under the current design, the sbr system cannot function without such software products. australasian journal of information systems volume 18 number 3 2014 34 on the front end, users need to obtain two components of sbr: auskey and sbr-enabled software. auskey is a single secure log on facility. with an auskey, businesses or their intermediaries such as tax agents no longer have to log on to different government agencies’ online services using multiple ids and passwords. since july 2010 businesses that have a valid australian business number (abn) can register for an auskey digital identity with the australian business register (abr). once registered, businesses are able to log onto a single web portal that provides access to all the participating agencies (madden, 2011). by mid-2013, 800,000 auskeys were in use by 400,000 entities. however, one should be careful when interpreting these figures in relation to sbr adoption because about 40 percent of entities have multiple auskeys (australian government, 2013). moreover, the auskey is an authentication mechanism for both sbr and non-sbr access to government agencies. in addition to the auskey, businesses need to obtain sbr-enabled software to lodge government forms and reports. businesses are expected to check with their own software providers to find out whether their existing software products support sbr. developed by the impact management group (img), interactive accounts manager (iam) is the first sbr-enabled software product that was available to businesses since may 2010. currently, the two most prominent software products are govdirect and govreports, which support sbr for 10 government agencies. to date most software developers provide sbr-enabled software products to businesses by adding sbr functionality to existing accounting software such as quickbooks and myob. based on the number of sbr-enabled software products available for individual forms, we believe the form most commonly submitted to date is the ato form 4195 business activity statement (bas). a comprehensive list of the sbr-enabled software for ato forms, its software provider, and the number of forms supported by the software is shown in appendix 1. the list is sorted chronologically according to the certified date to highlight the pioneers and late comers. as at mid-2012, 128 developers were licensed to develop sbr-enabled software (australian government, 2013). components on the sbr back end are the taxonomy, the core services, and various government information systems (madden, 2010b; 2011). the sbr taxonomy represents the dictionary of definitions and rules about tags. under the current design of the sbr program, the taxonomy is transparent to businesses via the sbr-enabled software they use. to lodge sbr forms or reports, businesses need to adopt the sbr international financial reporting standards (ifrs) au taxonomy, an xbrl taxonomy specific to australia. the taxonomy includes disclosure requirements set by the corporations act 2011, australian accounting standards board (aasb) accounting standards and australian securities exchange (asx) listing rules (sbr, 2012b). the development of the taxonomy involves standardizing items in forms mandated by various agencies. over 33,500 data elements have been standardized and reduced to around 6,600 since 2007 (australian government, 2013). while the sbr board is in charge of standardizing the data elements on government forms, it is the software developers that need to develop appropriate software according to the taxonomy. the sbr core services refers to the functionality provided to end users by the sbr system. it is an invisible interface between businesses and government agencies (miller, 2013). it provides users with the ability to pre-fill validated forms and reduces the lodgment time and recording errors. after forms are submitted via sbr-enabled software, the core services would then pass the information onto appropriate government agencies. the third component on the sbr back end is information systems employed by government agencies to receive forms from users. sbr users do not need to know about these systems. they just need to know the core services works purely as a gateway and data submitted to various government agencies are handled independently like the pre-sbr system. in summary, an australian business obtains an sbr-enabled software product and uses its unique identification key, auskey, to log on to the sbr core services. appendix 2 shows the screen when a australasian journal of information systems volume 18 number 3 2014 35 quickbooks user is ready to submit a bas via sbr. as shown in figure 1, company a can logon only once to submit a business activity statement (bas) to ato and form 388 to the asic. once company a has logged on to the core services, it can complete the forms quickly as forms are prefilled with data extracted from individual government systems. data sent from the sbr-enabled software are xbrl instance documents ready to be sent over the internet to the core services. after receiving forms from company a, the core services will pass the data to information systems at ato and asic for processing. to facilitate the readers, milestones of sbr development in australia are shown in appendix 3. standard business reporting adoption this section analyses the factors influencing sbr adoption based on the doi model and the toe framework. doi: relative advantage first, the primary benefit of sbr is cost savings for businesses when they file reports or forms to government agencies, and cost savings has been the focus of promotion by the australian government. it was estimated the sbr initiative would save australian businesses around a$800 million per year when fully implemented (brands, 2012b; madden, 2011). we argue that an emphasis of total cost savings is not an effective approach in enhancing the relative advantage of sbr as businesses are more interested in how much cost savings they can achieve individually. while the treasury had provided estimated cost savings figures for businesses of different sizes, those figures do not seem attractive. based on the 2006/07 figures from the business case prepared by the treasury, the annual cost savings for small and large businesses were expected to be a$324 and a$4,023 respectively. these figures show that the cost savings for australian businesses is rather small. if one takes into the account the unknown extra costs businesses incur in relation to sbr adoption, the cost savings might be smaller or become negative. hence, it is unsurprising that perceived relative advantage does not affect sbr adoption (azam & taylor, 2013). however, these figures are based on the treasury business case, not a real business. australian businesses would be more willing to take up sbr if empirical data, such as the united technologies corporation (utc) case in the u.s., are available. the utc case provides hard evidence that the total process time of financial reports is reduced by 17 percent after the switch to xbrl reporting (boritz & no, 2009; stantial, 2007). a real case of an australian business with actual cost saving figures would be more effective than reiterating the theoretical cost savings (madden, 2010b; miller, 2013; productivity commission, 2012a). second, two major contributing factors for cost savings are the number of participating government agencies and the number of sbr-enabled forms that agencies are ready to receive via the sbr channel. logically, the more participating government agencies and the more sbr-ready forms, the greater the cost savings. when sbr first began in 2010, 10 government agencies were sbr-ready and currently the treasury is pushing to get more agencies such as australian charities and not-for-profits commission (acnc) on board (miller, 2013). currently, asic has six and each sro has two to three sbr-ready forms. in the last three years, ato and australian prudential regulation authority (apra) have increased the number of sbr-ready forms steadily to more than 30 and 200 respectively (apra, 2013). however, the slow increase of forms due to resource constraints was identified by productivity commission (2012a) to be a reason for the low take-up rate of sbr by businesses. in view of political uncertainty and possible changes in funding priorities in the australian government, we agree with the productivity commission (2012b, p.46) that “it would be beneficial to bed down the existing scope of sbr before extending it to other sectors of the economy.” it would be more beneficial to focus on working with agencies already australasian journal of information systems volume 18 number 3 2014 36 involved in the project and identify forms that could lead to maximum cost savings instead of rushing to extending the sbr program to other agencies. third, apart from highlighting the cost saving for individual businesses, it would be a good idea to emphasize other potential benefits of sbr. for example, sbr reduces the need to rekey data and hence data errors. moreover, it overcomes the problem of incompatible financial report formats such as pdf files and excel files (debreceny & gray, 2001). the adoption of sbr might even improve internal processes and controls that will in turn lead to enhanced corporate governance (ahmadpour, 2011; baldwin & trinkle 2011; callaghan & nehmer, 2009). another major benefit of xbrl reiterated in the literature is its ability to meet the information needs of stakeholders (debreceny & gray, 2001). examples include improving the quality of financial information such as timeliness, reliability and transparency (baldwin et al., 2006; fang, 2011; richards & tower, 2004). shortening the time for companies to provide financial information to regulatory agencies or general investors is one of the objectives of sbr (arnold et al., 2012). yet such timeliness benefit is hardly mentioned in australia’s sbr. promoting the benefit of transparency of financial reports is likely to enhance businesses and general public’s acceptance of sbr (farewell & pinsker, 2005; thomson & iyer, 2011). the use of sbr may even reduce information asymmetry in a stock market context and lead to market efficiency (kim et al., 2012; yoon et al., 2011). sbr is also beneficial to auditing because it enhances the audit trail. it might lead auditors to better risk assessment because more updated information is used (baldwin & trinkle, 2011). chironna & zwikker (2010) argue that with the use of xbrl, auditors can conduct audit test procedures on the population to reduce the audit risk. all these benefits will ultimately help companies comply with government rules and regulations (i.e., sox/clerp9) and reduce the risk of litigation because of errors in reports (alles & gray, 2012). doi: compatibility and complexity based on the results of a survey, azam and taylor (2013) found that compatibility and complexity had no significant influence on sbr adoption, but they could not explain why that was the case. we believe compatibility is not an obstacle for australian businesses adopting sbr because software developers are responsible to provide sbr-enabled software after referring to the latest australian-specific taxonomy. business users just need to make sure they pick sbr-enabled software products that are compatible with their existing systems. similarly, the complexity of the sbr system is hidden at the back end as shown in figure 1. however, we would like to point out that the design of sbr taxonomy is a complex issue that is hidden from businesses and is hardly mentioned in the australian literature. based on the ifrs taxonomy, australia has developed its country-specific taxonomy after taking into account its own legislation and accounting reporting requirements. while evidence from u.s. businesses shows that extensions to preestablished taxonomies are common (bovee et al., 2002), australia has adopted a closed xbrl reporting approach as no industry specific taxonomies or taxonomy extensions are allowed. on the one hand, one can argue that the restriction is good for businesses because the single taxonomy simplifies the adoption process and reduces the complexity of sbr. it avoids the use of inappropriate taxonomy or errors that may arise from taxonomy extensions (boritz & no, 2008; debreceny et al., 2010; zhu & wu, 2011). studies in the u.s. have shown that errors such as creating tags unnecessarily can happen in taxonomy extension (weirich & harrast, 2010). the restrictions help investors and analysts compare financial reports of businesses not only within the same industry but across different industries. on the other hand, such restrictions disallow company or industry specific information to be captured in the xbrl-formatted financial reports. as a result, the quality of financial reports may suffer. australasian journal of information systems volume 18 number 3 2014 37 potentially businesses that have their own proprietary accounting system may need to make adjustments to their existing system and financial reports to suit the taxonomy. currently, acnc aims to align financial reports with the sbr taxonomy, but it is unsure whether the au sbr taxonomy is appropriate for submissions to acnc (acnc, 2011). this is an example of how a one-size-fit-all taxonomy might not be a good idea. before the sbr board considers whether industry specific taxonomies or extension to approved taxonomy should be allowed, we recommend it to investigate how well the current taxonomy is in meeting the reporting needs of australian businesses. for example, are certain tags never used in forms or would certain tags be appropriate for specific industries? we suggest researchers to conduct studies similar to bovee et al. (2002) or bonsón et al. (2009) and examine how closely the current australian taxonomy meet the reporting needs of businesses. after the quality of the existing taxonomy is examined, the sbr board can then consider whether industry specific taxonomies or extension to taxonomies should be allowed. doi: trialability and observability trialability is an issue for sbr. some sbr-enabled software such as lodgeit allows businesses and individuals who have an asukey to test sbr submission using the software. however, one needs to be an existing customer and the number of forms allowed to be submitted is limited. another problem is even if some software is free, businesses often do not know about it. as shown in appendix 1, there are already more than 20 software products for ato forms and the list is growing. we believe it is the responsibility of the developers to promote their sbr-enabled software. perhaps they should let more people in particular accountants try their software. moreover, the sbr board may consider providing temporary auskeys (session keys) to facilitate software trials. the u.s. securities and exchange commission (sec) website (www.sec.gov) allows people to easily access company reports and compare them in different formats. with the black box design of sbr in australia and limited information on costs, observability is also an issue. it is difficult for anyone to see how an xbrl instance document looks like, how sbr works, or any relative advantages brought by sbr. we would recommend the sbr board consider providing sample documents in both xbrl and non-xbrl format for the general public’s perusal. toe technological perspective: perceived costs reducing business compliance costs is the primary objective of sbr. it was estimated the sbr initiative would cost around a$320 million over six years (brands, 2012b; madden, 2011). nevertheless, the cost of sbr to businesses such as development or purchase of sbr-enabled software, quality assurance, training, and software maintenance which might offset the targeted cost savings is hardly mentioned by the australian government agencies (alles & gray, 2012). cost to other groups of stakeholders, such as software developers, accountants, and financial reports users, also have not been mentioned. prior literature provides a good guidance of the potential costs related to xbrl in general (alles & gray 2012; pinsker & li 2008). for example, alles & gray (2012) estimated the various types of costs related to the preparation of xbrl reports over the phasing-in periods in the u.s. in another study, the initial filing of xbrl reports to the sec required over 120 hours and the estimated cost was between $47,000 and $89,000 (weirich & harrast, 2010). yet because of the specific design of sbr, costing estimates for u.s. companies are not applicable in the australian context. one possible explanation for the insignificant correlation between perceived financial cost and sbr adoption in azam & taylor (2012) is that chief financial officers had insufficient knowledge about sbr costing to answer the survey questions. therefore, we believe empirical studies of the costs of sbr in the australian environment are required to provide businesses australasian journal of information systems volume 18 number 3 2014 38 with first-hand information. for example, for those businesses that have adopted sbr, which software product have they adopted and how much did they spend in the first year of adoption? a breakdown of costs would be useful to australian businesses. such information would be particularly useful in persuading businesses if the sbr board decides to keep sbr voluntary for asic. toe technological perspective: quality an important question that seems to be overlooked is how much businesses, government agencies and other stakeholders can trust and rely on the sbr-enabled software and ultimately the submitted forms and reports. the quality of forms and reports submitted through sbr depends largely on the quality of software. prior research in the u.s. repeatedly shows that the tagging process is error-prone and poor quality control reduces the usefulness and acceptance of xbrl financial reports (boritz & no, 2008). it is easy for software developers to use tags incorrectly because of the large number of tags in a taxonomy (weirich & harrast, 2010). other errors include tagging a figure with negative instead of positive value (bartley et al., 2010; debreceny et al., 2010). research has also shown that some xbrl software products provide better data checking than the others (debreceny et al., 2010). yet in australia, sbr-enabled software products are allowed to be self-certified and the issue of quality assurance is hardly mentioned by sbr promoters. the fact that software developers re-issue the same version of sbr-enabled software proves that quality of sbr-enabled software is not guaranteed (sbr, 2013). but how should sbr-enabled software be tested by businesses? perhaps validation software like that described in bortiz & no (2008) need to be considered. future research that examines the quality of sbr-enabled software would be useful to businesses. currently apra is the only agency that specifies the audit requirements of different sbr forms. all other agencies seem to have a trusting attitude toward sbr-enabled software and sbr-enabled forms and have not set specific requirements. such a lax attitude of the participating agencies in australia is surprising given the u.s. experience. under the sec mandatory filing program, 25 percent of approximately 400 companies had errors in their furnished xbrl reports (debreceny et al., 2010) and the quality of xbrl reports do not necessarily improve over time (boritz & no, 2008). such results suggest that mandatory filing or reduced data re-entry does not guarantee data accuracy. hence, measures should be taken to assure that lodged sbr forms are accurate. the sbr board and australian businesses might find validation frameworks proposed by researchers useful (boritz & no, 2011; srivastava & kogan, 2010). moreover, enforcing accountability could help tackling the quality assurance problem. for example, who should be responsible for the quality of financial reports generated from the sbr-enabled software and lodged with the asic? moreover, should businesses be given a grace period to comply with the quality assurance requirements? of course, the level of assurance required or expected from financial reports depends on whether sbr supplements or replaces traditional reporting channels or formats (plumlee & plumlee, 2008). if asic requires business to submit xbrl forms in addition to pdf or paper forms, it needs to state clearly whether chief executive officers (ceos) will be asked to certify the xbrl documents. the level of responsibility of auditors is another vital issue surrounding quality (richards & tower, 2004). in the u.s., the auditing standards board specifies the requirements for auditing xbrl documents (farewell & pinsker, 2005). currently asic forms submitted via sbr are not required to be audited and external auditors have no responsibility for the optional xbrl financial reports (madden, 2011). if auditors have to certify the accuracy of xbrl reports, the key issue is not just what constitutes an error, but what constitutes a material error that would warrant auditors to flag the reports and how sampling should be done (plumlee & plumlee, 2008). given the importance of data quality of xbrl forms, the sbr board should clarify the expected requirements on ceos, auditors, and any other related parties. researchers may want to explore the quality issue surrounding sbr. australasian journal of information systems volume 18 number 3 2014 39 toe organizational perspective: financial resources the availability of financial resources to invest in a new technology is a prerequisite for its success. hence, large businesses are more likely to have the ability to adopt sbr. when the u.s. sec began its phase-in approach of mandatory xbrl reporting, it started from the largest listed companies. it is important to remember that sbr in australia was originally developed to target businesses that already use electronic channels. hence, we believe it would be unreasonable to mandate all businesses to use sbr. toe organizational perspective: expertise another factor that affects sbr adoption is the availability of experts that understand xbrl and sbr. with the successful development of taxonomies, australia does not seem to have an expertise problem similar to that of new zealand (cordery et al., 2011). similarly, the continual development of sbrenabled software products suggests that there is no lack of sbr expertise in software developers. nonetheless, experts are required to help businesses ensure a smooth transition to the use of sbr. the international education practice statement (ieps) 2 information technology for professional accountants describes the general it and it control knowledge and competencies that professional accountants are expected to have to fulfil their roles of manager, evaluator, or designer (iaesb, 2007). although xbrl is included in the ieps2, research has shown that accountants often are ill-trained or even completely ignorant of the new technology and only few universities offer relevant courses or cover the topic in their curriculum (fang, 2011; pinsker, 2003). more effort is definitely required to promote the awareness of sbr and provide appropriate training to bookkeepers, accountants and managers. an increased coverage of xbrl and sbr in the accounting curriculum is necessary. existing accounting information systems textbooks have a limited coverage of xbrl and even less on sbr. we would suggest australian educators to fill this gap and write books on sbr. in the meantime, the basic information of sbr provided in this paper can be used temporarily by relevant courses. toe organizational perspective: top management support azam and taylor (2012) found that top management support is important for sbr adoption in australia. top management is unlikely to support sbr unless they have at least some knowledge of it. yet recent studies show that ceos lack knowledge of sbr (aig, 2011; productivity commission, 2012b). government agencies could consider sending out easy to understand information flyers with the basic information about the new approach of report submission to businesses. we believe our description of sbr components would be useful to the government agencies. toe environmental perspective: external pressure the sbr board has failed to encourage businesses to adopt the reporting system voluntarily. in view of the ‘wait-and-see’ attitude of australian businesses (troshani & rao, 2007 p. 107), the government is shifting gradually toward a mandatory approach. the association of superannuation funds australia (asfa) began mandatory sbr since july 2013. superannuation funds are required to adopt the superstream data and payment standards that are based on sbr (parkinson, 2012). the asfa mandatory sbr reporting will be extended to large and medium-sized companies in 2014 (parkinson, 2012). ato also has decided to transit from its electronic lodgment service (els) to sbr completely in 2015. what has not been decided is the submission requirement to asic. the current submission to asic requires a pdf format report or a hard copy, and xbrl format is optional. in november 2012 a consultative paper was issued to gather feedback on the use of sbr for financial reports lodged with asic (australian government, 2012). australasian journal of information systems volume 18 number 3 2014 40 our response to the consultative paper is that ixbrl is a better choice than xbrl because ixbrl facilitates users of the asic forms. but more information and education on ixbrl to various groups of stakeholders is required. in view of the u.s. experience and the limited availability of sbr-enabled software products for asic (currently only two), we recommend a transition period of at least five years. however, we would suggest the sbr board win businesses over by demonstrating the costs and benefits in a clearer and more demonstrable way instead of focusing on mandating businesses to lodge reports to asic using sbr. toe environmental perspective: external support because of a lack of knowledge of sbr, businesses need external support to help them participate in the program. yet currently the support provided by the government is inadequate. the first place businesses would go to obtain relevant information is probably the sbr official website (www.sbr.gov.au) or websites of participating government agencies. our review of those websites shows that the information for businesses is scattered, limited, inconsistent and sometimes outdated (e.g., information on sbr-enabled software). moreover, information on the sbr official website is mainly for software developers. for example, it provides an implementation checklist and a step-bystep explanation on what software developers need to do to create products that support sbr (sbr, 2012a). yet when business users click on ‘how to get sbr’, all they see is a list of participating agencies and a list of sbr-enabled software. if the sbr board wants to encourage the adoption of sbr by businesses, it needs to improve the sbr website and provide more concrete information such as a better description of the components of sbr. currently the website provides video case studies that describe the potential benefits of sbr. addition of information such as cost saved and cost incurred would be useful. several conferences on sbr have been held in australia in the last few years. an example is the sbr/xbrl international conference held in brisbane in november, 2007. however, the attendance of such conferences was limited. a more effective way to introduce the sbr program to the general public especially the accounting professions is free or low-cost training classes held by regulatory or professional accounting organizations or webinars run by software developers (brands 2012a; weirich & harrast, 2010). currently, such training is limited in australia. we recommend the sbr board cooperate with the professional accounting associations or universities to organize low or no cost workshops on a regular basis. such co-operation should lead to better training and hence benefit australian businesses. another way to support businesses is through publications. except for government publications that promote sbr (madden, 2010a; 2010b; 2011; miller, 2013), to date there are limited publications that are relevant to australia’s sbr (azam & taylor, 2012; 2013; troshani & rao, 2007). publications aim at educating the accounting professionals are available (farewell, 2006; gomaa et al., 2011; mahoney & white, 2007; phillips et al., 2008). for example, gomaa et al. (2011) teach students how to extract xbrl reports from u.s. sec website and compare companies using financial ratio analysis. unfortunately, these publications focus on u.s. sec filing and are inapplicable to the australian context. more australian specific publications are required to help stakeholders understand the program. another source of support for businesses is non-governmental organizations that promote xbrl such as xbrl international. being a branch under xbrl international, xbrl australia has played a minor role in promoting the adoption of xbrl in australia since the initiation of the sbr program in 2010. currently information on its website, www.xbrl.org/au, seems to be outdated (e.g., on taxonomy development), and there are broken links on the website. to help promote sbr, education material on this website should not be restricted to its members but be made available to the general public. australasian journal of information systems volume 18 number 3 2014 41 toe environmental perspective: software developers software developers is another important group of stakeholder whose role is often neglected in the promotion of sbr. with the exception of apra, which allows businesses to download a software tool, direct2apra (d2a), from apra website to enter data directly and submit forms, a key obstacle to the take-up of sbr is the limited availability of sbr-enabled software, particularly for large businesses (miller, 2013). it is a catch-22 situation (doolin & troshani, 2007; productivity commission, 2012a). because of limited resources, software developers are likely to be unwilling to invest in sbr-enabled software unless a critical mass of users exists. yet businesses would not be interested in sbr unless a critical mass of relevant software is available. according to the sbr website, ato forms appear to be well supported by popular accounting packages such as myob and quickbooks. however, software developers of large and complicated accounting systems do not seem to have participated. this is unsurprising as developing sbr-enabled software for large organizations is particularly costly because of the complexity involved. currently two sbr-enabled software products are available for lodging asic reports. moreover, even though asic has six forms ready for sbr, only four forms are supported by software products. businesses are unlikely to be interested in adopting sbr if only several forms can be submitted as the time saved is little. the number of software developers that provide sbr-enabled software for ato has risen sharply in the last several years. however, about half support only one or two forms (appendix 1). we believe that it is pointless mandating sbr unless more forms are supported by existing software. even though several software products do support many sbr forms, how likely would businesses be willing to switch from their current accounting system to another one for the sake of sbr? conclusion and recommendation sbr is an australian government initiative aimed at enhancing business productivity by reducing compliance costs. as outlined in the paper, general awareness or knowledge of the program is limited and the adoption rate by business is far below government’s expectations. to date, the savings for government and australian businesses have been far below the projected $800 million savings. in this study, we fill a gap in the literature by providing a comprehensive description of the different components of sbr and also provide a clear overview of how the components work together. in addition, based on the doi model and the toe framework, we conducted an in-depth meta-analysis of the factors influencing sbr adoption. results show that in the context of sbr, the most relevant factor in the doi model is relative advantage. the current focus of the potential benefits of sbr is on cost savings. yet the cost savings were estimated from an official business case and appeared to be minimal. the potential costs to non-government stakeholders such as businesses are hardly mentioned. we argue that the promotion of cost savings should focus on the individual business and not on the total savings. in addition, we also suggest that other potential benefits of sbr should be emphasized. instead of rushing to extending the program to more agencies, it would be better to increase sbr-forms from those agencies already involved. compatibility does not appear to be a problem to businesses because the responsibility is shifted to software developers. currently, australia has adopted a closed xbrl reporting approach with no industry specific taxonomy or taxonomy extension. the restriction simplifies the adoption process and reduces the complexity of sbr. however, the restriction disallows company or industry specific information to be captured in xbrl, which may ultimately influence the quality and usability of the reports. trialability and observability are also key issues for sbr. with the black box design of sbr in australia, we believe it is the responsibility of the developers to promote sbr-enabled software. we australasian journal of information systems volume 18 number 3 2014 42 also suggest that the sbr board promote trialability by providing temporary auskeys to facilitate software trials. in the context of the technological perspective in the toe framework, perceived costs is an obstacle to sbr adoption as insufficient information is available to businesses. the quality of sbr reports has hardly been mentioned or discussed in the development process but it is an important factor for the long term success of the sbr. the most salient issue in the organizational perspective is the lack of awareness and expertise on sbr and xbrl among business managers and accountants. in the environmental perspective, the sbr board seeks to extend the initiative to more government agencies. moreover, asic is considering making sbr compulsory. currently, external support to help businesses appears to be insufficient. based on the above meta-analysis, we propose the following six specific recommendations to the sbr board for consideration: 1. promotion: use a real case of an australian business with actual cost saving figures to promote the program. promote benefits of sbr other than cost savings: improving corporate governance, meeting the information needs of stakeholders such as investors, enhancing audit trails, and so on. 2. taxonomy: examine how closely the australian taxonomy meets the reporting needs of businesses before making any changes to the current one-size-fits-all approach. 3. quality: develop better measures to validate the quality of sbr software and reports. clarify accountability in the case of misleading reports. 4. expertise: communicate with educators to enhance the training of xbrl and sbr in universities. co-operate with professional accounting bodies and universities to organize low or no cost workshops on sbr. provide temporary auskeys to facilitate trial of sbrenabled software. 5. development: focus on working with agencies already involved in the project and identify forms that could lead to maximum cost savings instead of rushing to extending the program to more agencies. regarding future submissions to asic, allow businesses continue to submit pdf or paper forms or submit ixbrl forms instead. adopt a phase-in admission process like the one in the u.s. 6. support: improve the government websites to provide more useful information to businesses. provide sample documents in both xbrl and non-xbrl formats for the general public’s perusal. develop additional measures such as information flyers to help businesses understand sbr. this research provides a comprehensive analysis of sbr adoption from different perspectives but the analysis is based on secondary data. future studies could gather first-hand data from business managers and accountants on issues such as how closely the current australian taxonomy meets the reporting needs of businesses. empirical data on data quality or the costs of sbr adoption in the australian environment would also be useful. references ahmadpour, a. (2011) “the improvement of governance decision making using xbrl”, international journal of e-business research, vol 7 no 2, pp. 11-18. ajzen, i. (1991) “the theory of planned behaviour”, organizational behavior and human decision processes, vol 50(2), pp. 179-211. australasian journal of information systems volume 18 number 3 2014 43 alles, m. g. & gray, g. l. (2012) “a relative cost framework of demand for external assurance of xbrl filings”, journal of information systems, vol 26 no 1, pp. 103-126. arnold, v., bedard, j. c., phillips, j. r., & sutton, s. g. (2012) “the impact of tagging qualitative financial information on investor decision making: implications for xbrl”, international journal of accounting information systems, vol 13 no 1, pp. 2-20. australian charities and not-for-profits commission (acnc) (2011) australian charities and notfor-profits commission: implementation design, acnc discussion paper. australian industry group (aig) (2011) national ceo survey: business regulation, september. http://www.aigroup.com.au/portal/binary/com.epicentric.contentmanagement.servlet.contentd eliveryservlet/live_content/publications/reports/2011/10259_ceo_survey_web.pdf accessed 10 may 2013. australian government (2012) options paper: use of standard business reporting (sbr) for financial reports. http://www.treasury.gov.au/ accessed 10 january 2013. australian government (2013) report of the australian business registrar 2012-13. https://abr.gov.au/ accessed 25 february 2014. australian prudential regulation authority (apra) (2013) plain english taxonomy (pet). http://apra.gov.au/sbr-pet/ accessed 20 june 2013. azam, s. & taylor, d. (2012) “the impact of organizational factors on sbr adoption in australia”. in proceedings of the accounting and finance association of australia and new zealand (afaanz) 1-3 july, melbourne, australia, 2012. azam, s. & taylor, d. (2013) “adopting standard business reporting in australia: are cfos persuaded by technology attributes?”, malaysian accounting review, vol 12 no 1, pp. 27-54. baldwin, a. a., brown, c. e., & trinkle, b. s. (2006) “xbrl: an impacts framework and research challenge”, journal of emerging technologies in accounting, vol 3(1), pp. 97-116. baldwin, a. a. & trinkle, b. s. (2011) “the impact of xbrl: a delphi investigation”, the international journal of digital accounting research, vol 11(17), pp. 1-24. bartley, j., chen, y. s. a., & taylor, e. (2010) “avoiding common errors of xbrl implementation”, journal of accountancy, february, pp. 46-51. bonsón, e., cortijo, v., & escobar, t. (2009) “towards the global adoption of xbrl using international financial reporting standards (ifrs)”, international journal of accounting information systems, vol 10(1), pp. 46-60. boritz, j. e. & no, w. g. (2008) “the sec's xbrl voluntary filing program on edgar: a case for quality assurance”, current issues in auditing, vol 2 no 2, pp. a36-a50. boritz, j. e. & no, w. g. (2009) “assurance on xbrl-related documents: the case of united technologies corporation “, journal of information systems, vol 23 no 2, pp. 49-78. boritz, e. & no, w. g. (2011) computer-assisted functions for auditing xbrl-related documents working paper, pp. 1-48. bovee, m., ettredge, m. i., srivastava, r. p., & vasarhelyi, m. a. (2002) “does the year 2000 xbrl taxonomy accommodate current business financial-reporting practice?” journal of information systems, vol 16 no 2, pp. 165-182. brands, k. (2012a) “get ready to catch the xbrl wave!” strategic finance, vol 93 no 9, pp. 64-65. brands, k. (2012b) “global xbrl reporting update”, strategic finance, vol 93 no 11, pp. 64-65. callaghan, j. & nehmer, r. (2009) “financial and governance characteristics of voluntary xbrl adopters in the united states”, international journal of disclosure & governance, vol 6 no 4, pp. 321-335. australasian journal of information systems volume 18 number 3 2014 44 chironna, j. & zwikker, e. (2010) “xbrl for grc”, strategic finance, vol 92 no 1, pp. 44-47. church, r. m. (2001) “the effective use of secondary data” learning and motivation, vol 33(1), 3245. cordery, c. j., fowler, c. j., & mustafa, k. (2011) “a solution looking for a problem: factors associated with the non-adoption of xbrl”, pacific accounting review, vol 23 no 1, pp. 69-88. crombie, i. k. & davies, h. t. (2010) “what is meta-analysis?” what is…? series, http://www.whatisseries.co.uk/whatis/pdfs/what_is_meta_analy.pdf, accessed 2 february, 2014. davis, f. d. (1989) “perceived usefulness, perceived ease of use, and user acceptance of information technology”, mis quarterly, vol 13 no 3, pp. 319-340. debreceny, r., farewell, s., piechocki, m., felden, c., & gräning, a. (2010) “does it add up? early evidence on the data quality of xbrl filings to the sec”, journal of accounting and public policy, vol 29 no 3, pp. 296-306. debreceny, r. & gray, g. l. (2001) “the production and use of semantically rich accounting reports on the internet: xml and xbrl”, international journal of accounting information systems, vol 2(1), pp. 47-74. doolin, b. & troshani, i. (2007) “organizational adoption of xbrl”, electronic markets, vol 17 no 3, pp. 199-209. fang, j. (2011) “why is the u.s. xbrl conversion process so slow?” the cpa journal, january, pp. 6-10. farewell, s. m. (2006) “an introduction to xbrl through the use of research and technical assignments”, journal of information systems, vol 20 no 1, pp. 161-185. farewell, s. & pinsker, r. (2005) “xbrl and financial information assurance services”, the cpa journal, may 2005, pp. 68-69. gomaa, m. i., markelevich, a., & shaw, l. (2011) “introducing xbrl through a financial statement analysis project”, journal of accounting education, vol 29(2-3), pp. 153-173. international accounting education standards board (iaesb). (2007). international education practice statement (ieps) 2: information technology for professional accountants. new york: international federation of accountants (ifac). jeyaraj, a., rottman, j. w., & lacity, m. c. (2006) “a review of the predictors, linkages, and biases in it innovation adoption research”, journal of information technology, vol 21(1), pp. 1-23. kendall, j. d., tung, l. l., chua, k. h., hong, c., ng., d., & tan, s. m. (2001). “receptivity of singapore’s smes to electronic commerce adoption”, journal of strategic information systems, vol 10 no 3, pp. 223-242. kim, d. & ammeter, t. (2014). “predicting personal information system adoption using an integrated diffusion model”, information & management, vol 51 no 4, pp. 451-464. kim, j. w., lim, j., & no, w. g. (2012) “the effect of first wave mandatory xbrl reporting across the financial information environment”, journal of information systems, vol 26 no 1, pp. 127153. kuan, k. k. y. & chau, p. y. k. (2001) “a perception-based model for edi adoption in small businesses using a technology-organization-environment framework” information & management, vol 38(8), pp. 507-521. madden, p. (2010a) “cut compliance costs”, cio, august 2010, p. 19. madden, p. (2010b) “standard business reporting an idea whose time starts now”, in proceedings of the 2010 australian regulatory reforms conference, sydney, nsw, pp. 1-18. australasian journal of information systems volume 18 number 3 2014 45 madden, p. (2011) “greater accountability, less red tape: the australian standard business reporting experience”, international journal of e-business research, vol 7 no 2, pp. 1-10. mahoney, l. s. & white, c. jr. (2007) “creating xbrl instance documents in excel”, the cpa journal, july, pp. 66-71. miller, g. (2013) “standard business reporting – improving business reporting now and into the future”, keeping good companies, march, issue 2, pp. 70-74. oliveira, t. & martins, m. f. (2011) “literature review of information technology adoption models at firm level” the electronic journal information systems evaluation, vol 14 no 1, pp. 110-121. pan, m. & jang, w. (2008) “determinants of the adoption of enterprise resource planning within the technology-organization-environment framework: taiwan’s communications industry”, the journal of computer information systems, vol 48 no 3, pp. 94-102. parkinson, m. (2012) “super sbr.” superfunds, july, pp. 16-18. phillips, m. e., bahmanziari, t. e., & colvard, r. g. (2008) “six steps to xbrl” journal of accountancy, february, pp. 34-37. pinsker, r. (2003) “xbrl awareness in auditing; a sleeping giant?” managerial auditing journal, vol 18 no 9, pp. 732-736. pinsker, r. & li, s. (2008) “costs and benefits of xbrl adoption: early evidence”, communications of the acm, vol 51 no 3, pp. 47-50. plumlee, r. d. & plumlee, m. a. (2008) “assurance on xbrl for financial reporting”, accounting horizons, vol 22 no 3, pp. 353-368. power, b. (2013) “a new wave of sbr change”, charter, march, pp. 32-33. productivity commission. (2012a) impacts of coag reforms: business regulation and vet, research report, volume 2 – business regulation. report, april 2012. productivity commission. (2012b) coag’s regulatory and competition reform agenda: a high level assessment of the gains, report, june 2012. ramdani, b., kawalek, p. & lorenzo, o. (2009) “predicting smes’ adoption of enterprise systems”, journal of enterprise information management, vol 22 no 1/2, pp. 10-24. richards, j. & tower, g. (2004) “progress on xbrl from an australian perspective”, australian accounting review, vol 14 no 1, pp. 81-88. rogers, e. m. (1995) diffusion of innovations, 2nd ed., thousand oaks, new york, free press. sbr (2010) forms in scope. www.sbr.gov.au/about-sbr/what-is-sbr/forms-in-scope accessed 18 october 2012. sbr (2012a) building sbr into my application. www.sbr.gov.au/software-developers/enabling-sbr-inmy-application accessed 18 october 2012. sbr (2012b) ifrs au taxonomy 2012 guide, production release, pp. 1-41. sbr (2013) sbr products register. http://www.sbr.gov.au/sbr-products-register/ accessed 24 june 2013. srivastava, r. p. & kogan, a. (2010) “assurance on xbrl instance document: a conceptual framework of assertions”, international journal of accounting information systems, vol 11 no 3, pp. 261-273. stantial, j. (2007) “roi on xbrl”, journal of accountancy, june, pp. 32-35. teo, t. s. h., ranganathan, c., & dhaliwal, j. (2008) “key dimensions of inhibitors for the deployment of web-based business-to-business electronic commerce”, ieee transactions on engineering management, vol 53 no 3,pp. 395-411. australasian journal of information systems volume 18 number 3 2014 46 thomson, j. c. & iyer, u. (2011) “xbrl and erm: increasing organizational effectiveness”, strategic finance, vol 92 no 11, pp. 64-69. tornatzky, l. g. & fleischer, m. (1990) processes of technological innovation, lexington books, lexington, massachusetts. troshani, i. & doolin, b. (2007) “innovation diffusion: a stakeholder and social network view”, european journal of innovation management, vol 10 no 2, pp. 176-200. troshani, i. & rao, s. (2007) “drivers and inhibitors to xbrl adoption: a qualitative approach to build a theory in under-researched areas”, international journal of e-business research, vol 3 no 4, pp. 98-111. venkatesh, v., thong, j. y. l., & xu, x. (2003) “consumer acceptance and use of information technology: extending the unified theory of acceptance and use of technology” mis quarterly, vol 36 no 1, pp. 157-178. weirich, t. r. & harrast, s. a. (2010) “xbrl filings: what have we learned?” the journal of corporate accounting & finance, july/august, pp. 35-39. xbrl international (n.d.) about xbrl international http://www.xbrl.org/aboutxbrl, accessed 24 june, 2013. yoon, h., zo, h., & ciganek, a. p. (2011) “does xbrl adoption reduce information symmetry”, journal of business research, vol 64(2), pp. 157-163. zhu, h. & wu, h. (2011) “interoperability of xbrl financial statements in the u.s.”, international journal of e-business research, vol 7 no 2, pp. 19-33. australasian journal of information systems volume 18 number 3 2014 47 glossary aasb: australian accounting standards board abn: australian business number abr: australian business register abs: australian bureau of statistics acnc: australian charities and not-for-profits commission acra: accounting and corporate regulatory authority apra: australian prudential regulation authority asb: auditing standards board asfa: association of superannuation funds australia asic: australian securities and investments commission asx: australian securities exchange ato: australian taxation office bas: business activity statement b2b: business-to-business b2g: business-to-government ceo: chief executive officer clerp: corporate law economic reform program coag: council of australian governments doi: diffusion of innovation d2a: directtoapra els: electronic lodgment service fbt: fringe benefit tax hmrc: her majesty revenue & customers html: hypertext markup language iam: interactive accounts manager icaa: institute of chartered accountants in australia ieps: international education practice statement ifrs: international financial reporting standards img: impact management group itsa: insolvency and trustee service australia ixbrl: inline-xbrl australasian journal of information systems volume 18 number 3 2014 48 payg: pay as you go pet: plain english taxonomy sbr: standard business reporting sec: securities and exchange commission sox: sarbanes-oxley act sros: state and territory revenue offices tfn: tax file number tam: technology acceptance model toe framework: technology,organization, and environment framework tpb: theory of planned behavior utaut: unified theory of acceptance and use of technology utc: united technologies corporation wysiwyg: what you see is what you get xbrl: extensible business reporting language xml: extensible markup language australasian journal of information systems volume 18 number 3 2014 49 appendix 1: sbr-enabled software for ato (sbr, 2013) software product certified since software provider no. of forms @ 24/6/2013 interactive accounts manager (iam) 13/05/2010 impact management group 7 govdirect 25/05/2010 govdirect (deloitte digital & business driven systems) 7 technology one financials 05/08/2010 technologyone 1 technology one hr and payroll 05/08/2010 technologyone 3 nominal accounting 13/08/2010 nominal small business accounting software 3 reckon govconnect 13/09/2010 reckon limited 6 onesource e-filer 18/09/2010 thomson reuters (professional) australia 7 lodge it 20/09/2010 lodge it 25 craftsman 06/10/2010 ocs software 2 biz administrator 16/04/2011 rapid business systems 2 govreports 28/04/2011 impact management group 26 free accounting software 30/05/2011 free accounting 11 globalbake 01/12/2011 globalbake 3 class super 21/12/2011 super ip 1 etax sbr forms 05/03/2012 etax accountants 2 cashflow manager 08/03/2012 cashflow manager 8 supermate 16/04/2012 supersorp technology 1 sbr assistant 26/03/2013 software assistant 8 muli project risk, accounts & process management 10/04/2013 muli management 1 fbt return form preparer 23/04/2013 national tax and accountants’ association 1 xero business 11/06/2013 xero australia 1 bluedoor superstream 14/06/2013 dst bluedoor 1 mercertransaction 18/06/2013 mercer outsourcing (australia) 1 my super solution 18/06/2013 administration partners 1 smartstream 03/06/2013 synchronised software 1 tranzact super systems (tss) 20/06/2013 tranzact financial services 1 australasian journal of information systems volume 18 number 3 2014 50 appendix 2: lodgment with australian taxation office using sbr australasian journal of information systems volume 18 number 3 2014 51 appendix 3: sbr milestones year australia overseas 2002 apra became the first australian entity to adopt xbrl reporting 2003 2004 netherland sbr began 2005 u.s. sec voluntary reporting began 2006 australian regulation taskforce report recommended sbr u.k. decided to make ixbrl tax reporting mandatory 2007 australia sbr development began by the treasury (aug.) mandatory reporting to acra in singapore began 2008 coag endorsed and supported sbr (jul.) 2009 u.s. sec mandatory reporting phase 1 began (jun.) 2010 australia sbr became available (jul.) auskey became available (jul.) u.s. sec mandatory reporting phase 2 began (jun.) 2011 sbr is extended to apra sbr taxonomy xbrl 2.1 was released (jul.) u.k. ixbrl mandatory tax reporting began (apr.) u.s. sec mandatory reporting phase 3 began (jun.) 2012 evaluation of sbr in productivity commission research report (apr.) sbr ifrs au taxonomy was released (jul.) consultative paper: use of sbr for financial reports was released (nov.) mandatory tax filing began in the netherlands ifrs taxonomy 2012 and xbrl taxonomy for sustainability reporting were released (mar.) xbrl climate change reporting taxonomy was released (nov.) 2013 consultative paper – call for comments closed (mar.) mandatory sbr reporting to asfa began (jul.) auskey is available to acnc (jul.) 2014 sbr is extended to acnc. asfa mandatory sbr reporting is extended to large and medium-sized companies (jul.) mandatory filing of financial statements begins in the netherlands 2015 ato aims to switch from els to sbr (jul.) microsoft word vol 15 2 final.doc australasian journal of information systems volume 15 number 2, 2009 23 a citation analysis of australian information systems researchers: towards a new era? roger clarke xamax consultancy pty ltd email: roger.clarke@xamax.com.au abstract citation analysis is a potentially valuable means of assessing the contributions of researchers, in information systems (is) as in other disciplines. in particular, a combination of raw counts and deeper analysis of citation data can deliver insights into the impact of a researcher's publications on other researchers. despite this potential, the limited literature in the is discipline has paid very little attention to the use of citation analysis for this purpose. meanwhile, the federal department responsible for education funding has convinced successive federal governments to develop research quality measures that can be used as a basis for differential funding. the howard government's proposed research quality framework (rqf) has been abandoned, but a number of aspects of it survive within the rudd government's excellence in research for australia (era) initiative. the era also appears likely to involve a highly formalised process whereby 'research groupings' within individual universities will be evaluated, with (as yet unclear) impacts on the distribution of research funding. funding agencies have an interest in score-keeping, whether or not their enthusiasm is shared by australian researchers. it is therefore highly advisable that australian disciplines, and especially less well-established and powerful disciplines like information systems, achieve a clear understanding of their performance as indicated by the available measurement techniques applied to the available data. this paper reports on citation analysis using data from both the longstanding thomson/isi collection and the more recently developed google scholar service. few australian is researchers have achieved scores of any great significance in the thomson/isi collection, whereas the greater depth available in google scholar provides a more realistic picture. quality assessment of the thomson/isi collection shows it to be seriously inappropriate for relatively new disciplines generally, and for is in particular. yet the howard government had selected the thomson/isi collection as the data-source for citation analysis for the now-abandoned rqf, and the rudd government appears likely to re-select it for the era. it is critical to the ongoing acessibility of research funding by australian is researchers that steps be taken to ensure that decision-makers are made very well aware of the inadequacies of the isi data collection, and that alternative, more appropriate processes for impact assessment be developed. australasian journal of information systems volume 15 number 2, 2009 24 introduction the last two decades have seen dramatic changes in tertiary education. successive federal governments, through the department responsible for education (until the end of 2007, the department of education, science and training dest, now the department of education, employment and workplace relations deewr), have re-badged institutes of technology and colleges of advanced education as universities, strongly encouraged a rapid change from collegial processes to managerialism, reduced funding per student, imposed a range of wasteful administrative processes, forced most institutions into heavy financial dependence on full-feepaying international students, and driven them into commercially risky overseas ventures. during 2005-07, a further major intervention was in development. the basis for distribution of research funding was to be changed to a vastly bureaucratic process. the term 'research quality framework' (rqf) referred to a scheme whereby dest would determine the quality and impact of research undertaken by groups of academics in institutions. the scores arising from that process would in turn be a major factor in the allocation of research funding. the rqf was notionally abandoned by the rudd government which took office at the end of 2007. the alternative scheme currently in development appears very likely, however, to inherit some, and perhaps many, features of the predecessor rqf proposal. assessments of research quality and impact can be based on a wide range of measures. this paper considers one key element the count of citations of a researcher's publications. it commences with a review of the now-dead rqf and the general frame of its replacement, and some of the challenges inherent in such schemes. an overview is then provided of the study that was undertaken, followed by an analysis of the results arising from the two selected citation databases, and of the quality of the collections. the implications for the information systems (is) discipline in australia are argued to be potentially negative, and very seriously so. conclusions are drawn regarding actions that need to be taken. measures of the quality of research in recent years, several countries in australia's reference group have implemented mechanisms for evaluating the impact of individual researchers and research groups. these have generally been an element within a broader activity, particularly relating to the award of block grants to research centres. the key examples have been the u.k. research assessment exercise (rae 2001, 2005), which was undertaken in 1986, 1989, 1992, 1996 and 2001, and the new zealand performancebased research fund (pbrf 2005). the u.k. rae has involved lengthy and bureaucratic specifications of how research groupings and individuals are to fill in forms, including definitions of the publications that can be included, large evaluation panels comprising people from disparate disciplines, and lengthy, rule-bound assessment processes. it has been a very highly resourceintensive activity. it had been suggested that the rae was to be abandoned, on the basis that it has achieved its aims (macleod 2006), but a 2008 process is under way, with only moderate changes from the previous iteration. during 2005-07, the then dest devised an australian research quality framework (rqf), along similar lines, and for similar purposes (dest 2005, 2006a, 2006b, 2007a and 2007b). critical consideration of the rqf's likely impact on is is in fisher et al. (2007). even as it moved towards implementation, many key aspects of the australian rqf remained vague. it sought to measure: australasian journal of information systems volume 15 number 2, 2009 25 • "the quality of research", which included "its intrinsic merit and academic impact"; and • the "broader impact or use of the research", which referred to "the extent to which it is successfully applied in the broader community". the unit of study of the measurement process was to be 'research groupings' within institutions. these were to be decided by each institution, but were to be subject to considerable constraints in terms of disciplinary focus and minimum size. it was envisaged that each research grouping would be assessed against 5-point scales. the outcomes were to be rankings, which were to be used (in a manner that was never explained) in decision-making about the allocation of research-funding. the iteration current at the time of its abandonment by the incoming government at the end of 2007 restricted each researcher to their "four best outputs" during "the six-year assessment period", although a case could be made for inclusion of impacts of "original research conducted during the preceding six years" (dest 2006a, p. 15). this represented a very narrow window into each researcher's product, product quality, and product impact. yet worse, this was coupled with a demand that each discipline nominate a fixed list of a+, a, b and c journals, with no more than 5% of them as a+. this left no scope at all for subtlety or diversity in relation to such matters as subdisciplines, research domains, the emergence of new sub-disciplines and domains, and the progressive re-definition of discipline scope. a further aspect of the scheme that warranted concern was the suggestion that a measure be used comprising "the proportion of [those] publications ... which have citations [i.e. citation-counts] that would put them in the top decile for the discipline" (dest 2006a). such an assessment would have required not only a clear definition of complex questions about the boundaries of disciplines, but also a comprehensive database capable of supporting analysis of all citations of all publications in all disciplines. the architects of the scheme appeared to be unaware of the realities of bibliometrics, or unprepared to permit practicalities to interfere with a political imperative. there was considerable evidence of muddy thinking within the dest documents. during the first two years, attempts were made by academics to achieve some clarity and logic in the scheme, but as the date of implementation was drawing near, their energy was diverted to desperate endeavours to defend the interests of individual universities and disciplines. of particular relevance were the following problems: • it appeared that 'merit' and 'impact on researchers', two fundamentally distinct notions, were to be subsumed into a single measure; • it appeared that "[broader] impact ... in the wider community" was to be conflated with the "significance" of the area in which the work was undertaken; • shortly before implementation, it remained unclear what specific measures were to be used; • measures of 'broader impact' on business, government and the community remained challenging to devise, not least because less formal publications such as the trade press, professional magazines and newsletters, and even some government reports, are sometimes coy and often imprecise in their attribution to sources. the abandoned rqf is to be replaced by an excellence in research for australia (era) initiative. at the date of writing, only two authoritative documents were available, both ministerial media releases (carr 2008a, 2008b). era, "to be developed by the australian research council (arc) in conjunction with the department of innovation, industry, science and research [diisr], will assess research quality using a combination of metrics and expert review by committees comprising australasian journal of information systems volume 15 number 2, 2009 26 experienced, internationally-recognised experts". it is to be "a transparent, workable system to assess the quality of home-grown research". (all quotations in these paragraphs are from carr 2008a). the 'discipline cluster' notion has survived: "era will use leading researchers to evaluate research activity progressively in each of the arc discipline clusters and several clusters covering health and medical research that will be informed by experts from the national health and medical research council (nhmrc)". verbal presentations have indicated that there are to be precisely 6 discipline clusters within the arc and 2 within nh&mrc. this inevitably lumps together virtually all physical sciences into one 'cluster', and similarly all social sciences, all humanities and all creative arts. attitudes to the appropriateness and worth of different categories of publishing venue are contentious even within individual disciplines, and vary enormously between disciplines. unless the era somehow enables differentiation within each 'discipline cluster', there will inevitably be vicious battles over evaluation methods, and inevitably the large, well-established and wellconnected disciplines will win out. the idea that the quality measures will influence the allocation of research resources has also survived: "each cluster report will detail by institution and by discipline those areas that are internationally competitive, together with emerging areas where there are opportunities for development and further investment", and "it will also assist with the government's plans to introduce funding compacts for universities". departmental officers also succeeded in retaining at least some aspects of the prior deliberations about metrics, despite their confused and unsatisfactory nature. this is apparent because "the era will build on work done to date", and "consultation [would continue] with [some?] disciplines about metrics appropriate to their disciplines, noting that some measures will be appropriate to all disciplines and that for all disciplines expert review of the metrics is essential". it remains unclear as to whether the word 'disciplines' is to be taken at face value, or really means 'discipline clusters'. in the meantime, the era was to "start with those disciplines where the metrics were most widely accepted, for example, in the physical and biological sciences". this appears to have the advantage for is of leaving more time for study unless some parts of it are drawn into the 'physical sciences', which is possible because the codes for is are split between 'information and computing sciences' division and the 'business and management' group (arc 2006, anzsrc 2008). the deferral has the corresponding disadvantage that the evaluation framework might be set in stone without is and disciplines with similar characteristics even being involved in the negotiations. schemes such as the rae, pbrf, rqf and the emergent era are political in nature, designed to provide a justification for funds-allocation decisions. moreover, any evaluative regime of this nature is inherently backward-looking. in rewarding prior performance, it overlooks those individuals and teams that are poised to undertake breakthrough research if only they are provided with sufficient resources. the research reported on in this paper, while acknowledging the political dimension, seeks to adopt a rational approach to the question of how to appropriately measure one particular, key aspect of research quality. the 'merit' component of 'research quality' might be measured by a count of publications, and/or a count weighted by the venue in which each publication appeared. this is not a primary focus of the research reported on in this paper, but it was reflected to some extent in the research design. the concept 'impact on researchers' is multi-faceted, and a variety of indicators could be used. for example, re-publication of papers in collections and anthologies, and translations into foreign australasian journal of information systems volume 15 number 2, 2009 27 languages, both indicate that a researcher's work has been both noticed and considered to be worthy. reputation or esteem is indicated by international appointments, prizes and awards, memberships of academies and editorial boards, keynote-speaker invitations, and collaborations with other highlyreputed researchers. a count of citations within journals is an attractive metric for several reasons. it produces quantitative data and hence statistical techniques can be applied. journal publication is mainstream across a great many disciplines, and hence cross-disciplinary comparison may be feasible. many issues arise, however. there is likely to be a preference for citation-counts to be weighted in some manner, depending on the publication-venue of the cited paper and/or of the citing paper. conference proceedings may be ignored, or weighted very low, irrespective of the norms in the discipline or research domain. in addition, the publications impact of some researchers may be tightly focussed, even on a single work, whereas the impact of other researchers may involve the summation of many works. citation-count was to be the primary focus of the rqf, and it appears likely to figure prominently in the era as well. the research reported on in this paper focusses on citation-counts, but places them in context, and considers various measures associated with them, rather than counts alone. overview of the study this study was undertaken within two important contexts. one was the increasing bureaucratic interest in measuring the quality of research, as discussed in the previous section. the other context was a study of the history of the information systems (is) discipline in australia. 'retrospectives' on the discipline were published in clarke (2006 and 2008), the first within a special issue of ajis (2006) and the second, at greater length, in gable & smyth (2008). a full description of the citation analysis method adopted is reported in clarke (2008), which also reports the outcomes of analyses of the citation records of leading and 'middle-ground' academics internationally. briefly, the method involved the following steps: • a set of 34 refereed venues was selected as representing the core of the discipline, with subsets distinguishing a few 'aa' journals, plus 'generalist a', 'specialist a' and 'regional a' journals; • a list of authors active in is in australia from the late 1970s to 2005 was assembled; • sources of citation-data were evaluated, and two selected; • citation-data was extracted from the thomson/isi citation-indices, and subjected to analysis. this is discussed in detail in section 4 below; • citation-date was extracted from google scholar, and subjected to analysis. this is discussed in detail in section 5 below; • anomalies were investigated; • implications were drawn. the remainder of this section discusses two aspects that are specific to this part of the study and hence were not addressed in clarke (2008). australasian journal of information systems volume 15 number 2, 2009 28 the first topic is the manner in which the list of australian is researchers was developed. there are currently about 700 members of the is discipline in australia. in order to compile a list of names, the author drew on his experience in the field since about 1970. this includes the establishment of the australian is academics directory (clarke 1988), and involvement in the subsequent incarnations as the australasian is academics directory (clarke 1991), the asia pacific directory of is researchers (gable & clarke 1994 and 1996), and the (a)isworld faculty directory (1997-). in addition, the broader project of which this was a part included the development of a directory of full is professors in australia (clarke 2007). somewhat arbitrary decisions were taken as to who was an expatriate australian, and how long immigrants needed to be active in australia to be treated as being australian for the purposes of this analysis. data was sought in relation to about 100 leading australian is researchers, plus 4 well-known and successful expatriates. the second aspect of the method relates to the set of refereed venues that was selected in early 2006 as a basis for evaluating the scope of citation databases. as explained in clarke (2008), this reflected the considerable body of publications on publication venues, including the primary international register of is journals (which was established, and is maintained and published, in australia lamp 2005). by the time this paper was completed, it was known that four other relevant projects had been undertaken in parallel with this research, and published after it was completed. firstly, the international body of is researchers, the association for information systems (ais) has addressed concerns within the usa about inappropriate valuations of is publications by deans of graduate schools of business. a 'senior scholars forum' proposed that the ais publish "a 'basket' of journals that the ais deems as 'excellent'", and nominated "misq, isr, jais, jmis, ejis, and isj [plus] jsis and jit" (depending on whether six or eight journals was decided on being appropriate) (ssf 2007). secondly, two papers were published in a special featured theme in ajis 14, 1 (june 2007) focussing on academic journals. fisher et al. (2007) derived a ranking list for is journals relevant to is in australia. this identified 10 a and 19 b journals. another paper (sellito 2007) used data submitted to dest by universities to identify the journals in which australian is researchers published. finally, in late 2007, the australian council of professors and heads of information systems (acphis) released a list of 182 is journals, allocating 9 a+ (limited by rqf rules to 5% of the total), 29 a, 32 b and 112 c-grade journals. comparisons of these later lists with that used in this study demonstrates several differences of detail, which reflect the specific purposes of five quite distinct projects. one aspect of significance is the distinction drawn in the present study among general, specialist and regional journals. there is, however, a sufficiently broad similarity that it appears reasonable to infer that the results of the study would not have been greatly different had any of the other lists of journals developed in those studies been available and been used instead. australasian journal of information systems volume 15 number 2, 2009 29 results of the thomson/isi study thomson scientific now owns a service previously known as the institute for scientific information (isi). isi was established in 1960, and has been widely available since 1988, and on the web since 1997. what is referred to here as thomson/isi comprises three sub-collections marketed under the brand-name 'web of science'. the sub-collections are the science citation index (sci), the social science citation index (ssci) and the arts & humanities citation index (a&hci). the collection is well-known and much-used for bibliometric analyses generally, and citation analyses in particular. sometime in late 2007, dest announced that thomson scientific was to be the data supplier for the rqf. (the only authority that could be located was on a web-page entitled 'rqf 2008 documents and forms' dest 2007b, although a thomson media release of 2 november 2007 also mentioned it). on the basis of comments made in presentations by arc and diisr executives in early 2008, it appears likely that the rudd government will select a single supplier of a citation database to support its era initiative, and that database will be thomson/isi. for these reasons, it was essential that it be utilised as part of this study. method data was extracted from the collection over several days in late january 2006. access was gained through the anu library reverse proxy, by means of thomson's 'web of science' offering. both sets of searches were restricted to 1978-2006, across all three citation indices (sci, ssci and a&hci). multiple name-spellings and initials were checked, and where doubt arose were also cross-checked with the ais elibrary and the (a)isworld faculty directory. following the google study (reported on in the following section), supplementary research was then undertaken within the thomson/isi database in april 2006. during the 3 months between the two rounds of analysis using isi, the database and hence the citation-counts of course continued to accumulate. re-sampling was undertaken in june 2007, in order to provide information about the stability of the data collections, and the rate of change of citation-counts. further experiments were performed, in order to enhance understanding of the quality of the counts. citation-counts for australian is researchers for a small number of australian is researchers moderate counts were found, but the majority of australian is academics barely register on the isi scales. reasons for this are discussed below. an arbitrary cut-off of 100 total citations was applied. (as it happened, none fell between 86 and 166). this resulted in the inclusion in the table of the 4 expatriates and 7 local researchers. column 1 shows the total citation-count, and column 2 the number of papers found. column 3 shows the citation-count for the person's most-cited paper, primarily to provide an indication of the upper bounds on citation-count-per-article. australasian journal of information systems volume 15 number 2, 2009 30 caveat: for reasons that are discussed progressively in this paper and in greater depth in clarke (2008), there are strong arguments for not utilising the data in this table, and for not utilising the isi 'general search', as a basis for assessing the impact of individual researchers or individual articles citation count number of articles largest perarticle count expatriates iris vessey (as i) 601 35 111 rick watson (as rt), since 1989 485 28 78 ted stohr (as ea) 217 12 108 peter weill (as p) , since 2000 178 13 47 locals marcus o'connnor (as m) 354 31 66 ron weber (as r) 328 22 38 philip yetton (as p and pw), since 1975 270 26 57 michael lawrence (as m) 208 27 66 michael vitale (as m and mr), since 1995 179 14 107 ross jeffery (as dr, and as r) 172 28 38 marianne broadbent (as m) 166 24 36 exhibit 1: isi data for leading australian is researchers, january 2006 the thomson/isi service supports several alternative measures. as described in clarke (2008), the 'cited ref search' generates higher scores, because citations within papers in the isi collection are counted even if the cited paper itself is not in the isi collection. although the analysis was problematical, the results do lead to insights of some value and are provided in exhibit 2. the people selected include one of those in exhibit 1, together with several others from the 'middleground' behind the citation-count leaders. the first group of columns provides the same data as exhibit 1, derived using isi's 'general search' function. the second group of columns shows the additional citation-counts disclosed by isi's 'cited ref search'. the last two columns show the sum of the two citation-counts and the expansion factor, i.e. the ratio of the summed citation-counts to the general search citation-count. the data in exhibit 2 enables the following inferences to be drawn: • as expected, the wider scope of the 'cited ref search' generates a count that is in all cases higher than the 'general search'; • the expansion factor varies a great deal, from moderate (an additional 20-30%) to high (3-5 times); • the researchers who benefit most from the alternative measure are those who have publications that appeared in venues outside the limited isi collection, which have significent numbers of citations in articles that fall inside the isi collection. australasian journal of information systems volume 15 number 2, 2009 31 ---in isi database ----not in isi database researcher citationcount articlecount highest citecount citationcount articlecount highest citecount total citations expansion factor iris vessey 622 32 114 186 52 76 808 1.3 philip yetton 278 20 59 65 51 6 343 1.2 peter seddon (p, pb) 81 4 69 67 30 22 148 1.8 graeme shanks (g) 66 10 15 51 32 7 117 1.8 paula swatman (pmc) 43 4 31 102 39 23 145 3.4 roger clarke (r, ra) 41 11 17 176 131 8 217 5.3 guy gable (gg) 35 4 29 73 24 39 108 3.1 exhibit 2: isi general search cf. cited ref search, april 2006 quality assessment a quality assessment of the isi data-holdings was undertaken, reported on at length in clarke (2008), and summarised in appendix 8 of that paper. key conclusions were that: • the isi collection's coverage is seriously deficient, in multiple ways, including nontransparency of venues, many venue-exclusions, and inconsistencies in venue-inclusion; • these deficiencies impact is particularly severely. key examples include the omission of early volumes of 'aa' journals, the severely inadequate coverage of 'generalist a' journals, and the even more inaequate coverage of 'specialist a' and 'regional a' journals, with the result that only about 40% of the core body of is publications appeared to be included when the main body of the research was conducted in 2006; • although the most glaring problems arise from the exclusion of many key venues for quality is research, there are also instances of over-inclusiveness; • even in respect of the journals that are included, the data quality is very patchy, which can result in serious anomalies; • the name-based discovery process suffers from many vagaries; • the nature of the search capabilities is not clear, and the presentation of results is not conducive to accurate research. this is partly a result of the modest quality of the data on which the service depends, but also partly a result of deficient design and implementation. further discussion of the implications arising from this part of the study is provided in section 5.3 below. australasian journal of information systems volume 15 number 2, 2009 32 results of the google study google scholar is an experimental value-adding service provided by the well-known search-engine company. it applies the same search-engine, but to a specialised sub-set of its vast index. although it was introduced in 2004, it was still labelled a 'beta' service in early 2008. from a bibliometric perspective, google scholar is crude, because it is based on brute-force freetext analysis, without recourse to metadata, and without any systematic approach to testing venues for quality before including them. on the other hand, it has the advantages of substantial reach, ready accessibility, and popularity. it is inevitable that it will be used as a basis for citation analysis, and therefore important that it be compared against the more formal isi database. it was selected because it is very different from thomson/isi, it is in a growth-phase, it appears to be the most extensive collection available, it is readily available, and it has attracted widespread attention. method google scholar was searched for the 30 australian researchers who were most highly cited in the thomson/isi collection. these elements of the empirical work were performed during april 2006. the approach adopted was necessarily different from that used for thomson/isi data. the method is discussed in some detail in clarke (2008). re-sampling was undertaken in june 2007, in order to provide information about the stability of the data collections, and the rate of change of citation-counts. it was apparent from instability in the results that changes had occurred during the intervening period in both the google collection and the google service. further experiments were performed, in order to enhance understanding of the quality of the counts. some further re-sampling was conducted in april 2008. citation-counts for australian is researchers the results for the people who qualified for exhibit 1 are shown in exhibit 3. column 1 shows the 'h-index'. this is a measure of recent origin, proposed by hirsch (2005) and enthusiastically adopted in the bibliometrics literature. the h-index is computed by sorting a person's publications in descending order of citations, then counting down them until the publication-count matches or exceeds the citation-count. hence a person with an h-index of 15 has 15 papers with at least 15 citations each. further discussion of the h-index method is provided in clarke (2008). column 2 shows the 'h-count'. this is the total citation-count for the papers that are within each author's 'h-index'. column 3 shows the largest citation-count for a single paper. caveats: 1. this data is based on an experimental service, and a collection of undeclared extent 2. the scores for researchers marked with asterisks may be seriously under-stated due to the non-inclusion of the proceedings of conferences at the technical end of the is discipline australasian journal of information systems volume 15 number 2, 2009 33 h-index h-count largest peritem count expatriates iris vessey 26 1,481 196 rick watson, since 1989 26 1,755 236 ted/ed stohr 15 1,077 689 peter weill, since 2000 21 1,673 268 locals ** marcus o'connnor 11 357 84 ron weber 20 1,469 172 philip yetton, since 1975 16 1,000 488 ** michael lawrence 12 252 42 michael vitale, since 1995 14 1,152 296 ** ross jeffery 18 763 76 marianne broadbent 13 1,002 230 exhibit 3: google data for leading australian is researchers, june 2007 the google scholar data is deeper and more finely-grained than that in isi. a test was therefore undertaken to determine whether meaningful impact measures could be generated from google citation-counts for the next level of researchers. this was done using a purposive sub-sample of 8 australians, all of whom are chaired or chairworthy, but whose isi citation-counts fell between 34 and 70 well below the threshhold of 100 used in exhibit 1. in exhibit 4 below, the comparative data for the 8 researchers is shown, sorted in descending order of isi citation-count. isi-derived data april 2006 google-derived data june 2007 citation count number of articles largest perarticle count hindex h count largest per-item count peter seddon (p, pb) 70 6 60 13 674 220 graeme shanks (g) 61 13 14 13 469 114 paula swatman (pmc) 53 9 29 15 597 165 roger clarke (r, ra) 44 22 16 18 684 113 michael rosemann (m), since 1999 41 14 18 17 618 97 chris sauer (c), until 2000 40 18 13 11 493 180 simpson poon (s), until 2003 37 3 29 12 496 165 guy gable (gg) 34 9 23 12 516 141 exhibit 4: isi/google comparison for 'the middle ground' the isi data in the left-hand section can be seen to be too thin to support any meaningful analysis. the google data in the right-hand section, on the other hand, is much deeper. there are several australasian journal of information systems volume 15 number 2, 2009 34 instances where researchers' h-indices exceed the entire count of articles found on isi. although the data extraction exercises were conducted 18 months apart, isi's collection is stable. for example, in one re-test, the isi citation-count increased by 30% during the period. in contrast, the gap between the two count columns averages around a factor of 20. the google scholar data therefore appears to be capable of more reliable interpretation. the google data gives rise to substantial h-indices which overlap with the measures for the australian leaders in exhibit 3. on the other hand, the google h-counts of between 469 and 684, and the largest per-item counts of between 97 and 220, are both noticeably lower. although ranking within this sample of 'middle ground' researchers would be contentious, the data provides a basis for comparison both with the impact leaders in the main samples above, and with other researchers internationally. re-tests were performed in november 2007 and april 2008. in the google collection, over the period from june 2007 to april 2008, as shown in exhibits 5a and 5b, the h-counts for a microsample of two researchers increased at annualised rates of 70% and 50% respectively. among the leaders, each of whom was by definition building on a larger base, growth rates ranged downwards philip yetton about 50%, iris vessey and rick watson about 35%, michael vitale about 30%, peter weill 20% and ed stohr 10%. in the isi collection, on the other hand, over the period from april 2006 to april 2008, swatman's and clarke's total citation-counts grew at annualised rates of 28% (53 to 82) and 20% (44 to 62), reflecting the smaller number of items in the isi collection and the narrower catchment of citing articles. june 2007 nov 2007 apr 2008 h-index 15 17 17 h-count 597 758 866 largest 165 195 231 exhibit 5a: growth in google scores paula swatman june 2007 nov 2007 apr 2008 h-index 18 19 21 h-count 684 807 890 largest 113 133 145 exhibit 5b: growth in google scores roger clarke quality assessment further experiments were conducted, in order to throw light on the quality of the google scholar collection, but also on that of the thomson/isi data service. the results of the tests on google suggested that it is feasible to specify a procedure to generate meaningful measures from the collection, but that the undertaking was dependent upon a great deal of knowledge about the author's works, and on fuzzy matching. experiments were conducted to test the quality of the isi collection, using google as a benchmark. a sub-set of 7 of the australian researchers was selected, including 3 of the 7 leaders and 4 of the 'middle ground' sample. their top-10 google citation-counts were extracted, and comparison made australasian journal of information systems volume 15 number 2, 2009 35 with the isi results. in each case, careful comparison was necessary, to ensure accurate matching of the articles uncovered by google against those disclosed by isi. the data is shown in exhibits a1 to a7, in appendix a. google finds many more items than isi, and finds many more citations of those items than isi does. in the sample, the isi count includes only 39/70 items, and even for those 39 the total isi citationcount is only 45% of the total google citation-count. to some extent, this is a natural result of the very different approaches the two services adopt: indiscriminate inclusiveness on the one hand, and narrow exclusivity on the other. however, a number of aspects throw serious doubt on the adequacy of isi as a basis on which to assess is academics' research impact: • there are multiple instances of papers that should be in isi but that are not. for example, 4 of iris vessey's articles with a total of 267 google citations appear to be erroneous omissions from the isi database (see exhibit a1); • many papers in 'a-list' journals that are missing from isi have high citation counts in google. for example, among the 70 papers are 7 in jis, tis , it&p and ejis, all of which are excluded from isi's coverage but have a total of 475 citations or an average of 68 each (see exhibits a2, a5, a6 and a7); • the net effect of those two deficiencies is that, of the 7 authors' 10 highest google-count papers, 59% are not counted by isi (41/70). even if books, lower-ranked conferences and government reports are removed from the lists, 45% are omitted (24/53); • if the analysis is narrowed to the authors' top 4 works based on google-citations, 15/28 are missing from isi, and the missing items account for 54% of the combined google citationcount of 2,204; • the difference between isi and google counts varies very substantially between individuals: o the isi counts represent as high as 70% of the google counts (iris vessey and philip yetton) and as low as 38% (ron weber), 35% (peter seddon), 28% (paula swatman) and 17% (roger clarke) a range of 17-70% with a mean of 50%. figures of 25-100% and 40% were reported by meho & yang (2007) for a sample of 25 highly-published researchers in information science / librarianship; o a comparison between the google citation-count for each author's top 10 papers against the total isi count for all of the author's papers is even more stark: iris vessey's partial google count is only 15% higher than her total isi count, but ron weber's and philip yetton's are double, peter seddon's 4 times, paula swatman's 7 times, and roger clarke's and guy gable's between 8 and 9 times. in the study of international is researchers, further, serious and potentially very harmful anomalies in the isi data collection were unearthed in respect of both an article (delone & mclean's seminal 'information systems success: the quest for the dependent variable') and two individuals (clarke 2008). in short, this analysis confirmed the provisional conclusions reached earlier in the paper: google scholar suffers a number of inadequacies as a basis for score-keeping for australian is academics, whereas the thomson/isi service is so grossly inadequate that use of it would be to the serious detriment of the discipline. australasian journal of information systems volume 15 number 2, 2009 36 implications a wide range of inferences from the overall study are identified in clarke (2008). this section focusses on those with direct relevance to the australian context. any form of citation analysis will have defects and undesirable consequences. on the other hand, it is feasible to describe a collection and analysis tool and supporting database that would be of sufficient quality to have value in supporting assessments of individuals' impacts on the research community. for well-established disciplines, particularly in some branches of the sciences and social sciences, the quality of the thomson/isi service may be adequate to support assessments. it is likely to be generally seriously inadequate for relatively new disciplines, for such reasons as the long delay in accepting journals into its collection, its failure to add volumes prior to its adoption date, and its exclusion of refereed conference proceedings. the analysis reported on in this paper and in clarke (2008) shows it to be an extremely poor-quality means of measuring the impact of is researchers generally, and particularly is researchers outside the usa. if thomson/isi general search were imposed as a means of measuring australian is researchers' impact on other researchers, it would disadvantage every australian is researcher, and essentially disqualify from consideration for research funding all but about 7 people, of whom 2 have retired, 1 is employed as a consultant, 1 is a dean, and 1 regards himself as adjacent to is rather than firmly within it. if instead the broader thomson/isi cited ref search were imposed, the field might expand, but by an uncertain amount. the proposal in the now-dead rqf that each researcher would be constrained to their "four best outputs" would create a further serious hurdle. for example, if a threshhold of 100 total citations across each person's "four best outputs" were to be imposed, only a small middleground of australian is researchers would qualify; and if a threshhold of 100 citations for a single paper were imposed, virtually every is academic in australia would be excluded from consideration. in comparison with thomson/isi, google scholar has much greater scope and depth; but to date it lacks in transparency, and in stability. however its advantages already outweigh its disadvantages, at least when compared with the complete inadequacy of thomson/isi, especially for australian is researchers. a rational evaluation of a researcher's 'merit' and 'impact on researchers' would take into account multiple measures, such as the following: • the count of publications in the collection; • the total citation-count for those publications; • the mean citation-count per item; • the citation-count(s) of the highest-scoring item(s); • the full reference(s) for the highest-scoring item(s), in particular the venue and date of publication; • the h-index; australasian journal of information systems volume 15 number 2, 2009 37 • the h-count, i.e. the total citation-count for the publications in the h-index; • re-computed scores based on the above, applying a published weighting scheme that reflects the quality of publishing venues of the person's articles and/or of the articles citing them. on the basis of the research reported on in this paper, a process to generate measures of those kinds from google scholar data appears capable of being implemented. because of the nature and quality of the data, it appears very unlikely that the process could be fully automated. on the other hand, automated tools appear feasible that would significantly reduce the resource requirements to perform, and re-perform the analysis. if reduction to one or two measures is demanded, then the least-worst candidate would appear to be the google scholar h-index and h-count. the rationale underlying this proposition is that the two measures reflect both breadth and depth, rather than over-weighting the researcher's few most meritorious and/or most-heavily-cited papers. an australian management academic has developed a tool to support such analysis, called publish or perish (harzing 2007). if a mechanism of this kind were adopted, the threshholds at the end of 2007 might be perceived to be, for an outstanding australian is researcher, an h-index of 25 and/or an h-count of 750, and for successful australian is researchers, an h-index of 12 or 15 combined with an h-count of 500. because citations are cumulative, and the collections are not entirely stable, any such heuristic requires re-consideration at least annually. conclusions the research reported in this paper was undertaken during the formative phases of the rqf initiative, and the paper was completed after the rqf's demise but before any clarity had emerged about the details of its replacement, the era. some aspects of the conclusions must therefore be expressed in a qualified and contingent manner. the u.k. rae, the n.z. pbrf, and the australian rqf and era are political mechanisms aimed at focussing research funding on a small proportion of research centres within a small proportion of institutions. they are mass-production exercises, and are subject to heavily bureaucratic processes and definitions. citation analysis used in all such processes will be inevitably largely mechanical, with simple rules applied to all disciplines, irrespective of their appropriateness. the is discipline lacks a power-base, and hence it appears unlikely that special treatment can be achieved on such grounds as the significance of refereed conference proceedings, the diversity and rapidity of change in the research domain and hence the substantial proportion of its publications that appear in specialist journals that attract low weightings, and the applied and instrumentalist nature of the discipline and hence the substantial proportion of its publications that appear in unrefereed venues. the analysis undertaken in this paper shows that the use of thomson/isi would be a seriously inappropriate decision in respect of the is discipline, and would result in very few is researchers being recognised as having had a significant impact. this would be very likely to have a negative impact on access by australian is researchers to research funding from deewr and arc sources. several measures were suggested in clarke (2008) that could be taken by the is discipline as a whole, presumably through the association for information systems (ais). exhibit 6 identifies a australasian journal of information systems volume 15 number 2, 2009 38 number of measures that can be taken by australian institutions, in particular the australasian chapter of ais (aais) and the australian council of professors and heads of information systems (acphis). • replicate relevant aspects of the work performed in this study, using the acphis list of publications as a tool in the analysis • support ais in representations to thomson in order to achieve back-loading into isi of large volumes of missing publications • either alone, or in concert with ais, sponsor the development of a search tool that will accept as input a list of published items and generate citation-counts and links to the citing items. this might be readily achieved through enhancement of harzing's public or perish product • maintain and further develop the acphis quality-classification scheme of publishing venues • establish a clear policy regarding measurement of researcher merit and impact • make successive representations to the relevant ministers and agencies (currently diisr and the arc), to working parties and panels within the era framework, and to other relevant bodies, with a view to ensuring an appreciation of the untenability of the use of thomson/isi as a basis for score-keeping, and the seriously negative impact it would inevitably have on access by australian is academics to research funding • make readily available to all relevant parties summaries of the deficiencies of thomson/isi and of the policy regarding measurement of researcher merit and impact exhibit 6: actions needed by aais and/or acphis australasian journal of information systems volume 15 number 2, 2009 39 appendix a: isi cf. google comparisons for selected researchers this appendix provides detailed comparisons of results extracted from both isi and google scholar. seven australian academics were selected, from among both expatriates and local researchers. all but one were selected because of their relatively uncommon names, in order to ease the difficulties of undertaking the searches and thereby achieve reasonable quality in the data. the other (the author of this paper) was selected because, for researchers with common names, full knowledge of the author's publications list makes it much easier to confidently achieve a reasonable degree of accuracy. hence no significance should be inferred from inclusion in or exclusion from this short list. in each of the following tables: • the first column shows the citation-counts for the top 10 works found for the relevant author by google scholar; • the second column shows the count for each of those works as disclosed by isi; • the third column shows the nature of the venue in which the work was published, including the name of the journal in those instances where the work is not indexed by isi; • in the bottom row: o the total isi count shows the sum of the citation-counts for those 10 works. the figure in brackets shows the total for that author as displayed in exhibit 1; o the total google count for the 10 works is shown. the second figure is the total for the sub-set of papers that is also indexed by isi, and is therefore comparable with the first figure in the isi total. google count thomson count venue 145 111 journal 92 unindexed (!!) journal (isr) 88 83 journal 86 unindexed (!!) journal (cacm) 56 26 journal 52 unindexed conference (icis) 52 unindexed journal (ijmms) 48 unindexed (!!) journal (cacm) 41 unindexed (!!) journal (ieee software) 31 9 journal 691 or 320 229 (of 601) totals exhibit a1: isi cf. google iris vessey, january/april 2006 australasian journal of information systems volume 15 number 2, 2009 40 google count thomson count venue 125 38 journal 102 unindexed journal (jis) 106 30 journal 87 36 journal 72 26 journal (commentary) 65 20 journal (commentary) 45 unindexed book 34 unindexed journal (jis) 34 22 journal 31 24 journal 701 or 520 196 (of 328) totals exhibit a2: isi cf. google ron weber, january/april 2006 google count thomson count venue 302 unindexed book 55 11 journal 42 12 journal 32 12 journal 31 34 journal (1988) 27 unindexed book 26 57 journal (1982) 20 23 book 18 6 journal (1985) 18 unindexed government report 571 or 224 155 (of 270) totals exhibit a3: isi cf. google philip yetton, january/april 2006 google count thomson count venue 133 60 journal 47 unindexed journal (cais) 43 unindexed conference (icis) 33 unindexed conference 22 unindexed (!) journal (db, 2002) 24 2 journal (i&m, 1991) 18 2 journal 18 unindexed journal (jis) 13 unindexed conference (ecis) 9 0 journal (jit, editorial) 360 or 184 64 (of 70) totals exhibit a4: isi cf. google peter seddon, january/april 2006 australasian journal of information systems volume 15 number 2, 2009 41 google count thomson count venue 117 29 journal 73 unindexed journal (int'l mkting rev) 61 unindexed journal (tis) 43 unindexed journal (jsis) 29 unindexed (!) journal (ijec) 26 12 journal 26 unindexed journal (jis) 24 6 journal 22 unindexed conference 20 unindexed journal (em) 441 or 167 47 (of 53) totals exhibit a5: isi cf. google paula swatman, january/april 2006 position google count thomson count venue 57 81 14 journal 59 85 16 journal 102 60 unindexed journal (it&p) 148 47 unindexed journal (tis) 253 33 unindexed conference 325 28 unindexed conference 373 25 3 journal 407 23 unindexed journal (jsis) 539 18 unindexed journal 558 17 unindexed conference 417 or 191 33 (of 44) totals exhibit a6: isi cf. google roger clarke, january/april 2006 google count thomson count venue 102 unindexed (!) journal (ejis, 1994) 56 unindexed (!) journal (isf, 2000) 40 unindexed journal (jgim) 27 6 journal (ms) 27 unindexed conference 24 23 journal (i&m, 1991) 23 unindexed conference 14 unindexed conference 13 unindexed conference 10 unindexed conference 336 or 51 29 (of 34) totals exhibit a7: isi cf. google guy gable, january/april 2006 australasian journal of information systems volume 15 number 2, 2009 42 references except where otherwise noted, all urls were most recently access on 16 april 2008. ajis (2006) `ajis featured theme: the information systems discipline in australian universities' australasian journal of information systems 14, 1 (november 2006) 123-140, at http://dl.acs.org.au/index.php/ajis/issue/view/1 anzsrc (2008) '1297.0 australian and new zealand standard research classification' australian bureau of statistics, march 2008, at http://www.abs.gov.au/ausstats/abs@.nsf/latestproducts/1297.0contents12008?opendo cument&tabna arc (2006) 'research fields, courses and disciplines classification (rfcd)' australian research council, undated, apparently of 21 february 2006, withdrawn at the end of 2007 from http://www.arc.gov.au/apply_grants/rfcd_seo_codes.htm, and superseded by anzsrc (2008) cais (2007) 21 (2007) special volume on the is academic discipline in pacific asia in 2006, communications of the ais, 2007 carr k. (2008a) 'new era for research quality: announcement of excellence in research for australia initiative' minister for innovation, industry, science and research, 26 february 2008, at http://www.arc.gov.au/media/releases/media_26feb08.htm carr k. (2008b) 'new classification to assist era development' minister for innovation, industry, science and research, 31 march 2008, at http://www.arc.gov.au/media/releases/media_31march08.htm clarke r. (ed.) (1988) 'australian information systems academics: 1988/89 directory' australian national university, november 1988 clarke r. (ed.) (1991) 'australasian information systems academics: 1991 directory' australian national university, april 1991 clarke r. (2006) 'key aspects of the history of the information systems discipline in australia' australasian journal of information systems 14, 1 (november 2006) 123-140, at http://dl.acs.org.au/index.php/ajis/article/view/12/11 clarke r. (2007) 'a retrospective on the information systems discipline in australia: appendix 4: professors' xamax consultancy pty ltd, march 2007, at http://www.anu.edu.au/people/roger.clarke/sos/aishistapp4.html clarke r. (2008) 'an exploratory study of information systems researcher impact' commun. ais 22, 1 (january 2008), at http://cais.aisnet.org/articles/default.asp?vol=22&art=1, and at http://www.anu.edu.au/people/roger.clarke/sos/cit-cais.html dest (2005) 'research quality framework: assessing the quality and impact of research in australia: final advice on the preferred rqf model' department of education, science & training, december 2005, withdrawn at the end of 2007 from http://www.dest.gov.au/sectors/research_sector/policies_issues_reviews/key_issue/research_ quality_framework/final_advice_on_preferred_rqf_model.htm dest (2006a) 'research quality' department of education, science and training, undated but apparently of mid-2006, withdrawn at the end of 2007 from australasian journal of information systems volume 15 number 2, 2009 43 http://www.dest.gov.au/nr/rdonlyres/26afca18-a4ca-481e-b54f2a83c29528b0/13523/rsearchquality.pdf dest (2006b) 'research quality framework: assessing the quality and impact of research in australia: the recommended rqf' department of education, science and training, october 2006, withdrawn at the end of 2007 from http://www.dest.gov.au/sectors/research_sector/policies_issues_reviews/key_issue/research_ quality_framework/documents/the_recommended_rqf_pdf.htm dest (2007a) 'research quality framework fact sheet' undated, but apparently of may 2007, withdrawn at the end of 2007 from http://www.dest.gov.au/sectors/research_sector/policies_issues_reviews/key_issue/research_ quality_framework/documents/rqf_factsheet_2006/rqf_pdf.htm dest (2007b) 'rqf 2008 documents and forms', department of education, science and training, september 2007, , withdrawn at the end of 2007 from http://www.dest.gov.au/sectors/research_sector/policies_issues_reviews/key_issue/research_ quality_framework/documents_and_forms.htm fisher j., shanks g. & lamp j. (2007) 'a ranking list for information systems journals' australasian j. of infor. syst. 14, 2 (june 2007), at http://dl.acs.org.au/index.php/ajis/article/view/469/417 gable g. & clarke r. (eds.) (1994) 'asia pacific directory of information systems researchers: 1994' national university of singapore, 1994 gable g. & clarke r. (eds.) (1996) 'asia pacific directory of information systems researchers: 1996' national university of singapore, 1996 gable g. & smyth r. (eds.) (2008) 'the information systems discipline in australian universities' forthcoming, anu epress, 2008 harzing a.-w. (2007) 'google scholar a new data source for citation analysis', university of melbourne, 2007, at http://www.harzing.com/pop_gs.htm hirsch j.e. (2005) 'an index to quantify an individual's scientific research output' arxiv:physics/0508025v5, 29 september 2005, at http://arxiv.org/ps_cache/physics/pdf/0508/0508025v5.pdf, accessed july 2007 lamp j. (2005) 'the index of information systems journals', deakin university, version of 16 august 2005, at http://lamp.infosys.deakin.edu.au/journals/index.php macleod d. (2006) 'research exercise to be scrapped' the guardian, 22 march 2006, at http://education.guardian.co.uk/rae/story/0,,1737082,00.html pbrf (2005) 'performance-based research fund', n.z. tertiary education commission, july 2005, at http://www.tec.govt.nz/downloads/a2z_publications/pbrf2006-guidelines.pdf rae (2001) 'a guide to the 2001 research assessment exercise', u.k. department for employment and learning, apparently undated, at http://www.hero.ac.uk/rae/pubs/other/raeguide.pdf rae (2005) 'guidance on submissions' department for employment and learning, rae 03/2005, june 2005, at http://www.rae.ac.uk/pubs/2005/03/rae0305.pdf australasian journal of information systems volume 15 number 2, 2009 44 sellito c. (2007) 'a study of journal publication diversity within the australian information systems sphere' australasian j. of infor. syst. 14, 1 (june 2007), at http://dl.acs.org.au/index.php/ajis/article/view/26/409 ssf (2007) 'letter to the ais president' ais senior scholars forum subcommittee on journals, 23 april 2007, at http://home.aisnet.org/associations/7499/files/senior%20scholars%20letter.pdf acknowledgements the work reported on in this paper was conducted within the context of a major collaborative project on the is discipline, led by guy gable and bob smyth at qut. a report on australia was published in a special issue of the australasian journal of information systems 14, 1 (ajis 2006), and a further report on pacific asia appeared as a special issue in cais (2007). the author acknowledges valuable input from peter seddon of the university of melbourne and colin steele of the anu, and the feedback from the associate editor and reviewers. responsibility for all aspects of the work rests, of course, entirely with the author. author affiliations roger clarke is principal of xamax consultancy pty ltd, canberra. he is also a visiting professor in the cyberspace law & policy centre at the university of n.s.w., a visiting professor in the ecommerce programme at the university of hong kong, and a visiting professor in the department of computer science at the australian national university. paper title (paper title style) australasian journal of information systems xiao, meredith & gao 2017, vol 21, selected papers from acis 2016 managers use of tablets in decision making 1 an exploratory study investigating how and why managers use tablets to support managerial decisionmaking meng xiao decision support systems laboratory faculty of information technology monash university rob meredith decision support systems laboratory faculty of information technology monash university rob.meredith@monash.edu shijia gao decision support systems laboratory faculty of information technology monash university abstract managers are often mobile and a large proportion of their work is dealing with decisions. although many managers currently use tablet computers in their work, there is little research on the use of tablets for managerial decision-support. this exploratory study aims to investigate the ways in which managers use tablets to support their decision-making and the reasons why they do so. using task-technology fit theory, semi-structured interviews were conducted with 20 managers, 17 of whom used tablets for their work-related decision-making. the study reveals managers’ tablet usage patterns in terms of location, tablet applications, decision activities and types. this study has also found that a range of tablet characteristics and decision-task characteristics affect managers’ use of tablets to support decision-making at work. this exploratory study contributes to both academia and industry by providing evidence on the tablet decision-support area, and affording organisations, tablet vendors and tablet application developers informative findings for further improvement in the provision of tabletbased decision support. keywords: managers, technology use, tablets, decision support, mobile decision support, business intelligence, mobile business intelligence, task-technology fit. 1 introduction the popularity of tablet computers grew sharply after apple first introduced the ipad in 2010. while sales of new tablets slowed from 2014 to 2016 (lee et al., 2017), adoption rates among consumers are high: in january 2014, 42% of us adults owned a tablet (zickuhr & rainie, 2014). with the widespread consumer adoption of tablet computers, it is common to see employees, including managers, using these devices in the workplace (willis, 2011). despite their initial status as a consumer device, tablets offer a wide range of business functionality. the business category was the fastest growing category of applications within apple’s ios app store in 2014 (sacco, 2015). managerial users of tablet devices are doing so to read documents and communicate (mayer, winter, stock, & scholl-steinepreis, 2014), and some are pushing their it departments to offer greater access to management support systems such as business intelligence (bi). bi vendors such as tableau, business objects and cognos offer versions of their software for tablets, while developers highlight mobile decision-support as a top issue driven by managerial demand at all levels of the organisation including senior executives (o'donnell, sipsma, & watt, 2012). australasian journal of information systems xiao, meredith & gao 2017, vol 21, selected papers from acis 2016 managers use of tablets in decision making 2 managers are an important group of business users with a large proportion of their work dealing with decisions (simon, 1960). however, there is little research investigating the use of tablets by managers to support managerial decision-making. there are two main research questions being addressed in this paper: 1. in what ways do managers use tablet devices to support their managerial decisionmaking? 2. where tablet devices are used by managers for managerial decision-support, why do they do so? in addition, a sub-question related to research question two was also addressed: 2a. where tablet devices are not used by managers who have access to them for managerial decision-support, why do they not do so? the reason for including the sub-question above is partly opportunistic, as three participants in the study indicated that they no longer, or never have, used their tablets for decisionsupport. understanding why a manager does not use a tablet for decision support when they have the opportunity to do so adds context and further insight into the reasons why managers who do use their tablets for decision-support do so. the project is exploratory in nature and to address the research questions we conducted semistructured interviews with 20 managers from a large government authority, 17 of whom used tablets to support some aspect of their decision-making. we used the task-technology fit (ttf) theory (goodhue & thompson, 1995) to inform the design of the interview protocol and guide the analysis of the data collected. the paper is structured as follows. in the following section, we review the literature on management support using mobile technology. we also offer a brief outline of the ttf theory as it applies to this study. the research design is then presented, followed by results, discussion and a conclusion. 2 background 2.1 mobile decision-support for the purposes of our study, we adopt mintzberg’s (1973) definition of a manager as someone who ensures their organisation achieves its purpose through “planning, organising, staffing, directing, coordinating, reporting, and budgeting” (p.95). as knowledge workers, managers rely on a variety of information sources to inform these activities, including enterprise it systems such as bi and other management support systems. managers also tend to be highly mobile, with studies such as kim (2008) and yuan, archer, connelly, and zheng (2010) suggesting some managers spend between 30%-60% of their time away from their primary work location. it follows that mobile technologies may have a significant role to play in the provision of itbased managerial decision-support. findings from gebauer and shaw (2004) support this premise, showing that managerial users of mobile technologies used mobile applications more often than other categories of employees. mobile technologies are those which are easily transported by their users, in particular those which can be carried on a regular basis regardless of location such as mobile phones / smartphones, tablets and small notebooks. some authors also include laptop computers within this definition, although arguably there is a significant difference between the desktop-like experience of using a full laptop computer, and the more ubiquitous but less functional experience of devices using mobile operating systems such as android and ios. research on the use of mobile technology for management support pre-dates the introduction of tablet computers in 2010 (e.g. cil, alpturk, & yazgan, 2005). however, there is a lack of research on managerial mobile decision-support, with the majority of mobile decision support australasian journal of information systems xiao, meredith & gao 2017, vol 21, selected papers from acis 2016 managers use of tablets in decision making 3 systems (dss) studies focusing on individual consumers and specific industry applications such as health (sneha & varshney, 2009) and agriculture (rose et al., 2016). gao (2013) suggests that more mobile dss research should be conducted on managerial users, especially senior managers. gao’s study also shows a peak in academic papers addressing mobile dss in 2007 followed by a sharp decline leading to a small resurgence in 2012. this suggests that tablet computing might have revived interest in this area. tona and carlsson (2014) also note an increase in the popularity of mobile bi coincided with the introduction of the ipad in 2010, and o'donnell et al. (2012) note that developers’ interest in mobile bi is strongly driven by senior executive demand for support for the ipad platform. mayer et al. (2014) found that tablets are the dominant preferred platform for mobile access to management support systems over smartphones, although they note that it is tablet “functionality” rather than tablet devices themselves that is important. given the potential benefits of mobile decision-support, the importance placed on the topic by decision-support vendors and developers, the demand from managerial users as well as the dominance of tablets as a mobile decision-support platform, it is important to understand managerial tablet use for decision-support. to date there has been little empirical research on how tablets are used by managers for decision-support and why they do so. tona and carlsson (2014) call for more research into the usage of mobile technologies for decision-making, and similarly mayer et al. (2014) argue strongly for future research and development efforts to focus on managerial usage situations and work styles. tona and carlsson (2017) investigate mobile bi use by employees in mobile workplaces including middleand top-level managers (63% of their sample), but do not exclusively focus on managerial use of tablet-based bi tools, with tablet-use making up just 24% of their sample. verkooij and spruit (2013) conducted a literature review of mobile bi research, identifying six themes in the literature (value creation, application deployment, information security, workforce mobilisation, content delivery and device management) but also did not focus on tablet use specifically, or decision-making processes. kautzner, gao, yeoh, and wong (2015) studied managerial use of tablets, but did not specifically investigate their use for decision-support. buchana and naicker (2014), on the other hand, did investigate the use of mobile technologies for organisational decision-making. however, their use of the technology acceptance model (tam) as a theoretical framework for their study means that their results focussed on antecedents leading to an intention to use, rather than a detailed understanding of usage behaviour. while their results showed a positive correlation between behavioural intention to use mobile decision-support and organisational decision-making, a detailed understanding of the nature of mobile technology use for decision-making tasks was not explored. our intent in this paper is to investigate both the ‘how’ (the ways in which) and the ‘why’ (the reasons for) of managers use of tablet computers to support their decision-making processes. 2.2 task-technology fit (ttf) theory given that our study aims to understand how tablets are used by managers to support decisionmaking, not just why they do so, we have adopted the ttf theory (goodhue & thompson, 1995), rather than the tam theory used by buchana and naicker (2014). ttf offers constructs that allow us to investigate characteristics of the user (i.e. managers who are individuals with characteristics different from other non-managerial users), the technology (i.e. tablets that have distinguished characteristics such as mobility and multi-touch interfaces), and the task (i.e. decision making that is a special kind of managerial task). ttf has been used previously in studies of mobile technology (gebauer & shaw, 2004; gebauer, shaw, & gribbins, 2010; lee, cheng, & cheng, 2007), as well as dss (jarapathirun & zahedi, 2007; parkes, 2012). ttf posits that the fit between the technology and the task, and the fit between the individual and the technology, determine performance levels when using a technology. goodhue (1995) argues that “the better fit between technology functionalities, task requirements, and individual abilities will lead to better performance” (p. 1828). in a similar vein, dishaw and strong (1999) argue that it will be used if, and only if, its functions support the user’s task. while this study is not concerned with the two “fits” and managerial decision performance as australasian journal of information systems xiao, meredith & gao 2017, vol 21, selected papers from acis 2016 managers use of tablets in decision making 4 such, ttf does provide us with a theoretical framework to structure our investigation of how and why managers use tablets for decision making by focussing on the following three constructs: i. characteristics of tablet devices (technology characteristics): features of tablet devices that may contribute to a manager using it for decision-support, including physical characteristics such as portability, adaptability, and network connectivity as well as application features such as functionality and user interface design. ii. characteristics of managerial decision tasks (task characteristics): aspects of the decision-making tasks that managers engage in, including ‘structured-ness’, mobility, interdependence, time criticality and the operational, tactical or strategic nature of the decision. iii. characteristics of the managers (individual characteristics): personal characteristics such as demographic attributes and computer expertise, as well as positional characteristics such as managerial level (senior, middle or operational). our purpose for adopting ttf is neither to test the theory, nor to measure the final ttf construct of fit itself. rather, the three constructs from ttf described above, rather than the full set of constructs making up ttf, have been used to inform the design of the interview protocol for the study and guide the data analysis. 3 research design semi-structured interviews were used to collect data because this technique is well suited for exploration of the perceptions and opinions of respondents and it enables probing for more information and clarification of answers (barriball & while, 1994). the interview participants were recruited via the professional links of the researchers. the criteria for inclusion in the study were that the participant: i. must be employed in a managerial role; ii. own or have access through their employer to a tablet device; iii. have the opportunity to use their tablet device for work purposes, if they wish. a total of 20 participants from a business unit in a large government authority in australia took part in the interviews. of these, 17 used tablets to support at least some of their decisionmaking tasks at work. the summarised participants’ demographic information and the tablets they use are shown in table 1. there was a reasonable spread of participant gender, managerial level and experience, but only a single participant from the younger ‘gen y’. however, there was an overwhelming preference among tablet-using participants for apple’s ipad with only one using an alternative device, possibly reflecting consumer preference in the tablet market. gender (n=20) generation managerial level managerial experience tablet male 9 ‘gen y’ 1 senior 4 less than 5 years 6 ipad 16 female 11 ‘gen x’ 10 middle 10 5-15 years 11 ipad mini 2 ‘boomer’ 9 operations 6 greater than 15 years 3 dell windows android 1 1 table 1. participant demographics an interview protocol was developed and organised into three main sections: 1) background of participants – to collect demographic data such as age, managerial level, managerial experience and some basic information about the participant’s current tablet (if relevant); 2) tablet usage for work – to probe how managers use tablets to support their work if at all; and 3) tablet usage for decision making – to investigate how and why managers use tablets to specifically support their work-related decision-making. the latter includes the reasons why australasian journal of information systems xiao, meredith & gao 2017, vol 21, selected papers from acis 2016 managers use of tablets in decision making 5 they use tablets, the types of decisions they make using tablets, and desired improvements (if any) to the tablet so as to better support decision-making. in the case of participants who do not use tablets for work or decision-support, they were asked why this was the case and what, if anything, would need to change for them to do so. most of the interview questions were inspired by or adapted from the prior literature and the theoretical constructs from ttf, as discussed in section 2.2. in-person interviews of approximately half an hour were conducted with each participant. with the permission of the participants, all interviews were audio-recorded. all audio recordings were transcribed and loaded into nvivo, a qualitative data management and analysis software package (neuman, 2011). a thematic coding approach was employed to analyse the data. the coding process started with the first author reading through each transcript and assigning codes according to the ttf-derived constructs as discussed in section 2.2. new codes were also created to represent emerging themes that were found to be common among the research participants. this coding was refereed by the second and third authors. discussions took place where the authors were able to challenge or confirm any questions or issues as they arose. these discussions during the data analysis process improved the consistency and reliability of the coding, and as a result, improved the validity of the findings presented below. 4 results and discussion this section presents the results of the data collection and analysis, divided into three main sub-sections. the first describes the nature of how managers use tablets to support their decision-making at work. the second examines reasons why they do so, while the third describes the reasons why those who don’t use their tablets for decision-support don’t do so. 4.1 managerial use of tablets for decision-making at work the participants exhibited different degrees of tablets usage to support their decision-making. one participant didn’t use a tablet at work at all, while two further participants didn’t use tablets for decision support. of those who did use tablet devices for decision-support (17 participants), the majority spent less than half an hour per day on average doing so, with only three participants spending more than one hour per day using tablets to support their decisionmaking at work. while tablets were used by the participants for various tasks, on average approximately 80% of the time managers spent using tablets for work was for decisionsupport. in relation to decision-making tasks, participant 12 said, “most of my work is dealing with these kinds of activities”. to illustrate how the 17 participants who used tablets to support decision-making at work do so, we discuss four themes: usage location, tablet applications, decision activities, and decision types. 4.1.1 usage location the locations where the participants used tablets for decision-support included meeting rooms, at home, while commuting, offices, public places such as cafes, and hotel rooms. meeting rooms and home were the most common locations. for meetings a tablet was considered to be a more professional work device than a mobile phone and less obtrusive than a laptop. the most common activities supported by tablets in meetings were reading the agenda and looking up information. participant 17 found the tablet “indispensable for finding information and reading.” a number of participants said taking a tablet to meetings was a normal phenomenon within the organisation. participants 13 and 17 said that everyone in meetings had a tablet. tablets were also used at home by managers, most commonly in bed. participant 11 noted, “(when i) wake up, i wonder what i need to do today, so i might pick it up and look at email.” evidence suggests that tablets are mostly used when managers were away from their primary work location (only 4 participants out of 17 noted they use a tablet in their office). in addition, the decisions supported with a tablet tended to involve a high degree of mobility. most managers mentioned when they were able to access their desktop or laptop computers at their australasian journal of information systems xiao, meredith & gao 2017, vol 21, selected papers from acis 2016 managers use of tablets in decision making 6 primary work location, they tended to use those rather than their tablet. network connectivity, in particular, the sim card function, was believed to be an important aspect which assists managers to access information while away from their primary work location. as participant 17 said, “if i go somewhere on weekends, if (clients) want me to check something, i can do it with sim card.” 4.1.2 tablet applications participants used various applications on their tablets to support their decision-making. a summarised list of application types is shown in table 2, with application types grouped into higher-level categories and the number of participants who mentioned using that application type. category application participants (n=17) web-based applications web browsers 17 communication-driven applications email applications video chat applications 15 1 document-driven applications document writers document readers cloud solutions 7 7 2 self-organisation applications calendars task management applications 6 2 analytical-support applications spreadsheets remote access for business intelligence 3 1 other decision-support applications presentation applications graphical modelling applications social network applications 1 1 1 table 2. summary of tablet application types for decision-making the three most common types of tablet applications used by managers to support decision making at work were web-based, communication-driven, and document-driven applications. although there is a wide range of applications available on tablets, only several managers used more than five types to support decision-making. most participants used the default applications shipped by the tablet manufacturer such as built-in web browsers, email applications, and note-taking tools. as participant 7 said, “i don’t ask for anything fancy, i am not very technical, so i don’t try too much with it, just use what is there, i don’t have many applications.” the default applications seem to be adequate for supporting decision activities. participant 17 said “some of the applications are already there, the most useful ones are already there, the main applications.” a closer examination of the relationship between the managerial levels and the number of tablet applications for decision support reveals that senior managers used many more tablet applications for decision-support than middle managers, who in turn used slightly more than operations managers. a number of managers highlighted the capability of cloud-based applications to support decision-making. managers were able to access, share, and store content through cloudenabled applications, keeping the same set of files among multiple devices. managers used desktops, laptops, smartphones and tablets as a ‘managerial technology portfolio.’ cloudbased solutions helped synchronise files within that portfolio consistently. participant 18 said one of the important features of his tablet “is saving documents to the cloud, it is safe, i can print, that is really nice, that is a really nice feature of pages, i can really go to parks, work, no need to bring usb keys, laptops... that is really nice.” participant 10 mentioned his tablet australasian journal of information systems xiao, meredith & gao 2017, vol 21, selected papers from acis 2016 managers use of tablets in decision making 7 has “perfect integration with desktop environment, actually on two levels, i have my stuff integrated both with the desktop environment as well as with my mobile phone, which ensures i can access all my information at any time, and that is the most important thing.” 4.1.3 decision activities mintzberg, raisinghani, and theoret (1976) define a decision as a commitment to action and the decision-making process as a set of actions and dynamic factors that begins with the identification of a stimulus for action and ends with a specific commitment to action. using their phase-model of decision-making, the decision tasks performed by the participants on tablets were classified into four different activities: problem/opportunity identification, information gathering, solution/decision development, and solution/decision selection. all 17 participants indicated that they used their tablet to look for more information to understand a problem or opportunity. the web browser was used in 10 out of 15 examples to support information gathering. other applications either accompanied the web browser or were used alone such as spreadsheets, bi applications, document readers, cloud storage, and email. approximately two-thirds of the participants had used their tablet to identify a problem/opportunity, or to develop a solution to a problem. of the examples that involve problem/opportunity identification, 9 out of 12 began with email or email accompanying video chat. the others began with reading meeting agendas, viewing data in a spreadsheet, and viewing a web page, respectively. in supporting the development of a solution or decision, tablet applications were split fairly evenly, with four examples mentioning email, three web browsers, two document readers, one cloud storage, and one spreadsheet. only around half of the participants mentioned they used their tablet to select a solution or make a final commitment to a course of action. email was used in five examples to support the selection of a solution or choice. for instance, participant 8 said, “in a meeting … i would find some [information] … in email correspondence with other staff that would possibly be a way of solving an issue that i had discovered [during the meeting], and i would use that information to make a decision.” a common pattern was found among the responses. managers all mentioned they used a suite of applications to support the different activities/phases of decision-making. for example, participant 1 said, “email received from (client), want to inquiry about something... i have to search for information by using safari, the web browser on ipad... i have to use... emails... to recall myself of how i solved the previous similar problem...” 4.1.4 decision types to understand the types of decisions the participants were making using tablets, two classic decision typologies from anthony (1965) and simon (1960) were adopted. participants were asked to estimate the percentage of decisions they made that were of each type in the two typologies. according to anthony’s (1965) strategic, tactical/managerial, operational typology, participants estimated 66% were operational decisions, 20% were tactical decisions, and 13% were strategic decisions. of the 17 participants using tablets for decision support, 16 participants used their tablets for operational decisions, 13 for tactical, and 9 for strategic. on the other hand, using simon’s (1960) structured, semi-structured, and unstructured typology, 48% of tablet-supported decisions were structured, 33% were semi-structured, and 18% unstructured. of the 17 participants, all of them used their tablets for structured and semistructured decisions, while 13 used tablets for unstructured decisions. this finding contrasts with er’s (1988) characterisation of traditional pc-based dss users. where traditional dss users apply the system to support less-structured and under-specified problems, tablet users tend to use applications to support more structured and clearly specified problems. after organising the managers based on managerial level, we calculated and summarised the mean of the percentage of each decision type. figure 1 below shows the results of this analysis. the decision types made by senior managers on their tablets split evenly from strategic to australasian journal of information systems xiao, meredith & gao 2017, vol 21, selected papers from acis 2016 managers use of tablets in decision making 8 operational while operations managers and middle managers made more operational decisions than the other two types of decisions. senior managers show a similarly even distribution of structured, unstructured, and semi-structured decisions, while operations managers make many more structured decisions and middle managers slightly more, than the other two types of decisions. figure 1. mean of percentages of anthony’s (1965) (left) and simon’s (1960) (right) decision types sorted by managerial level the figures above indicate that operations managers make more operational and structured decisions on tablets than the other two types of managers while senior managers tend to make more strategic and unstructured decisions on tablets than the other two types of managers. this is consistent with the characterisations of these types of managers by both anthony (1965) and simon (1960). however, in the case of all manager types, tablet-based decision-support was dominated by use for operational / structured decisions. it is unclear whether this dominance is reflective of a genuine need for support for this mix of decision tasks, or whether usage for more strategic/unstructured decision tasks is hampered by a lack of suitable tabletbased decision-support applications. 4.2 reasons for using tablets for decision-making all participants claimed that they were not required to use tablets for decision-making as part of their work routine. the most common keyword managers mentioned in relation to the reasons for using tablets for decision making were ‘portability’ (mentioned by 12 participants), followed by ‘convenience’ (6 participants), and ‘ease of use’ (5 participants). we have summarised and grouped these concepts into two categories: tablet characteristics and decision-task characteristics. 4.2.1 reasons for use: tablet characteristics the characteristics of a technology and the consequent affordances offered to users are key drivers of its adoption. a number of tablet characteristics were identified during the study that contributed to their use for decision-support. these characteristics include portability, usability, network connectivity, and customisability. portability is the most evident inherited tablet characteristic from other mobile devices (kristoffersen & ljungberg, 1999; liang, huang, yeh, & lin, 2007). a number of participants explicitly and implicitly mentioned the size and weight of tablets. compared to laptops, tablets are “smaller, lighter, more portable” (participant 6) than a laptop and desktop. participant 8 said, “the only reason that i use my ipad instead of my laptop is that i can take it with me.” in addition, both participants (6 and 12) whose current tablets were ipad minis, the smallest apple tablet, cited the size of the tablet as one of the most important reasons they used it to australasian journal of information systems xiao, meredith & gao 2017, vol 21, selected papers from acis 2016 managers use of tablets in decision making 9 support decision-making. participant 12 said, “i like to take a small tablet to a meeting, rather than taking a big laptop... i like the size of ipad mini, this is why i use it more.” compared to mobile phones, tablets are larger, providing more space for users to interact. participant 17 mentioned “how i started using tablet is because i read things on iphone initially, but the screen size was not big enough, so i bought an ipad.” this is in line with the argument raised by ozok, benson, chakraborty, and norcio (2008) that tablets gained popularity by taking the mobility of laptop pcs one step further and providing a larger screen than smartphones. mcclard and somers (2000) proposed that applications on tablets should contain features perceived as useful, and comfortable to use. the screen size should be large enough to be usable. evidence supporting each of these features can be found in the participants’ responses. usability (ease of use and usefulness) is one of the tablet’s most cited characteristics. although a number of participants equated usability with portability, others referred to usability in terms of particular tablet applications, screen size and resolution, and the user interface. participant 17 used their tablet for “a lot of reading” and they noted that “the design is very easy to use” in terms of reading documents. participant 10, who uses a graphical modelling tool for decision making, said, “i draw a lot of diagrams, it is extremely simple to draw on tablet.” the screen size on a smartphone is too small to effectively read a document (mentioned by participant 17). however, in landscape orientation, a tablet screen is about as wide as a piece of paper and some participants said that it was suitable. the screen size is also better suited to writing than a smartphone’s. although there were mixed impressions of a tablet’s on-screen keyboard, some managers who compared it to their smartphones were more satisfied with typing on a tablet than on a smartphone. others mentioned the tablet’s resolution, which is about the same screen resolution as a laptop’s, even on a small ipad mini. participant 3 noted “the resolution is good, very clear”. also, the user interface was one aspect the managers found easy to use. in particular, the ability to scroll through documents displayed on the screen using the touch interface provided most managers with a good user experience, similar to the experience of physically touching a piece of paper. it also offered the ability to navigate by finger gestures such as zoom in, zoom out, and screen switching, without relying on a physical input device such as a keyboard or a mouse. participant 14 described using their tablet as “easy to navigate.” all of the participants regarded network connectivity as an important tablet characteristic for decision support. although only participant 8 explicitly mentioned network connectivity as one of their reasons for using a tablet to support decision-making, the concept is present in a number of participants’ answers to other questions. as discussed in section 4.1.2, the top two categories of tablet applications used by almost all managers who use tablets to support decision making are web-based and communication-driven applications. both of these application types require good network connectivity. different forms of network connectivity enable users to access the internet in different ways. wi-fi is the most common. most participants explicitly mentioned that they have access to wi-fi in their workplace and home. a sim card allows user to access the internet through a mobile phone network such as the third/fourth-generation mobile network (3g/4g). however, only six participants made use of the sim function on their tablet despite all of the participants’ tablets having sim card functionality. participant 17 said, “if i go somewhere on weekends, if (clients) want me to check something, i can do it with sim card”. tethering to a smartphone or mobile hotspot device is another way of accessing the internet when there is no wi-fi connection. participant 14 said, “if i am somewhere where i do need connection, i use my 3g smartphone. then i can hotspot to here (the tablet).” although in general the participants agreed that adaptation of the tablet applications to their own needs and use situation is a desirable feature, their perceptions on the importance of such customisation capability of tablet applications is mixed. only participant 8 mentioned it as a reason for using the tablet for decision-making. one explanation for this could be that for the majority of participants, the use of tablets to support decision-making was voluntary. in such a context, a user is likely to have more freedom to customise the tablet and its applications than in a mandatory context. therefore, it might be natural for them to think they can choose the australasian journal of information systems xiao, meredith & gao 2017, vol 21, selected papers from acis 2016 managers use of tablets in decision making 10 applications they need to support their decision-making activities because the “ownership, (of the tablet) is mine” (participant 18). beyond the tablet’s size and lightness, its overall physical design also affects its adoption. many agree with participant 6, who said “it is more personal than a laptop”. this theme became evident when participants talked about their work in meetings. in a meeting where a laptop was being used, other meeting attendees might feel the person was disconnected from the conversation. a tablet is also easier to carry, hold, and keep out of the way: as participant 10 said, “i hate taking a laptop with me to meetings, it sits between you and someone else, and that is obtrusive.” 4.2.2 reasons for use: decision task characteristics apart from the characteristics of the tablet itself, the nature of the decision task also affects the use of the device to support decision-making. participants brought up task mobility, time criticality, and complexity. decision task mobility was the most mentioned characteristic. gebauer et al. (2010) define mobility as the “extent to which a mobile is is used at different geographic locations and while the user is in motion” (p. 261). a number of participants emphasised the need to access the tablet while away from their primary work locations. participant 10 said, “complex decisionmaking does not have a particular place or time, it’s an ongoing process in your brain, and i need to be able to whip out my whole computation infrastructure at any moment at any place.” participant 20 said, “i use a tablet because it means that it is easier to get information on the spot, so it brings the information to the location which the decision is being made.” participant 13 said, “it is a good way of having a lot of document together, in one place, easy for me to carry around, mobility is the important thing, ’cause i just never know where i am going to use the data, ’cause in meetings, you would go to different locations.” in terms of time criticality and task complexity, participant 12 said, “if i provided a quick answer or solution to something, or one that is not too difficult, that i can easily access resources and answer it, then that is the kind of thing that i would do with tablet. if something needs a bit more consideration, then i probably, would prefer to be... better a desktop, have different things open at once, things like that.” similarly, participant 20 mentioned the immediacy of accessing information on a tablet as one of their reasons for making tabletsupported decisions, indicating that their decisions had high “immediacy.” in short, the reason why some managers used a tablet for decision-making was that the decision task was relatively simple and there was a need to deal with the task quickly. 4.3 reasons for not using tablets for decision-making at work all participants in the study owned a tablet or had access to one through their employment, as these were criteria for inclusion in the project. however, of the 20 participants three had made the decision to cease using their tablets for work (participants 2 and 16), or very rarely did so (less than 30 minutes per week) and never to support decision-making (participant 15). this allowed us to explore the reasons why managers might not use a tablet for managerial decisionsupport. this adds context and further understanding to the reasons for why managers do use their tablets for decision-support. despite not using their tablets for decision-support at work, two of these three participants nevertheless are enthusiastic users of mobile devices. participant 16 said “i’m all for using mobile devices” and “once you are exposed to a mobile device, you won’t go back to a desktop.” participant 2 reported using their tablet at home for personal use on average an hour and a half per night for email, browsing the world wide web and gaming. one each of the three was from ‘gen y’, ‘gen x’ and the ‘boomer’ generation, and all had at least two years’ experience in using tablet devices. one was an operational manager, while the other two were middle managers. two were female, one male. participants 15 and 16 ultimately gave their reason for not using tablets to support decisionmaking at work as a lack of perceived value. participant 15 said “[w]hat i know is that [the] australasian journal of information systems xiao, meredith & gao 2017, vol 21, selected papers from acis 2016 managers use of tablets in decision making 11 technology doesn’t support, doesn’t give me any additional value, so that’s why i never use [it].” participant 16 said “it wasn’t as good as having a piece of paper!” and “i just found that whole setup to be too artificial, too cumbersome, and it wasn’t worth it.” participant 2 also hinted at a lack of perceived value, saying that they would use it if the organisation had purchased their tablet: “i’d feel like i was meant to be [using] it if they’re paying for an ipad or whatever, but other than that i don’t feel a need.” interestingly, the reasons cited by participants 2 and 16 for the lack of perceived value largely relate to issues of form factor, portability and usability, cited by other participants as important reasons for why they did use tablets for their work. both participants suggested that because a tablet’s size is larger than a mobile phone but smaller than a laptop, it offered neither significant portability benefits over a mobile phone, nor the usability benefits offered by a laptop computer with a larger screen and physical keyboard. participant 2 said “i find an ipad hard to type on with a keypad…it takes me too long to do things”, and participant 16 “i could not type a document.” participant 16 found it particularly difficult and “cumbersome” to work with large documents, citing one occasion when they ended up printing a hard-copy of a large, complex document because of difficulties in searching, navigating and annotating. participant 2 said that “when i go to meetings at work, i use my phone, which i find easier, if i’m at a meeting i feel more comfortable using this as opposed to having an ipad, because i can probably get away with, you know, they think i’m still listening, and then at [my desk] i obviously have my desktop which i find much easier to use.” participant 16 recognised the task characteristics that lend mobile devices to supporting decision-making discussed above in section 4.2.2: mobility, time criticality and complexity. however, they indicated that for them time critical, mobile and simple tasks were adequately supported with a mobile phone, while more complex tasks, or those less time critical or that could be performed at a desk were far better supported with a laptop computer than a tablet. further, the difficulties in navigating and working with large documents on a tablet added to the level of complexity of the task: “for me to add another complexity with … trying to find information…, it was just too much, too, too difficult.” the nature of a tablet as a device situated between a mobile phone and a full computer meant that for both participant 2 and 16 it had no significant value over either. 5 conclusion the results of the interviews with the 17 participants who use their tablets at work and to support their managerial decision-making highlight the importance of device portability, usability, network connectivity and customisability. however, these are also areas where poor execution of the design of the tablet hardware and software can mean that managers will abandon their usage of the devices, as described in section 4.3. it is interesting to note that apart from information access, the tasks that these three participants referred to when discussing their reasons for not using the devices were not decision-support specific. this suggests that one possible factor in tablet use for decision-support is its value in supporting other work tasks not specifically related to decision-making. tablet devices need to offer a clear value proposition over both laptops and smart phones for managers to use them for decision support. ease of use, connectivity and security are essential, and if not well-executed, users will ignore pressures of task mobility and time criticality where complexity is substantially increased. where the task is simple and not made unnecessarily complex by the use of a tablet, managers will use them for decision-support. of the 17 managers that we interviewed who used their tablets for work, about 80% of their work use dealt with decision activities. the most common locations where managers did so were meeting rooms and at home. neither of these locations are their primary work locations. web-based, communication-driven, and document-driven applications were the most common applications in supporting decision-making. all participants who used their tablets for decision support used them for information-gathering during decision-making. managers tended to use communication driven applications to support problem/opportunity australasian journal of information systems xiao, meredith & gao 2017, vol 21, selected papers from acis 2016 managers use of tablets in decision making 12 identification and solution/decision selection, and used web-based applications to support information gathering. senior managers used more applications than middle and operational managers. structured and operational decisions were the most common types of decisions made by managers on their tablet. also, the results further confirmed that the decision types were in line with the levels of managers, that is, on a tablet, operational managers tended to make more structured and operational decisions while senior managers made more unstructured and strategic decisions. this research has addressed a gap in the area of tablet research, and contributes to the research field of mobile decision support. tablets are still a relatively new type of mobile technology, and the technology has distinct characteristics from other mobile devices (kautzner et al., 2015). this study provides a deeper understanding of the use behaviour of this mobile technology and the reasons behind it. in addition, this study focuses on managers, who are fundamentally different from other employees in organisations. the majority of prior studies on mobile decision support focus on students, general household users, operational workers, and professionals like medical practitioners (gao, 2013). investigating real managers offers potential value as they represent a group of users who are organisationally powerful and can influence technology adoption and use. therefore, by focusing on tablets and managers, and investigating decision making which is an important task, this study fills two research gaps in the literature for mobile technology use. as an exploratory study, this research has laid some of the groundwork for further investigation of tablet-based decision support and business intelligence. in particular, future research should investigate the construct of fit from ttf, looking at the alignment of mobile devices with the type of manager and decision tasks they are engaging in and how this impacts the nature of the decision support technology managers use. of note is the fact that few managers used formal business intelligence applications on their tablets, preferring instead a more ad hoc approach, using standard communication and information management applications like email, pdf readers and the like. this may be due to a lack of fit between mobile business intelligence applications and the decision tasks of managers, or other organisational issues that inhibit access to these applications. investigating this issue can inform mobile business intelligence vendors and application developers as to the kinds of affordances that they should focus on in developing their products. this is an area for future research. a limitation of the study is that only 20 managers were interviewed, all from one business unit of one organisation. the results are therefore suggestive only. that being said, as an exploratory qualitative study that emphasised depth rather than breath, the number of participants exceeds that recommended by neuman (2011). another limitation of this study is the concentration of one specific tablet platform with 18 of the 20 participants using an ipad or ipad mini. only one participant used an android device, while one used a windows device. references anthony, r. n. (1965). planning and control systems: a framework for analysis. cambridge, ma: harvard university press. barriball, k. l., & while, a. e. (1994). collecting data using a semi-structured interview: a discussion paper. journal of advanced nursing, 19(2), 328-335. buchana, y., & naicker, v. (2014). the effect of mobile bi on organisational managerial decision-making. the journal of applied business research, 30(4), 1003-1018. cil, i., alpturk, o., & yazgan, h. r. (2005). a new collaborative system framework based on a multiple perspective approach: inteliteam. decision support systems, 39(4), 619-641. dishaw, m. t., & strong, d. m. (1999). extending the technology acceptance model with tasktechnology fit constructs. information & management, 36(1), 9-21. australasian journal of information systems xiao, meredith & gao 2017, vol 21, selected papers from acis 2016 managers use of tablets in decision making 13 er, m. c. (1988). decision support systems: a summary, problems, and future trends. decision support systems, 4(3), 355-363. gao, s. (2013). mobile decision support systems research: a literature analysis. journal of decision systems, 22(1), 10-27. gebauer, j., & shaw, m. j. (2004). success factors and impacts of mobile business applications: results from a mobile e-procurement study. international journal of electronic commerce, 8(3), 19-41. gebauer, j., shaw, m. j., & gribbins, m. l. (2010). task-technology fit for mobile information systems. journal of information technology, 25(3), 259-272. goodhue, d. l. (1995). understanding user evaluations of information systems. management science, 41(12), 1827-1844. goodhue, d. l., & thompson, r. l. (1995). task-technology fit and individual performance. mis quarterly, 19(2), 213-236. jarapathirun, s., & zahedi, f. (2007). exploring the influence of perceptual factors in the success of web-based spatial dss. decision support systems, 43(3), 933-951. kautzner, e., gao, s., yeoh, w., & wong, s. f. (2015). an exploratory study on the factors influencing managers' use of mobile tablets. paper presented at the pacific asia conference on information systems (pacis) 2015, singapore. kim, s. h. (2008). moderating effects of job relevance and experience on mobile wireless technology acceptance: adoption of a smartphone by individuals. information & management, 45(6), 387-393. kristoffersen, s., & ljungberg, f. (1999). mobile use of it. paper presented at the 22nd information systems reserch seminar in scandinavia, keuruu, finland. lee, c. c., cheng, h. k., & cheng, h. h. (2007). an empirical study of mobile commerce in insurance industry: task–technology fit and individual differences. decision support systems, 43(1), 95-110. lee, p., stewart, d., calugar-pop, c., talbot, e., mandal, s., & tarigoppula, a. k. (2017). technology, media and telecommunications predictions 2017 . deloittetouche tohmatsu limited. retrieved from https://www2.deloitte.com/content/dam/deloitte/global/documents/technologymedia-telecommunications/gx-deloitte-2017-tmt-predictions.pdf liang, t. p., huang, c. w., yeh, y. h., & lin, b. (2007). adoption of mobile technology in business: a fit-viability model. industrial management & data systems, 107(8), 11541169. mayer, j. h., winter, r., stock, d., & scholl-steinepreis, n. (2014). management support systems on different devices a business perspective accommodating managers' growing range of use situations. paper presented at the 47th hawaii international conference on system science. mcclard, a., & somers, p. (2000). unleashed: web tablet integration into the home. paper presented at the sigchi conference on human factors in computing systems, new york, usa. mintzberg, h. (1973). the nature of managerial work. englewood cliffs, usa: prentice-hall. mintzberg, h., raisinghani, d., & theoret, a. (1976). the structure of 'unstructured' decision processes. administrative science quarterly, 21(2), 246-275. neuman, w. l. (2011). social research methods: qualitative and quantitative approaches (7th ed.). boston, usa: pearson/allyn and bacon. australasian journal of information systems xiao, meredith & gao 2017, vol 21, selected papers from acis 2016 managers use of tablets in decision making 14 o'donnell, p., sipsma, s., & watt, c. (2012). the "hot" issues in business intelligence: the view of practitioners. paper presented at the ifip working group 8.3 international conference on decision support systems: fusing decision support systems into the fabric of the context, anavissos, greeve. ozok, a. a., benson, d., chakraborty, j., & norcio, a. f. (2008). a comparative study between tablet and laptop pcs: user satisfaction and preferences. international journal of human-computer interaction, 24(3), 329-352. parkes, a. (2012). the effect of task-individual-technology fit on user attitude and performance: an experimental investigation. decision support systems, 54(2), 9971009. rose, d. c., sutherland, w. j., parker, c., lobley, m., winter, m., morris, c., twining, s., ffoulkes, c., amano, t., & dicks, l. v. (2016). decision support tools for agriculture: towards effective design and delivery. agricultural systems, 149, 165-174. sacco, a. (2015). leading app stores see dramatic increase in business apps. retrieved from http://www.cio.com/article/2868067/mobile-apps/leading-app-stores-see-dramaticincrease-in-business-apps.html simon, h. a. (1960). the new science of management decision. new york, usa: harper. sneha, s., & varshney, u. (2009). enabling ubiquitous patient monitoring: model, decision protocols, opportunities and challenges. decision support systems, 46(3), 606-619. tona, o., & carlsson, s. a. (2014). the impact of the organizing vision on mobile bi adoption. paper presented at the 2014 ifip working group 8.3 international conference on decision support systems: dss 2.0, paris. tona, o., & carlsson, s. a. (2017). enhancing decision-making efficiency through m-bi use. paper presented at the 25th european conference on information systems, guimarães, portugal. http://aisel.aisnet.org/ecis2017_rp/30/ verkooij, k., & spruit, m. (2013). mobile business intelligence: key considerations for implementations projects. journal of computer information systems, 54(1), 23-33. willis, d. a. (2011). ipad and beyond: the media tablet in business. retrieved from https://www.gartner.com/doc/1589917/ipad-media-tablet-business yuan, y., archer, n., connelly, c. e., & zheng, w. (2010). identifying the ideal fit between mobile work and mobile work support. information & management, 47(3), 125-137. zickuhr, k., & rainie, l. (2014). tablet and e-reader ownership. retrieved from http://www.pewinternet.org/2014/01/16/tablet-and-e-reader-ownership/ copyright: © 2017 xiao, meredith & gao. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems xiao, meredith & gao 2017, vol 21, selected papers from acis 2016 managers use of tablets in decision making 15 paper title (paper title style) australasian journal of information systems lane, middleton, zarnekow & alam 2016, vol 20, research on rural and remote ict editorial 1 editorial for the special section on rural and remote ict: broadband rural, regional and national impacts michael lane school of management and enterprise university of southern queensland michael.lane@usq.edu.au catherine middleton ted rodgers school of management ryerson university catherine.middleton@ryerson.ca ruediger zarnekow institut für technologie and management technische universitaet berlin ruediger.zarnekow@tu-berlin.de khorshed alam school of commerce university of southern queensland khorshed.alam@usq.edu.au the rapidly increasing digitalization of businesses and everyday life, combined with increased uptake of information and communication technology (ict) enabled services such as cloud computing, big data analytics and e-commerce, puts high demand on local, national and global network infrastructures. high-speed, affordable and reliable broadband services have become a critical success factor for socio-economic transformation of any economy and society. many broadband initiatives are underway at local and national levels, defining and rolling out future-proof broadband infrastructures needed for a digital age. despite significant progress in the development and deployment of the internet in general and broadband services in particular, many open questions remain on technological, economic and societal levels. what network technologies and infrastructures are best suited for broadband access? how should investments in broadband infrastructures be financed? how can highspeed broadband services be rolled out in the most effective manner for rural and regional communities? and in which ways can society best benefit from broadband infrastructures and services? with this overarching theme of broadband rural regional and national impacts in this special section of the australasian journal of information systems – research on rural and remote ict we want to foster discussion on broadband initiatives from different points of view (technological, economic and societal) and from different scopes (rural, regional and national). several important themes emerged from these four papers for this special section of the australasian journal of information systems. the first paper titled “the supply and use of broadband in rural australia: an explanatory case study of the western downs region” by lane, tiwari and alam looks at the relationship between the supply and use of broadband and how this impacts on household satisfaction with broadband services in rural communities in the western downs region in queensland and it also considers policy implications for delivering high speed broadband in australian rural communities. the second paper titled “improving rural broadband deployment with synergistic effects between multiple fixed infrastructures” by limbach, kuebel and zarnekow looks at a technoeconomic broadband deployment model as an efficient way to roll out broadband infrastructure by making use of existing physical infrastructure and cross-industry cooperation to deliver broadband infrastructure more efficiently for rural communities in germany. australasian journal of information systems lane, middleton, zarnekow & alam 2016, vol 20, research on rural and remote ict editorial 2 the third paper titled “the importance of broadband for socio-economic development: a perspective from rural australia” by freeman, park, middleton and allen focuses on the poor broadband connectivity in rural communities which is exacerbating urban/rural disparities. it finds that current broadband policy frameworks need strategic adaptations to account for the socio-economic contexts of rural communities in order to narrow the current rural-urban digital divide in australia. finally the fourth paper titled “the social surplus of broadband initiatives in compulsory education” by parsons examines the critical role that high speed broadband infrastructure plays in delivery of compulsory education in new zealand. findings indicate that broadband can deliver a social surplus nationally justifying the investment in broadband infrastructure via the more effective provision of broadband based education. these four research papers demonstrate the complexity and multi-disciplinarily of nature of broadband initiatives and the significance of their impact rural, regional and national levels. we would like to end this editorial by acknowledging the reviewers and their contribution to this theme of broadband rural regional and national impacts in this special section. their constructive feedback and comments to the prospective authors were invaluable and without their efforts this special section of ajis would not be possible. lisa mcdonald arifeen mamun anup shrestha sophia imran marhaini noor frank elston ann starasts reza rajabiun we hope that you will find the papers in this special issue interesting and provide an impetus for future work in this important and under-researched area of information systems. we would also like to acknowledge the wonderful support that we received from the editor-inchief, associate professor john lamp. copyright: © 2016 lane, middleton, zarnekow & alam. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ http://creativecommons.org/licenses/by-nc/3.0/au/ microsoft word 1707-copy.docx australasian journal of information systems palekar &sedera 2018, vol 22, research on recent advances in social media influence of sbns on news consumption 1 influence of social broadcasting networks on news consumption shailesh palekar queensland university of technology s.palekar@qut.edu.au darshana sedera monash university abstract this study assesses potentially sensitive effects of a specific sub-class of digital social technologies called social broadcasting networks (sbns). adopting the explanatory mechanisms of social network structures (borgatti et al. 2009; kane et al. 2014), and adapting herzberg et al’s (1959) two factor theory, it investigates how the structure of an sbn (conceptualized as its relative advantage), and the influence of users in an sbn empower two key news consumption behaviours viz. switching and continuance. employing the survey (qualitative) method, the study empirically validates the news consumption behaviours of 325 respondents. findings reveal a more nuanced role of sbns indicating the potential benefits to news firms because of individuals switching to traditional news mediums while in the same breadth highlights the potential of sbns as competitive news providers. it shows intermittent use of sbns and signifies that when online and offline news mediums co-exist, influence of users in the incumbent (online) medium plays a powerful role in inducing or refraining the use of a substitute offline medium. this study develops new ontological and theoretical understanding of the entanglement of technology and social activity that is dynamic, at times ephemeral and pervasively ubiquitous. keywords social media; two-factor theory; news media 1 introduction digital social technologies such as facebook and twitter have also been defined as social broadcasting networks (sbns) (shi et al. 2014). on a more holistic and higher-order level they simultaneously provide a broadcasting service as well as enable social networking (shi et al. 2014). sbns broadcast information similar to traditional subscribed relationships through followers-following on twitter, friends on facebook or groups in linkedin (shi et al. 2014). sbns focus on converging micro-blogging, user-generated-content and social networking (kang and lee 2017). for example, onlooker janis krums tweeted the dramatic crash landing of u.s. airways flight 1549 on the frigid waters of the hudson river as it unfolded in real time (krums 2009). his tweet was quickly consumed and re-tweeted by hundreds of other twitter users even before the news media caught wind of the dramatic incident. thus, sbns blur the contemporary precincts of information-based businesses such as traditional news mediums (e.g. television, newspapers) due to its (i) decentralized structure converged by a diverse spectrum of information (wu et al. 2011) (ii) open and loose network of people that are unilaterally established with ease across geographical zones (shi et al. 2014), and (iii) freedom over information creation, dissemination and consumption (kwak et al. 2010; palekar et al. 2015). news consumer are no longer tied to any specific news mediums (e.g. television, online news website). rather, they traverse through diverse online news sources and different sbns australasian journal of information systems palekar &sedera 2018, vol 22, research on recent advances in social media influence of sbns on news consumption 2 (pew research center 2013; williamson et al. 2012) expecting and demanding news in fast, convenient and smart ways consummate with a deep social and human touch. by amalgamating computing, communication and connectivity technologies (bharadwaj, el sawy, pavlou, & venkatraman, 2013; oestreicher-singer & zalmanson, 2013), sbns have deeply impacted news consumption by creating novel ways for communicating and creating newsworthy information (bruns, highfield, & ann lind, 2012; pentina & tarafdar, 2014). this study investigates how sbns and user influence in such networks affect the news consumption of people. media and user influence effects are important for understanding the nomological net of information systems (viz. sbns in this paper) and explicating the opportunities or threats they pose. online viral effects of news consumption, diffusion and dissemination exhibit actions similar to real-life contagion behaviours such as spontaneity, unpredictability, irrationality or disorderliness (coleman 1990; sedera et al. 2016b). anecdotal commentary suggests that user influence and affinity spawned by mass individuals in sbns result in the continuance of news consumption in the networks and aid their evolving roles as powerful platforms for broadcasting, sourcing and consuming news. thus, it is germane to investigate sbns for developing new ontological and theoretical understanding of the entanglement of technology and social activity that is dynamic, at times ephemeral and pervasively ubiquitous. specifically, this paper investigates a key research question: what factors influence news consumption on social broadcasting networks? 1.1 news consumption behaviours on social broadcasting networks sbns render users the choice and freedom to switch news consumption to substitutable news mediums based on the utility and gratifications afforded by the medium (kang and lee 2017; yuan 2011). for example, individuals reading news tweeted by a news firm can instantly switch to the news firms’ website by clicking on the web links provided with the news tweet. likewise, individuals on sbns can also switch their news consumption to other internet-based (e.g. news website) or offline news mediums (e.g. television) based on social and psychological needs for selecting a specific medium or news content area (diddi & larose, 2006) for more information and news verification. in such transformative trends, consumers dictate key elements serving their consumption choices, which include demand and control over how, when and where news should be consumed and ownership of self-generated content (setia et al. 2013). thus, the relative advantage of sbns in facilitating (i) open access and connectedness to information and its providers (ii) ownership of information (iii) immediacy of news, and (iv) control over what, when and how news is consumed, challenges the “dialectic tensions between the public-interest role of the news media” (hirst 2011, p. 15). in sbns, users exerting influence on other users in sbns is a critical driver of mass consumption and the networks’ successful functioning (shi et al. 2014). such mass information exchange creates a “web of cooperative relationships that breeds norms, trust, common purpose, and coordination – that is social capital” (kane et al. 2014, p. 277). people of repute (e.g. entertainers, business and political leaders) frequently use sbns to connect with all stakeholders, garner, reclaim and retain public support and trust, and influence future social, business and political ideas, agenda’s and trends. this provides users in sbns the opportunity to connect with such personalities, which otherwise is difficult in the real world. consuming such content provides individuals “information, influence, or social support” (kane et al. 2014, p. 277). news practitioners (e.g. reporters) also utilize sbns, to instantly disseminate news, and influence people to engage with the news companies they represent. past studies on post-adoption use of it treat continuous use or the decision for continuing the use of an innovation, technology australasian journal of information systems palekar &sedera 2018, vol 22, research on recent advances in social media influence of sbns on news consumption 3 or product as similar (sedera et al. 2016a; ye et al. 2008). such treatment relies on the assumption that specific technologies (e.g. web 2.0 applications) represent the innovation (i.e. sbns). this is a reasonable assumption based on the setting of many digital social technologies (e.g. sbns) that have multiple products, which are similar in functionality and substitutability (e.g. yammer and linkedin). however, in the context of this study, users switching news consumption from sbns (e.g. twitter) to a substitute that serves similar needs, such as a television news channel or a news website constitutes partial or intermittent replacement of news mediums fulfilling similar needs (bhattacherjee et al. 2012). in such scenarios, users opt for both, sbns and offline news mediums on a concurrent basis. based on the above, this study develops and tests a model depicting individuals (i) switching their news consumption from sbns to traditional news firms, and (ii) continuing their news consumption on sbns. the research model serves as a strong start-up for understanding it switching as well as continuance behaviours in the same breadth by incorporating the supply side effects (e.g. availability of related products) as well as the demand side effects (e.g. user needs and preferences). its main goal is to empirically evidence changes in news consumption behaviours based on the influence of sbns (i.e. its advantages as a medium) and influence in sbns (i.e. users influencing other users in sbns). the issues discussed provide the premise and the motivation for exploring how sbns disrupt the ways in which people understand and enhance their news creation and consumption experiences. in addition, the defiant roles of sbns in disrupting traditional ways of consuming news and threating the identity of traditional news firms (kietzmann et al. 2011) has been largely under-researched or overlooked in past is studies, which this paper explores. thus, this study responds to calls for researching the strengths and drawbacks of digital social technologies and theoretically explain their mechanisms and dynamics. this paper is further structured as follows: it begins with theoretical foundations followed by the section on the process model development. next, it details hypothesis development and methodology followed by data analysis and findings. it concludes with contributions to research and implications for practice and then lists the study limitations. 2 theoretical foundations for analysing digital social networks, borgatti & foster (2003) offer a parsimonious framework that describes four canonical types of explanatory mechanisms consisting of explanatory goals, which are built around the structure of a network and the content in the network. these explanatory goals capture the social homogeneity or disparities in network performance and performance variations (kane et al. 2014, p. 277). sbns offer users media choices that are motivated by a networks’ structure, content and users. thus, this study adopts the explanatory mechanisms centred around the (i) structure of an sbn, which relates to its relative advantage for understanding the social homogeneity in consuming news, and (ii) influence of users in an sbn for understanding how user performance impacts news consumption in an sbn. this provides the foundation in building up the factors that affect news switching and continuance behaviours. 2.1 relative advantage of social broadcasting networks relative advantage is conceptualized as a multi-dimensional construct (choudhury and karahanna 2008) based on the diffusion of innovation theory (refer to rogers 1995). through the diffusion of innovation theory, relative advantage captures the gains and benefits of an australasian journal of information systems palekar &sedera 2018, vol 22, research on recent advances in social media influence of sbns on news consumption 4 innovation on multiple dimensions such as increase in comfort levels, saving on costs, time and efforts (lokuge and sedera 2014a; rogers 1995), or trust and convenience (choudhury and karahanna 2008). underlining rogers (1995 diffusion of innovation theory and cited by choudhury and karahanna (2008, p. 180), relative advantage is defined as “the degree to which an innovation is perceived as being better than using the practice it supersedes.” a study by bhattacherjee et al. (2012) applying the relative advantage construct for understanding user’s it switching behaviour find that performance expectancy based on the unified theory of acceptance and use of technology (venkatesh et al. 2012) is a “stable and consistent predictor of it acceptance over time” (bhattacherjee et al. 2012, p. 328). a study on user’s it switching behaviour by ye et al. (2008) find that the relative advantage of a competing it product or service is positively related to users switching usage of existing it products to a substitute that satisfies similar needs. this study adopts the relative advantage constructs as the best predictor of media choice in a post-adoption it usage setting. it associates the relative advantage of sbns through its value propositions relating to the (i) immediacy of news (kietzmann et al. 2011) (ii) timeliness of news (pew research center 2013), and (iii) connectedness to global issues (kietzmann et al. 2011). 2.2 user influence in social broadcasting networks the mass dissemination and sharing of user-generated-content in real time (kaplan and haenlein 2012) provides a unique perspective of how users influence news consumption (lee and ma 2012) in sbns. the theoretical lens of social impact (latané 1981) evidences that individuals are influenced by the actions and expectations of other individuals. such influence is based on the (1) strength of the social force – i.e. the importance of the influencer group to the individual (2) immediacy of the social force – i.e. the temporal or physical proximity of the influencer group to the individual at the time of influence, and (3) number of people comprising the influencer group (latané 1981). thus, this study defines user influence as the extent to which a user can cause an effect in an indirect or intangible way on other users to follow or act in a certain way. 2.3 news consumption behaviours in social broadcasting networks this study defines news consumption on sbns as diverse activities undertaken by users of sbns, which involves interacting with information. these activities range from (1) posting text, photos, videos, random ideas, reading (surfing, browsing), sharing content, and commenting (alarifi and sedera 2014; xu et al. 2012) (2) tweeting (kwak et al. 2010), (3) listing (consuming information by grouping other users) (4) following (other users) (alarifi et al. 2015) and, (4) subscribing (velichety and ram 2013). switching: switching involves either partially or fully replacing the use of an it service, application or product with a substitute that potentially fulfils similar needs and objectives (bhattacherjee et al. 2012; ye et al. 2008). however, it switching may not involve the user in completely abandoning the existing use of it artefacts or services but concurrently using diverse yet competing it products and services based on users’ interests, necessities and needs (ye et al. 2008). this study contextualizes switching as concurrent or alternating news consumption between sbns and traditional news mediums (e.g. television). it defines switching as, users’ reducing their news consumption on an incumbent news medium (viz. sbns) while replacing it in part with an alternate news medium (viz. traditional news mediums) that fulfils identical needs. the concept of switching is underlined by the continuance-discontinuance paradigm (bhattacherjee et al. 2012; ye et al. 2008) in a post-adoption it/is use settings where australasian journal of information systems palekar &sedera 2018, vol 22, research on recent advances in social media influence of sbns on news consumption 5 adoption, use and familiarity with the new product or service has already taken place (ye et al. 2008). this study posits that the influence of users in sbns and the relative advantage of sbns significantly impact user switching behaviour in consuming news on sbns. continuance: continuance of news consumption in sbns (and other online mediums) is important and critical for the sustenance and survival of news companies and is dependent on existing customers and their recurring demand for goods and services (bhattacherjee 2001). continuance research in is has been examined through different ways such as for example, implementation (zmud 1982), and infusion and routinization (cooper and zmud 1990). past studies draw on several models for explaining it continuance. for example, hsu et al. (2014) investigate users’ extrinsic and intrinsic motivations for explaining the continuing use of social networking sites by integrating the technology acceptance model (davis 1989) and the theory of planned behaviour (venkatesh and davis 2000). the theoretical constructs that explain intentions for technology acceptance (e.g. ease of use) are generally applied as natural extensions for describing it continuance. lin and huang (2014) propose a conceptual model explaining factors based on the pushing, pulling and mooring effects. this study defines continuance as persistently consuming news on an incumbent news medium while not replacing it with an alternate news medium that fulfils similar needs. it extends the existing it/is continuance literature by studying the influence of an sbn and influence in an sbn on a user rather than user intentions, motivations or cognitions. 2.4 two-factor theory this study uses the two-factor theory as the overarching theoretical lens to explain two factors (viz. influence of sbns – relative advantage, and influence in sbns – user influence) that influence two opposing news behaviours of switching and continuance. psychologist frederick herzberg theorized that satisfaction and dissatisfaction in the workplace are caused by a set of factors that are independent of each other (herzberg et al. 1959) (also refer park and ryoo 2013). the two set of factors correspond to two opposing behaviours, intentions, or state of mind. each set of factors and its corresponding impact variable are like two sides of a coin. they do not impact each other but are neutral to each other (hulin and smith 1967). based on herzberg et al. (1959) study, as dependent constructs in the theory are unrelated, one does not increase as the other diminishes. although the two-factor theory has been applied in diverse research streams (e.g. consumer research (maddox 1981), human behaviour (park and ryoo 2013)), its adoption and adaption in is research is fairly recent (cenfetelli and schwarz 2010; lee et al. 2009; liu et al. 2011; park and ryoo 2013). in is studies, the theory is referred to factors that encourage or provoke and prevent users towards adopting a product or service (cenfetelli and schwarz 2010). however, most studies explain only one side of user behaviour by looking at the set of factors that prevent and enable it, but in different contexts. 3 the process model based on the above, this study proposes a process model (refer to figure 1) for illustrating the progression of news consumption and its lead up to the comparative duration of news consumption in sbns. the process model depicts how an individual on an sbn forms two distinct behaviours in response to prevailing conditions in the network. first, an individual in an sbn wishing to acquire more information and be connected to global issues seeks information as events unfold in real time. this triggers a rational decision-making process influenced by the relative advantage of sbns. the australasian journal of information systems palekar &sedera 2018, vol 22, research on recent advances in social media influence of sbns on news consumption 6 individual either (i) continues consuming news on the network [based on postadoption studies on it continuance (ortiz de guinea and markus 2009)], or (ii) switches to traditional news mediums for more information or verification [based on the post-adoption studies on it switching behaviour (ye et al. 2008)]. second, when an individual in an sbn sees news, posts or updates by friends, other users or news firms, the same rational decision-making process (as stated above) occurs leading the individual to either continue in the network or switch to a traditional news medium. the above observations help in conceptualising the antecedents of the two opposing news behaviours (i.e. switching news consumption to traditional news mediums and continuing news consumption in sbns). the influence of sbns through its value propositions of news immediacy (i.e. individuals can consume news whenever desired), timeliness (i.e. news is available as it happens) and connectedness (i.e. individuals can create, comment or share news thereby connecting to content and other individuals) reflect the relative advantage of the network. influence in sbns is observed through users influencing other users in sbns. both behaviours have an overall impact on the comparative duration of news consumed in sbns thus highlighting the significance of sbns. the process model describes users on sbn influenced by the network as well as users in the network. this is depicted through two situations which users generally face on sbns. these further translate into two outcomes, which are depicted through the news behaviours of switching and continuance. the two-factor theory is illustrated for understanding the relationships between the influence of sbns and the influence in sbns on the news behaviours of switching and continuance. although, as per the theory, each factor has a relationship with only one side of an individual’s state of mind, behaviour or intent, this study extends the theory by testing the relationship of both factors on both news behaviours of switching and continuance. these news behaviours reflect the comparative duration of news consumption in sbns, which is included in the research model to test the theory in its nomological net. figure 1. process model of news consumption on sbn australasian journal of information systems palekar &sedera 2018, vol 22, research on recent advances in social media influence of sbns on news consumption 7 4 hypothesis development the ubiquitous nature of sbns as public networks has necessitated this study to look at two key factors constituting an important and integral part of the network itself. this differs from previous studies that have utilized the two-factor theory for understanding typically only one behaviour such as switching in a bounded organisational environment. the two factors used in this study are: 1. influence of sbns: this relates to the relative advantage of sbns, which in turn is based on the premise of the “feature-centric view of technology” (jasperson et al. 2005, p. 527). technology features are interpreted as well as adapted by it users in order to “constitute a technology-in-use” (jasperson et al. 2005, p. 529) in post-adoption it contexts. [also refer to desanctis and poole (1994)]. technology features and their value propositions are important, as they influence and determine work outcomes at any given points in time (desanctis and poole 1994). in addition, technology attributes reflect the “core of the technology, collectively representing its identity” (jasperson et al. 2005, p. 529). thus, the relative advantage of sbns over their traditional counterparts (i.e. traditional news mediums) delineates the value propositions offered by the network for influencing user behaviour. 2. influence in sbns: this factor is associated with user influence in sbns. these networks comprise a public environment wherein news consumption is a voluntary activity. the networks constitute large online communities (e.g. groups, associations, friends) with common shared interests (ren et al. 2012). individuals in such networks create and share information with others in the network (wasko and faraj 2005) making the networks important digital spaces for professional, political and social discussions (ren et al. 2012). social psychological theory informs the formation of individual attachments in a group, which happen through (1) the group’s identity whereby people are attracted and feel connected to each other based on their goals, activities and character of the group (hogg and turner 1985), and (2) interpersonal bonds, friendships and other social relations developed with other members of the group. user influence on sbns follow the notion of bond-based attachment arising from the exchange of personal and social information (ren et al. 2012). this shifts the attention from the identity of the network to its individual members. 4.1 relative advantage relative advantage is conceptualized as a multi-dimensional construct (choudhury and karahanna 2008) based on the diffusion of innovation theory (refer to rogers 1995). taking a feature-centric view (jasperson et al. 2005) of sbns, this study observes three prime features that reflect the core advantage of sbns and represents the gratifications they offer: 1. immediacy of newsworthy information (kietzmann et al. 2011): individuals use sbns as they (i.e. the networks) transcend temporal and geographical boundaries and provide immediate access to information (xu et al. 2012). these networks facilitate conversations within network users creating instant pathways for information dissemination (kietzmann et al. 2011) 2. timeliness of newsworthy information (li and ng 2013): recent studies on user behaviours on twitter find that instant news reporting (e.g. breaking news, headline news) and commenting on current events are some of the main intentions for using australasian journal of information systems palekar &sedera 2018, vol 22, research on recent advances in social media influence of sbns on news consumption 8 twitter (kwak et al. 2010; li and ng 2013). kwak et al. (2010 exploring the topological attributes of twitter finds that more than 80% of trending topics are headline news occurring in real life. likewise, creating real-time status updates on sbns (e.g. facebook) creates an ambient cognizance of issues and events (kaplan and haenlein 2012), and 3. connectedness (kietzmann et al. 2011). these networks have become platforms of influence by engendering radically new ways of interacting and connecting with both, users and information (hanna et al. 2011). a study by huang (2009) finds that young adults see digital news mediums (e.g. online websites, facebook) as important platforms for connecting and staying informed on the world around them. social connectedness that enhances information sharing in sbns is limited in traditional news mediums in the absence of active and effectual diffusion channels (lee and ma 2012). sbns also support four different types of ties or simple connections between users (kane et al. 2014) such as (1) proximities representing shared physical or social spaces for ties to form (2) relations reflecting social connectedness between nodes such as role-based (e.g. friends) and affective (e.g. likes) connections (3) interactions, and (4) flows such as intangible materials (e.g. beliefs, information) between interacting nodes in the network. sbns also facilitate connecting with influential users (e.g. celebrities, experts, leaders) (kane et al. 2014). collectively, the above-mentioned features represent strong value propositions reflecting the identity of sbns. such features constitute a clear advantage that increases the likelihood of users on sbns to continue their news consumption on sbns. thus, this study proposes: hypothesis 1a: relative advantage of a social broadcasting network will lead to greater continuance of news consumption in the network. the relative advantage of sbns is likely to be less salient in relation to switching news consumption to traditional news mediums. this is because substitutable products (i.e. traditional news mediums such as television) have to prove their unique advantages over the incumbent product (i.e. sbns) (ye et al. 2008). however, if individuals expect other advantages or benefits from the substitutable product or service, then they are likely to switch to the substitutable product (ye et al. 2008). as a ubiquitous news media, sbns offer users a wide array of affordances (majchrzak et al. 2013) and content selection (lee and ma 2012). in the present digital age, individuals are active users and processors of information (ball-rokeach and jung 2009) and therefore likely to evaluate the potential benefits of an sbn based on their usage and experience, for further use (lee and ma 2012). sbns have increased public debates and social discussions (agarwal et al. 2008) resulting in widespread proliferation of its use (kane et al. 2014). based on the above, this study posits: hypothesis 1b: relative advantage of a social broadcasting network will reduce switching to traditional news media for consuming news. 4.2 user influence as glynn et al. (2012) posits, the prime motivation for joining digital social networks is often social. however, when exposed to constant information posted by friends, colleagues or acquaintances, users start following other users in the networks (glynn et al. 2012). diverse activities influence users in these networks to actively engage in the network, wherein they do not have to rely on (i) news flows controlled by traditional news firms, and (ii) news content determined by editors and news practitioners (lee and ma 2012). users on sbns are australasian journal of information systems palekar &sedera 2018, vol 22, research on recent advances in social media influence of sbns on news consumption 9 influenced by celebrities, experts and leaders, as that (i) increases the users’ reputation in social and professional networks (kane et al. 2014) (ii) provides indirect connections, which are not possible in offline environments, and (iii) keeps users informed of cels personal, social and professional lives in real time. a study by williamson et al. (2012) on young users’ news activities on facebook finds that users prefer personal news in the form of information posted by friends or through status updates and reporting issues and events in real time (e.g. facebook status update – raining heavy, homes likely flooded). a respondent in the study by williamson et al. (2012) says, “i suppose from blogs and facebook and all those sorts of things, i get personal news, that i find out from other people” (williamson et al. 2012, p. 262). based on the above, this study hypothesizes the relationship between user influence in sbns and continuance of news consumption in the network as hypothesis 2a: user influence in a social broadcasting network will lead to greater continuance of news consumption in the network. the diverse goals of institutional users (e.g. business, political and non-commercial entities) on sbns include creating buzz, advertising, marketing, delivering products and services, understanding customer sentiments and gauging product popularity and customer loyalty (divol et al. 2012; hong 2012). news firms, as institutional users engage sbns for (i) actively disseminating news (hong 2012) (ii) ensuring that news reaches the masses through diverse user networks (hong 2012), and (iii) diverting individuals from sbns to their traditional mediums by embedding click back links (pew research center 2013). this has made sbns viable sources of information and news (glynn et al. 2012; shi et al. 2014). prior studies on news media have described its influence on society and public opinion as well as powerful behavioural, cognitive, emotional and attitudinal effects on news consumers. a study by williamson et al. (2012) on the role of news media and young adults finds that people prefer several sources for news and still find printed newspapers having high information credibility. news firms also permit their employees (e.g. reporters) to use sbns for officially sharing information on behalf of the company. they routinely hire social media editors to manage the production and delivery of news content on sbns (glynn et al. 2012). in doing so, they deliver news to mass audiences with the aim of influencing them to consume news on their company’s news mediums. hypothesis 2b: user influence in a social broadcasting network will increase switching to traditional news media, for consuming news. 4.3 post-adoptive impact of news consumption literature on it post-adoptive behaviour postulate that during a period of utilitarian or functional technology use, individuals engaging in the reflective use of technology trigger “technology sense making” (jasperson et al. 2005, p. 544). this is based on how individuals use the technology’s attributes or features rather than their own habitual use of the technology (bhattacherjee 2001; jasperson et al. 2005; sedera et al. 2017). post adoptive behaviours leading to future use of it are also derived from self-orchestrated learning experiences. these are based on it features and the work systems in which they are situated, which potentially modify user cognitions on the future use of the technology (jasperson et al. 2005). this happens through experimentation with it features and applications (desanctis and poole 1994; lokuge and sedera 2016) as well as by observing how others are using similar it applications (bandura 1986; lokuge et al. 2016). in this study, it is observed that individuals reflecting switching and continuance news behaviours tend to rely less frequently in reflecting on other attributes of australasian journal of information systems palekar &sedera 2018, vol 22, research on recent advances in social media influence of sbns on news consumption 10 sbns. instead, they rely on their news consumption experience for future news consumption in an sbn. thus, it is pragmatic to apply prior-use experiences to imprint future news consumption, which occur automatically in response to an urge for consuming news. according to jasperson et al. (2005) past it use behaviour generally produces a tendency for similar future actions in a given context, which dissipate reflective cognitive processing and result in more routinized behaviours. based on the above, this study posits that news consumption of individuals on sbns will depend on their existing news behaviours rather than user intentions or decisions. with respect to continuing news consumption in sbns, this study hypothesizes that the habitual continuance of news consumption in these networks is a dominant predictor of a network supporting higher consumption of news compared to traditional news mediums. hypothesis 3: continual consumption of news in a social broadcasting network will increase the duration of news consumed in the network compared to a traditional news medium. present-day technologies that cater to individual use are often highly substitutable products (or services) that have similar functionalities and compete with each other (e.g. web browsers) (ye et al. 2008). such settings induce users to terminate, reduce or concurrently use the incumbent it/is application and switch to functionally related products or services. however, such switching behaviours occur only in post-adoption it/is use settings where adoption, use and familiarity with the new product or service has taken place (salim et al. 2014; ye et al. 2008). a study by the pew research centre relating to news on facebook finds news consumers regularly watch news on television and read printed newspapers (pew research center 2013). thus, this study hypothesizes, hypothesis 4: switching news consumption from a social broadcasting network to traditional news media will reduce the duration of news consumption in the network. the influence of sbns and the influence in sbns form the antecedents to the news behaviours of switching and continuance. these behaviours are self-reported activities comprising news consumption on sbns. the outcome of these behaviours is depicted through the comparative duration of news consumption on sbns. 5 methodology this study employs the survey (qualitative) method. this is influenced by the study objectives, which seek to test and verify the relationship between individuals’ news behaviours and the factors influencing those behaviours. the quantitative method is also used based on a careful review of related studies. for example, bhattacherjee et al. (2012) use a survey to empirically test users switching from one it product to another (contextualized to web browsers) in an environment of competitive technology choices. park and ryoo (2013) employ the survey method to empirically examine the relationship between two switching factors (viz. switching enablers and switching inhibitors) and intention of users to switch to cloud services. the quantitative method is effective in endorsing or challenging hypothetical relationships and uncovering new associations through statistical analysis and objective verifications (danziger and kraemer 1991; gable 1994; kling 1991). past studies have also shown that the quantitative method (i) has superior deducibility over the qualitative (e.g. case studies) method (vidich and shapiro 1955; walther et al. 2013b) (ii) can accurately document norms, identify outcomes and demarcate relationships between variables (attewel and rule 1991; lokuge and sedera 2014b; vidich and shapiro 1955) (iii) is a sound methodology for data verification (gable 1994; australasian journal of information systems palekar &sedera 2018, vol 22, research on recent advances in social media influence of sbns on news consumption 11 walther et al. 2013a), and (iv) facilitates data analyses at aggregate as well as individual levels thus offering better explanations about the validity of the survey instrument and the research model (benbasat 1984; sedera 2006). this study employed a seven point category scale for capturing the frequency (zikmund 2000) of news consumption. the scale measures how individuals rate the extent of their news consumption activities. the categories on the scale move from the left (negative) to the right (positive) side of the scale. they are: never, rarely, just now & then, sometimes, often, very often, all of the time. numbers from 1 (never) to 7 (all of the time) are assigned as anchors to each category for data analysis. the interval between each category is treated as equal. thus, the scale can also be used as interval measurement scale (cavana et al. 2001; sedera et al. 2003). the category ‘sometimes’ (the 4th in the 7-point scale) is taken as the midpoint of the scale and treated as neutral to the scale or a “passing level” (refer to cavana et al. 2001, p. 205) for rating the extent of news consumption. in comparison: (i) often (the 5th in the 7-point scale) and points thereafter to its right (6th and 7th) show an ascending degree of frequency wherein the extent of news consumption is considered positive, and (ii) just now & then (the 3rd in the 7point scale) and points thereafter to its left (2nd and 1st) show a descending degree of frequency wherein the extent of news consumption is considered negative. all questions in the survey were mandatory. the sole criterion for taking the survey was respondents had to be existing users of sbns (e.g. twitter, facebook) and traditional news mediums (e.g. news websites, television or newspapers). in order to eliminate and reduce the influence of common method variance, the study employed different sources of information during measures development and assorted the order of questions to reduce the likelihood of theory-in-use bias (chang et al. 2010). further, this study ensured that survey questions were (i) appropriately worded to convey precise information, and (ii) shortened to reduce ambiguity, maintain clarity, and influence participation and response (williams et al. 2003). 5.1 survey deployment this study employed face validation procedures (similar to content validation) wherein the appropriateness of survey items for accuracy, language and appearance was evaluated by subjective judgements of experts (keszei et al. 2010). it consulted a group of nine respondents comprising post-graduate students, doctoral researchers, academics and the public. the group provided constructive critique and feedback on (i) improving the construct definitions (ii) refining the construct measures and measurement items, and (iii) improving the measurement items. the feedback improved the item wordings, readability, meaning and content validity of the instrument (schouten et al. 2010). the instrument was then pre-tested with 21 respondents (10 individuals responded to the survey online while 11 responded to the survey offline by completing a paper-based questionnaire). next, the survey instrument was pilot tested with 40 respondents for assessing reliability and validity of individual items, which yielded acceptable results. see appendix a. the main survey was offered simultaneously in three ways: (1) a link to the online survey was emailed to students and staff at a tertiary institution (2) printed copies of the survey were handed out to students, at random, on campus at the institution, and (3) it was posted online on sbns (e.g. facebook and linkedin). the criteria for taking the survey was respondents had to be potential or existing news consumers on sbns as well as traditional news mediums (e.g. tv). a total of 325 individuals successfully completed the survey (note: the online survey recorded 969 clicks). given the limitations of time and available resources, a small australasian journal of information systems palekar &sedera 2018, vol 22, research on recent advances in social media influence of sbns on news consumption 12 representative sample of the population had been pre-planned. in deploying quantitative methods, smaller groups are useful in making inferences about bigger groups that are potentially exorbitant to study (barlett et al. 2001). the sample size comprising 325 respondents is deemed reasonable and representative of the population based on cochran (1977) formula for determining basic sample size. in doing so, the alpha level a-priori was set at 0.05, and based on the 7-point category scale the level of acceptable error was set at 3% with standard deviation as 1.167. the sample size is also adequate based on the number of items used in gathering data (note: the survey instrument contains 23 items). 6 data analysis and findings 6.1 reliability tests based on the work of barclay et al. (1995), this study examined the construct reliability of the research model by assessing individual measurement item reliability, internal consistency and discriminant validity. structural equation modelling (sem) was employed for analysing the data using amos 22.0 and the measurement model was examined prior to assessing the structural (research) model (hsieh and wang 2007). the measurement attributes of all constructs were estimated with confirmatory factor analysis and items with low loadings were deleted. the resultant indices show an acceptable baseline fit. the rmsea and cfi of the model is on par with the desired 0.08 and 0.90 levels respectively (browne and cudeck 1993). the research model indices relating to the tucker-lewis index (tli) (hsieh and wang 2007) is baseline with the desired criterion level. next, this study examined the discriminant validity, internal consistency and convergent validity by evaluating the cronbach’s alpha, composite reliability (cr), and average variance extracted (ave) of each construct. cronbach’s alpha and composite reliability values are consistent with the recommended 0.70 levels (nunnally et al. 1967) while ave values are acceptable, as they meet the desired 0.50 level (fornell and larcker 1981). thus, convergent validity level is acceptable confirming that all items are consistent, reliable, and reflect the properties that the construct represents. further, this study conducted the discriminant validity test to measure the difference between constructs in the research model. this was done through amos (version 22.0) by assessing the average value of each construct and comparing its squared correlations with other constructs. the analysed data also shows the average shared variable (asv) and when compared to the average variance extracted (ave), the asv is less thereby offering strong support for discriminant validity of constructs (fornell and larcker 1981). (note: due to page limitations, internal consistency and convergent validity assessment, and comparisons of ave and squared correlations have not been tabled in this paper) 6.2 research model testing the predictive power of standardised path coefficients, variance (r2) and path significance in the research model were examined using spss and its add-on module amos (version 22.0). the tested model using the sample data (n=325) is illustrated in figure 2. (note: due to page limitations, regression weights explaining variances between the constructs are not tabled in this paper). australasian journal of information systems palekar &sedera 2018, vol 22, research on recent advances in social media influence of sbns on news consumption 13 figure 2. tested research model (n=325) the examined sample (n=325) reveals: (1) a significant path between user influence and continuing news consumption on sbns. this shows that users in sbns substantially sway other users in the network to consume news in the network (2) the relationship between user influence and switching news consumption to traditional news medium is significant. this implies that news practitioners and news firms (viz. institutional users) are successful in attracting news consumers to their contemporary news channels, although such relationships are potentially weak (3) the relative advantage of sbns also shows a meaningful relationship with continuing news consumption in sbns. this denotes that individuals consider the relative advantage of sbns for seeking news on an immediate and timely basis and for connecting to both, information and users (4) the relative advantage of sbns is not significant when individuals choose to switch their news consumption to traditional news mediums. this denotes that the relative advantage of sbns is largely irrelevant or unimportant while deciding to switch to traditional news mediums for consuming news (5) collectively, both constructs viz. user influence and relative advantage explain 74% of variance for continuing news consumption on sbns while the same two constructs viz. user influence and relative advantage of sbns demonstrate just 5% variance in switching news consumption to traditional news mediums. this has serious implications for news firms that engage with sbns for attracting people to their traditional news mediums, and (6) collectively, switching and continuing news behaviours explain 41% variation in the duration of news consumption on sbns compared to the duration of news consumption on traditional news mediums. this indicates that although people intermittently switch to traditional news mediums, the extent of such switching is low. rather, most individuals consume news on sbns on a continuing basis. thus, although sbns may benefit news firms by providing their platforms for delivering news, most people may not switch to traditional news mediums when they are on sbns. 6.3 hypothesis testing 6.3.1 hypothesis 1a testing confirms that continuance of news consumption on sbns increases when individuals evaluate the value propositions of sbns for consuming news. the correlation between relative advantage and continuing news consumption is significant and shows positive path coefficient (β = 0.179 with p < 0.001). the features and value propositions of sbns represent a strong identity of the networks for consuming news. the data shows that individuals consider australasian journal of information systems palekar &sedera 2018, vol 22, research on recent advances in social media influence of sbns on news consumption 14 these more significant when making a media choice between sbns and traditional news mediums. therefore, the relative advantage of sbns is a given fact, which does not require significant independent consideration when choosing a media for consuming news. the data supports hypothesis 1a, “relative advantage of a social broadcasting network will lead to greater continuance of news consumption in the network.” 6.3.2 hypothesis 1b testing shows a negative path coefficient between relative advantage of sbns and switching news consumption to traditional news mediums. this relationship is evidenced as being not significant (β = -0.005). as hypothesized, substitutable products such as traditional news mediums must prove their unique advantages or strengths over incumbent products such as sbns to attract usage (note: this study situates individuals already engaging with sbns). therefore, individuals are likely to switch to competitive products only if the expected benefits are greater than the incumbent products. however, the relative advantage of sbns is not an important consideration for users wishing to switch to traditional news mediums for consuming news. although features and value propositions of sbns represent a strong identity of the networks for consuming news, users (specifically younger generations such as new millennials) do not consider those as significant when choosing between sbns and traditional news mediums. this is because they are already actively engaged in consuming information beyond news in sbns (e.g. emailing, chat, games etc.). thus, there is little resistance to adopting and using sbns. so, to them, the relative advantage of sbns is a given fact, which does not require significant and independent considerations if switching to other or traditional news mediums. in addition, our study also observed that news switching may not involve the user in completely abandoning the existing use of digital news artefacts or services but rather concurrently using complementing (yet competing news mediums) services. for example, an individual on twitter may simultaneously watch a news channel on television while surfing the same news on twitter. in such scenarios, switching (i.e. the complementing use of traditional news mediums) is likely to make the relative advantage of sbns redundant. therefore, the data does not support hypothesis 1b, “relative advantage of a social broadcasting network will reduce switching to traditional news media, for consuming news.” 6.3.3 hypothesis 2a the tested model shows a highly significant path coefficient between user influence in an sbn and continuing news consumption in the network. it shows a positive path coefficient β = 0.740 with p < 0.0001. this is a significant finding that evidences the influences exerted by users in sbns on other individuals in the network, which strongly contributes towards users continuing their news consumption on sbns. this denotes that users do not switch over to traditional news mediums as a result thereof. thus, findings concur with hypothesis 2a, “user influence in a social broadcasting network will lead to greater continuance of news consumption in the network.” 6.3.4 hypothesis 2b the tested model shows positive support but differing levels of significance between users influencing other users in sbns and users switching their news consumption to traditional news mediums. the sample show a positive and significant path coefficient with β = 0.229 with p < 0.05. this is suggestive of a significant effect that institutional users such as news firms can australasian journal of information systems palekar &sedera 2018, vol 22, research on recent advances in social media influence of sbns on news consumption 15 have on individuals in sbns. the evidence of switching news consumption to traditional news mediums happening simultaneously with continuing news consumption on sbns suggests that sbns have a dual effect on individuals in the network. however, compared to hypothesis 2a, user influence on switching is lower than on continuing news consumption on sbns. findings support hypothesis 2b, “user influence in a social broadcasting network will increase switching to traditional news media, for consuming news.” 6.3.5 hypothesis 3 the tested model finds the outcome of continuing news consumption on sbns to higher duration of news consumption in the network to be highly significant and shows positive support for hypothesis 3, “continually consuming news in a social broadcasting network will increase the duration of news consumed in the network compared to a traditional news medium.” findings evidence that continuance of news consumption on sbns is a leading forecaster of the networks’ supporting higher duration of news consumption compared to traditional news mediums. this has strong implications for news firms. it suggests that sbns are superior news providers compared to traditional news firms. 6.3.6 hypothesis 4 findings show that when users switch from sbns to traditional news mediums, it decreases their overall duration of news consumption on sbns. thus, switching to traditional news mediums has a negative impact on the overall duration of news consumption in sbns. this supports hypothesis 4, “switching news consumption from a social broadcasting network to traditional news media will reduce the duration of news consumption in the network.” 6.4 key findings the empirically validated research model depicts that sbns influence individual news consumption showing greater duration of individual news consumption on the networks compared to traditional news mediums. yet, it also shows support for individuals switching their news consumption to traditional news mediums. summarized findings are: (1) users in sbns show a dual way of consuming news. on the one hand, they sometimes switch to traditional news mediums for consuming news while on the other they sometimes continue their news consumption in sbns. overall, users often spend more time in consuming news in sbns compared to traditional news mediums (2) the antecedents to news switching and continuance, which is the relative advantage of sbns has a significant impact on continuing news consumption on the networks. it has a negative impact on switching news consumption to traditional news mediums, which is not significant. user influence in sbns has a highly significant impact on individuals continuing news consumption on sbns while its impact on switching to traditional news mediums is found to be significant (3) individuals switch to traditional news mediums only for specific information or for verifying information but overall continue consuming news for longer durations on sbns. this indicates strong support for sbns as news providers (4) news firms as institutional users deliver news through sbns and potentially influence individuals to consume news through their traditional platforms as evidenced by the switching behaviour. however, higher duration of news consumption on sbns (compared to traditional news mediums) denote that any benefits to news firms accrued because of switching news consumption is perhaps temporary and unsustainable in the long run (5) the data shows elevated levels of switching news consumption to traditional news mediums compared to continuing news consumption on sbns. this is perhaps due to the exposure to both, traditional and digital media in equal measure. however, the duration of australasian journal of information systems palekar &sedera 2018, vol 22, research on recent advances in social media influence of sbns on news consumption 16 news consumption on sbns shows an increasing trend indicating their potential superiority as news providers (6) the data shows a general decline in switching news consumption to traditional news mediums, and (7) user influence is evidenced as a key variable having a highly significant effect on individuals switching and continuing their news consumption on sbns. although social influence has been investigated by past studies, actual user influence has largely been under researched. so, this study clarifies and empirically validates the nuanced role of users in digital social networks. note: this study also analysed data relating to demographic features of respondents using descriptive statistics. the analysis featured sample demographic statistics classified through respondent’s age, gender and geographical location. the data represents four applicable generation cohorts, viz. baby boomers (born before 1964), generation x (1965-1978), generation y (1979-1989), and new millennials (1990 and resulting thereafter). the analysed data showed 47.38% of news consumers on sbns to be new millennials (154 respondents), which comprises the biggest cohort amongst all groups, followed by generation y (24.31%), generation x (22.15%) and baby boomers (6%). the analysis depicted the increasing use of sbns by successive generations. the male to female gender ratio was reasonably balanced in all groups with the new millennials showing the biggest gap with 29.23% males compared to 18.25% females using sbns for consuming news. post-hoc testing was conducted on the sample (n=325), which was polarized into two groups: (1) baby boomers + generation x + generation y, and (2) new millennials. 7 conclusion social broadcasting networks such as facebook and twitter (sbns) (shi et al. 2014) have infused the social element in the production and dissemination of news by giving users of such networks the power to report, share and consume information. such information does not constitute news as understood in the traditional sense. rather, they represent bits and pieces of information relating to incidences (e.g. events, actions) that occur on a 24/7 basis. this study investigated the news behaviours of switching and continuance on sbns for explaining how they (i.e. sbns) transform news consumption. it specifically examined the simultaneous switching and continuance of news consumption of 325 respondents in a post-adoption technology setting, i.e. individuals already engaging with sbns for consuming news. it explained news behaviours through the notion of social homogeneity shaped by the environment of sbns and empirically validated it through the constructs of (i) relative advantage of sbns, as influence of the network, and (ii) user influence on other users as the influence in the network environment. thus, by testing a model of news behaviours on sbns, this study brings new understanding on the role of digital news media, communications and digital social networks, and the dependency relations between network environments and users. 8 contributions to research and implications for practice past studies have investigated news consumption on traditional news mediums and the internet through broad themes such as (i) forming news repertoires across multiple media platforms (yuan 2011) (ii) information overload and sense making (pentina and tarafdar 2014), and (iii) displacement effects of media (lee and leung 2008). however, those studies do not (i) define news consumption and consumption activities on any specific news medium australasian journal of information systems palekar &sedera 2018, vol 22, research on recent advances in social media influence of sbns on news consumption 17 (ii) investigate news consumption on a specific sub-class of digital social networks such as sbns (iii) explain news behaviours such as switching and continuance (iv) examine the environmental influences of and in specific news mediums on news behaviours, and (v) highlight the role of digital social networks based on the empirical validation of news behaviours, all of which this study accomplishes. by adapting the two factor theory, this study expands the body of knowledge to a new sub-class of digital social technologies called sbns (shi et al. 2014). it explains how information is consumed at the individual level in sbns due to the influence of sbns and the influence in sbns. focusing on the domain of news media it builds and empirically validates a research model depicting news consumption on sbns. first, it directs the attention of researchers towards distinguishing the role of digital social technologies such as facebook or twitter as sbns. it clarifies the role of facebook, twitter and similar technologies as information broadcasters by highlighting their social and broadcasting features by comparing them with traditional news mediums. such evaluations facilitate understanding the epistemic value of sbns as news providers. there is disagreement on the effectiveness of digital social technologies in enabling citizen journalism and taking up the role of gatekeeper in informing society and enhancing civic knowledge (pentina and tarafdar 2014). however, this study reveals a more nuanced role of sbns, through the news behaviours of individuals. it demonstrates the potential benefits to news firms because of individuals switching from sbns to traditional news mediums while in the same breadth highlights the potential of sbns becoming competitive news providers. this is perhaps one of the first studies in is providing empirical evidence on individual news consumption behaviours on a digital social network, which leads the study (future) towards interpreting the role of technology (i.e. sbns) at the organisational level. second, it switching is an emerging yet under-researched area in is (bhattacherjee et al. 2012) while it continuance has attracted relatively more attention as a set of affective responses to it acceptance and use (kim and malhotra 2005). this study concisely demonstrates the integration of both, switching and continuance for explaining news consumption. it positions switching and continuance as post-it adoption behaviours and explains actual news consumption rather than intentions. third, in addressing the causes of it continuance behaviours, this study takes a more recursive stance in positioning switching and continuance as two opposing news consumption behaviours. it adapts the theoretical lens of the two-factor theory and develops the constructs of relative advantage (signifying influence of sbns) and user influence (indicating influence in the networks) as two factors for explaining why users switch or continue consuming news on sbns. it explains new dynamics of intermittent or alternate use of sbns, which has not been investigated by prior studies. also, it demonstrates that when online and offline news mediums co-exist, influence of users on other users in the incumbent (online) medium plays a key role in inducing the decision to use or refrain from using a substitutable offline medium. relative advantage has been described in past studies as the best predictor of it adoption leading to actual usage (karahanna et al. 1999) as well as in it switching intentions (bhattacherjee et al. 2012). however, this study finds that it does not have a significant effect on users deciding to either switch news consumption to traditional news mediums or continue news consumption on sbns. this study tests the two-factor theory in a post-adoption setting on two opposing news behaviours of switching and continuance within the context of social digital technologies, which has not been attempted by past studies in is. australasian journal of information systems palekar &sedera 2018, vol 22, research on recent advances in social media influence of sbns on news consumption 18 fourth, findings contribute to the is, media and communications domains by constructing and validating a model of news behaviours through two key factors (viz. relative advantage and user influence). in general, this study systemically assesses unique or potentially sensitive effects of a specific sub-class of digital social technologies (i.e. sbns) at the individual level. fifth, in recent times there has been much debate (especially in the u.s. in the aftermath of the 2017 presidential elections) on whether contemporary media manipulates news (popularly termed as fake news). this study provides a conceptual guide (refer to figure 1) into how consumers can alternate between traditional news mediums and sbns for newsworthy information. this implies that news consumers have a choice of selecting traditional news channels that purportedly propagate fake news, or consume news aggregated, curated or broadcast at grassroot levels by people using sbns. such participatory, social and partisan nature of news reporting suggests (as per our preliminary observations) that consumers are likely to be persuaded by family and close friends, and people of repute such as entertainers, business and political leaders. however, in recent times even sbns have been plagued (e.g. youtube, twitter) by hoaxes peddling false and misleading information created by unverified channels. so, although user influence impacts news consumption behaviours (especially continuity on sbns, as evidenced by our study) people and channels masquerading as newsworthy outlets make it difficult to filter contextual expression and validity of information flows from genuine news reporting this phenomenon requires further and in-depth attention and research. 8.1 implications for practice this study evidences that sbns are no longer static channels for information flows between senders and receivers. rather, they have become veritable repositories enabling: (i) dynamic interactivity in real time between mass communicators (ii) reporting, sharing and creating diverse content and views by mass users (iii) selectivity and control over information and usage of the network (iv) personalization of content (v) convergence of technological boundaries between communication forms and social elements, and (vi) global reach and media portability (metzger, 2009). such strategic choices that exploit or explore the opportunities offered sbn are important determinants of firm survival. news consumption on sbns shows that news firms engage with sbns as institutional users to complement their business function of news delivery. this can happen due to major shifts within the news firm in relation to (i) limitations of traditional business models, as any modern technology that enhances readership is more than likely to limit the firm’s incumbent business model including its scope and scale, making it imperative for the firm to adopt the technology and fuse it with existing products and processes (ii) limited trans-functional role of it: for example, the function of a television news-broadcasting firm requires a different technology (i.e. broadcasting), which is different from the technology required for creating content (i.e. publishing and editing system), and (iii) increased familiarity with it: sbns are easy to use. their client-interface and usability is intuitively designed for use and is less cumbersome than the applications of legacy systems. however, given the low cost of engagement and no requirements for discontinuing existing products, services, functions and processes, news firms need to be vigilant in engaging with sbns for delivering news to mass audiences in a bid to attract them to their news mediums. this implies that although sbns provide their platforms for all users, and never meant to compete with news firm, users in an sbn influence other users and in the process, build the potential competing power of the network as a superior news provider. hence, news firms need to provide more contextual information or australasian journal of information systems palekar &sedera 2018, vol 22, research on recent advances in social media influence of sbns on news consumption 19 in-depth news coverage and analysis for ensuring longer news consumption on their traditional mediums. this can potentially strengthen their subscription-based revenues for trustworthy and premium news content in the face of rising unverifiable information by online user-generated reports. the powerful significance of user influence, as found by this study can be applied to other domains of information consumption. for example, peer influence or biased product reviews can intensify or demystify the information asymmetry of individuals and limit (or increase) their exposure and choice to new product information and pricing. behavioural economics proposes that people are generally affected by a paradox of choice, in which increasing number of options increases the difficulty of decision-making. so, information from familiar or authoritative sources is perceived as having greater value and significance in the decision-making process. through this study we observe that users (i) are influenced by who (e.g. other users) communicates newsworthy information (ii) are motivated by what (they think) other users do (e.g. commenting, posting on sbns) (iii) are influenced by user cues (e.g. like’s) that shape actions, and (iv) act and feel better about what they do based on emotional attachments or applause from other users. therefore, organizations (e.g. hotels, airlines) need to evaluate and manage ways in which patrons and consumers provide comments and/or information relating to the organization’s products and services as well as their (i.e. consumers’) experiences. curating and sharing product information through personal blogs can negatively (or positively) impact the buying decisions of consumers. this study can be further extended by three academic domains viz. journalism, communications, and management. sbns and consumption behaviours can be analysed at three levels: (1) individual users and society (public), (2) intermediaries and platforms, and (3) organisations and industries. 8.2 limitations the generalized findings are limited by the data sample size and the chosen news media. both, sbns and traditional news mediums are freely available and constitute reasonably strong substitutes. users can voluntarily choose one over the other. thus, further research is required for identifying and evaluating more salient factors other than those developed by this study. a bigger data sample size is also proposed (future research) for improving generalized findings. sbns are not conceptualised by this study as monolithic it artefacts. rather, they represent a collection of many mediums, platforms and applications that have varied capabilities (e.g. twitter is not the same as facebook). therefore, it is possible that the generic attributes of different sbns may not be homogenous across all sbns. so, the reflective role of sbns may differ across networks. in addition, choosing news mediums exhibiting strong disparities in relative advantage can reveal stronger effects on switching and continuance behaviour than the findings reported by this study. the research model and the interpretations of findings are situated in a post-it adoption setting pertaining to personal use. references agarwal, r., gupta, a. k., and kraut, r. 2008. "editorial overview – the interplay between digital and social networks," information systems research (19:3), pp. 243-252. alarifi, a., and sedera, d. 2014. "peripheral, central and coercive routes for promoting enterprise social networks," australasian conference on information, auckland, new zealand: acis. australasian journal of information systems palekar &sedera 2018, vol 22, research on recent advances in social media influence of sbns on news consumption 20 alarifi, a., sedera, d., and recker, j. 2015. "posters versus lurkers: improving participation in enterprise social networks through promotional messages," in: international conference on information systems. fort worth, texas, usa: ais. attewel, p., and rule, j. b. (eds.). 1991. survey and other methodologies applied to it impact research: experiences from a comparative study of business computing. boston, ma: harvard business school press. ball-rokeach, s. j., and jung, j.-y. 2009. "the evolution of media system dependency theory," in the sage handbook of media processes and effects, r.l. nabi and m.b. oliver (eds.). thousand oaks, california: sage publications, inc., pp. 531-544. bandura, a. 1986. social foundations of thought and action: a social cognitive theory. prenticehall, inc. barclay, d., higgins, c., and thompson, r. 1995. "the partial least squares (pls) approach to causal modeling: personal computer adoption and use as an illustration," technology studies (2:2), pp. 285-309. barlett, j. e., kotrlik, j. w., and higgins, c. c. 2001. "organizational research: determining appropriate sample size in survey research," information technology, learning, and performance journal (19:1), p. 43. benbasat, i. (ed.) 1984. an analysis of research methodologies. boston, m.a: harvard business school press. bhattacherjee, a. 2001. "understanding information systems continuance: an expectationconfirmation model," mis quarterly (25:3), pp. 351-370. bhattacherjee, a., limayem, m., and cheung, c. m. k. 2012. "user switching of information technology: a theoretical synthesis and empirical test," information & management (49:7–8), pp. 327-333. borgatti, s. p., and foster, p. c. 2003. "the network paradigm in organizational research: a review and typology," journal of management (29:6), pp. 991-1013. borgatti, s. p., mehra, a., brass, d. j., and labianca, g. 2009. "network analysis in the social sciences," science (323:5916), pp. 892-895. browne, m. w., and cudeck, r. 1993. alternative ways of assessing model fit. thousand oaks, ca.: sage. cavana, r. y., delahaye, b. l., and sekaran, u. 2001. applied business research: qualitative and quantitative methods. milton, queensland: john wiley & sons australia, ltd. cenfetelli, r. t., and schwarz, a. 2010. "identifying and testing the inhibitors of technology usage intentions," information systems research (22:4), pp. 808-823. chang, s.-j., van witteloostuijn, a., and eden, l. 2010. "from the editors: common method variance in international business research," j int bus stud (41:2), pp. 178-184. choudhury, v., and karahanna, e. 2008. "the relative advantage of electronic channels: a multidimensional view," mis quarterly (32:1), pp. 179-200. cochran, w. 1977. sampling techniques. new york: wiley and sons. australasian journal of information systems palekar &sedera 2018, vol 22, research on recent advances in social media influence of sbns on news consumption 21 coleman, j. s. 1990. foundations of social theory. cambridge, mass: belknap press of harvard university press. cooper, r. b., and zmud, r. w. 1990. "information technology implementation research: a technological diffusion approach," management science (36:2), pp. 123-139. danziger, j. n., and kraemer, k. l. (eds.). 1991. survey research and multiple operationalism. the urbis project methodology. boston, ma: harvard business school press. davis, f. d. 1989. "perceived usefulness, perceived ease of use and user acceptance of information technology," mis quarterly (13:3), pp. 319-340. desanctis, g., and poole, m. s. 1994. "capturing the complexity in advanced technology use: adaptive structuration theory," organization science (5:2), pp. 121-147. divol, r., edelman, d., and sarrazin, h. 2012. "demystifying social media," in: mckinsey quarterly. mckinsey & company. fornell, c., and larcker, d. f. 1981. "evaluating structural equation models with unobservable variables and measurement error," journal of marketing research (18:1), pp. 39-50. gable, g. g. 1994. "integrating case study and survey research methods: an example in information systems," european journal of information systems (3:2), pp. 112-126. glynn, c. j., huge, m. e., and hoffman, l. h. 2012. "all the news that's fit to post: a profile of news use on social networking sites," computers in human behavior (28:1), pp. 113119. hanna, r., rohm, a., and crittenden, v. l. 2011. "we're all connected: the power of the social media ecosystem," business horizons (54:3), pp. 265-273. herzberg, f., mausner, b., and snyderman, b. 1959. the motivation to work. new york: wiley. hirst, m. 2011. news 2.0: can journalism survive the internet? crows nest, nsw 2065, australia: allen & unwin. hogg, m. a., and turner, j. c. 1985. "interpersonal attraction, social identification and psychological group formation," european journal of social psychology (15:1), pp. 51-66. hong, s. 2012. "online news on twitter: newspaper's social media adoption and their online readership," information economics and policy (24:1), pp. 69-74. hsieh, p.-a. j. j., and wang, w. 2007. "explaining employees' extended use of complex information systems," european journal of information systems (16), pp. 216-227. hsu, c.-l., yu, c.-c., and wu, c.-c. 2014. "exploring the continuance intention of social networking websites: an empirical research," information systems and e-business management (12:2), pp. 139-163. huang, e. 2009. "the causes of youths' low news consumption and strategies for making youths happy news consumers," convergence: the international journal of research into new media technologies (15:1), pp. 105-122. hulin, c. l., and smith, p. a. 1967. "an empirical investigation of two implications of the two-factor theory of job satisfaction," journal of applied psychology (51:5, pt.1), pp. 396402. australasian journal of information systems palekar &sedera 2018, vol 22, research on recent advances in social media influence of sbns on news consumption 22 jasperson, j. s., carter, p. e., and zmud, r. w. 2005. "a comprehensive conceptualization of post-adoptive behaviors associated with information technology enabled work systems," mis quarterly (29:3), pp. 525-557. kane, g. c., alavi, m., labianca, g., and borgatti, s. p. 2014. "what's different about social media networks? a framework and research agenda," mis quarterly (38:1), pp. 275304. kang, j., and lee, h. 2017. "modeling user interest in social media using news media and wikipedia," information systems (65), pp. 52-64. kaplan, a. m., and haenlein, m. 2012. "social media: back to the roots and back to the future," journal of systems and information technology (14:2), pp. 101-104. karahanna, e., straub, d. w., and chervany, n. l. 1999. "information technology adoption across time: a cross-sectional comparison pf pre-adoption and post-adoption beliefs," mis quarterly (23:2), pp. 183-213. keszei, a. p., novak, m., and streiner, d. l. 2010. "introduction to health measurement scales," journal of psychosomatic research (68:4), pp. 319-323. kietzmann, j. h., hermkens, k., mccarthy, i. p., and s., s. b. 2011. "social media? get serious! understanding the functional building blocks of social media," business horizons (54), pp. 241-251. kim, s. s., and malhotra, n. k. 2005. "a longitudinal model of continued is use: an integrative view of four mechanisms underlying postadoption phenomena," management science (51:5), pp. 741-755. kling, r. (ed.) 1991. adapting survey methods to study the social consequences of computerization: a response to attewel and rule based on five survey-based studies. boston, ma: harvard business school press. krums, j. j . 2009. "http://twitpic.com/135xa there's a plane in the hudson. i 'm on the ferry going to pick up the people. crazy. [tweet]." https://twitter.com/jkrums/status/1121915133. kwak, h., lee, c., park, h., and moon, s. 2010. "what is twitter, a social network or a news media?," in: world wide web 2010. raleigh, north carolina, usa.: international world wide web conference committee (iw3c2), pp. 591-600. latané, b. 1981. "the psychology of social impact," american psychologist (36:4), pp. 343-356. lee, c. s., and ma, l. 2012. "news sharing in social media: the effect of gratifications and prior experience," computers in human behavior (28:2), pp. 331-339. lee, p. s. n., and leung, l. 2008. "assessing the displacement effects of the internet," telematics and information (25:25), pp. 145-155. lee, s., shin, b., and lee, h. g. 2009. "understanding post-adoption usage of mobile data services: the role of supplier-side variables," journal of the association for information systems (10:12), pp. 860-888. li, h. l., and ng, v. t. y. 2013. "discovering associations between news and contents in social network sites with the d-miner service framework," journal of network and computer applications (36:6), pp. 1651-1659. australasian journal of information systems palekar &sedera 2018, vol 22, research on recent advances in social media influence of sbns on news consumption 23 lin, t.-c., and huang, s.-l. 2014. "understanding the determinants of consumers' switching intentions in a standards war," international journal of electronic commerce (19:1), pp. 163189. liu, c.-t., guo, y. m., and lee, c.-h. 2011. "the effects of relationship quality and switching barriers on customer loyalty," international journal of information management (31:1), pp. 71-79. lokuge, s., and sedera, d. 2014a. "enterprise systems lifecycle-wide innovation," americas conference on information systems (amcis 2014), savannah, georgia: ais. lokuge, s., and sedera, d. 2014b. "enterprise systems lifecycle-wide innovation readiness," pacific asia conference on information systems (pacis 2014), chengdu, china: ais. lokuge, s., and sedera, d. 2016. "is your it eco-system ready to facilitate organizational innovation? deriving an it eco-system readiness measurement model," the international conference on information systems (icis2016), dublin, ireland: ais. lokuge, s., sedera, d., and grover, v. 2016. "thinking inside the box: five organizational strategies enabled through information systems," pacific asia conference on information systems (pacis 2016), chiyai, taiwan: ais. maddox, r. n. 1981. "two-factor theory and consumer satisfaction: replication and extension," journal of consumer research (8:1), pp. 97-102. majchrzak, a., faraj, s., kane, g. c., and azad, b. 2013. "the contradictory influence of social media affordances on online communal knowledge sharing," journal of computermediated communication (19:1), pp. 38-55. nunnally, j. c., bernstein, i. h., and berge, j. m. t. 1967. psychometric theory. new york: mcgraw-hill. ortiz de guinea, a., and markus, m. l. 2009. "why break the habit of a lifetime? rethinking the roles of intention, habit, and emotion in continuing information technology use," mis quarterly (33:3), pp. 433-444. palekar, s., atapattu, m., sedera, d., and lokuge, s. 2015. "exploring spiral of silence in digital social networking spaces," the international conference on information systems (icis 2015), fort worth, texas: ais. park, s. c., and ryoo, s. y. 2013. "an empirical investigation of end-users’ switching toward cloud computing: a two factor theory perspective," computers in human behavior (29:1), pp. 160-170. pentina, i., and tarafdar, m. 2014. "from “information” to “knowing”: exploring the role of social media in contemporary news consumption," computers in human behavior (35), pp. 211-223. pew research center. 2013. the role of news on facebook: common yet incidental, pew research center, washington d.c, united states, pp. 1-24. ren, y., harper, f. m., drenner, s., terveen, l., kiesler, s., reidl, j., and kraut, r. e. 2012. "building member attachment in online communities: applying theories of group identity and interpersonal bonds," mis quarterly (36:3), pp. 841-864. rogers, e. m. 1995. diffussion of innovations, (4th ed.). new york: the free press. australasian journal of information systems palekar &sedera 2018, vol 22, research on recent advances in social media influence of sbns on news consumption 24 salim, s. a., sedera, d., sawang, s., and alarifi, a. 2014. "technology adoption as a multistage process," australasian conference on information systems (acis), auckland, new zealand: acis. schouten, l., grol, r., and hulscher, m. 2010. "factors influencing success in qualityimprovement collaboratives: development and psychometric testing of an instrument," implementation science (5:1), p. 84. sedera, d. 2006. "enterprise systems success measurement model," in: school of information systems. brisbane: queensland university of technology, p. 456. sedera, d., gable, g., and chan, t. 2003. "survey design: insights from a public sector-erp impact study," proceedings of the 7th pacific asia conference on information systems, j. hanisch, d. falconer, s. horrocks and m. hillier (eds.), adelaide, australia: association for information systems, pp. 595-610. sedera, d., lokuge, s., atapattu, m., and gretzel, u. 2017. "likes—the key to my happiness: the moderating effect of social influence on travel experience," information & management (54:6), pp. 825-836. sedera, d., lokuge, s., grover, v., sarker, s., and sarker, s. 2016a. "innovating with enterprise systems and digital platforms: a contingent resource-based theory view," information & management (53:3), pp. 366–379. sedera, d., lokuge, s., salleh, n., moghavvemi, s., and palekar, s. 2016b. "spoilt for choice: when user-system relationship becomes one-to-many," the international conference on information systems (icis2016), dublin, ireland: ais. setia, p., venkatesh, v., and joglekar, s. 2013. "leveraging digital technologies: how information quality leads to localized capabilities and customer service performance," mis quarterly (37:2), pp. 565-a564. shi, z., rui, h., and whinston, a. b. 2014. "content sharing in a social broadcasting environment: evidence from twitter," mis quarterly (38:1), pp. 123-142. velichety, s., and ram, s. 2013. "a cross-sectional and temporal analysis of information consumption on twitter," thirty fourth international conference on information systems, milan, italy. venkatesh, v., and davis, f. d. 2000. "a theoretical extension of the technology acceptance model: four longitudinal field studies," management science (45:2), pp. 186-204. venkatesh, v., thong, j. y. l., and xu, x. 2012. "consumer acceptance and use of information technology: extending the unified theory of acceptance and use of technology," mis quarterly (36:1), pp. 157-178. vidich, a. j., and shapiro, g. 1955. "a comparison of participant observation and survey data," american sociological review (20:1), pp. 28-33. walther, s., sarker, s., sedera, d., and eymann, t. 2013a. "exploring subscription renewal intention of operational cloud enterprise systems-a socio-technical approach," european conference on information systems, utrecht, the netherlands: ais, p. 25. australasian journal of information systems palekar &sedera 2018, vol 22, research on recent advances in social media influence of sbns on news consumption 25 walther, s., sedera, d., sarker, s., and eymann, t. 2013b. "evaluating operational cloud enterprise system success: an organizational perspective," european conference on information systems (ecis 2013), utrecht, p. 16. wasko, m. m., and faraj, s. 2005. "why should i share? examining social capital and knowledge contribution in electronic networks of practice," mis quarterly (29:1), pp. 35-57. williams, l. j., edwards, j. r., and vandenberg, r. j. 2003. "recent advances in causal modeling methods for organizational and management research," journal of management (29:6), pp. 903-936. williamson, k., qayyum, a., hider, p., and liu, y.-h. 2012. "young adults and everyday-life information: the role of news media," library & information science research (34:4), pp. 258-264. wu, s., hofman, j. m., mason, w. a., and watts, d. j. 2011. "who says what to whom on twitter," world wide web, hyderabad, india: international world wide web conference committee (iw3c2), pp. 705-714. xu, c., ryan, s., prybutok, v., and wen, c. 2012. "it is not for fun: an examination of social network site usage," information & management (49), pp. 210-217. ye, c., seo, d., desouza, k. c., sangareddy, s. p., and jha, s. 2008. "influences of it substitutes and user experience on post-adoption user switching: an empirical investigation," journal of the american society for information science and technology (59:13), pp. 2115-2132. yuan, e. 2011. "news consumption across multiple media platforms," information, communication & society (14:7), pp. 998-1016. zikmund, w. g. 2000. business research methods (sixth edition). orlando, florida, usa: the dryden press. zmud, r. w. 1982. "diffusion of modern software practices: influence of centralization and formalization," management science (28:12), pp. 1421-1431. australasian journal of information systems palekar &sedera 2018, vol 22, research on recent advances in social media influence of sbns on news consumption 26 appendix a demographics gender  female  male age i was born  before 1964  between 1965 and 1978  between 1979 and 1989  1990 or after location i live in  australia / new zealand  north america / canada  europe  middle east  south east asia  china / north east asia  indian sub-continent / sri lanka  africa please indicate below the extent and frequency of your news related activities news is defined as information on events and issues that include more than just friends and family social media refers to facebook, twitter or similar traditional news media refers to televison, news website, newspaper and radio australasian journal of information systems palekar &sedera 2018, vol 22, research on recent advances in social media influence of sbns on news consumption 27 [relative advantage of social broadcasting networks] question: i use social media for news because… items scale never sometimes always 1 2 3 4 5 6 7 ra1: the news is more current (e.g. breaking news, headline news) compared to traditional news mediums (e.g. t.v)   ra2: the news is available immediately as compared to pre-scheduled news on traditional news mediums (e.g. t.v)   ra3: i like personalized opinions or eyewitness accounts of people, as they give me a better understanding of what is happening in the world compared to news provided by news companies   ra4: i can surf, browse, read, follow or subscribe to tweets or posts of people with expert insights, knowledge or reputation (e.g. celebrities, politicians, ceos), which is not possible on traditional news mediums   [user influence in social broadcasting networks] question: i get my news through social media… items scale never sometimes always 1 2 3 4 5 6 7 ui1: by reading, browsing or surfing the news tweeted or posted by others   ui2: by following tweets or post of reporters from news companies   ui3: by following tweets or posts of people with expert insights, knowledge or reputation (e.g. celebrities, politicians, ceos)   ui4: by reading, browsing or surfing the news tweeted or posted only by news companies  australasian journal of information systems palekar &sedera 2018, vol 22, research on recent advances in social media influence of sbns on news consumption 28 [continuing news consumption in social broadcasting networks] question: when i see news from news companies on social media or when my friends post or tweet news on social media… items scale never sometimes always 1 2 3 4 5 6 7 cnt1: i check my friend’s tweets or posts for more information or for verifying the news   cnt2: i read the news and then share it (e.g. re-tweet or re-post) on social media   cnt3: i follow tweets or posts of people with expert insights, knowledge or reputation (e.g. celebrities, politicians, ceos)   cnt4: i comment on the news that my friends post or tweet   cnt5: i click ‘like’ on the news that my friends post or tweet   [switching news consumption to traditional news mediums] question: when i see news from news companies on social media or when my friends post or tweet news on social media… items scale never sometimes always 1 2 3 4 5 6 7 swt1: i go to their traditional news mediums (e.g. tv channels) for more information   swt2: i watch tv, read newspapers, listen to radio, surf or browse news websites to verify the news   swt3: i search traditional news mediums (e.g. t.v, news websites) for more information   swt4: i watch, listen, read, browse news on traditional news mediums (e.g. tv or news websites) for more information   australasian journal of information systems palekar &sedera 2018, vol 22, research on recent advances in social media influence of sbns on news consumption 29 [comparative duration of news consumption on social broadcasting networks] question: please indicate the comparative time you spend on social media items scale never sometimes always 1 2 3 4 5 6 7 cnc1: i spend more time every day reading news on social media compared to traditional news mediums (e.g. t.v or news websites)   cnc2: i spend more time every day checking news on social media compared to traditional news mediums (e.g. t.v or news websites)   cnc3: i spend more days in a week for all my news related activities (e.g. reading, commenting, tweeting or posting) on social media compared to reading, watching or listening to news on traditional news mediums (e.g. t.v or news websites)   copyright: © 2018 palekar &sedera. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. paper title (paper title style) australasian journal of information systems mclachlan, craig, coldwell-neilson 2016, vol 20, research article students’ computing use & study students' computing use and study: when more is less christine a. mclachlan deakin university, geelong, australia. annemieke craig deakin university, geelong, australia. jo coldwell-neilson deakin university, geelong, australia. abstract since the turn of the century there has been a steady decline in enrolments of students in senior secondary computing classes in australia. a flow on effect has seen reduced enrolments in tertiary computing courses and the subsequent predictions of shortages in skilled computing professionals. this paper investigates the relationship between students’ computing literacy levels, their use and access to computing tools, and students’ interest in and attitudes to formal computing study. through the use of secondary data obtained from australian and international reports, a reverse effect was discovered indicating that the more students used computing tools, the less interested they become in computing studies. keywords: computing literacy, computing enrolment, computing education, computer use, computer access, computing interest, secondary data. 1 introduction in the 21st century the use of computers has become an expected and integral component of education for all school students in australia. in the thirty or so years since computers were introduced into secondary schools there have been dramatic changes with access increasing from, in many instances, one or no computer per school to one device per student. computing studies have also moved from a specific learning area/subject to an integrated component across the curriculum. currently there is a concern that there are insufficient students at all stages of the computing education pipeline to satisfy the future needs of the industry. sterling (2012) argues that one of the great challenges that currently exists for the australian industry is having enough sufficiently skilled computer professionals, as there is a growing need for people with ict skills in all occupational sectors (australian workforce and productivity agency [awpa], 2013). this study provides an investigation into victorian students’ computer use and access, computing attitude and proficiency, and their enrolment in formal computing education. data from studies conducted over the last fifteen years from multiple large scale public secondary resources has been obtained and re-analysed. this work was motivated by recent trends of declining enrolments in both senior secondary and tertiary computing courses despite the increased accessibility and usage of computing devices within schools and homes. this paper argues that the widespread use of computers has not resulted in an increase in students studying computing, nor even an increase in their proficiency with computers; rather it has had the reverse effect. 2 background a variety of terms are used to describe computing and computing related activities. the term ‘computing’ in this paper is used as a generic term encompassing the entire gamut of the discipline however specific terms such as information technology (it), information systems 1 australasian journal of information systems mclachlan, craig, coldwell-neilson 2016, vol 20, research article students’ computing use & study (is) and information communication technology (ict) are used where more appropriate. an interesting observation, adding to the overall confusion, is that in industry the common nomenclature appears to be ict, in secondary schools it is computing and in post-secondary education it is more frequently it when associated with the science domain and is within the business domain. koppi and naghdy (2009) report that there are multiple stakeholders in post-secondary computing education beyond the students who are undertaking formal courses. at one end of the computing pipeline is the industry which reports a lack of skilled australian workers and increasing reliance on foreign workers. at the other end is the education system, where students in primary and secondary schools use computing devices as tools, and computing skills and knowledge are regarded as a general capability in the curriculum. a career in the ict industry has many positive aspects to offer tertiary graduates including a diverse range of roles and strong employment prospects (australian government 2011). however with less young people completing relevant degrees (lynch 2007; mclachlan, craig & coldwell 2010) the industry is suffering through a shortage of suitably trained and skilled personnel resulting in 'a labour shortage' that impacts on the management and potential failure of major projects (australian government 2007, p. 21). last century, pearcey (1988, p. 125) argued that the 'demand for people with computing expertise has always outstripped the capacity of the tertiary sector to supply it' and now with fewer students undertaking such studies the situation seems to be worsening and unlikely to change. while the extent of the skill shortages in the ict industry in australia is inconclusive, the department of immigration has advertised for computer related professionals on its skills australia needs skilled occupation list, enabling the migration to australia of both temporary and permanent ict skilled workers (australian government & department of immigration and border protection n.d.-a). the list details a four-digit level australian and new zealand standard classification of occupations (anzsco) unit group number and brief occupation description, followed by the set ceiling value of applications and the number of requests issued to date. the australian computer society (acs) has authorised the assessment of ict occupations for the anzsco (australian computer society n.d.). consequently there has been an increase in the migration of skilled ict professionals over the last decade (acs 2013; awpa 2013). recent media reports suggest that ict employees on 457 visas are replacing australian workers, as they cost less to employ (australian government 2013; pennington 2013). whether employees are foreign workers or australian nationals, multimedia victoria (2012) argues that the supply of qualified ict professionals is decreasing while the demand continues to increase. therefore, there is still a shortage of, and a need for, more ict workers in australia (australian government & department of immigration and border protection n.d.-b). 3 research method this study undertook an examination of school students computing education over the last fifteen years in victoria, australia. secondary data was obtained from key international, national, and state-based studies. the international studies comprised the international computer and information literacy study (icils) conducted in 2013 and secondly the oecd’s programme for international student assessment (pisa) 2000 through to 2012: • the icils investigated students’ computer and information literacy skills to determine their readiness for life in the digital age (fraillon et al. 2014). it focused on students’ computer device use, attitude, and proficiency (de bortoli et al. 2014). approximately 60,000 eighth grade students with a mean age of 13.5 years at testing time took part, including 5326 australian students (de bortoli et al. 2014). 2 australasian journal of information systems mclachlan, craig, coldwell-neilson 2016, vol 20, research article students’ computing use & study • pisa includes an ict familiarity questionnaire which examines student access to, how they use, and what they feel when using computing devices (thomson & de bortoli 2007). pisa began triennial student assessments in 2000; by 2012, approximately 510,000 15 year old students from 65 countries participated (australian council for educational research [acer], 2014). national data was sourced from acer as well as the australian curriculum, assessment and reporting authority (acara). acer has a long history of providing evidenced-based research to improve learning across students’ ‘lifespan’ from all levels of schooling through to vocational and workplace education. acara, a newer body formed in 2009, is responsible for the governance of the national curriculum, the national assessment program (nap) as well as national data collection and reporting (acara 2013). the victorian data used in this study was obtained from the victorian curriculum and assessment authority (vcaa), an organisation which aims to provide quality curriculum, assessment and reporting for all of the state’s school students (vcaa 2013). the vcaa is involved in administering the nap, including tri-annual tests for ict literacy known as the napictl. in 2011 over 11,000 australian students aged approximately 12 and 16 years took this test which focused on computer use, skill development, creativity, and confidence (acara 2012b). a total of 13,494 victorian students have participated in either the napictl, pisa or icils from 2000 to 2013 (see table 1). year napictl icils pisa napictl age 12 years 13.5 years 15 years 16 years 2000 810 2003 2354 2005 575 593 2006 2272 2008 898 850 2009 2296 2011 930 878 2012 2406 2013 813 total 2494 813 7866 2321 table 1: victorian students surveyed for napictl, pisa, and icils. the vcaa is also responsible for the victorian certificate of education (vce), which is a qualification spanning the final two years of secondary school for students aged 17 to 18. approximately 50,000 senior victorian students complete their vce each year (vcaa 2013). a vce ‘study’ consists of 4 units per subject. during their first year students normally study units 1 and 2, with units 3 and 4 in their second year (vcaa 2011). senior students can combine studying vocational education certificates (vetis) alongside their vce, giving them the opportunity of gaining nationally recognised qualifications (vcaa 2010). the vocational education program began in the 1990’s in three areas, expanding to include more than 30 areas with over 390 certificates (vcaa 2013). in 2012 almost 70,000 senior victorian students studied a vocational education certificate varying from level i to iv (vcaa 2013). 3 australasian journal of information systems mclachlan, craig, coldwell-neilson 2016, vol 20, research article students’ computing use & study vce, vetis and naplan data in this paper only includes those students who achieved the minimum standard or greater. combining data from the above sources forms a picture of victorian students’ interests and interactions with computing from the ages of 12 to 18. the data collected therefore is all secondary data which is publically available. this provides both advantages and disadvantages for research. obtaining data through secondary sources grants the researcher access to a variety of current and historical information, procures information previously collected, saves researchers time and money, eliminates the need for ethical approval, and provides 'rich potential' (cowton 1998, p. 427) and 'valuable insights' for researchers (oakshott 2012, p. 431). data collected in large amounts reduces the likelihood of bias and overcomes low response rates (cowton 1998). the use of on-line government and public sector produced information provides researchers with confidence that the data is trustworthy, valid, reliable and non-identifiable (oakshott 2012). however, utilizing data from secondary sources may provide an incomplete picture, display unneeded data, be timeconsuming, and may cause the researcher to feel that they lack control of data collection and presentation (cowton 1998). the data quality may be questionable, yet when using official sources this is less of a problem (oakshott 2012). further issues with secondary data is that it may be duplicated in other databases, or be presented in different formats, contributing to the problem of data inconsistency (atkinson & brandolini 2001). to overcome any perceived shortcomings in the use of secondary data, this research has used data that is available from australian government sources and large international organisations only, therefore increasing data validity and reliability and reducing possible inconsistencies. analysing or re-analysing multiple secondary data sources can be achieved in a number of ways. cowton (1998, p. 426) suggests that 'reanalysis can involve more than one set of research results. one of the benefits of this is that it generates a larger effective sample than an individual study in isolation'. this paper uses descriptive statistical techniques for data analysis, the aims of which are 'to provide images and/or summaries that can help the reader understand the nature of the variables and their relationship' by making predictions that lead to inferences about the population studied (tashakkori & teddlie 1998, p. 113). the following sections present and examine statistics on student computing studies throughout school education in victoria focussing on students’ computing literacy outcomes, senior secondary students’ computing studies, computer use and access, and attitudes towards computers and computing. 4 computing education courses on programming and the application of computers were first introduced at several universities in the late 1950s (sale 1994), although it was not until the 1970’s and 1980’s that computers began to seep into australian secondary schools (larkin & finger 2011; tatnall & davey 2006). in victoria in 1985 the state board of education (1985) directed all schools to provide students with opportunities to develop their computing confidence. however, when computing instruction began in schools, access was often restricted to students in particular subjects with the technology being isolated in one room (state board of education 1985). until the mid-1980s many schools were not able to provide computers for student use; or if they did, one computer was often shared by many, if not all, students in the school (tatnall & davey 2006). despite computing now being taught throughout primary and secondary school, koppi et al. (2008) argue that there is an insufficient number of students studying computing at all levels. critical issues identified in the literature include cross-curricular teaching methods, a lack of specialist and inspirational computing teachers, poor promotion of computing in schools, and a lack of parental encouragement (for example australian government 2013; genrich, toleman & roberts 2014; koppi et al. 2008; lynch 2007; sterling 2012). over a quarter of a century ago, educational planners suggested that targets be set to increase the number of students participating in technology, mathematics and science (working group on education for science and technology to the victorian government through the hon. ian cathie 1987). 4 australasian journal of information systems mclachlan, craig, coldwell-neilson 2016, vol 20, research article students’ computing use & study attempts continue to be made by the industry, government departments and researchers to encourage greater numbers of students to take up computing studies. computing curricula in secondary schools have been described as irrelevant, mundane, uninteresting, and repetitive (craig 2006; genrich, toleman & roberts 2014; lynch 2007). students report that they continue to work on the same software year after year doing presentations, essays, and worksheets (fraillon et al. 2014), and when students do learn exciting and fun content they find that there is no continuing course (genrich, toleman & roberts 2014). younger secondary students have a narrow understanding of what senior computing education entails and have formed their perception of the discipline based on their junior years’ experiences and hearsay, although the subject content is different in the senior years (lynch 2007). genrich, toleman and roberts (2014) report that some middle year teachers tend to teach computing out of a book as it is not one of their specialisations, leaving students with incorrect opinions of computing studies and the ict work environment. further, in australia secondary school computing subjects are not designed as a pathway into university and are often not required as prerequisite subjects for entry into tertiary computing courses (lynch 2007). aligned with reducing interest in computing studies, enrolments in tertiary computing courses are also decreasing at a time when there are increasing numbers of students attending tertiary institutions. the phenomenon of declining secondary and tertiary enrolments is not a local problem; the issue is evident nationally and internationally (downes & kleydish 2007; genrich, toleman & roberts 2014; koppi et al. 2008; lomerson & pollacia 2006). 5 students computing decision influences students consider computing subjects to be 'irrelevant to their aspirations' and view computing as a 'bad strategic choice' for 'long-term career prospects' (lynch 2007, p. 10), despite technology becoming part of everyday life (george, valacich & valor 2005). genrich, toleman and roberts (2014) assert that there is little encouragement from parents and insufficient incentives from teachers for students to seriously consider computing careers. students rely on information from others as well as the image of an industry when making decisions about career choices. the computing industry has been stereotypically portrayed as nerdy, blokey, geeky, boring, too technical, working long hours and in isolation for too long, as well as there being significant confusion about what computing workers really do, resulting in shying away from the industry (see for example harris & wilkinson 2004; lynch 2007; rommes et al. 2007). the reality is that working in computing is nothing like these stereotypes, it involves working a 'rewarding career' in a dynamic environment (tate 2012). employers look for people with a broad range of skills in the so-called soft areas such as communication and teamwork, as well as the required qualifications and technical skills (australian government 2011, 2013). teachers are crucial to student learning and the quality of education provided is dependent on quality and inspirational teachers and their schools (sterling 2012). teachers’ enthusiasm, computer knowledge, expectations, pedagogical practices, and attitudes toward computing can significantly affect students learning abilities and use of technology (lynch 2007) and change their perceptions of computing (george, valacich & valor 2005). according to hattie (2003, pp. 2-3) 'the answer lies … in the person who gently closes the classroom door and performs the teaching'. recent research adds to this debate. genrich, toleman and roberts (2014) suggest that secondary students gained the opinion from teachers that computing was not interesting and not an appropriate career, while other students that considered themselves as having higher computing competence were likely to be more engaged and show higher interest in following an ict career path (christoph et al. 2015). 5 australasian journal of information systems mclachlan, craig, coldwell-neilson 2016, vol 20, research article students’ computing use & study 6 victorian schooling state-wide, victorian students typically begin school in the year they turn 6 years old (acara 2010) and by the end of their primary education are around 12 years old. in most cases students then enter secondary school and by the end of their compulsory learning years they are around 16. most students continue into post-compulsory learning, also referred to as their senior years, until they are around 18 years old when they commence work or tertiary studies. students in victoria are legally required to undertake school studies until they turn 17 (state government of victoria 2009). nationally, until 2011, all states and territories operated their own educational systems, mostly independently of each other, although from the mid 1990’s they have been working towards common goals (national curriculum board 2009). in 2009 all australian states, territories, and the commonwealth agreed upon a national education system (drabsch 2013) which victorian schools began to implement from 2013 (vcaa 2012). all students in australian schools will eventually be taught using this common curriculum base (national curriculum board 2009). the placement of computing in the victorian curriculum has altered over time. initially it was placed under commerce (curriculum branch 1985; ministry of education (schools division) 1988) and then moved to technology (board of studies 1995). the concept of embedding computing emerged (board of studies 2000), followed by its integration into other subjects (curriculum corporation 2006). computing then moved to interdisciplinary learning (vcaa 2008) where it remains to the present day. in the very near future, with the implementation of the national curriculum, computing will become multidisciplinary (acara 2012a). students at all year levels are expected to learn aspects of computing as a general competency. the availability of computing studies for senior students is determined by individual schools, and would be based on school council approval, equipment, teacher availability, timetabling, and student interest. senior computing studies began in the 1980’s with secretarial studies and electronic typewriting (christophersen 2001), moving onto hsc computer science, computers in science and engineering, and computers in business and government by 1990 (tatnall & davey 2008). the introduction of the two year victorian certificate of education (vce) program in 1990 introduced 2 units each of information technology (first year studies), information processing and management, information systems, and information technology in society (second year studies) (victorian curriculum and assessment board 1990), however the last subject was dropped due to low enrolment numbers in 1998 (christophersen 2001). the current vce information technology curriculum comprises at unit 1 it in action, unit 2 it pathways and at units 3 & 4 information technology applications, and software development (vcaa 2006). 7 student computing literacy computing literacy provides students with the skills and abilities to interact with and operate computing devices effectively. the definition of computing literacy adopted for use in victorian schools is: the ability of individuals to use ict appropriately to access, manage and evaluate information, develop new understandings, and communicate with others in order to participate effectively in society (ministerial council on education employment training and youth affairs [mceetya] & performance measurement and reporting taskforce 2008, p. 2). the definition of computing literacy as used by the icils is 'an individual’s ability to use computers to investigate, create, and communicate in order to participate effectively at home, at school, in the workplace, and in society' (fraillon et al. 2014, p. 17). napictl results show that many students are not meeting the set proficiency level for their age. twelve year old junior students computing literacy results indicate that a significant 6 australasian journal of information systems mclachlan, craig, coldwell-neilson 2016, vol 20, research article students’ computing use & study increase in proficiency occurred between 2005 and 2008, although this was followed by a decrease in 2011. this situation was not repeated with 16 year old middle secondary students; they remained relatively stable in proficiency throughout the three assessment cycles (table 2) (acara 2012b; mceetya 2007; 2010). age 2005 2008 2011 12 year olds 57.9% 66.1% 64.0% 16 year olds 66.5% 69.8% 68.0% table 2: percentage of victorian students at or above napictl proficiency level students who are 13.5 year olds would be expected to achieve the second level of computing literacy on the icils standard. extracted data shows that 19% of victorian students were at or below level 1, 42% level 2, 35% level 3, and 4% level 4; 81% of students met the minimum level for their age. victorian students self-reported that they were very confident at accomplishing basic computing tasks, but less confident when it came to advanced skills. (de bortoli et al. 2014) 8 senior secondary computing studies involvement data about senior secondary students who successfully completed vce computing education studies are presented in table 3 (vcaa 2013). while the number of students undertaking the vce has increased slightly from 46,000 to 50,000 between 2001 and 2013, there has been a significant decline in students undertaking vce computing studies. these include information technology (it); information processing and management (ipm) prior to 2006, and information technology applications (ita) from 2006; and information systems (is) prior to 2006 and system development (sd) from 2006. year computing subjects available to vce students satisfactor y completio n vce (all students) it unit 1 it unit 2 ipm unit 3 ipm unit 4 is unit 3 is unit 4 2001 17113 16354 13344 12963 3346 3266 46,308 2003 11940 11142 9772 9431 2842 2763 47,585 2006 6983 6878 5174 5027 1819 1771 48,352 ita unit 3 ita unit 4 sd unit 3 sd unit 4 2007 6508 6311 4516 4374 1500 1456 47,069 2010 5258 5171 3495 3381 1208 1179 50,057 2013 4304 4177 2595 2529 1335 1304 50,014 % change from 2001 to 2013 -74.8% -74.4% -80.5% -80.5% -60.1% -60.1% +1.1 table 3: vce and vce it satisfactory completions students who undertake computing studies within vetis are able to choose from a variety of certificates, which range from level i to iv, in topics such as it, digital media, and computer systems. students may study more than one type of vetis certificate during their vce. the 7 australasian journal of information systems mclachlan, craig, coldwell-neilson 2016, vol 20, research article students’ computing use & study type and number of it certificates on offer have increased and changed focus over the years, for example in 2001 five certificates were available which increased to 11 by 2013 (vcaa 2013). in 2013 there was almost a three-fold increase in the number of students enrolled in vetis certificates compared to 2001, although the number of enrolments in vetis it studies had decreased during this time (table 4). year vetis it certificate enrolments all vetis certificate enrolments % it of all vetis enrolments 2001 3692 24,685 14.9% 2013 2328 68,463 3.4% % change from 2001 to 2013 -36.9% +277.3% table 4: vetis it and all vetis certificate enrolments 9 student use of and access to computing devices in addition to testing students on their computing knowledge, napictl testing also included a section on student use of and access to ict. student frequency of computer usage has increased for both 12 and 16 year olds over the past three testing cycles (table 5) (acara 2012b; mceetya 2007; 2010). age year computer used > than 5 years frequency of computer use usage (mean days pm) used daily at home at school at home at school 12 years 2005 60.4% 13.4 10.4 2008 64.4% 18.7 14.3 54.3% 21.4% 2011 70.0% 63.0% 39.0% 16 years 2005 69.9% 15.6 11.6 2008 70.6% 23.0 13.7 73.4% 31.7% 2011 79.0% 83.0% 50.0% table 5: (napictl) computer usage by 12 and 16 year old students in 2013, students who completed the icils indicated how long they had been exposed to computing devices. victorian students who had used a computer for more than 5 years accounted for 80%, 89% used a computer at least once a week at home and similarly, 89% used computers at school more than weekly. almost all students had internet access at home (99%), while every student had internet access at school. computer access was high, 94 % of students reported having access to two or more computers at home, all students had access to computer devices at school, with 66 % indicating they had access to their own device (de bortoli et al. 2014). student computer use and access has progressively increased for 15 years olds since pisa testing began (table 6). the pisa questionnaire structure has changed over the years, in the later years the format of some questions were worded differently to reflect a changed focus. where there was significant change in the wording the results are not reported here. the 8 australasian journal of information systems mclachlan, craig, coldwell-neilson 2016, vol 20, research article students’ computing use & study number of victorian homes with more than one computer has increased slightly, from an already high rate (93.1%) in 2000. daily internet use at school has increased significantly, as has home internet connections and home computer use. the number of students who have used a computer for more than 5 years has also increased since 2003. year 2000 2003 2006 2009 2012 have at least one computer at home 93.1% 95.1% 96.4% 96.7% 97.8% have an internet connection at home 68.6% 86.2% 90.6% 94.9% 96.3% use the internet at school more than once a week 67.8% 71.4% 70.4% 92.2% 95.8% used computers for more than 5 years 69.5% 81.9% 87.8% used a computer at home almost every day 50.9% 53.7% 75.5% used a computer at school almost every day 17.2% 14.1% 23.9% table 6: pisa 15 year old computer access and use 10 attitudes and interests toward computing students from all data sources were asked the same computing attitude questions as those used in the pisa questionnaire. this provides good consistency of results over the three sources. data adjustments have been made for three particular years so a comparison can be made over the years and age groups. data for 2005 and 2008 were presented differently in napictl reports; for 2005 an average was calculated by gender, and for 2008 findings were given as an average over both age groups (12 and 16 year olds). the icils in 2013 did not provide victorian percentage results, although those from australia have been provided, these were the same in scale scores as victoria. in addition to these data changes, two years have been omitted. pisa 2006 did not include computer attitude questions, and 2012 pisa questions were not comparable with those listed and were therefore not included. table 7 provides the percentage of students who replied as agreeing to and strongly agreeing to the computer attitude statements. it can be seen that students generally have positive attitudes towards using computers whether for work or play. however, their interest in technology generally has remained relatively static, although with some significant fluctuations over the 14 year period. year age (years) it is very important to work with computers work or play with computers is fun i use computers because i feel interested in technology 2000 15 70.5% 81.1% 64.3% 2003 15 83.1% 84.2% 70.8% 2005 12 84.7% 96.0% 80.3% 16 85.9% 87.8% 69.9% 2008 mixed 83.5% 91.2% 53.4% 2009 15 74.7% 81.3% 45.0% 2011 12 81% 95% 65% 16 80% 91% 59% 2013 13.5 88% 93% 65% table 7: student attitudes towards computing 9 australasian journal of information systems mclachlan, craig, coldwell-neilson 2016, vol 20, research article students’ computing use & study 11 discussion computing literacy levels for 12 and 16 year old students provide a somewhat depressing result. today’s society and education systems have an expectation that students are able to operate computing devices effectively. the napictl findings indicate that many students are still not competent in computing literacy; less than two thirds of victorian 12 year old students and only a slightly higher proportion of 16 year olds have been able to meet computing literacy expectations in 2011. the findings also suggest that students in 2011 had computing literacy skills at a similar level as students in 2005, despite increased access to computing devices. the napictl results back-up what was reported by the acs (2009) in reference to students’ inadequate preparation with computing skills, which seems to be impacting on choices in study prior to entering into senior years. details of napictl computing literacy in victoria predict that this is an area of educational concern and needs to be addressed, possibly by increasing the emphasis on student computing literacy learning by providing adequate training for teachers. icils results for 13.5 year olds provide a different and encouraging picture; just over 80% of students met their age level computing literacy requirements. positioning these students onto a computing literacy continuum for 12 to 16 years olds tested under napictl, indicates that they are performing better than expected. this surprising result from the same cohort of students, although at a different age, may be due to a number of factors such as the testing mechanism used, the items tested, question format, presentation and wording, and testing environment. despite a substantial increase in students completing vce, there has been a massive downturn in those completing computing studies at this level (see table 3), despite a steady increase in the number of senior computing students before 2001 (christophersen 2001). students are 75% less likely to undertake information technology units in 2013 than they were in 2001, 80% less likely to study information technology applications, and 60% less likely to study information systems. in the few instances where there were slight increases, these were very short lived with little indication of students continuing on with further computing studies. students are decreasingly less interested in studying senior computing, despite an increase in the number of senior secondary students. does this reflect the perceived relevance, amongst students, of the computing curricula in secondary schools? the dramatic decrease in vce computing completions occurred despite computing being taught to all students during their first ten compulsory learning years at schooling. the significant drop in enrolment numbers at senior levels will ultimately contribute to vce computing units being removed from the curriculum in the future, aggravating the skill shortages problem. the decrease in student numbers at senior levels has already been reflected in fewer students undertaking computing studies in higher education, contributing to a lack of qualified graduates entering the work force (christophersen 2001). anecdotal evidence from the authors' own experiences suggest that schools are already removing senior computing studies from their portfolio of offerings. the question has to be asked, but cannot yet be answered, will the new it curriculum that is being deployed nationwide make a difference to students’ perceptions of computing and the ict industry, increasing student engagement with the discipline? a similar situation exists in the vetis sector; the increasing number of students studying a variety of certificates has tripled yet over a third fewer are venturing into computing related certificates (table 4). there seems to be an ongoing issue in attracting students into computing courses at both the senior secondary and post-secondary levels, with the situation seemingly worsening every year (australian government n.d.; vcaa 2013). student use and access to computing devices has definitively increased since 2000; the findings in all the studies examined reflect this. many 12 year old students indicated that they have been using computers for at least five years suggesting that most started using computers before they were seven. these students were using computers almost daily at home and at 10 australasian journal of information systems mclachlan, craig, coldwell-neilson 2016, vol 20, research article students’ computing use & study school. similarly almost 80 % of 16 year olds reported using computers for longer than five years, and accessing them regularly at home and school. continual growth has occurred in these students using computers for more than five years, and older students are more habitual users than younger ones both at school and at home. napictl students experience in, and frequency of, computer usage has increased dramatically, although this is not reflected in proficiency results; higher usage does not seem to impact computing literacy levels. one has to ask if the definition of computing literacy is appropriate in this day and age. computer access and use for 13.5 year olds showed similar results. most students had access to at least two computers at home that were connected to the internet that were used frequently; all students had computer and internet access at school through a range of computing devices. most of these students (80%) had used a computer for more than five years. according to the pisa data, the number of victorian homes that have more than one computer at home has increased slightly since 2000 but, not surprisingly, internet connectivity has increased significantly; nearly every household has a computing device. daily use of computers and the internet at home and at school is now a regular occurrence. computer use at school in the years 2000, 2003 and 2006 was about a third as often than use at home. the number of 15 year olds with at least 5 years of computer use under their belt has also increased. combining data from all four sources clearly shows a steady increase in computer use and access over the last fifteen years. the computing situation in schools and homes in victoria has certainly changed, considering that thirty-five years ago most schools did not have any computers (tatnall & davey 2006); in 2013 almost every student has access to and use of a computing device for educational, leisure and personal use. the findings related to student interest and attitude to computing highlight some quite remarkable details. a high percentage of students felt it is important to know how to use computers and regarded computers fun to work with; there were only small differences between the ages and years that data was presented for. younger students enjoyed computing more than older students and most students could see the importance of computers for work. interest in technology varied however; overall interest in using computers and technology was significantly lower than the recognition of the importance of working with computers or the entertainment (fun) value that the technology could afford. despite technology interest by 12 year olds being high in 2005, in 2009 it was the lowest recorded for 15 year olds. overall technology interest by students is positive, yet despite these encouraging findings, the disheartening result is that students are still not interested in undertaking further computing study. this suggests that there are other factors influencing student career choices. 12 conclusion this study searched for affiliations between students’ computer literacy levels, senior computing study interest, students’ use and access to computing tools, and students’ interest and attitudes to computing. educational attainment of computing literacy has not greatly improved and is of particular concern in victoria. the number of senior students undertaking computer studies has diminished greatly. meanwhile computer access and use, and computer attitude and interest are on the rise. the presented results show that the situation of decreasing enrolments and interest in computing study areas are not improving; in fact, it is getting worse. the napictl results reported above have not improved, recently released results show no improvement. in victoria, 12 year olds computer literacy level remained constant, while those for 16 year olds have declined to 55% (acara 2015b). mr robert randall, acaras ceo, has called for 'a renewed focus' in relation to declining performance concerns of computing in schools, and for schools to work together and with acara to 'turn this around because this is an important area' (acara 2015a; ford 2015). mr randall commented that although students are 'prolific users' they might not be 'proficient users', and that 'explicit teaching' was required for students to attain 'deeper knowledge and understanding' (ford 2015). it is reasonable to 11 australasian journal of information systems mclachlan, craig, coldwell-neilson 2016, vol 20, research article students’ computing use & study assume that students' computer literacy levels are influenced by what they are taught by their teachers. the integration of computing has brought about the situation where almost all teachers teach computing concepts, although only a small number are qualified to do so (tate 2012). the presented data shows a clear drop in interest and future research is required to find ways in which to engage students in computing studies. the research reported here suggests that school computing subject offerings, the delivery of the computing curriculum, availability of suitably qualified and enthusiastic teachers, minimal computing career encouragement from teachers, and the lack of computing knowledge in non-specialised teachers are ultimately contributing to the lack of interest in senior secondary students. these issues should be considered for future research to investigate how they impact on students’ decisions to undertake further studies in the area. change is certainly needed in the teaching of computing at secondary schools, only time will tell if the new national curriculum digital technology content will solve any of these problems. the differences in outcomes of the various standard measures of computing literacy suggests that maybe what is being measured is not within the skill set that students are developing through their computing activities. young students are using a range of mobile devices to access services and information, are engaging with social media for fun and entertainment and use assorted software to support their educational needs. are the standards of computing literacy an accurate measure of their skills with using technology? do we need to reconsider what this literacy is purporting to measure? do we need to update current perceptions of what computing literacy is (under all its pseudonyms) and construct one or more technology related literacies such as social media literacy, digital literacy, information literacy and so on? enjoying using computers and recognising their potential for work and play is seemingly not sufficient to translate into a career in the ict industry. maybe students’ perception of the ict industry is sufficiently off putting? popular media portrayal of ict professionals and the industry, through programs such as the it crowd and big bang theory reinforce the ongoing perception that ict is for geeks and nerds and not for the everyday student. students career choices are strongly influenced by their parents, amongst others, and their perceptions of the ict industry would have been influenced by the dotcom crash in 2000, the more recent global financial crisis and the ongoing debate regarding outsourcing and offshoring of ict work, all providing strong negative drivers to avoid ict. this study has shown that the widespread access to and use of computers and positive computing attitudes does not result in students increasing their computer study options or their level of computer literacy, rather the reverse is evident. it seems that students may still be carrying stereotypical perceptions about computing, or that frequent use has provided students with an unrealistic perception of their computer abilities. alternatively the pervasiveness and ubiquity of computing technologies may be encouraging a view that there is nothing new or engaging behind the screens that is worthy of their attention. future research needs to address these misconceptions. references atkinson, ab & brandolini, a 2001, 'promise and pitfalls in the use of "secondary" data-sets: income inequality in oecd countries as a case study', journal of economic literature, vol. 39, no. 3, pp. 771-99, retrieved 19/10/2012, http://www.jstor.org/stable/10.2307/2698313 australian computer society 2013, australian ict statistical compendium 2013: inspiring success, sydney, nsw, retrieved 25/7/2014, http://acs.org.au/__data/assets/pdf_file/0004/28570/australian-ict-statisticalcompendium-2013.pdf australian computer society n.d., migration skills assessment, australian computer society, retrieved 9/12/2015, https://www.acs.org.au/migration-skills-assessment 12 australasian journal of information systems mclachlan, craig, coldwell-neilson 2016, vol 20, research article students’ computing use & study australian council for educational research 2014, programme for international student assessment (pisa) australia, australian council for educational research, retrieved 2/10/2014, http://www.acer.edu.au/ozpisa/ australian curriculum, assessment and reporting authority 2010, national report on schooling in australia 2009, retrieved 9/4/2013, http://www.acara.edu.au/reporting/national_report_on_schooling/schools_and_sch ooling/school_structures.html australian curriculum, assessment and reporting authority 2012a, draft shape of the australian curriculum: technologies, australian curriculum, assessment and reporting authority, sydney, nsw, retrieved 15/5/2012, http://www.acara.edu.au/curriculum/technologies.html australian curriculum, assessment and reporting authority 2012b, national assessment program: ict literacy years 6 & 10 report 2011, sydney, nsw, retrieved 28/2/2013, http://www.nap.edu.au/verve/_resources/nap_ictl_2011_public_report_final.pdf australian curriculum, assessment and reporting authority 2013, about us, australian curriculum, assessment and reporting authority, retrieved 7/1/2015, http://www.acara.edu.au/about_us/about_us.html australian curriculum, assessment and reporting authority, 2015a, 2014 nap ict literacy report shows a decline in ict literacy, 10/12/2015, http://www.acara.edu.au/news_media/acara_news/acara_news_2015_11.html#20151 117 australian curriculum, assessment and reporting authority 2015b, national assessment program ict literacy years 6 & 10 report 2014, by australian curriculum, assessment and reporting authority, retrieved 17/11/2015, http://www.nap.edu.au/verve/_resources/d15_8761__napict_2014_public_report_final.pdf department of finance and administration & australian government information management office 2007, meeting the demand for ict skills in the australian public service today and for the future: report of the ict professional and skills development taskforce, by australian government, commonwealth of australia, retrieved 12/10/2008, http://www.amigo.gov.au/ department of education, employment and workplace relations 2011, australian jobs 2011, by australian government, retrieved may 2012, http://www.deewr.gov.au/employment/researchstatistics/pages/australianjobs.aspx department of education, employment and workplace relations 2013, labour market research information and telecommunications (ict) professions, by australian government, retrieved 6/9/2013, http://foi.deewr.gov.au/system/files/doc/other/ictprofessions.pdf department of industry, innovation science research and tertiary education n.d., higher education statistics data cube (ucube), by australian government, retrieved 11/9/2013, http://www.highereducationstatistics.deewr.gov.au/ australian government & department of immigration and border protection n.d.-a, skillselect, australian government, retrieved 9/12/2015, http://www.border.gov.au/trav/work/skil australian government & department of immigration and border protection n.d.-b, subclass 457 temporary work (skilled) visa statistics, retrieved 14/1/2015, http://www.immi.gov.au/media/statistics/statistical-info/temp-entrants/subclass457.htm 13 australasian journal of information systems mclachlan, craig, coldwell-neilson 2016, vol 20, research article students’ computing use & study australian workforce and productivity agency 2013, ict workforce study, australian workforce and productivity agency, retrieved 10/7/2013, http://www.awpa.gov.au/publications/documents/ict-study-final-28-june2013.pdf board of studies 1995, curriculum and standards framework: technology, board of studies, carlton, vic. board of studies 2000, curriculum and standards framework ii: technology, board of studies, carlton, vic, retrieved 8/3/2013, http://pandora.nla.gov.au/pan/99103/200905051614/csf.vcaa.vic.edu.au/te/kote.html christoph, g, goldhammer, f, zylka, j & hartig, j 2015, 'adolescents' computer performance: the role of self-concept and motivational aspects', computers & education, vol. 81, no. 0, pp. 1-12, retrieved 1/12/2014, doi http://dx.doi.org/10.1016/j.compedu.2014.09.004 christophersen, p 2001, 'computer studies in victoria', australian educational computing, vol. 16, no. 1, pp. 42-7, retrieved 27/2/2014, http://acce.edu.au/sites/acce.edu.au/files/pj/journal/aec_vol16_1_2001%20comput er%20studies%20in%20victoria.pdf cowton, cj 1998, 'the use of secondary data in business ethics research', journal of business ethics, vol. 17, pp. 423-4, retrieved 19/10/2012, http://link.springer.com/article/10.1023%2fa%3a1005730825103?li=true craig, a 2006, young girls ict into computing too, australian women in information technology and science entity. curriculum branch, education department of victoria, 1985, curriculum frameworks p-12: an introduction, education department of victoria, melbourne. curriculum corporation 2006, statements of learning for information and communication technologies (ict), doi isbn-13: 978-1-86366-633-6, isbn-10: 1 86366 633 8, <http://www.curriculum.edu.au/verve/_resources/stmntlearning_ict_2008.pdf de bortoli, l, buckley, s, underwood, c, o'grady, e & gebhardt, e 2014, icils 2013: australian students' readiness for study, work and life in the digital age, australian council for educational reasearch ltd, camberwell, victoria, retrieved 25/11/2014, http://research.acer.edu.au/ict_literacy/6/ downes, t & kleydish, f 2007, gait working paper: curriculum, participation and performance in senior secondary computing and it subjects in new south wales, victoria and south australia, charles sturt university, australia, retrieved 27/2/2014, http://www.csu.edu.au/__data/assets/pdf_file/0020/205139/curriculumanalysiswo rkingpaper.pdf drabsch, t 2013, the australian curriculum: briefing paper no. 1/2013, nsw parliamentary research services, february, issn: 1325-5142, isbn: 978-0-7313-1897-1, briefing paper, retrieved 8/3/2013, http://www.parliament.nsw.gov.au/prod/parlment/publications.nsf/0/b18363c26ec 0f93aca257b1800144fde/$file/the%20australian%20curriculum.pdf ford, m 2015, school students' digital skills in decline despite prolific use: report, 17/11/2015, retrieved 17/11/2015, http://www.abc.net.au/news/2015-11-17/school-students-digitalskills-in-decline-despite-prolific-use/6945672 fraillon, j, ainley, j, schultz, w, friedman, t & gebhardt, e 2014, preparing for life in a digital age: the iea international computer and information literacy study international report, springerlink.com, melbourne, retrieved 25/11/2014, http://research.acer.edu.au/ict_literacy/8/ 14 australasian journal of information systems mclachlan, craig, coldwell-neilson 2016, vol 20, research article students’ computing use & study genrich, r, toleman, m & roberts, d 2014, 'impacting it enrolments: what factors most influence student career decisions', paper presented to 25th australasian conference on information systems, auckland, new zealand, 8-10 december, retrieved 9/12/2014, http://aut.researchgateway.ac.nz/bitstream/handle/10292/8144/acis20140_submissi on_134.pdf?sequence=1&isallowed=y george, jf, valacich, js & valor, j 2005, 'does information systems still matter? lessons for a maturing discipline', communications of the association for information systems, vol. 16, no. 1, p. article 8, retrieved 28/3/2008, http://aisel.aisnet.org/cais/vol16/iss1/8/ harris, r & wilkinson, ma 2004, 'situating gender: students' perceptions of information work', information technology & people, vol. 17, no. 1, pp. 71-86, retrieved 31/3/2008, http://www.emeraldinsight.com/insight/viewcontentservlet;jsessionid=2b7111af416 fda543395f0290b0eb3b0?filename=published/emeraldfulltextarticle/articles/16 10170104.html hattie, j 2003, 'teachers make a difference: what is the research evidence?', paper presented to building teacher quality: what does research tell us?, melbourne, vic, october, retrieved 22/5/2014, http://www.acer.edu.au/documents/rc2003_hattie_teachersmakeadifference.pdf koppi, t & naghdy, f 2009, managing educational change in the ict discipline at the tertiary educational level, university of wollongong, strawberry hills, nsw, retrieved 25/3/2010, http://acdict.edu.au/userdocs/koppi_naghdyexecutivesummaryfinal.pdf koppi, t, naghdy, f, chicharo, j, sheard, j, edwards, s & wilson, d 2008, 'the crisis in ict education: an academic perspective', in hello! where are you in the landscape of educational technology? melbourne 2008, pp. 505-8, retrieved 25/5/09, http://www.ascilite.org.au/conferences/melbourne08/procs/koppi-poster.pdf larkin, k & finger, g 2011, 'informing one-to-one computing in primary schools: student use of netbooks', australasian journal of educational technology, vol. 27, no. 3, pp. 51430, retrieved 11/4/2012, http://ajet.org.au/index.php/ajet/article/view/958/234 lomerson, wl & pollacia, l 2006, 'declining cis enrollment: an examination of pre-college factors', information systems education journal, vol. 4, no. 35, pp. 1-13, retrieved 8/2/2010, http://isedj.org/4/35/isedj.4(35).lomerson.pdf lynch, j 2007, 'introduction: exploring the gender and it problem and possible ways forward', in j lynch (ed.), gender and it: on going challenges for computing and information technology education in australian secondary schools, common ground publishing, altona, vic, pp. 1-26. mclachlan, c, craig, a & coldwell, j 2010, 'student perceptions of ict: a gendered analysis', in t clear & j hamer (eds), twelfth australasian computing education conference (ace2010), brisbane, australia, vol. 103, pp. 127-36. ministerial council on education, employment training and youth affairs 2007, national assessment program: ict literacy, years 6 and 10, report 2005, carlton south, vic, retrieved 11/06/09, http://www.mceetya.edu.au/verve/_resources/nap_ictl_2005_years_6_and_10_r eport.pdf ministerial council on education, employment training and youth affairs 2010, national assessment program: ict literacy, years 6 and 10, report 2008, carlton south, vic, retrieved 3/9/2013, http://www.nap.edu.au/verve/_resources/2008_nap_ictl_public_report.pdf ministerial council on education employment training and youth affairs [mceetya] & performance measurement and reporting taskforce 2008, national assessment program information and communication technology literacy years 6 and 10, an 15 australasian journal of information systems mclachlan, craig, coldwell-neilson 2016, vol 20, research article students’ computing use & study assessment domain for ict literacy, ministerial council on education, employment, training and youth affairs, melbourne, retrieved 3/7/09, http://www.mceetya.edu.au/verve/_resources/nap_ictl_2008_assessment_domai n.pdf ministry of education (schools division), victoria, 1988, the school curriculum and organisation framework, p-12: a curriculum for all, ministry of education (schools division), victoria, melbourne, vic. national curriculum board 2009, the shape of the australian curriculum, by national curriculum board, retrieved 21/4/2010, http://www.acara.edu.au/verve/_resources/shape_of_the_australian_curriculum.pd f oakshott, l 2012, essential quantitative methods for business, management and finance, 5th edn, palgrave macmillan, new york, ny. pearcey, t 1988, a history of australian computing, chisholm institute of technology, caulfield, vic. pennington, s 2013, 'employers want a 'cheapie, just arrived off the boat', aussie it workers told', the sydney morning herald, 15/3/2013, it pro, retrieved 22/3/2013, http://www.smh.com.au/it-pro/government-it/employers-want-a-cheapie-justarrived-off-the-boat-aussie-it-workers-told-20130315-2g515.html rommes, e, overbeek, g, scholte, r, engles, r & de kemp, r 2007, ''i'm not interested in computers': gender-based occupational choices of adolescents', information, communication & society, vol. 10, no. 3, pp. 299-319, retrieved 25/1/2013, doi 10.1080/13691180701409838, <http://dx.doi.org/10.1080/13691180701409838 sale, a 1994, 'computer science teaching in australia', in jm bennet, r broomhan, pm murton, t pearcey & rw rutledge (eds), computing in australia: the development of a profession, hale and iremonger pty limited, sydney, nsw, pp. 151-4. state board of education 1985, technology in education, information paper no. 1: an introduction to computers in learning, state board of education, melbourne, vic. state government of victoria 2009, education and training reform amendment (school age) act 2009 17 november 2009, retrieved 10/4/2013, http://www.austlii.edu.au/au/legis/vic/num_act/eatraaa200962o2009475/ sterling, l 2012, how can schools get kids engaged in it, retrieved 13/7/2012, http://www.swinburne.edu.au/chancellery/mediacentre/mediacentre/news/2012/07/how-can-schools-get-kids-engaged-in-it tashakkori, a & teddlie, c 1998, mixed methodology: combining qualitative and quantitative approaches, vol. 46, applied social research methods series, sage publications inc, thousand oaks, ca. tate, n 2012, 'let's learn from the british ict overhaul', the australian, 31 january, business/technology, retrieved 10/4/2015, http://www.theaustralian.com.au/business/technology/lets-learn-from-the-british-ictoverhaul/story-e6frgb0o-1226257642933 tatnall, a & davey, b 2006, 'early computer awareness courses in australian secondary schools curricula from the late 1970s and early 1980s', in j impagliazzo (ed.), ifip 19th world computer congress, wg 9.7, tc9: history of computing, proceedings of the second conference on the history of computing and education, santiago, chile, vol. 215, pp. 107-16. tatnall, a & davey, b 2008, 'computer education support structures in victorian schools in the 1980's', in j impagliazzo (ed.), ifip international federation for information processing, 20th world computer congress, proceedings of the third ifip conference 16 australasian journal of information systems mclachlan, craig, coldwell-neilson 2016, vol 20, research article students’ computing use & study on the history of computing and education wg9.7/tc9, history of computing 3, milano, italy, vol. 269, pp. 1-22, retrieved 24/1/2013, http://download.springer.com/static/pdf/58/chp%253a10.1007%252f978-0-38709657-5_1.pdf?auth66=1380245576_db2fcdaebc1a8eb7ca29948a3b138a0b&ext=.pdf thomson, s & de bortoli, l 2007, pisa 2003 australia: ict use and familiarity at school and home, vol. acer research monoghraph no 62, australian council for educational research, camberwell, vic, retrieved 1/10/2014, https://www.acer.edu.au/files/mono_62_pisa-ict_report.pdf victorian curriculum and assessment authority 2006, information technology: victorian certificate of education study design, victorian curriculum and assessment authority, east melbourne, vic. victorian curriculum and assessment authority 2008, interdisciplinary learning strand: information and communications technology, retrieved 14/5/09, http://vels.vcaa.vic.edu.au/downloads/vels_standards/velsrevisedict.pdf victorian curriculum and assessment authority 2010, an overview of vet in the vce and vcal 2011, east melbourne, vic, retrieved 27/2/2013, http://www.vcaa.vic.edu.au/documents/vet/publications/vetoverview.pdf victorian curriculum and assessment authority 2011, vce and vcal administrative handbook 2012, issn: 1448-4552, east melbourne, vic. victorian curriculum and assessment authority 2012, introducing ausvels, victorian curriculum and assessment authority, retrieved 3/5/2013, http://www.vcaa.vic.edu.au/pages/foundation10/curriculum/intro.aspx victorian curriculum and assessment authority 2013, senior secondary certificate statistical information, state government of victoria, retrieved 5/8/2013, http://www.vcaa.vic.edu.au/pages/vce/statistics/subjectstats.aspx victorian curriculum and assessment board 1990, information technology study design, victorian curriculum and assessment board, carlton, vic. working group on education for science and technology to the victorian government through the hon. ian cathie, minister of education, 1987, improving the science, mathematics and technology education base in victoria. part 1: an overview and recommendations, melbourne, vic. copyright: © 2016 mclachlan, craig, coldwell-neilson. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. 17 http://creativecommons.org/licenses/by-nc/3.0/au/ http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 background 3 research method 4 computing education 5 students computing decision influences 6 victorian schooling 7 student computing literacy 8 senior secondary computing studies involvement 9 student use of and access to computing devices 10 attitudes and interests toward computing 11 discussion 12 conclusion microsoft word ajis vol 15-1 m.doc australasian journal of information systems volume 15 number 1 december 2007 25 generating a gender balance: making introductory information systems courses a positive experience rosemary stockdale institute of information and mathematical sciences massey university, auckland, nz email: r.j.stockdale@massey.ac.nz susan stoney school of management edith cowan university, perth, australia email: s.stoney@ecu.edu.au abstract there is a growing shortfall of graduates entering the it profession. the situation is exacerbated by the continuing decline in the number of women undertaking it related degrees. however, there are an increasing number of students taking business degrees that have a small information systems component, although few students choose to major in is. using a qualitative reflective approach we identify the perceptions and experiences of female undergraduates taking introductory is courses in two universities, one in australia and one in new zealand. we discuss ways of improving the delivery of introductory is courses in order to make information systems more interesting to women undergraduates, thus enhancing their learning experiences and encouraging further uptake of is majors. the paper concludes with some reflections on other influences that impact on the ability of is departments to deliver appropriate introductory courses. introduction there is a real and growing shortage of information technology (it) professionals in the workplace (department of labour, 2005), in both traditional and non-traditional areas such as supply chain management and education (sujdak, 2002). whilst there are over 45000 hits on google scholar regarding the shortage of it professionals generally, there are over 63,000 that specifically address the shortage of women. australasian journal of information systems volume 15 number 1 december 2007 26 the declining number of women in it is well-documented in both the business press and in academic research. the reasons given for the decline are many and varied. armstrong, riemenschneider, ried & allen’s (2004) study discusses women who have followed an it career path but then leave the profession. the authors conclude that the reasons are contained within three loops; managing family, need for flexibility of work schedule and stress. this is borne out by an department of women (australian) report (2002), which found that women who work in the it industry rate it highly for flexibility and career support, but very low when it came to a family friendly workplace. these findings are reflected across all industry sectors where there are a steady stream of women in all professions who opt out of the workplace after five to ten years. work-life balance and difficult workplace environments are major factors in this drain of experienced, professional women leaving their careers (stone, 2007). however, this does not explain why women are not entering the it professions in the first place. fewer women are studying it related courses with a consequent impact on their future careers whilst the work-life balance is an established reason for women leaving the it profession, it does not explain why women are not entering the profession. the lack of interest in it is already evident in university enrolments where the number of women studying it-related courses continues to decrease, with a consequent impact on their future careers. high enrolment levels in business degrees mean that many women are exposed to information systems (is) courses as first year undergraduates. these courses are often compulsory for business students and can be seen as an opportunity to recruit more students to is majors via a business pathway. our research suggests that such courses are unpopular with female undergraduates and do not attract them to consider changing from business studies to it related majors. the contribution of this paper is to determine ways to develop more positive experiences of studying for an it related degree. we concentrate specifically on introductory is courses and determine what steps we, as academics responsible for teaching these courses, need to take to encourage greater uptake of is majors by women. this paper addresses the first steps taken towards a proactive process of change to achieve a greater gender balance in is courses at our two universities, one in new zealand and one in australia. we have taken a qualitative approach to analysing the current situation regarding the delivery and marketing of information systems courses. using our own experiences as course co-ordinators for introductory information systems, we use focus groups and interviews to identify student perceptions of the courses. subsequent analysis of the data, underpinned by existing literature, has enabled us to identify where improvements can be made to encourage a more balanced gender distribution in is degrees. although this paper specifically addresses is courses, the literature encompasses the broader spectrum of it as the issues relating to the it sector include those of is and the latter is rarely addressed separately. the term it is therefore used in its widest sense, although we have avoided literature that specifically addresses computer science and software engineering. the gender gap in information technology “as an industry some of our role models aren’t necessarily the coolest people out there. i’m not sure that bill gates, for example, is the natural turn-on for a 12 year old girl” (katrina troughton, ibm nz managing director) technology is stereotypically seen as a male domain in a popular culture that continues to associate occupations with either a male or a female bias (lee, 2005). despite arguments that gender is no australasian journal of information systems volume 15 number 1 december 2007 27 longer an issue in a dynamic labour market where there is a high demand for is professionals (sumner & niederman, 2004) women account for less than 20% of the it workforce (goodwin, 2006). this has become a political issue as the lack of women in it amounts to a classic example of horizontal occupational segregation (griffiths, keogh, moore, tattersall, & richardson, 2006). griffiths et al., explain that horizontal occupational segregation shows how different groups of people, often defined by gender, tend to work in separate occupations. women are often clustered in traditional careers; teaching, nursing, and domestic service work (anker, 1998 cited in griffiths, et al., 2006). furthermore, within particular industries, women tend to be under-represented at the higher managerial levels (such as chief information officers in is), and are more strongly represented in lower level occupations. this is a identified as vertical gender segregation. much of the research on women in is and it related managerial positions focuses on the workgender imbalance (griffiths et al., 2006; griffiths & moore, 2006; moore, griffiths, & richardson, 2005), citing the unsuitability of the it workplace, the deeply ingrained masculine culture of it, the long hours, “presenteeism”, and difficulties with obtaining part-time employment. the reasons attributed to the gender imbalance also focus on the different attitudes to computer technology where women are seen as computer-phobic or computer reticent (turkle in christensen, knezek, & overall, 2005). it has an image problem where boys who excel at computing are seen as ‘nerds’, ‘geeks’ or ‘trainspotter types’ with a lack of social skills (lee, 2005) while girls are perceived to be more social and find it subjects in school ‘boring’. research suggests that lack of role models and the occupational culture of it project a negative image to women (sumner & niederman, 2004). lack of career information for a technology orientated career is another cited problem (morris, 2002) and research has found that women lack confidence in their own abilities and assess their skills at a lower level than their male peers (hargittai & shafer, 2006). these issues are less prevalent for females with a family member or friend in the it industry as this contributes to a more accurate picture of such careers than those who relied on second hand knowledge (lee, 2005). pinkard adds unequal access and an adverse climate in higher education as barriers to encouraging women into it (2005). she argues that a key cause of the different attitudes towards computer use is the perception of software as male orientated with stereotypical characteristics that did not target or attract women. this identification of software as dominantly male resonates through the literature (baroudi & igbaria, 1995; morris, 2002; trauth, 2002). lee (2005) states that not only software but technology “is gendered from the outside by those appropriating it rather than necessarily on the inside or at the point of design” (p123). this argument supports research into the consideration of two viewpoints into perceptions of it and gender: essentialist theory versus social construction theory (trauth, 2002). essentialist theory is based on the assertion of inherent differences between the nature of men and women. this argues that biological determinants dictate reaction to it and we must therefore concede that women will require different methods of training than men to enter the it workforce (trauth, 2002). the implications for bridging the gender divide at school and university levels are significant. the identification of ‘gendering from the outside’ (lee, 2005) adheres to the theory of social construction. if it is seen as male dominated because of societal perceptions then females are less attracted to the technical workplace and this reinforces the identification of computing as ‘men’s work’ (trauth, 2002). agreeing with this theory and addressing the problems it raises are two very different things. australasian journal of information systems volume 15 number 1 december 2007 28 one significant initiative that addresses the male stereotyping of it is the itbeat project in the uk (lee, 2005). the project aims to make ict glamorous and appealing to girls by introducing appropriate role models, associating fun with computers and using images that relate to girls’ perceptions of themselves. in making computing more of a ‘female domain’ (webster in lee, 2005) the project encourages positive images amongst those girls involved and supports their development of technical knowledge. however, lee highlights the problem of applying feminine stereotyping and developing gender specific affinities rather than unifying social constructs in a genderless context. it/is education and gender “the school curriculum doesn’t have a lot of credibility…..it is not regarded as a prestigious subject. it’s the sort of thing they quarantine the hopeless into, really” (t clear, aut in hendery, 2006) there is widespread concern at the low level of enrolments in it degrees in many countries (denning & mcgettrick, 2005). in a recent study across australia and nz a recruitment firm found that tertiary it qualifications were considered of major importance to 77% of employers surveyed (rossi, 2006). in nz universities the total number of students enrolling for it related degrees has fallen for each of the past four years this was attributed by academics at a recent government conference to the ‘geeky image’ and lack of broad appeal (hendery, 2006). another factor was considered to be the lack of credibility in the school curriculum and the failure of the tertiary sector to adapt to ‘what the kids can do with ict’(hendery, 2006). the drop in student numbers is particularly prevalent amongst women; for example in australia women account for less than one fifth of it graduates (von hellens & nielsen, 2005). there are several factors that are held to influence the low take-up of it degrees by women including social and cultural issues (von hellens & nielsen, 2005), lack of role models (trauth, 2002) and an overwhelmingly male environment (baroudi & igbaria, 1995). research into school age children has found that the girls’ technical abilities are as good, if not better, than boys at primary age (christensen et al., 2005). differences begin to appear after grade 6 and as children reach high school the enjoyment that girls find in using computers falls. more importantly, there is evidence that their self-assessment of skills declines and they are less confident of their technical abilities (hargittai & shafer, 2006; jewell & maltby, 2002; von hellens & nielsen, 2005). nevertheless, girls’ use of email and their level of web skills remain high and appear to be influenced by the social enjoyment aspects gained from online use (christensen et al., 2005). in high school, a distinct difference becomes visible in gender attitudes towards computing and it and the ‘geek’ image gains more credence, with girls finding the studying of it to be boring and difficult (lee, 2005). there is a perception amongst girls of this age that careers in it lead to a solitary work environment with little contact with people and an emphasis on mathematical abilities (von hellens & nielsen, 2005). they appear to believe that a special aptitude for computing is necessary, while males see computing as a technical challenge that can be mastered (jewell & maltby, 2002). however, it appears from jewell & maltby’s study that girls are not deterred by the level of difficulty but rather by the perceived lack of social involvement arising from an it career. girls’ aversion to it related courses may also be affected by the lack of women role models in the teaching profession (von hellens & nielsen, 2005). this point is echoed in trauth’s (2002) call for more women professors of it and sumner and niederman’s (2004) observation that less than 7% of faculty it positions are held by women. some support for the role model argument lies with the australasian journal of information systems volume 15 number 1 december 2007 29 observation that single sex schools have more success in promoting it as a career (trauth, 2002) although it is not clear what percentage of it teaching staff in such schools are women. lang (2003) opens up the role model argument to suggest that the inequality in numbers is also evident in other disciplines such as law and medicine, but that these faculties have few difficulties in attracting female students. therefore image may have a bigger influence than lack of role models on the low numbers of women in the it discipline. these low numbers, of either gender, have serious consequences for the it sector and the broader economy. in 2003, the nz government concluded that the shortage of appropriately educated graduates was a major constraint on the growth of the country’s ict sector (department of labour, 2005); a situation that has not changed. lang (2003) also argues that intervention programmes rarely sustain improvement. the concerns of it education must therefore be addressed at several levels and academics responsible for introductory university it related courses must play a role in making courses more accessible and relevant to students. research design this paper is based on the experiences of two academics, from new zealand and australia, who teach introduction to information systems courses. the australian case is from a school of management information systems within a business faculty while the new zealand study is drawn from an information systems discipline group within a college of sciences. the studies were originally undertaken independently, with the two researchers beginning collaboration after data collection had commenced. in both cases data were gathered towards the end of semester one, before exam results were known. two different approaches were taken to the collection of data and while the preliminary content analysis was conducted separately similar guiding themes were used. collaboration led to a further refinement of data analysis themes as summarised in figure 1 and the subsequent joint discussion of the results was used to compare and contrast our experiences, again using the themes in figure 1. a reflective interpretive approach was taken to meet the needs of our research that derive from our efforts to further develop our first year is courses to make them more applicable, accessible and even enjoyable for female students. figure 1: approach to data collection and analysis course feedback forms themes information systems as a university course perceptions of the job market is as a career for women gender aptitude gender issues australia focus groups new zealand interviewspreliminary analysis joint analysis course feedback forms australasian journal of information systems volume 15 number 1 december 2007 30 the australian experience approximately half of all students studying first year information systems are women, and yet less than 10% of women graduate in an is discipline. a series of focus groups was used to explore the issues; namely, why women rarely choose an information systems major. there were six focus groups in all with six students who had recently passed the course in each group. three groups consisted of only women, and three consisted of an equal mix of men and women. groups were drawn from the course cohort and were representative of school leavers and mature age students, and local and international students were equally represented. the focus group sessions lasted for an average of one hour each and extensive notes were taken during the discussions. the new zealand experience introductory information systems is a compulsory course for all business and communications students in the nz university. therefore slightly more than half of the course are women. informal, semi-structured interviews were conducted with 14 students (11 female and 3 male students) at the end of semester to explore their perceptions and experiences in taking the course. interviews lasted an average of 40 minutes and notes were taken and cross checked with the students for confirmation of their accuracy. additionally, secondary data was extracted from 211 course feedback forms. the forms were not directly related to this research but are commonly used to collect data for course development. the forms are completed anonymously and for the purposes of this research the general comments section was used to identify any comments directly related to gender issues. such comments were coded and analysed to gain further insights in support of the interview data. cultural factors were not specifically addressed although we recognise that there are different perceptions between cultures on the issues in technology (von hellens & nielsen, 2005). student perceptions of is courses the findings from the analysis are discussed in relation to the literature in order to examine the issues that affect female students in regard to is within the university environment. this supports identification of where changes can be made to improve the learning experience of female students and encourage their greater uptake of is studies. studying information systems a common facet of teaching is in the tertiary sector is the lack of understanding of what the term information systems means. first year students rarely show an understanding of what the course might comprise and cannot differentiate is from other technology courses. study participants admitted that they were confused as to what constituted is, with few students able to differentiate between is and other it disciplines before the course. even by the end of semester the majority saw is as a subset of it and felt that is would not give the students as many skills as other it related majors. this view is exemplified by the following comments: “yeah, everyone knows what it is, but who’s ever heard of is?” and “…my friends always ask me to explain what i’m studying, and can’t understand what it is – perhaps because my own thoughts about it are a bit vague”. many of the women involved in the focus groups were headed for the “soft” majors of marketing and management, and felt that is would not help them in their careers. australasian journal of information systems volume 15 number 1 december 2007 31 “i look at the jobs that appeal to me and they have titles like ‘marketing manager’, hr professional – i don’t see anything with the words information systems.” even the girls choosing accounting and/or finance, felt that is was “someone else’s problem”. these views were articulated repeatedly in the focus groups with comments like: “everyone understands what an accountant does, but what does an is professional do? we learn a lot about where it fits into organisations, and how it can lead to a job like a cio, but i don’t know any cios, i know accountants and financiers.” “my dad works for [medium sized company, name withheld], and he reckons that the is people are there to mop up problems – they only see the problems, don’t have a lot of status”. from a nz perspective, the understanding of information systems as distinct from it was addressed early in the course, when it became clear that few students could discern the difference. responses to ‘why is?’ were closely aligned to those of the australian students with most of the women taking business majors in management and finance. interestingly, it was the men who saw it as necessary to understand the role of is in business, “so you can tell your workers what to do and understand what they are capable of”. excel and access are taught and tested in practical lab sessions in the nz case by request of the business faculty. labs are optional and the higher percentage of women attending bears out the argument that women have a less confident view of their own skills (hargittai & shafer, 2006). “i found the labs really difficult. i had a problem learning excel although access was easier. i think it will be useful but i didn’t enjoy the labs.” the difference in ability was not reflected in the lab outcomes where test results were even across the genders. both the interviews and the feedback forms showed that women were more positive about the use of these software applications, and the male students thought that learning them was a ‘waste of time’. is as a job and is as a career despite discussion of the roles of chief information officers within lectures, almost all the students saw a career in is as sitting in front of a computer all day, either programming or manipulating software. only two australian students (both men) understood that is is a ‘people’ profession, involving talking to people, understanding job requirements, making decisions and managing effectively based on the information provided by the system. all the women felt that a job in is would be “boring” or “uninspiring”, with constant effort required to keep up to date, reflecting the argument that it disciplines have an image problem (von hellens & nielsen, 2005). the women also saw is as a “nerdy” profession, and cited the stereotype of the “geek” living in a solitary, antisocial world (lee, 2005). “i want to work in a social environment. computers and things don’t give you that i think you just sit there all day facing a screen.” not one of them saw the area as dynamic or exciting. many of the women felt that if they were to major in is they would be relegated to jobs in web design or simply users of a system. they had no concept that they could have input into the design of the system itself, or that they could influence the ways in which technology was used in an organization. in nz, a positive experience was found in the use of guest lecturers who were able to relate is to the lives of the students. this was particularly true in health informatics, mobile business and security where feedback from students was overwhelmingly positive. these were topics that bore direct relevance to student lives and could be seen as support for the argument of role models in that australasian journal of information systems volume 15 number 1 december 2007 32 feedback forms had comments such as “i could see myself doing that” “using is in health care really contributes to society” (sumner & niederman, 2004) and “the guest lecturers were really interesting. i could relate to what they said because it was real. the security stuff was great.” from a career perspective, the reflections on the perceived employment tasks were mirrored in a lack of enthusiasm for pursuing a career in is, with virtually no students planning a career in this sector. “…look i’ve done the first is unit and i can’t see is as being something that would get me promoted at work. i know it’s useful as a tool, but i can’t imagine doing it for a living.” in the australian university two of the women and five of the men agreed that is could be a worthwhile career for women, but when asked why, they struggled to articulate any reasons. this reflected the case in nz where only one of the women interviewed had any intention of working in an is career. this student is a pacific islander and wished to contribute to her country’s urgent need for more trained it people; an ambition more in line with the traditional career choice of contributing to society that underpins the dominance of women in the so-called caring professions (griffiths et al., 2006), than a commitment to is. one australian student did ask the researcher why, if is was such a great career for women, was there only one woman academic in the school of is? this was a very pertinent question, and it is one that the researcher, the lone woman, has struggled with herself and echoes trauth’s call for more women is academics (2002). although the nz university has two women out of nine lecturing staff, all four is tutors are women. this gives the impression of a greater gender balance than actually exists as the tutors are responsible for practical labs and are not research active. half of the students thought that a woman lecturer was employed to teach the first year course to make the subject “more accessible”, implying that female academics are more suited to a nurturing role. this gender stereotyping was more evident on the part of the women than the men. failure to attract women into it disciplines in academia must have a significant impact on the way the profession is viewed. the under-representation of women in is schools does not go unnoticed by the students and may have a significant influence on the image of it careers as a male dominated domain (sumner & niederman, 2004; trauth, 2002; von hellens & nielsen, 2005). gender aptitude anecdotal evidence or stereotyping suggests that males have great aptitude for the technical aspects of is despite evidence to the contrary. a review of course results over the last three semesters at the australian university highlights the fact that the high distinctions in introductory information systems are split fairly well into 75% for women and 25% for men, and the students who fail are split evenly between the two genders. however, students seem to accept gender stereotyping and when asked which group would have greater aptitude for is, 34 of the 36 students said that men would be better. when asked why, they thought that men had more logical minds, were better at computing because they played games all the time and experimented with software for their personal use, are able to use the technology more proactively, and enjoy technology more. when it was pointed out to the students that women tend to shine in is, and that the top five students in the previous course were women, they all felt that this was a statistical aberration and that the women must have been “nerdy, mature age types”. two of the mature age women in the focus groups felt that they struggled with is because they wanted to understand exactly how everything worked, although they did not feel like this about other australasian journal of information systems volume 15 number 1 december 2007 33 technologies such as the telephone, television and motor car. both students were in the top 10% of the course. comments from the women students suggested that they felt out of place in the classes: “minorities [women] feel they are in the wrong place. is is a young male dominated, geeky, environment and we feel like interlopers.” “even though we have been told that girls do better than the boys, we feel that this has to be a generalisation. i know lots of girls who have failed, and i think i might fail. even though it’s not supposed to be technical, i struggle to understand some of the concepts… i mean, just how can this stuff work?” these results were mirrored in the outcomes from nz although the data were only available from one semester. two of the three top students and nine out of the top twenty were women. failure rates were relatively evenly split with slightly more males failing than women. mature age women tended to succeed more than any other group and their experiences reflected those of the australian students in that they felt they had to know and understand everything relating to is technology. further gender issues further issues that turn women away from is also reflect the literature on ‘gendering’ (lee, 2005). the identification of it as ‘men’s work’ results from societal perception of gender roles. the male dominance in the it workplace reinforces the identification of computing as something of little interest to women (trauth, 2002). many of the women students said that they were not stimulated to learn about computers and is, and saw technology either as a tool or as some science fiction fantasy with which they would probably never engage. “i just can’t get interested in it. the classes look at the future technologies and i can’t get excited, even though i want to and i know that it will help me get a job.” it is tempting to assign essentialist theory and assert that these attitudes arise from inherent differences between the genders (trauth, 2002). however, women students in nz who were familiar with practical applications of technology in the workplace were more interested in computing and what could be done with it. although their views supported the identification of technology as a tool, they did agree that social benefits of technologies such as the internet and mobile phones were extensive and could be fun. nevertheless, women saw themselves as users of the technology, rather than as drivers. they struggled to see themselves in a position where they would dictate is policy within an organization, in spite of having studied numerous case studies that had women in roles of cio. male participants could not think of many reasons why women would not want to study is, particularly as there were so many men involved. this, of course, led two of the women to state that that would be exactly why they would not want to participate, again supporting the stereotyping of it subjects as male dominated. one nz male student thought that is would be a great job for women as they could work from home and “go part time if they had families”. he also felt that women probably made better managers where men concentrated more on the ‘computer stuff’. this is an illustrative example of a male student also having problems identifying it careers without preconceptions and tending to adopt stereotyping as reality. another recurring theme was how ‘boring’ is was at school and the women particularly, though not exclusively, had few good things to say about the subject as taught in schools. “i did systems at school, it was taught by a man, the class was full of boys who didn’t wash, and who swore all the time, and the teacher never told them they were australasian journal of information systems volume 15 number 1 december 2007 34 offensive. i was horrified when i found that i had to do an is unit as part of my bbus – i tried to get an exemption”. “i must say that i enjoyed the unit more than when i studied it at school. it was horrible there, smart arse boys who showed off”. this echoes the findings of a government conference in nz where the school it curriculum was castigated by academics as failing to inspire pupils (hendery, 2006). this is of concern for is academics who must address students who have preconceptions of the subject as boring before they enter the lecture rooms. when initiatives such as winit (griffiths & moore, 2006) and itbeat (lee, 2005) were discussed with students there was great enthusiasm for the idea that technology could be made relevant to women. although the focus groups and interviews were conducted in an attempt to understand the issues for women and is, they did have the bonus benefit of helping to educate the students about the role of the is professional. “i must say that i have learned loads about is through this process” “i have actually felt more part of the group during this session, than i do in the class – can we do more of these?” in spite of extensive discussion of these issues in lectures, it was not until the students had to think about themselves that they could see that there were some possibilities. when asked if any of them had changed their minds about undertaking an is major, three students said they would consider it when they received their exam results – unfortunately, none of the three were women. “i’m beginning to understand what it’s about, i’ve never really thought about it before. i still don’t think it’s the career for me – sorry.” outcomes for course development although women leaving the is profession is a difficult issues for academics to address, the many women entering tertiary education that are exposed to information systems as part of their university course are our concern. these courses are an opportunity to highlight many of the positives of undertaking is as a career. however, it is often at this point that we, the academics, fail to inspire women to continue with is studies. we found from our data analysis that a key point raised by the female students was the need to make the course relevant to them. by this we are led to understand that female students fail to find the connection between the course content and their own understanding of the workplace. these students had difficulty relating to the course content in the abstract, but responded well to guest lecturers who presented real situations such as how information systems were implemented and where and why projects had failed. this was of particular importance where students had little personal contact with any it professionals and industry speakers could be perceived as role models. these speakers also often emphasised the ‘people’ nature of is as a career. women are often good at working in teams because of an inherent preference for connected knowing or learning, as opposed to separated learning (morris, 2002). however, many find groupwork in university to be a frustrating experience particularly in large first-year groups where students, with diverse interests and skills, expend different degrees of effort. more use of online collaborative learning both peer to peer and student to lecturer is likely to encourage women who are more attuned to cooperative learning styles (rajagopal & bojin, 2003). the use of chat-room discussions and promoting group-learning through discussions and emails appeals to women in a learning environment (rajagopal & bojin, 2003). greater application of such methods to is courses would greatly benefit them and improve study methods. australasian journal of information systems volume 15 number 1 december 2007 35 there is also a need to relate concepts to the personal experiences of students using an active handson approach. we have found that case studies to which the students can relate increases interest and promotes questioning. case studies from the text books are not seen as ‘real’ and time taken to develop more ‘live’ material produces a better response. this is particularly true when case studies have a social element to which the students can directly relate (for example, health, non-profit and female-orientated cases). one of the outcomes in the australian university is a complete rewrite of the undergraduate degree. part of the rewrite involves the introduction of four units based on integrated studies. these four units are taught outside of the normal school or department disciplines and involve staff from throughout the faculty teaching communications, numeracy, ethics, problem solving, negotiation, teamwork and information literacy through integrated activities. it is hoped that the women will enjoy learning about is in the context of information literacy, and will utilise computers for a variety of purposes such as research, interpreting data, and writing and presenting reports. the students will create e-portfolios and gain career advice as part of one unit, and there will be a strong push to expose them equally to all the disciplines offered by the faculty. the new subset of units will also involve industry and employers, and there will be a series of lectures and networking opportunities for the students. several women cios have been identified as being important for this part of the program, and it is hoped that they will be able to inspire more female students to join the profession. this is in line with the proposed action in the business/higher education round table report (bhert, 2005). in the nz university, the approach to date has been more low-key. is sits in the science faculty, which raises barriers in the minds of our business students despite the introduction of a joint business/is major. the discipline has developed along more technical lines over the years than many other is departments due to its co-location with other it related disciplines. to counteract this we have recently identified and promoted a business pathway through the degree programme to enable students to choose the emphasis of their is degree. the first year course has two academics appointed to reform the content over a period of three years to make it more accessible and relevant to the students from is and other disciplines. interactive, visual teaching aids are being developed to present technical concepts to the students and plans are underway to encourage third year and postgraduate projects to develop multimedia learning programmes for first year students. guest speakers, although difficult to find, have been used to great effect and other lecturers from within the discipline group have also contributed talks on their research areas. this approach has been very well received by students who have been able to relate is to the ‘real world’. recommendations for marketing is courses marketing of courses is not traditionally a responsibility of academics. however, in the current climate of tertiary education there needs to be recognition of the economic realities and the need to attract students to our own courses. marketing is of paramount importance in attracting women into is courses. the influence of parents should not be overlooked (morris, 2002) and therefore there is a need to target parents at open days and encourage families to is display. the australian university is creating a set of materials that will provide prospective students with better publicity about is and more enjoyable experiences through interactive displays. one female graduate from the school is now a high ranking is professional within a large organisation, and another is a chief information officer. they have been interviewed on video and the recording is now used as promotional material for the school. it is too australasian journal of information systems volume 15 number 1 december 2007 36 early to determine whether this approach will make a difference, but it is proposed to use the video as part of a focus group research item to see whether the students feel that it makes a difference. the nz university has revised its promotion of is courses to emphasise the ‘business pathway’ through the courses offered. it has also taken more control of planning for open days (previously left to the university administrators) and gained a role in both the science and business faculties parents’ evenings. marketing events now emphasise the ‘people, technology, information’ definition of is. it is intended that such initiatives should overcome the ‘geek’ image problem expressed by the students in the study, and have a more positive influence on prospective students before they enrol. further marketing initiatives need to be encouraged at school level. the women in ict task force report (bhert, 2005) argues that in order to attract women into computer-related professions, ‘educators need to broaden the meaning and improve the values associated with the computer per se as a tool for productive and creative technological work across the life-career stages’ (p.26). this report also identified the lack of women role models in the media; for example, there are no female it professionals in sitcoms. in order to attract younger women into it courses, career development officers and teachers in schools should be targeted, as there is no doubt that these people do make a difference to subject selection and impressions about academic disciplines. finally, on a more general note, we have found that university marketing personnel often have little idea of what any it related courses entail and therefore do not highlight them to schools. more interaction with university marketing departments by it academics is a necessary step to improving enrolments of either gender. reflections and conclusions we reached some conclusions from this study on action that we could take to improve the interest levels of these introductory is courses for female students as discussed in the outcomes and marketing sections above. however, our collaboration also led us to reflect on other influences that affect the way such courses are presented and which are, to a great extent, beyond the control of individual lecturers. from a university perspective the low numbers of women in it is reflected in both the student and academic bodies. both universities in this study suffer from an imbalance between male and female is academics. the whole discipline of information systems needs to encourage the appointment of more female role models. the need for more women academics is impacted by a shortage of women graduates and this “catch 22” situation needs to be addressed urgently if we are to encourage more women into is degrees. from a strategic perspective a reworking of the curriculum at both secondary and tertiary levels would be of benefit to improve the content of such courses and make them more personally relevant to both men and women. from an academic perspective, course content and delivery must be improved and more closely related to the key elements of technology, business and social issues and the effect of it on people and their lives. this, we believe, is a difficult task when the contribution of information systems as a discipline does not retain a high regard within the current university climate, following the downturn in student numbers in recent years. there is a division between is academics as to whether is should be regarded as a technical or a business orientated discipline. the two universities in this study approach delivery of courses from australasian journal of information systems volume 15 number 1 december 2007 37 different perspectives with one course being delivered from a business faculty and one from a science faculty. recent changes in both universities reflect our perception of the downgrading of the value of information systems. the school of information systems in australia has lost its identity by being subsumed into the school of management with further restructuring of the course to ‘make it appeal to employers’. there is little apparent cognisance of the need to attract more women (and men) into is degrees in these changes. in the nz university an artificial ‘division of the body of knowledge’ has resulted in any business-related content being moved to the business faculty while the existing discipline group retains responsibility for delivery of the technical content. this move could be seen to mirror the divisions in the workplace which is textbooks highlight as a significant cause of systems inefficiencies. generating a gender balance by making first year is courses more interesting for women is a necessary step towards improving their representation in it-related careers in the workplace. this requires constant reflection on the nature and content of these courses, but also requires that the discipline achieves greater recognition both within universities and in industry. academics must play a part in this and step outside their teaching role to take a hands-on approach to addressing the drab image of the it professions. they need to work towards greater appreciation of the value of the is discipline and towards ensuring that marketing of their courses informs industry, prospective students, their parents and careers advisors of the true nature, content and value of information technology degrees. limitations and further research this research addresses the presentation and content of introductory is courses. it is also written from the perspective of two women academics and does not address the views of male colleagues. more research into the progress of students through second and third year is courses would contribute to a clearer picture of how not only to encourage but also retain students. such research should also address the questions raised from both male and female perspectives to ensure that a more holistic view is given. at the university level we believe that any dominance of male characteristics in system design, human/computer interaction and programming needs to be more clearly addressed, together with developing an emphasis on usability that is more appealing to women. specific research on how to teach women so that the discipline appeals to them should continue to be conducted. there is a scarcity of literature, although there are now several academics actively conducting research in this area. a review of a similar problem within the sciences highlights the fact that such research has been happening in the pure sciences for a long time and has salient lessons for the information systems discipline. references armstrong, d., riemenschneider, c., k, reid, m., & allen, m. (2004) “voluntary turnover and women in it: a cognitive study of work-family balance”. proceedings of the fourth annual sig is cognitive research workshop, washington, dc. australasian journal of information systems volume 15 number 1 december 2007 38 baroudi, j., & igbaria, m. (1995). “an examination of gender effects on career effects of information systems employees”. journal of management information systems, vol 11 no 3, pp 181-201. bhert. (2005). “the gender gap in the ict industries: failing to fully utilise a national resource” joondalup, wa: business/higher education round table. edith cowan university, report 106. christensen, r., knezek, g., & overall, t. (2005). “transition points for the gender gap in computer enjoyment” journal of research on technology in education, vol 38 no 1, pp 2337. denning, p., j, & mcgettrick, a. (2005). “recentering computer science”. communications of the acm, vol 48 no 11, pp 15-19. department for women. (2002). “information technology framework. government of nsw. available online at: http://www.women.nsw.gov.au/pdf/archived/ 2002%20information%20technology% (accessed 6 june 2006). department of labour. (2005). “information technology professional: occupational skill shortage assessment”. wellington. http://www.dol.govt.nz accessed 10 july 2006. goodwin, b. (2006). “better networking keeps men in top it jobs, research suggests”. computer weekly, p 42. griffiths, m., keogh, c., moore, k., tattersall, a., & richardson, h. (2006). “managing diversity or valuing diversity? gender and the it labour market”. working paper, information systems institute, university of salford, manchester. griffiths, m., & moore, k. (2006). “issues raised by women in it (winit) project in england”. in e. trauth (ed.), encyclopaedia of gender and information technology. ideas group, hershey, pa. hargittai, e., & shafer, s. (2006). “differences in actual and perceived online skills: the role of gender”. social science quarterly, vol 87 no 2, pp 432-448. hendery, s. (2006). “it grappling with image problem”. nz herald. http://www.nzherald.co.nz accessed 11 july 2006. jewell, h., & maltby, j. (2002). “female involvement in information technology degrees: perception, expectation and enrolment”. proceedings of the 12th australasian conference on information systems, coffs harbour, nsw. lang, c. (2003). “how girls make decisions about education and careers in information technology”. university of melbourne, phd confirmation document. http://www.ict.swin.edu.au/personal/clang/ accessed 4 july 2006. lee, l. (2005). “tackling technology's image problem among young girls”. the international journal of sociology and social policy, vol 10/11, pp 119-130. moore, k., griffiths, m., & richardson, h. (2005). “moving in, moving up, moving out? a survey of women in ict”. symposium on gender and ict; working for change. manchester. morris, l. d. (2002). “women in information technology literature review: recruitment, retention and persistence factors”. in a. m. o. t. m.-s. e. r. association (ed.). chattanooga: educational resources information center, us dept. of education. australasian journal of information systems volume 15 number 1 december 2007 39 pinkard, n. (2005). “how the perceived masculinity and/or femininity of software applications influences students' software preferences”. journal of educational computing research, vol 32 no 1, pp 57-78. rajagopal, i., & bojin, n. (2003). “a gendered world: students and instructional technologies”. first monday, vol 8 no 1. http://www.firstmonday.org accessed 18 june 2006. rossi, s. (2006). “aussies seek local recruits, but women still spurn it”. computerworld new zealand. http://www.computerworld.co.nz accessed 3 july 2006. stone, p. (2007). “opting out? why women really quit careers and head home”. university of california press, berkeley, ca. sujdak, e. j. (2002). “impact of the shortage of it professionals on logistics and supply chain management”. logistics spectrum apr-june. sumner, m., & niederman, f. (2004). “the impact of gender differences on job satisfaction, job turnover, and career experiences of information systems professionals”. journal of computer information systems, winter, pp 29-39. trauth, e. (2002). “odd girl out: an individual differences perspective on women in the it profession”. information technology and people, vol 15 no 2, pp 98-118. von hellens, l., & nielsen, s. (2005). “australian women in it”. communications of the acm, vol 44 no 7, pp 46-52. australasian journal of information systems volume 15 number 1 december 2007 40 microsoft word ajis vol 15-1 m.doc australasian journal of information systems volume 15 number 1 december 2007 9 gender and equity in e-learning j. coldwell deakin university, geelong, australia email: jo.coldwell@deakin.edu.au a. goold deakin university, geelong, australia email: annegret.goold@deakin.edu.au a. craig deakin university, geelong, australia email: annemieke.craig@deakin.edu.au j. mustard deakin university, geelong, australia email: jamie.mustard@deakin.edu.au abstract literature to date suggests contrary indicators of acceptance of the use of technology to support learning by females. with the increasing adoption of information technology to support teaching and learning, it is imperative that factors which may impede student learning are identified. the research reported here is of a large-scale survey of the perceptions of university students about elearning and their use of the online learning environment. the aim of the survey was to gather data to inform about online learning practices at the university. the results were explored, amongst other factors, by gender. findings include no significant differences between the female and male students with respect to being able to use the online learning environment confidently and effectively. in general the female students were more willing to participate in online discussions. however, there was no difference between the female and male students regarding their willingness to voice their opinions online. an unexpected result was the greater value placed by female students on using the online environment for communicating and collaborating with students of diverse background. australasian journal of information systems volume 15 number 1 december 2007 10 introduction there is contradictory evidence in the literature as to the adoption and acceptance of information technologies by females. on the one hand, reports suggest that females are being disadvantaged by the introduction of information and communication technologies (ict) in the learning arena. on the other, there is evidence that in some areas, females are more likely to have a positive experience of using technology. with the increased adoption of online learning technologies to support teaching and learning in higher education institutions, it is imperative that any factors which may impede student learning are identified with the aim of removing them. all learners should learn in environments that support their needs yet online learners are not a homogenous group (mcgee 2002). the identification of possible gender inequities in online learning environments (oles) is the focus of this paper. as asserted by mcgee (2002) whatever the reasons higher educational institutions have for delivering courses online, it is the student’s success that is the desired outcome. in the context of this research elearning can be described as learning processes and interactions between students and teachers that are supported by information and communication technologies (ict). the level of elearning may range from static delivery of key documents to fully supported communication and collaboration online. higher education institutions are increasingly adopting online technologies to support teaching and learning and deakin university is no exception. as part of the university's commitment to preparing students for lifelong learning, it has mandated that every student experience at least one wholly online unit as part of their undergraduate degree programme (deakin university 2003). a wholly online unit is defined as one where there are no face-to-face classes at all, and all teaching and administrative support is provided through an ole. a key objective of a wholly online unit is for students to develop skills in online communication and collaboration while studying the unit curriculum. communication and the ability to work in virtual teams (collaboration) are seen as important lifelong learning skills. online learning offers students a different type of learning experience. it can ‘provide an educationally superior alternative to traditional lectures’ and it can ‘also provide a model for students on how to become self directed independent learners, which may assist them to become lifelong learners’ (singh et al. 2005). haywood et al. (2004) suggest the ‘differential experience, confidence, or preparedness to participate, of any group must be considered as a potential source of discrimination and inequity’ (p. 14) in online environments. there are numerous perspectives which may shed light on equity issues in education including culture, age, race, gender, and socio-economic status (meyers et al. 2004). meyers et al. (2004) suggest that it is ‘from examining these diverse perspectives that the developing field of ict in elearning will most benefit’ (p. 4). such research can then provide the evidence base to inform the development of elearning strategies in the educational context (haywood et al. 2004). educators at deakin university have supported elearning for a number of years. the focus now is on improving the learning in these environments in an endeavour to ensure successful learning outcomes for all students. the university population is becoming more diverse, with increasing numbers of international students, and with local students having more varied skills and backgrounds. the factor of culture is having an increasing impact on teaching and learning, both in the traditional face-to-face classroom as well as the online virtual classroom. the research presented in this paper was undertaken as part of a strategic teaching and learning grant at deakin university in 2005. the overall aim of the larger study was to investigate the impact of cultural diversity in the ole. the larger study involved a review of literature and data australasian journal of information systems volume 15 number 1 december 2007 11 collection from academics involved with online teaching at deakin university. the findings were then benchmarked with external experts. the final part of the study was a large scale online survey that was designed to gather the perceptions of the entire university student population about elearning, online environments and cultural diversity. this paper reports on particular outcomes from the student survey (2711 respondents). in the survey, students were asked to self assess their confidence and level of competence when using the ole. they were also asked a number of questions about how they used and participated in discussion forums. the responses were analysed by culture, age and gender to determine whether there were any indications of differences in students’ perceived level of confidence and competence in using the ole. the results from the gender breakdown are reported here. in the discussion section, findings which support or contradict the literature are emphasised. background over the last 20 years the tertiary education sector in australia has changed dramatically in a number of ways. overall more local students are attending tertiary institutions; there are more international students and the number of female students is increasing. during the first 130 years of tertiary education in australia male students outnumbered female students (maslen and slattery 1994). since the late 1980s however, female enrolments have exceeded male enrolments in the undergraduate population of universities, a situation mirrored at deakin university with 57% of the student population being female (deakin university 2006). a large body of research exists on the male-dominated culture of computing (craig 1996). research from the 1980s and 1990s show that women are less enthusiastic about using and working with icts than men. these findings are reiterated in a recent the educause survey (katz 2006) which found that ‘female respondents…prefer less technology in their courses than others’. the reasons for this are complex and include an array of factors which interact to produce the ict experiences and outcomes for women (craig 2005). kirkup (2002) suggests that, with a more diverse body of students attending university, finding ways to be inclusive of all through an electronic medium, will not be any easier than teaching in a face-to-face classroom. according to kirkup (2002, p. 13) there is ‘now a wealth of research on the gender differences of male and female online activity, all of which demonstrate the online environment creating a gendered world operating in similar ways to the material world’. observable changes are, however, apparent. from a study of 1200 students at edinburgh university, haywood et al. (2004) found for example, that a difference in the level of computer ownership between males and females was no longer statistically significant. they suggest that with regard to technology, the inequality of access and computer literacy has become a diminishing problem. yet there is evidence to the contrary. in trent and slade’s (2001) study for instance, they discovered that while 80% of the boys indicated that they mostly used computers at home, only 40% of the girls did the same. huyer (2003, p. 120) suggests there needs to be ‘more sex-disaggregated data collected at every level of implementation’ of icts, to ensure that new applications such as online learning environments are inclusive of all students and that there is no differential impact on female or male students. the introduction then of elearning has prompted a new range of studies in gender-related issues. many have focussed on particular aspects of the technologies used in oles and suggest that the online learning experience is different for female and male students (gunn et al. 2003; astleitner australasian journal of information systems volume 15 number 1 december 2007 12 and steinberg 2005). factors that appear to impact on the online learning experience relate to confidence and having access to computers at home (gunn et al. 2003). one area where gender differences continue to persist is in students’ self confidence with the technology (haywood et al. 2004). however the gunn et al. (2003) study found that female students outperformed male students in an online web design unit and suggested that the issue was not a lack of confidence in the female students but rather an overestimation of confidence and ability by the male students who were therefore not putting in the necessary effort to perform well in the unit. many studies have targeted online discussion forums as an area for gender research. these studies have often focussed on small groups of students in case-study type research. the meyers et al. study (2004) of mature-age rural women and the giannini-gachago and seleka (2005) study of students in a masters course from botswana and the united states are examples here. in both of these studies there were perceived differences in the use of asynchronous communication, such as discussion boards, in online learning environments, by women compared with men. consequently the researchers postulated that there were implications for the design and moderation of discussion forums in oles to ensure that they are fair and equitable arenas for all participants. according to an open university uk survey ‘female students appreciate the use of computer conferencing for contact with other students, course directors, and tutors, suggesting that women students may experience more support in an online environment’ (huyer 2003). bissell et al. (2002) reported that only 58% of the men appreciated the computer conferencing compared to 82% of the women. yet other researchers have identified only small gender differences in online discussions and activities (kirkup 2002; davidson-shivers et al. 2003). the literature therefore has indicated that this is an area where there are differences. the research reported here makes a significant contribution because of the large number of students involved in the study. additionally, for these students, much of their learning took place through online communication. the study enables a direct comparison between both male and female students. methodology deakin university has five physical campuses, covering a capital city and regional areas, and has a strong history and extensive experience as a provider of distance education. deakin has adopted a policy of teaching the same course on and off campus using technology to support delivery and interaction. deakin is ‘at the forefront of the convergence of distance and on campus teaching and learning through internet based technologies’ (evans and stacey 2002). the provision of high quality online learning is a variant of the high quality distance learning that deakin is known for. in 2005 deakin university had an enrolment of over 32,000 students across five faculties (deakin university 2006). in a normal semester a full-time student would be expected to undertake four of some 1500 undergraduate or 700 postgraduate unit offerings. a standard three-year undergraduate programme is 24 units completed over six semesters. deakin studies online (dso) is the acronym deakin uses for its ole. dso incorporates a suite of integrated teaching and learning technologies including webct vista (now known as blackboard), elive, ilecture and turnitin. additional tools such as respondus, studymate and social software enhance the functionality and useability of the online teaching and learning environment. every unit taught by deakin has an online presence in dso however each unit may vary in their level of onlineness. some units that are taught in face-to-face mode may only have a basic online presence, with the unit outline and study materials available for downloading, and a discussion forum australasian journal of information systems volume 15 number 1 december 2007 13 available for contact with teaching staff. at the other end of the scale are wholly online units in which all teaching takes place in the ole and there is a heavy emphasis on communication and collaboration. in 2005 there were 23 nominated wholly online units at deakin. these units represent core units in each of the five faculties and are part of deakin university's commitment to preparing students for lifelong learning, thus ensuring that each student completes at least one wholly online unit during their degree program (deakin university 2003). the aim of the overall research study was to ascertain students’ perceptions about their learning experience in the ole. a survey questionnaire was deemed the most appropriate method of collecting the data. in the context of this research it was important to gather the opinions of a diverse range of students with the aim of obtaining a representative sample of the university population. according to thomas (2004) representativeness means that what is learnt about the sample would be essentially the same if the entire population was included and thus generalizations can be made for the population as a whole. the student survey questionnaire was posted on the university’s intranet in september 2005 and was made available to the entire university student population for a period of two weeks. the survey responses were anonymous and voluntary, in accordance with the university ethical guidelines. the 60 questions on the questionnaire were developed by the researchers. the questions were based on issues highlighted in the literature and identified as likely to inform online learning practices. the questions covered: • demographics (age, gender, country of birth, language, cultural background); • studies at deakin (mode of study and enrolment, enrolled faculty, major area of study, experience with online units); • perceptions of the ole; • perceptions of their ability to use the ole; • perceptions of virtual group work; • perceptions of their role as a student; and • expectations of learning generally, and of learning online. most of the questions required a five-point likert scale response from strongly agree (1) to strongly disagree (5). a number of questions required open-ended responses. the data from the online survey was gathered into a spreadsheet and various types of statistical analyses were performed. for most of the questions listed above the results were broken down by gender and basic summary results were determined for females and males in the sample. hypothesis tests for the difference between two population proportions were then performed to determine if the differences found between females and males in the sample were statistically significant; in other words, determine if the results found held for the entire university population. (all results are reported in the next section). the literature suggested some key areas where similarities or differences between male and female students might be significant in online learning. these areas were the level of confidence and ability to effectively use the ole, and the degree of student participation in the online discussion forums. gunn (2003, p 24) suggests that the challenge is to ‘identify the characteristics that make learning environments friendly to both male and female students’ and then one is able to encourage participation by, and success for, all. australasian journal of information systems volume 15 number 1 december 2007 14 results the total student population at deakin university in 2005 was 32,354 (deakin university 2006). table 1 shows the break-up of the students with respect to gender, mode of study and mode of enrolment. the total number of completed surveys was 2711. although this response rate is low, the rate is in line with a university-wide student evaluation survey done on webct vista in semester 1 2005. in that survey 2546 students responded and this was considered to be “adequate to provide reliable indicators” (deakin university 2005). of the 2711 surveys completed here, 10 were unusable and were therefore not included in the data analysis. students enrolled in 2005 n= 32,354 survey data n =2701 male 43% female 57% male 39% female 61% full-time 59% part-time 41% full-time 71.5% part-time 28.5% oncampus 65% offcampus 35% oncampus 69% offcampus 31% table 1: university population table 2: survey population table 2 shows the break-up of the survey data. gender, mode of study and mode of enrolment in the survey data indicated that the completed surveys were a representative sample of the 2005 student population (table 1). there were only minor differences between the survey data and the reported university demographics in 2005. the survey data had a slightly higher representation of females (4% more); students studying full-time (13% more); and on-campus students (4% more). table 3 shows the break-up of students by (owning) faculty for the university population as well as the survey sample. again there were some minor differences in the response rates. responses for students from the arts and education faculties were slightly lower while the response rates from the health and behavioural science, and the science and technology students were higher. interestingly, the proportion of business and law students responding to the survey was the same as their proportion of the university population. again the minor differences are not significant. overall there is reasonable confidence that the data collected from the survey data is representative of the university population as a whole, and are adequate to provide reliable measures of student perceptions. australasian journal of information systems volume 15 number 1 december 2007 15 faculty university population survey data arts 6424 20% 384 15% health & behavioural sciences 4544 14% 421 17% business & law 11803 37% 926 37% education 4373 14% 276 11% science and technology 4864 15% 474 19% table 3: total enrolment and completed survey by faculty overall confidence/effective use in ole students were asked whether they were confident in their use of dso (the ole). they were also asked whether they believed that they were able to make effective use of this environment. the majority of respondents (84%) reported being confident in using dso by selecting either agree or strongly agree (table 4). five percent of students stated that they were not confident in their use of dso (disagree or strongly disagreed). the remainder (nearly 11%) were unsure. similarly the majority of students believed that they used dso effectively (85% agreed or strongly agreed) with only 5% believing that they were not able to use dso effectively (table 5). to ascertain whether there were any differences between females and males in terms of confidence and effective use, students were divided by gender. of the 1,646 female students who responded, 86% generally agreed when asked whether they were confident is using the ole and 87% claimed they could use the ole effectively. the corresponding percentages for the 1055 males who participated were 82% (for confidence) and 82% (for effective use). female students were sightly more positive about their confidence and effectiveness in their use of the ole than the male students. a hypothesis test for the difference between two population proportions was performed and although the difference in the sample is only slight, it is, nevertheless, statistically significant (z=2.84, p<0.01 for confidence and z=3.43, p<0.01 for effective use). gender gender i am confident in using dso f m total i am able to use dso effectively f m total strongly agree 38.9% 36.2% 37.9% strongly agree 36.6% 31.5% 34.6% agree 47.0% 45.7% 46.5% agree 50.0% 50.2% 50.1% neutral 10.0% 11.9% 10.8% neutral 8.7% 12.3% 10.1% disagree 2.3% 3.4% 2.7% disagree 3.3% 4.7% 3.9% strongly disagree 1.6% 2.7% 2.0% strongly disagree 1.3% 1.1% 1.2% not applicable 0.1% 0.1% 0.1% not applicable 0.1% 0.1% 0.1% total 100% 100.% 100% total 100% 100% 100% table 4: confidence table 5: effective use apart from gender, participation at the start of the semester was found to be another important factor in determining levels of self confidence or effective use. students were asked to indicate whether they took the time to explore dso unit sites at the start of semester and 1,802 (67%) of all students generally agreed. of these students, approximately 90% were both confident and believed australasian journal of information systems volume 15 number 1 december 2007 16 they used dso effectively. of the remaining students, only 73% were confident and 74% believed they used dso effectively (see table 6). these differences between the two groups are considerable and are clearly statistically significant (z=11.87, p<0.01 for confidence and z=11.02, p<0.01 for effective use). explore dso at start of semester explore dso at start of semester i am confident in using dso generally agree generally disagree total i am able to use dso effectively generally agree generally disagree total generally agree 90.2% 72.6% 84.4% generally agree 90.1% 73.9% 84.7% generally disagree 3.6% 7.5% 4.9% generally disagree 3.7% 8.1% 5.2% neutral 6.2% 19.9% 10.8% neutral 6.2% 18.0% 10.1% total 100.0% 100.0% 100.0% total 100.0% 100.0% 100.0% table 6: explore dso at start of semester it is noteworthy that more females (70%) compared to males (61%) explored dso at the start of the semester. this may explain in part the differences found between genders described above, but further study is required to confirm these findings. participation in discussion forums about 45% of students indicated that they liked posting messages and participating in online discussion forums with 25% saying they disliked it and 30% indicating that they neither liked nor disliked posting messages and participating in online discussions (table 7). gender gender i do not like posting messages in dso f m total dso discussion enable students to help each other f m total strongly agree 4.9% 5.5% 5.1% strongly agree 25.1% 18.3% 22.4% agree 19.0% 20.9% 19.7% agree 50.4% 49.5% 50.1% neutral 28.1% 31.7% 29.5% neutral 18.7% 23.0% 20.4% disagree 37.2% 29.8% 34.3% disagree 4.9% 7.3% 5.8% strongly disagree 10.0% 11.1% 10.4% strongly disagree 0.9% 1.9% 1.3% not applicable 0.8% 1.1% 0.9% not applicable 100.0 % 100.0% 100.0% total 100% 100% 100% total 100% 100% 100% table 7: posting messages table 8: help each other 'posting messages' was another important factor for effective use of dso. the majority of students (76%) who did not believe they used dso effectively also disliked (or were neutral about) posting messages. the female students were, in general, more willing to participate in the online discussion forums (table 7) with 47% generally disagreeing to not liking posting messages compared to only 41% of males. the results are statistically significant (z=3.27, p<0.01). further to this, 75% of female students believed that they derived a benefit from students assisting each other in these forums, australasian journal of information systems volume 15 number 1 december 2007 17 compared with only 68% of the male students (table 8). the difference is again statistically significant (z=4.40, p<0.01). there were also more female students (50%) compared with male students (45%) that selected communication via dso as one of the advantages of the technology (table 9). gender gender it is easy to communicate with others f m total ask for help when i'm not sure f m total no 50.3% 55.3% 52.2% no 13.4% 24.6% 17.8% yes 49.7% 44.7% 47.8% yes 86.6% 75.4% 82.2% total 100.0% 100.0% 100.0% total 100.0% 100.0% 100.0% table 9: advantage of dso table 10: my role as a student perceptions of what students believed their role at university to be, were explored. eighty-seven percent of the female students believed it was their role as a student to ask questions when they were unsure, yet only 75% of male students agreed (table 10). this result is statistically significant (z=7.44, p<0.01). in contrast, when asked if they would ask the teacher or other students for help, there was no discernable difference between the responses of male and female students (z=1.24, p=0.21). it is interesting to note that both males and females are less willing to ask other students for help compared with seeking help from their teachers (tables 11 and 12). gender gender i am prepared to ask the teacher for help f m total i am prepared to ask other students for help f m total strongly agree 25.9% 24.0% 25.1% strongly agree 22.3% 20.7% 21.7% agree 48.6% 48.4% 48.5% agree 47.1% 46.7% 46.9% neutral 19.3% 19.3% 19.3% neutral 21.6% 23.0% 22.2% disagree 5.5% 7.4% 6.2% disagree 8.0% 8.9% 8.3% strongly disagree 0.7% 0.9% 0.8% strongly disagree 1.0% 0.7% 0.9% total 100.0% 100.0% 100.0% total 100.0% 100.0% 100.0% table 11: getting help from staff table 12: getting help from each other voicing opinions in discussion forums overall students claimed that they are willing to voice their opinions on dso. fifty-two percent agreed or strongly agreed and only 17% disagreed or strongly disagreed (table 13). there was no significant difference between the responses of male and female students (z=1.10, p=0.27). voicing an opinion does not seem to influence whether or not a student perceives themselves as an effective user of the dso technology. australasian journal of information systems volume 15 number 1 december 2007 18 gender i am willing to voice my opinion in discussions in dso f m total strongly agree 13.0% 10.9% 12.2% agree 40.2% 40.1% 40.1% neutral 28.9% 31.1% 29.7% disagree 14.3% 14.7% 14.5% strongly disagree 2.3% 2.0% 2.2% not applicable 1.3% 1.2% 1.3% total 100.0% 100.0% 100.0% table 13: voicing an opinion not surprisingly most of the students who were not willing to express their opinions also did not like posting messages. however, there was a small number (14%) of those students who did not express their opinions that did enjoy posting messages. tables 14, 15 and 16 show that when split by gender, the results indicate that there is practically no difference between the opinions of males and females. gender my role as a student: express my opinion f m total no 33.2% 34.2% 33.6% yes 66.8% 65.8% 66.4% total 100.0% 100.0% 100.0% table 14: my role is to express my opinion gender disadvantage of dso: my opinions are not heard f m total no 95.3% 92.1% 94.0% yes 4.7% 7.9% 6.0% total 100.0% 100.0% 100.0% table 15: in dso my opinions are not heard australasian journal of information systems volume 15 number 1 december 2007 19 gender disadvantage of dso: some students dominate when online f m total no 74.1% 75.3% 74.5% yes 25.9% 24.7% 25.5% total 100.0% 100.0% 100.0% table 16: in dso some students dominate collaboration with others students were asked whether they thought it was valuable to work with students from diverse backgrounds in the ole. while 81% of the female students agreed or strongly agreed with this statement, only 70.3% of the male students were in agreement (table 17). this is a statistically significant difference with z=6.45, p<0.01. i think it is valuable to work with students from diverse backgrounds gender f m total strongly agree 37.5% 31.2% 35.1% agree 43.5% 39.1% 41.8% neutral 17.4% 25.1% 20.4% disagree 1.5% 4.0% 2.5% strongly disagree 0.1% 0.7% 0.3% total 100.0% 100.0% 100.0% table 17: working with others from diverse backgrounds a curious result, not necessarily of importance to gender research is that whereas 90.8% of the females have the expectation that they should submit work which is their own, only 82.7% of males have a similar expectation. it should be of concern to all academics that almost 18% of males and 9% of females do not have such an expectation! nationality nationality and culture of students, and the influences these factors have on students and their use of dso, were part of the larger study as described in the introduction. the detailed results about nationality are to be reported elsewhere. for completeness, we note here that there are considerable variations between students of different nationalities and their use of dso. in particular, those students from asian countries compared to other countries show the most prominent differences; students from asian countries tended to be less confident about using dso, did not like posting messages and did not consider themselves to be effective users of dso, compared with local australasian journal of information systems volume 15 number 1 december 2007 20 students and students of other nationalities. the most relevant part of these results here is that the impact of gender appears to be independent of nationality. discussion overall confidence/effective use in ole the literature (gunn et al. 2003, haywood et al. 2004) had suggested that the main areas for possible discrepancies between male and female students could be in the areas of confidence and effective use of the ole and students’ level and type of participation in the online discussion forums. females often rate themselves as being less competent and having less experience with the internet (gunn et al. 2003) and consistently underestimate their technology skills regardless of what their skills really are (sanders 2005). the results of our study are contrary to this earlier literature. there is a statistically significant difference between female and male students with more females being confident in using the ole and more believing they effectively use the technology. several explanations for this are possible. the majority of students 1487 (55%) indicated that they had completed a wholly online unit; 917 (34%) were female and 570 (21%) were male. in wholly online units all teaching takes place in the ole and students are exposed to a wide range of online technologies. one would expect that students who had completed a wholly online unit would be confident about using the ole and making effective use of it. the relatively higher number of female students who responded and who considered that they had completed online units could have skewed the results here. other explanations are possible. for example the imperative to use the ole may well have had a positive impact on females’ attitudes towards the ole, impacting on their perceived confidence and effectiveness. participation in discussion forums at the open university in the uk (as cited in huyer 2003) 82% of females appreciate computer conferencing compared with 58% of men; 91% of females used web resources compared with 50% of men. these findings align themselves with gunn where the possibility that ‘male students may be over-confident in their ability and think they do not have to put in as much effort to meet the course requirements’ (gunn et al. 2003) is presented. the results of our research are in agreement with the earlier literature. enjoying posting messages in the ole and being willing to participate is an important factor in determining whether a student is confident and can use the ole effectively. there are (statistically) more females than males willing to participate and also more who list the communication aspect of the ole as an advantage. perceptions of what students believed their role at university to be, were explored. eighty-seven percent of the female students believed it was their role as a student to ask questions when they were unsure, yet only 75% of male students agreed (table 10). this may suggest that male students expect to have all details of the learning materials provided, or for some other reason, are not prepared to ask when they are unsure. in contrast, when asked if they would ask the teacher or other students for help, there was no discernable difference between the responses of male and female students but slightly fewer students overall are willing to ask other students for help compared to asking their teachers (tables 11 and 12). some research (gferer and pauschwein. 2002; bissell et al. 2002, as cited in huyer 2003) suggested that the anonymity afforded by online environments suits females allowing them to work to their australasian journal of information systems volume 15 number 1 december 2007 21 maximum competence. in a study of ict-based lectures at the fh-joanneum in austria ‘female students tend to appreciate the privacy of virtual education, with no pressure or fear of appearing stupid in front of male students’ (huyer 2003, p. 118). the literature also suggests that females are better at communication than males. this skill may outweigh any perceived shortcomings in technical skills in the ole, increasing female’s perception of their competence. the differences described above, in communication via an ole, did not extend to voicing opinions over the ole. there was no significant difference between females and males in terms of expressing opinions. thus it is possible that the greater use in communication by females in the ole has improved the willingness of female students to express their opinions such that there is now no significant difference between genders. however, further research would be required to confirm this. further investigation is also required in two other areas. firstly, female students appeared to perceive their role as students to be different from male students. however this is not straightforward as the results are mixed. secondly, the area of female students valuing the ability to work with people from diverse backgrounds much more highly is of interest and warrants further in-depth study. limitations a limitation of the study is that the sample was non random. as the survey was conducted in the ole, students who were not comfortable in participating online may have chosen not to participate. conversely those who were more active online may have participated because they felt comfortable in doing so. conclusions hiltz and turoff (2005) suggest that as online learning environments are no longer confined to tertiary institutions but are beginning to appear in k-12, adult learning and corporate training, it is imperative that these environments are suitable and inclusive of all students. because of this wide spread and growing use, it is important that all students are comfortable with oles and can use the technology effectively. this study has found, in higher education at least, that such environments were generally used confidently by all groups of students, regardless of a students’ gender or study mode. another finding of our study suggests that female students are not disadvantaged by the use of icts in the learning environment as they have a similar perceived level of confidence and effectiveness in using the ole as males, as well as being willing to participate in the online environment. the findings support the assertions in astleitner and steinberg (2005, p.60) who suggest that ‘the gender gap in technology-related education like web-based learning begins to close’. it is important to note however that this study does not contradict other studies which have identified numerous gender differences in areas such as learning styles, communication patterns or participation barriers. finally, it was clear that students who actively explored the ole early in the semester had greater self confidence and were more effective users. consequently it is recommended that all students should be encouraged to immerse themselves in the ole early in their course as this will contribute to their ultimate success. unlike many of the studies reported in the literature, our study had a very large base, the sample being representative of the university student population. the findings disagree with some of those australasian journal of information systems volume 15 number 1 december 2007 22 reported in the literature. in depth investigation is required to confirm or refute these findings as well as to explore the discrepancies in the perceived roles between the genders and the conundrum of female communication skills versus male technical skills. references astleitner h. and steinberg, r. (2005). are there gender differences in web-based learning? an integrated model and related effect sizes. aace journal, vol. 13(1), pp 47-63. bissell, c., chapman, d., herman, c. and robinson, l. (2002). some gender issues in the teaching of information and communication technologies at the uk open university. in improving the gender balance in engineering education using ict methods and contents. oulu polytechnic, institute of technology and university of oulu, faculty of technology, oulu, finland. campbell, k. (2000). gender and educational technologies: relational frameworks for learning design. journal of educational multimedia and hypermedia. vol 9(1), pp.131-149. craig, a. (1996). encouraging female students in business computing. masters thesis, victoria university of technology. craig, a. (2005). girls not choosing computing: causes and effects. redress, journal of the association of women educators, vol. 14(3), pp 2 7. davidson-shivers, g. v., morris, s. b. and sriwongkol t. (2003). gender differences: are they diminished in online discussions? international journal on e-learning, vol. 2(1), pp 29-36. deakin university (2003). online technologies in courses and units operational policy. accessed last on 24th january 2006 at http://theguide.deakin.edu.au/thedeakinguide.nsf/web+visitors?openframeset& frame= webcontent&src=wi2.1?openpage&choice=0&access=visitor deakin university (2005) dso (webct vista) evaluation report: semester 1, 2005. internal report. deakin university (2006) 2005 pocket statistics. planning unit, accessed last july 2006 at http://www.deakin.edu.au/planning-unit/statistics/pocket/2005.php evans, t., and stacey, e. (2002) flexible learning for quality in higher education. proceedings of edu-com 2002, higher education without borders. sustainable development in higher education, khon kaen, thailand, 25-27 november gferer, m. and pauschwein, j. (2002). is the change from traditional teaching methods to ictbased methods going to attract more female students to study engineering? an analysis of ict-based lectures at the fh-joanneum. in improving the gender balance in engineering education using ict methods and contents. oulu polytechnic, institute of technology and university of oulu, faculty of technology, oulu, finland. giannini-gachago, d. and seleka, g. (2005). experiences with international online discussions: participation patterns of botswana and american students in an adult education and development course at the university of botswana. international journal of education and development using ict, vol. 1(2). australasian journal of information systems volume 15 number 1 december 2007 23 gunn, c., mcsporran, m., macleod, h. and french, s. (2003). dominant or different? gender issues in computer supported learning. journal of asynchronous learning networks, vol. 7(1), pp 14 30. haywood, j., macleod, h., haywood, d., mogey, n. and alexander, w. (2004). the student view of ict in education at the university of edinburgh: skills, attitudes & expectations. in association for learning technologies. exeter. hiltz, s. r. and turoff, m. (2005). education goes digital: the evolution of online learning and the revolution in higher education. communications of the acm, vol. 48(10), pp 59-64. huyer, s. (2003). gender, ict, and education. engendering ict. world bank, pp 100 -124. jones, l. (2005). maintaining the curricular philosophies of a multicultural education course: from face-to-face to elearning environments. aace journal. vol 13(1), pp.91-99. katz, r. n. (2006). the ecar study of undergraduate students and information technology, 2006. educause center for applied research. accessed last on 20 september 2007 at http://www.educause.edu/ir/library/pdf/ekf/ekf0607.pdf. kirkup, g. (2002). ict as a tool for enhancing women's education opportunities; and new educational and professional opportunities for women in new technologies. seoul, republic of korea, united nations division for the advancement of women (daw), expert group meeting on "information and communication technologies and their impact on and use as an instrument for the advancement and empowerment of women", pp 1-15. maslen, g. and slattery, l. (1994). why our universities are failing. melbourne, wilkinson books, information australia. mcfarlane, a. bradburn, a. and mcmahon, a. (2003) e-learning for leadership: emerging indicators of effective practice. national college for school leadership, autumn 2003. accessed last on 12 june 2007 from http://www.ncsl.org.uk/media/f7b/98/randd-litreview-e-learning-full.pdf mcgee, p. (2002) web-based learning design: planning for diversity. united states distance learning association journal vol 16(3) accessed last on 12 june 2007 from http://www.usdla.org/html/journal/mar02_issue/article03.html meyers, w., bennett, s. and lysaght, p. (2004). asynchronous communication: strategies or equitable e-learning. in beyond the comfort zone: proceedings of the 21st ascilite conference. (eds: atkinson, r., mcbeath, c., jonas-dwyer, d. and phillips, r.), perth, pp. 655-662. sanders, j. (2005). gender and technology: what the research tells us, women and ict, baltimore md, june, 2005. singh, g., o'donoghue, j. and worton, h. (2005). a study into the effects of elearning on higher education. journal of university teaching and learning practice vol. 2 (2), pp 13-24. thomas, s. j. (2004). using web and paper questionnaires for data-based decision making: from design to interpretation of the results, corwin press, thousand oaks, california. trent, f. and slade, m. (2001) declining rates of achievement and retention. the perception of adolescent males. evaluations and investigations programme, higher education division. department of education, training and youth affairs. the flinders university of south australia australasian journal of information systems volume 15 number 1 december 2007 24 microsoft word ajis special 2002.doc ajis special issue december 2002 30 information systems as dialectic, tool-mediated activity helen hasan head of discipline, information systems university of wollongong, helen_hasan@uow.edu.au abstract information systems (is) draws its significance from the uniqueness of computer-based information and communications tools and their place in shaping recent human history. advances in the field come from a better understanding of how to develop and use these tools and what impact they have on the way we work, and live. as is is still an evolving field of study, two views, the objective and the subjective, are in constant tension and, though these may be considered complementary, it is rare that they come together as a unified whole. a more balanced, and integrated, foundation for is may be found in the subject-object dialectic arising out of the german philosophical tradition. an extension of this approach from the cultural-historical activity theory is presented in this paper. this theory views all human endeavour as a purposeful, dynamic, dialectic relationship between subject and object, mediated by tools, such as technology and information, and by the social environment or community. an adaptation of this holistic theory, to incorporate the best of other theoretical approaches used by is researchers, could span and integrate the breadth of the field is providing it with unity and identity. introduction there is a legitimate tendency for researchers to categorise, classify and compartmentalise the basic constructs of their discipline. information systems (is) is a new and evolving field of study whose basic constructs are still being established. is is also an applied field so there is a need to embed such constructs in practice, yet look for constant, enduring concepts that will give the field a firm foundation. finding constants is difficult in a field that is intimately enmeshed in a technology that is constantly evolving at such a rapid rate that the only constant seems to be change itself. in pursuit of a firm foundation for the field, this paper begins with an identification of the characteristics of two extreme views of is, the objective and the subjective, and then suggests that a more balanced, and integrated, foundation for is may be found in the subject-object dialectic (spencer & krauze 1999, vygotsky 1978). an extension of this approach from the cultural-historical activity theory (chat) will then be presented. chat views all human endeavour as a purposeful, dynamic relationship between subject and object, mediated by tools, such as technology and information, and by the social environment or community. extremes of objectivity and subjectivity an objective starting point in the quest for constructs in the foundation of is could be the two component concepts, information and systems. information is usually classified as one of four entities: data, information, knowledge and wisdom. these are defined either independently or in a circular, recursive manner (eg. information is processed data; knowledge is information made actionable). the distinctions between data, information and knowledge are frequently debated, with a contentious point being whether each can be stored or enabled by it. this debate surfaces from time to time in forums such as isworld with only superficial resolution. the concept of system implies an integrated, working whole that is active, purposeful and more than just the sum of its parts. there is a relatively simplistic view that an information system is a tool that processes data into information and is based on information and communication technology (ict). however there is general consensus that ict, while a fascinating invention and development of people is not, in itself, the main focus of the field of is. this highlights the confusion that comes from the association between the name of the field, “information systems”, and the artefacts studied; that is, actual information systems themselves. introductory textbooks in is tend to focus on the artefacts, identifying five components (hardware, software, storage, people, procedures) and define these as having a basic function of processing data into information. information systems are also categorised by their level of application, whether they are at the operational (tps), tactical (mis) or strategic (dss, eis, kms) levels of organisations and whether they automate or “informate” work (zuboff 1988). if is is to be considered a discipline, it may be helpful to hold a more subjective view of information as a universal concept that supports work practices and systems as dynamic, purposeful and holistic entities. however, this still leaves much to consider in establishing a comprehensive and stable foundation for “information systems” as a discipline or field of study. approaches to is can therefore be grounded in different worldviews, two extremes of which are a purely objective view and a purely subjective one, with many variations or combinations of these adopted in practice. pictures of the two extremes will now be painted in order to mark out the boundaries of the is territory. an extremely objective approach to is sees data as atomic facts, while information is processed data, abstracted from its context and highly codified, or codifiable, and therefore readily diffused by ict. in this approach, information is usually intentionally sought and used by people in rational decision-making processes. closely ajis special issue december 2002 31 aligned to an object view of information is that of knowledge as either explicit or, if tacit, able to be converted into an explicit form so that it can be codified and stored in a repository or even captured in an expert system. methodologies for objectively based is research tend to be positivist, experimental, quantitative and technical. the ideal information system automates as much information processing as possible in accord with the artificial intelligence paradigm. in an extremely subjective view, information is viewed as data that is meaningful and in context, and hence is not abstract, codifiable or easily diffused. knowledge, quite distinct from information, is viewed as tacit, embedded in people and not able to be codified or stored using ict. wisdom plays a critical role in decisionmaking to determine what is right and ethical. information systems may, or may not, involve the use of technology, although advances in technology have greatly changed the significance of information. ict is only able to store and manipulate data in transaction processing systems or databases that are relatively uninteresting. the systems of most interest in is research are those that provide support for the work of people such as dss, gdss, ess, kms, intranets and portals. such systems allow for browsing and general sense-making rather than intentional information seeking. methodologies for subjective is research tend to be interpretivist and qualitative. the subject-object dialectic although in practice is research generally falls between the two extremes it is rare that the objective and subjective aspects are treated with equal emphasis and even more rarely are they integrated. a balanced theoretical position, bringing together the objective and the subjective, is found in the german philosophical tradition, in particular the work of hegel, who asserts that, in order to advance a thesis, its antithesis should be proposed (spencer & krauze 1999). the thesis and its antithesis should then be worked through to a more advanced synthesis. taking the objective view of information systems as thesis and the subjective as its antithesis, an integrated approach to is, involving a subject-object dialectic would then be a synthesis of the two extreme views. this is not an average or middle view but a more mature understanding of the field. for example, currently in the field of knowledge management there are two opposing views of organisational knowledge. one, the thesis, sees knowledge as an object that can be codified and then stored in a computerized system to be made available on demand and so “the fundamental purpose of all knowledge management activity is to acquire, capture, access and reuse knowledge throughout the organisation” (fowler 2000). the implication is that knowledge can be separated from its source and context. the antithesis says that knowledge can only reside in people and a knowledge management system can only enable individual knowledge seekers to identify and communicate with knowledge sources, i.e. experts. the implication here is that group knowledge is simply the sum of the knowledge of its members and “the goal a knowledge management system is to create a connected environment for knowledge exchange a technical embodiment of corporate memory” (mentzas et al, 2001). the following quote is a step towards the synthesis of the objective and subjective views of knowledge management: “knowledge management caters to the critical issues of organisational adaptation, survival, and competence in face of increasingly discontinuous environmental change. essentially it embodies organisational processes that seek synergistic combination of data and information processing capacity of information technologies and the creative and innovative capacity of human beings.” (malhotra 2000) a candidate for a mature integrated theoretical basis of is may be found in the social psychology of vygotsky (1978), which takes the dialectic relationship between subject and object as a fundamental unit of analysis for all human endeavour. vygotsky, a contemporary of pavlov in the early 20th century, argued against the extrapolation to humans of the stimulus-response behaviour observed by the latter in animals. according to vygotsky, what makes humans distinct from animals is that all activity is purposeful, much more complex than a response to stimuli, involves the use of tools and produces outcomes that are distinct from the perceived object. vygotsky’s work, continued after his death by his students (leontiev, luria and others), evolved into the cultural-historical activity theory, where, in all human activity, the subject (or person) has a relationship with the object (the purpose) of the activity and tools and community mediate this relationship. the theory is premised on the notion that all meaningful work and learning take place in a social setting where, according to chat, the most powerful human tool is language. in the latter half of the 20th century, ict has been the tool that has revolutionised human work and society. it is therefore reasonable that tool mediated activity, as understood by chat, could be the synthesis of the subjective and objective that ideally suits research into information systems. in order to substantiate this claim a short description of chat is now presented. the reader is referred to the original works such as those of vygotsky (1978) and leontiev (1981) for a more in depth exposé of the theory. ajis special issue december 2002 32 the cultural historical activity theory (chat) although chat has its roots in the work of the russian psychologist vygotsky during the first half of the 20th century, his writings were not published in the west until several decades later (vygotsky, 1978). the theory has a well-established research tradition and has been applied in a range of fields and disciplines, including education, linguistics, anthropology and cultural research. it has the same standing in eastern european research communities that cognitive psychology has in the west, but it is much richer in terms of its interpretation of human behaviour when using technology such as computers. the significant difference that chat brings to the study of is is that it places the focus of study on the activities that are carried out by people in support of their interpretations of their role, the opportunities available, and the purpose for which the activity exists. this is both subjective, in the sense that it is a matter for individual interpretation, and objective, in the sense that the motives, purpose and context are a vital part of the reality of human work. in contrast to western cognitive science, thinking, feeling and acting are considered as integrated parts of the one object in chat. the theory recognises an objective reality. the object, or purpose, of all human activity is what defines that activity and that object is real, whether physical or ideal. indeed activities are often poly-motivated as, for example, employees may be good corporate citizens, and therefore be motivated to cooperate with fellow employees, but also compete with them when they have their own careers to consider. a failure by management to take these different motives into account can have disastrous consequences. in chat, an activity is the only complete meaningful unit of analysis of work and includes purpose, motive and context. in is terms, this has implications for what is studied and the approach taken research into information systems as artefacts. what is objective is not the rational analysis of what should be done but what really is done, affected by messy contexts and driven by conflicting motives. schultze and boland (2000) report low success rates of around 30% for information support systems, attributable to technologists’ lack of understanding of the situated work practices of user communities. they believe that systems designers do not have accepted models for the large invisible and complex nature of work that systems are expected to support. rather than see things such as personal motives and particular contextual factors as soft and inconsequential, they should be part of the objective reality of the whole to be analysed in the system. activity, defined by the dialectic relationship between subject-object, both mediates and is mediated by the tools used and the social context of the work activity. this two-way concept of mediation implies that the capability and availability of tools mediates what is able to be done and the tool, in turn, evolves to hold the historical knowledge of how the communities works and is organised. with ict, this mediation process is happening at a rapid rate that is proving stressful for many workers and disruptive for many work practices. new technology is driving changes to organisational structures and activities and this in turn is placing increasing demands on the capability and capacity of the technology. the changes that ict, and in particular the internet, has made to the way information is perceived and used in society today, is illustrative of this concept. it is through the dynamic process of mediation that learning and development occurs, both in the individual and in the society as a whole (hasan & crawford, 2002). chat uses the term “always active subject” to describe the way people are not static subjects of activities but are continually changing and growing as they engage in those activities. another exclusively chat concept is that of the “zone of proximal development” or zpd. this describes the potential for an individual to proceed to the next phase of develop beyond their present level, aided by teachers, co-workers and such, known as “significant others”. figure 1 kaptelinin's (1996) view of the computer as an extension of the ipa ajis special issue december 2002 33 much of vygotsky’s work is concerned with the study of human development through the processes of internalisation and externalisation. his insight into the dynamics of human consciousness was that it is essentially subjective and shaped by the history of each individual’s cultural experience. learning and development occurs through a social process where what happens in the external world is manipulated in the individual’s internal plane of action (ipa), a type of working storage before being internalised. correspondingly, when an individual externalises what they know, it is rehearsed in the ipa. it is this human process of reflection before acting that intervenes in the automatic pavlovian response to stimuli observed in animals. kaptelinin (1996) has done significant work, which conceptualises a computer as an extension to the ipa as shown in figure 1. vygotsky’s work was continued by his students, amongst them leontiev (1981), who developed a popular conceptual framework for a complete theory of human activity. leontiev saw the theory of activity as the foundation of a unified, monolithic psychology that makes possible a consistent, coherent reconstruction of nonreductionist psychological reality. activity, according to leontiev, is neither a reaction nor a totality of reactions but a system that has structure, its own internal transitions and transformations and its own development. activity motive ↓↑ ↓↑ action goal ↓↑ ↓↑ operation conditions figure 2 the hierarchical structure of activity (leontiev 1981) leontiev (1981) proposed that “activity” should be the unit of analysis in the study of sustained human endeavour and placed this at the top of the hierarchy shown in figure 2, associated with purpose and motive over the long term. this is a conceptual level above that at which most business analysis takes place, which is at the level of actions, undertaken towards specific and often short-term goals. under certain conditions, conscious actions can be driven to a lower level of automation, often in computer systems, as they become standardised as operations. the leontiev hierarchy is probably the best-known chat concept although it can be difficult to fully appreciate the distinction between activities and actions on first encounter. an activity is comprised of sets of actions (creative new responses towards specific goals) and operations (routine and well known habitual cognitive or behavioural processes, now commonly the domain of it systems). whereas an activity is defined by purpose and motive and is typically a long-term affair, actions are more planned with specific goals and a more limited time span. actions are not meaningful in themselves unless they are part of an activity. for example it makes no sense to drive to work (an action) unless there is a work activity to go to. there may be legitimate alternative sets of actions that can enable the successful performance of an activity. there are many instances of this kind of situation. for example, it is common practice in is development to assess the feasibility of different solutions to an organisational problem once the problem is defined. it is usual to then choose one solution to implement on a cost benefit analysis. however there may be instances where it is feasible to allow different solutions (i.e. different sets of actions) for an activity under different circumstances (e.g. in different countries where cultures vary or in different divisions of a company that operate differently). it is important, however, to have a common understanding of the object of the activity itself. in practical terms, engeström’s (1987) triangular representation of vygotsky's concept of activity, shown in figure 3, can be used as a means of identifying and representing an activity. this image shows the central subject-object relationship of the activity leading to outcomes and being mediated by tools (instruments) and the community. the community imposes rules on the subject and establishes the division of labour needed to conduct the activity, which is defined by its object. the components of this representation of an activity are listed and described in table 1. ajis special issue december 2002 34 figure 3 the components of an activity (engeström, 1987) table 1 components of activities according to engeström (1999) component: definition and clarification object the purpose and motives that define the activity. subjects: the person or people who carry out the activity outcomes: both intended and unintended results of carrying out the activity tools/instruments: both physical and non-physical instruments that are used in the conduct of the activity community: the community in which the subjects carry out that activity rules the formal and informal rules that the community imposes on the subject division of labour relationships in the community that determine the roles that subject have in carrying out the activity the dynamic nature of chat is apparent in the activity system approach of engestrom (1987), who uses activity as a unit of analysis in his research into developmental work in organizations. in longitudinal case studies in the workplace he follows the progress of a dominant activity, together with any interacting secondary activities, as an activity system (idealised in figure 4). learning occurs from contradictions and tensions within activities and between the activities and their environment. engestrom depicts this in the cycle of expansive learning (figure 5) as the dominant activity evolves into a more advanced form after each cycle 1: primary inner contradiction (double nature) within each constituent component of the central activity. 2: secondary contradictions between the constituents of the central activity. 3: tertiary contradiction between the object/motive of the dominant form of the central activity and the object/motive of a culturally more advanced form of the central activity. 4: quaternary contradictions between the central activity and its neighbour activities. figure 4: four levels of contradictions in a network of human activity systems (engestrom 1999) activity systems are constantly working through contradictions within and between their elements and with other related activities. in this sense, an activity system is generates a virtual disturbance and thus becomes an ajis special issue december 2002 35 innovation-producing machine. there are periods of incremental growth interrupted by discontinuities of growth spurts leading to knowledge creation and organisational learning. figure 5 engestrom’s view of a dynamic activity system the chat concepts of internalisation and externalisation are echoed in the current knowledge management literature much of which distinguishes tacit from explicit knowledge based on polanyi’s (1966) original concepts. the model describes a dynamic process in which explicit and tacit knowledge in organisations are exchanged and transformed through four modes. socialisation enables tacit knowledge to be transferred from one individual to another. combination allows the existing explicit knowledge to be combined into new explicit forms. externalisation converts tacit knowledge into explicit knowledge in the forms of concepts and models. internalisation allows individuals to absorb explicit knowledge and broaden their tacit knowledge so that new knowledge could be developed. this has led to the knowledge creation spiral of nonaka and takeuchi (1995), shown in figure 6, which views organisational knowledge creation as a process involving a continual interplay between explicit and tacit dimensions of knowledge. figure 6 the knowledge creation spiral research using chat as a theoretical framework. researchers in various disciplines, related to the multi-disciplinary field of is, are using frameworks based on the cultural-historical activity theory for the study of technology, cultural, organisational and managerial issues. several research teams have successfully applied a chat framework to areas such as organisational theory (blackler 1993), organisational learning (engestrom 1999), organisational memory (kuutti & virkkunen 1995), human-computer interaction (bodker 1991) and organisational sense-making in knowledge management (hasan 2000). bodker’s seminal work (bodker, 1991) concerned the interaction of two activities in it: the design activity whose outcome is the tool for the use activity. according to kuutti (1996), chat is a philosophy and cross-disciplinary framework for studying different forms of human practices and offers a set of concepts, structures and terms that are eminently suited to research undertaken within the communities of practice. blackler (1993) eloquently describes his reasons for adopting a chat approach, stating that it offers a way of synthesising and developing various notions of knowledge, ajis special issue december 2002 36 organisations and management and it deals with the messy problems encountered at the strategic level in organisations by attributing significance to incoherency and dilemma. choo (1998) also recommends that organisational knowing relate to an understanding of its socially distributed activity systems. these are collective, situated and pragmatic. they involve the active participation of employees and must incorporate the significance of history as well as an understanding of anticipated changes and new goals. there is a prevalence of incoherence and dilemma in activities, which may have more that one motive. the contradictions within and between activity systems can be leveraged for organisational innovation and growth. choo (1998) sees this approach as public common sense, where language is the most important tool, opposing the tendency for organisations to view talking as socialising and detracting from work. chat provides a unifying holistic approach to the work of organisations, incorporating culture, history, situation and context, and allowing for contradictions and unintended consequences of activities. the chat approach of engestrom (1987) is particularly well known. his work uses cycles of expansive learning shown in figure 5, and demonstrates how positive interventions into activity systems can promote growth and innovation. as already noted this is not unlike the knowledge spiral of nonaka and takeuchi (1995) in figure 6 where the contradictions between tacit and explicit knowledge promotes new learning. whereas most approaches categorise or reconcile the anomalies and dichotomies that arise in sophisticated information systems, chat welcomes these as contradictions that stimulate the activities into a more advanced state. to illustrate how chat can be applied in the field of is, two research projects conducted by the author and published elsewhere in more detail, are now described. the first of these uses the construct of the cycle of expansive learning as a holistic and dynamic tool for the analysis of a complex long-term case study. the second is a project where the chat concept of “activity” is used as the building block of a knowledge architecture in a knowledge management system. project 1 the first project was a case study, using an action research methodology, involving performance evaluation on strategic objectives in tertiary education (hasan 2001b). the site of the case study was an australian university. among its strategic objectives, the university management had agreed on nine attributes that all graduates should acquire through any course of study at that institution. these attributes included independent learning, discipline knowledge, communication skills, teamwork, analysis skills, diversity, information literacy, continuous improvement and individual rights. in order to demonstrate that this strategic objective was being achieved, senior management believed that it was necessary to audit what was taught and assessed in each subject7, counting each separate instance of servicing one of the nine attributes. these counts could then be summed over whole courses. low counts would reveal which attributes were not addressed in a particular course taken by students and this knowledge would result in appropriate changes to the relevant course curricula. the project began in february 2000 when it was decided to begin the auditing process by ascertaining the extent to which the nine attributes were taught and assessed in subjects in the undergraduate commerce degree. the bachelor of commerce had the largest enrolment of any course offered by the university and, while there is one generic award, there are 13 single majors and 57 combined majors on offer. there was a very large set of possible combinations of subjects that could be taken by any one student in the course of their study. the project therefore had two parts: one to ascertain and tally the attributes that were addressed in each subject and the other to calculate the coverage of each attribute over all possible sets of subject combinations. the problem was complex as there was no obvious means of identifying how educational activities contributed to the graduate attributes, and complicated by the fact that subject content changed when coordinated by different lecturers. the author began as one of the subject coordinators participating in the project and was subsequently asked to design and develop an appropriate database information system for recording the collected data. the direct involvement in the restructuring of the data collection and responsibility for the design of the supporting system provided the author with rich insights into the events as they unfolded. over a period of two years the project went through a number of major changes of personnel and techniques for data gathering and analysis, each major change triggering a new cycle of expansive learning. for example, the project was initially coordinated by a senior, experienced academic, who left the university soon after the project began, and a junior, inexperienced person then took over. this was a disruptive change but brought with it new enthusiasm and a questioning of the purpose of the study, which proved beneficial to the project. another cycle was triggered when the attribute identification process began. this involved a series of voluntary workshops that were later replaced by individual interviews to make the process more focussed. this corresponded to the need for a more structured 7 the university uses the word “subject” to refer to each unit of study undertaken by the student (e.g. microeconomics i). ajis special issue december 2002 37 form of the data for each subject compared with the initially free form nature of data reported at the workshops so that it could be recorded and analysed in a database when the project concluded at the end of 2001 there was no clear outcome in the form of a set of proposed changes to subjects so that courses would produce graduates with the desired attributes. however, it was felt by participants that the project had been a positive learning exercise in that, on the one hand, teaching staff were now aware of the attributes and, on the other, management had a much improved understanding of what these entailed. the chat approach provided a technique suitable for examining these issues, enabling the identification of the cycles of learning and the events that triggered them. chat also allows, at least from a research point of view, that a project may have useful learning outcomes when not a success from a business perspective. project 2 the second project involved a much more explicit use of chat (hasan 2001a). researchers and practitioners in information systems have, over several decades, developed and refined techniques for modelling the real world and these techniques are routinely used to design computer-based business systems. the resulting systems have architectures determined by “units of analysis”, such as a record in a relational database, an object in an oo program or a rule in an expert system. given the close relationship of organisational knowledge to work practices, it was proposed that a promising unit of analysis for the architecture of a knowledge management system (kms) is that of “activity”, as understood in chat. a knowledge architecture with activity as the unit of analysis has been developed and constructed from the structure of activity. this activity-based knowledge architecture combines the activity system of engeström (see figure 3 with tables 2 and 3) and the activity hierarchy of leontiev (see figure 2) integrating their structures into a workable architecture of group knowledge that is quite pragmatic and so is implementable in a practical computer-based system. a prototype of a kms using this architecture is currently being evaluated. table 2. elements of the activity-model for kms activities: who is doing what, for what purpose components of each activity as listed in table 2 relationships between those activities. actions and operations by which activities are carried out an historical record of the above elements table 3. components of activities according to engeström component: definition and clarification object the purpose and motives that define the activity. subjects: the person or people who carry out the activity outcomes: both intended and unintended results of carrying out the activity tools/instruments: both physical and non-physical instruments that are used in the conduct of the activity community: the community in which the subjects carry out that activity rules the formal and informal rules that the community imposes on the subject division of labour relationships in the community that determine the roles that subject have in carrying out the activity the meaningfulness of this representation of work has been tested with three typical workgroups where members were asked to identify their activities, their components and relationships. a paper describing this work has been recently accepted for publication (hasan 2003) and the reader is referred to this article for a further explanation of the research that is beyond the scope of this paper. discussion and conclusion it is generally agreed that is is not yet a mature discipline and may never have a distinct, theoretical basis as in other more traditional disciplines. is may continue for some time as a multidisciplinary research field that borrows and adapts theories and methods from more established areas. at the current time researchers in is come from different backgrounds or have placed emphasis on a variety of issues from technical to ajis special issue december 2002 38 organisational, from quantitative to qualitative, from economic to social. it may prove important to the field to maintain its breadth with specialists is different skills and issues. eventually a theory or methodological approach that can span and integrate the breadth of the field could provide unity and identity, or at least a line of communication between a multidisciplinary band of specialists. chat is one such theory that, among others, can be used as a basis for is research. its basic philosophy, that all human activity is a relationship between subject and object, mediated by tools and community, makes it relevant to the study of ict as a tool used to support human work in organisation. it has the advantage of being holistic, flexible and dynamic. currently, the question is not which approach is best for is in an absolute sense, but what is useful for particular research approaches to particular problems. it is worth the investment of time and effort by individual is researchers to understand one such theory in depth, identifying its relevance to is and informing the is community of insights it provides. in chat terms these theories would be considered tools, which mediate the research activity, and synthesise the range of epistemologies from objective to subjective. as noted above, the mediation works in both directions so not only could the theories contribute to the field of is, but the field of is could be a vehicle for these theories to adapt and grow, incorporating concepts from other theories, into one that particularly suits is. references ashford, j. (1997) “an information-space model for the development and application of computer–based tools in information creation and dissemination”, journal of documentation, 53(4), pp 351-373. blackler, f. (1993) “knowledge and the theory of organisations: organisations as activity systems and the reframing of management”, journal of management studies, 30(6), pp 863-884. bødker, s. (1991) through the interface: a human activity approach to user interface design, hillsdale, nj, erlbaum. choo, c.w. (1998) the knowing organisation, oxford university press, new york. davydov, v. v. (1991) “the content and unsolved problems of activity theory”, in multidisciplinary newsletter for activity theory, 7(8), pp 1-35 engeström, y. (1987) learning by expanding: an activity-theoretical approach to developmental research, helsinki: orienta-konsultit. engeström, y (1999) “cultural-historical activity theory” http://www.helsinki.fi/~jengestr/activity (accessed 2002) fowler, a. (2000) “the role of ai-based technology in support of the knowledge management value activity cycle”, journal of strategic information systems, 9, pp 107-128. hasan, h. (2000) “the mediating role of technology in making sense of information in a knowledge intensive industry, knowledge and process management 6(2), pp 72-82. hasan, h. (2001a) “a knowledge architecture using activity as the key unit of analysis”, proceedings of conference on active knowledge management for intelligent decision support (akmids) 2001, melbourne. hasan, h. (2001b) “knowledge creation in organisations: the role of computer-based kms”, accepted for presentation at the 2nd european conference on knowledge management. hasan, h. (2003) “an activity-based model of collective knowledge”, accepted for publication in the proceedings of the 36th hawaiian international conference on systems sciences (hicss36). hasan, h. & crawford, k. (2002) codifying or enabling: the challenge of knowledge management systems, journal of operation research. kaptelinin, v. (1996) “activity theory: implications for human-computer interaction”, in b.nardi (ed) context and consciousness, mit press, pp 103-116. kuutti, k. (1996) “activity theory as a potential framework for human-computer interaction”, in b.nardi (ed) context and consciousness, mit press, pp 17-44. kuutti, k. & virkkunen, j. (1995) “organisational memory and learning network organisations: the case of finnish labour protection inspectors”, proceedings of the 28th hawaiian international conference on systems sciences (hicss28), pp 313-322. leontiev, a. n. (1981) problems of the development of mind, moscow: progress. malhotra, y. (2000) “knowledge management for e-business performance: advancing information strategy to ‘internet time.’”, information strategy : the executive's journal, 16(4), (summer 2000): pp 5-16, available at brint.com mentzas, g., apostolou, d., young, r. & abecker a. (2001) “knowledge networking: a holistic solution for leveraging corporate knowledge”, journal of knowledge management 5(1), pp 94-106. nonaka, i. & takeuchi, h. (1995) the knowledge creating company: how japanese companies create the dynamics of innovation, oxford university press, new york. polanyi, m. (1966) the tacit dimension, new york, doubleday schultze, u. & boland, r (2000) “knowledge management technology and the reproduction of knowledge work practices”, journal of strategic information systems, 9, pp 193-212. ajis special issue december 2002 39 spencer, l. & krauze, a. (1999) “excerpts from hegel for beginners” http://werple.net.au/~andy/ wenger, e. (1998) communities of practice, cambridge university press, cambridge. virkkunen, j. & kuutti, k. (2000) “understanding organisational learning by focusing on ‘activity systems’”, accounting, management and information technology, 10, pp 291-319. vygotsky, l.s. (1978), mind and society, harvard university press. walsham, g. (1995) “interpretive case studies in is research: nature and method”, european journal of information systems, 4(2), pp 74-81. zuboff, s. (1988) in the age of the smart machine, heinemann professional, oxford the australasian journal of information systems (ajis) began in 1993 and was based on the simple realisation that there was a need for an academic ‘outlet’ for is research in australia australasian journal of information systems volume 18 number1 2013 3 ajis editorial the australasian journal of information systems is a refereed journal that publishes articles contributing to information systems theory and practice. manuscripts from all areas related to information systems are sought to reflect the needs and interests of the information systems community. while research is global, ajis contributions have an emphasis on the australasian context of the work. ajis is an academic journal that has adopted a double blind refereeing process. my sincere thanks to aais and acphis for the help and support that they have provided for this issue, and the acs for providing hosting services for the journal. not only is their help financial, but also in the support for the journal that they are fostering in the academic community. i would also like to thank the university of the sunshine coast for the financial support they have given to the journal in employing dr sherre roy as an assistant to the editor-in chief; her dedication in moving the submissions through the new checklist process has dramatically improved the pre-review process. i also thank dr john lamp for his technical assistance, and dr acklesh prasad for his dedicated work on the copyedit and journal issue administration. the acceptance rate for ajis18(1) was 27%. an online version of ajis18(1) can be accessed at http://dl.acs.org.au/index.php/ajis/issue/view/36 this issue of ajis contains eight articles covering a range of topics of interest to information systems researchers, professionals and practitioners. the first paper, data, information, knowledge, wisdom (dikw): a semiotic theoretical and empirical exploration of the hierarchy and its quality dimension, by sasa baskarada and andy koronios (university of south australia) presents a theoretical and empirical exploration of the data, information, knowledge, wisdom (dikw) hierarchy and its quality dimension. they review relevant literature from a range of perspectives, and develop and contextualise a theoretical dikw framework through semiotics. the second paper, factors affecting consumer's continued use of internet banking: empirical evidence from australia, by sujana adapa and ray cooksey (university of new england) investigates the factors that influence consumers continued use of internet banking in australia. their study sets out to develop a conceptual framework based on theoretical models related to the acceptance of technology and diffusion of innovations theory. the third paper, paradigmatic approaches used in enterprise resource planning systems research: a systematic literature review, by kevin burgess (cranfield university, uk), don kerr (university of the sunshine coast) and luke houghton (griffith university) determines the range of research paradigms employed in a smaller subset of information systems (is) literature, namely enterprise resource planning (erp) systems. this research also identifies areas where existing paradigm evaluation methods could be enhanced and refined in respect to non-positivist classifications. the fourth paper, barriers and motivations affecting information systems usage by hajj–umrah religious tourism operators in saudi arabia, by hani brdesee, brian corbitt and siddhi pittayachawan (rmit university) outlines the motivations and challenges that affect the utilisation of various is services in the saudi religious tourism industry of which hajj & umrah religious tourism accounts for seven million visitors each year in saudi arabia. the fifth paper, using historical methods in information systems: a primer for researchers, by janet toland and pak yoong (victorian university of wellington, nz) describes the use of historical methods in information systems research and provides a practical example of how this technique was http://dl.acs.org.au/index.php/ajis/issue/view/36 australasian journal of information systems volume 18 number1 2013 4 used in a recent research project which explored the relationship between icts and regional development in new zealand between 1985 and 2005. the sixth paper, beyond normal competencies: understanding organisation designs to devlop and sustain it-related capabilities, by acklesh prasad (queensland university of technology), peter green and jon heales (university of queensland) suggest an environment in which organisations could build new and sustain their existing it-related capabilities., together with suggestions on the development of valid and reliable measures for this environment. the seventh paper, charting diversity and change in is publications: a tri-continental journal analysis, by susan keller and darryl coulthard (deakin university) explores the extent to which the increased diversity in application and use of information systems in the past decade has been reflected in increased diversity of the publications of three key journals of three continents – north america, europe, and australasia. the paper also reflects on diversity within these journals and discusses the findings in relation to both exogenous and endogenous pressures on the discipline more broadly. the final paper, decision making model for business process outsourcing of enterprise content management, by zhuojun yi (sun yat-sen university, china) and dongming xu (university of queensland), presents a decision making model for enterprise content management business process outsourcing to fill the literature gap and guide industry practitioners with ecm sourcing decision making strategies. dr kathy lynch ajis editor-in-chief (formerly of the university of the sunshine coast) september 2013 paper title (paper title style) australasian journal of information systems lang, fisher, craig & forgasz 2020, vol 24, research article computing, girls and education 1 computing, girls and education: what we need to know to change how girls think about information technology. catherine lang monash university catherine.lang@monash.edu julie fisher rmit university annemieke craig deakin university helen forgasz monash university abstract despite significant efforts and many intervention programs over the years to encourage girls to study computing, we continue to see a declining interest. girls’ lack of engagement with technology at school is resulting in fewer women entering the information technology (it) workforce. our research investigated whether a long-term intervention program with a specifically designed school-based curriculum could change girls’ minds about computing generally and increase their confidence and interest in an it career. qualitative and quantitative data were collected from girls and teachers before, during, and after this program was implemented. a conceptual model of the school-based influences on girls’ attitude was developed from the literature and used to explore the data. findings from this four-year project added rich insights and resulted in a comprehensive model of ‘factors that influence girls’ attitude to it.’ this research demonstrates that a carefully designed it curriculum, delivered in single-sex classes, reinforced by opportunities to interact with role models, and timetabled in regular class time, can and does change girls’ attitudes to it. we also found that the students reported improved confidence and increased interest in it. we posit that our refined model of ‘factors that influence girls’ attitude to it’ is a valuable reference tool. teachers, academics and professionals who are implementing programs to promote it to girls can use it. keywords: intervention programs, gender and it, education, curriculum design, computing and girls 1 introduction diversity in any workforce is regarded as important (evans, 2012). “a wealth of research in the past decade shows that diversity improves problem-solving, productivity, innovation, and ultimately, the bottom line” (ashcraft, eger, and friend, 2012, p. 2). for example, a report from the national centre for women in information technology (ncwit, 2014) found that work teams with equal male and female membership were more experimental and more efficient than single-sex teams (all male or all female). further, when women have engaged in information technology (it), they have been able to create high-tech start-ups with lower funding and fewer failures than the average (ncwit, 2014). however, we continue to see low numbers of women working in it in many western developed nations while simultaneously australasian journal of information systems lang, fisher, craig & forgasz 2020, vol 24, research article computing, girls and education 2 we see an increasing demand for it skills (klawe, whitney, and simard, 2009; dennis, 2013; evans, 2012; shein 2018). the australian computer society reported in 2013 that 28% of the total australian ict workforce are women (dennis, 2013). in 2017 this percentage had only changed to 28.1% (deloitte access economics, 2017). the situation is similar in other western nations, for example, in 2014 in the united states (us), women held 26% of computing positions (ncwit, 2015a; shein, 2018). in europe women make up between 10% and 25% of the ict workforce depending on the country (e-skills, 2014). to understand why we have so few women working in it we need to look at what is happening in education. the “incredible shrinking pipeline” was a phrase coined by camp (1997) in response to the declining numbers of women studying computer science in the us. twenty plus years on, the figures relating to girls studying computing in schools, which may be the start of the pipeline, remain low particularly in western developed nations (e-skills, 2014; anderson, lankshear, timms and courtney, 2008; timms, moyle, weldon, and mitchell, 2018). in australia, lewis, lang and mckay (2009) drew attention to the steep decline of interest in careers in it in australia this century. in 2014, ncwit reported that in the usa less than 20% of the school students taking advanced computing were girls (ncwit, 2014). in the united kingdom in 2012 just 6.5% of those studying computing at a level were girls (eskills, 2014). the number of women studying it at university in europe on average is less than 20% (fatourou, papageorgiou, petousi, 2019). in victoria, australia, the state where our intervention program was developed and delivered, only 16% of senior secondary students studying it applications were girls, and for software development this dropped to just 6% (butt, 2014). mclachlan, craig, and coldwell‐neilson (2016) suggest that while students’ overall technology interest is positive, “the disheartening result is that students are still not interested in undertaking further computing study.” these statistics are presented to draw attention to the current paradox. if girls are not engaging with it curricula at school, they are not building confidence or expertise in it; they then do not elect to study computing subjects in senior school and are even less inclined to select tertiary it courses which would lead to it careers. the result is fewer women in the it workforce (ashcraft, eger, and friend, 2012). adya and kaiser (2005) recognised the problem and proposed a research agenda that included longer-term studies exploring the role of school in influencing girls’ future career choices. consistent with this, our research sought to address girls’ declining interest in it in schools. our digital divas club was a school-based program for australian middle-school girls (14 to 16 years old). the program was implemented in ten australian secondary schools and data was collected from these schools for four years (2010– 2014). the questions guiding our research agenda were: • can a curriculum, designed specifically for girls, change their attitude to it? • can such a program lead to an increase in girls’ confidence in technology? • will the changes be sustained over the longer term? we use the term ‘it’ to refer to the wider industry area of digital technology. the term ‘computing’ is used primarily in the context of computer-based activities. the broader field of information and communication technology (ict) is referred to when appropriate. we begin by providing a discussion from the literature of research about previous it intervention programs designed to change girls’ perceptions about the discipline. we then provide an outline of eccles’ (2005) ‘expectancy value’ model with an explanation of how this australasian journal of information systems lang, fisher, craig & forgasz 2020, vol 24, research article computing, girls and education 3 underpinned the design of our program. this combined body of literature informed our theoretical model which captures these factors diagrammatically. next, we describe our research approach, including an overview of our digital divas club, and the results of our study. in the discussion, we revisit the theoretical model, highlighting how details, which emerged from our research, contributed to the key factors. some of the findings from this research have been presented in an earlier conference paper (fisher, lang, craig and forgasz, 2015). in this article, we comprehensively analysed a wider range of data from the teachers and the schools, which was not provided in the earlier version. this has enabled a richer discussion of the factors that influenced the program. 2 girls, schools and the teaching of computing researchers have shown that girls generally use technology outside of school in similar numbers to boys (mims-word, 2012; underwood, 2016). in a report from the international computer and information literacy study (icils) (de bortoli, buckley, underwood, o'grady, gebhardt, 2014) it was noted that australian girls’ computer information literacy was significantly higher than boys. despite this, girls generally had a lower interest in studying it. although there are many factors which are known to influence girls’ disinclination to pursue it studies further (craig, fisher, and lang, 2007; lang, 2010), our research focused on schoolbased factors. several themes emerged from the literature related to schools and the teaching of computing which we discuss next. 2.1 the school environment and principal school is one place where all students can, or should, be encouraged to think and work positively with technology (de bortoli et al., 2014; barrett, 2018). how schools react to, support, and equip classrooms and teachers obviously influences it learning outcomes. school principals are the strategists and decision-makers playing a key role in technology related decisions including it support and implementation (sheppard and brown, 2014). the principal is also responsible for establishing policies around it use, ensuring appropriate security and determining computer access (de bortoli et al., 2014). a school principal’s views about and attitude towards technology will influence teachers’ classroom-based computer use (machado and chung, 2015; raman, don, and kasim, 2014). raman et al. (2014) found that teachers were more likely to adopt technology and use it more in their classes when there is support from the school principal. when there is less technology available in a classroom girls were less positive about technology than boys (abidin, mathrani and hunter, 2018). it was reported in a major study (icils) that in australia, 90% of teachers have used computers in their teaching, with the majority reporting a high level of ict self-efficacy (de bortoli et al., 2014). support for teachers in the form of professional development (pd) and other activities is important in promoting the value of technology, as well as ensuring teachers are sufficiently equipped to use that technology effectively (hall, atkins, and fraser, 2014). more than twothirds of schools in the icil study reported that their teachers had participated in courses on ict use in teaching (de bortoli et al., 2014). when asked, 57% of teachers reported haven taken some computer related pd activity (de bortoli et al., 2014), but the extent or frequency of the pd was not reported. some of the pd was defined as discussing the use of ict during meetings. australasian journal of information systems lang, fisher, craig & forgasz 2020, vol 24, research article computing, girls and education 4 2.2 the classroom environment a positive learning environment with effective teaching practices is critical to all teaching (ashcraft, eger and friend, 2012). substantial research over the years has found that teachers and their behaviour in the classroom have a significant impact on learning (flanders, 1970; hattie, 2003; prestridge, 2010). early seminal research by rosenthal and jacobson (1992) highlighted the influence of teacher expectations of students. a ‘pygmalion effect’ occurs: “…when teachers expect students to do well and show intellectual growth, they do; when teachers do not have such expectations, performance and growth are not so encouraged and may in fact be discouraged in a variety of ways” (rheem, 1999, pp. 1–2). classroom teachers can, often unwittingly, suggest that boys are better than girls with computers, and attribute computer expertise more to boys than girls (adya and kaiser, 2004). the same authors also reported that teachers can have a negative effect, discouraging girls when it comes to it. this is consistent with the work of margolis and fisher (2002) who argued that the expectation of teachers’ is that boys will excel in computing whereas the girls will struggle. the media often portrays a stereotypical image that it is ‘masculine and geeky’ and not for women (ridley and young 2012). this is often reinforced in the way schools teach computing (ashcraft, eger, and friend, 2012). when computing is taught in the abstract—for example, when students are asked to program a computer to say ‘hello world’—students do not see how technology can help solve societal problems (ashcraft, eger, and friend, 2012). in the digital divas curricula, programming languages were not taught, but were integrated into solving a problem or challenge. for example, students were asked to create a healthy menu using traffic light colours to indicate highly calorific foods. the australian council for educational research (acer) report that current traditional classroom settings and curricula tend not to accommodate different forms of learning (masters, 2016). the physical environment of a computing classroom is also often not encouraging and does not engender a feeling of belonging, conveying the wrong message to girls. often there are rows of single computers set up in a way to preclude group work, and wall signage that warns of misbehaviour or forbids food for example. researchers report that computing classes are often not comfortable for girls because they can feel a level of intimidation from the boys in the classroom (anderson et al., 2008). when there are low numbers of females in the class in the first place, the male dominated environment can have the feeling of entering a male ‘clubhouse’ (margolis and fisher, 2002), and therefore girls feel unwelcome or ‘other.’ 2.3 it curriculum design and delivery australian researchers found that by early secondary school the barriers to girls contemplating it careers have already been established (anderson et al., 2008). newmarch, taylor-steele, and cumpston (2000, p. 9) found that many girls considered computing subjects to be “too theoretical, rigidly structured and boring.” this view does not appear to have changed over time. craig and horton (2009) reported that classroom lessons need to include activities that are creative, challenge the stereotypical image of it being boring and not just about programming. similarly, ashcraft, eger, and friend (2012) stated that the curriculum should also demonstrate how technology can improve people’s lives. they found that girls are more positive towards it when the curriculum incorporates group work or cooperative assignments australasian journal of information systems lang, fisher, craig & forgasz 2020, vol 24, research article computing, girls and education 5 rather than individual projects. the preference for girls is for topics that tap into their interests and prior knowledge (ncwit, 2015b). 2.4 role models of women working or studying in it visible and appropriate female it role models are recognised as important to encourage young women to consider an it career (ahuja, 2002; haddad, 2019; souza, 2018 ). more recently, research has shown that “contact with, or exposure to, same-sex experts will subtly but systematically benefit female students’ implicit self-concept [in science technology engineering and mathematics (stem) environments]…. after multiple exposures or contact experiences this benefit may linger” (stout, dasgupta, hunsinger, and mcmanus 2010, p. 3). the literature argues for a strategy of providing appropriate role models to girls (ahuja, 2002; ashcraft, eger, and friend, 2012; kaygan 2016). generally female speakers from industry describe what an it career might look like, with the aim of dispelling some of the stereotypical myths that it is boring and involves working alone at a computer all day (ashcraft, eger, and friend, 2012). role models can have a positive impact on encouraging and supporting women in their it careers, as is the case in other disciplines (bandias and warne, 2009; klawe, whitney, and simard, 2009), although there is little empirical data on the impact such speakers have on stimulating girls’ it interest over the longer term. similarly, drury, siy, and cheryan (2011, p. 267) claimed that “maximising a sense of perceived similarity to role models is key in both recruiting and retaining women in stem fields” but that “sharing the same gender might not be a sufficient source of similarity.” 2.5 reinforcing positive messages about it and it careers ashcraft, et al., (2012) report that both boys and girls have either little knowledge, or an incorrect perception of, what an it career involves. girls typically see an it career as for men not women and think that it is ‘nerdy’ and involves sitting at a computer for most of the day with little interaction with other people (ashcraft et al., 2012). in promoting it careers for girls, many programs aim to counter the stereotypical images that girls have of it (craig, lang and fisher 2008). the research outcomes reported from these previous programs focus on the changes in girls’ attitudes to it careers from the beginning to the end of the program, but not changes over the longer term. the computer clubs for girls (cc4g) experience in the uk highlighted the importance of sustained reinforcement (fuller, connor, johnston and turbin, 2009). similarly bystydzienski, eisenhart and bruning (2015) argue that intervention programs need to be ongoing to ensure girls do transition to a technical career. 2.6 factors that influence girls’ attitudes to it. other factors reported in the literature may also have an impact on girls’ interest in it. these factors include: the influence of parents, family and peers; the availability and use of technology at home; early exposure to it; informal education outside the school; and the stereotypical image of it careers promoted in the media and popular culture (bain and rice, 2006; ashcraft, eger, and friend, 2012; clayton, beekhuyzen, and nielsen, 2012). however, in designing our school-based intervention program these factors were considered to be outside of our control. the conceptual model presented below diagrammatically presents the key factors identified from the literature that informed our research design and data analysis. australasian journal of information systems lang, fisher, craig & forgasz 2020, vol 24, research article computing, girls and education 6 figure 1: conceptual model of factors that influence girls’ attitudes to it. the school environment in our conceptual model (figure 1) includes school technology policies, the attitude of the school towards technology and the message the principal conveys about the importance of it in their school. the classroom environment encompasses the ability of teachers to encourage students in their it learning and student-led learning through exploration. ongoing reinforcement refers to the continual emphasis in the classroom that women can and do work in it and that it is a rewarding career. the curriculum design and delivery relates to how it is taught in schools, specifically curricula designed to encourage girls’ interest in it, rather than to teach a computer program or application. role models are women who actively engage with and promote it careers in the classroom. 3 theories informing our research expectancy-value theorists maintain that “individuals’ choice, persistence, and performance can be explained by their beliefs about how well they will do on the activity and the extent to which they value the activity” (wigfield and eccles, 2000, p. 68). to explain gender differences in decisions about mathematics course selection, eccles (1985) proposed the expectancy-value model of academic choice. the model includes a range of interacting psychological and sociological factors that are considered to shape and influence expectations of success and perceptions of the value of the task. contributing factors include: the cultural milieu; socialisers’ behaviours, self-concepts, attitudes and expectations, as well as the individual’s interpretations of them; and the individual’s prior academic history and interpretations of it with respect to success and failure attributions, and perceptions of ability and of task difficulty. the model of academic choice has since been extended beyond mathematics and has been applied to the choice of it career paths. according to zarrett and malanchuk (2005, pp. 75–6): australasian journal of information systems lang, fisher, craig & forgasz 2020, vol 24, research article computing, girls and education 7 individuals’ choice to pursue an it career relates to their perceived ability or mastery of the field and its precursors and how much they value it, as well as the culmination of their experiences and subjective interpretations with the subject matter, cultural norms and stereotypes, and the influence of socializers and peers. the model of academic choice underpinned the contents of the evaluation tools adopted in the digital divas research project. eccles’ model provides insights into the stereotyped perceptions of it that the digital divas club program aimed to challenge. when designing the curriculum for the digital divas club we drew upon vygotskian theory (vygotsky, 1978), in particular, with the zone of proximal development, collaboration with a peer, and the context in which learning occurs. vygotsky describes the zone of proximal development as the “difference between the actual developmental level as determined by independent problem solving and the level of potential development as determined through problem solving … with more capable peers” (vygotsky, 1978, p. 86). the pedagogical approach in designing modules of work encouraged pair and group work in classrooms. a summary of the modules of work are provided below in table 1. the table also provides details of the program assumption (see table 2) that module aimed to address. note that each themed topic was decided in collaboration with a current information technology teacher and year 8 home group leader with an impressive understanding of what topics captured the imagination of girls. topic theme software/hardware learning objectives/outcomes and assumption shake the bottle, wake the brand branding flash or photoshop colour printer internet google apps (online survey) powerpoint assumption 1: curriculum design creating a digital diva identity. designing and creating logos and slogans using a vector-based program. understanding why organisations value branding. brainstorming, planning, branding and marketing to design a unique logo and slogan to represent digital divas for that semester. lights, camera, action communication, collaboration video editing software moviemaker or premier pro or imovie internet assumption 3: it career awareness creating a movie for a specific audience outside of australia. planning, research and design involves script writing, storyboards, mindmapping, production and evaluation receiving feedback from intended audience. (teamwork) chefs’ delight design and creation of an online system excel and internet assumption 1: curriculum design brainstorming, collecting and analysing data, transferring data into a spreadsheet to create an online system. creating combo boxes, macros, if statements, formulae, vlookups, 3d referencing and conditional formatting. fab and famous developing a critical eye for media photoshop assumption 3: it career awareness looking at how the advertising industry edit and modify images for magazines and commercial products for advertising. using photoshop to create a magazine cover that has edited images. discussing the ethics and morals of models and how women are portrayed in the media. australasian journal of information systems lang, fisher, craig & forgasz 2020, vol 24, research article computing, girls and education 8 topic theme software/hardware learning objectives/outcomes and assumption mythbusters research careers (pairs) powerpoint software to watch external movies .avi windows media player/quicktime assumption 2: challenging stereotypes to broaden students’ views on how it is used in different industries. job opportunities for working in the it industry. external speakers. combined with visit to deakin for girls it day. busting four myths about it. alice programming alice 3d free software assumption 1: curriculum design storytelling with alice—introduction to object-oriented programming. understanding algorithms and basic programming concepts such as program variable declaration, conditional and looping constructs. wiki wiki web technologies microsoft office assumption 3: it career awareness web 1.0 verses web 2.0. what does it mean? and what's the difference? broaden your it terminology and compare methods of communication. table 1: summary of digital divas club modules. each module included teacher instructions for projects (called challenges), resource examples, and assessment suggestions. the modules were intended to take four to five weeks (twenty hours of class time) to complete. the modules were not designed to teach a particular it programming language or software application but were focused on developing a crosscurricular broad knowledge and skills in it focused on building computing self-efficacy, while ensuring the activities were collaborative, motivating and interesting to girls in this age group. the first module introduced students to many computing concepts through design and branding (see appendix 1) and involved them in creating artefacts to identify their club (posters, keyrings and t-shirts). the costs of these artefacts were covered with the grant money. their value in allowing female students to identify as part of the club helped in normalising computing as a space for females and contributed to a stronger expectancy-value of it. resources provided to the teachers gave them strategies to promote active student-led learning in their classroom. we were not aware of the levels of it confidence and skill each student brought to the program, but research and experience told us that students enjoyed learning with and from peers (nuthall, 2007) and that they come into the classroom with knowledge from their life experiences (gonzález, moll, and amanti, 2006). in designing our curriculum, we were consciously designing a non-standard approach to delivering it subjects in schools. more detail about the curriculum is presented in the following section and in appendix 1. 4 our research approach our research was conducted within the discipline domains of information systems (is) and education. our methodological stance was interpretivist, that is, we focused on people and the ‘social world’ rather than the natural world; we did not set out to test hypotheses. interpretive research approaches are particularly valid when looking at rich phenomena that cannot be easily described or explained by existing concepts or theories (walsham, 1995). our philosophy in designing the research was that one approach would not provide us with the depth of understanding we sought. in the words of miles and huberman (1994, p. 40), “we australasian journal of information systems lang, fisher, craig & forgasz 2020, vol 24, research article computing, girls and education 9 have to face the fact that numbers and words are both needed if we are to understand the world.” our study employed a mixed methods approach, that is, one in which both quantitative and qualitative data are gathered and analysed (venkatesh, brown and bal, 2013). consistent with quesenberry and trauth (2012) we recognise the issues relating to girls and technology are organisational and social and “not essentialist generalisations about men and women” (quesenberry and trauth, 2012, pp. 457–8). we therefore sought to understand how schools and curricula influence girls’ attitudes to it. in the next sections we briefly describe the program, the participants, the data gathered, the approach taken to evaluate our program, and how we analysed the data. 4.1 our digital divas club the digital divas club was embedded in the school curriculum as recommended by fuller et al. (2008). it ran from 2010 to 2012. ten schools participated in the program during this time. the program was typically implemented as an elective (non-core) subject. delivery varied in each school; some schools implemented the program for between ten and twenty weeks with two to four lessons each week. five schools ran the program more than once, some over several years. in co-educational schools classes were run as girls’ only, a condition set by the research team. the classes varied in size from thirteen to twenty-four girls. the schools varied in type, geographic location, socio-economic background, and ethnic mix of the students. some participating schools were co-educational (6), others were single-sex (girls only) (4); some schools were in economically deprived areas (6), others in wealthier suburbs (4). one school was a select entry school; another was a single-sex independent (nongovernment) girls’ school. appendix 2 provides more information on the schools and the years they delivered the digital divas club. data were gathered from six schools in 2010 and three new schools in 2011. four schools with new teachers ran the program for a second time and data were gathered from them. data were collected from one interstate school in 2012. as the program was running each school was visited regularly by the researchers and classes were observed. each class also had a visit from a female it professional (a role model) who spoke about her career experiences and school and university pathways that led to her current career. a female undergraduate it student, called an expert diva (a role model), was assigned to each class and supported the classroom teacher. she was paid by the research grant. the visiting speaker was an important component of the program to reinforce that women do work in the it industry; the expert diva usually highlighted a more recent pathway into an it career. 4.2 evaluation of the program an important reason for conducting program evaluation is to assess its effectiveness both for the program designers/organisers and future program organisers (rossi, freeman, and lipsey, 1999). in designing the digital divas club we were cognisant of the criticisms expressed of other intervention programs such as: “too much of the research into the gender composition of computing includes only formative evaluations (participant satisfaction with aspects of the program) rather than summative evaluations that measure whether predicted outcomes and impact were achieved.” (cohoon and aspray 2006, p. 144). additionally, a detailed investigation into major intervention programs conducted over the last twenty years for women in computing in australia found that the program champions had some australasian journal of information systems lang, fisher, craig & forgasz 2020, vol 24, research article computing, girls and education 10 understanding about how and why they expected their program to work, yet this theory was rarely tested through evaluation (craig, 2014). “detailed evaluations would provide the evidence to point to programs which should be replicated and those which should be modified or abandoned. without this knowledge there is a real danger that the efforts of creating intervention programs will not translate into more girls undertaking computing in the future and that program champions may burn-out and abandon their efforts” (craig 2014, p107). our evaluation therefore included not just preand post-surveys, interviews, and focus groups, but also focus groups held twelve to twenty-four months after the completion of the program. the research team adapted an evaluation framework designed by one of the authors (craig, 2016) and influenced by the logic models developed by the university of wisconsin (2003). the key elements of this framework were the identification of the underlying assumptions about the impact of the program and designing the evaluation to test those assumptions. table 2 details those assumptions. as detailed in table 1, assumptions 1, 2 and 3 were considered and influenced the design of the modules. assumption 4 relates to the teacher and their approach to teaching the modules and assumption 5 underpinned the philosophy of the program and was evaluated at the end. no assumption assumption description 1 curriculum design curriculum modules would engender greater it selfefficacy/confidence in female students 2 challenging stereotypes prevailing stereotypes about women and it would be challenged 3 it career awareness students would gain a wider and deeper understanding of it careers 4 teaching approach teachers delivering the program would encourage students to be creative and curious 5 maintaining enthusiasm student motivation and enthusiasm would be maintained after the conclusion of the program table 2 program assumptions 4.3 data gathering a unique feature of our research was the extensive data gathering undertaken using a variety of research instruments. as far as we are aware this has not been previously done when investigating a school-based intervention program. data were gathered throughout the research period (2010 to 2014) from three sources: teachers, students and expert role-models. the data gathering focused of the five assumptions detailed above as we sought to understand the extent to which we achieved what we set out to achieve. the research adopted ten different data gathering instruments consisting of preand postsurvey instruments, student focus group guiding questions and a series of interview questions. each instrument is available in full at (fisher et al, 2016, pp. 158–74). data gathering from each of the three groups from whom data were gathered are described next. this includes the research instrument used (more detail provided in table 3) and the assumptions we were exploring. student feedback was gathered through: • preand post-surveys (assumptions 1, 2, 3); australasian journal of information systems lang, fisher, craig & forgasz 2020, vol 24, research article computing, girls and education 11 • preand post-program focus groups (assumptions 1, 2, 3, 5); and • focus groups conducted with students at least one year after the program ended (assumptions 2, 3, 5). student survey questions were drawn from a number of previously validated survey instruments (for example; fogarty, cretchley, harman, ellerton, and konki, 2001; dawson and rakes, 2003). they included quantitative data including yes/no questions and questions requiring likert scale type responses (strongly agree to strongly disagree) and follow up questions (please explain). these included how confident the girls reported feeling with computers, their perceptions of an it career and what they thought of women working in it. we also sought their responses to the digital divas club and the modules including the modules they liked the best and the least. thirty-three statements related to confidence, careers, feelings, perceptions, jobs and stereotypes. post surveys gauged the girls’ responses to each of the modules and the program overall. thirty-three statements were the same as those students answered in the pre-survey. students were identified by a number enabling us to compare individual responses and identify any changes in attitudes. focus groups of between four and six girls were conducted before and after the program in each school; in four schools further focus groups were conducted 12 to 24 months after the girls had completed the digital divas club. girls provided feedback on their overall responses to digital divas, the modules and their perceptions now of a career in it groups. we tested assumption 5 in these post focus groups to determine if girls’ attitudes had changed longer term and had maintained their enthusiasm for it as a career. expert role model (expert divas) data was gathered via: • regular journal entries (assumptions 1, 2, 3, 4); and • a post-program interview (assumptions 1, 2, 3, 4). in the interviews and journal entries expert divas were asked for their reflections on how well digital divas had worked in the classroom and school and how engaged they thought the girls were with the materials. in addition, we tested assumption 4 relating to the teaching of the modules. teacher feedback was gathered through: • a preand a post-survey (assumptions 1, 2, 3, 4); and • a preand a post-interview (assumptions 1, 2, 3, 4). we asked teachers a range of questions in interviews before and after they taught the program. questions included how the classes ran, theirs and the girls’ responses to the modules. personal and school demographics were obtained via a survey. we tested assumption 4 through interviews asking questions relating to how they taught the modules 4.3.1 data collected all data were collected at different intervals at each school between 2010–2014. table 3 provides summary details of the data collected over the duration of the program. australasian journal of information systems lang, fisher, craig & forgasz 2020, vol 24, research article computing, girls and education 12 data legend data type participants mixed-methods (predominately quantitative) pre-survey for students 265 mixed-methods (predominately qualitative) post-survey for students (full instrument: fisher et al., 2016, pp. 162–5) 199 qualitative pre-focus groups with students (31 conducted) (full instrument: fisher et al., 2016, p. 170) 134 qualitative post-focus groups with students (30 conducted) (full instrument: fisher et al., 2016, pp. 170–1) 108 qualitative follow-up student focus groups conducted one to two years later (11 focus groups), 4 schools (full instrument: fisher et al., 2016, p. 171) 33 mixed-methods (predominately quantitative) teacher pre-survey (full instrument: fisher et al., 2016, pp. 166–7) 14 mixed-methods (predominately qualitative) teacher post-survey (full instrument: fisher et al., 2016, pp. 168–9) 10 qualitative teacher pre-interview (full instrument: fisher et al., 2016, pp. 171–2) 11 qualitative teacher postinterview (full instrument: fisher et al,. 2016, pp. 172–3) 13 qualitative expert divas post-interviews (full instrument: fisher et al., 2016, pp. 173–4) 13 note: some teachers taught the program more than once, and some teachers did not complete the post-survey. table 3: summary of the data collected 4.4 data analyses to analyse the quantitative data we used the statistical package for the social sciences (spss). it should be noted that the data from both the first and second rounds of data collection were combined and analysed together. although there were small variations in the findings between the schools, there was little concern regarding bias, as all bar one teacher in the second round of delivery was new to the program. nvivo, a qualitative research tool, was used to code, and assist with the analysis of all qualitative data including survey free-text responses (see appendix 3 for the codes used). we used a grounded theory approach, described by bryant and charmaz (2007), urquhart (2013), and saillard (2011), as follows, “categories are grounded in the data thanks to the line-by-line coding, but the categories derived are part of the analytic thinking of the researcher” (saillard, 2011). because we began with defined assumptions, we had some ideas of the likely categories which guided the initial coding decisions; we then allowed the data to ‘speak for themselves’. three of the researchers coded all the data; an inter-coding reliability test was conducted and indicated a 90% agreement on the codes. 5 results in this section we present the results from the perspective of both our starting assumptions and the factors in our conceptual model (figure 1). 5.1 assumptions identifying assumptions at the start was important for the design of our data gathering. we tested each assumption through a wide range of data gathering approaches ensuring each australasian journal of information systems lang, fisher, craig & forgasz 2020, vol 24, research article computing, girls and education 13 assumption was thoroughly investigated. other studies of intervention programs relating to girls and it have typically used only one method to gather data; a survey at the end of the event. none that we are aware of investigated one to two years after the intervention if there was a sustained impact. given we used surveys, interviews, focus groups and observations this we argue strengthens our findings. next, we discuss the findings as they relate to each of our assumptions. assumption 1: curriculum design we found that the girls participating in the digital divas club did increase their selfefficacy/confidence in it. a paired t-test was conducted on the preand post-survey responses. the results are shown in table 4. statement mean (pre-) mean (post-) t sig change 6.2 if something goes wrong on the computer, i panic 3.12 2.78 3.72 < .001 agree to disagree 6.5 boys are better than girls at fixing a computer 2.42 2.22 2.26 < .05 disagree more strongly 6.10 girls are more likely than boys to ask for help when they are not sure what to do next when working on the computer 3.29 3.09 2.32 < .05 agree less strongly table 4: preand post-survey question t-test results the data in table 4 reveal how, after the program, the girls indicated they were significantly less likely to panic if something went wrong with a computer, boys were not considered better at fixing a computer and boys were just as likely to ask for help when having computer problems. the post-survey if students answered in the affirmative to the question, ‘has your confidence with computers changed?’, they were then prompted to explain in what way their confidence had changed. three quarters of the girls who answered this question (128: 76%) indicated that their confidence had improved. of the girls who answered in the negative (67: 34%), seventeen (26%) wrote they were already confident with computers before participating in the program. in the focus group interviews held immediately after the digital divas club, the girls were also asked if their confidence with computers improved after completing the program; the majority answered affirmatively. a matrix-coding query found forty-two instances where the girls self-reported an increase in their confidence. examples of the girls’ comments highlight this: “i felt more confident after the course. most of the stuff was on word, and excel, and photoshop, and my dad, all my family members, use those applications, so i teach them a whole lot of stuff, it makes me feel smart”; and “if i found something was wrong in my computer i got like scared, but here i think i know how to fix it”. there were two questions in the teacher pre-survey that asked them to indicate whether boys or girls were more confident and competent using it. all teachers responded that they thought boys were both more confident and more competent using it. it is apparent that ten to twelve years after margolis and fisher (2002) and adya and kaiser (2004) reported similar findings, there has been no shift in this gendered perception of inherent ability. australasian journal of information systems lang, fisher, craig & forgasz 2020, vol 24, research article computing, girls and education 14 there was not a matching question in the post-survey of teachers. however, there were questions related to student engagement and enjoyment in the post teacher interviews. all the digital divas club teachers said the girls’ confidence had increased. one teacher explained that their increased confidence meant that the girls had enjoyed the class more than expected. while we believe that the digital divas program increased girls’ confidence in it, we do not know if we changed teachers’ gendered perceptions of ability. assumption 2: challenging stereotypes a key focus in the design of the program was that girls would be more aware that women could work successfully in it challenging the stereotype of it being only for men. focus groups conducted prior to the program, indicated that most girls had a negative attitude towards computers and/or a career in it. there were ninety-nine negative comments relating to it stereotypes and only fifteen positive comments. post focus group comments from girls indicate that stereotypes were challenged. comments included: “before, i didn't really see girls with computers going together that much, but now i know it can be a domain for girls” and “this program has shown me that computers are not only for men, but women can enjoy technology as well. i know this because i have enjoyed the program.” assumption 3: it career awareness digital divas gave girls a wider perspective on it careers. in the post focus groups girls recalled information on the different it career possibilities they had learned about. it was clear that girls now saw it a possible career for them. girls’ comments included: “doing this subject i've learnt there's a lot of career opportunities in the it work field”, “it’s not always boring and sitting in front of the computer screen”, “i now know a lot of jobs involve women and computers” and “i think that females can bring more of a difference…they can contribute a lot to that field”. a number of the girls expressed an interest in continuing in it and were considering an it career. one girl, a year after the conclusion of the program when asked about what she remembered said “the most interesting things i learnt from the [digital divas club] were the speakers that came in and talked about what possibilities and opportunities there are in technology.” her eyes had been opened to the possibility of it as a career. in the post focus groups, students were asked questions relating to further it study. a matrixcoding query identified thirty-six positive, twelve negative and seven mixed comments about future study. results from the focus groups conducted one and two years after the program indicated this result was sustained. further, girls who had changed their minds and expressed an interest in further it study, remained interested. assumption 4: teaching approach our research found that the teachers encouraged students to be creative and curious. teachers enthusiasm was demonstrated by the fact that they modified and extended the modules to encourage the girls further. one teacher described how she changed one module by having the girls “go out and take photographs and play with their own photographs”. she went on to explain how the modules gave her ideas of what else could be done “and i was able to adapt them a little bit to suit our student’s needs and likes.” the survey responses from the girls also highlighted this, one student said she enjoyed “[b]eing able to use my creativity while doing it”. the module ‘shake the bottle wake the brand’ used photoshop and was designed to demonstrate to the girls that you can be creative australasian journal of information systems lang, fisher, craig & forgasz 2020, vol 24, research article computing, girls and education 15 in it. one girl’s comment highlights that this was successful she said “i liked working with photoshop especially to edit images and i learnt how to use the presentation tool and i think it would help me in the future as well.” another girl in the post focus group reflected on how you could be creative working in it. the module ‘chef’s delight’ had girls create a website one girl’s comment highlights the change in girls’ attitude to creativity using it “you can design like websites ... you can make them more interesting than normal websites.” assumption 5: maintaining motivation in contrast to other programs focusing on girls and it that we were aware of, we investigated if girls had sustained their interest in it as a result of the digital divas club and if this motivation and enthusiasm was sustained over time. we found students continued to be enthusiastic about it. we revisited four participating schools two years post data collection and conducted eleven focus groups involving thirty-three girls who had participated in the program one or two years previously. we acknowledge that we cannot draw any hard conclusions from these data because of the limited number of girls who participated in these focus groups. the responses, however, do provide some insights into the extent to which the learnings from the program were sustained. girls remembered the class was fun, the different classroom activities and their confidence in using it also appeared to have been sustained: “it extended my interest. it brought it up a bit more, now that i know how to use certain programs. i can be a bit more specific in what it is i want to learn.” several girls said they continued to study it at school and three were considering studying it at university which they would not otherwise have done. this quote from one girl sums this up well: “by doing digital divas we actually considered doing it in university because in [school name] everyone says medicine, engineering. that’s their stereotype. by doing digital divas we actually had an opportunity to learn about it and careers in it and everything. that was just great.” the sustained effect of the program was also evident in its lifespan in different schools and in the wider school community. the year following the final data collection three schools we contacted indicated they would be offering the digital divas club again, and two schools thought it was likely to be offered again. the digital divas club website was made open access through creative commons at the end of 2012, enabling schools to continue to run the program after it had officially finished. further, in 2014 and 2015, several schools that had not participated in the research project contacted our team seeking information about how they could implement the program. one of the teaching modules (healthy menus using excel) is now available on a government funded website, scootle, that provides digital technology curricular resources for practicing teachers (https://www.scootle.edu.au/ec/p/home). our resources demonstrate innovative and interesting ways to engage all students in computing. 5.2 factors that influence girls’ attitudes to it our conceptual model (figure 1) presented earlier was based on factors highlighted in the literature which were found to influence girls’ attitudes to it. our research sought to assess each of those factors with a school-based curriculum. we next discuss each of the factors and our findings. curriculum design and delivery in the post-survey the girls were asked if they enjoyed being in the digital divas club class. of the 198 girls who responded, 172 (87%) said that they had enjoyed it, and twenty-six (13%) said they had not. although most girls elected to join the digital divas club, some were put australasian journal of information systems lang, fisher, craig & forgasz 2020, vol 24, research article computing, girls and education 16 into the class to make up numbers, or because there was no alternative class for them. using an nvivo matrix coding query for the post-focus group interview data, 265 positive comments were identified relating to the curriculum, thirty-eight negative comments, and fourteen mixed comments. examples of positive comments included the following: “the new modules/activities. it was something new we could learn, so it wasn't boring but fun”; “i enjoyed having a challenge. sometimes, it made me have to try really hard to accomplish my goals”. the words the girls used to describe the digital divas club curriculum highlight how the materials had excited their interest. an nvivo text search from the post-survey data identified that the words ‘fun’ or ‘enjoy’ were used 249 times (in a positive way) to describe the program or its content. examples of the comments the girls made included: ‘‘it was fun”; “i enjoyed learning what different tools could do”; “i learnt that it can be fun and involve girls”; and “[t]hat it isn't just for boys and that it's fun”. the school environment and principal implementation of the digital divas club required the support and encouragement of the principal in each school, suggesting that our principals were enthusiastic about technology. pre-surveys completed by the teachers indicated that most of the participating schools did not have a strong focus on teaching it, although they recognised the importance of providing students with access to technology. in interviews, teachers were asked how it was promoted, taught and supported in the school. promotion of it subjects was usually through school handbooks. a couple of the schools did not promote it at all. one teacher explained, “it seems to be a bit incidental that we’ve got a lot of computers around the school, but we don’t seem to promote it as such”. how it is taught also presented challenges. the same teacher said, “ ..there’s still a fair bit of confusion about where ict fits into the curriculum”. another teacher described how students had to wait until year 9 to any it. and “by then they’ve already decided it’s pretty much just word processing and there’s nothing to learn, which is a bit sad.” the ratio of computers to students in our participating schools ranged from 1: 1.2 to 1: 6. seven schools provided data on the availability of it as a subject in students’ final year (year 12); of these only four offered a year 12 it subject. although the pre-interviews with teachers suggested strong support for computing across the school, there were several aspects related to the way this was managed to suggest computing did not have a high profile. one teacher requested a computer laboratory for each of her five digital divas club classes. this was initially provided, but later the laboratory for two classes became unavailable. in another school, despite the fact there was significant interest from the girls for a computing class, it was not timetabled. two schools had no facility for students to print their work. technical issues caused numerous problems, for example, the inability to log on, networks dropping out, reformatting of machines without a teacher’s knowledge, and poor technical support. the classroom environment in our study, although it was not mandated, all but one of the teachers was female, most wanted to teach the digital divas club and had attended a training session during their school holidays. of the fourteen teachers who completed the survey, ten considered their computing skills to be ‘moderately high’ or ‘high’, and four indicated that their skills were ‘moderate’. the background of the teachers varied considerably: eight teachers had a formal it background, one teacher was an art teacher, two predominantly taught humanities, one taught australasian journal of information systems lang, fisher, craig & forgasz 2020, vol 24, research article computing, girls and education 17 science, and another taught outdoor education. despite these diverse backgrounds, all but one teacher expressed enthusiasm for teaching the program, the one who did not said she lacked confidence; however she reported that this improved the second time she ran the program. the teachers’ enthusiasm was reflected in how they modified and changed the curriculum. in some cases, modules were adapted to suit a particular cohort; in others, new materials were found to complement the existing resources. this is consistent with the findings of miranda and russell (2011), whose research established that innovative teachers are likely to seek out external resources. some of the teachers themselves grew in confidence as their proficiency with technology increased, as illustrated by the following quotes (school names provided are pseudonyms): i enjoyed the first four modules and i’m not an it teacher. i just teach myself and actually, i think, became pretty good at using those programs in particular. i enjoyed it myself, and i learned a lot myself because i had to. (moffatt) others saw the need to update their knowledge as a positive aspect of the program: it was good to get back into it and see what other people are doing in it and some of the possibilities. it also gave me something to aim for, like, i need to learn something about that. (goldstine) the most significant impact on the girls’ responses to the digital divas club was that the program was run as an all-girls class in all six co-educational schools, as part of a series of electives offered to year 8 or 9 students. the reaction of the girls to all-girls classes was very positive, with sixty-one specific positive comments. students commented positively on the classroom atmosphere, that they enjoyed the learning, that their confidence increased in this environment and they preferred this type of class. a single-sex environment for this elective was a very important aspect and had a strong impact on the students. table 5 provides illustrative comments grouped according to the four themes in the comments made. classroom atmosphere • everyone was really friendly, and everyone listened to the teacher, and it was all really, really, really good • it was fun and comfortable • no boys meant it was quieter. guess what happened, in oral presentations you could hear them • i’m in the [digital divas club] and i’m in another it that is co-ed, and i think i like it with the girls better because everyone’s talking about the same things and it’s more like everyone’s talking with each other • it’s like a more friendly environment and there’s not like boys mucking around and stuff • i’ve been in an it class last year but it was mostly boys and you just sit there and go ‘i don’t want to put my hand up to [show i] need help’. learning was fun • it was just girls, you’d talk, they all knew what we were talking about, and everyone was just happy • it's fun to escape from disruptive boys and have a laugh and work in a class of girls • it was so much fun and i learnt lots of new things. i also liked a class all full of girls increased confidence • they [boys] brag and, like, put you down • because it’s all girls... if it was boys, the boys will take over, say ‘oh you’re doing this wrong’ • it gives us a bit more confidence to actually stand up and give our ideas. if guys were in the class they would just be talking and everything, and you wouldn’t be able to get your point across australasian journal of information systems lang, fisher, craig & forgasz 2020, vol 24, research article computing, girls and education 18 • where it’s all girls, you’re more confident to do stuff. you’re more confident to put your hand up and say, ‘i need help’, because with the boys, they just go, ‘that person needs help. we’re still not going to do any work’, and ‘ha, that person needs help’ better learning environment • i think it’s just to get more work done without the guys being there because they just tend to be loud and don’t really pay attention • most boys muck around in the class, and we actually got work done • compared to a maths class, or a science class, there’s boys that keep on talking, and talking, and the teacher doesn’t focus on you • i reckon we get less done with boys. i’d probably get distracted more if the boys were in the class. i probably wouldn’t participate as much as i did • being in an all-girls class it is easier to learn without disruptions table 5: responses about all-girls classes role models women studying or working in it the digital divas club exposed students to role models at two levels: current it university students (expert divas), and professional women (speakers). the girls’ responses to the students were overwhelmingly positive. typically, the it students acted as classroom assistants, helping the school students with their projects. the girls were impressed with the knowledge of the expert diva, exemplified by comments such as: “i realised she actually knew all the answers to the questions that i had so that was good”; and “she knew what she was talking about”. several girls made reference to the fact that their student was young which they liked, and that they discussed topics with her including university studies and it as a career. industry speakers also had a positive impact. a matrix-coding query of the qualitative survey responses found sixteen specific positive comments from the girls about the speaker who came to their school. comments from the girls highlight this: • by talking to sally, she’s the one who kind of gave us the bigger picture of careers in it and what role women play and everything, and she’s kind of the one who actually made me consider it • when alison came in, i didn’t really know that you could do that and that was part of it, what she did. but it was a change to what i thought about it and that girls can do it as well. • i was amazed with some of the things i could do with it. it was putting words into reality, seeing this person [who] actually does this job and [that] she’s really enjoying it and all that. reinforcement of positive messages about it throughout our program design, we embedded continual reinforcement of positive messages about it, reiterating that it was fun, creative and a good career for women. in part, this was done by the expert divas’ regular attendance in classes, and by the speakers reinforcing the messages. however, the main reinforcement was through the teaching modules, two of which focused specifically on investigating it careers. using film making software the girls were required to make a movie about someone working in it. another module, ‘myth busters’, required the girls to prepare a fiveto ten-minute multimedia presentation busting myths around women and it. other modules also highlighted the types of work someone employed australasian journal of information systems lang, fisher, craig & forgasz 2020, vol 24, research article computing, girls and education 19 in it might do, for example, in the teaching of photoshop, the girls learned about how it is used in the advertising industry to reimage models’ faces. when coding the data, we created nodes around stereotypes in the girls’ comments at the pre (negative and positive) and post-focus groups (challenged, no change and reinforced). from the pre-focus groups, there were ninety-nine negative comments relating to stereotypical views of it. the post-focus group data showed a change, with 286 comments indicating girls’ views had been challenged. representative examples of comments from the girls in the postsurvey included: “i just thought it was like working behind a bench but now i know you can go places and do stuff”; “this program has shown me that computers are not only for men, but women can enjoy technology as well. i know this because i have enjoyed the program”; “doing this subject i've learnt there's a lot of career opportunities in it”; and “i now know a lot of jobs involve women and computers”. 6 discussion our research is unique in that it was an in-depth in class program delivered to girls. we set out to investigate a number of assumptions based on the literature (figure 1). through our extensive data collection over four years we identified the factors specifically important in delivering a curriculum-based intervention program focused on girls and it. the most important finding from our research is that with specifically designed materials focusing on girls’ interests you can change girls’ minds about it. we have demonstrated that it is possible to challenge the stereotype that it is only for men, we can improve girls’ confidence in using technology and very importantly learning about computing can be fun. lastly, when designed carefully, an intervention program such as ours can have a sustained impact and did encourage girls to study it further. for too long intervention programs have been short term, typically one to five days in length and have assumed that the solution to encouraging more girls to study it is to provide them with more information about working in it and it careers (clayton et al., 2012; doerschuk, liu and mann, 2007). to date there has been only one other reported long-term intervention program designed to change girls’ minds about it. this was computer clubs for girls (cc4g) a uk based after school program which introduced girls to it, but this program was not delivered within the school curriculum, it was an optional club outside school hours and did not have the same girls attending each week (fuller et al., 2009). the digital divas club owes elements of its success to findings and suggested improvements from other programs, particularly cc4g. as in our study, fuller et al., (2009) found that cc4g could influence change through the inclusion of it activities which were fun. they found longer term activities (those which ran over several weeks) to be more successful than short term activities such as game playing. learning from cc4g and informed by the methodological stance of the model of academic choice that we had employed when designing the program, we embedded the digital divas club in schools as part of the curriculum to validate the importance of computing to girls. the content was carefully designed to be consistent with vygotsky’s pedagogical principles of peer-to-peer learning and align with the expectations of the victorian state-wide curriculum. the comments from the students related to the classroom environment indicate that this innovative pedagogical approach had a lasting effect on attitudes towards it. australasian journal of information systems lang, fisher, craig & forgasz 2020, vol 24, research article computing, girls and education 20 the curriculum alone is not enough, we investigated a range of school-based factors identified from the literature as presented in our conceptual model (figure 1) that are also important in changing girls’ minds about it. however, there has been no research that we are aware of which examined all these factors in the one study. in figure 2, presented below, we have extended the conceptual model identifying the important elements for each of the five factors to facilitate attitudinal change. figure 2: factors that influence girls’ attitude to it. in summary, the factors needed to effect change are: • the school must be supportive and willing to implement an intervention. in our experience this was critical. those schools where the principal supported the program are the ones that continued with digital divas. • the teacher is important in terms of the messages they might deliver about computing and gender. they are also important for promoting a learning environment that encourages creativity and curiosity as well as embraces peer-to peer learning. all girl classes were an important component in the girls’ positive engagement with the curriculum. • the curriculum designed for girls is key; it must tap into their interests if we are to encourage girls’ further it study. messages reinforcing that it is for girls and a career in it can be interesting must be imparted throughout the program through the curriculum and role models. australasian journal of information systems lang, fisher, craig & forgasz 2020, vol 24, research article computing, girls and education 21 • role models are regularly emphasised in the literature as important. a role model closer to the girls’ ages (expert diva) was very effective as the girls were more likely to ask questions about it and it careers of someone not so old. it was beyond the scope of our research to determine if one of these factors was more important than any of the others. however, we know from other research that teachers are second only to parents and family members in influencing student course and career choices (lang 2010, p. 295) so we may be able to extrapolate that a supportive, enthusiastic teacher who encourages girls to consider it is the strongest factor in the classroom. 7 conclusion adya and kaiser (2005) called for a better understanding of factors that influence girls’ it career decisions. they argue that research is needed at three levels one of which is in schools. our research explored this through a longer-term school-based intervention with a curriculum specifically designed for girls. we established that such a program could change their attitude to it, increase their it confidence, and these changes could be sustained over time. to date there has been a paucity of research on how successful a larger, longer term intervention initiative such as digital divas might be in addressing girls’ declining interests in it courses and careers. there is also limited understanding of the types of teaching materials that might best encourage girls to continue with it. our contributions to school level it curricular resources demonstrates how specifically designed materials can support girls’ learning outcomes and extend knowledge and understanding in the field. while it intervention programs for girls previously delivered in australia had been considered successful by the organisers, we found that only limited evaluations had been conducted, making it difficult to identify what factors contributed to the success (craig, lang and fisher, 2008). very few programs were evaluated as extensively as ours, nor were evaluations undertaken to establish if attitudinal changes were sustained over time. we were determined to ‘do it right’ and believe we have avoided this salient warning by an expert in the field: too often well-intentioned individuals embark upon intervention programs without a clear understanding of what 'the problem' is for which the intervention is the solution. at best, such endeavours could be ineffective; at worst such 'interventions' could end up doing more harm than good if they are reinforcing damaging gender assumptions. (trauth, 2012 p. 53). we posit that our digital divas club has delivered more good than harm and provides a roadmap for those looking to implement a similar intervention program. our model ‘factors that influence girls’ attitude to it’ provides guidance for any school wishing to implement a similar school-based it intervention. it is also possible that our approach may have value for intervention programs for other disciplines with similar gender disparities, such as those encompassed in the various stem (science, technology, engineering and maths) areas. the digital divas club is an example of success in the it discipline in particular, linking universities, professional organisations, and school teachers in the collaborative design and implementation of a program that had a positive impact on girls’ perceptions and attitudes to computing. australasian journal of information systems lang, fisher, craig & forgasz 2020, vol 24, research article computing, girls and education 22 ultimately the it industry needs more women and, unless the problem is addressed at the school level, we will continue to see low rates of women undertaking an it career. 7.1 limitations and future research one limitation of this study is that it was conducted in a small sample of australian schools, most in metropolitan areas of one state. however, our findings and recommendations are transferrable. it is possible that researchers from other countries could take our program as a starting point and adapt it to suit their local environment. a second limitation is that the study did not employ measures to weight the importance of each of the factors that influence girls’ attitude. while we posit that teachers are the most important, future research could be conducted to determine which, if any of these factors is more important than any other. acknowledgements this research was funded by an australian research council linkage grant #lp0989469 ethics approval was obtained through monash university: muhrec cf09/2617-2009001507 references abidin, z., mathrani, a and hunter, r. (2018) gender-related differences in the use of technology in mathematics classrooms .the international journal of information and learning technology, 35 (4), pp. 266-284 adya, m. & kaiser, k. (2005). early determinants of women in the it workforce: a model of girls’ career choices. information technology & people, 18(3), pp. 230–59. ahuja, a. (2002). women in the information technology profession: a literature review, synthesis and research agenda. european journal of information systems, 11, pp. 20–34. anderson, n., lankshear, c., timms, c., & courtney, l. (2008). 'because it's boring, irrelevant and i don't like computers': why high school girls avoid professionally-oriented ict subjects. computers & education, 50(4), pp. 1304–18. ashcraft, c., eger, e. & friend, m. (2012). girls in it: the facts. national center for women in information technology (ncwit), usa. bain, c. & rice, m. (2006). the influence of gender attitudes, perceptions, and uses of technology. journal of research on technology in education, 39(2), pp. 119–32. bandias, s. & warne, l. (2009). women in ict – retain and sustain: an overview of the acsw survey. acis 2009 proceedings, 103. http://aisel.aisnet.org/acis2009/103 barrett, (2018) improving women's participation in the security field. in j. tietjen (ed) women in security: changing the face of technology and innovation springer pp. 129-136 bryant, a. & charmaz. c. (2007). the sage handbook of grounded theory. london, sage. butt, c. (2014, sept 8). it crowd shrinking as vce enrolments plummet. the sydney morning herald. retrieved from https://www.smh.com.au/technology/it-crowd-shrinking-as-vceenrolments-plummet-20140907-10dizx.html https://search-proquest-com.ezproxy.lib.rmit.edu.au/abiglobal/docview/2075447159/d0cab4ffdae242c2pq/1?accountid=13552 https://search-proquest-com.ezproxy.lib.rmit.edu.au/abiglobal/docview/2075447159/d0cab4ffdae242c2pq/1?accountid=13552 https://search-proquest-com.ezproxy.lib.rmit.edu.au/abiglobal/docview/2075447159/d0cab4ffdae242c2pq/1?accountid=13552 australasian journal of information systems lang, fisher, craig & forgasz 2020, vol 24, research article computing, girls and education 23 bystydzienski, j., eisenhart, m. & bruning, m. (2015). high school is not too late: developing girls' interest and engagement in engineering careers. the career development quarterly, 63(1), pp. 88–95. camp, t. (1997). the incredible shrinking pipeline. communications of the acm, 40(10), 103–10. clayton, k., beekhuyzen j. & nielsen, s. (2012). now i know what ict can do for me! information systems journal, 22, pp. 375–90. cohoon, j. & aspray. w. (2006). women and information technology: research on underrepresentation. cambridge, ma, mit press. craig, a. (2014). interventions for women in computing: are we evaluating? australasian journal of information systems, 18(2), pp. 91–110. craig, a. (2016). theorising about gender and computing interventions through an evaluation framework. information systems journal, 26(6), pp. 585–611. craig, a., fisher, j. & lang, c. (2007). ict and girls: the need for a large scale intervention programme. acis proceedings, 5–7 dec 2007, toowoomba. pp. 761–9. craig, a., lang, c. & fisher, j. (2008). twenty years of girls into computing days: has it been worth the effort? journal of information technology education, 7, pp. 339–53. craig, m. & horton, d. (2009). gr8 designs for gr8 girls: a middle-school program and its evaluation. sigcse’09. tennessee, acm, pp. 221–5. dawson, c. & rakes, g. c. (2003). the influence of principals' technology training on the integration of technology into schools. journal of research on technology in education, 36(1), pp. 29–49. de bortoli, l., buckley, s., underwood, c., o’grady, e. & gebhardt, e. (2014). icils international computer and information literacy study 2013: australian students’ readiness for study, work and life in the digital age. melbourne, australian council for educational research. deloitte access economics. (2017). australia's digital pulse: policy priorities to fuel australia's digital workforce boom. sydney, australian computer society inc. dennis, i. (2013). australian ict statistical compendium 2013. sydney, australian computer society inc. doerschuk, p., liu, j., & mann, j. (2007). pilot summer camps in computing for middle school girls: from organization through assessment. sigcse bulletin, 39(3), pp. 4–8. drury, b. j., siy, j. o., & cheryan, s. (2011). when do female role models benefit women? the importance of differentiating recruitment from retention in stem. psychological inquiry: an international journal for the advancement of psychological theory, 22(4), pp. 256–9. e-skills. (2014). the women in it scorecard. swindon, e-skills uk. eccles, j. (1985). model of student's mathematics enrollment decisions. educational studies in mathematics, 16(3), pp. 311–14. eccles, j. (2005). studying gender and ethnic differences in participation in math, physical science, and information technology. new directions for child and adolescent development, 110, pp. 7–14. australasian journal of information systems lang, fisher, craig & forgasz 2020, vol 24, research article computing, girls and education 24 evans, c. (2012). recruitment initiatives aimed at increasing the gender diversity within itec employment. equality, diversity and inclusion: an international journal, 31(8), pp. 741–52. fatourou, p., papageorgiou, y & petousi, a. (2019) women are needed in stem: european policies and incentives." association for computing machinery. communications of the acm, 62 (4), pp. 52-57 fisher, j., lang, c., craig, a., & forgasz, h. (2015) "if girls aren't interested in computers can we change their minds?" (2015). ecis 2015 completed research papers.paper 45.isbn 978-3-00-050284-2http://aisel.aisnet.org/ecis2015_cr/45 fisher, j., lang, c., craig, a., & forgasz, h. (2016). digital divas: putting the wow into computing for girls. melbourne, monash university publishing. flanders, n. (1970). analyzing teacher behaviour. reading, ma, addison-wesley publishing. fogarty, g., cretchley, p., harman, c., ellerton, n. & konki. n. (2001). validation of a questionnaire to measure mathematics confidence, computer confidence, and attitudes towards the use of technology for learning mathematics. mathematics education research journal, 13(2), pp. 154–60. fuller, a., connor, h., johnston b. & turbin, j. (2009). the evaluation of computer clubs for girls: final report to seeda. southampton, university of southampton. gonzález, n., moll, l. c., & amanti, c. (2006). funds of knowledge: theorizing practices in households, communities, and classrooms. new york, routledge. haddad, c (2019) ungendering technology: women retooling the masculine sphere. abingdon, oxon ; new york, ny: routledge hall, r., atkins, l. & fraser, j. (2014) defining a self-evaluation digital literacy framework for secondary educators: the digilit leicester project. research in learning technology 22. hattie, j. (2003). teachers make a difference. what is the research evidence? retrieved from https://www.det.nsw.edu.au/proflearn/docs/pdf/qt_hattie.pdf. july 2014. kaygan, p. (2016). gender, technology, and the designers work: a feminist review. design and culture 8(2), pp. 235-52 klawe, m., whitney t. & simard. c. (2009). women in computing—take 2. communications of the acm, 52(2), pp. 68–76. lang, c. (2010). happenstance and compromise: a gendered analysis of students computing degree course selection. computer science education, 20(4), pp. 317–45. lewis, s., lang, c., & mckay, j. (2009). an inconvenient truth: the invisibility of women in ict. australasian journal of information systems, 16(1), pp. 59–76. machado, l. j. & chung, c. j. (2015). integrating technology: the principals' role and effect. international education studies, 8(5), pp. 43–53. margolis, j. & fisher, a. (2002). unlocking the clubhouse: women in computing. cambridge, mass., the mit press. masters, g., (2016). five challenges in australian school education policy insights issue #5 australian council for educational research australasian journal of information systems lang, fisher, craig & forgasz 2020, vol 24, research article computing, girls and education 25 mclachlan, c., craig, a. & coldwell‐neilson, j. (2016). students' computing use and study: when more is less. australasian journal of information systems, 20, pp. 1–17. doi: 10.3127/ajis.v20i0.990. miles, m. b. & huberman, m. a. (1994). qualitative data analysis. london, sage. mims-word, m. (2012). the importance of technology usage in the classroom, does gender gaps exist. contemporary issues in education research, 5(4), pp. 271–8. miranda, h. & russell, m. (2011). predictors of teacher-directed student use of technology in elementary classrooms: a multilevel sem approach using data from the useit study. journal of research on technology in education, 43(4), pp. 301–23. national centre for women and information technology (ncwit). (2014). ncwit scorecard: a report on the status of women in information technology. usa, ncwit. national centre for women and information technology (ncwit). (2015a). women in it: the facts infographic. retrieved from https://www.ncwit.org/resources/women-it-factsinfographic-2015-update. july 2015, national centre for women and information technology (ncwit). (2015b). compugirls: fostering innovation and developing technosocial change agents (case study 1): engaging a diverse range of girls in technology through culturally responsive computing. retrieved from https://www.ncwit.org/resources/how-can-you-engage-diverse-range-girlstechnology/. july 2015. newmarch, e., taylor-steele s. & cumpston, a. (2000). women in it—what are the barriers? paper presented at network of women in further education conference showcase of strategies for women in education on information technology. adelaide, australia. nuthall, g. (2007). the hidden lives of learners. wellington, new zealand, council for educational research. prestridge, s. (2010). the alignment of digital pedagogy to current teacher beliefs. paper presented at acec2010: digital diversity conference. melbourne, australia. quesenberry, j. & trauth e. (2012). the (dis)placement of women in the it workforce: an investigation of individual career values and organisational interventions. information systems journal, 22(6), pp. 457–73. raman, a., don, y., & kasim. a. l. (2014). the relationship between principals' technology leadership and teachers' technology use in malaysian secondary schools. asian social science, 10(18), 30–6. rheem, j. (1999). pygmalion in the classroom. the national teaching & learning forum, 8(2), pp. 1–2. ridley, g. & young, j. (2012). theoretical approaches to gender and it: examining some australian evidence. information systems journal, 22(5), pp. 355–73. rosenthal, r. & jacobson, l. (1992). pygmalion in the classroom: teacher expectation and pupil’s intellectual development. new york, irvington publishers. rossi, p., freeman, h. & lipsey m. (1999) evaluation: a systematic approach. thousand oaks, calif., sage. australasian journal of information systems lang, fisher, craig & forgasz 2020, vol 24, research article computing, girls and education 26 saillard, e. (2011). systematic versus interpretive analysis with two caqdas packages: nvivo and maxqda. forum: qualitative social research, 12(1), art. 34,http://nbnresolving.de/urn:nbn:de:0114-fqs1101345. shein, e. (2018) broadening the path for women in stem. communications of the acm, 61(8), pp. 19-21 sheppard, b. & brown, j. (2014). leadership for a new vision of public school classrooms. journal of educational administration, 52(1), pp. 84–96. souza, e. (2018) the era of homo digitus in j. tietjen (ed) women in security: changing the face of technology and innovation springer pp. 47-52 stout, j. g., dasgupta, n., hunsinger, m., & mcmanus, m. a. (2010). steming the tide: using ingroup experts to inoculate women's self-concept in science, technology, engineering and mathematics (stem). journal of personality and social psychology, 100(2), pp. 255–70. timms, m., moyle, k., weldon, p. and mitchell, p. (2018). challenges in stem :learning in australian schools. literature and policy review , australian council for educational research . pp 1-35. retrieved from https://research.acer.edu.au/cgi/viewcontent.cgi?article=1028&context=p olicy_analysis_misc aug 2019 trauth, e. (2012). are there enough seats for women at the it table? inroads, 3, pp. 49–54. underwood, c. (2016) how are computers and the internet being used for out-of-school activities? snapshots july australian council for educational research pp. 1-5 university of wisconsin. (2003). enhancing program performance with logic models course. retrieved from http://www.uwex.edu/ces/lmcourse/. july 2006. urquhart, c. (2013) grounded theories for qualitative research: a practical guide. london, sage. venkatesh, v., brown, s. & bala, h. (2013). bridging the qualitative-quantitative divide: guidelines for conducting mixed methods research in information systems. mis quarterly, 37(1), pp. 21–54. vygotsky, l. (1978). mind in society: the development of higher sociological processes. cambridge, mass., harvard university press. walsham, g. (1995). interpretive case studies in is research: nature and method. european journal of information systems, 4(2), pp. 74–81. wigfield, a., & eccles, j. s. (2000). expectancy-value theory of achievement motivation. contemporary educational psychology, 25, pp. 68–81. zarrett, n. r., & malanchuk, o. (2005). who’s computing? gender and race differences in young adults’ decisions to pursue an information technology career. new directions for child and adolescent development, 110, pp. 65–84. australasian journal of information systems lang, fisher, craig & forgasz 2020, vol 24, research article computing, girls and education 27 appendix 1 “shake the bottle, wake the brand” the challenge you are to give the digital divas a new brand image. create an original logo for “digital divas” and a slogan to match using a vector drawing multimedia software. the logo and slogan will be used to publicise the course and may be used on merchandise such as key-rings, posters, tee-shirts, lanyards etc. the logo will also appear on all letterheads relating to any correspondence from the digital divas. you should research existing logos to gain inspiration to design your own but should adhere to copyright laws. the website below should answer any questions on copyright issues: study the rubric below to ensure that you hand in all the work to receive your best mark. you have the option of working in groups or individually http://www.smartcopying.edu.au an online vote will be available for both the logo and slogan. challenge objectives: ❖ creating vector images ❖ organising work using layers ❖ use the drawing tool bar to draw objects http://www.smartcopying.edu.au/ australasian journal of information systems lang, fisher, craig & forgasz 2020, vol 24, research article computing, girls and education 28 rubric provided key • h = high • m = medium • l = low • n = not satisfactory logo marks available marks awarded h 20 m 10 l 5 n 0 the logo is uniquely drawn, using original ideas and ideas from existing logos. explain in detail the tools that you have used to create your final logo demonstrate an organised structure to your drawing by including layers and name each layer with an appropriate name. provide original images of the photos that you used to create your final picture. reference any websites that you used for research to show where you got your inspiration. a suitable slogan has been created total mark 110 grade australasian journal of information systems lang, fisher, craig & forgasz 2020, vol 24, research article computing, girls and education 29 appendix 2 schools participating in digital divas club school type years run bartik government co-educational 2010* (pilot), 2011*, 2012 (two semesters) * clark government girls only 2010*, 2013 forsythe government girls only 2011*, 2012 goldstine government co-educational 2010*, 2011* holberton government co-educational 2010*, 2011*, 2012 koss government co-educational, regional 2011* mayer government, co-educational 2011* mcallister government girls only 2010*, 2011*, 2012 moffat government, co-educational 2010* spertus non-government girls only (new south wales) 2013* pseudonyms have been used for the school names. some schools elected to run digital divas club more than once outside our research program, an * indicates the year data were gathered from each school. all but one school was based in the city. australasian journal of information systems lang, fisher, craig & forgasz 2020, vol 24, research article computing, girls and education 30 appendix 3 nvivo coding1 copyright: © 2020 lang, fisher, craig & forgasz. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v24i0.1783 1 nvivo screen shots reproduced with permission from qsr international http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems lang, fisher, craig & forgasz 2020, vol 24, research article computing, girls and education 31 paper title (paper title style) australasian journal of information systems lim 2016, vol 20, post publication review publication anxiety, quality & journal rankings 1 post publication review coulthard, d., & keller, s. (2016). publication anxiety, quality, and journal rankings: researchers views. australasian journal of information systems, 20. doi: http://dx.doi.org/10.3127/ajis.v20i0.1262 review journal ranking systems are impacting academics in research-intensive universities worldwide. the paper by coulthard and keller contributes to existing knowledge in the area by empirically investigating the effects of journal ranking systems on research and the welfare of researchers, specifically in the discipline of information systems. using chi-square tests and descriptive analyses on survey responses from a sample of 275 attendees of information systems conferences, the paper concludes that journal ranking systems have mixed impacts on research and the welfare of researchers. more specifically, in terms of research, the study finds that though journal ranking systems have been perceived as increasing publication quality (positive impact), such systems have also been perceived as inhibiting innovative, risky research and encouraging safe, conforming mainstream orthodoxy, which result in a lack of diverse, innovative, and ground breaking research (negative impacts). in terms of the welfare of researchers, the study finds that journal ranking systems contribute to the pressure to publish and to publication anxiety, especially in instances when time for research is insufficient and when regional, cross-disciplinary and qualitative research are threatened by publication bias by journals and its editors (negative impacts); no positive impacts of such systems on the welfare of researchers were reported. notwithstanding the noteworthy empirical contributions from the study by coulthard and keller, this review identifies three substantive issues that may be of interest for future research. first, the state of theoretical contribution in the existing literature resulting from journal ranking systems needs to be framed and discussed more clearly and objectively. for example, the use of the technology acceptance model as a theoretical lens in information systems research is characterized as a form of orthodox, incremental research, which is considered undesirable by coulthard and keller as well as several other scholars (benbasat and barki, 2007; grover and lyytinen, 2015). however, proponents of the theory may have an alternative and justifiable view of its use and value—e.g. the relevance and value of technology acceptance studies are predicated on its ability to identify avenues for attitudinal and behavioural change to encourage greater acceptance of myriad technologies among diverse communities. it is important to note that technology acceptance studies today are not limited to basic, conventional concepts, such as perceived ease of use and perceived usefulness. instead, noteworthy efforts are being made to extend (e.g. through theoretical integration; lim, 2015) or apply (e.g. in experimental settings; teh et al., in press) the technology acceptance model meaningfully. this is part and parcel of theoretical extension (i.e. a type of theoretical contribution accepted by top journals), and thus its importance should not be downplayed. more importantly, it is necessary to acknowledge that not all research will start from scratch or will challenge existing theory (i.e. other types of theoretical contribution accepted by top journals). a good way to put this understanding in perspective is that the existing literature, especially in top journals, has been largely shaped by the former but not the latter two types of theoretical contribution, and thus, greater visibility of the latter types of theoretical contribution may be encouraged. second, a realist perspective on research and journal ranking systems suggests that it may not be realistic to expect all research investigations to be highly innovative and ground breaking. any potential attempt to pressure top journals to only progress manuscripts into the review process on the basis of such ideals will likely produce adverse impacts, such as greater pressure and anxiety among researchers. the case of the senior editor of an a* publication that rarely receives, let alone publishes, innovative exciting research, despite the many editorials he wrote or the associate editors he appointed to solicit highly innovative and ground breaking research australasian journal of information systems lim 2016, vol 20, post publication review publication anxiety, quality & journal rankings 2 papers, is an excellent example by coulthard and keller that clearly shows that not all researchers are highly innovative and capable of producing ground breaking research. indeed, it is easier to produce incremental contributions as compared to highly innovative, ground breaking contributions. thus, it might be more realistic for researchers to focus on meeting criteria such as clarity (e.g. easy to understand, sharp and straight to the point), novelty (e.g. conceptual and practical originality), and rigor (e.g. search for and/or rule out alternative explanations, triangulation) in research investigation and reporting, rather than the degree of research contribution (e.g. incremental to ground breaking), when considering to publish in top journals. after all, academics are not superheroes who can solve world problems overnight. instead, it is reasonable to say that the incremental contributions contributed by the global network of scholars in the field forms the extant contribution in totality, which may equate the impacts made by a single ground breaking research, which may rarely or never occur. third, academic support for the positive impacts of journal ranking systems on research as well as on researchers themselves is lacking or virtually non-existent. if we consider the fact that the absence of research ideals of being highly innovative and ground breaking is lamented by academics in the work of coulthard and keller (and if these academics insist to strictly follow and materialize these ideals), then further investigation into the negative impacts of journal ranking systems on research and the welfare of researchers are discouraged on the basis of these ideals, so as to avoid falling into the loop of incrementalism and orthodoxy. instead, further research is highly encouraged to take up the challenge to use highly innovative methods to produce ground breaking insights into the perspective of researchers who have garnered benefits from and are in strong support of journal ranking systems. a possible way to do this may be to recruit academics who are in favour of journal ranking systems and subsequently conduct studies and use methods that triangulates the researched phenomenon in greater depth. in short, the findings by coulthard and keller and the substantive issues raised in this review should contribute to greater understanding of the impacts and usage of journal ranking systems. more importantly, academics in research-intensive institutions need to acknowledge that academic success requires hard and smart work. in the contemporary era, where competition is becoming increasingly intense, academics need to form an entrepreneurial mindset (i.e. a mindset that thrives on curiosity, opportunities, innovation, and new value creation and that stimulates work behaviour beyond paid hours) instead of an employee mindset (i.e. a mindset bounded by a glass ceiling and that simulates work behaviour within paid hours). with an entrepreneurial mindset, stress will likely not be taken negatively, but rather more positively. this should help academics cope better with the pressure and anxiety that they often encounter in the global academic world that revolves around journal ranking systems. such a mindset should also motivate greater exploration of the alternatives to journal ranking systems among opposing academics engaged in the pursuit of finding a more acceptable means of effectively and efficiently measuring, evaluating, and enhancing research performance of academics and universities. weng marc lim swinburne university of technology (sarawak campus) wlim@swinburne.edu.my / lim@wengmarc.com acknowledgment many thanks to darryl coulthard and susan keller for contributing to an empirical piece on the impacts of journal ranking systems on research and the welfare of researchers in the discipline of information systems. their work is useful to understand the contemporary struggles of academics in research-intensive universities. author response we thank dr lim for his kind review. within his review, he makes three interesting observations: (1) that the contribution of theoretical work needs to be discussed more australasian journal of information systems lim 2016, vol 20, post publication review publication anxiety, quality & journal rankings 3 objectively. dr lim, using tam as an example, suggests that incremental work does have utility. we accept that view but agree with grover and lyytinen (2015) that much of the research in is, including tam, is incrementalist with limited reach or impact; (2) that most scientific work is incremental in nature and thus most research will be incremental. we do not deny that most scientific work is incremental: “normal science” as thomas kuhn (1970) put it. our fear, supported by our research, is that the journal ranking system is changing the nature of research by exacerbating incrementalism, further heightening a publish or perish culture, and to quote powell and woerndl (2008), ending ‘important work’. the moot question is the degree to which this is occurring as dr lim points out it is easier to produce incrementalist work. (3) the discouragement of incremental work would lead to an impasse in research and thus further increase researcher anxiety. this is only so if one accepts a binary view of research – that it is either incremental or it is innovative. we suggest that it is a matter of degree and motivation and the journal system seems to be weighted towards encouraging incrementalism. something our respondent researchers are questioning. references benbasat, i., & barki, h. (2007). quo vadis tam? journal of the association for information systems, 8 (4), 211-218. coulthard, d., & keller, s. (2016). publication anxiety, quality, and journal rankings: researchers views. australasian journal of information systems, 20. doi: http://dx.doi.org/10.3127/ajis.v20i0.1262 grover, v., & lyytinen, k. (2015). new state of play in information systems research: the push to the edges. mis quarterly, 39(2), 271-296. kuhn, t. s. (1970). the structure of scientific revolutions (2d ed., enl ed.). chicago: university of chicago press. lim, w. m. (2015). antecedents and consequences of e-shopping: an integrated model. internet research, 25 (2), 184-217. doi: https://doi.org/10.1108/intr-11-2013-0247 powell, p., & woerndl, m. (2008). time to stop researching the important things? european journal of information systems, 17(2), 174-178. doi:10.1057/ejis.2008.11 teh, p.-l., lim, w. m., ahmed, p. k., chan, a. h. s., loo, j. m. y., cheong, s.-n., & yap, w.j. (in press). does power posing affect gerontechnology adoption among older adults? behaviour and information technology. doi: http://doi.org/10.1080/0144929x.2016.1175508 copyright: © 2016 lim. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ ajis vol. 4 no. 2 may 1997 revisiting the usability of graphical displays: an experimental approach dan hawthorn1 & ananth srinivasan2 'department of information systems and computing unitec institute of technology auckland, new zealand email: dhawthorn@unitec.ac.nz 2department of management science and information systems university of auckland auckland, new zealand email: ananth@auckland.ac.nz abstract the paper reports on the results of an experimental study of the usability of graphical displays for decision making. the literature points to inconsistent results that have been obtained by researchers looking at this issue. instead of trying to find the 'best format* for a given task, our approach in this study examines common factors across various graph formats. our focus is on the cognitive demands placed on a user instead of the task itself. it was found that assisting users to extract information from graphs with design features such as a mouse driven display has a significantly positive effect on performance. introduction information systems have acquired increasingly powerful display facilities but our knowledge of how best to use these for communicating numerical information to the user is remarkably limited. it is the position of this paper that the long history of conflicting results often referred to as the "graphs versus tables' debate, is a persuasive argument for rethinking the terms on which this debate is based. the more recent form of the debate examines the claims of competing graphical formats to best fit particular tasks (lalomia, coovert, and salas; 1992). since clear findings have failed to emerge from this line of work, we should ask whether the graph type versus task typology is a useful way to characterise the problem. although users can easily recognise different graph formats such as line or bar graphs, this does not automatically mean that such classifications are good predictors of decision making performance. again, the debate has its origins in a style of computer system which addressed single tasks for which potentially a best format existed. current users of information systems are more likely to be engaged in the exploration of data and its implications. this implies that the a user is likely to be faced with a multiplicity of exploratory tasks, each of which may be supported by a different format. an alternative approach to the question of how best to display data is based on understanding the cognitive load imposed on the user by specific features of graphs such as scales, keys and legends which may be shared over several graph formats. from the viewpoint of designing useful information systems, the validity of each approach needs to be measured in terms of two related activities: (1) how useful are the results of a particular approach to a designer producing an information display, and (2) how well do the results of such a design effort suit the needs of users. the experimental study reported here contrasts different design approaches and shows that the approach of understanding the load that design features place on the user has the potential to lead to a markedly superior system. the remainder of the paper will briefly review the existing literature, describe the research aims of the study and then present the method and results before concluding with a discussion of the findings. research background studies of the merits of graphs for displaying information date back to the 1920s. concern as to the nature of good versus bad displays has been an issue for both experimental psychologists and statisticians. the two major works on what constituted desirable graphics, berlin (1967) and tufte (1983) were prescriptions based on the authors' reasoning and intuition, not on experimental results. numerous other studies have attempted to provide research based guidelines on what formats were good or bad and what discriminations readers of graphs were capable of. the topic has been adopted as part of the is field with the emergence of readily available graphic displays as part of the toolkit of the modern computer interfaces. reviewers such as jarvenpaa and dickson (1988) and desanctis (1984), note that the existing findings are inconsistent and plagued by methodological problems. the current state of affairs is admirably 46 ajis vol. 4 no. 2 may 1997 figure 1. line graphs and trends up by yuen (1993) when she states: "the current trend in graphs research has been to perform one shot, ad-hoc studies without any significant effort to build upon the work of others and achieve a state of relatedness among studies". jarvenpaa and dickson (1988), in a summary of experimental work contrasting the use of graphs versus tables in decision support problems, and yuen (1993) in a summary of later work in the same area, show that there is no consensus of findings supporting the notion that graphical "feel" for the data automatically translates to better decision making when graphs are used as a way of presenting information. this is, on the face of it, a problem since if one has a graph such as figure 1, one can clearly state that site 1 has faster growth in units sold than site 2. the obvious solutions are that either the data often does not follow such conveniently clear patterns as the example or that more realistically, typical tasks need information which does not match that which is easily available from the particular graphs that studies have used. following the latter argument, vessey (1991) suggested that one could look at the cognitive fit between the task and the graph format. in their review, jarvenpaa and dickson (1988), found some empirical support for a fit between some tasks and some graph formats for simple tasks, but were unable to extend this to more complex, decision support tasks. they also suggested that some of the earlier confusion could be resolved by looking at the type of task the display was attempting to support. tasks depending on detailed use of point values would be better supported by tables, while tasks which required a summary of data would be better supported by graphs. bar graphs should best fit tasks which required comparison of values while line graphs .should best support tasks which dealt with trends. however the more recent studies reported below do not provide consistent support for these recommendations. although the area is sometimes referred to as the "graphs versus tables" debate, the concern is more generally with what presentation format is best when users need to extract information about sets of numeric data. while traditionally studies have compared graphs versus tables, there has recently been a lively interest in comparing different graph formats. in the last few years several themes can be seen in publications about graphical and tabular formats. the first theme is the development of a concern for which particular forms of display may best be used for a particular type of task. lalomia, et al, (1992), illustrate a recent example of this approach. a second (related) theme is the pursuit of approaches from psychophysics to contrast particular graphical display formats. typical studies are those of legge, gu and luebker (1989), spence (1990) and spence and lewandowsky (1991). the third theme is the use of cognitive modelling to understand the effect of features common to more than one graph format, typified by the work of lohse (1993a, 1993b). in these studies, the techniques for research into the perception of graphs use methods derived from the initial goms study by card, moran and newell (1980). a fourth theme is the use of graphic versus tabular displays as one factor within a study of more complex and realistic decision making. yuen (1993) provides a review summarising the divergent findings from this approach. research aims and derivation of research design the intention of this study is to answer two basic questions. • is graph type a more powerful predictor of performance than variations in features common to several graph formats? • can a design, based on pursuing detailed understanding of such common features, outperform traditional graph displays across several graph formats? assisting users of graph displays the study looked at providing assistance to users based on the details of our view of task difficulty. extensive pilot studies on the effect of scales on point estimation indicated that two significant cognitive tasks were involved. first the subject had to fix and remember two visual positions: one on the graph and the other on the scale. this became more difficult as the distance between the two increased. second, the labelling on the scales required the subject to estimate the position in tenths, fifteenths or twentieths of units from the labelled points. difficulty increased as the mental subdivision of the scale became more complex. a logical issue to study in this context is the manner in which a user 47 ajis vol. 4 no. 2 may 1997 may be assisted in overcoming these difficulties associated with extracting meaning from the display. it is important to gain an understanding of the efficacy of a feature such as scale, in terms of the cognitive load imposed by its subtasks. two useful approaches of providing assistance were pursued. the first approach, a grid superimposed over a graph, can be seen as providing a traditional way of assisting a user with these difficulties. this approach, while providing assistance for interpretation of the display, ignores the issue of user control over the invocation of such assistance. an alternative approach is based on the notion of user control in the determination of whether assistance is required during task execution. knowledge of the subtasks involved in scale reading was therefore used to form a new design for assisting graph users in scale reading. this approach utilised a mouse tool to superimpose necessary grid-like lines on the display as required by the user. details of the implementation of this approach are presented in subsequent sections. task complexity using a graph implies that the user is engaged in an overall cognitive task for which information is required. a desirable graph display allows the extraction of accurate information but also imposes minimal cognitive load on the user. this ensures that the effort of using the graph competes as little as possible with the cognitive effort being expended on the overall task. to this end, graph displays can be contrasted as to their transparency by comparing their effect over a range of tasks of varying cognitive difficulty. the more transparent display should have less effect in impeding complex tasks. to provide a test for this, the tasks addressed by subjects in the study varied over a range of complexities. experimental factors the dismal record of conflicting findings in this area means that a single study under limited conditions has limited external validity. the study therefore varied the conditions found to be significant in scale reading: distance from the scale and labelling interval used on the scale. the study also aimed for a degree of replication by repeating the basic study over three different graph formats; it is believed that this adds considerably to the generality of the findings. the presence of the different graph formats also allows us to examine our first research question on the contribution of graph format to task performance. performance traditionally experimental studies on this topic have measured the percentage of accurate responses or the time taken for task completion as performance measures. in the present study (involving point estimation from fine scales), the proximity of the estimate to the correct answer is a more appropriate indicator of performance than the number of trials in which absolute accuracy was achieved. this paper will concentrate on reporting the average size of the errors made by subjects under varying conditions (though data was also collected on time taken per trial, and the subjects' self reported estimates of: (a) maximum error in a trial and (b) stress). time has been used as an indirect measure of the cognitive effort involved in using the display where the time unaccounted for after allowing for actions such as keystrokes, eye movements and target acquisition is presumed to indicate cognitive activity (lohse 1993a). a point of contention here is whether task completion time is a direct and simple measure of the goodness of a display format. subjects, experimental method and detailed design subjects were 45 polytechnic students engaged in full time business computing studies in years 1 and 2 of a three year diploma. they were experienced windows 3.x users. subjects were tested for colour vision and ability to follow the experimental procedure in a preliminary practice session. the experiment was run under windows 3.1 on 486/33 machines under vga resolution. the experimental design consisted of nine experiments divided into three groups of three each. each experiment differed in terms of graph format (bar graphsgroup g, curve and line graphsgroup h, and line graphs group i) and level of assistance offered to the user (no assistance level 1, superimposed grid on the display level 2, and availability of a user invocable mouse tool level 3). subjects were randomly allocated to one of the graph type groups, 15 subjects per group, and then did the three experiments (varying assistance levels) within their group in randomly assigned order. each experiment is henceforth referred to by a letter-number combination representing the graph type (g/h/i) and assistance level (1/2/3). hence experiment h3, for example, refers to the curve and line graph type with a mouse tool. 48 ajis vol. 4 no. 2 may 1997 each of the nine experiments was a randomised 2 x 2 x 3 factorial design varying the labelling interval used on the graph's scale, the distance of the decision point from the scale and the level of cognitive difficulty of the task given to the subject. labelling intervals were set at either 10 or 20. distances between the critical point on the graph and the scale wetc set at either 4cm or 17.5cm. cognitive level for questions was varied as follows. • level 1 restricted subjects to information contained in the question and within the graph. (example: "what is the height of the red bar at site 5 ?"). • level 2 requires subjects to use (and hold in working memory) an extra source of information in order to be able to relate the question to the graph. (example: "what is the value of copper at site 1 ?" where a key relates the colour of the relevant line to the term "copper" used in the question). • level 3 asks questions which extend the memory load of level 2 by additionally requiring the user to manipulate the information obtained. (example: "what is the difference between the price of lead at site 1 and zinc at site 2 ?"). an experiment thus consisted of twelve trials. an extra trial was included at the beginning of each experiment to provide practice; results from this first trial were discarded from subsequent analysis. in each trial the experimental program displayed a graph. when the user clicked a button to begin, a question pertaining to the data displayed by the graph was presented and the answer, along with the time taken to obtain it was recorded. following each trial the subject was prompted for their estimates of accuracy and stress. screen snapshots for two such trials are shown below in figures 2 and 3. the mouse tool allowed users to create a horizontal line from any point on the graph to the scale. the line was created by positioning the mouse cursor and pressing the left mouse button. if the mouse was moved with the button still held down, the line could be dragged to a new position. this allowed target acquisition by dragging the line onto the target. when the left button was released the line became fixed. the system labelled the line with a number indicating the position of the line on the scale. any number of such lines could be added to a graph when the mouse tool was available. pressing the right hand mouse button erased all lines drawn up to that point allowing the user to start again. at the end of each trial, any lines drawn previously were automatically erased. for the h3 experiment (which displayed a curve intersected by a line and both x and y scales) the mouse tool was extended to drop an additional labelled vertical line to the scale on the x axis. //hot is the y value of the line when xis? graph characteristics group h. (line + curve) assistance level: grid cognitive level: 1 distance from scale: 1 labeling interval: 20 answer buttons tiialnunbet fl |of 13 figure 2: sample screen from group h showing grid display with line + curve graph 49 ajis vol. 4 no. 2 may 1997 seabird numbers by site what is the value of gannet at site 5 answer buttons lets 28 13 29 14 30 15 31 16 32 next trial 17 33 18 34 19 35 20 36 begin 21 37 zz 38 23 39 24 40 25 41 26 42 27 more trial numbet |g~] ol 13 graph characteristics group i (line graphs) assistance level: mouse tool cognitive level: 2 distance from scale: 5 labeling interval: 10 figure 3. sample screen from group i showing mouse tool used with line graph. the availability of the mouse tool was indicated by the presence of a mouse tool icon next to the answer buttons and by the mouse cursor changing to cross hairs inside the graph area. a manipulation check was included to make sure that in g3, h3 and 13, the subjects actually remembered to make use of the mouse tool. two subjects out of 15 in g3 were found to have made no use of the mouse tool and results from these subjects were excluded from the analysis. all subjects in h3 and 13 and all remaining subjects in g3 used the mouse tool for all trials. analysis of the results suggested that two types of error were present: small errors due to subjects' limits in reading points against scales and very large errors where the subject had misread the question and attempted to answer for the wrong line or wrong bar. cases involving such extreme errors and/or times taken (>= 60 seconds) were excluded. vertical distance from the baseline, answer button position, offset from scale labels and order of trial as were controlled for by randomisation and by treating these values as covariates. results although our dependent variable of primary interest is the error amount (i.e. number of units away from the correct response), data was also collected for users' estimates of their error levels and stress, and the amount of time taken for task completion. it can be seen from table 1 below that cognitive level, distance, graph type and level of assistance all strongly affect error amounts. labelling interval also affected errors though not as strongly. variable graph type assistance level cognitive level distance from scale labelling interval error amount 11. 227 (.000) 176.684 (.000) 49.262 (.000) 9.807 (.002) 4.182 (.041) error estimate 44. 158 (.000) 37 1.536 (.000) 42.777 (.000) 5.565 (.01 8) 1.202(ns) stress 32.341 (.000) 352.520 (.000) 29.616 (.000) 3.622 (.045) 0.964 (ns) time 74.232 (.000) 2.505 (ns ) 576.846 (.000) 1.813 (ns) 1.486(ns) n=1525 table 1. analysis of variance: table shows f-values with significance levels in parenthesis. (analysis excludes cases with extreme error values and time > 60 seconds) there are a number of significant interactions which are examined more closely below: for error amounts as the dependent variable: distance x cognitive level, f=7.339 (p=.001); assistance x graph type, f=4.507 (p=.001); assistance x cognitive level, f=4.312 (p=.002); assistance x distance from scale, f=10.457 (p=.000); assistance x labelling interval, f=5.352 (p=.005). similar patterns were 50 ajis vol. 4 no. 2 may 1997 found with the other dependent variables namely self reported estimates of error and stress, and task completion time. table 2 and chart 1 below focus on the impact of graph type and level of assistance provided to the subject. it is clear that the major separation of results is by level of assistance rather than by graph format. the mouse tool offers a significant improvement over the grid except for the case of line graphs (13 vs. 12) though this difference is in the same direction. the grid in turn is consistently significantly better than the unassisted condition. bar graphs have a significant advantage over line graphs only in the mouse tool condition but this can be shown to relate to a potential design flaw in the mouse tool as used with line graphs, not to the graph formats themselves (see the concluding discussion). although overall errors go down as the level of assistance increases it is of more interest to see how the form of assistance interacts with the source of difficulty. it can be seen in chart 2, that both the grid and the mouse tool remove the effect of distance on error amounts compared with the unassisted condition. this is in contrast to the results shown in charts 3 and 4. the grid makes little improvement over the unassisted condition in either reducing the effects due to labelling interval or those due to cognitive level. expt g3 h3 13 12 g2 h2 gl 11 hi mean .0588 .4971 .7184 .7870 .8571 1.1029 1.7314 1.9176 2.0536 g3 * * * * * * * h3 * * * * 13 * * * 12 * * * g2 * * * h2 * * * gl 11 table 2. means of error amounts by experiment (using modified lsd (bonferroni), asterisks indicate significant differences between the column and row group means. 95 % c on fi de nc e in te rv al s on e rr or a m oa nt 2.0' 1.0' .5' 0.0' -.5 h .r ~r ; * : .... '[_"" ~r.. r . = 17! 170 tt t» at 17! w3 174 173 no assistance grid displayed moose tool graph type bar graph a i^lie graph ^ line-tj carve chart 1. error amount: assistance level by graph type 51 ajis vol. 4 no. 2 may 1997 j.u1 2.5' 2.0. 1.5' 1.0' .5' n n n _ _ , 4 1 . ^ , if ' ~~{~ ' 256 2s7 200 252 251 2« no assistance grid displayed mouse tool distance from scale t 5 chart 2. error amount by level of assistance by distance. charts 3 and 4 may be interpreted as showing that the grid only assists problems due to distance. the grid does not ease the problem of interpolation between scale points as shown by the continued spread between easier and more difficult scaling intervals and cognitive levels in the grid condition. 2.5 2.0 1.5 1.0 0.0 t' 9 labelling interval 10 20 259 257 no assistance 258 294 grid displayed 291 24t mouse tool chart 3. error amount by level of assistance by labelling interval. 8 i h i i d.u1 2.5' 2.0' 1 5' 1.0' .5' nn n . . — r ~ -i-"1-tv — 1 <p i3z— • 173 174 199 174 175 193 171 199 194 no assistance grid displayed mouse tool level 1 r * 2 ..,. * 3 chart 4. error amount by level of assistance by cognitive level. the mouse tool however can be seen to make substantial reductions to the effects of both these factors over the grid or the unassisted condition. the overlap for the three levels of cognitive difficulty which only shows for the mouse tool in chart 4 is a strong indicator that the mouse tool is successful in reducing the sources of cognitive load due to scale reading thus freeing the users' cognitive capacity for the overall problem. the patterns shown in these three charts were repeated with little change over each different graph format when considered individually. 52 ajis vol. 4 no. 2 may 1997 it was noteworthy that task completion time was almost unaltered by the level of assistance provided. thus time as a predictor of performance would not have indicated any particular virtue for either the grid or the mouse tool over the unassisted condition. performance subgroups individual scores were studied and it was found that it was possible to identify three subgroups of subjects whose performance stood out from the general level of performance. there were those who avoided any extreme errors, those who were highly accurate once extreme errors were discarded and those who were much faster to complete the trials. interestingly membership on any one subgroup was not a significant predictor of membership of another group. discussion and conclusions the specific findings on the mouise tool, graph format and cognitive load are held to be of interest in themselves. however it is argued that beyond these findings the achievement of the study lies in its support for an approach to design based on pragmatic task analysis rather than design rules. the superior performance of the mouse tool over both the grid and the unassisted case supports the argument that designers can be well served by the strategy of identifying and analysing the cognitive subtasks associated with features common to several display formats rather than searching for a best fit between format and task. the argument that the mouse tool has emerged as an effective response to users' needs is buttressed by the significantly more positive response accorded the mouse tool than to the grid as competing forms of assistance. it is clear that graph format makes a less significant contribution to performance than the cognitive load of the subtasks involved in scale reading. the graph format by task approach leads to designers working within set rules regarding the best graph for job. it seems to be that such guidelines tend to suffer from an over elaboration of rules which exhausts the ability of the designer to take them all into consideration. rule based guidelines also tend to lack guidance about where the applicability of a given rule breaks down as formats are varied. it is not claimed that graph format is rendered irrelevant by undertaking cognitive analysis of task demands. graph formats can themselves impose variations on the cognitive subtasks the designer is addressing. it was observed for example that the mouse tool did not exactly match the needs of users when dealing with line graphs since the tool did not provide a descending line to identify the point of intersection on the x axis. this was needed to accurately locate the site point on the graph line, especially in cases where the graph line did not indicate the site with a sharp angle. thus to be effective, cognitive task analysis should proceed with reference to any special problems arising from a graph format. the abnormally high reading for error amounts when using the mouse tool with line graphs in chart 1 can be seen as a consequence of not making such an adjustment. the mouse tool in its present form manages to combine some of the virtues of tabular and graphic displays, approaches which for years have been seen as strict alternatives. focusing the designer's attention on identification and analysis of areas of difficulty rather than on rules for best design or for best graphs appears to yield payoffs. the view taken in the current study allows the designer to step outside the frame of current approaches and effectively target design issues that need addressing. the weakness of time in predicting the usability of the various designs is important. it is apparent that time spent in manipulation has much less impact on cognitive load (and hence overall transparency) than time spent attempting to remember point locations and estimating sub-intervals from scales. the mouse tool, as implemented at present, supports a single function: evaluating points on a graph. it would be of interest to see if the tool could be usefully extended to support multiple functions where the user chooses the mouse tool function from a menu. the obvious extension to evaluate is to allow the user to set the mouse tool to draw slope lines between pairs of points on the graph. in theory this would mean that trend comparisons are enhanced on bar graphs and trend comparisons between non adjacent sites become easier on line graphs. another area in which the mouse tool could potentially be adapted to deal with areas of difficulty shown by users in the current study lies in the extreme error problem arising from incorrectly identifying the point on the graph which matches the question. it would be possible to get the mouse tool to label its lines with the location information for the point chosen, say "copper at site 3" in a way which the user could check against the question the user brings to the data. it is apparent in this study, that while users appreciate the mouse tool they do not find it quick to use. it would be possible to enhance the tool so as to allow the user to choose a "snap to" option in which the line drawn by the mouse tool snapped to the end of a bar or the nearest site for a line so that the user has only to acquire the bar or line as a target rather than the much smaller point representing the end of the bar or the site point on a line. this imposes significant extra programming complexity but could well be 53 ajis vol. 4 no. 2 may 1997 worth while for pre-packaged graphical tools. potentially the mouse tool or variations on it allow the designer to provide the user with optional access to the desirable features of several display formats including tabular displays while still preserving a straight forward graphic presentation. it was argued in the introduction to this study that we should now see graphs as a tool for exploration of data and that this role for graphs is no longer consistent with a focus on a single match between one task and one graph format. it must be noted though that this remains an assumption and that there is an urgent need for investigation of the range of actual uses to which graphs are put. it is acknowledged that there is a need to check that the findings apply to actual users as well as to student suubjects. in such a study one could also look at the levels of experience which typify users. additionally, given that jarvenpaa and dickson's (1988) objection to allowing users to choose their own graph formats has been overtaken by events, is their prediction of this flexibility leading to increased numbers of poorly formatted graphic displays borne out? are there frequent patterns of reading mistakes or poor design in actual use of graphs which could be addressed by designers. further is it possible to design help systems which can guide users in their choice of graphic displays. it was observed that there were separate subgroups of proficient users who were fast, who were accurate and who avoided extreme errors. it is well worth investigating to what extent the skills of each of these subgroups represent learnable skills rather than innate ability. if a finding of learnable strategies is supported, could one then design training packages which guide users of graphs to adopt techniques used by the most proficient users? overall, it seems that the approach of analysing the cognitive demands of the subtasks involved in dealing with the components of graphic displays offers a solid basis for beginning to tackle the multitude of differences possible in combinations of task and graph format. it is hoped that a shift to pragmatic task analysis can prove valuable for general interface design as we move towards creating interfaces which accurately reflect and respond to users' needs. references berlin, j. (1967) semiologie graphiques [english translation berg, w.j. (1983) seminology of graphics, university of wisconsin press, madison, wisconsin.] card, s.k., moran, t. p., and newell, a. (1980) computer text editing: an information processing analysis of a routine cognitive skill, cognitive psychology, 12, 32-74. culbertson, h.m and powers, r.d (1959) a study of graph comprehension difficulties, av comm review, 7,97-100. grant, s. and mayes, t. (1991) cognitive task analysis in r. s. weir and j. l. ally (eds.) humancomputer interaction and complex systems, academic press, harcourt brace, london. jarvenpaa, s.l. (1989) the effect of task demands and graphic format on information processing strategies, management science, 35, 285-303. jarvenpaa, s.l. and dickson, g.w. (1988) graphics and managerial decision making: research-based guidelines, communications of the acm, 31,764-774. lalomia, m.j, coovert, m.d and salas, e. (1992) problem-solving performance as a function of problem type, number progression and memory load, behavior and information technology, 11,268-280. lewandowsky, s. and spence, i. (1989) discriminating strata in scatterplots, journal of the american statistical association, 84,682-688. legge, g.e., gu, y. and luebker, a. (1989) efficiency of graphical perception, perception and psychophysics, 40, 365-374. lohse, g.l (1993a) eye movement based analysis of graphs and tables: the next generation, behavior and information technology, 12, 213-224. lohse, g.l. (1993b) a cognitive model for understanding graphical perception, human-computer interaction, 8, 353-388. spense, i. (1990) visual psychophysics of simple graphical elements, journal of experimental psychology: human perception and performance, 16,683-692. spense, i. and lewandowsky, s. (1991) displaying proportions and percentages, applied cognitive psychology, 5, 61-77. tufte, e.r. (1983) the visual display of quantitative information, graphics press, cheshire, connecticut. vessey, i. (1991) cognitive fit: a theory based analysis of the graphs versus tables literature, decision sciences, 22, 219-240. yuen s (1993) the effects of what-if analysis and user interface type on decision making, unpublished masters thesis, msis dept, auckland university. 54 paper title (paper title style) australasian journal of information systems tomer & mishra 2019, vol 23, research on the role of tech in workforce mgt expectation from tech & career satisfaction 1 expectation from technology and career satisfaction: a study among it professionals in india gunjan tomer indian institute of management nagpur f10gunjant@iimidr.ac.in sushanta mishra indian institute of management indore abstract technology is an integral part of the work life of any information technology (it) professional. however, the influence of technology on career related outcomes of it professionals is at best limited. using the p-e fit perspective, the present study investigated the implications of the perceived fit/ misfit between the expected and the experienced career growth and work-life balance from the technology the it professionals are working in on their career satisfaction. based on response surface methodology and polynomial regression analysis of the data collected from 286 it professionals, the study found that not only the fit, but also the misfit between expected and experienced technology led career growth and work-life balance is important in explaining career satisfaction. implications of the study for both research and practice were discussed. keywords: it professionals, technology, p-e fit, career satisfaction, work-life balance. 1 introduction it professionals play a critical role in sustaining competitive advantage of an organization (ahmed, taskin, pauleen, & parker, 2017; armstrong, brooks, & riemenschneider, 2015). hence, retaining the professionals and ensuring their high motivation level are important for organizations (ahmed et al., 2017). to ensure their retention and productivity, it is essential that it professionals hold positive valence to their work and career (sarker, sarker, & jana, 2010). in the present study we focused on career satisfaction among it professionals for two reasons. one, it represents “an overall summary of how a person feels about a lifetime of work” (lounsbury et al., 2007, p. 174) and signifies individual’s perceived level of satisfaction in achieving desired career outcomes (jiang & klein, 1999). moreover, studies in different context have found that career satisfaction is related to individual outcomes such as positive valence (fu & chen, 2015), life satisfaction (lounsbury et al., 2004) and self-efficacy (ngo & hui, 2018). two, career satisfaction is related to work outcomes such as employee commitment, work engagement (ngo & hui, 2018), citizenship performance (jawahar & liu, 2016) and turnover intention (rambur et al., 2005). given the importance of career satisfaction, studies investigating the predictors of career satisfaction is at best limited (ng, eby, sorensen, & feldman, 2005). in fact, scholars have argued that “challenge to increase employees’ career satisfaction” is difficult (han, 2010, p. 437), especially among it professionals. since technology1 is core to the functioning of it professionals, in the present study, we proposed 1 given technology refers to the technology in which an it professional is trained and working with. australasian journal of information systems tomer & mishra 2019, vol 23, research on the role of tech in workforce mgt expectation from tech & career satisfaction 2 that perceived career related features of a technology might impact career satisfaction of these professionals. scholars have argued that individuals in the it workforce are heterogeneous (joseph et al., 2012), i.e., individuals evaluate a given technology differently based on their own perceptions and expectations (speier & venkatesh, 2002). in fact, career outcomes of an individual working in a given technology depend on his/her own preferences and the ability of the technology to fulfil his/her career and work-related expectations. since interaction with a given technology defines individual’s work and career outcomes, in case of it professionals, the unmet expectations can arise due to mismatch of expectations in a given technology. scholars have found that unmet expectations from the job setting lead to negative work outcomes among it professionals (chilton, hardgrave, & armstrong, 2005; igbaria, greenhaus, & parasuraman, 1991; lee, 2011). unfortunately, studies providing explanations regarding how an it professional evaluates a given technology are limited. technology is not a homogeneous factor. some technologies are widely accepted in the industry compared to other technologies. as a result, working in some technologies increase market value of the professionals compared to working in other technologies. although demand of it professionals is growing with time, professionals working in certain technologies may receive better job offers than their counterparts (moreno et al., 2012). this leads to increased preferences for acquiring skillsets in some technologies compared to other technologies. it creates differential expectations for high demand technologies among the ambitious professionals. however, the expectations from a technology depend on both the individual and technological prospects. for instance, one professional might prefer to maintain work-life balance instead of working in a demanding technology. on the other hand, another professional may prefer working in a demanding technology over work-life balance. thus, the fit/ misfit between the professional’s expectation and the technology he/she is currently working in might explain his/her career satisfaction. despite a reasonable literature on career satisfaction, challenges persist, indicating, the need for further research especially among the it professionals. in the present study, we have investigated career satisfaction through the lens of person-environment (p-e) fit. our argument is based on the assumption that the fit/misfit between the expectation and experience of working in a technology will explain career satisfaction among the it professionals. 2 literature review while all the diverse activities and experiences comprise one’s career (lounsbury et al., 2007), the extent to which an it professional expresses a positive orientation towards his or her career is termed as career satisfaction (fu & chen, 2015). career satisfaction depends on how well individual career desires are getting fulfilled by the current job (jiang & klein, 1999). scholars suggest that fulfillment of career growth expectation leads to many positive organizational outcomes (gupta, guimaraes, & raghunathan, 1992). given the importance of career for the it professionals, there is a pressing need for the organizations to explore ways to manage career expectations of these professionals (igbaria et al., 1991). in the present study, we investigated career satisfaction of it professionals by exploring the dynamics related to technology. technology is argued to be at the core of the functioning of these professionals and it defines their work fulfillment, job satisfaction and career growth (hsu et al., 2003; speier & venkatesh, 2002). surprisingly, studies exploring the role of ‘technology’ on career prospects of it professionals are limited. australasian journal of information systems tomer & mishra 2019, vol 23, research on the role of tech in workforce mgt expectation from tech & career satisfaction 3 in the present study, we borrowed from the person-environment (p-e) fit theory (edwards et al., 2006, kristof-brown et al., 2005) to explain the linkages between the aspects of technology and career satisfaction of it professionals. the fit approach is well accepted in the information systems (is) literature to understand individual’s perception about the job and the work settings (ayyagari et al., 2011; igbaria et al., 1991; venkatesh et al., 2017). the basic premise of the p-e fit theory is that perceptions, attitudes, and behaviours of individuals are not determined only by the person level factors or the environmental factors independently but from the interactions between them. the ‘environmental’ factors have been redefined in various theories to explain the congruence of individuals with different aspects of their environment such as person-job fit and person-organization fit. we focused on technology aspect of the ‘environment’ in explaining the work outcomes of it professionals. 2.1 technology led career growth (tcg) drawing from the p-e fit theory, the need–supply framework explains individual behaviour. it captures two important dimensions, first, the needs of an individual and second, how the needs of an individual are fulfilled by the environment. the conceptualization of need-supply fit implies that when an individual’s job fits his/her career needs, it will lead to increased career satisfaction (kristof-brown et al., 2005; rehfuss, gambrell, & meyer, 2012). thus, the fit between it professional’s expectation from the work and the work positions he/she currently holds is positively related to career satisfaction (igbaria et al., 1991). based on the importance of congruence, several studies in is literature has employed the p-e fit theory to explain career satisfaction (jiang et al., 2012; venkatesh et al., 2017). career growth captures an individual’s perceptions of the development and advancement possibilities within an organization (jans, 1989; weng et al., 2010). career growth thus consists of two aspects: one employee’s efforts in developing his/ her professional abilities and meeting his/ her own career goals; two, receiving rewards and promotions from the organization commensurate to his/her abilities and contributions (weng & hu, 2009). scholars have argued that individuals’ progress in attaining their career goals influence their outcomes such as career satisfaction (weng et al., 2010). technology being central to the working of it professionals, it can offer better work opportunities and better chances of role progression. for example, aspects of technology, specifically job market demands are argued to influence the career consequences of an it professional (moreno et al., 2012). since technology is accepted as central to the career growth of it professionals, we argue that experienced growth in career due to working in a technology will positively impact their career satisfaction. hypothesis 1: the fit between the expected and the experienced career growth while working in a technology is positively related to career satisfaction. 2.2 technology led work-life balance (twlb) the degree “to which an individual is simultaneously able to balance the temporal, emotional, and behavioural demands of both paid work and family responsibilities” is termed as worklife balance (hill et al., 2001, p. 49). literature has suggested that decreased work-life balance (wlb) among the it professionals may induce negative outcomes such as higher levels of stress and turnover intention (moore, 2000). it professionals experience both ‘time based’ and ‘strain based’ conflict. coordination activities with the global work teams contribute to creating time-based conflict as it professionals have to work according to different time zones (sarker et al., 2010). on the other hand, highly technical and sometimes routine work contributes to strain-based conflict among it professionals. for example, a certain set of australasian journal of information systems tomer & mishra 2019, vol 23, research on the role of tech in workforce mgt expectation from tech & career satisfaction 4 technologies require the it professionals to stay available professionally even beyond office hours. the technologies involved with maintenance, troubleshooting and debugging can be categorized in this group. it professionals working in these technologies tend to experience conflicts in balancing their personal and professional lives. the perceived conflict between an individual’s personal and work life while working in a given technology is expected to influence the perceived work-life balance (chen, powell, & greenhaus, 2009). with increasing demand in the workplace, scholars have noticed decreasing levels of wlb among the it professionals (tarafdar et al., 2007). as individuals differ in their needs and preferences, their career expectations differ significantly. in fact, ginzberg and baroudi (1988) found varying degrees of career desires among it professionals. they further noticed that a set of individuals give priority to personal lives over their professional lives. these individuals assign more emphasis on maintaining work-life balance (sarker et al., 2010). studies suggest that technology that one is working in influences his/her work life balance by influencing work load, work hours (greenblatt, 2002) and work timings (sarker et al., 2010). the discrepancy between the expected and experienced work-life balance might reduce career satisfaction (hsu et al., 2003; igbaria et al., 1991). we propose that if the expected work-life balance level is not achieved while working in a given technology, the individual will experience negative work outcomes such as decreased career satisfaction. on the other hand, if the individual achieves expected work-life balance while working in a given technology, it will enhance his/her career satisfaction. hypothesis 2: the fit between the expected and the experienced work-life balance while working in a technology is positively related to career satisfaction. 2.3 incongruence between person and technology in contrast to the concept of fit, misfit explains the mismatch between the expected and experienced work outcomes from the technology an it professional is currently working in. while fit has been associated with significant work outcomes, a good number of studies investigated outcomes associated with misfit or incongruence (chen et al., 2009; furnham & schaeffer, 2011). the evidence from the literature suggest that incongruence between a person and his/her environment such as ‘technology’ leads to strong perceptions of work outcomes as compared to convergence or perceived fit. in the present study, we have explained whether misfit between the expected and experienced technology impacts it professionals’ career satisfaction. to argue this, we borrowed from the expectation theory (porter & steers, 1973). theoretically, expectations are argued to be impactful as they create anticipation about the situation in the mind of an individual and create a cognitive schema about what should occur (locke, 1976). thus, according to the expectation hypothesis, unmet expectation leads to increase in negative outcomes (shockley & allen, 2018). in addition, the conceptualization of need-supply fit implies that an individual will experience higher positive mental state and lesser psychological stress when congruence occurs, i.e., his/her needs are met by the supply from the environment (shockley & allen, 2018). based on the above discussion, we propose that career dissatisfaction will be more when there is incongruence, i.e., the expectations for career growth or work-life balance is more, compared to the perceived career growth and work-life balance while working in a technology. hypothesis 3(a): misfit between the expected and the experienced technology led career growth, such that the higher the expected career growth compared to the experienced career growth from working in the technology, there is a decrease in career satisfaction. australasian journal of information systems tomer & mishra 2019, vol 23, research on the role of tech in workforce mgt expectation from tech & career satisfaction 5 hypothesis 3(b): misfit between expected and the experienced technology led work-life balance, such that the higher the expected work-life balance compared to the experienced work-life balance from working in the technology, there is a decrease in career satisfaction. 3 method 3.1 operationalization of constructs the operationalization of constructs was one of the methodological challenges in our study. in the p-e fit literature, the operationalization of constructs has been debated in multiple studies. the challenge lies in measuring the extent of fit between the person and the dimensions of the environment. while conceptualizing a fit model, it is imperative to understand the distinction between various models of fit under the overarching p-e fit theory. the first distinction is defined in terms of whether it is a need-supply fit (congruence between the need of an individual and what the environment is supplying) or abilities-demand fit (congruence between the abilities of an individual to meet the demand placed by the environment). another distinction is between commensurate and non-commensurate measures. in the commensurate measure, the discrepancy between the reported p (person) and e (environment) are calculated. the commensurate measures ensure that the measurement of fit is approached on the same dimension. on the other hand, the noncommensurate dimension approach allows the researcher to use different dimensions to measure both the p and the e and then reconcile the two measures to capture the fit by some appropriate methods. finding appropriate construct to operationalize both p and e dimension is another methodological challenge in the measurement of p-e fit. in their excellent work on p-e fit, edwards et al. (2006) favoured the use of commensurate measures to address research questions grounded in the p-e fit model. we thus used the commensurate measures: the person dimension captured the preferences for the outcomes of an it professional while working in a technology and the environmental dimension captured the technology led outcomes supplied by the technology in which the concerned it professional is currently working in. 3.1.1 atomistic approach versus molecular approach there are three basic approaches which are widely used in linking person and environment relationship. the atomistic approach allows the researchers to capture the person and environment component separately and then deduce the combination scores using statistical techniques. the molecular approach is followed when the assessment of fit is direct in a way where “how less” or “how more” (with respect to individuals’ expectations) is captured directly from the respondents. in the molar approach the respondents are asked to assess the fit while responding and subsequently rate the extent of the perceived fit/match. while evaluating and comparing these approaches, edward and colleagues (2006) have suggested adopting these approaches in accordance with the research problem. in the present study, we have adopted the atomistic approach because it allows studying and observing the relative conceptual relevance of person and environment component measures. the atomistic approach also allows the use of commensurate measures which are believed to be more efficient in the context of p-e fit studies (brown et al., 2012; chen et al., 2009). research design based on the atomistic approach records two component measures for each predictor variable on a commensurate scale. this presents another challenge of measuring and combining the component measure. australasian journal of information systems tomer & mishra 2019, vol 23, research on the role of tech in workforce mgt expectation from tech & career satisfaction 6 3.1.2 difference score approach in organizational research, the difference score approach has been widely used in studies of congruence and fit. this approach offers a simplistic approach to obtain difference scores of two component measures. this difference, however, can be an algebraic difference, absolute value of algebraic difference or squared difference based on the proposed relationship (dawis & lofquist, 1984). however, the extant literature establishes various methodological concerns (edwards, 1994; edwards & parry, 1993; klein et al., 2009; venkatesh & goyal, 2010) regarding usage of difference scoring while combining the component measures. difference score approach is also criticized for its assumption of a linear relationship, as linear models do not adequately represent the relationship between two component measures (edwards et al., 2006). 3.1.3 polynomial regression analysis literature suggests that if non-linearity is expected, component measures should be analysed using polynomial regression analysis (pra), which allows interaction of higher order terms in the measurement model (atapattu et al., 2014; edwards et al., 2006). in pra, congruence is not conceptualized as a single score such as difference score, but it rather emphasizes the correspondence between the component measures. congruence or fit is analysed as a continuous curve, not a point, thus, allowing a better understanding of the complex relationship dynamics of fit. despite the advancement in analytical power, coefficients of pra are complex to understand and interpret. to analyse the coefficients more clearly and to extract additional information response surface methodology (rsm) is employed along with pra. response surface analysis involves studying the resulting surfaces which correspond to the polynomial regression equations. thus, it provides both statistical and visual testing of the data. 4 research design the present study attempts to explain the impact of the fit and misfit between the expectations and the experiences (career growth and work-life balance) while working in a given technology on career satisfaction of it professionals. 4.1 sample design the sample frame included it professionals working in service-based it organizations in india. we received 286 usable questionnaires for the study. about 58 percent of the respondents were in the age group of 24 to 28 years, 35 percent in the age group of 28 to 32 years and rest were beyond 32 years. the mean experience of the respondents was 42 months and median experience was 36 months. about 73 percent of the respondents were male, about 66 percent of the respondents were single, 24 percent were married, and 10 percent were married with children. 4.2 control variables the most crucial challenge faced by researchers in establishing the robustness of their findings is the elimination of alternate explanations. based on prior research, we controlled for age, gender, marital status (greenhaus et al., 1990; igbaria et al., 1991; sarker et al., 2010; weng et al., 2010), experience (igbaria et al., 1991), promotability (greenhaus et al., 1990), and negative affectivity (moore, 2000). australasian journal of information systems tomer & mishra 2019, vol 23, research on the role of tech in workforce mgt expectation from tech & career satisfaction 7 4.3 research instruments the measurement scales were adapted from the well-established constructs in literature. scale for technology led career growth was sourced from the work of weng & hu (2009) where they have established a relationship between the perceived career growth and turnover intention. we have adapted the valid scale developed by kopelman, et al., (1983) for measuring technology led work-life balance dimension. the career satisfaction scale was adapted from greenhaus et al. (1990). the scale has been widely used in the literature involving it professionals (jiang et al., 2012). all the responses were recorded on a five-point scale. we performed pre-tests to ensure (a) construed meaning is same for different individuals; (b) the construed meaning is similar as intended. the face validity process was conducted with fellow academicians along with the small subset of the intended population. a pilot test was conducted with 65 it professionals to conduct validity and reliability test on scale items. all the constructs displayed reliabilities above the acceptable limit of alpha greater than 0.70. we examined both convergent and discriminant validity. the average variance extracted (ave) score of at least 0.50 demonstrates that a set of items can represent a single underlying construct, thus indicating construct validity (fornell & larcker, 1981). to ensure discriminant validity, square root of the ave score need to be more than the correlations between the constructs. the ave scores of our constructs ranged from 0.68 to 0.81 (see table 1), and the square root of the ave scores were more than the correlations (fornell & larcker, 1981). thus, all the variables indicated discriminant validity. we mean centred the variables to control for multicollinearity, which is common while using commensurate measures. we also confirmed the absence of influential outliers and problematic deviations. australasian journal of information systems tomer & mishra 2019, vol 23, research on the role of tech in workforce mgt expectation from tech & career satisfaction 8 note: tlcg: technology led career growth; twlb: technology led work-life balance; sd: standard deviation; aalpha: cronbach alpha score n = 286; *p<0.05; **p<.01; the values at the diagonal represent ave scores for the respective variables. table 1: mean, standard deviation and correlation of all constructs. sl. no. variables mean sd alphaa 1 2 3 4 5 6 7 8 9 10 11 1. age 1.70 4.10 -- 2. gender 1.70 0.50 -0.01 - 3. marital status 1.20 0.50 -0.57** -0.12* - 4. previous experience 2.60 0.89 -0.52** 0.05 0.43** - 5. promotability 2.70 0.90 -0.09 0.12* 0.06 0.14* - 6. negative affectivity 4.01 1.60 --0.23* 0.09 -0.16* -0.14* -0.19* - 7. tlcg (person) 6.25 0.96 0.94 -0.03 -0.08 0.01 0.01 0.05 0.00 0.68 8. tlcg (technology) 5.15 1.36 0.94 0.05 0.03 0.02 0.10* 0.35* -0.18* 0.31** 0.72 9. twlb (person) 5.91 1.21 0.91 -0.02 -0.15* -0.02 0.04 -0.04 -0.05 0.36** 0.09 0.80 10. twlb (technology) 4.37 1.70 0.93 0.02 0.13* 0.01 0.07 -0.01 0.31** 0.08 0.10* 0.09 0.81 11. career satisfaction 4.07 1.49 0.90 0.12* 0.09 0.08 0.13* 0.63* -0.24* 0.02 0.50** -0.02 0.06 0.81 australasian journal of information systems tomer & mishra 2019, vol 23, research on the role of tech in workforce mgt expectation from tech & career satisfaction 9 5 results 5.1 analytical representation of the hypotheses the general equation to relate career satisfaction and the fit is represented by the algebraic expression (1) cs = b0 + b1p + b2t + b3p2+ b4p×t + b5t2+e ................................................................................. (1) where, b0, b1, b2, b3, b4, and b5 represent the regression coefficient p – it professionals’ individual expectation t – it professionals’ experienced values p2 – squared term for individual expectation pt – interaction term for both expectation and experienced values t2– squared term for experienced values cs career satisfaction 5.2 testing hypotheses hypothesis 1 proposed that the fit between the expected and the experienced career growth while working in a technology is positively related to career satisfaction. we conducted regression analyses for the fit in career growth aspects of technology with career satisfaction. we entered the terms (p, t, p x t, p2, t2) in the regression analysis after controlling the effect of demographic and other important variables. the results indicated adjusted r square value of 0.27. in hypothesis 2, we proposed that the fit between the expected and the experienced work-life balance while working in a technology is positively related to career satisfaction. to test the hypothesis, we followed the above method and found the adjusted r square value to be 0.13 (see table 2). fit on career satisfaction career growth work-life balance constant 3.76** 3.91** age 0.00 0.00 gender 0.10* 0.06* marital status 0.36 0.30* previous experience 0.10 0.10 promotability 0.17* 0.19 negative affectivity -0.22** -0.40** p -0.24* -0.28** t 0.43** 0.17** p2 -0.03 0.09 p x t -0.09** -0.04** t2 0.05* -0.13** f statistics 5.56* 6.35* adjusted r2 0.27 0.13 r2 0.29 0.16 note: n= 284; *p<0.05; **p <0.01; f statistics represents the score for the higher order terms; b0, b1, b2, b3, b4, and b5 represent the regression coefficient table 2: results of polynomial regression analysis australasian journal of information systems tomer & mishra 2019, vol 23, research on the role of tech in workforce mgt expectation from tech & career satisfaction 10 according to edward and parry (1993), when the joint coefficient of the three higher order terms (p2, p x t, and t2) is significant, there is a significant effect of fit on the outcome variables. we tested both the models (career growth and work-life balance) and found the f-statistic values to be significant, thus, supporting hypotheses 1 and 2. the details are provided in table 2. edwards and parry (1993) further illustrated the use of response surface methods to test hypotheses along with polynomial regression analysis. using response surface method threedimensional graphs were generated which depicted the position of all constructs in a threedimension frame. the rsm suggests some additional tests (using specific co-ordinates) to test the hypotheses. figure 1 and figure 2 illustrates the output of response surface analysis conducted on predictor and criterion variable. the 3-dimensional plots are generated by plotting the regression coefficient for each regression equation. for hypothesis testing, we focused on three key features of the graph. 1. stationary point (the point at which slope of the surface is zero in all directions) 2. principal axes (they run perpendicular to each other and intersect at the stationary point) 3. slope of the surface along various lines of interest (principal axes, lines where principal components are equal {y=x}. description of relevant tests for our study is discussed in the following text. test 1: one of the significant coordinates in three-dimensional graphs is the stationary point (x0, y0) where the slope of the surface is zero in all directions. the equation for the stationary points can be represented as, x0 = (b2b4) – (2 b1b5) / (4 b3b5) – b42 y0 = (b1b4) – (2 b2b3) / (4 b3b5) – b42 the variables (b1, b2… b5) are used to generate equations for the first and second principal axis. these values are derived from the result of polynomial regression analysis and represent the regression coefficient for p, t, p×t, p2, t2. equations can be written as, y = p10 + p11 x0 ............................................................................................................................... (2) where, p11 = (b5 – b3 + sqrt ((b3 – b5)2 + b42)) / b4 and p10 = y0 – p11 x0 as adapted from khuri and cornell (1987) y = p20 + p21 x0 ............................................................................................................................... (3) where, p21 = (b5 – b3 – sqrt ((b3 – b5)2 + b42)) / b4 and p20 = y0 – p21 x0 we analysed slope of three-dimensional graphs at various coordinate to test our hypotheses. hypotheses 1 and 2 proposed that fit impacts career satisfaction. this implies that slope of the graph will be positive when there is a congruence between individual expectations and experiences from a given technology. table 3 displays results of hypothesis testing using the response surface model. hypotheses 1 and hypothesis 2 will be supported if the value of p21, which represents the slope of the response surface, is significant (refer equation 3) and the value of p21 is not significantly different from 1 (test 1). in addition, the value of p11 (refer equation 2); which represents the slope of the response surface curve, is not significantly australasian journal of information systems tomer & mishra 2019, vol 23, research on the role of tech in workforce mgt expectation from tech & career satisfaction 11 different from 1 (test 2). the results in table 3 provided additional support to hypothesis 1 and hypothesis 2. hypotheses stationary point x0 y0 p21 p11 test results (p) hypothesis supported hypotheses 1 2.50 9.50 0.72 1.12 0.09* (test 1) 0.11* (test 2) yes hypotheses 2 -0.25 -1.30 0.79 1.05 0.01** (test 1) 0.10* (test 2) yes response surface feature (x=-y, misfit line) hypotheses slope (b1-b2) curvature (b3-b4+b5) hypotheses 3(a) -0.19* -0.18* hypothesis 3(b) -0.45* 0.18* note: *p <0.05; **p <0.01 table 3: results of response surface analysis examining career satisfaction to test our hypothesis 3 (a, b), we were particularly interested in effects on career satisfaction when it professionals exhibit a higher degree of expectations than the experienced aspects of technology. thus, hypothesis 3 particularly involve the movement along the misfit or incongruence line (x=-y). we examined both slope and curvature of the graph along the incongruence line and found both to be significant. this implies that as we move along the misfit line higher the expectations regarding (a) tcg and (b) twlb as compared to the experienced tcg and twlb, lesser will be the career satisfaction. hypotheses 3 (a, b) will be supported if slope (b1-b2) and the curvature (b3-b4+b5) are significant (see table 3) for both tcg and twlb. we found support for both the hypotheses (table 3 and figure 1 & 2). figure 1: response surface graph predicting career satisfaction against tcg figure 2: response surface graph predicting career satisfaction against twlb 1 3.4 5.8 0 2 4 6 8 1 2 .2 3 .4 4 .6 5 .8 7 y z x 1 3.4 5.8 -2 0 2 4 6 8 10 1 2 .2 3 .4 4 .6 5 .8 7 y z x australasian journal of information systems tomer & mishra 2019, vol 23, research on the role of tech in workforce mgt expectation from tech & career satisfaction 12 6 discussion our findings highlighted the need to investigate relevant ‘environmental’ aspects to understand work outcomes of it professionals. in the present study, we examined ‘technology’ as a critical ‘environmental’ factor to explain career satisfaction of it professionals. the is literature is rich in studies exploring different aspects of technology. while most of the studies focus on usage and adoption of technology, studies exploring the interpretation of technology from the career perspective of it professionals are at best limited. in the present study, we have explored two distinct technology-related aspects, i.e., technology led career growth and technology led work-life balance. summarizing the findings of the study, we propose that career satisfaction of an it professional is driven by the fit between his/her expectations and experiences while working in a technology. our findings indicate that individuals who perceive that they are working in a favourable technology are more prone to experience greater career satisfaction. in addition, the support for our divergence hypotheses (hypotheses 3a and 3b) indicated that the misfit between expectation and experience while working in a technology lead to reduced career satisfaction. our study further highlighted that it is not only about the misfit but the nature of the misfit that affected career satisfaction of it professionals. for instance, our results indicated that when expectation (on both career growth and work-life balance) is more than the experienced outcomes, compared to the other way (when experienced outcomes are more than the expectations) the impact on career satisfaction is high. 6.1 implications for theory there is a substantial body of knowledge in the is literature that has demonstrated that technology is critical towards the growth and profitability of organizations (mithas & rust, 2016; sabherwal & jeyaraj, 2015). in addition, technology has been accepted as an enabler of the digital transformation of business markets (hess et al., 2016; majchrzak et al., 2016). our study contributes to this body of knowledge in a meaningful way by exploring aspects of technology as important predictors of career satisfaction among the it professionals. our study builds and extends the existing theory in the individual-technology interface. recognizing the contribution of prior studies such as affordance theory (leonardi, 2011; markus & silver, 2008), expectation disconfirmation theory (edt) and p-o fit/p-j fit theory (venkatesh & goyal, 2010; venkatesh et al., 2017) the present study proposed a set of unique technology led aspects which are relevant for the it professionals, especially their career satisfaction. our findings extend the p-e fit literature by reconceptualizing technology as the “environment” component of the framework to explain career satisfaction of it professionals. considering centrality of technology in the career of it professionals, we chose two important aspects, namely career growth and work-life balance aspects of technology in the present study. the findings of our study depicted that the match between their expectations (related to career growth and work-life balance) from the technology the it professionals are working in and their experiences with those aspects of technology explained significant variance in their career satisfaction. our study contributes to this stream of literature by (1) extending the p-e fit framework in a unique context and (2) examining that individual expectation related to technology plays a major role in determining significant work-related outcomes such as career satisfaction. australasian journal of information systems tomer & mishra 2019, vol 23, research on the role of tech in workforce mgt expectation from tech & career satisfaction 13 6.2 implications for practitioners the study has significant implications for practitioners, as supervisors are facing serious challenges in managing it professionals. with increasing instances of shorter employment relationships, the management of careers has become more self-directed (sullivan & baruch, 2009) more so for the it professionals (joseph et al., 2012). career satisfaction is important as it influences significant outcomes such as productivity, commitment and job satisfaction (maccrory et al., 2016). there is a need for the organizations to “develop more sophisticated strategies to acquire and retain top talents for organizational success” (huang & zhang, 2016, p. 803). given that it professionals experience higher levels of stress (moore, 2000) and stress related outcomes, our finding provides a novel approach to look at these issues. our study demonstrated that the ‘technology’, the it professionals work in, is central to their work outcomes. technology being a significant aspect of their work life, the match/ mismatch between expectations from the technology and experience while working in that technology play an important role in explaining career related outcomes such as career satisfaction. scholars agree that “it professionals derive motivation from what they do” (ahmed et al., 2017, p. 04). hence, organizations need to explore ways to factor in the expectations of professionals while allocating technology to them. our study demonstrated that by devising mechanisms which incorporate it professionals’ technology-related expectations, in project/ technology allocation may be effective in enhancing career satisfaction of these professionals. 6.3 limitations and future research we believe that adopting subjective measures in the study to capture person-environment dimensions have generated findings which are based on individual perceptions and beliefs. objective measures tend to report p-e fit which is believed to be free from human bias. the objectivity can be maintained by employing multi-source data reporting. however, by synthesizing literature from both cognitive and behavioural perspectives, we noticed that subjective measures may be more relevant and pertinent than objective measures (edwards et al., 2006; venkatesh et al., 2017). further, subjective measures are more proximal indicators of individual perception and attitudes than are the objective measures (edwards et al., 2006). for these reasons, we found subjective measures to be more appropriate for the present study. in the present study, we have limited the work outcome to career satisfaction. future studies may explore the expectation-experience match/mismatch from the technology one is working in to explain other important work outcomes such as stress and turnover. in addition, our study is limited to indian cultural context; similar studies may be conducted in different cultural settings to understand the cultural variance of the expectation-experience (mis-)match in predicting work outcomes. 7 conclusion technology is core to the it professionals’ work-life. hence, it is important to understand, the expectations of these professionals from the technology they are working in. surprisingly the research in this domain is scarce. the present study explained two important expectations from the technology (namely career growth and work-life balance) and demonstrated their impact on career satisfaction. based on a rigorous study and using a novel methodology, the study highlighted the need to refocus the attention on expectations and experiences of it professionals’ while working in a technology to explain their work outcomes. australasian journal of information systems tomer & mishra 2019, vol 23, research on the role of tech in workforce mgt expectation from tech & career satisfaction 14 references ahmed, s., taskin, n., pauleen, d. j., & parker, j. (2017). motivating information technology professionals: the case of new zealand. australasian journal of information systems, 21, 130. armstrong, d. j., brooks, n. g., & riemenschneider, c. k. (2015). exhaustion from information system career experience: implications for turn-away intention. mis quarterly, 39(3), 713-727. atapattu, m., & sedera, d. (2014). agility in consumer retail: sense-response alignment through the eyes of customers. australasian journal of information systems, 18(2), 111-132. ayyagari, r., grover, v., & purvis, r. l. (2011). technostress: technological antecedents and implications. mis quarterly, 35(4), 831-858. brown, s. a., venkatesh, v., & goyal, s. (2012). expectation confirmation in technology use. information systems research, 23(2), 474-487. chen, z., powell, g. n., & greenhaus, j. h. (2009). work-to-family conflict, positive spillover, and boundary management: a person-environment fit approach. journal of vocational behavior, 74(1), 82-93. chilton, m. a., hardgrave, b. c., & armstrong, d. j. (2005). person-job cognitive style fit for software developers: the effect on strain and performance. journal of management information systems, 22(2), 193-226. dawis, r. v., & lofquist, l. h. (1984). psychological theory of work adjustment: an individualdifferences model and its applications. minneapolis: university of minnesota press. edwards, j. r., cable, d. m., williamson, i. o., lambert, l. s., & shipp, a. j. (2006). the phenomenology of fit: linking the person and environment to the subjective experience of person-environment fit. journal of applied psychology, 91(4), 802-827. edwards, j. r., & parry, m. e. (1993). on the use of polynomial regression equations as an alternative to difference scores in organizational research. academy of management journal, 36(6), 1577-1613. fornell, c., & larcker, d. f. (1981). evaluating structural equation models with unobservable variables and measurement error. journal of marketing research, 18(1), 39-50. fu, j-r., & chen, j. h. f. (2015). career commitment of information technology professionals: the investment model perspective. information & management, 52, 537-549. furnham, a., & schaeffer, r. (2011). person‐environment fit, job satisfaction and mental health. journal of occupational and organizational psychology, 57(4), 295-307. ginzberg, m. j., & baroudi j. j. (1988). mis careers: a theoretical perspective. communications of the acm, 31(5), 586-594. greenblatt, e. (2002). work/life balance: wisdom or whining. organizational dynamics, 31(2), 177-193. greenhaus, j. h., parasuraman, s., & wormley, w. m. (1990). effects of race on organizational experiences, job performance evaluations, and career outcomes. academy of management journal, 33(1), 64-86. australasian journal of information systems tomer & mishra 2019, vol 23, research on the role of tech in workforce mgt expectation from tech & career satisfaction 15 gupta, y. p., guimaraes, t., & raghunathan, t. s. (1992). attitude and intentions of information center personnel. information and management, 22(3), 151-160. han, g. (2010). trust and career satisfaction: the role of lmx. career development international, 15(5), 437-458. hess, t., matt, c., benlian, a., & wiesböck, f. (2016). options for formulating a digital transformation strategy. mis quarterly executive, 15(2), 123-139. hill, e. j., hawkins, a. j., ferris, m., & weitzman, m. (2001). finding an extra day a week: the positive influence of perceived job flexibility on work and family life balance. family relations: interdisciplinary journal of applied family science, 50(1), 49-58. hsu, m. k., jiang, j. j., klein, g., & tang, z. (2003). perceived career incentives and intent to leave. information and management, 40(5), 361-369. huang, p., & zhang, z. (2016). participation in open knowledge communities and jobhopping: evidence from enterprise software. mis quarterly, 40(3), 785-806. igbaria, m., greenhaus, j. h., & parasuraman, s. (1991). career orientations of mis employees: an empirical analysis. mis quarterly, 15(2), 151-169. jans, n. a. (1989). organizational commitment, career factors and career/life stage. journal of organizational behavior, 10, 247−266. jawahar, i. m., & liu, y. (2016). proactive personality and citizenship performance: the mediating role of career satisfaction and the moderating role of political skill. career development international, 21(4), 378-401. jiang j., & klein, g. (1999). supervisor support and career anchor impact on the career satisfaction of the entry-level information systems professional. journal of management information systems, 16(3), 219–240. jiang, j. j., klein, g., & saunders, c. (2012). discrepancy theory models of satisfaction in is research. in y. l. dwivedi, m. r. wade, & s. l. schneberger (eds.), information systems theory: explaining and predicting our digital society (pp. 355-381). new york: springer publishing. joseph, d., boh, w. f., ang, s., & slaughter, s. a. (2012). the career paths less (or more) travelled: a sequence analysis of it career histories, mobility patterns, and career success. mis quarterly, 36(2), 427-452. khuri, a. l. & cornell, j. a. (1987). response surfaces: designs and analyses. new york: marcel dekker inc. kopelman, r. e., greenhaus, j. h., & connolly, t. f. (1983). a model of work, family, and interrole conflict: a construct validation study. organizational behavior and human performance, 32(2), 198-215. kristof‐brown, a. l., zimmerman, r. d., & johnson, e. c. (2005). consequences of individuals’ fit at work: a meta-analysis of person-job, person-organization, person-group and person-supervisor fit. personnel psychology, 58(2), 281-342. lee, p. c. b. (2011). computer professionals and their perceived workload. australasian journal of information systems, 17(1), 5-22. australasian journal of information systems tomer & mishra 2019, vol 23, research on the role of tech in workforce mgt expectation from tech & career satisfaction 16 leonardi, p. m. (2011). when flexible routines meet flexible technologies: affordance, constraint, and the imbrication of human and material agencies. mis quarterly, 35(1), 147-167. locke, e.a. (1976). the nature and causes of job satisfaction. in m. d. dunnette (ed.), handbook of industrial and organizational psychology (pp.1297-1349). chicago: rand mcnally college publishing. lounsbury, j. w., moffitt, l., gibson, l. w., drost, a. w., & stevens, m. (2007). an investigation of personality traits in relation to job and career satisfaction of information technology professionals. journal of information technology, 22, 174–183. lounsbury, j. w., park, s. h., sundstrom, e., williamson, j., & pemberton, a. (2004). personality, career satisfaction, and life satisfaction: test of a directional model. journal of career assessment, 12, 395–406. maccrory, f., choudhary, v., & pinsonneault, a. (2016). designing promotion ladders to mitigate turnover of it professionals. information systems research, 27(3), 648-660. majchrzak, a., markus, m. l., & wareham, j. (2016). designing for digital transformation: lessons for information systems research from the study of ict and societal challenges. mis quarterly, 40(2), 267-277. mithas, s., & rust, r. t. (2016). how information technology strategy and investments influence firm performance: conjectures and empirical evidence. mis quarterly, 40(1), 223-245. moore, j. e. (2000). one road to turnover: an examination of work exhaustion in technology professionals. mis quarterly, 24(1), 141-168. moreno, a. m., sanchez-segura, m. i., medina-dominguez, f., & carvajal, l. (2012). balancing software engineering education and industrial needs. journal of systems and software, 85(7), 1607-1620. ng, t.w.h., eby, l.t., sorensen, k. l., & feldman, d. c. (2005). predictors of objective and subjective career success: a meta-analysis. personnel psychology, 58(2), 367-408. ngo, h-y., & hui, l. (2018). individual orientations and career satisfaction: the mediating roles of work engagement and self-efficacy. journal of career development, 45(5), 425-439. porter, l.w., & steers, r.m. (1973). organizational, work, and personal factors in employee turnover and absenteeism. psychological bulletin, 80(2), 151-176.rambur, b., mcintosh, b., palumbo, m.v., & reinier, k. (2005). education as a determinant of career retention and job satisfaction among registered nurses. journal of nursing scholarship, 37, 185–192. rehfuss, m.c., gambrell, c.e., & meyer, d. (2012). counselors’ perceived person-environment fit and career satisfaction. career development quarterly, 60(2), 145-151. sabherwal, r., & jeyaraj, a. (2015). information technology impacts on firm performance: an extension of kohli and devaraj (2003). mis quarterly, 39(4), 809-836. sarker, s., sarker, s., & jana, d. (2010). the impact of the nature of globally distributed work arrangement on work-life conflict and valence: the indian gsd professionals’ perspective. european journal of information systems, 19(2), 209-222. australasian journal of information systems tomer & mishra 2019, vol 23, research on the role of tech in workforce mgt expectation from tech & career satisfaction 17 shockley, k.m., & allen, t.d. (2018). it’s not what i expected: the association between dualearner couples' met expectations for the division of paid and family labor and wellbeing. journal of vocational behavior, 104, 240-260. speier, c., & venkatesh, v. (2002). the hidden minefields in the adoption of sales force automation technologies. journal of marketing, 66(3), 98-111. sullivan, s. e., & baruch, y. (2009). advances in career theory and research: a critical review and agenda for future exploration. journal of management, 35(6), 1542-1571. tarafdar m., tu, q., ragu-nathan, b. s., & ragu-nathan t. s. (2007). the impact of technostress on role stress and productivity. journal of management information systems, 24(1), 301-328. venkatesh, v., & goyal, s. (2010). expectation disconfirmation and technology adoption: polynomial modeling and response surface analysis. mis quarterly, 34(2), 281-303. venkatesh, v., windeler, j.b., bartol, k.m., & williamson, i.o. (2017). person-organization and person-job fit perceptions of new it employees: work outcomes and gender differences. management information systems quarterly, 41(2), 525-558. weng, q. x., & hu, b. (2009). the structure of career growth and its impact on employees’ turnover intention. industrial engineering and management, 14(1), 14−21. weng, q. x., mcelroy, j. c., morrow, p. c., & liu, r. (2010). the relationship between career growth and organizational commitment. journal of vocational behavior, 77(3), 391-400. copyright: © 2019 tomer & mishra. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems dang-pham, pittayachawan & nkhoma 2015, vol. 19, research article demystifying vietnamese online personas demystifying online personas of vietnamese young adults on facebook: a q-methodology approach duy dang-pham school of business it and logistics rmit university duy.dang@rmit.edu.au siddhi pittayachawan school of business it and logistics rmit university siddhi.pittayachawan@rmit.edu.au mathews nkhoma department of business it and logistics rmit international university vietnam mathews.nkhoma@rmit.edu.vn abstract as online social networking sites (sns) are becoming a part of many people’s daily life, they have become a topic of interest that attracts the attention of both academics and practitioners. the users’ behaviours on those sites are complex and have important implications. individuals can use a wide range of sns’s functionalities, for example, to socialise and promote their online identities or find friends of common interests. reckless online behaviours on sns have been shown to result in breaches of privacy and other risks such as cyber-bullying, cyber-stalking, and job loss. as a result, there are many concerns over these problems that prevent sns from reaching their full potential. given the vast number of user’s concerns, identified by prior studies, there is a need to explore a typology of users that classifies online personas. this typology would reveal more about the users’ needs and help improve their experience on sns. to achieve that goal, we collected data from fourteen vietnamese young adults of different backgrounds who use facebook, which was then analysed with q factor analysis. the findings suggested three distinctive online personas that include the outgoing connector, the reserved trustor, and the threats perceiver. implications for marketing tactics and design of better users’ experiences on social networking sites were discussed, and the proposed typology could serve as a starting point for future research to further explore the different types of social media’s users. keywords: facebook; social networking site; q methodology; online identity; online privacy; information behaviour 1 introduction 1.1 background social networking sites (sns) have been receiving tremendous attention not only from the end users but also practitioners and researchers who are interested in these virtual, multifaceted domains. by gaining more knowledge of the users’ behaviours and preferences on sns, better services and protection could be developed to cater their needs and enhance the users’ online experiences. furthermore, research about social media uses also brings implications to domains such as online education and business (a. carrillat et al. 2014; boyd and ellison 2007). with the large amount of research investigating users’ concerns about the different aspects of sns, recent literature reviews that summarised these findings were able to explain why people use social media (e.g. nadkarni and hofmann 2012). nevertheless, there remain opportunities to bring out more value from the existing research by studying them as a whole, especially when they have been examined separately as an attempt to gain an in-depth 1 australasian journal of information systems dang-pham, pittayachawan & nkhoma 2015, vol. 19, research article demystifying vietnamese online personas understanding about their respective topics. as a consequence, we are attempting to construct a typology of users based on their concerns that have been identified by prior studies. more specifically, we anticipate elaborating the relations between the concerns about different aspects and hence explore the unique personalities of the sns users. potential findings derived from the typology would add insights to the current discussion by explaining why different types of users are using social media in their own ways. there are few studies that investigated typologies of online citizens. for instance, meyen et al. (2010) grouped internet users based on their everyday life’s activities and types of capital (i.e. cultural and social) with qualitative analysis. brandtzaeg and heim (2011) used cluster analysis to classify users based on the intensity of their participation on sns (i.e. high or low) and objectives of participation (i.e. informational or recreational). their study collected data from 5,233 users on four popular sns in norway, and found five types of users that fall into the four quadrants as determined by the two mentioned criteria. most recently, hassouneh and brengman (2014) took a mixed methods approach to explore traits of social virtual world users with exploratory factor analysis. our research is different from the mentioned typological studies in that we focus on the concerns of the end users about facebook as a popular sns, rather than the broader theme of internet behaviours or more specific activities such as a virtual social game. furthermore, the anticipated typology of personalities will be developed based on multiple concerns in relation to the aspects identified by prior studies such as identities, information sharing, online privacy, and so on. this scope allows the extracted personalities to provide more complete portraits of groups of users, rather than focusing solely on their motivations to use snss or intensity. 1.2 research context of our particular interest is the sns facebook which has been growing rapidly since its first appearance in 2004. to date, it has reached more than 890 million daily active users from around the world (facebook 2015). this sns offers a wide range of functionalities that allow the users to customise their personal pages and profiles, as well as communicate with the others by private messaging or posting contents (e.g. updates, news, websites, music and video) on their pages. facebook also hosts community’s pages and online games where users sharing common interests can participate in and socialise. according to the 2014 global survey by pew research center (2014), 87 per cent of young-adult internet users whose ages range from 18 to 29 years old use facebook, and it was found to attract the highest number of online adults when compared to other sns. as facebook’s high popularity attracts end users of varied demographics, we decided to limit our investigation to the personalities of young adults who are the dominant cohort of users. our research context further focuses on vietnamese young adult users of facebook, whose concerns and use of social media are influenced by their unique cultural values as a part of the larger asian context. similar to other societies, facebook is also popular among vietnamese social media users. according to the latest report by wearesocial (2015) about internet and social media usage around the world in january 2015, vietnam had a 40 per cent growth of social media users (accounted for 31 per cent of total population) since 2014 when there were 20 million active facebook accounts (wearesocial 2014). 2 australasian journal of information systems dang-pham, pittayachawan & nkhoma 2015, vol. 19, research article demystifying vietnamese online personas figure 1: facebook usage compared to other sns by vietnamese (left) and global users (right) – adapted from pew research center (2014) and wearesocial (2015) this present research is a part of a larger project which aims at capturing and comparing vietnamese users’ concerns on facebook and zing me, a local sns (www.zing.me) which has functionalities that resemble facebook and was its main competitor in 2012 when both had 8.5 and 8.4 million users respectively (wearesocial 2012). in this study, we focus on analysing and classifying the users’ concerns on facebook by performing q factor analysis, so to explore the distinctive facebook personas of vietnamese young adult users. we start with consolidating concerns of sns users by conducting a literature review in the next section, to assist us in developing an instrument to collect data. the research methodology and analyses are then presented, followed by research findings and our discussion. finally, we conclude our study with implications and limitations. 2 literature review 2.1 concerns about socialisation and functionalities one of the fundamental and critical aspects of most sns is the user’s disclosure of information which can be observed from their personal profile or communication with others (kietzmann et al. 2011; krasnova et al. 2010). disclosure of information is fundamental as every user is required to submit their information (to some extent depending on the sns) and to register for an sns account, yet critical because the strategies for establishing and maintaining identities relate to the user’s goals and concerns about other important aspects (boyd and ellison 2007). to begin with, one important implication of those strategies is for selfpromotion or self-branding (kietzmann et al. 2011). these purposes, which are termed selfpresentation, are among the central desires that motivate users to disclose more information (boyd 2007; krasnova et al. 2010), especially to promote their attractive traits and receive social approvals (mehdizadeh 2010). in particular, self-presentation can be achieved by disclosing many types of information, including those that are identifiable (e.g. real names, genders, and profile images) and more private such as hobbies, tastes of music and movies, relationship status, or sexual preferences (gross and acquisti 2005; krasnova et al. 2010; stutzman 2006). in relation to identities, maintaining and building online relationships were regarded as the main benefits of using sns and disclosing information (krasnova et al. 2010). for example, the availability of accurate information such as names and pictures can allow other users to assess and validate the authenticity of one’s account (boyd and ellison 2007). likewise, ellison et al. (2011) differentiate between making friends indiscriminately with those who share similar sns contexts (e.g. playing the same games) and “social browsing”, which refers to selectively finding more information about contacts that have similar offline backgrounds. in 3 australasian journal of information systems dang-pham, pittayachawan & nkhoma 2015, vol. 19, research article demystifying vietnamese online personas addition, users can also search for information about products and organisations, provided that online stores and organisations are leveraging social media to reach more potential customers (kietzmann et al. 2011). the next step after finding information about other people is establishing relationships with them. the studies of lampe et al. (2006) and joinson (2008) reveal that re-acquiring lost contacts or keeping in touch with close friends ranked highly in the list of purposes of using facebook, followed by making new friends and having in-person encounter with people met online. interestingly, kietzmann et al. (2011) discussed connections with like-minded people and even potential dates result from conversations between individuals and groups on sns. furthermore, some users view sns as a mean to acquire and accumulate social capital, especially by creating ties with new contacts that can provide them unique information (krasnova et al. 2010). similarly, users may expand their businesses and professional networks on sns, provided that sns have been recently adopted as a recruitment channel by organisations (a. carrillat et al. 2014). sns offer a range of interactional functionalities that facilitate establishing identities and building relationships. blogging is a type of interaction that conveys one’s message to the audience, while online gaming and social groups or pages provide a common ground for finding people with similar interests (ellison et al., 2011; joinson, 2008; kietzmann et al., 2011). sharing content such as music, video, and photos is another way to make connections among individuals (joinson 2008; kietzmann et al. 2011). however, kietzmann et al. (2011) discussed that whether sharing would result in a relationship between individuals depends on the shared objects fit with the sns’s functional objective (e.g. pictures for flickr and careers for linkedin). finally, tan et al. (2012) found the sns’s perceived ease of use and usefulness motivates users to continue using it, which indicates the importance of the functions’ usability. 2.2 concerns about threats of privacy privacy threats have been a primary interest of scholarship concerning sns since 2004 (boyd and ellison 2007). one of the earliest and widely cited articles about privacy on facebook found the surveyed users reveal a generous amount of personal information while appearing unconcerned about their privacy (gross and acquisti 2005). furthermore, krasnova et al. (2010) discussed that the privacy risks on sns include not only those mentioned in ecommerce’s literature such as having information collected by third-party companies but also secret sharing, identity theft, and misuse of personal information (e.g. for phishing purposes). in addition, krasnova et al. (2009) found the users were concerned about their personal data being collected and used by the sns providers. similarly, other researchers studying users’ privacy on snss such as stutzman et al. (2011) and kietzmann et al. (2011) also discussed those threats in their studies. however, debatin et al. (2009) suggested not every user would recognise these threats as they argued that organisations’ violations of personal data such as data mining remain invisible to the average users. apart from the aforementioned privacy risks that involve losing control over personal data, there are several other threats that emerge from the online environment. gross and acquisti (2005), debatin et al. (2009), and krasnova et al. (2009) discussed social threats such as stalking (offline and online), bullying, and uncontrollable actions of other users such as inappropriate comments, harassment, or denigration. moreover, debatin et al. (2009) found rumours and gossips on facebook play central roles by keeping the sns interesting while causing social violations of privacy. stories shared among the participants in these authors’ focus group further illustrate privacy incidents in which insults were exchanged via posting of embarrassing comments and abusive tagging as well as modifying personal pictures. nevertheless, krasnova et al. (2009) found these social threats do not affect information disclosure, and they argued that such insignificant impact could be due to privacy controls perceived to be effective against social threats. moreover, there has been an emerging number of studies investigates the impact of sns’s contents on one’s career. for example, the study by karl et al. (2010) revealed that the surveyed participants would be inclined to post “problematic content” (e.g. self-photo drinking alcohol 4 australasian journal of information systems dang-pham, pittayachawan & nkhoma 2015, vol. 19, research article demystifying vietnamese online personas or being provocative) and private information on their profile such as religious beliefs and political affiliations. similarly, stoughton et al. (2013) studied posting of bad-mouthing behaviour and substance use in their research, and they shared the common view with karl et al. (2010) that employers have been paying extra attention to inappropriate content for screening potential candidates and monitoring “off-the-clock” behaviour of their employees. similarly, users often weigh the benefits against the costs of disclosing information or making connections before initiating any social interaction, and online trust has been regarded as an important factor that mitigates the mentioned privacy threats as a major cost (krasnova et al. 2010). trust is multi-dimensional and contains several meanings which include the truster’s assessment of the trustee’s competence, predictability, benevolence, and integrity (pittayachawan et al. 2008). furthermore, krasnova et al. (2010) argued that only the latter two would affect information disclosure as sns do not hold specific topics that can be used to evaluate one’s competence to converse on it. nevertheless, discussing in-depth online trust is outside the scope of this research, and this present study will only focus on the user’s trust in the snss’ protective services and functions, as well as in their online friends who would not harm them. in addition to trust is the user’s perceived control over their information, which is reflected in how much they are aware of or using the privacy settings of the sns (krasnova et al. 2010). more specifically, these authors discussed that sns providers need to have transparent policies about collecting and maintaining their users’ information, as well as to give them clear tools to manage their privacy (e.g. limit access to personal profile, remove and report inappropriate contents). as a consequence, it is expected that multiple settings and effective usability of the sns’s privacy functions would be among the users’ concerns. 3 research methodology we conducted this study using q-methodology which was developed by stephenson (1953) to explore and classify sns users into distinct groups or “online personas”. this was achieved by using a combination of enquiries including q sorts (i.e. card-sorting), semi-structured interviews, and observation. each participant was requested to sort the cards from the “most concerned” to the “most unconcerned” issues on a scale, which ranges from +6 to -6 based on the distribution illustrated in figure 2. semi-structured interview was used before card-sorting to collected data about demographic and sns experience, and after card-sorting to ask the participants why they sorted the cards in such a way, especially issues located at the extreme (i.e. +6 and -6) or middle positions (i.e. 0). the card-sorting data was then analysed with qmethod package in r (zabala 2014). the interview data was used to aid interpretation of the findings. figure 2: q distribution 5 australasian journal of information systems dang-pham, pittayachawan & nkhoma 2015, vol. 19, research article demystifying vietnamese online personas 3.1 development of the statements since none of the prior studies used q methodology, we had to adopt the idea from previous researches. according to watts and stenner (2012), a q set should contain items with a good coverage related to the research question. we developed the statements for the sorting activity based on our literature review in the previous section. specifically, we identified six major themes including relationships, sns functionalities, interactions, finding information, privacy threats, and identity. “relationships” refers to the use of facebook to connect with family and friends, or for purposes such as expanding professional network (ellison et al. 2011; joinson 2008; kietzmann et al. 2011; lampe et al. 2006). “sns functionalities” includes items such as playing games, sharing music and video, as well as the usability and protective measures of facebook (ellison et al. 2011; joinson 2008; tan et al. 2012). “interactions” lists out activities such as communications, sharing pictures, writing blog, and building online trust (boyd 2007; ellison et al. 2011; krasnova et al. 2010). “finding information” refers to the need to search for information about people, events, products, or to read online news (kietzmann et al. 2011). “privacy threats” covers threats such as insulting, online scams, identity theft, cyberstalking, and posting sensitive contents (boyd and ellison 2007; debatin et al. 2009; gross and acquisti 2005; stutzman et al. 2011). “identity” describes the concerns over putting which types of information on the user’s profile, as well as the need for self-presentation (boyd 2007; kietzmann et al. 2011; mehdizadeh 2010). we composed the q set’s statements that reflect the content of the identified themes. the comprehensiveness of the statements, as well as their relation to the existing functionalities of facebook, was carefully considered during the development process. the final list of 50 statements can be found in the appendix. 3.2 profiles of the participants to recruit the participants, public advertisement that details the project’s objectives and procedures has been posted on facebook and online forums for one month. the participants were invited to attend the individual interview and q-sort session at rmit vietnam university’s campus in ho chi minh city. initial rejections to participate suggested that the card sorting procedure was perceived to be confusing as compared to the traditional survey method and traveling to the campus was inconvenient. consequently, small gifts were used as incentives to encourage participation. this research is a part of a larger project which aims at capturing and comparing users’ concerns on facebook and zing me, which is a popular vietnamese sns that resembles facebook, and this paper only discusses the concerns of facebook’s users. as a result, one of the key selection criteria includes the participants must have used both facebook and zing me for at least one year. among the 24 participants who met our criteria and were set up appointments to participate in the project, only 14 attended to the actual sessions. the demographics of the 14 participants are displayed below, with the average age of 23.07 years old and an even ratio of genders. id age gender occupation id age gender occupation 1 20 female he student 2 25 male it consultant 5 21 female he student 3 25 male pharmacist 7 21 female he student 4 22 male it intern 9 23 female vet intern 6 25 male marketer 10 21 female he student 8 28 male marketer 11 21 female he student 13 21 male he student 12 27 female hr executive 14 23 male accountant table 1: demographics of participants 3.3 data analysis we used principal component analysis with spearman’s correlation and cluster rotation to identify three factors in r with the qmethod package. the total explained variance of the three 6 australasian journal of information systems dang-pham, pittayachawan & nkhoma 2015, vol. 19, research article demystifying vietnamese online personas factors is 49% (table 2). one of the participants was not loaded on any factor (i.e. a confounded case), thereby being dropped from subsequent analysis. factors 1 and 2 are clearly loaded by the majority of the interviewees. all factors are highly reliable (>0.80). participants factor 1 factor 2 factor 3 14f_xdai 0.792 -0.141 -0.053 9f_danoi 0.741 0.168 0.087 6f_dlong 0.591 0.299 0.08 8f_khang 0.584 0.146 0.114 4f_longt 0.487 -0.185 0.317 11f_nga 0.085 0.783 -0.154 5f_diem -0.166 0.650 0.371 7f_linda 0.283 0.574 -0.305 13f_qhai -0.134 0.549 0.059 10f_life 0.243 0.483 0.038 12f_uyen 0.145 -0.099 0.768 1f_trang -0.098 0.285 0.636 2f_thinh 0.453 -0.178 0.555 n 5 5 3 eigenvalue 2.5 2.2 1.7 expl var (%) 19 17 13 reliability 0.95 0.95 0.92 table 2: factor loadings table 3 summarises correlations (in normal fonts) and squared correlations (in italic fonts) between factor z-scores. it is evident that a degree of similarity between factor 1 and factor 2 is 23% and that a degree of similarity between factor 1 and factor 3 is 21%. however, surprisingly, factors 2 and 3 are not similar (10%). factor 1 z-score factor 2 z-score factor 3 z-score factor 1 z-score 1 0.23 0.21 factor 2 z-score 0.48 1 0.10 factor 3 z-score 0.46 0.32 1 table 3: correlation between factor z-scores 4 research findings––the three personas 4.1 persona 1: the outgoing connector persona 1 consists of five young adults who have been working in different industries, including two marketers, one accountant, one it intern and one vet intern. four of them are males, and the average age of this persona is 24.2. this persona has an eigenvalue of 2.5 and explained variable of 19 per cent. these users in persona 1 stand out from the rest with their high concerns about establishing relationships with other people, especially those they met offline (7: +4; 11: +2). this, of course, includes connecting with family members and close/old friends (1: +6; 9: +5). they appear to like making new friends online (10: +1), but the statement was ranked at +1 which may indicate low interest. finding dates (2: -2) was also seen not important, yet interestingly they ranked higher on it than the other groups. the statement about “connecting selectively” is ranked significantly lower which further justifies the outgoing personality of these participants. 7 australasian journal of information systems dang-pham, pittayachawan & nkhoma 2015, vol. 19, research article demystifying vietnamese online personas nevertheless, a careful observation could tell that these people feel more concerned about connecting with known acquaintances than strangers. given such an outgoing personality, it is consistent to find that they are concerned about sns having a large number of users (40: +5) and finding more information about people in the real world (46: +4; 48: +3). this would support their strong need for establishing relationships. other than that, they see information about products, companies, celebrities, and events as unimportant (4: -4; 43: 0; 47: -4). they are also interested in reading news (49: +3) and sharing resources, especially sharing music and video (5: +4). one participant said that he enjoys sharing clips about outing events with his friends on facebook. overall, they do not seem to care very much about facebook’s usability (45: +1) as they agreed that the interface is simple and can be figured out easily. these users do not seem to be interested in facebook’s conversational functionality, perhaps because they are more attracted to establishing online relationships with people they know in real life. this is reflected by the low concern about the need to communicate with public audiences (29: -2). the only statement being ranked higher than the other groups is “joining sns pages of common interests” (41: +2), but it was explained by one participant that he joined groups for gaining knowledge of specialised topics rather than socialising. in this sense, joining pages of interests is more or less similar to reading news. it is also worth noticing that receiving approvals from friends is of high concern to this persona (37: +2). one participant actually explained that the reason and motivation for blogging his opinions about a general event (15: 0) are to portrait himself to his friends as being up-to-date. these users are not concerned about trusting others online (16: -1; 22: 0), as one discussed that he would want to be constantly aware of people whether or not they are his friends. these users’ views about privacy threats are unique among the three personas (20: -4). they feel concerned more about the possibility of having their profiles checked by potential job recruiters as well as the risk of losing jobs due to complaints (38: +1; 39: 0). however, these two statements were ranked at the middle of the sort, and two participants discussed that they would limit their employers’ access to their profiles, which together suggest these statements may indicate neutrality. they feel concerned about expanding professional networks on facebook (8: +3). furthermore, they do not care about the threats of insult and racism (31: -5; 35: -6) because they can block inappropriate content, whereas insults are perceived as unimportant unless they come from their friends. regarding other threats, they generally do not care much about their information being collected by firms (42: -3; 50: -3) or misused (24: -3; 33: -1). this is mostly due to these users believing that they do not post important information in their accounts (13: -1; 19: 0); therefore, they do not have to be worried about what would happen to their information. similarly, they are also not bothered by being located offline which links to the threat of cyberstalking (30: -2; 34: -2). one participant reported not usually “checking in” at places while the other said that even if he were tagged by his friends’ check-ins, it would not be threatening as long as his home address is not revealed. again, the emphasis on the nature of the information is displayed here. these users are also not concerned about the threats of scams or trickery (32: -2; 36: 0), which two of them expressed confidence in not getting themselves caught in such schemes. such beliefs may explain why they do not see much importance in whether facebook has multiple privacy settings as well as the settings’ usability (26: +1; 44: 0). in fact, one participant shared that as long as facebook has underlying privacy protection, he would not care about how easy it is to configure the settings. this user said that he trusts sns’s protection (23: +1), but believed it would be more difficult for local users to seek support from a global company such as facebook. 4.2 persona 2: the reserved trustor there are five participants representing persona 2, among which four are female. all participants were higher education students, and were 21 years old on average. persona 2 has an eigenvalue of 2.2 and explained variance of 17 per cent. 8 australasian journal of information systems dang-pham, pittayachawan & nkhoma 2015, vol. 19, research article demystifying vietnamese online personas the users of persona 2 appear to contradict much those of persona 1 as discussed previously. to begin with, they are interested in finding information about people at school or workplace (46: +5) and events (e.g. technology expos and on-sales events), as well as products or companies (e.g. browsing online shop’s catalogue) (4: +4; 43: +2). some reported to have followed fan pages of singers but only for getting updates of the latest songs (47: 0), which explains the rank of this statement being at zero point. they also enjoy reading news from multiple sources (49: +3) like the users in persona 1. these users are concerned about keeping their networks really personal and intimate. this argument is supported by the high rankings of the statements about keeping in touch with old friends, family members, or close friends (1: +5; 9: +6). to let their friends search for them more easily and perhaps to reciprocate trust, these participants feel particularly concerned about using real information to create their facebook accounts (6: +3, 13: +2). more important, their approach to establish relationships is different to those in persona 1 in that they see connecting selectively as more important (12: +3), and they display a strong disinterest towards people whom they just met physically (7: -6; 48: 0). one participant emphasised that his friends on facebook are those to whom he had talked in real life and remembers who they are, and having met in-person is not the only criteria to make one become his friend. of course, finding dates on an sns (2: -5) is unimportant and untrustworthy to these participants. extending business networks on facebook (8: +1) is also not much concerning because they believed that it would be better to do it in-person. in terms of their interaction preferences, building online trust is ranked highest in this persona among the three (16: +2; 22: 0), which indicates the unique concerns regarding this aspect. it is indeed consistent to the overall personality of these users who feel more inclined to keep their networks personal. joining pages of common interests is also concerning to these participants (41: +2), as one of them said that she joined her university’s class facebook page to discuss assignments and share materials. on the other hand, they do not care much about writing blog as those of persona 1 do (14: -2; 15: -1). in fact, they are not concerned about gaining any approval from friends (37: -2). furthermore, the need to make their accounts appear good is ranked higher than other personas (3: 0) yet at the neutrality point of the sort. one participant discussed that he is not concerned about that because it is more important for his friends to accept him as who he is regardless of the content that makes up his profile. they view sharing music and video as important (5: +4), and more interestingly, one participant sees this activity as a mean to express subtly her feelings rather than to socialise. however, they are not interested in playing games on facebook (28: -4) because of several reasons including slow connection, and many games require the users to wait in real time for more ingame “energy” to continue playing. this suggests that they consider gaming as purely for entertainment, and socialising with a larger audience in the game environment may also not be of these users’ main interest. in overall, good usability of facebook (45: +4) is important to them, and one participant admitted that he is quite “low-tech”. scams and trickery such as fake charity events or hacking accounts and disguising as friends to ask for mobile phone’s top-up are relatively concerning to these users (32: +3). despite being ranked at zero position, these users’ concern about the threats of insulting and threatening on facebook is still higher than the other groups (31; 0). one participant mentioned that after strangers on facebook gave inappropriate comments on her post about her family issues, her family members complained to her that she should not have brought personal matters online. this left her a negative experience, and she believed such incidents would not happen again in the future. another participant in this persona admitted to having insulted people on facebook when he was in high school, and discussed that the impact of such cyber-bullying would not leave much damage. however, he admits not being able to deny that online insulting is happening. in the first case, the female student displayed confidence in not repeating the incident, but she also expressed concern about the threats should they happen again in the future. the male student in the second case stated his concern about the existence of cyberbullying, yet he rejected the severity of his own bullying in the past. both cases suggested that concerns about these threats are conditional (e.g. feel confident because having learnt the 9 australasian journal of information systems dang-pham, pittayachawan & nkhoma 2015, vol. 19, research article demystifying vietnamese online personas lesson but concerned if the insulting actually happens again; feel concerned about cyberbullying in general but unconcerned about own action). this could explain why the participants ranked the statement in the middle of the q sort, which gives a tentative and unsure answer about their concerns. the threat of cyber-stalking is ranked as not concerning (34: -1), despite three participants were fully aware of such problem. in particular, two of them reported to have friends who were cyber-stalked and had their pictures modified and publicly teased on facebook. similarly, the other user said that she felt terrified when read news about a high school student who also had pictures ridiculed online and committed suicide because of that. nevertheless, perhaps because these participants did not expect such situation would happen to them so they ranked this threat as unconcerned. in fact, one of them discussed that she accepts her pictures being publicly visible to the public, which could explain why the need for sharing personal photos to public audience was viewed as less concerning (25: -2). in contrast with participants with persona 1 that worry about having their contents seen by employers (38: -4; 39: -3), these users do not display concerns about those threats. one participant believed recruiters would not do that while the other discussed empathetic employers would not misunderstand the staff’s online actions. likewise, the threats of having personal information misused or shared by organisations (24: -3; 42: -2) are also not an issue as one user argued that her information (only contains name and school) that if leaked would not bring her any harm in any way. 4.3 persona 3: the threats perceiver persona 3 consists of three participants having different genders and occupations. more specifically, there are two females and one male. they are higher education student, hr executive and it auditor respectively. the average age of persona 3’s participants is 24 years old. the factor has an eigenvalue of 1.7 and explained variance of 13 per cent. the users of persona 3 differentiate themselves from the rest with their unique concerns about establishing relationships, sns functionalities, and privacy threats. in particular, they saw expanding professional networks (8: +4) as most important. they also share high concerns about connecting selectively with friends and family members (1: +3; 9: +6; 12: +3) as compared to the other personas. they also value connections with people whom they met inperson (7: +3) rather than online (10: -3; 29: -5). their need to connect selectively (perhaps with business contacts) is supported by the low concern about being able to connect with everyone met in real life (11). finding information (4: -3; 43: -2; 46: +1; 47: -3, 48: 0; 49: -2) is not these participants’ main purpose. one participant complained that her facebook’s feeds contains a mixture of some actual news and many stories and pictures that are not credible and inadequate (49: -2), while another said that they would rather go to official websites to search for information about events or products (4: -3; 47: -3). similarly, these users care the least about making their accounts appear good (3: -5), yet feel quite concerned about receiving approval from friends (37: +2). all of them agreed that they do not want to tailor too much or fake their profiles to make themselves more attractive, and one participant discussed that approvals are only valuable when coming from their realistic features on their profiles. they also display disinterest towards sharing sensitive information such as gender preferences or drinking habits (18: -4), which one participant explained that such information hold no value for building their images. likewise, they are also not interested in sharing or joining discussions about religions or politics (27: -2), as they explained that people are entitled to their own opinions and debating these topics may result in legal issues. if there is a need to share anything, one participant explained that he would write blogs and set it available only to him (14: +1; 15: 0). in this case, a personal blog is seen as a tool to release stress. not exposing much information on facebook is slightly concerning (19: +1). one participant especially feels concerned about relationship status because they would cause gossip. this may relate to their uniquely high concerns about the privacy threats. among these participants’ top concerns is the belief that their privacy being threatened (20: +5), followed by scams and trickery (32: +5). they also told the fear of having their children’s photos misused on facebook, 10 australasian journal of information systems dang-pham, pittayachawan & nkhoma 2015, vol. 19, research article demystifying vietnamese online personas as well as scams that attempted to collect profile’s data for marketing purposes, trick them into a pyramid scheme or request top-up for mobile credit by using their friend’s hijacked account. two participants advised their friends to contact them if received strange requests from their account. these viewpoints are related to the high concern about identity theft and cyberstalking (24: 0; 33: +2, 34: +2), which they believe the main purpose is for gathering information about their behaviours and friends circle so that the stalker may craft realistic trickery (e.g. cloning their accounts). on the other hand, they are not concerned about having locations displayed on facebook as they explained one could stay at home and check in at somewhere else, and this viewpoint is similar to those in persona 2. these participants also feel concerned about having their information collected and shared by organisations, including facebook (42: +2; 50: 0). however, as much as they value business networks on sns, they are not bothered by the risk of having profiles checked by recruiters and especially losing jobs because of inappropriate contents (38: -2; 39: -4). a participant who was an hr executive shared that she does not usually screen candidates using social media profiles but rather calls and checks their references. the other participants reported they had never complained on facebook, and if they did such a thing, their friends would come to their rescues. another also argued that the employers would care more about the staff’s abilities rather than what they post on facebook. again, this explanation is similar to one of persona 2’s users. in response to the high concerns about the privacy threats, these participants see facebook having multiple privacy settings with good usability as important (26: +4, 44: +3), yet they do not feel concerned about trusting its protection (23: 0). one of them expressed that he would generally trust facebook’s protection, but feels unsure what facebook would do to help him if his account got stolen. on the other hand, building online trust is unreliable thus unimportant to them (16: -1; 22: -1). they justified that it is important to maintain being careful with the others, especially when their friends’ accounts could be faked any time. 4.4 summary of the three personas we summed the ranks of the items grouped by their respective domains, then normalised those summations by the number of items so to create a radar chart (figure 3) that compares the three personas against each other. at first glance, persona 1 has the most concerns about relationships and sns functionalities domains, and cares the least about privacy threats. on the other hand, persona 2 contradicts the concerns of persona 1 and focuses on finding information. persona 3 appears to be a combination of personas 1 and 2, especially in terms of their concerns about relationships and sns functionalities. however, they care the most about privacy threats and the least about interactions and finding information. these brief visual analyses are consistent with the previous within-persona discussions. 11 australasian journal of information systems dang-pham, pittayachawan & nkhoma 2015, vol. 19, research article demystifying vietnamese online personas figure 3: summary of three personas first, persona 1––the outgoing connector––is interested in establishing relationships with a stronger alignment towards known acquaintances and business contacts in offline world rather than online. to achieve this goal, their connection strategy is quite unselective and reflects their highest concerns about relationships domain. they are interested much in finding information about people rather than events or products. furthermore, they also care the most among the three personas about facebook having a large number of users, which resulted in their highest concerns about sns functionalities. one explanation provided by a persona 1’s participant was that the amount of users is associated with the number of online threats. however, another reason behind this concern could be to ensure a greater chance for connecting with the people of their interest, especially in the offline world. the users of persona 1 appear to be most confident in handling the privacy threats on facebook, which resulted in their lowest concerns regarding those threats as compared to the other personas. in fact, these participants reported to know well how to limit others’ access to their profiles as well as block the undesired contents. second, persona 2––the reserved trustor––pays attention to information about not only people at school but also products, organisations, and events. consequently, this broad and intensive attention to information made them having the highest concerns regarding this domain. unlike persona 1, these participants’ connection approach is selective, and meeting physically alone does not make that person become their online friend. this subsequently explains their slight interest in establishing business network on facebook, which they argued that it should be better done in-person. they also joined facebook pages to exchange information as persona 1 does, except that those pages are university or class’s online communities that consist of known acquaintances only. they also do not care about gaining social approval as much as personas 1 and 3 do. one participant actually expected his trusted circle to accept him as who he is, and sharing music and video was discussed as the mean to express feelings by another participant. after all, building trust on facebook is highly concerning to persona 2. these female students have a higher concern about the threats of scams and trickery on facebook than the other two personas, which may reveal them as the specific threats that these students encounter. furthermore, they have a more optimistic 12 australasian journal of information systems dang-pham, pittayachawan & nkhoma 2015, vol. 19, research article demystifying vietnamese online personas expectation than persona 1 that their employers would not judge the profile’s contents. however, it is worth mentioning that these users do not have work experience like those of personas 1 and 3, which may make their expectations unrealistic. third, persona 3––the threats perceiver––appears to be a mixture of both personas. to start with, they resemble those in persona 1 that they are interested in connecting with offline people and especially for expanding professional network. however, they prefer to connect selectively like persona 2. this could explain why they do not feel concerned about making their accounts appear good, and friends’ approvals only matter if they are about the true and real information on these users’ profiles. unlike the other personas, finding information is not persona 3’s purpose in using facebook. they especially displayed concerns about the credibility and appropriateness of the information on facebook. more interestingly, persona 3 has the most concerns about privacy threats. perhaps these concerns explain their selective and reserved connection approach, unlike persona 2 which would be based on trust and intimacy. persona 3 feels bothered by the belief that their online privacy is threatened, which includes threats such as identity theft, cyber stalking, scams and trickery. in contrast to persona 1, persona 3 does not feel concerned about the threats of losing job and having recruiters check for inappropriate contents. finally, they also have higher interests in facebook having multiple privacy settings and their usability. 5 discussion 5.1 concerns about finding information finding and using information have been supported to be one of the core activities on sns (lampe et al. 2006), especially for establishing relationships (boyd and ellison 2007; ellison et al. 2011). our results are consistent with prior studies as the concerns about finding information on facebook (i.e. “social searching”) were ranked relatively high. more importantly, we detected dissimilarities in these concerns and anticipate that they may be associated with the participants’ different demographics as well as other concerns. while the working, male-dominant persona 1 was interested in finding information about anyone they met offline, especially for making new business connections, the female students in persona 2 only felt concerned about those at school and prioritised information about products, events and parties instead. in fact, this reflected the study of cha (2009) which found a positive relationship between female gender and attitude towards shopping on sns, based on their survey of 167 college students in the us whose average age was 20.71 years old. on the other hand, a possible explanation for the prior result is that the participants in persona 1 who have been working realised a greater need to expand their network, while persona 2’s students do not have much capability to reach professional contacts. all users of persona 3 questioned the credibility and appropriateness of information sources on facebook, as much as being critical about privacy threats. while they did not explain their disinterests in finding information in general, the attributes of information on facebook could determine the users’ motivation to find and use them. additionally, the lack of interest in finding information about other people could also be due to the fact that their connections had been carefully selected and established as a result of their offline investigations. 5.2 concerns about identity establishing and maintaining online identities is another major topic in sns research (boyd and ellison 2007). our findings support that the three personas expected their profiles’ information (e.g. real names and pictures) to be searched by friends (lampe et al. 2006), and some intentionally included identifiable information for that purpose while accepting the privacy risks. unlike the recent research by karl et al. (2010) which surveyed us and german students about sensitive and private contents on facebook such as gender preferences and drinking habit, the young adults in vietnamese context consistently demonstrated disinterests in posting or joining discussions about those topics on facebook. interestingly, the recent research by nadkarni and hofmann (2012) hypothesised that individualistic cultures would disclose more private information and more likely to promote controversial topics than 13 australasian journal of information systems dang-pham, pittayachawan & nkhoma 2015, vol. 19, research article demystifying vietnamese online personas collectivistic ones. as the sample of our study consists of vietnamese whose culture is collectivistic (hofstede 2001), this finding may be an indicator that supports that hypothesis. self-presentation and self-promotion (boyd and ellison 2007; kietzmann et al. 2011; krasnova et al. 2010; mehdizadeh 2010) were also examined in our study. the three personas did not feel concerned about making their accounts appear good, whereas receiving approvals from the others was more concerning. most participants understood the prior statement as intentionally using enhanced photos and desirable information to the extent of presenting fake information, and they opposed against such activities. on the other hand, participants of personas 1 and 3 admitted to feel concerned about receiving friends’ approvals, whereas persona 2 saw it as not concerning. this suggests self-presentation may not only be about the different needs to display desirable traits, but also depend on the existing relationships which subsequently characterise the target audiences. furthermore, discussed findings about a persona 1’s participant who sought knowledge and portrayed himself as being up-to-date by blogging his opinions imply that certain types of self-presentation are more accepted than the others. 5.3 concerns about interactions and facebook’s functionalities interactions on facebook are reflected in activities such as broadcasting messages via writing blogs or participating in common environments such as online games or social pages (ellison et al. 2011; joinson, 2008; kietzmann et al. 2011). on the other hand, facebook facilitated these activities with its functionalities, thus we discuss the concerns of these two domains together in this section. joining groups on facebook was categorised in the theme of “making new contacts” by joinson (2008), and it was discussed to find people with common interests by ellison et al. (2011). however, participants in both personas 1 and 2 did not mention about socialising on those pages but rather joined for exchanging information and acquiring knowledge. this is supported by the low concern about wanting to communicate with the public audience displayed among the three personas. furthermore, combined with the low concerns about finding dates, these findings suggest that facebook may not be suitable for making new contacts. as discussed previously, online trust was not highly concerned by most participants, except persona 2. even so, all three personas held a common belief that they could not simply trust online profiles easily, which also includes those that belong to their friends. a possible explanation is that for persona 1, their trusting stance is high in nature since they are outgoing and like to make new connections online, thereby not being concerned over a lack of trust; while for persona 2, they aim to make connections with either professionals or those met in real life, which also makes them feel less worried about untrustworthy people. as a consequence, this questions the socialising purpose of gaming and joining pages of common interests on facebook. likewise, a participant of persona 3 explained writing blogs as a tool to relieve stress, and the blog’s availability was set to private and available only to the author. on the other hand, persona 2 provided an example where a participant subtly expressed her moods and feelings by sharing songs and video. in these cases, writing blogs and sharing music and video appear to have more purposes than simply broadcasting the messages. 5.4 concerns about privacy threats the different concerns about privacy threats distinguished the three personas. overall, personas 1 and 2 hold much fewer concerns about privacy threats as compared with persona 3. between persona 1 and the others, the prior is bothered more by the threat related to occupations while the other two personas have higher concern about scams and trickery. by considering the demographics of personas 1 and 2, it could be hypothesised that the types of concerned privacy threats are associated with the differences in occupations if not even genders. nevertheless, occupational threats were also not consistently concerning among the working groups as it was shown in the comparison between personas 1 and 3. we believe such dissimilarities could be explained by the different approaches in establishing relationships, which persona 1 appears to connect with wider networks than persona 3. as a consequence, 14 australasian journal of information systems dang-pham, pittayachawan & nkhoma 2015, vol. 19, research article demystifying vietnamese online personas persona 1 may be aware of their larger exposure to the threats of having careers affected by inappropriate contents or having their profile checked by recruiters. among the three personas, persona 3 holds the highest concerns about their online privacy being threatened by a wide range of threats mentioned in previous studies (krasnova et al. 2010, 2009; stutzman et al. 2011). moreover, these participants may have the attributes of the users who are aware of third parties’ violation of privacy, which would normally be invisible to other “average users” (debatin et al. 2009). the varied concerns about one’s online privacy being threatened may also be accounted by the different perceived values of information. in fact, most participants of personas 1 and 2 who had little concern about this aspect agreed so, and one even explained that her information being leaked would not bring severe consequences. interestingly, this conjecture was supported by the three personas’ consistently low concern about being located physically by check-ins. a majority of participants discussed that virtual check-ins could be easily faked (i.e. not checking in at actual locations). consequently, they argued that such information does not hold much meaning and would result in little harm. on the other hand, the higher concern about cyber-stalking was tied to the information in their profiles such as school or home addresses, rather than checked-in places. while the threat of spam and spim is evolving and migrating from the traditional email to social media platforms (symantec 2014), there was not much concern about this threat. some participants noticed the increased amount of spam received since they joined facebook, but were also unsure whether sns was the cause. nonetheless, they reported to either set up spam filter or register facebook’s account with unimportant mailboxes to mitigate the problem. additionally, we considered racism as a possible social threat on sns, but all personas displayed no concern about this threat and denied its existence. on the other hand, the threat of insult and threatening was ranked consistently at a higher position, and the concern about it was discussed as a yes-and-no answer. as presented in the findings’ sections, some participants revealed that they had experiences with this threat before and now believed it would not happen again, whereas others shared that it is not concerning as long as the insults do not come from their friends. this finding reflects the medium concerns regarding social threats as surveyed in the study by krasnova et al. (2009). 5.5 concerns about relationships keeping in touch with friends and family members have been commonly discussed by prior research (ellison et al. 2011; park et al. 2009; subrahmanyam et al. 2008). our study also found maintaining these relationships is of consistently high concern to the three personas’ participants. in addition, the users of personas 1 and 3 who had work experiences were concerned about expanding their professional networks. this finding reflects the discussions in relevant studies (a. carrillat et al. 2014; dimicco and millen 2007; skeels and grudin 2009) that argued facebook is becoming more related to organisational aspects such as business networking and recruitment. furthermore, two distinct networking approaches were detected from these personas even though they share similar demographics. specifically, persona 1 is more interested in connecting with everyone met offline and online, whereas persona 3 only feels concerned about connecting selectively with people met in person. 6 implications and limitations we believe our typology of facebook personas and their specific concerns hold practical and theoretical implications. the findings could be helpful for designing marketing tactics and for social media developers, especially facebook, to enhance user’s experiences. while our finding about the relationships between demographics (i.e. genders and occupations) and their information of interest (i.e. about people or products/events) cannot be generalised to a larger population, it was consistent with the result of a prior research in the us context about young female students having a positive attitude towards online shopping on sns. the female students of persona 2 were highly concerned about building online trust and connecting with known acquaintances. it may be worthwhile to explore whether referrals and testimonials 15 australasian journal of information systems dang-pham, pittayachawan & nkhoma 2015, vol. 19, research article demystifying vietnamese online personas based on marketing would be effective for the demographics of this persona. on the other hand, finding information about celebrities, as well as events (e.g. expos and sales events) and parties, was ranked uniformly low by personas 1 and 3. while this could be due to these personas’ preferences, it could also hint that facebook pages may not be the best platform to disseminate these types of information to the vietnamese young adult audiences. the high level of concern about “connecting with acquaintances” supports this item to be the main purpose of using facebook. furthermore, previous analyses on the low ranking items such as “finding dates” and “connecting with everyone online” suggest that facebook may not be a suitable platform for connecting with strangers. on the other hand, expanding business network on facebook was highly ranked by participants with work experiences. while there are other sns that are exclusively designed for professional networks such as linkedin, perhaps facebook with its unique socialising functionalities would enable different types of interactions that may be interesting for future research to explore. for example, linkedin focuses on effectively encapsulating and showcasing its users’ work experiences, which are most suitable for disseminating profiles and reaching new people. on the other hand, users on facebook would exploit the interactional functions to expand their business networks via richer communication, or perhaps by showcasing their desirable traits to potential contacts in a different way. in addition, social pages and blogs are the interactional tools that are available on facebook for its users to socialise with people of common interests and exchange information. however, some users shared that they joined social pages on facebook mainly for obtaining information rather than socialising. perhaps organisations that run facebook pages may want to focus more on delivering the desired content to the audience, as well as control the types of information and sources on their page that would attract the users’ attention (e.g. other users’ comments). moreover, we also found multiple ways to express feelings from the users. for instance, a female student from persona 2 reported to communicate her feelings by sharing music and video, whereas a more matured user from persona 3 chose to write down thoughts in his blog and set it available only to him as a way to relieve stress. consequently, we learned that facebook’s functions may not always be used for their original purposes and could contain different meanings. we also found the participants to hold different views about their online privacy. the three personas seemed to understand well which information would be appropriate to be disclosed, and some intentionally accepted the risks to achieve their purposes (e.g. included identifiable information to make accounts searchable by friends). our sample’s participants also considered the threat of racism as not existing, which may be due to the fact that vietnam’s population is generally homogeneous or that they have a high level of racism tolerance. as a result, it would be interesting to replicate this research in heterogeneous and multicultural countries (e.g. australia) and compare the findings. on the other hand, the concerns about usability of facebook’s multiple privacy settings were relatively high in general. moreover, participants of persona 3 especially feel concerned about the privacy threats on facebook. while these participants could have many reasons and past experiences that led to such high concerns, it may also be due to them not knowing much about the available privacy settings. sns should continue to educate their users as well as make it easy for them to effectively use the provided protection. furthermore, a majority of our participants felt unsure about whether facebook as a global organisation would provide them assistance if their online privacy was harmed, provided that they are local users. in this case, the perceived presence of sns organisations would play a crucial role in providing the global users a state of security and may subsequently affect their behaviours, thus its visibility should be enhanced. even though we have attempted to ensure rigour and richness in analysis and discussion, our research inevitably contains limitations. first, the proposed typology of the three personas is exploratory and we cannot confirm that their unique concerns had not occurred by sampling’s chance. secondly, 49% of explained variance stems from the small sample size of 14 participants, and there may be more profiles that are not discovered in this study. 16 australasian journal of information systems dang-pham, pittayachawan & nkhoma 2015, vol. 19, research article demystifying vietnamese online personas our analysis suggested three as the optimum number of unique personas that could be extracted. with a larger number of participants and items, there could be more different personas to be found. these limitations, combined with our focus on vietnamese young adults, clearly need follow-up studies in the future. in that case, this typology may serve as the first step to classify facebook’s young adult users, and it may be changed as being tested by future studies in larger samples or different contexts. more importantly, the items used to construct this typology would be especially useful for developing future research instruments and bring forward an important theoretical implication. one of the theoretical implications includes the consensus and distinct items as shown in the appendix. the consensus items can be used to develop an invariant measurement which is stable across all personas and can be subsequently tested with factor analysis. researchers can use the distinct items to develop a measurement to classify users into different personas that can be tested with latent class analysis, since each item can be used to capture specific traits of each persona. in addition, our study also demonstrated a new way to classify social media users’ personalities by using q-methodology. the virtual world on sns such as facebook is indeed multifaceted and consists of diverse end users’ concerns and behaviours. we look forward to seeing our practical and theoretical implications contribute to future studies that further expand our knowledge of this vast research domain. references a. carrillat, françois, alain d’astous, and emilie morissette grégoire. 2014. “leveraging social media to enhance recruitment effectiveness.” internet research, 24 (4): 474–495. boyd, danah m., and nicole b. ellison. 2007. “social network sites: definition, history, and scholarship.” journal of computer-mediated communication, 13 (1): 210–230. brandtzaeg, petter bae, and jan heim. 2011. “a typology of social networking sites users.” international journal of web based communities, 7 (1): 28. cha, jiyoung. 2009. “shopping on social networking web sites.” journal of interactive advertising, 10: 77–93. debatin, bernhard, jennette p. lovejoy, ann-kathrin horn, and brittany n. hughes. 2009. “facebook and online privacy: attitudes, behaviors, and unintended consequences.” journal of computer-mediated communication, 15 (1): 83–108. dimicco, joan morris, and david r millen. 2007. “identity management: multiple presentations of self in facebook.” in group’07, 0–3. ellison, n. b., c. steinfield, and c. lampe. 2011. “connection strategies: social capital implications of facebook-enabled communication practices.” new media & society, 13: 873–892. facebook. 2015. “company info.” newsroom. http://newsroom.fb.com/company-info/. gross, ralph, and alessandro acquisti. 2005. “information revelation and privacy in online social networks.” in acm workshop on privacy in the electronic, 71–80. hassouneh, diana, and malaika brengman. 2014. “a motivation-based typology of social virtual world users.” computers in human behavior, 33: 330–338. hofstede, geert. 2001. culture’s consequences: comparing values, behaviors, institutions, and organizations across nations. second edi. thousand oaks ca: sage publications. joinson, adam n. 2008. “‘looking at’, ‘looking up’ or ‘keeping up with’ people? motives and uses of facebook.” in chi 2008, 1027–1036. karl, katherine, joy peluchette, and christopher schlaegel. 2010. “who’s posting facebook faux pas? a cross-cultural examination of personality differences.” international journal of selection and assessment, 18 (2): 174–186. 17 australasian journal of information systems dang-pham, pittayachawan & nkhoma 2015, vol. 19, research article demystifying vietnamese online personas kietzmann, jan h., kristopher hermkens, ian p. mccarthy, and bruno s. silvestre. 2011. “social media? get serious! understanding the functional building blocks of social media.” business horizons, 54 (3): 241–251. krasnova, hanna, oliver günther, sarah spiekermann, and ksenia koroleva. 2009. “privacy concerns and identity in online social networks.” identity in the information society, 2 (1): 39–63. krasnova, hanna, sarah spiekermann, ksenia koroleva, and thomas hildebrand. 2010. “online social networks: why we disclose.” journal of information technology, 25 (2): 109–125. lampe, cliff, nicole ellison, and charles steinfield. 2006. “a face (book) in the crowd: social searching vs. social browsing.” in proceedings of the 2006 20th anniversary conference on computer supported cooperative work. acm, 167–170. mehdizadeh, soraya. 2010. “self-presentation 2.0: narcissism and self-esteem on facebook.” cyberpsychology, behavior and social networking, 13 (4): 357–364. meyen, m., s. pfaff-rudiger, k. dudenhoffer, and j. huss. 2010. “the internet in everyday life: a typology of internet users.” media, culture & society, 32 (5): 873–882. nadkarni, ashwini, and stefan g hofmann. 2012. “why do people use facebook ?” personality and individual differences, 52 (3): 243–249. park, namsu, kerk f kee, and sebastián valenzuela. 2009. “being immersed in social networking environment: facebook groups, uses and gratifications, and social outcomes.” cyberpsychology & behavior : the impact of the internet, multimedia and virtual reality on behavior and society, 12 (6): 729–733. pew research center. 2014. “demographics of key social networking platforms.” social media update 2014. http://www.pewinternet.org/2015/01/09/demographics-of-keysocial-networking-platforms-2/. pittayachawan, s, m singh, and brian corbitt. 2008. “a multitheoretical approach for solving trust problems in b2c e-commerce.” international journal of networking and virtual organisations, 5: 369–395. skeels, meredith m, and jonathan grudin. 2009. “when social networks cross boundaries: a case study of workplace use of facebook and linkedin.” in group’09, 10:95–103. stephenson, william. 1953. “the study of behavior: q-technique and its methodology.” stoughton, j william, lori foster thompson, and adam w. meade. 2013. “big five personality traits reflected in job applicants’ social media postings.” cyberpsychology, behavior and social networking, 16 (11): 800–806. stutzman, f. 2006. “an evaluation of identity-sharing behavior in social network communities.” international digital and media arts journal, 3: 10–18. stutzman, fred, robert capra, and jamila thompson. 2011. “factors mediating disclosure in social network sites.” computers in human behavior, 27 (1): 590–598. subrahmanyam, kaveri, stephanie m reich, natalia waechter, and guadalupe espinoza. 2008. “online and offline social networks: use of social networking sites by emerging adults.” journal of applied developmental psychology, 29 (6): 420–433. symantec. 2014. “internet security threat report 2014.” vol. 19. http://www.symantec.com/content/en/us/enterprise/other_resources/bistr_main_report_v19_21291018.en-us.pdf. tan, xin, li qin, yongbeom kim, and jeffrey hsu. 2012. “impact of privacy concern in social networking web sites.” internet research 22, (2): 211–233. 18 australasian journal of information systems dang-pham, pittayachawan & nkhoma 2015, vol. 19, research article demystifying vietnamese online personas watts, s., and stenner, p. 2012. “doing q methodological research: theory, method and interpretation.” qualitative research in psychology, 2 (1): 67–91. wearesocial. 2012. “social, digital and mobile in vietnam.” http://wearesocial.sg/blog/2012/10/social-digital-mobile-vietnam-oct-2012/. wearesocial. 2014. “social, digital & mobile in apac in 2014.” http://wearesocial.net/blog/2014/01/social-digital-mobile-apac-2014/. wearesocial. 2015. “digital, social & mobile worldwide in 2015.” http://wearesocial.net/blog/2015/01/digital-social-mobile-worldwide-2015/. zabala, aiora. 2014. “qmethod: a package to explore human perspectives using q methodology.” 19 australasian journal of information systems dang-pham, pittayachawan & nkhoma 2015, vol. 19, research article demystifying vietnamese online personas appendix domains # items relationships 1 connect with family or close friends 7 connect with people just met outside 9 keep in touch with old friends 10 can connect with everyone online 11 can connect with everyone met in real life 12 connect selectively 2 find dates 8 expand business and professional network sns functionalities 28 want to play games on this sns 5 share music and video 26 sns has multiple privacy settings 40 sns has large number of users 44 good usability of privacy settings 45 good usability of sns 23 trust protective service and function of sns interactions 25 want to share personal pictures to public audience 29 want to communicate with public audience 41 join sns pages of common interests 15 write blog about general experiences or events 14 write blog about private and personal experiences 16 building trust on this sns is important 22 trust my friends on sns will not harm me finding information 46 find more info about people at school or workplace 48 find more info about people just met outside 47 find info about celebrities 43 find info about products or companies 49 read news from multiple sources 4 find info about events and parties privacy threats 31 there can be threats of insult, threatening 32 there can be threats of scams, trickery 33 there can be threats of identity theft 34 there can be threats of cyber-stalking 30 cannot locate me in real life 35 there can be threats of racism 36 there can be threats of spam or spim 38 threats of recruiters check info on sns 39 threats of losing job due to complaint or talk bad about job 42 threats of having personal info shared by organisations 50 sns company could use my personal info 20 my privacy is threatened on this sns 24 my identifiable info can be used for bad purposes identity 6 use real info to create account 13 use real name on this sns 17 publish my real pictures 3 my account needs to appear good 21 create snss accounts with similar info 19 not expose much info on this sns 18 share sensitive info e.g. gender preferences or drinking habit 27 share info about religion or political points of view 37 receive approval from sns friends e.g. facebook likes table a1: list of items (statements) 20 australasian journal of information systems dang-pham, pittayachawan & nkhoma 2015, vol. 19, research article demystifying vietnamese online personas distinguishes # items all f1 only f2 only f3 only consensus 1 connect with family or close friends o 2 find dates o 3 my account needs to appear good o 39 threats of losing job due to complaint or talk bad about job o 13 use real name on this sns o 25 want to share personal pictures to public audience o 31 there can be threats of insult, threatening o 32 there can be threats of scams, trickery o 38 threats of recruiters check info on sns o 40 sns has large number of users o 48 find more info about people just met outside o 50 sns company could use my personal info o 4 find info about events and parties o 7 connect with people just met outside o 16 building trust on this sns is important o 18 share sensitive info e.g. gender preferences or drinking habit o 28 want to play games on this sns o 37 receive approval from sns friends e.g. facebook likes o 47 find info about celebrities o 5 share music and video o 10 can connect with everyone online o 11 can connect with everyone met in real life o 14 write blog about private and personal experiences o 20 my privacy is threatened on this sns o 24 my identifiable info can be used for bad purposes o 29 want to communicate with public audience o 33 there can be threats of identity theft o 34 there can be threats of cyber-stalking o 41 join sns pages of common interests o 42 threats of having personal info shared by organisations o 43 find info about products or companies o 49 read news from multiple sources o 6 use real info to create account o 9 keep in touch with old friends o 12 connect selectively o 15 write blog about general experiences or events o 17 publish my real pictures o 21 create snss accounts with similar info o 22 trust my friends on sns will not harm me o 30 cannot locate me in real life o 35 there can be threats of racism o 36 there can be threats of spam or spim o 44 good usability of privacy settings o 8 expand business and professional network 19 not expose much info on this sns 23 trust protective service and function of sns 26 sns has multiple privacy settings 27 share info about religion or political points of view 45 good usability of sns 46 find more info about people at school or workplace table a2: list of distinguishing/consensus items (statements) 21 australasian journal of information systems dang-pham, pittayachawan & nkhoma 2015, vol. 19, research article demystifying vietnamese online personas copyright: © 2015 dang-pham, pittayachawan & nkhoma. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. 22 http://creativecommons.org/licenses/by-nc/3.0/au/ http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 1.1 background 1.2 research context 2 literature review 2.1 concerns about socialisation and functionalities 2.2 concerns about threats of privacy 3 research methodology 3.1 development of the statements 3.2 profiles of the participants 3.3 data analysis 4 research findings––the three personas 4.1 persona 1: the outgoing connector 4.2 persona 2: the reserved trustor 4.3 persona 3: the threats perceiver 4.4 summary of the three personas 5 discussion 5.1 concerns about finding information 5.2 concerns about identity 5.3 concerns about interactions and facebook’s functionalities 5.4 concerns about privacy threats 5.5 concerns about relationships 6 implications and limitations the australasian journal of information systems (ajis) began in 1993 and was based on the simple realisation that there was a need for an academic ‘outlet’ for is research in australia australasian journal of information systems volume 18 number 2 2014 149 where is my pay? critical success factors of a payroll system – a system life cycle approach mohan thite griffith university m.thite@griffith.edu.au kuldeep sandhu griffith university k.sandhu@griffith.edu.au abstract majority of firms deploy technologies in hr administrative applications. while payroll processing is a routine transactional activity, poor design and implementation of payroll system can cause immense harm to employee and organizational well-being. based on the case study of a flawed payroll system in a large and complex public sector organization in australia, we highlight the key success factors using the system life cycle approach underpinned by the agile philosophy. it highlights the critical importance of strategic organizational review, user involvement and ongoing communication with diverse stakeholders during the planning, analysis, design, implementation and review stages of a payroll project. it reinforces the need for the adoption of and adherence to sound project and change management methodologies. we also explore the limitations of shared service center approach. keywords: e-hrm; payroll system implementation; queensland health; system development life cycle (sdlc), project management; critical success factors; shared service center introduction today, organizations, both in public and private sectors are under constant pressure to seek ways and means of ‘doing more with less’ and healthcare industry is no exception (abraham and junglas, 2011). one way of doing this is by reengineering the business processes and deploying latest technologies that help enhance efficiency and effectiveness. however, this is considered more difficult in a large and complex healthcare environment (bohmer, 2010). indeed, by some estimates, the failure rates of new it systems in general, as measured in their ability to meet deadlines, to be within budget and fulfilling the key objectives, are approaching 50% (fisher and howell, 2004). however, research also clearly indicates that it is not technology per se but its faulty implementation and poor understanding of social processes, including inherent organizational rigidity, inertia and resistance to change that is the main reason for firms not being able to achieve all or most of their key goals in any technology project implementation (klein and ralls, 1997; lengnick-hall and lengnick-hall, 2006; waring, 2004). payroll processing is one of the first and most frequent automations of hr activities (thite et al., 2012). while by itself it adds no value to a firm’s competitive advantage, pay slips are the most scrutinized documents by employees and therefore, an incorrect payment of pay can lead to intense employee dissatisfaction. this paper looks at payroll project implementation at queensland health, the third largest employer in australia, which turned out to be a ‘debacle’ and caused widespread financial losses, decline in employee morale and created a political storm which continued to haunt the organization more than two years after its hasty and ill-conceived implementation. this case mirrors australasian journal of information systems volume 18 number 2 2014 150 shrivatsava and shaw’s (2003) assertion that “horror stories of failed technology implementation efforts abound and hardly inspire confidence” (p. 201). the purpose of the paper is two-fold: one, to ascertain the main reasons for the failure of the new payroll implementation project in the case study organization using the key findings of published independent reviews, and two, to develop a theoretically and practically derived system development life cycle model identifying key success factors for large scale and complex technical project implementation in a public sector environment that uses the whole-of-government shared service center approach. the significance and contributions of the study include: an examination of a payroll system with a holistic, multiple stakeholder focus which helps us better understand the factors influencing the outcome of information system (is) implementation. such a focus pays due attention to the dynamic interaction between organizational factors, technical system, process environment and project/change management. it reinforces the relevance of the traditional system development life cycle (sdlc) approach when it is underpinned by new agile methodologies, combining both rigor and flexibility in contemporary organizational settings. in addition to the key success factors, the study includes a critical examination of the shared service center approach, as it had a major impact on the payroll implementation in the case study organization, an aspect that has not been sufficiently recognized and addressed in the is implementation literature of large and complex organizations. the paper starts with a brief introduction to the critical success factors in technical project implementation, with specific reference to payroll system. it is followed by an over view of the case study organization and the design and implementation of its new payroll system under the umbrella of shared service center. it then examines what went wrong, its fallout and the lessons learnt. the paper concludes with a discussion of findings and their implications for managerial practice. project implementation: critical success factors at the macro level, the organizational context, in terms of the nature and complexity of its operations in which the technology implementation takes place, is a crucial consideration. for example, chang et al. (2000) who looked at enterprise resource planning implementation issues in the australian public sector found that knowledge management, system development concerns and operational deficiencies were of particular concern. in a complex environment, such as a healthcare organization, even routine and trivial software implementation can have a profound effect (stefanou and revanoglou, 2006). to successfully acquire an enterprise resource planning (erp) software, verville et al., (2005) demonstrate that clarity of authority, a structured, rigorous and user-driven planning process, well defined evaluation criteria and partnership between implementation team, users and the vendor are the key success factors. researchers also emphasize the importance of top management commitment to provide adequate resources (ngai and wat, 2006), total employee involvement and buy-in (barker and frolick, 2003; ehie and madsen, 2005), continuous monitoring and feedback (varma and gopal, 2011), thoughtful consideration of system-design elements (fisher and howell, 2004), particularly based on the modern socio-technical view that prefers empowerment over control (benders et al., (2006), careful change management and cultural readiness (motwani et al., 2005) and communication processes (stone and lukaszewski, 2009). similarly, in the post-implementation stage, a systematic and planned review of overall project scope and planning, driving principles of project development, misfit resolution strategies and evaluation of benefits and user learning are critical (nicolaou and bhattacharya, 2006). australasian journal of information systems volume 18 number 2 2014 151 at a more general level, the technology acceptance model proposes that perceived ease of use and usefulness are primary determinants of technology acceptance (ma and liu, 2004). however, technology implementation can lead to both functional and dysfunctional or intended and unintended consequences. in order to minimize unintended consequences of technology implementation, such as users avoiding to use the system (inertia) or work around it (reinvention), some researchers highlight the importance of human agency perspective that directs attention toward social contexts and processes whereby rather than technology, variety of stakeholders or actors exercise social influence to alter the pattern of use leading to desired organizational change (boudreau and robey, 2005; light and wagner, 2006). in contrast with the hard-wired, inflexible routines embedded in technology, particularly in vanillabased erp packages, the socio-technical view and human agency perspective prefer empowering employees with the required flexibility and discretion to overcome technology constraints as they learn to navigate the new system under the influence of different social actors. this approach supports orlikowski’s (2000) ‘technology-in-practice’ framework which argues that technology’s consequences emanate from user enactment and improvised learning rather than technical features of a new system and therefore should be proactively managed. here, organizational factors, such as the nature and structure of work play an important role influencing how individuals use technologies (dery et al., 2006). further, orlikowski and scott (2008) propose a ‘sociomaterial’ approach to study the fusion of technology and work in organizations with a view that “there is an inherent inseparability between the technical and the social” (p.434). taken together, the critical success factors in any it project can be summarized under the five stages of a typical system development life cycle (sdlc), namely, planning, analysis, design, implementation and maintenance/review (alge and upright, 2012). we will later use the sdlc approach to highlight what went wrong with the new payroll system in the case study organization. the typical traditional approaches to sdlc (often called incremental, waterfall or stage-gate models) are deemed to be rationalized, rigid and purely engineering-based with lack of attention to fast changing needs of the customers and the impact of social factors in technology implementation, such as organizational culture and teamwork (nerur, mahapatra and mangalaraj, 2005). in contrast, the newer agile methodologies focus more on people than processes, more responsive to changing environments and user requirements and are characterized by flexibility, leanness and collaboration (dyba and dingsoyr, 2008). another study recommends an iterative system development cycle (isdc) with inbuilt feedback loops or prototyping cycle in a public sector environment (rubin, 1986). payroll system: design & implementation the payroll function is generally administered by the finance department in close coordination with hr which feeds employee compensation and benefits data. as labor costs are typically the largest variable cost of most firms and pay check is the most visible symbol of employment relationship, it is vital that payroll records are timely and accurate. a survey by the american payroll association reported that more than two-thirds of americans live paycheck to paycheck and expect to be paid on time, every time (the pak banker, 2010). therefore, payroll should be flawless for the organizational and employee well-being (fay and nardoni, 2012). considering that an enormous amount of data needs to be generated, received and processed to ensure timely and accurate salary payments as well as compliance with legislative requirements, “the union of technology and payroll seems like a perfect marriage” (danilewicz, 1997, p. 6). the “y2k” crisis highlighted the challenge of using up to date technologies to constantly and continually update or replace legacy systems for effective and seamless payroll service delivery (dennis, 1997). further, with the increasing emphasis on performance based pay, the payroll system has to be substantially australasian journal of information systems volume 18 number 2 2014 152 transformed to link pay with employee stock ownership schemes and other progressive hr policies and practices, such as self service portals that let employees to access payslips online, electronic transfer of pay directly into employee bank accounts and adopting hr shared service center concept (phelan and hayes, 2005; jereb et al., 2009). based on a study of payroll application users, hunton and price (1997) highlight the importance of user participation and involvement in developing the system as it enhances their perception of procedural justice and control. an increasing trend in large organizations to derive further benefits of technology in hr service delivery is to bundle hr administrative tasks within a shared service center (ssc) (isenhour, 2012). it is a collaborative strategy whereby several related functions and processes are clustered under the umbrella of an independently managed strategic business unit or center of excellence in order to reduce costs and improve quality (goh et al., 2007). for this strategy to succeed, the business units whose functions are being transferred to ssc need to establish an overarching vision, trust each other, be open and transparent and follow a thoughtful and thorough change management approach (accenture, 2007). jereb et al. (2009) demonstrate that within the ssc architecture, transparency and comparability of payroll processes can be significantly enhanced by focusing on key performance indicators (kpis) and benchmarking. there have been several instances of payroll system implementation going horribly wrong with severe financial consequences for the organization and employees and resulting in poor industrial relations. for example, the roll out of a new payroll system for the southern california district in the usa as part of a new integrated technology system lead to underpayments, overpayments and incorrect withholdings for over 35,000 employees (trotter, 2007). it was acknowledged that the payroll system’s rollout was rushed and tainted by numerous programming glitches, such as insufficient system testing and poor training of payroll officials. similarly, the australian defense force (adf) payroll system overpaid allowances to some military staff deployed overseas and the adf’s attempts recover them became a sensitive issue in the media and lead to the announcement of a high level task force to fix the problems (dodd, 2010). case study research design considering the exploratory nature of our study, the qualitative case study method is an appropriate choice. according to doz (2011), qualitative research is uniquely suited to “opening the black box of organizational processes” and can make a “substantial contribution” to areas, such as theory building as it stimulates deeper thought by “providing rich, thick descriptions of real phenomena and action instances (p. 583-584). we used ‘document analysis’ as our primary source of data for this particular case study because considering the enormous size and scale of the payroll implementation failure in terms of financial loss, public outrage and employee unrest, several reputed consulting organizations were asked by the government to probe and deliver the findings on this case. these consultants were given unhindered access to all the key stakeholders to prepare their reports and what is more, these reports have been placed in the public domain. as yin (2003) points out, the strengths of documents as a source of evidence are that they can be viewed repeatedly, they are exact (in details), they have a broad coverage (of time, events and settings) and they are unobtrusive (not created as a result of the case study) (p. 86). the following case study on queensland health (hereafter referred to as qld health), the queensland state government run health department in australia, has been prepared with the following key official documents that have been publicly made available by the queensland government, as a primary source of data: australasian journal of information systems volume 18 number 2 2014 153 reports by the auditor-general of queensland submitted to parliament on the queensland health payroll system (agq, 2010a; 2010b) an independent review of queensland health payroll implementation by kpmg (kpmg, 2012; 2010a; 2010b) an independent review of queensland health payroll and rostering solutions by ernst and young (ernst & young, 2010) and an independent review of the shared service model for queensland government by pricewaterhousecoopers (pwc, 2010) the independent reviews by kpmg, ernst and young and pricewaterhousecoopers involved extensive consultations with all the key stakeholders, including qld health managers, payroll staff, employees and trade unions, the owner of shared service centre and the vendor. as these reports incorporate the voices of all key stakeholders and written by independent experts, the data provided in these reports over a two year period can be considered sufficient enough to undertake an informed analysis of the case study organization. in addition, qld health web site, newspaper reports and government announcements and notifications were used as secondary source of data. ‘applied social research’ is most suitable for the subject in this study because of its potential for actionable outcomes, as it helps us better understand the problem in order to find the most suitable solution, particularly in the context of government agencies (richie and spencer, 1994). document analysis, as part of grounded theory approach, is a potent way to carry out qualitative research (bowen, 2009). one of the problems in undertaking exhaustive applied research in an institutional context is the lack of access to key policy makers and implementation actors for reasons of confidentiality. therefore, this study has relied on publicly available consultancy reports commissioned by the government and tabled before the legislature as the researchers who submitted these reports were given unhindered access to all key documents and stakeholders. in terms of data analysis, the authors first ‘familiarized’ themselves with the data from different sources, ‘identified a thematic framework’ and then ‘indexed, charted and mapped’ the data before objectively ‘interpreting’ it (richie and spencer, 1994). organizational profile qld health aims at achieving good health and well being for all queenslanders. through a network of hospitals, healthcare facilities and health service districts, qld health delivers a range of integrated services including hospital inpatient, outpatient and emergency services, mental health services, primary healthcare and chronic disease management in hospitals and community health centers. with over 85,000 staff in permanent, casual and temporary positions in both clinical and non-clinical areas, qld health is the third largest employer in australia. the organization has a complex system of employment terms and conditions governed by 12 awards and 6 industrial agreements which provide for over 200 different allowances and up to 24,000 different combinations of pay. there are over 1000 new starters and leavers in a standard fortnight reflecting the fluidity in its workforce composition. qld health pays on an average $250 million in salaries and entitlements on a fortnightly basis to its employees spread over 300 work sites. overview of previous payroll system in terms of the payroll process, the information on all rosters, shift changes, allowances and leave entitlements is provided by qld health payroll staff to a contracted firm to produce printed payslips. it is noteworthy that about 40,000 emails and faxes containing payroll processing requests are received by payroll staff per pay period. since not all employees regularly use computers, payslips are printed australasian journal of information systems volume 18 number 2 2014 154 as agreed with the trade unions representing the workforce. there is a small time period of two days available to make corrections and adjustments to perform ad hoc pay runs before the electronic file is provided to the various banks to deposit pay directly into employee bank accounts. this often results in ad hoc pay and differences between net pay shown on the payslip and the amount deposited into the employee bank account. this has been a normal part of the payroll process leading to an increase in rate of errors reported by employees and 15,000 adjustments or ‘reworks’ required after each pay run. the previous system was built on lattice in 1997. due to the limitations of the lattice payroll system and the underlying complexity of the qld health awards and allowances, the payroll ‘rework’ rate was approximately 20% of total payees. this situation was clearly unsustainable and needed urgent attention. in addition, vendor support for the lattice payroll system expired in june, 2008 leading to an urgent need for its replacement. design & implementation of new payroll system the queensland government established corp tech, a shared service entity, with a ‘whole of government’ approach over the acquisition of information technology infrastructure (including hardware and software) for all queensland government agencies. under the shared service initiative, all agencies were mandated to implement a standard software suite, including ‘sap hr, workbrain rostering software and sap finance’ with ibm as the prime contractor to both manage and implement payroll and rostering systems. the first pilot project was done by ibm at the queensland department of housing in 2007 and qld health was the next agency in line. three key players were involved in the replacement of qld health payroll system: corp tech, a specialized government business unit under the shared service concept, and the owner of sap hr and workbrain systems ibm, as the prime contractor to corp tech, to direct, manage and control the project and to implement sap hr and workbrain solution to replace lattice qld health, the business user responsible for the payment of salaries and entitlements to its employees. primarily responsible for ensuring that scope of works reflected business requirements, data cleansing and migration, user acceptance processes and staff training. ibm was required to replace the qld health payroll system by implementing the department of housing model of sap hr ‘with very little customization’. corp tech approved the initial planning and scoping of the lattice replacement ‘interim solution’ with the specifications for the design, configuration, build, testing and implementation documented in a statement of work in january, 2008. however, in october, 2008 “detailed planning revealed that the size, complexity and scope of this phase of the program had been severely underestimated, with the consequence that its revised implementation cost estimates significantly exceeded the original tender proposal” (agq, 2010a, page 16). corp tech approved over 47 change requests from ibm which “were mainly due to the business requirements not being clearly articulated and agreed to at the outset of the project” which resulted in the continuous failure of the user acceptance testing criteria. consequently, ‘the system implementation was over 18 months after the scheduled go-live date and approximately 300% over the original cost budget’. when the new payroll system went live in march, 2010, qld health experienced significant payroll errors resulting in staff being overpaid, underpaid, or not paid at all. what went wrong? the auditor-general of queensland (agq, 2010a), highlighted several key reasons for the failure of the new payroll system as below: australasian journal of information systems volume 18 number 2 2014 155 under-estimation of project scope & complexity: the qld health payroll system is very complex due to a large workforce and multiple employment awards and agreements and therefore, needed careful attention from start to finish. the prime contractor was expected to use the same sap hr system that was used for queensland housing with ‘very little customization’ and provide ‘like for like replacement’ for the previous lattice system. however, the requirements of qld health were significantly more complex than that of the department of housing because the latter had about 1300 employees with one employment award whereas the former had 78,000 employees with multiple awards. in addition, the business rules for lattice and sap hr are very different with the latter requiring more rigor and discipline making the ‘like for like’ assumption invalid (page 23 and 25). poor governance: the governance structure of the system implementation between the key agencies, that is corp tech, ibm and qld health, was not clear, causing confusion over the roles and responsibilities, authority and accountabilities of various parties. in the complex shared services arrangement, despite attempts to clarify roles and responsibilities, there still existed some tension between qld health as owner of data and corp tech as owner of the system (page 3). poor user involvement & buy-in: the ‘the identification of the customer in this project was confusing’ (page 21). while key project deliverables were signed between shared service entity (corp tech) and prime contractor (ibm), there was no explicit endorsement by qld health, the system user. ill-defined project scope: the ‘project scope was not formally agreed to by qld health, and negotiations over the scope occurred throughout the project, resulting in over 47 change requests’ (page 21). the scope definition was formally approved by qld health, ‘20 months after the commencement of the project’ (page 25).the ‘project creep’ occurred because there was inadequate documentation of business requirements at the commencement of the project. the absence of periodic review of the business needs contributed to subsequent difficulties with system testing and implementation. the ‘absence of an agreed scope from all key stakeholders from the beginning of the project’ lead to cost blow out and delays in implementation. poor project management: the pre-implementation system and process testing failed to identify significant implementation risks. moreover, a number of critical business readiness activities and practices were not fully developed prior to implementation. ‘no specific project management methodology was applied throughout the life of the project’ (page 3). there was ‘no one entity monitoring and managing total project budget versus costs being incurred by all of the various stakeholders’ resulting in no accountability for project costs (page 27). mad rush to go live: parallel payroll testing was done at the same time as user acceptance testing for a sample of 10% of employees and 60% of functionality of the new system. ‘sap recommended a full parallel pay run comparison’ between the old and the new system prior to go-live; however, this ‘recommendation was not accepted by the project board due to the size and complexity of undertaking this task’ (page 29). poor risk management: before the final decision to go-live, a full risk profile was undertaken; however, ‘the risks were not quantified to indicate the extent of the problem, should the risk materialize, that is how many or what category of staff may not be paid’. the code fixes were migrated to the production environment prior to the business cut-over which ‘is not considered good practice’ as ‘it increased the risk that the live system could become unstable’ (page 30). there was no ‘contingency planning for business cut-over’. adapting from jereb, kuchem and sohn (2009), figure-1 presents the key causes of failure of the qld health payroll system under four headings, namely, organization, project/change management, systems and process environment. australasian journal of information systems volume 18 number 2 2014 156 the fallout while ibm as the prime contractor defended the quality of its work, the payroll staff identified 70 major problems with the system and according to the australian services union, there were 35,000 wage anomalies and around three-quarters of the issues involved system or configuration errors (the australian, 2010a and 2010b). with the press reporting that the ‘sick’ payroll system that cost $64.5 million ‘does not work’ and calls for the resignation of the health minister, the queensland government went into damage control promising a quick fix and terminating the services of two senior executives (the australian, 2010c). it was reported in the media that the system was deducting overpayments from employees’ bank accounts automatically and without consent in breach of employment agreement but qld health expressed its inability to stop it as it was a ‘whole-of-government’ problem (courier mail, 2010a). with thousands of qld health employees continuing to report errors in their pay checks for several pay cycles, the government appointed kpmg to review the payroll system and according to the media, kpmg report revealed ‘turmoil and incompetence’ with the government admitting that qld health’s initial efforts were “not good enough” and that there was no sufficient contingency plan to protect staff from the system’s failure (courier mail, 2010b). later, with the release of the auditor general’s report that ‘found the botched payroll system was approved despite serious awareness of the extent of its risks’, the queensland public sector union called it an ‘unprecedented failure of public administration’ (courier mail, 2010c). with the change of government in 2012, the new government commissioned another enquiry by kpmg which reported that the total cost of the payroll system would be around a whopping $1.2 billion, between 2010-2017 (kpmg, 2012). lessons learned the auditor general’s report highlights the following major lessons for future projects (agq, 2010a, page 3): simplification of employment awards and agreements to remove complexities that affect the efficiency and effectiveness of payroll process establishing clear lines of accountability, roles and responsibilities to ensure end to end governance structure ensure assessment of full impact of system change and readiness of the business before going live and identify all risks and put in place robust contingency plans and risk management strategies. australasian journal of information systems volume 18 number 2 2014 157 figure 1. key causes of failure of the qld health payroll system australasian journal of information systems volume 18 number 2 2014 158 an independent review by ernst and young (2010) after extensive stakeholder consultation identified ‘ongoing people, process and technology issues with the current solution’ (page 2). its vendor analysis revealed that sap was an appropriate payroll solution for qld health. while it recommended to ‘stabilize and optimize’ the current system rather than replacing it, its central recommendation was the need for comprehensive support for subsequent project activities in the form of “change management, training planning and execution to support staff and minimize negative impacts, create consistent and broadly disseminated communication, and effective, targeted training and education” (page 3). similarly, kpmg (2010a; 2010b) recommended improved project governance and management, better system design and technology support and improved change management, including, user involvement throughout the process, integrated approach to communicate with diverse stakeholders and training. the report by ernst and young (2010, page 31) listed several critical success factors that should have been taken into account to ensure the smooth implementation of this project: well defined business requirements and clearly articulated statement of works; alignment with project management methodology; mature and consistent testing approach; comprehensive change management and training schedule; effective and timely communication; implementation in a controlled, staged manner; early establishment of conditions for stakeholder cooperation and expectation management; alignment of people, process and technology; availability of resources, including staff training and ongoing vendor management. pricewaterhousecoopers (pwc, 2010) which reviewed the shared services arrangement concluded that while all levels of government in australia have moved toward various models of shared services in line with western countries, its successful introduction within the public sector is ‘challenging’ due to increased complexity of stakeholder relationships, public ownership and scrutiny of government activities and lower appetite for risk. it recommended an “integrated operating model which combines the benefits of central control, scale and standardization with a flexible, service-centric approach” (page viii). the key recommendations for improvement as a result of the above independent reviews can best be summarized under the system development life cycle (sdlc) approach that classifies the key success factors under five distinct and iterative phases, namely, planning, analysis, design, implementation and maintenance/review. as presented in figure-2, before addressing the issues raised under the sdlc approach, an organization should conduct a thorough strategic review of its external and internal environment to identify the best possible solution. the case study clearly highlights two critical issues of technical project management, namely. user involvement and ongoing and integrated communication with diverse stakeholders throughout the process and as such, figure-2 presents them as being critical during all the five phases. australasian journal of information systems volume 18 number 2 2014 159 figure 2. the sdlc approach australasian journal of information systems volume 18 number 2 2014 160 discussion & conclusions considering that the hr technologies are predominantly used for hr administrative applications, including payroll and benefits administration (cedarcrestone survey, 2010), this case study has profound implications on the evaluation of their effectiveness. while transactional hr processing adds little value to an organization’s bottom line, failure to pay people on time and the right amount can cause immense employee dissatisfaction and distress which in turn can severely dent the employer branding. apart from the indirect cost of loss of employer reputation, the direct costs of rectifying the flaws of a new payroll system can run in to millions of dollars for a large organization. as we have seen in the case of qld health, the initial budget is likely to exceed by 2000% from $50 million to over $1 billion between 2010-2017. by any standards, this is an utter disaster and if qld health was a private enterprise, it’s very future would have been in doubt. an analysis of what went wrong by the government and independent agencies reveals exactly all the key lessons found in the literature on technology implementation: lack of clarity on business requirements, poor needs analysis, ill-defined project scope leading to major scope creep, confusion on roles and responsibilities between stakeholders, lack of user involvement in system design, inadequate risk assessment and planning for risk mitigation, poor project and change management, insufficient parallel pay run prior to going live, poor alignment between people, process and technology, lack of user training and inconsistent communication. in addition to the above, there were additional complexities arising out of the organizational profile and whole of government approach to it infrastructure management. the payroll process by its very nature is complex and challenging at qld health brought about by multiple employment awards. this complexity was completely ignored during the planning and design process as the prime contractor was asked to provide like-for-like replacement modeled on another much smaller government department. by adopting the system development life cycle (sdlc), figure-2 captures the key factors that should have been taken into account in designing, implementing and rolling out the new payroll system at qld health. it reconfirms the findings in another case study of a successful is implementation in a healthcare organization where the key success factors included link with organizational strategy, planned change with stakeholder consultations and collaboration, role and process transparency and accountability, ensuring training appropriateness and fostering continuous learning and improvement (abraham and junglas, 2011, p. 189). thus, the factors that contribute to the success or failure of project implementation are the same which reinforce their reliability and validity. the project should have begun with a thorough organizational environmental analysis and a strategic review of business needs. for example, an academic review of qld health payroll case recommends the integration of it design with organizational design to adequately respond to the dynamic and complex organizational environment (silva and rosemann, 2012). the agile methods to sdlc have certain distinct advantages in the context of this case study as they particularly focus on key nontechnical factors, such as diverse organizational cultures and collaborative work teams that emphasize shared responsibilities in order to be lean, flexible and customer-centric. however, some studies find that the superiority of agile methods have not been conclusively established and that they are not particularly suitable for large and complex projects (dyba and dingsoyr, 2008; awad, 2005) while others recommend integration of the traditional methods with agile methods to improve cost control, product functionality and on-time delivery (karlstrom and runeson, 2005). accordingly, figure-2 depicts both traditional and agile methods as complimentary rather than alternate approaches to sdlc for successful technical project implementation. the whole of government approach to it under the shared service center (ssc) concept added another layer of complexity. while ssc is a well accepted model in the western countries, its implementation australasian journal of information systems volume 18 number 2 2014 161 is ‘almost invariably been problematic’ in a public sector environment due to ‘systematic underestimation of the costs and/or risks of implementation caused by behavioral factors (such as) optimism bias by proponents at the whole-of-government level and fear of loss of control by players at the individual agency level’ (aim, 2011, page 2). in this case, the ssc entity owned the system and signed the key project deliverables with the contractor whereas the end user who owned the data and was responsible for payment to employees was kept out of the loop and it took over 20 months for the end user to sign on the project scope. the governance structure between the three key parties was conspicuous by its absence. this resulted in 47 change requests throwing the budget and implementation schedule into chaos. while the ssc approach to central control has its benefits, a large organization also requires flexibility and autonomy to effectively manage its internal complexity. the authors recognize that hindsight-bias is one of the weaknesses of the study. however, research suggests that use of multiple informants increases the reliability and validity of informant reports (kumar, stern and anderson, 1993) and as the study relies on multiple reports conducted over a considerable period of time by multiple agencies, it lends credibility to the findings reported in the study. the case study confirms the central message in the literature that it is not technology per se but the underlying social and organizational processes that mainly derail the implementation and cause the failure to realize the intended benefits. in this case, the vendor analysis by the independent agency revealed that the technologies chosen were sound, well tested and well suited for the organization’s requirements. as the cedarcrestone survey (2010) cautions, hr technologies cannot fix inadequate or broken processes. the case study highlights the importance of the social actors, their motives and interactions underpinned by the socio-technical view and the human agency perspective in minimizing the unintended consequences of technology implementation. references abraham, c. and junglas, i. (2011), ‘from cacophony to harmony: a case study about the is implementation process as an opportunity for organizational transformation at sentara healthcare’, journal of strategic information systems, 20, 177-197. accenture (2007). ‘managing shared services change: beyond communications and training’. https://microsite.accenture.com/sharedservices/research_and_insights/documents/120168_ma nagingshared_v9.pdf agq (2010a). report to parliament no. 7 for 2010. information systems governance and control, including the queensland health implementation of continuity project. auditor-general of queensland. queensland audit office, brisbane, australia. http://www.qao.qld.gov.au/files/file/reports/2010_report_no.7.pdf agq (2010b). report to parliament no. 13 for 2010. results of audits at 31 october 2010. auditorgeneral of queensland. queensland audit office, brisbane, australia. http://www.qao.qld.gov.au/files/file/reports/2010_report_no.13.pdf aim (2011), ‘shared services in the public sector: a triumph of hope over experience?’, green paper, september 2011, australian institute of management, north sydney. alge, b. j. and upright, k. b. (2012), ‘human resource information systems needs analysis’, in human resource information systems: basics, applications & directions, ed., m. j. kavanagh, m. thite, r. johnson. thousand oaks, ca: sage. awad, m. a. (2005), ‘a comparison between agile and traditional development methodologies’, unpublished dissertation, school of computer science and software engineering, the university of western australia. australasian journal of information systems volume 18 number 2 2014 162 barker, t. and frolick, m.n. (2003), ‘erp implementation failure: a case study’, information systems management, 20, 43-49. benders, j., hoeken, p., batenburg, r. , and schouteten, r. (2006), ‘first organise, then automate: a modern socio-technical view on erp systems and teamworking’, new technology, work and employment, 21, 242-251. bohmer, r. (2010). fixing healthcare on the frontlines, harvard business review, 88 (4), 62-69. boudreau, m.-c. and robey, d. (2005), ‘enacting integrated information technology: a human agency perspective’, organisation science, 16, 3-18. bowen, g. a. (2009), ‘document analysis as a qualitative research method’, qualitative research journal, 9(2), 27 – 40. cedarcrestone 2010-2011 hr systems survey. ‘hr technologies, service delivery approaches, and metrics, 13th annual edition’. http://www.euroinvestor.co.uk/news/story.aspx?id=11083093&bw=20100524006449 chang, , s., gable, g., smythe, e. and timbrell, g. (2000), ‘a delphi examination of public sector erp implementation issues’. in 21st international conference on information systems, 10-13 december 2000, brisbane, australia courier mail (2010a). ‘queensland health payroll debacle spread with overpayments deducted illegally’. may 27, 2010. courier mail (2010b). ‘queensland health payroll report from auditor kpmg reveals turmoil and incompetence’. may 10, 2010. courier mail (2010c). ‘state knew of queensland health payroll system risk: two bureaucrats axed’. june 29, 2010. danilewicz, d. (1997), ‘how technology has changed the payroll function’, management accounting, 78, 6-9. dennis, a. (1997), ‘the importance of payroll’, journal of accountancy, 184, 47-49. dery, k., hall, r., and wailes, n. (2006) , ‘erps as ‘technologies – in – practice’: social construction, materiality and the role of organizational factors’, new technology, work and employment , 21, 229-241. dodd, m. (2010). ‘task force to fix defence payroll’. http://www.theaustralian.com.au/news/nation/task-force-to-try-and-fix-defence-payroll/storye6frg6nf-1225826128569 doz, y. (2011), ‘qualitative research for international business’, journal of international business studies, 42, 582-590. dyba, t. and dingsoyr, t. (2008), ‘empirical studies of agile software development: a systematic review’, information and software technology, 50, 833-859. ehie, i.c. and madsen, m. (2005), ‘identifying critical issues in enterprise resource planning (erp) implementation’, computers in industry, 56, 545-557. ernst and young (2010). ‘queensland health review of payroll and rostering solutions’. ernst and young, australia. september, 2010. http://www.health.qld.gov.au/qhpip/docs/e_y_report_23nov.pdf fay, c. h. and nardoni, r. e. (2012), ‘performance management, compensation, benefits, payroll and the human resource information system’, in human resource information systems: basics, applications and directions, ed. m. j. kavanagh, m.thite, r. johnson, thousand oaks, ca: sage. australasian journal of information systems volume 18 number 2 2014 163 fisher, s.l. and howell, a.w. (2004), ‘beyond user acceptance: an examination of employee reactions to information technology systems’, human resource management, 43, 243-258 goh, m., prakash, s., and yeo, r. (2007), ‘resource-based approach to it in a shared services manufacturing firm’, industrial management and data systems, 107, 251-270. hunton, j.e. and price, k.h. (1997), ‘effects of the user participation process and task meaningfulness on key information systems outcomes’, management science, 43, 797-812. isenhour, l. c. (2012), ‘hr administration and hris’, in human resource information systems: basics, applications and directions , eds. m. j. kavanagh, m. thite, r. johnson, thousand oaks, ca: sage. jereb, c., kuchem, r., and sohn, w. (2009), ‘shed light on the black box: how deutsche post dhl achieved global transparency over payroll processes’, strategic human resource review, 8, 23-31. karlstrom, d. and runeson, p. (2005), ‘combining agile methods with stage-gate project management’, ieee software, 22(3), 43-49. klein, k. j. and ralls, r. s. (1997), ‘an exploration of the unintended organizational consequences of technology training: implications of practice for training theory and research’, in improving training effectiveness in work organizations , ed., j.k. ford, mahwah, nj: lawrence. kpmg (2012). ‘review of the queensland health payroll system’. kpmg, australia. 31 may 2012. http://delimiter.com.au/wp-content/uploads/2012/06/kpmg_audit.pdf kpmg (2010a). ‘queensland health payroll implementation review’. stage 1 status report. kpmg, australia. 8 may 2010. http://www.premiers.qld.gov.au/publications/categories/reviews/assets/health-payroll-reviewstatus-report.pdf kpmg (2010b). ‘queensland health payroll implementation review’. interim report – stage 2. kpmg, australia. 18 may 2010. http://www.premiers.qld.gov.au/publications/categories/reviews/assets/health-payroll-reviewstage2.pdf kumar, n., stern, l. and anderson, j. c. (1993), ‘conducting interorganizational research using key informants’, academy of management journal, 36(6), 1633-1651. lengnickhall, c. a. and lengnick-hall, m.l. (2006), ‘hr, erp, and knowledge for competitive advantage’, human resource management, 45, 179-194 light, b. and wagner, e. (2006), ‘integrating in erp environments: rhetoric, realities and organisational possibilities’, new technology, work, and employment, 21, 215-228. ma, q. and liu, l. (2004), ‘the technology acceptance model: a meta-analysis of empirical findings’, journal of organisational and end user computing, 16, 59-72. motwani, j., subramanian, r. , and gopalakrishna, p. (2005), ‘critical factors for successful erp implementation: exploratory findings from four case studies’, computers in industry, 56,529544. nerur, s., mahapatra, r. and mangalaraj, g. (2005), ‘challenges of migrating to agile methodologies’, communications of the acm, 48(5), 72-78. ngai, e.w.t. and wat, f.k.t. (2006), ‘human resource information systems: a review and empirical analysis’, personal review, 35, 297-314. nicolaou, a. i. and bhattacharya, s. (2006), ‘organisational performance effects of erp systems usage: the impact of postimplementation changes’, international journal of accounting information systems, 7, 18-35 australasian journal of information systems volume 18 number 2 2014 164 orlikowski , w.j. (2000), ‘using technology and constitution structures: a practice lens for studying technology in organisations’, organizational science, 11, 404-428. orlikowski , w.j. and scott, s.v. (2008), ‘sociomateriality: challenging the separation of technology, work and organization’, the academy of management annals, 2(1), 433-474. phelan, s. e. and hayes, m. (2005), ‘payroll, a tie that binds’, journal of accountancy, 199, 29-35. pwc (2010), ‘review of the shared services model for queensland government’. pricewaterhousecoopers, brisbane, australia. september, 2010. http://www.premiers.qld.gov.au/publications/categories/reviews/assets/shared-services-review30-sep.pdf richie, j. and spencer, c. (1994), ‘qualitative data analysis for applied policy research, in analyzing qualitative data, eds. a. bryman and r. g. burgess. taylor and francis. pp.173-194. rubin, b. m. (1986), ‘information systems for public management: design & implementation’, public administration review, 46, 540-552. silva, a. r. and rosemann, m. (2012), ‘integrating organizational design with it design: the queensland health payroll case’, in peffers, k, rothenberger, m. and kuechler, b. (eds.), desrist 2012, lncs 7286, 271-286. shrivastava, s. and shaw, j. b. (2003), ‘liberating hr through technology’, human resource management, 42, 201-222. stefanou, c. j. and revanoglou, a. (2006), ‘erp integration in a healthcare environment: a case study’, journal of enterprise information management, 19(1), 115-130. stone, d. l. and lukaszewski, k. m. (2009), ‘an expanded model of the factors affecting the acceptance and effectiveness of electronic human resource management systems’, human resource management review, 19, 134-143. the australian (2010a). ‘we delivered quality payroll solution to qld health’. april 23, 2010. the australian (2010b). ‘qld health payroll system in chaos: asu’. may 05, 2010. the australian (2010c). ‘queensland health to change payroll system again after bungle’. june 29, 2010. the pak banker (2010). ‘majority of americans live pay check to paycheck’. september 21, 2010. thite, m., kavanagh, m.j., and johnson, r. (2012), ‘evolution of human resource management and human resource information systems: the role of information technology’, in human resource information systems: basics, applications & directions, ed., m. j. kavanagh, m. thite, r. johnson. thousand oaks, ca: sage. trotter, a. (2007). ‘glitches in los angeles payroll system spark furor’. education week. 26, 1-23. varma s. and gopal r. (2011), ‘the implications of implementing electronic –human resource management (e-hrm) systems in companies’, journal of information systems communication, 2, 10-29 verville, j. and bernadas, c. (2005), ‘so you’re thinking of buying an erp? ten critical factors for successful acquisitions’ , journal of enterprise information management, 18, 665-677. waring, t. (2004), ‘communicating the complexity of integrating information systems: a case study of the procurement of a payroll-personal system’, international journal of human resource development and management, 4, 431. yin, r. k. (2003). case study research: design and methods. thousand oaks, ca: sage paper title (paper title style) australasian journal of information systems parker, burgess & al-qirim 2015, vol. 19, pp. s153-s173 review of sme journal articles a review of studies on information systems and smes in high ranked is journals (2000-2014) craig m. parker deakin business school deakin university craig.parker@deakin.edu.au stephen burgess college of business victoria university stephen.burgess@vu.edu.au nabeel al-qirim college of information technology united arab emirates university united arab emirates nalqirim@uaeu.ac.ae abstract this paper identifies novel approaches to future small and medium enterprise (sme) research from a review of articles, and then introduces the papers in this ajis special section which evidence these approaches. more specifically, the paper makes an important contribution by reviewing 61 articles in high ranked is journals (2000-2014) and introducing three new facets which are used to analyse research on sme adoption/use of is (units of analysis, sme sizes and sme types) not considered in previous literature review studies. these facets provide the basis for proposing various future research opportunities. the editorial then introduces the four papers in this special section covering the research theme on smes, and highlights the contributions they make using the three facets. keywords: information systems; small and medium enterprises (smes); units of analysis; literature review; high quality journals; sme sizes; sme types. 1 introduction small and medium enterprises (smes) are not just small versions of large organisations. they have unique characteristics with regard to their adoption and use of is, such as limited resources for implementing is and the dominance of owner-managers in is-related decisionmaking (cragg et al. 2011). small businesses in particular contribute between 40-50% of private sector employment in various countries (e.g. australian government 2014, federation of small businesses uk 2014, united states census bureau 2012). however, the number of sme-related articles published in leading is journals (see keller and coulthard 2013) is much less than the value contribution they make, so that continuing studies on smes are needed to advance the is discipline. it is important that sme research is undertaken which results in new theoretical and empirical directions to be taken up by scholars in the future. to that end, this editorial prefaces this ‘research theme on small and medium enterprises’ by identifying novel approaches to future sme research, and by showing these new approaches are evidenced by the papers in this ajis special section. this editorial therefore serves two aims which will advance future is research relating to smes. first, we review 61 articles on sme adoption/use of is in high quality ‘a*’ ranked is journals (see the list of journals in the appendix) from 2000 to 2014 according to three facets (units of analysis (uoas) they use, the sme types studied and sme sizes used) to identify opportunities for future research. earlier reviews of articles on is and smes have focused on other facets of articles. for instance, parker and castleman (2007) analysed four facets of articles from 2003 to 2006 (research approaches, types of is, and countries studied, and primary research objective), and powell and woerndl (2008) reviewed the research approaches used in is s153 australasian journal of information systems parker, burgess & al-qirim 2015, vol. 19, pp. s153-s173 review of sme journal articles articles on sme adoption/use of is from 1995 to 2005. this editorial makes a contribution to knowledge by conducting original research into three new facets, which were not analysed in these past reviews, and by identifying new research opportunities. in addition, it presents current insights into state of the literature when compared to past reviews, which are now seven or more years old. the second aim of the editorial is to introduce the four papers in this special section in ajis covering the research theme on smes. we highlight that each paper makes a contribution to knowledge through their use of novel uoas and/or their focus on specific sme types or sme sizes. this give the papers unique perspectives when compared to the a* journal articles reviewed in this editorial. the editorial is structured as follows. first, it justifies the approach used to select and analyse the a* is journal articles for review. then, it reports on each facet of the selected articles (the uoas employed and the sme types and sizes studied) in separate sections, including the future research opportunities revealed by this analysis. finally, the editorial introduces the papers included in this ajis special section covering the research theme on smes. 2 research approach our overall approach to this review was informed by schultze’s (2015) useful framework outlining a continuum of pluralistic literature review methods. in particular, we followed the principles associated with the ‘interpretive literature review’ end of this continuum to guide our article selection and analysis, as explained further in the next section. this was because the interpretive approach is most suited to identifying new future research opportunities (schultze 2015) and to dealing with complexities in literature reviews (boell and cecez-kecmanovic 2010), such as interpreting units of analysis (as explained later). 3 criteria employed in article selection twelve ‘a*’ ranked journals were identified from the australian council of professors and heads of information systems’ (acphis 2013) is journal ranking list. we chose this list because it was revised recently to evaluate the quality of international is journals, it subsumes the eight journals in the senior scholars’ basket of journals (ais 2011) into the ‘a*’ category of journals, and it resulted in a wider range of a* is journals to increase the scope for finding relevant articles (compared to the basket of eight) whilst retaining a focus on high quality journals. while the focus on a* is journals means the review is not exhaustive, is journal editorials and reviews often use a* is journals (e.g. sarker et al. 2013) because they are representative of high quality is research (shen et al. 2014). we investigated articles from 2000 to 2014 to cover a longer period than previous literature reviews. this meant we could identify many high quality articles, gauge the state of is research on sme types, sme sizes and the articles’ uoa used, and then identify future research opportunities. in order to ensure that we identified high quality is journal articles related to is use by smes, we searched the a* journals for articles whose title or abstract contained the terms “sme”, “smes”, “small” or “micro”. articles were removed if they used the terms in other contexts, such as an article that referred to a “small sample size” or one that referred to a “micro-level analysis”. we did not do a full-text search because we anticipated only articles with these terms in their titles/abstracts would have an sme focus. this resulted in 95 articles being selected initially. 4 approach to analysing the articles an interpretive, hermeneutic circle approach, similar to that proposed by schultze (2015) and boell and cecez-kecmanovic (2010) was used to enable new themes and future research opportunities relating to the three facets of the articles to emerge inductively. after initial cycles of reading, interpreting and coding the articles, the research aims and the methods used lead us to conclude the importance of the three facets focused on in this editorial: the uoa s154 australasian journal of information systems parker, burgess & al-qirim 2015, vol. 19, pp. s153-s173 review of sme journal articles used; the types of smes and sizes of smes examined. this hermeneutic process resulted in excluding articles which did not support our objective of exploring these three facets in articles focusing on the ajis research theme on sme adoption/use of is. the types of is studied by the articles included enterprise resource planning (erp) systems, electronic commerce systems, websites, anti-malware software, electronic marketplaces, decision support systems and software-as-a-service. some articles were excluded because they had little focus on smes. two of these developed models which provided little insight into sme-specific issues (chen and wu 2013, kauffman and mohtadi 2004). six included only one hypothesis (ada et al. 2012, bose and leung 2013, bose and pal 2012, kuk 2004) or research question (zhu et al. 2003) relating to company size, or controlled for firm size (baird et al. 2012), but again provided almost no sme-related insights nor examined sme-specific issues. however, we included studies which compared smes and large firms (e.g. daniel and grimshaw 2002, francalanci and morabito 2008) if there were clear analyses and insights focused on smes. nine articles purportedly focused on smes but provided no sme details to enable us to explore how they characterised sme types and sizes. for example, they did not specify sme sample selection criteria, characterise the firms as smes, or engage with the is literature on smes to characterise their sample (dahl and derigs 2011, delen et al. 2013, hung et al. 2011, lee and myers 2004, paolucci et al. 2002, tallon 2010). this was because the sme context appeared to be incidental to the article’s research aims. for example, tsatsou et al. (2010) focused on trust, vidgen (2002) focused on a website development methodology, and bandyopadhyay and bandyopadhyay (2009) focused on online auction results. we also excluded articles which were not relevant to our focus on sme adoption/use of is, as well as articles which studied is use by large businesses to manage sme clients (tsaih et al. 2004) or online user activities on websites (mccart et al. 2013), because these did not relate directly to sme adoption/use of is. additionally, we excluded 13 articles on sme software developer firms (e.g. ågerfalk and fitzgerald 2008, butler and murphy 2008, carlo et al. 2012, ceccagnoli et al. 2012, mehra et al. 2014) because the study in this editorial related to how smes in general adopt/use is. however, we included studies relating to (sme) software vendors (e.g. liang and xue 2004) if the article reported on any aspect of their relationship with sme clients. finally, we excluded a non-empirical literature review (powell and woerndl 2008) and an editorial (chau et al. 2007) relating to sme adoption/use of is because these articles did not enable us to explore the main facets of our interest, such as unit of analysis. table 1 summarises the 61 a* articles satisfying our selection criteria by publication year and journal (see the appendix for the journal names corresponding to the abbreviations). the table reveals interesting insights. first, it shows that a few journals published the majority of articles of interest (ejis and i&m, and to a lesser extent jsis, isj and jit), and that articles on smes and is adoption/use have appeared fairly consistently over the years. second, the table shows some journals in the 2000-2014 analysis period had no articles which matched our criteria for inclusion, as explained above. it should be noted, however, that the remainder of the editorial does not analyse the articles further by year because there are too few articles each year to comment on any yearly trends. further hermeneutic cycles of reading, interpreting and coding of the 61 articles were carried out by the first and second authors, consistent with the approach by schultze (2015) and boell and cecez-kecmanovic (2010), to understand how the three facets were manifested in the articles, and to develop emergent coding themes. the first author started the hermeneutic approach by reading the full text of all articles to identify initial coding themes relating to each facet. cycles then involved both authors: 1) discussing and refining the coding themes; 2) dividing and examining the articles using the refined coding themes; and 3) documenting the themes in a spreadsheet with articles in rows and coding themes in columns. cycles were undertaken until agreement was formed between the authors. further details of the coding themes and interpretations relating to each facet are expanded upon in later sections. s155 australasian journal of information systems parker, burgess & al-qirim 2015, vol. 19, pp. s153-s173 review of sme journal articles journal year 20xx total00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 acm 0 dss 1 1 ejis 1 1 2 2 2 1 1 3 2 1 1 17 i&m 1 2 2 4 1 1 1 1 1 14 i&o 1 1 2 isj 1 2 1 2 1 7 isr 0 jais 1 1 1 3 jaist 0 jit 1 1 1 1 2 1 7 jmis 0 jsis 1 1 1 1 2 1 1 8 misq 1 1 2 total 4 6 5 4 11 2 5 5 4 3 2 5 2 3 0 61 table 1: is a* journals and the number of articles on sme adoption/use of is each year note: blank cells are “0” and have been left blank to aid table readability. 5 facet 1: units of analysis used in a* is articles on smes 5.1 interpreting units of analysis (uoas) determining the unit of analysis (uoa) of a study is important because it helps to determine, for instance, the type of data to collect (singer 1961). various uoas can be used such as the individual, group, organisation, inter-organisation, industry, state/province, nation, event, government policy (singer 1961, yin 2014), artefact, social interaction (herring 2004, trochim 2006), activity or practice (nicolini 2012), as well as sentences in websites and social media content (herring 2004, parker et al. 2011). it must be noted that case studies of smes do not always mean that the uoa is the firm (or sme), because the firm examined in a case study can be the context for focusing on group interactions, events and other uoas (grunbaum 2007, yin 2014). table 2 lists and describes each uoa identified from the 61 articles. we anticipated that examining the uoas in the a* articles, and identifying those which have received little attention, could help to identify future research avenues. yin (2014) points out, however, that if authors do not explicitly state their uoa, readers often come to different conclusions. indeed, table 2 shows that 51 articles did not explicitly state their uoa. in these instances we inferred uoas using the hermeneutic circle approach of repeated reading and interpretation of each article’s research objectives, research questions and/or hypotheses. this often led us to lean toward, for instance, two uoas such as the firm or process/activity for an article. in some cases we settled on the is process/activity as the uoa when the research objectives and questions/hypotheses focused on a process/activity and used smes as case studies. however, there were articles where the distinction was still not clear. in these instances, the findings and, if available, survey and interview questions, were examined. where the majority of these focused on the process/activity (as in our example), this was selected as the uoa. s156 australasian journal of information systems parker, burgess & al-qirim 2015, vol. 19, pp. s153-s173 review of sme journal articles uoa explicit not stated uoa description external to firm vendor-sme relationship 1 4 nature of the relationship between an sme and is vendor-consultant(s) such as delegation, knowledge sharing, etc. industry 1 is-related events, discourse, etc relating to the sme sector within or external to the industry. cluster 1 a location-specific cluster and is-related events, governance, knowledge sharing, etc relating to sme members. intervention 1 is-related aims, management, success, etc of an intervention targeting the sme sector or a specified group of smes. e-marketplace 1 socio-technical factors (e.g. ownership, management) of an e-marketplace (involving smes) explaining its success/failure. firm-level firm 4 26 impact of is, including internal and external circumstances, which influence sme: is (non-)adoption; or owner-manager views on the ability of the firm to achieve firm-level benefits from is. process/daily activity 12 nature of a process/daily activity relating to is or influenced by is in an sme, excluding the firm-level impact of the process/activity. is capability 2 nature or development of an is-related resource (capability, skill, competency) in an sme, excluding the firm-level impact. is artefact 2 design of artefacts (e.g. is architectures, website, social media) used by smes, excluding the firm-level impact of these artefacts. group 1 team/group dynamics in an sme on is-related matters. sentence 1 sentence level analysis (e.g. text, images, videos) in an sme’s is artefact, report output by is, etc (excluding artefact design level). multiple uoas combination 1 3 process and vendor-sme relationship combination 2 1 (firm) firm and process total 10 51 table 2: units of analysis used in is a* journals 5.2 findings from the analysis of uoas table 2 shows the most common uoa was the sme or firm, with five articles stating this explicitly. four of the articles examined factors affecting the firm’s adoption (li et al. 2011, mackay et al. 2004, mehrtens et al. 2001, reardon and davidson 2007) and use of is (li et al. 2011, reardon and davidson 2007). in other words, these articles investigated, and included firm-level findings and data collection questions about, the (anticipated) positive and/or negative impact of is on various aspects of the firm. the fifth article, included as combination 2 under multiple uoas, explicitly stated the firm as its uoa was pob-nzaou and raymond (2011). we examine this article in more detail later because our analysis suggests they combined the firm uoa with a second unstated uoa (is process). table 2 shows the remaining a* articles without explicit statements about their uoa included 26 with a firm uoa (31 articles in total). many of these were adoption factor studies (e.g. caldeira and ward 2002, eikebrokk and olsen 2007, lee and larsen 2009, quaddus and hofmeyer 2007) like the articles with an explicit firm uoa. but other studies examined the impact of business-it alignment (bergeron et al. 2004, cragg et al. 2002), is integration (francalanci and morabito 2008), it capability (zhang et al. 2008; 2013), software-as-as157 australasian journal of information systems parker, burgess & al-qirim 2015, vol. 19, pp. s153-s173 review of sme journal articles service use (lee et al. 2013) and is in general (rivard et al. 2006) on firm performance. others looked at is/it investment and impact on firm-level costs, benefits and risk (love and irani 2004, love et al. 2005). our analysis in table 2 shows that the next most common uoa inferred from 12 a* articles without explicit statements of their uoa was an is process or a daily activity within smes: • is strategy formulation process (duhan et al. 2001, levy and powell 2000, ray and ray 2006, walters et al. 2003); • is adoption process (molla et al. 2006); • is/vendor selection process (chang et al. 2012, olsen and sætre 2007); • is implementation process (newman and zhao 2008, ray and ray 2006); • improvements or changes to daily activities using is (choi et al. 2004, harwood 2011, levenburg and klein 2006); and • is governance process and activities (huang et al. 2010). these articles focused on describing or exploring smes’ processes or daily activities, not the broader impact which these processes/activities had at the firm-level. some reported on the internal and/or external issues which influenced these processes/activities (molla et al. 2006, newman and zhao 2008, olsen and sætre 2007, ray and ray 2006), but these firm-level issues were used to provide context for understanding, in detail, the process/activity. the firm and process/daily activity uoas can be considered firm-level uoas or, in other words, looking at the firm or a narrower aspect of the firm. table 2 shows there were a range of other firm-level uoas stated explicitly in, or inferred from, other articles: • group: the case study by street and meister (2004) reports on how the management team in a small firm handled is-related change in response to planned firm growth. • the is capabilities which smes can develop (cragg et al. 2011), with another article examining how smes developed these is capabilities (caldeira and ward 2003). these capabilities do have implications for firm-level success, but this aspect was not the focus of these two articles. indeed, caldeira and ward (2003) stated this aspect had been examined in their earlier work (caldeira and ward 2002); • the is artefact, which related to the design of smes’ is architectures (bidan et al. 2012) and contents of their websites (merono-cerdan and soto-acosta 2007). the dominant focus of these articles was on the is architectures and websites, although a second uoa of the latter could also be considered the firm-level because they examined the impact of website content on overall firm performance; • sentence: alonso-mendo et al. (2009) investigated the reasons sme ownermanagers stated for changes the researchers observed in the smes’ websites over time, and characterised their uoa as “website change reason sentences”. the vendor-sme relationship uoa was the next most common in table 2. it was evident in only five articles, and is the first example of a uoa which bridges the firm-level and external context of the sme. three of these articles, without an explicit uoa statement, explored intermediary e-business models to encourage smes to engage in e-commerce (brown and lockett 2004), a non-government organisation’s attempt to develop is solutions for sme farmers (dobson et al. 2013), and an erp vendor’s methods of engaging with sme clients (liang and xue 2004). the fourth non-explicit article, mcgovern and hicks (2004), was similar because it examined the power relationships between the authors, who acted as is consultants, and the owner-manager of a small business during an action research project. only currie et al. (2004) explicitly stated their uoa as the business model between vendors and smes, which we labelled vendor-sme relationship. they explored the business models s158 australasian journal of information systems parker, burgess & al-qirim 2015, vol. 19, pp. s153-s173 review of sme journal articles (strategic positioning, product/service portfolios and value propositions) offered by application service provider (asp) vendors to sme clients. table 2 shows there were a range of other uoas stated explicitly in, or inferred from, other articles which focused on examining areas outside the context of a single sme: • intervention: vega et al. (2008) reported on a policy intervention aimed at encouraging sme adoption of e-business, and aimed to understand the mechanisms and effectiveness of the intervention itself. • industry: currie (2004) examined the asp industry and how its marketing discourse during the late 1990s and early 2000s affected the adoption of asp by smes. • e-marketplaces (websites facilitating sme-to-business/government/consumer trade) and the socio-technical factors (e.g. ownership structure, profile of the rural regional locations) influencing their success and failure (gengatharen and standing 2005). we did not consider the uoa to be the is artefact because the article did not focus on the design, features or functionality of the e-marketplace websites (see table 2), but instead focused on socio-technical factors associated with managing e-marketplaces. • is-enabled clusters involving smes and the communication, intellectual property and research collaboration issues between stakeholders in these clusters (jaegersberg and ure 2011). we did not consider the uoa in this article to be the industry-level, because the clusters were location specific. thus, the focus is more on the location than the ‘is’ itself. an industry-level analysis, as in the case of currie (2004), explores the entire industry at the national or international level. the difficulties of inferring uoas was notable for three articles because they reported in-depth findings relating to two uoas (adam and o'doherty 2000, howcroft and light 2006; 2010). they all described in-depth an is selection or implementation process, but also examined in some detail aspects of vendor-sme relationships associated with these processes. similarly, a fourth article, pob-nzaou and raymond (2011), stated that their uoa was the firm-level, but the article mainly detailed the activities within the erp implementation process of four smes. an alternative interpretation, therefore, is that the uoa in this article was the process-level and that the internal firm-level and external influences they examined contextualised the execution of the processes. 5.3 uoa-related future research opportunities while our interpretation of the articles’ uoas is subjective, our analysis makes three important contributions. first, it suggests the need for article authors to have clear statements of and rationale for their uoa(s) so that it is not left to interpretation. second, it provides the starting point for a scholarly debate on what constitutes different uoas in the context of sme adoption/use of is, including the notion that an article could potentially have more than one uoa. third, figure 1 illustrates the interrelationships between firmand non-firm-level uoas which we identified above, and highlights areas requiring further research; the darker the shading, the greater the opportunity because fewer a* articles have used the particular uoas. s159 australasian journal of information systems parker, burgess & al-qirim 2015, vol. 19, pp. s153-s173 review of sme journal articles figure 1: interrelationships between units of analysis in the context of sme adoption/use of is figure 1 suggests there are various future research opportunities arising from firm-level uoas. for example: • our analysis shows there is a lack of research on post-implementation is activities by smes and how these change over time. harwood (2011) was one of few a* studies in this area. it shows the value of such research because it provides insights into the complex socio-historical issues which shape is activities and how is (fails to) become embedded or ‘invisible’ in sme daily work, which is so far not well understood. such studies could also explore in detail how sme is capabilities evolve/develop over time. • there are opportunities for research into the design, architectures, content, etc of is artefacts used by smes. this could include exploring information outputs of their is, whether it is used and understood by sme owner-managers, whether it is in a form suitable for their use, etc. another example is studying the content of sme social media presences (including client postings) to examine the effectiveness of use. figure 1 illustrates that smes can have marketplace relationships with third-party firms and they can use inter-organisational is artefacts, and that third-parties can engage in is-related processes/activities with smes or initiate interventions to support smes with is. additionally, it shows that smes operate within industry and government level contexts which can influence perceptions and use of is. whilst this is not new and is well acknowledged in the is literature on smes, our analysis of the a* articles adds that these can be uoas in their own right and the source of future research opportunities. for example: • there is emerging, but still limited, in-depth analysis of interactions between smes and is vendors-consultants (vcs) during is (post-)implementation. for example, future studies can explore how (post-)implementation processes/activities of vcs help smes develop, or obviate the need to develop, is capabilities when vcs sustain ongoing relations with smes, and with various groups (e.g. management) in larger smes. this can include, as per howcroft and light (2010), observing (and interviewing) vcs, and treating the sme-vendor relationship as the uoa. another example is exploring the processes/activities used by vcs to gather sme information needs for, and design/develop, packaged software (e.g. erp systems) s160 australasian journal of information systems parker, burgess & al-qirim 2015, vol. 19, pp. s153-s173 review of sme journal articles for sme markets to determine, for instance, possible improvements to these processes and designs of their is artefacts. • there are opportunities to build on the work of vega et al. (2008) to explore indepth the design, execution and impact of is-related interventions initiated at government and industry levels. for instance, future research can explore the impact of ‘green’ is/it regulations (parker and scheepers 2012) to explore the impact on the sme sector (e.g. their it purchasing decisions). other examples include examining the impact of government, industry and trading partner led isrelated information campaigns on sme behaviours, including the impact of different designs (e.g. types of messages, media used, etc). • the work of currie (2004) highlights the opportunity to explore how is-related industry norms evolve and become known to and practiced by smes. for example, it is unclear in the a* articles how best practices are selected and embedded in erp systems, whether these are based on the practices of large firms rather than those of smes, and how these industry-level norms affect innovation in the sme sector. 6 facet 2: sme types studied in a* is articles we analysed the types of smes studied in the a* is articles by examining, for instance, the industries studied to identify future research opportunities, as shown in table 3. industries total percent business focus within an industry total percent multiple 29 47.5% export-focus 2 3.3% two 4 6.6% business growth 1 1.6% three 5 8.2% family-run 1 1.6% one 23 37.7% none 57 93.5% total 61 100.0% total 61 100% table 3: sme types studied in a* is articles table 3 shows we found that nearly half of the articles studied smes across a range of industry sectors (e.g. khazanchi and sutton 2001, kuan and chau 2001, levy et al. 2003, levy et al. 2001, poon 2000, riemenschneider et al. 2003). very few of these articles (exceptions included lee and larsen 2009, love et al. 2005, quaddus and hofmeyer 2007, riemenschneider and mykytyn jr 2000) examined the impact of sme industry on sme adoption, use or experiences of is, and instead researched their aims without considering the impact of industry. those that did compare industries quantitatively reported differences among the industries regarding aspects such as it investment (love et al. 2005) and businessto-business trading exchange adoption (quaddus and hofmeyer 2007). table 3 also shows that four articles examined two broad industry sectors: • manufacturing and oil (olsen and sætre 2007); and • manufacturing and service (bergeron et al. 2004, francalanci and morabito 2008, pob-nzaou and raymond 2011). a further five articles studied three broad sectors: • manufacturing, service and one each of the following sectors: o not-for-profit (huang et al. 2010), o retail (lee et al. 2013, merono-cerdan and soto-acosta 2007) or o trade (kendall et al. 2001); and • transport, tourism and food-beverage (eikebrokk and olsen 2007). s161 australasian journal of information systems parker, burgess & al-qirim 2015, vol. 19, pp. s153-s173 review of sme journal articles interestingly, merono-cerdan and soto-acosta (2007) was the only article among those examining three industries which quantitatively examined the impact of industry, and found some differences in website design by industry. this article, and those cited earlier, highlight the need for future research to investigate whether there are industry-specific idiosyncrasies among smes with respect to the uoa research opportunities identified above, such as their response to is-related interventions or the evolution of ‘best’ is practices. finally, the existence of industry-specific idiosyncrasies is emphasised further by the observation that the remaining 23 articles focused on a specific industry sector. eight articles studied sme manufacturers (caldeira and ward 2002; 2003, choi et al. 2004, cragg et al. 2002, hussin et al. 2002, mcgovern and hicks 2004, street and meister 2004, walters et al. 2003, zhang et al. 2008). some articles examined similar industry sectors such as engineering (cragg 2002), construction (love and irani 2004) and solar energy (jaegersberg and ure 2011). the remaining industries were career management (howcroft and light 2006; 2010), farming (dobson et al. 2013), knowledge-based (duhan et al. 2001), hotel (harwood 2011), not-for-profit (mackay et al. 2004), wine (molla et al. 2006), retail (newman and zhao 2008), sporting (ray and ray 2006) and medical physician (reardon and davidson 2007) smes. in many cases, authors emphasised unique characteristics of smes in their chosen industry relative to other industries. for example, mackay et al. (2004) points out that small not-forprofit businesses have unique dilemmas compared to for-profit firms which may result in different is uses, such as recruiting/servicing volunteers and competing for grants/donations, in addition to providing (selling or donating) their products/services to beneficiaries. when combined with the uoa future research opportunities above, this raises interesting questions such as whether their everyday is activities, interactions with third-parties to develop is capabilities and is artefact use are different to other smes. interestingly, three of the 29 articles examining various industry sectors focused upon specific types of smes irrespective of their industry: ‘born global’ and export-focused smes (zhang et al. 2013); smes focused on business growth (levy and powell 2000) and family-run smes (wang and ahmed 2009). similarly, an additional article examining a single manufacturing sector narrowed the focus further to export-focused smes (zhang et al. 2008). these sme types were justified in the articles on the basis they were more pertinent to the articles’ research goals (levy and powell 2000, zhang et al. 2013), or because the articles argued these smes had characteristics differentiating them from other smes (wang and ahmed 2009). given the heterogeneous nature of smes, future research can explore these and other idiosyncrasies (e.g. home-based businesses) to determine if these sme types use and experience is differently to other types of smes. these sme types could also be considered within the context of uoa research opportunities identified earlier. for example, do the everyday is activities of homebased smes differ manifestly from those of otherwise similar smes working out of dedicated business premises? how do is capabilities or is artefacts evolve and develop as smes grow? 7 facet 3: sme sizes studied in a* is articles table 4 summarises the a* articles in terms of how they characterised the sme sizes they examined. the table shows that over half the articles focused on smes broadly, while the next common size was small business. table 4 also reports on the types of sme size stated in the articles (e.g. staff numbers) determined hermeneutically by analysing sme definitions or descriptions in article introductions and method sections (e.g. sample selection criteria). s162 australasian journal of information systems parker, burgess & al-qirim 2015, vol. 19, pp. s153-s173 review of sme journal articles sme sizes studied total percent size definition/description types total percent smes and large 2 3% precise staff numbers and revenue 6 9.8% small, medium and large 1 2% precise staff numbers 36 59.0% smes 35 57% precise revenue or assets 1 1.6% medium 1 2% imprecise staff numbers 2 3.3% small and medium 2 3% industry specific size definition 2 3.3% micro, small and medium 2 3% size evident only from case overview 4 6.6% small 16 26% smes size described in generic terms 10 16.4% micro and small 1 2% micro 1 2% sole-operators 0 0% total 61 100% total 61 100% table 4: sme sizes studied and size definition/description types used in a* is articles our analysis suggests that a major issue facing is scholars is the various sme size definitions. 44 articles used staff numbers. others (adam and o'doherty 2000, alonso-mendo et al. 2009, bergeron et al. 2004, cragg et al. 2011, kendall et al. 2001, molla et al. 2006) combined staff numbers with, or solely (chang et al. 2012) used, annual turnover and/or level of assets. most articles used previous studies or regional definitions to derive upper limits on staff numbers and/or revenue, but this resulted in inconsistencies. for example, the upper limit for smes was typically 450-500 (15 studies) or 249/250 staff (8 studies), reflecting north american/asian or european union (eu) focus (respectively). similar differences were evident for small and micro firms: some articles defined small firms as up to 100 staff (9 articles) and others up to 50 (alonso-mendo et al. 2009, levenburg and klein 2006); and micro firms having up to 10 staff (3 articles) and another up to 50 staff (molla et al. 2006). the challenge of varying definitions was notable in cragg et al. (2011) who defined smes in usa, portugal and new zealand as less than 500 staff; and a large firm in new zealand using the definition of less than 200 staff from other articles (mehrtens et al. 2001). some articles were imprecise because they did not specify an upper limit (merono-cerdan and soto-acosta 2007, zhang et al. 2008) to clarify if large firms were omitted, while others had no definition and described their case studies with ‘staff’ numbers (duhan et al. 2001, howcroft and light 2006, mcgovern and hicks 2004, street and meister 2004). these issues make cross-national comparisons difficult both across studies, and with future research. further difficulties with comparisons occurs with the 10 articles noted in table 4 without a clear sme size definition, and instead only characterising smes or small firms in more general terms. it is important to note that staff numbers may be problematical in industries with a large casual or seasonal labour (al-qirim 2005, burgess et al. 2009). this was evident in two articles which used an industry-specific size definition: number of rooms for hotels (harwood 2011) and number of medical physicians without mention of other staff (reardon and davidson 2007). another issue which emerged was that some articles included subsidiaries of large firms as smes (adam and o'doherty 2000, huang et al. 2010, olsen and sætre 2007, pob-nzaou and raymond 2011), others excluded them specifically (cragg 2002, cragg et al. 2011, eikebrokk and olsen 2007, mackay et al. 2004, reardon and davidson 2007, street and meister 2004, walters et al. 2003, wang and ahmed 2009), and most articles did not confirm if subsidiaries were included. only two studies (cragg et al. 2002, hussin et al. 2002) compared independent and subsidiary smes and concluded the results were consistent, which means there is insufficient empirical evidence to conclude whether they are equivalent. this analysis of the articles raises implications for future research. first, we implore is scholars to include clear sme size definitions, including lower/upper staff numbers and (if relevant) revenue limits, and whether subsidiaries are included to enable comparisons of future research. second, our analysis suggests that future research can explore the feasibility of consistent size definitions between nations. for example, is there evidence that smes up to 500 staff in the usa are equivalent to smes up to 200 in new zealand due to national context? s163 australasian journal of information systems parker, burgess & al-qirim 2015, vol. 19, pp. s153-s173 review of sme journal articles or are smes up to 200 staff (or micro firms up to 10, etc) similar regardless of national context? third, future research can explore if independent smes are different, or equivalent, to subsidiaries. fourth, research can potentially determine industry-specific differentiators which better describe sme size (instead of or in addition to staff numbers) and the impact of these on sme uoas such as is activities, is artefacts, etc. another insight from table 4 is that few a* is articles contrasted their findings by sme size, which is often a categorical variable comprising micro, small and/or medium sized firms (e.g. alonso-mendo et al. 2009, levenburg and klein 2006) or continuous variable (e.g. lee and larsen 2009). this is important because sme size is one dimension on which heterogeneous smes can be differentiated. while some studies using sme size as a categorical (alonsomendo et al. 2009) or continuous variable (lee and larsen 2009) did not find statistical differences, others did find differences which suggest further research is warranted. for example, levenburg and klein (2006) found significant differences between micro and medium firms regarding use of some is applications. others treated sme size as a continuous variable and found that firm size explained significant differences between sme adopters and non-adopters of e-commerce (grandon and pearson 2004) and between smes gaining some strategic benefits from it and being willing to adapt to changes resulting from it (love et al. 2005). there is no consensus and this implies that there is potential for future research to identify peculiarities relating to how smes of different sizes adopt and/or use is, and to compare findings relating to size to understand better differences between different sized firms. 8 introduction of articles in the ajis special section the papers in this ajis special section offer diverse insights into sme adoption or use of different is, across various countries, and using different theories and methodologies. by shedding more light on the idiosyncratic features of the sme sector with respect to is, it is evident from the papers that smes experience success with is, but also face many challenges. the hurdles are in part due to fast-paced technological change, but also due to other issues specific to smes which can overwhelm an owner-manager and/or their venture. the papers introduced in this research theme on smes provide a broad picture about such advantages and challenges concerning is and smes. additionally, we state how they contribute to the future research areas identified above. in the first paper, kreuzer, born and bernius shed light on improving sme adoption of interorganisational is (iois) by exploring the under-studied area of firms’ non-coercive persuasion strategies using a methodologically novel controlled experiment examining the outcome of a large german firm using information and expertise support with its sme business partners. the authors state their uoa is the sme partners, and address a knowledge gap relating to interventions we identified in a* is articles. the initial analysis of 203 smes showed no significant result concerning the impact of the strategy on adoption. a cluster analysis, however, found two adopter configurations which varied in their response to the strategy, thereby addressing knowledge gaps we found relating to sme size (using the eu definition of less than 10, 50 and 250 for micro, small and medium enterprises respectively) and sme type (micro-sized service providers in a community-based configuration, and a dyadic configuration of medium-sized parts manufacturers). their findings suggest the success of providing information and expertise to sme business partners about an iois to foster adoption may succeed or fail depending on adoption configurations of the smes. the findings support adopter configurations as an alternative conceptual view of adoption than that used in prior studies, which examine factors influencing individual sme owner-managers’ iois adoption decisions without considering the firm’s adopter configuration context. the second paper by bradshaw, cragg and pulakanam explores how consultants affect the it knowledge of smes when consultants and smes managers interact by exploring relations between it consultants and smes, using the new zealand size definition of less than 50 staff, during it implementation projects. the authors thus address a uoa knowledge gap relating s164 australasian journal of information systems parker, burgess & al-qirim 2015, vol. 19, pp. s153-s173 review of sme journal articles to sme-vendor (i.e. consultant) relationships which we identified in a* is articles by interviewing it consultants and sme staff with knowledge of the project. further, the paper sheds light on the knowledge gap we report concerning how smes develop is capabilities by exploring the processes by which it knowledge is acquired by smes, shared between smes and consultants, the barriers to knowledge transfer, and the types of knowledge that consultants help smes to create. another important contribution of the paper is modifying an existing theoretical framework of knowledge sharing mechanisms to include three types of knowledge assets that were shared between consultants and sme clients during it implementation projects. salim, sedera, sawang, alarifi and atapattu, in the third paper, point out that most is literature treats sme adoption of is as a single activity when examining the impact of adoption factors, rather than as a multi-stage process whereby each factor in each adoption stage will have a different level of importance. the paper explores this notion in a survey of 162 malaysian smes (using the size definition of less than 200 full-time staff for manufacturers, and 75 staff for other sectors) across various industry sectors using the theory of planned behaviour to explore factors which influence two stages of cloud-based enterprise resource planning (erp) adoption: evaluation and trial. the authors state that the uoa for this study was the sme owner-manager, because they represented the views of the firm as a whole. the study sheds light on a knowledge gap we identified in is a* articles relating to is processes/activities by finding that the factors (attitude, subjective norms and perceived behavioural control) influencing the intention to adopt cloud-erp vary significantly across the two adoption stages. in the final paper, bi, davison and smyrnios argue that the ability of smes to gain it business value depends on how they employ it resources to develop it capability, because it facilitates partnership processes along inter-organisational value chains. the paper focuses on fastgrowth, independent (non-subsidiary) smes across multiple industries with turnover greater than 500,000 aud (and not greater than 50% of sales from one customer) and less than 200 full-time staff. the authors address a knowledge gap we identified in a* is articles relating to sme type based on the argument that fast-growth smes are different to other smes because they are risk takers, more entrepreneurial, and active in leveraging their it resources. the paper uses specific sme size criteria (i.e. staff numbers and revenue, excluding subsidiaries) which will help future is scholars compare their findings against this study. the paper tests and finds support for a research model linking it resources, it capabilities, it-enabled interfirm processes and firm performance, based on the resource-based view theory, from their survey of 310 fast-growth smes (i.e. firm-level uoa). the paper thus addresses another knowledge gap relating to is capabilities which we identified in a* is articles. the editors of this ajis special section, covering the research theme on smes, commend these papers to the readers of ajis. 9 acknowledgements we would like to thank the authors of the papers in this ajis special section, and all the reviewers of these papers, for their diligence and patience. without their valuable work, there would be no special section. we would also like to thank ajis associate editor, rosemary stockdale, and the two anonymous reviewers, for their valuable feedback which helped us improve this editorial. references acphis (2013) is journal ranking: rank order, australian council of professors and heads of information systems. available from http://www.acphis.org.au/index.php/isjournal-ranking/rank-order [accessed 19 may 2015]. ada, s., sharman, r. & balkundi, p. (2012) "impact of meta-analytic decisions on the conclusions drawn on the business value of information technology", decision support systems, 54(1): 521-533. doi: 10.1016/j.dss.2012.07.001 s165 australasian journal of information systems parker, burgess & al-qirim 2015, vol. 19, pp. s153-s173 review of sme journal articles adam, f. & o'doherty, p. (2000) "lessons from enterprise resource planning implementations in ireland – towards smaller and shorter erp projects", journal of information technology, 15(4): 305-316. doi: 10.1080/02683960010008953 ågerfalk, p.j. & fitzgerald, b. (2008) "outsourcing to an unknown workforce: exploring opensourcing as a global sourcing strategy", mis quarterly, 32(2): 385-409. ais (2011) senior scholars' basket of journals, association for information systems. available from http://aisnet.org/?seniorscholarbasket [accessed 19 may 2015]. al-qirim, n.a.y. (2005) "an empirical investigation of an e-commerce adoption-capability model in small businesses in new zealand", electronic markets, 15(4): 418-437. doi: 10.1080/10196780500303136 alonso-mendo, f., fitzgerald, g. & frias-martinez, e. (2009) "understanding web site redesigns in smalland medium-sized enterprises (smes): a u.k.-based study on the applicability of e-commerce stage models", european journal of information systems, 18(3): 264-279. doi: 10.1057/ejis.2009.14 australian government (2014) small business data card, the treasury, australian government, canberra. available from http://www.treasury.gov.au/publicationsandmedia/publications/2012/sml-bus-data [accessed 29 september 2015]. baird, a., furukawa, m.f. & raghu, t.s. (2012) "understanding contingencies associated with the early adoption of customer-facing web portals", journal of management information systems, 29(2): 293-324. bandyopadhyay, s. & bandyopadhyay, s. (2009) "estimating time required to reach bid levels in online auctions", journal of management information systems, 26(3): 275-301. bergeron, f., raymond, l. & rivard, s. (2004) "ideal patterns of strategic alignment and business performance", information & management, 41(8): 1003-1020. doi: 10.1016/j.im.2003.10.004 bidan, m., rowe, f. & truex, d. (2012) "an empirical study of is architectures in french smes: integration approaches", european journal of information systems, 21(3): 287-302. doi: 10.1057/ejis.2012.12 boell, s.k. & cecez-kecmanovic, d. (2010) "literature reviews and the hermeneutic circle", australian academic & research libraries, 41(2): 129-144. doi: 10.1080/00048623.2010.10721450 bose, i. & leung, a.c.m. (2013) "the impact of adoption of identity theft countermeasures on firm value", decision support systems, 55(3): 753-763. bose, i. & pal, r. (2012) "do green supply chain management initiatives impact stock prices of firms?", decision support systems, 52(3): 624-634. doi: 10.1016/j.dss.2011.10.020 brown, d.h. & lockett, n.j. (2004) "potential of critical e-applications for engaging smes in e-business: a provider perspective", european journal of information systems, 13(1): 21-34. doi: 10.1057/palgrave.ejis.3000480 burgess, s., sellitto, c. & karanasios, s.s. (2009) effective web presence solutions for small businesses: strategies for successful implementation, information science reference, hershey, pennsylvania. butler, t.b. & murphy, c.m. (2008) "an exploratory study on is capabilities and assets in a small-to-medium software enterprise", journal of information technology, 23(4): 330344. doi: 10.1057/jit.2008.19 caldeira, m.m. & ward, j.m. (2002) "understanding the successful adoption and use of is/it in smes: an explanation from portuguese manufacturing industries", information systems journal, 12(2): 121-152. doi: 10.1046/j.1365-2575.2002.00119.x s166 australasian journal of information systems parker, burgess & al-qirim 2015, vol. 19, pp. s153-s173 review of sme journal articles caldeira, m.m. & ward, j.m. (2003) "using resource-based theory to interpret the successful adoption and use of information systems and technology in manufacturing small and medium-sized enterprises", european journal of information systems, 12(2): 127-141. doi: 10.1057/palgrave.ejis.3000454 carlo, j.l., lyytinen, k. & rose, g.m. (2012) "a knowledge-based model of radical innovation in small software firms", mis quarterly, 36(3): p865-a810. ceccagnoli, m., forman, c., huang, p. & wu, d.j. (2012) "cocreation of value in a platform ecosystem: the case of enterprise software", mis quarterly, 36(1): 263-290. chang, s.-i., yen, d.c., ng, c.s.-p. & chang, w.-t. (2012) "an analysis of it/is outsourcing provider selection for smalland medium-sized enterprises in taiwan", information & management, 49(5): 199-209. doi: 10.1016/j.im.2012.03.001 chau, p.y.k., kuan, k.k.y. & liang, t.-p. (2007) "research on it value: what we have done in asia and europe", european journal of information systems, 16(3): 196-201. doi: 10.1057/palgrave.eijs.3000666 chen, p.-y. & wu, s.-y. (2013) "the impact and implications of on-demand services on market structure", information systems research, 24(3): 750-767. doi: 10.1287/isre.1120.0451 choi, h.r., kim, h.s., park, b.j., park, y.j. & whinston, a.b. (2004) "an agent for selecting optimal order set in ec marketplace", decision support systems, 36(4): 371-383. doi: 10.1016/s0167-9236(03)00027-7 cragg, p.b. (2002) "benchmarking information technology practices in small firms", european journal of information systems, 11(4): 267-282. cragg, p.b., caldeira, m.m. & ward, j.m. (2011) "organizational information systems competences in small and medium-sized enterprises", information & management, 48(8): 353-363. doi: 10.1016/j.im.2011.08.003 cragg, p.b., king, m. & hussin, h. (2002) "it alignment and firm performance in small manufacturing firms", the journal of strategic information systems, 11(2): 109-132. doi: 10.1016/s0963-8687(02)00007-0 currie, w.l. (2004) "the organizing vision of application service provision: a process-oriented analysis", information and organization, 14(4): 237-267. doi: 10.1016/j.infoandorg.2004.07.001 currie, w.l., desai, b. & khan, n. (2004) "customer evaluation of application services provisioning in five vertical sectors", journal of information technology, 19(1): 39-58. doi: 10.1057/palgrave.jit.2000006 dahl, s. & derigs, u. (2011) "cooperative planning in express carrier networks an empirical study on the effectiveness of a real-time decision support system", decision support systems, 51(3): 620-626. doi: 10.1016/j.dss.2011.02.018 daniel, e.m. & grimshaw, d.j. (2002) "an exploratory comparison of electronic commerce adoption in large and small enterprises", journal of information technology, 17(3): 133147. doi: 10.1080/0268396022000018409 delen, d., zaim, h., kuzey, c. & zaim, s. (2013) "a comparative analysis of machine learning systems for measuring the impact of knowledge management practices", decision support systems, 54(2): 1150-1160. dobson, p., jackson, p. & gengatharen, d.e. (2013) "explaining broadband adoption in rural australia: modes of reflexivity and the morphogenetic approach", mis quarterly, 37(3): 965-991. duhan, s., levy, m. & powell, p. (2001) "information systems strategies in knowledge-based smes: the role of core competencies", european journal of information systems, 10(1): 25-40. s167 australasian journal of information systems parker, burgess & al-qirim 2015, vol. 19, pp. s153-s173 review of sme journal articles eikebrokk, t.r. & olsen, d.h. (2007) "an empirical investigation of competency factors affecting e-business success in european smes", information & management, 44(4): 364-383. doi: 10.1016/j.im.2007.02.004 federation of small businesses uk (2014) small business statistics, federation of small businesses, lancashire, uk. available from http://www.fsb.org.uk/stats [accessed 29 september 2015]. francalanci, c. & morabito, v. (2008) "is integration and business performance: the mediation effect of organizational absorptive capacity in smes", journal of information technology, 23(4): 297–312. doi: 10.1057/jit.2008.18 gengatharen, d.e. & standing, c. (2005) "a framework to assess the factors affecting success or failure of the implementation of government-supported regional e-marketplaces for smes", european journal of information systems, 14(4): 417-433. doi: 10.1057/palgrave.ejis.3000551 grandon, e.e. & pearson, j.m. (2004) "electronic commerce adoption: an empirical study of small and medium us businesses", information & management, 42(1): 197-216. doi: 10.1016/j.im.2003.12.010 grunbaum, n.n. (2007) "identification of ambiguity in the case study research typology: what is a unit of analysis?", qualitative market research: an international journal, 10(1): 78-97. doi: 10.1108/13522750710720413 harwood, s.a. (2011) "the domestication of online technologies by smaller businesses and the 'busy day'", information and organization, 21(2): 84-106. doi: 10.1016/j.infoandorg.2011.03.002 herring, s.c. (2004) "computer-mediated discourse analysis: an approach to researching online behavior", in: s.a. barab, r. kling & j.h. gray (eds) designing for virtual communities in the service of learning, cambridge university press, new york, pp. 338-376. howcroft, d. & light, b. (2006) "reflections on issues of power in packaged software selection", information systems journal, 16(3): 215-235. doi: 10.1111/j.13652575.2006.00216.x howcroft, d. & light, b. (2010) "the social shaping of packaged software selection", journal of the association for information systems, 11(3): 122-148. huang, r., zmud, r.w. & price, l.r. (2010) "influencing the effectiveness of it governance practices through steering committees and communication policies", european journal of information systems, 19(3): 288-302. doi: 10.1057/ejis.2010.16 hung, y.-h., chou, s.-c.t. & tzeng, g.-h. (2011) "knowledge management adoption and assessment for smes by a novel mcdm approach", decision support systems, 51(2): 270-291. doi: 10.1016/j.dss.2010.11.021 hussin, h., king, m. & cragg, p.b. (2002) "it alignment in small firms", european journal of information systems, 11(2): 108-127. doi: 10.1057/palgrave/ejis/3000422 jaegersberg, g. & ure, j. (2011) "barriers to knowledge sharing and stakeholder alignment in solar energy clusters: learning from other sectors and regions", the journal of strategic information systems, 20(4): 343-354. doi: 10.1016/j.jsis.2011.03.002 kauffman, r.j. & mohtadi, h. (2004) "proprietary and open systems adoption in eprocurement: a risk-augmented transaction cost perspective", journal of management information systems, 21(1): 137-166. keller, s. & coulthard, d. (2013) "charting diversity and change in is publications: a tricontinental journal analysis", australasian journal of information systems, 18(1): 5-35. s168 australasian journal of information systems parker, burgess & al-qirim 2015, vol. 19, pp. s153-s173 review of sme journal articles kendall, j.d., tung, l.l., chua, k.h., ng, c.h.d. & tan, s.m. (2001) "receptivity of singapore's smes to electronic commerce adoption", the journal of strategic information systems, 10(3): 223-242. doi: 10.1016/s0963-8687(01)00048-8 khazanchi, d. & sutton, s.g. (2001) "assurance services for business-to-business electronic commerce: a framework and implications", journal of the association for information systems, 1(1). kuan, k.k.y. & chau, p.y.k. (2001) "a perception-based model for edi adoption in small businesses using a technology–organization–environment framework", information & management, 38(8): 507-521. doi: 10.1016/s0378-7206(01)00073-8 kuk, g. (2004) "effectiveness of vendor-managed inventory in the electronics industry: determinants and outcomes", information & management, 41(5): 645-654. doi: 10.1016/j.im.2003.08.002 lee, j.c. & myers, m.d. (2004) "dominant actors, political agendas, and strategic shifts over time: a critical ethnography of an enterprise systems implementation", the journal of strategic information systems, 13(4): 355-374. doi: 10.1016/j.jsis.2004.11.005 lee, s., park, s.b. & lim, g.g. (2013) "using balanced scorecards for the evaluation of “software-as-a-service”", information & management, 50(7): 553-561. doi: 10.1016/j.im.2013.07.006 lee, y. & larsen, k.r. (2009) "threat or coping appraisal: determinants of smb executives' decision to adopt anti-malware software", european journal of information systems, 18(2): 177-187. doi: 10.1057/ejis.2009.11 levenburg, n.m. & klein, h.a. (2006) "delivering customer services online: identifying best practices of medium-sized enterprises", information systems journal, 16(2): 135-155. doi: 10.1111/j.1365-2575.2006.00212.x levy, m., loebbecke, c. & powell, p. (2003) "smes, co-opetition and knowledge sharing: the role of information systems", european journal of information systems, 12(1): 3-17. doi: 10.1057/palgrave.ejis.3000439 levy, m. & powell, p. (2000) "information systems strategy for small and medium sized enterprises: an organisational perspective", the journal of strategic information systems, 9(1): 63-84. doi: 10.1016/s0963-8687(00)00028-7 levy, m., powell, p. & yetton, p. (2001) "smes: aligning is and the strategic context", journal of information technology, 16(3): 133-144. li, x., troutt, m.d., brandyberry, a. & wang, t. (2011) "decision factors for the adoption and continued use of online direct sales channels among smes", journal of the association for information systems, 12(1): 1-31. liang, h. & xue, y. (2004) "coping with erp-related contextual issues in smes: a vendor's perspective", the journal of strategic information systems, 13(4): 399-415. doi: 10.1016/j.jsis.2004.11.006 love, p.e.d. & irani, z. (2004) "an exploratory study of information technology evaluation and benefits management practices of smes in the construction industry", information & management, 42(1): 227-242. doi: 10.1016/j.im.2003.12.011 love, p.e.d., irani, z., standing, c., lin, c. & burn, j.m. (2005) "the enigma of evaluation: benefits, costs and risks of it in australian small–medium-sized enterprises", information & management, 42(7): 947-964. doi: 10.1016/j.im.2004.10.004 mackay, n., parent, m. & gemino, a. (2004) "a model of electronic commerce adoption by small voluntary organizations", european journal of information systems, 13(2): 147159. doi: 10.1057/palgrave.ejis.3000491 s169 australasian journal of information systems parker, burgess & al-qirim 2015, vol. 19, pp. s153-s173 review of sme journal articles mccart, j.a., padmanabhan, b. & berndt, d.j. (2013) "goal attainment on long tail web sites: an information foraging approach", decision support systems, 55(1): 235-246. mcgovern, t. & hicks, c. (2004) "how political processes shaped the it adopted by a small make-to-order company: a case study in the insulated wire and cable industry", information & management, 42(1): 243-257. doi: 10.1016/j.im.2004.01.004 mehra, a., langer, n., bapna, r. & gopal, r. (2014) "estimating returns to training in the knowledge economy: a firm-level analysis of small and medium enterprises", mis quarterly, 38(3): 757-771. mehrtens, j., cragg, p.b. & mills, a.m. (2001) "a model of internet adoption by smes", information & management, 39(3): 165-176. doi: 10.1016/s0378-7206(01)00086-6 merono-cerdan, a.l. & soto-acosta, p. (2007) "external web content and its influence on organizational performance", european journal of information systems, 16(1): 66-80. doi: 10.1057/palgrave.ejis.3000656 molla, a., heeks, r. & balcells, i. (2006) "adding clicks to bricks: a case study of e-commerce adoption by a catalan small retailer", european journal of information systems, 15(4): 424-438. doi: 10.1057/palgrave.ejis.3000623 newman, m. & zhao, y. (2008) "the process of enterprise resource planning implementation and business process re-engineering: tales from two chinese small and medium-sized enterprises", information systems journal, 18(4): 405-426. doi: 10.1111/j.13652575.2008.00305.x nicolini, d. (2012) practice theory, work, and organization: an introduction, oxford university press, oxford. olsen, k.a. & sætre, p. (2007) "it for niche companies: is an erp system the solution?", information systems journal, 17(1): 37-58. doi: 10.1111/j.1365-2575.2006.00229.x paolucci, m., sacile, r. & boccalatte, a. (2002) "allocating crude oil supply to port and refinery tanks: a simulation-based decision support system", decision support systems, 33(1): 39-54. doi: 10.1016/s0167-9236(01)00133-6 parker, c.m. & castleman, t. (2007) "new directions for research on sme-ebusiness: insights from an analysis of journal articles from 2003 to 2006", journal of information systems and small business, 1(1-2): 21-40. parker, c.m., saundage, d. & lee, c.y. (2011) "can qualitative content analysis be adapted for use by social informaticians to study social media discourse? a position paper", 22nd australasian conference on information systems, 29 november 2 december, sydney, australia. parker, c.m. & scheepers, r. (2012) "applying king et al.’s taxonomy to frame the is discipline’s engagement in green is discourse", 23rd australasian conference on information systems, 3-5 december, geelong, australia. pob-nzaou, p. & raymond, l. (2011) "managing erp system risk in smes: a multiple case study", journal of information technology, 26(3): 170-192. doi: 10.1057/jit.2010.34 poon, s. (2000) "business environment and internet commerce benefit a small business perspective", european journal of information systems, 9(2): 72-81. powell, p. & woerndl, m. (2008) "time to stop researching the important things?", european journal of information systems, 17(2): 174-178. doi: 10.1057/ejis.2008.11 quaddus, m. & hofmeyer, g. (2007) "an investigation into the factors influencing the adoption of b2b trading exchanges in small businesses", european journal of information systems, 16(3): 202-215. doi: 10.1057/palgrave.ejis.3000671 s170 australasian journal of information systems parker, burgess & al-qirim 2015, vol. 19, pp. s153-s173 review of sme journal articles ray, a.w. & ray, j.j. (2006) "strategic benefits to smes from third party web services: an action research analysis", the journal of strategic information systems, 15(4): 273-291. doi: 10.1016/j.jsis.2006.07.001 reardon, j.l. & davidson, e. (2007) "an organizational learning perspective on the assimilation of electronic medical records among small physician practices", european journal of information systems, 16(6): 681-694. doi: 10.1057/palgraveejis.3000714 riemenschneider, c.k., harrison, d.a. & mykytyn jr, p.p. (2003) "understanding it adoption decisions in small business: integrating current theories", information & management, 40(4): 269-285. doi: 10.1016/s0378-7206(02)00010-1 riemenschneider, c.k. & mykytyn jr, p.p. (2000) "what small business executives have learned about managing information technology", information & management, 37(5): 257-269. doi: 10.1016/s0378-7206(99)00052-x rivard, s., raymond, l. & verreault, d. (2006) "resource-based view and competitive strategy: an integrated model of the contribution of information technology to firm performance", the journal of strategic information systems, 15(1): 29-50. doi: 10.1016/j.jsis.2005.06.003 sarker, s., xiao, x. & beaulieu, t. (2013) "qualitative studies in information systems: a critical review and some guiding principles", mis quarterly, 37(4): pii-xviii. schultze, u. (2015) "skirting slr's language trap: reframing the 'systematic' vs 'traditional' literature review opposition as a continuum", journal of information technology, 30(2): 180-184. doi: 10.1057/jit.2015.10 shen, z., lyytinen, k. & yoo, y. (2014) "time and information technology in teams: a review of empirical research and future research directions", european journal of information systems, in press. doi: 10.1057/ejis.2014.8 singer, j.d. (1961) "the level-of-analysis problem in international relations", world politics, 14(1): 77-92. doi: 10.2307/2009557 street, c.t. & meister, d.b. (2004) "small business growth and internal transparency: the role of information systems", mis quarterly, 28(3): 473-506. tallon, p.p. (2010) "a service science perspective on strategic choice, it, and performance in u.s. banking", journal of management information systems, 26(4): 219-252. trochim, w.m.k. (2006) unit of analysis, http://www.socialresearchmethods.net/kb/unitanal.php [accessed 24 may 2015]. tsaih, r., liu, y.-j., liu, w. & lien, y.-l. (2004) "credit scoring system for small business loans", decision support systems, 38(1): 91-99. tsatsou, p., elaluf-calderwood, s. & liebenau, j. (2010) "towards a taxonomy for regulatory issues in a digital business ecosystem in the eu", journal of information technology, 25(3): 288-307. doi: 10.1057/jit.2009.22 united states census bureau (2012) statistics of u.s. businesses (susb) main, united states census bureau, washington, dc. available from http://www.census.gov/econ/susb/ [accessed 29 september 2015]. vega, a., chiasson, m. & brown, d.h. (2008) "extending the research agenda on diffusion: the case of public program interventions for the adoption of e-business systems in smes", journal of information technology, 23(2): 109-117. doi: 10.1057/palgrave.jit.2000135 vidgen, r. (2002) "constructing a web information system development methodology", information systems journal, 12(3): 247-261. doi: 10.1046/j.1365-2575.2002.00129.x s171 australasian journal of information systems parker, burgess & al-qirim 2015, vol. 19, pp. s153-s173 review of sme journal articles walters, b.a., jiang, j.j. & klein, g. (2003) "strategic information and strategic decision making: the eis/ceo interface in smaller manufacturing companies", information & management, 40(6): 487-495. doi: 10.1016/s0378-7206(02)00063-0 wang, y. & ahmed, p.k. (2009) "the moderating effect of the business strategic orientation on ecommerce adoption: evidence from uk family run smes", the journal of strategic information systems, 18(1): 16-30. doi: 10.1016/j.jsis.2008.11.001 yin, r.k. (2014) case study research: design and methods, 5th edition, sage publications, thousand oaks, california. zhang, m., sarker, s. & sarker, s. (2008) "unpacking the effect of it capability on the performance of export-focused smes: a report from china", information systems journal, 18(4): 357-380. doi: 10.1111/j.1365-2575.2008.00303.x zhang, m., sarker, s. & sarker, s. (2013) "drivers and export performance impacts of it capability in ‘born-global’ firms: a cross-national study", information systems journal, 23(5): 419-443. doi: 10.1111/j.1365-2575.2012.00404.x zhu, k., kraemer, k.l. & xu, s. (2003) "electronic business adoption by european firms: a cross-country assessment of the facilitators and inhibitors", european journal of information systems, 12(4): 251-268. doi: 10.1057/palgrave.ejis.3000475 s172 australasian journal of information systems parker, burgess & al-qirim 2015, vol. 19, pp. s153-s173 review of sme journal articles appendix: list of a* is journals abbreviation journal name acm acm transactions on computer-human interaction dss decision support systems ejis european journal of information systems i&m information and management i&o information and organization isj information systems journal isr information systems research jais journal of the association for information systems jaist journal of the association for information sciences and technology jit journal of information technology jmis journal of management information systems jsis journal of strategic information systems misq mis quarterly copyright: © 2015 parker, burgess & al-qirim. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. s173 paper title (paper title style) australasian journal of information systems namvar, cybulski, phang, ee & tan 2018, vol 22, research note simplifying sensemaking 1 simplifying sensemaking: concept, process, strengths, shortcomings, and ways forward for information systems in contemporary business environments morteza namvar university of wollongong morteza.namvar@uow.edu.au jacob l. cybulski deakin university cynthia su chen phang swinburne university of technology yaw seng ee swinburne university of technology kevin tee liang tan swinburne university of technology abstract sensemaking is a popular and useful organizational behaviour concept that is gaining visibility in the field of information systems. however, it remains relatively unknown compared to more established information systems concepts like technology acceptance and resistance. to enhance and propel greater use of sensemaking in information systems, this article offers a systematic explanation of sensemaking, specifically focusing on its concept, process, strengths, and shortcomings, as well as discussing ways forward for information systems in contemporary business environments. keywords: sensemaking; decision making; information systems; concept; process; strengths; limitations; future research directions. 1 introduction sensemaking, which was introduced and popularized by karl weick in 1969, is a concept that is accompanied by great irony. on the one hand, sensemaking aims to provide clarity and meaning to obscure phenomena (weick, 1969, 1979, 1995); on the other hand, the conceptual boundaries (or process) of sensemaking have become obscure due to the mushrooming of sensemaking perspectives, leading to a state where “there is no single agreed definition of sensemaking” (brown et al., 2015, p. 266). moreover, unlike other concepts in information systems—such as technology acceptance and resistance—that have been extensively reviewed to provide a snapshot of their progress in the field, sensemaking lacks a useful reference point that can offer scholars who are interested in the concept with such an overview. to this end, the present paper aims to provide a systematic explanation of sensemaking, specifically focusing on its concept, process, strengths, and shortcomings, as well as ways forward for information systems in contemporary business environments. in doing so, this paper hopes to consolidate the essence of diverse sensemaking perspectives into a simplified blueprint for sensemaking. the aim is to stimulate greater usage of sensemaking to advance australasian journal of information systems namvar, cybulski, phang, ee & tan 2018, vol 22, research note simplifying sensemaking 2 our understanding and application of information systems in contemporary business environments. 2 sensemaking sensemaking exists due to the perception-expectation gap, which may occur in the form of breakdowns, surprises, discrepancies, or opportunities in organizational settings. however, to many entities, especially decision-makers, making sense of the data required to close this gap is often a challenging task. this is because data in its existential form is complicated, complex, and chaotic, and therefore, overwhelming. the following section on the concept of sensemaking deals with this challenge, and the subsequent sections introduce and discuss sensemaking in detail. 2.1 the concept sensemaking has been defined in many ways by various scholars over time. for example: • russell et al. (1993, p. 269) considered sensemaking as “the process of searching for a representation and encoding data in that representation to answer task-specific questions,” • dervin (1998, p. 36) described sensemaking as “a methodology disciplining the cacophony of diversity and complexity without homogenizing it,” • weick et al. (2005, p. 409) suggested sensemaking as “the ongoing retrospective development of plausible images that rationalize what people are doing,” and • brown et al. (2008, p. 1055) proposed sensemaking as “the processes of organizing using the technology of language—processes of labelling and categorizing for instance—to identify, regularize and routinize memories into plausible explanations.” to comprehensively put the essence of extant definitions of sensemaking into a straightforward perspective for future reference, this article considers the “what,” “how,” and “so what” questions relating to sensemaking. in doing so, it proposes a simple but holistic definition for sensemaking: sensemaking is a process of clarifying and removing ambiguity and uncertainty by searching for and organizing similarities and differences from data sources through which goal-directed interpretations for decision-making are established. therefore, sensemaking as a process is the foundation of knowledge creation, where the quality of sensemaking affects the quality of knowledge produced and the outcome of decisions predicated on that knowledge. however, the peculiarities of this process are complex and varied. the process of sensemaking is scrutinized and discussed in the next section. 2.2 the process the sense that one initially and subsequently develops over time is predicated on a set of underpinnings, which has taken multiple forms over the years. for example: • dervin (1983; 1998) contends that sensemaking as a methodology involves identifying and bridging three gaps that exist between situation and outcome— namely time, space, and movement. these gaps also interact with each other to create additional gaps—namely time-space, time-movement, and space-movement gaps. australasian journal of information systems namvar, cybulski, phang, ee & tan 2018, vol 22, research note simplifying sensemaking 3 • russell et al. (1993, p. 271) posit that sensemaking consists of three learning loops— namely the generation loop (or the search for representations, where appropriate representations are created to capture important regularities in data sources—e.g. establish categories, comparison dimensions, specifications for clustering), the data coverage loop (or the instantiation of representations, where data of interest are identified and encoded into appropriate representations), and the representational shift loop (or the shift of representations, where sensemaking iterates between the generation and data coverage loops—e.g. to account for the discovery of residue, where original representations may be merged or split, for otherwise new representations may be created). • weick (1995; 2008, pp. 1404-1405) proposes that sensemaking is a project whose process is grounded on seven properties—namely identity construction (or the sense of self, as an individual [such as skills, knowledge, experience] or as an organization [such as structure, strategy, culture]), social context (or the associated meanings assigned to social support, consensual validation, and shared relevance in the actual, implied, or imagined presence of others—e.g. culture, education, and interaction among interdependent people shape the interpretation of events), retrospection (or the reflection on past events relevant to current interpretations [e.g. structured or unstructured internal and external data], which may be influenced by the extent of lapse time and recollection capabilities), salient cues (or the diagnosis and elaboration of traces for confirming evidence [e.g. alerts, metrics, patterns, key performance indicators]), ongoing projects (or the continuing or future events that can alter and shape current interpretations [e.g. data from business intelligence and analytics]), plausibility (or the adequacy of interpretation and associated justifications to proceed with current projects [e.g. plausible models]), and enactment (or the actions that one takes to inspect, interpret, or interact with observations—e.g. asking questions and inserting probes to make sense of a situation). note that sensemaking is applicable to both business processes and projects. • pirolli and card (2005) suggest that sensemaking consists of a set of sequential tasks—namely information development (or the gathering of information of interest), schema development (or the representation of gathered information in a schema that aids analysis), insight development (or the insights produced through the manipulation of representations), and product development (or the action or creation of a product predicated on the knowledge from sensemaking insights). the sequence of sensemaking tasks can follow two approaches: a bottom-up approach or a top-down approach. the former requires the sensemaker to search and filter, read and extract, schematize, build a case, and relate the information of interest (story tell), whereas the latter requires the sensemaker to search for support, evidence, relations, and information from the final product. regardless of its approach, the quality of sensemaking is assured through two loops: the foraging loop (or the exploration–enrichment–exploitation cycle) and the sensemaking loop (or the problem structuring [i.e. the generation, exploration, and management of hypotheses], evidentiary reasoning [i.e. the marshalling of evidence to support or disconfirm hypotheses], and decision-making [i.e. the choosing of the best australasian journal of information systems namvar, cybulski, phang, ee & tan 2018, vol 22, research note simplifying sensemaking 4 alternative for further action from the set of alternatives produced through the foraging loop]). • namvar et al. (2016) shed light on the notion of information systems in the form of business intelligence (e.g. standard reports, ad-hoc reports, query drilldowns, and alerts) and business analytics (e.g. statistical analysis, forecasting, predictive modelling, and optimization). the authors identify these as the key factors of consideration (or enablers) that sensemakers should take into account and use to improve the process (e.g. empower and maximize the returns) of sensemaking. when an individualistic view is employed, the process of sensemaking proposed by each scholar appears distinct. however, when a collective view is taken, the crossroads between these underpinnings suggest that the different propositions by past scholars are, in fact, complementary to one another. thus, similar to the consolidation of sensemaking definitions in the previous section, this article contends that an integration of underpinnings is necessary to address existing piecemeal understanding and to solidify future understanding of the sensemaking process (see figure 1). more specifically: • the contention by dervin (1983, 1998) sets the overarching tone to explain the motivation to engage in sensemaking—that is, to identify and bridge the gap—of time (past, present, future), space (context), movement (changes), or its combination (interaction) between an existing situation and an intended outcome through which sensemaking is applied as a knowledge-based solution-generation mechanism. • the suggestion by pirolli and card (2005) lays the foundations for the sensemaking process in terms of its tasks (i.e. information, schema, insight, and product development), sequence (i.e. bottom-up or top-down), and quality-assurance loops (i.e. foraging and sensemaking loops). • the proposition by russel et al. (1993) in the form of learning loops (i.e. generation, data coverage, and representational shift loops) acts as sub-loops within the foraging loop (or as part of the exploration–enrichment–exploitation cycle). • finally, the sensemaking properties proposed by weick (1995, 2008) and the business information systems suggested by namvar et al. (2016) are positioned as factors to consider (or enablers) for the sequential tasks and quality-assurance loops in the sensemaking process. that is, data in the form of salient cues and ongoing projects serve as inputs to information development; identity construction, social context, retrospection, and plausibility serve as considerations that must be taken into account by the sensemaker as they affect the interpretations made during insight development; and the development of information received and the quality assurance loops are enacted with the support of business information systems in the form of business intelligence and business analytics. 2.3 the strengths the concept and process of sensemaking provided in this article have several upshots. first, sensemaking offers decision-makers with a structured process of dealing with ambiguity and uncertainty (weick, 1995, 2008). second, sensemaking offers explanations regarding the australasian journal of information systems namvar, cybulski, phang, ee & tan 2018, vol 22, research note simplifying sensemaking 5 mental models of reality, their causes, and their use in reaching informed decisions with the support of existing and future information systems and technology (boland, 2008; namvar et al., 2016). third, sensemaking links elements of the natural world, the social world, and the technology world, and thus is well-positioned to contribute to research inquiry in contemporary environments, such as in business information systems research (gregor, 2006). fourth, sensemaking offers the opportunity to address the challenges of using business information systems for decision-making (hasan & gould, 2001), especially in modern datacentric organizations where business and information systems processes are intertwined (e.g. managerial decision-making and business intelligence and analytics) (namvar & cybulski, 2014). in short, sensemaking, in a general sense, offers the opportunity to learn about, organize, and give meaning to obscure phenomena, and in a business sense, enables the transformation of complicated, complex, and chaotic organizational and marketplace data into the precipice of business acumen and realization among decision-makers, on behalf of organizations and in contemporary business environments. 2.4 the shortcomings when sensemaking was first introduced by weick in 1969, information technology was limited to supporting core organizational processes, such as the management of human records, payroll processing, and producing financial reports. the notion of computer-based decision support was still in its infancy and modern communication technology was not available. it is not surprising that, over the years, sensemaking developed predominantly as a methodology for dealing with complex organizational phenomena (e.g. strategic change) (gioia & chittipeddi, 1991; maitlis & christianson, 2014) rather than as a practical information systemsbased tool for improving decision-making in organizations. in spite of weick’s foresight in casting sensemaking as a data-intensive approach to organizational learning, the development of sensemaking missed out on opportunities offered by years of technological innovation. for instance, only in the last decade have we seen the advent and boom of new and inexpensive technologies, widely accessible to organizations and individuals, such as personal wireless networks (e.g. wi-fi and bluetooth), mobile computing (e.g. smartphones), online media (e.g. youtube) and social media (e.g. twitter), on-demand online services (e.g. uber), crowdsourcing (e.g. mechanical turk), cryptocurrencies (e.g. bitcoin), and more. all these developments resulted in new socio-technological environments (e.g. online businesses), new types of business transactions (e.g. paypal), new types of products (e.g. audio and video streams), new means of enacting business processes (e.g. blockchain), new ways of sensing and collecting environmental cues (e.g. with remote sensors and meters), new ways of generating data (e.g. by leaving digital footprints via mobile phone), new kinds of business data or data that previously could not be processed (e.g. streams of unstructured text, audio, and video), fuzzy boundaries between an organization and an individual (e.g. work anywhere and anytime), new ways of creating personal and business identities (by creating instant impressions—e.g. instagram), new ways of instant collaboration (e.g. via skype or trello), and more. business circumstances are more uncertain than in the 1980s; change is proceeding at a higher pace and innovation has a steeper curve. in this world of technology-inspired change, the questions for sensemaking theorists are: how can we make sense of these new socio-organizational phenomena? how could sensemaking assist decisionmaking in a world disrupted by the introduction of new technologies? how could new technology improve sensemaking in support of decision-making? australasian journal of information systems namvar, cybulski, phang, ee & tan 2018, vol 22, research note simplifying sensemaking 6 the consolidation of the concept and process of sensemaking outlined here, along with the ways forward for sensemaking presented in the next section, contributes some insights and paves the way for future research to provide more elaborate thoughts predicated on empirical research to respond to these questions. 2.5 the ways forward sensemaking results in individuals gaining an understanding of organizational complexity, which leads to desired action in the organization and its environment. the contemporary business environment is immersed in data and saturated with new business and personal technologies, some of which enhance while others disrupt decision-making processes. by relying on sensemaking processes, information systems-based decision-support tools—such as business intelligence and analytics—can be used and further improved to facilitate a more nuanced understanding of contemporary business environments, thus leading to more informed business decisions. this requires consideration of sensemaking properties when developing and using tools such as business intelligence and analytics, as follows: • retrospection: traditional information systems and analytics tools are useful to provide a retrospective view of business environments by collecting, processing, and presenting structured data for decision-makers. however, new data analytics systems featuring text analytics and stream analytics tools can also process massive amounts of unstructured data, which is (often freely) available to business from internal or external sources. • salient cues: many businesses today experience pressure from internal and environmental change, which confounds the collection of data cues and potentially leads to inferior data quality and decision-making uncertainty. through data modelling, aggregation, and clustering, business intelligence and analytics tools can assist in rapidly reducing the volume, veracity, and volatility of data to a small number of relevant cues, such as alerts, key performance indicators, metrics, and patterns, among others, which can then be used to assign meaning to equivocal business situations and establish triggers for action. • plausibility: due to the quality and volatility of data collected from open data sources, decisions based on such sources are subject to considerable uncertainty and risk. plausible models developed with predictive analytics methods can, however, provide decision-makers with measures to weigh in various risk factors and with opportunities to apply their own expertise and domain knowledge, especially in situations where all the relevant factors and parameters cannot be incorporated into predictive models. • identity construction (individual) and social context: decision-makers can also rely more extensively on the knowledge and experience of their team members, who can be accessed via collaborative (e.g. microsoft teams) and business social media (e.g. yammer) facilities for sharing and reviewing online reports. many such reports can be delivered in actionable form on portable devices. thus, they readily fit the work patterns of a new generation of decision-makers who work anywhere and anytime. tough decisions and hard cases (handling which may exceed inhouse skills), could be referred to the wisdom of crowds in the form of questions, problems, and competitions (e.g. quora and kaggle). australasian journal of information systems namvar, cybulski, phang, ee & tan 2018, vol 22, research note simplifying sensemaking 7 • identity construction (organization), ongoing projects, and enactment: notwithstanding the advances in and proliferation of technology in recent times, one of the major limitations of current technologies is the lack of any direct link between decisionsupport tools and enactment processes, which is an important element in organizational sensemaking. the enactment of a decision-maker’s own vision of the organizational process is still elusive; possible solutions include involving decision-makers in report generation and model building via self-service workflow analytics with interactive user-friendly drag-and-drop interfaces (e.g. microsoft power bi). the main requirement of the latter activity would be to alter the organizational identity (or capacity) in using business intelligence and analytics, as reflected in the structure, strategy, culture, and processes (whether formalized or enacted), all of which support internal collaborative activities, external presence, and overall behaviour. if resistance is present, potential behavioural remedies through priming measures may be considered (lim, 2015). 3 conclusion in the increasingly competitive business landscape of today, organizations rely on massive amounts of data to monitor and analyse changes in their business environment. business information systems, such as business intelligence and analytics, assist in this process by turning data into evidence that is capable of supporting decision-making. with rapid organizational and technological changes, and given the growing amount of shared data and data availability, it goes without saying that business decisions must also rest on a sound understanding of this complexity, for which sensemaking is crucial. specifically, this article has made clear that sensemaking as an approach is suitable for continuously gaining an understanding of contemporary business environments and providing additional insights into decision-making processes. sensemaking provides a useful theoretical lens for studying factors that influence organizations in their move toward informed decision-making. this article provides a consolidated overview of sensemaking in the form of its concept, process, strengths, and shortcomings, as well as ways forward for information systems in contemporary business environments, with a special focus on new ways of developing and using existing and future business intelligence and analytics more effectively. acknowledgement many thanks to weng marc lim, the editor, and the anonymous reviewers for their constructive feedback to improve the quality of the paper. australasian journal of information systems namvar, cybulski, phang, ee & tan 2018, vol 22, research note simplifying sensemaking 8 figure 1 the sensemaking process australasian journal of information systems namvar, cybulski, phang, ee & tan 2018, vol 22, research note simplifying sensemaking 9 references boland, r. j. (2008). decision making and sensemaking. in handbook on decision support systems 1 (pp. 55–63). new york: springer. doi: https://doi.org/10.1007/978-3-540-48713-5_3 brown, a. d., stacey, p., & nandhakumar, j. (2008). making sense of sensemaking narratives. human relations, 61(8), 1035-1062. doi: https://doi.org/10.1177/0018726708094858 brown, a. d., colville, i . , & pye, a. (2015). making sense of sensemaking in organization studies. organization studies , 36(2), 265-277. doi: https://doi .org/10.1177/0170840614559259 dervin, b. (1998). sense-making theory and practice: an overview of user interests in knowledge seeking and use. journal of knowledge management, 2(2), 36-46. doi: https://doi.org/10.1108/13673279810249369 gioia, d. a., & chittipeddi , k. (1991). sensemaking and sensegiving in strategic change ini tiation. strategic management journal , 12, 433–448. doi: https://doi .org/10.1002/smj.4250120604 gregor, s. (2006). the nature of theory in information systems. mis quarterly, 611–642. doi: https://doi.org/10.2307/25148742 hasan, h., & gould, e. (2001). support for the sense-making activity of managers. decision support systems, 31(1), 71–86. doi: https://doi.org/10.1016/s0167-9236(00)00120-2 lim, w. m. (2015). enriching information science research through chronic disposition and situational priming: a short note for future research. journal of information science, 41(3), 399-402. doi: https://doi.org/10.1177/0165551515577913 maitlis, s., & christianson, m. (2014). sensemaking in organizations: taking stock and moving forward. academy of management annals, 8(1), 57–125. doi: https://doi.org/10.1080/19416520.2014.873177 namvar, m., & cybulski, j. (2014). bi-based organizations: a sensemaking perspective. proceedings of the international conference on information systems 2014 (icis 2014), auckland, new zealand. namvar, m., cybulski, j., & perera, l. (2016). using business intelligence to support the process of organizational sensemaking. communications of the association for information systems, 38, 330–352. doi: https://doi.org/10.17705/1cais.03820 pirolli, p., & card, s. (2005). the sensemaking process and leverage points for analyst technology as identified through cognitive task analysis. proceedings of international conference on intelligence analysis (vol. 5). russell, d. m., stefik, m. j., pirolli, p., & card, s. k. (1993). the cost structure of sensemaking (pp. 269–276). proceedings of the interact'93 and chi'93 conference on human factors in computing systems, acm. doi: https://doi.org/10.1145/169059.169209 weick, k. e. (2008). sensemaking. in clegg, s. r., & bailey, j. r., international encyclopaedia of organization studies (pp. 1404-1406). thousand oaks, ca: sage publications. weick, k. e. (1995). sensemaking in organizations. thousand oaks, ca: sage publications. weick, k. e. (1969). the social psychology of organizing. reading, ma: addison-wesley. australasian journal of information systems namvar, cybulski, phang, ee & tan 2018, vol 22, research note simplifying sensemaking 10 weick, k. e. (1979). the social psychology of organizing (2nd ed.). new york: mcgraw-hill. wei ck , k. e . , sutcl i f fe , k. m. , & obstfel d , d. (2005) . organizi ng and the process of sen semaki ng. organizat i on science , 16 (4 ) , 409–421 . doi : ht tps: / /doi .org/10.1287/orsc .1050 .0133 copyright: © 2018 namvar, cybulski, phang, ee & tan. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 sensemaking 2.1 the concept 2.2 the process 2.3 the strengths 2.4 the shortcomings 2.5 the ways forward 3 conclusion microsoft word ajis vol 19.docx australasian journal of information systems lin & parinyavuttichai 2015, vol. 19, pp. 1-22 project risks as emergent phenomena 1 is project risks as emergent phenomena: towards a model of risk escalation and its management angela y lin the university of sheffield a.lin@shef.ac.uk nipon parinyavuttichai rajabhat rajanagarindra university nparinya@gmail.com abstract while the number of substantive investments in is projects continues to grow, the number of failing projects also continues to increase at an alarming rate. both the academic and industry literature suggests that inadequate attention to risk and its management continues to be a key factor in project failure. the typical approach taken is to identify and map potential risks, to act as a planning and diagnostic tool, and to prepare a contingency plan has been a factorbased approach. while it remains a valuable tool for mapping anticipated risks the factorbased approach is less effective when viewing project risks as emergent phenomena that unfold during the course of the project, and require ongoing attention and risk management. informed by a case study of a failing university is development project, this paper focuses on the phenomenon of risk escalation. the case findings suggest that rather than being defined ahead of the project, some project risks may emerge during the project as a consequence of escalation factors that were both antecedent to and a consequence of actual risk management decisions. the article concludes with suggestions as to how project managers can better man-age the emergent rather than static nature of risk phenomena. keywords: information systems project management, risk, risk management, risk escalation, project escalation 1 introduction substantive investments in information systems (is) continue to be made as managers seek a source of leverage for improving the efficiency and productivity of the organizations within which they work. at the same time a high volume of is projects continue to end up as failing projects, either cancelled or abandoned through poor management or over spend, or otherwise incomplete in some way through not working or not meeting the expectations of their users (creswell et al., 2011, mcleod and macdonell, 2011, mcmanus and wood-harper, 2007, morrison et al., 2011, pankratz and basten, 2013). while factors that contribute to is project failures are diverse (fortune and white, 2006), a lack of identification of the risks involved in developing is along with a lack of adequate management of these risks have been common findings in many studies (de bakker et al., 2010, kerr and houghton, 2010, kitsch et al., 2013, pm solutions, 2011, rozenes, 2011, wilton, 2005). is project risks can be characterized as the uncertainties that prevent a project team from delivering a planned system on time and within budget. is project risk management can therefore be defined as the effort required to develop and implement strategies that counter the uncertainties that threaten the possible success of a project (sherer, 1992). traditional is risk management typically involves a factor-based approach, the first and foremost stage of which involves the risk manager in identifying and defining the potential risks (dey et al., 2007, project management institute, 2013). at this stage, foreseeable project risks are identified and contingency plans drawn up to mitigate the risks and limit their impact should they arise on a project (kim and park, 2006, project management institute, 2013, teller, 2013). the premise of the factor-based approach to is risk management is that risks are entities australasian journal of information systems lin & parinyavuttichai 2015, vol. 19, pp. 1-22 project risks as emergent phenomena 2 that can be identified and defined prior to the commencement of a project; that they are time and context independent, and that the nature and attributes of such risks remain static throughout the lifetime of a project. such a ‘factor-based’ approach to risk management, has been widely adopted (barki et al., 1993, dagher and kuzic, 2011, kappleman et al., 2006, tiwana and keil, 2004), nevertheless the approach overlooks the emergent nature of risks (leonard, 2012, wright and capps, 2010). first, a factor-based approach takes little account of how risks and their management depend on a decision maker's or group of decision-makers’ own understandings and attitudes towards those risks and the situation to which they relate; as a consequence poor risk management can itself become a risk for a project. second, the approach does not take into account the fact that risks can emerge as a result of complex social interaction; that risks can emerge in situ and not necessarily be predicted (lyytinen and robey, 1999). third, a factor-based approach has normally not taken into account the interdependence of different risks and how the occurrence of one risk can be the cause of or the consequence of another risk (belassi and tukel, 1996, nandhakumar, 1996). fourth, since risks can be emergent and interdependent, the nature of these risks and their properties can vary from one project stage to another, and from project to project (larsen and myers, 1998). finally, is project risks may emerge during a project as a result of unexpected factors; either within the project (e.g. changes in users’ expectations of a system or a change of project champion), or within the environment of the project (e.g. changes in organizational or market environments, or in government policy). in summary, while a factor-based approach remains a valuable tool for mapping anticipated risks the factor-based approach is less effective when viewing project risks as emergent phenomena that unfold during the course of the project, and require ongoing attention and risk management. while a factor-based approach is capable of anticipating what and why risks occur, for the purposes of predicting and controlling these risks through effective risk management; a more dynamic approach begins to ask how risks emerge as a function of ongoing risk management decisions. the purpose of this article is to bridge this gap by focusing on the phenomenon of risk escalation which refers to the tendency for decision makers “ to become locked into a course of action, throwing good money after bad or committing new resources to a losing course of action” (staw, 1981, p. 578). informed by a case study of a university is development project, the findings suggest that rather than being defined ahead of the project, different types of project risk emerged during the project as a consequence of escalation factors that were both antecedent to and a consequence of risk management decisions. the article concludes with suggestions for how project managers can better manage the emergent rather than static nature of risk phenomena. the paper is organized as follows. the next section presents a review of three main categories of risk that can arise on an is development project: requirements risk, personnel risks, and resource management risks. this is followed by a description of research strategy including the case study approach used, data collection, data analysis methods and escalation model as analytical framework. a findings section initially describes the risks that arose on the project, and why before introducing a model of risk escalation. the article concludes with suggestions as to how project managers can better manage the emergent rather than static nature of risk phenomena. 2 theoretical background in this review, we first focus on project risks occurring in the areas of requirements, personnel, and resource management. while uncertainties and risks can occur in a number of other areas of an is project including organizational environment, user commitment, project management, resource management, personnel, requirements, development process and technology; it is these areas that are particularly relevant to our case, and which subsequently inform the development of our risk escalation model. in the second part of the review we discuss escalating factors that contribute to project as well as risk escalation. the discussion is based on keil’s (1995) model and organised under the headings of project, psychological, social and structural factors. australasian journal of information systems lin & parinyavuttichai 2015, vol. 19, pp. 1-22 project risks as emergent phenomena 3 2.1 risk factors on information systems projects 2.1.1 requirements risks requirements risks refer to project uncertainties that arise from changing requirements, requirements misunderstanding, and incomplete requirements (schmidt et al., 2001; robinson et al., 2003; keil et al., 2006). changing requirements is a common requirement risk that almost all projects would encounter and the reasons for changing requirements include changes in business environment which inevitably invokes changes in requirements so that the envisaged system can reflect the change (land, 1982); conflicts or politics concerning system specifications between user departments (sherer and alter, 2004; wallace et al., 2004), and the lack of understanding of the envisioned system on user’s part means that users are likely to change their mind about what they want from the system (jiang et al., 1999; kumar, 2002). continuous changes in requirements means that it is difficult for the project team to control the progress of the project, resource allocation, and budget (tiwana and keil, 2004). risk of requirement misunderstanding occurs for a number of possible reasons. these include differing perceptions of the system or outcome of is project development between stakeholders (sumner, 2000, schmidt et al., 2001); neither of the project team nor users having a good understanding of the envisioned system (dey et al., 2007); lack of communication between project team and user groups to clarify the requirements (coughlan et al., 2003); and stakeholders are unable to articulate their requirements to the project team. furthermore, user requirements may not always be carefully defined and effectively translated into the design by the project team (kim et al., 2005; dey et al., 2007; han and huang, 2007). an implication from misunderstanding requirements for the system development is the failure to develop necessary or the development of unnecessary functionalities (ropponen and lyytinen, 2000; kim et al., 2005) which leads to subsequent system modifications and hence project delays (mcallister, 2006). incomplete requirement risks arise when not all of user requirements are collected and considered or some user requirements are overlooked or ignored either by project team or by users (lauesen and vinter, 2001; yeo, 2002). there are reasons for why requirements are not collected or overlooked including changes in the system requirements or in expectation of the system because the situation has changed during the course of the project; the project team does not have appropriate or use inappropriate methods to collect user requirements; and the project team fails to verify their understanding with users but develops the system based on their own assumptions (howcroft and wilson, 2003). incomplete user requirements can result the project team in spending more time on collecting user requirements from users and correcting or modifying the design accordingly in the later stage of the project (lauesen and vinter, 2001). 2.1.2 personnel risks personnel risks are often cited as reasons for project failure (sherer and alter, 2004; kim and park, 2006; han and huang, 2007). the risks arise due to insufficient number of skilled personnel to deliver the project (keil et al., 2006); unrealistic estimation of staffing time which causes more work per capita (ropponen and lyytinen, 2000; peterson et al., 2002; kim and park, 2006; han and huang, 2007); high staff turnover rate that causes legacy in terms of knowledge and time and hence project delays(wallace et al., 2004; kim and park, 2006); and over relaying on one or few people which puts the project at risk if the personnel resigns or leaves the project (schmidt et al., 2001).project mangers’ ability to manage personnel can contribute to personnel risks for example overestimating project staff’s ability can result in inappropriate resource allocation such as human and time (ropponen and lyytinen, 2000); failing to organize sufficient training sessions for the staff(peterson et al., 2002; kim and park, 2006; han and huang, 2007) can create situations where staff does not have sufficient up to date skills and knowledge; and failing to motivate staff to commit to the project (sherer and alter, 2004; wallace et al., 2004). australasian journal of information systems lin & parinyavuttichai 2015, vol. 19, pp. 1-22 project risks as emergent phenomena 4 2.1.3 resource management risks resource management risks refer to uncertainties in relation to project planning and control of project resources. a project can face resource management risk from the outset for example when a project has insufficient budget to deliver the project (jiang et al., 1999; sumner, 2000; sherer and later, 2004; dey et al., 2007). resource management risks can arise from mismanagement of resources because of not using or lack of appropriate tools to calculate required resources and failing to estimate required resources correctly (ropponen and lyytinen, 2000; barki et al., 2001; schmidt et al., 2001; han and huang, 2007); and failure to involve key project stakeholders in the budget planning process (barki et al., 2001). mismanaging budget refers to the situations where project budget is unevenly allocated and used for example a project may use up much of its resources early in the project (keil, 1995; schmidt et al., 2001) or the resources are misused by being allocated to support the activities that are not relevant to the project (keil and mann, 1997; wallace et al., 2004). 2.2 escalation factors escalation of commitment has received much attention in the is domain in particular the studies in is project failures (drummond, 1996, 1998, jamieson and hyland, 2006, keil and mann, 1997, mähring and keil, 2008, pan et al, 2009). an extensive study of the failing taurus project at the london stock exchange has for example shown and explained how decision makers were constrained by the unique relationship between the london stock exchange and the market; and the complex interactions, e.g. negotiation and compromise, between different departments and those who had the power to influence decisions (drummond, 1996). consequently they were trapped in a situation of escalation, and committed to allocating more resources to a failing project until such time as the project was cancelled. a study of an is project in a utilities service company shows that an escalation of commitment was an outcome of recursive interactions between the project, organizational work activities, and their contexts (pan et al., 2009). a customer centric organizational culture for example created a situation where decision makers found it difficult not to accept requests for changes from users. the commitment to accept constant changes in user requirements soon led to project delays, which in turn led to a problem in resource planning. the project further deviated from the initial plan as some of the systems features were no longer required because of changes in the business environment e.g. the decrease in customer numbers. the mandata project, which was sponsored by the australian public service board and aimed to assist office automation, information management, and records processing for the entire australian public service, is another example of project escalation that led to project abandonment (sauer, 1993). the project ran through 1970s and was finally terminated in 1981 and the total cost of the project was about 30 million australian dollars. according to sauer (1993) the project suffered from a chronic haemorrhage of support, lack of sufficient resources, insufficient project planning (e.g. no definite implementation date), lack of strategic applications in the system that were useful to the stakeholders, and so on. sauer (1993:69) states that a possible explanation for why authority let an obvious failing project continue because the board wanted to avoid embarrassment or political damage from loss of face; and the project might have been subject to a form of escalating commitment. these studies have observed that escalations are usually determined by a combination of different factors; that they don’t ‘just happen’ but the outcomes of a subtle interplay between people, context, and technology which develop over time; and that they are not one-phase phenomena but cut through all phases (alvarez et al., 2011, winch, 2008). escalating factors can be categorized into four main types: project, psychological, social, and structural (staw and ross, 1987). project factors are the attributes of a project that play a role in determining the continuation or termination of an existing course of action e.g. the size and scope of a project, the economic structure of a project, the foreseeable expenditure or costs required to complete the project, and the availability of alternative courses of action. decision makers tend to keep allocating resources to a project when they believe that the payoff will australasian journal of information systems lin & parinyavuttichai 2015, vol. 19, pp. 1-22 project risks as emergent phenomena 5 benefit the organization in the long-term or when they believe that the project is close to reaching completion (keil et al., 2000). figure 1. summary model of factors influencing escalation source: keil (1995, p.436) psychological factors are a further salient influence on people’s behaviour in escalation situations. these include self-justification, self-inference, and bias when processing information. self-justification is an important element in studying the reasons for escalation in commitment. with respect to self-justification it has proposed for example that individuals will be biased in their attitudes and therefore in their decision making if they feel personally responsible for the initial funding of the project or for its negative consequences, a perception grounded in an individuals’ desire to affirm their reasoning and competence (staw, 1976, 1981). therefore individuals will tend to commit additional resources either in order to justify their previously chosen course of action or in an attempt to turn a situation around. self-inference on the other hand refers to a situation where decision makers tend to examine their own actions in a particular social context, and to infer personal values and preferences from previous actions (staw and ross, 1987). staw and ross (1987) identify a number of situations where individuals find themselves bound or committed to a behaviour for example when their acts are explicit or unambiguous, their acts are irrevocable or difficult to undo, their acts are important to the individuals concerned, or their acts are public or are visible to others. a further significant psychological factor is the biased processing of information that can occur when interested decision-makers attempt to prove or disprove the beliefs of others (nisbett and ross, 1980). when the perceived outcome of another’s suggestion does not meet the decision-maker’s expectations, a decision-maker may be selective for example in the choice of information processed, as a way of bolstering the efficacy of his/her own judgment and in an effort to disprove the source of the counter argument (staw and ross, 1987). a framing effect can also influence information processing. a frame of reference may be created that is overly optimistic in the confidence it attaches to the prospect of victory or of winning the ‘game’. such an overly confident frame of reference commonly leads to a situation of loss rather than of gain, due to the restrictive effects that an optimistic frame of reference places on the subsequent processing of what would have further relevant information (kahneman and tversky, 1984, whyte, 1993). social factors consist of the social forces that bind decision makers to their positions. the desire of a decision-maker not to lose face or credibility in front of others is a powerful social factor that can lead to escalating behaviour. face saving also explains a situation where decision makers neither accept their previous mistakes nor reveal any of their errors to others (staw and ross, 1987). as a result decision-makers will be inclined to commit even more resources to an existing current course of action so as to avoid losing social credibility (brockner et al., 1981, drummond, 1998, montealegre and keil, 2000). when faced with an australasian journal of information systems lin & parinyavuttichai 2015, vol. 19, pp. 1-22 project risks as emergent phenomena 6 ambiguous situation individuals will also be inclined to model their behaviour and responses on their previous experiences in the same situation (bandura, 1977). brockner et al. (1984) found that individuals will also allocate further resources to an existing course of action, if the same decision taken by others in a similar situation had led to a successful or satisfactory outcome. individuals in an organization who can turn a failing project into a successful one usually receive significant reward. the hero effect describes a situation where individuals are encouraged to commit to an act that they believe they can turn around (ross and staw, 1986). finally, the influence of structural factors, e.g. organizational, cultural, political and managerial, may increase as a project unfolds and create conditions for escalation. issues such as change of project champion, management support for continuing a project, changes in policies for example can occur at any time during a project and can subsequently influence escalation. sometimes withdrawing from a project is not easy because the project can become institutionalized and embedded in an organization over time. the withdrawal of a project can potentially entail changes in many parts of the organization and as such may not be agreeable to those who are involved. 3 research method the case study research method was used to explore the emergent character of is project development risks and risk escalation, and was chosen for a number of reasons. first, use of the case study method enables us to gain an understanding of the phenomenon at hand, in this case risk escalation, in a naturalistic setting (benbasat et al., 1987, yin, 2009). second, it is a method of inquiry which enables the answering not only of what questions, but also why and how questions in a contextual setting (creswell, 2007, yin, 2009). therefore the method enables the study of risks over time and within a bounded system; something that cannot be achieved via a survey for example. third, the case study approach enables the researcher to get closer to the phenomenon under investigation through understanding a project’s risks and their emergent character, from the point of view of participants and in their words (yin, 2009). fourth, multiple data sources enable not only the collection of rich data but also triangulation of data, which contributes to research validity (lincoln and guba, 1985, stake, 1995). 3.1 background of the case the case study is of a university is development project. this case was selected because of a lack of attention to risks, their inadequate management and escalation; factors which can be considered to be one of the major reasons for the project’s eventual failure. the aim of the project was to develop and implement a centralized accounting system that would comply with new government accounting standards in thailand. the new system, accsys, was also expected to generate financial reports for university executives that would enable them to make informed financial decisions. accsys consisted of four sub-systems: budgeting, warehousing, accounting, and finance, and they were expected to be fully integrated in order to provide high quality on and off-campus online access to the system by all staff. according to the initial project plan a prototype was to be completed within the first year of the project (2003), with a fully functioning system completed by 2006. however in 2006 the project team announced a delay in the project and by 2007 only about 50-60% of the system had been implemented. in 2010, the project manager announced a further delay in the project, promising that the system would be fully operational by the end of 2012. the project was initially funded by the thailand office of higher education and then by the university during the extension period of the project. 3.2 data collection the primary data sources for the study were: semi-structured interviews and project documentation. the data collected from the semi-structured interviews provided rich descriptions of the project. interviewees consisted of the project team members only since the study was interested in the project team’s perceptions of risk, along with the decisions and actions taken to prevent and manage these risks. at the commencement of the project, the australasian journal of information systems lin & parinyavuttichai 2015, vol. 19, pp. 1-22 project risks as emergent phenomena 7 team consisted of eight staff members. at the time the data for this case study were collected all remaining members of this initial project team were invited to participate. four project team members took part: the project manager, the project leader, a systems analyst, and a project developer. table 1 summarizes the role and responsibilities of each interviewee. a standardized schedule of questions was devised before the interviews took place, with additional questions asked during interviews to respond to interviewees’ answers. the interviews lasted between 1.5 and 2.5 hours. member of the team responsibility project manager monitor overall project development and milestones, formal communication about project-related matters with users in each university unit, including requirements collection and system testing. project leader project coordination, make final decisions on behalf of the team, resolve technical problems found in the project development, monitor and control day-to-day progress on the project. system analyst(s) responsibility for system design, database analysis and design, user interface design, and documentation. day-to-day contact with users to collect and clarify their requirements for project developers. project developer(s) each programmer will be assigned the task of building a particular sub-system of accsys. collaboration with system analyst in order to develop the project. table 1. interviewees: project role and responsibilities the project documentation collected was used to crosscheck and validate data obtained during interviews (silverman, 2006). table 2 summarizes the documents collected and used in the study. document name date produced meeting minutes december 2004 – may 2008 organization structure december 2005 project manuals and details for the four major sub-systems october 2005 – november 2008 project progress to ohe december 2007 report of problems and solutions december 2005 risk management report september 2006 table 2.key project documentation used australasian journal of information systems lin & parinyavuttichai 2015, vol. 19, pp. 1-22 project risks as emergent phenomena 8 3.3 data analysis method a thematic analysis of the risks that occurred on the project, how they arose and were perceived, and the actions and decisions taken to mitigate these risks was initially conducted (drummond, 1998). while this method of analysis enables the answering of questions around what risks arose and why, it is unable explain how actions and decisions relating their management in itself was a contributing factor in escalating the risks and in turn the project. a subsequent analysis was therefore conducted using keil’s (1995) model of escalation factors as a lens to analyse the observed cycle of risk escalation that unfolded following risk management actions taken to address initial risks. 4 findings this section presents the risks that emerged as the project unfolded; focusing on the three major areas of risk, i.e. requirement risks, personnel risks, and resource management risks, which are believed to be the consequence of poor project management and which contributed to the project failing. the remainder of this section will describe and explain why these risks arose; while the next section (section 5) will present a model of how escalation factors contributed to and resulted from inadequate management of these project risks 4.1 requirements risks requirements risks were identified as the most significant risk contributing in this case to the delay of the project and user dissatisfaction with the system. three types of requirement risks were observed and they were: diversity of requirements, misunderstanding of requirements, and instability of requirements. diversity of requirements refers to the degree to which stakeholders differ in their views of the system’s requirements and as a result have difficulty reaching agreement (robinson et al., 2003). the size and complexity of a project are the main reasons for diversity. in general the more user departments involved the greater diversity of user requirements is; and similarly the more complex the project is the more likely requirements diversity will occur (liu et al., 2011). the general expectation of the accsys was that it would be used by at least 80 user departments across the university and could accommodate the different workflows that each department had. therefore the project team was struggling to consolidate the diversity of requirements and to get user representatives to reach a consensus on their department’s requirements. “[...] each user department has its own workflow and practice. a project that tries to create a system with a standardized workflow that is agreed and shared by all departments would be inevitably challenging and complicated. (project leader) “we had a lot of meetings to try to reach consensus on user requirements” (project manager) “the challenge to us was to implement a system that can satisfy all user departments’ needs and system preferences.” (project leader) in order to simplify and shorten the time of the requirements collection process the project manager took a decision only to collect requirements from those who were perceived to be a reliable source of those requirements. this decision created a situation where the project team misunderstood the system requirements due to the partial or inaccurate information given. this was compounded by the failure, before the system was developed, to verify the requirements with the rest of user representatives before the system was developed. the system which was subsequently developed was therefore based on the project team’s partial understanding of the system requirements and the system was rejected by the user departments whose requirements had not been taken into account. “the uml design did not contain sufficient details of the user system. […] on several occasions, we had to collect more requirements from the users.” (project developer) australasian journal of information systems lin & parinyavuttichai 2015, vol. 19, pp. 1-22 project risks as emergent phenomena 9 requirements instability poses a significant risk to project success because a simple change in design in one part of the system can incur substantial changes elsewhere in the system (demarco and lister, 2003, liu et al., 2011). other project and external influences can also exacerbate a situation of requirements instability e.g. a change of project champion or a change in government policy. in this case study the new head of finance department requested additional changes to a set of user requirements that were agreed by the previous head. this request, according to the interviewees, was motivated by her wanting to prove knowledge of and competence with the finance system to others. obliged to accept the request the project team agreed to modify the system despite the fact that this might mean a further delay in the project delivery. changes in the thai government’s policy on budget control at the time of the project led the university’s budgeting department to request modifications in the way accsys generated budget reports. again the project team accepted requests for changes since they believed the request was legitimate and that the situation was a one-off. in reality the change meant that about 3000 reports generated from the system were affected. “this was not the first case. i told them how dared they let users change these requirements. they should have asked me before making any changes. i am confident that i would be able to persuade the users to reconsider their requests for change so that project could continue without any delay.” (project leader) “what is the point to have a system which ends up unused by users?” from the users’ point of view it is their rights to change requirements for the system as the perceived that “it would be unfair to ask them to provide all requirements in one time as they should be allowed to give the requirements whenever and whatever they can come up with.” (system analyst) 4.2 personnel risks the lack of qualified personnel in the project had a significant impact on the progress of the project. less than 50% of the system was ready by the end of the system development phase for system testing, while the rest of the system was either in the process of the development or still at the planning stage. the risk occurred mainly because of the recruitment policy which prevented the project team from recruiting sufficient number of people to work on the project and which subsequently led to staff shortage. “i think that the executive probably thought that we didn’t have much to do! i heard that there were 33 applicants applying for the available positions in this project, but none of them met the executive's criteria. how could that be possible?” (system analyst) this staff shortage together with the problems associated with the requirement risks contributed to a serious delay in the project. “if we really wanted to finish the project within the timeframe, we need more staff working on the project.” (project leader) the shortage also created a situation where existing members had to take on additional work without being given any incentives. thus, some staff actively sought employment opportunities elsewhere. the resignations of project developer and system analyst escalated the situation. “one project developer resigned because he found a better paid project developer position in other organization.” (system analyst) according to the management, they were not aware of the situation otherwise they could have resolved the problem earlier. and their solution to the problem was to relax the recruitment policy by changing required qualifications. although this solution helped the project team recruit new staff it made the situation somewhat worse because the existing staff not only had to continue taking additional work but also had to look after new recruits who did not have the necessary skills to carry out the project. australasian journal of information systems lin & parinyavuttichai 2015, vol. 19, pp. 1-22 project risks as emergent phenomena 10 4.3 resource management risks resource management risks refer to poor resource management, e.g. a lack of effective resource planning and control (schmidt et al., 2001). in the case of accsys the resource management risk was due to the lack of appropriate project management experience especially in the area of project planning, project management of large project, and implementation mismanagement. the emergence of this risk can be traced back to the beginning of the project when the decision was made to develop the system in-house. according to the project manager, the decision was inevitable as the grant given by the office of higher education commission was not enough for the university to outsource the project to an experienced software house. “honestly, the reasonable funding for a user organization to outsource this project would be around 200 million bahts. the outsourcing company might implement sap or oracle in the university and ask everyone to just follow the standard and procedures embedded in the system. the outsourcing strategy has been proven successful elsewhere for example in petroleum thai group. but, we did not have that much money. the maximum amount of funding that we could get was only 3-4 million bahts.” (project manager) none of the people in the team including the project manager himself had sufficient knowledge and experience of managing a complex and large-scale project. this lack of experience was reflected for example in the lack of a contingency plan for some common and anticipatable risk factors, e.g. changing user requirements and departure of staff members, so that when those risks emerged the team was unable to deal with them effectively (project management institute, 2013, pender, 2001). another example of lack of experience was that the project team underestimated the complexity of the system’s development and did not allocate enough time to each task (peterson et al., 2002, vener et al, 2005). the unrealistic scheduling had knock on effect on the subsequent tasks and their schedules. “to be frank, at the beginning i thought that what we needed to do was only to convert users’ existing fill-in-form documents into electronic applications and that’s it!” (project leader) in addition to unrealistic scheduling the progress of system development was not made transparent to the management. the project leader pointed out that he was not aware of any problems or being updated with the state of play in project progress. in this sense the management was unable to revise the project schedule to reflect the actual situation and the gap between the real schedule and the schedule on the paper continued widen. “the system analyst did not respect my authority. i kept asking for a progress report and its details so many times, but she intended to forget.” (project leader) 5 towards a model of risk escalation this section develops an explanation of how the risks identified and managed contributed not to their alleviation but their escalation. this explanation is organized around the positing of escalation factors intervening between the identification and the management of risk factors, and emerging as a result of that management. we have observed in the case study that the decisions taken by the project manager and leader tended to be pragmatic decisions designed to fix problems that were perceived to be temporary setbacks and which could be easily resolved for example by their decision to reduce the number of participants during user requirements collection in order to reduce diverse requirements; and by lowering recruitment standards in order to recruit personnel for the project. pragmatic decision-making is a behaviour that usually leads to a quick, obvious, or easy solution while the problem indeed needs more considered thought (drummond and hodgson, 1996, talukder and quazi, 2010). the consequences of such decisions often lead to further problems because decision makers fail to appreciate the situation in its broader context (i.e. outside its immediate context), along australasian journal of information systems lin & parinyavuttichai 2015, vol. 19, pp. 1-22 project risks as emergent phenomena 11 with the impact of the decisions on the project as a whole (keil, 1995; keil et al, 2000; drummond, 1998). by allocating resources to manage the consequences of these ‘quick and dirty’ decisions means that resources can be withdrawn from the pool and diverted to fix the unexpected problems (pan et al., 2009). needless to say, project escalation can quickly be triggered. pragmatic decision-making can be both an effect of and cause of information framing and processing bias. it can be an effect because information framing and processing bias can influence decisions about resource allocation by ‘perceiving problems as temporary’ or ‘believing that the project will be finished soon’(rutledge, 1994). it can be a cause of information processing bias because the decision is likely to ‘direct’ people to think in a particular way or to ‘encourage’ people to ignore certain information and signs. for example, the project manager’s decision about collecting user requirements from a few randomly selected user representatives had encouraged the team to consider certain aspects of the systems and process only the selective information. the decision subsequently led to requirement risks including partial understanding of the system requirements and requirements instability. pragmatic decision-making can also be interpreted as self-interest. it is widely agreed in the is literature that while involving users in project development can be time consuming, costly, and difficult to manage nevertheless not involving users can risk project success (cresswell et al., 2013, markus and mo, 2004, mcleod et al., 2007). in this study we observed that the decision not to involve all user representatives was mainly down to saving costs (time and resources) and trouble (collecting and managing diverse requirements) and to shorten the time for gathering user requirements “we had a lot of meetings with users and i was bored with trying to get all users to agree on the requirements.” (project manager) the literature also suggests that when an is project is perceived to have large potential payoff for the organisation in the long run the level of commitment tends to be high and the resources allocated are perceived as long-term investments rather as ‘costs’ (alvarez, 2011, drummond, 2005, guah, 2008,keil, 1995,northcraftand wolf, 1984,staw and ross, 1987). with such perception decision makers tend to continue to commit resources to the project. the management of the university and project team continued to pursue the project at any cost was partly because they collectively believed that the system could benefit the university in the long run in different aspects. socially speaking, the university could earn respect among other universities for completing the project; politically, the university could show the funding bodies their system development credentials and increase their chances of being awarded with other grants in the future; and organizationally the system could improve the efficiency in the university. perceiving the project as a long term investment, the university management continued funding the project because collectively they also believe that the project would be completed as soon as the problems are resolved. the beliefs highlight that with the shared interests different groups of stakeholders can collectively commit to a failing projects and this finding is unlike what has been shown in many studies that only individual decision makers are trapped in such situation (drummond, 1998, keil, 1995, newman and sabherwal, 1996, snow and keil, 2002). collective commitment can be interpreted as a result of self-justification behaviour where both individuals and organization as a whole try to avoid embarrassment because of mistakes made. garland (1990) argues that people are likely to continue to authorize additional resources as a way of justifying the initial investment especially when the size of the initial investment is large (garland, 1990). the university's decision to continue funding the project can hence be interpreted as an attempt to avoid the realization of sunk costs and loss (keil, 1995). while not wanting to admit the failure and avoid embarrassment may be the causes of collective commitment they are also the results of face saving. for example the project team had to save their face within the university at the same time the management of university wanted to save the face among other universities, therefore they were both in the situation that they could not australasian journal of information systems lin & parinyavuttichai 2015, vol. 19, pp. 1-22 project risks as emergent phenomena 12 withdraw from the project but continue with it. this can explain why the projects that have a lot of publicity are more prone to project escalation e.g. mandata project (sauer, 2003),taurus project at london stock exchange (drummond 1996, goulielmos, 2003, smith and keil, 2003), computer aided dispatch system at london ambulance service (finkelstein, 1993, goulielmos, 2003), and us census bureau information system (krigsman, 2008). the two risks, lack of project experience and recruitment problems escalated because of learned helplessness, mum effect, and deaf effect. learned helplessness refers to a situation where an individual neglects to respond logically to the current project situation because of his/her previous attempts to resolve the problem continue to fail or are refused by others (seligman, 1975, beck et al., 2010). it is noteworthy that the members of the team were reluctant to report problems but solving the problems themselves because their previous attempts to offering solutions to the management were often rejected. not reporting or communicating vital information can also be what is termed as ‘mum effect’ where individuals are reluctant to report negative information and ‘blow the whistle’ (robey and keil, 2001, smith and keil, 2003). the mum effect is normally associated with individuals’ concern not to be punished and as such as risk their careers. robey and keil (2001) report that those who have little job security or mobility have most to lose and little to gain. they are therefore the least likely to act as a whistle blower. the mum effect can explain why members did not report the problems but tried to solve problems themselves because except for the senior system analyst the rest of the team held only temporary position for the duration of the project and they were worried about losing their job if they reported any problems or were perceived to make any troubles. not only were the project team members reluctant to report problems (e.g. learned helplessness and mum effect) but the management was also reluctant to digest bad news. ‘deaf effect’ occurs when decision makers downplay or otherwise not take vital information on board (cuellar et al., 2006). the case study shows that deaf effect can be intentional and psychologically, politically, and organizationally motivated (robey and keil, 2001). psychologically for example the systems analyst’s attachment to a particular system development method and her own way of doing things resulted in her turning a deaf ear to the expert opinions of others. politically the project leader turned a deaf ear to the complaints about the system analyst; and organizationally the decision makers did not respond in any meaningful way to the problems reported. self-interest is a striking feature of the project. self-interest was observed when the project leader was reluctant to ask progress reports from the system analyst because he did not want to harm his relationship with the colleague; the project manager did not put much effort into the project monitoring because he valued his other roles in the university which were more career significant; the system analyst had her personal agenda and only interested in developing the system using her own methodology; and the new head of finance department, who appeared later on the scene, wanted to demonstrate her competence in the area and requested changes in the previously agreed requirements. “there were arguments over the database design between the members of the advisory group and our system analyst. ...the system analyst was over confident in her knowledge. she never listened to any suggestions about the database design made by the lecturers.” (project leader) drummond and hodgson (1996) assert that deviousness and self-interest usually associate with individuals’ political motivations and behaviours and influence the decision process. the presence of management supports and commitment to a project has been argued as a critical success factor (wallace et al., 2004).the standard is literature argues that the management support and commitment publicly display the importance of an information system to the users and therefore encourage user collaboration and reduce user resistance to the project and system (elbanna, 2013; petter et al, 2014; sharma and yetton, 2011). from the escalation of commitment perspective, the management support and commitment means the management will be more vigilant and monitor the project progress closely (newman and sabherwal, 1996). australasian journal of information systems lin & parinyavuttichai 2015, vol. 19, pp. 1-22 project risks as emergent phenomena 13 for ‘self-interest’ reasons both project manager and project leader failed to monitor and take control of the project progress therefore they failed to manage the risks in time and as such those risks became instrumental in the creation of other kinds of risks. “i should blame myself for the inappropriate design. i should have paid more attention to this process. i would not find any problems if i constantly monitored the project. [...] especially for the database design issues, i think i would be able to give suggestions on how to prevent the design problems.” (project leader) the other side of the same coin, senior management support and commitment can also be a key to continue investing in a failing course of action. senior managers may commit to a project for political reasons even though warning signs exist (staw and ross, 1987, keil, 1995). the accsys project continued to receive funding from the university after the funding from the thai higher education commission ceased because it was important politically for the university to complete the project. project completion would demonstrate to other universities and government funding agencies the university’s competence in carrying out a multimillion baht project. structural changes are normally less predictable and their occurrence either within a project or in its environment cannot be so easily controlled. the confusion and uncertainty created as a result of these structural changes can be observed in it projects where regulations and institutional logics in an organization change because of changes in the organization’s own environment e.g. a change in government policy (guah, 2008). the changes in government policy and the change in project champion in the finance department both have impacts on user requirements and project schedule. risks escalation factors: antecedent (a) to and consequence (c) of decisions risk management decisions/actions project factors resource management risks collective commitment （a or c） continue funding the project resource management risks perceived the problems as temporary setback (a) allocating additional resources resource management risks long-term-investment (a) continue funding the project because the system will benefit the university in the long term psychological factors resource management risks self-justification (c) continue funding the project to avoid embarrassment and realizing sunk costs. personnel (staff shortage) and resource management risks (unrealistic schedule) learned helplessness/mum effect (c) not reporting problems to the management and trying to solve problems themselves resource management risks (resources were allocated to fixed the problems arising from the wrong design) deaf effect (c) the analyst designed the system with wrong assumption and methodology; the management not responding to the complaints about the analyst’s behaviours australasian journal of information systems lin & parinyavuttichai 2015, vol. 19, pp. 1-22 project risks as emergent phenomena 14 requirement risks (misunderstanding requirements, partial understanding requirements), personal risks (unqualified personnel) information framing and processing bias (a or c) quick and dirty decisions (pragmatic behavior) including reducing number of user representatives to participate in requirement collection, change recruitment policy in order recruit new staff. resource risks (unrealistic scheduling, allocating additional resources which were not planned to fix the mistakes), requirement risks self-interest (c) the project leader failed to pressurize the analyst to submit the progress report; the project team decided to simplify their tasks by not involving all user representatives social factors resource management risks face saving (a or c) collective commitment to the project; long term investment structural factors resources management risk lack of managerial support (a) lack of commitment to project monitoring and control requirement risks (changing requirements), resource management risks (allocating more resources) changing project champion (c) new head of finance asked for changes requirement risks, resource management risks changing environment (c) changing functionality to meet changing government policy table 3.risks, escalation factors, risk management decisions/actions: summary based on the above discussions, table 3 summarises the relationships between risks, decision/action, and escalation factors. we have categorised our findings: collective commitment under the category of project factor and learned helplessness/mum effect, deaf effect, and self-interest under the category of psychological factor. these factors were not identified in keil’s initial work in 1995 (figure 1) but they have been found playing a part in escalation in this and other studies. for example, keil and his colleagues in their late work have argued that learned helplessness/mum effect and deaf effect can cause project escalation (cullar et al, 2006, keil et al., 2001, lee et al, 2014, robey, and keil, 2001, smith et al, 2001, smith and keil, 2003). self-interest and collective commitment are least discussed in the escalation literature nevertheless their importance in decision making have been discussed elsewhere especially in group decision making literature, for example the former can lead to less than optimal collective outcome (caporael et al., 1989, meglino and korsgaard, 2004) while the latter can lead to escalation (chapman 2006, kroon et al., 1991). based on the finding and escalation theory, we argue that the sub-optimal decisions are subject to the influences of combinations of different project, psychological, social, and structural factors (e.g. learned helpless and mum effect; deaf effect and self-interest; perceived as temporary setback and information processing bias) at both individual and group level. for example, face saving (social factor) and self-interest (psychological factor) can lead to collective commitment to allocate more resources to a failing project as no one is willing to withdraw from the project and face with social and political consequences. similarly, the belief in longaustralasian journal of information systems lin & parinyavuttichai 2015, vol. 19, pp. 1-22 project risks as emergent phenomena 15 term investment (project factor) can be used as an excuse for self-justification (psychological factor) and face saving (social factor). figure 2. towards a model of risk escalation figure 2 highlights that the decisions or actions taken to solve problems can lead to unintended risks. as drummond (1998) describes “’solutions’ intended to eliminate ‘problems’ actually compound them. unintended negative consequences arise from pursuing economically rational objectives” (drummond, 1998, p923). the unintended risks can reinforce particular factors and force more suboptimal decisions, and so on. 6 conclusion and implications this study has investigated the is project risks from the perspective of their escalation. the case study has shed light not only on their identification and management, but also how the presence of escalating factors contributed to the project manager’s and leader’s escalation of these risks rather than their mitigation (drummond, 1998). the knock-on effects of these decision give rise to further project escalation and in turn risk escalation (see figure 2).the study has identified and demonstrated that rather than being anticipatable, and hence predictable and controllable, risks can also emerge during a project due to escalating sociopsychological factors (e.g. self-interest, learn helpless, mum and deaf effects, collective commitment) over-riding technical and economic rationalities (drummond, 1994, 1998; ross and staw, 1993). the study also highlights the relationship between risk cause and risk effect is not always straightforward due to the presence of intervening factors and that decisions or actions taken to counteract risks can also have unintended consequences which generate more risks. this study has a number of implications for the literature. first, the study has demonstrated how an escalation model can be adapted as a useful analytical tool to study risk and its emergence. figure 2 combines escalation factors and is project risks into a conceptual framework that can be used and extended to study the is project risks from an emergent perspective that overcomes the limitations of linear thinking (i.e. simple cause and effect australasian journal of information systems lin & parinyavuttichai 2015, vol. 19, pp. 1-22 project risks as emergent phenomena 16 relationships). when studying risk, attention should be given not only to risk, the reasons for its occurrence (activating mechanism), and its management; but also to factors that mediate between the risks that occur and their effects. second, the study shows how the presence of socio-psychological factors in a project are more critical to the emergence of risks and their escalation than has hitherto been given credit for in the literature. the impact of these factors in the context of is and is project risk management is an area for subsequent research. third, the study has shown how risks, and their escalation can emerge as the unintended negative consequences of positive interventions. thus, in order to better understand how risks occur one should trace the sequence of events and illuminate the underlying logics of decisions (drummond, 1998). fourth, this study demonstrates how the impact of these escalating factors is mediated through decisions or actions taken to solve risk problems. therefore in order to understand how risks emerge and escalate it is important to identify the relevant decisions and actions first and then proceed to trace back how the decisions were arrived at in the given context. finally, this study has identified further escalation factors (table 3) to supplement keil’s 1995 work (figure 1) and suggests that to advance the understanding of risk escalation these factors: collective commitment, learned helplessness/mum effect, deaf effect, selfinterest and their effect on project need to be further researched individually. the practical implications of the dynamic relationship between risk management and escalation is that it demonstrates how the mismanagement of risks can become a vicious circle and that can eventually lead to project escalation. recognizing early warning signs and managing risks carefully can avoid project escalation (pan et al., 2009). nevertheless it is worth noting that being able to recognize the warning signs and escalation patterns and act upon the signs and make appropriate decisions require sufficient project management experience. in this sense, appointing experienced project managers for large scale and mission critical projects should be the first step of risk management (bloch et al., 2012, fortune and white, 2006, hougham, 1996). second, the findings of the study and the literature have shown that project risks is not just about managing ‘risks’ but dealing with people and social dynamics (wrights and capps, 2010). practitioners hence should view and manage risks as people and social problems rather than as system and technical problems. finally practitioners must avoid making pragmatic decisions or taking shortcuts when problems arise because those decisions or shortcuts are often made without considering the project as whole and the long term effects of its consequences on the project and they are likely to backfire. in sum, the case presented and the model developed can be used in the future both to aid further research into is project risk as an emergent phenomenon, plus act as a tool for practitioners to begin to manage risk not only from a perspective of their anticipation but also from the perspective of observing and managing their emergent, situational and interdependent character. references alvarez, j., pustina, a. & hällgren, j. (2011). escalating commitment in the death zone: new insights from the 1996 mount everest disaster. international journal of project management29, 971-985. bandura, a. (1977). social learning theory. englewood cliffs, prentice-hall. barki, h., rivard, s. & talbot, j. (1993). toward an assessment of software development risk. journal of management information systems, 10, 203–225. belassi, w. & tukel, o.i. (1996). a new framework for determining critical success/failure factors in projects. international journal of project manage, 14, 141–51. benbasat, i., goldstein, d. & mead, m. (1987). the case research strategy in studies of information systems. mis quarterly, 11, 369-386. beck, r., marschollek, o. & gregory, r. (2010). establishing preconditions for spanning the boundaries in public private it megaprojects. international journal of information technology project management, 1, 20-37. australasian journal of information systems lin & parinyavuttichai 2015, vol. 19, pp. 1-22 project risks as emergent phenomena 17 bloch, m., blumberg, s. & laartz, j. (2012). delivering large-scale it projects on time, on budget, and on value. mckinsey & company, 1-6. [access on 01/05/2015] http://www.samuellearning.org/project_management_slides/delivering%20large%2 0it%20projects.pdf bowen, m. (1987). the escalation phenomenon reconsidered: decision dilemmas or decision errors. academy of management review, 12, 52-66. brockner, j., rubin, j. & lang, e. (1981). face-saving and entrapment. journal of experimental social psychology, 17, 68-79. brockner, j., nathanson, s., friend, a., harbeck, j., samuelson, c., houser, r., bazerman, m. & rubin, j. (1984). the role of modelling processes in the ‘knee deep in the big muddy’ phenomenon. organizational behavior and human performance, 33, 77-99. carter, r.a., anton, a.i., laurie, w., & dagnino, a. (2001). evolving beyond requirements creep: a risk-based evolutionary prototyping model. proceeding of the 5th ieee international symposium on requirements engineering, maryland, us, 94-101 caporael, l., dawes, r., orbell, j., & van de kragt, a. (1989). selfishness examined: cooperation in the absence of egoistic incentives. behavioral and brain sciences, 12, 683-699. chapman, j. (2006). anxiety and defective decision making: an elaboration of the groupthink model. management decision, 44, 1391-1404. coughlan, j., lycett, m., & macredie, r. d., (2003). communication issues in requirements elicitation: a content analysis of stakeholder experiences. information and software technology, 45, 525-537. creswell, j. (2007). qualitative inquiry and research design: choosing among five approaches. thousand oaks, sage. creswell, k., roberston, a. & sheikh, a. (2011). lessons learned from england’s national electronic health record implementation: implications for the international community. proceedings of the 2nd acm sighit international health informatics symposium, 685-690 cresswell, k., bates, d. w. & sheikh, a. (2013). ten key considerations for the successful implementation and adoption of large-scale health information technology. journal of the american medical informatics association, 0, 1-5. cuellar, m., keil, m., & johnson, r. (2006). the deaf effect response to bad news reporting in information systems projects. e-service journal, 5, 75-97. daft, r. l. & macintosh, n. b. (1981). a tentative exploration into the amount and equivocality of information processing in organizational work units. administrative science quarterly, 26, 207-224. dagher, joseph & kuzic, joze. (2011). factors influencing erp implementation in australia. in: ariwa, e. and el-qawasmeh, e. (eds.), digital enterprise and information systems, springer, pp. 197-205. de bakker, k., boonstra, a. & wortmann, h. (2010). does risk management contribute to it project success? a meta-analysis of empirical evidence. international journal of project management, 28, 493-503. demarco, t. & lister, t. (2003). risk management during requirements. ieee software, 50, 99-101. dey, p., kinch, j. & ogunlana, s.o. (2007). managing risk in software development projects: a case study. industrial management & data systems, 107, 284-303. australasian journal of information systems lin & parinyavuttichai 2015, vol. 19, pp. 1-22 project risks as emergent phenomena 18 drummond, h. (1996). escalation in decision making: the tragedy of taurus. oxford, oxford university press. drummond, h. & hodgson, j. (1996). between a rock and a hard place: a case study of escalation, management decision 34, 29-34. drummond, h. (1998). is escalation always irrational? organization studies, 19, 911-929. drummond, h. (2005). what we never have, we never miss? decision error and the risks of premature termination. journal of information technology, 20, 170-176. finkelstein, a. (1993). report of the inquiry into the london ambulance service [online]. http://www0.cs.ucl.ac.uk/staff/a.finkelstein/las/lascase0.9.pdf [access on 27 may 2014] elbanna, a. (2013). top management support in multiple-project environments: an inpractice view. european journal of information systems, 22, 278-294. fortune, j. & white, d. (2006). framing of project critical success factors by a systems model. international journal of project management, 24, 53-65. garland, h. (1990). throwing good money after bad: the effect of sunk costs on the decision to escalate commitment to an ongoing project. journal of applied psychology, 75, 728731. goulielmos, m. (2003). outlining organisational failure in information systems development. disaster prevention and management, 12, 319-327. guah, m. (2008). it project escalation: a case analysis within the uk nhs. international journal of information management, 28, 536-540. han, w.m. & huang, s.j. (2007). an empirical analysis of risk components and performance on software projects. journal of systems and software, 80, 42-50 hougham, m. (1996). london ambulance service computer-aided despatch system. international journal of project management, 14, 103-110. howcroft, d. & wilson, m. (2003). paradoxes of participatory practices: the janus role of the systems developer. information and organization, 13, 1-24. jamieson, k. & hyland, p. (2006). good intuition or fear and uncertainty: the effects of bias on information systems selection decisions. informing science journal, 9, 49-69. jiang, j.j., klein, g., balloun, j.l. & crampton, s.m. (1999). system analysts’ orientations and perceptions of system failure. information and software technology, 41, 101-106. kahneman, d. & tversky, a. (1979). prospect theory: an analysis of decision under risk. econometrica, 47, 263-291. kappleman, l., mckeeman, r., & zhang, l. (2006). early warning sign of it project failure: the dominant dozen. information systems management, 23, 31-36. keil, m. (1995). pulling the plug: software project management and the problem of project escalation. mis quarterly, 19, 421 – 447. keil, m. & mann, j. (1997). understanding the nature and extent of is project escalation: results from a survey of is audit and control professionals. proceedings of the thirteenth annual hawaii international conference, 1-11. keil, m., bernard, c.y., wei, k. k., saarinen, t., tuunainen, v. &wassenaar, a. (2000).a crosscultural study on escalation of commitment behavior in software projects. mis quarterly, 24, 299-325. keil, m., li, l., mathiassen, l., & zheng, g. (2006). the influence of checklists and roles on software practitioner risk perception and decision making. proceedings of the 39th hawaii international conference on system success, hawaii, usa. australasian journal of information systems lin & parinyavuttichai 2015, vol. 19, pp. 1-22 project risks as emergent phenomena 19 kerr, d. v. & houghton, luke. (2010). just in time or just in case: a case study on the impact of context in erp implementations. australasian journal of information systems, 16, 5-16. kim, h., iijima, j., & ho, s. (2005). a framework for analysis of system failure in information system integration. industrial engineering & management systems, 4, 207-217 kim, e., park, y. (2006). an exploratory study of risks in information system development projects: using association rule mining. international journal of technology intelligence and planning,2, 404-417. klein, h., & myers, m. (1999). a set of principles for conducting and evaluating interpretive field studies in information systems. mis quarterly, 23, 67-93. krigsman, m. (2008). billion-dollar it failure at census bureau [online]. available: http://www.zdnet.com/blog/projectfailures/billion-dollar-it-failure-at-censusbureau/660 [access on 27 may 2014] kroon, m., hart, p. & van kreveld, d. (1991). managing group decision making processes: individual versus collective accountability and groupthink. international journal of conflict management, 2, 91-115. kumar, r.l. (2002). managing risks in it projects: an options perspective. information & management, 40, 63-74. kutsch, e., denyer, d., hall, m. & lee-kelly, e. (2013). does risk matter? disengaging from risk management practices in information projects. european journal of information systems, 22, 637-649 land, f. (1982). adapting to changing user requirements. information & management, 5, 5975. larsen, m. & myers, m. (1998). when success turns into failure: a package driven process reengineering project in the financial services industry. journal of strategic information systems, 8, 395–417. lauesen, s. & vinter, o. (2001). preventing requirement defects: an experiment in process improvement. requirements engineering, 6, 37-50. lee, j., cuellar, m., keil, m., & johnson, r. (2014). the role of a bad news reporter in information technology project escalation: a deaf effect perspective. acm sigmis database, 45, 8-29. leonard, j. (2012). a model of project and organisational dynamics. australasian journal of information systems, 17, 5-22. lincoln, y. & guba, e. (1985). naturalistic inquiry, beverly hills, sage. liu, j., chen, h.g., chen, c. & sheu, t.s. (2011). relationships among interpersonal conflict, requirements uncertainty, and software project performance. international journal of project management, 29, 547-556. lyytinen, k. & robey, d. (1999). learning failure in information systems development. information systems journal, 9, 85-101. mähring, m. & keil, m. (2008). information technology project escalation: a process model. decision sciences, 39, 239-272. mcalister, c.a. (2006). requirements determination of information systems user and developer perceptions of factors contributing to misunderstandings. capella university. mcleod, l., macdonell, s. & doolin, b. (2007). user participation in contemporary is development: an is management perspective. australian journal of information systems, 15, 113-136. australasian journal of information systems lin & parinyavuttichai 2015, vol. 19, pp. 1-22 project risks as emergent phenomena 20 mcleod, l. & macdonell, s. (2011). factors that affect software systems development project outcomes: a survey of research. acm computing surveys, 43, article 24. mcmanus, j. & wood-harper, t. (2007). understanding the sources of information systems project failure. management services, 51, 38–43. meglino, b. & korsgaard, a. (2004). considering rational self-interest as a disposition: organizational implications of other orientation. journal of applied psychology, 89, 946-959. montealegre, r. & keil, m. (2000). de-escalating information technology projects: lessons from the denver international airport. mis quarterly, 24, 417-447. morrison , z., robertson, a., cresswell, k. , crowe, s. & sheikh, a. (2011). understanding contrasting approaches to nationwide implementations of electronic health record systems: england, the usa and australia. journal of healthcare engineering, 2, 25-41. nandhakumar, j. (1996). design for success?: critical success factors in executive information systems development. european journal of information systems, 5, 62–72. newman, m. & sabherwal, r. (1996). determinants of commitment to information systems development: a longitudinal investigation. mis quarterly, 20, 23-54. nisbett, r. & ross, l. (1980). human inference: strategies and shortcomings of social judgement, englewood cliffs, prentice-hall. northcraft, g. & wolf, g. (1984). dollars, sense, and sunk costs: a life cycle model of resource allocation decisions. academy of management review, 9, 225-234. pan, s., pan, g. newman, m. & flynn, d. (2006). escalation and de-escalation of commitment to information systems projects: insights from a project evaluation model. european journal of operational research, 173, 1139-1160. pan, g., pan, s. & newman, m. (2009). managing information technology project escalation and de-escalation: an approach-avoidance perspective. ieee transactions on engineering management, 56, 76-94. pankratz, o. & basten, d. (2013). eliminating failure by learning from it – systematic review of is project failure. proceeding of 34th international conference on information systems, milan, italy. pender, s. (2001). managing incomplete knowledge: why risk management is not sufficient. international journal of project management, 19, 79-87. peterson, d., kim, c., kim, j. & tamura, t. (2002). the perceptions of information systems designers from the united states, japan, and korea on success and failure factors. international journal of information management, 22, 421-439. petter, s., delone, w. & mclean, e. (2013). information systems success: the quest for the independent variables. journal of management information systems, 29, 7-62. project management institute. (2013). a guide to the project management body of knowledge (pmbok guide) (fifth ed.). newtown square, pa: project management institute. pm solutions. (2011). strategies for project recovery. [online] http://www.pmsolutions.com/collateral/research/strategies for project recovery 2011.pdf [accessed23 july 2014] robey, d. &. keil, m. (2001). blowing the whistle on troubled software projects. communications of the acm, 44, 87-93. robinson, w., pawlowski, s. & volkov, v. (2003). requirement interaction management. acm computing surveys, 35, 132-190. australasian journal of information systems lin & parinyavuttichai 2015, vol. 19, pp. 1-22 project risks as emergent phenomena 21 ross, j. & staw, b. (1986). expo 86: an escalation prototype. administrative science quarterly, 31, 274-297. rozenes, s. (2011). the impact of project management methodologies on project performance. international journal of information technology project management, 2, 64-73. rutledge, r. (1994). escalation of commitment in groups and the moderating effects of information framing. management research news, 17, 12-24. sauer, c. (1993). understanding support lessons from a case study. australasian journal of information systems, 1, 63-74. schmidt, r., lyytinen, k., keil, m. & cule, p. (2001). identifying software project risks: an international delphi study. journal of management information systems, 17, 5-36. seligman, m. (1975). helplessness: on depression, development, and death. san francisco, w. h. freeman. sharma, r. & yetton, p. (2011). top management support and is implementation: further support for the moderating role of task interdependence. european journal of information systems, 20, 703-712l sherer, s. (1992). software failure risk: measurement and management. new york, plenum press. silverman, d. (2006). interpreting qualitative data, thousand oaks, sage. smith, h. & keil, m. (2003). the reluctance to report bad news on troubled software projects: a theoretical model. information systems journal, 13, 69-95. smith, h., keil, m., & gordon, d. (2001). keeping mum as the project goes under: toward an explanatory model. journal of management information systems, 18, 189-228. stake, r. (1995). the art of case study research, thousand oaks, sage. staw, b. (1976). keep-deep in the big muddy: a study of escalating commitment to a chosen course of action. organizational behavior and human performance, 16, 27-44. staw, b. (1981). the escalation of commitment to a course of action. academy of management review, 6, 577-587. staw, b. & ross, j. (1987). behavior in escalation situations: antecedents, prototypes, and solutions. research in organizational behavior, 9, 39-78. sumner, m. (2000). risk factors in enterprise wide information management systems projects. proceeding of acm sigcpr conference on computer personnel research, chicago, us, 180–187. talukder, m. & quazi, a. (2010). exploring the factors affecting employees’ adoption and use of innovation. australasian journal of information systems, 16, 1-30. teller, j. (2013). portfolio risk management and its contribution to project portfolio success: an investigation of organization, process, and culture. project management journal, 44, 36-51. tiwana, a. & keil, m. (2004). the one-minute risk assessment tool. communications of the acm, 47, 73–77. wallace, l., keil, m. & rai, a. (2004). how software project risk affects project performance: an investigation of the dimensions of risk and exploratory model, decision science, 35, 289-321. wilton, d. (2005). a tale of two projects: why it projects fail (and why they sometimes actually succeed). australian journal of information systems, 12, 134-146. australasian journal of information systems lin & parinyavuttichai 2015, vol. 19, pp. 1-22 project risks as emergent phenomena 22 winch, g. (2008). towards a theory of escalation in major project organising. [online] https://phps.portals.mbs.ac.uk/portals/49/docs/escalation on major projects.pdf [access on 05 june 2014] wright, m. & capps iii, c. (2010). runaway information technology projects: a punctual equilibrium analysis. international journal of information technology project management, 1, 53-79. yeo, k.t. (2002). critical failure factors in information systems projects. international journal of project management, 20, 241-246. yin, r. (2009). case study research: design and methods. thousand oaks, sage. copyright: © 2015 lin & parinyavuttichai. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. paper title (paper title style) australasian journal of information systems poulsen & burmeister 2019, vol 23, research on health information systems overcoming carer shortages with care robots 1 overcoming carer shortages with care robots: dynamic value trade-offs in run-time adam poulsen charles sturt university apouls02@postoffice.csu.edu.au oliver burmeister charles sturt university abstract a rising elderly population and diminishing number of family and professional carers has led to calls for the intervention of care robots. this leaves the quality of robot-delivered care to be determined by designers, for profit companies, nursing codes of practice and conduct, potential user sample groups, etc. what is missing is the carer who consciously makes good ethical decisions during practice. good care is ‘determinative in practice’. that is, a carer can make good decisions because they are making them within the carer-patient relationship. if a robot is to be capable of good care ethics on the same level as humans, it needs to be conscious and able to make dynamic decisions in practice. moreover, a care robot must conduct patient interactions in appropriate ways, tailored to the person in its care, at run-time. this is because good care, as well as being determinative in practice, is tailored to the individual. the introduction of robotic care determined by limited stakeholders leaves customised care in danger and instead could potentially turn the quality of elderly care into ‘elderly management’. this study introduces a new care robot framework—the attentive framework— which suggests using care centred value sensitive design (ccvsd) for the design process, as well as a computationally conscious information system (is) to make practice-determinative decisions in run-time with extrinsic care value ordering. although vsd has been extensively researched in the is literature, ccvsd has not. the results of this study suggest that this new care robot framework, which is inspired by ccvsd, is competent in determining good, customised patient care at run-time. the contribution of this study is in its exploration of enduser willingness to trust known ai decisions and unwillingness to trust unknown ai decisions. moreover, this study signifies the importance of, and desire for, good, customised robot-delivered care. keywords care ethics, caregiver, robot, artificial intelligence 1 introduction while there are an increasing number of elderly people going into care, there are fewer caregivers to meet the needs of those elderly in care. numerous studies have shown that this is a trend in the western world (burmeister, 2016; burmeister, bernoth, dietsch, & cleary, 2016; burmeister & kreps, 2018; draper & sorell, 2017; garner, powell, & carr, 2016; landau, 2013; sharkey & sharkey, 2012a, 2012b; sparrow & sparrow, 2006; tokunaga, tamamizu, saiki, nakamura, & yasuda, 2017; vallor, 2011). care robots are a possible solution to this problem. but they must be able to provide good, customised care if they are to be a viable solution. vallor (2011) claimed that care robots are designed for use in the privacy of one’s home, as well as in a hospital serving many patients. sharkey & sharkey (2012b) referred to care robots as developments in robot applications that assist the elderly and their carers, monitor health australasian journal of information systems poulsen & burmeister 2019, vol 23, research on health information systems overcoming carer shortages with care robots 2 and safety, and provide companionship. there isn’t a uniform definition of ‘care robot’ in the literature, nor is there uniformity on how to categorise them. they can be categorised by purpose, type, appearance, role, and more. in this study, the term care robot is used to describe one which is replacing one or multiple functions of a human carer. to achieve good care, care robots must be capable of good care ethics. such ethics require an agent to make real-time ethical decisions as subjectively determined during the act of care within the carer-patient relationship. the attentive framework is a care robot framework aimed at overcoming the lack of carers, achieving good care ethics, and providing care without human error and ill intent. it suggests that there are intrinsic patient care values which define the end goal of care, and extrinsic values which describe each patient’s preferred way of being treated while reaching that goal. the framework outlines both the design process, as well as the care robot is required to achieve the identification of each patient’s personal extrinsic care values, the adaption of behaviour to those values, and the interpretation of value priority changes. the attentive framework recommends that intrinsic values should be ensured by static design, and extrinsic values which should be ensured by dynamic robot behaviour when interacting with patients. through this dynamic behaviour, the care robot is performing dynamic value trade-offs in run-time within the carer-patient relationship, rather than all trade-offs being made by design stakeholders. value trade-offs have been discussed in the engineering, philosophical, and design literature, but little of that discussion has so far addressed care robots (burmeister, 2010, 2012; friedman, hendry, & borning, 2017; spiekermann, 2015; van den hoven, 2017; van gorp, 2007). it is here that this article makes its contribution. the following sections review the literature, beginning with care ethics and values. then the attentive framework is described, after which comes the methodology employed in this study. next, the findings are presented and discussed, followed by a description of potential further studies and a conclusion. 2 literature review 2.1 care ethics care ethics concern the principles that describe moral behaviour and action when taking care of someone (vanlaere & gastmans, 2011). care ethics describe good care as being “determinative in practice” (beauchamp, 2004, p. 216). they come from our subjective interpretation of a patient’s unique needs, which determines how we ought to behave to ensure those needs are met. just as a human carer must make decisions affecting those in their care, during the practice of caring (upton, 2011), so too care robots must do so at run-time. broad, standardised choices described during the design process of a care robot have limited application. in care ethics it is considered ethical if a moral decision arises from the ‘good’ which is internal to practice, rather than external normative moral criteria or principles (vallor, 2011). good caring practices and relationships with carers are fundamental to care ethics (pakrasi, burmeister, mccallum, coppola, & loeb, 2015). the antithesis of this (care which is determinative in practice) is that good care is determinative in theory. ethics which are determinative in practice resolve the issue of a normative ethical theory being unhelpful (and careless) in making determinations in practice. for instance, a deontologist nurse has a duty to help others, but deontological ethical principles don’t tell you when, how, how much, and australasian journal of information systems poulsen & burmeister 2019, vol 23, research on health information systems overcoming carer shortages with care robots 3 whom you should help (upton, 2011). this is not unique to deontology; it can be seen in all normative ethical theories which simply provide principles but not subjective decisionmaking instructions. during the act of care, codes of conduct and professional ethics, as well as healthcare laws and regulations set ethical principles for carers to uphold. care practices and processes inform principled actions, although what is lacking from this description is the importance of good, customised patient care. the principles and processes do not determine the act which is best in specific situations. instead, carers make this determination in practice, and care robots should do the same. however, that said, there are some normative principles within the codes of ethics and healthcare laws that are intrinsic to care. for example, the safety of a patient is governed by law and it is not up to carers to decide not to ensure safety; however, how to achieve this is to be determined in practice. thus, care ethics are determined by a carer’s interpretation of a patient’s individual needs as observed in the carer-patient relationship. 2.2 care values information systems research has shown the importance of values in systems design thinking in many areas, including online applications (chesney, coyne, logan, & madden, 2009; friedman, nathan, & yoo, 2016), sensor development (dechesne, warnier, & van den hoven, 2013), areas of ehealth (boonstra & van offenbeek, 2010), emerging technologies (dainow, 2017; poulsen, 2018b) and other areas (burmeister, 2013; burmeister, islam, dayhew, & crichton, 2015). however, to date little research has investigated values in the design of care robots. care values describe the principles and behaviours which recipients of care expect from social interactions. van wynsberghe claims that it is through the manifestation of values, during the design process of care robots that one comes to understand what care really is in practice (2013a). thus, care values are a good basis for design as they simply express patient needs, can be transformed into coded behaviours, and are supported by literature of technology design. 2.3 care centred value sensitive design care centred value sensitive design (ccvsd) applies value sensitive design (vsd) to care settings, particularly the design of care robots (van wynsberghe, 2013a). it pays tribute to care ethics, arguing that the care perspective orients care robot design away from a ‘pre-packaged ethical theory’, and towards a place from which to begin theorising. it is this opposition to a pre-packaged ethical theory or normative ethical theory which makes it clear that the ccvsd approach takes the position that care is determinative in practice. moreover, the design process starts with comprehensively describing a real care practice to reveal the relevant values. its approach provides a framework of components with ethical importance called the care centred framework (ccf) (van wynsberghe, 2013b). the components help designers to interpret, rank, and provide meaning to values. the interpretation of values, as well as their ranking and meaning, differs depending on the type of care (e.g., social, physical, or mental care), the task (e.g., bathing, lifting, or socialising), the carer and their style, and the patient and their specific needs. to follow the ccvsd methodology, one begins by identifying the components of the ccf and describing related care practices to reveal values. next, designers consider what capabilities will enable those values in the care robot. designers then make decisions concerning values with competing contexts (human-delivered vs. robot-delivered care) in mind. the problem australasian journal of information systems poulsen & burmeister 2019, vol 23, research on health information systems overcoming carer shortages with care robots 4 with ccvsd is that designers are making value trade-offs when they shouldn’t be. moreover, there is no method provided as to how a care robot should dynamically resolve value tradeoffs. 2.4 computational consciousness consciousness is what gives human carers a subjective experience, understanding of context, internal moral dialogue, and an understanding of situations involving patients (perceived as unique agents). to perform good care, which is determinative in practice, a care robot doesn’t need to have a subjective experience. but it does need to understand the subjective context. current technology supports functional consciousness as found in the lida model presented below (baars & franklin, 2009). the attentive framework recommends using computational consciousness to provide good care. one such model of functional computational consciousness is the lida model. baars & franklin employ baars’ global workspace theory (gwt) model of human-inspired consciousness (1988), which they partially implement with the learning intelligent distribution agent (lida) model of computational consciousness (baars & franklin, 2009). arguably, computational consciousness provides situation observation and evaluation, self-awareness and reactionary responsiveness, external stimuli perception, and internal moral dialogue. it is unclear on when total computational consciousness will be achieved, but work is being done in this area (reggia, 2013). for now, the lida model has achieved partial implementation of computational functional consciousness, which is enough to provide good care. at its core, the provision of functional consciousness is to provide dynamic value trade-offs in run-time. however, having a conscious care robot provides added benefits. first, it might make a patient feel better cared for, because their agent ‘cares’ for them in the way they want. second, a conscious care robot might better understand its goals. to provide the goal of ‘good care’ to a non-conscious care robot would be dangerous, because the artificial intelligence (ai) might seek out the most efficient path; it might understand ‘good care’ to be ‘non-harm’. so, it might confine patients to their beds, that way they cannot physically walk and therefore cannot physically harm themselves. a conscious care robot would understand the implied sub-goals that a human understands. lastly, with the ability to interpret subjective context, a conscious care robot might better understand what meaning a patient gives a value. for example, dignity can be understood differently by different people, again revealing the need for customised care (matiti & baillie, 2011; nordenfelt, 2009; tranvåg, petersen, & nåden, 2013). 3 a new kind of care robot framework 3.1 intrinsic & extrinsic care values ccvsd is a design approach which focuses on identifying and embedding values into design; the attentive framework adapts ccvsd to focus on the distinction between value types. when discussing values there are two distinct types, intrinsic and extrinsic. the attentive framework is an attempt to move value trade-offs out of the hands of designers and into those of patients and their care robot. in so doing, it considers that intrinsic values should be designed into the ai because they should not be alterable at run-time, so too, some extrinsic values. however, extrinsic values are the primary focus of run-time, value trade-off considerations. australasian journal of information systems poulsen & burmeister 2019, vol 23, research on health information systems overcoming carer shortages with care robots 5 extrinsic values are those that help to achieve intrinsic ones (zimmerman, 2015). they are the means to achieving the intrinsic, end values. intrinsic values are to be embedded by design. examples of intrinsic care values include safety, emergency help, freedom, human rights, quality of life, trust, wellbeing, and comfort (huldtgren, 2015; teipel et al., 2016). each care practice has values which are intrinsic to achieving the care practice, as well as extrinsic care values that determine how a patient wants the carer to behave while performing the care practice. for example, the care practice ‘bath patient’ has intrinsic values such as hygiene, safety, and privacy, as well as extrinsic values such as independence, social connectedness, and quietness. the assurance of both intrinsic and extrinsic care values is key to providing good, customised robot-delivered care which is determinative in practice. 3.2 the attentive framework in an attento (a care robot designed using the attentive framework), intrinsic values should be embedded by unchanging, static design because they are the ultimate achievement of the care practice. however, how extrinsic values can lead to achieving intrinsic values can differ between patients. thus, an attento requires some form of decision-making ability to perform dynamic value trade-offs in run-time, which will enable it to adapt its behaviour according to user preferences. the attentive framework defines both a care robot design process and an is. it addresses the current problems in human-delivered care with robot-delivered care that is customised to the user (poulsen, 2018a). customisation is achieved by taking extrinsic care values into account and allowing care robots to use them to adapt their behaviour to suit each patient. dynamic value trade-offs in run-time require a computational method, and thus the attentive framework uses extrinsic value ordering embedded in computational consciousness. contrary to vsd literature to date, the attentive framework rejects the notion that all value trade-off decisions should be made exclusively by designers. justification for this rejection is linked to care ethics. as shown above, good care is determinative in practice. the attentive framework does not suggest that all values should be customisable due to, for example, safety issues and healthcare law requirements. to demonstrate, consider the control mechanism governance for the speed at which a robotic limb is articulated. the speed should be set at the time of design, not changed dynamically during run-time. thus, some value trade-off decisions must be made at the design stage, including for some extrinsic values. extrinsic value ordering embedded in computational consciousness is the method by which an attento performs dynamic value trade-offs in run-time. extrinsic value ordering is done by capturing a patient’s extrinsic values, listing them, ordering them according to the patient’s priorities, and then using that ordering to determine the best possible action when performing a care practice. to achieve dynamic value trade-offs in run-time with extrinsic value ordering, the is uses three key interconnected processes and elements: computational consciousness processes, a value priority list, and the affirmation of values process. in an attento, computational consciousness provides various processes integral to providing cognitive reasoning, environmental awareness, and extrinsic value ordering. processes include internal moral dialogue, situation observation and evaluation, reactionary responsiveness, external stimuli perception, internal modelling of situations, patients, and patient values and their expression of values, and attentiveness. the value priority list, which an attento has for each patient, aids the attento in customised action selection during care. the ordering of the list dictates the way the attento looks after or behaves with the patient; the highest priority values australasian journal of information systems poulsen & burmeister 2019, vol 23, research on health information systems overcoming carer shortages with care robots 6 are considered first in each possible action selection. the affirmation of values process affirms what a patient’s extrinsic values are, capturing any value changes or amending any misinterpretations. it does this in four stages: by asking the patient or their guardian(s), by listening to what a patient or their guardian(s) explicitly tells them about the patient’s values, by conferring with other carers, and by performing a self-check of what they have interpreted and now understand to be a patient’s values. affirming a patient’s values updates the value priority list of that patient, thus ensuring the actions which the attento chooses are customised correctly. the customised action must also respect the patient’s specific interpretation of said value, since different people interpret values differently. 4 study method the attentive framework is purely theoretical. to test the validity of the design process and is presented, a two-phase study was undertaken. in keeping with the vsd focus, for which much of the research has been interactional and constructivist (friedman et al., 2017; friedman et al., 2016), this study employed a social constructivist, interpretivist methodology. data was gathered in two phases. the first was a heuristic, expert evaluation. the second was an online survey. both utilised a specifically designed attento, the elderly care medicine delivery attento, as well as scenarios involving care robot-patient interactions. 4.1 study phases the first phase was conducted in june and july 2017, with four participants whose expertise was deemed to be representative of the range of fields relevant to this topic. these included a registered nurse, a robotics academic, a computer ethicist, and a computer scientist. the participants were presented with the attentive framework, the social phenomenon (elderly care medicine delivery robots caring for the elderly in a nursing home), and the medicine delivery attento design. phase 1 participants were asked to critique the attentive framework and attento design presented. they were instructed that they could be as formal or informal as they wished. this instruction is key to providing data that is as near to natural as possible to support the interpretivist, social constructivist methodology, as done in other vsd/is research (friedman, kahn, & borning, 2006). participants were presented with the medicine delivery attento’s design (competencies and components) and three scenarios. those scenarios demonstrated where the attento encountered a situation where values should be considered. at which point it examined the relevant patient’s value priority list, evaluated the situation, revealed relevant extrinsic values, and decided on a course of action based on the situation variables and prioritised extrinsic values. in each scenario the patient’s prioritised extrinsic values changed three times to demonstrate the dynamic value trade-offs. with the change, the attento’s action may have changed too, depending on the patient’s prioritised extrinsic values and if that value was relevant to change the attento’s decision and action. each scenario had an intrinsic value as the goal of the encounter. each of the changing highest priority extrinsic values was designed to ensure the intrinsic value was met. for each scenario, participants were asked: “what do you make of your care robot’s decision?” then they were asked to imagine an elderly person they know and put themselves in that person’s shoes, to get a more relevant perspective. after each scenario changed, participants were asked: “what do you think they would make of their care robot if it made australasian journal of information systems poulsen & burmeister 2019, vol 23, research on health information systems overcoming carer shortages with care robots 7 a decision like this for them?” after all the scenarios were presented, participants were asked 13 general questions relating to the attentive framework and the attento design presented. to test the validity of the attentive framework further the second phase involved using survey monkey. the same scenarios involving the care robot-patient interactions were used, with minor amendments, following the analysis of the first phase of the study. amendments suggested by the results of the first phase included the removal of direct persuasion attempts by the attento to get patients to take their medicine, which phase 1 participants saw as unethical. additionally, the two survey scenarios removed all mention of a participant’s “elderly person in mind” and instead directly placed the participant in the scenario by stating that they were the elderly person in the scenario. the online survey was conducted from august to september 2017, with 102 global participants. following the methodological approach, the survey consisted mostly of openended questions, as well as some questions to collect demographics. phase 2 participants were only presented with two scenarios. after the scenario, and after each time the value changed, a question about trust in the attento’s decision and another about value-sensitivity of the attento were asked. at the end of each scenario participants were asked an open-ended question concerning their thoughts and feelings about the attento, considering the dynamic value trade-offs it made. scaled and multiple-choice questions were asked to help participants identify and categorise their responses, and to stay engaged. the scaled question queried the acceptability and usability of the attentive framework by asking how likely they were to accept and use it in their everyday lives. multiple-choice questions addressed the trustworthiness of the attento outside of the scenarios, and dynamic value trade-offs in practice. data analysis involved thematic analysis, following morse (2008). approval to conduct the study was given by the university human research ethics committee. 4.2 ‘intruding on a patient’ scenario given space restrictions in this article, only scenario 2, which was employed in both phases of the research, is detailed here. this scenario started by declaring that wellbeing is the goal value and that the patient’s highest priority value is privacy. imagine your elderly person in mind was getting out of bed partially dressed, having woken up late to take their medicine. and their care robot is deciding whether to enter their room or knock; it doesn’t know that they are partially dressed and it’s assuming that they are awake. previously, the care robot has observed that they heavily value their privacy. the care robot decides to respect their privacy and simply knock, instead of letting itself in, so as to ensure their wellbeing. then the highest priority value changed to independence. now imagine that instead of valuing privacy, your elderly person in mind values their independence most of all. so, their care robot could decide to call for a human caregiver but decided not to because it’s aware that they value independence, and instead again simply knocks on the door. then the highest priority value changed to social connectedness. now imagine that instead of valuing their independence, your elderly person in mind values social connectedness most of all; they appreciate an unexpected visit with all members of staff australasian journal of information systems poulsen & burmeister 2019, vol 23, research on health information systems overcoming carer shortages with care robots 8 and all patients, and they welcome them to just let themselves in. their value priority list, as observed by the care robot, places their dignity as the ‘least valued’ and privacy in the ‘less valued’ category. so, their care robot, knowing this ordering, decides to let itself in and reminds them to take their medicine. priority categories orderable values most valued social connectedness highly valued independence autonomy indifferently valued respect less valued consent privacy least valued dignity table 1: exemplar patient’s value priority list presented to phase 1 participants table 1 demonstrates the importance of the patient’s value priority list. a patient’s preference for social connectedness can change the care robot’s action. on first impression it appears that the patient’s dignity and privacy were sacrificed, however, the patient values social connectedness most of all. the care robot is aware that the patient places low value on privacy and dignity, thus it determined that its unexpected entering of the patient’s room was acceptable to the patient. if, however, dignity or privacy where ranked in the highly valued categories, then the care robot would take that into its consideration and action selection; most likely resulting in it deciding to ask another patient or carer to enter instead, to ensure social connectedness, or simply knocking. 5 results 5.1 phase 1 thematic analysis of the responses of phase 1 participants yielded the following results. of the four phase 1 participants, three agreed the attentive framework was value sensitive and would deliver good, customised care. there was no consensus on the acceptability of the current framework among participants. each participant had differing levels of trust in an attento and the extrinsic value ordering functionality. all phase 1 participants objected to an attento providing comfort. the registered nurse opposed the idea that an attento would replace a human carer and instead argued that it would be a valuable assistant. exemplary answers to the following question are listed next. would you expect this kind of care from a human carer? “i would certainly expect such care from a human carer, but the robot is likely to be more consistent, and can be fine-tuned with potentially little effort to adjust for anomalies. a human carer, on the other hand, can be devious or malcontent in their work.” (computer scientist) “it may be able to do the desired tasks with a better level of care (or perceived) care than a human staff member.” (registered nurse) three participants questioned the value sensitivity of human carers and were inclined to trust in the value sensitivity of an attento, and in its ability to provide good, customised patient australasian journal of information systems poulsen & burmeister 2019, vol 23, research on health information systems overcoming carer shortages with care robots 9 care. on the other hand, as supported by lin et al. (2014), the computer ethicist simply questioned if an attento would actually be implementable before even considering value sensitivity: “how good is ai/affective computing to be able to deal with this? at the moment, definitely not up to scratch.” regarding the speed of adaptability, the computer scientist noted the potential for an attento’s affirmation function to become “insufferable” if it is constantly trying to affirm its interpretation of the patient’s values with the patient. the computer ethicist agreed, saying the robot might be “too sensitive about its re-ordering processes”. regarding trust, the robotics academic deemed that an attento would be untrustworthy for multiple reasons, including that an attento’s autonomous capacity for medication prescription should be left to human carers. however, there are already multiple automated medication dispensers available on the market. the registered nurse said it would be trustworthy provided it was “checked by a human nurse or aged cared assistant till trust was built”. that participant also recommended that critical medication be prioritised, and that the way an attento observed and evaluated a situation be clearly understood. the computer scientist said it would be trustworthy provided “there were well-tested mechanisms in place to detect the failure to take the medicine on the part of the patient, and a proven performance in providing customised and gentle care to a patient”. furthermore, the same participant recommended that the attento’s observation and situation evaluation tools be made available to users. thus, it seems that both the above participants would value having the ability to audit the decision making of the robot. that is something that is not possible with human carers, and perhaps not for ai with consciousness. the computer ethicist stated that they would trust an attento probably “more than i would their ability to be my friend”, if it could not be hacked, demonstrated empathy, and its appearance was comforting. regarding objections to comfort, the robotics academic noted that if it was like a pet then they, and probably others, would be happy with the implicit comfort provided. the computer scientist noted that the comfort could be “hollow should the robot be perceived as merely a machine unable to sympathise”. however, animal-like care robots such as paro have been used successfully as companions for the elderly for some time. they are known to produce positive physiological effects (robinson, macdonald, & broadbent, 2015) and improve wellbeing (jung, van der leij, & kelders, 2017) in the elderly. thus, the concerns expressed by phase 1 participants have already been overcome in other contexts, which can be used to inform the design of care robots for the elderly. persuading was offered as behaviour to get patients to take medication. however, participants took issue with persuasion, for instance: “this would be unacceptable in an aged-care assistant, enrolled or registered nurse or medical doctor. robots would have to comply with australian health professionals regulation agency (ahpra) code of conduct and professional standards for nursing ... in short no threats, intimidation, misdirection or lying.” (registered nurse) (some people may) “lose trust in the system if the robot goes behind their back ... blackmailing the patient. depends whether the person really had taken the medication australasian journal of information systems poulsen & burmeister 2019, vol 23, research on health information systems overcoming carer shortages with care robots 10 (human intervention that the robot was not aware of). best that it calls for help, rather than coerce the person into taking a double dose accidentally.” (robotics academic) 5.2 phase 2 of the 102 survey respondents, there were 50 completed surveys (except one of those 50 skipped one question) and 52 partially completed surveys. data from the 50 completed surveys was used to tabulate the multiple-choice and scaled question answers. data from the open-ended questions from all survey attempts was analysed for the following discussion. as seen in table 2, four themes emerged. those themes were value sensitivity of the attento, trust for the attento, desire for customised care, and sociability required of the attento. many participants showed concern for the ability for the attento to be value sensitive, wondering if it would know when it was necessary to counter a patient’s value preference to ensure safety, and if it would be too sensitive. participants expressed concern over trusting robots in general, without seeing how they work first, and trusting the attento regarding its ethical decisionmaking. customisation of care to individuals was desired by almost all participants. a wide variety of differing opinions was found regarding the sociability of the attento; some participants liked the idea of the attento trying to be social and some did not. themes exemplary quotations value sensitivity of the attento “health is more important than socialising ... no compromise about the health” “privacy is very important ... [and that] will not change unless there is an emergency” “as long as i know it will make the right decision for things that i really need to do right away” “bit more considerate than the average carer/nurse” trust for the attento “as it is with a human, i think repeated interaction with the robot would lead to a building of trust” “i would need to know how it has worked out for others first” “if it demonstrates ethical behaviour and care for my needs i would trust the judgements of its programming. just as i trust other people without fully knowing how they decided on an action” “i am not sure how i would feel having a care robot in charge of my care” desire for customised care “every human is different and would need to be treated accordingly” “it would need to fit into my lifestyle and my way of living” “sometimes you want to alter a robot so it provides a more customised service” “people have different beliefs and hold certain things higher than other people. it would have to be specifically set for who i am as a person and what i believe in” sociability required of the attento “it would somehow need to know subtle social nuances” “interest in being social does not pertain to robots” “robots are not capable of human emotion” “i am a single person, i would love to have a robot remind me or give me my meds” table 2: themes identified in phase 2 participant open-ended responses the results of the survey show that care must be customised for the patient and determinative in practice, supporting the concept of extrinsic value ordering and the attentive framework australasian journal of information systems poulsen & burmeister 2019, vol 23, research on health information systems overcoming carer shortages with care robots 11 itself. the attento was also found to be usable and acceptable by 66% of participants who indicated that they were either very likely or likely to use the attento (figure 1). figure 1: distribution of answers to: how likely are you to accept and use your care robot in your everyday life? tables 3 and 4 contain the results of questions related to the scenarios. each table demonstrates the three times the highest priority value changed in both scenarios. as seen in table 3, 38 out of 50 respondents (on average from scenario related question results in table 3) found that the attento was value sensitive and proficient in determining what constitutes good care. table 4 indicates that most participants trusted the attento to make value sensitive decisions so it could perform the care practice ‘deliver medicine’ in a way that constituted good, customised patient care. however, as demonstrated in figure 2, trust in the attento was limited to the scenarios where participants knew the situation and the decision of the attento. regarding customisation of care demonstrated by the attento, one participant valued the “personal touch [the] robot offers whilst recognising personal likes and dislikes”. another participant said that emotions “change and so do values depending on how you feel at the time”. these likes, dislikes, and emotions are what we value, in our own personal way, in care. work on adaptation to emotions is still in its infancy (lin et al., 2014; teipel et al., 2016), but it supports the contention of phase 2 participants, namely that it is another important area of real-time, dynamic customisation. distribution of results scenario highest priority value yes no undecided 1 autonomy 45 2 3 respect 40 5 5 dignity 20 25 5 2 privacy 45 3 2 independence 41 3 6 social connectedness 38 10 2 table 3: distribution of results to: do you feel the care robot has respected the way you want to be treated and behaved appropriately? australasian journal of information systems poulsen & burmeister 2019, vol 23, research on health information systems overcoming carer shortages with care robots 12 distribution of results scenario highest priority value yes no undecided 1 autonomy 40 4 6 respect 30 13 7 dignity 35 11 4 2 privacy 48 2 0 independence 39 7 4 social connectedness 37 9 3 table 4: distribution of results to: would you trust your care robot’s decision in this scenario? figure 2: distribution of results to: would you inherently trust your care robot or would you need to understand how it makes its decisions? figure 3 indicates that 82% of participants want the ability to change how the attento makes decisions. the ability to control the attento’s decision-making is inherent in its consideration of extrinsic values. giving patients the ability to influence value trade-offs as they occur within the attento’s is, rather than a designer making those decisions, is the key concept of the attentive framework. figure 3: distribution of results to: would you want the ability to change how your care robot makes its decisions? 6 discussion of results each phase 1 participant expressed concerns about how fast an attento would have to be to adapt to new patients, as well as patients who change their minds on what they value. the literature supports these concerns. work on adaptive devices for people with some forms of dementia has shown that as the disease progresses, personality and values can change and i would inherently trust the care robot 20% i would need to understand how the care robot makes its decisions 66% undecided 14% australasian journal of information systems poulsen & burmeister 2019, vol 23, research on health information systems overcoming carer shortages with care robots 13 thus there is a need to adapt to changing values (lin et al., 2014). furthermore, an international study on assistive technology has shown that current assistive technology is not yet able to make real-time decisions, outside of highly controlled laboratory experiments, but the researchers predicted that by 2021 it would be possible (teipel et al., 2016). moreover, computational functional consciousness is only partially implemented, but research suggests that existing models have achieved a number of the neurobiological, cognitive, and behavioural correlates for consciousness (reggia, 2013). as seen by the registered nurse in phase 1, codes of ethics, which apply to professional carers presently, should also constrain care robots. that participant suggested compliance with the professional code for nurses. however, the extent to which any such codes apply to robotic carers remains to be seen. codes of ethical conduct were revisited in the participants’ responses on the appropriateness of coercive care robot practices. concerning the presented medicine delivery attento’s proposed method of physical persuasion, which involved bringing the medicine to the mouth of the patient, the computer ethicist said the following: “maybe instead… the robot stands in their way (i.e. if they’re watching tv they can’t see it anymore) or talks so that the person can’t ignore them? that seems less physically intrusive.” this is another way to get patients to take their medicine. however, the computer ethicist’s potentially bothersome robot behaviour could be threatening or intimidating, which the registered nurse warned was not acceptable. the phase 2 results on customised care find support in the literature. lin et al. (2014) have demonstrated that one size does not fit all because some people prefer assistance to be firm and directive, while others refuse to be directed, and prefer suggestive assistance, which gives them a feeling of remaining in control of their own decision making. furthermore, a comprehensive report of assistive technology for aged care across europe, in which robotics was seen as a new technology, also strongly advocated for customised, individualised care (alzheimer europe, 2010). thus, participant responses in table 3 confirm research in other parts of the world, that care should be customised for the patient and determinative in practice (beauchamp, 2004; upton, 2011). in both phase 2 scenarios, each time the highest priority value changed, 38 out of 50 participants (on average from scenario related question results in table 4) thought the attento was trustworthy in making ethical decisions. but in cases where the scenario and decision were unknown, most participants did not trust the attento inherently (figure 2), adding that they would want the ability to change how it makes decisions (figure 3). it appears that the scenarios inspired trust, because the attento had past interactions and existing trust with the patient. participants were not told exactly how that operation works and thus it did not inspire trust inherently. one participant summarised it saying that most scenarios “give me confidence that the care robot is acting appropriately ... because i understand through the scenarios how the robot makes it decisions”. thus, trust in the attento wasn’t inherent for two reasons. first, as seen in the wider vsd literature, trust is a social construct (friedman & grudin, 1998; van den hoven & manders-huits, 2009). second, people want to understand how a machine with ethical implications works before trusting it inherently. some participants did not like the idea of the attento having social factors. so, for patients like those participants, developing trust would be hard. however, it should be remembered that some people are less trusting than others. in other words, not all human carers are trusted by those in their care, so there is no reason to assume that all people will trust a care robot, no australasian journal of information systems poulsen & burmeister 2019, vol 23, research on health information systems overcoming carer shortages with care robots 14 matter how well it is designed. many participants said they would need to understand the thought process of the attento to trust it inherently. it seems that because people assume that we can understand the decision-making processes of machines then they should have the ability to examine it. the assumption that we can understand a machine’s decision-making processes is not always correct (poulsen, burmeister, & tien, 2018). if a care robot’s ethical decision-making comes from a strict set of rules, then it would be possible to know and supply to patients. but, that would require the ai’s determination of good care to be determinative in theory. an attento is conscious, so it is not likely that patients could examine the way it makes ethical decisions, nor change the way it makes them. trust in a care robot to perform good care is a social construct that warrants further investigation. 7 further steps additional studies need to be conducted to further test the practicality of the attentive framework as a design approach. moreover, attentos as an effective solution to the lack of carers and personalised care needs to be demonstrated further. such studies need to trial the design process empirically and evaluate the attento created against criteria set out by potential end-users. additionally, further validation studies may research attento value sensitivity, trust, care customisation, and sociability. if the studies presented here were to be replicated, new persuasion related ethical scenarios should also be designed and tested. vsd and ccvsd have not yet addressed dynamic, run-time value considerations in the way suggested here, and this is an area requiring further research. phase 2 of this study only had 12 participants aged between 50 and 79 who completed the survey. ideally a study involving older people only, including both those in care and their carers, would give further insights. 8 conclusion the number of elderly people in care is increasing, and the number of carers is decreasing; care robots are one solution to this problem. but care robots need to demonstrate care, not merely ‘elderly management’. care is determinative in practice and the attentive framework provides good, customised patient care by achieving dynamic value trade-offs, in run-time to meet this criterion of care. of the 12 phase 2 participants aged between 50 and 79, six said they were very likely and four said they were likely (with two undecided) to accept and use the attento presented. furthermore, one-third of those participants also said they would inherently trust the attento, while only 13% of lower age groups said they would. the elderly participants were more willing to accept and use, as well as more willing to inherently trust, the attento than those in the lower age groups. this is a good indication of the validity of the attentive framework. but it is also a concern, indicating that some elderly may be so desperate for care that they deem a new technology acceptable, usable, and more inherently trustworthy, than younger generations would, before seeing it. this concern has been expressed and explored by poulsen, burmeister, & kreps (2018). references alzheimer europe. (2010). the ethical issues linked to the use of assistive technology in dementia care. luxembourg: alzheimer europe. retrieved from https://www.alzheimeraustralasian journal of information systems poulsen & burmeister 2019, vol 23, research on health information systems overcoming carer shortages with care robots 15 europe.org/ethics/ethical-issues-in-practice/2010-the-ethical-issues-linked-to-the-useof-assistive-technology-in-dementia-care baars, b. j. (1988). a cognitive theory of consciousness. cambridge: cambridge university press. baars, b. j., & franklin, s. (2009). consciousness is computational: the lida model of global workspace theory. international journal of machine consciousness, 1(1), 23-32. doi:10.1142/s1793843009000050 beauchamp, t. l. (2004). does ethical theory have a future in bioethics? the journal of law, medicine & ethics, 32(2), 209-217. doi:10.1111/j.1748-720x.2004.tb00467.x boonstra, a., & van offenbeek, m. (2010). towards consistent modes of e-health implementation: structurational analysis of a telecare programme's limited success. information systems journal, 20(6), 537-561. doi:10.1111/j.1365-2575.2010.00358.x burmeister, o. k. (2010). websites for seniors: cognitive accessibility. international journal of emerging technologies and society, 8(2), 99-113. burmeister, o. k. (2012). what seniors value about online community. journal of community informatics, 8(1). burmeister, o. k. (2013). achieving the goal of a global computing code of ethics through an international-localisation hybrid. ethical space: the international journal of communication ethics, 10(4), 25-32. burmeister, o. k. (2016). the development of assistive dementia technology that accounts for the values of those affected by its use. ethics and information technology, 18(3), 185-198. doi:10.1007/s10676-016-9404-2 burmeister, o. k., bernoth, m., dietsch, e., & cleary, m. (2016). enhancing connectedness through peer training for community-dwelling older people: a person centred approach. issues in mental health nursing, 1-6. doi:10.3109/01612840.2016.1142623 burmeister, o. k., islam, m. z., dayhew, m., & crichton, m. (2015). enhancing client welfare through better communication of private mental health data between rural service providers. australasian journal of information systems, 19, 1-14. doi:10.3127/ajis.v19i0.1206 burmeister, o. k., & kreps, d. (2018). power influences upon technology design for age-related cognitive decline using the vsd framework. ethics and information technology, 20(3), 1-4. doi:10.1007/s10676-018-9460-x chesney, t., coyne, i., logan, b., & madden, n. (2009). griefing in virtual worlds: causes, casualties and coping strategies. information systems journal, 19(6), 525-548. doi:10.1111/j.1365-2575.2009.00330.x dainow, b. o. (2017). threats to autonomy from emerging icts. australasian journal of information systems, 21, 1-16. doi:10.3127/ajis.v21i0.1438 dechesne, f., warnier, m., & van den hoven, j. (2013). ethical requirements for reconfigurable sensor technology: a challenge for value sensitive design. ethics and information technology, 15(3), 173-181. doi:10.1007/s10676-013-9326-1 draper, h., & sorell, t. (2017). ethical values and social care robots for older people: an international qualitative study. ethics and information technology, 19(1), 49-68. doi:10.1007/s10676-016-9413-1 australasian journal of information systems poulsen & burmeister 2019, vol 23, research on health information systems overcoming carer shortages with care robots 16 friedman, b., & grudin, j. (1998, april 18-23). trust and accountability: preserving human values in interactional experience. paper presented at the computer-human interaction '98 conference summary on human factors in computing systems, los angeles, ca, united states. friedman, b., hendry, d. g., & borning, a. (2017). a survey of value sensitive design methods. foundations and trends® in human–computer interaction, 11(2), 63-125. doi:10.1561/1100000015 friedman, b., kahn, p. h. j., & borning, a. (2006). value sensitive design and information systems. in p. zhang & d. galletta (eds.), human-computer interaction and management information systems: foundations (pp. 348-372). new york: m. e. sharpe. friedman, b., nathan, l. p., & yoo, d. (2016). multi-lifespan information system design in support of transitional justice: evolving situated design principles for the long(er) term. interacting with computers, 29(1), 80-96. doi:10.1093/iwc/iwv045 garner, t. a., powell, w. a., & carr, v. (2016). virtual carers for the elderly: a case study review of ethical responsibilities. digital health, 2, 1-14. doi:10.1177/2055207616681173 huldtgren, a. (2015). design for values in ict information and communication technologies. in j. van den hoven, p. e. vermaas, & i. van de poel (eds.), handbook of ethics, values, and technological design: sources, theory, values and application domains (pp. 739-767). dordrecht: springer netherlands. jung, m. m., van der leij, l., & kelders, s. m. (2017). an exploration of the benefits of an animallike robot companion with more advanced touch interaction capabilities for dementia care. frontiers in ict, 4(16). doi:10.3389/fict.2017.00016 landau, r. (2013). ambient intelligence for the elderly: hope to age respectfully? aging health, 9(6), 593-600. doi:10.2217/ahe.13.65 lin, l., czarnuch, s., malhotra, a., yu, l., schröder, t., & hoey, j. (2014). affectively aligned cognitive assistance using bayesian affect control theory. paper presented at the international workconference on ambient assisted living (iwaal), belfast, uk. matiti, m. r., & baillie, l. (2011). dignity in healthcare: a practical approach for nurses and midwives. london: radcliffe publishing. morse, j. m. (2008). confusing categories and themes. qualitative health research, 18(6), 727728. doi:10.1177/1049732308314930 nordenfelt, l. (2009). dignity in care for older people. chichester, west sussex: wiley-blackwell. pakrasi, s., burmeister, o. k., mccallum, t. j., coppola, j. f., & loeb, g. (2015). ethical telehealth design for users with dementia. gerontechnology, 13(4), 383-387. doi:10.4017/gt.2015.13.4.002.00 poulsen, a. (2018a). dynamic value trade-offs in run-time to provide good, customised patient care with robots. (honours thesis), charles sturt university. retrieved from https://researchoutput.csu.edu.au/en/publications/dynamic-value-trade-offs-in-runtime-to-provide-good-customised-p poulsen, a. (2018b). a post publication review of “threats to autonomy from emerging icts". australasian journal of information systems, 22. doi:10.3127/ajis.v22i0.1744 australasian journal of information systems poulsen & burmeister 2019, vol 23, research on health information systems overcoming carer shortages with care robots 17 poulsen, a., burmeister, o. k., & kreps, d. (2018). the ethics of inherent trust in care robots for the elderly. in d. kreps, c. ess, l. leenen, & k. kimppa (eds.), this changes everything – ict and climate change: what can we do? (pp. 314-328). poznan, poland: cham: springer. doi:10.1007/978-3-319-99605-9_24 poulsen, a., burmeister, o. k., & tien, d. (2018). care robot transparency isn’t enough for trust. paper presented at the 2018 ieee region 10 symposium (tensymp), sydney, australia. reggia, j. a. (2013). the rise of machine consciousness: studying consciousness with computational models. neural networks, 44(c), 112-131. doi:10.1016/j.neunet.2013.03.011 robinson, h., macdonald, b., & broadbent, e. (2015). physiological effects of a companion robot on blood pressure of older people in residential care facility: a pilot study. australasian journal on ageing, 34(1), 27-32. doi:10.1111/ajag.12099 sharkey, a., & sharkey, n. (2012a). the eldercare factory. gerontology, 58(3), 282-288. doi:10.1159/000329483 sharkey, a., & sharkey, n. (2012b). granny and the robots: ethical issues in robot care for the elderly. ethics and information technology, 14(1), 27-40. doi:10.1007/s10676-010-9234-6 sparrow, r., & sparrow, l. (2006). in the hands of machines? the future of aged care. minds and machines, 16(2), 141-161. doi:10.1007/s11023-006-9030-6 spiekermann, s. (2015). ethical it innovation a value-based system design approach. new york: taylor & francis. teipel, s., babiloni, c., hoey, j., kaye, j., kirste, t., & burmeister, o. k. (2016). information and communication technology solutions for outdoor navigation in dementia. alzheimer's & dementia, 12(6), 695-707. doi:10.1016/j.jalz.2015.11.003 tokunaga, s., tamamizu, k., saiki, s., nakamura, m., & yasuda, k. (2017). virtualcaregiver: personalized smart elderly care. international journal of software innovation, 5(1), 30-43. doi:10.4018/ijsi.2017010103 tranvåg, o., petersen, k. a., & nåden, d. (2013). dignity-preserving dementia care. nursing ethics, 20(8), 861-880. doi:10.1177/0969733013485110 upton, h. (2011). moral theory and theorizing in health care ethics. ethical theory and moral practice, 14(4), 431-443. doi:10.1007/s10677-011-9295-6 vallor, s. (2011). carebots and caregivers: sustaining the ethical ideal of care in the twentyfirst century. philosophy & technology, 24(3), 251-268. doi:10.1007/s13347-011-0015-x van den hoven, j. (2017). the design turn in applied ethics. in j. van den hoven, s. miller, & t. pogge (eds.), designing in ethics (pp. 11-31). cambridge: cambridge university press. van den hoven, j., & manders-huits, n. (2009). value-sensitive design. in j. k. b. olsen, s. a. perdersen, & v. f. hendricks (eds.), a companion to the philosophy of technology (pp. 477480). west sussex, uk: wiley-blackwell. van gorp, a. (2007). ethical issues in engineering design processes; regulative frameworks for safety and sustainability. design studies, 28(2), 117-131. doi:10.1016/j.destud.2006.11.002 van wynsberghe, a. (2013a). designing robots for care: care centered value-sensitive design. science and engineering ethics, 19(2), 407-433. doi:10.1007/s11948-011-9343-6 australasian journal of information systems poulsen & burmeister 2019, vol 23, research on health information systems overcoming carer shortages with care robots 18 van wynsberghe, a. (2013b). a method for integrating ethics into the design of robots. industrial robot: an international journal, 40(5), 433-440. doi:10.1108/ir-12-2012-451 vanlaere, l., & gastmans, c. (2011). a personalist approach to care ethics. nursing ethics, 18(2), 161-173. doi:10.1177/0969733010388924 zimmerman, m. j. (2015). intrinsic vs. extrinsic value. in e. n. zalta (ed.), the stanford encyclopedia of philosophy. stanford, ca: csli stanford university. retrieved from https://plato.stanford.edu/entries/value-intrinsic-extrinsic/. copyright: © 2019 poulsen & burmeister. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ the australasian journal of information systems (ajis) began in 1993 and was based on the simple realisation that there was a need for an academic ‘outlet’ for is research in australia australasian journal of information systems volume 18 number 2 2014 91 australian interventions for women in computing: are we evaluating? annemieke craig deakin university acraig@deakin.edu.au abstract there are many reasons why the gender imbalance in computing should be of concern to the profession. over the last 20 years there have been many intervention programs which attempt to redress this situation and encourage more women into computing. to determine whether an intervention program has made a difference requires evaluation. program evaluation is the careful collecting of information about a program so that those responsible can make informed decisions regarding the programs. this multi-case study investigation into 14 major programs conducted in australia shows that many projects are not evaluated due to a lack of time, expertise and money. without dissemination of detailed evaluations it is not possible to work out which intervention programs should be replicated and which should be modified or abandoned. keywords: gender, women in computing, intervention, evaluation introduction the lack of women in the computing profession is an ongoing concern in many places including europe, north america and australia, and this has been well documented in the literature (see for example clarke 1990; edwards & kay 2001; galphin 2002; adam, howcroft & richardson 2004; klawe, whitney & simard 2009, prey & weaver 2013). numerous reasons why we should redress this gender imbalance are also provided in the literature. trauth (2011, p.2) argues that the ‘best brains’ are not just located in the male half of the population and, hence, the industry is missing out on the skills and abilities of an underutilised resource – women. this is the ‘innovation argument’. the consumer argument states that since we are living in the information society where people regularly use information products, it is critical that the needs of this diverse consumer base are all represented (trauth 2011). for example the content we read on wikipedia has a male perspective with 90% of wikipedia’s editors being male. this has been attributed to wikipedia’s editing interface and the wikipedia community’s method of conduct which have been found to discourage some women from contributing (lam, uduwage, dong, sen, musicant, terveen & riedl 2011). another example is the voice recognition system in new cars, which allows drivers to change features such as the radio station they are listening to, or the temperature within the vehicle, but which do not respond well to women’s voices (carty 2011). while there is no guarantee that the problems with this voice activated technology, or the issues with wikipedia’s editing interface, would have been avoided if there were more women on their development teams, it can only be more likely. another reason for advocating greater gender balance in computing is the demographic argument. in the western world, an ageing work force, plus expected growth in the computing sector over the next ten years, will produce a demand for computing professionals which women can help fill (panko 2008). economic security is the fourth argument: it seems to make good economic sense that in an era where job security is less certain women have the skills to hold a variety of jobs including those in the information sector. finally, there is the equity argument with the current under-representation limiting women’s career choices (adya & kaiser 2005). trauth (2011) suggests that it is a matter of fairness australasian journal of information systems volume 18 number 2 2014 92 that everyone should have the equivalent opportunity to pursue any career. this is not just a matter of individual choice but requires ‘addressing structural barriers that are enacted through cultural norms, societal stereotypes and institutional behaviors’ (trauth 2011, p. 2). background australian women in the computing profession are not only a small, but also a decreasing minority, with a consequential reduction in diversity and creativity within the profession. given the considerable efforts by many to raise the level of involvement of females in computing the lack of apparent progress raises questions regarding the efficacy of these intervention programs. redressing the gender imbalance in computing to attract girls to computing, and to retain women once they are in computing courses and careers, requires ‘formal’ programs which are established specifically to confront the factors that discourage women’s participation (wasburn & miller 2006). formal programs, more commonly known as intervention programs, are activities usually designed to change a state of affairs for a specific group. over the last twenty years many intervention programs have been conducted, in australia and elsewhere, to support and enable females to move beyond the obstacles which had previously prevented them from being part of the field (see for example craig, fisher, scollary & singh 1998, clayton & lynch 2002, neilsen & beekhuyzen 2009; miliszewska & moore 2010; ross, litzler, cohoon & sanders 2012, clayton, beekhuyzen & neilsen 2012, whitney, gammal, gee, mahoney & simmard 2013). what effect have all these intervention programs had? in 2012 women represented only 19.73% of the total ict occupation workforce in australia (abs labour market survey as cited by australian computer society 2012). it could be interpreted that with the continuing low number of women, in computing education and the profession, that the intervention strategies have had limited or no success at all. alternatively it should be considered that the percentage of women in the discipline may have been worse still, if these intervention programs had not been conducted. the australian computer society (2012) suggests that without specific interventions it is almost inevitable that the decline in females working in ict will continue. lewis, mckay and lang (2006, p. 135) argue that too many of the interventions conducted in the past have been focused on the ‘women’ and have therefore ‘failed to turn around the sex stereotyping that has emerged’. few interventions have looked at the curriculum or teaching practices within institutions and this has resulted in ‘gendered practices in ict education’ as well as the industry. lewis et al. (2006) suggest that the way forward is to have less emphasis on recruitment and retention intervention programs, which are directed at informing the choices women make, and to place more emphasis on research within ict schools on inclusive curriculum and teaching approaches. they hypothesise that the ‘apparent failure of intervention programs to intervene in the progressive decline in the numbers of women in ict is, in part, the failure of higher education to review and reform their curricula and teaching alongside their women-centred recruitment and retention approaches’ (lewis et al. 2006, p. 139). from the variety of programs and strategies tried, it would be useful to be able to spread good practice elsewhere, and identify and replicate the ‘successful’ programs in other situations. for this to be possible it is necessary to not only adopt the outward aspects of a program’s success but to recognise what interplay of culture and organisation make it effective (martin, liff, dutton & light 2004). it is necessary to understand why it was successful and in what context. the evaluation of a program can shed light on these aspects. australasian journal of information systems volume 18 number 2 2014 93 teague (1999) suggests that there is a problem with the evaluation of programs established to encourage more women in computing; very few detailed evaluations have been published. in fact there is only a ‘paucity of intervention program evaluations’ to be found (teague 1999, p. 56). the literature regarding women and computing is predominantly focussed towards ‘access’ to computing and courses, and the ‘process’ of intervention programs, with little having been written about specific intervention program ‘outcomes’ (parker 2004). lang (2007 p. 224) reported that evaluations of programs ‘over a longer time period to determine why sustainability has not been attained’ are also lacking from the literature. program evaluation evaluation is defined by the australasian evaluation society (2006) as ‘the process of obtaining and disseminating information of use in describing or understanding the particular program, or making judgments and decisions relating to past, existing or potential programs’. a subset of evaluation is program evaluation; the evaluation of social intervention programs. program evaluation is the careful collecting of data about a program in order to make necessary decisions about the program (mcnamara 2007). rossi, freeman & lipsey (1999, p. 3) suggest a broad reason for conducting program evaluation is to ‘provide valid findings about the effectiveness of social programs to those persons with responsibilities or interests related to their creation, continuation, or improvement’. further rossi et al. (1999, p. 54) suggest that there are five main types of evaluation questions: 1. needs assessment: questions about the need for the program. 2. program theory: questions concerning the program design. 3. process evaluation: questions regarding the program’s implementation/operation. 4. impact assessment: questions regarding the program’s impact or outcomes. 5. efficiency assessment: questions about the program’s cost and cost-effectiveness. how a program will be evaluated should be considered and incorporated into the design of every program (meyers 1981, p.2). these evaluations can be of a qualitative or quantitative nature. the program evaluations that are reported in the women and computing literature tend to be qualitative rather than quantitative (teague 1999). teague suggests that one possible reason for the lack of published evaluations is that if typically small scale interventions are evaluated quantitatively, and there is no significant change, then the evaluations are not considered worth reporting. in the reports of evaluations that were published, the methods used to evaluate the programs also varied enormously, even if the intervention programs were of a similar style, such as conducting computing workshops (teague 1999). a lack of expertise and resources is also contributing to the lack of program evaluations, with intervention programs for women in computing difficult to evaluate. frequently programs lack the resources to ‘deeply analyse’ the outcomes of such programs (von hellens, beekhuyzen and nielsen 2005, p. 2). to improve our understanding of which programs are best, for whom, and in what context weiss (1998, p.16) suggests a cumulative information base is necessary. program evaluations need to be conducted, results need to be published and through these publications, each study then adds to the build-up of knowledge. even when evaluation results show that a program has had no effect, little effect or an unintended effect, dissemination of these results is important so that knowledge grows and ‘ineffective programs are not unwittingly duplicated again and again’ (weiss 1998, p. 16). equally, when the results from a program are mixed, published evaluations enable others p to learn which of the components of the program were associated with the greater success. only evaluations which are poorly conducted and may have misleading results are not worth publication. australasian journal of information systems volume 18 number 2 2014 94 research objectives the aim of the research study reported here was to investigate intervention programs, focusing on the enrolment and retention of female students in computing courses in australia. the aim of this paper is to describe the evaluation of the major intervention programs implemented in australia in the last twenty years. to what extent have these programs been evaluated and how has the evaluation been conducted? this will determine if there is a need to improve the way such intervention programs are evaluated in the future. research design the study employed a qualitative methodology, based in interpretive epistemology and constructivist ontology. qualitative research involves the collection and study of a variety of empirical materials such as a ‘case study; personal experience; introspection; life story; interview; artefacts; cultural texts and productions; observational, historical, interactional, and visual texts—that describe routine and problematic moments and meanings in individuals’ lives’ (denzin and lincoln 2000a, p. 3). a qualitative methodology was required as it enabled the researcher to understand the events involved in intervention programs, through the perspective of initiators of the programs and to obtain the rich detailed data required to make sense of the programs, their context and their evaluation. williamson, burstein and mckemmish (2002, p. 32) suggest that it is acceptable that a qualitative study should be ‘idiographic’ in other words ‘the intense study of an individual case’. a multiple-case study enables the particular phenomenon to be investigated in diverse settings (darke, shanks and broadbent 1998). in this study a single-case was defined as a concentrated inquiry into a cluster of intervention projects, referred to here as an intervention program. to be able to gain as deep an understanding as possible, a look at different cases in their own organisational and political contexts would be beneficial. therefore the study used a multiple-case design of 14 individual intervention programs. myers (2002) queries why more than one case is actually necessary but miles and huberman (1994, p.173) suggest cross-case analysis enhances generalisability as it will deepen the understanding and ability to explain what has occurred. eisenhardt (1989) proposes that at least four separate cases are required for multiplecase research. miles and huberman (1994, p.30) warn that a study with more than 15 cases can, however, become unwieldy. since 1994 a total of 17 major intervention programs have been conducted in australia to encourage more women into computing. from these, the cases to include in this research were selected based on that a program’s objectives included attempts to increase the number of females in computing; the program was a sustained activity but could consist of one or more projects; the program’s champion was able to participate in the research and finally that the programs chosen would provide diversity in terms of location and focus. fourteen cases met the criteria: eight cases from universities, three from government bodies and three from the industry sector (see table 1). more cases were from the university sector because of the proliferation of programs in this sector. some programs and projects were completed, others were ongoing. australasian journal of information systems volume 18 number 2 2014 95 educational institutions who spectrum interviews other data sources uni1 school of cs megan; champion 2 published papers 4 newspaper articles uni2 school of is nikki; champion 3 grant applications 12 published papers 3 unpublished reports 6 surveys 2 profiles 1 video tape 1 resource booklet uni3 faculty of ict sarah; champion 1 grant application 7 published papers 2 unpublished reports 1 website 2 interview transcript with 5 facilitators uni4 school of is kim; champion helen; major contributor 2 published reports 3 proceedings 1 newspaper article 1 media release 1 newsletter uni5 school of business and technology ann and alison; joint champion 1 grant application 8 published papers 1 unpublished paper 1 newspaper article 4 video tapes 7 profiles 1 website uni6 faculty of it jodie; champion 2 published papers promotional material 2 brochures 2 websites (for different audiences) uni7 department of is kerrie; champion 1 published paper 1 draft evaluation report – unpublished 1 program outline 10 assorted surveys 3 profiles uni8 school of cs and se tania; evaluator and major contributor 1 published paper 4 unpublished reports 2 technical reports 1 newspaper article 1 award application government departments who spectrum interviews other data sources gov1 education clair; champion 2 grant applications 4 published documents 2 unpublished reports 2 websites various support materials for teachers numerous workshop materials gov2 ict karen; major contributor 3 research reports 1 task force report 1 evaluation report 2 websites 1 media release 1 video tape 22 profiles various support materials for teachers gov3 ict debbie; champion 1 evaluation report 1 article 1 website 1 newspaper article various promotional material and brochures industry organisations who spectrum interviews other data sources ind1 ict cheryl and bev; joint champions joan; evaluator 6 various surveys 2 evaluation reports 3 newspaper articles 1 website ind2 technology lesley; champion fiona; major contributor 1 constitution 2 newspaper articles 8 various surveys 1 project plan 1 final report 1 published paper 1 website ind3 telecommunications & it stacey; champion 3 reports 2 newspaper articles promotional materials 1 media release 2 surveys 1 website table 1: data sources australasian journal of information systems volume 18 number 2 2014 96 the data sought for each case study included details about the intervention program in context, the way the program was expecting to work (the assumptions it was based on) and the level of success of the program from the perspective of the program champion. in-depth interviews with the program champion of each program were conducted as well as detailed document and artefact analysis (see table 1). nineteen interviews were conducted as in some of the cases more than one person was interviewed. the analysis of the documents and artefacts complement and provided some validation for conclusions drawn from the interviews. exploring the evidence used to indicate or measure success provided an understanding of the nature of the evaluation conducted, both formal and informal evaluation, what its purpose was and how this evaluation was ultimately used. the analysis of each individual case was followed by a cross-case comparison. all data was brought together in one nvivo project file enabling sorting, searching and linking. an initial set of categories for coding was created based on the themes identified in the gender literature such as awareness (clarke 1990; lang 2007), recruitment and retention (clarke 1990; clayton & lynch 2002; klawe et al 2009; adya 2000). additional codes came from the evaluation literature: program operation, success of program, purpose and design of evaluation, stakeholder involvement, resources available and dissemination of findings (kellogg foundation 1998; rossi et al 1999; weiss 1998). concepts emerged from the source data which enabled further refinement of the categories (eg program operation; prior research conducted, the way it was expected to work, difficulties with program operation, follow up with target audience, grant requirements, support from others, sustainability/burnout, using stakeholder participation). having all the data in one project file enabled the creation of a meta-matrix as described by miles and huberman (1994, p.178) to facilitate an analysis for patterns in responses and opinions. yin (1994, p.54) agrees that this approach is a good method for analysing multiple-case study data. intervention programs each of the intervention programs conducted by the case study entities was a multifaceted cluster of numerous intervention projects. for example uni1’s intervention program consisted of projects conducted at both the pre-tertiary and tertiary level (see table 2). the time span of uni1’s intervention program was five years with limited activity currently taking place. pre-tertiary projects created secondary teacher awareness through a careers booklet created a website profiling a wide variety of computer careers had female undergraduate students speak at school careers events conducted a shadowing program where girls in years 10 and 11 were able to ‘shadow’ first year computing students provided a series of interviews to newspapers and radio to dispel the nerdy image of computing all publicity (eg websites and brochures) was checked to ensure the photos in any publication incorporated 50% women. frequently photos were of groups of students rather than computers to emphasise the people-side of computing an annual girls in computing day was held for 50-60 girls at a time tertiary level projects a compulsory orientation camp geared towards networking was conducted a scholarship program was offered female students were provided with professional development through industry partners table 2: the projects which made up uni1’s intervention program australasian journal of information systems volume 18 number 2 2014 97 each of the 14 case study entities had a unique set of intervention projects which made up their intervention program. while some of the intervention programs are still relatively new, some have been operating for a lengthy period of time. table 3 summarises the collective reach of the more common major activities conducted by the case study entities. table 3 does not capture the more local projects, that are too numerous to list, such as a mentoring breakfast with female ict mentors through a city council for 30 students and seven teachers from six different schools. nor does it capture other projects which are not measurable by the number of participants they directly reach such as the creation of a ‘how to’ kit for girls and icts role models events; or the publicity that was generated by girls and ict articles appearing in numerous school and parent magazines, teacher journals and articles in local and national newspapers. as can be seen from this section, the activities conducted by the 14 case studies were considerable. activity no. of events held no. of participants girls in computing days or role model events 40 14,000 girls 500 educators 200 parents 200 role models multimedia workshops 9 200 girls professional development for teachers 20+ 400 educators awareness-raising events 370 23,044 students computer clubs established 150+ unknown videos produced and distributed 6 unknown significant websites created 12 34,700 hits in a two year period orientation programs for female tertiary it students 3 unknown national women in computing conferences 5 400+ computer camps for secondary girls 4 120+ state based technical workshops 2 unknown scholarship/awards/bursary schemes 8 unknown practical ideas booklet for encouraging girls with ict 1 all schools in one state table 3: direct reach of common major activities conducted by the case study entities. findings and discussion success of the programs this research found that from the perspective of the program champions there was little doubt that their programs were successful. fantastic, incredibly successful. they had way more people there than they thought they would get … it is actually the whole movement that is successful. (debbie_gov3) australasian journal of information systems volume 18 number 2 2014 98 the end result was that it was a mind-blowing experience. it was a fantastic effort by all, we just did the unimaginable and pulled it off and we had 1500 girls by the end and we were able to pay our expenses and still manage to do very well. (cheryl_ind1) some of the program champions however did acknowledge that though the programs were successful, the programs might have had less impact then they would have liked. i think all the events were successful, it depends how you are measuring the success. (clair_gov1) were the conferences successful? very much so. i probably would have liked more people at the second one. no, i think they were great. (kim_uni4) while the program champions were convinced of the success of their programs it was often not formal evaluation that lead them to their conclusion but anecdotal evidence. almost all of the respondents talked about individual instances of having made a difference in one persons’ life and that this was enough to make the program a success; yes. i know that i personally have touched lots of women’s lives from that point of view. that satisfies me but does it really justify the amount of time and effort that i have put into it? i don’t care. really you cannot always measure things by the direct and tangible benefits so to me if we supported an activity and it made the difference between no girls going into a course and one girl going in i would be quite satisfied with it, i don’t think it is an issue. (stacey_ind3) there were also many perceived outcomes of the programs that the program champions did not know how to evaluate or quantify: • increased ict skills amongst participating girls; • increased confidence by girls in using icts; • increased willingness to take a leadership role in the use of icts; • better networks between entities and therefore better distribution of key information; • increased understanding of possibilities and options in ict study paths and careers; • students who felt they were 'cared about', a sense of 'belonging' which assisted with their retention in courses; • workplace strategies which have been shared and professional development activities carried out; • improved curriculum in numerous universities and/or an improved learning environment developed; and • numerous supportive communities which have been created enabling networking and mentoring programs to be conducted. the intervention programs at the focus of this study were mostly conducted by female volunteers who were passionate about the need for more females in computing and were prepared to devote large amounts of their own time to make a difference. this meant that frequently the tasks to implement the program were on top of their ‘day’ job and family responsibilities (craig 2009). for many of the projects the champions were practitioners and being time-poor, they were more concerned about the implementation of the program, which they ‘knew’ would work anyway, then evaluating how it did work. almost all of the intervention programs were considered successful yet this was often based on anecdotal evidence since only limited formal evaluation may have been conducted. australasian journal of information systems volume 18 number 2 2014 99 the next sections will explore how the programs were evaluated and whether the key elements as identified in the literature were taken into consideration by those conducting the evaluations. purpose of conducted evaluations using rossi et al.’s (1999, p. 54) five main types of evaluation questions, if evaluation was conducted on a particular project by the case study entities, it was mostly to assess the implementation or operation of the project (process evaluation) or an assessment of its outcomes or impact (impact evaluation). a few of the evaluations also assessed longer term outcomes which, although not articulated as such, is moving towards assessing the assumptions made in the project design (program theory). a needs assessment was conducted before the implementation of a number of projects: the entire research program is aimed at gaining a deeper, conceptual understanding of this phenomenon as the basis for informed intervention at a range of levels. (sarah_uni3) none of the evaluations looked at a cost-benefit analysis of their programs, the fifth purpose of evaluation which rossi et al (1999, p. 54) describe as efficiency assessment. the audience for whom the evaluation was conducted was either the project champions or program committee so that they could modify and improve the project; the sponsors of the project so that they would see that the project had the desired outcome; or for future sponsors to show the impact of the interventions and to encourage their contributions. the purpose of this report is to provide feedback to the organising committee of the [event] … the report summarises students' and teachers' views about it careers and analyses the impact of this event on the students' it career intentions. (uni8_documents) a large number of individual projects were not evaluated at all; kim (uni4) indicated that their project of running women in computing conferences was a great success but when asked how they were evaluated the response was ‘we didn’t because we didn’t think about it’. when queried further, it was the enthusiasm of the participants and anecdotal comments which had led to her conclusion that the intervention was a success. similarly websites were not tracked to see for example how many unique visitors came to the site (uni5_documents) and artefacts such as videos were not formally evaluated either (nikki_uni2, anne_uni5). nikki explained: ‘i think we just put it out there. i do not even know if we followed up to find out how many times it was used. it was literally a product we put out’. fiona (ind2) described the development of a comprehensive information kit for teachers which was made freely available, was not formally evaluated, but was considered a success. a role model day was also not evaluated but it too was considered successful because; … it was aimed at year eight and nine girls from memory and yet we had one of the women that was involved in the committee she brought along a group of her grade five students to help out on the day. at the end of the day they were so excited, these grade five girls, they went back and started a girl’s computer club. (fiona_ind2) sarah (uni3) and kerrie (uni7) explained that as part of their funding applications there was no need to mention how the project would be evaluated. clair (gov1) and megan (uni1) agreed; i think we looked at numbers at events and those sorts of things. i think that was all we indicated. we had to say what our outcomes were; we didn’t have to say how we were going to evaluate it if that makes sense. (clair_gov1) the university special projects funding was part of it in the days when there was actually a budget with money in it. it was maybe a three or four page proposal outlining what you wanted to do and what it was going to cost and what you hoped the outcome of it would be. (megan_uni1) australasian journal of information systems volume 18 number 2 2014 100 sarah (uni3) whose program is still in its early stages, has not yet considered when and how the evaluation will be conducted: ‘the evaluation of that, i do not know, we have not discussed how that is going to happen’. nikki (uni2) explained that it was the project itself which was the priority and evaluation was not the main concern; … we had a very much practitioner view. there was not a lot of evaluation… we were not required, as part of our funding, to deliver anything other than the product. as teachers i think we always evaluated at a level of have your sheet checked with people afterwards and i know emily did a lot of follow-up work on statistics on numbers as evidence to see whether we had made any difference. with attitudinal change, it is very hard to have a good evaluation instrument anyway. so evaluations would have been against whatever the funding criteria was. if, as a practitioner, you wanted a product like a video, that was really probably our early focus and it was only when it became seen as appropriate to take what we were doing to research conferences rather than practitioner gatherings, that we started to look more seriously at the validity of what we had done and whether we had made any difference. (nikki_uni2) bev (ind1) also felt that evaluation was not the priority. the program itself was the primary focus though having better evaluation could make it easier to encourage potential sponsors to support the initiative; [the role model day] produces the survey that shows [students’] perceptions have shifted from this opinion to that opinion on the day. very subjective stuff. anybody with a statistical background is going to go well that’s great. what were they thinking the next day? three months, six months? where is the follow-up work? of course we are not doing any, we are not researchers. we are working stiffs who are volunteers in our “spare” time … (bev_ind1) each of the program champions had some understanding about how and why they expected their program to work but this process or theory, was not tested through evaluation. while there are good resources available in the literature and online (see for example: http://www.olt.gov.au/evaluation), to guide the development of evaluation for social intervention programs, these were seldom used by the program champions. reasons provided by the champions included a lack of time; lack of priority compared to running the program; and the general nature of the sources that were explored which did not focus specifically on intervention programs to raise awareness, recruitment and retention. while the program champions would not have undertaken implementation of the intervention programs unless they thought they could make a difference, the reality is that in they have little evaluationbased evidence of the success or worth of their programs. the evaluator and inclusion of stakeholders when evaluation of a project was designed and conducted, in almost all cases it became the responsibility of a team member, or members, from within the program, and rarely involved other stakeholders. these evaluations were conducted by the project team members. (uni8_documents) an exception to this was the evaluation of uni7’s summer school intervention and evaluations performed by government entities which were frequently conducted by external consultants to the projects such as the following; …we have commissioned a consultant to do the evaluation of that and that is twelve months in now and it runs through to the end of this year… their report will probably come in to us early next year. (karen_gov2) australasian journal of information systems volume 18 number 2 2014 101 the analysis of major role-model interventions conducted by ind1 and uni8 were undertaken by external evaluators however they did not contribute to the design of the actual evaluation itself; my involvement started with the [completed] forms that we got. we didn’t have any input into the design of the forms which was a bit of problem, the questionnaire. when we got the questionnaires we looked at them and thought well we could have done quite a lot better knowing the analysis that we could do on it. we could have done a lot better if we had some input into the design of the questionnaires. (joan_ind1) there was minimal indication of any other stakeholders being involved in the evaluation process therefore any evaluation that was conducted did not consider the perspectives or the needs of these stakeholders. resources provided for evaluation to conduct evaluations requires access to sufficient resources in terms of time, funding and skills. these resources were hardest to come by for the university sector. anne (uni5) indicated that ‘we didn’t ever conduct any official investigation or surveys on how much impact it has had simply because we didn’t have the time or funding to do it.’ as far as the tertiary interventions were concerned at uni5, a lack of funding also contributed to the type of evaluation conducted: ‘only in 1996 and 1997 did we have sufficient funds to employ a part-time research assistant to conduct structured interviews and focus groups’ (uni5_documents). the government entities on the other hand, had greater access to funding for evaluation; i think we are spending 40 to 50 [thousand dollars] on the evaluation, that is ok … typically 30 to 40 will give you a good evaluation. i know the attitudinal research was whole lifecycle too so that was a lot more, that was probably close to 200 i reckon. i think that went out to selective tender. that was depressing, but good stuff. (karen_gov2) karen also spoke about a project which was regarded as being resource-intensive and hence was not sustainable; ‘[this project] was a major resource-heavy showcase opportunity for schools around the state to see the breadth and excitement around it careers pathways. that ran for about two or three years’. it was decided to discontinue the project. it was only after the decision to wind-up the project was made, that an evaluation of the project was commissioned; my understanding of it is that it had been closed off; we are not doing this again. but i guess it is part of government process that you evaluate everything. you plan it, you run it, you evaluate it. regardless of its future. that just finishes the research cycle. (karen_gov2) yet not all interventions conducted by the government entities had access to the same level of funding; no further funding was available to commit resources to investigating what impact the funded projects had on participants in their career choices and whether some projects worked better than others. (gov1_documents) a lack of knowledge of how to conduct evaluations and having the required skills were also problematic: the other thing is we are project officers writing these questions. we don’t have a background in writing valid survey questions either and we don’t pay anybody to do that but that is something i would be interested in; looking at how valid our questions are and whether we should look at rewording them. (clair_gov1) participants do not believe that males write better programs than females. these results could be an indication that perceptions are changing in this cohort. however, such an interpretation australasian journal of information systems volume 18 number 2 2014 102 may be premature. the question may have measured the general issue of differences in capabilities between females and males rather than this specific skill. further research is required to resolve the issue and a redrafting of this question will be attempted for administration at the next summer school. (uni7_documents) clair (gov1) also spoke of a project involving establishing a ‘brand name’ for a series of girls and computing initiatives. when queried on whether she had tried to evaluate how well-known the brand was becoming, her response was ‘i have no idea how to do that but i would love to’. fiona (ind2) raised several issues regarding a lack of confidence with evaluation skills and the difficulty of getting useful evaluation; again it is a volunteer thing, who has time to do something with it and who has the skills to maybe do something with it? susan and i have the research interests and maybe nobody else really has that research interest on the committee. it has pretty much come back to us each time. [what we do] is more research analysis of the results. susan then puts it through spss …so how many girls thought this was influential? who was their most influential role model and things like that? then we got some of the qualitative stuff as well. we haven’t really had anything concrete to go back to [sponsors] with and i know that is part of the problem of getting money to run projects, is not having that written on paper saying this is what is really going on…how do you measure that anyway to say this is what is really happening? and give us funds that equal that. it doesn’t work that way … we just don’t have a way of doing that so it is really hard. (fiona_ind2) the evaluation that was conducted by the case studies was predominately dependent upon the time available, skills and expertise of the program champion or other key team members. scarce resources were used to implement the program with seldom sufficient funds and time to consider what evaluation was necessary and appropriate to be conducted. a lack of evaluation expertise among the team members contributed to a lack of understanding of its pivotal role in informing the team of the true worth of the intervention programs. design of the evaluation the methods used to evaluate the intervention projects formally did not vary greatly. most frequently questionnaires, interviews and focus groups were used. some projects also incorporated observation. the quantitative analysis will provide a statistical evaluation of the intervention program and identify other factors in the students' backgrounds which are associated with success in the first programming subject …a variety of data collection methods were used including diaries, structured interviews, and focus groups involving the first year women computing students and end of semester written reports from the senior student. … the qualitative analysis will provide detailed accounts of the female students' experiences in the intervention program and in the first year of the computing course. it will also be used to help interpret the results of the quantitative analysis. (uni5_documents) for evaluation of changes in attitude, confidence or skills it was common to adopt a preand post-event survey. however as some events became larger this proved to be problematic and the pre-event survey was abandoned: i think a couple of years ago we did a pre-survey and a post-survey. that was our approach in the beginning. i think by the next year it was just too much work because there were too many girls and we ended up just having the one survey. (fiona_ind2) australasian journal of information systems volume 18 number 2 2014 103 in the case of gov3 the post-event survey then incorporated two questions which still tried to gauge the students’ self-perception of any change in attitude; q16. would you consider it as a career option? yes □ no □ q17. had you considered it as a career option before today? yes □ no □ (gov3_documents) a number of comments surfaced around post-event surveys of role model or girls in computer days really being a ‘happy sheet’ and indicating whether the girls had had an enjoyable day rather than whether their awareness of, or interest in, computing had changed at all. lesley (ind2) suggested that ‘happy smiley is not a fair evaluation’ but evaluating whether the girls had had a good day was easily measurable. ‘is it because they are suspecting that they are not going to get good results and they don’t want to hear it is this why people use happy sheets?’ asked lesley. tania (uni8) agreed and suggested that asking the right questions can, however, be difficult; the way a lot of people think about evaluation is that you just give them a survey and we will be fine. just ask some questions and then we will get some information back and that will be fine. that is the level that a lot of people think. there is no understanding of how difficult it can be to design something that is appropriate, or to think ahead. what information do we want? they just know they want some feedback as a general rule. (tania_uni8) some surveys therefore did not provide quality feedback; when you get a box of survey forms like that it is quite a lot. that is why it was so disappointing the second year when the form itself wasn’t well-designed and you could see that the students really weren’t able to give a proper evaluation of what they had been to. the second day they couldn’t say anything was boring because they didn’t have that column. (joan_ind1) the surveys told us that the girls had a wonderful time and they thoroughly enjoyed it and they thought technology was great. but when we did the longitudinal study. it would have been a couple of years later. it really didn’t show anything. there was no clear cut outcome. when it was written up and when the results were looked at it really showed nothing. i think you could say with confidence that it demonstrated nothing. (kerrie_uni7) gov2 was the only entity who had the resources to use external consultants to undertake each of their evaluations and karen explained how this would happen; the way that we will go out to market is we will write a brief and scope it out ourselves saying these are the things that we want to find out about. we will put it out to the market with three or four selected companies ask them to come back to us on how they will approach it. some methodology around how they will approach it and some recommendations, which is vital for us. we will just select on the best proposal. (karen_gov2) to be effective evaluation must have a clear purpose. specific data can then be carefully collected and analysed to answer the evaluation questions. the process or theory of how an intervention program should work requires testing through evaluation yet this was rarely undertaken. learning and sharing it was strongly suggested in the literature (rossi et al 1999, p.24; weiss 1998, p. 16) that not only do program evaluations need to take place, but results of these evaluations need to be used, and then disseminated so that each evaluation can add to the build-up of knowledge. australasian journal of information systems volume 18 number 2 2014 104 the purpose of many of the evaluations undertaken by the case study entities revolved around improving the process and implementation of the project; the group that organised the day used [evaluation]. primarily it was to tweak the program for the next time around. (nikki_uni2) it was predominantly [used by] me, for planning for the future ones. (megan_uni1) we learnt lessons from 2002 where schools don’t read emails because they just don’t. you need to fax to about five or six different parts of the school to make sure that somebody reads it. we asked them “which part of the year would you like to have it? is november suitable?” it came back that march was the appropriate time. so we collated a lot of lessons learned and we tried to implement them for 2004. (cheryl_ind1) using an overall-rating of the project on a likert scale of 1 to 5, all students rated the project 3 (good) or above, with 33% of the students nominating 5 (excellent). it was pleasing to find that in the following year 55% of the students rated the project as excellent showing a positive reaction to the improvements made in the project delivery from the feedback of previous years. (uni8_documents) tania (uni8) expressed frustration that the funding for the summer school intervention was cut off and that this decision was not based on the evaluation of the project. there were other instances where it was unclear if the evaluation conducted was actually used; i haven’t seen it to be honest; i have never seen anything from it, no. because i was still on the ground running everything, she was collecting the feedback and helping me out a lot …and i have never actually seen it to be honest. (fiona_ind2) jo has got all the analysis of the data and i just get to flick through it and have a look. i wanted to do that a lot earlier but i just didn’t have the access. (jodie_uni6) in some situations the evaluation was used to affirm the project leader’s decision to conduct the intervention; ‘we just looked at it and thought oh didn’t we do a good job’ and ‘it told us that it was worthwhile doing’ or to apply for an equity award; ‘we were just using it for an award that we went for’. for government entities evaluation reports were either internal documents or posted on an external website for interested parties to obtain. we either post it on the website as a complete report …or we just picked the eyes out of the report which was 200 pages and we just made some highlighted comments and we have sent that out as a media release and we have sent it out to stakeholders and also posted that on the website. (karen_gov2) yes [we wrote reports]. usually they are just internal though. (clair_gov1) it is not clear who should write up the results of the evaluation and where it should be distributed. the funding agency and the program committee generally receive copies but members of the project do not necessarily see the results, nor is it easy to access results of projects that other people have conducted; we wrote a report for the grant, the final status report for that and i know that marg and sue from the university actually presented some papers on those events at conferences. my role isn’t to write academic papers but i did do the final status report. (clair_gov1) i think in the end sally did … quite a lot of it and she wrote up a big report and things because it was government [funded] and i don’t believe i really saw that report but i am sure she would have a copy. again, this doesn’t come back around. being in government and having funds i australasian journal of information systems volume 18 number 2 2014 105 think they had to do a formal evaluation of it but i am not quite sure what that was but i know she had a really hard time with it. (fiona_ind2) some grant applications indicated that dissemination would occur but not specifically who to and how other interested parties could access the information; thirdly, the dissemination of research outcomes to interested parties and increased enrolments in it courses will enhance uni3's reputation as being a female-friendly university and progressive in its approach to course design and delivery, which assist positive branding for the university. (uni3_documents) as previously mentioned by nikki (uni2) and others, numerous projects started as ‘practitioner’ based with the focus on conducting the project and without concern to any evaluation that might be of interest to others. in these situations the university sector does not allow dissemination of results as the university’s ethical committee will not have sanctioned the project before its commencement: [we have not published the results]… because we didn’t have ethical clearance for it... this is the beauty of having jo around because she does this kind of thing and i just don’t have time and also her background is in ethics. in fact in future any of the projects we plan we will actually get ethic clearance first. (jodie_uni6) there was little evidence that the dissemination of the outcomes from the conducting of interventions was a priority for the team members of the case studies. yet without this, a build-up of knowledge regarding what makes a program effective, or ineffective, will not occur within the women in computing community. therefore unsuccessful programs may continue to be conducted taking scarce resources away from those programs that evaluation shows do have the desired effect. difficulties with evaluation an inability to conduct anything beyond short-term evaluation was a common concern. this difficulty arose through being unable to track students due to privacy laws, lack of resources, or project restrictions. the timeframes weren’t such that you could necessarily evaluate very much anyway. (nikki_uni2) all of the case-study entities undertook some intervention projects aimed at pre-tertiary girls. consequently the participants were under the age of 18 and by law, not yet adults. as a result there are legal requirements that must be abided by such as parental permission is required before these girls can be interviewed or take part in any research endeavours. this severely restricted anything other than anonymous short-term feedback; we have not been able to do any follow-up with any of the girls. i would like to but no, we haven’t. in terms of looking at young school girls and ethics it is a bit tricky and nobody has had the time to be honest. i think it does need to be done, as well as looking at where these girls go, and what they are doing, and things like that, but again it is getting the resource and getting the ethics to do a privacy thing and things like that. i think no, we just haven’t. (fiona_ind2) we can’t really do that for privacy reasons. (clair_gov1) we could not follow up more due to the privacy act; we could not get their addresses; we would have kept in contact but it was just not possible. (megan_uni1) you can’t use the information that you have got from one survey or one contact with the students for something else without making it clear to them. they are very strict on that. (tania_uni8) australasian journal of information systems volume 18 number 2 2014 106 most of the large events did not get a list of the names of the students that were attending the event, and only registered the name of the school with the number of students and staff attending. therefore even the checking, in future years, to see whether these same girls enrol at their university, was often not possible. surveys conducted on the day were all completed on a voluntary basis and without any identifying names or addresses. consequently any follow-up would have to go via the schools to the parents. this could only be done after clearance from ethics committees at the university level and from the education department have been obtained (a process which can take many months) and then also requires the consent of the principal of the school as well as the cooperation of individual teachers to encourage students to return the surveys. this would also require that the school actually had retained a list of the attending girls. constraints placed by sponsors and funding agencies also dictated that if evaluation was to be undertaken then it often needed to be reported back within a short period of time. the largest and most ambitious of the intervention programs was expected to show results within a three-year period while smaller projects’ timeline was generally from a few months to one year in length. when you get funding grants you have to indicate how you are going to evaluate but the evaluation measures are not medium-term or long-term, they are all just kind of on the spot. you do have to say what your performance indicators will be and you have to quantify them, but they tend to be things like, we did a survey on the day and 80% of the girls that came in that said they were not interested in it said that when they left that they are. so that is a bit of a measure about the culture and the impact on the day … but yeah, it does ask for evaluation but there is no follow up or medium or long term evaluation associated with it because you have to report back in a certain time. (lesley_ind2) i am not going to get numbers in three years. that is where we have got to convince them that qualitative is important.… (sarah_uni3) a common difficulty with evaluation was what criteria to use and how to measure the criteria accurately, so that it did provide useful information; how can you measure how successful you are? at this point we can’t. the only proof is the fact that we have bums on seats. absolutely. if it was a negative experience for the girls there would have been no registrations, the schools would have found out by now. (cheryl_ind1) it is very hard to measure when we are talking about attitudes towards technology and the way it is being used in the classroom obviously if it is being used across the curriculum more that is good, but how to measure that is near impossible as far as i know. (clair_gov1) .. you need that quantifiable information on; did it make a difference and all this kind of stuff. susan was really struggling with it because how do you quantify this? (fiona_ind2) we tried to measure it quantitatively and we couldn’t. you will see that in the journal paper. we couldn’t relate anything really to the program but our qualitative work tells us that women students perceived that the program was just a great benefit and that is enough for me. (alison_uni5) not consulting with all stakeholders was problematic with ind1’s evaluation resulting in much less-rich data then might have been obtained otherwise; the design of the survey questionnaire with the 3 and 4 point-rating scales made analysis of responses difficult. providing a 5 point-rating scale for the participants to rate the sessions would allow more meaningful analysis of the data. furthermore, the career-interest questions on the questionnaires were not relevant to the teachers. for future events, we recommend that a separate questionnaire be designed for them. (ind1_documents) australasian journal of information systems volume 18 number 2 2014 107 we said that we would be happy to give our time to help with the design of the questionnaire… we asked if we could have some input into this, … but things get underway and things get forgotten and things happen before you realise they are done and we thought, well we did give feedback last time and so they will pay attention to that … the next year they made it worse. the format wasn’t very good. nor were the questions wonderful. (joan_ind1) even when evaluation had been undertaken it was not always possible to enact the recommendations (such as moving a role-model day to earlier in the year when the sponsor provided facilities were not available any earlier) and there was not always confidence in the results; so the first evaluation was exceptionally positive, much more positive then the second one but i am sure the [consultants] got the facts wrong because the second one was in my opinion as positive when we were reading the evaluation forms. the statistics came back a little bit lower than the first one which was something like ‘88% would do it that weren’t thinking about doing it before they came to the showcase’ and the second one came a little bit lower at 69% or something. all in all it was very positive. (cheryl_ind1) to conduct effective evaluation on intervention programs designed to encourage girls into computing was not a trivial task. those program champions who did recognise that it would be useful to be able to conduct formal evaluation found many difficulties which hampered their ability to provide rigorous evidence-based evaluation. conclusion to determine whether an intervention program has made a difference requires evaluation. the purpose or aims of conducting an evaluation will guide the design of the process (rossi et al 1999) while evaluation skills are necessary to create the evaluation instruments. the evaluation should represent the needs of all the stakeholders with sufficient resources provided to ensure that the appropriate data is collected and analysed (weiss 1998 p.79). the learnings from an evaluation should be published or shared so that others in the community can learn from what is, or is not, successful (weiss 1998 p. 17). fourteen major intervention programs, conducted over the last 20 years for women in computing in australia, were investigated. each of the program champions had some understanding about how and why they expected their program to work yet this theory was rarely tested through evaluation. program champions conducting the intervention programs were passionate about encouraging and supporting girls to move into higher computing education and/or careers. this was expected to come about from the intervention programs that they conducted and ‘knew’ would be successful. therefore little priority was given to formal evaluation. when evaluation was considered a lack of time, funding and expertise restricted its conduct. the evaluation’s design and application rarely included multiple stakeholders. problems with privacy laws, project restrictions and resources also hampered the ability to evaluate the longer term impacts of the programs. frequently the analysis of the evaluation data was used only by the project team rather than disseminated more widely to enable the build-up of knowledge in the women in computing community. champions of intervention programs such as those highlighted in this study do not currently have access to sufficient resources (time, money and skills) to enable them to undertake appropriate evaluation of the girls in computing programs that they are conducting. nor do they always recognise its importance. yet detailed evaluations would provide the evidence to point to programs which should be replicated and those which should be modified or abandoned. without this knowledge there is a real danger that the efforts of creating intervention programs will not translate into more girls undertaking computing in the future and that program champions may burn-out and abandon their efforts. australasian journal of information systems volume 18 number 2 2014 108 future research needs to focus on the development of an evaluation framework which can be implemented by practitioners who are not expert in evaluation and who have limited resources at their disposal. such a framework should be based on a theory of change and guide the program champions in what needs to be evaluated, who should be involved and when. such a framework may not be able to address all the difficulties faced by program champions such as issues related to privacy laws and project restrictions. however it will enable those without skills in evaluation to articulate the way their programs are expected to work and then better evaluate the components within their control. better evaluation should enable the creation of a knowledge base to show which intervention projects work for whom and under what conditions, and which have only limited effects. this information is urgently needed by the computing profession. references australian computer society (2012) australian ict statistical compendium, retrieved from http://www.acs.org.au/__data/assets/pdf_file/0014/13541/2012_statcompendium_final_web.p df on 25 april, 2013 australasian evaluation society (2006) definition of evaluation, retrieved 5 february 2006, from www.phcris.org.au/resources/research/evaluation_ mainframe.html adam, a. howcroft, d. & richardson, h. (2004) a decade of neglect: reflecting on gender and is, new technology, vol. 19, no. 3, pp. 222 40. adya, m. & kaiser, k. (2005) early determinants of women in the it workforce: a model of girls’ career choices, information technology & people, vol. 18 no. 3, pp. 230-59. carty, s. (2011) many cars tone deaf to women's voices, retrieved 10 may 2011 from http://autos.aol.com/article/women-voice-command-systems/ clarke, v. (1990) sex differences in computing participation: concerns, extent, reasons and strategies, australian journal of education, vol. 34, no. 1, pp. 52-66. clayton, d. & lynch, t. (2002) ten years of strategies to increase participation of women in computing programs: the central queensland university experience, 1999-2001, sigcse bulletin, vol. 34, no. 2, pp. 89-93. clayton, k, beekhuyzen, j. & nielsen, s. (2012) now i know what ict can do for me!, information systems journal, vol. 22, no. 5, pp. 375-90. craig, a. (2009) intervention programmes to recruit female computing students: why do the programme champions do it?', paper presented to the eleventh australasian conference on computing education, wellington, new zealand, 20 to 23 january 2009. craig, a, fisher, j, scollary, a & singh, m (1998) closing the gap: women education and information technology courses in australia, journal of systems software, vol. 40, no. 1, pp. 7 15. darke, p, shanks, g & broadbent, m (1998) successfully completing case study research: combining rigour, relevance and pragmatism, information systems journal, vol. 8, no. 4, pp. 273-89. denzin, n. & lincoln,y. (2000) the discipline and practice of qualitative research. in handbook of qualitative research. (eds: denzin, n. & lincoln, y.) thousand oaks, california, sage, pp. 1 29. edwards, j. & kay, j. (2001) a sorry tale a study of women's participation in it higher education in australia, journal of research and practice in information technology. eisenhardt, k. (1989) building theories from case study research, academy of management review, vol. 14, no. 4, pp. 532-50. australasian journal of information systems volume 18 number 2 2014 109 galpin, v. (2002) women in computing around the world. inroads sigcse bulletin, vol. 34(2), pp. 94 100. kellogg foundation (the w.k.) (1998), evaluation handbook, the w. k. kellogg foundation. klawe, m. whitney, t. & simard, c. (2009) women in computing take 2, communications of the acm, vol. 52, no. 2, pp. 68 – 76 lam, s. uduwage, a. dong, z. sen, s. musicant, d. terveen, l. & riedl, j. (2011), wp:clubhouse?: an exploration of wikipedia's gender imbalance, proceedings of the 7th international symposium on wikis and open collaboration, mountain view, california. lang, c. (2007) twenty-first century australian women in it: exercising the power of choice, journal of computer science education, vol. 17, no. 3, pp. 215 26. lewis, s. mckay, j. & lang, c. (2006) the next wave of gender projects in it curriculum teaching at universities, proceedings of the 8th australian conference on computing education, hobart, australia martin, u. liff, s. dutton, w. & light, a. (2004) discussion paper no. 3 rocket science or social science? involving women in the creation of computing's intellectual property, oxford internet institute, university of oxford. mcnamara, c. (2007) basic guide to program evaluation. retrieved 16 december, 2007, from http://www.managementhelp.org/evaluatn/fnl_eval.htm. meyers, w. (1981) the evaluation enterprise. san francisco, california jossey-bass inc. miles, m.& huberman, a.(1994) qualitative data analysis: an expanded sourcebook, 2nd edn, sage publications inc., usa. miliszewska, i. & moore, a. (2010) encouraging girls to consider a career in ict: a review of strategies, journal of information technology education, vol. 9, pp. 143-66. myers, m. d. (2002) qualitative research workshop. faculty of computing. presented at monash university 2002. nielsen, s. & beekhuyzen, j. (2009) narratives of identity formation: tech girls are chic (not just geek), australasian women in information technology (ozwit), melbourne, australia, monash university, 1 december panko, r. (200) it employment prospects: beyond the dotcom bubble, european journal of information systems, vol. 17, no. 3, pp. 182-97. parker, l. (2004) gender and technology in the information society: networking to influence information and communications technology in education. in gist-gender perspectives opening diversity for information society technology. bremen, germany. prey, j & weaver, a. (2013) fostering gender diversity in computing, computer, vol. 46, no. 3, pp. 22-3. ross, j. litzler, e. cohoon, j. & sanders, l. (2012) improving gender composition in computing, communications of the acm, vol. 55, no. 4, pp. 29 31. rossi, p., freeman, h. & lipsey, m. (1999) the social context of evaluation, in evaluation: a systematic approach, 6th edn, sage publications, thousand oaks, ca, pp. 397 416. teague, g. j. (1999) perceptions and misperceptions of computing careers. phd thesis, geelong, deakin university, school of information systems. trauth, e. (2011) what can we learn from gender research? seven lessons for business research methods, the electronic journal of business research methods, vol. 9, no. 1, pp. 1 – 9. von hellens, l. beekhuyzen, j. & neilsen, s. (2005) thought and action: the winit perspective strategies for increasing female participation in it, paper presented to women, work and it australasian journal of information systems volume 18 number 2 2014 110 forum: contemporary perspectives on the reproduction of gender inequalities in employment, brisbane, australia, 23 to 24 june 2005. wasburn, m. & miller, s. (2006) still a chilly climate for women students in technology: a case study. in women, gender, and technology. (eds: fox, m. johnson d. & rosser, s.), board of trustees of the university of illinois, pp. 60 79. weiss, c. h. (1998). evaluation: methods for studying programs and policies. englewood cliffs, nj: prentice-hall. whitney, t. gammal, d. gee, b. mahoney, j. & simard, c. (2013), priming the pipeline: addressing gender-based barriers in computing, computer, vol. 46, no. 3, pp. 30-6. williamson, k., burstein, f. and mckemmish, s. (2002). the two major traditions of research. in research methods for students, academics and professionals. (ed: williamson, k.) wagga wagga, centre for information studies, charles stuart university, pp. 25 47. yin, r. (1994) case study research: design and methods, 2nd edn, sage publications, newbury park. microsoft word 1911-5567-1-le.docx australasian journal of information systems de silva, bedingfield & alahakoon 2018, vol 22, research on recent advances in social media preface 1 special section on recent advances in social media: preface guest editors: daswin de silva la trobe university australia. d.desilva@latrobe.edu.au susan bedingfield monash university australia. damminda alahakoon la trobe university australia. editorial from humble newsgroup beginnings to the recent hubris of distorted democracy, social media has endured a multifarious and multifaceted journey into every aspect of contemporary life. despite recent misgivings, the overarching outcomes and general sentiment towards social media continues to be pragmatic and positive. platforms and applications with direct decision value for business (culnan et al. 2010), government (bertot et al. 2010), social empowerment (carlson 2013; rodriguez 2016), healthcare (bandaragoda et al. 2018), and education (gikas and grant 2013), among many others, represent a rigorous evidence base of its significance. in this context, it is pertinent to postulate that social media has successfully embedded itself as an increasingly indispensable medium of identity propagation, social networking, information dissemination and communication. it can be further argued that the scheme proposed by (kaplan and haenlein 2010) for classification of social media platforms (social presence, media richness, self-presentation and self-disclosure) has rapidly evolved into a de facto guideline for actual human behaviour and social interactions. this development represents an expanding window of opportunity to harvest precise insights on individual social media users and their group behaviours. precise information can be utilised for good or exploited for bad. this conundrum of equivocality motivates our special section. diverging from the rush to harness decision value from behaviours and interactions, we posit the significance of research focused on the individual in social media. we received 16 submissions, out of which 6 articles were rigorously peer-reviewed for publication. these six articles represent a broad spectrum of recent advances permeating importance of the individual in social media; subjectivity of event perception, factors of social capital, influence of social broadcasting networks, impact on intimate relationships, knowledge sharing in higher education, and impartiality of social media discourse in regional australia. australasian journal of information systems de silva, bedingfield & alahakoon 2018, vol 22, research on recent advances in social media preface 2 article 1: an extended cognitive situation model for capturing subjective dynamics of events from social media (yujie wang, damminda alahakoon, and daswin de silva) cognitive situation models are representations of the psychology of understanding and comprehension from textual sources, such as social media. the collaborative nature of social media can lead to subjective interpretation of such content and discussions. this article proposes an extended cognitive situation model and a computational implementation that can continuously construct and update event-centric knowledge from social media data streams, accounting for subjectivity. authors use a twitter dataset to demonstrate and evaluate the proposed model. article 2: which factors determine our online social capital? an analysis based on structural equation modelling (janine hacker, michael grottke, and carolin durst) this article addresses two research gaps in the association between online social networking and social capital; the absence of a valid and appropriate measure of online social capital and key characteristics of an individual’s social network that influence online social capital. structural equation modelling is utilised to analyse and evaluate data collected from 1000 facebook users. authors report varying degrees of influence across variables for online and offline social capital. article 3: influence of social broadcasting networks on news consumption (shailesh palekar and darshana sedera) social media platforms have recently been classified and popularised as social broadcasting networks (sbn) of news and other information. this article explores the role and impact of sbn structure and influence of its users on two news consumption behaviours, switching and continuance. a qualitative study of 325 sbn users was conducted as empirical validation of individuality on these two behaviours. authors indicate sbn serves a different purpose to traditional news media, with intermittent switching and continuance. article 4: helpful or harmful? exploring the impact of social media usage on intimate relationships (naomi whiteside, torgeir aleti, jason pallant, and john zeleznikow) a number of recent studies have reported on the negative impact of social media usage on personal relationships. this article presents a thorough investigation of this association, using facebook as a case study. a survey was conducted on 518 australian facebook users, exploring usage patterns, relationship satisfaction and household income. authors report four prominent usage profiles with corresponding degree of usage and relationship satisfaction. article 5: using social media to enable staff knowledge sharing in higher education institutions (aidan duane and niall corcoran) organisational implementations of social media platforms continue to receive limited attention in current literature. a limitation addressed in this article, as it examines the role and impact of enterprise social networks in employee knowledge sharing, within a tertiary education setting. authors adopt an action research approach to develop a conceptual model that is empirically evaluated via focus groups and interviews. organisational structure and culture are reported as primary barriers to participation and knowledge sharing. australasian journal of information systems de silva, bedingfield & alahakoon 2018, vol 22, research on recent advances in social media preface 3 article 6: power to the people: social media discourse on regional energy issues in australia (kerri morgan, susan bedingfield, and marc cheong) although touted as platforms of equality, social media can conceive bias towards overpowering personalities and group behaviours. this article presents a novel approach for investigating this bias, using text clustering and emergent communication patterns based on graph theory. authors empirically evaluate the proposed approach using a collection of tweets in response to the closure of a coal-fired power station in regional australia. acknowledgement – associate editors are sincerely thankful to all authors for their original research contribution. we kindly acknowledge and appreciate the time and effort expended by all reviewers. finally, we thank associate professor john lamp for his guidance and support from inception to completion of this special section. references bandaragoda, t., ranasinghe, w., adikari, a., de silva, d., lawrentschuk, n., alahakoon, d., persad, r., and bolton, d. 2018. "the patient-reported information multidimensional exploration (prime) framework for investigating emotions and other factors of prostate cancer patients with low intermediate risk based on online cancer support group discussions," annals of surgical oncology), pp. 1-9. bertot, j. c., jaeger, p. t., and grimes, j. m. 2010. "using icts to create a culture of transparency: e-government and social media as openness and anti-corruption tools for societies," government information quarterly (27:3), pp. 264-271. carlson, b. (2013). the ‘new frontier’: emergent indigenous identities and social media. in m. harris, m. nakata & b. carlson (eds.), the politics of identity: emerging indigeneity (pp. 147-168). sydney: university of technology sydney e-press culnan, m. j., mchugh, p. j., and zubillaga, j. i. 2010. "how large us companies can use twitter and other social media to gain business value," mis quarterly executive (9:4). gikas, j., and grant, m. m. 2013. "mobile computing devices in higher education: student perspectives on learning with cellphones, smartphones & social media," the internet and higher education (19), pp. 18-26. kaplan, a. m., and haenlein, m. 2010. "users of the world, unite! the challenges and opportunities of social media," business horizons (53:1), pp. 59-68. rodriguez, n. s. 2016. "communicating global inequalities: how lgbti asylum-specific ngos use social media as public relations," public relations review (42:2), pp. 322-332. copyright: © 2018 de silva, bedingfield & alahakoon. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. australasian journal of information systems de silva, bedingfield & alahakoon 2018, vol 22, research on recent advances in social media preface 4 paper title (paper title style) australasian journal of information systems ramaprasad, win, syn, beydoun, dawson 2016, vol 20, research article ontology of australian health programs 1 australia’s national health programs: an ontological mapping arkalgud ramaprasad university of illinois at chicago khin than win university of wollongong win@uow.edu.au thant syn texas a&m international university ghassan beydoun university of technology, sydney, linda dawson university of wollongong, abstract australia has a large number of health program initiatives whose comprehensive assessment will help refine and redefine priorities by highlighting areas of emphasis, under-emphasis, and non-emphasis. the objectives of our research are to: (a) systematically map all the programs onto an ontological framework, and (b) systemically analyse their relative emphases at different levels of granularity. we mapped all the health program initiatives onto an ontology with five dimensions, namely: (a) policy-scope, (b) policy-focus, (c) outcomes, (d) type of care, and (e) population served. each dimension is expanded into a taxonomy of its constituent elements. each combination of elements from the five dimensions is a possible policy initiative component. there are 30,030 possible components encapsulated in the ontology. it includes, for example: (a) national financial policies on accessibility of preventive care for family, and (b) local-urban regulatory policies on cost of palliative care for individual-aged. four of the authors mapped all of australia’s health programs and initiatives on to the ontology. visualizations of the data are used to highlight the relative emphases in the program initiatives. the dominant emphasis of the program initiatives is: [national] [educational, personnelphysician, information] policies on [accessibility, quality] of [preventive, wellness] care for the [community]. however, although (a) information is emphasized technology is not; and (b) accessibility and quality are emphasized cost, satisfaction, and quality are not. the ontology and the results of the mapping can help systematically reassess and redirect the relative emphases of the programs and initiatives from a systemic perspective. keywords: ontology; health initiatives; health program; ontological framework 1 introduction health programs promote, restore and maintain health (world health organization 2011). national health programs vary significantly across countries based on their needs, initiatives, policies and stakeholders involved (núñez mondaca, ramaprasad, and syn 2015, ramaprasad, nanda kumar, and syn 2015). they evolve with the changing environment, demographics, and the requirements of the country’s population. australia’s department of health has several programs and initiatives to support the nation’s health system. its states, territories, and local governments have their own programs which operate concurrently. we focus on australia’s national programs and initiatives. health in all policies—one of the outcomes of the 1986 ottawa charter for health promotion— introduced the concept of better health, improved population health outcomes, and closing the health gap (kickbusch, mccann, and sherbon 2008) through policies in all sectors and areas. nations all over the world have been seeking ways to implement this. in the usa, healthy australasian journal of information systems ramaprasad, win, syn, beydoun, dawson 2016, vol 20, research article ontology of australian health programs 2 people 2020 initiatives built on the social determinants of health approach (koh et al. 2011). in australia, policies across different health sectors recognise the social determinants of the health approach too (fisher et al. 2016). government policy and health planning set the direction for the distribution of health services, workforce, resource allocation, and social environment (heilbrunn-lang et al. 2016). health service needs of the population have to be satisfied by the healthcare workforce (lopes, almeida, and almada-lobo 2015). healthcare disparities or inequalities exist in different health communities. a culturally responsive framework was proposed in 2012 for the culturally diverse population in australia (gill and babacan 2012) to provide (a) better links between access, equity, quality, and safety, (b) better health outcomes for culturally and linguistically diverse (cald) populations, (c) enhanced cost effectiveness of service provision, and (d) culturally responsive and existing standards compliant services (seeleman et al. 2015). accessibility to health services, workforce distribution, recruiting and retaining staff, and having quality care services have been the challenges for rural australia (malatzky and bourke 2016, bourke et al. 2012). support in workforce, training for rural and regional healthcare, continuing professional development access, family support, recognition, and remuneration have been identified as needs for the workforce in australia (larkins and evans 2014). australian national health policy investment and deinvestment timeline was presented by smith, crawford, and signal (2016). release of national preventive health strategy in 2008 and national primary healthcare strategic framework in 2013 are noteworthy while the deletion of australian national preventive health agency and the cessation of the national partnership agreement on preventive health in 2014 are of particular concern (smith, crawford, and signal 2016). significant health policy changes under government policy changes or budget will have impact on health outcome (moore and yeatman 2014). in this context, we analyse australia’s national health programs systematically and systemically, at a point in time. the analysis provides the ‘big picture’ of the areas of strong, moderate, and no emphasis in the programs. as such, it highlights the strengths and weaknesses of the programs and provides a roadmap for changing the future trajectory. australia’s department of health’s vision, purpose, and strategic priorities are as follows (australian government department of health 2016c): vision: better health and wellbeing for all australians, now and for future generations purpose: lead and shape australia’s health system and sporting outcomes through evidence based policy, well targeted programs, and best practice regulation strategic priorities: better health outcomes and reduced inequality through: • an integrated approach that balances prevention, primary, secondary, and tertiary care • promoting greater engagement of individuals in their health and healthcare • enabling access for the most disadvantaged including aboriginal and torres strait islander people, people in rural and remote areas and people experiencing socio-economic disadvantage affordable, accessible, efficient, and high quality health system through: • partnering and collaborating with others to deliver health programs • better, more cost-effective patient care through innovation and technology • regulation that protects the health and safety of the community better sport outcomes through: • boosting participation opportunities for all australians australasian journal of information systems ramaprasad, win, syn, beydoun, dawson 2016, vol 20, research article ontology of australian health programs 3 • optimising international performance • safeguarding integrity in sport to fulfil the above aspiration, the department has a large number of programs and initiatives. the current list is available at australian government department of health (2016d). in april 2015 when this study was conducted, there were 163 programs listed on the site. we posed the question: do the programs and initiatives fit the aspiration? the feedback from this study will help answer this question and aid future program redirection and development. health programs are formulated over long periods of time, by different governments, and different agencies of the governments. they may originate in legislation passed in the parliament, the reports of committees, the judgment of courts, and response to public pressure. sometimes the programs are products of systematic thinking about a problem by a committee, think-tank, or other entities. often they are formulated opportunistically to respond to the needs of the time, and incrementally to change or correct programs of the past. australia’s department of health was established in 1921—its name, structure, and function have been changed a number of times since then (australian government department of health 2016b). the annual reports of the department summarize the performance in terms of a variety of outcome measures such as: (a) population health, (b) access to pharmaceutical services, (c) access to medical and dental services, (d) acute care, (e) primary health care, (f) private health, (g) health infrastructure, regulation, safety and quality, (h) health workforce capacity, (i) biosecurity and emergency response, and (j) sport and recreation. comprehensive assessment including an understanding of interactions and relationships of a country’s health programs and initiatives will help refine and redefine priorities by highlighting areas of emphasis, under-emphasis, and non-emphasis. without an appropriate means to grasp the ‘big picture’ of the entire nation’s healthcare policies, policy-makers play the risk of missing the forest for the trees. bennett and peters (2015, 10) suggest four reasons for such an assessment: 1. to motivate health system reforms; 2. to promote harmonization and alignment across actors in the health system; 3. to translate health systems reforms into meaningful tracking and evaluation of systems performance; and 4. to facilitate learning through cross-country comparisons. the description of australia’s programs and initiatives constitutes a large volume of text data. there are few techniques to systematically and systemically assess the programs based on this ‘big text data’. in this paper we use meta-analysis and synthesis of the data using an ontology as a tool to mine the data (ramaprasad and syn 2015). the ontology provides a set of explanatory terms and relations that anchor the description of the domain enabling consensus and explicit sharing of reflections on australia’s health programs and initiatives. we will illustrate its use to map the topography of this knowledge domain and visualize the “knowledge structure” (zhang et al. 2012) of the domain. the health programs and initiatives are treated as realizations of the concepts articulated in the ontology. they provide textual empirical data that sample the ontological topography. the topography represents a “virtual knowledge landscape” (scharnhorst 2001, 505) based on textual empirical data. it will help to visually recognize the coherence and lack of it in the cumulative domain knowledge, and therefore help correct the lacuna when appropriate (hoeffner and smiraglia 2013, noar and zimmerman 2005). it will help visualize the bright, light, and blank/blind spots in the knowledge domain. thus, it will provide “support for navigating the knowledge landscape.” (kazimierczak et al. 2012, 1) further, “[i]ncrementally computed information landscapes are an effective means to visualize longitudinal changes in large document repositories…” (syed et al. 2012, 352). “given diverse rationales for undertaking hsas [health system assessments], there is no one standardized framework to guide them. frameworks and tools should be selected to reflect australasian journal of information systems ramaprasad, win, syn, beydoun, dawson 2016, vol 20, research article ontology of australian health programs 4 proposed use…” (bennett and peters 2015, 9). in the following, we present an ontology-based assessment of australia’s health programs and initiatives. it is adapted from an earlier effort to analyse and synthesize the healthcare policies of chile (núñez mondaca, ramaprasad, and syn 2015). we will describe the deconstruction of the construct of ‘healthcare policy’ into its constituent dimensions, and the dimensions into their constituent elements. we will then present the mapping of australia’s national health programs and initiatives onto the ontology to illuminate the ‘bright’, ‘light’, and ‘blind/blank’ spots which indicate the policy areas with significant emphasis, little emphasis, and no emphasis, respectively. lastly, we will discuss how the ontological framework can be used to articulate the key themes and questions systematically and systemically. such a meta-analysis of the health programs in their entirety will help develop a roadmap for the future. 2 ontology of health policies the ontology of healthcare policy represents our conceptualization of the healthcare policy domain (gruber 2008). it is an “explicit specification of [our] conceptualization,” (gruber 1995, 908) and can be used to systematize the description of a complex system (cimino 2006) such as national healthcare policies. the ontology organizes the terminologies and taxonomies that constitute healthcare policies. “our acceptance of [the] ontology is… similar in principle to our acceptance of a scientific theory, say a system of physics; we adopt, at least insofar as we are reasonable, the simplest conceptual scheme into which the disordered fragments of raw experience can be fitted and arranged.” (quine 1961, 16) the method of constructing and presenting an ontology for a domain is explained in greater detail in ramaprasad and syn (2014a, 2014b). it is an iterative process that systemically and systematically deconstructs healthcare policy into its dimensions and associated elements. it is a new method of analysing and synthesizing knowledge in a domain (ramaprasad and syn 2015, tate et al. 2015). it has been applied to analysing and synthesizing chile’s and india’s national healthcare policies (núñez mondaca, ramaprasad, and syn 2015, ramaprasad, nanda kumar, and syn 2015). here we apply it to policies governing australia’s healthcare system. the challenge is to construct an ontology which is a logical, parsimonious, and complete description of australia’s national healthcare policy. it has to be logical in the deconstruction of the domain, and parsimonious yet complete in the representation of the domain. it has to be a closed description of a country’s healthcare policy in its entirety yet adaptable to changes in the domain. there can be multiple points of views to study a complex domain like national healthcare policy. each view can be represented in its own ontology. it is a ‘wicked’ problem (churchman 1967) with many potential formulations (ramaprasad and syn 2014a). each ontology can be seen as a lens to study a country’s healthcare policy. the ontological analysis provides a holistic approach to understanding healthcare policies; the traditional methods tend to be linear focusing on a subset of its elements, fragments, or components. in fact, our method simply reorganizes the traditional bodies of knowledge derived from traditional methods in a novel way to provide a holistic understanding. thus, dimension and taxonomies constituting the ontology are not new; most of them are in the literature on healthcare, public policy, and development. the ontology helps synthesize the knowledge from these disciplines in the context of healthcare policies to provide a holistic perspective. 2.1 construction of the ontology the domain of interest is healthcare policy. it can be conceptualized as a composite of two subdomains—‘healthcare’ and ‘policy’—each with a considerable corpus of research and practice but in different disciplines. each subdomain can be further divided as discussed below. australasian journal of information systems ramaprasad, win, syn, beydoun, dawson 2016, vol 20, research article ontology of australian health programs 5 the construct of healthcare, especially in the context of healthcare policy, commonly refers to: (a) a population whose health is of concern to the policy makers; (b) the different types of care which constitute healthcare; and (c) the various outcomes desired of healthcare. the distinction by population is made to recognize their different needs, to cater to them effectively. some needs have to be addressed at the individual level; some may be addressed at the family or community level. some needs are very specific (for example, pre-natal children); some are more general (for example, adults). the distinction by type of care is necessary to accommodate the different types, amount, and cost associated with each type. healthcare policies have to account for these differences to be effective. healthcare policies seek many outcomes from the healthcare of the population. these outcomes are specified in the design of the policies, measured to assess them, and used to provide feedback to (re)direct their trajectory. healthcare policies seek multiple outcomes for the target population based on the different types of care. policies have many attributes. among them, their scope and focus directly impact the outcomes of different types of care for the various populations. healthcare is a subject that is concurrently addressed at the global, national, local, and institutional levels. although the subject of our study is australia’s national healthcare policies, some of their scope may be more general, specific, or both. the scope can affect the breadth and depth of a policies effects on the health outcomes. simultaneously with the scope (which is the equivalent of geographical focus) the substantive focus of the policies varies. policies seek to focus on the key attributes of the system necessary to deliver care to the population. some policies may have a singular focus and some may have multiple foci. in conjunction with their scope the focus affects the outcomes. thus, we deconstruct healthcare policy into (a) the scope of the policy, (b) the focus of the policy, (c) the desired outcomes of the policy, (d) the type of care which is the objects of the policy, and (e) the population for which it is intended. a healthcare policy is ideally seen as having a defined scope, being focused on an aspect of healthcare, with desired outcomes, for particular types of care, targeted at a given population. the underlying argument is: healthcare policy = scope + focus + outcome + care + population these five dimensions define the domain of healthcare policies. in practice, all healthcare policies may not cover all the dimensions. for example, some may not specify the outcome, some the focus, and some a combination of these. we will refer to policies in which some dimensions are excluded from the definition as fragments of policies rather than full policies. each dimension of the ontology is expressed by a taxonomy of its constituent elements (figure 1). the elements are defined in the glossary in appendix. the taxonomies are derived from the common terminology in the body of knowledge on each dimension, especially in the healthcare policy domain. (note: we will capitalize the words which refer to the dimensions and elements in the ontology, except in describing full or partial components.) thus, the scope of a policy may be global, national, local-urban, local-rural, or restricted to the provider. they are primarily defined geographically. the categories are arranged from the most general to the specific and are grounded in practice in australia and other countries (chile, india, and china) that have been studied by two of the authors (núñez mondaca, ramaprasad, and syn 2015, ramaprasad, nanda kumar, and syn 2015). the research literature addresses these levels but does not present it as a taxonomy. they evolved iteratively through abstraction from the corpus of policies and application to the same corpus to test the fit. the last category, provider, for example, was added to accommodate the distinctive policies of these entities which are not inherited from the different levels (of the government) above. australasian journal of information systems ramaprasad, win, syn, beydoun, dawson 2016, vol 20, research article ontology of australian health programs 6 figure 1 ontology of national healthcare policy with illustrative components the scope categories represent the units of attention and action of the policies—the levels at which policies address the healthcare of a population. they reflect the porosity of geographical boundaries in healthcare, especially in the context of air travel and fast spreading epidemics, making it necessary to address the issues at all levels simultaneously. they also reflect the particularity of many health problems which have to be addressed locally. urban health problems, for example, are often different from rural ones. at the lowest level of the dimension, the provider’s policies may ultimately determine the care delivered and its outcome. thus: scope ⊂ [global, national, local (urban, rural), provider] the focus of the policy may be drugs, educational, financial, insurance, information, personnel, regulatory, technology, treatment, or administration. the personnel focus may be on the physician, nurses, or staff. these can be said to be the common instruments of healthcare policy to achieve the outcomes. they are not presented in any particular order. the categories are grounded in literature (bourke et al. 2012, larkins and evans 2014, malatzky and bourke 2016) and practice, derived iteratively through abstraction from and application to the corpus. they cover all the primary foci of healthcare policies. a policy may have a singular focus or multiple foci. however, a combination of these foci/instruments is essential for the delivery of care for different populations to achieve the outcomes. the emphases on these foci may vary significantly based on the type of care, the population, and the desired outcomes. drugs, for example, may be important for illness care; on the other hand, education may be important for preventive care. thus: focus ⊂ [drugs, educational, financial, insurance, information, personnel, regulatory, technology, treatment, administration] personnel ⊂ [physician (general, specialist), nurses, staff] scope focus outcomes care population global [+] drugs accessibility preventive individual national educational cost wellness children local financial quality pregnancy pre-natal urban insurance satisfaction illness post-natal rural information safety episodic other provider personnel parity chronic adolescents physician timeliness palliative adults general mothers specialist workers nurses other staff aged regulatory family technology community treatment indigenous administration other illustrative components (total = 5*13*7*6*11 = 30,030): national financial policies on accessibility of preventive care for family. local urban regulatory policies on cost of palliative care for individual aged. provider administration policies on cost of illness episodic care of individual adolescents. policy [p ol ic ie s o n] [o f/ w ith ] [c ar e fo r/ of ] australasian journal of information systems ramaprasad, win, syn, beydoun, dawson 2016, vol 20, research article ontology of australian health programs 7 the desired outcomes of a healthcare policy have been articulated by the us government (ramaprasad, syn, and thirumalai 2015), the who (world health organization), and in the policies themselves. collating and combining them, the outcomes of the policy may be accessibility, cost, quality, satisfaction, safety, parity, or timeliness of healthcare. these categories comprehensively cover the desired outcomes of the policies. they are not presented in any particular order; accessibility however appears to be the primary concern universally. understandably, without accessibility the other outcomes would not be relevant. outcomes ⊂ [accessibility, cost, quality, satisfaction, safety, parity, timeliness] the types of healthcare are fairly well established, although they may not be presented as a formal taxonomy. historically, healthcare has been synonymous with illness care; preventive, wellness, and palliative care are of recent origin. within illness care, chronic care become a distinctive subcategory with the increasing prevalence of chronic illnesses such as diabetes, and even some types of cancers being designated as chronic conditions. pregnancy care is defined as a separate category with the increasing focus on maternal and child health and reproductive health. thus, the care could be preventive, for wellness, pregnancy, illness (episodic or chronic), or palliative. care ⊂ [preventive, wellness, pregnancy, illness (episodic, chronic), palliative] last, population cared for may be the individual, family, or community. the individuals may be children (pre-natal or post-natal), adolescents, adults (mothers, workers), and aged. (gill and babacan 2012, heilbrunn-lang et al. 2016, lopes, almeida, and almada-lobo 2015) the sub-categories and the sub-sub-categories in the population taxonomy indicate the finegrained attention to these groups in the policies based on their healthcare needs. they are grounded in the policies themselves and extracted through the iterative process of abstraction and application described earlier. for example, australia’s health programs pay special attention to indigenous communities. in contrast, chile and india’s policies (núñez mondaca, ramaprasad, and syn 2015, ramaprasad, nanda kumar, and syn 2015) do not highlight their indigenous (tribal in india) populations. hence they form a separate subcategory. thus: population ⊂ [individual, family, community] individual ⊂ [children (pre-natal, post-natal, other), adolescents, adults (mothers, workers, other), aged] community ⊂ [indigenous, other] thus, the ontology is a product of: (a) logical decomposition and recomposition of the domain, (b) abstraction from and application to the policies in the domain, and (c) appropriation and adaptation of taxonomies from the research and policy corpus of the domain. it is a systemic and systematic representation of the domain. in the next section, we discuss how the ontology can be used as a lens to map the domain and study it. 2.2 ‘bright’, ‘light’, ‘blind/blank’ spots the ontology in figure 1 helps visualize the dimensions, sub-dimensions, elements, fragments, and components of healthcare policy in natural english. each dimension is represented by a column. the taxonomy of elements constituting each dimension/sub-dimension is listed in the respective column. all the elements together constitute the elements of the system of policies. the columns are connected by adjacent words/phrases—they help concatenate the elements from different columns into phrases/sentences which describe the fragments/components which constitute the system. thus, for example: (a) accessibility of preventive care for family can be a fragment of a system, and (b) local-urban regulatory policies on cost of palliative care for individual-aged can be a component of a system. there are (5*13*7*6*11 = 30,030) potential first-order components of the system of policies encapsulated in the ontology. if one considers interactions among the elements of a column (for example, combination of different types of care) then there are even more second-order and higher-order components. further, there are a very large number of australasian journal of information systems ramaprasad, win, syn, beydoun, dawson 2016, vol 20, research article ontology of australian health programs 8 fragments of these components also encapsulated in the ontology. the ontology encapsulates the combinatorial complexity of a nation’s healthcare policy. three illustrative components derived from the ontology are shown in figure 1. they are: 1. national financial policies on accessibility of preventive care for family. for example, policies to provide financial incentives for families to travel to obtain preventive care. 2. national regulatory policies on cost of wellness for individual-aged. for example, policies to limit the cost of annual check-ups for senior citizens. 3. provider administration policies on parity of illness-episodic and illness-chronic care for family. for example, policies on satellite clinics in low-income neighbourhoods. a component, fragment, or element in the ontology may or may not be instantiated in a particular policy. studying across policies, some of them may be instantiated frequently, some infrequently, and others not at all. we will label the frequently instantiated portions the ‘bright’ spots, the infrequent ones the ‘light’ spots, and the overlooked ones the ‘blind/blank’ spots. the luminosity of each spot is a product of two opposing dynamics. a ‘bright’ spot may be so because it is effective and important; it may also be a consequence of habit and herd effect (‘more of the same’), irrespective of whether it is effective or important. a ‘light’ spot may be so because it is ineffective and unimportant; it may also be a consequence of difficulty of implementing it (‘path of least resistance’), irrespective of its potential effectiveness or importance, or its recent emergence in importance. a ‘blind/blank’ spot may have been simply overlooked by design or by accident; or, it may be infeasible. knowing the ‘bright’, ‘light’, and ‘blind/blank’ spots in the policies and their antecedent reasons will help develop more systemic and systematic approaches to the challenge of healthcare policies. in the following we present an ontological map of health programs and initiatives in australia, highlight the ‘bright’, ‘light’, and ‘blind/blank’ spots, and discuss possible reasons for the same. before presenting the results, we will first describe the method we used for mapping. in the conclusion we will present the potential implications of this program of research and the planned extensions to what is presented here. the distinguishing feature of the proposed ontology is that it articulates the logic of the system in structured natural english words, phrases, and sentences. it is neither an abstract, philosophical discussion for meta-cognition (imagination) of the system, nor a concrete, syntactically precise formalization of the system for machine cognition. it is between the two extremes—a pragmatic, semantically rich articulation of the system for human cognition. the ontology itself can be refined by adding subcategories of elements, coarsened by aggregating categories of elements, extended by adding additional dimensions, and reduced by eliminating current dimensions. thus, it can be adapted to design healthcare policy at different levels of complexity and granularity (ramaprasad and syn 2015, ramaprasad, syn, and win 2014, ramaprasad and syn 2014a). 2.3 validity of the ontology the validity of the ontology will determine how well the ontology captures and represents a national healthcare policy. the common methods focus on induced ontologies at a finer level of detail, greater formalism, and machine readability, but not on deduced ontologies at a higher-level of abstraction such as ours. hence, we draw on the traditional constructs of validity and assert the ontology’s face validity, content validity, semantic validity, systemic validity, and external validity, which are commonly used in social science research (brennan, voros, and brady 2011, horn and lee 1989). we can judge the validity of the ontology of national healthcare policy based on the following questions: how explicit is the conceptualization? how logical is the conceptualization? how specific is the conceptualization? how systematic is the description? how systemic is the australasian journal of information systems ramaprasad, win, syn, beydoun, dawson 2016, vol 20, research article ontology of australian health programs 9 description? are the dimensions basic to the domain? are the taxonomies basic to the dimensions? are the components basic to the domain? we address these questions below to establish the ontology’s validity. the ontology is a complete and closed description of australia’s national healthcare policy. it is logically derived as described earlier. it is a novel reorganization and representation of traditional bodies of knowledge from multiple disciplines such as public policy, healthcare, and social science. it derives part of its validity from that of the knowledge in the underlying disciplines. it comprehensively describes a country’s healthcare policy using structured natural english understandable to novices and experts alike. it deconstructs the combinatorial complexity of a construct and presents it visually as an easily readable, parsimonious text-table (tufte 1990). the ontology encapsulates all 30,310 possible components of healthcare policy in a readable form in a single sheet of paper. thus, its face validity is high. each logical component of national healthcare policy derived from the ontology is semantically meaningful as illustrated above; thus, its semantic validity is high. it is easy to ascertain whether a component is instantiated or not. obversely, it is also easy to ascertain the component that matches an instantiation. its dimensions are based on the common bodies of knowledge from underlying disciplines. the taxonomies include the basic categories of knowledge from these disciplines. hence, the content validity of the dimensions, the taxonomies, and the large number of consequent components is high. further, the ontology encapsulates all the possible components of australia’s national healthcare policy. it has high systemic validity. lastly, the ontology is also validated through application to the data (programs and initiatives). the application will highlight errors of omission of significant elements and of commission of irrelevant elements in the ontology. these errors are easily corrected—the ontology is easily extensible and reducible—and the validity of the ontology enhanced. in the same vein, most of the ontology has been validated through application to chile’s and india’s healthcare policies and learning from the same. 3 methods we downloaded the list of all health programs and initiatives from the department of health’s website (australian government department of health 2016d) in april 2015. a total of 163 programs and initiatives were imported into an excel spreadsheet together with links to the details. four of the authors used a spreadsheet tool to map each program/initiative to the ontology. each coder coded about half the programs. the coding assignment ensured that each coder’s programs overlapped about 50% with a second coder’s and the rest 50% with a third coder’s. thus, each program was coded by at least two coders, and each coder’s coding was compared with two others’ for validation. the programs and initiatives were coded in two rounds with significant feedback and learning between the rounds. the coders were instructed to base their judgments on explicit attributes of the programs and initiatives in the description. they were also instructed to resist imputing attributes. the first round of coding was based on the coders’ understanding of the terms based on the project discussions. a comparison of the coding of overlapping sets (as described above) revealed significant variations in the same. the feedback on differences was provided to the coders on a color-coded spreadsheet as well as an ontological map (described later). the feedback revealed to each coder his/her bias with reference to two other coders. the feedback was discussed by the coders. part of the differences was attributed to the absence of glossary of terms. in other words, coders’ interpretations of the terms were slightly different australasian journal of information systems ramaprasad, win, syn, beydoun, dawson 2016, vol 20, research article ontology of australian health programs 10 in some cases. consequently, a consensus glossary as shown in appendix was developed, after a few iterations, to aid the second round of coding. the second round of coding was based on the glossary. the comparison of coding revealed much greater agreement than in the first round. the coders debated whether they should undertake a third round to achieve greater consensus or base subsequent analysis on the union of their coding. the latter was adopted for the following reasons. it would be more inclusive— the final data would include what one might expect to be natural differences in interpretation of the text data. it may not be possible to achieve complete consensus given the nature of the data, the backgrounds of the coders, and the interpretation necessarily involved in coding. thus, overall, the final coding traded possible errors of omission for possible errors of commission. the mapping of health programs and initiatives to the ontology is based on the presence or absence of concepts related to individual ontological elements. the coding is symmetric in nature since both presence (coded as ‘1’) and absence (coded as ‘0’) of an element are significant. the similarity between symmetric variables is generally measured by simple matching coefficient (smc) (cheetham and hazel 1969, gower 1971). the average smc of the final coding is 90.63%, a significant agreement among coders. 4 results we present the results of the analysis as an ontological map (figure 2) and a dendrogram (figure 3). the former highlights the emphasis on the elements of the ontology in the 163 programs and initiative. the latter highlights the clustering of elements of the ontology in the corpus. they provide two different but related perspectives on australia’s national health programs and initiatives. we discuss the two in greater detail below. figure 2 ontological map of monads 4.1 ontological map the ontological map of individual ontological elements (monads) is shown in figure 2. the number in parentheses adjacent to an element indicates its frequency of occurrence in the 163 health programs and initiatives. the data bar below the element is a visual indicator of the same scaled to the maximum frequency of occurrences of any one element, national (138) in scope focus outcomes care population global (3) drugs (16) accessibility (58) preventive (43) individual children pre-natal (5) national (138) educational (56) cost (18) wellness (41) individual children post-natal (14) local urban (9) financial (28) quality (35) pregnancy (7) individual children others (8) local rural (39) insurance (4) satisfaction (6) illness episodic (14) individual adolescents (17) provider (39) information (45) safety (18) illness chronic (20) individual adults mothers (20) personnel physician general (48) parity (21) palliative (1) individual adults workers (16) personnel physician specialist (29) timeliness (20) individual adults others (0) personnel nurses (21) individual aged (14) personnel staff (28) family (16) regulatory (27) community indigenous (25) technology (6) community others (80) treatment (35) administration (39) [c ar e fo r/ of ] [+ ] [p ol ic ie s/ pr og ra m s o n] [o f/ w ith ] policy/program australasian journal of information systems ramaprasad, win, syn, beydoun, dawson 2016, vol 20, research article ontology of australian health programs 11 this case. a program/initiative may be coded on multiple elements on a dimension (column) or no items. hence, the sum of the numbers in each column may not equal 163—it may be more or less. the visualization in figure 2 clearly highlights the areas of emphasis, limited emphasis, and no emphasis. we will discuss the map in detail below from right to left. 4.2 population the dominant focus of the programs is the community-others (80), the general community, and community-indigenous (25), the indigenous community. the programs are dominantly community-based, not familyand individual-based. the secondary focus of the programs is individual-adults-mother (20), individualadolescents (17), individual-adults-workers (16), family (16), individual-aged (14), and individual-children-post-natal (14). the tertiary focus is on individual-children-other (8) and individual-children-pre-natal (5). it is noted, however, that there are 5 or more programs that address the needs of each segment of the population identified in the ontology. there are no ‘blind/blank’ spots in the population dimension. 4.3 care the dominant focus of the programs is on preventive (43) and wellness (41) care. the secondary focus is on illness-chronic (20) and illness-episodic (14) care. the tertiary focus is on pregnancy (7). one program focuses on palliative (1) care. preventive and wellness care are the dominant drivers of the programs. combining the care and population dimensions it may be reasonable to infer that the dominant emphasis of the programs is on the preventive/wellness care of the community. 4.4 outcomes the dominant outcomes sought are accessibility (58) and quality (35). the secondary outcomes are parity (21), timeliness (20), cost (18), and safety (18). satisfaction (6) is tertiary. accessibility followed by quality is the dominant driver of the programs. combining the outcomes, care, and population dimensions it may be reasonable to infer that the dominant emphasis of the programs is on the accessibility/quality of preventive/wellness care of the community. 4.5 focus the dominant focus of the programs is educational (56), personnel-physician-general (48), information (45), administration (39), and treatment (35). the secondary focus is on personnel-physician-specialist (29), financial (28), personnel-staff (28), regulatory (27), and personnel-nurses (21). the tertiary focus is on drugs (16), technology (6), and insurance (4). combining the focus, outcomes, care, and population dimensions it may be reasonable to infer that the dominant emphasis of the programs is on education/general physician/ information/administration/treatment policies for accessibility/quality of preventive/wellness care of the community. 4.6 scope the dominant scope is national (138)—it is to be expected given that only national programs and initiatives were coded. there are however many programs that are local-rural (39) and provider related. very few are local-urban (9) and global (1). combining all the dimensions it may be reasonable to infer that the dominant emphasis of the programs is on national education/general physician/information/administration/treatment policies for accessibility/quality of preventive/wellness care of the community. australasian journal of information systems ramaprasad, win, syn, beydoun, dawson 2016, vol 20, research article ontology of australian health programs 12 figure 3 cluster analysis of elements of national health programs rescaled distance cluster combine 2520151050 y palliative individual_adults_others global insurance individual_children_prenatal satisfaction technology pregnancy individual_children_others individual_children_postnatal local_urban family individual_adolescents individual_adults_workers individual_aged illness_episodic individual_adults_mothers drugs safety cost illness_chronic timeliness parity personnel_nurses community_indigenous regulatory financial personnel_staff personnel_physician_specialist preventive treatment administration local_rural information quality wellness provider personnel_physician_general accessibility educational community_others national 2 42 7 19 11 5 27 21 10 4 18 17 26 12 14 8 15 41 13 24 25 30 20 23 6 36 29 39 37 35 40 3 33 34 28 16 22 32 9 1 38 31 g g g g australasian journal of information systems ramaprasad, win, syn, beydoun, dawson 2016, vol 20, research article ontology of australian health programs 13 5 cluster analysis the dominant themes can only be inferred from the ontological map (figure 2); it cannot be confirmed because it lacks the data on the co-occurrence of the elements in a program. the clusters of ontology elements based on their co-occurrence are shown in figure 3. the clusters are formed based on the coding similarity between pairs of ontology elements in the corpus measured by smc (cheetham and hazel 1969, gower 1971). as mentioned above, smc is a symmetric similarity measure which considers presence (coded as ‘1’) and absence (coded as ‘0’) of elements in the articles, in contrast to other well-known binary similarity/ distance measures such as jaccard and sørensen-dice which only consider presence of elements. in ontological analysis both presence and absence of elements in articles convey equally important information. if for example two elements are not mapped to any of the articles, both jaccard and sørensen-dice methods will return a value of zero to indicate a complete dissimilarity whereas smc will return a value of one to indicate a complete agreement which is the appropriate representation for the ontological analysis. in cases where articles include or exclude two elements in no particular pattern, all three similarity measures are comparable. however, the denominator in jaccard and sørensen-dice formulas is a variable based on the number of articles coded to one or both of a pair of elements. thus, the denominator can vary from one pair to another. the denominator in smc formula is a fixed number that represents the total number of programs and initiatives coded. smc will thus provide a more consistent comparison across pairs of elements. further, the cluster analysis is used to descriptively summarize the data about the population of articles and not to make statistical inference about the population from a sample of programs and initiatives. the clustering was conducted in spss based on the single linkage (or nearest neighbour) between the clusters. the single linkage method, as the name implies, links cluster elements based on the nearest neighbour in the existing clusters. the computation in this method is simple, and consequently the agglomeration coefficients of and the dendrogram from the cluster analysis can be translated to the data and interpreted as presented, as discussed below. (this is not the case with the centroid method, for example.) as such the method provides a direct visualization of the association between the ontology elements. these associations can be interpreted in conjunction with the frequencies of the elements in the ontological map (figure 2). the dendrogram in figure 3 visually depicts the matching of the ontology elements in the corpus of programs based on the smc. at the top are the two most infrequent elements in the ontological map—palliative (1) and individual-adults-others (0). their smc (0.994) reflects their matched absence in 162 of the 163 programs, with no matched presence. at the bottom are the two most frequent elements in the ontological map—national (138) and communityothers (80). based on their smc (0.607) one can compute that they have matched presence in 77 programs and matched absence in 22 programs. in the rest (66) one of them is present. one may compute similar values for each pair of joined elements based on their frequency of occurrence (from the ontological map) and their smc (from the dendrogram). it may be observed in the dendrogram that: (a) the less frequently present elements in the ontological map are at the top and the more frequently present elements are at the bottom, and (b) the adjacent elements are more closely matched than distant ones. by using the five equidistant divisions of smc we can infer the following themes in descending order of dominance in the programs corpus. considering more divisions will result in finer-grained themes, fewer divisions will result in coarser-grained themes. they are: 1. national programs/policies for care of community-others. this is the dominant theme of the corpus. 2. personnel-physician-general and educational policies/programs for accessibility of care. australasian journal of information systems ramaprasad, win, syn, beydoun, dawson 2016, vol 20, research article ontology of australian health programs 14 3. local-rural/provider information/administration/treatment/financial/personnel-physicianspecialist/personnel-staff policies/programs for quality of preventive/ wellness care. 4. drugs/regulatory/personnel-nurses policies/programs for parity/timeliness/cost/ safety of illness-chronic/illness-episodic care for/of individual-adults-mothers/ individual-adults-workers/individualaged/community-indigenous care. 5. global/local-urban technology/insurance policies/programs for satisfaction with pregnancy/palliative care for individual-adolescents/individual-children-postnatal/individual-adults-others/family care. the first two may be said to be the dominant themes of the corpus. the next two are non dominant themes. the last is really a non-theme rather than a theme, highlighting what is virtually absent from the corpus. in the following section we will discuss the themes and nontheme inferred from the ontological map and cluster analysis of australia’s national health programs. 6 discussion national accessibility of healthcare for the community emerges as the core theme of australia’s health programs and initiatives, based on the ontological map and the dendrogram. further, within cluster 1, accessibility and the general community are very closely associated. accessibility is explicitly articulated in the department of health’s strategic priorities. logically, it is a precondition for all other outcomes. its reflection in the country’s programs and initiatives signifies the effective translation of the priorities into programs and initiatives. on the other hand, community-based healthcare is articulated only for “[r]egulation that protects the health and safety of the community” (australian government department of health 2016c) as a strategic priority. the emphasis on community in the programs appears to be broader than that envisaged in the strategic priorities. the theme of the second cluster may be parsed by studying the underlying sub-clusters. the bottom-most sub-cluster of eight elements highlights the emphasis on the timeliness of care of individuals and families. the next sub-cluster of four elements highlights the emphasis on information and treatment for preventive and wellness care of the individuals and families. these are aligned with the strategic priorities of “[a]n integrated approach that balances prevention, primary, secondary and tertiary care” and “[p]romoting greater engagement of individuals in their health and healthcare” (australian government department of health 2016c). the complex theme of the third cluster may be best understood as a combination of two distinct sub-themes. the bottom sub-cluster focuses on provider educational/drug programs for cost/parity both episodic and chronic illness care in indigenous communities. the second subcluster focuses on personnel and financial programs for rural care. these are aligned with the strategic priority of “[e]nabling access for the most disadvantaged including aboriginal and torres strait islander people, people in rural and remote areas and people experiencing socioeconomic disadvantage” (australian government department of health 2016a). the fourth cluster consists of the residual elements. there does not appear to be an underlying theme. however, we note the following dyads. palliative care is grouped with individualadults-others to form a dyad of elements virtually absent in the programs. the same is true of global scope and insurance programs. pregnancy care is paired with individual-childrenprenatal—a logical association, but infrequent among the programs. by the same token, the association of regulatory and administration programs is understandable—they are part of many programs. last, local-urban is paired with safety and then linked to quality. it is interesting to note that quality is not strongly associated with the care elements. it is, at the next level, associated with the regulatory-administration dyad. it may be because quality is australasian journal of information systems ramaprasad, win, syn, beydoun, dawson 2016, vol 20, research article ontology of australian health programs 15 seen as an administrative and regulatory issue rather than as a care issue in the programs. part of this may be the realization of the strategic priority of “[a]ffordable, accessible, efficient, and high quality health system through… [r]egulation that protects the health and safety of the community” (australian government department of health 2016c). however, in the same cluster technology is associated with satisfaction—both occur infrequently in the programs. logically, technology programs can affect satisfaction but that doesn’t appear to be a strong sub-theme in the programs. it is contrary to the strategic priority of “[b]etter, more costeffective patient care through innovation and technology” (australian government department of health 2016c). moreover, without an emphasis on technology (and innovation) it would be difficult to achieve the overall purpose to “[l]ead and shape australia’s health system” (australian government department of health 2016c) in today’s healthcare environment. in summary, australia’s health programs and initiatives map well to its vision, purpose, and strategic priorities. there are no normative benchmarks on the relative emphasis on the different elements of the ontology and the clustering of these elements. it may be impossible and inappropriate to emphasize all elements equally. moreover, some of the elements may be emphasized in the programs of the states, territories, regional, and local areas. for example, regional programs are likely to have dominant rural focus in contrast to the national programs. yet, the ontological map and the clusters also reveal several ‘light’ spots in the corpus which may limit the programs’ overall effectiveness. the ontological map and the dendrogram provide a ‘big picture’ of the programs for further reflection. we suggest a few based on the above analysis and discussion. accessibility to healthcare is necessary but not sufficient to achieve the vision of the department of health. it is the first step but subsequently the other outcomes—cost, quality, satisfaction, safety, parity, and timeliness—assume greater importance. without these, accessibility by itself may be of little value. initiatives for providing accessibility of healthcare in rural areas include “rural clinical training support”, “mental health services in rural and remote area”, and “bonded medical services scheme”. some programs such as the “primary health network”, ataps “access to allied psychological services”, “national partnership agreement indigenous early childhood development program”, although national, have a rural and regional scope. further, the limited emphasis on technology programs may also need reflection. today, technology has become sine qua non for realizing all the outcomes. it would also be appropriate to prioritize it at the national level to facilitate interoperability, standardization, etc. in the same vein, there has been some emphasis on aged care. the “national palliative care program” emphasizes palliative care. as the demographics change and chronic illnesses increase, more palliative care programs may become necessary. focus on preventive and curative care alone, as in the past, may not be sufficient for the healthcare system. in the next section, we conclude with a discussion of the contributions of this paper, some of its assumptions and limitations, and plan for future research. 7 conclusion traditional health policy analyses often consider the outcome of the policies from a particular stakeholder’s perspective (policy makers, doctors, allied staff etc.), in a narrow domain (communicable diseases, maternal and child health, etc.), or a geographical area (rural, urban, etc.). a key contribution of the present analysis presented is that it provides a unified view of all these various perspectives and foci. the highest level of the ontology provides a unification of concepts corresponding to these multiple perspectives of analyses: scope, focus, outcomes, care, and population. the use of an ontology as a knowledge sharing mechanism is not new. however, the use of an ontology to enable combining and integrating the analysis of policies is new and is an innovative approach to gain a broader and a deeper understanding of the impact of health policies regardless of their scope or the stakeholders involved. australasian journal of information systems ramaprasad, win, syn, beydoun, dawson 2016, vol 20, research article ontology of australian health programs 16 ideally research should find application in programs; programs should provide feedback and pose problems for research; and policy should follow research and programs, and provide feedback for both. in this context we believe that the ontology can help guide, inform, and direct the research, programs, and policies on national healthcare systemically and systematically instead of fragmentarily and opportunistically. the ontology can be used to develop a roadmap for future research on national healthcare policy. 1. the paper makes three basic contributions to understanding comprehensive assessment and direction of national health programs. they are: 1. an approach using an ontology based on national healthcare policy. the approach can be extended, refined, revised, and adapted to other contexts (countries, states, territories, cities, etc.); 2. a method of mapping the state-of-the-programs for national healthcare using the ontology based on text data about the programs; and 3. a method of analysing the dominant, less dominant and non-dominant focus of a nation’s programs, and the consequent gaps in them. there is a need for policy makers to understand the implications of the above findings for future planning, prioritizing for formulation (and reformulation) of policy. our research will allow the policy makers to ask, as discussed above, for example: are the dominant foci e.g., preventative care, accessibility balanced against the low focus elements such as palliative care, satisfaction, and technology? do they need to be? ultimately ontological analysis such as this can provide a pragmatic basis for recommendations by policy makers and interest groups for policy formulation. future work will investigate broadening the effectiveness of the approach to provide detailed input that can be of used to reflect on federal budget allocation across the programs and initiatives. this will require enhancing the coding with numeric values to reflect the “degree of emphasis” in monetary terms. in ending the discussion we should also highlight some of the limitations of the research. despite our best efforts the ontology may be incomplete or over-specified. we are reasonably confident, based on the study of the corpus while coding, that the errors of omission and commission are minimal. in the future, should it be necessary, the ontology can be extended, reduced, refined, or coarsened as appropriate. the results are based on the population of australia’s national health programs at a point in time. there may be state and local programs which may be complementary and relevant. in the future, the domain could be systematically broadened to be more inclusive. considerable effort has been expended in the construction of the glossary and the monitoring of coding to minimize errors. while the coders tried to stay true to the text of the programs without imputing their own expectations, one cannot exclude the possibility of over-coding and under-coding. given the large population of programs (163) and the significant variation in the frequency of the elements, despite the potential errors, the results are likely to be robust. given the data, errors in the ontological map and dendrogram are unlikely. however, there is room for variation in the interpretation of the luminosity of the different elements and the determination of clusters. since the method of construction of the map and dendrogram are completely transparent, it would be easy to compare and contrast different interpretations of the same. in summary, despite the limitations, the insights are strong. their explanation as to why the corpus is as described may vary, but there is little room for variation in the description of the programs corpus. australasian journal of information systems ramaprasad, win, syn, beydoun, dawson 2016, vol 20, research article ontology of australian health programs 17 references australian government department of health. 2016a. "annual reports." accessed february 11, 2016. http://www.health.gov.au/internet/main/publishing.nsf/content/annual+reports-3. australian government department of health. 2016b. "history of the department." accessed february 11, 2016. http://www.health.gov.au/internet/main/publishing.nsf/content/health-history.htm. australian government department of health. 2016c. "our vision." accessed february 11, 2016. http://www.health.gov.au/internet/main/publishing.nsf/content/annual-report-1415toc~14-15-our-vision. australian government department of health. 2016d. "programs & initiatives." accessed february 11, 2016. http://www.health.gov.au/internet/main/publishing.nsf/content/programsinitiatives-menu. bennett, sara, and david h. peters. 2015. "assessing national health systems: why and how." health systems and reform 1 (1):9-17. doi: 10.1080/23288604.2014.997107. bourke, lisa, john s. humphreys, john wakerman, and judy taylor. 2012. "understanding rural and remote health: a framework for analysis in australia." health and place 18 (3):496-503. doi: 10.1016/j.healthplace.2012.02.009. brennan, linda, joseph voros, and erica brady. 2011. "paradigms at play and implications for validity in social marketing research." journal of social marketing 1 (2):100-119. doi: 10.1108/20426761111141869. cheetham, alan h., and joseph e. hazel. 1969. "binary (presence-absence) similarity coefficients." journal of paleontology 43 (5):1130-1136. doi: 10.2307/1302424. churchman, c. west. 1967. "wicked problems." management science 14 (4): b-141-42. cimino, james j. 2006. "in defense of the desiderata." journal of biomedical informatics 39 (3):299-306. doi: 10.1016/j.jbi.2005.11.008. fisher, matthew, frances e. baum, colin macdougall, lareen newman, and dennis mcdermott. 2016. "to what extent do australian health policy documents address social determinants of health and health equity?" journal of social policy 45 (03):545-564. doi: 10.1017/s0047279415000756. gill, g. k., and h. babacan. 2012. "developing a cultural responsiveness framework in healthcare systems: an australian example." diversity and equality in health and care 9 (1):45-55. gower, j. c. 1971. "a general coefficient of similarity and some of its properties." biometrics 27 (4):857-871. doi: 10.2307/2528823. gruber, thomas r. 1995. "toward principles for the design of ontologies used for knowledge sharing." international journal human-computer studies 43 (5-6):907-928. doi: 10.1006/ijhc.1995.1081. gruber, thomas r. 2008. "ontology." in encyclopedia of database systems, edited by ling liu and m. tamer özsu. new york: springer-verlag. heilbrunn-lang, adina y., lauren m. carpenter, seona m. powell, susan l. kearney, deborah cole, and andrea m. de silva. 2016. "reviewing public policy for promoting population oral health in victoria, australia (2007-12)." australian health review 40 (1):19-26. doi: 10.1071/ah15013. australasian journal of information systems ramaprasad, win, syn, beydoun, dawson 2016, vol 20, research article ontology of australian health programs 18 hoeffner, lori ann rung, and richard p. smiraglia. 2013. "visualizing domain coherence: social informatics as a case study." advances in classification research online 23 (1):49-51. doi: 10.7152/acro.v23i1.14261. horn, b. ray, and in-hee lee. 1989. "toward integrated interdisciplinary information and communication sciences: a general systems perspective." in proceedings of the 22nd hawaii international conference on system sciences (hicss 1989), 244-255. kailuakona, hawaii, usa: ieee. kazimierczak, karolina a., zoë c. skea, mary dixon-woods, vikki a. entwistle, deb feldmanstewart, james m. o. n'dow, and sara j. maclennan. 2012. "provision of cancer information as a “support for navigating the knowledge landscape”: findings from a critical interpretive literature synthesis." european journal of oncology nursing 17:110. doi: 10.1016/j.ejon.2012.10.002. kickbusch, ilona, warren mccann, and tony sherbon. 2008. "adelaide revisited: from healthy public policy to health in all policies." health promotion international 23 (1):1-4. doi: 10.1093/heapro/dan006. koh, howard k., julie j. piotrowski, shiriki kumanyika, and jonathan e. fielding. 2011. "healthy people: a 2020 vision for the social determinants approach." health education & behavior 38 (6):551-557. doi: 10.1177/1090198111428646. larkins, sarah, and rebecca evans. 2014. "greater support for generalism in rural and regional australia." australian family physician 43 (7):487-490. lopes, mário amorim, álvaro santos almeida, and bernardo almada-lobo. 2015. "handling healthcare workforce planning with care: where do we stand?" human resources for health 13 (1):1-19. doi: 10.1186/s12960-015-0028-0. malatzky, christina, and lisa bourke. 2016. "re-producing rural health: challenging dominant discourses and the manifestation of power." journal of rural studies 45:157164. doi: 10.1016/j.jrurstud.2016.03.005. moore, michael, and heather yeatman. 2014. "'killer budget' attacks prevention and primary health care." australian and new zealand journal of public health 38 (4):301. doi: 10.1111/1753-6405.12276. noar, seth m., and rick s. zimmerman. 2005. "health behavior theory and cumulative knowledge regarding health behaviors: are we moving in the right direction?" health education research 20 (3):275-290. doi: 10.1093/her/cyg113. núñez mondaca, alicia, arkalgud ramaprasad, and thant syn. 2015. "national healthcare policies in chile: an ontological meta-analysis." in medinfo 2015: ehealth-enabled health, edited by indra neil sarkar, andrew georgiou and paulo mazzoncini de azevedo marques, 1105. quine, willard van orman. 1961. from a logical point of view. second, revised ed. boston, ma, usa: harvard university press. ramaprasad, arkalgud, b. sastry nanda kumar, and thant syn. 2015. "national healthcare programs and policies in india: an ontological analysis." in proceedings of the 10th annual international conference on public policy and management (10 cppm 2015). bangalore, india. ramaprasad, arkalgud, and thant syn. 2014a. "design thinking and evaluation using an ontology." in design science: perspectives from europe, edited by markus helfert, brian donnellan and jim kenneally, 63-74. switzerland: springer international publishing. ramaprasad, arkalgud, and thant syn. 2014b. "ontological topography: mapping the bright, light, blind/blank spots in healthcare knowledge." in proceedings of the 2nd australasian journal of information systems ramaprasad, win, syn, beydoun, dawson 2016, vol 20, research article ontology of australian health programs 19 international conference on big data and analytics in healthcare (bdah 2014). singapore. ramaprasad, arkalgud, and thant syn. 2015. "ontological meta-analysis and synthesis." communications of the association for information systems 37:138-153. ramaprasad, arkalgud, thant syn, and mohanraj thirumalai. 2015. "ontological analysis of meaningful use of healthcare information systems (muhis) requirements and practice." in biomedical engineering systems and technologies, edited by guy plantier, tanja schulz, ana fred and hugo gamboa, 315-330. switzerland: springer international publishing. ramaprasad, arkalgud, thant syn, and khin than win. 2014. "ontological meta-analysis and synthesis of hipaa." in proceedings of the 18th pacific asia conference on information systems (pacis 2014). chengdu, china. scharnhorst, andrea. 2001. "constructing knowledge landscapes within the framework of geometrically oriented evolutionary theories." in integrative systems approaches to natural social dynamics, edited by m. mathies, h. malchow and j. kriz, 505-515. springer berlin heidelberg. seeleman, conny, marie-louise essink-bot, karien stronks, and david ingleby. 2015. "how should health service organizations respond to diversity? a content analysis of six approaches." bmc health services research 15 (1). doi: 10.1186/s12913-015-1159-7. smith, james a., gemma crawford, and louise signal. 2016. "the case of national health promotion policy in australia: where to now?" health promotion journal of australia 27 (1):61-65. doi: 10.1071/he15055. syed, kamran, mark kröll, vedran sabol, arno scharl, stefan gindl, michael granitzer, and albert weichselbraun. 2012. "dynamic topography information landscapes – an incremental approach to visual knowledge discovery." in data warehousing and knowledge discovery, edited by alfredo cuzzocrea and umeshwar dayal, 352-363. springer berlin heidelberg. tate, mary, elfi furtmueller, joerg evermann, and wasana bandara. 2015. "introduction to the special issue: the literature review in information systems." communications of the association for information systems 37:103-111. tufte, edward r. 1990. envisioining information. cheshire, ct: graphics press. world health organization. "quality of care." last modified september 5, 2016. http://www.who.int/management/quality/assurance/qualitycare_b.def.pdf. world health organization. 2011. "world health organization health systems strengthening glossary." accessed february 6, 2016. http://www.who.int/healthsystems/glossary_january2011.pdf. zhang, juan, jun xie, wanli hou, xiaochen tu, jing xu, fujian song, zhihong wang, and zuxun lu. 2012. "mapping the knowledge structure of research on patient adherence: knowledge domain visualization based co-word analysis and social network analysis." plos one 7 (4). doi: 10.1371/journal.pone.0034497. australasian journal of information systems ramaprasad, win, syn, beydoun, dawson 2016, vol 20, research article ontology of australian health programs 20 appendix: glossary policy: healthcare policy. scope: reach of the healthcare policy. global: policy applicable in all countries of the world. national: policy applicable everywhere in australia. local: policy applicable within a defined part of australia. urban: policy applicable within local urban areas. rural: policy applicable within local rural areas. provider: policy applicable to a healthcare providing institution. focus: focus of the healthcare policy. drugs: policies regarding drugs used in healthcare. educational: policies regarding healthcare education. financial: policies regarding healthcare finance. insurance: policies regarding healthcare insurance. information: policies regarding healthcare information. personnel: policies regarding healthcare personnel. physician: policies regarding physicians. general: policies regarding general physicians. specialist: policies regarding specialist physicians. nurses: policies regarding healthcare nurses. staff: policies regarding healthcare staff. regulatory: policies regulating healthcare. technology: policies regarding healthcare technology. treatment: policies regarding healthcare treatment. administration: policies regarding healthcare administration. outcomes: the intended outcomes of healthcare policy. accessibility: the accessibility of healthcare to the population. cost: the cost of healthcare to the population. quality: the quality of healthcare delivered to the population. satisfaction: the population's satisfaction with healthcare. safety: the safety of healthcare delivered to the population. parity: the parity of healthcare delivered to the population segments. timeliness: the timeliness of healthcare delivery to the population. care: the different types of healthcare. preventive: care to prevent illnesses and diseases in the population. wellness: care to maintain wellness of the population. pregnancy: care during pregnancy. illness: care of illnesses when they occur. episodic: care during illness episodes -time bound. chronic: care of chronic illnesses -continuing. palliative: care to alleviate pain and suffering. population: the population targeted by the policy. individual: individual recipients of healthcare. children: children who are recipients of healthcare. pre-natal: unborn children receiving care during pregnancy. post-natal: newborn children receiving care. other: children beyond post-natal care, pre-adolescent. adolescents: adolescents receiving healthcare. adults: adults receiving healthcare. mothers: mothers receiving maternal healthcare. workers: workers receiving occupational/work related healthcare. australasian journal of information systems ramaprasad, win, syn, beydoun, dawson 2016, vol 20, research article ontology of australian health programs 21 other: adults other than mothers and workers. aged: older people receiving elderly healthcare. family: family, as an entity, receiving healthcare. community: community as an entity receiving healthcare. indigenous: healthcare of the indigenous populations. other: healthcare of other identified communities. copyright: © 2016 ramaprasad, win, syn, beydoun, dawson. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 ontology of health policies 2.1 construction of the ontology 2.2 ‘bright’, ‘light’, ‘blind/blank’ spots 2.3 validity of the ontology 3 methods 4 results 4.1 ontological map 4.2 population 4.3 care 4.4 outcomes 4.5 focus 4.6 scope 5 cluster analysis 6 discussion 7 conclusion paper title (paper title style) australasian journal of information systems campbell & heales 2016, vol 20, research article individual consequences for teleworking professionals factor structure of individual consequences for teleworking professionals john campbell anu college of business and economics the australian national university john.campbell@anu.edu.au jon heales uq business school university of queensland abstract despite the increasing popularity of telework, little is known about individual outcomes that arise particularly for teleworking professionals. we build on earlier research by examining the conceptual constructs of individual consequences from telework. we initially categorized consequences identified in the extant literature into five areas. however, this model was not supported by confirmatory factor analysis of survey data collected from 250 teleworking accounting professionals. subsequent exploratory factor analysis revealed a more complex sixfactor structure relating to (1) effectiveness; (2) self-assurance; (3) working with others; (4) work pressure; (5) professional image; and (6) physicality. the results extend our understanding of latent constructs underlying telework by professional knowledge workers, and provide a basis for further refinement of our model based on empirical research and theoretical development. keywords: telework, individual consequences, structural analysis 1 introduction the incidence of telework in professional service occupations is significant and continues to increase. a recent usa survey showed that seventy-six percent of the organisations surveyed allowed some employees to routinely work a portion of their regular work hours from home (matos and galinsky, 2014). as telework gains wider acceptance, practitioners and researchers need to understand better the underlying factors involved in telework. much of the existing research has only examined the advantages and disadvantages of telework and has not adopted the necessary broader research perspectives required to examine the deeper issues and the roles played by affected stakeholders. furthermore, much of the research undertaken to date focused on organizational aspects so there is a need to balance this with research that better examines the personal/individual factors involved. we believe it is important to examine drivers of telework from an individual's perspective because the individual forms 50 percent of the relationship between telework and the organization, and ultimately makes the decision whether or not to telework. individuals should also be aware of the factors involved that affect them as teleworkers, and our work seeks to help inform them. we do know that telework introduces distinct discontinuities from traditional face-to-face work environments, and that these discontinuities have been shown to bring about significant changes in communication patterns (campbell, 2006; rafaeli and ravid, 2003). the motivation for this paper is to explore the factor structure of individual impacts associated with telework and of their association with telework practice in professional occupations. the following section briefly discusses existing research frameworks developed for studying telework. 1 australasian journal of information systems campbell & heales 2016, vol 20, research article individual consequences for teleworking professionals 2 theories of telework adoption economic factors such as increased productivity and cost reductions are frequently cited as the main organizational drivers for telework adoption (see hill et al., 1998; watad and disanzo, 2000). while economic considerations are generally important drivers for the adoption of workplace technologies, it is also important to consider the legal, ethical, and human issues, particularly for professional roles (davenport and pearlson, 1998). telework is also seen as a strategy to help ensure business continuity in the face of a pandemic or other disaster (the telework coalition, 2009). the establishment of sound working relationships between teleworkers, non-teleworkers, and managers is critical for a successful telework program (guimaraes and dallow, 1999; reinsch, 1997). alternative conceptual models for telework adoption and utilization have been proposed. some of these models focus on the relationship between different telework task configurations and employee attitudes and behaviours (feldman and gainey, 1997; shamir and salomon, 1985, hunton and harmon, 2004). others have proposed a supply/demand approach (gray, 1997), or emphasized the relationship between different telework practices and organizational outcomes (bélanger and collins, 1998). siha and monroe (2006) proposed a research model following an extensive review of the telework/telecommuting literature. for siha and monroe, competition and government regulation provide the impetus for organizations to consider telework strategies. consequently, their model follows a top-down perspective beginning with a strategic organizational dimension that is influenced by the regulatory and competitive environment within which an organization operates. this strategic view is further moderated by the level of support from both employees and management, and is reliant on the deployment of appropriate technologies to adequately support telework activities (siha and monroe, 2006 p. 466). according to siha and monroe (2006), the success of an organizational telework program is measured in terms of regulatory compliance; positive environmental impact; productivity increases and cost reductions; and worker satisfaction. however there are several substantive limitations with this model. first, telework adoption in some organizations may not follow a rational decision-making process. for example, telework may evolve over an extended period of time from the work practices of only a few individuals. second, the criteria for successful telework programs are reliant on stakeholder perspectives that are not well-defined and likely to change over time. third, there is a lack of focus on the human challenges associated with telework particularly in how work practices are altered and how these might infringe on family and personal life. in contrast, the telework behavioural model (tbm) developed by hunton and harmon (2004) focuses on psychological effects, individual outcomes, and organizational consequences. in the tbm, the link between human motivation and telework choices (whether to telework, where to telework, and how often to telework) is mediated by the range of telework options allowed in organizational policy (hunton and harmon, 2004). hunton and harmon argue that telework choices have psychological properties (cognitions and affects) that in turn influence individual outcomes such as more or less work, family conflict, reduced commute time, scheduling flexibility, and other individual-level effects. telework choices can also impact organizations in ways that will require a modification of behaviour. for example, an organization’s telework policy may result in work practices that reduce real estate costs and staff turnover, or alternatively reduce customer service and workplace collegiality. consequently, organizations will need to monitor and modify their telework policies with the aim of accruing the most value while minimizing negative outcomes. like the siha and monroe model, little is offered by the tbm for those organizations where telework practices are slowly evolving through processes other than a formal organizational policy, or those situations where individuals are able to negotiate or expropriate a unique telework regime. 2 australasian journal of information systems campbell & heales 2016, vol 20, research article individual consequences for teleworking professionals gajendran and harrison (2007) put forward a theoretical framework, based on psychological mediators and individual outcomes, to direct their meta-analysis of the telecommuting literature. their framework identified three proximal consequences (perceived autonomy, work-family conflict, and supervisor/co-worker relationship quality), and five distal consequences (job satisfaction, performance, turnover intention, role stress, and perceived career prospects). while the framework provided a useful basis for analysing the literature, it contained a number of inconsistencies in how the proximal and distal consequences were selected and categorised. for example, it can be argued that an individual’s perceptions of performance and role stress (both identified as distal consequences) are in fact proximal outcomes. this is especially relevant for professional knowledge workers. also, it can be argued that other proximal factors are missing such as professional quality of work performed, and the personal costs incurred when teleworking. more recently, campbell and mcdonald (2009) proposed a systems-based telework framework (sbtf) that presented a dynamic view of the evolution of telework practices. their model built on elements from hunton and harman (2004) and siha and monroe (2006) and drew these perspectives together to reflect three fundamental structural elements of telework practice within organizations: the motivation for adopting telework (telework drivers), telework activities and processes (telework processes), and the outcomes and consequences of telework activities (telework outcomes). these structural elements are all highly interrelated. the sbtf is also a meta-model with the potential to accommodate a range of philosophical perspectives in telework research from investigations on technology appropriation (orlikowski, 2000) through to critical studies on the micro-political processes associated with the performance and management of work (deetz, 2005). 3 research approach because the objective of our research was to examine the latent constructs underlying telework for professional knowledge workers, we first examined the literature to identify potential constructs for testing. we analysed the sbtf (employee impact), gajendran and harrison’s (2007) meta-model (proximal consequences), and the tbm (personal outcomes) approaches from the perspective of direct consequences for professional teleworkers. the nature of individual outcomes, as described in the sbtf and tbm models, is at this stage somewhat uncertain and unspecified. while gajendran and harrison’s (2007) framework identified three proximal outcomes (perceived autonomy, work-family conflict, and relationship quality), it can be argued that other important constructs are missing (e.g., professional quality and personal cost of remote work), or that some proximal outcomes have been misclassified as distal consequences (e.g., performance and role stress). however, we used these models as the starting point for specifying proximal consequences for teleworking professionals. earlier studies have also examined a range of work and home-related outcomes for teleworkers including expectations and perceived consequences (maruyama and tietze, 2012), impact on work-life balance (maruyama et al., 2009), and differences in perceived outcomes for workers from different work locations (morganson et al., 2010). in contrast our study is not causal in approach, but addresses a more fundamental examination of how individual outcomes from telework are structured. from the literature we identified five outcome themes for teleworking professionals: (1) personal costs and savings associated with telework; (2) autonomy; (3) workload and work/home-life balance; (4) working with others; and (5) professionalism and personal image. these five broad areas emerged as a way to make sense of the higher-level themes reflected in the literature, and to assist in item identification for confirmatory factor analysis (cfa) with the aim of achieving as complete a set of measurement variables as possible. these themes are discussed in further detail below. 3.1 individual outcomes for teleworking professionals an extensive review of the extant literature was undertaken to broadly identify individual outcomes relating to professional teleworkers. the outcomes from the review were categorized into the five areas detailed above, and broken down into 24 distinct items. 3 australasian journal of information systems campbell & heales 2016, vol 20, research article individual consequences for teleworking professionals 3.1.1 personal costs and savings associated with telework travel substitution was one of the earliest benefits associated with telework programs (apgar, 1998), predicting reductions in travel costs (nilles et al., 1976). however, more recent studies suggest that telework has a wider impact that may eliminate any direct benefit from travel reduction, and may in fact increase road traffic or create other travel related costs (mokhtarian, 1997; pinsonneault and boisvert, 2001; riswadkar and riswadkar, 2009; seaman, 1997). bailey and kurland (2002) found telework programs can help organizations to reduce realestate and other operating overhead costs. however, there can be significant costs for organizations implementing telework programs, and teleworkers themselves may bear some of these costs in providing home-based work areas and work infrastructure (bailey and kurland, 2002; crede, 1995; seaman, 1997; nilles, 1996). it would seem the literature is equivocal in terms of telework costs and savings of the three outcomes presented in table 1. individual outcomes literature source travel between home and the office apgar 1998; nilles et al. 1976; pinsonneault and boisvert 2001; salomon 1985; seaman 1997 work related travel apgar 1998; mokhtarian, 1997; riswadkar and riswadkar, 2009; salomon 1985; seaman 1997 costs relating to telework activities bailey and kurland 2002; crede 1995; seaman 1997; nilles 1996 table 1. costs and savings for teleworkers 3.1.2 work autonomy metzger and von glinow (1988) detailed the importance of having governance and management mechanisms in place for the effective management of telework. further, a study by guimaraes and dallow (1999) found that the characteristics of supervisors, employees, tasks, and work environment were all significant factors for successful telework programs. consequently, some researchers suggested having a different set of work rules for teleworkers (feldman and gainey, 1997; wright and oldford, 1993), and this is consistent with watad and will (2003) who found that telework can change corporate culture. flexibility in scheduling work and fewer interruptions are frequently cited in the literature as significant benefits for professional teleworkers (baruch and nicholson, 1997; chevron and primeau, 1996; hunton, 2005; pinsonneault and boisvert, 2001). empirical studies of telework have shown that there are perceived increases in efficiency (apgar, 1998) and effectiveness (bélanger, 1999), suggesting that fewer interruptions and greater work flexibility enable teleworkers to have an increased capacity to work (apgar, 1998; dubrin, 1991) and undertake more complex work tasks (bélanger, 1999; duxbury et al., 1998). task and technology fit considerations are also important for telework practice (bell and kozlowski, 2002; campbell, 1998; campbell, 2006). table 2 details eight distinct outcomes for teleworking professionals that relate to work autonomy. 4 australasian journal of information systems campbell & heales 2016, vol 20, research article individual consequences for teleworking professionals individual outcomes literature source the level of managerial control by others over work feldman and gainey 1997; guimaraes and dallow 1999; metzger and von glinow 1988; wright and oldford 1993; watad and will 2003 flexibility in scheduling work baruch and nicholson 1997; chevron and primeau 1996; pinsonneault and boisvert 2001 the complexity of work tasks bélanger 1999; duxbury et al. 1998 ; guimaraes and dallow 1999 capability to get things done in your job bélanger 1999 the time it takes to get things done apgar 1998; bélanger 1999 the number of interruptions experienced when working hunton 2005; pinsonneault and boisvert 2001 the number of different work tasks able to be undertaken bell and kozlowski 2002; campbell 1998; campbell 2006 capacity to work apgar 1998; dubrin 1991 table 2. work autonomy 3.1.3 workload and work/home-life balance a study by duxbury et al. (1992) found that after-hours teleworkers work significantly longer hours. they also found significant role overload and work spillover interference with family life. ahuja et al. (2007) also found that work-family conflict was a key source of stress for remote workers. in contrast, other studies cite greater work-family balance as the major benefit for teleworkers (dixon and webster, 1998; mokhtarian et al., 1998; shamir and salomon, 1985). the findings from investigations of the impact of telework on workload and work related pressures are also mixed. while telework can be seen as a way of alleviating a high workload and work related stress for some teleworkers (baruch, 2001; kanellopoulos, 2011), it may also increase organizational stress and workload for all employees (bailey and kurland, 2002). table 3 presents the three outcomes identified from the literature relating to workload and work/home-life balance. individual outcomes literature source time for family and friends ahuja et al. 2007; dixon and webster 1998; duxbury et al. 1992; mokhtarian et al. 1998; shamir and salomon 1985. work related stress ahuja et al. 2007; baruch 2001; bailey and kurland 2002; kanellopoulos, 2011 workload ahuja et al. 2007; baruch 2001 table 3. workload and work/home-life balance 3.1.4 working with others professional and social isolation are cited as negatives for intense telework environments (apgar, 1998; baruch and nicholson, 1997; chevron and primeau, 1996). telework can also affect personal relationships with colleagues (apgar, 1998; baruch, 2001). in some cases nonteleworkers must handle situations created by the absence of teleworkers from the usual place of work (duxbury and neufeld, 1999). kompast and wagner (1998) also reported that 5 australasian journal of information systems campbell & heales 2016, vol 20, research article individual consequences for teleworking professionals teleworkers can add to the workload of colleagues by sending requests from remote locations whilst remaining largely unavailable themselves. consequently the structure of intraorganizational communication associated with remote work can play an important role in the working relationship between teleworkers, non-teleworkers and managers (golden, 2007; harrington and ruppel, 1999; watson-manheim et al., 2012). we identified four outcomes from the literature that relate to collaborative activities and working with others. these outcomes along with their literature sources are presented in table 4. individual outcomes literature source ability to collaborate with others apgar 1998; duxbury and neufeld 1999; harrington and ruppel 1999; kompast and wagner 1998 relationship with work colleagues apgar 1998; baruch 2001; golden 2007; watson-manheim et al. 2012 collegiality in the workplace chevron and primeau 1996; kompast and wagner 1998; the ability to be contacted by work colleagues and/or clients baruch 2001; kompast and wagner 1998; table 4 working with others 3.1.5 professional and personal image as discussed earlier, professional isolation can be a major disadvantage for teleworkers and is a particularly important factor for teleworkers working in professional roles. professional isolation can lead to teleworkers feeling cut off from others and out of the loop in office interactions (golden et al., 2008). this can lead to teleworkers engaging in image management activities to convince co-workers, clients and others that real work is being performed (bailey and kurland, 2002; haddon, 1998; johnson, 1998). teleworkers can also have concerns that work performed remotely is not professional (bélanger and allport, 2008; baruch and nicholson, 1997; pinsonneault and boisvert, 2001) and not of sufficient quality (bélanger and allport, 2008; pinsonneault and boisvert, 2001). perceptions of social and professional isolation associated with telework can also impact feelings of personal achievement (ahuja et al., 2007; pinsonneault and boisvert, 2001). in a similar way, isolation can diminish the organizational status of a teleworker by reducing their visibility in the workplace (bailey and kurland, 2002; baruch, 2001). paradoxically, other research has reported that there is a positive relationship between job autonomy and organizational commitment for remote workers (apgar, 1998; hill et al., 1998). this relationship has been shown to be moderated by perceptions of work-family conflict (ahuja et al., 2007). nevertheless, a reduced sense of belonging and greater professional isolation has been shown to reduce organizational commitment (golden et al., 2008). the six potential outcomes relating to professionalism and personal image are described in table 5 along with literature sources. 6 australasian journal of information systems campbell & heales 2016, vol 20, research article individual consequences for teleworking professionals individual outcomes literature source the regard that others have for work produced bailey and kurland 2002; haddon 1998; johnson 1998 level of professionalism bélanger and allport 2008; baruch and nicholson 1997; pinsonneault and boisvert 2001; the overall quality of work bélanger and allport 2008; pinsonneault and boisvert 2001; feelings of personal achievement ahuja et al. 2007; pinsonneault and boisvert 2001 status within the organization bailey and kurland 2002; baruch 2001 commitment to the organization ahuja et al. 2007; apgar 1998; golden et al. 2008; hill et al. 1998 table 5. professional and personal image 3.2 survey development a survey instrument was developed incorporating the 24 items identified above and detailed in tables 1-5 (see appendix for instrument items). as part of the development process, the face validity of the survey items was established through a focus group consisting of three practitioners and two academic researchers who were all involved with telework and familiar with the process of survey design. a randomized anonymous survey of cpa australia members was conducted by telephone during december 2007. telework is particularly important to accounting professionals because of the nature of accounting work and the need to work effectively from remote locations. also, accounting professionals often undertake telework tasks, monitor the performance of others involved in telework, or are responsible for the allocation of resources for telework activities. because these work practices have been undertaken over many years, relationships, factors involved, outcomes, and interactions are expected to have stabilized and therefore are more likely to be valid over the long-term, not only making this a suitable environment to study, but also one that is likely to enhance the external validity of this research. for the purposes of the survey, telework was defined as the use of information and communication technology to support work activities away from the office. participants were asked to indicate the impact of telework on each of the survey items using a five-point scale ranging from “a lot more” (1) to “a lot less” (5). of the 1718 individuals contacted, 600 valid responses were obtained representing an overall response rate of 34.9 percent of the 600 accounting professional respondents, 250 reported that they undertook some level of telework representing almost 42 percent of the total sample. the 250 participants who identified as undertaking telework activity were then asked to respond to the additional 24 survey items seeking to measure the impact of telework on individual outcomes. 4 results an initial analysis was conducted to examine the potential for demographic differences both between and within the teleworker and non-teleworker groups (see table 6). approximately sixty-four percent of teleworkers were male (p<.001). there was no significant difference between the numbers of teleworkers and the number of non-teleworkers in the age group bands, with the exception of the 21 – 30 years of age where non-teleworkers were significantly higher than teleworkers (p<.004). non-teleworkers were more likely to have a partner (p<.012), and there was no significant difference in dependent children living at home between 7 australasian journal of information systems campbell & heales 2016, vol 20, research article individual consequences for teleworking professionals teleworkers and non-teleworkers. teleworkers had been with their current organization for 6.5 years on average and spent almost 11 hours per week working away from their main place of employment (6.5 hours during office hours and 4.3 hours outside of normal office hours). in contrast, non-teleworkers had been with their organizations for 5.4 years on average, but this difference was not significant at p<.05. teleworkers reported working longer hours than nonteleworkers (3 hours extra per week on average) and this was significant ( p<.001). teleworkers non-teleworkers p-value# cases (n=600) 250 (41.7% of sample) 350 (58.3% of sample) gender 36% female 51% female .001 age 21-30 years (34.8%) 21-30 years (47.4%) .004 full time position 95.2% 96% .635 partner 73.2% 63.4% .012 dependents living at home 32.8% 27.1% .134 tenure with current organization 6.5 years 5.4 years .063 hours worked per week 45.2 hours (range: 15-80 hours) 42.2 hours (range: 10-80 hours) .001 total hours of telework per week 10.8 hours per week (6.5 hours during office hours and 4.3 hours outside office hours) not applicable # chi square results testing for differences between teleworkers and non-teleworkers table 6. respondent profiles figures 1a and 1b show summary response data for each of the 24 items reported by the 250 teleworkers. in figure 1a, user responses were separated into five response categories: a lot more; a bit more; no difference; a bit less; a lot less. at one extreme the responses for nine items (5, 7, 10, 11, 13, 14 18, 20, and 21) showed that respondents associated substantial changes with telework. while change was less pronounced for the remaining fifteen items, clear patterns still emerged. for example, there were strong increases reported in terms of getting things done (11), capacity to work (7), flexibility in scheduling work (5), collaboration (10), ability to be contacted (20), and number of different work tasks undertaken (18). somewhat surprisingly there was a perception that telework had little impact on travel between home and the office (1), managerial control by others (4), and collegiality in the workplace (16). 8 australasian journal of information systems campbell & heales 2016, vol 20, research article individual consequences for teleworking professionals figure 1a. perceptions of change in work practices due to telework figure 1b. perceptions of change versus no change in work practices due to telework it could be argued that any change from normal working practices could be disadvantageous. consequently responses were recoded to enable comparison between the proportion of respondents reporting no change, and the proportion of respondents reporting either an increase or decrease from normal work practices. this data is presented in figure 1b. while no particular pattern is evident, it is clear that a diversity of opinion existed in how telework impacted professional work practices. 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 24. time for family and friends 23. status within your organization 22. your workload 21. work related stress 20. the ability to be contacted by work colleagues and/or clients 19. your commitment to the organization 18. the number of different work tasks you can undertake 17. your feelings of personal achievement 16. collegiality in the workplace 15. your relationship with your work colleagues 14. the number of interruptions you experience when working 13. the time it takes to get things done 12. the overall quality of your work 11. your capability to get things done in your job 10. your ability to collaborate with others 9. your professionalism 8. the regard that others have for your work 7. your capacity to work 6. the complexity of your work tasks 5. flexibility in scheduling work 4. the level of managerial control by others over your work 3. costs relating to your telework activities 2. your work related travel 1. your travel between home and the office a lot more a bit more no difference a bit less a lot less 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 24. time for family and friends 23. status within your organization 22. your workload 21. work related stress 20. the ability to be contacted by work colleagues and/or clients 19. your commitment to the organization 18. the number of different work tasks you can undertake 17. your feelings of personal achievement 16. collegiality in the workplace 15. your relationship with your work colleagues 14. the number of interruptions you experience when working 13. the time it takes to get things done 12. the overall quality of your work 11. your capability to get things done in your job 10. your ability to collaborate with others 9. your professionalism 8. the regard that others have for your work 7. your capacity to work 6. the complexity of your work tasks 5. flexibility in scheduling work 4. the level of managerial control by others over your work 3. costs relating to your telework activities 2. your work related travel 1. your travel between home and the office no change increase or decrease 9 australasian journal of information systems campbell & heales 2016, vol 20, research article individual consequences for teleworking professionals to test whether the hypothesized five-factor model provided a good fit to the data, confirmatory factor analysis was used to estimate the model and assess its fit to the 250 teleworker data cases. for the dataset of teleworkers (n=250), bartlett's test of sphericity is significant (p< 0.000), thus the hypothesis that the intercorrelation matrix involving the 24 survey items is an identity matrix is rejected. thus from the perspective of bartlett's test, factor analysis is feasible. as bartlett's test is almost always significant, a more discriminating index of factor analyzability is the kaiser-meyer-olkin (kmo) indicator. for this data set, it is 0.757 which is large, so the kmo also supports factor analysis. the five factors identified from the literature included (1) personal costs and savings associated with telework; (2) work autonomy; (3) workload and work/home-life balance; (4) working with others; and (5) professional and personal image. however, the model showed very poor fit (χ2 = 1008.4; df = 252). attempts to improve the model using the modification index and akaike’s information criteria (kaplan, 1991) reduced the chi-square value only marginally (χ2 = 921; df = 252). thus, the initial model was deemed an unsuitable representation of the data and its underlying constructs. follow up exploratory factor analysis (efa) was then used to examine whether the 24 items could be factored to a smaller number of constructs that provide a more meaningful basis for understanding how telework outcomes are structured (sass and schmitt, 2010; schmitt, 2011). factors were extracted using principal axis factoring with promax oblique rotation method and kaiser normalization. oblique rotation is preferred when factors are correlated (tabachnick and fidell, 2007) as was expected and later confirmed in subsequent analysis (see table 8), and promax rotation is relatively efficient at achieving simple oblique structure (abdi, 2003). initially seven factors returned eigenvalues greater than one (i.e., the k1 rule of thumb). however, on closer inspection, the seventh factor returned an eigenvalue only marginally above 1 and had only 2 items with loadings greater than 0.3 (item 13 with 0.365, and item 14 with 0.368). after considering these issues and reviewing the scree plot (see figure 2), it was concluded that a six-factor solution should be retained for further analysis. while the k1 rule has been shown to overstate the number of factors in previous studies, the scree test is frequently cited as one of the best methods available for deciding how many factors to retain for efa (costello and osborne, 2005; horn, 1965). the resulting factor structure is provided in table 7. figure 2. scree plot factor 1 was interpreted as reflecting effectiveness as it contained items relating to the effective performance of work including ability to collaborate, get things done, being contactable, 10 australasian journal of information systems campbell & heales 2016, vol 20, research article individual consequences for teleworking professionals quality of work and scheduling. factor 2 was interpreted as self-assurance as this factor contained three items concerned with organizational commitment, personal achievement, and status. factor 3 was interpreted as working with others as it contained items concerned with interpersonal relationships and collegiality in the workplace. factor 4 was interpreted as work pressure as it grouped variables reflecting work-related stress, workload, and time available for personal relationships. factor 5 was seen to reflect items related to an individual’s personal image including professionalism and the regard others have for one’s work. factor 6 was more difficult to interpret because of the multiplicity of items that loaded. however, this factor was labelled physicality as the items with higher loadings reflected aspects of location and travel between locations. more than 55 percent of total variance was explained by the six factors. only three variables (items 9, 12 and 18) were found to load on more than one factor (i.e., loading scores ≥ 0.5) suggesting that each factor reflected a different aspect of the consequences of telework. as very little item complexity was evident, the rotated factor solution was deemed to approximate simple structure and therefore suitable for analysis of the underlying factor structure. factor reliability was assessed using cronbach’s alpha with each factor examined to assess if the removal of items improved reliability. the examination showed that factor reliability could be improved for factor 3 (working with others) and factor 4 (work pressure). factor 3 initially returned a cronbach’s alpha coefficient of .55. however this increased to .77 when item 14 (the number of interruptions you experience when working) was excluded. factor 4 returned a cronbach’s alpha coefficient of .63 which increased to .68 when item 24 (time for family and friends) was excluded. items were also removed from factor 6 (physicality). however cronbach’s alpha could not be substantially improved. three items were removed from this scale to improve internal consistency between the remaining items because of low factor loadings. the three items removed were: costs relating to your telework activities (item 3); the complexity of your work tasks (item 6); and the time it takes to get things done (item 13). although removed from further consideration, items 3, 6, 13, 14 and 24 are reported in subsequent tables for completeness. table 8 reports correlations for demographic variables and scores for each of the six factors. the analysis shows that there are strong correlations between demographic variables suggesting strong links between such attributes as age, tenure, hours worked, number of dependents and full-time status. most interesting was that the amount of telework hours per week was not significantly correlated with any personal attributes, but was significantly correlated (p<.05) level with number of hours worked. this result suggests that telework may facilitate longer work hours which has implications for outcomes such as work-life balance (see maruyama et al., 2009). 11 australasian journal of information systems campbell & heales 2016, vol 20, research article individual consequences for teleworking professionals individual consequences item no. factor 1: effectiveness factor 2: self-assurance factor 3: working with others factor 4: work pressure factor 5: professional image factor 6: physicality your capability to get things done in your job 11 .780 .409 .308 .230 .371 .093 your ability to collaborate with others 10 .668 .383 .479 .334 .316 .297 the ability to be contacted by work colleagues and/or clients 20 .640 .381 .423 .384 .203 .190 the number of different work tasks you can undertake 18 .628 .583 .353 .329 .217 .278 your capacity to work 7 .627 .153 .132 .156 .312 .103 the overall quality of your work 12 .613 .508 .295 .110 .530 .087 flexibility in scheduling work 5 .364 .173 -.056 -.031 .109 .067 your commitment to the organization 19 .375 .759 .384 .126 .383 .058 your feelings of personal achievement 17 .295 .737 .250 .078 .337 .098 status within your organization 23 .397 .541 .394 .227 .341 .178 your relationship with your work colleagues 15 .303 .438 .885 .216 .395 .064 collegiality in the workplace 16 .202 .330 .660 .038 .235 .055 the number of interruptions you experience when working 14 .201 -.042 .410 .391 -.113 .302 work related stress 21 .290 .133 .261 .817 .020 .499 your workload 22 .263 .249 .201 .663 .132 .289 time for family and friends 24 -.013 .104 .010 -.387 .121 -.032 your professionalism 9 .540 .501 .305 .120 .747 .167 the regard that others have for your work 8 .352 .359 .307 .118 .737 .107 your work related travel 2 .181 .049 .048 .295 -.035 .560 your travel between home and the office 1 .070 .041 .003 .150 -.026 .525 the level of managerial control by others over your work 4 .130 .158 .211 .171 .221 .424 the time it takes to get things done 13 -.027 .090 .135 .194 .028 .316 the complexity of your work tasks 6 .081 .079 .137 .192 .112 .296 costs relating to your telework activities 3 .044 -.039 -.036 -.018 .003 .262 eigenvalue 5.393 2.261 1.759 1.534 1.306 1.154 percent of total variance explained 22.47% 9.42% 7.33% 6.39% 5.44% 4.81% cronbach’s alpha .80 .71 .77^ .68^ .75 .53^ participants were asked to indicate the impact of telework on each item using a five-point scale ranging from “a lot more” (1) to “a lot less” (5). factors were extracted using principal axis factoring and promax rotation with kaiser normalization. rotation converged in 7 iterations. ^ cronbach’s alpha for factor 3 (working with others) increased to .77 (up from .55) when item 14 was removed, and to .68 (up from .63) for factor 4 (work pressure) when item 24 was removed. cronbach’s alpha for factor 6 reduced slightly to .53 (down from .55) when items, 3, 6 and 13 were removed to improve internal consistency. table 7. factor analysis of user perceptions of the impact of telework 12 australasian journal of information systems campbell & heales 2016, vol 20, research article individual consequences for teleworking professionals mean sd 1 2 3 4 5 6 7 8 9 10 11 12 13 14 1. gender (male=1; female=2) 1.36 .48 1 2. age (mean category = 21-30 years) 3.33 1.27 -.435** 1 3. full time status (ft=1; pt=2) 1.05 .21 -.050 .281** 1 4. partner (yes=1; no=2) 1.27 .44 .116 -.263** -.051 1 5. dependents (yes=1; no=2) 1.67 .47 .217** -.256** .037 .250** 1 6. tenure (years) 6.45 7.88 -.276** .474** .068 -.153* -.142* 1 7. hours worked per week 45.19 8.96 -.223** .173** -.536** -.053 -.162* .174** 1 8. hours of telework per week 10.85 12.98 -.083 .125 .077 -.092 -.049 .007 .155* 1 9. effectiveness (7 items) 15.55 4.11 .044 -.143* -.035 -.063 -.027 -.066 -.013 -.181** 1 10. self-assurance (3 items) 7.94 1.61 -.079 -.053 -.133* -.065 -.041 -.026 .192** .063 .469** 1 11. working with others (3 items) 8.65 1.99 -.100 .041 .031 -.005 -.103 159* .095 -.015 .304** .241** 1 12. work pressure (3 items) 8.73 1.88 -.052 .059 -.137* -.003 -.120 .003 .049 -.063 .213** .104 .192** 1 13. professional image (2 items) 5.07 1.27 .041 -.128* -.091 .005 -.044 -.024 .009 -.110 .502** .421** .207** .073 1 14. physicality (6 items) 17.00 2.82 -.036 -.034 -.007 -.014 -.094 .067 -.045 -.103 .136* .084 .156* .246** .123 1 ** correlation is significant at the 0.01 level (2-tailed). * correlation is significant at the 0.05 level (2-tailed). table 8. correlations between demographic variables and factor scores 13 australasian journal of information systems campbell & heales 2016, vol 20, research article individual consequences for teleworking professionals the correlations for the six factors revealed interesting patterns. as might be expected, scores for all six factors were on the most part strongly associated with one another. however, only the effectiveness factor was significantly correlated with hours of telework per week (p<.01) and this association was a negative one. the strongest correlations (>.4) were recorded between the three factors self-assurance, effectiveness and professional image. 4.1 scale construction with factors relating to individual consequences of telework identified along with items that related to each of those factors, we were in a position to extend our work to develop an initial scale for the measurement of an individual’s consequences of telework. our procedures are consistent with (devellis, 2003) and the practice of using items from the literature. we further analysed the items relating to the six factors with a view to develop a more parsimonious model than would be suggested by the factor analysis. a review of the factor loadings indicated that all factors loaded primarily onto one factor (convergent validity), and the constructs ave values are larger than correlations with other constructs. we used ibm spss amos version 22 to test our measurement model. convergent and discriminate validity for each of the six factors was evaluated resulting in no significant loadings on other factors, and a chi-square value significantly smaller than the independent model. the chi-square/df value of 3.2 is acceptable and an rmsea of 0.094 represents a mediocre fit (hu and bentler, 1999). a gfi of 0.774 does not represent a strong fit, indicating further work is needed to better refine the scale. 5 discussion we initially categorized individual consequences of telework based on five broad areas identified in the extant literature. although confirmatory factor analysis did not support our hypothesized structure, follow-up exploratory factor analysis revealed six latent factors highlighting the complexity of individual consequences for teleworking. although caution is required in interpreting our results, the findings signal that two of the three proximal outcomes identified by gajendran and harrison (2007) are perhaps multi-dimensional and, as argued earlier, other important elements have been overlooked. gajendran and harrison’s (2007) proximal outcomes personal autonomy and relationship quality did not emerge as latent constructs for teleworking professionals in this study. however, items reflecting these issues were embedded in the effectiveness, collegiality, and physicality factors. table 9 maps the five factors outlined in the literature against the six factors that emerged from exploratory factor analysis. the workload and work-home life balance construct proposed from the literature was fully accounted for by the work pressures factor. items identified in the literature describing professionalism and personal image were largely included in either the professional image or self-assurance factors. similarly items identified in the literature concerned with work autonomy were largely contained in the effectiveness and physicality factors. work autonomy, which was prominent in the literature, proved to be a particularly complex construct thereby highlighting the need to more fully explore the nature and consequences of telework for professional knowledge workers. the costs and savings construct gleaned from the literature was accounted for by physicality which emphasised travel and time overheads. however, costs didn’t seem to matter very much to individuals (but they certainly do to organisations). work related stress is an important factor, and while it has a relatively high loading onto physicality (0.499), it loads much higher on work pressure (0.817). it is important to physicality, but not as important as work related travel (0.560) and travel between home and the office (0.525). working with others was subsumed by the collegiality and effectiveness factors which suggested that teamwork in a telework environment may be dependent on individual traits relating to the individual’s personality and their desire to be open to contact from others to collaborate. 14 australasian journal of information systems campbell & heales 2016, vol 20, research article individual consequences for teleworking professionals the findings from our analysis suggest that professional workers have perspectives on telework that incorporate aspects from all three models, campbell and mcdonald (2009), gajendran and harrison (2007), and hunton and harmon (2004). the next step in our work is to develop a more formalised and testable model that will better inform telework from the individual’s perspective and determine the drivers for a successful telework arrangement. 6 summary and conclusion this study examined the underlying structure of individual outcomes for teleworking professionals. the research was motivated by the need to build on earlier work by campbell and mcdonald (2009), gajendran and harrison (2007), and hunton and harmon (2004) to develop a better understanding of individual outcomes and their association with telework practices, particularly in a professional worker context. from the literature, we identified 24 consequences affecting teleworkers with the aim of asking professional knowledge workers how each item impacted on them as teleworking individuals (not from an organisation perspective, as most previous studies have done). to achieve this we surveyed 600 accounting professionals, of whom 250 undertook telework. the resultant exploratory factor analysis of these 250 cases revealed six significant constructs that defined individual outcomes for teleworking professionals. these outcomes are not only associated with effectiveness and work pressure, but also with perceptions of professionalism, collegiality, physicality and professional image. 15 australasian journal of information systems campbell & heales 2016, vol 20, research article individual consequences for teleworking professionals factor item no. telework activities costs and savings work autonomy workload and workhome life balance working with others professional and personal image 1. effectiveness 11 your capability to get things done in your job x 10 your ability to collaborate with others x 20 the ability to be contacted by work colleagues and/or clients x 18 the number of different work tasks you can undertake x 7 your capacity to work x 12 the overall quality of your work x 5 flexibility in scheduling work x 2. self-assurance 19 your commitment to the organization x 17 your feelings of personal achievement x 23 status within your organization x 3. working with others 15 your relationship with your work colleagues x 16 collegiality in the workplace x 14 the number of interruptions you experience when working x 4. work pressure 21 work related stress x 22 your workload x 24 time for family and friends x 5. professional image 9 your professionalism x 8 the regard that others have for your work x 6. physicality 2 your work related travel x 1 your travel between home and the office x 4 the level of managerial control by others over your work x 13 the time it takes to get things done x 6 the complexity of your work tasks x 3 costs relating to your telework activities x table 9. mapping of the six factor loadings against the literature domains 16 australasian journal of information systems campbell & heales 2016, vol 20, research article individual consequences for teleworking professionals we suggest that the six factors greatly inform modelling of proximal outcomes of telework thereby enhancing opportunities for further empirical research and theoretical development. future research should seek to better understand how each of the six factors relates to actual telework behaviour. information and communication technologies are rapidly changing the workplace and this study has shown there is latent structure underlying outcomes for teleworkers. future research is also required to test the relationship between these individual factors and organizational consequences, and social impact as suggested by the sbtf. 7 limitations there are three important limitations resulting from this study that also create opportunities for future research (e.g., see padsakoff et al., 2003). firstly, our analysis relied on a crosssectional design so our results may have been affected by differences within our sample where confounding variables such as experience were not equally distributed. although this possibility was mitigated to some extent by focusing on accounting professionals, our understanding of individual consequences would be greatly strengthened through further research using in-depth case studies. secondly, reliance on self-report data may have introduced method bias to our results. padsakoff et al. (2003) identified several potential method effects relating to biases introduced by item characteristics and individual behaviours such as affectivity, social desirability, assumptions about relationships amongst items, and acquiescence to attitudinal statements. future research should test our findings using data obtained from different sources. for example, it may be particularly advantageous to contrast the perspective of teleworkers with other stakeholder viewpoints including those of family members, colleagues, managers, and clients. thirdly, and perhaps the most significant study limitation, concerns the method used to analyse the data. the exploratory factor analysis must be interpreted carefully as the results have not been confirmed against other sample data sets. our results could be sample specific and not generalize well to the population of teleworkers. also the measurement instrument requires additional refinement to improve construct and criterion reliability. the initial description and classification of items into different outcome domains and later interpretation of constructs was a subjective process relying on published measurement items. further work is required to establish the validity of these items and the constructs that were subsequently identified. in particular, factor 6 requires further development and refinement to improve internal reliability. acknowledgment this research was supported under australian research council's discovery projects funding scheme (project number dp120104521). references abdi, h., 2003. factor rotation in factor analyses, in the sage encyclopedia of social science research methods, m. s. lewis-beck, a. e. bryman and t. f. liao eds., california: sage publications, 792-795. ahuja, m.k., chudoba, k.m, kavmar, c.j., mcknight, d.h., george, j.f., 2007. it road warriors: balancing work-family conflict, job autonomy, and work overload to mitigate turnover intentions. mis quarterly 31 (1), 1-17. apgar, m., 1998. the alternative workplace: changing where and how people work. harvard business review 76 (3), 121-30. bailey, d.e., kurland, n.b., 2002. a review of telework research: findings, new directions, and lessons for the study of modern work. journal of organizational behaviour 23 (4), 383-400. 17 australasian journal of information systems campbell & heales 2016, vol 20, research article individual consequences for teleworking professionals baruch, y., nicholson, n., 1997. home, sweet work: requirements for effective home working. journal of general management 23, 15–30. baruch, y. 2001. the status of research on teleworking and the agenda for future research. international journal of management reviews 3 (2), 113-129. bélanger f., collins, r.w., 1998. distributed work arrangements: a research framework, information society 14 (2), 137-152. bélanger f., 1999. workers’ propensity to telecommute: an empirical study. information & management 35, 139-153. bell, b.s., kozlowski, s.w.j., 2002. a typology of virtual teams implications for effective leadership. group & organization management 27 (1), 14-49. campbell, j.a., 1998. participation in videoconferenced meetings: user disposition and meeting context. information & management 34 (6), 329-338. campbell, j., 2006. media richness, communication apprehension and participation in group videoconferencing. journal of information, information technology, and organizations 1, 87-96. campbell, j. and mcdonald, c., 2009. defining a conceptual framework for telework and an agenda for research in accounting and finance. international journal of business information systems 4 (4), 387-402. chevron, j., primeau, m., 1996. the telecommuting innovation opportunity. journal of consumer marketing 13 (4), 40-8. costello, a.b., osborne, j.w., 2005. best practices in exploratory factor analysis: four recommendations for getting the most from your analysis. practical assessment, research & evaluation 10 (7), 1-9. crede, k.l., 1995. the environmental effects of the computer age. ieee transactions on professional communication 38 (1), 33-40. davenport, t., pearlson, k., 1998. two cheers for the virtual office. sloan management review 39 (4), 51-65. deetz, s., 2005. critical theory, in engaging organizational communication theory: multiple perspectives, s. may and d. mumby eds., thousand oaks, ca: sage, 85-11. devellis, r.f. 2003. scale development: theory and applications. thousand oaks, california: sage publications. dixon, t.l., webster, j., 1998. family structure and the telecommuter’s quality of life. journal of end user computing 10 (4), 42-50. dubrin, a.j., 1991. comparison of the job satisfaction and productivity of telecommuters versus in-house employees: a research note on work in progress. psychological report 68, 1223-34. duxbury, l.e., higgins, c.a., mills, s., 1992. after hours telecommuting and work-family conflict: a comparative analysis. information systems research 3, 173-90. duxbury, l.e., neufeld, d., 1999. an empirical evaluation of the impacts of telecommuting on intra-organizational communication. journal of engineering & technology management 16, 1-28. duxbury, l.e., higgins, c.a., neufeld, d., 1998. telework and the balance between work and family: is telework part of the problem or part of the solution? in the virtual workplace, m. igbaria and m. tan eds., hershey, pa: idea group publishing, 218–255. feldman, d., gainey, t., 1997. patterns of telecommuting and their consequences: framing the research agenda. human resource management review 7 (4), 369-88. 18 australasian journal of information systems campbell & heales 2016, vol 20, research article individual consequences for teleworking professionals gajendran, r. s., harrison, d. a., 2007. the good, the bad and the unknown about telecommuting: meta-analysis of psychological mediators and individual consequences. journal of applied psychology 92, 1524-1541. golden, t., 2007. co-workers who telework and the impact on those in the office: understanding the implications of virtual work for co-worker satisfaction and turnover intentions. human relations 60 (11), 1641-1667. golden, t., veiga, j.j., dino, r.n., 2008. the impact of professional isolation on teleworker job performance and turnover intentions: does time spent teleworking, interacting face-to-face, or having access to communication-enhancing technology matter? journal of applied psychology 93 (6), 1412-1421. gray, p., 1997. a demand-side approach to telecommuting: the integrated workplace strategies concept. information systems management 14 (4), 21-8. guimaraes, t., dallow, p., 1999. empirically testing the benefits, problems and success factors for telecommuting programs. european journal of information systems 8 (1), 40-54. haddon l., 1998. the experience of teleworking: a view from the home. in teleworking: international perspectives, p.j. jackson and j.m. van der wielen eds., london: routledge press, 136–143. harrington, s.j., ruppel, c.p., 1999. telecommuting: a test of trust, competing values, and relative advantage. ieee transactions on professional communication 42 (4), 223. hill, e.j., miller, b.c., weiner, s.p., colihan, j., 1998. influences of the virtual office on aspects of work and work/life balance. personnel psychology 51 (3), 667-83. horn, j.l., 1965. a rationale and test for the number of factors in factor analysis. psychometrika 32, 179-185. hu, l.-t., bentler, p.m. 1999. "cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives.," structural equation modeling 6 (1), pp. 1-55. hunton, j.e., harmon, w.k., 2004. a model for investigating telework in accounting. international journal of accounting information systems 5 (4), 417-427. hunton, j.e., 2005. behavioural self-regulation of telework locations: interrupting interruptions! journal of information systems 19 (2), 111-140. johnson, s.a., 1998. teleworking service management: issues for an integrated framework. in teleworking: international perspectives. from telecommuting to the virtual organization, jackson pj, van der wielen j eds. london: routledge press, 185–206. kanellopoulos, d.n., 2011. how can teleworking be pro-poor? journal of enterprise information management 24 (1), 8-29. kaplan, d. 1991. on the modification and predictive validity of covariance structure models. quality & quantity 25, 307-314. kompast m., wagner, i., 1998. telework: managing spatial, temporal and cultural boundaries. in teleworking: international perspectives (pp. 95–117), jackson pj, van der wielen jm eds., london: routledge press. matos, k., galinsky, e., 2014. 2014 national study of employees, families and work institute, http://familiesandwork.org/downloads/2014nationalstudyofemployers.pdf, [accessed 15 april 2016]. metzger, r.o., von glinow, m.a., 1988. off-site workers: at home and abroad. california management review 30 (3), 101-111. 19 australasian journal of information systems campbell & heales 2016, vol 20, research article individual consequences for teleworking professionals mokhtarian, p.l., 1997. the transportation impacts of telecommuting: recent empirical findings. in understanding travel behaviour in an era of change, eds. p. r. stopher and m. lee-gosselin. oxford, great britain: pergamon (elsevier science ltd.). mokhtarian, p.l., bagley, m.n., salomon, i, 1998. the impact of gender, occupation, and presence of children on telecommuting motivations and constraints. journal of the american society for information science 49 (12), 1115-34. maruyama, t., hopkinson, p.g., james, p.w., 2009. a multivariate analysis of work–life balance outcomes from a large-scale telework programme. new technology, work and employment 24 (1), 76-88. maruyama, t., tietze, s., 2012. from anxiety to assurance: concerns and outcomes of telework. personnel review 41 (4), 450-469. morganson, v.j., major, d.a., oborn, k.l., verive, j.m., heelan, m.p., 2010. comparing telework locations and traditional work arrangements: differences in work-life balance support, job satisfaction, and inclusion. journal of managerial psychology 25 (6), 578-595. nilles, j., 1996. what does telework really do to us? world transport policy and practice 2 (1), 15-23. nilles, j., carlson, jr., f.r., gray, p., hanneman, g.j., the telecommunicationstransportation tradeoff: options for tomorrow. new york, ny: john wiley and sons, 1976. orlikowski, w., 2000. using technology and constituting structures: a practice lens for studying technology in organizations. organization science 11 (4), 404-428. pinsonneault, a., boisvert, m., 2001. the impacts of telecommuting on organizations and individuals: a review of the literature, in johnson, n.j. ed., telecommuting and virtual offices: issues and opportunities, hershey, pa: idea group publishing, 163-85. rafaeli, s., ravid, g., 2003. information sharing as enabler for the virtual team: an experimental approach to assessing the role of electronic mail in disintermediation. information systems journal 13 (2), 191–206. riswadkar, a., riswadkar, a.v., 2009. balancing the risks of remote working: walking the telecommuting line. the john liner review 23 (2), 1-5. reinsch, n.l., 1997. relationship between telecommuting workers and their managers: an exploratory study. journal of business communication 34 (4), 343-69. salomon, i., 1985. telecommunications and travel: substitution or modified mobility? journal of transport economics and policy 19, 219–235. sass, d.a., schmitt, t.a., 2010. a comparative investigation of rotation criteria within exploratory factor analysis. multivariate behavioral research 45 (1), 73-103. schmitt, t.a., 2011. current methodological considerations in exploratory and confirmatory factor analysis. journal of psychoeducational assessment 29 (4), 304-321. seaman, m.j., 1997. telecommuting: a transportation planner’s view. information systems management 14 (4), 73-5. shamir, b., salomon, h., 1985. work-at-home and the quality of working life. the academy of management review 10 (3), 455-65. siha, s.m., monroe, r.w., 2006. telecommuting’s past and future: a literature review and research agenda. business process management journal 12 (4), 455-482. tabachnick, b.g., fidell, l.s. 2007. using multivariate statistics, 5th ed., upper saddle river, nj: pearson allyn & bacon. 20 australasian journal of information systems campbell & heales 2016, vol 20, research article individual consequences for teleworking professionals watad, m.m., disanzo, f.j., 2000. case study: the synergism of telecommuting and office automation. sloan management review 41 (2), 85-96. watad, m.m., will, p.c., 2003. telecommuting and organizational change: a middlemanagers’ perspective. business process management journal 9 (4), 459-72. watson-manheim, m.b., chudoba, k.m., crowston, k., 2012. perceived discontinuities and constructed continuities in virtual work. information systems journal 22 (1), 29-52. wright, p., oldford, a., 1993. telecommuting and employee effectiveness: career and manager. the international journal of career management 5 (1), 4-10. 21 australasian journal of information systems campbell & heales 2016, vol 20, research article individual consequences for teleworking professionals appendix survey items used for efa analysis participants were asked to indicate the impact of telework on each item using a five-point scale ranging from “a lot more” (1) to “a lot less” (5). (1 = a lot more; 2 = a bit more; 3 = no difference; 4 = a bit less; 5 = a lot less) 1. your travel between home and the office 2. your work related travel 3. costs relating to your telework activities (home office, broadband connection, etc.) 4. the level of managerial control by others over your work 5. flexibility in scheduling work 6. the complexity of your work tasks 7. your capacity to work 8. the regard that others have for your work 9. your professionalism 10. your ability to collaborate with others 11. your capability to get things done in your job 12. the overall quality of your work 13. the time it takes to get things done 14. the number of interruptions you experience when working 15. your relationship with your work colleagues 16. collegiality in the workplace 17. your feelings of personal achievement 18. the number of different work tasks you can undertake 19. your commitment to the organization 20. the ability to be contacted by work colleagues and/or clients 21. work related stress 22. your workload 23. status within your organization 24. time for family and friends copyright: © 2016 campbell & heales. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. 22 http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems campbell & heales 2016, vol 20, research article individual consequences for teleworking professionals 23 1 introduction 2 theories of telework adoption 3 research approach 3.1 individual outcomes for teleworking professionals 3.1.1 personal costs and savings associated with telework 3.1.2 work autonomy 3.1.3 workload and work/home-life balance 3.1.4 working with others 3.1.5 professional and personal image 3.2 survey development 4 results 4.1 scale construction 5 discussion 6 summary and conclusion 7 limitations paper title (paper title style) australasian journal of information systems nathawitharana, alahakoon & matharage 2015, vol. 19, pp. s55-s75 improving decision value of hierarchical text improving the decision value of hierarchical text clustering using term overlap detection nilupulee nathawitharana la trobe university 18585515@students.latrobe.edu.au damminda alahakoon la trobe university d.alahakoon@latrobe.edu.au sumith matharage sumith.matharage@gmail.com abstract humans are used to expressing themselves with written language and language provides a medium with which we can describe our experiences in detail incorporating individuality. even though documents provide a rich source of information, it becomes very difficult to identify, extract, summarize and search when vast amounts of documents are collected especially over time. document clustering is a technique that has been widely used to group documents based on similarity of content represented by the words used. once key groups are identified further drill down into sub-groupings is facilitated by the use of hierarchical clustering. clustering and hierarchical clustering are very useful when applied to numerical and categorical data and cluster accuracy and purity measures exist to evaluate the outcomes of a clustering exercise. although the same measures have been applied to text clustering, text clusters are based on words or terms which can be repeated across documents associated with different topics. therefore text data cannot be considered as a direct ‘coding’ of a particular experience or situation in contrast to numerical and categorical data and term overlap is a very common characteristic in text clustering. in this paper we propose a new technique and methodology for term overlap capture from text documents, highlighting the different situations such overlap could signify and discuss why such understanding is important for obtaining value from text clustering. experiments were conducted using a widely used text document collection where the proposed methodology allowed exploring the term diversity for a given document collection and obtain clusters with minimum term overlap. keywords: term overlap, growing self organizing map, hierarchical clustering, text document clustering 1 introduction with the introduction of world wide web, an enormous amount of text data has become available. also, the usage of electronic media including electronic format of text data has been increased in every aspect of human life due to easy storage and accessibility. text mining has recently emerged as a popular research field to find hidden patterns in electronic text data. this research field has received attention of researchers due to availability of massive volumes of text and sophisticated text mining techniques are introduced to analyse text data in different classification (grouping of text data into predefined classes) and text clustering (grouping of text data into similar groups without known class labels) are two main areas which fall under the umbrella of text mining. however, text clustering techniques are preferred over text classification techniques in most of the cases, due to lack of categorical information in most text data sets (gunasinghe et al. 2012). document clustering is a widely used, useful technique to explore hidden patterns in a collection of text documents. during the document clustering process, documents are grouped into a number of clusters, in which documents belonging to the same cluster demonstrate a high degree of homogeneity and documents belonging to two different clusters demonstrate a s55 australasian journal of information systems nathawitharana, alahakoon & matharage 2015, vol. 19, pp. s55-s75 improving decision value of hierarchical text high degree of heterogeneity. since documents need to be converted into a numerical format prior to clustering, they are represented as a vector of weighted terms, where terms can comprise of individual words or word sequences and the clustering process is carried out based on the detailed term similarities of the document vectors (kohonen et al. 2000). humans group documents into different topics or categories based on the content relevancy to those topics. due to semantic or syntactic necessity, words or word sequences can be repeated in documents grouped under different topics so it is not possible to expect a one to one mapping between the terms of the documents and the topics. for example, the word ‘scheme’ can be used in explaining housing schemes, pension schemes, recycling schemes, and colour schemes which are associated with different themes according to human categorization. because of above reason traditional cluster evaluation measures become less important when clustering text documents and we cannot expect the documents which belong to the same human categorized group to be clustered together. in general, documents which are associated with similar topics are expected to have significant term overlap and a low term overlap is expected in the documents which belong to very different topics. despite this general belief there might be situations where documents describing vastly different topics share a considerable percentage of terms, as well as documents describing similar topics share a lesser number of terms. since term frequencies are used for text document clustering, if same terms with similar frequencies exist within two documents there is very high possibility they will be clustered together regardless of the human categorized group they actually belong to. this highlights the importance of analysing term overlap since the term overlap between the documents have considerable impact in results we obtain by text documents clustering. in addition overlap calculation is highly beneficial to identify the term diversity of a given document set. different types of clustering algorithms exists which include hierarchical clustering, partitional clustering, fuzzy clustering, and artificial neural networks (jain et al. 1999). several most popular clustering algorithms are k-means, self-organizing map and som based clustering algorithms. out of the many clustering techniques, self-organizing map (som) (kohonen 1998) based techniques are widely used by the data analysts (kaski et al. 1998; kohonen et al. 2000). the growing self organizing map (gsom) (alahakoon et al. 2000) is a prominent member of the som family and it has shown great potential in different types of clustering tasks. the gsom algorithm has been used across diverse disciplines due to its capabilities and several applications of the gsom can be found in hsu et al. (2003), amarasiri et al. (2005), matharage et al. (2009), ahmad et al. (2010), matharage et al. (2011), and gunasinghe et al. (2012). in this paper a new methodology is proposed utilizing the unique feature of the gsom algorithm to analyse a document set with the possibility of obtaining clusters with low term overlap using hierarchical clustering. the rest of the paper is organized as follows. section 2 provides a detailed explanation on hierarchical clustering as a term overlap detection technique. section 3 describes the proposed methodology. section 4 provides the experimental results and the section 5 conclude the paper. 2 hierarchical clustering as a term overlap detection technique 2.1 hierarchical clustering of text hierarchies are a commonly used data structure in data mining and knowledge discovery (schkolnick 1977). hierarchical structure facilitates visualizations of data in different granularity levels which let the data analyst to obtain more detailed relationships by initially exploring the abstract relationships. some of the previous research work have revealed that documents in a collection naturally lead themselves to a hierarchical structure which makes it a suitable data structure to analyse textual data (merkl 1998). s56 australasian journal of information systems nathawitharana, alahakoon & matharage 2015, vol. 19, pp. s55-s75 improving decision value of hierarchical text 2.2 the som and the gsom algorithms the self-organizing map (som) algorithm was introduced by kohonen (1982) and is capable in mapping high dimensional data distribution to a lower dimensional output, mostly to a two dimensional output space. since the som algorithm is capable in visualizing patterns in input data it is widely used as a clustering technique. some of the application of the som algorithm for text clustering can be found in kaski et al. (1998), merkl and rauber (1999), and kohonen et al. (2000). however there are certain limitations associated with the som algorithm and the main limitation is size of the output network need to be predefined. when clustering a dataset, output map size cannot be defined correctly prior to the clustering as there is no knowledge about the inputting data. in that case there might be a need of running the som algorithm for different map sizes and select an appropriate map size. some of the other limitations include lack of hierarchical clustering capabilities and possibility of learning only from stationary data. the growing self-organizing map (gsom) algorithm (alahakoon et al. 2000) which has the ability to grow dynamically has overcome the predefined map size limitation in the som. similar to most of the som based algorithms, the gsom also consists with two activation modes namely training mode and testing mode. actual network growth and smoothing out of weights occur in training mode, while the final calibration of the network for known inputs take place in the testing mode. several key characteristics and advantages of the gsom algorithm include structurally unconstrained learning, exploratory dynamics, efficient computing, and visualization (matharage 2012) which makes it a highly suitable algorithm for text clustering. matharage (2012) has investigated the usability of the gsom algorithm as a text clustering algorithm and concluded that when compared to the som algorithm, the gsom algorithm is more efficient, more accurate and has better topology preservation capabilities. the details of the gsom algorithm is provided in alahakoon et al. (2000). 2.3 the gsom with spread factor (sf) for hierarchical text clustering 2.3.1 the gsom with spread factor (sf) in the gsom, hierarchical clustering is facilitated by the parameter spread factor (sf) which controls the growth of the network. when a low sf value is used, the gsom output becomes a more abstract map whereas more detailed map is given when a high value for sf is used. this characteristic of the gsom algorithm is very advantageous in data mining since it is possible to obtain an abstract map initially and then further explore the map using a higher sf. some of the examples of using hierarchical clustering facilitated by the gsom includes alahakoon et al. (2000) for the zoo data set (bache and lichman 2013) and in haiying et al. (2004) for the sleep apnea data set (goldberger et al. 2000). 2.3.2 automatic cluster identification in the gsom in general clusters in a gsom output can be identified by human inspection. when cluster identification is carried out by humans, clusters are identified as a group of hit nodes which are separated by dummy nodes or non-hit nodes. when the gsom output is obtained using a high sf value cluster identification via visual inspection becomes easy as the output is a more detailed map with increased branching out. if a low sf value is used, the output will be an abstract map where two groups of hit nodes might not be separated by non-hit nodes. in such situation it might not be possible to identify clusters accurately by human inspection as cluster boundaries are unclear. ahmad et al. (2010) emphasises the requirement of having an automatic cluster identification mechanism when cluster boundaries are not clear. the k-means and davies-bouldin (db) index based method (davies and bouldin 1979) and data skeleton model (dsm) based method (alahakoon et al. 2001) has been used for the automatic cluster identification in the gsom outputs (ahmad et al. 2010; amarasiri et al. 2003). when using the k-means and db index based method, value for k needs to be predetermined and value range 2 to is suggested by vesanto and alhoniemi (2000) where n is the total number of hit nodes in the map. the k value which minimizes the db-index value s57 australasian journal of information systems nathawitharana, alahakoon & matharage 2015, vol. 19, pp. s55-s75 improving decision value of hierarchical text is selected as the final number of clusters. in data skeleton modelling the path of the spread (pos) during node growth is manipulated to identify and separate clusters from the gsom output. 3 methodology figure 1 illustrates the proposed methodology for using gsom based hierarchical clustering to capture term overlap in a document corpus. each stage of the methodology is explained in following subsections. figure 1: proposed methodology for using gsom based hierarchical clustering to capture term overlap in a document corpus 3.1 pre-process text documents input documents are pre-processed to reduce the insignificant terms. stemming and stop word removal are executed and lower and upper thresholds are applied where the lower threshold helps to remove terms which occur very infrequently and the upper threshold helps to remove terms which occur very frequently. after pre-processing text documents need to be represented as term vectors prior to the clustering process (kohonen et al. 2000). vector space model (vsm) (salton et al. 1975) is a popular document representation technique where documents are represented as term vectors, in which the dimensionality is equivalent to the total number of terms extracted from the complete document set. the importance of each term is represented by a weight value and three popular term weighting techniques associated with the vsm are binary representation, term frequency (tf), and term frequency – inverse document frequency (tf-idf). binary representation indicates existence of a term in a document where weight 1 is given if term exists s58 australasian journal of information systems nathawitharana, alahakoon & matharage 2015, vol. 19, pp. s55-s75 improving decision value of hierarchical text and 0 is given when term does not exist. the equations for tf and tf-idf term weighting techniques are presented by (1) and (2) respectively. , = , (1) tft,d – term frequency of term t in document d, nt,d – number of occurrences of term t in document d, nd – total number of terms in document d , = , × log | | (2) tf – idft,d – term frequency – inverse document frequency of term t in document d, tft,d – term frequency of term t in document d, |d| – total number of documents, dft – number of documents which contain term t. in the proposed methodology term vectors are prepared (coded) using the tf-idf weighting technique. 3.2 execution of the gsom and automatic cluster identification term vectors are directed to the gsom algorithm and sf value is chosen based on the level of the hierarchy. lower sf value is selected for the first level of the hierarchy with the aim of obtaining an abstract map. clusters in the output map are obtained using the k-means and db index based automatic cluster identification mechanism. 3.3 overlap factor based hierarchical implementation clusters obtained are directed to the term overlap calculation phase which uses two novel parameters introduced in the methodology, namely overlap factor (of) and overlap threshold (ot). of is introduced based on the jaccard index (jaccard 1908). jaccard index which is also known as jaccard similarity coefficient measures the similarity between finite sample sets (jaccard 1908). the calculation of jaccard index is presented by (3). j(a, b) = ( )( ) (3) where, a and b are finite sample sets of measures the term overlap between the cluster pairs which is an effective indicator of cluster purity. when calculating the of only the most frequent terms which are considered as representative terms for clusters are taken into consideration. one major concern regarding the of calculation is how many terms should be selected from each cluster. selecting a limited number of highly frequent terms will not provide satisfactory results as the overlap in terms tend to be low. therefore it is important for the analyst to decide how many terms are needed for the term overlap calculation and the best approach for such selection. if the sizes of resultant clusters obtained via proposed methodology varies vastly then it is more appropriate to select fixed number of terms rather than a percentage of top most terms for each cluster. the reason is that if a particular cluster size is small and only few documents are mapped to the cluster, the total term count will be comparatively low than the other clusters. even though total term count is low, most of the terms exist within the cluster might be very frequent within the cluster and if we select only the top most percentage only few terms out of frequent terms will be selected as representative terms for relevant cluster which will have negative impact for of calculation. the equation for of calculation for cluster a and b is provided in (4). of(a, b) = ( )( ) (4) where, ta and tb ot is the threshold value which helps to decide whether further analysis is required for the s59 australasian journal of information systems nathawitharana, alahakoon & matharage 2015, vol. 19, pp. s55-s75 improving decision value of hierarchical text phase of value is calculated for each cluster pair and the cluster pairs which have of greater than the ot are selected and grouped for further analysis. each cluster in selected cluster pair group is then compared with other selected cluster pair groups and if same cluster is found in two cluster pair groups then they are grouped together. afterwards clusters within groups are merged and selected for further spreading. for each analysis, value for ot is predefined and this value can be consistent with hierarchical analysis or can be adjusted to different values in different hierarchy levels. when higher value is selected for ot only few clusters are selected for further analysis whereas large number of clusters are selected if low ot is selected. by varying the value of the ot, the data analyst can control the number of clusters which are chosen for further analysis. according to the methodology if cluster groups are selected for further analysis, similar steps to the first iteration will be followed. during the execution of the gsom, appropriate sf value can be selected based on the level of the hierarchy. this process executes iteratively until there is no further analysis required in the obtained clusters as of calculated for cluster pairs is less than the predefined ot. this approach allows hierarchical clustering of a text document collection based on term overlap and obtain set of document clusters with minimum term overlap. algorithmic form of the process is presented by algorithm 1. input : i input text document collection initialise sf to a lower value initialise overlap threshold ot cluster(i) function repeat select d = termset(i) based on thresholds i input text document collection generate weight matrix w using tf-idf as the term weighting technique feed w to gsom g with relevant sf value let’s take c = clusters(g) for iteration i 1 to n do n number of clusters j = documentset(ci) assign the document set mapped into the cluster ci as the input collection select f = toptermset(j) assign the top most n% or n number of terms with highest frequency into f where toptermset(j) termset(i) end for for each cluster pair x(ci,cj) 1 to p do p number of distinct cluster pairs calculate of = overlapfactor( , ) if (of >ot ) then ot overlap threshold g = x(ci,cj) assign the overlapping cluster pair to group g end if end for for each cluster group in g, gi do for each cluster group in g, gj where (gi gj) do if ci gj then check whether any of the clusters in the group gi is similar to any of the clusters in the group gj set gi = merge(gi, gj) merge the clusters in the groups gi and gj and remove gj from g end if end for end for s60 australasian journal of information systems nathawitharana, alahakoon & matharage 2015, vol. 19, pp. s55-s75 improving decision value of hierarchical text for each cluster in g do i = documentset(gi) assign the document set mapped into the cluster gi as the input collection cluster(i) recursively calls the clustering function. sf value should be increased based on the level of the hierarchy end for until overlap factor value for each obtained cluster pair is lower than the predefined overlap threshold algorithm 1: the gsom and term overlap based hierarchical text clustering algorithm 4 experimental results 4.1 dataset a benchmark dataset known as ‘banksearch dataset’ (sinka and corne 2005) which was used in many document clustering experiments (borgelt and nürnberger 2004; d’hondt et al. 2010) is used for experiments. the banksearch dataset was formed using web pages and contains 11 document categories with each category containing 1000 documents. the dataset includes distant categories which are associated with different themes and close categories which are associated with the same theme according to human categorization. distant and close groups exist within the dataset facilitate performing autonomous clustering experiments for close and distant categories separately. in addition one category which is ‘sport’ is added as parent for ‘soccer’ and ‘motor sport’ categories to facilitate hierarchical clustering. dataset id dataset category associated theme a commercial banks banking and finance b building societies banking and finance c insurance agencies banking and finance d java programming languages e c/c++ programming languages f visual basic programming languages g astronomy science h biology science i soccer sport j motor sport sport k sport sport table 1: data set categories and associated themes in banksearch data set (sinka and corne 2005) 4.2 investigate the effect of overlap factor for hierarchical clustering with the gsom the main objective of the experiment was to identify the impact of term overlap for hierarchical analysis of text documents. the experiments were conducted for two distant categories, two close categories and mixture of distant and close categories which were selected from the above mentioned dataset (sinka and corne 2005). during the experiments how documents are structured hierarchically based on term similarity was investigated. steps in the proposed methodology were followed to obtain the output clusters and document pre-processing was carried out as the first step. first web documents were converted to plain text documents removing the tags used for web page design. afterwards stop words and special characters were removed and porter’s stemming algorithm (porter 1980) was applied to the s61 australasian journal of information systems nathawitharana, alahakoon & matharage 2015, vol. 19, pp. s55-s75 improving decision value of hierarchical text document set. lower threshold of 2% and upper threshold of 50% were selected for thresholding and only the terms which had frequency in between were selected. after extracting the terms, term vectors were prepared using tf-idf. for the first level of the hierarchy we have used sf value of 0.1 and the term vectors were directed to the gsom algorithm. a learning rate of 0.05 with 50 training iterations and 100 smoothing iterations were used as the gsom parameters. the nodes generated by the gsom algorithm were separated into clusters by using the k-means and db index based algorithm. during cluster separation we have executed k-means algorithm for k = 2 to k = where n was the total number of hit nodes. in addition we observed that for text clustering more accurate k-means results were obtained when most hit nodes were used as cluster centroids instead of random initialization. because of that when running the k-means and db index based automatic cluster separation algorithm weight vectors of the most hit nodes were used as the cluster centroids. following sections provide details of the experiment results for selected document categories. under each experiment gsom output maps are presented where nodes with identical colour represent the same cluster and grey nodes represent the non-hit nodes. clusters in the output maps were labelled based on the most frequent terms and few of most frequent terms are presented for each cluster. top 35% of most frequent terms were used from each cluster for overlap calculation in each hierarchy level and calculated overlap values are rounded to 2 decimal places and presented in tables. in figures and tables term ‘c’ is used to represent clusters, ‘sc’ is used to represent sub clusters, and ‘ssc’ is used to represent further sub clusters obtained for sub clusters. 4.2.1 experiment results obtained for two distant categories for the experiment categories ‘commercial banks’ (a) and ‘soccer’ (i) were selected which are associated with two different themes, ‘banking and finance’ and ‘sport’ respectively. the total document count used for the experiment was 2000. the main objective in this experiment was to identify how term overlap plays a role in document clustering when the input documents belong to vastly different themes in human categorization perspectives. figure 2 illustrates the gsom output obtained for the first level of the hierarchy. seven clusters were identified in the gsom. from the representative terms of each cluster, only one cluster mainly represented the ‘commercial banks’ category and other six clusters mainly represented the ‘soccer’ category. the overlap for each cluster pair was calculated using (4) and the resulting overlap is shown in table 2. the predefined ot was 0.4 and according to the overlap matrix cluster groups c2 and c3, c2 and c4, and c2 and c7 had of greater than the predefined ot. as mentioned by the proposed algorithm if the selected cluster groups have common clusters they should be merged together for further analysis. since all above cluster groups included c2 cluster, we have merged them all for further analysis. in addition it was noted that all four clusters selected for further analysis were mainly representing the ‘soccer’ category. all the documents mapped to the selected four clusters were pre-processed following a similar approach used in the first hierarchy level and clustered using the gsom algorithm with sf 0.3. cluster separated gsom output for the second level of the hierarchy is presented in figure 3. s62 australasian journal of information systems nathawitharana, alahakoon & matharage 2015, vol. 19, pp. s55-s75 improving decision value of hierarchical text figure 2: gsom hierarchical clustering with sf 0.1 for two distant categories from the dataset (first level of the hierarchy) table 2: overlap matrix of each cluster pair for the 1st level of the hierarchy for distant categories we have observed that the merged four clusters were divided into six sub clusters in the next level of clustering. the merged four clusters mainly represented the ‘soccer’ category and the resultant sub clusters obtained in the second level of the hierarchy were also mainly representing the ‘soccer’ category. to investigate the impact of hierarchical clustering for term overlap we have calculated the of for cluster pairs and relevant overlap matrix is provided in table 3. id c1 c2 c3 c4 c5 c6 c7 c1 0.30 0.23 0.28 0.15 0.25 0.25 c2 0.30 0.41 0.41 0.25 0.39 0.42 c3 0.23 0.41 0.36 0.24 0.33 0.38 c4 0.28 0.41 0.36 0.29 0.38 0.39 c5 0.15 0.25 0.24 0.29 0.28 0.26 c6 0.25 0.39 0.33 0.38 0.28 0.31 c7 0.25 0.42 0.38 0.39 0.26 0.31 s63 australasian journal of information systems nathawitharana, alahakoon & matharage 2015, vol. 19, pp. s55-s75 improving decision value of hierarchical text figure 3: gsom hierarchical clustering with sf 0.3 for merged c2, c3, c4, and c7 clusters obtained for distant categories in the first level of the hierarchy id sc1 sc2 sc3 sc4 sc5 sc6 sc1 0.26 0.36 0.23 0.39 0.16 sc2 0.26 0.36 0.33 0.27 0.27 sc3 0.36 0.36 0.39 0.37 0.18 sc4 0.23 0.33 0.39 0.24 0.18 sc5 0.39 0.27 0.37 0.24 0.24 sc6 0.16 0.27 0.18 0.18 0.24 table 3: overlap matrix of each cluster pair for the 2nd level of the hierarchy for merged c2, c3, c4, and c7 clusters obtained for distant categories according to the overlap matrix overlap was minimized between the cluster pairs in this second hierarchy level. an interesting observation was that number of highest important terms repeated across the clusters was minimized in the second hierarchy level. in the first hierarchy level more important terms were repeated across the clusters including ‘cup’, ‘football’, ‘club’, and ‘team’. however in the second level of the hierarchy number was reduced to two and only ‘cup’ and ‘team’ were repeated, confirming higher cluster purity for distant document groups. figure 4 presents the gsom hierarchical clustering overview diagram for the selected distant categories including the distribution of the categories for each cluster. figure 4: gsom hierarchical clustering overview diagram for two distant categories from the dataset to further verify the accuracy of the methodology we have executed experiments for same two distant categories changing the two parameter values which are number of terms selected for s64 australasian journal of information systems nathawitharana, alahakoon & matharage 2015, vol. 19, pp. s55-s75 improving decision value of hierarchical text the overlap calculation and ot. rather than 35% of most frequent terms, most frequent 200 terms were used from each cluster for overlap calculation and ot 0.3 was used for the analysis. in the first hierarchy level when clustered with sf 0.1 seven clusters were obtained where one cluster represented ‘commercial banks’ category and other six clusters mainly represented the ‘soccer’ category. two clusters representing ‘soccer’ has been selected for further analysis and when clustered with sf 0.3 for the second level of the hierarchy seven sub clusters were obtained where one sub cluster represented ‘commercial banks’ and all others were ‘soccer’. the overlap was minimised in the second level of the hierarchy where no further analysis was required. detailed results of the experiments were not provided due to space limitations. 4.2.2 experiment results obtained for two close categories close categories used for the experiment were ‘commercial banks’ (a) and ‘insurance agencies’ (c) and the total document count used for the experiment was 2000. the main objective was to identify the possibility of obtaining clusters with low term overlap for documents which are associated with similar themes according to human categorization. figure 5 illustrates the gsom output obtained for the first level of the hierarchy after automatic cluster separation. according to the results four clusters mainly represented ‘commercial banks’ and the remaining two clusters representing the ‘insurance agencies.’ the result overlap matrix (calculated as previously) is provided in table 4. our pre-defined overlap threshold was 0.5 and according to the overlap matrix presented in table 4 the cluster groups c1 and c3, c1 and c5, c2 and c6, c3 and c5 had of more than the pre-defined ot. the selected cluster groups which have common clusters are merged together for further analysis. so clusters c1, c3, and c5 and the clusters c2 and c6 were merged as separate groups. figure 5: gsom hierarchical clustering with sf 0.1 for two close categories from the dataset (first level of the hierarchy) s65 australasian journal of information systems nathawitharana, alahakoon & matharage 2015, vol. 19, pp. s55-s75 improving decision value of hierarchical text id c1 c2 c3 c4 c5 c6 c1 0.45 0.51 0.42 0.52 0.43 c2 0.45 0.40 0.34 0.45 0.59 c3 0.51 0.40 0.36 0.52 0.38 c4 0.42 0.34 0.36 0.43 0.32 c5 0.52 0.45 0.52 0.43 0.42 c6 0.43 0.59 0.38 0.32 0.42 table 4: overlap matrix of each cluster pair for the 1st level of the hierarchy for close categories according to figure 5 c1, c3, and c5 clusters grouped together for further analysis mainly represented the ‘commercial banks’ category and the clusters c2 and c6 represented the ‘insurance agencies’ and were further analysed separately. all the documents mapped into a group were processed using a similar approach followed in the first hierarchy level. figure 6 illustrates the resultant map obtained by further analysis of c1, c3, and c5 clusters and figure 7 illustrates the resultant map obtained by further analysis of c2 and c6 clusters with sf 0.3. figure 6: gsom hierarchical clustering with sf 0.3 for merged c1, c3, and c5 clusters obtained for close categories in the first level of the hierarchy s66 australasian journal of information systems nathawitharana, alahakoon & matharage 2015, vol. 19, pp. s55-s75 improving decision value of hierarchical text figure 7: gsom hierarchical clustering with sf 0.3 for merged c2 and c6 clusters obtained for close categories in the first level of the hierarchy in figure 6, merged c1, c3, and c5 cluster group was separated to 9 sub clusters. out of the 9 sub clusters 8 represented the ‘commercial banks’ category and 1 represented the ‘insurance agencies’. the initial clusters which were further spread in this second level of the hierarchy were mainly representing ‘commercial banks’ category. however in this second hierarchy level, we observe a cluster which was well separated from other clusters and represent the ‘insurance agencies’ category which confirm that few ‘insurance agencies’ documents included in the clusters c1, c3, and c5 were separated in this second level of the hierarchy. these results prove the ability of the gsom algorithm to well separate text documents belonging to different categories using the terms when high sf value is used. we have calculated the of for each sub cluster pair and relevant overlap matrix is given in table 5. according to table 5 some clusters still have high term overlap which exceeds the predefined ot. in addition figure 6 illustrate that several most important terms are repeated across sub clusters so clusters are not well separated based on terms. in order to verify whether it is possible to obtain higher cluster purity we can merge sub clusters with of greater than the ot by following the steps in the proposed methodology and further analyse them with higher sf value than 0.3. table 5: overlap matrix of each cluster pair for the 2nd level of the hierarchy for merged c1, c3, and c5 clusters obtained for close categories according to figure 7, c2 and c6 clusters were separated to five sub clusters. the clusters c2 and c6 mainly represented the ‘insurance agencies’ category and the resultant sub clusters were also representing the same category. overlap matrix for further analysis of merged c2 id sc1 sc2 sc3 sc4 sc5 sc6 sc7 sc8 sc9 sc1 0.48 0.47 0.37 0.63 0.44 0.41 0.42 0.41 sc2 0.48 0.46 0.45 0.47 0.54 0.50 0.51 0.46 sc3 0.47 0.46 0.44 0.45 0.42 0.46 0.40 0.43 sc4 0.37 0.45 0.44 0.37 0.41 0.56 0.37 0.39 sc5 0.63 0.47 0.45 0.37 0.44 0.41 0.43 0.41 sc6 0.44 0.54 0.42 0.41 0.44 0.44 0.63 0.54 sc7 0.41 0.50 0.46 0.56 0.41 0.44 0.40 0.43 sc8 0.42 0.51 0.40 0.37 0.43 0.63 0.40 0.56 sc9 0.41 0.46 0.43 0.39 0.41 0.54 0.43 0.56 s67 australasian journal of information systems nathawitharana, alahakoon & matharage 2015, vol. 19, pp. s55-s75 improving decision value of hierarchical text and c6 clusters is given in table 6 and according to table 6 we can say that none of the sub clusters exceeded the ot such that no further analysis was required. figure 8 presents the gsom hierarchical clustering overview diagram for the selected close categories including the distribution of the categories for each cluster. id sc1 sc2 sc3 sc4 sc5 sc1 0.43 0.35 0.36 0.34 sc2 0.43 0.42 0.43 0.42 sc3 0.35 0.42 0.43 0.28 sc4 0.36 0.43 0.43 0.32 sc5 0.34 0.42 0.28 0.32 table 6: overlap matrix of each cluster pair for the 2nd level of the hierarchy for merged c2 and c6 clusters obtained for close categories figure 8: gsom hierarchical clustering overview diagram for two close categories from the dataset 4.2.3 experiment results obtained for mixture of close and distant categories for this experiment we have selected ‘commercial banks’ (a), ‘insurance agencies’ (c), and ‘soccer’ (i). ‘commercial banks’ and ‘insurance agencies’ are associated with the same theme which is ‘banking and finance’. however ‘soccer’ is associated with a different theme which is ‘sport’. during this experiment requirement for running more iterations was expected as close and distant categories tend to separate in the first level of the hierarchy and further drill down might be required for clusters representing close categories to obtain minimal term overlap. the total document count used for the experiment was 3000 and the steps in the proposed methodology were followed. the first hierarchy level generated 8 clusters after automatic cluster separation was executed. figure 9 illustrates the cluster separation in the gsom output. according to the results four clusters mainly represented the ‘soccer’ category, one cluster represented the ‘commercial banks’ category and three clusters represented the ‘insurance agencies’ category. the resulting overlap matrix is provided in table 7. ot value used for the experiment was 0.5 and according to the overlap matrix presented in table 7 the cluster groups c1 and c2, c2 and c5 had of more than the pre-defined overlap threshold value. so the clusters c1, c2, and c5 were merged for further analysis. two clusters selected for further analysis mainly represent ‘insurance agencies’ whereas the other cluster represent ‘commercial banks’. this indicates that it is likely to obtain higher term overlap between the clusters associated with similar themes. documents associated with ‘soccer’ theme are distributed among four clusters which prove that the gsom s68 australasian journal of information systems nathawitharana, alahakoon & matharage 2015, vol. 19, pp. s55-s75 improving decision value of hierarchical text algorithm has well separated the ‘soccer’ related documents in the first hierarchy level. in addition the of calculated between the clusters labelled as ‘soccer’ didn’t exceed the predefined ot which verified that those clusters were well separated based on terms even though they belonged to the same theme. when looking into the cluster separation and overlap matrix we can make the conclusion that gsom algorithm has well separated the distant category from the close categories in the first hierarchy level. figure 9: gsom hierarchical clustering with sf 0.1 for mixture of close and distant categories from the dataset (first level of the hierarchy) id c1 c2 c3 c4 c5 c6 c7 c8 c1 0.58 0.49 0.28 0.47 0.39 0.21 0.12 c2 0.58 0.46 0.33 0.52 0.44 0.22 0.14 c3 0.49 0.46 0.38 0.46 0.36 0.34 0.17 c4 0.28 0.33 0.38 0.44 0.37 0.38 0.25 c5 0.47 0.52 0.46 0.44 0.46 0.28 0.20 c6 0.39 0.44 0.36 0.37 0.46 0.23 0.17 c7 0.21 0.22 0.34 0.38 0.28 0.23 0.18 c8 0.12 0.14 0.17 0.25 0.20 0.17 0.18 table 7: overlap matrix of each cluster pair for the 1st level of the hierarchy for mixture of close and distant categories for the second hierarchy level we have used the sf 0.3 for the gsom algorithm. the merged three clusters were separated into ten sub clusters which is illustrated in figure 10. five sub clusters mainly represented ‘commercial banks’ category, four sub clusters ‘insurance agencies’ and one sub cluster was ‘soccer’. the merged three clusters used for further analysis were mainly representing ‘commercial banks’ and ‘insurance agencies’ categories. however in this second hierarchy level we obtain a sub cluster which represent ‘soccer’ category and it verifies that some of the ‘soccer’ related documents assigned to the three clusters selected for further analysis were separated in the second hierarchy level. we observed that several sub clusters had of greater than the ot and most frequent terms were still repeated among number of sub clusters. this repetition of most frequent terms was significantly visible for sub clusters labelled as ‘commercial banks’. this observation verified that the sub clusters labelled as ‘commercial banks’ consist of similar terms and in the term overlap matrix all of those s69 australasian journal of information systems nathawitharana, alahakoon & matharage 2015, vol. 19, pp. s55-s75 improving decision value of hierarchical text clusters had of greater than the ot which confirmed the observation. the overlap matrix for the second hierarchy level is presented in table 8 and according to table 8 sub cluster groups sc1 and sc3, sc1 and sc6, sc1 and sc7, sc2 and sc6, sc2 and sc7, sc5 and sc7 showed of greater than the pre-defined ot. following the merging rule in our proposed algorithm we have merged the sub clusters sc1, sc2, sc3, sc5, sc6, and sc7 for further analysis. five sub clusters selected for further analysis mainly represented the ‘commercial banks’ category and the other sub cluster mainly represented the ‘insurance agencies’ category. figure 10: gsom hierarchical clustering with sf 0.3 for merged c1, c2, and c5 clusters obtained for mixture of close and distant categories in the first level of the hierarchy id sc1 sc2 sc3 sc4 sc5 sc6 sc7 sc8 sc9 sc10 sc1 0.49 0.52 0.42 0.44 0.51 0.52 0.40 0.34 0.42 sc2 0.49 0.43 0.43 0.49 0.55 0.53 0.46 0.28 0.41 sc3 0.52 0.43 0.42 0.37 0.39 0.47 0.41 0.35 0.46 sc4 0.42 0.43 0.42 0.40 0.43 0.40 0.36 0.30 0.42 sc5 0.44 0.49 0.37 0.40 0.41 0.51 0.49 0.29 0.42 sc6 0.51 0.55 0.39 0.43 0.41 0.46 0.37 0.25 0.38 sc7 0.52 0.53 0.47 0.40 0.51 0.46 0.49 0.30 0.41 sc8 0.40 0.46 0.41 0.36 0.49 0.37 0.49 0.28 0.39 sc9 0.34 0.28 0.35 0.30 0.29 0.25 0.30 0.28 0.32 sc10 0.42 0.41 0.46 0.42 0.42 0.38 0.41 0.39 0.32 table 8: overlap matrix of each cluster pair for the 2nd level of the hierarchy for merged c1, c2, and c5 clusters obtained for mixture of close and distant categories figure 11 illustrates the output map using sf 0.5 and according to figure 11 we observed that merged six clusters were separated to six sub clusters. four sub clusters mainly represented ‘commercial banks’, one sub cluster ‘insurance agencies’ and the other sub cluster ‘soccer’. similar to the results for the second hierarchy level we observed that one sub cluster represent ‘soccer’ category even though the merged clusters directed for further analysis mainly represented ‘commercial banks’ and ‘insurance agencies’ categories. this observation proves that few documents belonging to ‘soccer’ category were included to the clusters used for further analysis and they were separated in the third hierarchy level. s70 australasian journal of information systems nathawitharana, alahakoon & matharage 2015, vol. 19, pp. s55-s75 improving decision value of hierarchical text overlap matrix for third hierarchy level is shown in table 9. within the clusters further analysed in this third hierarchy level there was one cluster mainly representing ‘insurance agencies’. it can be seen that same most important terms presented for relevant cluster in figure 10 were repeated for the sub cluster labelled as ‘insurance agencies’ obtained in the third hierarchy level. however when comparing relevant two clusters labelled as ‘insurance agencies’ in the second and third hierarchy levels, we can say that the cluster purity is higher for the cluster we obtained in the third hierarchy level. since the term overlap calculated in the third hierarchy level didn’t exceed the ot which shows that relevant cluster does not have significant term overlap with any other sub cluster in the third level of the hierarchy. according to figure 11 we notice that there were three sub clusters which still had of greater than the ot. those clusters mainly represent ‘commercial banks’. however, one sub cluster which represented ‘commercial banks’ category was separated well as it did not have significant term overlap with other sub clusters. we can further analyse the three sub clusters which had of greater than the ot to verify whether it is possible to obtain high cluster purity by minimising the term overlap between the resultant clusters. figure 12 presents the gsom hierarchical clustering overview diagram for the selected close and distant categories including the distribution of the categories for each cluster. figure 11: gsom hierarchical clustering with sf 0.5 for merged sc1, sc2, sc3, sc5, sc6, and sc7 sub clusters obtained for mixture of close and distant categories in the second level of the hierarchy id ssc1 ssc2 ssc3 ssc4 ssc5 ssc6 ssc1 0.42 0.37 0.31 0.37 0.35 ssc2 0.42 0.46 0.49 0.56 0.59 ssc3 0.37 0.46 0.37 0.48 0.47 ssc4 0.31 0.49 0.37 0.47 0.49 ssc5 0.37 0.56 0.48 0.47 0.49 ssc6 0.35 0.59 0.47 0.49 0.49 table 9: overlap matrix of each cluster pair for the 3rd level of the hierarchy for merged sc1, sc2, sc3, sc5, sc6, and sc7 sub clusters obtained for mixture of close and distant categories s71 australasian journal of information systems nathawitharana, alahakoon & matharage 2015, vol. 19, pp. s55-s75 improving decision value of hierarchical text figure 12: gsom hierarchical clustering overview diagram for mixture of close and distant categories from the dataset 5 conclusion in this paper we have introduced a novel methodology to analyse text documents hierarchically based on the terms in the documents. according to our experiment results we can conclude that the gsom hierarchical clustering can be used together with term overlap between the clusters to conduct an exploratory analysis of a given document set. the experiment results showed that the proposed methodology will lead to obtaining clusters with minimal term overlap when executed for distant categories. for close categories it might be required to execute the steps several times iteratively to obtain clusters with minimal term overlap. when the document set consists with a mixture of distant and close categories, distant categories tend to be separated within first few levels of the hierarchy showing minimum term overlap whereas close categories might require more iterations to obtain minimum term overlap between the clusters. however it was noticed that the analysis output we obtained by following the novel methodology is highly dependent on the repetitive terms within the given document set regardless of the themes they are actually associated with. one advantage of the novel methodology is that it allows investigating the diversity of the document content for a given document set. in addition it allows making decisions on what documents are more likely representing a similar or close document category based on term overlap. organizations collect large volumes of documents which contain information necessary for decision making at various levels. automating the separation of documents into useful groups is highly beneficial since this will cut down lengthy manual processing which becomes almost impossible due to large volumes and high frequency of document collection. traditionally documents are grouped in to pre-defined categories but a document may belong to several such categories or better fit into a new category (no existing category) based on the content. the methodology proposed in this paper enables exploration of a document corpus by content and hierarchically separating into categories and sub categories based on term overlap. s72 australasian journal of information systems nathawitharana, alahakoon & matharage 2015, vol. 19, pp. s55-s75 improving decision value of hierarchical text references ahmad, n., alahakoon, d., and chau, r. 2010. "cluster identification and separation in the growing self-organizing map: application in protein sequence classification," neural computing & applications (19:4), pp. 531-542. alahakoon, d., halgamuge, s. k., and srinivasan, b. 2000. "dynamic self-organizing maps with controlled growth for knowledge discovery," ieee transactions on neural networks (11:3), pp. 601-614. alahakoon, d., halgamuge, s. k., and srinivasan, b. 2001. "mining a growing feature map by data skeleton modelling," data mining and computational intelligence (68), pp. 217250. amarasiri, r., alahakoon, d., and smith, k. a. 2005. "hdgsom: a modified growing selforganizing map for high dimensional data clustering," his'04: fourth international conference on hybrid intelligent systems, proceedings, pp. 216-221. amarasiri, r., wickramasinghe, l., and alahakoon, l. d. 2003. "enhanced cluster visualization using the data skeleton model," in intelligent systems design and applications. springer, pp. 539-548. bache, k., and lichman, m. 2013. {uci} machine learning repository. university of california, irvine, school of information and computer sciences. borgelt, c., and nürnberger, a. 2004. "experiments in document clustering using cluster specific term weights," proc. workshop machine learning and interaction for textbased information retrieval (tir 2004), pp. 55-68. d’hondt, j., vertommen, j., verhaegen, p.-a., cattrysse, d., and duflou, j. r. 2010. "pairwiseadaptive dissimilarity measure for document clustering," information sciences (180:12), pp. 2341-2358. davies, d. l., and bouldin, d. w. 1979. "cluster separation measure," ieee transactions on pattern analysis and machine intelligence (1:2), pp. 224-227. goldberger, a. l., amaral, l. a., glass, l., hausdorff, j. m., ivanov, p. c., mark, r. g., mietus, j. e., moody, g. b., peng, c. k., and stanley, h. e. 2000. "physiobank, physiotoolkit, and physionet: components of a new research resource for complex physiologic signals," circulation (101:23), pp. e215-220. gunasinghe, u., matharage, s., and alahakoon, d. 2012. "a sequence based dynamic som model for text clustering," ieee international joint conference on neural networks (ijcnn). haiying, w., azuaje, f., and black, n. 2004. "an integrative and interactive framework for improving biomedical pattern discovery and visualization," ieee transactions on information technology in biomedicine, (8:1), pp. 16-27. hsu, a. l., tang, s. l., and halgamuge, s. k. 2003. "an unsupervised hierarchical dynamic self-organizing approach to cancer class discovery and marker gene identification in microarray data," bioinformatics (19:16), pp. 2131-2140. jaccard, p. 1908. "nouvelles recherches sur la distribution florale," bulletin de la sociète vaudense des sciences naturelles (44), pp. 223-270. jain, a. k., murty, m. n., and flynn, p. j. 1999. "data clustering: a review," acm computing surveys (csur) (31:3), pp. 264-323. kaski, s., honkela, t., lagus, k., and kohonen, t. 1998. "websom self-organizing maps of document collections," neurocomputing (21:1-3), pp. 101-117. s73 australasian journal of information systems nathawitharana, alahakoon & matharage 2015, vol. 19, pp. s55-s75 improving decision value of hierarchical text khadjeh nassirtoussi, a., aghabozorgi, s., ying wah, t., and ngo, d. c. l. 2014. "text mining for market prediction: a systematic review," expert systems with applications (41:16), pp. 7653-7670. kohonen, t. 1982. "self-organized formation of topologically correct feature maps," biological cybernetics (43:1), pp. 59-69. kohonen, t. 1998. "the self-organizing map," neurocomputing (21:1-3), pp. 1-6. kohonen, t., kaski, s., lagus, k., salojarvi, j., honkela, j., paatero, v., and saarela, a. 2000. "self organization of a massive document collection," ieee transactions on neural networks (11:3), pp. 574-585. lu, y., zhang, p., liu, j., li, j., and deng, s. 2013. "health-related hot topic detection in online communities using text clustering," plos one (8:2). matharage, s., alahakoon, o., alahakoon, d., kapurubandara, s., nayyar, r., mukherji, m., jagadish, u., yim, s., and alahakoon, i. 2011. "analysing stillbirth data using dynamic self organizing maps," ieee international workshop on database and expert systems applications (dexa), 2011 22nd, pp. 86-90. matharage, s., gunasinghe, u., and alahakoon, d. 2009. "growing self organizing map with an imposed binary search tree for discovering temporal input patterns," 2009 ieee international conference on industrial and information systems, pp. 222-226. matharage, s. s. 2012. an autonomous incremental learning model for efficient mining of text data. in: faculty of information technology. clayton school of information technology. monash university. merkl, d. 1998. "text classification with self-organizing maps: some lessons learned," neurocomputing (21:1–3), pp. 61-77. merkl, d., and rauber, a. 1999. "uncovering associations between documents," proc. international joint conference on artificial intelligence (ijcai99). knime: online reviews vs. social media," 2014 37th international convention on information and communication technology, electronics and microelectronics, mipro 2014 proceedings, pp. 1464-1468. porter, m. f. 1980. "an algorithm for suffix stripping," program-automated library and information systems (14:3), pp. 130-137. salton, g., wong, a., and yang, c. s. 1975. "a vector space model for automatic indexing," commun. acm (18:11), pp. 613-620. schkolnick, m. 1977. "a clustering algorithm for hierarchical structures," acm trans. database syst. (2:1), pp. 27-44. sinka, m. p., and corne, d. w. 2005. "the banksearch web document dataset: investigating unsupervised clustering and category similarity," journal of network and computer applications (28:2), pp. 129-146. vesanto, j., and alhoniemi, e. 2000. "clustering of the self-organizing map," ieee transactions on neural networks, (11:3), pp. 586-600. copyright: © 2015 nathawitharana, alahakoon & matharage. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. s74 australasian journal of information systems nathawitharana, alahakoon & matharage 2015, vol. 19, pp. s55-s75 improving decision value of hierarchical text s75 paper title (paper title style) australasian journal of information systems gallagher, burmeister & sixsmith 2016, vol 20, post publication review engaging students in blended environments post publication review pye, g., holt, d., salzman, s., bellucci, e. & lombardi, l. (2015). engaging diverse student audiences in contemporary blended learning environments in australian higher business education: implications for design and practice. australasian journal of information systems, 19, research article. doi: http://dx.doi.org/10.3127/ajis.v19i0.1251 review the above paper by pye et al 2015, provides an interesting insight into student engagement in a blended learning environment across both the face-to-face and online education realms through presenting findings from a student engagement survey. in the initial parts of the paper pye et al report on the engagement of undergraduate students in an introductory subject in the faculty of business at their university. discussion was made with regard to the student academic success, satisfaction and retention rates in relation to contemporary online blended learning management system (lms) environment at their own university. the survey appears to cover this engagement from an online perspective only, given the six key dimensions listed in the paper, which would be useful for future research in this area. from these dimensions, findings are presented across a number of different themes focusing on the student perceptions and finally several valuable areas for blended learning design and practice are provided. effective engagement of students in their own learning is a goal sought by many academics. active learning is a key element in engaging students. through participation in activities which maximize learning, students not only grasp the learning outcomes, but also understanding the underlying context on which the activity was based (maznevski 1996). through this process, learning then becomes an experience and provides students with the knowledge to perform effectively, as recently demonstrated in a similar course at another university (gallagher & sixsmith 2014a and 2014b). these studies build on a tradition of student engagement in information and communications technology (ict) (al-saggaf & burmeister, 2012; burmeister, 1995; simpson, nevile, & burmeister, 2003), which demonstrate that it is this active learning experience that has a profound influence on learning outcomes. the authors provide background to the study, through discussion on: 1) the blended learning environment at their university; 2) the emergence of moocs (massive open online courses); and 3) the online education model (which appears to be the new ‘distance model of education’). each of the above has emerged in its own way from the extended use of ict in the education sector. a major focus in the background (and the overall study) is the concept of ‘clouddeakin’, which appears to be the lms. an important facet of this current study is the focus on the student and their perceptions of engagement in the online and blended learning environments. background literature is presented in this area to justify this direction of the study. the survey instrument demonstrates a predominant focus toward the online environment. while the key characteristics of blended learning environments are presented in the background information (table 1), this appears to be lacking when the authors describe the survey instrument (sections 3.2 and 3.3). in fact, the survey is framed around six key dimensions each of which use ‘online’ rather than ‘blended’. the survey in our opinion lacks one key dimension, that of the actual subject content. while students can collaborate and interact using ict, it is the subject content that they truly need to engage with. an example of blended learning is that of scenario based interaction between the online content and teacher led instruction to the student (gallagher & sixsmith 2014a and 2014b). one limitation of this study is the discussion about blended learning. the distinction between on campus and off campus students seems at conjecture with a blended learning environment, as this environment is a combination of face to face and elearning. this doesn’t seem to reflect the blended learning ideology, as the authors quite frankly state that they were focusing from a student and not the teaching staff perspective. the discussion is further blurred when in section 2.1 where the authors bring into the review the issue of mature age distance education, 1 australasian journal of information systems gallagher, burmeister & sixsmith 2016, vol 20, post publication review engaging students in blended environments making claims that are not well supported. if they wished to explore this topic in more depth, ajis recently published articles on the undergraduate to post-graduate distinctions for is students (richards & marrone, 2014), and on post-graduate work specifically burmeister (2015). that latter article addresses another issue raised by these authors, that of online learning communities, which it addresses in terms of communities of practice. sandra gallagher school of systems management and leadership university of technology sydney sandra.gallagher@uts.edu.au oliver k burmeister charles sturt university alan j sixsmith university of technology sydney references al-saggaf, y., & burmeister, o. k. (2012). improving skill development: an exploratory study comparing a philosophical and an applied ethical analysis technique. computer science education, 22(3), 1-19. burmeister, o. k. (1995). evaluating the factors that facilitate a deep understanding of data analysis. australasian journal of information systems, 3(1). burmeister, o. k. (2015). improving professional it doctorate completion rates. australasian journal of information systems, 19, 55-70. doi: 10.3127/ajis.v19i0.1073 gallagher, s. & sixsmith, a. (2014a) “engaging it undergraduates in non-it content: adopting an elearning information system in the classroom”, interactive technology and smart education, (11, 2), 2014, pp 99-111 gallagher, s. & sixsmith, a. (2014b) “integrating an elearning information system into the classroom to engage it undergraduate students in non it content”, proceedings of the iadis information systems 2014 conference, 28 feb-2 mar 2014, madrid, spain maznevski, m.l (1996) grading class participation, teaching concerns january, pp 1-3 richards, d., & marrone, m. (2014). identifying the education needs of the business analyst: an australian study. australasian journal of information systems, 18(2), 165-186. simpson, c., nevile, l., & burmeister, o. k. (2003). doing ethics: a universal technique in an accessibility context. australasian journal of information systems, 10(2). author response the authors of the paper under review value the positive contributions made and thank their colleagues for their contribution. copyright: © 2016 gallagher, burmeister & sixsmith. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. 2 http://creativecommons.org/licenses/by-nc/3.0/au/ http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems gallagher, burmeister & sixsmith 2016, vol 20, post publication review engaging students in blended environments 3 paper title (paper title style) australasian journal of information systems madlberger & matook 2017, vol 21, research article theorizing e-commerce business models 1 theorizing e-commerce business models: on the impact of partially and fully supported transaction phases on customer satisfaction and loyalty maria madlberger webster vienna private university austria maria.madlberger@webster.ac.at sabine matook university of queensland abstract although considerable research has been conducted on the definition and classification of ecommerce business models, little research has integrated the support and impact of distinct transaction phases within e-commerce business models. indeed, any channel design decisions depend on the arrangement of the three transaction phases, viz. information phase, agreement phase and fulfilment phase. whereas in literature, a complete support of transaction phases is implicitly assumed, in practice, e-commerce firms exist that consciously do not support all three transaction phases through the online channel. in cases for which only certain phases are supported in one channel, the customer is required to switch to another channel to complete the transaction. to gain an initial understanding of the relevance of transaction phases, we examine the influence of transaction phase support in one channel on satisfaction and reuse intention of an e-commerce website under consideration of users’ prior experiences. we conduct a laboratory experiment involving two e-commerce business models that differ only in the number of supported transaction phases via the online channel. empirical data was analysed using a bayesian network approach. our analysis indicates the types of users who value partial support compared to those users who prefer full support of transaction phases. the results will assist firms in meeting future challenges regarding user engagement and attraction to their online stores. keywords: electronic commerce business model; transaction phases; channel design; channel switching; customer satisfaction; reuse intention; tourism industry; bayesian network 1 introduction after years of constant growth, mobile and electronic commerce (e-commerce) have reached a significant level and account for a substantial share of overall retail sales (emarketer 2015). firms of all sizes use internet technologies for conducting their business transactions and customer interactions online (slade & van akkeren 2002). recent figures make this development evident: as of 2015, e-commerce sales worldwide amounted to 1.67 trillion us$ and sales are predicted to grow to 3.58 trillion us$ by 2019 (statista 2016). indeed, projected annual e-commerce sales are expected to reach growth rates of between 18.7% and 22.7% each year and the worldwide online purchase penetration – that is, the percentage of online buyers among internet users – is now 51.5% (emarketer 2015). mobile commerce via locationindependent devices is also growing, especially among younger demographics whereby, in the 18-24 age group, 65% of survey respondents said they use their smartphones for conducting purchasing transactions (lindner 2016). these figures illustrate the advancement of mobile and e-commerce, which has been recently enhanced through multi-channel customer engagement (piotrowicz & cuthbertson 2014). providing multiple channels offers customers various access points and interaction opportunities that can ultimately increase customer satisfaction and loyalty (wallace et al. 2004). companies enjoy a number of benefits through a multi-channel design that are not present in a single channel design, such as closer customer relationships, more tailored products, better service offerings and potentially more sales australasian journal of information systems madlberger & matook 2017, vol 21, research article theorizing e-commerce business models 2 (steinfield et al. 2002). decisions about how to engage with customers have become even more important for companies given the advent of social media as an additional channel (dalla pozza 2014; piotrowicz & cuthbertson 2014; verhoef et al. 2015) and the increased functionalities of mobile devices (bang et al. 2013; huang et al. 2016). channel design is a core element of a company’s e-commerce business model whereby the focus is on the purchasing transaction. an e-commerce business model reflects the business logic of a company (daidj 2015; morris et al. 2005) and explains how e-commerce is integrated into the company’s wider business strategy (teece 2010). chesbrough & rosenbloom (2002) note that a simplified view of a business model is that it represents the company’s business activities “by which a company can sustain itself, that is, generate revenue” (p. 529). hence, a business model describes “the content, structure and governance of transactions designed so as to create value” (amit & zott 2001, p. 511). business models that consider a multichannel strategy implicitly utilize the concept of segmenting the customer transaction into a number of individual phases. existing research has identified different transaction phases, viz. the information, agreement and fulfilment phases (choudhury & karahanna 2008; shankar et al. 2003; wiesel et al. 2011). once a transaction phase has been completed, customers are free to switch to another channel to continue the transaction (pookulangara et al. 2011). for example, customers can initiate a purchasing transaction on a smartphone but may choose, due to limitations of the mobile channel (e.g., screen or image size), to switch to a desktop computer to finalize the transaction. similarly, customers may be invited to search for and select a product on a company’s website but, in order to complete the transaction, are forced to switch the channel and establish a direct communication channel with the company. despite a large body of research on e-commerce business models on the one hand and transaction phases in e-commerce on the other hand, both research areas have been studied by the literature isolated from each other. in fact, research on e-commerce business models has so far paid little attention to the concept of distinct transaction phases. more specifically, many studies on e-commerce business models do not differentiate between separate transaction phases and instead understand the purchasing transaction as a monolithic, indispensable component (e.g., osterwalder et al. 2005; zott et al. 2011). how companies might offer customers different channels and allow for channel switching is not detailed in ecommerce business model descriptions. in fact, existing research on the topic implicitly suggests that one channel supports all transaction phases, while empirical research shows that most channel design solutions offer multiple channels through which customers can complete their transactions (zott & amit 2007; zott & amit 2010). in contrast to the academic debate, some e-commerce firms strategically position their business models to focus on increasing user engagement (attfield et al. 2011). to achieve this objective, they implement forced channel switching by only supporting parts of a transaction in one channel (e.g., mobile or online) and allowing customers to complete a transaction only after switching to a direct customer-company communication channel. this channel switching is implemented despite the technical abilities of the website operators to support a full transaction phase approach on their websites. examples in practice of such business models that include such forced channel switching can be found in the tourism industry, whereby the booking of a flight or hotel package is only possible through direct interaction with a company service operator. in such cases, the forced switching is from the online or mobile channel to the direct customer-company communication channel via email. a specific example of a business model that offers only partial transaction support with forced channel switching is flightfox (flightfox.com). flightfox.com wishes its customers to complete the transaction with its personal selling channel. therefore, customers first submit their planned itinerary on the website and, subsequently, an operator is assigned who contacts each customer to search for flights and carry out the booking once the cheapest flight offer has been found. another example for a partial transaction support is the australian travel agency flight centre that processes holiday packages only through direct customer-company communication, despite customers’ ability to search through available deals and request quotes on the company’s website (flightcentre.com.au/holidays). further examples of forced australasian journal of information systems madlberger & matook 2017, vol 21, research article theorizing e-commerce business models 3 channel switching include special conditions that make the switch necessary, such as customers’ payment options (e.g., a hawaiian airline requires customers to book an interisland flight via their call centre if they do not have a us credit card), time constraints (e.g., allowing the purchase of tickets for an event by phone only if the event date is in the near future) or availability constraints (e.g., despite availability, vacation packages may no longer be bookable through the online channel closer to the travel date, necessitating customers’ direct contact with the travel agent). the forced channel switching represents an e-commerce business model by which the company seeks to maximize customer service and user engagement through personal interactions with customers (lehmann et al. 2012; van doorn et al. 2010). this strategic decision is made consciously and does not take full use of technological opportunities to provide full support via multiple channels for customers. the enhanced interaction intensity that results from the forced channel switching allows for the offering of additional services to customers and provides opportunities for extra value creation. however, e-commerce business models that employ such forced channel switching have not yet been discussed in the literature. hence, the question remains as to how customers perceive the partial support of a transaction in one channel and the subsequent forced channel switching. in particular more clarification is needed whether an increased service level through personal interaction can offset the inherent delay of completing the transaction which can be perceived as a flaw by customers (everard & mccoy 2010). furthermore, the literature is silent as to which types of customers prefer an e-commerce business model by which one transaction can be completed in one channel (full transaction support) compared to switching the channel after the first or second phase (partial transaction support). on the one hand, customers may appreciate the extra service offerings and closer attachment to the company provided by partial transaction support. on the other hand, customers may prefer to complete a transaction swiftly in one channel, especially the online and mobile channels. consequently, this study seeks to fill the gaps in the literature by examining how e-commerce business models, which support transaction phases via the online channel, influence to different extents customer satisfaction and reuse intention in light of user experience considerations. the limited body of literature on transaction phase considerations in e-commerce business models and the research gap on e-commerce business models that incorporate forced channel switching as a means of enhancing customer engagement motivated our research. consequently, we examine extant research on e-commerce business models and transaction phases in e-commerce. we draw on theoretical work regarding transaction phases in ecommerce settings (choudhury & karahanna 2008; veit et al. 2014; wiesel et al. 2011) and channel expansion theory (carlson & zmud 1999). subsequently we empirically test the research model with an experimental research design in the domain of online travel websites. motivated to focus on causality (lauría & duchessi 2007; zheng & pavlou 2010), the data are analysed using a bayesian network approach. our findings contribute to the information systems (is) and marketing literature by providing an improved understanding of user satisfaction and the reuse intention of an online store’s customers, considering varying degrees of transaction phase support. the practical implications of this research are relevant for ecommerce companies and customers alike, as they facilitate improving firms’ e-commerce presence so as to achieve higher levels of customer satisfaction. 2 background 2.1 the concept of transaction phases transactions aim to initiate, arrange and complete contractual agreements between trading partners (schmid & lindemann 1998). a transaction can be split into a number of transaction phases. indeed, when analysing purchasing processes, neither the marketing literature (bunn 1993; kotler & keller 2015) nor the is literature (choudhury & karahanna 2008; pavlou & fygenson 2006) considers a transaction to be a monolithic block, but rather a series of transaction phases. this "purchase funnel" captures the customer’s decision-making process australasian journal of information systems madlberger & matook 2017, vol 21, research article theorizing e-commerce business models 4 towards the purchasing act, whereby the number of phases depends on the level of granularity (wiesel et al. 2011). conceptually, a transaction is composed of three stages, viz. the pre-purchase, purchase and post-purchase phases (wiesel et al. 2011). these stages have been captured in the literature as (1) information phase, (2) agreement phase and (3) fulfilment phase (bauer et al. 2006; chen & chang 2002; holzmüller & schlüchter 2002; kalakota & whinston 1997; yu et al. 2002). it should be noted that, in the literature, multiple variations to this three-phase model exist, for example by adding a fourth, after-sales phase (bauer et al. 2006; di noia et al. 2004; schubert 2002) or by splitting the agreement phase into two phases: the intention and contract phases (anagnostopoulos et al. 2004). nevertheless, in this research we focus on the three-phase model because it is the most generic and independent of particular industries or supply chain members. in the information phase, where all pre-purchase activities take place, information acquisition is central to aiding in the customer’s decision-making and preparation of the purchasing act (choudhury & karahanna 2008; pavlou & fygenson 2006). this initial stage is largely cognitive and represents the mental processes of forming an intention to purchase a product based on the gathered information (wiesel et al. 2011). in the agreement phase, the actual purchase is made (pavlou & chai 2002; steinfield et al. 2002) and, thus, this phase is a conative activity (wiesel et al. 2011). the fulfilment phase includes the post-purchase activities and covers after-sales services (di noia et al. 2004; schubert 2002; steinfield et al. 2002) and after-purchase evaluations (choudhury & karahanna 2008). the three transaction phases are presented in detail as follows and summarized in figure 1 (choudhury & karahanna 2008; kim & lee 2002; veit et al. 2014; wiesel et al. 2011; zumpe & madlberger 2007b): 1) in the information phase, also referred to as information search, customers look for information about offered products and services, as well as companies that offer them. in e-commerce, this phase begins when prospective customers enter a mobile or ecommerce system (kim & lee 2002). information search includes product and price comparisons as well as checking the availability of products or services. once an offer is submitted the information phase is concluded and the transaction proceeds to the next phase. 2) in the agreement phase, the customer interacts with the seller by agreeing upon purchasing conditions. key parameters are product prices, product characteristics, terms of delivery and terms of payment. once both the customer and the seller accept the agreed upon conditions, a legally binding contract is established. this activity constitutes the end of the agreement phase, requiring both parties to fulfil the conditions of the contract (delivery and payment, respectively). this phase is carried out entirely by means of electronic communication. 3) the fulfilment phase comprises the fulfilment of the contractual obligations set up in the preceding agreement phase. it consists of activities such as transportation of goods or download enablement, storage, packaging and payment. in cases involving services, fulfilment includes reservations and booking, or the provision of a ticket that acts as a document entitling the customer to consume the service. the purchasing transaction is completed once the fulfilment phase is finished, although after-purchase phases are still possible (e.g., after-sales service). australasian journal of information systems madlberger & matook 2017, vol 21, research article theorizing e-commerce business models 5 information phase  request of information  providing an offer agreement phase  contract negotiation  agreement on contract fulfilment phase  payment is made  delivery is processed contract offer demand for service or product service or product demand is satisfied figure 1. the e-commerce transaction phases model (zumpe & madlberger 2007b; zumpe & madlberger 2007a) 2.2 e-commerce business models comprehensive research has been conducted on the structure, functioning and impacts of ecommerce business models over the past decade (choudhury & karahanna 2008; hedman & kalling 2003; veit et al. 2014). the findings are characterized by a variety of methods, classifications, frameworks and components. for example, wang et al. (2016) provide an empirical analysis of success factors of group-buying websites to illustrate a business model that differs from conventional business-to-consumer online stores. indeed, e-commerce business models can differ substantially in their effect on sales generation because important factors such as price reductions or dynamics in sales processes can look very differently (wang et al. 2016). considering this rich body of work, it comes as a surprise that e-commerce business models are described as the most discussed yet least understood concepts in the ecommerce field (veit et al. 2014). consequently, we see a need to provide a detailed and comprehensive definition of e-commerce business models as we understand them in this research. following osterwalder et al. (2005), we define an e-commerce business model as “a conceptual tool that contains a set of elements and their relationships and allows expressing the business logic of a specific firm […] being a description of the value a company offers to one or several segments of customers and of the architecture of the firm and its network of partners for creating, marketing, and delivering this value and relationship capital, to generate profitable and sustainable revenue streams” (p. 3). the value provided by an e-commerce business model is decisive for customers’ choice of the e-commerce channel (lim 2014) and, as a further consequence, also for the reuse intention (adapa & cooksey 2013; wang et al. 2016). nowadays e-commerce business models are often based on multiple channels, also referred to as bricks and clicks business models (herhausen et al. 2015). several studies consider channels a part of a business model’s value network that describes how companies conduct transactions through coordination and collaboration with other parties (al-debei & avison 2010). the design of a company’s distribution channels is a strategic company decision because it impacts on the organization’s business model and any changes to the channel design alter customer target groups and propositions of shopping value, entailing a redesign of the value chain. as framing theory suggests, location and presentation of content can determine the specific focus of a website (parker et al. 2014), hence particular features of e-commerce business models can be emphasised this way. e-commerce business models are, however, influenced by decisions not only regarding the provision of different channels but also regarding whether they are run separately or are integrated (cao 2014). integrated channels allow customers to switch between them, whereas separate channels do not provide this functionality. a highly relevant phenomenon in multi-channel business models is customers’ channel switching behaviour (forman et al. 2009; gupta et al. 2004; pookulangara et al. 2011; viswanathan 2005). each channel has its respective advantages that potentially drive australasian journal of information systems madlberger & matook 2017, vol 21, research article theorizing e-commerce business models 6 customers to terminate a purchasing process in one channel only to continue it in another (gupta et al. 2004; kollmann et al. 2012). particular advantages of online over offline channels include saving time and effort, as well as search facilitation and comparison of products (rohm & swaminathan 2004), more comprehensive product information and a larger product assortment (alba et al. 1997). customers can enjoy additional benefits of the mobile channel that go beyond those benefits of the online channel. for example, transactions via the mobile channel benefit from the ubiquity of mobile devices, which increases customers’ opportunities to shop independently of their location. yet limitations in the size of mobile devices, together with potentially limiting situational factors (e.g., time pressure, lack of concentration), can drive customers back to the online channel (bang et al. 2013). another new channel for the acquisition of products and services involves web 2.0 social media technologies (liang & turban 2011). social media applications, such as linkedin, facebook and twitter, provide features for users to initiate and complete transactions. a strength of social media-enabled commerce (i.e., social commerce) is the availability of the known crowd of close others in the form of friends, contacts or followers. these people are often powerful advisers for a consumer’s purchasing decision, as research on word of mouth has shown (amblee & bui 2011; matook et al. 2015). in contrast, offline channels demonstrate the strengths of human interaction and in-store experience, providing product feeling, touching and testing possibilities prior to purchase, the satisfaction of immediate needs and an established infrastructure (prasarnphanich & gillenson 2003). the increased number of available channels and the channel switching phenomenon draw attention to the issue of transaction phase support. a number of studies examined the components of different e-commerce business models and studied their use, effectiveness and attractiveness for customers (for a detailed discussion of these prior research studies see zumpe & madlberger 2007a). most of these studies conclude that a lack of transaction focus in e-commerce business models exists and highlight the need for more research on the impact of transaction phases on the purchasing process. for example, pateli & giaglis (2004) define a research agenda on e-commerce business models and request clarification of the relationship between e-commerce and the related business transactions. in their understanding, if a business model does not encompass the concept of transactions and how different phases are to be supported, the business logic of the company is imperfect (hammer & champy 2003). similarly, in a recent publication, veit et al. (2014) theorize about the role of technology in linking business partners and customers via distinct channels to ensure value creation through these models. the authors again call for future research to foster a better understanding of channel composition and transaction support. in doing so, they highlight the persistent lack of theoretical and empirical studies in the literature about transaction phases in e-commerce business models. consequently, we concur that an e-commerce business model that lacks discussion of transactions, especially regarding transaction phases and how they are supported in different channels, is incomplete from a theoretical perspective. 3 hypotheses development 3.1 impact of transaction phase support on customer satisfaction and usage intention one of the key goals of e-commerce business models is the optimization of technology-enabled purchases and maximization of customer retention by employing customer relationship management (hamid & mcgrath 2005). to get a better understanding of how to achieve these goals, is researchers have studied the antecedents of online purchasing and customer retention variables, e.g., satisfaction and loyalty (chen 2012a; kumar et al. 2013; suh & yi 2012). in the context of e-commerce, customer satisfaction and loyalty are of major importance for online purchase intention or purchasing behaviour (massad et al. 2006; mckinney et al. 2002; shankara et al. 2003). as such, customer satisfaction leads to loyalty to the company and results in repeated purchases (anderson & srinivasan 2003; zeithaml 2000). further research on satisfaction with services in e-commerce settings demonstrates a significant influence of satisfaction on intention to use an online service (zhang et al. 2006). particularly in the online australasian journal of information systems madlberger & matook 2017, vol 21, research article theorizing e-commerce business models 7 space, customers have high expectations and, hence, push e-commerce companies in their efforts to achieve customer satisfaction (bhattacherjee 2001a). the number of transaction phases supported through a channel (e.g., online) translates into how a company seeks to achieve customer satisfaction. different phase support means different opportunities to offer a service or product to a customer. if all phases in an ecommerce business model are supported a customer can carry out the entire transaction within one channel and the transaction can be completed without the need to switch to another channel. in this case, the customer can finalize the transaction quickly and without any delays (berry et al. 2002). the full-phase transaction support represents a component in an ecommerce business model that requires only low levels of user engagement. however, some e-commerce business models offer customers additional services and enhanced customer care through partial transaction support in one channel. these additional offerings can be provided because customers are directed and hence, requested to switch from the online or mobile channel to the direct customer-company communication channel. the rationale for the company may be that the online channel is not appropriately equipped to offer a particular service or because the company seeks to enhance customer intimacy, as suggested in the business models of travel service providers such as flightcentre.com or flightfox.com. in such cases, opportunities for additional sales or improved customer services are presented. from the customer's viewpoint, partial transaction support results from companies' wish to have direct interaction with the customer and thus increase their customer engagement (attfield et al. 2011). moreover, tate et al. (2005) point out that close customer relationships in e-commerce settings can increase retention rates and, thus, raise profitability for the company. at the same time, the forced channel switching means that the customer cannot complete the transaction and must wait until the company contacts him or her. this implementation of the website is despite technological abilities to support all transaction phases. for example, additional support that goes beyond a website’s channel capability – as in the case of flightcentre.com and flightfox.com – is only accessible if the customer leaves the online channel (bang et al. 2013). in any case, the forced channel switch causes a delay of the transaction completion. as is research has shown, delays in online transactions can be perceived as a flaw that may ultimately have a negative impact on continuation to use the website (everard & mccoy 2010). in a similar vein, any features that irritate customers during the interaction with a website (e.g., interruptions) deteriorate the customer experience and hence, have a negative impact on attitude and usage intention (lim 2014). to summarize, partial transaction support will result in positive customer experiences due to higher customer engagement and interaction, but may also be less appreciated by customers due to delays of transaction completion. the literature argues that customers value convenience in their online shopping endeavours and high levels of convenience have resulted in increased degrees of customer satisfaction and usage intention (zhang et al. 2006). perceived convenience includes transaction convenience, which captures the fact that low mental effort and little time is required of the customer to complete a transaction (berry et al. 2002). the ability to complete a transaction online without any interruptions is, therefore, likely to increase convenience. hence, we expect that a website that supports all transaction phases will evoke a more favourable reaction from users than a website that requires switching to another channel. in a similar vein, we suggest that a website that provides value in the form of convenience has a higher chance of continued use, i.e., that it is reused (adapa & cooksey 2013). previous research from a different angle, i.e., the ecommerce success model (wang 2008) has shown that satisfaction impacts reuse intention of an e-commerce website (wang et al. 2016). conversely, we argue that customers do not value any extra services they may receive through direct customer-company communication enough to outweigh the lack of convenience of partial transaction support. more specifically, we posit that a website that fully supports all transaction phases will achieve higher satisfaction and a higher intention to reuse the website, than a website that provides only partial transaction support. australasian journal of information systems madlberger & matook 2017, vol 21, research article theorizing e-commerce business models 8 hypothesis 1: satisfaction with a website is more likely if all transaction phases are supported. hypothesis 2: intention to reuse a website is more likely if all transaction phases are supported. 3.2 impact of experience on customer satisfaction and usage intention user satisfaction with a website and the user’s decision to reuse the website is tightly connected with their prior experiences. the literature shows that prior experiences influence user skills and, therefore, users’ expectations of an is (delone 1988; fuerst & cheney 1982; hong et al. 2002; lee 1986). in the context of internet experience, kuhlmeier & knight (2005) demonstrate that internet skills positively impact online purchases. channel expansion theory (carlson & zmud 1999) aids in explaining the relationship between users’ experiences and online purchases. this theory considers user experience an important antecedent for the user’s perception of an information and communication technology (ict) (e.g., mobile channel, online channel). among other claims, the theory asserts that users’ prior experiences with the technology and the communication context increase the perceived value of the ict and, in turn, users’ satisfaction with the channel (ogara et al. 2014). the literature distinguishes between two different types of prior experience because, in a technology-enabled environment, the experience construct is multilayered. first, experience with the product or service itself, also referred to as cross-domain experience with the service (bhatnagar et al. 2003), and second, experience with the online transaction that relates to the service, also known as within-domain experience (bhatnagar et al. 2003). user experience of both types creates knowledge that allows users to apply leaner channels for more demanding tasks (carlson & zmud 1999). as brunelle (2009) demonstrates in a quantitative study, a customer’s experience with the online channel and their experience with product purchase both positively influence their intention to use an online store for information gathering as well as for online purchasing (brunelle 2009; carlson & zmud 1999). in this study, we distinguish between experience resulting from the frequency of service consumption in general (usage frequency of a service) and experience resulting from the frequency of purchasing this service online (online purchasing frequency of a service). as prior research shows, experiences with a service from a different channel (cross-domain) and experiences with a service purchased in the same channel (within-domain) equally influence the formation of online expectations (bhatnagar et al. 2003). frequent users of the general service and the online execution of the service will, therefore, have higher expectations. according to the expectation-confirmation theory, higher expectations lead to lower confirmation that, in turn, leads to lower satisfaction (bhattacherjee 2001b). thus, we propose: hypothesis 3: users with high service experience are less likely to be satisfied with a website that only partially supports transaction phases online. hypothesis 4: users with high service experience are less likely to reuse a website that only partially supports transaction phases online. hypothesis 5: users with high experience with online service purchasing are less likely to be satisfied with a website that only partially supports transaction phases online. hypothesis 6: users with high experience with online service purchasing are less likely to reuse a website that only partially supports transaction phases online. finally, we propose that users who are experienced in both experience types – that is, the service itself and online service purchasing – display a lower satisfaction and intention to reuse if the website only supports a single transaction phase. this argument is again in line with the expectation-confirmation theory, which postulates that higher expectations lead to lower confirmation that, in turn, leads to lower satisfaction (bhattacherjee 2001b). australasian journal of information systems madlberger & matook 2017, vol 21, research article theorizing e-commerce business models 9 hypothesis 7: users with high experience with both the service and online service purchasing are less likely to be satisfied with a website that only partially supports transaction phases online. hypothesis 8: users with high experience with both the service and online service purchasing are less likely to reuse a website that only partially supports transaction phases online. 4 research methodology the empirical testing of the above hypotheses was performed using an experimental research design. experiments are a preferred methodology when the researchers seek to control for extraneous factors that would otherwise confound the results (cook & campbell 1979). indeed, experiments allow for increased internal validity and specifically tailor the empirical investigation to a particular problem (zikmund 2003). as we elaborate in more detail below, our empirical research design involves exposure to two websites, one offering a service with support of all transaction phases and the other offering support of the information phase only. the experimental design was set up as a laboratory experiment to maximize precision (dennis & valacich 2001). the experiment was designed as a one-shot case study (malhotra et al. 2012) with two treatments (i.e., the two websites) followed by measurement with a questionnaire. we followed guidelines for experimental research in determining several requirements for setting up the websites to be used as stimuli in the experiment (kirk 2013; shadish et al. 2002). first, to ensure participants concentrated on the transactions rather than various other aspects of usage of the website, the websites had to have low complexity. in doing so, we minimized participants' cognitive load and a possibly confounding impact of participants' varying levels of navigation and interaction skills and thus, ease of use. further the website design aimed at avoiding distraction from carrying out the transaction (agarwal & karahanna 2000). second, the websites needed to be based on otherwise identical e-commerce business models that only differed in the transaction phase support. we have purposefully not included any customercompany interactions that take place once the direct link between the customer and the company has been established, as it is the case in the partial transaction phase support. third, any other exogenous influences on satisfaction and reuse (for example promotions, special packages or visual elements) needed to be eliminated, thus requiring that the websites were identical in all respects other than the support of transaction phases. these conditions led to the conclusion that real commercial websites were not eligible for the experiment. in an attempt to ensure that all the above-mentioned conditions were met, the websites were developed specifically for the experimental design. 4.1 subjects subjects in this experiment were 54 undergraduate and graduate business students at an austrian university. researchers need to ascertain that their sample size is adequate to avoid that an effect is found in the data that may not reflect a true effect (hair et al. 2009). to ensure the size of the sample is adequate, we determined the statistical power using the software g*power 3 (faul et al. 2007). assuming an alpha of 0.05, a statistical power of 0.8 and an effect size of 0.4 resulted in a minimum required sample size of 32, which is less than the sample size in the study. hence, we concluded a sufficient large sample size for explaining moderate population effects, however, the sample would not be sufficient for explaining strong population effects using covariance-based path modelling. a student population is considered appropriate for this research as this group is influential and a representative gatekeeper for spreading opinion and information regarding internet content (su 2007). university students are suitable subjects on whom to conduct an experiment that focuses on internet purchasing transactions given that internet buyers tend to be young, well educated and have high levels of computer literacy (swinyard & smith 2003). the average age in the sample was 25 years and 29.4% were female. the gender bias was a result of a high proportion of male students in the university programme. however, male internet users still comprise the majority of online shoppers according to various statistics. in australasian journal of information systems madlberger & matook 2017, vol 21, research article theorizing e-commerce business models 10 2015, 62.6% of men shopped online within the preceding 12 months, whereas only 52.9% of women did (statistik austria 2016). similarly, certain age groups dominate among online shoppers. together comprising more than 80% of online shoppers, the age groups of 16 to 24 years and 25 to 35 years still dominate among internet consumers (statistik austria 2016). based on these statistics, we conclude that the sample sufficiently represents internet shoppers. subjects were invited to participate in the study during a regular class and the experiment was conducted immediately after that class. the experiment took place in a computer lab with pcs of identical configuration. the subjects were randomly assigned to commence the experiment on one of the two websites. once each group finished their booking with their starting website, they swapped and performed the second experimental task, i.e., booking via the other website. 4.2 materials the materials used were the two websites and an accompanying questionnaire. both were pretested with a separate group of five students. as a result, the websites were optimized in terms of wording. the two websites were developed using html and simulated simplified travel agency websites with identical visual design and navigation, offering the necessary functionalities to book domestic and international flights, which was also confirmed in our pretest. in the laboratory experiment both websites only included features that were necessary for carrying out the transaction in order to avoid distraction from carrying out the booking tasks. the websites were assigned invented names: “dream flight” and “flight heaven”. the study welcome page introduced the study, explained the experimental task, provided links to both travel websites and obtained consent from the students. the structure of the two websites was as follows. dream flight allowed the user to conduct all transaction phases online. it comprised five single web pages: the welcome page of the travel agency, a page with domestic flight offers (time, price, airline), a page with international flight offers (time, price, airline), a page with booking and payment details, and a final confirmation page with the reference number. no additional services or assistance were available during the transaction. the second website, flight heaven, supported the information phase only. it also comprised five single web pages: the welcome page and pages with domestic and international flight offers were duplicated from dream flight, while the remaining two pages unique to flight heaven covered the following. on one page, the benefits of the website only supporting the information phase were presented. customers were told that they could receive a number of value-adding services through personal contact with the service operators, such as advice about accommodation options, offerings of discounted hotels, tailored tours, travel insurance or particular flight times to the chosen destination. furthermore, this page requested customer contact details for future contact through flight haven and a final web page with information on the response time for the flight requests. hence, this website provided the promise of a soon callback service as an additional service offering, but terminated the online transaction after the information phase. 4.3 procedure each subject conducted the entire experiment without interruption. an instructor provided the subjects with the url and a password to access the study. the subjects were instructed to use the assigned website to book two flights, viz. a domestic flight and an international, longdistance flight within a specified date range and to look for the cheapest flights. in the first phase of the experiment, one portion of the subjects conducted the experiment with the website dream flight, while the remaining subjects used the website flight heaven. after completion of these online tasks, the subjects were asked to fill out a questionnaire. in the second phase of the experiment, the two groups performed the same tasks with the other website. again, the subjects were asked to conduct the experimental tasks of booking the two flights, before completing a survey questionnaire. australasian journal of information systems madlberger & matook 2017, vol 21, research article theorizing e-commerce business models 11 4.4 measurements the measurement items utilized were adapted from the literature. the questionnaire measured satisfaction, reuse intention, frequency of flight booking (experience with the service), proportion of online booking (experience with online purchasing of the service) and socio-demographic data. the items used to measure customer satisfaction and reuse intention were taken from the web customer satisfaction scale by mckinney et al. (2002) using a 7-point likert scale. we slightly modified the items of both constructs to adapt them to the experimental task and the tourism context, in particular the air travel industry (granados et al. 2007). table 1 and table 2 show the results of the reliability analyses including consistency, convergent validity and discriminant validity. cronbach’s alpha and composite reliability values are higher than the recommended value of 0.7 and thus deemed acceptable (hair et al. 2009; nunnally 1978). convergent validity is satisfactory if the average variance extracted (ave) is higher than 0.5 (fornell & larcker 1981). with 0.775 and 0.946 respectively, this condition is met for both multi-item constructs. further, the loadings of the individual items on the corresponding variables are well above the recommended value of 0.5 for appropriate discriminant validity. variable items min max mean sd loadings cronbach’s alpha composite reliability ave satisfaction (sati) sati1 sati2 sati3 sati4 1 1 1 1 7 7 7 7 3.81 3.84 3.99 4.19 1.560 1.243 1.225 1.147 0.867*** 0.908*** 0.907*** 0.838*** 0.903 0.932 0.775 reuse intention (reuse) reuse1 reuse2 1 1 7 7 3.63 3.34 1.698 1.642 0.974*** 0.971*** 0.943 0.972 0.946 *** p < 0.001 table 1: reliability measures of the variables. the frequency of general service consumption (flight booking) was measured as a single item (chuanlan & forsythe 2010) with a 4-point scale (less than once a year, once to twice a year, three to five times a year, and more than five times a year). the online flight booking experience was also measured as a single item by asking for the proportion of flights booked online (chang & samuel 2004) using four answer categories (never, less than half of the flights, more than half of the flights, and for each flight). table 2 presents the numbers concerning discriminant validity. the numbers in italics are the square roots of the ave. for adequate discriminant validity, these values should exceed the inter-construct correlations. this condition is met for all constructs. given the strong results of the consistency, convergent validity and discriminant validity, the results of the measurement model are considered acceptable. variable mean sd satisfaction reuse intention product or service experience online channel experience satisfaction 3.96 1.139 0.880 reuse intention 3.49 1.625 0.832** 0.973 product or service experience 2.38 0.930 -0.042 -0.055 n/a online channel experience 2.09 1.077 0.208* 0.158 0.025 n/a * p < 0.05 (2-tailed), ** p < 0.01 (2-tailed) table 2: descriptive statistics and discriminant validity. australasian journal of information systems madlberger & matook 2017, vol 21, research article theorizing e-commerce business models 12 5 results 5.1 the bayesian network data analysis was performed by means of a bayesian network, which is an appropriate approach for building models in a context with intrinsic uncertainty (lauría & duchessi 2007). whereas commonly used statistical tests (e.g., structural models) seek to generalize findings and provide statistical significance, their use is questionable in cases of small data samples. indeed, zheng & pavlou (2010) elaborate on the use of bayesian networks to overcome limitations of structural equation modelling. bayesian networks permit supporting measurement models, but they also help to identify the most likely causal structure in a structural model. consequently, the bayesian network approach has been applied in various is studies to gain enhanced insights into novel research fields, e.g., concerning success factors in it projects (gingnell et al. 2014) and is security factors (feng et al. 2014). bayesian networks are well established in computer science and operations research as they enable researchers to make predictions using conditional probabilities (korb & nicholson 2004). our choice was motivated by the benefits of bayesian networks. they are sensitive to the smallest changes in the data and, hence, detect variances on the dependent variable. further, they are independent of the sample size. finally, bayesian networks are particularly powerful in representing uncertain knowledge (ghosh & valtorta 2000) and permit the analysis of “what-if” questions (lauría & duchessi 2007). in general, bayesian networks can be used either for diagnostic reasoning or predictive reasoning (korb & nicholson 2004). we use them here for the latter. bayesian networks are also termed belief networks, causal probabilistic networks or probabilistic influence diagrams (xenos 2004). a bayesian network contains two elements: nodes (displayed as tables) and directed arcs (displayed as arrows). the nodes represent the mathematical variables of the constructs (satisfaction, reuse intention, experiences). the arcs are the relationships between the nodes (jensen 1997). the output of a bayesian network allows for a consistent estimation of the way in which the input probabilities impact uncertain conclusions – for example, on the internet users’ future reuse intentions. to utilize the bayesian network, we had to conduct certain preparatory analyses. first, we needed to test whether the order in which the subjects had used the two websites had influenced their answers. a randomly assigned portion of the subjects tested dream flight first and flight heaven second, while the other group worked the other way around. an anova showed no significant differences between these two groups. therefore, we excluded a bias due to the order of the website interactions. second, a bayesian network requires entering probabilities for each investigated node. thus, we determined the probabilities of the variables, i.e., experience with flight booking, experience with online flight booking, satisfaction and intention to reuse. for this purpose, we reduced complexity and performed a scale transformation on the data by transforming the original 7-point and 4-point scales into simplified 3-point and 2-point scales. for satisfaction and reuse intention, answers ranging from one to three were summarized as “negative” answers, answers ranging from five to seven were treated as “positive” answers, and answers that were exactly four were assigned as “neutral” answers. similarly, we transformed the experience items into two categories respectively by assigning the first two answer categories to “low” experience and the last two categories to “high” experience. following this scale transformation, the frequencies of each variable’s different states in the sample were computed. all frequencies of combinations between the variables were recorded in truth tables and then entered as probabilities into the bayesian network. therefore, the bayesian network presents only data that represent the structure of probabilities within the experimental sample. the bayesian network was configured using the open source software bayesbuilder. this tool was developed by stichting neural networks (snn) at the university of nijmegen, netherlands (snn-adaptive-intelligence 2013). australasian journal of information systems madlberger & matook 2017, vol 21, research article theorizing e-commerce business models 13 5.2 initial model of the bayesian network the calculation of the bayesian network using the transformed data created the following probability structures for both experimental websites, that is, dream flight with full transaction support and flight heaven with support of the information phase only. to analyse the data with a bayesian network the creation of an initial model is required. its structure is depicted in figure 2 and is interpreted as follows: to measure the impact of experience on satisfaction and reuse intention, we considered flight booking experiences (left node) and online booking experiences (right node). in the first step, we computed the probabilities based on the data, providing the results in figure 2. dream flight (full support of transaction) flight heaven (support of information phase only) figure 2. bayesian network of both websites. figure 2 can be interpreted as follows: in the flight booking experience node, the majority of participants have low experience with flight booking since 63% answered they had little or no experience and 37% answered that they had high or very high booking experience. therefore, the probability in the sample that one subject is experienced in flight booking is 37%. in contrast, the online booking experience node shows that most of the participants (78%) book the majority of their flights online and thus are experienced with online flight booking. both websites show similar probabilities of satisfaction – that is 48% for dream flight and 46% for flight heaven. the difference between these two probabilities is small, thus we do not accept hypothesis 1. concerning intention to reuse, the probability of reuse intention is 33% for dream flight and 28% for flight heaven. therefore, the website that supports the information phase only is less likely to be reused than the website that supports the entire transaction, as postulated in hypothesis 2. at this point, it is important to notice that the acceptance or rejection of each proposition is not derived from statistical significance tests but from differences in probabilities in the research sample. to test the further propositions with the bayesian networks, the probabilities were punched, that is, the conditional probabilities of satisfaction and reuse intention were calculated by determining the levels of experience in advance. in other words, it was defined that one particular subject shows a certain level of experience before computing the corresponding probabilities of satisfaction and reuse intention. in terms of the bayesian network, this means that we refer to an observed case (ypma et al. 2004), that is, a subject who is experienced in booking flights or purchasing flights online. australasian journal of information systems madlberger & matook 2017, vol 21, research article theorizing e-commerce business models 14 5.3 high flight booking experience in the bayesian network first, we calculated the probabilities of various satisfaction levels and reuse intention for users who have high experience with flight booking. in figure 3, the punches are the blue bars and the red bars refer to the conditional probabilities that occur among these experienced users. thus, figure 3 presents the impact of user experience regarding general service consumption on satisfaction and reuse intention for both websites. dream flight (full support of transaction) flight heaven (support of information phase only) figure 3. probability structure with predefined high experience with flight booking the probability of high satisfaction is considerably larger for dream flight (40%) than for flight heaven (34%). hypothesis 3 is therefore supported, which presumes that users experienced with flight booking are less likely to be satisfied with the website that supports only the information phase. when it comes to intention to reuse, we observe that the probability is again larger for dream flight, which offers the entire transaction online (28% for dream flight vs. 20% for flight heaven). therefore, hypothesis 4 can also be supported. 5.4 high online booking experience in the bayesian network figure 4 presents the probabilities in case of high experience with online flight booking for both websites. for this purpose, we punched the experience with online booking, i.e., we defined that one particular subject shows a high level of online booking experience. we then computed the corresponding probabilities of satisfaction and reuse intention. dream flight (full support of transaction) flight heaven (support of information phase only) figure 4. probability structure with predefined high experience with online flight booking australasian journal of information systems madlberger & matook 2017, vol 21, research article theorizing e-commerce business models 15 in the context of users who book most of their flights online, we observe that users are again more likely to be satisfied with dream flight (47%) than with flight heaven (40%). it can further be noticed that dream flight has a considerably lower likelihood of dissatisfied users (41%) than flight heaven (50%). this finding conforms to hypothesis 5. reuse intention is also higher for dream flight (33% probability) than for flight heaven (24%). hence, flight heaven achieves worse results than dream flight in this regard, which supports hypothesis 6. 5.5 high flight booking experience and high online booking experience in the bayesian network finally, we investigated the effect when users have high levels of both types of experience simultaneously. for this purpose, we punched high experiences with both flight booking in general and online flight booking. figure 5 shows the results. dream flight (full support of transaction) flight heaven (support of information phase only) figure 5. probability structure with predefined high experience with flight booking and online flight booking as the above results indicate, the largest differences between the two websites are found here. amongst users who are highly experienced in both dimensions, dream flight achieves considerably better results than flight heaven. the probability that experienced users are satisfied with dream flight is 40% while it is 27% for flight heaven. with regard to dissatisfaction, users return a likelihood of 47% for dream flight and 53% for flight heaven. the differences are also considerable concerning reuse intention. given that the users are experienced both with booking flights and booking flights online, the likelihood that dream flight is reused is 28% while it is only 16% for flight heaven. with regard to user reports of not intending to reuse the website, the likelihood for dream flight is 59% versus 65% for flight heaven. thus, among experienced users, dream flight achieves considerably better results than flight heaven, which supports hypotheses 7 and 8. 5.6 summary of results summarizing the findings, the results show that customers are equally likely to be satisfied with the fullyand partly-supported booking transactions. however, reuse intention is more likely for the fully-supported transaction. despite the promise of potential additional service offerings, we can observe that, for all combinations of experience (experience with the booking process generally, experience with online booking, and both), the full support of all transaction phases enjoys higher probabilities for positive customer reactions than does the support of only the information phase. table 3 summarizes our results. australasian journal of information systems madlberger & matook 2017, vol 21, research article theorizing e-commerce business models 16 hypothesis supported/ rejected h1: satisfaction with a website is more likely if all transaction phases are supported. rejected h2: intention to reuse a website is more likely if all transaction phases are supported. supported h3: users with high service experience are less likely to be satisfied with a website that only partially supports transaction phases online. supported h4: users with high service experience are less likely to reuse a website that only partially supports transaction phases online. supported h5: users with high experience with online service purchasing are less likely to be satisfied with a website that only partially supports transaction phases online. supported h6: users with high experience with online service purchasing are less likely to reuse a website that only partially supports transaction phases online. supported h7: users with high experience with both the service and online service purchasing are less likely to be satisfied with a website that only partially supports transaction phases online. supported h8: users with high experience with both the service and online service purchasing are less likely to reuse a website that only partially supports transaction phases online. supported table 3. summary of results 6 discussion our research aimed to identify the impact of online transaction phase support on user satisfaction and reuse intention in e-commerce business models. in particular, we examined the impact of partially-supported transaction phases on customers’ perception of the two variables. in addition, we sought to characterize the type of user that prefers different levels of online support of transaction phases. we found evidence in a first exploratory study that experienced users – that is, users who are familiar with the general service consumption and/or online purchase of the service – are less satisfied and have a lower intention to reuse the website if it forces them to switch to another channel of communication or distribution. experienced users evaluate the website that supports only the information phase less favourably as compared to a website offering complete transaction support. 6.1 contributions to theory our research contributes in various ways to the existing literature in is and marketing, especially in the areas of e-commerce business models, e-commerce, channel design and channel switching, and to prior research on customer satisfaction and reuse. we elaborate on these four main contributions in detail as follows. first, we contribute to the literature on e-commerce business models by introducing the concept of transaction phases into a business model. prior research has produced core definitions for these phenomena and created a number of frameworks, typologies and research agendas to classify, evaluate and innovate e-commerce business models (veit et al. 2014). these studies have laid a solid foundation for further research to delve more deeply into specific aspects of the e-commerce business model and perform empirical research but have not integrated the view of transaction phases into e-commerce business models. our study highlights the e-commerce business model’s concept of transaction phases and varying transaction support in one channel as a strategic factor for companies to create value and coordinate their business resources (osterwalder et al. 2005). testing the extent to which full transaction support impacts satisfaction and reuse intention as compared to partial support provides initial empirical evidence for this concept of a business model. the results clearly show that benefits of e-commerce business models that provide partial support of transaction phases in the form of increased customer engagement cannot offset the negative impacts of the inherent delay in transaction completion through channel switching. thus, the study endorses australasian journal of information systems madlberger & matook 2017, vol 21, research article theorizing e-commerce business models 17 findings of e-commerce literature on general drivers of customer satisfaction and reuse intention at the level of single transaction phases (wang 2008; chen 2012; liao et al. 2011). extending our understanding of e-commerce business models in this regards is important considering the increasing variety of electronic devices that can be used for online purchasing. ubiquitous mobile devices facilitate shopping online independent of location (bang et al. 2013) and, as such, offer opportunities to complete the entire purchasing transaction in the mobile channel alone without the need to switch to another channel, while benefiting from location information in the form of more tailored service offerings. our second contribution is to the e-commerce literature, demonstrating the impact of transaction phase support and varying degrees of user experience while conducting online shopping. prior research has theorized about transaction phases in e-commerce, their impact on website adoption and the advantages of e-commerce over traditional purchasing processes (choudhury & karahanna 2008; pavlou & fygenson 2006). our findings show that transaction phases also influence a user’s perception of a website regarding satisfaction and reuse intention. by investigating the role of prior experiences, we accounted for an antecedent of satisfaction and reuse intention that has been investigated in different is contexts (roca et al. 2006; wang 2003) but less frequently in e-commerce research (chen 2012). the construct of prior experiences concerning customers’ familiarity with online purchasing and the service/product booking offered both have an impact on how users evaluate the website. the third contribution relates to the marketing literature on channel design and channel switching. prior research has indicated the importance of multi-channel design and the opportunities possible for customers when switching between channels (brynjolfsson et al. 2013; hansen & sia 2015; pookulangara et al. 2011; verhoef et al. 2015). however, this body of research did not discuss switching in regards to the phases of the transaction and when best to implement the switch. it is implicitly assumed that customers value switching between different channels during the purchasing process. our research findings illustrate that when customers are forced to switch channels they experience lower degrees of satisfaction and are less likely to reuse the website. as such, our research suggests that, when multiple channels are offered, each channel should provide the possibility of completing the transaction. fourth, we contribute to the literature on customer satisfaction and reuse. our findings demonstrate that satisfaction and reuse intention – two core concepts of customer loyalty creation – are affected by the support of transaction phases. extensive research on satisfaction and reuse in e-commerce settings has shown that these two variables are impacted by service quality (kettinger et al. 2009; setia et al. 2007), perceived value (wang 2008), perceived usefulness and system quality (setia et al. 2007), and regret (liao et al. 2011). with our study, we propose transaction phase support as a new antecedent of satisfaction and reuse intention of websites. understanding which phases need to be supported within a channel to achieve high customer satisfaction and reuse intention is important as it determines the degree to which a customer is ultimately loyal to the business. 6.2 implications for practice this research provides a number of implications for practice. first, from a managerial perspective, we suggest considering transaction phases an essential aspect of the e-commerce business model for a company to define on a fine-grained basis. by explicitly referring to the support of transaction phases, companies are encouraged to realize a more strategy-oriented plan of their e-commerce websites. in fact, support of transaction phases may be regarded as a means of differentiation. we do not entirely call for a maximization of transaction phases supported online, but rather point to the necessity of identifying the optimal number and kind of supported online transaction phases. if a firm decides to support only single transaction phases online, this must be part of a well-considered e-commerce strategy. in particular, firms should consider the nature of the offered products or services as well as key contextual factors. for example, in case of products or services that are usually purchased after an extensive purchase decision (d'astous et al. 1989), customers may feel more satisfied if they are directed to a service-oriented channel with personal interactions and additional assistance. on the australasian journal of information systems madlberger & matook 2017, vol 21, research article theorizing e-commerce business models 18 other hand, in case of limited or habitual purchase decisions or if products of low complexity are involved, customers might prefer to complete the transaction in the original selected channel. in any case e-commerce firms should not settle for only partial online support of transaction phases due to limitations of their online or mobile presence. second, we raise the issue of target groups of e-commerce customers. if companies cater to highly experienced users – a situation that applies, for example, to industries with standardized products, a relatively “long” e-commerce history and digital natives – these users have an expectation that they will be able to complete a transaction quickly, without delays or disruptions. however, companies can potentially offer extra value-adding services through direct customer-company interactions once users switch from the online/mobile channel to the direct customer-company communication channel. it seems that customers do not appreciate these services to the same extent as they value full transaction phase support. hence, companies must be aware of customers’ experience levels and the low ratings of satisfaction and reuse intention partial transaction phase support triggers. indeed, companies should consider that a lack of transaction phase support may have negative impacts on their image and the other channels they offer. furthermore, the decision of transaction phase support in an e-commerce business model becomes even more prevalent in environments of technological constraints where forced channel switching is not a customer engagement strategy but the only possibility to sell products and services. compared with industrialized countries, the growth rate of the internet is significantly higher in developing countries despite some economic and technological constraints such as affordable internet access (worldbank 2016). consequently, the relevance of e-commerce business models is rising due to increasing internet penetration in developing countries assuming improvements in economic conditions and technological infrastructure (mccoy et al. 2012). hence, companies that expand to regions with rapidly growing internet penetration may be faced with technological constraints that require their e-commerce business models to enforce channel switching. 6.3 limitations and future research like any research, this study has several limitations that should be considered when examining the results. first, we have compared just two channels: the online and the offline channel. it may be that this selection of channels influenced users’ satisfaction and reuse intention. second, we have tested the differences in transaction support in the service sector, more specifically the tourism industry. therefore, it is possible that the results are valid for this scenario alone and not transferable to other sectors, such as the retail sector. third, the present study is a very simplistic model of user-specific antecedents of satisfaction and reuse intention, which only incorporates prior experiences. therefore, this research suggests a number of future research directions. first, although our channel design involved only two channels, a variety of different channels exist – including mobile and social media channels – that entail a multitude of channel switching opportunities. thus, we suggest including different channels when testing for partial transaction support. it could potentially be the case, for example, that the disadvantages and advantages of the mobile channel, such as small screen size and a lack of appropriate mobile apps, in concert with the benefits of transmitting location data, result in different findings regarding satisfaction and reuse intention. second, we only differentiated between three transaction phases. while the trisection (i.e., information, agreement and fulfilment) is the division that is most often used in the literature, we suggest examining forced channel switching with more than three transaction phases. in such an investigation, the channel switching could be forced at multiple points of the transaction and potentially result in different findings, as users might appreciate the receipt of after-sales services in a different and more personal channel. third, we have examined prior experience as an impact factor for how satisfied a customer is with the website and whether he/she intends to reuse it. there are a number of other factors australasian journal of information systems madlberger & matook 2017, vol 21, research article theorizing e-commerce business models 19 that may also have such an impact (monsuwé et al. 2004). these factors include personal characteristics of the customer (e.g., self-efficacy, online trust), characteristics of the purchasing process (e.g., task/product/service complexity, task/product/service familiarity) and characteristics of the company (e.g., domain knowledge of the provider). examining the impact these factors can have on user experience enlarges our collective knowledge base regarding e-commerce business models and provides a better understanding of appropriate transaction phase support in various channels. another future research topic may compare the satisfaction and reuse of the combined channel experience including the forced switching. in this way, an entire purchasing process is compared and potentially changing dynamics of satisfaction and re-use intention may emerge. finally, we set up the experiment as a laboratory experiment to achieve a high internal validity. future research should complement these insights with a field experiment involving existing websites that apply forced channel switching in their business models. 7 conclusion in this research, we investigate the impact of varying degrees of trasaction phase support on customer satisfaction and reuse intention in e-commerce settings. our theoretical foundation is based on the literature of e-commerce business models, transaction phases and channel expansion theory. we test a model by providing users with two online websites: one with full transaction support and the other with only partial transaction support. for each condition and based on the users’ levels of experience, we determined their degree of satisfaction and reuse intention. our exploratory study shows that varying online support of transaction phases influences reuse intention depending on the experience a user possesses. in case of high experience with the service in general, and particularly the online purchasing of the service, reuse intention and satisfaction decrease if not all transaction phases are supported. these findings contribute to a more nuanced conceptualization of e-commerce business models considering transaction phases and user experience, and an improved understanding of channel design and users’ perceptions of forced channel switching. references adapa, s., & cooksey, r. (2013) "factors affecting consumers' continued use of internet banking: empirical evidence from australia", australasian journal of information systems, 18(1): 5-31. agarwal, r., & karahanna, e. (2000) "time flies when you're having fun: cognitive absorption and beliefs about information technology usage", mis quarterly, 24(4): 665-694. al-debei, m.m., & avison, d. (2010) "developing a unified framework of the business model concept", european journal of information systems, 19(3): 359-376. alba, j., lynch, j., weitz, b., janiszewski, c., lutz, r., sawyer, a., & wood, s. (1997) "interactive home shopping: consumer, retailer, and manufacturer incentives to participate in electronic marketplaces", journal of marketing, 61(3): 38-53. amblee, n., & bui, t. (2011) "harnessing the influence of social proof in online shopping: the effect of electronic word of mouth on sales of digital microproducts", international journal of electronic commerce, 16(2): 91-114. amit, r., & zott, c. (2001) "value creation in e-business", strategic management journal, 22(6/7): 493-520. anagnostopoulos, i., anagnostopoulos, c., loumos, v., & kayafas, e. (2004) "classifying web pages employing a probabilistic neural network", ieee proceedings software, 151(3): 139-150. anderson, r.e., & srinivasan, s.s. (2003) "e-satisfaction and e-loyalty: a contingency framework", psychology and marketing, 20(2): 123-138. australasian journal of information systems madlberger & matook 2017, vol 21, research article theorizing e-commerce business models 20 attfield, s., kazai, g., lalmas, m., & piwowarski, b. (2011) "towards a science of user engagement (position paper)", wsdm'11 conference, hong kong. bang, y., lee, d.-j., han, k., hwang, m., & ahn, j.-h. (2013) "channel capabilities, product characteristics, and the impacts of mobile channel introduction", journal of management information systems, 30(2): 101-126. berry, l.l., seiders, k., & grewal, d. (2002) "understanding service convenience", journal of marketing, 66(3): 1-17. bhatnagar, n., lurie, n., & zeithaml, v. (2003) "reasoning about online and offline service experiences: the role of domain-specificity in the formation of service expectations", advances in consumer research, 30(1): 383-384. bhattacherjee, a. (2001a) "an empirical analysis of the antecedents of electronic commerce services continuance", decision support systems, 32(2): 201-214. bhattacherjee, a. (2001b) "understanding information systems continuance: an expectation-confirmation model", mis quarterly, 25(3): 351-370. brunelle, e. (2009) "introducing media richness into an integrated model of consumers’ intentions to use online stores in their purchase process", journal of internet commerce, 8(3/4): 222-245. bunn, m.d. (1993) "taxonomy of buying decision approaches", journal of marketing, 57(1): 38-56. cao, l. (2014) "business model transformation in moving to a cross-channel retail strategy: a case study", international journal of electronic commerce, 18(4): 69-96. carlson, j.r., & zmud, r.w. (1999) "channel expansion theory and the experiental nature of media richness perceptions", academy of management journal, 42(2): 153-170. chang, j., & samuel, n. (2004) "internet shopper demographics and buying behaviour in australia", journal of american academy of business, cambridge, 5(1/2): 171-176. chen, y.-y. (2012) "why do consumers go internet shopping again? understanding the antecedents of repurchase intention", journal of organizational computing & electronic commerce, 22(1): 38-63. chesbrough, h., & rosenbloom, r.s. (2002) "the role of the business model in capturing value from innovation: evidence from xerox corporation's technology spin-off companies", industrial and corporate change, 11(3): 529-555. choudhury, v., & karahanna, e. (2008) "the relative advantage of electronic channels: a multi-dimensional view", mis quarterly, 32(1): 179-200. chuanlan, l., & forsythe, s. (2010) "post-adoption online shopping continuance", international journal of retail & distribution management, 38(2): 97-114. cook, t.d., & campbell, d.t. (1979) quasi-experimentation: design and analysis issue, houghton mifflin, boston, ma. d'astous, a., bensouda, i., & guindon, j. (1989) "a re-examination of consumer decision making for a repeat purchase product: variations in product importance and purchase frequency", advances in consumer research, 16(1): 433-438. daidj, n. (2015) developing strategic business models and competitive advantage in the digital sector, igi global, hershey, pa. dennis, a.r., & valacich, j.s. (2001) "conducting research in information systems", communications of the association for information systems, 7(5): 1-41. emarketer (2015) worldwide retail ecommerce sales: emarketer's updated estimates and forecast through 2019. available at: australasian journal of information systems madlberger & matook 2017, vol 21, research article theorizing e-commerce business models 21 http://www.emarketer.com/public_media/docs/emarketer_etailwest2016_worldwid e_ecommerce_report.pdf [accessed 8 march 2016]. everard, a., & mccoy, s. (2010) "effect of presentation flaw attribution on website quality, trust, and abandonment", australasian journal of information systems, 16(2): 5-19. faul, f., erdfelder, e., lang, a.-g., & buchner, a. (2007) "g*power 3: a flexible statistical power analysis program for the social, behavioral, and biomedical sciences", behavior research methods, 39(2): 175-191. feng, n., wang, h.j., & li, m. (2014) "a security risk analysis model for information systems: causal relationships of risk factors and vulnerability propagation analysis", information sciences, 256(1): 57-73. fornell, c., & larcker, d.f. (1981) "evaluating structural equation models with unobservable variables and measurement error", journal of marketing research, 18(1): 39-50. ghosh, j.k., & valtorta, m. (2000) "building a bayesian network model of heart disease", 38th annual southeast regional conference clemson, acm press, south carolina, 239240. gingnell, l., franke, u., lagerström, r., ericsson, e., & lilliesköld, j. (2014) "quantifying success factors for it projects an expert-based bayesian model", information systems management, 31(1): 21-36. granados, n.f., gupta, a., & kauffman, r.f. (2007) "it-enabled transparent electronic markets: the case of the air travel industry", information systems and e-business management, 5(1): 65-91. gupta, a., su, b., & walter, z. (2004) "an empirical study of consumer switching from traditional to electronic channels: a purchase-decision process perspective", international journal of electronic commerce, 8(3): 131-161. hair, j.f., black, w.c., babin, b.j., & anderson, r.e. (2009) multivariate data analysis, pearson, harlow. hamid, n.r.a., & mcgrath, m.g. (2005) "the diffusion of internet interactivity on retail web sites: a customer relationship model", communications of the iima, 5(2): 35-46. hammer, m., & champy, j. (2003) reengineering the corporation: a manifesto for business revolution, harper collins, new york, ny. herhausen, d., binder, j., schoegel, m., & herrmann, a. (2015) "integrating bricks with clicks: retailer-level and channel-level outcomes of online-offline channel integration", journal of retailing, 91(2): 309-325. huang, l., lu, x., & ba, s. (2016) "an empirical study of the cross-channel effects between web and mobile shopping channels", information and management, 53(2): 265-278. jensen, f. (1997) an introduction to bayesian networks, springer, new york. kettinger, w.j., park, s.-h., & smith, j. (2009) "understanding the consequences of information systems service quality on is service reuse", information and management, 46(6): 335-341. kim, j., & lee, j. (2002) "critical design factors for successful e-commerce systems", behaviour and information technology, 21(3): 185-199. kirk, r.e. (2013) experimental design. procedures for the behavioral sciences, sage publications, thousand oaks. kollmann, t., kuckertz, a., & kayser, i. (2012) "cannibalization or synergy? consumers’ channel selection in online-offline multichannel systems", journal of retailing and consumer services, 19(2): 186-194. australasian journal of information systems madlberger & matook 2017, vol 21, research article theorizing e-commerce business models 22 korb, k.b., & nicholson, a.e. (2004) bayesian artificial intelligence, chapman & hall/crc, london, uk. kotler, p.t., & keller, k.l. (2015) marketing management, pearson education, prentice hall. kuhlmeier, d., & knight, g. (2005) "antecedents to internet-based purchasing: a multinational study", international marketing review, 22(4): 460-473. lauría, e.j.m., & duchessi, p.j. (2007) "a methodology for developing bayesian networks: an application to information technology (it) implementation", european journal of operational research, 179(1): 234-252. lehmann, j., lalmas, m., yom-tov, e., & dupret, g. (2012) "models of user engagement", umap'12 20th international conference on user modeling, adaptation, and personalization, springer, 164-175. liang, t.-p., & turban, e. (2011) "introduction to the special issue social commerce: a research framework for social commerce", international journal of electronic commerce, 16(2): 5-14. liao, c., liu, c.-c., liu, y.-p., to, p.-l., & lin, h.-n. (2011) "applying the expectancy disconfirmation and regret theories to online consumer behavior", cyberpsychology, behavior and social networking, 14(4): 241-246. lim, w.m. (2014) "antecedents and consequences of e-shopping: an integrated model", internet research, 25(2): 184-217. lindner, m. (2016) younger consumers turn increasingly to mobile devices for buying as well as browsing . available at: https://www.internetretailer.com/2016/04/25/younger-consumers-turn-increasinglymobile-devices-buying [accessed 25 may 2016]. malhotra, n.k., birks, d.f., & wills, p. (2012) marketing research. an applied approach, pearson education, essex. matook, s., brown, s.a., & rolf, j. (2015) "forming an intention to act on recommendations given via online social networks", european journal of information systems, 24(1): 76-92. mccoy, s., cha, h.s., & durcikova, a. (2012) "modeling internet diffusion in developing countries", australasian journal of information systems, 17(2): 5-23. mckinney, v., yoon, k., & zahedi, f.m. (2002) "the measurement of web-customer satisfaction: an expectation and disconfirmation approach", information systems research, 13(3): 296-315. monsuwé, t.p., dellaert, b.g.c., & de ruyter, k. (2004) "what drives consumers to shop online? a literature review", international journal of service industry management, 15(1): 102-121. morris, m., schindehutte, m., & allen, j. (2005) "the entrepreneur's business model: toward a unified perspective", journal of business research, 58(6): 726-735. nunnally, j.c. (1978) psychometric theory, mcgraw-hill, new york, ny. ogara, s.o., koh, c.e., & prybutok, v.r. (2014) "investigating factors affecting social presence and user satisfaction with mobile instant messaging", computers in human behavior, 36(2): 453-459. osterwalder, a., pigneur, y., & tucci, c. (2005) "clarifying business models: origins, present, and future of the concept", communications of the association for information systems, 15(1): 1-25. australasian journal of information systems madlberger & matook 2017, vol 21, research article theorizing e-commerce business models 23 parker, c.m., bellucci, e., torlina, l., fraunholz, b., & zutshi, a. (2014) "conceptualising how smes incorporate green content in their websites", australasian journal of information systems, 18(3): 137-160. pateli, a., & giaglis, g. (2004) "a research framework for analysing ebusiness models", european journal of information systems, 13(4): 302-314. pavlou, p.a., & chai, l. (2002) "what drives electronic commerce across cultures? a crosscultural empirical investigation of the theory of planned behavior", journal of electronic commerce research, 3(4): 240-253. pavlou, p.a., & fygenson, m. (2006) "understanding and predicting electronic commerce adoption: an extension of the theory of planned behavior", mis quarterly, 30(1): 115143. piotrowicz, w., & cuthbertson, r. (2014) "introduction to the special issue information technology in retail: toward omnichannel retailing", international journal of electronic commerce, 18(4): 5-16. pookulangara, s., hawley, j., & xiao, g. (2011) "explaining consumers’ channel-switching behavior using the theory of planned behavior", journal of retailing and consumer services, 18(4): 311-321. prasarnphanich, p., & gillenson, m.l. (2003) "the hybrid clicks and bricks business model", communications of the acm, 46(12): 178-185. roca, j.c., chiu, c.-m., & martínez, f.j. (2006) "understanding e-learning continuance intention: an extension of the technology acceptance model", international journal of human-computer studies, 64(8): 683-696. rohm, a.j., & swaminathan, v. (2004) "a typology of online shoppers based on shopping motivations", journal of business research, 57(7): 748-757. schmid, b., & lindemann, m. (1998) "elements of a reference model for electronic markets", 31st hawaii international conference on system sciences (hicss-35), hawaii, hi, 193201. setia, p., boyer, k.k., calantone, r.j., & voorhees, c.m. (2007) "complementary effects of value and satisfaction on customer's behavioral intentions", academy of management annual meeting, 1-6. shadish, w.r., cook, t.d., & campbell, d.t. (2002) experimental and quasi-experimental designs for generalized causal inference, hougthon-mifflin, boston, ma. slade, p., & van akkeren, j. (2002) "business on-line? an empirical study of factors leading to the adoption of internet technologies by australian smes", australasian journal of information systems, 10(1): 50-64. snn-adaptive-intelligence (2013) bayesbuilder: a tool for constructing and testing bayesian networks. available at: www.snn.ru.nl statista (2016) retail e-commerce sales worldwide from 2014 to 2019. available at: http://www.statista.com/statistics/379046/worldwide-retail-e-commerce-sales/ [accessed 30 june 2016]. statistik austria (2016) online-shopper 2015 . available at: www.statistik.at/wcm/idc/idcplg?idcservice=get_pdf_file&revisionselectionmet hod=latestreleased&ddocname=022211 [accessed 25 april 2016]. steinfield, c., bouwman, h., & adelaar, t. (2002) "the dynamics of click-and-mortar eletronic commerce: opportunities and management strategies", international journal of electronic commerce, 7(1): 93-119. australasian journal of information systems madlberger & matook 2017, vol 21, research article theorizing e-commerce business models 24 su, b.-c. (2007) "consumer e-tailer choice strategies at on-line shopping comparison sites", international journal of electronic commerce, 11(3): 135-159. swinyard, w., & smith, s. (2003) " why people (don’t) shop online: a lifestyle study of the internet consumer", psychology and marketing, 20(7): 567-597. tate, m., hope, b., & coker, b. (2005) "the buywell way: seven essential practices of a highly successful multi-channel e-tailer", australasian journal of information systems, 12(2): 147-163. teece, d.j. (2010) "business models, business strategy and innovation", long range planning: international journal of strategic management, 43(2-3): 172-194. van doorn, j., lemon, k.n., mittal, v., nass, s., pick, d., pirner, p., & verhoef, p.c. (2010) "customer engagement behavior: theoretical foundations and research directions", journal of service research, 13(3): 253-266. veit, d., clemons, e., benlian, a., buxmann, p., hess, t., kundisch, d., leimeister, j.m., loos, p., & spann, m. (2014) "business models an information systems research agenda", business and information systems engineering research notes, 6(1): 45-53. wallace, d.w., giese, j.l., & johnson, j.l. (2004) "customer retailer loyalty in the context of multiple channel strategies", journal of retailing, 80(4): 249-263. wang, w.-t., wang, y.-s., & liu, e.-r. (2016) "the stickiness intention of group-buying websites: the integration of the commitment-trust theory and e-commerce success model", information and management, 53): 625-642. wang, y.-s. (2003) "determinants of user acceptance of internet banking: an empirical study", international journal of service industry management, 14(5): 501-519. wang, y.s. (2008) "assessing e-commerce systems success: a respecification and validation of the delone and mclean model of is success ", information systems journal, 18(5): 529-557. wiesel, t., pauwels, k., & arts, j. (2011) "practice prize paper—marketing's profit impact: quantifying online and offline funnel progression", marketing science, 30(4): 604611. worldbank (2016) internet users (per 100 people) . available at: http://data.worldbank.org/indicator/it.net.user.p2?end=2015&start=2010 [accessed 2016/11/29]. xenos, m. (2004) "prediction and assessment of student behaviour in open and distance education in computers using bayesian networks", computers and education, 43(4): 345-359. ypma, a., westerdijk, m., de walle, h.-j., & heskes, t. (2004) "bayesian techniques for modelling dynamic patterns", 4th european symposium on intelligent technologies, hybrid systems and their implementation on smart adaptive systems, aachen, germany. zeithaml, v.a. (2000) "service quality, profitability, and the economic worth of customers: what we know and what we need to learn", journal of the academy of marketing science, 28(1): 67-85. zhang, x., prybutok, v., & huang, a. (2006) "an empirical study of factors affecting eservice satisfaction", human systems management, 25(4): 279-291. zheng, z., & pavlou, p.a. (2010) "toward a causal interpretation from observational data: a new bayesian networks method for structural models with latent variables", information systems research, 21(2): 365-391. zikmund, w.g. (2003) business research methods, thomson/south-western, mason, oh. australasian journal of information systems madlberger & matook 2017, vol 21, research article theorizing e-commerce business models 25 zott, c., & amit, r. (2007) "business model design and the performance of entrepreneurial firms", organization science, 18(2): 181-199. zott, c., & amit, r. (2010) "designing your future business model: an activity system perspective", long range planning, 43(2010): 216-226. zott, c., amit, r., & massa, l. (2011) "the business model: recent developments and future research", journal of management information systems, 37(4): 1019-1042. zumpe, s., & madlberger, m. (2007a) "serving online customers better in the tourism industry", information age, 2007(11/12): 48-52. zumpe, s., & madlberger, m. (2007b) "a transaction-based framework for business models in electronic commerce", 11th pacific asia conference on information systems (pacis 2007), auckland, new zealand, 106-121. copyright: © 2017 madlberger & matook. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ the australasian journal of information systems (ajis) began in 1993 and was based on the simple realisation that there was a need for an academic ‘outlet’ for is research in australia australasian journal of information systems volume 16 number 2 2010 5 effect of presentation flaw attribution on website quality, trust, and abandonment andrea everard university of delaware scott mccoy college of william & mary abstract using scenario-based experiments we examine how users’ perceptions of online store quality and trustworthiness are affected by their attribution of website flaws. the attribution of online store website flaws can be internal (i.e., an action taken by the online store's site developer) or external (i.e., an action taken on the part of the site’s service provider) to the website. perceived quality of the online store was found to be lower for users who attributed the flaws to internal factors. findings also showed that the presence of a flaw, regardless of whether it was attributed to an internal or external condition, negatively affected the users’ level of trust in the website. introduction several characteristics of e-commerce websites can affect consumers’ impressions of them, for example, ease of navigation, the appearance of online advertisements, clearly stated privacy and security policies, up-to-date links, and available contact information. everard and galletta (2005) looked at various types of presentation flaws, such as typographical errors, incompleteness of the site, stylistic issues, and delay and how these flaws affect users’ impressions of online stores, their trust in these stores and their intention to purchase from these stores. building on this work, we attempt to explore the impact of attribution of the flaws on website quality, trust, and decisions to abandon the website. although past research has identified various factors that influence users’ online experiences (mccoy, everard, and loiacono, 2009; mccoy, everard, polak, and galletta, 2008). none to our knowledge, has studied the effect of presentation flaw attribution on users’ impressions of a site. this paper reports the empirical results of scenario-based experiments that look at how attributing on-line store website flaws either to external or internal factors affects people’s perceptions of the quality of and trust in the website. we propose that depending on whether a flaw is attributed to an internal (i.e., an action by the on-line store’s site developer) factor or an external (i.e., an action on the part of the site’s service provider) factor the effect on the user’s attitudes and beliefs about a website will differ. our focus is on users’ first time experiences with the website when users have no prior knowledge of the store or its products. the following research questions are addressed in this paper: (1) is a user’s perception of quality of an online store affected by the attribution (internal or external) made of a presentation flaw? (2) is a australasian journal of information systems volume 16 number 2 2010 6 user’s trust in an online store affected by the attribution (internal or external) made of a presentation flaw? and (3) given the attribution of the presentation flaw, what action (either leave the online store or remain on it) would the user take? we provide a sound background to this study by examining several areas of relevant research. in looking at the effect of users’ attributions of presentation flaws on the perceived quality of and trust in the on-line store, we integrate research on attribution theory, trust, impression formation, and presentation flaws to form the study’s theoretical foundation. the remainder of the paper proceeds as follows: the next section reports on prior research that is relevant to the current study, which leads to the development of hypotheses. the research methodology and experimental design used to test the hypotheses as well as the data collection procedure are then described. the analysis is then detailed and the results reported and discussed. finally, we present the limitations of this research as well as potential research extensions to this study. background attribution theory attribution is an explanation that is given to account for the occurrence of an event. attribution theory helps to understand how people explain things. in the case of this research, the explanation that someone provides for an event can either be attributed to something internal or something external. if it is believed that a factor internal to a person or an organization is the cause of an event, this is deemed an internal attribution. an external attribution assigns causality to a factor outside of the person or firm. in such a case, the event is deemed to be motivated by an outside force. a person’s behavior will be affected by whether the event is attributed to an internal or external factor. there are three main dimensions of attributions (brewer & crano, 1994). first, there is locus of causality which deals with whether the outcome is attributed to an internal or external cause. the second dimension is stability and focuses on whether the cause is stable or changeable. the third and most important dimension that individuals consider is controllability (anderson, 1991). individuals tend to be held more responsible for events and outcomes that they are believed to be able to control than for those that are not deemed controllable. trust trust refers to a “positive belief about the perceived reliability of, dependability of, and confidence in a person, object, or process” (fogg & tseng, 1999, p.81). trust has been defined in various ways, often depending on the context in which it appears; it has been recognized that trust is difficult to define and to measure (corritore et al., 2003; grabner-kräuter & kaluscha, 2003; ), and only exists in an uncertain and risky environment (mayer, davis, & schoorman, 1995). some definitions have focused on the element of risk involved (johnson-george & swap, 1982), others on the vulnerability of one of the parties concerned (boss, 1978), while still others on the presence of a significant motivation or incentive at stake (kee & knox, 1970). the view of trust that we focus on in this research is trust in the online store via its website. initial trust is a particular form of trust that develops at the initial point of contact with a vendor. it is especially relevant in electronic commerce environments, where users frequently access unfamiliar online stores. this type of trust occurs when a relationship is initiated with an unfamiliar trustee, that is where parties lack credible information about each other and no prior ties exist (serva, benamati, & fuller, 2005; mcknight, choudhury, & kacmar, 2002; mcknight, cummings, & chervany, 1998). australasian journal of information systems volume 16 number 2 2010 7 users’ impressions of an online store’s website may lead to differing perceptions of the quality of the online store. since users form impressions of the online store from the signals obtained from the website, it is in the best interest of the store to manage these signals. to expand on this critical concept, we now review the impression formation literature. impression formation the manner in which information is presented to users helps them form perceptions of a website. impression formation refers to how this initial information serves as the basis of users’ perceptions. because a user’s initial point of contact with the vendor is often through the website, it is crucial that the user is presented with an initial favorable image (anderson, 1965; asch, 1946; fiske, 1980). “the seven-second rule” refers to the time period after which a prospective customer will disengage and be “turned off” for good from a website (cotlier, 2001) should the initial impression not be positive. it is therefore imperative to focus on the user’s initial contact. research on impression formation considers the way people perceive others as a process by which an integrated impression is formed from stimulus information. early models of impression formation (asch, 1946; anderson, 1965) assume that when an individual is presented with information about a previously unknown or unfamiliar person, the individual creates a mental slot in which information is received and processed and thus impressions of others are formed. the negativity effect in impression formation refers to the fact that negative attributes are weighted more heavily in an individual’s overall impression than positive ones. also, items with extreme evaluative meaning are given more weight than neutral items in an individual’s impression formation. negative and extreme factors are deemed, in general, more novel and unusual and as such, are given more attention. traits that are paid more attention to are usually weighted more heavily in a user’s impression formation (fiske, 1980). rather than concentrate on specific attributes or basic traits, individuals focus on person or object types. a “type” is a set of traits or attributes that, based on historical evidence, are likely to be clustered together. for example, one would expect a professional website to evoke a sense of clarity and purpose, to be consistent across pages, to contain easily navigable pages, up to date information, and so forth. presentation flaws there is an abundance of practical advice on several sites on how to improve website design (for i.e., websitesthatsuck.com and useit.com) and many books on this same issue (for i.e., homepage usability: 50 websites deconstructed by nielsen & tahir (2001)). while trust can be enhanced by users’ perceptions of reliable and accurate information being supplied by the computer, flaws in the information provided may serve to destroy that trust (wright & marett, 2010); “virtually all researchers agree that computer errors damage credibility – at least to some extent” (fogg & tseng, 1999, p.82). a presentation flaw is any undesirable feature of a website that could interfere with reading or understanding its content or intent. such flaws include any features of the system that may make the system difficult to learn and remember, ineffective, and unpleasant to use. examples of flaws are poor aesthetics, confusing organization, difficulty in navigation, broken links, and pictures that do not load correctly. spelling errors can also be used to form negative impressions about competency and attention to detail (liu & ginther, 2001). flaws such as typographical errors, notation that is unfamiliar to or not easily understood by the users, and inconsistent or faulty formatting issues may result in a loss of the users’ trust. as in everard & galletta (2005), we focus on four presentation flaws, namely, (1) english usage and spelling errors, (2) non-loading pictures, (3) broken links and (4) delay. australasian journal of information systems volume 16 number 2 2010 8 english usage and spelling errors. english usage errors such as typographical, grammatical, and factual errors make up this set of flaws. spelling errors, according to molich and nielsen (1990) “distract users and make them suspect a generally poor quality of the system” (p.344). moreover, spelling errors can be used to form impressions about competency and attention to detail (liu & ginther, 2001). non-loading pictures and broken links. the perceived incompleteness of a website can be affected by non-loading pictures and broken links. these flaws can discourage a user’s confidence and trust. although incompleteness can be thought of as a temporary state, a site owner must use judgment to determine when to release a site to the public. allowing users to arrive at an incomplete site can be a risky venture and could lead to lasting, negative impressions. websites need to be maintained, as do their links, which are perishable and must be updated periodically. users are likely to be disappointed and to not return when sites go stale. obsolete links can shake the users’ confidence in the validity and timeliness of the content (lynch & horton, 2002). delay. the effects of delay over the world wide web have been thoroughly studied and it was found that users especially dislike delays that are associated with the information retrieval process and result in longer wait periods for sought after information (sears, jacko, & borella, 1997). the time that elapses between the user’s input and the computer’s response is called the system response time. increased levels of annoyance, frustration, and impatience were reported as system response time increased. users also rated their well-being as lower when delays in response time grew (kuhman, boucstein, schaefer, & alexander, 1987; schleifer, & amick, 1989). based on the prior research presented here, the following sections develop the hypotheses that will be empirically tested. attribution and perception of quality a user’s perception of the quality of a website that contains flaws may vary depending on whether the user believes that the flaws are a result of some issues that are external to the company or whether the website is flawed because of an issue that is internal to the firm. in this research we posit that users who are presented with a website which contains a presentation flaw (english errors, non-loading pictures, broken links, or delay) and who attribute that flaw to an issue internal to the online store, for example the online store’s developer’s negligence, will perceive the quality of the online store to be lower than if they attributed the presentation flaw to an issue external to the online store. in the first case, the user may feel that the flaw is present despite the ability of the firm to do something about it or to control it. if the flaw is attributable to some external force, the user may feel that the online store can do nothing to stop the flaw from occurring. from the user’s perspective, it is less desirable if the online store can control the presence of the flaw but does not than if the online store is not in a position to control the flaw. this leads to the first hypothesis: h1. an internal attribution of website presentation flaws will result in lower perceived quality of the online store than an external attribution of website presentation flaws. attribution and trust similarly, users’ attribution of presentation flaws can affect the level of trust that users place in the website. we hypothesize that an internal attribution of a flaw (for example, an online store’s developer incorrectly specifies the path to the picture file and hence no picture loads) will more negatively affect the user’s level of trust in an online store than a flaw which is attributed to something outside of the online store’s control. as with the user’s perception of quality of an online australasian journal of information systems volume 16 number 2 2010 9 store, if the user feels that the flaw was present in the online store’s website despite the firm’s ability to prevent it the user will place less trust in the site than if the user feels that the firm could not have done anything to control the presence of the flaw. therefore, we hypothesize: h2. an internal attribution of website presentation flaws will result in a lower level of trust in the online store than an external attribution of website presentation flaws. another question that this research is interested in answering is what action would the user take given the attribution of the presentation flaw. given the above hypotheses, we hypothesize that a user presented with an internal attribution condition will be more likely to leave the online store as we expect the user’s perception of quality (h1) and level of trust (h2) to be lower than in the external attribution condition. we also expect that users in the external attribution condition to more likely either continue to browse and buy from the store or to continue to browse being wary. we do not expect such users to leave the store. we also expect that users in the external attribution condition will consider that the presence of the presentation flaw is through the action of something not under the control of the online store and therefore users will perceive the quality of the online store less negatively than if the presentation flaw was due to issues controllable by the online store. therefore, we hypothesize: h3.users in the internal attribution condition will be more likely to leave the website than users in the external attribution condition, who will more likely continue to browse the store. research methodology scenario-based vignettes were created to conduct this research. in this experimental setting the vignettes were used to control the type of flaw which users were presented as well as whether the flaw was attributed to an internal or an external issue of the online store’s website. the use of vignettes is useful when trying to elucidate attitudes by personalizing a situation (couger, 1989) and when problems of internal validity (for example, subjects attempting to gain experimenter approval) (harrington, 1996) may exist. in the case of this research, the use of vignettes was especially useful as the experimenter was able to communicate the attribution of the flaw to the subjects. without the use of the vignettes, it would have been more difficult to ascertain the attribution of the flaw made by the subject. a similar methodology was used in gattiker and kelley (1999) who looked at ethical computer-related behavior. a 4x2 factorial design was used: (1) english usage and spelling errors, (2) non-loading pictures, (3) broken links, and (4) delay are the four different types of flaws that are investigated; attribution of the flaw was either internal or external. operationalization of variables a vignette experiment was designed to collect the required data for this research. the vignettes each included the presence of a flaw (either, english usage and spelling errors, non-loading pictures, broken links or delay) and this flaw was either attributed to a cause internal to the online store’s website or an issue external to the website. the vignettes were identical in their composition except for the type of flaw depicted and its attribution. each of the four flaw types was in turn attributed to internal issues and to external issues, resulting in eight different versions of the vignettes. that is, all combinations of flaw-attribution were created in the fully-factorial design. the vignette described a scenario in which a user (by the gender-neutral name of chris) is faced with a flaw and is informed whether the flaw occurred because of something internal to the website or external to the website. australasian journal of information systems volume 16 number 2 2010 10 flaw types and attribution as described above, four different types of flaws were represented in the vignettes (see appendix a). in the vignette which includes english usage and spelling errors, the scenario reads: “chris notices that some words are misspelled and that there are numerous grammatical errors. some of those problems were quite obvious, making the site look fairly bad.” chris then contacts the store to report the problem. in the internal attribution condition the store’s reply “made it clear that the errors were the fault of the in-house on-line store’s site developer who had not spent enough time proofing the site’s text before the site went live.” in the external attribution condition the store’s reply “made it clear that, although the site was secure and customer accounts had not been compromised, the store’s internet provider suffered a security breach and a hacker defaced the site and introduced the misspellings. the spelling errors were a result of that and had nothing to do with the on-line store’s management or the professionals who built the site.” in the scenarios involving non-loading pictures, chris “finds the desired item and starts to read the description and information about the product. chris notices that some of the pictures on the page being viewed do not appear.” as in the english usage and spelling errors scenario, chris reports the problem; from the store’s reply, it is clear that in the internal attribution condition “the in-house online store’s site developer had incorrectly specified the path to the picture file and had not spent enough time proofing and testing the site before it went live,” while in the external attribution condition the cause lay with the service provider eol that was “using content filtering and hence some pictures had been blocked from viewing by mistake. although the on-line store’s site developer had correctly specified the path to the picture file, the problem was occurring due to the service provider’s decision to use software to block certain types of content.” similar to the non-loading picture flaw, the scenario built around the broken link flaw also depicts chris as finding the desired item and clicking on the link to access additional information but instead an error message appears. chris then contacts the store and receives a reply. in the internal attribution condition, “the in-house on-line store’s site developer had incorrectly specified the file path and had not spent enough time proofing and testing the site before it went live.” in the external attribution condition, the problem’s cause lay with the service provider that was “using content filtering and hence some information had been blocked from viewing by mistake. although the on-line store’s site developer had correctly specified the file path, the problem was occurring due to the service provider’s decision to use software to block certain types of content.” the final flaw, delay, was illustrated by chris clicking on a link to go to another page and noticing a longer than usual delay. in the internal attribution condition the scenario then continues “because this happens repeatedly, chris, who recently learned a little about developing sites, examines the page more closely and finds that the in-house on-line store’s site developer had not properly tested the site. the graphics on the page were from large graphic files that were not compressed at all, which is a problem that is easily remedied. in fact, chris experiments a little and finds that two elementary changes reduced the delay by 95%.” in the external attribution condition, chris contacts the store and finds out that the problem lies with the service provider. “network problems are the cause of the delay. the waiting period between the loading of pages has nothing to do with the on-line store.” dependent variables action after each of the scenarios the respondent was asked what action s/he would take given the scenario just presented. the options were: (1) continue to browse and buy the item, if it’s what you want, at the right price; (2) continue but be wary of the online store; or (3) leave the online store and browse another online store with similar merchandise. finally, a question that serves as the manipulation australasian journal of information systems volume 16 number 2 2010 11 check was asked in order to verify that the attribution manipulation (internal and external) was successful. the following dependent variables were measured using a paper-based instrument which respondents filled out after having read one of the eight scenarios. perceived quality the perceived quality of the website was measured using a set of four items adapted from the sitequal instrument developed by yoo and donthu (2001). the sitequal instrument was designed to measure the perceived quality of an internet site (7-point scales) (alpha = .87). trust in the online store’s website trust in the online store website was measured by a set of six items adapted from jarvenpaa and tractinsky (1999). these six items measure the online store’s website’s trustworthiness (7-point scales) (alpha = .74). subjects 259 volunteer undergraduate business students from two u.s. universities participated in the study. each participant was randomly assigned to read one of the eight scenarios and to complete the subsequent questionnaire. subjects were given the opportunity to participate during one of their class periods, and an incentive in the form of extra credit points to those who completed the experiment was offered to stimulate interest in, and completion of, the task. results and discussion data were coded and hypotheses were tested using spss version 12. tests of the hypotheses are reviewed below, along with the results. a manipulation check was included in the instrument in order to ensure that respondents perceived the experimental variation in the attribution treatment. a chisquare test indicates that respondents understood the attribution treatment (chi-square = 39.910, 1 d.f., p = .000). h1 predicted that users would perceive the quality of the online store as lower if they were presented with a presentation flaw that was attributed to a cause internal to the online store, for example an error by the in-house online store’s site developer. this hypothesis was supported. the analysis of site quality produced a significant effect for attribution (t=2.045, p=.042). subjects who were exposed to the internal attribution condition perceived the quality of the online store to be worse (4.62) than subjects who were exposed to the external attribution condition (4.33) 1 (table 1). attribution n mean standard deviation perceived quality internal 130 4.62 1.22 external 129 4.33 1.03 table 1: means of perception of quality of online store’s website 1 for the instrument used the lower the score the higher the perceived quality. a score of 1 would indicate the highest perception of quality, while a score of 7 indicates the lowest perception of quality. australasian journal of information systems volume 16 number 2 2010 12 this result is consistent with what one would expect given attribution theory. just as individuals are judged more responsible for events whose outcomes they can control, the website is held more responsible if it is deemed it could have done something about the flaws, that is, that the presence of the flaw was in fact under the control of the website. as can be seen from the results, users perceived the quality of the website to be better when an external attribution was used to explain the flaw. one explanation is that users felt that though the flaw was present, it was attributed to something outside of the control of the online store. h2 predicted that users’ level of trust would be lower for those exposed to an internal attribution of website presentation flaws than those exposed to an external attribution of the flaws. this hypothesis was not supported. the analysis of trust produced an insignificant effect for attribution (t=.-363, p=.717), where subjects who were exposed to the internal attribution condition indicated a 4.29 level of trust and subjects who were exposed to the external attribution condition indicated a 4.33 level of trust (table 2). attribution n mean standard deviation trust internal 130 4.29 .896 external 129 4.33 .976 table 2: means of trust of quality of online store’s website although the results of this analysis are in the predicted direction, the non-significant result is contrary to our expectation. one explanation could be that web users, who are increasingly informed and knowledgeable about the web, have no tolerance for any type of presentation flaw. essentially, the above reported results indicate that no matter whether the attribution of the flaw is internal or external the respondents’ trust is decreased. h3 predicted that users in the internal attribution condition would be more likely to leave the website than users in the external attribution condition who, it was predicted, would more likely continue to browse the store. this hypothesis was not supported. the analysis produced a marginally significant effect for action (p=.007), however this effect was in the opposite direction of our predictions. for subjects who were exposed to the internal attribution condition, 71 chose to keep browsing the site while 56 of them chose to leave the site 2 . for the subjects who were exposed to an external attribution of the flaw 46 chose to remain on the site and continue browsing while 77 chose to leave the site. 2 respondents were given a choice of three possible actions: (1) “continue to browse and buy the item, if it’s what you want, at the right price,” (2) “continue but be wary of the online store” or (3) “leave that online store and browse another online store with similar merchandise”. in the analysis for h3, we are interested in whether the user continues to browse the site or leaves the site. as a result, in the statistical tests we add the number of users who chose either to continue to browse and maybe even buy from the site and those who would continue to browse but be cautious; this number represents those users that remain on the site. we compared this aggregate number with the number of users who chose to leave the site (action #3). australasian journal of information systems volume 16 number 2 2010 13 because the above results are contrary to our expectations, and because we wanted to gain further insight into the possible reasons for the findings, some exploratory analysis was conducted. the manipulation check, although confirmed to have elucidated the intended treatment, can serve as the subject’s perception of the attribution of the flaw. we now report on how the perception of the attribution of the flaw affects the users’ action, that is whether to continue to browse and even buy from the site if the product is what the user is looking for, to continue to browse but to be wary, or to leave the site altogether. crosstabs were run with the scores of the manipulation check and the action taken. it is interesting to note that although half of the subjects were clearly exposed to the internal attribution condition and the other half to the external attribution condition, the respondents’ perceptions of the attribution were heavily skewed on the internal attribution side. of the 250 respondents, 179 perceived the flaw to be caused by an internal cause while 71 of the respondents felt the flaw was due to something external to the online store. the analysis produced a significant effect for action (p=.074), with subjects who attributed the flaw to an internal cause to be more likely to leave the site (98) than to continue to browse and perhaps buy (37) or to continue to browse but to be wary (44). of the subjects who perceived the flaw to be caused by an issue external to the online store 35 chose to leave the site altogether, 24 chose to continue browsing and perhaps buy the product from the site, and 12 chose to continue to browse but be wary. these results are similar to the above results in that no matter what the attribution a majority of the respondents are more likely to leave the site when a flaw is present. limitations and conclusions as in most studies, there are several limitations that should be kept in mind when considering the results of this study. first, the subjects were college students, and the results might not generalize to the rest of the population. fortunately, voich (1995) found students to be particularly representative of values and beliefs of individuals employed in a variety of occupations. in addition, given that college students comprise a significant portion of online shoppers and are not expected to react differently when encountering flaws, we believe that students are suitable participants for such a study. another potential limitation in this study is the seldom-used vignette experimental technique. although this type of research instrument is perhaps more common in psychology research, is researchers have also made use of it (for example, gattiker and kelley, 1999). a laboratory or field experiment involving sites that participants navigate in their typical context may produce different results than the vignette experiments used here. the findings suggest that the attribution that users make of presentation flaws has a significant effects on users’ perceptions of the online store’s quality and the action that they choose to take, whether it be to remain on the site or to leave. in terms of users’ perception of an online store’s quality, users seem to hold websites more responsible if the flaw is attributed to something internal; they are more forgiving when the attribution is an external one. this supports the notion that website owners should take the necessary steps to have a complete website free of flaws. although statistically the effect of attribution on users’ trust in the online store was insignificant, the results from this study demonstrate that regardless of whether users attribute a flaw to an internal or an external cause the presence of a flaw decreases the users’ level of trust in the website. therefore, website owners must continually review their sites to make sure they are free of all flaws, regardless of their source. australasian journal of information systems volume 16 number 2 2010 14 when asked whether the user would remain on the site and continue browsing or leave the site, the results from this study were contrary to our expectations. more respondents exposed to the external attribution condition chose to leave compared with those exposed to the internal attribution condition. although this result was unexpected, we can understand it given the informed and knowledgeable web users who expect websites to be free from flaws, and who have innumerable options from which to choose from when browsing the world wide web. an interesting future direction of this research is to investigate the impact of the individual flaws on the dependent variables. in other words, do specific flaws have an impact different than grouping them together as was done in this research? this is a promising avenue of future research. references anderson, c.a. (1991) “how people think about causes: examination of the typical phenomenal organization of attributions for success and failure”, social cognition, vol 9, pp 295-329. anderson, norman h. (1965) “averaging versus adding as a stimulus-combination rule in impression formation”, journal of experimental psychology, vol 70 no 4, pp 394-400. asch, solomon e. (1946) “forming impressions of personality”, journal of abnormal and social psychology, vol 41, pp 258-290. boss, r.w. (1978) “trust and managerial problem solving revisited”, group and organization studies, vol 3, pp 331-342. brewer, marilynn b. and crano, william d. (1994) social psychology, st paul, mn: west publishing company. corritore, cl; kracher, b; wiedenbeck, s (2003) “on line trust: concepts, evolving themes, a model”, international journal of human computer studies, vol 58, pp 737-758. cotlier, moira (2001) “electronic catalogs: judging a site by its home page”, catalog age, may 1. couger, d.j., (1989) “preparing is students to deal with ethical issues,” mis quarterly, vol 13 no 2, pp 211-218. everard, andrea and galletta, dennis (2005) “how presentation flaws affect perceived site quality, trust, and intentions to purchase from an online store”, journal of management information systems, special issue on hci in mis, vol 22 no 3. fiske, s.t. (1980) “attention and weight in person perception: the impact of negative and extreme behavior”, journal of personality and social psychology, vol 38, pp 889-906. fogg, b.j. and tseng, hsiang (1999) “the elements of computer credibility”, proceedings of chi’99 (pittsburgh, may 15-20), pp 80-87, new york: acm press. gattiker, urs, e. and kelley, helen (1999) “morality and computers: attitudes and differences in moral judgments”, information systems research, vol 10 no 3, pp 233-254. grabner-kräuter, sonja and kaluscha, ewald a. (2003) “empirical research in online trust: a review and critical assessment”, international journal of human-computer studies, vol 58 no 6, pp 783-812. harrington, s.j. (1996) “the effects of code of ethics and personal denial of responsibility on computer abuse judgment and intentions,” mis quarterly, vol 20 no 3, pp 257-278. jarvenpaa, sirkka l. and tractinsky, noam (1999) “consumer trust in an internet store: a crosscultural validation”, journal of computer-mediated communication, vol 5 no 2. http://portal.acm.org/citation.cfm?id=941185&dl=guide&coll=guide&cfid=35930392&cftoken=68757451 http://portal.acm.org/citation.cfm?id=941185&dl=guide&coll=guide&cfid=35930392&cftoken=68757451 http://portal.acm.org/citation.cfm?id=941185&dl=guide&coll=guide&cfid=35930392&cftoken=68757451 australasian journal of information systems volume 16 number 2 2010 15 johnson-george, c. and swap, w. (1982) “measurement of specific interpersonal trust: construction and validation of a scale to access trust in a specific other”, journal of personality and social psychology, vol 43, pp 1306-1317. kee, h.w. and knox, r.e. (1970) “conceptual and methodological considerations in the study of trust”, journal of conflict resolution, vol 14, pp 357-366. kuhman, w., boucsein, w., schaefer, f., and alexander, j. (1987) “experimental investigation of psychophysiological stress-reactions induced by different system response times in humancomputer interaction”, ergonomics, vol 30 no 6, pp 933-943. liu, yuliang and ginther, dean (2001) “managing impression formation in computer-mediated communication”, educause quarterly, november 3, pp 50-54. lynch, patrick j. & horton, sarah (2002) web style guide, new haven, ct: yale university press mayer, r. c., davis, j. h. and f. d. schoorman, (1995), “an integrative model of organizational trust”, academy of management review, vol 20 no 3, pp 709-734. mccoy, scott, everard, andrea and eleanor loiacono (2009) “online ads in familiar and unfamiliar sites: effects on perceived website quality and intention to reuse,” information systems journal (isj), 19:4, 437-458. mccoy, scott, everard, andrea, polak, peter, and galletta, dennis (2008) “an experimental study of antecedents and consequences of online ad intrusiveness,” international journal of humancomputer interaction (ijhci), 24:7, 672-699. mcknight, h., choudhury, v., & kacmar, c. (2002). developing and validating trust measures for ecommerce: an integrative typology. information systems research, 13, 334–359. mcknight, d. harrison, cummings, l.l. and chervany, n.l. (1998) “initial trust formation in new organizational relationships,” academy of management review, vol 23 no 3, pp 473-490. molich, rolf and nielsen, jakob (1990) “improving a human-computer dialogue”, communications of the acm, vol 33 no 3, pp 338-348. nielsen, jakob and tahir, marie (2001) homepage usability: 50 websites deconstructed, indianapolis, il: new riders publishing. schleifer, l. m., and amick, b. c. (1989) “system repines time and method of pay: stress effects in computer-based tasks”, international journal of human-computer interaction, vol 1 no1, pp 23-39. sears, andrew, jacko, julie a., and borella, michael s. (1997) “internet delay effects: how users perceive quality, organization, and ease of use of information”, chi proceedings, pp 353 354. serva, m., benamati, s., & fuller, m. (2005). trustworthiness in b2c e-commerce: an examination of alternative models. data base for advances in information systems, 36(3), 89–118. voich, d., (1995) comparative empirical analysis of cultural values and perceptions of political economy issues, westport, ct: praeger publications. wright, ryan t., marett, kent (2010) the influence of experiential and dispositional factors in phishing: an empirical investigation of the deceived,” journal of management information systems, 27:1, 273-303.yoo, boonghee and donthu, naveen (2001) “developing a scale to measure the perceived quality of internet shopping sites (sitequal)”, quarterly journal of electronic commerce, vol 2 no 1, pp 31-47. australasian journal of information systems volume 16 number 2 2010 16 appendix a scenarios 1. spelling errors internal chris is browsing a new web retail store, and finds it to be well-organized, easy to navigate and aesthetically pleasing. chris finds the desired item and starts to read the description and information about the product. chris notices that some words are misspelled and that there are numerous grammatical errors. some of those problems were quite obvious, making the site look fairly bad. chris was kind enough to contact the store to report the problems, and the reply made it clear that the errors were the fault of the in-house on-line store’s site developer who had not spent enough time proofing the site’s text before the site went live. if you were chris, what would you do next? continue to browse and buy the item, if it’s what you want, at the right price. continue but be wary of the on-line store. leave that on-line store and browse another on-line store with similar merchandise. other: __________________________________________________________ based on the above scenario, who do you blame for the spelling errors? should the blame be directed internally or externally to the on-line store? internal  external  2. spelling errors – external chris is browsing a new web retail store, and finds it to be well-organized, easy to navigate and aesthetically pleasing. chris finds the desired item and starts to read the description and information about the product. chris notices that some words are misspelled and that there are numerous grammatical errors. some of those problems were quite obvious, making the site look fairly bad. chris was kind enough to contact the store to report the problems, and the reply made it clear that, although the site was secure and customer accounts had not been compromised, the store’s internet provider suffered a security breach and a hacker defaced the site and introduced the misspellings. the spelling errors were a result of that and had nothing to do with the on-line store’s management or the professionals who built the site. if you were chris, what would you do next? continue to browse and buy the item, if it’s what you want, at the right price. continue but be wary of the on-line store. leave that on-line store and browse another on-line store with similar merchandise. other: __________________________________________________________ based on the above scenario, who do you blame for the spelling errors? should the blame be directed internally or externally to the on-line store? internal  external  australasian journal of information systems volume 16 number 2 2010 17 3. non-loading pictures internal chris is browsing a new web retail store and finds it to be well-organized, easy to navigate and aesthetically pleasing. chris finds the desired item and starts to read the description and information about the product. chris notices that some of the pictures on the page being viewed do not appear. after contacting the store to report the problem, it became clear that the in-house on-line store’s site developer had incorrectly specified the path to the picture file and had not spent enough time proofing and testing the site before it went live. hence, the pictures could not be seen. if you were chris, what would you do next? continue to browse and buy the item, if it’s what you want, at the right price. continue but be wary of the on-line store. leave that on-line store and browse another on-line store with similar merchandise. other: __________________________________________________________ based on the above scenario, who do you blame for the non-loading pictures? should the blame be directed internally or externally to the on-line store? internal  external  4. non-loading pictures external chris is browsing a web retail store using a new service provider called everyone on line (eol). the site is well-organized, easy to navigate and aesthetically pleasing. chris finds the desired item and starts to read the description and information about the product. chris notices that some of the pictures on the page being viewed do not appear. after contacting the store to report the problem, it became clear that eol is using content filtering and hence some pictures had been blocked from viewing by mistake. although the on-line store’s site developer had correctly specified the path to the picture file, the problem was occurring due to the service provider’s decision to use software to block certain types of content. if you were chris, what would you do next? continue to browse and buy the item, if it’s what you want, at the right price. continue but be wary of the on-line store. leave that on-line store and browse another on-line store with similar merchandise. other: __________________________________________________________ based on the above scenario, who do you blame for the non-loading pictures? should the blame be directed internally or externally to the on-line store? internal  external  australasian journal of information systems volume 16 number 2 2010 18 5. broken links internal chris is browsing a new web retail store, and finds it to be well-organized, easy to navigate and aesthetically pleasing. chris finds the desired item and clicks on the link but an error message appears instead of the appropriate page. after contacting the store to report the problem, it became clear that the in-house on-line store’s site developer had incorrectly specified the file path and had not spent enough time proofing and testing the site before it went live. hence, the broken links prevented chris from finding the desired item. if you were chris, what would you do next? continue to browse and buy the item, if it’s what you want, at the right price. continue but be wary of the on-line store. leave that on-line store and browse another on-line store with similar merchandise. other: __________________________________________________________ based on the above scenario, who do you blame for the broken links? should the blame be directed internally or externally to the on-line store? internal  external  6. broken links external chris is browsing a new web retail store using a new service provider called everyone on line (eol). the site is well-organized, easy to navigate and aesthetically pleasing. chris finds the desired item and clicks on the link but an error message appears instead of the appropriate page. after contacting the store to report the problem, it became clear that eol is using content filtering and hence some information had been blocked from viewing by mistake. although the on-line store’s site developer had correctly specified the file path, the problem was occurring due to the service provider’s decision to use software to block certain types of content. if you were chris, what would you do next? continue to browse and buy the item, if it’s what you want, at the right price. continue but be wary of the on-line store. leave that on-line store and browse another on-line store with similar merchandise. other: __________________________________________________________ based on the above scenario, who do you blame for the broken links? should the blame be directed internally or externally to the on-line store? internal  external  7. delay internal chris is browsing a new web retail store, and finds it to be well-organized, easy to navigate and aesthetically pleasing. whenever chris clicks on a link to go to another page, there is a longer than australasian journal of information systems volume 16 number 2 2010 19 usual delay. that is, there is a noticeable wait before the page is complete. because this happens repeatedly, chris, who recently learned a little about developing sites, examines the page more closely and finds that the in-house on-line store’s site developer had not properly tested the site. the graphics on the page were from large graphic files that were not compressed at all, which is a problem that is easily remedied. in fact, chris experiments a little and finds that two elementary changes reduced the delay by 95%. if you were chris, what would you do next? continue to browse and buy the item, if it’s what you want, at the right price. continue but be wary of the on-line store. leave that on-line store and browse another on-line store with similar merchandise. other: __________________________________________________________ based on the above scenario, who do you blame for the delay? should the blame be directed internally or externally to the on-line store? internal  external  8. delay – external chris is browsing a web retail store using a new, free service called everybody on line (eol). the site is well-organized, easy to navigate and aesthetically pleasing. whenever chris clicks on a link to go to another page, there is a longer than usual delay. that is, there is a noticeable wait before the page is complete. because this happens repeatedly, chris contacts the store to report the problem. it then becomes clear that the problem lies with eol. network problems are the cause of the delay. the waiting period between the loading of pages has nothing to do with the on-line store. if you were chris, what would you do next? continue to browse and buy the item, if it’s what you want, at the right price. continue but be wary of the on-line store. leave that on-line store and browse another on-line store with similar merchandise. other: __________________________________________________________ based on the above scenario, who do you blame for the delay? should the blame be directed internally or externally to the on-line store? internal  external  paper title (paper title style) australasian journal of information systems north & north 2016, vol 20, research article virtual reality & immersive components a comparative study of sense of presence of traditional virtual reality and immersive environments max m. north visualization & simulation research center information systems department-coles college of business computer science department-college of computing and software engineering kennesaw state university, kennesaw, ga 30144, usa max@kennesaw.edu sarah north visualization & simulation research center information systems department-coles college of business computer science department-college of computing and software engineering kennesaw state university, kennesaw, ga 30144, usa snorth@kennesaw.edu abstract user experience is a critical component in the development of virtual environment systems and applications. to create an effective and efficient user experience for virtual environment systems, it is imperative to conduct empirical studies to explore factors that contribute to a heightened sense of presence, or immersion, for users, who increasingly demand an experience in a virtual environment that seems as real as possible. the primary hypothesis of this study is that participants will experience higher levels of presence in an immersive visualization environment (ive) than in a traditional virtual reality environment (tvre). the empirical methodology consists of two different experiments designed, developed, and implemented using seventy participants. a simulated virtual airplane scenario was created and conducted for each participant. after completion of each simulation session, each participant was given a questionnaire that measured the participant’s sense of presence in either virtual or real environments. analysis of the collected data indicated that there was a statistically significant difference in the level of presence experienced by participants between the tvre and ive systems. the primary conclusion is that participants experienced a higher sense of presence in the ive experiment when compared with the tvre experiment, indicating significantly more natural and richer user experience through the ive system’s interface. keywords: sense of presence; immersive environments; virtual reality 1 introduction over the last few decades, the importance of user experience in virtual environment systems has become paramount, and this element is recognized as a critical component of the design and development of these systems. consequently, there is an increased need to conduct empirical studies to explore factors that create a higher sense of presence, or immersion, for users. the main goal is to ultimately develop a scientific body of knowledge and a framework to assist researchers in the development of future virtual environment systems and applications that provide real and natural interface and user experiences. there are different aspects that make virtual reality feel as real as possible to users. one wellknown aspect of virtual reality is a sense of presence in the virtual environment. it is often thought of as the sense of “being there.” in the 1990s, a few theoretical articles exploring virtual presence were published in the journal of presence: teleoperators and virtual environments, published by the massachusetts institute of technology, mit (pausch, proffitt & williams, 1997; sheridan, 2000; hendrix & barfield, 1995; slater, usoh and steed, 1994; slater, 1999; witmer & singer, 1998; zeltzer, 1992). since that time, more and more researchers have been harvesting the power of presence, which in turn has contributed new knowledge to extend a 1 australasian journal of information systems north & north 2016, vol 20, research article virtual reality & immersive components deeper understanding of different the causes and nature of this sense of presence; these studies include post-traumatic stress disorder treatment (hodges et al., 1999; rizzo et al., 2010), virtual reality therapy (north, north & coble, 1996a), fear of flight phobia treatment (north, north & coble, 1996b), arachnophobia treatment (bouchard, 2006), treatment for burn pain (hoffman et al., 2000; palyo, schopmeyer & mcquaid, 2010), and vr based therapy for the treatment for impotence and premature ejaculation (optale et al., 1998); influence of personality on the sense of presence experienced in anxiety triggering virtual environments (jurnet & maldonado, 2010); measuring sense of presence and user characteristics to predict effective training in an online simulated virtual environment (de leo et al., 2014); beyond presence: how holistic experience drives training and education (chertoff & schatz, 2014). although development of research has progressed through the years, our understanding of the theoretical aspects of presence remains limited, and more experimentation is needed in this area. we reiterate from our earlier research in the field along with many pioneers here: there is a great need to develop a scientific body of knowledge and a framework to assist researchers in the development of efficient and effective virtual environment applications, and within this framework to rigorously investigate the sense of presence. this current research has been designed and implemented to explore the sense of presence and its relationship in real and virtual environments. to do so, authors focus on investigating sense of presence comparing two different virtual environments: traditional virtual reality environment (helmet-based system with a head-tracking device) an innovative virtual reality environment using an immersive environment (spherical-based virtual environment). 1.1 sense of presence how is “presence” defined? in 1980, marvin minsky coined the term “telepresence” and described telepresence as “refer[ing] to the phenomenon that a human operator develops a sense of being physically present at a remote location through interaction with the system’s human interface” (coelho et al., 2006, p. 2). over time, the study of “telepresence” yielded to the general study of “presence,” since telepresence is more commonly used with teleoperations. over time, many articles and journals defined presence differently, but all had a similar general meaning. in this research paper, the main definition of presence that will be used is the awareness or state of the mind of being in an environment, either real or virtual (usoh et al., 1999; nunez & blake, 2001). experiencing a sense of presence can happen whenever a person mentally feels that he or she is present in a situation. for example, when reading a book, some people may feel what the characters are feeling, and they become wrapped up in the book— they may feel as though they are watching a movie or have an even deeper connection with the text. the traditional way of thinking of presence stems from the writings of rené descartes and the german philosopher immanuel kant. they made a clear distinction between objects located in the mind and objects located outside of the mind (sheridan, 1999). one of the first theoretical articles on presence described it as the feeling of “realness, vividness, and feeling very much alive” (fontaine, 1992, p.41). there was also a proposed division of the definition of presence, separating it into subjective presence (a conscious state-of-mind) and behavioral presence (an unconscious state-of-mind) (slater, 2002; slater, 2009). a more recent definition of presence is “how realistically participants respond to the environment as well as their subjective sense of being in the place depicted by the virtual environment” (yu et al., 2012, p.76). yet other researchers, sanchez-vives & slater, (2005) define the concept of presence as a phenomenon of behaving and feeling as if participants are in the virtual world created by computer displays. in their article, they argue that study of presence is essential to neuroscientists, that in turn it might aid the study of perception and consciousness. as an example of the power of this phenomenon, juan et al. (2006) show how the sense of presence played a dominant role in an augmented reality system for the treatment of acrophobia using immersive photography. most participants in this study showed a higher subjective sense of presence with immersive photography. 2 australasian journal of information systems north & north 2016, vol 20, research article virtual reality & immersive components while there are a good number of applied researches in virtual environment field that directly utilize sense of presence and ultimately contribute to the comprehension of this phenomenon, there seems to be a shortfall of in-depth studies into the pure theoretical aspect of sense of presence. in addition, because of the complex nature of the sense of presence, past investigations have been very focused and limited to specific studies, leaving unanswered questions. this study expands on some of those unanswered questions: what is presence exactly? does presence go beyond the physical attributes and into the cognitive mind? what is virtual reality, and how does a person’s sense of presence change in the virtual environment? what factors contribute to the sense of presence in an environment, either virtual or real? what is the best method or device used for measuring presence? therefore, one of the main goals of current and future research is to study the sense of presence by proposing a series of testable propositions and by using significant experimentations to build theories that in turn will assist future researchers and application developers. 1.2 factors affecting presence there are many factors that contribute to one’s sense of presence. one classic study in particular from sheridan at mit summarizes and gives some additional enlightenment on these factors. sheridan proposes that there are four major variables that affect a person’s immersion in an environment. the first is the quantity of information that gets through to the participant. this depends on whether the participant is paying attention and on the level of visual or auditory distractions in the environment. the second variable is sensor position and orientation, which in the context of head-tracking devices deals with the degree of corresponding visual feedback. the third is the change of relative location of objects in response to static feedback (such as gravity) and direct manipulation commands (sheridan, 1999). for example, a study by usoh et al. (1999) indicated that naïve subjects in an immersive virtual environment experience a higher subjective sense of presence when they move by walking in place than when they “push-button fly.” the fourth variable to be considered is an active imagination in suppressing disbelief. this factor is the most difficult to control and generally relies on a large enough participant base to even out differences. steuer (1992) proposed three different factors involved with a person’s presence in virtual reality. those factors are vividness, interactivity, and the influence of the user’s characteristics. vividness is the representational richness of a mediated environment defined by its formal features. interactivity can be defined as the extent to which users can participate in modifying the form and content of a mediated environment in real time. getting a person more engaged in what is being seen can have a major effect on how a person perceives the virtual world. the final factor steur talks about is the influence of a user’s characteristics. this derives from the individual differences in the sense of presence when subjects are confronted with the same virtual environments. the individual’s personal experiences contribute to this factor. because there was no direct manipulation in the airplane simulation, steur’s second factor, interactivity, was not used in either experiment. however, care was taken to keep a standard across participants so that the other variables introduced above, especially auditory and visual distractions, as well as the head-tracking device, were constants for the data sample. sheridan focused predominately on environmental factors, while another study focusing on therapy through virtual reality focused much more on the psychological factors affecting immersion. psotka and davison found that “immersion was most affected by how claustrophobic one is. the more claustrophobic you are the more often you think about the other person(s) in the real world with you there. how immersed one feels appears to be determined by a complex set of physical components and affordances of the environment, as well as psychological processes that as yet are poorly understood” (psotka & davison, 1993, p.71). this study did not take into account any atypical psychological issues, but rather relied on averaging to represent the majority. this complex set of factors, both environmental and psychological, is what makes typical subjective measures of presence challenging. because this study was measuring sense 3 australasian journal of information systems north & north 2016, vol 20, research article virtual reality & immersive components of presence and not these variables themselves, as long as the variables were turned into constants as much as possible, the data sample was considered representative of the average. according to jurnet and maldonado (2010), who investigated the influence of personality and individual abilities on the sense of presence experienced in anxiety-triggering virtual environments, spatial intelligence has an influence on the sense of presence in an immersive virtual reality system. there is also a strong relationship between users’ personality characteristics and their sense of presence in a virtual environment. a ten-year critical review of empirical research of educational applications of virtual reality was conducted by mikropoulos and natsisl (2011). researchers reported multisensory interaction channels and visual representation within virtual reality as a dominant factor effecting sense of presence. 1.3 measuring presence there has been some discussion on the ways to measure presence. one way to measure presence is to observe how people physically react to stimulation; we can call this behavioral presence (witmer and singer, 1998; zeltzer, 1992; meehan et al., 2002; ijsselsteijn et al., 2002; north, 2002; riva et al., 2007; lee, chou & sun, 2015). behavioral presence focused mainly on the movements of the subjects and on spontaneous speech. a study headed by dillon et al. (2001) out of new cross london discussed some of the issues that arise when attempting to measure presence. physiological measures of presence have been proposed to be the most accurate and objective possible measurement. dillon notes that the “unifying idea behind this approach is that as presence within a displayed environment increases, physiological reaction will tend towards those that would be observed in a real environment. alternatively, these objective, continuous measures could provide additional information about viewing experiences which are not tapped by post-test subjective rating scales, which may be prone to demand characteristics and memory biases” (dillon et al., 2001, p.83). however, such measures may not be appropriate in cases when even a high amount of presence does not physically stimulate the subject, such as when a subject moves a chair in a virtual environment. after an in-depth study that compared heart-rate monitoring, skin-conductance monitoring, and survey methods, dillon concluded that physiological measures are an addition to subjective presence measures, but not a replacement (dillon et al., 2001). since this experiment was a simulation of simply being in an airplane, it was determined that physiological measures would not have been a very valuable addition to the subjective measures. presence is a subjective condition, and it has most commonly been measured using self-reports by administrating questionnaires either during or immediately after the experiment (juan et al., 2009). to properly measure presence, several questionnaires have been suggested (barfield et al., 1995; lombard & ditton, 1997; sheridan, 1996; slater et al., 1994; slater, 1999; june et al., 2009; north et al., 2002; lin et al., 2002; lee et al., 2015). based on the research briefly described above, particularly the questionnaire originally developed by slater et al. (1994), questionnaires that measured subjective self-reports were administered immediately after the completion of the simulation sessions for both ive and tvre experiments. 2 empirical research methodology 2.1 hypothesis, objectives, and terminology usage the primary hypothesis of this research is that participants will experience higher levels of presence in an immersive visualization environment (ive – a spherical, projected virtual environment) than in a traditional virtual reality environment (tvre – a helmet-based system with a head-tracking device). the first objective of this research was to design, configure, develop and implement an innovative ive. the second objective was to investigate the sense of presence induced in participants by the immersive visualization environment and compare it with the sense of presence felt by participants who experienced the traditional virtual reality environment. please note that for improved readability and comprehension, the terminology “immersive visualization environment” and “ive” will be used 4 australasian journal of information systems north & north 2016, vol 20, research article virtual reality & immersive components interchangeably and simultaneously throughout this paper, as will “traditional virtual reality environment” and “tvre”. 2.2 experimental design based on preceding comparative studies there are two main types of comparisons. the first type is the comparison of extreme opposites, like cave (cave automatic virtual environment, an immersive virtual reality with projectors are pointed to between three and six walls of a room-sized cube) and standard desktop computers (gruchalla, 2004; shim & kim, 2004; juan & perez, 2009). the second type refers to studies that utilize external factors such as field-of-view, stereoscopy, headtracking, sounds, and augmentation (hendrix & barfield, 1995; waterworth & waterworth, 2003; juan, calatrava, 2011; north & north, 2002; arino et al., 2014; lee, chou & sun, 2015). the design of this empirical research methodology is primarily based on this second class of comparisons, incorporating limited features from the first type. we use best practices of the studies mentioned in prior sections. 2.3 participants and site for tvre and ive experiments seventy volunteer college students ranging in age from twenty to forty-eight years old participated in the study. their mean age was twenty-eight years. they were recruited by advertisements on the university campuses. participants were counterbalanced and arbitrarily assigned to the following two experiments: • tvre experiment: traditional virtual reality environment (n=35), 23 male and 12 female. • ive experiment: immersive visualization environment (n=35), 25 male and 10 female. the site of both experiments was visualization & simulation research center at kennesaw state university. tvre and ive systems were implemented to work independently, but in the same facility. each experiment was conducted in a different time frame, so there was no communication or interaction between participants of each experiment until after the research was concluded. this arrangement allowed for monitoring with high validity the collected data through the individually administered questionnaires. 2.4 instrument used for tvre and ive experiments presence is a subjective condition most commonly measured using subjective self-reports by administrating questionnaires either during or immediately after the experiment. in both experiments, a streamlined version of the ucl presence questionnaire, also termed the subjective measure of presence (smop) and originally developed by slater et al. (1994), was used. this post-experience subjective measure of presence was administered immediately after each simulation for both experiments. participants were asked to provide ratings on an 11point likert scale (with 0 being the lowest and 10 being the highest) on two selected questions from the questionnaire—the subjective questions that were directly related to presence and correlated to the hypothesis of this study. the same questions were posed to participants of both experiments: • question 1-rate your sense of presence in the real world while experiencing the virtual world. • question 2-rate your sense of presence in the virtual world during the experiment. based on our prior experiences, questions were kept simple in order to allow participants to focus on their responses and report accurate subjective ratings. for this reason, we used “virtual world” term to denote to “tvre” or “ive” in each distinct experiment. all participants were given an explanation of the experiments and instructions about interaction protocols with the environments before actually experiencing the virtual environments. authors replied to all the participants’ questions and secured their signed consent forms for participating in experiments. 5 australasian journal of information systems north & north 2016, vol 20, research article virtual reality & immersive components 2.5 traditional virtual reality environment experiment (tvre) apparatus: the virtual reality system for this part of the study consisted of a pentium-based computer using a 6d head-tracking system and display. the virtual environments were created using the sense8tm virtual reality development software package and libraries. each session was executed using the virtual i-o glasses with a head-tracking device. the vr helmet provided a 360o field of view, which allows the participants to see the image at any direction they look. the virtual environment scene used was a moderately simple rendition of a windowseat passenger’s view of an airplane taking off and flying during bad weather (see figure 1). this environment was called traditional virtual reality environment or tvre. procedure: the participants individually were seated in an aircraft-like seat equipped with vibration capabilities which simulated engine vibration and flight turbulence. the participants could partially view their bodies. the subject listened to sounds such as the flight attendant speaking, engine noises, and noises due to turbulence. the only control the participants had over the simulation was through the head-mounted tracking system that responded to their head movements by moving the view in the virtual world. participants experienced thirty minutes in the airplane simulation before being administered the smop sense of presence questionnaire immediately afterward. the definition of presence, given beforehand, was “the sense of being there,” where “there” meant the particular environment referenced. participants rated the questions quantitatively on a scale of 0 to 10 (likert scale), with zero being equal to no sense of presence and ten being equal to complete immersion in the environment. there were no other rules imposed on how subjects could score their sense of presence. (a) (b) figure 1. depicts (a) a traditional virtual reality environment system and (b) a participant wearing a head-mounted display and head-tracking device immersed in the traditional virtual reality environment. 2.6 immersive visualization environment experiment (ive) apparatus: the device used for this part of the experiment was the immersive visualization environment (spherical-based virtual environment), also known as a dome-shaped system (see figures 2 and 3). this state-of-the-art equipment enhances the immersive virtual environment imaging with its four digital projectors and a large cylindrical screen. the particular immersive system used for the experiment is a spatially immersive visualization and features four digital projectors, omnifocus™ series 500 wide angle single lens projection system, and an 8ft by 10ft cylindrical screen with multi-channel visual display, coupled with integrated hardware or software warp and blend technology (see figure 3, a schematic configuration of the ive system). numerous variations of ive are designed and implemented at research and training laboratories in prime industries, such as boeing, bmw, lockheed martin, and in branches of the department of defense, army, air force, navy, national security agency, just to list a few. such installations have a wide range of applications, such as architectural design, flight simulation, military simulation, scientific visualization, experiential marketing, and industrial simulation (maxwell, griffith & finkelstein, 2014). 6 australasian journal of information systems north & north 2016, vol 20, research article virtual reality & immersive components (a) (b) figure 2. illustrates (a) an airplane in the immersive visualization environment system flying over terrain; and (b) several subjects engaged in the immersive visualization environment figure 3. a schematic configuration of the immersive visualization environment system procedure: participants individually were seated in an aircraft-like seat placed a specific distance away from the screen, providing the individual with a fully immersed environment, and cancelling out any outer disturbances. identical to the prior experiment, this seat had a vibration apparatus to correspond to the engine and flight turbulence. although the environment was controlled, the participants had the ability to freely move their body parts without the restrictions of a helmet or hand held devices. the simulation used here was exactly the same as that used with the tvre experiment. the participants could listen to sounds such as the flight attendant speaking, engine noises, and noises due to turbulence. it must be noted that the experiments were intentionally focused on visual and auditory stimuli, with limited interaction and use of directed manipulation by participants. participants could view the virtual world by just moving their heads in the desired direction. participants experienced thirty minutes in the airplane simulation and responded to the smop sense of presence questionnaire immediately afterward. the same definition of presence described in the prior experiment was given to participants of this experiment beforehand. participants individually rated the questions quantitatively on a scale of 0 to 10 (likert scale), with zero being equal to no sense of presence and ten being equal to complete immersion in the environment. there were no other rules imposed on how subjects could score their sense of presence. 3 results and conclusions data from the subjective measure of presence (smop) instrument, a post-experience subjective measure of presence administered immediately after each simulation for both tvre and ive experiments, were collected and analyzed using appropriate statistical methods described below. 7 australasian journal of information systems north & north 2016, vol 20, research article virtual reality & immersive components 3.1 data analysis of traditional virtual reality environment tvre the questionnaire (smop) was administered immediately after the experiment in the tvre. the smop consisted of 10 questions, of which only 2 were directly related to the hypothesis of this part of the study. each question had 11-point likert scale; participants would select a number between 0 and 10 indicating their level of sense of presence within the environment. the higher score indicates a higher sense of presence (with 0 being the lowest and 10 being the highest presence). graphically, figure 4 (a and b) illustrates the distribution of smop measures for rating the sense of presence in the real world while experiencing the virtual world in the tvre (q1), and for rating of the sense of presence in the virtual world during the experiment in the tvre (q2). the mean (m=6.42, sd=2.25) for question-1 in the tvre experiment was slightly higher than mean (m=6.29, sd=2.16) for question-2. however, a matched pair t-test (t=0.56, df=68, p<0.05) analysis applied to the scores given to both questions showed very strong evidence that there is no significant difference between the level of sense of presence of the real world while experiencing the virtual world when compared with sense of presence in the virtual world (see table 1). this indicates that the reported sense of presence in the real world and virtual world were almost equally divided (approximately equal). interestingly, the total means reported (total means of 12.71 = calculated mean of 6.42 for the real world scores+ mean of 6.29 for the virtual world scores) appear to be more than the expected 10 (expected 5 for the real world scores + 5 for the virtual world scores). this finding will be described further in the comparison section of tvre and ive experiments. (a) (b) figure 4. smop measures for (a) rating of the sense of presence in the real world while experiencing the virtual world in the tvre, and for (b) rating of the sense of presence in the virtual world during the experiment in the tvre. 0 2 4 6 8 1 2 3 4 5 6 7 8 9 10 n um be r o f p ar tic ip an ts q1-sense of presence in the real world while experiencing the virtual world tvre experiment sense of presence in the real world 0 5 10 1 2 3 4 5 6 7 8 9 10 n um be r o f p ar tic ip an ts q2-sense of presence in the virtual world during the experiment tvre experiment sense of presence in the virtual world 8 australasian journal of information systems north & north 2016, vol 20, research article virtual reality & immersive components tvre-traditional virtual reality environment q1sense of presence in the real world while experiencing the virtual world q2sense of presence in the virtual world during the experiment analysis (n=35) (n=35) (df=68) measure mean sd mean sd t p smop* 6.42 2.25 6.29 2.16 0.56 <0.05 *subjective measure of presence table 1. means and standard deviations of sense of presence scores (q1 and q2) for tvre, and matched pair t-test analysis. 3.2 data analysis of immersive visualization environment ive for the ive experiment, the same questionnaire (smop) was administered immediately after each session. graphically, figure 5 (a and b) exhibits the distribution of smop measures for (a) rating of the sense of presence in the real world while experiencing the virtual world in the ive, and for (b) rating of the sense of presence in the virtual world during the experiment in the ive. the mean (m=5.01, sd=1.66) for question-1 of ive was considerably lower than the mean (m=7.69, sd=1.10) for question-2. a matched pair t-test (t=7.95, df=68, p<0.05) analysis applied to the scores given to both questions showed very strong evidence that there is a significant difference between the level of sense of presence of the real world while experiencing the virtual world and the sense of presence in the virtual world during the ive experiment (see table 2). this indicates that the reported sense of presence in the real world and virtual world were not equally distributed. participants sensed a significantly higher sense of presence in the virtual world than in the real world during the experiment. similar to the tvre experiment, the total means (total means of 12.69 = calculated by mean of 7.69 for the real world scores + mean of 5.0 for the virtual world scores) appear to be more than the expected 10 (expected 5 for the real world + 5 for the virtual world scores). this finding will be described further in the comparison section below. (a) (b) figure 5. smop measures for (a) rating of the sense of presence in the real world while experiencing the virtual world in the ive, and for (b) rating of the sense of presence in the virtual world during the experiment in the ive. 0 5 10 1 2 3 4 5 6 7 8 9 10 n um be r o f p ar tic ip an ts q1-sense of presence in the real world while experiencing the the virtual world ive experiment sense of presence in the real world 0 5 10 15 1 2 3 4 5 6 7 8 9 10 n um be r o f p ar tic ip an ts q2-sense of presence in the virtual world during the experiment ive experiment sense of presence in the virtual world 9 australasian journal of information systems north & north 2016, vol 20, research article virtual reality & immersive components ive-immersive visualization environment q1sense of presence in the real world while experiencing the virtual world q2sense of presence in the virtual world during the experiment analysis (n=35) (n=35) (df=68) measure mean sd mean sd t p smop* 5.01 1.66 7.69 1.10 7.95 <0.05 table 2. means and standard deviations of sense of presence scores (q1 and q2) for ive, and matched pair t-test analysis. 3.3 comparison of tvre and ive experiments table 3 shows the results of smop measurements for question-1 and for question-2 of the tvre and ive experiments. in addition, this table illustrates the analysis of a comparison between each set of scores. a matched pair t-test (t=3.60, df=68, p≤0.05) provided strong evidence that there was a significant difference between the means of tvre and ive scores for question-1, indicating that the sense of presence in the real world while experiencing the ive was lower than in the tvre experiment. for question-2 scores of both experiments, a matched pair t-test (t=2.16, p≤0.05) provided strong evidence that there was a significant difference between the means of tvre and ive scores, indicating that the sense of presence in the virtual world during the experiment of the ive was higher than in the tvre experiment. based on these results, it is reasonable to state that the participants were much more immersed in the virtual world while using the ive environment. comparison of tvre-traditional virtual reality environment and ive-immersive visualization environment q1sense of presence in the real world while experiencing the virtual world analysis q2sense of presence in the virtual world during the experiment analysis tvre ive tvre ive (n=35) (n=35) (df=68) (n=35) (n=35) (df=68) measure mean sd mean sd t p mean sd mean sd t p smop* 6.42 2.25 5.01 1.66 3.60 ≤0.05 6.29 2.16 7.69 1.10 2.16 ≤0.05 *subjective measure of presence table 3. comparison of tvre-traditional virtual reality environment and ive-immersive visualization environment an interesting finding of this study is that data analysis indicated the total sense of presence scores for the virtual and real worlds in both experiments exceeded 100% (mean of trve of real world scores + mean of trve of virtual world scores, that is 12.71; mean of ive of real world scores + mean of ive of virtual world scores, that is 12.70) for both trve and ive experiments (see figure 6). while commonsense would indicate that participants’ total sense of presence in concurrent environments should not exceed 100%, or 10 on the rating scale in this study, the analysis of data indicated this value was approximately 120% for both experiments. these results could have many implications, all of which would need to be independently studied in the future research endeavors. 10 australasian journal of information systems north & north 2016, vol 20, research article virtual reality & immersive components figure 6. this graph shows the sense of presence in the real world while experiencing the virtual world, sense of presence in the virtual world during the experiment, and total sense of presence in both real and virtual worlds during experiments using the tvre and ive. 4 concise discussions, implications, and extended concluding inferences two experiments, one using a trve and one using an ive were designed, developed and implemented to investigate the sense of presence and to determine which environment would induce a higher sense of presence in the virtual world. to the best of our knowledge, this is the first study in which these two specific environments were compared for the sense of presence they induce. the analysis of collected data supported the hypothesis of this study: participants were more immersed in the virtual world while using the ive comparing with the tvre. as a general interpretation, it was also noted that the ive group indicated a smaller sd's than the tvre (please see table 3). this signifies less variance within ive group, which may have enhanced the response consensus for ive compared to tvre. the idea of response consensus coming out of sd might be connected to "focus" of response, making the implications proposed from ive stronger. furthermore, the data analysis indicated that participants’ total sense of presence in concurrent environments, either tvre or ive, exceeded 100% (10 on the rating scale in this study). this study has some limitations. first, the number of participants was limited to seventy, and they were only from college student population. second, the sense of presence was measured only by subjective self-reported measure using a limited number of questions from questionnaires. third, although the measurement questionnaires used in this study were validated and repeatedly tested in many other experiments, physiological measurements could certainly be useful supplements to further study the sense of presence. these findings of this study raise several possibilities for future research endeavors. for instance, research to determine why participants’ overall ratings of sense of presence varies so widely, why the total subjective degree of sense of presence in both environments is higher than 100%, or the make-up of individual factors within participants’ reported sense of presence. as clearly stated in the preceding experiments in this article, a limited, direct manipulation intentionally was incorporated, in order to keep the investigation focused on the effect of manageable visual and auditory stimuli on the participants’ sense of presence. however, the researchers will be gradually introducing an increasing variety of direct manipulation within 11 australasian journal of information systems north & north 2016, vol 20, research article virtual reality & immersive components the future studies. as a consequence, the questionnaire will be extended accordingly, and the scope of the studies will be broadened. while research in this particular field shows that subjective measurements positively correlate with objective measurement, physiological measurements—such as eye-tracking and eeg (electroencephalogram – noninvasive monitoring method to record electrical activity of the brain)—will be utilized during all stages of the future experiments. specifically, in addition to subjective data, physiological data from each session will be collected and used for evaluation purposes. for example, eye tracking will be used to evaluate the level of attention allocated to areas of interest, and eeg will be used to evaluate whether cognitive problem states occurred less frequently. as an example of the implications of this technology being investigated by the authors, the following learning aspect is briefly described: there is a growing gap between the pace of technical and scientific innovation and the speed with which humans can learn new concepts and knowledge. while the former is accelerating, the latter has thus far remained fairly constant (mikropoulos & natsisl, 2011; gilbert, 2001). consequently, this gap is more apparent with the younger generations of learners. new and stimulating innovative educational techniques, concepts and materials will have to be developed to address this imbalance. the direction of the research described in this article not only will improve user experiences, but ultimately has the potential to drastically improve the efficiency and effectiveness of innovative technologies used in a broad range of training environments and fields. clearly, an immersive environment is becoming an increasingly important factor to users interacting with any application. various kinds of displays for immersive environments have been developed, from small (720 x 480 head-mounted display that create the perception of a large screen) to the large displays such as imax, to induce an immersive feeling in viewers. furthermore, with the recent demand for “real-feeling” media, we see a new market for various types of displays being formed (e.g., new technology like google glass – an optical headmounted display that induces an immersive experience for users). all immersive virtual environment systems—small or large, simple or complex—can take advantage of improved interface design to advance a more natural user experience. further investigation into the sense of presence will need to be conducted in order to better understand this phenomenon. research of this kind enhances opportunities for the discovery and generation of new knowledge and theories, which in turn advances the efficiency and effectiveness of humancomputer interaction. acknowledgment this effort was supported by an equipment grant from the u.s. army research office (aro) and assistance from pslsamp program (state-wide grant by nsf–national science foundation). the content of this work does not reflect the position or policy of the aro or nsf and no official endorsement should be inferred. we sincerely thank the editor, kathy lynch, and reviewers for their professional reviews and constructive suggestions. references arino, j. j., juan, m. c., gil-gomez, j. a. & molla, r. (2014). a comparative study using autostereoscopic display with augmented and virtual reality, behaviour & information technology, 33 (6), 646-655. barfield, w., sheridan, t.b., zeltzer, d., & slater, m. (1995). presence and performance within virtual environments, in virtual environments and advanced interface design, barfield, w. & furness (eds.), london: oxford university press. bouchard, s., cote, s., st-jacques, j., robillard, g. & renaud, p. (2006). effectiveness of virtual reality exposure in the treatment of arachnophobia using 3d games, technol health care, 14(1), 19-27. 12 australasian journal of information systems north & north 2016, vol 20, research article virtual reality & immersive components chertoff, d. & schatz, s. (2014). beyond presence: how holistic experience drives training and education. in handbook of virtual environments (pp. 959-1000), hale, k.s. & stanney, k.m. (eds.), boca raton: crc press-taylor & francis. coelho, c., tichon, j., hine, t., wallis, g. & riva, g. (2006). media presence and inner presence: the sense of presence in virtual technologies, in from communication to presence: cognition, emotions and culture toward the ultimate communicative experience, riva, g., anquera, m.t., wiederhold, b.k. & mantovani, f. (eds.), 2-5. de leo, g., diggs, l.a., radici, e. & mastaglio, t.w. (2014). measuring sense of presence and user characteristics to predict effective training in an online simulated virtual environment. simulation in healthcare, the journal of the society for simulation in healthcare, 9(1), 1–6 dillon, c., keogh, e., freeman, j. & davidoff, j. (2001). presence: is your heart in it? proceedings of the 4th annual international workshop on presence, philadelphia, usa. fontaine, g. (1992). the experience of sense of presence in intercultural and international encounters, presence: teleoperators and virtual environments. 1(4), 482-489. gilbert, s.w. (2001). no moore’s law for learning, syllabus –technology implementation, 14 (8), 28. gruchalla, k. (2004). immersive well-path editing: investigating the added value of immersion. ieee virtual reality, 157-164. hendrix, c. & barfield, w. (1995). presence in virtual environments as a function of visual and auditory cues, vrais '95 proceedings of the virtual reality annual international symposium, 74-79. hodges, l.f., rothbaum, b.o., alarcon, r., ready, d., shahar, f., graap, k., pair, j., herbert, p., gotz, d., wills, d. & baltzell., d. (1999). virtual vietnam: a virtual environment for the treatment of vietnam war veterans with post-traumatic stress disorder, cyber psychology & behavior 2(1). hoffman, h.g., patterson, d.r. & carrougher, g. (2000). use of virtual reality for adjunctive treatment of adult burn pain during physical therapy: a controlled study, clinical journal of pain, 16(3), 244-250. ijsselsteijn,w.a., ridder, h., freeman, j. & avons, s.e. (2002). presence: concept, determinants, and measurement, proceedings of spie 3959, human vision and electronic imaging, 520. juan, m. c., banos, r., botella, c., perez, d., alcaniz, m. & monserrat, c. (2006). an augmented reality system for acrophobia: the sense of presence using immersive photography, presence: teleoperators and virtual environments, 15(4), 393-402. juan, m. c. & calatrava, j. (2011). an augmented reality system for the treatment of phobia to small animals viewed via an optical see-through hmd, comparison with a similar system viewed via a video see-through, international journal of human-computer interaction, 27(5), 436-449. juan, m. c. & perez, d. (2009). comparison of the levels of presence and anxiety in an acrophobic environment viewed via hmd or cave, presence: teleoperators and virtual environments, 18(3), 232-248. jurnet, a. & maldonado, j. (2010). influence of personality and individual abilities on the sense of presence experienced in anxiety triggering virtual environments, international journal of human-computer studies, 68(10), 788–801. lee, c., chou, c. & sun, t. (2015). evaluating presence for customer experience in a virtual environment: using a nuclear power plant as an example, human factors and ergonomics in manufacturing & service industries, 25(2), 484-499. 13 australasian journal of information systems north & north 2016, vol 20, research article virtual reality & immersive components lin, j.j.-w., duh, h.b.l., parker, d.e., abi-rached, h. & furness, t.a. (2002). effects of field of view on presence, enjoyment, memory, and simulator sickness in a virtual environment, proceedings of virtual reality, ieee, 164-171. lombard, m. & ditton, t. (1997). at the heart of it all: the concept of presence, journal of computer mediated communication, 3(2), 1-43. maxwell, d., griffith, t. & finkelstein, n.m. (2014). use of virtual worlds in the military services as part of blended learning strategy, in handbook of virtual environments (pp. 959-1000), hale, k.s. & stanney, k.m. (eds.), boca raton: crc press-taylor & francis. meehan, m., insko, b., whitton, m. & boorks, f.p. (2002). physiological measures of presence in stressful virtual environment, acm transaction on graphics, 21(3), 645-652. mikropoulos, t.a. & natsisl, a. (2011). educational virtual environments: a ten-year review of empirical research (1999–2009), computers & education, 56(3), 769–780. munro, a., patrey, j., biddle, e.s. & carroll, m. (2014).cognitive aspects of virtual environment design, in handbook of virtual environments (pp. 391-410), hale, k.s. & stanney, k.m. (eds.), boca raton: crc press-taylor & francis. north, m.s. (2002). a study of the sense of presence in virtual environment experiment, cisst ’02 international conference, 623-626. north, m. m., north, m. s. & coble, j. r., (1996a). virtual reality therapy, an innovative paradigm, ipi press. north, m. m., north, s. m. & coble, j. r. (1996b). application: psychotherapy, flight fear flees, cyberedge journal, 6(1), 8-10. nunez, d. & blake, e. (2001). cognitive presence as a unified concept of virtual reality effectiveness. proceedings of the 1st international conference on computer graphics, virtual reality and visualization, south africa 115-118. optale, g., munari, a., nasta, a., pianon, c., verde, j. & viggiano, j. (1998). a vr based therapy for the treatment of impotence and premature ejaculation, in virtual environments in clinical psychology and neuroscience, giuseppe riva, brenda k. wiederhold and enrico molinari (eds.), amsterdam: ios press. palyo, s. a., schopmeyer, k. a. & mcquaid, j. r. (2012). tele-pain management: use of videoconferencing technology in the delivery of an integrated cognitive–behavioral and physical therapy group intervention, psychological services, 9(2), 200-202. pausch, r., proffitt, d. & williams, g. (1997). quantifying immersion in virtual reality. proceedings of siggraph 97, computer graphics, 13-18. psotka, j. & davison, s. (1993). cognitive factors associated with immersion in virtual environments. proceedings of the intelligent computer-aided training and virtual environment technology, houston, tx. riva, g., mantovani, f., capideville, c.s., preziosa, a., morganti, f., villani, d., gaggioli, a., botella, c. & alcaniz, m. (2007). affective interactions using virtual reality: the link between presence and emotions, cyberpsychology & behavior, 10(1), 45-56. rizzo, a. s., difede, j., rothbaum, b.o., reger, g., spitalnick, j.s., cukor, j., & mclay, r. (2010). development and early evaluation of the virtual iraq/afghanistan exposure therapy system for combat-related ptsd. annals of the new york academy of science, 1208, 114-125. sanchez-vives, m.v. & slater, m. (2005). from presence to consciousness through virtual reality, nature reviews neuroscience, 6(4), 332–339. 14 australasian journal of information systems north & north 2016, vol 20, research article virtual reality & immersive components sheridan, t.b. (1996). further musings on the psychophysics of presence, presence: teleoperators and virtual environments, 5(2), 241-246. sheridan, t.b. (1999). descartes, heidegger, gibson, and god toward and eclectic ontology of presence, presence: teleoperators and virtual environments, 8(5), 551-559. sheridan, t.b. (2000). interaction, imagination and immersion: some research needs, proceedings of the acm symposium on virtual reality software and technology,1-7. shim, w. & kim, g. j. (2004). designing for presence and performance: the case of the virtual fish tank, presence: teleoperators and virtual environments, 12(4), 374-386. slater, m. (1999). measuring presence: a response to the witmer and singer presence questionnaire, presence: teleoperators and virtual environments, 8(5), 560-565. slater, m. (2009). place illusion and plausibility can lead to realistic behaviour in immersive virtual environments, philosophical trans. royal soc. london, 364(1535), 3549–3557. slater, m. (2002). presence and the sixth sense, presence: teleoperators and virtual environments, 11(4), 435-439. slater, m., usoh, m., & steed, a. (1994). depth of presence in virtual environments, presence: teleoperators and virtual environments, 3(2), 130-144. steuer, j. s. (1992). defining virtual reality: dimensions determining telepresence, journal of communication, 4, 73-93. usoh, m., arthur, k., whitton, m., bastos, r., steed, a., slater, m. & brooks, f. (1999). walking > walking-in-place > flying, in virtual environments, proceedings of the 26th annual conference on computer graphics and interactive techniques, 359-364. waterworth, j.a. & waterworth, e.l. (2003). being and time: judged presence and duration as a function of media form. presence: teleoperators and virtual environments, 12(5), 495-513. witmer, b.g. & singer m.j. (1998). measuring presence in virtual environments: a presence questionnaire, presence: teleoperators and virtual environments, 7(3), 225-250. yu, i., mortensen, j., khanna, p., spanlang, b. & slater, m. (2012). visual realism enhances realistic response in an immersive virtual environment, ieee computer graphics & applications, 32(6), 36-45. zeltzer, d. (1992). autonomy, interaction and presence, presence: teleoperators and virtual environments, 1(1), 109-112. copyright: © 2016 north & north. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. 15 http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 1.1 sense of presence 1.2 factors affecting presence 1.3 measuring presence 2 empirical research methodology 2.1 hypothesis, objectives, and terminology usage 2.2 experimental design based on preceding comparative studies 2.3 participants and site for tvre and ive experiments 2.4 instrument used for tvre and ive experiments 2.5 traditional virtual reality environment experiment (tvre) 2.6 immersive visualization environment experiment (ive) 3 results and conclusions 3.1 data analysis of traditional virtual reality environment tvre 3.2 data analysis of immersive visualization environment ive 3.3 comparison of tvre and ive experiments 4 concise discussions, implications, and extended concluding inferences microsoft word ajis_18(3)_v10.docx australasian journal of information systems volume 18 number 3 2014 331 mobile learning anytime, anywhere: what are our students doing? angela murphy australian digital futures institute university of southern queensland toowoomba, australia angela.murphy@usq.edu.au helen farley australian digital futures institute university of southern queensland toowoomba, australia helen.farley@usq.edu.au michael lane school of management and enterprise university of southern queensland toowoomba, australia michael.lane@usq.edu.au abdul hafeez-baig school of management and enterprise university of southern queensland toowoomba, australia abdul.hafeez-baig@usq.edu.au brad carter school of agricultural, computational and environmental sciences university of southern queensland toowoomba, australia brad.carter@usq.edu.au abstract recent developments in mobile technologies have provided unique opportunities for learning and teaching. this paper reports on recent research undertaken at a regional australian university in order to understand how higher education students are using mobile devices to support their learning. a survey instrument was developed and deployed and the data collected analysed quantitatively. upon analysis, these data demonstrate that students are predominantly using laptop computers to support their learning, but their use of smart phones and tablets are also used for a number of specific learning activities. further analysis indicates that in spite of the limitations in the formal university infrastructure, many students would like to use their mobile devices for formal learning as well as informal learning. keywords: mobile learning, m-learning, smart technologies, student usage, higher education introduction recent research suggests that mobile technologies offer unique opportunities for students to become more engaged with learning activities beyond the boundaries of the learning management system australasian journal of information systems volume 18 number 3 2014 332 (matias & wolf, 2013). the functionality of these devices is continuously enhanced through the inclusion of features from established technologies such as personal digital assistants (pda), portable media players, gps navigation, digital cameras and ebook readers (alley & gardiner, 2012). this has resulted in devices that enable the development of ubiquitous or seamless learning environments that combine real-world and virtual resources. as a result, many educational researchers are exploring the potential for mobile technologies to enhance student learning, with mobile learning becoming one of the fastest growing areas within the field of icts in education (pegrum, oakley & faulkner, 2013). if mobile technologies are to be effectively used for education delivery, a key question becomes how do we best use mobile technologies for teaching and learning? in addressing this, the first step is to explore how students are already using mobile technologies to support their learning. the research presented in this paper investigates the extent to which students currently have access to mobile technologies and whether they are using these to support their learning. this research progresses previous work conducted into student ownership of mobile devices (e.g. oliver & goerke, 2007; mcneill, diao, gosper, 2011), by taking into account the rapid evolution of these technologies and shift in relative popularity of different devices, with smart phones and tablets dominating the market in more recent times. these devices have greater functionality and enable interactivity, thereby enhancing their potential affordances for both formal and informal learning. the data collected for this research will be used to inform the next phase of this research project which is to develop a mobile learning evaluation framework (mlef). the mlef project is part of an australian government-funded collaborative research network program led by the university of southern queensland (usq) with the australian national university (anu) and the university of south australia (unisa). researchers from a range of disciplines at these three universities are working together to develop a framework that will support institutional leaders and educators in higher education institutions with providing sustainable mobile learning opportunities to students. as illustrated in figure 1, the framework is being developed across four levels within the higher education environment: 1) pedagogical (from the student perspective), 2) pedagogical (from the teacher perspective), 3) technical and 4) organisational (murphy & farley, 2012). the trends, expectations and challenges experienced by stakeholders at each of these levels will be explored prior to the development of the framework. the data obtained during this study will be used to inform the second level of the framework. australasian journal of information systems volume 18 number 3 2014 333 figure 1: four levels forming the foundation of the mobile learning evaluation framework (adapted from murphy & farley, 2012) literature review over the past ten years, a number of pilot studies have been conducted across a number of sectors to investigate the impact of mobile technologies on learning and teaching (e.g. elias, 2011; wong, 2012). one of the most consistent conclusions of these studies is that there remain a number of barriers that influence the adoption of mobile learning initiatives, both at an institutional and at a user level. higher education institutions are cautious about investing in programs to provide students with mobile devices for learning, due to the rapidly changing nature of mobile technologies (alrasheedi & capretz, 2013). bring your own device (byod) refers to the use of a student’s own mobile technologies for learning and has been identified as a viable and cost-effective alternative to institutions supplying technologies for learning. it brings the added advantage of individuals being more comfortable and competent in using their own devices, therefore enhancing the potential for mobile learning adoption (shim, mittleman, welke, french & guo, 2013). a thorough understanding of how students are already using their mobile devices to support learning could lead to the more effective and sustainable deployment of mobile learning initiatives across the higher education sector. though ict infrastructure, staff and student culture, levels of technical support, appropriate skillsets of staff and the ability to adjust pedagogical approaches to teaching and learning are important considerations when planning for the deployment of learning technologies, the student voice is also important and is frequently overlooked (gosper, woo, muir, dudley, & nakazawa, 2007). this research seeks to address this deficiency but placing emphasis on the student voice. in 2005 and again in 2007, oliver and goerke surveyed some 413 and 290 undergraduate students respectively at an australian university to determine their levels of mobile device ownership. in both cohorts, just under half of the students owned laptop computers. interestingly in 2005, some 8 percent of students owned ‘handheld computers’, elsewhere described as pdas. by the time of the next survey, this percentage had decreased slightly. mobile phones were the most popular devices with nearly 97 percent of both cohorts owning them. even though the apple iphone was released in 2007 and smart phones were becoming more visible in the marketplace, no distinction was made in the survey between feature phones and smart phones. tablets were absent from the technology landscape with mp3 players and ipods being the only other technologies investigated. ownership levels were reasonably high with 40 percent of students owning one in 2005, jumping to some 70 percent in 2007 (oliver & goercke, australasian journal of information systems volume 18 number 3 2014 334 2007). though the authors did not specifically investigate whether or not students used their devices to support their learning, they did determine that students did use them for instant messaging (17.7 percent in 2005, 12.0 percent in 2007), accessing blogs (79.3 percent and 70.2 percent), and listening to podcasts (93.4 percent and 78.5 percent) (oliver & goercke, 2007). a study conducted by kennedy, judd, churchward, gray, and krause in 2008 questioned the widely held assumption that students had sufficient access to technology and digital literacy levels to maximise their learning using technology. the majority of students in this study had unrestricted access to mobile phones (96 percent), desktop computers (90 percent), mp3 players (69 percent) and laptop computers (63 percent). learning activities were mostly undertaken on desktop or laptop computers, with mobile phones reserved for voice calls and text messages. again, no distinction was made between smartphones and feature phones. a more recent study found that student ownership of laptop computers had increased with most in the sample owning a laptop computer. (mcneill & diao, 2011). interestingly, these students left their laptop computers at home rather than bring them to university due to the weight of the devices. also, though most students had phones that connected to the internet, only one student had access to a smartphone and that was owned by a family member. the low levels of smartphone ownership were attributed to the high cost of purchasing and using the devices. most students only used the basic functions of mobile phones including making phone calls and sending text messages with few accessing the internet. kobus, rietveld and van ommeren (2013) surveyed 3132 students from a dutch university in the netherlands in november, 2011. their study investigated student ownership and on-campus use of laptops, tablets and smartphones and found that at that stage, only 11 percent of students owned or shared a tablet computer yet 68 percent owned a smartphone. their research explored attitudes towards making laptop computers mandatory for use at university and found that 87 percent of students found laptop computers were too cumbersome to carry, supporting mcneill and diao’s (2011) conclusions. while these studies provide important insights into the uptake of mobile devices by university students, there remains a need to obtain information that is both applicable to the australian higher education context and is up-to-date with respect to recent rapid advances in mobile technology. further, there was very little research conducted into how students used mobile technologies to support their learning. this research seeks to address these issues. research method a quantitative survey was designed in april 2013 consisting of 28 closed and one open form question. it was divided into four sections: 1) student demographics; 2) the availability and quality of internet access; 3) ownership and access to mobile devices; and 4) usage of mobile devices to support learning. participants were asked about their access and use of both mobile and tethered technologies such as desktop and laptop computers to enable comparison. the research instrument was initially piloted on a small sample of students and a few minor revisions were made. two additional closed questions about the types of operating systems on tablets and smartphones were included in the revised survey instrument. the survey was administered using the online survey tool, qualtrics. questions about the learning activities engaged in by students were filtered according to the devices that students either owned or had the use of. this ensured that students only completed questions about technologies that they owned or actively used. the survey was piloted during march and april 2013. course examiners from 13 online courses at usq disseminated the link to their students for the pilot. four of usq’s five faculties were represented: the faculties of sciences, business and law, arts and the faculty of education. students were invited via email or via course discussion forums to participate. after refinement of the survey, a link was placed on the university announcement portal and was available to students during june and july 2013. participation was voluntary and no incentives were offered to students. australasian journal of information systems volume 18 number 3 2014 335 a total of 186 potential participants accessed the online survey and after removal of unusable responses, a sample of 100 participants was retained. of the total sample, 37 completed responses were obtained during the survey pilot and 63 during the second data collection stage. the majority of the sample represented australian residents (87 percent) and the remainder were international students (13 percent). both on-campus (53 percent) and distance students (46 percent) were equally represented. undergraduate students made up 78 percent of the sample and 22 percent were postgraduate students. a third of the sample consisted of first year students (35 percent). the sample consisted of more female (63 percent) than male (37 percent) participants and the age of students ranged from 17 to 61 with a mean age of 33 (sd=10.53). furthermore, most of the participants were employed in addition to studying (63 percent), working a minimum of 4 hours and a maximum of 80 hours a week (m=26, sd=18.52). research findings ownership and use of mobile technologies for learning purposes survey participants were presented with a list of mobile technologies and asked whether they owned the device, had access to the device or neither owned nor had access to the device. later, students were asked questions about how they used the devices they owned or had access to. figure 2 indicates student ownership of mobile and tethered computing technologies. as illustrated, 87 percent owned and 8 percent used a smartphone, similarly 88 percent owned a laptop and 6 percent used one. half of the participants owned (49 percent) or used (7 percent) an mp3 player; slightly fewer than half owned (46 percent) a tablet computer. an additional 21 percent of students used tablet computers resulting in a total of 67 percent of students having a tablet device at their disposal. ownership of ebook readers (17 percent), netbook computers (15 percent) and feature phones (21 percent) was less common. figure 2: ownership or access to mobile technologies further analysis revealed that students owned or used on average five (sd = 1.28) of the mobile technologies on the list, and the minimum number of technologies owned by each student was two. half of students who owned a smartphone (45, 52 percent) also owned a tablet computer. the four students who did not own or use a smartphone all had a feature phone and one had a tablet computer. of the six students who did not own or use a laptop, all had a desktop computer and three had a netbook, two had a tablet computer and all had a smartphone. during the second data collection stage, a question australasian journal of information systems volume 18 number 3 2014 336 was added to ask about the type of operating system participants had on their smartphones or tablet computers. students who owned smartphones either had apple ios operating systems (35, 58 percent) or android systems (21, 35 percent). three students owned a phone with a windows operating system (5 percent). those who owned tablet computers mostly had ios operating systems, i.e. they owned ipads (25, 56 percent), 24 percent (11) had windows operating systems and 20 percent (9) had android systems. participants were requested to indicate which of the technologies they owned or used, were used to support their learning. they were informed that this could refer to activities including accessing course materials, participating in discussion forums, communicating with other students or lecturers, or searching for information for their studies. unsurprisingly, the dominant technology used to support learning was the laptop computer with 85 percent of respondents using them for this purpose. tablet computers (72 percent) and smartphones (64 percent) were also used extensively by students to support their learning. technologies such as mp3 players (23 percent), ebook readers (30 percent) and feature phones (17 percent) were used less often for learning purposes. half of the students who owned netbook computers (50 percent) used them to support their learning. this suggests that lecturers could consider incorporating mobile learning initiatives into learning settings without potentially disadvantaging students without mobile technologies as most, if not all students own or have access to one or more mobile device. furthermore, as smartphones or tablet computers are favoured by students as tools to support learning, there is scope to incorporate more sophisticated learning activities within classroom environments. these may include quizzes and polls which enable students to engage interactively with the learning content and the lecturer. frequency and location of use of mobile technologies for learning all participants who owned or used each device were asked how frequently they used them to support their learning activities (see figure 3). laptop computers were used most frequently by students to support learning activities with 77 percent using them at least daily. smartphones were the second most frequently used device with 52 percent using them on a daily basis to support learning, just surpassing the desktop computer (47 percent). only some 42 percent of tablet computer owners were using them on a daily basis to support their learning, although a further 26 percent were using them for this purpose a few times a week. this suggests that tablet computers are being used more frequently than desktops computers over a period of a week. australasian journal of information systems volume 18 number 3 2014 337 figure 3: frequency of use of mobile technologies for learning there is some debate in the literature (sharples, taylor, & vavoula, 2005) as to whether students really do learn while physically moving between places. to explore this issue, participants were asked to indicate on a range of options, all of the locations where they used each of the technologies they owned or used specifically for learning. they were also encouraged to enter other locations if the location was not otherwise listed. the desktop computer was excluded in the analysis of the results as it is not mobile and the feature phone was excluded due to the low number of students who used this device for learning. figure 4 shows that the home is the primary location for use of larger as well as easily portable mobile technologies such as smartphones (81 percent), ebook readers (82 percent) and tablet computers (82 percent). although a surprisingly large proportion of students use smartphones (66 percent) and tablet computers at university (57 percent). netbook computers (57 percent) are also more likely to be used at university than laptop computers (47 percent), most probably due to their size. australasian journal of information systems volume 18 number 3 2014 338 figure 4: use of mobile technologies for learning: by location a large number of students are engaging in learning activities using their mobile technologies in locations other than at home or at university. a majority (67 percent) are using their smartphones to learn while travelling as a passenger in a vehicle, and 42 percent are using their tablet computers in this context. smartphones are also being used extensively in other non-traditional locations for learning activities such as at work (47 percent), while walking (44 percent) and in public locations such as parks (41 percent). mp3 players are popular for learning while walking (44 percent). additional open comments provided by students included locations such as on holiday or on work trips away from home, in transit between daily activities such as waiting in a doctor’s office, at a child’s school or sports event or at a friend’s house. a comment from a participant which signifies the transitory nature of mobile learning was “everywhere, it’s attached to me.” these findings indicate that students do take advantage of physical mobility between locations as well as locations outside designated study spaces to engage in learning. therefore lecturers who provide students with podcasts, video recordings or other multimedia content that can be downloaded and accessed from a mobile device are well placed to capture the attention of students seeking to engage with their studies outside of formal study locations and times. using mobile devices to support learning very few previous studies have attempted to discern the type of learning activities that students use mobile technologies for, particularly smartphones and tablet devices. to this end, students were provided with a list of potential learning activities and requested to indicate which they conducted on each of the mobile technologies that they owned or used. they were also provided with an open entry field to describe additional activities not on the list. students could also indicate if they did not participate in the activity. unsurprisingly, laptop computers were used for the majority of learning activities, except for taking photos or videos (23 percent), which suggests that this is still the dominant technology used by most students for learning purposes. desktop computers were used for fewer activities, and most students used desktop computers for accessing or reading course materials (57 australasian journal of information systems volume 18 number 3 2014 339 percent), searching the internet for course related information (57 percent), communicating using emails (52 percent), and accessing the university's lms (51 percent). tablet computers and smartphones were used for very different activities suggesting that students consider them to be useful for significantly different aspects of their study. tablet computers were used by students mostly for engaging with their formal learning materials, for example course materials (63 percent) and course textbooks (50 percent), searching the internet (56 percent) and taking notes (55 percent). smartphones were most likely to be used for interactive communication and engagement with peers and teachers through emails (62 percent), social media (59 percent) and information sharing (52 percent). smartphones were also used extensively to take photos or videos to support learning (66 percent). figure 5: frequency of learning activities undertaken with mobile technologies figure 5 (detailed figures provided in table 1) provides some interesting insights into the extent to which students are using mobile technologies across a range of learning activities. firstly, a feature phone is rarely used for the learning activities indicated, while the laptop is the most widely used device for learning. the laptop computer has logical troughs in use in some learning activities such as taking photos or videos. smartphone usage provides some interesting insights with some learning activities such as sending and receiving email showing conducted often by students, whereas completing assignments was relatively uncommon. tablets computers are sitting in the middle overall in terms of usage for the range of learning activities assessed in this study, again with some logical troughs and peaks in usage across the learning activities. these findings demonstrate that students are actively using their mobile technologies in specific ways to support their learning. as smartphones are used as devices to provide instant access to online information there is scope for these technologies to be used effectively to engage students more interactively during on-campus lectures. for example, students could be asked to search on a topic and provide feedback in small groups on information found rather than rely on the lecturer to transmit the information to them. for distance education students, learning australasian journal of information systems volume 18 number 3 2014 340 content that is accessible on mobile technologies offer opportunities to take advantage brief moments between family and work responsibilities to engage in learning. learning activities laptop desktop computer tablet computer smartphone netbook standard mobile phone mp3 player e-book reader sending and receiving emails (to/from the course leader or other students) 75% 53% 48% 61% 24% 0% 0% 0% accessing or reading course materials 81% 56% 60% 26% 40% 0% 0% 26% listening to course audio materials such as lectures or podcasts 72% 39% 28% 14% 24% 0% 19% 0% watching course videos such as video recordings of lectures 74% 40% 33% 9% 20% 0% 2% 0% reading prescribed course textbooks 62% 33% 48% 11% 16% 0% 0% 32% searching the internet for course related information 76% 54% 57% 41% 28% 0% 2% 5% completing assignments 78% 47% 18% 2% 28% 0% 0% 0% participating in discussion forums 76% 39% 40% 22% 16% 0% 0% 0% searching online databases such as journals or publications 73% 43% 28% 12% 24% 0% 0% 0% accessing the university learning management system 78% 50% 50% 44% 36% 0% 0% 0% using video or audio conference tools such as skype to communicate with fellow students or course leaders 60% 26% 18% 15% 12% 0% 0% 0% communicating on social media sites (such as facebook or twitter) about your studies 69% 33% 48% 60% 20% 4% 0% 5% taking photos or videos to support your learning 22% 8% 37% 65% 0% 4% 0% 0% take notes 65% 29% 53% 38% 28% 4% 0% 0% share information with other students 69% 39% 48% 54% 32% 4% 0% 0% none of these 12% 31% 20% 21% 52% 88% 81% 58% table 1. detailed frequencies of learning activities australasian journal of information systems volume 18 number 3 2014 341 preferences and attitudes towards use of mobile technologies for learning mobile learning can be described as the ability to learn anytime, anywhere (traxler, 2005). to identify student preferences for and attitudes about mobile learning, participants were requested to respond to a series of statements on a five point scale from strongly agree (5) to strongly disagree (1). the overwhelming majority of students want to be able to learn anytime, anywhere (m=4.49, sd=0.84) and would like to be able to use their mobile technologies to support their learning (m=4.47, sd=0.86) (see figure 6). the majority of students also indicated that they would like to use their mobile devices in formal learning environments such as in lectures or on excursions. interestingly, a third of students disagree (12 percent) or strongly disagree (17 percent) that they would like to use their mobile devices in this context. further examination was undertaken using pearson's correlation to identify a potential reason for this finding. a strong negative correlation was found between this statement and the statement: “i would not want to use my mobile device in class or as part of my learning it is for staying in touch with my family and friends” (r=-0.46, p<.000, n=93). weaker negative correlations were also found for the perceived cost of this type of use: “using mobile devices for learning will be too expensive for me” (r=0.37, p<.000, n=93) and availability of sufficient internet access: “the internet access is too limited to effectively use mobile learning” (r=-0.34, p<.05, n=93). this suggests that there is a cohort of students who are opposed to using mobile technologies in formal learning spaces due to the cost of using the devices, the limited availability of internet access in these locations, and the perception that mobile devices are personal objects only to be used for personal communications. figure 6: attitudes towards using mobile for learning activities and learning materials designed for mobile technologies have not yet been effectively accommodated in the learning policies of the university where the research took place. students were requested to indicate on a five point scale from very unlikely (1) to very likely (5), how likely they would be to use smartphones, tablet computers, e-book readers or mp3 players for learning if appropriate resources were available to support these devices. the results are presented graphically in figure 7. the reason for this question was to determine whether use of mobile technologies for learning purposes would increase with the improved availability of appropriate resources. the tablet computer proved to be the device that the most students would want to use to support learning with 76 percent indicating that they would be likely or very likely to use this technology for learning (m=4.10, sd=1.35). students indicated that smartphones were marginally less desirable for use as learning devices (m=3.87, sd=1.35). this suggests that tablet computer may yet surpass smartphones as the learning technology of choice for students in the near future. ebook readers (m=3.00, sd=1.46) and mp3 players (m=2.63, australasian journal of information systems volume 18 number 3 2014 342 sd=1.39) were considered to be beneficial for learning by fewer students, yet still appear to be preferred by some students to support learning. figure 7: preferences for using mobile technologies for learning discussion findings from this research demonstrate that the adoption of mobile technologies, particularly smartphones and tablet computers, has increased considerably over the past three or four years. the results from this research markedly differ from the findings of previous studies which suggest that students’ everyday use of technology centres on well-established technologies such as laptop computers and mobile phones without the internet (gosper, malfroy, & mckenzie, 2013). in contrast, most students in the sample had access to smartphones and a large proportion had tablet computers, which is surprising as these technologies were only gained significant market share in early 2011. students are also using their mobile devices to support increasingly sophisticated learning activities previously only possible with computers. up until relatively recently, desktop computers were the primary technology to support learning activities and mobile phones were considered to be tools for interpersonal communications through calls, text messages or emails (ellis & newton, 2009). students in this sample appear to have leveraged the greater functionality and connectivity of smartphones and tablet devices and are actively using these technologies to support their learning. the smartphone in particular is being used by students for collaborative activities such as sharing information and communicating through social media. this supports findings by gikas and grant (2013) who found that students with mobile devices in classrooms benefited extensively from the constant connectivity available to them and the ability to communicate with classmates and instructors. the results from this study demonstrate that students use tablet computers very differently to smartphones. students appear to use tablet computers more as a transmissive technology to access course materials and the internet, rather than as a collaborative tool. this may simply be because students are still early adopters of tablet computers and may be unaware of the ways to use the devices to support greater collaboration and interactivity with peers and learning materials. the level of access students have to sophisticated devices, as well as the extensive use of these devices to support learning, clearly indicates that students would benefit from the provision of initiatives to support mobile learning both on-campus and at a distance. this study shows that students engage with learning while they are on the move and in non-traditional learning locations as they go about their daily lives. one limitation of this study is that it is not possible to identify the exact nature of the mobile learning activities that students are engaged in. a series of online and on-campus focus groups are currently underway to gain greater insight into these trends. australasian journal of information systems volume 18 number 3 2014 343 data about the attitudes and preferences of students towards the ability to learn anytime, anywhere raises a few concerns when considering the piecemeal support of this kind of learning by institutions. the majority of students appear to want to be able to learn anytime, anywhere and prefer to use their mobile devices for informal learning. there appears to be some resistance to the use of these devices for formal learning purposes as a third of students are uncertain or opposed to the use of their own technologies within classrooms or on excursions. these findings have significant implications for institutions that wish to rely on byod policies to encourage greater interactivity during classroom sessions. the high correlation with statements that indicate concern about the cost implications of this type of use and the availability of internet access does suggest that some of these concerns could be rectified by ensuring students have adequate access to free on-campus internet. the potential for mobile technologies to enhance and enrich student learning also poses challenges for higher education institutions. if academics start to embrace the possibilities for enhancing and enriching student learning through well designed learning activities which are fit for purpose for the different types of mobile technologies, then this may also be a double edged sword. students who don’t have access to these mobile technologies and the accompanying connectivity may be severely disadvantaged in their learning. conclusion this research was aimed at obtaining exploratory data about the ownership of mobile technologies among higher education students and their use of these technologies to support informal learning. the findings of this research clearly indicate that a range of mobile technologies which are fit for purpose for a range of learning activities, are being embraced by students. laptops, smart phones and tablets devices appear to be dominant technologies being used by students for a range of learning activities. the use of these mobile technologies across these learning activities have logical troughs and peaks indicating that students are using these mobile technologies in a common sense and programmatic manner to best effect to support their learning. furthermore our findings indicate that mobile technologies present both opportunities and challenges to academics and institutions to create learning activities which better utilise the multi-functionality of mobile technologies such as laptops, smart phones and tablets. but at the same time this also creates some dilemmas in that learning activities which are facilitated by mobile technologies may actually disadvantage students who don’t have access to these technologies and connectivity. this study has the potential to support institutional planning by providing insights into the ways students are currently using technologies for learning and their attitudes preferences for future use. next steps and future research directions it is a limitation of this study is that it only presents findings from a very early exploratory investigation of the use of mobile technologies for personal and professional use within the student cohort of a large distance education institution within regional australia. as such, the study does not address the potential impact of specific mobile learning interventions on student learning outcomes. the survey developed for this research will soon be deployed with students at anu and unisa. this will increase the representativeness of the results and gain a more complete picture of mobile device ownership and use to support learning by australian students in a range of institutional contexts (go8 versus regional universities network, distance versus face-to-face). the survey has also been deployed in a number of asian countries including malaysia, thailand, vietnam and china to see if identified trends are consistent across other asian countries. focus groups are currently underway with students at usq, anu and unisa to further tease out how students are using mobile devices to support their learning, and the personal and institutional barriers to use they face. the resulting data from these studies as well as the focus groups conducted at the three universities will be used to develop a model for the pedagogical learning level of the mlef (see figure 1). a survey will be developed to test this model australasian journal of information systems volume 18 number 3 2014 344 and will be disseminated to the three universities involved in the study as well as other australian and international higher education institutions. the aim of the final mlef will be to provide institutions with a model representative of the mobile learning trends and preferences of students across a range of cultures and disciplines. the data from these studies will also be made available for use as benchmarking data for other organisations. references alrasheedi, m. and capretz. l.f. 2013. “can mobile learning maturity be measured? a preliminary work,” proceedings of the canadian engineering education association. retrieved 2 august, 2013, from http://library.queensu.ca/ojs/index.php/pceea/article/view/4805 ally, m., and gardiner, m. 2012. “application and device characteristics as drivers for smart mobile device adoption and productivity,” international journal of organisational behaviour (17: 4), september-december, pp 35-47. elias, t. 2011. “universal instructional design principles for m-learning,” international review of research in open and distance learning (12:2), february, pp 143-156. ellis, a. and newton, d. 2009. “first year university students' access, usage and expectations of technology: an australian pilot study” in world conference on e-learning in corporate, government, healthcare, and higher education (2009:1), vancouver, canada, pp. 2539-2546. gikas, j. and grant, m. 2013. “mobile computing devices in higher education: student perspectives on learning with cellphones, smartphones & social media,” the internet and higher education, (19), october, pp 18-26. gosper, m., malfroy, j., and mckenzie, j. 2013. “students' experiences and expectations of technologies: an australian study designed to inform planning and development decisions,” australasian journal of educational technology (29:2), pp 268-82. gosper, m., woo, k., muir, h., dudley, c., and nakazawa, k. 2007. “selecting ict-based solutions for quality learning and sustainable practice,” australasian journal of educational technology (23:2), may, pp 227-247. kennedy, g., judd, t. s., churchward, a., gray, k. and krause, k. 2008. “first year students’ experiences with technology: are they really digital natives?,” australasian journal of educational technology (24:1), november, pp 108-122. kobus, m.b.w., rietveld, p., and van ommeren, j.n. 2013. “ownership versus on-campus use of mobile it devices by university students,” computers & education (68), october, 29-41. matias, a., and wolf, d.f. 2013. “engaging students in online courses through the use of mobile technology,” in increasing student engagement and retention using mobile applications: smartphones, skype and texting technologies (cutting-edge technologies in higher education, volume 6), l.a. wankel, and p. blessinger (eds.), emerald group publishing limited, pp 115-142. mcneill m., & diao, m.m. 2011. mq student uses of technologies for learning: an ethnographic study. retrieved 2 august, 2013, from http://www.mq.edu.au/ltc/altc/student_it_experience/index.htm mcneill, m., diao, m.m., and gosper, m., (2011) "student uses of technology in learning: two lenses", interactive technology and smart education (8:1), pp.5 17. murphy, a., & farley, h. (2012). “development of a framework for evaluating the impact and sustainability of mobile learning initiatives in higher education,” in proceedings of the ascilite australian society for computers in learning in tertiary education annual conference, m. brown, m. hartnett and t. stewart (eds.), wellington, nz, november, pp 678680. australasian journal of information systems volume 18 number 3 2014 345 oliver, b., and goerke, v. 2007. “australian undergraduates' use and ownership of emerging technologies: implications and opportunities for creating engaging learning experiences for the net generation,” australasian journal of educational technology (23:2), may, pp 171-86. pegrum, m., oakley, g., and faulkner, r. 2013. “schools going mobile: a study of the adoption of mobile handheld technologies in western australian independent schools,” australasian journal of educational technology (29:1), pp 66-81. peters, k. 2009. “m-learning: positioning educators for a mobile, connected future”, in mobile learning: transforming the delivery of education and training. m. ally (ed.), edmonton, canada: athabasca university press, pp 113-134. sharples, m., taylor, j., & vavoula, g. 2005. “towards a theory of mobile learning”, proceedings of mlearn20054th world conference on mlearning, cape town, south africa, october. retrieved 2 august, 2013, from http://www.mlearn.org.za/cd/papers/sharples%20theory%20of%20mobile.pdf shim, j. p., mittleman, d., welke, r., french, a. m., and guo, j. c. 2013. “bring your own device (byod): current status, issues, and future directions,” in proceedings of the 19th americas conference on information systems (amcis). retrieved 2 august, 2013, from http://aisel.aisnet.org/cgi/viewcontent.cgi?article=1296&context=amcis2013. traxler, j. and kukulska-hulme, a. 2005. “mobile learning in developing countries”, in knowledge series: a topical, start-up guide to distance education practice and delivery, vancouver, bc: commonwealth of learning. retrieved 2 august, 2013, from http://www.academia.edu/2810630/mobile_learning_in_developing_countries wong, w. 2012. “tools of the trade: how mobile learning devices are changing the face of higher education,” community college journal (82:5), apr-may, pp 54-61. acknowledgements this project is supported through the australian government's collaborative research networks (crn) program. an earlier version of this paper was presented at the australasian conference on information systems (acis) 2013 in melbourne, australia. ajis vol. 6 no. 1 september 1998 evaluating method engineer performance: an error classification and preliminary empirical study steven kelly, matti rossi department of computer science and information systems university of jyvaskyla p.o. box 35 fin-40351 jyvaskyla finland email: (stevek, mor}@hyeena.jyu.fi tel:+358 41603036 fax:+358 41 603011 abstract we describe an approach to empirically test the use of metacase environments to model methods. both diagrams and matrices have been proposed as a means for presenting the methods. these different paradigms may have their own effects on how easily and well users can model methods. we extend batra's classification of errors in data modelling to cover metamodelling, and use it to measure the performance of a group of metamodellers using either diagrams or matrices. the tentative results from this pilot study confirm the usefulness of the classification, and show some interesting differences between the paradigms. keywords method engineering, metamodelling introduction in recent years we have seen a tremendous growth in the number of available software development methods. as has been stated, it is a practical impossibility to build a new case environment for every method, and there is no suggestion that everyone will begin to use the same method, or even one of a small set of methods (bubenko 1988). the solution to this conundrum can thus lie only in a case environment which can be customised to support any method. a proposed solution, called case shells (bubenko 1988) or metacase environments (ald91), has produced promising research prototypes, and recently also commercial products. we believe that we are now at a stage where these environments can be evaluated by users to shed light on their usability and expressive power. we describe an approach that can be used to empirically test the use of a metacase environment in modelling systems development methods. modelling a method into a metacase environment is in itself a form of system development, along with its own paradigms, tools and methods: we could call these 'meta-methods'. in evaluating such meta-methods, as with any method, we should try to eliminate tool-centred factors, as these are not essentially part of the method. similarly, we should not concern ourselves with the purely symbolic parts of the method's representations, as these may easily be changed without affecting the essence of the method. rather, we should concentrate on the conceptual aspects of the meta-method, i.e. the metamodelling language. it is however possible to use completely different representational paradigms with the same conceptual meta-method: both diagrams (ros95) and matrices (kel95, kin94) have been proposed as a means for representing metamodels. these different paradigms may have their own effects on how easily and well users can model methods. most evaluations of metamodelling methods have so far taken place on a purely conceptual level, ignoring the effects of the way that different representational paradigms may affect method definition. in this paper we aim to develop a research instrument that can be used to evaluate the effects of different representational paradigms by means of a controlled experiment of how accurately users model a method in a metacase environment. in this pilot study we use one metamodelling language, but two different representational paradigms, matrices and diagrams. batra (bat93) summarised the results of four experiments which studied errors made when performing data modelling, a similar activity to modelling methods. he classified the errors according to the facet (type of element, e.g. entity, attribute or relation) that was modelled, and concluded: "the analysis shows that the modeling of different facets requires different cognitive thinking, and leads to different types of errors". we adapt his classification and follow his approach, looking at each facet to see whether there is any difference in the performance between matrix and diagram representations. we believe that the developed classification can be used to study metamodelling errors in other settings as well and it should be easy to adapt to other environments. we first describe the background of the study and key terminology of method engineering. we take batra's classification (bat90) and extend it to create a set of facets of metamodelling. then we describe the metamodelling 39 ajis vol. 6 no. 1 september 1998 task the subjects were asked to perform and use the classification to analyse our preliminary results. finally we conclude and provide some future research issues. background and terminology in its simplest form we can say that a metamodel is a conceptual model of a development method (bri90). consequently, metamodelling can be defined as a modeling process, which takes place one level of abstraction and logic higher than the standard modeling process (gig91). at its simplest, a metamodel captures information about the concepts of a method and rules about how they may be used and connected together. for example, in data flow diagrams the concepts used to model systems are processes, data stores, external entities and data flow relationships between them. a metamodel may also include information about the presentation forms (or signs, cf. lep94) and uses of a method. metacase environments are configurable case environments that use a set of primitives (their meta-metamodel or metamodelling language) which allow them to describe a given method, and mechanisms to implement a case tool to support the defined method. design of methodologies and of applications systems are very comparable exercises in human endeavour: in both cases one has to decide what data is needed and what processes are to be supported. brinkkemper (bri95) has defined method engineering as "a discipline to design, construct and adapt methods, techniques and tools for the development of information systems". if the method engineering process is supported by specific computer aided tools we call the engineering discipline computer aided method engineering (came), and the supporting tools came tools. a person responsible for developing and implementing method specifications is called a method engineer (kum92). in this study we are interested in the utility of the came tools. the power of came comes from the definitive power of the metacase environment's metamodelling language and the ease of use of the came tools for a method engineer. most came tools to date have used textual languages to represent methods, as in the very earliest tools (e.g. see isd85), but recently there has been a move towards a more visual approach. the development of visual metamodelling languages has been heavily focused on diagrammatic paradigms, motivated by the fact that a large amount of methods themselves use diagrammatic representations. hofstede and weide (hof93) emphasise the importance of diagrammatic formalisation in instantiating a method, and develop a general approach for such formalisation. smolander et al. (smo91), protsko et al. (pro89), and several others have proposed languages to represent graphical notations of a method, their connections, and/or graphical constraints. in addition, kelly (kel94) and kinnunen et al. (kin94) have investigated the utilisation of a matrix format as a visual representation form for metamodelling. in appendix 1 we give a brief description of the metaedit+ tool and goprr metamodelling language used in this experiment. one (implicit) assumption for the shift from a textual metamodelling language to a visual one lies in the belief of the improved user friendliness and accuracy. for instance, a study of data modelling with graphical e-r diagrams and a textual language (bat90) found "overwhelming evidence in favor of the graphical model for the greater correctness of the models produced. however, no studies in the method engineering field have been performed to support this belief, beyond a short comment in (gol93). this necessitates empirical research, for example laboratory experiments, not only to evaluate this belief, but also to investigate user preferences for different visual representational paradigms. the implementation of visual paradigms should also be improved to address the wide spectrum of functionality which the metamodelling languages could support. this includes, for instance, the investigation of the role of visual languages for the querying and retrieval of reusable method fragments. classification of facets in metamodelling in this chapter we develop a classification of user errors in metamodelling into various facets. as a starting point for our classification of facets we took that which batra defined and used over a series of experiments (bat90, bat93). batra's studies were of errors made while performing data modelling, i.e. modelling the structure of data for a database, an activity very similar to modelling methods; to our knowledge there has been no previous classification of facets specific to metamodelling. batra's classification fits our needs particularly well because it has been successfully applied to models made using the entity-relationship method, from which our goprr method is descended. also, batra's analysis concentrated mainly on relationships, the subject of our hypothesis. batra claims that the errors arise from the hutchins, hollan and norman's 'gulf, or directness distance between a user's goals and the human-computer interface (hut85). we can argue that the metamodelling task is in essence a 40 ajis vol. 6 no. 1 september 1998 mapping task from a part of 'reality', which happens here to be the description of a software development method, into a conceptual representation of the method in a given metamodelling language and its representation forms. we have modified batra's classification to make it more appropriate to metamodelling (see table below). as properties in goprr are more complicated that attributes in er, we supplemented the 'descriptor' attribute facet with a new 'property data type' facet. similarly, properties in goprr can contain objects, something not found in er: this caused the addition of a 'complex property type' facet. batra's 'category' facet representing the use of inheritance was omitted from this analysis, because use of inheritance in goprr is more a matter of taste than correctness: all inheritance hierarchies produced by subjects were flattened before analysis. in place of the 'category' facet we examined whether the right metatype was chosen for each element, e.g. that an object was not modelled as a relationship. other minor changes could be made: for the relationship facets, 'binding' would be a better name within goprr, and 'n-ary' more accurate than 'ternary'; we maintain batra's naming conventions for simplicity. the facets can usefully be grouped into those dealing with objects (o), properties (p), and relationships (r). the last group is subdivided for the purposes of our hypothesis into simple relationships (unary and binary, r(s)), and complex relationships r(c). 0 o o p p p r (s) r (s) r (c) facet 1 . entity 2. metatype choice (object, relationship etc.) 3. complex property type 4. identifier 5. descriptor 6. property data type 7. unary relationship 8. binary relationship 9. ternary relationship, role cardinalities description object, role or relationship type identified e.g. relationship modelled as object is the right complex meta property type used (list, combo box, reference etc.) does a property uniquely identify its owner are there the right set of properties for a type is the data type right for each property (string, number etc.) relationships with two roles, in both of which is the same object relationships with two roles, each with a different object relationships with more than two roles, either explicitly or by >1 cardinality for a role facet in batra? yes changed from category added yes yes added yes yes yes research hypothesis and scoring in this research we wanted to shed light onto some key aspects of the usability of advanced method engineering environments. there has been a clear move in the field from textual or programmatic method definition formats into graphical, matrix or form-based paradigms. we wanted to compare the performance of these new paradigms with each other: which paradigm was better for which facets of metamodelling. to do this, we needed to remove other factors such as the underlying metamodelling language, symbols and tools. we thus decided to analyse the difference between metamodelling with the matrix and diagram editors of metaedit+: both work with the same underlying conceptual metamodelling language, goprr, and the tools are mutually consistent in their user interface: similar menu structures, use of popup menus, same toolbar etc. the matrix representation shows object types on the axes of the matrix and the relationship or role types in cells: the relationships shown in the matrix are from the object on the left axis to the object on the top axis. the diagram representation shows objects as rectangles, connected by lines via roles (circles) and relationships (diamonds). as independent variable we took the use of the matrix or diagram editor, and as dependent variables the metamodelling correctness in each facet or area of metamodelling. the working hypothesis was: matrix is better with simple relationship modelling, diagram better otherwise this hypothesis is based on observations about the readability of the models and the ways that the diagrams and matrixes are used (kel94). we believed that simple relationships (i.e. unary and binary) would be easier to read in the matrix representation, because the relationship information can be presented more compactly in a matrix without 41 ajis vol. 6 no. 1 september 1998 the clutter and line crossings common in diagram representations. for example, figure 1 shows the legal relationships for the objects of the real time structured analysis metamodel, represented as both a diagram and a matrix. to find what relationships are allowed from a data transformation to a control transformation, we can follow the grey arrows in the matrix and read across the data transformation row to the control transformation column, and find that only a signal relationship is allowed. trying to find the same information from the diagram representation is somewhat more difficult. data transformation control transformation terminator buffer data store data transformation signal, discrete, continuous signal, activation, deactivation signal, discrete signal, discrete discrete control a transformation signal jj ^ signal signal signal terminator signal, discrete signal buffer signal, discrete signal data store discrete figure 1. real time structured analysis oprr metamodel, as a graph (properties hidden) and a matrix (showing relationship names) in the case of the more complex relationships (i.e. ternary or higher order), on the other hand, we expected the diagram representation to be easier to use, because whilst the matrix representation is still more compact, the ternary relationship components are scattered throughout the matrix. outside of relationship handling, the differences between matrix and diagram formats seemed likely to be smaller, but we envisaged diagrams performing somewhat better because of their familiarity and more graphical nature. to test our hypothesis we built an experiment, which is described in the next section. a scoring scheme was developed for giving consistent scores for different types of errors. to reduce subjectivity in analysing the metamodels we used an automatic report to provide the conceptual metamodel from each assignment in textual form, thus removing any effects the representational paradigm used might have on our ability to analyse the results, and reducing the possibility of bias towards our hypothesis in our analysis. the resulting report was in each case compared to that of a reference 'correct' metamodel, which had been made by the authors and checked by an 42 ajis vol. 6 no. 1 september 1998 independent method engineer. as the use of inheritance in goprr was allowed, but would give distorted results (there is no 'correct' inheritance hierarchy) to the analysis, all inheritance hierarchies were first flattened and purely abstract types removed before the analysis. there may be several different acceptable ways to model a method, depending on our approach, goals, and even personal style. thus in our approach to classifying 'errors' in modelling methods we mainly classify as 'errors' cases where there is a clear omission, addition, internal inconsistency or undeniably incorrect model. whilst we found it relatively easy to spot correct components and incorrect components, additions or omissions, and to decide which facet the component belonged to, it was harder to determine how to score these occurrences within each class. the main problem was the lack of any definite number of correct components: although we had a reference metamodel, we recognised that our having e.g. 4 object types did not mean that was the only possible solution. thus we decided to count the correctly modelled, incorrectly modelled, unnecessarily added, and necessary but omitted components separately for each facet in a subject's metamodel, and to form a percentage correctness within each facet from the number of correct components divided by the total number of components modelled by that subject for that facet (correct+incorrect+added+omitted). thus we obtained percentage correctness scores for each subject for each facet. experiment set-up and tools used the task given to the subjects was to build a metamodel for the class diagram of the booch/rumbaugh unified method (boo95) using the diagram or matrix editor in metaedit+. we had originally envisaged that the task would also include a second diagram type, but time constraints forced us to restrict the task to the class diagram. the unified method, published only one month earlier, was chosen to ensure that previous experience with the given method would not interfere with the results. there were 11 study subjects, who were students from a method engineering course given at the university of jyvaskyla in fall 1995. each subject had 20 hours of lectures about method engineering before the assignment, and they had been given two hours of training with die metaedit+ tools and eight hours of lessons about the goprr metamodelling language it uses. the students were divided randomly into two groups to model the unified method with either the matrix or the diagram editor of metaedit+. each subject was given instructions to make the metamodel of the class diagrams in a three hour modelling session. the material given them was part of the documentation of the unified method from rational (boo95). the students were instructed to only use the matrix or diagram editor for the task. the task performance time was controlled and the students had no possibility to talk to others, but they could obtain help from the researchers in possible technical problems with the tools. throughout the experiment experienced users of metaedimand goprr metamodellers were on hand to assist the modellers. they were instructed to answer questions on metaedit+ and goprr, but not on issues regarding decisions about how to model the given method. they were also instructed to actively look for models that showed a clear misapprehension of some part of metaedit+ or goprr, but similarly not to intervene in the case of errors in the model of the method. the result of the modelling session was either a matrix or diagram model of the unified method. the results were analysed according to criteria given in section 0. preliminary analysis of user metamodels the results were analysed by the authors, who first analysed one assignment together to try to ensure that their use of the facet classification would be similar. the remaining assignment method models were divided between the researchers, half each, to be analysed. one subject was excluded from the results, as he only managed to model a tiny fraction of the method. the analysis of the remaining subjects' work has been done, but the interpretation of the findings is still underway. here we present preliminary results. figure 2 presents the mean percentage score of all diagram users, and all matrix users, in each facet. marked differences are few outside of the relationship facets: the clearest is in the identifier facet, but this is probably to be explained by the small number of subjects and the overall lack of understanding in the subject group of how and when to use identifiers. the modelling of identifiers was not immediately visible in the matrix or diagram formats, and was performed in exactly the same way in both tools, thus we can safely disregard this as a statistical anomaly. descriptor had the next largest difference of the non-relationship facets, and here we see the diagram format performing better: descriptors are immediately visible in a diagram, but in a matrix the user must open a separate dialog to see them. 43 ajis vol. 6 no. 1 september 1998 100% facet figure 2. mean percentage correctness score in each facet q. s 8 §> si 100%-r90 % • 80 % 70 % • 60 % • 50 % • 40 % • 30 % • • 20 % • • 10% • • 0 % • • diagram d matrix , unary+ binary other class 1st 2nd 3rd 4th ranking 5th figure 3. mean score in unary and binary relationships, and elsewhere. figure 4. each user's mean score over all facets (best matrix user paired with best diagram user etc.) to test our hypothesis, we combined each subject's scores in the unary and binary facets, and again in all other facets, and compared the mean results of diagram and matrix users. figure 3 shows the results, which would tend to support at least the first part of our hypothesis: that matrix users would be better on unary and binary relationships (73% matrix, 54% diagram). the second part of the hypothesis, that elsewhere diagrams would be better, is not as clear (62% diagram, 60% matrix). it would be better supported if we omitted the anomalous identifier results (giving 71% diagram, 61% matrix); alternatively, it may be that the matrix format did indeed perform better overall than we had envisaged, further strengthening its claims as a metamodelling representation paradigm. overall, taking all facets into account, users of the matrix and diagram formats performed remarkably similarly: individual differences of subjects within a format were much greater than those between formats, as seen in figure 4, which shows the overall mean score for each of the ten subjects. the mean was 60% for diagrams and 63% for matrices, with standard deviation of 16 percentage points in both cases. by a happy coincidence the best matrix user obtained almost the same score as the best diagram user, and so on down to the worst. whilst this of course has in itself no statistical significance, it allows us to give a good subjective impression of the overall similarity of performance of the diagram and matrix users, showing each user of each format in order of mean score in a clear head-to-head comparison. 44 ajis vol. 6 no. 1 september 1998 in addition to these preliminary summary results, we have also begun analysing the test scores in more detail using spss. using different subjects for the diagram users and matrix users gives us two independent samples, enabling better analysis than if the same subject had performed the modelling task with each tool. the scores in each facet are continuous interval variables (there is no clear zero point), although it may be safer to regard them as discrete, given that the scores are the quotient of two discrete values. the independent variable, use of matrix or diagram, is strictly a nominal variable: however, taking it as a binary variable with values of 0 and 1 is not unusual and allows the use of more sophisticated statistical tests. looking at pearson product-moment correlation coefficients, we see the largest correlation between scores on objects (facets 1-3) and on properties (4-6): 0.6785 (p=0.016), followed by that between simple relationships (7-8) and complex relationships (9): 0.4721 (p=0.084). given our hypothesis concerned only simple relationships, we can examine the partial correlation between use of matrix and simple relationship score, controlling for the complex relationship score: this gives 0.59 (p=0.047). the similar test for matrix and object, controlling for property scores, gives -0.30 (p=0.44). in other words, there is a statistically significant direct relationship (p<0.05) between the use of a matrix and the score obtained for simple relationships, controlling for the score for other relationships; there is no such correlation between use of matrix and scores for objects or properties. multiple regression results seem to show that use of matrix and ternary score form a good predictor of unary scores (r2=0.61, significance of f = 0.038): the ternary has partial regression coefficient b=0.53 (significance of t = 0.029) and matrix has b=0.41 (significance of t = 0.032). if the use of matrix is excluded from the model, the significance of t for the ternary term is no longer meaningful (0.18). these spss results remain however preliminary, subject to more detailed examination of the data and assumptions of the statistical tests used. conclusions in this paper we have performed a pilot study for an empirical experiment which would analyse the usability of different method engineering representational paradigms. the study served as a preliminary test for our metamodelling error classification. the classification was developed from batra's error facet classification for database design (bat90).. we found our extension of batra's classification to be a useful tool, and believe that it can be a valuable aid in empirical evaluation of metamodelling languages and tools. comparing our results with batra's, we find some overall similarities, but a striking difference in the property group of facets. batra's subjects were "usually successful" in modelling attributes, whereas that was the worst area for our subjects: a mean score of 49% compared to 65% for relationships and 71% for objects. this is probably due to the greater variety and complexity of properties used in modelling methods than in data modelling, or maybe the fact that the attributes do not have a separate representation in goprr. on the basis of this pilot study we can make some tentative conclusions about the performance of matrix and diagram formats for metamodelling, and use these to generate hypotheses for further research to confirm the findings. we believe that this kind of approach can give us important feedback about the performance and usability of method engineering tools. this kind of research is needed, because there is a lack of empirical evidence for the claimed usefulness and ease of use of metamodelling and method engineering environments. our initial findings tend to support our hypothesis that the matrix is a better format for working with simple relationship types in metamodels, whilst in other areas the diagram format seems to perform slightly better. due to the small sample size (n = 10), however, these results cannot be regarded as conclusive. should further research confirm these findings, we can draw at least two practical conclusions: • came environments should support more than one representation paradigm for metamodelling, as different parts of the metamodelling task are better performed by some paradigms than others. • when metamodelling with matrix and diagram tools, it would be useful to perform most work in the diagram tool, but addition of simple (unary and binary) relationships could best be performed while viewing the metamodel as a matrix. the latter result is doubly important for binary relationships, as (1) these form the bulk of all relationships in general in methods, and (2) they had the second worst overall percentage score for both diagram and matrix users (after identifier and descriptor respectively). to confirm our tentative findings from this experiment we would need to extend the setting in three ways: a larger number of participants, larger metamodel, and greater method engineer experience. another extension would be to include more metamodelling paradigms, such as textual and form-based: this would give further guidance to metamodellers as to which paradigm to use when, and to came environment designers as to which tools to include. 45 ajis vol. 6 no. 1 september 1998 when using multiple paradigms for metamodelling there would be important effects from how well integrated the paradigms were in the came environment. studies should thus also be performed on came environment use with multiple paradigms: does switching between paradigms form a problem for the method engineer, possibly even outweighing the benefits of the different paradigms for different tasks. another approach to analysing how well method engineers perform with different paradigms could be a comprehension task, such as the ones described in (par96). because a comprehension task uses a predefined model from which the subjects find answers to predefined questions, assessing the results would be easier. it would shed light on the problems of interpreting the metamodels, whereas here we analysed the problems of developing the metamodels. references alderson, albert, "meta-case technology," pp. 81-91 in software development environments and case technology, a. endres, h.weber (ed.), springer-verlag, (1991). batra, d., j. hoffer and p. bostrom, "comparing representations with relational and eer models," cacm 33(2) (1990) pp.126-139. batra, d., "a framework for studying human error behavior in conceptual database modeling," information & management 25 (1993) pp.121-131. booch, g., j. rumbaugh, "unified method for object-oriented develoment," rational software corporation, santa clara, us (1995). brinkkemper, sjaak, "formalisation of information systems modelling," ph.d. thesis, univ. of nijmegen, thesis publishers, amsterdam (1990). brinkkemper, sjaak, "method engineering: engineering of information systems development methods and tools," information & software technology 37(11) (1995) p.1-6. bubenko, j. a., "a method engineering approach to information systems development," the proceedings of the ifip wg8.1 working conference on information systems development process (1988) pp. 167-186. gigch, j. van, "systems design and modeling and metamodeling," plenum press, new york (1991). goldkuhl, goran, stefan cronholm, "customizable case environments: a framework for design and evaluation," accepted to cope it '93. lith-ida-r-93-42, linkoping university, sweden (1993). hofstede, a. h. m. ter, th. p. van der weide, "formalisation of techniques: chopping down the methodology jungle," information & software technology 34(1) (1993) p.57-65. hutchins, e. l., j. d. hollan and d. a. norman, "direct manipulations interfaces," human computer interaction 1(1985) pp.311-338. isdos, "system encyclopedia manager, language definition manager: user manual (sem/ldm)," version 1.4 (june 1985). kelly, steven, "a matrix editor for a metacase environment," information and software technology 36(6) (1994)pp.361-371. kelly, steven, "what's in a relationship: on distinguishing property holding and object binding," in proceedings of 3rd international conference on information systems concepts, isco 3, w. hesse and e. falkenberg (ed.), university of marburg, lahn, germany (1995). kelly, s., k. lyytinen and m. rossi, "metaedit+: a fully configurable multi-user and multi-tool case and came environment," pp. 1-21 in proceedings of the 8th international conference on advanced information systems engineering, caise'96, springer-verlag (1996). kinnunen, kimmo, mauri leppanen, "o/a matrix and a technique for methodology engineering," in proceedings of the fourth international conference on information systems development, j. zupansis and s. wrycza (ed.), moderna organizacija, kranj, slovenia (1994). kumar, kuldeep, richard j. welke, "methodology engineering: a proposal for situation specific methodology construction," pp. 257-269 in challenges and strategies for research in systems development, kottermann w. w. and senn j. a. (ed.), john wiley & sons, washington (1992). leppanen, mauri, "metamodelling: concept, benefits and pitfalls," pp. 126-137 in proceedings of the fourth international conference on information systems development, j. zupansis and s. wrycza (ed.), moderna organizacija, kranj, slovenia (1994). parsons, j., "on experimental evaluation of classification in information modelling: local versus global schemas," proceedings of the emmsad'96 workshop, university of british columbia, vancouver (1996). 46 ajis vol. 6 no. 1 september 1998 protsko, l. b., p. g. sorenson and j. p. tremblay, "mondrian: system for automatic generation of dataflow diagrams," information and software technology 31(9) (1989) pp.456-471. rossi, m., "the metaedit came environment," proceedings of the metacase 95, university of sunderland press, sunderland (1995). smolander, kari, kalle lyytinen, veli-pekka tahvanainen and pentti marttiin, "metaedit — a flexible graphical environment for methodology modelling," pp. 168-193 in advanced information systems engineering, proceedings of the third international conference caise'91, trondheim, norway, may 1991, r. andersen, j. a. bubenko jr. and a. solvberg (ed.), springer-verlag, berlin (1991). welke, r. j., "the case repository: more than another database application," in challenges and strategies for research in systems development, william w. cotterman and james a. senn (eds.), wiley, chichester uk (1992). appendix 1 a short description of metaedit+ and its goprr metamodelling language is given below, for a longer discussion see (kel96). metaedit+ allows the user to specify the isd method to be used, and to use several different tools to make and manipulate models of the is according to that method. these different tools represent different representational paradigms: the same underlying conceptual data can be shown in various formats, including graphical diagram, matrix and table formats. the same information can be handled in a way appropriate to the current task: for some tasks it is easier to work with the data in diagram form, for others in matrix form. both forms rely on the underlying conceptual information, which contains no information about what the representation is like. metaedit+ models the underlying conceptual information of both models and metamodels with goprr (kel96), which forms an evolutionary extension of the oprr model, used successfully in specifying methods for metaedit (wel92, smo91). the fundamental goprr modeling constructs are: • graphs, which are aggregates of a set of objects and their connections. examples of graphs are an entity relationship diagram or a data flow diagram. • objects, which are considered as independent and identifiable design objects. examples of objects are an entity in an entity relationship diagram or a process in a data flow diagram. • properties are attributes of graphs, objects, relationships and roles. an example of a property is the number of a process in a data flow diagram. • relationships are associations between objects. an example of a relationship is a data flow in a data flow diagram. • roles define the ways in which objects participate in specific relationships. an example of a role is the 'to' end of a data flow relationship, which is indicated by an arrowhead. the goprr model is equally applicable to both the metamodel (type) and model (instance) levels of case data, and thus is particularly well suited to a metacase environment, as the same concepts can be used both for making methods and for making models with those methods. a particular benefit is the ability to make a model of a method on the instance level, and then transform that by a simple mapping into a meramodel for the method on the type level. in addition to these fundamental concepts or metatypes goprr also uses some auxiliary constructs, the most important of which is binding (kel95). a binding specifies a relationship, a collection of roles, and for each role a collection of objects taking part in that relationship in that role. in other words, the objects, relationships and roles themselves store no information about each other; rather, a graph includes a set of bindings that specify how its various elements are connected together. in a metamodel bindings work as rules, specifying the legal ways in which objects of the different types can be connected via relationships of various types. 47 microsoft word ajis_18(3)_v10.docx australasian journal of information systems volume 18 number 3 2014 257 creating value through virtual teams: a current literature review akemi takeoka chatfield school of information systems & technology university of wollongong wollongong, australia akemi@uow.edu.au vivian najem shlemoon school of information systems & technology university of wollongong wollongong, australia vns983@uow.edu.au wilbur redublado school of information systems & technology university of wollongong wollongong, australia wr803@uowmail.edu.au gary darbyshire school of information systems & technology university of wollongong wollongong, australia gd197@uowmail.edu.au abstract globally, virtual teams (vt) as ict-enabled emergent network organisation forms have gained international validity by innovative organisations, with a corresponding surge of interest in understanding how organisations can leverage vt to create business value. despite growing deliberations in vt literature on managing vt, tasks and outcomes, however, creating business value through vt remains an unresolved theoretical and pragmatic conundrum. a review of prior relevant literature is essential to advancing knowledge. the paucity of published review articles seems to have impeded the field’s accumulation of vt knowledge. this research, therefore, reviews the current literature on case studies of vt to address the question: what are organisational challenges in creating business value through vt in the organisation? the key challenges found in the literature are effective communication, knowledge sharing, trust, and interpersonal skills in the new virtual boundary-less environment. drawing on the it business value model, we also discuss their resource-based implications. keywords: virtual teams, business value creation, organisational challenges, resourcebased implications, literature review introduction in the digital economy both private-sector and public-sector organisations alike increasingly depend on smart information technology (it) infrastructure for timely information sharing, effective operational control, rapid innovation, speed to market, and customer satisfaction. on the other hand, recent global financial crisis and economic recessions encourage trends for increased managerial scrutiny to reduce australasian journal of information systems volume 18 number 3 2014 258 it spending and to increase business value of it investments (coleman and chatfield, 2011). globally, virtual teams (vt) as ict-enabled emergent boundary-less network organisation forms (jarvenpaa et al., 1997) have gained international validity by innovative organizations to leverage geographically and culturally dispersed knowledge and expertise worldwide, with a corresponding surge of interest in understanding how organisations can leverage vt to create business value. in literature on vt, there are various terms in use such as vt, global virtual teams, multinational and multicultural distributed teams, ict-mediated dispersed teams among others. while all these terms and their conceptions share common characteristics with those of traditional face-to-face teams, the former differ significantly from the latter with regard to their high-level virtuality. moreover, even among the existing definitions of vt, there are differences in focus. in a literature review research on early vt, a virtual team is defined as “a group of geographically, organisationally and/or time dispersed workers brought together by information technologies to accomplish one or more organisation tasks” (powell et al. 2004, p.7). as this definition suggests, it is central in facilitating tasks performed through vt. like many other similar definitions of vt, however, it seems to focus on task-level, team-level analysis, and hence failing to view vt as a new organisational resource for creating business value through investments in developing and deploying vt in the organisation. with the increasing workplace trends for vt deployment options over the recent years, there has been a corresponding surge of interest in understanding how organisations can leverage vt to create business value. despite growing deliberations in information systems literature, however, creating business value through vt in the global workplace remains an unresolved theoretical and pragmatic conundrum. to reduce the gap in the literature, we aim to address one central research question: what are the key organisational challenges in creating business value through leveraging vt in the organisation? in this research, we will answer the question by conducting a systematic review of the current literature on vt, directing our attention specifically to published case studies on vt in an organizational setting. a review of prior, relevant literature is essential to advancing knowledge. in is literature, there are few published review articles, which seem to have impeded the field’s accumulation of is knowledge (webster and watson, 2002). the same problem seems to exist with vt literature. this study provides a systematic review of previously published journal or review articles on the findings from current virtual team research in an effort to develop a better understanding of the key challenges in creating business value through leveraging virtual teams as the ict-enabled emergent network organisation forms. our review examined the publications over the recent years from 2004 to 2012. our review of the case studies on vt found the key organisational challenges: effective communication, knowledge sharing, trust, and interpersonal skills in the virtual environment. the review findings are categorized according to issues/challenges, tools in use for vt, and organization forms of vt. we draw on prior conceptual framework, “an integrative model of it business value” (melville et al. 2004) to discuss the key organizational challenges in creating business value through vt: effective communication, knowledge sharing, trust, and interpersonal skills and to discuss their resource-based implications for the future of organizational practice through leveraging vt. the remainder of this paper is structured as follows: the next section presents a brief description of our methodology used to conduct a systematic review of the current literature on vt with our research focus on case studies of vt in the organisation. the third section presents our review findings on vt research. the fourth section draws on the it business value model (melville et al., 2004) as our guiding conceptual framework to organise and discusses the key organisational challenges found in the literature in creating business value through vt. this discussion and conclusion section also identifies our research limitations and directions for future research. australasian journal of information systems volume 18 number 3 2014 259 methodology in this research we have addressed the central research question: what are the organisational challenges in creating business value from leveraging virtual teams in the organisation? in order to answer this question, we first conduct a systematic review of the existing literature on vt to identify the key organisational challenges and then we draw on the it business value model (melville et al., 2004) to organize and discuss these key challenges. a systematic search accumulates a relatively complete census of relevant literature (webster and watson, 2002). there are three structured approaches recommended by webster and watson (2002) to determine the source material for review, (1) the major contributions are likely to be in the leading journals; (2) going backward by reviewing the citations for the articles identified in step 1 to determine prior articles we should consider, and finally (3) going forward by using any journal database to identify articles identified in the previous steps. in academic practice, the usefulness of a piece of research often is evaluated by its uptake by other researchers and not by the fact it has been published. one, albeit very imperfect, way of looking at this is the use of citation counts (ginieis et al., 2012). so in this study, we use citation counts provided by academic databases to evaluate the usefulness of a given vt publication. previously literature review studies of early vt were published in 2004 or earlier (for example, powell et al. 2004). so in this study, our review is directed to review and analyse the literature in consecutive years from 2004 to 2012. in contrast to the prior review work on vt by powell et al. (2004), where 67% of vt reviewed were student teams, we explored the literature on vt operating in organisational settings by focusing primarily on case studies. we draw on the methodology suggested by webster and watson (2002) and moustaghfir (2008). first, we used “virtual team” as the relevant primary keyword and “case study” as the secondary keyword for our search strategy. second, we identify five major academic databases through which we search highly cited journal articles on vt: scopus, web of science, ieee, springer, and ais. therefore, while there exist the differences in search engines in use across the databases, we consistently employed the following generic query strategy: (title or abstract) contains ("virtual team") and (publication year) = (2004-2012) and (publication type) = (journal article), which was constrained by the use of secondary keyword ("case study"). this search strategy result in 132 published journal articles. then, after reviewing the abstract of these articles, 12 case studies have been selected for further analysis because these case studies focused on key organisational challenges that are relevant to answer our central research question. although the concept of vt is still new and emerging, the literature has been rapidly growing over the past decade. so we have decided to exclude conference papers and books from our systematic review and analysis. we further excluded published journal articles written in other languages than english. literature review recent trends in vt research: 2004-2012 using the search strategy discussed in the methodology section but without further constraint, namely the use of secondary keyword (“case study”), we identified 254 journal articles and conference papers across the five major academic databases. they were published in ieee transactions on professional communications (20 articles, or 8%), group decision and negotiation (9 articles, or 4%), proceedings of the american conference on information systems (amcis) (8 papers, or 3%), behavior and information technology (5 articles, or 2%), business & information systems engineering (5 articles, or 2%), information systems frontier (5 articles, or 2%), journal of business ethics (5 articles, or 2%) and other journals which published 4 or less articles on vt (197, or 78%). in summary, figure 1 shows a bar graph for an overall trend in vt research published over nine years from 2004 to 2012. the bar graph shows the frequency distribution of these published vt studies for the five databases. of the five databases we examined, with 90 published case studies, springer leads australasian journal of information systems volume 18 number 3 2014 260 the number of publications, which is followed by scopus (66), web of science (52), ieee (29) and ais (17). figure 1: overall trend in published vt studies across major academic databases: 2004 – 2012 in contrast, figure 2 shows time series graph for dynamically changing trends over the same period across the five databases. the graph shows that overall research interests in vt are on the rise, showing the three (springer, scopus and ieee) out of the five databases are publishing more studies in the recent years (2011-2012) vis-à-vis the earlier year (2004), even though conceptions and definitions of vt are still new and emerging in the literature. figure 2: yearly trend in published vt studies: 2004 2012 definitions of vt as figures 1 and 2 have shown, the concept of vt has gained significant and growing research attention during the last decade. moreover, vt as ict-enabled emergent network organisation forms have gained international validity by innovative organizations worldwide. however, defining what constitutes a vt or a global vt has been challenged in the literature partly due to many different terms in use: virtual teams, global virtual teams (jarvenpaa and leidner, 1999), geographically distributed teams (hertel et al., 2005) and ict-mediated teams. however, in early vt literature the term “virtual team” refers to “groups of people who work closely together even though they are geographically separated by miles australasian journal of information systems volume 18 number 3 2014 261 or even continents” (henry and hartzer 1998, p. 5). this earlier definition refers neither to the virtual team’s temporary nature nor to the critical role of ict for facilitating the communication across vt members. in contrast, one of the most cited definitions of a vt is “a group of geographically, organisationally and/or time dispersed workers brought together by information technologies to accomplish one or more organisation tasks” (powell et al., 2004, p. 7). although this definition refers to the central role of ict, it does not underscore the temporary nature of vt. the recent literature on global vt shows that vt are temporary in nature and they are assembled in an ad hoc basis to meet the business needs in an agile manner in the dynamically changing global business environment (shachaf, 2008). the literature on global vt also identifies the strategic nature of tasks for which vt are particularly in demand. they are typically assigned tasks that are strategically important and highly complex (maznevski and chudoba, 2000). finally, the literature is mixed with regard to the use of face-to-face meetings by vt. on the one hand, vt members rarely meet in person, conducting almost all of their interaction and decision making using ict (maznevski and chudoba, 2000). on the other hand, case studies on global vt show that vt members also are involved in traditional teams and also are involved with collocated face-to-face project meetings (may and carter, 2001). in consequence, there have been calls for defining vt as having high-level virtuality based on four dimensions of temporal, spatial, cultural, and organizational dispersion (shin, 2004; hertel et al., 2005). benefits and challenges of virtual teams a large number of studies explored advantages and disadvantages relating to virtual teaming (jarvenpaa and leidner, 1999; may and carter, 2001; breankek and martz, 2005; anderson et al., 2007; munkvold and zigurs, 2007; rosen et al., 2007; lee-kelley and sankey, 2008; ebrahim et al., 2009). vt often operate under such time-compressed schedules that they need to perform tasks and projects more rapidly than traditional face-to-face teams, because they do not work at the same time or place (munkvold and zigurs, 2007). in consequence, in their study on virtual team meetings, anderson et al. (2007) argue that the effective use of communication technologies plays an important enabling role in providing the flexibility (reducing relocation time and cost) of agile vt. moreover, ict-mediated vt perform well and reduces time-to market, which is one of the significant successful keys in some organisations (may and cater, 2001). lee-kelley and sankey (2008) found that mangers were willing to use technology because of their propensity for self-management and interpersonal awareness. on the other hand, jarvenpaa and leidner (1999) found that managers felt uncomfortable with ict-mediated vt concept because it implied new challenges in using technology as new methods of supervision. however, vt have produced higher quality outcomes because vt permit the highest qualified members for a particular job to join vt regardless of their location (ebrahim et al., 2009). in contrast, munkvold and zigurs (2007) found that cultural or language diversity within vt leads to differences in the members’ thinking process, which will affect their performance negatively. similarly, implementing vt could impact on trust negatively because of the geographical distance, difference in time zone, and other characteristics (khazanchi and zigurs, 2006). australasian journal of information systems volume 18 number 3 2014 262 table 1: summary of case studies with organisational challenges finally, table 1 lists the key organisational challenges discussed in the literature on case studies of vt in organisational settings. they are effective communication; knowledge sharing; trust; and interpersonal skills in the virtual workplace environment. these four challenges are discussed more fully in the next section. table 1 also identifies ict tools in use (thissen et al., 2007) to mediate virtual team task/project processes and performance in the 12 published case studies we reviewed in this study. while some of the case studies do not discuss ict tools, those case studies that identified specific ict tools show that synchronous ict such as email, instant messaging/chat, web conferencing and telephone as well as asynchronous ict such as groupware/shared services, remote access control, file transfer, and video conferencing. more recently, synchronous ict have been increasingly used to facilitate effective and immediate communication across vt members. finally, table 1 also identifies four distinct organisational forms of vt proposed by cascio and shurygailo (2003): telework (one manager and one location), remote team (one manager and multiple locations), matrixed telework (multiple managers and single location), and matrixed remote team (multiple managers and multiple locations). across the case studies reviewed in this study, all the vt have shown organisational forms of remote teams or matrixed remote teams, while none of the vt showed the other organisational forms of telework and matrixed telework. discusion in contrast to the prior review work on vt (powell et al., 2004), where 67.4% of vt reviewed were student teams and the publication years ranged from 1988 to 2002, with the mode in 2001, our review has focused on more recently published reviews or articles (2004-2012) on case studies on vt operating in the real organizational settings. in this section we will discuss the four key organizational challenges in creating business value through vt in the virtual environment. australasian journal of information systems volume 18 number 3 2014 263 effective communication communication is a powerful tool that can directly influence the social dimensions of vt and performance of vt, which in turn has a positive impact on satisfaction with vt (ebrahim et al., 2009). collaboration within the virtual team across time and space is enabled by a heavy reliance on computer mediated communications (kanawattanachai and yoo, 2007). on the one hand, the communication among virtual teams is very complex, requiring a detailed analysis of both the team members and the social dimensions of the shared common technology being used (burlea, 2007). communication breakdown can wreak havoc on a project as vt members struggle to effectively communicate and work with one another (daim et al., 2012). on the other hand, communication complexity is also the result of diverse competences of the virtual team members (rich, 1997). while communication can be viewed as a traditional face-to-face team issue, effective communication problems and challenges within vt can be magnified by distance, time, and increased cost of interaction due to the slower pace of nonnative speakers’ communication (shachaf, 2008). furthermore, cultural diversity had a negative impact on communication (shachaf, 2008) and cultural differences seem to affect vt performance poorly in maintaining effective cross-functional communication (daim et al., 2012). finally, communication difficulties (lee-kelley and sankey, 2008) are commonly found in the case studies depicted in table 1 even where those firms and industries leveraged the technology and embraced vt as a new organization form. according to hollingshead et al. (1993), both advances in ict and well-designed organisational arrangements will enable well-prepared virtual team members to communicate more effectively and more clearly across time and across different geographical locations than traditional face-to-face teams (hertel et al., 2005). may and carter (2001) in their case study of vt working in the european automotive industry have shown that enhanced communication and collaboration between geographically distributed engineers at automotive manufacturer and supplier sites make them get benefits are better quality, reduced costs and a reduction in the time-to-market (between 20% to 50%) for a new product vehicle. vorakulpipat et al. (2010) highlighted the need for a shared project knowledge base in a virtual team context to promote value creation through improved communication. vt need norms that describe how communication technology will be used (malhotra et al., 2007) where vt need frequent and effective communication (kanawattanachai and yoo, 2007) and where vt need to avoid miscommunications (daim et al., 2012). finally, anderson et al. (2007) observe that organisations which adopt vt must not only ensure good communication among all members of the dispersed team but also provide regular and timely communication feedback. similarly, the importance of implementing governance guidelines, rules and policies must be clearly understood (jarvenpaa and leidner 1999). importantly, having a well-defined strategy relating to vt is imperative to overcome these communication challenges. knowledge sharing in the 21st century, the most valuable organizational resources are knowledge, knowledge workers and their productivity, replacing the most valued asset of capital and equipment in the 20th century (drucker, 1999). similarly, davenport (1997) holds that the most valuable asset firms have is the knowledge of their employees as knowledge workers. boeing-rocketdyne established special vt, which are known as virtual cross-value-chain collaborative creative teams (or vc3 teams) for creatively and rapidly designing a new innovative low-cost engine (malhotra et al., 2001). vt faced many challenges in creative work, non-routine problem-solving and unpredictable solutions without the merit of having face-to-face meetings. however, they were successful in producing new ideas/designs quickly and continuously throughout the project life cycle. the outcome was new innovative engine design created under budget and achieving the goal of vt. to create business value, collaborative technology called ‘internet notebook’ was used as knowledge repository for recording and sharing knowledge, ideas, designs and comments. in addition, teleconferencing was held twice a australasian journal of information systems volume 18 number 3 2014 264 week to share ‘just-in-time’ analysis findings during these teleconference meetings and provide immediate feedbacks about problem solutions and feasibility of a design idea. these it resources and adaptive working processes drove the performance of this radical innovation project. another study found that maintaining "mutual knowledge" was a central problem of geographically dispersed virtual collaboration. based on 13 case studies of geographically dispersed teams, cramton (2001) identified five types of problems related to failures of maintaining mutual knowledge: “failure to communicate and retain contextual information, unevenly distributed information, difficulty communicating and understanding the salience of information, differences in speed of access to information, and difficulty interpreting the meaning of silence”. in terms of value creation, knowledge, which is distributed throughout vt environment, could create value when it is identified and transferred from a source location and applied where it is needed (alavi and tiwana, 2002, p.1030). from a resource-based view of the firm, knowledge can sustain long-term competitive advantage as it is not easy to imitate and socially complex (alavi and leidner, 2001). the groupware and ict such as electronic discussion forums and repositories can support knowledge storage/retrieval, knowledge sharing and knowledge transfer, whereas workflow systems support knowledge application (alavi and leidner, 2001). trust in vt within any organisational context, trust is a pervasive global challenge. therefore, not surprisingly, trust has received much attention in various research fields, confirming the importance of trust in sustaining the effectiveness of organisations during the past decade (jarvenpaa et al., 2004; mayer et al., 1995; colquitt et al., 2007; baruch and lin, 2012). in contrast, there has been little theoretical work that explains the effect of trust on vt task performance in it-enabled interpersonal relationships in the virtual environment (jarvenpaa et al., 2004). the definition of trust that has been offered by mayer et al. (1995) is: “the willingness of a party to be vulnerable to the actions of another party based on the expectation that the other will perform a particular action important to the trust or, irrespective of the ability to monitor or control that other party”. another study suggested that trust can be defined as a belief that increases the vulnerability and reliance between members and between team and their leader (baruch and lin, 2012). literature on vt shows that trust has been found as more critical in global vt, whose members are separated by location, culture, and time, than the traditional teams because of the absence of face-toface interactions (jarvenpaa et al., 2004). in a study on vt performance it is found that trust is necessary for adding value to vt performance as an important determinant of the team member’s decision for cooperation with or competition against other team members in the virtual workplace environment, where team members work independently (baruch and lin, 2012). a case study of orange group illustrates the difficulty of establishing trust in vt, even though the organization has been actively using vt. orange is a fast moving business in the highly competitive mobile communication market and a branch of france telecom and one of the uk’s leading mobile phone service providers, with services to 57 million customers across 17 countries. as a consequence, most of their traditional face-to-face team members also are involved as members of vt (lawley, 2006). the case study finds that a lack of trust among vt was a major hindrance to new product development, because collaboration among vt was often absent. the key challenge facing orange is to improve their business performance by developing and maintaining trust within their vt. the knowledge management team found that their vt were performing in several different forms of organisational structures, requiring orange to develop a framework for best practice in vt (lawley, 2006). australasian journal of information systems volume 18 number 3 2014 265 interpersonal skills in the virtual environment for effective vt, it is critical to understand the importance of the relationship between people, processes and technology (bal and gundry, 1999; ebrahim, 2009). according to pazos et al. (2012) model, and from a management and skills, the model represents an important way to enhance vt effectiveness, which discovered that teams managed effectiveness while the teams were given more constant feedback (pazos et al., 2012). therefore, the mechanisms of the collaboration and coordination of the teams are more preferred than a controlled style, which were more effective in reaching their goals (pazos et al., 2012). an important finding was that leaders were there to merely create the environment for their teams to gain the best out of their collaborative skills (pazos et al., 2012). therefore, the impact of leadership style and vt effectiveness is one of the significant organisational challenges in vt (daim et al., 2012; pazos et al., 2012; ebrahim, 2009; malhotra et al., 2007; bell and kozlowski, 2002; pauleen, 2003). in addition, the relational links among group members is a critical and a fundamental component of vt processes and their ability to exchange information that positively affects the group’s performance (daim et al., 2012; bell and kozlowski, 2002; burke and chidambaram, 1995). although, bell and kozlowski (2002) showed that vt provides the capability for more flexible organisational response, they also found that vt could also create conflicts of the roles attributed to vt members. however, understanding goals, objectives, task requirements, roles and responsibilities among team’s members lead to an effective vt (ebrahim et al., 2009; daim et al., 2012). another underlying problem with interpersonal skills mentioned by daim et al. (2012) and malhotra et al. (2007), which is organisational and cultural barriers are perhaps serious barriers as the technological barriers are among vt members to the effectiveness of vt. however, melville et al. (2004) model also showed that value is being created autonomously within each team and then processed as value through the technology resources as the human recourses are. moreover, it is critical to the leaders of vt to understand the unavoidable need to rely on technological rather than personal resources of communication (malhotra et al., 2007; daim et al., 2012; pazos et al., 2012). furthermore, powell et al. (2004) also mentioned in their study that some researchers who have investigated the impact of members’ technical expertise have found evidence of its effect on team performance. therefore, by using emerging technology, the vt environment allows working effectively and also creates competitive advantages by enabling several parties to work together under a more consultative leadership rather than a single layered team (malhotra et al., 2007; daim et al., 2012). overall, it is essential to promise a greater understanding of alignment in a virtual and traditional fashion from interpersonal skills perspective. organisational challenges in creating business value through vt we have discussed the four key organisational challenges found in the case studies on vt: effective communication, knowledge sharing, trust, and interpersonal skills in the emergent virtual environment. in discussing these organisational challenges in creating business value through the deployment of vt, it is useful to draw on the it business value model developed by melville et al. (2004). figure 3 shows this integrative model. the model was built on jay barney’s resource-based view of the firm that integrates the various findings on how business value can be created through it investments. the integrative model (melville et al., 2004) holds that it is valuable in creating business value in the organisation but the extent and dimensions of it roles are contingent on internal and external resources, including complementary organisational resources of the firm and its trading partners, as well as the competitive dynamics of macro environment. this model is particularly relevant to organize the literature on vt because, as we have discussed earlier in the literature review section, vt represent emerging boundary-less network organization forms (jarvenpaa et al., 1997) and hence they as organizational resources can potentially generate new australasian journal of information systems volume 18 number 3 2014 266 business value through leveraging dispersed knowledge and expertise beyond the traditional firm boundary. on the one hand, as we summarized our findings in table 1, vt literature identifies various types of it resources that have been deployed in the organizations to support vt. specifically, asynchronous such as lotus groupware (shachaf 2008), electronic discussion threads (malhotra et al., 2007), basic knowledge management solution (vorakulpipat et al., 2010), intranets (breu and hemingway, 2004), file transfer (paul and mcdaniel jr., 2004) and synchronous such as e-mail, telephone, videoconferencing (hertel et al., 2005; lee-kelley and sankey, 2008; anderson et al., 2007;daim et al., 2012), netmeeting (lee-kelley and sankey, 2008), team technology (anderson et al., 2007), emeetings (shachaf, 2008), remote access (breu and hemingway, 2004; pazos et al., 2012), text-based computer mediated communication (shachaf, 2008) have been found to be in use by vt in the literature. figure 3: it business value model (melville et al. 2004) on the other hand, an analysis of the literature also has shown the relatively under-developed complementary organisational resources both in the focal firm and its trading partners which have formed vt in terms of virtual team members who experienced difficulty in establishing trust (or swift trust) with other members whom they do not meet face-to-face and difficulty in effectively communicating through it resources. in consequence, despite the sufficient support of it resources, members among vt cannot easily maintain mutual knowledge and share knowledge even when their strategic tasks or projects require timely sharing of information and knowledge for effective performance. the literature on vt tends to focus research attention on the virtual team level rather than the firm level. in consequence, relatively little has been written about business processes, business process performance, and very rarely about the relationships between vt and organizational performance. in summary, while the literature on vt in general and on case studies on vt contributes to our understanding of organizational challenges in creating business value through ict-enabled vt in the organization, the literature still lacks the organizational and strategic levels of focus in solving these organizational challenges. conclusion this study has reviewed published current case study research on vt from 2004 to 2012. four key organisational issues: communication, people and skills, trust, and knowledge have been identified and australasian journal of information systems volume 18 number 3 2014 267 discussed. how an organisation capture and create business value has been argued and related to the it business value creation model of melville et al. (2004) as the most comprehensive model at this time. basically, vt environment leads to the challenges and problems that more difficult than a traditional way as without the benefits of face-to-face interactions. according to the case studies and ground concepts, it is presented that all issues depended and influenced each other. communication is an infrastructure of interaction between team members. people need special skills in leadership and technical tools usage to maximize the utilization of communication channels. trust and knowledge cannot be disregarded as no organisation can drive the performance without trust and knowledge in the team and work. overall, this study showed that business value creation through it is still a myth for both traditional and vt. more research in empirical study to prove and refine the model is necessary. this study has some research limitations, because the review is based on existing literature on virtual teams. specifically, our study has focused on organisational challenges on vt so we have reviewed the published case studies and field studies on vt. this focus has excluded prior research on vt using other research methods such as experiment, simulation, and survey. despite these limitations, it has discussed future research directions suggested by the conceptualization of virtual teams as innovative organisational forms that need to be aligned with strategic goals of the organisation and expand the existing organisational resources to create business value through organisational investments in these innovative organisational forms; ict-mediated virtual teams. references alavi, m. and leidner, d. e. 2001. “review: knowledge management and knowledge management systems: conceptual foundations and research issues,” mis quarterly (25,1), pp 107-136. alavi, m. and tiwana, a. 2002. “knowledge integration in virtual teams: the potential role of kms,” journal of the american society for information science and technology (53,12), pp 1029-1037. anderson, a. h., mcewan, r., bal, j. and carletta, j. 2007. “virtual team meetings: an analysis of communication and context,” computers in human behavior (23:5), pp 2558-2580. bal, j. and gundry, j. 1999. “virtual teaming in the automotive supply chain,” team performance management (5:6), pp 174-193. baruch, y. and lin, g. 2012. “all for one, one for all: coopetition and virtual team performance,” technological forecasting and social change (79:6), pp 1155-1168. bell, b.s. and kozlowski, s. w. j 2002. “a typology of virtual teams: implications for effective leadership,” group and organisation management (27:1), pp 14-49. breu, k. and hemingway, c.j. 2004. “making organisations virtual: the hidden cost of distributed teams,” information systems research centre (19:3), pp 191-202. burke, k. and chidambaram, l. 1995. “developmental differences between distributed and face-toface groups in electronically supported meeting environments: an exploratory investigation,” group decision and negotiation (4:3), pp 213-33. burlea, a.s. 2007. “the communication process in virtual teams,” informatica economica (11:1), pp 113-116. cascio, w.f. and shurygailo, s. 2003. “e-leadership and virtual teams,” organisational dynamics (31:4), pp 362-376. coleman, t. and chatfield, a. t. 2011. “promises and successful practice in it governance: a survey of australian senior it managers,” preceding by15th pacific asia conference on information systems, pp 1-15. australasian journal of information systems volume 18 number 3 2014 268 crampton, c. 2001. “the mutual knowledge problem and its consequences for dispersed collaboration,” organization science (12: 3), pp 346-371. daim, t.u., ha, a., reutiman, s., hughes, b., pathak, u., bynum, w. and bhatla, a. 2012. “exploring the communication breakdown in global virtual teams,” international journal of project management (30:2), pp 199-212. davenport, t.h. 2010. “process management for knowledge work,” handbook on business process management: introduction, methods and information systems, j brocke and m rosemann (eds), new york: springer, pp 17–35. drucker, p.f. 1999. “knowledge-worker productivity: the biggest challenge,” california management review (41:2), pp 79-94. ebrahim, n.a., ahmed, s. and taha, z. 2009. “virtual teams: a literature review,” australian journal of basic and applied sciences (3:3), pp 2653-2669. ginieis, m., sánchez-rebull, m.v. and campa-planas, f. 2012. “the academic journal literature on air transport: analysis using systematic literature review methodology,” journal of air transport management, (19:1), pp 31-35. henry, j.e. and hartzler, m. 1998. tools for virtual teams. milwaukee, wi: asq quality press. hertel, g.t., geister, s. and konradt, u. 2005. “managing virtual teams: a review of current empirical research,”human resource management review (15:1), pp 69-95. hollingshead, a.b., mcgrath, j.e. and o’connor, k.m. 1993. “group task performance and communication technology: a longitudinal study of computer-mediated versus face-to-face work groups,” small group research (24:1), pp307-33. jarvenpaa, s.l., knoll, k. and leidner, d.e. 1997. “is anybody out there? antecedents of trust in global virtual teams,” journal of management information systems, 14 (4), pp. 29-64. jarvenpaa, s.l. and leidner, d.e. 1999. “communication and trust in global virtual teams,” organisation science (10:6), pp 791-815. jarvenpaa, s.l., shaw, t.r. and staples, d.s. 2004. “toward contextualized theories of trust: the role of trust in global virtual teams,” information systems research (15:3), pp 250-267. kanawattanachai, p. and yoo, y. 2007. "the impact of knowledge coordination on virtual team performance over time," mis quarterly (31:4), pp 783-808. lawley, d. 2006. “creating trust in virtual teams at orange,” knowledge management review (9:2), pp. 12-17. lee-kelley, l. and sankey, t. 2008. “global virtual teams for value creation and project success: a case study,” international journal of project management (6:1), pp 51-62. malhotra, a., majchrzak, a. and renson, b. 2007. "leading virtual teams," academy of management perspectives (21:1), pp 60-70. may, a. and carter, c. 2001. “a case study of virtual team working in the european automotive industry,” international journal of indus trial ergonomics (27:3), pp 171-186. mayer, r.c., davis, j.h. and schoorman, f.d. 1995. “an integrative model of organisational trust,” academy of management review (20:3), pp 709-734. maznevski, m. and chudoba, k. 2000. “bridging space over time: global virtual team dynamics and effectiveness,” organization science (11:5), pp 473-492. melville, n., kraemer, k. and gurbaxani, v. 2004. “review: information technology and organizational performance: an integrative model of it business value,” mis quarterly, (28:2), pp 283-322. australasian journal of information systems volume 18 number 3 2014 269 moustaghfir, k. 2008. “the dynamics of knowledge assets and their link with firm performance,” measuring business excellence (12:2), pp 10-24. munkvold, b.e. and zigurs, i. 2007. “process and technology challenges in swift-starting virtual teams,” information and management (44:3), pp 287-299. paul, d.l. and mcdaniel jr., r.r. 2004. "a field study of the effect of interpersonal trust on virtual collaborative relationship performance," mis quarterly (28:2), pp183-227. pazos, p. 2012. “conflict management and effectiveness in virtual teams,” team performance management (18:7), pp 401-417. powell, a., piccoli, g. and ives, b. 2004. “virtual teams: a review of current literature and directions for future research,” the data base for advances in information systems (35:1), pp 6-36. rich, m. 1997. “a learning community on the internet: an exercise with masters students,” proceedings of the third annual conference of the ais, indianapolis, in. rosen, b., furst, s. and blackburn, r. 2007. “overcoming barriers to knowledge sharing in virtual teams,” organisational dynamics (36:3), pp 259-273. shachaf, p. 2008. “cultural diversity and information and communication technology impacts on global virtual teams: an exploratory study,” information and management (45:2), pp 131142. shin, y. 2004. “a person-environment fit model for virtual organizations,” journal of management, (30:5), pp 725-743. vorakulpipat, c., rezgui, y. and hopfe, c.j. 2010. “value creating construction virtual teams: a case study in the construction sector,” automation in construction (19:2), pp 142-147. webster, j. and watson, r.t. 2002. “analyzing the past to prepare for the future: writing a literature review,” mis quarterly (26:2), pp xiii-xxiii. an earlier version of this paper was presented at the australasian conference on information systems (acis) 2013 in melbourne, australia. paper title (paper title style) australasian journal of information systems fletcher & islam 2018, vol 22, research article comparing sets of patterns with the jaccard index 1 comparing sets of patterns with the jaccard index sam fletcher school of computing and mathematics, charles sturt university, bathurst, australia sam.pt.fletcher@gmail.com md zahidul islam school of computing and mathematics, charles sturt university, bathurst, australia abstract the ability to extract knowledge from data has been the driving force of data mining since its inception, and of statistical modelling long before even that. actionable knowledge often takes the form of patterns, where a set of antecedents can be used to infer a consequent. in this paper we offer a solution to the problem of comparing different sets of patterns. our solution allows comparisons between sets of patterns that were derived from different techniques (such as different classification algorithms), or made from different samples of data (such as temporal data or data perturbed for privacy reasons). we propose using the jaccard index to measure the similarity between sets of patterns by converting each pattern into a single element within the set. our measure focuses on providing conceptual simplicity, computational simplicity, interpretability, and wide applicability. the results of this measure are compared to prediction accuracy in the context of a real-world data mining scenario. keywords: machine learning; metrics; data mining; patterns; rules; utility measures; quality evaluation. 1 introduction the discovery of patterns in data is the cornerstone of data mining; sometimes for predicting the future, and other times for extracting meaning. it is the latter case that this paper will concern itself with. by extracting meaning, statisticians and data analysts are able to elevate otherwise meaningless data into usable information; information that can be acted on or used to discover knowledge (han et al. 2006). the extraction of meaning from data is most often explicated as mining patterns from data, preferably in a form that is interpretable by humans. many branches of data mining have concerned themselves over the years with this endeavour, not limited to frequent pattern (itemset) mining (han et al. 2007), decision trees (quinlan 1993) and forests (breiman 2001a), and association rule mining (ordonez & zhao 2011). constructing models from data in order to make accurate predictions is useful, but it is important to remember that the accuracy of a model does not guarantee the truthfulness of the patterns contained in the model. the history of science is littered with untruthful models that accurately predicted observations, from bohr's model of the atom to the geocentric model of the solar system. assessing the patterns discovered by data mining algorithms is often overlooked in the current zeitgeist, instead favouring a dogged pursuit of higher prediction accuracy (hand 2006, wagstaff 2012). we argue that this is to the detriment of the discipline and provide a simple but powerful method for comparing sets of patterns, aiding a renewed focus in pattern assessment seen in recent years (aodha et al. 2014, letham et al. 2013). australasian journal of information systems fletcher & islam 2018, vol 22, research article comparing sets of patterns with the jaccard index 2 two terms warrant further explanation: data and patterns. when we talk about data, we are talking about the contents of a dataset x, where a dataset is a two-dimensional matrix of rows and columns, with each cell in the dataset containing a single datum (i.e., piece of data, or value). each row r represents the record of a unique participant in the dataset, and each column a represents an attribute or feature of the participants. the value v that a record r has for attribute a is the value that corresponds to some real-life fact about the participant. an attribute a can use either discrete (i.e., categorical or unordered) values, or continuous (i.e., numerical or ordered) values. we write ra when referring to the value that r has for attribute a. we use n to denote the number of records in the whole dataset, and m to denote the number of attributes. by “pattern”, we refer to a set of criteria, where if a record meets all of the criteria, allow an analyst to infer additional information about the record that was not previously known. more formally, patterns take the form ψ→c∈c. that is to say, if criteria ψ is met, attribute c is predicted to equal c. c is known as the consequent in this context, but is functionally the same as a class attribute. each record r has a class attribute, referred to as rc. functioning as the antecedent, ψ is a set of conditions for a subset of the m attributes used in the data, where each attribute a has values v∈a. conditions take the form a=v, or can use other operators such as a>v if a is ordered. an example would be {𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸 = 𝑃𝑃ℎ𝐷𝐷,𝐴𝐴𝐴𝐴𝐴𝐴 > 45} → 𝐼𝐼𝐸𝐸𝐸𝐸𝐸𝐸𝐼𝐼𝐴𝐴 = 𝐻𝐻𝐸𝐸𝐴𝐴ℎ . another example could be {𝐼𝐼𝐸𝐸𝑚𝑚𝑚𝑚, 𝐴𝐴𝐴𝐴𝐴𝐴𝑒𝑒} → {𝑏𝑏𝑏𝑏𝐴𝐴𝐸𝐸𝐸𝐸} , where milk and eggs can be thought of as boolean (binary) attributes. other examples can be seen in table 1. patterns in any of these forms are sometimes referred to as decision rules or association rules, and when collected together can be called a decision list or rule list (letham et al. 2013).1 𝒊𝒊 𝝍𝝍𝒊𝒊 𝒄𝒄 0 𝐶𝐶𝑚𝑚𝐸𝐸𝐼𝐼𝐶𝐶 𝑇𝑇ℎ𝐸𝐸𝐸𝐸𝑚𝑚𝐸𝐸𝐴𝐴𝑒𝑒𝑒𝑒 ≤ 5.5 𝐴𝐴𝐴𝐴𝐷𝐷 𝑈𝑈𝐸𝐸𝐸𝐸𝑈𝑈𝐸𝐸𝑏𝑏𝐼𝐼𝐸𝐸𝐸𝐸𝑈𝑈 𝐸𝐸𝑈𝑈 𝐶𝐶𝐴𝐴𝑚𝑚𝑚𝑚 𝑆𝑆𝐸𝐸𝑆𝑆𝐴𝐴 ≤ 2.5 𝐴𝐴𝐴𝐴𝐷𝐷 𝐵𝐵𝐸𝐸𝑏𝑏𝐴𝐴 𝐴𝐴𝐸𝐸𝐸𝐸𝑚𝑚𝐴𝐴𝐸𝐸 ≤ 4.5 benign 1 𝑈𝑈𝐸𝐸𝐸𝐸𝑈𝑈𝐸𝐸𝑏𝑏𝐼𝐼𝐸𝐸𝐸𝐸𝑈𝑈 𝐸𝐸𝑈𝑈 𝐶𝐶𝐴𝐴𝑚𝑚𝑚𝑚 𝑆𝑆𝐸𝐸𝑆𝑆𝐴𝐴 > 2.5 𝐴𝐴𝐴𝐴𝐷𝐷 𝑈𝑈𝐸𝐸𝐸𝐸𝑈𝑈𝐸𝐸𝑏𝑏𝐼𝐼𝐸𝐸𝐸𝐸𝑈𝑈 𝐸𝐸𝑈𝑈 𝐶𝐶𝐴𝐴𝑚𝑚𝑚𝑚 𝑆𝑆ℎ𝐸𝐸𝐶𝐶𝐴𝐴 ≤ 2.5 benign 2 𝐶𝐶𝑚𝑚𝐸𝐸𝐼𝐼𝐶𝐶 𝑇𝑇ℎ𝐸𝐸𝐸𝐸𝑚𝑚𝐸𝐸𝐴𝐴𝑒𝑒𝑒𝑒 > 6.5 𝐴𝐴𝐴𝐴𝐷𝐷 𝑈𝑈𝐸𝐸𝐸𝐸𝑈𝑈𝐸𝐸𝑏𝑏𝐼𝐼𝐸𝐸𝐸𝐸𝑈𝑈 𝐸𝐸𝑈𝑈 𝐶𝐶𝐴𝐴𝑚𝑚𝑚𝑚 𝑆𝑆𝐸𝐸𝑆𝑆𝐴𝐴 > 2.5 𝐴𝐴𝐴𝐴𝐷𝐷 𝑈𝑈𝐸𝐸𝐸𝐸𝑈𝑈𝐸𝐸𝑏𝑏𝐼𝐼𝐸𝐸𝐸𝐸𝑈𝑈 𝐸𝐸𝑈𝑈 𝐶𝐶𝐴𝐴𝑚𝑚𝑚𝑚 𝑆𝑆𝐸𝐸𝑆𝑆𝐴𝐴 ≤ 4.5 𝐴𝐴𝐴𝐴𝐷𝐷 𝑈𝑈𝐸𝐸𝐸𝐸𝑈𝑈𝐸𝐸𝑏𝑏𝐼𝐼𝐸𝐸𝐸𝐸𝑈𝑈 𝐸𝐸𝑈𝑈 𝐶𝐶𝐴𝐴𝑚𝑚𝑚𝑚 𝑆𝑆ℎ𝐸𝐸𝐶𝐶𝐴𝐴 > 2.5 𝐴𝐴𝐴𝐴𝐷𝐷 𝐵𝐵𝐸𝐸𝑏𝑏𝐴𝐴 𝐴𝐴𝐸𝐸𝐸𝐸𝑚𝑚𝐴𝐴𝐸𝐸 > 2.5 𝐴𝐴𝐴𝐴𝐷𝐷 𝑀𝑀𝐸𝐸𝐸𝐸𝐸𝐸𝑒𝑒𝐴𝐴𝑒𝑒 ≤ 1.5 malignant 3 𝑈𝑈𝐸𝐸𝐸𝐸𝑈𝑈𝐸𝐸𝑏𝑏𝐼𝐼𝐸𝐸𝐸𝐸𝑈𝑈 𝐸𝐸𝑈𝑈 𝐶𝐶𝐴𝐴𝑚𝑚𝑚𝑚 𝑆𝑆𝐸𝐸𝑆𝑆𝐴𝐴 > 4.5 𝐴𝐴𝐴𝐴𝐷𝐷 𝑈𝑈𝐸𝐸𝐸𝐸𝑈𝑈𝐸𝐸𝑏𝑏𝐼𝐼𝐸𝐸𝐸𝐸𝑈𝑈 𝐸𝐸𝑈𝑈 𝐶𝐶𝐴𝐴𝑚𝑚𝑚𝑚 𝑆𝑆ℎ𝐸𝐸𝐶𝐶𝐴𝐴 > 2.5 𝐴𝐴𝐴𝐴𝐷𝐷 𝐵𝐵𝑚𝑚𝐸𝐸𝐸𝐸𝐸𝐸 𝐶𝐶ℎ𝑏𝑏𝐸𝐸𝐼𝐼𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸 > 4.5 malignant table 1 – some examples of patterns discovered by cart in the wbc dataset. 1 for example, a decision tree might be “flattened” so as to no longer have roots or leaves, and it merely becomes an unordered set (a rule list) of unordered sets of attribute conditions (rules). the decision tree would then be indistinguishable from a rule list. australasian journal of information systems fletcher & islam 2018, vol 22, research article comparing sets of patterns with the jaccard index 3 1.1 problem statement consider the following question: given two sets of patterns ψ1 and ψ2, how similar are they? to the best of our knowledge, this question has yet to be answered in the literature, and is answered in this paper. the ability to answer this question has clear benefits in many areas of data mining and knowledge discovery, such as: • comparing different classifiers, or classifiers with different parameters (islam & giggins 2011, shotton et al. 2013); • comparing patterns discovered manually by statisticians, to patterns discovered with machine learning techniques (breiman 2001b); • comparing the quality of the patterns discovered in data before and after applying privacy-preserving techniques to the data (fletcher & islam 2014, 2015a, friedman & schuster 2010); and • finding differences in different samples of data, including temporal scenarios with time series (baron et al. 2003). answering the above question requires a measurement of some kind, and it is a measure (more specifically, a metric) that we propose in this paper. we convert the patterns in ψ1 and ψ2 into discrete elements, and take advantage of the jaccard index to measure the similarity between sets made up of these elements. this is discussed in full in section 3. in order to be a useful measure, we consider several external factors to be part of the problem statement. these are factors that often determine whether researchers and data miners wilfully choose to use a measure: • conceptual simplicity; • computational simplicity; • interpretability; and • wide applicability. we discuss how our proposed measure fulfils these external factors in section 3. section 2 provides additional background information and related work. in section 4, we demonstrate our measure in action with two real-world scenarios: one in which a user wishes to compare two classifiers; and one where a user wants to compare to privacy-preserving data mining algorithms. we conclude with section 5. 2 related work patterns play a key role in the knowledge discovery and decision-making process. several fields of machine learning – notably frequent pattern mining (han et al. 2007, 2000) – focus specifically on finding patterns. fields such as classification can also find patterns by using decision forests (breiman 2001a) or other classifiers (han et al. 2006). patterns can be assessed for their usefulness (geng & hamilton 2006) and their interpretability (letham et al. 2013), or monitored for any changes in temporal scenarios (baron et al. 2003). while differing in methodology, approaches such as these agree on the importance of patterns and attest to the value of patterns for gaining knowledge. australasian journal of information systems fletcher & islam 2018, vol 22, research article comparing sets of patterns with the jaccard index 4 it is important to note that assessing the quality of patterns in these ways is not the same as measuring the performance of a model at achieving a goal (caruana & niculescu-mizil 2004, sokolova & lapalme 2009). prediction accuracy is often used to measure a model’s performance (cheng et al. 2007, letham et al. 2013), but is disconnected from any reliable assessments of pattern quality (fletcher & islam 2014, islam et al. 2003). a user should be aware of the specific goals they wish their model to achieve and how important the truthfulness (kifer & gehrke 2006) or interpretability of the patterns in the model are, and then use multiple measures to assess if their needs are met. our proposed application of the jaccard index assesses if two sets of patterns are similar. attempts to measure the similarity between trusted patterns and newly discovered patterns have been made in the past (islam & brankovic 2011), but the measure used was designed for a specific problem, lacking any applicability in a wider context. we discuss the value of widely applicable measures in section 3.3. the field of privacy-preserving data mining (dwork 2008, fung et al. 2010) is known for struggling with the inherent trade-off that must be made between the amount of privacy provided and the quality of the perturbed data (fung et al. 2005, nergiz & clifton 2007). a naive approach would be to use measures like rmse to compare the original data to the perturbed data directly (willmott 1982, willmott et al. 2009), however this ignores the correlations in the data necessary for information discovery (agrawal & aggarwal 2001, fletcher & islam 2015b). just like how frequent pattern mining and other fields use prediction accuracy heavily, so too does privacy-preserving data mining (chaudhuri et al. 2011, friedman & schuster 2010, fung et al. 2005, mohammed et al. 2011), but the same problems encountered by the former when assessing patterns also plague the latter (fletcher & islam 2014, islam et al. 2003). attempts at directly measuring the loss of pattern retention as privacy needs are increased have been made (fletcher & islam 2014), but the measure is not applicable in any wider context outside of privacy preservation. there is therefore a need for insightful ways to compare between patterns gained by different means, regardless of what data mining algorithms or data are used. we do so in this paper, taking both practical and mathematical considerations into account (meila 2007). 3 comparing sets of patterns with the jaccard index the jaccard index (jaccard 1901) is a well-known measurement of the similarity between two sets s and t, defined as the size of the intersection divided by the size of the union of the two sets: 𝐽𝐽(𝑆𝑆,𝑇𝑇) = |s ∩ t| |s ∪ t| , where we say j(s,t)=1 if |s∪t|=0. by reinterpreting a set of patterns as a set of elements, we are able to use the jaccard index to measure the similarity between two sets of patterns. in section 3.1 we describe how we convert patterns in the sets ψ1 and ψ2 into elements for sets s and t. in sections 3.3 and 3.4 we outline the practical and mathematical benefits of using the jaccard index to measure the difference between two sets of patterns. 3.1 converting patterns into elements in a set our aim is to compare two sets of patterns and describe their similarities with an intuitive, quantitative number. to do so with the jaccard index, we must first translate each pattern ψ→c australasian journal of information systems fletcher & islam 2018, vol 22, research article comparing sets of patterns with the jaccard index 5 (such as those seen in table 1) into element s of set s. each antecedent ψ is made up of attributes – continuous, discrete or binary attributes – that specify the conditions that must be met in order for the consequent c to be predicted to equal c. to condense the set ψ as well as c into a single element s, we use the following equation: 𝑒𝑒 = 𝟏𝟏𝜓𝜓(𝐴𝐴1),𝟏𝟏𝜓𝜓(𝐴𝐴2), … ,𝟏𝟏𝜓𝜓(𝐴𝐴𝑚𝑚), 𝐸𝐸𝜓𝜓 where we use cψ to simply refer to the class value (consequent) of pattern ψ, and 1ψ(a) is the indicator function: 𝟏𝟏𝜓𝜓(𝐴𝐴) ∶= �1 𝐴𝐴 ∈ 𝜓𝜓 0 𝐴𝐴 ∉ 𝜓𝜓 . essentially, equation 2 is recording the presence or absence of each attribute in pattern ψ, as well as recording the consequent c. table 2 illustrates this with converted versions of the patterns seen in table 1. the three 1's in ψ0 refer to the positions in the wbc dataset of the three attributes used by ψ0 in table 1, and similarly for the other patterns. we include the consequent in s because of the role it plays in the definition of a pattern – without a consequent, an antecedent hardly means anything at all. the conversion described in equation 2 is lossy when used on non-binary attributes, but is lossless for binary attributes. while there are certainly disadvantages to being lossy with some attributes, there are also several advantages, such as being able to treat categorical and numerical attributes equally, without bias. for example, weather=sunny is clearly a different pattern to weather=cloudy, but what about weather=27.85° compared to weather=27.84°? heuristics could be developed to handle these cases, but ultimately it still relies on heuristics. instead, our approach asks the question “does weather make up part of the pattern?” – if it is, then it is more similar to another pattern that also cares about weather than it is to a pattern that does not care about weather. experimentally, it is found in section 4 that making these less granular comparisons such as “do the two patterns both care about weather?” does a good job of distinguishing between two sets of patterns. simultaneously, the proposed approach maintains a lossless conversion for patterns with binary attributes such as {milk, eggs}→bread. 𝒊𝒊 𝒔𝒔𝒊𝒊 0 110001000a 1 011000000a 2 111001001b 3 011000100b table 2 the encoded versions of the patterns shown in table 1. 3.2 steps the steps for calculating the similarity between two sets of patterns are as follows: 1. take ψ1 and ψ2 as input. 2. convert ψ1 and ψ2 into sets s and t respectively, using the process in section 3.1. 3. for each element in s, scan t to see if that element exists in t. the number of pairs found equals |s∩t|. australasian journal of information systems fletcher & islam 2018, vol 22, research article comparing sets of patterns with the jaccard index 6 4. deduce the number of unpaired elements in s and t using the results from step 3. this number plus |s∩t| equals |s∪t|. 5. calculate j(s,t). 3.3 practical considerations as discussed in section 1.1, a good measure should not be difficult for data analysts to harness effectively. we assess our proposed application of the jaccard index with four factors that influence an analyst's decision to use a measure: 3.3.1 conceptual simplicity a good measure is one that can be easily and intuitively understood. if a measure has too many moving parts or variables, it can quickly become a “black box” of sorts, where analysts can no longer conceptualize all the possible outputs that the measure could produce. our implementation of the jaccard index avoids these risks by being very straight-forward – it is the number of patterns two sets of patterns share, divided by the number of unique patterns across both sets. there are no variables beyond the two sets of patterns, and there are no parameters that need expert knowledge to properly adjust. our encoding of the patterns into single elements is simple and intuitive, making the conceptualization of the processes involved in the measure no more difficult than in a standard application of the jaccard index. incorporating attribute value conditions into the encoding process would require a robust definition of “similarity” or “distance” for both continuous and discrete attribute values that did not induce any biases in the calculation, and would undoubtedly increase the conceptual complexity of the measure. this is one avenue for future work, but currently appears to be infeasible due to the fundamental differences between continuous and discrete attributes. 3.3.2 computational simplicity a measure can become infeasible if it does not scale well as the variables become large. the computation time of our measure is very satisfactory, mostly due to the fact that it does not use the underlying raw data in any way, or even a classifier – it just needs the sets of patterns. encoding all the patterns in ψ1 and ψ2 as the sets s and t requires each pattern in both sets to be read and converted once, with the presence of each attribute being checked once in each pattern. therefore the computational complexity is ο(𝐼𝐼(|ψ1| + |ψ2|)). once s and t have been generated, the intersection and union in the jaccard index can both be calculated by comparing each element in s to every element in t, where each element is m digits long. this gives a computational complexity of ο(𝐼𝐼 ∙ |𝑆𝑆| ∙ |𝑇𝑇|). since the length of the sets of patterns and the respective sets of encoded elements will always be equal, the total computation time of our measure is ο(𝐼𝐼(|ψ1| + |ψ2| + |ψ1| ∙ |ψ2|)). as a rough example, classification algorithms such as random forest (breiman 2001a) might generate several hundred patterns, and the kinds of datasets that random forest might be applied to generally have less than 1000 attributes. 3.3.3 interpretability in order for analysts to judge the result of a measure and act on it in a meaningful way, the result needs to be interpretable. this can be as simple as being able to parse the result into a sentence. in our case, our measure can be interpreted as “out of all the patterns that appear in sets of patterns ψ1 and ψ2, (j(s,t)×100)% can be found in both sets”. an addendum could be added about how the patterns are compared at a level of granularity that ignores differences in attribute value conditions, without making the measure any more difficult to understand and interpret. australasian journal of information systems fletcher & islam 2018, vol 22, research article comparing sets of patterns with the jaccard index 7 3.3.4 wide applicability measures that accurately encapsulate specific scenarios are undeniably useful, but measures capable of crossing academic discipline boundaries are far more appealing. they provide ways for researchers and professionals to interface with work outside of their personal scope and build connections between otherwise isolated fields of science. historically, the statistics and machine learning disciplines have learned this lesson the hard way, with the relatively new field of machine learning often recreating mathematics and methods invented previously by statisticians (breiman 2001b). the ubiquity of prediction accuracy as a measure of performance and the role it plays in creating dialogue among researchers demonstrates the advantages of measures with wide applicability. our proposed encoding method and application of the jaccard index is general enough to be viable in any situation involving two groups of patterns discovered from data with the same attributes. our measure is completely independent of how the patterns were discovered, built, or connected. it can handle non-binary consequents – a common constraint for other measures (felkin 2007). it can handle continuous consequents as well, but only if the range of values are first discretised into “buckets” (kotsiantis & kanellopoulos 2006). it can even work in scenarios without a c component, where ψ might merely represent a collection of attributes that appear together frequently. in this situation, s would intuitively drop the c component and become equal to 𝟏𝟏𝜓𝜓(𝐴𝐴1),𝟏𝟏𝜓𝜓(𝐴𝐴2), … ,𝟏𝟏𝜓𝜓(𝐴𝐴𝑚𝑚). our measure is conceptually and computationally simple, it produces easily interpretable results, and it is an appropriate measure in a wide variety of scenarios. not constraining ourselves by requiring definitions of similarity or distance for continuous and discrete attribute value conditions allows these qualities to be improved even further. 3.4 mathematical properties a good measure should be capable of distinguishing between slightly different sets of patterns and very different sets of patterns. the jaccard index possess several properties that make this possible, as well as possessing properties that strengthen the versatility of the measure. our encoding method is designed in a way that does not interfere with these properties. 3.4.1 bounds the jaccard index has the bounds 0≤j(s,t)≤1. we can narrow down the upper bound further by using the difference in size between the sets. in situations where |s|>|t|, the maximum similarity is when t⊆s, in which case j(s,t)=|t|/|s|. the larger the size difference between the sets, the smaller the upper bound is. to put it formally: 𝐽𝐽(𝑆𝑆,𝑇𝑇) ≤ |𝑇𝑇| |𝑆𝑆| ,𝑤𝑤ℎ𝐴𝐴𝑏𝑏𝐴𝐴 |𝑆𝑆| ≥ |𝑇𝑇| . it is reasonable to assume that the user will know the size of sets, making this upper bound easy to incorporate when interpreting the measure's result, and possibly being quite informative. a similar situation exists for the popular prediction accuracy measure, where the lower bound is equal to the relative frequency of the most common class value.2 if the flexibility of the jaccard index's upper bound is undesirable, a user is free to shorten s until both sets have equal length. the frequent pattern mining discipline does this often, only 2 theoretically it can go lower, but then the model is providing no benefit to the user and is actually causing harm – it is worse than a random guess. australasian journal of information systems fletcher & islam 2018, vol 22, research article comparing sets of patterns with the jaccard index 8 selecting patterns with high support (han et al. 2007, 2000). many other measures could be used to remove the least valuable patterns depending on how the user defines “valuable” (geng & hamilton 2006). a user could also divide the jaccard index result by the upper bound and reinterpret the result as “the ratio between the actual result and the best possible result”, but this approach could easily misguide the user about how similar the sets of patterns actually are and is not recommended. the lower bound can also be narrowed in very specific scenarios: when the number of patterns in s and t is more than the number of unique patterns that could exist. this indicates that |s∩t|>0; some overlap between the sets must exist. this can occur when the number of unique patterns that could exist given the number attributes and class values in the dataset (i.e. the number of ways you could write s, |𝐶𝐶|(2𝑚𝑚 − 1) is less than |s|+|t|.3 4 it is also possible that the pattern creation process puts a constraint k on the number of attributes that could be present in any one pattern (webb & brain 2002). it has been demonstrated that increased pattern length can actually decrease the information gained due to the decreased generality of the patterns (cheng et al. 2007), as well as decreasing the interpretability of the patterns (freitas 2013, huysmans et al. 2011, letham et al. 2013, vellido et al. 2012). note that the data mining processes used to generate ψ1 and ψ2 might have different k's, denoted kψ1 and kψ2, but the combinations possible with a smaller k are a subset of a larger k's combinations. we can write the number of combinations possible in a scenario with constraint k as ∑ �𝑚𝑚𝑖𝑖 � 𝑘𝑘 𝑖𝑖=1 , where �𝑚𝑚𝑖𝑖 � is the binomial coefficient and equals 𝐼𝐼!/𝐸𝐸! (𝐼𝐼 − 𝐸𝐸)!. strictly speaking, the lower bound of our measure is 𝐽𝐽(𝑆𝑆,𝑇𝑇) ≥ 𝐼𝐼𝐸𝐸𝑚𝑚 � |𝑆𝑆| + |𝑇𝑇| − |𝐶𝐶| ∙ ∑ �𝑚𝑚𝑖𝑖 � 𝑘𝑘 𝑖𝑖=1 |𝐶𝐶| ∙ ∑ �𝑚𝑚𝑖𝑖 � 𝑘𝑘 𝑖𝑖=1 , 0� , where 𝑚𝑚 = 𝐼𝐼𝐸𝐸𝑚𝑚(𝑚𝑚ψ1 ,𝑚𝑚ψ1) ≤ 𝐼𝐼. if k=m (i.e. no constraint was put on the length of the patterns), |𝐶𝐶| ∙�� 𝐼𝐼 𝐸𝐸 � = |𝐶𝐶|(2𝑚𝑚 − 1) . 𝑚𝑚 𝑖𝑖=1 to use some example numbers, if |s|+|t|=100 and there are five attributes and two class values, there are 2(25-1)=62 possible combinations of attributes and class values. the lower bound then becomes 0.61, ruling out over half the original range. we reiterate that in most situations, the lower bound will simply equal 0. these simple upper and lower bounds allow the user to interpret the measure's results with far more insight than would be possible if the bounds were ignored. 3.4.2 metric properties metrics are a subset of measures, defined by four mathematical properties they possess: nonnegativity; identity of indiscernibles; symmetry; and triangle inequality. we can describe each of these, respectively, with the jaccard index: • j(s,t)≥0 • j(s,t)=1⇔s=t 3 we subtract one from 2m because an empty ψ is not a legal antecedent. 4 note that |s∪t|≤|s|+|t|. australasian journal of information systems fletcher & islam 2018, vol 22, research article comparing sets of patterns with the jaccard index 9 • j(s,t)=j(t,s) • 𝐽𝐽(𝑆𝑆,𝑈𝑈) ≥ 𝐽𝐽(𝑆𝑆,𝑇𝑇) + 𝐽𝐽(𝑇𝑇,𝑈𝑈). note that we use “≥” since the jaccard index is traditionally a metric of similarity, not distance. if a distance is preferred, the jaccard distance can be very easily used: 𝐸𝐸𝑗𝑗(𝑆𝑆,𝑇𝑇) = 1 − 𝐽𝐽(𝑆𝑆,𝑇𝑇). it is straight-forward to see how the jaccard index satisfies the first three properties, since its maximum bounds are 0 ≤ 𝐽𝐽(𝑆𝑆,𝑇𝑇) ≤ 1 and equation 1 does not change if s and t are swapped. a proof of the jaccard index satisfying the triangle inequality has been previously constructed (levandowsky & winter 1971, lipkus 1999). because the jaccard index has these properties, mathematicians can use them when constructing proofs and can use more assumptions that are guaranteed to hold true. the triangle inequality especially is well-known as a strong mathematical property when analysing metrics (chawla et al. 2005) and designing efficient data structures and algorithms (meila 2007). measures that are not metrics often output results that require unintuitive interpretations (meila 2007, willmott et al. 2009). 4 experiments below, we explore two example scenarios of how our proposed measure can be used as a complementary measure to prediction accuracy, discovering new information that prediction accuracy cannot tell us. in section 4.1 we compare two variations of the cart algorithm in terms of pattern similarity and prediction accuracy. in section 4.2 we use the same two measures, but this time we compare two privacy-preserving decision tree algorithms. 4.1 scenario 1: comparing classifiers in this section we perform a short case study on one of the example scenarios given in section 1.1: comparing classifiers with different parameters. specifically, we use the implementation of the cart classifier (breiman et al. 1984) found in the scikit-learn software (pedregosa et al. 2011), and we compare two different objective functions for the splitting criteria: the gini index (breiman et al. 1984) and information gain (quinlan 1996). this comparison represents a reasonably straight-forward scenario where a common question is being asked: “how different would our results be if we changed classifier?” by using single decision trees (rather than decision forests or other classifiers), we generate a manageable number of patterns directly from the classifier, rather than needing to filter a larger number of patterns down using additional processes. by using a simple experimental set-up, our results are easily reproducible. australasian journal of information systems fletcher & islam 2018, vol 22, research article comparing sets of patterns with the jaccard index 10 dataset jaccard index jaccard upper bound prediction accuracy difference wbc 0.077 6/8=0.750 0.007 vehicle 0.184 21/24=0.875 0.011 banknotes 0.750 7/7=1.000 0.027 redwine 0.125 21/24=0.875 0.013 spambase 0.100 21/23=0.913 0.008 pageblocks 0.546 14/16=0.889 0.004 optdigits 0.019 26/29=0.897 0.022 penwritten 0.000 21/24=0.875 0.020 gammatele 0.310 18/20=0.900 0.005 shuttle 0.714 6/6=1.000 0.000 credit 0.136 12/13=0.920 0.014 yeast 0.364 14/16=0.875 0.004 cardio 0.143 7/9=0.778 0.007 adult 0.400 17/18=0.944 0.001 bank 0.172 17/17=1.000 0.008 tictactoe 0.161 17/19=0.895 0.004 car 0.400 6/8=0.750 0.028 nursery 0.615 10/11=0.909 0.000 chess 0.476 14/17=0.824 0.005 table 3 the jaccard index and prediction accuracy difference between cart-g and cart-i for 19 datasets. for this experiment, patterns in the form ψ→c are extracted from the decision trees by “flattening” each root-to-leaf path. that is, a chain of nodes from the root node to a leaf node (where the most common class value in the leaf node is the predicted class value) forms a pattern. each leaf node is part of one pattern. root-to-leaf paths are an ordered list, and the same attribute may appear multiple times, but these properties disappear when a root-to-leaf path is flattened into a list of conditions ψ. we apply the cart classifier on 19 datasets from the uci machine learning repository (bache & lichman 2013) using five-fold cross-validation with stratified folds5, and measure the jaccard index between cart with the gini index (referred to as cart-g) and cart with information gain (referred to as cart-i). we also measure the prediction accuracy of both cart-g and cart-i on the test data from the unused fold. our aim is to see whether the user could learn anything new by going beyond a simple comparison of prediction accuracies and also comparing the classifiers with the jaccard index. for each dataset in our experiments, the minimum support threshold of each leaf in the decision trees is 2% of the records. the lower bound of the jaccard index is 0 for all datasets. the lower bound for prediction accuracy depends on the relative frequency of the majority class label in each dataset. the upper bound for the jaccard index depends on the number of patterns found with each classifier and is reported along with the jaccard index results in table 3. since our question is “how different would our results be if we changed classifier?”, we 5 stratified folds have the same distribution of class values as the dataset as a whole. australasian journal of information systems fletcher & islam 2018, vol 22, research article comparing sets of patterns with the jaccard index 11 report the absolute difference between the prediction accuracy of cart-g and cart-i, also in table 3. we display the same information as a scatter plot in figure 1, where we graph the jaccard index and prediction accuracy difference for each dataset as a point in 2d space. figure 1 the jaccard index compared to the prediction accuracy difference for each dataset. note the scope of the axes – the maximum prediction accuracy difference is small, while the jaccard index results make use of the full range between the bounds. we can see that the detected differences in prediction accuracy are small – too small to trust that the detected difference is solely due to the classifiers (hand 2006), or that the result is precise enough to remain consistent after repeated runs of the experiment (fletcher & islam 2014). if a user found themselves in the scenario described above, the only thing they could reasonably conclude from the prediction accuracy results is “the two classifiers are very similar”. however the jaccard index results inform the user that this is not true at all – the patterns uncovered by the two classifiers are similar for some datasets, but on others they can have very few patterns in common. it turns out that while there is almost no difference in prediction accuracy when choosing either the gini index or information gain, the structure of the trees can be very different for some datasets! the jaccard index allows the user to learn information such as “only 40% of the patterns in the adult dataset found by either classifier were found by both classifiers”, while prediction accuracy tells the user “for the adult dataset, the two classifiers differ by only a tenth of a percent when predicting the consequent of unseen records”. our proposed measure does not replace prediction accuracy, but instead provides information that prediction accuracy (or any other measure, to the best of our knowledge) is incapable of providing. australasian journal of information systems fletcher & islam 2018, vol 22, research article comparing sets of patterns with the jaccard index 12 4.2 scenario 2: comparing anonymization techniques in this section we perform another case study on one of the example scenarios given in section 1.1; this time we compare two privacy-preserving data mining algorithms. both are decision tree algorithms that use differential privacy to ensure that the trees do not leak personal information, but they do so in different ways. we refer the reader to the respective papers for details of the algorithms if they are so inclined: fi (fletcher & islam 2015a) and fs (friedman & schuster 2010). figure 2 the similarity between the sets of patterns produced by fi and cart, compared to the similarity between the patterns produced by fs and cart. we compare fi to fs using the jaccard index in figure 2. here, we are measuring how many patterns discovered by fi are also discovered by the (non-private) cart-g algorithm (breiman et al. 1984), and then doing the same with fs. we build one tree with each algorithm6, extract all the root-to-leaf paths in each tree, convert the paths into elements of a set as described in section 3.1, and measure their similarity to each other. this process was performed on five datasets and repeated with five-fold cross-validation with stratified folds, and figure 2 presents the aggregated results. by using the jaccard index, we learn that fi finds more of the patterns discovered by cart (where privacy requirements are not perturbing the model) than fs does. since the patterns discovered by cart are not perturbed, we can trust that they are more representative of real-world phenomena than patterns discovered under privacy restrictions. 6 the same parameters are used for each tree: maximum depth of δ=5; privacy budget of ε=2; forest size of τ=1. 0.0 0.2 0.4 0.6 0.8 car connect4 mushroom nursery tic-tac-toe ja cc ar d in de x fi vs. cart fs vs. cart australasian journal of information systems fletcher & islam 2018, vol 22, research article comparing sets of patterns with the jaccard index 13 figure 3 the prediction accuracy of a tree built using fi (fletcher & islam 2015a), fs (friedman & schuster 2010) and cart (breiman et al. 1984). accompanying figure 2, we also provide the prediction accuracy of the three algorithms in figure 3. here we confirm that cart achieves higher accuracy, as expected for a non-private algorithm, and discover that fi outperforms fs in terms of prediction accuracy. by using two different measures, and learning different things from each of them, the usefulness of having workload-specific measures becomes clear. no measure can claim to be a “one size fits all” solution for quantifying what the user is interested in, and our jaccard index measure is no exception. it does, however, perform well as a “one size fits some” solution (as is true of any good measure), and informs us about the quality of the patterns discovered by different algorithms. 5 conclusion in this paper we propose a method for measuring the similarity between two sets of patterns. the method was designed with a focus on conceptual simplicity, computational simplicity, interpretable results, and applicability in a wide variety of scenarios. one such scenario was explored in detail to demonstrate the information a user could learn from our proposed measure. strong mathematical properties are provided to aid users in interpreting their results and making comparisons between results. for example if a third classifier was added to the scenario portrayed in section 4, the triangle inequality allows users to use their intuition that the similarity between classifier a and c cannot be lower than the sum of the similarities between classifiers a and b and classifiers b and c. our use of the jaccard index successfully measures an aspect of data mining results that no pre-existing measure is able to do. as the data mining community continues to put more focus on the discovery of interpretable patterns in data, the ability to distinguish between different sets of patterns will be highly useful. 0.00 0.20 0.40 0.60 0.80 1.00 car connect4 mushroom nursery tictactoe pr ed ic tio n a cc ur ac y cart fi fs australasian journal of information systems fletcher & islam 2018, vol 22, research article comparing sets of patterns with the jaccard index 14 references agrawal, d. & aggarwal, c. (2001), “on the design and quantification of privacy preserving data mining algorithms”, 20th acm sigmodsigact-sigart symposium on principles of database systems, acm, pp. 247–255. aodha, o. m., stathopoulos, v., terry, m., jones, k. e., brostow, g. j. & girolami, m. (2014), “putting the scientist in the loop – accelerating scientific progress with interactive machine learning”, 22nd international conference on pattern recognition, ieee, stockholm, sweden, pp. 9–17. bache, k. & lichman, m. (2013), uci machine learning repository, url: http://archive.ics.uci.edu/ml/ baron, s., spiliopoulou, m. & gunther, o. (2003), “efficient monitoring of 21 patterns in data mining environments”, 7th east-european conference on advances in databases and informations systems, springer, dresden, germany, pp. 253–265. breiman, l. (2001a), “random forests”, machine learning, 45(1): 5–32. breiman, l. (2001b), “statistical modeling: the two cultures”, statistical science, 16(3): 199– 231. breiman, l., friedman, j., stone, c. & olshen, r. (1984), classification and regression trees, chapman & hall/crc. caruana, r. & niculescu-mizil, a. (2004), “data mining in metric space: an empirical analysis of supervised learning performance criteria”, 10th sigkdd international conference on knowledge discovery and data mining, acm, seattle, washington, pp. 69– 78. chaudhuri, k., monteleoni, c. & sarwate, a. (2011), “differentially private empirical risk minimization”, the journal of machine learning research, 12(1): 1069–1109. chawla, s., dwork, c. & mcsherry, f. (2005), “toward privacy in public databases”, theory of cryptography, pp. 363–385. cheng, h., yan, x., han, j. & hsu, c.w. (2007), “discriminative frequent pattern analysis for effective classification”, 23rd international conference on data engineering, ieee, pp. 716– 725. dwork, c. (2008), “differential privacy: a survey of results”, theory and applications of models of computation, springer, xi’an, china, pp. 1–19. felkin, m. (2007), “comparing classification results between n-ary and binary problems”, quality measures in data mining, springer, pp. 277–301. fletcher, s. & islam, m. z. (2014), “quality evaluation of an anonymized dataset”, 22nd international conference on pattern recognition, ieee, stockholm, sweden, pp. 3594–3599. fletcher, s. & islam, m. z. (2015a), “a differentially private decision forest”, 13th australasian data mining conference, sydney, australia, pp. 1–10. fletcher, s. & islam, m. z. (2015b), “measuring information quality for privacy preserving data mining”, international journal of computer theory and engineering, 7(1): 21–28. australasian journal of information systems fletcher & islam 2018, vol 22, research article comparing sets of patterns with the jaccard index 15 freitas, a. (2013), “comprehensible classification models: a position paper”, acm sigkdd explorations newsletter, 15(1): 1–10. friedman, a. & schuster, a. (2010), “data mining with differential privacy”, 16th sigkdd conference on knowledge discovery and data mining, acm, washington, dc, usa, pp. 493–502. fung, b., wang, k., chen, r. & yu, p. (2010), “privacy-preserving data publishing: a survey of recent developments”, acm computing surveys, 42(4): 1–53. fung, b., wang, k. & yu, p. (2005), “top-down specialization for information and privacy preservation”, 21st international conference on data engineering, ieee, pp. 205–216. geng, l. & hamilton, h. j. (2006), “interestingness measures for data mining: a survey”, acm computing surveys, 38(3): 1–32. han, j., cheng, h., xin, d. & yan, x. (2007), “frequent pattern mining: current status and future directions”, data mining and knowledge discovery, 15(1): 55–86. han, j., kamber, m. & pei, j. (2006), data mining: concepts and techniques, morgan kaufmann publishers, san diego. han, j., pei, j. & yin, y. (2000), “mining frequent patterns without candidate generation”, acm sigmod record, acm, dallas, texas, 12(1): 1–12. hand, d. j. (2006), “classifier technology and the illusion of progress”, statistical science, 21(1): 1–14. huysmans, j., dejaeger, k., mues, c., vanthienen, j. & baesens, b. (2011), “an empirical evaluation of the comprehensibility of decision table, tree and rule based predictive models”, decision support systems, 51(1): 141–154. islam, m. z., barnaghi, p. & brankovic, l. (2003), “measuring data quality: predictive accuracy vs. similarity of decision trees”, 6th international conference on computer & information technology, dhaka, bangladesh, pp. 457–462. islam, m. z. & brankovic, l. (2011), “privacy preserving data mining: a noise addition framework using a novel clustering technique”, knowledge-based systems, 24(8): 1214– 1223. islam, m. z. & giggins, h. (2011), “knowledge discovery through sysfor: a systematically developed forest of multiple decision trees”, 9th australasian data mining conference, australian computer society, inc., ballarat, australia, pp. 195–204. jaccard, p. (1901), “etude comparative de la distribution florale dans une portion des alpes et du jura”, bulletin de la soci´et´e vaudoise des sciences naturelles 37(1): 547–579. kifer, d. & gehrke, j. (2006), “injecting utility into anonymized datasets”, sigmod international conference on management of data, acm, new york, new york, usa, pp. 217–228. kotsiantis, s. & kanellopoulos, d. (2006), “discretization techniques: a recent survey”, gests international transactions on computer science and engineering 32(1): 47–58. australasian journal of information systems fletcher & islam 2018, vol 22, research article comparing sets of patterns with the jaccard index 16 letham, b., rudin, c., mccormick, t. h. & madigan, d. (2013), “interpretable classifiers using rules and bayesian analysis: building a better stroke prediction model”, technical report 609, university of washington. levandowsky, m. & winter, d. (1971), “distance between sets”, nature, 234(5323): 34–35. lipkus, a. (1999), “a proof of the triangle inequality for the tanimoto distance”, journal of mathematical chemistry, 26(1-3): 263–265. meila, m. (2007), “comparing clusterings – an information based distance”, journal of multivariate analysis, 98(1): 873–895. mohammed, n., chen, r., fung, b. c. & yu, p. s. (2011), “differentially private data release for data mining”, 7th sigkdd international conference on knowledge discovery and data mining, acm, new york, new york, usa, pp. 493–501. nergiz, m. e. & clifton, c. (2007), “thoughts on k-anonymization”, data & knowledge engineering, 63(3): 622–645. ordonez, c. & zhao, k. (2011), “evaluating association rules and decision trees to predict multiple target attributes”, intelligent data analysis, 15(1): 173–192. pedregosa, f., varoquaux, g., gramfort, a., michel, v., thirion, b., grisel, o., blondel, m., prettenhofer, p., weiss, r., dubourg, v., vanderplas, j., passos, a., cournapeau, d., brucher, m., perrot, m. & duchesnay, e. (2011), “scikit-learn: machine learning in python”, journal of machine learning research 12(1): 2825–2830. quinlan, j. r. (1993), c4.5: programs for machine learning, 1st edn, morgan kaufmann. quinlan, j. r. (1996), “improved use of continuous attributes in c4.5”, journal of artificial intelligence research 4(1): 77–90. shotton, j., sharp, t., kohli, p., nowozin, s., winn, j. & criminisi, a. (2013), “decision jungles: compact and rich models for classification”, advances in neural information processing systems, pp. 234–242. sokolova, m. & lapalme, g. (2009), “a systematic analysis of performance measures for classification tasks”, information processing & management 45(4): 427–437. vellido, a., martin-guerroro, j. d. & lisboa, p. j. (2012), “making machine learning models interpretable”, european symposium on artificial neural networks, bruges, belgium, pp. 163–172. wagstaff, k. l. (2012), “machine learning that matters”, 29th international conference on machine learning, acm, edinburgh, scotland, pp. 1–6. webb, g. & brain, d. (2002), “generality is predictive of prediction accuracy”, pacific rim knowledge acquisition workshop, pp. 117–130. willmott, c. j. (1982), “some comments on the evaluation of model performance”, bulletin of the american meteorological society, 63(11): 1309–1313. willmott, c. j., matsuura, k. & robeson, s. m. (2009), “ambiguities inherent in sums-ofsquares-based error statistics”, atmospheric environment, 43(3): 749–752. australasian journal of information systems fletcher & islam 2018, vol 22, research article comparing sets of patterns with the jaccard index 17 copyright: © 2018 fletcher & islam. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 1.1 problem statement 2 related work 3 comparing sets of patterns with the jaccard index 3.1 converting patterns into elements in a set 3.2 steps 3.3 practical considerations 3.3.1 conceptual simplicity 3.3.2 computational simplicity 3.3.3 interpretability 3.3.4 wide applicability 3.4 mathematical properties 3.4.1 bounds 3.4.2 metric properties 4 experiments 4.1 scenario 1: comparing classifiers 4.2 scenario 2: comparing anonymization techniques 5 conclusion microsoft word ajis vol 15-1 m.doc australasian journal of information systems volume 15 number 1 december 2007 59 an inconvenient truth: the invisibility of women in ict sue lewis institute for social research swinburne university of technology email: suelewis@swin.edu.au catherine lang faculty of communication and information technologies swinburne university of technology clang@it.swin.edu.au judy mckay faculty of communication and information technologies swinburne university of technology jmckay@it.swin.edu.au abstract this paper presents research findings from an australian faculty of information and communication technology (fict) against a backdrop of declining interest amongst women in courses and careers in information and communication technology (ict). it poses the question of why research conversations around women in ict are still pedagogically ignored, even after over twenty years of gender research and projects. in reporting the findings of this research project it seeks an understanding of and remedy for the steep decline of interest and uptake of ict places at tertiary institutions in australia, particularly amongst young women. in addition to understanding gender differences in educational motivation and performance, and appreciating the challenges posed to ict education, readers must also be cognisant of the steep decline of interest in careers in ict in general. introduction the ‘women in information technology – swinburne’ project (wit-s) is a three-year research project at swinburne university of technology aimed at gaining a deeper, conceptual understanding of the gendered participation in information and communication technology (ict) education as the basis for informed intervention at a range of levels. the whole area of the relative lack of diversity in ict education and careers has been under-theorised in both the literature and at the level of australasian journal of information systems volume 15 number 1 december 2007 60 assumptions underpinning recruitment and retention projects in ict and science, engineering and technology education (set) more broadly (trauth 2006). wit-s is underpinned by the research knowledge that women and men have differential experiences and interactions with the curriculum and teaching environments and with computers (crawford and macleod 1990, mahony and van toen 1990, cobbin 1995, collins et al. 2000, margolis and fisher 2002, barker and aspray (2006). the wit-s project was originally informed by the multilayered intervention project at carnegie mellon university (cmu) in the us where the numbers of women studying computer science were dramatically improved following a range of successful interventions (margolis and fisher 2002). whilst the wit-s program is geared towards the under-representation of women in tertiary ict education, the project is also designed to widen the appeal of ict to ‘non-geek’ boys as well as more girls as the ict sector faces declining student numbers overall in post-secondary education. at one level the wit-s project is part of the next generation of gender and ict projects in the australian university system after an earlier 1980’s generation of projects and reforms (cobbin 1995). however, this wit-s gender project at swinburne’s faculty of information and communication technologies triggers a much wider set of issues concerning gender research projects and their findings more generally in the higher education sector both in australia and internationally over the past 20 or more years. whilst located at and funded by swinburne university of technology, the project could be in most australian and indeed overseas universities where the drop in student interest in studying ict and the subsequent downsizing and amalgamations of ict teaching units have become commonplace (lang et al. 2005, charles and bradley 2006). throughout this paper ict will be used to represent all variations in specialisations within this discipline, including information systems, computer sciences, information technology and other computing courses. wit-s is focused on understanding the gender differences in learning and studying ict for the benefit of our university and our corollary aim is to contribute to a more cohesive theory about why women are ‘deserting’ ict education at a higher rate than men and take a fresh look at strategies for interventions in higher education. historically, when there are crises in procuring sufficient numbers of undergraduate students in the set curriculum areas in general, the ‘what about the women?’ issue commonly appears (wacjman 1991). historically, this concern about numbers of women has been driven by policy notions of social and economic justice for women (cobbin 1995), but the flow of funds for short-term projects such as wit-s have waxed and waned over the past 20 years in the australian tertiary system. long-term funding for equity research, policy and program level responses is very rare indeed in australia with few universities maintaining their commitment over time. currently, these questions ‘about the women’ also coincide with prosperous economic times nationally and together create the possibility of some innovative responses to this research. the national prosperity in australia however contrasts with a university sector that is seriously under funded by international standards (oecd 2007) and it is difficult to see where the circuit breaker will occur as high quality and equitable education will require appropriate resourcing. unlike secondary and primary education in australia (yates 1993; murphy and gipps 1996) there has never been a system-level focus on understanding what gender and culturally inclusive education means in set and ict education at the tertiary level despite the continuing low numbers of women in these courses at many of our universities (lang et al. 2005, dest 2006). whilst equity based research in the primary/secondary sectors have seesawed with political changes in state governments, funding for system-level equity-based research and pedagogical reviews in the post-secondary sector have been neglected by successive federal governments. ict faculties, and indeed set university faculties in general, rarely have the resources to investigate which groups of girls and which groups of boys have access to or are achieving in their curriculum, teaching and assessment practices (littleton 1996, furger 1998, durndell et al. 1998, meredyth 1999, collins et australasian journal of information systems volume 15 number 1 december 2007 61 al. 2000). an ict education at an australian university undoubtedly provides an opportunity for students to gain access to an economically beneficial career and the critical questions about who benefits and does not benefit in the current system are rarely asked. related to this pattern, we are hypothesising that the evidence based and reproducible research findings focusing on the gendered ict classroom over the past 20 years have been rendered largely invisible in tertiary institutions. pedagogical practices of academic staff are fundamental to how teaching and learning enhances or undermines student learning and achievement in the ict classroom (ramsden 1998). the three ‘message systems’ of education are commonly described as curriculum, pedagogies and assessment (bernstein 1971). all these ‘message systems’ need to be research based and, we argue informed by gender theory as well as the traditional educational and discipline-based knowledge that underpins ict education. our experience, and the learnings of gender theory itself, suggests that unless equity issues have an active role and contribution in both policy and practice terms within ict faculties, then intervention programs and their findings will continue to be marginalised and rendered invisible through the hidden faculty processes that maintain the traditional gender order (connell 1995, connell 2002). currently, most tertiary institutions do not question the way the ‘gender order’ is institutionalised within faculty processes such as curriculum and teaching decisions, course design and selection, sequencing of subjects within courses, course evaluation and review processes and teaching and assessment practices. gender theory informs us that there is a regime of gendered social relations that operates in everyday relationships, committee structures, email exchanges and committee processes in organisations that devalue and ignore gender learnings and questions (eveline 1994; fletcher 2001, adam et al 2004). whilst ict education started well in the 1980’s when relative numbers of women students were higher than other set curriculum areas, the steady decline in numbers of women choosing ict education and careers has not as yet resulted in a system-level response. in the tertiary education sector in australia, unlike the us (cohoon and aspray 2006) and the greenfield report in the uk (peters et al 2002), there has not been any professional review of ict education at the teaching, curriculum and assessment levels in response to either the significant cultural diversity issues facing the sector or the continued decline in the gender balance. there is little historical base to understand the gender, class and cultural components of ict education and the traditional and hidden social practices embedded in teaching and learning ict (meredyth 1999). senior women in male dominated workplaces experience more strongly sexualised gender stereotypes and roles in the workplace (fletcher 2001; kantor 2003) and this appears to spill over very directly into curriculum and teaching decisions in male dominated faculties where the gendered nature of curriculum and teaching needs to be considered (cobbin 1995; dest 2004; james et al. 2004). the educational processes within ict faculties and teaching units support the existing ways of doing curriculum, pedagogy and assessment and do not usually allow for curriculum review and reform. there is considerable disjunction between the research based findings that live in the knowledge and practices of many academics that have researched and considered the gender and ict educational learnings and the invisibility of these learnings in faculty processes (trauth 2006). we will describe some of these below. the intent of this paper is to bring to the fore conversations about gender-based research findings within ict in higher education. it is an inconvenient truth in ict education that the findings of gender-based research over the last twenty years have been ignored, and now the ict discipline is facing a crisis in student numbers. women in ict, while visible statistically, have been rendered invisible in the gendered organisational context of these faculties. not all women ict students have the same teaching and learning needs, however there are broad gender-based trends that point to a diversity of student needs that are not understood or recognised in most ict faculty pedagogies. in this paper, we will first consider the crisis in ict enrolments at university which in part motivated australasian journal of information systems volume 15 number 1 december 2007 62 this research project, and then the under-representation of women in ict courses. the paper will then broadly discuss a range of gender issues and how these impact on educational contexts, particularly in under-represented areas of education for women, including ict. we then consider some of the local research findings, which, using terminology adopted by cohoon and aspray (2006) we have called ‘things we already know’ at swinburne. this synthesis of theoretical knowledge from the literature and findings form our empirical work are finally discussed with our key recommendations. the crisis in enrolments first to the crisis in enrolments in ict. there are analyses elsewhere about the recent downturn in student numbers (lang et al. 2005; lewis et al. 2006) despite the seemingly contradictory evidence of the skill shortage in the ict industry. whilst the percentage of women participating in ict courses at australian universities has fluctuated according to the specialisations within the discipline, apart from the three years after 1997 when there was a growth in aggregate female enrolments of an average of half a percent per year, there have been declining female enrolments for 20 years (lewis et al. 2006). from 2000 to 2005, the popularity of the ict discipline has plummeted in australia with fewer women and men entering the ict field at university: there has been a decline of 29 per cent in males, and an alarming 51 per cent decline for women. the aggregate proportion of females enrolling as new undergraduate ict students in australian universities has also declined from 26 per cent in 2001 to 20 per cent in 2005 (dest 2006). we are hypothesising that this is no ‘new’ crisis at all in relation to the participation of women in ict education but simply a further consolidation of a pattern that has been evident for 20 years. as early as 1995 a national study of ‘women’s participation in non-traditional fields of study’ concluded that computer science in australia was the one field of study that had shown a decrease in the representation of women during the study period 1989 to 1994 (cobbin 1995). the 2004 equity review of ‘non-traditional courses’ in australian higher education concluded that ict should be retained as a non-traditional field of study equity group with targets of 40% participation for women (james et al. 2004). australasian journal of information systems volume 15 number 1 december 2007 63 figure 1: it courses as first choice university course selection (vtac 2006) at a time when the overall participation rates of women in higher education in australia have increased to 56.7% of all students in 2004 (dest 2004), and disciplines such as medicine and law have more than 50% undergraduate women (medicine 57% in 2003, law more than 50% since 1996), female aggregate enrolments in ict courses have decreased to 20% overall. since 2001, ict has witnessed a 46% collapse in the number of students selecting ict courses as their first preference for higher education in victoria (vtac 2006, figure 1). when this figure is analysed by gender, there is an overall decline in ict course selection of 65% amongst females compared to 40% amongst males (vtac 2006). with the overall participation of women in the ict field of study close to 20%, this leaves many ict courses and year levels with cohorts of women far less than 15% (dest 2006). interestingly, australia also reflects the global downturn where the numbers of women enrolling in ict courses at universities in western industrialized countries has also been spiralling down over the past two decades. wright (1997) surveyed the numbers of women studying in ict /computing courses between 1985 and 1990 in 19 countries and found that 12 countries – including the us and the uk – witnessed a decline during these years (wright 1997). the number of women studying in all ict/computing courses in the uk fell from 28% in 1990 (wright 1997) to 18% in 1999, and this decline has continued in more recent years (ucas 2005). a recent international study of the proportional gender representation in the discipline carried out by analysing graduation trends from twenty-one industrialised countries emphasised the widespread masculinity of the field (charles and bradley 2006). however there was disparity between countries in the degree of over-representation of males. the male to female ratio was three times stronger in some countries than others (charles and bradley 2006 p.191). the male ‘over-representation factor’ was the lowest for turkey (1.79) and the highest for czech republic 6.36, with australia low at 2.86 (charles and bradley 2006 p.190). 0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 2000-1 2001-2 2002-3 2003-4 2004-5 st ud en t n um be rs females males totals australasian journal of information systems volume 15 number 1 december 2007 64 the gender and ict jigsaw few ict faculties employ staff with specialist expertise in education or gender theory and herein lies some of the dilemmas underpinning our broader argument about the invisibility of previous research into ict education and the related tendency to reinvent the wheel. gender is a critical element in the gender/class/culture socio-demographics in shaping our social and organisational lives and all our social interactions. as collins, kenway and mcleod (2000) outline, the socio economic differences within groups of boys and within groups of girls can outweigh the gender differences; for example socio economic status (ses) makes a larger difference than gender to educational participation and year 12 performance in australia (collins et al. 2000). contemporary thinking in gender and education is more concerned about the ‘which boys, which girls?’ approach to understanding differences in educational outcomes as an important development in thinking about difference (collins et al. 2000). the so-called ‘gender wars’ or ‘gender gap’ has kept the public debate within the binary ‘boy versus girl’ discourse rather than as collins et al (2000) advocate, into understanding the gender jigsaw. to understand the ‘gender jigsaw’ the argument needs to move beyond a binary ‘boy versus girl’ discourse to an understanding of why ict is not attractive to a greater cohort of students, male and female (collins et al. 2000). equally, it is important to avoid using essentialist language that treats all young women and young men as having identical educational needs: hence the importance of asking ‘which girls, which boys?’ in relation to educational access and success. however, it is not only the gender differences that are important to educationally understand but the privileging of characteristics associated as masculine over characteristics associated as feminine that is even more important (adam 2004, wacjman 2004, trauth, 2006). in addition, how these differences get translated into advantage and disadvantage in tertiary institutions is also critical (acker 1990, eveline 1994). consequently, we need to understand and review the organizational and educational processes that shape this very valuing of the masculine over the feminine if we are to change anything substantial in our staffrooms and classrooms (wacjman 2004, webster 2005). this includes our hypothesis that gender research findings are themselves not valued historically in informing and determining change in relation to curriculum and teaching of ict. we will return to this theme when discussing our findings. it can be problematic to regard ict as a ‘single’ discipline, as the juxtaposition of ‘information’, ‘communication’, and ‘technology’ suggests some sort of separation of interests. the predominant focus, particularly in the earlier years of the discipline, was on the technology itself, and saw the development of a range of technical theories, frameworks and practices geared to underpinning the technical challenges, generally at the exclusion of social theories and practices (mahoney and van toen 1990, adam et al. 2004). more recently, there has been greater focus on information as an organisational resource, and in the more information-oriented sub-disciplines in ict, namely information systems and information management, there has been an increasing emphasis on both the technical, and social theories and practices (adam et al. 2004). indeed the ubiquitous application of ict to all aspects of society and its increasingly ‘human’ focus should necessitate a greater investigation as to how to configure the discipline to be attractive to an increasingly diverse, yet diminishing cohort (trauth 2006). the contradiction that emerges in contemporary ict higher education is that student numbers mean economic survival for faculties, yet in an era of declining enrolments, we hypothesise that ict faculties still marginalise ‘diversity’ projects, and overlook strategies that are informed by research and would attract and retain a larger cohort of students both men and women to the discipline. australasian journal of information systems volume 15 number 1 december 2007 65 during the 1980’s and 1990’s, most of the ‘access and equity’ discussions in ict in higher education were centred on recruitment where outreach programs targeted young women in schools to consider ict as a career (cobbin 1995; sandler et al. 1996). these models ultimately carried an assimilatory set of assumptions: influence young women to change their choices and then they would participate in the same university courses historically offered to men, and they would be indistinguishable from the male students in their curriculum and teaching needs. women had no choice but be assimilated into a male dominated faculty, and theorists have argued that this required women to ‘become one of the guys’ in terms of their gender identity in ict and other densely masculine disciplines (mclean et al. 1997; wacjman 2004). these ‘first generation’ intervention programs were largely based on a liberal discourse of equal opportunities similar to the wise (women into science and engineering) initiatives since 1984 in the uk (henwood 1996). this discourse inadvertently placed the responsibility on women who were seen as ignorant of the opportunities and choices that ‘non-traditional’ education and careers had to offer (henwood 1996). consequently many universities designed recruitment and retention programs directed at ‘informing’ women’s choices but the parallel challenge of changing curriculum and teaching practices as well as cultural changes inside the ict academy was overlooked. there was not a corollary lens on how ict and its institutions could be reshaped to accommodate rather than assimilate women into existing structures and cultures. many of the findings of gender and ict projects, like wit-s, have been found before in earlier ict education research contexts (margolis and fisher 2002, cohoon and aspray 2006, trauth 2006). during the 1990’s, there was a proliferation of literature about why these secondary and tertiary courses acted as an unwelcoming learning environment for women (cobbin 1995; murphy and gipps 1996; sandler et al. 1996). common criticisms of ict education focus on the over-emphases on abstract ‘mathematical formalism’ (mahony and van toen 1990) and on the technical skills (cohoon and aspray 2006). different mathematics choices operate at school for girls and boys and career options for girls, given their more limited subject selection in secondary school, can be more restricted (cobbin 1995). in the 1995 review of the ‘non-traditional’ fields of higher education study, cobbin (1995) cited the following reasons from the literature for the differential participation of girls and boys in ict: sex stereotyping of toys and activities; sex-biased computer software and games; differential availability of female and male role models; different learning experiences of girls and boys in the gendered classroom where boys’ had greater access to school resources and teacher attention. other frequently cited reasons for women choosing not to enter ict education in the same numbers as men are: the construction of the ict curricula as abstract and disconnected from social and human concerns; software being written by and for men and boys that results in greater informal exposure of boys to ict; domination of computer training programs by boys, men and male values; and the perceptions of computer professionals as geeks, nerds and antisocial ‘computer heads’ (cobbin 1995, cohoon & aspray 2006). there are also differences evident in self-confidence, self esteem and risk taking behaviours of young women and men (eckel and grossman 2002; gneezy et al. 2003). indeed recent studies confirm that knowledge and understandings of the technical at the expense of the socio-technical, ‘real-world’ and relational skills is still predominantly valued in most ict faculties (margolis and fisher 2002, cohoon and aspray 2006). there is even less discussion of the importance of informal learning in the production of these gendered patterns. tully (1997) examined informal pathways to computer knowledge and argued that while 'the knowledge of how to handle a computer should be taught mainly via the school education system’, in reality 'the acquisition of computer skills is primarily ensured by informal learning in the home’ (tully 1997). students attitudes to and competence with computers in the australasian journal of information systems volume 15 number 1 december 2007 66 classroom are related to at-home access and that most learning about computers occurred at home for those students who had home access (kersteen and linn 1988; martinez and mead 1988). this theory was verified by the national ‘real time’ study where informal learning was the primary means of learning the information technology skills possessed by both australian teachers and students (meredyth et al.1999). the ‘real time’ study also confirmed the digital divide where boys had more access to and spent more time developing ict skills at home than girls (meredyth et al. 1999). if the home is the primary means of learning ict skills, then schools and universities need to include this understanding in the way they design and market ict courses, for example providing different entry pathways to degree courses to overcome these pre-existing patterns of inequality of access and experience and recognise difference in the cultural and gender backgrounds of students. all these studies of home, school and university use of computers are overshadowed by the literature theorising the very gendering of the industry and the technologies themselves (wacjman 1991; wacjman 2004; webster 2005). wacjman and webster have argued that the very technologies around us are socially shaped as masculine because over time the “culture of masculinity is conterminous with the culture of technology’ (wacjman 2004). as wacjman explains, it makes no difference whether women or men apply and use the technology it is the technology itself that is now gendered (wacjman, 2004). this gendering of technologies means that the current flight of young women from ict education will embed the more serious structural employment inequalities that surround the numbers inequalities. it reinforces and reproduces the sexual divisions of labour in our society and renders the ict sector sex stereotyped at best and excluding women at worst. ‘things we already know’ at swinburne here we summarise the swinburne research findings and our realisation that most of these research findings are already well substantiated in the literature. an important question for faculty learning is how gender influences learning in our ict classes and the following summary outlines the key findings from our quantitative survey of over 700 first year undergraduate students across the ict faculty and the five other university faculties. the goals of this survey were to glean a better understanding of the student experience, and their attitudes to ict; their backgrounds in terms of experiences with ict both in educational and social contexts, their perceptions of courses and career choices, and their experiences at university. more detailed data on the survey results are included elsewhere (lang et al. 2006; lewis et al. 2006)). the summary findings from the wit-s quantitative and gender audit components of our project lead us to make a list of ‘things we know’ at swinburne using the terminology coined by cohoon and aspray (2006) when reviewing the research on women’s participation in postsecondary ict education. the ‘things we know’ at swinburne are: the ict class of 2005 are diverse and have non-traditional pathways to study: the question about ‘which girls and which boys?’ study ict at our university show some interesting gender and culture patterns for the ‘class of 2005’. a high proportion of the women were overseas students (48%) compared with only 18% of the men. of the 52% of women resident in australia, nearly 30% were bridging from technical and further education (tafe) courses, revealing low recruitment of young women directly from secondary school. only 8% of females compared with over 50% for male undergraduates came to university ict courses directly from high school. there was also an interesting pattern of single-sex secondary school education for a high percentage of first year ict australasian journal of information systems volume 15 number 1 december 2007 67 women (42%) compared with the men (27%) with the majority of these from non-elite private schools. male ict undergraduates have more programming experience prior to university: the overall finding was that male and female ict students have more in common with each other about their experiences of using computers at school, home, and at university than not. using a one-way analysis of variance, the only statistically significant variance between the genders relating to computer use (using a likert response to statement about programming experience) was that male ict students as a group arrive at our courses with significantly more programming expertise and experience than new ict female students (p.002). as discussed later this was the same finding that led carnegie mellon to introduce different entry pathways for different students. both female and male ict students significantly differed from the rest of the university students surveyed in their responses about computer use and indicate a greater competence and confidence with ict for both the women and men undergraduates (p.000), as would be expected with students enrolled in undergraduate ict courses. male ict undergraduates were also significantly more likely to assess coursework timing as adequate to understand the coursework (p.003) and expressed more confidence in their computer skills when compared with their male friends compared with our women ict undergraduates when compared with their women peers (p.003). women experience more stress in transition to university life: our survey asked the first year students a number of questions related to their first-year experiences. women and men responded similarly on a range of questions related to the climate of the ict classroom or laboratory but differently in their attribution of course progress and success. however, significantly more women than men thought the course workload was ‘heavier than expected’ (p.003) and more men than women agreed that ‘the coursework timing is adequate to understand the things we have to learn (p.003)’ suggesting a proportionately higher number of stressed female students in their transition to university study. anxiety levels show a further gender difference with more women than men feeling anxious about how they are coping with study compared with the men. our male ict undergraduates reported significantly more difficulty balancing their time between study, work and social activities (p.034) compared with women. the data however shows male ict students undertaking more paid employment during semester and these data may relate to these longer paid work hours. ict classes are more disruptive than other faculties: there was statistical significance in the responses of both women and men students to questions related to disruptive student behaviour in ict classes compared with students in other faculties (p.000), indicating a level of disruptive behaviour in the ict laboratory that supports the well-documented culture of masculinity or ‘clubhouse’ environment associated with ict (margolis and fisher 2002). it is also evident that our male students have stronger self-efficacy related to their ict abilities than our female students and that the climate in our classrooms is less than desirable for a considerable proportion of our students. pedagogical factors detrimental to student achievement: on the basis of the gender audit of student achievement for the 2005 first year ict cohort, a range of pedagogical factors were significant when clustered together (lang et al. 2007). when a low female critical mass was combined with no female teaching staff, a masculinised disruptive environment resulted which was detrimental to the learning outcomes for the australasian journal of information systems volume 15 number 1 december 2007 68 women students. the three aspects of pedagogy shown to significantly affect all students performance were a contextualised curriculum as apposed to an abstract curriculum, varied assessment tasks including continuous assessment as apposed to exam based assessment and faculty teaching by academic staff with teaching qualifications compared with no teaching qualifications (lang et al. 2007). interestingly two lowest performing subjects where students received a fail more than any other mark were the two programming subjects in 2005. these subjects had the lowest participation by women (8%) and were almost exclusively taught by male staff with no formal teaching qualifications. in the recent review of the research on women’s participation in post-secondary ict education cohoon and aspray (2006) conclude that we are ‘closer to the beginning than to the end of research into the gender imbalance in postsecondary computing education’ (p.170). cohoon and aspray (2006) created two interesting tables based on their critical review of the literature; ‘things we know’ (p.171) about the gender composition of ict and also include another table on ‘things we believe and expect to establish’ based on their literature review of the research available (p.172). we have separated the cohoon and aspray (2006) list into factors connected to student background and factors connected to faculty interventions. from their extensive review of the research in women’s participation in post-secondary ict education, their list of the ‘things we know’ related to student background are (cohoon and aspray, 2006, p 171): computing culture is masculine. whether it has to be masculine, and whether the culture is a cause or a consequence of its gender composition, are different questions ict specialisations have different cultures and gender compositions women generally reach college with less programming experience than men computing experience has a positive effect on both computer science education grades and confidence women generally have less confidence than men in their ability to do ict differences in academic fitness are at most weak contributors to women’s underrepresentation these review findings are reinforced by our university based research and similar to the cmu study findings that helped shape our wit-s project, where a ‘roller coaster of uncertainty and doubt’ was experienced by many of their female students, particularly in the early stages of their course (margolis & fisher 2002). these data show a marked similarity with other findings of the gendered student undergraduate experience of ict curriculum and teaching as well as the cmu findings (clegg et al. 2000; margolis and fisher 2002)) where the ict classroom experience contains different gender dynamics for the women and men studying in our university. women are often part of a ‘minority’ below the critical mass of 15% participation: this percentage was adopted as an earlier yardstick in the study of male dominated educational settings (cobbin 1995). however research in the business field placed the critical mass at 25%, where valian found that “when women are in a minority [less than 25%] they are judged more in terms of their difference than their ability, … being in a minority increases a woman's likelihood of being judged in terms of her difference from the male majority, rather than in terms of her actual performance" (valian 1999 p.140). kantor (2003) describes the cultural dynamics for women under 25% of a cohort in employment as being judged as a representative of their sex rather than as an individual. in such situations, it has been argued that the environment is ‘skewed’ towards men feeling comfortable and being able to determine the cultural and classroom dynamics in positive or negative terms (margolis & fisher 2002, kantor 2003). the ict and education research literature now provides evidence of the importance of students’ perceptions of relevance and the socio-technical contextualisation of the curriculum. a large scale australian study of 16-year-old science students reported their studies in science as rarely relevant australasian journal of information systems volume 15 number 1 december 2007 69 and not connecting with their interests and experiences (goodrum et al. 2001). a victorian government sponsored project found that many 17 to 19 year olds stereotypically believed the ict career path to be too boring to consider as a future career option (mmv 2004). this evidence points to ict faculties needing to move from the content driven to context driven curriculum. the teaching of abstract concepts disconnected from the interests and lives of students has been roundly criticised in a recent review of physics research in education in the uk (murphy & whitelegg, 2006). in this review, instigated by the uk institute of physics in response to plummeting numbers of post compulsory physics students murphy and whitelegg (2006) conclude that changing the curriculum is not simply the addition of applications on the end of a theory lesson but the reconstruction of the curriculum as context driven and not content driven that is important. it has been well established that girls more than boys engage with learning and teaching when the curriculum is contextualised and the relevance clearly established (boaler 1997, murphy & elwood 1998). however, context curricula has been curiously under-theorised and researched as a curriculum model. aikenhead (1994) defined context curricula as ‘an organiser’ for contemporary curriculum transformation and murphy and whitelegg (2006) outlined the characteristics of a context curriculum from their exhaustive review of research into physics education. context curriculum is where social situations organise and determine the content studied and assessed; the situation and the problems within it provide the purpose for learning; situations vary between those of relevance to students’ daily lives and concerns, and wider social issues of concern to societies generally; physics represented as a social practice, physics knowledge as a social construction that is open to change and influenced by social, political, historical and cultural factors; and the values implicit in physics practices and knowledge are matters for examination (murphy and whitelegg, 2006 p.19). this then places the social context as the driver of the subject knowledge for learning. clearly such an approach would challenge most conventional curriculum and teaching programs in tertiary education; however there is enough evidence from other curriculum areas to suggest that these approaches are worth pursuing in ict education to attract more students, and particularly more women. context curricula are also essential in the challenge to break down the rusted stereotype of ict employees as ‘geeks in front of the computer all day’. the educational research and debates surrounding the importance of context curricula in reviewing and reforming threatened curriculum areas such as ict also connect to the criticisms from employers in the ict industry. spencer (2003) challenged all involved in ict education in australian universities to address the narrow focus of the ict curriculum, and claimed that few australian degree programs reflected current ict applications in business and the workforce and “this marginalises the users of information technology” (spencer 2003 p.65; spencer 2003 p.65). spencer suggests we need to reconstruct the domain of ict as a first step to equalising the gender imbalance and grounding the problem solidly within the discipline (spencer 2003). we are hypothesising that this cluster of different student experiences and confidence in programming, masculinised learning environment, content-driven abstract curriculum, and limited understanding of effective pedagogies all have connections to one of the hidden educational phenomena underpinning this cluster: the gendered attribution of success and failure. we consider that these threads of poor pedagogy in higher education contribute to the pattern of women and men undergraduates often experiencing curriculum, teaching and assessment systems differently and have outcomes that are as yet under-researched in ict teaching and learning contexts. we will trace these theoretical connections in the following discussion. australasian journal of information systems volume 15 number 1 december 2007 70 discussion the gendered attribution of success and failure needs some explanation first. there have been many studies in the last twenty years reported in educational and psychological literature on the gendered nature of psychological factors other than ability that affect success in cognitive tasks. during the 1980’s, when gender related differences in abilities had been discredited as being insignificant as an explanation for the absence of women from many disciplines, both educational and psychological researchers started to explore the possibility that women and men tended to use their abilities in different ways (head 1996). this led to the focus on cognitive style as a frame for analysing gender differences. the important characteristics of cognitive style are that the scale is bipolar and people are located on a continuum according to their preference between two contrasting ways of working (head 1996). here the context is very important as the advantage of any particular style is only conferred by context and not any measured level as would be the case on a uni-polar scale of ability. as head (1996) describes, cognitive styles ‘are all concerned with how individuals organise their environment and develop reasonably consistent strategies in dealing with unfamiliar tasks’ (p.60). whilst there are many styles described in the literature, one of the important clusters important in the context of this research is described as the attribution of success or failure or the ‘locus of control’. as head (1996) describes, males tend to develop a defence mechanism of attributing success to their own efforts (internal locus of control) and failure to external factors (external locus of control). girls however show the opposite and attribute failure to their own internal factors and success to external factors. for example a boy will attribute failure to ‘too much sport’, ‘not enough study’, ‘hopeless teaching’ or ‘got sick’ whereas a girl will blame herself ‘for not studying enough’ or ‘not good at maths’. girls on the other hand will externalise success as due to ‘luck in studying the right questions’, ‘great teacher’, ‘friends helped me’ while boys will internalise success as being smart, able and capable. of course not all boys and all girls behave in these ways all the time but these gender patterns are well understood from the social and anecdotal to educational research (head 1996). we consider here that the educational implications of these findings have been under-theorised and under-utilised when understanding gender differences in success and failure attribution as well as in the gendered construction of unequal outcomes from education. there are learnings and implications from this research for the way that faculties organise curriculum, teaching and assessment systems as well as student course and career choice and even their motivation and interest in studying ict. twenty years ago, dweck reviewed the socio-cognitive approaches to the study of motivational processes and how they affect success on cognitive tasks (dweck 1986). she concluded that there was an educationally sound and replicable body of findings that ‘highlighting performance goals relative to learning goals can have negative motivational impacts on achievement’ (dweck 1986). a focus on performance judgements can result in ‘a tendency to avoid and withdraw from a challenge’, whereas a focus on learning goals and effort creates a tendency to ‘seek and be energised by challenge’ (dweck 1986). this is a powerful pedagogical argument for formative and continuous assessment as apposed to the summative exam-based approach that our assessment audit found does indeed undermine student achievement. bandura (1997) also observed different attitudes to learning and experiences in education due to gendered self-efficacy beliefs. according to bandura, boys tend to overestimate their ability in a task, and when things do not go as well as anticipated, often blame an external factor. girls on the other hand tend to underestimate their abilities and blame themselves first before any external factor australasian journal of information systems volume 15 number 1 december 2007 71 (bandura 1997; pajares 2002). the classroom implications of these gender differences is that girls may sink into a state of ‘learned helplessness’ where the perception of ‘personal failure inhibits subsequent performance’. this also explains the other well known contemporary pattern of girls having to be ‘super’ high achievers to even contemplate pursuing studies in the physical and technical science subjects and their documented fragility in maintaining their self efficacy in these curriculum areas (murphy and whitelegg 2006) an 80% result on a maths test can trigger some young women to question whether they ‘are good enough’ to continue with maths at secondary school. it also assists in understanding the pattern of girls choosing to study science and technology subjects in higher numbers in single-sex secondary schools when compared with their coeducational sisters. here they experience less disruptive classes and have a less stereotyped view of these subjects being masculine in their rarefied single sex atmosphere. the current numbers crisis in higher education is perpetuating many aspects of the ‘single sex school for boys’ and we are hypothesising that this produces a more fragile learning environment for women where self-efficacy at university may be fragile in the masculinised classroom alongside the abstract content-driven curriculum. the corollary challenge for boys is learning to accept responsibility for their poor work (dweck 1986, head 1996). it should be noted that this gender difference is independent of ability. girls proneness to internal distress underpins the theoretical reasons why girls do not pursue ict study in secondary and tertiary education and have different experiences when they do. girls who are doing well in performance terms are still vulnerable to evaluating themselves negatively and experiencing ‘internal distress’. conclusions and recommendations the wit-s project recommendations are multi-layered in response to these gender based research findings and the wider research in post secondary ict participation. the student-focused layer of recommendations is directed at recruitment, retention and advancement strategies for women students and have been well established for 2006/7 with the women in it @ swinburne network. the ‘women in it’ network set up this year at swinburne has been very successful already in providing learning and social networks for many students. this is a well-documented and successful strategy used around the world, including at carnegie mellon university, but will require ongoing university support to underpin the student energy and involvement (margolis and fisher 2002). this network becomes an umbrella for careers events, mentoring activities, role models from industry and networking with the wider networks of ict women. new students in the faculty in 2007 are being offered a series of research and information sessions to support women in understanding, contesting and transforming these identities. this innovative strategy is based on earlier research that showed how educators could assist young people to understand and transform the gender discourses prevalent in their peer group (connell 2002). our goal is to raise awareness as well as shift behaviour in relation to both self and others when students attribute success and failure in their course. the faculty-focused layer is directed at a range of professional and curriculum development strategies that will develop research material and professional development units that describe more fully the threshold knowledge to support a gender inclusive pedagogy in the faculty. these research files would cover the gender research areas discussed in this paper; attribution of success and failure, cognitive style, single-sex secondary schooling and implications for faculty learning environments, performance centred vs. learning centred assessment. these will require significant australasian journal of information systems volume 15 number 1 december 2007 72 leadership and resources to implement. if we were to take the research findings seriously then the faculty would be pedagogically different with a range of policies as guidelines for context and learning centred courses and sequencing. the other half of the cohoon and aspray (2006) ‘things we know’ (p.171) list focused on factors connected to faculty interventions that have informed our project initiatives also: when faculty mentor for diversity (this action equalises the retention of undergraduate men and women) encouragement from faculty helps to equalise undergraduate retention same sex peers help to equalise undergraduate retention paired programming improves overall student retention. our findings reported here are consistent with other studies both in australia and internationally. however, we also ‘wonder’ as well as theorise about the educational dissociation where the pedagogical implications of these findings are consistently ignored in practice. if half of our young women come from overseas, are at an age when they are vulnerable to evaluating themselves negatively even when they are high achievers, have less background experience of computers and less computer confidence, learn better in a learning-centred rather than performance-centred classroom, prefer the socio-technical contextual curriculum over the abstract, then why are they pedagogically invisible? it would appear that the very organisational processes that render women’s knowledge as less status than men’s are not even recognised as being gendered processes (acker 1990). acker would say these gendered discourses are mobilised at various decision–making sites within faculty and university life and thereby become part of the educational policies and practices that enshrine this invisibility (acker 1990). the valuing of masculine standpoints and experiences over the feminine is the basic conclusion in male dominated sectors of our society. are there similar processes at work in densely masculine institutions such as ict faculties where the educational needs of women in faculties are given scant attention? one important reason may stem from the frequent separation between educational and gender researchers, and ict curriculum developers and educators. however, as we outline in this paper, there have been consistent gender research findings relating to the experiences of young women in ict education for over twenty years, and yet these findings are consistently ignored in the curriculum and teaching development in ict faculties. perhaps in this current climate of haemorrhaging enrolments, both male and female, a greater degree of professional review or internal ‘naval-gazing’ is needed in ict faculties to ensure they remain relevant to students in the twenty-first century. references acker, j. (1990). "hierarchies, jobs, bodies: a theory of gendered organisations." gender and society 4(2): 139-158. adam, a., d. howcroft and h. richardson (2004). "a decade of neglect: reflecting on gender and is." new technology. work and employment 19(3): 222-240. aikenhead, g. (1994). what is sts science teaching? sts education: international perspectives on reform. j. solomon and g. aikenhead. new york, teachers college press. bandura, a. (1997). self-efficacy: the exercise of control. new york, w.h. freeman & company. barker, l. j. and w. aspray (2006). the state of research on girls and it: research on underrepresentation. women and information technology. j. m. cohoon and w. aspray. cambridge massachusetts, the mit press: 3-54. australasian journal of information systems volume 15 number 1 december 2007 73 bernstein, b. (1971). on the classification and framing of educational knowledge. knowledge and control. m. f. d. young. london, collier macmillan: 47-69. boaler, j. (1997). experiencing school mathematics: teaching style, sex and setting. buckingham, open university press. charles, m. and k. bradley (2006). a matter of degrees: female underrepresentation in computer science programs cross-nationally. women and information technology. j. m. cohoon and w. aspray. cambridge massachusetts, the mit press: 183-204. clegg, s., d. trayhurn and a. johnson (2000). "not just for men: a case study of the teaching and learning of information technology in higher education." higher education 40: 123-145. cobbin, d. (1995). women's participation in nontraditional fields of study at the undergraduate level of higher education 1989-1993. deet higher education division, evaluations and investigations program. canberra, australian government publishing service. cohoon, j. m. and w. aspray (2006). a critical review of the research on women's participation in postsecondary computing education. women and information technology. j. m. cohoon and w. aspray. cambridge massachusetts, the mit press: 137-180. collins, c., j. kenway and j. mcleod (2000). factors influencing the educational performance of males and females in school and their initial destinations after leaving school, deakin university connell, r. w. (1995). masculinities. sydney, allen & unwin. connell, r. w. (2002). gender. cambridge, polity press. crawford, m. and m. macleod (1990). "gender in the college classroom: an assessment of the "chilly climate" for women." sex roles 23: 101-122. dest (2004). review of the higher education equity programme (heep). canberra, dest: 24. dest (2006). http://www.dest.gov.au/sectors/higher_education/publications_resources/profiles/students_2 005_selected_higher_education_statistics.htm. department of education, science and training. durndell, a., p. glissov and g. siann (1995). "gender and computing: persisting differences." educational research 37(3): 219-227. dweck, c. s. (1986). "motivational processes affecting learning." american psychologist 41(10): 1040-1048. eccles, j. s., a. wigfield and u. schiefele (1998). motivation to succeed. handbook of child psychology: vol 3. social, emotional, and personality development. n. eisenberg. new york, wiley: 1017-1095. eckel, c. c. and p. j. grossman (2002). forecasting risk attitudes: an experimental study of actual and forecast risk attitudes of women and men., virginia tech: 1-31. eveline, j. (1994). "the politics of advantage." australian feminist studies 19: 129-154. faludi, s. (1991). backlash: the undeclared war against american women. new york, crown. fletcher, j. k. (2001). disappearing acts: gender, power, and relational practice at work. cambridge, massachusetts, the mit press. australasian journal of information systems volume 15 number 1 december 2007 74 furger, r. (1998). does jane compute?: preserving our daughters' place in the cyber revolution. new york, warner books inc. gneezy, u., m. niederle and a. rustichino (2003). "performance in competitive environments: gender differences." the quarterly journal of economics 118(3): 1049-1074. goodrum, d., m. w. hackling and l. rennie (2001). the status and quality of teaching and learning in science in australian schools. canberra, the department of education, training and youth affairs. head, j. (1996). gender identity and cognitive style. equity in the classroom: towards effective pedagogy for girls and boys. p. f. murphy and c. v. gipps. london, the falmer press: 5969. henwood, f. (1996). "wise choices? understanding occupational decision -making in a climate of equal opportunities for women in science and technology." gender and education 8(2): 199-214. james, r., g. baldwin, h. coates, k.-l. krause and c. mcinnis (2004). analysis of equity groups in higher education 1991-2002. centre for the study of higher education, the university of melbourne: 114. kantor, r. m. (2003). men and women of the corporation. reader in gender, work and organisation. r. j. ely, e. g. foldy, m. a. scully, maldon, usa, blackwell publishing: 3448 kersteen, z. a. and m. c. linn (1988). "previous experience and the learning of computer programming: the computer helps those who help themselves." journal of educational computer research 4: 321-334. lang, c., s. v. lewis and j. mckay (2005). it courses: the re-emergence of it as a 'boys' subject area. vitta, melbourne. lang, c., j. mckay and s. lewis (2007). seven factors that influence ict student achievement. the 12th annual conference on innovation and technology in computer scienceinclusive education in computer science, dundee, university of dundee. lewis, s., j. mckay and c. lang (2006). the next wave of gender projects in it curriculum and teaching at universities. eighth australasian computing education conference (ace2006), hobart, australia, crpit. littleton, k. (1996). girls and information technology. equity in the classroom. p. f. murphy and c. v. gipps. london, the falmer press: 81-96. mahony, k. and b. van toen (1990). "mathematical formalism as a means of occupational closure in computing why 'hard' computing tends to exclude women." gender and education 2: 319 331. margolis, j. and a. fisher (2002). unlocking the clubhouse: women in computing. cambridge, massachusetts, the mit press. martinez, m. e. and n. a. mead (1988). computer competence: the first national assessment. princeton, nj, the national assessment of educational progress and educational testing service. mclean, c., s. lewis, j. copeland, b. o'neill and s. lintern (1997). "masculinity and the culture of engineering." australian journal of engineering education volume 7(2): 143-156. australasian journal of information systems volume 15 number 1 december 2007 75 meredyth, d., n. russell, l. blackwood, j. thomas and p. wise (1999). real time: computers, change and schooling, commonwealth department of education. canberra, australian key centre for cultural and media policy. mmv (2001). reality bytes. multimedia victoria. melbourne, department of infrastructure, state of victoria. mmv (2004). attitudes to ict careers and study among 17-19 year old victorians. multimedia victoria. melbourne, department of infrastructure, state of victoria. murphy, p. f. and j. elwood (1998). "gendered experiences, choices and achievement exploring the links." international journal of inclusive education 2(2): 95-118. murphy, p. f. and c. v. gipps, eds. (1996). equity in the classroom: towards effective pedagogy for girls and boys. london, falmer press unesco publishing. murphy, p. f. and e. whitelegg (2006). girls in the physics classroom: a review of the research on the participation of girls in physics. london, institute of physics: 61. oecd (2007). education at a glance: oecd indicators. oecd. paris. pajares, f. (2002). self-efficacy beliefs in academic contexts: an outline., http://www.des.emory.edu/mfp/efftalk.html peters, j., n. lane, t. rees and s. gill (2002). set fair: a report on women in science, engineering, and technology from the baroness greenfield cbe to the secretary of state for trade and industry. london, uk department of trade and industry: 98. ramsden, p. (1998). learning to lead in higher education. london, routledge. sandler, b. r., l. a. silverberg and r. hall (1996). the chilly classroom climate: a guide to improve the education of women. washington dc, the national association for women in education. spencer, s. (2003). can you do addition? questioning the domain of it. australian women in it conference, university of tasmania. trauth, e. (2006). agenda for research on gender diversity in the global information economy. encyclopedia of gender and information technology. e. trauth. hershey, idea group reference. 1: xxix xxxiii. trauth, e. m., ed. (2006). encyclopedia of gender and information technology. london, idea group reference. tully, c. j. (1997). "'informal education by computer ways to computer knowledge'." computers in education 27(1): 31-43. ucas (2005). http://www.lmu.ac.uk/ces/ped/apdedict/index.htm. valian, v. (1999). why so slow? : the advancement of women. cambridge massachusetts, the mit press. vtac (2006). victorian tertiary admissions centre. http://www.vtac.edu.au/publications/index.html. wacjman, j. (1991). feminism confronts technology. sydney, allen and unwin. wacjman, j. (2004). technofeminism. cambridge, polity press. australasian journal of information systems volume 15 number 1 december 2007 76 webster, j. (1996). shaping women's work: gender, employment and information technology. london and new york, addison wesley longman limited. webster, j. (2005). women in it professions: corporate structures, masculine cultures. 3rd european symposium on gender & ict: working for change, manchester, uk. wright, r. (1997). women in computing: a cross-national analysis. women in computing. r. lander and a. adam. exeter, intellect books. yates, l. (1993). the education of girls: policy, research and the question of gender. hawthorn, the australian council for educational research. paper title (paper title style) australasian journal of information systems watson, lacey, kerr, salmon & goode 2019, vol 23, research article effects of misuse of personal details on older people 1 understanding the effects of compromise and misuse of personal details on older people judy watson university of the sunshine coast jwatson@usc.edu.au david lacey university of the sunshine coast don kerr university of the sunshine coast paul salmon university of the sunshine coast natassia goode university of the sunshine coast abstract an increasing number of older adults are being affected by scams that can lead to the compromise and misuse of their personal details. previous research has investigated factors that increase the likelihood of falling victim to identity compromise and misuse; less is known about the non-financial impacts that result from the event or about the factors that influence these impacts. the aims of this study are to describe the identity compromise events experienced by older adults, and explore the non-financial impacts (i.e. behavioural, physiological, emotional and psychological), and external factors that might influence these impacts. the study used data collected by an identity compromise and misuse support service in australia. the manner of compromise/misuse, the type and number of credentials involved, and the organisations/agencies that were contacted for advice following the event, were recorded. it was found that most events had an online element, were detected by the victim, and involved multiple identity credentials. participants experienced a variety of behavioural, physiological, emotional and/or psychological impacts. more impacts were experienced where the event had an online element, where more credentials were compromised and where more points of contact were made to reach comprehensive advice. misuse was not a contributory factor to the impacts experienced. implications for further research are discussed. keywords: older adults; identity compromise and misuse; non-financial impacts; identity credentials; online 1 introduction identity compromise and misuse continues to cause individuals financial and non-financial impact (newman & mcnally, 2005). research has investigated strategies and methods for detecting identity compromise and misuse events, and has assessed the effectiveness of such approaches (jenab & moslehpour, 2016; shah, ahmed & soomro, 2016). researchers have explored the demographic factors (anderson, 2006; harrell, 2015) and personality traits (holtfreter, reisig, pratt & holtfreter, 2015) that increase the likelihood of succumbing to identity compromise. society has sought to overcome the issue through technical and australasian journal of information systems watson, lacey, kerr, salmon & goode 2019, vol 23, research article effects of misuse of personal details on older people 2 educational solutions, and financial reimbursement of loss (australian payments clearing association, 2016). though important, financial reimbursement of the victim is not considered a holistic solution as the impacts of identity compromise and misuse are considerably wider, ranging from financial loss to physical, social, emotional and psychological issues (benibo, chambers & thorne, 2016; button, lewis & tapley, 2009a; 2009b; 2012; cross, 2012; golladay & holtfreter, 2017; identity theft resource centre [itrc], 2016; sharp, shreve-neiger, fremouw, kane & hutton, 2004). although financial impacts may be relatively small, many victims experience impacts that negatively affect their general well-being (attorney-general’s department [agd], 2016). physical, behavioural, emotional and psychological impacts may also vary in intensity; victims have reported a variety of levels of severity for each of these impacts (button, lewis & tapley, 2009b). understanding the impacts resulting from events where the amount of financial loss is not the overriding factor may provide the basis for researchers and practitioners to develop better tailored solutions. as a society we are obliged to consider the well-being of an individual whose identity has been compromised and find means to address the non-financial impacts. identity compromise and misuse has been found to corrode faith (australian crime commission [acc], 2013) and allow fear to permeate through an individual’s life (anderson, 2006) to a point where victims may experience years of emotional distress (schmitz, 2008). furthermore it compromises trust and reduces confidence in the internet (australian communications and media authority [acma], 2009), a medium which has been demonstrated to improve the quality of life and health and wellbeing of older adults (hasan & linger, 2016; shillair, rikard, cotton & tsai, 2015; sum, mathews, pourghasem & hughes, 2009). additionally, if the faith that older people have in online transactions (both financial and non-financial) can be restored and extended, government and institutional services may save money with online solutions (griffiths et al., 2017). this may enhance health and well-being through better interventions whilst providing significant savings for taxpayers and the nation. therefore it is important to increase understanding of the antecedents of the impacts that victims’ experience enabling the construction of better support frameworks to assist older adults to cope with an event. the aim of this study is to explore and understand the non-financial impacts of identity compromise and misuse, and the influences that might shape the impacts experienced by adults aged 65 years and older. 2 background and theoretical basis 2.1 identity compromise and misuse affecting older adults it is recognised that the methods criminals use to achieve identity compromise are diverse and are constantly evolving (agd, 2016). reports published in australia and internationally show that increasing numbers of older adults are falling victim to identity compromise and misuse. for example, a survey undertaken in australia in 2014/15 revealed that compared to 2010/11 more australians, aged 55 and over, were exposed to at least one scam (57% and 33% respectively) (australian bureau of statistics [abs], 2016a). it was also reported that credit card fraud in this age group increased to an estimated four percent of the population (abs, 2016a). in the uk, between 2014 and 2015, the number of identity compromise and misuse victims aged over 60 increased by 52 percent (cifas, 2016). in the us, an increasing number of older adults are reported to be falling victim australasian journal of information systems watson, lacey, kerr, salmon & goode 2019, vol 23, research article effects of misuse of personal details on older people 3 to identity compromise and misuse; almost half a million more adults aged over 65, were victims of identity compromise and misuse in 2014 than in 2012 (harrell, 2015). although research has reported that some victims are able to overcome the experience and return to their normal life pattern relatively quickly (benibo et al., 2016; turville, yearwood & miller, 2010), others are devastated by the experience (button, lewis & tapley, 2012a; itrc, 2016; cross, richards & smith, 2016) to the extent that there are a few reported cases of victims committing suicide (cross, smith & richards, 2014). 2.2 impacts of identity compromise on the wellbeing of the victim research has reported a range of physical impacts with detrimental consequences to health, such as insomnia, heart palpitations and loss of appetite (cross et al., 2016; itrc, 2016; sharp et al., 2004). previous studies report that the majority of the participants experience intense emotional and psychological impacts (itrc, 2016) with participants describing that they had experienced a combination of emotions (cross et al., 2016). the range of emotional and psychological impacts include but are not limited to anger, embarrassment, guilt, humiliation, paranoia, stress and worry (cross et al., 2016; itrc, 2016; sharp et al., 2004). in turn, each of these emotional and psychological impacts may lead to further emotional states and behavioural actions as the victim attempts to cope with the circumstance they find themselves in (cross et al., 2016). previously it has been reported that victims of online fraud change their behaviour (smith & hutchings, 2014; turville et al., 2010) and may be less prepared to use online environments (button et al., 2009a; hille, walsh, brach & dose, 2011). other behavioural changes, such as becoming more cautious, have also been noted (abs, 2016b; button et al., 2009b). though these impacts are varied it should be noted that the true range of impacts may not be understood as research acknowledges that victims do not always report to a central authority the fact that they have experienced identity fraud (benibo et al., 2016; smith, 2008). further, what influences the impacts that individuals experience is unclear and there is no specific research on this with regard to older adults. 2.3 theoretical basis technology threat avoidance theory (ttat) (liang & xue, 2009) suggests that when individuals perceive a malicious threat in information technology and believe that the threat is avoidable they are motivated to use problem focused coping measures. if they believe the threat is not completely avoidable emotion focused coping mechanisms will be initiated. ttat proposes that threat perception is greater where perceived susceptibility to a threat and belief in the severity of the threat is increased. furthermore the perception of threat is proposed as the motivating factor in commencing avoidance behaviour (liang & xue, 2009). liang and xue (2009) posit that threat avoidance is instigated with a coping appraisal process which has three antecedents; perceived effectiveness, perceived costs, and self-efficacy. ttat proposes that these antecedents influence the perception of the avoidability of a threat. the theory suggests that where an information technology threat is apparent individuals will be motivated to protect themselves by undertaking a safeguarding action and/or performing emotion-focused coping. the adoption of a problem focussed coping strategy through the action of taking a safeguarding measure is more likely when an individual feels in control of the situation. emotion focused coping strategies are more likely to be adopted where individuals feel they have less situational control (liang & xue, 2009). indeed, coping theories such as protection motivation theory (rogers, 1975) and the extended parallel process model (witte, 1992; witte australasian journal of information systems watson, lacey, kerr, salmon & goode 2019, vol 23, research article effects of misuse of personal details on older people 4 & allen, 2000) posit that emotional responses, such as fear, concern and anxiety, may be caused by threat and coping appraisals. 2.4 hypotheses development the perceived severity of a situation is seen as a critical determinant of protective behaviour (boss, galletta, lowry, moody & polak, 2015; witte, 1992). in a situation where an individual’s identity has been compromised/misused, perceived severity has been seen as an individual’s subjective assessment of the damage caused by the fraudulent use of that information (li, yazdanmehr, wang & rao, 2018). the environment in which an identity compromise and misuse event occurs may be a factor that influences that subjective assessment. indeed, there are fundamental differences between identity compromise events that occur in online and offline environments. for example, individuals who experience identity compromise in an offline environment, such as the physical theft of a purse or wallet, may quickly become aware of the loss and therefore recognise the risk to their identity credentials. where the event occurs online the individual may not know that a compromise has occurred for some period of time afterwards (reyns & randa, 2017) and in the case of romance scams they may be in denial that any compromise has occurred (whitty & buchanan, 2016). additionally, in such cases the credentials that have been compromised may be unknown and difficult to assess adding another layer of complexity to the event and reducing the amount of control a victim has over the unfolding situation. it has been suggested that fear of the unknown may be a causal factor of anxiety and the tipping point for experiencing psychological response (carleton, 2016). any loss of control felt by an individual may make the difference between an individual adopting a problem focussed coping strategy and an emotion focussed coping strategy where more impacts are likely to be experienced. therefore we hypothesise: h1 events that have an online element will result in a greater number of impacts being experienced. it has been reported that there are three main stages of identity compromise and misuse (turville et al., 2010). stage one represents the compromise event where a perpetrator acquires an individual’s identity information, stage two is the misuse stage and stage three occurs when the individual becomes aware of the misuse of their identity (turville et al., 2010). in some cases the misuse stage may be prevented through intervention, this may result in fear of the unknown being reduced or prevented. as a result individuals may feel more in control of the situation and hence more likely to adopt a problem focused coping strategy. therefore we hypothesise: h2 fewer impacts will be experienced where misuse of the compromised identity credentials has not occurred. previous research has acknowledged that there are many channels for reporting financial loss from fraud (button, tapley & lewis. 2012b; cross et al., 2016; smith, 2008). loss of identity credentials that do not necessarily have financial implications (e.g. passports and driving licences) have the potential to extend the number of organisations to which the event may need to be reported. this suggests that the greater the number of credentials compromised, the greater the number of organisations that an individual will have to contact to report the issue. where more organisations have to be contacted it is likely that the time to resolve the issue will be increased. it has been found that the longer it takes a victim to resolve the issue the more likely it is that there will be more impacts of a greater intensity (benibo et al., 2016; harrell, 2015; sharp et al., 2004), and the perceived severity of victimisation is impacted (li et australasian journal of information systems watson, lacey, kerr, salmon & goode 2019, vol 23, research article effects of misuse of personal details on older people 5 al., 2018). additionally the extent of misuse of personal information has been found to contribute to the self-assessment of severity of victimisation, which was positively correlated with perceived distress (li et al., 2018). furthermore, where more credentials are compromised individuals may feel an increased loss of control of their identity. therefore, we hypothesise: h3 more impacts will be experienced where more credentials have been compromised. in addition, we also examined whether older adults experience more impacts when they contact a greater number of organisations/agencies in order to reach comprehensive advice. a delay in accessing comprehensive advice may negatively affect feelings of situational control resulting in individuals being less likely to adopt a problem focused coping strategy. we hypothesise: h4 more impacts will be experienced when a greater number of points of contact are made to reach comprehensive advice. by exploring these hypotheses this study will investigate whether there is a relationship between the level of the identity compromise/misuse event and the impacts experienced following the event in a sample of older adults. 3 method 3.1 research design this study used pre-existing data collected by an identity compromise and misuse support service (ids) from 354 older adults. the aim of the ids is to support individuals from australia and new zealand that experience identity compromise and misuse (www.idcare.org). the ids is a not-for-profit and registered charity. the study was formally approved, by exemption, by the human research ethics committee of the university of the sunshine coast. 3.2 description of data source the dataset was sourced from the call centre associated with the ids. the data had been collected and logged by the ids during telephone calls between the call centre case managers and the individuals who were searching for assistance. the data was collected when an individual first made contact with the service. given the nature of the ids this means that data is collected at a point in time when the individual is still searching for assistance in overcoming the threat to their identity. the data used in this study represents one moment in time and it is important to note that the identity compromise/misuse event may have occurred at a considerably earlier time. interviews vary in length depending on the nature of the identity compromise/misuse event and are tailored to support the individual. prior to data collection, the ids routinely seek, from all individuals, consent to use the data for research purposes. the data was collected and recorded by experienced case managers, trained in psychology, who used semi-structured telephone interviews to gather information from older adults who contacted the ids call centre for assistance over an eight month period. for this study older adults are defined as individuals aged 65 years or older. the interview schedule, designed by the ids, focussed on the ‘journey’ the individual had taken in reaching the ids, other questions were designed to provide greater insight into the event itself (see appendix 1). data was recorded in the customer relationship management australasian journal of information systems watson, lacey, kerr, salmon & goode 2019, vol 23, research article effects of misuse of personal details on older people 6 (crm) software knack, which is a customisable online database platform. responses were notated into the crm software at the time of the telephone call; responses to open ended questions were paraphrased using uniformity of language terminology within the ids organisation. to ensure anonymity, the interview process recorded very little demographic data other than gender, age range and whether or not the individual’s first language was english. various questions (see appendix 1) had predetermined categories of responses from which case managers selected an appropriate response, the predetermined responses were clearly defined resulting in negligible margin for error (see appendix 2). individuals are also asked if they have experienced any impacts following the compromise of their identity, and case managers identify and record the individual’s emotional and psychological response to the event (see appendix 3) using adapted scales. the scales used to measure responses to questions regarding the behavioural, physiological, and emotional and psychological impacts that individuals experience were adapted from counselling theories and approaches. these include passive observations of depression and anxiety indicators including the depression anxiety stress scales (dass) (lovibond & lovibond, 1995) and the state-trait anxiety inventory (balsamo et al., 2013). the details of the adapted scales can be found in appendix 4. 3.3 sample all participants were 65 years of age or over; however the ids does not record specific age data. to be included, participants had to be a resident of australia and had at least one form of their identity information compromised. following data collection, the dataset was examined for cases that related to those not resident in australia (n = 70) and for cases where credential compromise did not occur (n = 12). a total of 82 cases were removed from the data set resulting in 272 australian cases where at least one form of identity compromise had occurred. the sample comprised 144 males (52.9%) and 127 females (46.7%). gender was not recorded for one individual. english was the first language of 265 (97.4%) therefore only 7 (2.6%) were considered to be linguistically diverse. 3.4 data analysis the interview data was coded into the qualitative data software program, nvivo, allowing responses from open questions and those that did not follow the predetermined scheme to be coded accordingly. organisational names nominated in response to other questions were analysed and classified into generic categories representing organisational sectors, for example financial institutions, telecommunication organisations etc. responses from questions regarding the behavioural, physiological, emotional and psychological impacts were analysed and coded to the following four categories by the first named author: behavioural impacts, physiological impacts, self-reported psychological impacts, and identified psychological and emotional impacts. coding examples can be found in table 1. a second analyst undertook an inter-rater coding reliability test on the whole dataset, any differences in coding being resolved through discussion between the analysts. based on the percentage of analytical agreement a reliability score of 80.6 percent was realised. the australasian journal of information systems watson, lacey, kerr, salmon & goode 2019, vol 23, research article effects of misuse of personal details on older people 7 reliability score was calculated by dividing the number of agreements by the total number of times coding was required. this is considered an appropriate way to determine reliability scores with qualitative data and a score above 80 percent is considered to be acceptable (jentsch & bowers, 2005). text coded type of impact i do not really go on the internet for anything apart from searching for information, now i definitely won’t i have always been vigilant, but now am more so. behavioural not sleeping had a huge spike in blood pressure. physiological i don’t trust myself on the computer anymore i feel anxious and worried about what might happen with my details. self-reported psychological paranoid distrusting identified psychological and emotional table 1: coding examples descriptive statistics were used to describe the data. tests of normality indicated that the data was not normally distributed, therefore nonparametric tests were conducted. the mann-whitney u test was used to investigate h1 and h2. the mann-whitney u test is used to test differences between conditions where nonparametric data has been identified (field, 2009). to investigate h3 and h4 scatter plots were used to show relationships and trends within the data. to further explore the associations, bivariate correlations using spearman’s rho were undertaken to identify any significance in the associations. spearman’s rho is used to test for associations where non-parametric data is present (field, 2009). 3.4.1 dependent variables the dependent variables represent self-reported impacts experienced by each participant and impacts identified by case managers. the self-reported impacts variable is a count variable constructed by summing the total number of behavioural, physiological and self-reported psychological impacts reported by participants. likewise, the identified impacts variable is a count variable and again was constructed by summing the total number of emotional and psychological impacts identified by the case managers. 3.4.2 predictor variables the predictor variables represent the factors being investigated, table 2 details the predictor variables and where each was sourced. dependent and predictor variables were coded and calculated by an analyst following which a second analyst undertook an inter-rater coding reliability test, as described in section 3.4, on 20 percent of the dataset. a reliability score of 97.5 percent was realised which was also considered to be acceptable. australasian journal of information systems watson, lacey, kerr, salmon & goode 2019, vol 23, research article effects of misuse of personal details on older people 8 predictor variable source place of compromise/misuse (online/not online/unknown) case notes; compromise methods reported; misuse methods reported compromise only or misuse occurred (compromise only/misuse) case notes; compromise methods reported; misuse methods reported; whether attempted misuse of information was successful number of credentials compromised count variable calculated by summing the number of identity credentials the individual reported as being compromised number of points of contact to reach comprehensive advice count variable calculated by summing the people/organisations/agencies/sources that individuals reported they had connected with while searching for comprehensive advice table 2: source of each predictor variable 4 results 4.1 compromise/misuse event the majority of compromise and/or misuse events (76.84%, n=272) contained an element of online activity, in other words either the compromise or the misuse or both occurred online (59.19%, n=161). a further 34.56 percent (n=94) of the events did not have an online element leaving 6.25 percent (n=17) of the participants not knowing how their identity credentials were compromised or misused. almost all participants (98.53%, n=268) nominated the way in which they were notified of the issue, the majority (76.10%, n=207) detected the compromise/misuse themselves. a further 7.72 percent (n=21) percent were alerted by family, friends or acquaintances, whilst financial institutions and other businesses/agencies each notified 5.88 percent (n=16) of the participants. 4.1.1 compromise event almost 90 percent of the sample (89.97%, n=242) reported that at least one form of their personal identity credentials had been compromised. the remaining participants (11.03%, n=30) did not consider that their personal details had been compromised even though they reported being subject to events such as data breach incidents, remote access scams or household theft of personal papers. of those who were able to report where their credentials were compromised (n=177), almost half (n=79); reported various online situations whilst 55.37 percent (n=98) reported offline situations. twenty four different sources of compromise were cited of which the top five were telephone phishing (33.90%, n=60), data breach (10.17%, n=18), email phishing (7.91%, n=14), website scams (6.21%, n=11) and household theft (5.65%, n=10). 4.1.2 misuse event participants were asked if they believed their compromised credentials had been misused. just over half (58.82%, n=160) reported no misuse, however 41.18 percent (n=112) believed misuse had occurred. of those that believed misuse had occurred 55.36 percent (n=62) reported that the misuse of their information had been successful. a further 14.29 percent (n=16) did not know if the misuse had been successful or not. of those that believed misuse had occurred 70.54 percent (n=79) reported that they were aware of between one and six methods of misuse. on average, 1.67 methods of misuse were reported australasian journal of information systems watson, lacey, kerr, salmon & goode 2019, vol 23, research article effects of misuse of personal details on older people 9 (sd=1.08, n=79). the most common first reported method of misuse was accessing bank funds that did not involve superannuation (39.24%, n=31), followed by credit card fraud (24.05%, n=19) and tax return misuse (8.86%, n=7). 4.2 credentials compromised an average of 4.94 credentials were reported to be compromised (sd=3.35, n=234). most commonly one or four credentials were compromised (15.07%, n=41). the actual credentials reported as compromised were varied, 25 percent (n=68) reported their driving licence as compromised. other credentials reported as compromised, that have high value to criminals (agd, 2016), were passports (21.70%, n=59), credit cards (21.30%, n=58), tax file numbers (19.50%, n=53) and medical card numbers (11.00%, n=30). 4.3 points of contact of those who could name the organisation they first contacted (n=161) most participants reported that they initially contacted a relevant financial institution (24.85%, n=40). state/territory police (19.88%, n=32) were the second most frequent initial point of contact followed by the australian tax office (13.66%, n=22). on average, participants reported that 2.93 points of contact were required to reach comprehensive advice about resuming control of their identity (sd=1.32, n=272). the majority (94.85%, n=258) made multiple points of contact. 4.4 impacts 4.4.1 overview almost three quarters of the participants (74.27%, n=202) self-reported or were identified (by a case manager) as experiencing at least one non-financial impact (behavioural, physiological, emotional and/or psychological). on average, participants who experienced an impact experienced just under three impacts (mean = 2.86; sd = 2. 08, n=202). almost a quarter of the participants (23.53%, n=64) experienced impacts of more than one type, reporting or being identified as experiencing a mixture of behavioural, physiological, emotional and psychological impacts. 4.4.2 impacts reported by participants over a third of the participants (35.66%, n=97) reported experiencing at least one impact. psychological impacts were the most common type of self-reported impact experienced (18.75%, n=51). self-reported psychological impacts of those that self-reported an impact over half of the participants (52.58%, n=51) reported at least one psychological impact. as shown in table 3, the most frequently self-reported psychological impact was an awareness and concern that personal information was in the hands of others (25.49%, n=13). self-reported behavioural impacts of those that reported an impact, almost half 47.42 percent (n=46) described at least one behavioural impact. the most frequently reported behavioural impact was considering no longer using an online device (19.57%, n=9), as shown in table 3. australasian journal of information systems watson, lacey, kerr, salmon & goode 2019, vol 23, research article effects of misuse of personal details on older people 10 self-reported physiological impacts of those that self-reported an impact almost one third of the participants (30.93%, n=30) reported at least one physiological impact. as shown in table 3, the most frequently reported physiological impact was a feeling of nausea (73.33%, n=22). impact type impact % participants self-reported psychological impacts aware/concerned that information is in hands of others 25.49 worried about the future 19.61 stressed 17.65 feels reputation has been affected 11.76 feels isolated 7.84 other self-reported psychological impact 62.75 behavioural impacts is considering no longer using online device 19.57 negative impact on relationships 15.22 closed affected accounts 13.04 stopped using online device 8.70 time off work 4.35 stopped online shopping 2.17 is considering no longer shopping online 2.17 other behavioural impact 47.83 physiological impacts feels sick 73.33 trouble sleeping 40.00 trouble staying asleep 6.67 other physiological impact 3.33 table 3: most common self-reported impacts by impact type (multiple responses possible) 4.4.3 impacts identified by case manager impacts were identified in 62.50 percent (n=170) of the participants. emotional impacts were the most common type of identified impact with 58.82 percent of participants (n=160) being affected. emotional impacts identified by case manager over 90 percent of those identified by case managers as experiencing impacts (94.12%, n=160) were identified as experiencing at least one emotional impact. on average, these participants experienced 1.88 emotional impacts (sd=0.98, n=160). as shown in table 4, the most frequently identified emotional impact was stupidity (26.25%, n=42), psychological impacts identified by case manager over 50 percent of those identified by case managers as experiencing impacts (51.18%, n=87) were identified as experiencing at least one psychological impact. on average, these participants experienced 1.33 psychological impacts (sd=0.68, n=87). as shown in table 4, the most frequently identified psychological impact was being watchful and on guard (50.58%, n=44). australasian journal of information systems watson, lacey, kerr, salmon & goode 2019, vol 23, research article effects of misuse of personal details on older people 11 impact type impact % participants emotional impacts stupid 26.25 silly 18.13 vulnerable 16.25 anxious 16.25 embarrassed 15.00 foolish 14.38 distrusting 13.75 overwhelmed 13.13 regretful 12.50 violated 9.38 irritable 9.38 angry 9.38 deceived 6.25 helpless 5.63 paranoid 4.38 psychological impacts watchful and on-guard 50.57 no longer want to engage online 19.54 scared that they know where i live 12.64 have waves of strong feeling about it 11.49 trouble concentrating 8.05 other psychological impact identified by case manager 25.29 table 4: impacts most commonly identified by case manager (multiple responses possible) 4.5 h1 events that have an online element will result in a greater number of impacts being experienced a mann-whitney test indicated that the number of self-reported impacts was significantly greater when the compromise and/or misuse of personal information occurred in an online environment (mdn=0, range = 0-7) rather than an offline environment (mdn=0, range = 0-5); u = 6439.50, p = .01. however, a mann-whitney test indicated that the number of impacts identified by case manager was not significantly greater when the compromise and/or misuse of personal information occurred in an online environment (mdn=1, range = 0-8) than an offline environment (mdn=1, range = 0-7), u = 7002.50, p = .15. 4.6 h2 fewer impacts will be experienced where misuse of the compromised identity credentials has not occurred a mann-whitney test indicated that the number of self-reported impacts was not significantly fewer when identity credentials were only compromised (mdn=0, range = 0-5) than where misuse also occurred (mdn=0, range = 0-7), u = 8939.50, p = .47. similarly, a mann-whitney test indicated that the number of impacts identified by case manager was not fewer when only identity credentials were compromised (mdn=1, range = 0-7) than where misuse also occurred (mdn=1, range = 0-8), u = 8009.00, p = .06. 4.7 h3 more impacts will be experienced where more credentials have been compromised as shown on the left hand side of figure 1, there was a small to medium significant positive correlation between the number of credentials compromised and the number of self-reported impacts, rs = .21, p <.01. as shown on the right hand side of figure 1, there was also a small significant positive correlation between the number of credentials compromised and the number of impacts identified by case manager, rs = .13, p =.03. australasian journal of information systems watson, lacey, kerr, salmon & goode 2019, vol 23, research article effects of misuse of personal details on older people 12 figure 1: scatter plot graphs for the relationships between the number of credentials compromised and the number of self-reported and identified impacts 4.8 h4 more impacts will be experienced when a greater number of points of contact are made to reach comprehensive advice as shown on the left hand side of figure 2, there was a small significant positive correlation between the number of points of contact to reach comprehensive advice and the number of self-reported impacts, rs = .16, p <.01, the effect size was small. as shown on the right hand side of figure 2, there was also a medium significant positive correlation between the number of points of contact and the number of impacts identified by case manager, rs = .37, p <.01. figure 2: scatter plot graphs for the relationship between the number of points of contacts made to reach comprehensive advice and the number of self-reported and identified impacts 5 discussion the aims of this study were to understand the impacts that older adults experience following identity compromise and misuse, and to build knowledge regarding the factors that influence them by testing the hypotheses detailed in section 2.4. approximately three quarters of the participants experienced at least one non-financial impact from falling victim to identity compromise/misuse; the greater percentage experienced more than one impact. over one third of the participants were able to self-report a non-financial australasian journal of information systems watson, lacey, kerr, salmon & goode 2019, vol 23, research article effects of misuse of personal details on older people 13 impact, while two thirds were identified, by a case manager, as experiencing some form of this type of impact. most detected the compromise and/or misuse of their credentials themselves and almost all connected with multiple points of contact to reach comprehensive advice. not all knew how their credentials were compromised or misused however the majority involved an online element. further the majority had multiple credentials compromised and at least one of the credentials was of high value, value being determined by financial worth in illegal online marketplaces (agd, 2016). the dataset provided evidence to support h1, the place of compromise and/or misuse (online) affected the number of self-reported impacts that were experienced. however there was no evidence to support h2, the added knowledge of misuse occurring did not increase the number of impacts experienced. a level of support was found for h3, suggesting that where a greater number of credentials are compromised more impacts will be experienced. further, support was also found for h4 indicating that where more points of contact are made before reaching comprehensive advice more impacts will be experienced. 5.1 main non-financial impacts undoubtedly older adults are impacted behaviourally, physiologically, emotionally and psychologically, and many but not all recognise the non-financial impacts they experience. in terms of frequency, emotional and psychological impacts dominated the experiences. most commonly the experience left older adults more watchful and on-guard with participants being worried for the future and concerned that their personal information was in the hands of others. this suggests that they are aware that whether or not misuse has already occurred there is a potential that it could occur in the future. the reported physical impacts of feeling nauseous and experiencing sleep pattern disruption are likely to be the physiological manifestation of this worry and concern. 5.2 behavioural change almost one third of those reporting a behavioural impact either considered stopping or had stopped using their online device. further, no longer wanting to engage online was the second most identified psychological impact affecting almost 20 percent of the participants. this a concern when the drive is to move organisational contact methods to digital communication, and when research has reported that internet use can enhance older adults’ quality of life and general well-being (hasan & linger 2016; shillair et al., 2015; sum et al., 2009). 5.3 influential factors identity compromise and misuse events may be multifaceted with regard to many factors. these factors include place of compromise, place of misuse, methods of misuse, credentials involved, and knowledge of how to recover from the event. notably, victims may not comprehend all of these factors. research has reported that victims do not always know how compromise of their identity credentials occurs (roberts, indermaur & spiranovic, 2013). in this study most participants were able to describe how their identity credentials were compromised and/or misused. for this dataset, the online element of the event was significantly related to participants experiencing a greater number of self-reported impacts. however, the fact that compromise had extended to misuse also occurring did not influence the number of impacts experienced. this study has shown variance in the number of credentials compromised. potentially, where more credentials are compromised, victims may feel greater confusion in knowing who to australasian journal of information systems watson, lacey, kerr, salmon & goode 2019, vol 23, research article effects of misuse of personal details on older people 14 contact and the reporting timescale may be extended by more organisations being involved. prior research has identified that the amount of time spent resolving any financial impacts is directly related to amount of emotional distress and relationship issues encountered (harrell, 2015). the results from this study indicate a link between the impacts experienced and both the number of credentials compromised, and the number of points of contact made to reach comprehensive advice. 5.4 the response journey the response and recovery journey commences at the point of discovery. the study showed that on discovery the participants first contacted an array of different people, organisations or agencies. financial institutions were the most common first point of contact; this may be because a victim’s first thought is to secure their finances and prevent further loss, or because the credential compromised was issued by a financial institution. other first points of contact include law enforcement agencies and organisations that issue credentials (e.g. passport office) or manage accounts (e.g. telco provider). whatever the victims’ intention, this study corroborates prior research that acknowledges a network of organisations involved in victims’ response journeys (button et al., 2012b; cross et al., 2016), and in the identity ecosystem (watson, salmon, lacey & kerr, 2015). research on fraud has shown that having many reporting channels leads to confusion in knowing who to report the crime to (button et al., 2012b; smith, 2008; uk attorney general’s office, 2006). the people, organisations and agencies that older adults connect with before they reach comprehensive advice are an important link in the recovery process. research has reported that following identity compromise and misuse, distrust commonly extends to almost everyone that the victim comes into contact with and this is exacerbated when it takes longer to restore their name (benibo et al., 2016). this study has provided confirmatory evidence in support of ttat. circumstances where emotional and psychological responses were heightened resulted from instances where there was less situational control. furthermore, it was found that the theory applied in a context where users had already succumbed to a threat, identity compromise and misuse. additionally, environmental factors (place, extent of event and length of response journey) have been found to influence the likelihood of users adopting emotional coping strategies, thereby building on ttat. future research should take into consideration the role that online environments, number of credentials compromised and length of response journey play in the post event experience of victims, and therefore in the likelihood of victims undertaking safeguarding actions or performing emotion-focused coping. future research must consider the well-being of individuals whose identity has been compromised. 5.5 future directions more detailed understanding of the response network would develop greater knowledge of the response journey, highlight opportunities for feedback between stakeholders, and identify other individuals who provide support, such as social support, to the victim. research has found that social support from family members results in less negative consequences for the victim (golladay & holtfreter, 2017). greater understanding facilitates the opportunity to inform policy and practice, and thereby bolster support through better interventions. this study has added to the knowledge-base by reporting on the impacts experienced by older adults and querying links between non-financial impacts and both the journey to seek australasian journal of information systems watson, lacey, kerr, salmon & goode 2019, vol 23, research article effects of misuse of personal details on older people 15 comprehensive advice, and the diversity of the identity compromise and misuse event. future research should document the network of people, organisations and agencies that are involved in the response journey of older adults. given the diversity of identity compromise and misuse events, investigation may be best served by undertaking case studies. further research could extend across other demographic profiles. 5.6 limitations the study was limited by the small sample size. additionally, due to further data limitations, it was not possible to compare results across different age groups. however, the findings are in line with previous multi-age group research which reports that impacts go beyond financial loss, and that there are many avenues for reporting an event. the data was also limited by the small percentage (11.7%) of the participants reporting any behavioural impact. this may be affected by the timing of the data collection as approximately two thirds of this study’s cases were collected within seven days of the participant first becoming aware of the event. in that narrow time interval, participants may not have been conscious of behavioural impacts resulting from the event. future research should investigate behavioural change at a later point in the response and recovery journey. 6 conclusion this study has increased knowledge around the identity compromise and misuse events that are experienced by older adults in australia. the findings show that events are often diverse in occurrence and outcome. many of the participants experienced non-financial impacts. these impacts were behavioural, physiological, emotional and/or psychological and it was not uncommon to experience multiple impacts. the findings suggest that when identity compromise and/or misuse occurs in an online environment older adults experience more impacts which may affect their behaviour. furthermore, the actions that an older adult takes in responding to an identity compromise event appear to affect the non-financial impacts that are experienced. finally the findings indicate that the number of credentials compromised matters as more impacts were experienced when more credentials were involved. acknowledgement the authors would like to thank the reviewers for their helpful comments. paul salmon's contribution to this article and research programme is supported through an australian research council future fellowship (ft140100681). references anderson, k.b. (2006). who are the victims of identity theft? the effect of demographics. journal of public policy & marketing, 25, 160–171. doi: 10.1509/jppm.25.2.160 attorney-general’s department (agd). (2016). identity crime and misuse in australia 2016. retrieved from https://www.ag.gov.au/rightsandprotections/identitysecurity/documents/identitycrime-and-misuse-in-australia-2016.pdf australian bureau of statistics (abs). (2016a). personal fraud, 2014-15. in focus: experiences of personal fraud for persons aged 55 and over (cat no. 4528.0). retrieved from http://www.abs.gov.au/ausstats/abs@.nsf/lookup/4528.0main+features17201415?opendocument australasian journal of information systems watson, lacey, kerr, salmon & goode 2019, vol 23, research article effects of misuse of personal details on older people 16 australian bureau of statistics (abs). (2016b). personal fraud, 2014-15 (cat no. 4528.0). retrieved from http://www.abs.gov.au/ausstats/abs@.nsf/latestproducts/4528.0main%20features15201 4-15?opendocument&tabname=summary&prodno=4528.0&issue=201415&num=&view= australian communications and media authority (acma). (2009). australia in the digital economy, report 1: trust and confidence. retrieved from http://www.acma.gov.au/webwr/aba/about/recruitment/trust_and_confidence_aust_in _digital_economy.pdf australian crime commission (acc). (2013). identity crime. retrieved from https://www.crimecommission.gov.au/sites/default/files/identity%20crime%20jul y%202013.pdf australian payments clearing association. (2016). media release apca releases interim payments fraud data. retrieved from http://www.apca.com.au/docs/default-source/2016-mediareleases/apca-releases-interim-payments-fraud-data.pdf author (2015). balsamo, m., romanelli, r., innamorati, m., ciccarese, g., carlucci, l., & saggino, a. (2013). the state-trait anxiety inventory: shadows and lights on its construct validity. journal of psychopathology and behavioral assessment, 35(4), 475-486. doi: 10.1007/s10862-013-9354-5 benibo, b.r., chambers, v., & thorne, b. (2016). breaking bad news to victims of identity theft: lessons from medical doctors. journal of accountancy, 222(2), 30–34. retrieved from https://www.journalofaccountancy.com/issues/2016/aug/giving-bad-news-to-identitytheft-victims.html boss, s.r., galletta, d.f., lowry, p.b., moody, g.d., & polak, p. (2015). what do systems users have to fear? using fear appeals to engender threats and fear that motivate protective security behaviours. mis quarterly 39(4): 837-864. button, m., lewis, c., & tapley, j. (2009a). fraud typologies and victims of fraud: literature review. london: national fraud authority. retrieved from https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/118469 /fraud-typologies.pdf button, m., lewis, c., & tapley, j. (2009b). a better deal for fraud victims: research into victims’ needs and experiences. london: national fraud authority. retrieved from https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/118468 /better-deal-for-fraud-victims.pdf. button, m., lewis, c., & tapley, j. (2012a). not a victimless crime: the impact of fraud on individual victims and their families. security journal, 27(1), 36–54. doi: 10.1057/sj.2012.11 button, m., tapley, j., & lewis, c. (2012b). the ‘fraud justice network’ and the infra-structure of support for individual fraud victims in england and wales. criminology and criminal justice, 13, 37-61. doi: 10.1177/1748895812448085 carleton, r.n. (2016). into the unknown: a review and synthesis of contemporary models involving uncertainty. journal of anxiety disorders, 39, 30–43. doi: 10.1016/j.janxdis.2016.02.007. australasian journal of information systems watson, lacey, kerr, salmon & goode 2019, vol 23, research article effects of misuse of personal details on older people 17 cifas. (2016). criminals target uk youth as identity fraud rises. retrieved from https://www.cifas.org.uk/press_centre/criminals_target_uk_youth_as_dentity_fraud_r ises cross, c. (2012). the donald mackay churchill fellowship to study methods of preventing and supporting victims of online fraud. retrieved from: http://eprints.qut.edu.au/view/person/cross,_cassandra.html cross, c., richards, k., & smith, r.g. (2016). the reporting experiences and support needs of victims of online fraud. trends and issues in crime and criminal justice, no. 518. retrieved from http://www.aic.gov.au/media_library/publications/tandi_pdf/tandi518.pdf cross, c., smith, r.g., & richards, k. (2014). challenges of responding to online fraud victimisation in australia. trends and issues in crime and criminal justice, no. 474. retrieved from http://www.aic.gov.au/media_library/publications/tandi_pdf/tandi474.pdf field, a. (2009) discovering statistics using spss. (3rd ed.) london: sage publications ltd. golladay, k., & holtfreter, k. (2017). the consequences of identity theft victimisation: an examination of emotional and physical health, victims & offenders, 12, 741-760. doi: 10.1080/15564886.20161177766 griffiths, f., bryce, c., cave, j., dritsaki, m., fraser, j., hamilton, k.,,…sturt, j. (2017). timely digital patient-clinician communication in specialist clinical services for young people: a mixed-methods study (the lync study), journal of medical internet research, 19(4), e102. doi: 10.2196/jmir.7154 harrell, e. (2015). victims of identity theft, 2014. retrieved from http://www.bjs.gov/content/pub/pdf/vit14.pdf hasan, h., & linger, h. (2016). enhancing the wellbeing of the elderly: social use of digital technologies in aged care. educational gerontology, 42, 749-757. doi:10.1080/03601277.2016.1205425. hille, p., walsh, g., brach, s., & dose, d. (2011). why online identity theft poses a major threat to e-business. proceedings of the acm websci’11, 1–2. retrieved from: http://journal.webscience.org/518/1/207_paper.pdf. holtfreter, k., reisig, m.d., pratt, t.c., & holtfreter, r.e. (2015). risky remote purchasing and identity theft victimization among older internet users. psychology, crime & law, 21, 681– 698. doi: 10.1080/1068316x.2015.1028545 identity theft resource centre (itrc). (2016). identity theft: the aftermath 2016. retrieved from http://www.idtheftcenter.org/images/page-docs/aftermathfinal_2016.pdf jenab, k. & moslehpour, s. (2016). cyber security management: a review. business management dynamics, 5(11), 16–39. retrieved from https://www.researchgate.net/profile/kouroush_jenab/publication/305220294_cyber_s ecurity_management_a_review/links/578510d408aef321de2a8f90.pdf jentsch, f., & bowers, c. (2005). team communication analysis. in n. a. stanton, a. hedge, k. brookhuis, e. salas, & h. hendrick (eds.), handbook of human factors and ergonomics methods. boca raton, fl: crc press. australasian journal of information systems watson, lacey, kerr, salmon & goode 2019, vol 23, research article effects of misuse of personal details on older people 18 li, y., yazdanmehr, a., wang, j., & rao, h.r. (2018). how do you cope: individuals’ responses to identity theft victimization. paper presented at the twenty-fourth americas conference on information systems, amcis 2018, new orleans, la, august 16-18. liang, h. & xue, y. (2009). avoidance of information technology threats: a theoretical perspective. mis quarterly, 33(1), pp.71–90. doi:10.2307/20650279. lovibond, p.f., & lovibond, s.h. (1995). the structure of negative emotional states: comparison of the depression anxiety stress scales (dass) with the beck depression and anxiety inventories. behaviour research and therapy, 33, 335-343. doi: 10.1016/00057967(94)00075-u newman, g.r. & mcnally, m.m. (2005). identity theft literature review, national institute of justice focus group meeting. retrieved from https://pdfs.semanticscholar.org/bb56/26ec1b5d31ff299c325a4bca93eee725de24.pdf. reyns, b.w., & randa, r. (2017). victim reporting behaviors following identity theft victimization: results from the national crime victimization survey. crime & delinquency, 63, 814–838. doi: 10.1177/0011128715620428 roberts, l.d., indermaur, d., & spiranovic, c. (2013). fear of cyber-identity theft and related fraudulent activity. psychiatry, psychology and law, 20, 315-328. doi: 10.1080/13218719.2012.672275 rogers, r.w. (1975). a protection motivation theory of fear appeals and attitude change1. the journal of psychology, 91(1), pp.93–114. doi:10.1080/00223980.1975.9915803. schmitz, c. (2008). identity theft victim claims emotional distress. inside counsel, (196), pp.65– 66. shah, m.h., ahmed, j. & soomro, z.a. (2016). investigating the identity theft prevention strategies in m-commerce. proceedings of the international conferences its, icedutech and ste, 59–66. retrieved from http://files.eric.ed.gov/fulltext/ed571576.pdf sharp, t., shreve-neiger, a., fremouw, w., kane, j., & hutton, s. (2004). exploring the psychological and somatic impact of identity theft, journal of forensic sciences, 49, 1-6. doi: 10.1520/jfs2003178. shillair, r., rikard, r. v., cotten, s.r., & tsai, h-y. s. (2015). not so lonely surfers: loneliness, social support, internet use and life satisfaction in older adults. proceedings of the iconference 2015. retrieved from http://hdl.handle.net/2142/73666 smith, r.g. (2008). coordinating individual and organisational responses to fraud. crime, law and social change, 49, 379-396. doi: 10.1007/s10611-008-9112-x smith, r.g., & hutchings, a. (2014). identity crime and misuse in australia: results of the 2013 online survey. retrieved from http://www.aic.gov.au/media_library/publications/rpp/128/rpp128.pdf sum, s., mathews, r.m., pourghasem, m., & hughes, i. (2009). internet use as a predictor of sense of community in older people. cyberpsychology & behavior, 12(2): 235–239. doi:10.1089/cpb.2008.0150 australasian journal of information systems watson, lacey, kerr, salmon & goode 2019, vol 23, research article effects of misuse of personal details on older people 19 turville, k., yearwood, j., & miller, c. (2010). understanding victims of identity theft: preliminary insights. proceedings of the 2nd cybercrime and trustworthy computing workshop, ctc 2010, 60–68. doi: 10.1109/ctc.2010.12 uk attorney general’s office. (2006). fraud review: final report. retrieved from: http://webarchive.nationalarchives.gov.uk/20070222120000/http://www.lslo.gov.uk/pdf /fraudreview.pdf whitty, m.t., & buchanan, t. (2016). the online dating romance scam: the psychological impact on victims both financial and non-financial. criminology and criminal justice, 16, 176-194. doi: 10.1177/1748895815603773 witte, k. (1992). putting the fear back into fear appeals: the extended parallel process model. communication monographs, 59, pp.329–349. doi:10.1080/03637759209376276 witte, k. & allen, m. (2000). a meta-analysis of fear appeals: implications for effective public health campaigns. health education & behavior, 27(5), pp.591–615. doi:10.1177/109019810002700506 australasian journal of information systems watson, lacey, kerr, salmon & goode 2019, vol 23, research article effects of misuse of personal details on older people 20 appendix 1 interview questions questions do you believe your personal details have been compromised? what do you believe was the method or source of your information being compromised? what was the other source of compromise? what credentials do you believe are compromised? which other credentials? do you believe there has also been misuse of your identification? in what ways are you aware your compromised information has been misused? what other way was information used? was the attempted misuse of your information successful? in what way were you notified of the issue? notified by other? who did you first contact? other first contact have you already engaged another organisation for assistance with this matter? which organisation? #1 is a 2nd organisation already engaged? 2nd org: which organisation? is a 3rd organisation already engaged? 3rd org: which organisation? how did you find out about the ids? which partner organisation? case notes (text about the individual’s case) gender state of residence age range linguistic diversity australasian journal of information systems watson, lacey, kerr, salmon & goode 2019, vol 23, research article effects of misuse of personal details on older people 21 appendix 2 questions with predetermined categories question predetermined response what do you believe was the method or source of your information being compromised? cyber stalking/digital stalking known to me cyber stalking/digital stalking not known to me data breach email face-to-face – known to me job application lost phishing – email phishing sms/text message phishing – telephone remote access scam social media telephone theft from household theft from letterbox theft from person theft from vehicle australasian journal of information systems watson, lacey, kerr, salmon & goode 2019, vol 23, research article effects of misuse of personal details on older people 22 question predetermined response unsolicited contact through mail virus malware/spyware website competition/survey website – job application website – pop-up website – romance/dating scam website – travel visa website scam other unknown what credentials do you believe are compromised? address australian tax office/tax file number bank account centrelink certificates birth/death/marriage computer/networking credit/debit card date of birth driver’s licence australasian journal of information systems watson, lacey, kerr, salmon & goode 2019, vol 23, research article effects of misuse of personal details on older people 23 question predetermined response email address full name insurance login info medicare mobile phone mygov passport phone photos superannuation utilities account details other unknown in what ways are you aware your compromised information has been misused? acquiring personal loan bank – access funds (not super) bank new account credit card computer – key-logger australasian journal of information systems watson, lacey, kerr, salmon & goode 2019, vol 23, research article effects of misuse of personal details on older people 24 question predetermined response computer – malware/virus computer ransomware damage reputation impersonate email impersonation on chat sites manipulate social media mobile phone account (including porting) mobile phone new account obtain investment money/shares obtain medical benefits obtain debit cards open online account (email/social media/etc) personal info passed to others purchase products redirection of mail tax return tfn used transfer funds other australasian journal of information systems watson, lacey, kerr, salmon & goode 2019, vol 23, research article effects of misuse of personal details on older people 25 question predetermined response none in what way were you notified of the issue? another business/agency australian tax office bank or financial institution credit bureau family member/friend/acquaintance self-detected police utility organisation other how did you find out about the ids? community outreach friend/acquaintance media partner organisation web search other australasian journal of information systems watson, lacey, kerr, salmon & goode 2019, vol 23, research article effects of misuse of personal details on older people 26 appendix 3 interview questions designed to identify impacts questions in what way, if any, has your behaviour changed since this event? in what other ways has this use of your information impacted you? psychosomatic effects – physical reactions emotional impacts/responses australasian journal of information systems watson, lacey, kerr, salmon & goode 2019, vol 23, research article effects of misuse of personal details on older people 27 appendix 4 scales used to measure responses (adapted from counselling theories and approaches) question scale in what way, if any, has your behaviour changed since this event? affected reputation closed affected accounts is considering no longer shopping online is considering no longer using online device negative impact on relationships stopped online shopping stopped using online device time off work other psychosomatic effects – physical reactions aware have a lot of feelings about it, but don't want to deal with them feel as if it hadn't happened or isn't real feel sick feeling numb about it have dreams about it i think about it when i am not meaning to i try to remove it from my memory jumpy and easily startled no longer want to engage online australasian journal of information systems watson, lacey, kerr, salmon & goode 2019, vol 23, research article effects of misuse of personal details on older people 28 other things keep making me think about it pictures about it pop into my mind scared that they know where i live trouble concentrating trouble sleeping trouble staying asleep try not to talk about it try not to think about it watchful and on-guard waves of strong feeling about it emotional impacts/responses angry anxious deceived distrusting embarrassed foolish helpless irritable overwhelmed paranoid australasian journal of information systems watson, lacey, kerr, salmon & goode 2019, vol 23, research article effects of misuse of personal details on older people 29 regretful silly stupid violated vulnerable copyright: © 2019 watson, lacey, kerr, salmon & goode. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems eze, gleasure & heavin 2019, vol 23, research on health information systems factors that influence appraisal of mhealth tools 1 understanding the factors that influence the primary appraisal of mhealth tools in developing countries: an exploratory case-study in nigeria emmanuel eze department of business information systems, university college cork, cork, ireland emmanuel.eze@ucc.ie rob gleasure university college cork, cork, ireland ciara heavin university college cork, cork, ireland abstract shortages of health workers, infrastructural deficiencies, limited access to medical care are just a few of the many barriers to care in developing countries. the integration of smartphones and mobile devices into healthcare systems has been proposed to address some of the physical barriers to care and service delivery. these mhealth solutions extend the reach of medical care into rural areas of developing countries. however, it is not clear how mhealth solutions designed and tested in one developing region can be positively appraised for use in others. this study frames this problem using a coping theory approach based on an exploratory casestudy to understand the factors that influence primary appraisal of smartphone-enabled clinical guidelines (mhealth tool) for accessing, classifying and eliciting treatment recommendation for sick children under the age of five by rural healthcare workers (rhcws). findings identified a set of factors which are bound as an emerging explanatory positivity model that influence primary appraisal of an mhealth tool in a new context. these factors are the set of individual and social factors that governments, funding bodies and nongovernmental organisations should consider before embarking on the introduction of an mhealth tool in rural communities of developing countries. it is envisaged that by understanding the factors that influence primary appraisal, that is, either as an opportunity or a threat, practitioners and organisations will support positive appraisal and minimise the occurrence of negative ones when introducing mhealth tools. these findings have implications for theory, practice, and future research as explained in the concluding section of this paper. keywords: healthcare; developing countries; mobile technology; coping theory; mhealth; rural healthcare workers. 1 introduction the ubiquitous nature of mobile information technology (it) presents an opportunity to stimulate developmental activities in rural areas of developing countries (datta, byrd, okoli, & mbarika, 2005; furuholt & matotay, 2011). mobile devices have the potential to overcome australasian journal of information systems eze, gleasure & heavin 2019, vol 23, research on health information systems factors that influence appraisal of mhealth tools 2 some of the physical challenges and infrastructural deficiency that hold back these areas (aker & mbiti, 2010; lee, levendis, & gutierrez, 2012). this is due in part to the unique mobility and smaller infrastructural requirements when compared to landlines (aker & mbiti, 2010; lee et al., 2012). the developmental paradigm surrounding mobile phones has shifted from one that simply reduces communication and coordination costs to one that could transform lives through transformative applications of mobile services (aker & mbiti, 2010; kahn, yang, & kahn, 2010). one example is the integration of smartphones and mobile devices into healthcare systems to address some of the challenges to care and service delivery in rural areas of developing countries (donner & mechael, 2012; free et al., 2013). the strategies of incorporating mobile technologies in healthcare services are collectively known as mobile health (mhealth) (donner & mechael, 2012; kahn et al., 2010). the use of mhealth tools can vary in focus (eze, gleasure, & heavin, 2016b, 2018). first, mprevention/education tools provide preventive, advisory, counselling, and educational services (e.g. hacking et al., 2016; nhavoto, grönlund, & klein, 2017). second, mdatacollection tools are used to collect data that may inform other aspects of healthcare delivery (e.g kabuya, wright, odama, & o'mahoney, 2014; simon & seldon, 2012). third, mdiagnosis applications are used to support the diagnosis of particular conditions (e.g. chib & chen, 2011; mavhu et al., 2017). fourth, mtreatment apps are used to guide remedial healthcare interventions for specific patients (e.g. alam, khanam, & khan, 2010; hufnagel, 2012). the potential of these mhealth tools to navigate some of the barriers to medical care in developing countries has prompted a number of initiatives by governments, nongovernmental (ngos), and research organisations to invest in innovative mhealth approaches to healthcare delivery. however, research has shown that most of these initiatives have struggled with deployment, particularly during the progression from pilot stages to large-scale nation-wide roll-out (chib, van velthoven, & car, 2015; heeks, 2006). although many scholars have used various models, e.g., venkatesh, morris, davis, and davis (2003); taylor and todd (1995), and rogers (2003) to understand users’ adoption processes but a few have examined users’ appraisal process before adoption or use. it is argued that understanding an individual’s cognitive appraisal process which provides information about the individual’s behaviours or emotions would help a researcher understand the individual’s disposition (hareli & hess, 2010). this implies, that individual’s behaviours or emotions influence individual’s appraisal processes towards, for example, an it occurrence in their environment (e.g. beaudry & pinsonneault, 2005; wisniewski, xu, & chen, 2014). in a health context, scholars posit that a coping theory/framework can be effectively employed in the intervention, assessment or evaluation of an individual’s psychological stress and coping responses (fadel & brown, 2010; lyon, 2000). this study uses coping theory to explore perceptions around new mhealth initiatives, with particular attention to perceived threats and opportunities as appraisal outcomes. more specifically, we ask what are the factors that influence the primary appraisal of an mhealth tool in a developing country? the rest of the paper is outlined as follows. section 2 discusses the theoretical background of coping and appraisal as it applies to mhealth in developing countries. section 3 describes an exploratory case-study approach based on the potential introduction of a new mhealth tool for assisting the treatment of sick children under the age of five in nigeria. section 4 presents the findings of the study, which are bound as an emerging explanatory model for the primary australasian journal of information systems eze, gleasure & heavin 2019, vol 23, research on health information systems factors that influence appraisal of mhealth tools 3 appraisal of mhealth tools in developing countries. sections 5 and 6 discuss the findings in relation to existing studies and presents a summary and conclusions. 2 primary appraisal and coping with new technologies to understand primary appraisal and coping we turn to the theory by lazarus and folkman (1984) on stress, appraisal and coping from the social psychology literature. coping in information systems (is) research is conceptualised as adaptation strategies, and this allows us to understand the individuals’ behaviours that occur before, during, and after the implementation of a new technology (beaudry & pinsonneault, 2005), such as mhealth tool. lazarus and folkman (1984) assert that individuals employ two-way processes to cope with a disruptive new it occurrence, i.e., appraisal and coping. 2.1 coping theory coping theory is used to explore and understand the underlying relationships on how individuals respond to an it occurrence in their environment (beaudry & pinsonneault, 2005; bhattacherjee, davis, connolly, & hikmet, 2017). coping is defined as the “cognitive and behavioural efforts to manage specific external and/or internal demands that are appraised as taxing or exceeding the resource of the person [individual]” (lazarus & folkman, 1984, p. 141). ‘internal’ demands are personal needs or requirements such as the desire to excel, perform or execute, and ‘external’ demands refer to those activities impacted or influenced by the external environment (beaudry & pinsonneault, 2005; bhattacherjee et al., 2017). coping is a significant concept in is for theory and research on it adaptation (claggett, 2010; fadel & brown, 2010). coping theory explains the processes by which individuals frame and respond to disruptive events in their environment/workplace, such as a new it occurrence (beaudry & pinsonneault, 2005; wisniewski et al., 2014). is scholars have applied coping theory in organisational settings to understand the individual cognitive responses to new it in a work environment, three examples include: 1) in the context of an it adaptation in a banking setting, beaudry and pinsonneault (2005), by building on the works of lazarus and folkman (1984) offered us an integrated model for understanding users’ adaptation to an it occurrence in a workplace, known as coping model of user adaptation (cmua). in applying coping in it banking settings, cmua adopts a process-oriented approach to coping and outlined four adaptation responses (beaudry & pinsonneault, 2005). 2) in the context of an it avoidance in a security oriented setting, liang and xue (2009) used coping in conjunction with cybernetics to present us with an integrated processes theory of coping and variance theory known simply as the technology threat avoidance theory (ttat). by applying coping and variance theories in it business settings, ttat adopts a process-oriented approach to coping and variance theories to explain the individual it user’s behaviour of avoiding ’threat’ of malicious information technologies (liang & xue, 2009). 3) in the context of an it appraisal and coping in a healthcare setting, fadel and brown (2010), utilised the cmua model in a developed country environment to set the first step toward integrating theories of is ‘adoption and use’ with coping theory by examining how adoptionrelated is perceptions influence individual-level post-adoptive is appraisal. these studies underline the significance of the application of coping processes in is research to understand individuals’ cognitive responses to the introduction of new it in a work environment. australasian journal of information systems eze, gleasure & heavin 2019, vol 23, research on health information systems factors that influence appraisal of mhealth tools 4 2.2 appraisal appraisal is defined as the cognitive evaluation and classification of an it encounter in its various aspects with respect to the individual’s well-being (beaudry & pinsonneault, 2005; lazarus & folkman, 1984). appraisal processes are mediated by the individual’s reactions and in every situation each individual appraises differently (beaudry & pinsonneault, 2005; lazarus & folkman, 1987). in cmua model, two types of appraisals of interest are identified in the study of coping process, namely, 1.) primary and 2.) secondary appraisal processes (beaudry & pinsonneault, 2005). it is argued that these two processes interact and may occur simultaneously (christophe elie-dit-cosaque & straub, 2011). the process of ‘primary appraisal’ describes where individuals evaluate the importance of an event as a consequence of their situations and interests (beaudry & pinsonneault, 2005; folkman, lazarus, gruen, & delongis, 1986). the outcome of such an evaluation is usually as either an opportunity or threat (beaudry & pinsonneault, 2005; christophe elie-dit-cosaque & straub, 2011). for example, when a change occurs in an individual’s workplace (e.g., introduction of an mhealth tool), the individual asks himself/herself, “what is at stake for me in this situation” (beaudry & pinsonneault, 2005, p. 495). the four outlined adaptation strategies in cmua model (beaudry & pinsonneault, 2005) are: 1) benefit maximising – when the new it occurrence is perceived as an ‘opportunity’ and individuals feel they have ’high level’ of control; 2) benefit satisficing – when the new it occurrence is appraised as an ’opportunity’ but with a ‘low level’ of control; 3) disturbance handling – when the new it occurrence is perceived as a ‘threat’ and individuals feel they have a ’high level’ of control, and 4) self-preservation strategies – when the new it occurrence is perceived as a ‘threat’ but with a low level of control. individuals undertake the assessment of how much control they have over the new event and the opportunities or the threats it presents them in respect to their environment, and resources provided by their management (beaudry & pinsonneault, 2005; nach & lejeune, 2010). individuals have high levels of control when they believe they have control over the event. high control users engage in ‘problem-focused coping’, for example, by expressing self-confidence in the ability to adapt themselves to the new environment or being able to manipulate features and functionality of the new (mhealth) it (beaudry & pinsonneault, 2005; christophe elie-dit-cosaque & straub, 2011). individuals have low levels of control when they believe they have insufficient control over the event, thus engaging in ‘emotion-focused’ coping in which they believe there is little or nothing they can do about this change (beaudry & pinsonneault, 2005; nach & lejeune, 2010). however, how people positively cope in the context of mhealth in developing countries remains unclear. against this background we offer a new context on how to understand the ‘positivity’ of primary appraisal, namely, primary appraisal of an mhealth tool. this study applies the coping process to understand the positive actions or activities that would influence the assimilation of an mhealth tool in the rural communities of developing countries. the next section discusses the positivity of primary appraisal. 2.3 positivity of primary appraisal positivity of primary appraisal describes an individual’s tendency to have a positive or optimistic attitude towards some new it in their work-environment. positivity describes a summative judgement of the extent to which positive (desirable) outcomes overcome negative (undesirable) outcomes (beaudry & pinsonneault, 2005; lazarus & folkman, 1984). positive and negative outcomes in a primary appraisal process are regarded as ‘opportunities’ or australasian journal of information systems eze, gleasure & heavin 2019, vol 23, research on health information systems factors that influence appraisal of mhealth tools 5 ‘threats’ respectively (claggett, 2010; christophe elie-dit-cosaque & straub, 2011). ‘opportunity’ refers to a situation that has been assessed as having ‘positive outcomes’ for the individual, invoking emotions of excitement and anticipation (bhattacherjee et al., 2017; claggett, 2010). for example, a ‘strong task-technology fit’ (goodhue & thompson, 1995) could be considered by a user as an opportunity to improve his/her performance in a workplace (beaudry & pinsonneault, 2005; fadel & brown, 2010). ‘threat’ refers to the individual’s feeling or belief that the change may negatively affect him/her. this negative feeling could be referred to a situation where a loss (e.g., loss of power or position) or harm is anticipated and could be categorised by emotions of anger, fear or anxiety (bhattacherjee et al., 2017; wisniewski et al., 2014). to promote goal-oriented work attitudes and behaviours, organisations’ actions must support those factors that foster positivity (avey, luthans, & youssef, 2010). a number of factors could impact the positivity of primary appraisal for a new mhealth tool. researchers have stressed the need to attend to social, cultural, and contextual factors of stresscoping (e.g. aldwin, 2007; chun, moos, & cronkite, 2006). social and cultural variations significantly influence the degree of positivity of primary appraisal for a stressful it (e.g. mhealth tool) occurrence (kuo, 2011; newton & mcintosh, 2010). following the transactional nature of lazarus and folkman (1984) coping theory, these factors, i.e., ‘individual’ and the ‘social’ (environment) are viewed as being in a dynamic and mutual relationship (christophe elie-dit-cosaque & pallud, 2010; christophe elie-dit-cosaque & straub, 2011). individual factors are internal behavioural or emotional factors affecting how the individual appraises a particular context or situation (beaudry & pinsonneault, 2005; bhattacherjee et al., 2017). for example, research has shown that an individual’s previous experience with technology has an impact on the way they perceive new technology in their environment (hackbarth, grover, & mun, 2003; venkatesh & morris, 2000). specifically, innovative individuals have been found to be positively predisposed to it in their work environment (lee, qu, & kim, 2007; lewis, agarwal, & sambamurthy, 2003). that is, individuals’ cognitive processes underline the basic tenants of an individual’s reaction to a stressful event (e.g. new it) (krohne, 2002; miller & kaiser, 2001). this is especially true for the following reasons: first, individuals’ ‘cognitive skills’ mediate the type of reaction they have towards an it occurrence in their workplace (beaudry & pinsonneault, 2005; fadel & brown, 2010); second, ’cognitive appraisal styles’ significantly impact on how individuals appraise and adapt to stressful situations (christophe elie-dit-cosaque & straub, 2011). thus: proposition 1 (p1). individual factors influence the positivity of the individual’s primary appraisal of an mhealth tool in developing countries. social factors are conceptualised in this paper as external factors that are outside the control of the user (or exceeding the resource of the person). social factors are situationally, contextually or environmentally dependent (beaudry & pinsonneault, 2005; mathieson, 1991). is scholars posit that ‘social factors’ influence individual’s primary appraisal (bhattacherjee et al., 2017; christophe elie-dit-cosaque, pallud, & kalika, 2011). ‘social factors’ include for example, organisational mechanisms (e.g. training and resource support), peers support (e.g., from coworker, family and friends) and environmental conditions (e.g. culture and working conditions) (johnston, warkentin, mcbride, & carter, 2016; terry, 1994). findings show that social factors may deny an individual the opportunity to use it even when the individual feels australasian journal of information systems eze, gleasure & heavin 2019, vol 23, research on health information systems factors that influence appraisal of mhealth tools 6 he/she could benefit from doing so (claggett, 2010; ragu-nathan, tarafdar, ragu-nathan, & tu, 2008), for example, network coverage (e.g. stanton et al., 2015). thus: proposition 2 (p2). social factors influence the positivity of an individual’s primary appraisal of an mhealth tool in developing countries. this allows a preliminary model to be developed representing high-level constructs that require deeper exploratory propositions (figure 1). individual factors individual threat & opportunity social factors social threat & opportunity positivity of primary appraisalp1 p2 figure 1: preliminary/sensitising research model 3 method 3.1 research methodology and site selection the study adopts an exploratory case-study approach (yin, 2013) aimed to understand the primary appraisal processes that influence the assimilation of an mhealth technology for use in new areas of developing countries. the area selected for study was the nsukka local government area in enugu state, in the south eastern region of nigeria. this area was selected for two main reasons: (i) poverty has historically been high, meaning infrastructural and cultural challenges are significant (ii) one of the researchers is from the area, meaning phenomena could be studied with a high degree of access and immersion. these qualities accommodate a case-study; an approach suitable to explore domains that maybe too complex for other research methods (e.g., surveys or experimental) (jensen & vatrapu, 2015; sarker, sarker, sahaym, & bjørn-andersen, 2012). this investigation focused on the primary appraisal of an mhealth tool that was designed and developed for a country in east africa. the mhealth tool is underpinned by the who and unicef’s integrated community case management (iccm) guidelines1. iccm are a briefer version of the integrated management of childhood illness (imci) guidelines aimed at reducing child mortality. iccm guidelines are region specific, they are used by healthcare workers in rural communities to assess, classify and treat children between the age of 2 months and five years for illnesses such as malaria, cholera and diarrhoea. the mhealth tool digitalises the existing paper-based iccm clinical guidelines decision support rule engine by defining the classification and treatment rules in extensible mark-up language (xml). 3.2 data collection and sampling a purposeful sampling approach (patton, 1990) was used to promote the selection of ‘information rich’ sources for this study (ram & khatri, 2005). following knoke (1994), interviewees/stakeholders were selected based on reputational and positional methods in the 1 for more details, please see the unicef website at https://www.unicef.org/ australasian journal of information systems eze, gleasure & heavin 2019, vol 23, research on health information systems factors that influence appraisal of mhealth tools 7 target communities in nsukka local government area. these interviewees/stakeholders occupy key roles, participate in key binding policy decisions, have the actual power to make changes, and have the important political relational power with other systems (knoke, 1994) in the enugu state healthcare delivery system (see appendix a). the researchers engaged with four key groups of stakeholders in the rural healthcare delivery system (eze et al., 2016b), specifically, parents/guardians, rural healthcare workers (rhcws), developers, and facilitators. according to this classification, the parents/guardians (pgs) are individuals that help their children to receive preventative or curative care from the healthcare system; the rhcws were those directly involved in healthcare processes, they are the direct users of the mhealth tool; the facilitators were those individuals or bodies that expedite or enable the development, implementation and delivery of mhealth processes, and the developers were those responsible for building and maintaining the mhealth system. empirical data were collected between 2nd and 23rd september 2016, and between 25th february and 25th march 2017 in both nigeria and europe. data collection exercises were conducted at the headquarters of enugu state’s civil service, ministry of health (moh), enugu state university of technology and science teaching hospital (esut), local government headquarters, health centres in the rural communities in nsukka local government area enugu state, and a university in north-west europe participating in an mhealth project. ethical approval was obtained in both the primary host institution of the researchers and a local university in nigeria involved with the research initiative. data gathering involved, in-depth interviews, participant observation, and document/record analysis, field notes and photographs from clinics in the target rural communities. all interview participants had been exposed to a new mhealth tool for accessing, classifying and treatment of children under the age of 5. interview questions therefore focused on this app, though much of the discussion ended up being at the level of mhealth more generally. interviews were conducted in igbo or english languages and recorded (with informed consent) for subsequent analysis. all recordings were transcribed verbatim into english, along with the written notes from interviews. contact time averaged 240 minutes for each group of stakeholders. initial interview questions are available in the appendixes b, c, d, and e. additional paper-based documentation included standard operation procedure (sop); facility registers, summary form, wall photographs of his related charts, graphs, and paper forms. these documents were reviewed in order to get a background information about enugu state’s his and to validate data from interviews and observations. 3.3 analysis grounded theory (gt) coding techniques (corbin & strauss, 1990, 2008, 2014; glaser & strauss, 1967) were applied to expand on preliminary theorising in this study. gt coding techniques are appropriate in this study for two reasons: first, when theorising is exploratory (gasson, 2004; glaser & strauss, 1967), as it is here, gt techniques help researchers to generate, or discover theory (glaser, 2017). second, gt techniques use a systematic set of procedures to inductively derive theory about a given phenomenon (corbin & strauss, 1990, 2008, 2014; strauss & corbin, 1998). in this way, gt techniques enable researchers to remain close to the studied environments and to develop an integrated set of theoretical concepts from their empirical data-sets (charmaz, 2011; urquhart, 2000). these techniques not only help researchers to synthesize and interpret data, but also help to show relationships in the analysis of data (charmaz, 2011, 2014). thorough description of coding is presented in appendix f. australasian journal of information systems eze, gleasure & heavin 2019, vol 23, research on health information systems factors that influence appraisal of mhealth tools 8 potential inconsistency or misinterpretation of data during the coding processes were minimised in four ways. first, the emerging themes were discussed (among researchers) and compared with insights generated that collaborated with secondary literature. second, findings were made known to stakeholders as a form of ‘venting’ or ‘member checking’ exercise, thus testing the validity and reliability of our interpretation (borman, lecompte, & goetz, 1986; miles & huberman, 1994). third, collaboration with stakeholders/participants who were actively involved as co-researchers further added to the credibility of our accounts (creswell & miller, 2000). fourth, data transcripts were revisited and recoded to set the final themes and constructs which reinforces the validity and trustworthiness of the research (roberts, priest, & traynor, 2006). that is, revisiting transcripts during the coding and constructs framing processes helped to ensure trust and rigor in the method adopted. in the next section, we present the research findings regarding the core categories resulting from the data analysis and the other major categories influencing them. 4 findings and theory building this research identified the factors that influence the primary appraisal of an mhealth tool in developing countries, the refined model is illustrated in figure 4. the figure shows the categories and concepts that emerged from the data. this process of identification of key concepts that describe the results of the primary appraisal is proposed as the initial step towards the adoption and implementation of an mhealth tool in developing countries. we make no claim that the concepts presented here are exhaustive. positivity of primary appraisal individual factors social factors perceived opportunity for improved speed and efficiency perceived threat from process uncertainty perceived opportunity for new information channels perceived opportunity for improved healthcare outcomes in rural communities perceived threat of social exclusion perceived threat from technical limitations perceived opportunity for improved reliability perceived threat from lack of reliability of infrastructure perceived threat from lack of government support perceived opportunity for simplicity of tasks figure 4 – refined research model proposition 1 was supported, as five constructs emerged relating to individual threat and opportunity appraisals: perceived opportunity for improved speed and efficiency, perceived opportunity for improved reliability, perceived opportunity for simplicity of tasks, perceived threat from technical limitations, and perceived threat from process uncertainty. proposition 2 was also australasian journal of information systems eze, gleasure & heavin 2019, vol 23, research on health information systems factors that influence appraisal of mhealth tools 9 supported, as five constructs emerged relating to social threat and opportunity appraisals: perceived opportunity for new information and communication channels; perceived opportunity for improved healthcare outcomes in rural communities; perceived threat from lack of government support; perceived threat from lack of reliability of infrastructure, and perceived threat from social exclusion. the following sections describe these emerging constructs in a refined model, as well as the themes that characterised them. 4.1 positivity of stakeholders’ primary appraisal of an it in their environment positivity of primary appraisal was characterised by two themes, illustrated in table 1. these themes collectively explain the results of the influential role of the individual and social factors on the stakeholders’ primary appraisal process – positive or negative. these are evident in the stakeholders’ accounts of the expected perceived impacts of mhealth tool on 1) the performance of rhcws (direct users of the mhealth tool), 2) the reliability of results thereof from clinics, and 3) the lifesaving outcomes for the communities in the long run. the first variable (opportunity) suggests that positive appraisal would be greatest when stakeholders perceive that they have all they need, internally and externally, to support the use of the mhealth tool. the second theme suggests negative appraisal is high when stakeholders perceive a lack of internal or external resources needed to make use of an mhealth tool in their workplace. these may take a variety of forms, e.g. resources, training and support. however, their absence means stakeholders feel likely to become overwhelmed or isolated and unable to perform tasks with the expected levels of efficiency and reliability. construct explanation themes: stakeholders believes that… positivity of primary appraisal. the extent to which a stakeholder believes the new mhealth tool will improve conditions • stakeholders are reassured by resources that are being made available to make productive use of the new mhealth tool. • stakeholders are concerned that key resources are missing and the impact of the new mhealth tool will ultimately not provide the expected value. table 1 – themes for stakeholders’ positivity of primary appraisal 4.2 emerged constructs around individual factors five constructs emerged for individual factors. the first is the perceived opportunity for improved speed and efficiency. three themes emerged around this construct, illustrated in table 2. the first theme describes the potential for improving the rate at which stakeholders could perform basic tasks, e.g. “i believe it will make our job faster” (rhcw2). the second theme concerns the potential to reduce costs associated with transferring health data to the moh. health data could easily be transferred via internet into the central database by the click of a button; a significantly simpler alternative to the current method of transporting hardcopies of health records by road, e.g. “no more paying for transportation to all the places where you are required to send the data, so you just click a button and the data goes wherever” (rhcw5). the third theme describes an expected reduction in time spent by parents/community members at the healthcare centres during diagnosis and treatment. several stakeholders saw the use of the mhealth tool as a way of quickly going through the process of diagnosis and treatment in a much shorter period, e.g. “with this mhealth tool we will not be wasting too much time at the centre, since the tool will make them work faster” (parent7). australasian journal of information systems eze, gleasure & heavin 2019, vol 23, research on health information systems factors that influence appraisal of mhealth tools 10 in stakeholders’ terms, the perceived opportunity for improved speed and efficiency was seen as an important factor that would influence positive appraisal of the mhealth tool. thus: h1a. stakeholders’ perceived opportunity for improved speed and efficiency will result in a positive primary appraisal constructs (axial codes) explanation (theoretical memos) themes: stakeholders believe that by using the mhealth tool (open codes) perceived opportunity for improved speed and efficiency stakeholders believe new practices will be less demanding of their time • there is a perceived opportunity for improved speed and efficiency for diagnosis and treatment among stakeholders • there is a perceived opportunity for improved speed and efficiency for capturing and sending (uploading) health data by stakeholders • there is a perceived opportunity for improved speed and efficiency for diagnosis and treatment time spent at rural healthcare centres among stakeholders perceived opportunity for improved reliability stakeholders believe new practice will be less demanding of rework and error correction • there is a perceived opportunity for improved reliability of diagnosis and treatment results among stakeholders • there is a perceived opportunity for improved reliability of results among stakeholders from rural healthcare centres • there is a perceived opportunity for improved reliability health data entries among stakeholders perceived opportunity for simplicity of tasks stakeholders believe new practices will be less demanding in terms of their concentration and understanding • there is a perceived opportunity for simplicity of diagnosis and treatment procedures among stakeholders • there is a perceived opportunity for simplicity of health data handling among stakeholders perceived threat from technical limitations stakeholders believe the technical resources will be insufficient for new practices • there is a perceived threat from the technical limitation of mhealth is respect to task execution among stakeholders • there is a perceived threat from technical limitation of the features regarding other diagnosis and treatments among stakeholders • there is a perceived threat from technical limitation regarding the sturdiness of the mhealth tool among stakeholders perceived threat from process uncertainty stakeholders believe the support in breaking from longstanding habits will be insufficient for new practices • there is a perceived threat from process uncertainty for the in-built tasks among stakeholders • there is a perceived threat from process uncertainty in procedural healthcare steps among stakeholders • there is a perceived threat from process uncertainty when interacting with the community’s values and norms among stakeholders table 2 themes for each of the emerging constructs relating to individual factors. australasian journal of information systems eze, gleasure & heavin 2019, vol 23, research on health information systems factors that influence appraisal of mhealth tools 11 the second construct was the perceived opportunity for improved reliability, which manifested three distinct themes. the first theme described stakeholders’ anticipation that the quality of diagnosis and treatment outcomes from rural health centres would improve e.g. “mhealth tool will help rhcws in making better decisions resulting in improved quality of diagnosis and treatment” (parent5). the second theme was stakeholders’ perception that results from the new system would be more reliable, e.g. “it might bring changes, because right now from the way i am seeing things, people will tend to trust [have faith on] devices and people will trust being diagnosed with devices” (parent2). the third theme describes rhcws’ anticipation that new systems could implement error-proof data entry forms for rural healthcare centres, e.g. “i know that using mhealth tool will help in reducing errors in our treatment” (rhcw6). stakeholders were acutely aware that records are not always accurate; an issue that creates frequent and unwelcome uncertainty during the diagnosis and treatment process. improved reliability was seen by stakeholders as a significant factor that would influence positive appraisal for mhealth tools. thus: h1b. stakeholders’ perceived opportunity for improved reliability will result in a positive primary appraisal the third construct was the perceived opportunity for simplicity of tasks. two themes emerged within this construct. the first theme describes that most stakeholders believe an mhealth tool would make rhcws tasks effortless compared with the existing paper format, e.g. “the app is easy to locate on the phone; you can easily use it. it is good, the guide is there for you” (rhcw7). for some, it referred to the easy understanding of their tasks when using the mhealth tool, e.g. “it is easy for me to manipulate this tool, the app as i can say is very comfortable at our own level” (rhcw3). the second theme describes the simplification of data management for stakeholders, e.g. “data recording is not needed. as you progress through the app, data is being saved and stored for you at the same time” (rhcw2). this appealed to stakeholders, for whom data recording was often a cumbersome secondary activity distracting them from core treatment and diagnosis responsibilities. perceived opportunity for simplicity of tasks was also identified. stakeholders’ were enthusiastic about the possibility of an mhealth tool simplifying and improving their tasks. this was identified as a significant factor that would influence a positive primary appraisal. thus: h1c. stakeholders’ perceived opportunity for simplicity of tasks will result in a positive primary appraisal the fourth construct was the perceived threat from technical limitations, which manifested three themes. the first theme was the concern around the technical limitation features of the mhealth tool in performing the envisaged tasks. stakeholders raised concerns regarding how well the mhealth tool performs the diagnosis and treatment tasks e.g. “if they have overwhelming failure in the app then that can put a lot of people off” (developer6). this implies that a first-time user could be influenced to reject an application that is not performing as one anticipated. the second theme expressed concerns around the limited technical features of the mhealth tool regarding treatments e.g. “i feel that the app development should go further than the stage it is at now, for example, the issue of treatment is still being done manually” (facilitator1). for others, it goes much deeper, they want the mhealth tool to be used in treating adults, e.g. “i want the tool to be developed to include adults, like pregnant australasian journal of information systems eze, gleasure & heavin 2019, vol 23, research on health information systems factors that influence appraisal of mhealth tools 12 women” (rhcw4). that is, if mhealth tool could not be used to do these other activities that he/she would have wished it could do, then, he or she might negatively appraise it. the third theme expressed concerns around the technical limitations regarding the ruggedness features of the mhealth tool e.g. “the smartphone looks fragile and might break when it falls, so, one would like to use a tool that could break incurring damages from user” (facilitator2). technical limitations were highlighted by stakeholders as one of the significant factors that would influence a negative appraisal for the mhealth tool. thus: h1d. stakeholders’ perceived threat from technical limitations will result in a negative primary appraisal the fifth construct was the perceived threat from process uncertainty, which exhibited three themes. the first theme explained the anxiety felt by stakeholders with regard to using computers, e.g. “i have not used a computer before, i do not know whether i can use it” (rhcw5). to some stakeholders the prospect of using technology evoked a deep emotion, e.g. “for some of us it will be hard you know, i feel too old now to start learning how to use computer, well we will see" (facilitator6). the second theme describes the concern by stakeholders that mhealth introduction might alter current work practices, e.g. “people find it very difficult to change from their comfort zones, they feel uncomfortable to change to an unknown way of doing things” (facilitator3). the third theme describes the effect of norms and cultural values that may negatively influence stakeholders’ behaviour towards the mhealth tool, e.g. “people in rural communities liken technology as a sign that we are nearing the end of the world [end-time] due to their beliefs” (facilitator1). process uncertainty was also identified by stakeholders as a convincing factor that would influence a negative appraisal for the mhealth tool. thus: h1e. stakeholder’s perceived threat from process uncertainty will result in a negative primary appraisal. 4.3 emerged constructs around the social factors five constructs emerged around the social factors, themes for which are illustrated in table 3. the first construct was the perceived opportunity for new information and communication channels, which exhibited two themes. the first theme describes the new communication channels between stakeholders, e.g. “it will create communication between rural healthcare officers and patients regarding health-related matters” (rhcw6). for others, new information channels created more potential for supervision, e.g. “it will help open up conversation between rural healthcare officers and their superiors about their tasks” (developer5). the second theme describes a new source of health information through the internet, e.g. “with the phone, people would be looking for diagnosis or treatment about ailments in the net” (facilitator1). stakeholders identified perceived opportunity for new information and communication channels as a compelling factor that would influence positive primary appraisal of an mhealth tool. thus: h2a. stakeholders’ perceived opportunity for new information and communication channels will result in a positive primary appraisal the second construct was the perceived opportunity for improved healthcare outcomes in rural communities, which displayed two themes. the first theme highlighted the impact such a healthcare delivery tool would have on rural community members, e.g. “it could create a happier community since this could mean that less children would be dying from childhood diseases” (facilitator4). the general impression among stakeholders is that using mhealth tool australasian journal of information systems eze, gleasure & heavin 2019, vol 23, research on health information systems factors that influence appraisal of mhealth tools 13 in rural communities would encourage members to send their children for diagnosis and treatment, e.g., “once they [parents] know that we are using phone [mhealth tool], they would rush [avail of such opportunity] for it” (rhcw3). there is also the belief that the work ethic of stakeholders would improve tremendously as a result of mhealth introduction in enugu state healthcare system. for example, stakeholders would be motivated to work in rural healthcare centres, e.g. “it would improve my confidence and i would be respected in my community for using mhealth tool” (rhcw1). the second theme concerns the ‘reach’ capacity of an mhealth tool, e.g. “using mhealth tool by rhcws is the best way to bring treatment to the rural communities” (developer5). that is, the mhealth tool would help in extending healthcare services to rural community members into the healthcare systems. perceived opportunity for improved diagnosis and treatment was seen by stakeholders as a substantial factor that would influence primary positive appraisal of an mhealth tool. that is, positive response to the external impact on the healthcare services that is quite different from internal (self) impact. thus: h2b. stakeholders’ perceived opportunity for improved diagnosis and treatments will result in a positive primary appraisal the third and the most frequently discussed construct described the perceived threat from lack of government support. this construct is considered to be the most persuasive, as it emphasises the need for government approval and support for the mhealth implementation process. the nature of the support expected from government is varied. government support significantly plays a central role in moderating the effect of negative appraisals. the first theme focused on the origin of the mhealth tool e.g. “you have to convince these policy makers seriously [persuasively] before they can buy-into it, we need to convince them that this [mhealth tool] belongs to them” (facilitator1). this suggests that for successful implementation of mhealth tool in enugu state, policy-makers would need to be co-opted as partners in its introduction, for example, policy-makers would need to enact laws and regulations to afford the provision of healthcare via mhealth a legal status in the healthcare delivery system. the second theme refers to the concern around the consistency in policy implementation by successive governments, e.g. “one of the things i have seen is, you know somersaults, inconsistency in implementing policies and things they set out to do” (developer5). implying there is a lack of consistency on the part of governments regarding policy implementation. often, a change of government means policies are reversed or tweaked in such a way as to suit the new government agenda. the third theme centred on the concern by stakeholders for the provision of adequate financial resources. stakeholders expect the government to provide financial support and the needed incentives to stimulate the use of an mhealth tool, e.g. “mhealth implementation could be jeopardised by lack of funds, and its sustainability depends on the availability of funds as well” (facilitator4). the fourth theme describes the concern around the provision of training for users. for some, lack of training could mean not doing their tasks as expected e.g. “it is a new app that are going to have to be embedded within their daily work practices, and for this to work, they have to be trained properly on how to use it” (developer2). for others, it has much deeper implications, e.g., “without good training, it may have a consequential effect on the continued use of mhealth for a long time after its introduction, the tool could be abandoned” (facilitator5). the fifth theme is the concern expressed by stakeholders around the need for supervision during mhealth tool use. this theme stressed the importance of supervising users during use to make sure that the mhealth tool is used as australasian journal of information systems eze, gleasure & heavin 2019, vol 23, research on health information systems factors that influence appraisal of mhealth tools 14 anticipated, e.g. “they will also need a lot of supervision from their superiors to make sure they are doing the correct thing” (facilitaor6). constructs explanation themes: stakeholders believe that by using the mhealth tool perceived opportunity for new information and communication channels stakeholders believe new practices will be less demanding of existing social connections • there is a perceived opportunity for new information and communication channels among stakeholders – between rhcws and parents, with rhcws, and rhcws and their supervisors or superiors • there is a perceived opportunity for new information and communication channels through internet for parents and community members at large perceived opportunity for improved healthcare outcomes in rural communities stakeholders believe new practices will be less demanding of subsequent ancillary or emergency care • there is a perceived opportunity for improved healthcare outcomes as mhealth tool introduction would lead to less under-five deaths • there is a perceived opportunity for improved healthcare outcomes in rural communities since healthcare services would reach the unserved perceived threat from lack of government support stakeholders believe the government support will be insufficient for new practices • there is a perceived threat from lack of government support with regards to participation or partnering • there is a perceived threat from lack of government support by the creation of enabling policies for mhealth tool implementation and upscaling • there is a perceived threat from lack of government support for the provision of required or necessary funding for implementation and sustainability • there is a perceived threat from lack of government support to provide training for end-users • there is a perceived threat from the lack of government support in the areas of supervision and monitoring perceived threat from lack of reliability of infrastructure stakeholders believe the infrastructure will be insufficient for new practices • there is a perceived threat from lack of reliability of infrastructure with regards to internet availability • there is a perceived threat from lack of reliability of infrastructure with regards to steady supply of power (electricity) perceived threat from social exclusion stakeholders believe the consideration of personal roles will be insufficient for new practices • there is a perceived threat from social exclusion for doctors who might feel that their primary job is being taken away by the introduction of mhealth tool • there is a perceived threat from social exclusion for rhcws who feel that it might mean the loss of their job table 3 themes for each of the emerging constructs relating to social factors. perceived threat from lack of government support was viewed by stakeholders as the most significant factor that would influence the primary negative appraisal for mhealth tool. thus: h2c. stakeholders’ perceived threat from lack of government support will result in a negative primary appraisal the fourth construct was the perceived threat from lack of reliability of infrastructure. stakeholders raised concerns around the impact of unreliable infrastructure. the first theme was the concern australasian journal of information systems eze, gleasure & heavin 2019, vol 23, research on health information systems factors that influence appraisal of mhealth tools 15 around the non-availability of internet which could hamper the use of the mhealth tool, e.g. “the external networks that could impact on the health care delivery as regards mhealth is internet availability” (rhcw2). suggesting that in remote areas, the external input, such as poor internet connection could make mhealth use unworkable, being a characteristic feature, it could make people not be enthusiastic about mhealth introduction. the second theme centred on the need for a constant power supply e.g. “the epileptic nature of electricity supply…where people can’t even charge their phones or pcs because they don’t have light [power] is going to be a big problem for mhealth tool” (parent2). stakeholders are referring to the unreliable electricity supplies across nigeria, which to them could jeopardise the opportunities afforded by mhealth. reliability of infrastructure was highlighted as one of the important factors that would influence the primary negative appraisal for mhealth tool. thus: h2d. stakeholders’ perceived threat from lack of reliability of infrastructure will result in a negative primary appraisal the fifth construct was the perceived threat from social exclusion. stakeholders are concerned that mhealth introduction into enugu state healthcare system could bring along with it such issues as social exclusion and class distinction. the first theme made reference to the anxiety that an mhealth tool will reduce their job security, e.g. “i am worried that it might make some of us redundant in our work place” (rhcw3). this perception stems from the deduction that using an mhealth tool could mean executing more tasks than one or more rhcws at any given time, thus rendering some of them redundant. these redundant workers could be sacked or reassigned. the second theme focused on the job status of some stakeholders (e.g., doctors). for some, implementing mhealth technologies might mean losing the professional autonomy they have over diagnosis and treatment, e.g. “some doctors may not accept it for given away their primary duty” (developer6). these concerned stakeholders argue that diagnosis and treatments are at the core of their profession, so, why give it away to other stakeholders by way of mhealth technologies. perceived threat from social exclusion was identified as one of the factors that would influence a negative primary appraisal. thus: h2e. stakeholders’ perceived threat from social exclusion will result in a negative primary appraisal 5 discussion this paper explores the factors that influence stakeholders’ primary appraisal of mhealth technologies in rural contexts. the analysis in the previous section presents several important findings. first, opportunity was found to play an important role in explaining the internal and external factors that positively (positivity) influence stakeholders’ primary appraisal via five constructs. the perceived opportunity for improved speed and efficiency construct describes the stakeholders’ opinions that captured the practical benefits that mhealth would have on healthcare delivery (e.g. gurman, rubin, & roess, 2012; paina & peters, 2011). such internal perception for an opportunity for improved speed and efficiency may lead to stakeholders’ positive appraisal of an it tool and may ultimately influence intention to use (beaudry, 2009; claggett, 2010). stakeholders’ beliefs that using mhealth tool would result in improved quality data, diagnosis and treatment emanated from the perceived opportunity for improved australasian journal of information systems eze, gleasure & heavin 2019, vol 23, research on health information systems factors that influence appraisal of mhealth tools 16 reliability construct. these perceptions are consistent with existing literature (e.g. akter, d'ambra, & ray, 2010; chib et al., 2015). self-efficacy which emanates from the stakeholders’ beliefs about their abilities emerged within the perceived opportunity for simplicity of tasks construct. self-efficacy perception reflects an individual’s internal beliefs in his/her own capabilities to perform a sequence of action to meet a given social demand (healthcare delivery) (ajzen, 2002; barbeite & weiss, 2004). self-efficacy has been found in literature to influence positive appraisal of an it tool and intention to engage (e.g. agarwal, sambamurthy, & stair, 2000; ajzen, 2002). this specifically holds true for mhealth tool in research conducted by xue et al. (2015) in ethiopia. they posit that perceived behavioural control which could be aligned with self-efficacy can make individuals more motivated to perform a target behaviour (goal) (xue et al., (2015). the perceived opportunity for new information and communication channels construct arose out of the stakeholders’ perceptions around these other ‘things’ that one could use the mhealth tool for. this positive influencing factor emerged from interacting with the material agency of mhealth tool (external material). one such activity is third party information access via the internet. parents could access health information available in the internet, but this space is unregulated and may jeopardise the structured healthcare delivery processes meant for rural communities (moreland, french, & cumming, 2016; murray et al., 2003). the other is the new communication channels between stakeholders created by the availability of these mobile tools. patients could reach rhcws through this tool, it facilitates communication amongst rhcws and between rhcws and their supervisors (e.g. higgs et al., 2014; leon, schneider, & daviaud, 2012). the perceived opportunity for improved healthcare outcomes in rural communities construct emerged from the stakeholders’ belief that healthcare services would reach the unserved in rural areas. that is, the positive external impacts of using an mhealth tool. these findings resonate with previous work demonstrating how perceived improvement in health outcomes could influence stakeholders’ decision-making process towards positively appraising an mhealth tool (aranda-jan, mohutsiwa-dibe, & loukanova, 2014; miah, hasan, hasan, & gammack, 2017). second, threat was found to play a significant role in delineating the internal and external factors that negatively influence stakeholders’ primary appraisal of an mhealth tool through five constructs. the perceived threat from technical limitation construct emerged for the stakeholders’ internal concerns around technical functionalities and limited capabilities (j.-n. lee, huynh, & hirschheim, 2008; lim, benbasat, & ward, 2000) of the mhealth tool (chang et al., 2013). first impression has been shown to influence the decision-making process to either positively (opportunity) or negatively (threat) appraise an it tool (kim, shin, & lee, 2009; nicolaou & mcknight, 2006). first impression is considered a significant factor in an appraisal process, since one may not get a second chance to test-out a particular is tool (frost, pike, & kenyo, 2008; reinecke et al., 2013). limited capabilities were found to influence stakeholders’ decisions to either positively or negatively appraise an mhealth tool. this finding resonates with evidence in literature regarding stakeholders’ high expectation of an mhealth tool (chang et al., 2013). the perceived threat from process uncertainty construct emerged for stakeholders’ internal perception of concerns around the fear of computers, pre-existing practices, and counter interactions with culture and norms. computer anxiety arises out of the fear of computers when using a computer or fearing the possibility of using one in the future (barbeite & weiss, 2004; shu, tu, & wang, 2011). findings in literature echo previous works demonstrating the influence of computer anxiety on primary appraisal that impacts intention (e.g. fagan, neill, australasian journal of information systems eze, gleasure & heavin 2019, vol 23, research on health information systems factors that influence appraisal of mhealth tools 17 & wooldridge, 2004; venkatesh, 2000). habit as one of the perceived threats has been identified in previous works (maier, laumer, weinert, & weitzel, 2015; recker, 2014). habit could be defined as an acquired or cultured behavioural sequences of acts to achieve a specific goal (de guinea & markus, 2009; polites, 2005). in is research pre-existing practice or habit is a critical predictor of technology use (polites, 2005; venkatesh, thong, & xu, 2012). cultural and social norms was found to influence stakeholders’ decisions (e.g. carter & weerakkody, 2008; srite & karahanna, 2006). in the context of this study, culture could be defined as communicable knowledge produced across humanity’s social life (im, hong, & kang, 2011; jahoda, 2012). the perceived effect of culture and social norms have important conceptual similarity with habit. in that vein, one could infer that since technology is generally used in cultural contexts, culture can be said to play a significant role in technology appraisal (im et al., 2011). the perceived threat from lack of government support construct emerged for stakeholders’ concerns around government support for mhealth intervention. this concern resonates with previous work showing the influence of government support on primary appraisal of an it. for example, lack of support from governments (external to the user) in areas of promulgation of enabling policy, high level strategic planning and financial support have shown to inhibit implementation of mhealth in developing countries (e.g. leon et al., 2012; mechael, 2009). funding shortages have been found to discourage users to continue with mhealth services (chib, 2010; chib, lwin, ang, lin, & santoso, 2008). absence of this support may lead to negative (threat) primary appraisal of an mhealth tool (aranda-jan et al., 2014; leon et al., 2012). support in areas of training and supervision has been shown to lead to positive or negative appraisals. evidence of the need for these types of support is found in existing literature (e.g. leon et al., 2012; modi et al., 2015). the perceived threat from lack of reliability of infrastructure construct reflects the reality that the non-availability of power supply and internet access could pose negative influence on the successful implementation of mhealth in developing countries (akter et al., 2010; sanner, manda, & nielsen, 2014). threat appraisal of the reliability of infrastructure which is an external factor is particularly significant for stakeholders in rural communities where power outages and network coverages are more pronounced. the perceived threat from social exclusion construct manifested for stakeholders’ concerns around the internal fear for technology and job loss as a result of introducing an mhealth tool (chang et al., 2013; maeder, 2014). some stakeholders expressed concern for the security of their jobs as the introduction of an mhealth tool might mean fewer workers would be required (chang et al., 2013; xue et al., 2015). other stakeholders (e.g., doctors) expressed concern over the possible change to the traditional way treatments are done (desai, khan, mistry, & gaikawad, 2016; malvey & slovensky, 2014). for others, it could mean the loss of autonomy where a skilled professional is acting exclusively on the guidance of a specialist located at a remote area (morrison, shrestha, hayes, & zimmerman, 2013). 6 summary and conclusion the study developed a novel research model that describes how primary appraisal influences the introduction of an mhealth tool in a new context. in the model, the emergent constructs from both the individual and social factors combine to tell a story of how primary appraisal could positively or negatively affect mhealth introduction in rural communities of developing australasian journal of information systems eze, gleasure & heavin 2019, vol 23, research on health information systems factors that influence appraisal of mhealth tools 18 countries. the model presents a set of individual and social factors that governments, funding bodies and non-governmental organisations should consider before embarking on the introduction of an mhealth tool in rural communities of developing countries. at the individual level, the perceived opportunities for improved speed and efficiency, reliability of results, and simplification of tasks by the tool were seen on as possible motivating factors that would influence stakeholders to positively appraise a new mhealth tool. the perceived threats from the technical limitations of the tool’s functionalities, and process uncertainty were seen by stakeholders to negatively impact the introduction of an mhealth tool. at the social level, the perceived opportunities for new information and communication channels and improved availability of healthcare services would positively impact on the primary appraisal. yet, the perceived threats from lack of government support, lack of reliable infrastructure, and the resultant social exclusion associated with the introduction of an mhealth tool were seen by stakeholders to negatively affect primary appraisal. this research has several important contributions to research and practice. first, the model offers new perspectives for researchers into the primary appraisal processes and dynamics involved in the introduction of mhealth tools for new areas of developing countries. second, the model offers a new way to understand how users arrive at their primary appraisal behaviour and thus can provide a useful framework through which we can incorporate adoption and resistance studies (eze, gleasure, & heavin, 2016a). this contribution could be considered significant in modelling the factors that influence primary appraisal. third, it is envisaged that by understanding the process of primary appraisal, either as an opportunity or a threat, practitioners and organisations will support positive appraisal and minimise the occurrence of negative ones when introducing mhealth tools. fourth, this research contributes to the growing evidence that the cognitive processes can be broken down into internal and external components (e.g. aizawa, 2017; braver, 2012; paradis, 2011; wedgwood, 2006). the findings of this study were not without some limitations. first, the study made use of a singlecase design, and thus make no claims of statistical generalisability (yin, 2013). second, the study was exploratory in nature. we therefore recommend a longitudinal study that could reveal other contributing factors that may arise due to re-appraisal processes, as users may reevaluate and adjust their prior primary and/or secondary appraisals (beaudry & pinsonneault, 2005; bhattacherjee et al., 2017). third, the research focused on technology-enabled guidelinedriven treatment of the mhealth delivery service. other forms of mhealth initiatives exist, e.g. those focused on data gathering (chang et al., 2011; medhanyie et al., 2015) or those focused on remote diagnosis and treatment (hufnagel, 2012; knoble & bhusal, 2015). we, therefore, call for similar research on other delivery approaches in order to compare findings. references abraham, c., boudreau, m.-c., junglas, i., & watson, r. (2013). enriching our theoretical repertoire: the role of evolutionary psychology in technology acceptance. european journal of information systems, 22(1), 56-75. agarwal, r., sambamurthy, v., & stair, r. m. (2000). the evolving relationship between general and specific computer self-efficacy—an empirical assessment. information systems research, 11(4), 418-430. aizawa, k. (2017). cognition and behavior. synthese, 194(11), 4269-4288. australasian journal of information systems eze, gleasure & heavin 2019, vol 23, research on health information systems factors that influence appraisal of mhealth tools 19 ajzen, i. (2002). perceived behavioral control, self‐efficacy, locus of control, and the theory of planned behavior. journal of applied social psychology. 32(4), 665-683. aker, j. c., & mbiti, i. m. (2010). mobile phones and economic development in africa. center for global development working paper(211). akter, s., d'ambra, j., & ray, p. (2010). user perceived service quality of m-health services in developing countries. paper presented at the 18th european conference on information systems (pp. 1-12), pretoria, south africa: university of pretoria. alam, m., khanam, t., & khan, r. (2010). assessing the scope for use of mobile based solution to improve maternal and child health in bangladesh: a case study. paper presented at the international conference on information and communication technologies and development, london, uk. aldwin, c. m. (2007). stress, coping, and development: an integrative approach. new york, ny: guildford. andriopoulos, c., & lowe, a. (2000). enhancing organisational creativity: the process of perpetual challenging. management decision, 38(10), 734-742. aranda-jan, c. b., mohutsiwa-dibe, n., & loukanova, s. (2014). systematic review on what works, what does not work and why of implementation of mobile health (mhealth) projects in africa. bmc public pealth, 14(1), 188. avey, j. b., luthans, f., & youssef, c. m. (2010). the additive value of positive psychological capital in predicting work attitudes and behaviors. journal of management, 36(2), 430-452. barbeite, f. g., & weiss, e. m. (2004). computer self-efficacy and anxiety scales for an internet sample: testing measurement equivalence of existing measures and development of new scales. computers in human behavior, 20(1), 1-15. beaudry, a. (2009). coping with information technology handbook of research on contemporary theoretical models in information systems (pp. 516-528): igi global. beaudry, a., & pinsonneault, a. (2005). understanding user responses to information technology: a coping model of user adaptation. mis quarterly. 493-524. bhattacherjee, a., davis, c. j., connolly, a. j., & hikmet, n. (2017). user response to mandatory it use: a coping theory perspective. european journal of information systems, 1-21. bohm, a. (2004). 5.13 theoreticai coding: text anaiysis in grounded theory. a companion to qualitative research. 270. böhm, a. (2004). theoretical coding: text analysis in grounded theory. dans u. flick, e. von kardorff, & i. steinke (éds), a companion to qualitative research (pp. 270-275): london: sage. borman, k. m., lecompte, m. d., & goetz, j. p. (1986). ethnographic and qualitative research design and why it doesn't work. american behavioral scientist. 30(1), 42-57. braver, t. s. (2012). the variable nature of cognitive control: a dual mechanisms framework. trends in cognitive sciences. 16(2), 106-113. australasian journal of information systems eze, gleasure & heavin 2019, vol 23, research on health information systems factors that influence appraisal of mhealth tools 20 carter, l., & weerakkody, v. (2008). e-government adoption: a cultural comparison. information systems frontiers, 10(4), 473-482. chang, l. w., kagaayi, j., arem, h., nakigozi, g., ssempijja, v., serwadda, d., . . . reynolds, s. j. (2011). impact of a mhealth intervention for peer health workers on aids care in rural uganda: a mixed methods evaluation of a cluster-randomized trial. aids and behavior, 15(8), 1776-1784. chang, l. w., njie-carr, v., kalenge, s., kelly, j. f., bollinger, r. c., & alamo-talisuna, s. (2013). perceptions and acceptability of mhealth interventions for improving patient care at a community-based hiv/aids clinic in uganda: a mixed methods study. aids care. 25(7), 874-880. charmaz, k. (2006). constructing grounded theory: a practical guide through qualitative research. sagepublications ltd, london. charmaz, k. (2011). grounded theory methods in social justice research. the sage handbook of qualitative research. 4(1), 359-380. charmaz, k. (2014). constructing grounded theory: 416 pages | sage publications ltd chib, a. (2010). the aceh besar midwives with mobile phones project: design and evaluation perspectives using the information and communication technologies for healthcare development model. journal of computer‐mediated communication, 15(3), 500-525. chib, a. (2013). the promise and peril of mhealth in developing countries. mobile media & communication, 1(1), 69-75. chib, a., & chen, v. h.-h. (2011). midwives with mobiles: a dialectical perspective on gender arising from technology introduction in rural indonesia. new media & society, 13(3), 486501. chib, a., lwin, m. o., ang, j., lin, h., & santoso, f. (2008). midwives and mobiles: using icts to improve healthcare in aceh besar, indonesia. asian journal of communication, 18(4), 348-364. chib, a., van velthoven, m. h., & car, j. (2015). mhealth adoption in low-resource environments: a review of the use of mobile healthcare in developing countries. journal of health communication. 20(1), 4-34. chun, c.-a., moos, r. h., & cronkite, r. c. (2006). culture: a fundamental context for the stress and coping paradigm. handbook of multicultural perspectives on stress and coping. handbook of multicultural perspectives on stress and coping, 29-53. claggett, j. l. (2010). understanding primary appraisal in user adoption: an exploratory case study of a telehealth project. paper presented at the proceedings of the southern association for information systems conference. connolly, a. j., & bhattacherjee, a. (2011). coping with the dynamic process of technostress, appraisal and adaptation. paper presented at american conference on information systems. corbin, j., & strauss, a. (1990). grounded theory research: procedures, canons and evaluative criteria. zeitschrift für soziologie, 19(6), 418-427. australasian journal of information systems eze, gleasure & heavin 2019, vol 23, research on health information systems factors that influence appraisal of mhealth tools 21 corbin, j., & strauss, a. (2008). basics of qualitative research: techniques and procedures for developing grounded theory. (3rd ed.). thousand oaks, ca: sage. corbin, j., & strauss, a. (2014). basics of qualitative research: techniques and procedures for developing grounded theory: sage publications. creswell, j. w., & miller, d. l. (2000). determining validity in qualitative inquiry. theory into practice. 39(3), 124-130. datta, p., byrd, t. a., okoli, c., & mbarika, v. w. (2005). the neglected continent of is research: a research agenda for sub-saharan africa. journal of the association for information systems, 6(5), 6. de guinea, a. o., & markus, m. l. (2009). why break the habit of a lifetime? rethinking the roles of intention, habit, and emotion in continuing information technology use. mis quarterly. 433-444. derenzi, b., findlater, l., payne, j., birnbaum, b., mangilima, j., parikh, t., . . . lesh, n. (2012). improving community health worker performance through automated sms. paper presented at the proceedings of the fifth international conference on information and communication technologies and development. desai, p., khan, m. s., mistry, c., & gaikawad, v. (2016). study of concerns related to implementation of m-health in india: understanding urban doctor’s perspective. international journal of computer applications (0975 – 8887) international conference on advances in information technology and management icaim 2016. donner, j., & mechael, p. (2012). mhealth in practice: mobile technology for health promotion in the developing world: a&c black. elie-dit-cosaque, c., & pallud, j. (2010). user adaptation and is success: an empirical investigation among french workers. paper presented at the international conference on information systems. elie-dit-cosaque, c., pallud, j., & kalika, m. (2011). the influence of individual, contextual, and social factors on perceived behavioral control of information technology: a field theory approach. journal of management information systems. 28(3), 201-234. elie-dit-cosaque, c., & straub, d. w. (2011). opening the black box of system usage: user adaptation to disruptive it. european journal of information systems, 20(5), 589-607. eze, e., gleasure, r., & heavin, c. (2016a). how can mhealth applications that are developed in one area of the developing world be adapted for use in others? journal of decision systems. 25(sup1), 536-541. eze, e., gleasure, r., & heavin, c. (2016b). reviewing mhealth in developing countries: a stakeholder perspective. procedia computer science, 100, 1024-1032. eze, e., gleasure, r., & heavin, c. (2018). mobile health solutions in developing countries: a stakeholder perspective. health systems, 1-23. fadel, k. j., & brown, s. a. (2010). information systems appraisal and coping: the role of user perceptions. communications of the association for information systems. communications of the association for information systems, 26(1), 6. australasian journal of information systems eze, gleasure & heavin 2019, vol 23, research on health information systems factors that influence appraisal of mhealth tools 22 fagan, m. h., neill, s., & wooldridge, b. r. (2004). an empirical investigation into the relationship between computer self-efficacy, anxiety, experience, support and usage. journal of computer information systems, 44(2), 95-104. folkman, s., lazarus, r. s., gruen, r. j., & delongis, a. (1986). appraisal, coping, health status, and psychological symptoms. journal of personality and social psychology. 50(3), 571. free, c., phillips, g., watson, l., galli, l., felix, l., edwards, p., . . . haines, a. (2013). the effectiveness of mobile-health technologies to improve health care service delivery processes: a systematic review and meta-analysis. plos medicine. 10(1), e1001363. frost, r., pike, j., & kenyo, l. (2008). generating student interest in the information systems major: a strategic framework for the introductory course. issues in information systems, 9(1), 188-195. furuholt, b., & matotay, e. (2011). the developmental contribution from mobile phones across the agricultural value chain in rural africa. the electronic journal of information systems in developing countries, 48. gasson, s. (2004). rigor in grounded theory research: an interpretive perspective on generating theory from qualitative field studies. the handbook of information systems research. 4, 79-102. glaser, b. (1978). theoretical sensitivity: advances in the methodology of grounded theory: sociology pr. glaser, b. (2017). discovery of grounded theory: strategies for qualitative research: routledge. glaser, b., & strauss, a. (1967). grounded theory: the discovery of grounded theory. sociology the journal of the british sociological association, 12, 27-49. gleasure, r. (2015). resistance to crowdfunding among entrepreneurs: an impression management perspective. the journal of strategic information systems, 24(4), 219-233. goodhue, d. l., & thompson, r. l. (1995). task-technology fit and individual performance. mis quarterly. 213-236. gurman, t. a., rubin, s. e., & roess, a. a. (2012). effectiveness of mhealth behavior change communication interventions in developing countries: a systematic review of the literature. journal of health communication. 17(sup1), 82-104. hackbarth, g., grover, v., & mun, y. y. (2003). computer playfulness and anxiety: positive and negative mediators of the system experience effect on perceived ease of use. information & management. 40(3), 221-232. hacking, d., haricharan, h. j., brittain, k., lau, y. k., cassidy, t., & heap, m. (2016). hypertension health promotion via text messaging at a community health center in south africa: a mixed methods study. jmir mhealth and uhealth, 4(1). hareli, s., & hess, u. (2010). what emotional reactions can tell us about the nature of others: an appraisal perspective on person perception. cognition and emotion, 24(1), 128-140. heeks, r. (2006). health information systems: failure, success and improvisation. international journal of medical informatics. 75(2), 125-137. australasian journal of information systems eze, gleasure & heavin 2019, vol 23, research on health information systems factors that influence appraisal of mhealth tools 23 higgs, e. s., goldberg, a. b., labrique, a. b., cook, s. h., schmid, c., cole, c. f., & obregón, r. a. (2014). understanding the role of mhealth and other media interventions for behavior change to enhance child survival and development in low-and middle-income countries: an evidence review. journal of health communication. 19(sup1), 164-189. hufnagel, h. (2012). compendium 2012: innovative health technologies for low resource settings. paper presented at the 13th world congress on public health (april 23-27, 2012). im, i., hong, s., & kang, m. s. (2011). an international comparison of technology adoption: testing the utaut model. information & management. information & management, 48(1), 1-8. jahoda, g. (2012). critical reflections on some recent definitions of “culture”. culture & psychology, 18(3), 289-303. jensen, t., & vatrapu, r. (2015). ships & roses: a revelatory case study of affordances in international trade. paper presented at the european conference on information systems. johnston, a. c., warkentin, m., mcbride, m., & carter, l. (2016). dispositional and situational factors: influences on information security policy violations. european journal of information systems, 25(3), 231-251. kabuya, c., wright, g., odama, a., & o'mahoney, d. (2014). routine data for disease surveillance in the undeveloped region of the or tambo district of the eastern cape province. stud health technol inform, 197, 103-107. kahn, j. g., yang, j. s., & kahn, j. s. (2010). ‘mobile’health needs and opportunities in developing countries. health affairs, 29(2), 252-258. kay, m., santos, j., & takane, m. (2011). mhealth: new horizons for health through mobile technologies. world health organization, 64(7), 66-71. kim, g., shin, b., & lee, h. g. (2009). understanding dynamics between initial trust and usage intentions of mobile banking. information systems journal, 19(3), 283-311. knoble, s. j., & bhusal, m. r. (2015). electronic diagnostic algorithms to assist mid-level health care workers in nepal: a mixed-method exploratory study. international journal of medical informatics. 334-340. knoke, d. (1994). networks of elite structure and decision making. sage focus editions, 171, 274-274. krohne, h. w. (2002). stress and coping theories. international encyclopedia of the social behavioral sceinces, 22, 15163-15170. kuo, b. c. (2011). culture’s consequences on coping: theories, evidences, and dimensionalities. journal of cross-cultural psychology, 42(6), 1084-1100. lazarus, r. s., & folkman, s. (1984). stress, appraisal, and coping: springer publishing company. lazarus, r. s., & folkman, s. (1987). transactional theory and research on emotions and coping. european journal of personality. 1(3), 141-169. lee, h. y., qu, h., & kim, y. s. (2007). a study of the impact of personal innovativeness on online travel shopping behavior—a case study of korean travelers. tourism management, 28(3), 886-897. australasian journal of information systems eze, gleasure & heavin 2019, vol 23, research on health information systems factors that influence appraisal of mhealth tools 24 lee, j.-n., huynh, m. q., & hirschheim, r. (2008). an integrative model of trust on it outsourcing: examining a bilateral perspective. information systems frontiers, 10(2), 145163. leon, n., schneider, h., & daviaud, e. (2012). applying a framework for assessing the health system challenges to scaling up mhealth in south africa. bmc medical informatics and decision making. 12(1), 123. lewis, w., agarwal, r., & sambamurthy, v. (2003). sources of influence on beliefs about information technology use: an empirical study of knowledge workers. mis quarterly. 657-678. liang, h., & xue, y. (2009). avoidance of information technology threats: a theoretical perspective. mis quarterly 71-90. lim, k. h., benbasat, i., & ward, l. m. (2000). the role of multimedia in changing first impression bias. information systems research, 11(2), 115-136. lowe, a. (1996). an explanation of grounded theory: [helsinki]: swedish school of economics and business administration. lyon, b. l. (2000). stress, coping, and health. handbook of stress, coping and health: implications for nursing research, theory, and practice. 3-23. maeder, a. (2014). evaluating success of mobile health projects in the developing world. obal t, 7. maier, c., laumer, s., weinert, c., & weitzel, t. (2015). the effects of technostress and switching stress on discontinued use of social networking services: a study of facebook use. information systems journal, 25(3), 275-308. malvey, d., & slovensky, d. j. (2014). mhealth: transforming healthcare: springer. mathieson, k. (1991). predicting user intentions: comparing the technology acceptance model with the theory of planned behavior. information systems research, 2(3), 173-191. mavhu, w., willis, n., mufuka, j., mangenah, c., mvududu, k., bernays, s., . . . weiss, h. a. (2017). evaluating a multi-component, community-based program to improve adherence and retention in care among adolescents living with hiv in zimbabwe: study protocol for a cluster randomized controlled trial. trials, 18(1), 478. mechael, p. (2009). the case for mhealth in developing countries. innovations. 4(1), 103-118. medhanyie, a., spigt, m., dinant, g., & blanco, r. (2012). knowledge and performance of the ethiopian health extension workers on antenatal and delivery care: a cross-sectional study. human resources for health. 10(1), 44. medhanyie, a. a., moser, a., spigt, m., yebyo, h., little, a., dinant, g., & blanco, r. (2015). mobile health data collection at primary health care in ethiopia: a feasible challenge. journal of clinical epidemiology. 68(1), 80-86. miah, s. j., hasan, n., hasan, r., & gammack, j. (2017). healthcare support for underserved communities using a mobile social media platform. information systems, 66, 1-12. miles, m. b., & huberman, a. m. (1994). qualitative data analysis: an expanded sourcebook: sage. australasian journal of information systems eze, gleasure & heavin 2019, vol 23, research on health information systems factors that influence appraisal of mhealth tools 25 miller, c. t., & kaiser, c. r. (2001). a theoretical perspective on coping with stigma. journal of social issues 57(1), 73-92. modi, d., gopalan, r., shah, s., venkatraman, s., desai, g., desai, s., & shah, p. (2015). development and formative evaluation of an innovative mhealth intervention for improving coverage of community-based maternal, newborn and child health services in rural areas of india. global health action. 8(1), 26769. moreland, j . , french, t. , & cumming, g. (2016) . exploring online health information seeking in scotland. paper presented at the proceedings of the 3rd european workshop on practical aspects of health inf ormatics. elgin, uk. retrieved from http://radar.gsa.ac.uk/4582/1/exploring% 20ohis% 20scotland. pdf. morrison, j., shrestha, n. r., hayes, b., & zimmerman, m. (2013). mobile phone support for rural health workers in nepal through ‘celemedicine’. journal of nepal medical association, 52(191). murray, e., lo, b., pollack, l., donelan, k., catania, j., lee, k., . . . turner, r. (2003). the impact of health information on the internet on health care and the physician-patient relationship: national us survey among 1.050 us physicians. j med internet res, 5(3), e17. nach, h., & lejeune, a. (2010). coping with information technology challenges to identity: a theoretical framework. computers in human behavior, 26(4), 618-629. newton, a. t., & mcintosh, d. n. (2010). specific religious beliefs in a cognitive appraisal model of stress and coping. international journal for the psychology of religion, 20(1), 39-58. ngabo, f., nguimfack, j., nwaigwe, f., mugeni, c., muhoza, d., wilson, d. r., . . . binagwaho, a. (2012). designing and implementing an innovative sms-based alert system (rapidsms-mch) to monitor pregnancy and reduce maternal and child deaths in rwanda. the pan african medical journal, 13, 1-15. nhavoto, j. a., grönlund, å., & klein, g. o. (2017). mobile health treatment support intervention for hiv and tuberculosis in mozambique: perspectives of patients and healthcare workers. plos one. 12(4), e0176051. nicolaou, a. i., & mcknight, d. h. (2006). perceived information quality in data exchanges: effects on risk, trust, and intention to use. information systems research, 17(4), 332-351. orlikowski, w. j. (1993). case tools as organizational change: investigating incremental and radical changes in systems development. mis quarterly. 309-340. paina, l., & peters, d. h. (2011). understanding pathways for scaling up health services through the lens of complex adaptive systems. health policy and planning. 27(5), 365-373. paradis, j. (2011). individual differences in child english second language acquisition: comparing child-internal and child-external factors. linguistic approaches to bilingualism, 1(3), 213-237. patton, m. q. (1990). qualitative evaluation and research methods: sage publications, inc, berverly hills, ca. australasian journal of information systems eze, gleasure & heavin 2019, vol 23, research on health information systems factors that influence appraisal of mhealth tools 26 polites, g. l. (2005). counterintentional habit as an inhibitor of technology acceptance. paper presented at the proceedings of the 2005 southern association for information systems conference, 264-271. ragu-nathan, t., tarafdar, m., ragu-nathan, b. s., & tu, q. (2008). the consequences of technostress for end users in organizations: conceptual development and empirical validation. information systems research, 19(4), 417-433. ram, s., & khatri, v. (2005). a comprehensive framework for modeling set-based business rules during conceptual database design. information systems, 30(2), 89-118. recker, j. c. (2014). towards a theory of individual-level discontinuance of information systems use. paper presented at the proceedings of the 35th international conference on information systems. reinecke, k., yeh, t., miratrix, l., mardiko, r., zhao, y., liu, j., & gajos, k. z. (2013). predicting users' first impressions of website aesthetics with a quantification of perceived visual complexity and colorfulness. paper presented at the sigchi conference on human factors in computing systems. roberts, p., priest, h., & traynor, m. (2006). reliability and validity in research. nursing standard. 20(44), 41-45. rogers, e. m. (2003). elements of diffusion. diffusion of innovations,. 5, 1-38. sanner, t. a., manda, t. d., & nielsen, p. (2014). grafting: balancing control and cultivation in information infrastructure innovation. journal of the association for information systems, 15(4), 220-243. sarker, s., sarker, s., sahaym, a., & bjørn-andersen, n. (2012). exploring value cocreation in relationships between an erp vendor and its partners: a revelatory case study. mis quarterly. 36(1). seidel, s., & urquhart, c. (2013). on emergence and forcing in information systems grounded theory studies: the case of strauss and corbin. journal of information technology, 28(3), 237-260. shu, q., tu, q., & wang, k. (2011). the impact of computer self-efficacy and technology dependence on computer-related technostress: a social cognitive theory perspective. international journal of human-computer interaction, 27(10), 923-939. simon, s. k., & seldon, h. l. (2012). personal health records: mobile biosensors and smartphones for developing countries. studies in health technology and informatics 182, 125-132. srite, m., & karahanna, e. (2006). the role of espoused national cultural values in technology acceptance. mis quarterly. 679-704. stanton, m. c., mkwanda, s. z., debrah, a. y., batsa, l., biritwum, n.-k., hoerauf, a., . . . kelly-hope, l. a. (2015). developing a community-led sms reporting tool for the rapid assessment of lymphatic filariasis morbidity burden: case studies from malawi and ghana. bmc infectious diseases 15(1), 214. strauss, a., & corbin, j. (1998). basics of qualitative research: procedures and techniques for developing grounded theory: thousand oaks, ca: sage. australasian journal of information systems eze, gleasure & heavin 2019, vol 23, research on health information systems factors that influence appraisal of mhealth tools 27 taylor, s., & todd, p. a. (1995). understanding information technology usage: a test of competing models. information systems research, 6(2), 144-176. terry, d. j. (1994). determinants of coping: the role of stable and situational factors. journal of personality and social psychology. 66(5), 895. urquhart, c. (2000). an encounter with grounded theory: tackling the practical and philosophical issues. qualitative research in is: issues and trends. 104-140. urquhart, c., lehmann, h., & myers, m. d. (2010). putting the ‘theory’back into grounded theory: guidelines for grounded theory studies in information systems. information systems journal, 20(4), 357-381. venkatesh, v. (2000). determinants of perceived ease of use: integrating perceived behavioral control, computer anxiety and enjoyment into the technology acceptance model. information systems research, 11(4), 342-365. venkatesh, v., & morris, m. g. (2000). why don't men ever stop to ask for directions? gender, social influence, and their role in technology acceptance and usage behavior. mis quarterly. 115-139. venkatesh, v., morris, m. g., davis, g. b., & davis, f. d. (2003). user acceptance of information technology: toward a unified view. mis quarterly. 425-478. venkatesh, v., thong, j. y., & xu, x. (2012). consumer acceptance and use of information technology: extending the unified theory of acceptance and use of technology. mis quarterly. 157-178. wedgwood, r. (2006). the internal and external components of cognition. contemporary debates in cognitive science. 307-325. wisniewski, p., xu, h., & chen, y. (2014). understanding user adaptation strategies for the launching of facebook timeline. paper presented at the proceedings of the 32nd annual acm conference on human factors in computing systems. xue, y., liang, h., mbarika, v., hauser, r., schwager, p., & getahun, m. k. (2015). investigating the resistance to telemedicine in ethiopia. international journal of medical informatics. 84(8), 537-547. yin, r. k. (2013). case study research design and methods third edition. applied social research methods series. 5, 1-312. australasian journal of information systems eze, gleasure & heavin 2019, vol 23, research on health information systems factors that influence appraisal of mhealth tools 28 appendix a – data sources and interviewees’ roles data sources role 8* interviews with seven parents/guardians (pgs) mothers – these are parents to the children under the age of five in the target community whose primary tasks amongst others is to take care of their children’s health in their homes. 8* interviews with seven rural healthcare workers (rhcws) rhcws – these are trained healthcare professionals working in the community health centres. rhcws are the lowest cadre of health professional in the health system. they are a mixture of nurses and those trained specifically to help in healthcare services. rchws are called community healthcare workers (e.g. derenzi et al., 2012), health extension workers (e.g. a. medhanyie, spigt, dinant, & blanco, 2012), and local health workers (e.g. ngabo et al., 2012) in other jurisdictions. rhcws were the direct users of the mhealth tool in this study. 8* interviews with six facilitators head of service – head of the entire public service or public servants that work in enugu state civil service. responsibility include to make sure that all adapt appropriately in their workplace and working in order that they deliver on their mandate. local government chairman – chairman of the transitional committee of nsukka local government. one of the 17 local governments in enugu state. health data manager – head, enugu state’s health management system officer. work responsibility include, human resource officer, health information system officer and in-charge of the health accounts of the state. provost of college of medicine – responsibilities include, train medical students, and support them through their medical training. director clinical services – facilitation of service delivery by all the clinical staff, the doctors, the nurses, the medical laboratory scientists, the pharmacies, the therapists and all the other medical or healthcare workers. director, primary health care (local government services commission) – work responsibilities include the facilitation of national programmes at the local government levels. 8* interviews with six developers principal investigator – the head of the impact project. lead the designing the app, and decides on what the app ultimately becomes. software programmer – involves mainly in software development, software design, and user interface design, and usability analysis. research partner – lead collaborator representing impact project. makes contribution towards the designing and customising the app. research collaborator – offers advice on the clinical aspects of the app design and development. member of the collaborator’s team – former director of disease control in the state’s health ministry. insights on challenges during guidelines developments. former director of public and primary healthcare at the national level. participated in writing the health policy and the health guidelines. field notes from observations of pgs’ homes, edem-ani, alor-unor, ibagwa-ani, okpuje, and okwutu health centres. medical papers & photographs images of rural health centres, some pictures of the social actors, the paper-based standing operation procedure (sop); paper-based facility registers, paper-based summary form, wall photographs of health information systems (his) related charts, graphs and paper forms. 8* interviews with six system developers australasian journal of information systems eze, gleasure & heavin 2019, vol 23, research on health information systems factors that influence appraisal of mhealth tools 29 appendix b – research questions for rural healthcare workers (rhcws) how do you feel about this new mhealth app? to what extent do you think that this new app would have a positive impact on your work practices? to what extent do you think that this new app would have a positive impact on co-workers’ work practices? to what extent do you think that this new app would have a negative impact on your work practices? to what extent do you think that this new app would have a negative impact on co-workers’ work practices? to what extent do you believe this new app would be part of a broader positive/negative trend in healthcare delivery in enugu state? to what extent do you intend to explore different features on this new app? what, if any, other things do you think this new app could do for you? what, if any, challenges did you face connecting to the internet? to what extent do you see this new app changing the way you perform your duties? to what extent do you think you can perform your duties using this new app without outside help? what, if any, challenges did you face when trying to get familiar using this new app? is there any reason why you would avoid using this new app in the future? how do you feel after using this new mhealth app? after using this new app, to what extent do you think that it would have a positive impact on your work practices? after using this new app, to what extent do you think that it would have a positive impact on co-workers’ work practices? after using this new app, to what extent do you think that it would have a negative impact on your work practices? after using this new app, to what extent do you think that this new app would have a negative impact on co-workers’ work practice? after using this new app, to what extent do you believe it would be part of a broader positive/negative trend in healthcare delivery in enugu state? australasian journal of information systems eze, gleasure & heavin 2019, vol 23, research on health information systems factors that influence appraisal of mhealth tools 30 appendix c – research questions for parents/guardians (pgs) how do you feel about this new mhealth app? to what extent do you think that this new app would have a positive impact on the way your child would be assessed at the health centre? to what extent do you think that this new app would have a positive impact on fellow parents in your community? to what extent do you think that this new app would have a negative impact on fellow parents in your community? to what extent do you think that this new app would have a negative impact on fellow parents in your community? to what extent do you believe this new app would be part of a broader positive/negative trend in healthcare delivery in enugu state? to what extent do you think that healthcare workers would like to explore the different features on this new app? what, if any, other things do you think this new app could do for rural healthcare workers? what, if any, challenges do you think that rural healthcare workers would face connecting to the internet? to what extent do you see this new app changing the way rural healthcare workers perform their duties? to what extent do you think that healthcare workers can perform their duties using this new app without outside help? what, if any, challenges do you think that rural healthcare workers would face when trying to get familiar with using this new app? is there any reason why you think that rural healthcare workers would avoid using this new app in the future? how do you feel after using this new mhealth app? after using this new app, to what extent do you think that it would have a positive impact on healthcare practices in enugu state? after using this new app, to what extent do you think that it would have a positive impact on rural healthcare workers’ practices? after using this new app, to what extent do you think that it would have a negative impact on health practices in enugu state? after using this new app, to what extent do you think that this new app would have a negative impact on rural healthcare workers’ practices? after using this new app on your child/children, to what extent do you believe it would be part of a broader positive/negative trend in healthcare delivery in enugu state? australasian journal of information systems eze, gleasure & heavin 2019, vol 23, research on health information systems factors that influence appraisal of mhealth tools 31 appendix d – research questions for facilitators how do you feel about this new mhealth app? to what extent do you think that this new app would have a positive impact on the way you want children to be assessed in enugu state? to what extent do you think that this new app would have a positive impact on fellow facilitators in enugu healthcare system? to what extent do you think that this new app would have a negative impact on fellow facilitators in enugu healthcare system? to what extent do you think that this new app would have a negative impact on fellow facilitators in enugu healthcare system? to what extent do you believe this new app would be part of a broader positive/negative trend in healthcare delivery in enugu state? to what extent do you think that rural healthcare workers would like to explore the different features on this new app? what, if any, other things do you think this new app could do for rural healthcare workers? what, if any, challenges do you think that rural healthcare workers would face connecting to the internet? to what extent do you see this new app changing the way rural healthcare workers perform their duties? to what extent do you think that rural healthcare workers can perform their duties using this new app without outside help? what, if any, challenges do you think that rural healthcare workers would face when trying to get familiar with using this new app? is there any reason why you think that rural healthcare workers would avoid using this new app in the future? how do you feel after using this new mhealth app? after using this new app, to what extent do you think that it would have a positive impact on healthcare practices in enugu state? after using this new app, to what extent do you think that it would have a positive impact on healthcare workers’ practices? after using this new app, to what extent do you think that it would have a negative impact on health practices in enugu state? after using this new app, to what extent do you think that this new app would have a negative impact healthcare workers’ practices? after using this new app, to what extent do you believe it would be part of a broader positive/negative trend in healthcare delivery in enugu state? australasian journal of information systems eze, gleasure & heavin 2019, vol 23, research on health information systems factors that influence appraisal of mhealth tools 32 appendix e – research questions for developers how do you feel about this new mhealth app? to what extent do you think that this new app would have a positive impact on the way you want children to be assessed in enugu state? to what extent do you think that this new app would have a positive impact on fellow developers in enugu healthcare system? to what extent do you think that this new app would have a negative impact on fellow developers in enugu healthcare system? to what extent do you think that this new app would have a negative impact on fellow developers in enugu healthcare system? to what extent do you believe this new app would be part of a broader positive/negative trend in healthcare delivery in enugu state? to what extent do you think that rural healthcare workers would like to explore the different features on this new app? what, if any, other things do you think this new app could do for rural healthcare workers in enugu state? what, if any, challenges do you think that rural healthcare workers would face connecting to the internet? to what extent do you see this new app changing the way healthcare workers perform their duties? to what extent do you think that rural healthcare workers can perform their duties using this new app without outside help? what, if any, challenges do you think that rural healthcare workers would face when trying to get familiar with using this new app? is there any reason why you think that rural healthcare workers would avoid using this new app in the future? how do you feel after using this new mhealth app? after using this new app, to what extent do you think that it would have a positive impact on healthcare practices in enugu state? after using this new app, to what extent do you think that it would have a positive impact on rural healthcare workers’ practices? after using this new app, to what extent do you think that it would have a negative impact on health practices in enugu state? after using this new app, to what extent do you think that this new app would have a negative impact rural healthcare workers’ practices? after using this new app, to what extent do you believe it would be part of a broader positive/negative trend in healthcare delivery in enugu state? australasian journal of information systems eze, gleasure & heavin 2019, vol 23, research on health information systems factors that influence appraisal of mhealth tools 33 appendix f – details of open, axial, and selective coding the gt coding process included three major types of coding, namely: through open, axial, and selective coding processes (orlikowski, 1993; urquhart, lehmann, & myers, 2010). f1 open coding open coding refers to classifying/breaking data into concepts that may explain important incidences or happenings about the phenomenon (böhm, 2004; gasson, 2004). in order to stay deeply connected to the research topic we followed the line of questioning provided by glazer (1978, p. 57) that is used in generating codes: 1) "what is this data a study of?" 2) "what category does this incident indicate?" 3) "what is actually happening in the data?" open coding began with a ‘line-by-line’ analysis of the data (charmaz, 2006, 2014). through this process, we created 35 codes that were given conceptual labels that related to 450 word-based data-sets from thirty-two interviews along with written notes (30 pages) from the interviews, and documentation. subsequently, conceptually similar incidences were grouped together to form common categories (corbin & strauss, 1990). in open coding, we focused on the stakeholders’ primary appraisal of the proposed mhealth tool for healthcare delivery in enugu state. for example, we coded a portion of rhcws’ interview, i.e., “many people find it difficult to change the way they do things” as ‘habit’, and a portion of facilitators’ interview, i.e., “the people who are not good with technology will be afraid of its introduction” as ‘computer anxiety’. open codes were developed for each portion of the data-set as presented in tables 2 and 3. emerging codes from each data-set were subsequently compared against varying viewpoints as recurring themes emerge from the data for consistency. f2 axial coding axial coding refers to the comparisons of the emerging themes or subthemes to classify them into meaningful categories which enable the creation of a more hierarchical groupings (abraham, boudreau, junglas, & watson, 2013; gasson, 2004). that is, it helps to fine-tune and differentiate themes or subthemes and lends them into other status or levels of classifications in relation to the data. axial coding entails the search for relationships between coded concepts identified during open coding and by ensuring that the evolving interview instruments captured emerging constructs and relationships (gasson, 2004; gleasure, 2015). the iterations between the researchers and the data allowed the initial model to be expanded and delineated into a clearly defined and well-articulated hypothesis-based model and the underlining processes. following this technique, we related and combined codes to form themes representing sources of threat and opportunity appraisals towards mhealth assimilation. these themes fall under the ‘causal conditions’ category of strauss and corbin paradigm (bohm, 2004; seidel & urquhart, 2013). for example, we created relationship between the codes of ‘habit’, ‘computer anxiety’ and the effect of norms and cultural values to form the theme ‘perceived threat from process uncertainty’. f3 theoretical memos theoretical memos are write-ups of ideas relating to codes and themes, and between themes themselves which ultimately form the basis for writing theory (bohm, 2004; gasson, 2004). memos provide avenues to capture insights into the analysis process and contain clues to integration in so far as the researchers have systematically recognised the properties of the ideas together with their dimensions (strauss & corbin, 1998). for example, the memo ‘the inadvertent threat perceived by stakeholders with regards to changes that would affect habit/practice.’ refers to the proposed introduction of an mhealth tool. for it to be ‘positively’ appraised by the target communities, the programme initiators need to design technological solutions that reflect local realities and needs (chib, 2013; kay, santos, & takane, 2011). australasian journal of information systems eze, gleasure & heavin 2019, vol 23, research on health information systems factors that influence appraisal of mhealth tools 34 omitting memos and moving directly from coding to writing-up may impact the conceptual detail and integration of ideas (corbin & strauss, 1990; glaser & strauss, 1967). constructs and relationships identified in theoretical memos must be supported by further data analysis or it would just speculation and not theory (gasson, 2004). f4 selective coding selective coding is the integrating and refining of emerging core categories at the later stages of a coding process (corbin & strauss, 1990, 2008, 2014). this integrative procedural feature is considered an essential force in theory building or in explaining phenomenon (seidel & urquhart, 2013; urquhart, 2000). the refining process involves constant comparison between categories and data (lowe, 1996). that is, moving up and down the levels of analysis and looking for traces of relationships which it might explain and by incorporating relevant data up to a point where no more evidence is discovered (data saturation) (andriopoulos & lowe, 2000). it was at this stage that poorly developed categories were discovered and refined by revisiting data to fill-in the gaps. subsequently, core categories were defined and labelled. two core categories resulting from axial coding were consistent with the classification that evolved from contemporary scholars’ work in primary appraisals of technology application in organisations setting: ‘threats’ and ‘opportunities’ (connolly & bhattacherjee, 2011; wisniewski et al., 2014). copyright: © 2019 eze, gleasure & heavin. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems mcgrath & lipman 2016, vol 20, design research for decision support systems development green growth tourism dss 1 construction and use of a ‘green growth’ tourism decision support system: a multi-model approach g. michael mcgrath victoria university michael.mcgrath@vu.edu.au geoffrey lipman victoria university greenearth travel abstract for design, development, implementation and use of an information system (is) to constitute a valid research activity, the system should support the solution of a non-trivial and important problem and it should be original, drawing on existing theories and knowledge. the design of one such system is described in this paper: specifically, a decision support system (dss) designed to support the development of ‘green growth’ (gg) strategies for travelism (travel & tourism) destinations. a sound gg strategy is important: first, because tourism is a major contributor to the global economy particularly for developing and island states; second because it represents some 5% of greenhouse gas (ghg) emissions and these are increasing faster than the global norm; and third because the environment is an essential element of destination attractiveness. thus, the problem domain is certainly non-trivial and important. it is further argued that the design of the dss artefact described is original and novel in the sense that: i) it supports the entire gg strategy development process (which is actually cyclical); ii) it allows for the sharing of data, functionality and knowledge between different dss applications and different strategy development exercises in a seamless, integrated manner; and iii) it will be deployed in a global community based program in 2016. system design draws heavily on previous is, information management and software engineering research; particularly with regard to use of abstraction and interfaces in support of component sharing and reuse. keywords: decision support systems; green growth strategies; tourism; travel; travelism; design science. 1 introduction travelism is a complex socio-economic phenomenon, consisting of transport, hospitality, events, travel services, as well as the soft and hard infrastructure that enables the movement of people and their related goods (lipman, delacy, vorster, hawkins and jiang, 2012). it has emerged in the past half century, as a strong and growing part of global, regional, national and local economies, trade and development, and has many societal benefits. an increasing number of tourism destinations have investigated the adoption of ‘green growth (gg)’ strategies as a means of addressing a variety of critical concerns and issues (simpson et al., 2008; scott et al., 2008). these include: i) climate change and its current and future impacts (scott, defreitas, and matzarakis, 2009); ii) severe environmental and social problems such as pollution, degradation and loss of forest, bushland and coastal land areas, critical energy, water and land shortages, acute traffic congestion and other major infrastructure problems, and rising unemployment, crime and delinquency rates (unep, 2010); iii) the need to rejuvenate destinations that have reached the stagnation or decline stages of their life-cycles (butler, 2006); and iv) an apparent willingness on the part of visitors to pay a premium where tourism business operators have adopted sound environmental practices (hawkins and bohdanowicz, 2011). development of appropriate strategies is, however, not a simple matter. in part, this is because the policy development process demands that a substantial volume (and diverse range) of data australasian journal of information systems mcgrath & lipman 2016, vol 20, design research for decision support systems development green growth tourism dss 2 be analyzed. in addition, the policy domain contains a large number of variables, covering the economic, environmental and social dimensions, with variables interacting with each other in a complex myriad of ways; i.e a classic case of a “wicked’ or ‘messy’ problem (buchanan, 1992; vennix, 1996). obviously, information technology can assist in managing this complexity and, in addition, the analytical tools, scenario generation functionality and simulation capabilities characteristic of modern decision support systems (dss) can be used to advantage in evaluating the possible impacts of proposed strategies. while much research and tourism industry activity has focused on the design of dss that address part of the gg strategy development process, there is no current tool that supports the total, end-to-end process: from benchmarking, through forecasting, to the specification of adaptation and mitigation strategies. given that each of the later phases in the total process depends heavily on earlier phases, and the major variable interactions within and between the three fundamental dimensions of the sustainable tourism domain (economic, environmental and social) (brundtland, 1987), it would seem reasonable to require that a gg strategy development tool should be highly-integrated. the design of such an integrated dss is presented in this paper. from a design science (ds) research perspective, our dss artefact represents a ‘new solution for a known problem’ (categorised as “improvement” by gregor and hevner, 2013, p345), with a major contribution being the detailed specification of the end-toend gg strategy development process as a ds ‘method’ (march and smith, 1995). the design artefact has been instantiated as a working dss and used in the field to assist in the development of gg strategies at a number of tourism destinations, with further applications planned. the paper is organized as follows: relevant, background literature is presented in the following section, the research approach is detailed in section 3 and this is followed (in sections 4-8) by a discussion of the construction and use of our ‘green economy tourism system’ (gets) dss, broken down into the design science research phases identified by peffers, tuunanen, rothenberger and chatterjee (2007-08): specifically problem identification and motivation, solution objectives, design and development, demonstration and evaluation, and communication. section 9 contains concluding remarks. 2 background it is estimated by unwto (2015) that more than a billion people a year travel away from their normal place of residence, across international borders, for business or leisure purposes. the number travelling domestically is estimated at 3-4 times that figure. furthermore, tourism’s economic impact is calculated by wttc (2012) at 3-10% of direct, indirect and induced global gdp, with similar impacts on jobs, trade and investment. travel and tourism, labelled ‘travelism’ lipman et al. (2012), have consistently grown ahead of gdp at an average of 4% per annum for the past 3 decades and is estimated to show similar results for the next twenty years (unwto, 2015). travelism is, at the same time, a major source of greenhouse gas (ghg) emissions worldwide (scott et al., 2008; gossling and peeters, 2015; scott, gossling, hall and peeters, 2015), as well as having significant waste, water food and other resource impacts, in parallel with this, many tourism destinations are facing serious problems as a result of an over-emphasis on economic objectives and too little attention being paid to environmental and social needs (brundtland, 1987; lipman et al., 2012). as a consequence, an increasing number of destinations have embarked upon strategic planning exercises centred upon a ‘green growth’ premise (mcgrath, law and delacy, 2012). gg strategy development though, is an extremely complex process and, as such, it is our contention that strategy development teams could benefit very much from the use of an appropriate dss. a generic gg strategy development process is illustrated in figure 1 and a key requirement of the dss is that it must be capable of providing useful advice to strategy developers at each australasian journal of information systems mcgrath & lipman 2016, vol 20, design research for decision support systems development green growth tourism dss 3 process step. at the top-most level, the key steps are: i) benchmarking, where crucial current metrics (e.g. for water and energy usage, waste and ghg emissions, tourism growth and development, and for transport and infrastructure) are located or derived; ii) projections and forecasts, which involves predicting future trends for all key measures – both under ‘businessas-usual’ (bau) and alternative scenarios; iii) mitigation, where strategies to alleviate the worst impacts of projected, future difficulties are developed and evaluated; iv) adaptation, including mitigation strategies, but supplemented with plans to deal with events that cannot be prevented (e.g. low-lying island flooding due to global climate change); v) resilience, collectively the cumulative results of all previous steps, involving the adaptation, reorganization, and evolution of a destination into a more desirable configuration that improves its sustainability; and, finally, vi) validation, the ongoing evaluation and feedback of the accuracy and effectiveness of all measures, projections, strategies and impacts: making for a self-correcting long-term ecosystem. figure 1: generic gg strategy development process (derived from a gg activity classification scheme developed by law, 2015). over the years, a considerable number of dss addressing elements of this total strategy development process have been developed. much early work was designed and developed using the ‘systems dynamics’ (sd) approach (senge, 1990; vennix, 1996). examples include the ‘tourism futures simulator’ of walker et al., 1999), the ‘hotel value chain profitability’ model of georgantzas (2003) and the ‘tourism enterprise planning system’ (teps) of mcgrath, (2007). perhaps the most comprehensive set of models and simulators developed under the broad sd umbrella is the unep ‘green economy report’s’ ‘threshold 21 (t21) world’ model (unep, 2010). more recently, a number of agent-based tourism demand prediction tools have begun to appear. better-known examples include the scenario generation and analysis system, ‘auronzowinsim’ (balbi et al., 2010), a simulator designed to estimate the impact of climate change on tourism water consumption (soboll and schmude, 2011) and baggio’s (2011) ‘tourdestmem’ model and simulator, which assesses the impacts of tourism destination marketing initiatives (and consequential impacts on competitors). to our knowledge though, there is no existing gg strategic planning dss that assists with every stage (see figure 1) of the strategy development process. the gets dss described in this paper provides end-to-end decision support for this process and does so in an integrated, seamless manner. notable features of the system are that: i) all data is defined consistently with an abstracted conceptual schema and may be shared between activities at each process step; ii) data may also be seamlessly shared between system functions (applications), regardless of the software platform on which they have been constructed and implemented; and iii) knowledgebased outcomes and lessons learned in earlier strategy development exercises are available for reuse in later studies. a feature of our study is that it is based on a ds research methodology (hevener, march and park, 2004), where the major objective is the design, construction, implementation and evaluation of a research artifact: in our case the gets dss. for a discussion of the use of design science in dss research, see arnott and pervan (2008). gregor and hevner (2013), building upon wilson’s (2002) argument, assert that a ds research project must be ‘interesting’, meaning that it: i) should advance knowledge of theory, methods or applications; ii) should be an advance on previous work; iii) might be used to solve australasian journal of information systems mcgrath & lipman 2016, vol 20, design research for decision support systems development green growth tourism dss 4 important problems elsewhere; and iv) should clarify our understanding of the research area addressed. according to gregor (2006), a theory is a form of knowledge, an abstract entity comprised of a set of connected statements that aims to describe, explain enhance understanding and, possibly, make predictions. gregor goes on to argue that design theories are what merton (1968, p39) has described as “theories of the middle range”: neither the routine heuristics or hypotheses that inevitably arise during a research study or unified, all-inclusive grand theories, but somewhere in between. in general, such theories are concerned more with prescriptive, rather than descriptive, knowledge. goes (2014) asserts that ds research is not so much concerned with creating or testing theory as with problem solving through the design and development of artifacts. as such, it is not necessary that a ds research study should be explicitly tied to theory. nevertheless, gregor and jones (2007) have stressed the importance of ‘justificatory knowledge’: knowledge used to inform design science research (including formal heuristics from field practitioners) and a concept that builds upon the ‘kernel theories’ idea of walls, widemeyer and el sawy (1992, p41). within the ds research community, it now seems to be generally accepted that an is artifact is comprised of constructs, models, methods and instantiations (march and smith, 1995). the strategy development process depicted in figure 1 is the highest-level view of the gets ‘method’. this process has been broken down into much more-detailed views (see mcgrath, meijerink and gutterson, 2014 for examples) and it is contended that, given that the gg strategy development process seems to be only very loosely-specified in the mainstream sustainable tourism literature, this detailed process specification (at various levels of abstraction) represents a considerable contribution to the domain. gregor and hevner (2013) have argued that a ds research project’s knowledge contribution should be assessed in terms of problem maturity and solution maturity. furthermore, improvement involves developing “… new solutions for known problems (p345) and we contend that our research fits into this category: specifically, while the sustainable tourism literature is rich in policy development studies focused on elements of the strategic planning process (see e.g. inskeep, 1991; bramwell and lane, 1993; lane, 1994), the concept of building gg ‘roadmaps’ (encompassing every area of a tourism destination’s activities) is relatively new, with most major studies having taken place within the past five years. moreover, the mostcomprehensive and widely-used of the integrated, sustainable tourism policy development and decision support frameworks and aids (see richie and crouch, 2003) are not automated and do not allow seamless sharing of data and processes between activities. a novel feature of the gets artifact is that data and processes of all components are integrated and integrated in a way that allows for: i) maintenance and enhancements to be realized without impacts (e.g. significant code rewrites) on existing models and instantiations; and ii) modules (e.g forecasting applications) developed using multiple tools and software platforms to communicate without modification. this end-to-end process integration is due to two ds ‘models’ that largely inform the design and development of the gets artifact and its implementation as the gets dss: specifically, we refer to the highly-abstracted (and levelled) data models (feldman and miller, 1986) and the accompanying iso 3-schema architecture (van griethuysen, 1986). neither abstracted data modelling nor the 3-schema architecture are new of course but, as was the case 20 years back, truly-integrated instances of complex, multiapplication dsss (and indeed any sort of is) would appear to be still rare today. for example, at a recent us dod conference, it was reported that database and is integration efforts are still characterized more by failure than success and that more than 50% of all large, commercial data warehouse programs (which are underpinned by the assumption of effective integration of multiple datasets) fail (curtis and campbell, 2015). care needs to be taken though, when discussing dss integration. in presenting a framework for evaluating dss integration, liu, duffy, whitfield and boyle (2009) argued that integration australasian journal of information systems mcgrath & lipman 2016, vol 20, design research for decision support systems development green growth tourism dss 5 extent can be assessed according to: perspective (data, model, process, service and presentation); dimension (horizontal and vertical); level (null, in-project and generic); and type (tight and loose). in this project we are concerned with loose integration, where components interact via middleware according to some ‘agreement’ (linthicum, 2003) (encapsulated in our case in a common, abstracted conceptual schema). we also require high levels of both horizontal integration (i.e. where multiple parties can work on common activities) and vertical integration (seamless integration between upstream and downstream activities), and we have a focus on data, model and process perspectives. we are less-concerned with service and presentation integration, taking the view that, at the model interaction level, a common user interface is of less value than interfaces designed specifically for model and process types i.e. a ‘horses for courses’ approach (curtis, kellner and over, 1992). 3 research approach the last two decades have seen the rise of the ds research approach; sometimes described as the ‘design science research methodology’ (dsrm) (peffers et al., 2007-08). as far back as the 1960s, simon (1969, p55) argued “whereas natural sciences and social sciences try to understand reality, design science attempts to create things that serve human purposes”. hevner et al. (2004) proposed seven guidelines for conducting ds research. these are: i) that the research must produce an “artefact created to address a problem”; ii) the artifact should be relevant to the solution of a “heretofore unsolved and important business problem”; iii) its “utility, quality, and efficacy” must be rigorously evaluated; iv) the research should represent a verifiable contribution; v) rigor must be applied in both the development of the artifact and its evaluation; vi) the development of the artifact should be a search process that draws from existing theories and knowledge to come up with a solution to a defined problem; and, finally, vii) the research must be effectively communicated to appropriate audiences. pfeffers et al. (2007-08) reviewed much of the ds research and specified the process model presented in figure 2 as a consensus of the better-known dsrms proposed to that point in time. this represents the research design of our study and further detail will be presented in the following sections, which are broken up consistent with the major phases of the model. figure 2: dsrm process model (reproduced from hevner et al., 200, p52). nunamaker, chen and purdin (1991) have argued that is design using prototyping is similar to case study research, with each new prototype version building upon experience gained with previous versions. a prototyping approach underpinned the design and development of the dss described in this paper, with the system undergoing the following major revisions: i) an initial version was developed circa 2009, building on lessons learned in developing an earlier dss based largely on the ‘tourism area life cycle’ (pornphol and mcgrath, 2011); ii) this australasian journal of information systems mcgrath & lipman 2016, vol 20, design research for decision support systems development green growth tourism dss 6 initial version was refined and extended during the period 2010-11, when employed to assist with a gg strategy development exercise at sharm el sheik, egypt (law, delacy, mcgrath, whitelaw, lipman and buckley, 2012); and iii) the system underwent further major refinement (circa 2012-14) when used in follow-up gg strategy development exercises at jeju island in the republic of south korea (lipman, delacy and whitelaw, 2013) and in bali, indonesia (mcgrath, law and delacy, 2015). as detailed in section 7, it is anticipated that additional extensions will result from further studies anticipated to take place during 2016-18 finally, as noted by peffers et al. (op cit.), there are two sets of ds literature. one revolves around issues of actually doing academic design work; i.e., design research, while the second set addresses the meta-level of conducting research at a higher level of abstraction; it is research about design research. our study contributes to the first of these research areas. 4 problem identification and motivation this process involves defining the research problem and justifying the value of a solution (peffers et al., 2007-08, p52). in this instance, the research problem is to: “develop and implement a dss designed to support and assist those parties involved in the specification of gg strategies within travelism destinations.” as noted by mcgrath et al. (2012), the gg domain is extremely complex, comprised as it is of a large number of variable types and instances. reasons for this complexity include: i) the relevant variables encompass the economic, environmental and social dimensions (and relations within and between these); ii) the gg domain is a classical instance of a ‘messy’ problem (vennix, 1996), characterised (among other features) by a wide variety of vicious (and virtuous) circles; iii) the substantial volume and diversity of data referred to above needs to be modelled, structured and organized; and iv) tourism destinations vary greatly in terms of both their size and character, ranging as they do from large, semi-global regions to much smaller, special-purpose areas (e.g. ski and beach resorts). thus, any dss strategic planning aid must be both ‘scaleable’ and amenable to convenient customization. while development of a dss that addresses the needs detailed above is probably a worthwhile exercise from a pure, technical dss standpoint alone, the research problem (and, more specifically, the solution) aims to contribute towards the realization of a very important environmental and social objective: specifically, the maintenance of sustainable environmental and social goals while promoting economic growth (brundtland, 1987), moreover to quote from the ‘leaders declaration’ at the g20 meeting held in mexico during 2012 (g20, 2012, pp12-13): “we highlight that green growth and sustainable development have strong potential to stimulate long-term prosperity and well-being. . . . we recognize the role of travel and tourism as a vehicle for job creation, economic growth and development, and, while recognizing the sovereign right of states to control the entry of foreign nationals, we will work towards developing travel facilitation initiatives in support of job creation, quality work, poverty reduction and global growth.” however, as noted, travelism is a major contributor to global greenhouse gas (ghg) emissions and scott et al. (2008) estimate that emissions could grow by 161% by 20351. given the new tough national carbon reduction commitments and global targets agreed at the recent un climate change conference in paris (cop21) during december 2015, climate change mitigation, adaptation and resilience is an increasingly critical issue for many travelism destinations. in addition, the sector faces a number of industry-specific problems; notably: it is highly energy-intensive with aviation likely to be highly-dependent on fossil fuel for the forseeable future, water use can be extremely high (gossling, 2005) and tourists tend to generate (relatively) very high levels of waste (hamele and eckhardt, 2006; unep, 2010). 1 these figures were recently updated by gossling and peeters (2015), with similar conclusions. australasian journal of information systems mcgrath & lipman 2016, vol 20, design research for decision support systems development green growth tourism dss 7 moreover, gg strategy development is resource-intensive. for example, in the case study detailed in (law et al., 2012) a team of ten researchers was involved for a period of close to six months. moreover, two senior researchers and a principal research assistant were involved in the study on a close to full-time basis. cost is also an issue, in that many (newer and evolving tourism destinations especially) may not be able to easily afford the outlays demanded by commercial consultancy firms required for development of a comprehensive gg roadmap, which can easily run into millions of dollars all of this, we believe, points to the need for some sort of automated strategy development support: specifically, the need for a dss that provides appropriate process guidance, allows the sharing of knowledge, experience and results in an affordable and convenient way so that local communities may chart their own lifestyle destiny. finally, gg strategy development is a complicated process, characterized by highly-complex and uncertain relationships between gg drivers and inhibitors. to our knowledge, currently there is no existing, automated planning framework designed to assist gg strategy developers in understanding and leveraging key causal connections among these inhibitors and drivers. this, together with the other factors mentioned above, point to the need for the type of dss intended as the principal output artefact of this design-based research project. 5 solution objectives the purpose of this phase is to: “infer the objectives of a solution from the problem definition and knowledge of what is possible and feasible” (peffers et al., 2007-08, p53). thus, the major objective is to: “specify, develop, implement, evaluate and validate a gg strategy development support dss for use within tourism destinations of all types and sizes.” again, referring to peffers et al. (2007-08), objectives should flow directly from the problem specification and, ideally, both of these (problem and solution) should be “atomized” and represented “conceptually” (p53). in this instance, we have chosen to represent our solution as the requirements taxonomy presented in figure 3. australasian journal of information systems mcgrath & lipman 2016, vol 20, design research for decision support systems development green growth tourism dss 8 figure 3: solution objectives – requirements taxonomy. problem-solution linkages can be verified easily enough by referring back to section 4. in figure 3, the topmost section deals with domain requirements and the bottom section deals with supporting, dss functional and technical needs. thus, the overriding objective for tourism destinations is to address critical issues they are confronting; particularly those related to climate change, projected rapid increases in ghg emissions and the consequent need to conserve precious resources (notably energy, water and land). as noted in the previous section, strategy development must address the economic, environmental, social and climate change dimensions, and interactions between key variables, within and between dimensions, results in substantial domain complexity. any gg strategy developed and implemented must take adequate account of these interactions or risk serious, unintended consequences (senge, 1990). this brings us to our major research objective (shown in the central section of figure 3), as specified at the beginning of this section; the design, development, implementation, evaluation and validation of our gg strategy development dss. the dss must include the following (toplevel) functions: a strategy development guide, establishment, maintenance and access to a gg knowledge repository, knowledge sharing between all dss functions and the ability to reuse both dss functionality and accumulated experience from one destination to another. further detail on lower-level functionality is provided in the following section. 6 design and development 6.1 overview during the design and development phase, the artifact is created. as peffers et al. (2007-08, p53) have noted: australasian journal of information systems mcgrath & lipman 2016, vol 20, design research for decision support systems development green growth tourism dss 9 “such artifacts are potentially constructs, models, methods, or instantiations (each defined broadly) ----. conceptually, a design research artifact can be any designed object in which a research contribution is embedded in the design. this activity includes determining the artifact’s desired functionality and its architecture and then creating the actual artifact.” a view of the gets domain and its representation is presented in figure 7. the domain (or, more accurately, an instance of a domain) is what van griethuysen (1982) asserts to be the universe of discourse (uod) that collection of objects, from a real or postulated world that is being described. in our case, the world of interest is centered on green growth travelism, as a contributor to better sustainable community lifestyles. further detail on domain representation is presented in the following section but its specification is encapsulated in the conceptual schema (at the top of figure 4). the conceptual schema is a highly-abstracted specification of the uod and a set of mappings is employed to translate this view to and from external schemas associated with the individual applications that constitute the gets system2. these applications are represented in figure 4 at the centre of the diagram. all decision support applications utilise the data captured and organized during benchmarking. as noted earlier, the applications themselves are developed using a variety of languages and software platforms and are concerned with forecasts, validation, the discovery and application of expert rules and connections to previous, similar cases through case-based reasoning (cbr – and discussed in more detail in section 6.4). markus et al. (2002) have argued that is development constitutes original research where the development and implementation are original and new knowledge is generated about how to manage data and information in complex situations. we argue that the gets domain is certainly complex. specifically: i) a large number of data types and a large volume of data must be managed; and ii) the system must be capable of controlling the evolution of database schemas (and associated data manipulation code) as the system is employed with each new destination and as new functionality is added. concerning the other criterion, effective and efficient data and information management is a key objective of the gets system: specifically, we contend that the following aspects of our research constitute novel and ‘original’ contributions: i) the system provides end-to-end support for the gg strategy development process and allows for automated, integrated data sharing and reuse within and between all major sub-processes (see figure 4); 2 thus, the gets design is essentially based on the iso 3-schema architecture (van griethuysen, 1982). beyond the conceptual and external schemas are the internal-level schemas, the details of which are outside the scope of this paper. australasian journal of information systems mcgrath & lipman 2016, vol 20, design research for decision support systems development green growth tourism dss 10 figure 4: the gets domain and its representation. ii) the system is truly-integrated, in the sense that data may be seamlessly shared between different applications regardless of type and the supporting software platform3; and iii) data, findings and knowledge may be conveniently shared among destinations conducting (or who have previously conducted) gg strategy development exercises. this is facilitated by the common experience repository, containing the accumulated sum of all strategy development (and follow-up) exercises, the entire contents of which must conform to a common conceptual schema. a key function of the dss is its use as a guide, providing useful advice to strategy developers at each process step. it also encompasses the basic functionality required to support all strategy development process steps. in the remainder of this section, we provide an overview of this functionality, along with a description of the system architecture and its models. the discussion is broken down consistent with the process steps specified in figure 2. 3 for example, time series data produced by a system dynamics simulation may be input to a rule-based expert system. australasian journal of information systems mcgrath & lipman 2016, vol 20, design research for decision support systems development green growth tourism dss 11 6.2 benchmarking in figure 5, an abstracted view of the core of the benchmarking guide model is illustrated. the model is presented in entity-relationship (er) form (chen, 1976) and the modelling approach utilised is based on the ‘resources-events-agency’ approach proposed by mccarthy (1982). figure 5: abstracted core of the benchmarking guide model. benchmarking is primarily concerned with measuring activities. in an activity, a party uses a resource, resulting in some outcome. examples are: a hotel using energy and producing ghg emissions; a resident using transport and, again, producing ghg emissions; and a visitor consuming food and producing waste. parties, resources, activities and events or all related to themselves in specific involvement roles. a particularly important involvement role is the subtype. examples are presented in figure 6, where it can be seen that instances of all fundamental entities are related to each other in subtype hierarchies. thus, provided measures at the lowest level of the hierarchy are captured, measures at higher levels can be easily derived using (for example) prolog code (bratko, 1986) such as: supertype(entityx, entityy) : ppi(entityx, entityy, subtype) ; rri(entityx, entityy, subtype). supertype(entityx, entityy) : ppi(entityz, entityy) ; rri(entityz, entityy), supertype(entityx, entityz). thus, using the above and the set of assertive facts presented in figure 6 (plus the corresponding set of actual instances for a particular destination), we could automatically derive higher-level measures such as total energy usage for all accommodation operators and overnight visitors, carbon-intensive energy (cie) usage for all tourism visitors etc. note that, should we like to employ different hierarchies (e.g. separating accommodation enterprises into hotels, motels and resorts), no application recoding is necessary: all that is required is to outcome ooir ooi resource rrir rri aair aai activity party ppippir australasian journal of information systems mcgrath & lipman 2016, vol 20, design research for decision support systems development green growth tourism dss 12 modify our database entries. this demonstrates one of the major advantages of the abstracted modelling approach: more convenient maintenance (feldman and miller, 1986). figure 6: sample benchmarking guide instances. the benchmarking process though, can be abstracted further. specifically, each activity must be measured (at the lowest level); either to assess activity resource consumption or outcome production (see figure 4). this meta-process is illustrated in figure 7, where it can be seen that each roadmap exercise involves many benchmarking measures and, in turn, each measure must be associated with one usage instance. each usage instance applies to one activity, one period and one usage direction (input or output), and there is one derivation algorithm that applies to each measure (which, inversely, may apply to many usage measures). finally, there is a data sources set linked to each derivation algorithm, which is an instance of one of the three generic approaches (bottom-up, top-down or combination) identified by becken and patterson (2006). basically, the bottom-up approach uses detailed activity data while, with the top-down approach, estimates are made from macroeconomic data. for example, road transport ghg emissions measures would employ actual data on distances travelled by individuals or vehicles in a bottom-up approach but might (for example) use data on tourism fuel purchases, plus shares of destination-wide fuel usage and known ghg fuel-type emission factors, when estimating top-down. once appropriate benchmark data has been collected, it is then used in later strategy construction stages: particularly projections/forecasts and the development of adaption, mitigation and resilience policies and procedures. australasian journal of information systems mcgrath & lipman 2016, vol 20, design research for decision support systems development green growth tourism dss 13 figure 7: abstracted view of the benchmarking activity measuring process. 6.3 projections and forecasts once adequate benchmarking data has been gathered in a roadmap exercise, this can be utilised in the projections and forecasts phase of strategy development: a task that often involves simulation and ‘what-if’ style testing of generated, possible future scenarios. within gets, most projection and forecasting functionality applications implemented to date have been specified using system dynamics (sd) constructs and software. with a sd approach, models are generally initially specified using ‘causal-loop-diagrams’ (clds) (manni and cavana, 2000). an example of a cld model concerned with energy supply and cost is presented in figure 84. at level 1, high energy demand leads to greater energy usage (and vice versa) and greater usage, in turn, leads to a diminished energy supply. if energy supply is low though, this will probably lead to an increase in energy cost and this may have a consequent negative impact on economic activity. this leads us back to our starting point. note though that there are additional constructs at the bottom of level 1: specifically, if the cost of energy is high, companies will be more inclined to involve themselves in increased energy exploration and, in turn, one would hope (and probably expect) that this will ultimately increase the energy supply. aspects of this model may be decomposed further however: specifically, the relationship between traditional carbon-intensive energy (cie) (oil, coal and natural gas) and renewable energy (re) (hydro, wind, biomass, waste etc.) needs to be taken into account. the cld in level 2 is the result of breaking down the variable, energy cost, into greater detail. the total energy cost is a function of both the cie cost and re cost. a high cie cost may result in more re research, leading to greater re supply. this may reduce cie demand and, in turn, also reduce cie supply. finally, less supply may make cie more expensive. this logic, roughly speaking, underpins the many carbon pricing and ets schemes currently being introduced worldwide (see e.g. callan et al., 2009). 4 an arrow indicates a causal connection and may be annotated with a ‘+’ or ‘-‘. the former means that both variables move in the same direction while the latter means they move in opposite directions. data sources set generic approach benchmarking measure usage derivation algorithm roadmap exercise activity period australasian journal of information systems mcgrath & lipman 2016, vol 20, design research for decision support systems development green growth tourism dss 14 figure 8: cld for energy supply and demand forecasting. in sd both time and feedback loops are important. in our example: i) re research may well escalate as a result of increasing cie prices but this will probably not be immediate; and ii) re research → re supply → cie demand → cie supply → cie cost → re research is a classic feedback loop. time dependencies (e.g. delays) and feedback loops are common in tourism models (see e.g. ritchie and crouch, 2003, pp60-78) and richardson and pugh (1981) have noted that sd is ideally-suited to this type of modelling exercise. as noted previously, a further strength of clds is their simplicity and this facilitates modelling sessions where key problem stakeholders can fully participate. generally, however, translation to stock-flow form is undertaken by expert modellers, proficient in the particular sd modelling and simulation package employed. this process is more complex but it does allow the impacts of major external incidents to be factored into models (as well as providing the capacity to deal with phenomena such as queues, delays, step functions and other discrete events). the reader interested in this is referred to mcgrath et al. (2015). this paper contains an example demonstrating how time series data generated by a sd simulation may be exported to other applications via the conceptual schema. 6.4 adaptation, mitigation and resilience in section 6.3 the gets forecasting modules were described: functionality that relies on information captured, structured and classified during benchmarking (see section 6.2). basically, much of the forecasting process stage involves evaluating tactics, strategies and policy changes aimed at mitigating (or adapting to) the worst impacts of potential, critical problems and issues exposed during gg strategy development. these tactics etc. however, need to be identified and this is accomplished using a combination of rule-based and casebased reasoning. development of this gets component builds upon previous research (pornphol and mcgrath, 2011) aimed at constructing a dss based upon the ‘tourism area life cycle’ (talc). according to talc theory (butler, 1980), destinations evolve through a number of stages. highlydeveloped destinations tend to experience a number of problems associated with high visitor numbers and over-exploitation of resources (e.g. environmental degradation). at this point, a destination may stagnate and go into decline or it may embark upon a rejuvenation plan. various heuristics are associated with rejuvenation strategies and these, along with a collection of actual rejuvenation cases, were ported into the gets gg experience repository (see figure australasian journal of information systems mcgrath & lipman 2016, vol 20, design research for decision support systems development green growth tourism dss 15 4). as new strategy development exercises are undertaken, new case material is added to the experience repository and existing rules, taxonomies, data structures etc. are modified in light of new knowledge gained. furthermore, stakeholders in previous destinations where gets has been used are able to access much of this new knowledge and may also take advantage of a virtual collaboration space (again, see figure 4) to discuss latest developments with each other (and also, perhaps, to offer advice relevant to the current roadmap exercise in light of their own post-strategy development experiences). this iterative knowledge evolution process is illustrated in the remainder of this (sub) section. adaptation, mitigation and resilience functionality is based largely on a case-based reasoning (cbr) approach. with cbr (kim, 2004), users are gradually and iteratively guided towards a solution from a range of options (previous cases). beemer and gregg (2008) have argued that the case-based approach is more effective than the traditional approach to expert systems development based upon rules elicited from experts. in part, this seems to be due to the fact that users seem to find descriptions of the actual application of successful strategies and methods (i.e. actual cases) more convincing than logic-based justifications of why combinations of rules are correct (hammond, 1989). part of the cbr functionality user interface is presented in figure 9. users are required to rate various, key parameters related to the economic, environmental and social health of the destination they are concerned with. collectively, these parameter assignments define a new case. the new case is then compared to existing cases, with a similarity measure5 being calculated for each of these. existing cases are then ranked in order of their similarity measures and the user may then access case summaries in turn (normally, of course, starting with the most similar cases). a link at the bottom of the webpage to a complete description of the case is provided (specifically, a copy of the paper cited in the summary). figure 9: user interface for cbr functionality. 5 briefly, for each parameter, the distance between the new case and the existing case is calculated. these are then summed and normalized, so that the result is a value between 0 and 1, with higher values indicating greater similarity. it must be noted that measures are dependent on parameter assignments and these, of course, are (with some exceptions) subjective. the key to accuracy is getting input from as wide a range of experts within a destination as possible. australasian journal of information systems mcgrath & lipman 2016, vol 20, design research for decision support systems development green growth tourism dss 16 concerning types of rejuvenation strategies, various tourism researchers have suggested classification schemes and frameworks for these. choy (1992) divides strategic responses to decline into life-cycle extension and new product development, cooper (1995) sees the choice of strategic options as being determined by a destination’s competitive position in the market place and its life-cycle characteristics, and agarwal (2002) suggests that common elements across strategies can be classified as either product reorganization or product transformation. oft-cited examples of (what so far appear to be) successful rejuvenation attempts include: i) the introduction of casinos at atlantic city (stansfield, 1978); ii) convention centres and a focus on business events at brighton and bridlington in the uk and boston and waikiki in the usa (cooper, 2006); iii) new attractions – such as aquaria at rhyl, wales and long beach, usa (harmon, 1999) and theme parks and water parks at acapulco in mexico and port aventura in spain (curtis, 1997) – to update tired resorts; and iv) coordinated action by regional and national tourism authorities in wales (wales tourist board, 2000) and ireland (irish tourist board, 2001). increasingly, rejuvenation strategies appear to have a strong environmental quality enhancement focus (see e.g. jordan, 2000; agarwal, 2002; smith, 2002) and this is precisely why talc and gg research overlap so much. a very good recent example of a gg strategy development exercise is that undertaken in sharm el-sheik (law et al., 2012), a popular egyptian diving and water sports resort, with excellent beaches, a coral reef and diverse marine life. local authorities have determined that, if things continue as they are6, the destination will not be sustainable as a tourist resort in the medium-to-long term (elfrefaie, 2007). consequently, they have embarked upon an aggressive environmental enhancement initiative consisting of the following 10 ‘green sustaining and enabling programs’ (booz, 2009): i) green air access; ii) green energy infrastructure; iii) operational efficiency improvement for hotels; iv) operational efficiency improvement for other buildings; v) green building design; vi) green land transport; vii) green water transport; viii) resilient water supply; ix) effective waste management; and x) conservation of bio-diversity. these 10 initiative types provide quite a useful framework for classifying cases of rejuvenation actions with an environmental enhancement component. the problem is, however, that the types are not mutually exclusive and are linked to each other in a variety of ways. this complexity is magnified when dealing with specific actions within types, as higher-level relationships may or may not be inherited. consequently, a reasonably complex ontology is needed to assist in guiding the search for nearest cases when either producing a ‘best match’ report for a new case or searching for appropriate gg transformation strategies. for a more detailed discussion of this ontology, the reader is referred to pornphol and mcgrath (2011). the rule-based and cbr approaches though, are by no means incompatible and, in our implementation, we employ rules to supplement cbr similarity matches in our search for the most relevant cases. a more complete account of this has been detailed by pornphol and mcgrath (2011) but one important outcome was the decision table presented in figure 10. this table was employed as the basis for an initial attempt at the implementation of a module designed to guide users towards the type of rejuvenation strategies that might be appropriate for their destinations. this module was implemented in the flex expert system shell, with each of the columns in figure 10 translated into a rule. an actual example of one of our rules (from column 5) is: 6 i.e. with an economy based primarily on low-cost, mass tourism and with relatively poor energy, water and waste management practices. australasian journal of information systems mcgrath & lipman 2016, vol 20, design research for decision support systems development green growth tourism dss 17 rule solncat04a if the stagdeclineind of solninds is greater than 0.7 and the economicproblemsind of solninds is greater than 0.7 and the envtsocialproblemsind of solninds is greater than 0.7 and the espreversalind of solninds is yes and the visdropreversalind of solninds is yes then the productmixchangeind of solninds becomes maybe and the yieldimproveind of solninds becomes easieroptions and write('improving or changing the prod mix may be an option') and nl . and, in this instance the rule suggests that, because there may be opportunities to reverse both environmental and social problems and a current drop in visitor numbers, improvement of the current product mix may be the way to go – as distinct from a radical product transformation (specifically, e.g., targeting fewer, higher-yielding visitors seeking an exotic/exclusive location). note that the ‘x’ reference in the actions section of figure 10 refers to a rule number. figure 10: decision table for dss gg strategy rule derivation. the arguments of this rule are compound variables derived from 16 parameter values, input by users and related to destination economic, environmental and social health. the actual parameters are location type, visitor category, destination development, development rate drop, environment despoilment, man-made attractions level, man-made attractions state, heritage attractions state, tourism alternatives level, seasonality, crime, vandalism, visitor antipathy, disruption, access denial, character incompatibility, lost opportunity cost, visitor numbers, visitor numbers drop, visitor yield, business visitors, profitability, profitability drop, closures, ownership turnover and facilities standard. these variables were derived from work done on evaluating tourism event success by fredline, deery and jago (2006). it is demonstrated that, as the dss evolves through a number of cycles (each cycle being the result of dss application to a new destination), our knowledge of the gg strategy development conditions 1 2 3 4 5 6 7 8 9 stagnation/decline? n y y y y y y y y economic problems? n y y y y y y y envt/social problems (esp)? n y y y y y y esp reversal if vis. no's increase? n y y y n n esp reversal if vis. no's decrease? y n n increase yield if esp reversal? y current facilities and products poor? y n y/n n n high seasonality? y n n economic alternatives to tourism? y n actions nfa x (1a) warning signs take care x (2a) product transformation move to exotic/exclusive x (5) product reorganisation improve current product mix x (3b) x (4a, 6) product reorganisation add to current product mix x (4a, 6a) decrease seasonality via product reorganisation x (6c) move to tourism alternatives x (6d) ?????? x (6e) cases australasian journal of information systems mcgrath & lipman 2016, vol 20, design research for decision support systems development green growth tourism dss 18 domain is enhanced and refined in much the same way as would occur with a more conventional case study approach. 7 demonstration and evaluation these two phases are closely related. according to peffers et al. (2007-08, p54), demonstration involves: “---use of the artifact to solve one or more instances of the problem.” while evaluation: “---involves comparing the objectives of a solution to actual observed results from the use of the artifact in the demonstration.----at the end of this activity the researchers can decide to iterate back to step three to try to improve the effectiveness of the artifact or to continue on to communication.” over the past six years, gets has been successfully employed to assist with gg strategy development in the field at two locations, sharm el-sheik, egypt and bali, indonesia. each of these exercises was undertaken as a research case study, the details of which (including use of the dss) have been reported by law et al. (2012), lipman et al. (2012), lipman et al. (2013), delacy, jiang, lipman and vorster (2014) and mcgrath et al. (2015). during and after each of these studies, additional functionality was added to the system. this was done without the need to undertake extensive modifications of existing system applications, thereby demonstrating the realization of a number of key objectives detailed in section 6: namely, convenient maintenance, knowledge sharing between applications and studies, and reuse of functionality. it is anticipated that the dss will be extended further during proposed strategy development exercises in china (qiqihar province), australia (alpine shire, victoria) and belgium (genk) to be undertake during the period 2016-18 (delacy et al., 2015). in a related, further important initiative, gets is the key software component of “the sun program”, launched in december 2015 by greenearth travel (tjolle, 2015). this is a global initiative designed to help travel & tourism destinations develop through sustainable green growth. aligned with the post-2015 climate and sustainability agenda, sun helps communities achieve climate resilience, the sustainable development goals (sdgs) and emergency responses (to climate-induced and other disasters). linking dedicated solar powered sun-arks, through the internet cloud and manned by trained local post-graduates, each sun-ark is compact, efficient and low cost and facilitates the engagement of public, private, academic and civil society partners in market-responsive, inclusionary, long term change. sun’s single goal is to help communities and their stakeholders move along the green growth and travelism adaptation path, including: i) understanding the system dynamics and focusing on targets; ii) monitoring numbers and analytics; iii) helping local policy makers/stakeholders stay on track; iv) seeking out and building upon replicable good practice, innovation and investment cases; v) linking local culture with green growth and climate resilience; and vi) serving as a focal point for response to emergencies it is intended that the gets model will be available to destinations joining the ‘sun-ark community’. the first sun-ark was announced for april 2016 in limburg in belgium as part of the national park climate response, science project in conjunction with the university of hasselt and victoria university melbourne. 8 communication the task here (peffers et al., 2007-08: 55) is to: “communicate the problem and its importance, the artifact, its utility and novelty, the rigor of its design, and its effectiveness to researchers and other relevant audiences.” australasian journal of information systems mcgrath & lipman 2016, vol 20, design research for decision support systems development green growth tourism dss 19 as detailed in the previous section, the gets dss has been discussed in a number of scholarly papers over recent years (specifically law et al., 2012; mcgrath et al., 2012; mcgrath et al., 2015). it is anticipated that further papers will flow from the case studies detailed in section 7. these may well focus on additional functionality to be built during these studies and consequent issues of knowledge sharing, knowledge reuse and system maintenance, modification and enhancement. 9 conclusion we have described the construction of a gg strategy development dss using a classical design science approach. while it is now close to six years since system construction commenced, the system continues to undergo major modification and enhancement, particularly on the occasions it has been used to support major strategic planning exercises in the field. significantly, to date, enhancements have been accomplished without the need to make major changes to existing code or schemas, thereby providing some evidence that reuse and maintenance objectives have been realized. furthermore, application modules have been developed using system dynamics, rule-based expert systems, cbr, prolog and excel platforms, with data being shared between all modules in a way that is invisible to users. thus, seamless integration and data sharing objectives have been realized as well. with each new strategic planning exercise, the dss knowledge repository has grown, with the addition of new cases (and supporting case material) constituting a particularly valuable practitioner and researcher resource. thus, it is probably fair to say that the dss now has value beyond its original purpose (i.e. supporting the production of individual, destination-specific, gg strategy roadmaps see section 4). we believe that integration into the sun program will provide a clear development trajectory to help communities better meet their post-2015 agenda targets (sdg and climate) through gg and travelism. with design science, the problem statement drives objectives and system design. the fact that, with time, use of a system does not entirely match its specified, formal problem statement does not invalidate its design and construction. rather, carlsson and turban (2002) have identified the issue of designing dss for flexible use, through separation of data and application processes, as one of four main areas urgently in need of design science research. this, outside of any tourism domain-specific significance, constitutes the major contribution of our research. references argarwal, s. (2002). “restructuring seaside tourism: the resort life-cycle”, annals of tourism research, 29(1), 25-35. arnott, d. and pervan, g. (2008). “eight key issues for the decision support systems discipline”, decision support systems, 44, 657-672. baggio, r. (2011). “the mechanism for spreading online reputation”, academica turistica – tourism and innovation journal, 23(2), 7-15. balbi, s., perez, p. and giupponi, c. (2010). “a spatial agent-based model to explore scenarios of adaptation to climate change in an alpine setting”, proceedings of the 5th biennial international congress on environmental modelling and software, july 2010, ottawa, canada, 700-707. beemer, b.a. and gregg, d.a. (2008). “advisory systems to support decision making”, handbook on decision support systems, springer, berlin, 511-528. booz (2009). green sharm: strategic directions for green transformation, client presentation, booz and co. inc. bramwell, b. and lane, b. (1993). “sustainable tourism: an evolving global approach”, journal of sustainable tourism, 1(1), 1-5. australasian journal of information systems mcgrath & lipman 2016, vol 20, design research for decision support systems development green growth tourism dss 20 bratko, i. (1986). prolog programming for artificial intelligence, adison-wesley, wokingham, uk. brundtland, g.h. (1987). our common future brundtland report, oxford university press, oxford, uk, 204. buchanan, r. (1992). “wicked problems in design thinking”, design issues, 8(2), 5-21 butler, r.w. (1980). “the concept of a tourism area cycle of evolution: implications for management resources”, the canadian geographer, 24(1), 5-16. butler, r.w. (ed.) (2006). the tourism area life cycle (vol.2): conceptual and theoretical issues, channel view, clevedon, uk callan, t., lyons, s., scott, s. and tol, r.s.j. (2009). “the distributional implications of a carbon tax in ireland”, energy policy, 37(2), 407-412. carlsson, c. and turban, e. (2002). “dss: directions for the next decade”, decision support systems, 33, 95-110. chen, p.p.s. (1976). “the entity-relationship model: towards a unified view of data”, acm transactions on database systems, 1(1), 9-36. choy, d.j.l. (1992). “life cycle models for pacific island destinations”, journal of travel research 30(3), 26-32. cooper, c.p. (1995). “strategic planning for sustainable tourism: the case of the offshore islands in the uk”, journal of sustainable tourism, 3(4), 191-209. cooper, c. (2006). “the anatomy of the rejuvenation stage of the talc”, in (ed. r.w. butler), the tourism area life cycle: conceptual and theoretical issues, channel view publications, clevedon, uk, 183-200. curtis, b., kellner, m.i. and over, j. (1992). “process modelling”, communications of the acm, 35(9), 75-90. curtis, r.j. and campbell, d.e. (2015). “cybersecurity requires a clear systems engineering approach as a basis for cyberstrategy”, in (ed. j-l. richet), cybersecurity strategies for cyberwarfare prevention, igi global, hershey park, pa, pp. 102-120. curtis, s. (1997). “seaside resorts: spanish progress and british malaise”, in insights (c9-18), british tourism authority, london. delacy, t., jiang, m., lipman, g. and vorster , s. (eds) (2014). green growth and travelism concept: policy and practice for sustainable tourism, routledge, abingdon, uk. elrefaie, m. (2007). an integrated approach for sustainability assessment of tourism development in sharm el-sheikh, available at http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.134.9964&rep=rep1&type= pdf [accessed 15 october 2015]. feldman, p. and miller, d. (1986). “entity model clustering: structuring a data model by abstraction”, the computer journal, 29(4), 348-360. fredline, l., deery, m. and jago, l. (2006). development of a scale to assess the social impact of tourism within communities, sustainable tourism crc technical report, stcrc, gold coast, australia. g20 (2012). g20 leaders declaration, g20 meeting, los cabos, mexico, 18-19 june, 2012. georgantzas, n.c. (2003). “tourism dynamics: cyprus’ hotel value chain and profitability”, system dynamics review, 19(3), 175-212. goes, p.b. (2014). “design science research in top information systems journals”, (editor’s comments), mis quarterly, 38(1), pp.iii-viii. australasian journal of information systems mcgrath & lipman 2016, vol 20, design research for decision support systems development green growth tourism dss 21 gossling, s. and peeters, p. (2015). “assessing tourism’s global environmental impact 19902050”, journal of sustainable tourism, 25(3), 639-659. graham, r.l, knuth, d.e. and patashnik, o. (1990). concrete mathematics (2nd edition), addison-wesley, reading, ma. gregor, s. (2006). “the nature of theory in information systems”, mis quarterly, 30(3), 611642. gregor, s. and jones, d. (2007). “the anatomy of a design theory”, journal of the association of information systems, 8(5), 312-335. gregor, s. and hevner, a.r. (2013). “positioning and presenting design science research for maximum impact”, mis quarterly, 37(2), 337-355. hammond, k.j. (1989). case-based planning: viewing planning as a memory task, academic press, boston, usa. harmon, m. (1999). long beach, successful meetings, available at http://global.umi.com/pqdweb?ts=98...&sid=7&idx=10&deli=1&rqt=309&dtp=1 [accessed 16 october 2015]. hawkins, r. and bohdanowicz, p. (2011). responsible hospitality: theory and practice, goodfellows, oxford, uk. hevner, a.r., march, s.t. and park, j. (2004). “design research in information systems research”, mis quarterly, 28(1), 75-105. inskeep, e. (1991). tourism planning: an integrated and sustainable development approach, von nostrand reinhold, new york. irish tourist board (2001). available at http://www.ireland.travel.ie/aboutus/tourismfacts.asp [accessed 16 october 2015]. jordan, p. (2000). “restructuring croatia’s coastal resorts: change, sustainable development and the incorporation of rural hinterlands”, journal of sustainable tourism, 8(6), 525-539. kim, k.j. (2004). “toward global optimization of case-based reasoning systems for financial forecasting”, applied intelligence, 21(3), 239-249. kowalski, r.a. (1979). logic for problem solving, north-holland, new york. lane, b. (1994). “sustainable rural tourism strategies: a tool for development and conservation”, journal of sustainable tourism, 2(1-2), 102-111. law, a. (2015). a green economy indicator framework for tourism destinations: the case study of bali, indonesia, unpublished phd thesis, college of business, victoria university, melbourne. law, a., delacy, t., mcgrath, g.m., whitelaw, p.a., lipman, g. and buckley, g. (2012). “towards a green economy decision support system for tourism destinations”, journal of sustainable tourism, 20(6), 823-843. le blanc, d. (2015). “towards integration at last? the sustainable development goals as a network of targets”, desa working paper no. 141, st/esa/2015/dwp/141, division of sustainable development, united nations department of economic and social affairs (un-desa). linthicum, d.s. (2003). next generation application integration: from simple information to web services, addison-wesley, boston. lipman, g., delacy, t., vorster, s., hawkins r. and jiang, m. (2012). green growth and travelism; letters from leaders, goodfellow publishers, oxford, uk. australasian journal of information systems mcgrath & lipman 2016, vol 20, design research for decision support systems development green growth tourism dss 22 lipman, g., delacy, t. and whitelaw p. (2013). “competitiveness, jobs and green growth: a “glocal” model, in (blanke, j, and chiesa, t. eds.), the travel & tourism competitiveness report 2013: reducing barriers to economic growth and job creation, world economic forum, geneva, 71-77. liu, s., duffy, a.b., whitfield, r.i. and boyle, i.m. (2009). “integration of decision support systems to improve decision support performance”, knowledge and information systems, 22(3), 261-286. mccarthy, w.e. (1982). “the rea model: “a generalized framework for accounting systems in a shared data environment”, the accounting review, 57(3), 554-578. mcgrath, g.m. (2007). “towards improved business planning decision support for small-tomedium tourism enterprise operators”, (in w. pease, m. rowe and m. cooper eds.), information and communication technologies in support of the tourism industry, idea group, hershey, pa, 215-241. mcgrath, g.m., law, a. and delacy, t. (2012). "decision support for the development of green growth tourism strategies: managing complexity", journal of knowledge management practice, 13(4). mcgrath, g.m., meijerink, h. and gutterson, b. (2014). “an australia-china joint research project: it gaps and needs in support of ‘green tourism’ initiatives”,proceedings of the 28th australian and new zealand academy of management conference (anzam2014), 3-5 december, sydney, 1-14. mcgrath, g.m., law, a. and delacy, t. (2015). “green economy planning in tourism destinations: an integrated, multi-method decision support aid”, journal of developing areas, 49(6), 145-155. maani, k.e. and cavana, r.y. (2000). systems thinking and modelling: understanding change and complexity, prentice-hall, auckland, new zealand. march, s. and smith, g. (1995). “design and natural science research on information technology”, decision support systems, 15, 251-266. markus, l., majchrzak and gasser, l. (2002). “a design theory for systems that support emergent knowledge processes”, mis quarterly, 26(3), 79-212. merton, r. (1968). social theory and social structure (enlarged edition), the free press, new york. nunamaker, j.f., chen, m. and purdin, t. (1991). “systems development in information systems research”, journal of management information systems, 37(3), 89-106. peffers, k., tuunanen, t., rothenberger, m.a. and chatterjee, s. (2007-08). “a design research methodology for information systems”, journal of management information systems, 24(3), 2007-08, 45-77. pornphol, p. and mcgrath, g.m. (2011). "information systems development as an emergent knowledge research process", international journal of interdisciplinary social sciences, 6(1), 301-317. richardson, g.p. and pugh, a.l. (1981). introduction to system dynamics modeling, productivity press, oregon, usa. ritchie, j.r.b. and crouch, g.i. (2003). the competitive destination, cabi publishing, cambridge, ma. scott, d., amelung, b., becken, s., ceron, j. p., dubois, g. and gössling, s. (2008). climate change and tourism: responding to global challenges, world tourism organization (unwto), united nations environment programme (unep), madrid, paris. australasian journal of information systems mcgrath & lipman 2016, vol 20, design research for decision support systems development green growth tourism dss 23 scott, d., defreitas, c. and matzarakis, a. (2009). “adaptation in the tourism and recreation sector”, in (k.l. ebi, i. burton and g.r. mcgregor eds.) biometerology for adaptation to climate variability and change, kluwer, dordrecht, netherlands, 171-194. scott, d, gossling, s., hall, c.m., and peeters, p. (2015). “can tourism be part of the decarbonized global economy? the costs and risks of alternate carbon reduction policy pathways”, journal of sustainable tourism, 24(1), 52-72. senge, p.m. (1990). the fifth discipline: the art and practice of the learning organization, random house, milsons point, nsw. simon, h. (1969). the sciences of the artificial, mit press, cambridge, ma. simpson, m.c., gossling, s., scott, s., hall, c.m. and gladin, e. (2008). climate change adaptation and mitigation in the tourism sector: frameworks, tools and practices, united nations environment program (unep), madrid smith, m. (2002). “two parallel paths to a common goal? a critical analysis of sustainable tourism and cultural regeneration in the context of english seaside towns”, tourism research 2002, 4-7 september, cardiff. soboll, a. and schmude, j. (2011). “simulating tourism water consumption under climate change conditions using agent-based modeling: the example of ski areas”, annals of the association of american geographers, 101(5), 1049-1066. stansfield, c.a. (1978). “atlantic city and the resort cycle: background to the legalization of gambling”, annals of tourism research, 5(2), 238-251. tjolle, v. (2015). “sun launches soft power think tank”, travelmole news – 30/12/2015, available at: http://www.travelmole.com/news_feature.php?news_id=2019139 [accessed 30/12/2015]. unep (2010). green economy report, towards a green economy: pathways to sustainable development and poverty eradication, available at: http://www.unep.org/greeneconomy/v2/greeneconomyreport/tabid/29846/default. aspx [accessed 23/12/2011]. unwto (2015). unwto tourism highlights, 2015 edition, available at unwto: elibrary: http://publications.unwto.org/ [accessed 30/12/2015]. van griethuysen, j.j. (1982). “concepts and terminology for the conceptual schema and the information base”, iso technical report iso/tc9/sc5/wg3, iso, geneva, switzerland. vennix, j.a.m. (1996). group model building: facilitating team learning using system dynamics, wiley, chichester, uk. wales tourist board (2000). annual report 1999-2000, available at: http://www.walestourist-board.gov.uk/attachmrnts/1.pdf [accessed 30/12/2015]. walker, p.a., greiner, r., mcdonald, d. and lyne, v. (1999). “the tourism futures simulator: a systems thinking approach”, environmental modelling and software, 14, 59-67. walls, j, widemeyer, g. and el sawy, o. (1992). “building an information system design theory for vigilant eis”, information systems research, 3(1), 36-59. wilson, j. (2002). “responsible authorship and peer review”, science and engineering ethics, 8(2), 155-174. wttc (2012). the comparative economic impact of travel and tourism, world travel and tourism council, oxford, uk. australasian journal of information systems mcgrath & lipman 2016, vol 20, design research for decision support systems development green growth tourism dss 24 copyright: © 2016 mcgrath & lipman. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 background 3 research approach 4 problem identification and motivation 5 solution objectives 6 design and development 6.1 overview 6.2 benchmarking 6.3 projections and forecasts 6.4 adaptation, mitigation and resilience 7 demonstration and evaluation 8 communication 9 conclusion the australasian journal of information systems (ajis) began in 1993 and was based on the simple realisation that there was a need for an academic ‘outlet’ for is research in australia australasian journal of information systems volume 18 number 2 2014 59 the relationship between organisational strategic it vision and cio roles: one size does not fit all moyassar al-taie university of mosul iraq university of southern queensland moyassar_z_a@yahoo.com michael lane university of southern queensland michael.lane@usq.edu.au aileen cater-steel university of southern queensland aileen.cater-steel@usq.edu.au abstract the organisational strategic it vision has been considered as a key contingency that affects the role of information technology (it) leadership in organisations. using the theoretical lens of the contingency approach to leadership, this study investigates the influence of strategic it vision of an organisation on the chief information officer (cio) role and structural power. a large-scale survey was conducted with cios of australian organizations. the results of the data analysis show that the educator role of the cio seems to be more important in organisations that articulated a ‘transform’ vision of it compared to organisations that focus on it for automation. the results also show that cios in organisations that strategically view it as a transformation or informate-down orientation gave the role of information steward more attention than cios working in organisations where it is viewed as a means to automate processes. the findings also show that there is significant positive association between the organisation’s strategic it vision and the cio’s structural power in terms of reporting structure and cio job title. this study has implications for practitioners as the findings indicate the necessity for cios to align their roles with their firm’s it vision and suggest that chief executive officers (ceos) should empower their firm’s cios in terms of their reporting structure and job title as the role of it in the organisation progresses from the lower strategic vision of it (automate) to the highest vision (transform). keywords: contingency approach to leadership, organisational strategic it vision, chief information officer (cio) role, cio’s reporting structure, cio’s job title, australia. introduction the strategic information technology (it) vision is a key contingency affecting the role of it leadership in organisations (feeny, edwards, and simpson 1992; kaarst-brown 2005; preston and karahanna 2009; smaltz , sambamurthy, and agarwal 2006; smaltz 1999). the literature indicates that chief information officers (cios) play multiple roles in their leadership of it in an organisation (agarwal and beath 2007; earl 1996; ibm 2009; mclean and smits 1993; peppard, edwards, and lambert 2011; smaltz et al. 2006). these multiple roles can be grouped into operational (supply side) roles and strategic (demand side) roles (broadbent and kitzis 2005; chen, preston, and xia 2010; mark australasian journal of information systems volume 18 number 2 2014 60 and monnoyer 2004). prior studies have acknowledged that it is viewed by organisations in different ways and used for different purposes (dahlbom and mathiassen 1997; kaarst-brown 2005; marchand 2007; mcfarlan, mckenny, and pyburn 1983; schein 1989, 1992; spitze and depaschalis 2005). an extensive literature review indicates that few studies have examined the relationship between the strategic vision of it in organisations and the multiple roles of the cio in an organisation. while smaltz (2000) and li, ding, and wu (2012) found that cios in organisations that viewed it as a transformation tool perceived their strategic cio roles to be the most important, periasamy and seow (1998) reported conflicting points of view regarding the role of the cio working in firms that espouse an ‘automate’ view of it. other studies have failed to identify any significant effect of the organisational it vision on cios’ effectiveness in their roles (brown 2006 ; smaltz 1999). one study, conducted by grover, jeong, kettinger, and lee (1993) found that as the use of it matures, the strategic role of the cio does not become more important. there is also a lack of empirical research that has investigated the direct impact of the organisational view of it on the cio structural power in an organisation. investigating the structural power of the cio is crucial as the legitimacy of this executive manager has not been fully established in many organisations (kaarst-brown 2005). the lack of previous academic research in this area and the inconsistency in the results of previous empirical studies provides the justification for further investigation of the association between the strategic vision of it in organisations on one hand and the role of the cio and the structural power of the cio job position in organisations on the other hand. this study takes a novel approach to provide better understanding of the relationship between the strategic it vision of the organisation and the cio roles and their structural power. based on the contingency approach to leadership, it is anticipated that the vision of it for an individual organisation requires a cio to adopt an appropriate configuration of roles that is aligned with the it vision of an organisation. the mismatch between the it vision of an organisation and the cio roles could result in dire effects. for example, weak alignment between it and the firm’s strategy can lead to a reduction in the business value of it, a decrease in the effectiveness of the cios, the corruption of the relationship between the cio and the chief executive officer (ceo). finally, the misalignment of the current cio roles with the it vision of an organisation could threaten the survival of the cio position in an organisation. therefore, the cio who is adopting the right roles and is given the required structural power to implement the vision of it in an organisation is more likely to succeed and survive in the job. hence, this study is important because it aims to investigate the relationship between the organisational it vision and the configuration of cio roles and structural power of a cio in an organisation. the general research question investigated in this paper is as follows: is the configuration of cio roles and structural power of a cio associated with an organisation’s strategic vision of it? this paper is structured as follows. first the relevant literature regarding the strategic it vision in organisations and the cio role is reviewed and a set of hypotheses is formulated from the literature. next, the research methodology used in this study is described and justified. after that, the results of the data analysis of the survey responses of 162 cios are presented and discussed. finally, implications of the key findings for existing theory and practice are discussed and some suggestions for future research are provided. australasian journal of information systems volume 18 number 2 2014 61 literature review the contingency approach to leadership since 1950s, empirical studies tend to show that a leader is required to behave in a flexible manner in an organisation, and needs to adopt the appropriate leadership style that fits with the situation (tannenbaum and schmidt 1957; fiedler 1967; house and mitchell 1974; vroom and yetton 1973; hersey and blanchard 1993). the contingency approach to leadership (also known as situational approach) proposes that different leaders’ behaviours are effective in different situations. therefore, leadership from the contingency approach perspective is a dynamic process that implies different roles, style, or behaviour in different organisational situations. these perspectives refute the notion of “one size fits all” as there is no optimal leadership style appropriate to all situations. early research regarding the it leadership conducted by rockart, ball, and bullen (1982) emphasized the importance of studying the organisational contingencies that shape the cio role in an organisation. it is anticipated that studying the cio role without considering the organizational context can result in misunderstanding that crucial role. this study investigates how different organisational visions of it articulated by modern organisations can affect the cios’ roles and their structural power. strategic it vision strategic it vision refers to the shared, aspired state of the role that it should play in the firm (robbins and duncan 1988; zmud 1988 as cited in armstrong and sambamurthy1999). it is worth mentioning that different names have been used interchangeably for this construct such as the ceo’s basic assumption about it (kaarst-brown 2005; schein 1989, 1992); is/it orientation (weill and broadbent 1998; teo and too 2000); is/it role (chen, mocker, preston, and tuebner 2010; feeny et al. 1992; hallikainen, hu, frisk, päivärinta, eikebrokk, and nurmi 2006; ramakrishna and lin 2002; richardson and zmud 2002); organisation’s attitude to it (earl 1996); and the main purpose of it (weiss, thorogood and clark 2006). the previous literature summarised in table 1 highlights 10 different typologies for understanding how the role of it could be viewed in an organisation. organisational views and assumptions about the role of it references support; factory; strategic; and turnaround mcfarlan et al. 1983 exploiter/innovator; competitor/early adopter; and participant/ effective or efficient follower cash et al. 1988 automate; and informate zuboff 1985 automate; informate-down; informate-up; and transform schein 1989, 1992 automate; informate; and transform venkataraman 1991 build things; help people; and change things dahlbom et al. 1997 support tool; critical resource; means of transformation; and unclear earl 1996 utilitarian; strategic weill and broadbent 1998 necessary evil; support not a partner; it rules!; business can do it better; and equal partner kaarst-brown 2005 technical resource; business enabler; and strategic weapon weiss et al. 2006 it doesn’t matter; it pushes the business; business pushes it; and it does matter marchand 2007 table 1. literature summarymain it role typologies (source: developed for this study) australasian journal of information systems volume 18 number 2 2014 62 the strategic it vision typology (schein 1989, 1992) has been widely adopted by previous is researchers (e.g. armstrong and sambamurthy 1999; bassellier, reich,and benbasat 2001; feeny et al. 1992; hallikainen et al. 2006; sherer 2004; smaltz 2000; smaltz et al. 2006; smaltz 1999; tripp 2008). the strategic it vision of an organisation (schein 1989, 1992) can be classified into four distinct views: automate, informate-down, informate-up, and transform. a brief description of each of these views is provided in table 2. vision description purpose (feeny 1997) automate the potential of it is cost saving or quality improvement through automation. the role of it is to replace expensive, unreliable human labour, or at least transform its productivity. cost-displacement and efficiency informate-down the potential of it is to empower employee driven performance improvement. the role of it is to provide data and transaction that yield a far fuller picture at ‘operator’ level, with members of the staff gaining greater insight into their own activities. empower employees informate-up the potential of it is to increase managerial control of the organisation. the role of it is to provide data and transaction that allow management to have more clear and organized views of the state and dynamics of the organisation. increase management control transform the potential of it is to transform the organisation. the role of it is to fundamentally change the organisation and /or industry through new products and services often including redefinition of relationships with the organisation’s customers and /or suppliers. achieve radical change in some aspect of business table 2. organisational strategic views of the role of it (sources: feeny (1997); feeny et al. (1992); schein (1989); smaltz (1999)) chief information officer (cio) roles the term ‘role’ refers to an ‘organized set of behaviours belonging to an identifiable office or position’ (sarbin and alan 1968). from the literature it is evident that there are four main configurations of roles that have been developed for general managers over the last four decades: mintzberg’s managerial roles; paie (producer, administrator, entrepreneur, and integrator); cvf (competing values framework); and the integrated model of executive leadership roles. table 3 summarizes these four key typologies. australasian journal of information systems volume 18 number 2 2014 63 typology/configuration title used in literature number of roles roles label mintzberg 1973 mintzberg’s managerial roles 10 interpersonal roles: figurehead, leader, liaison informational roles: monitor, disseminator, spokesperson decisional roles: entrepreneur, disturbance hander, resource allocator, negotiator adizes 1976; adizes 2004 paei 4 producer, administrator, entrepreneur, integrator quinn, faerman, thompson, mcgrath, & clair 2006; quinn & rohrbaugh 1981; quinn & rohrbaugh 1983 competing value framework cvf 8 mentor, facilitator, innovator, broker, monitor, coordinator, producer, director hart & quinn 1993 integrated model of executive leadership roles 4 vision setter, motivator, analyser, task master table 3. key classic managerial roles typologies the cio role based on the classic managerial roles configurations since the emergence of the cio role in the early 1980s (synnott 1987; synnott et al. 1981) much has been written about it (chen and preston 2007; fisher 2003; karimi , gupta, and somers 1996; stephens, ledbetter, mitra, and ford, 1992). brown (1993) asserts that prior theories regarding managerial work and leadership are applicable to the cio role and it could be a useful base to build on provided the cio is a general manager. consequently, many researchers have attempted to apply the roles developed for general managers to the position of cio. twelve key empirical studies that adopted a classical managerial roles configuration in order to gain a better understanding of the cio role(s) are summarised in table 4. some interesting observations can be made on closer examination of the 12 empirical studies listed in table 4. first, it is confirmed that applying general managers and leadership frameworks to the cio role provides a greater understanding of the importance of cio roles. second, it appears that the cio role is evolving from focusing on communication-based roles such as spokesman and liaison (grover et al. 1993) to play a more strategic role focusing on change and innovation as an entrepreneur in the organisation (gottschalk 2000b; karlsen, gottschalk, and andersen 2002; lineman 2006; milliron 2008). furthermore, it is clear that the mintzberg (1973) framework that was used in 10 out of 12 cio roles empirical studies reviewed (carter, grover, and thatcher 2011; gottschalk 2000a, 2000b; grover et al. 1993; karlsen et al. 2002; lineman 2006; milliron 2008; stephens 1995; yang 2008), was the most widely used framework. moreover, the extensive review of the relevant literature indicated a gap in the cio literature that needs to be addressed as the vast majority of previous empirical studies examining the cio roles in organisations were based on mintzberg’s framework whereas almost none used any of the other three key management roles typologies of adizes (1976, 2004), quinn et al. (2006, 1981, 1983), and hart and quinn (1993). australasian journal of information systems volume 18 number 2 2014 64 reference methodology sample country important roles reported grover et al. 1993 quantitative survey 71 cios and is middle managers usa spokesperson and liaison roles stephen 1995 observation 5 cios from different industries usa occurrence frequency : resource allocator, monitor, and distributer handler time spent: resource allocator, entrepreneur, distributer handler, and monitor roles gottschalk 2000a quantitative survey 101 is//it leaders norway new is/it leaders more time in mintzberg’s informational roles than established is/it leaders. gottschalk 2000b quantitative survey 101 is//it leaders norway strategic responsibilities and is stage of growth influences extent of informational roles. extent chief executives uses it influences extent of decisional roles, and extent which subordinates use it influences extent of interpersonal roles. gottschalk 2002 quantitative survey 128 norway entrepreneur karlsen, gottschalk & andersen 2002 quantitative survey 128 it executives and 80 it project managers norway for it leaders: monitor for it project managers: leaders, resource allocator, and entrepreneur lineman 2005 quantitative survey 232 higher education cios usa entrepreneur, resource allocator, and leader gottschalk & karlsen 2005 quantitative – survey 80 it leaders of firms with internal it projects + 84 it leaders of firms with outsourcing projects norway in firms with internal it project: leader in firms with outsourcing it project: spokesman yang 2008 quantitative – survey it managers of financial industry taiwan liaison and spokesperson roles milliron 2008 mixed interviews + survey 10 cios of community colleges usa monitor, liaison, and entrepreneur roles tufts & jacobson 2010 q-methodology based on the cvf 8 roles survey 67 public cios usa results oriented pragmatist; compassionate managers; leading edge powerbroker; and goal oriented powerbroker carter, grover, & bennett 2011 quantitative – survey 45 cios usa monitor and spokesperson table 4. empirical cio studies based on classical managerial roles typologies australasian journal of information systems volume 18 number 2 2014 65 the cio distinctive roles configurations some information systems scholars have attempted to suggest other distinctive roles for the cio which differ from the classic managerial roles discussed in the previous section and consider the unique characteristics required for this role. this different approach to conceptualising the cio roles was based on the fact that mintzberg (1971) himself has asserted that all managers are indeed specialists and therefore their roles tend to be dependent on the functional area they lead. as part of the extensive review of the literature, 24 studies that specified configurations of cio roles over the last couple of decades are summarised in table 5. reference methodology country roles citations dixon & john 1989 conceptual usa 3 100 klenke 1993 conceptual usa 4 14 mclean & smits 1993 conceptual usa 4 5 klenke 1996 conceptual usa 4 4 csc 1996 conceptual uk 6 not available feeny 1997 qualitative (interviews) uk 4 8 earl 1998 qualitative (interviews) uk 8 32 smaltz 1999 quantitative (survey) usa 6 12 mclean & smits 2003 qualitative (interviews) usa 4 not available cash & pearlson 2004 conceptual usa 5 7 perchthold & sutton 2005 conceptual usa 3 not available sojer, schlager, & locher 2006 quantitative (surveys) usa 4 7 smaltz , sambamurthy,& agarwal 2006 quantitative (survey) usa 6 88 agarwal & beath 2007 qualitative (interviews) usa 7 5 tansley, loughran. edwards, lammert, & peppard 2008 qualitative (semistructured interviews) uk 5 not available weiss & adams 2010 mixed (online survey+ interviews) usa 3 2 wu, chen, & sambamurthy 2008 quantitative (survey) taiwan & china 8 6 ibm 2009 qualitative (interviews) global 78 countries 3 pairs not available chun & moony 2009 mixed (secondary data + interviews) usa 4 36 cio magazine 2009 quantitative (survey) usa 3 not available peppard, edwards, & lambert 2011 qualitative (semistructured interviews) uk 5 4 chen & wu 2011 quantitative (survey) taiwan & china 8 5 nicolet 2011 mixed (survey + interviews) usa 6 0 mclean & smits 2012 qualitative (interviews) usa 4 0 table 5. summary of previous studies of cio key roles configurations australasian journal of information systems volume 18 number 2 2014 66 several important conclusions can be drawn from these 24 studies of the role configurations of cios. first, the cio role configurations studies can be classified as conceptual (n=7) and empirical (n=17). the vast majority of the early research studies on the cio role presented in table 5 were conceptual. the seventeen studies on cio role configurations that were empirically based can in turn be categorised by their methodology into three groups: quantitative (n= 7); qualitative (n=7); and mixed methodology (n=3). two thirds of these studies on cio role configurations were conducted in the usa (n=15), nearly one third in the uk (n=5), with the rest conducted in china and taiwan (n=2) or worldwide (n=1). the number of suggested cio roles in each of these 24 studies ranges from three to eight. studies with a large number of roles in configurations (7 and 8 roles) tend to break down the main roles into much more specific roles for the sake of further explanation, while studies with fewer cio roles in configurations (3 and 4 roles) tend to merge two or more key cio roles for the sake of parsimony. earlier and later studies presented in table 5 include both operational (supply side) roles and strategic (demand side) roles and indicate that the shift of cio role from a largely technical and supply side role to a more strategic and demand side role started back in the late 1980s. these 24 studies of cio role configurations confirmed that the cio has multiple roles (a configuration of roles) and not one or two specific roles. is scholars have given different names to the cio roles. the most cited cio role configuration based on the data provided by google scholar (as december 2012) is smaltz et al.’s (2006) model of six key roles, followed by chun and mooney’s (2009) configuration of four key roles. table 6 provides brief definitions of these six roles in the cio roles configuration model developed and validated by smaltz et al. (2006). supply side roles demand side roles educator: the role of the cio as an it missionary, who provides insight and understanding about key information technologies to raise top management savviness, awareness, and appreciation of it and help them to make appropriate judgments about the business value of it and wise it investment decisions. strategist: the organisational desire for the cio to be an effective business partner and help their organisation leverage valuable opportunities for it-based innovation and business process redesign. information steward: the desirability of the cio to be an organisational steward for high quality data and operationally reliable systems. relationship architect: the desirability of a cio to build relationships both across the enterprise as well as outside the enterprise with key it service providers. utility provider: the role of the cio as a builder of sustaining solid, dependable, and responsive it infrastructure services. integrator: the desirability of the cio providing leadership in enterprise-wide integration of processes, information, and decision-support as digital options for the business table 6. smaltz et al.’s (2006) six key cio roles (source: smaltz et al. 2006) after rational examination and comparison of the quantitative studies of cio role configurations, this research adopted the six cio roles configuration developed by smaltz et al. (2006) for several reasons. first, this cio roles configuration model was developed from the comprehensive inventory of cio roles identified from previous literature and empirical data obtained from in-depth interviews with cios and top management members. second, it represents a comprehensive configuration that accommodates all of other empirical configurations previously identified as shown in table 7. also, despite the fact that this configuration of cio roles was developed within the healthcare sector, the cio roles that emerged were similar to the ones that have been identified in the existing literature in general (strickland 2011). furthermore, agarwal and beath (2007) found that all of the roles included in this australasian journal of information systems volume 18 number 2 2014 67 cio roles configuration were important in grooming the future cios regardless of their industry. moreover, this configuration has been proved to be a valid typology within the australian context (seddon, walker, reynolds, and willcocks 2008). what is more, smaltz et al. (2006) classified these six roles into two groups: supply side roles (utility provider, information steward, and educator) and demand side roles (integrator, relationship architect, and strategist) following the modern classification of the cio roles proposed by is scholars such as mark et al. (2004) and broadbent and kitzis (2005). furthermore, this cio roles configuration has been measured by a survey instrument that demonstrated high validity and reliability (chen et al. 2010; chen and wu 2011; li et al. 2012; wu, chen, and sambamurthy 2008). finally, the survey instrument for this cio configuration is concise which is crucial as the targeted survey respondent is the most senior it executive in the organisations. these managers are usually extremely busy and less likely to complete a lengthy questionnaire. australasian journal of information systems volume 18 number 2 2014 68 reference three demand side roles three supply side roles strategist relationship architect integrator educator information steward utility provider feeny 1997 system thinker relationship builder technical virtuoso supply manager earl 1996 visionary / systems thinker relationship builder / alliancemanager tactician /reformer deliverer /architect mclean & smits 1993; 2003; 2012 strategist/ innovator innovator/ enabler innovator enabler technologist sojer et al. 2006 driver enabler supporter/ cost cutter agarwal & beath 2007 strategist relationship architect integrator / leader educator information steward/ leader utility provider ibm 2009 savvy value creator collaborative business leader insightful visionary chun & mooney 2009 innovator & creator opportunit y seeker triage nurse landscape cultivator weiss & adams 2010 change agent / business expert change agent technologi st technologist peppard et al. 2011; tansley, loughran, edwards, lambert, & peppard 2008 innovator facilitator evangelist utility it director/ agility it director chen & wu 2011; wu et al. 2008 business visionary, business system thinker, entrepreneur; & value configure relationship builder ; value configure; & informed buyer organisati onal designer (coordinat ing) organisatio nal designer (staff) infrastructure builder nicolet 2011 strategist facilitator / manager educator technologist / implementer table 7. mapping of six cio roles configuration (smaltz et al. 2006) with other researchers’ cio roles configurations australasian journal of information systems volume 18 number 2 2014 69 research hypotheses development the relationship between the it vision and cio role the prior literature emphasised two central facts regarding the roles of cios. the first is that cios perform a configuration of roles rather than one specific role (agarwal and beath 2007; chen and wu 2011; earl 1996; smaltz et al. 2006), while the second fact acknowledges that the importance of these roles differs according to the organisational contingency (earl 1996; kaarst-brown 2005; peppard et al. 2011; rockart, ball, and bullen 1982; smaltz et al. 2006). some conceptual and empirical studies anticipated that the strategic it vision of the firm may affect the cio’s role (broadbent and kitzis 2005; dahlbom and mathiassen1997; kaarst-brown 2005; marchand 2007; spitze and depaschalis 2005). teo and too (2000) asserted that the organisation’s strategic it vision evolves in stages from automate to informate and finally, to transform, thus firms progress along an evolutionary path that parallels with the maturing of their deployment of it. mclean and smits (2003) confirmed that the role of cio has evolved and expanded into a number of roles in parallel with the evolution of the role of it in organisations starting with the technologist role, then to be an enabler, then an innovator, and lastly as a strategist role. results from a recent empirical study of 129 cios and senior business executives from china and taiwan (li et al. 2012) indicate that the strategic it vision has a significant moderating effect on the relationship between cio strategic roles’ effectiveness and the innovative usage of is. an empirical study of 168 senior it executives in the healthcare sector in the usa revealed that cio roles as business partner (strategist) and integrator were significantly more important to organisations that expressed a ‘transform’ vision of it (smaltz 2000). hence, the literature provides support for the following hypothesis: h1: the perceived importance of each cio role (strategist, integrator, relationship architect, educator, utility provider, and information steward) differs according to the organisation’s strategic it vision (automate, informate-down, informate-up, and transform). the relationship between the it vision and cio structural power the cio’s structural power refers to the cio’s level of legitimate power in their formal position within the hierarchy of the organisation (chen et al. 2010). for the purpose of this research two variables will be used to measure the cio’s structural power: the cio’s reporting structure and the cio’s job title in an organisation. the literature suggested that a shared conception of the role of it in an organisation is the key to an excellent relationship between the ceo and the cio (feeny et al. 1992) and it was critical to create and implement is strategic alignment in an organisation (preston et al. 2009). furthermore, it has been found that the cios with higher reporting levels had greater influence and control over the extent of it strategy implementation (gottschalk 1999). preston, chen, and leidner (2008) found that cio structural power, measured by reporting structure, is positively associated with the cio’s level of strategic decision-making authority in an organisation. cohen and dennis (2010) found a significant relationship between the cio reporting level and the cio organisational positioning. other studies have confirmed the significant relationship between the rank of it leader and the organisation’s it strategic orientation (grover et al. 1993; raghunathan and raghunathan 1989) which indicates that the higher the rank of a cio in an organisation structure, the higher the vision of it is ranked in an organisation. karimi et al. (1996) emphasise that the it leader’s rank and role must align with the firm’s competitive strategy. saldanha and krishnan (2011) found that it-enabled business innovation is more likely when the cio reports to the ceo. it is argued that the cios reporting structure should align with the organisational main purpose of it (leonard 2007). banker, hu, pavlou, and luftman (2011) found that the firm’s strategic positioning (differentiation or cost leadership) and it orientation are a primary determinant of its cio reporting structure. a cio is more likely to report to the ceo and have a high level job title when it is considered to be a strategic enabler of organisational strategy whereas a cio australasian journal of information systems volume 18 number 2 2014 70 is more likely to report to the cfo and have a lower level job title when it is considered to be mainly a way to reduce costs in an organisation. the literature provides support for the following hypothesis: h2: the it leader’s structural power is positively associated with the firm’s strategic it vision. figure 1 illustrates the research model showing the hypotheses that the organisational vision of it is associated with the cio’s role and their structural power. methodology the aim of this study is to examine the relationship between the strategic vision of it in an organisation and the configuration of cio roles and structural power of the cio. a quantitative approach based on a postal mail survey was chosen in order to establish generalizability, allow replication, and gain adequate statistical power (straub 1989). senior it executives (cios) were the targeted survey respondent for this study. the rationale behind choosing the cio as the most appropriate person in an organisation to provide answers on the research constructs (cio roles, strategic vision of it, cio structural power) was that they are the most experienced and knowledgeable persons in terms of their roles, position in the organisation and their organisation’s view of it. thus cios can provide appropriate responses to the survey questionnaire leading to accurate results. research measures the questionnaire used in this study comprises three sections (see appendix a). the first section presents the strategic it vision scale developed by feeny et al. (1992) based on schein’s (1989, 1992) typology. this scale is ordinal and gives the respondents four brief statements that describe four visions of the role of it in an organisation (automate, informate-down, informate-up, and transform). the respondents were asked to choose one option that best describes their firm’s vision of it. this measure was successfully adopted by previous researchers (e.g., armstrong and sambamurthy 1999; feeny et al. 1992; smaltz 2000; smaltz et al. 2006; smaltz 1999) whereas we did not find any research that used the scales developed by subramanian and nosek (1993) and ramakrishna and lin (2002). the second section of the research questionnaire includes the cio role expectations instrument developed by smaltz et al. (2006). this instrument has chosen after a rational comparison of developed instruments related to the cio role identified from the literature (e.g., mccall and segrist (1980); arthur andersen company (1986) cited in passino jr and severance (1988); karimi et al. (1996); gottschalk (2000b); smaltz et al. (2006); and chen et al. (2011)) this scale was used to identify the perceived importance of six key cio roles proposed as strategist, integrator, relationship architect, educator, australasian journal of information systems volume 18 number 2 2014 71 utility provider, and information steward. as explained earlier, there are two main reasons for the choice of this instrument. firstly, this instrument has demonstrated high validity and reliability (chen and wu 2011; li et al. 2012; wu et al. 2008). secondly, this instrument is concise which is crucial as the targeted survey respondent is the most senior it executive in the organisation and extremely busy, over surveyed and is unlikely to fill out a lengthy questionnaire. the third section of the questionnaire includes questions regarding the participants’ demographic details including their reporting structure and job title. data collection data for this research were collected through a large scale mail survey carried out in australia in early 2012. a list of postal addresses of 954 senior it executives in australian private sector firms was purchased from dun & bradstreet australia (2011) and provided the basis for the survey working population for this study. the mail survey was administrated in two waves: an initial mail out and a follow up mail out to ensure reasonable response rate. in february 2012 a cover letter along with a copy of the questionnaire and pre-paid reply envelope was sent to the senior it executives’ addresses. table 8 provides some statistics regarding the survey administration. round sent date responses complete and usable incomplete r.t.s. not willing to participate one 954 28/2/2012 97 95 2 105 4 two 950 19/3/2012 67 67 105 1 total 1908 164 162 2 210 5 table 8 survey administration statistics a total of 105 questionnaires were returned as undeliverable due to invalid addresses, and emails were received from five firms not willing to participate in this survey. with 162 complete and usable responses the response rate of this study was estimated to be 19.19 per cent which is considered to be a reasonable response rate for a postal mail survey compared to similar studies involved cios where response rates ranged from 7 to 22.5 per cent (chen and wu 2011; gerow 2012; oh and pinsonneault 2007; preston et al. 2006; weiss and adams 2010; wu et al. 2008). data analysis measure validation the validity and reliability of the cio roles instrument developed by smaltz et al. (2006) was checked prior to any further inferential analyses. this step is not applicable for the strategic it vision scale as it is a single item ordinal measure. the data collected were analysed using component-based structural equation modelling method (sem), partial least squares (pls). hair, ringle, and sarstedt (2012) asserted that pls/sem is more favourable with smaller sample sizes. the cio roles were modelled as reflective constructs; hence five major areas should be tested to ensure measurement validity (henseler, ringle, and sinkovics 2009): reliability at the construct level; reliability at the indicators level; convergent validity; discriminant validity at the construct level; and discriminant validity at the australasian journal of information systems volume 18 number 2 2014 72 indicators level. plsgraph (version 03.12 build 01) software was used to test the measurement (outer) model. following common criteria suggested by chin (2010); and henseler et al. (2009) we examined the inter-construct correlations, composite reliabilities, average variance extracted for each construct, item loadings on their constructs and items cross loadings on other constructs. these statistics are presented in tables 9 and 10. construct* cr ave strategist relationship architect integrator educator utility provider info. steward strategist 0.88 0.55 0.74** relationship architect 0.88 0.71 0.34 0.84 integrator 0.83 0.55 0.52 0.33 0.74 educator 0.90 0.76 0.56 0.26 0.48 0.87 utility provider 0.84 0.57 0.22 0.32 0.45 0.22 0.75 info. steward 0.81 0.59 0.46 0.43 0.40 0.44 0.43 0.77 *all items measured using seven point likert scale ranging from ‘ not at all important’ (1) to ‘critically important’ (7). ** square root of ave on diagonal; cr = composite reliability; ave = average variance extracted. table 9 inter-correlation among cio roles and reliability coefficients as can be seen in table 9, the composite reliability (cr) for all constructs exceeds the satisfactory level of 0.7 proposed by werts, linn, and joreskog (1974) which supports internal consistency reliability. reliability at the indicators level can be checked by examining the items loading on their respective constructs (see table 10). chin (1998) and henseler (2009) suggested 0.7 as a rule of thumb for a standardized outer loading to ensure that the indicator has captured at least half of the variance. the factor loadings of all items exceed the standardized cut off except for five items of which three are over 0.6 and two are below 0.4. a decision was made to keep the first three items as long as the composite reliability for their respective constructs is still over the satisfactory level of 0.7 (henseler et al. 2009) and remove the two items with the lower factor loadings of less than 0.4. the average variances extracted (aves) proposed by fornell and larcker (1981) for all research constructs as shown in table 9 exceed the acceptable cut off 0.5 which indicates sufficient convergent validity. discriminant validity at the indicators level is evident in table 10 as all remaining items are strongly related (loadings) to the constructs they were intended to measure and they do not have a stronger connection with another construct (cross loadings). discriminant validity at the construct level is confirmed, as the square root of the ave values of all constructs are larger than the inter-correlation of the constructs in the model which means that all constructs shared more variance with their own measures than with others (see table 9). since the reliability and validity of each construct for the six cio roles was confirmed, the mean for each set of items retained for each of the six cio roles was calculated in order to perform the anova and correlation analysis required for the hypotheses testing. australasian journal of information systems volume 18 number 2 2014 73 table 10 six cio roles construct’s items loadings and cross loadings (source: pls results) non-response bias test in order to assess non-response bias and following the guidelines presented in armstrong and overton (1977) and sivo, saunders, chang, and jiang (2006) a comparison was conducted between the early respondents (n=21) and late respondents (n=13) in terms of the research variables. it is assumed that late responders share similarities with non-responders and, if no significant differences are found between early and late responses, the likelihood is strong that non-response bias did not occur. a mannwhitney u test was used for this purpose since the data comprise some categorical variables. the results of the mann-whitney u test conducted on 26 items (presented in appendix b) found statistically significant differences in only one item which means that there are no major differences between early and late respondent cios. these results indicate the absence of non-response bias. testing research hypotheses having confirmed the reliability and validity of the research measures for the six key cio roles and established the absence of non-response bias, the next stage of the data analysis involved splitting the total data set (162 responses) into four sub groups based on the organisation’s strategic it vision. these four groups across the 162 respondent organisations were: automate (32); informate-down (31); informate-up (29); and transform (70). these four groups provide the basis for testing the research hypotheses. table 11 exhibits the mean and standard deviation of the six cio roles for the overall sample and the subsamples of organisations grouped according to their vision of it. items strategist integrator relationship architect educator utility provider information steward stra1 0.67 0.32 0.27 0.32 0.35 0.11 stra2 0.74 0.53 0.26 0.39 0.35 0.25 stra3 0.75 0.48 0.25 0.38 0.29 0.25 stra4 0.76 0.54 0.36 0.54 0.41 0.24 stra5 0.78 0.26 0.23 0.43 0.35 0.05 stra6 0.76 0.20 0.17 0.40 0.29 0.30 integ1 0.45 0.75 0.21 0.25 0.42 0.49 integ2 0.37 0.64 0.25 0.46 0.26 0.24 integ3 0.27 0.76 0.22 0.33 0.26 0.31 integ4 0.46 0.81 0.31 0.42 0.24 0.28 rear1 0.25 0.26 0.81 0.13 0.28 0.20 rear2 0.33 0.32 0.90 0.30 0.37 0.30 rear3 0.29 0.26 0.82 0.22 0.45 0.30 edu1 0.41 0.48 0.21 0.84 0.43 0.28 edu2 0.55 0.4 0.24 0.88 0.35 0.10 edu3 0.48 0.36 0.23 0.89 0.37 0.18 utpr1 0.11 0.23 0.25 0.11 0.68 0.29 utpr2 0.17 0.37 0.21 0.22 0.80 0.33 utpr3 0.08 0.31 0.24 0.13 0.81 0.34 utpr4 0.33 0.45 0.27 0.19 0.73 0.35 infst2 0.36 0.41 0.41 0.30 0.40 0.70 infst3 0.44 0.33 0.36 0.43 0.29 0.78 infst4 0.26 0.19 0.24 0.28 0.31 0.81 australasian journal of information systems volume 18 number 2 2014 74 cio roles overall n= 164 mean sd automate n=32 mean sd informate-down n=31 mean sd informate-up n=29 mean sd transform n= 70 mean sd strategist 5.37 0.98 5.1 0.99 5.29 1.06 5.27 1.21 5.58 0.79 relationship architect 5.58 0.94 5.35 1.28 5.82 0.90 5.54 0.77 5.60 0.83 integrator 5.01 0.99 5.12 1.00 5.26 1.15 4.81 1.10 4.94 0.84 educator 4.79 1.20 4.28 1.42 4.95 1.09 4.78 1.33 4.97 1.03 information steward 5.74 0.82 5.44 0.97 5.91 0.89 5.55 0.87 5.87 0.63 utility provider 5.60 0.90 5.66 0.84 5.72 0.92 5.53 1.08 5.55 0.84 table 11 descriptive statistics for six key cio roles across four it visions (source: developed for this study) testing hypothesis one a one-way analysis of variance (anova) was conducted to determine whether the means of the six key cio roles differed across the four groups of it vision. results partially support research hypothesis one as they show that there are significant differences at the p < .05 level among two of the six key cio roles across the four groups of an organisation’s it vision. next, to determine which cio roles were significantly different across the four groups of it vision, tukey post-hoc comparisons of the four groups of it visions were conducted and indicated that there are significant differences between the cio role as an educator in the firms that articulate a ‘transform’ vision (m= 4.97, sd = 1.03) and those firms that articulate an ‘automate’ vision (m= 4.28, sd 1.42). results also show that there are significant differences between the cio role as an information steward in the firms that articulate an it ‘transform’ vision (m= 5.87, sd = .64) and those firms that articulate an it ‘informate-down’ vision (m = 5.91, sd = .90) and those firms that articulate an ‘automate’ vision (m= 5.44, sd = 0.97). the effect size calculated using eta squared was 0.05 for the educator role and 0.05 for the information steward role. according to cohen (1988) the effect size of 0.05 can be considered a medium effect. table 12 provides a summary of anova results including levene’s tests for homogeneity which are all not significant at the p > .05 level indicating that the population variance for each group are approximately equal. cio role levene statistic f(3,159) sig eta squared strategist 1.86 n.s 2.10 0.10 n.s n.a relationship architect 2.08 n.s 1.35 0.25n.s n.a integrator 1.63 n.s 1.28 0.28 n.s n.a educator 2.28 n.s 2.70 0.04* 0.05 information steward 2.46 n.s 3.06 0 .03* 0.05 utility provider 1.04 n.s 0.37 0.77 n.s n.a n.s = not significant; * significant at p < 0.05 table 12. anova results regard the cio roles across four it visions australasian journal of information systems volume 18 number 2 2014 75 the results shown in table 12 partially support research hypothesis one. testing hypothesis two in order to test the relationship between the cio’s structural power (job title and reporting structure) and the organisation’s view of it, and due to the ordinal nature of these variables, the authors ranked the data regarding the three research variables, strategic it vision, cio reporting structure, and cio job title in a logical ordinal rank order from lowest to highest. this allowed us to perform a nonparametric correlation using spearman’s rank-order correlation. table 13 illustrates how we recoded the research variables into ordinal rank order for the purpose of testing research hypothesis two. strategic it vision cio’s reporting structure common cio’s job title item rank importance rank automate to others edp / mis manager 1 low informatedown to coo is/it manager/director 2 moderate informate-up to cfo cto 3 high transform to ceo cio / vice president it 4 very high table 13 ranking of three research variables for hypothesis two the total data set of 162 valid responses from australian senior it executives was used to test research hypothesis two. spearman’s rank-order correlation was performed between the three variables (strategic it vision, cio reporting structure, and cio job title). results of the correlation analysis indicate that an organisation’s strategic it vision is significantly and positively associated with both the level of the cio reporting structure (r =.178, p < .024, 2 tailed) and the level of the cio job title in an organisation (r = .207, p < .008, 2 tailed). this finding supports the notion that the more progressively an organisation views it from ‘automate’ up to ‘transform’, the more likely it is that the cio will report to the upper level of senior management and will have higher level job title in an organisation. these results provide support for research hypothesis two. table 14 shows descriptive statistics of a cross tabulation regarding the cio’s reporting structure and job title across four different contexts of it vision which provide further support for the results of spearman’s rank-order correlation. as evidenced in table 14 the percentage of it leaders that reported to the most senior executive in an organisation (ceo) markedly increased from 34.4 per cent in firms that adopted the automate vision for it to 52.9 per cent in firms that articulated the transform vision of it. as shown in table 14, it is worth mentioning that a comparison of the proportion of australian cios reporting to the ceo with other surveys also conducted in australia e.g., watson (1990) and leonard (2007) suggests a significant increase over time of cios reporting to the ceo. this percentage increased from 14 per cent in 1989 (watson 1990), to 33 per cent in 2007 (leonard 2007), to 42.7 per cent in 2012 in our study. australasian journal of information systems volume 18 number 2 2014 76 it visions cio status overall n= 162 automate n= 32 informate down n= 31 informateup n= 29 transform n= 70 freq. % freq. % freq. % freq. % freq. % cioreportin gto other 1 11.7 5 15.6 5 16.1 3 10.3 6 8.6 to the coo 14 8.6 4 12.5 2 6.5 3 10.3 5 7.1 to the cfo 60 37.0 12 37.5 14 45.2 12 41.4 22 31.4 to the ceo 69 42.7 11 34.4 10 32.3 11 37.9 37 52.9 cio job title: edp/ mis manager 12 7.4 4 12.5 3 9.67 1 3.4 4 5.7 is/it manager 71 43.8 16 50.0 16 51.6 14 48.2 25 35.7 cto 6 3.7 1 3.1 2 6.5 1 3.4 2 2.9 cio/ vice president it 73 45.1 11 34.3 10 32.3 13 44.8 39 55.7 table 14 cio reporting structure / job title within the context of strategic it vision. as far as the cio job title is concerned, table 14 shows that nearly two thirds of the it leaders working within firms that articulated transform vision of it have a senior job title (cio, vp it, and cto) while only one third have the lowest job title such as is/it manager/director and vice versa for the firms that articulated the automate vision of it. figure 2 summarises the results of testing the research hypotheses. discussion our study found that the importance of the six distinct cio roles differs partially in regards to their organisation’s strategic it vision as perceived by their cios. the results of this study suggest that there is a significant effect of the it vision of the firm on the cio roles of educator and information steward. australasian journal of information systems volume 18 number 2 2014 77 the australian cios who oversee it in an organisation with a ‘transform’ vision of it perceived the educator and the information steward roles more important than other cio roles in comparison to those who work within organisations with an ‘automate’ it vision. the information steward was also perceived to be more important than the other roles by the cios within organisations with an ‘informate–down’ it vision in comparison to those who work within organisations with an ‘automate’ it vision. the possible explanation for the importance of the cio’s educator role in an organisation that adopted the ‘transform’ vision of it might be due to the major cultural change required in firms going through transformational change (schein 1989, 1992) which in turn requires the cios to pay more attention to the educator role. the cios within organisations that articulate a ‘transform’ vision of it need to perform two types of educational activities in order to promote it as an agent of business transformation (kadlec 2004). the first is the facilitating educational activities that are important to provide the top management team (tmt) with required knowledge regarding the emerging technology and how it can transform business (tmt mental model building). the second type of educational activities is empowerment activities which are important to provide the top management team with required knowledge regarding the established technologies used and how the firm can invest in these technologies to transform its business (tmt mental model maintenance). the importance of the cio’s educator role in australian firms that articulate the ‘transform’ vision of it, provides further support for pervan’s (1998) finding that australian cios have a greater need for the it education of senior management. also, the role of the cio as an information steward in ‘transform’ vision organisations was found to be more important possibly because of the emerging concern of how to ‘keep the lights on’, and the need to provide high quality information, protect organisation data, customer privacy, and recruit qualified it staff within the radical change brought about from adopting this transformational view of it. the same concern will emerge for cios who work within organisations that expressed the ‘informatedown’ vision of it as according to schein (1989, 1992) this view required the whole system to be transparent to employees which challenges the cio’s role as an information steward responsible for organisational data security and privacy. surprisingly, as the view of it matures from ‘automate’ to ‘transform’ the strategic roles of the cios do not become more important which is consistent with the finding of kaarst-brown (2005) as she found that, despite the strategic potential of it in the investigated organisations, the cios are often not granted the same strategic decision-making authority as other business executives. also, these findings align with grover et al.’s (1993) study which found that as it matures the cios’ strategic roles do not become more important as one might expect. on the other hand, those findings conflict with those of smaltz (2000) who found an increase in the importance of the cios’ strategic roles in organisations that expressed the ‘transform’ vision of it. two possible explanations are offered. firstly, smaltz (2000) conducted his research within the healthcare sector which has special characteristics, is information intensive and is still undergoing a continuous transformation process toward greater use of it. the second explanation might be as the role of it matures in an organisation, the other executives in the top management team will play a more proactive role in setting the it strategic vision for the organisation. our study also found that it leaders in organisations with transformational vision of it are more powerful in terms of their job title and their reporting level than their counterparts in organisations that articulated lower visions of it such as ‘informate-up’, ‘informate-down’ and ‘automate’ views. that means the it leaders in organisations that articulated the higher transformational view of it are more likely to have a higher level report to the ceo and hold the higher title of cio or vice president it, whereas it leaders in organisations that articulated a lower ‘informate-up’, ‘informate-down’ or an ‘automate’ view of it are more likely have a lower level report to the cfo or coo and more likely to australasian journal of information systems volume 18 number 2 2014 78 hold the lower title of cto or it /is manager/director. these results are consistent with the results of karimi et al. (1996); grover et al. (1993); raghunathan and raghunathan (1989); cohen and dennis (2010); and banker et al. (2011) who argued that the rank of the it leaders should align with the firm’s competitive strategy and it orientation. secondly these finding provide further support for the proposition developed by sherer (2004) which argued that the reporting structure of it leaders is influenced by the strategic vision of it. conclusion our study has contributed to the body of knowledge in several ways. first, it is one of few studies that has examined the perceived importance of the configuration of cio roles across organisations in relation to the different strategic views of it and it presents some interesting results. furthermore, the results of this study support the validity of the configuration of the cio roles instrument developed by smaltz et al. (2006) and indicate that this cio roles configuration instrument is relevant to cios in different industries and not solely in the healthcare sector, which provides further support for the findings of agarwal and beath (2007) and strickland (2011). the results provide support for the contingency approach to leadership confirming that when it comes to cio roles, one size does not fit all organisations. this study has several useful implications for different stakeholders. first, the findings of this study are important to cios as it is indicates the need to adapt their configuration of roles according to their organisation’s strategic view of it. secondly, the key findings of this study can provide guidance to top management for recruitment of cios who will be able to play the configuration of roles that fit with the organisation’s strategic view of it. moreover, the training programmes for the specialist institutions responsible for preparing cios can be enhanced by the key findings of this study regarding the need to consider a configuration of cio roles that are aligned with organisation’s strategic vision of it. furthermore, this study has highlighted some gaps in the literature. the vast majority of the literature was based on mintzberg’s framework (1973) whereas almost none have used the other three key management roles typologies such as paie (adizes 1976, 2004); cvf (quinn et al. 2006); and the integrated model of executive leadership roles (hart and quinn 1993). limitations despite the key findings reported from this study, some limitations should be acknowledged. the findings of this study represent the perceptions of australian cios which might not match with the perceptions of cios in other countries. in addition, identifying the organisational view of it is based on the perception of a single manager (the cio) rather than considering multiple perceptions (e.g. all top management team) hence the findings of this study regarding the relationship between the six cio roles and the firm’s it vision should be treated with caution and investigated in greater detail from multiple perspectives of all the top management team. future research the key findings and the gaps identified by this study warrant further research. first, studying the impact of the strategic it vision on the configuration of cio roles in different countries is needed in order to validate the generalizability of our study’s findings and facilitate conducting a comparison among the it leaders in different countries. also, using different instruments whether for the cio role or for the it vision is required to better capture a comprehensive picture of the participants’ perceptions in this area. examining the relationship between the same constructs with a bigger sample size or extending the identification of the organisational view of it to include other members of the firm’s top management team could give different results. finally, investigating the impact of other organisational australasian journal of information systems volume 18 number 2 2014 79 contingencies such as organisational information intensity, organisational culture, organisational climate, organisational life cycle, and organisational it maturity on the configuration of cio roles is central to clarifying that vital role and filling the gaps in the body of knowledge. finally, applying some neglected classic managerial roles configurations such as adizes (1976,2004), quinn et al. (2006) and hart and quinn (1993) could give another perspective and further insights regarding cio roles. references adizes, i 1976, 'mismanagement styles', california management review, vol. 19, no. 2, pp. 5-20. adizes, i 2004, management/mismanagement styles, the adizes institute publishing. agarwal, r & beath, c 2007, grooming the 2010 cio, a report for the society for information management, advanced practice council. armstrong, c, & sambamurthy,v 1999, 'information technology assimilation in firms: the influence of senior leadership and it infrastructures', information systems research, vol. 10, no. 4, pp. 304-27. armstrong, js & overton, ts 1977, 'estimating nonresponse bias in mail surveys', journal of marketing research (jmr), vol. 14, no. 3, pp. 396-402. banker, r, hu, n, pavlou, p, & luftman, j 2011, 'cio reporting structure, strategic positioning, and firm performance', mis quarterly, vol. 35, no. 2, pp. 487-504. bassellier, g, reich, b, & bensabat, i 2001, 'information technology competence of business managers: a definition and research model', journal of management information systems, vol. 17, no. 4, pp. 159-82. broadbent, m, & kitzis, e. 2005, the new cio leader: setting the agenda and delivering results, harvard business school press. brown, c 1993, 'the successful cio: integrating organizational and individual perspectives', paper presented to sigcpr'93. brown, w 2006, 'cio effectiveness in higher education', educause quarterly, vol. 29, no. 1, p. 48. carter, m, grover,v, & thatcher, j 2011, 'the emerging cio role of business technology strategist', mis quarterly executive, vol. 10, no. 1, pp. 19-29. cash, j, mcfarlan, f, mckenney, j, & vitali, m 1988, corporate information systems management: text and cases, homewood, il: irwin. chen, d, & preston, d 2007, 'understanding cio role effectiveness: the antecedents and consequents'. 40th annual hawaii international conference on systems sciences, jan. 2007. chen, d, preston, d, & xia, w 2010, 'antecedents and effects of cio supply-side and demand-side leadership: a staged maturity model', journal of management information systems, vol. 27, no. 1, pp. 231-71. chen, d, mocker, m, preston, d, & tuebner, a 2010, 'information systems strategy: reconceptualization, measurement, and implications', mis quarterly, vol. 34, no. 2, pp. 23359. chen, y, & wu, j 2011, 'it management capability and its impact on the performance of a cio', information & management, vol. 48, no. 4–5, pp. 145-56. chin, ww 1998, 'issues and opinion on structural equation modelling', management information systems quarterly, vol. 22, no. 1, pp. 7-16. australasian journal of information systems volume 18 number 2 2014 80 chin, w, w 2010, ‘how to write up and report pls analyses’. in v. esposito vinzi, w. w. chin, j. henseler, & h. wang (eds.), handbook of partial least squares: concepts, methods and methods and application (pp. 645–689). new york. chun m , & mooney j. 2009, 'cio roles and responsibilities: twenty-five years of evolution and change', information & management, vol. 46, no. 6, pp. 323-34. cohen, j 1988, statistical power analysis for the behavioral sciences, lawrence erlbaum. cohen, j & dennis, c 2010, 'chief information officers: an empirical study of competence, organisational positioning and implications for performance', south african journal of economic and management sciences, vol. 13, no. 2, pp. 203-21. dahlbom, b & mathiassen, l 1997, 'the future of our profession', communications of the acm, vol. 40, no. 6, pp. 80-9. dun and bradstreet 2011, company 360 database, dun and bradstreet (australia) pty ltd, viewed 10th june 2012, <http://dnb.com.au/sales_and_marketing/company_information_and_research/index.aspx> earl, m.j. (1996). the chief information officer: past, present and future. in: earl, m.j., information management. oxford university press, oxford. feeny, d 1997, 'the five year learning of ten it directors', managing it as a strategic resource. maidenhead: mcgraw hill. feeny, df, edwards, br & simpson, km 1992, 'understanding the ceo/cio relationship', mis quarterly, vol. 16, no. 4, pp. 435-48. fiedler, f. 1967, a theory of leadership effectiveness. new york: mcgraw-hill fisher, jf 2003, 'cio then and now: an historical perspective of the cio role and the it function', paper presented to 39 th annual meeting of the midwest business. fornell, c & larcker, df 1981, 'evaluating structural equation models with unobservable variables and measurement error', journal of marketing research, vol.18, february, pp. 39-50. gerow, je 2012, 'what should firms look for in a cio?' proceedings of the southern association for information systems conference, atlanta, ga, usa march 23-24. gottschalk, p 1999, 'strategic management of is/it functions: the role of the cio in norwegian organisations', international journal of information management, vol. 19, no. 5, pp. 389-99. gottschalk, p. 2000a. the chief information officer: a study of managerial roles in norway. [electronic version]. retrieved on april 14, 2011 from http://webster.hibo.no/okobit2000/idragsfiler/gottschalkp.doc ---2000b, 'information systems executives: the changing role of new is/it leaders', informing science, vol. 3, no. 2, pp. 31-40. grover, v, jeong, s, kettinger, w & lee, c 1993, 'the chief information officer: a study of managerial roles', journal of management information systems, vol. 10, no. 2, pp. 107-30. hair, jf, ringle, cm & sarstedt, m 2011, 'pls-sem: indeed a silver bullet', journal of marketing theory & practice, vol. 19, no. 2, pp. 139-52. hallikainen, p, hu, q, frisk, e, päivärinta, t, eikebrokk, tr & nurmi, a 2006, 'the use of formal it investment evaluation methods in organizations: a survey of european countries'. proceedings of the 12th americas conference on information systems, acapulco, mexico august 4-6 2006. hart, sl & quinn, re 1993, 'roles executives play: ceos, behavioral complexity, and firm performance', human relations, vol. 46, no. 5, pp. 543-74. henseler, j, ringle, cm & sinkovics, rr 2009, 'the use of partial least squares path modelling in international marketing', advances in international marketing, vol. 20, no. 1, pp. 277-319. australasian journal of information systems volume 18 number 2 2014 81 hersey, p & blanchard, kh 1993, management of organizational behavior: utilizing human resources, prentice-hall, inc. house, r & mitchell, t 1974, 'path-goal theory of leadership', journal of contemporary business, vol, 3, pp. 81-97. hu, n, pavlou, p, & banker,r. 2004, 'it orientation, cio reporting structure and firm performance: to whom should the cio report?'. workshop on information systems and economics (wise). ibm 2009, the new voice of the cio: insight from the global chief information officer study, ibm corporation. kaarst-brown, m 2005, 'understanding an organization's view of the cio: the role of assumptions about it', mis quarterly executive, vol. 4, no. 2, pp. 287-301. kadlec, c 2004, 'educator roles of the cio', paper presented to the 7th annual conference of the southern association for information systems. karimi, j, gupta, y & somers, t 1996, 'the congruence between a firm's competitive strategy and information technology leader's rank and role', journal of management information systems, vol. 13, no. 1, p. 88. karlsen, jt, gottschalk, p & andersen, es 2002, 'information technology management roles: a comparison of it executives and it project managers', proceedings of the 35th annual hawaii international conference on system sciences (hicss), 7-10 jan. 2002. leonard, j 2007, ‘sharing a vision: comparing business and is managers’ perceptions of strategic alignment issues’, australian journal of information systems, vol 15, no.1,pp 95-112. li, d, ding, f, & wu, j, "innovative usage of information systems: does cio role effectiveness matter?" (2012). pacis 2012 proceedings. paper 81. lineman, j 2006, the chief information officer in higher education: a study in managerial roles, phd dissertation, capella university. marchand, da 2007, 'realizing it value: a shared responsibility between senior managers and the cio', information resources management journal. no.147, may 2007. mark, d & monnoyer, e 2004, 'next-generation cios', the mckinsey quarterly, july. pp.2-8. mccall, mw, & segrist, ca 1980, in pursuit of the manager's job: building on mintzberg, center for creative leadership. mcfarlan, w. f., mckenny, j. l., & pyburn, p. 1983. “the information archipelago: plotting a course,” harvard business review. 1983 jan-feb; 61(1), pp. 145-156. mclean, e & smits, s 1993, 'the i/s leader as `innovator'', proceedings of the twenty-sixth hawaii international conference on system sciences, 5-8 jan 1993. mclean, e & smits, s 2003, 'a role model of is leadership', amcis 2003 proceedings, pp. 1273-1282. mclean, e & smits,s. 2012, management, leadership, and the roles of the cio. retrieved 5 january, 2013, from < http://cob.unt.edu/itds/faculty/becker/bcis5520/readings/cio_leadershipmclean-smits-june%202011_ed1.pdf> milliron, mt 2008, management roles and the community college chief information officer, proquest. mintzberg, h 1971, 'managerial work: analysis from observation', management science, vol. 18, no. 2, pp. b-97-b110. mintzberg, h 1973, the nature of managerial work, harper & row, new york. nicolet, t 2011, leadership in higher education: the cio role and the leadership team. phd dissertation, north carolina university at greensboro. australasian journal of information systems volume 18 number 2 2014 82 oh, w & pinsonneault, a 2007, 'on the assessment of the strategic value of information technologies: conceptual and analytical approaches', mis quarterly, vol. 31, no. 2, pp. 239-265. passino jr, jh & severance, dg 1988, 'the changing role of the chief information officer', strategy & leadership, vol. 16, no. 5, pp. 38-42. peppard, j, edwards, c & lambert, r 2011, 'clarifying the ambiguous role of the cio', mis quarterly executive, vol. 10, no. 1, pp. 31-44. periasamy, kp, & seow, a,'cio: business executive or technical expert?, paper presented to searcc'98, hong kong computer society. hong kong. pervan, g. (1998). "how chief executive officers in large organizations view the management of their information systems." journal of information technology, vol.13. no. 2, pp. 95-109. preston, d & karahanna, e 2009, 'how to develop a shared vision: the key to is strategic alignment', mis quarterly executive, vol. 8, no. 1, pp. 1-8. preston, ds, karahanna, e & rowe, f 2006, 'development of shared understanding between the chief information officer and top management team in u.s. and french organizations: a crosscultural comparison', ieee transactions on engineering management, vol. 53, no. 2, pp. 191206. preston, d, chen, d & leidner, d 2008, 'examining the antecedents and consequences of cio strategic decision-making authority: an empirical study', decision sciences, vol. 39, no. 4, pp. 605-42. quinn, r. e., & rohrbaugh, j. (1981). a competing values approach to organizational effectiveness. public productivity review, vol. 5, no. 2, jun, pp.122-140. quinn, re & rohrbaugh, j 1983, 'a spatial model of effectiveness criteria: towards a competing values approach to organizational analysis', management science, vol. 29, no. 3, pp. 363-77. quinn, r, faerman, s, thompson, m, mcgrath, m & clair, l 2006, becoming a master manager: a competing values approach, john wiley & sons inc. raghunathan, b & raghunathan, t 1989, 'relationship of the rank of information systems executive to the organizational role and planning dimensions of information systems', journal of management information systems, vol.6, no.1, pp. 111-126. ramakrishna, h & lin, x 2002, 'perception of the role of information technology function in organizations: toward the development of a measure', acm sigcpr computer personnel, vol. 20, no. 4, pp. 39-54. richardson, vj & zmud, rw 2002, 'the value relevance of information technology investment announcements: incorporating industry strategic it role', . proceedings of the 35th annual hawaii international conference on system sciences (hicss), 7-10 jan. 2002. robbins, s, & duncan, r 1988, the role of the ceo and top management in the creation and implementation of strategic vision. d.c.hambrik, ed., the executive effect: concepts and methods for studying top managers. jai press, greenwich,ct.pp. 205-233. rockart, j, ball, l & bullen, c 1982, 'future role of the information systems executive', mis quarterly, vol. 6, special issue, pp. 1-14. saldanha, t & krishnan, m 2011, “leveraging it for business innovation: does the role of the cio matter?”. proceedings of the international conference on information systems (icis) 2011. sarbin, t. r., & allen, v. l. 1968 "role theory." pp. 488-567 in gardner lindzey and elliot aronson (eds.), the handbook of social psychology, vol. i. reading, mass.: addison-wesley. schein, e.h. (1989), the role of the ceo in the management of change: the case of information technology, sloan school of management working paper series, paper # 3072, massachusetts institute of technology 1989. australasian journal of information systems volume 18 number 2 2014 83 ---, ( 1992). the role of the ceo in the management of change: the case of information technology. in ta, kochan and m. useem (eds.). transforming organizations. oxford: oxford university press, pp. 80-96. seddon, pb, walker, d, reynolds, p and willcocks, l, 2008 "a case-based assessment of the descriptiveness of three cio typologies and validity of two cio-effectiveness models". acis 2008 proceedings. paper 90. sherer, sa 2004, 'is project selection: the role of strategic vision and it governance'. proceedings of the 37th annual hawaii international conference on system sciences. sivo, sa, saunders, c, chang, q & jiang, jj 2006, 'how low should you go? low response rates and the validity of inference in is questionnaire research', journal of the association for information systems, vol. 7, no. 6, pp. 351-413. smaltz, d 2000, 'the elevation of cio roles: organizational barriers and organizational enablers', journal of healthcare information management: jhim, vol. 14, no. 1, pp. 81-91 smaltz, d, sambamurthy, v & agarwal, r 2006, 'the antecedents of cio role effectiveness in organizations: an empirical study in the healthcare sector', ieee transactions on engineering management, vol. 53, no. 2, pp. 207-22. smaltz, dh 1999, antecedents of cio effectiveness: a role-based perspective, florida state university tallahassee. sojer, m, schläger,c, & locher, c, "the cio – hype, science and reality" (2006).ecis 2006 proceedings. paper 46, http://aisel.aisnet.org/ecis2006/46 spitze, jm, & depaschalis, eg (2005), the “quals” of the ideal cio, scc, retrieved 28 june, 2012, from http://www.scc.cc/voice/cio%20quals%20research%20report.pdf stephens, c, ledbetter, w, mitra, a & ford, f 1992, 'executive or functional manager? the nature of the cio's job', mis quarterly, vol. 16, no. 4, pp. 449-67. stephens, cs 1995, the nature of information technology managerial work:the work life of five chief information officers, westport,ct, quorum books. straub, dw 1989, 'validating instruments in mis research', mis quarterly, vol. 13, no. 2, pp. 14769. strickland, s 2011, how the role of the chief information officer contributes to the organisation. university of manchester. subramanian, gh & nosek, jt 1993, 'the development and validation of an instrument to measure perceived strategic value of information systems', , proceeding of the twenty-sixth hawaii international conference on system sciences, 5-8 jan 1993. synnott, w 1987, the information weapon: winning customers and markets with technology, john wiley & sons inc. synnott, wr & gruber, wh 1981, information resource management: opportunities and strategies for the 1980s, john wiley & sons, inc tannenbaum, r & schmidt, wh 1958, 'how to choose a leadership pattern', harvard business review, vol. march/april, pp. 95-102. tansley, d., loughran, c., edwards, c., lambert, r. & peppard, j.. "realizing value from a cio: navigating the silicon ceiling," deloitte llp and cranfield university report, 2008, pp. 1-16. teo, tsh & too, bl 2000, 'information systems orientation and business use of the internet: an empirical study', international journal of electronic commerce, vol. 4, no. 4, pp. 105-30. tripp, jf., "transformational focus in strategic it vision: an industry-level conceptual model" (2008). mwais 2008 proceedings. paper 7. australasian journal of information systems volume 18 number 2 2014 84 venkataraman, n (ed.) 1991, “it-induced business reconfiguration” in ms scott morton ed. “the corporation of the 1990’s: information technology and organisational transformation”, oxford university press, new york. vroom, vh & yetton, pw 1973, leadership and decision-making, university of pittsburgh press, pittsburgh, penn. watson, r. t. (1990). "influences on the is manager's perceptions of key issues: information scanning and the relationship with the ceo." mis quarterly, vol. 14, no. 2, pp. 217-231. weill, p & broadbent, m 1998, leveraging the new iinfrastructure: how market leaders capitalize on information technology, harvard business school press, boston. weiss, jw & adams, sm 2010, 'changing roles of technology leaders: strategic partners or high level mechanics?', 43rd hawaii international conference on system sciences (hicss), 5-8 jan. 2010. weiss, jw., thorogood, a., & clark, kd (2006). "three it-business alignment profiles: technical resource, business enabler, and strategic weapon." communications of the association for information systems 18: 676-691. werts, ce, linn, rl & jöreskog, kg 1974, 'intraclass reliability estimates: testing structural assumptions', educational and psychological measurement, vol. 34, no. 1, pp. 25-33. wu, j-h, chen, y-c,; and sambamurthy, v, 2008 "the impacts of btm capability and cio role effectiveness on firms' information technology assimilation: an empirical study" . icis 2008 proceedings. paper 76. yang, s 2008, the managerial roles of information technology manager: an empirical study of financial industry in taiwan. masters thesis, national taiwan university of science and technology. zmud, r 1988, building relationships throughout the corporate entity, icit press, washington dc. zuboff, s 1985, 'automate/informate: the two faces of intelligent technology', organizational dynamics, vol. 14, no. 2, pp. 5-18. australasian journal of information systems volume 18 number 2 2014 85 appendix a: extract from survey questionnaire the full questionnaire is available on request from the authors. 1. the organisational strategic it vision: which statement best describes the present vision of your top management team/executive staff of the role of information technology (it) in your organization? (tick one box only) a) the potential of it is cost saving or quality improvement through automation — that is, the role of it is to replace expensive, unreliable human labour, or at least transform its productivity. 1 b) the potential of it is to empower employee driven performance improvement — that is, the role of it is to provide data and transactions that yield a far fuller picture at ‘operator’ level, with members of the staff gaining greater insight into their own activities. 2 c) the potential of it is to transform the organisation — that is, the role of it is to fundamentally change the organisation and/or industry through new products or services often including redefinition of relationships with our customers and/or suppliers. 3 d) the potential of it is increased managerial control of the organisation — that is, the role of it is to provide data and transactions that allow management more clear and organized views of the state and dynamics of the organisation. 4 2. the cio role: the following are general expectations that apply to the cio role in varying degrees in organizations. please indicate how important each expectation is in your organisation. (tick one box only for each question) not at all important critically important 1 2 3 4 5 6 7 (a) keep key systems operational. (b) establish and maintain an it department that is responsive to user requests/problems. (c) establish electronic linkages throughout the organisation. (d) ensure the organization’s users have adequate workstations (pcs/laptops/tablets) to accomplish their jobs. (e) establish electronic linkages to external entities (customers, suppliers, partners, etc.). (f) direct efforts to build an integrated delivery system. (g) build and maintain an it staff with skill sets that match your current and planned technology base. (h) champion digital literacy throughout the organisation. (i) provide insight to the top management team/executive staff (tmt) on new emerging technologies. australasian journal of information systems volume 18 number 2 2014 86 (j) assist top management team/executive staff (tmt) in improving their digital literacy. (k) migrate organisation from legacy, departmental applications to cross-departmental, integrated applications. (l) develop/acquire an electronic document management capability throughout the organisation. (m) develop an understanding of the industry delivery process. (n) provide executive oversight for all it contracts with external vendors. (o) negotiate with vendor it organizations on new external contract proposals. (p) ensure it contracts with external vendors remain within scope and budget. (q) develop and implement a strategic it plan that aligns with the organisation’s strategic business plan. (r) develop/maintain metrics that measure the value of it to the organisation. (s) direct it-enabled business process restructuring/ reengineering. (t) provide expertise on multidisciplinary business process improvement teams. (u) interact often with non-it managers throughout the organisation. (v) be intimately involved in shaping the mission/vision of the organisation. (w) be intimately involved in business strategic planning and decisions. (x) provide oversight for quality assurance of organisational data. (y) ensure confidentiality and security of organisational data. 3cio demographic information: please answer the following questions regarding yourself, your job and your organisation. 3-1. cio reporting structure: to whom do you primarily report? 1 chief executive officer (ceo) 2 chief financial officer (cfo) 3 chief operating officer (coo) 4 other (please specify)________________ 3-2. job title: which of the following categories best describes your current job title? 1 it/is manager australasian journal of information systems volume 18 number 2 2014 87 2 it/is director 3 mis manager 4 edp manager 5 cio 6 cto 7 vice president it 8 other (please specify)________________ australasian journal of information systems volume 18 number 2 2014 88 appendix b: non-response bias test results (mann-whitney u test) items group n mean rank sum of ranks mann whitney u wilcoxon w z-score asymp. sig. (2-tailed) stra1 early late 21 13 17.26 17.88 362.50 232.50 131.50 362.50 -0.19 0.84 stra2 early late 21 13 17.19 18.00 361.00 234.00 130.00 361.00 -0.24 0.81 stra3 early late 21 13 18.07 16.58 379.50 215.50 124.50 215.50 -0.45 0.65 stra4 early late 21 13 17.69 17.19 371.50 223.50 132.50 223.50 -0.14 0.88 stra5 early late 21 13 15.64 20.50 328.50 266.50 97.50 328.50 -1.43 0.15 stra6 early late 21 13 16.05 19.85 337.00 258.00 106.00 337.00 -1.12 0.26 integ1 early late 21 13 18.31 16.19 384.50 210.50 119.50 210.50 -0.62 0.53 integ2 early late 21 13 19.50 14.27 409.50 185.50 94.50 185.50 -1.54 0.12 integ3 early late 21 13 18.40 16.04 366.50 208.50 117.50 208.50 -0.69 0.48 integ4 early late 21 13 16.90 18.46 355.00 240.00 124.00 355.00 -0.46 0.64 rear1 early late 21 13 20.38 12.85 428.00 167.00 76.00 167.00 -2.26 0.02* rear2 early late 21 13 19.71 13.92 414.00 181.00 90.00 181.00 -1.73 0.08 rear3 early late 21 13 19.60 14.12 411.50 163.50 93.50 183.50 -1.63 0.10 edu1 early late 21 13 15.74 20.35 330.50 264.50 99.50 330.50 -1.36 0.17 edu2 early late 21 13 16.10 19.77 338.00 257.00 107.00 338.00 -1.08 0.27 edu3 early late 21 13 15.07 21.42 316.50 278.50 85.50 316.50 -1.88 0.06 utpr1 early late 21 13 18.79 15.42 394.50 200.50 109.5 200.50 -1.00 0.31 utpr2 early late 21 13 18.55 15.81 389.50 205.50 114.50 205.50 -0.81 0.41 utpr3 early late 21 13 18.38 16.08 386.00 209.00 118.00 209.00 -0.677 0.49 utpr4 early late 21 13 19.21 14.73 403.50 191.50 100.50 191.50 -1.30 0.19 infst2 early late 21 13 17.57 17.38 369.00 226.00 135.00 226.00 -0.05 0.95 infst3 early late 21 13 16.17 19.65 339.50 255.50 108.50 339.50 -1.02 0.30 australasian journal of information systems volume 18 number 2 2014 89 infst4 early late 21 13 17.36 17.73 364.50 230.50 133.50 364.50 -0.11 0.90 s.it.v early late 21 13 15.36 20.96 322.50 272.50 91.50 322.50 -1.72 0.08 cio. j.t early late 21 13 17.86 16.92 375.00 220.00 129.00 220.00 -0.288 0.77 cio.r.s early late 21 13 16.07 19.81 337.50 257.50 106.50 337.50 -1.12 0.26 * sig. p< 0.05 microsoft word ajis_18(3)_v10.docx australasian journal of information systems volume 18 number 3 2014 161 an assessment of risk response strategies practiced in software projects vanita bhoola s. p. jain institute of management and research india vanita@spjimr.org s b hiremath national institute of industrial engineering india hiremat.nitie@gmail.com debasis mallik s. p. jain institute of management and research india debasis.mallik@spjimr.org abstract risk management and success in projects are highly intertwined – better approaches to project risk management tend to increase chances of project success in terms of achieving scope & quality, schedule and cost targets. the process of responding to risk factors during a project’s life cycle is a crucial aspect of risk management referred to as risk response strategies, in this paper. the current research explores the status of risk response strategies applied in the software development projects in india. india provides a young it-savvy english-speaking population, which is also cost effective. other than the workforce, the environment for implementation of software projects in india is different from the matured economies. risk management process is a commonly discussed theme, though its implementation in practice has a huge scope for improvement in india. the paper talks about four fundamental treatments to risk response – avoidance, transference, mitigation and acceptance (atma). from a primary data of 302 project managers, the paper attempts to address the risk response factors that lead to successful achievement of project scope & quality, schedule and cost targets, by using a series of regressions followed with seemingly unrelated regression equations (sure) modelling. mitigation emerged as the most significant risk response strategy to achieve project targets. acceptance, transference, and avoidance of risk were mostly manifested in the forms of transparency in communication across stakeholders, careful study of the nature of risks and close coordination between project team, customers/end-users and top management. keywords: project risk management; risk response strategies; software projects introduction information technology (it) projects have inherent risks, some of which can be managed to an acceptable level (kwak & stoddard, 2004). implementation of an it project is a double edged sword – on one hand, it can give tremendous opportunities to gain revenues, and on the other, it can pose a threat to both the it service provider and also the customer (chandra & calderon, 2009). though there is abundance of research in the area of it project risk management, there is serious dearth of such studies in the indian context (thomas & bhasi, 2011). australasian journal of information systems volume 18 number 3 2014 162 the national association of software and services companies (nasscom), a trade association of indian information technology and business process management (it-bpm) industry, predicts that the industry expenditure is expected to grow by 6 percent in 2013-14 and to touch usd 2 trillion, in the next couple of years. information technology (it) and information technology enabled services (ites) provide about 7.5 percent of the indian gdp and expected to increase by another 2 percent in the coming years (nasscom, 2013). on one hand, this portrays a unique opportunity for major it companies in india to capitalize on this, and on the other, it triggers an alarm on a major source of threat, which many indian it firms are blissfully unaware or tend to oversee – the associated risk. the purpose of this paper is to understand risk response strategies in software projects in india and the factors that determine them. based on primary study, data was collected from 302 software project managers and project leads on risks factors and its risk solutions which influence project outcomes. in other words, the focus of this empirical study is on risk response strategies that are practiced to alleviate these risks and its implications on achieving scope & quality, schedule and cost targets. four major forms of risk response strategies are considered avoidance, transference, mitigation and acceptance. the findings of the study open a new dimension of looking at risk response strategies to deal with scope & quality, schedule and cost targets. as the respondents of the study are practitioners in the area of software development projects, the different forms of application of risk response strategy adopted in diversified scenarios, is a significant contribution of the current research. software project risk management software development projects have a dismal track-record of cost and schedule overruns and quality and usability problems (kwak & stoddard, 2004). the focus of kwak & stoddard (2004) is on project risk management by creating an organizational environment and culture which is conducive. in similar lines, doherty & king (2003) and warkentin & willison (2009) discuss how organizational risks stemming from organizational culture, structure and business processes impact technical software development issues, creating a wide range of potential trouble points. a study revealed that 65 per cent of the project failures were due to management issues (mcmanus, 2004). alter & ginzberg (1978) focused on problems associated with the organizational acceptance and implementation of the information system. they argued that implementing any system would involve uncertainty from the managerial point of view. while the focus of the abovementioned papers has been on organizational issues that influence project risks, other factors such as those which are project specific, human resource related or associated with the overall environment in which the project is to be implemented are ignored. apparently, this implies that software project development is a tightrope walk. therefore, managing risks with a holistic perspective is of primary importance in software project development, especially in the case of long duration software projects (yong et al., 2006). boehm's model (1991) suggests a comprehensive set of steps and guidelines to manage software development risks. for each of the top 10 risk items, boehm developed a set of risk-management techniques. these risk management practices include award-fee contracts, benchmarking, contracts, design to cost, early user's manuals, incremental development, information hiding, instrumentation, mission analysis, morale building, pre-award audits, pre-scheduling, prototyping, requirements scrubbing, software re-use, team building, tuning, user surveys, compatibility analysis, competitive design, cost-benefit analysis, cross-training, detailed multisource cost and schedule estimation, inspections, job-matching, modeling, organizational analysis, reference checking, scenarios, simulation, staffing with top talent, task analysis, technical analysis and user characterization. having said that, the model does not allow for continuous improvement of project performance, through updating changing risk profiles in software projects. davis' model (1982) is based on the basic premise that projects often use inappropriate methodologies for obtaining and documenting user requirements. hence, he suggests “packaging” different methods into alternative strategies, which can be then linked to risk management model. these strategies include australasian journal of information systems volume 18 number 3 2014 163 asking from users, deriving from existing systems, synthesis from characteristics of the utilizing system and discovering from experimentation. established software companies dealing with projects would, by contrast, have effective systems and methods for procurement and documentation of clients’ requirements. cavusgil & sikora (1988) discuss reactive and proactive strategies for countering gray market channels. while reactive approach can help project managers and organizations to prevent repetition, proactive risk management has to be more useful as it focuses on the minimization of the likelihood of the emergence of a risk factor, instead of waiting for its occurrence. focusing mostly on large long-term projects, pavlak (2004) discusses the importance of maintaining a balance between reactive and proactive risk management strategies in projects. in similar lines, alter & ginzberg (1978) recognized and classified several risk resolution strategies into inhibiting or compensating strategies. inhibiting strategies are ex ante while compensating strategies are ex post. mcfarlan (1982) classified risk resolution techniques into four types, namely external integration, internal integration, formal planning and formal control mechanisms. this simplified outlook of classifying risks primarily focuses on organizational or corporate culture and planning and control mechanisms, largely ignoring the greater challenges of risk response strategies developed by later authors like abdul-rahman et al., (2012). rivard et al., (2011) discusses the relevance of estimating risk exposure. it implies assessing the loss associated with a deviation from each of the project’s objectives and the probability of occurrence of each of these undesirable outcomes, hence estimating risks in the beginning of the projects will help mitigating risks. an extended discussion on identification of risk and strategies to overcome such risks would have further strengthened the research. in the above literature we have observed the diversified approaches used by researchers in developing risk management practices in software projects. one of the key elements of risk management is controlling these risks with appropriate response strategies. the next sections discuss these aspects of risk management. project failure and dynamic risk management keil et al., (2013) documents that it/is projects are mired with problems related to poor process performance, budget spillovers and cost overruns. according to the authors, exercising formal and informal monitors and controls or identifying and managing key risks can help prevent project failures, but the authors admit that there are no full-proof mechanisms to ensure project success. keil et al., (1998) asserts that too many software projects end up in failure. hence, identifying and countering the threats to failure, is the key to success. according to wallace et al., (2004) the inability to identify and understand the depth and breadth of inherent project risks and respond to them accordingly, is a major cause of project failures. abdul-rahman et al., (2012) identifies risk uncertainties as the major causes to software project failures, which generally manifest in the form of cost and time overruns. though failure is majorly identified from risk uncertainties and manifest in the form of cost and time overruns, the process of alleviation of such risks lacks empirical evidence. drummond, h. (2005) studies a small business to discuss the concept of “entrapment” of organizations in not embracing change as a part of organizational development. complacency to restore existing gains, risk aversion, myopic vision, ego-defensiveness and social pressures are some of the attitudinal deficiencies, discussed by drummond (2005), the restrain organizations to achieve sustainability and growth. schmidt, et al., (2001) disclose the sad state of affairs of software projects, internationally. according to the authors 25 percent of all software projects are cancelled outright, 80 percent of all software australasian journal of information systems volume 18 number 3 2014 164 projects encounter cost overruns and 75 percent of large systems are operational failures. hence, the sticking point here is to identify and control risk occurrence (boehm, 1989; karolak, 1996). lehtinen et al., (2014) study the causes for software project failures and their interconnectedness. using a root cause analysis, the reasons for project failure were detected. lack of cooperation, collaboration and communication, task backlogs and lack of software testing resources were some of the common causes across tasks, people and processes that were responsible for project failures. the nature of it/software risks is often dynamic in nature. to develop a comprehensive risk management plan based on past data and information, often repeated occurrences to risks are unavailable. hence, risk assessment on a current project is mostly based on soft data, rather than hard data from historical experiences (gasparini, et al., 2004). even then, the company will surely make an attempt to counter risks by taking preventive actions through thorough assessments on a dynamic basis across phases of the project life cycle. this leads to a process of dynamic risk control using preventive steps and assessment, in an iterative manner (gasparini, et al., 2004). an example of a dynamic risk management loop, in the form of re-scoping loop of assessment, has been provided by the authors as given in figure 1. figure: 1: dynamic risk management loop (gasparini, et al., 2004) dynamic risk control typically has the rescoring and the prevention loop. the job of the preventive loop is to change the future course of action of the perceived risks, based on the changed scenario and subjective risk assessment. iterative exercises performed by the project team to identify new risks, assess and monitor, reassess and perform the same process throughout the project life cycle is an important component of dynamic risk management. the paper, built on bayesian approach, is theoretical. the paper fails to illustrate how the procedure can be implemented over multiple projects with diversified complications. in the context of software project management, there is serious dearth of research in dynamic risk management. the reasons for the same could be two-fold. duration of software projects is generally short. hence, the iterative process of learning from previous mistakes, identifying new risks and providing new set of risk response strategies is not very relevant. effective planning, continuous monitoring and timely implementation are often the keys to success. with the advent of agile methodologies, software projects can now face dynamic business risks and changing clients’ requirements more effectively (augustine et al., 2005). hence, the scope of the current research is static risk management and not an iterative process of implementing changes through learning from previous mistakes within the same project life cycle. this necessitated the creation of a comprehensive and exhaustive list encompassing all the risk response strategies as discussed in the previous sections. australasian journal of information systems volume 18 number 3 2014 165 risk response strategies the success of the information technology (it) projects is influenced by various factors like customer involvement, schedules and budgets, flexible project tools, and a positive attitude towards project success (pieterse, 2012). agarwal & rathod (2006) find that the performance of internal stakeholders is the most significant determinants of project success. they propose that the cost, time and scope (software functionality and quality combined) are the parameters that signify the internal characteristics of the process adopted in software projects. the study, as it appears, depends on a purposive sample, rather than a random sample and limits the evaluation of project success indicators from the standpoint of cost, time and scope. march & shapira (1987) study the variability in the outcome of project success in the context of decision makers’ attitudes to risk aversion. the authors suggest greater the variance involved, higher should be the return on investment, and vice versa. the paper does not discuss the complications involved in risk identification and the factors that lead to high diversity leading to changing patterns on risk solutions. investing on high risk projects with high returns is acceptable, provided risk calculations are in place, explicitly (armour, 2005). managerial issues related to risk response strategies therefore appears to be more critical than technical issues (abdul-rahman et al., 2012). fairley & willshire (2005) coined the term ‘risk handling’ that implies a course of action taken in response to risk factors that includes risk acceptance, risk avoidance, risk transfer and risk mitigation. the study defines risk response strategies which are categorized as: avoidance, transference, mitigation and acceptance. once risks associated in a project are identified, a proper risk response strategy has to be formulated and be made accountable for implementation. each of these responses to the identified risk would require a responsible owner. risk response is decided based on the risk category, proximity, probability, frequency and impact on the project (hillson, 2002, pmbok®, 2013; salmeron & lopez, 2012; tesch et al., 2007). identification of risks alone cannot ensure project success. implementation of risk response strategies is crucial to meeting project targets. using integer programming model, zhang & fan (2014) propose a novel method for selecting the right risk response strategy in an iterative manner. risk response strategies, in this study, are broadly classified into four categories namely, avoidance, transference, mitigation and acceptance. the study could have been further enriched by considering the contribution of human elements like, attitudes, incentives, experiences, etc. it also considers the occurrence or nonoccurrence of each risk element independent of the other risks. fairley (2005) states that risk handling includes all the four types of risk response strategies namely, avoidance, transference, mitigation and acceptance. fairley (2005) suggests that selection of a particular response to risk should always be a strategic decision. the next sections discuss each of these in greater detail. avoidance eliminating activities with a high probability of loss by making it difficult for risk to occur, or by executing the project in a different way which will achieve the same objectives but which insulates the project from the effect of the risk can be termed was risk avoidance. a snapshot of risk avoidance variables is given below. communication between project head and management is crucial to the successful implementation of project. this is generally influenced by the principal–agent relationship between the parties and the contract type chosen (müller & turner, 2005). bond-barnard et al., (2013) show that a balance between formal and informal communication between project manager and other stakeholders reduces mistrust australasian journal of information systems volume 18 number 3 2014 166 and conflict of interest. a study conducted by fisher & urich (1999) introduces the models of instrumental and transformative participation and the way they influence flow and communication between project manager and other stakeholders. rosenkranz et al., (2013) argues that knowledge transfer, communication, and shared understanding between project stakeholders are important requirements to projects. the ability of the project manager to facilitate communication among stakeholders, create the desired commitment level and reduce uncertainty can help avoiding the risk of project failure (burström & jacobsson, 2011). blackstone et al., (2009) points out the importance of project managers’ skills and leadership capabilities, user involvement, top management commitment and organizational engagement in successful implementation of projects. shiferaw et al., (2012) find that weak links between project stakeholders affect the effectiveness of project governance system. laursen (2011) asserts that there are often weak links that exist between what is known by the staff as “ordinary problems” of organizations and the objectives and goals of the project. though these studies discuss about the weak links they fail to consider the softer issues related to attitude towards work, personality traits, motivating factors, interpersonal conflicts, etc. and its impact on project stakeholders. while investigating the role of phased project planning in project success, tasevska et al., (2014) study four measures namely, business case development, scope planning, baseline plan development and risk planning. kutsch et al., (2011) state that among many reasons behind project failure, ‘planning fallacy’, i.e., over-optimism in the planning phase in the project due to resource misallocation and miscommunication is crucial. it is preferable if a project is budgeted, one phase at a time, instead of budgeting at a time. khamooshi & cioffi (2013) develop a model for phase-wise project budgeting and scheduling under uncertainty. conversely, projects which involve cross-cultural teams working together from remote locations often require an overall plan and budget which is implemented through intertwining of phase-wise planning and budgeting. keil et al. (1998) discusses the importance of contingency plans that constitutes a serious threat to the successful completion of a software development project. thal & martínez (2011) suggest that contingency plans help project teams to deal with uncertainties such as, handling new product development, enforcing innovative actions, increase plan flexibility, etc. hanisch & wald (2012) studies 1,622 articles and present a meta-analysis of project contingency theory as appears in project management journals. a comparative analysis of the significance of contingency planning vis-à-vis ex ante and ex post risk management, is however, lacking. several projects use a preliminary feasibility study to facilitate the analysis of go/no go decisionmaking processes (sungmin & caldas, 2009). alkass et al., (2006) appraise that feasibility study can help in selecting the best alternative during the strategic planning phase of a project, more so before the starting the project life cycle. quartel et al., (2012) use an architecture-based approach to it software project evaluation. the approach offers organizations an instrument to assess the business value of a project in terms of its contribution to a selected set of business goals. software industry is heavily technology-driven and hence, changes and upgrades frequently. a pilot run is often conducted to appreciate whether the investment on a new technology is worth (cardenas et al., 2014). though running a pilot survey is gratifying in exploratory research, it is often a costly affair and does not provide a true platform to ensure successful implementation of software. continuously meeting on agreed customers’ requirements at the lowest cost, without sacrificing on quality and other deadlines, after realizing the potential of the existing team members and the support from organization, has its inherent risks (scurr, 1990). abd et al., (2014) finds that companies often australasian journal of information systems volume 18 number 3 2014 167 employ flexible manufacturing systems (fms) capable of dealing with unexpected events and constantly changing customer expectations. the implementation of agile methods can often be a precursor to employing fms in software projects can be difficult. performing an in-depth analysis of the risks associated with assignment of roles and responsibilities of key stakeholders that includes project team members, clients, consultants, and contractors can avoid project failure (doloi, 2013). a roles and responsibility assignment matrix including the participation of all concerned stakeholders is useful (melnic & puiu, 2011). tesch et al. (2007) suggest that the keys to managing risks at each stage of the project are to assign an experienced project manager skilled in change management and monitoring progress. this can act as an avoidance strategy to provide risk solutions. parker et al. (2013) states that the high failure rate of change interventions suggests that improvements are required in terms of better management, monitoring and control. transference transferring a portion or entire risk to a third party, by identifying another stakeholder to manage the risk activities with low probability of recurrence, but with a large financial impact, is termed as risk transference. this section discusses the various variables and their reference to literature with risk transference. project failures highlight the importance of measuring, understanding and managing risks. identification of the sources of risks and breakdowns in detecting risks can provide significant clues to project failures (eicher & ruder, 2007). identifying the root cause and detecting the sources of risk can help in risk transference in an efficient manner. byoun et al., (2013) finds that more leverage is used when project risk is high and projects use less leverage in the presence of risk-reducing features including off-take agreements. there are a set of attributes that ensures the presence of risk in different phases of the project based on changes in the external environment, cost reduction attributes, etc. (zavadskas et al., (2010). a detailed knowledge sharing on the nature of the risks identified and evaluated, can enable shifting risks to another phase of the project to avoid delays, cost overruns, change orders and avoid design omissions and errors from unforeseen contingencies (rodriguez et al., 2010). this runs the risk of not mitigating it at the source and transferring it to another phase where the degree of risk may escalate or change its form. chan and au (2007), in this context, argues that when risk is transferred there is generally a time overrun in the project, as the team members and the nature of job in changed phase could be different. when it is possible to shift risk after taking a well-thought-after decision and clearly communicating the impact to the possible stakeholders, mitigating risk is faster and smoother (zofi, 2012). when risk transfer is not smooth and adequately disclosed, mismatch in the expectations of related parties may lead to the emergence of new risks. these risks need to be examined and evaluated for cost effective risk management plans (golden et al., 2007). skyner (2006) discusses the requirement of balance that is to be maintained between risk transference and risk mitigation. it is not necessary that effective functioning of the prior smoothly leads to the later. with the presence of unclear regulations, lack of transparency, changing market conditions and lack of proper communication between stakeholders, transference of risk may create problems in mitigation (rosenbaum, 2005). often risks originating from external environmental factors like natural hazards or war are transferred to the end users or customers in the form of time or cost overruns. this can lead to mismatch of expectations between stakeholders, thereby leading to the creation of a new set of risks (golden, et al., australasian journal of information systems volume 18 number 3 2014 168 2007). evolution of such new risks must be examined and evaluated as a part of ascertaining cost effective risk management plans. mouraviev & kakabadse (2014) discuss risk transference from the point of view of public private partnership (in short, ppp). they argue that even when the transference is smooth, the cost and time involved in such process are often extensive and the governments’ financial outlays are seriously compromised. armstrong & paolucci (2010) argue that risk transfers within a project improve efficiency, if it is a well-thought-after decision with mutual agreement, knowledge transfer and communication. risk transfer and delivery efficiency of the party or project phase to which risk has been transferred go hand in hand (witt & liias, 2011). the papers discuss risk transfer from the viewpoint of sectors like insurance and construction in the context of ppp. hence, the applicability of the arguments in favor of risk transference is yet to be tested for the it sector. mitigation reducing the risk in order to make it more acceptable to the project or organization, by reducing its impact can be termed as mitigation of risk. tesch et al., (2007) identify several mitigation strategies as risk response solutions. as a mitigation strategy the authors suggest escalating risk issues to top management, obtain signoff on commitments and stop the project and discuss with sponsor and management on further steps. in case there is lack of commitment from the management or the customer, the authors also suggest working with them to understand the reasons for indifference. laurentiu & gabriela (2013) discuss the importance of a cost-benefit analysis on existing risks in the project. the authors suggest using a sensitivity analysis to identify risk parameters that may impact during project development and operational period and may lead to failure and varied points in the project life cycle. funding plays a crucial role to conduct risk mitigation activities and enabling the system to restore its usual functioning (hecker, 2002). funding deficits are an integral part of cost of time overruns. infrastructure projects are prone to more funding deficits than projects in manufacturing or even software sectors (little, 2010). such funding deficits are more prevalent in large infrastructure or multihazard mitigation projects with where investment stakes are high. according to vizard (2008) it firms have enough funding for their ongoing projects, but lack funding for infrastructure required for business continuity. according to goble & bier (2013) periodic communication of risk assessment results can mitigate risks in projects. according to the authors risk assessments are repositories of structured information and a medium for communication. hence, the judicious use of risk assessment tools with adequate communication can mitigate risks to a great extent (veil & husted, 2012). alexandra-mihaela & danut (2013) point out that internal communication is one of the most important factors for success in project management. project manager should tract the internal communication to ensure project deliverables to make ends meet (aubry, 2011). an effective project team’s skills, experience motivation and teamwork are enhanced with a proactive and supportive management (bagdonien , 2012). the significance of proper communication between stakeholders has been discussed by various researchers from the viewpoints of different risk response strategies. from the perspective of risk mitigation, internal communication within project team and immediate stakeholders within the organization commands more relevance. having said that, this does not undermine the significance of periodic review meetings with clients and customers to enhance risk mitigation (swink, 1996). as the nature of project complexity and scope are different, different processes or more strict and extensive use of well-known practices are required with specific tools and techniques (besner & hobbs, australasian journal of information systems volume 18 number 3 2014 169 2008). according to bolles et al., (2006) project management requires total commitment and support of executive management and strict adherence to documented processes. project success and client satisfaction are results of collaborative actions by project actors throughout its life cycle. hence, risk mitigation is effectively conducted if all changes are discussed with both customers and top management (holzmann & panizel, 2013). the focus of the paper revolves around communications management in agile software development projects and not managing risks using diversified response strategies. acceptance recognizing that residual risks (i.e., risk that remains after a risk response has been taken) will exist and responding either actively by allocating appropriate contingency, or passively doing nothing except monitoring the status of the risk can be termed as risk acceptance. johnstone (2000) identifies three strategies to ensure effective evaluation and acceptance of risk factors. these strategies are as follows: firstly, screening all stakeholders based on their risk profile; secondly, stringency of reporting standards; and lastly, more proactively using risk adaptation strategies. discussing on software project risk management, mcmanus (2003) points out that if risk is evaluated and acknowledged, risk planning is easier. risk acceptance would also mean that taking no action on risk was a carefully thought-after decision. hence, if a decision is taken not to take any action of the existing risk and to accept it the way it is, it can be termed as risk acceptance strategy (fairley, 2005). risk acceptance can act as a double-edged sword, if not monitored and inspected by senior management. it can become a potential threat to organizations if it crosses a predetermined threshold level, thereby raising other forms of risks. according to sang-hoon & elsaid (1985), risk evaluation of a project is an examination process of whether the positive net present value (npv) estimate provides a large enough safely margin to compensate for the uncertainties with its estimated cash flows. yelin et al., (2012) designed a practical and computerized risk evaluation model where a risk hierarchy structure composed of 17 weighted risk factors are developed. the risk evaluation model is then developed using a fuzzy synthetic evaluation approach. despite the availability and practice of such sophisticated risk identification, assessment methodologies and response strategies, projects fail. this can be partly because of the gap between theoretical foundations of risk management and its practical implementation and partly because of the organizational approach to risks. unless the attitude to risk response follows a top down approach with management supervision, implementation at the project level often fails, despite the availability of several control mechanisms to achieve targets. berkel (2014) analyses that risk evaluation and monitoring process are often seen as administrative burden by project team members. quadruple targets software projects are often entangled with constraints. a good project manager should be able to balance his resource requirements and plan his work procedures effectively to ensure achievement of scope & quality, schedule and cost targets to encounter risks (samáková, 2013). achieving scope & quality, schedule and cost targets are the major goals for successful implementation of projects. while the triple targets of scope, time, and cost were the major objectives to be achieved to meet the goals of the projects, at the same time, the delivery of quality product to the client cannot be ignored (pinto & mantel, 1990). by contrast, thakurta (2013) considers quality delivery of a project to be one of the most important determinants of project success. scope creep directly impacts the project quality. improper scope australasian journal of information systems volume 18 number 3 2014 170 planning or identifying requirements leads to scope creep, which, in turn, can adversely impact project quality, e.g., high severity defects resulting in deterioration of product quality (ferreira et al., 2009). a project is considered successful if it achieves its business objective, within the defined scope of work, agreed upon schedule and cost budgets and with acceptable quality. project management is performed to achieve these objectives by managing various aspects of the project such as scope, time, cost, quality, human resources, organizational culture, communications, and procurement management in an integrated manner (mir & pinnington, 2014). most researchers concentrate on the impact of software risk management on achieving the quadruple targets of scope & quality, schedule and cost. a closer look into the quadruple targets is given below. scope: project scope determines the boundaries of the project. it specifies what features and characteristics of the product and the functionalities that is included or not included in the project. in project management there are two types of scope – product scope and project scope. product scope is what the project would create at the end. project scope describes all the work involved in creating the product scope. it includes all the required work to complete the project deliverable (pmbok®, fifth edition). quality: quality represents the extent in which the product is fit to meet customer’s expectations. it has agreed functionalities as per set standards, so that there is value for money for the customer. the purpose of project quality management is to ensure that the project will satisfy the needs for which it was undertaken. quality, therefore, must be on equal footing with project scope, time and cost. if the stakeholders are not satisfied by the quality of the deliverables, the project can be termed as unsuccessful or failure (tuman, 1986). schedule: schedule target is a limitation in a project that affects the start and end date of an activity. it takes form of fixed dates for activities. projects have a variety of schedule constraints such as, imposed dates, key resource availability, logical task and activity duration limitation, etc. the goals of schedule targets include knowing status of the schedule, influencing the factors that cause schedule changes, determining that the schedule was changed, and managing changes when they occur (lewin, 2001). cost: of all the above constraints cost keeps most businesses awake. the three challenges posed by cost constraints are lack of accountability, cost centers as commodities and unexpected budget cuts. to keep projects on track, project managers should also try to identify and eliminate non-critical physical resources before planning to retrench labor. labor retrenchment or high attrition rates often leads to increased responsibilities or outsourcing jobs to independent contractors (doloi, 2013). motivation behind the study india provides a young it-savvy english-speaking population, which is also cost effective. other than the workforce, the environment for implementation of software projects both in the manufacturing sector as well as the services sector, appear very conducive in india. a comparative study on us, china and india on implementation of software projects, shows india on a high pedestal (sykes, 2014). over the past decade or more, india has emerged as a leader in it offshore outsourcing world, by significantly expanding its offering across different verticals to enhance the depth of services. india’s advantage lies in its ability to provide a complete spectrum of services related to it, in english-speaking countries, especially the us. hence, the current study focuses on the indian context of software project management. there are many studies conducted in the area of software project risk management with a perspective to implementable risk solutions. yet software projects fail. this shows that there is a gap between the researchers and practitioners approach (geethalakshmi & shanmugam, 2008). most of the research has been done in western countries. by contrast, the indian scenario is largely different. this is because the australasian journal of information systems volume 18 number 3 2014 171 indian market is not matured, approach to risk management is either non-existent or rudimentary and organization and hr perspective to risk management is highly unstructured (geethalakshmi & shanmugam, 2008). taking these factors into consideration, there are not many studies conducted on software project risk in india. thomas & bhasi, (2011) reaffirm the above and state that there is scarcity of literature on software project risk management, especially from india. research objectives and hypotheses the objective of the study is to determine the impact of diversified risk response strategies on achievement of project targets and overall project success. risks can be either avoided, transferred, mitigated or accepted, each of which or a combination of which can have impact on the project outcomes. hypotheses for the study are divided into two components. hypotheses h11 to h14 are related to multiple regression models considering the achievement of project targets at an aggregative level. the next set of hypotheses i.e., h21 to h24 are associated with the equations of the sure model. the overall structure of the hypotheses is presented in figure 2. the rightward portion of the figure describes individual multiple regressions, while the leftward portion captures the composite impact of the variables from the four risk response strategies on aggregate target achievement in the form of a set of simultaneous equations model. the hypotheses of the study can be summarized as under. figure 2: structure of the hypotheses h11: risk avoidance variables and factors enable meeting quadruple targets of scope & quality, schedule and costs targets; this hypothesis tests whether the factors associated with risk avoidance influence achievement of project targets. accepting that risk response can range from avoidance to acceptance, cook (2005) australasian journal of information systems volume 18 number 3 2014 172 argues that risk avoidance can be a serious affair, if the chances of risk mitigation are unacceptably low. koller et al., (2012) discusses how risk-averse organizations can reduce risk by using effective risk avoidance strategies. h12: risk transference variables and factors enable meeting project targets of scope & quality, schedule and costs; the project management team often finds it convenient to transfer risk from the core project team to the it department, especially in the case of software projects, despite tight controls and professional approaches (legault & chasserio, 2012). javed et al., (2013) discusses how project target achievement can be enhanced with appropriate risk transfer, after understanding users’ requirements through clear and concise output specifications, detailed study of the impact of the transfer and the costs and time associated with it. h13: risk mitigation variables and factors enable meeting targets of scope & quality, schedule and costs targets; the next set of hypotheses discusses whether risk mitigation, as a risk response strategy influences meeting project targets. verner et al., (2014) identified 85 risks and 77 risk mitigation strategies, categorizing them into four groups namely, outsourcing rationale, software development, human resources, and project management, the largest group of risks being related to project management. h14: risk acceptance variables and factors enable meeting targets of scope & quality, schedule and costs targets. according to drummond (2005), risk acceptance primarily consist of issues like, inaction against risk and its impact, evaluation of impact of risk occurrence, consciously accepting risk, evaluation of impact and time frame of risk, monitoring of change in the nature of risks across phases, and so on, that affect project outcomes. the hypotheses h21 to h24 captures a group of four equations considered simultaneously. however, the independent variables in each of these equations are filtered based on the rejection of the null hypotheses, which are selected based on the findings of the prior set of regressions. the dependent variable in these set of equations remain the same, i.e., achievement of overall project targets. the statements of alternative hypothesis for each of these equations are given below. h21: selected avoidance variables influence achievement of overall project targets; h22: selected transference variables influence achievement of overall project targets; h23: selected mitigation variables influence achievement of overall project targets; h24: selected acceptance variables influence achievement of overall project targets. it is expected that when the set of equations are considered in a composite manner the influence on the dependent variable may be captured differently. the output of the sure model will throw further light on this aspect. methodology the analytical framework of our study is based on estimating overall achievement of project success in terms of scope & quality, scheduling and cost targets – henceforth referred as ps_sqsc. the study is based on how various risk response strategies, namely avoidance (avd), transference (tfr), mitigation (mtg) and acceptance (act) impact ps_sqsc. in other words, these four factors, i.e., avd, tfr, mtg and act (referred to as atma, in short) form the independent variables in the study. these four factors constitutes of 15, 9, 14, and 8 items, respectively as described in table 1. australasian journal of information systems volume 18 number 3 2014 173 the study is based on primary data, which was collected using a questionnaire. in the absence of any existing validated questionnaire that deals with risk response strategies in software projects, the researchers were required to develop a questionnaire. to design this questionnaire the study has extensively used the different facets of factors that influence the identification of risks in software projects and the risk response strategies as described by pmbok®, (2013). the questionnaire also incorporated inputs from literature, extensively (tesch et al., 2007, carcary, 2013, stoneburner et. al., 2002). key informant technique was used to interview knowledgeable people in the industry and academia to understand whether such a study should be conducted. a long list of papers, as mentioned in literature was referred, to prepare an exhaustive list of variables and four categories, namely, avoidance, transference, mitigation and acceptance, was created. tesch et al., (2007) have provided a detailed list of factors for two broad categories of risk response strategies, namely, avoidance and mitigation. carcary (2013) has mentioned about the four risk handing strategies as considered for this study. the abovementioned paper builds more on capacity maturity model perspective, rather than focusing on risk response strategies. sillars & o’connor (2009) discusses about alternative design solutions to reduce risks. the initial section of this questionnaire attempted to gather the personal profile of the respondents. most of the questions were administered over a 5-point likert scale, with 1 representing “strongly disagree” and 5 representing “strongly agree”. as coding of variables and questions is crucial in quantitative research, all questions were coded before administering the questionnaire, which was later decoded to covert in an excel worksheet for constructing quantitative models. care was taken to ensure that the questionnaire does not consist of leading questions. in certain cases, when there was a scope of answering in a specific direction or pattern, the question was changed accordingly. in such cases, questions were based on operational aspects or project management, rather than asking what should be done. for example, in a likert scale, two types of statements can be ranked. statement – i: “the project manager attempted hiding project complexity from the customer.” statement – ii: “it is not necessary to disclose all the complexities associated with the project to the customer.” statement – i is a leading question as compared to statement – ii, as the respondent would not easily agree that project complexity was hidden from the customer. this ensured proper diversity in responses across respondents. risk avoidance, as a risk response strategy typically considers 15 variables that address issues like, adequate contingency plans, analysis of customers’ requirements, planning of project in phases, commitment made only after carefully studying team’s skillsets, monitoring of progress and change management, evaluation of business value of the project, pilot run of new technology, careful study of alternatives to new technology, etc. the next response strategy is risk transference. it consists of 9 variables and addresses questions like, whether the impact of risk transference was carefully studied, whether new risk was created after being transferred, whether mitigation was easier after risk transference, whether impact on overall risk was studied after transference decision was taken, whether risk transfer was a well thought after decision, whether cost and time of the risk transfer was low and whether transfer of risk increased project efficiency. mitigation was the next risk response strategy consisted of 14 variables. the issues covered under mitigation generally included concerns like careful study of procedures, escalation of risk issues, when required, reconsideration of cost-benefit analysis, management’s perspective to risk, awareness of funding requirements, willingness of management to work closely with project team and provide adequate support, clear communication and information flow across stakeholders, keeping customers informed and updated at all phases of project and conducting periodic meetings, etc. australasian journal of information systems volume 18 number 3 2014 174 the last risk response strategy is risk acceptance comprising of 8 variables. risk acceptance, as a risk response strategy primarily consist of issues like, not taking any action on risk and its impact, careful evaluation and acknowledgement of the probability of occurrence of risk, placing risk factors in watch list and studying their impact, conscious decision on accepting risk as it is, evaluation of impact and time frame of risk, monitoring of change in the nature of risks across phases, etc. acronym description acronym description avoidance variables mitigation variables avd_cpm communication between project head and management mtg_pri if required, stop project or escalate risk issues avd_csh communication between project head and stakeholders mtg_crs lack of commitment on risk issues studied avd_wl project head identifies and resolves weak links mtg_mps management's perspective to risk studied avd_pp project planned in phases mtg_mct management's readiness to accept other communication tools avd_pbt project budgeted, one phase at a time mtg_cbc re-evaluate cost-benefit calculations. avd_acp adequate contingency plans are maintained mtg_frp awareness on funding requirements, at each phase avd_pfs project feasibility study was conducted mtg_crp communicate risk assessment results periodically avd_bve business value of the project was evaluated mtg_sfd keeping stakeholders informed about funding deficits avd_pnt pilot run of new technology mtg_mdm management aware of deliverables & means to meet avd_cra risks to meeting customers’ requirements analyzed mtg_wmw willingness of management to work with project team avd_ant alternatives to new technology mtg_psf processes strictly followed avd_rrm roles and responsibility matrix mtg_agc updating customers through regular meeting avd_css commitment was made after studying skills mtg_acc project achievement gap communicated to customer avd_cmm progress and change management was monitored mtg_dcm all changes discussed with customer and management avd_upf end user participated in feedback australasian journal of information systems volume 18 number 3 2014 175 transference variables acceptance variables trf_dsr possibility of detecting source of risk act_rea risk factors evaluated and acknowledged tfr_prs presence of risk for each phase studied act_rws risk factors were in watch-list and & studied tfr_ors impact of shifting risk to another phase studied act_rtd no action of risk was taken as a decision tfr_rts impact on overall risk after shifting, studied act_rep probability of occurrence of risk evaluated periodically tfr_rtm when risk transferred, it is easily mitigated act_rfe impact and time frame of risk factors evaluated tfr_tnr transfer of risk does not create new risk act_rsa each risk factor studied for immediate action tfr_trl cost and time to transfer risk is low act_crm change in nature of risk across phases, monitored tfr_rie transfer of risk increases efficiency act_tri taking no action on risk & its impact tfr_rtd risk transfer was well-thought decision table 1: the list of variables constituting atma factors are as listed below: questionnaire design and sample selection to identify the sample for the above administered questionnaire, a "population frame" was first defined. the population frame helped us addressing a select group of respondents, yet maintaining the randomness and representativeness of the population. in order to do so, it was a desirable trait that the project managers who were interviewed had some knowledge about risk management, and the methodologies, techniques and tools in this area. also, companies that they work for should be undertaking projects regularly, conforming to definition of project, as per the pmbok® guide. keeping this in mind, the following criteria were identified to determine the population frame. before selecting the sample, it was checked whether the respondent had at least minimum of 5 years of work experience and had attended a training program in project management. the whole focus of data collection was concentrated on it and ites companies. no response was taken from any individual, unless the project size she dealt with was more than inr 10 million or more. there was no rationale in deciding this cut-off – the purpose was to make sure responses from very small projects are avoided. from pmi india chapters training centers (registered education providers) were identified. from this exhaustive list of 70 training centers a database of companies which provided project management training was identified. out of the 70 training centers, 31 centers shared this data, under strict confidentiality clause, where it was agreed upon that the data will be used solely for academic purpose, without disclosing names of companies or respondents. in some cases, individuals were directly contacted for response, while in others, the training & development department or the hr departments were contacted to identify employees who could be potential respondents. in this way, the online questionnaire was circulated using survey monkey, a commonly used online survey portal, to 3448 persons. out of these, responses were received from 1206 respondents. out of these 1206 respondents, there were 612 responses which were incomplete, despite repeated follow-ups. from the rest 594 responses, 292 responses were invalid. in these 292 cases, it was observed that the respondents, may be due to certain pressure from seniors, had filled in the responses, but there was no variability across answers, or the cross-check questions proved that they did not respond seriously. for example, for australasian journal of information systems volume 18 number 3 2014 176 several respondents it was observed that all the responses had a single answer “5”. hence, after removing all such irrelevant data, the researchers were left with 302 responses. later, coding / decoding was performed on the raw data from 302 respondents for further statistical analysis. once the questionnaire was designed based on the literature, key informant technique was used for taking inputs from five reviewers who serve as project management trainers in the corporate sector. before administering the questionnaire, a pilot survey was conducted amongst 7 pmi trainers and12 project leads. critical comments from these 19 responses were incorporated in the questionnaire. this process was very useful and strengthened the questionnaire. data was collected using an online tool called surveymonkey, that automatically helps collaging the data in excel worksheets. the format of the questionnaire in survey monkey required coding for several questions. hence, after collection of raw data, the excel worksheets were processed by decoding the data and converting it to usable format. analytical framework to test for internal consistency of these items, derived from likert scale, cronbach’s alpha was used to test for reliability of data. the cut-off for cronbach’s alpha was 0.70. after the reliability test the number of items for the variables avd, tfr, mtg and act and reduced to 14, 8, 14 and 8 items, respectively. the summary results of cronbach’s alpha is given in table 2. variable categories initial cronbach's alpha original number of items revised cronbach's alpha revised number of items list of items deleted avoidance (avd) 0.6865 15 0.7232 14 alternatives to new technology (avd_ant) transference (tfr) 0.6846 9 0.7989 8 possibility of detecting source of risk (trf_dsr) mitigation(mtg) 0.8473 14 acceptance (act) 0.7471 8 table 2: reliability test: summary results from cronbach's alpha quantitative models were performed in two stages – in the first stage four multiple regressions were conducted considering ps_sqsc as the dependent variable in each case and the variables under the atma factors as the independent variables. multiple regressions were performed using eviews 8.0. all the regression outputs are white’s heteroscedasticity consistent. the regression outputs have also been substantiated with multicollinearity tests using tolerance. tolerance captures the extent of variability in an independent variable not captured by other independent variables. hence, the higher the value, the more unique is the variable. generally, a tolerance value higher than 0.30 is considered as good and ensures relief from multicollinearity problem. findings from regression models the first regression was performed considering 14 independent variables under avoidance with ps_sqsc as the dependent variable. it was observed that the overall explanatory power (r-squared = 0.30) of the independent variables is not very impressive with only one variable – avd_pnt emerging australasian journal of information systems volume 18 number 3 2014 177 significant at 99 percent level of confidence. the variable represents the extent to which pilot run on new technology was performed, before implementing on the project. a high score would imply that effective pilot runs on new technology would largely avoid recurrence of project risk. if pilot runs are conducted on new technologies, it can help complete projects on time, thereby shortening the project cycle (li, 2009). two variables namely, avd_bve and avd_rrm emerged significant at 95 percent level of confidence. the variables represent adequate evaluation of business value of a project and the correct use of “roles and responsibility” matrix to identify problem areas. both the variables being directly proportional to ps_sqsc, it would imply that higher values of avd_bve and avd_rrm will lead to higher efficiency in avoiding risk. quartel et al., (2012) asserts that an architecture-based approach to it software project evaluation can assess the business value of a project better and helps in meeting project targets. again projects can be highly successful if there is clarity over roles and responsibilities of project team members (evans et al., 1983). the other two variables which were mildly significant at 90 percent level of confidence are avd_cpm and avd_pp. while avd_cpm implies free flow of communication between the project head and the management, avd_pp implies proper phased planning of project. both the variables are directly proportional to overall achievement of targets. massey & kyriazis (2007) discusses the importance of smooth and transparent communication between project head and management. tasevska et al., (2014) highlights the significance of phased planning of project and its positive implications on project success. the second multiple regression was performed considering the transference variables, again, ps_sqsc being the dependent variable. the overall explanatory power of this model is similar to avoidance (rsquared = 0.29). in this model, tfr_prs was significant at 99 percent level of confidence. in lines with the study of korrapati & kocherla (2010), it appears that if the presence of risk is carefully studied to understand and appreciate the nature of risks, transference of risk would not be a threat to success. the other two variables, namely tfr_rtd and tfr_trl are significant at 95 percent level of confidence. this would imply that if cost and time involved in transference of risk is low and the transfer was a well-thought-after decision, project success can be achieved in terms of ps_sqsc. armstrong & paolucci (2010) observe that a systematic planned transfer of risk within departments of project can enable achievement of project targets. mouraviev & kakabadse (2014) argue that a project is benefited if risk transference is prompt and cost effective. australasian journal of information systems volume 18 number 3 2014 178 dependent variable: ps_sqsc method: least squares sample: 1 302 white heteroskedasticity-consistent standard errors & covariance [risk response strategy: acceptance] [risk response strategy: transference] variable coeff. prob. tolerance variable coeff. prob. tolerance c 1.1740 0.0000 c 1.0233 0.0000 act_crm 0.0655 0.1148 0.6588 tfr_ors 0.0808 0.1257 0.5818 act_rea 0.0484 0.3668 0.6332 tfr_prs 0.2145 0.0000 0.7417 act_rep 0.0212 0.6968 0.7296 tfr_rie 0.0449 0.4206 0.5566 act_rfe 0.0864 0.0542 0.7834 tfr_rtd 0.1135 0.0323 0.6126 act_rsa 0.1859 0.0001 0.6586 tfr_rtm -0.0655 0.2435 0.5983 act_rtd -0.0531 0.3494 0.5477 tfr_rts 0.0260 0.6040 0.5053 act_rws 0.0606 0.2445 0.5283 tfr_tnr 0.0761 0.1413 0.6755 act_tri 0.1902 0.0000 0.8000 tfr_trl 0.1277 0.0162 0.6897 r-squared 0.3267 r-squared 0.2903 adjusted r-squared 0.3084 adjusted r-squared 0.2684 f-statistic 17.7746 f-statistic 13.2286 [risk response strategy: avoidance] [risk response strategy: mitigation] variable coeff. prob. tolerance variable coeff. prob. tolerance c 0.7301 0.0017 c 0.2229 0.1866 avd_acp -0.0773 0.2134 0.6308 mtg_acc 0.0558 0.0993 0.5557 avd_bve 0.1052 0.0324 0.6351 mtg_agc 0.0596 0.0741 0.5906 avd_cmm 0.0552 0.4042 0.6251 mtg_cbc 0.0254 0.3726 0.7442 avd_cpm 0.0839 0.0862 0.5579 mtg_crp 0.1542 0.0000 0.5973 avd_cra 0.0157 0.7558 0.6895 mtg_crs 0.0052 0.8421 0.9059 avd_csh 0.0213 0.6283 0.7962 mtg_dcm 0.1200 0.0004 0.5577 avd_css 0.0381 0.5048 0.6184 mtg_frp 0.0392 0.3359 0.6123 avd_pbt -0.0146 0.7763 0.7568 mtg_mct 0.0923 0.0039 0.6397 avd_pfs 0.0560 0.1257 0.7245 mtg_mdm 0.1493 0.0001 0.5847 avd_pnt 0.2263 0.0003 0.6240 mtg_mps 0.0692 0.0253 0.7319 avd_pp 0.0770 0.0914 0.5622 mtg_pri 0.0378 0.1511 0.6916 avd_rrm 0.0860 0.0475 0.5827 mtg_psf 0.2150 0.0000 0.7159 avd_upf 0.0219 0.6537 0.6967 mtg_sfd -0.0306 0.3806 0.5528 avd_wl 0.0182 0.6668 0.5741 mtg_wmw -0.0324 0.3286 0.5030 r-squared 0.3019 r-squared 0.6799 adjusted r-squared 0.2653 adjusted r-squared 0.6643 f-statistic 8.2442 f-statistic 43.5432 table 3: multiple regression of atma the third multiple regression considered mitigation variables with ps_sqsc as the dependent variable, as before. this is the regression that reaped the highest explanatory power with 0.6799 r-squared. this implies that mitigation is the most frequently used risk response strategy for project managers in the software industry. out of 14 variables, five variables, namely, mtg_crp, mtg_dcm, mtg_mct, mtg_mdm and mtg_psf emerged significant at 99 percent level of confidence. there was one australasian journal of information systems volume 18 number 3 2014 179 variable significant at 95 percent level (mtg_mps) and two variables significant at 90 percent level of confidence (mtg_acc, mtg_agc). risk mitigation being an aggressive stance to risk management, organizational consent is important. to gain organization’s confidence and convince the top management to mitigate risk, periodic communication even before the commencement of the project, is essential. following this, periodic communication of project progress, and other risk assessment measures and their impact has to be shared with the top management. goble & bier (2013) opines that having a clear communication channel can itself be termed as a risk mitigation strategy. if there are changes in customers’ expectations or changes in the scope of the project, it is important to discuss the possibility of incorporating such changes in the project clearly between the clients, project managers and top management (holzmann & panizel, 2013). management’s perspective to risk and their readiness to accept other forms of communication makes meetings project targets easier. a strong internal communication and keen interest would also ensure management to be completely aware of the project deliverables and the means to attain them (aubry, 2011). implementation of strict processes and detailed documentation along with commitment and support of top management would also help attainment of project targets bolles et al., (2006). from the perspective of the project team, it is required to be aware of management’s perspective to risk handling (tesch et al., 2007). moreover, if risk assessment results are communicated periodically to all stakeholders and all changes associated with project communicated to customers and management, thereby making customer aware of achievement gaps, project success can be largely attained in terms of scope & quality, scheduling and cost. the fourth and the last regression was conducted using 8 acceptance variables. the overall r-squared for this regression is 0.3267, which is approximately similar to avoidance and transference, though slightly higher than the two. three variables namely, act_rsa, act_tri and act_rfe emerged as significant – the first two variables are significant at 99 percent level and the third at 90 percent level of confidence. this largely implies that careful study of risk factors, postponement of immediate action on current risk and its impact, and evaluation of impact and duration of risk factors are the principal ways in which acceptance strategies are taken up to respond to risk. olawale & sun (2010) identify a list of similar factors for risk acceptance and immediate action. hence, in a nutshell, the output of the four multiple regression models indicate that project managers are more inclined to consider mitigation as a risk response strategy. having said that, acceptance, transference and avoidance also provide interesting dimensions to risk response strategies, in the current study. if the impact of risk is carefully studied, accepting or transferring risks has emerged as a viable option which is predominantly applied by project managers to achieve project targets. along with transparency in the flow of communication, advanced planning of projects in a phase-wise manner and correct evaluation of the business value of a project, avoidance can also be considered an effective strategy in providing risk solutions. findings from sure model as there are four different risk response strategies impacting the same dependent variable ps_sqsc, an alternative model was tried that can consider all the four equations as a system and evaluate the impact on ps_sqsc, jointly. when the problem is to estimate project success from more than one equation, the disturbances in these different equations at a given point of time are likely to reflect some common immeasurable or omitted factors, and hence could be correlated. for example, in the present study of estimating ps_sqsc, the effect of avoidance, treated as a set of independent variables, under the four risk response strategies, mentioned above, is likely to have related effects on the disturbances of the four equations. correlation between disturbances from different equations at a given point of time is known as contemporaneous correlation. australasian journal of information systems volume 18 number 3 2014 180 when contemporaneous correlation exists, it may be more efficient to estimate all equations jointly, rather than to estimate each one separately using least squares. separate estimation of each of the relations ignores the possible relatedness of the equation errors. the appropriate joint estimation technique is often known as seemingly unrelated regression estimation (sure) (zellner, 1962). although the technique is described in the context of estimating a number of equations using timeseries data, they can be equally relevant for cross-sectional data. if cross-sectional data were used to estimate different types of project success functions, then it is quite likely that some immeasurable characteristics of a given project could have similar effects on the disturbances of all the project success functions. the distinguishing feature of seemingly unrelated regressions as a method of pooling timeseries and cross-sectional data, is contemporaneous correlation in the disturbances and the assumption that each cross-sectional unit has a different coefficient vector. under the sure method we want to take into account of the fact that the errors for the simultaneous equations may be contemporaneously (same time period) correlated and thus the off-diagonal blocks in the above equation may not be equal to zero. the system of seemingly unrelated regression equations (sure) provides the researchers with the scope of taking the contemporaneous correlation into account and the conditions under which it is possible to improve on separate least squares estimation of a number of equations. if we have m seemingly unrelated regression equations, the complete model may be written as miuxy iiii ,,2,1 , (1) the complete set of equations is given as: mmmm uxy uxy uxy ............ 2222 1111 (2) where yi and ui (i=1, 2, …, m) are of dimensions (n 1), xi is (n ki), and i is (ki 1). we should note that each equation need not have to have the same number of explanatory variables. in matrix form we write m 2 1 m 2 1 m 2 1 m 2 1 u u u x000 000 00x0 000x y y y (3) or as y=x u (4) australasian journal of information systems volume 18 number 3 2014 181 where y, x, and u are matrices of dimensions respectively, (mn 1), (mn × k), (k × 1) and (mn 1) with . 1 m i ikk since, the design matrix x is of the block-diagonal form, the application of ols to equation (4) treated as a simple regression, would be exactly equivalent to the application of ols to each equation of the m equations in equation (2) separately. it is useful to test the presence of heteroscedasticity and contemporaneous correlation embodied in the systems equations. primary study of residual plots generated by eviews 8.0 verifies the presence of contemporaneous correlation. there are four equations determining project success in terms of scope & quality, scheduling and cost (ps_sqsc). there are intercept terms with all the equations ( 10, 20, 30, 40 and 50), which implies that if the other factors influencing project success are absent, there is an autonomous component responsible for it. the exogenous variables are quantitative. on an a priori basis, we assume that, the coefficients of the exogenous variables are not equal to zero, i.e. ij, ij 0. ‘i’ denotes the source from which project success is measured, based on which the equation is framed and ‘j’ denotes the exogenous variable. that is, each project success equation denotes the project success for the ith source from the jth respondent. in the present set of equations i = 1, 2, 3 and 4 – all representing ps_sqcs as the dependent variable. the independent variables are the same as considered for the four independent multiple regression models conducted earlier. it is also to be notes at this point that all the variables which were considered under the atma are not considered for the sure model. for the sure model we have considered only those variables from atma, which were significant at 99, 95 and 90 percent level of confidence. other variables were dropped out from the sure model. hence, the system of equations for the sure model is as follows: cpmavdbveavd ppavdrrmavdpntavdsqscps __ ____ 1514 13121110 rtdtfrtrltfrprstfrsqscps ____ 23222120 agcmtgmpsmtgaccmtgmctmtg dcmtfrmdmmtgpsfmtgcrpmtgsqscps ____ _____ 38373635 3433323130 rfeacttriactrsaactsqscps ____ 43424140 the output of the sure model is different and also interesting, as compared to the individual multiple regressions performed earlier. let us at first discuss the explanatory power of the four equations as generated by eviews 8.0. in sharp contrast to the earlier results, here it is found that avoidance and transference variables have very little impact on ps_sqsc, both registering an r-squared of 0.07. acceptance, represented by the last equation in the sure model, has showed an r-squared of 0.12, which is higher than avoidance and transference. as expected, mitigation has generated an r-squared of 0.56, which is substantially higher than all the other three factors of risk response strategies. this clearly indicates that project managers do not consider the other mechanisms to respond to risk more conspicuous compared to mitigation. acceptance, as a risk response strategy is the next most influential, after mitigation. australasian journal of information systems volume 18 number 3 2014 182 system: untitled estimation method: seemingly unrelated regression date: 12/04/13 time: 13:55 sample: 1 302 included observations: 302 total system (balanced) observations 1208 linear estimation after one-step weighting matrix coefficient std. error t-statistic prob. 10 2.7231 0.099969 27.23953 0.0000 11 0.036464 0.02522 1.445834 0.1485 12 0.017626 0.019776 0.891291 0.3730 13 0.010502 0.021254 0.494144 0.6213 14 0.011315 0.024931 0.453838 0.6500 15 0.008897 0.022177 0.401208 0.6883 20 2.736259 0.094499 28.95543 0.0000 21 0.037303 0.019766 1.88723 0.0594 22 0.017697 0.021486 0.823669 0.4103 23 0.026438 0.021474 1.231141 0.2185 30 1.348957 0.094228 14.31595 0.0000 31 0.08806 0.023816 3.697474 0.0002 32 0.128626 0.023889 5.384337 0.0000 33 0.083709 0.023167 3.613223 0.0003 34 0.068328 0.024644 2.772575 0.0056 35 0.057879 0.022048 2.625182 0.0088 36 0.047473 0.022874 2.075382 0.0382 37 0.049858 0.021289 2.341888 0.0194 38 0.041301 0.025334 1.630249 0.1033 40 2.643774 0.097333 27.16226 0.0000 41 0.047718 0.026421 1.806063 0.0712 42 0.048899 0.021759 2.247259 0.0248 43 0.017278 0.024248 0.71256 0.4763 table 4: sure model the output under the sure model did not witness all independent variables to emerge as significant. in fact, other than variables pertaining to mitigation, no other risk response strategy emerged to be significant at 99 percent level of confidence. the variables that emerged significant at 99 percent level of confidence are mtg_crp, mtg_psf, mtg_mdm, mtg_dcm and mtg_mct. the other variables which were significant at 95 percent level of confidence are mtg_acc, mtg_mps. two variables were significant as far as acceptance is concerned – one at 95 percent level of confidence australasian journal of information systems volume 18 number 3 2014 183 (act_tri) and the other act_rsa, significant at 90 percent level of confidence. the only variable which was significant at 90 percent level of confidence in transference is tfr_prs. unfortunately, there was no variable that emerged to be significant from avoidance. the results of the sure model are indeed interesting, as the independent variables used in the set of equations are a selected few, which emerged significant in the multiple regressions. just like the results of the multiple regressions, mitigation has again displayed the highest influence on achievement of project targets. according to the study, risk mitigation mostly revolves around effective communication across all parties involved in the project. the findings reveal that periodic communication of risk assessment results including discussions on change of project scope with customers and management, keeping management and other stakeholders aware of project deliverables and means to meet and strict adherence to procedures which have been set by top management in concurrence with project manager and team members, management’s readiness to accept different forms of communication tools, transparency in continuous gap analysis of target achievements with customers and top management and study of management’s perspective to risk are the significant determinants of risk mitigation. it was also found that risk can be transferred to another phase of the project, after careful study of its nature in the current phase. hence, if transference of risk has the likelihood of increasing its impact after transference, it should be avoided. korrapati & kocherla (2010) suggest that ex post phase-wise analysis of risks over several projects can provide such crucial knowledge on whether transference would be the best strategy. acceptance of risk in the form of not taking any action, after thorough study of the potential threat of each and every risk factor, has emerged as an effective risk response strategy (berkeley, et al., 1991). often, in the indian context, despite various sophisticated studies in risk management, practical and easily adoptable risk framework is largely missing (dey, 2012). hence, taking no action on risk and its impact is often a strategy that project managers would adopt. this variable has emerged significant in the current study. discussion the study demonstrates that indian companies prefer mitigation as the most prominent response strategy to achieve the quadruple targets of scope & quality, schedule and cost. though major international studies reiterate about all four responses strategies applied in project risk management at different levels, in india the major response strategy as perceived by project managers is mitigation. the findings of the present study offer a lot of insights on the practices being followed in the implementation of risk management. the study reveals that project stakeholders who perceived that organizations that follow processes strictly, as laid down by the management tended to misreport less. as observed by smith et al., (2006) the existence of team spirit and a caring environment did not necessarily imply a robust reporting structure (smith et al., 2006). in other words, a process-oriented approach to risk management reaps better benefits than loosely bound processes, which are not well-defined. assessing risk management periodically and communicating it to the management and customers, can help resolve major uncertainties and lead to project success. using a phase-wise approach towards risk assessment and management can occur with effective interaction and cooperation between project managers and other stakeholders involved in the process. methods to explore and develop free flow of communication within the organization and between the stakeholders can help managing risk more effectively (thompson & bloom, 2000). at different phases of the project, top management, project steering committee, and internal & external stakeholders should be aware of the deliverables. the steering committee plays a critical role together with the sponsor in approving the project scope, australasian journal of information systems volume 18 number 3 2014 184 deliverables and financials. their responsibilities with respect to project management are to provide support to the project team to ensure a smooth project delivery process and implementation. the specific response strategies under the overall atma factors, mentioned above play a fundamental role to impact the quadruple targets of scope & quality, schedule and cost. processes, which are mostly tailor-made to meet the objectives and requirements of the projects, are targeted to meet the deadlines, maintain quality, avoid rework and conflict and improve productivity, in a realistic manner. for example, if a software product is to be deployed in a client location, there is a possibility that testing the smooth installation is accidentally left out, due to the lack of existence of a properly defined checklist. this can, in turn, delay the process of activation of the software and lead to several other problems in project implementation. communicating risks in projects requires special skills, project managers often share information about project risks ineffectually or not at all, and hence projects suffer unnecessary set back. communicating risks at a scheduled time regularly scheduled team meetings, in scheduled progress reports and uppermanagement reviews, and in special meetings will increase the likelihood of controlling projects from any kind of risks that might increase. in a nutshell, the findings suggest that risk management is an integral part of project management and it needs to be weaved with all the process groups of initiating, planning, executing, monitoring and controlling, and closing. risk can be effectively controlled through proper communication using different modes, for example, formal or informal, spoken or written, manual or electronic, amongst various stakeholders, as applicable to each milestone to achieve the targets of scope & quality, schedule and cost (pmbok®). conclusions the objective of the study was to investigate the risk response strategies that influence the achievement of scope & quality, schedule and cost targets in software firms in india. based on the literature, key informant technique and pilot survey, a questionnaire was designed to collect primary data from 302 respondents. variables under four broad categories namely, avoidance, transference, mitigation and acceptance (atma) were considered to form the list of independent variables. applying cronbach’s alpha, two variables were removed as a part of reliability check. we have used two forms of quantitative models in this study. four multiple regressions were conducted using project success to achieve the quadruple targets (ps_sqsc), as the dependent variable and the variables under each of the risk response strategies (atma) as the independent variables in the different regression models. separately, a set of seemingly unrelated regression equations (sure) model was also used as a part of the analysis where the dependent variable was the same but the independent variables includes only those which emerged significant in the prior multiple regression models. this helped the study to get a holistic picture of the risk response strategies’ impact on meeting project targets. irrespective of the category of risk response strategy, clear, transparent and free flow of information and communication have emerged as key solutions to risk. apart from that, in-depth knowledge of all stakeholders on the changing nature of risks in software projects is of paramount importance. mitigation of risk emerged as the most significant factor in both the models used in the study. the other three factors namely, avoidance, transference and acceptance have nearly the same influence on ps_sqsc. in the development phase of a project, mitigation of risks plays an important role. sakthidaran (2010) includes user involvement, requirements, user and management support, managing innovation, complexity, scope of work, project management practices, and training as important variables that influence project success. the common thread that bonds all these variables in risk mitigation is the effective flow of communication across levels. poor communication practices often lead to unclear project objectives and improper project control (kerzner, 2013). australasian journal of information systems volume 18 number 3 2014 185 the future scope of research in the topic would include a couple of options. a cross-border analysis of the atma model and its influence on achieving project targets would be an interesting study. it will be interesting to note whether mitigation emerges as the most significant risk response strategy in other emerging economies. as this study is limited to software industry, there is a scope of expanding it to other industries to produce a comparative analysis of risk response strategies. the study also considers risk management from the software development life cycle perspective. however, the approach to risk management in software projects can also be undertaken for agile processes like scrum, extreme programming and crystal methodologies. while the paper largely assumes that project managers have the authority of employing risk solutions, the reality to be far from this. there can be either a steering committee or even active involvement of top management and their perspective to risk that can determine the implementation of risk response strategies. also, as discussed earlier, approach to risk has two perspectives (gasparini et al., 2004) – the static model and the dynamic model. there is a future scope of research to apply sophisticated statistical models in the area of dynamic risk management, where learning from each sage of the project and previous projects can be incorporated on an iterative basis to manage project risks better. as software or it projects have low gestation periods, a static model to risk management is largely acceptable and the projects’ scope & quality requirements do not change much. for large projects in the area of manufacturing, especially, infrastructure, dynamic risk management models would be more effective, as the requirements for the projects and the identified risks can change. the authors conclude that planning of risk is vital to every project success but implementing it with the appropriate risk response strategy is a must. apart from planning, transparency of communication within project stakeholders is imperative. software industry is ever-evolving and currency is crucial to the industry, which differentiates it from other industries. this factor of continuous evolution of the industry makes it more vulnerable to risks. hence, a major key to software project success is impromptu application of risk response strategies. references abd, k.; abhary, k.; marian, r.; (2014); simulation modeling and the analysis of scheduling in robotic flexible assembly clls using taguchi method; international journal of production research; 52 (9), 2654-2666. abdul-rahman, h., mohd-rahim, f. a. and chen, w. (2012) reducing failures in software development projects: effectiveness of risk mitigation strategies. journal of risk research, 15, 417-433. agarwal, n., & rathod, u.; (2006). defining ‘success’ for software projects: an exploratory revelation. international journal of project management, 24(4), 358-370. alexandra-mihaela, p., & danut, d. d. (2013); the measurement and evaluation of the internal communication process in project management; annals of the university of oradea, economic science series, 22(1). augustine s., bob p., fred s., and susan w. (2005). agile project management: steering from the edges. communications of the acm, 48(12), 85-89. alkass, s.; al-jibouri, s.; techakosol, v.; (2006); feasibility studies: a case for using a stochastic approach; aace international transactions; 6.1-6. alter, s., ginzberg, m., (1978), managing uncertainty in mis implementation, sloan management review, 23 – 31. australasian journal of information systems volume 18 number 3 2014 186 armour, p. g. (2005). project portfolios: organizational management of risk. communications of the acm, 48, 17-20. armstrong, j.; paolucci, f.; (2010); risk equalization in ireland and australia: a simulation analysis to compare outcomes; geneva papers on risk & insurance issues & practice; 35 (4), 521-538. aubry, m.; (2011); the social reality of organizational project management at the interface between networks and hierarchy; international journal of managing projects in business; 4 (3), 436457. bagdonien , d.; simanavi ien , a.; (2012); effective project team training; economics & management. 17 (3), 1165-1170. berkel, r.; (2014); quasi-markets and the delivery of activation – a frontline perspective; social policy & administration; 48 (2), 188-203. berkeley, d.; humphreys, p. c.; thomas, r. d.; (1991); project risk action management; construction management & economics; 9 (1), 3. blackstone, j. h.; cox, j. f.; schleier, j.g.; (2009); a tutorial on project management from a theory of constraints perspective; international journal of production research; 47 (24), 7029-7046. boehm, b. w. (1989). software risk management tutorial, ieee computer society press, los alamitos, california. boehm, b. w. (1991). software risk management: principles and practices. software, ieee, 8, 32-41. bolles, dennis l.; hubbard, d. g. (2006), integrating projects with business strategies and objectives: 4.5 key points for project business management planning success; power of enterprise-wide project management. p41-42. burström, t.; jacobsson, m.; (2011); the role and importance of ‘glue people’ in projects; iup journal of soft skills; 5 (1), p7-15. byoun, s.; kim, j.; yoo, s. s.; (2013); risk management with leverage: evidence from project finance; journal of financial & quantitative analysis;48 (2), 549-577. cardenas, j. a.; gemoets, l.; ablanedo r., jose h.; sarfi, r.; (2014); a literature survey on smart grid distribution: an analytical approach; journal of cleaner production; 65, p202-216. carcary, m. (2013). it risk management: a capability maturity model perspective. electronic journal of information systems evaluation, 16(1). cavusgil, s. t.; sikora, e.; (1988); how multinationals can counter gray market imports; columbia journal of world business; 23 (4), 75-85. chan, e. h.w.; au, m. c.y. (2007); building contractors’ behavioral pattern in pricing weather risks; international journal of project management; 25 (6), 615-626. chandra, a., & calderon, t. g. (2009). information intensity, control deficiency risk, and materiality. managerial auditing journal, 24(3), 220-232. cook, p.; (2005); formalized risk management: vital tools for project and business success; cost engineering; 47 (8), p12-13. davis, g. b.; (1982); strategies for information requirements determination; ibm systems journal; 21(1), 4-30. dey, p. k.; (2012); project risk management using multiple criterion decision-making technique and decision tree analysis: a case study of indian oil refinery; production planning & control; 23 (12), 903-921. doherty, n. f., king, m., & al-mushayt, o., (2003), the impact of inadequacies in the treatment of organizational issues on information systems development projects, information & management. 41, 49-63. australasian journal of information systems volume 18 number 3 2014 187 doloi, h.; (2013); cost overruns and failure in project management: understanding the role of key stakeholders in construction projects; journal of construction engineering & management; 139 (3), 267-279. drummond, h.(2005); what we never have, we never miss? a study of entrapment in a small business; journal of information technology; 20 (3), 170-176. dvir, d. and shenhar, a. j., (1992) measuring the success of technology based strategic business units, engineering management journal, 4, 33-38. eicher, j.; ruder, d.; (2007); business process analytics: a new approach to risk; journal of alternative investments; 10 (2), 76-84. evans, m. w., piazza, p., dolkas, b., (1983). principles of productive software, john wiley and sons, inc. new york, ny, usa. fairley, m. e.; (2005); software risk management; ieee software;. 22 (3), 101-101. ferreira, s., collofello, j., shunk, d., & mackulak, g. (2009); understanding the effects of requirements volatility in software engineering by using analytical modelling and software process simulation. journal of systems and software, 82, 1568-1577. gasparini, m., margaria, g., & wynn, h. p. (2004). dynamic risk control for project development. statistical methods and applications, 13(1), 73-88. golden, l. l.; wang, m.; yang, c.; (2007); handling weather related risks through the financial markets: consideration of credit risk, basic risk and hedging; journal of risk & insurance; 74 (2), 319-346. goble, r.; bier, v. m.; (2013); risk assessment can be a game-changing information technology – but too often it isn’t; risk analysis: an international journal; 33 (11), 1942-1951. hanisch, b.; wald, a.; (2012); a bibliometric view on the use of contingency theory in project management research; project management journal; 43(3), 4-23. hecker, j. z.; (2002); hazard mitigation: proposed changes to fema’s multihazard mitigation programs present challenges; gao reports. 46. hillson, d. (2002). extending the risk process to manage opportunities. international journal of project management, 20(3), 235-240. holzmann, v.; panizel, i.; (2013); communications management in scrum projects; proceedings of the european conference on information management & evaluation; 67-74. fisher, k. t.; urich, p. b.; (1999), information dissemination and communication in stakeholder participation: the bohol cebu water supply project; asia pacific viewpoint; 40(3), 251. geethalakshmi. s.n., and shanmugam. a., (2008); success and failure of software development: practitioners’ perspective; proceedings of the international multiconference of engineers and computer scientists; 1, hong kong. goble, r.; bier, v. m.; (2013); risk assessment can be a game-changing information technology – but too often it isn’t; risk analysis: an international journal; 33 (11), 1942-1951. javed, a. a.; lam, p.t.i.; zou, p. x.w; (2013); output-based specifications for ppp projects: lessons for facilities management from australia; journal of facilities management; 11(1), p5-30. johnstone, k.m.; (2000); client-acceptance decisions: simultaneous effects of client business risk, audit risk, auditor business risk, and risk adaptation; auditing; 19 (1), 1-25. karolak, d. w.; (1996); software engineering risk management; los alamitos, ca; ieee computer society press. australasian journal of information systems volume 18 number 3 2014 188 khamooshi, h.; cioffi, d. f.; (2013); uncertainty in task duration and cost estimates: fusion of probabilistic forecasts and deterministic scheduling; journal of construction engineering & management; 139 (5), 488-497. keil, m.; rai, a.; liu, s.; (2013); how user risk and requirements risk moderate the effects of formal and informal control on the process performance of it projects; european journal of information systems; 22 (6), 650-672. keil m., cule p., lyytinen, k., and schmidt, r., (1998) a framework for identifying software project risks. communications of the acm, 41, 11, 76-83. kerzner, h. r. (2013). project management: a systems approach to planning, scheduling, and controlling. wiley. koller, t., lovallo, d., williams, z.,(2012) overcoming a bias against risk; mckinsey quarterly; 4, p15-17. korrapati, r.; kocherla, s.;(2010); a qualitative study on determining managerial styles for software development life cycle stages; allied academies international conference: proceedings of the academy of information & management sciences (aims); 14 (1), 54-57. kwak, y. h., & stoddard, j. (2004). project risk management: lessons learned from software development environment. technovation, 24(11), 915-920. kutsch, e.; maylor, h.; weyer, b.; lupson, j.; (2011); performers, trackers, lemmings and the lost: sustained false optimism in forecasting project outcomes – evidence from a quasi-experiment; international journal of project management; dec2011, 29 (8), 1070-1081. laursen, e.; (2011); organizational learning through development projects; international journal of manpower; 32(5/6); 567-580. laurentiu, d.; gabriela, d.; (2013); sensitivity assessment modelling in european funded projects proposed by romanian companies; annals of the university of oradea, economic science series; 22 (2), 361-371. lehtinen, t.o.a.; mäntylä, m. v.; vanhanen, j.; itkonen, j.; lassenius, c.; (2014); percieved causes of software project failures – an analysis of their relationships; information &software technology; 56 (6), 623-643. legault, mj.; chasserio, s.; (2012); professionalization, risk transfer and the effect on gender gap in project management; international journal of project management; 30(6), p697-707. lewin, m. d. (2001). better software project management: a primer for success. john wiley & sons. li, d. c.; tsai, t. -i.; shi, s.; (2009); a prediction of the dielectric constant of multi-layer ceramic capacitors using the mega-trend-diffusion in powder pilot runs: case study; international journal of production research; 47 (1), p51-69. little, r. g.; (2010); towards a new federal role in infrastructure investment: using us sovereign wealth to rebuild america; public works management & policy; 14 (3), 288-306. march, j. g. & z. shapira, z. (1987). managerial perspectives on risk and risk taking. management science, 33, 1404-1418. massey, g. r.; kyriazis, e.; (2007); interpersonal trust between marketing and r&d during new product development projects; european journal of marketing; 41 (9/10), 1146-1172. mcfarlan, f. w., (1982). portfolio approach to information systems. journal of systems management (january), 12-19. mcmanus, j. (2004). a stakeholder perspective within software engineering projects. in engineering management conference, 2004. proceedings. 2004 ieee international (vol. 2, pp. 880-884). ieee. australasian journal of information systems volume 18 number 3 2014 189 mcmanus, j.; (2003); the application of risk planning in software development projects; management services; 47 (9), 16-18. melnic, a.s.; puiu, t.; (2011); the management of human resources within projects: the structures of the project team, the responsibility assignment matrix; economy transdisciplinarity cognition; 14(1), 476-484. mir, f. a., & pinnington, a. h. (2014). exploring the value of project management: linking project management performance and project success. international journal of project management, 32(2), 202-217. mouraviev, n.; kakabadse, n.k.; (2014); risk allocation in a public-private partnership: a case study of construction and operation of kindergartens in kazakhstan; journal of risk research; 17 (5), 621-640. müller, r.; turner, j. r..; (2005); the impact of principal-agent relationship and contract type on communication between project owner and manager; international journal of project management; 23(5), 398-403. olawale, y. a.; sun, m.; (2010); cost and time control of construction projects: inhibiting factors and mitigating measures in practice; construction management & economics; 28 (5), 509-526. parker, d.; charlton, j.; ribeiro, a.; pathak, r. d.; (2013); integration of project-based management0020and change management intervention methodology; international journal of productivity & performance management; 62 (5), 534-544. pavlak, a.; (2004); project troubleshooting: tiger teams for reactive risk management; project management journal, 35(4), 5-14. pieterse, m.; (2012), critical success factors in information technology projects; uj library and information centre. pinto, j. k., & mantel jr, s. j. (1990). the causes of project failure. engineering management, ieee transactions on, 37, 269-276. project management institute, inc. a guide to the project management body of knowledge (pmbok® guide), fifth edition (2013). rivard, s., st-james, y., & cameron, a. (2011). software project risk drivers as project manager stressors and coping resources. in system sciences (hicss), 2011 44th hawaii international conference on (pp. 1-10). ieee. rosenbaum, d.; the bold choice; (2005); cio:18 (21), 40-44. quartel, d.; steen, m. w.a.; lankhorst, m. m.; (2012); application and project portfolio valuation using enterprise architecture and business requirements modeling; enterprise information systems; 6 (2), 189-213. rodriguez, e.; edwards, j.; koenig, d.; (2010); the board of directors, executives and risk knowledge management; proceedings of the international conference on intellectual capital, knowledge management & organizational learning. 394-404. rosenkranz, c.; corvera c. m.; holten, r.; (2013), language quality in requirements development: tracing communication in the process of information systems development; journal of information technology (palgrave macmillan); 28(3), 198-223. salmeron, j. l., & lopez, c. (2012). forecasting risk impact on erp maintenance with augmented fuzzy cognitive maps. software engineering, ieee transactions on, 38(2), 439-452. samáková, j.; šujanová, j.; koltnerová, k.; (2013); project communication management in industrial enterprises; proceedings of the european conference on information management & evaluation; 155-163. australasian journal of information systems volume 18 number 3 2014 190 sakthidaran, r. t. (2010). how can an acquirer mitigate risks in software engineering projects?; open software engineering journal, 4. sang-hoon k.; elsaid, h. h. (1985);safely margin allocation and risk assessment under the npv method; journal of business finance & accounting; 12 (1), 133-144. schmidt, r.; lyytinen, k.; keil, m.; cule, p.; (2001); identifying software project risks: an international delphi study; journal of management information systems17 (4), 5-36. scurr, c.; (1990); management services and total quality management; management services; 34 (7), 16-18. shiferaw, a. t.; klakegg, o. j.; haavaldsen, t..; (2012); governance of public investment projects in ethiopia; project management journal; 43(4), 52-69. sillars, d. n., & o’connor, m. b. (2009). risk-informed transit project oversight. journal of management in engineering, 25(3), 106-114. skyner, l.; (2006); a viable framework of private investment in the utility sector: an analysis of the 2005 rf law of concession agreement; review of central & east european law; 31 (2), 155177. smith, d., eastcroft, m., mahmood, n., and rode, h., (2006);risk factors affecting software projects in south africa; south african journal of business management; 2, 55-65. sungmin y.; caldas, c. h. (2009); analyzing decision variables that influence preliminary feasibility studies using data mining techniques; construction management & economics; 27(1), 73-87. stoneburner, g., goguen, a., & feringa, a. (2002). risk management guide for information technology systems. nist special publication, 800(30), 800-30. swink, m. l.; sandvig, j. c.; mabert, v. a.; (1996); customizing concurrent engineering processes: five case studies; journal of product innovation management; 13 (3), 229-244. sykes, k.; (2014), global jobs report: 6 sectors to watch; pm network; 28 (1), p58-67. tasevska, f.; damij, t.; damij, n.; (2014); project planning practices based on enterprise resource planning systems in small and medium enterprises – a case study from the republic of macedonia; international journal of project management; 32(3), 529-539. tesch, d.; timothy j. k.; and mark n. f.; (2007); it project risk factors: the project management professionals perspective; journal of computer information systems 47 (4) 61-69. thakurta, r. (2013). impact of scope creep on software project quality. vilakshan: the ximb journal of management, 10, 37-46. thal jr., a. e.; martínez, c.; (2011); contingency action helps projects deal with unknown unknowns; sustainability is good news; industrial management; 53(6), 8. thomas, s., & bhasi, m. (2011). how does software project risk vary across projects? an exploratory study from india. international journal of intercultural information management, 2, 301-316. thompson, b. (2000). q-technique factor analysis: one variation on the two-mode factor analysis of variables. in l. grimm & p. yarnold (eds.), reading and understanding more multivariate statistics (pp. 207-226). washington, dc: american psychological association. tuman j., (1986), success modeling: a technique for building a winning project team.; proc., 18th annual seminar/symposium, project management institute, montreal, canada, 29-34. veil, s. r.; husted, r. a.; (2012); best practices as an assessment for crisis communication; journal of communication management; 16 (2), 131-145. verner, j.m.; brereton, o.p.; kitchenham, b.a.; turner, m.; niazi, m.; (2014); risks and risk mitigation in global software development: a tertiary study; information & software technology 56 (1), p54-78 australasian journal of information systems volume 18 number 3 2014 191 vizard, m.; (2008); why there is no business continuity; baseline, 88, 18-18. wallace, l., keil, m., & rai, a. (2004). how software project risk affects project performance: an investigation of the dimensions of risk and an exploratory model. decision sciences, 35, 289321. warkentin, m., & willison, r. (2009). behavioral and policy issues in information systems security: the insider threat. european journal of information systems, 18(2), 101. witt, e.; liias, r.;. (2011); comparing risk transfers under different procurement arrangements; international journal of strategic property management; 15 (2), 173-188. yelin x.; yujie l.; chan, a.p. c.; skibniewski, m. j.; yeung, j.f. y. (2012), a computerized risk evaluation model for public private partnership projects and its application; international journal of strategic property management. 16 (3), 277-297. yong, h., juhua, c., zhenbang, r., liu, m., & kang, x. (2006, december). a neural networks approach for software risk analysis. in data mining workshops, 2006. icdm workshops 2006. sixth ieee international conference on (722-725). ieee. zavadskas, e. k.; turskis, z.; ustinovichius, l.; shevchenko, g.; (2011); attributes weights determining peculiarities in multiple attribute decision making methods; engineering economics; 21(1), 32-43. zellner, a. (1962). an efficient method of estimating seemingly unrelated regressions and tests for aggregation bias. journal of the american statistical association, 57, 348-368. zhang, y.; fan, z.p.; (2014); an optimization method for selecting project risk response strategies; international journal of project management; 32 (3), 412-422. zofi, y.; (2012); getting deliverables out the door, industrial engineer; 44 (7), 35-40. acknowledgements the authors would like to sincerely thank the editor– dr. john w lamp for his continuous support. the authors are also appreciative of the two anonymous referees for useful comments that helped to structure the paper better. paper title (paper title style) australasian journal of information systems lim 2016, vol 20, post publication review moving from evaluation to trial 1 post-publication review salim, s. a., sedera, d., sawang, s., alarifi, a. h. e., & atapattu, m. (2015). moving from evaluation to trial: how do smes start adopting cloud erp? australasian journal of information systems, 19, s219-s254. doi: http://dx.doi.org/10.3127/ajis.v19i0.1030 review cloud-based enterprise resource planning (erp) is an approach to business process management with low subscription overhead that uses a network of remote servers hosted on the internet to provide businesses with a system for automating business functions (e.g. accounting, human resources, inventory, order management). by improving the accessibility of internal data (employees can upload and access company data in real time via the cloud from any device with an internet connection), cloud erp provides work flexibility and mobility and enables greater collaboration among employees from both the same and different departments than traditional on-premises erp solutions. recognizing that erp adoption in business settings is complex and thus goes beyond a single activity, salim et al. (2015) sought to understand the adoption process of cloud erp and the critical adoption factors in the evaluation and trial stages of the adoption process. the study— premised on the theory of planned behavior (tpb), ettlie’s multi-stage adoption model, and a sample of 162 owners of small and medium-sized enterprises (smes)—is novel and makes important contributions by identifying potential differences in significance and impact of factors that influence the intention of smes to adopt cloud erp across the two adoptive stages. salim et al.’s (2015) article offers several positive takeaways: 1. the article is exemplary in its articulation of the research problem in terms of the limitations of existing studies (i.e. simple technology adoption, technology adoption within a single broad stage) and the critical need for greater clarity on the factors that influence complex technology adoption (e.g. cloud erp; i.e. to improve decision making, to understand changes in critical adoption factors, to understand the requirements for adequate vendor involvement, and to understand the rationales for switching vendors or dropping out, all of which pertain to the complex technology adoption process; dubey and wagle 2007; law and ngai 2007; muscatello et al. 2003; willcocks and sykes 2000; winters et al. 2008). 2. the article makes a good case for examining cloud erp in the context of smes—that is, the accessibility to (and benefits derived from) a complete range of it applications that were once restricted to larger organizations—and is supported by recent statistics showing significant adoption intentions. 3. the article provides a good explanation of its contributions to theory and practice by addressing the identified research problems in the context of smes’ adoption of cloud erp. that is, by combining two complementary theories in the form of tpb and ettlie’s multi-stage adoption model, the study demonstrates the multi-stage process of cloud erp adoption and contributes to the understanding of the different levels of significance of each factor in each of the adoption stages, including the reasons certain factors are more or less prominent in a particular stage. note that though only two of the five adoption stages (i.e. evaluation and trial stages) were studied, the article provides ample support from previous studies to justify such a narrow focus of investigation (i.e. most critical stages in the process of adoption, reasons for adoption dropouts related to limited information from these two stages, apparent differences before and after experiencing cloud erp, and completion of stages leading to actual decision on cloud erp usage; arthur 1989; au and kauffman 2003; howard and sheth 1969). australasian journal of information systems lim 2016, vol 20, post publication review moving from evaluation to trial 2 4. the article is based on solid theoretical foundations. up-to-date (i.e. 1967–2015) studies related to technology adoption were covered in detail, a thorough explanation of the adoption stages was provided (i.e. awareness, interest, evaluation, trial, and commitment), determinants of decision making were adopted from a widely used behavioral model (i.e. tpb; attitude, subjective norms, perceived behavioral control, and intention), and adequate rationales from previous studies were used to support five research propositions under study (i.e. attitudes, subjective normal, perceived behavioral control, intention, and construct fluctuations between evaluation and trial stages in the adoption process). 5. the article is rigorous and practice oriented in the methods used and analysis conducted. that is, constructs were developed using closely related studies (e.g. ettlie 1980; fichman and kemerer 1997; harrison et al. 1997), appropriate guidelines were used for identifying formative variables (i.e. do not need to covary, must not be interchangeable, must cause the core construct and not be caused by it, and must allow variability in antecedents and consequences under different nomological nets; cenfetelli and bassellier 2009; jarvis et al. 2003; petter et al. 2007), practical relevance for using formative constructs were provided (i.e. specific and actionable attributes of formative constructs, as weight of constructs can be used to draw practical implications on the importance of specific details; furneaux and wade 2011; mathieson et al. 2001), a pretest (with a sample of 30 respondents) was carried out to ensure adequate reliability and validity of items measuring constructs before full-scale survey administration, sampling criteria were transparent (e.g. 162 malaysian smes were selected according to the national sme development council’s definition of smes, key informants were owners of smes whose opinions were deemed as representing the voice of the entire firm, questionnaires were administered at a conference targeted to smes, participation was voluntary and no incentives were provided), and the partial least square technique of structural equation modeling was justified (e.g. purpose of study involves predictive ability of constructs rather than theoretical validation, small sample size; hair et al. 2011) and rigorously conducted (i.e. assessment of internal consistency, discriminant and convergent validity, and multicollinearity and use of bonferroni-corrected p-value to guard against global type i error; diamantopoulos and winklhofer 2001; fornell and larcker 1981; gable et al. 2008; nunnally and bernstein 1991; shaffer 1995). 6. the article offers a meaningful discussion of the implications of the findings to theory and practice. the study extends the assumption of technology adoption from a singlestage process to a multi-stage process (theory), extends the scope of tpb (i.e. cloud erp as a new research context, evidence of negative perceived behavioral control–intention relationship, modeling of tpb in a multi-stage process, and support for treating tpb components such as subjective norms as formative constructs), and provides the cloud erp and sme ecosystem with a nuanced understanding of potential users’ perceptions of and behaviors toward the adoption of cloud erp (practice). nonetheless, in addition to the five limitations acknowledged (i.e. limited to evaluation and trial stages in the adoption process, limited explanation on intention–behavior relationship, limited in generalizability to other types of complex technology adoption, limited to smes whose important decisions are made by one individual, and limited to smes in malaysia), salim et al.’s (2015) work contains two notable shortcomings that pave the way for further research in the area: 1. although the study reports some differences between critical adoption factors in evaluation and trial stages of the adoption process for cloud erp among smes, these differences were only analyzed descriptively. moreover, observation of path coefficients (β) imply that identified differences (e.g. between the attitude–intention and subjective norms–intention paths) might not be significant (vs. significance of paths when taken individually) as a result of marginal improvements between predictors (e.g. in the evaluation stage, the difference in β between the attitude–intention and subjective australasian journal of information systems lim 2016, vol 20, post publication review moving from evaluation to trial 3 norms–intention paths is only 0.023 [i.e. 0.385 – 0.362]). thus, further research can improve the rigor of analysis for multi-stage adoption processes by testing for significant differences in impact (or significance of improvements) between critical adoption factors within and across stages. 2. although the critical adoption factors are well grounded in tpb and items measuring the constructs are adapted from existing studies in the area, an optimistic outlook suggests that more can be done to identify and examine factors that are more contextualized in order to induce greater adoption of cloud erp among smes. for example, a qualitative study can be conducted to explore from sme owners’ (and employees’) perspectives the facilitators and barriers to cloud erp adoption, which can subsequently be tested quantitatively (and potentially as an elaborated part of perceived behavioral control). in short, salim et al.’s (2015) article is a well-written piece whose positive takeaways can be used as a guide for writing a good research paper. in addition, their article goes beyond addressing important gaps in current understanding by offering new pathways for theoretical and practical extensions in the area. thus, further research can improve the adoption of and reduce the attrition rates of cloud erp in business settings using the avenues discussed both in salim et al.’s (2015) article and in this post-publication review. weng marc lim swinburne university of technology (sarawak campus) wlim@swinburne.edu.my references arthur, w. b. (1989). competing technologies, increasing returns, and lock-in by historical events. the economic journal, 99 (394), 116-131. au, y. a., & kauffman, r. j. (2003). what do you know? rational expectations in information technology adoption and investment. journal of management information systems, 20 (2), 49-76. cenfetelli, r. t., & bassellier, g. (2009). interpretation of formative measurement in information systems research. mis quarterly, 33 (4), 689-707. diamantopoulos, a., & winklhofer, h. m. (2011). index construction with formative indicators: an alternative to scale development. journal of marketing research, 19 (2), 269-277. dubey, a., & wagle, d. (2007). delivering software as a service. the mckinsey quarterly. available at http://www.lcis.com.tw/paper_store/paper_store/delivering_software_as_a_service20154515250718.pdf ettlie, j. e. (1980). adequacy of stage models for decisions on adoption of innovation. psychological reports, 46 (3), 991-995. fichman, r. g., & kemerer, c. f. (1997). the assimilation of software process innovations: an organizational learning perspective. management science, 43 (10), 1345-1363. fornell, c., & larcker, d. f. (1981). evaluating structural equation models with unobservable variables and measurement error. journal of marketing research, 19 (1), 39-50. furneaux, b., & wade, m. (2011). an exploration of organizational level information systems discontinuance intentions. mis quarterly, 35 (3), 573-598. gable, g. g., sedera, d., & chan, t. (2008). re-conceptualizing information system success: the is-impact measurement model. journal of the association for information systems, 9 (7), 377-408. australasian journal of information systems lim 2016, vol 20, post publication review moving from evaluation to trial 4 hair, j. f., ringle, c. m., & sarstedt, m. (2011). pls-sem: indeed a silver bullet. journal of marketing theory and practice, 19 (2), 139-152. harrison, d. a., mykytyn jr., p. p., & riemenschneider, c. k. (1997). executive decisions about adoption of information technology in small business: theory and empirical tests. information systems research, 8 (2), 171-195. howard, j. a., & sheth, j. n. (1969). the theory of buyer behavior. new york: wiley. jarvis, c. b., mackenzie, s. b., & podsakoff, p. m. (2003). a critical review of construct indicators and measurement model misspecification in marketing and consumer research. journal of consumer research, 30 (2), 199-218. law, c. c., & ngai, e. w. (2007). erp systems adoption: an exploratory study of the organizational factors and impacts of erp success. information and management, 44 (4), 418-432. mathieson, k., peacock, e., & chin, w. w. (2001). extending the technology acceptance model: the influence of perceived user resources. acm sigmis database, 32 (3), 86-112. muscatello, j. r., small, m. h., & chen, j. j. (2003). implementing enterprise resource planning (erp) systems in small and midsize manufacturing firms. international journal of operations and production management, 23 (8), 850-871. nunnally, j. c., & bernstein, i. h. (1991). psychometric theory. new york: mcgraw-hill. petter, s., straub, d., & rai, a. (2007). specifying formative constructs in information systems research. mis quarterly, 31 (4), 623-656. salim, s. a., sedera, d., sawang, s., alarifi, a. h. e., & atapattu, m. (2015). moving from evaluation to trial: how do smes start adopting cloud erp? australasian journal of information systems, 19, s219-s254. doi: http://dx.doi.org/10.3127/ajis.v19i0.1030 shaffer, j. p. (1995). multiple hypothesis testing. annual review of psychology, 46 (1), 561584. willcocks, l. p., & sykes, r. (2000). enterprise resource planning: the role of the cio and it function in erp. communications of the acm, 43 (4), 32-38. winters, d. b., lindley, j. t., topping, s., & lindley, l. t. (2008). the hidden financial costs of erp software. managerial finance, 34 (2), 78-90. author response the authors of the paper under review value the positive contributions made and thank their colleague for his contribution. copyright: © 2016 lim. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems lim 2016, vol 20, post publication review moving from evaluation to trial 5 paper title (paper title style) australasian journal of information systems o’connor, heavin, gallagher & o’donohue 2017, vol 21, research on applied ethics and ict predicting participant consent in mhealth trials 1 predicting participant consent in mhealth trials – a caregiver’s perspective yvonne o’connor health information systems research centre cork university business school ireland y.oconnor@ucc.ie ciara heavin health information systems research centre cork university business school ireland joe gallagher ghealth research group university college dublin ireland john o’donohue global ehealth unit imperial college london uk abstract informed consent is sought prior to conducting a healthcare intervention on a person. when a healthcare intervention involves a young child, their caregiver is required to provide informed consent on their behalf. however, little is known on the behavioural intentions of participants to provide consent when a mobile health (mhealth) intervention is involved in a clinical trial scenario. understanding this phenomenon is important, without consent appropriate data may not be collected to empirically examine the implications of mhealth initiatives when delivering healthcare services to children in a ‘real world context’. the objective of this paper is to explore the behavioural intentions of caregivers to provide consent for children (under five years of age) to participate in mhealth randomised control trials (rct) in developing countries and subsequently develop a predictive model for consent giving. data was captured vis-à-vis interviews with malawian caregivers in africa. the findings reveal that emotional response stimuli play a major role during the participant informed consent process resulting in the involvement (or not) of a child within an rct. the study contributes to, and opens up, avenues for critical research on the role of informed consent as part of rct-related projects, especially concerning the involvement of children. this new knowledge may be leveraged to address participant uncertainties and subsequently improve the rate of paediatric recruitment in mhealth trial scenarios. keywords: emotional response stimuli; rational decision making; mhealth; consent; developing countries 1 introduction mhealth refers to the application of mobile information and communication technologies within the healthcare domain to support the delivery of healthcare services (lester, ritvo et al. 2010) and represents a shift in focus from traditional paper-based to digitised approaches to delivering healthcare services in an effort to improve the nature of care delivery (gianchandani 2011). in developing countries mhealth initiatives range from disease surveillance and control (e.g. malaria, hiv/aids, and diabetes), emergency response systems, human resource coordination, management and supervision, mobile-learning to health services monitoring and reporting (mechael 2006; varshney 2014). the increase of mhealth initiatives in australasian journal of information systems o’connor, heavin, gallagher & o’donohue 2017, vol 21, research on applied ethics and ict predicting participant consent in mhealth trials 2 developing countries may be attributable to the recent phenomenon of ‘information and communication technology for development’ (ict4d or ict4dev) which seeks to generate sustainable development among developing countries through the effective utilisation of ict (unwin 2009). the overall objective of ict4d is to foster economic and socio-economic growth in marginalised communities across the world (heeks 2008; unwin 2009). to complement ict4d, global initiatives in the form of millennium development goals (mdg) were proposed at the start of the decade (2005) which stimulated unprecedented efforts to meet the needs of the world’s poorest (i.e. focusing on issues of poverty, hunger, education, gender equality, child mortality, maternal health and environmental sustainability to name but a few). as the deadline for achieving mdg drew to an end (year: 2015) governments worldwide established sustainable development goals (sdg) to build and expand the existing goals to ensure longevity and sustainability of projects in developing countries. sdg commenced in january 2016 which inevitably will see the introduction of more mhealth projects within low and middle income countries, using the statistics of the previous decade as a basis. to understand mhealth initiatives and their associated implications, pilot studies are often performed prior to the deployment of the solution in routine clinical practice. while existing mhealth studies based around pilot projects have provided rich insights, it is argued that a more rigorous approach (i.e. a comparative study under control conditions) is required to fully appreciate if an intervention (i.e. mhealth application) is successful or not (cole-lewis and kershaw 2010; mechael 2010). such an approach is referred to as a randomised control trial (rct) and requires approval from participants before proceeding with the study. while the implementation of mhealth initiatives is relatively well understood, evidence of behavioural intentions towards consenting to participating in rcts is less clear. understanding this phenomenon is important as without consent, appropriate data may not be collected to empirically examine the implications of mhealth initiatives when delivering healthcare services to patients in a ‘real world context’. more specifically, the motivation to study rcts and the informed consent process particularly in developing countries is underpinned by the fact that introducing mobile technology in a clinical domain within low and middle income countries is contextually different from implementation initiatives in the developed world (walsham and sahay 2006; avgerou 2008). contextual factors reflect dynamic external forces constituted in the user groups’ social, cultural, economic, political, technological and institutional environment and, as such, comprise the environment or conditions for decision making tasks (edwards and steins 1999). such factors could potentially influence the informed consent process for approving child participant in a mhealth intervention within a clinical domain. moreover, nabulsi, khalil & makhoul (2010 p.420) argues that studies exploring parental perceptions in paediatric trials have been primarily investigated in developed countries with few studies investigating “similar parental experiences from non-industrialised countries, where clinical research faces economic, cultural and practical obstacles”. the next section explores the notion of research trials. 1.1 randomised control trial: definition and characteristics a randomised control trial (rct) is often conducted when an intervention project matures and its efficacy needs to be empirically established (nilsen, kumar et al. 2012). simply, an rct is performed to examine whether an intervention works. embracing this methodological evaluation requires the (1) use of a control condition to which the experimental intervention is compared; and (2) random assignment of participants to conditions (cf. gamble, haley et al. 2014). in the information systems (is) field pilot/feasibility studies are often performed to explore the feasibility of technological artefacts. however, when used in conjunction with clinical studies pilot/feasibility studies are considered “preliminary studies conducted specifically for the purposes of establishing whether or not a full trial will be feasible to conduct, and that all the necessary components of a trial will work together” (abbott 2014 p.555). rcts aim to investigate the effectiveness or efficacy of a healthcare intervention whereas preliminary studies (i.e. feasibility and pilot) aim to determine and to assess, respectively, the intervention australasian journal of information systems o’connor, heavin, gallagher & o’donohue 2017, vol 21, research on applied ethics and ict predicting participant consent in mhealth trials 3 under examination. thus, feasibility and pilot studies are very practical and descriptive in nature (lancaster, dodd et al. 2004) and used to underpin rcts. the predominant distinction between the three study types lies with the aim of the study. table 1 summarises the work of abbott (abbott 2014) who distinguishes between feasibility, pilot and rct studies. study type objective sample activities explored feasibility study to determine whether or not it will be feasible to conduct an rct of a particular intervention in a particular setting. willingness of clinicians to recruit participants, response rates to questionnaires, and loss to follow-up. pilot study to assess the key processes necessary for conducting the proposed main rct. processes for assessing eligibility, conducting baseline assessments, randomization procedures, treatment fidelity, and follow-up assessment. rct to investigate the efficacy or effectiveness of an intervention(s) compared with a comparison group. the null hypothesis that intervention a is not more effective than a comparison (typically either a control group or another intervention), in the case of a superiority trial. table 1. distinguishing between feasibility, pilot and rct studies (source: (abbott 2014)) it is important to recognise that different classifications of rcts exist: namely, parallel-group, crossover, cluster and factorial (hopewell, dutton et al. 2010). each classification is briefly described in table 1. notably, this list is not exhaustive and the authors acknowledge that additional classifications exist outside of the most commonly used classifications identified in table 2. rct classification description parallel-group each participant is randomly assigned to a group, and all the participants in the group receive (or do not receive) an intervention. cross-over over time, each participant receives (or does not receive) an intervention in a random sequence. cluster pre-existing groups of participants are randomly selected to receive (or not receive) an intervention. factorial participants are randomly assigned to individual interventions or a combination of interventions. table 2. description of rct classifications (source: gamble, haley et al. 2014) the classification criterion for an rct is dependent upon the research objective of the study. while rcts are primarily part of an epidemiological research tradition (richards and hamers 2009) their importance when investigating mhealth diagnostic tools is becoming widely recognised (chib 2013; davis 2014). one example of a study employing rcts to examine the efficacy of mhealth include watts et al., (2013) who conducted a rct comparing the delivery modality (mobile phone/tablet or fixed computer) of a cognitive behavioural therapy intervention for the treatment of depression. other examples based in developing countries can be found in the work of chang et al., (2011), hoffman et al., (2010), jones et al.,(2012), lester et al.,(2010); mbuagbaw et al.,(2012), pop-eleches et al., (2011), & zurovac et al., (2011). australasian journal of information systems o’connor, heavin, gallagher & o’donohue 2017, vol 21, research on applied ethics and ict predicting participant consent in mhealth trials 4 1.2 behavioural intention research behavioural intention research is well documented in the is field. common theories applied to exploring behavioural intentions include the theory of reasoned action (fishbein and ajzen 1975), theory of planned behaviour (ajzen 1991) and technology acceptance model (davis 1989). behavioural intention is a predictor of future behaviour (ajzen 1991). the majority of behavioural intention research focuses on the individual who participates in the research initiative themselves. however in paediatric trials/research, consent is obtained by proxy from the child’s parent(s) or guardian(s) (referred herein as caregiver) (peart 2000). it is argued (caldwell, murphy et al. 2004 p.805) that caregivers “are uncomfortable with this referred responsibility because of concerns about unknown or unexpected future side-effects and the possibility that the treatment their child receives might later be discovered to be ineffective or even harmful.” as a result, emotions and rational decision making are important factors when providing consent on the behalf of another individual for which one is responsible for. yet, a dearth of research focuses on child participation in studies from the caregivers’ perspective when technological artefacts are involved (carvalho and costa 2013). to add to this complexity rcts have certain characteristics which differentiate them from mainstream feasibility or pilot studies (see table 1). these characteristics have gone unexplored as part of behavioural intention research in the is domain to date. building from this, the objective of this paper is to explore behavioural intentions of caregivers to provide consent for children to participate in mhealth rcts in developing countries and subsequently develop a predictive model for the provision of informed consent. the remainder of this paper is structured as follows: the theoretical grounding to this study is first discussed focusing on emotional response stimuli and rational decision making. the methodology employed for the study is subsequently outlined. the findings are revealed resulting in the development of a conceptual model. the findings are then discussed in relation to extant literature and the implications of this study to both theory and practice. 2 theoretical grounding vast amounts of research have been conducted on the topic of emotions versus reasons (i.e. rational decision making) in different scientific fields since the early 1990s (e.g. macmurray (macmurray 1937; simon 1959; sousa 1979; toda 1980). the author(s) acknowledge that an association exists, and is well documented in literature, between emotions, rational decision making and behavioural intentions. the aim of this research is not to reinvent the wheel in this domain but instead to embrace such classical perspectives to enhance current understanding surrounding a relatively new phenomenon, namely the behavioural intentions of caregivers to provide consent for children to participate in mhealth trial scenarios. the decomposition of the broad constructs of emotional response stimuli and rational decision making provides a more holistic view of the consent process from a caregiver’s perspective within a healthcare context. healthcare is very much a personalised experience (rigby, roberts et al. 2000) which represents a markedly different social and technical context compared with many of the industries (e.g. finance and manufacturing) where research is conducted (chiasson and davidson 2004). healthcare consists of an extraordinarily diverse set of activities (lyons, woloshynowych et al. 2005) with a strong consumer focused perspective (kay 2007). additionally, finnell et al., (2003) argues that in an electronic health (ehealth) environment data is considered at the level of the community, rather than solely on an institutional basis. these views, aligned with the fact that mhealth technologies in developing countries are relatively new, provide a novel perspective in the area of emotions and reasoning. building on this, the following sections describe emotional response stimuli and rational decision making in the context of caregivers intending to provide (or not) consent for children to participate in mhealth trial scenarios. australasian journal of information systems o’connor, heavin, gallagher & o’donohue 2017, vol 21, research on applied ethics and ict predicting participant consent in mhealth trials 5 2.1 emotional response stimuli emotion is defined as a mental/cognitive reaction that transpires when individuals encounter significant relationships with others or with their environment (barrett and campos 1987). that is, emotions are “subjectively experienced state that can be described qualitatively and is accompanied by changes in feeling, physiology, and expression” (adam, gamer et al. 2011 p.4). this paper focuses on the stimuli which give rise to emotional responses in rct scenarios. more specifically, the authors concentrate on this approach to gain a rich understanding of the objects or events which can cause an emotional response (adam, gamer et al. 2011). applying this approach, referred herein as ‘emotional response stimuli’, often incorporates a conscious, cognitive appraisal of the stimulus/stimuli by the individual in certain circumstances. in the context of this research, rct emotional response stimuli refer to parallel-group (see table 1 for description) rct based stimulus/stimuli which could potentially be encountered by participants during the clinical trial. researchers have long advocated the importance by which emotional responses play in an individual’s decision making process (e.g. angie, connelly, waples & kligyt (2011); lerner & keltner (2000); paulus & yu (2012); sanfey (2007); schwarz (2000)). in 2015, the united nations children’s fund (unicef), world health organisation (who), world bank, united nations department of economic and social affair (un desa) population division recently reported 5.9 million children died worldwide before the age of 5 years (unicef 2015). that is a staggering 16,000 children who die on a daily basis due to preventable or treatable diseases. without doubt, the death of a child of any age is a profound, difficult, and painful experience. emotions can therefore have an impact on caregivers’ decision making. emotions either play a facilitating or hindering role in the decision-making process (de guinea and markus 2009; li, ashkanasy et al. 2014). for instance, chown, jones & henninger (2002 p.352) argue that “emotions are often seen as being disruptive to rational thought”. this viewpoint is also expressed by other theorists such as ashton-james & ashkanasy (2008) and lerner & tiedens (2006). conversely, damasio (damasio 1994; damasio 1998) has shown that emotions can improve an individual’s decision-making process. further, it is argued that emotions play an integral role in the decision-making process (tyszka and zaleskiewicz 2012; li, ashkanasy et al. 2014). building from this, if caregivers in developing countries perceive that involving their child(ren) in rct studies will increase their livelihoods and chance of survival then the likelihood of participant consent increases (jansen-van der weide, caldwell et al. 2015). yet, if they perceive that involvement will put their child(ren) at risk then the chance of caregivers’ providing consent is reduced (jansen-van der weide, caldwell et al. 2015). 2.2 rational decision making decision making is “a process of identifying a problem, evaluating alternatives, and selecting one alternative” (cole 2004 p.151). one traditional approach to understanding individual decision making is based upon edwards’ classical decision theory (edwards 1954). this theory focuses on instrumental rationality which employs a strategy by seeking the best possible alternative to maximise the achievement of goals and objectives. this implies that a clear set of alternate choices can be generated and their likely outcomes can be predicted with a significant degree of confidence. additional approaches are proposed (march 1958; tversky and kahneman 1974; mintzberg, raisinghani et al. 1976; simon 1979) which attempt to understand individual decision making. although the approach to understanding decision making varies in literature there appears to be widespread consensus that decision making often occurs under three conditions: certainty (outcomes of actions are certain), risk (outcomes are not certain but their probabilities are known, as in some games of chance), and uncertainty (probabilities of outcomes are unknown) (simon 1959). these are utilised to bring greater clarity to the decision making process. australasian journal of information systems o’connor, heavin, gallagher & o’donohue 2017, vol 21, research on applied ethics and ict predicting participant consent in mhealth trials 6 further to this, rational decision-making involves choosing between available alternatives so as to maximise ensuing benefits (simon 1979). that is, a good decision is perceived as having high outcome benefits (it is worthwhile) and low outcome costs (it is worth it). by this criterion, “utility maximisation could be seen as a rational decision-making model that follows shared and accepted rules of decision-making (li, ashkanasy et al. 2014 p.294)”. bearing this in mind we consider the case of malawi, africa. malawi is ranked as one of the ten poorest countries in the world with a high rate of child mortality and morbidity (callaghan-koru, gilroy et al. 2013). therefore, malawians may perceive that the advantages (e.g. improved healthcare services with a focus on reducing child mortality and morbidity) of participating in rct based studies may out-weigh the disadvantages (remaining with the status quo healthcare system). figure 1 represents a diagrammatic model of the association between emotional response stimuli, rational decision making and behavioural intentions. as part of our study, we set out to decompose this model to enhance our understanding of caregivers’ consenting their child(ren) in paediatric rcts in developing countries. figure 1. emotional response stimuli and rationale decision making model of caregivers’ behavioural intentions to provide consent for children to participate in mhealth randomised control trials the following section describes the methodology employed by the researchers to explore this model. based on the results, a conceptual model is proposed before the implications of this research conclude this paper. 3 methodology this research explores the behavioural intentions of caregivers to provide consent for children to participate in mhealth randomised control trials in developing countries and subsequently develop a predictive model for consent giving. a case study approach (i.e. mzuzu, northern malawi) was employed by the researchers as this facilitates an in-depth understanding of the phenomenon and its context (yin 1994; cavaye 1996). considered one of the most important sources of information in qualitative research (yin 1994; stake 1995) interviews were conducted for collecting data in an effort to describe the meanings of central themes in the world of study (kvale 1996). as a result, a qualitative case study approach was deemed appropriate to yield data regarding caregivers’ feelings about the process used to allocate their child(ren) to rcts. ethical approval was granted by the social research ethics committee (srec) within university college cork, ireland. each caregiver was provided with a description of the research prior to seeking informed consent. this description was made readily available in english or tambuku (local dialect) by members of an ngo (ungweru) who conduct community based activities in the local region. each caregiver was also provided with an informed consent form indicating that the purpose and nature of the study had been explained to the participant (i.e. caregiver); they were participating voluntarily; they understood that they could withdraw from the study, without repercussions, at any time, whether before it starts or while they were participating; they understood that anonymity would be ensured in the write-up by disguising any identity. once the caregivers understood the study and were willing to participate in it informed consent was obtained from the participant vis-à-vis a signature and date. working closely with community health workers (chws) in the community, ten households were identified which had sick children under the age of five who previously attended a clinic australasian journal of information systems o’connor, heavin, gallagher & o’donohue 2017, vol 21, research on applied ethics and ict predicting participant consent in mhealth trials 7 in the community of dunduzu and doroba in mzimba north, malawi, africa. in their study of two african countries, guest, bunce & johnson (2006) undertook a detailed data analysis of the transcript coding process of sixty interviewees. the results of this study showed that 92 percent of codes identified for the entire sample were found in the early stage of data analysis amongst the first twelve interviews (guest, bunce et al. 2006). for this study, ten interviews with female caregivers were performed in collaboration with our african partners using a purposive sampling approach (patton 1980). like guest et al., (2006), this study included a comparatively homogenous population and had a focused objective. these factors are important in order for theoretical saturation to be achieved in a study such as this (eisenhardt 1989; guest, bunce et al. 2006). interviews were conducted in the local dialect of tambuku in the caregivers’ homes. each participant was given a standardised briefing prior to the interview about rcts and what they entail. the data was analysed using open, axial and selective coding as advocated by strauss & corbin (strauss and corbin 1990). the rationale for employing their techniques is that it is favourable for a research study engaged in advancing current understanding engaged in theory building. moreover, these content analysis techniques can be utilised in the absence of, or in conjunction with existing theory (strauss and corbin 1990; urquhart 2001). operationalising this content analysis approach required the researchers to first examine the data ‘word-by-word’/‘line-byline’ to ascertain the main ideas (open coding). through comparative analysis across interviews and with regards to similarities and differences, the researchers then grouped codes together and formed, where applicable, more abstract categories or themes. the next step examined the data to establish if relationships between categories and other (sub) categories exist (axial coding). finally, selective coding was undertaken to identify the relationships between categories using hypothesised conditions, context, strategies and consequences. 4 findings this section presents the findings of this study and discusses its implications for the a priori model (figure 1). findings enable the researchers to refine the conceptual model derived from existing literature, more specifically in terms of emotional response stimuli and rational decision making. as a result, a revised model is developed and presented (figure 2). 4.1 overview of interviewees table 3 provides an overview of the interviewees’ demographics. data was gathered during october 2014 and all interviews were transcribed into english for analysis. question overview age 50% (n=5) between 18-28 years 50% (n=5) between 29-39 years level of education 10% (n=1) junior primary 30% (n=3) senior primary 20% (n=2) junior certificate of education 40% (n=4) malawi school certificate of education table 3. overview of interviewees’ demographics 4.2 emotional response stimuli in this paper, the authors identified a number of rct based emotional response stimuli which have an impact on caregivers rational decision making when intending to provide consent for australasian journal of information systems o’connor, heavin, gallagher & o’donohue 2017, vol 21, research on applied ethics and ict predicting participant consent in mhealth trials 8 a child to participate in mhealth rct. these include (1) perceptions of rcts involving children, (2) perceptions of chance allocation, (3) perceptions of experimentation, (4) perceived (mis)trust of mobile technology and (5) perceived novelty. the first stimulus which triggers emotional responses is that of ‘perceptions of rcts involving children’. interviewees’ perceived that clinical research involving children was a positive initiative. this is exemplified in a comment from one caregiver who stated “its [clinical research that involves children less than five years of age] a wonderful area to explore because children by themselves fail to communicate effectively the signs and symptoms of what they suffer from” (caregiver 1). this viewpoint is similarly expressed by another caregiver stating “children don’t express properly what they suffer from. i feel it will help to find an efficient way of helping out children” (caregiver 3). the second stimulus to elicit emotional responses from caregivers was that of ‘perceptions of chance allocation’. as outlined earlier (section 1.1), rcts often require the use of a control condition to which the experimental intervention is compared and a random assignment of participants to conditions (note: for the purpose of this research, the control condition would involve using the existing paper-based approach to delivering healthcare services to children less than five years of age). the findings reveal that the perception of chance allocation did not have a significant influence on their decision making process providing the correct diagnosis and treatment were received by the sick child. this is reflected in the following comments; “i want help for my sick child. both approaches [paper based or mobile] will help our sick children. i don’t see any issue using this arrangement” (caregiver 3); “i feel there is no problem because both approaches help the children” (caregiver 6); and “to use both approaches is ok with me, because the approaches work to help our children. what i want is to get my child treated regardless of the approach used” (caregiver 9). interestingly, one care-giver (number 5) stated that this controlled approach was beneficial as “it’s an opportunity to compare the performance of the two approaches.” in some situations however, caregivers perceived that this element of chance would only cause confusion in the community. this is reflected in the following comment such as “i feel they should use one approach, with paper-based or mobile technology to avoid confusing the community” (caregiver 7). the third stimulus, ‘perceptions of experimentation’, also triggered some emotional responses from the caregivers interviewed in this study. it is during an rct that the efficacy of mhealth needs to be empirically established thus, there remains an element of ‘trial and error’. the findings reveal that caregivers do not fear the experimentation element associated with the technological artefact provided that the chws (end users of the mhealth during the rct) continue to rely on their tacit knowledge and experience. that is, they have control over the technology during the trial. this is exemplified in the following comments; “the community health workers have been diagnosing and treating my children all along in this village. they will use the mobile technology, so i feel they will have control” (caregiver 2) and “it will be controlled by trained and experienced community health workers” (caregiver 3). the fourth stimulus is closely related to the previous stimulus and includes ‘(mis)trust of the technology’. perceptions of the trustworthiness of the mhealth technology varied across caregivers. for instance comments ranged from “i trust the technology because i have been told that mobile technology works the same way as the paper-based approach” (caregiver 4), “it’s a new technology. i will trust it” (caregiver 5), “if the community health workers will operate it, i will trust it” (caregiver 6) and “i believe the mobile technology is like any other technologies in big hospitals to diagnose different diseases. based on that, i will trust the mobile technology” (caregiver 9). conversely, one caregiver doubted the technology stating that “i do not trust the technology because i am not sure about its performance” (caregiver 10). another caregiver would only trust the technology providing that the output would confirm what the caregiver already was aware of: “before i go to the clinic for diagnosis and treatment of my child i will have the knowledge and behaviour of the child. if the mobile technology will confirm what i had observed at home, then i will trust the mobile technology” (caregiver 1). australasian journal of information systems o’connor, heavin, gallagher & o’donohue 2017, vol 21, research on applied ethics and ict predicting participant consent in mhealth trials 9 the fifth and final stimulus to trigger emotional responses from the caregivers is ‘perceived novelty’. the caregivers interviewed in this study had little to no experience with smartphone devices so introducing such an innovation into their community was perceived as an exciting initiative. the perceived novelty of the technology was found not to discourage caregivers from providing consent but in fact welcomed the initiative. for example, “i welcome the idea of clinical research which uses mobile technology because that is the way to go these days. the world has become technological” (caregiver 7) and “i’ve not seen the mobile technology before. i look forward to seeing it being tested here” (caregiver 9). while the perceived novelty was not seen as an obstacle for the majority of caregivers, there was one caregiver who expressed his/her concerns with the technology stating “i’ve doubts about its functionality” (caregiver 10). the findings presented in this section demonstrate that perceptions of rcts involving children, chance allocation, experimentation, (mis)trust of mobile technology and novelty are emotional response stimuli often encountered by caregivers in an mhealth trial scenario. the following section now focuses on rational decision making. 4.3 rational decision making existing studies argue that rational decision making can impact the behavioural intentions of individuals. for this study, two rational decision making concepts were identified. these include (1) perceived net benefits and (2) perceived uncertainty costs. first, the ‘perceived net benefits’ associated with the mhealth rct include improved diagnosis and treatment of a sick child, improved delivery of effective and efficient healthcare services at the point-of-care and improved accuracy in terms of maintaining the medical records of a sick child. these perceived net benefits were reported by all caregivers with comments such as “one of the advantages is that my child will get the right diagnosis and right treatment” (stated by caregiver 1 but similarly expressed by caregivers 2,3,5,6 and 9). improved effective and efficient services were reflected in comments such as “my child will get a transparent assessment” (caregiver 4) and “my opinion is that this is an efficient way of patient assessment” (caregiver 8). caregiver 5 expressed that “the community health workers will greatly benefits as much as the community” which depicts the societal wide perceived net benefits associated with mhealth rct initiatives. the most important perceived net benefit was reflected in a comment from caregiver 10 who stated that “the idea is to help children to have a healthy life.” the second rational decision making concept identified in this paper is that of ‘perceived uncertainty costs’. essentially an rct is performed to examine whether an intervention works thus, there is an element of uncertainty associated with the mhealth intervention. this uncertainty is reflected in various comments from caregivers such as “i feel it is a risk because i don’t know perfectly well how it will work on our children here” (caregiver 2) and “it has a risk” (caregiver 8). caregiver 10 reveals that the status quo of using the paper-based approach is precise when delivering healthcare services to children as “it’s been tested”. the findings presented in this section demonstrate that perceptions of net benefits and uncertainty are also encountered by caregivers in a trial scenario. overall, the findings revealed that emotional response stimuli affect rational decision making. this is exemplified in table 4. the chain of evidence vis-à-vis beaudry & pinsonneault (2005) presented in table 4 acts to underpin the proposition that emotional response stimuli influences rational decision making. australasian journal of information systems o’connor, heavin, gallagher & o’donohue 2017, vol 21, research on applied ethics and ict predicting participant consent in mhealth trials 10 relationship evidence perceptions of rcts involving children > net benefits and uncertainty costs “i feel the clinical research will help ways that will enable our children to grow healthy. i also see it as a challenge because i’m not really sure to what extent children will be directly involved” (caregiver 4). “my child will set an example to others about the importance of participating in clinical research for the good of others” (caregiver 5). “i want my child to get diagnosed and get the right treatment… i (my child) look forward to participating in clinical research despite the fact that i’m not sure about its functionality” (caregiver 6). perceptions of chance allocation > net benefits and uncertainty costs “i prefer mobile technology because the paper based might have a higher risk of making a wrong diagnosis and treatment” (caregiver 1). “i prefer the mobile technology because i feel the mobile technology will be more accurate than paper based approach in diagnosis and treatment” (caregiver 4). perceptions of experimentation > net benefits and uncertainty costs “because the mobile technology will work just as the paper based, then i don’t have any problem. however, i would like that the mobile technology be tested first” (caregiver 1). “my child will set an example to others about the importance of participating in clinical research for the good of others” (caregiver 5). (mis)trust of technology > net benefits and uncertainty costs “it will be difficult in the first place to completely trust the clinical research that uses mobile technology but i fell it will provide better and efficient health services to children” (caregiver 2). “while i’m happy that my child may be exposed to clinical research using mobile technology, i still need convincing that it will work perfectly well” (caregiver 4). perceived novelty> net benefits and uncertainty costs “it will be completely new and strange. it will be a challenge to understand it when they are used at the clinic” (caregiver 3). “i agree that i have not seen it before. however, i fell this clinical research is important for the good health of our children” (caregiver 9). table 4. evidence of emotional response stimuli influencing rational decision making 4.4 outcome: intentions to provide consent for children to participate in mhealth rct from the ten caregivers interviewed, only one caregiver would refuse to give consent for their child(ren) to participate in mhealth rct studies. the key concerns stemmed from the emotional responses experienced by the caregiver, primarily surrounding ‘fear of experimentation’, ‘mistrust of technology’ and ‘inclusion of children in clinical trials’. this caregiver in this situation perceived that the advantages of participating did not outweigh the disadvantages. yet, despite this one caregiver refraining to provide consent, the remaining caregivers intend to provide consent for their child (ren) to participate in mhealth rct studies if asked. the findings reveal that emotional response stimuli primarily facilitated rational decision making however, there were instances whereby emotional response stimuli would hinder rational decision making. based on the findings presented a decomposed conceptual model (figure 2) is illustrated for future empirical testing and validation. australasian journal of information systems o’connor, heavin, gallagher & o’donohue 2017, vol 21, research on applied ethics and ict predicting participant consent in mhealth trials 11 figure 2. predictive conceptual model of caregivers’ behavioural intentions to provide consent for children to participate in mhealth randomised control trials with any initiative, identifying the pros and cons to a project may assist in an individual’s decision making process. while our findings support this argument it was also identified that emotional response stimuli can influence how a project is perceived in terms of advantages and disadvantages. this new conceptual model (figure 2) portrays these associations. 5 discussion the impact of emotional response stimuli and rational decision making on caregivers’ behavioural intentions to provide consent for children to participate in mhealth rct has gone relatively unnoticed in existing literature. from synthesising the literature a preliminary model is initially proposed, which is further refined vis-à-vis a case study of caregivers in mzimba north, malawi africa. the findings presented in this paper corroborate decision making research which has found that emotional response stimuli can both hinder (caldwell, butow et al. 2003) and facilitate (zupancic, gillie et al. 1997) rational decision making thus, ultimately impacting whether or not caregivers consent their children to participate in mhealth rct studies in developing countries. the findings identified five emotional response stimuli associated with mhealth rct characteristics including (1) perceptions of rcts involving children, (2) perceptions of chance allocation, (3) perceptions of experimentation, (4) perceived (mis)trust of mobile technology and (5) perceived novelty. two rational decision making concepts were also identified influencing the behavioural intentions of caregivers to provide consent on behalf of their children to participate in mhealth rct (1) perceived net benefits and (2) perceived uncertainty costs. figure 3 depicts the recommendations which will be further outlined. australasian journal of information systems o’connor, heavin, gallagher & o’donohue 2017, vol 21, research on applied ethics and ict predicting participant consent in mhealth trials 12 figure 3. improving child recruitment in mhealth clinical trials perceptions of rcts involving children increases caregivers’ vulnerability and contradictory feelings about the trial as the caregiver is not directly receiving diagnosis/treatment based on the output from the mhealth technology. similarly, caldwell et al., (2004) identified that caregivers were more willing to provide consent for their own participation in trails than assenting their child(ren). caregivers’ fear of harm or adverse events which potentially could be faced by the child is further enhanced through their perceptions of experimentation. often parental misconceptions about the research process such as the meaning of randomisation and perceived risks associated with the intervention (i.e. mhealth technology) influences their decision (nabulsi, khalil et al. 2010). yet, caregivers’ perceive that the benefits for their child(ren) participating in mhealth rcts outweighs the risks involved. a possible reason for this may be attributable to the contextual environment caregivers’ are situated in. the current delivery of healthcare services in malawi, africa is quite fragmented, with insufficient resources and suffers from a brain drain of highly skilled workers (coloma and harris 2009) which leaves a shortage of well-educated professionals. research using mhealth technologies in trial scenarios involving children found that caregivers perceived that community health workers using mhealth technologies provided (a) a more thorough examination of their child and (b) were more knowledgeable as a result (mitchell, getchell et al. 2012). in the work of mitchell et al., (2012), caregivers had never seen or interacted with the mhealth technology. therefore, the perceived novelty of the technological artefact isn’t seen as an inhibitor but more a facilitator in the consent process. based on these results recruitment to trials in developing countries in terms of the consent giving process can be improved by considering the contextual nature in which mhealth interventions are being introduced. the critical component is providing education to the caregiver surrounding the intervention itself (i.e. mhealth technology and how it will be used in the trial). given the important role that non-governmental organisations (ngos) play in developing countries, research teams should corroborate with local ngos in the area for which the trial is envisioned to take place as ngos currently provide information to communities in developing countries on various initiatives. individuals within ngos should be educated on the trial protocol. it is also clinically imperative that the community health workers are educated on the trial and more specifically receive training to ensure that they can utilise the australasian journal of information systems o’connor, heavin, gallagher & o’donohue 2017, vol 21, research on applied ethics and ict predicting participant consent in mhealth trials 13 mhealth technology according to trial protocol. to ensure that both community health workers and ngos are sufficiently educated the research team must have the trial coordination activities clearly articulated and trial information documented prior to commencing the trial. trial information should explicitly detail how the mhealth intervention will be used during the trial and what the implications are for the child involved (i.e. individual and community-wide benefits and risks). this information, supplemented with additional information around mobile technology and healthcare, should be relayed in a clear, concise manner by the ngo to caregivers prior to the trial to increase awareness around the intervention. caregivers should also be made aware that participation in mhealth rcts are on a voluntary basis whereby the caregiver and his/her child are free to leave the study at any stage. to ensure full commitment by caregivers it is imperative that a two-way communication path is implemented between the consent giver and the health workers/local ngos to facilitate any question and answer sessions. in some situations financial incentives may encourage caregivers to provide consent but no evidence of this was found in this study. 6 conclusion the traditional paper-based approach for recording and exchanging clinical data in healthcare environments is progressing towards automation. the arrival of mhealth has created an opportunity to document healthcare information in electronic format at the point-of-care. recognising the profound benefits that such technological tools can offer many mhealth initiatives have been and will continue to be deployed in developing countries. as healthcare interventions, mhealth technologies are undergoing rigorous efficacy and/or effectiveness trials in resource poor settings. the majority of these interventions target cohorts of children under the age of five years in an effort to address the high mortality rates which exists in such regions of the world. however, in order to conduct such investigations (commonly referred to as randomised clinical trials) requires consent to be given by a caregiver. yet, a dearth of research exists highlighting the behavioural intentions of caregivers to provide consent for children to participate in trials involving mobile technology (carvalho and costa 2013). this paper attempts to close this gap by exploring perceptions of caregivers in malawi, africa on the informed consent process, focusing on emotional response stimuli and rational decision making in order to better predict the likelihood of participant consent in studies such as these. the findings identified five emotional response stimuli and two rational decision making concepts which influence the behavioural intentions of caregivers to provide consent, on behalf of their children, to participate in mhealth rct. the findings presented in this paper should be considered in the context of the study’s limitations. firstly, this study did not take into account the severity of a child’s illness and how this may have an impact on caregivers’ behavioural intentions. secondly, this study focused on caregivers providing (or not) for children specifically under the age of five years. thirdly, this research solely focused on one classification of rcts (i.e. parallel-group). fourth, it is important to note, that an inverse association between emotional stimuli and rational decision making can exist. that is, rational arguments can affect emotional stimuli. for instance, data presented in this paper reveals that ‘mistrust of technology affects net benefits and uncertainty costs’, but the researchers acknowledge (although no evidence was found in the dataset) the counter argument that net benefits and uncertainty costs affect mistrust. while the research presented in this paper has emphasised a clear gap in the existing literature and the case study results highlight the influence of emotional response stimuli and rational decision making on the behavioural intentions of caregivers to provide consent for children to participate in mhealth rct, the authors argue that future research in this area would require a broader spectrum of participants across a number of demographics to provide a richer picture of the findings. more specifically, researchers should examine testable / verifiable elements around the model (i.e. creating hard data) which address the limitations of this study to enhance current understanding in this domain. the model could be further improved by introducing a rating scale of ‘very emotional’ to ‘very rational’ to determine if contextual factors australasian journal of information systems o’connor, heavin, gallagher & o’donohue 2017, vol 21, research on applied ethics and ict predicting participant consent in mhealth trials 14 (e.g. age of the child, severity of the disease, etc.) affect the emotional and/or rational responses of caregivers. nonetheless, it is imperative to reemphasise that clinical trials involving children are argued to have resulted in significant improvements in the diagnosis and treatment of paediatric healthcare services (caldwell, murphy et al. 2004; molyneux, mathanga et al. 2012). this study highlights the importance of obtaining informed consent as part of mhealth studies. in doing so, this paper contributes to research and practice by (1) deriving a predictive model of consent giving, (2) identifying the role of emotions and reasoning as part of informed consent and (3) understanding caregivers’ perceptions surrounding trial participation, facilitating improved engagements with participants may result in terms of paediatric recruitment. it is essential that caregivers are educated about mhealth rcts and the opportunities that they afford in terms of potential improvements in the availability, efficiency and effectiveness of health services with the opportunity to improve patient health outcomes in the long term. acknowledgement the supporting life project (305292) is funded by the seventh framework programme for research and technological development of the eu. references abbott, j. h. (2014). "the distinction between randomized clinical trials (rcts) and preliminary feasibility and pilot studies: what they are and are not." journal of orthopaedic & sports physical therapy, 44(8): 555-558. adam, m., m. gamer, et al. (2011). measuring emotions in electronic markets. thirty second international conference on information systems, shanghai 2011 ajzen, i. (1991). "the theory of planned behavior." organizational behavior and human decision processes, 50(2): 179-211. angie, a. d., s. connelly, et al. (2011). "the influence of discrete emotions on judgement and decision-making: a meta-analytic review." cognition & emotion, 25(8): 1393-1422. ashton-james, c. e. and n. m. ashkanasy (2008). "affective events theory: a strategic perspective." research on emotion in organizations, 4: 1-34. avgerou, c. (2008). "information systems in developing countries: a critical research review." journal of information technology, 23(3): 133-146. barrett, k. c. and j. j. campos (1987). perspectives on emotional development: a functional approach to emotions. handbook of infant development, 2nd edition. . j. d. osofsky. new york, wiley: 555–578. beaudry, a. and a. pinsonneault (2005). "understanding user responses to information technology: a coping model of user adaptation." mis quarterly, 29(3): 493-524. caldwell, p. h., p. n. butow, et al. (2003). "parents' attitudes to children's participation in randomized controlled trials." the journal of paediatrics, 142(5): 554-559. caldwell, p. h., s. b. murphy, et al. (2004). "clinical trials in children." the lancet, 364(9436): 803-811. callaghan-koru, j. a., k. gilroy, et al. (2013). "health systems supports for community case management of childhood illness: lessons from an assessment of early implementation in malawi." bmc health services research, 13(1): 55. carvalho, a. a. and l. r. costa (2013). "mothers’ perceptions of their child’s enrollment in a randomized clinical trial: poor understanding, vulnerability and contradictory feelings." bmc medical ethics, 14: 52-52. cavaye, a. l. m. (1996). "case study research: a multi‐faceted research approach for is." information systems journal, 6(3): 227-242. australasian journal of information systems o’connor, heavin, gallagher & o’donohue 2017, vol 21, research on applied ethics and ict predicting participant consent in mhealth trials 15 chang, l. w., j. kagaayi, et al. (2011). "impact of a mhealth intervention for peer health workers on aids care in rural uganda: a mixed methods evaluation of a clusterrandomized trial." aids and behavior, 15(8): 1776-1784. chiasson, m. w. and e. davidson (2004). "pushing the contextual envelope: developing and diffusing is theory for health information systems research." information and organization, 14(3): 155-188. chib, a. (2013). "the promise and peril of mhealth in developing countries." mobile media & communication, 1(1): 69-75. chown, e., r. m. jones, et al. (2002). an architecture for emotional decision-making agents. proceedings of the first international joint conference on autonomous agents and multiagent systems: part 1, acm. cole-lewis, h. and t. kershaw (2010). "text messaging as a tool for behavior change in disease prevention and management." epidemiologic reviews, mxq004. cole, g. a. (2004). management theory and practice, cengage learning emea. coloma, j. and e. harris (2009). "from construction workers to architects: developing scientific research capacity in lowincome countries." plos biol, 7(7): e1000156. damasio, a. (1994). "descartes’ error: emotion, reason, and the human brain. grosset/putnam." new york. damasio, a. r. (1998). "emotion in the perspective of an integrated nervous system." brain research reviews, 26(2): 83-86. davis, f. d. (1989). "perceived usefulness, perceived ease of use, and user acceptance of information technology." mis quarterly, 13(3): 319-340. davis, t. (2014). "going mobile: how mobile technology is evolving in clinical trials." url: http://www.clinicalleader.com/doc/going-mobile-how-mobile-technology-is-evolvingin-clinical-trials-0001 [accessed: nov 26th 2015] de guinea, a. o. and m. l. markus (2009). "why break the habit of a lifetime? rethinking the roles of intention, habit, and emotion in continuing information technology use." mis quarterly, 33(3): 433-444. edwards, v. and n. steins (1999). "a framework for analysing contextual factors in common pool resource research." journal of environmental policy & planning, 1(3): 205-221. edwards, w. (1954). "the theory of decision making." psychological bulletin, 51(4): 380. eisenhardt, k. m. (1989). "building theories from case study research." the academy of management review, 14(4): 532-550. finnell, j. t., j. m. overhage, et al. (2003). community clinical data exchange for emergency medicine patients. amia annual symposium proceedings, american medical informatics association. fishbein, m. and i. ajzen (1975). belief, attitude, intention and behavior: an introduction to theory and research, reading, ma: addison-wesley. . gamble, t., d. haley, et al. (2014). "designing randomized controlled trials (rcts)." public health research methods (eds. greg guest & emily namey), pp. 224-247. sage publications. gianchandani, e. p. (2011). "toward smarter health and well-being: an implicit role for networking and information technology." journal of information technology, 26(2): 120-128. guest, g., a. bunce, et al. (2006). "how many interviews are enough? an experiment with data saturation and variability." field methods, 18(1): 59-82. australasian journal of information systems o’connor, heavin, gallagher & o’donohue 2017, vol 21, research on applied ethics and ict predicting participant consent in mhealth trials 16 heeks, r. (2008). "ict4d 2.0: the next phase of applying ict for international development." computer, 41(6): 26-33. hoffman, j. a., j. r. cunningham, et al. (2010). "mobile direct observation treatment for tuberculosis patients: a technical feasibility pilot using mobile phones in nairobi, kenya." american journal of preventive medicine, 39(1): 78-80. hopewell, s., s. dutton, et al. (2010). the quality of reports of randomised trials in 2000 and 2006: comparative study of articles indexed in pubmed. bmj, mar 23;340:c723. doi: 10.1136/bmj.c723 jansen-van der weide, m. c., p. h. caldwell, et al. (2015). "clinical trial decisions in difficult circumstances: parental consent under time pressure." pediatrics, 136(4): e983-e992. jones, c. o., b. wasunna, et al. (2012). "“even if you know everything you can forget”: health worker perceptions of mobile phone text-messaging to improve malaria casemanagement in kenya." plos one, 7(6): e38636. kay, m. j. (2007). "healthcare marketing: what is salient?" international journal of pharmaceutical and healthcare marketing, 1(3): 247-263. kvale, d. (1996). interviews. london, sage publications. lancaster, g. a., s. dodd, et al. (2004). "design and analysis of pilot studies: recommendations for good practice." journal of evaluation in clinical practice, 10(2): 307-312. lerner, j. s. and d. keltner (2000). "beyond valence: toward a model of emotion-specific influences on judgement and choice." cognition & emotion, 14(4): 473-493. lerner, j. s. and l. z. tiedens (2006). "portrait of the angry decision maker: how appraisal tendencies shape anger's influence on cognition." journal of behavioral decision making, 19(2): 115-137. lester, r. t., p. ritvo, et al. (2010). "effects of a mobile phone short message service on antiretroviral treatment adherence in kenya (weltel kenya1): a randomised trial." the lancet, 376(9755): 1838-1845. li, y., n. m. ashkanasy, et al. (2014). "the rationality of emotions: a hybrid process model of decision-making under uncertainty." asia pacific journal of management, 31(1): 293308. lyons, m., m. woloshynowych, et al. (2005). "error reduction in medicine." final report to the nuffield trust, the nuffield trust. macmurray, j. (1937). "reason and emotion." london, d. appleton-century company incorporated. march, j. g. (1958). "a behavioral theory of decision making." personnel administration, 21(3): 8-10. mbuagbaw, l., l. thabane, et al. (2012). "the cameroon mobile phone sms (camps) trial: a randomized trial of text messaging versus usual care for adherence to antiretroviral therapy." plos one, 7(12): e46909. mechael, p. (2010). opportunities and challenges of integrating mhealth applications into rural health initiatives in africa. handbook of research on developments in e-health and telemedicine: technological and social perspectives (2 volumes) (eds. a. t. m. cruz-cunha, & r. simoes). igi global, 2010. 1-1486. web. 3 oct. 2016. mechael, p. n. (2006). exploring health-related uses of mobile phones: an egyptian case study, london school of hygiene and tropical medicine. url: https://wiki.ihris.org/mediawiki/upload/patriciamechaelthesisfinaldecember2006.p df [accessed: 17th july 2016] australasian journal of information systems o’connor, heavin, gallagher & o’donohue 2017, vol 21, research on applied ethics and ict predicting participant consent in mhealth trials 17 mintzberg, h., d. raisinghani, et al. (1976). "the structure of" unstructured" decision processes." administrative science quarterly, 21(2): 246-275. mitchell, m., m. getchell, et al. (2012). "perceived improvement in integrated management of childhood illness implementation through use of mobile technology: qualitative evidence from a pilot study in tanzania." journal of health communication, 17(sup1): 118-127. molyneux, e., d. mathanga, et al. (2012). "practical issues in relation to clinical trials in children in low-income countries: experience from the front line." archives of disease in childhood, 97(9): 848-851. nabulsi, m., y. khalil, et al. (2010). "parental attitudes towards and perceptions of their children's participation in clinical research: a developing-country perspective." journal of medical ethics, jme. 2010.035899. nilsen, w., s. kumar, et al. (2012). "advancing the science of mhealth." journal of health communication, 17(sup1): 5-10. patton, m. q. (1980). qualitative evaluation methods, sage publications beverly hills, ca. paulus, m. p. and a. j. yu (2012). "emotion and decision-making: affect-driven belief systems in anxiety and depression." trends in cognitive sciences, 16(9): 476-483. peart, n. (2000). "health research with children: the new zealand experience." curr legal issues, 3: 421-439. pop-eleches, c., h. thirumurthy, et al. (2011). "mobile phone technologies improve adherence to antiretroviral treatment in a resource-limited setting: a randomized controlled trial of text message reminders." aids (london, england), 25(6): 825. richards, d. a. and j. p. hamers (2009). "rcts in complex nursing interventions and laboratory experimental studies." international journal of nursing studies, 46(4): 588592. rigby, m., r. roberts, et al. (2000). taking health telematics into the 21st century, the radcliffe press. sanfey, a. g. (2007). "decision neuroscience new directions in studies of judgment and decision making." current directions in psychological science, 16(3): 151-155. schwarz, n. (2000). "emotion, cognition, and decision making." cognition & emotion, 14(4): 433-440. simon, h. a. (1959). "theories of decision-making in economics and behavioral science." the american economic review, 49(3): 253-283. simon, h. a. (1979). "rational decision making in business organizations." the american economic review, 69(4): 493-513. sousa, r. d. (1979). "the rationality of emotions." dialogue, 18(01): 41-63. stake, r. (1995). the art of case study research. thousand oaks, ca, sage. strauss, a. and j. corbin (1990). basics of qualitative research: grounded theory procedures and techniques, newbury park, sage publications inc., california. toda, m. (1980). "emotion and decision making." acta psychologica, 45(1): 133-155. tversky, a. and d. kahneman (1974). "judgment under uncertainty: heuristics and biases." science, 185(4157): 1124-1131. tyszka, t. and t. zaleskiewicz (2012). "the strength of emotions in moral judgment and decision-making under risk." polish psychological bulletin, 43(2): 132-144. australasian journal of information systems o’connor, heavin, gallagher & o’donohue 2017, vol 21, research on applied ethics and ict predicting participant consent in mhealth trials 18 unicef (2015). levels & trends in child mortality. http://www.childmortality.org/files_v20/download/igme%20report%202015_9_3% 20lr%20web.pdf [accessed: july 17th 2016] unwin, p. (2009). ict4d: information and communication technology for development, cambridge university press. urquhart, c. (2001). an encounter with grounded theory: tackling the practical and philosophical issues, in qualitative research in is: issues and trends. e. t. (ed.), idea group publishing: pp. 104-140. varshney, u. (2014). "mobile health: four emerging themes of research." decision support systems, 66(0): 20-35. walsham, g. and s. sahay (2006). "research on information systems in developing countries: current landscape and future prospects." information technology for development, 12(1): 7-24. watts, s., a. mackenzie, et al. (2013). "cbt for depression: a pilot rct comparing mobile phone vs. computer." bmc psychiatry, 13(1): 49. wilson, c. z. and m. alexis (1962). "basic frameworks for decisions." academy of management journal, 5(2): 150-164. yin, r. k. (1994). case study research, design and methods, sage publications, newbury park. zupancic, j. a., p. gillie, et al. (1997). "determinants of parental authorization for involvement of newborn infants in clinical trials." pediatrics, 99(1): e6-e6. zurovac, d., r. k. sudoi, et al. (2011). "the effect of mobile phone text-message reminders on kenyan health workers' adherence to malaria treatment guidelines: a cluster randomised trial." the lancet, 378(9793): 795-803. copyright: © 2017 o’connor, heavin, gallagher & o’donohue. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems chua & zhang 2020, vol 24, research article open source developers & forking 1 applying a systematic literature review and content analysis method to analyse open source developers’ forking motivation interpretation, categories and consequences bee bee chua faculty of engineering and information technology university of technology, sydney choiceno1@mailcity.com ying zhang faculty of engineering and information technology university of technology, sydney abstract in open source (os) environments, forking is a powerful social collaborative technique that creates a social coding community and increases code visibility but it has not been adopted by os software (oss) developers. this paper investigates os forking divergence using contextual frameworks (systematic literature review and content analysis) to analyse oss developer forking motivation, interpretation, categorisation and consequences. we identified five theoretical forking patterns: 1) forking can revive original project health; 2) few effective frameworks exist to describe project-to-project developer migration; 3) there is a literature on social forking community behaviour; 4) poor guidance is a threat to forking; and 5) most research uses mixed methods. we introduce guidelines for oss communities to reduce organisational barriers to developer motivation and highlight the important of understanding developer forking. the challenge remains to analyse forking and sustainability from a social community perspective, particularly how programming language, file repositories and developer interest can predict forking motivation and behaviour for both novice oss developers or experienced developers who want to improve forking performance. keywords: open source, forking, motivation, sustainability, systematic literature review, fork visibility 1 introduction github is a hosting website for developing open source software (oss) through social coding by multiple developers. github stores projects, files, programing languages, licenses and developer profiles. in may 2019, github reported having over 37 million users since its inception, and more than 100 million repositories (including at least 28 million public repositories), making it the largest global host of open source (os) code (gousios et al., 2014). github currently has 26 million registered developers from 110,000 organisations and an additional 20 million developers and users visit github daily without registering (alexa, 2017). github has long-term viability and remains on the cutting edge of technology, particularly the forking feature, which many developers adopt and use. forking is an important feature in github, allowing developers to make a copy of original source code, download it into their own environment to learn from or make changes, then submit adapted code back to the project owners (sometimes referred to as ‘upstream’). when australasian journal of information systems chua & zhang 2020, vol 24, research article open source developers & forking 2 a file is forked by developers in github, the developer may indirectly adapt it to enhance the programming language longevity. developers may download a programming language not only because the language file repository is interesting and unique but because it also may have strong compliance and interoperability with local developmental environments. however, most os projects do not receive high forking counts and there is currently no reliable method of determining whether developer motivation behind projects with the most forked files is ‘genuine’ or ‘non-genuine’. genuine motivation would be developers who are willing to contribute, rewrite source codes and submit them upstream for owners to accept and merge; non-genuine developers would simply retain the code – adapted or not – for their own purposes, without submitting it upstream. moreover, programming language use, adoption and forking varies, based on the number of projects and file repositories, so the evidence base on developer forking motivation behaviour is unclear. a project can have one or multiple programming languages to allow one or more developers to create single or multiple file repositories. github hosts 339 active programming languages yet less than one twelfth are sustainable or widely adopted in projects by organisations (meyerovich & rabkin, 2013). however, there are other factors beyond popular use that influence sustainability of a programming language, including organisational and project boundaries, the programming languages themselves, and above all, social psychology aspects such as developer motivation, preference and interest. flexible coding provides many software development companies and developers the freedom to submit their source codes on github and allow other developers to respond and fork the code. despite a number of published os forking studies that highlight critical factors attributed to successful software forking and forking failure (glass, 2003; fung, aurum & tang, 2012; gamalielesson & lundell, 2013; fujita & ikuine, 2014; jiang, lo, he, xia, singh & zhang, 2016; azarbakht & jensen, 2017), there has been no systematic study mapping understanding of forking motivation, interpretation, categorisation and consequences. this paper therefore presents a systematic review of studies to compare, contrast, summarise and synthesise existing studies to inform future decisions about os forking research by providing an understanding of why some projects are forked more than others, through the lens of project and programming language characteristics. to the best of our knowledge, there are currently few studies that have identified or classified developer forking motivation to enhance forking visibility, and little knowledge about potential differences in forking motivation between junior and senior developers across software engineering, computing science and information systems literature. therefore, clarifications are required. there is no framework to categorise forking motivation behaviour and its effect on forking visibility. a methodological framework would be useful for researchers to implement sustainable ways to motivate developers to fork more programming language files. the objective of this research was therefore to identify types of developer forking motivation and forking consequences cited in the existing os literature through a systematic literature review (slr) adopted from biolchini, j et al. (2005) of conference papers and literature in relevant databases. a slr uses specific search criteria to identify appropriate papers that are then read and analysed carefully using content analysis (a qualitative research technique) from hsieh and shannon (2016) to extract themes and words, in this instance, describing forking. australasian journal of information systems chua & zhang 2020, vol 24, research article open source developers & forking 3 each paper is scrutinised to understand research methodology, methods of data collection, units of analysis and conclusions. the contributions of this paper include: 1) summarising the existing evidence base on forking motivation and consequences into a methodological framework; 2) providing a reference check for those interested in conducting research on understanding developer forking motivation and consequences influencing the ability of projects and organisations to predict project survivability and sustainability [survivability as in the duration of a programming language and sustainability as in measuring a programming language’s continued use by developers]; 3) filling a gap on forking risk literature to inform future research; and 4) proposing a strategy to map how forking motivation and programming language influence forking visibility. we aim to support oss communities and researchers with theoretical insights on developer forking motivation, consequences and impacts. the paper is organised as follows: section 2 discusses the research study motivation and research questions; section 3 describes the slr, content analysis methods and the proposed framework; section 4 presents the findings (forking interpretations and response to the research questions); then section outlines conclusions and possible future research directions. 2 research study motivation and research questions 2.1 research study motivation this study was designed primarily to contribute to a theoretical understanding of os forking and to potentially identify new influencing factors. it is important to address the current disparity in the literature around a theoretical understanding of what forking features and functions can offer in oss, that is, perspectives on interpreting and defining forking as software, project, file repository and programming language source code. there is also a need to understand what influencing factors can cause os project forking to succeed or fail. forking activity has been reported using a variety of measures, including activity growth, developer interest and licensing (dabbish, stuart, tsay & herbsleb, 2012; fung et al., 2012; robles & gonzalez-barahona, 2012; jiang et al., 2016) but there are few analyses measuring forking motivation implicitly or explicitly. moreover, there is limited evidence to confirm forking activeness in spin-off projects that may be strongly influenced by project topic, organisation and license, or developer forking motivation (genuine or non-genuine). further, a myriad of programming languages have tried to spur developer interest but not all succeed or sustain developer forking interest. lastly, there is little evidence on whether genuine developers are more positively motivated to fork compared with non-genuine developers; for example, murgia et al. (2014) noted that developers feel emotions about oss artefacts, such as joy, love, anger, surprise, sadness and fear. 2.2 research questions forking is the creation of a new software repository by copying another repository (jiang et al., 2016). software forking is increasingly adopted by many oss communities for various reasons, including social and political. for instance, a relational database management system project – mysql, owned by sun microsystems – was forked into another project – , called maria db – due to uncertainty whether oracle stewardship could maintain mysql’s survivability (wikipedia, 2001). australasian journal of information systems chua & zhang 2020, vol 24, research article open source developers & forking 4 for new os projects, it is critical to seek developers’ participation and collaboration. interestingly, most junior developers prefer to fork new projects more than old projects, despite less involvement from senior developers, and junior developers seem to prioritise forking in favour of using new programming languages (meyerovich & rabkin, 2013). the number of terminated projects is also increasing due to low sustainable community participation and collaboration to fix bugs and improve features (jiang et al., 2016). it is therefore important to identify types of developer forking motivational behaviour and risk to prevent project termination due to low developer interest. identifying forking motivation may help communities increase sustainability and build more long-term contributors. three research questions (rqs) guided this study. rq1: how do researchers interpret forking and categorise developer forking motivational behaviour? types of developer motivation to fork oss were captured to address rq1, referencing a definition of ‘motivational behaviour’ as a reason or reasons for acting or behaving in a particular way (merriam-webster dictionary, 1999). as the topic is closely related to the study of human behaviour, databases spanning a variety of disciplines – such as humanities and social science, management science, policy, psychology and sociology – were selected to search for oss papers. rq2: what were the most popular methodologies used to research forking from 1990 to 2017? the open source software initiative (osi) started in 1990 with support from many of the world’s largest oss projects and contributors, including debian, drupal association, freebsd foundation, linux foundation, mozilla foundation, wikimedia foundation and wordpress foundation (open source initiative, 1990). the aim was to uphold the osi’s mission and open source definition through the osi affiliate agreement (osi affiliate agreement). while the evolution of forking started in 1990, it is unclear what forking research papers have been published over the past nearly three decades. through rq2 we therefore aim to provide upto-date information on forking throughout the period of os development. rq3: what aspects of os forking have been researched and reported? open source forking is not a new topic but has gained popularity in recent years, with many researchers and communities interested in investigating forking reliability (jiang et al., 2016; fung et al., 2015). when github launched there was an overwhelming response from researchers investigating forking technique performance to analyse forking in sustainable projects by programming language committees or version control files (ernst, easterbrook & mylopoulos, 2010). unfortunately, research findings remain unclear, particularly a lack of data to understand possible impacts and consequences of negative forking. therefore rq3 sought to find barriers to forking to better guide further research. 3 methodology: systematic literature review and content analysis method the slr method was employed to examine and review developers’ motivational forking behaviour in os literature as the topic has been published across multiple disciplines for a number of years. slr was chosen to provide a rigorous and vigorous literature review, as the method can synthesise controversial views and dilemmas when discussing different perspectives on the same topic. slr is one of the most reliable methods for conducting a australasian journal of information systems chua & zhang 2020, vol 24, research article open source developers & forking 5 software engineering literature review and is widely used in computer science, software engineering, social science and information systems research (biolchini, mian, natali & travassos, 2005; okoli & schabram, 2010; salazar, lacerda, nunes & von gresse, 2013). software engineering researchers (kitchenham, 2004; kitchenham & charters, 2007; kitchenham & brereton, 2013) even proclaimed that slr is a form of evidence-based software engineering that can address many engineering questions posed by researchers. here we outline the process for conducting a slr by specifying research questions, describing the search and retrieval process, collecting evidence, synthesising the evidence and providing results. applying slr guidelines provided discrete steps to locate and review appropriate documents describing os forking motivation. as the content of each paper was comprehensive the content analysis method (cam) was then applied to analyse and interpret articles (figure 1), as it is a flexible method for analysing text data, with approaches ranging from impressionistic, intuitive and interpretive to systematic and strict textual analyses (cavanagh, 1997; rosengren, 1981). highly cited content analysis researchers hsieh and shannon (2016) defined three approaches: 1) a conventional analysis where coding categories are derived directly from the text data; 2) a directed approach where user analysis begins with a theory or relevant research findings as guidance for initial codes; and 3) summative content analysis that involves counting and comparing keywords or content followed by interpreting the underlying context. here we adopted a summative content analysis of the slr articles to identify and count common themes and words used to describe forking motivation and sustainability (figure 1). figure 1. combined approaches: systematic literature review and content analysis methods 3.1 systematic literature review search criteria to ensure the literature search was specific and to identify the most relevant, high-quality articles, the inclusion criteria were: 1. peer-reviewed conference or journal papers, published and indexed either in google scholar, acm, ieee, science direct, springer or misq; and 2. written in english; and 3. titles or content included phrases “open source forking motivation”, “open source software forking”, “open source project forking”, “open source social forking”, systematic literature review method search strategy papers retrieved from databases content analysis method word frequency (title, abstract and introduction) australasian journal of information systems chua & zhang 2020, vol 24, research article open source developers & forking 6 “open source code forking”, “open source language forking” or “file repository forking”; and 4. published from 1990 to 2017; and 5. published from top quality information systems conferences or journals; and 6. described the research methodology used – systematic study, stratified sampling, case study, survey, interview, experiment, quasi-experiment or other study types – to collect, analyse and interpret results to address research questions in the paper. this criterion was necessary to determine common and similar research methodologies used by oss researchers to inform the methods and reduce bias of method selection to study forking patterns, frequency, etc. when searching for quality papers, exclusion criteria were articled that: 1. were too short (e.g., less than five pages), general, based on a different perspective or did not include empirical evidence to demonstrate the authors’ claim; or 2. did not identify positive and/or negative impacts or consequences of motivating factors, and did not discuss challenges or barriers, as the objective was to understand developer forking motivation. 3.2 search strategy two approaches were applied to conduct the slr search (figure 2). the first search was conducted on 1 october 2017 on google scholar for the term “open source forking behaviour”, resulting in 21,200 urls. results were then sorted by relevance and filtered for papers published from 1996 to 2017, resulting in 9,530 urls. these papers were both peer-reviewed and non-peer-reviewed, spanning a variety of disciplines, from economics, management and software engineering through sociology (biolchini et al., 2005; okoli & schabram, 2010; salazar et al.; 2013). as each google scholar results page lists 10 urls linking to peer-reviewed articles cited in databases, the first five pages were reviewed by clicking each link to each url, and the summary or abstract and introduction were read to confirm relevancy and suitability. in total, 13 papers were identified in acm, ieee, science direct and or misq databases plus 8 other relevant papers in other databases (table 1). australasian journal of information systems chua & zhang 2020, vol 24, research article open source developers & forking 7 figure 2. the systematic literature review search strategy for research papers database number authors google scholar 8 biazzini & baudry (2014); moen (1999); ernst et al. (2010); ikuine & fujita (2014); fujita & ikuine (2014); fung et al. (2012); gamalielesson & lundell (2013); nyman, mikkonen, lindman & fougère (2012) acm 6 glass (2003); neville-neil (2011); dabbish, et al. (2012); ray & kim (2012); nyman (2014); ray, posnett, filkov & devanbu (2014) ieee 2 chua (2015); cosentino, javier, izquierdo & cabot (2017) misq 1 krogh, haefliger, spaeth & wallin (2012) springer 4 robles & gonzalez-barahona (2012); azarbakht & jensen (2017); jiang et al. (2016); nyman & mikkonen (2011) table 1. the systematic literature review identified 21 relevant and suitable papers 3.3 methodological framework of the 21 papers, five focused on forking sustainability, three on forking challenges and 17 on lessons learnt. forking motivation, sustainability and lessons learnt were synthesised into a methodological framework with three steps to address the research questions via retrieval, categorisation and reporting (table 2). 1) identify variables used to define motivation and its interpretation from both broad and specific perspectives by applying the three rqs via the slr to select and review papers. 2) categorise forking interpretations into three categories (os forking motivation, sustainability and lessons learnt) by applying the cam using the same theme or word. 3) group similar keywords and papers that describe the three categories of australasian journal of information systems chua & zhang 2020, vol 24, research article open source developers & forking 8 forking motivation, sustainability and lessons learnt. conclusions were then drawn from these findings regarding forking challenges and lessons to be learnt. purpose process outcome 1. identify variables that describe forking motivation and its interpretation apply slr to select relevant papers from selective databases retrieve relevant papers on forking motivation 2. categorise forking into motivation, sustainability and lessons learnt apply cam and classify common themes or words categorise forking motivation into three classes 3. group similar keywords to describe os developer forking motivation, sustainability and forking lessons learnt analyse word count frequency (title , abstract and introduction) report forking motivation factors table 2. a forking motivation methodological framework 3.4 content analysis method each of the 21 papers identified was scrutinised for context using content analysis. papers were first scanned to confirm the word ‘fork*’ was mentioned and the research evidence was empirical, then themes and key words were extracted. next, each title was checked, abstract read, and adjectives that described ‘fork*’ quantified (table 3). for example, when reviewing the papers “code forking in open-source software: a requirements perspective” (ernst et al., 2010) and “perspective on code forking and sustainability in open source software” (nyman et al., 2012) the word ‘code’ occurred twice so ‘2’ was entered under ‘code’ forking type identified by the google scholar search in table 2. occurrences of forking motivation (n=10), forking sustainability (n=4), consequences (n=2), impacts (n=2) and threats (n=1) were also noted. paper content was then analysed, noting research method, unit of analysis and results, then the introduction and conclusion were reviewed in more detail. forking type paper identified via total acm ieee springer misq google scholar open source 1 1 project 4 1 1 1 7 software 1 2 3 social 2 1 2 5 code 1 2 3 language 1 1 file repository 1 1 total 6 2 4 1 8 21 table 3 forking interpretation types next, papers were grouped into four categories to address rq1: 1. developer forking interpretations: 7 interpretations of forking (table 2). 2. developer motivation and reasons: a subset of papers reported similar variables (table 3). for instance, krogh et al. (2012), fung et al. (2012), glass (2013) and jiang et al. (2016) reported divergent specialisation, objective misalignment, poor governance and leadership and culture. australasian journal of information systems chua & zhang 2020, vol 24, research article open source developers & forking 9 3. forking sustainability: four groups of researchers (ernst et al., 2010; nyman et al., 2012; gamalielesson et al 2013; jiang et al., 2016) undertook real-world projects, comparing original versus forked projects (table 3). successful and sustainable projects included community-level projects, such as mariadb forked by mysql, the software level of ms word and libreoffice and ecosystem levels of libreoffice forked from openoffice. 4. forking lessons learnt on project compatibility issues: 19 papers cited forking lessons and seven described more than one type of forking reason, including no guidance or direction, copyright, licensing conflict, project ownership or dividing the forking community (moen, 1999; glass, 2003; neville-neil, 2011; ikuine & fujita, 2014; fujita & ikuine, 2014; cosentino et al., 2017; azarbakht & jensen, 2017). neville (2011) pointed out that technical developers’ roles are becoming specialised. 4 forking motivation interpretations although a number of motivating factors identified in previous os studies are applicable in the forking context, a number of diverse forking motivation factors were detected in this literature review, including project revival and alignment, culture traits, divergent specialisation, individual ownership, license and software compliance, community disintegration, community practice and extending community social coding development. therefore prior to investigating forking motivation factors, an additional research question was posed. 4.1 how do researchers interpret developer forking and categorise forking motivational behaviour? these findings reveal a diversity of forking interpretations (table 3), with project forking most common (7 papers), and os, programming language and file repository the least (1 each). however fork type was interpreted differently by different researchers, due to the metadata of the dataset they downloaded from the hosting server. for example, github was the only hosting server to categorise file repository forking. to further understand the forking interpretation each paper, the categories were defined in more detail (paper classifications shown in table 4). 4.1.1 open source forking the early 1990s saw a proliferation of research on os motivation. krogh and colleagues (2012) reviewed seven years of publications and identified 40 papers that focused on os developer motivation, including hars (2002), stewart & gosain (2006), hertel et al. (2003), lerner & tirole (2002) and shah (2006). they synthesised findings across these papers into three classes of motivation: intrinsic, internalised intrinsic and extrinsic. intrinsic motivation included ideology, altruism, kinship and fun, and can drive developers to fork software. internalised intrinsic motivation included reputation, reciprocity, learning and own-use. extrinsic motivation may include being paid for the work or finding a career in coding. hippel & krogh (2003) and goode (2005; 2014) studied organisational information sharing in adopters and non-adopters of oss and innovation models as influencing factors on motivation. they found more reputable organisations and innovative projects are more likely to attract oss developer attention to download or copy repository files. australasian journal of information systems chua & zhang 2020, vol 24, research article open source developers & forking 10 4.1.2 project forking nyman and mikkonen (2011) defined that a project fork takes place when software developers copy source code from one software package and use it to begin an independent development work. in general, forking results in an independent version of the system that is maintained separately from its origin. nyman and mikkonen (2011) looked at forking behaviour in the context of forked project survivability, quantifying project forking as the number of original projects forked by developers and comparing the number of original projects versus forked projects in github. many researchers seek to understand how forking impacts an original forked project and nyman and mikkonen provided real-life examples of current high profile os projects that either started from a fork or were common targets for forking. 4.1.3 software forking ikuine and fujita (2014) referred to software forking as the continuous development of software, by the original developer or others. when other developers take over, the original developer must share the source code. software forking focuses on the product itself, such as microsoft software, facebook software and email applications. 4.1.4 social forking fung, aurum and tang (2012) defined social forking in their study of nine javascript development communities in github, with the highest amount of forks to identify the relationships within them and study how forks are used to facilitate oss development. in their analysis, almost 7,000 developers made approximately 8,000 forks in different communities, with the most active developers making contributions to multiple communities. their research indicated that forks are actively used by the development community to fix defects and to experiment with new features. what separates these forks from normal branching is that the changes do not necessarily need to be promoted to the original project upstream and can live in a separate fork that can still take any changes and improvements from the original project as updates. what separates a fork from a branch even more is that a fork can originate from either a subset of the forked predecessor’s artefacts or from multiple predecessors’ artefacts. a branch in turn is a copy of all the predecessor’s artefacts (fung et al., 2012). 4.1.5 code forking code forking is defined as a forked project copied from existing code base and moved in a direction different from the project leadership. forking the code base allows developers to leverage existing functionality while also addressing new requirements. although flexible, forking has inherent difficulties, such as maintenance, evolution, and social factors concerning the development community. a broad definition of a code fork is when the code from an existing program serves as a fork (nyman et al., 2014); it is the basis for a new version of the program, more specifically, a version that seeks to continue to exist apart from the original. 4.1.6 programming language forking chua (2015) examined language forking from the perspective of programming language adoption by project owners, finding three projects where apache, mozilla and ubuntu javascript languages were actively forked by developers. chua and zhang (2019) then proposed three forking pattern types (‘once-only’, intermittent or steady) and potential reasons behind short-lived programming languages. australasian journal of information systems chua & zhang 2020, vol 24, research article open source developers & forking 11 4.1.7 file repository forking a file repository fork is mainly used to make contributions to original repositories and is beneficial for the oss community (jiang et al., 2016). actions such as submitting pull requests, fixing bugs, adding new features and keeping copies are motivations for developers to fork repositories. a repository written in a developer’s preferred programming language is more likely to be forked and developers mostly fork repositories from creators. attractive repository owners include organisations, as they have more followers. type interpretation studies citing authors within paper set forking motivation coding for revising requirements requirement change ernst et al. (2010) ernst et al. (2010) cited by fung et al. (2012); jiang et al. (2016) seeking a coding job recruitment of contributors biazzini & baudry (2014) nil licensing compliance licensing compliance biazzini & baudry (2014); dabbish et al. (2012); jiang, et al. (2016) nil software compliance software interoperability krogh et al. (2012); meyerovich, & rabkin (2013); nyman (2014); tegawendé, bissyandé, thung, lo, jiang & réveillère (2013) nil reviving original project development duration cessation of original project nyman (2014); robles & gonzalezbarahona (2012); ray & kim (2012); tegawendé et al. (2013); chua (2015) nyman (2014) cited by jiang et al. (2016) extending community social coding development more community driven development dabbish et al. (2012) ray et al. (2014) cited by jiang et al. (2016) ownership implication legal implication on ownership and conflict over brand ownership fung, aurum & tang et al. (2012); nyman (2014); nyman & mikkonen (2011); ray & kim (2012) business strategy risk commercial strategy forks dabbish et al. (2012) team coding skill inequality differences among developer team nyman (2014) community socialisation building new community through social interaction, sharing and collaboration dabbish et al. (2012); fung et al. (2012); robles, & gonzalez-barahona (2012) coding by socialising social network coding jiang et al. (2016); fung et al. (2012) divergent specialisation new specialisation, divergent technical views nyman (2014); nyman & mikkonen (2011); ray & kim (2012) nil objective misalignment different technical objectives poor leadership poor project governance nyman (2014); nyman & mikkonen (2011); robles & gonzalez-barahona (2012) culture trait cultural differences australasian journal of information systems chua & zhang 2020, vol 24, research article open source developers & forking 12 type interpretation studies citing authors within paper set software activity project specialty to generate commits ray & kim (2012); tegawendé et al. (2013) ecosystem system between system sharing resources and infrastructure forking sustainability community activity communities retention ernst, et al. (2010); gamalielesson & lundell (2013). jiang et al. (2016); nyman et al. (2012); azarbakht & jensen (2017); cosentino et al. (2017) ray et al. (2014) cited by jiang et al. (2016); gamalielesson & lundell (2013) forking lessons learnt no formal process no guidance/ direction ikuine & fujita (2014); fujita & ikuine (2014); azarbakht & jensen (2017) nil legal implication copyright glass (2003); azarbakht & jensen (2017) licensing conflict moen (1999); azarbakht & jensen (2017) transfership project ownership ikuine & fujita (2014); fujita & ikuine (2014); cosentino et al. (2017); azarbakht & jensen (2017) product expertise shortage technical developers become product expert neville-neil (2011) upgrade of developer role to product role role movement glass (2003); ikuine & fujita (2014); cosentino et al. (2017) glass (2003) cited by fung et al. (2012) community divide divide community fork azarbakht & jensen (2017); cosentino et al. (2017) nil table 4. fork categorisation, sustainability and lessons learnt 4.2 what were the most popular methodologies used by forking researchers from 1990 to 2017? figure 4 presents data relating to methodologies across the 21 papers after they were carefully reviewed for study type, research methodology and data collection methods and type. thirteen of the 21 papers were qualitative with data collection methods including stratified sampling (n=8), systematic study (n=5), qualitative interview (n=2), qualitative case study (n=2), survey and interview (n=1), stratified sampling and survey (n=2) and qualitative interview and survey (n=1). australasian journal of information systems chua & zhang 2020, vol 24, research article open source developers & forking 13 figure 3. data collection methods in the 21 papers 4.3 what aspects of os forking have been researched and reported? figure 4 shows the units of analysis used in the 21 papers. in seven papers this was a comparison between non-forking and forking projects. of the remaining 14 papers, six papers focused on the forking relationship on software releases, version control files and file repository and eight focused on os project interactions with components, such as popular programming languages, the product and the successful system, and analysing forking behaviour between the manager, developer, and end user (github versus non-github). figure 4. units of analysis in the 21 papers 0 2 4 6 8 10 stratified sampling systematic study qualitative interview qualitative case study stratified sampling and survey survey and interview qualitative interview and survey single method mixed method 0 2 4 6 8 project software releases file respository version control product system programming language manager developer github end users non-github end user australasian journal of information systems chua & zhang 2020, vol 24, research article open source developers & forking 14 figure 5 shows eight types of forking lessons learnt on project compatibility issues that were identified in the 21 papers. in order of decreasing frequency of reporting, these were: no project ownership (n=4), no project guidance and the developer role becoming specialised (n=3); copyright, licensing and the software less likely to become proprietary, and a split community (all n=2 each). there was also one paper on losing developers as technical developers become product experts. figure 5. forking lessons learnt across the 21 papers 5 conclusions and future work forking is one of the most critical technique in os research today. our analysis of 21 papers can help the os community – educators, academicians, developers, project investors – to improve awareness of forking as a sustainable way to revive project health. the categories of forking lessons learnt highlight that forking consequences are likely to continue and remain a survival challenge to oss developers. for example, if forking life span becomes short-lived developers could close the project or terminate the file repository. to the best of our knowledge, there is no research discussing how a lack of sustainable programming languages could reduce forking sustainability and viability. programming language attractiveness drives and motivates developer desire to fork, helping to maintain forking health and activity. the usefulness of a programming language is the likelihood a fork can be generated effectively by developers. we strongly believe it is important to investigate how competitive programming languages can impact forking sustainability and to seek ways to prevent low forking performance, if necessary. this paper provides a quick reference for oss researchers to understand categories of developer forking motivation, introduce guidelines for oss communities on ways to reduce organisational barriers to developer motivation, and, most importantly, highlight that new or existing project sponsors should focus on understanding developer forking motivation, to positively influence achieving a healthy source code. this study also identifies some challenging areas for future work. 0 2 4 6 project ownership no guidance role becomes very specalised copyright licensing never make the software become proprietary community divide technical developers become a product expert australasian journal of information systems chua & zhang 2020, vol 24, research article open source developers & forking 15 1. append new findings into the body of knowledge on os forking behaviour. applying the combined approaches of slr and cam revealed seven forking types interpreted by academic researchers and the latest interpretation found is file language repository fork. this novel insight will assist researchers on how forking is presented and interpreted and industry practitioners in reviewing project forking health, especially projects with programing language file repositories that are less adopted or forked by developers. 2. understanding forking consequences. case studies are an important way to highlight lessons learnt by researchers. this paper identified forking impacts and consequences, with one of the worst impacts being a political strategy that divides a project community and forms a new community. forming a new community results in less contributions by developers to the original file repository, bug fixes or feature enhancement. allowing accumulated bugs and feature enhancements to remain unfixed for a period of time can affect project health risk. 3. more research is required on forking sustainability. reviewing these 21 papers revealed the importance of forking sustainability investigation as a top priority with two specific areas of interest. a. analysing forking from a social community perspective. for instance, azarbakht & jensen (2017) adopted a developer-oriented statistical approach to determine what causes people in complex software development networks to decide to fork (break away), and what changes a community goes through when deciding to divide different or conflicting goals, communication styles, or values can positively or negatively influence community interactions. b. understanding the relationship between programming languages, repositories and developer forking interest to more accurately predict oss forking motivation and behaviour. 4. studying forking sustainability using a slr for software development with github. valentio, javier, izquierdo and cabot (2017) used a slr to show that forking is a good indicator of project longevity and the chance of forking is highly dependent on the project, where developers provide additional contact information (e.g., emails, personal website urls that are clearly active or aligned with popular project owners) to increase social connections between a project owner and forker, and increase developer community size for medium-size projects and projects that are written in a forker’s preferred programming language. future work could include developing a prediction model for fork effectiveness from forking motivation classifications in response to language repository files, where programming language survival time is critical to an os projects’ health and survivability. australasian journal of information systems chua & zhang 2020, vol 24, research article open source developers & forking 16 references alexa, (2017). https://www.alexa.com/siteinfo. (accessed on 1 october 2017) azarbakht, a.e., & jensen, c. (2017). longitudinal analysis of the run-up to a decision to breakup (fork) in a community. proceedings of the ifip international conference on open source systems. oss 2017, springer. biazzini, m., & baudry, b. (2014). “may the fork be with you”: novel metrics to analyze collaboration on github. proceedings of the 5th international workshop on emerging trends in software metrics, june 2014, hyderabad, india. biolchini, j., mian, p. g., natali, a. c., & travassos, g. h. (2005). systematic review in software engineering. technical report rt-es 679/05, coppe/ufrj, rio de janeiro, brazil. cavanagh, s. (1997). content analysis: concepts, methods and applications. nurse researcher 4(3), 5–16. chua, b. (2015). detecting sustainable programming languages through forking on open source projects for survivability. proceedings of the ieee international symposium on software reliability engineering (issre) 2015 in conjunction with a wosar workshop, ieee, gaithersburg, usa. 120–124. chua, b. (2017). a survey paper on open source forking motivation reasons and challenges. proceedings of the pacific asia conference of information systems (pacis), july malaysia, langakawi chua, b and zhang y. (2019). predicting open source repository by programming language survivability from forking data. the 15th international symposium on open collaboration (opensym), august 20–22, 2019, skövde, sweden. association for computing machinery (acm). dabbish, l., stuart, c., tsay, j., & herbsleb, j. (2012). social coding in github: transparency and collaboration in an open software repository. proceedings of the acm 2012 conference on computer supported cooperative work ernst, n. a., easterbrook, m. a., & mylopoulos, j. (2010). code forking in open-source software: a requirements perspective. corr, abs/1004.2889. fujita, h., & ikuine, f. (2014). open source, a phenomenon of generation changes in software development: the case of denshin 8 go. annals of business administrative science, 13(1), 1–15. fung, k. h., aurum, a., & tang, d. (2012). social forking in open source software: an empirical study. caise forum, 50–57. gamalielesson, j., & lundell, b. (2013). sustainability of open source software communities beyond a fork: how and why has the libreoffice project evolved? journal of systems & software, 89, 128–145. glass, r. l. (2003). a sociopolitical look at open source. communications of the acm 46(11), 21– 23. goode, s. (2005). something for nothing: management rejection of open source software in australia’s top firms. information & management, 42(5), 669–681. australasian journal of information systems chua & zhang 2020, vol 24, research article open source developers & forking 17 goode, s. (2014). exploring organizational information sharing in adopters and non-adopters of open source software: evidence from six case studies. knowledge and process management, 21(1), 78–89. gousios, georgios; vasilescu, bogdan; serebrenik, alexander; zaidman, andy. (2014) lean ghtorrent: github data on demand. the netherlands: delft university of technology & eindhoven university of technology. hars, s. o. (2002). working for free? motivations for participating in open-source projects. international journal of electronic commerce, 6(3), 25–39. hippel, e. v., & krogh, g. v. (2003). open source software and the “private-collective” innovation model: issues for organization science. organizational science, 14(2), 209–223 hertel, g., niedner, s., & herrmann, s. (2003). motivation of software developers in open source projects: an internet-based survey of contributors to the linux kernel. research policy, 32(7), 1159–1177. hsiu-fang h., & shannon, s. e. (2016). three approaches to qualitative content analysis. journal of qualitative health research, 15(9), 1277–1288. ikuine, f., & fujita, h. (2014). how to avoid fork: the guardians of denshin 8 go, japan. annals of business administrative science, 13, 283–298. jiang, j., lo, d., he, j. j., xia, x, singh p. k., & zhang, l. (2016). why and how developers fork what from whom in github. journal of empirical software engineering, 100(21), 1–32. kitchenham, b. (2004). procedures for performing systematic reviews, technical report. tr/se0401, department of computer science, keele university, keele, uk. kitchenham, b., & brereton, p. (2013). a systematic review of systematic review process research in software engineering. information & software technology, 55(12), 2049–2075. kitchenham, b., & charters, s. (2007). guidelines for performing systematic literature reviews in software engineering, joint report ebse 2007-001, keele university and durham university. krogh, v. g., haefliger, s., spaeth, s., & wallin, m. w. (2012). carrots and rainbows: motivation and social practice in open source software development. mis quarterly, 36(2), 649–676. lerner, j., & tirole, j. (2002). some simple economics of open source. journal of industrial economics, 50(2), 197–234. mariadb. in wikipedia. https://en.wikipedia.org/wiki/mariadb. retrieved 17 february 2017. meyerovich, l. a., & rabkin, a.s. (2013). empirical analysis of programming language adoption. proceedings of the 2013 acm sigplan international conference on object oriented programming systems languages & applications, october 29–31, 2013. indianapolis, indiana, usa. moen, r. (1999). fear of forking. http://linuxmafia.com/faq/licensing_and_law/forking. html. retrieved 22 november 2016. australasian journal of information systems chua & zhang 2020, vol 24, research article open source developers & forking 18 murgia, a., tourani p., adams, b., & ortu, m. (2014). do developers feel emotions? an exploratory analysis of emotions in software artifacts. proceedings of the 11th working conference on mining software repositories, 262271. neville-neil, g. v. (2011). kode vicious: think before you fork. communication of the acm (54), 34–35. doi: 10.1145/ 1953122.1953137 nyman, l. (2014). hackers on forking. proceedings of the international symposium on open collaboration, acm, new york, ny, usa, isbn: 978-1-4503 30169. 4–12. nyman, l., & mikkonen, t. (2011). to fork or not to fork: fork motivations in sourceforge projects. international journal of open source software & processes, 3(3), 1–9. nyman, l., mikkonen, t., lindman, j., & fougère, m. (2012). perspective on code forking and sustainability in open source software. proceedings of the ifip international conference on open source systems, open source systems: long-term sustainability, 274–279. okoli, c., & schabram, k. (2010). a guide to conducting a systematic literature review of information systems research. available at ssrn: dx.doi.org/10.2139/ssrn.1954824 open source initiative. (1990). https://opensource.org/node/755. retrieved 25 november 2016. open source initiative affiliate agreement. (1998) www.opensource.org. retrieved 25 november 2016. ray, b., & kim, m. (2012). a case study of cross-system porting in forked project. proceedings of the 20th acm sigsoft international symposium on the foundation of software engineering. november 2012. ray, b., posnett, d., filkov, v., & devanbu, p. (2014). a large scale study of programming languages and code quality in github. proceedings of the 22nd acm sigsoft international symposium on foundations of software engineering, november 16–21 2014. hong kong, china. ray, b., wiley, c., & kim, m. (2012). repertoire: a cross-system porting analysis tool for forked software project. proceedings of the 20th acm sigsoft international symposium on the foundation of software engineering, november 2012. robles, g., & gonzalez-barahona, m. (2012). a comprehensive study of software forks: dates, reasons and outcomes. open source systems: long-term sustainability. ifip advances in information & communication technology, 378(1), 1–14. rosengren, k. e. (1981). advances in scandinavia content analysis: an introduction. in advances in content analysis, k. e. rosengren (ed.), beverly hills, ca: sage, 9–19. salazar, l. h. a., lacerda, t., nunes, j. v., & von gresse, c. (2013). systematic literature review on usability heuristics for mobile phones. international journal of mobile human computer interaction, 5(2), 12–22. shah, s. k. (2006). motivation, governance, and the viability of hybrid forms in open source software development. management science, 52(7), 1000–1014. stewart, k. j., & gosain, s. (2006). the impact of ideology on effectiveness in open source software development teams. mis quarterly, 30(2), 291-314 australasian journal of information systems chua & zhang 2020, vol 24, research article open source developers & forking 19 tegawendé, f., bissyandé, t., thung, f., lo, d., jiang, l.x., & réveillère, l. (2013). popularity, interoperability, and impact of programming languages in 100,000 open source projects. computer software and applications conference (compsac), 2013 ieee 37th annual conference 23-26 july, kyoto, japan valentio. c, javier, l. izquierdo, c. cabot, j. (2017). a systematic mapping study of software development with github. access ieee, 5(1), 7173–7192. copyright: © 2020 chua & zhang. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v24i0.1714 http://creativecommons.org/licenses/by-nc/3.0/au/ ajis vol 4 no. 1 focusing information systems post-graduate research projects gail ridley information systems group department of computer science university of tasmania gpo box 252c hobart tasmania 7001 email: gail.ridlcy@cs.utas.edu.au abstract this paper reports on an investigation of mechanisms that assist information systems post-graduate research students to focus their projects. an evaluation is presented of the experiences of information systems research students in focussing their research projects based on a survey conducted of students who participated in two of the first three information systems doctoral consortia to be held in australia. the survey sought to determine whether a doctoral consortium or 'systematic expert review' is the most valuable mechanism for focussing a research proposal. systematic expert review was considered by the students to be more effective than the doctoral consortium process for the purpose of focussing their research project proposals. introduction recently criticism has been levelled at some practices and achievements of research post-graduate education and supervision in australia. against this background information systems (is) and other responsive research communities have recently adopted and implemented the post-doctoral consortium process. it is unclear how much evaluation of the doctoral consortium mechanism took place before it was instituted. having participated in doctoral consortia and systematic expert review, the author was motivated to determine how common it was for post-graduate students to take part in the latter, and which of the two strategies was considered by postgraduate students to be the most effective for focussing a research project. another motivation for the investigation was to learn of other methods that the post-graduate students had used to focus their projects. it is assumed that one of the most difficult phases of a research higher degree is the period taken to focus on a research question. as information systems is a relatively recent and still developing discipline where traditions and theory are being established (farhoomand 1992; miller 1993), this must be particularly so. the appropriate methodologies and content of is are both under debate. shanks et al. (1993) report that some is researchers argue that interpretivism is the preferred framework. keen (1991) observes however, that the positivist perspective dominates is journals. methodological pluralism is advocated by galliers (1992). such diverse views suggest that knowledgeable review of a student proposal is warranted. because of is's interdisciplinary nature, there are few clearly defined boundaries (shanks et al. 1993). such a characteristic may make it difficult for novice researchers to conduct literature searches and find someone with sufficient background in the knowledge domain to review a research proposal. however, as is is still an emerging discipline (miller, 1993), opportunities for contact with active is researchers may be limited. even a cursory examination of the 7996 asia pacific directory of information systems researchers reveals a wide range in the faculty and departmental placement of is researchers in australia. australian researchers who identify with the is field work within departments or schools as disparate as: school of environmental and information science, department of computer science and computer engineering, graduate school of management, department of information management and marketing and department of information systems. where university teaching and research in is are conducted within a larger and more dominant field with different paradigms such as computer science or management, and information systems researchers are a small minority, greater difficulty in observing is researchers at work may be anticipated. such characteristics of is may make it difficult for a research student to be able to select a focus and plan a project with confidence, particularly in geographically isolated areas or where a proposed project is in a less dominant knowledge domain. australian post-graduate students commencing research frequently spend lengthy periods trying to determine a topic and methodology. however, in american phd programmes, the emphasis on graduate coursework before starting a research project allows students 'to develop research proposals systematically...' (higher education council, 1990). due to differences in the structure of australian phd programs, it is argued in this paper that australian post-graduate research students have special requirements in order to determine the scope and approach of their project when compared to their american counterparts. these special requirements may be met by strategies such as those proposed in this paper. 80 ajis vol 4 no. 1 cullen et al. (1994) have identified the need to formulate a feasible topic as being significant to the successful completion of a phd project. assistance from experienced academics at the time of deciding on a research topic and writing a proposal may be critical to the ultimate success of the project. locke and spirduso (1976) firmly discount the idea that it is 'vaguely immoral to seek assistance in preparing a proposal'. this paper assumes that such help is pivotal to the outcome of the research project and is primarily concerned with just one aspect of supervision, the provision of opportunities to help the student select and focus a research proposal. background approaches to supervision the report, establishing effective phd supervision, written for the (australian) department of employment, education and training (deet) by cullen et al. 1994, states that the american phd system makes greater use of a panel of supervisors and thesis advisers than does the australian system. in australia, as in some other nations, a single supervisor (or sometimes a supervisor and associate supervisor) is more common. however, the same authors state that 'no one supervisor can be expected to provide all that is needed all the time to different students.' this points to an inadequacy in the traditional means of supervising australian postgraduate research projects which may translate into a difficulty for the student in formulating a project. one possible solution to achieve effective supervision is proposed in the same report. it is recommended that supervisory panels be established to give students access to a wider range of expertise and skills. cullen et al. also state in the same report that '...there is a need to [...] restructure practice to ensure that responsibility for quality is co-ordinated'. it seems likely then, that where a student's proposal is exposed to other researchers for feedback, its quality will be greater. cullen et al. recommend that a study be made of other strategies to restructure graduate education. it is proposed that the mechanisms examined in this paper will assist research students to determine an appropriate topic. as discussed earlier, the process of determining an appropriate topic is related to the successful completion of a phd project. the analysis and findings reported in this paper contribute to satisfying the information need identified by cullen et al. student contact with other researchers cullen et al. (1994) state that australian graduate programs generally have less students than those in the united states of america. it is likely, then, that there is less appropriate expertise to draw upon within australian university departments to assist the student, and as a consequence, it could be argued that there is a need for greater contact with academics from relevant departments elsewhere. recognition was given to the need for greater contact between universities when the higher education council in australia recommended in 1990 that higher education institutions examine student research proposals and 'develop formal and informal links with other institutions' for mutual benefit. however, studies by powles (1988) and cullen et al. (1994) indicate that a significant to majority proportion of post-graduate students have little contact with staff and students at other institutions in order to obtain feedback on their research. the findings of a survey of students and supervisors conducted by cullen et al. (1994) give 'sources of critical assistance' used by phd students. only 22% obtained critical assistance from academics in the same department, and just 8% obtained the same kind of help from academics elsewhere. in another australian study, powles (1988) found that 55% of students had received no advice from staff in other departments and 36% had no advice from academic staff other than their supervisor, resulting in intellectual isolation for many. powles also found that 31% of students enrolled for four years or more had not attended one local, state or national meeting or conference, a startling indication of limited contact with research students and academics from outside their own department. these statistics give credence to cullen et al.'s (1994) plea to '...go beyond individual supervisory interactions' for post-graduate students. mechanisms to assist supervision in the deet report, supervising higher degree research students, parry and hayden (1994) state that consensual approaches to determine that a post-graduate thesis is acceptable are much more useful than 'attempting to define in advance the acceptable characteristics of theses, against which an individual student's work could be judged.' such 'consensual approaches' include presenting work at conferences and seminars, and publishing to give both supervisors and students confidence that acceptable progress is being made. it can 81 ajis vol 4 no. 1 be seen, then, that obtaining feedback on a student's work from outside the student's department can be proposed on the grounds of increased efficiency in addition to the advantages regarding quality. povvles (1988) found that '47% of phd students had not been required to submit a written progress report and 48% had not had to give an oral progress report of a formal type during their candidature.' it is likely that such written and oral progress reports would be reviewed by other academics in addition to the student's supervisor, broadening the expertise and skills base of those reviewing the student's project. however, despite the belief by 86% of the students surveyed in the powles study that 'the oral reports [were] "very useful" or "somewhat useful" ', almost half of the students surveyed had not had this opportunity. doctoral consortia a recent phenomenon to australia (conrad, 1991), designed to assist in the definition of post-graduate research projects, is the doctoral consortium. both doctoral and research masters students have participated in doctoral consortia. due to its recent introduction, there is little literature available. doctoral consortia are better established outside australia, particularly in europe and the united states, where their function may be different. in australian doctoral consortia, conference organisers group students before matching with one or more expert researchers in the area to review the research proposals. both specific responses to the individual student and general advice to the group are usually offered. the doctoral consortium was first launched in australia with marketing phd students at the inaugural australia and new zealand doctoral consortium in march 1991 (conrad, 1991). since then they have been organised by the accounting association of australia and new zealand conference (aaanz) and the department of accountancy at the university of wollongong and in the area of management accounting at the university of new south wales (leech, s. 1994, pers. comm., december). the process has been applied more recently to the is field in australia, although consortia have been held in conjunction with the most recent international conferences in information systems. the second and third australian is doctoral consortia were held in conjunction with two major conferences in 1994, at which the author was present. other mechanisms parry and hayden (1994) recommend that departments 'provide mechanisms whereby students are actively encouraged to participate in intellectual exchange with other scholars'. they review strategies to expose students' work to other scholars, including the presentation of work in progress at departmental seminars to provide students with feedback from students and academic staff of the department. it is clear from the literature that there is a need for mechanisms that will broaden the provision of feedback on the students' work. systematic expert review of post-graduate proposals may be another method of achieving the same objective. such reviews involve a deliberate strategy of obtaining detailed and specific feedback in a sequential fashion from researchers familiar with the methodology or knowledge domain of the research proposal. such researchers may work within or outside of the field of is, and may be found among academics other than the student's supervisor/s within the same department, elsewhere in the same university, or at universities within and outside the country. study methodology the study objectives were to determine the value of the doctoral consortium from the post-graduate students' perspective in helping focus their research proposals. in addition, the investigation sought to learn whether the research proposals were reviewed by anyone other than the supervisor/s and the consortium facilitators, and the value of the experience for focussing the project. a final objective was to compare the value of doctoral consortium participation with other forms of non-supervisor review. a largely quantitative approach was used for the study, but one that has been supported by qualitative data in order to validate the findings and to enrich the data in a way that would not have been possible had a strictly positivist epistemology been adopted (jick, 1983). while quantitative analysis was employed, 'enumerative induction' has been used rather than the 'the more elaborate form of statistical induction in which probability theory is involved' (mitchell 1983). in enumerative induction 'the characteristics of a class of phenomena are established ... by listing them' (mitchell 1983). as the nature of the study was exploratory, it was considered that a combination of descriptive statistical analysis and qualitative investigation would be better suited to an initial investigation than use of inferential statistics. sophisticated statistical analysis would have been inappropriate considering the objectives of the study and the nature of the data sought. 82 ajis vol 4 no. 1 a survey was the obvious method for contacting participants to obtain their perspective. use of personal or telephone interviews would have been expensive and difficult as the research students were dispersed across australia and new zealand. because of the nature of their discipline, all the subjects were likely to be familiar with, and have access to a computer and internet facilities. as a result, and because it was immediate and inexpensive, an emailed survey was chosen. the emphasis of this paper is to examine strategies used to focus a research project. for this reason, and as research masters students and phd students were not segregated at the doctoral consortia and the numbers in both groups were not large enough to be independently representative, the responses from the two groups were not differentiated. the two conferences with their affiliated doctoral consortia were chosen because they were the second and third is doctoral consortia to have been held in australia at the time that this paper was written. it was decided not to survey participants of the first is doctoral consortia to allow for the refinement of the first iteration of the process. not all respondents had participated in expert review and where they had, the process had not been uniform. for this reason it was decided that the respondents would be asked to detail the advantages and disadvantages of the doctoral consortium, but not for systematic expert review. the study subjects were 39 participants of the doctoral consortium that was held in association with the 5th australasian conference on information systems (acis'94) as well as 18 participants of the doctoral consortium held with the international federation for information processing technical committee 8 open conference on business process re-engineering, 1994 information systems international working conference (ifip tc8aus) at the gold coast, australia in may 1994. the majority of the post-graduate attendees were enrolled at universities in australia, and six participants travelled from new zealand. after initial piloting, an emailed questionnaire was developed that posed four questions. the first question asked the students to list the advantages and disadvantages of the doctoral consortium process for them personally. the second question used a five point likert scale graduated from strongly agree to strongly disagree for the student to indicate the value of the doctoral consortium in helping to focus their research. the third question asked whether the student's research proposal had been reviewed by anyone other than their supervisor or consortium facilitators, and requested comments on the value of the experience for focussing the project. the last question required the respondent to state and compare the value of any form of review of their research proposal (excluding supervisor review) against participation in the doctoral consortium. the questionnaire asked for some data that was designed to be analysed quantitatively, using 'enumerative' rather than 'statistical' deduction (mitchell, 1983). other open-ended questions that were designed to broaden understanding of the topic were subjected to content analysis. these responses were listed and clustered into groups to permit analysis. in this way, phrases were extracted that gave support to the quantitative data or enriched an understanding of the respondents. some of this material has been included in the results section that follows. results of the 57 doctoral consortium participants surveyed, 37 responses were received, a return rate of 65%. this figure includes two students who participated in both doctoral consortia. as the same questions were used for each conference, these two students were surveyed only once. if the two students had been asked to complete two identical questionnaires, it is likely that their responses to the second questionnaire would have been influenced by the experience of completing the same questionnaire earlier. consequently, the responses from these two students were counted twice as the students were part of both conference groups. as over 30 questionnaires were returned by participants at the two doctoral consortia and no generalisations were made from the two subgroups, the responses were taken as being representative of the range of participants at the two consortia. this approach is consistent with that recommended by roscoe (1975). other mechanisms used for focussing a research proposal many of the post-graduate students recommended mechanisms other than doctoral consortium participation that they had used to help focus their research proposal. these mechanisms are summarised in table 1. by far the most popular of these mechanisms is expert review (ie to have academics other than their supervisor review their proposal). doctoral consortia are not included in the table as all the students surveyed had participated at least once. the next most popular mechanism was to present papers and seminars. three students stated that discussion of the research project with staff was helpful. such staff may be located in the student's department of enrolment, 83 aj1s vol 4 no. 1 or with post-graduates who were also academics elsewhere, at the student's workplace. two students wrote of the value of submitting papers to conferences, while the same number suggested that discussion with other students working in similar areas was of use. one student identified the value of the 'panel meeting' approach, also referred to by another student as a trial consortium. in this arrangement a panel consisting of the supervisor, external supervisor (if relevant) and another person external to the student's department listened, reviewed and criticised the student's research, focussing on monitoring and support. mechanisms used expert review presenting seminars, papers discussion with staff submitting papers to conferences discussion with other students trial doctoral consortium (ie panel meeting) applying for a faculty grant in similar area preparing submissions for ethics committee talking over research with friends number of responses 16 5 3 2 2 2 1 1 1 table 1 mechanisms used for focussing research proposal as suggested by research students. more than one response was possible. not all students suggested other mechanisms to help focus their project. three students suggested one of each of the following: where a student was employed as an academic, applying for a faculty grant in an area related to her post-graduate research and getting feedback on the application, preparing a submission for the university's ethics committee and talking over the research with friends. strengths of doctoral consortia many advantages of the doctoral consortium process were identified by the participants. the most popular of these are summarised in table 2. main strengths of doctoral consortia social and academic interaction independent appraisal by peers & researchers focus on method exposure to other research projects support for common problems number of responses 19 17 8 8 6 table 2 main strengths of doctoral consortia as identified by research students surveyed. more than one response to the question was possible. nineteen of the post-graduate students wrote of the strengths of being able to network with others working in similar areas through social and academic interaction. another strength identified by seventeen of the consortium participants was the opportunity for critical, independent appraisal by peers and experienced researchers, particularly regarding research methods. more specifically, the opportunity to be able to focus on method was mentioned by eight students. eight of the participants found exposure to other research projects of value, while six students stated that they appreciated the support for common problems that the consortium provided. weaknesses of doctoral consortia the disadvantages of the doctoral consortium mentioned by the participants were more disparate than the stated advantages. a lengthy list of disadvantages suggested only once has been omitted for brevity. the main disadvantages are summarised in table 3. 84 ajis vol 4 no. 1 main weaknesses of doctoral consortia limitations in management of sessions limitations in facilitation of groups size of groups too large insufficient time for feedback research area too broad within groups number of responses 8 8 5 5 4 table 3 main weaknesses of doctoral consortia as identified by research students surveyed eight students made comments relating to poor management of the sessions, such as 'one project dominated' and 'my facilitator was from my university'. eight students commented negatively on the quality of the facilitation, with statements such as 'facilitator's review superficial [sic] as insufficient time for them to become familiar with your work' and 'facilitators weak in dealing with students, especially mature students, nearly all of whom were lecturers'. five of the students stated that the size of the groups had been too large and another five considered that there had not been enough time for feedback. four participants wrote that the areas of research had been too broad within the groups, both in terms of content and method, so that it was hard to maintain the interest of the other participants. comparison of doctoral consortia and systematic expert review some students were critical of the match between other students and the reviewers to their own work in the doctoral consortium they had attended. one student commented, 'ideally, the student should be able to determine beforehand which student's work is most relevant to them and be grouped accordingly. secondly, they should have more say in picking their reviewer some reviewers can obviously input more ideas in a given field than other reviewers.' in comparison, feedback from expert review was considered to have greater relevance to the student. one student commented that expert review was of more benefit as 'the reviewers had more expertise in my field and [...] a better understanding of what i was trying to do. secondly, comments were made that applied to each individual's work...'. a similar view was communicated by another student in the statement, expert review has '...been more benefit than the doctoral consortium. this is particularly true if one considers the cost benefit relationship. other sources are naturally one-on-one and therefore have the potential to being [sic] more productive or targeted.' several students saw doctoral consortia and expert review as fulfilling different roles. a comment that exemplifies this view is, 'the expert review evaluation was of more value because they were able to spend more time on looking at the content of my work. the focus at the doctoral consortium was more on research method.' another student wrote, expert review was '...more valuable because comments made were more critical. [the comments] served to focus attention on specific strengths and weaknesses. [it] would be fair to say that the doctoral consortium did serve to highlight the general issues and therefore gave a good base for the comments from the [expert review ...] which came at a later date'. a third student summarised this perspective by writing that expert review and 'doctoral consortia are different, both are beneficial but for different reasons.' when the students were asked if the doctoral consortium had been of value in helping them focus the research proposal, 23 (62%) agreed with the statement (ten strongly), while only eight (22%) disagreed with it (two strongly disagreed). the discrepancy in the total represents the students who were uncertain. some of the participants of the two doctoral consortia had their research proposal reviewed by researchers other than their supervisor/s or the facilitators at the doctoral consortium. when this group was asked about the value of expert review all thirteen who answered the question were positive. this totally positive response must be contrasted with the 62% response that participation in the doctoral consortium had been of value. when asked .to evaluate the most effective focus mechanism, twelve students had taken part in both expert review and a doctoral consortium and stated a preference. of these, nine preferred expert review while only three preferred the doctoral consortium process. discussion of results the student preference for expert review to help focus a research proposal is revealed by the survey. during systematic expert review, students have contact with a range of academics over a long period to develop the proposal. one student referred to the ongoing nature of systematic expert review when he wrote, 'the whole 85 ajis vol 4 no. 1 project has evolved over time. each individual, at various stages, helped shaped [sic] my thoughts about the project.' as only two students enrolled in both doctoral consortia, evidence from the survey suggests that once a student has participated in a doctoral consortium, enrolment in future consortia is unlikely. eighteen or 53% of the students had not had their proposal reviewed by anyone other than their supervisor/s or doctoral consortia facilitators, either from inside their department or outside. it may be that those having experienced expert review feel less need to attend a doctoral consortium. however, this explanation is not supported by studies by powles (1988) and cullen et al. (1994) of students' general recourse to non-supervisor review of research proposals. cullen et al.'s (1994) findings that 22% of the population of the students surveyed at an australian university obtained 'critical assistance' from academics in the same department, and 8% obtained the same kind of assistance from academics outside their department are considerably lower than the results obtained in this study. contrary to the conjecture of the previous paragraph, students participating in a doctoral consortium may be more likely to seek outside assistance to focus their project. the nature of is itself may also encourage students to obtain outside assistance. these factors may explain the higher proportion of students in this study who sought a review of their proposal than in the cullen study. powles' (1988) findings may provide some support to the suggestion that the group surveyed in the current study may be more likely to seek assistance from academics other than their supervisors. he found that more than 30% of phd students surveyed enrolled for four years or over had not attended a conference. as the two doctoral consortia were held in association with a conference, and the great majority of the students attending one of the consortia also attended the conference, the students captured in this survey do not fit the category identified by powles. there is another possible explanation for the discrepancy between the findings of cullen et al. and this study. it may be that in cullen et al.'s study more of the students than indicated sought outside assistance but that assistance had not been of a 'critical' nature. the figure from cullen et al. indicates not only the proportion that sought help, but also comments on the significance of that assistance. however, in findings of an earlier study completed at an australian university, powles (1988) obtained figures closer to those obtained by this author. in that study, 55% of higher degree research students indicated that they had received no advice from staff in departments outside their own, and 36% had no help from academic staff other than their own supervisor in their own department. powles' study broke the contact with academics into two categories, from within and outside the department, and considered advice from staff, a broader grouping than academic staff. however, there is some parallel with the findings of this study that 53% had not had their proposal reviewed by an academic other than their supervisor or a doctoral consortium facilitator. there are some advantages of doctoral consortium participation that are not found in systematic expert review, such as the potential for networking, exposure to other research projects and support for common problems. however, many of these strengths are also offered by gatherings of is researchers, as occur at is conferences. moreover, many of these advantages are of only marginal relevance in helping a student focus and develop a research topic. in fact, one participant commented that the doctoral consortium was not the forum for focussing research proposals, 'especially since most participants did not have access to the domain knowledge necessary to assist in focussing. discussion of method is much more important.' most of the comments from students relating to the weaknesses of doctoral consortia are concerned with management issues regarding use of time and grouping students, and the difficulties of obtaining specific and appropriate feedback for each participant. an interesting discrepancy was noted between one advantage of doctoral consortia given by 17 students, 'independent appraisal by peers and researchers' and individual comments that revealed frustration in obtaining specific feedback on an individual's research. the reason for the mismatch may be confusion about the purpose of doctoral consortia, with many students anticipating individual feedback but not obtaining it. nearly all the drawbacks mentioned would not occur with systematic expert review, where the feedback provided would relate specifically to the relevant student. a participant wrote, 'in my trial doctoral consortium my subject was the focus of attention unlike the consortium where i felt it was not really appropriate within the session group.' although specific advantages and disadvantages of systematic expert review were not canvassed in the survey, one obvious disadvantage of systematic expert review when conducted on a large scale is the considerable cost of individual interaction. another is finding suitable reviewers. many of the advantages of the doctoral consortium process are also advantages of systematic expert review, for example they both allow academic interaction. however, it was found that some strengths of systematic expert review are not offered by doctoral consortium participation. 86 ajis vol 4 no. 1 limitations of study and suggestions for future research the investigation was exploratory. all students had experienced a doctoral consortium of a similar form. however, as not all students had been exposed to systematic expert review and it had not been implemented in a uniform manner, any comparison of the two mechanisms cannot be rigorous. for a more formal evaluation of the mechanisms, the trial of a uniformly administered systematic expert review program is recommended. as more is doctoral consortia take place, it may be interesting to compare feedback from students related to individual doctoral consortia against variations in how they were run, in order to refine the process. conclusions it is clear that doctoral consortia and systematic expert review each have different strengths and weaknesses. moreover, responses from the students suggest that each of the two mechanisms serves a different purpose and one should not be considered a replacement for the other. the following conclusions are specific to doctoral consortia: • doctoral consortia may function as a preparation for an expert review at a later date. • doctoral consortia are discrete events. students may not wish to enrol in more than one. • provision of detailed individual feedback at doctoral consortia may not be appropriate. doctoral consortia are most suitable for: • social and academic interaction to overcome the isolation of research students • networking and support for common problems • focussing on methodological and other issues of general interest • provision of general advice the following conclusions are specific to systematic expert review: • as it is likely that a reviewer's field will be of relevance to the student's project, an expert review is likely to result in targeted advice. • systematic expert reviews have a developmental nature in that they place emphasis on the process of change. such an emphasis may lead to an increased number of iterations of the proposal, and as a result, greater refinement in its focus and quality. • comments of reviewers will validate reservations or the confidence of a student or supervisor/s regarding a proposal and its viability. systematic expert review is most suitable for: • provision of detailed feedback to the student • helping to focus a research proposal • building ongoing research relationships with experts other than the student's supervisor/s although some students had arranged expert review of their proposals, it was found that systematic expert review was not undertaken by the majority of the students who participated in two is doctoral consortia. a range of other mechanisms were used to focus a research proposal, but none were considered as highly as participation in a doctoral consortium or external review. students who had been involved in both procedures considered systematic expert review to be the most valuable mechanism for focussing an is phd research project. the author is currently a member of an advisory group assisting the chair of the doctoral consortium for the 7th australasian conference on information systems. recommendations although the following recommendations have been derived from a study set within an is context, they can be applied to the task of focussing a post-graduate research degree in most disciplines. the following recommendations are directed towards organisers of doctoral consortia and supervisors of australian postgraduate research students. • recognition needs to be given to the special requirements of australian post-graduates in determining the scope and approach of their project when compared with some other 87 ajis vol 4 no. 1 international post-graduate research programs. strategies designed to overcome such difficulties, such as those evaluated in this study, need to be considered. • doctoral consortia are recommended for research students early in enrolment for exposure to broad methodological issues and to establish networks, before or at the time of writing a project proposal. • after a student has developed a proposal, systematic expert review is the best mechanism for refinement of the proposal through expert feedback. • systematic expert review is of particular value for students in universities with small is establishments or in isolated locations. • to ensure a match between the aims of the students and organisers of a doctoral consortium, students need to be made aware of the emphasis of the doctoral consortium, and be advised that detailed individual feedback on their research proposal may not be possible. • doctoral consortium organisers need to allow students some choice in the most relevant peer group to join for discussion. • doctoral consortium organisers need to allow students some choice in which facilitator sessions to join, to ensure that the background and interests of the facilitator are the most relevant to the student's work. • doctoral consortium organisers need to provide for input from students with regard to discussion topics to ensure relevancy of the program. references conrad, l. (1991) the doctoral consortium: a cross-institutional approach to supplementing postgraduate research supervision, herds a news, vol 13, no 2, pp 6-8. cullen, d.j., pearson, m., sana, l.j. & spear, r.h. (1994) establishing effective phd supervision, department of employment, education and training, canberra: australian government publishing service. farhoomand, a.r. (1992) scientific progress of management information systems, in r. galliers (ed.) information systems research: issues, methods and practical guidelines, oxford: blackwell scientific publications. gable, g.g. & clarke, r.a. (1995) 1996 asia pacific directory of information systems researchers, brisbane: information systems management research concentration, queensland university of technology. galliers, r. (1992) choosing information systems research approaches, in r. galliers (ed.) information systems research: issues, methods and practical guidelines, oxford: blackwell scientific publications. higher education council (1990), higher education courses and graduate studies, canberra: australian government publishing service. jick, t.d.(1983) mixing qualitative and quantitative methods: triangulation in action, in j. van maanen (ed.) qualitative methodology, beverly hills: sage publications. keen, p.g.w. (1991) relevance and rigour in information systems research: improving quality, confidence, cohesion and impact, in h.-e. nissen, h.k. klein and r.a. hirschheim (eds.) information systems research: contemporary approaches and emergent traditions, proc. ifip tc8/wg 8.2 working conference, dec. 1990, amsterdam: north-holland. locke, l. & spirduso, w. (1976) proposals that work, a guide for planning research, new york: teachers college press, colombia university. miller, d.m. (1993) mis as a discipline: a structured definition, sigcse bulletin, vol 25, no 4, pp 27-32. mitchell, j. c. (1983) case and situation analysis, sociological review, vol 312, no 2, pp 186-211. parry, s. and hayden, m. (1994) supervising higher degree research students; an investigation of practices across a range of academic departments, canberra: department of employment, education and training, australian government publishing service. powles, m. (1988) know your phd students and how to help them, melbourne: centre for the study of higher education, university of melbourne. roscoe, j.t. (1975), fundamental research statistics for the behavioral sciences, 2nd edn, new york: holt, rinehart & winston inc. shanks, g., rouse a. & arnott, d. (1993) a review of approaches to research and scholarship in information systems, melbourne: working paper series 3/93, monash university. 88 ajis vol 4 no. 1 acknowledgements the author would like to thank dr chris keen, associate professor glenn lowry and members of the information systems group of the university of tasmania, and dr pauline o'connor for their valuable comments on the draft of this paper. 89 paper title (paper title style) australasian journal of information systems nathawitharana, huang, ong et al 2017, vol 21, selected papers from ausdm towards next generation rubrics 1 towards next generation rubrics: an automated assignment feedback system nilupulee nathawitharana la trobe university nathawitharana.n@students.latrobe.edu.au qing huang la trobe university kok-leong ong la trobe university peter vitartas la trobe university madhura jayaratne la trobe university damminda alahakoon la trobe university sarah midford la trobe university aleks michalewicz la trobe university gillian sullivan mort la trobe university tanvir ahmed la trobe university abstract as the use of blended learning environments and digital technologies become integrated into the higher education sector, rich technologies such as analytics have shown promise in facilitating teaching and learning. one popular application of analytics is automated writing evaluation (awe) systems. such systems can be used in a formative way; for example, by providing students with feedback on digitally submitted assignments. this paper presents work on the development of an awe software tool for an australian university using advanced text analytics techniques. the tool was designed to provide students with timely feedback on their initial assignment drafts, for revision and further improvement. moreover, it could also assist academics in better understanding students’ assignment performance so as to inform future teaching activities. the paper provides details on the methodology used for development of the software, and presents the results obtained from the analysis of text-based assignments submitted in two subjects. the results are discussed, highlighting how the tool can provide practical value, followed by insights into existing challenges and possible future directions. keywords: learning analytics, automated writing evaluation, text analysis, assignment feedback 1 introduction with the emergence of big data and advances in text analytics, information retrieval, machine learning and natural language processing, academics are able to explore new opportunities to understand student performance in assignment submissions. this is of particular importance to academics involved in extra-large subjects and massive open online courses (moocs). australasian journal of information systems nathawitharana, huang, ong et al 2017, vol 21, selected papers from ausdm towards next generation rubrics 2 while much of this work is still being advanced in a range of disciplines, the opportunity arises to explore how such techniques can assist and be applied to the management of assignment marking and moderation in large subjects. the adoption of digital submissions for assignments via the learning management system (lms) and the development of text-analytic algorithms has seen a range of automated writing evaluation (awe) or automated essay scoring (aes) systems emerge (deane, 2013; shermis & burstein, 2003), which have been shown to be comparable to human markers (mcnamara, crossley, roscoe, allen, & dai, 2015). examples of automated analysis and evaluation of written text are emerging in a variety of contexts, from formative feedback in writing instruction (from primary through tertiary education), to summative assessment (e.g. grading essays or short answer responses with or without a second human grader). as class sizes increase, there is a corresponding increase in the use of large-scale exams and the generation of large volumes of writing to be evaluated and assessed (e.g. 1000+ students in first year university classes, naplan in australia, exams based on the common core state standards initiative in the us and the rise in popularity of moocs). at the same time, the increase in interest of marketers to explore the meaning of messages and brand sentiment in social media such as facebook, twitter and blogs has seen increased demand for ever-more sophisticated text analysis tools. such developments in text analysis are finding their way into applications for education and demonstrate potential for the assessment of text based writing such as assignments. as a result, awe systems have drawn on multidisciplinary insights from computer science, linguistics, writing research, cognitive psychology, and educational data mining (edm). in this research, the use of business analytics methods is being applied to the development and refinement of software that can not only provide formative feedback to students, but also assist teachers to understand their students’ performances and areas where guided instruction could be directed to understanding terms and concepts covered, or not covered, in assignments. 1.1 next generation rubrics the next generation rubrics (ngr) project (vitartas et al., 2016) was established at an australian university as a collaboration between a small number of academics and a newly appointed business analytics team. the project was supported by an internal learning and teaching grant and sought to develop a tool that provided students and academics with information about the performance of text based assignments as a proof of concept. the ngr software aims to provide descriptive timely feedback to students on their digitally submitted text assignments as well as assist the academic to better understand students’ assignment performance and learning outcomes. the starting point for the development of the ngr software was a marking rubric, as these underpin many standardised marking schemes at universities. while it is acknowledged that rubrics have come to have a range of meanings to various people (dawson, 2017), in this paper its meaning is based on popham’s definition: “a scoring guide used to evaluate the quality of students’ constructed responses” (popham, 1997, p.72) and consists of an evaluative criterion, and guidance on expectations for associated scores or marks (popham, 1997). the project analysed assignments from two subjects. the first is an introductory first-year subject in the bachelor of arts (ba). the assessment regime required students to submit a 1500-word assignment and a total of 115 assignment scripts were included for analysis. students answered one of five questions, analysing the ways individualism, imperialism or secularisation has impacted contemporary society. the second subject is at the masters level from the marketing discipline (mkt) in the business school at the same university. students were required to present a 2500-word report on a strategic analysis of a new product introduction for a video streaming company. a total of 80 assignment scripts were analysed from this subject. the software was developed to evaluate students’ performance for the purpose of providing feedback. moreover, cluster analysis of the assignments was also conducted based on calculated evaluation measures in order to provide the academic with a detailed view of australasian journal of information systems nathawitharana, huang, ong et al 2017, vol 21, selected papers from ausdm towards next generation rubrics 3 students’ assignment performance, e.g. how evaluation measures are performed across the student cohort. by examining the results from the analysis, a greater understanding of the computer’s ability to assess student performance could be attained, and the results could be compared to the human issued marks for each assignment. the remainder of the paper first provides the background to the development of the awe tool and its role in facilitating teaching and learning. the methodology used for ngr’s development is then outlined, followed by the presentation and discussion of the results from this study. furthermore, we also share the clustering results and discuss how they could assist academics to extract further insights into their cohort. prior to the conclusion, we discuss the challenges and limitation in the field and suggest possible future directions. 2 background innovative analytical tools are providing opportunities for educational designers and teachers to understand student performance in much greater detail than ever before. tools such as text analytics, information retrieval, machine learning, natural language processing and learning analytics form part of the suite of big data analytics that has the potential to provide evaluative feedback on students’ work (shermis & burstein, 2013). in this section, we provide a brief review of some key concepts underlying the technology being applied in awe, drawing on insights from computer science, linguistics, writing research, cognitive psychology, educational data mining (edm) and learning analytics (la). we provide a synopsis of the awe tools and then discuss how awe tools could possibly assist both the students and the academic to gain valuable insights from text-based assignments. 2.1 educational data mining (edm) and learning analytics (la) the edm community website describes edm as “an emerging discipline, concerned with developing methods for exploring the unique and increasingly large-scale data that come from educational settings, and using those methods to better understand students, and the settings which they learn in” ("educational data mining," 2017). with the advent of moocs and publicly available data, such as the pittsburgh science of learning center datashop, edm research has accelerated in recent years. learning analytics (la) was defined in the first international learning analytics and knowledge (lak) conference as “the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimising learning and the environments in which it occurs” (1st international conference on learning analytics and knowledge, 2010). it draws on the increasing range of data available from digital learning tools and environments. siemens and baker (2012) promote a closer collaboration between the edm and la communities, as the two groups share the goals of improving both educational research and practice through the analysis of large-scale educational data. one interesting development from edm/la that is relevant to awe is white and larusson’s ‘point of originality’ tool (white & larusson, 2010). this is designed to help instructors in large university courses, such as first year gateway courses, to monitor students’ understanding of key concepts. the system uses wordnet (miller, 1995), a large lexical database of english words, to “track how a student’s written language migrates from mere paraphrase to mastery, isolating the moment when the student’s understanding of core concepts best demonstrates an ability to place that concept into his or her own words, a moment we’ve chosen to call the ‘point of originality’” (white & larusson, 2010, p. 817). 2.2 natural language processing (nlp) put simply, nlp is “an area of research and application that explores how computers can be used to understand and manipulate natural language text or speech to do useful things” (chowdhury, 2003, p. 51). some of these ‘useful things’ include machine translation, speech recognition, information retrieval and extraction, summarisation, and, relevant to awe, text australasian journal of information systems nathawitharana, huang, ong et al 2017, vol 21, selected papers from ausdm towards next generation rubrics 4 processing. liddy (2001) points out that nlp can operate at various levels of linguistic analysis, including phonology, morphology, lexical, syntactic, semantic, discourse and pragmatic. 2.3 automated writing evaluation (awe) awe systems can be classified as either simulation-based assessments or response-type systems (williamson, xi, & breyer, 2012). the former present computerised simulations of real-life scenarios, and are usually specific to a certain test (such as the united states medical licensing examination). the latter are more generalisable in that they score a typical type of response such as mathematical equations, short written responses, spoken responses, or essays. essay scoring has been a particular focus for many automated systems and numerous essay evaluation systems are now used in formative feedback as well as high-stakes testing. in these tests the automated assessor acts either as a second rater to assist human scorers (e.g. in ets’ toefl test ets 2015) (ets®, 2017b), or as the sole rater (e.g. the pearson test of academic english (pte academic) uses automated scoring for writing and speaking (pearson, 2012)). the following section provides a synopsis of a number of available tools. a history of awe generally begins with the work of english teacher turned researcher ellis page, and his project essay grade (peg) beginning in the 1960s. in a 1966 article in the phi delta kappan, page insisted “we will soon be grading essays by computer, and this development will have an astonishing impact on the educational world” (page, 1966, p. 238). ellis had an optimistic vision for writing feedback being provided to students much more extensively and in a timely manner than could be achieved by english teachers in school or college. something of page’s vision is now closer to reality in various automated formative tools, offering immediate descriptive feedback on writing (e.g. writelab (writelab, n.d.), turnitin’s revision assistant (turnitin, 2017), pearson’s writetolearn™ (pearson, 2017), ets’ criterion® (ets®, 2017a), and vantage learning’s myaccess! (vantage learning, 2016)). in addition to these proprietary products, a number of freely available services from the academic domain are also available to examine text and extract phrases including coh-metrix (cohmetrix, n.d.), wordnet (miller, 1995), termine (frantzi, ananiadou, & mima, 2000), mallet (mccallum, 2002), stanford core nlp (manning et al., 2014) and natural language toolkit (nltk) (bird, klein, & loper, 2009). 2.4 the role of awe tools for both student and teacher insights a considerable amount of work being undertaken in the development of awe tools is focused on more accurately evaluating how students perform in their writing assignments. however, evaluation results are only as valid as when they are able to provide meaningful insights to both students and teachers in terms of informing future learning and teaching. therefore, a critical role of awe tools could be providing the opportunity to understand better how and why students are performing in this way. in fact, the ngr project was initiated by the university’s need to provide students with timely and personalised feedback coupled with the desire to extract further insight into students’ assessment writing for academics. while most of the existing awe tools focus on providing automated scoring rather than generating feedback, our project aims to cater to the specific requirements of the university and provide a customised solution to the need for high-quality feedback on written assessment as a formative learning tool. therefore, the research presented in this paper focuses on automatically providing descriptive feedback to students via the analysis of text-based assignments. software was developed on the basis of five dimensions of analysis to provide formative feedback. furthermore, clustering techniques were applied to uncover any hidden patterns in the text data that may contain useful information for enhancing an understanding of the results. a stronger understanding of students’ assignment performance could potentially provide meaningful insights for academics, informing future teaching activities and subject design. the project is continuing australasian journal of information systems nathawitharana, huang, ong et al 2017, vol 21, selected papers from ausdm towards next generation rubrics 5 and has already provided useful insights into students’ writing performance and the effectiveness of teaching activities. 3 methodology 3.1 ngr software functionality and architecture the ngr software assists both students and academic staff. the main functionality is assessing digitally submitted assignments and providing descriptive feedback to students. this process allows students to submit an unlimited number of draft versions of their assignments, obtain feedback and improve them before they make a final submission for (human) marking. upon submission of an assignment, the software conducts content analysis, evaluates the assignment based on an evaluation criterion, and generates feedback based on the analytical results explaining which areas need further improvement. for academic staff, the software allows the performance of individual students and classes to be monitored. academics can add subjects and assignments for their subjects using the software. in addition, they can set evaluation criteria for those assignments which will be used during the analysis process. the software facilitates the analysis of different types of written assignments via a generalised evaluation mechanism. however, evaluation of assignments might depend on the nature of the writing task, and the importance of different evaluation measures might depend on the type of assignment and subject context (e.g. report or essay). therefore, the software facilitates the functionality to customise evaluation measures based on the type of assignment. the software is accessed by both academic staff and students, and the user interface is implemented to be simple but user-friendly. more importantly, different functionalities need to be provided by the software for the different users. students are able to access the analysis results and descriptive feedback generated for the assignments they have uploaded and the academic staff are able to view the performance of the class as well as query and observe the performance of individual students. the software caters for this requirement by maintaining different access levels for different types of users. the high-level architecture of the ngr software is presented in figure 1. the software consists of four layers; a user interface tier is the topmost level and defines the presentation to the users. this tier communicates with the business logic tier to retrieve essential information for the presentation. the business logic tier manages users, grading, subjects, assignments, and dashboard presentation. the various analytics are implemented on the service tier and the database tier stores assignments and other data essential for the analysis. during the analysis, various analytical functions implemented in the service tier communicate with the data tier to retrieve critical information, and afterwards the analysis output is presented to the business logic tier, which organises the results and presents them to the end-user via a user interface tier. australasian journal of information systems nathawitharana, huang, ong et al 2017, vol 21, selected papers from ausdm towards next generation rubrics 6 figure 1: high-level architecture of the ngr software the ngr project is ongoing and the software continues to be refined. the current version of the software facilitates the calculation and evaluation of measures and presents the outcome to dashboards. academic staff can view the performance of classes and individual students. likewise, students can access the software and view the values of the evaluation measures for the assignments they have uploaded. the weighting of evaluation measures and personalised feedback are currently under development and will be incorporated into the next version of the software. the following subsections provide details of the implementation of the analytical component, core dimensions used for the evaluation, and tools and technologies utilised in the development of the software. 3.2 implementation of the analytical component identifying proper evaluation criteria to be implemented for the assignment analysis logic has been one of the main objectives during the implementation phase of the ngr’s analytical component. in addition, the evaluation criteria expected to assess assignments and identify the areas have been a focus of the software development, but require further improvement. rubrics which defined the scoring guide play a vital role for the marker during manual assignment grading. therefore, the rubric was considered the starting point for the development of the analytical component. a set of already marked assignments was analysed to identify important evaluation measures, and for this project, assignments from two different subjects were used to identify valid evaluation criteria. the discipline and level of the degree were two controlling factors used for assignment assessment, and it was also noted that trialling subjects from different disciplines and degree levels provided insights into a wider range of issues which might occur during the analysis process. therefore the two subjects selected for developing the analytical component belonged to different disciplines and degree levels. the first subject was selected from the bachelor of arts (ba) degree and therefore represented the humanities disciplines. several prominent concepts that emerged during the european enlightenment (such as individualism, imperialism, secularisation and freedom/democracy) are explored from their origins to their manifestation in contemporary society in this first year core subject. the assignment selected from this subject was an essay of 1500 words based on five themed questions. approximately 800 student essays were submitted to this subject, and one themed question with 115 submissions was selected for analysis. the second subject selected for the analysis was chosen from the masters of management marketing (mkt). this australasian journal of information systems nathawitharana, huang, ong et al 2017, vol 21, selected papers from ausdm towards next generation rubrics 7 subject aimed to provide guidance and develop the necessary skills to understand and implement marketing strategy in a business context. the selected assignment was a 2500 word executive report in which students conducted a strategic marketing analysis of a new video streaming company. there were 80 submissions of this assignment. 3.3 core dimensions for assessment as mentioned in the previous section, a marking rubric provides the expectations for each grade and describes different areas which need to be considered for the allocation of marks. in general, the marking criteria for an assignment includes multiple components, and the marking guide for the selected two assignments were consulted to identify important measures for the evaluation. it has been noted that the assignment structure, proofreading errors, evidence for research and critical thinking, use of discipline terms, and use of accurate referencing mechanism are important for mark allocations. consequently, the following five dimensions were incorporated into the analysis. 1. assignment statistics 2. proofreading errors 3. research, critical thinking, and discipline words and phrases 4. assignment readability 5. referencing information assignment statistics measure how well each assignment complies with the submission guidelines. the assignment guide included the submission requirements stating page and word limits, and final format for the submission. word count, paragraph count, and page count are calculated as assignment statistics to verify whether an assignment follows the prescribed format. proofreading errors, spelling and grammar error counts are calculated per 1000 words. considering the length of the assignment provides a standardised representation of proofreading errors as the occurrence of those errors are proportional to the length of the document. research, critical thinking and discipline terms are captured to verify whether the assignment covered required disciplinary concepts, contains research related content, and critically evaluated the evidence. the use of research, critical thinking, and discipline terms are calculated by referring to a set of term lists. therefore during the initial development phase, subject experts were consulted to retrieve research and discipline terms for each assignment. for measuring critical thinking, a term list developed by paul (1995) was used. grading an assignment involves looking into writing style as well. the structure of sentences, diversity of vocabulary, and sophistication of the writing style are considered when allocating marks, and the writing style has an enormous impact on the readability of a document. therefore readability is included as a dimension to cover that aspect. flesch reading ease (flesch, 1948) and flesch-kincaid grade level (kincaid, fishburne jr, rogers, & chissom, 1975) are very popular readability measurements, and the analysis incorporates those two measures to calculate readability scores. for most assignments from the disciplines selected for the project, students are required to conduct literature reviews, critically evaluate situations, and support arguments and conclusions with evidence gathered from relevant literature. when referring to their research it is essential to provide citations (in-text references) and a list of references or bibliography at the end of the assignment. therefore, reference information plays a vital role during the analysis process. the ngr software calculates the number of correctly formatted citations, number of all citations disregarding the format, the number of distinct authors in the citations, and the number of references in the reference list or bibliography. in most instances, assignment submission guidelines include a preferred referencing style. therefore the format of the citation is checked to verify whether the correct referencing style is followed. also, the number of distinct authors helps to identify whether the student has conducted a comprehensive literature search. australasian journal of information systems nathawitharana, huang, ong et al 2017, vol 21, selected papers from ausdm towards next generation rubrics 8 3.4 tools and technologies used for the implementation for the development of the software both c#.net (wagner, 2015) and java (oracle, 2017) programming languages have been used. the analytical component is implemented using c#.net, whereas the user interface is developed using java. for the user interface development, kibana dashboard (elasticsearch, 2017) has been utilised. also, the software uses functionalities provided by the microsoft developer network (msdn) when calculating the assignment statistics, proofreading errors and readability (microsoft developer network, 2017a, 2017b). the termine web tool (frantzi et al., 2000) has been utilised to capture phrases from the assignments which are useful when identifying research, critical thinking, and discipline terms. 4 analysis and results 4.1 pre-processing and sampling for initial analysis, it is necessary to pre-process and clean the assignment files. this includes de-identifying the assignments and cleaning the content by removing cover pages, page numbers, table of contents, figures, and other irrelevant characters and symbols. samples were selected from the two assignments for initial analysis. for the ba subject 77 assignments were selected, and for the mkt subject 60 assignments were selected. the remaining assignments were excluded so that they could be used as a test sample to validate the tool’s functionality once the software implementation was completed. 4.2 analysis during the analysis, values for the key evaluation dimensions are calculated for the selected samples from the two assignments. also, the average value for each dimension is calculated, and the distribution of the average values across grades was investigated to discover relationships between those dimensions and the grades. the main objective of the analysis was to identify relationships between grades and evaluation dimensions, as that information can be utilised when generating feedback. initially, the selected assignments were pre-processed and directed to the ngr software to calculate assignment statistics and proof-reading errors. references and appendices were excluded from the calculation. when calculating the research, critical thinking, and discipline terms both words and phrases are extracted from the assignments to calculate the frequencies. in addition provided word lists for research, critical thinking, and discipline terms were stemmed, as stemmed words represent the root form and provide a more generalised view of those terms. for stemming, porter’s stemming algorithm (porter, 1980) was used. frequencies were calculated for both stemmed and non-stemmed words and phrases, and the calculation excluded references and appendices. for assignment readability, flesch reading ease and flesch-kincaid grade level were calculated. finally, referencing information was calculated for the selected assignments using the software. 4.3 results and discussion table 1 and 2 present a summary of the results calculated using the selected samples for the ba and mkt subjects. australasian journal of information systems nathawitharana, huang, ong et al 2017, vol 21, selected papers from ausdm towards next generation rubrics 9 human marked grade 0-49 (f/<50%) 50-59 (d/ 50-59%) 60-69 (c/ 60-69%) 70-79 (b/ 70-79%) 80-100 (a/80+%) all # of assignments 10 14 17 26 10 77 critical thinking, research, & discipline terms av # of critical thinking words 11.4 10.2 10.9 12.0 12.9 11.5 av # of research words 19.7 20.6 22.9 23.9 24.8 22.7 av # of discipline words 54.9 64.4 65.8 70.8 91.1 69.1 citations & references av citations 11.0 9.3 8.4 14.6 13.6 11.7 av distinct authors 2.8 5.0 3.0 5.2 5.1 4.4 av # of references 4.9 7.9 7.5 8.9 8.4 7.8 word statistics & proofreading errors av word count 1445 1503 1650 1726 1765 1637 av paragraph count 7 10 6 7 9 8 av spelling error per 1000 words 5.6 6.0 6.6 7.9 7.7 6.9 av grammar error per 1000 words 2.5 1.3 1.0 1.3 0.6 1.3 readability flesch reading ease 44.1 41.4 38.4 33.3 28.2 36.6 flesch-kincaid grade level 13.5 13.6 14.5 15.6 16.2 14.8 table 1: summary results for ba subject human marked grade 0-19 (f/<50%) 20-23 (d/ 50-59%) 24-27 (c/ 60-69%) 28-31 (b/ 70-79%) 32-40 (a/80+%) all # of assignments 3 12 18 20 7 60 critical thinking, research, & discipline terms av # critical thinking terms 16.7 17.5 15.7 17.7 18.7 16.3 av # of research words 41.3 49.4 40.1 42.5 40.6 42.9 av # of discipline words 49.3 55.3 49.7 46.0 87.6 54.0 citations & references av citations 9.7 17.7 12 13.2 30.7 15.2 av distinct authors 5.3 8.25 7.2 6.8 13.6 7.9 av # of references 9.7 12.1 12.4 12.6 18.6 13.0 word statistics & proofreading errors av word count 2151 2603 2697 2758 3284 2739 av paragraph count 45 66 66 68 110 70 av spelling error per 1000 words 4.3 10.3 10.3 9.6 9.6 9.7 av grammar error per 1000 words 0.6 2.5 3.8 2.7 1.3 2.7 readability flesch reading ease 35.3 39.3 39.3 38.7 33.4 37.8 flesch-kincaid grade level 14.5 12.8 12.7 12.9 14.1 13.0 table 2: summary results for mkt subject the two assignments selected for analysis belong to different disciplines, and degree levels. this meant each had quite different expectations of the students. the ba assignment was an essay on a themed question which expected students to refer to literature, present arguments and draw conclusions. however, the mkt assignment was focused on preparing an executive report on the marketing analysis of a company. the initial analysis results conveyed the differences between the two assignments. as an example, the average number of paragraphs australasian journal of information systems nathawitharana, huang, ong et al 2017, vol 21, selected papers from ausdm towards next generation rubrics 10 for marketing assignments was very high when compared with the ba assignment, as content was presented as a report with bullet points or short paragraphs. the analysis results also conveyed that the readability values had a significant relationship with the assignment marks, particularly for the ba subject. according to the flesch reading ease score, when the readability score decreases, it indicates that the document is more suited to an audience with a higher education. this relationship seems more prominent for the essay-type assignments based on findings from the analysis. figure 2 presents the relationship between the flesch reading ease score and assignment marks for the ba subject. according to above analysis, it can be concluded that different types of assignments are made up of different characteristics. therefore, it is vital to consider the assignment type when designing the assessment criteria. the requirement for customising the evaluation measures based on the assignment type is important for obtaining accurate analysis outcomes and therefore will be incorporated into the software by introducing a weighting mechanism for each evaluation measure. the academic staff can prioritise each evaluation measure by assigning a weight which provides the opportunity to conduct a more customised evaluation for each assignment. figure 3 and 4 present the assignment evaluation page and the dashboard for academic staff respectively. similarly, students will have the opportunity to view their values for evaluation measures for assignments they have submitted, as well as suggestions for improving those assignments. figure 2: average flesch reading ease across marks categories for ba subject australasian journal of information systems nathawitharana, huang, ong et al 2017, vol 21, selected papers from ausdm towards next generation rubrics 11 figure 3: assignment evaluation page in the software based on the values calculated for the evaluation measures, feedback will be generated for students. moreover, threshold values will be maintained by the software to decide whether sufficient information is presented for each evaluation measure. as an example, a threshold for reference count and number of distinct authors can be defined. if the assignment statistics for reference count and number of distinct authors is less than the pre-defined threshold values, then feedback will be generated to inform the student that they need to conduct a comprehensive literature review by referring to articles published by different authors. similarly, assignment statistics will be used to advise students about the issues associated with the structure of their assignments. research, critical thinking, and discipline term counts will inform students whether they have sufficient content from each category. australasian journal of information systems nathawitharana, huang, ong et al 2017, vol 21, selected papers from ausdm towards next generation rubrics 12 figure 4: dashboard for academic staff in the software 5 further analysis 5.1 cluster analysis as previously discussed, clustering of the assignment data could provide academics with more insights into students’ performance. therefore, attribute clustering has been conducted for all the assignments from the two subjects. the attributes selected for clustering are listed below. • spelling errors per 1000 words • grammar errors per 1000 words • number of references • readability measured by flesch reading ease score • research terms • critical thinking terms australasian journal of information systems nathawitharana, huang, ong et al 2017, vol 21, selected papers from ausdm towards next generation rubrics 13 • discipline terms sas enterprise miner (sas, 2016) was used to generate clusters. after an initial exploration of the data, it was decided to use five clusters for the extraction of segments for the ba assignments and four for the mkt assignments. the decision for the total number of clusters was made by trialling different cluster sizes with the main focus to obtain approximately five segments aligned with the five marking grades (a, b, c, d, and f). for the ba subject five segments could be obtained with a reasonable number of assignments in each segment. however, when the same procedure was executed for the mkt subject, one segment was very small when compared to the other segments. therefore, clustering was executed to obtain only four segments for the mkt subject. clustering is a useful approach when looking for similarities across the assignments and is also useful in informing academics how assignments have similar characteristics which can be used when providing feedback to students and evaluating the performance of students on assignments. 5.2 discussion based on the seven dimensions used in the study, sas enterprise miner has generated segment plots and segment profiles for the two subjects. the segment plots are presented in figures 5 and 6 for the ba and mkt subjects respectively. figure 5: segment plot for ba subject australasian journal of information systems nathawitharana, huang, ong et al 2017, vol 21, selected papers from ausdm towards next generation rubrics 14 figure 6: segment plot for mkt subject an examination of the segment plots highlighted the performance of each dimension for each segment and, in association with the segment profiles (not presented here) allowed for a picture of segments to be revealed. a summary of the key features of each segment is provided in tables 3 and 4 below for ba and mkt subjects respectively. segment segment description 1 (n=10) high on research and discipline terms, low on readability 2 (n=18) average performance across the discipline and research terms with low readability and critical thinking terms 3 (n=21) high readability with low performance on the discipline and research terms 4 (n=30) low discipline and research terms with moderate readability, and referencing 5 (n=36) high critical thinking terms with moderate readability, discipline and research terms. low spelling and grammar errors table 3: a summary of key features of each segment for the ba subject segment segment description 1 (n=9) high spelling errors and moderate discipline terms, readability, critical thinking and referencing 2 (n=36) high readability with remaining items low or moderate 3 (n=28) low grammar errors but high critical thinking and research terms 4 (n=7) high on all dimensions except for grammar errors and critical thinking table 4: a summary of key features of each segment for the mkt subject five segments have been identified in the clustering results for the ba subject. segment one consisted of high research and discipline terms and low readability, indicating stronger writing skills. segment two had mainly moderate or low measures across the descriptors, while segment three had a high readability score with low discipline term/phrase use, research and high spelling errors, suggesting an overall poor performance. segment four also exhibited overall poor performance with low discipline and research terms, with moderate readability and referencing. high critical thinking scores with moderate readability, discipline and research terms were noticeable in segment five, where spelling and grammar errors were low. australasian journal of information systems nathawitharana, huang, ong et al 2017, vol 21, selected papers from ausdm towards next generation rubrics 15 therefore, a good analysis by students but only average coverage of key concepts in the assignment was evident in segment five. according to the cluster analysis results, segment one was the best performing cluster, and when compared with the grades allocated by academics, it was observed that all the assignments belonging to segment one were awarded higher marks by human markers (marks ranged from 76% to 90%). when the allocated marks were considered for each other segment, it was observed that those segments consisted of a mixture of low, moderate, and high performing assignments. in particular, segment four, which exhibited overall poor performance, had the highest number of fail grades. four segments were identified in the cluster results for the mkt subject. high spelling error rate and moderate to low scores on the remaining dimensions were observed in segment one. segment two was the largest sized group and had a high readability score, but was low on most of the remaining dimensions suggesting poorer performing assignments. segment three also consisted of a large sample which was characterised by low grammar errors and high critical thinking and research terms. higher values for all dimensions except for grammar errors and critical thinking were noticeable in segment four. high performance in most of the dimensions and a low grammar error score suggested better performance by students. when examined in relation to the marks allocated by academics, it was observed that most of the assignments belonging to this segment had higher marks, where four out of the seven assignments had marks equal to or greater than 77.5%. for each other segment, it was observed that those segments contained a mixture of low, moderate, and high performing assignments. the results provide useful insights to academics, explaining how rubric elements are performed across the cohort. for example, in the ba subject, the largest segment performed well on critical thinking but only moderately on readability and discussion of discipline and research terms. in contrast, better overall performance was identified for segment one, especially regarding research and discipline terms and readability, although the use of critical thinking terms was only moderate. furthermore, weaker writing skills were exhibited in segment three due to higher readability scores combined with infrequent use of discipline and research terms. subject coordinators can learn from these findings, identifying areas of their curriculum that require revision and identifying students who require additional support in key areas of the curriculum. a similar approach undertaken in the mkt subject revealed groupings of assignments that performed highly across multiple dimensions (segment four) and poorly across most of the dimensions (segment two). of concern in this latter segment was a significant number of students doing poorly in comparison to the remaining cohort. while the research reported here has highlighted an approach to assignment analysis, of equal importance in this area has been developing an understanding of what is expected in an assignment by the subject coordinator and markers. in undertaking this work, it has been observed that markers do not take a structured approach to assessment marking, and provide inconsistent feedback to the subject coordinator on the shortcomings of an assignment. furthermore, it is often difficult for the subject coordinator to grasp a clear overview of performance across the whole subject, particularly because of the individual biases of different markers. the approach being developed in the ngr project provides much greater insight into the performance of students at the macro level because of its consistency. of interest in this analysis was the performance of each subject across multiple dimensions. by looking at how each subject performed on each dimension, insights can be provided to academics about large classes that assist them to better moderate assessment, better scaffold assessment skills acquisition in class, and gain a better understanding of student performance overall. as the cluster analysis facilitates valuable information about assignments, clustering functionality will be incorporated next into the software to provide more advanced analysis and efficient feedback generation. australasian journal of information systems nathawitharana, huang, ong et al 2017, vol 21, selected papers from ausdm towards next generation rubrics 16 6 future directions 6.1 challenges and limitations many challenges are faced by the ngr project while assessing student assignments. one of the key challenges is conducting the analysis when the assignment does not comply with submission guidelines. a common mistake by students during assignment submission is that they only pay attention to the main question, ignoring other important factors such as how the assignment should be formatted or how references should be presented. as an example, if references are included as footnotes in the assignment it becomes difficult to calculate referencing information accurately. however, as mentioned in the methodology section, the assignment analysis process is designed to be aligned with the rubric and submission guidelines. in situations where assignments do not comply with the submission guidelines, it leads to difficulties conducting analysis and obtaining accurate results. as such it was considered a limitation in the current version of the software. another major challenge is identifying research and discipline terms. currently, this process is done manually by academics and thus inefficient. moreover, measuring assignment coverage for research and discipline concepts based on a term list is not efficient and requires enhancements. one possible solution is to create an ontology or refer to an existing one to obtain a diverse set of terms related to a subject discipline. such an approach would be highly advantageous to identify whether assignments have addressed topics related to the subject discipline. in addition, the software possesses limitations in measuring referencing information as well, as it requires the citations and references to follow an acceptable style. moreover, limiting the evaluation to a set of pre-defined evaluation measures is also a limitation associated with the ngr software. if a new evaluation measure needs to be incorporated, then the software needs to be modified accordingly. as future work above mentioned, limitations can be addressed to enhance the usability of the ngr software and the accuracy of the analysis results. 6.2 future work as the current analysis techniques only facilitate the primary content analysis of the assignments, an investigation has been undertaken to identify further analytical methods which might be advantageous and integrated into the software. relevant techniques are listed below. • advanced style, spelling and grammar check • topic mining • capturing lexical chains • discriminant analysis the current version of the software captures spelling and grammar errors, however, incorporating advanced style, spelling and grammar checks could provide valuable insights into the analysis. such analysis allows capturing redundant phrases, non-standard phrases, commonly confused words, misuse of capitalisation and punctuation errors. moreover, differentiation between australian, american and british english can be conducted to verify whether the assignment complies with the preferred language. languagetool (languagetool, 2017) is an open source proofreading software which facilitates advanced style, spell and grammar checks. therefore, in the future, functionality provided by languagetool will be integrated to achieve a more comprehensive analysis. another important analysis technique is topic mining, which allows the discovery of major topics associated with each assignment. discovering topics is important as it identifies the key concepts covered in the assignments and theories referred from textbooks. several topic mining software tools are currently available, including mallet (mccallum, 2002) a popular open source tool which facilitates topic mining via latent dirichlet allocation (lda) (david m blei, ng, & jordan, 2003), pachinko allocation (li & mccallum, 2006), and hierarchical lda australasian journal of information systems nathawitharana, huang, ong et al 2017, vol 21, selected papers from ausdm towards next generation rubrics 17 (david m. blei, jordan, griffiths, & tenenbaum, 2003) algorithms. the software captures topics with topical terms and topic composition for the documents. in the future, the applicability of topic mining functionality will be investigated and if successful, integrated into the current version of the software using mallet. lexical chains allow the capture of sequences of related words in a document and considered a useful mechanism to capture semantic content of a document. lexical chains have been utilised for document summarisation tasks, as they capture the main themes discussed in the documents (wei, lu, chang, zhou, & bao, 2015). therefore, the possibility of integrating lexical chains into the analysis process will be investigated. capturing lexical chains will be beneficial in measuring whether assignments cover the necessary discipline concepts and can be utilised in grading and the feedback generation process. discriminant analysis is a popular classification technique and consists of different variations that include linear and quadratic forms. this technique can be used to classify assignments into grades based on key assignment characteristics and useful for assignment evaluation and feedback generation. however, this technique requires a set of already marked assignments for training, as it learns characteristics of each grade using those samples. therefore, the successful implementation of discriminant analysis will depend on the availability of training samples and how each grade can be clearly distinguished based on distinct characteristics of the assignments. integrating the above techniques will be beneficial for a more comprehensive analysis of the assignments and will be considered for future work on the project. 7 conclusion the results presented in this paper highlight opportunities for academics, and in particular subject coordinators of large classes, to undertake automated analysis of assignments in order to obtain greater understanding of student performance in major pieces of assessment against marking rubrics using tools from learning analytics. work on the project is continuing with the preparation of software to automate much of the feedback process, and to present data in a user-friendly dashboard for subject coordinators that can also be used to provide feedback to students on individual performance in their assignments. this would include the opportunity for subject coordinators to drill down into the data to identify students at-risk. further work to refine the accuracy of the tool could include the use of additional measures, including behavioural measures. acknowledgments the authors gratefully acknowledge support from a digital learning strategy innovation grant provided through la trobe learning and teaching at la trobe university, melbourne. they would also like to acknowledge the assistance of james heath in research assistance associated with this work. references 1st international conference on learning analytics and knowledge. (2010). lak '11: 1st international conference on learning analytics and knowledge 2011. retrieved from https://tekri.athabascau.ca/analytics/ bird, s., klein, e., & loper, e. (2009). natural language processing with python: analyzing text with the natural language toolkit: o'reilly media, inc. blei, d. m., jordan, m. i., griffiths, t. l., & tenenbaum, j. b. (2003). hierarchical topic models and the nested chinese restaurant process. paper presented at the proceedings of the 16th international conference on neural information processing systems, whistler, british columbia, canada. australasian journal of information systems nathawitharana, huang, ong et al 2017, vol 21, selected papers from ausdm towards next generation rubrics 18 blei, d. m., ng, a. y., & jordan, m. i. (2003). latent dirichlet allocation. journal of machine learning research, 3(jan), 993-1022. retrieved from http://www.jmlr.org/papers/v3/ chowdhury, g. g. (2003). natural language processing. annual review of information science and technology, 37(1), 51-89. doi:10.1002/aris.1440370103 coh-metrix. (n.d.). coh-metrix. retrieved from http://cohmetrix.com/ dawson, p. (2017). assessment rubrics: towards clearer and more replicable design, research and practice. assessment & evaluation in higher education, 42(3), 347-360. doi:10.1080/02602938.2015.1111294 deane, p. (2013). on the relation between automated essay scoring and modern views of the writing construct. assessing writing, 18(1), 7-24. doi:https://doi.org/10.1016/j.asw.2012.10.002 educational data mining. (2017). retrieved from international educational data mining society website: http://www.educationaldatamining.org/ elasticsearch. (2017). kibana. retrieved from https://www.elastic.co/products/kibana ets®. (2017a). criterion®. retrieved from https://www.ets.org/criterion ets®. (2017b). toefl: for academic institutions: scores. retrieved from https://www.ets.org/toefl/institutions/scores flesch, r. (1948). a new readability yardstick. journal of applied psychology, 32(3), 221-233. doi:http://dx.doi.org/10.1037/h0057532 frantzi, k., ananiadou, s., & mima, h. (2000). automatic recognition of multi-word terms:. the c-value/nc-value method. international journal on digital libraries, 3(2), 115130. doi:10.1007/s007999900023 kincaid, j. p., fishburne jr, r. p., rogers, r. l., & chissom, b. s. (1975). derivation of new readability formulas (automated readability index, fog count and flesch reading ease formula) for navy enlisted personnel (no. rbr-8-75). retrieved from defense technical information center website: http://www.dtic.mil/get-tr-doc/pdf?ad=ada006655 languagetool. (2017). languagetool style and grammar check. retrieved from https://www.languagetool.org/ li, w., & mccallum, a. (2006). pachinko allocation: dag-structured mixture models of topic correlations. proceedings of the 23rd international conference on machine learning, 577-584. doi:10.1145/1143844.1143917 liddy, e. d. (2001). natural language processing. in m. drake (ed.), encyclopedia of library and information science (2nd ed., pp. 2126-2136). new york, ny: marcel decker, inc. manning, c. d., surdeanu, m., bauer, j., finkel, j., bethard, s. j., & mcclosky, d. (2014). the stanford corenlp natural language processing toolkit. proceedings of 52nd annual meeting of the association for computational linguistics: system demonstrations, 5560. doi:10.3115/v1/p14-5010 mccallum, a. k. (2002). mallet: a machine learning for language toolkit. retrieved from http://mallet.cs.umass.edu mcnamara, d. s., crossley, s. a., roscoe, r. d., allen, l. k., & dai, j. (2015). a hierarchical classification approach to automated essay scoring. assessing writing, 23(supplement c), 35-59. doi:https://doi.org/10.1016/j.asw.2014.09.002 microsoft developer network. (2017a). document.computestatistics method (wdstatistic, object) (microsoft.office.tools.word). retrieved from https://msdn.microsoft.com/enus/library/microsoft.office.tools.word.document.computestatistics.aspx australasian journal of information systems nathawitharana, huang, ong et al 2017, vol 21, selected papers from ausdm towards next generation rubrics 19 microsoft developer network. (2017b). proofreadingerrors interface (microsoft.office.interop.word). retrieved from https://msdn.microsoft.com/enus/library/microsoft.office.interop.word.proofreadingerrors.aspx miller, g. a. (1995). wordnet: a lexical database for english. communications of the acm, 38(11), 39-41. doi:10.1145/219717.219748 oracle. (2017). oracle technology network for java developers | oracle technology network | oracle. retrieved from http://www.oracle.com/technetwork/java/index.html page, e. b. (1966). the imminence of... grading essays by computer. the phi delta kappan, 47(5), 238-243. retrieved from http://www.jstor.org/stable/20371545 paul, r. (1995). critical thinking: how to prepare students for a rapidly changing world: foundation for critical thinking. pearson. (2012). objective fact sheet. retrieved from https://pearsonpte.com/wpcontent/uploads/2014/07/objectivefactsheet.pdf pearson. (2017). writetolearntm. retrieved from https://www.pearsonassessments.com/products/100000030/writetolearn.html#tabdetails popham, w. j. (1997). what's wrong-and what's right-with rubrics. educational leadership, 55(2), 72-75. retrieved from http://www.ascd.org/publications/educationalleadership/oct97/vol55/num02/what's-wrong%e2%80%94and-what'sright%e2%80%94with-rubrics.aspx porter, m. f. (1980). an algorithm for suffix stripping. program-automated library and information systems, 14(3), 130-137. doi:10.1108/eb046814 sas. (2016). data mining software, model development and deployment, sas enterprise miner | sas. retrieved from https://www.sas.com/en_us/software/enterpriseminer.html shermis, m. d., & burstein, j. (2013). handbook of automated essay evaluation: current applications and new directions. new york, ny: routledge. shermis, m. d., & burstein, j. c. (2003). automated essay scoring: a cross-disciplinary perspective. mahwah, new jersey: lawrence erlbaum associates, inc. siemens, g., & baker, r. s. j. d. (2012). learning analytics and educational data mining: towards communication and collaboration. proceedings of the 2nd international conference on learning analytics and knowledge, 252-254. doi:10.1145/2330601.2330661 turnitin. (2017). turnitin what we offer : revision assistant. retrieved from http://www.turnitin.com/en_us/what-we-offer/revision-assistant vantage learning. (2016). my access!® writing and assessment solution | vantage learning. retrieved from http://www.vantagelearning.com/products/my-access-school-edition/ vitartas, p., ahmed, t., alahakoon, d., midford, s., nathawitharana, n., ong, k. l., & sullivanmort, g. (2016, 5-7 december). using learning analytics to guide learning: an analysis of marketing assignments. paper presented at the proceedings of the australian and new zealand marketing academy conference (anzmac), christchurch, new zealand. wagner, b. (2015). introduction to the c# language and the .net framework. retrieved from https://docs.microsoft.com/en-us/dotnet/csharp/getting-started/introduction-to-thecsharp-language-and-the-net-framework wei, t., lu, y., chang, h., zhou, q., & bao, x. (2015). a semantic approach for text clustering using wordnet and lexical chains. expert systems with applications, 42(4), 2264-2275. doi:https://doi.org/10.1016/j.eswa.2014.10.023 australasian journal of information systems nathawitharana, huang, ong et al 2017, vol 21, selected papers from ausdm towards next generation rubrics 20 white, b., & larusson, j. a. (2010). detecting the "point of originality" in student writing. proceedings of the 6th nordic conference on human-computer interaction: extending boundaries, 817-820. doi:10.1145/1868914.1869037 williamson, d. m., xi, x., & breyer, f. j. (2012). a framework for evaluation and use of automated scoring. educational measurement: issues and practice, 31(1), 2-13. doi:10.1111/j.1745-3992.2011.00223.x writelab. (n.d.). writelab homepage. retrieved from http://www.writelab.com copyright: © 2017 nathawitharana, huang, ong, vitartas, jayaratne, alahakoon, midford, michalewicz, sullivan mort, ahmed. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems hacker, wickramasinghe & durst 2017, vol 21, research on health information systems health 2.0 & obesity 1 can health 2.0 address critical healthcare challenges? insights from the case of how online social networks can assist in combatting the obesity epidemic janine hacker universität erlangen-nürnberg janine.hacker@fau.de nilmini wickramasinghe epworth healthcare & deakin university carolin durst universität erlangen-nürnberg abstract one of the serious concerns in healthcare in this 21st century is obesity. while the causes of obesity are multifaceted, social networks have been identified as one of the most important dimensions of people's social environment that may influence the adoption of many behaviours, including health-promoting behaviours. in this article, we examine the possibility of harnessing the appeal of online social networks to address the obesity epidemic currently plaguing society. specifically, a design science research methodology is adopted to design, implement and test the health 2.0 application called “calorie cruncher”. the application is designed specifically to explore the influence of online social networks on individual’s health-related behaviour. in this regard, pilot data collected based on qualitative interviews indicate that online social networks may influence healthrelated behaviours in several ways. firstly, they can influence people’s norms and value system that have an impact on their health-related behaviours. secondly, social control and pressure of social connections may also shape health-related behaviours, and operate implicitly when people make food selection decisions. thirdly, social relationships may provide emotional support. our study has implications for research and practice. from a theoretical perspective, the article inductively identifies three factors that influence specific types of health outcomes in the context of obesity. from a practical perspective, the study underscores the benefits of adopting a design science methodology to design and implement a technology solution for a healthcare issue as well as the key role for online social media to assist with health and wellness management and maintenance. keywords: online social networks; health 2.0 application; design science methodology; obesity. 1 introduction a central aspect of participatory medicine is an empowered patient who understands critical issues of their underlying health issue (lejbkowicz, caspi, & miller, 2012). the following examines critical issues concerning patient empowerment in the context of obesity. the article proffers a technology solution to enable both heightened patient empowerment and understanding by the patient of the key dynamics in this context. one of the serious concerns in healthcare in this 21st century is obesity (who, 2000). today, there exists more than one billion overweight adults out of which at least 300 million are medically obese (who, 2011b). not only is obesity itself problematic but it also leads to various secondary disorders such as diabetes, heart diseases and cancer, (who, 2011b), which in turn adds burdens to the challenged healthcare delivery systems, especially in terms of costs caused by obesity and the likelihood of recovery (wickramasinghe, bali, goldberg, & troshani, 2012). australasian journal of information systems hacker, wickramasinghe & durst 2017, vol 21, research on health information systems health 2.0 & obesity 2 the change of an individual’s behaviour is the key to successful primary prevention of obesity. traditional measures are established by human-to-human communication (e.g., group therapies, counselling sessions) or mediated-communication (e.g., media campaigns). a wellestablished human-to-human measure is personal fitness coaching. but as new technologies are arising, there are new possibilities through web, mobile, and other ambient technologies to establish human-computer interaction (hci) to foster a healthier lifestyle (oinas-kukkonen & harjumaa, 2009). mobile devices, like smartphones, facilitate an easy and fast way to communicate (oulasvirta, rattenbury, ma, & raita, 2012) by creating, accessing, and sharing information (oinaskukkonen, 2010). in the healthcare context, such services can connect individuals with similar interests, for example, via online communities. providing social support (hwang et al., 2010) and informational support, online health communities allow people to seek information, communicate with others with the same or similar diseases, share health guidance, and compare treatment and medication strategies. the ubiquitous accessibility at any time, the anonymity of the medium, as well as the access to greater expertise are regarded as the main benefits of online health communities (maloney-krichmar & preece, 2002). interestingly, social networks have also been identified as one of the most important dimensions of people's social environment that may influence the adoption of many behaviours, including health-promoting behaviours (mcneill, kreuter, & subramanian, 2006). previous studies have focused on the spread of obesity within traditional (offline) social networks (christakis & fowler, 2007; fowler & christakis, 2008). as ideas, behaviours and trends are passed on within people's social environment, a person-to-person spread of obesity within social networks has been suggested (christakis & fowler, 2007). lately, the dramatic growth of electronic (online) social networks has resulted in a blurring of the boundaries between the real and the virtual world (bvdw, 2010). online social networks have become part of many people's everyday life and are – just as “real world contacts” – suggested to influence the diffusion and adoption of health-related behaviours (ma, chen, & xiao, 2010). it is the ultimate goal of this research to address obesity by reinforcing positive health-related behaviours with innovative health 2.0 solutions. such interventions require a better understanding of the relationship between an individual’s online social network and their health-related behaviours. building upon our prior work on this topic (durst, viol, & wickramasinghe, 2013; wickramasinghe, teoh, durst, & viol, 2013), this article explores the relationship between online social networks, health-related behaviours, and an individual’s body weight based on qualitative interviews. embedded within a design science methodology, the interviews are used to derive design implications for developing a health 2.0 facebook application, which collects data in order to investigate the aforementioned relationship. moreover, the interviews enable us to extend our previously developed conceptual model (durst et al., 2013). the remainder of this article is organised as follows. after elaborating on obesity, social networks, and health, we review current work regarding obesity in the context of health 2.0. we then present our conceptual model and develop a health 2.0 facebook application using design science research methodology. finally, we discuss our findings and the limitations of the study. 2 background 2.1 classification the most common measure to classify underweight, overweight and obesity in adults is the body-mass-index (bmi), developed by the belgian mathematician adolphe quetelet in 1871 (quetelet, 1871). the bmi is defined as an individual's body weight in kilograms divided by the square of his or her height in meters (kg/m²): australasian journal of information systems hacker, wickramasinghe & durst 2017, vol 21, research on health information systems health 2.0 & obesity 3 𝐵𝑀𝐼 = 𝑚𝑎𝑠𝑠 (𝑘𝑔) (ℎ𝑒𝑖𝑔ℎ𝑡 (𝑚))² a person with a bmi of less than 18.5 is considered underweight (who, 2011b). a bmi greater than 25 is regarded as overweight and above 30 is regarded as obese (who, 2011a). 2.2 causes of overweight and obesity the causes of overweight and obesity are multifaceted and result from the co-occurrence and interaction of several factors (who, 2000, 2011a, 2011b). individual medical conditions may lead to obesity; yet, the dramatic increase in obese people worldwide cannot be explained by medical conditions alone (who, 2000). the who identifies an energy imbalance between consumed and expended calories as the fundamental cause of obesity (who, 2011a, 2011b). this energy imbalance is determined by individual behaviours and lifestyle choices, such as dietary and physical activity patterns (huffman, 2011; who, 2011a, 2011b). in turn, individual behaviours and lifestyle choices are shaped by people's social and economic environment. thus, in this study we focus on social factors influencing an individual’s propensity to engage in a healthy lifestyle. 2.3 social networks, social capital, and health mcneill et al identify social networks as one of the most important dimension of people's social environment to influence the adoption of health-promoting behaviours (mcneill et al., 2006). as such, social networks consist of a finite set of actors or people and the relationships between them (wasserman & faust, 1994). drawing on their social networks, people have access to different kinds of resources forming their social capital. social capital theory is rooted in the social sciences (hyyppä, 2010) and has been used to explain a wide range of societal phenomena. for instance, marmot & wilkinson (2005) study social capital as a determinant of public health in medical sociology. while different notions of social capital exist (cf. rostila, 2011), this study is grounded in the networkbased notion of social capital. thus, social capital is defined as the resources that are embedded in one’s social network and that can be accessed or mobilised through relationships in the network (lin, 2001, p29). commonly, these relationships are categorised into strong and weak ties. strong ties are intimate bonds between family members or close friends that are maintained regularly and permanently (granovetter, 1973). tending to be concentrated in particular groups, strong ties are of informal nature and occur between network members with a shared social identity (ibid). strong ties can be related to bonding social capital (putnam, leonardi & nanetti, 1994). in contrast, weak ties emerge as non-intimate bonds between acquaintances (granovetter, 1973). maintained infrequently and inconsistently, weak ties may be formal contacts and are more likely to link members of different small groups (rostila, 2011). weak ties tend to generate informational support whereas informal and strong ties are associated with the provision of emotional support (ibid). weak ties overlap with the idea of bridging social capital (putnam et al., 1994). regarding epidemic diseases in social networks, gershenson (2011) distinguishes between communicable diseases transmitted through e.g. bacteria and viruses, such as influenza and hiv, and non-communicable diseases, e.g. cancer and diabetes, that can – by definition – not be transmitted (gershenson, 2011). however, he acknowledges, the spread of ideas and behaviours as well as contagious trends and habits provides a powerful substitute for physical mechanisms to spread diseases and leads to a so-called "social infection" (ibid). thus, the risk factors for non-communicable diseases are spread across social networks. in the context of this research, the association between social networks, social capital, health-related behaviours and obesity is of particular importance. having conducted a longitudinal study over 32 years, christakis and fowler provide three explanations for the collective dynamics of obesity in a large social network (christakis & fowler, 2007). further, they argue that the susceptibility of a person to become obese increases when close members in the social network become obese. thus, the chance of becoming obese is positively correlated with the closeness and strength of australasian journal of information systems hacker, wickramasinghe & durst 2017, vol 21, research on health information systems health 2.0 & obesity 4 the relationship. though the design and findings of their study have been criticised (e.g. in cohen-cole & fletcher, 2008), a number of papers support their line of argument. for instance, madan et al. found that health-related behaviours of peers correlate (madan, moturu, lazer, & pentland, 2010). moreover, a literature review by hammond (2010) examining the relationship between social influence and obesity points to social influence and social network structures being important factors in obesity. finally, moore, daniel, gauvin & dubé (2009) found a relationship between social capital and likelihood of overweight and obesity. 2.4 obesity in the context of health 2.0 social media and web 2.0 technologies are nowadays applied in various areas. as to healthcare, the use of web 2.0 technologies in the context of this research is referred to as health 2.0, which is defined as the "combination of health data and health information with (patient) experience through the use of ict" (bos, marsh, carroll, gupta, & rees, 2008) or "user generated healthcare" (bos, carroll, & marsh 2008). embracing patient-centred health informatics, health 2.0 applications may become an effective self-care information and disease self-management tool in the future. for instance, online health communities allow people to seek information, communicate with others with the same or similar problems, to share health guidance and compare treatment and medication strategies. the accessibility from everywhere at any time, anonymity of the medium as well as the access to greater expertise are regarded as the main benefits of online health communities (maloney-krichmar & preece, 2002). hwang et al. (2010) found an online weight loss community to be an important source of social support for its members. a number of social media applications deal with obesity. dedicated blogs, such as diet blog (http://www.diet-blog.com), aggregate diet news and advice and cover topics such as exercise, body image and weight loss success stories. moreover, health social network sites, e.g. sparkpeople (http://www.sparkpeople.com), enable users to connect with each other, to share information, and to support each other in reaching diet and exercise goals. besides health 2.0 applications on the internet, mobile healthcare applications implemented for smartphones are increasingly used for health promotion and disease prevention (durst, hamper, & müller, 2013). these applications educate users in terms of risks of specific behaviours, provide users with feedback through tracking and coaching mechanisms, enable social interaction with other users, and include gamification features to keep up user motivation. a great number of mobile healthcare applications focusing on physical activity use sensor technologies, such as gps sensors for position localisation and tracking, externally connected heart rate sensors for biofeedback or the built-in accelerometer for counting steps. nutrition-related applications primarily make use of the camera module for scanning nutritional fact labels. relatively few health 2.0 applications include social interaction and gamification features (for a comprehensive overview see durst, hamper, & müller (2013)). however, the interaction between a user and his/her social network may play a crucial role in shaping health-related behaviours as explained in the following section. 2.5 possible role for online social networks online social networks are applications that allow users to build a semi-public or public profile within a bounded system, to create explicit linkages to other users and to communicate by sharing information or sending messages between each other (arnaboldi, passarella, tesconi, & gazze, 2011). organised around people, online social networks are structured as egocentric networks within which the individual is at the centre of their personal network (kaplan & haenlein, 2010). online social networks exhibit a high similarity to offline social relationships, as they offer similar functionalities as unmediated spaces (arnaboldi et al., 2011). they display people's (extended) offline social network and are primarily used to maintain and reinforce existing offline relationships (boyd & ellison, 2007). australasian journal of information systems hacker, wickramasinghe & durst 2017, vol 21, research on health information systems health 2.0 & obesity 5 as individual behaviours and lifestyle choices are shaped by an individual’s social environment (who, 2011b), online social relationships might exert similar influences. a conceptual model developed in an earlier step of this research suggests a relationship between people’s online social network, social capital, health-related behaviours, and their bodyweight (durst et al., 2013). the conceptual model is based on social capital theory in general as well as on literature focusing on the relationship between social capital and health and the association between online social networks and social capital. in the course of developing the conceptual model (durst et al., 2013), we identified two main research gaps. 1. little is known about the types of relationships that matter for specific types of health outcomes in the context of obesity (moore, 2010). 2. the lack of longitudinal data, that is the cross-sectional nature of many studies, makes it difficult to establish causality in that it cannot be determined whether social networks are a consequence of good health or vice versa (hyyppä, 2010; kim, subramanian, & kawachi, 2008; moore, 2010). while social networks and social capital have been identified as important determinants of health outcomes, it is not quite clear how they shape health-related behaviours. thus, as shown in figure 1, there may be other factors (situated in the “black box”) that influence the relationship between social capital and health-related behaviours. figure 1. relationship between social networks, social capital, health-related behaviours and health hence, investigating the relationship between social capital and health-related behaviours, including dietary and physical activity patterns and body weight forms the focus of this research. in this article, we report on the design of a health 2.0 facebook application developed to support a (future) quantitative study investigating the aforementioned relationship. preceding this study, explorative interviews with 14 study participants were conducted to gain qualitative insights regarding the relationship between (online) social networks and health-related behaviours. the identified variables help refine the conceptual model and inform the development of features of the health 2.0 facebook application. 3 design science research methodology the design science research methodology (dsrm) process model is one of the most influential design science research methodologies in information system (peffers, tuunanen, rothenberger, & chatterjee, 2007). the model is consistent with the principles and guidelines of design science research (hevner, march, park, & ram, 2004), as explained below. table 1 shows the application of the dsrm to the facebook application calorie cruncher developed in this paper. black box offline online health-related behaviours social capital social networks bodyweight (bmi) australasian journal of information systems hacker, wickramasinghe & durst 2017, vol 21, research on health information systems health 2.0 & obesity 6 dsrm activities activity description application in this study problem identification and motivation identify and describe the importance of the problem data collection for the measurement of the influence of facebook friends on the individual's body weight, design implications for the health 2.0 facebook application definition of objectives for the application define the artefact that accomplishes the objectives design of an application that provides access to the users’ network of facebook friends and that gathers data on the users’ health-related behaviours design and development identify the requirement for the solution and create the artefact design of the calorie cruncher facebook application based on 14 qualitative interviews demonstration use artefact to solve problem proof of concept of the calorie cruncher facebook application evaluation iterate back to the design of the artefact identification of areas to improve the calorie cruncher application communication publish and communicate the results conference publication to communicate and discuss the results in order to develop this project further table 1. design science research methodology (dsrm) adopted from (peffers et al., 2007) 3.1 problem identification and motivation the motivation of this study is to address an important gap in literature, that is, to measure the influence of different online relationships on the individual's body weight (moore, 2010). as a first step, this study requires a large dataset containing personal online social network data, as well as information on an individual’s health-related behaviours. thus, we chose to use data from the most popular social network site facebook by designing a health 2.0 facebook application to collect the required data that would lead to higher data reliability and a more efficient data collection process than the use of self-reported survey data (mccarty, 2002; vitak, ellison, & steinfield, 2011). 3.2 definition of objectives for the application we designed the application according to the steps of the dsrm. the solution used for the data collection needs to fulfil the following objectives: 1. provide access to user profiles on facebook 2. provide access to the users’ network of friends (full personal networks) 3. provide access to the users’ interaction with their facebook friends 4. gather data on the users’ health status and health-related behaviours an application in the area of fitness and sports should facilitate the collection of longitudinal data to facilitate a better understanding of people’s behaviour over the course of time and thus enable the detection of causal inferences. 3.3 application design and development the design and development requirements are through qualitative interviews to determine the artefact’s desired functionality and its architecture. by exploring the influence of people's offline social networks on their health-related behaviours and body weight, the findings of the interviews will inform the design and the development of the application's features. translating the influencing social factors into application features also supports the australasian journal of information systems hacker, wickramasinghe & durst 2017, vol 21, research on health information systems health 2.0 & obesity 7 continuous engagement with the application on the long run and enables us to collect longitudinal data. 3.4 design sampling using a purposeful sampling (flick, 2009), the interviewees were selected based on currently experiencing problems with overweight and obesity or having experienced weight issues in the past (table 2). interviewee age sex occupation marital status weight status* 1 29 f student, journalist single obese 2 26 f dental assistant single overweight 3 28 f dental assistant engaged overweight 4 26 f works in the public sector in a relationship normal 5 26 f sales assistant, works freelance as an educationalist single overweight 6 21 f management assistant in office communication single obese 7 22 f student, waitress single normal 8 24 m hairdresser single normal 9 24 m media designer single overweight 10 39 m real estate agent married obese 11 23 m it technician single normal 12 24 m student single overweight 13 39 m senior manager married obese 14 35 m senior manager single overweight *according to the bmi classification of (who, 2011a) a person with a bmi of less than 18.5 is considered underweight, greater than 25 is regarded as overweight and above 30 is regarded as obese table 2. overview of the interviewees all selected interviewees were required to either currently engage in activities to lose weight or to have substantially reduced their body weight in the past. we included people from different stages of the weight loss process, since the study participants needed to be aware of their weight issues, and be able to reflect on the influence of their real-world social networks in different stages of the previous weight gain and current weight loss process. being a member of facebook was a third requirement. 3.5 design data collection the interview guideline included six sections addressing different areas of the process of weight gain and weight loss. dealing with the very delicate and personal topic of obesity, data was collected through semi-structured open ended interviews (hopf, 2007). as such, the interviews left room for spontaneous questions and allowed for in-depth exploration of the relationship between the interviewees' personal traits, social environment and their healthrelated behaviours. direct interaction made it possible to control the flow, to avoid misunderstandings, to shift the interview's focus to particularly interesting aspects or to australasian journal of information systems hacker, wickramasinghe & durst 2017, vol 21, research on health information systems health 2.0 & obesity 8 neglect certain areas of interrogation if the interviewee did not seem comfortable. open-ended questions were advantageous as the interviewees had to provide answers based on their personal experiences (flick, 2009). 3.6 design data analysis the data coding was conducted using nvivo, with inductive category building of free nodes categorizing the environment, reasons for weight gain and motivation to lose weight (mayring, 2000). following this process resulted in 154 free nodes. an example for a free node coding in “comparison with others” is “my friends do sports, too, and that motivates me to engage in physical activity as well (…)” [interviewee 9]. subsequently, the free nodes were classified into categories and organised into tree nodes. table 3 shows a sample of tree node coding. redundancies were checked for and eliminated, resulting in a final number of 131 nodes. the analysis of interview data followed approaches recommended by boyatzis (1998), kvale (1996) and yin (2009). tree nodes references social pressure slim friends 8 bad conscience 1 comparison with others 21 competition with others 4 common weight loss 5 sports together 8 table 3. example tree node coding 3.7 design results based on reports on offline social networks, online social networks may influence healthrelated behaviours in several ways. firstly, social networks are suggested to influence people’s norms and value system that have an impact on their health-related behaviours. the family social environment was mentioned as an important determinant of norms in terms of eating behaviours [interviewee 10]. secondly, social control and pressure of social connections may also shape health-related behaviours, and operate implicitly when people make food selection decisions [interviewee 7]. “in the gym, i met many sportive and body-conscious people who have now become my friends. even if i don't aim for a six pack, which some of my friends have, it does motivate me.” thirdly, social relationships may provide emotional support (rostila, 2011) for individual health-related behaviours, and interviewees’ success and progress in the weight loss process made them feel more attractive, admired and socially accepted. however, a negative influence on people’s emotional state is best illustrated by the fact that people engage in eating as a redirection activity due to experiencing emotional distress [interviewee 7]. further, a feeling of loneliness resulting from a lack of friends was mentioned as a reason for less physical activity on the one hand [interviewee 9] and comfort in eating on the other hand [interviewee 2]. considering these findings, an award system as well as features enabling the comparison with other users were incorporated in the facebook application. 3.8 design implications to facilitate a better understanding of the relationship between people’s social networks and their health-related behaviours, the three factors (1) value system, (2) social pressure and control and (3) emotional support were included as intervening variables in the presented model (figure 3). table 4 provides an overview of the main findings of the qualitative australasian journal of information systems hacker, wickramasinghe & durst 2017, vol 21, research on health information systems health 2.0 & obesity 9 interviews according to the three factors with the derived design implication for the facebook application. intervening factors summary of findings design implications value system  social networks are an important determinant of norms in terms of healthrelated behaviours  social networks are a source of informational support  enabling the comparison with others, social networks influence an individual's norms and values  facilitate documentation of sports activities  provide information such as nutritional information or information of calorie expenditure  include comparison features social pressure and control  social networks influence health-related behaviours by providing role models  social networks exert social control and create social pressure through competition with others  enable monitoring of friends’ sports activities  include competition features emotional support  social networks are a source of emotional support (e.g. recognition)  a lack of social contacts negatively effects health-related behaviours  provide a social network “on the go”  allow users to obtain feedback from their online social network table 4. summary of the interview findings and design implications 3.9 demonstration the beta version of the calorie cruncher application launched in october 2012 and counts 30 active users as of today. 3.10 data collection the calorie cruncher successfully fulfilled the four main objectives by collecting online social network data, including (1) users’ profile information, (2) users’ network of friends and (3) users’ interaction data, as well as (4) users’ health data. australasian journal of information systems hacker, wickramasinghe & durst 2017, vol 21, research on health information systems health 2.0 & obesity 10 online social network data health data users’ profile information users’ network of friends users’ interaction data  user id  gender  birthday  work and education history  hometown  current location  activities and interests  friends’ user ids  friendship status between the user’s friends  status updates  wall posts  check-ins  picture ids  comments and likes on the previously mentioned interactions  picture tags  body height  body weight  activity level  health-related behaviour table 5. data collected via the calorie cruncher application all facebook ids are made anonymous to protect the privacy of the participants. with regard to the users’ profiles, we collected information on gender, age, work and education history, hometown, their current location and their activities and interests – depending on the completeness of the facebook profile information. the access to data from friends’ networks enabled us to identify full personal networks, which facilitate the calculation of network-based measures like density or the identification of cliques within an individual facebook network. for users’ interaction data, the solution collects information on status updates, wall posts, check-ins and picture ids and the respective facebook likes and comments on those interactions. additionally, we have access to picture tags to identify if one or more facebook users are in the same picture. with respect to the health data, calorie cruncher gathered data on the user’s health status, tracks activities and monitors the body weight over time. 3.11 implementation of design implications calorie cruncher was developed according to findings from the qualitative study. users of this application can “crunch” (burn) calories or take in calories, as well as share these activities with their friends. additionally, we implemented the design implications based on the identified intervening factors (see figure 2), to ensure frequent use and spread of the application on facebook in order to monitor health data and online social network data over time. calorie cruncher works like a diary that documents all activities (value system [1]). by providing statistics on the user’s activities and bmi measures, the application enables the monitoring of the user’s activities over time and, provides statistics on the activities of friends (value system [2]). on the landing page (screen 1) of the calorie cruncher, the application shows a health radar to indicate the current health status of the user (value system [3]). australasian journal of information systems hacker, wickramasinghe & durst 2017, vol 21, research on health information systems health 2.0 & obesity 11 figure 2. screenshots of the calorie cruncher application with regard to emotional support, we included features fostering the socialising aspect (yee, 2006) to provide users with a sports club network “on the go”. users can document activities performed with other facebook friends, who do not need to be users of calorie cruncher. moreover, every calorie cruncher user can like – and in this manner support – the activities a user performed (emotional support). all activities can be shared on the facebook wall, to allow user’s facebook friends to like and comment on those activities. hence, a connection between calorie cruncher activities and the facebook wall can be established and other facebook members may be encouraged to start using the application. 3.12 evaluation the facebook application calorie cruncher successfully collected all the required online social network data and the users’ health data. the challenge lies in the cleaning, preparation, aggregation and management of the data in order to use them for further analysis, and in the identification of meaningful relations. most users only communicate with a very small and distinct number of people out of their total number of online friends (byron, lento, marlow, & rosenn, 2009), and these friends are expected to exert a higher influence on the individual, and they are identified by the frequency of interaction (ellison, steinfield, & lampe, 2007). in this case interaction measures have to be developed based on the raw data set. with regard to the design of the calorie cruncher application, we are still in the pilot stage, since as of today we only have 30 active users. more promotion is planned in order to boost the user base. the application testing was successful, but we still need to conduct user acceptance testing. 4 discussion and conclusion finding innovative, effective and efficient ways to enable health and wellness to ensue is indeed a global priority and a huge societal challenge today. the preceding has served to outline a role for an innovative use of technology in addressing the current complex and pressing issue of the obesity epidemic. specific contributions to theory and practice are identified below. 4.1 theoretical implications this study has successfully refined our conceptual model (figure 3) based on the findings of the qualitative interviews. australasian journal of information systems hacker, wickramasinghe & durst 2017, vol 21, research on health information systems health 2.0 & obesity 12 figure 3. revised relationship between social networks, health-related behaviours and health we have inductively identified three factors that influence specific types of health outcomes in the context of obesity (moore, 2010): (1) value system, (2) social pressure and control and (3) emotional support. by carefully examining the extant literature and then designing and developing a technology solution together with user input that focuses on empowering and supporting individuals, our study contributes to existing literature. building on the aboveidentified three factors has enabled this study to enter into the black box, that is, to shed light on the relationship between online social networks and health-related behaviours (figure 3). in particular, the extended conceptual model has the potential to establish causality between online social networks and good health, which has been found difficult to achieve to date (hyyppä, 2010; kim et al., 2008; moore, 2010). it is important to note that healthcare delivery is changing (wickramasinghe & schaffer, 2010), and is influenced by rising healthcare costs, advances in technology, growth in chronic disease such as diabetes and obesity, and patient expectations and desires to be more empowered (eysenbach, 2000; wickramasinghe & schaffer, 2010). eysenbach shows that individuals are desirous to be more empowered and supported in their own health and well-being maintenance (figure 4) (eysenbach, 2000). in particular, he asserts that this has contributed significantly to the rise of the emerging and nascent domain of consumer health informatics (ibid). hence, what we have is an evolution in the practice of medicine with an emphasis on prevention and self-help which necessitates an engaged, empowered and informed patient. this can only be realised if ict solutions are innovative or applied in an innovative fashion (as is the case with calorie cruncher). calorie cruncher facilitates the collection of user profile data, interaction data, and health-related data. as such, it enables the application of big data analytics in the context of obesity and online social networks, e.g. to analyse the relationship between a user’s interaction patterns, friendship network, and their health status. we contend therefore, that online social networks will have a growing and significant role to play in the delivery of effective, efficient high value healthcare in today’s information age. further, our study is perhaps one of the first to theoretically identify the underlying reasons for the importance of online social networks to support and facilitate superior healthcare delivery. black box offline online health-related behaviours social capital social networks bodyweight (bmi) value system emotional support social control and pressure australasian journal of information systems hacker, wickramasinghe & durst 2017, vol 21, research on health information systems health 2.0 & obesity 13 hospital medicine industrial age healthcare cyber medicine information age healthcare telemedicine ambulatory medicine prevention/self help self care assisted care & empowerment assisted care & disease management consumer health informatics provider oriented medical informatics traditional new & emergent pa rt ici pa to ry m ed ici ne clinical m edicine figure 4. the changing face of healthcare delivery (adapted from: eysenbach, 2000) 4.2 practical implications lessons learned from designing this health 2.0 facebook application would enable medical and healthcare practitioners to (1) understand the effect of obesity propagation in online social networks; (2) identify potential online intervention strategies to disseminate health-related information to the right group of people and (3) to offer services fostering positive healthrelated behaviours or to promote behaviour change advice. by exploring the social aspects of obesity and their effect in online social networks, this research will help to facilitate appropriate health-related behaviours using innovative consumer healthcare technologies, and will serve to inform the development of possible online intervention strategies. further, for patients to be truly empowered to address key drivers impacting their chronic disease they need to understand and identify factors that influence and support healthy lifestyles. the proffered solution addresses this and thus supports heightened patient empowerment. in so doing, it also provides an example of how current technologies can support and facilitate the fostering of participatory medicine. 4.3 limitations and future research the application designed in this exploratory study is based on interview data gathered from a relatively small number of participants. however, after the pilot study and demonstration, we were able to collect sufficient data to enable deeper analysis. to date, modern living and or more sedentary life style typically has been blamed for the rise in many chronic diseases such as obesity and diabetes. kling noted that technology has both positive and negative impacts (kling, 1996). to date however, as connected with chronic disease, we too often only hear the negative aspects or consequences of technology. we believe our research has served to underscore in a systematic fashion how technology might be able to assist in a significant, positive way to address a key global challenge and complex societal issue of addressing many chronic disease epidemics such as obesity and diabetes. thus, the implications of this study are important, significant and far-reaching. acknowledgements all appropriate ethics clearances have been received by the respective universities of the coauthors. a study of this type challenges aspects of privacy and the ability to withdraw as the facebook technology used moves some of the control outside the researchers’ immediate control. the respective ethics committees were satisfied that the nature of the study was fully explained and as individuals recruited were facebook users they were familiar with facebook. further, identifiable data was only made available by the individuals themselves to those friends they chose to share the data; i.e. the researchers did not share any data to others except australasian journal of information systems hacker, wickramasinghe & durst 2017, vol 21, research on health information systems health 2.0 & obesity 14 as aggregated de-identified data. individuals were free to withdraw at any time. their own facebook accounts were their accounts to manage as they desired so this was never impacted or impeded by the study. the authors acknowledge funding received from atn daad joint research co-operation scheme 1 january 2012 31 december 2013 which assisted them in conducting their research. references arnaboldi, v., passarella, a., tesconi, m., & gazzè, d. (2011). towards a characterization of egocentric networks in online social networks. in r. meersman, t. dillon, & p. herrero (eds.), on the move to meaningful internet systems: otm 2011 workshops (pp. 524– 533). crete: springer berlin heidelberg. doi: 10.1007/978-3-642-25126-9_64 bos, l., carroll, d., & marsh , a. (2008). the impatient patient. studies in health technology and informatics, 137, 1-13. bos, l., marsh, a., carroll, d., gupta, s., & rees, m. (2008). patient 2.0 empowerment. paper presented at the swws. boyatzis, r. (1998). thematic analysis and code development: transforming qualitative information. london: sage. boyd, d. m., & ellison, n. (2007). social network sites: definition, history, and scholarship. journal of computer-mediated communication, 13, 210-230. doi: 10.1111/j.10836101.2007.00393.x brewer, j. & hunter, a. (1989). multimethod research. a synthesis of styles. newbury park: sage. bruderl, j. & preisendorfer, p. (1998). network support and the success of newly founded businesses. small business economics, 19, 213-225. bvdw. (2010). social media kompass 2010/2011. düsseldorf: fachgruppe social media im bundesverband digitale wirtschaft (bvdw) e.v. byron, l., lento, t., marlow, c., & rosenn, i. (2009). maintained relationships on facebook. available at: https://www.facebook.com/notes/facebook-data-science/maintainedrelationships-on-facebook/55257228858/ [accessed 29 january 2017] christakis, n. a., & fowler, j. h. (2007). the spread of obesity in a large social network over 32 years. the new england journal of medicine, 357, 370-379. doi: 10.1056/nejmsa066082 cohen-cole, e., & fletcher, j. m. (2008). is obesity contagious? social networks vs. environmental factors in the obesity epidemic. journal of health economics, 27, 13821387. doi: 10.1016/j.jhealeco.2008.04.005 durst, c., hamper, a., & müller, t. (2013). in j. kandampully (ed.), service management in health and wellness. kendall hunt publishing. durst, c., viol, j., & wickramasinghe, n. (2013). online social networks, social capital and health-related behaviours: a state-of-the-art analysis. communications of the association for information systems, 32, 134–158. ellison, n., steinfield, c., & lampe, c. (2007). the benefits of facebook “friends:” social capital and college students’ use of online social network sites. journal of computermediated communication, 12, 1143-1168. doi: 10.1111/j.1083-6101.2007.00367.x eysenbach, g. (2000). recent advances: consumer health informatics. bmj: british medical journal, 320(7251), 1713. flick, u. (2009). an introduction to qualitative research (4th ed.). london: sage. australasian journal of information systems hacker, wickramasinghe & durst 2017, vol 21, research on health information systems health 2.0 & obesity 15 fowler, j. h., & christakis, n. a. (2008). estimating peer effects on health in social networks: a response to cohen-cole and fletcher; trogdon, nonnemaker, pais. journal of health economics, 27, 1400. doi: 10.1016/j.jhealeco.2008.07.001. gershenson, c. (2011). epidemiology and social networks. cir cir, 79(3), 99–200. granovetter, m. s. (1973). the strength of weak ties. the american journal of sociology, 78, 1360-1380. doi: 10.1037/a0018761 hammond, r. a. (2010). social influence and obesity. current opinion in endocrinology, diabetes, and obesity, 17, 467-471. hevner, a. r., march, s. t., park, j., & ram, s. (2004). design science in information systems research. mis quarterly, 28, 75-105. doi: 10.1.1.110.8133 hopf, c. (2007). qualitative interviews ein überblick. in u. flick, e. von kardoff, & i. steinke (eds.), qualitative forschung: ein handbuch (5th ed., pp. 349–360). reinbek bei hamburg: rowohlt taschenbuch verlag. huffman, s. k. (2011). bmi changes in russian adults: the role of lifestyles and spousal relationships. agricultural and applied economics association. available at: http://econpapers.repec.org/repec:ags:aaea11:102653 [accessed 29 january 2017] hwang, k. o., ottenbacher, a. j., green, a. p., cannon-diehl, m. r., richardson, o., bernstam, e. v., & thomas, e. j. (2010). social support in an internet weight loss community. international journal of medical informatics, 79(1), 5-13. hyyppä, m. t. (2010). healthy ties. media. doi: 10.1007/978-90-481-9606-7 kaplan, a. m., & haenlein, m. (2010). users of the world, unite! the challenges and opportunities of social media. business horizons, 53, 59-68. kim, d., subramanian, s. v, & kawachi, i. (2008). social capital and physical health. in i. kawachi, s. v subramanian, & d. kim (eds.), social capital and health (pp. 139–190). springer new york. doi: 10.1007/978-0-387-71311-3_8 kling r. (1996). computerization and controversy: value conflicts and social choices. san diego: academic press. kvale, s. (1996). interviews: an introduction to qualitative research interviewing. thousand oaks: sage publications. lejbkowicz, i., caspi, o., & miller, a. (2012). participatory medicine and patient empowerment towards personalized healthcare in multiple sclerosis. expert review of neurotherapeutics, 12(3), 343-352. lin, n. (2001). social capital: a theory of social structure and action. cambridge: cambridge university press. ma, x., chen, g., & xiao, j. (2010). analysis of an online health social network. ihi' 10. madan, a., moturu, s. t., lazer, d., & pentland, a. s. (2010). social sensing. wireless health 2010 on wh '10. new york, new york, usa: acm press. maloney-krichmar, d., & preece, j. (2002). the meaning of an online health community in the lives of its members: roles, relationships and group dynamics. paper presented at the 2002 international symposium on technology and society, 2002. (istas'02). marmot, m., & wilkinson, r. g. (eds.) (2005). social determinants of health (2nd ed.). oxford university press. mayring, p. (2000). qualitative inhaltsanalyse. forum qualitative sozialforschung / forum: qualitative social research, 1. mccarty, c. (2002). measuring structure in personal networks. journal of social structure, 3. australasian journal of information systems hacker, wickramasinghe & durst 2017, vol 21, research on health information systems health 2.0 & obesity 16 mcneill, l. h., kreuter, m. w., & subramanian, s. v. (2006). social environment and physical activity: a review of concepts and evidence. social science & medicine (1982), 63, 10111022. doi: 10.1016/j.socscimed.2006.03.012 moore, s., daniel, m., gauvin, l., & dubé, l. (2009). not all social capital is good capital. health & place, 15(4), 1071–7. doi:10.1016/j.healthplace.2009.05.005 moore, s. (2010). social networks, social capital and obesity: a literature review. in l. dubé, a. bechara, a. dagher, a. drewnowski, j. lebel, p. james, & r. y. yada (eds.), obesity prevention (pp. 673 – 685). san diego: academic press. oinas-kukkonen, h. (2010). behavior change support systems: a research model and agenda persuasive technology (pp. 4-14). berlin: springer-verlag. oinas-kukkonen, h., & harjumaa, m. (2009). persuasive systems design: key issues, process model, and system features. communications of the association for information systems, 24(1), 28. oulasvirta, a., rattenbury, t., ma, l., & raita, e. (2012). habits make smartphone use more pervasive. personal and ubiquitous computing, 16(1), 105-114. peffers, k., tuunanen, t., rothenberger, m. a., & chatterjee, s. (2007). a design science research methodology for information systems research. journal of management information systems, 24, 45-77. doi: 10.2753/mis0742-1222240302 putnam, r. d., leonardi, r., & nanetti, r. y. (1994). making democracy work: civic traditions in modern italy. new jersey: princeton university press. quetelet, l. a. j. (1871). anthropométrie ou mesure des différentes facultés de l'homme. bruxelles. rostila, m. (2011). a resource-based theory of social capital for health research: can it help us bridge the individual and collective facets of the concept? social theory & health, 1-21. doi: 10.1057/sth.2011.4 rostila, m. (2011). the facets of social capital. journal for the theory of social behaviour, 41(3), 308–326. doi:10.1111/j.1468-5914.2010.00454.x vitak, j., ellison, n., & steinfield, c. (2011). the ties that bond: re-examining the relationship between facebook use and bonding social capital. proceedings of the 44th hawaii international conference on system sciences. wasserman, s., & faust, k. (1994). social network analysis: methods and applications. cambridge: cambridge university press. who. (2000). obesity: preventing and managing the global epidemic. report of a who consultation. who technical report series 894, geneva. who. (2011a). bmi classification. available at: http://apps.who.int/bmi/index.jsp?intropage=intro_3.html [accessed 29 january 2017] who. (2011b). obesity and overweight. available at: http://www.who.int/mediacentre/factsheets/fs311/en/index.html [accessed 29 january 2017] wickramasinghe, n., & schaffer, j. (2010). realizing value driven e-health solutions. report for ibm. washington dc. wickramasinghe n., bali r., goldberg s., & troshani i. (2012). pervasive health knowledge management. new york. wickramasinghe, n., teoh, s. y., durst, c., & viol, j. (2013). insights from an investigation of the design of a consumer health 2.0 application to address the relationship australasian journal of information systems hacker, wickramasinghe & durst 2017, vol 21, research on health information systems health 2.0 & obesity 17 between on-line social networks and health-related behaviours. in proceedings of the 24th australasian conference on information systems. melbourne. yee, n. (2006). motivations for play in online games. cyberpsychology & behavior, 9, 772775. doi: 10.1089/cpb.2006.9.772 yin, r. k. (2009). case study research: design and methods (vol. 5). thousand oaks: sage. copyright: © 2017 hacker, wickramasinghe & durst. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems vella 2016, vol 20, post publication review students' computing use and study post publication review mclachlan, c., craig, a., & coldwell-neilson, j. (2016). students' computing use and study: when more is less. australasian journal of information systems, 20. doi:http://dx.doi.org/10.3127/ajis.v20i0.990 review in their analysis mclachlan, craig and coldwell-nielsen (2016) gave some insight into the complex issues around implementing information and communication technology (ict) in schools and the potential flow-on impact on our future workforce for suitably trained and skilled personnel. they begin with a fascinating review of the literature to show reasons for declining interests in ict careers, and focus on data from victoria to demonstrate their reasoning. the use of secondary data analysis is interesting. the paper presents a thorough and informative investigation. in their paper the authors re-presented publicly available secondary data and discussed on the one hand the decrease in enrolments in computing subject and courses (sections 4 and 8), and the negative influence on students to not enrol in ict subjects (section 5), as well as, on the other hand, state and national government influence on victorian schools (section 6), students being computer literate (section 7), the increasing use and availability of technology (section 9) and the generally positive attitude students had towards computers (section 10). the authors revealed that “the widespread use of computers has not resulted in an increase in students studying computing, nor even an increase in their proficiency with computers; rather it has had the reverse effect” (p 1). this is curious because similarly studies have shown that since the australian computer society made ethics teaching for ict courses compulsory for accreditation, graduates have actually decreased in their ability to engage in professional ethics reasoning, when compared with ict workers who have not had university training (al-saggaf, burmeister, & weckert, 2015; lucas & mason, 2008; simpson, nevile, & burmeister, 2003). the authors suggested, further research was needed to correct the misconceptions keeping students from studying technology and becoming trained and skilled labour in the future. in particular: “…stereotypical perceptions about computing, or that frequent use has provided students with an unrealistic perception of their computer abilities.” and “… the pervasiveness and ubiquity of computing technologies may be encouraging a view that there is nothing new or engaging behind the screens that is worthy of their attention.” although mclachlan et al. (2016) also concluded that study options and computer literacy do not necessarily improve with increasing access to computers or using computers more, the finding is not new. for example, the technology-productivity paradox was termed from data collected in 2006 (oecd, 2010) and recently not integrating technology well was thought to cause more harm than good ("gesf agenda: rethinking collective responsibility for public education," 2016). subsequently, the article could have had a broader scope by considering a wider range of perception and factors critical for ict integration. perceptions such as those of school leaders (vella, 2015) and among other factors, professional development and training; executive and administrative level support; stakeholder involvement; skill level and commitment of faculty and students; and the availability of resources and financial support. (dennison, 2013; vella, 2014). in other words, whilst the paper shows the student perspective, it doesn’t appear to delve sufficiently into other causes, such as the role of thought leaders in schools. that is, stereotypical views of students are shown, but not the ways in which those stereotypes might be influenced or altered. aside from thought leaders in schools, other areas to investigate is the professionalism of ict teachers and measures they might take to improve their understanding 1 australasian journal of information systems vella 2016, vol 20, post publication review students' computing use and study of the industry (burmeister, 2015). that is, whether ict teachers also view themselves as bound by ict professional codes (bowern, burmeister, gotterbarn, & weckert, 2006; burmeister, 2013), not just as teachers? if so, then they have a responsibility, as part of such professionalism, to promote their profession and have it be seen in a positive light, worthy of pursuit. overall the paper achieves what it sets out to do. as seen above, it leaves significant room for further research, to investigate further ways in which schools can do their part to increase student perceptions of ict careers as being worthy pursuits for them. steven vella doctor of it student charles sturt university author response we thank the author of the post publication review for his interesting and positive feedback and for the number of options he has suggested for future research. the aim of the paper was to demonstrate support for the reverse effect on interest by use. the studies suggested by vella indicate a lower capability by tertiary students to undertake ethical analysis. the authors are not clear as to the relevance of this comment in the context of computer use by secondary school children. further, the authors are aware of a similar decrease in student interest in ict/it courses at the tertiary level. we thank the reviewer for highlighting the oecd finding regarding the technologyproductivity paradox and this will be taken into account in future work as will the findings in the more recent publications that he has identified and were published after the acceptance of this paper. the authors are well aware of the wider range of perceptions and factors critical for ict integration and we are currently supervising a phd project which explores exactly this issue. we agree that teachers have a responsibility to promote their profession (education or their discipline or both?) in a positive light. however, since the australian new curriculum recommends integration of ict teaching into all subject areas, the issue of professionalism needs to go beyond that of ict teachers and include all teachers. as vella suggests, there is significant room for further research; probably two or three phds (if not more) can comfortably be accommodated in this topic area! christine a. mclachlan deakin university, geelong, australia. annemieke craig deakin university, geelong, australia. jo coldwell-neilson deakin university, geelong, australia. references al-saggaf, y., burmeister, o. k., & weckert, j. (2015). reasons behind unethical behaviour in the australian ict workplace: an empirical investigation. journal of information, communication & ethics in society, 13(3/4), 235-255. bowern, m., burmeister, o. k., gotterbarn, d., & weckert, j. (2006). ict integrity: bringing the acs code of ethics up to date. australasian journal of information systems, 13(2), 168-181. 2 australasian journal of information systems vella 2016, vol 20, post publication review students' computing use and study burmeister, o. k. (2013). achieving the goal of a global computing code of ethics through an international-localisation hybrid. ethical space: the international journal of communication ethics, 10(4), 25-32. burmeister, o. k. (2015). improving professional it doctorate completion rates. australasian journal of information systems, 19, 55-70. doi: 10.3127/ajis.v19i0.1073 dennison, w. (2013). critical success factors of technological innovation and diffusion in higher education. available at http://works.bepress.com/wayne_dennison/6 gesf agenda: rethinking collective responsibility for public education. (2016). [press release]. available at https://educationandskillsforum.org/gesf-agenda-rethinkingcollective-responsibility-for-public-education/ lucas, r., & mason, n. (2008). a survey of ethics and regulation within the ict industry in australia: ethics education. journal of information, communication and ethics in society, 6(4), 349-363. doi:10.1108/14779960810921141 mclachlan, c. a., craig, a., & coldwell-neilson, j. (2016). students' computing use and study: when more is less. australasian journal of information systems; vol 20 (2016). oecd. (2010). are the new millennium learners making the grade? : technology use and educational performance in pisa 2006. doi:10.1787/9789264076044-en simpson, c., nevile, l., & burmeister, o. k. (2003). doing ethics: a universal technique in an accessibility context. australasian journal of information systems, 10(2). vella, s. (2014). comparing perceptions: a secondary analysis. paper presented at the 2014 higher degree research symposium, wagga wagga. available at http://svstudyblog.files.wordpress.com/2015/10/2014symppaper.pdf vella, s. (2015). external influencers of school leaders in rural australia deploying and integrating ict. paper presented at the 2015 higher degree research symposium, wagga wagga. available at http://svstudyblog.files.wordpress.com/2015/10/stevenvella-2015-symposium-paper.pdf copyright: © 2016 vella, mclachlan, craig & coldwell-neilson. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. 3 http://creativecommons.org/licenses/by-nc/3.0/au/ http://creativecommons.org/licenses/by-nc/3.0/au/ microsoft word ajis_18(3)_v10.docx australasian journal of information systems volume 18 number 3 2014 391 information security risk management: an intelligence-driven approach jeb webb department of computing and information systems melbourne school of engineering university of melbourne victoria, australia sean maynard department of computing and information systems melbourne school of engineering university of melbourne victoria, australia email: sean.maynard@unimelb.edu.au atif ahmad department of computing and information systems melbourne school of engineering university of melbourne victoria, australia graeme shanks department of computing and information systems melbourne school of engineering university of melbourne victoria, australia abstract three deficiencies exist in the organisational practice of information security risk management: risk assessments are commonly perfunctory, security risks are estimated without investigation; risk is assessed on an occasional (as opposed to continuous) basis. these tendencies indicate that important data is being missed and that the situation awareness of decision-makers in many organisations is currently inadequate. this research-in-progress paper uses endsley's situation awareness theory, and examines how the structure and functions of the us national security intelligence enterprise—a revelatory case of enterprise situation awareness development in security and risk management—correspond with endsley’s theoretical model, and how facets of the us enterprise might be adapted to improve situation awareness in the information security risk management process of organisations. keywords: information, security, risk management, enterprise situation awareness, intelligence introduction the modern organisation is essentially built of information: almost everything the organisation is and does involves information’s storage, use, or communication. information security is a broad term that essentially refers to the practice of protecting information and the ways in which it is used to serve the goals of an organisation (whitman and mattord 2004; ahmad, bosua, scheepers 2014). given the critical role of information in the functions of organisations, the importance of information security is australasian journal of information systems volume 18 number 3 2014 392 widely acknowledged (baskerville 1991; shedden et al. 2010a). laws and standards designed to guide information security practices have become more prevalent worldwide. these laws and standards typically endorse a “risk management” approach to information security. the object of risk management is to identify sources of risk and deal with them appropriately. managing information security risks effectively, however, requires accurate appraisal of the organisation’s overall information security situation, and there is evidence that, under prevailing practices, much of the information required to model risk representatively is simply not being incorporated into organisations’ information security risk assessments (parker 2007; shedden et al. 2011; shedden et al. 2010a; utin et al. 2008). a review of information security literature revealed that organisations tend to conduct information security risk assessments in a perfunctory manner (johnson 2009; matwyshyn 2009; shedden et al. 2010a; young and windsor 2010; ahmad, maynard, park 2014); to estimate risk based on speculation rather than evidence (parker 2007; richardson 2011; shedden et al. 2011; utin et al. 2008); and to assess risk on an occasional—as opposed to continuous—basis (rees and allen 2008; schmittling 2010; hulme 2004). each of these tendencies describes a way in which important security status information is being omitted from information security risk management (isrm) decision making. the decisions that senior managers need to make relating to their organizations' information security postures (e.g. whether to change the way the organization does things to avoid negative situations that could cost it money, damage its reputation, or otherwise impede its accomplishment of strategic objectives) need to be informed by accurate understanding of the risks their particular organizations are actually faced with. we argue that this problem can be described as one of leaders and their subordinates lacking situation awareness in regard to the security states of their organizations. the decision-makers in many organizations are neither registering how developments within their operational environments affect the security of their information and information systems, nor recognizing how the functionality and strategic interests of the organization as a whole are contingent upon this security. this research-inprogress paper is part of a research project which aims to develop an intelligence-driven isrm process that maximizes situation awareness among the decision-makers involved in that process. in this initial stage of the project we use a document-based case study to identify whether endsley’s situation awareness model can be utilized at an organisation level. ultimately, our aim is to answer the following research question: “how can situation awareness be increased in information security risk management?” this research-in-progress paper is organized as follows. the background to the study is discussed including the literature on information systems security and an introduction of endsley’s situational awareness model (1995). next the research approach for the research is presented. following this, a document-based case study is presented in which endsley’s model is used to develop an a priori version of the isrm model. we conclude this research-in-progress paper with a discussion of the project’s outlook and its potential implications/contributions. background this section reviews two relevant literature areas. it first examines work in information security risk assessment. secondly, it discusses situational awareness, in particularly endsley’s model of situational awareness. information security and risk assessment in the literature on information security risk management practices three apparently endemic deficiencies were uncovered: (1) information security risk assessment is commonly perfunctory; (2) australasian journal of information systems volume 18 number 3 2014 393 estimations of information security risks commonly rely on speculation rather than evidence; and (3) information security risk assessments are not typically carried out continuously and historically. security risk assessment is commonly perfunctory there is considerable evidence in the literature that suggests that organizations are not identifying significant sources of risk during the risk management process (parker 2007; utin, utin & utin 2008; shedden et al. 2009; shedden et al. 2010b; shedden et al. 2012). for example, the traditional perspective of assets being discrete and enumerated leads to the omission of intangible knowledge assets such as distributed tacit knowledge amongst personnel (shedden et al. 2011). another type of frequently omitted risks are those resulting from the exploitation of vulnerabilities arising out of complex relationships between multiple information assets (parker 2007; utin et al. 2008). malicious threats such as fraud and espionage, or sabotage are also not recognized as indications of risks (kowalski, cappelli and moore 2008; colwill 2009; stewart and lacey 2012). the inability to systematically and continuously learn from past information security incidents also results in the omission of risks (ahmad, hadgkiss, & ruighaver, 2012); and risk identification suffers from failures to identify attack patterns, or to model complex or persistent attack scenarios in order to develop countermeasures / counterstrategies against them (park 2008, raju 2012). security risks are estimated on the basis of speculation rather than evidence whilst the existence of standards and legal requirements reinforces the message that information security is important, the degree to which information security is unique to each organisation appears to be widely misunderstood (parker 2007; richardson 2011; shedden et al. 2011; utin et al. 2008). standardized approaches to risk estimation are rarely useful toward estimating the specific risks that a particular organisation is faced with. rather, these methods lead practitioners to settle on guessed values and imagined probabilities virtually prima facie (baskerville 1991). nevertheless, such approaches are commonly accepted by many organisations as adequate means for managing information security risk (parker 2007; utin et al. 2008). risk cannot be properly managed unless it is fully understood (humphreys 2008). to understand information security risk fully, organisations must ensure that the roles and characteristics of the information assets their business processes depend on are understood through ongoing, conscientious investigation, and that lessons learned become matters of record. security risk is not assessed historically and continuously many organisations conduct information security risk assessments as infrequent events occurring somewhere between quarterly and yearly (rees and allen 2008). when carried out this way, a detailed assessment can become overwhelming because all of the pertinent information must be gathered within a limited time frame. furthermore, information gathered at any one point in time only constitutes a status “snapshot” of the organisation’s actually fluid information security environment (schmittling 2010). hulme notes that “a risk assessment conducted on the first day of the month can be quite different than the same assessment conducted several weeks later;” and that risk can be most effectively minimized by keeping “eternally vigilant” (2004). ahmad, hadgkiss and ruighaver (2012) argue that it is also important for organisations to retain memory of past security incidents and identified problem indications. in their failure to remain attentive and retentive, many organisations are missing out on vital risk-pertinent data about current developments and long-term trends that could afford them advance warning, by means of predictive analysis, of impending incidents. summary of risk assessment deficiencies each of the three flaws outlined above describes a way that important information about risk is misapprehended or simply missed altogether. without this information, managers' understandings of their organisations' security situations are fragmentary. we argue that this problem can be described as australasian journal of information systems volume 18 number 3 2014 394 one of leaders and their subordinates lacking situation awareness concerning the security states of their organisations. the phenomenon known as situation awareness (sa), as it is explained by endsley’s (1995) sa theory, occurs when an actor, whose function is to decide and act appropriately on (or in response to) a situation, has the relevant status information he or she requires about the elements of (i.e. the “different things going on” within) this situation of interest to decide and act appropriately. as sa theory describes how decision-making actors come to understand the contexts within which they function, we argue that it is an appropriate theory for the current project, which aims to increase information security risk managers’ awareness of their organizations’ respective security situations. theoretical basis: endsley’s theory of situation awareness endsley’s theory of sa is by far the most widely accepted and validated theory of sa (salmon 2008). while other authors have modelled some aspects of sa differently, or have argued that an sa theory should draw on theoretical underpinnings different from those endsley has drawn upon, a review of the literature failed to uncover any arguments that genuinely undermine the validity of her theory or that offer better—or, we would argue, even significantly different—explanations of how people come to develop awareness of situational states in the context of goal-oriented activities. endsley defines sa as “the perception of the elements in the environment within a volume of time and space, the comprehension of their meaning, and the projection of their status in the near future” (endsley 1988, in endsley and jones 2011). in endsley’s model, sa is achieved in progressive stages. in level 1 sa, one perceives, or becomes aware of, “the status, attributes, and dynamics of relevant elements in the environment” (endsley 1988, in endsley and jones 2011). failure to achieve level 1 sa essentially amounts to a failure to perceive relevant information about the environment, given one’s information requirements in light of one’s goals and objectives. in level 2 sa, one compares perceptions of the environment against one’s internally held understanding of, or associations regarding, this incoming information (“prototypical situations in memory;” endsley 1995). failure to achieve level 2 sa amounts to a failure to understand what has been perceived, which can stem from information overload or from having inadequate informational templates (e.g. “mental models” held in human memory) to reference in processing and interpreting the sensed/incoming information (endsley 1995). when level 2 sa is achieved, one is aware of information’s intrinsic meaning(s), as well as its significance in the context of functional goals and objectives. level 3 sa occurs when one is able to extrapolate the implications of things perceived about the environment, to predict what will happen “at least in the very near term,” based on one’s extant understanding of cause and effect relationships between the elements of a situation (endsley 1995). to achieve level 3 sa, one must already have developed level 2 sa. level 3 sa enables one to anticipate and plan for alternative future scenarios. failure at this level can stem from information overload or inadequate subject matter knowledge (endsley and jones 2011). endsley’s model portrays sa as a phenomenon that occurs in the context of decision-making, as it recognizes sa to be purposeful or goal oriented: “goals form the basis for most decision making in dynamic environments” (endsley 1995). what we might call “high fidelity” level 3 sa, or level 3 sa borne out of assessing the situation of interest accurately, and interpreting it rationally in the context of goals and objectives, becomes the basis for informed decision making (endsley and jones 2011). endsley’s model of sa is depicted in figure 1. australasian journal of information systems volume 18 number 3 2014 395 figure 1: endsley’s situation awareness model (adapted from endsley 1995) above and below the chain from perception to action upon a situation (shown as the orange, red, blue and green boxes in figure 1) are task/system and individual factors that bear on the sa development and decision making processes. all of these factors only become meaningful in the context of a specific individual or system. as our project concerns coordinated or “team” sa development, however, the roles of “goals and objectives”—which are shared between team members—have special relevance and are ultimately singled out for representation in our adapted model. endsley explains that sa within a team involves “a specific set of sa elements” for each member, pertinent to his or her functions within the team, with some overlapping between these elements (endsley 1995). overlapping occurs where goals and objectives are shared between team members. similarly, any member may acquire information that meets the sa needs/requirements of another member and, in such a case, this information should be shared (endsley 1995). endsley defines the sa of the team as “the degree to which every team member possesses the sa required for his or her responsibilities” (endsley 1995). research approach this is a design science research project employing mixed qualitative analytical methods. the overall project’s design comprises the following phases: (1) a literature review to identify a gap/need in the information security subject area; (2) adaptation of endsley’s theoretical model of sa to include supported second-party decision making; (3) a single case revelatory case study (yin 2009) in which open source documents were textually analysed and coded (carley 1993; neuman 2011) to determine whether the functions of the usnsie correspond with the adapted model; (4) the specification of an a priori model for an isrm process in particular, based on findings from the literature review and logical analogues between us national security intelligence enterprise components and the components of typical organisations; (5) expert interviews with information security risk managers to validate or revise components of the a priori model; (6) the specification of a revised model based on expert input; and (7) focus groups to refine and validate the revised model. of these 7 phases, 1-4 have been completed. australasian journal of information systems volume 18 number 3 2014 396 case study of the us national security intelligence enterprise: enterprise sa production we argue that the us stands as the world’s richest organisation in terms of both its wealth and the complexity of its critical national infrastructure. its national security intelligence enterprise represents perhaps the ultimate example of sa in the interest of asset protection. the us intelligence community (ic) is composed of 17 specialised but coordinated elements dedicated to collecting and processing information required by decision-makers to establish awareness vis-à-vis national security issues (odni 2011). as the us national security intelligence enterprise (usnsie), which comprises the us ic and the military and governmental decision-makers that the ic serves, describes an sa enterprise devoted to informing the management of risks to national security, we argue that the usnsie provides a revelatory case of how sa can be achieved across a security risk management enterprise. the object of the case study was to understand how the usnsie develops sa and facilitates the sa of decision makers in government; the purpose of doing this was to derive a process model for enterprise sa development. the case study involved the analysis of 71 documents. the documents analysed included us law, ic policy documents (intelligence community directives or “icds”), military manuals and other us government publications, us government websites, and authoritative works produced by subject matter experts. initially, open coding (neuman 2011) was carried out on 32 icds and 5 pieces of us legislation to compile a lexical concordance of named actors and functions (carley 1993). the concordance was constructed, using microsoft word, in the form of a large (171pages) table consisting of 154 entries. icds and legislation were selected for analysis based on their titles and apparent themes. axial coding followed open coding. the object of axial coding is to evaluate previously developed codes for their utility in describing themes of interest within the study (neuman 2011). we realized that the vast majority of the data we had collected concerned details that—though they concerned the structure of the usnsie at the highest strategic level—were still too specific (and complex) for adaptation into analogue components for a realistic organizational isrm process model. furthermore, while we now had a fair idea of the usnsie’s structure and role in government, our understanding of how intelligence is actually created and disseminated within this enterprise was still lacking. a search of available literature revealed that no theory has been accepted as a definitive explanation of the process by which a national security intelligence enterprise informs decision making (treverton et al. 2006). typically, the overall process of identifying the intelligence needs of decision-makers and carrying out the operations required to meet these needs is described in terms of a conceptual framework known as the “intelligence cycle” (johnson 2012). the intelligence cycle is a feedback loop divided into phases of activity. the number of discrete phases composing this cycle is a matter of opinion and varies across authors. we argue that the intelligence cycle is most accurately depicted as a twelve phase cycle, as this allots a discrete phase for each component activity. while the intelligence cycle is a useful heuristic for understanding how intelligence generally informs decision making, it does not in itself constitute a detailed process description, however. thus, our next step was to assemble a suitable process description. we created an outline of the intelligence cycle in which each phase represented the heading of a section. we then reviewed the entries of the actors and functions table to determine the most key/central actors at the highest enterprise levels. next, we constructed a step-by-step account of what happens within the usnsie throughout the intelligence cycle. as we drafted this account, its components were considered for their overall representativeness, centrality, and level of granularity within the usnsie: actors that were too specific to the peculiar functions of the enterprise, or which served less than central functions, were considered poor candidates for adaptation into a high level process model and were omitted from our account on that basis. essentially, this phase consisted of multiple iterations of selective coding, during which data was reorganized under themes to further inform our understanding of “major themes or concepts” of interest (neuman 2011). australasian journal of information systems volume 18 number 3 2014 397 at this stage we still lacked a robust theoretical template on which to propose a process model. a search for theory relating to the role of situation awareness in decision making and action ultimately led us to mica endsley’s sa theory. to determine whether endsley’s theoretical model of sa could be used to describe the usnsie, we then applied the pattern matching variation of the illustrative method (neuman 2011). neuman explains that the illustrative method involves deriving “empty boxes” from components of pre-existent theory and then filling these boxes with findings from research data (2011). in our case, these “empty boxes” were structural components of endsley’s theoretical model, which we “filled” with case study data. we compared concepts within sa theory (i.e. task and system factors, stages of sa development, decision and action phases, and its feedback loop) to the phases of the intelligence cycle to identify points of analogy. we then sought to adapt endsley’s model to better describe the way a team of actors can serve as “purveyors of situation awareness” to decision makers the context of an intelligence enterprise. this involved successive approximation, described by neuman as “(a) method of qualitative data analysis that repeatedly moves back and forth between the empirical data and the abstract concepts, theories, or models, adjusting theory and refining data collection each time” (2011). evidence from the case study was considered and reconsidered within the context of endsley’s sadevelopment/decision/action cycle to yield a process template describing enterprise sa production in support of a single decision making actor. figure 2 illustrates how the intelligence cycle framework can be represented as an adapted situation awareness process model. the process of developing sa, whether it is confined to individual experience or applied to a distributed enterprise, describes the organisation and interpretation of situational element status data into an overall understanding of a situation. while endsley’s model of individual sa pertains to an individual acting directly upon or in response to a situation, figure 2 portrays sa development through the concerted effort of multiple actors whose overall function is to develop and then export this sa to a consumer who then acts (or directs action) on or in response to the situation of interest. in figure 2, us ic actors have been simplistically divided into two types: collection components and analysis components. table 1 maps correspondences between the intelligence cycle and sa theory. australasian journal of information systems volume 18 number 3 2014 398 figure 2: enterprise sa in the context of the intelligence cycle though integral to the usnsie, issuing feedback is not, of course, the decision-maker's definitive function. the role of the decision-maker is to make decisions that lead to action on, or in reaction to, a situation of interest, as necessary in light of strategic goals and objectives. if the incoming intelligence suggests that a situation requires action, then the decision-maker will make a decision concerning what action is required and, to some extent, how it should be carried out. in many cases, the decision-maker in government does not carry out an action but rather directs one or more proxy actors to do so (johnson 2012; lowenthal 2000). alternatively, if intelligence suggests that the situation is already conducive to the goals and objectives of the decision-maker, action may not be required. yet another possibility is that the intelligence presents the situation differently than the decision-maker had previously conceived of it, and the decision-maker must revise his or her goals and objectives to accommodate this new understanding (lowenthal 2000). if action affecting the relative state of the situation is carried out by the decision-maker or proxy, the situation changes and a new enterprise sa development and synthesis process must be undertaken by the us ic and the decision-maker to establish current awareness of it. discussion in the previous section, we have argued that the usnsie is actually an example of enterprise sa. we will now argue that this enterprise model can be applied toward the design of a situation-aware isrm process for organizations. just as an enterprise consists of layered activities, enterprise sa must also be considered at multiple levels. in figure 2, the combined efforts of the entire intelligence community have been simplified into one tier of a two-tiered model. while this may accurately depict the nature of the enterprise at a high level, the reader must remember that the model can be adapted to describe any transaction in which sa is developed by an actor or team of actors and provided to another actor who then decides and acts in response to the situation of interest. this is to say that the model can be adapted to describe activity between all layers of the sa enterprise. australasian journal of information systems volume 18 number 3 2014 399 intelligence cycle phases 1, 2 and 3: requirements, planning, and direction sa theory analogue goals and objectives explanation the goals and objectives of decision-makers determine intelligence requirements. these requirements are then translated into operational requirements at the ic element level (icd 900; jp 2-0). in phases 2 and 3, planning and direction by ic leadership determines the goals and objectives of operators within ic elements (odni 2011). these goals and objectives provide the context by which situational element states are judged—determining what needs to be perceived and why, and informing understanding of developments’ implications, given the goals and objectives of the decision-making intelligence consumer (odni 2011). intelligence cycle phases 4, 5, and 6: collection, processing, and exploitation sa theory analogue level 1 sa (perception) explanation: during collection, situational element state data is gathered by human or technical assets (johnson 2012; jp 20). perception occurs when element states are perceptible by conscious agents (endsley 1995). in human assets, collection and processing can occur concurrently; human perception of technical data occurs after some machine processing (jp 2-0; miller 2004). exploitation requires basic relevance recognition and is often automated to some extent (jp 2-0; icd 300; odni 2011). humans facilitate exploitation by labelling and classifying information for easy discovery or by forwarding information where useful (miller 2004; odni 2011). intelligence cycle phase 7: analysis sa theory analogues level 2 sa (comprehension); level 3 sa (projection); decision explanation situational comprehension occurs at the analysis phase of the intelligence cycle (katter, montgomery and thompson 1979). analysis can focus on anywhere from current to long term situations, and often draws on the expertise of multiple specialist analysts to develop multidimensional comprehension of a particular situation (jp 2-0). analysts are commonly expected to apply their knowledge in the subject area to anticipate the implications of a situation's current status, and its likely future state, for decision-makers' goals and objectives (usg 2009). analysts decide and act across analysis, production, evaluation and dissemination. intelligence cycle phase 8: production sa theory analogues decision; action explanation the process of creating a coherent piece of finished intelligence explaining the analyst's findings, rationale, limitations, recommendations, etc., is referred to as "production;" the finished intelligence, which can be in any media format, is often referred to as an "intelligence product" (odni 2011). the analyst must make decisions that amount to findings as well as decisions about how to communicate these findings in the final product, given the stated needs of the consumer. intelligence cycle phase 9: evaluation sa theory analogues decision; action explanation the analyst, and under some circumstances the analyst's superior, must also carry out an evaluation of the intelligence product, to verify that it fulfils the consumer/decision-maker’s stated intelligence requirements, prior to the product's dissemination (icd 203). both production and evaluation can involve reiterations of decision and action when products are evaluated negatively (jp 2-0; icd 203). intelligence cycle phase 10: dissemination sa theory analogue action explanation the action of disseminating a useful and appropriate intelligence product is the ultimate functional objective of the intelligence analyst (treverton and gabbard 2008). intelligence products may be disseminated physically, electronically, presented formally or simply communicated, depending on the type of intelligence involved, consumer needs and specifications, and temporal, locational, or other practical considerations (odni 2011). intelligence cycle phase 11: consumption sa theory analogue level 1 sa (perception); level 2 sa (comprehension); level 3 sa (projection) explanation the disseminated product is consumed (listened to, read, or otherwise perceived) by the decision-maker (odni 2011). consumption initiates the process of sa synthesis in the mind of the consumer. consumed intelligence is not always passively assimilated; the decision-maker often compares the product’s contents to information held in memory to develop personal understanding of the situation (lowenthal 2000). while perception and lower level comprehension may occur in rapid succession, higher level comprehension of the situation and the internal formation of projections by the decisionmaker may require a period of reflection and rumination. intelligence cycle phase 12: feedback sa theory analogue action explanation in the context of the intelligence cycle, feedback refers to the decision-maker's confirmation or denial, to an ic liaison, that the intelligence product has met his or her stated information requirements regarding the situation of interest (odni, 2011). it is an action by the decision-maker that is inherent to the cycle itself. table 1: correspondences between the intelligence cycle and sa theory australasian journal of information systems volume 18 number 3 2014 400 in the next section we present an adapted, a priori model describing a situation-aware/intelligencedriven isrm process at the business process level. it is the second of three diagrams (depicting [1] transactions between department level collection and analysis assets and business process owners; [2] transactions between business process owners and the isrm manager; and [3] transactions between the isrm manager and an executive level officer, such as a chief information security officer, or chief security or risk officer, where such a relationship exists within the hierarchy of an organization) that have been developed to describe the proposed process. the others have been omitted from this paper due to space considerations. an a priori model for intelligence-driven isrm we propose that enterprise sa development in isrm should start with collection and analysis at the department level. the departments are intrinsically specialized to carry out particular functions within the organisation, and should be most familiar with the information assets located under their respective functions. responsibility for isrm should be distributed across business process owners (coles and moulton 2003), who have tasking authority over collection and analysis components embedded within the departments. the business process owners report up to the isrm head, subordinate to the organisation’s risk or security executive (where one exists). at the department level, the security statuses of specific information assets are the collection targets. if multiple information assets that fall under the same department are interdependent or otherwise interact with each other, the department-level intelligence component is responsible for collecting status information relating both to the separate information assets as well as to their interactions between each other. "information assets" includes pieces of hardware, software, network assets, individual people, specific procedures, and data assets (whitman and mattord 2004). while the security states of hardware, software, and network assets generally fall under the purview of the organisation's it department regardless of the business process concerned, other information assets supporting a business process, such as the people, procedures and pieces of information involved, may be distributed across several departments. collection targets should be developed with guidance from the enterprise and business process levels in the form of refined and prioritized intelligence requirements. intelligence forwarded from departmental collection and analysis components is received and aggregated by business process owners, who then perform business process level analyses. the products resulting from these analyses are then disseminated upward to the isrm head, who aggregates the findings to perform a whole-enterprise security assessment. figure 3 depicts our a priori model of intelligence-driven isrm at the business process level. the diagram illustrates the relationship between each business process owner and the information security risk manager. the relationship between all business process owners and the manager can be similarly modelled, however, in much the same way as the upper tier of figure 2 represents the combined efforts of us ic components. in figure 3, authority to act on a process level situation is generally delegated to the business process owner, subject to oversight by the isrm manager. a similar diagram applies to the relationships between department level collection and analysis assets and each business process owner, wherein the department level assets assume the upper tier of the diagram, the business process owners assume the lower tier, and authority to act on a department level situation is held by the business process owner(s) involved, but may be delegated to an actor at the department level. australasian journal of information systems volume 18 number 3 2014 401 figure 3: intelligence-driven isrm at the business process level conclusion this research-in-progress paper has discussed the development of an intelligence-driven isrm process model for organisations. it has described three types of sa deficiencies identified in information security literature and proposes an enterprise sa model, derived from a case study of the usnsie, to improve sa in isrm. the next step for this research project will be to conduct interviews with subject matter experts from the isrm field. the point of these interviews will be to elicit expert advice concerning the general feasibility of the a priori model as presented here, as well as general, high level recommendations for improving it. following incorporation of this input, we will then present the resulting model to a focus group. while the purpose of the initial expert interviews will be to determine what components—if any—of the model are considered valid propositions in the broad sense, the purpose of the focus groups will be to revise, refine, and validate lower level components of the formerly validated high level model. the intelligence driven isrm process model has several important implications for practitioners and researchers. first, it will enable more accurate estimations of risk in information security by distributing the assessment workload across the enterprise, rendering more detailed assessments practicable. second, it will institute continuous monitoring and reporting/documentation practices, increasing the chances of threat detection while also enabling trend mapping and predictive analysis. the net result of this for practitioners will be betterinformed decision-making in isrm at both the business process and whole enterprise levels. as the model approaches isrm from a business process security perspective, it links isrm directly to the strategic business interests of the organisation, increasing the likelihood of enthusiastic support by upper management. australasian journal of information systems volume 18 number 3 2014 402 the model will also provide the basis for further research in situation-aware/intelligent information security risk management. the model should ultimately be developed into a complete method that guides organizations through the process of sa/intelligence requirements identification, as well as through isrm-specific collection and analysis procedures. empirical work needs to be done to evaluate the actual feasibility of the model and any resulting method. action research studies implementing isrm methods based on the model resulting from this project would be particularly useful. references ahmad, atif, justin hadgkiss, and a.b. ruighaver. 2012. "incident response teams – challenges in supporting the organisational security function." computers & security 31, 643-652. ahmad, a., maynard s., park, s. 2014. information security strategies: towards an organizational multi-strategy perspective. journal of intelligent manufacturing. 25(2). 357-370 ahmad, a., bosua, r., scheepers, r. 2014. protecting organizational competitive advantage: a knowledge leakage perpective. computers & security. vol. 42. 27-39. baskerville, r. 1991. “risk analysis: an interpretive feasibility tool in justifying information systems security.” european journal of information systems 1, no.2: 121-130. broderick, j. stuart."isms, security standards and security regulations." information security technical report 11, 26-31. carley, kathleen. 1993. "coding choices for textual analysis: a comparison of content analysis and map analysis.” sociological methodology 23, no. 1: 75-126 coles, robert s., and rolf moulton. 2003."operationalizing it risk management." computers & security 22.6: 487-493 colwill carl. 2009, human factors in information security: the insider threat—who can you trust these days? information security technical report 2009; 14(4):186—96. director of national intelligence. intelligence community directive number 116: intelligence planning, programming, budgeting, and evaluation system. washington, d.c.: office of the director of national intelligence, 2011. director of national intelligence. intelligence community directive number 200: management, integration, and oversight of intelligence community analysis. washington, d.c.: office of the director of national intelligence, 2007. director of national intelligence. intelligence community directive number 203: analytic standards. washington, d.c.: office of the director of national intelligence, 2007. director of national intelligence. intelligence community directive number 300: management, integration, and oversight of intelligence collection and covert action. washington, d.c.: office of the director of national intelligence, 2006. director of national intelligence. intelligence community directive number 900: mission management. washington: office of the director of national intelligence, 2006. endsley, m.r. 1988. “design and evaluation for situation awareness enhancement.” proceedings of the human factors society 32nd annual meeting, 97-101. santa monica, california: human factors society. endsley, m.r. 1995. “toward a theory of situation awareness in dynamic systems.” human factors 37, no. 1: 32-64. endsley, mica r. and debra g. jones. 2011. designing for situation awareness: an approach to usercentered design. boca raton, florida; london: crc press. hulme, george v. “getting at risk.” in management of information security, by michael e. whitman and herbert j. mattord, 307-308. boston: thomson course technology, 2004. australasian journal of information systems volume 18 number 3 2014 403 humphreys, edward. 2008. "information security management standards: compliance, governance and risk management." information security technical report 13, no. 4: 247-255. johnson, alice m. 2009. "business and security executives views of information security investment drivers: results from a delphi study." journal of information privacy & security 5, no. 1: 327. johnson, loch k. 2012. national security intelligence: secret operations in defense of the democracies. cambridge, uk; malden, ma. joint chiefs of staff. joint publication 2-0: joint intelligence. washington, d.c.: office of the joint chiefs of staff, 2007. katter, robert v., christine a. montgomery, and john r. thompson. 1979. "human processes in intelligence analysis." research report 1237. us army research institute for the behavioral and social sciences, us army intelligence and security command. kowalski eileen, dawn cappelli, andrew moore. 2008. insider threat study: illicit cyber activity in the information technology and telecommunications sector. united states secret service national threat assessment center and cert program. washington, d.c., pittsburgh, pa: carnegie mellon university software engineering institute; 2008. available at: http://www.secretservice.gov/ntac/final_it_sector_2008_0109.pdf. lowenthal, mark m. 2003. intelligence: from secrets to policy. washington, d.c.: cq press. matwyshyn, andrea. 2010. "csr and the corporate cyborg: ethical corporate information security practices." journal of business ethics 88, 579-594. miller, j.o. 2004. “modeling the us military intelligence process.” department of operational sciences, air force institute of technology. wright patterson, oh. neuman, william lawrence. 2011. social research methods: qualitative and quantitative approaches. boston, massachusetts: allyn & bacon. office of the director of national intelligence. 2011. us national intelligence: an overview. intelligence consumer’s guide. washington, dc. parker, donn b. 2007. "risks of risk-based security." communications of the acm 50, no. 3: 120. rees, j, and j allen. 2008. "the state of risk assessment practices in information security: an exploratory investigation." journal of organisational computing and electronic commerce 18, no. 4: 255-277. richardson, robert. 2011. 2010/2011 csi computer crime and security survey. new york: computer security institute. salmon, paul., and n. stanton. 2008. distributed situation awareness: advances in theory, measurement and application to team work. phd. brunel university. schmittling, ron. 2010. “performing a security risk assessment.” isaca journal, vol. 1: 1-7. shedden, piya, a. b. ruighaver, and atif ahmad. 2010a. "risk management standards the perception of ease of use." journal of information system security 6, no. 3: 23-41. shedden, p., smith, w., ahmad, a. 2010b. information security risk assessment: towards a business practice perspective, proceedings of the 8th information security management conference (pp. 127-138), perth, australia: edith cowan university. 30 nov – 2nd dec. shedden, p, scheepers, r., smith, m., ahmad, a. 2009. towards a knowledge perspective in information security risk assessments – an illustrative case study, proceedings of the 20th australasian conference on information systems (pp. 74-84), melbourne, australia: monash university. australasian journal of information systems volume 18 number 3 2014 404 shedden, piya, rens scheepers, wally smith, and atif ahmad. 2011. "incorporating a knowledge perspective into security risk assessments." vine: the journal of information & knowledge management systems 41, no. 2: 152. siponen, m. 2006. "information security standards focus on the existence of process, not its content." communications of the acm 49, no. 8: 97-100. siponen, mikko and robert willison. 2009. "information security management standards: problems and solutions." information & management 46, 267-270. stewart geordie, lacey david. 2012. death by a thousand facts: criticising the technocratic approach to information security awareness." information management & computer security 20(1):29. treverton, gregory f., and c. bryan gabbard. 2008. "assessing the tradecraft of intelligence analysis." rand national security research division technical report. treverton, gregory f., seth g. jones, steven boraz, philip lipscy. 2006. "toward a theory of intelligence." rand national security research division workshop report (conference proceedings). us government. 2009. a tradecraft primer: structured analytic techniques for improving intelligence analysis. washington, d.c.: us central intelligence agency, center for the study of intelligence. utin, daniil m., mikhail a. utin, and jane utin. 2008. "general misconceptions about information security lead to an insecure world." information security journal: a global perspective 17, no. 4: 164-169. von grebmer, andreas. 2007. information and it risk management in a nutshell: a pragmatic approach to information security. norderstedt, germany: books on demand. whitman, michael e., and herbert j. mattord. 2004. management of information security. boston, mass. thomson course technology. yin, robert k. 2009. case study research: design and methods. thousand oaks, california: sage. young, randall, and john windsor. 2010. "empirical evaluation of information security planning and integration." communications of ais 2010, no. 26: 245-266. an earlier version of this paper was presented at the australasian conference on information systems (acis) 2013 in melbourne, australia. microsoft word ajis_18(3)_v10.docx australasian journal of information systems volume 18 number 3 2014 213 a critical realist perspective of enterprise architecture evolution: conditioning and outcomes ayed alwadain computer science department king saud university riyadh, saudi arabia aalwadain@ksu.edu.sa erwin fielt school of information systems queensland university of technology brisbane, australia e.fielt@qut.edu.au axel korthaus college of business, information systems discipline victoria university melbourne, australia axel.korthaus@vu.edu.au michael rosemann school of information systems queensland university of technology brisbane, australia m.rosemann@qut.edu.au abstract this paper investigates how enterprise architecture (ea) evolves due to emerging trends. it specifically explores how ea integrates the service-oriented architecture (soa). archer’s morphogenetic theory is used as an analytical approach to distinguish the architectural conditions under which soa is introduced, to study the relationships between these conditions and soa introduction, and to reflect on ea evolution (elaborations) that then take place. the paper focuses on reasons for why ea evolution could take place, or not and what architectural changes could happen due to soa integration. the research builds on sound theoretical foundations to discuss ea evolution in a field that often lacks a solid theoretical groundwork. specifically, it proposes that critical realism, using the morphogenetic theory, can provide a useful theoretical foundation to study enterprise architecture (ea) evolution. the initial results of a literature review (a-priori model) were extended using explorative interviews. the findings of this study are threefold. first, there are five different levels of ea-soa integration outcomes. second, a mature ea, flexible and well-defined ea framework and comprehensive objectives of ea improve the integration outcomes. third, the analytical separation using archer’s theory is helpful in order to understand how these different integration outcomes are generated. keywords: enterprise architecture, ea, service-oriented architecture, soa, critical realism and morphogenetic theory. australasian journal of information systems volume 18 number 3 2014 214 introduction a widely adopted approach providing the required conceptual understanding of an enterprise as well as the way is facilitates its business processes, is enterprise architecture (ea). ea is “a complete expression of the enterprise; a master plan which ‘acts as a collaboration force’ between aspects of business planning such as goals, visions, strategies and governance principles; aspects of business operations such as business terms, organisation structures, processes and data; aspects of automation such as information systems and databases; and the enabling technological infrastructure of the business such as computers, operating systems and networks” (schekkerman 2005, p. 18). as a discipline, ea faces the challenge of responding to the emerging capabilities of both business and it domains and integrating them within its frameworks in order to enable true representations of existing organisational elements and their relationships. ea needs to evolve on both representational and architectural descriptions levels. architectural descriptions are the vehicle for building architectural representations (martin et al. 2009). the representational level entails keeping ea models up-to-date with instances changes such as changes to business processes or applications data (martin et al. 2009; shah et al. 2011). the architectural descriptions level includes managing the concepts or properties of a system, their relationships and the principles of the system design and evolution (martin et al. 2009). ea needs to evolve and accommodate emerging trends (jung 2009; mckendrick 2010; postina et al. 2010; sampaio 2010; sousa et al. 2009) in order to accurately develop models capable of representing those emerging trends, concepts and relationships. yet, ea’s literature lacks empirical studies that investigate ea evolution due to emerging trends. further, ea evolution management is still a challenge (buckl et al. 2009). in a recent forrester research survey, current business and it trends such as pervasive business intelligence, service-orientated architecture (soa) and cloud are considered challenges for ea over the next three years (mckendrick 2010). this study focuses on ea architectural descriptions evolution, through the focus on one specific new development with substantial impact on ea, namely the emergence of soa (martin et al. 2009; sousa et al. 2009). the open group (2010) defines soa as “an architectural style that supports service orientation, and service orientation is a way of thinking in terms of services and service-based development and the outcomes of services”. while many studies have argued for soa integration within ea (khoshnevis et al. 2009; postina et al. 2010), there is a lack of empirical studies that address soa and ea integration (kistasamy et al. 2012; viering et al. 2009). so far, there has been a lack of consensus on the form of such integration and its outcomes. the integration of elements and viewpoints of soa has been approached differently and inconsistently (postina et al. 2010). a literature survey of current approaches of soa integration within ea was conducted and a survey of the ways the five most popular ea frameworks accommodate soa was performed. these studies also have revealed significant variations in terms of how soa is integrated within ea (alwadain et al. 2011; alwadain et al. 2013a). in order to understand the integration of soa within ea outcomes and improve the integration, the use of a critical realist (cr) theory, namely archer morphogenetic theory (1995), is proposed. it is applied to understand the ea evolution process by clarifying the role of pre-existing ea, soa introduction and their relevant generative mechanisms on soa integration within ea. we argue that by taking a critical realism stance using archer morphogenetic theory (1995), we are able to investigate the issue in a much deeper way leading to richer insights into the integration of soa within ea. the rationale of a critical realist study is to explicate a given set of outcomes by uncovering the hypothesised existence of mechanisms that, once activated, could have generated these outcomes (wynn et al. 2012). we argue that ea evolution could be improved by using archer’s concept of analytical dualism where structure (ea) and action (soa introduction) are analytically held apart in order to appropriately study their interaction and understand their outcomes. we acknowledge the power and the relevance of all the analytical phases of archer’s model, but we will leave the architectural interaction (soa introduction) phase out of discussion due to space limitation. australasian journal of information systems volume 18 number 3 2014 215 summarising, this paper will address the following research questions: (1) how does ea evolve, in particular in relation to the introduction of soa? (2) what are the possible different outcomes of soa integration within ea? and (3) can critical realism, in particular archer’s morphogenetic theory, provide a theoretical foundation for ea evolution? the remainder of this paper is organised as follows. the next section presents the research methodology. then, the theoretical foundations and the a-priori model are introduced. subsequently, the revised research model and the findings are reported. finally, the summarised findings, limitations and an outlook are presented. research methodology the study started with a literature review. relevant empirical and analytical studies have been identified on soa integration within ea. they have been identified through ieeexplore, springerlink, sciencedirect, ebsco host, business source elite and ais electronic library. a combination of “service-orientation”, “soa”, “service oriented architecture”, “ea” and “enterprise architecture” was used as keywords when searching for articles. the literature review findings were used to develop the a-priori research model based on archer’s theory (alwadain et al. 2013b). to further enrich our understanding and refine the a-priori model, explorative interviews were conducted. from a critical realist perspective, sayer (1992, p. 179) states that “qualitative analysis of objects is required to disclose mechanisms”. semi-structured interviews were conducted with twenty enterprise architects and ea consultants who were involved with soa projects. the selection was based on their involvement with soa integration within ea projects to find information-rich participants, and the snowball sampling technique, where interviewing one person leads to another (patton 2001), was used to select further participants. candidates from private sector, public sector and consultancy groups were approached. twenty interviews with twenty-two persons involved in initiatives that included both ea and soa were conducted between june 2010 and july 2011, see table 1. we interviewed thirteen enterprise architects and nine ea consultants. most of the participants have a long experience with ea (more than 3 years) and between 5 to 20 years of various business/it jobs. interviews (i-5) and (i-6) were done separately with two participants of the same originations. in each interview session of (i-4) and (i-11) two participants were interviewed at the same time. each interview lasted between 45 minutes and one hour. an interview protocol was developed prior to the commencement to guide the interview sessions. most of the participants reported that they use togaf. this is consistent with other findings that togaf is the most widely used ea approach (infosys 2009). australasian journal of information systems volume 18 number 3 2014 216 interview job title industry sector ea framework i-1 chief enterprise architect banking proprietary ea framework i-2 chief enterprise architect multi-business modified togaf i-3 strategic architect government archimate i-4 (a) chief enterprise architect and (b) senior enterprise architect health modified togaf i-5 senior enterprise architect banking modified togaf i-6 senior enterprise architect banking modified togaf i-7 ea consultant consultancy togaf i-8 architecture manager health dodaf i-9 ea consultant consultancy togaf i-10 ea consultant consultancy modified togaf i-11 (a)ea manager and (b)architecture manager education in-house-developed ea based on togaf i-12 ea consultant consultancy togaf, zachman i-13 enterprise architect government meta-group methodology, now togaf i-14 architecture manager banking built-in framework (partial models) i-15 enterprise architect banking built-in framework i-16 ea consultant consultancy gartner i-17 ea consultant consultancy togaf i-18 ea consultant consultancy modified togaf i-19 ea consultant consultancy togaf i-20 ea consultant consultancy togaf, dodaf table 1. participants information thematic analysis was conducted in this research to analyse the interview data (fereday et al. 2006). each interview was recorded and transcribed. after reading each interview and getting familiar with its content, it is imported to qualitative analysis tool nvivo v9 for analysis. a predefined codebook derived from the a-priori research model was used as a starting point for coding and was updated during the coding process. both deductive and inductive coding techniques (fereday et al. 2006) were used. at the completion of the interviews analysis, more than 70 codes were in the codebook including the ones derived from the a-priori research model. then, newly identified codes were grouped and organised into possible themes (either generative mechanisms or outcomes). it was an iterative process back and forth in order to come to meaningful themes that describe the grouped codes. the a-priori model development the findings of the literature review in regard to soa integration within ea outcomes and the possible generative mechanisms are redescribed using archer’s morphogenetic theory to explain ea evolution. the result of this activity is the development of the a-priori model of this study, which is briefly described below using archer’s (1995) analytical lens. australasian journal of information systems volume 18 number 3 2014 217 archer (1995) proposed a morphogenetic theory for studying change. interaction between structure and agency is complex to properly investigate in social situations and thus an “analytical” dualism whereby structure and agency are analytically separated is proposed in order to appropriately study their interaction (archer 1995). structure is defined as the “set of internally related objects or practices” (sayer 1992, p. 92) that comprise the real entities we seek to examine in a particular contextual situation (wynn et al. 2012). archer (1995) states structures have properties (causal powers/generative mechanisms) which enable them to influence the world around them. the morphogenetic cycle has three analytical phases: structural conditioning, social interaction and structural elaboration. first, structural conditioning represents the existing structural properties prior to the intervention. they are consequences of past actions (archer 1995). second, social interaction is the second analytical level. during social interaction, agents engage with the pre-existing structures (archer 1995). at this level, actions are enabled or constrained by the pre-existing structures. finally, the third analytical level is the structural elaboration (either reproduction or transformation) of existing structures. archer’s theory as presented earlier was adopted for three reasons. it provides an analytical approach that (1) distinguishes ea and soa interaction into three phases and (2) facilitates the retroduction from the outcomes (phase three) to the causes (phase one and two). (3) soa introduction also has different properties (dependent on related generative mechanisms such as different perception, different scopes and different perceived benefits) which need to be understood to comprehend the outcomes. therefore, it was deployed as a means to better understand how the interaction between existing architectural settings and an soa introduction can lead to different integration outcomes. it serves as an analytical tool to represent conditions, actions and outcomes during ea evolution. such analysis is helpful when introducing soa by considering the pre-existing conditions and soa aspects in order to better integrate soa within ea. the structural conditioning at t1 is called “architectural conditioning” to reflect the scope of this study comprising ea and its subsequent changes as a result of soa introduction. the social interaction is called “architectural interaction (soa introduction)”. soa activities that are performed over the time period t2 to t3 are greatly influenced by both the pre-existing architectural conditions and soa related generative mechanisms. then, “architectural elaboration” at t4 is the result of the interplay between pre-existing architectural conditions and soa introduction. we argue that soa is introduced in different architectural conditions in different contexts. soa introduction itself has multiple generative mechanisms such as different perspectives of soa, scopes and perceived benefits which diversify soa implementation in different contexts. when soa is introduced, the interplay between theses architectural conditions and soa introduction leads to the different evolution outcomes. the a-priori model is described using archer’s (1995) morphogenetic analytical phases. phase one of the morphogenetic cycle is “architectural conditioning”. it represents the conditions in which the agents find themselves (archer 1995). the architectural conditioning could be it artefacts or design specifications. if the object of analysis, for instance, were an implementation, the structure would be a business process or processes that would be changed as a result of the implementation (cuellar 2010). in this study, architectural conditioning is represented through ea settings and ea maturity prior to soa introduction. ea is the main structure that is investigated with respect to its evolution due to soa introduction. in archer’s terms, ea is a structure that has been shaped by previous morphogenetic cycles and soa introduction results in its architectural elaboration. ea has a generative mechanism, namely its maturity which has an influence on the world around it. organisations need mature ea in order to successfully implement soa and realise expected soa benefits (perko 2008). the availability of detailed business architecture models during soa implementation and architects’ skills affect soa implementation (kokko et al. 2009). several ea maturity models have been proposed. they were studied and compared; for limitation of space, however, they are not discussed in this paper. this study adapted the nascio enterprise architecture maturity model (nascio 2003). it is considered a good example of ea maturity models by the open group and could be used to assess government and private ea maturity (the open group 2009). it australasian journal of information systems volume 18 number 3 2014 218 also conforms to the well-known maturity model sei smm (nascio 2003) and is widely used (gosselt 2012). the second analytical level is the social interaction between t2 and t3, in this study “architectural interaction (soa introduction)”, which means soa implementation related activities. in archer (1995) terms during social interaction, actors engage with the pre-existing architectural conditions. soa introduction triggers generative mechanisms that impact soa introduction and interact with other generative mechanisms in a given context to generate the observed outcomes. bygstad et al. (2011) note that it is often a group of objects that trigger a mechanism and generate an outcome that is dependent on the objects, but not reducible to them. the interaction of agents and technology may activate a group of mechanisms pertinent for the is discipline. triggering the mechanism and the result it might produce is not predetermined, but will depend on other active mechanisms and the context (architectural conditioning mechanisms). nevertheless, it tends to result in certain outcomes (bygstad et al. 2011). we have identified some generative mechanisms relevant to soa introduction from the literature and they are ‘view of soa’, ‘perceived soa benefits’ and ‘soa scope’. agents introduce soa entertaining a certain perspective of service-orientation, anticipating certain benefits and determining a certain scope. however, as said earlier, this phase is crucial to the explanation of the different integration outcomes but the details of this phase are excluded due to space limitations. the third analytical level is the architectural elaboration t4 (reproduction or transformation). the interaction between pre-existing architectural settings and soa introduction explains the architectural elaboration (integration outcomes) at t4. it results in either architectural transformation or reproduction of ea that existed prior to soa introduction. the literature findings suggest that there are different architectural transformation outcomes of the interaction between pre-existing ea and soa introduction. due to the fact that ea frameworks use different architectural layers, this study aimed at finding common layers of ea to use them to structure the findings. this study adopted the widely used layers of ea; business, information systems (information, applications) and technology (joachim et al. 2011). ea could be transformed or reproduced on all or some of the three architectural layers. the literature review findings of soa integration within ea were covered in alwadain et al. (2013b). this study hypothesises that ea is transformed or reproduced due to soa integration on three architectural levels: business architecture, is architecture and technical architecture. the outcomes of the integration could be transformation or production of ea on one or many of the three architectural levels. the first form of architectural transformation is service-orientation of business architecture. it means that soa and its relevant elements such as business services, service channels, soa vision, drivers, services slas and qos are integrated within business architecture models. it often builds on lower architectures service-orientations (e.g., is and infrastructure architectures are integrated with soa or going to be integrated). several studies have been identified which have integrated soa within ea on this level (e.g. see correia et al. 2007; khoshnevis et al. 2009). the second noticed level of integration is soa integration and relevant soa elements such as is services, service descriptions and slas. for example, soa is adopted to reduce the complexity of its distributed application landscape. enterprise services were integrated within the integration architecture while basic services (software components) are integrated within the software architecture (schelp et al. 2009). other examples are presented in jung (2009) and kistasamy et al. (2010). the third aspect of architectural elaboration is soa and its related elements such as technical services, services monitoring, services security, esb, xml standards and web services integration with the technology architecture. for example, the nsw departments of lands adopted a service-oriented architecture approach to transform its technical architecture using soa. it started when an esb was first introduced in 2005. second, a logistics operator in finland adopted soa during 2005. it was an it driven project using a technical bottom-up approach and soa was almost reduced to the use of web services. later, it was expanded into multiple projects to integrate the legacy system landscape employing soa technology to mainly expose legacy system services via an integration platform (kokko et al. 2009). australasian journal of information systems volume 18 number 3 2014 219 the extended theoretical (a-priori) model in this section we present the refined a-priori model (figure 1) that explains soa integration within ea outcomes underpinned by archer morphogenetic theory (1995). it is developed based on the literature review (a-priori model) and extended based on the interview findings. the (*) sign on some elements of the model entails that they were inductively identified from the interview findings. the interview analysis was guided by the a-priori model. particularly, the focus was on the architectural conditioning aspects, soa introduction aspects and the integration outcomes. following the analysis of the interviews, four new themes (two generative mechanisms related to the architectural conditioning phase and two integration outcomes) were identified. other generative mechanisms related to soa introduction emerged. however, they are not discussed in this paper due to space limitations and therefore we considered the second phase a black box as shown in figure 1. the new emerged soa integration within ea outcomes are “ea methods and tools” and “ea governance”. it means that soa introduction interaction with the pre-existing architectural conditioning could lead to changes in “ea methods and tools” and/or “ea governance”. the interview findings are presented in the following sections and figure 1 shows the theoretical (extended a-priori) model. t1 architectural conditioning t2 architectural interaction t3 architectural elaboration t4 business architecture is architecture technical architecture ea methods and tools* ea governance* soa introduction ea maturity ea framework* ea objectives* figure 1. enterprise architecture evolution (soa integration) t1 architectural conditioning the interview results showed different architectural conditions of ea (pre-existing ea). besides the hypothesised generative mechanism (ea maturity), two other generative mechanisms emerged from the data. they are ea framework and ea objectives. the results showed that used ea frameworks vary. some of the reported ea frameworks are based on well-known ea frameworks and others are developed in-house. it could be concluded that frameworks developed in-house could restrict the integration outcomes due to the limited external guidance compared to the well-developed ea frameworks such as togaf, which has incorporated soa. it also showed that ea was implemented prior to soa introduction and that ea framework and methodologies have gone through some cycles of changes in theses organisations prior to soa introduction. ea is often implemented prior to soa introduction in organisations. ea as a discipline producing related frameworks and methodologies also existed before soa emerged. in some cases where ea and soa could be implemented at the same time, it is still arguably valid to assume that ea as a structure (its framework, methodology) has existed before soa. it also could be argued that even if an organisation has not implemented a formal ea, there still is an informal ea comprised of applications, business processes and infrastructure its stakeholders are familiar with. the findings also indicated that ea was adopted for various objectives australasian journal of information systems volume 18 number 3 2014 220 based on how it is seen. they are classified into strategic, operational, it and governance oriented eas. the objectives of ea are seen to influence its use and soa integration. for example ea consultant, i16, commented that: “there is a widely differing opinion on what enterprise architecture is and if you start with a different definition of ea you end up with a different relation to soa”. the interview findings also confirmed the relevance of ea maturity in this study as a generative mechanism. ea consultant, i-17, suggested there is a link between the architectural practice and the maturity of ea. he reported that togaf architecture development methodology (adm) is used differently in alignment with the level of ea maturity. he stated: “there’s very few people who use the whole adm but typically people will align their architecture practice at their level of maturity with the relevant aspects of the adm”. ea consultant, i-12, reported based on many engagements with ea implementations that ea is still in its early maturity stages. he elaborated that a low level of maturity of ea impacts the involvement of ea within soa and soa introduction. he acknowledged that “if you don’t have a mature ea capability in organisation, soa is very unlikely to even get a look in…..if you don’t have a good ea function and have it engaged and accountable, in my opinion you cannot get soa properly implemented”, i-12. in summary, the interview findings suggest that there are two other important generative mechanisms besides ea maturity. they are ea framework and ea objectives. the participants reported different ea frameworks e.g. in-house-developed and modified well-known frameworks. some of these frameworks have been changed in many morphogenetic cycles to match organisational changes. ea is also adopted for various objectives and there are different opinions about ea. the findings suggested that ea objectives have influence on soa integration with ea. the findings also supported the proposed hypothesis that ea maturity (through its multiple dimensions) contingently affects the outcomes of soa integration within ea. t2 architectural interaction (soa introduction) t3 the interview findings supported the proposed generative mechanisms and additional generative mechanisms were inductively identified from the interview analysis. in this study, soa introduction is hypothesised to be influenced by related generative mechanisms that lead to different integration outcomes within ea in different contexts. due to space limitations they are not discussed in this paper, but this phase is mentioned here to emphasise that the action (soa introduction) has generative mechanisms that interact with the architectural conditioning mechanisms to generate the different integration outcomes. architectural elaboration t4 (reproduction or transformation) the interview findings confirmed the three architectural elaboration levels (integration outcomes) of the a-priori model. the integration outcomes could be explained by looking at the interaction of the previous two phases (architectural conditioning and architectural interaction). ea is transformed or reproduced due to soa integration on the three architectural levels identified from the literature. they are business architecture, is architecture and technical architecture. also two other levels of architectural elaboration emerged from the interview data. they are “ea methods and tools” and “ea governance”. these two aspects of ea are also found to be transformed or reproduced dependent on the interaction of architectural conditioning and architectural interaction phases in a given context. the details of soa integration within ea outcomes on these five architectural levels are examined in the following paragraphs. australasian journal of information systems volume 18 number 3 2014 221 business architecture the interview findings showed some examples (seven cases) of this level of transformation where the soa is integrated with the business architecture. all the cases that have reported soa integration with the business architecture also integrated soa with the is and technology architectures. for example, participant, i-2, said “we don’t differentiate between soa architecture and ea, you know for us, soa is very much part of the enterprise architecture... you know, until you start getting up into the information and business layers you start achieving true service orientation”,i-2. information system architecture soa integration with the is architecture is also supported in some of the interviews (three cases). all the cases that integrated soa within the is architectures have integrated it with the technology architectures as well. for example, participant, i-11, stated that soa is integrated within the is and the technical architectures. he noted “the applications level has service and [service] components. at the operations level or the infrastructure level we talk about security and monitoring in terms of services”. chief enterprise architect, i-1, also stated that soa is integrated within the integration architecture. he said “integration architecture is where we have these artefacts and everything you described right down to the service descriptions, the slas and the detail, all exist within it”. technical architecture the interview data supported this level of architectural elaboration. the abovementioned cases at the service-orientation of business and is architectures level reported that soa is integrated within ea at this level as well. in addition, participant, i-13, described their soa integration at this architectural level. enterprise architect, i-13, presented that soa implementation started using a point-to-point integration using web services and a couple of years later, an enterprise service bus was introduced. he also reported that their documentation of ea and soa was very limited. he said “we’re not strong; we haven’t been strong in publishing a huge range of artefacts…. on a five point scale we’re somewhere between one, one and a half on soa and probably enterprise architecture”. ea consultant, i-20, also described this limited integration aspect. he said “people look at the service oriented architecture and then straight away they think of the technology adaptors, enterprise service buses and things like that and they forget about the business reason”, i20. ea methods and tools some participants argued that soa introduction requires some changes to the architectural design and development methods and processes. for example, chief enterprise architect, i-1, argued that such changes need to be considered at a very early stage of soa adoption in order to identify and build the right services. he claimed that: “not laying foundations first such as changes to sdlc, governance processes…” leads to “…building the wrong services”, i-1. chief enterprise architect, i-2, argued that upon soa adoption, their guidelines, methods, processes that are required to implement service-oriented projects are established. ea consultant, i-7, also supported the idea of the changes of ea methodology to be able to identify services. he said australasian journal of information systems volume 18 number 3 2014 222 “so what we believe is that after the requirements, the business requirements, we need to have a service identification space. which basically allows us to see what are the solutions that are needed and where can we actually get it... so we believe that there is a service identification and a service realisation phase that need to be a part of the life cycle of it services in the future”. however, participant, i-13, argued that soa is not a new thing that needs changes to ea development methods and processes. this observed integration outcome in some cases is convincing when existing ea methods are integrated with project and solution portfolios. this possible integration outcome is also argued for in the literature. for example, changes to ea methodology are needed to manage the service architecture (postina et al. 2010). to do so, ea needs to be embedded in established it management and processes. in particular, ea practices, roles and artefacts need to be integrated within the existing it management processes and tasks (löhe et al. 2013). ea and projects also need to be integrated to make sure that the project team develops a solution that aligns with the target architecture. this includes the definition of the architectural components, the specification of interfaces and activities needed to implement the solution (ahlemann et al. 2012). ea governance the fifth possible level of architectural elaborations (integration outcomes) is soa governance integration with ea governance. soa governance is an important aspect to be considered when introducing soa. however, the participants have different perspectives on soa governance in relation to ea governance. some participants emphasised soa governance is part of ea governance. for example, participant, i-12, argued that “soa is just an architectural pattern, ea governance is the thing that is going to make this work, so soa is part of your overall enterprise architecture governance”, i-12. enterprise consultant, i-18, declared that soa is delivered as part of ea, and thus ea governance covers and influences soa governance and practices. he said “that [ea] governance should stem into anything that is delivered as part of the enterprise architecture which includes soa”, i-18. however, ea consultant, i-20, argued that soa is a large initiative and requires its own governance practices that are aligned with architectural governance practices. it is also supported by another ea consultant, i-16. he stated that there is a relationship between ea and soa governance on high level aspects, but ea should not get involved in low technical aspects of soa. “enterprise architecture has set certain boundaries within those boundaries you can solve things in many different ways.…and especially when it gets to the lower layers of the soa where you implement the technical services, the low level services i don’t think ea should necessarily have much to say about that”, i-16. this level of integration, whether to integrate or not, is even discussed in recent literature. some argue for the integration of the existing governance mechanisms with soa governance while others argue for a stand-alone governance body for soa (joachim et al. 2013). this could even be used as an argument for the value of this study model, once completed and tested, to be able to explain why soa governance is or is not integrated within ea governance using the hypothesised existence of many generative mechanisms at the first two phases of the model that interact to produce the observed integration outcome. australasian journal of information systems volume 18 number 3 2014 223 conclusion and outlook as a result of the dynamic environment, the increasing pace of change and technological innovation, organisations are faced with huge challenges not only of assessing how their businesses could be affected but also how the emerging trends impact their eas. eas need to co-evolve with organisations in response to new technical and business paradigms. one of the more significant developments in recent years is the emergence of soa. several studies have attempted to provide approaches for how to integrate soa within ea. yet, there are different outcomes, different emphases and different representations of soa elements. moreover, there is a lack of empirical studies that examine ea evolution due to emergent trends. thus, this paper aimed at better understanding the ea evolution through soa by taking a closer look at both literature and practice based on a solid theoretical underpinning for understanding change. it employed archer’s morphogenetic theory to understand the process of ea evolution when soa is introduced. it built the foundations for subsequent empirical phases to further understand and explain ea evolution. through the understanding of ea evolution, the viability of improvements arises. such analysis is helpful to identify whether the architectural settings are needed to be enhanced prior to the dedication to soa introduction. the paper derives its significance and relevance from interrelating important contemporary phenomena based on a theoretical foundation. it uses a sound theoretical underpinning to enrich the understanding of the ea evolution process and outcomes. archer’s morphogenetic theory is employed as an analytical framework to exemplify the interaction between pre-existing architectural settings, the actions taken to introduce soa and the elaborations occurring as a result of that interaction. the main findings of this study are threefold. first, in archer’s terms, architectural conditions at t1, before an soa introduction, are the results of previous actions. these architectural conditions have an influence through their generative mechanisms (ea maturity, ea framework and ea objectives) on the action of agents (soa introduction). the maturity of ea, its framework and objectives either enable or restrict the action of soa introduction between t2 and t3. we hypothesise that the likelihood of comprehensive architectural transformation (integration of soa within ea’s five architectural levels), contingent upon the introduction of soa interaction, increases when (i) the architectural conditions are mature, (ii) the adopted framework is capable of handling soa, is comprehensive and well-defined, and (iii) ea has comprehensive strategic, operational, it and governance objectives. second, the interaction between the architectural settings (t1) and soa introduction (t2-t3) results in the different architectural elaboration (t4) either transformation or reproduction. the existing architectural settings could be transformed or reproduced on one or many of the five architectural outcomes. they are business architecture, is architecture and technology architecture, ea methods and tools and ea governance. third, archer theory is adopted and its analytical separation was used in order to understand how these different integration outcomes are generated. it is a powerful theory that offered a way of analysing ea evolution due to soa introduction in three analytical phases to explain the integration outcomes. a limitation of the paper is the lack of overarching evidence that link the evolution process building blocks because it is out of scope of this paper. the discussion of the second phase (the architectural interaction) was removed due to space limitations. the preliminary findings presented above need to be examined in different contexts to further explore the model and its explanatory power. multiple longitudinal case studies are appropriate to examine the architectural settings before soa introduction, the way soa is introduced through the examination of its related generative mechanisms and their integration outcomes. further, the proposed model could be reused to examine ea evolution as a result of other emerging trends. this could be easily conducted by replacing the inner phase (the architectural interaction) with an emerging trend of interest such as cloud computing or enterprise mobility. australasian journal of information systems volume 18 number 3 2014 224 references ahlemann, f., stettiner, e., messerschmidt, m., legner, c., and lux, j. 2012. "embedding eam into the project life cycle," in strategic enterprise architecture management, f. ahlemann, e. stettiner, m. messerschmidt and c. legner (eds.), springer berlin heidelberg, pp. 141-168. alwadain, a., fielt, e., korthaus, a., and rosemann, m. 2011. "where do we find services in enterprise architectures? a comparative approach. ," in the 22nd australasian conference on information systems (acis 2011). sydney, australia. alwadain, a., fielt, e., korthaus, a., and rosemann, m. 2013a. "a comparative analysis of the integration of soa elements in widely-used enterprise architecture frameworks," international journal of intelligent information technologies (ijiit) (9:2), pp 54-70. alwadain, a., fielt, e., korthaus, a., and rosemann, m. 2013b. "service-oriented architecture integration within enterprise architecture: a-priori model," in the 24nd australasian conference on information systems (acis 2013). melbourne, australia. archer, m. s. 1995. realist social theory: the morphogenetic approach, (cambridge university press: cambridge, uk. buckl, s., ernst, a. m., matthes, f., and schweda, c. m. 2009. "visual roadmaps for managed enterprise architecture evolution," in software engineering, artificial intelligences, networking and parallel/distributed computing, 2009. snpd '09. 10th acis international conference on, pp. 352-357. bygstad, b., and munkvold, b. e. 2011. "in search of mechanisms. conducting a critical realist data analysis," in icis 2011 proceedings. correia, a., and silva, m. 2007. "modeling services in information systems architectures.," in digital enterprise technology perspectives and future challenges p. f. cunha and p. g. maropoulos (eds.), springer.: new york, usa, pp. 157-164. cuellar, m. j. 2010. "using realist social theory to explain project outcomes.," international journal of information technology project management (1:4), pp 38-52. fereday, j., and muir-cochrane, e. 2006. "demonstrating rigor using thematic analysis: a hybrid approach of inductive and deductive coding and theme development," international journal of qualitative methods (5:1), pp 1-11. gosselt, r. 2012. "a maturity model based roadmap for implementing togaf," in 17th twente student conference on it: enschede, the netherlands. infosys. (2009). enterprise architecture expands its role in strategic business transformation, infosys enterprise architecture survey 2008/2009. retrieved february, 05, 2010, from http://www.infosys.com/offerings/it-services/architecture-services/easurvey/pages/index.aspx joachim, n., beimborn, d., schlosser, f., and weitzel, t. 2011. "does soa create or require it/business collaboration? investigating soa’s potential to reduce the gap between it and business," in the 2011 international conference on information systems (icis). joachim, n., beimborn, d., and weitzel, t. 2013. "the influence of soa governance mechanisms on it flexibility and service reuse," the journal of strategic information systems (22:1) 3//, pp 86-101. jung, c. 2009. "actionable enterprise architecture.," in the 10th acis international conference on software engineering, artificial intelligences, networking and parallel/distributed computing, pp. 294-299. khoshnevis, s., aliee, f. s., and jamshidi, p. 2009. "model driven approach to service oriented enterprise architecture.," in the 2009 ieee services computing conference., pp. 279-286. australasian journal of information systems volume 18 number 3 2014 225 kistasamy, c., van der merwe, a., and de la harpe, a. 2010. "the relationship between service oriented architecture and enterprise architecture," in 14th ieee international enterprise distributed object computing conference, pp. 129-137. kistasamy, c., van der merwe, a., and de la harpe, a. 2012. "the role of service oriented architecture as an enabler for enterprise architecture," in the 18th americas conference on information systems seattle, washington kokko, t., antikainen, j., and systa, t. 2009. "adopting soaexperiences from nine finnish organizations.," in 13th european conference on software maintenance and reengineering. löhe, j., and legner, c. 2013. "overcoming implementation challenges in enterprise architecture management: a design theory for architecture-driven it management (adrima)," information systems and e-business management) 2013/02/01, pp 1-37. martin, r., purao, s., and robertson, e. 2009. "evolving enterprise architecture," in next generation enterprise architecture iceimt workshop: bled, slovenia. mckendrick, j. (2010). top 10 enterprise architecture trends, identified by forrester. retrieved march 02, 2011, from http://www.zdnet.com/blog/service-oriented/top-10-enterprisearchitecture-trends-identified-by-forrester/5949 nascio. (2003). nascio enterprise architecture maturity model. retrieved 20 march, 2011 from http://www.nascio.org/publications/documents/nascio-eamm.pdf patton, m. q. 2001. qualitative research & evaluation methods, (sage publications: thousand oaks, ca. perko, j. 2008. it governance and enterprise architecture as prerequisites for assimilation of serviceoriented architecture. phd thesis., tampere university of technology. postina, m., trefke, j., and steffens, u. 2010. "an ea-approach to develop soa viewpoints," in 14th ieee international enterprise distributed object computing conference., pp. 37-46. sampaio, a. 2010. an approach for creating and managing enterprise blueprints. unpublished thesis. sayer, r. a. 1992. method in social science : a realist approach, (2nd ed.) routledge: london. schekkerman, j. 2005. the economic benefits of enterprise architecture., (trafford: victoria, bc, canada. schelp, j., and aier, s. 2009. "soa and ea—sustainable contributions for increasing corporate agility.," in the 42nd hawaii international conference on system sciences., pp. 1-8. shah, h., and golder, p. 2011. "adappt: enterprise architecture thinking for information systems development," international journal of computer science issues (8:1). sousa, p., lima, j., sampaio, a., and pereira, c. 2009. "an approach for creating and managing enterprise blueprints: a case for it blueprints," a. albani, j. barjis and j. l. g. dietz (eds.), springer berlin heidelberg, pp. 70-84. the open group 2009. the open group architecture framework togaf, (van haren publishing: lj zaltbommel, the netherlands. the open group. (2010). definition of soa retrieved january, 20, 2010, from http://www.opengroup.org/projects/soa/doc.tpl?gdid=10632 viering, g., legner, c., and ahlemann, f. 2009. "the (lacking) business perspective on soa–critical themes in soa research. ," in wirtschaftsinformatik. , pp. 45-54. wynn, d., and williams, c. k. 2012. "principles for conducting critical realist case study research in information systems," management information systems quarterly (36:3), pp 787-810. australasian journal of information systems volume 18 number 3 2014 226 an earlier version of this paper was presented at the australasian conference on information systems (acis) 2013 in melbourne, australia. paper title (paper title style) australasian journal of information systems wilson, carlson & sciascia 2017, vol 21, research on indigenous ict reterritorialising social media 1 reterritorialising social media: indigenous people rise up alex wilson university of saskatchewan alex.wilson@usask.ca bronwyn carlson macquarie university acushla sciascia independent researcher social media is transforming the way indigenous peoples interact and connect with each other at a local, regional, national and global level. facebook and other social media facilitate this interaction and allow users to maintain relationships across vast distances and time zones, thereby increasing social and political connectivity and impact (carlson 2013). indeed, social media users transcend time and space, yet they produce content, set up links and connect communities of practice. there is now a constantly networked world in every dimension of human experience. social movements, like every other aspect of life, are becoming increasingly reliant on online networking and information sharing. researchers are beginning to examine the use of social media for political communication and social movements (carlson & frazer 2016). generally, however, such research has focused on movements that have received attention from mainstream media, (such as the #occupywallstreet movement for example) whereas indigenous activism, does not always receive such attention. even though indigenous populations worldwide are avid social media users, indigenous activism and social media research has yet to gain any real traction in academia. the rapid rise of the use of social media as a means of social, cultural and political interaction among indigenous peoples and groups is an intriguing development. while this is not to suggest there is no digital inequality, it does counter any assumptions that indigenous peoples may have little interest in the possibilities of technology and the online environment (carlson & frazer 2015). social media as a ‘new frontier’ is where indigenous peoples are busy interacting and networking in what can be argued as a cultural and political reterritorialisation of social media spaces (carlson 2013). the power and potential of social media to promote indigenous movements on a global level is now manifesting. in the us, the sioux people of north dakota demonstrated effectively— though not yet successfully—against the development of an oil pipeline that would run through traditional homelands. social media is being deployed to gather a global audience of allies who oppose the development for both indigenous justice and climate justice reasons. in australia, led by aboriginal activist sam cook, a movement against the forced closure of aboriginal communities was focused by the hashtag #sosblakaustralia. through effectively using facebook and twitter in combination with on-the-streets protesting, the movement sustained a global audience even while traditional forms of media actively sought to ignore or undermine the movement (carlson & frazer 2016). to increase māori political engagement the mana movement (formally known as the mana party) actively used facebook to raise awareness of political issues and to engage māori now living in australia. in what o’carroll (2013) refers to as “the virtual marae” (a communal meeting ground) māori are using social media to maintain kinship relationships and advocate for maintaining language and other cultural practices around the world. in hawaii, indigenous hawaiians have engaged in a protracted protest against the redevelopment of a major telescope on top of the dormant volcano mauna kea, which is considered a significant sacred site (kamelamela 2016). while protesters used mass blockages and sit-ins to prevent the development, the story of the protests was shared virally through social media—particularly instagram—where photos of the protests were shared by high profile celebrities drawing global attention to the campaign. australasian journal of information systems wilson, carlson & sciascia 2017, vol 21, research on indigenous ict reterritorialising social media 2 in canada the movement idle no more has used social media to highlight issues facing first nations, metis and inuit people. the movement began following the harper government passing bill c-45, which sought to divest first nations sovereign command of lands and waterways. the movement, utilising the hashtag #idlenomore, has used social media to promote environmental justice issues globally (mcmillan, young & peters 2013). other indigenous groups, such as ‘aotearoa in support of idle no more’ and ‘protect mauna kea’ have shown indigenous solidarity with the movements. thus, indigenous groups have deployed social media effectively in diverse efforts to coordinate collective political practice and express political will in a climate where their voices are often ignored. in this special section of the australasian journal of information systems (ajis) entitled, “research on indigenous use of information and communication technologies: reterritorialising social media: indigenous people rise up” authors describe specific aspects of “reterritorialisation”. mainstream media cannot necessarily be counted upon to take interest in issues specific to or of concern to indigenous peoples. social media however, is providing the means whereby indigenous people can “reterritorialise” and “indigenise” the information and communication space. the ability to create international solidarity as well as elevating indigenous issues to a global platform remain key strengths for indigenous activism. the level of visibility social media has given indigenous issues is unprecedented. this special section begins with a powerful poem entitled, “social media exploration 1.1” written by māori poet, performer and activist te kahu rolleston. the poem was written especially for the 2015 international symposium, ‘reterritorialising social media: indigenous people rise up’ which was held at the university of wollongong, nsw australia and was hosted by the forum for indigenous research excellence (fire). the video of rolleston’s performance can be accessed via the journal of global indigeneity at the following link, http://ro.uow.edu.au/jgi/vol1/iss2/9/). led by indigenous australian journalist and scholar jack latimore, david nolan, margaret simons, elyas khan provide a theoretical ground for considering the inherited and shifting spaces of indigenous media representation in their paper, “reassembling the indigenous public sphere”. noting that more indigenous people are choosing digital media as a source of news this paper discusses the rise of indigenous news outlets that cover indigenous issues and events. this paper introduces the wakul app software infrastructure tailored for connecting indigenous communities and highlighting the issues that matter to indigenous people as opposed to what is being covered by mainstream media. following, native american scholar marisa duarte’s article “connected activism: indigenous uses of social media for shaping political change” explores the use of digital tactics in social and political campaigns. drawing on three case studies, duarte compares the kinds of community knowledge work that takes place as part of any online indigenous rights movements. exploring ezln, idle no more, and the ongoing rio yaqui water rights movement duarte reveals the significance of political and social connections forged via digital tools and in doing so reveals that indigenous movements are always grounded in indigenous ways of knowing and understanding the world. indigenous australian scholar bronwyn carlson and colleagues lani jones, michelle harris, nelia quezada and ryan frazer investigate the ways in which indigenous australians respond individually, and collectively, to racial vilification by means of social media sites in their paper, “trauma, shared recognition and indigenous resistance on social media”. they argue that indigenous people collectively experience a sense of anger and frustration when traumatic events in the public domain act as reminders of ongoing colonialism. drawing on sara ahmed’s (2004) notion of “affective economies” this paper unpacks the politics of pain, shame and pride in the aftermath of both the four corners program and the bill leak cartoon. importantly, this paper also focuses on the indigenous led #indigenousdads twitter campaign that emerged in response to these events and demonstrates the use of social media in the ongoing resistance to colonial narratives. australasian journal of information systems wilson, carlson & sciascia 2017, vol 21, research on indigenous ict reterritorialising social media 3 in her paper “protecting oak flat: acts of resilience as observed through digital activism” native american graduate student nicholet deschine parkhurst highlights the ongoing battle to protect lands and resources. this paper focuses on a specific case study of oak flat and how native american activist deployed social media to protest the destruction of sacred lands by mining companies. in this case, as with others, efforts included a combination of on-theground efforts, online efforts, solidarity efforts, and legislative support efforts. the paper, “the whiteness of redmen: indigenous mascots, social media and an antiracist intervention” authored by idle no more co-founder sheelah mclean, cree scholar alex wilson and cree activist erica lee presents the story and analysis of how the grassroots movement idle no more’s (inm) social media power contributed to the successful ‘retirement’ of a racist high school mascot. mclean, wilson and lee, all inm organisers, connect social media and anti-racist education, providing a way for indigenous peoples to intervene and reclaim digital and real time space and representations. eric otenyo’s article “being left behind amidst africa’s rising imagery: the maasai in the world of information and communication technologies (icts)” provides and overview of the exploitation of maasai imagery and culture providing a counter narrative to the idea of “africa rising”. otenyo reminds us that although indigenous groups are active social media users, governments also use social media to present a particular reality of indigenous people. in the example of maasai people, the digital divide is very real with many maasai marginalised and life chances worsening rather than improving. the way maasai people and communities are represented on social media is often out of the control of maasai themselves. however, maasai are utilising online platforms to campaign against injustices. demonstrating how indigenous people use humour as a strategy of resistance scholar jeff berglund showcases the work of native american comedy troupe the 1491s in his paper “‘go cry over someone else’s tragedy’: the youtube activism of the 1491s”. as berglund argues, the 1491s have successfully used social media to reach a global audience and to advocate for positive change for native american people and communities. using humour to challenge mainstream representation of native american people the comedy troupe have been successful in attracting vast attention to their political stance on a variety of topics including the recent #nodapl movement. indigenous australian graduate student andrew farrell discusses the use of social media by indigenous lbgtqi communities in his paper, “archiving the aboriginal rainbow: reflections on building an aboriginal lbgtqi portal and archive online”. drawing on the work of leesa watego, who conceptualised online blogging as a kind of “midden” (watego 2015) this paper explores the layers of knowledges that are shared and stored online via social media. this paper highlights the need for a counter narrative to the negative representation of the lbgtqi community and the power of social media in enabling an “indigenising” of the online world of blogging. in summary, the articles in this special section challenge the reader to engage with on the ground activists and educators who seek to protect indigenous and environmental rights on our traditional territories through the use of the modern technologies of social media. although not necessarily designed for facilitating political activism, social media has become a common entry point for many indigenous led campaigns. it has become a wellspring of new forms of political expression and action, which appear to complement rather than supplant more traditional forms of activism (bennett & segerberg 2012). social media is now undoubtedly a great force for political organisation and transformation allowing for the amplification of indigenous voices. these voices, as represented in this special section, reaffirm indigenous sovereignty and connections to ancestral homelands. indigenous people indeed are reterritorialising social media and as always will continue to rise up. australasian journal of information systems wilson, carlson & sciascia 2017, vol 21, research on indigenous ict reterritorialising social media 4 references bennett, l. w. & segerberg, a. (2012) the logic of connective action: digital media and the personalization of contentious politics. information, communication & society, 15(5): 739-768. carlson, b. (2013) the ‘new frontier’: emergent indigenous identities and social media. in harris, m., nakata, m. & carlson, b. (eds.), the politics of identity: emerging indigeneity, university of technology sydney e-press: sydney. carlson, b. & frazer, r. (2015) “it’s like going to a cemetery and lighting a candle”: aboriginal australians, sorry business and social media. alternative: an international journal of indigenous peoples, 11(3): 211-224. carlson, b. & frazer, r. (2016) indigenous activism and social media: the global response to #sosblakaustraia. in mccosker, a., vivienne, s., & johns, a. (eds.), rethinking digital citizenship: control, contest and culture, rowman and littlefield international: england. kamelamela, k. (2016) the protect mauna kea movement: since before the overthrow in 1893. journal of global indigeneity, 1(2). available at, http://ro.uow.edu.au/jgi/vol1/iss2/5/ mcmillan, l.j., young, j. & peters, m. (2013) commentary: the ‘idle no more’ movement in eastern canada. canadian journal of law and society, 28(3): 429-431. o’carroll, a. d. (2013) virtual whanaungatanga. māori utilising social networking sites to attain and maintain relationships. alternative: international journal of indigenous people, 9(3): 230-245. watego, l. (2015) even the activist’s gotta eat: taking ownership, building platforms. journal of global indigeneity, 1(2). available at: http://ro.uow.edu.au/jgi/vol1/iss2/7/ copyright: © 2017 wilson, carlson & sciascia. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems thomas, burmeister, islam, dayhew & crichton 2016, vol 20, post publication review client welfare & communication of mental health data post publication review burmeister, o.k., islam, m.z., dayhew, m., & crichton, m. (2015) enhancing client welfare through better communication of private mental health data between rural service providers. australasian journal of information systems, 19. doi: http://dx.doi.org/10.3127/ajis.v19i0.1206 review this paper seeks to highlight and advance the ongoing debate of undertaking multi practitioner service delivery in regional, rural/remote locations and the impact of effective data warehousing and sharing technologies. what is apparent from the research undertaken by burmeister, islam, dayhew and crichton (2015), is the fragmentation of communication and information exchange. as proposed in the paper, one way to solve this is the instigation of technical software capability to bridge the existing service delivery data exchange divide. the authors do clearly articulate the need to consider the privacy, confidentiality and patient/client rights who are accessing services both on a voluntary or involuntary basis. the discussion regarding the ethical and unsolicited receipt of private client data is key, however, i see the need to challenge the assumption that professionals do not have a role in maintaining their professional competence around changes to privacy legalisation. whilst what is proposed is a significant step forward, it does not negate the need to maintain contemporary knowledge for practice. what is demonstrated in this paper are the clear practice points implications such as practitioners maintaining knowledge of their statutory obligations, the handling of personal information across vast distances, and the mobility of clients across these rural/regional area that was utilised in the sample population (p. 3). the paper may also be a platform to challenge the raised service delivery issues on a policy level and advocate for less metro centric policy and service delivery (russell, humphreys, ward, chisholm, buykx, mcgrail, & wakerman, 2013; smith, humphreys, & wilson, 2008; wakerman, humphreys, wells, kuipers, entwistle, & jones, 2008). the methodological approach was a sound one. the use of a qualitative approach in this particular case assisted the researchers to build meaning from the data collection. it also provides the reader with clear case scenarios and vignettes throughout that highlight the issues at hand in a way and directly links issues and solution building, both from a human service delivery respond, and technological infrastructures and applications. in examining the proposed data capturing, sharing, warehousing functionality, this is a dramatic step forward, but the paper raises further questions. how does this application interface with existing ehealth systems and applications at both commonwealth and state levels (australian department of health, 2016; nsw government, 2016)? who determines user access and on what level of access? if his model does in fact successfully achieve data integration with a range of other existing platforms and provides an upgrade of functionality and local responsive to existing service systems and providers, then such technology should be further explored. what is refreshing is that the application has multiple functionality and actually goes beyond just improving communication between practitioners to deliver better outcomes. if the forecasting ability is in fact functional then this would assist human service delivery agencies (and academics) to examine forecasts and trends of potential health and social implications into the future such as the example described on pages 10-11. but again the question is raised in terms of risk management… who is actually responsibility for the data, who manages the upkeep of the data sharing capability and further questions are raised regarding who determines the indicators for forecasting purposes. for example, different health professionals will have different parameters and determinants of health and well-being. so this presents the question of purpose – what comes first – academic considerations, professional applicability 1 australasian journal of information systems thomas, burmeister, islam, dayhew & crichton 2016, vol 20, post publication review client welfare & communication of mental health data or client outcomes? is this process one of default standardised decision making (falzer, 2013; mossman, 2015)? more contextual discussion behind the qualitative aspects of the research would have provided additional insight to the diversity and breath of professionals that could potentially utilise this software program, as to, more conversation around the integration and functionality of existing systems. coming from a human services perspective, the paper, even though clearly aiming to improve and enhance client outcomes for those accessing mental health services in the nominated catchment areas, could have strengthened practical examples of how this in fact would take place. the authors are to be commended on taking a practical approach to resolving a local referral and communication issues for patients and service providers with the generation of software which responds directly to the need. however, from a human services systems view, the software needs to be seen as valuing-adding to existing online, data warehousing systems that are available as opposed to a separate system that may be in competition as if not may potentially open an information gap. whilst there is no doubt of the competence of the software development team, and of course due consideration given to ethical and privacy legislation, further discussion and debate of the practicalities of implementation is required. catherine thomas charles sturt university cthomas@csu.edu.au references australian department of health. (2016). digital health. retrieved 9 march 2016, from http://www.health.gov.au/internet/main/publishing.nsf/content/ehealth burmeister, o. k., islam, m. z., dayhew, m., & crichton, m. (2015). enhancing client welfare through better communication of private mental health data between rural service providers. australasian journal of information systems, 19, 1-14. doi: http://dx.doi.org/10.3127/ajis.v19i0.1206 falzer, p. r. (2013). valuing structured professional judgment: predictive validity, decisionmaking, and the clinical-actuarial conflict. behavioral sciences & the law, 31(1), 4054. doi: 10.1002/bsl.2043 mossman, d. (2015). from group data to useful probabilities: the relevance of actuarial risk assessment in individual instances. j am acad psychiatry law, 43(1), 93-102. nsw government. (2016). hsnet. retrieved 9 march 2016, from https://www.hsnet.nsw.gov.au/ russell, d. j., humphreys, j. s., ward, b., chisholm, m., buykx, p., mcgrail, m., & wakerman, j. (2013). helping policy-makers address rural health access problems. australian journal of rural health, 21(2), 61-71. doi: 10.1111/ajr.12023 smith, k. b., humphreys, j. s., & wilson, m. g. a. (2008). addressing the health disadvantage of rural populations: how does epidemiological evidence inform rural health policies and research? australian journal of rural health, 16(2), 56-66. doi: 10.1111/j.14401584.2008.00953.x wakerman, j., humphreys, j. s., wells, r., kuipers, p., entwistle, p., & jones, j. (2008). primary health care delivery models in rural and remote australia – a systematic review. bmc health services research, 8(1), 1-10. doi: 10.1186/1472-6963-8-276 author response we appreciate the thorough and generally affirming review. the issue of challenging the assumption that practitioners do not have a role in maintaining their professional competence around changes to privacy legislation is not what we were trying to say. rather, our article 2 australasian journal of information systems thomas, burmeister, islam, dayhew & crichton 2016, vol 20, post publication review client welfare & communication of mental health data simply states that it is difficult for service providers to keep up with legislative changes, not that they don’t have such a role. linking to the ehealth system is an interesting concept that we didn’t explore and that will be a reasonable future action when it is recognised that ehealth is being widely adopted. the issue of forecasting and data use also raises an opportunity for future exploration and managing the diverse paradigms that exist over the various professional disciplines with which a patient with a mental health issue may interact. data management and security are directed by the legislation so these are a given and would be attended in the development of any proprietary offering. as the review indicates, our concern was for the clients. whilst early detection of mental illness, particularly in the young, will often facilitate return to health, recovery, that is learning to live with the illness and managing its effects is the experience of many, particularly people who experience such in later life (burmeister & marks, 2016; young & ensing, 1999). many consumers of mental health services can be considered vulnerable users and for some such vulnerability may even increase (bernoth, dietsch, burmeister, & schwartz, 2014). another example recently seen in this journal was the vulnerability of people with suicidal ideation (carlson, farrelly, frazer, & borthwick, 2015). thus a system that protects consumer privacy is important and even more so for vulnerable consumers: the young, people with intellectual disabilities, and older people with neuro-degenerative diseases, as shown in various studies (pakrasi, burmeister, mccallum, coppola, & loeb, 2015; teipel et al., 2016; van wynsberghe, 2015). it is a pleasure to see that the review acknowledges the contribution of the proposed private data sharing model in going beyond just an improved communication system between various parties through proposing backend data mining facilities through classification (siers & islam, 2015; adnan & islam, 2016), clustering (rahman & islam, 2014) and data pre-processing (rahman & islam, 2012) for better knowledge discovery (beg & islam, 2016). various privacy preserving data mining techniques (islam & brankovic, 2011; fletcher & islam, 2015) can also be modified and used in this scenario to enhance privacy while allowing knowledge discovery. regarding the question raised in the review on the authority of the forecasting parameter selection, we suggest that different parties can choose their parameters as they need and then can get their data mining results accordingly. the suggestion made by the review for interfacing the existing e-health systems is a very reasonable one. although we did not explore this in detail in this paper we always had that in mind. a more specific study on this can definitely be an interesting future work. oliver burmeister charles sturt university oburmeister@csu.edu.au md zahidul islam charles sturt university zislam@csu.edu.au miriam dayhew charles sturt university mdayhew@csu.edu.au merrilyn crichton charles sturt university mcrichton@csu.edu.au references adnan, m. n. and islam, m. z. (2016): forest cern: a new decision forest building technique, in proc. of the 20th pacific asia conference on knowledge discovery and data mining (pakdd) 2016, april 19-22, auckland, new zealand. 3 australasian journal of information systems thomas, burmeister, islam, dayhew & crichton 2016, vol 20, post publication review client welfare & communication of mental health data beg, a. h. and islam, m. z. (2016): a novel genetic algorithm-based clustering technique and its suitability for knowledge discovery from a brain dataset, in proc. of the ieee congress on evolutionary computation (ieee cec 2016), vancouver, canada, july 24 29, 2016. bernoth, m., dietsch, e., burmeister, o. k., & schwartz, m. (2014). information management in aged care: cases of confidentiality and elder abuse. journal of business ethics, 122, 453-460. doi: 10.1007/s10551-013-1770-7 burmeister, o. k., & marks, e. (2016). rural and remote communities, technology and mental health recovery. journal of information, communication and ethics in society, 14(2). carlson, b. l., farrelly, t., frazer, r., & borthwick, f. (2015). mediating tragedy: facebook, aboriginal peoples and suicide. australasian journal of information systems, 19. doi: 10.3127/ajis.v19i0.1174 fletcher, s. and islam, m. z. (2015): a differentially-private random decision forest using reliable signal-to-noise ratios, in proc. of the 28th australasian joint conference on artificial intelligence (ai 2015), canberra, australia, 30 november 4 december, 2015, lecture notes in computer science (lncs), pp. 192-203, vol. 9457, isbn 978-3-31926350-2. islam, m. z., and brankovic, l.(2011): privacy preserving data mining: a noise addition framework using a novel clustering technique, knowledge-based systems vol. 24, issue 8, isbn 0950-7051,(december 2011) pg. 1214-1223. pakrasi, s., burmeister, o. k., mccallum, t. j., coppola, j. f., & loeb, g. (2015). ethical telehealth design for users with dementia. gerontechnology, 13(4), 383-387. doi: 10.4017/gt.2015.13.4.002.00 rahman, m. a., and islam, m. z. (2014): a hybrid clustering technique combining a novel genetic algorithm with k-means, knowledge-based systems, vol. 71, november 2014, pp. 345-365. rahman, m. g., islam, m. z., bossomaier, t., and gao, j. (2012): cairad: a co-appearance based analysis for incorrect records and attribute-values detection, in proc. of ieee international joint conference on neural networks (ijcnn 12), brisbane, australia. june 10 june 15, 2012, pg. 2190-2199. siers, m., and islam, m. z. (2015): software defect prediction using a cost sensitive decision forest and voting, and a potential solution to the class imbalance problem, information systems, vol. 51, pg. 62-71. teipel, s., babiloni, c., hoey, j., kaye, j., kirste, t., & burmeister, o. k. (2016). information and communication technology solutions for outdoor navigation in dementia. alzheimer's & dementia: the journal of the alzheimer's association, 1-13. doi: 10.1016/j.jalz.2015.11.003 van wynsberghe, a. (2015). healthcare robots: ethics, design and implementation: ashgate publishing, ltd. young, s., & ensing, d. (1999). exploring recovery from the perspective of people with psychiatric disabilities. psychiatric rehabilitation journal, 22(3), 219–231. copyright: © 2016 thomas, burmeister, islam, dayhew & crichton. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. 4 http://creativecommons.org/licenses/by-nc/3.0/au/ http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems thomas, burmeister, islam, dayhew & crichton 2016, vol 20, post publication review client welfare & communication of mental health data 5 paper title (paper title style) australasian journal of information systems o riordan, lohan & conboy 2015, vol. 19, selected papers from acis mind the gaps mind the gaps: increasing the impact of is research on isd performance improvement niamh o riordan ucd school of business university college dublin dublin 4, ireland niamh.oriordan@ucd.ie garry lohan lero research centre & whitaker institute nui galway galway, ireland garry.lohan@nuigalway.ie kieran conboy je cairnes school of business & economics nui galway galway, ireland kieran.conboy@nuigalway.ie abstract poor performance has pervaded the last forty years of software development, evident across industry sector, project size, budget, geographic location, system quality and functionality, and exacerbated by increased criticality of information technology (it) in organisational mission and strategy. a significant body of research has investigated the potential of emerging development methodologies to address these shortcomings but the effectiveness of these methods is largely supported by anecdotal evidence. at the same time, metrics and measurement are known to affect isd performance but the existing literature on information systems development (isd) metrics is misaligned with practitioners’ needs, leading to a lack of clarity about isd metrics in practice. this paper presents an interdisciplinary literature review on isd metrics to identify the underlying reasons for this misalignment and evaluate the extent to which existing literature can be used to better understand the impact of emerging software development methodologies on isd performance. keywords: information systems development, performance improvement, metrics, measurement 1 introduction investments in isd are substantial but poor performance is a pervasive and significant problem (bharadwaj, keil and mähring, 2009). in the past decade, a significant body of research has investigated the potential of emerging software development methodologies (e.g. xp and scrum) to address these issues but partly due to uncertainty regarding the applicability of traditional measurement practices (kulas, 2012), the effectiveness of these methods is largely supported by anecdotal evidence (lee and xia, 2010). at the same time, an extensive body of research attests to significance of measurement in software development processes (scotto et al., 2005; harjumaa et al., 2008; hazzan and hadar, 2008). however, the existing literature on isd metrics and measurement practices is misaligned with practitioners’ needs (johnson et al., 2005). this has led to a persistent lack of clarity about isd metrics in practice, which is recognized as one of main causes of poor isd performance (jalali and wohlin, 2011; olague et al., 2007; bertoa et al., 2006). to address these issues, this paper presents an interdisciplinary review of the peer reviewed studies on isd metrics that have been published in leading information systems (is) and software engineering (se) journals from 2001 to 2013. its main aim is to identify some of the underlying reasons for this misalignment and also 1 australasian journal of information systems o riordan, lohan & conboy 2015, vol. 19, selected papers from acis mind the gaps to evaluate the extent to which existing literature can be used to better understand the impact of emerging software development methodologies on isd performance. the paper begins with a discussion of the role of emerging software development methodologies and software measurement programs in achieving isd performance improvements, highlighting the need for research on the misalignment of the existing literature on isd metrics with practitioners’ needs. the literature review process and analytical framework are described. in the next section, the analysis identifies and explains the existence of significant gaps in the literature, arguing that these gaps effectively limit the potential impact of the literature on practice. the paper concludes with a discussion of directions for future is research on isd metrics. 2 problem statement information systems have become ubiquitous and are recognized as important enablers of both global productivity improvements and lifestyle advancements. thus, investments in isd are not only significant but are set to increase in the coming years (gartner, 2013). as a result, isd practitioners have a growing need to protect their investments in isd project, to maximise the return on their investments and to ensure that their investments have a significant and sustained effect on their competitive positioning. by extension, is research that can maximise the return on those investments is of great practical significance. nevertheless, practitioners have also struggled to overcome persistent and significant problems with poor isd project performance: organizations continue to undertake projects that consume more resources than they should before they are either cancelled or turned around, produce systems and platforms that are poorly aligned with stakeholder requirements or non-compliant with demands unique to the domain. thus, the direct annual cost of isd project failure has been estimated to be over $6 trillion (sessions, 2009). in 1995, for example, the standish group report estimated that more than half of all isd projects would cost nearly twice the originally estimated cost and that 30% would ultimately be cancelled (the standish group, 1995). though a more recent study (chaos, 2011) indicates that performance improvements have been achieved, it reports that only 37% of isd projects meet scope, time and cost goals. further, that one fifth of all projects (21%) are ultimately cancelled. the persistence of these problems is due to the numerous challenges faced by is developers including variability in project size and duration, emerging trends in the areas of open systems, web services and distributed configurations, decreasing product lifecycles, and increasing requirements volatility (port and bui, 2008; petersen and wohlin 2010). as well as this, there is a lack of strong theoretical and conceptual foundations to many studies of is development methods (abrahamsson et al., 2009, conboy, 2009). in the absence of sound, systematic research, there are few lessons learned across studies, and thus, the existing body of knowledge is somewhat fragmented and inconclusive. 2.1 agile methods: a possible solution? in the past decade, a significant body of research has investigated the potential of emerging software development methodologies (e.g. xp and scrum) to address these issues. these agile methodologies promise to deliver higher customer satisfaction, lower defect rates, faster development times and a solution to rapidly changing requirements by means of a shift in the “centre of gravity" of software engineering from creating a technology-cantered solution toward satisfying the stakeholders (suryn and abran, 2003). though agile methodologies are now used by the majority of software development teams, the effectiveness of these methods is largely supported by anecdotal evidence and rhetorical argument (lee and xia, 2010). this is partly due to uncertainty regarding the applicability of traditional measurement practices in agile settings (kulas, 2012). more specifically, it is argued that (continuous) measurement of both product and process is inherent in agile software development processes (kulas, 2012) and that traditional modes of evaluation may either be incompatible with agile values and principles (hartmann and dymond, 2006) or cannot be 2 australasian journal of information systems o riordan, lohan & conboy 2015, vol. 19, selected papers from acis mind the gaps directly transferred into agile development (nerur and balijepally, 2007; kunz et al., 2008). the use of agile methods is also often superficially judged as used or not used, whereas the actual implementation can be subtle, partial and inconsistent, and so categorising a method as used or not used may be overly simplistic (conboy and fitzgerald, 2010). as a result, few agile methods studies have used metrics to evaluate software product or process quality (olague et al., 2007) and the research on the effectiveness of agile methods “is yet to yield significant systematic and insightful knowledge that can either guide future research or inform effective adoption and use of these methods in practice” (wang, conboy et al., 2010). in practice, agile teams therefore continue to struggle to measure performance (downey and sutherland, 2013). 2.2 isd metrics: a possible solution? in recent years, there has also been a dramatic increase in the number of isd metrics used to drive isd performance improvements in practice (gencel et al, 2013). these metrics, sets of figures or statistics used to measure results (oed, 2013), are used by developers to provide a quantitative basis to support the development, control and validation of software processes and products. similarly, project managers can also use them to assess status, track risk, uncover problems before they become critical, adjust work flows and tasks, and evaluate the team’s ability to control quality. in addition, other organisational stakeholders including customers and program managers can use metrics to better understand and control the software process and products as well as the relationships between them (basili, 1992). though the bulk of the literature on isd metrics focuses on the metrics themselves, a significant body of research investigates the efficacy of isd metrics and evaluates their impact on isd performance (cf. harjumaa et al., 2008; hazzan and hadar, 2008). however, the results of these studies are inconclusive. on the one hand, these studies show that isd metrics can have a positive impact on project performance by facilitating improved resource estimation (boehm et al., 2002), improved resource allocation (basili, 1996) and improved information for decision making (appari and benaroch, 2010). on the other hand, they problematise the validity of the metrics themselves in some instances (e.g. fenton, 1994). (more recently, meneeley et al., (2012) identify 47 validation criteria to help identify valid metrics). they also reveal that isd metrics can have a negative effect on isd performance. for example, hartmann and dymond (2006) argue that inappropriate measurement practices can drive dysfunctional behaviours, such as wasting resources and distorting team behaviours in counter-productive ways. similarly, harrison (2004) cautions that particular metrics may become obsolete over time or may require different interpretations depending on the given context. thus, a number of authors have pointed to this lack of consistency and clarity surrounding isd metrics (cf. jalali and wohlin, 2011; olague et al., 2007; bertoa et al., 2006). for example, johnson et al (2005) argue that practitioners face various barriers in selecting and implementing isd metrics that are inadequately addressed in the literature, pointing to the existence of a misalignment or gap between isd metrics research and practice. to address these issues, this study presents a literature review that identifies some of the underlying reasons for this misalignment and evaluates the extent to which existing knowledge of isd metrics can be used to better understand the impact of emerging software development methodologies (e.g. agile methods) on isd performance. our approach is described in the next section. 3 research design in order to better understand the reasons for the misalignment or gap between the isd metrics research and practice, we conducted an interdisciplinary review of the literature on isd metrics. our approach was informed by both fink’s general recommendations and by the recommendations of kitchenham et al. (2007) and petersen (2011) in terms of se reviews. more specifically, we began by defining (i) a search strategy and (ii) the inclusion and exclusion criteria that could be used to govern the selection of specific articles. once we had identified 3 australasian journal of information systems o riordan, lohan & conboy 2015, vol. 19, selected papers from acis mind the gaps the relevant articles, we analysed them using an analytical framework in order to both ‘map’ and ‘review’ the literature (cf. petersen, 2011). more specifically, we mapped the literature by imposing a structure upon it and analysing the distribution of individual studies within that structure and we reviewed the literature by evaluating the evidence within those studies in order to derive particular recommendations to guide future research. each of these activities is discussed in turn. 3.1 search strategy and paper selection dybå et al. (2007) propose four phases to define a comprehensive search strategy. in the first phase a search string is defined to search for relevant articles. in the second and third phase, inclusion and exclusion criteria are applied to the articles based on title (phase 2) and abstract (phase 3). finally, the relevant articles are retrieved for an in-depth evaluation. in this study, we began by identifying the leading information systems (is) and software engineering (se) journals. it was necessary to consider both schools as they both occupy the domain of isd in terms of research and practice (barry and brown, 2003). in is, we analysed eight leading is journals in the senior scholars' basket of journals: management information systems quarterly (misq), information systems research (isr), journal of the association for information systems (jais), european journal of information systems (ejis), information systems journal (isj), journal of management information systems (jmis), journal of strategic information systems (jsis), journal of information technology (jit). we were less familiar with the leading journals in se so our selections were informed by similar literature reviews in the area (cf. petersen, 2011). more specifically, we analysed the acm transactions on software engineering and methodology (tosem), ieee transactions on software engineering (tse), ieee software (ieeesw), journal of software evolution and process formerly software process: improvement and practice (jsep). we subsequently added two journals because our analysis of the literature revealed that many of the most heavily cited articles on isd metrics in both is and se appear in these journals. these were the journal of systems and software (jss) and the journal of information and software technology (ist). study selection criteria are intended to identify those primary studies that provide direct evidence about the research question. in order to reduce the likelihood of bias, we defined the selection criteria prior to the review. we developed a protocol which specified the search strategy, search string, and inclusion and exclusion criteria. we implemented the research protocol by searching all articles published in the chosen journals between 2001, the year the agile manifesto was released, and 2013. the titles, abstracts and keywords of the journal papers were searched and the search string used was formulated as follows (search strings were adapted to match the individual requirements of each of the search engines used): ((“metric” or “measure” or “indicator”) and (“agil*” or “incremental” or “scrum” or “extreme” or “software develop*” or “information systems develop” or “iterative”)) the abstracts, titles and keywords of the result set were merged and duplicates removed. in order to minimise any bias in the search procedure and to ensure that the result set covers the literature on isd metrics, we preselected a set of papers that should be picked up by a thorough search procedure and then checked to see that they were returned by the search. we also analysed the reference lists of the included articles to identify any additional papers that might have been inappropriately excluded from the search. figure 1 shows the number of studies identified at each stage. in phase 2 and 3 of the study, inclusion and exclusion criteria were applied to the titles and abstracts of the papers (see figure 1). as noted by brereton (2007) the standard of abstracts is often too poor to rely on when selecting primary studies. we therefore examined the conclusions of papers at this stage. the reading of titles, abstracts and conclusions was done by two researchers to minimise the possibility of bias when including or excluding articles. when there was any disagreement we discussed the issues until we reached agreement. 4 australasian journal of information systems o riordan, lohan & conboy 2015, vol. 19, selected papers from acis mind the gaps inclusion criteria: the paper appears in one of the journals listed above and (the paper addresses one or more isd metrics, defined as figures or statistics used to measure results in a substantial way or the paper addresses one or more isd measures, defined as standard units used to express the size, amount or degree of some aspect of an isd process or product, in a substantial way or the paper addresses one or more isd indicators, defined as a thing that indicates the state or level of some aspect of an isd process or product, in a substantial way) exclusion criteria: the paper is an introduction to a special issue or a book review or the paper is not centrally concerned with isd metrics or measurement issues figure 1: the process of applying inclusion and exclusion criteria to papers analysed in this review 3.2 in-depth evaluation at this stage, the relevant articles were retrieved for an in-depth evaluation. a predefined extraction form was used to extract data from each of the 57 primary studies included in this review. in the remainder of this document, sources that are referenced using square brackets are part of the literature review whereas sources referenced in the normal fashion were not. the following details were extracted: bibliographic reference: author, year, title, source; number of citations; primary research focus: technical, non-technical; primary focus of the paper: validation, selection, utilization, evaluation; metrics used; research method: qualitative, quantitative (experiment, survey); data collection method: questionnaire, observation, interview. we also extracted the type of metric used. two types of metric are commonly distinguished in the literature: (i) process metrics are aimed at reducing risk and gaining confidence in the process as the isd process progresses and (ii) product metrics are aimed at ensuring a high-quality product (hazzan and hazard, 2008). in our initial evaluation of the literature, we found that a third type of metric is prevalent. we refer to this as a productin-process metric. this type of metric is calculated using the product as it is being created during the isd process. as such, our analysis distinguishes between three types of metric: (i) process metrics, (ii) product-in-process (mid-process or interim product) metrics, and (iii) product-in-use (post release) metrics. finally we assessed the research design. the evaluation was also conducted by two researchers working independently at first and then collaboratively. where ambiguities arose, the argumentation presented in the paper itself was analysed and the apparent perspective of its authors was adopted. in [12] for example, one of the researchers felt that architectural quality over time could be classified as product-in-use metric but the intent of the paper was to attempt to “foresee the possible impact of changes before their realization” and so the metric was classified as a product-in-process metric. 5 australasian journal of information systems o riordan, lohan & conboy 2015, vol. 19, selected papers from acis mind the gaps 4 findings and discussion we analysed 57 publications, the bulk of which were published in the two journals straddling information systems (is) and software engineering (se). though the number of relevant publications is small given the duration and number of journals studied, figure 2 illustrates that the significance of the topic manifests in terms of citations. in the field of is, each of the papers receives an average of eight citations per year taking into account the number of years since publication. this is significantly greater than the impact factor for each of the is journals included in the sample and cannot be explained in terms of outliers: all ten papers had an average of at least three citations per year. information systems (n=10) info sys / software eng (n=30) software engineering (n=17) ejis (1.59) isj (1.38) jmis (1.26) jit (3.53) n=3 n=3 n=2 n=1 misq (4.66) isr (2.14) jais (2.25) jsis (1.50) n=1 n=0 n=0 n=0 jss (1.24) ist (1.42) n=19 n=11 tse (3.37) ieeesw (2.58) tosem (1.55) swep (1.27) n=11 n=1 n=0 n=5 figure 2: the configuration of found papers across is and se figures 2(a) and 2(b) present an overview of the literature. figure 2(a) compares the fields, analysing the metric type(s) studied (row 1) and the research design employed (row 2) in each field. our analytical framework distinguishes between several experimental designs that may not be familiar to is researchers including in vitro, in silico, in vivo and in situ studies(see above for further information). in our assessment of research design, we wanted to be able to clearly distinguish the different types of experimental methods that are used in the studies so we distinguished between eight different research designs. in particular, our analytical framework distinguishes between in vitro, in silico, in vivo and in situ studies, which may not be familiar to is researchers. in vitro studies use data gathered from industrial setting but generate observations outside of those settings; in silico studies use simulated data rather than data from industrial settings for simulation, in vivo studies approximate industrial settings to an extent (e.g. by using student participants) and in situ studies are carried out in an industrial setting. figure 2(b) examines the literature as a whole, analysing the research designs used to study each category of metric. 6 australasian journal of information systems o riordan, lohan & conboy 2015, vol. 19, selected papers from acis mind the gaps figure 2a: comparing the metrics studied and methods used in is and se figure 2b: analysing the methods used to study software metrics in the two fields in terms of the metrics, 40 studies focus on a single type of metric with the remainder focusing on two or more metric types. a closer inspection of figure 2(a) reveals that the prevalence of studies investigating a single type of metric is similar in both fields. but there is a stronger emphasis on process metrics in is than in se, which is more balanced in its coverage of the different metric types. 17 studies focus on multiple metric types and they are evenly distributed across the fields. over half examine process metrics together with product-in-process metrics but only three examine all three metric types and only one combines product-in-process and 7 australasian journal of information systems o riordan, lohan & conboy 2015, vol. 19, selected papers from acis mind the gaps product-in-use metrics. a wide variety of research designs has been used but 21 of the 57 studies use an in vitro approach. survey-based studies (n=9) and in situ studies (n=11) are also common, though mixed methods are rare (n=2). a closer inspection of figure 2(a) reveals that there are more in vitro studies in the is/se domain (n=14) than in is (n=1) or in se (n=5) while figure 2(b) reveals that there is less diversity in the methods used to study product-inuse metrics or to study different combinations of metric types compared to the methods used to study process and product-in-process metrics. in particular, 12 of the 13 papers studying product-in-use metrics rely on in vitro methods. due to space constraints, we are unable to present a more in-depth analysis of the studies, but would like to draw the reader’s attention to some key points that help explain the misalignment of literature with practitioner needs and suggest particular recommendations for is research on metrics going forward. 4.1 the literature is heavily focused on the identification and validation of metrics many of the papers in both is and se are primarily designed to identify and validate new metrics ([18] and [55] are typical of papers in this style). in addition, the majority of the papers in our sample focused on metrics that would evaluate the functionality of software rather than the usability of software ([23] is an interesting exception). this is unfortunate because significant preventable losses are incurred every year when companies build highly functional and technically successful systems that are nevertheless unused or underused because of poor usability (markus and keil, 2004). in addition, a significant portion of the studies focus on a specific metric with only a handful of papers comparing individual metrics (e.g. [13]) to one another or comparing suites of metrics to one another (e.g. [35]). as a result, there is little or no integration across studies and the literature as a whole lacks coherence. this means that there are important gaps in the literature that go undetected our analysis did not find any papers that proposed metrics for program level management, for example. but it is especially problematic for practitioners who must integrate assessments of (sets of) process and product metrics into the software development process and struggle to clearly define measures that can be derived directly from evolving engineering artefacts and fully integrated into all activities and teams (royce, 2002). 4.2 the literature is heavily focused on particular types of research question many of the papers have a strong technical focus (our analysis found that 38 papers had an exclusively technical focus), frequently overlooking managerial and human (individual or social) measures of performance. this is problematic because of the complex relationship that exists between social interaction and software development success [44] and because “human aspects are the source of the majority of problems associated with software development projects” [19]. as a result, many of the papers fail to provide an adequate description of the context in which their studies are carried out. this is problematic given the diversity of the domain and the broad range of contextual factors such as application domain and application criticality, research methodology employed, programming language used, age and physical distribution of developers that affect the software development product and process. one exception is [12] which takes into consideration the fact that different components of the software being measured are to be used in mixed critical contexts. thus, many of the papers fail to provide adequate insights to practitioners on when, where and how to use a particular metric or suite of metrics. in particular, the literature provides very little guidance on maximizing the utility of metrics in practice and sheds very little light on the impact of measurement programs on organisational behaviour. 4.3 the literature is heavily focused on particular types of research design finally, our analysis suggests that many of the shortcomings listed above have arisen because a significant proportion of the studies rely on data that has been extracted from industrial settings but is analysed in artificial settings. whilst this approach is well suited to the identification and validation of software metrics, it is of little use in terms of (i) identifying principles to guide practitioners on the selection, utilization and evaluation of metrics and 8 australasian journal of information systems o riordan, lohan & conboy 2015, vol. 19, selected papers from acis mind the gaps measurement programs in practice and in term of (ii) developing a better understanding the organisational impact of measurement programs and practice. at the same time, a number of studies carried out either in situ or in collaboration with practitioners have produced some significant findings. several studies, for example, report on practitioners’ efforts to streamline their measurement programs and to reduce the number of metrics used without reducing measurement program performance (cf. concas et al., 2008; staron, 2012; kehan et al., 2011). in one well known study that was carried out in microsoft, eight organisational metrics were found to be more effective predictors of fault proneness than traditional metrics (cf. nagapan et al., 2008). 5 conclusion this paper highlights the positive impact that metric and measurement programs have had on isd project performance outcomes and the potential of isd metrics research to contribute to the reduction of costly isd project performance issues in practice. the paper also suggests that isd metrics may have a role to play in evaluating the impact of emerging is development methodologies (e.g. agile and model driven development) on isd project performance, particularly in comparison with more traditional approaches. nevertheless, our initial analysis reveals the existence of a significant gap between the contributions that are made by the existing isd metrics literature and the needs of practitioners. the paper therefore presents an interdisciplinary review of 57 peer reviewed studies on isd metrics that have been published in leading information systems (is) and software engineering (se) journals between 2001 and 2013. one of the main features of the paper is that it both maps and reviews the literature. the mapping of the literature is done so that a clear picture can emerge of the topography of the field and the gaps within it. this analysis is then complemented by the literature review which done so that the underlying reasons for these gaps can also emerge. though the study has its limitations, the composition of the sample size and the exclusive reliance on journal publications are perhaps its most significant limitations, a number of important observations arise from the analysis that significantly enhance the capacity of is research to impact upon isd performance improvement practice. the citation analysis confirms that there is significant interest in existing literature on isd metrics. this underscores the practical and theoretical significance of research in this area and should encourage both editors and researchers to publish in the domain. the analysis also shows that in many instances, researchers fail to provide adequate information on software development methods that had been used in their studies. this is at least partly due to the reliance on research designs that are based on artificial data or are carried out in artificial settings but if the research is to be used to evaluate the impact of is development methodologies on isd project performance then researchers should be encouraged to more carefully distinguish between traditional, plan driven, isd settings and other settings. the main contribution of the paper, however, is in confirming that the majority of isd metrics studies in the sample are designed to contribute a narrow technical validation of specific isd metrics rather than to inform the selection, utilization or evaluation of them. it also identifies some of the reasons why this is the case. first, the primary research focus of the studies is an issue with the majority of the studies in the sample having a purely technical focus and with only a tiny minority of published papers examining purely non-technical (i.e. individual, social, behavioural and organisational) issues. second, the number and configuration of metrics studied is an issue. more specifically, a significant proportion of the studies focus on a single metric or on a single type of metric. the studies that focus on two types of metric typically focus on particular combinations of metric type (e.g. they investigate process metrics in conjunction with product-in-process metrics) and very few studies examine all three types of metric at once. this helps to explain the lack of integration across studies and the lack of coherence in the research as a whole. third, the analysis reveals that a significant proportion of the literature is based on empirical studies that rely on artificial data and/or artificial settings and are ill 9 australasian journal of information systems o riordan, lohan & conboy 2015, vol. 19, selected papers from acis mind the gaps suited to the investigation of the efficacy of isd processes, the usability of isd products and to quantify the impact of isd products on the generation of business value. acknowledgements this work was supported, in part, by science foundation ireland grant 10/ce/i1855 to lero the irish software engineering research centre (www.lero.ie). references abrahamsson, p., conboy, k., and wang, x. 2009. "“lots done, more to do”: the current state of agile systems development research,"). barry, c., and brown, j. 2003. "finding a home for web-based information systems-perusing the landscape," ecis, pp. 126-138. basili, v.r. 1992. "software modeling and measurement: the goal/question/metric paradigm". bharadwaj, a., keil, m., and mähring, m. 2009. "effects of information technology failures on the market value of firms," the journal of strategic information systems (18:2), pp. 66-79. brereton, p., kitchenham, b.a., budgen, d., turner, m., and khalil, m. 2007. "lessons from applying the systematic literature review process within the software engineering domain," journal of systems and software (80:4), pp. 571-583. conboy, k. 2009. "agility from first principles: reconstructing the concept of agility in information systems development," information systems research (20:3), pp. 329354. conboy, k and fitzgerald, b. 2010. "method and developer characteristics for effective agile method tailoring: a study of xp expert opinion," acm transactions on software engineering and methodology (20:1) concas, g., di francesco, m., marchesi, m., quaresima, r., and pinna, s. 2008. "study of the evolution of an agile project featuring a web application using software metrics," in product-focused software process improvement. springer, pp. 386-399. downey, s., and sutherland, j. 2013. "scrum metrics for hyperproductive teams: how they fly like fighter aircraft," 46th hawaii international conference on system sciences (hicss), pp. 4870-4878. dybå, t., and dingsøyr, t. 2008. "empirical studies of agile software development: a systematic review," information and software technology (50:9), pp. 833-859. dyba, t., dingsoyr, t., and hanssen, g.k. 2007. "applying systematic reviews to diverse study types: an experience report," first international symposium on empirical software engineering and measurement: ieee, pp. 225-234. fenton, n. 1994. "software measurement: a necessary scientific basis," ieee transactions on software engineering (20:3), pp. 199-206. fink, a. 1998. "conducting research literature reviews: from paper to the intern,"). harjumaa, l., markkula, j., and oivo, m. 2008. "how does a measurement programme evolve in software organizations?," in product-focused software process improvement. springer, pp. 230-243. harrison, w. 2004. "a flexible method for maintaining software metrics data: a universal metrics repository," journal of systems and software (72:2), pp. 225-234. hartmann, d., and dymond, r. 2006. "appropriate agile measurement: using metrics and diagnostics to deliver business value," agile conference, 2006: ieee, pp. 6 pp.-134. 10 australasian journal of information systems o riordan, lohan & conboy 2015, vol. 19, selected papers from acis mind the gaps jalali, s., and wohlin, c. 2012. "global software engineering and agile practices: a systematic review," journal of software: evolution and process (24:6), pp. 643-659. kitchenham, b.a., and charters, s. 2007. "guidelines for performing systematic literature reviews in software engineering". kulas, h. 2012. "product metrics in agile software development," in: school of information sciences. university of tampere. kunz, m., dumke, r.r., and zenker, n. 2008. "software metrics for agile software development," 19th australian conference on software engineering: ieee, pp. 673678. markus, m.l., and keil, m. 1994. "if we build it, they will come: designing information systems that people want to use," sloan management review (35), pp. 11-11. meneely, a., smith, b., and williams, l. 2012. "validating software metrics: a spectrum of philosophies," acm transactions on software engineering and methodology (tosem) (21:4), p. 24. nagappan, n., ball, t., and zeller, a. 2006. "mining metrics to predict component failures," proceedings of the 28th international conference on software engineering: acm, pp. 452-461. nerur, s., and balijepally, v. 2007. "theoretical reflections on agile development methodologies," communications of the acm (50:3), pp. 79-83. petersen, k. 2011. "measuring and predicting software productivity: a systematic map and review," information and software technology (53:4), pp. 317-343. petersen, k., and wohlin, c. 2010. "the effect of moving from a plan-driven to an incremental software development approach with agile practices," empirical software engineering (15:6), pp. 654-693. royce, w. 2002. "cmm vs. cmmi: from conventional to modern software management," the rational edge, pp. 2-9. sessions, r. 2009. "the it complexity crisis: danger and opportunity," white paper, november). staron, m. 2012. "critical role of measures in decision processes: managerial and technical measures in the context of large software development organizations," information and software technology (54:8), aug, pp. 887-899. wang, x., conboy, k., and cawley, o. 2012. "“leagile” software development: an experience report analysis of the application of lean approaches in agile software development," journal of systems and software (85:6), pp. 1287-1299. 11 australasian journal of information systems o riordan, lohan & conboy 2015, vol. 19, selected papers from acis mind the gaps appendix a the 57 papers that were analysed in this study are listed below. these papers are referenced in the paper using square brackets whereas all other sources are referenced in the normal fashion. list of publications analysed as part of the literature review (n=57) [1] alshayeb, m., and li, w. 2003. "an empirical validation of object-oriented metrics in two different iterative software processes," ieee transactions on software engineering (29:11), nov, pp. 1043-1049. [2] alshayeb, m., and li, w. 2005. "an empirical study of system design instability metric and design evolution in an agile software process," journal of systems and software (74:3), feb 1, pp. 269-274. [3] andersson, carina, and per runeson. "a spiral process model for case studies on software quality monitoring—method and metrics." software process: improvement and practice 12.2 (2007): 125-140. [4] bavota, g., de lucia, a., and oliveto, r. 2011"identifying extract class refactoring opportunities using structural and semantic cohesion measures," journal of systems and software (84:3), mar, pp. 397-414. [5] bertoa, m.f., troya, j.m., and vallecillo, a. 2006. "measuring the usability of software components," journal of systems and software (79:3), mar, pp. 427-439. [6] bibi, stamatia, et al. "bbn based approach for improving the software development process of an sme—a case study." software process: improvement and practice (2009). [7] binkley, d., feild, h., lawrie, d., and pighin, m. 2009. "increasing diversity: natural language measures for software fault prediction," journal of systems and software (82:11), nov, pp. 1793-1803. [8] chen, c.b., lin, c.t., wang, c.h., and chang, c.w. 2006. "model for measuring quality of software in dvrs using the gap concept and fuzzy schemes with ga," information and software technology (48:3), mar, pp. 187-203. [9] chilton, michael a., bill c. hardgrave, and deborah j. armstrong. "person-job cognitive style fit for software developers: the effect on strain and performance." journal of management information systems 22.2 (2005): 193-226. [10] conejero, j.m., figueiredo, e., garcia, a., hernandez, j., and jurado, e. "on the relationship of concern metrics and requirements maintainability," information and software technology (54:2), feb, pp. 212-238. [11] cubranic, d., murphy, g.c., singer, j., and booth, k.s. 2005. "hipikat: a project memory for software development," ieee transactions on software engineering (31:6), jun, pp. 446-465. [12] durisic, d., nilsson, m., staron, m., and hansson, j. 2013. "measuring the impact of changes to the complexity and coupling properties of automotive software systems," journal of systems and software (86:5), may, pp. 1275-1293. [13] etzkorn, letha h., et al. "a comparison of cohesion metrics for object-oriented systems." information and software technology 46.10 (2004): 677-687. [14] gencel, c., petersen, k., mughal, a.a., and iqbal, m.i. "a decision support framework for metrics selection in goal-based measurement programs: gqmdsfms," journal of systems and software (86:12), dec, pp. 3091-3108. 12 australasian journal of information systems o riordan, lohan & conboy 2015, vol. 19, selected papers from acis mind the gaps [15] gopal, anandasivam, et al. "measurement programs in software development: determinants of success." software engineering, ieee transactions on 28.9 (2002): 863-875. [16] gotel, olly, et al. "quality indicators on global software development projects: does ‘getting to know you’really matter?." journal of software: evolution and process 24.2 (2012): 169-184. [17] gou, lang, et al. "quantitative defects management in iterative development with bidefect." software process: improvement and practice 14.4 (2009): 227-241. [18] han, a.-r., jeon, s.-u., bae, d.-h., and hong, j.-e. "measuring behavioral dependency for improving change-proneness prediction in uml-based design models," journal of systems and software (83:2), feb, pp. 222-234. [19] hazzan, o., and hadar, i. 2008. "why and how can human-related measures support software development processes?," journal of systems and software (81:7), jul, pp. 1248-1252. [20] henderson, peter, yvonne margaret howard, and robert john walters. "a tool for evaluation of the software development process." journal of systems and software 59.3 (2001): 355-362. [21] iversen, j., and mathiassen, l. 2003. "cultivation and engineering of a software metrics program," information systems journal (13:1), jan, pp. 3-19. [22] johnson, p.m., kou, h.b., paulding, m., zhang, q., kagawa, a., and yamashita, t. 2005. "improving software development management through software project telemetry," ieee software (22:4), jul-aug, pp. 76-+. [23] joshi, a., sarda, n.l., and tripathi, s. "measuring effectiveness of hci integration in software development processes," journal of systems and software (83:11), nov, pp. 2045-2058. [24] kakarontzas, g., constantinou, e., ampatzoglou, a., and stamelos, i. "layer assessment of object-oriented software: a metric facilitating white-box reuse," journal of systems and software (86:2), feb, pp. 349-366. [25] kumar, k.v., ravi, v., carr, m., and kiran, n.r. 2008. "software development cost estimation using wavelet neural networks," journal of systems and software (81:11), nov, pp. 1853-1867. [26] lee, g., and xia, w. "toward agile: an integrated analysis of quantitative and qualitative field data on software development agility," mis quarterly (34:1), mar, pp. 87-114. [27] lee, g., and xia, w.d. 2005. "the ability of information systems development project teams to respond to business and technology changes: a study of flexibility measures," european journal of information systems (14:1), mar, pp. 75-92. [28] li, m., and smidts, c.s. 2003. "a ranking of software engineering measures based on expert opinion," ieee transactions on software engineering (29:9), sep, pp. 811-824. [29] lin, c.-t., and huang, c.-y. 2008. "enhancing and measuring the predictive capabilities of testing-effort dependent software reliability models," journal of systems and software (81:6), jun, pp. 1025-1038. [30] madeyski, l. "the impact of test-first programming on branch coverage and mutation score indicator of unit tests: an experiment," information and software technology (52:2), feb, pp. 169-184. 13 australasian journal of information systems o riordan, lohan & conboy 2015, vol. 19, selected papers from acis mind the gaps [31] manadhata, p.k., and wing, j.m. "an attack surface metric," ieee transactions on software engineering (37:3), may-jun, pp. 371-386. [32] mendes, e., mosley, n., and counsell, s. 2005. "investigating web size metrics for early web cost estimation," journal of systems and software (77:2), aug, pp. 157172. [33] miranda, e., and bourque, p. "agile monitoring using the line of balance," journal of systems and software (83:7), jul, pp. 1205-1215. [34] na, k.-s., simpson, j.t., li, x., singh, t., and kim, k.-y. 2007. "software development risk and project performance measurement: evidence in korea," journal of systems and software (80:4), apr, pp. 596-605. [35] olague, h.m., etzkorn, l.h., gholston, s., and quattlebaum, s. 2007. "empirical validation of three software metrics suites to predict fault-proneness of objectoriented classes developed using highly iterative or agile software development processes," ieee transactions on software engineering (33:6), jun, pp. 402-419. [36] palviainen, m., evesti, a., and ovaska, e. "the reliability estimation, prediction and measuring of component-based software," journal of systems and software (84:6), jun, pp. 1054-1070. [37] petersen, kai. "measuring and predicting software productivity: a systematic map and review." information and software technology 53.4 (2011): 317-343. [38] port, d., and bui, t. 2009. "simulating mixed agile and plan-based requirements prioritization strategies: proof-of-concept and practical implications," european journal of information systems (18:4), aug, pp. 317-331. [39] procaccino, j.d., verner, j.m., darter, m.e., and amadio, w.j. 2005. "toward predicting software development success from the perspective of practitioners: an exploratory bayesian model," journal of information technology (20:3), sep, pp. 187-200. [40] quah, tong-seng, and mie mie thet thwin. "prediction of software development faults in pl/sql files using neural network models." information and software technology 46.8 (2004): 519-523. [41] raffo, d.m. 2005. "software project management using prompt: a hybrid metrics, modeling and utility framework," information and software technology (47:15), dec, pp. 1009-1017. [42] sarkar, s., kak, a.c., and rama, g.m. 2008. "metrics for measuring the quality of modularization of large-scale object-oriented software," ieee transactions on software engineering (34:5), sep-oct, pp. 700-720. [43] sarker, s., munson, c.l., sarker, s., and chakraborty, s. 2009. "assessing the relative contribution of the facets of agility to distributed systems development success: an analytic hierarchy process approach," european journal of information systems (18:4), aug, pp. 285-299. [44] sawyer, s., guinan, p.j., and cooprider, j. "social interactions of information systems development teams: a performance perspective," information systems journal (20:1), jan, pp. 81-107. [45] sobel, ann e. kelley, and michael r. clarkson. "formal methods application: an empirical tale of software development." software engineering, ieee transactions on 28.3 (2002): 308-320. [46] stamelos, ioannis, et al. "code quality analysis in open source software development." information systems journal 12.1 (2002): 43-60. 14 australasian journal of information systems o riordan, lohan & conboy 2015, vol. 19, selected papers from acis mind the gaps [47] staron, m., meding, w., and soderqvist, b. 2010. "a method for forecasting defect backlog in large streamline software development projects and its industrial evaluation," information and software technology (52:10), oct, pp. 1069-1079. [48] subramanyam, r., and krishnan, m.s. 2003. "empirical analysis of ck metrics for object-oriented design complexity: implications for software defects," ieee transactions on software engineering (29:4), apr, pp. 297-310. [49] succi, g., pedrycz, w., stefanovic, m., and miller, j. 2003. "practical assessment of the models for identification of defect-prone classes in object-oriented commercial systems using design metrics," journal of systems and software (65:1), jan 15, pp. 1-12. [50] teasley, stephanie d., et al. "rapid software development through team collocation." software engineering, ieee transactions on 28.7 (2002): 671-683. [51] tomaszewski, p., and lundberg, l. 2005. "software development productivity on a new platform: an industrial case study," information and software technology (47:4), mar 15, pp. 257-269. [52] tunkelo, teemu, ari‐pekka hameri, and yves pigneur. "improving globally distributed software development and support processes–a workflow view." journal of software: evolution and process 25.12 (2013): 1305-1324. [53] van koten, c., and gray, a.r. 2006. "bayesian statistical effort prediction models for data-centred 4gl software development," information and software technology (48:11), nov, pp. 1056-1067. [54] van rompaey, b., du bois, b., demeyer, s., and rieger, m. 2007. "on the detection of test smells: a metrics-based approach for general fixture and eager test," ieee transactions on software engineering (33:12), dec, pp. 800-817. [55] wijayasiriwardhane, t., and lai, r. "component point: a system-level size measure for component-based software systems," journal of systems and software (83:12), dec, pp. 2456-2470. [56] wood, s., michaelides, g., and thomson, c. "successful extreme programming: fidelity to the methodology or good teamworking?," information and software technology (55:4), apr, pp. 660-672. [57] xia, w.d., and lee, g.h. 2005. "complexity of information systems development projects: conceptualization and measurement development," journal of management information systems (22:1), sum, pp. 45-83. an earlier version of this paper was presented at the australasian conference on information systems (acis) 2014 in auckland, new zealand. copyright: © 2015 o riordan lohan & conboy. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. 15 http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems o riordan, lohan & conboy 2015, vol. 19, selected papers from acis mind the gaps 16 1 introduction 2 problem statement 2.1 agile methods: a possible solution? 2.2 isd metrics: a possible solution? 3 research design 3.1 search strategy and paper selection 3.2 in-depth evaluation 4 findings and discussion 4.1 the literature is heavily focused on the identification and validation of metrics 4.2 the literature is heavily focused on particular types of research question 4.3 the literature is heavily focused on particular types of research design 5 conclusion paper title (paper title style) australasian journal of information systems nguyen & wickramasinghe 2017, vol 21, research on health information systems mediating role for nursing is 1 an examination of the mediating role for a nursing information system lemai nguyen department of information systems and business analytics deakin business school deakin university lemai.nguyen@deakin.edu.au nilmini wickramasinghe health informatics management epworth healthcare faculty of health, deakin university abstract this paper reports on findings from an examination of a nursing information system through the lens of activity theory. the information system was designed to support real-time nursing documentation in acute care hospital contexts. the objective was to enable superior nursing care to ensue by providing nurses with the opportunity to document patient care data into a tablet computer located at the patient bedside. the system was evaluated in a not-for-profit acute care hospital’s wards during its implementation. nurses’ interactions with the system and their perceptions were collected and analysed through the lens of activity theory. the analysis highlighted nurses’ positive attitude towards the system and identified potential mediation capabilities as well as areas for improvements. activity theory was found to be useful to examine the positive and potentially problematic aspects of this new nursing information system. keywords: activity theory; nursing informatics; health information systems; electronic documentation. 1 introduction increasingly health information systems (his) are introduced into healthcare as a mediating tool to support patient care delivery and to improve care quality, safety and efficiency. the introduction of his comes with many challenges. these are due to the dynamic and complex nature of healthcare operations; such as multi-disciplinary characteristics of critical tasks, the high stakes of life and death, the tensions between orderly routines and the need for flexibility, the need to combine rationale and intuitive judgement in key decision making, and various organisational and regulatory setting specifications (fichman et al., 2011; shaban, 2005; thompson & dowding, 2002: chapter 1). in such a complex context, a good understanding of the mediating role of technology, i.e. whether and how it can support the clinician user in care delivery, is important to facilitate successful his design, implementation and adoption as well as support superior care delivery. over the years, on-going research reports mixed and inconclusive results in regard to impacts of his implementation and its assimilation to clinicians’ practice and workflow (lau et al., 2010; mills et al., 2013; nguyen et al., 2014). industry surveys conducted by the american college of physicians and american ehr partners (2013) reveal an overall dissatisfaction of the clinician users. moreover, these reports highlight a need for a rich understanding of the role of his with the clinicians’ in the context of their activities and practice. the research reported in this paper contributes to addressing this need by examining the potential mediating role of a new his in an acute healthcare setting for front line clinical staff, particularly nurses. in order to do so, activity theory (engeström, 1999; vygotsky, 1978) was selected as a suitably rich theoretical lens to understand the interactions between individuals, australasian journal of information systems nguyen & wickramasinghe 2017, vol 21, research on health information systems mediating role for nursing is 2 their goals, and the mediating tool to achieve a specific outcome; namely better patient care. specifically in this study, activity theory serves as a lens to examine the interactions between the nurse, the nursing documentation, and a new electronic bedside documentation system (ebds) to improve patient care delivery. the paper is structured as follows. section 2 provides a background discussion about nurses and information systems. section 3 provides an overview of the key concepts in activity theory and its use in the information systems discipline. section 4 describes the adopted research approach. section 5 presents the results of the study. section 6 concludes the paper with a summary of the key findings, theoretical and practical implications, limitations and future research directions. 2 nurses and information systems in healthcare delivery, nurses provide front-line and continuous care for patients. they play a key role in patient safety including early detection of deterioration and prevention of complications such as pressure injuries, falls and infections (see for example, cornell et al., 2010; courtney et al., 2005; hegney et al., 2007). previous research has provided robust evidence of the relationship between the quality of nursing care and patient outcomes (australian nursing federation, 2009; rafferty et al., 2007). the nursing process includes assessment, diagnosis, planning, implementation, and evaluation (timby, 2009: chapter 2). this process involves various observations, decision-making, and interventions and associated patient health outcomes. nursing documentation is essential in keeping accurate records of delivered care and patient health status, as well as supporting clinical communication about nurses’ observations, decisions, actions and outcomes. not only is nursing documentation a legal requirement, it promotes the early identification of patient health issues and opportunities for care quality and safety improvements. nursing documentation is a routine, integral part of safe and effective patient care delivery (blair & smith, 2012). with the advancements of portable computers and electronic health records, electronic bedside documentation systems are increasingly introduced to nurses as a support tool for nursing documentation. the objective is to enable nurses to document and access patient care data at the patient bedside when care is delivered. common anticipated benefits include increased care quality, patient safety and process efficiency as a result of better access to realtime and accurate patient data. the reality has yet to reach this state and over one decade (2003-2014) of literature reviews report mixed findings including attitude of nurses toward adopting technology solutions and impacts of technology adoption in healthcare (häyrinen et al., 2008; lau et al., 2010; meijden et al., 2003; mills et al., 2013; nguyen et al., 2014). mixed findings are still found in recent research (lin et al., 2016; tilahun & fritz, 2015). nurses’ low satisfaction with system’ usability and fidelity of technology solutions in their practice emerge frequently as two strong themes in previous research. usability has been defined in iso 9241 as “the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use” (international organization for standardization, 1998). in this paper, fidelity refers to the system’s appropriateness or fitness with the context of use. table 1 summarises findings from previous research in relation to these themes. we note that while other issues such as security, networks, and other characterises of selected technologies are also important they are not considered in this study since in general for acute healthcare contexts if usability and fidelity are problematic, these other issues are not considered or treated as mute until usability and fidelity are addressed. australasian journal of information systems nguyen & wickramasinghe 2017, vol 21, research on health information systems mediating role for nursing is 3 themes summary of findings usability ease of use / efficiency complex user-interface and navigation mechanism, slow response, poor content design (darbyshire, 2004; kossman, 2006; lee et al., 2008; stevenson et al., 2010; vatnøy et al., 2014; weinhara et al., 2009; whittaker et al., 2009). moderate ease of use (weinhara et al., 2009; whittaker et al., 2009). observed increased time spent on direct patient care and nursing documentation, and decreased time spent on administrative tasks (banner & olney, 2009) in contrast to no significant change in time spent on patient care (westbrook et al., 2013). usefulness / effectiveness perceived usefulness: access to important patient care information and reduced written documentation (hoonakker et al., 2013; kossman, 2006; vatnøy et al., 2014; whittaker et al., 2009), improved information quality (aktharsha & anisa, 2011; darbyshire, 2004; tilahun & fritz, 2015), enabled monitoring of patients’ health parameters (miller et al., 2009). some functions were used less relevant, less frequently or never used (darbyshire, 2004; nakamura et al., 2010; tilahun & fritz, 2015). satisfaction majority of nurses have a moderately positive attitude toward using his (de veer & francke, 2010; mills et al., 2013; moody et al., 2004). mixed findings: moderate / high satisfaction (hoonakker et al., 2013; moody et al., 2004). low satisfactory / dissatisfaction (darbyshire, 2004; lærum et al., 2004; mills et al., 2013; stevenson et al., 2010; tilahun & fritz, 2015). dynamics of nurses’ experience: during the period of implementation, nurses’ attitude and satisfaction increased overtime as they experienced more with the system (ammenwerth et al., 2011; carayon et al., 2011; lee et al., 2008). fidelity his functionality did not meet the nurses’ needs nor support their workflows and practice (novak et al., 2013; stevenson et al., 2010); workflow interruptions, additional work (eley et al., 2009; lee et al., 2008), unintended consequences (dowding et al., 2009; koppel et al., 2005). a particular his may fit one context but not others (ackerman et al., 2012; gough et al., 2014; jeskey et al., 2011) table 1. summary of previous research in relation to usability and fidelity firstly, in regard to usability, stevenson et al. (2010) reviewed five empirical studies published from 2000 to 2009 and reported on nurses’ experiences with his systems. these studies took place in acute care hospital contexts in usa, australia, and the uk. according to their review, his functionality was not useful nor did it meet the nurses’ needs nor support their workflow; the user-interfaces were found to be difficult to use and navigate. overriding of the system, duplication of documentation, and a reversion to familiar systems such as paper recording were reported as major work-around strategies when technology solutions did not meet clinicians’ needs (dowding et al., 2009; koppel et al., 2005). more recently, mills et al. (2013) reviewed 21 studies published between 1995 and 2010 which took place in acute care contexts in australia and new zealand. as reported in their review, while nurses held positive attitudes toward his adoption they experienced dissatisfaction with technology implementations at the workplace. particularly, nurses reported problems with the system’s lack of intuitive userinterface and navigation design, complex content presentation, and slow response (darbyshire, 2004; kossman, 2006; lee et al., 2008; vatnøy et al., 2014; weinhara et al., 2009; whittaker et al., 2009). contradictory findings were found in two time and motion studies: banner and olney (2009) found that nurses spent more time on documentation and direct patient care and less time on administration. in contrast, westbrook et al. (2013) found no redistribution of time away from direct patient care or other tasks before and after the his implementation. usefulness of his was perceived by nurses to include the system’s capability to document and access patient care data, improved legibility and reduced data errors (e.g. aktharsha & anisa, 2011; hoonakker et al., 2013; vatnøy et al., 2014), and enabled monitoring of patients’ parameters (miller et al., 2009). mixed findings were reported in regard to nurses’ australasian journal of information systems nguyen & wickramasinghe 2017, vol 21, research on health information systems mediating role for nursing is 4 satisfaction with his. a majority of studies reported low satisfaction or dissatisfaction with his (darbyshire, 2004; lærum et al., 2004; mills et al., 2013; stevenson et al., 2010; tilahun & fritz, 2015). surveys with nurses 3 months and 12 months after his implementation at an us regional medical centre found nurses’ satisfaction toward the system increased over time (carayon et al., 2011). secondly, his functionality was criticised by nurses for its poor fit with their workflow and practice needs (darbyshire, 2004; novak et al., 2013). novak et al. (2013) observed a collision between nursing and technology frames of references as well as observing nurses’ adaptations to fit his with their work. his was found to cause workflow interruptions and increased workload (eley et al., 2009; lee et al., 2008), and unintended consequences (dowding et al., 2009; koppel et al., 2005). in addition, a particular his may fit one context but not others; the same his was successfully adopted in urgent care but was abandoned when it was introduced to routine patient care due to its poor fit (ackerman et al., 2012). in another study (jeskey et al., 2011), his designed for critical care was found to be unsuitable for acute care settings. gough et al. (2014) interviewed 125 nurses at five acute care hospitals across two australian states. they found that his supported nurses and enhanced their sense of autonomy at high dependency wards while it increased their workloads and reduced their interactions with patients in low dependency wards. in summary, usability and fidelity are frequently identified and remain on-going strong themes in the literature. as these concern technology’s capability and fitness with the context of use, we focus on examining them as mediating roles of technology in nursing documentation activity to see if this could address current key voids noted throughout the literature regarding usability and fidelity. having conducted a literature review in electronic health record (ehr) systems as real-time point of care information systems in the context of orders management, staggers et al. (2008) concluded that further research is needed to examine technology impact on the role of nurses and their activities, and suggested activity theory as one of the potentially useful approaches. this is in line with yen and bakken (2012) who noted that there are more quantitative than qualitative studies in evaluating his usability during systems’ implementation and routine use stages. furthermore, as there is a recognised need for more qualitative types of research when studying his in acute healthcare contexts as such studies tend to uncover more issues around using his including fidelity and usability (nguyen et al., 2014; yen & bakken, 2012). overall, mixed and inconclusive findings from the previous studies provide a strong case for needed further research using a rich analysis lens to gain in-depth understanding of the role of his and in particular electronic nursing documentation systems in the immediate context of use. hence to provide a rich lens of analysis to shed light on his in the context of nursing documentation , this study focuses on the mediating role of technology as a tool in the specific context of human activity; i.e., a nursing information system in an acute healthcare context and analyses this using a rich theoretical lens; i.e., activity theory. 3 activity theory in information systems given low satisfaction with is theories around user acceptance and adoption specifically in acute healthcare nursing contexts (see for example cresswell et al., 2010; holden & karsh, 2010), we investigate the benefits of framing the situation in terms of activity theory. 3.1 an overview of key concepts in activity theory activity theory, or cultural-historical theory of activity, (vygotsky, 1978, english translation) was first developed in the 1920s and 1930s by lev vygosky, a russian psychologist, together with his colleagues. the fundamental building block of the theory is a human activity involving a subject i.e. a human agent, and an object i.e. an objective or goal of activity. this subjectobject relationship is mediated by tools and signs (language/words) that essentially are linked to a culture. activity theory was extended by leontiev (2009, english translation) to describe a three layer model of human activity consisting of activity, actions and operations. along with these, there australasian journal of information systems nguyen & wickramasinghe 2017, vol 21, research on health information systems mediating role for nursing is 5 is another hierarchy of motivation consisting of motive, goals, and conditions. an activity is associated with motive at the top level and can be characterized as high-level, significant and holistic. activity can be achieved through a set of actions where each consists of operations. an action can be associated with an intention (to achieve a specific goal) and operations. an operation is a performed routine to complete the action under certain conditions. activity theory was further extended by engeström (1987, 1999) to include the collective characteristics of the human activity system. the extended model includes additional collective elements of rules, community and division of labour. rules and division of labour can be explicit or implicit. rules refer to conventions, regulations and social relations within a community while division of labour refers to the organisation of a community to enable the transformation. therefore, rules mediate the relationship between subject and community while division of labour mediates the relationship between object and community. the mediating roles of tools, rules and division of labour are historically formed and evolve over time. further, engeström’s structure of activity also emphasises the transformation of an object to an outcome (engeström, 1987). as shown in figure 1, the structure of an individual human activity includes a subject e.g. an individual worker or member of the workplace community, tools e.g. conceptual models or devices, the rules that govern their work, and the purpose to which the subject directs their activity. this structure of individual human activity can be further extended to denote the interaction between multiple human activities where the activities share common goals and rules of division of labour. figure 1. structure of human activity in activity theory (engeström, 1987, 1999) engeström (1987) described human activity as a goal oriented, tool and cultural-historically mediated. furthermore, he added the dynamic, cultural-historical dimension to the human activity through the concepts of externalisation and internalisation. externalisation can be described as the process of change driven by tensions and conflicts motivting changes and improvements; as a result new tools are created and introduced. internalisaiton can be described as the process of reproduction of culture through the training of members within the community to adopt changes and fit in the new culture. these two processes are refered to as the dynamic cycle of learning and enable for both the historical continuilty and adaptation of the culture of the human activity system. 3.2 3.2 use of activity theory in information systems some of the first instances when activity theory was recommended to be used for information systems (is) research include by nardi (1996: chapter 1) and kuutti (1996: chapter 2). both of these authors emphasise the notion of technology mediation in activity theory. technology can be seen as tools or “mediators of human thought and behaviour” (nardi, 1996: p. 7). thus, to design an effective mediating technology solution, we should understand users’ thoughts and behaviours in the context of use. since then, activity theory has been embraced by is researchers from different perspectives. table 2 summarises the key perspectives. australasian journal of information systems nguyen & wickramasinghe 2017, vol 21, research on health information systems mediating role for nursing is 6 perspective subject object tools community advocates 1. is development as a human activity the developer of an information system to build an information system analysis and design methods, computers, networks, programming languages and tools. developers, user representatives and their workplace environment(s) (kuutti, 1996: pp. 14-15). 2. is as a tool in various human activities the user of an information system user’s goals (for example healthcare, business, education, entertainment and social networking etc.) an information system(s) (and other tools) users’ workplace environment and/or their communities (hashim & jones, 2007). 3. is research study as a human activity the is researcher to answer an is research question(s) and extend the current human knowledge research methodologies and tools research environment, participants, their organisation(s) and the wider community (crawford & hasan, 2006) table 2. perspectives of using activity theory in is research with a view to examining the usability and fidelity of his, we are interested in the second perspective in table 2, i.e. is as a tool in a human activity in the healthcare context. activity theory has been found to be useful to explore and elicit healthcare practitioners’ requirements for an information system as a tool (häkkinen & korpela, 2007; sadeghi et al., 2014) and to unpack healthcare activities when assessing e-health readiness in community hospitals in south africa (coleman & coleman, 2013). activity theory has also been used in investigating is adoption in healthcare contexts. for example, factors affecting doctors’ intention to adopt is were found to include subject (human), tool (technology), and environment (organization) (juling et al., 2016). in the above-mentioned studies, doctors, nurses, social workers and hospital administrators were the subject; his was their to-be or adopted tool; and their task and work process was the objective according to activity theory. other studies (vega-barbas et al., 2015; waycott et al., 2014) consider patients as the subject interacting with technology tools for their own healthcare. waycott et al. (2014) conducted a multiple case study with 15 pregnant women with type 1 diabetes and concluded that motivated patients play an active role in aligning such interacting activity systems and adjusting their use of technology tools for their own health management. activity theory has also been adopted to review research efforts in is adoption within the healthcare domain (sun & qu, 2015). these authors selected 979 papers about his adoption published between 1990 and 2009 from online databases including pubmed, medline and embase. using a text-mining tool and the activity theory lens, the authors identified different types of subjects, tools and their tasks from the abstracts of the selected papers. their work revealed that (1) is adoption can be examined at an individual as well as a social level, and that (2) current research tends to examine is adoption at individual levels rather than adoption at a social level. our study builds on previous research in which activity theory was found to be a useful theoretical lens to examine is as a tool in human activities in their immediate context of use. particularly, we used the activity theory lens to examine the mediating i.e. supporting, role of a new point of care electronic documentation system in its natural, target patient care setting. australasian journal of information systems nguyen & wickramasinghe 2017, vol 21, research on health information systems mediating role for nursing is 7 thus, this is one of the first studies to use activity theory in an acute healthcare nursing context in this fashion. 4 research approach the study reported in this paper was one component of a large multi-disciplinary longitudinal funded research program (nguyen et al., 2015). a design science research methodology has been adopted to guide the cycles of problem/opportunity determination (relevance cycle), solution design and evaluation examine (design cycle), using the knowledge base to inform design (rigor cycle) (hevner & chatterjee, 2010; hevner et al., 2004) through the large research project. the problem addressed in the on-going research project is focused on the lack of a technology solution that is usable for nurses and reflects and responds to the complexity of nursing work (stevenson et al., 2010). this paper reports specifically on the findings from one of the design cycles in which activity theory was adopted to examine the mediating role for a new electronic bedside documentation system (ebds) in an acute care context. we note that the ebds solution was developed by an international vendor and this study is one of the first to examine the system in situ. the development of the ebds was informed through our study including the full cycle of design science research methodology. see table 3 below for how hevner’s et al. (2004) design science research guidelines apply in this project, i.e. in acute healthcare nursing context. design science research guidelines our application guideline 1: design as an artefact the ebds system a sophisticated, multi-functional point of care technology designed to support nurses in the delivery and documentation of patient care in the acute hospital setting. guideline 2: problem relevance to address the lack of a technology solution that is usable for nurses, reflects and responds to the complexity of acute healthcare nursing work. guideline 3: design evaluation nurses as users were trained to use ebds; subsequently, they used it in parallel with paper documentation; finally, champion nurses used it exclusively for documentation of all nursing care delivery for selected patients. guideline 4: research contributions in this study, nurses’ perspectives of the mediating role of his were explored. guideline 5: research rigor theoretical foundations and conceptual models drawn from information systems, nursing documentation and nursing process, healthcare quality and safety were used to inform the development cycles to evaluate the ebds system in the acute healthcare nursing context. guideline 6: design as a search process in this project, activity theory concepts (subject, object, tools, community, rules, division of labours) were used to guide data analysis and define areas and levels of mediation. the findings will inform future implementations. guideline 7: communication of research internal communication: the larger research project (of which this study is part of) is presented to technology and clinically-oriented users through focus groups, simulations exercises, brainstorming meetings, as well as technical and managerial meetings. external communication: progress and findings are reported in a book chapter and peer review papers submitted to international conferences and professional peer-reviewed journals in each discipline. table 3. application of design science research guidelines to the development of ebds (guidelines adopted from hevner et al., 2004) australasian journal of information systems nguyen & wickramasinghe 2017, vol 21, research on health information systems mediating role for nursing is 8 4.1 research setting the ebds system is a sophisticated, multi-functional point of care technology solution designed to support nurses in the delivery and documentation of patient care in the acute hospital setting. it was designed to allow nurses to document patient care and access the care data in real time, thereby providing potential enhancement to nurses’ work experience. the system was run on a server and a thin client and the frontend was on an ipad mounted at each patient bedside. ebds was put into an implementation for eight weeks from august to october in 2013. the implementation was conducted in two acute care wards across two sites of a single large metropolitan health service in victoria, australia. human research ethics approvals were obtained from the host healthcare service provider and affiliated university prior to the commencement of the study. the participating wards included a 25-bed general surgical ward and a 32-bed general medical and neurology ward involved over 80 nurses. nurses were the primary end users of the system. nurse managers were asked to provide lists of all registered and enrolled nursing staff working regularly on their wards more than one shift per week. agency nurses and nursing staff working in the participating wards less than one shift per week were excluded from the study as these staff did not have the opportunity for training to become familiar with the system prior to use in their routine work. all staff were individually sent written information about the study and invited to attend information sessions where they were able to ask questions. on agreement to participate, all participants were then allocated a unique code to protect their identity in all research data. at the completion of data collection and prior to analysis, all potentially identifying features were removed. the process (in the study reported in this paper) consisted of three stages. during stage 1, the ebds system was installed in eight beds on each ward; and nurse participants were recruited, trained and introduced to using ebds. stage 1 occurred over two weeks. stage 2 was designed to integrate the ebds system into selected nursing care processes for a sub-group of patients allocated to the beds where it was installed. during this stage the participating nurses began to adopt the system into care processes. stage 2 occurred over three weeks. while using ebds, nurse participants identified issues of concern related to usability and risk mitigation, these issues were then prioritized by the management committee; and in response ebds was modified in real time by the developers. during stage 3, the system was used exclusively for documentation of all nursing care delivery for selected patients. the participants were three nurses identified as early adopters or ‘super users’ from stage 2. to ensure patient safety, a supernumerary nurse ‘scribe’ was employed to ‘shadow’ all activities performed by nurse participants in order to manually duplicate all documentation of patient data into the permanent paper records. this was a requirement from ethics and legal. table 4 maps the key concepts in activity theory to our conceptualisation of the nursing documentation activity in the acute healthcare nursing context. nursing documentation is one of the human activities in the overall context of patient care. the overall objective is to ensure quality, safety, and efficiency of patient care, and that positive patient outcomes ensue. the ebds system is proposed to be a new electronic tool to replace paper forms and charts to support nursing documentation activity. australasian journal of information systems nguyen & wickramasinghe 2017, vol 21, research on health information systems mediating role for nursing is 9 key concepts nursing documentation subject nurse object documentation of patient care tools there is a range of tools to support nurses to communicate and document patient care data. the existing tools include paper forms and assessment charts, communication whiteboard, phones, medical equipment and other clinical tools. ebds is the new information system and is the system under study. community a community of nurses, patients, other healthcare professionals in acute care setting in the hospital. rules documentation is required as part of nursing discipline; other rules concern care process and workflows, and other hospital and broader healthcare regulations. division of labour distribution of work, power, and responsibilities among nurses and other healthcare professionals such as doctors, pharmacists, other specialists, allied health, etc. for example, nurse assesses vital signs while the medical professional / doctor assesses medical progress specific to the treatment under consideration. table 4. understanding nursing documentation using key concepts of activity theory 4.2 data collection and analysis focus groups and non-participant observations were used as data collection methods. during stage 2, two focus groups (fga and fgb) were conducted with eight nurses. at the end of stage 3, a third focus group (fgc) was conducted with the three nurses who participated in this stage. up to eight nurses participated in each focus group. during the focus groups, nurses were asked to reflect upon their experience with ebds. in particular, they were asked to talk about how they use and their experiences with the system to document and deliver nursing care, what features of the technology are most useful/ unhelpful to achieve these objectives, how this new technology influenced your work in acutecare setting in the hospital (time, nursing care documentation and delivery, interactions with patients and other healthcare professionals, nursing care process, workflows,...), and/or impacted on the quality and safety of patient care. nurses were encouraged to suggest changes. non-participant observations of nurses’ interactions with the system were also conducted. the observers followed the nurses around the wards during times pre-identified to capture ‘peak use’ of the system. observers used a purpose specific semi-structured observation data collection tool (a form on an ipad) to record the date, time, location, and an audio recorded description of each interaction between the nurse and the ebds system. the observed nurses were wearing a mp3 recorder and were asked to speak aloud when interacting with the system. their verbal interactions with patients and other hospital staff were also recorded during the observation session. in total, 48 hours of observation data were collected and eight participating nurses were observed during stages 2 and 3. this paper presents a conceptual mapping as well as empirical findings from our analysis of the collected qualitative data through the lens of activity theory. the data were analysed using content and thematic analysis (braun & clarke, 2006; kvale, 1996). data analysis steps include: becoming familiar with the data, initial coding using activity theory concepts, identifying themes (e.g. support for goal achievement, affecting and problematic, reflecting reality), iterative coding and reviewing themes, refining themes and producing the report. in the initial coding cycle, coding nodes were pre-determined based on essential activity theory constituents including subject, tool, object, rule, division of labour, and community. in the subsequent coding cycles, new coding nodes were generated to code relevant expanded triangles i.e. ternary relationships between tool and other constituents (triangles adapted from activity-oriented design method, see mwanza-simwami, 2011; sadeghi et al., 2014, see table 5). additional nodes included high and low usability, high fidelity and low fidelity; sub-nodes emerged from the observation and focus group data for example navigation, interface, efficiency, training and support, improved safety, safety concern, privacy. a combination of australasian journal of information systems nguyen & wickramasinghe 2017, vol 21, research on health information systems mediating role for nursing is 10 software tools including nvivo and xl were used for data coding, data management and theme review. nvivo was used to record the coding nodes and select and assign coded texts to the nodes, not for automated coding. the coding and theme identification was done and reviewed by the researchers. consensus was obtained though research meetings. the nvivo coded data grouped by initial themes were retrieved and grouped in xl for subsequent thematic analysis. 5 results as described in section 2, activity theory describes the human activity as mediated by tools and its cultural-historical context. the nursing documentation activity is thus mediated by different tools through the historical continuity of how nursing activities are organised and nursing practice is carried out in the specific hospital setting. the results from this study are presented in terms of i) nurse-tool-documentation relationship and cycle of expansive learning as well as (ii) the mediating role of the ebds. 5.1 nurse-tool-documentation relationship and the cycle of expansive learning the subject-object dialectic relationship between nurse and documentation of patient care was mediated by different tools at different stages of the implementation. during stage 1, ebds was introduced to nurses while they still used paper forms to document patient care data. in stage 2, to ensure patient safety requirements (requested by the hospital management committee; ethics and legal), nurses used the system in parallel with paper forms to document patient care. their documentation activity was mediated by both the paper forms and ebds. the observations revealed that data entry was the primary interaction between nurse and ebds. occasionally nurses used the system to check the patient’s id and vital signs. they still relied on paper forms and charts to access the patient care data. in stage 3, ‘super user’ nurses used the system to document and access patient care data without paper forms for a short period of time. the interactions between the nurse and paper and ebds as tools were found to be object-oriented, for the purpose of documenting and accessing patient data for care delivery. according to activity theory, tensions and conflicts drive change, leading to the creation of new tools and to changes to the current practice; this process is refereed to as externalisation (engeström, 1987). in this case, the ebds system was developed in response to tensions and conflicts recognised in a wider context of nursing practice rather than only within the involved hospital. the implementation of the system demonstrates that the hospital recognised the current tensions and conflicts in using paper-documentation in acute healthcare nursing, thus the need for such as system. externalisation was noted through nurses’ sharing of comments about the need for change in their documentation practice; “biggest problems is that the educators here are wanting it to stay the say way as the paper forms, and that's reflecting on the way the system has been working. until they embrace the change, it's not going to work.” [fgb] further, they expressed the vision that electronic nursing documentation is the way for the future: “it's the way to go, it's electronic, definitely. there's no going back” because it is about “being able to work at a fast pace” [fga] and having everything “in one place” [fgb]. therefore in this implementation, externalisation is manifested through the introduction of ebds as an innovation in documentation practice. internalisation involves the reproduction of culture as a result of innovation (engeström, 1987). in this case, internalisation is manifested through supporting and training nurses to use ebds as an electronic tool in stage 1, and to embrace associated cultural shifts in nursing documentation in stages 2 and 3. stage 1 was designed to install the ebds system and train the nurse participants. cultural shifts include significant changes from paper to electronic documentation, from delayed to real-time documentation, and a change in documentation location from nurses’ station to patient bedside. as observed throughout the three stages, clinical champions and the ebds team were around to provide training and assistance to the participating nurses when needed. training was found to be an important issue: training and assistance was appreciated by the nurses: “we've had a lot of support from the researchers and the ebds team” and “always someone here to help us, so that aspect is very good” [fga]. australasian journal of information systems nguyen & wickramasinghe 2017, vol 21, research on health information systems mediating role for nursing is 11 nurses expressed a willingness to learn to better use ebds: “we'll just carry on doing the best we can to come to grips with it” [fga] and “just learn it and deal with it” [fgc]. internalisation was manifested through nurses’ overall attitude and learning towards embracing technology solution and associated cultural shifts. 5.2 examination of the mediating role of ebds an examination of how the ebds system mediated existing relationships revealed its potential; i.e., positive aspects, as well as areas for improvement. the mediating role of ebds was examined in relevant ternary relationships with other constituents (adapted from mwanzasimwami, 2011; sadeghi et al., 2014) and a summary of findings is provided in table 5. generic relationships contextualised relationships mediating role of ebds facilitating (positive impacts of ebds) affecting (current negative impacts of ebds i.e. areas for improvement) subject, tool, and object (see subsection 4.2.1) nurse, ebds, and nursing documentation how does ebds facilitate/affect nurses in nursing documentation? ebds’s relative ease of use; systematic data entry; useful data source and functionality to support for care delivery and nursing judgement; improved efficiency. usability (navigation, presentation and data entry options). a roadmap of ebds and navigation. subject, tool and division of labour (see subsection 4.2.2) nurse, ebds, and distribution of work, power, and responsibilities in acute care wards how does ebds facilitate/affect nurses in undertaking their role, duties and responsibilities assigned to them? functionality to allocate nurses and assign access permission accordingly; retrieve patient data from ebds during handover (though limited). lack of shiftbased plan tool, rules, object (see subsection 4.2.3) ebds, nursing discipline, workflows, and other hospital and broader healthcare regulations, and nursing documentation how well does ebds-enabled documentation map with nursing disciplines, workflows, and hospital regulations? systematic documentation, ebds functions, forms and charts, alerts and prompts were designed based on the nursing disciplines and hospital forms and charts. potential to be more prompting. gaps in between ebds and the existing documentation charts, forms, and paper layout design. australasian journal of information systems nguyen & wickramasinghe 2017, vol 21, research on health information systems mediating role for nursing is 12 generic relationships contextualised relationships mediating role of ebds facilitating (positive impacts of ebds) affecting (current negative impacts of ebds i.e. areas for improvement) subject, tool and community (see subsection 4.2.4) nurse, ebds, and hospital healthcare community (other nurses, patients, healthcare professionals) how does ebds facilitate/affect nurses when working with other nurses and the healthcare community nurses visited patients more frequently and spent more time with the patients; potential of ebds in sharing and communicating patient data with healthcare professionals. patient private time might be affected. there may be a lack of nurse-patient interactions if the nurse is not familiar with the system. community, tool, object (see subsection 4.2.5) hospital healthcare community, ebds, and nursing documentation (thus quality and safety) of patient care how well does ebds facilitate/affect communication and collaboration among the healthcare community? potential data sharing to support collaborative care and care coordination. need to support synchronous multidisciplinary care coordination especially in emergency situations. table 5. the mediating role of ebds in key activity theory ternary relationships often, the mediating role of ebds manifested in more than one relationship. for example, as a result of effective mapping between the system and nursing disciplines and process (tool, rules, and object), ebds also facilitated nurses to document patient care data (subject, tool, and object). therefore, the mediating role of ebds is noted in relation to different relationships when appropriate. 5.2.1 subject, tool and object in the central relationship of subject, tool and object, the mediating role of ebds is concerned with how the system facilitates (or affects) nursing documentation. the mediating role of ebds manifested through different aspects: at different levels of ease of use, forcing systematic data entry in documenting, how well ebds forms, charts and data reflect the patient care ‘reality’, levels of support for nursing judgement, change the documentation location, shift to real-time documentation, and work efficiency. in stage 2, nurses identified a number of usability issues in regard to the user interface including information navigation, screen layout, data entry options. nurses suggested changes to improve the information presentation and data entry methods such as adding bed numbers and a menu for frequently used options. as a result of implemented changes in ebds, nurses found the ebds system to be easier to use in stage 3, compared to the version used in stage 2. for example, “the system became a lot more usable to us as the improvements and as things changed, it definitely became more user friendly” [fgc]. areas for further improvements include a roadmap of the complex system and better navigation mechanisms to allow them to move easily between different screens to locate and review information from multiple relevant charts simultaneously. the ebds system was observed to force nurses to enter patient data systematically. nurses were required to move through all the required forms and steps, complete documentation from one patient before their next task. the ebds bedside records were found to be a useful source of patient data that reflects care plans and delivered care: “(all is) in one place; everyone knows where it is and it doesn't move” [fgb] and “there's the patient's details, why they're here and their history. we have known in our head patients, but if you want something a lot australasian journal of information systems nguyen & wickramasinghe 2017, vol 21, research on health information systems mediating role for nursing is 13 more how long ago was that operation, you can do that at the bedside, which is useful” [fga]. therefore, ebds enforced systematic documentation, hence it also mediated the relationship rules-object. due to the limited scope of the implementation, not all paper charts were translated into ebds. hence, the electronic documented data were incomplete and did not truly reflect patients’ care plans and delivered care. in addition, as nurses used to colour code information in the paper charts, they found the limited grey colour of digital forms in ebds to potentially be misleading and confusing. gaps in between ebds and the documentation requirements (current forms and charts) can be perceived as a limitation of the system in mediating the relationship between rules and object, which is affecting the subject and object relationship. nurses found ebds to have the potential to reduce time wasted due to paper documentation, walking between patients and nurses’ stations, looking for paper forms, and data loss due to misplacement of papers: “it would streamline our work so that it was quicker and more efficient” [fga]. however, in stage 2, their efficiency decreased because of the extra time nurses spent with learning to use ebds and because of the usability issues, as agreed by nurses: “(there is the) potential to save time when you get used to it” [fgb]. in stage 3, the participating nurses became familiar with the system and many usability issues had been addressed, nurses found their work efficiency increased: “in the final stage, (ebds) definitely saved me time looking for charts” [fgc]. ebds was found to increase efficiency in subjectobject interactions in this final stage. 5.2.2 subject, tool and division of labour ebds was observed to provide support for division of labour although it was rather limited. it allowed nurse unit managers (num) to allocate nurses and their access permissions in the system according to their roles and responsibilities. during handover, nurses retrieved data from ebds mostly for patient identification and occasionally for observation data. a gap was found to be the lack of shift-based plan in the system. nurses were required to go to each patient’s bedside to access the patient care plan and data. therefore, they found it difficult to plan and prioritise their shift-based care activities for all the patients assigned to them: “so it's not really reflecting what we would really do with 12 patients” [fgc]. this can be seen as low fidelity of the system (as a tool) in facilitating the relationship between subject and division of labour, subsequently affecting the subject and object relationship. 5.2.3 tool, rules, and object in addition to ‘forcing’ nurses to systematically enter patient care data, the system ebds generated prompts for nurses to complete missing information and provided reminders to check on patients, thus potentially improved patient safety. particularly, nurses found the system’s automatic calculation of the patient fluid balance to be useful: “you don't need to calculate the vomit, how many and how much, as you cannot always hold a calculator, ebds saves time” (recorded observation audio). although compared to paper forms, ebds was found to better mediating the relationship between rules and object, nurses wanted the system to be “more prompting” rather than “being a mirror image of the paper forms” [fgb]. another benefit of the ebds prompts and alerts is that it reminds nurses about the scheduled tasks and ‘forced’ nurses to document them in real-time, rather then keeping information in head or paper notes and entering it into the system later, often at the end of each shift. realtime documentation at patient bedside is important to ensure accurate and complete data that reflects the delivered patient care (i.e. the reality). fidelity of the system in the above examples is manifested through enabling better practice, i.e. ensuring systematic documentation and preventing care delays or omissions. 5.2.4 subject, tool and community in regard to the subject, tool and community relationship, ebds appeared to affect the nursepatient interactions. it ‘forced’ nurses to document patient care data at the patient bedside; as a result they visited patients more frequently and spent more time with the patients. while, this could potentially increase nurse-patient interactions and improve patient safety, however australasian journal of information systems nguyen & wickramasinghe 2017, vol 21, research on health information systems mediating role for nursing is 14 a number of concerns were also noted. as nurses were not familiar with the system, their focus was ‘fixed’ on the screen rather than on the interaction with patients. in addition, nurses felt that bedside documentation could be an intrusion on patient’s privacy “i felt like an intrusion yesterday when i was entering stuff and they had visitors, or she had a phone call. i'm standing there and i feel like i'm eavesdropping” [fga]. further in regard to the subject, tool and community relationship, the participating nurses also recognised the potential of ebds in sharing and communicating patient data with other healthcare professionals if the system is integrated with other systems. “i think once clinicians get on board, i think it will potentially save a lot of time for them as well…whoever wants to access our kind of documentation, i think it would just be so much easier having everything in one place, and only that one place. everyone can come to it” [fgc]. therefore, ebds has the potential to support the relationship between community and object through providing information required for collaborative care between different clinical disciplines. 5.2.5 community, tool and object in addition to the above mentioned potential of the system to facilitate collaboration care between healthcare practitioners, there was a concern about the system being incapable to provide support for synchronous multi-disciplinary care coordination. nurses identified the central data access point of the system as a potential problem in complex emergency scenarios such as met (medical emergency team) calls: “(with paper) someone's got the history, someone's got the obs chart, someone's got something else, whereas if it's all just on that system, i think it would be difficult if someone's trying to input information while someone's trying to look up information” [fgc]. 6 discussion and conclusion nursing documentation is a complex human activity. it is object-oriented toward documenting and providing complete and accurate records of patient data and delivered care, partly due to legal requirements. data retrieved from the records are important to achieve high quality, safe care delivery, process efficiency, and positive patient outcomes. in current practice, the documentation activity has been mediated by paper forms and records. the new tool ebds was introduced. for a defined period of the implementation, we conducted this study using the lens of activity theory to facilitate a rich understanding of the role of the system in mediating nurses around documentation of patient care data in acute care wards. the introduction of ebds to acute care can be seen as an externalisation in response to the need for better technology solutions to support nurses as recognised within the nursing disciplines (mills et al., 2013; stevenson et al., 2010). as revealed in this implementation, the internalisation led to cultural shifts in nursing documentation. the nurse-documentation relationship was mediated by the new tool (ebds): change to their documentation location leading to more frequent visits to patients, time duration with patients, systematic data entry, real-time documentation, better support for nursing judgement, reduced time waste in looking for paper forms etc. a number of potential benefits as well as usability issues were identified. having identified various usability issues of the tool (e.g. user interface issues and lack of support for shift-based care plan), nurses suggested changes during the ebds implementation. this demonstrates the dynamics of the internalisation process when ebds and nurses’ activities are appropriated to reproduce the new ebds-enabled documentation practice (allen et al., 2011). the study also highlights the active participation of the subject in this central subject, tool and object relationship of the human activity system. previous research tends to examine technology adoption at the individual rather than social level (sun & qu, 2015). in this study, the mediating role of the tool in the relationship with community elements was also observed. fidelity of the system to nursing discipline and practice can be examined through the mediating role of the system in terms of how it reflects rules, division of labour, its potential to support communication and collective care between nurses and other healthcare professionals in the hospital community. low fidelity could be australasian journal of information systems nguyen & wickramasinghe 2017, vol 21, research on health information systems mediating role for nursing is 15 seen, for example, through the lack of a shift-based plan and of support for synchronous multidisciplinary care coordination. the community, tool, and object relationship needs further investigation as it involves the mediating elements (tool and community) in the object-oriented human activity. further, it was found that the mediating role of the system appeared to affect different key relationships in the activity theory as the elements are interconnected. for example, the relationship subject-object is mediated by the tool (usability) as well as by how well the tool reflects rules and division of labour in the organisation (fidelity). during this implementation, ebds was used by nurses (end users) and as a stand-alone system, not integrated with other systems nor used by other healthcare professionals in the hospital. therefore, the mediating role of the system in the other community relationships (not involving nurses as subject) was rather limited and not the focus of the study. future research will serve to address these aspects in detail. other limitations are briefly noted: it is a single case study that took place in two acute care wards, thus the findings are indicative and useful, but cannot be universally generalised. further, nurses’ individual/moderating factors such as age, gender, backgrounds, technology competency, and years of experience were not included in this analysis. in our future work, we will also expand our analysis to examine in detail moderating variables such as relative experience of nurses with technology as well as age and gender. other technical and organisational aspects such as network connection, data security, patient safety and risk of infection were not part of this study. in summary, the paper offers an examination of the mediating role for a nursing information system during an implementation in a naturalistic acute care hospital context. the study demonstrates an application of activity theory in information systems in a healthcare setting. the theoretical lens of activity theory reveals a socio-technical dialectic story of how the new nursing documentation culture and practice emerges, socially constructed and technically mediated. activity theory was found to be useful to unpack the issues around usability at the nurse-tool-documentation, and fidelity around the three social constituents including rule, community and division of labour. further, activity theory is significantly rich to enable the dynamic and complex operations within an acute healthcare nursing context to be examined in depth. hence, this study has significant implications for both theory and practice. the paper contributes to theory by being one of the first studies to incorporate activity theory as a theoretical lens of enquiry in the context of acute healthcare nursing with technology implementation. activity theory was found to be useful in examining technology impact in the context of human activities especially during a social change, through their interactions with existing and new tools and the community of practice toward their objective in a specific socioorganisational context. in this way we have also addressed in some fashion a key void as noted at the start of the paper; namely, the need for a rich analytic lens to gain an in-depth understanding of his in nursing contexts. there exist several opportunities for future research to integrate activity theory with other information systems theories to better conceptualise and evaluate the acceptance, use, and effectiveness of information system as a mediating/enabling tool for human activity system. for example, we believe that combining activity theory with theories like unified theory of acceptance and use of technology (v. venkatesh et al., 2003; viswanath venkatesh et al., 2012), information systems success model (delone & mclean, 2003), and effective use of information systems (burton-jones & grange, 2013) will enable a rich and full analysis to ensue which will in turn serve to unpack deep and far reaching implications. in summary then, we note that given the dynamic and complex nature of operations in an acute healthcare nursing context, activity theory provides a useful and rich lens to systematically analyse critical interactions. however, we believe an even more powerful analysis will ensue if activity theory is combined with other is theories and plan to investigate this in our future research. in terms of practical implications, the study strongly suggests the importance of engaging nurses in technology implementation, training and change management. nurses should take active participation in the expansive learning process when new patient care data, systems and australasian journal of information systems nguyen & wickramasinghe 2017, vol 21, research on health information systems mediating role for nursing is 16 practices are appropriated at both individual and community levels. therefore, practitioner engagement is critical and understanding how nurses may modify/adapt care path practice using the technology is an essential part of this. as this is an important finding, our future work will unpack this further. acknowledgement the authors would like to thank epworth healthcare and the research team including, prof bridie kent, prof mari botti, a/prof bernice redley, peter haddad and dr. imran muhammad. we are also indebted to the company smartward for their support. references ackerman, s. l., tebb, k., stein, j. c., frazee, b. w., hendey, g. w., schmidt, l. a., & gonzales, r. (2012). benefit or burden? a sociotechnical analysis of diagnostic computer kiosks in four california hospital emergency departments. social science & medicine, 75(12), 2378-2385. doi: http://dx.doi.org/10.1016/j.socscimed.2012.09.013 aktharsha, u. s., & anisa, h. (2011). impact of information technology on nursing practices. iup journal of information technology, 7(2), 33-46. allen, d., karanasios, s., & slavova, m. (2011). working with activity theory: context, technology, and information behavior. journal of the american society for information science & technology, 62(4), 776-788. doi: 10.1002/asi.21441 american college of physicians and american ehr partners. (2013). challenges with meaningful use: ehr satisfaction & usability diminishing, survey of clinicians: user satisfaction with electronic health records has decreased since 2010. from http://conference.himss.org/himss13/pdfs/61.pdf ammenwerth, e., rauchegger, f., ehlers, f., hirsch, b., & schaubmayr, c. (2011). effect of a nursing information system on the quality of information processing in nursing: an evaluation study using the his-monitor instrument. international journal of medical informatics, 80(1), 25-38. doi: http://dx.doi.org/10.1016/j.ijmedinf.2010.10.010 australian nursing federation. (2009). ensuring quality, safety and positive patient outcomes. why investing in nursing makes sense. banner, l., & olney, c. m. (2009). automated clinical documentation: does it allow nurses more time for patient care. cin computers informatics nursing, 27(2), 75-81. doi: 10.1097/ncn.0b013e318197287d blair, w., & smith, b. (2012). nursing documentation: frameworks and barriers. contemporary nurse, 41, 160+. braun, v., & clarke, v. (2006). using thematic analysis in psychology. . qualitative research in psychology, 3(2), pp. 77-101. . burton-jones, a., & grange, c. (2013). from use to effective use: a representation theory perspective. information systems research, 24(3), pp. 632-658. doi: doi:10.1287/isre.1120.0444 carayon, p., cartmill, r., blosky, m. a., brown, r., hackenberg, m., hoonakker, p., . . . walker, j. m. (2011). icu nurses' acceptance of electronic health records. journal of the american medical informatics association, 18(6), 812-819. doi: 10.1136/amiajnl-2010000018 coleman, a., & coleman, m. f. (2013). activity theory framework: a basis for e-health readiness assessment in health institutions. journal of communication, 4(2), pp. 95100. australasian journal of information systems nguyen & wickramasinghe 2017, vol 21, research on health information systems mediating role for nursing is 17 cornell, p., riordan, m., & herrin-griffith, d. (2010). transforming nursing workflow, part 2: the impact of technology on nurse activities. journal of nursing administration, 40(10), pp. 432-439. doi: 1090b1013e3181f1092eb1093f. courtney, k., demiris, g., & alexander, g. (2005). information technology: changing nursing processes at the point-of-care. nursing administration quarterly, 29(4), 315-2322 crawford, k., & hasan, h. (2006). demonstrations of the activity theory framework for research in information systems. 2006, 13(2). doi: 10.3127/ajis.v13i2.40 cresswell, k. m., worth, a., & sheikh, a. (2010). actor-network theory and its role in understanding the implementation of information technology developments in healthcare. bmc medical informatics and decision making, 10(1), 67. darbyshire, p. (2004). 'rage against the machine?': nurses' and midwives' experiences of using computerized patient information systems for clinical information. journal of clinical nursing, 13(1), 17-25. de veer, a. j. e., & francke, a. l. (2010). attitudes of nursing staff towards electronic patient records: a questionnaire surveyoriginal research article. international journal of nursing studies, 47(7), p. 846-854. delone, w. h., & mclean, e. r. (2003). the delone and mclean model of information systems success: a ten-year update. journal of management information systems, 19(4), pp 9-30. dowding, d., mitchell, n., randell, r., foster, r., lattimer, v., & thompson, c. (2009). nurses’ use of computerised clinical decision support systems: a case site analysis. journal of clinical nursing, 18(8), 1159-1167. eley, r., fallon, t., soar, j., buikstra, e., & hegney, d. (2009). barriers to use of information and computer technology by australia’s nurses: a national survey. journal of clinical nursing, 18(8), 1151-1158. engeström, y. (1987). learning by expanding: an activity-theoretical approach to developmental research. helsinki: orienta-konsultit. engeström, y. (1999). outline of three generations of activity theory retrieved 27/07/2015, from http://www.bath.ac.uk/research/liw/resources/models and principles of activity theory.pdf fichman, r. g., kohli, r., & krishnan, r. (2011). the role of information systems in healthcare: current research and future trends. information systems research, 22(3), 419-428. doi: 10.1287/isre.1110.0382 gough, r., ballardie, r., & brewer, p. (2014). new technology and nurses. labour & industry: a journal of the social and economic relations of work, 24(1), 9-25. doi: 10.1080/10301763.2013.877118 häkkinen, h., & korpela, m. (2007). a participatory assessment of is integration needs in maternity clinics using activity theory. international journal of medical informatics, 76(11–12), 843-849. doi: http://dx.doi.org/10.1016/j.ijmedinf.2006.11.003 hashim, n., & jones, m. l. (2007). activity theory: a framework for qualitative analysis. paper presented at the 4th international qualitative research convention (qrc), malaysia. häyrinen, k., saranto, k., & nykänen, p. (2008). definition, structure, content, use and impacts of electronic health records: a review of the research literature. international journal of medical informatics, 77(5), 291-304. hegney, p. d., buikstra, e., eley, r., fallon, t., gilmore, v., & soar, j. (2007). nursing and information technology: final report.: australian nursing federation, http://www.anf.org.au/it_project/. australasian journal of information systems nguyen & wickramasinghe 2017, vol 21, research on health information systems mediating role for nursing is 18 hevner, a., & chatterjee, s. (2010). design research in information systems: theory and practice (1st ed.): springer. hevner, a., march, s., park, j., & ram, s. (2004). design science in information systems research. mis quarterly, 28(1), pp. 75-105. holden, r. j., & karsh, b.-t. (2010). the technology acceptance model: its past and its future in health care. journal of biomedical informatics, 43(1), 159. doi: 10.1016/j.jbi.2009.07.002 hoonakker, p. l. t., carayon, p., brown, r. l., cartmill, r. s., wetterneck, t. b., & walker, j. m. (2013). changes in end-user satisfaction with computerized provider order entry over time among nurses and providers in intensive care units. journal of the american medical informatics association, 20(2), 252-259. doi: 10.1136/amiajnl-2012-001114 international organization for standardization. (1998). international standard iso 9241-11 ergonomic requirements for office work with visual display terminals (vdts) — part 11: guidance on usability, https://www.iso.org/obp/ui/ iso:std:iso:9241:-11:ed1:v1:en: technical committee iso/tc 159, ergonomics, subcommittee sc 4, ergonomics of human-system interaction. jeskey, m., card, e., nelson, d., mercaldo, n. d., sanders, n., higgins, m. s., . . . miller, a. (2011). nurse adoption of continuous patient monitoring on acute post-surgical units: managing technology implementation. journal of nursing management, 19(7), 863-875. doi: 10.1111/j.1365-2834.2011.01295.x ju-ling, h., rai-fu, c., hsiao, j.-l., & chen, r.-f. (2016). critical factors influencing physicians' intention to use computerized clinical practice guidelines: an integrative model of activity theory and the technology acceptance model. bmc medical informatics & decision making, 16, 1-15. doi: 10.1186/s12911-016-0241-3 koppel, r., metlay, j. p., cohen, a., & et al. (2005). role of computerized physician order entry systems in facilitating medication errors. the journal of the american medical association, 293(10), 1197-1203. doi: 10.1001/jama.293.10.1197 kossman, s. p. (2006). perceptions of impact of electronic health records on nurses' work. studies in health technology and informatics, 122, 337-341. kuutti, k. (1996). activity theory as a potential framework for human-computer interaction research. in b. a. nardi (ed.), context and consciousness: activity theory and humancomputer interaction (chapter 2, pp. 9-22). ma, usa: massachusetts institute of technology cambridge, mit press. kvale, s. (1996). interviews: an introduction to qualitative research interviewing. thousand oaks, calif.: sage publications. lærum, h., karlsen, t. h., & faxvaag, a. (2004). use of and attitudes to a hospital information system by medical secretaries, nurses and physicians deprived of the paper-based medical record: a case report. bmc medical informatics and decision making, 4. doi: 10.1186/1472-6947-4-18 lau, f., kuziemsky, c., price, m., & gardner, j. (2010). a review on systematic reviews of health information system studies. journal of the american medical informatics association, 17(6), 637-645. doi: 10.1136/jamia.2010.004838 lee, t.-t., mills, m. e., bausell, b., & lu, m.-h. (2008). two-stage evaluation of the impact of a nursing information system in taiwan. international journal of medical informatics, 77(10), 698-707. doi: http://dx.doi.org/10.1016/j.ijmedinf.2008.03.004 nguyen, l., haddad, p., moghimi, h., coleman, k., redley, b., botti, m., & wickramasinghe, n. (2015). developing an information system for nursing in acute care contexts paper presented at the pacis 2015 proceedings of the 19th pacific asia conference on information systems, singapore. http://aisel.aisnet.org/pacis2015/219 australasian journal of information systems nguyen & wickramasinghe 2017, vol 21, research on health information systems mediating role for nursing is 19 leontiev, a. n. (2009). problems of the development of mind. ohio, usa: marxists internet archive. lin, h.-c., chiou, j.-y., chen, c.-c., & yang, c.-w. u. n. e. t. (2016). understanding the impact of nurses' perception and technological capability on nurses' satisfaction with nursing information system usage: a holistic perspective of alignment. computers in human behavior, 57, 143-152. doi: 10.1016/j.chb.2015.12.001 meijden, m. j. v. d., tange, h. j., troost, j., & hasman, a. (2003). determinants of success of inpatient clinical information systems: a literature review. journal of the american medical informatics association, 10(3), 235-243. miller, a., scheinkestel, c., & steele, c. (2009). the effects of clinical information presentation on physicians' and nurses' decision-making in icus. applied ergonomics, 40(4), p. 753761. mills, j., chamberlain-salaun, j., henry, r., sando, j., & summers, g. (2013). nurses in australian acute care settings: experiences with and outcomes of e-health. an integrative review. international journal of management & information technology, 2(2), pp. 18 moody, l. e., slocumb, e., berg, b., & jackson, d. (2004). electronic health records documentation in nursing: nurses' perceptions, attitudes, and preferences. cin: computers, informatics, nursing, 22(6), 337-344. mwanza-simwami, d. (2011). aodm as a framework and model for characterising learner experiences with technology. journal of e-learning and knowledge society, 7(3), 7585. nakamura, m. m., ferris, t. g., desroches, c. m., & jha, a. k. (2010). electronic health record adoption by children's hospitals in the united states. archives of pediatrics and adolescent medicine, 164(12), 1145-1151. doi: 10.1001/archpediatrics.2010.234 nardi, b. a. (1996). activity theory and human computer interaction in b. a. nardi (ed.), context and consciousness: activity theory and human-computer interaction (chapter 1, pp. 4-8). cambridge: massachusetts institute of technology cambridge, mit press. nguyen, l., bellucci, e., & thuy, n. l. (2014). electronic health records implementation: an evaluation of information system impact and contingency factors. international journal of medical informatics, 83(11), pp: 779-796. . novak, l. l., holden, r. j., anders, s. h., hong, j. y., & karsh, b.-t. (2013). using a sociotechnical framework to understand adaptations in health it implementation. international journal of medical informatics, 82(12), e331-e344. doi: http://dx.doi.org/10.1016/j.ijmedinf.2013.01.009 rafferty, a. m., clarke, s. p., coles, j., ball, j., james, p., mckee, m., & aiken, l. h. (2007). outcomes of variation in hospital nurse staffing in english hospitals: cross-sectional analysis of survey data and discharge records. international journal of nursing studies, 44(2), 175-182. doi: 10.1016/j.ijnurstu.2006.08.003 sadeghi, p., andreev, p., benyoucef, m., momtahan, k., & kuziemsky, c. e. (2014). activity theory driven system analysis of complex healthcare processes. paper presented at the proceedings of twenty second european conference on information systems ecis'2014 tel aviv. http://aisel.aisnet.org/ecis2014/proceedings/track09/15 shaban, r. (2005). theories of clinical judgment and decision-making: a review of the theoretical literature. australasian journal of paramedicine, 3(1). staggers, n., weir, c., & s., p. (2008). patient safety and health information technology: role of the electronic health record. in e. in: hughes rg (ed.), patient safety and quality: australasian journal of information systems nguyen & wickramasinghe 2017, vol 21, research on health information systems mediating role for nursing is 20 an evidence-based handbook for nurses (chapter 47, pp. 91-133). rockville, md, us: agency for healthcare research and quality. stevenson, j., nilsson, g., petersson, g., & johanssson, p. (2010). nurses' experience of using electronic patient records in everyday practice in acute/inpatient ward settings: a literature review. health informatics journal, 16(1), 63-72. sun, j., & qu, z. (2015). understanding health information technology adoption: a synthesis of literature from an activity perspective. information systems frontiers, 17(5), 11771190. doi: 10.1007/s10796-014-9497-2 thompson, c., & dowding, d. (2002). clinical decision making and judgement in nursing. london: churchill livingstone. tilahun, b. b. t. u.-m. d., & fritz, f. (2015). comprehensive evaluation of electronic medical record system use and user satisfaction at five low-resource setting hospitals in ethiopia. journal of medical internet research, 17(5), 1-1. doi: 10.2196/medinform.4106 timby, b. k. (2009). nursing process fundamental nursing skills and concepts (9th ed., chapter 2). vatnøy, t. k., vabo, g., & fossum, m. (2014). a usability evaluation of an electronic health record system for nursing documentation used in the municipality healthcare services in norway. in f. f.-h. nah (ed.), hci in business: first international conference, hcib 2014, held as part of hci international 2014, heraklion, crete, greece, june 22-27, 2014. proceedings (pp. 690-699). cham: springer international publishing. vega-barbas, m., pau, i., martín-ruiz, m. l., & seoane, f. (2015). adaptive software architecture based on confident hci for the deployment of sensitive services in smart homes. sensors (14248220), 15(4), 7294-7322. doi: 10.3390/s150407294 venkatesh, v., morris, m. g., davis, f. d., & davis, g. b. (2003). user acceptance of information technology: toward a unified view. mis quarterly, 27, pp. 425-478. venkatesh, v., thong, j. y. l., & xin, x. (2012). consumer acceptance and use of information technology: extending the unified theory of acceptance and use of technology. mis quarterly, 36(1), pp: 157-178. vygotsky, l. s. (1978). mind and society: the development of higher mental processes. cambridge, ma: harvard university press. waycott, j., scheepers, r., davis, h., howard, s., & sonenberg, l. (2014). the individual in multiple interacting activity systems: it-supported diabetes management. information technology & people, 27(4), 463-481. doi: 10.1108/itp-11-2013-0195 weinhara, m., stoicu-tivadar, l., & dagres, c. (2009). early stage testing of user's satisfaction after implementation of a central electronic health record (ehr) system in serbia. journal on information technology in healthcare, 7(2), 127-133. westbrook, j. i., ling, l., georgiou, a., paoloni, r., & cullen, j. (2013). impact of an electronic medication management system on hospital doctors' and nurses' work: a controlled prepost, time and motion study. journal of the american medical informatics association, 20(6), 1150-1158. doi: 10.1136/amiajnl-2012-001414 whittaker, a. a., aufdenkamp, m., & tinley, s. (2009). barriers and facilitators to electronic documentation in a rural hospital. journal of nursing scholarship, 41(3), 293-300. doi: 10.1111/j.1547-5069.2009.01278.x yen, p.-y., & bakken, s. (2012). review of health information technology usability study methodologies. journal of the american medical informatics association : jamia, 19(3), 413-422. doi: 10.1136/amiajnl-2010-000020 australasian journal of information systems nguyen & wickramasinghe 2017, vol 21, research on health information systems mediating role for nursing is 21 copyright: © 2017 nguyen & wickramasinghe. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems adhikari, mathrani & scogings 2017, vol 21, selected papers from acis 2016 byod classrooms: access, capability, outcome divides 1 a longitudinal journey with byod classrooms: issues of access, capability and outcome divides janak adhikari open polytechnic new zealand janak.mailbox@gmail.com anuradha mathrani institute of natural and mathematical sciences massey university new zealand chris scogings institute of natural and mathematical sciences massey university new zealand abstract bring your own devices (byod) classrooms is the latest addition to the ongoing quest for transforming pedagogical practices and driving educational development outcomes. governments and policymakers around the world are embracing the idea of integrating digital learning technologies into educational settings backed by research on the benefits offered like the lifelong development of individual learners. while technological interventions open up unlimited possibilities for accessing information and improving collaboration, thereby engaging learners with more interactive learning activities, a new type of gap between individuals could also take shape as the penetration of technologies and adoption stages advance. results from past projects of similar nature within educational settings have indicated the possible rise in the gap among individuals based on digital access (i.e., equity in access/ownership of digital learning technologies), digital capability (i.e., equity in digital/information literacy skills and usage) and digital outcome (i.e., equity in knowledge acquisition and progression). therefore, we have conducted a longitudinal study to investigate a byod initiative by a new zealand school. this study shares rich insights in the context of technology-mediated pedagogies and specifically byod classroom, as to how digital divides moved beyond access and skills to ensure inclusive learning outcomes. as a part of the five-year study of the technology-mediated teaching and learning initiative, we have been able to explain some of the unanswered questions around the issue of digital divides in the learning process. we investigated issues pertaining to digital divide in the context of byod classrooms to make the following revelations. first, the byod classroom initiative did not end up accentuating existing gaps in access to digital tools and technologies, despite earlier studies indicating towards increase in gaps. second, our analysis strongly indicated the changing nature of digital divides with the presence of gaps in terms of information literacy and critical thinking ability, as the byod classroom progressed to mature stage. this was eventually bridged in the later stage, as students slowly adjusted to the classroom curricular structures in the byod classroom. third, learner self-efficacy has been identified as a determinant of learning outcomes. in the earlier phase of ict adoption, learner self-efficacy is influenced by a combination of information literacy, critical thinking ability, and positive motivation; however subsequently, self-efficacy influences affordances in various aspects of social cognitive abilities related to individual’s learning activities affecting how learners engage and apply technology to achieve learning outcomes. keywords: byod, schools, digital access divide, digital capability divide, digital outcome divide australasian journal of information systems adhikari, mathrani & scogings 2017, vol 21, selected papers from acis 2016 byod classrooms: access, capability, outcome divides 2 1 introduction during the late 1990s, innovations in the information and communication technologies disrupted the traditional way of life introducing many opportunities and services people never imagined. modern developments in digital technologies marked the beginning of the information age (sterling, 1997) and revolutionized our working lives by enabling better access to information, ability to transfer and share information quickly, and improving communication and collaboration between individuals. this marked the start of a shift in the economy from traditional industry to an economy based on the information, i.e., an information society. however, despite the potential benefits offered by the information society and the increased penetration of digital technologies in society, not everyone around us has the same privilege of using the services offered by digital technologies, thereby restricting opportunities in the information society for some. that includes the inability to access, operate, and use the digital tools and technologies meaningfully to benefit from it. as a result, many individuals and societal groups are still facing digital exclusion from the rest of the population. nonetheless, digital revolution has helped transform the society positively with many technology-driven services and opportunities despite challenges around the equity. with the increased ubiquity of digital technologies into almost every aspect of our lives, the need for appropriate digital and information literacy skills are on the rise. in this changing technological world, digital skills are now considered as the third most important life skill alongside numeracy and literacy (dfes, 2003; johnson, levine, smith, & stone, 2010). therefore, there is a common consensus among educators and policy makers that the integration of digital tools and technologies into pedagogical practices has a lot to offer to improve the quality and changing needs of the education. one example of the facilitation of better technological infrastructure resulted from a government policy is the ultrafast broadband (ufb) initiative in new zealand, which prioritized and mandated connectivity to education sectors (parsons, 2016). one of the aims of the ufb initiative is to bring the school curriculum up to speed with digital learning using the supporting broadband infrastructure. a great deal of transformation is possible in teaching and learning methods through innovation in pedagogical practices, facilitated by modern digital technologies. on one side, the digital learning technologies has opened up the unlimited possibility for access to information, ease of handling and processing information, and improved collaboration opportunities resulting in ever-improving academic activities and results. whereas on the other side, as the penetration of technologies and adoption stages in teaching and learning advances, the digital gap between the individual learners takes shape, which has been conceptualised in a research paper with a three level digital divide framework (wei, teo, chan, & tan, 2011). literature suggests that despite the potential of innovative learning technologies to improve learning outcomes; it could end up accentuating existing digital divides (parr & ward, 2005; rivers & rivers, 2004; winter, 2004) within the current teaching and learning environments. the findings from the study of the digital opportunities pilot projects (digiops)1 indicate that these projects have failed to contribute as were expected from them. in some cases, the initiative designed to fill the gap of digital divide potentially caused the gap to become even wider, challenging the objective of any such technology-mediated teaching and learning initiatives in future. supporting the earlier argument, wei et al. (2011) too caution that as adoption stages of icts advance, there may arise more levels of digital divides based on the equity of information literacy and learning outcomes. we have used the results from the digital opportunities pilot projects as the base to start our investigation into the digital divide phenomenon. . we selected a case of a secondary school in new zealand, which made a decision to transform into technology-mediated teaching and learning, dubbed as the bring your own devices (byod) initiative. we focused the initial phase of our study to confirm the results from the digital opportunities projects. in the first year of 1 the digital opportunities pilot projects (digiops) are partnerships between schools, organizations involved in ict and the ministry of education in new zealand. australasian journal of information systems adhikari, mathrani & scogings 2017, vol 21, selected papers from acis 2016 byod classrooms: access, capability, outcome divides 3 our study (during 2012 and 2013), our inquiry revealed issues related to digital access (i.e., equity in terms of the access/ownership of digital learning technologies among learners) and competency of using technologies available for meaningful learning activities (i.e., digital skills). our initial results also took account of public opinion, which had earlier shown strong resistance to the school’s initiative. these initial results provided a baseline to move the study forward. however, as the byod initiative gained momentum (leading up to the middle of 2014), there was much acceptance for technology-mediated teaching and learning among all the stakeholders, including students, teachers, and parents. the goal of the technology-mediated teaching and learning initiatives is to improve studentlearning outcome, leveraging the opportunities provided by the digital learning technologies. studies on what effect the technology-mediated teaching and learning will have on the existing digital divides, and how this may influence the digital outcomes of the learners subsequently is not currently present in the literature. with the increased penetration of digital technologies into the classroom teaching and learning practices like byod classrooms, studies on how the existing and new digital divides evolve are needed. this study therefore investigates beyond digital access and skills, as the byod initiative progressed into mature stages (i.e., from the middle of 2014 to 2016). as the ability of meaningmaking and critical thinking among students matured, we carried out a deeper investigation on factors pertaining to learning outcomes. during this stage of study, the progression of agency among students across different learning spaces has been analysed using the sociocultural ecological model (scem) for mobile learning (cook, pachler, & bachmair, 2011; pachler, bachmair, cook, & kress, 2010). the scem explains the interrelationships between enablers (technology, curriculum), actors (student’s ability to act) and cultural practices (social interactions in everyday life enabled by technology). this framework underpins our case study analysis to inform on the relationships between different aspects of the byod initiative. 2 digital divides in the learning the unprecedented developments in digital technologies generally referred as the ‘digital revolution’ created the opportunity for better access to information and services for everyone, and changed our view of society, business, and education. generally, access to and use of digital technologies has many benefits to offer. however, in the years after the digital revolution, the pattern of the distribution of icts still raises concern, since not everyone has proper access to them or has adequate skills for making meaningful use of them. many individuals, groups, and societies are still excluded digitally, from the rest of the population, causing digital division between them. the ‘digital divide’ is a term that broadly represents just about any aspect of our daily life and the phenomenon has been researched and defined in many contexts. it first emerged from academic research to signify the gap between people or society who have access to digital technology and those who do not (cullen, 2001; van dijk & hacker, 2003). however, the common understanding behind most of the research is that the digital divide is a complex issue and it is hard to understand the phenomenon within a single context and with a single definition. careful examination of the literature gives no clear evidence of the origin of the term ‘digital divide’ and its meaning is still unclear. the digital divide phenomenon has been described by many authors as the most pressing social, economic, and educational issue of the information age and is now receiving increased attention from researchers and policymakers around the world (dewan, ganley, & kraemer, 2005). the education sector has been an important area of inquiry for many digital divide researchers, as they strive to inform government initiatives on strategies to address issues prevalent here. when information and communication technologies (icts) are introduced into learning environments, they can give rise to innovative curricula changes, inform on novel learning activities and assessment methods, and provide valuable knowledge resources to enhance learners’ academic and social development (demiraslan & usluel, 2008). consequently, a number of research studies have emphasised the integration of digital technologies into existing pedagogies to transform australasian journal of information systems adhikari, mathrani & scogings 2017, vol 21, selected papers from acis 2016 byod classrooms: access, capability, outcome divides 4 teaching and learning (anderson, 2009; prestridge, 2007). however, questions still remain unanswered on the issues of digital divides with the embedding of icts into learning environments. the phenomenon of the digital divide is complex due to the variety of economic, demographic, individual and social variables associated with it. therefore, we have followed the grounded theory as a method for reviewing the literature in the research area as identified by the wolfswinkel, furtmueller, and wilderom (2013) in three stages. stage one defined the scope of the review, search terms and criteria to be considered while scoping the literature review. we have chosen all of the major databases (like scopus, science direct, and web of science, ebsco) to access journal and conference article for our source material. due to the nature of the issue we were investigating, we have also searched for relevant government articles and reports to reflect on the current practices. following table below details the search terms used in the database searches. concept search terms justification digital divide digital divide or digital access divide or levels of digital divide or nature of digital divide to cover the width and depth of the literature on the digital divide phenomenon education digital divide in education or digital divide in teaching and learning or digital divide in learning or digital divide in school capturing the literature on the digital divide in education byod technology enhanced learning or technology-mediated learning or technology mediated learning or byod find the literature on any form of integration of technology in education. table 1: criteria for the search of relevant literature stage two involved combination of search terms with the inclusion criteria and querying on the relevant article databases. stage three involved the task of filtering articles relevant to our study. that included removing any duplicate articles retrieved from across the databases, reading abstracts to find the suitability of the article for inclusion in the review. a backward and then forward analysis was performed to establish the quality of the article. this process resulted in a final set of 29 articles for inclusion. the final list of articles that are reviewed is representative of the existing body of knowledge on the issue of digital divide in the teaching and learning in the context of integration of technology in education. themes emerging from the review and analysis of the literature have been grouped into the following subsections and analysed accordingly. 2.1 access to digital technologies the issue of digital access is the oldest and most widely researched aspect of the digital divide, is characterised at both the individual and the social level, and has always been the easiest determinant for defining digital divides in any of these contexts. in fact, one of the earliest understanding of the digital divide phenomenon was entirely based on the level of access to digital technologies in everyday life. according to van dijk (2005), zhong (2011), and cullen (2001) the digital access divide is the divide between those who have access to icts and those who do not. a large number of studies have been conducted to explore this issue of access and the impact it is causing to individuals and the society in general. when we try to identify the most important background characteristics of the digital access divide; it appears that highly correlated variables of income, socioeconomic status, educational level, occupation, and gender are the main determinants (a. martin & rader, 2002; parker, 2001; parycek, sachs, & schossböck, 2011; van dijk, 2006). this is an indication that individuals and societies with australasian journal of information systems adhikari, mathrani & scogings 2017, vol 21, selected papers from acis 2016 byod classrooms: access, capability, outcome divides 5 lower financial status and educational level may have poor or no access to icts putting them into the wrong side of the digital divide. the digital access divide is also known as the first level digital divide. recent trends in formal education emphasise the integration of digital learning media into existing pedagogies to transform teaching and learning (anderson, 2009; prestridge, 2007). introducing ict provides potentially valuable resources for learners’ academic and social development, and introduces more visual stimulants in the learning environment (demiraslan & usluel, 2008; mardis & everhart, 2013). however, to realise that outcome, it is essential that every learner in the class or school be equipped with personal digital learning devices as a starting point, to enable learners to participate in technology enabled teaching and learning activities. financial sources for that are usually the parents in most cases, and we have discussed earlier that it appears from a previous study that income is a vital factor when it comes to ensuring the appropriate digital access. the world today is suffering from various economic crises, and in the current situation, not every parent will be able to afford expensive digital devices for their children. this can put both the parent and children in a disadvantaged situation with two consequences. one, the learners or children probably will go to school without the digital device, and that would make them feel less involved compared to their peers who have these devices, leading to a poorer learning experience and isolation within the learning process. second, the parents can feel embarrassed for not being able to afford any form of digital devices for their children, while other parents in the same school have been able to afford. this situation again has potential to lead to the feeling of disparity in society leading to a social gap. as a result, some initiatives have been taken by communities and governments around the world to tackle this issue of growing gap of digital access. one of such initiative is digital opportunities pilot project in new zealand discussed in earlier section, which aimed to bridge the gap of access to digital technologies to the students and teachers as a part of the educational initiative. despite the opportunities offered to overcome digital access within the formal educational setting, evaluation of the projects later indicate that any technological intervention does not guarantee the solution of the issue of digital access divide; since this is much more deeper and involves a whole spectrum of contextual factors that exist in society. 2.2 digital /information literacy the knowledge gap hypothesis theory by donohue, tichenor, and olien (1975) suggests that each new medium or technology increases the gap between the information rich and information poor. the differences in access to the medium, technology and control over its use in turn affects the abilities and skills of the users (gaziano, 2010). with the infusion of new media or technologies into our multi-segmented social system, the segments of population having higher socioeconomic status tend to acquire this technology at a faster rate than the lower status segments (yoori & se-hoon, 2009). as a result, the gap in knowledge between these segments increases further. an earlier study on digital divide found that merely offering individuals access to technology is not sufficient to ensure that they can use the medium appropriately to meet their needs or be on the right side of the digital divide. while equitable material access to technology can be a necessary first step to address the phenomenon of the digital divide, it is not sufficient to overcome the issue (hargittai, 2002b). it is equally important to have proper digital/information literacy skills to use the available technologies appropriately and meaningfully (cole, 2001; ghobadi & ghobadi, 2015; yoori & se-hoon, 2009); otherwise, individuals may not be able to take even basic advantage of the technology and resources available to them. after different perspective on the interpretation of digital divide to include the digital capability as one of the aspect (hargittai, 2002a, 2002b); several other authors have also discussed the digital media literacy (warschauer, 2003) and digital skills (van dijk, 2006) in relation to the digital divide phenomenon. this new interpretations of the digital divide phenomenon has been collectively termed as the second level or second order digital divide. australasian journal of information systems adhikari, mathrani & scogings 2017, vol 21, selected papers from acis 2016 byod classrooms: access, capability, outcome divides 6 the concept of the second level digital divide is very much relevant in the context of education. first, because this is already proven that the issue of access is a reality in the current educational landscape and with the help of the knowledge gap hypothesis, we can assume that the gap in the digital and information literacy might follow soon after. secondly, as mentioned in the earlier section, recent trends in formal education emphasise the integration of digital learning technologies into existing pedagogies. there is nothing wrong with this trend, but at the same time, it is vital that it be done in a right way. digital opportunities project referred earlier in the section concluded that the initiative could even accentuate the existing digital divide. the reason for coming to that conclusion from the analysis of the digital opportunities pilot projects is that these initiatives are focused on treating the symptoms of the digital divide phenomenon, not the cause. during the planning and implementation of the digital opportunities projects, the meaning of equity was understood only as a matter of material access and digital skills. instead, the initiatives around the digital teaching and learning should focus on various ways to create opportunities for learners to gain sufficient digital and information literacy skills to enable their participation in learning activities in technologymediated pedagogical context. to realize this, a more learner-centred culture is essential where education agencies and organisations focus on enriching the ability of the learners rather than the technology/medium itself. moreover, it can be done through the combined focus on improved connectivity (or, access to ict infrastructure for education), content (such as digital content extracted from a variety of sources), and with confidence and capability (or use of skills for converting information into knowledge) (ministry of education, 2006). 2.3 learner self-efficacy findings from prior literature on the digital divide phenomenon and outcomes from digital opportunity pilot projects show that equitable material access to digital learning technologies at home and school and having appropriate digital skills are necessary first steps, however, this alone is not sufficient for achieving digital inclusion for every learner. when it comes to the digital access divide, main predicators include the personal attributes like gender, educational level, and social attributes like affordability and socio-economic status. lack of adequate access to digital technologies then limits the individual’s digital/information literacy levels. this can further affect the learning activities of learners, restricting their ability to meet the learning objectives (ersanlı, 2015; wei et al., 2011; yusuf, 2011). when the digital access divide leads to the divide in terms of their learning capabilities, this can soon start influencing various facets of the social cognitive abilities related to the individual’s learning, usually referred as the self-efficacy of the individual/learner. this selfefficacy among learners is very important to translate the opportunities offered by the technology-mediated teaching and learning practices. eventually, the limited self-efficacy levels among learners can potentially cause a life-long impact on the individual’s learning tendencies and abilities which can restrict them in acquiring new skills and knowledge in using the available technologies thereby reducing their opportunities (garipagaoglu, 2013). therefore, there is a need to extend the digital divide research in the context of technologymediated pedagogical practices, towards additional fields of enquiry beyond just access and skills. 2.4 nature of the technology usage by learners in the context of the technology-mediated pedagogical practices, broader definition of equity reaches beyond level of physical access to technologies and skills to make use of those technologies. having adequate access and individual’s ability to utilise the available technologies are necessary first steps towards consuming information. to achieve wider digital inclusion in the context of technology-mediated pedagogical practices, engagement has to move beyond consumption to participation and creation. faced with rich variety of digital tools, learners can engage with digital media and exchange information with each other, for example via posting messages to chat rooms, newsgroup and forums, or use peer-to-peer file australasian journal of information systems adhikari, mathrani & scogings 2017, vol 21, selected papers from acis 2016 byod classrooms: access, capability, outcome divides 7 sharing sites all of which can lead to building up of social capital (brake, 2014) and achieving better learning outcomes. however, some communication tools can often be used for pursuing entertainment and leisure activities such as downloading music, streaming videos, surfing for fun, playing online games and cheering up each other to pass time (van deursen & van dijk, 2014). however, such forms of entertainment and leisure motives can interfere with classroom teaching and learning activities. hence, behavioural aspects of the individual learner are equally important to realise the equity in terms of the learning outcomes among learners. the way individual learners make use of the available technologies can have a huge impact on student achievement, as the usage of digital technologies can be done meaningfully or not (jones & issroff, 2007; wei et al., 2011). in addition to that, since the learners will be having access to all kinds of games and other resources in the internet at their fingertips, the classroom might experience disruptive behaviour from the students (bru, 2006). this remains one of the challenges for teachers to keep the learners on task and guiding them to make meaningful usage of available technologies during the learning activities. therefore, the nature of technology usage by learners is a very important aspect when investigating the issue of the digital divide, as the nature of the technology usage determines the productivity of the learning activities. 2.5 motivation/belief introduction of the technology-mediated teaching and learning requires a big shift in the way teachers teach in classrooms and how students engage in learning activities. redefinitions of the learning activities and changes in the expectation levels means that students often have to come out of their comfort zone and embrace the new way of learning. some might appreciate it and others not. therefore, the shift in the way of learning in the technology-mediated learning environment requires a careful transition through which the learners can be kept motivated. constructivist learning theory by jean piaget holds that learners learn more effectively through the active design and development of projects meaningful to them and the community around them (vogel-walcutt, gebrim, bowers, carper, & nicholson, 2011). through this process of learning, they actively construct new knowledge and become more deeply involved in their learning process. when learners are novices, they need to be guided and supported to construct knowledge and then integrate knowledge to create further knowledge. however, without proper instructional guidance to inform learners on how to add to existing knowledge in their learning process, their belief in themselves could waiver and thereby affect their motivation to learn. for that reason, it is a very important area to investigate and to find how motivational issues can affect the learning outcomes of the learners. 3 analysis and contextualisation of literature into study the world around us is transforming and changing every day, and therefore understanding the phenomenon of digital divide only from the notion of material access (haves and have nots) and adequate digital/information literacy skill (can and cannot) may not be sufficient. apart from those aspects, studies on how individuals engage with and make use of the available technologies, as an expressive tool, in and across different contexts to achieve set goals are very important (brandtzæg, heim, & karahasanović, 2011; brosnan, 1998). as explained in the three level digital divide by wei et al. (2011), the literature shows a clear shift in digital divide research around acceptance and adaptation of digital technologies, the nature of technology use, changes in attitude and behaviour, ability of meaning making, skill development, and knowledge acquisition in recent years. it has been collectively termed as the digital outcome divide or third level divide and is a more recent analysis of the digital divide phenomenon (brandtzæg et al., 2011; gunkel, 2003; lenhart et al., 2003; partridge, 2003; wei et al., 2011; zhong, 2011). australasian journal of information systems adhikari, mathrani & scogings 2017, vol 21, selected papers from acis 2016 byod classrooms: access, capability, outcome divides 8 figure 1: three level digital divide framework (wei et al., 2011) the figure 1 depicts the adoption of information technologies across three stages, having access to icts (digital access divide), developing usage capability (digital capability divide) and achieving outcomes (digital outcome divide) (wei et al., 2011). the importance of computing environments is growing and so is the penetration of digital tools and technologies within every aspect of our daily lives. with the ever increasing adoption of digital technologies in educational activities, it is important to find how that affects the ability to use and the level of outcomes gained through the adequate appropriation of the technologies. there is also an area to understand in this context that, if the digital access divide in anyway contributes to the following levels of the digital divides. this extends the area of enquiry to investigate the effects of the digital access divide on the subsequent two levels of divides. it is therefore evident that, there are still some unanswered questions around, if one level of divide leads to another in the context of technology-mediated teaching and learning process? it is especially unclear how digital access divide will affect the digital capabilities divide and the learning outcomes in turn. therefore, there is a need to extend the digital divide research towards additional fields of enquiry beyond just access and digital skills to understand the evolution of the digital divide phenomenon within the context of technology-mediated teaching and learning environments. specifically, with the increased penetration of digital technologies into the teaching methodologies and learning activities by the byod classroom initiative, the three level digital divide framework by wei et al. (2011) guides the study on how the existing and new digital divides will evolve within the byod classrooms. for that reason, we have adopted the three level digital divide framework and applied it to the context of our study. while adapting the framework for our study, we mapped the three levels of it adoption stages to the three levels of digital divides in the learning process. specifically, ict adoption stages includes access, capability and outcome divide stages, which matches with the digital access divide, digital capability divide and digital/learning outcome divide. figure 2: three levels of the digital divide in learning applied to the context of our study. australasian journal of information systems adhikari, mathrani & scogings 2017, vol 21, selected papers from acis 2016 byod classrooms: access, capability, outcome divides 9 as illustrated in figure 2 above, the main predicators for the digital access divide are individual (like educational level), socio-economic (like affordability) and gender, which has potential to contribute to the digital divide in learning directly or indirectly through its effect to the digital capability divide. further to that, the digital capability divide is influenced by various factors during the learning process as the technology-mediated learning activities can occur in formal or informal learning spaces, since contemporary learning technologies offer learners the ability to extend their learning beyond the formal learning spaces and continue their learning in informal spaces (murphy, farley, lane, hafeez-baig, & carter, 2014). in this context, the formal learning space constitutes classrooms and informal space constitutes anywhere outside of the school/classroom. specifically, the factors determining the digital capability divide are digital/information literacy of learners in performing different types of computing tasks, nature of technology usage by learners (motivation to learn), and computer self-efficacy measurements of their capabilities. we can consider these factors as the focal constructs for this study through which personal, behavioural and environmental factors further influence the knowledge acquisition, skill development, and progress in attitude and behavioural aspects in learner, resulting in digital outcome divide. as shown in the framework in figure 2, the nature of digital divide may change from one form to another over the different stages of technology adoption. therefore, to examine digital outcome divide, various factors in first two levels has to be investigated to find their effect on the extent of knowledge acquisition, skills development and changes in attitudes, behaviours, and progression in learning. 4 case narrative and research questions in 2011, a new zealand school decided to integrate digital learning technologies into everyday teaching and learning activities in the form of one-to-one portable digital devices for all students in a cohort. the school informed all parents and students that they were expected to bring a one-to-one digital learning device (preferably an ipad2) into the classroom in year 9 (students aged 13-14) for the 2012 academic year. the most controversial and unique aspect, which makes this initiative different from most others, is that the parents were told they must cover the full cost of the required digital learning devices for their children, whereas similar projects in the past (like the digital opportunities projects) had provided devices and resources through the schools. the school’s decision resulted in a high profile news story in the new zealand herald, a national newspaper, triggered by a complaint from a parent about being asked to buy a digital device for their child. this set off a significant public and media response, leading to news stories on tv, radio, and online debates on various news sites and forums. drawing on from the initial evaluation of the literature review on the research topic and the preliminary analysis, the purpose of this research study is (a) to investigate whether and, if so, how, the introduction of byod initiative has changed digital divides and affected teaching and learning process, in both formal and informal learning spaces; (b) and, to evaluate the effectiveness of byod initiative on students’ learning outcomes. therefore, the main research question posed in this paper is: how have the digital divides in teaching and learning changed over the years of byod classrooms? the following subsidiary questions influence the above research question: 1. how has the digital access divide evolved because of the byod classrooms? 2. how have the digital capabilities divide evolved because of the byod classrooms? 3. how have the learning outcomes (knowledge acquisition, skills development, and progression of attitudes and motivation) evolved because of the byod classroom? australasian journal of information systems adhikari, mathrani & scogings 2017, vol 21, selected papers from acis 2016 byod classrooms: access, capability, outcome divides 10 5 socio-cultural ecological model to analyse the byod classrooms the focal constructs identified in the three level digital divide framework applied in the context of our study (in figure 2) has potential to disrupt learning activities not only on formal spaces like classrooms but also in informal spaces, such as home/out of school. therefore, the study will need an analytical lens that helps to capture learning process within formal and informal learning spaces mediated by one-to-one digital learning devices. our analysis framework (figure 3 below) adapted from socio-cultural ecological model by pachler, bachmair, et al. (2010) has been used to analyse the interrelationships between three components, namely structures, agency and cultural practices, within formal and informal learning spaces mediated by one-to-one icts. the idea behind this framework is that teaching and learning practices using one-to-one devices in and around different learning spaces is influenced by a triangular relationship between structures (imposed by curricula, communication, technology), agency (such as self and other users/actors) and cultural practices (or social interactions in everyday life) (pachler, bachmair, et al., 2010; pachler, cook, & bachmair, 2010). figure 3: the sociocultural framework used in our analysis (adapted from pachler et al, 2010) each of the three components of the socio-cultural framework actually relates to the byod case study we are investigating and helps to scaffold the interrelationships between key stakeholders, practices, and structures governing the teaching and learning. for that reason, the sociocultural ecological framework for mobile learning by pachler, bachmair, et al. (2010) have been adapted for this study, as this provides much needed analytical lens for the analysis of data. we have further extended the socio-cultural ecological model to explain the interrelationships in a technology-mediated teaching and learning environment. figure 3 shows the various points of references within each components and helps to scaffold the interrelationship between key stakeholders, practices and structures that relates to the byod case study under investigation. one of the components, the agency, is all about the ability of actions made by individuals within their worlds. it includes the capacity to construct one’s own learning responsibly and to build information literacy skills and self-efficacy around media use. in the context of our study, the agency is having adequate digital/information literacy skills for using digital tools, thinking critically to process the information and personalizing that information knowledgeably. in this manner, a habitus of learning emerges from individualized expertise which can then be applied to formal learning practice (pachler, bachmair, et al., 2010). cultural practices on the other side emphasize the areas that can benefit learning, as they relate to collaboration, social interaction, meaning making and media use. cultural practices in the context of byod is australasian journal of information systems adhikari, mathrani & scogings 2017, vol 21, selected papers from acis 2016 byod classrooms: access, capability, outcome divides 11 enabled by a digital communications medium and is transforming how individuals interact and use media in everyday life. episodes from everyday life can be captured and shared to create cultural artefacts. once captured and in tangible form, they provide a context and can then be used for further discussion, reflection and analysis. at the end, structures relate to immersion in digital learning technologies, prevalent school infrastructure to facilitate byod classrooms, and the curriculum within which they are applied. for example, the educational experience is structured around projects, assignments, and presentations where the media can be used to generate and express relevant content for assessment purposes. 6 research methodology a number of approaches can be used to investigate the integration of digital learning technologies and strategies in classrooms. some of the approaches include social constructivist perspective (khalid, jurisic, kristensen, & ørngreen, 2014) and experimental approach using preand posttests (f. martin & ertzberger, 2013). cheung and hew (2009) suggest research designs to include surveys, interviews or observations, especially when investigating technology-mediated teaching and learning, and similar initiatives. the case study method is appropriate when the objective is to learn about some environment in more detail (dubé & paré, 2003). a case study explores a phenomenon in its natural environment by using multiple data collection methods to gather information from variety of sources associated with the phenomenon under study (baxter & jack, 2008). the boundaries of the phenomenon are not apparent at the initial stages of the research, and no intervention or manipulation tactic is to be used (benbasat, goldstein, & mead, 1987; dubé & paré, 2003; yin, 2003). case studies are common research designs for exploratory (theory building), explanatory (theory testing) and descriptive (description of the context) in social science research (yin, 2003). in this study, a descriptive case study design has been used to present a longitudinal case of a secondary school implementing technology-mediated pedagogical practices in the form of the byod classrooms. the case we have selected for conducting the study is co-educational secondary school, which is state funded and is ranked at the decile 92 in the new zealand school ranking system based on the socio-economic status of the area surrounding the school. according to yin (2003), case study research design with a single case is suitable when the study represents a unique, revelatory or critical case. the case chosen is unique, revelatory and representative of the research problem and field of enquiry being investigated, since the said case is one of the earliest adopters of byod classroom or similar initiative in new zealand. this study follows an in-depth single case study research design with the continued investigation over a period of five years. 7 research instruments and timeline of data collection research instruments such as surveys, interviews, and observations have been used to collect data from students, teachers and parents. a low-risk notification has been obtained to conduct the study from the university human ethics committee well before starting the data collection. parents were informed of the research being conducted through school newsletter, and permission was sought for students below the age of 16 years. all of the research participants were given a written description of the purpose of the research study, along with the list of the university officials to be contacted if the participants feel the ethical consideration is not adhered. participant provided either consent through online consent form (for an online survey) or signing the paper consent form (for interviews) before researcher started recording data. data have been gathered using different methods at various rounds as outlined in the following table. 2 deciles are a measure of the socio-economic position of a school’s student community relative to other schools throughout the country. australasian journal of information systems adhikari, mathrani & scogings 2017, vol 21, selected papers from acis 2016 byod classrooms: access, capability, outcome divides 12 stages data collection timeline data collection methods stage 1: baseline early-2012 to mid-2012  open-ended surveys with students, teachers, and parents  classroom observations in targeted subject areas stage 2: follow-ups mid-2012 to mid-2014  semi-structured interviews with a randomly selected sample of students, teachers, and parents  two rounds of open-ended follow-up surveys with students, teachers, and parents  classroom observations in targeted subject areas stage 3: summative mid-2014 to late-2016  two rounds of semi-structured interviews with teachers table 2: stages of data collection during the first stage of the study, the answers to the first research question related to the digital access divide (i.e. how has the digital access divide evolved because of the byod classrooms?) have been investigated. initially, themes that emerged from the public debate surrounding the byod classroom initiative raised some concerns towards the potentially digitally divided classroom. in a bid to find answers to concerns raised, our initial investigation focused on affordability and availability issues of one-to-one digital learning devices among the students. results from this stage of investigation then informed the nature of subsequent research questions to be posed for the remainder of the study. at the stage 2, the research activities were focused mostly on finding the answer to the second research question about digital capability divide (i.e. how have the digital capabilities divide evolved because of the byod classrooms?). this stage of analysis briefly revisits the access and competencies divide issue discussed in the earlier stage, and touches on how digital capability might influence the next level of digital divides in learning (i.e. learning outcomes divide) before investigating that in detail at the third stage. the previous stage of study along with the review of the literature helped us to identify a number of factors that have potential to influence digital capability. we analysed data from students, teachers, and parents to learn how certain factors influenced the capability of an individual learner. at the stage 3, the research activities were focused mostly on finding the answer to the third research question about digital capability divide, that is, how have the learning outcomes (knowledge acquisition, skills development, and progression of attitudes and motivation) evolved because of the byod classroom? we combined findings from the first two stages of research to investigate how digital outcome divide evolved during the last 5-year implementation of byod classrooms initiative. during this stage of investigation, we conducted further semi-structured interviews with teachers who had participated over the whole byod initiative period. this helped to gain more insights on changes in learners’ attitudes and self-efficacy over the five-year long journey with the byod policy. 8 data analysis and findings from the study the data to analyse the research questions came pre-dominantly from the surveys and semistructured interviews conducted with the students, teachers, and parents over the past 5 years. qualitative data provided personal narratives involving classroom interactions with technology, thereby revealing rich insights during the different implementation phases of the byod policy. interviews and text response data from surveys have been coded and categorized australasian journal of information systems adhikari, mathrani & scogings 2017, vol 21, selected papers from acis 2016 byod classrooms: access, capability, outcome divides 13 under three research questions to be analysed. thematic analysis was then performed on each of the category resulted from the qualitative coding using nvivo, a qualitative data analysis software. this is evident from the findings of this study that, factors pertaining to one level of divide can also affect and contribute to the divide in the subsequent level. for example, limited or no affordability and ability to operate digital technologies along with the social and environmental conditions of homes and schools (digital access divide), further influences affordances in various sources of social cognitive abilities related to individual’s learning activities and computer self-efficacy levels, demonstrating digital capability divide among individuals (wei et al., 2011). these both levels of divide (digital access divide and digital capability divide) will, in turn, affect how new skills and knowledge are honed; having further implications on the individuals’ learning outcomes leading to digital outcome divide. this section discusses the study findings based on the three levels of digital divide framework in learning adapted from wei et al. (2011). the sociocultural ecological model (scem) for mobile learning by pachler, bachmair, et al. (2010), has been used as the analytical lens to answer the three subsidiary research questions. 8.1 how has the digital access divide evolved because of the byod classrooms? initial results coming out of the analysis based on the baseline data indicated no presence of digital access divide of any nature within the byod classrooms. despite survey data indicating 100% access, few students turned-up at school without any kind of one-to-one device. the school tried to resolve this issue of potential gap in device ownership by providing notebook computers to those who did not have any learning devices to use during school time only. however, these students could not take those devices to home. further enquiry of the issue at the subsequent stage of the investigation revealed more details about access issue among students in terms of the varying level of access to digital learning tools and technologies. while majority of the students had access to the one-to-one digital learning devices and internet at school and home to support their learning, others in the classroom had limited levels of access (including no internet access at home, poor internet connection at home, computers at home but no one-to-one devices to use in classrooms) thereby restricting their opportunities. therefore, despite the school’s efforts to bridge the gap of access by facilitating learning devices during school hours, a significant number of students were missing the opportunities offered by the full access to digital learning tools and technologies. this finding was in contrast with the fact that the school area is ranked decile nine (representing a region where citizens belong to a relatively higher socioeconomic status). one student stated in the survey: “i usually do not spend much time with the tablet at home because i don't have the internet at home. sometimes i can't complete my work at home because of the internet”. similarly, another student too had responded: “well in my house we don't have dial-up so i only use my tablet for the project i have downloaded. i don't have the internet at home.” teachers also confirm the issue of internet access at home: “something that is a problem is internet access at home or broadband access. because children are saying honestly that, ‘i cannot do this at home’. i have one bright student in my class who said she doesn't have the internet in her home or the device either”. many of the parents also expressed their concerns about not being able to make the recommended device available to the students. some of them said: australasian journal of information systems adhikari, mathrani & scogings 2017, vol 21, selected papers from acis 2016 byod classrooms: access, capability, outcome divides 14 “one-to-one devices are great for education but there needs to be equity for families that cannot afford devices”. another issue that came up related to digital access was the compatibility issues between different types of one-to-one devices within the classroom learning. interview responses from student indicated that some students have been unable to carry out their usual learning activities during classroom because of device compatibility issues. responses from teachers confirmed that most of the learning activities are designed keeping ipads in mind, and these activities may be difficult to perform using a laptop and android devices. regarding compatibility issue, one student says: “i felt disadvantaged sometimes because i have a laptop and all the teachers talk about is apps for ipads”. however, students and teachers were prompt in finding alternative ways for continuing participation in learning activities for those affected in these situations. also, the overall survey responses do not reflect incompatibility issues to be prevalent on a larger scale in everyday learning, making this a relatively non-significant issue for the further investigation. despite some degree of access, compatibility and technology issues being discovered, byod initiative had certainly provided a greater degree of access to digital learning technologies to learners. in addition, the access to the internet at homes has improved in the subsequent years, which contributed to narrowing down of the gap in access to digital technologies and resources for students, marking a gradual improvement. looking into the positive change in terms of access/reach, byod initiative can be considered therefore an enabler in this context. 8.2 how have the digital capabilities divide evolved because of the byod classrooms? the baseline and follow-up survey data between 2012 and mid-2014 helped in laying the foundation of the study and framing aspects of digital divide issues as they evolved in the byod classrooms. as the pedagogical practices, including teaching methodologies and learning activities progressed, so did, the different nature of digital divides in learning which started to appear in the second set of the survey responses. the teaching and learning practices during this period seemed to concentrate more on critical thinking and analysis of available information rather than consuming as it is. many students found byod classroom much more relevant and useful because of those changes. because of this shift in teaching and learning practices/activities from consumption to analysing critically before applying it into their learning, the area of investigation shifted from just digital skills to much broader area of skills (i.e. information literacy and computer selfefficacy). survey and interview data from this phase suggest that a significant proportion of students lack critical analysis of information to apply to their learning activities. information literacy and computer-self efficacy skills are critical to lift the student’s ability around the meaningful usage of technology and critical thinking and analysis, which affects the acquisition of knowledge in students. therefore, it is necessary that information literacy and computer self-efficacy are considered as the key factor into the investigation of the digital capability divide and also the evolving nature of digital divides in learning, since that can either help or risk the learning outcomes, determining the digital outcomes divide. the investigation into this phase also looked into the motivational and behavioural aspects related to the students, potentially affecting their digital capability. when asked what have been their major challenges, 17% of teachers indicated classroom management in the byod context. this was mainly because of the extra effort teachers had to put in their classroom to prevent students going off task. a number of measures have been taken to caution students, but this had little or no effect. therefore, the issue remains one of the challenges for the teachers. parents in their responses, also clearly voiced their concerns regarding the unsupervised usage of devices by their child. a number of parents worried about the australasian journal of information systems adhikari, mathrani & scogings 2017, vol 21, selected papers from acis 2016 byod classrooms: access, capability, outcome divides 15 unsupervised nature of their child’s device usage and the potential harm it may cause. one parent worried for change in their children’s behaviour and social interactions say: “yes i constantly have to take the device off my child she seems to be constantly on it and it is a constant battle, she has lost interest in a lot of other activities”. the unsupervised access of device and the internet by students could also be a concern because of the widening digital skills gap between the students and parents. while students are exposed to the different forms of digital technologies as an integral part of their learning, their parents severely lack the ability to monitor their digital activities. in a different study, newhouse, lane, cooper, and twining (2014) reported similar findings in a learning environment using web 2.0 technologies. because of this, parents have a growing concern about the safety of their children from online content. in the latest surveys, some of the parents responded as: “negative impact: they spend a huge amount of time at home on their devices. it is often very difficult for us to know whether it is school related or not. as it is a condition of them attending school we are bound to allow them access to their devices.” “yes, the definite negative impact i have seen in our community and at home. a huge amount of social bullying and inappropriate use of the device to take photos, and send images, messages to others about others etc. children as young as year 7 and 8 being given complete access to the internet and everything on it getting into pornography (written and visual) and chat rooms talking to older men and women.” lately, there have been reports of students using some of the applications and sites that are used for internet bullying in new zealand schools. there is no report of that from the school where this research is based on, but this is clearly an alarm bell for school and parents involved in the byod classrooms. therefore, there is a mixed result from the analysis in this context. therefore, while the results show improvement in the digital/information literacy, they also raise concerns caused by the digital practices/habits of the students. on the other hand, initial skill levels of staff reported indicated towards potentially lower digital capability, which was improved in the later stage of the study. however, it should be noted that the staff were early adopters in the first year of the byod initiative who volunteered to take part into the byod classrooms initiative. this suggests that we cannot expect the digital skills of staff overall to reach its maximum potential until the byod policy has been fully rolled out across all school years so that all the staff has had the opportunity to fully develop their digital skills. nonetheless, the school has also helped in that period to setup workshops to foster peer-support into adopting and integrating one-to-one digital learning device into the classroom pedagogical practices through development of their own digital capability. this was vital for the success of the byod classroom initiative, as this method of professional development through the community of practicing teachers in the context of technologymediated teaching and learning environment is more sustainable (watson & prestridge, 2003), and teachers expressed appreciation of the ongoing peer-support they received from the community of learning. overall, some vicissitudes in the digital/information literacy skills and behavioural and motivational concerns have been observed in the byod classroom initiative. despite that, the digital capability is continuing to improve over the five-year period of the investigation. yet, the analysis of some focal constructs like computer self-efficacy is cannot be concluded at this level and has to be analysed also in the context of the learning outcomes next. 8.3 how have the learning outcomes (knowledge acquisition, skills development, and progression of attitudes and motivation) evolved because of the byod classroom? certainly, addressing and bridging of access and digital/information literacy skills are important aspects in the context of the byod classrooms. at the same time, having reasonable access and adequate skills will not guarantee overall digital inclusion. even with the innovative technologies and best digital/information literacy skills, students may still struggle in their australasian journal of information systems adhikari, mathrani & scogings 2017, vol 21, selected papers from acis 2016 byod classrooms: access, capability, outcome divides 16 learning activities. therefore, our analysis extended its focus on some of the factors contributing to the learning outcomes divide. one of the factors that closely affect the learning outcomes is knowledge acquisition. in the recent years since the introduction of byod classrooms, a shift in the teaching and learning practices and activities has been seen. the learning activities are now designed in such a way that it requires students to gather information, synthesize it and analyse critically before applying into their learning. many students have appreciated the way they are currently learning. whereas earlier a small proportion of students expressed concerns regarding not being able to meet that standard. however, recent interviews with teachers indicated that there is huge progress among students in critical thinking abilities, such as gathering, analysing and applying information into their learning. “the results that we are getting since having byod are improved. i definitely would be very surprised if results go down.” another factor that affects the learning outcomes is the attitude, behaviour, and motivation. apart from few exceptions, majority of learners find byod classrooms a great idea. interestingly, despite the potential opportunities offered by the byod classroom, some students earlier expressed some apprehensions with the initiative. however, the latest data from the teacher interviews shows a very different picture on the issue indicating positive changes in attitude, behaviour and motivation. “there far more engagement and there is far more openness with their work. i remember years ago students sort of covering their work and not showing it to teachers when they were writing, if their handwriting was bad or spelling was bad. whereas now the kids are lining up to show their work, whether it's the video, report or presentation.” learner self-efficacy is another factor that can affect the learning outcomes. it has its root in the three level digital divide framework (figure 2), adapted for our research. the framework describes self-efficacy as one of the major predicators of the digital capability divide, which in turn leads to digital outcome divide in combination with some other factors. recently, a number of teaching and learning strategies implemented have been designed to promote selflearning and maximize student engagement. all of that have been done with emphasis on student led learning which does not just maximize student engagement, but also gives them more freedom, responsibility and ownership of their learning. the following comments from different teachers show the change in students’ attitude in the new environment. “on a good day, you come here and you will see students outside. they are still on task but they are choosing different learning environment for them.” “very first thing is, let the students be and trust that they are going to take ownership of their learning.” as a result, the changes that are appearing now indicate that students are developing into independent learners. one comment states it simply: “for majority of students, they have improved on critical thinking ability.” at the same time, concerns about every student not being on the same page and not having same level of information literacy skills were also raised. “some students are lot more confident in finding, processing and applying information they come across and others don’t. i think that’s what separates your achieved students with excellent ones, because quite often your excellent students have higher level of information literacy.” interestingly there were no conflicting views about improvement in student learning outcomes over the years of byod policy. there was a strong feeling among teachers that byod has definitely contributed in improving learning outcomes. australasian journal of information systems adhikari, mathrani & scogings 2017, vol 21, selected papers from acis 2016 byod classrooms: access, capability, outcome divides 17 “the results that we are getting since having byod are improved. i definitely would be very surprised if results go down.” “i think the critical thinking ability in majority of students have improved because i find their essays lot more detailed and with in-depth information. they have lot more insightful comments.” overall, the latest round of data has shown improvements on many aspects, including school curricular practices, giving students’ agency and ability of meaning making all of which contribute to build students self-efficacy in the context of byod policy. this in turn has a positive effect in achieving better learning outcomes among students. 9 contributions this study reports on a longitudinal investigation of a technology-mediated teaching and learning initiative, in the form of byod classrooms in a new zealand secondary school. the study draws on findings from start of the byod initiative onto 5 years of its implementation. using a narrative approach, the study has described how the teaching methodologies and learning activities have changed over the years, and how the existing and new digital divides in learning evolved within those changes and with increased penetration and adoption of digital learning technologies across formal and informal learning spaces. the data for the study comes from a variety of stakeholders into the byod classrooms initiative (including teachers, students and parents). the longitudinal inquiry at each stage of study was guided by data collected from previous stage, as the issues that evolved at previous stage became part of the subsequent inquiry. the study reported these issues as they came up, discussed their resolutions and informed on new issues, which came at the forefront over the 5-year period. this research study has utilized two main frameworks to analyse transformations within the technology-mediated teaching and learning (byod classrooms) environments. the theoretical framework developed for the study (figure 2) has been adapted from the three level digital divide by wei et al. (2011) which helped the study to capture the continuously evolving nature of the digital divides with the advances in the technology adoption into teaching methodologies and learning activities. also, the study uses analytical lenses of sociocultural ecological framework for mobile learning (figure 3) by pachler, bachmair, et al. (2010) to allow the analysis to capture the interrelationships between the various stakeholders, the learning environments and curricular structure of the school. the key constructs identified for the study (which includes digital/information literacy, computer self-efficacy, and nature of technology usage) from the three level digital divide framework for learning ( i.e. theoretical framework developed for study) have been cross-referenced with the three components of the sociocultural framework for mobile learning (agency, cultural practices and structures). the combination of these two frameworks in analysis helped to capture the learning activities situated across the formal and informal learning spaces and investigate various sources of social cognitive abilities related to individual’s information literacy, learning activities and computer self-efficacy levels and its effect on the learning outcomes of the students. study posed a series of research questions in response to the research problem identified for the investigation. the main overarching question was aimed at the overall evolution of the digital divide in learning in the context of the technology-mediated pedagogical environments. to answer the overarching question three sub-questions aimed at the three levels – digital access divide, digital capability divide, and digital outcome divide – were posed. therefore, the study outcomes are mapped under these three broad digital divide levels. these outcomes from the study are explained below. 9.1 outcomes related to digital access divide despite the speculation and anticipation during the initial phase of the byod classrooms initiative, the findings suggested no evidence of a widening gap in terms of the digital access divide within the byod classrooms. to be precise, there were some reports of limited or no access to internet at homes, which eventually improved at the later stage of the study. the australasian journal of information systems adhikari, mathrani & scogings 2017, vol 21, selected papers from acis 2016 byod classrooms: access, capability, outcome divides 18 byod classroom also experienced some issues related to the compatibility of one-to-one devices in terms of participating into the classroom learning activities, as the majority of the resources and activities were found to be developed keeping recommended devices (i.e. ipad) in mind. this led to increased efforts by teachers to bridge the capability of the various devices. an earlier study had cautioned that technology integration initiatives in schools might accentuate existing digital divides (rivers & rivers, 2004). this study finds that such an initiative (i.e. the byod classroom initiative) did not trigger or widen the existing gaps/level of gaps in access to digital divides. it is specifically clear that the fear of having digitally divided school environment based on the previous notions about the integration of technologies proved to be unfounded. 9.2 outcomes related to digital capabilities divide the second stage of study started on a relatively firm base, after finding not much evidence of digital access divide within the byod classrooms, where various aspects of the digital capability divide were investigated using online surveys and semi-structured interviews. in addition, the continuing classroom observations data was also used to inform the analysis at this stage. a major finding from the analysis at this stage indicates that digital skills in learner do not support their ability of learning as understood earlier. students may have an excellent level of digital skills in operating the digital learning devices, but they may not be able to take advantage of that skill without the proper information literacy skills, suggesting ability of learning in learners is guided largely by the information literacy skills in the technologymediated teaching and learning situations. the byod classrooms initiative enabled teachers to be more innovative in classroom learning activities, for better student engagement. students were offered greater flexibility and agency making them more responsible for their own learning. at this time, the difference between digital skill and information literacy skills become evident. the analysis found an increase in frustration and motivational issues among the students with no or limited information literacy and critical thinking ability. because of this, the gap in ability to learn between students with and without the information literacy and critical thinking skills emerged. the analysis strongly indicated the presence of this gap mid-way during the five-year long research. however, this gap become narrow at the later stage of the analysis, which suggests a possible explanation that it took some time for students to accept the teaching and learning practices in the byod classroom and to develop the vital information literacy skills, with the huge change in the classroom curricular structure. 9.3 outcomes related to digital outcomes divide the findings of the second stage of the research provided the informed beginning of the third stage enquiry as more understanding was gained around factors pertaining to the digital capability divide. this was important, as the factors pertaining to the digital capability divides are the focal constructs for this study (digital/information literacy, computer self-efficacy, and nature of technology usage), through which personal, behavioural and environmental factors further influence learning outcomes resulting in digital outcome divide. from the analysis of these focal constructs, computer self-efficacy emerged as the factor with greater influence in shaping up the learning outcomes. the learner self-efficacy when combined with the other predicators of the digital capability divide, further influenced affordances in various sources of social cognitive abilities affecting individual learner’s knowledge acquisition, progression, and changes in their attitude and behaviours. this leads to difficulty in learners to engage and apply technology to attain their learning requirements, and therefore puts them to the wrong side of digital outcome divide. 10 limitations and future research as a part of the five-year study of the technology-mediated teaching and learning initiative, we have been able to explain some of the unanswered questions around the issue of digital divide australasian journal of information systems adhikari, mathrani & scogings 2017, vol 21, selected papers from acis 2016 byod classrooms: access, capability, outcome divides 19 in the learning process. rich insights have been shared in the context of technology-mediated pedagogies and specifically byod classroom, as to how digital divides moved beyond access and skills to ensure inclusive learning outcomes. however, the study also has some limitations as explained next. firstly, the choice of the case study was restricted by the availability of the suitable cases representative of the research problem at the start of the study. the case study selected for this research is one of the first schools to introduce the fully technology-mediated pedagogical practices in the form of the byod classrooms initiative. that has limited our options in terms of the selection of the case for the study. that led the study to use the single case of a byod classrooms initiative in a secondary school in new zealand with the decile ranking of nine based on the new zealand school ranking system. while this study is limited to a single case, it takes account of a five-year longitudinal period to reveal a comprehensive understanding about technology-mediated transformations in teaching and learning spaces. however, a comparative study involving more than one case study including schools with varying level of decile ranking relevant to the research problem would give a better understanding of digital divide in learning across various socio-economic regions. in a recent development in the education policy, the government reform to scrap the decile ranking of schools by 2019 is currently under discussion (parliament, 2017). the discussion is around the argument that decile ranking is a blunt tool, which measures schools with socio-economic region rather than quality of education. secondly, because of being the pioneer school to introduce such initiative in new zealand there was a huge media controversy surrounding the school’s decision and was covered in many of the national and local media (mass, 2011; moore & tasman-jones, 2011; tasman-jones, 2012). the media spotlight and speculation surrounding the initiative could have had some impact on steps taken by staff to better manage digital exclusion. however, triangulation of data sources (i.e., classroom observation, student surveys and parent and teacher interviews) has been used in data gathering process to strengthen the study findings. this study provides a very good understanding on the issue of digital divides in learning and captures the evolution of the existing and new digital divides in the context of the technologymediated pedagogical interventions and initiatives. while doing so, the study predominantly collects and analyses the qualitative data from the longitudinal case of a byod initiative, involving various stakeholders like students, teachers and parents. however, analysis of student learning outcomes based on quantitative measures like student performance data have never been included into the study. therefore, the future directions of research on this topic lie in the inclusion of the quantitative analysis of the student performance data captured through the school’s assessment process. further research can use the three level digital divide framework in learning (theoretical framework developed for this study in figure 2) as a base and combine it with different analytical lens, which are suitable for quantitative data analysis. this will not just help to validate the three level digital divide framework in learning from the quantitative prospective, but also provide an extension to this research study and contribute to teaching and learning theories. references anderson, n. (2009). equity and information communication technology (ict) in education (vol. 6): peter lang publishing inc., new york. baxter, p., & jack, s. (2008). qualitative case study methodology: study design and implementation for novice researchers. qualitative report, 13(4), 544-559. benbasat, i., goldstein, d. k., & mead, m. (1987). the case research strategy in studies of information systems. mis quarterly, 11(3), 369-386. brake, d. r. (2014). are we all online content creators now? web 2.0 and digital divides. journal of computer-mediated communication, 19(3), 591-609. doi: 10.1111/jcc4.12042 australasian journal of information systems adhikari, mathrani & scogings 2017, vol 21, selected papers from acis 2016 byod classrooms: access, capability, outcome divides 20 brandtzæg, p. b., heim, j., & karahasanović, a. (2011). understanding the new digital divide a typology of internet users in europe. international journal of human computer studies, 69(3), 123-138. brosnan, m. j. (1998). the impact of computer anxiety and self-efficacy upon performance. journal of computer assisted learning, 14(3), 223. bru, e. (2006). factors associated with disruptive behaviour in the classroom. scandinavian journal of educational research, 50(1), 23-43. cheung, w. s., & hew, k. f. (2009). a review of research methodologies used in studies on mobile handheld devices in k-12 and higher education settings. australasian journal of educational technology and society, 25(2), 153-183. cole, j. i. (2001) surveying the digital future year two: the ucla internet report 2001 los angeles, ca: ucla center for communication policy. cook, j., pachler, n., & bachmair, b. (2011). ubiquitous mobility with mobile phones: a cultural ecology for mobile learning. e-learning and digital media, 8(3), 181-196. cullen, r. (2001). addressing the digital divide. online information review, 25(5), 311-320. demiraslan, y., & usluel, y. k. (2008). ict integration processes in turkish schools: using activity theory to study issues and contradictions. australasian journal of educational technology, 24(4), 458-474. dewan, s., ganley, d., & kraemer, k. l. (2005). across the digital divide: a cross-country multi-technology analysis of the determinants of it penetration. journal of the association for information systems, 6(12), 409-431. dfes. (2003). 21st century skills : realising our potential : individuals, employers, nation. retrieved from http://dera.ioe.ac.uk/4747/. donohue, g. a., tichenor, p. j., & olien, c. n. (1975). mass media and the knowledge gap. communication research, 2(1), 3-23. doi: doi:10.1177/009365027500200101 dubé, l., & paré, g. (2003). rigor in information systems positivist case research: current practices, trends, and recommendations. mis quarterly, 27(4), 597-636. ersanlı, c. y. (2015). the relationship between students’ academic self-efficacy and language learning motivation: a study of 8th graders. procedia social and behavioral sciences, 199, 472-478. doi: http://dx.doi.org/10.1016/j.sbspro.2015.07.534 garipagaoglu, b. c. (2013). the effect of self-efficacy on the lifelong learning tendencies of computer education and instructional technologies students: a case study. 2013, 10(1), 13. gaziano, c. (2010). notes on "revisiting the knowledge gap hypothesis: a meta-analysis of thirty-five years of research.". journalism & mass communication quarterly, 87(3/4), 615-632. ghobadi, s., & ghobadi, z. (2015). how access gaps interact and shape digital divide: a cognitive investigation. behaviour & information technology, 34.4, 330-340. doi: 10.1080/0144929x.2013.833650 gunkel, d. j. (2003). second thoughts: toward a critique of the digital divide. new media & society, 5(4), 499-522. hargittai, e. (2002a). beyond logs and surveys: in-depth measures of people's web use skills. journal of the american society for information science & technology, 53(14), 1239-1244. australasian journal of information systems adhikari, mathrani & scogings 2017, vol 21, selected papers from acis 2016 byod classrooms: access, capability, outcome divides 21 hargittai, e. (2002b). second-level digital divide: differences in people's online skills. first monday, 7(4). johnson, l., levine, a., smith, r., & stone, s. (2010). the 2010 horizon report. austin, texas: the new media consortium. jones, a., & issroff, k. (2007). motivation and mobile devices: exploring the role of appropriation and coping strategies. research in learning technology, 15(3), 247-258. khalid, m. s., jurisic, o., kristensen, h. s., & ørngreen, r. (2014). exploring the use of ipads in danish schools. paper presented at the european conference on e-learning. lenhart, a., horrigan, j., rainie, l., allen, k., boyce, a., madden, m., & o'grady, e. (2003) pew internet and american life project report. washington, dc: pew internet and american life project. mardis, m., & everhart, n. (2013). from paper to pixel: the promise and challenges of digital textbooks for k-12 schools. in m. orey, s. a. jones & r. m. branch (eds.), educational media and technology yearbook (vol. 37, pp. pp 93-118). new york: springer new york. martin, a., & rader, h. b. (2002). information and it literacy : enabling learning in the 21st century / edited by allan martin and hannelore rader: london : facet, 2002. martin, f., & ertzberger, j. (2013). here and now mobile learning: an experimental study on the use of mobile technology. comput. educ., 68, 76-85. doi: 10.1016/j.compedu.2013.04.021 mass, a. (2011, 27/10/2011). orewa college ipad plans move ahead, stuff. retrieved from http://www.stuff.co.nz/national/education/5859716/orewa-college-ipad-plans-moveahead ministry of education. (2006). ict strategic framework for education.: the ministry of education, on behalf of the education sector agencies and the national library of new zealand. moore, c., & tasman-jones, j. (2011, 19/07/2011). school's ipad requirement 'divisive', stuff. retrieved from http://www.stuff.co.nz/national/education/5304084/schools-ipadrequirement-divisive murphy, a., farley, h., lane, m., hafeez-baig, a., & carter, b. (2014). mobile learning anytime, anywhere: what are our students doing? australasian journal of information systems, 18(3). doi: 10.3127/ajis.v18i3.1098 newhouse, p., lane, j., cooper, m., & twining, p. (2014). redefining education: sustaining 1 to 1 computing strategies in western australian schools. paper presented at the australian computers in education conference (acec). pachler, n., bachmair, b., cook, j., & kress, g. r. (2010). mobile learning : structures, agency, practices: new york : springer. pachler, n., cook, j., & bachmair, b. (2010). appropriation of mobile cultural resources for learning. international journal of mobile and blended learning, 2(1), 1-21. parker, b. (2001). māori access to information technology / [prepared by brett parker, te puni kōkiri]: wellington, n.z. : te puni kōkiri/ ministry of māori development, 2001. parliament, n. z. (2017). oral questions — questions to ministers. new zealand parliament. parr, m., & ward, l. (2005). evaluation of the digital oppertunities project farnet: learning communities in the far north. wellington: ministry of education, new zealand. parsons, d. p. (2016). the social surplus of broadband initiatives in compulsory education. australasian journal of information systems, 20. doi: 10.3127/ajis.v20i0.1188 australasian journal of information systems adhikari, mathrani & scogings 2017, vol 21, selected papers from acis 2016 byod classrooms: access, capability, outcome divides 22 partridge, h. (2003). moving beyond the digital divide: developing a psychological perspective of digital inequality. in m. khosrowpour (ed.), information technology and organizations: trends, issues, challenges and solutions (vol. 1, pp. 508-511): idea group inc (igi). parycek, p., sachs, m., & schossböck, j. (2011). digital divide among youth: socio‐cultural factors and implications. interactive technology and smart education, 8(3), 161-171. doi: doi:10.1108/17415651111165393 prestridge, s. (2007). engaging with the transforming possibilities of ici: a discussion paper. australian educational computing, 22(2), 3-9. rivers, j., & rivers, l. (2004). a summary of the key findings of the evaluations of the digital opportunities pilot projects (2001 2003). wellington: ministry of education, new zealand. sterling, b. (1997). the digital revolution in retrospect. communications of the acm, 40(2), 79-79. doi: 10.1145/253671.253715 tasman-jones, j. (2012, 15/05/2012). school ipad revolution may go nationwide, auckland now. retrieved from http://www.stuff.co.nz/auckland/local-news/6919789/schoolipad-revolution-may-go-nationwide van deursen, a., & van dijk, j. (2014). the digital divide shifts to differences in usage. new media & society, 16(3), 507-526. doi: doi:10.1177/1461444813487959 van dijk, j. (2005). the deepening divide : inequality in the information society: thousand oaks. van dijk, j. (2006). digital divide research, achievements and shortcomings. poetics, 34(4-5), 221-235. van dijk, j., & hacker, k. (2003). the digital divide as a complex and dynamic phenomenon. information society, 19(4), 315. vogel-walcutt, j. j., gebrim, j. b., bowers, c., carper, t. m., & nicholson, d. (2011). cognitive load theory vs. constructivist approaches: which best leads to efficient, deep learning? journal of computer assisted learning, 27(2), 133-145. warschauer, m. (2003). technology and social inclusion : rethinking the digital divide.: cambridge, mass. : mit press, 2003. watson, g., & prestridge, s. (2003). a networked learning community approach to sustain teacher ict professional development. australasian journal of educational technology, 19(2). wei, k. k., teo, h. h., chan, h. c., & tan, b. c. y. (2011). conceptualizing and testing a social cognitive model of the digital divide. information systems research, 22(1), 170-187. winter, m. (2004). digital opportunities pilot project (2001-2003) evaluation of generation xp. wellington: ministry of education, new zealand. wolfswinkel, j. f., furtmueller, e., & wilderom, c. p. m. (2013). using grounded theory as a method for rigorously reviewing literature. european journal of information systems, 22(1), 45-55. doi: 10.1057/ejis.2011.51 yin, r. k. (2003). case study research: design and methods (3rd ed.). newbury park: sage publications. yoori, h., & se-hoon, j. (2009). revising the knowledge gap hypothesis: a meta-analysis of thirty-five years of research. journalism & mass communication quarterly, 86(3), 513-532. australasian journal of information systems adhikari, mathrani & scogings 2017, vol 21, selected papers from acis 2016 byod classrooms: access, capability, outcome divides 23 yusuf, m. (2011). the impact of self-efficacy, achievement motivation, and self-regulated learning strategies on students’ academic achievement. procedia social and behavioral sciences, 15, 2623-2626. doi: http://dx.doi.org/10.1016/j.sbspro.2011.04.158 zhong, z. j. (2011). from access to usage: the divide of self-reported digital skills among adolescents. computers and education, 56(3), 736-746. copyright: © 2017 adhikari, mathrani & scogings. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems aleti, harrigan, cheong & turner 2016, vol 20, research article australian brewing industry on twitter 1 an investigation of how the australian brewing industry influence consumers on twitter torgeir aleti monash university torgeir.aleti@monash.edu paul harrigan university of western australia marc cheong monash university will turner independent researcher abstract in this paper we develop and test hypotheses around organisations’ behaviour on social media and its effect on consumers’ responses. we draw on the notion of the market maven to underpin the research and suggest that organisations on social media need to focus on acting in a maven-like manner in order to influence audiences in twitter. we collected data from the twitter accounts of the entire brewing industry in australia, analysing organisational postings and their impact on influence (follower numbers, retweets) of their respective twitter accounts. in particular, we look at message formulation and language, native platform behaviour, reciprocity and persistency variables. findings suggest that establishing a larger follower base requires an interactive, one-to-one and reciprocal approach. in order to influence audiences to retweet organisations need to speak the ‘native platform language’ and employ a soft-sell strategy. maven-like behaviour tends to reside in the small independent craft breweries. we offer the conclusion that these craft breweries have realised that, on social media, a different approach to marketing is required: the organisations must act in a mavenlike manner. keywords: mavenism; big data; social media; twitter; influence; engagement 1 introduction social media is said to be fundamentally changing the way we communicate, collaborate, consume and create content (aral, dellarocas, & godes, 2013; as-saber et al., 2014; malsbender, hoffmann, & becker, 2014). because of its dynamic nature and the potential of on-going consumer-to-consumer communication, it is vitally important for businesses to understand how to use it in order to reach and influence customers (augar & zeleznikow, 2014; mangold & faulds, 2009). the dynamic and on-going conversational nature of social media lends itself to a ‘pull’ rather than the traditional ‘push’ strategy in terms of attracting an audience (hodis, sriramachandramurthy, & sashittal, 2015). that is, consumers need to be drawn towards something of interest to them, rather than being the passive recipients of expressly promotional messages (fournier & avery, 2011). digital audiences can easily avoid advertising by using ad blockers that remove native ads from feeds on any social media platform. further, they can easily stop following an annoying or irrelevant brand. the key is to connect with the audience in such a way as to ‘softly sell’ a commercial message that is perceived as both non-intrusive and of interest to receivers. essentially, mastering social media relates to understanding how messages spread. traditionally, the view has been that a minority of members in a society hold certain abilities that make them exceptionally persuasive in spreading ideas to others. the assumption is that these exceptional individuals are highly informed, respected and well-connected and, as such, are needed in order for new trends to reach ordinary people (cha, haddadi, benevenuto, & australasian journal of information systems aleti, harrigan, cheong & turner 2016, vol 20, research article australian brewing industry on twitter 2 gummadi, 2010). in the two-step flow theory, these individuals are referred to as ‘opinion leaders’ (katz & lazarsfeld, 1955), in the diffusion of innovations theory they are called ‘innovators’ (rogers, 1983), while gladwell (2000) refers to then as ‘salesmen, connectors and mavens’. with the arrival of social media, this traditional view may be turned upside down. ordinary users can be as influential as these exceptional minorities. in essence, andy warhol’s ‘15 minutes of fame’ (violet, 1988) has been realised through online social channels. social media represents a virtual reality in which any user may at any given point in time perform the role of the ‘exceptional individuals’ referred to in the traditional view of how messages spread. from a marketing perspective, the traditional framework also poses a problem as to how to find these ‘mystical characters’ and further convince them to support your cause. instead of searching for special people with wholly abstract personality traits, we seek to investigate observable behaviours which organisations can use in their communication. 2 literature review gladwell’s (2000) framework, is typical in terms of the traditional view of how messages spread. referred to as ‘the law of the few’ it reflects the ‘pareto principle’ (sanders, 1987), which suggest that about 80% of effects come from 20% of causes (e.g. 80% of a business revenue comes from 20% of its customers). one implication of the law of the few is that small businesses necessarily need the help of powerful, influential individuals to spread their messages. however, this ‘law’ does not explain how small businesses can spread messages without the help of powerful ‘friends’. as we will show, this influence may be achieved more directly through social media channels. in gladwell’s framework, ‘connectors’ are individuals who have ties to many different realms and act as conduits between them, helping to engender connections, relationships, and ‘crossfertilisation’ that otherwise might never occur. on social media, such ‘cross-fertilisation’ can happen frequently when messages are forwarded. for example, when consumers re-post a message from a business to their friends, those friends re-post the message to their friends and so on (dang-pham, pittayachawan, & nkhoma, 2015). as such, special people are not a necessity in order to influence a large number of consumers. indeed, ordinary social media users can also be influential (cha et al., 2010). further, ‘salespeople’ are said to have an unusual charisma that allows them to be extremely persuasive in inducing others’ buying decisions and behaviours (gladwell, 2000). to be ‘extremely persuasive’ seems dangerous on social media where consumers typically resist advertising (fournier & avery, 2011) and instead tend to be looking for information (ho & dempsey, 2010) or entertainment (myrick, 2015). given this backdrop the concept of a market maven offers a more useful focus for attention. however the maven concept itself has undergone considerable reanalysis. market mavenism is a notion first developed by feick & price (1987). in the original conception market mavens are people with marketplace knowledge who have a strong compulsion to help other consumers by enabling them to make informed decisions (price, feick, & guskeyfederouch, 1988). they are described as special people with a particular personality trait, and in this respect the concept is closely related to opinion leaders. more recently, goodey & east (2008) have criticised the maven concept, finding that there are no unified personality traits for mavens, and further that they do not provide more consumer advice than ‘non-mavens’. such critique may explain why the maven concept has been extended from its original conceptualisation, a process that has been accelerated by developing communication technologies. the internet, and in particular the exponential growth of social media and user-generated content, have provided an ideal environment for a new type of maven to flourish: the e-maven (belch, krentler, & willis-flurry, 2005). in mainstream and niche product and service categories, e-mavens are using social media to build networks of followers, to share and comment on marketing messages, and to act as industry experts guiding consumers (ho & dempsey, 2010). australasian journal of information systems aleti, harrigan, cheong & turner 2016, vol 20, research article australian brewing industry on twitter 3 ho and dempsey (2010) define e-mavens as those who frequently forward online content. a recent study of mavens’ online referral behaviour found they were four times more likely to refer products or services than other consumers (walsh et al. 2012). such argumentation lends itself the notion of domain specific mavens. that is, feick and price’s (1987) market maven spreads market knowledge in general, while ho and dempsey’s (2010) e-maven is restricted to spreading content online. changing demographic and market conditions and their associated fragmentation is not a new idea (zeithaml, 1985). the internet’s tendency to split the market into consumer tribes is a key driver in such fragmentation (hosanagar, fleder, lee, & buja, 2014). this opens up the opportunity for ‘specialised e-mavens’ on social media in niche markets. if there are e-mavens, there may also be mavens within particular social networking sites as well as within particular product categories. mavenism could be said to have fragmented in the same way that markets have fragmented. however, the traits of maven-like behaviour are still relevant when referring to a person with a passion for spreading information, helping others and continuously updating their own knowledge on their area of interest. key aspects of market mavens are high sociability (feick et al. 1987) and strong social influence (clark et al. 2005). social media has created a perfect habitat for market mavens, as they can use it to easily gather and disseminate market information and become hubs of expertise. several studies have shown how mavens affect others through social media (cui et al. 2012; gnambs et al. 2011; goldenberg et al. 2009; stephen et al. 2010). for example, key bloggers can make or break brands (chau et al. 2012; luetjens et al. 2007). however, in the disruptive environment that is social media, the boundaries between organisation and consumer become blurred. we posit that modern business models require organisations to become maven-like to influence and provide value to consumers. rather than ‘hunting for the mavens’ we investigate how businesses can act as mavens themselves in order to produce messages of interest to consumers and hence to generate value to themselves as producers. in particular, the aim of this paper is to investigate maven-like behaviour by modelling big data collected from twitter. 3 conceptual development one of the most popular platforms for brands and individuals who wish to stand out and be heard is the microblogging site twitter. twitter is used by many for sharing news items, gossip and opinions relevant to events of the day (langley, hoeve, ortt, pals, & van der vecht, 2014). its messages or tweets can contain text, images, videos, links and #hashtags, all within a limit of 140 characters. by default, tweets are publicly available, making twitter a particularly rich source of big data. studies show that those who access twitter for informational purposes do so for its utilitarian value and cognitive stimulation whereas information needs on facebook are more related to socialisation with friends (hughes, rowe, batey, & lee, 2012). according to twittercounter.com’s ‘top 100’, the most followed businesses on twitter are news and sports organisations, reflecting that consumers use the platform when they are searching for information. maven-like behaviour from businesses on twitter needs to be informative, because consumers are using the platform for information search (smith, fischer, & yongjian, 2012). a need for cognition is related to mavenism because it is based on gathering and spreading information. indeed, the most successful businesses on twitter, in terms of influencing consumers, are actively spreading information that may be useful to them. a few recent examples from the most followed brands on twitter is warranted: • how to brew a soft cup of coffee [video insert]. tweeted by @starbucks which has 11.7 million followers. the tweet received 576 retweets. • be the first to know. the official go-to destination for #thenextgalaxy real-time news, stories and more. @samsungmobile account tagline; 11.5 million followers. australasian journal of information systems aleti, harrigan, cheong & turner 2016, vol 20, research article australian brewing industry on twitter 4 • the division launches tomorrow on ps4. read our q&a to learn about @ubisoft's infected nyc: http://... tweeted by @playstation which has 10.3 million followers. the tweet received 439 retweets. in all these examples the businesses are appealing to consumers’ search for information about their products. instead of simply advertising their products, they are offering expert advice and acting as a knowledge hub for others. starbucks is leading the way as a coffee connoisseur (in the eyes of its followers) by offering consumers expert advice about how to brew coffee at home. samsung is appealing to consumers who are looking for the latest in mobile technology, while at the same time putting themselves at the forefront of new knowledge. finally, playstation is offering the latest in game development, with a link to an interview they did with ubisoft – a video game developer and publisher. as such, they are all acting like mavens since they are actively collecting and assimilating relevant information for others. by doing so, these businesses have reached a substantial number of followers and their content is also frequently retweeted. that is, their maven-like behaviour is associated with influencing consumers in two different ways; retweeting and followers. 3.1 influence of twitter; followers and retweets focusing on the potential to lead others to engage in a certain act, we highlight two different interpersonal activities businesses can engage in on twitter. first, consumers interact by following updates of businesses who post interesting tweets. second, consumers can pass along interesting pieces of information from businesses to their own followers. this is known as retweeting, which can be identified by the use of rt @username. these activities represent different types of influence a business can have on consumers: 1. ‘indegree influence’, the number of followers of a user, indicates the size of the audience for that user (cha et al., 2010). 2. ‘retweet influence’, the number of retweets containing one’s name, indicates the ability of that user to generate content with pass-along value (cha et al., 2010). following someone on twitter means that you agree to see his or her messages in your own ‘feed’ on an on-going basis. it is of course possible to ignore the information from some of the accounts followed, but the number of followers is nevertheless a measure of potential reach or influence (twitter, 2015). either way, in commercial settings, a business needs to ‘persuade’ consumers to follow their accounts in the first place (de vries, gensler, & leeflang, 2012). being followed on twitter gives a business the potential to connect with consumers who will receive the communication the business place in their feed. however, they may be ‘passive’ followers, who never read the tweets of particular people they follow. as such, having a large number of followers is a necessary but insufficient condition; a business also need to focus on ‘retweet influence’ with consumers. indegree influence represents a user’s popularity. it is, however, not related to other important notions of influence related to audience engagement, i.e., retweets (cha et al., 2010). by reposting a message a connection has taken place, and the business has the potential of connecting with even more consumers through the network of the re-tweeter. in contrast to indegree influence, retweet influence is known to be driven by the content value of the tweet (cha et al., 2010). in order to achieve maven-like characteristics, businesses need to master both indegree and retweet influence. that is, they need to have reach of their messages through indegree influence and they need to provide content that is of value to consumers so that they are willing to retweet the content and further spread the message. research has not established how businesses should use the tools available on twitter to increase their popularity and what type of content is perceived as valuable to consumers. in the following, we draw on previous research on twitter in order to build hypotheses for how businesses can improve their influence. we look at four areas of communication on twitter and how this can be used to gain influence. these are: message formulation and language, native platform behaviour, reciprocity and persistency. australasian journal of information systems aleti, harrigan, cheong & turner 2016, vol 20, research article australian brewing industry on twitter 5 3.2 message formulation and language brands are mentioned in around 20 per cent of tweets (smith et al., 2012). however, tweets need to contain information in order to attract consumers’ attention (de vries et al., 2012). language in itself can be more or less engaging. as suggested by brodie et al. (2013), consumers are stimulated by the information they find online and will further engage with businesses that provide them with the information they need. twitter’s interactive nature is particularly suited to conversations rather than self-presentation (kietzmann, hermkens, mccarthy, & silvestre, 2011). broadcasting tweets is an important part in the process of gaining more followers (zhang, jansen, & chowdhury, 2011). however, it seems fair to assume that the same message broadcasted repeatedly will be less engaging than new and interesting stories. repeat messaging in an advertising-like manner may also be the antithesis of mavenlike behaviour; it does not show knowledge – nor does it help others. as such, we hypothesise that using a wide vocabulary and discussing a wide range of topics would result in a higher number of followers and a higher likelihood of material being shared. we assess average vocabulary by calculating the lexical diversity across tweets made by an organisation (cheong, 2013; wagner & strohmaier, 2010). the measure of lexical diversity is defined as the ratio of unique words divided by the total number of words, creating a mean vocabulary size per message (wagner & strohmaier, 2010). the efficiency of this metric lies in its ability to summarise the range of vocabulary deployed by a writer, which is modeand purpose-dependent (mccarthy, 2006). lexical diversity belongs to the family of ‘luhn summarisation techniques’, which depends on the frequency analysis of the words in a document as defined by luhn (1958). in the case of a twitter message, the ‘document’ is thus defined as the entire tweet (cheong, 2013). also, to remove ‘noise’ from the content of the tweet, ‘stopwords’ (commonly-occurring function words such as articles, pronouns, particles, and conjunctions) should not be counted (luhn, 1958). this will yield a subset of words which are more significant in capturing the essence of the tweet (cheong, 2013; luhn, 1958). we hypothesise that; h1a a higher lexical diversity of tweets is positively associated with higher levels of indegree and retweet influence. the frequency of using question marks gives us an indication of the level of interactivity sought in the communication. this is an important pointer to the use of twitter as a way to foster interactions, rather than broadcasting information (kumar et al., 2010; van bruggen, antia, jap, reinartz, & pallas, 2010). asking questions would be one means of encouraging consumers to engage through following and retweeting, and would show recognition of the power of user generated content (kaplan & haenlein, 2010; liu-thompkins & rogerson, 2012). one way of seeking connections and information through twitter is to ask questions to followers (naaman, boase, & lai, 2010). thus, we hypothesise that; h1b use of question marks in tweets is positively associated with higher levels of indegree and retweet influence. the use of exclamation marks indicates strong feelings or high volume (shouting). proclaiming strong feelings is a means to attract consumers’ attention, and thus to engage with them (de vries et al., 2012). exclamation marks are a means of conveying the importance of an announcement, which may include new important knowledge to consumers. thus, we hypothesise that; h1c more frequent use of exclamation marks in tweets is positively associated with higher levels of indegree and retweet influence. 3.3 ‘native’ platform behaviour native platform behaviour refers to using the communication tools native to that specific site. on twitter, this means posting photos and videos, using ‘hashtags’ (tagging a particular message with keywords preceded by a #hash symbol), and sending tweets directly to other users. we hypothesise that using such tools would contribute to a higher number of followers australasian journal of information systems aleti, harrigan, cheong & turner 2016, vol 20, research article australian brewing industry on twitter 6 and a higher likelihood of material being shared (the characteristics of a market maven). recent research by twitter themselves (and others) has found that posts including media such as photos and videos garner significantly higher levels of engagement than more ‘basic’ textbased posts (e.g. rogers, 2014). the top commercial organisations in the world all use images and videos on twitter, with many drawing their content from their consumers’ user-generated photos and videos (lawrence, fournier, & brunel, 2013; smith et al., 2012). thus, we hypothesise that; h2a more frequent use of pictures and videos in tweets is positively associated with higher levels of indegree and retweet influence. from authoritative literature on twitter and microblogging platform behaviour in the early 2010s (cheong, 2013), the #hashtag is an important part of the message as opposed to merely describing a message (huang, thornton, & efthimiadis, 2010). #hashtags increase the likelihood of a tweet being engaged with, as users are inclined to comment or share their views/commentary (huang et al., 2010) – particularly after a #hashtag has gained popularity. by using hashtags, organisations are classifying their tweets according to the wider, relevant conversations taking place on twitter. hashtagged keywords link the tweet to an existing conversational topic – or create one from scratch – for others to join (burton & soboleva, 2011). this recognises the power of c2c conversations on social media (bruns & burgess, 2012; libai et al., 2010), which are enabled by hashtags on twitter. the hashtags also make it easier for consumers to search for specific information. helping consumers find what they are looking for may also be perceived as maven-like behaviour and related to influence. thus, we hypothesise that; h2b more frequent use of hashtags in tweets is positively associated with higher levels of indegree and retweet influence. twitter offers the possibility of directly addressing another user in a tweet by including the recipient’s username in the tweet message, i.e. @user tweets. studies on direct interactions (tweets with ‘@’ signs), are intended to signal interpersonal correspondence, with emphasis on the twitter platform for purposes of conversation and collaboration (boyd, golder, & lotan, 2010; honeycutt & herring, 2009). we assessed this in two ways: (a) the percentage of tweets containing @users and (b) the number of unique @users tweeted to. this allows us to assess whether directly addressed tweets were sent between a large or small number of users. oneto-one interactions directly with consumers are important on social media in order to connect with them (baek, ahn, & choi, 2012; brodie, ilic, juric, & hollebeek, 2013; hollebeek, 2013; van doorn et al., 2010). in particular, organisations are likely to reach out to influential consumers, such as those with expertise or a large number of followers (aral & walker, 2012), which would increase the level of influence. such direct interaction would further entail maven-like traits, assuming that the information provided is of value. thus, we hypothesise that; h2c more frequent use of direct interactions in tweets is positively associated with higher levels of indegree and retweet influence. 3.4 reciprocity retweeted messages have elements of information sharing and social tagging (cheong, 2013), as in the presence of urls and hashtags [with the motivation of] spreading tweets, starting a conversation, and drawing attention to the originating use (boyd et al., 2010). everyday twitter users have a roughly-equal follower-to-friend ratio (ffr close to 1.0), regard twitter as a social network and tend to exhibit reciprocity in their relationships (krishnamurthy, gill, & arlitt, 2008). as a social network, these empirical observations in the early days of twitter indicate an inherent reciprocity. the most obvious way to reciprocate on social media is through retweeting another user’s message. retweeting illustrates the extent to which organisations would forward (or re-post) content posted by other users. this is important, as retweeting is a form of interacting with consumers and others through sharing their tweets. this is a particularly common behaviour australasian journal of information systems aleti, harrigan, cheong & turner 2016, vol 20, research article australian brewing industry on twitter 7 on twitter where content-sharing through retweeting is often easier for organisations than creating their own content (shi, rui, & whinston, 2014). also, retweets are shown in the originator’s account, which makes the process of reciprocation visible to others. we hypothesise that; h3a more frequent use of retweets is positively associated with higher levels of indegree and retweet influence. twitter also allows users to mark other users’ tweets as favourites. by marking a tweet as a favorited a user indicates that they have read and liked it (similar to a ‘like’ on facebook). this is important as it shows that organisations are using twitter as an information source, paying attention to the content posted by their consumers. this is a key benefit of using social media, not least in terms of product innovation ideas, consumer support, new consumers and business partners (chau & xu, 2012; moran & gossieaux, 2010). we hypothesise that; h3b more frequent use of favourites is positively associated with higher levels of indegree and retweet influence. attempts at generating followers on twitter may be maximised by reading, liking and retweeting the posts of others, and not just ‘announcing’ one’s own presence via a barrage of tweets. to be able to do this, organisations need to follow other users. this would certainly be evidence of an organisation embracing the two-way nature of social media (baek et al., 2012). thus, we hypothesise that; h3c more frequent use of following is positively associated with higher levels of indegree and retweet influence. 3.5 persistency finally, a measure for persistency on twitter is important. this is because it seems likely that both the number of followers and retweets increase with time and with a high volume of tweets. we suspected that a maven-like twitter account would continue to grow in terms of followers and retweets over time. likewise, a higher volume of tweets over time would increase the likelihood of more consumers following and sharing information from the account. length of time on twitter facilitates the identification of new accounts and the degree of seniority of users (cheong, 2013). further, messaging frequency (as a function of length of time on twitter and number of tweets) reflects the degree of a user activity on twitter. however, users typically show more activity in the early life of their twitter account, and after their first few tweets a significant proportion of users leave their twitter accounts idle (cheong, 2013). therefore, we posit that a certain degree of seniority is a prerequisite for a maven-like twitter account. thus, we hypothesise that; h4a length of time on twitter is positively associated with higher levels of indegree and retweet influence. h4b number of tweets is positively associated with higher levels of indegree and retweet influence. australasian journal of information systems aleti, harrigan, cheong & turner 2016, vol 20, research article australian brewing industry on twitter 8 constructs twitter variables description indegree influence total number of followers indegree influence is a function of the twitter audience’s willingness to follow the communication of a particular organisation. it directly indicates the size of the audience for that user. retweet influence total number of retweets by others others’ willingness to retweet a message means that the business communication is reaching further. it indicates the ability of businesses to generate content with passalong value. h1: message formulation and language lexical diversity exclamation marks! question marks? a message’s formulation is likely to relate to how it is perceived by the audience. e.g., repetition of advertising may have little influence on consumers. h2: ‘native’ platform behaviour photos and videos #hashtags (total and unique) @user (total and unique) consumers would be more likely to connect with and follow businesses and retweet their content when they appear ‘native’ on twitter. h3: reciprocity total number of retweets total number of favourites total number of following following and sharing others’ content signals that an organisation is interested in what others do. it also shows that they are reciprocating and are willing to build a relationship. h4: persistency month and year joined total number of tweets the age and volume of an account may accumulate followers over time (assuming users are not ‘unfollowing’). similarly, persistency in messaging should increase the likelihood of retweets. table 1: construct and variable summary 4 research context we focused our study on australia, since australian consumers are amongst the world’s heaviest users of social media (sensis, 2012). this includes commercial organisations, with almost 80 per cent of australian businesses being present on social media (sensis, 2014). the social media usage of australian organisations therefore represents a valuable context in terms of informing and understanding social media more widely. we also chose to focus on australia because of the composition of the industry under investigation. the australian brewing industry is mature and established, dominated by large multinational corporations (mass breweries). the two largest brewing organisations (ukbased sab miller and japanese kirin holding) control approximately 90% of the overall market. however, the recent emergence of numerous small, independent ‘craft breweries’ is noteworthy. within this wider context the way large and small organisations use twitter appears to differ. national statistics indicate that large organisations are more active than their smaller competitors on social media, as well as having their activities coordinated by a marketing department rather than the owner/management, which is often the case in small organisations (sensis, 2014). small organisations still perceive social media as having a positive impact on their business, although their interactions with consumers are often modest (augar & zeleznikow, 2014). the australian brewing industry can be said to be mature as well as highly competitive. in such industries, social media investments are related to firm value (plangger, 2012), which suggests that the australian brewing industry should be visible and active on social media. as a result of the competitive nature of this industry, the new entrants (craft breweries) are also likely to have an entrepreneurial orientation (watne & hakala, 2013), with a focus on innovation. such a focus is also said to be a key factor in successful social media implementation in large organisations (valos, polonsky, mavondo, & lipscomb, 2014). from a mass brewery perspective, consumers have difficulties telling beer products apart (jacoby, olson, & haddock, 1971). craft breweries are perceived as offering a ‘counter-culture’ australasian journal of information systems aleti, harrigan, cheong & turner 2016, vol 20, research article australian brewing industry on twitter 9 to the mass breweries (carroll & swaminathan, 2000). they routinely differentiate their products according to taste and the raw materials they use (adams, 2006) and they are typically more focused on educating consumers to know more about beer and brewing (watne, 2012). thus, craft brewers tend to act in a more maven-like way. they might focus on influencing those looking for information on twitter, in order to foster engagement in their social media marketing. 5 methodology all twitter streams (i.e. handles, or users) for all breweries and beer brands in australia with an active twitter account at the time were ‘harvested’ in mid-june 2014. this allowed us to compare and contrast the communication of large and small players from a complete sample of the industry. through such a comparison we were able to observe how influence on social media differs between large and small organisations in the same industry. to obtain this data, we searched the websites of the three major brewing organisations in australia, and further searched twitter for accounts related to the breweries they own as well as the brands in their portfolios. we further searched the member sections of industry organisations for craft breweries: the australian craft beer industry association (cbia) and the australian real craft brewers association. from this we were able to obtain a complete list of all beer brands and breweries in australia with an active twitter account; 95 accounts in total. of these, 83 were regarded as craft breweries, (according to cbia definition) and 9 were mass breweries. in order to harvest the twitter data from the breweries, we used a ‘best-effort’ data collection strategy (cheong & lee, 2010) involving the ‘twitter rest api’ (application programming interface). the twitter rest api is a service that obtains raw data about a particular handle or user and their messages, allowing researchers to harvest tweets for free from particular users (russell, 2011). our technique is preferred over conventional scraping methods – such as manually copying-and-pasting text from the twitter website – as the twitter rest api allows for automated and systematic harvesting of tweets. the rest api has been used in various research projects since 2010 (cheong, 2013; serrano & iglesias, 2016), which is testament to its stability and effectiveness in extracting twitter data. we harvested tweets posted by australian beer brands and breweries using a collection of ‘scripts’ written in the perl programming language by first using the search api (a subset of the rest api). for some of the measures, the persistence and volume of activity would greatly impact the validity of the findings. lexical diversity, for example, tends to converge when analysing larger amounts of text (cheong, 2013). since some of the breweries were more active than others, certain measures would be biased by such variation in activity. accordingly, we randomly selected 100 tweets from each account in order to analyse the measures related to message formulation and native platform behaviour (n=9.500 tweets from the 95 accounts). this was favoured over the most recent 100 tweets as this would give a disproportionate sample of tweets from the last few weeks (from some brewers who are active on twitter) but would return results from far back for more sporadic twitter users. thus, we avoided potential bias related to particular events that some brewers would have tweeted about. the use of stratified random sampling (of users and messages) – as opposed to using all the samples for a given stratum of users – for the study of twitter phenomena has been widely documented, e.g. (cheong & lee, 2009; cormode, krishnamurthy, & willinger, 2010; ehrlich & shami, 2010). 6 results most of the mass brewers had twitter accounts for their brands, as distinct from the breweries themselves, while all of the craft breweries had twitter accounts only for the brewery. large differences were found within the dataset in terms of individual activities on twitter. for example, the most active tweeter in the sample sent 21,900 tweets while the least active sent only 142 tweets. table 2 gives an overview of the popularity of mass and craft breweries as well as their twitter use. australasian journal of information systems aleti, harrigan, cheong & turner 2016, vol 20, research article australian brewing industry on twitter 10 table 2 shows some differences between the craft breweries and the mass breweries. overall, the differences were minor, but due to the low number of mass-brewers we were very lenient in the reporting of significant differences (<0.1), in order to tease out nuances of difference. in terms of how they communicate, craft brewers are using a wider vocabulary. this may be a common attribute of a maven-like behaviour; they talk about a large number of topics and engage in conversations as opposed to posting advertising messages. however, they use fewer question marks than the mass brewers. this could be related to them giving answers as opposed to asking questions. also, maven-like behaviour is seen in craft brewers posting more visual tweets and communicating directly with others (@user) more frequently. finally, craft brewers are more active on average, posting more tweets than the mass brewers. in the following, we investigate the hypotheses. activity mean t-test craft mass t sig. influence* indegree (followers) 1666.0 2149.3 .847 .399 retweet influence 318.3 206.4 -.713 .481 message formulation and language** lexical diversity 0.63 0.59 -2.036 .070 ? tweets 10.3 21.0 4.906 .000 ! tweets 35.4 36.7 .175 .865 ‘native’ platform behaviour** photo/video 127.7 61.6 -1.935 .064 # tweets 20.9 27.8 .984 .351 # unique 24.6 20.2 -.641 .536 @ tweets 64.3 48.0 -1.732 .087 @ unique 86.0 57.1 -1.788 .107 reciprocity* retweet 23.4 15.3 -1.431 .186 favourites 142.9 200.6 .555 .580 following 789.9 675.7 -.497 .629 persistency* joined (mm/yy) total tweets 1921.7 1022.3 -2.034 .050 * the measure relates to overall activity. ** the measure relates to a random sample of 100 tweets from each account. table 2: descriptive statistics and t-test for differences between craft and mass brewers there were large variations in activity between the breweries in the dataset. as such, the data was considered as non-normal (ho, 2014) so we recoded all the variables from the harvested tweets into 5-point scales. the data was recoded based on the variety within each variable. for example, number of followers were recoded into; 1.) 0-500; 2.) 501-1000; 3.) 1001-1500; 4.) 1501-2500; and 5.) >2500. the data was recoded in order to achieve a normal distribution, appropriate for parametric tests (ho, 2014). from this, a correlation analysis could be conducted between followers and retweets of consumers and twitter communication from the organisations. table 3 shows the correlations found within the dataset. australasian journal of information systems aleti, harrigan, cheong & turner 2016, vol 20, research article australian brewing industry on twitter 11 business activity consumer response: indegree influence consumer response: retweet influence message formulation and language lexical diversity 0.34** 0.19 question mark? -0.07 -0.06 exclamation point! 0.02 -0.19 ‘native’ twitter behaviour photo/videos 0.58** 0.27** # tweets -0.17 0.33** # unique -0.20 0.42** @ tweets 0.26** 0.58** @ unique 0.32** 0.57** reciprocity retweet 0.19 0.60** favourites 0.19 0.34** following 0.45** 0.21* persistency joined -0.49** 0.27** total tweets 0.61** 0.07 table 3: significant correlations between twitter popularity and communication style when it comes to message formulation and language, lexical diversity was correlated with indergee influence. there were no correlations with retweet influence. regarding the native platform behaviour, it was clear that these variables had a larger impact on both indegree and retweet influence. all the variables were related to retweet influence, and some also to the number of followers. reciprocity between the organisations and the audience was also evident, with strong correlations between activities and retweets. finally, in terms of persistence, strong correlations were also found with followers and volume of tweets. however, a negative correlation regarding year joined was also found, counter to our hypothesis h4a for indegree influence. our next analysis was to regress all the variables in the study with the dependent variables (indegree and retweet influence). many of the correlations in table 3 did not contribute to the variance in followers and retweets in the regression analysis. since number of followers and retweets were statistically unrelated to each other (confirming the work of cha et al., 2010), we looked at them separately. this also confirms that our measures of indegree and retweet influence are conceptually different – consumers follow and retweet for different reasons. the results of the stepwise regression analysis are illustrated in table 4. australasian journal of information systems aleti, harrigan, cheong & turner 2016, vol 20, research article australian brewing industry on twitter 12 variable std. β t (sig.) f (sig.) r sq. adj. r sq. retweet influence (constant) 2.928 (.004) 23.044 (.000) .506 .484 retweet .341 3.510 (.001) @ tweets .318 3.369 (.001) exclamation point! -.204 -2.717 (.008) # unique .200 2.524 (.013) indegree influence (constant) 6.265 (.000) 28.943 (.000) .563 .543 total tweets .342 4.042 (.000) joined -.418 -5.594 (.000) @ unique .197 2.503 (.014) following .190 2.295 (.024) table 4: stepwise regression analysis between business activity, retweets and followers hypothesis support message formulation and language indegree influence retweet influence h1a a higher lexical diversity of tweets is positively associated with higher levels of indegree and retweet influence. partly supported (correlation analysis) not supported h1b use of question marks in tweets is positively associated with higher levels of indegree and retweet influence. not supported not supported h1c use of exclamation marks in tweets is positively associated with higher levels of indegree and retweet influence. not supported not supported (opposite relationship) ‘native’ platform behaviour h2a more frequent use of pictures and videos in tweets is positively associated with higher levels of indegree and retweet influence. partly supported (correlation analysis) partly supported (correlation analysis) h2b more frequent use of hashtags in tweets is positively associated with higher levels of indegree and retweet influence. not supported supported h2c more frequent use of direct interactions in tweets is positively associated with higher levels of indegree and retweet influence. supported (unique @ user) supported (total @ user) reciprocity h3a more frequent use of retweets is positively associated with higher levels of indegree and retweet influence. not supported supported h3b more frequent use of favourites is positively associated with higher levels of indegree and retweet influence. not supported partly supported (correlation analysis) h3c more frequent use of following is positively associated with higher levels of indegree and retweet influence. supported partly supported (correlation analysis) persistency h4a length of time on twitter is positively associated with higher levels of indegree and retweet influence. not supported (negative correlation) partly supported (correlation analysis) h4b number of tweets is positively associated with higher levels of indegree and retweet influence. supported not supported table 5: hypotheses supported and not supported australasian journal of information systems aleti, harrigan, cheong & turner 2016, vol 20, research article australian brewing industry on twitter 13 retweeting others’ content, sending @ user tweets, avoiding exclamation marks and using unique hashtags contributed to 48 per cent of the variance in retweet influence. further, 54 per cent of the variance in indegree influence was explained by the regression. consumers are most likely to follow those who tweet often, use videos and photos, send direct @ user tweets to a large number of users (unique @ user tweets), and follow them back. it seems that consumers also follow newer accounts, indicated by the negative relationship with year joined. for conciseness, table 5 summarises the hypothesis testing. 7 discussion hypothesis 1 related message formulation and language to indegree and retweet influence. we found little support for this hypothesis. for the question and exclamation marks, we had proposed that these be used to attract attention which would lead to followers and retweets (de vries et al., 2012). however, using exclamation marks had a negative impact on retweets, which indicates that it may be perceived as ‘shouting’ and thus ignored. there was partial support for an association between tweets with a higher lexical diversity and followers, which supports brodie et al.’s (2013) assertion that consumers are information-seeking and that organisations providing it are more engaging. we did not find a relationship between lexical diversity and retweet influence, which suggests that retweeting is not a function of how much different information is provided, but rather whom it is provided to and how it is delivered. a few example tweets illustrate this: • tweet reflecting business shouting advertising messages about where to buy the beer in question: ”good to hear! @darren_rolfe pure at 1st choice liquor. i'm stocking up!!!”. this tweet may be seen as positive for the person it was directed at. nevertheless, it is unlikely that it will have any further influence on other consumers because it shouts advertising at the audience. • a business talking about a wide range of topics: 1. ”just wondering, what beer styles would you like to see mpb brewing in 2013? can't promise we'll brew 'em though...”. 2. “congratulations renee & leigh on the birth of corey leigh denett at 5.33pm today weighing in at 4.4kg.”. 3. “our specialty beer is an english style ipa at 5.1% abv. pouring golden amber in colour this ipa delivers sweet...”. first tweet is a call to action; encouraging followers to directly engage in a technical discussion about beer styles. the second tweet reflects a ‘resting maven’ that is being human and congratulating a friend that gave birth. in the last tweet, the maven is back with a technical description of their new product (note; without a direct selling proposition). hypothesis 2 related native platform behaviour to indegree and retweet influence. we found partial support for this hypothesis. there were associations between the number of pictures and videos in tweets and the number of direct interactions in tweets with followers and retweets. often, photos and videos from the breweries, and indeed the brewing process, were featured; demonstrating the expertise of the maven. visual media have been shown to garner higher engagement in other industries and on other platforms (lawrence et al., 2013; smith et al., 2012). likewise, the ability to interact directly with an individual consumer (@user) is an important means of improving influence on social media (baek et al., 2012; brodie et al., 2013; hollebeek, 2013; van doorn et al., 2010). we also found support for an association between the use of hashtags and retweets, but not followers. hashtags enable c2c conversations on particular topics on social media and thus their potential to facilitate connections is clear (libai et al., 2010). the link between hashtags and retweets is interesting as it suggests that consumers search and use hashtags to further circulate information they find useful or interesting. on twitter, retweet influence is a function of following themed conversations and trends (bruns & burgess, 2012), which suggests that consumers are adding information to what they already are interested in and know about. a few example tweets illustrate the usage of hashtags and @user: • tweets with purposeful usage of hashtags directly addressed to individual consumers: 1. “four tastes matched with #sunsetale @rhcl3121 #craftbeer #gbw2013 australasian journal of information systems aleti, harrigan, cheong & turner 2016, vol 20, research article australian brewing industry on twitter 14 #foodporn @ richmond hill cafe & larder http://…”. 2. “@pouringinmelb @theparkbeer must get out to the park at werribee for our american brown on hand pump for @goodbeerweek #gbw2013 any left?”. 3. “@calibrebeer wow dan. stiff competition. nice one. #realcraftbeer #australianowned #independent #loveyourwork”. the first two tweets are purposely using the hashtag ‘#gbw2013’; referring to good beer week – a melbourne based beer festival. this makes it easy for users to find and forward their events during the festival. the last tweet is using hashtags that signals craft and independence. further, the @user in the tweets are directed towards beer venues and blogs. these are users with larger networks and as such an increased opportunity for more exposure, followers and retweets. acting like a maven means that businesses need to navigate their way within their area of expertise and take advantage of other knowledge-hubs that they can easily reach within the twitter community. hypothesis 3 related reciprocity to indegree and retweet influence. we found partial support for this hypothesis, where the number of users followed by an organisation associates positively with the number of followers they have. this emphasises the interactive and reciprocal nature of social media where organisations must follow consumers to understand how best to engage with them (baek et al., 2012; brodie et al., 2013). there was also partial support for an association between following and retweet influence, which reinforces the point. we additionally found positive associations between the number of retweets and favourites by the organisation, and retweet influence on consumers. when organisations actively engage with consumers by sharing their material, current consumers will reciprocate (shi et al., 2014). the followers a business already has are those who are most likely to retweet their messages (zhang et al., 2011). as such, reciprocity is the key to connecting with existing followers by listening to them. it is also the key to attracting new followers by starting to follow them. here are a few examples of retweeting (rt): • 1. “rt @beer_maven: learn how to distinguish different beers http://…”. 2. “rt @goodbeerweek: learn to bbq with beer at @matildabaybeer! just a few tix left for some of the sessions! http://…”. the first tweet is forwarding a blog post about beer styles posted by a beer enthusiast under the name ‘@beer_maven’. an example of connectivity and acting like a maven by spreading information about beer. the second tweet is forwarding a tweet from the good beer week festival, where consumers can learn more about barbequing with beer. again, a maven at work spreading relevant information. finally, hypothesis 4 related persistency of activity on twitter to indegree and retweet influence. we found partial support for this hypothesis; the overall number of tweets is positively associated with followers, but not retweets. indeed, the second most followed brewery in the sample also had the highest volume of tweets. creating ‘noise’ through high volumes of tweet seems to impact indegree influence, especially when mixed with other effective means such as purposeful use of hashtags. however, retweet influence may require more interactive tweets over a longer period of time. the length of time an organisation is on twitter is positively associated with followers, but not retweets. this may be explained by the fact that the longer an organisation has been on twitter, the number of retweets can only increase. however, the number of followers can increase and decrease. 8 implications in total, craft breweries in australia had 143,276 followers on twitter. in contrast, only 19,344 twitter users followed the mass breweries. this means that the craft breweries are outperforming the mass breweries with an 8:1 ratio in terms of indegree influence on twitter. in most industries, small businesses struggle to match the level of impact large businesses have on social media. however, in the brewing industry in australia, the opposite is the case; the businesses that holds about two per cent of the market share, holds about 90 per cent of the share of the voice on twitter. this reflects that the pareto principle (20/80) may indeed be australasian journal of information systems aleti, harrigan, cheong & turner 2016, vol 20, research article australian brewing industry on twitter 15 turned upside down on social media. craft brewers are better than their multinational counterparts in terms of influence on consumers on twitter; consumers interested in the category will turn to them for information. thus, we characterise these organisations as displaying maven-like behaviour. social media is an ideal environment to behave like a maven. organisations can share market-related information and network effectively and efficiently by acting like mavens. this may also be a reflection of twitter as a social media platform. as suggested by hughes et al. (2012), twitter is more a forum for information while facebook is more geared towards social connections. this research makes methodological contributions around the collection and analysis of big data. we conceptualised publically available variables as reflective of maven-like behaviour through indegree and retweet influence – specifically, number of followers and number of retweets. we empirically illustrate that there is a range of variables around message formulation and language, native platform behaviour, reciprocity and persistency that are positively associated with such influence. this theoretical advancement allows us to investigate marketing concepts through big data, which is a key field for future research. our findings underline that acting in a maven-like manner can be an effective way of using social media by an organisation. firstly, the use of visual media on twitter is essential to influence consumers. these should be complemented with the use of relevant hashtags and, where possible, direct interactions with consumers. this enables marketing efforts to become part of a conversation and to join in on trending conversations relevant to their organisation. reciprocity is vital. to achieve indegree influence organisations need to follow consumers in order to encourage them to follow them back. retweet influence on the other hand relates to interacting with consumers through retweeting and spreading the content of other users. this will enable consumers to be powerful advocates for the organisation. although organisations may gain a following relatively quickly through tweeting, to become a maven necessitates the longer time that a reciprocal approach requires. finally, we recommend that organisations use the publicly available big data on twitter to investigate the nature of their own twitter behaviour, and draw on the findings from this paper to modify their behaviour in order to help them to think like and act as mavens on social media. 9 limitations / future research the observations made in this study suggesting that small companies do a better job than larger companies, might not be extended to other industries. however, other industries marked by a few ‘generalist’ large players, and several small ‘specialist’ players, may also benefit from acting like a maven in their social media strategy. examples might be found in producers of other alcoholic beverages, various food industries, supermarkets or other retailers. further research may look at a cross-section of ‘generalist’ and ‘specialist’ businesses in order to further investigate the differences between the two. in our data, we only had 9 generalists (mass brewers), which is insufficient for drawing statistically significant conclusions. by looking at more than one industry, or perhaps more than one country, differences in communication and response between particular groups can be made. however, cultural differences are likely to exist and the findings here may not be extended to other countries with different consumption patterns and a different composition of the industry in question. this study was conducted in twitter. in many ways, the twitter platform differs from the more widely used facebook platform. on facebook consumers often have regular user accounts while brands tend to create fan pages that consumers can ‘like’. in other words, interactions between businesses and consumers are not the same on the two platforms. the simple and public transparency of twitter, is not reflected on facebook. it is unclear to what extent consumers perceive a facebook ‘like’ as similar to following someone in twitter. thus, further research should investigate whether our framework can be implemented on other social media platforms. australasian journal of information systems aleti, harrigan, cheong & turner 2016, vol 20, research article australian brewing industry on twitter 16 10 conclusion this study demonstrates that big data can be used to investigate organisational influence on twitter. we demonstrate that the analysis of big data can be used to test hypotheses and explain relationships, and not just to develop propositions. consumers allow retweet influence from organisations that communicate directly with them, that join conversations relevant to them via #hashtags and that are open to a reciprocal relationship with them. further, indegree influence is related to how businesses communicate directly (one-to-one) with a large number of consumers and on a variety of topics. thinking like and acting as a maven by providing relevant information to consumers and actively engaging on delivering such information is one way of achieving greater influence on consumers through twitter. references adams, w. j. (2006). markets: beer in germany and the united states. journal of economic perspectives, 20(1), 189-205. doi:10.1257/089533006776526120 aral, s., dellarocas, c., & godes, d. (2013). introduction to the special issue—social media and business transformation: a framework for research. information systems research, 24(1), 3-13. doi:10.1287/isre.1120.0470 aral, s., & walker, d. (2012). identifying influential and susceptible members of social networks. science, 337(6092), 337-341. doi:10.1126/science.1215842 as-saber, s., rahman, s., rahim, m., cairns, g., härtel, c., & waheduzzaman, w. (2014). promises and pitfalls of using social media in public e-procurement: an appraisal. . australasian journal of information systems, 18(3), 227-241. doi:http://dx.doi.org/10.3127/ajis.v18i3.1097 augar, n., & zeleznikow, j. (2014). i just saw this on facebook, i need it now’: exploring small business use of facebook. australasian journal of information systems, 18(3), 243255. baek, h., ahn, j., & choi, y. (2012). helpfulness of online consumer reviews: readers' objectives and review cues. international journal of electronic commerce, 17(2), 99126. doi:10.2753/jec1086-4415170204 belch, m. a., krentler, k. a., & willis-flurry, l. a. (2005). teen internet mavens: influence in family decision making. journal of business research, 58(5), 569-575. doi:10.1016/j.jbusres.2003.08.005 boyd, d., golder, s., & lotan, g. (2010). tweet, tweet, retweet: conversational aspects of retweeting on twitter. paper presented at the 43rd hawaii international conference on system sciences (hicss '43), kauai, hi. brodie, r. j., ilic, a., juric, b., & hollebeek, l. (2013). consumer engagement in a virtual brand community: an exploratory analysis. journal of business research, 66(1), 105-114. doi:10.1016/j.jbusres.2011.07.029 bruns, a., & burgess, j. (2012). researching news discussion on twitter. journalism studies, 13(5-6), 801-814. doi:10.1080/1461670x.2012.664428 burton, s., & soboleva, a. (2011). interactive or reactive? marketing with twitter. journal of consumer marketing, 28(7), 491-499. doi:10.1108/07363761111181473 carroll, g. r., & swaminathan, a. (2000). why the microbrewery movement? organizational dynamics of resource partitioning in the u.s. brewing industry. american journal of sociology, 106(3), 715-762. doi:10.1086/318962 cha, m., haddadi, h., benevenuto, f., & gummadi, k. p. (2010). measuring user influence in twitter: the million follower fallacy. paper presented at the international aaai conference on web and social media, washington, dc. australasian journal of information systems aleti, harrigan, cheong & turner 2016, vol 20, research article australian brewing industry on twitter 17 chau, m., & xu, j. (2012). business intelligence in blogs: understanding consumer interactions and communities. mis quarterly, 36(4), 1189-1216. cheong, m. (2013). inferring social behavior and interaction on twitter by combining metadata about users & messages. (phd thesis), monash university, melbourne, australia. cheong, m., & lee, v. (2009). integrating web-based intelligence retrieval and decisionmaking from the twitter trends knowledge base. paper presented at the proceedings of the 2nd acm workshop on social web search and mining, hong kong, china. cheong, m., & lee, v. (2010). twitmographics: learning the emergent properties of the twitter community from sociology to computing in social networks: theory, foundations and applications (vol. 1, pp. 323-342): springer-verlag. cormode, g., krishnamurthy, b., & willinger, w. (2010). a manifesto for modeling and measurement in social media. first monday, 15(9). dang-pham, d., pittayachawan, s., & nkhoma, m. (2015). demystifying online personas of vietnamese young adults on facebook: a q-methodology approach. australasian journal of information systems, 19, 1-22. doi:http://dx.doi.org/10.3127/ajis.v19i0.1204 de vries, l., gensler, s., & leeflang, p. s. h. (2012). popularity of brand posts on brand fan pages: an investigation of the effects of social media marketing. journal of interactive marketing, 26(2), 83-91. doi:10.1016/j.intmar.2012.01.003 ehrlich, k., & shami, n. (2010). microblogging inside and outside the workplace. paper presented at the proc. the fourth international aaai conference on weblogs and social media, washington, dc. feick, l. f., & price, l. l. (1987). the market maven: a diffuser of marketplace information. journal of marketing, 51(1), 83-97. fournier, s., & avery, j. (2011). the uninvited brand. business horizons, 54(3), 193-207. doi:http://dx.doi.org/10.1016/j.bushor.2011.01.001 gladwell, m. (2000). the tipping point: how little things can make a big difference. boston: little, brown, and company. goodey, c., & east, r. (2008). testing the market maven concept. journal of marketing management, 24(3-4), 265-282. doi:10.1362/026725708x306095 ho, j. y. c., & dempsey, m. (2010). viral marketing: motivations to forward online content. journal of business research, 63(9-10), 1000-1006. doi:10.1016/j.jbusres.2008.08.010 ho, r. (2014). handbook of univariate and multivariate data analysis with ibm spss (2 ed.). boca raton, fl: taylor and francis. hodis, m. a., sriramachandramurthy, r., & sashittal, h. c. (2015). interact with me on my terms: a four segment facebook engagement framework for marketers. journal of marketing management, 1-30. doi:10.1080/0267257x.2015.1012535 hollebeek, l. d. (2013). the customer engagement/value interface: an exploratory investigation. australasian marketing journal (amj), 21(1), 17-24. doi:http://dx.doi.org/10.1016/j.ausmj.2012.08.006 honeycutt, c., & herring, s. c. (2009, 5-8 jan. 2009). beyond microblogging: conversation and collaboration via twitter. paper presented at the 42nd hawaii international conference on system sciences, (hicss '42). los alamitos, ca. hosanagar, k., fleder, d., lee, d., & buja, a. (2014). will the global village fracture into tribes? recommender systems and their effects on consumer fragmentation. management science, 60(4), 805-823. doi:10.1287/mnsc.2013.1808 australasian journal of information systems aleti, harrigan, cheong & turner 2016, vol 20, research article australian brewing industry on twitter 18 huang, j., thornton, k. m., & efthimiadis, e. n. (2010). conversational tagging in twitter. paper presented at the 21st acm conference on hypertext and hypermedia, toronto, ontario, canada. hughes, d. j., rowe, m., batey, m., & lee, a. (2012). a tale of two sites: twitter vs. facebook and the personality predictors of social media usage. computers in human behavior, 28(2), 561-569. doi:http://dx.doi.org/10.1016/j.chb.2011.11.001 jacoby, j., olson, j. c., & haddock, r. a. (1971). price, brand name, and product composition characteristics as determinants of perceived quality. journal of applied psychology, 55(6), 570-579. doi:10.1037/h0032045 kaplan, a. m., & haenlein, m. (2010). users of the world, unite! the challenges and opportunities of social media. business horizons, 53(1), 59-68. doi:http://dx.doi.org/10.1016/j.bushor.2009.09.003 katz, e., & lazarsfeld, p. (1955). personal influence: the part played by people in the flow of mass communications. new york: the free press. kietzmann, j. h., hermkens, k., mccarthy, i. p., & silvestre, b. s. (2011). social media? get serious! understanding the functional building blocks of social media. business horizons, 54(3), 241-251. doi:http://dx.doi.org/10.1016/j.bushor.2011.01.005 krishnamurthy, b., gill, p., & arlitt, m. (2008). a few chirps about twitter. paper presented at the proceedings of the first workshop on online social networks, seattle, wa, usa. kumar, v., aksoy, l., donkers, b., venkatesan, r., wiesel, t., & tillmanns, s. (2010). undervalued or overvalued customers: capturing total customer engagement value. journal of service research, 13(3), 297-310. doi:10.1177/1094670510375602 langley, d. j., hoeve, m. c., ortt, j. r., pals, n., & van der vecht, b. (2014). patterns of herding and their occurrence in an online setting. journal of interactive marketing, 28(1), 16– 25. doi:10.1016/j.intmar.2013.06.005 lawrence, b., fournier, s., & brunel, f. (2013). when companies don't make the ad: a multimethod inquiry into the differential effectiveness of consumer-generated advertising. journal of advertising, 42(4), 292-307. doi:10.1080/00913367.2013.795120 libai, b., bolton, r., bügel, m. s., de ruyter, k., götz, o., risselada, h., & stephen, a. t. (2010). customer-to-customer interactions: broadening the scope of word of mouth research. journal of service research, 13(3), 267-282. doi:10.1177/1094670510375600 liu-thompkins, y., & rogerson, m. (2012). rising to stardom: an empirical investigation of the diffusion of user-generated content. journal of interactive marketing, 26(2), 7182. doi:10.1016/j.intmar.2011.11.003 luhn, h. p. (1958). the automatic creation of literature abstracts. ibm journal of research and development, 2(2), 159-165. doi:10.1147/rd.22.0159 malsbender, a., hoffmann, s., & becker, j. (2014). aligning capabilities and social media affordances for open innovation in governments. australasian journal of information systems, 18(3), 317-330. doi:http://dx.doi.org/10.3127/ajis.v18i3.1100 mangold, w. g., & faulds, d. j. (2009). social media: the new hybrid element of the promotion mix. business horizons, 52(4), 357-365. doi:10.1016/j.bushor.2009.03.002 mccarthy, p. m. (2006). an assessment of the range and usefulness of lexical diversity measures and the potential of the measure of textual, lexical diversity (mtld). university of memphis. moran, e., & gossieaux, f. (2010). marketing in a hyper-social world the tribalization of business study and characteristics of successful online communities. journal of advertising research, 50(3), 232-239. doi:10.2501/s0021849910091397 australasian journal of information systems aleti, harrigan, cheong & turner 2016, vol 20, research article australian brewing industry on twitter 19 myrick, j. g. (2015). emotion regulation, procrastination, and watching cat videos online: who watches internet cats, why, and to what effect? computers in human behavior, 52, 168176. doi:10.1016/j.chb.2015.06.001 naaman, m., boase, j., & lai, c.-h. (2010). is it really about me?: message content in social awareness streams. paper presented at the proceedings of the 2010 acm conference on computer supported cooperative work, savannah, georgia, usa. plangger, k. (2012). the power of popularity: how the size of a virtual community adds to firm value. journal of public affairs, 12(2), 145-153. doi:10.1002/pa.1416 price, l. l., feick, l. f., & guskey-federouch, a. (1988). couponing behaviors of the market maven: profile of a super couponer. paper presented at the advances in consumer research, provo, ut. rogers, e. m. (1983). diffusion of innovations. new york: the free press. rogers, s. (2014). what fuels a tweet’s engagement? the twitter media blog: tracking cool, meaningful uses of twitter in tv, music, sports, government, journalism and more(twitter, inc.), accessed: 26 june, 2014. russell, m. a. (2011). mining the social web. sebastopol, ca: o'reilly media. sanders, r. (1987). the pareto principle: its use and abuse. journal of services marketing, 1(2), 37-40. doi:10.1108/eb024706 sensis. (2012). yellow social media report: what australian people and businesses are doing with social media. melbourne: sensis pty ltd sensis. (2014). yellow social media report: what australian people and businesses are doing with social media. melbourne: sensis pty ltd serrano, e., & iglesias, c. a. (2016). validating viral marketing strategies in twitter via agentbased social simulation. expert systems with applications, 50, 140-150. doi:http://dx.doi.org/10.1016/j.eswa.2015.12.021 shi, z., rui, h., & whinston, a. b. (2014). content sharing in a social broadcasting environment: evidence from twitter. mis quarterly, 38(1), 123-142. smith, a. n., fischer, e., & yongjian, c. (2012). how does brand-related user-generated content differ across youtube, facebook, and twitter? journal of interactive marketing, 26(2), 102-113. doi:10.1016/j.intmar.2012.01.002 twitter. (2015). twitter 101 for business: measure your impact. retrieved from: https://guides.co/g/twitter-101-for-business/12346 https://business.twitter.com/basics/measure-your-impact?lang=en&location=apac valos, m., polonsky, m. j., mavondo, f., & lipscomb, j. (2014). senior marketers’ insights into the challenges of social media implementation in large organisations: assessing generic and electronic orientation models as potential solutions. journal of marketing management, 31(7-8), 713-746. doi:10.1080/0267257x.2014.977931 van bruggen, g. h., antia, k. d., jap, s. d., reinartz, w. j., & pallas, f. (2010). managing marketing channel multiplicity. journal of service research, 13(3), 331-340. doi:10.1177/1094670510375601 van doorn, j., lemon, k. n., mittal, v., nass, s., pick, d., pirner, p., & verhoef, p. c. (2010). customer engagement behavior: theoretical foundations and research directions. journal of service research, 13(3), 253-266. doi:10.1177/1094670510375599 violet, u. (1988). famous for 15 minutes: my years with andy warhol. san diego: harcourt brace jovanovich. wagner, c., & strohmaier, m. (2010). the wisdom in tweetonomies: acquiring latent conceptual structures from social awareness streams. paper presented at the australasian journal of information systems aleti, harrigan, cheong & turner 2016, vol 20, research article australian brewing industry on twitter 20 proceedings of the 3rd international semantic search workshop, raleigh, north carolina. watne, t. (2012). agents of change: an investigation of how craft breweries educate their consumers. paper presented at the australia and new zealand marketing academy (anzmac) adelaide, australia. watne, t., & hakala, h. (2013). inventor, founder or developer? an enquiry into the passion that drives craft breweries in victoria, australia. journal of marketing development and competitiveness, 7(3), 54-67. zeithaml, v. a. (1985). the new demographics and market fragmentation. journal of marketing, 49(3), 64-75. zhang, m., jansen, b. j., & chowdhury, a. (2011). business engagement on twitter: a path analysis. electronic markets, 21(3), 161-175. doi:10.1007/s12525-011-0065-z acknowledgement this research was funded by victoria university’s research development grant scheme (rdgs), grant number 53/13. the authors would like to thank victoria university for the support for this project. copyright: © 2016 aleti, harrigan, cheong & turner. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 literature review 3 conceptual development 3.1 influence of twitter; followers and retweets 3.2 message formulation and language 3.3 ‘native’ platform behaviour 3.4 reciprocity 3.5 persistency 4 research context 5 methodology 6 results 7 discussion 8 implications 9 limitations / future research 10 conclusion paper title (paper title style) australasian journal of information systems vlahu-gjorgievska, nagapuri & win 2019, vol 23, research on health information systems monitoring using tele-monitoring technology 1 tele-monitoring technology as a tool for monitoring and management of patients with congestive heart failure elena vlahu-gjorgievska school of computing and information technology university of wollongong elenavg@uow.edu.au swetha nagapuri school of computing and information technology university of wollongong khin than win school of computing and information technology university of wollongong abstract telehealth interventions are designed to facilitate the remote exchange of information and data between patients and health care providers, improving the quality and safety of the patients and increasing efficiency and cost-effectiveness of health care providers. the development of telecommunications and virtual technology allowed a number of telehealth systems to be applied in different health care areas. these technologies can provide an alternative monitoring and solutions for decreasing the hospital readmission rates for patients with congestive heart failure (chf). in this paper, a systematic literature review regarding telemonitoring and its use in the management of patients with chf is conducted. the result indicated a that standardized tele-monitoring design would reduce a length of hospitalization and re-hospitalization rate. the other factors that moderate the effectiveness of the telemonitoring intervention include quality of life, mortality rate, and disease-specific knowledge (health literacy). all these factors align with the consumer-centred principle of the australian safety and quality framework for health care. keywords: tele-monitoring; health care; health literacy; continuity of care; congestive heart failure 1 introduction cardiovascular diseases are the number one cause of death globally. in 2013 cardiovascular disease caused 17.3 million deaths or about 32% of all deaths that year (zipes et al., 2018). congestive heart failure (chf) is defined as a complex chronic condition where the heart pumps the blood slowly and weakens the pumping power of blood, which can result in numerous heart disease conditions like coronary artery disease, heart attack and cardiomyopathy (velagaleti & vasan, 2007). this growth in the number of patients suffering from chf leads to an increase in hospital readmissions every year. thus, patients with heart failure need to be monitored after they have been diagnosed or discharged from the hospital to prevent progress in disease and improve their health. tele-monitoring systems are one of the solutions that can provide the required patients’ assistance in various ways (ponikowski et al., 2014). australasian journal of information systems vlahu-gjorgievska, nagapuri & win 2019, vol 23, research on health information systems monitoring using tele-monitoring technology 2 telehealth is the delivery of health care from a distance, indicating a shift from the traditional clinical contexts towards patients and where they live (burrows, coyle & gooberman-hill, 2018). it can be cost effective and promote self-management especially to people with chronic illness (trajkovik et al., 2014). tele-monitoring systems allow active monitoring of patients’ health parameters, provide health recommendations and educational materials, as well as clinical interventions in emergency situations. these systems can be used as a viable mechanism supporting more independent life and reducing the mortality and hospitalization for patients with heart failure (ware, seto & ross, 2018). there are other telehealth systems such as teleconsultation between healthcare providers and teleconsultations between patients and healthcare providers (maarop & win 2012). the scope of this study is on telehealth systems between heart failure patients and the healthcare providers. the self-determination theory postulates of autonomy, competence and relatedness would provide the patients’ behaviour of acceptance towards the tele-monitoring technology as a health care tool for patients with congestive heart failure. the study will examine whether the tele-monitoring can add a value to the patients’ need for autonomy, competency and the relatedness by examining the role of tele-monitoring in strengthening patients confidence; assisting the needed behaviour change and facilitating communication between the patients and the health care providers. on the other hand, many health policies work differently with different health care systems based on their primary concerns such as economic, political and social issues (cheow & win, 2007). the health system in australia is strategically aligned with the australian safety and quality framework for health care (australian commission on safety and quality in health care, 2010) that specifies three core principles for safe and highquality care: consumer centred, driven by information, and organised for safety. this framework provides 21 areas for action that all people in the health system can take to improve the safety and quality of care provided in all health care settings. several areas for action are related to consumer-centred health care: develop methods and models to help patients get health services when they need them; increase health literacy; partner with patients, families and carers to share decision making about their care; provide care that respects and is sensitive to different cultures; and improve continuity of care. similarly, hospital quality and safety assurance in victoria (duckett, 2017) complements the australian safety and quality framework, while focusing mainly on quality assurance in the hospital environment. the objective of this systematic literature review is to understand the role of tele-monitoring in the patients’ management and improvement in the continuity of care. since the number of patients with chf is growing rapidly, tele-monitoring as a technology for monitoring and management of health parameters can greatly contribute towards the improvement of the quality of life for this vulnerable group of patients. thus, this study is focused on the telemonitoring as a technology for the management and self-management of patients with chf in the context of self-determination theory (sdt) and consumer-centred principle of the australian safety and quality framework for health care (asqfhc). 2 theoretical perspectives self-determination theory distinguishes between autonomous as intrinsic and wellinternalized extrinsic motivation, and controlled motivations as regulation by external factors (ryan and deci, 2000). based on these motivations, sdt introduces three postulates as basic psychological needs: competence, autonomy, and relatedness. the theory underscores the idea australasian journal of information systems vlahu-gjorgievska, nagapuri & win 2019, vol 23, research on health information systems monitoring using tele-monitoring technology 3 that people’s relationships and cultural and social contexts must involve and support their fundamental human needs based on these three postulates (legault, 2017). in the health context, maintenance of behaviour over time requires patients to internalize values and skills for change, and experience self-determination. therefore, the selfdetermination theory is focused on the processes through which a person acquires the motivation for initiating new health-related behaviours and maintaining them over time (ryan et al., 2008). sdt predicts that competence (as confidence and competence to change) alone is not sufficient to ensure adherence, it must be accompanied by autonomy. two forms of autonomous motivation are identified and integrated regulation. in identified regulation, the person identifies the value or importance of a behaviour or health practice; and in integrated regulation, the person aligns its behaviour with lifestyle patterns. according to sdt, relatedness is connected with the practitioner-patient relationship. thus, a sense of being respected, understood, and cared for is essential for forming the experiences of connection and trust (ryan et al., 2008). the findings in (ng et al., 2012) suggest that sdt can become a foundation for the development of interventions within health promotion and health care contexts. in this context, sdt principles can be included in delivering health care messages or as a guideline for more effective and more cost-effective interventions. murray et al. (2015) apply the sdt as a base theory for their communication skills training intervention – connect. results from the investigation of gunnell et al. (2014) point out that various components of the sdt could be targeted in order to increase wellbeing and physical activity behaviour. hancox et al. (2018) examined the process of instructors’ understanding and adopting, and the practicalities of operationalising specific sdt-informed motivational strategies within the particular context of indoor group cycling classes. 3 methodology the process of papers selection was based on the prisma approach (liberati et al., 2009). the reviewed papers were selected by searching relevant publications in three electronic databases: science direct, sage and pub med central. these databases contain relevant papers in the field of application of tele-monitoring systems published in high-impact journals. the search terms were: (“mobile phone applications” or “self-management”) and (“selfmonitoring” or “tele-monitoring” or “telehealth”) and (“heart failure” or “congestive heart failure”). the search on the databases was conducted in february 2017. the tele-monitoring systems are enabled by the telecommunications and information technology that have progressive developments in health care over the past several years. thus, we have limited our search on articles published from 2010 to 2017. also, the studies needed to meet the following criteria to be included in the review: • a peer-reviewed article written in english, • an article focused on a study or trial for patients with chf that use tele-monitoring systems. the review papers, conference papers, case reports, as well as the papers related to other health conditions, or health conditions in combination with chf were excluded. australasian journal of information systems vlahu-gjorgievska, nagapuri & win 2019, vol 23, research on health information systems monitoring using tele-monitoring technology 4 tele-monitoring, as an innovative technology for monitoring health parameters of the patients, is an emerging approach used in hospital and home environments all over the world. to have a broader picture of the systems and its impact on the patients with heart failure, we didn’t exclude papers based on the country where the study/studies were conducted. all the papers that met the aforementioned criteria for inclusion and exclusion, regardless of the country, were included in this report. the initial cross-database search resulted in 500 articles: science direct (125 papers), sage (73 papers) and pub med central (302 papers). from this set of articles, 342 articles were excluded based on the article title and abstract. the remaining 158 articles were assessed for eligibility by full-text and a further 147 articles were excluded based on the introduction, conclusion and full-text review as they didn’t provide sufficient information on telehealth monitoring for congestive heart failure. the resulting 11 articles were included in the review. the process of the selection of the papers (prisma flow diagram) is presented on figure 1. figure 1. overview of data extraction for selecting relevant papers 4 results in table 1 are summarized the journals and its impact factor of the papers and studies included in this systematic review. author journal impact factor black et al., 2014 trials performance and findings of randomised trials in health from bmc blum & gottlieb, 2014 journal of cardiac failure if. 3.994 boyne et al., 2011 international journal of nursing studies if 3.656 cichosz et al., 2016 trials performance and findings of randomised trials in health from bmc domingo et al., 2012 european journal of cardiovascular nursing if 2.651 australasian journal of information systems vlahu-gjorgievska, nagapuri & win 2019, vol 23, research on health information systems monitoring using tele-monitoring technology 5 author journal impact factor eilattsanani et al., 2015 european journal of cardiovascular nursing if 2.651 ferrante et al., 2010 journal of american college of cardiology if 16.834 kao et al., 2016 telemedicine and e-health if 2.165 ong et al., 2016 journal of american medical association if 47.7 stut et al., 2015 patient preference and adherence if 1.733 veenstra et al., 2015 netherlands health journal if 1.476 table 1. description of the included trials and studies 4.1 study demographics the studies in the eleven reviewed papers were conducted in eight different countries (usa, netherlands, denmark, spain, israel, argentina, united kingdom and germany). most of the studies were conducted in the usa (black et al, 2014; blum & gottlieb, 2014; kao et al., 2016; ong et al., 2016) and two were conducted in netherland (boyne et al., 2011; veenstra et al., 2015). multi-countries study (united kingdom, germany and spain) was presented in one of the papers (stut et al., 2015). the duration period of the studies varied from 3 months to 3 years. most of the studies used randomized control trials (black et al., 2014; blum & gottlieb, 2014; boyne et al., 2011; cichosz, ehlers & hejlesen, 2016; ferrante et al., 2010; ong et al., 2016), and others are pre and post-study without using control group (veenstra et al., 2015), prospective intervention study (domingo et al, 2012), service-evaluation retrospective study (eliat-tsanani et al., 2016), observational study (stut et al., 2015) and retrospective cohort study (kao et al., 2016). in all studies, tele-monitoring was performed in the home environment. demographic and other details of the studies covered in the reviewed papers are presented in table 2 and table 3. beat-hf (black et al., 2014; ong et al., 2016) is a multi-centres randomized control trial conducted for a period of 180 days to investigate the effectiveness of transition of care to patients. 1437 participants with an average age of 73 years were selected from six academic health care centres located in california. cichosz, ehlers & hejlesen (2016) conducted a randomized control trial for a period of 12 months in denmark. participants were selected from three health care centres by handing out the questionnaires that measure the quality of life based on baseline physical measurements and demographic characters. another service-evaluation retrospective study that lasts for 12 months, conducted by eilat-tsanani et al. (2016) in israel, investigated the change in the quality of life and re-hospitalisation rate of patients. veenstra et al. (2015) conducted an optimized care program with 102 patients in the netherlands. for two years telehealth interventions were used to evaluate the unplanned admissions, quality of life and disease-specific knowledge in the hospital. australasian journal of information systems vlahu-gjorgievska, nagapuri & win 2019, vol 23, research on health information systems monitoring using tele-monitoring technology 6 author study study name country study period black et al., 2014 randomized control trial beat hf california, usa 6 months (180 days) blum & gottlieb, 2014 randomized control trial n/a baltimore, washington, maryland, usa 26±14 months (802±430 days) boyne et al., 2011 randomised control trial tehaf netherlands 12 months cichosz, ehlers & hejlesen, 2016 randomised control trial telecare north heart failure trial denmark 12 months domingo et al., 2012 prospective intervention study carme spain 12 months eilat-tsanani et al., 2016 service-evaluation retrospective study tschf israel 12 months ferrante et al., 2010 randomised control trial dial argentina 16 months (57 months before / 41 months after) kao et al., 2016 retrospective cohort study hbp cms usa 3 years ong et al., 2016 randomised control trial beta-hf california, usa 6 months stut et al., 2015 observational study heart cycle united kingdom, germany and spain 3 to 18 months veenstra et al., 2015 pre and post-study program without using a control group n/a netherlands 12 months prephase and 12 months postphase table 2. description of the included trials and studies the study of kao et al. (2016) analyses the mortality rate and hospitalization. the authors are using the health buddy program (hbp) for monitoring hf patients’ demographic conditions, baseline characteristics, medicare beneficiaries (cost analysis) and retrospective cohort. the heart cycle study was conducted by stut et al. (2015) at three hospitals located in uk, germany and spain for a period of 3 to 18 months enrolling 123 patients. this study evaluated the hf patients’ adherence to self-care behaviour in telehealth system by educating and coaching the patients based on their knowledge towards the disease. blum & gottlieb (2014) conducted a randomized trial at the medical centre of the university of maryland, baltimore veterans administration medical centre and other private practices in order to identify the quality of life, change in the frequency of re-hospitalisation rate and costs. tehaf is another randomized control trial with 390 patients (boyne et al., 2011) for a period of 12 months at 3 medical centres located in the netherlands. catalan remote management evaluation (carme) study (domingo et al., 2012) was conducted for 12 months at university hospital in spain. the aim of the study was to analyse australasian journal of information systems vlahu-gjorgievska, nagapuri & win 2019, vol 23, research on health information systems monitoring using tele-monitoring technology 7 the improvement of the quality of life of hf patients by using motiva healthcare system and supporting educational motivating tools. dial study, presented by (ferrante et al., 2010) is a randomized trial conducted at 51 centres in argentina. the objective of this intervention was to support patients’ health behaviours such as dietary, change in physical activity and medication adherence using telecommunication technology. author device /system monitor/follow up sample size data analysis black et al., 2014 ideal life body-manager, bpmanager, ideal life pod nurses, cardiologists 1500 patients multivariate regression analysis blum & gottlieb, 2014 philips electronic ecare system nurses, cardiologists 204 patients t-test, chi-square test, mann whitney u test boyne et al., 2011 health buddy nurses, cardiologists 390 patients t-test, man(c)ova cichosz, ehlers & hejlesen, 2016 telekit tablet connected to two external devices: (1)digital blood pressure monitor, and (2)precision health scale clinicians, nurses 316 patients t-test domingo et al., 2012 motiva system & motiva plus (philips) nurses, doctors 97 patients chi-square test, ttest, wilcoxon test eilattsanani et al., 2016 telemedicine system with tele weight scale nurses, general practitioners, cardiologists 141 patients t-test, regression analysis ferrante et al., 2010 telephone interventions nurses, cardiologists 1518 patients pearson chi-square, t-test kao et al., 2016 health buddy nurses, physicians, physician assistants 623 patients (in each group) wilcoxon test, chisquare test ong et al., 2016 equipment supplied by ideal life nurses out 1437 participants multivariable analysis stut et al., 2015 motiva telehealth system (philips) nurses 123 patients t-test veenstra et al., 2015 motiva telehealth system (philips) nurses, cardiologists 102 patients analysis of variance anova table 3. tele-monitoring systems, sample size and statistics used in the reviewed studies australasian journal of information systems vlahu-gjorgievska, nagapuri & win 2019, vol 23, research on health information systems monitoring using tele-monitoring technology 8 most of the studies categorise the patients using the new york heart association (nyha) functional classification. nyha classification focuses on the exercise capacity of the patient and the symptomatic status of the disease. based on the nyha classification system, the estimation of hf severity can be classified as a stage two, three, or four, allowing the experts to adjust the treatment of the patient and minimize the risk of adverse events (tripoliti et al., 2017). eight papers (boyne et al., 2011; cichosz, ehlers & hejlesen, 2016; domingo et al., 2012; eliattsanani et al., 2016; ferrante et al., 2010; ong et al., 2016; stut et al., 2015; veenstra et al., 2015 ) included patients suffering from nyha ii, iii or iv. in the heart cycle study (stut et al., 2015) the mean age of the patients is above 50 years and 66 % of the patients were classified as nyha iii. in the tehaf and carme studies, the patients were in the category of nyha iiiv. in the study of veenstra et al. (2015), most of the patients were diagnosed with nyha iiiv, and in the beat-hf study (ong et al., 2016) 61.2% of the patients were under the category of nyha iii-iv. 4.2 cultural background of the participants in the reviewed studies in six papers (black et al., 2014; blum & gottlieb, 2014; stut et al., 2015; eliat-tsanani et al., 2016; cichosz, ehlers & hejlesen, 2016; ferrante et al., 2010) basic priority is the ability of the patient to handle the electronic devices. in addition, other five papers (black et al., 2014; stut et al., 2015; kao et al., 2016; domingo et al., 2012; cichosz, ehlers & hejlesen, 2016) considered patients’ socio-economic and cultural factors. in the study of cichosz, ehlers & hejlesen (2016), only patients who can communicate in danish language and be able to handle landline or mobile phone by themselves or have carers that can translate them about the use of telehealth system were included in the study. in beat-hf study (black et al., 2014; ong et al., 2016), patients speaking english, spanish, farsi or russian were included. in tehaf study (boyne et al., 2011) only patients able to visit any of the 3 medical centres and can communicate in dutch were included. in carme study (domingo et al., 2012) all patients aged above 18 years, having a television and understanding and being able to adequately carry out self-monitoring were included. 4.3 telehealthcare approach cichosz, ehlers & hejlesen (2016) are using telekit system comprised of samsung galaxy tab 2 tablet and two external devices (digital blood pressure monitor and scale) wirelessly connected to transmit the data to the tablet. the tablet collects data from the external devices and disease-specific questionnaires, and transmit it to the central clinic system. the telekit sends the patients a reminder and an alarm to note the baseline measurements (blood pressure, pulse rate, and weight) and fill the questionnaires on health-related quality of life. the measurements (blood pressure, pulse, and weight) and symptom-specific information (from the questionnaire) are assessed by a nurse or clinician at fixed intervals. if a patient’s measurements are not received as expected or clinician observes abnormal measurements the patient will be contacted with further instructions. telemedicine system used in tschf study (eliat-tsanani et al., 2016) comprises of tele-weight scale and electronic medical record (emr). a central communication module automatically transmits the patient’s measurements (daily weight) to the emr. the hospital administration gets the notification when the patient’s baseline value triggers, so the medical staff can immediately contact the patients for further preventive measures. australasian journal of information systems vlahu-gjorgievska, nagapuri & win 2019, vol 23, research on health information systems monitoring using tele-monitoring technology 9 blum & gottlieb (2014) uses the philips electronic e-care system to monitor and record the patient’s data. data is encrypted and wirelessly transmitted by the hub. when received, the data is decrypted by the server software and assigned to the patient’s database file. nurses compare the readings of the individual hf patients and in emergency conditions consult cardiologists for further treatment compliance. 4.4 telehealthcare approach including health literacy health buddy is an electronic device with a high-resolution screen and four large buttons located in patients’ homes and connected to remote care managers. the health buddy asks patients for vital signs and other health-related information on a daily basis, providing feedback and education based on responses (kao et al., 2016; boyne et al., 2011). health buddy as part of the health buddy program, not only monitors the patient’s vital signs but also educate patients about their disease condition. the questionnaires are structured in order to improve patient’s disease-specific knowledge, symptoms and behaviour. the clinical staffs provide feedback on patient’s responses and categorize patient’s symptoms as a low, medium and high risk (needed further attention). in tehaf study (boyne et al., 2011), the health buddy device was used in dutch (translated by the distribution company) and followed by nurses and doctors according to the european guidelines. four programs focussed on self-educating and self-managing of hf patients were conducted. every program had a different level of monitoring symptoms and education. the beat-hf study (black et al., 2014 and ong et al., 2016) uses the ideal life equipment (ideal life pod, a bluetooth-enabled wireless gateway, the ideal life body-manager and the ideal life bp-manager). patients are using the devices every day to transmit the data (their weight, blood pressure, heart rate, and responses to 3 symptom questions) that is accessed by the nurses. the nurses contact the patients thru telephone calls in a regular time period or when readings exceed predetermined threshold variables. the beat-hf study besides the regular tele-monitoring, includes education of pre-discharge patients. study nurse educates patients and their family members or caretakers regarding the disease, symptoms, diet, importance of monitoring health parameters, as well as how to use the tele-monitoring equipment. motiva telehealth system is used in the studies of stut et al. (2015) and veenstra et al. (2015). this system consists of automated devices for regular measurement of blood pressure, heart rate and weight, and broadband home tv channel which is used to provide patients with educational videos, gentle reminders on medications, surveys on health-related issues and motivational messages for patients’ encouragement. the clinicians monitor the patient’s basic measurements and contact the patients that require high attention. in carme study (domingo et al., 2012) is also used a version of motiva system, but patients were divided into 2 groups. patients in group a were trained with educational videos and motivational messages using motiva system, whereas patients in group b were additionally using self-monitoring tools (motiva plus). the educational videos on different topics related to symptoms, medications, lifestyle, disease conditions, diet, and prevention were played in the spanish language. 4.5 improve continuity of care in the study of eilat-tsanani et al. (2016) the quality of life was measured by minnesota living with heart failure questionnaire. the change in the scores of the results indicated an australasian journal of information systems vlahu-gjorgievska, nagapuri & win 2019, vol 23, research on health information systems monitoring using tele-monitoring technology 10 improvement in the quality of life of the patients and a decrease in the rate of hospitalization. the beat-hf study (black et al., 2014; ong et al., 2016) is also using the minnesota living with heart failure questionnaire to evaluate the readmission rate and patients’ quality of life. according to the results, even though there was no significant difference in the readmission rate, a significant difference in improvement of the quality of life score (at 180-day) was recorded between the intervention and the control participants. the results from veenstra et al. (2015) study showed that after telehealth enrolment there were only 32 patients with unplanned hf admissions, and the number of days spent in a hospital was reduced from 975 to 662 days. the quality of life is measured by minnesota living with heart failure questionnaire showed that the quality of life of the patient was improved with statistical significance starting from 24 weeks after the enrolment in the program. in heart cycle study (stut et al., 2015), the self-care behaviours like daily weighing, fluid restriction, low salt diet, medication, and physical activity were assessed using european heart failure self-care behaviour scale. the results showed that all self-care behaviours except medication intake, self-reported, and self-care behaviour scores have improved significantly at the end of the study comparing to the beginning. the results of the carme study (domingo et al., 2012) showed that telemedicine program led to a significant reduction in hospital admissions and in the number of days in the hospital for heart failure and also other cardiac causes, as well as to a perception of better quality of life for the patients using both systems (motiva and motiva plus+). compliance is another factor which is measured for the intervention group in the dial study (ferrante et al., 2010). during the first 45 days of the trial, there was an improvement in medication compliance for 83% patients, 40.7% patients diet compliance and 34.9% in weight compliance. in general, there is a high degree of satisfaction with the tele-monitoring tools (domingo et al., 2012) and the tele-monitoring programs (tools combined with automated education and coaching program) can influence behavioural changes in improving self-care (stut et al, 2015), patient compliance and adherence (eliat-tsanani et al., 2016, ferrante et al., 2010), as well as improve survival and reduce hospitalization of the patients (kao et al., 2016, domingo et al., 2012; veenstra et al., 2015). also, need to be mentioned that the results in the study of blum & gottlieb (2014) study indicated that although the 30 days readmission rate for hospitalisation is decreased in the first year, there was no significant change in symptom improvement, mortality, total costs and hospitalizations. 5 discussion this review emphasizes the significance of including patients from different cultural backgrounds in studies and trials, and the importance for patients to understand spoken language and used technology. most of the studies include only the patients that speak a particular language, and in some of them, the requirement is the patients to be familiar with the used technology. this aligns with the asqfhc area of action related to the cultural background. provided review supports the assumption that (pre-prepared) information translated into languages (understandable for patients) can provide effective and respectful communication between patients and health care workers. australasian journal of information systems vlahu-gjorgievska, nagapuri & win 2019, vol 23, research on health information systems monitoring using tele-monitoring technology 11 considering the consumer-centred principle of asqfhc related to the developing methods and models to help patients get health services when they need them, tele-monitoring is one approach that can ease the communication between the patients and health care professionals and provide greater access to the health services. besides the communication and access to services, reviewed tele-monitoring systems offer educational materials and provide information regarding the health condition of the patients. thus, can be as well used as a health literacy tool. based on the asqfhc, improving the continuity of care has a positive effect on the care provided to the patients. there are several examples in the presented literature review that measure the improvement of the quality of life of the monitored patients. according to these studies, increasing the quality of life is one of the major benefits of tele-monitoring systems. additionally, monitoring and qualitative analysis (framing questions and collecting user views) improve patient’s self-care and self-confidence, as well as the compliance with the diet, exercises regime and medication adherence. the review also indicated that tele-monitoring systems allow patients to track their baseline measurements and identify the importance of self-care and self-compliance, support and advance the practitioner-patient relationship, provide information regarding the disease’s symptoms and management thus strengthen the patients’ competence and increase their confidence. 6 conclusion the most notable gaps found in this review were patients’ adoption and usage of technology, improper training and insufficient communication between clinicians and patients. also, challenges like socio-economic status and low health literacy can additionally contribute towards poor congestive heart failure conditions and health care utilization. the health care providers can contribute in bridging these gaps, supporting the patients’ participation in the health care by explaining the functionalities and outcomes, as well as the health literacy goals of the tele-monitoring systems. based on the literature review conducted in this paper, tele-monitoring systems support patients’ autonomy, competence and relatedness, and empower the patients’ self-care promoting motivation, education and self-management. thus provide consumer-centred health care, oriented towards patients' safety and improved quality of life. the australian safety and quality framework for health care describes a vision for safe and high-quality care for all australians and sets out the actions needed to achieve this vision. although the papers reviewed are studies conducted outside australia, we can conclude that the framework can be applied in the usage of the tele-monitoring technology for the management of patients with congestive heart failure. references australian commission on safety and quality in health care. (2010). australian safety and quality framework for health care. retrieved from http://www.safetyandquality.gov.au/national-priorities/australian-safety-and-qualityframework-for-health-care. australasian journal of information systems vlahu-gjorgievska, nagapuri & win 2019, vol 23, research on health information systems monitoring using tele-monitoring technology 12 black, j. t., romano, p. s., sadeghi, b., auerbach, a. d., ganiats, t. g., greenfield, s., ... & ong, m. k. (2014). a remote monitoring and telephone nurse coaching intervention to reduce readmissions among patients with heart failure: study protocol for the better effectiveness after transition-heart failure (beat-hf) randomized controlled trial. trials, 15(1), 124. blum, k., & gottlieb, s. s. (2014). the effect of a randomized trial of home telemonitoring on medical costs, 30-day readmissions, mortality, and health-related quality of life in a cohort of community-dwelling heart failure patients. journal of cardiac failure, 20(7), 513521. boyne, j. j., vrijhoef, h. j., de wit, r., & gorgels, a. p. (2011). telemonitoring in patients with heart failure, the tehaf study: study protocol of an ongoing prospective randomised trial. international journal of nursing studies, 48(1), 94-99. burrows, a., coyle, d., & gooberman-hill, r. (2018). privacy, boundaries and smart homes for health: an ethnographic study. health & place, 50, 112-118. cheow, m. f., & win, k. t. (2007). personal health record and healthcare systems. international journal of healthcare technology and management, 8(3-4), 209-225. cichosz, s. l., ehlers, l. h., & hejlesen, o. (2016). health effectiveness and cost-effectiveness of telehealthcare for heart failure: study protocol for a randomized controlled trial. trials, 17(1), 590. domingo, m., lupón, j., gonzález, b., crespo, e., lópez, r., ramos, a., & bayes-genis, a. (2012). evaluation of a telemedicine system for heart failure patients: feasibility, acceptance rate, satisfaction and changes in patient behavior: results from the carme (catalan remote management evaluation) study. european journal of cardiovascular nursing, 11(4), 410-418. duckett, s. (2017). review of hospital safety and quality assurance in victoria. retrieved from https://www2.health.vic.gov.au/about/publications/researchandreports/report-of-thereview-of-hospital-safety-and-quality-assurance-in-victoria eilat-tsanani, s., golovner, m., marcus, o., dayan, m., sade, z., iktelat, a., ... & oppenheimer, y. (2016). evaluation of telehealth service for patients with congestive heart failure in the north of israel. european journal of cardiovascular nursing, 15(3), e78-e84. ferrante, d., varini, s., macchia, a., soifer, s., badra, r., nul, d., ... & gesica investigators. (2010). long-term results after a telephone intervention in chronic heart failure: dial (randomized trial of phone intervention in chronic heart failure) follow-up. journal of the american college of cardiology, 56(5), 372-378. gunnell, k.e., crocker, p.r., mack, d.e., wilson, p.m. and zumbo, b.d., 2014. goal contents, motivation, psychological need satisfaction, well-being and physical activity: a test of self-determination theory over 6 months. psychology of sport and exercise, 15(1), pp.19-29. hancox, j. e., quested, e., ntoumanis, n., & thøgersen-ntoumani, c. (2018). putting selfdetermination theory into practice: application of adaptive motivational principles in the exercise domain. qualitative research in sport, exercise and health, 10(1), 75-91. kao, d. p., lindenfeld, j., macaulay, d., birnbaum, h. g., jarvis, j. l., desai, u. s., & page, r. l. (2016). impact of a telehealth and care management program on all-cause mortality australasian journal of information systems vlahu-gjorgievska, nagapuri & win 2019, vol 23, research on health information systems monitoring using tele-monitoring technology 13 and healthcare utilization in patients with heart failure. telemedicine and e-health, 22(1), 2-11. legault, l. (2017). self-determination theory. encyclopedia of personality and individual differences, 1-9. liberati, a., altman, d. g., tetzlaff, j., mulrow, c., gøtzsche, p. c., ioannidis, j. p., ... & moher, d. (2009). the prisma statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. plos medicine, 6(7), e1000100. maarop n., & win k.t. (2012). understanding the need of healthcare providers for teleconsultation and technology attributes in relation to the acceptance of teleconsultation in malaysia: a mixed methods study, journal of medical systems, 36(5) , 2881-2892, murray, a., hall, a. m., williams, g. c., mcdonough, s. m., ntoumanis, n., taylor, i. m., ... & lonsdale, c. (2015). effect of a self-determination theory–based communication skills training program on physiotherapists' psychological support for their patients with chronic low back pain: a randomized controlled trial. archives of physical medicine and rehabilitation, 96(5), 809-816. ng, j. y., ntoumanis, n., thøgersen-ntoumani, c., deci, e. l., ryan, r. m., duda, j. l., & williams, g. c. (2012). self-determination theory applied to health contexts: a metaanalysis. perspectives on psychological science, 7(4), 325-340. ong, m. k., romano, p. s., edgington, s., aronow, h. u., auerbach, a. d., black, j. t., ... & ganiats, t. g. (2016). effectiveness of remote patient monitoring after discharge of hospitalized patients with heart failure: the better effectiveness after transition–heart failure (beat-hf) randomized clinical trial. jama internal medicine, 176(3), 310-318. ponikowski, p., anker, s. d., alhabib, k. f., cowie, m. r., force, t. l., hu, s., ... & samal, u. c. (2014). heart failure: preventing disease and death worldwide. esc heart failure, 1(1), 4-25. ryan, r. m., & deci, e. l. (2000). self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. american psychologist, 55(1), 68. ryan, r. m., patrick, h., deci, e. l., & williams, g. c. (2008). facilitating health behaviour change and its maintenance: interventions based on self-determination theory. european health psychologist, 10(1), 2-5. stut, w., deighan, c., cleland, j. g., & jaarsma, t. (2015). adherence to self-care in patients with heart failure in the heartcycle study. patient preference and adherence, 9, 1195. trajkovik, v., koceski, s., vlahu-gjorgievska, e., & kulev, i. (2014). collaborative health-care system (cohesy) model. in s. adibi (ed.), mhealth multidisciplinary verticals (pp. 123140). crc press. tripoliti, e. e., papadopoulos, t. g., karanasiou, g. s., kalatzis, f. g., bechlioulis, a., goletsis, y., ... & fotiadis, d. i. (2017). estimation of new york heart association class in heart failure patients based on machine learning techniques. in ieee embs international conference on biomedical & health informatics, 421-424. ieee. australasian journal of information systems vlahu-gjorgievska, nagapuri & win 2019, vol 23, research on health information systems monitoring using tele-monitoring technology 14 veenstra, w., op den buijs, j., pauws, s., westerterp, m., & nagelsmit, m. (2015). clinical effects of an optimised care program with telehealth in heart failure patients in a community hospital in the netherlands. netherlands heart journal, 23(6), 334-340. velagaleti, r. s., & vasan, r. s. (2007). heart failure in the twenty-first century: is it a coronary artery disease or hypertension problem? cardiology clinics, 25(4), 487-495. ware, p., seto, e., & ross, h. j. (2018). accounting for complexity in home telemonitoring: a need for context-centred evidence. canadian journal of cardiology, 34(7), 897-904. zipes, d. p., libby, p., bonow, r. o., mann, d. l., & tomaselli, g. f. (2018). braunwald's heart disease e-book: a textbook of cardiovascular medicine. elsevier health sciences. copyright: © 2019 vlahu-gjorgievska, nagapuri & win. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 theoretical perspectives 3 methodology 4 results 4.1 study demographics 4.2 cultural background of the participants in the reviewed studies 4.3 telehealthcare approach 4.4 telehealthcare approach including health literacy 4.5 improve continuity of care 5 discussion 6 conclusion paper title (paper title style) australasian journal of information systems coulthard & keller 2016, vol 20, research article publication, journal rankings, researcher views 1 publication anxiety, quality, and journal rankings: researcher views darryl coulthard la trobe university d.coulthard@latrobe.edu.au susan keller la trobe university abstract journal ranking systems are increasingly used to measure research performance of academics and universities. a growing number of academic commentators have voiced concerns of possible undesirable outcomes such as increased publication anxiety and an increase in safe and conforming research, but there have been few empirical studies on the possible effects. to address this gap, we surveyed information systems (is) academics who published in one of three key is conferences in 2013, to gather their views of the effects of journal ranking systems. overall, we found that the concerns in the literature were strongly reflected in the views of those surveyed. academics believe the system has greatly increased their publication anxiety. while most believed that the quality of published research had improved, researchers believe the ranking systems inhibit innovative, risky research, and encourages safe, conforming, mainstream research. keywords: journal rankings; quality; publication anxiety; survey 1 introduction the rise of neoliberalism and the emphasis on user pays by governments in most developed countries has resulted in reduced funding to public universities over the past 25 years (bourdieu, 1998; marginson, 2011). encouraged by governments, universities have developed market-driven business models and responded to increased pressure to provide quantifiable and marketable evidence of the quality of educational and research outcomes (hammarfelt & de rijcke, 2014; kinman, 2014). simple metrics such as student satisfaction surveys, research quantum, and citation counts are in demand by government agencies, university marketers and student 'consumers' to demonstrate ‘value for money’, market success and brand differentiation and also feed local and international league tables designed to rank institutions and promote consumer choice and bragging rights (marginson, 2011). while the validity and veracity of these metrics are questionable, universities nevertheless may have little option but to participate as they compete in a global education market for full fee-paying students and are subject to government requirements (harvey, 2008). in such as climate, it is unsurprising that the use and importance of journal rankings schemes have increased since they provide a ready proxy for government agencies, funding bodies and universities to determine the quality of research outputs. such schemes may be convenient for monitoring research performance but they may also be leading to undesirable consequences. there are very few empirical studies, however a growing number of academic commentators have voiced concerns on the adverse effects of journal ranking systems (rijcke, wouters, rushforth, franssen, & hammarfelt, 2015). these concerns include: the reduction of research in innovative areas and cross disciplinary work; a narrowing of research methods; encouragement of orthodoxy; and a reduction in research dealing with big questions (laudel & gläser, 2014; lincoln, 2012; mingers & willmott, 2013). this may be particularly concerning for information systems as it grapples to understand the rapid changes in information technology and its uses and outcomes in contemporary social, organizational and business activities (cooper & zmud, 1990; grover & lyytinen, 2015). such technological changes and responses may fall outside well-established research programs and australasian journal of information systems coulthard & keller 2016, vol 20, research article publication, journal rankings, researcher views 2 ‘epistemic middle range scripts’ identified by grover and lyytinen (2015) in information systems and may require new innovative research programs, concepts and methods. such innovative programs may necessarily depart from established research paths and the ‘comfort zone’ of is scholars (grover and lyytinen, 2015) and are as a consequence more likely to have poorer, risky publication prospects. in effect, by rewarding orthodox and well-established research and potentially devaluing new research programs and approaches (mingers & willmott 2013), the discipline will not only restrict the knowledge it produces, but also its relevance since it will be unwilling to engage in the innovative research topics and methods needed to address important questions in a rapidly changing global environment. in addition to possible undesirable effects on research from a discipline perspective, there is some suggestion of undesirable effects at the individual researcher level too. a growing body of research suggests that academic staff are suffering increased work stress and low levels of psychological health (kinman, 2014). the use of journal ranking schemes to measure and monitor research output may be adding to such stress and contributing to highly skilled academics leaving higher education institutions (kinman & jones, 2003). universities may well have difficulties recruiting bright graduates and retaining academics. given the importance of university research to the economic, intellectual, cultural and social development of a country (rossi & geuna, 2015, pp. 165-166), any changes that lead to a contraction of research freedoms and loss of research staff would have very serious ramifications and thus warrants further investigation. in sum, there are two areas of concern with the rise of journal ranking systemsits effects on research and the effect on researchers. while academic opinion expressed in journals is overwhelmingly negative concerning these effects, there has been little if any empirical research into the effect on research and the welfare of researchers, and none that we could identify in the information system area. the purpose of this paper then is to investigate empirically is researchers’ views on the effects of journal ranking systems on both is research and is researchers. to this end, we surveyed information systems academics who published in three key is conferences in 2013, to gather their views on the effects of journal ranking systems. overall, we found that the concerns in the literature were strongly reflected in the views of those surveyed. academics believe the system has greatly increased their publication anxiety. while most believed that the quality of published research had improved, researchers believe the ranking systems inhibit innovative, risky research, and encourages safe, conforming, mainstream research. the remainder of the paper is structured as follows. sections two and three discuss the possible impact of the use of journal ranking systems on the discipline and the individual researcher. we then describe the empirical study and present the findings in sections four and five. this is followed by a discussion of the findings, and finally we conclude the paper by suggesting further research and a stronger role for the is discipline leadership. 2 possible consequences on research 2.1 the potential effects of ranking on disciplinary research mingers and willmott (2013) point out most academic work is now shaped by performance measures. it seems highly improbable that the measurement of research will have no appreciable unintended effect on research itself (van dalen & henkens, 2012). indeed, there are two possible opposing effects. the first possible effect is that ranking will have a positive, salutary effect where competition and measurement breeds and rewards excellence: the top journals will demonstrate exemplary research and rigour that direct junior researchers to the most important areas of research and the methods that are best used to research them. the second is that the ranking systems will result in undesirable effects on research directions, methods and topics as researchers pursue outcomes that are measured rather than outcomes that are desirable from a discipline perspective. in other words, they will pursue whatever is australasian journal of information systems coulthard & keller 2016, vol 20, research article publication, journal rankings, researcher views 3 required to publish in high ranking journals but not necessarily pursue high quality innovative research. the pivotal question then is whether the journal ranking systems can promote innovative, ground-breaking research in important areas when the driving force underpinning how the ranking system operates is the use of the system for promotion, tenure, and recruitment and the marketing and advancement of the university. the answer appears gloomy. published opinion uniformly warn that journal rankings and in particular their coupling to performance management will harm research and in particular will stymie innovation and heighten conformity and ‘incrementalist’ research within the disciplines (alvesson & sandberg, 2013; lincoln, 2012; macdonald & kam, 2007; ozbilgin, 2009; powell & woerndl, 2008; tourish, 2015; van dalen & henkens, 2012). the key allegations of harmful outcomes to research include: the end of important but risky or time consuming research; incrementalism and the promotion of orthodoxy, uniformity and reduction of diversity; the reduction of qualitative research; and fewer regional and interdisciplinary studies. arguably the most serious allegations are the ‘end of important work’ (powell & woerndl, 2008; tourish, 2015) and the rise of ‘incrementalism’ (alvesson & sandberg, 2013; lincoln, 2012). essentially, the argument is that academics will begin to research areas that they think are publishable, less risky, with good publishable returns for effort rather than in areas that are theoretically important but may be difficult, time consuming, or risky. the fear is that academics will be encouraged to undertake a form of cost benefit analysis for their career advancement and choose areas and methodologies that are least risky and most likely to be published for the least effort. ‘smart’ research may be to ride on the coat tails of mainstream trends and prevailing orthodoxies, rather than contest these orthodoxies and risk not finding a high ranking publication outlet (macdonald & kam, 2007). on the other hand, pursuing new ideas or difficult but important areas is high risk and more likely to be full of ‘cul de sacs’ (lincoln, 2012). in sum, there is a real fear that blue sky research will be eschewed in favour of risk averse, incremental, orthodox work. as hassan (2014) quipped, "after we have had the satisfaction of publishing in misq or isr, and at least after making tenure and full professor, it is time to start thinking about researching big ideas that make a difference". the problem though is that this may be too late. the full professor, trained in, compliant with and rewarded for his or her orthodoxy may be either unwilling or unable to undertake blue sky research. grover and lyytinen (2015) in their recent and important work on “epistemic scripts” demonstrate an already existing tendency in information systems towards orthodox, incremental research and the publishing rewards that are attached to such research. epistemic scripts according to grover and lyytinen refers to a ‘middle range theory’ that appropriates an abstract concept from reference or base disciplines and seeks an instance of the concept in an is related context. this establishes a form of institutionalisation of middle range theories or scripts such that subsequent scholars tinker with the theory and produce a vast body of incremental research that produces publishable work and career security. following the script, scholars mostly produce knowledge that seeks to get through reviewers looking to check boxes on theory and method. the name of the game is conformity with the received theoretical insights, and the truth is largely determined by the work’s coherence with received theory and less by its novel insight, value, or utility for the field. overall, examples of how to enact this script with dexterity are not hard to find; they are commonly taught as part of core ph.d. courses in is curricula. (grover & lyytinen, 2015, p. 275) the history of tam research epitomises the concept of the script (benbasat & barki, 2007; grover & lyytinen, 2015). the rise of journal rankings may exacerbate this tendency within the discipline. by virtue of the rankings systems, the importance of the top journals in terms of determining the directions, methods, and topics in a discipline is increased while the importance or impact australasian journal of information systems coulthard & keller 2016, vol 20, research article publication, journal rankings, researcher views 4 of lower ranked journals is reduced and thus ranking systems may act to reduce the diversity of journals and promote orthodoxy (tourish, 2015). for example, macdonald and kam (2007) warn that top journals in management studies encourage ‘homogenous research’ that ‘can impose a deadening uniformity on its field of endeavour’ (harley and lee, 1997, p. 1434 cited in macdonald & kam 2007 p. 647). lincoln (2012) has also expressed deep concern that qualitative work will suffer as a consequence of journal ranking systems since qualitative work is generally more researcher labour intensive than other methods and has fewer higher ranked outlets. there is some empirical support for a reduction in interpretivism in information systems. in our study of three key journals, we found a twelve percent decline in interpretivist research between 20042006 and 2007-2009 and a corresponding fifteen percent increase in positivist studies (keller & coulthard, 2013). others too have argued that under-resourced areas will fail to get the attention they deserve as researchers pursue better resourced areas published in the top journals (powell & woerndl, 2008). as early as 2004, chen and hirschheim expressed the fear that publication pressure would lead to a reduction in methodological pluralism and less meaningful research in information systems since ‘under the pressure of tenure and promotion, researchers would tend to choose a less time consuming approach positivism to avoid perishing’ (p. 225). methodological pluralism and research diversity are critical for information systems. information technology and our relations to technology are constantly changing and developing and we must adapt our methodological approaches and research problems accordingly. however, on pragmatic career grounds it may become tempting to simply adopt the current orthodoxy and research problems. it is one thing to advise a candidate to study a safe topic with an orthodox method for their phd but quite another to advise an academic to do the same until they ‘reach full professorship’. it may be the case that canny scholars have always followed fashion and were incremental in their research rather than iconoclastic and heterodox. kuhn invented a term for this: ‘normal science’ (kuhn, 1970). however, the rise of formal ranking systems and associated managerial setting of targets and performance monitoring, make these pressures more pointed, the possible publishing outlets more limited, and the competition for high ranked publications greater. the stakes of the research ‘game’ have increased substantially. a further allegation is that regional studies may suffer in the competition for top journals. is has been accused of having a north american bias (gallivan & benbunan-fich, 2007) and that north american authors predominate in the top journals (truex, cuellar, & takeda, 2009 , keller & coulthard, 2013). indeed, tourish (2015) has identified that 31 out of the top 33 journals in the rankings produced by the association of business schools in the uk were of us origin. research in information systems is highly contextual. a study of an implementation failure in bangladesh is arguably less interesting to the top western journals, and the majority of its readership, than one in sydney but even that may pale in interest to a failure in manhattan. in other disciplines at least, there is evidence that citations decrease with a reference to a specific country in the title (jacques & sebire, 2010). cross-disciplinary work and international collaboration may, in a similar fashion, also suffer (mcnay, 1998; rafols, leydesdorff, o’hare, nightingale, & stirling, 2012). the problem stems from the ranking and recognition of particular journals within disciplines, institutes, and countries. for instance, if an is researcher wished to collaborate with an academic in the health sector, a journal would need to be found that ‘counted’ across both disciplines. if the collaboration takes place across institutions or across international borders the search for acceptable outlets becomes even more difficult (hussain, liu, wang, & zuo, 2015). for information systems, this is particularly concerning since information and technology now underpin “most areas of both research and social life,[so] confronting the problématiques of the new century may require a more transdisciplinary perspective on the study of information and systems” (hovorka, 2010, p. 12). australasian journal of information systems coulthard & keller 2016, vol 20, research article publication, journal rankings, researcher views 5 macdonald and kam (2007) point out that journal ranking and performance monitoring systems are also likely to place increased pressure on academics to publish in top listed journals – and undervalue, ignore or even deride lesser-ranked journals. this emphasis limits publishing outlets and increases competition among academics for publishing in higher ranked journals and effectively reduces diversity of publication and research areas, since academics are ‘encouraged to write the sort of papers and perform the sort of research these journals favour’ (macdonald & kam, 2007, p. 647). journal ranking systems may therefore inadvertently increase the importance of the top journals in terms of determining the directions, methods, topics, and research thus defining research orthodoxy. this could be exemplary but as we have seen, many argue that it can also be overly conservative and restrictive. often ground breaking work comes from younger researchers publishing in newer, and therefore lesser ranked journals, that may be among other things nonconformist, exploring new, less established, and emerging research problems not (yet) recognised by the highly ranked journals. this potentially fertile field of innovation and perhaps less polished talent is discouraged through low rankings. again, in a frontier discipline such as information systems that is particularly concerned with the application of innovations in technology this may be particularly detrimental. finally, journal ranking systems may also act to reduce the diversity of journals by effectively providing barriers to entry of new journals and neutralising the threat of disintermediation of established publishing houses. the internet threatens journals with disintermediation through self or school/university publication or at the very least fierce competition. scholars could use the web, informal peer networks and social media to find articles of interest and weigh and test their value according to their scientific viewpoint, interests, and specialisations. however, with journal rankings, where something is published has become critical for the researcher (lamp, 2009). this provides a barrier against disintermediation, and a marketing opportunity for established commercial publishers to trade on the prestige that the rankings systems provide them. indeed, the top journals will themselves be under increased pressure to maintain their high status and may even restrict papers to increase scarcity. journals are increasingly branding, promoting, and reporting on the impact of their journals on promotion and tenure (janvrin, lim, & peters, 2015). as nobel prize winner, randy schekman angrily exclaimed of the most highly ranked science journals: these journals aggressively curate their brands, in ways more conducive to selling subscriptions than to stimulating the most important research. like fashion designers who create limited-edition handbags or suits, they know scarcity stokes demand, so they artificially restrict the number of papers they accept (schekman, 2013). others have accused journals of developing dual submission policies – inviting articles from highly cited authors to ensure high citation of the journal while a stringent review process ensures high failure rates for other authors (alvesson & sandberg, 2013). journals have also been accused of internal and coercive citation, that is, forcing or otherwise encouraging authors to cite their journal and reviewers (butler & spoelstra, 2015; wilhite & fong, 2012). moreover, lower ranked journals may also embark on a form of ‘sanskritisation’ – emulating the topics, methods, and rigour of the top journals to maintain authors and readers; a good thing from the point of maintaining standards but bad from the point of view of diversity. similarly, researchers will be doing the same as they compete for publication space with each other in the top journals. there may be something of an ‘arms race’ between researchers for publication. while innovation may be put to one side, increased resources – more extensive literature reviews, larger sample sizes, and so on – may come to the fore and more polished, extensive and better-resourced research from established researchers may be increasingly published in the face of competition. 3 possible consequences on the individual researcher having detailed possible unintended consequences of journal ranking systems on the discipline we now turn our attention to possible impacts on the individual is researcher. how australasian journal of information systems coulthard & keller 2016, vol 20, research article publication, journal rankings, researcher views 6 the researcher is affected or believes he or she will be affected by the rankings will in turn affect the outcome of the journal ranking system as well as having an impact on the academics themselves. academics have always had an incentive to publish and experienced a form of ‘publication anxiety’. the catchphrase ‘publish or perish’ encapsulates this anxiety and incentive. we have seen the argument that the increased incentives to publish afforded by the new journal ranking systems may negatively affect the content of research. the question now becomes one as to whether the journal ranking systems produces additional stress and anxiety rather than an additional positive incentive. the potential for additional stress must be considered in the light of the unprecedented level of accountability and transparency of academic activities (mingers & willmott, 2013). many activities have become transparent, accountable and calculable. these activates range from conformance to strategic directions and operational plans of the newly incorporated university, to hr performance plans and appraisals, teaching and learning measures and evaluations, research objectives and kpis, course compliance monitoring and ‘quality assurance’. such measures now direct and manage the performance of each academic and the performance of the university overall (mingers & willmott, 2013). within this performative context, increased publication anxiety is likely to provide a powerful social psychological force on academics since employment, job retention, and career prospects are important motivators, and job security has long been recognised as one of the most potent stressors in the workplace (brief & atieh, 1987). whether the pressure to publish and indeed to generate research income is seen as a stressor or a challenge depends in large part on the extent to which a person feels in control of the outcomes (ganster, kiersch, marsh, & bowen, 2011). academics may have some control over the number of papers they submit, however, most academics have far less control over the outcomes of publication, or research grant submissions or even what counts as a publication. in australia, for instance, conference proceedings do not ‘count’ towards research quotas. in the face of fierce competition for journal space and high rejection rates it is reasonable to think there may be a real fear amongst academics that many good research papers are rejected by reviewers who do not like or understand the work (armstrong, 1997; smith, 2006), by editorial boards who favour "strong empirical work that informs practice" (george, 2014, p. 1) over carefully considered theory development or possibly because there is insufficient room in the journal and that other papers are deemed superior or more closely aligned with journal, its editors and reviewers. such factors are largely out of the researcher’s control. given these pressures it seems likely that scholars will be considering closely what sort of articles get published in the highly ranked journals. academics may undertake research on topics and use research methods they believe will be rewarded by publication in top journals, or engage in other activities that they perceive will favourably influence editors and reviewers of top journals. in australia, there is talk of academics seeking collaboration with american academics in order to increase their chances of getting published in highly ranked us journals. stiff competition for publication may also encourage cronyism and gaming the system (gallivan & benbunan-fich, 2007; macdonald & kam, 2007). knowing, citing, and collaborating with senior editors and established researchers may become increasingly important, if not essential, to publish in the top journals, and ‘dirty tricks’ to increase ones’ publication rate will rise. ‘dirty tricks’ to improve publication include: self-plagiarism, selfserving citation (i.e., citing for reasons other than scholarly value), ‘salami slicing’ or ‘milking’ (artificially breaking studies into a number of publications to increase publication record), misrepresenting or fabricating results, multiple author cartels and including established researchers on papers where they have done little or no work (abraham, 2000; butler & spoelstra, 2015). johnston and riemer (2014) describe these activities as a ‘pernicious’ form of complicity by academics in the journal ranking systems where each academic and manager pursue locally optimal objectives. australasian journal of information systems coulthard & keller 2016, vol 20, research article publication, journal rankings, researcher views 7 but we do not want to paint this crucial step of linking simplified measures to operational activity as a management conspiracy. much more pernicious systemic forces than top-down power are at work here. academics at all levels themselves subtly conspire in the simplification and reduction of the practice they love. the reason is that once these simplified and supposedly objective measures become embedded, it becomes locally optimal for a wide range of actors to make use of these measures in various political and self-interested ways. academics present their achievements using them, selection committees compare candidates using them, promotion committees use them to set standards, phd supervisors use them to focus coursework content and select projects, journal editors use them to position their journals. each of these actions has local validity derived from the peer review system. however, the unintended effect of the collective use of these metrics as a currency is to increasingly endow them with an objectivity that is unjustified when their use is removed from specific academic contexts (p. 852). this complicity may be driven by publication anxiety generated and exacerbated by the journal ranking system. from this point of view, the potential increase in publication anxiety may well become the central driver of research and publications, and the intrinsic merits of research problems may take a backward step in importance. similar to our failure to find strong academic support for the journal ranking system of research, we found no academic support for the positive effects of the ranking system on the researchers themselves. rather, there is some evidence that the pressure to publish is having a deleterious effect on the wellbeing of individual researchers. a recent survey of dutch medical scientists found that a substantial proportion experienced publication pressure which negatively affected their work (tijdink, vergouwen, & smulders, 2013). while publication pressure may be only one of many pressures facing academics in the current environment it nevertheless causes immense pressure since academics who do not perform well may be ‘performance managed’ out by university managers (sangster, 2015). this was tragically illustrated by the suicide of a high profile uk academic after he was placed on performance review following a number of failed grant applications (parr, 2014). less extreme but still worrying is the growing level of workplace stress amongst academics. for instance, in a recent uk academic well-being survey, more than half of the 14,677 respondents indicated high or very high levels of stress, and almost a third reported they often or always experienced unacceptable levels of stress (kinman, 2014). workplace stress is a major contributor to serious health problems (ganster et al., 2011) and attrition. indeed, a 2009-10 survey of 1,414 academics from eight institutions in victoria, australia, found that less than 30% of respondents would stay at their current institution if they were offered a decent job elsewhere (fredman & doughney, 2011). the risk of the loss of highly skilled academics and the possible failure to attract the brightest students should be a concern for governments and society more broadly. finally, the emphasis on measurement of the performance of the individual researcher clouds the issue of institutional and collegial support for the researcher’s output. it assumes something of a level playing field for researchers and their disciplines. research is a collective enterprise requiring enormous social capital and physical resources. such support may range from informal discussions and a strong research culture to access to research grants. such support would generally be outside the control of the individual researcher but he or she will feel its effects in their performance review. overall, there are strong arguments that the negative consequences of the emerging journal ranking systems will outweigh the possible benefits such as the identification of exemplary research, clearer career strategies, and more transparent management of recruitment, retention, and rewarding academics by university managers. we further did not find any support in the literature for the use of journal rankings by academics apart from providing guidance to academics of where to publish for furthering their careers ("senior scholars' basket of journals," n.d.). australasian journal of information systems coulthard & keller 2016, vol 20, research article publication, journal rankings, researcher views 8 4 empirical study while academic opinion seems uniformly bleak, empirical evidence is scarce (hammarfelt & de rijcke, 2014). there have been few attempts to investigate the effects empirically and none that we could identify in the information systems area. the assertions of a negative impact may be strong but need to be tested particularly as there appears to have been few political or university administrative changes as a result of academic opinion and argument. the best and most obvious means of testing is to undertake a form of review or content analysis of the change. perhaps undertaking a content analysis of journal articles by experts utilising various categories such as ‘orthodox’ and ‘innovative’ and chart any change over time. the main problem with this is that by the time negative trends become obvious extensive damage will have already occurred. the views of academics can provide a good indication of the degree and direction that any pressure may be having on research as a consequence of that pressure and whether they feel that have the necessary support and resources to enable them to meet the increased demand for research outputs. while it may be said that their views may be influenced by the critical opinions expressed in journals, the same influence may well lead them to rethink their research for tenure and promotion and thereby bring about the very changes feared by academics. the foregoing suggests a number of empirical questions concerning the effects of journal rankings on research and the researcher. beginning with research, the first area of questioning concerns what is researchers think have been the effects of the journal ranking systems on research and on the journals themselves. a second area of investigation concerns whether researchers feel the pressure to publish is increasing as a result of journal rankings and their role in performance management, and whether they feel they have the resources, mainly time and access to research grants, to support them. to undertake the empirical study, we canvassed the views of is researchers to discover whether they believe that top journals are becoming more important and if they think that the rankings systems strengthen current orthodoxy. and also if they feel under (greater) pressure and whether they believe this pressure is the result of journal rankings. in summary, we asked is researchers who published a conference paper at one of the three is conferences their views on these matters in five areas:  researchers’ views on the nature of ‘top journals’;  researchers’ perceptions of how ranking systems have influenced the type of research that is published;  researchers’ views regarding the usefulness of journal ranking systems.  the extent to which researchers feel under pressure to publish; and  the extent to which researchers feel they have the support and resources to undertake research. 4.1 research design to obtain a sample of active is researchers we emailed all authors of research papers published in three information systems conferences during the latter half 2013. these were the australasian information systems conference (acis), the european conference of information systems (ecis) and the international conference of information systems (icis). is conferences form the ground-spring of research and provide an opportunity to survey a variety of active researchers who are likely to aspire to publishing in highly ranked journals and may already have done so. we also wished to survey researchers from various regions around the world and these three conferences tend to attract researchers who are geographically located close to where the conferences are held. that is, acis is usually held in australia so it tends to attract researchers from asia, ecis is usually held in europe so it tends to attract european scholars and the us based icis rotates between north america, europe australasian journal of information systems coulthard & keller 2016, vol 20, research article publication, journal rankings, researcher views 9 and asia. in 2013, icis was held in europe and thus tended to attract less north american or australasian researchers. table 1 shows the totals of emails sent and returned for all three conferences. the survey was internet based and designed in such a way as not to compel people to answer all questions and we include the results of partially completed surveys too. conference total authors total successful emails total returned* return rate acis 377 369 79 21.4 ecis 585 560 101 18.0 icis 657 644 95 14.8 total 1619 1573 275 17.5 * totally or partially completed table 1: survey return rates the survey was designed in five sections each with a number of 5 point likert scale style questions focusing on: (1) pressure to publish; (2) resources available to the researcher; (3) views on the research characteristics of the top journals; (4) views on journal ranking systems and what they contribute; and finally (5) demographic characteristics such as publication history, stage in career and location of the researcher. an initial version of the survey was piloted within our school and improved based on the resultant feedback. we also invited respondents to include additional comments at the end of survey and 51 people, approximately 20 percent, did so. 4.2 demographics of respondents the demographics for the respondents is shown in table 2. overall, 275 respondents answered at least part of the questionnaire. a large number of respondents (40%) have over ten publications and a large number also have not published in top journals (35%). most of our respondents are male (71%) between the ages of 35 and 55 (53%) and early or mid-career researchers (60%), holding tenured positions (53%). given that two out of the three conferences we surveyed were hosted in europe it is not surprising that europe is the largest group represented by respondents (47%). australasian journal of information systems coulthard & keller 2016, vol 20, research article publication, journal rankings, researcher views 10 publication # none 1 2-4 5-10 >10 n=243 8.81% 10.69% 22.01% 18.24% 40.25% top journal # none 1 2-4 5-10 >10 n=250 34.97% 11.66% 30.06% 12.88% 10.43% experience beginning early career mid-career established n=246 16.15% 29.81% 29.81% 24.22% position contract tenured phds / postdoc other position* n=247 12.6% 53.4% 28.7% 5.2% region europe nth america australasi a / pacific other region** n=250 46.6% 17.8% 23.5% 12.1% age <35 35-55 >55 n=247 31.2% 53.4% 15.4% gender man woman neither n=247 70.9% 28.3% 0.8% * included industry representatives and honorary positions ** other regions included africa and south america table 2: summary of demographic data 5 findings findings are presented in two sections. firstly, we detail academics views on the effects of journal rankings on research, and then we present views on the effects of the journal rankings on the individual researcher. 5.1 views on the effects of journal rankings on research 5.1.1 researchers views on the nature of ‘top journals’ while some academics pointed out that top journals did not form a homogenous group and therefore found it difficult to consider top journals as a whole, the qualitative and quantitative responses show that many academics were able to do so. the findings show that most researchers (88%) believe that ‘top journals’ have a strong influence on the direction of the discipline. table 3 also shows that 60 percent of authors believe top journals publish safe, incremental research which makes modest contributions to knowledge. around 20 percent believe that new advances are reported first in the top journals but the majority (54%), did not believe so, as summed up by one respondent: they always publish 'safe' research, which can be easily cross-checked, and if you don't mind me saying a bit boring to read. so, in my experience, as the ranking gets higher, the text gets more boring, the research more outdated, and overall less significant or interesting in some way [r8]. nearly two thirds of the authors believed that ‘mainly quantitative’ research is published. this is consistent with interpretivist fears expressed in the literature of not getting published (lincoln, 2012). one third of authors believed that the top journals have the most innovative research, however more (42%) did not believe this was the case. in terms of encouraging diversity of publications in top journals, 42% of authors are unconvinced while only 31 percent agreed. australasian journal of information systems coulthard & keller 2016, vol 20, research article publication, journal rankings, researcher views 11 statements about top journals agree or strongly agree top journals have a strong influence on the direction of is research 87.7% mainly quantitative research gets published in top journals 63.1% mainly publish safe, incremental research with modest gains in knowledge 59.9% top journals have the most innovative and ground-breaking research 33.7% (41.7%) top journals are diverse in the regions and methodologies they publish 31.3% (41.7%) new advances in is are mostly reported first in top journals 19.9% (54.2%) notes: percent in brackets refers to the number who strongly disagree or disagree with the statement. not all items or questions were completed by all respondents. unless otherwise stated, the sample size for an item ranged between 270 and 276 responses. table 3: authors’ views on the nature of ‘top journals’ overall, the vast majority of researchers believe top journals strongly influence the direction of the is discipline. however, it would appear that this influence is towards a research orthodoxy. in one sense, the journal ranking system is doing what was intended, providing guidance but it seems that for is researchers the direction of that guidance is a narrowing to a particular orthodoxy. as one respondent commented: there is need for more plurality in information systems research. i am not sure if it is the result of journal ranking systems (they exist in other fields after all) or that top journals have a very narrow focus but gradually is research is losing its potential to participate in public debates on critical contemporary social issues [r51] 5.1.2 views on the effect of journal rankings on what and how researchers work authors were asked to comment on their beliefs as to whether journal ranking systems have encouraged, inhibited, or had no effect on the type of is research published (table 4). the findings show a strong conservative effect with 82 percent believing that mainstream research is encouraged and 75 percent believing safe research is encouraged. only 28 percent believed that the ranking system encouraged work on difficult problems while nearly 40 percent believed that the ranking system inhibited such research. one respondent summed up the majority view stating: establishment of journal rankings drives behaviour of academics: they are looking into currently published research to be able to emulate it and increase chances of publishing. this encourages mediocre but technically well executed research and discourages innovative, risky and unusual research. [r20] it is however, not all one way, the results also show that most academics believe top journals encourage theory development and a significant number believe that the top journals encourage work on emerging issues and important problems (see table 4). in addition, over half believed that resource intensive research is encouraged. however, a significant minority of nearly one quarter of academics believe that journal rankings inhibit research on emerging issues and important issues. further, nearly 40% of respondents believe work on ‘difficult problems’ is inhibited. it is difficult to tease out what appears to be contradictory views here; a resounding view that mainstream, safe, conforming research is encouraged and research into difficult problems was not encouraged but at the same time many also believe that research on emerging and important problems is encouraged. we need to undertake further investigation but one resolution is to suggest that because of the power of the ranking system, what is written in the top journals is important and emerging because being in the top journals makes it so. like lewis carroll’s humpty dumpty, top journals make ‘important and emerging’ mean what they choose them to mean. australasian journal of information systems coulthard & keller 2016, vol 20, research article publication, journal rankings, researcher views 12 beliefs about affect journal rankings have on nature of research inhibited neither encouraged mainstream research 1.7 16.2 82.1 safe, conforming research 6.7 18.8 74.6 theory development 11.5 23.9 64.5 resource intensive research 12.3 36.5 51.2 research on emerging issues 23.2 27.8 48.9 research on important problems 24.2 33.1 42.8 research on difficult problems 38.5 33.2 28.3 table 4: effect of journal rankings on nature of is research the survey results suggests that authors believe that journal ranking have influenced the focus of is research with 57 percent believing it has encouraged research with a business focus (see table 5). less emphatically, authors also believe research on both social issues and technical issues is somewhat encouraged by journal ranking systems while applied and practical research is somewhat inhibited. beliefs about effect journal rankings have on research area inhibited neither encouraged research on business 12.5% 30.2% 57.3% research on technical issues 28.8% 32.2% 39.1% applied and practical research 39.9% 32.2% 27.9% social issues 21.9% 38.6% 39.5% table 5: effect of journal rankings on research area in terms of diversity in regions and methods, table 6 shows that over a third of surveyed authors believe that research on developing countries and areas outside north america and europe is inhibited (34% and 37% respectively). less than 20 percent however believed that such research was encouraged by journal ranking systems. researchers also believe that quantitative research is the big winner with 78 percent believing ranking schemes encourage this method of research while only 27 percent believe that qualitative research is encouraged. effect journal rankings have on diversity of regions and methods inhibited neither encouraged research on developing countries 34.4 47.6 17.9 research in areas outside north america and europe 36.8 44.3 18.9 qualitative research 36.6 36.6 26.7 quantitative research 2.9 18.8 78.3 table 6: effect of journal rankings there were several gender differences identified. women were significantly more likely to believe that qualitative research was inhibited by journal rankings – 55% compared with 30% of men (χ2=13.955, df2, p <0.001). women were also significantly more likely to believe that quantitative research was encouraged – 89% compared with 68% (χ2= 6.846, df2, p<0.033). finally, women were also more likely to believe that technically focused research (57% vs 33%) (χ2=10.999, df2, p<0.004) was encouraged and less likely to believe that applied and practical work was encouraged (42% vs 58%) (χ2=9.102, df2, p<0.011). australasian journal of information systems coulthard & keller 2016, vol 20, research article publication, journal rankings, researcher views 13 5.1.3 beliefs about the use of journal rankings while academics were in general critical of journal rankings effects on research and the pressure they place on academics, only 13 percent believed that they were of little or no value (table 7). nearly half (46%) of the academics believed that the rankings had improved the quality of journals. the researchers also saw journal rankings as assisting in determining where to publish (82%), what to read (74%) and helping to avoid unscrupulous journals (60%). comments on the benefits of ranking systems to academics included: this is an interesting and difficult topic. a key aspect of top journals is to provide rigorous research that provides a solid basis for future research. it is sometimes astonishing how quickly the quality of published research drops when one moves outside of the ais scholars' basket of journals. there is a lot of research sold as "relevance-oriented" that is just not solid enough to invest in by designing studies that build on those works [r50] it has helped me when filtering papers to reference. if for example, i have two sources providing similar information, the one from a more reputable source is naturally used even though the message might be equally well explained in a journal of less rank. thus, i think that journal ranking promotes itself, as these papers tend to be more referenced not necessarily because they contribute more but since they are safer and more regarded [r38] beliefs about the use of journal rankings disagree neither agree assist in determining the journal to publish in 6.1 11.8 82.0 provides a useful guide to quality literature 12.7 13.5 73.9 provides little or no value 67.8 19.2 13.1 assists in avoiding predatory journals 15.1 24.9 60.0 increased the quality of publications 18.5 35.8 45.7 primarily of benefit to managers of academics 24.3 30.0 45.7 table 7: beliefs about the use of journal rankings curiously, nearly half (46%) believe that the journal ranking systems are primarily of benefit to managers. 5.2 views on the effects of journal rankings on the researcher 5.2.1 pressure to publish the survey results show that academics feel under enormous pressure to publish. as shown in table 8, three quarters of academics place themselves under enormous pressure to publish and over 40 percent of academics are often worried that they will not get published and find worry a constant burden. it is as if the anxieties traditionally associated with undertaking a phd continue on into academic working life. i put myself under enormous pressure to publish 75.8% i am often worried that i will not get my work published 43.4% worrying about getting published is a constant burden 41.2% table 8: felt personal pressure (per cent agree or strongly agree) it is clear that much of the pressure comes from university and the journal ranking systems. as one respondent commented: i think the ranking of the journals now dictate where we need to publish and it’s really, really hard to get published in the top journals. actually the pressure on writing papers creates heaps of stress. [r6] australasian journal of information systems coulthard & keller 2016, vol 20, research article publication, journal rankings, researcher views 14 indeed, as table 9 shows, only one in ten academics believe there is little pressure to publish from their university and nearly two thirds did not agree that there was little pressure to publish. further, 90 percent believed that promotion and tenure were largely achieved by publishing in the top journals. despite many universities trumpeting the importance of good teaching, only six percent of authors believed that good teaching was the best way to gain promotion or tenure. academics consider themselves under more pressure to publish than a decade ago with only 5% believing this was not the case. indeed, 70 percent of authors believe that journal ranking systems are responsible for the increase in pressure to publish and over 80 percent did not believe that journal ranking systems had little influence over their careers. there is little pressure to publish at my university or organisation 10.4% (62.7%*) promotion and security of tenure is largely achieved by publishing in the top journals 89.5% academics are under less pressure to publish than a decade ago 5.1% (63.9%) good teaching not publications is the best way to get promotion or tenure 5.6% (82.7%) journal ranking systems have led to increased pressure to publish 70.1% journal ranking systems have little influence on academic careers at my university 11.2% (80.9%) * percentage in brackets refers to the number who strongly disagree or disagree with the statement table 9: organisational pressure (per cent agree or strongly agree) it is clear that authors feel under great pressure to publish and that much of this pressure derives from the universities and the importance of publications for promotion and tenure. pressure to publish may also be exacerbated by a reduction of university sanctioned research areas and outlets. as one respondent stated: it infuriates me that my organisation is only willing to support certain research areas for publication and place incredible pressure on us to publish in high quality journals without additional time and compensation [r2] there were some differences in the level of worry among different researcher groupings. perhaps not surprisingly, established researchers and researchers who had published more than ten articles were least worried. consistent with lincoln’s (2012) views of the effect of journal rankings on interpretivist research, researchers who identified themselves with interpretivism were more worried they would not get their work published. women were significantly more worried than men that they would not get published (χ2= 17.11, 2df, p<0.000) and were more likely to feel under a constant burden (χ2= 10.386, 2df, p<0.006). views of organisational pressure were fairly homogenous across different researcher groupings. however, european academics were more likely to agree that journal rankings have increased publication pressure than their north american colleagues (χ2=18.007 df6 p<0.006). this may reflect that over the past decade there has been ‘increased (coercive) institutional pressure to publish in the elite journals’ in europe (lyytinen, baskerville, iivari, & te'eni, 2007, p. 319). notwithstanding this increased pressure, optimism reigned with most authors confident in their ability to get published (65%) while nearly half (46.3%) were confident that their research would get published in the top journals. there were no significant differences in confidence of publication by region but well established researchers are more confident of being published than other groups (χ2=24.449, df6 p0<.000). women, on the other hand, were significantly less confident than men (χ2=13.240 df2 p <0.001) with only fifty one percent of women confident that they would get published compared with seventy two percent of men. australasian journal of information systems coulthard & keller 2016, vol 20, research article publication, journal rankings, researcher views 15 5.2.2 resources to undertake research access to resources and support to undertake research is equivocal (table 10). the most scarce resource is time, with nearly half the authors believing that they did not have reasonable time to focus on their research; did not have sufficient time to read the literature; or have good access to research only semesters. nearly seventy percent believed a significant amount of their research was unpaid, that is, they did their research in their own time and in excess of their paid hours. a significant amount of my research is unpaid (in excess of the number of hours i am paid to work) 69.8% i have good access to research-only semesters and study leave 24.5 (52.6*%) i have time to read deeply and widely to keep abreast of the key research developments discipline 23.4 (50.0*%) i believe i have reasonable time to focus on my research to produce quality research 33.7 (47.2*%) *percentage in brackets indicates those who disagree or strongly disagree with the statement table 10: time to undertake research (per cent agree or strongly agree) it is perhaps not surprising given the pressure to publish that academics want more time to research and make up time by undertaking a significant amount of their research out of hours. this would suggest that those who can afford to work additional hours will be in a better position to undertake research. women are significantly more likely to report that they do not have sufficient time to undertake research (χ2=16.1, 2df, p<0.000) or time to read widely and broadly to keep abreast of the field (χ2= 7.51, 2df, p<0.023). this reflects that they tend to be more worried about publication but it is possible that it reflects women generally reporting greater family responsibilities than men (cha & weeden, 2014) which reduces the time they have available outside of normal working hours to undertake their research work. north americans were more likely to agree or strongly agree (52%) that they had time to focus on their research than their australasian (29%) and european (27%) counterparts (χ2=16.561 df6 p<.011). academics who had published more than ten articles in top journal were also more likely to believe that they have time to undertake research (χ2=16.781 df8 p<0.032) and access to research semesters (χ2=16.865 df8 p<0.032). access to grants is also equivocal, with 40 percent believing they had access to research grants but there being over a third of academics believing they did not have reasonable access is hardly an endorsement for good access to grants (see table 11). authors with more than ten papers published in the top journals were more likely to agree that they had adequate access (χ2= 21.83, df8, p<0.005). they also tended to believe that their school, department or discipline (47.8%) and their university were supportive (41.9%). however, sizeable proportions of around 30 percent of authors disagreed. the majority of authors did believe that the resources that were available to them were of an international standard (57.7%). the only consistent finding at this aggregate level is that schools and universities need to support research through an increased allocation of time. i believe i have reasonable access to research grants to publish quality research in the top journals 39.9% the university as a whole provides me with a strong supportive research culture 41.9% my school/department/discipline provides me with a strong supportive research culture 47.8% the resources i have available are of an international standard 57.7% table 11: resources to undertake research (per cent agree or strongly agree) australasian journal of information systems coulthard & keller 2016, vol 20, research article publication, journal rankings, researcher views 16 6 discussion 6.1 discussion on findings the findings of this study provide the first empirical support in information systems for the concerns expressed more broadly in the literature. in particular, that journal ranking systems are contributing to pressure to publish and publication anxiety (tijdink et al., 2013) and that most academics surveyed feel they have insufficient time to undertake their research (kinman & jones, 2003). further, the findings also provide empirical support for the assertion that ranking systems have inhibiting effects on innovative, risky research and encourage safe, conforming mainstream orthodoxy (alvesson & sandberg, 2013; lincoln, 2012; macdonald & kam, 2007; ozbilgin, 2009; powell & woerndl, 2008; tourish, 2015; van dalen & henkens, 2012). the findings also support allegations in the literature that regional, cross-disciplinary (mcnay, 1998; rafols, leydesdorff, o’hare, nightingale, & stirling, 2012) and qualitative research is threatened (lincoln 2012, chen and hirschheim 2004). the results show a clear fear that driven by pressure to publish and constrained by lack of time, is academics will increasingly research what they see being successfully published rather than what is important. further, researchers believe journal ranking systems and the top journals do influence how and what is researched and while this may provide guidance and instruction, it also provides a model of conformity or model for success. nevertheless, it is clear that most academics also believe that journal rankings have increased publication quality and significant numbers of academics believe that the top journals do publish important work on emerging issues. few find that the journal ranking systems are of little or no value. this appears at first to be a contradiction between academics saying, on the one hand; that the top journals are influential and publish work on emerging issues and that journal rankings are useful and have increased quality, while on the other hand giving quite a uniform critical assessment concerning the nature and diversity of research published. however, if we consider top journals and ranking systems as determining value and determining what is published by that fact they are important and influential, then almost by definition they report on emerging important issues. those important issues are just not innovative or diverse but are safe and conforming. what the respondents meant by increased quality is unclear. however, it does not seem to mean an increase in innovative and ground breaking research. for the respondents, the increase in quality may refer not to more good research being published but simply less bad, poorly conceived and carried out research being published. most tellingly, while academics find ranking systems useful, almost half think their managers find them more useful. based on our findings, it would appear that the top journals tell them what is successful and important because that is what the top journals are publishing. there are particular groups who are more affected by publication anxiety and whose publications may fall as a result of the rise of journal rankings. certainly it appears, in line with the literature (lincoln, 2012), that qualitative researchers are most concerned. we also found that women were most worried about publication and reported that they had the least time to undertake research. gaining time to undertake research is clearly an issue for nearly everyone but it may also be an important distinguishing issue between men and women for a career trajectory in academia. cha and weeden (2014a) in their study of us current population and labour force statistics found that: the ability for staff to work longer hours rests on a foundation that is itself highly gendered -in other words, those employees who work longer hours do so only with the support of other household members, usually women, who shoulder the lion's share of household duties....women are less likely to choose positions where longer hours are embedded in the organisational culture, and the fact they are responsible for the majority of housework and childcare and therefore can't offer as much of their time to outside-hours (and paid) work (cha & weeden, 2014b, p. 16) australasian journal of information systems coulthard & keller 2016, vol 20, research article publication, journal rankings, researcher views 17 women may have a strong reason to be worried. it may be the case that the ranking systems overall benefit those with the trifecta of talent, research time, and resources as well as those with better social capital – better research connections with influential researchers, ‘insider knowledge’ on how to get published and to play the ‘game’. 6.2 cycle of strengthening orthodoxy it is interesting to consider the social psychological role publication anxiety may play in developing or strengthening safe incremental research. driven by pressure to publish, constrained by lack of time and monitored by performance regimes, it is likely that academics will try to research what they see being successfully published. the chance of publication is naturally greater when they go with, rather than against, the mainstream and hedge their bets accordingly. it is in this way, academics are complicit in the outcome (johnston & riemer, 2014). the supply of innovation may be low at the best of times but the journal ranking system may make that supply even lower as academics play ‘safe’ and a vicious cycle arises. an anecdote illustrates this supply side problem. just before a faculty meeting, when we had just completed our first data run and while waiting for the meeting to start, one of the authors was describing some of the initial findings. a senior academic across the table listened with interest. he then exclaimed in frustration that as a senior editor of an a* ranked publication he found that he rarely received, let alone published, innovative exciting material no matter how many editorials he wrote or who he appointed as associate editors. since academics do not see much innovation and diversity in the journals they therefore do not believe that is the way to success despite the pronouncement of senior editors. a further break on innovation is that whereas emerging new journals provided an outlet for new, diverse topics (lamp & fisher, 2010), the ranking system may raise a significant barrier to entry – new journals will necessarily be unranked and as such more or less worthless to publish in. the systematic effect may be summarised as follows: the rise of performative measures establishes the performance ladder that the academic must follow in order to gain recruitment, tenure and promotion with an increasing emphasis on research output in selected ‘quality journals’. to succeed academics must identify the particular qualities of such journals and reproduce them as closely and efficiently as they can. with one eye on their career and the other eye on their available workload, the academic is likely to make a rational choice of what to research and how. while the intended effect of this may be to provide clarity, objectivity of performative measures and to improve quality by providing exemplary guidance to academic research, the actual effect is increased anxiety and loss of diversity of methods, journals and topics. this should be particularly worrisome for a discipline that is essentially a ‘frontier discipline’ investigating and exploring changes in technology, its use and effects (yoo, 2010). 7 conclusions and further research overall, it would seem that academics believe the journal ranking systems and associated performative measures and recruitment and retention processes will lead to the impoverishment of research. being an academic working towards publishing important work is quite different to publishing to survive. here again we need to repeat that there has and always will be a tendency towards publishing mainstream research that offers the best opportunity for recruitment, retention and promotion. the point here is that the tendency has intensified in the minds of academics to an imperative. the performative net has tightened and the results strongly support the concerns with the ranking systems expressed by many of critical researchers in the literature. as we have seen too, there is increased stress on researchers, particularly those who work in qualitative areas but also those who may have reduced ability to work additional hours as it australasian journal of information systems coulthard & keller 2016, vol 20, research article publication, journal rankings, researcher views 18 would appear that most researchers claim to undertake their research in excess of regular office hours. this may affect women researchers particularly, who also are more worried or stressed over publication. we are aware that we a painting a rather cynical picture of the academic researcher, albeit one under great pressure. nevertheless, the results are consistent with the view that the journal ranking system is principally external to the processes of research and serves to not only rank journals but to rank academics for recruitment, tenure, and promotion, to rank universities on research league tables, and to secure the ascendancy of established journals. the results suggest that the intrinsic benefits to research are small: a handy guide to the literature and the absence of poor quality papers. such a view needs to be further examined. rankings have become so important to academic careers, to universities, and to journals that they need to be examined as a priority to ensure that meritorious and diverse research is found, published, and rewarded. ranking systems have globalised research and researchers are being measured and compete against their international peers and even across disciplines. the pressures to publish are likely to increase both ‘sharp’ research practices and publication tactics by all who are rewarded by publication or ranking systems namely researchers, reviewers, editors, journals and universities. the concerns and criticisms of journal rankings in the literature are very strongly reflected in the views of the researchers studied. researchers do fear and perhaps even see a rise in safe, conforming work and the pressures to do so appear enormous. the question is whether academic practices will change in accordance with the concerns expressed in the literature and here in our survey. key limitations of our study we believe are firstly the charge that ‘academics are simply repeating what they have read in the journals’. that is to say, influential critical researchers have swayed the discipline. secondly, that criticism of the journal ranking system, a system that is intended to make academic work transparent and accountable, is self-serving. people generally do not like being monitored. there is no getting around these criticisms, but it does not make it untrue and at least gives an indication of the feelings and thoughts of academics on the matter. most importantly, it is likely that academics will act on their views and become more conservative, incremental and mainstream in their work. finding an objective or neutral standpoint to these changes is difficult if not impossible at least in the short term and without hindsight. what may be for one researcher an improvement in quality or the establishment of a clear research direction for another may be seen as a reduction in diversity. one cannot simply ask those most senior because, to use the well-worn phrase, the victors write the history. research into this area will necessarily be partisan but it is no less important for that. such studies and research can monitor the rise and fall of research methodologies and attempt through content analysis or expert panels to gauge changes in types of research and their theoretical and practical importance and impact. as we have suggested, for the discipline of information systems, the reduction of diversity may be particularly worrisome for a discipline engaged in a technological frontier that is constantly moving and which is cross disciplinary in nature. should each discipline ‘look to their own disciplines’ for their journal rankings, cross disciplinary studies may suffer. further studies may look on the changing nature of academic work, the rise of performativity and the impact that has on current and future academics. as well as measuring rising or falling levels of work satisfaction, such studies may examine the attractiveness of academic work does it indeed attract and retain the brightest and the best? future studies can also investigate the question of the diversity of authorship and diversity of publications. such evidence that we have suggests a predominance of north american authorship (truex et al., 2009, keller & coulthard, 2013). will this increase or decrease? will citations and authorship groupings diversify or become more tightly networked? as the stakes get higher and work performance and promotion more closely related to journal rankings, we may see the rise of plagiarism, false australasian journal of information systems coulthard & keller 2016, vol 20, research article publication, journal rankings, researcher views 19 authorship, citation and research networks and other practices aimed at increasing the researcher’s publication standing. another area for research is whether the ranking systems do stymie new journals and effectively strengthen the brand of existing high ranked journals because researchers gain little institutional benefit for publishing in new journals and where one publishes trumps what one publishes. certainly our findings suggest that this is likely, but it needs to be further explored. now that the performative genie is out of the bottle it is difficult to push the genie back in or relax the expectations on academics in the ranking system. further, despite many calls for better measurement systems, all or most systems are likely to be gamed. of course, one further alternative is for the interview and promotion panels to read the articles. in summary, future research will involve further investigating:  the positive and negative impact of journal rankings on content and diversity of research;  the effects of journal rankings on research collaboration and publication for different research groups, including gender and region;  the occurrence of ‘sharp’ research practices to increase research publications;  the development of alternative internet publishing forums as outlets for research; and  the effects of performativity on recruitment, retention and advancement of academics. our empirical study has only been on the supply side of publications – the authors. studies of the demand side, the journals, publications companies, editors, reviewer, readers, and universities also need to be made. it is likely that similar pressures on the journals to provide ‘quality’ and to increase their rankings is occurring with zeal and as we have seen in the anecdote above, frustration. academics are not in charge of the system but neither is anyone else fully in charge. researchers are complicit in the production of orthodoxy (johnston & riemer, 2014). however, as we have seen perpetuating orthodoxy is in the interest of the individual researcher and calls for researchers to temper their immediate interests may ring hollow. what is required is a disciplinary approach to reducing the reliance on rankings in the discipline and the encouragement of new journals and new ideas and approaches. it is a matter of academic and disciplinary policy and it would seem an appropriate role for the association for information systems, national bodies, and peak academic forums such as the australian professor and heads of information systems to take charge and jointly investigate this matter through supporting research and consultation within the discipline. from such investigations a better system may be created; one controlled by the discipline itself rather than to any particular groupings within or outside of the discipline. references abraham, p. (2000). duplicate and salami publications. journal of postgraduate medicine, 46(2), 67-69. alvesson, m., & sandberg, j. (2013). has management studies lost its way? ideas for more imaginative and innovative research: has management studies lost its way? journal of management studies, 50(1), 128-152. doi:10.1111/j.1467-6486.2012.01070.x benbasat, i., & barki, h. (2007). quo vadis tam? journal of the association for information systems, 8(4), 7. bourdieu, p. (1998, december). utopia of endless exploitation: the essence of neoliberalism. le monde diplomatique. brief, a. p., & atieh, j. m. (1987). studying job stress: are we making mountains out of molehills? journal of organizational behavior, 8(2), 115-126. doi:10.1002/job.4030080203 australasian journal of information systems coulthard & keller 2016, vol 20, research article publication, journal rankings, researcher views 20 butler, n., & spoelstra, s. (2015). the dark arts of academia – and why journals must do more to tackle the problem. retrieved from http://theconversation.com/the-dark-arts-ofacademia-and-why-journals-must-do-more-to-tackle-the-problem-35796 cha, y., & weeden, k. a. (2014a). overwork and the slow convergence in the gender gap in wages. american sociological review, 79(3), 457-484. doi:10.1177/0003122414528936 cha, y., & weeden, k. a. (2014b). overwork and the slow convergence i n the gender gap in wages. retrieved from retreived from http://mypage.iu.edu/~cha5/youngjoo_cha_files/cha_weeden.pdf chen, w., & hirschheim, r. (2004). a paradigmatic and methodological examination of information systems research from 1991 to 2001. information systems journal, 14(3), 197-235. doi:10.1111/j.1365-2575.2004.00173.x cooper, r. b., & zmud, r. w. (1990). information technology implementation research: a technological diffusion approach. management science, 36(2), 123-139. fredman, n., & doughney, j. (2011). academic dissatisfaction, managerial change and neoliberalism. higher education, 64(1), 41-58. doi:10.1007/s10734-011-9479-y gallivan, m. j., & benbunan-fich, r. (2007). analyzing is research productivity: an inclusive approach to global is scholarship. european journal of information systems, 16(1), 3653. doi:10.1057/palgrave.ejis.3000667 ganster, d. c., kiersch, c. e., marsh, r. e., & bowen, a. (2011). performance-based rewards and work stress. journal of organizational behavior management, 31(4), 221-235. doi:10.1080/01608061.2011.619388 grover, v., & lyytinen, k. (2015). new state of play in information systems research: the push to the edges. mis quarterly, 39(2), 271-296. hammarfelt, b., & de rijcke, s. (2014). accountability in context: effects of research evaluation systems on publication practices, disciplinary norms, and individual working routines in the faculty of arts at uppsala university. research evaluation, 24(1), 63-77. doi:10.1093/reseval/rvu029 harvey, l. (2008). rankings of higher education institutions: a critical review. quality in higher education, 14(3), 187-207. doi:10.1080/13538320802507711 hassan, n. (2014). value of is research: is there a crisis? communications of the association for information systems, 34 (1). retrieved from http://aisel.aisnet.org/cais/vol34/iss1/41 hovorka, d. (2010). moving beyond is identity: concepts and discourses. paper presented at the 31st international conference on information systems icis 2010, st. louis, united states, 15th december 2010. hussain, s., liu, l., wang, y., & zuo, l. (2015). journal rankings, collaborative research and publication strategies: evidence from china. accounting education, 24(3), 233-255. doi:10.1080/09639284.2015.1037776 jacques, t. s., & sebire, n. j. (2010). the impact of article titles on citation hits: an analysis of general and specialist medical journals. jrsm short reports, 1(1), 2-2. doi:10.1258/shorts.2009.100020 janvrin, d. j., lim, j.-h., & peters, g. f. (2015). the perceived impact of journal of information systems on promotion and tenure. journal of information systems, 29(1), 73-93. doi:10.2308/isys-50963 johnston, r., & riemer, k. (2014). on putting the score ahead of the game. communications of the association for information systems, 34(1). retrieved from http://aisel.aisnet.org/cais/vol34/iss1/47 australasian journal of information systems coulthard & keller 2016, vol 20, research article publication, journal rankings, researcher views 21 keller, s., & coulthard, d. 2013. charting diversity and change in is publications: a tricontinental journal analysis, australasian journal of information systems, 18(1), 5-35. kinman, g. (2014). doing more with less? work and wellbeing in academics. somatechnics, 4(2), 219-235. doi:10.3366/soma.2014.0129 kinman, g., & jones, f. (2003). 'running up the down escalator': stressors and strains in uk academics. quality in higher education, 9(1), 21-38. doi:10.1080/13538320308162 kuhn, t. s. (1970). the structure of scientific revolutions ([2d ed., enl ed.). chicago: university of chicago press. lamp, j. w. (2009). journal ranking and the dreams of academics. online information review, 33(4), 827-830. doi:doi:10.1108/14684520910985747 lamp, j. w., & fisher, j. (2010). era distribution of information systems journals. australasian journal of information systems, 16(2). laudel, g., & gläser, j. (2014). beyond breakthrough research: epistemic properties of research and their consequences for research funding. research policy, 43(7), 12041216. doi:10.1016/j.respol.2014.02.006 lincoln, y. s. (2012). the political economy of publication: marketing, commodification, and qualitative scholarly work. qualitative health research, 22(11), 1451-1459. doi:10.1177/1049732312457713 lyytinen, k., baskerville, r., iivari, j., & te'eni, d. (2007). why the old world cannot publish? overcoming challenges in publishing high-impact is research. european journal of information systems(4), 317-326. macdonald, s., & kam, j. (2007). ring a ring o? roses: quality journals and gamesmanship in management studies. journal of management studies, 44(4), 640-655. doi:10.1111/j.1467-6486.2007.00704.x marginson, s. (2011). the new world order in higher education: research rankings, outcomes measures, and institutional classifications. in m. rostan & m. vaira (eds.), questioning excellence in higher education: policies, experiences and challenges in national and comparative perspective (pp. 3–20). rotterdam, the netherlands: sense publishers. mcnay, i. (1998). the research assessment exercise (rae) and after: 'you never know how it will all turn out'. perspectives: policy and practice in higher education, 2(1), 19-22. doi:10.1080/713847899 mingers, j., & willmott, h. (2013). taylorizing business school research: on the 'one best way' performative effects of journal ranking lists. human relations, 66(8), 1051-1073. doi: 10.1177/0018726712467048 ozbilgin, m. f. (2009). from journal rankings to making sense of the world. academy of management learning & education, 8(1), 113-121. doi:10.5465/amle.2009.37012185 parr, c. (2014, november 27, 2014). imperial college london to ‘review procedures’ after death of academic. retrieved from https://www.timeshighereducation.com/news/imperial-college-london-to-reviewprocedures-after-death-of-academic/2017188.article powell, p., & woerndl, m. (2008). time to stop researching the important things? european journal of information systems, 17(2), 174-178. doi:10.1057/ejis.2008.11 rafols, i., leydesdorff, l., o’hare, a., nightingale, p., & stirling, a. (2012). how journal rankings can suppress interdisciplinary research: a comparison between innovation studies and business &amp; management. research policy, 41(7), 1262-1282. doi:10.1016/j.respol.2012.03.015 australasian journal of information systems coulthard & keller 2016, vol 20, research article publication, journal rankings, researcher views 22 rijcke, s. d., wouters, p. f., rushforth, a. d., franssen, t. p., & hammarfelt, b. (2015). evaluation practices and effects of indicator use—a literature review. research evaluation. doi:10.1093/reseval/rvv038 rossi, f., & geuna, a. (2015). university and the economy: edward elgar publishing. sangster, a. (2015). you cannot judge a book by its cover: the problems with journal rankings. accounting education, 24(3), 175-186. doi:10.1080/09639284.2015.1055929 schekman, r. (2013, 2013/12/09/). how journals like nature, cell and science are damaging science. the guardian . retrieved from http://www.theguardian.com/commentisfree/2013/dec/09/how-journals-naturescience-cell-damage-science senior scholars' basket of journals. (n.d.). retrieved from http://aisnet.org/?seniorscholarbasket tijdink, j. k., vergouwen, a. c., & smulders, y. m. (2013). publication pressure and burn out among dutch medical professors: a nationwide survey. plos one, 8(9), e73381. doi:10.1371/journal.pone.0073381 tourish, d. (2015). some announcements, reaff irming the critical ethos of leadership, and what we look for in submissions. leadership, 11(2), 135141. doi:10.1177/1742715015577889 truex, d., cuellar, m., & takeda, h. (2009). assessing scholarly influence: using the hirsch indices to reframe the discourse. journal of the association for information systems, 10(7), 560-594. van dalen, h. p., & henkens, k. (2012). intended and unintended consequences of a publishor-perish culture: a worldwide survey. journal of the american society for information science and technology, 63(7), 1282-1293. doi:10.1002/asi.22636 wilhite, a. w., & fong, e. a. (2012). coercive citation in academic publishing. science, 335(6068), 542-543. doi:10.1126/science.1212540 yoo, y. (2010). computing in everyday life: a call for research on experiential computing. mis quarterly, 34(2), 213-231. retrieved from http://aisel.aisnet.org/cgi/viewcontent.cgi?article=2903&context=misq copyright: © 2016 coulthard & keller. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems taxén 2018, vol 22, research note reconfiguring sociomateriality from a neurobiological perspective 1 reconfiguring sociomateriality from a neurobiological perspective lars taxén linköping university lars.taxen@gmail.com abstract the aim of this paper is to propose a reconfiguration of sociomateriality (sm) from a neurobiological perspective, which maintains the relational ontology of sm without relapsing into untenable entanglement positions of strong sm (ssm). keywords: sociomateriality, neurobiological perspective. 1 introduction the sociomaterial (sm) strand of research—or the predominantly ‘strong’ variant of sm (ssm) that drew on agential realism as its foundation (barad, 2003)—was introduced in information systems (is) by orlikowski and scott (2008). notwithstanding the other variants of sm, such as ‘weak’ and sm based on critical realism sociomateriality, this paper focuses on the extant limitations of ssm and makes a case on how a neurobiological perspective can advance the sm strand of research in is. 2 limitations of ssm notwithstanding the general belief that ssm is “extremely theoretical” and therefore difficult to decipher (leonardi, 2013), it is safe to say that the fundamental tenets of ssm suggests that ssm is purported to be generally applicable—that is to say, “there is no reason, in principle, why this genre [sociomateriality] of scholarship would not be appropriate for historical manifestations of work and organizations” (orlikowski and scott, 2008, p. 463). indeed, the limitations of sm may be elucidated in a straightforward manner using the example of a cellist in concert (see figure 1). this approach adheres to the thesis that “if something is understood well, it can be explained simply” (paul, 2010, p. 380). figure 1 mstislav rostropovič in concert playing the cello voice in a quartet • ssm tenet “there are no stable pre-existing entities” (hassan, 2016, p.17). clearly, the musician, the score and the cello exist before the performance, and remain stable australasian journal of information systems taxén 2018, vol 22, research note reconfiguring sociomateriality from a neurobiological perspective 2 enough throughout the concert. concerning pre-existence, the musician’s playing skill, the instrument and several other activity-relevant entities need to be in place before the activity can be carried out. • ssm tenet “[entities] (whether humans or technologies) have no inherent properties, but acquire form, attributes, and capabilities through their interpenetration” (orlikowski & scott, 2008, p. 455). undoubtedly, the cellist and the cello have properties that exist before the concert. however, these may be modified by the activity—for example, the cellist’s playing skill may be improved and the timbre of the cello may change. • ssm tenet “[any] distinction of humans and technologies is analytical only” (orlikowski & scott, 2008, p. 456). certainly, most people have no problem in distinguishing rostropovič from his cello. thus, it seems bizarre to claim that this distinction is ‘analytical only.’ this tenet devaluates the role of the acting individual, which means that ssm is unable to elucidate uniquely human issues such as communication, meaning, interpretation, and action. the example demonstrates a concrete, every-day situation that ssm is unable to illuminate. therefore, if sm is to contribute to is, further work is needed to reconfigure its tenets. this paper intends to contribute a solution for this reconfiguration, specifically from the neurobiological perspective. 3 neurobiological reconfiguration for sm to address the limitations of ssm, a neurobiological point of departure is proposed. the main motivation for this position is to establish the ‘individual’ on par with the ‘social’ and ‘material’ in the sm discourse, and in doing so, providing a tripartite analysis ground for advancing sm: the individual, the social, the material. the proposal is based on the following foundational assumptions. 3.1 a dialectical ontology the relatum between individual and sociomaterial is seen as dialectical, which means that relata are pairs of opposites. these opposites are different but mutually depending on and impacting each other (israel, 1979)—e.g. a classic example is the relation between master-slave. in that sense, individuals cannot be understood without taking their social environment into account and vice versa. the individual (neural) and sociomaterial realms form a totality, which parts “cannot be separated or isolated without destroying the phenomenon that is studied” (toomela, 2014, p. 337). in essence, the dialectical relation implies a specific way of conceiving parts and whole: [the] ancient debate on emergence, whether indeed wholes may have properties not intrinsic to the parts, is beside the point. the fact is that the parts have properties that are characteristic of them only as they are parts of wholes; the properties come into existence in the interaction that makes the whole (levins & lewontin, 1985, p. 273). consequently, the dialectical ontology complies with substantialist and process metaphysics since it comprises both being—that is, “what there is”—and becoming—that is, “what is occurring” (cecez-kecmanovic, 2016). australasian journal of information systems taxén 2018, vol 22, research note reconfiguring sociomateriality from a neurobiological perspective 3 3.2 the substantialist aspect—the relata action requires that an infrastructure is in place before we can engage in any activity, including communication: there is “always something that exists first as a given, as an issue, as a problem” (latour, 2009, p. 5). such an infrastructure, which can be called as communal, comprises factors of three kinds: biological, social, and circumstantial (harris, 1996). biological factors concern the physical and mental capacities of the human being. a fundamental biological assumption is that brains evolved to control the activities of bodies in the world. the “mental is inextricably interwoven with body, world and action, where the mind consists of structures that operate on the world via their role in determining action” (love, 2004, p. 527). biological factors derive from neurobiological capacities developed during the phylogenetic evolution of humankind. such capacities, which every healthy human being is endowed with at birth, “are universal and inherent for all humans, independent of language and environmental conditions” (kotik-friedgut, 2006, p. 43). social factors are entities and practices that are relevant in a certain community. for example, the layout of score as in figure 1 has evolved over long periods in various musical activities. this artefact makes sense only in musical communities, as do the cello, norms for playing, musical schools, etc. circumstantial factors pertain to the particularity of situations. in the musical example, the musician’s playing skill is a biological factor, the layout of the musical score a social factor, and the presence of the player and his cello at the time of the concert a circumstantial factor. 3.3 the processual aspect—dialectics between relata the processual aspect, which can be referred to as communalization, is seen as a dynamical systems process (gibbs & cameron, 2008) ‘anchored’ by neural attractors in individual brains and communal attractors in communities. the neural anchoring implies that the individual is capable of preforming a multitude of potential actions, which are stabilized and constrained by communal anchoring, and thus considering “human action within its systemic anchoring” (boesch, 1991, p. 17) (see figure 2). external stimuli, such as someone uttering the word “cat” or the appearance of a cat (communal attractors), converge to the same neural attractor basin (e.g. “cat”) in the neural phase space of the individual. figure 2 the dialectical relation anchored by neural and communal attractors since every action changes both biological and social factors, the communal infrastructure is always in becoming and only temporally stabilized, however on vastly different timescales ”cat” ”cat” neural attractors communal attractors australasian journal of information systems taxén 2018, vol 22, research note reconfiguring sociomateriality from a neurobiological perspective 4 ranging from millions of years (biological evolutionary), cultural-historical time (communal), and millisecond (biological-neural). the communalization process can be illustrated by the musical example. the dynamics involved in transforming rostropovič into an outstanding cellist involves a long and arduous practicing with communal attractors such as the cello and the score. eventually, neural and communal attractors converged into a unity so tightly intertwined that rostropovič’s playing become virtually effortless: there no longer exist relations between us. some time ago i lost my sense of the border between us…. i experience no difficulty in playing sounds…. the cello is my tool no more (rostropovič, quoted in zinchenko, 1996, p. 295). therefore, action, whether individual or joint (as in the quartet example in figure 1), is from the outset social. joint action is based on individual action and “constituted by the fitting together of the lines of behaviour of the separate participants” (blumer, 1969, p. 70). 4 discussion any theory, which purports to elucidate the relation between the social and material in is theorizing, should be generally applicable and not limited to the is field. as the musical example shows, this is not the case for ssm. the inevitable conclusion is that the extreme relational ontology of ssm is untenable as a foundation of the is field. alternative ways out of this dilemma are discussed below. 4.1 weak sociomateriality the ‘weak’ variant of sm (wsm) rejects the “strong sociomaterialist claims that the existence of entities is wholly relational” (jones, 2014, p. 918). entities “exist independently of their enactment in practice, even while it is through relations between entities that agency can be explained” (cecez-kecmanovic et al., 2014, p.825). wsm concurs with the dialectical approach in several aspects. for instance, wsm may accommodate “the pre-existence of social structure” (jones, 2014, p. 919), which complies with the communal infrastructure. another point of adherence concerns the dynamics, where wsm claims that “sociomaterial entanglement may, in principle, be open to revision at any instant” (jones, 2014, p. 919). moreover, wsm can be used for inquiries in practical settings. nonetheless, wsm lacks a communicational theory and a clear conception of individuality. 4.2 imbrication the notion of ‘imbrication’ for conceptualizing the human-material interlocking aligns well with the dialectical approach. however, as leonardi (2011) concedes: “using the imagery of imbrices and tegulae has its problems. both tiles are made of “material” in the sense that they are physical creations of clay. the struggle to find a suitable image with which to describe the imbrication of human and material agencies points to the conceptual difficulty of integrating these phenomena” (leonardi, 2011, p. 151). this paper argues that the dialectical approach provides a solution to this dilemma, since one ‘tile’ is the neurobiology of the human being and another ‘tile’ the social conceived as communal attractors. australasian journal of information systems taxén 2018, vol 22, research note reconfiguring sociomateriality from a neurobiological perspective 5 4.3 further development 4.3.1 communication communication can be accommodated in the dialectical approach using the communication theory of integrational linguistics (e.g., harris, 1996) which suggests that any communicational act requires a communicational infrastructure comprising the same kinds of factors as the communal infrastructure. 4.3.2 is and information technology (it) artefacts is becomes socialized in the dialectical approach as communalized it artefacts, which means that the ‘system’ in is conceptualization includes the individual. in that sense, the it artefact becomes informative for the individual in communalization. moreover, it remains an artefact, possibly modified, throughout communalization. consequently, we “do not need to put humans inside the boundary of the it artefact in order to make these artefacts social” (goldkuhl, 2013, p. 94). 4.3.3 materiality the neurobiological point of departure implies that any external materiality, which our sensory system can perceive, may be communalized into a communal attractor, regardless of the particular physicality of sensations. the decisive point is whether that materiality signifies something relevant for individual or joint actions. to illustrate this point, consider the material basis of sheet music in figure 1. traditionally, this has been paper. now, the paper is gradually replaced by digital sheet music presented on tablets. however, the layout of the sheet music remains the same, which indicates that its composition is highly efficient with respect to our neurobiological capacities for acting in musical communities. 4.3.4 practice ssm holds that categories are constituently entangled—that is, only locally and temporarily possible to separate and stabilize for analytical purposes through ‘agential cuts’ (barad, 2003). however, how to perform this cut is problematic since categories are always indeterminate. this in turn aggravates the practical application of ssm. the dialectical approach, on the other hand, provides a distinct canvas for analysing and informing practice, where the individual, social and material are from the outset disentangled, although dynamically related. 5 conclusion the ssm relational ontology of agential realism is too extreme to be used as a basis for is theorizing. importantly, it is necessary to include substantialist elements without relapsing into the other extreme—that is, positivism. the dialectical approach is a potential solution to overcome those extreme positions. this requires the individual to be included as a prime constituent on the same par as the social and the material. the ‘inseparability’ of the social and material needs to be seen as an individual experience (as exemplified in the rostropovič statement above). concerning materiality, the sm tenet ‘all materiality is social’ needs to be reformulated as ‘any form of materiality may become social’, depending on whether it has undergone communalization or not. by reconstructing sm along the lines indicated, the ambition of sm to base is theorizing on a relational ontology can be kept, while simultaneously providing a foundation that is generally applicable and thereby making it intelligible to practitioners. australasian journal of information systems taxén 2018, vol 22, research note reconfiguring sociomateriality from a neurobiological perspective 6 references barad, k. (2003). posthumanist performativity: toward an understanding of how matter comes to matter. signs, 28(3), 801–831. doi: https://doi.org/10.1086/345321 blumer, h. (1969). symbolic interactionism: perspective and method. englewood cliffs, n.j.: prentice-hall. boesch, e. (1991). symbolic action theory and cultural psychology. berlin: springer. doi: https://doi.org/10.1007/978-3-642-84497-3 cecez-kecmanovic, d., galliers, r.d., henfridsson, o., newell, s., & vidgen r. (2014). the sociomateriality of information systems: current status, future directions. mis quarterly, 38(3), 809-830. doi: https://doi.org/10.25300/misq/2014/38:3.3 cecez-kecmanovic, d. (2016). from substantialist to process metaphysics – exploring shifts in is research. in lucas introna, donncha kavanagh, séamas kelly, wanda orlikowski, susan scott (eds.) beyond interpretivism? new encounters with technology and organization (pp. 35-57). ifip wg 8.2 working conference on information systems and organizations, is&o 2016. dublin, ireland, december 9–10, 2016. springer. doi: https://doi.org/10.1007/978-3-319-49733-4_3 gibbs, r.w., & cameron, l. (2008). the social-cognitive dynamics of metaphor performance. cognitive systems research, 9(1–2), 64–75. doi: https://doi.org/10.1016/j.cogsys.2007.06.008 goldkuhl, g. (2013). the it artefact: an ensemble of the social and the technical? – a rejoinder. systems, signs & actions, 7(1), 90-99. harris, r. (1996). signs, language, and communication: integrational and segregational approaches. london: routledge. hassan, n. r. (2016). editorial: a brief history of the material in sociomateriality. acm sigmis database: the database for advances in information systems, 47(4), 10-22. doi: https://doi.org/10.1145/3025099.3025101 israel, j. (1979). the language of dialectics and the dialectics of language. new york: humanities press. jones, m. (2014). a matter of li fe and death: exploring conceptualizations of sociomateriali ty in the context of cri tical care. mis quarterly , 38 (3), 895– 925. doi: https://doi .org/10.25300/misq/2014/38.3.12 kotik-friedgut, b. (2006). development of the lurian approach: a cultural neurolinguistic perspective. neuropsychology review, 16(1), 43-52. doi: https://doi.org/10.1007/s11065006-9003-9 latour, b. (2009). a cautious prometheus? a few steps toward a philosophy of design (with special attention to peter sloterdijk). in fiona hackne, jonathn glynne and viv minto (editors) proceedings of the 2008 annual international conference of the design history society – falmouth, 3-6 september 2009 (pp. 2-10). universal publishers. leonardi, p. m. (2011). when flexible routines meet flexible technologies: affordance, constraint, and the imbrication of human and material agencies. mis quarterly, 35(1), 147-167. doi: https://doi.org/10.2307/23043493 australasian journal of information systems taxén 2018, vol 22, research note reconfiguring sociomateriality from a neurobiological perspective 7 leonardi, p. (2013). theoretical foundations for the study of sociomateriality. information and organization, 23(2), 59-76. doi: https://doi.org/10.1016/j.infoandorg.2013.02.002 levins, r., & lewontin, rc. (1985). the dialectical biologist. cambridge, mass: harvard university press. love, n. (2004). cognition and the language myth. language sciences, 26(6), 525-544. doi: https://doi.org/10.1016/j.langsci.2004.09.003 orlikowski, w. j., & scott, s. v. (2008). sociomateriality: challenging the separation of technology, work and organization. the academy of management annals, 2(1), 433-474. doi: https://doi.org/10.1080/19416520802211644 paul, r. (2010). loose change. european journal of information systems, 19, 379–381. doi: https://doi.org/10.1057/ejis.2010.40 toomela, a. (2014). there can be no cultural-historical psychology without neuropsychology. and vice versa. in a. yasnitsky, r. van der veer & m. ferrari (eds.), the cambridge hand-book of cultural-historical psychology (pp. 315-349). cambridge: cambridge university press. doi: https://doi.org/10.1017/cbo9781139028097.019 zinchenko, v. (1996). developing activity theory: the zone of proximal development and beyond. in b. nardi (ed.) context and consciousness, activity theory and human-computer interaction (pp. 283-324). cambridge, massachusetts: mit press copyright: © 2018 taxén. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 limitations of ssm 3 neurobiological reconfiguration for sm 3.1 a dialectical ontology 3.2 the substantialist aspect—the relata 3.3 the processual aspect—dialectics between relata 4 discussion 4.1 weak sociomateriality 4.2 imbrication 4.3 further development 4.3.1 communication 4.3.2 is and information technology (it) artefacts 4.3.3 materiality 4.3.4 practice 5 conclusion paper title (paper title style) australasian journal of information systems adnan & islam 2017, vol 21, selected papers from ausdm forex++: knowledge discovery from decision forests 1 forex++: a new framework for knowledge discovery from decision forests md nasim adnan school of computing and mathematics charles sturt university madnan@csu.edu.au, zislam@csu.edu.au md zahidul islam school of computing and mathematics charles sturt university abstract decision trees are popularly used in a wide range of real world problems for both prediction and classification (logic) rules discovery. a decision forest is an ensemble of decision trees and it is often built for achieving better predictive performance compared to a single decision tree. besides improving predictive performance, a decision forest can be seen as a pool of logic rules (rules) with great potential for knowledge discovery. however, a standard-sized decision forest usually generates a large number of rules that a user may not able to manage for effective knowledge analysis. in this paper, we propose a new, data set independent framework for extracting those rules that are comparatively more accurate, generalized and concise than others. we apply the proposed framework on rules generated by two different decision forest algorithms from some publicly available medical related data sets on dementia and heart disease. we then compare the quality of rules extracted by the proposed framework with rules generated from a single j48 decision tree and rules extracted by another recent method. the results reported in this paper demonstrate the effectiveness of the proposed framework. keywords: decision tree; decision forest; random forest; knowledge discovery; dementia; heart disease 1 introduction due to their capabilities of expressing knowledge in a human-understandable form and dealing with high dimensional as well as redundant attributes, decision trees are applied in a wide range of real-world problems (murthy (1998), safavian and landgrebe (1991)). aside from those properties, decision trees are considered to be an unstable classifier as slight differentiation in a training data set can result in significant differences between decision trees obtained from the original and differentiated data sets. interestingly, an ensemble of classifiers is found to be effective for unstable classifiers such as decision trees (tan et al. (2005)). a decision forest is an ensemble of decision trees where an individual decision tree acts as the base classifier. the classification is performed by taking a vote based on the predictions made by each decision tree of the decision forest (tan et al. (2005)). a decision forest in general incorporates and extends most of the capabilities of decision trees, such as a decision forest is said to be more robust to noise(s) as well as more accurate compared to a single decision tree (bernard et al. (2008), polikar (2006), quinlan (1996a)). in order to achieve better ensemble accuracy a decision forest needs both accurate and diverse (in terms of classification errors) individual decision trees as base classifiers (adnan and islam (2016a), adnan and islam (2015a), tin kam (1998), polikar (2006)). an accurate decision tree can be generated by applying a decision tree algorithm such as c4.5 (quinlan (1993), quinlan (1996b)) on a training data set. however, a single decision tree can discover only one set of rules and thus may wrongly predict the class value of a test record which could have been predicted correctly by a more appropriate rule. a different decision tree can be obtained from a differentiated data set (such as a bootstrap sample) which may include a more appropriate rule for the given test record. if a decision forest contains a set of decision trees which are australasian journal of information systems adnan & islam 2017, vol 21, selected papers from ausdm forex++: knowledge discovery from decision forests 2 different from each other, then some of the trees may discover more appropriate rules for a set of test records while some other trees may discover for another set of test records, resulting in better generalization performance for the forest (adnan and islam (2016c)). besides, a decision forest can be seen as a pool of rules with great potential for knowledge discovery. the main challenge of knowledge discovery from a standard-sized decision forest (usually a 100-tree decision forest (adnan and islam (2015b)), adnan and islam (2015c), geurts et al. (2006)) comes from the enormous number of rules that it generates. for effective knowledge discovery, we need to extract accurate, generalized (meaning high in coverage), concise as well as surprising (previously unknown) rules (geng and hamilton (2006)). an obvious technique to extract a subset of rules is to apply some cut points based on accuracy, coverage or length of the rules. for example, we can extract only those rules that have accuracy >= 80%. however, any such cut points may react differently from data set to data set. as a result, for one data set a cut point may net more than manageable rules whereas for another data set the same cut point may acquire a few rules. to avoid such situation, a user may need to expedient on different cut points for a single variable (such as accuracy) and subsequently on all possible combinations of the variables for each data set which may not be manageable either. there are many subforest selection algorithms that prunes a number of decision trees from a decision forest while retaining or increasing ensemble accuracy (adnan and islam (2016c), lu et al. (2010), margineantu and dietterich (1997), martínez-muñoz et al. (2009), martínezmuñoz and suárez (2004), ruta and gabrys (2005)). it was shown in adnan and islam (2016c)) that if the number of trees in a subforest drops considerably, the ensemble accuracy also drops significantly. it was also shown that on an average the best subforest selection algorithm amassed around 45 trees from a 100-tree random forest (adnan and islam (2016c)). admittedly, subforest selection algorithms can shrink the size of a decision forest, yet the size remains large enough to hinder effective knowledge discovery. in addition, these algorithms do not consider any rule-level properties for selecting trees and thus trees selected by them may not contain rules that are comparatively more accurate, generalized and concise than others. similarly, there are some algorithms that intend to increase the comprehensibility of a decision forest by representing the whole decision forest into a single decision tree. in one algorithm (johansson et al. (2011)), the authors approximated the entire forest by selecting the nearest tree in terms of prediction results. undoubtedly this algorithm can increase the comprehensibility of a decision forest; however cannot exploit its knowledge discovery potential. extraction of rules from different types of classifiers is a popular area of research; in recent years a number of rule extraction methods have been proposed (mashayekhi and gras (2015), martens et al. (2008), schmitz et al. (1999)). in huysmans et al. (2006), the authors provided a wide range of survey on different rule extraction methods. most of those rule extraction methods were designed for some “black box” type classifiers such as artificial neural networks and support vector machines (martens et al. (2008), schmitz et al. (1999)). on the contrary, rule extraction from decision forests remains largely ignored (huysmans et al. (2006), mashayekhi and gras (2015)). some existing rule extraction techniques (liu et al. (2012), mashayekhi and gras (2015)) mainly prune forest rules in order to increase the prediction accuracy (just like pruning trees from a forest) and consequently do not solely focus on issues related to knowledge discovery. in order to facilitate effective knowledge discovery from decision forests, recently we have proposed a data set independent framework (forex) for extracting those rules that are comparatively more accurate and generalized (high in coverage) than others. forex has been accepted in a conference (ausdm 2016) (adnan and islam (2016b)) but we have never submitted/published any variant of forex in any journal. in this paper, we extend forex substantially to propose forex++ as follows.  in addition to accuracy and coverage, forex++ considers rule length as concise rules are said to be more comprehensible (geng and hamilton (2006)). on the other hand, forex does not consider rule length while extracting forest rules. australasian journal of information systems adnan & islam 2017, vol 21, selected papers from ausdm forex++: knowledge discovery from decision forests 3  a major limitation of forex is that its rules can be dominated by the majority class and thus may miss to net rules from minority class(es) (see section 3 for details). in order to oblige the presence of rules from minority class(es), forex++ considers accuracy, coverage and rule length independently for each class (see section 3 for details).  unlike forex, forex++ removes identical (exactly the same) rules from decision forests before extracting any rule. compounded by the randomness of decision forests, rules reported in this paper are substantially different from rules reported in our conference paper (adnan and islam (2016b)).  this paper involves wider experimentation and interesting knowledge/trends analysis. we apply forex++ on rules generated by two different decision forest algorithms from two different publicly available medical related data sets on dementia (oasis) and heart disease (lichman). forex++ rules are compared with the j48 (the weka implementation of c4.5 (quinlan (1993), quinlan (1996b)) rules and rules extracted by a recent method (mashayekhi and gras (2015)). furthermore, based on the rules extracted by forex++, both data sets are further explored using sql in order to find interesting knowledge/trends. on the other hand, in our conference paper (adnan and islam (2016b)), forex was applied on the dementia data set (not on the heart disease data set) and the extracted rules were compared only with rules generated from a j48 tree (hall et al. (2009)) (not with rules extracted by a recent method). no interesting knowledge/trends were reported in our conference paper. the remainder of this paper is organized as follows: in section 2 we introduce data set, decision tree, two different decision forest algorithms and the recent rule extraction method as background information. section 3 explains the proposed knowledge extraction framework. section 4 provides detailed description of the data sets used and the associated experimental results. finally, we offer some concluding remarks in section 5. 2 background information 2.1 data set a data set d is regarded as a two dimensional table with columns/attributes ({a1, a2, ..., am, c}) and rows/records ({r1, r2, ..., rn}). a data set can have two broad types of attributes: numerical (e.g. age and income) and categorical (e.g. city and degree). while a numerical attribute has a natural ordering among its domain values, a categorical attribute does not. out of all attributes, one categorical attribute is chosen to be the class attribute and all other attributes are termed as non-class attributes. a domain value of the class attribute ciϵc is the class label of a record ri and popularly termed as the class value. if minority records have a certain class value, the class value is termed as a minority class. conversely, if majority records have a certain class value, the class value is termed as the majority class. if there are more than two class values then one of them is the majority class and the others are minority classes. 2.2 decision tree a decision tree consists of nodes (denoted by rectangles) and leaves (denoted by ovals) as shown in figure 1. the node of a decision tree symbolizes a splitting event where the splitting attribute (label of the node) partitions a data set according to its domain values. as a result, a disjoint set of horizontal segments of the data set is generated and each segment contains one set of domain values of the splitting attribute. a leaf of a decision tree represents a horizontal segment of the data set where no further splitting is carried out. in this way, all records of a training data set are distributed among the leaves. the path from the root node to a leaf makes up a rule (i.e. pattern) that identifies a relationship between a set of non-class attributes (splitting attributes along the path) and the class values. for example, the rule for leaf 1 is “if degree = masters and income <= 85k → lecturer” as majority records (all four in this case) belonging to the segment represented by leaf 1 have the class value lecturer. here, “if degree = masters and income <=” is the antecedent of the rule australasian journal of information systems adnan & islam 2017, vol 21, selected papers from ausdm forex++: knowledge discovery from decision forests 4 and “lecturer” is the consequent. a decision tree is then used to predict the class values of unseen records of a testing data set for which the class values are unknown (i.e. records are unlabeled). based on the values of non-class attribute/s of an unlabeled record it passes through an antecedent to be predicted as the consequent (adnan and islam (2014)). figure 1: decision tree out of all rules obtained from a decision tree, some may have consequent from minority class(es) and some may have consequent from the majority class. in this paper, for simplification, “rules with consequent from minority class(es)” is called “rules from minority class(es)” and “rules with consequent from the majority class” is called “rules from the majority class”. 2.3 decision forest 2.3.1 random forest random forest (breiman (2001)) is regarded as a state-of-the-art decision forest algorithm (bernard et al. (2012), bernard et al. (2008)) which is technically a fusion of bagging (breiman (1996)) and random subspace (tin kam (1998)) algorithms. bagging (breiman (1996)) generates a new training data set di where the records of di are selected randomly from the original training data set d. a new training data set di contains the same number of records as in d. thus, some records of d can be selected multiple times and some records may not be selected at all. approximately, 63.2% of the original records are selected in a bootstrap sample and the remaining 36.8% records are repeated (han et al. (2011)). bagging generates a predefined number (t) of bootstrap samples (d1, d2, …, dt) using the above mentioned approach. a decision tree building algorithm is then applied on each bootstrap sample di (i = 1, 2, …, t) in order to generate altogether t number of trees for the forest. the random subspace algorithm (tin kam (1998)) algorithm randomly draws a subset of attributes (subspace) f from the entire attribute space m. f can be drawn either at the tree level or at the node level. when selected at the tree level, attributes in f remains the same for each node of a tree; on the other hand attributes in f may differ from one node to another in a tree when selected at the node level. the best attribute in f is determined to be the splitting attribute for the associated node. the random subspace algorithm is applied on the original training data set (not on bootstrap samples) for building decision trees. random forest in its simplest form uses bootstrap sampling and node level subspacing for generating decision trees ((breiman, 2001)). australasian journal of information systems adnan & islam 2017, vol 21, selected papers from ausdm forex++: knowledge discovery from decision forests 5 2.3.2 forest pa recently, we have proposed a new decision forest algorithm called “forest by penalizing attributes (forest pa)” (adnan and islam (2017)) that imposes penalties on attributes systematically in such a way that an attribute tested at lower level (such as in the root node) receives higher penalty (lower weight) than an attribute tested at higher level. the reason is that an attribute tested at lower level may influence more rules than an attribute tested at higher level as discussed earlier. thus in order to discover diverse set of rules, attributes tested at lower levels are supposed to be avoided in a future tree more seriously than those attributes that are tested at higher levels. furthermore, to increase the chance of having different weights among attributes in the same level, forest pa randomly selects the weight of an attribute from the weight-range (wr) allocated for the attribute’s level (adnan and islam (2017)). forest pa also has a mechanism to gradually increase weights (withdraw penalties) from the attributes that have not been tested in the subsequent trees. this addresses the situation where all good attributes (attributes with high classification capacity) suffer from high penalties and thus poor quality trees can be generated at some later stage of the forest building process. forest pa also uses bootstrap samples of the original training data set (breiman (1996)) to ensure further diversity. 2.4 the rf+hc method similar to finding the best subforest, finding the best subset of rules by exhaustive search over any criteria is an np-hard problem (adnan and islam (2016c), mashayekhi and gras (2015)). in order to avoid such computational burden, rule extraction problem can be solved by a heuristic or a hill climbing approach as suggested in the rf+hc method (mashayekhi and gras (2015)). in doing so, the rf+hc method first ranks all rules using equation 1. rule_score = 𝑐𝑐−𝑖𝑐 𝑐𝑐+𝑖𝑐 + 𝑐𝑐 𝑖𝑐+𝑘 + 𝑐𝑐 𝑟𝑙 equation 1 here, cc is the number of records that is covered and correctly classified by a rule. ic is the number of records that is covered and incorrectly classified by the same rule. we see, a positive constant k is used at the denominator of the second term in equation 1. here, k is used to prevent the denominator to become zero when all records covered by a rule are classified correctly (i.e. ic = 0). in the rf+hc method, the value of k is set to 4 (although, any value can be used instead). equation 1 also incorporates the influence of rule length to prioritize concise rules as they are more preferred (geng and hamilton (2006)). the denominator of the third term rl (which stands for rule length) of the equation imposes more penalties for longer rules. now, as suggested in the rf+hc method, a subset of rules can be selected using a heuristic or a hill climbing approach. in the heuristic approach, rules are sorted based on their individual ranks. then, a user defined number of top rules are extracted according to the ranks. in the hill climbing approach, at first a subset of rules are probabilistically selected on the basis of their ranks (meaning a rule with higher rank has greater chance to be selected than a rule with lower rank). then, each time one rule from the rest is added in the subset and if the change increases the overall accuracy then the change is retained (mashayekhi and gras (2015)). 3 the proposed rule extracting framework 3.1 basic concepts the path from the root node to a leaf node makes up the antecedent and the majority class of the leaf node is the consequent of the rule in a decision tree. for example, the rule for leaf 2 in figure 1is “if degree = masters and income >85k→ class value = asst. professor” as majority records (lecturer: 1, asst. professor: 3) belonging to leaf 2 have the class value asst. professor. the minority record(s) are viewed as if they are misplaced under a rule. thus, the australasian journal of information systems adnan & islam 2017, vol 21, selected papers from ausdm forex++: knowledge discovery from decision forests 6 accuracy (acc) of the rule for leaf 2 is: 3/4 = 0.75. all records of a data set from which a decision tree is built are distributed among the leaves. the coverage (cov) of a rule indicates the proportion of records that fall in the leaf to the total records of the data set. for example, cov of the rule for leaf 2 is: 3/11 = 0.27. acc and cov are independent of each other and thus no data set independent relationships exist between them. in general, acc interprets the reliability of a rule whereas cov interprets its generality (geng and hamilton (2006)) and thus both of them are important for a rule. for example, a lowcoverage rule is likely to be uninteresting from a business perspective as it may occur simply by chance even if the rule is highly reliable (tan et al. (2005)). hence, both acc and cov need to be considered when extracting a subset of rules. however, in literature we find no method considering both acc and cov for extracting subsets of rules from decision forests (huysmans et al. (2006)). the recently proposed rf+hc method (mashayekhi and gras (2015)) considers the number of correctly and incorrectly classified records that are covered by a rule (in effect, accuracy) and the rule length for calculating a score for the rule (see equation 1). hence, the influence of coverage is not directly considered for calculating scores in the rf+hc method. 3.2 priorities for the proposed rule extracting framework 3.2.1 rule extraction from each class we understand that no data set independent relationships exist between acc and cov; however they are independently comparable between themselves. based on this principle, recently we have proposed a rule extraction framework for decision forest forex (adnan and islam (2016b)) that finds the set of rules 𝐑𝐴𝑣𝑔 𝐴𝑐𝑐 that are more (or equally) accurate than the average accuracy 𝐴𝑐𝑐𝐴𝑣𝑔 of all rules ({r = r1, r2, …, rz}) in a forest as follows, where 𝑅𝑖 𝐴𝑐𝑐 is the accuracy of ri. 𝐑𝐴𝑣𝑔 𝐴𝑐𝑐 = {𝑅𝑖: 𝑅𝑖 𝐴𝑐𝑐 ≥ 𝐴𝑐𝑐𝐴𝑣𝑔}|𝐴𝑐𝑐𝐴𝑣𝑔 = 1 |𝐑| ∑ 𝑅𝑗 𝐴𝑐𝑐 |𝐑| 𝑗=1 equation 2 similarly, forex (adnan and islam (2016b)) that finds the set of rules 𝐑𝐴𝑣𝑔 𝐴𝑐𝑐 that have more (or equal) coverage than the average coverage 𝐶𝑜𝑣𝐴𝑣𝑔 of all rules in a forest as follows, where 𝑅𝑖 𝐶𝑜𝑣 is the coverage of ri. 𝐑𝐴𝑣𝑔 𝐶𝑜𝑣 = {𝑅𝑖: 𝑅𝑖 𝐶𝑜𝑣 ≥ 𝐶𝑜𝑣𝐴𝑣𝑔}|𝐶𝑜𝑣𝐴𝑣𝑔 = 1 |𝐑| ∑ 𝑅𝑗 𝐶𝑜𝑣 |𝐑| 𝑗=1 equation 3 now, by applying both equation 2 and equation 3, forex is able to recognize those high quality rules that have simultaneously more (or equal) accuracy and coverage than their averages in a forest (see equation 4). 𝐑𝐴𝑣𝑔 𝐴𝑐𝑐.𝐶𝑜𝑣 = 𝐑𝐴𝑣𝑔 𝐴𝑐𝑐 ∩ 𝐑𝐴𝑣𝑔 𝐶𝑜𝑣 equation 4 from equation 3 and equation 4 we understand that forex rules can be dominated by the majority class and thus may miss to net rules from minority class(es). the reason is: the records with class values from minority class(es) can be far less than the records with class values from the majority class and thus cov of rules from minority class(es) may fall short of 𝐶𝑜𝑣𝐴𝑣𝑔. as a result, rules from minority class(es) may not qualify for 𝐑𝐴𝑣𝑔 𝐴𝑐𝑐.𝐶𝑜𝑣 making forex ineffective for extracting interesting knowledge from imbalanced data (see equation 4). australasian journal of information systems adnan & islam 2017, vol 21, selected papers from ausdm forex++: knowledge discovery from decision forests 7 as an improvement to forex, in this paper we propose forex++ that obliges the presence of rules from minority class(es) by applying equation 2 and equation 3 separately on rules from each class. forex++ finds the set of rules 𝐑𝐴𝑣𝑔,𝑐𝑘 𝐴𝑐𝑐 that are more (or equally) accurate than the average accuracy 𝐴𝑐𝑐𝐴𝑣𝑔,𝑐𝑘 of all rules with the ck class value (𝐑𝐶𝑘 = {rh, ri, …, ry}) in a forest as follows. note that 𝑅𝑗,𝑐𝑘 𝐴𝑐𝑐 represents the accuracy of the j-th rule with the ck class value (i.e. rj∈ 𝐑𝐶𝑘 ) and |𝐑𝐶𝑘 | expresses the number of rules with the ck class value. 𝐑𝐴𝑣𝑔,𝑐𝑘 𝐴𝑐𝑐 = {𝑅𝑖: 𝑅𝑖,𝑐𝑘 𝐴𝑐𝑐 ≥ 𝐴𝑐𝑐𝐴𝑣𝑔,𝑐𝑘 }|𝐴𝑐𝑐𝐴𝑣𝑔,𝑐𝑘 = 1 |𝐑𝑐𝑘 | ∑ 𝑅𝑗,𝑐𝑘 𝐴𝑐𝑐 |𝐑𝑐𝑘 | 𝑗=1 equation 5 similarly, forex++ finds the set of rules 𝐑𝐴𝑣𝑔,𝑐𝑘 𝐶𝑜𝑣 that have more (or equal) coverage than the average coverage 𝐶𝑜𝑣𝐴𝑣𝑔,𝑐𝑘 of all rules with the ck class value 𝐑𝐶𝑘 = {rh, ri, …, ry}) in a forest as follows. 𝐑𝐴𝑣𝑔,𝑐𝑘 𝐶𝑜𝑣 = {𝑅𝑖: 𝑅𝑖,𝑐𝑘 𝐶𝑜𝑣 ≥ 𝐶𝑜𝑣𝐴𝑣𝑔,𝑐𝑘 }|𝐶𝑜𝑣𝐴𝑣𝑔,𝑐𝑘 = 1 |𝐑𝑐𝑘 | ∑ 𝑅𝑗,𝑐𝑘 𝐶𝑜𝑣 |𝐑𝑐𝑘 | 𝑗=1 equation 6 3.2.2 consideration of rule length the rf+hc method incorporates the influence of rule length to prioritize concise rules using the term 𝑐𝑐 𝑟𝑙 in equation 1. in 𝑐𝑐 𝑟𝑙 , rl ensures more penalty for a longer rule; however the penalty can be suppressed cc (cc is the number of records that is covered and correctly classified by a rule). for example, the score from the term 𝑐𝑐 𝑟𝑙 for rule 1 with cc = 40 and rl = 10 will be more (better) than that of rule 2 with cc = 15 and rl = 5 even when the length of rule 1 is double and rule 2 is more accurate with no incorrect classification (ic = 0). in such scenarios, it is really debatable to establish any direct relationship between cc and rl without considering other related variable (e.g. ic). on the other hand, forex++ prioritizes the set of concise rules 𝐑𝐴𝑣𝑔,𝑐𝑘 𝐿𝑒𝑛 that have less (or equal) length than the average length 𝐿𝑒𝑛𝐴𝑣𝑔,𝑐𝑘 of all rules with the ck class value (𝐑𝐶𝑘 = {rh, ri, …, ry}) in a forest as follows. 𝐑𝐴𝑣𝑔,𝑐𝑘 𝐿𝑒𝑛 = {𝑅𝑖: 𝑅𝑖,𝑐𝑘 𝐿𝑒𝑛 ≤ 𝐿𝑒𝑛𝐴𝑣𝑔,𝑐𝑘 }|𝐿𝑒𝑛𝐴𝑣𝑔,𝑐𝑘 = 1 |𝐑𝑐𝑘 | ∑ 𝑅𝑗,𝑐𝑘 𝐿𝑒𝑛 |𝐑𝑐𝑘 | 𝑗=1 equation 7 3.3 rule extraction by forex++ by applying equation 5, equation 6 and equation 7, forex++ is able to recognize those high quality rules that have simultaneously more (or equal) accuracy, coverage and conciseness than their averages for each class in a forest (see equation 8 and equation 9). 𝐑𝐴𝑣𝑔,𝑐𝑘 𝐴𝑐𝑐.𝐶𝑜𝑣.𝐿𝑒𝑛 = 𝐑𝐴𝑣𝑔,𝑐𝑘 𝐴𝑐𝑐 ∩ 𝐑𝐴𝑣𝑔,𝑐𝑘 𝐶𝑜𝑣 ∩ 𝐑𝐴𝑣𝑔,𝑐𝑘 𝐿𝑒𝑛 equation 8 australasian journal of information systems adnan & islam 2017, vol 21, selected papers from ausdm forex++: knowledge discovery from decision forests 8 𝐑𝐹𝑜𝑟𝐸𝑥++ = ⋃ 𝐑𝐴𝑣𝑔,𝑐𝑘 𝐴𝑐𝑐.𝐶𝑜𝑣.𝐿𝑒𝑛 𝑐𝑘∈𝐶,∀𝑐𝑘 equation 9 3.4 steps of forex++ the major steps of forex++ are organized as follows. forex++ is further illustrated through algorithm 1. step 1: in the first step, forex++ removes identical (exactly the same) rules from r (see step 1 of algorithm 1). step 2: in the second step, forex++ selects three sets of rules from each class based on accuracy, coverage and rule length using equation 5, equation 6 and equation 7. it then selects the intersection of the three sets of rules using equation 8 (see step 2 of algorithm 1). step 3: in the third step, forex++ accumulates rules for all classes using equation 9 (see step 3 of algorithm 1). algorithm 1: forex++ input: forest rules r. output: forex++ rules rforex++. begin rforex++← ∅; k ← |c|; /* |c| is the number of distinct classes */ step 1: r← remove_identical_rules(r); end step 1 step 2: for i =1 to k do 𝐑𝐴𝑣𝑔,𝑐𝑖 𝐴𝑐𝑐 ← ∅; 𝐑𝐴𝑣𝑔,𝑐𝑖 𝐶𝑜𝑣 ← ∅; 𝐑𝐴𝑣𝑔,𝑐𝑖 𝐿𝑒𝑛 ← ∅; 𝐑𝐶𝑖 ← get_rules_by_class(r,ci); /* 𝐑𝐶𝑖 is the set of rules with the ci class value */ /* implementation of equation 5 */ for j = 1 to |𝐑𝐶𝑖 | do 𝑅𝑐𝑖 𝐴𝑐𝑐← 𝑅𝑐𝑖 𝐴𝑐𝑐 + get_accuracy(𝑅𝑗,𝐶𝑖 ); /* 𝑅𝑗,𝐶𝑖 is the j-th rule of 𝐑𝐶𝑖 */ end for 𝐴𝑐𝑐𝐴𝑣𝑔,𝑐𝑖 ← 1 |𝐑𝐶𝑖 | × 𝑅𝑐𝑖 𝐴𝑐𝑐; for j = 1 to |𝐑𝐶𝑖 | do if get_accuracy(𝑅𝑗,𝐶𝑖 ) ≥ 𝐴𝑐𝑐𝐴𝑣𝑔,𝑐𝑖 then 𝐑𝐴𝑣𝑔,𝑐𝑖 𝐴𝑐𝑐 ← 𝐑𝐴𝑣𝑔,𝑐𝑖 𝐴𝑐𝑐 ∪ 𝑅𝑗,𝐶𝑖 ; end for /* implementation of equation 6 error! reference source not found.*/ for j = 1 to |𝐑𝐶𝑖 | do 𝑅𝑐𝑖 𝐶𝑜𝑣← 𝑅𝑐𝑖 𝐶𝑜𝑣 + get_coverage(𝑅𝑗,𝐶𝑖 ); end for 𝐶𝑜𝑣𝐴𝑣𝑔,𝑐𝑖 ← 1 |𝐑𝐶𝑖 | × 𝑅𝑐𝑖 𝐶𝑜𝑣; for j = 1 to |𝐑𝐶𝑖 | do if get_coverage(𝑅𝑗,𝐶𝑖 ) ≥ 𝐶𝑜𝑣𝐴𝑣𝑔,𝑐𝑖 then 𝐑𝐴𝑣𝑔,𝑐𝑖 𝐶𝑜𝑣 ← 𝐑𝐴𝑣𝑔,𝑐𝑖 𝐶𝑜𝑣 ∪ 𝑅𝑗,𝐶𝑖 ; end for /* implementation of equation 7 */ for j = 1 to |𝐑𝐶𝑖 | do 𝑅𝑐𝑖 𝐿𝑒𝑛← 𝑅𝑐𝑖 𝐿𝑒𝑛 + get_rule_length(𝑅𝑗,𝐶𝑖 ); end for 𝐿𝑒𝑛𝐴𝑣𝑔,𝑐𝑖 ← 1 |𝐑𝐶𝑖 | × 𝑅𝑐𝑖 𝐿𝑒𝑛; for j = 1 to |𝐑𝐶𝑖 | do australasian journal of information systems adnan & islam 2017, vol 21, selected papers from ausdm forex++: knowledge discovery from decision forests 9 if get_rule_length (𝑅𝑗,𝐶𝑖 ) ≤ 𝐿𝑒𝑛𝐴𝑣𝑔,𝑐𝑖 then 𝐑𝐴𝑣𝑔,𝑐𝑖 𝐿𝑒𝑛 ← 𝐑𝐴𝑣𝑔,𝑐𝑖 𝐿𝑒𝑛 ∪ 𝑅𝑗,𝐶𝑖 ; end for /* implementation of equation 8 */ 𝐑𝐴𝑣𝑔,𝑐𝑖 𝐴𝑐𝑐.𝐶𝑜𝑣.𝐿𝑒𝑛 ← 𝐑𝐴𝑣𝑔,𝑐𝑖 𝐴𝑐𝑐 ∩ 𝐑𝐴𝑣𝑔,𝑐𝑖 𝐶𝑜𝑣 ∩ 𝐑𝐴𝑣𝑔,𝑐𝑖 𝐿𝑒𝑛 ; end for end step 2 step 3: /* implementation of equation 9 */ for i = 1 to k do rforex++← ⋃ 𝐑𝐴𝑣𝑔,𝑐𝑖 𝐴𝑐𝑐.𝐶𝑜𝑣.𝐿𝑒𝑛; end for end step 3 return rforex++; end 4 experiments the experimentation is conducted on two different publicly available medical related data sets on dementia (oasis) and heart disease (lichman). we now provide detailed description of the data sets and their associated experimental results in the following sections. 4.1 dementia dementia is a medical term linked with memory loss which is severe enough to complicate daily life (dementia). alzheimer’s disease (ad) is the most common cause of dementia, accounting for almost 70% of all dementia cases (dem). ad is caused by abnormal deposition of proteins in the brain that destroys cells in those areas of the brain (i.e. cerebral cortex) that are responsible for memories, thoughts, actions and personality. unfortunately, ad is degenerative (destroys brain cells that causes to shrink the size of the brain over time), progressive (gradual decline of functioning of effected areas of the brain) and thus irreversible (dem).vascular dementia (vad) is mainly caused by full or partial blocking of arteries in the brain from deposits of fats, dead cells, and other debris that ultimately disrupts the blood flow. vascular dementia is often related to high blood pressure, high cholesterol, heart disease, diabetes, and other related conditions. treating those conditions can slowdown the progress of vascular dementia, but as usual the brain functions that are lost are not recoverable. unlike in ad, in vad the size the brain may not shrink at all (dem). 4.2 data set on dementia in this paper, we use a data set named “oasis: longitudinal mri data in nondemented and demented older adults” that consists of a collection of 354 observations (records) on 142 subjects (patients) aged between 60 to 98 and all the observations have one of the three class values: demented, nondemented and converted (oasis). converted class value refers to the patients that develop dementia during the period of the data collection. in addition to mri data, the data set also includes information about patient’s gender, age, education level and socio-economic status. some scores on dementia-related examinations are also included. intable 1, we present the attributes of the oasis data set and explain their meanings with their respective value ranges in detail. australasian journal of information systems adnan & islam 2017, vol 21, selected papers from ausdm forex++: knowledge discovery from decision forests 10 attributes explanation mri id the unique number of tests (354 in total). subject id the number of unique patients (142 in total). one patient may be visiting multiple times for mri tests, so the number of mri tests (354) is larger than the number of subjects. visit chronological visit number of a patient. mr delay the delay since the last visit. gender male (m) or female (f). hand right-handed (r) or left-handed (l). age ages of the patients vary between 60 to 98. educ education level of the patients vary between 6 to 23 representing years of education. ses socio-economic status of the patients assigned through the hollingshead index of social position. 1 representing the highest status to 5 representing the lowest status (hollingshead (1975)). mmse mini-mental state examination value ranges between 0 to 30. in mmse, a health professional asks a patient a series of questions designed to test a range of everyday mental skills. the questions mainly cover preliminary arithmetic problems, simple memory tests, and recognition of different orientations of objects. a score of 20 to 24 suggests mild dementia, 13 to 20 suggests moderate dementia, and less than 12 indicates severe dementia ((dementia), folstein et al. (1975)). cdr clinical dementia rating. 0 indicates no dementia, 0.5 indicates very mild dementia, 1 indicates mild dementia and 2 indicates moderate dementia (morris (1993)). etiv estimated total intra-cranial volume (in cm3) of the brain (proportional to the size of the skull, can be obtained from mri image) (buckner et al. (2004)). nwbv normalized whole-brain volume, expressed as a percent of all voxels (can be obtained from mri image) (buckner et al. (2005)). asf atlas scale factor is the volume scaling factor for brain size (proportional to nwbs and etiv (buckner et al. (2004)). distinct class values three distinct class values. demented, nondemented and converted. table 1: description of the oasis data set 4.3 experimentation on the oasis data set in (ertek et al. (2014)), the authors applied j48 (hall et al. (2009)) on a modified version of oasis data set in order to generate a decision tree. from the original oasis data set, at first the authors excluded the identifier attributes (“mri id” and “subject id”) and then built a preliminary decision tree. however, the first split of the tree based on the attribute “cdr” perfectly distinguished the demented patients (when cdr = 1) from others (non-demented and converted). this showed that cdr was too good attribute to be included in the analysis. furthermore, attributes “mr delay” and “visit” were found strongly dependent on cdr in the preliminary tree as the following splits were based on them (ertek et al. (2014)). observing the “near perfect” results in the preliminary decision tree due to cdr and the inherent dependency problem of “mr delay” and “visit”, the authors (ertek et al. (2014)) excluded them to generate the final decision tree. table 2 presents the rules generated from the final j48 decision tree as reported in ertek et al. (2014) with their respective accuracies (except we prune a few lengthy rules to contain at most australasian journal of information systems adnan & islam 2017, vol 21, selected papers from ausdm forex++: knowledge discovery from decision forests 11 five attributes in their antecedents for better understanding). we also report the coverages of the rules of the final j48 decision tree even though they were not reported in ertek et al. (2014). demented, nondemented and converted are abbreviated as d, nd and c respectively to be presented in subsequent tables. also antecedent, consequent, accuracy and coverage are abbreviated as ante, cons, acc and cov respectively. ante cons acc cov if mmse <= 26 d 94% 24% if mmse > 28 and gender = f nd 85% 36% mmse > 28 and gender = m and educ and educ <= 13 nd 79% 5% if mmse > 28 and gender = m and educ between 13 and 15 d 86% 2% if mmse > 28 and gender = m and educ <= 13 nd 80% 5% if mmse > 28 and gender = m and educ > 15 and asf <= 0.93 d 80% 1% if mmse > 28 and gender = m and educ > 15 and asf > 0.93 nd 69% 7% if mmse between 27 and 28 and gender = m and nwbv <= 0.68 nd 80% 1% if mmse between 27 and 28 and gender = m and nwbv > 0.68 d 62% 8% if mmse between 27 and 28 and gender = f and nwbv <= 0.71 d 63% 2% if mmse between 27 and 28 and gender = f and nwbv > 0.71 nd 66% 9% table 2: rules from j48 decision tree both random forest and forest pa use bootstrap samples of a training data set. bootstrap samples are generally used for inducing diversity among the base classifiers (martínez-muñoz and suárez (2010), quinlan (1996a)). approximately, 63.2% of the original records are selected and the remaining 36.8% records are repeated in a bootstrap sample (han et al. (2011)). as a result of this deviation, bootstrap samples cannot be regarded as a valid source of knowledge. therefore, for an even comparison among j48, random forest and forest pa, we do not use bootstrap samples for both random forest and forest pa. as a result, random forest is converted to random subspace (rs) (tin kam (1998)) and forest pa is converted to forest pa without bootstrap samples (forest pa wobs) (adnan and islam (2017)). after generating rules from both the forests, we first remove identical rules from them and then apply forex++ and the rf+hc method for extracting subsets of rules. for a fair comparison, we apply heuristic approach for the rf+hc method to select the same number of top-ranked rules (in terms of rule score) as extracted by forex++. for example, if forex++ extracts 50 rules, we select the top-ranked 50 rules for the rf+hc method. similarly, a 20-tree forest pa wobs generates as many as 658 rules from oasis, a cut point with accuracy >= 95% amasses 461 rules from them, whereas forex++ extracts as low as 55 rules. thus, 55 top-ranked rules are selected for the rf+hc method from forest pa wobs. though smaller in number, it is difficult to accommodate every forex++ and rf+hc rules for detailed comparison with each other and j48 rules. hence, we present a brief comparison among them in table 3. criteria j48 rules rs forest pa wobs rf+hc rules forex++ rules rf+hc rules forex++ rules total rules 11 62 62 55 55 average accuracy 76.73% 95.24% 97.84% 92.28% 98.06% average coverage 9.09% 5.75% 7.10% 5.62% 5.50% average rule length 3.36 4.52 3.98 4.62 4.27 distinct root nodes 1 3 4 3 3 distinct class values 2 2 3 2 3 table 3: comparison among j48, rf+hc and forex++ rules from the oasis data set from table 3, we see that j48 rules have more average cov than rf+hc and forex++ rules and this is due to a low number of rules (11) generated by the j48 tree from oasis. however, australasian journal of information systems adnan & islam 2017, vol 21, selected papers from ausdm forex++: knowledge discovery from decision forests 12 average acc of j48 rules is significantly lower than that of rf+hc and forex++ rules. besides, all j48 rules start with the same attribute (mmse) and this may restrict their ability to explore different angles for knowledge discovery. furthermore, none of these j48 rules have consequent with the “converted” class value (a minority class in oasis) and thus provide no knowledge about those patients who developed dementia during the period of the data collection. rf+hc rules offer a major improvement over j48 rules in terms of average acc with competitive average cov for both decision forests. also, a number of different root attributes (3) are present in rf+hc rules to facilitate broader knowledge discovery. however, similar to j48 rules, none of the rf+hc rules have consequent with the “converted” class value. on the other hand, forex++ rules have the highest average acc with competitive average cov for both decision forests and they outperform rf+hc rules in every criteria with better average acc, average cov (for one out of two decision forests), average len, distinct root nodes and distinct class values. more importantly, forex++ rules can accommodate consequent with the “converted” class value. as forex++rules from forest pa wobs are found to be the most accurate (see table 3), we intend to go deeper into those rules in order to discuss some of the interesting trends that exist in them as follows. trend 1: we know, lower the mini-mental state examination (mmse) value, higher the chance of dementia. forex++ rules show us that the impact of mmse on dementia can be influenced by other attributes. for instance, when mmse <= 26, 94% of the patients are found to be demented (rule acc: 94%, cov: 23%). when nwbv is considered with mmse (mmse <= 26 and nwbv <= 0.75), acc increases to 97% with 21% cov. similarly, “if ses = 2 and educ > 12 and mmse <= 26 then demented (acc: 100%, cov: 3%)”. besides, forex++ nets an interesting rule with mmse > 27 but having consequent “demented”. the rule is presented as follows.  if mmse > 27 and gender = m and educ <= 15 and age > 74 and etiv <= 1498.9 then d (rule acc: 100%, cov: 2%). based on the interesting rule, we now explore the oasis data set using sql in order to find some contra (adjacent) rules. the attributes in the antecedent of a contra rule remains the same as the original rule except some of the conditions are flipped. some of the contra rules of the interesting rule are presented in table 4 (flipped conditions are shown in bold+italic). ante cons acc cov if mmse > 27 and gender = f and educ <= 15 and age > 74 and etiv <= 1498.9 nd 73% 11% if mmse > 27 and gender = f and educ > 15 and age > 74 and etiv <= 1498.9 nd 91% 9% table 4: some of the contra rules from the top rule of table 4 we see that when gender = f (instead of m), patients are not demented. this implies that females are less prone to dementia than their male counterparts when all other conditions remain the same. in line with this proposition we again apply sql on the oasis data set and find that “if gender = f then nd (acc: 63%, cov: 58%)” and “if gender = m then d (acc: 51%, cov: 42%)” which further validates the proposition. the bottom rule of table 4 indicates that females with higher educational background are very less likely to be demented. trend 2: similar to mmse, with the decrease of normalized whole-brain volume (nwbv) the chance of dementia increases. from the oasis data set (oasis), we see that when nwbv <= 0.73, patients generally get demented. however, with lower socio-economic status (ses = 3, 4 or 5) patients can be demented with nwbv as high as 0.78. a related rule extracted by forex++ is presented below. australasian journal of information systems adnan & islam 2017, vol 21, selected papers from ausdm forex++: knowledge discovery from decision forests 13  if ses = 4 and age <= 72 and nwbv <= 0.78 and etiv > 1454.86 then d (acc: 100%, cov: 2%). based on the related rule, we explore the oasis data set using sql and present the percentage of demented patients with high and low ses for different nwbv in figure 2. from figure 2 we see that the impact of nwbv on dementia can be greatly exacerbated by ses. with lower nwbv and ses, patients are more prone to dementia. besides, patients with higher ses are less likely to be demented compared to those with lower ses when nwbv remain the same. trend 3: with ses = 3, 4 or 5 (meaning lower socio-economic status) we find many rules having consequent “demented”. however, with ses = 1 (meaning the highest socio-economic status), patients can be demented in comparatively early ages. the related rule is:  if nwbv <= 0.73 and ses = 1 and age <= 70 then d (acc: 100%, cov: 3%). to validate the proposition we apply sql on the oasis data set and find that the average age to be demented for patients with higher socio-economic status (ses = 1 or 2) is 75 years whereas the average age for demented patients with comparatively lower socio-economic status (ses = 3, 4 or 5) is 77 years. figure 2: percentage of demented patients for different ses and nwbv trend 4: similar to trend 2, with nwbv <= 0.73 patients generally get demented. however, when education (educ) is high (i.e. > 15) and ses = 2 and asf <= 1.24 and age > 66, patients with nwbv <= 0.73 are found not demented (acc: 100%, cov: 5%). thus, we reason that patients with higher educational background and socio-economic status have a lower chance to be demented than others. we then apply sql on the oasis data set and find the following rules (without having any dementia or mri related attributes) to hold the same reasoning.  if educ > 16 and ses in (1, 2) then nd (acc: 67%, cov: 24%).  if educ <= 16 and ses in (3, 4, 5) then d (acc: 48%, cov: 45%). another similar rule highlighting the importance of educ is:  if nwbv <= 0.73 and educ > 16 and asf > 0.92 and age between 66 and 78 then nd (acc: 100%, cov: 4%). this rule indicates that even when the brain volume decreases (nwbv <= 0.73) patients may not be demented if they are highly educated (educ > 16). by applying sql, we see that the average brain volume of non-demented patients with higher educational background (educ >16) is 0.73 and for comparatively lower educational background (educ <= 16) is 0.75, which verifies the indication. australasian journal of information systems adnan & islam 2017, vol 21, selected papers from ausdm forex++: knowledge discovery from decision forests 14 from trend 1 and trend 4, it is understandable that educational background plays an important role on dementia in conjunction with gender, ses and nwbv. we now explore the impact of different education levels on dementia independent of any other attributes and present them in figure 3. figure 3 clearly shows that higher the educational background, lower the chance of dementia. trend 5: patients are found “converted” when nwbv <= 0.72 even if they have the highest socio-economic status (ses = 1) (acc: 100%, cov: 1%). the related rule is: “if nwbv <= 0.72 and ses = 1 and age > 70 and educ <= 16 then converted. this indicates that with very small brain volume, patients can develop dementia very quickly. by applying sql, we find that out of 37 converted patients, 22 of them have brain volume <= 0.72. figure 3: percentage of patients for different educ 4.4 heart disease cardiovascular disease (or heart disease) remains one of the biggest causes of deaths around the world. each year about 30% of all deaths worldwide (15.6 million deaths) occur from heart diseases which is more than all communicable, neonatal, maternal and nutritional disorders combined and double the number of deaths caused by cancers (nichols et al. (2014)). moreover, the death from heart disease is predicted to rise approximately 23.4 million by 2030 with heart disease to remain as the leading cause of death for many years to come (nichols et al. (2014)). following global trends, heart disease is the first leading cause of death in australia amounting 33.7% of all deaths (shouman et al. (2011)). motivated by these facts, we now facilitate exploring different aspects on heart disease from available data for better understanding the disease. 4.4.1 data set on heart disease in this paper, we use a data set named “statlog (heart)” that consists of a collection of 270 observations (records) and all the observations are distributed among two class values namely absence (a) or presence (p) of heart disease (lichman). in literature (el-bialy et al. (2015)), the same data set was used for feature analysis of coronary artery diseases. the data set includes information about patient’s gender, age, chest pain type, blood pressure along with some medical test information. in table 5, we present the attributes of the statlog (heart) data set and explain their meanings in detail. australasian journal of information systems adnan & islam 2017, vol 21, selected papers from ausdm forex++: knowledge discovery from decision forests 15 attributes explanation age ages of the patients vary between 29 to 77. gender male (m) or female (f). chest pain type 1 for typical angina (angina is a medical term for chest pain), 2 for atypical angina, 3 for non-anginal pain, 4 for asymptomatic pain. resting blood pressure (systolic) resting blood pressure in mm hg of patients when admitted to the hospital. serum cholesterol serum cholesterol in mg/dl. fasting blood sugar if fasting blood sugar > 120 mg/dl then 1 else 0. resting electrocardiographic results 0 for normal, 1 for having st-t wave abnormality (t wave inversions and/or st elevation or depression of > 0.05 mv), 2 for showing probable or definite left ventricular hypertrophy by estes’ criteria. maximum heart rate achieved maximum heart rate achieved during exercise. exercise induced angina 1 for yes, 0 for no. oldpeak st depression induced by exercise relative to rest. slope of the peak exercise st segment 1 for upsloping, 2 for flat, 3 for downsloping. major vessels colored vessels (0-3) colored by flourosopy. thal a thallium stress test (in short, thal) is a nuclear imaging test that depicts how well blood flows into the heart while a person is exercising or at rest (nelson (2015)). 3 accounts for normal blood flow, 6 for fixed defect, 7 for reversible defect. a fixed defect refers to an area of the heart that does not get enough blood flow at rest or under stress. a reversible defect refers to an area of the heart that has good blood flow only at rest. distinct class values two distinct class values. absence (a) or presence (p) of heart disease. table 5: description of the statlog (heart) data set 4.5 experimentation on the statlog (heart) data set from experimental results, we see that a j48 tree generates only 19 rules from the statlog (heart) data set. compared to that, a 20-tree rs generates as many as 555 rules, a cut point with accuracy >= 95% extracts 288 rules, forex++ extracts as low as 59 rules. consequently, 59 top-ranked rules are selected for the rf+hc method from rs. similarly, a 20-tree forest pa wobs generates as many as 473 rules from the statlog (heart) data set, a cut point with accuracy >= 95% extracts 222 rules and forex++ extracts as low as 40 rules and thus 40 topranked rules are selected for the rf+hc method. we now present a brief comparison among j48, rf+hc and forex++ rules in table 6. australasian journal of information systems adnan & islam 2017, vol 21, selected papers from ausdm forex++: knowledge discovery from decision forests 16 criteria j48 rules rs forest pa wobs rf+hc rules forex++rules rf+hc rules forex++rules total rules 19 59 59 40 40 average accuracy 84.39% 88.05% 94.71% 88.78% 95.07% average coverage 5.26% 5.85% 13.11% 6.96% 16.37% average rule length 3.51 3.8 3.2 3.7 3.2 distinct root nodes 1 6 7 5 6 distinct class values 2 2 2 2 2 table 6: comparison among j48, rf+hc and forex++ rules from the statlog (heart) data set from table 6, we see that both rf+hc and forex++ rules outperform j48 rules in almost every criterion. between rf+hc and forex++ rules, we see that forex++ rules prevail quantitatively with better average acc, average cov, average len and distinct root nodes even when distinct class values remain the same. as was done before, we now discuss some of the trends that exist in forex++ rules from forest pa wobs as they are found to be the most accurate (see table 6). the trends are discussed as follows. trend 6: all 9 rules with thal = 3 have the consequent “absence” (average acc and average cov of these rules are 95% and 24% respectively). this indicates that when there is normal blood flow in the heart, there is lower chance of having heart disease. all 5 rules with thal = 7 have the consequent “presence” (average acc: 99%, average cov: 17%). this indicates that when the heart is not supplied with enough blood flow under stress, heart disease may occur. in this case, with thal = 7 the heart has good blood flow at rest and thus the defect is reversible (meaning the heart muscles are still salvageable and a surgical procedure may prevent a future heart attack). trend 7: all 11 rules with chest pain type = 2 or 3 have the consequent “absence” (average acc: 92%, average cov: 16%). this means that patients with atypical or non-anginal pain have lower chance of having heart disease. all 9 rules with chest pain type = 4 have the consequent “presence” (average acc: 96%, average cov: 15%). this means that patients with asymptomatic pain (who have had previous heart attack or diabetes) are especially at risk of developing a silent ischemia. trend 8: all 7 rules with major vessels colored = 0 have the consequent “absence” (average acc: 93%, average cov: 26%), which indicates that patients with no coronary arteries being blocked may not suffer from heart disease. all 12 rules with major vessels colored = 1 or 2 have the consequent “presence” (average acc: 98%, average cov: 8%), which indicates that if there are blockage or narrowing of some of the coronary arteries, there is a high chance of having heart disease. trend 9: all 4 rules with age <= 59 have the consequent “absence” (average acc: 95%, average cov: 24%). all 4 rules with age between 59 and 63 have the consequent “presence” (average acc: 98%, average cov: 9%). this indicates that higher the age, higher the chance of heart disease as ageing causes cardiac output to slow down and makes atherosclerosis (hardening and narrowing of the arteries) to deter enough blood circulation in the heart. trend 10: all 4 rules with exercise induced angina = 0 have the consequent “absence” (average acc: 94%, average cov: 21%). this indicates that if the heart gets enough blood flow under stress, there is lower chance of having heart disease (also see trend 6). three out of 4 rules with exercise induced angina = 1 have the consequent “presence” (average acc: 95%, average cov: 12%). this indicates that if the heart does not get enough blood flow under stress then there is high chance of having heart disease (also see trend 6). the exceptional rule is: australasian journal of information systems adnan & islam 2017, vol 21, selected papers from ausdm forex++: knowledge discovery from decision forests 17  if major vessels colored = 0 and exercise induced angina = 1 and oldpeak <= 1.5 and resting blood pressure (systolic) <= 156 then a (acc: 88%, cov: 6%). in the exceptional rule, we see that there is no blockage in the coronary arteries (major vessels colored = 0) and thus the pain may not originate from the heart. trend 11: 2 out of 3 rules with serum cholesterol <= 304 have the consequent “presence”. the exceptional rule is:  if exercise induced angina = 0 and major vessels colored = 0 and gender = f and serum cholesterol <= 304 then a (acc: 100%, cov: 1%). the exceptional rule indicates that females may withstand higher serum cholesterol level due to their hormonal protection. based on the indication, we apply sql on the statlog (heart) data set and find that the average serum cholesterol level for females with heart disease is 290 and without heart disease is 257.2. on the other hand, the average serum cholesterol level for males with heart disease is 249.8 and without heart disease is 233.7. 5 conclusion in this paper we propose a new, data set independent framework (forex++) for extracting a subset of decision forest rules that are comparatively more accurate, generalized and concise than others. we apply forex++ on rules generated by random subspace and forest pa without bootstrap samples from two different publicly available medical related data sets on dementia and heart disease. we also apply a recent rule extraction method on rules generated from the same setup. compared to j48 rules, rules extracted from forex++ and the existing method are far more accurate and also can accommodate different angles for knowledge exploration. in head-to-head, forex++ outperforms the recent method in almost all criteria. also, the knowledge discovery potential of forex++ is found to be impressive. we have found some limitations of forex++ that are described in the following. 1. rules extracted by forex++ may not be very diverse; meaning similar rules may be selected from each class. 2. rules extracted by forex++ may not cover the entire data set. in future, we intend to solve these problems and also apply forex++ on different types of data sets. acknowledgement: the oasis data set (oasis) is a contribution from the following grants: p50 ag05681, p01 ag03991, r01 ag021910, p50 mh071616, u24 rr021382 and r01 mh56584. references adnan, m. n., & islam, m. z. (2014). “combosplit: combining various splitting criteria for building a single decision tree”. the international conference on artificial intelligence and pattern recognition (aipr2014), 17-19 november, 2014 kuala lumpur, malaysia. 1-8. adnan, m. n., & islam, m. z. (2015a). “complement random forest”. the 13th australasian data mining conference. sydney, australia. adnan, m. n., & islam, m. z. (2015b). “improving the random forest algorithm by randomly varying the size of the bootstrap samples for low dimensional data sets”. 23rd european symposium on artificial neural networks, computational intelligence and machine learning, (esann 2015), 22-24 april, 2015 bruges, belgium. 391-396. adnan, m. n., & islam, m. z. (2015c). “one-vs-all binarization technique in the context of random forest”. 23rd international symposium on artificial neural networks, computational intelligence and machine learning, (esann 2015), 22-24 april, 2015 bruges, belgium. 391-396. australasian journal of information systems adnan & islam 2017, vol 21, selected papers from ausdm forex++: knowledge discovery from decision forests 18 adnan, m. n., & islam, m. z. (2016a). “forest cern: a new decision forest building technique”. 20th pacific-asia conference, pakdd 2016, auckland, new zealand, april 19-22, 2016, proceedings, part i. cham: springer international publishing. adnan, m. n., & islam, m. z. (2016b). “knowledge discovery from a data set on dementia through decision forest”. the 14th australasian data mining conference. canberra, australia. adnan, m. n., & islam, m. z. (2016c). “optimizing the number of trees in a decision forest to discover a subforest with high ensemble accuracy using a genetic algorithm”. knowledgebased systems, 110, 86-97. adnan, m. n., & islam, m. z. (2017) “forest pa: constructing a decision forest by penalizing attributes used in previous trees”. expert systems with applications, accepted. bernard, s., adam, s., & heutte, l. (2012). “dynamic random forests”. pattern recognition letters, 33, 1580-1586. bernard, s., heutte, l., & adam, s (2008). “forest-rk: a new random forest induction method”. icic (2). springer. breiman, l. (1996). “bagging predictors”. machine learning, 24, 123-140. breiman, l. (2001). “random forests”. machine learning, 45, 5-32. buckner, r. l., head, d., parker, j., fotenos, a. f., marcus, d., morris, j. c., & snyder, a. z. (2004). “a unified approach for morphometric and functional data analysis in young, old, and demented adults using automated atlas-based head size normalization: reliability and validation against manual measurement of total intracranial volume”. neuroimage, 23, 724-38. buckner, r. l., snyder, a. z., shannon, b. j., larossa, g., sachs, r., fotenos, a. f., ... & mintun, m. a. (2005). “molecular, structural, and functional characterization of alzheimer's disease: evidence for a relationship between default activity, amyloid, and memory”. j neurosci, 25, 7709-17. dementia [online]. available: http://www.dementia.com/causes.html [accessed 15 march, 2016 2016]. el-bialy, r., salamay, m. a., karam, o. h., & khalifa, m. e. (2015). “feature analysis of coronary artery heart disease data sets”. procedia computer science, 65, 459-468. ertek, g., tokdil, b., & günaydın, i̇. (2014). risk factors and identifiers for alzheimer’s disease: a data mining analysis. in: perner, p. (ed.) advances in data mining. applications and theoretical aspects: 14th industrial conference, icdm 2014, st. petersburg, russia, july 16-20, 2014. proceedings. cham: springer international publishing. folstein, m. f., folstein, s. e., & mchugh, p. r. (1975). ““mini-mental state”: a practical method for grading the cognitive state of patients for the clinician”. journal of psychiatric research, 12, 189-98. geng, l., & hamilton, h. j. (2006). “interestingness measures for data mining: a survey”. acm computing surveys, 38, 9. geurts, p., ernst, d., & wehenkel, l. (2006). “extremely randomized trees”. machine learning, 63, 3-42. hall, m., frank, e., holmes, g., pfahringer, b., reutemann, p., & witten, i. h. (2009). “the weka data mining software: an update”. acm sigkdd explorations newsletter, 11(1):10-18 doi:10.1145/1656274.1656278. han, j., pei, j., & kamber, m. (2011). data mining: concepts and techniques, morgan kaufmann publishers inc. australasian journal of information systems adnan & islam 2017, vol 21, selected papers from ausdm forex++: knowledge discovery from decision forests 19 ho, t. k. (1998). “the random subspace method for constructing decision forests”. ieee transactions on pattern analysis and machine intelligence, 20, 832-844. hollingshead, a. b. (1975). four-factor index of social status. new haven, ct: yale university. huysmans, j., baesens, b., & vanthienen, j. (2006). “using rule extraction to improve the comprehensibility of predictive models”. fetew research report, 1-55 doi:10.2139/ssrn.961358. johansson, u., sönströd, c., & löfström, t. (2011). “one tree to explain them all”. ieee congress of evolutionary computation (cec), 5-8 june 2011 2011. 1444-1451. lichman, m. uci machine learning repository [online]. available: https://archive.ics.uci.edu/ml/index.html [accessed 15 march, 2016 2016]. liu, s., patel, r. y., daga, p. r., liu, h., fu, g., doerksen, r. j., & wilkins, d. e. (2012). “combined rule extraction and feature elimination in supervised classification”. ieee transactions on nanobioscience, 11(3):228-236 doi:10.1109/tnb.2012.2213264. lu, z., wu, x., zhu, x., & bongard, j. (2010). “ensemble pruning via individual contribution ordering”. proceedings of the 16th acm sigkdd international conference on knowledge discovery and data mining. washington, dc, usa. margineantu, d. d., & dietterich, t. g. (1997). “pruning adaptive boosting”. proceedings of the fourteenth international conference on machine learning. morgan kaufmann publishers inc. martens, d., huysmans, j., setiono, r., vanthienen, j., & baesens, b. (2008). “rule extraction from support vector machines: an overview of issues and application in credit scoring”. rule extraction from support vector machines. springer berlin heidelberg. martínez-muñoz, g., hernández-lobato, d., & suárez, a. (2009). “an analysis of ensemble pruning techniques based on ordered aggregation”. ieee transactions on pattern analysis and machine intelligence, 31, 245-259. martınez-munoz, g., & suárez, a. (2004). “aggregation ordering in bagging”. international conference on artificial intelligence and applications (iasted 2004). martínez-muñoz, g., & suárez, a. (2010). “out-of-bag estimation of the optimal sample size in bagging”. pattern recognition, 43, 143-152. mashayekhi, m., & gras, r. (2015). “rule extraction from random forest: the rf+hc methods”. 28th canadian conference on artificial intelligence, canadian ai 2015, halifax, nova scotia, canada, june 2-5, 2015, proceedings. cham: springer international publishing. morris, j. c. (1993). “the clinical dementia rating (cdr): current version and scoring rules”. neurology, 43, 2412-4. murthy, s. k. (1998). “automatic construction of decision trees from data: a multi-disciplinary survey”. data mining and knowledge discovery. 2, 345-389. nelson, j. (2015). thallium stress test [online]. available: http://www.healthline.com/health/thallium-stress-test [accessed 15 november, 2016 2016]. nichols, m., townsend, n., scarborough, p., & rayner, m. (2014). “cardiovascular disease in europe 2014: epidemiological update”. european heart journal, 35, 2950-2959. polikar, r. (2006). “ensemble based systems in decision making”. ieee circuits and systems magazine, 6, 21-45. quinlan, j. r. (1993). c4.5: programs for machine learning, morgan kaufmann publishers inc. australasian journal of information systems adnan & islam 2017, vol 21, selected papers from ausdm forex++: knowledge discovery from decision forests 20 quinlan, j. r. (1996a). “bagging, boosting, and c4.5”. proceedings of the thirteenth national conference on artificial intelligence volume 1. portland, oregon: aaai press. quinlan, j. r. (1996b). “improved use of continuous attributes in c4.5”. journal of artificial intelligence research. 4, 77-90. ruta, d., & gabrys, b. (2005). “classifier selection for majority voting”. information fusion, 6, 63-81. safavian, s. r., & landgrebe, d. (1991). “a survey of decision tree classifier methodology”. ieee transactions on systems, man, and cybernetics, 21, 660-674. schmitz, g. p., aldrich, c., & gouws, f. s. (1999). “ann-dt: an algorithm for extraction of decision trees from artificial neural networks”. ieee transactions on neural networks, 10, 1392-1401. shouman, m., turner, t., & stocker, r. (2011). “using decision tree for diagnosing heart disease patients”. proceedings of the ninth australasian data mining conference volume 121. ballarat, australia: australian computer society, inc. tan, p. n., steinbach, m., & kumar, v. (2005). introduction to data mining, (first edition), addison-wesley longman publishing co., inc. what is dementia? [online]. available: http://www.alz.org/what-is-dementia.asp [accessed 15 march, 2016 2016]. what is oasis? [online]. available: http://www.oasis-brains.org/ [accessed 15 march, 2016 2016]. copyright: © 2017 adnan & zahidul. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems schwartz, bellou, garcia-castrillo, muraro & papadopoulos 2017, vol 21, research on health information systems mhealth in emergency response communities 1 exploring mhealth participation for emergency response communities david g. schwartz graduate school of business administration bar-ilan university ramat-gan, israel david.schwartz@biu.ac.il abdelouahab bellou harvard medical school beth israel deaconness medical center boston, ma, usa luis garcia-castrillo emergency dept. servicio cantabro de salud hospital universitario marques de valdecilla cantabria, spain antonella muraro food allergy referral centre veneto region padua general university hospital padua, italy nikolaos papadopoulos center for pediatrics and child health institute of human development the university of manchester manchester, united kingdom and department of allergy university of athens athens, greece abstract we explore the challenges of participation by members of emergency response communities who share a similar condition and treatment, and are called upon to participate in emergency events experienced by fellow members. smartphones and location-based social networking technologies present an opportunity to re-engineer certain aspects of emergency medical response. life-saving prescription medication extended in an emergency by one individual to another occurs on a micro level, anecdotally documented. we illustrate the issues and our approach through the example of an app to support patients prone to anaphylaxis and prescribed to carry epinephrine auto-injectors. we address unique participation challenges in an mhealth environment in which interventions are primarily short-term interactions which require clear and precise decision-making and constant tracking of potential participants in responding to an emergency medical event. the conflicting effects of diffused responsibility and shared identity are identified as key factors in modelling participation. keywords: mhealth; participation; smartphones; chronic disease; emergency medical response; social networking; anaphylaxis; ems; shared identity; diffused responsibility. 1 introduction mobile healthcare applications (mhealth apps) have ushered in a new age of participatory medicine. developments in mhealth have led to a sharp increase in the availability of apps supporting medically motivated real-world interaction between app users, leading to australasian journal of information systems schwartz, bellou, garcia-castrillo, muraro & papadopoulos 2017, vol 21, research on health information systems mhealth in emergency response communities 2 potentially significant changes in healthcare delivery (anderson & emmerton, 2015; malvey & slovensky, 2014; swan, 2009). of particular interest is the category of mhealth apps which facilitate spontaneous communication between stakeholders in response to an emergency event. smartphone applications can support the connection of app users to real-world medical emergency events and are being studied in diverse situations such as the mapping and response-use of aeds (automatic electronic defibrillators) (folke et al., 2009; merchant et al., 2013; peberdy et al., 2006; rea, blackwood, damon, phelps, & eisenberg, 2011; sakai et al., 2011), volunteer emergency response (cappato et al., 2006; elsner, meisen, ewert, schilberg, & jeschke, 2014; elsner, meisen, thelen, schilberg, & jeschke, 2013; elsner, schneiders, haberstroh, schilberg, & jeschke, 2013; groh et al., 2007; marshall et al., 2006), the physical proximity of diabetes patients (d. johnson, 2014) in need of glucose or monitors, and emergency response communities (erc) for anaphylaxis events (schwartz, bellou, garcia-castrillo, muraro & papadopoulos, 2014). such systems have the potential to enable a new form of emergency medical response in which the social network-based connection between individuals creates a lay-layer of support which can augment or potentially modify current forms of ems (emergency medical services). like most interactions between patients and ems, many of these apps rely on fleeting one-off interactions between patient and a responding participant, which is quite different from the ongoing long-term relationships fostered between patients and their caregivers in most forms of participatory medicine. in this paper we investigate the nature of this new form of participation against the backdrop of an emergency response community designed to support allergy-sufferers in the event of an anaphylactic reaction. participatory medicine envisions enhanced communication between all stakeholders to collaborate and contribute towards patients’ wellbeing (ahern, woods, lightowler, finley, & houston, 2011; dyson, 2009). these stakeholders are generally viewed in terms of the standard medical ecosystem of the individual patient – i.e. the patient’s family, caregivers, physicians, third-party hmos and institutions where the patient is treated and extend beyond the individual to include advocacy groups, clinical scientists, and regulatory authorities (de silva, burstein, stranieri, williams, & rinehart, 2013). much of the work in this important area views community participation as the measure of a long-term ongoing relationship in which all stakeholders maintain various levels of involvement and has focused on ways to enhance our understanding of interactions between people (mckemmish et al., 2012). the primary forms of participation are considered decision making, information gathering, information sharing between stakeholders, and condition monitoring. this type of deep and long-term participatory relationship in support of patient centred care and p4 medicine (predictive, preventive, personalized and participatory) is being applied in areas ranging from the management of autism spectrum disorder (de silva et al., 2013) to family-centric genomic sequencing and the digitization of medicine (hood & flores, 2012). among the key aspects of participatory medicine for people is knowing when and how to call on friends, family and other social networks for help and advice and have access to them (smith, graedon, & graedon, 2016). it is these latter aspects – the immediate short-term need for intervention that we focus on in studying erc and the participatory behaviour of community members in an emergency event. participatory health information systems (phis) serve a primary role in enabling these interactions. emergency response apps and their underlying phis necessitate new considerations to the study of participation. this is due to (a) the fundamental reciprocity built-in to such apps in which all community members can equally become one who alerts or one who responds and (b) the lack of any form of social interaction outside of the emergency event. contrary to what is found in most social networking communication, an alert is not communication directed at counterparts chosen by the member, but rather is sent to the full subset of members available to respond at that time. thus the potential participants in any given medical event are never known in advance and participation is not dependent on the relationship factors that dominate the study of participatory medicine. members of such communities may in fact maintain relative anonymity and inactive yet latent links with all australasian journal of information systems schwartz, bellou, garcia-castrillo, muraro & papadopoulos 2017, vol 21, research on health information systems mhealth in emergency response communities 3 other members. this results in participation models, user characteristics, and participation behaviours unlike those traditionally studied in participatory medicine. to illustrate and analyse the nature of these ties and serve as a backdrop to our discussion we present an app in support of an erc for allergy patients at risk of anaphylaxis events. the information system, which integrates patient, physician and ems participation, advanced matching of prescription medication, and location-based social networking, is being initially focused on anaphylaxis with broad potential for other chronic conditions. the erc system and anaphylaxis app design emerged by following a soft systems methodology (checkland, 2000) of modelling workflow and information flow (durugbo, tiwari, & alcock, 2013; hibberd & evatt, 2004; unertl, weinger, johnson, & lorenzi, 2009). to facilitate this process we engaged with a pool of 18 medical professionals across the fields of emergency medicine (8), allergy and clinical immunology (4), pharmacology (4) and public health (4). 1.1 anaphylaxis as an illustrative case anaphylaxis is a severe, potentially fatal, systemic allergic or hypersensitivity reaction of rapid onset. it may trigger life-threatening cardiopulmonary compromise, often with skin and mucosal changes such as urticaria and angioedema. with the onset of anaphylaxis an immediate intramuscular (im) adrenaline injection is the first-line of treatment and can be lifesaving and it has been noted that there are no absolute contraindications to the prescription and utilization of self-injectable adrenaline (song, worm, & lieberman, 2014).many chronic conditions involve patients maintaining a long-term prescription and current supply of medication. certain conditions, including asthma, acute allergies, and diabetes (glycaemic control), have significant global populations being treated using a narrow set of medications. in many cases the aforementioned conditions generally require that the patient carry one or more doses of medication on their person at all times. the combination of stable prescriptions, narrow medical regimens, and high availability presents a common basis for community formation (simons & world allergy organization, 2009; “standards of medical care in diabetes--2012,” 2011, us department of health and human services, 2007). this type of community formulation goes beyond the membership characteristics commonly considered for online health communities (ohc) (van der eijk et al., 2013) even when such communities are ‘closed’ and monitored by a community manager. adherence by chronic patients to medical regimens is recognized to be dismally low (sabaté, 2003). with anaphylaxis in particular, it has been noted that many patients who have been prescribed epinephrine auto-injectors, because of a previous anaphylactic attack, do not carry them with studies showing that <30% of these patients carried it at all times (johnson & parker, 2006; kastner, harada, & waserman, 2010; song et al., 2014). there are few studies on anaphylaxis management in pre hospital settings. in the usa, the number of 911 calls for anaphylaxis is 0.4–0.9% of all calls (maio et al., 1999). chung et al (2014) showed that in an urban canadian centre from 481 identified allergy-related case records managed in the pre hospital setting, 136 (28%) met guideline criteria for anaphylaxis and 56% were considered high acuity level by the dispatch centre. only 25% of patients self-administered epinephrine and 36% of patients received epinephrine by paramedics. overall, 44% of patients received epinephrine prior the arrival of ems team. it appears that paramedics are not comfortable to use epinephrine likely because of pre existing misconceptions regarding epinephrine. epinephrine is the essential medication for the treatment of anaphylaxis and epinephrine autoinjectors are universally recommended as first-aid treatment for anaphylaxis occurring in community settings. despite this, significant delay in epinephrine administration (i.e.,≥ 5 min) was also found to be an important gap in anaphylaxis management (bilò et al., 2016; kastner et al., 2010). the onset of anaphylaxis can be sudden however it is generally recognized (song et al., 2014) that there is sufficient time and awareness on the part of the patient to initiate use of an epinephrine auto-injector should one be available. during this same time period a patient would be capable of initiating emergency use of an erc app to summon help. australasian journal of information systems schwartz, bellou, garcia-castrillo, muraro & papadopoulos 2017, vol 21, research on health information systems mhealth in emergency response communities 4 2 social networking, mobile telephones, and health care delivery it is recognized that social networking has the potential to change patterns of health inequalities and access to health care, alter the stability of health care provision and lead to a reformulation of the role of health professionals (griffiths et al., 2012). the study of mobile telephony and social networking’s impacts on health care delivery is widespread but has been limited to the behavioural effects of information sharing (shrank, choudhry, swanton, & et al, 2011), patient monitoring (weitzman, kelemen, quinn, eggleston, & mandl, 2013), intervention and behaviour modification (gold et al., 2012), and peer influencing, with a more recent focus on the day-to-day management of chronic conditions (parmanto et al., 2013). a comprehensive meta-analysis of mobile-phone health interventions (klasnja & pratt, 2012) analyses 73 studies to elicit five health intervention strategies: tracking health information; involving the healthcare team; leveraging social influence; increasing accessibility of health information; utilizing entertainment. notably, neither the five derived categories nor any of the underlying studies contemplate a mobile-based intervention strategy that involves one patient actually bringing a critical dose to another patient in distress. a comprehensive review of mobile health applications is presented by martinez-perez et al. (martínez-pérez, de la torre-díez, & lópez-coronado, 2013). the 3,600 apps surveyed overwhelmingly deal with condition monitoring and information provision and none present any form of community member intervention coordinated with ems as discussed in the current work. most research that deals with emergency response through mhealth apps does so in the context of large scale disasters from the perspective of emergency case management (gill, alam, & eustace, 2015) rather than individual participation. to move beyond information sharing, patient monitoring, behaviour modification, and peer influencing and harness the social network for actual emergency intervention we apply a flow model (durugbo et al., 2013) to study community roles and participation, and identify supervisory control and health policy issues which influence the nature of participation. in the remainder of this paper we map out the potential, challenges, and risks, explain the nature of emergency response community information flow and medication flow, and present the example of such an emergency response community for anaphylaxis. we begin with a description of information flow and medication flow in the erc ecosystem. we then provide an illustrative example showing how an erc can address the community of chronic allergy sufferers at risk of anaphylaxis and focus upon the nature and challenges of participation. we summarize a number of significant issues that must be addressed and conclude with an agenda for future research. australasian journal of information systems schwartz, bellou, garcia-castrillo, muraro & papadopoulos 2017, vol 21, research on health information systems mhealth in emergency response communities 5 fig. 1. ems view of anaphylaxis alert scenario 3 design and flow of emergency response participation modelling workflow and information flow is an established methodology for studying medical scenarios, such as chronic patient care (unertl et al., 2009). this approach is based on the soft systems methodology (checkland, 2000; mingers, 2000) which is well documented as a methodology for process innovation (presley, sarkis, & liles, 2000). the soft systems methodology is seen as a practical way to operationalize vickers concept of social process as appreciative system (stowell, 2013). the process, as applied in the design of an erc for anaphylaxis, incorporated iteratively presenting the system design for feedback to 18 medical professionals across the fields of emergency medicine (8), allergy and clinical immunology (4), pharmacology (4) and public health (4). observe the scenario emergency shown in figure 1. here we present a map superimposed with internal erc member data for a defined geographical area and treatment range (referred to as the ems view as this information is hidden from individual members). in this scenario there are 6 registered patients, e1 through e6, carrying some form of epinephrine injection or alternative treatment in the observed area, with a 7th and 8th observed members (a1 and a2) carrying asthma inhalers. this scenario was developed through the insights gained from our information flow modelling process. 3.1 information flow and medication flow information flow models (durugbo et al., 2013) model the transfer of information between entities in a bounded organization or decision environment. we use information flow modelling (figure 2) to depict the interactions between the main entities in the erc ecosystem and identify design and health policy issues. these entities include: (a) patient or community member; (b) authorizing medical professional; (c) ems centre; and (d) the erc server. generically we refer to all patients with registered medications as ‘members’ of their emergency response community. entities can adopt multiple roles without changing identity so that, the ‘patient’ entity can be, at alternate times, a ‘registered member’ who has simply registered with the system; a ‘requesting member’ who has initiated an alert; ‘a responding australasian journal of information systems schwartz, bellou, garcia-castrillo, muraro & papadopoulos 2017, vol 21, research on health information systems mhealth in emergency response communities 6 member’ who is answering an alert call. the insights gained from the flow model served as the basis for the overall system architecture and application design. we illustrate through a specific emergency response scenario resulting from the implementation of an erc for anaphylaxis. the erc system design is based on the analysis of information, supervisory, and medication flows. fig. 2. participation in the erc ecosystem: information flow, supervisory flow, and medication flow. following the scenario introduced above, e3 experiences an allergic reaction with anaphylactic symptoms and realizes that his injector is missing. he initiates an erc alert on his smartphone (figure 3, panel a). this alert is immediately transmitted, via the erc server, to each of the other members with registered compatible medication in the valid alert range. concurrently, an ems alert is initiated enabling ems to view the medications registered to e3 and all those responding to his call. alert notifications (figure 3, panel b) are then sent to e1, e2, e4, and e6 (with e5 determined to be out of effective range). upon receipt of the alert, e1, e2, and e6 all indicate that they are responding (e4 does not reply) and receive guidance (figure 3, panel c). e2 reaches the distressed e3 first, notes that approval has been sent by ems (as indicated by the green checkmark in the app) and administers her epinephrine to e3. australasian journal of information systems schwartz, bellou, garcia-castrillo, muraro & papadopoulos 2017, vol 21, research on health information systems mhealth in emergency response communities 7 fig. 3. erc mobile application screens 3.2 information flow and medication flow a flow model (figure 2) was used to map out the different forms of participation and roles of participants that emerged from our discussions with the expert pool of medical professionals. participants include: member – who at different life-cycle points can be: (a) enrolling (awaiting approval); (b) in distress/alerting; (c) (un)available; and (d) responding; physician – whose initial participation is required to approve patient membership in the appropriate community and is updated whenever a supervised member is involved in an erc event; ems – which serves a supervisory role in approving responding member participation upon event occurrence; the flows shown in figure 2 are labelled f0 through f16. table 2 presents details and significance of the main information flows. australasian journal of information systems schwartz, bellou, garcia-castrillo, muraro & papadopoulos 2017, vol 21, research on health information systems mhealth in emergency response communities 8 f# from to description f0 enrolling member erc cloud community enrollment and medication registration; prescription data and medication details. f1 erc cloud physician request confirmation of patient details and prescription. f2 primary physician erc cloud confirmation of patient details and prescription at time of registration. f3 member in distress erc cloud emergency situation; urgent need of critical medication; inform local ems. f4 erc cloud ems central condition and script data for member in distress; script data and proximity for compatible members in range. f5 ems central erc cloud expert ems medical confirmation of recommended med sources in range. f6– f9 erc cloud members member in distress is within range and compatible medication detected; requesting action. f10– f12 available members erc cloud medication is available and willing to act; requesting guidance and confirmation. f13 erc cloud responding members tracking and guidance to arrive at member in distress. f14 ems central responding member verification and confirmation of medication suitability by ems physician. f15 responding member member in distress physical delivery of medication. f16 erc cloud primary physician post-incident report of alert and response. table 1. key flow summary 3.3 from gps social navigation technologies to ercs the ability to track large numbers of participants across broad geographic regions, including assessments of speed and probable mode of transportation, has been established through recent developments in social navigation systems such as google’s waze (www.waze.com /about) and apple’s route rating (fino, 2011). these systems treat the driver’s smartphone as a proxy for car location and movement thereby deriving traffic characteristics and even the existence of new roads not previously mapped. with waze activated, the location and movement of millions of participants is steadily transmitted to a server which continually reassesses traffic conditions and shares new routing recommendations with all participants in the relevant geographic area (disatnik, shmuelevitz, & levine, 2011). similarly, erc applications are able to treat the smartphone as a proxy for the location of specific medicines borne by members as per their registration. as this information is continually transmitted to the erc server by the erc application installed on each member’s smartphone, we are able to determine in real-time which community members are within reach of a member in distress when an alert is initiated. 4 discussion the result of the anaphylaxis erc design process highlights both immense potential and system risks. the patient safety and drug regulatory issues that are not dealt with in ‘information sharing’ communities lead us to a series of supervisory and safety considerations. by mapping the information, medication, and supervisory flows a robust ecosystem emerges which can serve as the basis for changing emergency response activities. in this section we address a number of regulatory, safety and privacy issues before addressing the core questions related to participation. 4.1 regulatory, safety, legal, and privacy challenges in all chronic emergency response scenarios the erc approach includes and comes in addition to immediate ems contact and dispatch. we advocate training in auto-injector use for all australasian journal of information systems schwartz, bellou, garcia-castrillo, muraro & papadopoulos 2017, vol 21, research on health information systems mhealth in emergency response communities 9 community members and note that recognition in the importance of such training is growing (kastner et al., 2010; manivannan et al., 2014). regulatory issues related to prescription drug sharing; human risks related to unsuitable or improperly administered treatments; liability issues related to adverse effects; privacy concerns related to the voluntary disclosure of medical needs; all must be addressed. sharing prescription drugs is dangerous, illegal in most jurisdictions, and the centre of much research and many prevention efforts (goldsworthy, schwartz, & mayhorn, 2008). to be clear, while there is the potential to misuse and abuse, a manifold potential for drug abuse exists in a variety of social networking environments (hanson, cannon, burton, & giraud-carrier, 2013). rather than promoting misuse, the erc ecosystem enables identification, tracking, study, professional guidance and follow-up. while this article is not a source of legal advice or guidance regarding the sharing of prescription medications, in order to advance the research in this area herein are some initial findings regarding good samaritan laws and medication provision in emergency situations. the analysis of each jurisdiction needs to be done separately, and must address the distinct questions of (a) legality of providing the medication which may be in violation of controlled drug legislation; and (b) immunity from prosecution in the event of a negative results which is commonly address by good samaritan laws. we bring two illustrative examples. according to british regulation 238 of the human medicines regulations 2012, certain prescription-only medicines can be administered by anyone for the purpose of saving life in an emergency. this includes adrenaline for intramuscular use in anaphylaxis (mhra, 2014). in the u.k. (england and wales) actions of volunteer samaritans are protected under the social action, responsibility and heroism (sarah) act (u.k., 2015, sec. 3). in the united states legislation varies according to state. for example, in may 2015 colorado passed bill 15-1232 allowing any person to “…provide an epinephrine auto-injector to an individual believed in good faith to be experiencing anaphylaxis or to provide an epinephrine auto-injector to a family member, friend, colleague, caregiver, or person with a similar relationship with the individual” and further extends immunity from prosecution for in the event of adverse effects to “an individual who provides or administers an epinephrine autoinjector” provided it was done in good faith and devoid of negligence. (colorado state legislature, 2015). each chronic condition and treatment response requires in-depth study to assess the risks and benefits. indirect effects such as increasing the level of patients filling and being in possession of their own required prescriptions (johnson & parker, 2006) must also be studied. certain treatment areas may be determined to be off-limits due to unacceptable risks. for example, sharing insulin pens and cartridges is against the explicit policies of the cdc (2012), and fda (2013), described by the former as a “never event” which can result in transmission of hepatitis viruses, hiv, or other blood-borne pathogens – yet perhaps it would be considered an acceptable risk if the alternative is the onset of diabetic shock. recent advances related to the availability of epinephrine in schools, may provide guidance as to the broader potential of ercs (madigan & hasbrouck, 2012) in this treatment area. privacy issues must be considered. members of an erc will have voluntarily put their use of a given medical treatment into the public sphere. this increases the ability of a third party to become aware of an individual’s medical needs and make determinations regarding his or her medical condition. this issue is already being considered by sites such as patientslikeme (wicks et al., 2010). each erc member retains complete anonymity from other members up until such time as he or she responds to an alert or initiates one. the primary questions to be addressed here are: is the privacy being willingly sacrificed, and is the individual afforded a benefit worthy of that sacrifice (shortliffe & cimino, 2006). in an erc the answer to both questions appears to be a resounding ‘yes’ – nonetheless this area warrants further study. australasian journal of information systems schwartz, bellou, garcia-castrillo, muraro & papadopoulos 2017, vol 21, research on health information systems mhealth in emergency response communities 10 4.2 focusing on specific chronic conditions the decision to address anaphylaxis and asthma as the first two treatment areas was made based on their high prevalence and narrow treatment ranges. similar characteristics can be found in use of glucose tablets to treat hypoglycaemia for diabetes which will be the subject of future research and development efforts. each condition requires independent assessment of its suitability to our approach. 4.3 challenges to participation in the study of crowd behaviour, an accepted definition of a crowd is that of a large group of individuals in the same physical environment, sharing a common goal. crowd members are identified by the interests, attitudes, abilities and/or personal characteristics they have in common. over time these common attributes provide a guide for action that conforms to their common social identification, but may not conform to individuals' self-identity when they are alone or in a small group (musse & thalmann, 1997; reicher, 2001; roloff, 1981). in the study of internet-based crowdsourcing (schenk & guittard, 2009; surowiecki, 2004) a crowd can be defined as a large set of anonymous individuals that might not be individually identified, who work on a shared or given subtask via an internet platform. in the context of mhealth, a crowd may be formed by individuals joined together through the social network community supported by a medical app. when one community member requires assistance from another the task is accomplished by a single member drawn from the crowd who provides the necessary response not by the crowd as a whole. the effectiveness of such mhealth apps depends on the response of individual members and their willingness to participate in a task demanding physical intervention when the responsibility of completion falls upon a single person rather than the crowd. this points us to studies that discuss crowd behaviour during actual emergencies in which a single person is in distress, and the theory of diffused responsibility (dr). crowd responses to single person emergencies (spe) has been investigated in the canonical work by latane and darley (1970). bothered by recurrent incidents in which dozens of bystanders did nothing to help, they studied bystanders' motivations and actions during emergencies to reveal the effect of diffused responsibility. the doctrine of diffused responsibility holds that even if a person has noticed an event and defined it as an emergency, the mere knowledge of other witnessing bystanders can make him less likely to intervene (darley & latane, 1968; latane & darley, 1970). dr, however, does not explain strangers’ behaviour regarding a distressed individual that share something in common with them, such as the same chronic illness or medication. while a moral obligation to respond may exist, in absence of a legal obligation it is important to understand what might motivate community members to respond. for that we turn to the theory of shared identity. piliavin, rodin & piliavin (1969) investigated the effect of several variables on helping behaviour to reveal that bystanders are more likely to help if the victim was a member of their own racial group. the identity shared between the victim and the bystander leads to mutual aid that counteracted the effect of diffused responsibility due to perceptions of similarity and increased feelings of responsibility for welfare of others (batson, 1987; batson & ahmad, 2009; coke, daniel, & mcdavis, 1978; dovidio, piliavin, gaertner, schroeder, & clark iii, 1991; j. a. piliavin, 2009; snyder & dwyer, 2012; stotland, 1969). shared identity (si) goes beyond social classification to include factors ranging from sports team affiliations (levine, prosser, evans, & reicher, 2005) to the identity shared between chronically ill patients (brownstein, brownstein, williams, wicks, & heywood, 2009; frost & massagli, 2008; stürmer, snyder, & omoto, 2005). shostack & fox (2012) explored the presence of a collective identity among people with epilepsy discovering strong evidence of group identity expressed as an expectation that people with epilepsy will help each other. based on the above theoretical background we believe that the participation behaviour in mhealth emergency response situations will follow a hybrid behavioural model in which these conflicting impulses moderate. australasian journal of information systems schwartz, bellou, garcia-castrillo, muraro & papadopoulos 2017, vol 21, research on health information systems mhealth in emergency response communities 11 4.4 future research directions there are many risk, effectiveness, policy, and regulatory questions to be addressed. studying both the request and response behaviours of community members and monitoring and studying the clinical results of interactions facilitated by erc applications need to be at the forefront of our efforts. we present a number of the questions being dealt with in ongoing and future research. what percentage of participants will respond to community-member requests for emergency assistance when they believe that they can help save a life and what levels of response are required to be effective? how willing are participants to request and accept emergency assistance from a previously unknown responding community member when in distress? will knowing they have the potential to help someone in distress increase member adherence levels and thus increase the potential for self-administration in times of need? (johnson & parker, 2006). conversely, will a free-rider phenomenon lower medication carrying amongst members who believe they can rely on a community member to come to their rescue? how might effective response radii differ as impacted by medical condition, medication match, transportation mode, member density, estimated time of arrive (eta), and other relevant variables? how must legal frameworks (such as good samaritan laws (pardun, 1998)) and regulatory restrictions be modified to support participation in ercs? these are a number of the challenges being addressed as we move forward in the development and testing of emergency response communities. in particular, user adaption and the correct strategies to engage with different user communities, is of particular importance in considering participatory medicine. studies such as (beaudry & pinsonneault, 2005) provide suitable strategies on which to base this future research. 5 conclusion our approach to the use of social networks for emergency response presents a new conceptualization of ‘social medicine’ and raises new questions regarding the nature of participation. we have all heard, or asked, some form of the question: “i have a headache – do you have an aspirin?” we’ve asked colleagues, flight attendants, teachers, cab drivers – and have offered the same when asked. the archetypal question being asked and answered in our conceptualization of social medicine is “do you have an aspirin?” where “aspirin” is replaced with a critical life-saving treatment and “you” is replaced with millions of people, any one of whom may be carrying such a treatment nearby at any given time. as the prevalence of chronic conditions continues to rise (gerteis et al., 2014; goodman, posner, huang, parekh, & koh, 2013), by creating emergency response communities around chronic conditions such as acute allergies, asthma, diabetes, or heart disease, we have the ability to reshape society’s response to individuals in medical distress – by reshaping the response of individuals in our society. emergency medicine presents medical and economic challenges, and recognized trade-offs when an individual is in need of immediate intervention and traditional ems support might not be able to respond immediately. as such it presents a new sub-domain for the study of participatory medicine and challenges that we have explored through the example of anaphylaxis. acknowledgements the authors would like to thank batya schwartz for designing the erc user interface and graphics. references ahern, d. k., woods, s. s., lightowler, m. c., finley, s. w., & houston, t. k. (2011). promise of and potential for patient-facing technologies to enable meaningful use. american journal of preventive medicine, 40(5, supplement 2), s162–s172. http://doi.org/10.1016/j.amepre.2011.01.005 australasian journal of information systems schwartz, bellou, garcia-castrillo, muraro & papadopoulos 2017, vol 21, research on health information systems mhealth in emergency response communities 12 anderson, k., & emmerton, l. m. (2015). contribution of mobile health applications to selfmanagement by consumers: review of published evidence. australian health review. retrieved from http://www.publish.csiro.au/?paper=ah15162 batson, c. d. (1987). prosocial motivation: is it ever truly altruistic? retrieved from http://psycnet.apa.org/psycinfo/1987-98851-002 batson, c. d., & ahmad, n. y. (2009). using empathy to improve intergroup attitudes and relations. social issues and policy review, 3(1), 141–177. http://doi.org/10.1111/j.17512409.2009.01013.x beaudry, a., & pinsonneault, a. (2005). understanding user responses to information technology: a coping model of user adaptation. mis quarterly, 493–524. bilò, m. b., cichocka-jarosz, e., pumphrey, r., oude-elberink, j. n., lange, j., jakob, t., … rueff, f. (2016). self-medication of anaphylactic reactions due to hymenoptera stings— an eaaci task force consensus statement. allergy, 71(7), 931–943. http://doi.org/10.1111/all.12908 brownstein, c. a., brownstein, j. s., williams, d. s., wicks, p., & heywood, j. a. (2009). the power of social networking in medicine. nature biotechnology, 27(10), 888–890. http://doi.org/10.1038/nbt1009-888 cappato, r., curnis, a., marzollo, p., mascioli, g., bordonali, t., beretti, s., … cas, l. d. (2006). prospective assessment of integrating the existing emergency medical system with automated external defibrillators fully operated by volunteers and laypersons for out-of-hospital cardiac arrest: the brescia early defibrillation study (beds). european heart journal, 27(5), 553–561. http://doi.org/10.1093/eurheartj/ehi654 cdc. (2012, january). cdc clinical reminder: insulin pens injections safety. retrieved september 7, 2013, from http://www.cdc.gov/injectionsafety/clinicalreminders/insulin-pens.html checkland, p. (2000). soft systems methodology: a thirty year retrospective. systems research and behavioral science, 17(s1), s11. chung, t., gaudet, l., vandenberghe, c., couperthwaite, s., sookram, s., liss, k., … rowe, b. h. (2014). pre-hospital management of anaphylaxis in one canadian urban centre. resuscitation. retrieved from http://www.sciencedirect.com/science/article/pii/s0300957214004729 coke, j. s., daniel, c., & mcdavis, k. (1978). empathic mediation of helping: a two-stage model. journal of personality and social psychology, 36(7), 752–766. http://doi.org/10.1037/0022-3514.36.7.752 colorado state legislature. concerning the emergency use of epinephrine auto-injectors by authorized entities, and, in connection therewith, making an appropriation, pub. l. no. 15–1232 (2015). retrieved from http://www.leg.state.co.us/clics/clics2015a/csl.nsf/fsbillcont3/b99cd6d4b75ffe848 7257dcf0069ce28?open&file=1232_enr.pdf darley, j. m., & latane, b. (1968). bystander intervention in emergencies: diffusion of responsibility. journal of personality and social psychology, 8(4p1), 377. de silva, d., burstein, f., stranieri, a., williams, k., & rinehart, n. (2013). a participatory information management framework for patient centred care of autism spectrum disorder. in 24th australasian conference on information systems (acis) (pp. 1–11). rmit university. retrieved from https://researchbank.rmit.edu.au/view/rmit:161076 disatnik, i., shmuelevitz, y., & levine, u. (2011, may 5). device, system, and method of dynamic route guidance u.s. patent application no. 12/588,786. australasian journal of information systems schwartz, bellou, garcia-castrillo, muraro & papadopoulos 2017, vol 21, research on health information systems mhealth in emergency response communities 13 dovidio, j. f., piliavin, j. a., gaertner, s. l., schroeder, d. a., & clark iii, r. d. (1991). the arousal: cost-reward model and the process of intervention: a review of the evidence. in prosocial behavior (pp. 86–118). thousand oaks, ca, us: sage publications, inc. durugbo, c., tiwari, a., & alcock, j. r. (2013). modelling information flow for organisations: a review of approaches and future challenges. international journal of information management, 33(3), 597–610. http://doi.org/10.1016/j.ijinfomgt.2013.01.009 dyson, e. (2009). why participatory medicine? journal of participatory medicine. retrieved from http://ojs.jopm.org/index.php/jpm/article/viewarticle/v1n1e1 elsner, j., meisen, p., ewert, d., schilberg, d., & jeschke, s. (2014). prescient profiling–ai driven volunteer selection within a volunteer notification system. in e℡emed 2014, the sixth international conference on ehealth, telemedicine, and social medicine (pp. 94–98). retrieved from http://www.thinkmind.org/index.php?view=article&articleid=etelemed_2014_5_20_ 40036 elsner, j., meisen, p., thelen, s., schilberg, d., & jeschke, s. (2013). emurgency–a basic concept for an ai driven volunteer notification system for integrating laypersons into emergency medical services. international journal on advances in life sciences, 5(3 and 4), 223–236. elsner, j., schneiders, m.-t., haberstroh, m., schilberg, d., & jeschke, s. (2013). an introduction to a transnational volunteer notification system providing cardiopulmonary resuscitation for victims suffering a sudden cardiac arrest. in e℡ emed 2013, the fifth international conference on ehealth, telemedicine, and social medicine (pp. 59–64). retrieved from http://www.thinkmind.org/index.php?view=article&articleid=etelemed_2013_3_10_ 40119 fda. (2013, august). information for healthcare professionals: risk of transmission of blood-borne pathogens from shared use of insulin pens [webcontent]. retrieved september 7, 2013, from http://www.fda.gov/drugs/drugsafety/postmarketdrugsafetyinformationforpatients andproviders/drugsafetyinformationforheathcareprofessionals/ucm133352.htm fino, j. s. (2015) user-specified route rating and alerts. u.s. patent 8,977,498, issued march 10, 2015. folke, f., lippert, f. k., nielsen, s. l., gislason, g. h., hansen, m. l., schramm, t. k., … others. (2009). location of cardiac arrest in a city center strategic placement of automated external defibrillators in public locations. circulation, 120(6), 510–517. frost, j. h., & massagli, m. p. (2008). social uses of personal health information within patientslikeme, an online patient community: what can happen when patients have access to one another’s data. journal of medical internet research, 10(3), e15. http://doi.org/10.2196/jmir.1053 gerteis, j., izrael, d., deitz, d., leroy, l., ricciardi, r., miller, t., & basu, j. (2014). multiple chronic conditions chartbook. rockville, md: agency for healthcare research and quality (ahrq) publications. gill, a. q., alam, s. l., & eustace, j. (2015). social architecture: an emergency management case study. australasian journal of information systems, 19(0). http://doi.org/10.3127/ajis.v19i0.979 gold, j., pedrana, a. e., stoove, m. a., chang, s., howard, s., asselin, j., … hellard, m. e. (2012). developing health promotion interventions on social networking sites: recommendations from the facespace project. journal of medical internet research, 14(1), e30. http://doi.org/10.2196/jmir.1875 australasian journal of information systems schwartz, bellou, garcia-castrillo, muraro & papadopoulos 2017, vol 21, research on health information systems mhealth in emergency response communities 14 goldsworthy, r. c., schwartz, n. c., & mayhorn, c. b. (2008). beyond abuse and exposure: framing the impact of prescription-medication sharing. american journal of public health, 98(6), 1115–1121. http://doi.org/10.2105/ajph.2007.123257 goodman, r. a., posner, s. f., huang, e. s., parekh, a. k., & koh, h. k. (2013). defining and measuring chronic conditions: imperatives for research, policy, program, and practice. preventing chronic disease, 10. http://doi.org/10.5888/pcd10.120239 griffiths, f., cave, j., boardman, f., ren, j., pawlikowska, t., ball, r., … cohen, a. (2012). social networks – the future for health care delivery. social science & medicine, 75(12), 2233–2241. http://doi.org/10.1016/j.socscimed.2012.08.023 groh, w. j., birnbaum, a., barry, a., anton, a., mann, n. c., peberdy, m. a., … mosesso jr., v. n. (2007). characteristics of volunteers responding to emergencies in the public access defibrillation trial. resuscitation, 72(2), 193–199. http://doi.org/10.1016/j.resuscitation.2006.06.036 hanson, c. l., cannon, b., burton, s., & giraud-carrier, c. (2013). an exploration of social circles and prescription drug abuse through twitter. journal of medical internet research, 15(9), e189. http://doi.org/10.2196/jmir.2741 hibberd, b. j., & evatt, a. (2004). mapping information flows: a practical guide. the information management journal, 38(1), 58–64. hood, l., & flores, m. (2012). a personal view on systems medicine and the emergence of proactive p4 medicine: predictive, preventive, personalized and participatory. new biotechnology, 29(6), 613–624. http://doi.org/10.1016/j.nbt.2012.03.004 johnson, d. (2014, july 24). dhf partners with helparound in an effort to connect people touched by diabetes | diabetes hands foundation. retrieved from http://diabeteshandsfoundation.org/dhf-and-helparound-work-together-to-bringpeople-touched-by-diabetes-closer/ johnson, t. l., & parker, a. l. (2006). rates of retrieval of self-injectable epinephrine prescriptions: a descriptive report. annals of allergy, asthma & immunology: official publication of the american college of allergy, asthma, & immunology, 97(5), 694– 697. http://doi.org/10.1016/s1081-1206(10)61102-7 kastner, m., harada, l., & waserman, s. (2010). gaps in anaphylaxis management at the level of physicians, patients, and the community: a systematic review of the literature. allergy, 65(4), 435–444. http://doi.org/10.1111/j.1398-9995.2009.02294.x klasnja, p., & pratt, w. (2012). healthcare in the pocket: mapping the space of mobile-phone health interventions. journal of biomedical informatics, 45(1), 184–198. http://doi.org/10.1016/j.jbi.2011.08.017 latane, b., & darley, j. m. (1970). the unresponsive bystander: why doesn’t he help? prentice-hall. levine, m., prosser, a., evans, d., & reicher, s. (2005). identity and emergency intervention: how social group membership and inclusiveness of group boundaries shape helping behavior. personality and social psychology bulletin, 31(4), 443–453. http://doi.org/10.1177/0146167204271651 madigan, l., & hasbrouck, l. (2012, august 13). emergency epinephrine act (public act 970361) physician’s toolkit. illinois department of public health. maio, r. f., garrison, h. g., spaite, d. w., desmond, j. s., gregor, m. a., cayten, c., … others. (1999). emergency medical services outcomes project i (emsop i): prioritizing conditions for outcomes research. annals of emergency medicine, 33(4), 423–432. malvey, d., & slovensky, d. j. (2014). mhealth: transforming healthcare. springer. retrieved from australasian journal of information systems schwartz, bellou, garcia-castrillo, muraro & papadopoulos 2017, vol 21, research on health information systems mhealth in emergency response communities 15 https://books.google.co.nz/books?hl=en&lr=&id=f2ygbaaaqbaj&oi=fnd&pg=pp5&d q=malvey+d,+slovensky+dj.+mhealth:+transforming+healthcare.+new+york:+spri nger,+2014.&ots=wwiyokrgu-&sig=futgdfhfbnuizmiiz1xptyxvr4u manivannan, v., hess, e. p., bellamkonda, v. r., nestler, d. m., bellolio, m. f., hagan, j. b., … campbell, r. l. (2014). a multifaceted intervention for patients with anaphylaxis increases epinephrine use in adult emergency department. the journal of allergy and clinical immunology: in practice, 2(3), 294–299.e1. http://doi.org/10.1016/j.jaip.2013.11.009 marshall, a. h., cairns, k. j., kee, f., moore, m. j., hamilton, a. j., & adgey, a. (2006). a monte carlo simulation model to assess volunteer response times in a public access defibrillation scheme in northern ireland. in computer-based medical systems, 2006. cbms 2006. 19th ieee international symposium on (pp. 783–788). ieee. retrieved from http://ieeexplore.ieee.org/document/1647666/ martínez-pérez, b., de la torre-díez, i., & lópez-coronado, m. (2013). mobile health applications for the most prevalent conditions by the world health organization: review and analysis. journal of medical internet research, 15(6), e120. http://doi.org/10.2196/jmir.2600 mckemmish, s., burstein, f., faulkhead, s., fisher, j., gilliland, a. j., mcloughlin, i., & wilson, r. (2012). working with communities: community partnership research in information technology, management and systems. information, communication & society, 15(7), 985–990. merchant, r. m., asch, d. a., hershey, j. c., griffis, h. m., hill, s., saynisch, o., … others. (2013). a crowdsourcing innovation challenge to locate and map automated external defibrillators. circulation: cardiovascular quality and outcomes, 6(2), 229–236. mhra. (2014). administration of injectable medicines for the purpose of saving life in an emergency. retrieved september 2, 2013, from http://www.mhra.gov.uk/howweregulate/medicines/availabilityprescribingsellingand supplyingofmedicines/exemptionsfrommedicinesactrestrictions/administrationofinje ctablemedicinesforthepurposeofsavinglifeinanemergency/index.htm mingers, j. (2000). an idea ahead of its time: the history and development of soft systems methodology. systemic practice and action research, 13(6), 733–755. musse, s. r., & thalmann, d. (1997). a model of human crowd behavior: group interrelationship and collision detection analysis. springer. retrieved from http://link.springer.com/chapter/10.1007/978-3-7091-6874-5_3 pardun, j. t. (1998). good samaritan laws: a global perspective. loyola of los angeles international and comparative law journal, 20, 591. parmanto, b., pramana, g., yu, d. x., fairman, a. d., dicianno, b. e., & mccue, m. p. (2013). imhere: a novel mhealth system for supporting self-care in management of complex and chronic conditions. jmir mhealth and uhealth, 1(2), e10. http://doi.org/10.2196/mhealth.2391 peberdy, m. a., ottingham, l. v., groh, w. j., hedges, j., terndrup, t. e., pirrallo, r. g., … sehra, r. (2006). adverse events associated with lay emergency response programs: the public access defibrillation trial experience. resuscitation, 70(1), 59–65. http://doi.org/10.1016/j.resuscitation.2005.10.030 piliavin, i. m., rodin, j., & allyn piliavin, j. (1969). good samaritanism: an underground phenomenon? journal of personality and social psychology, 13, 289–299. piliavin, j. a. (2009). altruism and helping: the evolution of a f ield: the 2008 cooley-mead presentation. social psychology quarterly , 72(3). http://doi.org/10.1177/019027250907200305 australasian journal of information systems schwartz, bellou, garcia-castrillo, muraro & papadopoulos 2017, vol 21, research on health information systems mhealth in emergency response communities 16 presley, a., sarkis, j., & liles, d. h. (2000). a soft-systems methodology approach for product and process innovation. ieee transactions on engineering management, 47(3), 379– 392. rea, t., blackwood, j., damon, s., phelps, r., & eisenberg, m. (2011). a link between emergency dispatch and public access aeds: potential implications for early defibrillation. resuscitation, 82(8), 995–998. reicher, s. (2001). the psychology of crowd dynamics. blackwell handbook of social psychology: group processes, 182–208. roloff, m. e. (1981). interpersonal communication: the social exchange approach. sage publications beverly hills, ca. retrieved from http://library.wur.nl/webquery/clc/163266 sabaté, e. (2003). adherence to long-term therapies: evidence for action. world health organization. sakai, t., iwami, t., kitamura, t., nishiyama, c., kawamura, t., kajino, k., … others. (2011). effectiveness of the new “mobile aed map”to find and retrieve an aed: a randomised controlled trial. resuscitation, 82(1), 69–73. schenk, e., & guittard, c. (2009). crowdsourcing: what can be outsourced to the crowd, and why? retrieved from http://halshs.archives-ouvertes.fr/halshs-00439256/ schwartz, d. g., bellou, a., garcia-castrillo, l., muraro, a., & papadopoulos, n. g. (2014). towards chronic emergency response communities for anaphylaxis. in 2014 ieee 15th international conference on information reuse and integration (iri) (pp. 98–103). https://doi.org/10.1109/iri.2014.7051878 shortliffe, e. h., & cimino, j. j. (2006). biomedical informatics: computer applications in health care and biomedicine. springer. shostak, s., & fox, n. s. (2012). forgetting and remembering epilepsy: collective memory and the experience of illness. sociology of health & illness, 34(3), 362–378. shrank, w. h., choudhry, n. k., swanton, k., & et al. (2011). variations in structure and content of online social networks for patients with diabetes. archives of internal medicine, 171(17), 1589–1591. http://doi.org/10.1001/archinternmed.2011.407 simons, f. e. r., & world allergy organization. (2009). epinephrine auto-injectors: first-aid treatment still out of reach for many at risk of anaphylaxis in the community. annals of allergy, asthma & immunology: official publication of the american college of allergy, asthma, & immunology, 102(5), 403–409. http://doi.org/10.1016/s10811206(10)60512-1 smith, c. w., graedon, j., & graedon, t. (2016). editorial policies. retrieved may 1, 2016, from http://www.jopm.org/editorial-policies/#focusandscope snyder, m., & dwyer, p. c. (2012). altruism and prosocial behavior. handbook of psychology, 5, 467–486. song, t. t., worm, m., & lieberman, p. (2014). anaphylaxis treatment: current barriers to adrenaline auto-injector use. allergy. http://doi.org/10.1111/ all.12387. standards of medical care in diabetes--2012. (2011). diabetes care, 35(supplement_1), s11– s63. http://doi.org/10.2337/dc12-s011 stotland, e. (1969). exploratory investigations of empathy. advances in experimental social psychology, 4, 271–314. stowell, f. (2013). the appreciative inquiry method—a suitable candidate for action research? systems research and behavioral science, 30(1), 15–30. australasian journal of information systems schwartz, bellou, garcia-castrillo, muraro & papadopoulos 2017, vol 21, research on health information systems mhealth in emergency response communities 17 stürmer, s., snyder, m., & omoto, a. m. (2005). prosocial emotions and helping: the moderating role of group membership. journal of personality and social psychology, 88(3), 532–546. http://doi.org/10.1037/0022-3514.88.3.532 surowiecki, j. (2004). the wisdom of crowds: why the many are smarter and how collective wisdom shapes business, economies, societies, and nations, doubleday. swan, m. (2009). emerging patient-driven health care models: an examination of health social networks, consumer personalized medicine and quantified self-tracking. international journal of environmental research and public health, 6(2), 492–525. u.k. social action, responsibility and heroism act 2015, § 3 (2015). retrieved from http://www.legislation.gov.uk/ukpga/2015/3/introduction/enacted unertl, k. m., weinger, m. b., johnson, k. b., & lorenzi, n. m. (2009). describing and modeling workflow and information flow in chronic disease care. journal of the american medical informatics association, 16(6), 826–836. us department of health and human services. (2007). expert panel report 3: guidelines for the diagnosis and management of asthma. retrieved from http://www.nhlbi.nih.gov/guidelines/asthma/asthgdln.htm van der eijk, m., faber, m. j., aarts, j. w., kremer, j. a., munneke, m., & bloem, b. r. (2013). using online health communities to deliver patient-centered care to people with chronic conditions. journal of medical internet research, 15(6), e115. http://doi.org/10.2196/jmir.2476 weitzman, e. r., kelemen, s., quinn, m., eggleston, e. m., & mandl, k. d. (2013). participatory surveillance of hypoglycemia and harms in an online social network. jama internal medicine, 173(5), 345–351. http://doi.org/10.1001/jamainternmed.2013.2512 wicks, p., massagli, m., frost, j., brownstein, c., okun, s., vaughan, t., … heywood, j. (2010). sharing health data for better outcomes on patientslikeme. journal of medical internet research, 12(2). http://doi.org/10.2196/jmir.1549 copyright: © 2017 schwartz, bellou, garcia-castrillo, muraro & papadopoulos. this is an open-access article distributed under the terms of the creative commons attributionnoncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ http://creativecommons.org/licenses/by-nc/3.0/au/ microsoft word ajis_18(3)_v10.docx australasian journal of information systems volume 18 number 3 2014 271 absorptive capacity and contextual factors that influence green it assimilation vanessa a. cooper school of business it and logistics rmit university melbourne, australia vanessa.cooper@rmit.edu.au alemayehu molla school of business it and logistics rmit university melbourne, australia alemayehu.molla@rmit.edu.au abstract the first wave of research in green it has often focused on organisational adoption. as green it matures in organisations it is important to look beyond adoption and to investigate the assimilation of green it. to this end we draw from and compare two theories – contextual theory and absorptive capacity – and investigate which of the two theories better explains the level of green it assimilation in organisations. results from an international survey of 148 large organisations show that both theories explain green it assimilation with a medium effect size and that while contextual theory has a slightly higher r2 value than absorptive capacity, the difference is not statistically significant. we then propose a parsimonious and integrated model of green it assimilation drawing on contextual and absorptive capacity theories and outline implications for practitioners. the integrated model is parsimonious and has a higher explanatory power implying that a combination of contextual and absorptive capacity factors influences why and how widely and deeply green it practices, technologies and values are embedded in the it people, in the it management and it infrastructure of organisations. keywords: green it, it assimilation, absorptive capacity, contextual theory, organisation technology environment context. introduction the worldwide use of information technology (it) consumes vast amounts of electricity which led gartner in its 2007-2010 technology analyses to feature green it, narrowly defined as data centre energy efficiency, as one of the top 10 strategic technologies. in gartner’s analysis, a strategic technology has “high potential for disruption to it or the business and requires a major dollar investment” to avoid the risk of late adoption. over the last few years, in addition to the increasing energy intensity, there is increased awareness about the environmental foot print of it. the manufacturing of it requires rare and toxic resources and raises environmental and ethical issues throughout the supply chain, while the electricity used to power it is related to emissions, and the disposal of it equipment generates e-waste that can have negative impacts on the natural environment (murugesan 2008). as a result the conception and practice of green it has moved from the narrow focus on data centre energy efficiency to cover strategies, practices and technologies that improve the environmental footprint of the production, use and disposal of it such as computers, servers, and associated subsystems (dedrick 2010). green it also includes the beliefs, responsibilities and actions (both individually as well as within the organisational context) of it professionals to prevent pollution, improve product stewardship and contribute to sustainable development (molla et al. 2014). australasian journal of information systems volume 18 number 3 2014 272 a number of previous studies have contributed to the understanding and practice of green it, however defined. the energy informatics framework proposes research regarding sensor networks, flow networks, information systems and sensitised objects to reduce society’s energy consumption (watson et al. 2010). the carbon productivity framework addresses the role of it in increasing carbon productivity (dedrick 2010). the belief–action–outcome framework offers questions to guide research on green it belief formation, action and outcome (melville 2010). it-enabled business transformation was developed to address key issues and questions about the role of it in environmental sustainability (elliot 2011). the multi-level framework for environmentally sustainable it provides several theoretical propositions and suggestions for future research in the area of green it including environmental sustainability motivating forces, environmental sustainability initiatives, environmental orientation and environmental impacts (jenkin et al. 2011). all are important to guide practice, inform policy, disseminate good practices and build a cumulative body of knowledge that is beginning to make an impact. for the is discipline, continued green it research demonstrates is scholarship and scholars’ engagement with and contribution to the development of environmentally sustainable social and business practices. our overall intention in this paper is therefore to build on and contribute to the green it body of knowledge by addressing three issues. first, a number of the green it studies (chen et al. 2010; gholami et al. 2013; mishra et al. 2014) focus on green it adoption. these studies explain either theoretically or empirically, or both, the variations in adoption and the relationships between adoption and use and other technological, organisational, environmental and institutional antecedent factors. the adoption of a new technology implies the initial success and tells little if that technology has moved beyond the initial trial and becomes embedded into the routines of organisations, that is, the extent to which it is assimilated. technology assimilation as opposed to adoption refers to the acquisition, full utilisation, and institutionalisation of a technology (zhu et al. 2006). technology assimilation reflects to what extent a technology is utilised, both in terms of depth and breadth, and helps to unravel how organisations and other adopting entities leverage the advantages of a given technology (zhu et al. 2006). in this paper, we focus on green it assimilation and extend previous green it adoption studies. second, existing studies offer two main perspectives on green it driven either by organisation, technology and environment contextual and institutional factors (chen et al. 2010; dao et al. 2011) and attitudinal and motivational factors (mishra et al. 2014; molla et al. 2012). while these perspectives provide useful insight, we argue that an absorptive capacity perspective might provide as good an explanation as the two perspectives but is largely missing from extant green it literature. environmental sustainability, which is the ultimate goal of green it practice, is inherently linked to learning and innovation (elliot 2013). organisational learning capability is usually anchored to the concept of absorptive capacity which is defined as the ability to recognise the value of new and external information, assimilate and apply it (zahra et al. 2002). roberts et al. (2012) provide a comprehensive review of the ways in which is researchers have conceptualised, analysed and measured absorptive capacity. their review of 98 papers highlights that the absorptive capacity perspective has been used in analysing it assimilation (such as enterprise systems) (roberts et al. 2012). this study is the first to apply the absorptive capacity perspective to green it assimilation. third, in addition to adopting an absorptive capacity perspective, the study compares contextual and absorptive capacity perspective models and investigates which of the two perspectives provide a better explanation of green it assimilation. this has important implications for influencing managerial practice as well as advancing the theoretical foundation of green it scholarship. we are not aware of other green it research that attempted such an exercise. the research questions we address are, therefore, what influences green it assimilation? and which of the two theories – absorptive capacity vs. contextual, better explain variation in green it assimilation? australasian journal of information systems volume 18 number 3 2014 273 the contributions to green it research include theory building that explains green it assimilation and the factors which influence this. further, following a comparison of the utility of contextual and absorptive capacity theories in explaining green it assimilation, we propose an integrated model of green it assimilation. these efforts can be used in future studies. in terms of contributions to practice, by identifying the constructs that have most effect on green it assimilation, we provide insight for practitioners that might guide them in terms of where they might best focus their efforts and limited resources. background literature four areas of literature are relevant – green it, it assimilation; organisation, technology and environment (ote) and absorptive capacity. these are discussed in the following section. green it green it aims to minimise the negative impact of it operations on the environment through the sourcing, operations and end of life management of it and it-related products in an environmentally friendly way (murugesan 2008). green it requires that environmental considerations be embedded in the technical it infrastructure as well as in the it human infrastructure and it managerial capability (molla et al. 2011). in terms of the technical it infrastructure, green it focuses on improving energy efficiency in the powering and cooling of it (kumar et al. 2014), including the use of renewable energy to power the it infrastructure (chen et al. 2010), and on reducing greenhouse gas emissions through technologies such as server and storage virtualisation (bose and luo 2011) and data centre airflow and power management (alaraifi et al. 2013). in terms of the it human and it managerial capability dimensions, the application and realisation of environmental sustainability-oriented practices in the sourcing, operation and disposal of the it infrastructure, such as the analysis of the environmental track record of it vendors as part of their procurement decisions (molla et al. 2011) and electronic waste disposal (pant 2014) is important. further, defining clear governance models for the administration of green it, establishing policies that prescribe systematic frameworks to apply environmental criteria in it-related activities (molla et al. 2011) and developing environmental awareness of it managers and professionals (gholami et al. 2013) are key considerations within the it human and it management dimensions of the it infrastructure. green it is of increasing importance to organisations. research has identified a range of drivers for green it including increasing energy costs (alaraifi et al. 2013); coercive and mimetic pressures (chen et al. 2010; gholami et al. 2013); environmental legislation that addresses the disposal of electronic waste and energy efficiency (bose et al. 2011); and public perceptions of corporate strategies and operations (molla et al. 2012). as indicated earlier in the introduction, most of these green it studies tend to focus on adoption and not assimilation. in the following section we introduce the it assimilation construct. it assimilation is researchers have distinguished between diffusion, the process whereby technology spreads across a population, and assimilation, being the process by which progress from initial awareness of an innovation to formal adoption and full-scale deployment is made (fichman and kremer 1999). the it assimilation construct has been defined in the is literature in different ways. massetti and zmud (1996) provided one of the most widely used definitions of it assimilation constituted of four facets: volume, diversity, breadth and depth. volume represents the percentage of an organisation's processes that are handled through a system. diversity refers to the variety of business functions that are performed routinely through a system. breadth represents the extent to which an organisation has used a technology to conduct routine functions and depth refers to the degree of a system’s functionalities that australasian journal of information systems volume 18 number 3 2014 274 has been established in performing the business processes. thus it assimilation can also be defined as “the extent to which the use of technology diffuses across organizational projects or work processes and becomes routinized in the activities of those projects and processes” (roberts et al. 2012, p. 636). it assimilation is of much interest to researchers and practitioners alike. we now turn to two theories which highlight the factors which influence it assimilation in organisations. the organisation technology environment (ote) framework tornatzky et al. (1990) developed the ote framework from rogers’s (1995) diffusion of innovation theory by adding the environmental context as a third factor together with roger’s two factors of technology and organisation. this addition was important for addressing innovation use in a complex environment in which the external environment could provide both constraints and opportunities (tornatzky et al. 1990). the ote identifies three categories of contextual factors that influence the assimilation of technology: the technological context which refers to existing and new technology, the organisational context which refers to firms’ measurable characteristics such as size, scope and resource availability, and the environmental context which refers to the external environment in which firms conduct their business (such as industry, market participants, and government) and extends to any factor or source that could directly or indirectly motivate and/or inhibit firms’ operation or decision towards innovation (tornatzky et al. 1990). the ote provides a generic foundation that integrates the organisational, technological and environmental contexts to understand the factors that could affect the assimilation of technologies. the ote framework has been widely used in is research to study the implementation and use of innovations such as object-oriented technology (cho and kim 2002), e-business (e.g. zhu et al. 2006), erp assimilation (liang et al. 2007), and green it (e.g. bose and luo 2011). however, the factors that constitute the three contexts differ from one study to another. for example, zhu et al. (2006) identify technology readiness, technology integration, firm size, global scope, managerial obstacles, competition intensity and the regulatory environment as important contextual explanations of ebusiness assimilation. cho and kim (2002) include expectation for market trend, maturity of technology, intensity of new technology education and satisfaction with existing technology. liang et al. (2007) in their study of the determinants of erp success have identified (a) under technological context, erp attributes and expertise; (b) under organisational context, top management support, strategic alignment, managers’ and users’ involvement, reward systems and culture; and (c) under environmental context, institutional pressures and consultant effectiveness (kouki et al. 2010; liang et al. 2007). in addition, researchers have introduced variables that are specific to their focus of study under the generic ote framework. examples include manufacturing context, manufacturing technologies, networking intensity (raymond et al. 2005), trading partner pressure (hsu et al. 2006), reward system, consultant effectiveness and vendor support (kouki et al. 2010). this implies that by building on the ote structure, researchers can develop the relevant factors that are specific to their research context and focus only on the most relevant factors. further, researchers (cho and kim 2002) argue that some of the technological, organisational and environmental factors that appear to be more important indicators for the adoption stage could be less important or irrelevant to the post-adoption stage. therefore, researchers should carefully select the important factors to their research context. as such, the ote has a generic nature that makes it suitable to study different types of technology innovation and accommodates context-based constructs. thus it is useful to be applied as one of the conceptual foundations for investigating green it assimilation. while ote factors can provide the contextual factors that influence it assimilation, they do not capture the dynamics of learning, thus a few researchers have followed the absorptive capacity perspective. australasian journal of information systems volume 18 number 3 2014 275 absorptive capacity absorptive capacity was first introduced to an organisational context by cohen and levinthal (1990) and refers to the ability of a firm to recognise the value of new, external information, assimilate it, and apply it to commercial ends. absorptive capacity is conceptualised by scholars in a variety of ways but most tend to model absorptive capacity as comprising either two, three or four underlying constructs. the most frequently cited work to date is the model provided by zahra and george (2002). this model identifies that absorptive capacity comprises the four processes of acquisition, assimilation, transformation and exploitation and these processes are grouped into two higher order dimensions, namely, potential and realised absorptive capacity. potential absorptive capacity comprises acquisition, which refers to a firm’s capability “to identify and acquire externally generated knowledge that is critical to its operations” (zahra and george 2002, p.189) and assimilation, being the firm’s “routines and processes that allow it to analyse, process, interpret and understand information obtained from external sources” (ibid, p. 189). zahra and george (2002) argue that while potential absorptive capacity enables the firm to be receptive to acquiring and assimilating external knowledge it does not guarantee that a firm will leverage and exploit this knowledge, rather, this requires realised absorptive capacity. realised absorptive capacity comprises transformation, which represents the firm’s capability to “develop and refine the routines that facilitate combining existing knowledge and the newly acquired and assimilated knowledge” (ibid, p. 190) along with exploitation, which is the firm’s ability to “refine, extend and leverage existing competencies or to create new competencies by incorporating transformed knowledge into its operations” (ibid, p. 190). absorptive capacity has been used to investigate innovation and firm performance (e.g. cohen and levinthal 1990; gluch et al. 2009), and importantly, it assimilation (saref et al. 2012; roberts et al. 2012). model specification and hypothesis development based on the review of the literature, in this section we introduce the two models and related hypotheses. model 1contextual-based model of green it assimilation the contextual model is composed of three constructs: organisational, technological, environmental (figure 1). generally speaking, organisational context refers to descriptive measures such as slack resources, structural characteristics and the characteristics of leaders (such as their commitment) (fichman 2000). perceived availability and allocation of sufficient organisational resources has been argued to be an important factor for the adoption and assimilation of it because if the firm does not perceive the benefits of technology can be achieved within the allotted resources, adoption, and therefore assimilation of it, becomes futile (kuan and chau 2001). structural characteristics include firm size and scope, the degree of centralisation, formalisation and the complexity of the managerial structure (zhu et al. 2004). top management support and managerial knowledge are often cited as key elements in facilitating the assimilation of it. top management influences the perceived importance of green it which in turn influences resource allocation and investment in green it (mithas et al. 2010). the support of top management, especially support of long term strategies, will influence the formation and creation of policy and governance for green it (molla et al. 2011) and provides a positive environment for the success of green it (armstrong and sambamurthy 1999). environmental commitment of managers can for example, lead organisations to embrace green it strategy. actions undertaken by senior management can introduce complementary structures to facilitate green it assimilation, and modify and reinforce the norms that value the use of green it. there is a greater likelihood of green it assimilation when positive top management mind-set has been communicated effectively to the users. indeed the wider attitude of it people, which encompasses their sentiment, values and norms toward climate change and eco-sustainability and it’s role in green it (gholami et australasian journal of information systems volume 18 number 3 2014 276 al. 2013; molla et al. 2011) is important to ensure green it practices and technologies are embedded across an organisation at an operational level. thus we hypothesize hc1: organisational context such as resource commitment, formal structure and top management commitment, positively influences green it assimilation within the context of ote and drawing from prior studies (zhu and kraemer 2005) we consider the technological context to represent the it assets including the it people of organisations. it refers to organisation’s it size in terms of the number of it staff, and the personal computer and server (both physical as well as virtual) fleets. the technology context shows the depth, breadth and sophistication of it usage in an organisation. it can influence green it assimilation in three ways. first, firms with more it resources may consider the environmental impact of those assets and may demonstrate greater commitment and managerial support to use of green it to minimize their footprint. second, previous research indicates that the level of it resources a firm accumulates is an indication of a positive attitude towards extending it use (hsu et al. 2006). further, studies have also found that there is a significant relationship between technological resources and organisational innovativeness (zhu et al. 2004). this leads us to the following hc2: a firm with more technological resources will have greater assimilation of green it hc3: a firm with more technological resources will develop favourable organisational context for green it assimilation environmental context refers to the institutional and market environment within which organisations operate. market and regulatory forces, when present and mature, can create conducive conditions for widening and deepening green it implementations (molla et al. 2012). stakeholders such as suppliers, customers, industry associations and regulatory agencies have influenced the assimilation of technologies within data centre (alaraifi et al. 2013) and green it policies (chen et al. 2010). market and non-market external pressures, in addition to directly affecting green it assimilation, can also influence managers to improve their environmental performance (gholami et al. 2013). therefore, as the institutional, industry and market dynamics for green it become favourable, organisations might commit resources, structure and support towards either exploring or exploiting various green it technologies and practices. on the basis of these, we formulate the following two hypotheses hc4: the environmental context for green it such as market, regulatory and institutional factors positively influences green it assimilation hc5: the environmental context for green it positively influences the organisational context for green it australasian journal of information systems volume 18 number 3 2014 277 figure 1: context-based model of green it assimilation model 2absorptive capacity based model of green it assimilation both the environmental sustainability and is literature use absorptive capacity to understand the adoption of environmentally friendly technologies and it assimilation respectively. in terms of environmental sustainability, gluch et al. (2009) found that in the construction industry absorptive capacity explains green innovation and environmental performance. absorptive capacity has been found to explain the undertaking of proactive environmental strategy (delmas et al. 2011), environmental commitment of canadian small-medium enterprises (smes) (roy et al. 2008), the diffusion of clean technology in the spanish pulp and paper industry (del río gonzález 2005) and the development of clean technology and production (vickers 1999). in terms of it assimilation, roberts et al. (2012) provide a comprehensive review of the ways in which is researchers have conceptualised, analysed and measured absorptive capacity. their review of 98 papers highlights that one of the themes in the application of absorptive capacity in is research is it assimilation. is researchers have consistently found that there is a positive relationship between absorptive capacity and it assimilation as high absorptive capacity in a domain increases the capability of the organisation to assimilate complex innovations in that domain (roberts et al. 2012; sharma et al. 2012). it education (e.g. new technology education and learning) and it capabilities (e.g. investment in it infrastructure), often triggered by external institutional pressures (saraf et al. 2012), have been found to increase related knowledge and the diversity of knowledge held (i.e. absorptive capacity). this in turn improves it assimilation. in zahra and george’s (2002) initial model of absorptive capacity, potential and realised absorptive capacity were seen to have separate but complementary roles, with both being necessary to improve firm performance. that is, firms cannot transform and exploit knowledge (i.e. develop realised absorptive capacity) without first acquiring and assimilating knowledge (i.e. developing potential absorptive capacity). zahra and george (2002) advocate that while potential absorptive capacity is necessary to improve firm performance it is not sufficient alone. more recent empirical studies (e.g. saref et al. 2012) have found that both potential absorptive capacity (weak support) and realised absorptive capacity (strong support) have a direct influence on is assimilation. most studies report that potential absorptive capacity, either directly or indirectly, that is, through realised absorptive capacity, influences is assimilation (cooper and molla. 2012; gluch et al. 2009; saraf et al. 2012; sharma et al. 2012).therefore our absorptive capacity model tests the relationships among potential green it australasian journal of information systems volume 18 number 3 2014 278 absorptive capacity, realised green it absorptive capacity and green it assimilation, with the following three hypotheses (figure 2) ha1: potential green it absorptive capacity significantly contributes to green it assimilation ha2: realised green it absorptive capacity significantly contributes to green it assimilation ha3: potential green it absorptive capacity significantly contributes to realised green it absorptive capacity figure 2: absorptive capacity-based model of green is assimilation research method to collect the data, we first developed a measurement instrument to test the hypotheses and compare the research models. green it assimilation is measured based on five items adopted from the greadiness instrument (molla et al. 2011). organisational context is measured based on fichman and carroll (1999) using three items covering top management support, formal organisational structure and resource commitment. technology context was measured using three items that reflect the it resources (hardware) and the size of the it function (it people). three items are drawn from molla et al. (2012) to capture the environmental context of regulatory, industry and market dynamics to benefit from green it. the potential and realised absorptive capacity constructs were modelled and measured as second-order reflective-formative constructs as illustrated in figure 3. the use of higher order constructs is appropriate to model multi-dimensional latent variables and to reduce the complexity of structural models (hair et al. 2014). potential green it absorptive capacity is measured as a reflective-formative second-order construct, with green it knowledge acquisition and assimilation as its underlying first order dimensions. likewise, realised green it absorptive capacity is modeled as a reflective-formative second-order construct, with green it knowledge transformation and exploitation as its underlying first order dimensions. items to operationalise the four first order constructs are adopted from gluch et al. (2009) and zahra et al. (2002). here six items are used to capture the ability to acquire new knowledge on green it; four items to measure assimilation, that is, the routines and processes to interpret external knowledge; four items to assess the mechanisms for transforming and integrating new knowledge with existing knowledge, and four items to measure the routines and processes to exploit green it knowledge. see appendix 1 for the list of questions and items. australasian journal of information systems volume 18 number 3 2014 279 figure 3: the measurement model of potential and realised green it absorptive capacity after the instrument was developed, a panel of three experts reviewed the items for content validity. it managers from large australian, new zealand and us organisations were sampled to complete the survey. respondents were asked to respond to questions on a five point likert scale ranging from 1 = very low to 5 = very high. the survey had 179 responses (109 australian; 21 new zealand, 14 u.s. and 35 missing) out of which 148 were usable. eighty percent of the respondents were senior it managers (such as cios) working in large organisations (71%) (more than 500 full time equivalent (fte) employees) in the services (23%), manufacturing (20%), utilities and transport (12%) and education and government (20%) sub sectors. the it departments of most of the respondents’ (72%) organisations have more than 10 fte staff. analyses and results the psychometric property (that is reliability and validity) of the instrument was tested using partial least square in two steps. in the first step, the first order measurement models were tested for convergent and discriminant validity. the result (appendix 1) shows that the measurement models are valid and reliable. in terms of convergent validity, all factor loadings are greater than 0.7 and the average variance extracted is greater than 0.5. the composite reliability and cronbach alpha values are also greater than 0.7 providing evidence of internal consistency of the measures. further, the square root of the ave is larger than the off diagonal inter-construct correlation (see table 1) and supports discriminant validity. 1 2 3 4 5 6 7 8 1 green it knowledge acquisition 0.77* 2.green it knowledge assimilation 0.74 0.87 3. environmental context 0.62 0.61 0.90 4. green it knowledge exploitation 0.64 0.78 0.63 0.94 5. green it assimilation 0.64 0.69 0.67 0.70 0.87 6. organisational context 0.60 0.59 0.66 0.65 0.72 0.87 7. technology context 0.18 0.11 0.16 0.04 0.05 0.13 0.92 8. green it knowledge transformation 0.68 0.80 0.58 0.86 0.72 0.60 0.07 0.92 * diagonal values in bold and italics are square root of the ave table 1: discriminant validity of the first order measurement model in the second step, the two second order constructs that is potential and realised green it absorptive capacity were validated following a repeated indicator approach as described in hair et al. (2014, p231) and tested for collinearity of indicators. since the t-values of the outer weights are significant, all vif australasian journal of information systems volume 18 number 3 2014 280 values are <5.0 and the tolerance values are >0.20 (hair et al. 2014, p. 125), there is no collinearity problem. this shows adequate measurement validity of the second order models. outer weight (outer loading) t value vif tolerance potential green it absorptive capacity green it knowledge acquisition 0.56 (0.93) 23.20 2.30 0.44 green it knowledge assimilation 0.52 (0.94) 33.77 3.62 0.28 realised green it absorptive capacity green it knowledge transformation 0.53 (0.96) 73.44 4.24 0.24 green it knowledge exploitation 0.51 (0.97) 70.43 4.80 0.21 table 2: validation of the second order measurement models after the instrument was validated the structural models were tested using partial least squarestructural equation modeling (pls-sem). pls-sem was selected because of its relative robustness in working with smaller and larger samples; flexibility achieved by having few limiting assumptions regarding the model specifications and data; provision of parameter estimates that maximize the explained variance of dependent constructs (thus supporting prediction or theory building aims); and its ability to also be used for confirmatory theory testing and model comparison (hair et al. 2014). to test the second order structural model, a two stage latent variable approach was used (hair et al. 2014). figures 4a and 4b show the result of the structural model tests. figure 4a: structural test of the contextual model australasian journal of information systems volume 18 number 3 2014 281 figure 4b: green it absoptive capacity structural model to estimate the significance of the path coefficients, 5000 sample bootstrapping was performed. further, in order to estimate the predictive accuracy of the structural models, we also examine the stonegeisser’s q2 values. to compare the two models, following hair et al.’s (2014) recommendations, we have used several tests, such as effect sizes, adjusted r2 and q2 effect size. the results are summarized in table 3. m od el h yp ot he si s paths pa th c oe ff ic ie nt t va lu es sig r2 r2adj q2 f2 ef fe ct si ze q2 ef fe ct si ze c on te xt hc1 organisational context -> green it assimilation 0.50 4.50 ** 0.59 0.60 0.59 0.30 0.28 hc2 technology context -> green it assimilation 0.07 1.13 ns hc4 environmental context -> green it assimilation 0.35 2.88 * hc3 technology context -> organsational context 0.02 ns 0.44 hc5 environmental context -> organisational context 0.66 9.88 ** a bs or pt iv e c ap ac ity ha1 potential green it absorptive capacity -> green it assimilation 0.34 2.50 * 0.58 0.59 0.57 0.26 0.23 ha2 realised green it absorptive capacity -> green it assimilation 0.46 2.9638 * ha3 potential green it absorptive capacity -> realised green it absorptive capacity 0.80 24.9 ** 0.65 *significant at 0.01; ** significant at 0.001; ns not significant table 3 structural model results and comparison discussion as indicated in table 3, six out of the eight paths in the two models are significant at a 0.01 confidence level and some at a 0.001 confidence level thus supporting six of the theoretical hypotheses. the contextual model explains 59% of the variance of green it assimilation whereas the absorptive capacity australasian journal of information systems volume 18 number 3 2014 282 model explains 58%. in addition, environmental and technology contexts explain 44% of the variance in organisational context, however the path coefficient of the technology context is statistically insignificant (hypothesis c3). potential green it absorptive capacity explains 65% of the variation in realised green it absorptive capacity. further, two out of the three exogenous variables in the two models significantly influence green it assimilation such that the total effects of the exogenous variables, that is, potential green it absorptive capacity, and environmental context on green it assimilation are 0.40 (p= 0.01), and 0.31 (p=0.01) respectively. further of all the variables included in the two models, potential green it absorptive capacity has a higher magnitude of total (direct and indirect) effect (but with medium effect size) whereas organisational context has a higher magnitude (table 1) of direct effect (with medium effect size) on green it assimilation. however the size of the it function doesn’t seem to have any influence on green it assimilation (hypothesis c2). both models explain more than 50% of the variance in green it assimilation. the model comparison test results show that the contextual model has a marginally larger adjusted r2 and q2 values with a medium effect size whereas the absorptive capacity model has relatively lower adjusted r2 and q2 values but still with medium effect size. this means, at least in the current sample, both learning based and organisational and contextual factors explain why and how widely and deeply green it practices, technologies and values are embedded in the it people, in the it management and it infrastructure of organisations. with regards to the contextual model, the support for hypothesis c1 indicates that organisational context has a strong impact on green it assimilation (0.50). the support of senior managers has been found to impact on it assimilation generally (armstrong and sambamurthy 1999) and this has been found to be transferrable to the context of green it assimilation. in fischman and carroll’s (1999) review of factors having an impact on it diffusion and assimilation, the communication environment was found to be important, as was structural characteristics of an organisation – although with mixed results. some studies found that organisations with lower centralisation, formalisation and vertical differentiation are more likely to innovate, whereas others found that such firms will have difficulty reaching consensus about adopting innovations and less likely to sustain them. our study suggests that there is a need for formal structures that enable the exchange of ideas to enhance green it assimilation. further, the provision of sufficient resources will increase green it assimilation and this is consistent with wider research in it assimilation whereby a firm’s ‘slack resources’ and investment in, for example, training, (fischman and caroll 1999) affects it assimilation. the support for hypothesis c4 shows that the environmental context has a medium effect on green it assimilation (0.35). market demand for green it products, processes and services was found to have a positive effect on green it assimilation in our study, which is consistent with the work of molla and abareshi (2012) wherein market forces were identified as a motive of early adoption of green it practices. institutional and industry dynamics were perceived to impact the ability of firms to assimilate green it. this finding, while consistent with studies that highlight institutional drivers as impacting on a firm’s environmental performance (e.g. zalani et al. 2012), is inconsistent with the finding of melville and salandha (2013) in their investigation of the antecedents for adoption of carbon management systems. this may be because their study operationalised external environmental factors in terms of global climate agreements whereas our survey did not tap into such agreements specifically. through support for hypothesis c5, our study indicates that the environmental context has a strong impact in terms of shaping the organisational context (0.66). this suggests that institutional theory (dimaggio and powell 1983) may shed further light on organisations’ response to environmental factors as they pertain to green it. with regards to model 2, absorptive capacity, the support for hypotheses a1 and a2 indicates that firms that are able to develop potential absorptive capacity and able to develop realised absorptive capacity, will be better able to assimilate green it. the results indicate that realised absorptive capacity australasian journal of information systems volume 18 number 3 2014 283 has a larger direct effect (0.46) on green it assimilation than does potential absorptive capacity (0.34). this is consistent with the theorization of absorptive capacity by zahra and george (2002). the dominance of realised absorptive capacity in explaining green it assimilation, however, does not render potential absorptive capacity unimportant. potential green it absorptive capacity has strong explanatory power for realised absorptive capacity (0.80), indicating that there must be sufficient understanding of green it issues in the first instance. further, our findings show that potential absorptive capacity has a significant direct effect on green it assimilation and this is consistent with saref et al. (2012) in their study on enterprise systems assimilation. other scholars have noted the importance of this relationship between potential and realised absorptive capacity (hypothesis a3), including in the is domain, where information systems have been argued to mediate the relationship between potential and realised absorptive capacity. for example, cepeda-carrion et al. (2012) found that while is capability is needed to enhance potential absorptive capacity to realised absorptive capacity, in turn, these two forms of capabilities, contribute to creating higher-order dynamic capabilities (hu et al. 2012) and mediate the influence of is capabilities (such as flexible is infrastructure, is integration and assimilation) on firm performance (francalanci and morabito 2008; liu et al. 2013). overall our study indicates that green it assimilation is influenced by potential and realised absorptive capacity through different pathways and thus adds to other studies that have explored potential and realised absorptive capacity on is assimilation. theoretical contribution: a parsimonious and integrated model of green it assimilation this study was set out to address the questions of what influences green it assimilation and which of the two theories – absorptive capacity vs. contextual – better explain variation in green it assimilation. we have shown, via a contextual model, that market, institutional and industry dynamics of green it as well as top management, formal organisational structure and organisational resources influence green it assimilation. we have also shown, via a model of absorptive capacity, that the green it knowledge acquisition and assimilation (potential absorptive capacity) together with transformation and exploitation processes (realised absorptive capacity), equally influence green it assimilation. the results further demonstrate that both models explain more than 50% of the variation in the green it assimilation of the sample. the explanatory powers of the models are better than chen et al.’s (2010) institutional theory based model, molla and abarehsi’s (2012) motivational theory based model and gholami et al.’s (2013) managerial perception and belief based model. this means researchers can use the models and hypotheses developed in this study in future studies. in terms of which model has better utility to understand green it assimilation, both models have medium effect but the contextual model shows a marginally higher r2 value. this finding has two implications. first, if one is seeking to choose between the two models, our results favour the contextual model which might be further improved by improving the measures of the technological context variables. nevertheless, as green it can be considered at the early stage of development, it remains to be seen if contextual factors continue to dominate the assimilation of green it or if this might change in the future as organisations move from reactive strategies and start integrating green it as part of building and renewing their overall it capabilities. second, in order to get a more comprehensive understanding of green it assimilation, a model that draws from both contextual and absorptive capacity theories and that integrates the external context as triggers of green it absorptive capacity and the organisational context as mediating mechanisms for building green it absorptive capacity can be considered. we believe that the integrated model will yield a better understanding and explanation. as a result, and as a theoretical contribution, we propose a parsimonious and integrated model of green it assimilation illustrated in figure 5. australasian journal of information systems volume 18 number 3 2014 284 figure 5: integrated model of green it assimilation the model explains 64% of the variation in green it assimilation, 68% of the variance in realised green it absorptive capacity and about 43% of variations in potential green it absorptive capacity, as well as 44% of organisational support and commitment for green it. all the paths are higher in magnitude and significant at 99% confidence interval. its predictive accuracy, that is, q2 is 64% showing large predictive relevance of the model to investigate three questions: what factors explain green it assimilation?; how can organisations develop their potential and realised green it absorptive capacity?; and what influences organisational commitment and support for green it? the model is parsimonious because it provides higher explanatory power with fewer paths. the integrated model and the paths have also strong foundation in the ote and absorptive capacity theories. the general proposition of the integrated model is that the external environment context for green it contributes positively to both organisational support and commitment, as well as the ability to acquire and assimilate green it knowledge. together these two contribute significantly to create a higher order and realized absorptive capacity by transforming and exploiting the acquired and assimilated knowledge. the higher order realised absorptive capacity and the continued management commitment, support and resource for green it lead to assimilation of green it. these general propositions have six individual hypotheses (the paths in figure 5) that can be retested in future studies. practical contribution: guiding practitioners towards green it assimilation this study provides a number of insights for practitioners endeavouring to assimilate green it in their organisations. first, our study provides insight into the environmental and organisational factors that influence green it assimilation. this understanding will become increasingly important to practitioners as organisations move beyond the initial stages of adoption and endeavour to fully assimilate green it (fichman and kremer 1999). importantly, our study has identified the strong influence that organisational factors such as formal structures, provision of resources and top management support have on green it assimilation. this indicates that practitioners should pay careful attention to these issues. second, we have shown that organisations can benefit from adopting a learning perspective towards green it. this is because developing higher levels of potential and realised absorptive capacity can lead to higher levels of green it assimilation. potential absorptive capacity in this context involves acquiring new knowledge from a diverse range of sources (e.g. from relationships with external stakeholders, participation in green it interest groups and undertaking formal green it training programs), and then assimilating the newly acquired knowledge through development of the routines and processes that enable an organisation to analyse, interpret and understand newly acquired knowledge. realised absorptive capacity requires firms to ensure that it staff understand green it australasian journal of information systems volume 18 number 3 2014 285 issues and concepts and enable staff to transform wider organisational routines and processes, and in turn, create new, refine, and exploit existing green it competencies. by unraveling the routines and processes organisations require for assimilating green it and by providing empirical evidence as to their influence, we provide insight for practitioners in terms of the capabilities they should learn to develop. conclusion and future work our study provides the foundation for undertaking theory building and explanation of green it as well as for influencing it-enabled actions for addressing environmental sustainability, which is an original contribution to the green it literature. in addition, the study adds to the empirical base of green it literature and can be considered as one of the few empirical studies to look beyond initial adoption of green it to understand its assimilation. thus, researchers can use our findings to compare what factors are common to influence both the adoption and the assimilation of green it and what factors are unique to each construct domain. the study has a number of limitations that offer avenues for future research. first, we have developed the models on the basis of a single sample and did not have a hold-out sample to re-test the model. second, our study has focused on large organisations and future research might investigate how each model explains green it assimilation in a variety of organisations. third the technology context as measured in this study has no direct effect in our model and researchers need to look into the measurement items as well as the possibility of moderating or controlling effect. the integrated model although theoretically sound, was driven from the data and requires further test in another sample and opens avenue for future research. research comparing green is assimilation and wider is assimilation in firms would be an interesting avenue for future research. references alaraifi, a., molla, a., and deng, h. 2013. "an empirical analysis of antecedents to the assimilation of sensor information systems in data centers," international journal of information technologies and systems approach (6:1), pp 57-77. bose, r., and luo, x. 2011. "integrative framework for assessing firms' potential to undertake green it initiatives via virtualization a theoretical perspective," the journal of strategic information systems (20:1), pp 38-54. chen, a. j., watson, r. t., boudreau, m. c., and karahanna, e. 2010. "an institutional perspective on the adoption of green is & it," australasian journal of information systems (17:1), pp 23-45. cohen, w. m., and levinthal, d. a. 1990. "absorptive capacity: a new perspective on learning and innovation," administrative science quarterly (35:1), pp 128-152. cooper, v., and molla, a. 2012. "developing green it capability: an absorptive capacity perspective," in pacis 2012 proceedings. paper 46. dao, v., langella, i., and carbo, j. 2011. "from green to sustainability: information technology and an integrated sustainability framework," the journal of strategic information systems (20:1), pp 63-79. del río gonzález, p. 2005. "analysing the factors influencing clean technology adoption: a study of the spanish pulp and paper industry," business strategy and the environment (14:1), pp 20-37. delmas, m., hoffmann, v. h., and kuss, m. 2011. "under the tip of the iceberg: absorptive capacity, environmental strategy, and competitive advantage," business & society (50:1), pp 116-154. australasian journal of information systems volume 18 number 3 2014 286 elliot, s. 2013. "a transdisciplinary exploratory model of corporate responses to the challenges of environmental sustainability," business strategy and the environment (22:4), pp 269-282. fichman, r. g. 2000. "the diffusion and assimilation of information technology innovations," in r. zmud, ed. framing the domains of it management: projecting the future through the past, pinnaflex publishing: cincinnati, ohio, pp. 105-128. fichman, r. g., and kemer, c. f. 1999. "the illusionary diffusion of innovation: an examination of assimilation gaps," information systems research (10:3), pp 255-275 gholami, r., sulaiman, a., ramayah, t. b., and molla, a. 2013. "senior management perception on green information systems adoption and environmental performance: results from a field survey," information and management (50:7), pp 431-438. gluch, p., gustafsson, m., and thuvander, l. 2009. "an absorptive capacity model for green innovation and performance in the construction industry," construction management and economics (27:5), pp 451-464. hair jr, j. f., hult, g. t. m., ringle, c. m., and sarstedt, m. 2014. a primer on partial least squares structural equation modeling (pls-sem), sage publications inc: thousand oaks, california. kouki, r., pellerin, r., and poulin, d. 2010. "investigating the determinants of effective enterprise resource planning assimilation: a cross-case analysis," international journal of business information systems (5:1), pp 58-83. liang, h., saraf, n., hu, q., and xue, y. 2007. "assimilation of enterprise systems: the effect of institutional pressures and the mediating role of top management," management information systems quarterly (31:1), p 6. massetti, b., and zmud, r. w. 1996. "measuring the extent of edi usage in complex organizations: strategies and illustrative examples," management information systems quarterly, pp 331345. mishra, d., akman, i., and mishra, a. 2014. "theory of reasoned action application for green information technology acceptance," computers in human behavior (36), pp 29-40. molla, a., and abareshi, a. 2012. "organizational green motivations for information technology: empirical study," journal of computer information systems (52:3), pp 92-102. molla, a., abareshi, a., and cooper, v. 2014. "green it beliefs and pro-environmental it practices among it professionals," information technology & people (27:2), pp 2-2. molla, a., cooper, v., and pittayachawan, s. 2011. "the green it readiness (g-readiness) of organizations: an exploratory analysis of a construct and instrument," communications of the association for information systems (29:1), pp 67-96. murugesan, s. 2008. "harnessing green it: principles and practices," it professional (10:1), pp 2433. roberts, n., galluch, p. s., dinger, m., and grover, v. 2012. "absorptive capacity and information systems research: review, synthesis, and directions for future research," information systems (6:1), pp 25-40. roy, m. j., and thérin, f. 2008. "knowledge acquisition and environmental commitment in smes," corporate social responsibility and environmental management (15:5), pp 249-259. saraf, n., liang, h., xue, y., and hu, q. 2012. "how does organisational absorptive capacity matter in the assimilation of enterprise information systems?," information systems journal (23:3), pp 245-267. sharma, s., daniel, e. m., and gray, c. 2012. "absorptive capacity and erp implementation in indian medium-sized firms," journal of global information management (20:4), pp 54-79. australasian journal of information systems volume 18 number 3 2014 287 vickers, i. 1999. "cleaner production and organizational learning," technology analysis & strategic management (11:1), pp 75-94. zahra, s. a., and george, g. 2002. "absorptive capacity: a review, reconceptualization, and extension," academy of management review, (27:2), pp 185-203. zhu, k., kraemer, k. l., and xu, s. 2006. "the process of innovation assimilation by firms in different countries: a technology diffusion perspective on e-business," management science (52:10), p 1557. an earlier version of this paper was presented at the australasian conference on information systems (acis) 2013 in melbourne, australia. australasian journal of information systems volume 18 number 3 2014 288 appendix 1: first order measurement model validity and reliability factor item factor loadings cr ave organisational context please rate the extent to which each the following has facilitated green it: o_fos formal organisational structures 0.90 0.91 0.84 0.76 o_res provision of sufficient resource 0.88 o_tms top management support 0.84 technology contexplease provide data for each of the following t_itstaff: number of it staff .99 0.95 0.92 0.85 t_pcs: number of personal computers .94 t_servers: number of servers .84 environmental context please rate the extent to which each the following has influenced green it: e_ind industry dynamics 0.92 0.93 0.88 0.81 e_inst institutional dynamics 0.91 e_mkt market demand 0.86 green it knowledge acquisition please rate the capability of your it department to acquire new knowledge on green it via: acq1 carrying out market research 0.71 0.90 0.86 0.59 acq2 sending it staff to complete training courses 0.74 acq3 participation in interest groups 0.83 acq4 relationships with external stakeholders 0.82 acq5 observation of approaches adopted by competitors 0.78 acq6 compliance with environmental legislation 0.72 green it knowledge assimilation please rate the capability of the routines and processes in your it department to: ass1 interpret and understand information on green it 0.86 0.93 0.89 0.76 ass2 ensure newly acquired green it knowledge is understood by it staff 0.88 ass3 conduct a lifecycle analysis of its products/services to identify their environmental impact 0.83 ass4 develop a plan of action on how to achieve environmental goals 0.90 green it knowledg transformation please rate your it department’s capability to: tfrm1 develop effective routines to facilitate the combination of newly acquired or assimilated knowledge with existing green it knowledge 0.92 0.96 0.94 0.85 tfrm2 refine its routines to facilitate the combination of newly acquired or assimilated knowledge with existing green it knowledge 0.92 tfrm3 develop green it performance goals (e.g. kpis) to reflect understanding gained from newly acquired or assimilated knowledge 0.91 tfrm4 refine green it performance goals (e.g. key performance indicators) to reflect understanding gained from newly acquired or assimilated knowledge 0.93 green it knowledge exploitation please rate the capability of your it department’s to develop and refine routines to: xplt1 ensure that existing green it competencies are effectively leveraged in its operations 0.95 0.97 0.96 0.89 xplt2 refine existing green it competencies 0.96 xplt3 extend existing green it competencies 0.95 xplt4 enable the creation of new green it competencies 0.91 green it assimilation please rate the extent to which your it department has: gr_att a positive green it attitude 0.79 0.94 0.92 0.75 gr_gov well-developed green it governance mechanisms 0.88 gr_pol well-developed green it policy frameworks 0.87 gr_prc well-developed green it practices 0.92 gr_tec acquired and build an environmentally effective it infrastructure 0.87 microsoft word 1734-5478-1-le.docx australasian journal of information systems de vass, shee & miah 2018, vol 22, research article iot and supply chain integration and performance 1 the effect of “internet of things” on supply chain integration and performance: an organisational capability perspective tharaka de vass college of business victoria university melbourne, australia tharaka.devass@vu.edu.au himanshu shee college of business victoria university melbourne, australia shah miah college of business victoria university melbourne, australia abstract the internet of things (iot) is a next generation of internet connected embedded ict systems in a digital environment to seamlessly integrate supply chain and logistics processes. integrating emerging iot into the current ict systems can be unique because of its intelligence, autonomous and pervasive applications. however, research on the iot adoption in supply chain domain is scarce and acceptance of the iot into the retail services in specific has been overly rhetoric. this study is drawn upon the organisational capability theory for developing an empirical model considering the effect of iot capabilities on multiple dimensions of supply chain process integration, and in turn improves supply chain performance as well as organisational performance. cross-sectional survey data from 227 australian retail firms was analysed using structural equation modelling (sem). the results indicate that iot capability has a positive and significant effect on internal, customer-, and supplier-related process integration that in turn positively affects supply chain performance and organisational performance. theoretically, the study contributes to a body of knowledge that integrates information systems research into supply chain integration by establishing an empirical evidence of how iot-enabled process integration can enhance the performance at both supply chain and organisational level. practically, the results inform the managers of the likely investment on iot that can lead to chain’s performance outcome. keywords: internet of things, supply chain integration, supply chain performance, firm performance, organisational capability theory 1 introduction internet of things (iot) represents an advancement in technological innovation connecting objects and devices through internet (atzori, iera, & morabito, 2010). the network of objects (e.g. devices, vehicles, machines, containers), embedded with sensors and software has the potential to collect and communicate data over internet (edwards & hopkins, 2018). iot australasian journal of information systems de vass, shee & miah 2018, vol 22, research article iot and supply chain integration and performance 2 platform thus facilitates “things” to be identified, located, sensed and controlled via the global platform (borgia, 2014). it is viewed as progression of information and communication technology (ict) (e.g. computers, erp, bar code, email, fax, phone, and wms) applications that are helpful to capture and share data in a network of organisations on real-time basis (borgia, 2014). this “digitally upgrading” of conventional objects via internet connectivity generates added capabilities to its functionality (mattern & floerkemeier, 2010). iot capabilities may differ to previous ict capabilities due to their ubiquity, intelligence and autonomy (constantinides, kahlert, & de vries, 2017). aside, conventional ict technologies help monitor supply chain functions such as purchasing, transportation, storage, distribution, sales and returns (vanpoucke, vereecke, & muylle, 2017). with many other smart devices recently joining the list under the newly coined iot umbrella of technologies, the potential to address the information capture and exchange in real-time has multiplied (atzori et al., 2010; borgia, 2014). although the advances in iot applications in the form of sensor is expected to revolutionise retail sector (kahlert, constantinides, & de vries, 2017), its acceptance and potential to integrate supply chain processes is largely under developed both for theoretical and practical implications. although there have been a number of studies that have investigated the ict-enabled supply chain processes integration in improving the performance (li, yang, sun, & sohal, 2009; qrunfleh & tarafdar, 2014; rai, patnayakuni, & seth, 2006; vanpoucke et al., 2017), no study is documented to empirically assess the effect of emerging iot adoption on supply chain integration. supply chain integration includes both interand intra-organisational integration across the entire supply chain through a collaborative mechanism (alfalla-luque, medinalopez, & dey, 2013), that can generate greater customer value by offering superior services (christopher & towill, 2001). in a scenario of network-based competition, the strength of a supply chain is determined by how effective the flow of goods and services is, information exchange, and funds flows (rai et al., 2006). iot is perceived to strengthen the supply chain integration by connecting the objects through internet (ping, liu, zhou, & wang, 2011; tu, 2018). adoption and use of iot help organisation building its capability. from organisational capability theory perspective, iot is likely to enhance the capability to integrate the suppliers, customers and intra-organisational logistics processes. iot bridges the gap between physical and digital world by synchronising the information flow with the physical flow for greater supply chain integration (ping et al., 2011). iot capability is defined as additional capabilities gained by supply chains with everyday objects being embedded with technology that provides identifying, sensing, networking and processing capabilities to communicate with other devices and services over the internet (whitmore, agarwal, & da xu, 2014). given that retail supply chains are primarily demand driven, intense digital connectivity and coordination within the supply chain via adoption of new technology can be considered as an intervention in service improvement (fleisch & tellkamp, 2005). the emerging iot paradigm, therefore, may play a significant role in the retail industry to manage supply networks in response to customer demands (yu, subramanian, ning, & edwards, 2015). moreover, the supply chain literature on iot application is broadly rhetoric, technology and architecture focused and quite nascent (mishra et al., 2016). due to enthusiasm on technological adoption in data transparency and visibility to achieve supply chain process integration, the research australasian journal of information systems de vass, shee & miah 2018, vol 22, research article iot and supply chain integration and performance 3 on iot within this context is timely (ben-daya, hassini, & bahroun, 2017; dubey, gunasekaran, childe, papadopoulos, & wamba, 2017; mishra et al., 2016; tu, 2018). the question is what extent the emerging list of technologies classified as 'iot' can facilitate external (i.e. suppliers and customers) and internal (cross-functional operations) process integration to deliver better supply chain performance and firm performance. this study, therefore, aims to empirically investigate the perceived effect of iot adoption on supply chain process integration that in turn enhances both supply chain, and firm performance. this paper is organised as follows. first, the literature on organisational capability theory, supply chain integration, and iot is reviewed to propose a conceptual framework for iotenabled supply chain process integration and performance. the methodology of data collection and analysis technique using sem modelling is then outlined. the discussion of results, theoretical and managerial implications of the findings are then discussed. finally, we summarise the work and outline some limitations within the study context. 2 background literature review 2.1 organisational capability theory iot adoption can be viewed as an additional capability that may add value to the current configuration of ict capability within any organisation. it is important for studies to utilise and treat iot capability as a progression of ict capability that may facilitate intraand interorganisational communication and information flows in more integrated way (borgia, 2014). the intraand inter-organisational information sharing (huh, yook, & kim, 2008), information communication (huh et al., 2008; kusunoki, nonaka, & nagata, 1998), and inter-firm relationships (lorenzoni & lipparini, 1999) together represent process integration capabilities. we, therefore, intend to develop an insight from the literature that organisations can build additional capabilities by adopting iot into the mainstream business running on the legacy of ict backbone. the interand intra-organisational communication and information exchange are perceived to be facilitated by iot capability. organisational capability theory therefore can be argued to underpin this research. organisational capability perspective, related to resource-based view theory, suggests that a firm must develop its own resources and capabilities for performance improvement (huo, 2012; rai et al., 2006). integration per se is a higher order process capability that can directly influence firm performance (huo, 2012; verona, 1999). further, organisational capability theory suggests that internal integration can directly affect external integration where internal process integration is the base for the development of the firm’s external process integration (huo, 2012; zhao, huo, selen, & yeung, 2011). verona (1999) suggest that internal capabilities include internal communication, process integration and job training, while external capabilities represent external communication and networks of partners. bharadwaj (2000) further argues that ict implementation itself cannot have a direct effect on performance; rather it needs to be blended with the other organisational resources (e.g. human and financial resources) for performance improvement. hence, iot can be thought of improving the integration capability of an organisation perceived the way ict does. 2.2 supply chain integration in the current dynamic and competitive business environment, where it is the supply chains that compete rather than individual organisations (christopher, 2016; christopher & towill, australasian journal of information systems de vass, shee & miah 2018, vol 22, research article iot and supply chain integration and performance 4 2001), the increased supply chain performance enhances sustainable competitive advantage (seuring & müller, 2008; spekman, kamauff jr, & myhr, 1998). in conceptualising supply chain management (christopher, crum, & holweg, 2011; ellram & cooper, 1993; ho, au, & newton, 2002), the supply chain integration represents a mechanism for improving supply chain performance (alfalla-luque et al., 2013; ataseven & nair, 2017). supply chain integration can be defined as collaborative interand intra-organisational management on the strategic, tactical and operational business processes to achieve effective and efficient flows of products, information and funds to provide the maximum value to the end customer at the lowest cost and the greatest speed (alfalla-luque et al., 2013; huo, 2012; yu, 2015; zhao et al., 2011). being a leading research theme that combines information systems knowledge into supply chain management literature, integration has been viewed as the digital connection of business processes within the organisation, and between organisations involving upstream suppliers and downstream customers (ataseven & nair, 2017). in addition, many scholars consider supply chain integration having three dimensions: internal process integration, upstream supplier integration and downstream customer integration (alfalla-luque et al., 2013; ataseven & nair, 2017; näslund & hulthen, 2012). internal integration refers to the breakdown of cross-functional barriers within an organisation via synchronised processes by facilitating real-time information sharing across business functions, strategic collaboration and coordination to achieve superior performance (yu, 2015; zhao et al., 2011). supplier and customer integration refers to strategic information sharing, mutual planning and collaboration between the focal organisation and its upstream suppliers and downstream customers in managing synchronised processes (huo, 2012; yu, 2015). supplier integration involves collaboration between a focal organisation and its suppliers in managing cross-firm business processes; customer integration enables a deeper understanding of market expectations and opportunities for a more accurate and rapid response to customer requirements (ataseven & nair, 2017; huo, 2012). conceptually, the goal of integration is to achieve cost efficiency and delivery effectiveness across the entire supply chain, while creating value for the customer (näslund & hulthen, 2012). frohlich and westbrook (2001) suggest the ‘arcs of integration’ theory where organisations with greater supply chain integration will benefit from superior performance outcomes. abundance of studies in literature have shown the positive effect of supply chain integration on its performance (ataseven & nair, 2017; flynn, huo, & zhao, 2010; huo, 2012). 2.3 internet of things (iot) in process integration the advancement of internet-enabled network of infrastructure from rfid readers to 4g highspeed wireless communication acts as catalyst behind the growth of these disruptive technologies designed with low-power, low-transmission, machine-to-machine level communication to send and receive data (souders, 2015). the data however is not disruptive by itself. ample of logistics related data was then gathered using even existing devices, but they were soiled between functional areas and was seldom available on a real-time basis. instead, capturing and analysing more relevant data become essential tasks upon which disruptive technologies are built (wasserman & mahmoodi, 2017). iot therefore promises substantially for this purpose. atzori et al. (2010) explain how the iot is converged with three key visions: things-oriented, internet-oriented and semantic-oriented. thus, iot architecture encompasses the objects that captures data, communicates with the real world and actuates; the internet global platform australasian journal of information systems de vass, shee & miah 2018, vol 22, research article iot and supply chain integration and performance 5 including cloud that facilitates transmission; and hosting and processing of data as well as its information synthesis and processing ability. even though current iot applications are reported to be in its infancy compared to its broader potential, its intelligence, autonomous and pervasive applications have made it popular through cost-effective miniaturisation (uckelmann, harrison, & michahelles, 2011). as iot technologies became more effective and affordable in recent years, its adoption and deployment in supply chain operations progressively grows despite challenges (verdouw, wolfert, beulens, & rialland, 2016). ict, as digital enabler for supply chain integration, promotes effective information flow (rai et al., 2006; yu, 2015). the additional capabilities generated by iot technologies are argued to potentially facilitate the information capture and sharing among the partners (ben-daya et al., 2017; tu, 2018). thus, iot, has the capability to sense supply chain processes, improving visibility, accuracy, traceability, interoperability and collaborative decisions along the chains (reaidy, gunasekaran, & spalanzani, 2015). authors acknowledge supply chains in general (borgia, 2014) and retail supply chains in particular (lee & lee, 2015; yu et al., 2015) as a key industry application domain of iot. ben-daya et al. (2017) argues that iot is one of the key founding technologies of industry 4.0. zhou, liu, and zhou (2015) argue for integration of iot into logistic information helping the partners to capture data and use them effectively in decision making. current supply chain applications of iot are reported to take various forms such as rfid which requires a more unified, standardised and investment-oriented approach to exploit widespread smart devices, gps and smartphones that come with various built in iot functionalities (borgia, 2014; ng, scharf, pogrebna, & maull, 2015; perera, member, jayawardena, & chen, 2015; yu et al., 2015). therefore, iot should not be viewed as a technology limited to large, resourceful operations, but as a generally available capability. however, iot deployment is reported currently as fragmented and lacks interoperability to realise its full potential due to issues with standardisation, architecture, security and privacy (borgia, 2014). therefore, further investigation on the perceived impact of iot on supply chain integration process is needed. literature on iot is emerging and quite rhetoric on its adoption (mishra et al., 2016). by analysing over 300 iot publications, xu, he, and li (2014) find mounting scholarly interest on iot. the focus so far is mainly on conceptual definitions, model building, key technology and features (liu & gao, 2014). whitmore et al. (2014) reviewed 127 iot focussed publications mostly confined to a technology perspective and not well represented in the management and operations literature. review of 1,556 articles published on iot from 2000 to 2015, mishra et al. (2016) find the field dominated by conceptualisations and few case studies on applications of iot. the authors reveal that greater part of the literature focuses on technology perspective and hardly any study looks at the relationship between iot adoption for improved supply chain and organisational performance. moreover, supply chain integration being an underlying concept (reaidy et al., 2015; tu, 2018), the vital link between iot adoption and supply chain performance is largely unexplored within the literature. ping et al. (2011)’s concept paper on agile supply chain management is one of the few which seeks to explain how iot can synchronise and strengthen the goods and information flow to improve supply chain integration. similarly, reaidy et al. (2015) propose an iot infrastructure for collaborative warehousing, recognising that the new environment requires supply chain with greater integration. further, yan et al. (2014), through a single case study under australasian journal of information systems de vass, shee & miah 2018, vol 22, research article iot and supply chain integration and performance 6 laboratory conditions explore the relationship between iot and supply chain integration. these authors suggest that iot is an effective approach for supply chain integration to achieve overall supply chain performance. nonetheless, the authors offer no empirical evidence of iot that can improve supply chain process integration from the perspective of organisational capability theory. overall, the academic literature does not provide enough evidence on the relationship between iot capability and supply chain process integration from the supply chain perspective or specifically, the retail supply chain context. despite the emergence of laboratory-based studies, quantitative studies on the capabilities of iot are not yet available (verdouw et al., 2016). therefore, this study addresses the knowledge gap by examining empirically whether iot can strengthen supply chain integration capability to influence performance in the australian retail industry. mishra et al. (2016) posit that socioorganisational implications of iot adoption will benefit both academics and practitioners than focusing only on technology per se. 2.4 conceptual research framework and hypotheses development the conceptual framework in this study is built upon the process integration model proposed by huo (2012), who empirically investigates operational and financial performance significantly influenced by integration mechanism from a manufacturing industry perspective. in his model, huo (2012) includes three sub-dimensions of integration: supplier, customer and intra-organisational integration. literature evidences a wide range of studies on integration primarily focused on the relationship between supply chain integration and performance (fabbe-costes & jahre, 2008; huo, 2012; kim, 2009). rai et al. (2006), while acknowledging the positive effect of digitally enabled supply chain integration on firm performance, draw upon ict capability as a separate construct. further, yu (2015), building on huo (2012)’s model, considers ict as a separate construct having a significant effect on the three dimensions of supply chain integration. however, none of the existing studies so far used iot as an enabler of suppliers, customers and internal integration in the context we have drawn in this study. this study draws on ict-enabled organisational capabilities perspective, that is the theoretical notion of how ict shapes higher-order process capabilities for performance gains (rai et al., 2006). this concept is extended to investigate the hierarchy of iot-related capabilities and their effect on organisational performance. while the studies on emerging iot technology remain rhetoric in the literature, this study fills the gap by examining the perceived effect of iot capability on the three dimensions of supply chain integration, and subsequent effect on supply chain and organisational performance. this addresses the gap in the targeted literature that we have discussed above. 2.4.1 hypotheses development iot is an emerging global internet-based information service architecture facilitating the exchange of goods in global supply chain networks by generating value for all partners (lee & lee, 2015; liu & sun, 2011; weber, 2009). the pervasive presence of rfid tags, sensors, actuators, smart devices and machines and mobile phones are reported to have the capability of conducting real-time monitoring of almost every link from purchasing, transportation, storage, distribution, sales and returns in contemporary supply chains (atzori et al., 2010; haddud, desouza, khare, & lee, 2017). this facilitates a safe and trustworthy platform to exchange information related to goods and services in a global supply chain (mishra et al., 2016). hence, iot has the potential beyond traditional ict to address the information gap in australasian journal of information systems de vass, shee & miah 2018, vol 22, research article iot and supply chain integration and performance 7 existing supply chains by capturing additional data flowing among supply chain entities, processes, equipment and people, and transfer, process and action in real-time (ben-daya et al., 2017; borgia, 2014). the extent literature claims that iot has the capability to assist with real-time data capture thereby facilitating interand intra-firm communication and integration (ping et al., 2011; yan et al., 2014). zara, for instance, achieves planning flexibility, effective replenishment, shorter lead times and product variations with the assistance of such handheld devices (qrunfleh & tarafdar, 2014). iot further helps in fleet tracking; shipment condition monitoring; storage conditions of cold chain/perishable products and; product tracking, thereby integrating suppliers into buyers’ communication process. further, product delivery process, shop guidance, customer shopping behaviour, and product tracking where iot can help customers by integrating them in to the process. therefore, the hypothesis can be formulated as, hypothesis h1: iot capability has a positive effect on supplier integration. h2: iot capability has a positive effect on internal integration. h3: iot capability has a positive effect on customer integration. the literature posits that internal integration improves external integration in terms of supplier and customer integration (huo, 2012). organisational capability theory suggests that internal integrative capabilities can directly affect external integration capabilities (huo, 2012; zhao et al., 2011). information exchange with suppliers using iot technology can improve the understanding of joint planning, forecasting and partnerships. information exchange via iot can help organisations to understand the customer requirements and to collaborate with customers for better demand planning and customer service. the use of internal iot can be a base to develop iot capability to the entire supply chain. therefore, it is hypothesised that, hypothesis h4: iot-enabled internal integration has a positive effect on supplier integration. h5: iot-enabled internal integration has a positive effect on customer integration. literature suggests that partner integration can minimise cost via waste reduction and asset utilisation (näslund & hulthen, 2012) and also help supply chains to be more flexible, adaptive, reactive and responsive to cope with risks and market uncertainty (reaidy et al., 2015). therefore, all three dimensions of supply chain integration improves supply chain performance (huo, 2012; vanpoucke et al., 2017). digitally enabled supply chain integration has a positive effect on performance (rai et al., 2006; yu, 2015). iot can improve the way people and systems collaborate and coordinate supply chain processes, analyse captured data. this helps to identify optimisation prospects and effective procedures for performance gains throughout the entire supply chain via operational efficiency, quality, flexibility, delivery reliability and customer experience. the three dimensions of supply chain integration via iot, therefore, are likely to influence supply chain performance. hence, the hypotheses can be formulated as, hypothesis h6: iot-enabled supplier integration has positive influence on supply chain performance. h7: iot-enabled internal integration has positive influence on supply chain performance. h8: iot-enabled customer integration has positive influence on supply chain performance. australasian journal of information systems de vass, shee & miah 2018, vol 22, research article iot and supply chain integration and performance 8 the literature further suggests that ict-enabled supply chain performance impacts organisational performance positively (qrunfleh & tarafdar, 2014; vanpoucke et al., 2017). the real time information provided by iot helps track supply chain activities, from product design to the end users, providing accurate and timely information to help organisations respond to the market changes (mishra et al., 2016). while supply chain strategy yields performance by focussing on cost, quality, delivery and flexibility improvement (gunasekaran, patel, & mcgaughey, 2004), it attempts to align with the firm objectives of improving triple bottom line performance to generate environmental, social, and economic benefits (carter & rogers, 2008; elkington, 1997). likewise, iot-enabled supply chain performance influences firms’ economic performance while caring for environment and social dimensions. hence, the following hypothesis is proposed. hypothesis h9: iot-enabled supply chain performance is positively related to organisations’ sustainable performance. 2.4.2 control variables organisational size, characterised by employee number, is often found to determine firms actions and performance (rai et al., 2006). large organisations offer strength to acquire capabilities due to its resource availability, the economics of scale and the capacity to influence supply chain partners (yu, 2015). this study includes organisations from multiple retail forms. the iot application could vary on each retail form. the e-tail model relies more on information sharing as compared to physical goods flows in traditional bricks and mortar model. therefore, organisational size and retail form were tested as control variables as they might have positive effect on retail performance. 3 methodology 3.1 sample characteristics the respondent firms represented 49.8 percent from large size (employees>200), 49.8 percent from medium size (between 20 to 200 employees) and only 4.4 percent were small size (employees<20) (abs retail industry analysis, 2014). about 55.8 percent of the sample reported that their supply chains extended worldwide, while regional and local supply chains represented 30 and 14 percent respectively. about 56.8 percent of the organisations were traditional bricks and mortar store-based walk-in retailers, 38.8 percent were multimodal (or omnichannel) and only 4.4 percent were e-tailing. table 1 shows sample distribution of retail sector (abs retail industry analysis, 2014). overwhelming 96 percent of the respondents had over 2 years of managerial experience within their respective organisations. the respondents had titles such as ceo/chairmen/md/director/general manager (15 percent), operations/supply chain/logistics manager (24.2 percent), middle management (28.6 percent), it manager (26.4 percent) and others (5.8 percent). about 62 percent believed that they were engaged in high-level strategic decision making in supply chain operations with 14 percent perceiving them as the key decision maker and 48 percent involved in a considerable extent. 36 percent stated that their involvement is at a moderate extent. the majority (90 percent) perceived ‘improve overall business performance’ as a key motive for iot adoption in their firm's supply chain operations, while 45 percent opted for cost reduction via operational efficiency and 35 percent for improve supply chain performance. australasian journal of information systems de vass, shee & miah 2018, vol 22, research article iot and supply chain integration and performance 9 retail sector (n=227) frequency percent restaurant, café, takeaway 26 11.5 supermarkets, grocery 37 16.3 household goods (e.g. hardware, furniture) 22 9.7 clothing, footwear and personal accessories 33 14.5 electrical, electronic, computer 22 9.7 pharmaceutical, cosmetic, toiletry 20 8.8 motor vehicles & parts 11 4.8 fuel and convenience stores 9 4.0 department stores 25 11.0 other 22 9.7 table 1: retail sectors represented in the sample figure 1: current/planned adoption of iot in respondents’ supply chain operation the figure 1 displays how the respondents have implemented or planned to implement each form of iot in their supply chain operations in late 2016. even though rfid is widely reported 0 50 100 150 200 250 r f id r e la te d t a g g in g a n d t ra ck in g in te rn e t b a se d b a rc o d e t e ch n o lo g y g p s b a se d l o ca ti o n a w a re n e ss in te rn e t b a se d s e n so rs a n d s ca n n e rs h a n d h e ld /p a lm -h e ld t a b le ts /s m a rt -d e v ic e s s m a rt p h o n e s a n d m o b il e a p p s in te rn e t b a se d a p p li a n ce s/ e q u ip m e n t in te rn e t b a se d w e a ra b le d e v ic e s in te rn e t b a se d s e cu ri ty a n d s u rv e il la n ce in te rn e t b a se d t ra n sp o rt a ti o n d e v ic e s in te rn e t b a se d l o g is ti c e q u ip m e n t in te rn e t b a se d i m m o b il e /f ix e d d e v ic e s r e a lti m e s tr e a m in g a n a ly ti cs v ia i o t d a ta d a ta a n a ly ti cs u si n g i o t d a ta a u to n o m o u s re p o rt in g /a le rt a u to n o m o u s d e ci si o n -m a k in g /a ct io n /r e a ct io n im a g e r e co g n it io n v ia i o t o th e rs no plans to implement within next 3 years planning to implement within next 3 years currenty implementing less than 3 years ago over 3 years ago australasian journal of information systems de vass, shee & miah 2018, vol 22, research article iot and supply chain integration and performance 10 as the primary form of iot, the use of rfid related tagging and tracking is relatively low in retail supply chains. on the other hand, internet-based barcode technology, gps-based location awareness, internet-based sensors and scanners, handheld/palm-held tablets/smart devices, smartphones and mobile apps and internet-based security and surveillance demonstrate a high concentration of deployment. the findings confirm that, although iot originates from the concept of rfid, it is now a central element on its own, with far-reaching capabilities and eclectic diffusion. every respondent firm had at least one form of iot currently deployed within their supply chain operations. while 89 percent of respondents had four or more forms, 33 percent respondents had ten or more iot forms deployed in their respective supply chain operations. these statistics confirm the diffusion of iot in many primary forms in the retail supply chains in australia. 3.2 sample and data collection the study used self-reported survey instrument for data collection from australian retailers across industries. retail firms from across industries in six states of australia were randomly chosen from the yellow pages of australian businesses directory and alphabetically listed to avoid duplication of cases (yellowpages, 2016; yu, 2015; zhao et al., 2011). a single key informant from each retailer was identified by inviting them through linkedin professional networking and facebook. prior research has successfully applied the similar online method to collect research data using questionnaire survey (baltar & brunet icart, 2011, 2012). the professional and academic connections were found to be successful as reported in prior similar studies (yu, 2015; yu, jacobs, salisbury, & enns, 2013; zhao et al., 2011). the use of social networking sites of informants was effective to identify this "hard-to-reach"/"hard-to-involve" population (baltar & brunet icart, 2011, 2012). then it was snowballed from initial connections to the second tier, third tier connections to identify possible informants belonged to the generated representative sample. when pre-contacted, potential informants from 546 retail firms agreed to participate in the survey. these potential respondents mainly represent either the supply chain and logistics area or it department within that retailing business enabling them to respond to the questionnaire on iot. the link to the online qualtrics survey was sent to the mailing list in a personally addressed email (dillman, 2011). the questionnaire accompanied a cover letter explaining the objective and the potential contributions of the research. following online reminders and follow-up phone calls, a total of 231 responses were collected. four responses were discarded for their unengaged responses, reducing the sample to 227 usable cases that represented a 41 percent response rate. 3.3 measurement items and survey design the reliable and valid items were mostly adapted from previous literature through an extensive review. the minor changes were made to the questionnaire items to ensure they are aligned with the research questions, objectives and the method. since iot capability has not been measured in prior academic research, twelve items were newly developed based on their functionality published in academic literature (atzori et al., 2010; borgia, 2014; perera et al., 2015) and consultancy manuscripts (dhl, 2015). the items were adapted keeping in view the core capabilities of iot such as identify, monitor, measure, control, automate, analyse, information sharing and collaboration (dhl, 2015; lee & lee, 2015). the integration measures australasian journal of information systems de vass, shee & miah 2018, vol 22, research article iot and supply chain integration and performance 11 were primarily adapted from huo (2012) and rai et al. (2006). they comprise of ten supplier integration items, ten internal integration items and nine customer integration items. all nine supply chain performance measurement items were adapted from a study by schoenherr and swink (2012). for firm performance, out of 11 items, six items were adapted from rai et al. (2006), one from yu et al. (2013) and additional social and environmental measurement items were adapted from paulraj (2011), dolbier, webster, mccalister, mallon, and steinhardt (2005) and zhu, sarkis, and lai (2012). following prior similar studies (huo, 2012; yu, 2015), a sevenpoint likert scale was chosen to measure all items with the responses ranging from 1 (strongly disagree) to 7 (strongly agree). the measurement items used in this study can be found in table 2. in accordance with previous supply chain and operations management studies, the respondents were instructed to evaluate improvement relative to the performance of their main competitors over the last 3 years (yu et al., 2013). it was clearly explained that the supply chain from the retail firm perspective not just includes their suppliers and customers but also the logistics intermediaries such as 3pl operators managing their transport and storage functions. the survey followed dillman (2011)’s design method guide on internet surveys. a pre-pilot panel of five academics and five industry practitioners were asked to critique the questionnaire for content validity, appropriateness and clarity of questionnaire (dillman, 2011). they identified some duplication and ambiguity in questionnaire items. after incorporating their feedback, the final version was entered in qualtrics software and a link was created. 4 data analysis 4.1 non-response bias and common method bias non-response bias was assessed by comparing demographic variables of early and late respondents (at 60% to 40% split) via an independent sample t-test (armstrong & overton, 1977). the t-test results indicated no significant differences between early and late responses at p < 0.05 indicating no non-response issues (armstrong & overton, 1977; huo, 2012). harman’s single-factor test was performed applying both exploratory factor analysis (efa) and confirmatory factor analysis (cfa) (podsakoff, mackenzie, lee, & podsakoff, 2003). first, the all-item efa revealed seven distinct factors with eigenvalue above 1.0, explaining 65.92 percent of the total variance. the first factor explained 40.48 percent of the total variance, which is not the majority of the total variance. second, the cfa was applied to harman’s single-factor model (flynn et al., 2010; podsakoff et al., 2003). the cfa demonstrated a poor model fit at χ²/df = 2904/945 = 3.073, gfi = 0.529, tli = 0.672, cfi=0.623, rmsea = 0.096 and srmr = 0.0846, with indices way inferior than the measurement model, confirming the absence of common method bias in the data set. 4.2 measurement model structural equation modelling (sem) was applied to estimate the conceptual model based on 227 cases. the two-step approach for structural equation modelling allows testing the measurement model first followed by the structural path model as suggested by anderson and gerbing (1988). before cfa was undertaken, efa was run to find the underlying factors generated from all the items (refer appendix for efa). cfa then confirmed the acceptability of the measurement model and the unidimensionality of the theoretical constructs as summarised in table 2 (bollen, 1989; hair, black, babin, & anderson, 2014; hu & bentler, australasian journal of information systems de vass, shee & miah 2018, vol 22, research article iot and supply chain integration and performance 12 1999). however, the chi-square value is found to be a poor model fit of data because of the associated p-value more than 0.05. hair et al. (2014, p. 584), however, state that when the model contains over 30 measurement items and number of observed variables are fewer than 250, pvalue less than 0.05 (p <.05) is acceptable. ideally p-value should be >0.05 for the model to be fit and accepted. therefore, bollen-stine p-value was estimated through bootstrapping procedure that provided p>0.194 (i.e. more than p>0.05) (bollen & stine, 1992). many previous studies have not reported p-value but appear to apply bootstrapping procedure in testing smaller samples to fit the model (qrunfleh & tarafdar, 2014; rai et al., 2006; ralston, blackhurst, cantor, & crum, 2015). therefore, we argue the model fit is acceptable. the model fit indices are: χ² (930) =1366.876, bollen-stine p=0.194, χ²/df=1.255, srmr=0.0451, rmsea=0.034, cfi=0.962 and tli=0.960. the details of the factor loadings and t-values of cfa are presented in table 2. item no measurement item factor loading t-value construct 1: iot capability iot has the capability in our supply chain: 1.1 to provide individual item level identification. dropped in cfa 1.2 to provide unit level (e.g. container/box/pallet) identification. 0.759 12.109* 1.3 to monitor, track and trace supply chain entities and people through autocaptured data. 0.749 11.919* 1.4 to measure supply chain activities, processes and its environmental conditions. 0.781 a 1.5 to help control supply chain processes remotely. 0.718 11.323* 1.6 to make autonomous supply chain decisions. dropped in cfa 1.7 to provide real-time information to optimise supply chain activities 0.798 12.88* 1.8 to provide real-time intelligence of supply chain operations. dropped in cfa 1.9 to provide large volumes and variety of data to apply data analytics for tactical and strategic decision making. 0.767 12.26* 1.10 to strengthen inter and intra organizational information sharing within the supply chain. 0.774 12.398* 1.11 to facilitate inter and intra organizational decision making within the supply chain. dropped in cfa 1.12 to strengthen communication and coordination between operators. dropped in cfa construct 2: supplier integration we have been able to improve the business processes with our suppliers to: 2.1 improve information exchange with our suppliers. 0.803 13.469* 2.2 establish a quick ordering of inventory from our suppliers. 0.798 a 2.3 accurately plan and adopt the procurement process in collaboration with our suppliers. dropped in cfa 2.4 stabilize procurement with our suppliers. 0.758 12.489* 2.5 share real-time demand forecasts with our suppliers. 0.721 11.71* 2.6 improve strategic partnerships with our suppliers. 0.758 12.478* 2.7 help suppliers improve their processes to better meet our needs. 0.738 12.057* 2.8 improve the account payable processes for suppliers. dropped in cfa 2.9 improve the transport/logistics processes of logistics partners to deliver orders just in time. 0.742 12.149* australasian journal of information systems de vass, shee & miah 2018, vol 22, research article iot and supply chain integration and performance 13 2.10 improve our receiving processes for delivered goods. 0.762 12.581* construct 3: internal integration we have been able to improve our internal logistics processes to: 3.1 improve the integration of data among internal functions. dropped in cfa 3.2 improve real-time communication and linkage among all internal functions. 0.806 14.38* 3.3 accurately plan and adopt internal processes in collaboration with cross functional teams. 0.819 14.718* 3.4 make and adopt demand forecasts in collaboration with cross functional teams. 0.827 a 3.5 improve inventory management in collaboration with cross functional teams. dropped in cfa 3.6 improve real-time searching of the inventory levels. 0.808 14.433* 3.7 improve real-time searching of logistics-related operating data. 0.717 12.17* 3.8 employ cross functional teams in process improvement. 0.706 11.918* 3.9 improve replenishment of shop floor shelves. 0.766 13.343* 3.10 reduce stock outs in the shop floor shelves. 0.789 13.912* construct 4: customer integration we have been able to improve the business processes with our customers to: 4.1 improve the strength of linkages with our customers. 0.796 12.812* 4.2 improve regular contacts with our customers. 0.798 12.856* 4.3 improve communication with our customers on products and promotions. dropped in cfa 4.4 make and adopt demand forecasts with a real-time understanding of market trends. 0.778 a 4.5 improve the customer shopping experience/time/ordering/customising processes. 0.771 12.325* 4.6 accurately plan and adopt the checkout/dispatch/delivery processes through a better understanding of market trends. 0.759 12.082* 4.7 improve the check-out/dispatch/delivery process of goods. 0.739 11.693* 4.8 improve and simplify the payment receivable process from our customers. 0.734 11.606* 4.9 improve customer feedback process. dropped in cfa construct 5: supply chain performance we have been able to develop our supply chain processes to: 5.1 improve product quality. dropped in efa 5.2 improve supply chain delivery reliability. dropped in cfa 5.3 improve fill rates. dropped in cfa 5.4 improve perfect order fulfilment (deliveries with no errors). 0.776 a 5.5 improve supply chain flexibility (react to product changes, volume, mix). 0.643 9.601* 5.6 reduce the cash-to-cash cycle time. 0.646 9.655* 5.7 reduce the total supply chain management cost. 0.74 11.24* 5.8 reduce the cost of goods sold. 0.764 11.665* 5.9 improve value-added productivity (sales per employee). 0.684 10.291* construct 6: organisational performance we been able to develop our organisational operations to: 6.1 improve the product delivery cycle time. 0.705 10.744* 6.2 improve productivity (e.g. assets, operating costs, labour costs). 0.719 10.981* 6.3 improve sales of existing products. 0.808 12.526* 6.4 find new revenue streams. dropped in cfa 6.5 build strong and continuous bonds with customers. 0.754 a australasian journal of information systems de vass, shee & miah 2018, vol 22, research article iot and supply chain integration and performance 14 6.6 gain precise knowledge of customer buying patterns. 0.753 11.574* 6.7 improve customer satisfaction. 0.75 11.52* 6.8 improve employee satisfaction. 0.693 10.544* 6.9 improve employee health and safety. 0.703 10.723* 6.10 reduce energy use. 0.719 10.984* 6.11 improve return/re-use/recycle. dropped in cfa *p<0.001 *a = the loadings were set to unity to scale the latent variable table 2: cfa results of the measurement model4.4. reliability and validity content validity of the questionnaire was confirmed by industry practitioners and academics prior to data collection. the unidimensionality of the theoretical constructs was confirmed by cfa test. the cfa results (table 2) conclude that the measurement model is acceptable and its unidimensionality is confirmed (byrne, 2013; hair et al., 2014; hu & bentler, 1999; kline, 2015). reliability and validity of the constructs were tested post data collection by performing a series of analyses. the cronbach’s alpha and composite reliability of all constructs exceed the specified value 0.7 confirming the construct reliability (hair et al., 2014; huo, 2012). cronbach’s alpha values for all constructs were above 0.8. the composite reliability values were strong at 0.75 and above. the highest correlation between constructs were 0.69 indicating no multicollinearity issues among the constructs (hair et al., 2014). the convergent validity of each measurement scale was assessed by cfa using the maximum likelihood approach (o'leary-kelly & vokurka, 1998). all indicators under respective constructs have statistically significant (p < 0.001) factor loadings greater than 0.50 (table 2), which confirms convergent validity of the theoretical constructs (anderson & gerbing, 1988; zhao et al., 2011). therefore, measurement items are significantly grouped to their underlying theoretical constructs. further, the average variance extracted (ave) of each construct exceeds the minimum value of 0.50 (fornell & larcker, 1981) that suggests the constructs have sufficient convergent validity. discriminant validity was evaluated by comparing the correlation between the construct and the square root of ave. referring table 3, the square root of ave of all the constructs shown along the diagonal is greater than the inter-construct correlation between any pair of them which shows the discriminant validity (fornell & larcker, 1981). table 3 also reports the means, standard deviations and inter-construct correlation coefficient. australasian journal of information systems de vass, shee & miah 2018, vol 22, research article iot and supply chain integration and performance 15 construct no. of items iot si ii ci scp fp mean sd cr. alpha cr ave iot capability 7 0.76 4.82 1.27 0.91 0.86 0.58 supplier integration 8 0.63 0.76 4.72 1.32 0.92 0.86 0.58 internal integration 8 0.64 0.70 0.78 4.50 1.37 0.93 0.87 0.61 customer integration 7 0.59 0.64 0.69 0.77 4.72 1.33 0.91 0.85 0.59 supply chain performance 8 0.66 0.63 0.63 0.60 0.71 4.65 1.45 0.86 0.75 0.51 organisational performance 9 0.65 0.60 0.58 0.62 0.62 0.73 4.44 1.41 0.91 0.84 0.54 the diagonal values represent the square root of ave extracted for each construct cr= composite reliability; cr. alpha=cronbach’s alpha; sd=standard deviation all correlation coefficients below the diagonal are significant at p < .001 table 3: descriptive statistics, reliability (cronbach alpha and composite) and discriminant validity tests 4.3 structural path model and hypotheses testing figure 2 and table 4 present the results of structural path model using amos 23 (hair et al., 2014; hu & bentler, 1999). bootstrapping was used to generate the p-statistic with 227 cases (qrunfleh & tarafdar, 2014; ralston et al., 2015). the structural model resulted in overall acceptable fit with χ²(936)=1236.586, bollen-stine p=0.095, χ²/df=1.321, srmr=0.0761, rmsea=0.038, cfi=0.952 and tli=0.949. all the goodness-of-fit indices are above or close to the recommended cut-off points, suggesting that the specified model adequately captures the hypothesised relationship among all constructs. the findings therefore validate the proposed theoretical model. figure 2: a conceptual framework for iot-enabled supply chain integration and performance australasian journal of information systems de vass, shee & miah 2018, vol 22, research article iot and supply chain integration and performance 16 the nine hypotheses are supported as shown in table 4. the standardised beta coefficients of the path along with the t-statistics are also presented. hypotheses path std. ß t-values results h1 iot capability → supplier integration 0.33** 4.299 supported h2 iot capability → internal integration 0.65** 6.285 supported h3 iot capability → customer integration 0.27** 3.450 supported h4 internal integration → supplier integration 0.49** 6.285 supported h5 internal integration → customer integration 0.52** 6.374 supported h6 supplier integration → supply chain performance 0.32** 3.726 supported h7 internal integration → supply chain performance 0.24* 2.349 supported h8 customer integration → supply chain performance 0.27** 3.188 supported h9 supply chain performance → organisational performance 0.68** 8.520 supported **<0.001, * p < 0.05 table 4: results of the structural path analysis the results confirm the hypothesised relationships among the constructs in the theoretical framework, suggesting that iot-enabled supply chain integration has a positive and significant effect on supply chain performance, that in turn impacts positively on organisational performance. iot capability has a positive and significant effect on three dimensions of supply chain integration processes confirming the acceptance of h1, h2, and h3. also, iot-enabled internal integration is positively and significantly related to iot-enabled supplier and customer integration process confirming the acceptance of h4 and h5. the findings show that iot-enabled supplier-, internaland customer integration are positively and significantly related to supply chain performance confirming the acceptance of h6, h7, and h8. hypothesis h9 is confirmed too when iot-enabled supply chain performance is found to be positively associated with organisational performance. 4.3.1 testing for control variables the results indicated that organisation size has no significant effect on iot-enabled supply chain performance (ß = 0.167, t = 1.521, p = 0.128) or iot-enabled organisational performance (ß = 0.104, t = 0.968, p = 0.333) of retail organisations. likewise, the retail form also did not exhibit a significant effect on iot-enabled supply chain performance (ß = 0.058, t = 0.881, p = 0.378) or iot-enabled organisational performance (ß = 0.09, t = 1.397, p = 0.162). therefore, the two control variables were not found to be significant. 4.3.2 the post hoc analysis a post hoc analysis was conducted to verify if any model re-specification was desirable. iot is perceived to help supply chain operations (atzori et al., 2010). thus, a direct link between iot capability and supply chain performance was drawn and analysed. the alternative model with a direct path between iot capability and supply chain performance was found significant (ß = 0.379, t = 4.440, p < 0.001). however, the relationships between iot-enabled supplier integration and iot-enabled customer integration with supply chain performance (i.e. h6 and h8) were found to be weaker and iot-enabled internal integration and supply chain performance (h7) australasian journal of information systems de vass, shee & miah 2018, vol 22, research article iot and supply chain integration and performance 17 was found insignificant (ß = 0.139, t = 1.419, p = 0.156). we, therefore, decline to accept this changed path relationship following the theory that iot should have a positive effect on internal, supplier and customer integration to influence supply chain performance (li et al., 2009; rai et al., 2006). so, the competing model was overruled. the post hoc analysis concluded that model re-specification was not feasible. 5 discussion and implications the empirical findings of this study show that iot-enabled integration is perceived to improve the performance of supply chain as well as organisations. iot adoption is found to have significant effect on three dimensions of integration namely suppliers, internal and customers. the iot paradigm that includes physical objects such as sensors, actuators, palm-held devices and smart phone has the capability of external integration (i.e. suppliers and customers) as well as internal integration (i.e. cross-functional operations) enabling information capture, sharing, communication and inter-firm relationships. drawing on the emerging iot-enabled organisational capability perspective, we suggest that retailers those who leverage on iot for supply chain integration capability can yield significant and sustained performance improvement. the iot-powered digital infrastructure enables the retailers to develop a higherorder capability of supply chain integration among partners (e.g. suppliers and customers). this capability enables retailers to capture, share and analyse demand information of customers and their preference of products and services. it helps integrate suppliers for their product information and delivery schedule. while inter-functional integration within the organisations will reap the benefit of iot capability, it brings more benefits to the organisations by integrating their customers and the suppliers, as perceived by the sample respondents. the results reveal that iot capability affects supply chain integration positively which is consistent with many earlier studies on ict-enabled digital integration (li et al., 2009; rai et al., 2006; yu, 2015). co-existence of iot functional capability along with ict capability of organisations is perceived to add integration capability. iot is viewed as added components to the exiting ict technologies to make them proficient in data capture and information sharing. the finding supports the organisational capability theory perspective in a way that an organisation with a high level of internal communication and coordination capabilities is more capable of achieving a high level of external integration capabilities (zhao et al., 2011). while scholars have conceptualised iot adoption as more of rhetoric in supply chain operations in general (atzori et al., 2010; borgia, 2014; zhou, chong, & ngai, 2015) and retail supply chains in specific (lee & lee, 2015; yu et al., 2015), this research reinforces further that iot adoption strategy can improve supply chain process integration, and in turn supply chain performance and organisational performance. while conceptualising the iot-enabled integration capability in supply chain, the study contributes theoretically into supply chain integration literature as well as is literature. while is literature focuses predominantly on emerging technologies such as iot, cloud and automation initiative, supply chain integration literature focus is primarily on their effective applications in logistics processes around suppliers, customers and internal functions. it extends the prior literature abounds with ict-enabled process integration to deliver the supply chain performance benefits (li et al., 2009; rai et al., 2006; vanpoucke et al., 2017; yu, 2015), the literature is however, they are deficient in embracing any emerging technology that will bring in added advantages in the integration mechanism. here we refer to iot as an emerging but ever pervasive technology that uses sensors, devices with short range radio australasian journal of information systems de vass, shee & miah 2018, vol 22, research article iot and supply chain integration and performance 18 frequency, smart mobile phones, actuators etc. with the capability to capture and share data online through internet. earlier studies have provided no empirical evidence of these iots in supply chain integration context. literature have argued iot more of a pervasive technology per se by focusing on vision, features, and likely applications (atzori et al., 2010; borgia, 2014; mishra et al., 2016; whitmore et al., 2014). the literature claim iot as having a capability to connect physical things with digital world, and argue that the ‘things’ can be read, recognised, located, addressed and controlled by using internet (borgia, 2014). our results indicate that iot adoption is positively connected to supply chain process integration of internal functions, supplier processes and customer processes. the integration thus achieved is likely to have positive influences on supply chain performance and organisational performance. this study incorporates the emerging iot adoption as an organisational capability which has not been studied previously in the domain of supply chain integration. from an organisational capability theory perspective, which is related to resource-based view, supply chain integration is a capability of organisations having iot technologies as organisational resources (rai et al., 2006). like ict-enabled integration affecting positively on supply chain performance (li et al., 2009; rai et al., 2006; yu, 2015), adoption and deployment of iot per se is perceived, in this study, to have positive influence on performance. aside, bharadwaj (2000) argues ict investment is to be coupled up with other resources like human and financial resources to gain competitive advantage (armstrong & shimizu, 2007; newbert, 2007). we argue in similar vein for iot investment that will complement the current ict capabilities in significant way. the perceived integration capability can improve the performance of supply chain, as well as firm sustainability. further, the findings reinforce several iot related conceptual discussions (ng et al., 2015; ping et al., 2011) and laboratory-based study outcomes (reaidy et al., 2015; yan et al., 2014) that corroborates its positive impact. the study addressed an important shortfall by conducting an empirical research in the area of the emerging iot applications in supply chain integration. while literature is conceptual and rhetoric about its business applications, this study has taken iot aboard in an empirical framework to see its significant effects on performance improvement. it can be argued that while iot plays a vital role on top of the current ict legacy systems in internal operations, it can be deployed to enhance positive performance outcomes further by connecting digitally the globally spread smart objects and customers through internet. while ict connects end to end computers globally over internet, iot can increase the reach by connecting smart objects to computers and other objects to capture data. practically, the results are beneficial to the managers in a transition to iot adoption. our findings will encourage investments on iot to complement ict capabilities currently in place. iot emerges as a relatively new sphere of study with majority of scholars still focus on development of technology per se, and its service architecture aspect (liu & gao, 2014; whitmore et al., 2014). however, the business application purview of iot from the supply chain perspective is yet to enter the mainstream research (mishra et al., 2016). this is the gap in current research in supply chain domain. while most retailers have some form of ict technologies (e.g. bar code, edi, wms) as their current capability, they need to acquire, integrate, reconfigure, and release resources to upgrade with the emerging iot technologies (e.g. rfid, sensors, palm-held devices, smart phones). retail managers will see these sensors and smart devices helping capture ample data from the things/objects in retail points. aligning iot adoption along with supply chain strategy is likely to improve the supply chain and firm australasian journal of information systems de vass, shee & miah 2018, vol 22, research article iot and supply chain integration and performance 19 performance (qrunfleh & tarafdar, 2014). the findings inform managers about iot capabilities as an extension in the long established digitally-enabled supply chain integration capability. for example, the deployment of palm-held smart electronic devices assists in autocapturing more crucial data around logistics helping mangers to explore analytics of these data. as data analytics gaining popularity for effective decision using quality data, firms increasingly find value in data-driven decision. iot adoption and deployment into logistics functions within the retail organisations will certainly benefit in their day to day operations. the key insight is that managers should develop iotcapability alongside existing ict capability to achieve sustainable competitive advantage. the results suggest that retail managers need to first establish internal integration within the internal functions while they go out for external integration with their customers or suppliers. as iot-enabled supplierand customer integration affects supply chain performance more significantly than iot-enabled internal integration, in contrast to general consensus (ataseven & nair, 2017; yu, 2015). it can be argued that iot-enabled external integration is related at a higher significance for supply chain performance gains beyond firm boundaries due to iot’s pervasiveness and omnipresent ability. retail managers should stretch their focus from retail management perspective to the entire supply chain to include suppliers and customers. further, they need to consider linking existing iot devices with suppliers (e.g. gps) and customers (e.g. smart phones), mustering all digital data into one internet-based platform to share, communicate and process information from each other. this unified supply chain system approach may derive greater benefits for all partners fulfilling the conceptual objective of iot platform to establishing a dynamic worldwide network (borgia, 2014). the findings also contribute to policy making by providing evidence that the investment in iot is a sound public investment. various developed and developing countries have pursued national strategies on iot technology deployment and allocated substantial funding on iot research. the findings may help advocate iot to be lifted to the national strategy for australian businesses, shadowing us, eu, china, korea and japan (borgia, 2014). 6 conclusion and limitations iot has emerged as an innovative technology with capabilities to improve supply chain information flow. however, the effect of iot on supply chain integration and in turn performance is not yet explored empirically. the survey-based cross-sectional study reveals a positive and significant relationship between iot adoption and the effect it can have on supplier, customers and internal supply chain functions of retail firms. co-existence of iot capability in combination with ict capability is perceived to have a significant improvement in supply chain as well as sustainable firm performance. this study contributes to the supply chain integration literature over and above the contribution by generic ict driven digitally enabled supply chain integration significantly associated with supply chain performance and in turn the performance of the retail firm. from organisational capability theory perspective, the adoption of iot helps achieving organisational integration capability. there are some limitations acknowledged in this study. first, the study sampled australian retailers across industries thereby limiting its generalisability with caution for retailers in the pacific region. the study holds good in australian context with relatively robust internet network and more recently with national broadband network (nbn). the future study can undertake a research whether this framework can be validated in other setting within the australasian journal of information systems de vass, shee & miah 2018, vol 22, research article iot and supply chain integration and performance 20 pacific regions where the internet technology is relatively poor. the smaller sample size (n=227) did not allow a sector-specific analysis of iot adoption to reveal its effect on integration mechanism and performance. future research with larger sample can undertake a comparative study to reveal more of each sector. adoption of emerging technology is attributed to institutional pressure from stakeholders such as suppliers, customers and local government (teo & pok, 2003). the future research needs to consider institutional pressure as a moderator to test its influence on the relationship between iot-enabled process integration and supply chain performance. the cross-sectional study was undertaken with the retailers at a point of time that excluded the views of the suppliers and customers. the future study can include the supply chain partners to get a more meaningful opinion about iot adoption in improving the organisational performance. also, we suggest that a longitudinal study along the time gap of three years may validate the perceptual findings of this study. the study also lacks descriptive information thus an in-depth interview-based study in future may explain the findings of this cross-sectional study. acknowledgement we acknowledge all the reviewers and editors for their valuable suggestion to improve the quality of paper. references abs retail industry analysis. (2014). retail and wholesale industries, australia,. retrieved from http://www.abs.gov.au/ausstats/abs@.nsf/latestproducts/8622.0main%20features32012 -13?opendocument&tabname=summary&prodno=8622.0&issue=201213&num=&view= alfalla-luque, r., medina-lopez, c., & dey, p. k. (2013). supply chain integration framework using literature review. production planning & control, 24(8-9), 800-817. anderson, j. c., & gerbing, d. w. (1988). structural equation modeling in practice: a review and recommended two-step approach. psychological bulletin, 103(3), 411. armstrong, c. e., & shimizu, k. (2007). a review of approaches to empirical research on the resource-based view of the firm. journal of management, 33(6), 959-986. armstrong, j. s., & overton, t. s. (1977). estimating nonresponse bias in mail surveys. journal of marketing research, 396-402. ataseven, c., & nair, a. (2017). assessment of supply chain integration and performance relationships: a meta-analytic investigation of the literature. international journal of production economics, 185, 252-265. atzori, l., iera, a., & morabito, g. (2010). the internet of things: a survey. computer networks, 54(15), 2787-2805. baltar, f., & brunet icart, i. (2011). the use of facebook in social research. the virtual snowball method applied to the study of immigrants entrepreneurs in spain: centro de documentación, facultad de ciencias económicas y sociales, universidad nacional de mar del plata. australasian journal of information systems de vass, shee & miah 2018, vol 22, research article iot and supply chain integration and performance 21 baltar, f., & brunet icart, i. (2012). social research 2.0: virtual snowball sampling method using facebook: centro de documentación, facultad de ciencias económicas y sociales, universidad nacional de mar del plata. ben-daya, m., hassini, e., & bahroun, z. (2017). internet of things and supply chain management: a literature review. international journal of production research, 1-24. bharadwaj, a. s. (2000). a resource-based perspective on information technology capability and firm performance: an empirical investigation. mis quarterly, 169-196. bollen, k., & stine, r. a. (1992). bootstrapping goodness-of-fit measures in structural equation models. sociological methods & research, 21(2), 205-229. bollen, k. a. (1989). a new incremental fit index for general structural equation models. sociological methods & research, 17(3), 303-316. borgia, e. (2014). the internet of things vision: key features, applications and open issues. computer communications, 54, 1-31. byrne, b. m. (2013). structural equation modeling with amos: basic concepts, applications, and programming: routledge. carter, c. r., & rogers, d. s. (2008). a framework of sustainable supply chain management: moving toward new theory. international journal of physical distribution & logistics management, 38(5), 360-387. christopher, m. (2016). logistics & supply chain management: pearson uk. christopher, m., crum, m., & holweg, m. (2011). “supply chain 2.0”: managing supply chains in the era of turbulence. international journal of physical distribution & logistics management, 41(1), 63-82. christopher, m., & towill, d. (2001). an integrated model for the design of agile supply chains. international journal of physical distribution & logistics management, 31(4), 235-246. constantinides, e., kahlert, m., & de vries, s. a. (2017). the relevance of technological autonomy in the acceptance of iot services in retail. paper presented at the 2nd international conference on internet of things, data and cloud computing, icc 2017. dhl. (2015). internet of things in logistics. retrieved from dhl trend research: http://www.dhl.com/en/about_us/logistics_insights/dhl_trend_research/internet_of_thi ngs.html#.ve_empdsluu dillman, d. a. (2011). mail and internet surveys: the tailored design method--2007 update with new internet, visual, and mixed-mode guide. new york: john wiley & sons. dolbier, c. l., webster, j. a., mccalister, k. t., mallon, m. w., & steinhardt, m. a. (2005). reliability and validity of a single-item measure of job satisfaction. american journal of health promotion, 19(3), 194-198. dubey, r., gunasekaran, a., childe, s. j., papadopoulos, t., & wamba, s. f. (2017). world class sustainable supply chain management: critical review and further research directions. the international journal of logistics management, 28(2), 332-362. edwards, c., & hopkins, j. (2018). the australian supply chain tech survey. retrieved from https://sclaa.com.au/2018-australian-supply-chain-tech-survey-results australasian journal of information systems de vass, shee & miah 2018, vol 22, research article iot and supply chain integration and performance 22 elkington, j. (1997). cannibals with forks. the triple bottom line of 21st century. ellram, l. m., & cooper, m. c. (1993). characteristics of supply chain management and the implications for purchasing and logistics strategy. the international journal of logistics management, 4(2), 13-24. fabbe-costes, n., & jahre, m. (2008). supply chain integration and performance: a review of the evidence. the international journal of logistics management, 19(2), 130-154. fleisch, e., & tellkamp, c. (2005). inventory inaccuracy and supply chain performance: a simulation study of a retail supply chain. international journal of production economics, 95(3), 373-385. flynn, b. b., huo, b., & zhao, x. (2010). the impact of supply chain integration on performance: a contingency and configuration approach. journal of operations management, 28(1), 58-71. fornell, c., & larcker, d. f. (1981). evaluating structural equation models with unobservable variables and measurement error. journal of marketing research, 39-50. frohlich, m. t., & westbrook, r. (2001). arcs of integration: an international study of supply chain strategies. journal of operations management, 19(2), 185-200. gunasekaran, a., patel, c., & mcgaughey, r. e. (2004). a framework for supply chain performance measurement. international journal of production economics, 87(3), 333-347. haddud, a., desouza, a., khare, a., & lee, h. (2017). examining potential benefits and challenges associated with the internet of things integration in supply chains. journal of manufacturing technology management, 28(8), 1055-1085. hair, j. f., black, w. c., babin, b. j., & anderson, r. e. (2014). multivariate data analysis (vol. 7): pearson upper saddle river, nj. ho, d. c. k., au, k. f., & newton, e. (2002). empirical research on supply chain management: a critical review and recommendations. international journal of production research, 40(17), 4415-4430. hofmann, e., & rüsch, m. (2017). industry 4.0 and the current status as well as future prospects on logistics. computers in industry, 89, 23-34. hu, l. t., & bentler, p. m. (1999). cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. structural equation modeling: a multidisciplinary journal, 6(1), 1-55. huh, s., yook, k.-h., & kim, i.-w. (2008). relationship between organizational capabilities and performance of target costing: an empirical study of japanese companies. journal of international business research, 7(1), 91. huo, b. (2012). the impact of supply chain integration on company performance: an organizational capability perspective. supply chain management: an international journal, 17(6), 596-610. kahlert, m., constantinides, e., & de vries, s. (2017). the relevance of technological autonomy in the customer acceptance of iot services in retail. paper presented at the proceedings of the second international conference on internet of things and cloud computing, cambridge, united kingdom. australasian journal of information systems de vass, shee & miah 2018, vol 22, research article iot and supply chain integration and performance 23 kim, s. w. (2009). an investigation on the direct and indirect effect of supply chain integration on firm performance. international journal of production economics, 119(2), 328-346. kline, r. b. (2015). principles and practice of structural equation modeling (vol. 4th edtion): the guilford press. kusunoki, k., nonaka, i., & nagata, a. (1998). organizational capabilities in product development of japanese firms: a conceptual framework and empirical findings. organization science, 9(6), 699-718. lee, i., & lee, k. (2015). the internet of things (iot): applications, investments, and challenges for enterprises. business horizons, 58(4), 431-440. li, g., yang, h., sun, l., & sohal, a. s. (2009). the impact of it implementation on supply chain integration and performance. international journal of production economics, 120(1), 125-138. liu, w., & gao, z. (2014). study on iot based architecture of logistics service supply chain. international journal of grid and distributed computing, 7(1), 169-178. liu, x., & sun, y. (2011). information flow management of vendor-managed inventory system in automobile parts inbound logistics based on internet of things. journal of software, 6(7), 1374-1380. lorenzoni, g., & lipparini, a. (1999). the leveraging of interfirm relationships as a distinctive organizational capability: a longitudinal study. strategic management journal, 317-338. mattern, f., & floerkemeier, c. (2010). from the internet of computers to the internet of things from active data management to event-based systems and more (pp. 242-259): springer. mishra, d., gunasekaran, a., childe, s. j., papadopoulos, t., dubey, r., & wamba, s. f. (2016). vision, applications and future challenges of internet of things. industrial management & data systems, 116(7), 1331-1355. näslund, d., & hulthen, h. (2012). supply chain management integration: a critical analysis. benchmarking: an international journal, 19(4/5), 481-501. newbert, s. l. (2007). empirical research on the resource-based view of the firm: an assessment and suggestions for future research. strategic management journal, 28(2), 121-146. ng, i., scharf, k., pogrebna, g., & maull, r. (2015). contextual variety, internet-of-things and the choice of tailoring over platform: mass customisation strategy in supply chain management. international journal of production economics, 159(1), 76-87. o'leary-kelly, s. w., & vokurka, r. j. (1998). the empirical assessment of construct validity. journal of operations management, 16(4), 387-405. paulraj, a. (2011). understanding the relationships between internal resources and capabilities, sustainable supply management and organizational sustainability. journal of supply chain management, 47(1), 19-37. perera, c., member, c. h. l., jayawardena, s., & chen, m. (2015). context-aware computing in the internet of things: a survey on internet of things from industrial market perspective. ieee access, 2(1ng), 1660-1679. australasian journal of information systems de vass, shee & miah 2018, vol 22, research article iot and supply chain integration and performance 24 ping, l., liu, q., zhou, z., & wang, h. (2011). agile supply chain management over the internet of things. paper presented at the management and service science (mass), 2011 international conference on 2011 aug 12. podsakoff, p. m., mackenzie, s. b., lee, j.-y., & podsakoff, n. p. (2003). common method biases in behavioral research: a critical review of the literature and recommended remedies. journal of applied psychology, 88(5), 879. qrunfleh, s., & tarafdar, m. (2014). supply chain information systems strategy: impacts on supply chain performance and firm performance. international journal of production economics, 147(1), 340-350. rai, a., patnayakuni, r., & seth, n. (2006). firm performance impacts of digitally enabled supply chain integration capabilities. mis quarterly, 30(2), 225-246. ralston, p. m., blackhurst, j., cantor, d. e., & crum, m. r. (2015). a structure-conductperformance perspective of how strategic supply chain integration affects firm performance. journal of supply chain management, 51(2), 47. reaidy, p. j., gunasekaran, a., & spalanzani, a. (2015). bottom-up approach based on internet of things for order fulfillment in a collaborative warehousing environment. international journal of production economics, 159(1), 29-40. schoenherr, t., & swink, m. (2012). revisiting the arcs of integration: cross-validations and extensions. journal of operations management, 30(1), 99-115. seuring, s., & müller, m. (2008). from a literature review to a conceptual framework for sustainable supply chain management. journal of cleaner production, 16(15), 1699-1710. spekman, r. e., kamauff jr, j. w., & myhr, n. (1998). an empirical investigation into supply chain management: a perspective on partnerships. supply chain management: an international journal, 3(2), 53-67. souders, andy (2015). from desert storm to the retail store: five technologies that are closing global supply chain gaps, retreived from http://www.supplychainquarterly.com/topics/technology/20151228-fivetechnologiesthat-are-closing-global-supply-chain-gaps/ on 20 may 2018. teo, t. s., & pok, s. h. (2003). adoption of wap-enabled mobile phones among internet users. omega, 31(6), 483-498. trappey, a. j. c., trappey, c. v., hareesh govindarajan, u., chuang, a. c., & sun, j. j. (2017). a review of essential standards and patent landscapes for the internet of things: a key enabler for industry 4.0. advanced engineering informatics, 33, 208-229. tu, m. (2018). an exploratory study of internet of things (iot) adoption intention in logistics and supply chain management-a mixed research approach. international journal of logistics management, the(just-accepted), 00-00. uckelmann, d., harrison, m., & michahelles, f. (2011). an architectural approach towards the future internet of things architecting the internet of things (pp. 1-24): springer-verlag berlin heidelberg. australasian journal of information systems de vass, shee & miah 2018, vol 22, research article iot and supply chain integration and performance 25 vanpoucke, e., vereecke, a., & muylle, s. (2017). leveraging the impact of supply chain integration through information technology. international journal of operations & production management, 37(4), 510-530. verdouw, c. n., wolfert, j., beulens, a. j. m., & rialland, a. (2016). virtualization of food supply chains with the internet of things. journal of food engineering, 176, 128-136. verona, g. (1999). a resource-based view of product development. academy of management review, 24(1), 132-142. wasserman, m., & mahmoodi, f. (2017). disruptive technologies: should you give them the green l ight? retrieved from http://www.supplychainquarterly.com/topics/technology/20170302disruptive-technologies-should-you-give-them-the-green-light/ on 20 may 2018. weber, r. h. (2009). internet of things–need for a new legal environment? computer law & security review, 25(6), 522-527. whitmore, a., agarwal, a., & da xu, l. (2014). the internet of things—a survey of topics and trends. information systems frontiers, 17(2), 261-274. xu, l. d., he, w., & li, s. (2014). internet of things in industries: a survey. industrial informatics, ieee transactions on, 10(4), 2233-2243. yan, j., xin, s., liu, q., xu, w., yang, l., fan, l., . . . wang, q. (2014). intelligent supply chain integration and management based on cloud of things. international journal of distributed sensor networks, 10(3), 1-15. yellowpages. (2016). australian business directory https://www.yellowpages.com.au/, retrieved on 14 december 12017. yu, j., subramanian, n., ning, k., & edwards, d. (2015). product delivery service provider selection and customer satisfaction in the era of internet of things: a chinese e-retailers’ perspective. international journal of production economics, 159(1), 104-116. yu, w. (2015). the effect of it-enabled supply chain integration on performance. production planning & control, 26(12), 945-957. yu, w., jacobs, m. a., salisbury, w. d., & enns, h. (2013). the effects of supply chain integration on customer satisfaction and financial performance: an organizational learning perspective. international journal of production economics, 146(1), 346-358. zhao, x., huo, b., selen, w., & yeung, j. h. y. (2011). the impact of internal integration and relationship commitment on external integration. journal of operations management, 29(1), 17-32. zhou, k., liu, t., & zhou, l. (2015). industry 4.0: towards future industrial opportunities and challenges. paper presented at the fuzzy systems and knowledge discovery (fskd), 2015 12th international conference on. zhou, l., chong, a. y. l., & ngai, e. w. t. (2015). supply chain management in the era of the internet of things. international journal of production economics, 159(1), 1-3. australasian journal of information systems de vass, shee & miah 2018, vol 22, research article iot and supply chain integration and performance 26 zhu, q., sarkis, j., & lai, k.-h. (2012). examining the effects of green supply chain management practices and their mediations on performance improvements. international journal of production research, 50(5), 1377-1394. australasian journal of information systems de vass, shee & miah 2018, vol 22, research article iot and supply chain integration and performance 27 appendix table: efa results of the measurement model item no measurement item factor loading variance explained construct 1: iot capability iot has the capability in our supply chain: 1.1 to provide individual item level identification. 0.669 57% 1.2 to provide unit level (e.g. container/box/pallet) identification. 0.779 1.3 to monitor, track and trace supply chain entities and people through autocaptured data. 0.732 1.4 to measure supply chain activities, processes and its environmental conditions. 0.770 1.5 to help control supply chain processes remotely. 0.705 1.6 to make autonomous supply chain decisions. 0.577 1.7 to provide real-time information to optimise supply chain activities 0.799 1.8 to provide real-time intelligence of supply chain operations. 0.620 1.9 to provide large volumes and variety of data to apply data analytics for tactical and strategic decision making. 0.788 1.10 to strengthen inter and intra organizational information sharing within the supply chain. 0.743 1.11 to facilitate inter and intra organizational decision making within the supply chain. 0.750 1.12 to strengthen communication and coordination between operators. 0.783 eigenvalue = 6.839; kmo =0.942; bartlett’s test of sphericity: x²= 1614.476, p<0.001 construct 2: supplier integration we have been able to improve the business processes with our suppliers to: 2.1 improve information exchange with our suppliers. 0.775 60% 2.2 establish a quick ordering of inventory from our suppliers. 0.791 2.3 accurately plan and adopt the procurement process in collaboration with our suppliers. 0.801 2.4 stabilize procurement with our suppliers. 0.780 2.5 share real-time demand forecasts with our suppliers. 0.712 2.6 improve strategic partnerships with our suppliers. 0.772 2.7 help suppliers improve their processes to better meet our needs. 0.735 2.8 improve the account payable processes for suppliers. 0.572 2.9 improve the transport/logistics processes of logistics partners to deliver orders just in time. 0.734 2.10 improve our receiving processes for delivered goods. 0.770 eigenvalue = 6.007; kmo =0.939; bartlett’s test of sphericity: x²= 1342.42, p<0.001 construct 3: internal integration we have been able to improve our internal logistics processes to: 3.1 improve the integration of data among internal functions. 0.720 65% 3.2 improve real-time communication and linkage among all internal functions. 0.802 3.3 accurately plan and adopt internal processes in collaboration with cross functional teams. 0.816 3.4 make and adopt demand forecasts in collaboration with cross functional teams. 0.818 australasian journal of information systems de vass, shee & miah 2018, vol 22, research article iot and supply chain integration and performance 28 3.5 improve inventory management in collaboration with cross functional teams. 0.824 3.6 improve real-time searching of the inventory levels. 0.818 3.7 improve real-time searching of logistics-related operating data. 0.713 3.8 employ cross functional teams in process improvement. 0.704 3.9 improve replenishment of shop floor shelves. 0.782 3.10 reduce stock outs in the shop floor shelves. 0.791 eigenvalue = 6.471; kmo =0.955; bartlett’s test of sphericity: x²= 1542.049, p<0.001 construct 4: customer integration we have been able to improve the business processes with our customers to: 4.1 improve the strength of linkages with our customers. 0.788 61% 4.2 improve regular contacts with our customers. 0.828 4.3 improve communication with our customers on products and promotions. 0.708 4.4 make and adopt demand forecasts with a real-time understanding of market trends. 0.791 4.5 improve the customer shopping experience/time/ordering/customising processes. 0.761 4.6 accurately plan and adopt the checkout/dispatch/delivery processes through a better understanding of market trends. 0.729 4.7 improve the check-out/dispatch/delivery process of goods. 0.707 4.8 improve and simplify the payment receivable process from our customers. 0.741 4.9 improve customer feedback process. 0.682 eigenvalue = 5.491; kmo =0.930; bartlett’s test of sphericity: x²= 1192.148, p<0.001 construct 5: supply chain performance we have been able to develop our supply chain processes to: 5.1 improve product quality. dropped in efa 55% 5.2 improve supply chain delivery reliability. 0.641 5.3 improve fill rates. 0.656 5.4 improve perfect order fulfilment (deliveries with no errors). 0.759 5.5 improve supply chain flexibility (react to product changes, volume, mix). 0.652 5.6 reduce the cash-to-cash cycle time. 0.653 5.7 reduce the total supply chain management cost. 0.753 5.8 reduce the cost of goods sold. 0.742 5.9 improve value-added productivity (sales per employee). 0.681 eigenvalue = 4.361; kmo =0.924; bartlett’s test of sphericity: x²= 719.727, p<0.001 construct 6: organisational performance we been able to develop our organisational operations to: 6.1 improve the product delivery cycle time. 0.715 55% 6.2 improve productivity (e.g. assets, operating costs, labour costs). 0.714 6.3 improve sales of existing products. 0.828 6.4 find new revenue streams. 0.606 6.5 build strong and continuous bonds with customers. 0.741 6.6 gain precise knowledge of customer buying patterns. 0.741 6.7 improve customer satisfaction. 0.735 6.8 improve employee satisfaction. 0.688 6.9 improve employee health and safety. 0.701 australasian journal of information systems de vass, shee & miah 2018, vol 22, research article iot and supply chain integration and performance 29 6.10 reduce energy use. 0.730 eigenvalue = 6.024; kmo =0.945; bartlett’s test of sphericity: x²= 1248.043, p<0.001 kmo=kaiser-meyer-olkin measure of sampling adequacy copyright: © 2018 de vass, shee & miah. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. paper title (paper title style) australasian journal of information systems al-saggaf & ceric 2017, vol 21, research article bullying in the australian ict workplace 1 bullying in the australian ict workplace: the views of australian ict professionals yeslam al-saggaf school of computing and mathematics, charles sturt university yalsaggaf@csu.edu.au arnela ceric school of management and marketing, charles sturt university abstract the aim of this study is to examine bullying in the workplace from the perspective of australian information communication technology (ict) professionals. the data collection for this project included conducting a quantitative survey with 2,315 participants and 43 qualitative interviews with members of australian computer society (acs). we found that 630 ict professionals, or 27.23% of all survey respondents, identified workplace bullying as an ethical problem. the majority of survey respondents who selected bullying as an ethical issue were permanent full time employees (n= 413, 65.6%). a significant relationship was found between respondents identifying bullying as an ethical issue in the survey and their job classification (deviance = 25.55, df = 11, p=0.0076), suggesting that job classification, among other things, does predict respondents’ selection of bullying. furthermore, our survey and interview findings indicate that the more mature respondents, as well as those in the managerial roles, have a greater concern about bullying. keywords: ict professionals; professional ethics; bullying; harassment. 1 introduction bullying is a major problem in the workplace (standen, paull & omari, 2014). it reduces employee’s productivity, motivation and wellbeing (cowie, jennifer, neto, angula, pereira, del barrio, & ananiadou, 2000) and affects the overall work environment and organisational climate (kelloway, bryant, buttigieg, & hanley, 2009; harvey, heames, richey, & leonard, 2006). overall, the outcomes are negative for both, employers and employees. however, to our knowledge, no research has been done into workplace bullying in the australian ict industry. the ict workplace is distinct due to gender imbalance (oleksy, just, & zapedowska‐kling, 2012) and requirements for high technical knowledge. thus, the purpose of this paper is to present findings on bullying from the perspective of australian ict professionals, based on a research project that examined professional ethics more broadly in the ict workplace in australia. as part of the project on ethics in ict workplace, two stages of data collection were used. in the first stage, a quantitative survey was conducted with 2,313 members of the australian computer society (acs), administered using surveymonkey. survey data identified workplace bullying to be one of the top ten ethical problems faced by ict professionals (see table 2). based on the survey data, we identified demographic variables of survey respondents who identified bullying as one of the ethical issues they faced. understanding these demographic variables seems relevant for identifying a general profile of ict professionals who are concerned about bullying. the second stage of the project involved conducting semi-structured interviews with 43 participants. interviewees were selected from the group of survey respondents in the earlier stage of the data collection process. qualitative interviews with ict professionals provided additional insights into the forms that bullying takes in the ict workplace. australasian journal of information systems al-saggaf & ceric 2017, vol 21, research article bullying in the australian ict workplace 2 the structure of the paper is as follows. literature on workplace bullying and the demographic characteristics of participants on bullying are explored in the first section. quantitative research methodology and the findings from the survey are presented in the second section, followed by discussion on semi-structured interviews and the findings from the interviews. in the last section of the paper, the findings are discussed and concluding remarks are provided. 2 workplace bullying workplace bullying and workplace harassment are used interchangeably in the literature (skinner, peetz, strachan, whitehouse, bailey & broadbent, 2015). workplace bullying can “involve one or more perpetrators and targets at various levels of the organization, occur at various degrees of severity and frequency, be intentional or unintentional, and manifest in a wide variety of unacceptable behaviours in response to various precipitants” (crawshaw, 2009, p.265). the definition provided here is a generic one so that it can encompass a range of bullying situations and contexts. for example, harvey et al. (2006) identified five groups of behaviours related to workplace bullying. these behaviours are related to isolation by preventing access to opportunities or withholding important information, threats to personal standing by insulting or teasing the victim overwork through an increase of workload to targeted individuals, as well as reminding the victim of their failures, and humiliating them in public which can damage their professional status. definitions of workplace bullying have several components, namely, a) the targets of bullying experience negative behaviour persistently, b) the targets of bullying experience psychological or physical harm, c) they label the experience as bullying, d) the targets of bullying cannot defend themselves, and e) the targets of bullying perceive themselves as having less power than the bully (keashly & neuman, 2004). definitions provided in the literature may be different from personal definitions of workplace bullying. saunders, huynh, and goodman-delahunty (2007) found that lay people include two essential aspects of workplace bullying commonly used in the literature. these are occurrences of a negative behaviour (reported by 98.3% of their participants) and some form of harm that such behaviour causes to the target (86.3%). in addition, participants identified two additional components of workplace bullying that are not included in the literature. according to their definition, bullying involves unprofessional behaviour that violates the way employees are expected to be treated in a workplace (25.5%), and it needs to be intentional in order to be defined as bullying (21.4%). moreover, participants rarely included non-essential elements of bullying in their definitions. according to them, bullying does not need to be persistent or involve power imbalance, and targets do not need to self-label their experiences as bullying. workplace bullying represents a major source of stress at work that can cause severe social, psychological and psychosomatic problems for the victims (standen, paull & omari, 2014). victims of workplace bullying may suffer from post-traumatic stress syndrome (leymann & gustafsson, 1996; rodríguez‐muñoz, moreno‐jiménez, sanz vergel, & garrosa hernández, 2010) as well as insomnia, melancholy, lack of concentration, apathy, socio-phobia (bjorkqvis, osterman, & hjelt-back, 1994), depression (zapf, 1999), chronic and cardiovascular diseases (kivimäki, virtanen, vartia, elovainio, vahtera, & keltikangas-järvinen, 2003). apart from targets, workplace-bullying has a significant effect on bystanders as well. for example, vartia (2001) found that both, victims and observers of bullying experience higher rates of general and mental stress in comparison to employees who did not experience or observe bullying. bullying affects organisational performance through higher rates of absenteeism, staff turnover, reduced efficiency, decline in work quality, increased error margins and an unsafe workplace (skinner et al., 2015; kelloway et al., 2009; sheehan, mccarthy, barker, & henderson., 2001). in addition, zapf (1999) reported that workplace bullying creates negative working conditions due to lack of communication, conscious miscommunication and conflicts. moreover, harvey et al. (2006) reported four outcomes of workplace bullying that negatively affect organisational performance. victims’ job-related abilities can decrease, their innovative behaviour can be eliminated, organisational level of flexibility and adaptability can be australasian journal of information systems al-saggaf & ceric 2017, vol 21, research article bullying in the australian ict workplace 3 decreased, and long-term success of the organisation can be shattered. participants in cowie et al.’s (2000) study report that bullying reduced their wellbeing, motivation and efficiency. there are multiple causes of workplace bullying (salin, 2003b; zapf, 1999). for example, leymann (1993) identified deficiencies in work design and leadership behaviour, as well as a socially exposed position of the victim and a low moral standard in the work environment as causes of harassment. hauge, skogstad, and einarsen (2009) found that being a male and being a target of bullying, as well as role conflict and interpersonal conflict strongly predict being a bully. in addition, notelaers, de witte, and einarsen (2010) found that role conflict and role ambiguity cause bullying at work. vartia (1996) found that psychological work environment and organisational climate explain 24% of the variance in bullying. furthermore, victims surveyed in vartia’s (1996) study reported envy, a weak superior and competition for tasks, advancement or superior approval as reasons for why they were bullied. zapf (1999) found that victims saw organisational climate, high stress and organisational problems as causes of workplace bullying. some victims reported personal reasons for bullying, such as low performance, deficits in social skills, ‘being difficult’, aggressive or moaning. in addition, claybourn (2011) examined the relationship between moral disengagement, work characteristics and workplace harassment. he found that employees who are morally disengaged are more likely to report that they have been subjected to workplace harassment, and ultimately, they themselves participate in harmful behaviours towards others. salin (2003b) concluded that bullying is a result of interaction between three groups of factors associated with workplace bullying. these are enabling structures (e.g., power imbalances, dissatisfaction), incentives (e.g., internal competition, expected benefits) and triggering circumstances (e.g., organisational changes). definitions of bullying discussed here (saunders et al., 2007) and the specific forms that bullying takes identified by harvey et al. (2006) will be further compared with the findings from the qualitative component of this study. this will shed more light into the forms that bullying behaviour takes in the australian ict workplace. 3 demographic characteristics of participants in studies on ethical issues and bullying 3.1 age age is an important demographic characteristic that influences ict professionals’ awareness of ethical issues. wilford and wakunuma (2014) found that more mature ict professionals have a significantly higher level of ethical awareness in comparison to younger ict professionals. the latter group was more concerned with process and compliance with the requirements of their organisation, and was relatively unaware of ethical issues. age also influences the emphasis that ict professionals place on ethical regulations. lucas and mason (2008) found that generation y ict professionals, that is, those born in 1980s and 1990s, are different from other generations in regards to the importance they place on ethical regulations. generation y put less emphasis on ethical regulations and think that they ought to be less important. this finding is relevant for decision-making in the ict field as the importance placed on ethical regulations can have an impact on the judgement of decisionmakers. however, the ethical awareness in wilford and wakunuma’s (2014) and the emphasis placed on ethical regulations in lucas and mason’s (2008) studies relate to issues in ict, such as trust, social exclusion, privacy, and misuse of technology, not to workplace bullying. notelaers, vermunt, baillien, einarsen, and de witte (2011) found that employees above 54 years and those less than 25 years of age are least likely to be bullied at work, while employees in the age group between 35 and 44 have the highest risk of workplace bullying, followed by those aged between 25 and 34. similarly, einarsen and raknes (1997) as well as bowling and beehr (2006) found that older men experience less victimisation than younger men. in contrast, einarsen and skogstad (1996) found that older workers had a higher risk of victimisation than the younger employees. vartia (1996) found a small correlation between age australasian journal of information systems al-saggaf & ceric 2017, vol 21, research article bullying in the australian ict workplace 4 and bullying. on the other hand, ortega, høgh, pejtersen, and olsen (2009) did not find age to be important in explaining the prevalence of bullying. 3.2 gender one of the characteristics of the ict industry is unequal participation of men and women in ict employment (oleksy, et al., 2012). there is some research that found that women are more ethical than men when it comes to ict ethical issues (mason & mudrack, 1996; valentine & page, 2006) and that this has a favourable impact on the organisation’s bottom line (quigley & mohamad, 2008). in contrast, young and case (2009) examined a general population and found that there were no significant gender differences in the perception of unethical ict behaviour. on the other hand, lucas and mason (2008) found that male ict professionals thought that ethical regulations should be less important, in comparison to female ict professionals. mudrack and mason (2013) concluded that gender does not impact on ethical judgement. however, in cases when the difference occurs, women judged ethically questionable activities as being less appropriate than men. salin (2003a) explains the complexity between gender and workplace bullying, although this has received little attention in research. perpetrators of bullying are most likely to be males who have been a target of bullying themselves (hauge et al., 2009). female workers report higher victimisation rates when they occupy middle and senior management roles (hoel, cooper, & faragher, 2001), and when they are in non-traditional jobs (gutek & morasch, 1982). male workers and supervisors report that they observe general and workplace bullying more often than women in the same roles (hoel et al., 2001), but they do not identify themselves with being bullied when asked directly (jóhannsdóttir & ólafsson, 2004). women report being bullied by their superiors, colleagues and subordinates of both genders, while men tend to be bullied by male supervisors and to a smaller extent by their colleagues (salin, 2003a). men tend to confront when bullied, women tend to seek help or avoid the situation (jóhannsdóttir & ólafsson, 2004), often by leaving the organisation (salin, 2003a). griffiths and moore (2010) found that bullying is one of the main reasons why women in the uk left the ict sector. 3.3 occupation occupations such as teachers and managers, as well as nurses and social services have a low chance of being bullied. this can be related to relatively lower workload and high level of autonomy. salin (2001) found that managers indeed have a lower risk of being a victim of bullying, while this risk increases for experts, middle managers and officials/clerks. workplace bullying among business professionals in managerial and expert positions is found to be higher when using a predefined list of negative acts of bullying (24.1%), than when respondents classify themselves as being bullied (8.8%) (salin, 2001). furthermore, managers and supervisors have been identified as perpetrators rather than victims of bullying (cowie et al., 2000). ortega et al. (2009) found that occupational status and type of work processes were important indicators of workplace bullying. that is, unskilled workers were found to have the highest prevalence of bullying, while managers and supervisors had the lowest prevalence. in addition, respondents working with things reported the highest prevalence of bullying, while respondents working with customers, the lowest. ortega et al. (2009) did not find age or gender as important as type of work in explaining the prevalence of bullying. notelaers et al. (2011) also found that blue-collar workers have a higher risk of workplace bullying as compared to white-collar employees. 3.4 job classification and years of experience prior research has shown that full-time employees have a higher chance of being bullied in comparison to temporary employees (notelaers et al., 2011). bowling and beehr’s (2006) meta-analysis found that employees with longer tenure were more likely to be victimised than were those with shorter tenure. cowie et al. (2000) found that length of current job was the australasian journal of information systems al-saggaf & ceric 2017, vol 21, research article bullying in the australian ict workplace 5 only demographic characteristic that significantly correlates experiencing bullying behaviours in their uk sample. these findings can be explained with the entrapment hypothesis which states that permanent employees are less likely to move jobs when working conditions deteriorate (cuyper, notelaers, & witte, 2009). the demographic characteristics discussed here will be further compared with the demographic characteristics of the participants who selected bullying in the survey and the interviewees who raised the issue of bullying during interviews. this should shed light into the groups of participants/interviewees who are concerned about bullying in the australian ict workplace. 4 methods and findings 4.1 the quantitative survey 4.1.1 survey procedure the first stage of the data collection process involved a survey administered using surveymonkey to allow the participants to complete the survey and return it over the internet. the survey was informed by the results of a 2006 survey conducted by lucas and weckert (2008) and also by the instrument they used. all active acs members (approximately 18,600) were invited to participate in the web-based survey by direct email sent to them by the acs once on 12 september 2013. the survey was closed on 6 november 2013 after the response rate reached 12.4%. the online survey was prefaced by the ethics consent sheet (including assurances of anonymity) and a description of the study. the questions comprised both closed and open-ended questions. this study reports only on the closed questions. 4.1.2 sample a total of 2,315 participants completed the survey. the average number of years of work experience for all respondents was 19 years. however, 61.6% of those who selected bullying as one of the most common problems had more than 20 years of experience. table 1 shows a summary of the demographic information of all participants as well as the demographic information for the participants who selected bullying as one of the most common problems. as can be seen from table 1, it appears that the overall profile of the survey respondents is not much different from the profile of the selectors of bullying as one of the most common problems. australasian journal of information systems al-saggaf & ceric 2017, vol 21, research article bullying in the australian ict workplace 6 demographic information survey participants participants who selected bullying n % n % gender female 356 15.5 113 17.9 male 1,940 83.9 513 81.4 n/a 17 0.7 4 0.6 age < 35 692 30 155 24.6 36 45 516 22.3 148 23.5 46 55 576 25 174 27.7 > 56 524 22.7 153 24.3 n/a 5 0.2 0 0 state act 247 10 80 12.7 nsw 696 30.4 199 31.6 nt 27 1.2 8 1.3 qld 279 12.2 81 12.9 sa 120 5.5 28 4.4 tas 42 1.8 10 1.6 vic 581 25.4 149 23.7 wa 218 9.5 57 9.0 overseas 80 3.5 14 2.2 n/a 23 1.0 4 0.6 occupation administrator 134 6.5 36 5.7 consultant 502 24.3 146 23.2 developer 307 14.8 77 12.2 education 150 7.3 43 6.8 manager 698 33.8 196 31.1 technical support 277 13.3 76 12.1 other 215 10.39 54 8.6 n/a 247 11.9 2 0.3 geographical location capital city 2,069 89.5 581 92.2 regional area 215 9.43 46 7.3 n/a 29 1.3 3 0.5 job classification business owner with employed staff 57 2.7 8 1.3 fixed term contract 251 11.8 83 13.2 indefinite contract 34 1.6 8 1.3 permanent full-time 1,388 65.4 413 65.6 permanent part-time 90 4.2 17 2.7 self-employed 112 5.3 26 4.1 temporary full-time 61 2.9 11 1.7 temporary part-time 63 3.0 16 2.5 volunteer 67 3.2 13 2.1 other 121 5.69 33 5.2 n/a 2 0.3 table 1: demographic characteristics of the survey participants and those who selected bullying as an ethical issue in the survey 4.1.3 statistical analysis multiple response frequency (mrf) analysis and cross tabulations participants were asked in the survey to choose among different ethical problems they experienced in the workplace. since respondents were allowed to select more than one answer, a multiple response frequency (mrf) analysis (al-saggaf, burmeister & weckert, 2016) was australasian journal of information systems al-saggaf & ceric 2017, vol 21, research article bullying in the australian ict workplace 7 judged to be the most appropriate technique for analysing the survey data. in addition, cross tabulations were performed to see if there are differences in responses based on the demographic information. the findings from the mrf analysis are summarised below. the mrf analysis revealed that bullying was ranked ninth in a list of the most common ethical problems experienced by ict professionals (n=630, 27.2%). table 2 lists the top 10 ethical problems along with the number of responses and their percentages. the focus of this paper is on workplace bullying, and thus discussion on other ethical issues presented in table 2 is outside the scope of this paper. ethical problems number of survey respondents n (%) compromising quality to meet deadlines 1104 47.7 blaming others for own mistakes 957 41.4 compromising functionality to meet deadlines 846 36.6 overworking staff 762 32.9 incompetence 750 32.4 conflict of interest 682 29.5 unprofessional behaviour 633 27.4 compromising user requirements to meet deadlines 632 27.3 bullying 630 27.2 compromising security to meet deadlines 611 26.4 table 2: the top 10 ethical problems faced by australian ict professionals generalised linear models to investigate which demographic variables predict respondents’ selection of bullying in the survey, generalised linear models (glms) were used. the responses to the bullying question are binomial (recorded as a yes/no), whereas all the demographic variables are categorical. for this reason glms were fitted to investigate the relationships between the predictor variables and the binomial response variable. the glms were carried out on the data using r (version 3.0.2) and all requirements of this analysis were verified. the analysis of deviance revealed that there is a significant relationship between respondents’ selection of bullying as an ethical issue in the survey and age, gender, geographic location, job classification, and years of experience (see table 3 for the results of the analysis of deviance). all other demographic variables showed no evidence of a relationship with respondents’ selection of bullying in the survey. the fact that the statistical analysis (the analysis of deviance) showed that age, gender, geographic location, job classification and years of experience all predict the survey respondents’ selection of bullying raises a question: what are the demographic characteristics of the interviewees who raised the issue of bullying during the interviews. this question will be addressed in the subsequent section titled ‘comparison between the survey participants and interviewees who raised the issue of bullying in the survey/interview’. demographic variable deviance degrees of freedom (df) p age 18.02 8 0.0211 gender 4.15 1 0.0416 geographic location 4.58 1 0.0324 job classification 25.55 11 0.0076 years of experience 13.92 1 0.0002 table 3: the analysis of deviance australasian journal of information systems al-saggaf & ceric 2017, vol 21, research article bullying in the australian ict workplace 8 glm was also used to examine whether there is a relationship between the perceived frequency of occurrence of unethical behaviour and the respondents’ selection of bullying. this technique is fitted to investigate this relationship since the predictor is also a categorical variable. the analysis of deviance revealed that there is a significant relationship between the perceived frequency of occurrence of unethical behaviour and respondents’ selection of bullying (deviance = 143.29, df = 4, p=0.0000) suggesting that this variable is also a predictor for respondents’ selection of bullying in the survey. notelaers et al. (2011) found that employees above 54 years and those less than 25 years of age are least likely to be bullied at work, while employees in the age group between 25 and 44 have the highest risk of workplace bullying. however, a more recent literature from the academic sector, specifically skinner et al. (2015), has indicated that older employees are more likely to be bullied than their counterparts. in the current study, the analysis of deviance showed that age predicted the selection of bullying in the survey. we found that 51.2% participants between 36 and 55 years of age selected bullying as a common ethical problem. in comparison, only 24.6% or younger participants between 18 and 35 years of age, and 24.3% of participants older than 56 years also identified bullying as an ethical issue in the survey. this suggests that middle-aged participants are more concerned about bullying than participants in other age categories. this finding is consistent with notelaers et al.’s (2011) finding that middle-aged employees are more likely to be bullied at work, but is not in line with skinner et al.’s (2015) finding that older employees are more likely to be bullied at work. gender was also found to be a predictor for the selection of bullying in this study. almost 18% of interviewees who selected bullying were females. in comparison to the 15.5% females who participated in the survey, this finding suggests a higher proportion of females working in ict could be victims of bullying; but this is a question for future research. similarly, job classification was also found to be a predictor for the selection of bullying. however, the finding that a higher proportion of permanent full time employees (65.6%) and fixed term contracts (13.2%) selected this problem in our survey is aligned with the literature. the literature suggests that full time employees have a higher chance of being bullied in comparison to temporary employees (notelaers et al., 2011). 4.2 the qualitative interviews 4.2.1 conducting the interviews and analysing the data the survey was followed by a set of semi-structured interviews with 43 participants selected from the pool of survey participants in the previous stage of the data collection process. the interviews were conducted during the month of february 2014 and took place in the six capital cities of the australian states. all interviews were audio recorded and transcribed verbatim. the invitation for participation was sent to all survey participants who indicated their willingness to participate in the interviews. however, during the interviewing stage only some confirmed their willingness to participate in the interviews. interviewees were selected based on their demographic characteristics to ensure that we captured a diversity of views representative of those who work in ict. the results of the glms did not influence the choice of the participants in the interviews as the purpose of these follow-up interviews was to explore participants’ perceptions of the nature of the ethical problems experienced in the ict workplace. the final sample of interviewees included professionals from a range of ict organisations, both large and small, representing different geographic locations, ages, genders, types of jobs, and employment experience. table 4 lists demographic characteristics of the interviewees who participated in this research project. it should be noted that only 10 (out of 43) interviewees raised the issue of bullying during interviews. their views on bullying are further reported in the next section. australasian journal of information systems al-saggaf & ceric 2017, vol 21, research article bullying in the australian ict workplace 9 interviewees’ characteristics interviewees interviewees who raised bullying n (%) n (%) gender female male 7 36 16.28 83.72 1 9 2.3 21 age < 35 36-45 46-55 > 56 5 9 16 13 11.63 20.93 37.21 30.23 1 2 4 3 2.3 5 9.3 7 occupation accreditor business analyst consultant database/it coordinator manager other 1 4 7 1 16 14 2.33 9.30 16.28 2.33 37.21 32.56 1 1 2 1 4 1 2.3 2.3 5 2.3 9.3 2.3 city adelaide brisbane canberra melbourne sydney perth 5 6 6 8 11 7 11.63 16.28 16.28 18.60 25.58 16.28 1 2 1 3 2 1 2.3 5 2.3 7 5 2.3 job classification fixed term contract permanent full-time 7 36 16.28 83.72 2 8 5 19 ict work experience (years) 10-19 20-29 30-39 > 40 7 10 16 10 16.28 23.26 37.21 23.26 3 3 2 2 7 7 5 5 table 4: characteristics of the interviewees the transcribed interviews were analysed using thematic (qualitative) analysis with the help of qsr nvivo 10. the unit of analysis was each transcribed interview document. data analysis proceeded as follows. first, the researchers read the interview documents several times in order to familiarise themselves with the data collected. next, themes were created based on keywords and key topics in the interview documents. researchers discussed the themes, and agreed on the mutual approach they were going to use to code text for each theme. text with the same themes was located within the interview documents and assigned to these themes. themes then acted as “buckets” in the sense that they held all the data related to a specific theme. these themes were then further divided into specific sub-themes. this was to create a hierarchy that made it easy to make sense of the data and facilitate interpretation. 4.2.2 findings from the qualitative interviews findings from the 43 interviews that were conducted as part of the larger research project are rich and wide-ranging. for the purpose of this paper, our focus is on the forms that bullying can take in the ict workplace, as identified by the interviewees. the forms that bullying takes in the ict workplace participants highlighted several forms that bullying can take. these are compared with five groups of bullying behaviours identified by harvey et al. (2006). first, interviewees reported that bullying behaviour in the ict workplace does include isolation of a victim through preventing their access to opportunities. interviewee 13 explains: “where i don’t like that person so i’ll give all the interesting work to that other person and if i just don’t give any interesting work to that person, maybe they’ll go away”. this quote also shows that bullying can be intended and it can be purposeful in encouraging some employees to leave the organisation. other interviewees also reported that the end goal of bullying behaviour is to destroy other people’s careers. interviewee 41 emphasised “he just got rid of them, he actually got rid of them” and the reason for this is that “he didn’t like the fact they said he couldn’t have australasian journal of information systems al-saggaf & ceric 2017, vol 21, research article bullying in the australian ict workplace 10 an apple mac or whatever”. such behaviour can have significant repercussions for the employers in terms of cost of recruiting and potentially training a new employee. another way to isolate a victim is done through withholding important information. interviewee 12 described this behaviour as “sabotage”: “that’s where you get some information that is disingenuous and they say okay go and do this work and you will go and conduct the work but really the intention is for you to never be successful in doing that work”. the intent behind such behaviour is malicious. the bully gives their victim a task that (s)he will fail at. this is intended behaviour that is practised covertly. the second form of bullying identified by harvey et al. (2006), threats to personal standing by insulting or teasing the victim was also mentioned by several interviewees. an example with sending inappropriate emails or text messages, described by interviewee 24 is related to this: “so a guy was fired for a text that was commenting on a girl or something like that. ... it was saying she’s was really overweight”. from this example it is clear that employers do take action when they have a written record as evidence of bullying. however, as discussed earlier, bullying behaviour is not always overt and easy to notice. this makes bullying behaviours hard to stop in the workplace. furthermore, employees may fear retribution and thus tolerate bullying behaviour, rather than report it. interviewee 12 emphasised that the fear of retribution is particularly strong in branch locations “where things are smaller and it becomes more obvious if there are differences or accusations made”. thus, when interviewee 23 states that there was never any bullying or a harassment charge in her/his organisation, the question remains if this is indeed due to absence of bullying, or because it has not been discovered yet. interviewee 28 agrees with the latter option when (s)he informs that “the corporate psychopath still lives and the bully will still thrive on”. this example questions the success of organisational approaches to dealing with bullying such as “they described bullying and they sent out dvds, educational dvds and they got a policy written about it” (interviewee 12). interviewees 28 pointed to a failure with this formal approach: “the corporate can say we've provided the training, but the actual one-on-one follow-up wasn’t there”. hence, a different strategy was suggested that might be more successful, and that is “to talk about levels of informal discussions with management and perhaps a support person” (interviewee 17). bullying behaviour can also take a physical form. interviewee 31 remembered a situation: “she boiled over one day and this poor guy was out near the coffee machine making himself a cup of coffee and she physically came over and pushed him and nearly spilt some hot water on him and swore at him”. in this example, the bully is a female, and the victim is a young man, age between 19 and 21. this is contrary to the findings in the literature that bullies are mostly men (hauge et al., 2009). interviewee 31 further explained that the ‘poor guy’ from this example was “trying to hold his emotions back saying …i don’t know what i did to her”. this indicates that bullying behaviour is not necessarily related to anything that a victim did or said, but it can be based on their lack of experience or power in the organisation. comment from interviewee 25 indicates that bullying behaviour is in the eyes of the perceiver, and it may not be recognised or intended by the so called bully “if a person feels they’ve been bullied then it is bullying, but from the other person’s point of view they just thought they were having that kind of casual banter, chat”. the third type of bullying behaviour identified by harvey et al. (2006) includes overwork through an increase of workload to targeted individuals. this was mentioned by interviewee 41 who explained that bullying can also involve exerting “pressure” on employees “to produce results”. surprisingly interviewee 41 also explained that bullying can also mean enforcing “poor processes” or “hiring the wrong people”. this is particularly detrimental for the organisational success, and can create a range of further issues in the long run. thematic analysis of 43 interviews with australian ict employees provided some rich insights into the bullying behaviour in the australian ict workplace. however, we haven’t found evidence of two of the bullying behaviours identified by harvey et al. (2006). the first one is in relation to destabilising the victim by reminding them of their failures, and the second one australasian journal of information systems al-saggaf & ceric 2017, vol 21, research article bullying in the australian ict workplace 11 is in relation to threats to victim’s professional status. in addition, interviewees included two essential aspects of workplace bullying commonly used in the literature, occurrences of a negative behaviour and some form of harm that such behaviour causes to the target (crawshaw, 2009). our discussion on the findings from the qualitative interviews also highlights that bullying behaviour in the australian ict workplace involves unprofessional behaviour in a workplace, and that it is intentional. these are two aspects of bullying behaviour not commonly included in the literature (saunders et al., 2007). 5 comparison between the survey participants and interviewees who raised the issue of bullying in the survey/interview since the glms revealed that participants’ age, gender, geographic location, job classification and years of experience all predict the selection of bullying as a response in the quantitative survey, it is worth comparing the percentage of the survey participants who identified bullying as an ethical issue with the percentage of the interviewees who brought up bullying during interviews. in terms of age, 14.3% of the interviewees who raised the issue of bullying were between 36 and 55 years of age. this is inconsistent with the percentage of those who selected bullying in the survey. that is, in the survey 51.2% of those who selected bullying as one of the common ethical problems in the survey were between 36 and 55 years of age. however, the percentages are similar with regards to gender and geographic location although the influence of these two aspects is smaller because the vast majority of participants are males anyway and all interviews were conducted in state capital cities. the percentage of the survey participants who selected bullying and the percentage of the interviewees, who raised the issue of bullying, are also dissimilar in the area of years of experience. whereas 61.69% of the survey participants who selected bullying had more than 20 years of experience in it, only 16.3% of the interviewees who raised the issue of bullying during interviews had more than 20 years of experience in it. however, a similarity between the percentage of the survey participants who identified bullying as an ethical issue and the percentage of the interviewees who raised the issue of bullying during interviews exists in the job classification area. while 13.2% of those who selected bullying in the quantitative survey classified their jobs as fixed term contracts, 13.9% of the interviewees, who raised the issue of bullying during interviews, indicated they are employed on the fixed term contract. this is a surprising finding and should be further examined in future. 6 discussion the statistical analysis showed that age, gender, geographic location, job classification and years of experience all predict the survey respondents’ selection of bullying. however, in comparison with the demographic characteristics of the interviewees who raised the issue of bullying during the interviews, there is a mismatch. in terms of age, 14.3% of the interviewees who brought up the issue of bullying during interviews were between 36 and 55 years of age compared to 51.2% of the same age groups who selected bullying as one of the most common ethical problems in the survey. notelaers, et al. (2011) found that employees between 36 and 55 years of age are more likely to be bullied at work compared to those above 54 years and those less than 25 years of age. however, skinner et al. (2015), has indicated that older employees are more likely to be bullied than their counterparts. further research is needed to shed light on both; the inconsistency in the literature and the inconsistency between the findings from the survey and the findings from the interviews. in terms of gender and geographic location the proportion of the survey participants who identified bullying is similar to the proportion of the interviewees who brought up this issue but the influence of these two variables is smaller because the vast majority of participants are males and all interviews were conducted in state capital cities. there is also a similarity australasian journal of information systems al-saggaf & ceric 2017, vol 21, research article bullying in the australian ict workplace 12 between the proportions of the survey participants and the interviewees in the job classification area. in the quantitative survey 13.2% of those who selected bullying classified their jobs as fixed term contracts whereas in the interviews 13.9% classified their jobs as fixed term contracts. interestingly, skinner et al. (2015) also found geographic location (i.e. city vs. regional) and job classification (i.e. length of current employment, continuing position and appointment) were predictive of the self-reporting of bullying in addition to individuals’ background (i.e. being indigenous, speaking english at home and age). but since in the academic sector skinner et al. (2015) found that the rate of bullying of academic staff was higher in regional universities than in metropolitan ones and given this study found geographic location to be a predictor of bullying, future research could compare bullying in metropolitan ict workplaces with those in regional areas. hopefully, this paper will pave the way for such future research. in the area of years of work experience the proportion of the survey participants who selected bullying and the proportion of the interviewees who raised this issue are also dissimilar (61.69% of survey participants compared to 16.3% of the interviewees had more than 20 years of experience in it). cowie et al. (2000) found that the length of current job was the only demographic characteristic that significantly correlates experiencing bullying behaviours in their uk sample. while this may explain why a significant proportion of survey participants with more than 20 years of experience selected bullying, it does not explain why a smaller proportion of the interviewees who had more than 20 years of experience selected bullying. further research is needed to understand better the correlation between bullying and years of experience. in terms of the forms that bullying takes in the ict workplace, the qualitative in-depth interviews highlighted several forms. bullying can be practiced covertly by someone giving another person a task that he/she knows the other person will fail at, to set them up for failure, or by not giving them work at all to deny them opportunities to flourish or by giving them the wrong information or withholding the information that they need to complete their tasks to sabotage their efforts or not including them in groups to make them feel excluded or isolated. bullying can also be practiced overtly by nit-picking over someone’s work or publically defaming or harassing them or by making or emailing offensive comments or by being physically aggressive. the frames of exerting “pressure” on employees “to produce results”, or enforcing “poor processes”, or “hiring the wrong people” can all be categorised under ‘corporate bullying’. with the exception of two frames that were not reported by the participants in harvey’s et al. (2006) study, namely ‘destabilising the victim by reminding them of their failures’ and making threats to a victim’s professional status, all the other frames reported in this study were highlighted in harvey’s et al. (2006) study. for example, the participants in the current study and harvey’s et al. (2006) study identified isolation such as by denying access to opportunities or withholding important information. they also identified overwork and threats to personal standing such as by insulting or teasing the victim. 7 conclusion the aim of this study was to examine bullying in the workplace from the perspective of australian information communication technology (ict) professionals. the data collection for this project included conducting a quantitative survey and conducting qualitative interviews. the methods related to each other in this way: the quantitative survey identified bullying as the ninth from the top in the list of the most common ethical problems experienced by ict professionals (n=630, 27.2%) (see table 2). the findings from the survey helped shape both the content of the follow-up interviews and the selection of those interviewed. the findings from the qualitative interviews highlighted the forms that bullying takes in the australian ict workplace. a significant relationship was found between respondents’ selection of bullying as a survey response and job classification. job classification, among other factors, predicted respondents’ selection of bullying. in both the survey and interviews fixed term contracts had australasian journal of information systems al-saggaf & ceric 2017, vol 21, research article bullying in the australian ict workplace 13 a greater concern about bullying than the permanent professionals. obtaining such findings would not have been possible had only one data collection method been used. this paper reported on different aspects of bullying in the australian ict industry as identified by ict professionals. this is important for developing appropriate anti-bullying strategies in the workplace (standen, paull & omari, 2014). in addition, the findings reported in this paper point to a specific group of ict professionals based on their demographic characteristics who identified workplace bullying as an ethical issue in the survey. this is an important finding that raises additional questions, such as do more fixed term contracts have a greater concern about bullying than the permanent ict professionals and how bullying affects the bullied and those who are witnessing the bullying. the research presented in this paper makes three contributions to the literature on workplace bullying in the ict workplace. first, the paper highlighted the forms that bullying takes in the ict workplace. second, the paper offered a comparison between the forms that bullying takes in the ict workplace and those in other workplaces as discussed in the literature. it is anticipated that this discussion will not only stimulate debate among researchers interested in this area, but that it will also raise the readers’ awareness about these forms. the third, and arguably the most important contribution, was the discovery that the group of participants, who selected bullying in the survey and raised the issue of bullying during the qualitative interviews, were not the likely victims, but more likely the witnesses of such behaviour in the workplace. two limitations to the present study are outlined. one limitation is that not all survey participants volunteered to do interviews; only 419 survey participants (21.6%) expressed an interest in taking part in interviews. another limitation is that a higher proportion of full time employees and those on fixed term contracts participated in the interviews. this could have skewed the results of this analysis, as participants with other job classifications did not participate in the interviews. in terms of suggestions for future research, the finding that 13.2% of those who selected bullying in the quantitative survey classified their jobs as fixed term contracts and 13.9% in the interviews classified their jobs as fixed term contracts is interesting. it is not clear why this group of ict professionals, who are the unlikely victims of bullying, selected bullying as a response in the survey and raised this issue during interviews. is it possible that these external professionals suffered while observing this wrongdoing happening to others? or does this mean that fixed term contracts have a greater concern about bullying than the internal employees? only future research can reveal the answers to these questions. also, the literature indicates that bullying negatively affects those who witness this behaviour, and in our study it is possible that these participants also suffered while observing the bullying. however, this is a question for future research. that is, a future study could compare the effect of bullying in the ict workplace on the victims of this behaviour with those who witness the occurrence of bullying incidents. hopefully, this paper will pave the way for such future research. organisations have an ethical responsibility to provide employees with a safe workplace. this means that they are responsible for preventing workplace bullying, which is a major issue that impacts health and productivity of employees as well as organisations. workplace bullying has rarely been investigated in the context of the ict industry, although the speed of change and levels of stress that ict professionals experience are strong indicators of high probability of bullying (hauge et al., 2009). acknowledgments the research reported here is supported by an australian research council linkage grant (lp130100808), for which the industry partner is the australian computer society. other team investigators were professor john weckert, a/prof oliver burmeister and mr john ridge. the authors wish to thank a/prof oliver burmeister for his significant contribution to the project in general and specifically for helping with the distribution of the survey, conducting some of the qualitative interviews and his suggestions with regards to the analysis australasian journal of information systems al-saggaf & ceric 2017, vol 21, research article bullying in the australian ict workplace 14 of qualitative data. the authors also wish to thank katherine bell for editing an earlier draft of this paper. references adam, a. (2000). gender and computer ethics. acm sigcas computers and society, 30(4), 17-24. al-saggaf, y., burmeister, o. k., & weckert, j. (2016). perceptions of incompetence in the ict workplace. acm sigcas computers and society, 45(3), 11-17 al-saggaf, y. & burmeister, o.k. (2013). a survey of australian ict professionals’ perceptions regarding the most common ethical problems they face in the workplace. in m. warren (eds.). proceedings of the seventh aice conference (pp. 43-48). rmit, melbourne, australia. december 3, 2013. bjorkqvis, k., osterman, k., & hjelt-back, m. (1994). aggression among university employees. aggressive behavior, 20(3), 173-184. bowling, n. a., & beehr, t. a. (2006). workplace harassment from the victim's perspective: a theoretical model and meta-analysis. journal of applied psychology, 91(5), 998. claybourn, m. (2011). relationships between moral disengagement, work characteristics and workplace harassment. journal of business ethics, 100(2), 283-301. cowie, h., jennifer, d., neto, c., angula, j. c., pereira, b., del barrio, c., & ananiadou, k. (2000). comparing the nature of workplace bullying in two european countries: portugal and the uk. transcending boundaries: integrating people, processes and systems, 128-133. crawshaw, l. (2009). workplace bullying? mobbing? harassment? distraction by a thousand definitions. consulting psychology journal: practice and research, 61(3), 263. cuyper, n., notelaers, g., & witte, h. (2009). transitioning between temporary and permanent employment: a two‐wave study on the entrapment, the stepping stone and the selection hypothesis. journal of occupational and organizational psychology, 82(1), 67-88. einarsen, s., & raknes, b. i. (1997). harassment in the workplace and the victimization of men. violence and victims, 12(3), 247-263. einarsen, s., & skogstad, a. (1996). bullying at work: epidemiological findings in public and private organizations. european journal of work and organizational psychology, 5(2), 185-201. griffiths, m., & moore, k. (2010). 'disappearing women': a study of women who left the uk ict sector. journal of technology management & innovation, 5(1), 95-107. gutek, b. a., & morasch, b. (1982). sex‐ratios, sex‐role spillover, and sexual harassment of women at work. journal of social issues, 38(4), 55-74. harvey, m. g., heames, j. t., richey, r. g., & leonard, n. (2006). bullying: from the playground to the boardroom. journal of leadership & organizational studies, 12(4), 1-11. hauge, l. j., skogstad, a., & einarsen, s. (2009). individual and situational predictors of workplace bullying: why do perpetrators engage in the bullying of others? work & stress, 23(4), 349-358. hoel, h., cooper, c. l., & faragher, b. (2001). the experience of bullying in great britain: the impact of organizational status. european journal of work and organizational psychology, 10(4), 443-465. australasian journal of information systems al-saggaf & ceric 2017, vol 21, research article bullying in the australian ict workplace 15 jóhannsdóttir, h. l., & ólafsson, r. f. (2004). coping with bullying in the workplace: the effect of gender, age and type of bullying. british journal of guidance & counselling, 32(3), 319-333. keashly, l., & neuman, j. h. (2004). bullying in the workplace: its impact and management. employee rights and employment policy journal, 8, 335-523. kelloway, k., bryant, m., buttigieg, d., & hanley, g. (2009). poor bullying prevention and employee health: some implications. international journal of workplace health management, 2(1), 48-62. kivimäki, m., virtanen, m., vartia, m., elovainio, m., vahtera, j., & keltikangas-järvinen, l. (2003). workplace bullying and the risk of cardiovascular disease and depression. occupational and environmental medicine, 60(10), 779-783. leymann, h. (1993). mobbing–psychoterror am arbeitsplatz und wie man sich dagegen wehren kann (mobbing–psychoterror in the workplace and how one can defend oneself). reinbeck: rowohlt. leymann, h., & gustafsson, a. (1996). mobbing at work and the development of posttraumatic stress disorders. european journal of work and organizational psychology, 5(2), 251-275. lucas, r., & mason, n. (2008). a survey of ethics and regulation within the ict industry in australia: ethics education. journal of information, communication and ethics in society, 6(4), 349-363. lucas, r., & weckert, j. (2008). regulation in the ict industry. canberra: centre for applied philosophy and public ethics, australian national university. mason, e. s., & mudrack, p. e. (1996). gender and ethical orientation: a test of gender and occupational socialization theories. journal of business ethics, 15(6), 599-604. mudrack, p. e., & mason, e. s. (2013). ethical judgments: what do we know, where do we go? journal of business ethics, 115(3), 575-597. nielsen, m. b., skogstad, a., matthiesen, s. b., glasø, l., aasland, m. s., notelaers, g., & einarsen, s. (2009). prevalence of workplace bullying in norway: comparisons across time and estimation methods. european journal of work and organizational psychology, 18(1), 81-101. notelaers, g., de witte, h., & einarsen, s. (2010). a job characteristics approach to explain workplace bullying. european journal of work and organizational psychology, 19(4), 487-504. notelaers, g., vermunt, j. k., baillien, e., einarsen, s., & de witte, h. (2011). exploring risk groups workplace bullying with categorical data. industrial health, 49(1), 73-88. oleksy, w., just, e., & zapedowska‐kling, k. (2012). gender issues in information and communication technologies (icts). journal of information, communication and ethics in society, 10(2), 107-120. ortega, a., høgh, a., pejtersen, j., & olsen, o. (2009). prevalence of workplace bullying and risk groups: a representative population study. international archives of occupational and environmental health, 82(3), 417-426. quigley, b. z., & mohamad, s. (2008). gender, the propensity for ethical behavior and the bottom line. journal of business and behavioral sciences, 19(20), 46-57. rodríguez‐muñoz, a., moreno‐jiménez, b., sanz vergel, a. i., & garrosa hernández, e. (2010). post‐traumatic symptoms among victims of workplace bullying: exploring gender differences and shattered assumptions. journal of applied social psychology, 40(10), 2616-2635. australasian journal of information systems al-saggaf & ceric 2017, vol 21, research article bullying in the australian ict workplace 16 salin, d. (2001). prevalence and forms of bullying among business professionals: a comparison of two different strategies for measuring bullying. european journal of work and organizational psychology, 10(4), 425-441. salin, d. (2003a). the significance of gender in the prevalence, forms and perceptions of workplace bullying. nordiske organisasjonsstudier, 5(3), 30-50. salin, d. (2003b). ways of explaining workplace bullying: a review of enabling, motivating and precipitating structures and processes in the work environment. human relations, 56(10), 1213-1232. saunders, p., huynh, a., & goodman-delahunty, j. (2007). defining workplace bullying behaviour professional lay definitions of workplace bullying. international journal of law and psychiatry, 30(4–5), 340-354. sheehan, m., mccarthy, p., barker, m., & henderson, m. (2001). a model for assessing the impacts and costs of workplace bullying. article presented at the standing conference on organizational symbolism (scos), dublin, uk. skinner, t. c., peetz, d., strachan, g., whitehouse, g., bailey, j., & broadbent, k. (2015). selfreported harassment and bullying in australian universities: explaining differences between regional, metropolitan and elite institutions. journal of higher education policy and management, 37(5), 558-571. standen, p., paull, m., & omari, m. (2014). workplace bullying: propositions from heider’s balance theory. journal of management & organization, 20(6), 733-748. valentine, s., & page, k. (2006). nine to five: skepticism of women’s employment and ethical reasoning. journal of business ethics, 63(1), 53-61. vartia, m. (1996). the sources of bullying–psychological work environment and organizational climate. european journal of work and organizational psychology, 5(2), 203-214. vartia, m. a.-l. (2001). consequences of workplace bullying with respect to the well-being of its targets and the observers of bullying. scandinavian journal of work, environment & health, 27(1), 63-69. wilford, s. h., & wakunuma, k. j. (2014). perceptions of ethics in is: how age can affect awareness. journal of information, communication and ethics in society, 12(4), 270283. young, k. s., & case, c. j. (2009). computer ethics: gender effects and employee internet misuse. issues in information systems, 10(2), 598-603. zapf, d. (1999). organisational, work group related and personal causes of mobbing/bullying at work. international journal of manpower, 20(1/2), 70-85. copyright: © 2017 al-saggaf & ceric. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems al-saggaf & ceric 2017, vol 21, research article bullying in the australian ict workplace 17 paper title (paper title style) australasian journal of information systems miller 2019, vol 23, research on applied ethics (cybersecurity) machine learning, ethics and law 1 machine learning, ethics and law seumas miller charles sturt university tu delft university of oxford semiller@csu.edu.au abstract recent revelations concerning data firm cambridge analytica’s illegitimate use of the data of millions of facebook users highlights the ethical and, relatedly, legal issues arising from the use of machine learning techniques. cambridge analytica is, or was – the revelations brought about its demise a firm that used machine learning processes to try to influence elections in the us and elsewhere by, for instance, targeting ‘vulnerable’ voters in marginal seats with political advertising. of course, there is nothing new about political candidates and parties employing firms to engage in political advertising on their behalf, but if a data firm has access to the personal information of millions of voters, and is skilled in the use of machine learning techniques, then it can develop detailed, fine-grained voter profiles that enable political actors to reach a whole new level of manipulative influence over voters. my focus in this paper is not with the highly publicised ethical and legal issues arising from cambridge analytic’s activities but rather with some important ethical issues arising from the use of machine learning techniques that have not received the attention and analysis that they deserve. i focus on three areas in which machine learning techniques are used or, it is claimed, should be used, and which give rise to problems at the interface of law and ethics (or law and morality, i use the terms “ethics” and “morality” interchangeably). the three areas are profiling and predictive policing (saunders et al. 2016), legal adjudication (zeleznikow, 2017), and machines’ compliance with legally enshrined moral principles (arkin 2010). i note that here, as elsewhere, new and emerging technologies are developing rapidly making it difficult to predict what might or might not be able to be achieved in the future. for this reason, i have adopted the conservative stance of restricting my ethical analysis to existing machine learning techniques and applications rather than those that are the object of speculation or even informed extrapolation (mittelstadt et al. 2015). this has the consequence that what i might regard as a limitation of machine learning techniques, e.g. in respect of predicting novel outcomes or of accommodating moral principles, might be thought by others to be merely a limitation of currently available techniques. after all, has not the history of ai recently shown the naysayers to have been proved wrong? certainly, ai has seen some impressive results, including the construction of computers that can defeat human experts in complex games, such as chess and go (silver et al. 2017), and others that can do a better job than human medical experts at identifying the malignancy of moles and the like (esteva et al. 2017). however, since by definition future machine learning techniques and applications are not yet with us the general claim that current limitations will be overcome cannot at this time be confirmed or disconfirmed on the basis of empirical evidence. keywords: applied ethics; machine-learning; law australasian journal of information systems miller 2019, vol 23, research on applied ethics (cybersecurity) machine learning, ethics and law 2 1 profiling and predictive policing profiling of potential suspects and geographical locations for crimes has been around for some time (schauer 2003). for instance, law enforcement resources have not only been directed at persons ‘known to the police’, such as persons with past convictions, but at persons who have features that correlate with the perpetration of crimes of a certain type, e.g. a gambling habit, bad debts, a recent divorce etc. might correlate with certain forms of white-collar crime. again, police resources can be directed to crime hotspots identified not simply on the basis of past crimes committed at that location, but on the basis of possession of features statistically correlated with crimes of the relevant type and, therefore, believed to make a location crimeprone, e.g. the location has a number of tourist attractions and is close to a main road and to train and bus stations. however, the advent of big data and machine-learning has given significant additional impetus to the use of profiling in law enforcement and, thereby, to socalled predictive policing (saunders et al 2016). offender profiling can involve making inferences from characteristics of offences to characteristics of offenders, and thereby identify the perpetrator of a particular crime or set of crimes. of course, in a general sense, police investigators have always been engaged in this kind of inference making. however, contemporary policing is able to use a range of theories, research methods and statistical tools to enhance traditional police practice, including machine learning techniques. many police services have access to crime analysts and specialist police staff who analyse crime according to specific criteria, for example, modus operandi, behaviour exhibited by offender, and links to other crimes and offenders. moreover, there are academics researching serious crime who make their findings available to police investigators, e.g. psychological research on different categories of offender. historically, the crimes that have involved the use of offender profiling have mainly been rape and motiveless, or sexually motivated, murder cases. however, profiling is not restricted to these crimes. indeed, high volume crime, such as burglaries and car theft, are in many respects more amenable to profiling techniques and, in particular, to profiling techniques using big data and machine learning, than murder or rape. for one thing, the data bases of murder and rape cases are comparatively small. the comparatively small number of terrorist cases in data bases is also an impediment to the use of profiling techniques dependent on big data and machine learning in this area of law enforcement. the profiling process is greatly facilitated by databases of offenders (rapists, paedophiles, etc.) in which the data held is detailed and specific, e.g. it describes their modus operandi and provides psychological/behavioural profiles. here, as elsewhere, the integration or linking of databases in different jurisdictions is also extremely helpful especially, as in the us, where there are a large number of jurisdictions many of which are very small. profiling in law enforcement has two main types. the first type involves developing a profile or set of characteristics of a category of persons, for instance, a typical police officer at risk of corruption, based on generalizations from the past behaviour of persons belonging to this category. so a set of characteristics might be developed, including such things as numbers of complaints, associating with known criminals, gambling habits, substance abuse, financial problems, and operating in a high-risk area such as drug law enforcement, and consolidated so as to constitute the profile of a typical police officer engaged in corruption. once the profile australasian journal of information systems miller 2019, vol 23, research on applied ethics (cybersecurity) machine learning, ethics and law 3 is constructed, the internal affairs investigators can, at least in theory, monitor individuals with that profile in their preventative anti-corruption strategies. however, big data and machine learning can take this first kind of profiling much further. obviously, as already mentioned, the size of the data base is of great importance in the development of profiles; so the establishment of detailed electronic databases and the integration of these data bases across multiple jurisdictions greatly facilitates profiling. crucially, however, machine learning techniques enable profiles to be significantly refined. for machine learning techniques can detect additional correlations and, thereby, generate new features of the pre-existing profile. importantly, these new resulting ‘features’ are not known or intuited by analysists in advance – unlike the pre-programmed features mentioned above in traditional profiling – and, indeed, even once known they might not be intuitively obvious. rather it is a matter of there being an algorithmically-based correlation with all those persons who fit the original profile. the second type of profiling involves developing a profile or set of characteristics of the person who has committed the crime being investigated but whose identity is unknown; for instance, the serial rapist – whoever that person is who raped mary, betty, and jane. the profile is based on generalizations from the modus operandi, past behaviour, and so on of this offender and of like offenders. as with the first type of profiling, machine learning techniques can be utilized to refine the profile. once the profile is constructed the investigators can access their databases, or otherwise look for the particular person that fits the profile, and do so for the purpose of solving the crime in question. recent developments in communications and information technology, including the creation and integration of large databases, high-speed, long-distance accessing and communication of content, for instance, via the internet, and, more recently, the use of machine learning techniques, have enormously facilitated profiling in both of these two forms. however, the existence and possibility of widespread profiling in law enforcement has raised a range of ethical problems. one issue or set of issues concerns privacy, as we saw above in relation to cambridge analytica. do the databases upon which machine learning techniques are applied consist of personal information or confidential information to which those using these techniques do not have a right or are otherwise not morally or legally entitled to access? there are, of course, complications here in relation to what counts as personal information. the content of telephone calls, email etc. is typically regarded as personal or confidential information, but what of metadata; data concerning the caller/called, duration of call etc.? some have argued that this is not personal or confidential any more than the sender and receiver’s name on a parcel sent through the postal service are items of personal data. however, a large bank of metadata extracted from, for example, from a person’s phone calls, emails etc., can enable a detailed picture of that person’s associates, movements and so on; a picture sufficiently detailed to count as an infringement of their privacy. again, is the purpose to which the machine learning applications are a means a legitimate purpose? presumably, the purposes involved in predictive policing are for the most part legitimate law enforcement purposes, such as reducing crime, whereas cambridge analytica’s purposes – and certainly the purpose of a foreign power to influence an election outcome – are not. let us, then, assume that profiling undertaken by law enforcement is in the service of legitimate purposes and that the data upon which the profiling depends is not in any obvious way personal or confidential in character, e.g. the data is either publicly available or is australasian journal of information systems miller 2019, vol 23, research on applied ethics (cybersecurity) machine learning, ethics and law 4 appropriately anonymized personal data.1 are there other ethical concerns? arguably, there is a further ethical concern or, at least, need for moral justification, in respect of profiling in our first sense, that is, where there is no specific (actual or reasonably suspected) past, imminent or planned crime under investigation. for example, in the case of police officers who are not known to have committed any crime, what is the justification for monitoring their behavior – other than for ordinary work performance purposes? are police any different from ordinary citizens in this regard? it might be argued that, given their position of trust and the fact that they have extensive powers of arrest and use of lethal force not possessed by ordinary citizens, that such monitoring is morally justified (kleinig 1996; miller and gordon 2014: 201223). moreover, their occupational role as a police officer is one freely chosen. however, these arguments justifying the profiling of police officers are not available in the case of ordinary citizens, even if profiling of citizens might serve the ultimate purpose of reducing crime. speaking generally, there ought to be knowledge that a crime has been, or is about to be, committed or is being planned, and reasonable suspicion that citizen smith has or will commit the crime in question (absent police intervention), prior to any process of monitoring. the fact that smith has committed such crimes in the past might in some cases constitute reasonable suspicion, as might the fact that smith had the ability and opportunity to commit an already committed, or about to be committed, crime. however, in the absence of police focus on any specific (actual or reasonably) past, imminent or planned crime, (e.g. the murder of jones), or ongoing spate of crimes, (e.g. recent burglaries in a specific location), surely the mere fact that some citizens might have the profile of individuals who tend to commit a certain crime type is not a sufficient justification for generating such profiles, determining who among the citizenry has these profiles and monitoring these citizens. such invasive law enforcement practices are evidently inconsistent with fundamental principles underpinning liberal democracy and, in particular, the individual’s right to freedom from state interference absent prior evidence of violation of its laws. in a liberal democratic state, it is generally accepted, the state has no right to seek evidence of wrongdoing on the part of a citizen whose actions have not otherwise reasonably raised suspicion of unlawful behavior. at the very least in a democracy these law enforcement practices would need to be consented to by the citizenry via the democratic process. these ethical problems with profiling are compounded in the case of the use of machine learning techniques by the so-called ‘black box’ issue. in the case of the profiles generated by machine learning the algorithmic based correlations may not be known or understood by either the citizens or law enforcement. so citizen smith might be being monitored without either smith or the police monitoring him knowing at any point what a key part of the justification for this monitoring is, i.e. what features he possesses that make him, in effect, an object of suspicion. frederick schauer’s work can be viewed, at least in part, as an attempt to provide an antidote to many of the fears that have arisen in relation to the practice of profiling. by schauer’s lights as an advocate of the morality of decisions based on generalizations, profiling is already an acceptable and ubiquitous practice. for example, he discusses the case of sokolow who was searched by customs officials at an airport and found to possess drugs (schauer 2003: 172). he was searched because he fitted the profile of a drug courier and this was taken to constitute reasonable grounds for suspicion. sokolow argued in court that fitting a profile did not 1 there are potential ethical problems with the latter which would take me too far afield to go into here. australasian journal of information systems miller 2019, vol 23, research on applied ethics (cybersecurity) machine learning, ethics and law 5 constitute reasonable grounds for suspicion, but he lost the case. schauer claims that, “the issue is not about profiling at all, for profiling is inevitable” and “profiling is largely unobjectionable” (schauer 2003: 174). whatever the value of schauer’s discussion of profiling understood in a general sense, it is unconvincing in relation to profiling that utilizes machine-learning techniques. for one thing, as just mentioned, the justification for the elements of the profile might not be known to law enforcement. for another, on many influential accounts of judgment schauer distorts the nature and function of discretionary judgment; discretionary judgment is not, many theorists would hold, simply personal, unscientific profiling. rather, discretionary judgment is called for in relation to matters that have an inherent particularity (see next section for instances of this). by the lights of some theorists (harre and madden 1975), such judgements include ones involved in establishing causation in the law. on this kind of view, causation is sharply distinguished from correlation by virtue of the inherent particularity of the former; causal powers are powers possessed by powerful particulars rather than mere regularities in behaviour. accordingly, the role of discretionary judgment may well be to supplement profiling – including profiling utilizing machine learning techniques in order to rule out or rule in instances where the profiling generates manifestly absurd outcomes, e.g. citizen jones fits the profile but has a rock-solid alibi. finally, it could be argued that schauer’s conception does not give due weight to the dangers of profiling, including that involving machine learning techniques, e.g. the danger of discriminatory algorithms. for instance, racial profiling may entrench existing racist attitudes and may generate over-policing leading to policecommunity tension which in turns obstructs law enforcement. 2 machine learning and legal adjudication another area in which machine learning techniques are being used is in the legal quagmire of divorce proceedings (zeleznikow 2017). although separation and divorce can be amicable, they can also be monumentally expensive as a result of legal fees. new machine learning software now attempts to predict the settlement outcome, based on a huge case history and relatively clear-cut legal criteria for settlement. if the protagonists accept the prediction, a lowcost agreement can be achieved. accordingly, the utilization of machine learning techniques in areas of the law involving a high volume of similar types of case and relatively clear-cut legal rules, such as divorce proceedings, may well be hugely beneficial. however, here as elsewhere, there is a need for caution. consider the following somewhat non-standard case.2 rachel had risen to vicepresident of a profitable company, leaving not enough time for looking after her 6 year old son. now she was hit a double whammy. her partner had found a new lover and wanted a split, while her company had merged with a larger company and she was out of a job. she would get back again, but it would take a while to find something of comparable level. but she was happy to have more time for her son, and, finances would not be too bad for a few years, given a reasonable settlement with custody. unfortunately, the software predicted she would not get custody! as noted above, predicting future legal outcomes of cases based on past outcomes assumes, firstly, a large data set of past cases and, secondly, that new cases have similar features to past 2 i owe this example to terry bossomaier. australasian journal of information systems miller 2019, vol 23, research on applied ethics (cybersecurity) machine learning, ethics and law 6 ones. determinations of likelihood of success in divorce proceedings are based on outcomes of past cases and weighting of criteria used in these past cases. however, past cases involve judicial errors, e.g. on the part of solicitors, barristers and magistrates. accordingly, these errors, especially if frequently made, can now enter into the predictive process. however, in doing so predictions in current cases in which adjudications do not repeat past errors might have turned out to be false predictions and, thereby, mislead those who have acted upon these predictions. moreover, the possibility of correcting these errors might well be lost if, for example, the prediction is simply accepted at face value and acted on and, in particular, acted on without going through a thorough process in which all the various arguments, evidence and so on are aired prior to a considered adjudication. moreover, complex, contested criminal cases are much less amenable to machine learning techniques than simple, high volume, legal adjudications, given the inherent particularity of many of these cases. consider the legal adjudication in the case of the serial murderer and rapist, robert black. the case warrants detailed description since the point of using it as an example is to demonstrate its inherent particularity. 3 in july 1990 a six-year-old girl, mandy wilson was abducted as she walked in her street in stow (a town in scotland close to the english border). by a stroke of immense good fortune, a neighbour saw her walk towards a parked white van and he could see her feet under the open passenger door beside those of a man. the girl’s feet vanished, the van drove off; the witness took the registration number and immediately called the police. the witness was describing the event to the girl’s father (a police officer) when the van reappeared and was immediately stopped by the police. the father found his daughter bound, gagged and stuffed in a sleeping bag behind the driver’s seat. she was terrified and had already been sexually assaulted. the driver of the van was robert black, a delivery driver who travelled throughout the uk. black was arrested, charged, pleaded guilty and was sentenced to life imprisonment. black now became the main suspect in the maxwell, hogg and harper murders, there being evident similarity between the cases in terms of mo (modus operandi) and other factors; black however declined to speak about the abductions and murders of the three girls. the investigators began a thorough and scrupulous examination of black’s movements and lifestyle between 1982 and 1990, where a key focus was on his job as a delivery driver. using work records, including wage records and fuel receipts, they built up a picture of his movements and were able to place him in the vicinity of each abduction at the appropriate time and also where the bodies had been found. the investigators also discovered an attempted abduction of a 15-year-old girl, which had failed because she had fought back and a friend had gone to her assistance. the witnesses’ descriptions of the assailant were an exact match to black. there was no forensic evidence and no admission by black – the case was built on the abovedescribed circumstantial evidence linking the various murders to one another and to black but in april 1992 the crown prosecution service elected to prosecute. importantly, for our purposes here, this was a unique case in uk criminal law. the defence argued there was no direct evidence to establish that black had committed the offences and argued that each murder should be treated separately. but the court allowed the murders to 3 this description of the case is taken from miller and gordon (2014, 127-132). australasian journal of information systems miller 2019, vol 23, research on applied ethics (cybersecurity) machine learning, ethics and law 7 be presented as a series and allowed evidence from the earlier case relating to the abduction of mandy wilson and the attempted abduction of the 15 year old. black appeared at newcastle upon tyne crown court in april 1994, when the prosecution detailed the striking similarities between the murder cases and also the attempted abduction and the actual abduction of mandy wilson. the series of murders exhibited a common modus operandi (mo) followed by black and black was linked to each of them in terms of his movements black was convicted of all the charges before the court and sentenced to life imprisonment on each one; there was a minimum term of 35 years recommended for each of the murders. in october 2011 black was further convicted of the abduction and murder in 1981, in county down northern ireland, of jennifer cardy, 9 years. this offence also relied on painstaking investigation of black’s work records and included testimony from detective chief superintendent (retired) roger orr, sio in the mandy wilson abduction, where he described black’s actions in that offence and in the maxwell/hogg/harper murders. the prosecution alleged the evidence given by mr orr amounted to “a signature for robert black and that the case of jennifer also bears that signature”. it is strongly believed that black may be responsible for a further twelve abduction and murders of young girls, in the uk, france and holland. to date he has declined to assist any police enquiry. to summarise: robert black was a serial rapist and murderer but there was only circumstantial evidence in each of the murder cases. accordingly, the prediction in each case considered on its own– including predictions using machine learning techniques would have been ‘not guilty’. however, there was an evidential link between each of these cases: the modus operandi of black. the prosecution argued that there was a distinctive ‘signature’ mo in each case and that this mo was used in the abduction case for which he was convicted as well as the murder cases for which there was insufficient evidence absent recourse to the signature mo. the point to be stressed here is not that there was a pattern i.e. the mo, although the existence of a pattern was a necessary condition for the legal outcome. nor is it that machine learning was not necessary to establish this pattern, although obviously machine learning was not required to discover a pattern in a handful of murder-rape cases. rather the point to be stressed is that a discretionary, and inherently particular, legal decision was made; a decision that allowed for the first time an evidential relationship between different cases to be useable in a single discrete case. in short, the robert black case was at the time unique and unique in a manner that made it unable to be predicted on the basis of adjudications in past cases of serial murder and rape. accordingly, the legal adjudication in the black case would have been immune to prediction on the basis of machine learning techniques. the more general point is that appropriate legal adjudications in complex cases may have an inherent particularity that renders them immune to prediction on the basis machine learning techniques. therefore, there are evidently limitations to the utilization of machine learning techniques in legal adjudication and attempts to exceed these limitations may well lead, not simply to error, but to injustice, whether in the form of punishing the innocent or failing to punish the guilty. 3 machines and compliance with legally enshrined moral principles evidently, the introduction of autonomous cars on the streets of australia and elsewhere is imminent. autonomous cars need to be able to comply with the road rules, e.g. stop at red australasian journal of information systems miller 2019, vol 23, research on applied ethics (cybersecurity) machine learning, ethics and law 8 lights and zebra crossings, but they also, indeed simultaneously in the case of many road rules, need to be able to comply with moral principles enshrined in laws, e.g. avoid running over pedestrians (as happened recently in the well-publicised case of a woman killed by a selfdriving car in arizona). this raises the issue of the possibility of machines complying with moral principles and, in particular, legally enshrined moral principles.4 ron arkin has been undertaking research with a view to building machines that could fight wars in accordance with the rules of war (arkin 2010). the actual conduct of war is governed by moral principles enshrined in international law (the so-called jus in bello of just war theory), notably the principles of (1) military necessity, (2) proportionality and (3) discrimination. accordingly, arkin has sought to demonstrate how weaponized autonomous robots could not only fight wars, but in doing so comply with the principles of military necessity, proportionality and discrimination. as will become evident, these principles are quite unlike the precisely defined rules and precisely defined and limited contexts of application of rules in games, such as chess and go, in which, as mentioned above, machines have had notable success in recent times. thus the rules of chess precisely define what moves the chess pieces, (e.g. pawns, bishops) can and cannot make, and precisely define also the context of application, i.e. the chessboard and the configured chess pieces (e.g. pawns, bishops etc.). they also precisely define what counts as winning, e.g. check-mating the king. for the moment i note that by contrast with the rule of chess, the ius in bello moral principles of war are not welldefined, (e.g. what counts as a disproportionate use of force?) and have to be applied in multiple, diverse and shifting military contexts, (e.g. conventional theatres of war and counterterrorism operations, war at sea and war in the jungle) that are typically not precisely defined or delimited, (e.g. terrorism in civilian areas of failed states). moreover, what counts as winning a war – and, therefore, what the definition of military necessity is (see below) is not precisely defined either, e.g. withdrawal of enemy forces from the defending nation’s territory, devastating the enemy’s cities with atomic bombs etc. importantly, unlike in the typical case of the application of the law by police officers, these principles apply at the collective level, as opposed to merely at the individual level. so the context of any or, at least, most applications of these principles are multi-levelled and applies to organizations and not merely to individuals. what do i mean by the collective level(s)? the principle of military necessity, in particular, but also the principles of proportionality and discrimination, apply at various collective levels, such as at the level of a battle or at the level of an ongoing war fought by a military organization. accordingly, it might be militarily necessary to bomb a munitions dump in order to win a battle and it might be, in turn, militarily necessary to win the battle in order to win the war. the contrast here is with the individual level, i.e. the level of an individual combatant’s lethal action considered as one-off selfcontained action. at the individual level, it might be necessary for private smith to shoot dead an approaching enemy combatant who will otherwise kill jones. accordingly, the context for the application of these moral principles is a multi-level (individual and collective) context. let me explain further. in essence, the principle of military necessity ultimately pertains to the long term, necessarily underspecified collective end of winning the war which generates in turn a nested, dynamic, series of short and medium term collective ends, such as winning particular battles or 4 an earlier version of a number of the arguments in this section appeared in miller (2015). australasian journal of information systems miller 2019, vol 23, research on applied ethics (cybersecurity) machine learning, ethics and law 9 firefights. these short and medium term collective ends are means to the long term collective end of winning the war, albeit means in need of further specification, adjustment or even abandonment in the light of the responses to them of the enemy armed forces. accordingly, the principle of military necessity is to be understood, firstly, in short/medium/long term means/end, i.e. diachronic, terms. something is necessary in this sense if, comparatively speaking, it is both an efficient and effective means to an end and there is no obviously superior means available. if it is the only means, then it is strictly necessary. however, this is frequently not the case and so to this extent ‘necessity’ is correspondingly less strict. secondly, the strength of the necessity to deploy a given quantum of lethal military force in (say) the context of a battle turns in large part on the moral weight to be accorded to the winning of that battle in light of its likely contribution to the ultimate (necessarily underspecified) collective end of winning the war (and, of course, the (somewhat indeterminate) moral weight to be attached to the latter). in the case of a crucial battle in the context of a war of collective selfdefence, the military necessity to deploy a large quantum of lethal military force might be both strong (there is much at stake) and strict (it is the only available means). thirdly, and notwithstanding the cooperative interdependence of their military actions, within a given armed force the principle of necessity is applied by multiple different military commanders within a given armed force, and each applies (or fails to apply) the principle on a given occasion on the basis of his own discretionary judgment, e.g. a bomber commander deciding whether to bomb a munitions factory in the vicinity of civilians, the us president deciding to drop atomic bombs on hiroshima and nagasaki. what of the principles of proportionality and discrimination? roughly speaking, the principle of discrimination forbids deliberately targeting innocent civilians and, also, foreseeably and avoidably putting their lives at unnecessary risk. the latter clause brings the principle of military necessity into play; a risk to civilians is unnecessary if the use of lethal military force which constitutes this risk is not militarily necessary. so the principles of military necessity and discrimination are conceptually (albeit, evidently nonalgorithmically) interdependent. an important consequence of this is that the context for the application of the principle of discrimination is also multi-level; it applies at both the individual and collective levels. moreover, since these levels are interconnected by virtue of nested collective ends, the application of the principle of discrimination necessarily involves taking into account the risks to civilians at these various levels and (possibly) adjudicating between them. for example, pursuing tactic a (aerial bombing) to realize the collective end of winning a battle might lead to many more civilian casualties in this present battle than pursuing tactic b (taking and holding ground without aerial bombing). however, pursuing a might be a more efficient and effective means of decisively winning the battle (because, say, of the much heavier enemy casualties inflicted prior to their retreat) and might, therefore, reduce the number of future civilian casualties in future battles joined in further pursuit of the collective end of winning war. what of the principle of proportionality? the principle of proportionately arises in contexts in which both the principle of military necessity and the principle of discrimination are applicable. roughly speaking, it requires that that the quantum of (unintended) civilian deaths resulting from the deployment of lethal military force should not be disproportionate to the strategic (and derivable moral) weight of collective military ends to be realized by that deployment. as such, the principle of proportionality is conceptually (but again, evidently non-algorithmically) interdependent with both the principle of military necessity and the principle of discrimination. accordingly, australasian journal of information systems miller 2019, vol 23, research on applied ethics (cybersecurity) machine learning, ethics and law 10 the context for the application of the principle of proportionality is also multi-level; it applies at both the individual and collective levels. what of machine compliance with the legally enshrined moral principle of military necessity, discrimination and proportionality? the weaponized autonomous robots in question can detect and respond to features of their environment and in many cases they have impressive storage/retrieval, calculative capacities and, importantly for our purposes here, the ability to learn new rules and finesse prior rules based on past correlations. however, the argument at this point is based on the assumptions, firstly, that moral principles, such as military necessity, proportionality and discrimination, can be reduced to sharply defined rules, contexts of application and outcomes to be aimed at and, secondly, that these rules, contexts of application and outcomes to be aimed at are such that they can be programmed in to computers. an initial problem is that while such robots are sensitive to physical features of the environment they are not sensitive to moral properties. after all, computers do not care about anyone or anything (including themselves), and cannot recognise moral properties, such as courage, moral innocence, moral responsibility, sympathy or justice. therefore, computers cannot act for the sake of moral ends or principles understood as moral in character, such as the principle of discrimination. given the non-reducibility of moral concepts and properties to physical ones, at best computers can be programmed to comply with some non-moral physical proxy for moral requirements. the proxy for ‘do not intentionally kill morally innocent human beings’ might be ‘do not fire at bipeds if they are not carrying a weapon or they are not wearing a uniform of the following description’. each moral principle needs to be expressible in a sharply defined rule couched in purely physical descriptive terms. given the non-reducibility of the moral to the physical or, at least, the lack of reliable, precise, detailed correlations (e.g. innocent civilian is a vague notion and terrorists consistently seek to thwart attempts to identify them as combatants), this is extremely doubtful especially in respect of relatively vague and quite general principles, such as the principle of discrimination. a second objection pertains to the ends in play and, in particular, the ends implicit in the principle of military necessity. putting civilian lives at risk may be justified if it is militarily necessary i.e. in the service of the end of winning the war. but, as already mentioned, the content of the notion of winning the war is underspecified. winning the war is not welldefined in the manner of winning the chess game. rather, as is the case with most large-scale human enterprises taking place over extended periods the ultimate and proximate collective ends are underspecified and only become fully specified in the course of undertaking the enterprise; moreover, being collective ends, the cooperating individual who pursue them do so under somewhat different descriptions and the process of further specification is itself collaborative. accordingly, the collective end of a war or a battle cannot readily be programmed into a computer in the manner in which a geographical located destination can be programmed into a self-driving car. a more holistic objection to arkin’s project arises from combining a number of the aboveidentified difficulties attaching to compliance, including compliance assisted by machine learning techniques, with all three of these conceptually connected ius in bello principles. let us remind ourselves what these difficulties are: (1) the necessarily underspecified nature of the collective ends constitutive of waging war prior to their realization; (2) the moral importance of realizing these collective ends, individually and in aggregate – an importance derivable australasian journal of information systems miller 2019, vol 23, research on applied ethics (cybersecurity) machine learning, ethics and law 11 ultimately and in large part from the moral benefits of successful collective self-defence and the moral costs of failure; (3) the conceptual (but evidently non-algorithmic) interdependence of the principles of military necessity, discrimination and proportionality; (4) the interplay between the applications of these principles at the various different collective levels and at the individual level, and; (5) the application of these principles in the context of an evolving, dynamic and diachronic process of responses to, and counter-measures against, the enemy. in short, it is not simply a matter of programming in each precisely defined rule, precisely defined outcome to be aimed at and a precisely defined set of contexts of application, and then ‘pushing the start button’. nor it is even a matter of specifying a set of independent rules (since the principles in question are interdependent), together with a meta-rule specifying which rule is to take precedence over which when there is a conflict, and mechanically applying the resultant set of rules in the relevant contexts. moreover, while machine learning techniques may well enable a computerized robot to learn precisely defined rules, such as chess rules, on the basis of a data set comprised of instances of past compliance with those rules, and learn and further refine successful tactics and strategies on the basis of relevant chess data sets, matters are somewhat different when it comes to the application of the ius in bello principles in war. as has already been stressed, these moral principles, outcomes to be aimed at, and contexts are far from being precisely defined. however, there are further problems. where are the large data sets comprised of instances of compliance and non-compliance to these principles to be found? perhaps from past wars? unfortunately, past wars are extraordinarily diverse in respect of the weapons and tactics used, and the outcomes aimed at; moreover, the actual outcomes of the use of these weapons and tactics in the service of these aims varied enormously depending on the diverse physical, psychological and social contexts in which these conflicts took place. further, and in part because of these differences in weaponry, tactics, aims and contexts, every war is significantly different from every other war with respect to the application by combatants and their leaders of moral principles. and, course, human beings interpret moral principles in different ways, can choose to apply moral principles to different degrees or, indeed, to ignore them. for instance, the principle of discrimination is applied with much greater diligence by some sides in some wars than by other sides or in other wars. the upshot of this is that there will need to be a heavy reliance on the human beings in charge of this r&d project in morally and legally compliant weaponized autonomous robots to interpret the ius in bello moral principles, to devise proxy physical rules, to identify or design data sets of incidents involving compliance and non-compliance, and so on. in short, what looked initially to be a project in the natural sciences governed by strict principles of deduction, induction etc., has morphed into a project that is heavily reliant on the prior moral judgements of the researchers themselves. notwithstanding the above, let us assume that suitable proxies for the ius in bello principles are found and that an appropriate data base is available to facilitate machine learning processes. let us now turn more directly, and briefly, to some of the moral challenges confronted by our (supposedly) morally and legally compliant weaponized autonomous robots. appropriate application of the principle of military necessity requires reasonably reliable, morally informed, radically contextually dependent, judgments at the various collective levels, as well as at the individual level. however, given the nested character of the individual and collective ends in play, their necessarily underspecified content, and the need to be responsive australasian journal of information systems miller 2019, vol 23, research on applied ethics (cybersecurity) machine learning, ethics and law 12 to the actions, including counter-measures, of enemy combatants and their leaders, there is a constant interplay between the various collective and individual levels. further, the various applications of the principles of necessity, proportionality and discrimination are interdependent, e.g. the application of the principle of proportionality depends on considerations of military necessity and vice-versa. accordingly, there is a need to adjudicate not only between the means to given ends, but also with respect to the moral weight to be accorded different competing ends at different levels. for example, the individual end to advance to assist a comrade-in-arms coming under heavy fire might compete with the collective end of one’s platoon or company to make a tactical retreat to avoid heavy losses. again, the collective military end to win firefights might be facilitated by relatively permissive rules of engagement (roe), but perhaps this end competes with the collective end to avoid large-scale casualties among civilians and the latter end is facilitated by a relatively restrictive roe. further, at the macro-collective level, the collective end of the military leadership to win an internecine war might compete with the collective end of the political leadership not to inflict losses of a magnitude that would undermine the prospects for a sustainable peace. so reasonably reliable, morally informed, multi-level contextually dependent judgments with respect to the use of lethal force must apply a principle of military necessity in a context in which there are: (i) other competing, but interdependent, moral principles, e.g. proportionality and discrimination; (ii) a complex and dynamic structure of nested individual and collective ends (many of which are necessarily underspecified) at multiple levels, and; (iii) variable possible responses on the part of ‘the enemy’ to any putative use of force. the idea that there a set of requirements for the reasonably rational and appropriately morally informed judgments with respect to the use of lethal force in each and every (or, at least, most) relevant situation in war, and that these requirements could be rendered into algorithmic form and programmed into a computer is to say the least doubtful and, in any case, is at this stage just that: an idea. 4 conclusion in this paper i have focussed on three areas in which machine learning techniques are used, or it is claimed should be used, and which give rise to problems at the interface of law and ethics. the three areas are, profiling and predictive policing, legal adjudication, and machines’ compliance with legally enshrined moral principles. in general terms, i conclude that while machine learning techniques have considerable actual and potential benefits, they also have limitations, and actual and potential ethical downsides. the use of machine learning techniques in profiling by law enforcement agencies is potentially inconsistent with the individual rights of citizens in liberal democracies not to be subject to unwarranted interference by the state. the limitations of machine learning techniques are evident, for example, in relation to compliance with legally enshrined, moral principles in the investigation and adjudication of complex crimes, and in the conduct of complex, morally charged enterprises such as war. references arkin, ronald, c, 2010. “the case for ethical autonomy in unmanned systems”, journal of military ethics, 9(4), 332-341. australasian journal of information systems miller 2019, vol 23, research on applied ethics (cybersecurity) machine learning, ethics and law 13 esteva, a., kuprel, b., novoa, r. a., ko, j., swetter, s. m., blau, h. m., and thrun, s., 2017. “dermatologist-level classification of skin cancer with deep neural networks”. nature, 542(7639), 115–118. kleinig, john, 1996. the ethics of policing. new york: cambridge university press. harre, rom, and madden, e. h., 1975. causal necessity: a theory of natural necessity. rowman and littlefield. miller, seumas and gordon, ian, 2014. investigative ethics: ethics for detectives and criminal investigators. oxford: wiley-blackwell. miller, seumas, 2015. “robopocolypse?: autonomous weapons, military necessity and collective moral responsibility” in (ed.) jai galliott and m. lotze super soldiers: the ethical, legal and social implications. london: routledge. pp. 153-166. mittelstadt, daniel, carsten stahl, bernd, and fairweather, n. ben, 2015. “how to shape a better future? epistemic difficulties for ethical assessment and the anticipatory governance of emerging technologies” ethical theory and moral practice, 18: 1027-1047. saunders, j., hunt, p., and hollywood, j. s., 2016. “predictions put into practice: a quasiexperimental evaluation of chicago’s predictive policing pilot” journal of experimental criminology, 12(3), 347–371. schauer, frederick, 2003. profiles, probabilities and stereotypes. mass.: belknap press. silver, d., schrittwieser, j., simonyan, k., antonoglou, i., huang, a., guez, a., hassabis, d. 2017. “mastering the game of go without human knowledge”. nature, 550(7676), 354. zeleznikow, john, 2017. “can artificial intelligence and on-line dispute resolution enhance efficiency and effectiveness in courts” international journal for court administration, 8(2). www.papers.ssrn.com. copyright: © 2019 miller. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems carson, farrelly, frazer & borthwick 2015, vol. 19, research on indigenous ict mediating tragedy mediating tragedy: facebook, aboriginal peoples and suicide bronwyn l carson university of wollongong bcarlson@uow.edu.au terri farrelly the echidna group: indigenous research and development consultancy drterrifarrelly@gmail.com ryan frazer university of wollongong rfrazer@uow.edu.au fiona borthwick university of wollongong fiona@uow.edu.au abstract some australian aboriginal communities experience suicide rates that are among the highest in the world. they are also, however, avid social media users—approximately 20 percent higher than the national average. this article presents some preliminary findings from a current national study, funded by the australian research council, titled aboriginal identity and community online: a sociological exploration of aboriginal peoples’ use of online social media. the purpose of the study is to gain insights into how aboriginal people utilise and interact on social media, and how these technologies can assist with suicide prevention strategies. it found that aboriginal people are engaging with facebook to both seek and offer help for issues relating to suicide and self-harm. an existing continuum of suicide prevention strategies was evident—from light emotional support to direct suicide intervention involving health services. these strategies can be leveraged to implement effective and appropriate suicide prevention programs. keywords: social media; indigenous; aboriginal; social networking; suicide 1 introduction in this article we present some preliminary findings from a current national study, funded by the australian research council, titled aboriginal identity and community online: a sociological exploration of aboriginal peoples’ use of online social media. the purpose of the study is to gain insights into the ways in which australian aboriginal people utilise and interact on social media. results demonstrate that aboriginal people are avid social media users, that they use social media for a range of activities, including those associated with mental health and well-being (carlson 2014; carlson and frazer 2015). use of social media is clearly not a peculiarity among some aboriginal people but rather a daily activity for the vast majority, with facebook being the most popular site (carlson 2013). while there is a growing body of research examining suicide and social media (see for example, robinson et al. 2014), there is a paucity of studies that focus on links between aboriginal people’s use of social media and suicide. given the high rates of both suicide and social media use in aboriginal communities (the elders report 2014:7), this is surprising. our research findings revealed that aboriginal people are engaging with facebook to both seek and offer help for issues relating to suicide and self-harm. an existing continuum of suicide prevention strategies was evident—from light emotional support to direct suicide intervention involving health and emergency services. the results thus demonstrate that online social technologies, such as facebook, offer aboriginal users new pathways to engage in help-seeking and help 1 australasian journal of information systems carson, farrelly, frazer & borthwick 2015, vol. 19, research on indigenous ict mediating tragedy giving behaviours. we propose these online practices suggest social media presents a potentially efficacious platform for developing culturally appropriate suicide interventions and prevention programs. 1.1 background 1.1.1 aboriginal suicide until the 1970s suicide among aboriginal people was a relatively rare phenomenon (elliottfarrelly 2004). now, however, some aboriginal communities experience rates of suicide and self-harm that are up to 40 percent higher than the general australian population (carlson 2014), making them among the highest in the world (the elders report 2014:7). reducing suicide and suicidal behaviour among aboriginal people is now a public health priority for all australian governments. 1.1.2 differing epidemiologies, methods, risk factors and aetiological theories key research conducted into aboriginal suicide and self-harm behaviour in northern queensland (hunter et al. 2001) and new south wales (tatz 2001) uncovered myriad factors which differentiate aboriginal suicide from non-aboriginal suicide, in terms of epidemiology, methods, risk factors and aetiological theories. these factors impact the (in)effectiveness of current prevention measures, the designs of which have largely been based on western cultural understandings of and attitudes towards suicide and self-harm behaviour and prevention. many of the mainstream social risk factors for suicide do not appear to be relevant to aboriginal people and their communities (tatz 2001). as tatz (2001) demonstrates, there exists a separate aboriginal suicidology—perhaps even suicidologies—which is further complicated by significant cultural and socio-economic variation between regions and communities. the sudden onset and increase in aboriginal suicide rates since the 1970s is often blamed on the rapid social transformation enforced upon australia’s indigenous peoples, exacerbated by social and economic disadvantage, systemic oppression and racism (hunter et al. 2001; tatz 2001; pink and allbon 2008). however, there have been few careful and systematic research studies into the causes of aboriginal suicide. it has long been held that psychiatric illness and other pathologies are present in cases of attempted and completed suicide. however, research has shown that while this may be the case in the general population, it does not appear to be applicable in aboriginal communities (tatz 2001). furthermore, while depression is often cited as the precipitating cause of aboriginal suicidality, there has been little reference to the cultural context and meanings of depression-like behaviour (reser 1991). thus tatz (2001) argues that aboriginal suicidality is not always the domain of mental health. he suggests there is little evidence of clinical depression in cases of aboriginal suicide, just as there appears to be little or no correlation between aboriginal suicide and diagnosable mental illness more broadly. rather, aboriginal suicide is often impulsive and frequently occurs in the context or aftermath of intoxication (hunter et al. 2001; tatz 2001). this research points to the need to understand aboriginal suicide in its cultural context, rather than uncritically imposing western notions of health and well-being. an emerging literature is providing a strong foundation for understanding aboriginal and torres strait islander perspectives of well-being (dudgeon and kelly 2014; ranzijn, mcconnochie and nolan 2009; westerman 2010). increasing consideration is being paid to how aboriginal people’s diverse notions of well-being dynamically interact with their ways of knowing, doing, and being (martin 2008; moreton-robinson and walter 2008); some research has paid particular attention to the role of information and technology (radoll 2012; rigney 2011). thus there is increasing academic acknowledgement of the need to establish a more robust knowledge of the diverse understandings of health aboriginal people draw on in their help-seeking and helpgiving behaviours. 2 australasian journal of information systems carson, farrelly, frazer & borthwick 2015, vol. 19, research on indigenous ict mediating tragedy 1.1.3 aboriginal people and social media research tells us aboriginal people have always been early adopters of technology, with social media being no exception (pascoe 2014). in 2010 the australian institute of aboriginal and torres strait islander studies (aiatsis) hosted a conference on ‘information technologies and indigenous communities’ (july 13th to 15th 2010). among the topics discussed, the ‘facebook and social networking’ theme claimed that sites such as facebook: …are going to be a critical part of life in your community in the future. new technology allows the creation of new linkages that have not been seen in our cultures before (2010:30). across australia, aboriginal and torres strait islander use of social media is approximately 20 percent higher than the national average and in remote communities over 60 percent of the population are active facebook users (callinan 2014). social media is transforming the way we communicate with each other and the way in which we are connected to other people at a local, regional, national and global level. certainly for aboriginal and torres strait islander people, facebook and other social media provide a site of connection to others and allow users to maintain contact and culture across both vast distances and time, increasing social connectivity (carlson 2013). the rapid advances in mobile technologies, and the uptake of these by aboriginal youth in particular, can be evidenced in many aboriginal communities (see kral 2011). while this is not to suggest there is no ‘digital divide’, it does counter any assumptions that aboriginal people may have little interest in the possibilities of technology and the online environment. kral’s (2011:5) research into how aboriginal youth in remote australia utilise digital technologies and new media documented popular use of facebook as a platform to: …upload their multimedia productions, comment on each others’ mobile phone ‘pics’ and announce the immediacy of their activities with online chat...[t]hey are also using these channels to air their thoughts and the cultural activities and concerns of their community (kral 2011:5). carlson (2013:147) likewise observed that the use of social media by aboriginal people has become an “everyday, typical activity”. aboriginal people are also participating in political and cultural activities on social media—particularly facebook (see also carlson 2013, 2014; carlson and frazer 2015). in a recent publication examining aboriginal people, social media and practices associated with death, carlson and frazer (2015) found that aboriginal people were using social media for notifications of deaths and funerals, offering condolences and extending support, and grieving and healing. they also revealed that social media can be both a productive and problematic space for aboriginal people. while social media provided an online space where families and extended kin could assemble, it was also a contested space where competing cultural understandings, values and performances of respect and bereavement played out publicly. social media can also provide a space where aboriginal people become subject to abuse. carlson (2014) has raised the issue of ‘r.i.p trolls’ (i.e., when instigators post abusive comments and memes on memorial pages dedicated to the deceased), racism and abuse targeted towards aboriginal people on social media. rauch and schanz (2013) argue that it is not difficult to locate negative comments about minority groups on social media, noting that at times posts can be subtly negative but are often overtly racist. australia’s race discrimination commissioner has also raised the issue of cyber racism, stating that there has been an increase in complaints about online racism (emery 2014). it is in this context that georgatos (2014) powerfully argues that racism is implicated in some suicides in aboriginal communities. 1.1.4 aboriginal help-seeking and social media help-seeking behaviour refers to the coping mechanisms an individual displays in the face of distress or pressure, or the communicative process of seeking help from others in the event of 3 australasian journal of information systems carson, farrelly, frazer & borthwick 2015, vol. 19, research on indigenous ict mediating tragedy particular difficulty or emotional distress (gourash 1978). globally, research has found that many people feel empowered while help-seeking online and are more confident in accessing information and talking about sensitive topics (gould et al. 2002). there is emerging evidence that technology-based interventions, such as offering information or access to professional advice, could be effective (griffiths et al. 2010). in an investigation into aboriginal suicide and self-harm, farrelly (2008) categorised helpseeking as either formal or informal. formal help-seeking involves services and professionals who have been formally trained as help sources, such as counsellors, psychologists, general practitioners and the like. informal help-seeking, on the other hand, involves untrained help sources such as family, friends or other community members and social networks. several studies have outlined the barriers that may hinder aboriginal and torres strait islander people from engaging in formal, face-to-face help-seeking (zubrick et al. 2005; rickwood et al. 2007; lumby and farrelly 2008 & 2009; farrelly 2008). mistrust of government services and intergenerational trauma related to past policies of removal and racism can impact help-seeking behaviours (zubrick et al. 2005). intergenerational stigma and feelings of shame associated with help-seeking remains a barrier in all age groups, but particularly among indigenous youth (price and dalgleish 2013). research has thus indicated that aboriginal and torres strait islander help-seeking is typically more informal than that practiced by non-indigenous people (farrelly 2008; rickwood et al. 2007). this is largely due to a general lack of help-seeking skills and a reluctance to invite the scrutiny of mainstream service providers into homes, lives and communities (lumby and farrelly 2008). the onus of help-giving then, typically falls to family, friends and social networks. finding solutions to the barriers that impede aboriginal and torres strait islander help-seeking is crucial given the high level of unmet need (rickwood et al. 2007: 35). we suggest social media may offer some such solutions. while researchers have recently begun to examine the potential of social media for aboriginal and torres strait islander help-seeking, the area remains underexplored, with little research involving large cohorts of aboriginal and torres strait islander social media users. studies have largely focussed on the help-seeking behaviours of aboriginal and torres strait islander youth specifically. price and dalgleish’s (2012) research, for instance, revealed that indigenous youth actively accessed self-help information online and peer support opportunities through social networking. edmonds et al’s (2012) study of aboriginal youth’s use of mobile devices and social networking found that aboriginal youth used social media for “maintaining connections and for pathways to assist them when facing big decisions” (2012:12) and that social media offered a sense of community that could provide support in stressful situations. young aboriginal people were using their online networks in what could be described as informal help-seeking. while this research is invaluable, little is known about the specifics of the complex relationships between aboriginal and torres strait islander use of social media and helpseeking and help-giving practices. it is also not known whether the same offline barriers that nurture a reluctance to seek formal help sources also exist online on social media. nor is it known whether informal help sources on social media are providing appropriate advice and/or intervention. additionally, little is known about the role of the informal help sources and helpgiving on social media and whether social media provides a forum where the needs of the helpseeker can be met. this knowledge gap is not isolated to australia. internationally, little is known about indigenous help-seeking behaviours on social media. research on indigenous peoples and social media is an emerging field and generally focuses on investigating the impact that social media is having on cultural practices (see, o’carroll 2013) or using technology to reach indigenous people and communities for health related issues (see, rushing and stephens 2011). others map indigenous use of social media for communicating, socialising and participating in cultural practices, when vast distances isolate communities (see, molyneaux et al. 2014). what we can ascertain from these studies is that indigenous groups are active social 4 australasian journal of information systems carson, farrelly, frazer & borthwick 2015, vol. 19, research on indigenous ict mediating tragedy media users and their online engagements include connecting with each other socially and culturally—and is likely to include practices of help-seeking and help-giving. 1.1.5 suicide and social media facebook, like other social media sites, is a platform where users can update their networks about how they are feeling by regularly posting ‘status updates’. manago et al. (2012:369) found “emotional disclosure”, or “self disclosure”, to be the major function of facebook’s status update feature. in a recent uk study it was revealed that social media may feed anxiety and increase feelings of inadequacy (paddock 2012). the researchers found that over half of the respondents claimed social media made them feel less confident and that they spent time comparing their lives to their online networks. further, the research also found that after online confrontations the participants reported experiencing difficulties in personal relationships (paddock 2012). conversely, professor jessica vitak of the college of information studies at the university of maryland argues that the relationship between social media and mental health is far too complex to suggest that it has blanket effects across users. she has found that, for many people, social media can provide social benefits in terms of social and emotional support and maintaining relationships that would have faded away without the technology. she also concedes, however, that in some instances, such as those involving cyber bullying and online harassment, the effects on well-being may be negative. in a comprehensive review of the literature associated with social media and suicide, it was revealed that some people use social media for communicating suicidal ideations. this study also reported a correlation between cyber bullying and suicide ideation (robinson et al. 2012:12), noting the potential for harm must be taken seriously (2012:18). there are several documented cases of people posting status updates that are intended to act as suicide announcements or suicide notes. a recent news article in the australian reported facebook being used by aboriginal people to post messages of suicidal intent, with one young girl posting, “feel lost right now hang myself” (martin 2014). such announcements are generally visible to large numbers of other users within that person’s network—from dozens to thousands of others. as ruder et al. (2011) point out, this offers a potential opportunity for people to intervene in the realisation of the suicide risk, either through directly contacting or approaching the person, or involving emergency services. similarly, robinson et al. (2014:7) found that social media platforms have the potential to enable users to “access information, support and counselling and to share their experiences in a flexible, timely and readily accessible format”. robinson et al’s (2014:7) study also found that help seeking and responding was a “valuable and therapeutic feature” of social media. in the uk, several public cases of suicide announcements on facebook prompted uk charity samaritans to team up with facebook to provide help for those feared to be in danger (see, luxton et al. 2012). from march 2011, uk users were able to report concerns prompted by their friends’ posts through the facebook help centre. facebook would then put samaritan counsellors in touch with those in distress. a comparable initiative is currently being tested in the united states. a collaboration between facebook and researchers from the school of social work at the university of washington has seen the development of suicide prevention and support tools (worrall 2015). if a user identifies a post that causes them concern over another’s safety, they can then report it to facebook for review. if the post is found to indicate distress or suicidal thoughts, facebook sends the person a series of notifications offering advice (worrall 2015). although the effectiveness of these or similar programs has not yet been assessed, these efforts to leverage social media’s power for improving the well-being of its users could be promising. while these social media intervention initiatives appear promising, there is also need for caution when dealing with vulnerable groups, such as aboriginal and torres strait islander people. luxton et al. (2012:197) note that social media may pose a risk to vulnerable groups and argue that more research is needed focussing on both social media’s “negative and positive influences”. they suggest that examination of “sub-groups that might be most vulnerable to suicide promoting influences of social media is also warranted” (luxton et al. 2012:197). 5 australasian journal of information systems carson, farrelly, frazer & borthwick 2015, vol. 19, research on indigenous ict mediating tragedy 2 methodology & research design to explore aboriginal people’s use of social media for help-seeking and help-giving, this study utilises nakata’s (2007) notions of an indigenous standpoint theory and the cultural interface as both an entry point for inquiry and a tool for analysis. nakata’s conceptualisation of a standpoint theory for indigenous research and researchers provides a way for understanding the contemporary complexities of indigenous experience and struggles. indigenous standpoint theory is a method for indigenous researchers to critically assess the corpus of knowledge written about us. nakata suggests, indigenous standpoint theory can assist a way to theorise knowledge from a particular and interested position—not to produce the ‘truth’ of indigenous position but to better reveal the workings of knowledge and how understanding of indigenous people is caught up and implicated in its work (2007:215). indigenous standpoint theory is complemented by and perhaps best understood alongside his concept of the cultural interface, a term he coined to denote the everyday site of struggle that continues to confine colonised people. the cultural interface is located at the intersection of the western and indigenous domain…the place where we live and learn, the place that conditions our lives, the place that shapes our futures and more to the point the place where we are active agents in our own lives—where we make decisions—our lifeworlds (nakata 2002:285). to explore the diversity of everyday experiences of aboriginal social media users, data was collected through a variety of qualitative methods. following formal ethics approval from the university of wollongong’s ethics committee, participants were recruited through contacting key figures from each community, who were thereafter employed as one-off research assistants. this method was deemed both ethical and effective. respected persons in communities could communicate the purposes and content of the research project to potential participants and help interviewers conduct the interviews in an appropriate manner. in this way, informed consent was ensured. in total, eight communities across new south wales, queensland, south australia and western australia participated in the project. these communities represented a broad social and geographical cross-section: from cities (sydney, brisbane, adelaide, perth), regional areas (illawarra, batemans bay, broome), and remote communities. participants came from a wide variety of ages (18-60 years of age) and backgrounds: political activists, university students, stay-at-home parents, community elders. to ensure anonymity participants were given a pseudonym. fifty semi-structured interviews were conducted, lasting between thirty and ninety minutes each. ten themes were covered in interviews, including: quantity and quality of social media use; expressions and concerns of culture on social media; indigenous identities; community formation and concerns. drawing on nakata’s (2007) indigenous standpoint theory and cultural interface, interview transcripts were then subject to a form of narrative discourse analysis. as social interactions are increasingly conducted on social media, there was also an opportunity to collect additional data through an online social media-driven survey created using surveymonkey and promoted via facebook and twitter, using a snowball sampling technique. the survey was anonymous and consent was tacit, indicated by the choice to complete the survey. simple quantitative analyses were conducted using surveymonkey’s inbuilt statistical tools. methodological rigour was ensured through research triangulation: using a multi-method approach and ensuring all analysis was conducted by at least two researchers. 3 results & discussion 3.1 a less daunting way to seek help most participants in this study esteemed social media as a place of mutual support and care, suggesting it offers a platform where users could receive support and encouragement through 6 australasian journal of information systems carson, farrelly, frazer & borthwick 2015, vol. 19, research on indigenous ict mediating tragedy interactions with friends and family. through posting their thoughts and feelings as ‘status updates’, or directly reaching out to others through comments and private messages, participants could feel supported by their online network. in this way, some participants reported that social media is a less daunting way for users to find support and seek help in regards to suicidal ideation and behaviour, as there is less embarrassment due to the removal of the face-to-face contact: it does make it easier. because you might be too embarrassed or not want to talk to a person physically about it, whereas you can just go online and talk to someone or even look it up on the internet…it’s not as confronting. if you sat down with somebody, you think they might judge you. whereas if you did it online you can’t really tell if they’re judging you, so it’s less confronting (daniel, male, 19). certainly, seeking help for suicidal behaviour from mainstream formal help sources can be a daunting task for aboriginal peoples. farrelly (2008) found that mainstream services were typically described as being culturally inappropriate, due to a lack of awareness of aboriginal culture, history, and a discipline base that does not account for aboriginal frames of reference for particular aspects of health and well-being. the procedures and protocols that must be followed in order to gain access to services can be intimidating to the uninitiated, unaccustomed, or those whom the service is not directly targeting. the capacity of social media to make help-seeking less daunting is thus an appealing option. likewise, participants described social media, and the act of writing public posts, as offering an alternative outlet for users’ feelings and thoughts, circumventing some of the social barriers to requesting help. for instance, one participant suggested: …writing is an incredibly powerful thing, and sometimes people write stuff that they wouldn’t necessarily say. so i have seen people express some really stressed, distressed feelings. …i think people with those sorts of issues can sometimes use that as a healing mechanism. what they’re keeping inside, through the mode of writing, they are actually able to get out (chantelle, female, 36). this suggests social media might encourage users to seek support in situations where they would not have otherwise done so. farrelly (2008) found that due to the close-knit nature of many aboriginal communities, it can be very difficult for aboriginal peoples to seek help for sensitive issues such as suicide, self-harm, sexual abuse, and family violence. accessing informal sources of help generally involves disclosing the sensitive issue to a community member—the potential ramifications of which could be overwhelming for the individual. farrelly (2008) noted her participants stated that fears of these potential ramifications would generally compel them to avoid disclosure altogether and avoid accessing any alternative informal help sources. this avoidance can result in such issues becoming hidden within the community—the discussion of which becomes taboo. likewise, participants in the current study expressed concerns regarding facebook’s policy of transparency, which leaves little room for anonymous activity. if engagement with services on facebook—mental health services for example—is publicly open for that person’s network to see, it might become a further barrier to seeking help. 3.2 an opportunity to offer help the majority of participants described facebook as providing an opportunity to offer help to someone they believed was at risk of suicide. for example: that’s a big part of what i am there for. and for whatever reason, there are various people that see me as a person they can trust and they do [reach out for help]. and so they lean on me to do that. and so you just try to be strong enough to do what you can to help people (thomas, male, 41). while this suggests a possible positive avenue for help-giving on facebook, some participants were ambivalent about public calls for help. they expressed scepticism towards the genuineness and seriousness of some threats of self-harm, deeming them to be ‘attention 7 australasian journal of information systems carson, farrelly, frazer & borthwick 2015, vol. 19, research on indigenous ict mediating tragedy seeking’ rather than help-seeking. for example, one participant noted the use of suicidal status posts for the purpose of gaining ‘likes’—a function on facebook where other users can indicate they ‘like’ your photo or written post: but sometimes i think it is just about getting attention. everyone just wants to be the focus of all the posts and get all the likes. it is pretty sad when you get a lot of likes if you post that you are suicidal or feeling miserable (maree, female, 48). other participants reported they restricted their offers of help to family only, and a small number expressed an unwillingness to offer help at all, either because they feared they might give the wrong advice or because they felt annoyed by what they deemed to be attentionseeking behaviour. as one participant stated: “i told them not to advertise their problems on facebook” (katie, female, 24). 3.3 aboriginal facebook suicide prevention strategies but when these barriers were not present, participants described what could be collectively understood as a continuum of facebook suicide prevention strategies that they would implement to varying extents if they were concerned that an individual was at risk of suicide. at one end of this continuum are interventions that are relatively non-intrusive and easily implemented, such as fostering a general ‘positive atmosphere’ in their facebook interactions with others, or letting people know that they are present as a source of support. interventions then progressed to using facebook to offer support, help, information about available services, and advice to people they felt could be at risk of suicide—generally through private messaging, or ‘inboxing’, as opposed to public messages. this was described as a more direct means of offering help to someone, while protecting his or her privacy: i wouldn’t do it in the public domain, you know. if it was a private message, something like that, like if it’s facebook, i wouldn’t be doing it so that everyone else could see. i wouldn’t want to humiliate them or make them feel targeted any more than what they already were (lara, female, 22). progressing along this continuum, it is apparent that facebook enables some participants to actively ‘keep an eye on people’ that they may be concerned are at risk of suicide, usually by monitoring their posts and checking in with them on a regular basis: i gave them the contact details for the counsellor at the ams [aboriginal medical service] and just let them know they were not alone and i checked in on them through the next few days (katie, female, 22). participants unanimously agreed that while social media such as facebook can provide a platform for aboriginal suicide help-seeking and helping, it should not replace face-to-face contact, but rather could complement it. and so the progression to the opposite end of the continuum includes aboriginal facebook suicide prevention strategies that utilise facebook to enable the organisation of face-to-face assistance. participants described rallying around with other friends on facebook who also knew the person at risk, or contacting their friends and family to try and ascertain what may be going on and what help the person may need. this could then lead to actually getting together in person and visiting the individual at their home, and/or calling emergency or other services. as one participant explains: i remember one young man was writing some, well, it seemed quite suicidal thoughts on that [facebook]. so it ended up a bunch of us actually rallying together to make sure that police were sent around and went to that person’s place and it was all okay (chantelle, female, 36). some participants even reported using their own contacts in the region in which the person at risk was living, which could be interstate, to rally together and go visit the person at risk: a couple of weeks ago, a lady from, well i don't really know her, but she's family, so i'm friends with her on facebook. and she wrote, and i thought it was an attentionseeking thing, but she wrote, because she had a warrant out for her arrest or 8 australasian journal of information systems carson, farrelly, frazer & borthwick 2015, vol. 19, research on indigenous ict mediating tragedy something, and she wanted to top herself off, she doesn't know what to do. so i messaged one of my cousins in darwin and just let them know what was going on…they went around and she was all right (louise, female, 25). in this respect, while the close-knit nature of many aboriginal communities can impede aboriginal help-seeking for suicidal behaviour, it can also be utilised with the assistance of facebook to implement effective aboriginal suicide prevention strategies. 3.4 rapid information dissemination all participants agreed that facebook and other social media has the potential to aid aboriginal suicide prevention by means of the provision of fast access to relevant information. this notion is empirically supported by research conducted by silenzio et al. (2009), who used ‘network modelling’ to track and map the flow of suicide prevention information through lesbian/gay/bisexual (lgb) identified social media users. like indigenous peoples, globally young lgbqti people are more likely to attempt suicide than general populations. silenzio et al. (2009) ultimately demonstrated the potential of social media in acting as a suicideprevention platform through information distribution. qualitative data from the current study further substantiates the results of silenzio et al. (2009) in an indigenous context. as many aboriginal people are active on facebook and other social media, it has the capacity to quickly and easily distribute information to a wide audience. although participants expressed some ambivalence as to whether this information would effectively reach the person at risk, it was suggested it might help friends and family within the network in identifying risk factors, taking action, and supporting them emotionally. for example: i suppose it’s good because, with younger people, i suppose their first thought would be to go on facebook, rather than googling depression, something like that (lara, female, 22). those sorts of things can be disseminated through social media, and really, really quickly (jacob, male, 34). certainly, information for users helping to identify suicide warning signs would be beneficial given the ambivalence of many participants about whether some behaviour was attentionseeking as opposed to help-seeking. it was also suggested that campaigns through social media could help reduce the stigma around mental health and instigate much-needed discussions around the issue: i think to start the discussion it's good. because when [the actor] robin williams passed away, a lot of people were talking about it. and a lot of the statistics around suicide actually came to life, which i find a lot of people that i knew, well i never knew that it was the leading cause of young adult deaths. i never knew that. so for me to actually find that out, i was taken aback by that (louise, female, 25). 3.5 mediating tragedy – bullying, racism, negativity, and suicide contagion silenzio et al.’s (2009:2) optimistic conclusion about the efficacy of social media in reducing suicidal behaviour and attitudes came with a notable qualification, however: “online networks have the dual potential to serve as platforms for prevention interventions as well as to serve as conduits for toxic influences such as suicide contagion.” as the current study also found, the flip side of this suicide prevention capacity of facebook and other social media is the potential for bullying, racism, negativity and suicide contagion. some participants reported that interacting on facebook is often a source of negative feelings about the self: facebook is a big problem for young people. some people see their message, some people talking about you on facebook, and they pass the message and pass the message. and that person, ‘oh, i’ve had enough, nobody like me on facebook, might kill myself’ (sam, female, 25). 9 australasian journal of information systems carson, farrelly, frazer & borthwick 2015, vol. 19, research on indigenous ict mediating tragedy but at the same time, sometimes [facebook] is the cause. because people on there, you know, you see the bullying sides of things, that could be pushing them to, you know, feel worthless (nicole, female, 20). and more worryingly: you could drive someone to suicide on twitter (thomas, male, 41). an episode of living black (a television show featuring stories of interest to australian indigenous people) titled ‘cyber wars’ (19 april 2010) highlighted concerns about how racism online is affecting indigenous people and communities. furthermore, the joint committee on cyber-safety in 2013 led an ‘inquiry into issues surrounding cyber-safety for indigenous australians’ after evidence revealed that indigenous people were increasingly subject to cyber bullying and online racism. the report confirmed that online racism does have a negative effect on indigenous people (2013:20). it also raised concerns about the link between cyber-bullying, racism and suicide (2013:23). cyber bullying among aboriginal people is a worrying trend. in 2012, aboriginal people living in a remote location were calling for a ban on facebook and social media, as it was believed that it was fuelling community tensions and what was referred to as “old family rivalry and blood feuds” (michael 2012). it was reported that a street brawl took place after exchanges on social media sites reignited a long-standing dispute. while some community members were calling for a social media ban, others were against censorship and were calling for more education so the community could have a better understanding of the capacity of social media. the other concern participants voiced about facebook and suicidal behaviour was in regards to suicide clusters. aboriginal suicide appears to occur in clusters at certain points in time in particular communities, and certainly more frequently than in the general population (hunter et al. 2001). such clusters can be described as suicide contagion, whereby exposure to suicidal behaviour appears to trigger further ‘copycat’ suicidal behaviour. the clearly defined, contained, shared physical environment of many aboriginal communities results in high visibility of and exposure to suicidal behaviour. suicides, particularly the common method of hanging, are frequently witnessed by many members of an aboriginal community, who experience firsthand the impact such deaths have on the community. these deaths often spark a cluster of suicides in aboriginal communities involving similar methods, gender and age groups. as hunter et al. (2001) notes, this suggests suicide can play a catalytic role in communities through observational learning, imitation, and modelling. one participant, who works in the health and community sector, suggested that in the aftermath of the suicide of a young aboriginal girl, several elders became concerned about the information being communicated via social media and the impact this may have on other young people in the community. they feared that this may foster suicide contagion. to quote at length: we had, unfortunately, a young girl in this community commit suicide on boxing day just passed. and…the content went throughout the network very, very quickly. so it was a matter of, we got rung up by a couple of aunties to get around to some of the young fellas to try to make sure everything was all right, because there was a lot of knowledge going out and a lot of hurt going around. so, look there’s two folds to it. threats and stuff are the main, and there’s you know there’s suicide that does happen, and how do people process that information? and when you’re accessing that you’ve got a 1000 people in your support network but no one physically around you, i think is a different situation. i think that’s what we’re finding with a lot of the suicide stuff that we get. we get big clusters. that we might have one young guy do it, and then we might have three or four within that immediate social group go down the same path. and it’s sort of hard to predict and manage (dave, male, 33). this concern is given empirical support by kramer et al.’s (2014) recent and controversial study on “massive-scale emotional contagion” on facebook. although it did not measure suicide contagion specifically, it showed negative emotion can travel through online networks. 10 australasian journal of information systems carson, farrelly, frazer & borthwick 2015, vol. 19, research on indigenous ict mediating tragedy with an almost unrivalled sample size (n=689,003), researchers altered the facebook newsfeed algorithm in order to restrict what types of content with which individual users were presented. through identifying and then either eliminating or emphasising content containing key emotive words and terms, the two test groups were presented with facebook content that was either primarily negative or primarily positive (while the control group saw an unaltered ‘news feed’). the experiment ran for a single week in 2012 and turned up small, but statistically significant results: those in the ‘negative’ group we more likely to likewise generate ‘negative’ content, while the ‘positive’ group were more likely to generate ‘positive’ content. although the effects were small, “given the massive scale of social networks such as facebook, even small effects can have large aggregated consequences” (kramer et al. 2014:8790). it seems fear of suicidal contagion passing through online aboriginal networks is justified. 4 conclusion this paper offers the first study of aboriginal people’s use of social media for help-seeking and help-giving not restricted to young people. it shows that aboriginal people already consider social media such as facebook as a place of mutual support and care. previous research has identified significant barriers for aboriginal people in accessing help for mental well-being, suggesting they are more likely to engage in informal methods of help-seeking. thus the above results demonstrate facebook offers a further, potentially vital avenue for aboriginal people to seek help outside formal services. participants in this study reported being more often on the receiving end of requests for help, articulating a number of already existing help-giving practices taking place on facebook. in mobilising help, they drew on a continuum of practices ranging from soft, informal help-giving techniques (such as creating a positive online environment) to direct intervention (in some cases involving emergency services). but there was also some ambivalence in facebook’s efficacy in acting as a platform for helpseeking and help-giving. while overcoming several barriers to help (such as bureaucracy and cultural inappropriateness), the use of facebook for these ends presented their own barriers. participants were often sceptical of public calls for help, instead identifying these posts as attention-seeking. others lacked confidence in providing help to those in need. most concerning, however, is facebook’s potential to contribute to mental ill-health, through facilitating racial abuse and suicide contagion. thus any potential health benefits of social media need to be weighed against these challenges. with this in mind, we here suggest that a promising option in addressing aboriginal suicide is to engage with these already existing—and from the accounts above, effective—practices. from this initial knowledge base, we encourage the development and evaluation of culturally appropriate suicide intervention programs, using facebook as a platform for facilitating help. as the participants indicated, this could not replace face-to-face interventions, but would rather complement and enhance them. the need is urgent. aboriginal suicide is among the highest in the world (ting and hunter 2015). aboriginal children under 14 years are almost eight times more likely to commit suicide than their non-aboriginal compatriots. aboriginal people are already engaging in help-seeking and help-giving on facebook. leveraging these in a thoughtful way could help reverse these tragic trends. acknowledgments the authors wish to acknowledge and thank the aboriginal people who participated in this study. your contributions, experiences and anecdotes have enriched this area of research. additionally, we would also like to thank the research assistants in each community who generously provided us with information on local cultural protocols. 11 australasian journal of information systems carson, farrelly, frazer & borthwick 2015, vol. 19, research on indigenous ict mediating tragedy ethics this research has ethics approval from the university of wollongong human research ethics committee. approval granted on 26 september 2013 – 2014 and extended to 25 august 2015. reference number: he13/325 funding this research is supported by the australian research council discovery (indigenous), project id: in130100036.references references australian institute of aboriginal and torres strait islander studies conference, 2010, ‘information technologies and indigenous communities’ http://www.aiatsis.gov.au/research/symposia/digi10/theme.html callinan, t. 2014, ‘remote indigenous australians rely on facebook to stay in touch’ 29 september 2014, http://www.sbs.com.au/news/article/2014/08/26/remoteindigenous-australians-rely-facebook-stay-touch carlson, b. 2013, ‘the ‘new frontier’: emergent indigenous identities and social media’ in m. harris, m. nakata, & b. carlson (eds.), the politics of identity: emerging indigeneity, university of technology sydney e-press, sydney, pp. 125-146. carlson, b. 2014, ‘well connected indigenous kids keen to tap new ways to save lives’ the conversation, 20 september 2014, https://theconversation.com/well-connectedindigenous-kids-keen-to-tap-new-ways-to-save-lives-30964 carlson, b. & frazer, r. 2015, ‘“it’s like going to a cemetery and lighting a candle”: aboriginal peoples, sorry business and social media’, alternative: an international journal of indigenous peoples, 11(3). dudgeon, p. & kelly, k. 2014, contextual factors for research on psychological therapies for aboriginal australians, australian psychologist, 49(1), pp. 8-13. edmonds, f., rachinger, c., waycott, j. morrissey, p., kelada, o. & nordlinger, r. 2012, ‘keeping intouchable: a community report on the use of mobile phones and social networking by young aboriginal people in victoria’, institute for a broadband-enabled society, university of melbourne. emery, k. 2014, ‘social media driving racism’, the west australia, 30 june, https://au.news.yahoo.com/thewest/wa/a/21068988/social-media-driving-racism/ elliott-farrelly, t. 2004, ‘australian aboriginal suicide: the need for an aboriginal suicidology?’ australian e-journal for the advancement of mental health, 3(3), pp. 1-8 farrelly, t. 2008, ‘the aboriginal suicide & self-harm help-seeking quandary’, aboriginal & islander health worker journal 32 (1), pp. 11-15. georgatos, g. 2014, ‘it is racism killing our people – suicides born of racism’, the stringer, 30 june, http://thestringer.com.au/it-is-racism-killing-our-people-suicides-born-ofracism-9152 griffiths, k., farrer, l., & christensen, h. 2010, ‘the efficacy of internet interventions for depression and anxiety disorders: a review of randomised controlled trials’, medical journal of australia, 192, s4-s11. gould, m., munfakh, j., lubell, k., kleinman, m. & parker, s. 2002, ‘seeking help from the internet during adolescence’, journal of the american academy of child and adolescence psychiatry, 41, pp. 1182-1189. gourash, n. 1978, ‘help seeking: a review of the literature’, journal of community psychology, 6(5), pp. 413-423. 12 australasian journal of information systems carson, farrelly, frazer & borthwick 2015, vol. 19, research on indigenous ict mediating tragedy hunter, e., reser, j., baird, m. & reser, p. 2001, an analysis of suicide in indigenous communities of north queensland: the historical, cultural and symbolic landscape, commonwealth department of health and aged care: canberra. . joint committee on cyber-safety, 2013, issues surrounding cyber-safety for indigenous australians. the parliament of the commonwealth of australia, 20 september 2014, http://www.aph.gov.au/parliamentary_business/committees/house_of_representativ es_committees?url=jscc/indigenous_australians/report.htm kral, i. 2001, ‘youth media as cultural practice: remote indigenous youth speaking out loud’, australian aboriginal studies, 1, pp. 4-16. kramer, a. d. i., guillory, j. e. & hancock, j. t. 2014, experimental evidence of massive-scale emotional contagion through social networks, proceedings of the national academy of sciences. living black, 2010, ‘cyber wars’ april 19th, special broadcasting service. lumby, b. & farrelly, t. 2008, ‘aboriginal ageing and disability issues in south west and inner west sydney’, aboriginal & islander health worker journal, 32 (5), pp. 27-34. lumby, b. & farrelly, t. 2009, ‘family violence, help-seeking & the close-knit aboriginal community: lessons for mainstream service provision’, australian domestic & family violence clearinghouse. issues paper 19, september, pp.1-24. luxton, d. d., june, j. d. & fairall, j. m. 2012, ‘social media and suicide: a public health perspective’, american journal of public health, 102, pp. 195-200. martin, s. 2014, facebook used by remote teens to post suicide threats, 29 september 2014, http://www.theaustralian.com.au/national-affairs/indigenous/facebook-used-byremote-teens-to-post-suicide-threats/story-fn9hm1pm-1226832109265 michael, p. 2012, ‘indigenous leaders want to ban or censor social media including facebook’, the courier mail, 27 september 2014, http://www.couriermail.com.au/news/queensland/indigenous-leaders-want-to-banor-censor-social-media-including-facebook/story-e6freoof-1226237942587 molyneaux, h., o’donnell, s., kakekaspan, c., walmark, b., budka, p., & gibson, k. 2014, social media in remote first nation communities. canadian journal of communication, 39, pp.275-288. moreton-robinson, a. & walter, m. 2009, indigenous methodologies in social research. social research methods, pp. 1-18. price, m. & dalgleish, j. 2013, ‘help-seeking among indigenous australian adolescents: exploring attitudes, behaviours and barriers’, youth studies australia, 32(1), pp.10-18. manago, a. m., graham, m. b., greenfield, p. m. & salimkhan, g. 2008, ‘self-presentation and gender on myspace’, journal of applied development psychology, vol. 29, pp. 446-458. nakata, m. 2007, disciplining the savages: savaging the disciplines, aboriginal studies press: canberra. nakata, m. 2002, indigenous knowledge and the cultural interface: underlying issues at the intersection of knowledge and information systems, ifla journal, 28(5/6), pp. 281-291. o’carroll, a. 2013, ‘an analysis of how rangatahi maori are using social networking sites’, mai journal, 2(1), pp. 46-59. pascoe, b. 2014, dark emu, magabala books: broome. paddock, c. 2012, ‘facebook use feeds anxiety and inadequacy says small study’, medical news today, 30 june, http://www.medicalnewstoday.com/articles/247616.php 13 australasian journal of information systems carson, farrelly, frazer & borthwick 2015, vol. 19, research on indigenous ict mediating tragedy pink, b. & allbon, p. 2008, the health and welfare of australia’s aboriginal and torres strait islander peoples, 2008. catalogue no. 4704.0, australian bureau of statistics: canberra. radoll, p. 2012, ‘information and communication technologies in the classroom: implications and considerations’, aboriginal and torres strait islander education: an introduction for the teaching profession, cambridge university press: port melbourne, pp. 113-130. ranzijn, r., mcconnochie, k. r. & nolan, w. 2009, psychology and indigenous australians: foundations of cultural competence, palgrave macmillan: australia. rauch, s. & schanz, k. 2013, ‘advancing racism with facebook: frequently and purpose of facebook use and the acceptance of prejudiced and egalitarian messages’, computers in human behaviour, 29(3), pp. 610-615. reser, j. p. 1991, ‘aboriginal mental health: conflicting cultural perspectives’ in j. reid and p. trompf (eds.), the health of aboriginal australia. harcourt brace & company: marrickville, australia. rigney, l. 1997, ‘internationalization of an indigenous anticolonial cultural critique of research methodologies: a guide to indigenist research methodology and its principles’, wicazo sa review, pp.109-121. rigney, l. 2011, ‘indigenous education and tomorrow’s classroom: three questions, three answers’, n., milgate, g., bell, hr (ed/s.), acer press, pp. 35-47. rickwood, d., deane, f., & wilson, c. 2007, ‘when and how do young people seek professional help for mental health problems?’, medical journal of australia, 187(7), pp. 35-39. robinson, j., rodrigues, m., fisher, s., & herman, h. 2012, ‘suicide and social media: findings from the literature review, young and well cooperative research centre, melbourne. ruder, t. d., hatch, g. m., ampanozi, g., thali, m. j. & fischer, n 2011, ‘suicide announcement on facebook’, crisis, 32, pp. 280-2. rushing, s.c. & stephens, d. 2011, ‘use of media technologies by native american teens and young adults in the pacific northwest: exploring their utility for designing culturally appropriate technology-based health interventions’, the journal of primary prevention, 31(1-2). silenzio, v. m., duberstein, p. r., tang, w., lu, n., tu, x. & homan, c. m. 2009, ‘connecting the invisible dots: reaching lesbian, gay, and bisexual adolescents and young adults at risk for suicide through online social networks’ social science & medicine, 69, pp. 46974. tatz, c. 2001, aboriginal suicide is different: a portrait of life and self-destruction, aboriginal studies press: canberra. the elders’ report into preventing indigenous self harm & youth suicide, 2014, culture is life campaign, 26 september 2014, http://apo.org.au/research/elders-reportpreventing-indigenous-self-harm-and-youth-suicide ting, i. & hunter, f. 2015, ‘indigenous suicide rate 12th highest in the world’, the sydney morning herald, 13 july, http://www.smh.com.au/national/health/indigenoussuicide-rate-12th-highest-in-the-world-20150710-gi9jyn.html vitak, j. 2014, ‘facebook makes the heart grow fonder: relationship maintenance strategies among geographically dispersed and communication-restricted connections’. in proceedings of the 17th acm conference on computer supported cooperative work & social computing (cscw ’14). acm, new york, ny, usa, 842-853. westerman, t. 2010, ‘engaging australian aboriginal youth in mental health services’, australian psychologist, 45(3), pp. 212-222. 14 australasian journal of information systems carson, farrelly, frazer & borthwick 2015, vol. 19, research on indigenous ict mediating tragedy whiteman, h. 2013, could facebook be making you miserable?’, medical news today, 30 june, http://www.medicalnewstoday.com/articles/264902.php worrall, a. 2015, ‘facebook introduces suicide prevention tools’, the sydney morning herald, 30 june, http://www.smh.com.au/national/facebook-introduces-suicide-preventiontools-20150227-13qciq.html zuberick, s., silburn, s., lawrence, d., mitrou, f., dalby, r., blair, e., griffing, j., milroy, h., de maio, j., cox, a. & li, l. 2005, the social and emotional wellbeing of aboriginal children and young people, the western australian aboriginal child health survey: the social and emotional wellbeing of aboriginal children and young people, curtin university and telethon institute for child health research, perth. copyright: © 2015 carlson, farrelly, frazer & borthwick. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. 15 http://www.smh.com.au/national/facebook-introduces-suicide-prevention-tools-20150227-13qciq.html http://creativecommons.org/licenses/by-nc/3.0/au/ http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 1.1 background 1.1.1 aboriginal suicide 1.1.2 differing epidemiologies, methods, risk factors and aetiological theories 1.1.3 aboriginal people and social media 1.1.4 aboriginal help-seeking and social media 1.1.5 suicide and social media 2 methodology & research design 3 results & discussion 3.1 a less daunting way to seek help 3.2 an opportunity to offer help 3.3 aboriginal facebook suicide prevention strategies 3.4 rapid information dissemination 3.5 mediating tragedy – bullying, racism, negativity, and suicide contagion 4 conclusion paper title (paper title style) australasian journal of information systems lim, zhang, gable & rai 2016, vol 20, post publication review principles of construct clarity 1 post publication review zhang, m., cable, g., & rai, a. (2016). toward principles of construct clarity: exploring the usefulness of facet theory in guiding conceptualization. australasian journal of information systems, 20. doi: http://dx.doi.org/10.3127/ajis.v20i0.1123 review theory development is at the heart of the scientific process in information systems. knowledge from extant literature and observations of an information systems phenomenon underpin hypothesis development in information systems research about what is causing the phenomenon (e.g. enablers of benefit, cost, opportunity, or challenge) and how systems (e.g. process) work. empirical studies (e.g. experiments) are conducted to test the hypothesis and a theory emerges when a hypothesis is confirmed. most often, each theory employs a unique vocabulary to articulate specific constructs (or factors) that are considered important in explaining why an information systems phenomenon occurs and in predicting what is likely to happen next. application of similar vocabulary, therefore, is indicative of potential replication. critical to theory development is conceptualization (markus and saunders, 2007; suddaby, 2010). the paper by zhang, cable, and rai makes a timely call for revisiting an underdeveloped area of conceptualization in information systems in the form of construct clarity. as the authors rightly point out, extant work in the information systems discipline has largely focused on the importance of construct validity and the ways in which construct validity can be met and enhanced (e.g. gefen and straub, 2005; kuo and dick, 2010; mackenzie, podsakoff, and podsakoff, 2011; sedera, 2015). by introducing and discussing facet theory and its methodological procedures (guttman, 1954a, 1954b, 1971; guttman and greenbaum, 1998), the authors shed light on a useful solution that can improve conceptualization and theory development efforts through better construct clarity in information systems research. more specifically, to achieve greater construct clarity, the authors suggest that “(1) concepts in a content universe should be specified in terms of all relevant facets, (2) the facets, collectively, should be logically exhaustive of the content universe, (3) the logical relationships among facets should be specified, whereby independence among facets is preferred, (4) each facet should be analyzed into a set of collectively exhaustive elements, (5) each facet should be analyzed into a set of mutually exclusive elements, (6) the logical relationships among elements of a facet should be specified, and (7) the relationships among the concepts defined according to the facets and the elements of facets should correspond to the focal phenomenon.” notwithstanding the benefit and usefulness of these guiding principles in producing a richer and stronger conceptual understanding of a construct’s relationships to extant related constructs, two key limitations are apparent (in addition to the other key limitations acknowledged in the paper—e.g. complications related to issue isolation and limited metatheoretical principles). these limitations should spur further conceptual research and discussion in the area. more specifically, the guiding principles proposed by the authors may only be relevant to constructs that are multidimensional. that is because salient facets (or the set of attributes that represent the underlying conceptual and semantic components within a content universe [guttman and greenbaum, 1998]) are absent in constructs that are unidimensional. this limitation is further amplified when construct validity takes precedence over construct clarity—for example, in studies that aim to obtain a macro understanding of constructs and to avoid aggravated responses (bergkvist and rossiter, 2007). thus, further exploration for an alternative set of guiding principles using alternative theoretical lenses for establishing construct clarity for unidimensional constructs is highly encouraged. besides that, the limitation of using an avoidance strategy when assuming a neutral philosophical stance was accentuated in their paper as they chose not to delve into the expectations of construct clarity presented by positivists and constructivists. hence, the australasian journal of information systems lim, zhang, gable & rai 2016, vol 20, post publication review principles of construct clarity 2 guiding principles that were proposed were limited to those that were adopted from mcgrath’s (1968) succinctly summarized set of principles for facet theory methodology based on implied logic. using an approach strategy by delving into alternative streams of philosophical thought and expectations held by positivists and constructivists could have realized the greater potential for increased construct clarity. for example, delving further into the positivistic stream of philosophical thought could have produced new, additional guiding principles for greater construct clarity, such as those that account for (or guide researchers on how to deal with) prediction with multidimensional constructs that are dynamic and non-linear in nature (falk, hammerschmidt, and schepers, 2010). the same could also be done for the constructivist stream of philosophical thought. thus, delving into the expectations of these philosophical stances simultaneously should produce a more inclusive set of guiding principles for greater construct clarity (while maintaining a neutral standing). in short, the paper by zhang and colleagues should directly contribute toward helping information systems scientists develop and maintain good construct clarity, and indirectly contribute to the accentuation of novelties and knowledge comparability in the field. it is hoped that the articulation provided herein and in the paper by zhang and colleagues would stimulate greater interest in conceptual work that enhances the rigorousness in establishing construct clarity for myriad types of constructs in information systems studies of different philosophical paradigms. weng marc lim swinburne university of technology (sarawak campus) wlim@swinburne.edu.my / lim@wengmarc.com acknowledgment many thanks to meng zhang, guy cable, and arun rai for contributing to the growing visibility of conceptual studies in the discipline of information systems. they have shed light on the importance of construct clarity and the ways in which information systems scientists can develop and maintain construct clarity to improve conceptualization for good theory development in information systems research. author response we thank weng marc lim for sharing his thoughts on our recent paper, toward principles of construct clarity: exploring the usefulness of facet theory in guiding conceptualization, published on australasian journal of information systems (zhang et al. 2016). we are very pleased to know our colleague has been convinced of the importance of construct clarity in conceptualization and theory development, which is the primary motivation driving the paper. lim brings two issues to light and believes these two issues are “two key limitations” of our paper. we herein offer a response. weng marc lim argues that our guidelines for construct clarity are relevant only to multidimensional constructs, but not to unidimensional constructs. we believe that lim misunderstood the distinction between unidimensional and multidimensional constructs and confused “dimensions” of a construct with “facets” (as defined in our paper) of a content universe. edwards (2001, p. 144) argues, “a construct is multidimensional when it refers to several distinct but related dimensions as a single theoretical concept.” regardless of ongoing debates on the value or existence of multidimensional constructs (edwards 2001), for a multidimensional construct, its every dimension must also be a meaningful construct and thus has variance in empirical settings (polites et al. 2012). for example, is success may be considered a multidimensional construct (delone and mclean 1992) such that each of its dimensions meaningfully represents a construct (i.e., “system quality”, “information quality”, “use”, “user satisfaction”, “individual impact”, and “organizational impact”). in contrast, a facet is defined as “a set of attributes (variables) that together represent underlying conceptual and semantic components within a content universe” (zhang et al. 2016, p. 6). several facets are used together not to define a particular construct, but to compare and contrast a number australasian journal of information systems lim, zhang, gable & rai 2016, vol 20, post publication review principles of construct clarity 3 of constructs. in the example illustrated by zhang et al. (2016, p. 6), two facets (i.e., “evaluative response” and “evaluative target”) are used to differentiate three constructs (i.e., “user satisfaction”, “user information satisfaction”, and “end-user computing satisfaction”). it makes little sense to say, for instance, “evaluative response” and “evaluative target” are two dimensions of “user satisfaction”, because neither are “evaluative response” and “evaluative target” constructs, nor do they have variances in a particular empirical setting (e.g., “evaluative response” has a constant value of either “attitude” or “belief” in a setting). a construct can always be compared to and contrasted with other similar constructs through the principles suggested, irrespective of its dimensionality. weng marc lim further suggests the principles are limited for not considering the differences in the criteria between positivist and constructivist research traditions. we agree with lim and believe there is further scope to appropriate and extend the guidelines with an eye to the distinctive expectations of positivist and constructivist studies. meng zhang, queensland university of technology, m.zhang@qut.edu.au guy gable, queensland university of technology, g.gable@qut.edu.au arun rai, georgia state university arun.rai@eci.gsu.edu references bergkvist, l., & rossiter, j. r. (2007). the predictive validity of multiple-item versus singleitem measures of the same constructs. journal of marketing research, 44(2), 175-184. doi: http://dx.doi.org/10.1509/jmkr.44.2.175 delone, w. h., and mclean, e. r. 1992. “information systems success: the quest for the dependent variable,” information systems research (3:1), pp. 60–95. edwards, j. r. 2001. “multidimensional constructs in organizational behavior research: an integrative analytical framework,” organizational research methods (4:2), pp. 144– 192. falk, t., hammerschmidt, m., & schepers, j. j. l. (2010). the service quality-satisfaction link revisited: exploring asymmetries and dynamics. journal of the academy of marketing science, 38(3), 288-302. doi: http://dx.doi.org/10.1007/s11747-009-0152-2 gefen, d., & straub, d. (2005). a practical guide to factorial validity using pls-graph: tutorial and annotated example. communications of the association for information systems, 16 (1), 91-109. guttman, l. (1954a). an outline of some new methodology for social research. public opinion quarterly, 18(4), 395-404. guttman, l. (1954b). a new approach to factor analysis: the radex. in mathematical thinking in the social sciences, p. f. lazarsfeld (ed.), new york, ny, us: free press, 258-348. doi: http://dx.doi.org/10.1086/266532 guttman, l. (1971). measurement as structural theory. psychometrika 36(4), 329-347. doi: http://dx.doi.org/10.1007/bf02291362 guttman, r., & greenbaum, c. w. (1998). facet theory: its development and current status. european psychologist 3(1), 13-36. doi: http://dx.doi.org/10.1027/1016-9040.3.1.13 australasian journal of information systems lim, zhang, gable & rai 2016, vol 20, post publication review principles of construct clarity 4 kuo, b., & dick, g. (2010). the greening of organizational it: what makes a difference? australasian journal of information systems, 16(2), 81-92. doi: http://dx.doi.org/10.3127/ajis.v16i2.592 mackenzie, s. b., podsakoff, p. m., & podsakoff, n. p. (2011). construct measurement and validation procedures in mis and behavioral research: integrating new and existing techniques. mis quarterly, 35(2), 293-334. markus, m. l., & saunders, c. (2007). editor’s comments: looking for a few good concepts…and theories…for the information systems field. mis quarterly, 31(2), 3-6. mcgrath, j. e. (1968). a multi-facet approach to classification of individual group and organizational concepts. in b. indik and k. berrien (ed.), people, groups, and organizations, new york, ny, us: teachers college press, 191-215. polites, g. l., roberts, n., and thatcher, j. 2012. “conceptualizing models using multidimensional constructs: a review and guidelines for their use,” european journal of information systems (21:1), pp. 22–48. sedera, d. (2016). does size matter? the implications of firm size on enterprise systems success. australasian journal of information systems, 20. doi: http://dx.doi.org/10.3127/ajis.v20i0.1224 suddaby, r. (2010). editor’s comments: construct clarity in theories of management and organization. academy of management review, 35(3), 346-357. doi: http://dx.doi.org/10.5465/amr.2010.51141319 zhang, m., cable, g., & rai, a. (2016). toward principles of construct clarity: exploring the usefulness of facet theory in guiding conceptualization. australasian journal of information systems, 20. doi: http://dx.doi.org/10.3127/ajis.v20i0.1123 copyright: © 2016 lim, zhang, gable & rai. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems palekar, atapattu & sedera 2015, vol. 19, pp. 41-53 black markets of pooled assets unveiling the black markets of pooled assets shailesh palekar queensland university of technology s.palekar@qut.edu.au maura atapattu queensland university of technology darshana sedera queensland university of technology abstract presence of black markets is not common for every industry but is a unique phenomenon in the industries such as asset pooling and leasing services. the unique business models and the asset flows that we see in such industries are susceptible for such threats and reveals the nature and extent of extent of industry-specific threats. this paper employs agility lens (overby et al. 2006; roberts and grover 2012) to understand how such firms could address the issue of black market threats with the help of network structure. through a case study of a global asset pooling and leasing company, we reveal the criticality of network structures, the difficulties, inadequacies and impracticalities of current tracking technologies that challenge firms in minimizing such threats. keywords: black markets, network structures, asset pooling, impact of it, responding 1 introduction in present-day hyper competitive business environments, firms need to sense and respond quickly to dynamic changes in business environments. this has become a strategic imperative for contemporary organizations (atapattu & sedera, 2014; overby, bharadwaj, & sambamurthy, 2006). however, possessing sensing and responding capabilities is inadequate for achieving competitive advantage. rather, organizations need to strongly match the sensing capabilities with the responding capabilities to become significantly dexterous in combating precarious and competitive conditions (roberts & grover, 2012). extant literature on enterprise agility has predominantly focused on sensing and responding to opportunities (chakravarty, grewal, & sambamurthy, 2013; roberts & grover, 2012) while innately ignoring its criticality in addressing threats. also, studies on agility in the information systems (is) discipline are mostly fixated on demonstrating the sensing-responding capabilities of firms for opportunities and on how firms could optimize their performance (sambamurthy, bharadwaj, & grover, 2003; saraf, christoph schlueter, & gosain, 2007). in the process, research on sensing and responding to threats has received less attention. we are thus motivated in investigating industry-specific threats and how firms sense and respond to such threats through value network structures and by integrating information technology (it) into business functions. collectively, sensing and responding to threats provide a novel way for achieving agility in addressing risks and reducing the imminent dangers to businesses. in such threatbased scenarios, a lack of agility is evident from negative firm performances such as revenue losses. 2 threats threats are defined in myriad ways such as coercive acts eliciting negative outcomes that intend to inflict harm or loss1. they are also described as risks indicating approaching or imminent dangers, events or the likelihood of events occurring that have negative consequences such as loss or damage to life and property. threats are also seen as acts 1 http://en.wikipedia.org/wiki/threat 41 australasian journal of information systems palekar, atapattu & sedera 2015, vol. 19, pp. 41-53 black markets of pooled assets (malicious or otherwise) that breach the security parameters of systems such as a computer or a social environment through know or unknown vulnerabilities. such acts are enforced through communications or threatening behaviours that force compliance, restrict the freedom of others, inflict harm or damage property2. as an “expression of interest to harm or act out violently against someone or something, it (threats) can be spoken, written, or symbolic”. these can be caused by factors internal to the organization or triggered by external factors in the environment (hill & westbrook, 1997). thus, threats are malicious things, risks, acts, occurrences or elements that may or may not exist or happen but have the capability and potential to impose or cause harm or damage and other negative outcomes3 (meier et al., 2009). threats are not created equally. as an “expression of intent to harm or act out violently against someone or something, (threats) it can be spoken, written, or symbolic” (jimerson, brock, & cowan, 2005). they are characterized by three key components: (1) types: a direct threat is a specific act or occurrence against a specific target (e.g. property, people, organization, environment) and happens in a clear, explicit and straightforward way. in comparison indirect threats are vague, unclear and inconspicuous that are tentative and suggest negative or violent occurrences. a veiled threat is strongly implied or threatened but may not explicitly happen whereas a conditional threat is a warning of a malicious or violent act that could happen if terms or conditions are not met (2) levels: a low level is suggestive of minimal risk where the incidence of occurrence is vague, lacks detailed information and realistic probability. in comparison, a medium level is more direct, potential and probable where a general indication of its potential occurrence is more certain and realistic. a high level poses a direct, credible and imminent danger of its occurrence. this level is indicative of the threat being clearly known and predictable through historically observed and recurrent patterns of occurrences. a very high level of threat is well-defined and expected to occur based on prior analysis of conditions and factors that strongly suggest its occurrence (3) extent: the extent of a threat is generally based on the vulnerability of the target (e.g. organization, people, property, environment) in countering the impact of the threat. a devastating impact is where the threat causes large-scale destruction completely destroying the target beyond habitable use or restoration. in comparison, a severe impact denotes damage caused by the threat, which is serious but sustainable. in such cases the target is susceptible to damage but remains intact and can be rebuilt or reorganized. a noticeable impact is where the threat temporarily disrupts the target causing minimal damage. in such cases, the target is able to sustain through short periods of interruptions. a minor impact causes no significant damage to the target other than creating heightened awareness of the threat and initiating risk assessment exercises and analyses (renfroe & smith, 2011). this paper investigates the unique threat of black markets related to pooled assets. pooled assets refer to the shared use of high quality durable assets such as farming tools, truck trailers, and equipment used for hauling, storing and transporting goods (e.g. wooden pallets, metal containers) by multiple clients that hire or rent such assets. however, pooled assets such as wooden pallets or metal and plastic containers are highly susceptible to leakages from their asset lease pools due to pilferages or negligence in handling the assets by network partners. when pooled assets trickle from their lease pools into ungoverned locations such as open farmers markets or storage yards of small-scale retailers, the identity of the assets together with their geographical locations, movements and use remain concealed and do not get reported to the leasing company. the assets then either idle or are used free of costs by people and organizations that are not connected to the leasing company. we term such locations 2 http://www.businessdictionary.com/definition/threat.html 3 http://www.techopedia.com/definition/25263/threat 42 australasian journal of information systems palekar, atapattu & sedera 2015, vol. 19, pp. 41-53 black markets of pooled assets collectively as black markets of pooled assets. such black markets can be termed as veiled threats to asset pooling and leasing companies (aplcs) and pose moderate to high risk. the collective impact of this threat over a period of time is considered significant and severe. thus, this research specifically investigates: (1) how black markets of pooled assets are formed? (2) what is the impact of a black market of pooled assets on firm performance? (3) what are the inadequacies of information technologies in responding to the threat of black markets of pooled assets? this study is scoped to investigate the threat of black markets in a unique business-to-business (b2b) scenario, where (i) pooled assets such as wooden pallets are not sold but leased to various clients, and (ii) there is no collective enforcement (e.g. it-enabled solutions, management strategies) for identifying, restraining or dismantling such black markets. 3 asset leasing and pooling asset pooling and leasing companies (aplcs) offer assets on lease through a structured network of clients that form an asset lease pool. the ownership and control of assets in such lease pools is retained by the aplc. the operationalization of a leased asset pool is illustrated in figure 1, wherein for example, aplcs lease metal containers to manufacturers who use it for storing and transporting goods to their wholesalers and in turn pass those back to aplcs after using it (refer pool a in figure 1). similar pools exist for other leased assets such as reusable plastic containers (refer pool b in figure 1) and wooden pallets (refer pool c in figure 1). figure 1. asset lease pools pooled assets constitute a strategic option for network partners (e.g. manufacturers, retailers, wholesalers and service providers) in an asset pool because pooled assets (a) reduce capital expenditure incurred on owning the assets (b) decrease risks and costs associated with asset repairs and replacements (c) lessen the burden for ensuring environmentally friendly impacts of storing and servicing the assets, and (d) increase the potential for generating revenues. the benefits of pooled assets to network partners in a leased pool are explained in table 1. 43 australasian journal of information systems palekar, atapattu & sedera 2015, vol. 19, pp. 41-53 black markets of pooled assets types of benefits description (benefits accrued by network partners) cost reduction (capital expenditure and related costs) no capital investment is required for acquiring and maintaining the assets including for information systems in tracking asset movements; firms only pay for the assets used; asset pooling helps in reducing storage, handling and loss of the assets. risk reduction (asset repairs and replacements) network partners do not repair or replace the assets; there is certainty of supply, which reduces the risk of production downtime; there is no obsolete equipment and therefore has no impact on the balance sheet of the organization (e.g. asset depreciation). waste reduction asset pooling reduces environmental waste, as pooled assets do not require packaging and related accessories. revenue increase pooled assets enhance the value and operability of existing goods and services, which in turn increase the competitiveness of goods and services and achieve best market price. table 1. benefits of pooled assets pooled assets are however, highly susceptible to leakages within their asset lease pools due to pilferages or negligence in handling the assets by network partners. for example, a transporter may forget to pick up some quantities of wooden pallets used in transporting goods from suppliers to sellers in an open farmers market. when pooled assets trickle from their lease pools to ungoverned locations such as open farmers markets or storage yards of small-scale retailers, the identity of the assets, geographical locations, movements and use remain concealed from the aplc that leased them in the first place. we term such locations collectively as black markets of pooled assets. assets in such black markets either (i) flow through illegitimate ‘black pools’ where non-clients of aplcs (e.g. small-scale grocery shops, transporters) freely use the assets without paying rent to aplcs while ensuring that assets do not enter official lease pools or (ii) re-enter the official lease pools incognito to both, aplcs and their client networks and adversely impact aplcs, which is indicated by the accumulation of rental revenue losses over time. 4 threat of black markets of pooled assets black markets such as the ones described above are distinct veiled threats specific to the asset pooling and leasing industry. such threats are compounded by the (i) uniqueness of the industry where client networks do not own the pooled assets but merely lease them from aplcs and pass the assets over to other clients in the network after using them. as a result, they take no responsibility for ensuring that assets remain in the lease pool and do not leak into black markets (ii) difficulty in assigning a unique identifier to an individual asset. for example, millions of similar color and type of containers (e.g. plastic and metal) are used daily for storing, hauling and transporting goods (iii) high volume of assets pooled and leased (iv) degree of difficulty in implementing technological solutions for tracking asset movements due to the rugged nature of the asset (e.g. wooden pallets) and additional costs of procuring and implementing it for tracking simple low cost assets (e.g. the average price of producing a wooden pallet is about $25), and (v) slow and sometimes erroneous reporting of asset stocks by client networks, which makes it hard for aplcs to track asset flows and quantities in real time. we focus on pooled assets because we are intrigued by (a) the high demand for simple yet highly robust assets such as metal and reusable plastic containers and wooden pallets (b) the degree of difficulty in making large quantities of such assets individually identifiable (c) the process of assets leaking from legitimate lease pools to black markets (d) challenges in tracking assets that leak from lease pools to black markets, and then re-entering the lease pool (e) 44 australasian journal of information systems palekar, atapattu & sedera 2015, vol. 19, pp. 41-53 black markets of pooled assets accumulation of revenue losses due to assets re-entering the lease pool incognito from the black markets. 5 asset flows and network structures pooled assets generally move in lease pools in two different ways: (1) bi-directional flow (dyadic network): aplcs lease assets to different clients that use them and after using the assets return them back to the respective aplc for servicing or repairs (refer figure 2. a), and (2) unidirectional flow (complex networks): assets such as wooden pallets flow between multiple clients without moving back to the aplc unless the assets require immediate repairs or servicing. in such scenarios, the assets are either (i) transferred between different client locations (e.g. large retailers hire pallets for carrying goods when delivering them to various stores) or (ii) transferred between different clients and their locations (refer figure 2. b). for example, a farmer hires plastic reusable containers from an aplc to deliver fruits to wholesalers, who in turn use the same containers to deliver the goods to various retailers that in turn use the same to deliver goods to their various stores. in this type of asset flow, assets move between network partners for an indefinite period of time. assets are returned to aplcs if the network partners (clients of aplcs) find that they have excess stock or assets require servicing, replacements or repairs. black markets for pooled assets originate specifically during the unidirectional flow of assets wherein assets leak from the lease pool comprising a large network of clients and other value partners (e.g. transporters). figure 2. network structures and asset flows 5.1 characteristics of network structures in lease pools the attributes of network structures in lease pools are derived by understanding the network’s architecture, which is conceptualized through (a) nodes comprising the networks (b) ties connecting the nodes, and (c) patterns and relationships resulting from the connections between nodes. such network architecture is associated with the “characteristics, identities and number of nodes; location, content and strength of ties; and the patterns of interconnections among nodes” (ahuja, soda, & zaheer, 2011). network structures change, dissolve or modify with time. so, their evolutionary patterns are derived through basic factors that influence and shape the ties and nodes in the network. based on the above, we characterize network structures in lease pools through the following elements. (1) agency: this is described as motivations and abilities of aplcs to shape relationships and form beneficial links or discontinue unprofitable ones with network partners (sewell, 1992). aplcs are highly motivated in creating, improving and retaining client networks and partners for commercial gains. however, the concept of agency between 45 australasian journal of information systems palekar, atapattu & sedera 2015, vol. 19, pp. 41-53 black markets of pooled assets nodes (i.e. clients comprising network partners) is rather passive because the motivation for retaining ties within network partners is only for using pooled assets rather than fulfilling financial goals. (2) opportunity: this reflects the structural context of actions where network partners prefer to be linked within groups (than be left out) in order to benefit from activities that flow within the network. for example, wholesalers prefer to be linked with transporters that not only provide (i.e. pass on) pooled assets to them but also transport the wholesalers’ goods to retailers (e.g. transporting goods on pallets). therefore, by linking to a value partner in the network, a wholesaler not only obtains a pooled asset but also additional services such as delivery and logistical support. such opportunities are available to all parties in the network. (3) inertia: this includes pressures for change by reflecting the “durability of structures and processes” that influence and restrict the focal firm’s actions (ahuja et al., 2011). for example, actions or activities by network partners in lease pools to collectively implement or disapprove it solutions for managing the assets can influence or restrict the actions of aplcs. the unique nature of the asset pooling and leasing industry and its network structure and the high voluminous uptake of pooled assets (e.g. about 40 million wooden pallets are leased in australia each year)4 pose two elusive challenges to aplcs. first, there is a perennial leakage of pooled assets from lease pools to black markets (i.e. ungoverned locations such as open markets). in such cases, the assets are deemed lost although they potentially re-enter lease pools incognito. lost or idling assets in black markets do not adversely affect the revenue stream of aplcs, as clients generally compensate the loss. however, assets from black markets re-entering the lease pools pose huge problem for aplcs. first, the assets are not individually identifiable making it difficult for network partners to know their origins. as a result, the network partners use the assets, intentionally or unintentionally. in such cases, aplcs lose future accruable rental revenues although the assets are used within and outside of lease pools. second, aplcs need data from their networks to track and plan asset movements and quantities, which are based on information flows from clients. however, the expected level of responsiveness from certain businesses such as for instance those involved in farming are deemed low due to (i) lack of data and asset management systems (ii) differing levels of literacy (iii) undisciplined approach for stock and asset checking and (iv) lack of expertise in compiling and communicating information digitally. 6 research design in order to capture the richness of our research topic and understand how black markets are formed and their impact on aplcs, we investigated a global firm specialized in asset pooling and leasing services. this provided a grounded approach for understanding how firms engage in responsive actions for countering risks and threats to their businesses. it also helped our study in exploring radical changes that firms make to their existing capabilities (e.g. technologies) when responding to perceived threats. 6.1 case organization our case organization (referred hereafter as ap&l inc.) is a global asset pooling and leasing company with over five decades of experience in pooling and leasing assets. it leases different types of assets (e.g. plastic reusable containers, wooden pallets) in over 50 countries, and employs more than 10,000 employees spanning over 1,000 locations including their service centers. on a global scale the company had revenues over us$5 billion (as on 30 june 2012). ap&l inc. provided an excellent setting for capturing the inherent feasibility of present-day technologies in addressing the threat of black markets of pooled assets. 4 http://www.ibisworld.com.au 46 australasian journal of information systems palekar, atapattu & sedera 2015, vol. 19, pp. 41-53 black markets of pooled assets 7 findings based on interviews with senior executives of ap&l inc., and information collected through industry-related reports, we found that a critical attribute of sensing the threat of black markets of pooled assets was in understanding asset flows and their vulnerability to leaking from lease pools into potential black markets. ap&l inc. endeavored to sense the threat of such black markets through timeliness and accuracy in sharing and communicating asset flow information within their client networks. we also found that responding to the threat of black markets of pooled assets involves taking pervasive actions, collectively as a network, to reduce its negative impact on business. this involves (i) reducing asset leakages (ii) recovering lost assets, and (iii) tightening information flows within the network structures. for ap&l inc., responsive actions for addressing black markets revolved around implementing it-enabled tracking solutions. however, it was found that to achieve this, nodes within network structures needed to be ready, collaborative and closely linked to the same objectives of eliminating black markets. integrating the network structures through a feasible it solution was (and is) deemed the best way to tackle the black markets of pooled assets. asset tracking: for ap&l inc., asset tracking and recovery is extremely important, as it affirms that their assets are performance worthy and safe as well as identifiable at locations where they are susceptible to enter the black market. in this regard, some of core issues that ap&l inc. face are (a) the unidirectional nature of the lease pool wherein pooled assets moves between several locations and clients make it susceptible to pilferage, misuse and theft and (b) the equipment itself, which in several instances is not individually identifiable (e.g. containers, pallets) goes through volatile and rapidly changing situations and environments accruing wear and tear. asset tracking is a challenging task faced by the company given the complexity of how revenue losses accrue due to asset leakages and specifically those that re-enter the leased pool and remain unidentifiable and unaccounted. as the average cost of producing a pooled asset such as a wooden pallet is low (approximately a$25), embedding any identifiable coding mechanism increases the leased value of the asset substantially. in addition, other mandatory related costs incurred in developing it infrastructures are significant, which client networks connected to lease pools are generally unwilling to share. the rugged nature of pooling assets (e.g. wooden pallets) make it difficult to embed any type of it enabled system-readable code including sophisticated radio frequency identification (rfid) tags. due to the high number of pooling assets in operation (e.g. about 40 million wooden pallets are leased in australia each year)5 implanting any individually identifiable materials on them is an expensive and timeconsuming task. in addition, any it initiatives undertaken by the company also require network partners (e.g. warehouse staff, transporters) to adopt and use the technology, which can be rather daunting given the differing levels of literacy, time and resource availability of the network partners. “we give the assets (equipment’s) on lease. we don’t pass the ownership at any point. we are no different from a rental car company.” “we encourage customers to keep the assets (equipment’s) as long as they want. the more they have it, the more rent we receive.” “we don’t sell assets (equipment’s). we own a pool.” quotes a 5 http://www.ibisworld.com.au (pallets and other wood products manufacturing in australia) 47 australasian journal of information systems palekar, atapattu & sedera 2015, vol. 19, pp. 41-53 black markets of pooled assets 8 formation of black markets of pooled assets a black market is formed when assets leak from lease pools either through deliberate action (e.g. pilferage, theft, vandalism) or unintended behaviors such as forgetting to transport (i.e. return) empty pallets or leaving behind a few pallets in less regulated locations such as open markets. in such situations, ap&l inc. deems the assets as lost and requisite compensation from their client. when assets flow into the black market (i.e. non-governed places such as the backyard of a retailer or trucker or open farmer markets) they idle or somebody eventually picks them up and begins to use them. assets picked from the black market are used for individual purposes through a black pool where ap&l inc. has no control or knowledge of the assets although they own the assets. also, the finder of the assets in black markets either uses the assets without informing ap&l inc. or sells them to other interested parties. alternatively, those picking up idling assets in black markets sometimes return them to businesses that may or may not be clients of ap&l inc. this scenario is of utmost concern to ap&l inc. because assets re-entering the lease pool may not provide any rental revenues to ap&l inc. if they are not reported as having come from a black market. this happens because clients may not know the origin of the assets, as they are not individually identifiable (e.g. wooden pallets). when a large number of assets re-entering the lease pool from the black market go unnoticed by ap&l inc. but remain in active use within the lease pool, they begin to accumulate rental revenue losses for ap&l inc. we depict a scenario to understand the formation of a black market of pooled assets, the eventually re-entry of the leaked assets into the leased pool and the impact on the revenues of ap&l inc. 8.1 the scenario ap&l inc. issues assets to client a – generally a farmer, producer or a manufacturer. client a then uses the assets and in turn passes it on to client c – generally a retailer or distributor through client b generally a transporter. within this movement some quantities of the assets potentially leak out and end up in less governed places such as the backyard of another retailer or in a corner shop storage area or an open farmers market or in the yard of the transporter (trucker) (which may not be a client of ap&l inc.). in addition, idling assets are also prone to be picked up by other parties either intentionally or unintentionally. when assets leak out from the lease pool in this manner, they accumulate over a period of time into a substantial pool or black market. refer figure 3 and quotes b. figure 3. asset leakage and formation of black market of pooled assets 48 australasian journal of information systems palekar, atapattu & sedera 2015, vol. 19, pp. 41-53 black markets of pooled assets “black pool is a big pool in millions of dollars (of) lost revenue.” “we lose them (assets) in less governed places like markets, farmers, grocery stores, which are not our accounts.” “(we do) not necessarily lose revenue if assets (assets) are idling because they (clients) are paying money for not using it.” “we don’t have visibility any more (assets) and we have given it up” “we have no knowledge of where our assets are.” quotes b 8.2 impact on ap&l inc. assets leaking from the lease pool, their subsequent re-entry into the lease pool and the impact on ap&l inc.’s revenues is now explained through the below scenario. impact 1: when a company c that is not a client of ap&l inc. realize that they have lost some assets, they inform client-b of ap&l inc. (from whom they received the assets) and offer compensation. in turn, client-b informs client-a and passes the compensatory amount. clienta realizing that it has lost some assets quantities informs ap&l inc. and passes the compensatory amount for the lost quantities to ap&l inc. in return, ap&l inc. replaces the lost asset quantities with new quantities. so, the contractual agreement between ap&l inc. and client a remains unaffected, which in turn maintains the revenue stream (refer figure 4. impact 1-compensation). however, the lost assets are susceptible to be utilized elsewhere knowingly or unknowingly. hence, ap&l inc. loses potential rental revenue. impact 2: some of the previously lost quantities of the asset (from the black market) now reenters the lease pool (at some point in time) and end up with client-d through a transporter (company-e) who might not be a client of ap&l inc. client-d finds excess asset quantities in their premises and presume them to be surplus stock (not knowing that it has come from the black market). so, they send some of the quantities back to ap&l inc. requesting a reduction to their original contracted asset stock. by doing so, client-d ends up paying less rental revenue for the asset although they have the same quantities of the asset originally contracted. therefore, the actual asset usage by client-d does not reduce. but, they end up paying less rent for their original stock in use. in this scenario, ap&l inc. loses future revenues because it continues to receive less rent from client-d (refer figure 4. impact 2 – excess assets returned). refer quotes c. “(the) challenge is we don’t know whose equipment’s are coming back. one-on-one correlation in a dynamic supply chain is hard to make.” “when equipment’s are lost somebody will have that (equipment). it (the equipment) may come back to the pool but without paying rent.” quotes c 49 australasian journal of information systems palekar, atapattu & sedera 2015, vol. 19, pp. 41-53 black markets of pooled assets figure 4. impact of black market of pooled assets on ap&l inc. lease pools, especially those that feature unidirectional flow of pooled assets depict complexity because (i) assets remain in the pool for an indefinite period of time making it difficult for ap&l inc. to monitor their movements (ii) assets move between several clients and locations making it hard for ap&l inc. to control the assets, specifically those that leak from the lease pools, and (iii) ap&l inc. clients (network partners) have no ownership over assets and therefore do not make adequate efforts to secure, use or manage them appropriately. this puts extra burden on ap&l inc. to aggressively track the assets and also ensure that the assets remain in their lease pools in order to maintain revenue continuity. refer quotes d. “pallets can flow (within the leased pool) without coming back to us until it is broken or needs servicing or the retailer, manufacturer has no utility for it. managing the assets (pallets) is a problem for the company.” “asset (pallet) pool gets complicated (when) assets (pallets) move from coke to woolworths to 7/11. we do not have any control over the assets (pallets).” “the challenge with every customer is that it is not their core product. they do not have to maintain, service, track, take care of it.” quotes d 9 challenges in implementing tracking technologies in response to the perennial problem of asset tracking to plugging leakages from lease pools into black markets and reduce subsequent revenue losses, ap&l inc. explored several it initiatives. historically, asset tracking is done generally through manual paper-based mechanisms. clients compile asset stocks and reports manually and fax or send them by surface mail to ap&l inc. this apparently causes delays as well as discrepancies in information flows due to the length of time taken, inaccurate compilation of data, misplaced reports and reporting errors. in order to address those issues, ap&l inc. trialed various types of technologies such as universal product codes (upc) barcodes and radio frequency identification (rfid) tags for facilitating asset identification and movements. however, such it initiatives have yielded mixed results, as the mentioned technologies have proved rather 50 australasian journal of information systems palekar, atapattu & sedera 2015, vol. 19, pp. 41-53 black markets of pooled assets impracticable in certain types of leased assets such as plastic crates, containers and wooden pallets. as a result, tracking and preventing assets leaking into black markets has proved to be difficult. [quote e: “we can’t eliminate the black market but only keep it in control”]. tracking capabilities is a strategic responsive action in managing pooled and leased assets than merely policing black markets. some it initiatives undertaken by ap&l inc. for tracking their assets are discussed below. [note: information was also extracted from industry-based reports (e.g. http://www.ibisworld.com.au).] universal product codes (upcs): although upc’s are successfully used in the retail industry (since its introduction in the 1970’s) barcodes are found to be unfeasible in identifying and tracking pooled assets. for example, barcode scanning usually requires a person to hold the item in front of a barcode scanner or point a scanner directly at the barcode in a proper reading angle within a fairly short distance (max 2 feet). besides, the scanning can be done only one item at a time. for ap&l inc., barcode scanning thousands of assets on a daily basis was difficult because (i) assets were mostly stacked beyond the readable distances of scanners or were hidden behind other assets such as pallets used for storing, hauling or transporting them (ii) scanning required more human resources to scan each asset at many different locations, and (iii) the amount of time spent in scanning meant longer lead-times for asset flows and reporting. radio frequency identification (rfid) tags: rfids resolved some drawbacks of upcs. for example, rfid tags could be read from up to 10 feet away without being in the line of sight. it was also possible to perform multiple tag readings with a single scan by simply walking or driving along the assets with a tag reader. however, rfids had several drawbacks: (i) proper readings were not possible when water or metal blocked radio waves (ii) attaching tags to certain assets (e.g. wooden pallets) was not practically feasible as glue could not properly fix the tag to the asset surfaces while placing tags inside wood or metal-based assets was not practical as it masked and interfered with radio frequency signals. a realistic solution was affixing two tags to an asset but that was deemed expensive, as clients required spending up to $10,000 per reader portal (unit required for rfid scanning) and other rfid related infrastructure development. although rfid had the potential to provide benefits, ap&l inc.’s clients were not willing to share financial costs. they saw no visible and immediate tangible benefits [quote f: “they (clients in the supply-chain) don’t have the kind of maturity to absorb technology without incurring costs”]. ubiquitous technologies: ap&l inc. is experimenting with new dynamic technologies such as mobile apps and social media to manage pooled assets in order to prevent it from leaking into black markets. however, there is scepticism on whether such technologies can improve asset tracking and reduce leakages into black markets. ap&l inc. is presently trialling mobile apps but is unsure whether that will yield expected outcomes for tracking assets in real time and stop assets from leaking into black markets. [quote g: “mobile apps do not track anything. but it can help in timely reporting (and) improve the accuracy in reporting so that when there is an issue, you can zero down on the page where there is a leakage”]. besides, how much a firm can reduce its revenue losses by implementing mobile apps remains uncertain. [quote h: “we still haven’t got concrete numbers to say that (the threat of) black market has reduced”]. 10 conclusions the context of our study (i.e. asset pooling and leasing service) investigated a unique industryspecific threat viz. black market of pooled assets. in the process we extended the theoretical understanding of firm agility (roberts & grover, 2012) by focusing exclusively on (a) sensing and responding to threats (b) reducing revenue losses as a firm’s performance indicator (c) the uniqueness of black markets as a potential threat, and (d) asset pooling and leasing as a novel context for framing how firms and network structures aided by it can sense and respond to threats. broadly, our investigations and insights suggest a theoretical mismatch between the 51 australasian journal of information systems palekar, atapattu & sedera 2015, vol. 19, pp. 41-53 black markets of pooled assets sensing and responding capabilities of firms towards the threat of black markets. while firms are highly aware and sensitive to the threat of black markets, their responsive actions are rather inadequate when dealing with such threats. our findings evidence that firms struggle to respond to industry-specific threats due to the unsuitability of technologies in providing assettracking capabilities. therefore, it in the context of our study is found to be unsuitable in mediating the sense-response capabilities of the firm in addressing threats. this reflects the importance of responding to threats in dynamic and hypercompetitive environments and the necessity of more investigations for understanding the nomological networks surrounding sensing and responding to threats. this would include precise antecedents to (i) sensing capabilities such as information flows and the role of technology in tapping supply-chain network partners, and (ii) responding capabilities such as dexterous it solutions, integrative structures, people and processes to counter threats. pooled assets comprising heavy equipment and machinery are excluded from the scope and extent of this study. although these assets are prone to theft and damage, they are easily recognizable, as they have identifiable indictors (e.g. codes, serial numbers) embedded or attached to them. this makes them highly visible. they are verifiable and can be easily accounted in routine stock checks. references ahuja, g., soda, g., & zaheer, a. (2011). the genesis and dynamics of organizational networks. organization science, 23(2), 434-448. doi: 10.1287/orsc.1110.0695 atapattu, m., & sedera, d. (2014). agility in consumer retail: sense-response alignment through the eyes of customers. australasian journal of information systems. 18(2) doi: 10.3127/ajis.v18i2.866 chakravarty, a., grewal, r., & sambamurthy, v. (2013). information technology competencies, organizational agility, and firm performance: enabling and facilitating roles. information systems research, 24(4), 976-997. doi: 10.1287/isre.2013.0500 hill, t., & westbrook, r. (1997). swot analysis: it's time for a product recall. long range planning, 30(1), 46-52. doi: 10.1016/s0024-6301(96)00095-7 jimerson, s. r., brock, s. e., & cowan, c. c. (2005). threat assessment: an essential component of a comprehensive safe school program. student counseling, october, pp.11-15 (www.nasponline.org/resources/principals/nassp_threat.pdf). meier, j. d., farre, c., taylor, j., bansode, p., gregersen, s., sundararajan, m., & boucher, r. (2009). improving web services security: scenarios and implementation guidance for wcf. united states: microsoft corporation. overby, e., bharadwaj, a., & sambamurthy, v. (2006). enterprise agility and the enabling role of information technology. european journal of information systems, 15(2), 120-131. doi: 10.1057/palgrave.ejis.3000600 renfroe, n. a., & smith, j. l. (2011). threat/vulnerability assessments and risk analysis. washington, dc: national institute of building sciences. roberts, n., & grover, v. (2012). investigating firm's customer agility and firm performance: the importance of aligning sense and respond capabilities. journal of business research, 65(5), 579-585. doi: 10.1016/j.jbusres.2011.02.009 sambamurthy, v., bharadwaj, a., & grover, v. (2003). shaping agility through digital options: reconceptualizing the role of information technology in contemporary firms. mis quarterly, 27(2), 237-263. saraf, n., christoph schlueter, l., & gosain, s. (2007). is application capabilities and relational value in interfirm partnerships. information systems research, 18(3), 320339,363,365. 52 australasian journal of information systems palekar, atapattu & sedera 2015, vol. 19, pp. 41-53 black markets of pooled assets sewell, w. h., jr. (1992). a theory of structure: duality, agency, and transformation. american journal of sociology, 98(1), 1-29. doi: 10.2307/2781191 copyright: © 2015 atapattu, palekar, sedera. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. 53 http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 threats 3 asset leasing and pooling 4 threat of black markets of pooled assets 5 asset flows and network structures 5.1 characteristics of network structures in lease pools 6 research design 6.1 case organization 7 findings 8 formation of black markets of pooled assets 8.1 the scenario 8.2 impact on ap&l inc. 9 challenges in implementing tracking technologies 10 conclusions business process redesign using edi: the bhp steel experience paula m.c. swatman department of information management and marketing university of western australia nedlands, w.a. 6009. abstract edi should not be seen as a competitive weapon it does not offer a sustainable competitive advantage to its users as did the earlier inter-organisational systems (such as the various airline reservation systems). instead edi provides a standardised infrastructure upon which an adopting organisation can reengineer its business processes and thus confers a long-term, strategic and comparative advantage upon such an adopter. the potential for sustained competitive advantage to be derived from edi arises from its integration with the organisational structure of its implementor, depending upon the implementing organisation's ability to redesign its business processes appropriately. this paper examines the case of (arguably) the most sophisticated edi-using company in australia, bhp steel, demonstrating the advantages obtainable by an organisation using edi as a foundation for its reengineered organisational structure and business processes and concludes that: • there are real benefits to be obtained from an inter-organisational electronic trading strategy founded on edi as the enabling mechanism benefits which are considerably more extensive than edi's comparatively simple technical nature would appear to offer; • organisational gateways utilising an application generic approach to system integration and isolating the communications issue by insisting on the use of international data communications standards do offer a realistic and successful solution to the problem of internal and external trade for large and sophisticated organisations; • the majority of these achievements would have been either unlikely or totally impossible without bhp steel's commitment to organisation-wide business process redesign. introduction the major problems associated with involvement in edi are not primarily technical in nature, but may rather be the result of inefficiencies in corporate information systems analysis and design or, alternatively, might result from a lack of top-level management support (this postulate was first discussed in swatman and clarke, 1990; and further developed in swatman and swatman, 1991a and swatman et al, 1993). this lack of organisational acceptance, in turn, implies that the broad organisational integration and re-evaluation of structure and function which wilmot (1988) and benjamin, de long and scott morton (1990) regard as essential for strategic gains from edi may require further exploration. it has now become almost an article of faith within the edi community to claim that "edi is 90% business and 10% technology". a wide variety of writers within both the trade and academic sectors provide support for the view that edi should be regarded as a strategic issue, rather than as a technical problem (see, for example, sadwani and sarhan, 1987; robinson and stanton, 1987; patrick, 1988; lyttle, 1988; skagen, 1989; rochester, 1989; emmelhainz, 1990; swatman and swatman, 1991a; 1991b; 199 ic; vogel, 1991). the failure to validate this view is largely due to the difficulties of separating edi's technical aspects from the organisational issues of implementation and integration. prospective users of edi have tended to view both aspects of the total edi system as falling within the domain of their own i.s. department. since the creation of an edi communications system is unquestionably a complex and difficult task, such a technical difficulty naturally becomes the focus of concern within a technicallyoriented department the results of multiple case study research reported in emmelhainz (1987), benjamin et al (1990) and swatman (1993) provide some support for this view, indicating that larger organisations in the u.s. and australia are taking an increasingly sophisticated view of edi's role as an enabler of long-term change. these case studies do not, however, provide a detailed picture of the decisions made by such organisations or of their longer-term, edi-based strategies. this paper is intended to extend that 55 ajis picture, describing a single, in-depth case study which complements the earlier research and offers insights into both the potential which edi holds for business process redesign and into the likely pattern of such edi integration. the paper is therefore a description of one company's experience of using edi as an enabling infrastructure for edi later research will explore the generality of the conclusions drawn from this exemplar case study. clearly, in an environment such as australia's, where fully-integrated edi is still in its infancy, any organisation which had consolidated edi with its business processes would be suitable for such an investigation. my aim was, however, to find an organisation which had: • reached a level of maturity sufficient to enable successful planning for technical and business process redesign; and • taken a considered decision to develop an integrated approach to edi and interorganisational systems a serendipitous fall into integrated edi would not be sufficient. identifying an organisation's level of maturity was the more difficult of these two conditions to meet. in defining the notws framework, scott morton (1991) describes organisations in terms of five forces which remain in dynamic equilibrium as the organisational context changes. of these five forces scott morton considers management processes, structure and individuals and their roles to be critical to any organisational transformation. i have already argued (in swatman and swatman, 1992) that if organisations are to change as a result of evolving technology, they must focus on the culture of the organisation (as represented by the organisational structure and by the roles and tasks of individuals). it is clear, therefore, that organisations which wish to enter a considered transformation (which can be equated with the term "re-engineering", or with the concept of business process redesign), must take a holistic view of these five forces and theninteraction. an appropriate candidate for this case study, then, must be aware of the implications which an integrated approach to edi holds for management processes, organisational structure and individuals within the organisation. galliers (1991) points out that is strategy should be strongly embedded in business strategy, feeding off and into the business strategy process which in turn feeds off and into the organisation's business environment. further assistance in identifying an organisation capable of acting as a "critical" case study of edi integration is provided by galliers and sutherland (1989,1991a, 1991b) in their model of the stages of lt. growth in organisations, which provides a six-level classification of the stages of organisational maturity. only those organisations which have reached stage 5 or 6 of the galliers and sutherland model could possess a sufficiently mature approach to the mtt90's five forces and have the ability to plan for the use of information technology (and edi in particular) as an enabler of business process redesign. to find an australian organisation matching both these desiderata i needed an organisation which had achieved a holistic approach to its management processes, organisational structure and individual staff roles and which had applied this holistic view to long-term planning for an edi-based approach to business process redesign. from the group of possible case study candidates (which included, inter alia, coles-myer, the four automotive manufacturers, australian paper manufacturers (afm) and the australian customs service) i selected the bhp steel electronic trading project (etp). bhp is australia's largest company, with assets of a$22 billion and the bhp steel subsidiary itself is larger than the majority of australian organisations. long before the commencement of the electronic trading project, bhp was already committed to formal planning for both organisational and technological change. the electronic trading project itself was the result of several years of preliminary analysis and strategic planning and followed a development path based on clearly defined goals. an internal bhp document reporting the results of preliminary investigations into the need for such a project stated: "traditionally edi has been driven by customers down to their suppliers, but bhp steel has seen a unique opportunity to be pro-active and initiate edi with our customers. by taking this initiative we can create business benefits to both bhp steel and our customers (the partnership approach) as well as achieving extensive savings in administrations costs, stock holding, error reduction, manufacturing efficiencies ... flowing from this was the realisation that bhp steel need to develop an edi strategy to cover all the elements of electronic links with our customers and how such links could be converted into value-added servicing that can give bhp steel a competitive edge." (bhp steel, 1989:4). a further advantage of the electronic trading project was its development along strictly controlled project management lines, with bhp information technology (bhp it) being contracted as technical may 1994 56 project manager and controller for the project in june 1990 (hunt, 1992). this approach enabled a separation of the strategic, top-management decisions regarding organisational processes from the tactical decisions regarding implementation issues and the resolution of technical problems. background to the electronic trading initiative the electronic trading project rationale the problem bssf although bhp had been using inter-organisational systems for many years, these tended to be nonstandard systems involving very limited groups of users (mosdy drawn from bhp subsidiaries) and having widely varying protocols for data, formats and media. hunt (1992) points out, however, that during the late 1980's a number of bhp steel's divisions became involved in true, mainstreamstandards-based edi projects, using a variety of value-added network providers (vans). as the number of these projects increased it became evident that this decentralised approach was likely to lead to longer-term problems bhp steel's trading partners would increasingly find it necessary to connect to a variety of vans, each potentially using either different standards or differing versions of the same standard. figure 1 (adapted from an unpublished 1990 bhp ft presentation) illustrates the problem of multiple connectivity as the organisation perceived it at that time. it was apparent that bhp steel needed a long-term solution to this problem which would provide both inter and intra-organisational connectivity and which would act as a strategic focus for the organisation's business restructuring. the electronic trading project solution in april 1990, following an 18-month study by a steel group steering committee (composed of senior managers from both business and technical areas within bhp steel divisions) which assessed the edi strategies of other companies world-wide, bhp steel initiated the "electronic trading project" (fuller, 1990). the result of the study, its analysis, discussion with other edi implementors overseas and a formal planning process was the following proposal: 57 ajis "... that bhp steel introduce international standard edi with customers via an electronic trading gateway, in order to gain competitive advantage by focusing on customer service. the gateway approach enables the different divisions of bhp steel to present a common face to all customers at minimal cost and with minimal duplication of effort, and will be a significant step in making bhp steel our customers' preferred supplier on a long-term basis" (bhp steel, 1990a:3). solution import^ ^ $ bhp 610* ebcranb •mm tig gateway figure 2 (adapted from a 1991 bhp steel newsletter) illustrates bhp steel's gateway solution to its multiple connectivity problem. despite the lack of widely-accepted texts relating to the subject of "corporate edi gateways", it would be generally agreed that such a gateway comprises a number of modules providing the management and control facilities needed by large and complex corporate edi systems. these facilities include: • edi message construction and translation:translation between application software package standards and some agreed-upon inhouse "standard" queue management for both in and outbound documents translation between this in-house standard and the various edi document formats (such as ansi x12, tdi or edifact) "compliance checking" of arriving messages to ensure correctness; • transmission between the organisation's internal data communications standards (x.400 or a local area network standard such as ieee 802.3) and those of the recipient, who may be a value-added network service provider (in which case x.25 would be the most likely standard) or a trading partner (who might very well be using x.400 messaging); • session management and directory services maintenance (such as x.500); • full delivery audit facilities; • security and management features; • call-logging facilities, designed to enable the operation of a service desk; • the creation of trading partner relationships and the establishment of trading partner profiles; and may 1994 58 banks, the australian customs service and transport companies) of both bhp steel and other bhp groups. longer-term plans anticipated the extension of the gateway to provide additional electronic services such as email and interactive database access. above all, the gateway was seen as a key element of bhp's communications strategy. another major objective of the project, identified in the original proposal, was the establishment of a steel industry working party (siwp) associated with the edi council of australia (edica) and involving bhp steel, bhp it, edica, major customers and value-added network providers. the working party's mission was to develop a set of agreed standards for the initial edi document set in the form of an industry-wide implementation guide for edi document standards and to involve as many other industry groups as possible. in view of its size and complexity, the electronic trading project was divided into three development stages, which are briefly described below: stage 1 initial pilot this stage had a strong marketing focus initial business documents were selected with the intention of improving ordering efficiency and materials management. activities included: • the development and testing of the electronic trading gateway • initial application link testing by means of inter-divisional pilots (s&cp ordering from sppd and lsi ordering from s&cp) and • a successful edi connection between one steel division and its major customers (tubemakers of australia merchandising division and email metals distribution group). fuller (1990) points out that the strongly market-oriented focus of this stage necessitated the development of a generic trading partner agreement to highlight the rights and responsibilities of each party when adopting electronic trading. two successful cross-industry initiatives have emerged from this work: • because bhp trades with companies in industries as diverse as automotive, heavy engineering, mining, hardware and retail there was a clear need for consistent document sub-sets across all these industry groups. to this end, the siwp and the existing heavy engineering, mining and minerals working party (hemmp) cooperated to produce a common implementation guide for cross-sectoral users, particularly members of the steel services centres of australia who both purchase from and supply to bhp divisions. the siwp has recently changed its name to become the australian metals industry working party (amiwp) a name which better reflects the wider scope of the group; • the construction industry working party comprising developers, architects, builders, fabricators and manufacturers decided to target a specific development project (the darling park building in sydney's darling harbour), both to examine the potential benefits of edi and to seek the involvement of other industries, such as the steel re-inforcement industry. stage 2 full-scale implementation this stage was planned to involve the development of further documents and application links, together with a controlled "roll-out" of the gateway services to trading partners (pilot testing of at least one document with each participating division and at least one of that division's major customers). it was also intended that this stage would include the extension of the project to support any other bhp steel initiatives wishing to use the gateway. four such initiatives were undertaken during stage 2 (each of which is discussed in more detail in later sections of this paper): • the integration of extemally-sourced and corporate business data • the development of the edefact qality test certificate • the incorporation of bar coding and product numbering technologies • the asia pacific economic cooperation (apec) edi pilot may 1994 60 stage 3 project extension this stage of the project, which commenced in mid-1992, includes further development of other electronic trading services such as electronic mail, electronic funds transfer and remote database access, in addition to continued "roll-out" to other major customers. the project was completed in mid 1993, although the steering committee later agreed that a central management role would be required for a further two years to ensure that the existing implementation momentum continued at its current level. the remainder of this paper provides a more detailed explanation of the electronic trading project, covering: • the organisational structure of the project • the electronic trading gateway • in-house integration software the customer trading management system • the development of the edifact electronic test certificate • the apec multi-lateral pilot scheme • the bar coding and product numbering pilot scheme. while there are clear conclusions to be drawn from this single case study, it is also important to note that wider research is required before any conclusive judgments can be obtained. the structure of the et project team b hp steel recognised that to accomplish the shortand long-term objectives of such an ambitious project would require a development strategy which: • had enthusiastic senior management support • took a "staged" approach to development • had realistic development milestones for installation, implementation and integration. etoctnonb tndhy dun tiy commte rauhem imuw ranim) reams team ttnkihg team the selection of b hp it as technical project manager and controller provided a consistent approach to project planning and control. in addition to strong management commitment, however, the project planners found that all parts of the organisation must be involved in the planning and implementation of a project which will impinge upon the activities of all divisions. figure 3 (bhp steel, 199la: 121) illustrates the organisational structure of the etc project: fuller (1992) explains that two organisational strategies, in particular, proved effective in educating and involving participants: 1. the cross-organisation steering committee, which sets edi policy and directs the edi project team through a project manager. the group comprises legal and audit representatives 61 ajis as well as marketing, information systems, supply and, more recently, international and finance division representatives. the group meets bi-monthly and provides the essential edi champion in each key area of the company; 2. the cross-organisation functional teams, which are the real implementation "engine-room" and are integral to achieving the project's objectives. functions include application integration, communication, total quality management, mapping of document standards and system design. the project team has also recommended that bmp business units join edica as subsidiaries and become involved with the relevant edica industry working parties. these strategies are exactly those articulated by writers on business process redesign (see, for example, davenport and short, 1990; hammer, 1990; morrow and hazell, 1992) who advocate the formation of just such cross-functional teams and viewpoints. internal/external document integration the electronic trading gateway (etc) the overall strategy of the gateway is defined in a bhp steel evaluation report as "... the ability to offer a variety of value-added electronic trading services on an international standard communications basis to customers, suppliers and other trading partners ... the electronic trading gateway is viewed as an "enabling" service. it will provide the communications and standards infrastructure on which bhp steel's business units can build their inter-enterprise application systems to the mutual advantage of bhp steel and its trading partners" (bhp steel, 1990b:9_10). the report identifies the key elements of that strategy as including: • a single gateway (a network of both centralised and decentralised multiple nodes) for all electronic transmissions whether via van or directly from individual trading partners, government agencies, etc. • the use of international standards for data communications (iso/ccitt), document translation (ansi x12 and un-edifact), office communications (oda/odif) and messaging (x.400 on x.25) • the provision of osi data communications between the gateway and all appropriate bhp application platforms (dg, digital, hp, ibm, pc's, pc lans, tandem) the provision of x.500 directory interface when available • the ability for bhp steel business applications to support international edi standards for data elements directly • the provision of remote data base access, enabling trading partners to access bhp steel data bases (either stored on the gateway host itself, or via the gateway to bhp steel host systems) • support for encryption and authentication codes for added security (if required). the electronic trading gateway is responsible for translation between the single internal document standard and the multiple external standards used by each application's trading partners each of bhp steel's applications exchanges files of document data with the etg, using x.400 and a single internal format defined by bhp. the gateway provides a true integration facility, enabling not only the association of internal business data with externally-sourced edi documents, but also the synthesis of data across internal applications. the electronic trading gateway offers bhp steel a level of integration equivalent to that of stage 3 in the model of edi integration presented in swatman and swatman (1991c) "seamless integration". the ability to link internal application systems to one another through the gateway, however, offers the possibility of stage 4 integration "business process redesign". the following section of this working paper discusses the process of integration between the gateway and bhp steel's various application systems, not only supporting the view that the etg provides a seamless approach to integrating applications (both internal and external), but also considering the potential for redesigned business processes. the effects of such integration upon organisational structure and implications for business process redesign are then considered a little later in the paper. may 1994 62 internal application integration the customer trading management system14 the customer trading management system (ctms) was designed to manage the electronic transmission of corporate business data from any of the organisation's ibm mainframe application systems to its recipients (either external trading partners, or other internal application systems) through the electronic trading gateway and back again. a universal design for ctms was initially stated in the application interface requirements of the electronic trading project design document (bhp steel, 199la) which has since been tailored to suit the requirements of the sales and marketing systems culminating in the ctms as it is today. this design required applications wishing to send documents to a trading partner (incoming documents from trading partners were to be treated in an analogous fashion) to: • read the application databases and create records in the appropriate bhp internal file format for that document type (e.g. purchase order), converting internal data values to the single external standard • combine these document records into a mail file (multiple documents of differing document types for multiple trading partners could be accumulated into the same mail file) write an image of the mail file to an audit database to enable tracking and error recovery • transmit the mail file to the etc using the mail agent appropriate to that particular application platform. ctms was originally developed to link only a single system to the gateway, but currently links three applications within the steel group, with a future extension to the slab and plate products division test certificate systems anticipated: • the sales administration and order management system (sams), which handles product sales to customers • the steel group invoicing system (sgis) and • the sales reporting and debtors management system (srdm). although these are separate applications, the same group of systems developers handles the mapping of outbound application-specific formats to ctms' consistent bhp-wide internal format (and vice versa for inbound documents) ensuring a consistent approach to the document conversion process for all application systems. bhp steel have therefore decided to implement the more maintainable "application-generic" approach, ensuring a minimum of later integration problems as further applications, databases, external trading partners and documents are added to the system. at first sight, this integration between external documents and applications (e.g. customer purchase orders and the generation of actual orders within bhp steel's manufacturing facilities) merely appears to replace a physical paper trail with its electronic equivalent the bhp steel initiative, however, is intended to do much more than this the gateway and the ctms pilot are the foundation for business process redesign which will effectively revolutionise the organisation's manufacturing and administrative structures, unking internal and external business information in the most effective and timely manner. the organisation's long-term approach to information systems and its edi and electronic trading infrastructures is based upon the view that it is information, not paper-based documents, which provides the life-blood of the organisation. the following sections of this working paper describe three specific, gateway-related projects which were designed to link corporate business information and telecommunications in an effective manner: • the first two projects relate to bhp steel internal initiatives (although both these projects have substantial inter-organisational components) • the third project (the asia pacific economic cooperation (apec) edi pilot) differs from the other bhp initiatives reported since, in this case, bhp steel is merely one member of a multi-organisational group. the apec project is included in this chapter for two reasons: • this is a significant bhp steel electronic trading gateway project and thus throws further light on the electronic trading gateway initiative; and 14 the following brief summary of the gateway's activities is drawn from a number of sources bhp steel 1991a; bhp steel 1991b; internal bhp steel and bhp it documentation: and interviews with bhp it and bhp steel personnel involved in the ctms and gateway development projects. 63 ajis • the multi-lateral integration involved in this project is directly related to the subject of this working paper the integration of edi into internal applications and organisational structure. gateway-related projects business data integration electronic test certificates15 test certificates are formal documents which certify the attributes of steel products (for example, their chemical and mechanical properties). bhp generates around 200,000 of such test certificates annually ranging in content from simple, 1,000-character documents to reports having as many as 8,000 characters for special steel products. certificates are usually needed by bhp customers for one of three major reasons: • to accompany re-sale of products by bhp steel merchants • to set up manufacturing processes • to re-use data (such as chemical analyses) in the generation of test certificates produced by customers themselves. production and management of bhp steel's test certificates was formerly confined to a number of stand-alone test laboratories and computer applications within the organisation. over recent years, the steel and derivative industries have increasingly adopted quality assurance systems, which have made test certificates even more necessary to steel distributors and their customers. simultaneously, however, the costs of paper-based document storage and retrieval have increased substantially the cost of administering a paper-based test certificate system has been estimated collectively by customers at well over $500,000 per annum. this problem was exacerbated by two further considerations: • only 12%-15% of all certificates in storage ever need to be retrieved and • certificates (designed individually at different times) were produced in a wide variety of formats and with a variety of headings and layouts. clearly, a more efficient and less costly method of issuing test certificates was needed. the proposal to issue test certificates by means of edi document transfer resulted from the findings of a cross-organisational review group. the group completely re-engineered the test certification process, recommending a two-stage implementation strategy: • in stage 1, certificates were received by trading partners in conjunction with steel deliveries (the pre-edi arrangement). manufacturers and fabricators gained benefits, however, from selective access to the information contained in the bhp test certificates and to complete certificates at need. in the case of manufacturers who undertake additional processing and require only a portion of the information contained in the certificate, electronic receipt enables the more efficient production of their own certificates avoiding the errors which can be introduced by manual data rekeying, as well as allowing faster data handling and the elimination of paper document copies • in stage 2, additional benefits accrue to trading partners from the establishment of a central database on bhp's electronic trading gateway, from which customers can request certificates "on demand". this service is particularly useful to those customers having only occasional need for physical certificates. the proposal offers overall benefits to users which include: • more rapid test certificate availability reduced handling requirements reduced storage requirements and need for retrieval as a result of "on demand" certificates • significant cost reductions costs to complete both stages of the project come to $100,000 for customers and $350,000 for bhp; potential on-going savings are estimated to be approximately $500,000 p.a. for customers and $100,000 p.a. for 15 the following brief summary of the electronic test certificate project is drawn from a number of sources fuller, 1992; bhp steel, 1992a; 1992b; harvey, 1992; internal bhp steel and bhp it documentation; and interviews with bhp it and bhp steel personnel involved in the test certificate project. may 1994 64 bhp for the foreseeable future, in addition to a saving of $0.30 per test certificate (on average) for bhp. the new aproach, however, necessitated changes to existing bhp systems and procedures in terms of: • the need for a bhp functional team to coordinate test certificate issue • the creation of an edica steel industry working party to coordinate and consolidate work on document standards (the development/amendment of relevant edefact/x12 messages) • necessary changes to test certificate print formats (test certificates are now standardised on a single a4 page format) • new and amended control/audit procedures • education both within bhp steel and of trading partners. maintenance of quality assurance standards in the new system is now achieved by means of: • the existence and application of formal and documented systems for certificate generation at both bhp and customer sites • auditing of customers' certificate generation systems prior to the receipt of bhp approval to produce bhp test certificates • periodic system audits at both bhp and customer sites to ensure adherence to and maintenance of the system. another major achievement of this programme has been the development of the edefact qality test certificate document as a result of the work of the siwp. the test certificate programme (a senior management, rather than it, initiative) has brought together all the parties involved in the production and use of test certificates across organisational boundary lines. the flow of information involved in the test certificate process has taken precedence over the previous paper flow, modifying work practices in both bhp and customer operations. further substantial changes are expected both in the use made of test certificate data and in the application systems geared around this data a clear indication of the organisation's move towards stage 4 organisational integration "business process redesign". business process integration • bar coding and product numbering initiatives16 product numbering is an internationally compatible system for identifying products and services by allocating a unique number to each item or service which may be traded worldwide. by identifying products unambiguously, these numbers provide a means for manufacturers, exporters, importers, wholesalers and retailers to communicate regarding the goods they trade. product numbers are represented by barcodes which can be scanned and thus entered directly into a computer. article numbering associations exist in almost all countries in australia, for example, the relevant organisation is the australian product numbering association (apna), in north america the uniform code council (ucc) provides services to both the u.s. and canada, while the british equivalent is the article numbering association (ana). these organisations exist to allocate codes based upon consistent international article numbering standards, provide technical assistance and education to companies, support retailers in introducing scanning technology and to offer information and research services (palazzolo, 1992a). de vries (1992) discusses the early use of product numbering, which began in the 1970's, when barcodes were placed on individual consumer articles to enable detailed recording of the flow of goods leaving the shop via the cash register. these original initiatives became considerably more sophisticated with the advent of edi and suppliers/customers now optimise communications which relate to business transactions by linking their automated barcoding systems. these more advanced systems were made possible by a significant extension to the international product numbering system die ucc/ean application identifier (ai) standard. palazzolo (1992b) explains mat the basic tools of the existing system (the ean/apn 13 or 8 digit consumer unit number and the itf 14 digit trade unit number) merely serve to identify products. the ai standard enables companies to add additional variable information to individual products or packs at the time of production or shipping ranging from customer purchase order numbers to serial shipping container 1 the following extremely brief summary of product numbering and bar coding is drawn from palazzolo (1992% 19925) and brown (1992), while the information cooceming bhp steel's product numbering and barcoding initiadves is based upon meakins (1992) and internal bhp steel memoranda and documents and upon discussions with bhp steel personnel involved in these activities. 65 ajis codes, batch numbers, use-by dates, or product weight/length information. email's service variant numbers differentiate products which are identical as sales items (and therefore have common product identifiers), but which use different and non-interchangeable spare parts. the service variant number tells the service technician which non-interchangeable spare parts must be used for that particular appliance (brown, 1992). meakins (1992) suggests that in an edi environment, product codes are adopted primarily as a means of communication across disparate application systems, each of which may identify products differently. steel manufacturers and distributors presently use non-standard, in-house product codes often structured and worded in such a manner as to indicate the product itself. although such codes may be exchanged directly by using translation tables, the increase in edi traffic will increasingly favour conversion to codes based upon international standards. four of bhp steel's major divisions have adopted the apna 13-digit consumer unit product numbers and there are currently more than 800 numbers in use for edi trading. products are being allocated numbers continuously and the list is expected to grow into the 10's of thousands. product numbers are incorporated into edi messages as a pair of data elements which comprise both a qualifier (such as buyer's or vendor's part number) and the product code itself. bhp steel's involvement with the edica-based steel industry working party is crucial to the acceptance of international product number standards. the siwp advocates the use of apna standards, although it has not yet agreed upon a set of steel industry bar code data elements for use within the apna's ai standard for supplementary information. the siwp and the apna are currently evaluating a range of options to match steel industry data elements to apna numbering systems and symbologies. the integration of bar coding with edi handling is complex, since document data must also either be transferred and mapped to produce bar code labels at the time of despatch, or be read and mapped from bar code labels at the time of receipt or inventory. agreement on the designation of data elements and their position within bar code fields in the edi environment is now underway within the siwp. more advanced and integrated use of bar codes will come from the development of evaluated receipt settlement to automate the link between product receipt and payment for goods. this requires the extraction of data elements from individual despatch advice bar codes and the reconciliation of product and quantity with either purchase orders or period contract information, followed by the automatic generation of either a payment order and remittance advice or, at the very least, a prepayment advice. such systems will enable the elimination of invoices and back-end document matching, lead to improved cash-flow forecasting and enable an interactive approach to contracts and sales/price catalogues. the initial bhp steel initiative involved an edi-eft pilot with major suppliers in early 1993 and eft pilots with customers are continuing. the entire project will take a considerable amount of time bhp steel anticipates that the flow-on of eft to customers beyond the most immediate group could take as long as 7-8 years. meakins (1992) points out that edi, barcoding and eft schemes compete for the same capital pool as general computing facilities and may take many years to be accepted by smaller organisations who see fewer benefits in such integration. it is clear, however, that bhp steel's longer-term ambitions will add another level of business process redesign to present system innovations the use of barcoding and eft will not only eliminate a number of existing documents and improve efficiency, but will also increase the organisation's move towards information-based, rather than document-based operations. multi-lateral edi integration the apec project17 the asia pacific economic cooperation (apec) work programme supports ten separate work projects, of which the telecommunications working group (two) is one. the two held its first meeting in singapore in july 1990 and agreed to undertake a short-term project to compile telecommunications profiles for apec members, together with three longer-term projects: edi, teleports and human resource development. the mission for the edi project was defined in the following terms: 17 the brief summary of the apec edi project which follows is derived from two internal edica documents and supported by interviews with bhp it and edica representatives of the edica telecommunications working group involved in the project may 1994 66 to implement a live, edi-based, exchange of trade data which facilitates the movement of goods purchased by a new zealand customer from an australian manufacturer, meeting regulatory obligations required to support international trading relationships (edica, 1992c:25). the edi council of australia proposed a multi-lateral, industry-based pilot edi project with two major components the commercial and the regulatory transactions involved in international trade. edica proposed b hp as the initiating company, due to the company's: • recognised commitment to edi • demonstrated in-house expertise • substantially wider experience in edi than any other australian organisation • prime trading position in the apec trading region • established business relationships with many apec countries. 1. in terms of the commercial relationship, the importer and exporter were to exchange the following documents: • purchase order (initiating the transaction) • purchase order acknowledgement • invoice • quality report (test certificate) • despatch advice (shipping note) 2. the relevant regulatory requirements are that: • bhp steel (australia) will send commercial information regarding the goods via the exit i system to the australian customs service (acs) in order to obtain a customs clearance number; • seatrans/sydney (the shipping company) will lodge the manifest (details of the movement of the steel products) with the acs via the exit ii system. this electronic manifest may go directly to new zealand customs; • to meet mandatory legislative requirements, new zealand customs requires documentation (a copy of the exit document and/or an invoice) directly from the importer (or from a broker commissioned by the importer), prior to clearing the goods for entry. inter-national trade is an inherently complex process and, even in the form of a pilot project which has been deliberately restricted in scope, the exchange of documents required is rather involved. this project is still in its early stages and, due to both the technical and diplomatic complexity of the relationships involved development is proving a little slower than the original projections. the edi project is largely intended to demonstrate to other apec member nations the effectiveness of edi as a means of expediting international trade flows. the apec project is also an ideal illustration of integrated edi, demonstrating: • the basic, physical benefits of edi as replacement for paper-based document flows. international trade is particularly dependent on the movement of huge quantities of paper which frequently arrive after the goods themselves have reached their destination. (bhp steel have actually noted occasions when it was cheaper to fly one of their own staff members over to new zealand with the paper-work relating to a particular shipment than to pay the fees which would result from holding goods in storage until the documentation had arrived.) although the restriction appears to be more crucial to exporters making substantial use of air freight, this case study shows that even companies solely reliant on sea freight (it is not cost-effective to air-freight steel products) can gain substantial advantages from the electronic replacement of paper document flows • the still more substantial benefits possible from integrating these basic paper trail replacements into the application systems of the organisations involved in international trade. almost every organisation taking part in this pilot scheme (exporter, shipper, importer, customs and forwarding agents, and regulatory bodies such as port authorities and customs) has its own internal application systems dedicated to a particular part of the trade cycle. linking these systems together adds a further dimension of effectiveness/efficiency to the trading process • the longer-term strategic benefits of thinking in terms of information flow, rather than document flow. each of the organisations taking part in the pilot scheme has 67 ajis begun to rethink and redesign the processes which form their own contribution to the trade cycle (for example, re-read the descriptions of the manifest information flows above). this project provides an illustration of large-scale business process redesign across organisational boundaries. the movement of international trade goods is an archetypal example of this problem many autonomous processes, which were created in the days before computerisation, being improved and redesigned by individual organisations with little, if any, improvement in overall efficiency and timeliness. the apec project has tackled this group (or at least a sub-set of this group) of autonomous processes across a wide variety of organisation types (both public and private sector) in an attempt to come to terms with the "second-order" or meta-processes which are needed by the international trading operation. there is, as yet, no term to describe the application of business process redesign across inter-organisational boundaries but it is clear that the need for such an approach will continue to grow. the apec project, like other such visionary projects (the european odette or cost 320 projects, for example see van maaren, 1993) is endeavouring to develop tools, technologies and organisational structures which will enable organisations to work together for greater overall effectiveness while retaining their individuality and competitive ability. implications for bmp internal processes and structure the implications of integrated edi for re-engineered business processes are critical to the long-term use and effectiveness of this technology. unfortunately, however, changes to organisational practices and structures tend to occur far more slowly than do computer-based applications and the organisational impacts may not emerge until quite some time after the technical changes have been put into place. in the case of b hp steel, for example, the electronic trading project has had a significant effect on both internal and inter-organisational information systems over the past eighteen months, but changes to organisational practice are only just becoming apparent. this section of the paper describes these early procedural changes, as they affect two separate order management operations: • customer orders, handled by the order entry sub-system of the sales administration and order management system (sams) • supplier orders and consequent inventory management approaches. the sams order entry sub-system was the first internal application system to be targetted for integration with the electronic trading project. documents involved in this process are of two types: • incoming purchase orders (po) and purchase order changes (poc); • outgoing purchase order acknowledgements (in practice purchase order change acknowledgements (poca), since a single document type is used for both purposes). 1. incoming documents. prior to the implementation of the electronic trading project, the inside sales representative (isr) would review each purchase order with the customer prior to processing although there was really no need for such review, save in the case of one-off production (which is a rare occurrence) or where there had to be changes to production or requirements. the first stage of integrating in-coming purchase orders with internal systems and processes still requires the active participation of the inside sales representative, but has limited the time involved. purchase orders are now received in the form of edi documents through the electronic trading gateway and appear on the inside sales rep's terminal for review with the customer where necessary. once past this stage, however, po's released by the isr are sent for automatic review within the order entry system. the next stage of integration involves the separation of purchase orders into: • automatic orders, where the po is passed directly into the order entry system's automatic review process which merely identifies syntactic errors; and • manual orders, where the inside sales rep. must review the po with the customer prior to order entry processing. may 1994 68 at the completion of this stage the inside sales rep, freed from what is presently a very timeconsuming and largely unnecessary task, is able to devote considerably more time to what is the true purpose of the position sales of bhp steel products to customers. 2. outgoing documents. out-going purchase order change acknowledgements, although more substantial than a purely functional acknowledgement, contain only minimal information (such as confirmed delivery date and price) except in exceptional cases where more detail is needed. the first stage of automating outgoing po change acknowledgements is to replace the current printed acknowlegement document which is mailed to customers with an edi message (the ansi x12 865 message). this stage is already running "live" with several trading partners and is either in test or in the final planning stages across all bhp steel divisions and almost all trading partners taking part in the pilot etp. the second stage of automating the outgoing document process will involve modification of the order entry system itself to enable integration of the edi message at all stages of the outward process. one step has already been taken towards streamlining the process, however, by eliminating the manual reconciliation of purchase orders and po change acknowledgements resulting in significant savings. it is apparent that changes to the sams order entry sub-system processes are in the early stages of redesign although the cross-sectional coordination between the various departments affected by the receipt of purchase order modifications suggests that an organisational view of the problem is being taken. supplier order processes have undergone significant changes since the introduction of the electronic trading gateway. the structural effect of system-based changes in the order management process has been to change the overall organisational approach from what might be termed a "central store" to a just-in-time approach (jit is a philosophy based on a number of other technologies edi, material requirements planning and distribution requirements planning which combines distribution with manufacturing to get the correct products to the correct people at the correct time): • the slab and plate products division believes it has saved one week's worth of inventory as a result of stage 1 order entry automation; • the division is now running the plant with 25%-30% of the previously-held inventory of spares; • divisional departments are now ordering directly from trading partners, rather than through bhp steel's own central stores; • the central stores themselves have now been able to reduce thenholdings to requirements for 24-48 hours:where electronic orders are placed before 11.00 a.m_, deliveries are guaranteed before 5.00 p.m. where orders are placed between 11.00 a.m. and 3.00 p.m., deliveries are guaranteed before 11.00 a.m. the next day; • larger trading partners are now pushing this just-in-time philosophy out to their own suppliers. a case in point is bisalloy, which, while using exclusively pc-based systems, has decreased data handling errors, improved timeliness of inventory information (reducing inward goods delays from 36 to 12 hours) and substantially improved the productivity of both supplier and customer order processing by integrating edi messages with internal system information (croll, 1992) suggesting that even comparatively small companies can take advantage of the benefits available from edi-based business process redesign. these changes in philosophy and organisational practice are, in turn, generating further and possibly more far-reaching changes. personnel involved in the supply process are now looking at the potential which electronic trading holds for the despatch of goods. despatch information passed through the electronic trading gateway offers the possibility of "evaluated receipts settlement", the replacement of invoices by regular payment upon receipt. this involves a purchaser issuing an electronic order and a supplier responding by shipping the goods and sending an electronic ship notice. payment is triggered either by electronically matching these two documents, or by matching them automatically to a third document a receipt from the receiving point 69 ajis the process is made possible by the fact that supply is on the basis of period contracts, so that amounts and prices are agreed in advance. electronic unking of payments with despatch information by means of bar-coding and product numbering would significantly accelerate the payment process and could offer further savings in terms of both electronic funds transfer efficiences and the greater total buying power generated by the elimination of data rekeying. conclusions this paper has summarised the activities of bhp steel's electronic trading initiative, with the aim of investigating the successful integration of edi into internal application systems and organisational structure within an australian organisation. the length of the case study is due to the complexity of the subject bhp steel's strategic approach to the use of electronic trading (both internally and externally) has required the rethinking of many of the organisation's existing methods of doing business and has led to the creation and development of a number of entirely new internal and external trading initiatives. the analysis of bhp steel's electronic trading project provides a number of conclusions: • that there truly are benefits to be obtained from an inter-organisational electronic trading strategy founded on edi as the enabling mechanism benefits which are considerably more extensive than edi's comparatively simple technical nature would appear to offer; • that organisational gateways utilising an application generic approach to system integration and isolating the communications issue by insisting on the use of international data communications standards do offer a realistic and successful solution to the problem of internal and external trade for large and sophisticated organisations; • that a top-down, organisational and strategic perception of edi has led to a fast, effective take-up of edi at a number of levels:the development of the electronic trading gateway as an application generic facilitator of internal and external sources of information, with a design which encourages continuing modification and enhancement the active involvement of all sections of the bhp steel organisation in planning and implementation including production, trading and support groups the successful involvement of trading partners and the spread of electronic trading to what svinicki (1988) described as the "second supplier tier' those suppliers who themselves must obtain production materials from other suppliers and who can therefore link another level of organisations into the electronic trading hierarchy and the involvement of related industry sector representatives into edica working parties to develop accepted document sub-sets in terms of edica implementation guidelines and the subsequent amalgamation of the siwp and hemmp working parties which ensures greater currency for the guidelines developed and, hence, greater inter-sectoral trading opportunities; • that many of these achievements would have been either unlikely to succeed or totally impossible without bhp steel's commitment to organisation-wide business process redesign, which has led to:the decision to use an overall project manager and controller for the project to ensure consistency at all stages of development the cross-organisation steering committee which sets edi policy and directs the project team through the project manager the cross-organisation functional teams which handle application integration, communication, total quality management, mapping of document standards and system design. although this paper has concentrated on a single exemplar case study, the highly successful of outcome of bhp steel's edi initiatives suggests that further case research, using a wider group of examples, might well offer conclusive evidence of edi's ability to provide a suitable infrastructure for business process redesign. may 1994 70 references baker r.h. (1991) edi: what managers need to know about the revolution in business communications, mcgraw hill, blue ridge summit, pennsylvania. benjamin r.i., de long d.w. and scott morton m.c. (1990) electronic data interchange: how much competitive advantage?, long range planning (uk), vol. 23, no. 1, february, 29-40. bmp steel (1989) edi study tour united states of america, b hp steel coated products division publication, wollongong, may. b hp steel (1990a) electronic trading gateway proposal, bhp steel coated products division publication, wollongong, march. bhp steel (1990b) electronic trading project phase 1: "study and evaluation" report, bhp information technology publication, wollongong, september. bhp steel (1991a) electronic trading project phase 2: "system design" report, bhp information technology publication, wollongong, march. bhp steel (1991b) customer trading management systems stage 1: system design document, bhp information technology publication, melbourne, may. bhp steel (1992a) bhp test certificates via edi, presentation to the edi council of australia's awareness meeting, sydney, 7 may. bhp steel (1992b) edi test certificates: bhp steel information kit, bhp steel publication, wollongong, may. brown r. (1992) a case study of the use of the ean/ucc application identifier standard, proc conf 1992 scanning technology conference, melbourne, australia, october 20-21. cash j.i. and konsynski b.r. (1985) is redraws competitive boundaries, harvard business review, 134-142, mar-apr. croll t. (1992) electronic information exchange: a business proposition, bisalloy steels pry. ltd. corporate presentation, wollongong, nsw, 29 september. de vries h.g. (1992) "the use of edi for trade transactions and the role of the ean coding system: experiences with uac-transcom", in j. gri ar (ed) edi: inter-organizational systems in the global environment proceedings of the 5th international electronic data interchange conference, bled, slovenia, september, 117-131. dubois g. (1990) how to improve the bottom line and reduce the cost of doing business -where does edi fit in? proc conf edica '90 "paperless trading strategies for the 1990's", sydney, march 29/30. davenport t.e. and short je. (1990) the new industrial engineering: information technology and business process redesign, sloan management review, summer 1990. emmelhainz m. a. (1987) the impact of electronic data interchange on the purchasing process, doctoral thesis, school of business administration, the university of dayton, dayton ohio, february 1987. emmelhainz m.a. (1990) electronic data interchange: a total management guide, van nostrand reinhold, new york. fuller m (1990) establishing the most cost-effective electronic links with trading partners and other organisations, proc conf "edica'91" 3rd national edi users' conference, sydney, march. fuller m. (1992) edi be prepared for change, proc conf "edica'92" 4th national edi users' conference, melbourne, 8-10 april. galliers r.d. (1991) strategic information systems planning: myths, reality and guidelines for successful implementation, furpean journal of information systems, vol. 1,55-64. galliers r.d. and sutherland a.r. (1991a) organizational learning and it: steps towards managing and planning strategic information systems, warwick business school working paper, university of warwick, january. galliers r.d. and sutherland a.r. (1991b) information systems management and strategy formulation: the "stages of growth" model revisited, journal of information systems, vol. l.no. 2, 89-114. hammer m. (1990) reengineering work: don't automate, obliterate harvard busines review, july/august, 104-112. harris b, parfett m. and sarson r. (eds.) (1992) electronic trader the edi yearbook 1993, ncc blackwell limited, oxford, england. 71 ajis harvey d. (1992) a discussion of the organisational impacts of edi, proc conf "isop'92" 3rd australian conference on information systems, wollongong, 5-8 october. hunt t.j. (1992) establishing a corporate electronic trading gateway using international standards for communications, proc conf "new horizons open systems conference", sydney, 1113 february. ins (1991) tesco breaking down the barriers of trade, international network services limited publication, sunbury-on-thames. lyttle r.m. (1988) will you be proactive or reactive on edi?, systems/3x world, november, 6874. meakins j. (1992) bar coding in the australian steel industry, proc conf 1992 scanning technology conference, melbourne, australia, october 20-21. morrow m. and hazell m. (1992) activity mapping for business process redesign management accounting, february, 36-38. palazzolo m. (1992a) australian product numbering, presentation made to the edica steel industry working party, metals distribution sub-group, barcoding action group meeting, september. palazzolo m. (1992b) what is ai standard and ban 128?, proc conf 1992 scanning technology conference, melbourne, australia, october 20-21. patrick g. (1988) "the challenges of edi decision making", in m. gifkins and d. hitchcock (eds) the edi handbook, blenheim online publications, london. robinson d.g. and stanton s.a. (1987) exploit edi before edi exploits you, information strategy, the executive's journal, spring, 32-35. rochester, j.b. (1989) the strategic value of edi, i/s analyzer, vol. 27, no. 8. sadhwani a.t. and sarhan m.h. (1987) electronic systems enhance jit operations, management accounting, december, 25-30. scott morton m.s. (1991) the corporation of the 1990's: information technology and organizational transformation, oxford university press, oxford. skagen a.e. (1989) nurturing relationships, enhancing quality with electronic data interchange, management review, february, 28-32. swatman p.m.c. (1993) integrating electronic data interchange into existing organisational structure and internal application systems: the australian experience, curtin university phd thesis, published as curtin university department of computer science technical report 19 -1993, june. swatman p.m.c. and clarke r.a. (1991) "the organisational, sectoral and international implications of electronic data interchange", in j. berleur and j. drumm (eds), information technology assessment, norm-holland, amsterdam, 117-130. swatman p.m.c. and swatman p.a. (1991a) electronic data interchange: organisational opportunity, not technical problem proc conf "dbis '91" 2nd australian conference on information systems and database, university of new south wales, sydney, february, 290-307 swatman p.m.c. and swatman p.a. (1991b) 'trading blocs and electronic data interchange: the implications of europe's single unified market", in j. gri ar (ed) edi: business strategy for the '90s proceedings of the 4th electronic data interchange conference, bled, yugoslavia, june, 182-196. swatman p.m.c. and swatman p.a. (1991c) integrating edi into the organisation's systems: a model of the stages of integration, proc conf 12th international conference on information systems (icis'91), new york, december. swatman p.m.c. and swatman p.a. (1992) edi system integration: a definition and literature survey, the information society, vol. 8, no. 3, summer, 169-205. swatman p.m.c., swatman p.a. and fowler d.c. (1993) "a model of edi integration and strategic business reengineering", the journal of strategic information systems, vol. 2, no. 4, december. van maaren h.c. (1992) "the actual and potential impact of edi on the transport sector and its actors cost320, a coordinated international survey in europe", in j. gri ar (ed) edi: inter-organizational systems in the global environment proceedings of the 5th international electronic data interchange conference, bled, slovenia, september, 132140. may 1994 72 vogel d.r. (1991) "edi enterprise analysis", in j. gri ar (ed) edi: business strategy for the '90s proceedings of the 4th electronic data interchange conference, bled, yugoslavia, june, 196-207. wilmot r.w. (1988) international trends and developments in edi, proc conf edi '88, london, november, 15-21. 73 ajis view publication statsview publication stats https://www.researchgate.net/publication/30062925 paper title (paper title style) australasian journal of information systems petersen, poon j, poon s & loy 2017, vol 21, selected papers from ausdm better rulesets by removing redundancy 1 better rulesets by removing redundant specialisations and generalisations in association rule mining henry petersen university of sydney hpetersen@sydney.edu.au josiah poon university of sydney simon poon university of sydney clement loy university of sydney abstract association rule mining is a fundamental task in many data mining and analysis applications, both for knowledge extraction and as part of other processes (for example, building associative classifiers). it is well known that the number of associations identified by many association rule mining algorithms can be so large as to present a barrier to their interpretability and practical use. a typical solution to this problem involves removing redundant rules. this paper proposes a novel definition of redundancy, which is used to identify only the most interesting associations. compared to existing redundancy based approaches, our method is both more robust to noise, and produces fewer overall rules for a given data (improving clarity). a rule can be considered redundant if the knowledge it describes is already contained in other rules. given an association rule, most existing approaches consider rules to be redundant if they add additional variables without increasing quality according to some measure of interestingness. we claim that complex interactions between variables can confound many interestingness measures. this can lead to existing approaches being overly aggressive in removing redundant associations. most existing approaches also fail to take into account situations where more general rules (those with fewer attributes) can be considered redundant with respect to their specialisations. we examine this problem and provide concrete examples of such errors using artificial data. an alternate definition of redundancy that addresses these issues is proposed. our approach is shown to identify interesting associations missed by comparable methods on multiple real and synthetic data. when combined with the removal of redundant generalisations, our approach is often able to generate smaller overall rule sets, while leaving average rule quality unaffected or slightly improved. keywords: association rule mining;redundancy 1 introduction association rule mining is an important task in knowledge extraction and data analysis. formally, let a = {𝑎1, 𝑎2, … , 𝑎𝑀}be a set of m attributes. we then define n data d = {𝑑1, 𝑑2, … , 𝑑𝑁} where each data di contains a subset of the attributes in a (i.e. di ⊆ a, ∀ di ∈ d). the association rule mining task seeks to find all interesting rules of the form x ⇒ y, where x and y are disjoint subsets of a. it is well known that the number of rules generated can be so large as to obscure their interpretation, presenting a barrier to practical use (zaki 2000). ideally, we wish to find only the most interesting rules. this can be broken into two tasks; identification of truly interesting rules, and the removal of those which are simply redundant artefacts of other rules. a rule is considered interesting if it produces a value superior to some predetermined threshold according to a chosen function of interestingness m(·) (e.g. support, confidence, fishers p). standard thresholds exist for some measures of interestingness (e.g. p < 0.05 for measures of australasian journal of information systems petersen, poon j, poon s & loy 2017, vol 21, selected papers from ausdm better rulesets by removing redundancy 2 statistical significance), while other for other measures the threshold is determined in a more ad hoc manor. our work focuses primarily on statistical measures of interestingness. when comparing two rules x ⇒ z and xq ⇒ z to evaluate redundancy, existing approaches ignore data containing only part of the antecedent. as a result, non-interesting rules can be retained, and potentially interesting rules discarded. this paper proposes an alternate definition of redundancy (which we call robust redundancy) that utilises such information to improve the quality of the discovered rules. examples are given demonstrating the ability of robust redundancy to correctly identify interesting rules which would otherwise have been missed by more classical approaches. an algorithm for generating rules with robust redundancy is also proposed and evaluated on both real and artificial data. within the literature, there are several sub-problems that have been studied. examples include rules with fixed (verhein and chawla 2007), or single attribute (hämäläinen 2012) consequents, numerical data (song and ge 2013), or negative associations (hämäläinen 2012, li and zaiane 2015) (e.g. rules of the form x ⇒ ⌐z). in this paper we focus on the problem of positive rules from binary data with single attribute consequents. in this paper we refer to the concepts of rule generalisations and specialisations. given two rules x ⇒ z and y ⇒ z (where z is a single attribute and x and y are sets of attributes), the rule y ⇒ z is considered to be a generalisation of x ⇒ z if y is a proper subset of x. similarly, rule y ⇒ z is a specialisation of x ⇒ z if y is a proper superset of x. 2 background a key problem for association rule mining is measuring how interesting a rule is. traditionally this is done using support and confidence (analogous to the sample probability of a rule and the conditional probability of y given x respectively). many alternate approaches for measuring interestingness have been proposed (piatetsky-shapiro 1991, brin, motwani et al. 1997), several of which are presented in table 1. the interested reader is directed to the literature for further information (tan, kumar et al. 2004). the experimental work in this paper focuses on statistical measures interestingness, where rule interestingness is analogous to statistical significance. the size of the search space is a major concern when mining association rules. prior work often employs heuristics such as maximum rule lengths, fixed consequents, or frequency thresholds in order to control this (hämäläinen 2010). to our knowledge, only hämäläinen's kingfisher algorithm is able to identify all significant rules using the current definition of nonredundancy. we extend the kingfisher approach for rule generation in section 4. it is well established that the number of rules identified can often be so large as to hamper their interpretation (aggarwal and yu 2001). the concept of redundancy can be used to control the number of rules. consider a hypothetical study of supermarket transactions which identifies that people who buy a soft drink will also buy chips. further analysis may also identify that people who buy soft drink on tuesday will buy chips. however the condition that it be tuesday does not improve the quality of the association. the rule likely exists because the association between people buying soft drink and chips holds regardless of whether it is tuesday or not. the association between people buying soft drink on tuesday and buying chips is redundant. australasian journal of information systems petersen, poon j, poon s & loy 2017, vol 21, selected papers from ausdm better rulesets by removing redundancy 3 measure formulae support (agrawal, imieli et al. 1993) 𝑚(𝑋) 𝑁 confidence (agrawal, imieli et al. 1993) 𝑚(𝑋𝑌) 𝑁 interest (brin, motwani et al. 1997) 𝑁 × 𝑚(𝑋𝑌) 𝑚(𝑋) × 𝑚(𝑌) leverage (piatetsky-shapiro 1991) 𝑚(𝑋𝑌) 𝑁 − 𝑚(𝑋)𝑚(𝑌) 𝑁2 χ2 𝑁5 − 𝐿𝑒𝑣𝑒𝑟𝑎𝑔𝑒(𝑋 ⇒ 𝑌)2 𝑚(𝑋)𝑚(¬𝑋)𝑚(𝑌)𝑚(¬𝑌) fisher’s p ∑ ( 𝑚(𝑋) 𝑚(𝑋𝑌)+𝑖 ) ( 𝑚(¬𝑋) 𝑚(¬𝑋¬𝑌)+𝑖 ) ( 𝑁 𝑚(𝑌) ) min (𝑚(𝑋¬𝑌),𝑚(𝑌¬𝑋)) 𝑖=0 table 1: several common interestingness measures for a rule x ⇒ y expressed in terms of partial frequency counts. n is the size of the data, and m(·) is the frequency function. several authors have proposed formal means for defining redundant rules (zaki 2000, aggarwal and yu 2001, ashrafi, taniar et al. 2004, webb 2006, webb 2007). a definition of redundancy suitable for use with a general goodness measure (assuming single attribute consequents) was first proposed in 2010 by hämäläinen (hämäläinen 2010). hämäläinen defines a rule r to be redundant if some more general rule (i.e. a rule whose antecedent is a subset of the antecedent of r) has equal or better utility with respect to some goodness measure. we repeat this more formally in definition 1. definition 1: classical redundancy consider two rules x ⇒ z and xq ⇒ z where x and q are disjoint sets of items, and z is a single item of value α. let m(·) be an increasing measure of rule interestingness. rule xq ⇒ z is redundant with respect to rule x ⇒ z if m(xq ⇒ z) ≤ m(x ⇒ z). we note that when comparing rules based on some arbitrary goodness measure, complications can arise due to complex interactions between constituent attributes. that such interactions can give rise to spurious associations has been studied (mcgrane and poon 2010, webb 2010), however less attention has been paid to how it might obscure useful relationships. models of redundancy that remove spurious generalisations is a problem that has seen limited interest within the community. removing spurious generalisations was first looked at in 2001 by liu et al. in their work on non-actionable rules (liu et al. 2001). for a rule r0 and the set of its decedents r = {𝑟1, 𝑟2, … , 𝑟𝑁}, they define a rule r0 to be non-actionable if it is not interesting over the domain where instances matching at least one antecedent in r are removed. essentially, they claim a rule must cover some unique set of instances (with respect to the set of its specialisations) in which the relationship described still holds. a rule has no utility with respect to the set of its specialisations if it does not cover such a set of instances. australasian journal of information systems petersen, poon j, poon s & loy 2017, vol 21, selected papers from ausdm better rulesets by removing redundancy 4 a similar concept to non-actionable rules was proposed by webb in his work on self-sufficient itemsets (webb 2010). this work builds upon the concept of an exclusive domain for a given itemset. formally, given an itemset s and its specialisations s, the exclusive domain of s is defined to the domain of s minus the union of the domains of all itemsets in s. after generalising the concepts of redundancy and productivity (webb 2006, webb 2007) for use with itemsets, an itemset is defined to be self-sufficient if it is productive and non-redundant both with respect to the entire data and its exclusive domain. in many respects, self-sufficient itemsets can be considered an extension of non-actionable rules for use in an itemset context. however, it is noteworthy that itemsets must also be productive and non-redundant. this pruning of both specialised and general itemsets is similar to our work with robust redundancy described in this paper, although it is performed in an itemset context. we also examine how to avoid pruning specialisations where redundancy is likely to be an artefact of interactions between constituent attributes. 3 robust redundancy a rule is considered redundant when it adds no information over another rule. we claim that classical redundancy makes such a comparison using incomplete information. depending on the interestingness measure m(·) in use, m(x ⇒ z) is computed using the frequencies xz, ⌐xz, x⌐z, and ⌐x⌐z. note that directly comparing rules m(x ⇒ z) and m(xq ⇒ z) does not consider transactions including only part of the rule antecedent (i.e. frequencies of x⌐qz, ⌐xqz, x⌐q⌐z, and ⌐xq⌐z). association rule mining can be confounded by noise and complex relationships between variables. potential lack of control over the data collection process can further complicate matters. such noise could artificially raise or lower the measured interestingness value of a rule, which could lead to interesting rules being incorrectly excluded. we propose using additional information in an attempt to avoid excluding interesting rules. we also seek to identify seemingly interesting rules that are simply artefacts of groups of their specialisations. we refer to these approaches as specialisation and generalisation redundancy respectively. 3.1 specialisation redundancy we propose an alternate approach to redundancy in definition 2. we augment the classical approach given in definition 1 by not eliminating a rule xq ⇒ z if the partial frequencies can be used to demonstrate the attributes in q add value. this is accomplished by computing the strength of the association between x and z conditioned on q, and comparing it against the strength of the marginal association. if the conditional association between x and z improves over the strength of the previous association, we obtain evidence that the addition of q adds value to the existing rule. definition 2: specialisation redundancy consider two rules x ⇒ z and xq ⇒ z where x and q are disjoint sets of attributes, and z is a single attribute. let m(·) be an increasing measure of rule interestingness. rule xq ⇒ z is specialisation redundant with respect to rule x ⇒ z if m(xq ⇒ z) ≤ m(x ⇒ z), and m(x ⇒ z | q) ≤ m(x ⇒ z). computing the conditional association requires frequencies for ⌐xqz and ⌐xq⌐z (in addition to the frequencies xqz and xq⌐z). we do not consider the association between x and z conditioned on ⌐q, as the rule we are seeking to obtain evidence for is xq ⇒ z, which contains q. 3.1.1 example consider hypothetical data sampling 3 binary variables x, y, and z. assume 1000 data points with the probabilities expressed in table 2a. from these probabilities, observe that a strong australasian journal of information systems petersen, poon j, poon s & loy 2017, vol 21, selected papers from ausdm better rulesets by removing redundancy 5 dependency exists between z and the itemset xy. we now examine the rules x ⇒ z, y ⇒ z, and xy ⇒ z as we vary the joint probability of variables x and y. figure 1: ln(p-values) for rules x ⇒ z, y ⇒ z, and xy ⇒ z versus conditional probability of x and y. figure 1 plots rule quality against the conditional probability of x given y. for larger conditional probabilities we can observe that the quality of the rule xy ⇒ z is superior to that of rule x ⇒ z. as the overlap between data containing x and data containing y decreases, the quality of the more general rule x ⇒ z surpasses its specialisation. consequently the rule xy ⇒ z is removed as redundant, obscuring the true underlying structure of the data. (a) (b) p(x) 0.3 (x) 0.5 p(y) 0.3 p(y) 0.5 p(z|x,y) 0.8 p(z|x,y) 0.5 p(z|x,⌐y) 0.4 p(z|x,⌐y) 0.1 p(z|⌐x,y) 0.4 p(z|⌐x,y) 0.1 p(z|⌐x,⌐y) 0.6 p(z|⌐x,⌐y) 0.1 table 2: marginal and conditional probabilities for several combinations of variables used in the motivating examples for robust redundancy. holding the marginal probabilities constant, the frequencies of both x and y decrease along with the conditional probability of x given y. in order to support the rule xy ⇒ z, we require data containing both (or neither) xy and z. hence, as the number of data with xy and z decreases, so too does the evidence available to evaluate it. that the general rule x ⇒ z surpasses the true rule xy ⇒ z in quality as the conditional probability decreases is a reflection of this. by comparing rules x ⇒ z and xy ⇒ z using robust redundancy (i.e. including the conditional dependencies) we make more effective use of available data to evaluate the rules. in the example given in figure 1, rule xy ⇒ z is retained as non-redundant for conditional probabilities greater than ~0.045. this is in contrast to classical redundancy, where the threshold for retaining xy ⇒ z is ~0.062. although in both cases the conditional probability of x given y eventually reaches a point where insufficient evidence for the specialised rule exists, the range of values for which robust redundancy can still retain xy ⇒ z is increased. australasian journal of information systems petersen, poon j, poon s & loy 2017, vol 21, selected papers from ausdm better rulesets by removing redundancy 6 3.2 generalisation redundancy it is possible for general rules to exist that only appear interesting due to the presence of one or more interesting specialisations. definition 3 outlines a concept we call robust generalisation redundancy. a rule x ⇒ z is generalisation redundant if for all non-redundant specialisations xq ⇒ z, the rule x ⇒ z | ⌐q is uninteresting (its goodness is less than the required threshold). if a rule x⌐q ⇒ z is interesting, we obtain evidence that the generalised rule is interesting even in the absence of the terms in q. if after identifying all other interesting rules we cannot find evidence that x ⇒ z is interesting in the absence of the additional terms in its specialisations, we consider it redundant. computing the conditional association on ⌐q uses the frequencies x⌐qz and x⌐q⌐z. hence, by applying both specialisation and generalisation redundancy we consider all frequencies in the sample data. definition 3: generalisation redundancy consider a rule x ⇒ z and the complete set of its non-redundant specialisations r. let m(·) be an increasing measure of rule interestingness, and α be the corresponding goodness threshold. rule x ⇒ z is generalisation redundant with respect to r if m(x ⇒ z | ⌐q) ≤ α for all rules xq ⇒ z in r. 3.2.1 example consider hypothetical data containing drug prescriptions and a corresponding binary patient outcome. assume there are two drugs (x and y) which work in combination to produce a positive outcome. neither drug will produce a positive outcome on its own (a baseline probability for positive outcome of 0.1 is used). the exact probabilities used can be found in table 2b. when the conditional probability of x given y is 1, the measured quality of the rules x ⇒ z, y ⇒ z, and xy ⇒ z will be identical and maximal. the quality of these rules decreases with this conditional probability, with the quality of the general rules decreasing at the greatest rate. however despite the underlying structure of the data indicating that neither x or y alone support a positive outcome, the strength of these associations will likely remain quite high. when comparing rules x ⇒ z and xy ⇒ z, examining the rule x ⇒ z | ⌐y (i.e. conditioned on the absence of the additional terms y) indicates that there is no evidence to support the rule x ⇒ z without also including the features y. as xy ⇒ z is the only identified specialisation of x ⇒ z, and we have no evidence to indicate x ⇒ z is valid without the additional features, we consider it redundant. finally, we acknowledge that such an approach could potentially over-fit and remove valid general rules. we address this concern in the following section on redundancy chaining. 3.3 redundancy chaining classical redundancy as defined in definition 1 is transitive. if a rule xqy ⇒ z is redundant with respect to a generalisation xq ⇒ z, and xq ⇒ z is redundant with respect to x ⇒ z, then xqy ⇒ z will be redundant with respect to x ⇒ z. this result is straightforward to prove. australasian journal of information systems petersen, poon j, poon s & loy 2017, vol 21, selected papers from ausdm better rulesets by removing redundancy 7 (a) (b) attr freq. rule ln(p) abcd 10 a ⇒ d -19.33 abd 10 ab ⇒ d -8.10 acd 10 abc ⇒ d -3.78 ad 10 a ⇒ d|b -18.75 bc 30 a ⇒ d|bc -20.56 bd 10 ab ⇒ d|c -7.83 cd 10 d 10 table 3: sample data and rules for lemma 1. unfortunately, the same relation does not hold for the proposed robust redundancy. a proof that specialisation redundancy is nontransitive is given in lemma 1. the fact that specialisation redundancy is non-transitive can lead to some interesting behaviour. assume a rule r exists that is specialisation redundant with respect to one or more generalisations 𝑟0, … , 𝑟𝑖. let 𝑟0, … , 𝑟𝑖 be redundant with respect to rules 𝑟𝑖+1, … , 𝑟𝑛. despite being a redundant specialisation of other rules, r is non-redundant with respect to all non-redundant generalisations. we take the view that in such a situation, the rule r should be considered non-redundant. lemma 2: using redundant rules when evaluating generalisation redundancy allows for additional rules to be included. consider three rules a ⇒ d, ab ⇒ d, and abc ⇒ d generated from the data in table 4 using the confidence measure with a threshold of 0.6. according to definition 3 and the confidence scores for the above rules, ab ⇒ d | ⌐c is uninteresting so ab ⇒ d is redundant w.r.t. abc ⇒ d. when evaluating generalisation redundancy without redundant rules, the uninteresting rule a ⇒ d | ⌐(bc) implied a ⇒ d is redundant. when evaluating generalisation redundancy with redundant rules, as a ⇒ d | ⌐b is interesting a ⇒ d is non-redundant. ■ we also prove that whether or not redundant attributes are counted effects generalisation redundancy in lemma 2. as above, we elect not to allow redundant rules to influence the redundancy of another rule. in contrast to specialisation redundancy, this will lead to the exclusion of additional rules (comparing against additional rules raises the chance of inclusion as generalisation redundancy requires a rule be uninteresting with respect to all its specialisations). lemma 1: specialisation redundancy is nontransitive. let a ⇒ d, ab ⇒ d, and abc ⇒ d be three rules generated from data in table 3 using the log of fishers p. as the interestingness of the rules ab ⇒ d and a ⇒ d | b is worse than that of the rule a ⇒ d, ab ⇒ d is redundant w.r.t. a ⇒ d. as the interestingness of the rules abc ⇒ d and ab ⇒ d | c is worse than that of the rule ab ⇒ d, abc ⇒ d is redundant w.r.t. ab ⇒ d. as the interestingness of the rule a ⇒ d | bc is better than that of the rule a ⇒ d, the rule abc ⇒ d is non-redundant w.r.t. a ⇒ d. ■ not using redundant rules to support retaining otherwise redundant generalisations can produce the following interesting situation. assume a rule r1:y ⇒ a where conf(y ⇒ a) = 1 and supp(y)= supp(a). then for all rules of the form r1 x ⇒ a where y=xq (i.e. generalisations of y ⇒ a), the frequency of x⌐qa will be 0, and the rule x⌐q ⇒ a will be uninteresting. by definition 3, r1 is the only possible non-redundant rule with consequent a. australasian journal of information systems petersen, poon j, poon s & loy 2017, vol 21, selected papers from ausdm better rulesets by removing redundancy 8 while it may in fact be desirable to keep such a rule, care must be taken to avoid confounding caused by the addition of independent attributes. we demonstrate how such confounding might occur by providing an extension of the above example. consider the rule yz ⇒ a for some variable z where supp(za) = 1. it is simple to see that conf(yz ⇒ a) = 1, supp(yz)=supp(a), and freq(y⌐za) = 0. the rule y ⇒ a will be considered redundant. (a) (b) attr freq. rule conf abcd 60 a ⇒ d 0.90 ab 20 ab ⇒ d 0.75 acd 10 abc ⇒ d 1.00 ad 10 a ⇒ d|⌐b 1.00 a ⇒ d|⌐b⌐c 0.50 ab ⇒ d|⌐c 0.00 table 4: sample data and rules for lemma 2. by occam's razor we prefer a more general rule over its specialisations unless evidence can be obtained to suggest otherwise. using only generalisation redundancy can violate this principle as no evidence is ever considered to support yz ⇒ a over y ⇒ a. in the worst case, for a given consequent only one highly specific rule will be selected with all others being made redundant. therefore specialisation (or classical) redundancy should usually be employed before generalisation redundancy. we note however that in some cases (such as those where we prefer to generate more specific rules), generalisation redundancy may be applied first. 4 rule mining algorithm pseudocode for our algorithm (an extension of the kingfisher algorithm (hämäläinen 2010, hämäläinen 2012) is given in algorithm 1. we find non-redundant rules using the natural log of fisher's p value (a decreasing measure) in a three stage process: 1. all potentially non-redundant rules with some minimum log p-value are identified. 2. rules identified in stage 1 are examined and specialisation redundant rules are pruned. 3. remaining rules are examined and generalisation redundant rules are pruned. stage 1 is a bfs over itemsets, which is described in algorithm 1. it is equivalent to the kingfisher algorithm, however uses less strict pruning to avoid removing potentially relevant rules prior to stages 2 and 3 (see section 4.1). australasian journal of information systems petersen, poon j, poon s & loy 2017, vol 21, selected papers from ausdm better rulesets by removing redundancy 9 4.1 identification of potentially non-redundant rules the search begins at step 1 by sorting attributes in decreasing order of frequency. any attribute whose frequency is too low to produce an interestingness value greater than α is removed at this stage. level 1 search nodes are then created for each remaining attribute (a node is called level k if it represents a set of k attributes). steps 2 – 4 describe the breadth first search over nodes. at each level i we create nodes by taking the union of two i-1 nodes. at steps 5-7, for each level k node x corresponding to attributes 𝑥1, 𝑥2, … , 𝑥𝑘, the p-values of the k rules 𝑋\𝑥𝑖 ⇒ 𝑥𝑖 are computed and compared against the minimum threshold α. the frequency of set x is calculated and stored, with p-values for rules being computed using frequencies of parent nodes. the number of iterations over the dataset is therefore limited to the number of nodes considered. each node maintains a length |a| bit vector of possible consequents (attributes a where the rule xq ⇒ a is possible). these vectors are initialised using bitwise and of vectors for parent nodes. as a node is processed, lower bounds on the log fisher's p value are computed for all rules of the form xq\a⇒a for all a in a. if bounds for all attributes exceed the relevance threshold (the vector of possible consequents is 0), the node is pruned from the search. the bounds used were first reported by hämäläinen (hämäläinen 2010), and are reproduced in table 5. this is described in steps 9-12, 17, and 18. each node x contains a vector with the best previous p-value for rules with consequent xi in x. this is computed in step 8. similar to the possible bit vector, these vectors are merged from parents when the node x is created. using classical redundancy, if the bound on p-values for rules with a given consequent exceeds the corresponding value in this vector that consequent algorithm 1: search for potentially non-redundant rules input: set of attributes a, dataset d, decreasing goodness measure m(), threshold α output: set of potentially non-redundant rules r 1. sort a decreasing by frequency in d 2. k = 1 3. while k ≤ |a| 4. for each candidate attribute k-set c 5. for each rule r: c\{a} ⇒ a ∀ a ∈ c 6. if m(r) ≤ α 7. add r to result set r 8. c.pbest[a] = min(c.pbest[a], m(r)) 9. for each rule r: c\{a} ⇒ a ∀ a ∈ a 10. bnd = lower bound on m for all specialisations of r 11. if bnd ≥ α 12. c.possible[a] = false 13. if a ∈ c 14. bndonq = lower bound on m for all specialisations of r conditioned on the additional attributes 15. if bnd > c.pbest[a] and bndonq > c.pbest[a] 16. c.possible[a] =false 17. if c.possible[a] == false for all a ∈ a 18. remove all attribute sets containing c from candidate attribute sets 19. k = k + 1 20. return r australasian journal of information systems petersen, poon j, poon s & loy 2017, vol 21, selected papers from ausdm better rulesets by removing redundancy 10 can also be considered impossible. using robust redundancy (see definition 2), we also test that the bound on rule xq ⇒ a|q is worse than the previous best value. the fishers p-value for rule xq ⇒ a|q takes its smallest value when the number of instances containing sets qa⌐x and qx⌐a are 0 and qxa and q⌐x⌐a are as large as possible. this occurs when freq(qxa) = freq(xa) and freq(q⌐x⌐a) = freq(⌐x⌐a). we therefore compute the bound for xq ⇒ a|q using bnd3 from table 5 with parameters f(xa)=freq(xa), f(x)=freq(xa), f(a)=freq(xa), and n=freq(xa)+freq(⌐x⌐a). this computation is performed in steps 13-14. algorithm 2: prune redundant specialisations input: set of rules r, dataset d, decreasing goodness measure m() output: set of non (specialisation) redundant rules r 1. for each consequent c 2. rc = all rules with consequent c 3. sort rc increasing on the length of the antecedent 4. for each rule ri in rc 5. for each rule rk in rc[i+1, |rc|] 6. x = antecedent(ri) 7. y = antecedent(rj) 8. if x ⊂ y 9. q = y\x 10. if m(x ⇒ c) ≤ m(y ⇒ c) and m(x ⇒ c) ≤ m(x ⇒ c | q) 11. delete rk 12. return r algorithm 3: prune redundant generalisations input: set of rules r, dataset d, decreasing goodness measure m() output: set of non-redundant rules r 1. for all r in r 2. keep(r) = false 3. hasspec(r) = false 4. for each consequent c 5. rc = all rules with consequent c 6. sort rc increasing on the length of the antecedent 7. for each rule ri in rc (reverse order) 8. if keep(ri) or not hasspec(ri) 9. for each rule rk in rc[i+1, |rc|] 10. y = antecedent(ri) 11. x = antecedent(rj) 12. if x ⊂ y 13. hasspec(ri) = true 14. q = y\x 15. if m(x ⇒ c | ⌐q) ≤ α 16. keep(rk) = true 17. for all r in r 18. if not keep(r) and hasspec(r) 19. delete r 20. return r australasian journal of information systems petersen, poon j, poon s & loy 2017, vol 21, selected papers from ausdm better rulesets by removing redundancy 11 4.2 pruning the search space the kingfisher algorithm (hämäläinen 2010) employs two additional pruning steps to control the size of the search space. the first, referred to as the lapis philosophorum principle, deals with the case where all rules of the form xq ⇒ a become impossible at a given node x{a}. in such a case, a is also an impossible consequence for children of the parent node x, and its possible consequents vector can be updated. this principle is also applied in our approach. the latter pruning step is pruning based on minimality. a rule x ⇒ a is considered minimal iff p(a|x)=1. for a given minimal rule x ⇒ a, any rule of the form xq ⇒ a or xqa ⇒ b will be either classically redundant or not significant (hämäläinen 2012). 𝒃𝒏𝒅𝟏(|𝑨|, 𝑵) = 𝒇(𝑨)! 𝒇(⌐𝑨)! 𝑵! 𝑏𝑛𝑑2(|𝑋|, |𝐴|, 𝑁) = 𝑓(⌐𝑋)! 𝑓(𝐴)! 𝑁! (𝑓(𝐴) − 𝑓(𝑋))! 𝑏𝑛𝑑3(|𝑋𝐴|, |𝑋|, |𝐴|, 𝑁) = 𝑓(𝐴)! 𝑓(⌐𝐴)! (𝑁 − 𝑓(𝑋𝐴))! 𝑁! 𝑓(⌐𝐴)! 𝑓(𝐴⌐𝑋)! table 5: lower bounds for fishers p. the function f(·) returns the frequency of its argument in d. pruning based on minimality cannot be employed when searching for rules with robust redundancy. we now prove that with robust redundancy it is possible for a specialisation of a minimal rule to be both significant and non-redundant. lemma 3: given data d, an increasing statistical goodness measure m(·), and rule x ⇒ a such that p(a|x)=1, xq ⇒ a may exist such that m(x ⇒ a) < m(x ⇒ a|q). x ⇒ a is minimal implies the frequency of set x⌐a is 0. the frequencies of sets xa, ⌐xa, and ⌐x⌐a are unknown. let q be a set of attributes whose corresponding rows in d exactly match the sets xa and ⌐x⌐a. m(x ⇒ a) increases with each occurrence of xa and ⌐x⌐a, and decreases with each occurrence of ⌐xa. it is easy to observe tat freq(xa)=freq(xqa), freq(⌐x⌐a)=freq(⌐xq⌐a), and freq(⌐xa) ≥ freq(⌐xqa). assuming d contains at least one occurrence of ⌐xa, m(x ⇒ a|q) will therefore be greater than m(x ⇒ a). ■ lemma 4: given data d, an increasing statistical goodness measure m(·), and a rule x ⇒ a such that p(a|x)=1, there may exist a rule xqa ⇒ b such that m(xa ⇒ b) < m(xa ⇒ b|q). x ⇒ a is minimal implies that the frequency of the set x⌐a is 0. hence freq(xa) ≥ freq(xqa). the frequencies of the sets xa, ⌐xa, and ⌐x⌐a are unknown. let q be a set of attributes whose rows in d exactly match the sets cb and ⌐c⌐b where c=xa. observe that freq(c)=freq(cq), freq(⌐c⌐b)=freq(⌐cq⌐b), and freq(⌐c) ≥ freq(⌐cq). assuming d contains at least one occurrence of ⌐cb, m(c ⇒ b|q) will therefore be greater than m(c ⇒ b) (or m(xa ⇒ b|q) > m(xa ⇒ b)). ■ 4.3 identification of redundant rules the process for pruning specialisation redundant rules is given in algorithm 2. after grouping rules based on their consequent, rules are sorted in increasing order based on the length of their antecedent (steps 1-3). all pairs of rules within each group are then considered (steps 47), and for those pairs which contain a specialisation / generalisation pair we compute the australasian journal of information systems petersen, poon j, poon s & loy 2017, vol 21, selected papers from ausdm better rulesets by removing redundancy 12 appropriate m() values to determine specialisation redundancy. rules which are found to be specialisation redundant are then deleted (steps 8 11). the process for removing redundant generalisations is described in algorithm 3. we begin by grouping rules based on consequent, then sort rules based on antecedent length (lines 4-7). we then traverse the rule list in reverse order, and for each rule consider each of its specialisations in turn (lines 9-12). as a rule only needs to be non-redundant w.r.t. one of its specialisations to be kept, we begin by tagging each rule as an exclude (line 2), then changing this value to true should an appropriate specialisation be found (lines 14-16). we also tag rules according to whether they have a specialisation to avoid erroneous removal (lines 3 and 13). rules are then removed according to these tags at the end of the algorithm (lines 17-19). finally, as discussed in section 3.3, we need to take care that rules which are already redundant w.r.t. one of their specialisations are not used when determining redundancy for their generalisations. to this end we skip any rule (line 8) which has a specialisation present in the database and is tagged for exclusion (recall that due to the order rules are traversed, such a rule would already have been compared against all its specialisations present in the ruleset). the running time for stages 2 and 3 are quadratic in the number of rules tested (in general this is dwarfed by the initial search in stage 1). when comparing two rules x ⇒ a and xq ⇒ a, specialisation redundancy requires the computation of m(x ⇒ a|q), and generalisation redundancy requires m(x ⇒ a|⌐q). for fisher's p, this requires us to obtain the frequencies for q, ⌐q, aq, and a⌐q. 5 evaluation performance is evaluated with respect to three characteristics: total number of rules, overall rule quality, and efficiency. all experiments were run on a pc running ubuntu linux, with an intel i7-4500 processor and 8gb ram. performance is also reported for rules generated with the classical definition of redundancy (definition 1), as well as a baseline with no redundancy based pruning. 5.1 data our evaluation uses the following data covering several domains. descriptive statistics are also given in table 6.  mushroom mushroom descriptions from the 1981 audobon society field guide to north american mushrooms. this data is available in the uci machine learning repository1.  t10i4d100k an artificial dataset representing market basket data, obtained from the frequent itemset mining dataset repository2.  t40i10d100k an artificial dataset representing market basket data, obtained from the frequent itemset mining dataset repository3.  diabetes collection of real world data reporting traditional chinese medical herbal prescriptions for diabetes. includes both the herbs prescribed and a binary classification of the patient outcome as 'good' or 'bad'.  fertility collection of real world data reporting traditional chinese medical herbal prescriptions for fertility. includes both the herbs prescribed and a binary classification of the patient outcome as 'good' or 'bad'. 1 https://archive.ics.uci.edu/ml/datasets/mushroom 2 http://fimi.ua.ac.be/data/ 3 http://fimi.ua.ac.be/data/ australasian journal of information systems petersen, poon j, poon s & loy 2017, vol 21, selected papers from ausdm better rulesets by removing redundancy 13  insomnia collection reporting traditional chinese medical herbal prescriptions for insomnia. includes both the herbs prescribed and a binary classification of the patient outcome as 'good' or 'bad'.  aspergillosis text documents (titles and abstracts) for articles considered for inclusion in a systematic review on aspergillosis (leeflang, deeks et al. 2008). each document is converted to a binary vector indicating the presence or absence of each of 100 words, as well as a binary variable indicating whether the title and abstract was potentially relevant to the review. the words selected were those with the greatest discriminative power when identifying articles relevant to the review. all values were obtained as the average of 10 independent experiments using a random 50/50 test/training split. where possible, results are reported with their 95% confidence interval. statistical significance tests are performed using a p-value of .05. in line with similar work, we measure interestingness using fishers exact test (we report the natural log of p-values) (hämäläinen 2012, li and zaiane 2015). thresholds for interesting rules were chosen to strike a balance between permissiveness and execution time, and differ between data and experiments. name # instances # attributes avg. instance length agg. attribute freq. aspergillosis 4377 101 15.93 ± 0.26 680.51 ± 66.05 mushroom 8124 119 23.00 ± 0.00 1624.80 ± 358.73 diabetes 1915 204 10.26 ± 0.11 105.21 ± 30.09 fertility 766 215 15.73 ± 0.32 59.62 ± 14.21 insomnia 460 112 13.48 ± 0.25 55.38 ± 11.10 t10i4d100k 100000 870 10.10 ± 0.02 1161.18 ± 74.73 t40i10d100k 100000 942 39.61 ± 0.05 4204.36 ± 249.57 table 6: summary of datasets used in the evaluation. 5.2 size of the rule set figure 2 shows the number of rules generated for each dataset, pruning approach, and threshold (raw values are given in a table in the appendix). not shown is the number of rules generated without redundancy based pruning; in all cases these values were substantially (often orders of magnitude) greater than with any type of pruning and were omitted to improve the readability of the graphs. the number of rules generated with robust redundancy is also significantly (p=.05) lower than for classical redundancy with most tested data. for the three exceptions (insomnia data with thresholds -30 and -35, and mushroom with threshold -2000), no significant difference in means is observed. australasian journal of information systems petersen, poon j, poon s & loy 2017, vol 21, selected papers from ausdm better rulesets by removing redundancy 14 figure 2: total ruleset size vs. goodness threshold (ln(p)) with classical, specialisation, and robust (specialisation and generalisation redundancy). each figure contains one spoke per tested threshold. values closer to the outside of the plots indicate more rules were produced. scales differ between subplots. the cases where no significant difference in the number of non-redundant rules is observed occur using the strictest thresholds. in addition, the difference between the mean number of rules generated appears to increase as the interestingness threshold is relaxed. the number of rules appears to converge as the bound on interesting rules is tightened, and diverge as it is relaxed. this supports the conclusion that our proposed approach is able to produce a practical number of rules from a larger number of potentially interesting associations. this quality is desirable as it allows the use of relaxed interestingness thresholds, lowering the risk of missing potentially useful associations. we now examine performance when exclusively removing redundant specialisations. given rule x ⇒ z, specialisation redundancy (definition 2) uses the conditional association x ⇒ z | q to provide an additional chance to obtain evidence for keeping rule xq ⇒ z (with respect to the classical approach defined in section 2). all specialisations that are not classically australasian journal of information systems petersen, poon j, poon s & loy 2017, vol 21, selected papers from ausdm better rulesets by removing redundancy 15 redundant will also not be robust redundant. robust specialisation redundancy will always return at least as many rules as the classical approach. 5.3 rule quality next we examine at the quality of the generated rules. as evidence has been given that we should not prune generalisations without first pruning specialisations, no results are reported for pruning generalisations exclusively. figure 3 shows the mean log p-values for each of the tested redundancy methods and thresholds (raw values are given in the appendix). despite the smaller generated rule set, it can be seen that in all cases the performance of robust pruning is equivalent or slightly better than for rules generated with classical redundancy. figure 3: ruleset performance (average ln(p) vs. goodness threshold (ln(p)) on hold out data with classical, specialisation, and robust (specialisation and generalisation redundancy). each figure contains one spoke per tested threshold. values closer to the outside of the plots indicate higher absolute mean quality (lower p values). scales differ between subplots. australasian journal of information systems petersen, poon j, poon s & loy 2017, vol 21, selected papers from ausdm better rulesets by removing redundancy 16 5.4 efficiency the expanded search for robust redundancy increases the amount of time and space required. the main factor that effects both computational time and memory requirements is the number of nodes generated during the search. this can be seen by observing the similarity of the trends for the number of nodes generated (figure 4) against time (figure 5) and memory (figure 6) (additional figures displaying these trends are included in the appendix). two factors contribute to the increased search space size. as robust specialisation redundancy is more permissive than classical redundancy, the pruning employed during the search must be less aggressive. additionally, we do not prune based on minimality with robust redundancy. we note however that as our initial search differs from the existing kingfisher algorithm only in the pruning strategies employed, it maintains the same worst-case time and space complexity. figure 4: number of nodes generated during rule search algorithm vs. goodness threshold (ln(p)) with classical, specialisation, and robust (specialisation and generalisation redundancy). each figure contains one spoke per tested threshold. values closer to the outside of the plots indicate more search nodes. scales differ between subplots australasian journal of information systems petersen, poon j, poon s & loy 2017, vol 21, selected papers from ausdm better rulesets by removing redundancy 17 figure 5: total search time (seconds) during rule search algorithm vs. goodness threshold (ln(p)) with classical, specialisation, and robust (specialisation and generalisation redundancy). each figure contains one spoke per tested threshold. values closer to the outside of the plots indicate higher run times. scales differ between subplots australasian journal of information systems petersen, poon j, poon s & loy 2017, vol 21, selected papers from ausdm better rulesets by removing redundancy 18 figure 6: peak memory usage (kb) during rule search algorithm vs. goodness threshold (ln(p)) with classical, specialisation, and robust (specialisation and generalisation redundancy). each figure contains one spoke per tested threshold. values closer to the outside of the plots indicate memory usage. scales differ between subplots figure 2 shows the number of nodes generated when searching with each data (values for robust and specialisation redundancy appear quite similar). observe the difference in number of nodes generated when searching with robust and classical redundancy varies substantially. some data (e.g. aspergillosis and t10i4d100k) differ very little, while the greatest difference is observed for the mushroom and t40i10d100k data. in addition to comparing the number of nodes when searching with classical and robust redundancy, it is interesting to examine the number of nodes when no redundancy based pruning is used. a substantial difference exists between the number of nodes generated without pruning when compared to robust redundancy. this implies the bounds computed during the search (reported in table 5) have a notable effect on the size of the search space. three exceptions occur with the aspergillosis, t10i4d100k, and t40i10d100k data. for aspergillosis and t10i4d100k we note there is also little difference between the number of nodes generated using robust and classical redundancy. the implication here is that the australasian journal of information systems petersen, poon j, poon s & loy 2017, vol 21, selected papers from ausdm better rulesets by removing redundancy 19 majority of the pruning is being done by lapis philosophorum. however, for t40i10d100k there is a substantial difference in performance with robust and classical redundancy. figure 5 reports the time for all searches (including pruning in algorithm 1 and 2). in all cases the required search time was quite manageable. even for t40i10d100k, all searches completed in less than 30 seconds (all other data completed much quicker). in practice memory appears to become an issue long before time required for the search. 6 conclusion this paper considers the problem of identifying and removing redundant associations in association rule mining. a new approach for identifying and removing redundant rules is presented, which we call robust redundancy. prior work compared rules based only with their respective contingency tables. such a comparison fails to consider information included in instances containing only part of the antecedent. robust redundancy is able to use this information to discover interesting specialisations that would be incorrectly removed with a classical approach. we also remove rules which are redundant artefacts of their non-redundant specialisations. unlike previous work (liu, hsu et al. 2001, webb 2010) that evaluate generalisations based on their exclusive domain with respect to the set of all specialisations, we base our method on comparisons to individual rules. we also present the first work using both specialisation and generalisation redundancy in a rule based context (as opposed to the work of webb (webb 2010) with itemsets). we demonstrate several situations in which classical redundancy based approaches can be confounded by interactions between variables, and show that the proposed approach is able to more accurately identify the correct underlying relationships in these situations. experimental analysis with multiple real and artificial data demonstrates that robust redundancy often produces smaller overall rule sets compared to classical redundancy. these rule sets hold as well or better in future data than those generated using classical redundancy. a limitation of our work is the increased time and space requirements of the rule generation compared to classical approaches. we generate rules using a modification of the kingfisher algorithm with less aggressive pruning; although the worst case complexity does not change, in practice the time requirements can increase substantially. however, as noted in section 5 performance on experimental data was still good enough to run in reasonable time on standard hardware. alternate search algorithms and pruning approaches to improve performance in this area are an interesting line of enquiry for future work. references aggarwal, c. c. and p. s. yu (2001). "a new approach to online generation of association rules." ieee trans. on knowl. and data eng. 13(4): 527-540. agrawal, r., t. imieli, #324, ski and a. swami (1993). "mining association rules between sets of items in large databases." sigmod rec. 22(2): 207-216. ashrafi, m. z., d. taniar and k. smith (2004). a new approach of eliminating redundant association rules. database and expert systems applications: 15th international conference, dexa 2004, zaragoza, spain, august 30-september 3, 2004. proceedings. f. galindo, m. takizawa and r. traunmüller. berlin, heidelberg, springer berlin heidelberg: 465-474. brin, s., r. motwani, j. d. ullman and s. tsur (1997). "dynamic itemset counting and implication rules for market basket data." sigmod rec. 26(2): 255-264. hämäläinen, w. (2010). efficient discovery of the top-k optimal dependency rules with fisher's exact test of significance. proceedings of the 2010 ieee international conference on data mining, ieee computer society: 196-205. australasian journal of information systems petersen, poon j, poon s & loy 2017, vol 21, selected papers from ausdm better rulesets by removing redundancy 20 hämäläinen, w. (2012). "kingfisher: an efficient algorithm for searching for both positive and negative dependency rules with statistical significance measures." knowledge and information systems 32(2): 383-414. leeflang, m., j. j. deeks, c. gatsonis and p. m. m. bossuyt (2008). "systematic reviews of diagnostic test accuracy." annals of internal medicine 149(12): 889-897. li, j. and o. zaiane (2015). associative classification with statistically significant positive and negative rules. proceedings of the 24th acm international on conference on information and knowledge management. melbourne, australia, acm: 633-642. liu, b., w. hsu and y. ma (2001). identifying non-actionable association rules. proceedings of the seventh acm sigkdd international conference on knowledge discovery and data mining. san francisco, california, acm: 329-334. mcgrane, m. and s. k. poon (2010). interaction as an interestingness measure. 2010 ieee international conference on data mining workshops. piatetsky-shapiro, g. (1991). discovery, analysis and presentation of strong rul. knowledge discovery in databases. g. a. f. piatetsky-shapiro, w. j., aaai press: 229--248. song, c. and t. ge (2013). discovering and managing quantitative association rules. proceedings of the 22nd acm international conference on conference on information & knowledge management. san francisco, california, usa, acm: 2429-2434. tan, p.-n., v. kumar and j. srivastava (2004). "selecting the right objective measure for association analysis." information systems 29(4): 293-313. verhein, f. and s. chawla (2007). using significant, positively associated and relatively class correlated rules for associative classification of imbalanced datasets. seventh ieee international conference on data mining (icdm 2007). webb, g. i. (2006). discovering significant rules. proceedings of the 12th acm sigkdd international conference on knowledge discovery and data mining. philadelphia, pa, usa, acm: 434-443. webb, g. i. (2007). "discovering significant patterns." machine learning 68(1): 1-33. webb, g. i. (2010). "self-sufficient itemsets: an approach to screening potentially interesting associations between items." acm trans. knowl. discov. data 4(1): 1-20. zaki, m. j. (2000). generating non-redundant association rules. proceedings of the sixth acm sigkdd international conference on knowledge discovery and data mining. boston, massachusetts, usa, acm: 34-43. australasian journal of information systems petersen, poon j, poon s & loy 2017, vol 21, selected papers from ausdm better rulesets by removing redundancy 21 appendices number of rules the following table describes the average number of rules generated across 10 runs (as described in section 5.1 of the manuscript). values are reported with their 95% confidence intervals. alpha value is the goodness threshold used (rules were evaluated using the natural log of fishers p values). dataset α no prune classic robust specialisations robust (both) aspergillosis -50 29548.60 ± 4461.94 389.40 ± 31.39 391.60 ± 31.48 268.30 ± 21.10 -75 4207.80 ± 352.07 119.20 ± 6.36 120.10 ± 6.41 88.90 ± 2.82 -100 1073.60 ± 124.63 55.90 ± 5.90 56.20 ± 5.87 41.70 ± 3.45 -125 383.10 ± 33.97 26.90 ± 2.55 27.00 ± 2.57 22.20 ± 1.77 -150 163.90 ± 16.29 19.50 ± 1.55 19.50 ± 1.55 16.50 ± 1.28 -175 89.90 ± 6.74 14.60 ± 1.08 14.70 ± 1.04 12.90 ± 1.02 -200 49.40 ± 5.33 10.00 ± 1.04 10.20 ± 1.10 9.20 ± 0.77 diabetes -15 34543.20 ± 12832.45 1327.80 ± 99.71 1676.30 ± 235.23 823.60 ± 83.15 -20 11816.40 ± 2574.97 613.00 ± 52.78 699.90 ± 73.42 394.40 ± 38.01 -25 4152.80 ± 227.20 343.40 ± 18.07 365.00 ± 21.57 224.90 ± 10.38 -30 2731.10 ± 229.94 244.50 ± 10.73 248.60 ± 11.75 162.40 ± 9.20 -35 1656.20 ± 102.51 180.80 ± 7.58 183.40 ± 7.73 126.30 ± 5.47 -40 1198.80 ± 86.62 143.60 ± 7.21 145.10 ± 7.38 102.50 ± 4.95 -45 782.30 ± 78.51 107.10 ± 11.24 107.20 ± 11.29 78.80 ± 7.13 fertility -15 362405.60 ± 86789.45 595.30 ± 45.95 618.30 ± 48.06 352.80 ± 25.40 -20 141740.50 ± 41787.17 278.00 ± 15.74 283.80 ± 15.80 176.80 ± 12.03 -25 42389.60 ± 11074.22 176.10 ± 5.92 178.20 ± 6.07 111.20 ± 5.71 -30 20686.90 ± 4609.10 132.50 ± 14.70 133.20 ± 15.04 85.00 ± 9.83 -35 12736.90 ± 2347.48 113.60 ± 6.68 114.00 ± 7.06 72.80 ± 4.27 -40 7713.60 ± 1830.05 90.50 ± 7.11 90.80 ± 7.11 62.40 ± 6.12 -45 4718.80 ± 687.50 74.50 ± 10.29 74.60 ± 10.33 48.70 ± 7.68 insomnia -15 12812.70 ± 2399.55 624.00 ± 48.35 747.80 ± 64.69 402.10 ± 33.42 -20 3180.50 ± 998.68 243.60 ± 37.06 269.00 ± 40.89 161.20 ± 24.88 -25 876.70 ± 362.33 104.50 ± 15.75 112.50 ± 14.98 72.30 ± 12.27 -30 271.10 ± 36.97 43.60 ± 6.41 46.80 ± 6.78 32.00 ± 4.42 -35 136.40 ± 23.64 24.30 ± 5.93 25.40 ± 5.86 17.40 ± 3.59 -40 59.40 ± 8.13 10.60 ± 1.25 12.40 ± 2.19 8.20 ± 1.20 -45 31.60 ± 7.02 5.20 ± 1.70 6.10 ± 1.93 4.60 ± 1.4 mushroom -1250 61767.80 ± 158.39 409.70 ± 5.32 568.70 ± 7.42 227.40 ± 2.95 -1375 37501.10 ± 4080.51 308.10 ± 7.70 342.40 ± 10.14 166.00 ± 6.38 -1500 22634.50 ± 92.39 229.70 ± 5.67 239.40 ± 5.97 125.50 ± 1.67 -1625 22049.80 ± 39.14 191.30 ± 2.54 196.30 ± 2.85 114.80 ± 1.77 -1750 19980.00 ± 2498.40 140.70 ± 5.51 141.70 ± 5.51 93.30 ± 6.17 -1875 7507.80 ± 78.31 88.60 ± 3.75 89.60 ± 3.75 56.70 ± 1.08 -2000 6430.80 ± 522.10 38.90 ± 5.39 39.90 ± 5.39 34.70 ± 4.15 t10i4d100k -500 17287.60 ± 191.08 6114.80 ± 53.65 6114.80 ± 53.65 4302.40 ± 43.14 -750 3484.70 ± 73.63 1568.00 ± 28.03 1568.00 ± 28.03 1217.30 ± 21.15 -1000 750.40 ± 31.17 411.90 ± 12.19 411.90 ± 12.19 353.50 ± 9.51 -1250 169.70 ± 3.67 99.80 ± 2.51 99.80 ± 2.51 85.30 ± 2.30 -1500 76.70 ± 4.39 41.70 ± 2.73 41.70 ± 2.73 36.50 ± 2.47 -1750 28.50 ± 3.82 16.90 ± 1.55 16.90 ± 1.55 15.60 ± 1.31 -2000 2.90 ± 1.53 2.90 ± 1.53 2.90 ± 1.53 2.90 ± 1.53 t40i10d100k -2000 297056.60 ± 16747.78 5477.00 ± 211.44 5477.00 ± 211.44 3675.70 ± 133.93 -2125 227409.30 ± 30167.75 4165.60 ± 181.86 4165.60 ± 181.86 2874.70 ± 116.69 -2250 80480.40 ± 31503.32 3001.90 ± 159.61 3001.90 ± 159.61 2195.80 ± 93.08 -2375 32533.60 ± 24486.95 1746.40 ± 372.79 1746.40 ± 372.79 1323.10 ± 264.55 -2500 5693.70 ± 611.02 660.20 ± 78.37 660.20 ± 78.37 528.70 ± 69.71 -2625 1933.20 ± 631.18 341.90 ± 58.24 341.90 ± 58.24 282.20 ± 47.46 -2750 615.10 ± 272.55 193.50 ± 57.87 193.50 ± 57.87 172.50 ± 51.47 australasian journal of information systems petersen, poon j, poon s & loy 2017, vol 21, selected papers from ausdm better rulesets by removing redundancy 22 average p values the following table describes the average p value on hold out data for rules on 7 data sets. results were generated across 10 runs using a 50/50 training/test split (as described in section 5.1 of the manuscript). values are reported with their 95% confidence intervals. alpha value is the goodness threshold used (rules were evaluated using the natural log of fishers p values). dataset α no prune classic robust specialisations robust (both) aspergillosis -50 -57.44 ± 3.85 -73.19 ± 2.67 -73.25 ± 2.64 -77.29 ± 2.86 -75 -91.80 ± 3.04 -115.51 ± 2.75 -115.36 ± 2.73 -121.00 ± 2.09 -100 -125.78 ± 5.52 -155.54 ± 9.36 -155.37 ± 9.22 -165.06 ± 7.93 -125 -157.39 ± 5.65 -206.80 ± 10.15 -206.60 ± 10.25 -215.07 ± 9.12 -150 -197.29 ± 8.45 -240.32 ± 7.31 -240.32 ± 7.31 -249.13 ± 7.18 -175 -224.52 ± 6.51 -265.57 ± 7.95 -265.10 ± 7.75 -273.72 ± 8.71 -200 -259.87 ± 11.65 -297.70 ± 13.49 -296.33 ± 14.14 -304.60 ± 11.22 diabetes -15 -12.17 ± 1.72 -16.78 ± 0.90 -15.30 ± 1.28 -17.67 ± 1.39 -20 -18.97 ± 2.38 -26.30 ± 1.74 -24.48 ± 1.91 -27.65 ± 2.27 -25 -30.79 ± 1.34 -36.92 ± 1.58 -35.67 ± 1.53 -39.42 ± 1.54 -30 -35.59 ± 2.38 -43.34 ± 2.05 -43.11 ± 2.05 -47.00 ± 2.27 -35 -44.02 ± 1.78 -50.70 ± 1.22 -50.44 ± 1.18 -54.14 ± 1.27 -40 -48.68 ± 2.62 -55.41 ± 2.06 -55.15 ± 2.07 -58.88 ± 1.94 -45 -56.41 ± 3.62 -62.17 ± 4.00 -62.14 ± 4.01 -66.32 ± 4.32 fertility -15 -14.63 ± 1.86 -20.65 ± 1.35 -20.18 ± 1.29 -21.54 ± 1.23 -20 -19.21 ± 3.37 -33.41 ± 3.24 -33.00 ± 3.06 -34.13 ± 3.22 -25 -29.80 ± 3.75 -43.14 ± 3.31 -42.84 ± 3.30 -42.72 ± 3.14 -30 -34.94 ± 3.51 -54.64 ± 3.28 -54.49 ± 3.30 -55.67 ± 2.92 -35 -38.45 ± 4.19 -56.66 ± 4.09 -56.55 ± 4.10 -57.15 ± 4.08 -40 -44.74 ± 5.33 -62.70 ± 6.00 -62.55 ± 5.92 -63.21 ± 5.56 -45 -50.15 ± 3.88 -66.96 ± 5.25 -66.92 ± 5.28 -66.96 ± 5.36 insomnia -15 -10.67 ± 1.41 -13.27 ± 0.74 -12.83 ± 0.80 -13.05 ± 0.72 -20 -16.50 ± 2.50 -18.77 ± 1.76 -18.68 ± 1.75 -18.61 ± 1.78 -25 -24.19 ± 3.16 -24.32 ± 1.88 -24.58 ± 1.97 -24.39 ± 2.06 -30 -31.51 ± 1.70 -30.45 ± 1.52 -30.45 ± 1.64 -30.64 ± 1.62 -35 -35.02 ± 3.08 -33.93 ± 2.99 -34.37 ± 3.04 -35.13 ± 3.09 -40 -44.25 ± 4.01 -41.28 ± 2.06 -40.81 ± 2.88 -42.70 ± 2.77 -45 -57.66 ± 5.25 -55.98 ± 6.25 -55.34 ± 6.45 -58.16 ± 7.72 mushroom -1250 -1542.40 ± 8.59 -1604.50 ± 9.79 -1535.21 ± 8.13 -1622.80 ± 9.53 -1375 -1674.28 ± 39.12 -1700.65 ± 14.68 -1676.63 ± 14.71 -1733.22 ± 20.36 -1500 -1853.86 ± 17.29 -1793.91 ± 19.56 -1787.94 ± 19.16 -1842.73 ± 17.71 -1625 -1860.36 ± 11.36 -1843.53 ± 10.95 -1840.13 ± 11.21 -1869.33 ± 10.59 -1750 -1886.27 ± 24.84 -1903.90 ± 8.54 -1904.92 ± 8.53 -1915.04 ± 9.57 -1875 -2058.56 ± 15.00 -1996.28 ± 15.97 -1997.05 ± 15.92 -2016.88 ± 13.73 -2000 -2064.61 ± 13.26 -2040.31 ± 15.01 -2040.44 ± 14.65 -2044.43 ± 13.54 t10i4d100k -500 -654.97 ± 3.75 -677.78 ± 2.91 -677.78 ± 2.91 -691.13 ± 3.47 -750 -901.33 ± 7.01 -923.40 ± 5.67 -923.40 ± 5.67 -934.32 ± 6.11 -1000 -1155.35 ± 11.23 -1171.87 ± 9.15 -1171.87 ± 9.15 -1174.81 ± 8.79 -1250 -1496.84 ± 15.98 -1489.76 ± 14.07 -1489.76 ± 14.07 -1497.53 ± 14.53 -1500 -1679.61 ± 30.62 -1690.91 ± 30.69 -1690.91 ± 30.69 -1702.46 ± 30.36 -1750 -1839.38 ± 44.39 -1865.57 ± 41.17 -1865.57 ± 41.17 -1869.08 ± 40.72 t40i10d100k -2000 -2205.18 ± 32.30 -2282.00 ± 17.71 -2282.00 ± 17.71 -2302.10 ± 18.75 -2125 -2201.06 ± 42.97 -2339.99 ± 33.51 -2339.99 ± 33.51 -2360.88 ± 33.30 -2250 -2305.75 ± 57.95 -2408.56 ± 28.13 -2408.56 ± 28.13 -2423.45 ± 26.10 -2375 -2410.26 ± 100.15 -2470.00 ± 58.98 -2470.00 ± 58.98 -2481.85 ± 56.55 -2500 -2585.09 ± 42.48 -2621.87 ± 51.48 -2621.87 ± 51.48 -2631.06 ± 52.50 -2625 -2691.38 ± 36.23 -2748.60 ± 26.65 -2748.60 ± 26.65 -2756.40 ± 25.11 -2750 -2686.73 ± 40.39 -2706.56 ± 33.16 -2706.56 ± 33.16 -2710.19 ± 33.49 australasian journal of information systems petersen, poon j, poon s & loy 2017, vol 21, selected papers from ausdm better rulesets by removing redundancy 23 search time average search time (measured in seconds) vs. goodness threshold (natural log of fishers p) for 7 data sets. number of search nodes number of search nodes generated vs. goodness threshold (natural log of fishers p) for 7 data sets. australasian journal of information systems petersen, poon j, poon s & loy 2017, vol 21, selected papers from ausdm better rulesets by removing redundancy 24 peak memory usage peak memory usage (measured in kb) vs. goodness threshold (natural log of fishers p) for 7 data sets. copyright: © 2017 petersen, poon j, poon s & loy. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems warren & burmeister 2017, vol 21, research on applied ethics and ict editorial 1 editorial: research on applied ethics involving emerging ict technologies matthew warren deakin university oliver burmeister charles sturt university oburmeister@csu.edu.au this is the inaugural special section on applied ethics for ajis. there is a link between information system and ethics. this link can be traced back to the early days of computers and the ethical issues regarding the manner in which information could be used or misused, this relationship is often described as “information ethics” (floridi, 2006). the concept of information ethics has never been more relevant than today, the way that organisations and governments can use and manipulate people’s information to change their purchasing habits or even their voting habits. two guest editors were appointed, professor matthew warren (deakin university) and associate professor oliver burmeister (charles sturt university). nine submissions were received, of which three were accepted for this special section. an overview of those three follows, after which the guest editor backgrounds are described. the three accepted articles cover a broad spectrum of applied ethics to do with emerging technologies, from mobile health (mhealth), to considerations of autonomy and to professional ethics. the paper by o'connor et al., focuses on applied ethics as it conducts a philosophical examination from a moral standpoint of a public health issue that necessitates moral judgment of caregivers of children. some mhealth technologies, such as smart phone technologies, have been around for a long time, yet new ones keep emerging, including new applications for old technologies, again including smart phones. the paper on autonomy considerations and emerging icts is a different sort of exploration of applied ethics. for example, it raises an old issue in the philosophy of technology, in a new context, by relating the role of autonomy in technology design. the paper on professional ethics brings a different perspective again. it examined the common challenges and strategies for solving them, related to all technology development and management, rather than any specific technology. it addresses how professionals can better consider ethical implications of any technology. in relation to the acceptance of this paper on professional ethics, which was co-authored by one of the guest editors, it should be noted that he was not involved in any of the reviewing and editorial decision making for that paper. instead those tasks were carried out by the editor-in-chief of ajis. the ajis editorial system was set-up such that dr burmeister could not see that submission, nor any processing of it. about the guest editors. this editorial marks the first joint publication between these two. however, for the past two decades they have frequently served together on committee work. that has been particularly evident on two fronts. firstly, they were both executive members of the australasian institute on computer ethics (aice), which in the early 2000s had over 30 institutional members, and represented not only ict professionals, but also those in the ict industry whose primary work was other than ict (such as engineering), and who therefore did not qualify for membership of the ict professional body, the australian computer society (acs). both, at different times, served as aice director. both served on numerous organising committees of the bi-annual aice conferences. secondly, both have been and continue to be members of the acs ethics committee. professor matthew warren has published eight times in ajis over the past 16 years (senarathna, et al, 2016, pye and warren 2006, pierce, et al 2006, leitch and warren 2003, warren and hutchinson 2003, hutchinson and warren 2002a, hutchinson and warren 2002b, hutchinson and warren 2001). professor warren has australasian journal of information systems warren & burmeister 2017, vol 21, research on applied ethics and ict editorial 2 recently changed his academic role at deakin university from being a chair of information systems to a professor of cyber security, much of his current research is focused on ethical issues in relation to cyber security. associate professor burmeister has published five times in ajis over the past 22 years (bowern, burmeister, gotterbarn, & weckert, 2006; burmeister, 1995, 2015; burmeister, islam, dayhew, & crichton, 2015; simpson, nevile, & burmeister, 2003). most of his ict ethics work been in the areas of value sensitive design (burmeister, 2010, 2012, 2016; burmeister, weckert, & williamson, 2011; pakrasi, burmeister, mccallum, coppola, & loeb, 2015; teipel et al., 2016) and professional ethics (berleur et al., 2008; burmeister, 2000, 2013, 2017; burmeister & sharma, 2005; burmeister & weckert, 2003). references berleur, j., burmeister, o., duquenoy, p., gotterbarn, d., goujon, p., kaipainen, k., whitehouse, d. (eds.). (2008). ethics of computing committees: suggestions for functions, form, and structure. laxenburg, austria: ifip press. bowern, m., burmeister, o. k., gotterbarn, d., & weckert, j. (2006). ict integrity: bringing the acs code of ethics up to date. australasian journal of information systems, 13(2), 168-181. doi:http://dx.doi.org/10.3127/ajis.v13i2.50 burmeister, o. k. (1995). evaluating the factors that facilitate a deep understanding of data analysis. australasian journal of information systems, 3(1). doi:http://dx.doi.org/10.3127/ajis.v3i1.390 burmeister, o. k. (2000). applying the acs code of ethics. journal of research and practice in information technology, 32(2), 107-120. burmeister, o. k. (2010). websites for seniors: cognitive accessibility. australian journal of emerging technologies and society, 8(2), 99-113. retrieved from https://www.scopus.com/inward/record.uri?eid=2-s2.079251542361&partnerid=40&md5=7a80d0de2e5e424552b196b2f71fac56 b u r m e i s t e r , o . k . ( 2 0 1 2 ) . w h a t s e n i o r s v a l u e a b o u t o n l i n e c o m m u n i t y . j o u r n a l o f c o m m u n i t y i n f o r m a t i c s , 8 ( 1 ) . r e t r i e v e d f r o m h t t p : / / c i journal.net/index.php/ciej/article/view/545 burmeister, o. k. (2013). achieving the goal of a global computing code of ethics through an international-localisation hybrid. ethical space: the international journal of communication ethics, 10(4), 25-32. burmeister, o. k. (2015). improving professional it doctorate completion rates. australasian journal of information systems, 19, 55-70. doi:10.3127/ajis.v19i0.1073 burmeister, o. k. (2016). the development of assistive dementia technology that accounts for the values of those affected by its use. ethics and information technology, 18(3), 185198. doi:10.1007/s10676-016-9404-2 burmeister, o. k. (2017). professional ethics in the information age. journal of information, communication & ethics in society, 15(2). burmeister, o. k., islam, m. z., dayhew, m., & crichton, m. (2015). enhancing client welfare through better communication of private mental health data between rural service providers. australasian journal of information systems, 19, 1-14. doi:http://dx.doi.org/10.3127/ajis.v19i0.1206 burmeister, o. k., & sharma, a. (2005). professionalism in ict: meeting the challenge of ethical dilemmas in the workplace. paper presented at the fourth australian institute of computer ethics conference, deakin university, geelong. burmeister, o. k., & weckert, j. (2003). applying the new software engineering code of ethics to usability engineering: a study of 4 cases. journal of information, communication & ethics in society, 3(3), 119-132. australasian journal of information systems warren & burmeister 2017, vol 21, research on applied ethics and ict editorial 3 burmeister, o. k., weckert, j., & williamson, k. (2011). seniors extend understanding of what constitutes universal values. journal of information, communication & ethics in society, 9(4), 238-252. doi:10.1108/14779961111191048 hutchinson w and warren m (2001) information warfare and ethics, vol. 8, pp. 58-62, australasian journal of information systems, wollongong, nsw, australia. hutchinson w, and warren m (2002a) information warfare: using the viable system model as a framework to attack organisations, vol. 9, pp. 67-74, australasian journal of information systems, wollongong, nsw, australia. hutchinson w and warren m (2002b) information warfare: using the viable system model as a framework to attack organisations, vol. 9, pp. 67-74, australasian journal of information systems, wollongong, nsw, australia. floridi, l. (2006) "information ethics, its nature and scope" sigcas computers and society (36:3), pp. 21 36 leitch s, and warren m (2003) designing systems for e-commerce, vol. 10, pp. 139-143, australasian journal of information systems, canberra, act, australia. pierce j, jones a, and warren m (2006) penetration testing professional ethics: a conceptual model and taxonomy, vol. 13, pp. 193-200, australasian journal of information systems, canberra, act, australia. pakrasi, s., burmeister, o. k., mccallum, t. j., coppola, j. f., & loeb, g. (2015). ethical telehealth design for users with dementia. gerontechnology, 13(4), 383-387. doi:10.4017/gt.2015.13.4.002.00 pye, g and warren, m (2006), striking a balance between ethics and ict governance, vol. 13, pp. 201-207, australasian journal of information systems, canberra, act, australia. senarathna, i, yeoh g, warren m, and salzman , s. (2016) security and privacy concerns for australian smes cloud adoption: empirical study of metropolitan vs regional smes, vol. 20, pp. 1-20, australasian journal of information systems, canberra, act, australia. simpson, c., nevile, l., & burmeister, o. k. (2003). doing ethics: a universal technique in an accessibility context. australasian journal of information systems, 10(2). doi:http://dx.doi.org/10.3127/ajis.v10i2.159 teipel, s., babiloni, c., hoey, j., kaye, j., kirste, t., & burmeister, o. k. (2016). information and communication technology solutions for outdoor navigation in dementia. alzheimer's & dementia: the journal of the alzheimer's association, 12(6), 695-707. doi:10.1016/j.jalz.2015.11.003 warren m, and hutchinson, w (2003) australian hackers and ethics, vol. 10, pp. 151-156, australasian journal of information systems, canberra, act, australia. copyright: © 2017 warren & burmeister. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems warren & burmeister 2017, vol 21, research on applied ethics and ict editorial 4 microsoft word ajis_18(3)_v10.docx australasian journal of information systems volume 18 number 3 2014 317 aligning capabilities and social media affordances for open innovation in governments andrea malsbender university of muenster ercis muenster, germany andrea.malsbender@ercis.uni-muenster.de sara hofmann university of muenster ercis muenster, germany sara.hofmann@ercis.uni-muenster.de jörg becker university of muenster ercis muenster, germany joerg.becker@ercis.uni-muenster.de abstract in recent years the proliferation of a new technological evolution has aroused great interest from science and practice: social media. subsuming different types of online applications where users can create, edit and share content, social media enables new forms of interaction with others. organisations already recognise its relevance and start leveraging on it. furthermore, organisations identify the role of service innovation as one major driver for economic growth and competitive advantage. nevertheless, although social media affords organisations to interact with their customers, the potential for utilising it for improving their service innovation activities is almost unexploited yet. therefore, this paper aims at providing a framework for supporting the use of social media affordances for service innovation. its applicability is tested in the domain “government”. based on theoretical constructs a conceptual analysis is presented, tested in a specific domain and supplemented by potential future research and implications for theory and practice. keywords: innovation, open innovation, innovation capabilities, government, egovernment, social media affordances. introduction the rise of social media technologies has provided organisations with new ways of engaging with their customers. besides publishing information, advertising products or recruiting employees (gregurec et al. 2011), companies increasingly use social media for open innovation activities, i.e. they integrate their customers’ suggestions into the innovation process. as companies need to stay competitive, they constantly have to search for innovations and implement them based on the needs, wishes and demands of their customers to prevail against competitors (menor and roth 2007). although public administrations differ from private sector companies as they do not have to compete with each other, governments, too, start to utilise social media for new ways of interacting with citizens, associations as well as organisations in open innovation activities (hofmann et al. 2013). their main drivers are the needs for saving costs, offering improved services and considering citizens’ needs in a digitalised environment (bekkers et al. 2006). swedish municipalities, for instance, have formed a network for collaborating both with each other as well as with citizens and organisations in order to increase their innovation potentials (feller et al. 2010). studies also suggest that taking part in open innovation australasian journal of information systems volume 18 number 3 2014 318 initiatives leads to an increased knowledge management for governments (chaston 2012). however, all in all, open innovation in government is still in its infancy (lee et al. 2012). although open innovation initiatives in the private sector are further advanced, a clearly structured process on how to leverage social media technologies for open innovation is missing there, too. motivated by this gap in the literature, our research objectives are the following: based on the existent literature, we (1) identify the general capabilities that are required for service innovations as well as the affordances of social media for engaging with customers. afterwards, we (2) develop a framework that matches the service innovation capabilities with the social media affordances in a structured way in order to identify how social media can support service innovation capabilities. finally, we (3) apply this framework to the government domain and analyse how governments can use social media to foster open innovation in social media. the remainder of our article is structured as follows. in the next section, we illuminate service innovations from a general viewpoint. based on the dynamic capability theory, we identify six service innovation capabilities. furthermore, we review the related work on social media before depicting the use of social media and especially its adoption for open innovation in governmental agencies. based on this fundament, we develop a domain-neutral framework that matches service innovation capabilities and social media affordances in the following section. in doing so, we first identify social media affordances for engagements with customers and subsequently attribute them to the service innovation capabilities they can support. in order to test our attribution as well as to identify the potentials of social media for open innovation in public agencies, we apply the framework to the government domain in the following section. we also outline challenges that governments might face as well as further positive effects in using social media besides developing service innovations. afterwards, we discuss our results before finally summarising our findings and giving an outlook to future research questions. related work service innovation during the last decade, research on innovation, originally introduced by schumpeter (1934), experienced a shift from solely focussing on innovations related to technological artefacts, respectively products, to research concerning both, products and services (johnson et al. 2000; spohrer and maglio 2008). as a result service innovation has become a focal point of attention. according to den hertog et al. (2010, p. 494), a “service innovation is a new service experience or service solution that consists of one or several of the following dimensions: new service concepts, new customer interaction, new value system/business partners, new revenue model, [or] new organizational or technological service delivery system.” the role of service innovation for economic growth and sustainable competitive advantage is increasingly recognised and acknowledged by managers of service organisations (menor and roth 2007). especially, issues regarding design and development aspects are highlighted (menor et al. 2002) and an extensive literature stream on service management, service marketing, service innovation, service-dominant logic perspective, open service innovation and, generally, service science already exists (den hertog et al. 2010). nevertheless, as menor and roth (2007, p. 825) state, “current theory and understanding of the strategies and tactics for developing new services is inadequate”. this is reflected by the fact, that, despite the significant rise of research on service innovation, many service firms still have to cope with their innovative efforts (jong and vermeulen 2003). due to the individual character of services, specific capabilities for managing new service development are needed. generally, an organisation’s capabilities can be differentiated into two types: operational capabilities and dynamic capabilities. operational capabilities are needed to perform “a coordinated set of tasks utilizing organizational resources, for the purpose of achieving a particular result” (helfat and peteraf 2003, p. 999). such capabilities are, e.g., marketing campaigns or distribution logistics (collis 1994). dynamic capabilities, introduced by teece et al. (1997), are the organisation’s ability to “integrate, australasian journal of information systems volume 18 number 3 2014 319 build and reconfigure internal and external competencies to address rapidly changing market environments” (teece et al. (1997), p. 516). for the development and management of new service experiences or solutions, six main dynamic service innovation capabilities are needed (den hertog et al. 2010). first, the organisation must be able to both, understand their customer by sensing their needs and promising technological options. second, the organisation needs the ability to detail and visualise services gradually and to conceptualise each component of the new service solution (e.g. target audience). third, as service innovations are often new configurations of existing elements, the organisation needs to be able to configure or (un-)bundle existing service activities to generate value. fourth, due to dependencies on other ‘co-designers’ of the new service, the organisation needs the capability to manage the service innovation across existing boundaries. fifth, for the introduction of the new service, the organisation has to be able to scale and stretch the service innovation from an experimental (i.e. r&d) to an organisation wide setting. sixth, based on already implemented innovation activities, organisations need to be able to learn from experiences and to adapt the overall service innovation process with respect to past failures and successes. social media social media has experienced a tremendous growth over the last decades and is increasingly recognised by organisations. motivated by the success of first movers, in 2003 the development of many social media technologies (e.g. myspace, linkedin) (boyd and ellison 2007) led to new ways for individuals to interact with others and for organisations to engage with customers (rishika et al. 2013). according to bolton et al. (2013) and kaplan and haenlein (2010), we define social media as a group of online applications through which users are allowed to create, edit, evaluate, share and/or link to a variety of user-generated content or to other users. this broad view of social media allows for the classification into different categories (e.g. photo services (flickr), music services (youtube), weblogs etc.). most of existing research focusses on two main social media technologies: social network sites (or online social networks or social networking sites), like facebook, and microblogging services, like twitter. this is caused by the fact, that conversations, reactions and personal information of users can be gathered and analysed easily. social network sites are web-based platforms, users can “(1) construct a public or semi-public profile within a bounded system, (2) articulate a list of other users with whom they share a connection, and (3) view and traverse their list of connections and those made by others within the system” (boyd and ellison 2007, p. 211). furthermore, they can be differentiated into interorganisational (so called enterprise social networks (esn)) and public available social network sites (sns). our research is not limited to a specific technology since different technologies enable different types of engagement with customers/citizens and can be leveraged diversely. open innovation in government in contrast to organisations, governments took their time before using social media on a large scale. an analysis of facebook pages in germany revealed that as of march 2012 only 12 of the 25 largest cities had actually launched a government page on facebook (hofmann et al. 2013). however, since then, public administrations have increased their social media activities and meanwhile use this new way of communication for various purposes. these new technologies are not only suitable for providing information to citizens but also for actively engaging and incorporating their ideas into government decisions (lee and kwak 2012). a popular branch of governments’ social media activities that summarises this trend is subsumed by the term open government, which was coined by us president barack obama. he defined open government as “creating an unprecedented level of openness in government” (obama 2008). this can be achieved by the three steps transparency, participation and collaboration. in order to reach the postulated goals, adequate technologies are needed to enable these steps. whereas transparency refers to providing information about government activities, participation australasian journal of information systems volume 18 number 3 2014 320 and collaboration involve the active contribution and integration of citizens into the processes of government. a promising way of integrating citizens into these processes is via open innovation (feller et al. 2010). open public innovations in this context are understood as “processes of coproduction between public agencies, nongovernmental organizations and community-based organizations” (edwards 2006, p. 175). on the one hand, taking part in open innovation activities increases the sense of community among the citizens because they feel that they have created value for their community. on the other hand, open innovation projects can improve government services, which is a crucial requirement for public agencies. the concept of open innovation has spilled over from the private sector where companies have long been relying on external resources for designing innovations (feller et al. 2010). open innovation has mainly been considered for businesses since innovations are often the outcome of a competition race between several companies. furthermore, innovations go along with revenue models, which governments do not develop. however, open innovation is not limited to the firm level (koldzin 2011). the need for integration and interoperability of (e-)government services forces governments to re-invent themselves (kinder 2010). in the uk, e-government projects have been found to reduce process costs within the administration. nevertheless, service delivery towards the citizens could not be improved. furthermore, e-government projects have so far only focused on technical perspectives but neglected the users’ side thus achieving only little improvements (feller et al. 2010). in order to foster innovations, which are essential for modernising governments, co-design between public authorities and citizen communities is a valuable instrument in achieving innovation. codesigning services by governments and other stakeholders is regarded as a central element in the capability to innovate. besides the benefits of improved services and an increased sense of community, open innovation is also a reaction to financial deficiencies in the public sector (chaston 2012). in case expert knowledge is not accessible or simply too expensive, open innovation solutions offer a valuable alternative. especially online open innovation is much cheaper and more time-saving compared to traditional calls for proposals (mergel 2011). however, open innovation does not simply mean setting up a social media site but it goes along with several success factors. a prerequisite for successful open innovation activities is the provision of data, i.e. open data that citizens can use for developing their ideas (thoreson and miller 2013). furthermore, governments need to be aware that open innovations will cause changing business models. therefore, innovation activities need to be aligned with the governments’ strategy (lee et al. 2012). since governments need to be aware both of their capabilities required for successful service innovation as well as the advantages that social media as platforms for open innovation offer, we develop a framework that matches service innovations capabilities with social media affordances and apply it to the governmental context. developing a framework for social media afforded innovation taking the theoretical stance of affordances and dynamic capability theory, we introduce a framework for service innovation enabled by social media. therefore, we present affordances of social media for engaging with external users. subsequently, we allocate the resulting affordances to each service innovation capability. the resulting framework can be used to analyse the usability of social media for service innovation in different domains. social media affordances for engagement with customers social media technologies afford users in various ways depending on specific domains and set of actions. taking an affordances lens allows for determining which action potential can be taken by using a specific technology (leonardi 2011). previous work on social media affordances focuses on their use within organisations. on the one hand, treem and leonardi (2012), focus on the affordances of social media for organisational communication in general. in particular, they identify four types of affordances, each divided into sub-affordances. first, they highlight the visibility of employees’ australasian journal of information systems volume 18 number 3 2014 321 behaviours, knowledge, preferences, and network connections provided by social media. second, social media enables persistent access to information as it remains accessible as originally provided by the author. third, social media affords users to edit information individually before it is viewed by others (e.g. profile information or content). fourth, the affordance association is the option of connections between users as well as users and content. on the other hand, majchrzak et al. (2013) also present four affordances from a knowledge sharing/management perspective and based on the identified types by treem and leonardi (2012). first, social media enables metavoicing, which is the ability to engage in ongoing online conversations. second, users (resp. employees of an organisation) can attend knowledge conversations triggered by timely automated alerts caused by changes to a specific content of interest. third, network-informed association affords users to engage in conversations based on relational ties to other users or content ties to specific information. fourth, users are afforded to conduct a generative role-taking, i.e. they can engage in conversations with individually defined roles in order to maintain a productive dialogue. contrary to existing research on social media affordances for an organisation’s internal use of social media, we focus on the affordances of social media for external interaction with unknown users (customers or citizens). in doing so, we change the context from internal to external and the task from general conversations or knowledge sharing to innovation design. based on the literature, our definitions of these affordances are the following: visibility. social media affords visibility of content, previously invisible in real life, to other users (individuals or organisations) (treem and leonardi 2012). on the one hand, in most social media technologies users provide information about their personal profile and their personal situation and habits by describing their age, location, interests and preferences. such information becomes visible to other users and can easily be analysed (boyd and ellison 2007). on the other hand, users are able to contribute (post) or consume (lurking or observing) content, opinions or general information. thereby, each contribution leads to visibility (in some cases circumscribed by privacy settings). once, content is visible, the consumption of information is possible (bolton et al. 2013). visibility is the ability of social media to make content generated by users (e.g. ideas, information, personal preferences) visible to other users (people or organisations). persistence. information in social media is persistent over time. constituted by the fact, that even when a user ‘logs out’ of a social media technology, the information or content s/he provided is still accessible for other users (treem and leonardi 2012). thereby, information is presented in the same form as it was displayed by the content generator and cannot be changed by other users. persistence is the ability of social media that content (e.g. posted ideas) remains accessible over time in the same form as the original display after the actor has finished his or her presentation. editability. the affordance editability refers to two main features provided by social media technologies (treem and leonardi 2012). first, users are able to form their contribution in isolation and in an asynchronous setting before presenting it to others. second, users can change (modify or revise) content communicated in the past. the latter is particularly relevant in wikis where editing helps improving the quality of existing content. editability is the ability of social media that content (e.g. ideas) can both, be generated in isolation and updated and refined by users (also influenced by other users’ reaction). association. social media technologies provide access to social networks of users, their knowledge, opinions and experiences (rishika et al. 2013). in most of these technologies connections are displayed (dependent on privacy settings), which enables tracking other users through the network graph (boyd and ellison 2007). furthermore, based on profile information users can be associated with content. this content can vary from preferences (e.g. “liking” of music styles) over following (e.g. specific actors or australasian journal of information systems volume 18 number 3 2014 322 athletes) to associations resulting from the “about me” section (e.g. preferences regarding political or religious orientation). association is the ability of social media to allow for associating ideas, people and other content. associations exist in two forms: first, an individual can be associated with another individual (social ties). second, an individual can be associated with a piece of information (content). metavoicing. with respect to the original interpretation of this affordance by majchrzak et al. (2013), metavoicing in social media generally affords users to react on content presented by other users. this feedback is generally sought by users posting an idea, content, or critical comment. as social media allows for feedback independent of specific skills or knowledge (e.g. “tweets” can be placed without any connection to a person or organisation in real life), it enables users to articulate their opinion to others’ postings. metavoicing is the ability of social media to enable users to react to others’ content (e.g. ideas, content, discussions, surveys, or polls). generativity. social media enables users to articulate their ideas or opinions. based on postings contributing content, creating or combining multiple sources, their personal preferences and thinking can be presented to others (bolton et al. 2013). this user-generated content is visible (visibility) to other users, can be commented (metavoicing) or connected (association). generativity is the ability of social media to enable users and communities to engage, learn, and innovate by expressing service ideas and service experiences. social media affordances for service innovation framework with respect to the affordances and the requirements specified by the dynamic service innovation capabilities, we present the underlying framework matching these theoretical constructs (see figure 1). therefore, we define for each capability the affordances having supportive features. generally, as each affordance supports all of the capabilities in varying degrees, we focus on highly relevant aspects. furthermore, although both perspectives (customer and organisation) can be afforded by social media, our focus is the benefit for the focal organisation. figure 1. social media affordances for dynamic service innovation capabilities (cf. den hertog et al. 2010) signalling of customer needs and technological developments. organisations constantly have to sense for customer needs and technological options (den hertog et al. 2010). for this purposes, they australasian journal of information systems volume 18 number 3 2014 323 can leverage social media based on the affordances generativity (g), visibility (v), metavoicing (m), association (a) and persistence (p). first, organisations can benefit from generativity initialising competitions motivating customers to present new innovations addressing a specific topic. second, organisations can present new ideas open for discussions. third, they can articulate a need for a new technology and for suggestions by experts (without paying salaries) to an unknown audience. fourth, organisations can present activities (i.e. competitions) related to innovations conducted by the organisation. overall, generativity supports both, engagement with and learning from customers and their experience. visibility of social media supports signalling by enabling access to formerly inaccessible information. organisations can extract customer opinions, experiences, or innovative ideas that become visible through social media and thereby identify potential needs. therefore, they can screen all existing (posted by users) ideas, technologies, innovations etc. second, based on profile information, the organisation can deduce implicit needs (kotler 1988). metavoicing supports signalling by reacting to comments/posts regarding an organisation’s own innovative activities or by reacting or commenting on innovations presented by customers. having an interactive relationship with customers improves customer perception of the overall organisation. based on content-related associations, potential innovations similar to already existing services can be identified by use of the affordance association. first, the organisation can identify who is interested in being part of ‘the organisational activities’ (follower, group associations etc.). these connections can be used to classify target customers based on their behaviours. second, the organisation can identify potential experts within ‘their network’. as information in conversations (comments, criticism etc.) is persistent over time, organisations can benefit from this affordance by analysing changes in the market environment, needs and thinking of the customers. furthermore, the service innovation development processes are documented over time. conceptualisation. within a conceptualisation of a new service innovation, organisations need to consider all relevant information (den hertog et al. 2010). for this purpose, different affordances can be helpful for an organisation: generativity (g), visibility (v), association (a), persistence (p) and editability (e). organisations can benefit from generativity by motivating customers to present relevant components that need to be considered within a conceptualisation of a new service innovation to improve quality. first, to allow for this engagement of the customer, this affordance enables organisations to present a concept to a wide audience. second, by presenting a concept, the organisation can test whether it is understandable and even feasible from a customer’s point of view. visibility can be utilised during conceptualisation by screening information about ideas, technological changes or opinions as input for new service concepts. furthermore, organisational activities (e.g. the new service development) become visible and can improve awareness of customers for co-creation. by use of association, the organisation can access relevant information. especially, information about the target audience (including detailed descriptions of their profiles) can be extracted based on a classification of relationships (people or content). furthermore, the organisation can benefit by searching for relevant aspects for the new service concept through associated content (e.g. competitors in this field or feasibility of the new concept based on experiences). organisations can benefit from persistence by incrementally developing new service concepts over time (treem and leonardi 2012). customer activities, ideas, and comments endure over time and can be used as potential input that needs to be considered within the conceptualisation of new services. editability enables the organisation to work on existing components of a service concept. in doing so, every follower is informed about the changes and can react directly. based on customer’s comments to a presented concept, the conceptualisation can progress incrementally to improve the quality of the resulting solution. configuration. aiming at (un-)bundling existing service activities to generate value for customers (den hertog et al. 2010), organisations can notably benefit from the affordances generativity (g), metavoicing (m), association (a) and persistence (p). generativity affords organisations to include customers’ opinions and experiences regarding feasibility of a new service bundle. thereby, motivated australasian journal of information systems volume 18 number 3 2014 324 for contribution, the external view of targeted customers can be included in analysing the fit of the requirement (customer) and capability (organisation). based on a request for innovative suggestions by customers, organisations are afforded by metavoicing to react to and discuss potential new service combinations. furthermore, they can discuss potential benefits of new combinations. by association, organisations can identify potential service bundles based on preferences of customers. for instance, a customer who has connections to two different existing services (preferences) has a tendency to prefer a combination of those. furthermore, affiliated in a group, additional service preferences can be derived based on others’ preferences. persistence supports storing past experiences for configuring existing service components in a new way (e.g. feedback to bygone service innovation activities to identify potential combinations of existing services). furthermore, customer needs can show whether new combinations should be bundled or existing bundles of service activities should be provided individually. learning. organisations are constantly forced to learn from past activities. especially, learning from failures or success of past service innovation endeavours can be supported by the affordances generativity (g), visibility (v), metavoicing (m) and persistence (p) of social media. generativity enables organisations to learn from experiences by actively asking for environmental reactions on implemented service innovations in the past. thereby, the organisation can use technologies like polls or motivate customers to comment on these activities. particularly, the opportunity to receive direct feedback by the ‘real’ customer and the opportunity to reach many customers is a benefit which is only afforded by social media. organisations can use visibility by analysing environmental reactions to implemented service innovations in the past. therefore, analyses of marketing campaigns or brand perception (especially for product development) have become focal points of attention for organisations. sentiments regarding new products or services can be analysed (bollen et al. 2011) and views of different customers (profile information) can be clustered regarding their relevance. organisations can benefit from metavoicing by being able to react to comments or posts on their past activities. in doing so, they are able to learn from customers’ perceptions of their service innovations. furthermore, they can divert an existing conversation to focus on specific topics of interest. being able to directly communicate with formerly unknown customers enables receiving concrete feedback. organisations can benefit from the affordance persistence by analysing reactions on past service innovations. furthermore a long-term analysis of failures and successes can be conducted. this analysis can then be used to identify (learn) potential for improvement for future activities. co-producing and orchestration, and scaling and stretching. for these capabilities none of the present affordances can be leveraged reasonably. as these capabilities primarily require internal or inter-organisational (with partners etc.) interaction, which is out of scope in our framework, none of the affordances for interaction with customers support auxiliary value. applying the framework to open innovation in the public sector like private sector companies, governments require dynamic service innovation capabilities to provide new services to citizens. they need to be able to “adapt to developments in different political, socioeconomic, technological and cultural environments” (bekkers et al. 2006, p. 10). furthermore, governments need to “anticipate the needs of different stakeholders in these environments, such as citizens, companies, societal organizations and other government organizations”. as shown above, these capabilities can be supported by the affordances of social media. in order to identify the advantages which arise for governments by using social media in open innovation processes, we apply the previously developed framework to a government context. signalling. in contrast to businesses, governments do not compete with other organisations due to their monopoly in offering public services. however, they, too, are facing financial shortcomings and need to cut costs (chaston 2012). this goal can be achieved by electronic services which can be processed australasian journal of information systems volume 18 number 3 2014 325 with fewer resources. nevertheless, the adoption of e-government services suffers from citizens refraining to use these services, which can partly be attributed to citizens’ distrust towards the government (bélanger and carter 2008). oftentimes, citizens feel that their interests are ignored by the government. in order to counteract this negative image, especially governments need to have the capability to sense citizens’ needs. generativity in social media allows to actively ask citizens to make suggestions to a challenge that the administration is facing. similar to this co-design approach is for instance participatory budgeting (http://www.participatorybudgeting.org.uk/). furthermore, the site challenge.gov offers us public bodies the possibility to post calls for participations asking citizens and organisations to contribute to solving a challenge. in order to identify citizens’ needs and wishes, governments can furthermore screen citizens’ posts on governments’ social media sites (visibility). citizens can also create petitions via social media (e.g. gopetition 2013) and thus display their wishes to public administrations. metavoicing allows governments to lead conversations into a certain direction or raise specific questions of interest. furthermore, it helps identifying citizens’ preferred ideas, e.g. based on the number of likes or the polarity of the answers. from a crowdsourcing perspective, citizens can jointly develop service ideas by building on and reacting to others’ posts. in doing so, innovations will contain several perspectives and thereby improve their quality. due to associations between citizens as well as between citizens and topics, governments can identify and address groups in social media that are concerned with a certain topic. in doing so, they can later on extract relevant content from these groups (cf. visibility). it is especially useful for governments since politically or locally active groups connect via social media. persistence, which is an enabler for the other affordances in service innovation, supports governments’ innovation capabilities in several ways. on the one hand, governments can analyse changes in the environment by extracting the stored information on the social media sites of other actors (cf. visibility) and thus identify requirements for their own services. they could, for instance, identify the potential to offer further mobile government services by identifying discussions that point at an increasing interest in mobile apps. on the other hand, permanently stored information enables them to evaluate how successful service innovations have evolved through past discussions on their social media site. therefore, governments can deduce how to behave in future service innovation activities. conceptualisation. as government services address the whole population, i.e. from youths to elderly people, it is crucial to develop target group-specific services or to group the services according to life events. elderly people, for example, are more likely to come to the governments’ sites in person whereas younger citizens prefer using electronic services (plattfaut et al. 2013). the affordances of social media can help governments to thoroughly conceptualise their services and business models. in addition to the advantages for signalling, social media can support the conceptualisation of new government services in various ways. governments can present new concepts for services via social media to which citizens can respond (generativity). the government digital service of uk government (https://twitter.com/gdsteam), for instance, publishes information on newly released services on twitter and asks their followers to comment on it. citizens can use the social media sites of governments for expressing their opinions of government services (visibility). as an example, citizens complained about too slow reactions of their government on the twitter account of @gov (https://twitter.com/gov). metavoicing is especially interesting for services that address a younger target group since this part of the population belongs to the heaviest users of social media (statista 2012). governments can evaluate a service concept by aggregating ‘likes’ as well as the streams of discussion. furthermore, social media affords governments to identify the needs of special target groups by following specific associations. for instance, university social media groups might reveal important information about how to tailor government services to the target group students. since social media posts are stored (persistence), governments can learn from past social media discussions and thus extract information relevant for the service conceptualisation. especially in wikis, governments can react to citizens’ critique and suggestions by editing the proposed service (editability). the san francisco homeless resource wiki, which is jointly operated by different interest groups and the city council, contains services for homeless australasian journal of information systems volume 18 number 3 2014 326 people that can be edited by everyone (http://sfhomeless.wikia.com/wiki/san_francisco_homeless_resource). configuration. in the governmental context, completely new services are seldom created from scratch as this would in most cases go along with major changes in the law. a typical way of structuring government services is to bundle them according to phases of life (http://www.qld.gov.au/queenslanders/). furthermore, governments might leave the core of the service unchanged but add another communication channel (e.g. the mobile channel). although, social media offer some benefits for configuration, compared to the capabilities signalling and conceptualisation, these advantages are rather minor ones. governments can post their configured service bundles and ask for citizens’ comments (generativity). furthermore, they can scan the citizens’ contribution (visibility) or they can follow discussion streams (metavoicing) in order to identify hints for service configurations. by analysing groups of interests (association) or past discussions (persistence), governmental agencies might get ideas for new configurations. however, currently we do not see any real world scenario in which configuration would be supported by the affordances of social media. learning. up to now, some governments struggle with implementing an efficient knowledge management (martin 2003). as open innovation is a rather new field for governments (lee et al. 2012), they have to identify best practices and learn from their experiences. furthermore, since government communication has mainly been mediated via mass media in the past and since public relations departments in public administrations are often understaffed (fisher liu and horsley 2007), governments have to figure out how to address citizens directly via social media – be it for open innovation activities or in general. governments can ask their citizens for direct feedback on previous service innovations as well as on the service innovation process on social media (generativity). this will help them to understand the citizens’ perspective of how their services both are created and perceived. examples from turkey and germany, for instance barnett (2013) and thaa (2013), suggest that people felt overlooked in public decisions. such misunderstandings could be accounted for through a productive dialogue on social media. analysing citizens’ posts (visibility) on the governments’ own social media site as well as in dedicated groups will help governments understand how the public perceives both the process as well as a service innovation itself. this is important since government services often go along with submitting private information and governments need to learn about citizens’ privacy concerns (bélanger and carter 2008). social media allow governments to directly address citizens by reacting to their posts (metavoicing). in doing so, they can interfere in the conversations such that they can learn how citizens have perceived previous service innovations. furthermore, they can analyse and evaluate citizens’ reactions (in terms of ‘likes’, comments, retweets etc.) on specific topics and identify best practices as well as improvements for future open innovation activities. as past content is stored in social media, governments can access all relevant information and use them for analysis purposes (persistence, cf. generativity, visibility, and metavoicing). discussion, conclusion, and limitations the developed framework provides a structure that shows how service innovation capabilities can be supported by social media. although, matching the social media affordances and capabilities worked quite well, the classification has not been completely unambiguous. in general, every social media affordance can somehow support most of the service innovation capabilities. however, we matched them only if an affordance provided a real value to a capability. the framework shows that especially generativity, visibility and metavoicing are affordances supporting the open innovation process to a high degree. it becomes clear that mainly the first steps in an innovation process represented by the capability signalling can benefit from open innovation approaches. from a governmental point of view, the framework has revealed several advantages and potential steps for governments to foster open innovation activities in social media. all proposed capabilities except australasian journal of information systems volume 18 number 3 2014 327 for configuration can be supported by social media activities. however, despite their advantages for governments, open innovation campaigns in social media provide several challenges. benefiting from generativity, for instance, requires resources to identify relevant contributions by citizens. this is regarded as one of the grand challenges for governments’ (and organisations’ in general) social media activities (chun and warner 2010). for visibility, too, resources and technological limitations play an important role in extracting relevant information. in addition, governments need to reassure their citizens that searching their profiles and their posts will not be used for purposes other than generating innovations. when extracting information from metavoicing, governments need to make sure to invest sufficient time and personnel resources. furthermore, they should abstain from influencing the conversations amongst citizens in such a way that citizens do not feel that their ideas will not be appreciated. certainly, public bodies need to ensure that their activities are not regarded as propaganda (fisher liu and horsley 2007) and that browsing through their citizens’ associations is not considered to be espionage. persistence raises the issue of a comprehensive strategy to archive previous discussions. finally, for editability, governments need to keep track of their previous posts. what is more, editing old statements needs to be done in a transparent way in order to prevent the suspicion that governments try to conceal previous actions. in addition to the described advantages, the six affordances come along with further positive side effects for governments. generativity, for instance, can also contribute to creating a sense of community among the citizens since they feel that their ideas are appreciated by government. this may strengthen the ties between government and citizens. similarly, visibility can connect different government organisations by sharing their ideas and thus learning from each other. governments are advised to openly appreciate citizens’ contribution to the innovation process. a side effect of metavoicing is governments being able to get into direct contact with citizens. previously, government communication has mainly been mediated through mass communication, which could lead to misunderstandings and misinterpretations (fisher liu and horsley 2007). using social media, governments can directly react to any information published about them. governments can benefit from association by attracting further ‘followers’ for their social media site and by creating new associations between governments and other actors. furthermore, governments can explicitly get into contact with groups that are supposed to hold a critical view of government activities and try to integrate them into the innovation design phase. persistence is also useful for governments in terms of learning about communication behaviour in social media by analysing the conversation streams and citizens’ reactions. we expect our research to make a significant contribution to theory, by matching capabilities organisations need to possess for successful service innovations with affordances provided by different social media technologies. generally, based on the combination of two theoretical foundations – dynamic capability theory and affordances theory – the developed framework can be classified as universally valid and hence, can be utilised in different scenarios and settings. more specifically, we defined several social media affordances for engaging with customers respectively citizens aiming at the development of service innovations. these affordances can be successfully utilised to support the capabilities of signalling, conceptualisation, configuration, and learning. for the support of coproducing and orchestration as well as scaling and stretching none of the identified affordances could be leveraged reasonably. from a practical perspective, organisations and in particular governments can use the presented framework to design their social media strategy and to identify potential improvements. with respect to each capability they can compare their actual social media activities to potential application possibilities afforded by social media. as organisations are forced to constantly react to technological evolutions, we present a structured approach to develop a social media strategy for organisations. this research is subject to two main limitations that will be part of future research. first of all, we tested our framework in the context of government, which is defined as non-profit organisation. although it australasian journal of information systems volume 18 number 3 2014 328 worked quite well, generalisability is not given yet. therefore, we need to apply the presented framework to further domains, each considering specific requirements. second, as this research is only conceptually we need to test it within real settings. with respect to these limitations, future research will concentrate on qualitative and quantitative analyses, evaluating whether the framework is applicable in each domain and thereby universally valid. furthermore, differences and best practices between public and private sector strategies regarding the encouragement of capabilities for open innovation strategies need to be identified. references barnett, a. 2013. “the battle for taksim square and the gezi park commune,” retrieved 1 august, 2013, from http://www.opendemocracy.net/anthony-barnett/battle-for-taksim-square-and-gezipark-commune. bekkers, v., van duivenboden, h., and thaens, m. 2006. “public innovation and information and communication technology: relevant background and concepts,” in information and communication technology and public innovation, v. bekkers, h. van duivenboden, and m. thaens (eds.), amsterdam, netherlands: ios press, pp. 3–21. bollen, j., pepe, a., and mao, h. 2011. “modeling public mood and emotion: twitter sentiment and socio-economic phenomena,” proceedings of the fifth icwsm11. bolton, r. n., parasuraman, a., hoefnagel, a., migchels, n., kabadayi, s., gruber, t., komarova, y., and solnet, d. 2013. “understanding generation y and their use of social media: a review and research agenda,” journal of service management (24:3), pp. 245–267. boyd, d. m., and ellison, n. b. 2007. “social network sites: definition, history, and scholarship,” journal of computer-mediated communication (13:1), pp. 210–230. bélanger, f., and carter, l. 2008. “trust and risk in e-government adoption,” the journal of strategic information systems (17:2), pp. 165–176. chaston, i. 2012. “open innovation and public administration: transformational typologies and business model impacts,” international journal of public administration (35:4), pp. 248–258. chun, s. a., and warner, j. 2010. “finding information in an era of abundance: towards a collaborative tagging environment in government,” information polity (15), pp. 89–103. collis, d. j. 1994. “research note: how valuable are organizational capabilities?,” strategic management journal (15:1), pp. 143–152. edwards, a. 2006. “empowering communities for environmental decision-making: innovative partnerships in cleveland (usa),” in information and communication technology and public innovation: assessing the ict-driven modernization of public administration, v. bekkers, h. van duivenboden, and m. thaens (eds.), amsterdam, netherlands: ios press, pp. 175–191. feller, j., finnegan, p., and nilsson, o. 2010. “open innovation and public administration: transformational typologies and business model impacts,” european journal of information systems (20:3), pp. 358–374. fisher liu, b., and horsley, j. s. 2007. “the government communication decision wheel : toward a public relations model for the public sector,” journal of public relations research (19:4), pp. 377–393. gopetition. 2013. “how to create a petition,” retrieved 1 august, 2013, from https://www.facebook.com/notes/gopetition/how-to-create-a-petition/122104031162717. gregurec, i., vranesevic, t., and dobrinic, d. 2011. “the importance of database marketing in social network marketing,” international journal of management cases (13:4), pp. 165–172. australasian journal of information systems volume 18 number 3 2014 329 helfat, c. e., and peteraf, m. a. 2003. “the dynamic resource-based view: capability lifecycles,” strategic management journal (24:10), pp. 997–1010. den hertog, p., van der aa, w., and de jong, m. w. 2010. “capabilities for managing service innovation: towards a conceptual framework,” journal of service management (21:4), pp. 490–514. hofmann, s., räckers, m., beverungen, d., and becker, j. 2013. “old blunders in new media? how local governments communicate with citizens in online social networks,” in proceedings of the 46th hawaii international conference on systems sciences, pp. 1–10. johnson, s. p., menor, l. j., roth, a. v., and chase, r. b. 2000. “a critical evaluation of the new service development process: integrating service innovation and service design,” in new service development creating memorable experiences, j. a. fitzsimmons and m. j. fitzsimmons (eds.), thousand oaks, ca: sage publications, pp. 1–32. jong, j. p. j. de, and vermeulen, p. a. m. 2003. “organizing successful new service development: a literature review,” management decision (41:9), pp. 844–858. kaplan, a. m., and haenlein, m. 2010. “users of the world, unite! the challenges and opportunities of social media.,” business horizons (53:1), pp. 59–68. kinder, t. 2010. “e-government service innovation in the scottish criminal justice information system,” financial accountability & management (26:1), pp. 21–41. koldzin, d. 2011. “government instruments to support open innovation-experiences from eu countries,” acta technica corviniensis bulletin of engineering (4), pp. 35–39. kotler, p. 1988. marketing management: analysis, planning, implementing and control, (6th ed.) englewood cliffs, nj: prentice hall. lee, g., and kwak, y. h. 2012. “an open government maturity model for social media-based public engagement,” government information quarterly (29:4), pp. 492–503. lee, s. m., hwang, t., and choi, d. 2012. “open innovation in the public sector of leading countries,” management decision (50:1), pp. 147–162. leonardi, p. m. 2011. “when flexible routines meet flexible technologies: affordance, constraint, and the imbrication of human and material agencies,” mis quarterly (35:1), pp. 147–167. majchrzak, a., faraj, s., kane, g. c., and azad, b. 2013. “the contradictory influence of social media affordances on online communal knowledge sharing,” journal of computer-mediated communication (19:1), pp. 38–55. martin, b. 2003. “knowledge management and local government: some emerging trends,” asia pacific management review (8:1), pp. 43–58. menor, l. j., and roth, a. 2007. “new service development competence in retail banking: construct development and measurement validation,” journal of operations management (25:4), pp. 825–846. menor, l. j., tatikonda, m. v., and sampson, s. e. 2002. “new service development: areas for exploitation and exploration,” journal of operations management (20:2), pp. 135–157. mergel, i. 2011. “crowdsorced ideas make participating in government cool again,” pa times, pp. 4–7. obama, b. 2008. “transparency and open government. memorandum for the heads of executive departments and agencies”. plattfaut, r., beverungen, d., hofmann, s., and räckers, m. 2013. “unravelling (e-)government channel selection : a quantitative analysis of individual customer preferences in germany australasian journal of information systems volume 18 number 3 2014 330 and australia,” in proceedings of the 46th annual hawaii international conference on system sciences, january 7-10, p. 9. rishika, r., kumar, a., janakiraman, r., and bezawanda, r. 2013. “the effect of customers’ social media participation on customer visit frequency and profitability: an empirical investigation,” information systems (7047), pp. 1–20. schumpeter, j. a. 1934. the theory of economic development, cambridge, ma: harvard university press. spohrer, j. c., and maglio, p. p. 2008. “the emergence of service science: toward systematic service innovations to accelerate co-creation of value,” production and operations management (17:3), 238–246. statista. 2012. “social networking: u.s. users by age group in 2012,” retrieved 1 august, 2013, from http://www.statista.com/statistics/221674/share-of-us-social-network-users-by-age-group/. teece, d. j., pisano, g., and shuen, a. 1997. “dynamic capabilities and strategic management,” strategic management journal (18:7), pp. 509–533. thaa, w. 2013. “‘stuttgart 21’ crisis or repoliticization of representative democracy?,” politische vierteljahresschrift (54:1), pp. 1–20. thoreson, k., and miller, t. 2013. “tapping technology to connect with the public”. treem, j., and leonardi, p. 2012. “social media use in organizations: exploring the affordances of visibility, editability, persistence, and association,” communication yearbook (36). acknowledgements this paper was written in the context of the research project networked service society (promotional reference apr 10/805) funded by the german federal ministry of education and research (bmbf). an earlier version of this paper was presented at the australasian conference on information systems (acis) 2013 in melbourne, australia. paper title (paper title style) australasian journal of information systems limbach, kuebel & zarnekow 2016, vol 20, research on rural and remote ict improving rural broadband deployment improving rural broadband deployment with synergistic effects between multiple fixed infra-structures felix limbach technical university berlin felix.limbach@tu-berlin.de hannes kuebel technical university berlin rüdiger zarnekow technical university berlin abstract the utilization of deployment synergies across infrastructure networks of different industries has been identified as a key to improve the broadband business case. thus, an increasing number of broadband plans require owners of physical infrastructures such as the electricity, pipeline, highway and railroad networks to host broadband infrastructure. however, crossindustry cooperation brings about new complexity to optimal utilization of deployment synergies. this paper explores cost savings that can be achieved if national non-telecommunication infrastructures are considered as source for broadband networks in rural areas. moreover, it assesses economic, political and regulatory measures required for improving synergy utilization. the presented approach is based on a techno-economic broadband deployment model, which is applied to all rural communities in germany. results indicate that synergy optimized network topologies can generally decrease rural broadband deployment cost. however, it is required that local authorities recognize the definition of broadband tender areas as a chance for the aggregation of demand. moreover, national regulators need to ensure that metro-aggregation, backbone and co-location costs, which are associated with nontelecommunication infrastructures, do not exceed the costs of the incumbent by more than 50%. keywords: broadband; rural; synergy evaluation; regulation; policy 1 introduction telecommunication networks are widely considered as a basis for economic growth and associated with high general economic benefits (ec, 2012b; ruhle, brusic, kittl, & ehrler, 2011). consequently, governments around the world define broadband plans to improve the availability of those infrastructures (falch & henten, 2010). the european commission has initiated the digital agenda to foster the provisioning of download rates of up to 100 mbps for european citizens (ec, 2010a). similarly, the australian government has proposed the 2016 corporate plan for australia’s broadband network (nbnco, 2015). irrespective of the employed fibre deployment variant and technology cost modelling, studies have repeatedly highlighted the economic challenges associated with rural broadband deployment (analysys mason, 2008, chatzi, lazaro, prat, & tomkos, 2013; hoernig et al., 2012; rokkas, katsianis, & varoutas, 2010). in the most sparsely populated areas, even a single infrastructure operator with a high market share may require public funding to become economically viable. however, to limit negative market effects and reduce aid to the necessary minimum all cost reduction measures should be exhausted before public funding is granted for a project (ec, 2013b). accordingly, recent proposals of the european commission suggest a variety of measures to maximize synergy utilization across physical networks (ec, 2013c). in this case regulation goes beyond the telecommunication sector and requires owners of physical 1 australasian journal of information systems limbach, kuebel & zarnekow 2016, vol 20, research on rural and remote ict improving rural broadband deployment infrastructures such as the electricity, pipeline, highway or railway network to host broadband infrastructures. similarly, the australian broadband plan denotes access to existing areal infrastructures of utilities as a key enabler for the national broad strategy (nbnco, 2015). the majority of broadband models consider mark-downs on calculations of a given greenfield network topology to account for synergy utilization in certain segments of the telecommunication access network (analysys mason, 2008; chatzi et al., 2013; jay, neumann, & plückebaum, 2014). moreover, cross-network synergy utilization in broadband deployment has been addressed for a combined network roll out scenario of utility and telecommunication companies (tahon et al., 2014). however, literature that assesses economic, political and regulatory implications associated with using national physical infrastructures such as railroads, highways, pipelines, power lines or railways as a fibre backbone in rural areas is hardly available. this contribution aims to fill this gap and will subsequently denote those infrastructures as alternative infrastructures. in the course of this paper, the following questions are explored: what is the financial magnitude of cost savings if alternative national infrastructures are considered as source for broadband networks in rural areas? what political and regulatory actions are required to fully leverage the cost reduction measures? the presented questions are addressed with a street-length aware synergy evaluation model based on openstreetmap (osm), i.e., an open-source geographical information system (gis). this article is structured into the following sections. the next section provides details on theoretical foundations and related work, and section 3 explains the considered deployment scenarios. section 4 presents the synergy evaluation model. in its subsections, first, a detailed overview of the model input is provided, and second, an internet access model is presented. subsequently, the model results are presented and discussed with respect to geographical preconditions for synergy utilization, topology characteristics and achievable cost advantages. thereafter, political and regulatory implications are discussed in section 6. finally, section 7 provides conclusions regarding the research objectives. 2 theoretical foundations and related work 2.1 characteristics of rural network topologies generally, telecommunication access networks can make use of coaxial cables, power line communications, wireless solutions and copper or fibre cables. of all available options, fibreto-the-home (ftth) is recognized as the most future-proof and reliable solution for broadband access. however, due to the high deployment costs, national broadband markets currently reach ftth saturation at a share of 20% (ftth council europe, 2013). especially for rural areas, fibre-to-the-cabinet (fttc) and fibre-to-the-building (fttb) have been proposed as intermediate steps in a gradual ftth deployment process (analysys mason, 2008). on average, they are up to 75% less expensive than ftth deployment (analysys mason, 2008; fcc, 2010). moreover, both deployment variants are suitable to meet the european commission broadband target: 100 mbps for 50% of the population (ec, 2010a, guenach et al., 2011). fig. 1 depicts a typical copper telecommunication access network that is connected to an incumbent’s central office (co) with a fibre backbone. it can be subdivided into a feeder cable and a distribution cable network. in rural areas, the feeder cable distributes the cable to different communities and the distribution points (dp) of the distribution cable network within a community. it is subdivided into joints (j) and forms a minimum spanning tree topology (vidmar, peternel, štular, & kos, 2010, dippon & train, 2000). that is, a connected graph of n verticals and (n-1) links, which minimizes the total cable length (christofides, 1975). verticals and dp constitute cable consolidation points between the co and the customer (u). generally, these consolidation points imply shared deployment costs in the subsequent cable segment. 2 australasian journal of information systems limbach, kuebel & zarnekow 2016, vol 20, research on rural and remote ict improving rural broadband deployment fig. 1 typical access network topology in rural areas (adapted from vidmar et al., 2010) fttc requires the replacement of copper cables in the feeder cable segment between the central office and a distribution point such as the street cabinet. similarly, fttb requires the replacement of the copper cable between the fibre backbone and the customer building. generally, these cables are deployed along existing roads, as national laws in many cases allow the government to use these rights of way at no charge for the setup of public telecommunication networks. moreover, this right can be transferred to private companies. 2.2 the role of public authorities in rural broadband deployment in rural areas, average feeder cable lengths are longer and account for a higher share of total broadband deployment costs than in areas with a higher population density (grubesic, 2008; schneir & xiong, 2013). moreover, these higher costs must be allocated to a smaller number of potential customers as in urban areas. consequently, economic feasibility is particularly challenging if a rural community is located far away from the co and encompasses few potential customers (analysys mason, 2008). under these conditions, private companies may not be willing to deploy or upgrade networks within a timeframe that allows reaching national or super-national broadband goals (ec, 2010a; nbnco, 2015). this will be referred to as market failure (ec, 2010b). in response to market failure, public authorities across the world pursue different approaches. authorities in europe have notified the european commission (ec) about large scale state aid measures that aim to deploy fibre networks along alternative infrastructures to foster rural broadband deployment (ec, 2013a, ec 2012a, ec, 2010b). these national broadband deployment frameworks allow a community to act as a co-investor that ensures economic feasibility of otherwise unprofitable broadband projects. typically, a local public authority defines a target area for market exploration and launching a tender. according to state aid guidelines, the authority is free to define the size of the target area. however, it should consider that a too small target area such as a single community may provide too little incentive to bid for public aid and that a too large target area may foreclose the outcome of the selection process (ec, 2013b). considering different economic, political and geographic preconditions other approaches to rural broadband deployment can be reasonable (given, 2010). the australian government has commissioned the state owned company the nextgen group to deploy additional fibre capacities to provide competitive wholesale backbone services in regional areas. in addition, the stated owned nbn corporation is commissioned to provide wholesale-only access network to all access seekers. the goal of this initiative is to provide download speeds of at least 50 mbps to 90% of the fixed line premises within the constraint that public equity capital is limited co/ fb j1 j2 j3 j4 j5 j6 j9 j10 dp dp dp u u u distribution cable network feeder cable network co/fb: central office/ fiber backbone j: joint dp: distribution point u: unit feeder cable distribution cable j8j7 3 australasian journal of information systems limbach, kuebel & zarnekow 2016, vol 20, research on rural and remote ict improving rural broadband deployment (nbnco, 2015). without additional regulatory measures this approach bares the risk that fibre is deployed only to the most profitable premises in a particular target area (given, 2010). both approaches exhibit an inherent risk of demand fragmentation in broadband deployment (sawhney, 1992). this paper will provide the political and regulatory implications for reducing the risk of demand fragmentation and improving synergy utilization of alternative infrastructures. 3 deployment scenarios and cost modelling 3.1 deployment scenarios it has been recognized that many states follow either fttc or fttb deployment strategies to reach broadband goals (cave, 2014). thus the relative magnitude of cost reduction effects that can be achieved by leveraging synergies with alternative infrastructures is explored in this paper for both fibre deployment variants. however, the political and regulatory implications which are presented in section 6 do also apply to other fibre deployment variants like for example fttp or fttn which is being discussed for rural broadband deployment in australia (nbnco, 2015). subsequently, fttc refers to the deployment of vdsl2 vectoring technology, which is capable of providing download speeds of up to 100 mbps (guenach et al., 2011). the required street cabinets are deployed within close proximity to the customer and host active equipment. it is assumed that required power grid access is available in every community. the fttb deployment scenario refers to a point to multipoint 10 gbps passive optical network (xg-pon), which is capable of providing minimum download speeds of 312 mbps if a splitting factor of 1:32 is used (schneir & xiong, 2013). this scenario assumes that electrical power is provided from the customer building. the analysis focuses on rural communities that do not host a co because in such a case households can receive broadband services directly from the co without additional fibre investments (grubesic, 2008). based on jay et al. (2014), the term rural will be used to refer to communities with a potential customer density of 130 or less inhabitants per square kilometre. moreover, it is assumed that one operator will upgrade a legacy copper access network in a given target area and that synergies with alternative infrastructures will be used if they are available. in the base case, infrastructure is deployed to all customers in a target area. moreover, the base case assumes equally distributed spare duct availabilities for a specific target area. furthermore, backbone and co-location costs for alternative infrastructures and the co are assumed to be at comparable costs in a competitive market (schäfer & schöbel, 2005). these assumptions are optimistic for several reasons. depending on country specific preconditions, cable and wireless networks can be important alternative rural access technology (oecd, 2009). to address this aspect, optimized topologies will also be calculated for 50%, 75%, 90% and 95% deployment scenarios. spare duct availability is likely to be higher along the pathway of the incumbent copper feeder cable topology. thus, the effect of lower spare duct availability will be assessed for the feeder cable topology that originates at the alternative infrastructure. finally, depending on market and regulatory preconditions, costs for backbone, co-location and metro aggregation networks could be either higher or lower than the incumbent offer. these aspects will be addressed with a sensitivity analysis. 3.2 cost modelling infrastructure costs have been derived from reviews of current literature on access network cost modelling and triangulated with publicly available information. table 1 summarizes the input values for fttc and fttb cost calculations. 4 australasian journal of information systems limbach, kuebel & zarnekow 2016, vol 20, research on rural and remote ict improving rural broadband deployment item fttc fttb € € fiber cable/m 1 1 street cabinet 13,000 / manhole / 850 y-branch / 29 table 1 capex for infrastructure elements the fttc scenario requires the deployment of new street cabinets that can host vectoring cards and other active equipment. in the fttb scenario, manholes are deployed at every road intersection within a community, and y-branch units are used to separate the individual drop cable from the distribution cable. both scenarios assume an average fibre cable price of 1 €/m. depending on the density of potential customers in rural areas, different shares and costs of installation methods will be assumed to account for the possibility of duct reuse, aerial deployment and decreasing surface restoration costs. the input parameters are provided in table 2. installation costs potential number of customers per km² aerial digging duct reuse % €/m % €/m % €/m 130 ≥ x > 75 0 15 80 60 20 2 75 ≥ x > 30 5 15 85 55 10 2 30 ≥ x > 15 10 15 85 50 5 2 <15 15 15 85 45 0 2 table 2 infrastructure installation methods following common practice, opex is considered as a mark-up on the modelled passive (1%) and active equipment (5%) (jay et al., 2014; schneir & xiong, 2013). moreover, metro aggregation, backbone and co-location costs are modelled as opex. in this sense, the monthly cost of 3.66 € per customer is derived from a market analysis that has been conducted by the german regulator (bundesnetzagentur, 2013). opex costs rise with the number of customers. therefore, for the first three years opex calculations consider a take-up of 25%, 50% and 70% (schneir & xiong, 2013). equal customer acquisition and customer churn rates are assumed to keep this take-up rate constant for the remaining years of the investment timeframe (schneir & xiong, 2013). it is assumed that networks are rolled out in equal portions over the first three years of a 10 year investment timeframe. moreover, costs are compared on a cumulative present value basis assuming a yearly discount rate of 10%. this model does not consider retail, service provisioning or customer equipment cost. moreover, it does not consider price declines of equipment. 4 synergy evaluation model 4.1 data and sources the subsequent assessment is based on map data provided by osm, which is an open-source, free-of-charge digital map of the world. osm data are crowd-sourced from a growing community of volunteers that has contributed to a high data density and quality in terms of completeness and accuracy comparable to geodata from commercial providers (girres & touya, 2010; haklay, 2010; neis, zielstra, zipf, & strunk, 2010; zielstra & zipf, 2010). the highest levels of data density, i.e., number of nodes and ways per area, can be found in the 5 australasian journal of information systems limbach, kuebel & zarnekow 2016, vol 20, research on rural and remote ict improving rural broadband deployment countries of central europe (openstreetmap, 2012). in the case of germany, the total osm street network even exceeds the information in commercial data sets by 27% (neis et al., 2010). the model is applied to osm data from germany that hold the locations of all german central offices, more than 11,000 municipalities and cities, 368,745 geodata points (gp) of the highway network, 870,831 gp of the railroad network, 184,031 gp of the power line network and 26,228 gp of the pipeline network. in addition, the complete osm street network is incorporated in the analysis. the use of described data has been accompanied by numerous validations in satellite pictures and other public data sources. the osm data are merged with demographic data provided by the german census to consider municipal information on the number of households and population density. table 3 provides an overview of the model data and sources. information description derived model characteristics data source feeder cable model central offices locations of all central offices in germany distance of co to communities in rural areas (openstreetmap, 2013) main roads length of main roads of a community required feeder cable trench length for fttb and fttc (openstreetmap, 2013) location centers center of a rural community approximated termination point of the feeder cable german census bureau (regionaldatenbank, 2013) alternative infrastructures coordinates of railroads, highways, electricity networks and pipeline distance from next railroad, highway, major electricity network or pipeline to community center in rural community (openstreetmap, 2013) distribution cable model demographic data households per community, population density, number of buildings per community, community center calculation of required deployment meters per household german census bureau (regionaldatenbank, 2013) residential roads aggregated length of all residential roads in a rural community approximation of required distribution cable trench length for fttb (openstreetmap, 2013) table 3 modelling data and sources 4.2 internet access model this section describes an internet access model, which assesses the required access network deployment costs per rural households, if savings through the usage of alternative backbone infrastructures are considered. the model is subdivided into a feeder cable and a distribution cable model. 6 australasian journal of information systems limbach, kuebel & zarnekow 2016, vol 20, research on rural and remote ict improving rural broadband deployment 4.2.1 feeder cable model the feeder cable model calculates the required trench length between a rural community centre and a fibre backbone interconnection point. in contrast to related literature on broadband access models, (analysys mason, 2008; lannoo et al., 2008; vidmar et al., 2010), this fibre backbone interconnection point does not necessarily have to be the incumbent’s central office. instead, the model also considers potential backbone interconnections at the next railroad, highway, major electricity network or pipeline. in the first step, airline distance calculations between all german community centres and the corresponding next alternative infrastructure are conducted on a national level to identify one national infrastructure that exhibits a particularly low average distance to rural communities. in the second step, a trench-length-optimized feeder cable is calculated considering the most promising alternative infrastructure and the incumbent’s central offices. this step is based on graph theory and takes advantage of the fact that telecommunication access networks are usually planned as a minimum spanning tree with the fibre backbone interconnection at its root (vidmar et al., 2010). a minimum spanning tree can be calculated with the wellunderstood kruskal's algorithm (kruskal, 1956). table 4 indicates how this algorithm is adjusted to provide one spanning tree that originates at the co and an additional one that originates at the alternative infrastructure. input: edges e containing all distances between the co, communities and alternative infrastructure interconnection points, set s_co containing the co, set s_infra containing all alternative infrastructure interconnection points; output: minimum spanning trees t_co and t_infra; (1) add temporary 0-weigth edges between s_co and every element of s_infra to e; (2) edges e2 = edges of kruskal_algorithm(e) with weight>0; (3) while(e2 is not empty) foreach edge (v,w) from e2 if(v or w ∈ s_co) add (v,w) to t_co; add v and w to s_co; remove (v,w) from e2; else if(v or w ∈ s_infra) add (v,w) to t_infra; add v and w to s_infra; remove (v,w) from e2; table 4 synergy evaluation algorithm the algorithm’s input requires the location of the co, alternative infrastructure interconnection points and the distances between those locations. alternative infrastructure interconnection points are located at an intersection between a public road and the alternative infrastructure. distances between the two locations v and w are calculated by using the route 7 australasian journal of information systems limbach, kuebel & zarnekow 2016, vol 20, research on rural and remote ict improving rural broadband deployment planning capability of osm. they are referred to as edges and have a weight that is equivalent to the distance between v and w. the first step of the algorithm adds 0-weight edges between all potential fibre backbone locations. this ensures that the standard kruskal's algorithm can be used to calculate a single minimum spanning tree that minimizes the length of the total network topology, in the second step. in the third step all weighted edges of the previously calculated spanning tree are iteratively sorted to one of two spanning trees that either originates at the co (t_co) or the alternative infrastructure (t_infra). the final feeder cable topology serves all communities that have previously been served by a co. if the total feeder cable length is smaller than the feeder cable length of the minimum spanning tree that serves all communities from the co, this difference will be referred to as feeder cable savings potential. 4.2.2 distribution cable model the distribution cable model considers a fttc and a fttb fibre deployment scenario, and relates the financial savings from an optimized feeder cable topology to the total costs of broadband deployment. the fttc distribution cable network model is based on the assumption that all households of a particular community are already served with a copperwire cable for purposes of wire-line telephony and internet services, which provide less than 30 mbps. to achieve customer download speeds close to 100 mbps it is assumed that street cabinets with vectoring technology are deployed within a distance of 1,000 m along the residential streets of a rural community (guenach et al., 2011). this results in a maximum distance of 500 m to the customer premises. as indicated in eq. (1) the number of street cabinets c is inferred from a community’s aggregated residential street length r. it is assumed that one additional cabinet is deployed if the aggregated street length is smaller than 1,000 m.         ≥    < = 000,1 000,1 000,11 )( rr r rc (1) accordingly, as described in section 2.1, the additional trench length d between the cabinets is calculated with the subsequent eq. (2): )1(*000,1 −= cd (2) the fttb deployment scenario is based on a street-length aware broadband deployment model proposed by lannoo et al. (2008). it is used to calculate the trench and fibre length required in addition to the fttc deployment scenario. the initial model has been developed for the city of ghent and assumes that trenches and fibre cables are required along both sides of a residential street. to adjust the model to rural distribution networks it is assumed that this percentage decreases with population density according to table 5. 8 australasian journal of information systems limbach, kuebel & zarnekow 2016, vol 20, research on rural and remote ict improving rural broadband deployment potential number of customers per km² share of trenches along both sides of a residential road 130 ≥ x > 75 90 75 ≥ x > 30 75 30 ≥ x > 15 50 <15 35 table 5 required trenches along roads following lannoo et al. (2008), trench and fibre length are calculated separately. the additional fibre cable length which is required in a fttb scenario is derived from the routing distance between the street cabinet and a customer premises of a rural community. for the defined distance between street cabinets this results in an average cable length of 250 m. furthermore, it is assumed that buildings are located in the middle of a customer premises. the analysis of german census bureau data on average rural premises sizes and cuts results in a mark-up of 17 m for connecting the building with the distribution point at the street (regionaldatenbank, 2013). 5 results this section presents the results of the synergy evaluation model in two steps. first, an overview on the data of the feeder cable analysis is provided. as illustrated in section 4.1.1, this part of the results is derived from air-line distance calculations and comparisons. the second part of the analysis builds on trench-optimized network topologies, which are based on minimum spanning tree calculations. 5.1 geographic preconditions for synergy utilization table 6 presents the percentage of communities and associated potential customers according to the proximity to the closest potential fibre backbone. nearest infrastructure to rural community center co railroads power lines highways pipeline air-line proximity (m) com. (%) pot. cust. (%) com. (%) pot. cust. (%) com. (%) pot. cust. (%) com. (%) pot. cust. (%) com. (%) pot. cust. (%) 1,000 11.4 11.7 18.6 26.7 6.7 8.5 3.9 4.2 1.0 1.3 3,000 18.5 21.6 11.3 11.4 5.4 3.9 3.9 2.8 0.7 0.8 5,000 11.0 4.6 3.7 1.4 1.1 0.4 0.7 0.2 0.3 0.1 7,000 1.5 0.4 0.3 0.1 0.0 0.0 0.0 0.0 0.0 0.0 >7,000 0.1 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 total 42.4 38.3 33.9 39.5 13.2 12.8 8.4 7.3 2.1 2.2 table 6 geographic proximity of rural communities to potential fibre backbones the table indicates that 42.4% of the rural communities (com.) in germany are situated closer to the central office than to an alternative infrastructure. of those communities, 11.4% are located within an airline proximity of 1,000 m or below. another 18.5% of the communities are located within a proximity above 1,000 and below or equal to 3,000 m. though at least 33.9% of the rural communities are situated closer to railroads than to a central office those communities host at least 39.5% of the potential customers (pot. cust.). this value is slightly higher than the number of potential customers in close proximity to the central office. for 9 australasian journal of information systems limbach, kuebel & zarnekow 2016, vol 20, research on rural and remote ict improving rural broadband deployment another 23.7% of the rural communities, other alternative infrastructures are closer to the community centre than to cos or railroads. a single community within a close proximity to an alternative infrastructure will usually not justify a broadband deployment project. consequently, it is of importance to assess the clustering degree of rural communities with a close proximity to an alternative infrastructure. fig. 2 addresses this aspect on a federal, state and county level. it depicts the percentage of communities with a close proximity to an alternative infrastructure within a county for three synergy density categories. counties that exhibit less than 33% rural communities that are located closer to an alternative infrastructure than to the co are assigned to a low synergy density category. similarly, counties which exhibit more than 66% rural communities that are located closer to an alternative infrastructure than to the co are assigned to a high synergy density category. the remaining counties are assigned to an intermediate category. fig. 2 density of rural communities with close proximity to alternative infrastructures fig. 2 shows that the density of rural communities with a closer proximity to alternative infrastructures as opposed to the co differs by county and federal state. the majority of german counties exhibit a medium synergy density. for 2 of 13 states the number of counties with a high synergy density equals the number of counties in the intermediate density category. only a small share of the federal states exhibits a low synergy density. 5.2 characteristics of the optimized feeder cable topology based on the findings of table 6, the subsequent analyses focus on assessing synergies that stem from the railroad network. for this purpose every rural community of a county is allocated to a minimum spanning tree that either starts at the central office or at a railroad. table 7 provides descriptive statistical figures of the resulting spanning trees. railroad spanning tree co spanning tree avg # of communities 2.25 2.07 sd # of communities 1.88 1.94 avg # of pot. customers 1,518 1,281 sd # of customers 1,313 1,573 table 7 minimum spanning tree analysis for optimized feeder cable topology 0% 10% 20% 30% 40% 50% 60% 70% 80% sn st bb th nw mv sl ni rp by bw sh he co un tie s i n fe de ra l s ta te federal state low synergy density medium synergy density high synergy density 10 australasian journal of information systems limbach, kuebel & zarnekow 2016, vol 20, research on rural and remote ict improving rural broadband deployment table 7 shows that an optimized feeder cable topology results in spanning trees that connect 2 communities per railroad and per co spanning tree on average. the average number of customers reachable from a potential railroad spanning tree is slightly higher than in the co spanning tree. the rather high standard deviation (sd) for the number of communities and customers that are served in a minimum spanning tree indicates that a variety of rural spanning tree topologies exist. fig. 3 optimized rural feeder cable topology in thuringia (germany) fig. 3 depicts an optimized feeder cable topology in a typical rural area in the german federal state thuringia (th). communities such as ballstädt, bufleben or eschenbergen exhibit a street routing distance of more than 6,000 m to the next co. in contrast, potential feeder cable lengths to the next railroad fibre connection are up to six times shorter. thus, in an optimized topology they are associated with a railroad spanning tree. communities such as nottleben or zimmern exhibit a long routing distance to railroads and are assigned to a co spanning tree. using its widely available fibre capacities along railroads, the german company arcor has deployed vdsl with download speeds of up to 50 mbps in ballstädt and announced a national roll-out (briegleb, 2008). after the acquisition of arcor by vodafone these capacities have primarily been used for the deployment of the wireless technology lte which requires less capex per connected customer than fttc and currently provides average rural down-load speeds of approximately 10 mbps (pages & pe, 2013). 5.3 cost advantages of alternative infrastructures in this section, cost advantages of alternative infrastructures are explored in two steps. first, savings of an optimized feeder cable topology are put in perspective to the total costs of fttc and fttb deployment. thereafter, a sensitivity analysis explores the effects of parameter variations on possible cost reductions. fig. 4 depicts the fttc and fttb investment per customer, which is required for connecting 50%, 75%, 90%, 95% or 100% of all households in a county with optimized feeder cable 11 australasian journal of information systems limbach, kuebel & zarnekow 2016, vol 20, research on rural and remote ict improving rural broadband deployment topologies. moreover, an alternative connection via the co is depicted for both fibre deployment variants. for determining the shares of connected households, communities have been ordered by a decreasing number of households. that is, communities with the lowest number of households within a county will only be connected in the 100% deployment scenario. fig. 4 financial impact of synergy utilization with railroads results indicate that in the base case absolute savings per customer vary between 205 € in the 100% deployment case and 128 € in the 50% deployment case. this results in a maximum relative savings potential of 13% for the fttc case and 7% for the fttb case. however, as noted in section 3.1 variations in the monthly metro-aggregation, backbone, co-location costs and general spare duct availability can impact the magnitude of these savings. thus, a sensitivity analysis is exploring these effects in table 8. in this analysis, parameters for the co deployment are kept constant while parameters of the alternative infrastructure topology are subject to parameter variations. fttc fttb parameter variati on (%) optim ized (€) difference to co deployment (%) optimiz ed (€) difference to co deployment (%) monthly metro-aggregation, backbone and co-location costs + 50 1,213 1 2,480 1 50 946 23 2,213 11 spare duct availability in feeder cable topology 50 1,088 11 2,356 5 100 1,097 11 2,364 5 table 8 sensitivity analysis for deployment to 75% of all households results show that a variation of the monthly metro-aggregation, backbone and co-location costs per customer have a strong impact on the relative cost advantages. if these costs are approximately 50% higher than the incumbent’s offer, the savings potential of alternative infra-structure use is close to zero. in contrast, 50% lower monthly costs can increase the savings potential to 23% in the fttc case and 11% in the fttb case. a lower share of spare 2,354 € 2,491 € 2,643 € 2,744 € 3,078 € 2,226 € 2,347 € 2,472 € 2,565 € 2,874 € 1,129 € 1,225 € 1,334 € 1,401 € 1,609 € 999 € 1,080 € 1,163 € 1,222 € 1,404 € 800 € 1,300 € 1,800 € 2,300 € 2,800 € 50% 60% 70% 80% 90% 100% in ve st m en t p er c us to m er share of connected households in county fttb co deployment fttb optimized fttc co deployment fttc optimized 12 australasian journal of information systems limbach, kuebel & zarnekow 2016, vol 20, research on rural and remote ict improving rural broadband deployment ducts in the alternative infrastructure feeder cable topology hardly reduces relative cost savings. this is due to three reasons. first, as noted in section 3.2, general spare ducts availability in rural areas is usually much lower than in more densely populated areas. second, an increase of feeder cable deployment costs is allocated to all households of a connected community. finally, spare duct availability in the distribution cable segment is not affected by an optimized feeder cable topology. 5.4 comparison with other studies as of today, few national broadband deployment cost models have explored synergy optimized feeder cable topologies. moreover, most existing cost models are not based on a street-length aware modelling approach that can account for the variety of rural cable topologies (cf. table 7). nevertheless, fig. 4 indicates that the investment costs per potential customer without synergy utilization align well with existing cost modelling studies. this study has identified fttb investment costs of 3,078 € per potential customer in rural areas. based on averaged real world cable length from several european operators, schneir and xiong (2013) calculate costs of 3,250 € per rural home connected for a single operator with a market share of 70%. for the fttc deployment scenario this model indicates investment costs without synergy utilization of 1,609 € per potential customer. analysys mason (2008) takes into account the costs of customer premises equipment and identified costs of 1,690 € for the most remote geotype at a market share of 70%. 6 political and regulatory implications the presented results indicate that cost savings can be achieved if alternative national infrastructures are considered as source for broadband networks in rural areas. as a consequence this paper contributes to current efforts to maximize synergy utilization across networks and reduce the civil engineering costs of broadband deployment (ec, 2013c, nbnco, 2015). however, the efficient use of those infrastructures depends on appropriate political and regulatory incentive structures. section 5.1 shows that geographic preconditions for synergy utilization can differ largely between federal states and counties. thus, political and regulatory efforts directed at improving efficiency gains across national networks should be focused on geographic areas with favourable preconditions for synergy utilization. under disadvantageous geographic preconditions for synergy utilization, incumbent cos are more likely to be an essential facility for co-location. if competition in the local-loop usage is desired, the regulator should ensure the availability of co-location options as the incumbent is migrating to fibre access networks and closing down facilities. in countries like australia the physical network is provided by a national company. in this case regulation needs to ensure through periodic wholesale price rebalancing and regulatory measures in the utility sector that the national company maximizes synergy utilization across physical networks. section 5.2 shows that an optimized feeder cable topology is connecting up to five communities with every fibre backbone connection. thus, local authority practice of starting separate tenders and market explorations for neighbouring communities results in artificial demand fragmentation (sawhney, 1992; bbb, 2014). to improve the use of alternative infrastructures, tenders should be started and coordinated at a county or federal state level. at this level, companies should be allowed to bid for lots within the target area. the presented cost analysis in section 5.3 highlights the importance of variations in the monthly metro-aggregation, backbone and co-location costs for the efficient use of alternative infrastructures. especially if owners of alternative infrastructures have been obligated to offer fibre or spare ducts capacities, these costs could be set prohibitively high. if necessary, national regulatory measures should ensure that the cost of using alternative infrastructure capacities does not exceed the incumbent’s offer by more than 50%. 13 australasian journal of information systems limbach, kuebel & zarnekow 2016, vol 20, research on rural and remote ict improving rural broadband deployment 7 conclusion to address the economic challenges associated with rural broadband deployment, public aid and cost reduction measures will be required. this paper has used a street-aware synergy evaluation model to assess the financial magnitude of cost savings that can be achieved if national alternative infrastructures are considered as source for broadband networks in rural areas. as such, it proposes an is pilot instrument to guide political and regulatory decisions. analyses suggest that an optimized feeder cable topology can reduce the costs of rural broadband deployment if alternative infrastructure fibre backbone, metro-aggregation and colocation costs do not exceed the incumbent’s offer by more than 50%. to foster the deployment of cost optimized network topologies, public authorities should aggregate customer demand in large tender areas which cover multiple rural communities. moreover, regulators need to be aware of regional differences in the preconditions for synergy utilization and should consider them as cos are closed down in the context of the migration to fibre networks. this paper finds that railroad networks exhibit the largest national synergy potential in the case of germany. because densities of railroad networks differ across countries, synergy potentials may be alike. thus, it would be worthwhile to apply the proposed model to other markets. although corresponding osm data are available for 69 countries in the world (openstreetmap, 2012) and its quality has been certified within several case studies, triangulation with other data sources should complement data usage. further research may build on this contribution and explore related aspects. first and foremost, research should be devoted to the amount of savings that can be achieved through the reduction of cos, as the incumbent is migrating infrastructure to fibre access networks as well as geographical aspects of regulation or co-investment strategies. references analysys mason. (2008). the costs of deploying fibre-based next-generation broadband infrastructure. final report for the broadband stakeholder group. ref: 12726-371. available at: http://www.analysysmason.com/pagefiles/5766/analysys-mason-finalreport-for-bsg-(sept2008).pdf [accessed 8 september 2008]. bbb. (2014). breitband-ausschreibungen. available at: https://www.breitbandausschreibungen.de/publicoverview [accessed 12 september 2014]. briegleb, v. (2008) arcor testet vdsl in thüringer gemeinde. available at: http://www.heise.de/netze/meldung/arcor-testet-vdsl-in-thueringer-gemeinde217119.html [accessed 20 april 2015]. bundesnetzagentur (2013). genehmigung von entgelten für den zugang zur teilnehmeranschlussleitung (monatliche überlassungsentgelte). bk 3c-13/002, bonn: beschlusskammer 3 der bundesnetzagentur. available at: http://beschlussdatenbank.bundesnetzagentur.de/index.php?lr=view_bk_overview&g etfile=1&file=5929 [accessed 20 january 2015]. cave, m. (2014). the ladder of investment in europe, in retrospect and prospect. telecommunications policy, in press, doi: http://dx.doi.org/10.1016/j.telpol.2014.04.012. chatzi s., lazaro, j. a., prat, j., & tomkos, i. (2013). a techno-economic study on the outside plant cost of current and next-generation fiber-to-the-x deployments. fiber and integrated optics, 32(1), 12-27. christofides, n. (1975). graph theory: an algorithmic approach. london: academic press. dippon, c. m., & train, k. e. (2000). the cost of the local telecommunication network: a comparison of minimum spanning trees and the hai model. telecommunications policy, 24(3), 253-262. 14 australasian journal of information systems limbach, kuebel & zarnekow 2016, vol 20, research on rural and remote ict improving rural broadband deployment european commission (ec). (2010a). a digital agenda for europe. brussels: european commission. available at: http://eurlex.europa.eu/lexuriserv/lexuriserv.do?uri=com:2010:0245:fin:en:pdf [accessed 6 march 2015]. european commission (ec). (2010b). state aid n 407/2009 – spain optical fibre catalonia (xarxa oberta). brussels: european commission. available at: http://ec.europa.eu/competition/state_aid/cases/232264/232264_1132944_78_1.pdf [accessed 6 march 2015]. european commission (ec). (2012a). state aid sa.34732 (2012/n) – italy –bulgas – fibersar – nga sardegna. brussels: european commission. available at: http://ec.europa.eu/competition/state_aid/cases/244582/244582_1390156_60_2.pd f [accessed 30 march 2015]. european commission (ec). (2012b). support for the preparation of an impact assessment to accompany an eu initiative on reducing the cost of high‐speed broadband infrastructure deployment. luxembourg: the publications office of the european union. available at: http://ec.europa.eu/information_society/newsroom/cf/dae/document.cfm?doc_id=18 77 [accessed 2 february 2015]. european commission (ec). (2013a). state aid sa.33656 (2012/nn) – ireland – next generation (backhaul) network (ngn) alongside a gas pipeline in galway and mayo. brussels: european commission. available at: http://ec.europa.eu/competition/state_aid/cases/243213/243213_1504550_221_2.pd f [accessed 12 march 2015]. european commission (ec). (2013b). eu guidelines for the application of state aid rules in relation to the rapid deployment of broadband networks. available at: http://eurlex.europa.eu/lexuriserv/lexuriserv.do?uri=oj:c:2013:025:0001:0026:en:pdf [accessed 23 january 2015]. european commission (ec). (2013c). regulation of the european parliament and of the council on measures to reduce the cost of deploying high-speed electronic communications networks (2013/0080). available at: http://eurlex.europa.eu/lexuriserv/lexuriserv.do?uri=com:2013:0147:fin:en:pdf [accessed 4 march 2015]. falch, m., & henten, a. (2010). public private partnerships as a tool for stimulating investments in broadband. telecommunications policy, 34(9), 496-504. federal communications office (fcc). (2010). connecting america: the national broadband plan. 125-136. washington, dc: u.s. government printing office. available at: http://www.umic.pt/images/stories/publicacoes5/national-broadband-plan.pdf [accessed 5 june 2014]. ftth council europe. (2013). annual report april 2012 april 2013. available at: http://www.ftthcouncil.eu/documents/publications/ftthcouncil_ar2012_2013_fin al.pdf [accessed 9 april 2013]. girres, j. f., & touya, g. (2010). quality assessment of the french openstreetmap dataset. transactions in gis, 14(4), 435-459. given, j. (2010). take your partners: public private interplay in australian and new zealand plans for next generation broadband. telecommunications policy, 36(2), 96-112. grubesic, t. h. (2008). spatial data constraints: implications for measuring broadband. telecommunications policy, 34, 540-549. guenach, m., meas, j., timmers, m., lamparter, o., bischoff, j. c., & peeters, m. (2011). vectoring in dsl systems: practices and challenges. in: proceedings of the 2011 global telecommunications conference. houston, texas. available at: 15 australasian journal of information systems limbach, kuebel & zarnekow 2016, vol 20, research on rural and remote ict improving rural broadband deployment http://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=06133821 [accessed 7 april 2014]. haklay, m. (2010). how good is volunteered geographical information? a comparative study of openstreetmap and ordnance survey datasets. environment and planning. b: planning & design, 37(4), 682-703. hoernig, s., jay, s., neumann, k.-h., peitz, m., plückebaum, t., & vogelsang, i. (2012). the impact of different fibre access network technologies on cost, competition and welfare. telecommunications policy, 36(2), 96-112. jay, s., neumann, k., & plückebaum, t. (2014). comparing ftth access networks based on p2p and pmp fibre topologies. telecommunications policy, 38(5-6), 415-425. kipouridis, o., machuca, c.m., autenrieth, a., & grobe, k. (2012). street-aware infrastructure planning tool for next generation optical access networks. in: proceedings of the 16th international conference on ondm, colchester, pp 1-6. kruskal, j. b. (1956). on the shortest spanning subtree of a graph and the traveling salesman problem. in: proceedings of the american mathematical society, 7(1), 48-50. lannoo, b., casier, k., van ooteghem, j., wouters, b., verbrugge, s., colle, d., picavet, m., & demeester, p. (2008). economic benefits of a community driven fiber to the home rollout. in: proceedings of the 5th international conference on broadband communications, networks and systems. london, england. available at: http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=4769123 [accessed 7 april 2014]. nbnco, nbn co limited (2015). corporate plan 2016, nbn – australia’s new broadband network. available at: http://www.nbnco.com.au/content/dam/nbnco2/documents/nbn-corporate-plan2016.pdf [accessed 10.12.2015]. neis, p., zielstra, d., zipf, a., & struck, a. (2010). empirische untersuchungen zur datenqualität von openstreetmap erfahrungen aus zwei jahren betrieb mehrerer osmonline-dienste. in: strobl, j., blaschke, t. & griesebner, g. (eds.). angewandte geoinformatik 2010: beiträge zum 22. agit-symposium salzburg. berlin: wichmann, 420-425. openstreetmap. (2012). openstreetmap density of data by region. available at: http://www.odbl.de [accessed 14 january 2013]. openstreetmap. (2013). openstreetmap. available at: http://www.openstreetmap.org [accessed 12 january 2013]. organisation for economic co-operation and development (oecd). (2009). working party on communication infrastructures and services policy – indicators of broadband coverage. available at: http://www.oecd.org/internet/broadband/44381795.pdf [accessed 14 november 2013]. pages m. & pe d. (2013). will lte kill dsl? the delta perspective. regionaldatenbank. (2013) regionaldatenbank. available at: https://www.regionalstatistik.de/genesis/online/logon [accessed 11 april 2013]. robey d., sahay s. (1996). transforming work through information technology: a comparative case study of geographic information systems in county government. information systems research, 7(1), 93-110. rokkas, t., katsianis, d., & varoutas, d. (2010). techno-economic evaluation of fttc/vdsl and ftth roll-out scenarios: discounted cash flows and real option valuation. journal of optical communications and networking, 2(9), 760-772. 16 australasian journal of information systems limbach, kuebel & zarnekow 2016, vol 20, research on rural and remote ict improving rural broadband deployment ruhle, e. o., brusic, i., kittl, j., & ehrler, m. (2011). next generation access (nga) supply side interventions an international comparison. telecommunications policy, 35(9), 794-803. schäfer, r. g. & schöbel, a. (2005). stand der backbone-infrastruktur in deutschland – eine marktund wettbewerbsanalyse. wik diskussionsbeiträge, nr. 265. sawhney, h. (1992). demand aggregation strategies for rural telephony. telecommunications policy, 16(2), 167-178. schneir, j. r., & xiong, y. (2013). economic implications of a co-investment scheme for ftth/pon architectures. telecommunications policy, 37(10), 849-860. segarra, j., sales, v., & prat, j. (2012). planning and designing ftth networks: elements, tools and practical issues. in: proceedings of the 14th icton, coventry, 1-6. tahon, m., van ooteghem, j., casier, k., verbrugge, s., colle, d., pickavet, m., & demeester, p. (2014). improving the ftth business case – a joint telco-utility network rollout model. telecommunications policy, 38(5-6), 426–437. vidmar, l., peternel, b., štular, m., & kos, a. (2010). broadband access network investment optimization in rural areas. in: proceedings of the 15th ieee mediterranean electrotechnical conference. valletta, malta. available at: http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=5476228 [accessed 14 september 2014]. zielstra, d., & zipf, a. (2010). quantitative studies on the data quality of openstreetmap in germany. in: proceedings of the 6th international conference on geographic information science. zurich, switzerland. available at: http://www.giscience2010.org/pdfs/paper_187.pdf [accessed 10 february 2015]. copyright: © 2016 limbach, kuebel & zarnekow. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. 17 http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 theoretical foundations and related work 2.1 characteristics of rural network topologies 2.2 the role of public authorities in rural broadband deployment 3 deployment scenarios and cost modelling 3.1 deployment scenarios 3.2 cost modelling 4 synergy evaluation model 4.1 data and sources 4.2 internet access model 4.2.1 feeder cable model 4.2.2 distribution cable model 5 results 5.1 geographic preconditions for synergy utilization 5.2 characteristics of the optimized feeder cable topology 5.3 cost advantages of alternative infrastructures 5.4 comparison with other studies 6 political and regulatory implications 7 conclusion microsoft word ajis_18(3)_v10.docx australasian journal of information systems volume 18 number 3 2014 119 manager satisfaction in using the enterprise resource planning (erp) system and the managerial performance weli faculty of economics atma jaya catholic university of indonesia, jakarta weli.imbiri@gmail.com abstract this study is aimed to test the manager’s satisfaction of the managers using the erp and to test the influence of job satisfaction towards the performance in the decision process by the manager. the measurement of the satisfaction towards the erp system will be conducted using the end user’s satisfaction computing model (eucs), using the content, accuracy, formal ease of use, and timelines attributes (doll and torkzadeh, 1988). whereas, the performance is measured using the instrument which was developed by nicolaou (2000) consisting of 2 (two) question items about the reporting and continuing control, 7 (seven) items of questions adopted from the study results of spathis and constantinides (2004). the data were collected through a survey using questionnaires distributed to companies that have used the erp system. the data analysis used the partial least square method with smartpls program. the results show that the concept of eucs has a conformation and there is a significant effect between the erp system user satisfaction with the manager’s performance, especially in the decision making process. this result contributes to the concept of eucs and supports previous research related to end user satisfaction. keywords: enterprise resource planning, erp, information system, end user computing satisfaction, eucs, manager performance. introduction nowadays the use of the enterprise resource planning (erp) system has experienced a rapid growth, also in indonesia. sap indonesia as one of erp vendor has succeeded for the iv 2009’s quarter, to have a significant growth rate of 30%, compared with 2008, that is, 47 companies have become new customers. other information that has been reported in kompas of 11 july 2009 is, that large companies in indonesia have used the erp system with the sap vendors having reached more than 250 sap companies and according the sap indonesia report, until 2012, the sap users have reached more than 800 companies. the growth of the erp system users is not free from the wishes of the company to integrate all business functions so that streamlining data processing will occur via a centralized database. besides, the hope, being the basis of the use of the erp is to create an efficient and an effective business organization process, increasing the ability in making managerial decisions, and monitoring company’s performance. similarly, according to sajadi et al. (2008) that the use of the erp systems may affect the effectiveness of information systems through improved managers decision-making processes, internal controls, and improving the quality of financial reporting and facilitates transaction processing companies. furthermore, an effective information system is expected to improve the quality of decisions, and improving the company’s performance. previous studies have reported both success stories and failures in the process of implementing erp systems. several reasons were found from the literature that many erp projects fail because of complexity of the implementation processes, the lack of conformity between the management project and the erp system and the business organization’s process, the lack of users skilled, the coordination australasian journal of information systems volume 18 number 3 2014 120 issues, the lack in planning and project management, as well as consultant’s quality. however, previous study in using system has shown the importance of the system user’s satisfaction. the user satisfaction was determined by the user system satisfaction towards factors of the success of the system implementation. the interaction between user and implemented system will result in the perception of the user alone. the user’s perception is directed towards the satisfaction or on the contrary, towards the no satisfaction of the system. considering the aim of the use of the erp is create the integrated flow of the information, it is expected that the information produced by the system will be more qualified. the information quality becomes an important weapon for decision makers in solving business problems. therefore, this study will examine the satisfaction of manager of the erp system user and whether the satisfaction towards the use of the erp system has an influence towards the company’s performance, especially in the decision making process by the manager. the satisfaction measurement towards the erp system is conducted through the end user’s computing satisfaction (eucs) model, using the content, accuracy, format, ease of use, and timelines attributes (doll and torkzadeh, 1988). the eucs model to measure the satisfaction of the user of the erp has already been conducted by hees and hightower (2002), rose (2006), deng et al. (2008), and mitakos et al. (2010). in general, the eucs analytical report for the erp system shows that the eucs concept can also be used for the erp system, as well as to other systems at the previous studies. although the study using the eucs have already been conducted many times before, however, all those studies have only confirmed the measurement model for the system user’s satisfaction and have not yet seen the system user’s influence towards the organization, especially in the decision making process by the management. therefore, this study will analyze the satisfaction measurement of the erp system user and to test this satisfaction influence towards the organization, especially, at the decision making process by the manager. therefore, the aim of this study is as follows: 1. conducting an analysis on the content, accuracy, format, ease of use and timeliness as the makers of the satisfaction factors for the erp users. 2. to analyze the influence of the manager’s satisfaction of the use of the erp system towards the manager performance in the decision making. whereas the theoretical framework, regarding the relationship between the variables, is reflected in figure 1. figure 1 research model australasian journal of information systems volume 18 number 3 2014 121 literature review and the hypothetical formulation the satisfaction measurement of the information system user. the model to measure the system user’s performance has been developed long time ago by the researchers in the field of the information system. one of the models that are frequently used is the end user competing satisfaction (eucs). the eucs was developed in 1988 by doll and torkzadeh to measure the computer’s end user satisfaction. doll and torkzadeh have found five factors (content, accuracy, format, ease of use and timeliness) strongly measured the satisfaction of the end-user, so that this instrument became the right instrument to measure the end-user’s satisfaction. another model to measure the system user’s satisfaction is the successful information system, which has developed since the beginning of 1990 (delone, and mclean, 1992). this model has further developed, parallel with the many studies conducted in testing the success of the system (delone, 2003). the satisfaction model of the system user as developed by delone (2003) values that the satisfaction in using the information by relating the satisfaction of the system user with the usage and the organization’s performance. the higher the satisfaction rate towards the new implemented system, the higher the intention to use it and at the end will make the individual’s and organization’s performance better and more efficient. user satisfaction can be defined as the level of user confidence over an information system use that is available to meet their needs for information. when users use the system they will evaluate the various aspects of the use based on their experience compared to the expectations over the system. although the study about the satisfaction of the end user has already been developed since the last decade, however this study has still drawn the researcher’s as well as practitioner’s interest (khalifa and liu, 2003). since the delone and mclean (1992, 2003) model was developed, many researchers have continued investigating the information system, extra construct, and the different points of view. the test of the user’s satisfaction at the of the erp system has also been carried out, among others by hees and hightower (2002), roses (2006), deng et al. (2008) and mitakos et al (2010). hees and hightower (2002) have proposed a framework to test the erp system users using the equity theory. the aim of their study is to look into the attitudes of the users to complete the framework of the satisfaction of the system user model. roses (2006) has tested the antecedent factor of the erp system users satisfaction, using the information system’s success and the eucs model. the result of the study has confirmed that the application of the eucs model can be used in the application of the erp system. deng et al. (2008) have analyzed the multi-group variance towards the eucs instrument for the use of the enterprise system in five countries (us, western europe, saudi arabia, india and taiwan). the analysis report has indicated that the eucs instrument has provided a measurement result that is equal among the five countries. hence, conclusion can be derived that the eucs instrument can be globally used. mitakos et al. (2010) has connected the success of the implementation and operation of erp with the user. they have tested the demographic of the individuals (the department in which the user works, the user position in the organization, the education level, age, computer experience, gender) who influence the satisfaction of the erp system user by adding the self-efficacy factor. however, not a single individual demographic factor proposed, have proofed that it can satisfy the erp system user. based on previous studies , especially at the erp system and the accounting information, this study will test again the erp system user’s satisfaction model using the end user computing satisfaction (eucs) as proposed by doll & torkzadeh (1988). the result of above mentioned studies indicates that the eucs instrument was a measurement in which the validity has been tested and has often been used as an instrument to measure the effectiveness of the system in the study of the accounting information system. another reason being the base of the use of the eucs measurement is that the erp system is designed to solve the information disintegration by consolidating all business operations to be an australasian journal of information systems volume 18 number 3 2014 122 integrated system. the integration of the system is hoped to produce better information as seen from the perspective of the content, accuracy, format, ease of use and timeliness. therefore, the hypothesis formula in this study is as follows: h1: there is a direct influence between content from the output of the erp system and the manager’s satisfaction using the erp system. h2: there is a direct influence between accuracy of the output of the erp system and the manager’s satisfaction using the erp system. h3: there is a direct influence between the form of the output system of the erp and the manager’s satisfaction using the erp system. h4: there is a direct influence between the ease of use of the erp system and the manager’s decision using the erp system. h5: there is a direct influence between the timeliness of the output of the erp system the manager’s satisfaction. the enterprise resource planning (erp) system user’s satisfaction and the managerial performance. the user’s satisfaction at the previous studies was identical with the valuation of the effectiveness of an information system (delone and mclean, 1992). the system user’s satisfaction in interacting with the system can be seen from their experience with the information output as produced by the system. the produced physical output by the system can be valued through the content, accuracy, format, ease of use and timeliness. further, is the user’s satisfaction towards the output quality in accordance with what is in need, whether the output system can increase the work productivity, increase the work performance, and increase the control towards the decision, related with the resulting information. therefore, it is hoped that the information result can make the decision making process more effective, the information reach easier to interpret, and better understood and the distribution of information to the sections or to the related functions can be upgraded (ugboma, 2004). further, according to kim (1988), an effective system depends on the user’s perception regarding the information quality result, whereas the quality of the information is dependent on the reliability, the reporting form, the timeliness, and relevance for the decision making. further, nicolaou (2000) has defined the perception about the effectiveness of a system as seen from the perspective of the decision maker regarding the information output, which is originated from the transaction process which is adapted to their needs in the coordination and the management controlling activities. according to sajady et al. (2008), the information system effectiveness is also dependent on the decision maker’s perception towards the usefulness of the information, resulted from the system, to fulfill the information needs in the operational process, the managerial report, the budget, and the organizational control. the result of sajadi et al. (2008) study has indicated that the implementation of the information system will result in betterment of the decision making process by the manager, the internal control and the financial reporting quality, and the facility of the company’s transaction process. further, in this study the evaluation of the managerial performance influence towards the information system user’s satisfaction is focused towards the information usefulness for the users. the measurement is derived from the value of the user’s satisfaction towards the information quality, consisting of the form, content, and the presentation presented in the erp system. based on above mentioned observation, this study will also value the influence of the manager satisfaction in using the erp system, as measured with the eucs towards the manager’s performance, especially in the process of the decision making. this can be reached because the erp system has the characteristics of integrating all business functions. with the existence of the information system we australasian journal of information systems volume 18 number 3 2014 123 can hope that that it will increase the information to those using it as well as that it will also improve the dispatch of the information to those using it besides, it will also improve the consistence of the data and the information resulted from the system. hence, the measurement of the user’s satisfaction will become relevant in value the managerial performance effectiveness. therefore, the measurement of the user’s satisfaction will become more relevant to value the managerial performance effectiveness. further, the hope, as the basis for the performance increase is the believe that the information technology equipment can catch information and spread it more rapidly to the whole organization to help the individual to do work and to make better decision (arnold, 2006). since one of the goals for using the erp system is the data integration, the system will provide data that can be used collectively by all individuals in the organization. besides, it is expected that the system will create an up to date data (nah et al., 2001). therefore, there are 6 hypothesis formulations, as follows: h6: there is direct influence between the manager’s satisfaction using the erp system and the manager’s performance, especially in the decision making activities. research method the research population is the erp system users at the manager level from all companies in jakarta that have already used the erp system. however, because the list of companies that have been using the erp system is not available, so that it is not possible to have a sampling framework and therefore the sampling can not be conducted randomly. the collection of information about the company is done by searching the search engine (google). after the information about the company is collected, the next stage is to ask the willingness of the company to collaborate in the survey and who can be contacted to fill the questioners. the data collection process was done during 4 (four) month, that is, from beginning october 2010 until february 2011. the data collection was conducted by sending the questioners via the post office service, e-mail and visiting the related company directly. until the time limit, the rate of return via the post office, and e-mail have only reached 12 copies, while directly sent only 38 copies. this is far from the expectation. therefore, the researcher has extended the time limit for collecting the data until april 2011. the last result of the collection process, only 18 copies were collected via the post office and email, and 53 copies were sent directly, hence, the total was 71 copies. method sent received response rate via post 162 9 5,56% directly visit 54 53 98,15% via e-mail 52 9 17,31% total 268 71 26,49% table 1. response rate the validity instrument although the eucs instrument is much used in previous studies, and the validity has been tested, because the instrument is translated into the bahasa indonesia, hence it is important to test the instrument in the bahasa indonesia. therefore, before the survey in the field took place, the researcher has asked the help of some experts to read and to provide input regarding the instrument that have been developed. this pre-test is useful to understand whether the questioner is applicable, can be read, and be understood, as seen from the perspective of the format, the time needed to fill, as well as from the point of the questioner, whether is has already included the expected goal. questioners that have already been developed were sent to three experts who understand the product and the application of the erp australasian journal of information systems volume 18 number 3 2014 124 system of the company. from the review of these experts a small revision has been done relating to the use of the sentences. apart from the importance of the validation and reliability of the eucs instrument, this study will examine all five variables of the eucs instrument with the factor analysis, using the spss 16 program, where the result will be presented at the next section. operational variable and measurement the variable that forms the erp system user’s satisfaction is adopted from the end user computing satisfaction (eucs) concept as proposed by doll & torkzadeh (1988), in which the erp system user’s satisfaction is influenced by 5 variables, that are content, accuracy, format, ease of use and timelines. the measurement of the the five constructs is using the seven points likert scale, from not very agreed up to very agreed. the questions items of every construct are presented in the following table 2. code explanation content used to measure the erp user’s perception regarding how far the erp output system is on hand in accordance with their needs, consisting of 4 (four) questions items. c 1 the erp system provides the right information as needed. c2 the erp system content fulfill the needs c3 the erp system presents the reports exactly as requested c4 the erp system provides the needed information accuracy is used to measure the erp system user’ perception regarding the accuracy of the available erp system output consisting of 2 question items a1 the erp system used is accurate a2 the accuracy of the erp output system as presented is satisfactory format used to measure the erp user’s perception regarding the format of the output of the erp system on hand consisting of 2 question items f1 the erp output system is shown in an usefull format f2 the presented information by the erp system is very clear. ease of use is used to measure the erp user’s perception regarding the easiness of the use of the erp system, consisting of 2 questions items e1 the erp system is user friendly f2 the erp system is very easy to use timelines is used to measure the erp user’s perception regarding the timelines to get information from the erp system output consisting of 2 (two) questions items. t2 the erp system provides the best information t1 the needed information can be timely used source: doll & torkzadeh (1988) table 2. eucs construct managerial performance the managerial performance is used to measure the manager’s perception who uses the erp system regarding the manager’s task in making decisions. previous studies nicolaou (2000) has developed the measurement for test the effectiveness system consisting of 2 (two) question items about reporting and extended controlling. in accordance with the objectives of this study the measurement of managerial performance will adopt nicolaou (2000) instruments. australasian journal of information systems volume 18 number 3 2014 125 besides, it is also measured through 7 (seven) question items, adopted from the spathis and constantinides (2004) study result. the question items for managerial performance is presented in table 3: km1 the use of the erp system makes that the controlling report, such as the daily or the weekly report occasionally available, systematic, and in order. km2 in general, the erp system provides important information, which is useful for the continuing control of the decisions and activities km3 use of the erp system has increase the financial reporting quality km4 use of the erp system has increase the decision making process km5 use of the erp system has made the company’s information system more accurate and can be easily assessed km6 use of the erp system has decreased the information processing cost km7 use of the erp system has increased the information/visibility flow km8 use of the erp system has increased the flexibility km9 use of the erp system has increased the company’s integration source: nicolaou (2000), spathis and constantinides (2004) table 3. managerial performance construct result respondent profile the process of the data shows that most of the correspondents who have answered the questioners have managerial positions in the information technology and in accounting (see table 4). they have been in these positions for about 3 years (see table 5), and have been working in miscellaneous industries (automotive, cable, electronic, see table 6). whereas the majority of the erp products used, were originated from the sap vendors (see table 8). australasian journal of information systems volume 18 number 3 2014 126 position total percentage branch manager 1 1.4% corporate secretary 1 1.4% director 5 7% finance director 2 2.8% accounting manager 18 25.4% group manager 1 1.4% hrd manager 6 8.5% it manager 19 26.8% finance manager 2 2.8% marketing manager 1 1.4 % project manager 3 4.2% relation manager 3 4.2% sales manager 2 2.8% sipil manager 1 1.4% treasury manager 2 2.8% senior manager 2 2.8% supervisor 2 2.8% total 71 100% table 4. respondent profile position tenure total percentage < 3 year 34 47.9% 3 5 year 14 19.7% 5 10 year 13 18.3% 10 15 year 6 8.5% > 15 year 4 5.6% total 71 100% table 5. position tenure australasian journal of information systems volume 18 number 3 2014 127 industry total percentage agriculture 3 4% mining 4 6% basic industry and chemicals 7 10% miscellaneous industry (automotive, cable, electronic) 15 21% consumer goods industry 8 11% property, real estate and building construction 9 13% infrastructure, utilities & transportation 8 11% finance 8 11% trade, service&investment 9 13% total 71 100% table 6. industries non-response bias test as presented at table 7, the result of the non-response bias test between the group of 50 respondents that have responded earlier, (october 2010 – february 2011) and the group who have responded at the end of 21 respondents (march 2011 – april 2011), the p-value > 0,05. therefore, the conclusion is that there are no differences between the groups for the testing the group that has responded earlier and the group that has responded at the end. therefore, this data is applicable for further tests. description variable satis km earlier (n=50) mean 0.0741 .0099 standard deviation 1.0515 1.131 late (n=21) mean -0.176 -0.238 standard deviation 0.891 0.645 probability 0.343 0.875 table 7. non-response bias-test factor analysis this study has used structural equation model the alternative method partial least square (pls). the program to be used is the smartpls 2.0 m3, which is derived from http://www.smartpls.de (ringle, wende, and will, 2005). the reason for the use of this partial least square (pls) is the small sample size. besides because it can estimate a complex model with a small sample, partial least square (pls) does not assume that the data must be normally distributed. further, the use of the pls is also appropriate if the conceptual model and its measurement has not yet been rightly developed or its characteristic is still in the exploration stage of the theoretical development (ghozali, 2011). however, the use of the pls, are limited to exploratory model so that the results of this research is to proposed a model of user satisfaction and managerial performance. testing hypotheses using pls will be done in australasian journal of information systems volume 18 number 3 2014 128 three stages. at the next part a hypothetical test will be conducted, using the pls. the test will be carry out in three phases, that is: vendor total persentage vendor total persentage agresso 1 1.4% oracle 7 9.9% averis 1 1.4% oracle, erp eracomp 1 1.4% erasoft 1 1.4% peoplesoft 3 4.2% erp eracomp 1 1.4% platinum 1 1.4% fova system 1 1.4% sap 32 45.1% ifs 4 5.6% sap, dynamix ax 1 1.4% ifs, mas 1 1.4% sap, jd edwards 1 1.4% ifs, sap 1 1.4% sun systems 2 2.8% in house 4 5.6% sunfish mcs 1 1.4% mikrotek indonesia 1 1.4% wd erp sys 1 1.4% mincom 3 4.2% wd erp sys-my sys 1 1.4% ms d axapta 1 1.4% grand total 71 100% table 8. erp vendor phase 1: the test of the measurement model (outer model) the convergent validity test convergent validity measures the consistency of the loading factor for many operations tested, using the loading factor value towards the construct, which is statistically significant. in this study, the loading factor value used is above 0.7 as proposed by chin, (1998) and ghozali, (2011). besides that, each construction must have the average variance extracted (avt) above 0.5 (fornell and larcker, 1981). at figure 2, it can be observed all loading factor values above 0.7 and the result of the smartpls test, which shows that the ave value of above 0.5, so that the whole indicator can be applied to test the hypothesis. australasian journal of information systems volume 18 number 3 2014 129 source: output smart pls 2.0 m3 figure 2. the convergent validity research model original sample t statistics original sample t statistics a1 <accu 0,9389 43,7384 f1 -> satis 0,8047 15,4149 a1 -> satis 0,8116 13,5616 f2 <form 0,9469 83,2522 a2 <accu 0,9499 90,4288 f2 -> satis 0,9056 40,3067 a2 -> satis 0,8933 31,5605 km1 <km 0,846 26,9722 c1 <cont 0,9096 34,0322 km2 <km 0,8745 28,9261 c1 -> satis 0,8975 33,3979 km3 <km 0,8294 29,8634 c2 <cont 0,9269 41,2957 km4 <km 0,8123 17,8409 c2 -> satis 0,8526 17,9641 km5 <km 0,8839 28,931 c3 <cont 0,9257 44,342 km6 <km 0,8165 15,6028 c3 -> satis 0,8521 16,8733 km7 <km 0,9233 38,2907 c4 <cont 0,9215 39,5405 km8 <km 0,8449 20,6689 c4 -> satis 0,895 30,161 km9 <km 0,842 19,8449 e1 <ease 0,9613 80,3301 t1 <time 0,9648 89,4489 e1 -> satis 0,7533 11,2113 t1 -> satis 0,9286 50,186 e2 <ease 0,9621 83,4785 t2 <time 0,9619 72,9349 e2 -> satis 0,7616 11,0604 t2 -> satis 0,8922 29,261 f1 <form 0,9322 42,7493 table 9. loading factor australasian journal of information systems volume 18 number 3 2014 130 the discriminant validity test the valuation of the discriminant validity at this study was conducted using 2 (two) methods. first, the valuation was carry out based on the cross loading measurement with the construct, if the construct correlation using the item is greater than the other construct. this means that the latent construct will predict better the measurement at their block than the measurement in other blocks. the result of the cross loading test shows that all the loading factor value of each indicator is larger than the measurement at other constructs. the conclusion of this analysis is that each latent construct will be able to predict the measurement of their block better so that this research model has a good discriminant validity value. accu cont ease form km time a1 0,939 0,742 0,512 0,732 0,612 0,763 a2 0,950 0,811 0,596 0,849 0,684 0,848 c1 0,782 0,910 0,628 0,830 0,758 0,859 c2 0,743 0,927 0,546 0,715 0,767 0,824 c3 0,695 0,926 0,616 0,687 0,821 0,791 c4 0,811 0,922 0,645 0,786 0,690 0,893 e1 0,541 0,647 0,961 0,663 0,639 0,618 e2 0,591 0,627 0,962 0,681 0,633 0,618 f1 0,718 0,702 0,656 0,932 0,563 0,710 f2 0,854 0,834 0,657 0,947 0,681 0,837 km1 0,696 0,805 0,595 0,660 0,846 0,735 km2 0,700 0,845 0,574 0,695 0,875 0,834 km3 0,541 0,621 0,539 0,519 0,829 0,615 km4 0,505 0,628 0,525 0,502 0,812 0,578 km5 0,664 0,708 0,633 0,649 0,884 0,695 km6 0,471 0,681 0,621 0,445 0,817 0,638 km7 0,588 0,704 0,558 0,586 0,923 0,662 km8 0,465 0,639 0,534 0,439 0,845 0,531 km9 0,580 0,631 0,479 0,542 0,842 0,591 t1 0,838 0,900 0,637 0,814 0,775 0,965 t2 0,809 0,863 0,600 0,779 0,717 0,962 table 10. cross loading the second valuation was carry out by comparing the square root of average variation extracted (ave) of each construct with the correlation between the construct and other construct in the model. the result of the calculation shows that the square root of ave value is larger than the construct’s correlation value. therefore, is can be said that the model in general has a good discriminant validity (fornell and larcker, 1981). australasian journal of information systems volume 18 number 3 2014 131 ave ave composite reliability r square cronbachs alpha accu 0,892 0,944 0,943 0,818 0,879 cont 0,848 0,921 0,957 0,903 0,940 ease 0,925 0,962 0,961 0,620 0,919 form 0,883 0,940 0,938 0,833 0,868 km 0,728 0,853 0,960 0,677 0,953 time 0,928 0,963 0,963 0,894 0,923 table 11. validity measurement composite reliability value composite reliability is a block indicator that measures a construct reflexively. the composite reliability with a value of more than 0.7 shows that the internal consistency is good. the result of the smart pls of 2.0m3, as seen at table 11, shows that the composite reliability value of the whole construct is above 0.7. therefore, we can conclude that all constructs have a good internal consistency. all measurements of above measurement model show that a value that fulfill the measurement criteria. therefore, we can conclude that all indicators that have already been revised, can be used to test the hypothesis. all items that explain the construct of this study have a relatively high reliability. phase 2: structural model test (inner model) after testing the research model (outer model), we further conduct a test towards the structural model. this test is meant to evaluate the relationship between the constructs that have been proposed at this study’s hypothesis. information that shows how good the structural model predicted can be observed from the r square value, which is the goodness-fit test model, to explain the percentage of the construct variation towards the whole model. the result of the test using the smartpls for the whole level 1 construct (accuracy, content, format, timelines) has shown that a fairly strong explanation power, that is larger that 67% , except for ease of use construct which is only 62%, however, this value is quite substantial (chin,1998). also, for the r square value, the managerial performance (km) shows a fairly high explanation power, which is 67.7%. this means that the eucs construct is applicable to measure the user’s satisfaction of the erp system. besides, the construct of the user’s satisfaction together explain as much as 67.7% of the variation towards the managerial satisfaction. phase 3: hypothetical test the hypothetical test was conducted through the coefficient parameter and the statistical t significant value. conducting the bootstrapping procedure of 300 samples a coefficient path, that shows the power of the relationship between the two constructs. the coefficient path value to relate the construct accu, cont, ease, form, and time towards the whole satis construct is above 0,7 and it is significant at p=0,05 with a t value higher than 1.96. therefore, hypothesis 1 up to hypothesis 5 which conclude that accuracy, content, ease of use and timelines influence directly the satisfaction of the erp system user is accepted. the results of the tests of hypothesis 1 up to hypothesis 5 have proven that the whole eucs construct is applicable to measure the user’s satisfaction of erp. also, the track coefficient value to influence the user’s satisfaction (satis) towards the managerial performance (km) is 0.823 and it is significant at p=0.05 with a t value higher than 1.96, that is 17.471. therefore, hypothesis 6 that proposes that there is a direct influence between the satisfaction of the erp user and the managerial australasian journal of information systems volume 18 number 3 2014 132 performance is accepted. this means that the erp user’s satisfaction influences directly the managerial performance. original sample t statistics satis -> accu 0,905 29,702 satis -> cont 0,950 55,219 satis -> ease 0,788 12,363 satis -> form 0,913 40,101 satis -> km 0,823 17,471 satis -> time 0,945 71,183 table 12. coefficient path discussion and conclusion this study is conducted with two aims, e.g. to analyze whether the content, accuracy, format, ease of use, and timeliness are measurements to measure the manager’s satisfaction in using the erp system, and whether there is a satisfaction influence of the manager in using the erp system towards the manager’s performance in the process of making decisions. the result of the data from 71 respondents shows that the construct of content, accuracy, format, ease of use, and timeliness influence significantly the construct of the manager’s satisfaction using the erp system. based on this analytical result, and the test using the factor analysis, we can conclude that the concept as proposed by doll & torkzadeh (1988) regarding the end user computing satisfaction (eucs) can be used to measure the manager’s satisfaction using the erp system in this study. the second analysis is to test the manager’s satisfaction, using the erp system, as measured by the content, accuracy, format, ease of use and timeliness towards the managerial performance, in which the managerial performance is explained through the questions items regarding the frequency and the reporting quality, the effectiveness of the monitoring activities, and the coordination by the management, the decision making process, the easiness to the information’s availability, the reduction of information management, increase in the information flow, increasing the flexibility and the company’s integration. hence, from the result of the analysis, it can be concluded that in general, this study has supported the end user computing satisfaction (eucs) concept as proposed by doll & torkzadeh (1988), and other studies using the same concept and other information systems as well as the erp system such as hees and hightower (2002), roses (2006), deng et al. (2008) and mitakos et al. (2010). therefore, this study has confirmed the eucs concept for the erp system, besides for other information systems, such as for previous studies. besides, this study confirms the model as proposed in this study that the manager’s satisfaction using the erp system can influence the manager’s performance in the decision making process. limitation and future agenda the limitedness of this study concerns the limitedness of the data, because this study could not used the data by random sampling because there is no information which shows that the quantity and the name of the companies which have already officially used the erp system. besides, the size of the samples of this study is also small, which is caused by the many companies that have not yet given the permission for the publication of information about the use of the erp system for the public. the other limitedness is the managerial performance measurement, using the respondent’s perception, is represented by only one respondent and satisfaction was measured quantitatively. considering the australasian journal of information systems volume 18 number 3 2014 133 limitedness of this study, the advice for the coming study is to use the case study method in order to delve more the viewpoint of the ultimate user of the organization as a whole, thus a satisfaction measured not only quantitatively but also qualitatively. references arnold, v. (2006) behavioral research opportunities: understanding the impact of enterprise systems. international journal of accounting information systems, 7(1), 7-17. chin, w. w. (1998) commentary: issues and opinion on structural equation modeling. mis quarterly, vii-xvi. delone, w. h., & mclean, e. r. (1992) information systems success: the quest for the dependent variable. information systems research, 3(1), 60-95. delone, w. h. (2003) the delone and mclean model of information systems success: a ten-year update. journal of management information systems,19(4), 9-30. deng, x., doll, w. j., al-gahtani, s. s., larsen, t. j., pearson, j. m., & raghunathan, t. s. (2008) a cross-cultural analysis of the end-user computing satisfaction instrument: a multi-group invariance analysis. information & management, 45(4), 211-220. doll, w. j., & torkzadeh, g. (1988) the measurement of end-user computing satisfaction. mis quarterly, 259-274. fornell, c., & larcker, d. f. (1981) evaluating structural equation models with unobservable variables and measurement error. journal of marketing research, 39-50. ghozali i. (2011) “structural equation modeling: metode alternatif dengan partial least square (pls)”. edisi 3, semarang: badan penerbit universitas diponegoro. hess, t. j., & hightower, r. (2002) using equity theory to understand user satisfaction with erp systems: extending and advancing the equity-implementation model. in twenty-third international conference on information systems (icis) december. khalifa, mohamed and liu, vanessa. (2003) the state of research on information system satisfaction, journal of information technology theory and application 5(4), article 4. kim k. (1988) organizational coordination and performance in hospital accounting information systems: an empirical investigation.” the accounting review, 6: 85-99. mitakos, t., almaliotis, i., & demerouti, a. (2010) an auditing approach for erp systems examining human factors that influence erp user satisfaction. informatica economica, 14, 78-92. nah, f.f.h., lau j.l.s., dan kuang, j. (2001) critical factors for successful implementation of enterprise systems, business process management journal, 7(3): 285--296. nicolaou a.i. (2000) a contingency model of perceived effectiveness in accounting information systems: organizational coordination and control effects, international journal of accounting information systems, 1: 91--105. ringle c.m., wende s., dan will s. (2005) smartpls 2/0 (m3) beta, hamburg, http://ww.smartpls.de. accessed 29 september 2011. roses, l. k. (2006) information system success: antecedents of end-user satisfaction with erp. enanpad-encontro nacional de programas de pós-graduação em administração. sajady h., dastgir m., dan nejad h.h. july (2008) evaluation of the effectiveness of accounting information systems, international journal of information science and technology, 6 (2). spathis c. dan constantinides s. (2004) enterprise resource planning systems’ impact on accounting processes, business process management journal, vol. 10 (2):234--47. australasian journal of information systems volume 18 number 3 2014 134 ugboma e. (2004) assuring information systems’ effectiveness through data integrity: essential guidelines for information systems databases, proceeding isecon 2004, v21. australasian journal of information systems volume 18 number 3 2014 135 appendix cfa result for eucs variable component matrixa component matrixa component matrixa component component1 component 1 1 1 t1 .963 e1 .962 a1 .945 t2 .963 e2 .962 a2 .945 extraction method: principal component analysis. a. 1 components extracted. extraction method: principal component analysis. a. 1 components extracted. extraction method: principal component analysis. a. 1 components extracted. component matrixa component matrixa component matrixa component component1 component 1 1 1 f1 .940 c1 .907 km1 .839 f2 .940 c2 .929 km2 .873 extraction method: principal component analysis. a. 1 components extracted. c3 .928 km3 .836 c4 .920 km4 .810 extraction method: principal component analysis. a. 1 components extracted. km5 .887 km6 .801 km7 .939 km9 .845 extraction method: principal component analysis. a. 1 components extracted. paper title (paper title style) australasian journal of information systems scott, richards & adhikari 2015, vol. 19, selected papers from acis security risks & safety on mobile health apps a review and comparative analysis of security risks and safety measures of mobile health apps karen scott the children’s hospital at westmead university of sydney karen.scott@health.nsw.gov.au deborah richards macquarie university deborah.richards@mq.edu.au rajindra adhikari macquarie university rajindra.adhikari@students.mq.edu.au abstract in line with a patient-centred model of healthcare, mobile health applications (mhealth apps) provide convenient and equitable access to health and well-being resources and programs that can enable consumers to monitor their health related problems, understand specific medical conditions and attain personal fitness goals. this increase in access and control comes with an increase in risk and responsibility to identify and manage the associated risks, such as the privacy and security of consumers’ personal and health information. based on a review of the literature, this paper identifies a set of risk and safety features for evaluating mhealth apps and uses those features to conduct a comparative analysis of the 20 most popular mhealth apps. the comparative analysis reveals that current mhealth apps do pose a risk to consumers. to address the safety and privacy concerns, recommendations to consumers and app developers are offered together with consideration of mhealth app future trends. keywords: mhealth, security, privacy, apps. 1 introduction mobile health (mhealth) is the provision of health services and information via mobile devices, such as smartphones, tablet and laptop computers, and personal digital assistants (pdas) (akter et al. 2011). mhealth can enable healthcare that is “more accessible, affordable and available” (akter et al. 2013 p. 181), given that the majority of people throughout the world (more than 5.5 billion people) own a mobile phone (akter et al. 2013) and their tendency to keep them close at hand at all times (klasnja and pratt 2012). mhealth also enables healthcare communication, such as phoning new mothers (tamrat and kachnowski 2012) or sending patients an sms with information about their medical condition (technologies 2014). mobile health applications (health apps) are software programs that offer health-associated facilities for mobile phones and tablet computers (remedy health media 2014). the use of health apps has exploded with the introduction of the smartphone, including apple’s iphone and google’s android platform (schulke 2013). health apps are available to consumers to use anywhere and anytime, such as while they are at home, at work, studying or travelling (techtarget 2009-2014). health apps have been integrated into the field of health care in an attempt to address a wide variety of issues. health apps can significantly improve the availability and affordability of healthcare for patients, especially those in rural and remote locations (mirza et al. 2008). they can reduce physical and scheduling difficulties between healthcare specialists and patients living in dispersed locations, and help patients interact with healthcare providers (healthcarebusinesstech 2014). health apps can enable physicians to keep track of a patient’s condition remotely (brookings 2013, supplemental health care 2014) and assist consumers to 1 australasian journal of information systems scott, richards & adhikari 2015, vol. 19, selected papers from acis security risks & safety on mobile health apps self-monitor their own condition by measuring and collecting personal data, such as food intake, exercise and blood sugar levels (ucsf 2012). despite these promised benefits, there are some limitations. consumers may not have smartphones or may not fully understand how to use them and may be unable to use an app if one is recommended to them (webmd llc 1994 2014). incorrect medical advice provided by health apps can be harmful because consumers may rely on the apps to diagnose a condition or delay seeking necessary care (schulke 2013). health apps that provide false clinical measurements can lead to inaccurate or unnecessary care for consumers, which could potentially prove fatal. prior to the existence of mhealth apps, software in general was identified as being responsible for 20% of medication errors (hicks 2004), with particular concerns around the lack of regulation and policy for regulating safety-critical healthcare applications. even if regulations existed, “it is an overwhelmingly difficult task for the client to keep up with all regulations manually, and to check if the provider is meeting the compliance” (khan and bai (2013, p. 719). additionally, health apps may pose significant risks to the privacy and security of consumers’ protected health information. sensitive data gathered by health apps may be accessible to the patient, physician, family or scientific researchers, but may also be shared with third parties, such as advertisers (vodafone group 2013), putting the confidentiality of consumers’ data at risk. given the increasing use of electronic health records (ehr) and electronic healthcare (ehealth), the confidentiality, integrity and availability of consumer data are today’s major issues for health service providers in terms of security and data privacy (wang et al. 2013). the research scope for this project focuses on the data privacy and security issues associated with consumer use of health apps. the research questions are: 1. what issues related to data privacy and security are involved in using health apps? 2. what data privacy and security measures can be used in health apps? 3. what is the risk or safety status of current health apps? 4. what recommendations can be made to improve the privacy and security of health apps? to identify the data privacy and security issues (question 1) and measures in use (question 2), we conducted a review of the literature. to understand whether current health apps adhere to the measures or pose a risk (question 3), we analysed the 20 most popular health apps from apple and google stores, further described in our approach. after presenting our findings, we provide recommendations (question 4) for consumers and developers and our conclusion. 2 literature review verasoni’s research division, ahha! insights, executed a study of mobile application usage in the usa in 2012 (verasoni 2012). the research reviewed the top 150 applications on google android and the top 150 applications on apple iphone. their findings were: 1. medical apps on such topics as emergency medicine, medications and especially medical references were downloaded more by android than iphone users. 2. more sleep and meditation apps were available to iphone than android users. 3. when total download numbers were considered, exercise was the most prominent app category across iphone and android platforms. this category led the field with 102 (34%) of the top 150 apps. other categories in the top 150 apps were: medical reference guides with 34 apps (11%), weight loss with 31 (10%), sleep and meditation with 26 (9%), women’s health with 21 (7%), medical tools and instruments with 19 (6%), medication with 15 (5%), pregnancy with 14 (5%) and other health apps with 38 (13%). 4. the top earning apps for android and iphone users were exercise, weight loss, sleep and meditation, and women’s health. 2 australasian journal of information systems scott, richards & adhikari 2015, vol. 19, selected papers from acis security risks & safety on mobile health apps 5. downloads for paid apps were seven times higher for iphone than android users. the increasing number of health apps could improve the quality and affordability of health care and allow patients to safely and securely connect with their physicians. however, schulke (2013) warns that the increase of free and paid health apps on the market might pose health risks to patients due to a lack of health professional involvement in the development of the apps. schulke (2013) defines two broad classes of health apps: provider-focused and patientfocused. this literature review considers data privacy and security risks of health apps from a consumer perspective and includes use of a health app by a provider, patient and the general public. we present below a number of security and privacy themes identified from the literature, including security and privacy challenges, poorly protected consumer data, data security breaches, lack of app standards/guidelines and health app cloud storage. finally, we present suggested security measures for mobile devices to minimise these risks. 2.1 security and privacy issues data security and privacy are major concerns for the use of ehr (kharrazi et al. 2012) and health apps by healthcare providers and consumers (ford faudree and ford (2013), and pose a major barrier to their widespread use (kharrazi et al. 2012). if healthcare providers are unable to provide adequate safeguards to patient privacy when accessing ehr from a mobile device, the consequences can be significant. as evidence of the security and privacy challenges, kharrazi et al. (2012) reported that a small survey found 93% of clinicians use smartphones to access ehr but only 38% follow a formal mobile privacy policy. kharrazi et al. (2012) also note that individual healthcare providers and consumers may lose their devices or may not use any security authentication to protect their data. therefore to keep data secure and private, it is the responsibility of individual health professionals and consumers to set up device and app passwords. privacy issues related to using health apps include a breach of consumer confidentiality, data privacy shortcomings and security problems. to accelerate adoption of mhealth, it is important for individual apps to put in place supportive privacy policies (p. a. consulting 2012). a clear privacy policy can tell consumers what permissions an app requires of the device before downloading it, such as geo-location services access, book access, camera access, phone call access and contacts access. if consumers are not comfortable with health apps that are asking many permissions, they should avoid downloading them. 2.2 poorly protected consumer data the poor protection of consumer data in health apps has been reported by mccarthy (2013). in the study of 43 health and fitness apps, it was found that only 74% of the free apps and 60% of the paid apps had a privacy policy, available either in the app or on the developer’s website. however, only 25% of the free apps and 48% of the paid apps informed consumers about the privacy policy. furthermore, none of the free apps and only a few of the paid apps encrypted the data that consumers entered into the apps. encryption is the conversion of data into a form that cannot be easily understood by unauthorized people. therefore health apps that do not encrypt consumers’ information can pose a threat to data privacy. similarly, nasiri (healthcarebusinesstech 2014) found the information that consumers entered into health apps may carry privacy risks. he reported a survey of 20 of the 23 most popular free health apps that found 50% send data to third-party advertisers and 39% send data to unidentified parties without any data encryption. he found paid health apps are safer than free health apps to a certain degree and that many free health apps send data, connect to third-party sites, use unencrypted connections, allow for data collection by third parties and store data externally. most of the time users are not notified that this has occurred. based on a clearinghouse review of 43 popular health and fitness apps, leventhal (2013) concluded that “consumers should not assume any of their data is private in the mobile application environment even health data that they consider sensitive” leventhal (2013, p.1). 3 australasian journal of information systems scott, richards & adhikari 2015, vol. 19, selected papers from acis security risks & safety on mobile health apps some health apps involve an internet-enabled mobile device connecting wirelessly to portable or embedded sensors that track or measure a patient’s health condition or a user’s activities, which can also pose a security risk (vodafone group 2013). the tracking of the user’s health can occur in real-time, with or without the user’s involvement or approval and can be simultaneously shared with others. the data gathered by such health apps not only carry detailed information about a user’s health but also about their habits, location and movements, which can put the user’s sensitive information at risk if such information is disclosed. 2.3 data security breaches data security breaches in healthcare have become common, according to figg and kam (2011), with many on-line mhealth providers, including doctors and scientific researchers, able to view patients’ medical records without patients’ knowledge. the authors noted that the data security breaches in healthcare are a privacy threat that might lead to medical identity theft. the world privacy forum describe medical identity theft as an occurrence in which a person uses another person’s identity, such as a person’s name or medicare number, without the person’s knowledge and consent, which would impact many consumers’ security (dixon 2006). according to the us federal trade commission (ftc, 2015), in 2014 identity theft topped the list of consumer complaints for the 15th consecutive year. medical identity theft could result in some unauthorized benefits to the offender. the offender may steal patients’ records to sell on the black market or alter patients’ records for fun, such as adding false entries regarding diagnosis, blood types, drug allergies and other health information. victims who have their medical records altered by offenders may receive false medical treatment that may have negative consequences to their health, possibly causing death. 2.4 lack of app standards and guidelines exposure to the risks identified above, including fraud and identify theft, unauthorised access to personal information and loss or theft of a mobile device with unencrypted consumer data, need to be monitored and mitigated. a secure mhealth environment that builds trust among consumers and healthcare service providers would need to adopt standard app development guidelines to increase security and protect consumers from any unauthorized attacks (faudree and ford 2013). currently, security concerns about health apps are increasing due to the lack of standard app development guidelines (kharrazi et al. 2012). kharrazi et al. (2012) emphasise that a thorough verification process is required by the app stores that could detect unsafe programs and thereby reduce the security risks of health apps. the problem goes beyond the need for development guidelines to a need for regulation or supervision in the development and publication of mobile apps. some legislation does exist that could potentially be adapted. for example, the health insurance portability and accountability act (hipaa) seeks to protect the privacy rights of individuals by requiring healthcare providers to obtain consent from individuals prior to any disclosure of data to others. however, hipaa does not explicitly identify and apply the regulations to the mobile context. although in some countries, such as the united states of america, there have been initiatives to regulate mobile medical apps since 2011, these initiatives have not been approved, as evidenced in a statement on 6th of september 2013 that the “food and drug administration safety innovation act (fdasia) wants action on the mobile medical applications guidance” (jarrin, 2013, p.1). an additional factor for consideration is whether regulations are complied with and how compliance is monitored. currently, verification of compliance to hipaa is undertaken by independent auditors (munro 2013). 2.5 mhealth cloud storage mobile cloud computing brings a set of new challenges, especially when it comes to the availability of services and the security and privacy of consumers (gu and guirguis 2014). security issues are critical when a healthcare provider plans to deploy a cloud-based ehr management system because moving patient data to the cloud means patient files are hosted 4 australasian journal of information systems scott, richards & adhikari 2015, vol. 19, selected papers from acis security risks & safety on mobile health apps on the servers of the cloud service provider (piette et al. 2011). according to rui and ling (2010), when moving patient data to the cloud, healthcare providers are exposing information to several external threats because the data is available via the internet. as this data is subject to the same health insurance privacy and security rules, such as hipaa, as data stored in any other way, both cloud service providers and healthcare providers must understand the consequences in relation to the privacy risks of consumers’ sensitive data: the healthcare provider must guarantee the security of patient data by ensuring that the cloud platform has the needed security mechanisms in place (research 2013); and it is the cloud provider’s responsibility to protect the security and privacy of information by providing the security needed to avoid external attacks to steal or even delete the information. however, compliance with hipaa’s security requirements is harder in the cloud computing context (moyle 2013) because the implementation of compliance resides with the cloud provider, while the legal responsibility remains with the healthcare provider. the us department of health and human services recently modified the hipaa legislation to include cloud providers in the legal framework under the term business associate agreement (baa) to specify the legal and financial liabilities of cloud service providers (munro 2013). despite this change, due to the common use of third-party providers by cloud providers, data protection agencies in the us and european union (eu), place ultimate accountability with the clients of cloud computing. 2.6 mobile device security measures mobile technology is the technology used for cellular communication. the us office of civil rights (ocr) recently launched an educational initiative for mobile devices (healthit 2014), which identified the following guidelines for app developers: know the risks; take the steps; protect and secure health information. table 1 summarises measures suggested by ocr (healthit 2014; senft 2013) to ensure mhealth information is secure when using mobile devices such as smartphones and tablets. the ocr has reported that a significant number of data privacy breaches each year arise from lost or stolen mobile devices. with increasing access to unsecured wi-fi networks and file sharing features, security is a greater concern as breaches could occur from unauthorized access to data stored on mobile devices (wikipedia 2014). 5 australasian journal of information systems scott, richards & adhikari 2015, vol. 19, selected papers from acis security risks & safety on mobile health apps measure description use of a password or other user authentication mobile devices can be configured to require passwords, personal identification numbers (pins) or passcodes to gain access. these can be masked to prevent people seeing them. install & enable encryption encryption renders data unusable, unreadable and undecipherable to unauthorized individuals. activate remote wiping and/or remote disabling remote wiping allows an individual to permanently erase all data stored on a mobile device remotely if the device is lost or stolen. if the mobile device is later recovered, it can be unlocked. file-sharing applications or software file sharing allows internet users to connect and share or trade files. enabled file sharing could provide unauthorized users with access to a mobile device without users’ knowledge. firewall a personal firewall can protect against unauthorized connections by intercepting incoming and outgoing connection attempts and blocking or permitting the connection based on an established predetermined set of rules security software security software can be installed and regularly updated to protect against malicious applications, viruses, spyware and malwarebased attacks, which can exist in email attachments, websites or downloaded programs. using non-secured wi-fi network or hotspot without using a secured wi-fi network, there is a risk that communications will be intercepted. users should avoid sending or receiving information when connected to public wireless network. delete mhealth information all data stored on a mobile device needs to be deleted before the device is discarded. software can be installed to overwrite the data or devices can purge or destroy the data. table 1. measures to ensure mhealth information is secure (healthit 2014) (senft 2013) based on findings from the selected literature, it appears that security and data privacy in health apps are growing concerns among healthcare providers and consumers. the lack of privacy of user data and lack of regulation and guidelines for the development of health apps need to be considered for the improvement of health apps. 3 methodology to determine the extent to which current health apps pose a data privacy and security threat and, where necessary, recommend solutions, we conducted a comparative analysis as a case study of the 20 most popular free and paid health apps from apple and google stores. the process involved the following three steps: 1. selection criteria for 20 health apps 2. identification of data privacy and security features and issues of the 20 health apps 3. comparative analysis of 20 health apps the selection of the apps was determined by the highest number of downloads on the health app market, with high ratings from consumers, search engines or app stores, and recommendations through social media. apps were trialled on appropriate mobile devices: apps from apple stores on an ipod touch and apps from google stores on an android phone. 6 australasian journal of information systems scott, richards & adhikari 2015, vol. 19, selected papers from acis security risks & safety on mobile health apps from the literature we identified a set of nine features, presented in the results section, that we categorised as security risks (first three) or safety measures (last six). to measure each app against these features we asked the questions below. note question 2 is used to identify risk (i.e. cloud storage) and safety measure (i.e. local storage of data that is password restricted). 1. does the app ask for user registration (name, address, birthday and email)? most health apps available on the market ask for consumers’ details prior to registration. it is consumers’ responsibility whether or not to provide information to health apps. providing detailed information may result in compromising data privacy. 2. where is data stored (locally on a device or in a cloud)? data storage is the recording and storing of consumer information. data storage can either be possible locally on the mobile device or in cloud storage, depending upon the development of the health app. 3. is consumer data shared with a third party or advertiser? sharing of health consumer data with a third party or advertiser is becoming a more serious concern as many apps share consumers’ data to generate revenue. health apps collect detailed personal and health information from consumers to provide better services for health and well-being, but sharing sensitive health information with third parties and advertisers impacts many consumers’ data privacy and security. 4. does an app ask for user authentication (user name and password)? some of the most common threats to data security and patient privacy come from unauthorized access. the purpose of the security evaluation is to check security mechanisms (username and password) are implemented to guarantee the privacy of consumers’ data. 5. are consumers able to update and correct their personal profiles? this allows consumers to change their individual profiles according to the policy of the health apps. updating patient records improves data accuracy and supports better patient care. 6. can consumers delete any personal information completely? it is a prime concern whether health apps allow consumers to delete their personnel information completely. when consumers stop using a health app, they need to be able to delete it. they also need to be able to delete their personal profile and any data archives that have been created with their personal information. 7. are consumers informed about any data privacy and security measures? ensuring the privacy and security of health information, including ehr, is a key component to building the trust required to realize the potential benefits of electronic health information exchange. 8. is there a privacy policy? the privacy policy sets out how a health app uses and protects any information that consumers give to app owners when using the health app. 4 results of selection process of 20 apps in this section we present the evaluation results of the 20 most used health apps on the market based on their privacy and security features. to conduct the comparative analysis, all 20 apps were downloaded onto a corresponding device (iphone or android smartphone). using the above questions, we analysed each app’s authentication functions and features relating to consumer data privacy, as well as the privacy policy on the developers’ websites. a description of each app and how it met the selection criteria for inclusion in our evaluation is provided below. 7 australasian journal of information systems scott, richards & adhikari 2015, vol. 19, selected papers from acis security risks & safety on mobile health apps 4.1 apple stores myfitnesspal (myfitnesspal 2005 2014) is an online and app-based wellness-tracking platform, which has a calorie counter and exercise tracker. it helps to evaluate how many calories the user has eaten versus how many they have burnt. the app can also help to track over time any changes, such as weight or waist size. it provides the services for informational purposes only. no medical professionals were involved in its development. medscape (medscape 1994 2014), developed by webmd, is the leading medical resource, mostly used by physicians, medical students, nurses and other health care professionals for clinical information. it supports clinicians with professional tasks, including decision-making at the point-of-care, medical news and professional development. this is the highest rating and fastest growing free health app, with over 4 million registered users. epocrates (epocrates 2014) is the most popular health app among u.s physicians for clinical content and decision support at the point of care. epocrates has a reliable link of more than a million health care professionals and is used to find providers, review drug prescribing and safety information for thousands of brand and generic medications, and identify pills by imprint code and physical characteristics. the app, built by epocrates, received the 2ndhighest rating out of 15 apps reviewed by the imedicalapps team (imedicalapps team 2011). neuromind (digitalneurosurgeon 2014) offers interactive clinical decision support. it contains more than 120 clinical classification and grading systems, and some anatomical images for explanation to patients and students. this application has been developed with the involvement of surgical neurology international, the european association of neurosurgical societies (eans), and neurosurgic.com. it is considered the best app for neurosurgery in the world, with over 200,000 downloads. smart blood pressure (smartbp) (evolve medical systems 2012) is an easy-to-use blood pressure management tool that helps patients manage their health using their mobile device. smartbp allows users to record, analyze and share blood pressure information. it tracks progress and manages blood pressure measurements with an overall goal of improving blood pressure. blood pressure, pulsing rate and weight can be shared with anyone using email and text message. this application was developed with the involvement of evolve medical systems. pill monitor (appato 2012) is designed to manage and remind users to take pills on time and at the same time every day, which is good for health. this app is very simple and easy to use. key features are a schedule for a pill reminder, consumer reminder time, repeat date and dosage of pills, a check of current reminders and upcoming reminders, and a facility that allows users to add photos for each pill. the app was built by maxwell software and got 4 stars out of 5 based on 353 ratings and 53 user reviews. pregnancy & baby (apple inc. 2014) is designed to look at pregnancy phases. based on the baby’s due date, consumers can receive personalized content and get access to the latest parenting news and health information. it also includes a short video of common symptoms and recommendations, and helps consumers to access a variety of online communities. the app was built by everyday health inc. and got 4 stars out of 5 based on 3330 votes. diabetes tracker plus (apptism 2013) helps people with diabetes to control blood glucose and stay in good health. logging and tracking blood sugar using the app supports consumers to self-manage and self-track diabetes. blood sugar logs and reports can be shared with a healthcare provider by email. there is no information about health professional involvement in the development of the app. this app is recommended by mynetdiary diabetes (mynetdiary diabetes inc 2013). growth (clafou apps 2014) helps to track the growth curves of newborn babies and older children, and compare progress with expected growth rates. this app also helps consumers to share the results with family, friends or health professionals. the development of the app did not involve any health professionals. growth charts are designed in accordance with and recommended by world health organization (who) and centre for disease control. 8 australasian journal of information systems scott, richards & adhikari 2015, vol. 19, selected papers from acis security risks & safety on mobile health apps instant heart rate (azumio 2012) uses iphones’ camera to detect pulse from the fingertip. prior to using the app, consumers need to place the tip of their finger on the camera for few seconds. a real time chart will show every heartbeat. this app has received the best health and fitness app on mobile premier awards 2011. azumio inc. was involved in the development of the app and more than 25 million users already use it. there was no health professional involvement in the development of the app. 4.2 google store apps ob (pregnancy) wheel (google play 2014d) is a free android pregnancy calculator. many clinicians find this app useful, such as those working in primary care, emergency departments and obstetrics. it has numerous adjustable preferences and settings, ultrasound exam dating and dating ordered patient lists. though developed without professional input, this app is recommended by imedicalapps team (imedicalapps team 2011) and has got 4.2 stars out of 5 based on 1208 user reviews. calorie counter (google play 2014a) is an app to help users find nutritional information about the food they eat and easily keep track of meals. it also counsels patients about diet, exercise and weight. users are able to look up most types of food, including fast food, supermarket food and pre-prepared food. consumers can scan barcodes with a camera and the app identifies the type of food, along with allocating the appropriate calories. the app was built by myfitnesspal, inc and has got 4.7 stars out of 5 based on 618,588 user reviews. skyscape medical resources (google play 2014f) is a decision support tool that helps physicians, nurses and students to find the right answers of medical resources, such as medical calculators, periodically updated medical news alerts, practice guidelines and disease monographs. more than 2.7 million healthcare professionals access the medical resources in the app. this app is recommended by imedicalapps team (imedicalapps team 2011). endomondo (google play 2014b) helps track consumers’ workouts and analyze their training. this app can be used as a personal trainer and social fitness partner and is used for running, cycling and walking. it is one of the highest rated apps on the android market. though it was developed with health professional input, more than 20 million users are taking advantage of endomondo app, which has got 4.5 stars out of 5 based 166,459 user reviews. itriage (itriage 2013) is a consumer healthcare company founded in 2008 by two emergency medicine physicians. over 7 million users have downloaded itriage app and used it to locate nearby providers based on their symptoms, make appointments, store their personal health record, save medication refill reminders and learn about medications, diseases and procedures. itriage aims to help answer questions such as: “what medical condition could i have?” and “where should i go for treatment? appointuit (appointuit 2012) helps consumers make an appointment with doctors anywhere, anytime. this app can also cancel and reschedule an appointment if needed. the android apps market recommended this app. cardiograph (macropinch ltd 2011 2013) is used to measure users’ heart rate. the app can save results for future reference and keep track of multiple people with individual profiles. it uses a mobile device’s built-in camera to take pictures of a user’s fingertip and calculate their heart rhythm. the app was built by macropinch and has got 4 stars out of 5 based on 61,195 user reviews. quit now (google play 2014e) offers real-time stats to help users quit smoking. the indicators tell users how long it has been since they last smoked and the money the user saved. it also shows various indicators for how much the user’s heath has improved since they stopped smoking. the app was built by fewlaps and has 4 stars out of 5 based on 9,780 user reviews. gi monitor (google play 2014c) is a symptom logging application for patients with ibd (inflammatory bowel disease), crohn’s or ulcerative colitis. this app allows patients to log symptoms and provide data to their doctors for optimal treatment. no healthcare personal 9 australasian journal of information systems scott, richards & adhikari 2015, vol. 19, selected papers from acis security risks & safety on mobile health apps were involved in the development of the app. this app is recommended by wellapps, medivo (wellapps 2009 2011). stress check (google play 2014g) is an app available for quantifying the level of psychological or physical stress. by measuring patient heart rate through the camera and light features on smartphones, stress check can estimate the level of stress in real time. the app was built by azumin inc. and has got 3.5 stars out of 5 based on 4,223 user reviews. 5 results: comparative analysis table 2 shows a summary of the comparative analysis of the 20 mhealth apps, also identifying the extent to which each safety measure or security risk was evident. the results illustrate that not all health apps are free of issues. out of the 20 apps, only one (5%) enables consumers to delete personal information completely and mentions this facility to users. over half of the health apps 65% (13/20) asked consumers to enter personal information, such as name, address, email and date of birth, but only two apps asked for consumers’ authentication prior to log-in. half (50%, or 10/20) the apps stored data in a cloud that significantly poses risks to consumers’ data privacy and 65% (13/20) of the apps shared consumers’ information to a third party or advertisers. few of the apps (20%, or 4/20) informed consumers about data privacy and security measures, however, 90% (18/20) of the apps have a privacy policy that explained the details of the apps’ privacy and security measures. even though apps such as “medscape” and “skyscape” have higher safety scores (4 out of 6 in table 2) these apps also have considerably higher risk scores. as shown in table 2, some apps, such as “diabetes tracker plus” and “growth,” ask users’ personal information, such as name, address and email, but do not provide any security authentication, such as username and password, prior to log-in; therefore, such apps potentially put confidentiality of consumers’ data at risk. apps such as “quit now”, “pill monitor” and “ob pregnancy” have risk scores of 1 out of 3 but these apps are low risk to use as none asks users for personal information, such as name, address and email, prior to registering the apps. the apps “neuromind”, “smart bp” “cardiograph” and “stress check” apps were found to be relatively safe to use compared to other apps. these apps allow consumers to store results locally on the mobile device and do not share consumer data with a third party or advertisers. even though none of these apps ask consumers for authentication, such as username and password, to log-in, they are safe to use because they do not require consumers to enter personal information. only 20% (4/20) of the apps implement security measures to ensure privacy and security of consumers’ information. these involve the use of firewalls, secure connections on websites, frequently the use of secured socket levels (ssl) to encrypt pages that collect user information and security methods such as authentication to determine the identify of registered users, so that appropriate rights and restrictions can be enforced for that user. 10 australasian journal of information systems scott, richards & adhikari 2015, vol. 19, selected papers from acis security risks & safety on mobile health apps table 2. health app comparison results 6 discussion this study aimed to investigate the data security and privacy features of 20 popular free and paid health apps. from our review, it appears the main risks posed to data protection by health apps are lack of information about the app for consumers, insufficient security measures to safeguard consumers’ sensitive data, information shared with third party or advertisers and lack of user authentication prior to log-in to the app, such as username and password. we recommend that apps should only collect data that is strictly necessary for the app to perform the desired functionality. we also recommend that app developers include a privacy policy and inform consumers about it. a set of recommendations to consumers and app developers can be found in table 3. d oe s th e a p p a sk fo r u se r r eg is tr at io n d et ai ls d at a st or ed i n a c lo u d is c on su m er d at a s h ar ed w it h a t h ir d p ar ty o r a d ve rt is er d oe s an a p p a sk a or u se r a u th en ti ca ti on ( u se r n am e a n d p as sw or d ) a re c on su m er s a bl e to u p d at e an d c or re ct t h ei r p er so n al p ro fi le s c an c on su m er s c om p le te ly d el et e a n y p er so n al in fo rm at io n d at a st or ed l oc al ly o n a d ev ic e a re c on su m er s in fo rm ed a n y d at a p ri va cy a n d s ec u ri ty m ea su re s is t h er e a p ri va cy p ol ic y r is k s co re o u t o f 3 s af e sc or e o u t of 6 myfitnesspal 1 1 1 1 1 1 3 3 medscape 1 1 1 1 1 1 2 4 epocrates 1 1 1 1 1 3 2 neuromind 1 1 0 2 smart bp 1 1 0 2 pill monitor 1 1 1 1 pregnancy & baby 1 1 1 1 1 1 3 3 diabetes tracker plus 1 1 1 1 1 3 growth 1 1 1 1 1 3 instant heart rate 1 1 1 1 2 2 ob (pregnancy) 1 1 1 1 calorie counter 1 1 1 1 3 1 skyscape 1 1 1 1 1 1 1 3 4 endomondo 1 1 1 1 3 1 itriage 1 1 1 1 1 1 3 3 appointuit 1 1 1 1 1 1 3 3 cardiograph 1 1 0 2 quit now 1 1 1 1 2 gi monitor 1 1 1 1 1 1 3 3 stress check 1 1 0 2 total 13 10 13 2 8 1 14 4 18 11 australasian journal of information systems scott, richards & adhikari 2015, vol. 19, selected papers from acis security risks & safety on mobile health apps consumers application developers research the app before downloading it sensitive consumers’ information should always be stored encrypted so attackers cannot retrieve data. try to use apps without entering personal information if permitted provide user with information about the implementation of security measures and authentication, and what/how/where data is stored. look for user reviews and the privacy policy of an app, either through the app store or online. include user authentication. provide options so users can safely retrieve login details if forgotten. only 10% (2/20) of the apps in our study ask for user authentication prior to log-in. remove data when usage stopped. this may prevent unauthorised use of stored data when consumers no longer use the apps. minimise sharing information with third parties or advertisers and ask users to confirm agreement before sharing. in our study 65% (13/20) of the apps shared users’ information with third parties or advertisers. give feedback on product: users’ feedback on the features, privacy and policy, and functions of an app will help the developers to restructure the app appropriately. apps should allow consumers to delete their personal information completely. only 5% (1/20) of the apps in our study mentioned in its privacy policy that consumers can delete information completely; therefore, this criteria needs to be improved appropriately. table 3: recommendations to consumers and app developers the variety and number of health apps currently available make mhealth an important alternative to traditional face-to-face forms of healthcare. however, as presented in this paper, there are numerous privacy and safety concerns associated with these tools. nasiri (healthcarebusinesstech 2014) found that 50% of health apps transfer information to a third party and 39% send data to unidentified parties without consumer consent. in our study of 20 health apps, 65% send consumer data to a third party or advertisers; in a 2013 study of 43 health and fitness apps, only a few of the paid apps encrypted the data entered by consumers (mccarthy 2013). we believe this high rate is due to a lack of appropriate guidelines for app developers and owners. furthermore, although 88% (15 out of 17) of the free health apps and 100% (all three) of the paid health apps in our study have a privacy policy, only 23% (4 out of 17) of the free apps and none of the paid apps informed consumers about security measures. the study by mccarthy (2013) found that 74% of free and 60% of paid apps had a privacy policy in the app or on the developer website. related to safety, but not evaluated in the apps reviewed here, is the lack of health professional involvement in the development of the health apps, lack of evidence-based medical practices, lack of effectiveness and deficient functionality (scott et al. 2015). for example, burke et al (2012, p.3) reported that a small survey conducted in the uk on the ten most popular parenting and child health apps available on itunes (until that time) showed that only one of the apps had been developed with healthcare professional involvement in the design and evaluation. in addition, leroux and rivas (2013, p.1-2) from stanford university refer to the lack of regulation, effectiveness and evidence-based medical practices in the majority of health apps. similarly, mccartney (2013) highlights that users of health apps do not have any source of information to identify which apps provide evidence-based medical content and which do not. she reports that a study in jama dermatology found that many apps designed to diagnose melanoma had a failure of 30%. in the united kingdom the national health service (nhs) has 12 australasian journal of information systems scott, richards & adhikari 2015, vol. 19, selected papers from acis security risks & safety on mobile health apps created a database of health apps they have reviewed and approved; however, some of these apps lack real life tests to quantify the potential benefit or harm they contain. another issue is related to the lack of independent assessment of the app software before publication. mack (2013, p 4-5) gives one example of a health app that had to be recalled because the formula used to measure rheumatoid arthritis was wrong. fortunately the app owner noticed the error and recalled it for a bug fix, accompanied by information to potential users. if there is no independent software verification, it means that several other apps may contain errors and these may provide false results to users. 7 future trends for mobile health apps a study conducted by rodrigues et al (2012), p.167-168) found that the future of mhealth will evolve from individual-focused apps to those that integrate an element of social networking to promote healthy behaviours in groups. currently, social networking plays a key role within individuals’ daily lives and has made a revolution in the way people interact globally. they also note that social network sites, such as facebook, have health-related support groups that provide educational and emotional support for users (rodrigues et al 2012). the development of mobile health projects that incorporate components of social networking for patients can help endorse group and community health. in terms of numbers, walsh (2013, p.1) predicts there will be an explosion in the mhealth market. mobile health trends and figures 2013-2017 presents the future development of health apps, emphasizing the following eight important trends that will shape the mhealth market until 2017: 1. smartphone user penetration will be the main driver for the mobile health acceptance; 2. mobile health applications will be designed specifically for smartphones or tablets; 3. mobile health applications will be native (developed especially for mobile environment) rather than web-based applications; 4. lack of regulation is the main market obstacle during the commercialization phase; 5. customers will continue to drive the market; 6. applications will adopt traditional health distribution channels; 7. the mhealth market will evolve mainly in countries with high smartphone penetration and health expenditure; 8. second generation health apps will focus on chronic diseases. to aid compliance with health regulations, there are efforts to automate the compliance checking through the use of tools and testing suites, such as those developed by the certification commission for health information technology (cchit) and the national institute of standards and technology (nist) (khan and bai, 2013). an alternative approach has been offered by considering earlier phases of the software development lifecycle to ensure that non-functional requirements, including privacy and security, are elicited from health regulations (austin et al. 2010; ingolfo et al. 2011). work that specifically seeks to provide automated verification of compliance with health regulations in the context of cloud based services and third party service providers has been conducted by khan and bai (2013). they address this critical and challenging task by reasoning over regulations that have been transformed into machine-processable language and real-time data from cloud machines on service specific compliance. as kan and bai (2103) note, the approach is limited by the relevancy and accuracy of the data they are able to obtain and they are exploring approaches to ensure the data they collect is accurately identified and up to date. 13 australasian journal of information systems scott, richards & adhikari 2015, vol. 19, selected papers from acis security risks & safety on mobile health apps 8 conclusion this comparative analysis of 20 health apps illustrated that not all health apps available in the app stores are free of privacy and security issues. major areas that need to be focussed on in order to develop secure apps are standard app development guidelines and security authentication measures, such as device and app passwords, appropriate encryption mechanisms and an informative privacy policy on each app. as future work, it would be important to analyse the security mechanisms and encryption methods of apps. as it is unlikely that vendors will provide this information, discovery of these methods would probably involve trying to hack into an app, a dubious activity that we did not want to undertake. the study seeks to warn consumers, healthcare personal and app developers to take caution when adopting and developing health apps by providing them with the knowledge about app issues, as well as benefits and risk associated with health apps in healthcare. references akter, s., d'ambra, j., and ray, p. 2011. "trustworthiness in mhealth information services: an assessment of a hierarchical model with mediating and moderating effects using partial least squares (pls)," journal of the american society for information science and technology (62:1), pp. 100-116. akter, s., d’ambra, j., and ray, p. 2013. "development and validation of an instrument to measure user perceived service quality of mhealth," information & management (50:4), pp. 181-195. appato. 2012. "pill monitor freemedication reminders and logs." retrieved 07 may 2014, from http://www.appato.com/maxwell-software/pill-monitor-free-medicationreminders-and-logs/# apple inc. 2014. "pregnancy & baby what to expect." retrieved 09 may 2014, from http://itunes.apple.com/au/app/pregnancy-baby-what-to-expect/id289560144?mt=8 appointuit. 2012. "appointuit." retrieved 10 may 2014, from http://appointuit.com/why apptism. 2013. "diabetes tracker plus health & fitness app." retrieved 09 may 2014, from http://www.apptism.com/health-fitness/ooober-inc/diabetes-tracker-plus/ austin, a., smith, b., and williams, l. 2010. "towards improved security criteria for certification of electronic health record systems," proceedings of the 2010 icse workshop on software engineering in health care: acm, pp. 68-73. azumio. 2012. "instatnt heart rate." retrieved 10 may 2014, from www.azumio.com/apps/heart-rate/ brookings, c. f. t. i. a. 2013. "improving health care through mobile medical devices and sensors." retrieved 10 april 2014, from http://www.brookings.edu/~/media/research/files/papers/2013/10/22%20mobile%2 0medical%20devices%20west/west_mobile%20medical%20devices_v06.pdf clafou apps. 2014. "follow your child’s growth." retrieved 10 may 2014, from www.growthapp.net digitalneurosurgeon. 2014. "description." retrieved 04 may 2014, from http://blog.digitalneurosurgeon.com/ & http://itunes.apple.com/us/app/neuromind/id353386909?mt=8 dixon, p. 2006. "medical identity theft: the information crime that can kill you," the world privacy forum. epocrates. 2014. "epocrates, with you at the moment of care." retrieved 04 may 2014, from www.epocrates.com 14 australasian journal of information systems scott, richards & adhikari 2015, vol. 19, selected papers from acis security risks & safety on mobile health apps evolve medical systems. 2012. "smart blood pressure." retrieved 04 may 2014, from http://www.evolvemedsys.com/> and <https://itunes.apple.com/au/app/bloodpressuresmart-blood/id519076558?mt=8 faudree, b., and ford, m. 2013. "security and privacy in mobile health," cio journal). figg, w. c., ph.d, and kam, h. j., m.s. 2011. "medical information security," international journal of security (ijs) (5:1). ftc, 2015. identity theft tops ftc’s consumer complaint categories again in 2014: imposter scams on the rise. ftc international monthly. march 2015. retrieved 9 july 2015 from https://www.ftc.gov/policy/international/ftc-international-monthly/march-2015. google play. 2014a. "calorie counter by fat-secret." retrieved 08 may 2014, from http://play.google.com/store/apps/details?id=com.fatsecret.android google play. 2014b. "endomondo sports tracker." retrieved 10 may 2014, from http://play.google.com/store/apps/details?id=com.skyscape.android.ui google play. 2014c. "gi monitor." retrieved 12 may 2014, from http://play.google.com/store/apps/details?id=com.wellapps.gimonitor google play. 2014d. "ob pregnancy wheel." retrieved 09 may 2014, from http://play.google.com/store/apps/details?id=com.quartertone.medcalc.obwheel google play. 2014e. "quit smokingquit now." retrieved 10 may 2014, from http://play.google.com/store/apps/details?id=com.eaginsoftware.dejaloya google play. 2014f. "skyscape medical resources." retrieved 10 may 2014, from http://play.google.com/store/apps/details?id=com.skyscape.android.ui google play. 2014g. "stress check by azumio." retrieved 12 may 2014, from http://play.google.com/store/apps/details?id=com.azumio.android.stresscheck gu, q., and guirguis, m. 2014. "secure mobile cloud computing and security issues," in high performance cloud auditing and applications. springer, pp. 65-90. healthcarebusinesstech. 2014. "mobile health apps create privacy risk, study says." retrieved 18 march 2014, from http://www.healthcarebusinesstech.com/mobilehealth-apps-privacy/ healthit. 2014. "your mobile device and health information privacy and security." retrieved 26 april 2014, from http://www.healthit.gov/providers-professionals/your-mobiledevice-and-health-information-privacy-and-security hicks, r. 2004. medmarx 5th anniversary data report: a chartbook of 2003 findings and trends 1999-2003. us pharmacopeia. imedicalapps team. 2011. "top 15 free android medical apps for healthcare professionals." retrieved 10 may 2014, from http://iims.uthscsa.edu/sites/iims/files/top%2015%20free%20android%20medical %20apps%20for%20healthcare%20professionals.pdf ingolfo, s., siena, a., and mylopoulos, j. 2011. "establishing regulatory compliance for software requirements," in conceptual modeling–er 2011. springer, pp. 47-61. itriage. 2013. "itriage take charge of your health." retrieved 10 may 2014, from https://www.itriagehealth.com/ jarrin r., 2013. fdasia workgroup wants action on mobile medical apps guidance, mhealth news, retrieved 9 july 2015 from http://www.mhealthnews.com/news/fdasiaworkgroup-wants-action-mobile-medical-apps-guidance. 15 australasian journal of information systems scott, richards & adhikari 2015, vol. 19, selected papers from acis security risks & safety on mobile health apps khan, k.m., bai, y.: 2013. automatic verification of health regulatory compliance in cloud computing. in: ieee 15th international conference on e-health networking, applications and services, pp. 713–717. kharrazi, h., chisholm, r., vannasdale, d., and thompson, b. 2012. "mobile personal health records: an evaluation of features and functionality," international journal of medical informatics (81:9), pp. 579-593. klasnja, p., and pratt, w. 2012. "healthcare in the pocket: mapping the space of mobile-phone health interventions," journal of biomedical informatics (45:1), pp. 184-198. leventhal r. 2013. study: popular mobile health apps carry considerable privacy risks. retrieved 9 february 2015 from: http://www.healthcare-informatics.com/newsitem/study-popular-mobile-health-apps-come-considerable-privacy-risks macropinch ltd. 2011 2013. "cardiograph – personal heart rate meter." retrieved 10 may 2014, from http://macropinch.com/cardiograph/ mccarthy, m. 2013. "experts warn on data security in health and fitness apps," br. med. j (347), p. 1. mccartney m. 2013. how do we know whether medical apps work? bmj 2013; 346 doi: http://dx.doi.org/10.1136/bmj.f1811 medscape. 1994 2014. "definition." retrieved 04 may 2014, from http://www.medscape.com/ mirza, f., norris, t., and stockdale, r. 2008. "mobile technologies and the holistic management of chronic diseases," health informatics journal (14:4), pp. 309-321. moyle, e. 2013. "why cloud computing changes the game for hipaa security," in: technewsworld. munro, d. 2013. "hipaa support widens in cloud vendor community," forbes [internet]). myfitnesspal. 2005 2014. "lose weight with myfitnesspal." retrieved 04 may 2014, from http://www.myfitnesspal.com/ mynetdiary diabetes inc. 2013. "mynetdiary diabetes." retrieved 22 may 2014, from http://www.mynetdiary.com/diabetes-tracker-for-iphone.html p a consulting, g. 2012. "policy and regulation for innovation in mobile health." retrieved 16 april 2014, from http://www.gsma.com/mobilefordevelopment/wpcontent/uploads/2012/04/policyandregulationforinnovationinmobilehealth.pdf piette, j. d., mendoza-avelares, m. o., ganser, m., mohamed, m., marinec, n., and krishnan, s. 2011. "a preliminary study of a cloud-computing model for chronic illness self-care support in an underdeveloped country," american journal of preventive medicine (40:6), pp. 629-632. remedy health media, l. 2014. "health mobile apps raise concerns about safety, privacy, health costs." retrieved 11 april 2014, from http://www.healthcentral.com/dietexercise/c/255251/162564/apps-concerns-privacy/ research, j. o. m. i. 2013. "analysis of the security and privacy requirements of cloud-based electronic health records systems." retrieved 03 april 2014, from http://www.jmir.org/2013/8/e186/ rodrigues j, diez i, abajo b. 2012. telemedicine and e-health services, policies, and applications: advancements and developments. hershey: medical information science reference. rui, z., and ling, l. 2010. "security models and requirements for healthcare application clouds," cloud computing (cloud), 2010 ieee 3rd international conference on, pp. 268-275. 16 australasian journal of information systems scott, richards & adhikari 2015, vol. 19, selected papers from acis security risks & safety on mobile health apps schulke, d. f. 2013. "the regulatory arms race: mobile-health applications and agency posturing," boston university law review (93:5). scott, k. m., gome, g. a., richards, d., and caldwell, p. h. 2015. "how trustworthy are apps for maternal and child health?," health and technology (4:4), pp. 329-336. senft, d. j. 2013. "mobile devices: technology aid--security risk," geriatric nursing (new york, n.y.) (34:2), pp. 149-150. supplemental health care. 2014. "health care on the go." retrieved 01 may 2014, from http://www.supplementalhealthcare.com/blog/2013/healthcare-go-pros-and-consmobile-health-apps tamrat, t., and kachnowski, s. 2012. "special delivery: an analysis of mhealth in maternal and newborn health programs and their outcomes around the world," maternal and child health journal (16:5), pp. 1092-1101. tech-target. 2009-2014. "definition: mhealth." retrieved 31march 2014, from http://searchhealthit.techtarget.com/definition/mhealth> technologies, a. 2014. "mhealth its advantages and disadvantages." retrieved 16 april 2014, from http://attunelive.com/blog/mhealth-its-advantages-disadvantages/ ucsf. 2012. "self-tracking may become key element of personalized medicine." retrieved 01 may 2014, from http://www.ucsf.edu/news/2012/10/12913/self-tracking-maybecome-key-element-personalized-medicine verasoni. 2012. "mobile health applications: 2012 study ", from http://verasoni.com/2012/08/mobile-health-applications-2012-study/ vodafone group. 2013. "evaluating mhealth adoption barriers: privacy and regulation." retrieved 01 april 2014, from http://mhealthregulatorycoalition.org/wpcontent/uploads/2013/01/vodafoneglobalenterprise-mhealth-insights-guideevaluating-mhealth-adoption-privacy-and-regulation.pdf wang, j., zhang, z., xu, k., yin, y., and guo, p. 2013. "a research on security and privacy issues for patient related data in medical organization system," international journal of security & its applications (7:4). webmd llc. 1994 2014. "public health in the smartphone era." retrieved 15 april 2014, from http://www.medscape.com/viewarticle/776278_3 wellapps. 2009 2011. "well apps." retrieved 12 may 2014, from http://www.wellapps.com/ wikipedia. 2014. "mobile technology." retrieved 20 april 2014, from http://en.wikipedia.org/wiki/mobile_technology an earlier version of this paper was presented at the australasian conference on information systems (acis) 2014 in auckland, new zealand. copyright: © 2015 scott, richards & adhikari. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. 17 http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems scott, richards & adhikari 2015, vol. 19, selected papers from acis security risks & safety on mobile health apps 18 1 introduction 2 literature review 2.1 security and privacy issues 2.2 poorly protected consumer data 2.3 data security breaches 2.4 lack of app standards and guidelines 2.5 mhealth cloud storage 2.6 mobile device security measures 3 methodology 4 results of selection process of 20 apps 4.1 apple stores 4.2 google store apps 5 results: comparative analysis 6 discussion 7 future trends for mobile health apps 8 conclusion microsoft word ajis_18(3)_v10.docx australasian journal of information systems volume 18 number 3 2014 137 conceptualising how smes incorporate green content in their websites craig m. parker deakin university craig.parker@deakin.edu.au emilia bellucci deakin university emilia.bellucci@deakin.edu.au luba torlina monash university lioubov.torlina@monash.edu bardo fraunholz deakin university bardo.fraunholz@deakin.edu.au ambika zutshi deakin university ambika.zutshi@deakin.edu.au abstract this paper presents a framework on how small and medium enterprises (smes) can proactively incorporate content relating to their ecological responsibility (or green) activities in their websites. sme studies offer limited guidance on, and conceptualisation of, how organisations can incorporate different types of content into their website designs. this paper addresses this problem by presenting the results of an exploratory, qualitative content analysis of australian sme websites where emergent themes are interpreted using framing and legitimacy theories. the themes are grouped under three dimensions (location, presentation, and specificity) to form a theoretically-informed framework. the paper outlines how scholars can use the framework to develop models and carry out evaluations regarding how smes embed green content, and potentially other specific content types, in their websites. it also summarises how the framework can assist smes (or website developers serving them) make informed decisions regarding framing their websites as green, or de-emphasising this content, by paying attention to its location (e.g. homepage, navigation bars) and presentation (e.g. how paragraphs, images, etc are used) within webpages. the legitimacy or credibility of the green content can be enhanced using different types of specificity (e.g. statistics, detail of processes and actions, and third-party substantiation). keywords: small and medium enterprises (smes), ecological responsibility, websites, content analysis, framework, green information systems introduction small and medium enterprises (smes) increasingly use websites to gain marketing advantage (e.g. sensis, 2013), but their websites are often low quality due to their limited e-commerce/ marketing skills (burgess, 2008; fisher et al., 2007; sanders & galloway, 2013). this has led scholars to develop frameworks for evaluating sme website content/designs which consider if particular content (e.g. australasian journal of information systems volume 18 number 3 2014 138 policies, history) and design features (e.g. multimedia) are included (e.g. bingley et al., 2010; burgess, 2011; sanders & galloway, 2013). however, very few studies look at how selected content is embedded in sme websites. nevertheless, research in adjacent knowledge domains suggests that examining the effectiveness of sme websites requires considering not just if a specific type of content is included, but also how. this is important because, for example, variation in how privacy content is embedded can effect consumer behaviour, online sales (tsai et al., 2011) and trust (resnick & montania, 2003). our study thus addresses the above mentioned gap in the literature and suggests a framework which can assist smes to decide how to embed selected content in their websites, and which enables scholars to explore this in future evaluations of sme websites. the paper focuses on “green content” which some smes embed in their websites to promote “green activities” resulting in reduced negative environmental impact (parker & scheepers, 2012), such as recycling to reduce landfill and reducing energy to lower carbon emissions (parker et al., 2009). green content is an interesting case for identifying how specific information can be embedded because smes use information variously to promote green activities, attract clients and improve their reputation (williams & schaefer, 2013). for instance, for some of these smes being green is core business (spence et al., 2011) and they may include green content prominently. for others being green is not core business and they thus find it hard to design websites to appeal both to green customers and those who are not interested in their green activities (gueben & skerratt, 2007), so they may need to embed green content subtly. this study makes three contributions to is knowledge when answering the following research question: “what theory helps conceptualise how smes embed green content in their websites?” first, it fills a gap in knowledge concerning how smes can incorporate a specific content type by presenting a framework for exploring how smes embed green content in their websites, and suggests how it can be used potentially to examine other content types. second, the paper identifies, and then reports on using, framing and legitimacy theories to show how they inform the framework. this is important because is studies of sme websites do not provide insights into theory which could be used for such a framework. third, this study addresses the call by some scholars for the is field to examine the nexus of is, organisations and the environment (e.g. melville, 2010), including for smes (coffey et al., 2013; elliot, 2009), known as “green is” (watson et al., 2010). this includes how smes can use websites to communicate green activities (parker et al., 2010). the rest of the paper is structured as follows. first, it reviews sme-specific studies (because they are different to large firms) relating to their websites and/or green activities to argue the need for a framework to conceptualise how smes embed green content in their websites. the paper then summarises a three-staged study resulting in a theoretically-informed framework: 1) an inductive, qualitative content analysis of australian sme websites which led to identifying themes relating to how they embedded green content; 2) a search of literature on large firms to identify theories to help conceptualise and interpret the themes; and 3) a reinterpretation of the stage 1 data using the stage 2 theories which lead to new insights and the framework. the paper concludes by presenting the framework, summarising the study’s limitations, and outlining the potential for scholars to use the framework for other content types and to inform website design guidelines which can be developed for smes. literature review is researchers have undertaken sme-specific research for over a decade (keller & coulthard, 2013) to examine such issues as website adoption barriers/drivers and benefits (see reviews by parker & castleman, 2007; wymer & regan, 2011). sme-specific research is required because they are different to large firms, such as having limited financial/personnel resources (jones et al., 2014), and hence findings from studies of large firms may not always apply to smes (macgregor & kartiwi, 2010). australasian journal of information systems volume 18 number 3 2014 139 many smes, despite their limitations, have found websites can promote them and their products/services, and help them to gain new customers and sell online (e.g. jones et al., 2014; saffu et al., 2012; sensis, 2013). is scholars have conducted content analyses of sme websites to determine if they include specific content (e.g. firm history, policies, customer reviews and product details) and design features (e.g. transactions, interactivity and multimedia) (alonso mendo & fitzgerald, 2005; andreopoulou et al., 2009; bingley et al., 2010; burgess, 2011; neilson et al., 2010). some studies have created stage models or frameworks based on this work (alonso mendo & fitzgerald, 2005; burgess et al., 2011; cooper & burgess, 2000; zhu et al., 2009) to compare industries, evaluate website maturity within an industry and to compare website changes over time. other is scholars have reported that sme websites are often low quality because smes have limited resources and expertise with e-commerce and marketing (burgess, 2008; fisher et al., 2007; sanders & galloway, 2013). this highlights the need for smespecific studies which leads to guidance on how smes can design their websites in an effective manner within their constraints. this has led to research, for instance, on process models to help smes decide what website content/features to include (burgess, 2008). few is studies, however, focus on a specific content type and explore how it is embedded in sme websites. this is important because, for instance, experimental studies of consumers suggest that how privacy content is embedded in websites (e.g. top versus bottom of homepages) influences online sales (tsai et al., 2011) and trust (resnick & montania, 2003). this calls for a framework relating to how specific content types can be embedded in sme websites, because such design decisions may affect whether visitors find the content and how they act upon it. only two studies (dincer & dincer, 2010; parker et al., 2010) had findings relevant to how a specific content type (green/social responsibility content and green content respectively) was embedded in sme websites: where it was in website hierarchies; whether webpages were dedicated to the content; and if navigation titles related to the content. however, neither study developed a framework relating to how the content was embedded. we noted earlier that we selected green content as the specific content type for this paper because some smes are increasingly engaging in green activities (e.g. revell et al., 2010; williams & schaefer, 2013) and proactively promoting their efforts online (dincer & dincer, 2010; nielsen & thomsen, 2009) to gain new customers and improve their reputation (gueben & skerratt, 2007). the green activities of smes include reducing waste and carbon emissions, procuring green suppliers and/or from green suppliers, requiring staff to engage in recycling, offering products/services which enable customers to be green (aragon-correa et al., 2008; hofmann et al., 2012; revell et al., 2010; williams & schaefer, 2013) and reducing the environmental impact of their is/it (coffey et al., 2013; dick & burns, 2011; elliot, 2009). in this paper we focus on text, images and other semiotic forms which are used on sme websites to convey these green activities, and on identifying themes relating to how this green content was embedded in sme websites to inform the development of our framework. this paper contributes to the growing green is field, as mentioned earlier. green is research explores: 1) how organisations can reduce the negative impact of their is infrastructure; and 2) how organisations can use is to reduce their (and/or encourage their stakeholders such as customers to reduce their) impact (lei & ngai, 2013; melville, 2010; watson et al., 2010). while most green is research has focused on large firms, sme studies in the first (e.g. dick & burns, 2011; elliot, 2009) and second (e.g. coffey et al., 2013; nowduri & al-dossary, 2012) areas of green is are now more common. this paper falls into the second category because it explores how smes can use websites to communicate how they reduce and/or how their customers can reduce their environmental impact (parker et al., 2010). we were especially interested in the potential of theory to inform our framework and help to conceptualise how green content can be embedded in sme websites. this was important because some authors (alonso-mendo et al., 2009; alonso mendo & fitzgerald, 2005) used theory to underpin their australasian journal of information systems volume 18 number 3 2014 140 framework for evaluating how sme website designs change over time and found theory resulted in richer interpretations of findings. our review of sme studies did not reveal theories suitable for our purpose. this led to the research question: “what theory helps conceptualise how smes embed green content in their websites?” research approach the above-mentioned research question was answered using a three-stage research approach. stage 1: the first stage involved analysing relevant sme literature and collecting/analysing empirical data. the is literature was reviewed (see above) for insights relating to embedding specific content on sme websites. data collection involved selecting sme websites which proactively promoted green activities and incorporated green content in different ways, but manageable in number to facilitate an in-depth semiotic analysis to identify emergent themes. this is similar to the approach used by bingley et al. (2010) to develop their classification scheme for analysing what types of web 2.0 features were used on sme websites. combining our findings from sme specific literature and data analysis enabled us to identify themes regarding the different ways green content can be embedded in sme websites. stage 2: our stage 1 literature review did not reveal theories suitable for conceptualising how sme websites embed specific content. this meant we had to collect additional literature to identify suitable theory to "foster conceptualisation" (strauss & corbin, 1998) of the themes from stage 1. stage 2 therefore involved reviewing studies of large firms’ websites to identify theory they used to conceptualise how specific content was embedded, and to underpin richer interpretation of our stage 1 themes. framing and legitimacy theories have been used in studies of large firms, so we decided to employ them for our research on smes. stage 3: the stage 1 data and emergent themes were then reinterpreted using theory identified in stage 2. more specifically, further analysis of the themes identified during the first stage, through the lenses of legitimacy and framing theories, revealed new insights contributing to the final framework. this paper shows how theory can inform a framework which conceptualises how green content can be embedded in sme websites. there are, arguably, elements of a grounded theory approach in our three-stage research method. nonetheless, we do not consider our qualitative study as an exemplar of grounded theory because our study involves conceptions based on the work of others (goulding, 2005) and because we are not building theory from empirical data alone. the next sections explain the three stages in more detail, with particular emphasis on stage 3. stage 1: data collection and preliminary analysis sampling and data collection stage 1 required a sufficient number of sme websites to enable in-depth semiotic analysis leading to emergent themes regarding the different ways smes can proactively embed green content. we chose australian websites because adoption rates in australia had reached levels where many smes own and use websites for promotion (sensis, 2013). australian sme websites were identified from the yahoo small business directory (2010) database in which smes can promote themselves proactively by selecting their industry sector and entering details such as product/service descriptions and an optional hyperlink to their website. we focused on smes in the information media and telecommunications (imt) sector, as labelled in the yahoo directory, because most had websites and there was evidence (elliot, 2009; parker et al., 2010) some australian imt smes engaged in green activities. the imt sector included internet service australasian journal of information systems volume 18 number 3 2014 141 providers, website and web portal developers, internet and paper-based printers/publishers, (internet) broadcasters and motion picture/sound recorders. we identified all the imt sme websites which mentioned green activities typical of smes as found and listed in detail by parker et al. (2013). for each website we archived the homepage and webpages including green content because websites are highly dynamic (alonso mendo & fitzgerald, 2005). this ensured we could analyse the data in its original context without concern the website would disappear or its content would change. two authors independently analysed all the identified websites and, after reaching consensus over embedded content about green activities, there were 12 imt sme websites. table 1 summarises the products/services of each sme and the pseudonyms used to identify each website. the use of 12 australian imt sme websites has limitations such as emergent themes being more indicative of smes which proactively promote themselves and have more website design experience than those in other sectors, and omitting themes due to the small sample. nevertheless, the 12 analysed websites were sufficient for answering our research question. first, sampling required identifying websites which embedded green content differently, and the findings revealed the same (e.g. some had prominent green content while others embedded it subtly). second, 12 selected websites provided rich material from which themes emerged and were then reinterpreted in order to answer the research question. answering the question necessitated in-depth semiotic analysis of about 40 webpages contained in those websites, including homepages and those stating green activities (and associated text and images). we completed analysis when no new information seemed to emerge from the research process, which indicated no need for further data collection (strauss and corbin, 1998). finally, smes in other sectors have been found to commission imt smes to develop their websites (carey, 2008), so the framework will indirectly help such non-imt smes. pseudonym description of the products-services offered printmagazine published a magazine aimed at those in the vegan community. printproducts sold printed diaries and organisers via its website. webhost provided website hosting, domain name registration, and online tools (website builders) so companies could design their own websites. webdev designed websites in addition to the webhost services. epromo1, epromo2 provided graphics/logo design, email marketing and/or mobile phone texting marketing services in addition to many of the webdev services. epromoprint1, epromoprint2, epromoprint3 provided print marketing services in addition to many of the epromo1/epromo2 services. ecrm provided internet-based or client-run customer relationship management (crm) software so companies could, for instance, manage their clients, disseminate email newsletters, etc. teleco1, teleco2 provide information technology infrastructure table 1: sample of australian sme websites and their products-services initial data analysis an exploratory, inductive and iterative approach was used to analyse the webpages containing green content to identify emergent themes concerning how the content was embedded. we employed some quantitative content analysis (e.g. sentence and paragraph counts) but we mainly used qualitative content analysis which involved identifying themes and capturing the effects of narrative and context on the meaning of the content (beck et al., 2010). in this research, content analysis was used more broadly than solely for text analysis. analysis of semiotics, which is at the core of content analysis (beck et al., 2010), was employed to analyse, in addition to text, other elements such graphics and australasian journal of information systems volume 18 number 3 2014 142 photos (ramo, 2011). elements specific to website design (e.g. menus, interactive links, icons) are important for web-based communications, and these elements distinguish websites from other types of media such as reports (lodhia, 2012). these elements were included in our analysis as well. we analysed the archived content for all the websites using a cyclic process similar to the collaborative coding and reconciliation procedure used by other scholars (e.g. berente et al., 2011; du & vieira, 2012). the process started by two authors independently analysing the websites to gain a general sense of how the green content was embedded in the designs and to identify initial themes. themes were collated, differences in conclusions were discussed among all authors, and themes were consolidated. this approach was repeated with rounds of independent analysis by two authors (e.g. to look for detailed evidence of themes identified or consolidated after each round) followed by collation and discussions until we arrived at stable themes, based on consensus, grouped under three dimensions: 1) location of the content: emergent themes in this dimension related to green content being located on homepages and/or other webpages nested deeper in the website structures. these other webpages were either dedicated to green content or covered other topics but also included environment-related content to varying degrees. 2) presentation of the content: emergent presentation themes related to the use of logos, bullet points, paragraphs, hyperlinks and headings which emphasised or, in their absence, deemphasised the green content. 3) specificity of the content: we found that the websites varied in terms of how specific or detailed they were about the firm’s green activities. emergent themes in this dimension included the use of statistics, the provision of information to enable visitors to cross-check the accuracy of the content using third party sources, and the amount of detail provided about the firm’s green processes, actions and/or products-services. stage 2: theories identified from the literature on large firms stage 2 involved reviewing content analyses which examined how websites of large firms embedded specific content so that we could identify any respective use of theory. we found studies which had selected content relating to green activities (cho & roberts, 2010; hsieh, 2012; yu et al., 2011), green activities and social responsibility (du & vieira, 2012; herzig & godemann, 2011; lodhia, 2012), employment diversity (mestre, 2011), drug benefits versus risks (hicks et al., 2005), governance (jiang et al., 2009) and privacy (larose & rifon, 2006; park, 2011). most did not employ theories but those which did used legitimacy theory (cho & roberts, 2010; du & vieira, 2012) and framing theory (du & vieira, 2012). we therefore explored the literature on these theories further. we next present an overview of the two theories and why they had potential to help us conceptualise the dimensions and themes from stage 1. the later section on stage 3 explores this in more depth. framing theory originates from the communications field (chong & druckman, 2007) and has been used to examine how messages or ideas are presented or (de-)emphasised in mass media and to explore how these techniques influence the way in which receivers of the communication interpret the messages (chong & druckman, 2007; de vreese, 2005; matthes & kohring, 2008). framing mechanisms can include, among other things, presentation techniques which (de-)emphasise specific messages or ideas within media communications such as the use of headings, subheadings, logos, photos and photo captions (tankard, 2001). du and vieira (2012) identified two framing mechanisms which corresponded with our sme findings. the first was accessibility of green content from the homepage (location dimension), whereby the greater number of clicks it took to reach the green content the more the content was de-emphasised by the website. the second was the use of multimedia, which du and vieira (2012) stated australasian journal of information systems volume 18 number 3 2014 143 enhanced/emphasised the green content because it often drew more attention than text alone. in the next section we present, additionally, three location and four presentation framing mechanisms which were evident on sme websites. in this paper we also consider how some location and presentation themes can influence the interpretation of green content from a framing perspective. legitimacy theory posits that entities such as organisations carry out actions and/or wish to be seen in a way which they perceive that stakeholders (e.g. customers) will find desirable or appropriate (cho & roberts, 2010; du & vieira, 2012). these perceptions are driven by socially constructed contexts such as social norms, ideologies and regulations (du & vieira, 2012). effective legitimacy management by organisations is largely achieved through marketing tactics whereby they can present themselves in a positive manner and enable them to gain credibility (cho & roberts, 2010; du & vieira, 2012). organisations are often seen as more credible or legitimate if disinterested third-parties confirm the organisation’s claims compared to instances where firms only make the claims themselves. none of the legitimacy themes of cho and roberts (2010) were comparable to the emergent themes from our study. however, two legitimacy or communication tactics identified by du and vieira (2012) were similar to our sme website findings: presenting factual (statistical) statements about the firm’s green practices; and presenting third-party evidence (e.g. accreditations) about the firm’s green achievements. our results also included sme-specific examples of these two themes, as well as introduced an additional specificity theme which was not evident in the work of du and vieira (2012). stage 3: theme reinterpretation using framing and legitamcy theories stage 3 involved reinterpreting the themes from each dimension arising in stage 1 using framing and legitimacy theories identified during stage 2. this section reports the findings and these interpretations for each dimension in terms of framing theory (the location and presentation dimensions) or legitimacy theory (specificity dimension). the discussion of the findings suggests that the two theories can conceptualise the dimensions and associated themes which comprise this framework. for example, the next sections use the theories to explain the possible effects which many of the themes may have on website visitors, such as the ease of locating the content or how they may be interpreted. location of green content the first dimension grouped themes relating to the location of green content on various types of webpages. for this dimension we considered that webpages communicated green activities if they had any image, word or phrase relating to a green activity. we examine the nature of this content later when exploring the presentation and specificity dimensions. “clicks” from the homepage the first location theme related to the number of “clicks” from the homepage it took to reach green content and is a framing mechanism (du & vieira, 2012) because the content is deemphasised with increasing numbers of clicks. we found that all but three websites had accessible green content because it was located on the homepage, so that their website was framed as green. we explore the nature of the green content and the degree of green framing on the homepages of these nine websites later when examining the presentation dimension. the remaining three websites (ecrm, teleco1 and teleco2) required one click to a non-homepage before the green content was reached. this is shown in figure 1 which summarises the webpages which included green content and how they could be reached on the websites. the elements of figure 1 are explored in more detail in the next location sections. australasian journal of information systems volume 18 number 3 2014 144 figure 1: location of green content on sme websites dedicated versus subset webpages the second location theme concerned the type of non-homepages on which the content was located (see figure 1): dedicated webpages where all content on the webpage was green; and subset webpages which covered other topics but also embedded some green content. du and vieira (2012) did not make this distinction and therefore did not identify this as a framing mechanism, possibly because the websites of the large firms they studied only had dedicated webpages. we consider this to be a framing mechanism, at least for sme websites, because dedicated versus subset webpages vary the emphasis of green content. for example, if a visitor accesses a dedicated webpage (e.g. epromoprint3, webhost) they are more likely to notice the green content because it is all about the firm’s green activities. this can be contrasted with subset webpages (e.g. epromo1, teleco2) where visitors need to read the webpage before they find the green content. in the presentation dimension we consider framing mechanisms which can be used to emphasise green content on subset webpages. webpage framing using menu titles a third location theme related to the main-menu or sub-menu titles under which the dedicated and subset webpages were located within the website’s structure (see figure 1). du and vieira (2012) did not discuss how menu (or webpages) titles can be a framing mechanism in their own right. instead, they listed examples of titles but used the titles collectively to describe how many clicks the green content was from the homepage. we believe titles are a framing mechanism in their own right by examining the nature of the titles to give richer insights into how they frame webpages. this can include how the titles contextualise the webpages and influence the ease by which webpages can be found on the website. figure 1 suggests that menu titles can frame dedicated and subset webpages. all dedicated webpages were framed using environment-related terms and thus made the green content easier to find or more accessible. but the titles also contextualised the webpages. webhost’s title “green hosting” framed its dedicated webpage in a product-service context while other titles framed them in terms of the firm’s news (“green news”), identity (“we are carbon neutral”), policy or actions (“we plant trees”). similar framing was evident on the subset webpages. some had environment-related titles (epromoprint1, australasian journal of information systems volume 18 number 3 2014 145 epromoprint3, printproducts) for subset webpages and framed them in a product-service context. the other subset webpages did not have environment-related terms and framed the content in terms of external stakeholder relationships (e.g. epromo1), philosophy/policies (printmagazine, ecrm), news (webhost) or products-services (teleco2). in other words, the title of the webpage can contextualise how the webpages may be interpreted by the visitor: firm identity or principles; product-service offerings; news; actions; and stakeholder relationships. the lack of green titles also deemphasises the green content and could mean it is harder to locate. the main-menu titles under which dedicated webpages were located (see figure 1) may also influence how the webpages may be interpreted and how easily they would be found and accessed. all the dedicated webpages (except epromoprint3, webhost) were located two clicks from the homepage under an “about us” type main-menus. this “about us” framing implies the owner-managers may have seen green activities as part of their own, or their firm’s, values. this is interesting because this framing separates the product-service webpages from the green content so that visitors may not encounter the content if they only look for product-service information. this framing also deemphasises green content because it requires visitors to explore the webpages under this menu option before the content would be found, unless green content is also on the homepage. the ability of main-menu titles to frame subset webpages was also evident in the case of the three websites which did not have green content on homepages (epromo1, teleco1, teleco2) because it would potentially impact on whether visitors would find the content. for instance, if a visitor only wished to read about the firm’s products-services then out of these websites only teleco2’s green content would have been accessed because its green content was located on its virtual servers (service) webpage. for the other websites the content would only have been accessed if the visitor explored other menu options. environmental framing using navigation bars a fourth location theme related to whether and how sme websites incorporated green content into navigation bars. du and vieira (2012) implied that the menu titles listed were in the navigation bars of the large firms’ websites but they did not identify this as a framing mechanism. our findings suggest, at least in an sme context, that navigation bars should be considered a framing mechanism because those including green content in one of their navigation bars (all except ecrm, printmagazine, teleco1, teleco2, webdev) framed the entire website in a green context. this is because the navigation bars, and thus the green content, were visible on all webpages with those bars. websites which did not include green content in navigation bars, by contrast, de-emphasised the content because it was visible only by accessing specific webpages. however, the degree of emphasis for those including green content in their navigation bars depended on which bar(s). for example, epromo1 was the only website with green content in the top bar while some websites de-emphasised the content by locating it in the bottom bar (epromo2, epromoprint1, printproducts); often requiring scrolling down the webpage to be seen. presentation of green content the second dimension grouped emergent themes relating to the presentation of green content. our website analysis led to the categorisation of presentation approaches into three themes: textual forms (paragraphs and bullet points); emphasises in the textual forms such as section headings; and imagery such as photos. we examine each separately next. textual forms: paragraphs and bullet points table 2 summarises the first presentation theme which related to textual forms used to present green text: paragraphs entirely or partially including green content; and/or bullet lists entirely or partially including green actions, values/principles, commitments and/or product-service features. du and vieira australasian journal of information systems volume 18 number 3 2014 146 (2012) did not identify entirely or partially green paragraphs and bullet point lists as framing mechanisms. website paragraphs (all webpages) bullet point lists (all webpages) epromoprint2 green text comprised 7 paragraphs of 1-2 sentences each. listed 20 green actions and named one green award. epromo2 green text comprised 7 paragraphs of 1-3 sentences each. listed 9 green actions. listed 7 hyperlinks to information to help visitors wanting to be green. webhost green text comprised 18 paragraphs of 1-5 sentences each. lists 4 green actions in addition to its carbon neutral achievements. lists 3 features of its web hosting on the homepage with one green feature. epromoprint3 green text comprised 10 paragraphs of 2-5 sentences each. lists 7 features of 2 types of business cards with 2 green features each. epromoprint1 green text comprised 8 paragraphs of 1-4 sentences each. listed 3 green features of its recycled paper supplier or its products. webdev green text comprised 10 paragraphs of 1-4 sentences each. not applicable. teleco1 green text comprised 3 paragraphs of 1-2 sentences each. listed 4 green commitments of the firm. printmagazine green text comprised 1 sentence of a 2-sentence paragraph. listed 10 ethical guidelines and 4 were green practices. teleco2 green text comprised less than half a 2-sentence paragraph. listed 7 benefits of server consolidation and 1 was green related. ecrm green text comprised 1 paragraph of 3 short sentences. not applicable. printproducts green text comprised 1 sentence of 2-sentence product overviews. not applicable. epromo1 not applicable. listed 14 charities/not-for-profits supported and 2 were green initiatives. table 2: textual forms of green content on the sme websites the findings of this study (table 2) suggests paragraphs and bullet lists are framing mechanisms in sme websites because some had paragraphs and lists which only partially embedded green content and therefore deemphasised the content by making it easier or harder for visitors to identify or find. this did not apply to websites with dedicated webpages (epromoprint2, epromo2, epromoprint3, epromoprint1, webhost and webdev) because all the paragraphs or listed items were devoted to green content and was easier to find. this is apparent when considering websites (printmagazine, teleco2, printproducts, epromo1) which embedded the content within paragraphs and lists. the following example from teleco2 is typical and implies visitors would need to read the paragraph to find the content and that they may overlook the content due to the lack of emphasis: “we consolidate your physical servers and support your business with less cost in addition to helping reduce your organisations environmental footprint. your organisation can benefit from lower power, cooling and management costs.” the findings of this study also suggest that entirely green paragraphs on subset webpages could still be overlooked when they were embedded among many non-environmental paragraphs. for example, ecrm’s paragraph devoted to green content on its “ethical policy” webpage was one of ten paragraphs as part of their ethical principles. overall, embedding green sentences, bullet list items or paragraphs amongst non-green content may result in visitors overlooking it unless other framing mechanisms are used to draw a visitor’s attention to this content as explored next. australasian journal of information systems volume 18 number 3 2014 147 section headings, coloured/bold text and hyperlinks the second presentation theme involved using one or more textual approaches to emphasise the green content: section headings, coloured and/or bold text, and hyperlinks. du and vieira (2012) did not identify these as framing mechanisms but our findings suggest that with the six websites (epromoprint3, epromoprint1, epromoprint2, webhost, epromo1, epromo2) which used at least one of these approaches that these emphasised the green content. these can be contrasted with the remaining websites which only used paragraphs and lists (as reported in the previous section) without these framing mechanisms and which consequently deemphasised the green content. the findings of this study suggest that section headings using environment-related words or phrases (e.g. “green”, “recycled”, “eco”) can frame textual forms. some subset webpages had headings which could draw attention to green content. for example, webhost had an “earth hour” heading for one of its three news items on its “news and media centre” webpage. interestingly, it was more common that subset webpages did not use green headings. for example, ecrm used a heading implying a broader societal context (“to our community”) under which sat an entirely green paragraph, possibly assuming that visitors will logically connect it to environment issues, instead of a heading such as “to the environment”. other subset webpages did not use green headings at all. green headings were more common on dedicated webpages with four websites (webhost, epromo2, epromoprint3, epromoprint1) which used environment-related green headings to break up the green content. for example, epromoprint3 had headings such as “more about our green stock” and sub-headings such as “ecf (elemental chlorine free) & tcf (totally chlorine free)” which helped to frame or contextualise the green paragraphs. additionally, the results of this study suggest that bold formatting and colours can emphasise or frame the green content on subset and dedicated webpages. two websites (epromoprint1, epromo2) used these approaches on their subset webpages, perhaps to draw visitors’ attention to the entirely green paragraphs within the webpage. for example, epromo2’s subset webpage had two paragraphs whereby one was not green-related and the other was devoted to green activities. the following sentence within the latter paragraph had bolded, green coloured text “go green”: “more recently we and our hosting division [name] hosting made a commitment to ‘go green’ and at no additional cost to our clients”. four websites (epromoprint2, epromoprint3, epromo1, epromo2) had hyperlinks to internal webpages or external websites. hyperlinks were often bolded and/or different colours compared to the surrounding text and were therefore more visible. this finding suggests that hyperlinks can also be used as a framing mechanism. this was most notable on epromo1 which had a subset “charity/community support” webpage listing various community organisations the firm supported. one list item included the bold and red hyperlink “no carbon here” which made this text more visible than other list items on the webpage. imagery: logos, icons and photos the third presentation theme concerned the use of logos, icons and photos on nine of the 12 sme websites (see table 3). in this paper we define icons as computer-drawn images (as opposed to photos) without text and logos as computer-drawn images with text. du and vieira (2012) identified multimedia as a framing mechanism because it is often more effective than text at conveying complex information and at attracting attention because they argue it is often seen before text. they focused on large firms’ use of videos to convey green content which, at least in the smes we studied, did not apply. this raises the question of how other forms of imagery on sme websites can be used as framing mechanisms. australasian journal of information systems volume 18 number 3 2014 148 website(s) image type logo/icon description epromo2, webdev webhost third-party logo a white rectangle with green text. large “green power” and smaller “accredited renewable energy” beneath. some websites used a version with small green text “this business powered by” above “green power”. the logo was owned by the australian government and could be used when firms purchased renewable energy plan from a provider accredited by the government as renewable. epromoprint2, epromo1, webdev, webhost third-party logo a black square with large, thick “60” text which shows an image of the earth with body of the text, and small white “earth hour” text underneath. firms could use this logo if they participated in earth hour. epromo1 company logo a rectangle with dark green border and light green within, dark green “no carbon here” with a leaf on the left, and a dark green tick on right with “verified”. this logo belonged to another firm run by the owner-manager of epromo1. epromo2 third-party logo a white circle with image of the earth (blue and green) with a white tick through the middle and “climate positive” around it in blue. this logo was owned by climate positive and could be used by clients who purchased its carbon offsets. photo a black banner across the top of the webpage with an earth photo on the right and the climate positive logo in the lower right hand corner. on the right was the name of the firm in large white text, underneath “smarter greener [green text] online solutions” and under in larger text “no extra cost to you or the earth…” epromoprint1 company logo four horizontal circles (blue, pink, yellow, black) with two green leaves from the yellow circle and underneath green larger text “printed with” and smaller text “vegetable based inks” beneath this. company logo a green leaf-shaped image with “green news” in green against a white background. icon a frog-shaped image with a green head and legs, orange feet and eyes, a white stomach and tinges of blue. icon the recycling icon was similar to epromoprint2’s except the arrows were green against a white background. epromoprint2 icon three white, thick arrows in a triangular path in a clockwise direction against a green background (that is, the universal recycling symbol). third-party logo [program] partner: a rectangle with black/white text “[program] partner” against a white/black background. logo was owned by a state government. firms could use the logo in promotions if they participated in the government program. photo owner holding a certificate for completing the government program. epromoprint3 third-party logo carbon neutral: five fanned leaf-shaped images of different colours (green, black, yellow, magenta, blue) with carbon australasian journal of information systems volume 18 number 3 2014 149 neutral’s trademark logo on the green leaf. under this was “we plant trees”, then “carbon neutral green member”. company logo shows on one side of a large “=” three fanned rectangles (underneath “print business cards” and on the other a green tree image (underneath “plant trees”). icon two green fanned rectangles (representing business cards) and a green leaf-shaped image overlaying one of the rectangles. icon the recycling icon was similar to epromoprint2 except the arrows were purple (on a white background) to match the adjacent purple section heading. photo webpage 1: photo of a forest with large text “choose recycled” and above this in smaller text “help save trees”. webpage 2: photo of a hand holding a green plant and next to this “we plant trees” and underneath “sustainable printing”. print-magazine photo homepage included an image of the magazine cover with the firm’s name with the by-line “animal free food and living”. printproducts photo a webpage for one diary had a photo of the cover showing a lake, hills and a tree overlayed with a listing diary features including “enviro – contents printed on 65% pcw recycled paper”. webdev company logo white hollowed star with “environmentally friendly” in white and located underneath. the image was enclosed in a box with a background colour of green. webhost company logo white text against a green circular background. the largest text was the firm’s initials “dp” in the centre (a small leaf at the bottom of the “p”) and, circling these initials, was “eco host powered by green hosting”. the website states this logo could be embedded on their clients’ websites hosted by webhost. table 3: summary of logos and icons used on sme websites in the case of three websites with dedicated webpages (epromoprint1, epromoprint3, epromo2) the images emphasised and complemented the text in a similar manner as for the subset webpages. however, some of the dedicated webpages differed because they also included third-party logos (epromo2, epromoprint2, webdev, webhost) and, in one case, a photo of the owner-manager holding a certificate relating to the firm’s green activities (epromoprint2). the next section on the specificity dimension considers the impact such images may have on the perception of credibility of the websites and their green claims. specificity of green content the third dimension comprised of three emergent themes relating to website design approaches conveying or communicating varying degrees of legitimacy based on the extent of specificity of the green content. in line with legitimacy theory, the focus of these themes is on communication or legitimacy tactics firms can use on their websites to increase their credibility (du & vieira, 2012). in other words, they concern how the websites describe the firm’s green qualities of their processes, actions or products-services. australasian journal of information systems volume 18 number 3 2014 150 statistics relating to the firm’s green activities the first theme related to factual information in the form of statistics on the sme websites. this is consistent with the finding by du and vieira (2012) that legitimacy tactics on large firm websites were statements (especially statistics) on the amount of resources the firm devotes, or facts on the impact of their practices. our findings suggest that on sme websites these statements of statistics can take various forms. the first type of statistic related to the firm’s environmental impact reduction as found by du and vieira (2012). these offer a greater sense of legitimacy compared to a statement that a firm has reduced its impact without statistics because it implicitly or explicitly states the firm is measuring their impact reductions. epromo2 was the only website with such a statistic: “we've implemented strategies to reduce our energy consumption in our offices that include turning off equipment and lighting not required and only leaving on critical equipment out of business hours. we are measuring the results of this and have seen more than a 25% reduction in our weekly electricity consumption.” this was an interesting finding because sme-specific research suggests they would have difficulty providing such statistics because smes often do not measure green outcomes (clarke-sather et al., 2011) or conduct audits (brammer et al., 2012; hofmann et al., 2012). of even greater surprise was that with epromoprint2 we discovered that the website of the government green program which the firm completed stated the percentage and quantity reductions made by epromoprint2 in energy, greenhouse gases and waste. we were surprised the epromoprint2 website itself did not include these statistics and this thus perhaps emphasises the notion that some smes may have, but choose not to communicate, these statistics. alternatively, the website may not have been updated to include this information. the second type of statistic which was more common on the sme websites was citing statistics about the third-party products they used from suppliers. for example, some cited statistics regarding the supplier’s products such as stating they use “100% recycled paper” (printmagazine, epromoprint1, epromoprint2, epromoprint3) or their recycled paper was made from “100% post consumer waste” (epromoprint1). epromoprint3 included other statistics about its supplier’s products: “during the process of making this recycled stock 92% of the energy used was renewable”. again, these statistics imply a greater level of legitimacy than similar statements without statistics because they imply measurement or specific details of “greenness” such as the proportion of recycled waste in recycled paper. the final type of measurable numbers on two websites (epromoprint3, webhost) was the inclusion of third-party statistics relating to the impact of their firm’s industry on the environment or the benefits of national environmental initiatives. for example, webhost cited supporting quantifiable evidence from third party sources to substantiate the impact of their industry on the environment and the total carbon emissions businesses in their industry generate. epromoprint3 cited statistics about the emission reductions which can be achieved from planting trees to justify why they planted a tree for every business card order. these examples highlight that smes can provide a range of different types of statistics which may convey legitimacy in addition to the more difficult task of measuring their reduced environmental impact. the next section outlines another form of factual information. specifics of green qualities of the firm’s processes, actions and/or products-services the second theme concerned varying degrees of detail provided on the sme websites about the green qualities of the firm’s products-services, processes and/or actions. du and vieira (2012) did not identify australasian journal of information systems volume 18 number 3 2014 151 this theme as a legitimacy tactic but our findings, especially when comparing the different levels of detail provided via sme websites, suggests this is a relevant legitimacy theme. the first example of this specificity theme related to the amount of detail provided by nine websites which mentioned green credentials of the firm’s products-services. of the nine websites, three (epromo1, printmagazine, teleco2) provided no/little detail on how or why the products-services reduced environmental impact. two websites provided some detail such as general statements of policies/practices which ensure their services reduce impact (teleco1) or the percentage of postconsumer waste comprising the paper in one of its diaries (printproducts). four websites provided more comprehensive details of how they achieved carbon neutrality for their web hosting services (epromo2, webhost) or the credentials of the recycled paper used in their print products (epromoprint3 and epromoprint1). the latter examples are likely to be seen as more credible because the detail provides specifics on why and how the products-services reduce environmental impact, although it is still up to visitors to decide if they believe this detail. a second example related to variations in how the websites described the firm’s green processes. for example, epromo1 had a logo stating “no carbon here – verified” but did not provide any information about how this was achieved. this can be contrasted with webhost which provided specific detail about the process by which it made its website hosting service carbon neutral, adding a sense of legitimacy to the firm’s claims: “[webhost] has committed to conducting quarterly reviews of its carbon footprint, estimating the emissions generated for every aspect of its business including the daily operation and maintenance of hosting servers, routers and air conditioners. following each review the company will purchase carbon credits to cover one hundred per cent of its estimated emissions. this ensures that whatever power [webhost] uses, an equivalent amount is fed back into the electricity grid from wind or solar power, rather than non-sustainable means.” a third example related to the detail provided about the firm’s green actions. for instance, ecrm merely stated it reduces paper. some websites by contrast, as exemplified by epromoprint2, provided details of how they reduced paper such as: “utilizing old sheets of paper for writing notes instead of purchasing writing pads; reduction in printing of emails, by saving straight to the clients file; proofs, proposals and invoices are delivered electronically saving on paper, ink cartridges, postage and envelopes; …”. this extra detail adds a greater sense of legitimacy because the basis of the firm’s claim that it reduces paper is made clearer with the specific examples provided. overall, these findings suggest that a communication tactic which smes can use to increase the legitimacy or credibility of their green claims, in the absence of statistics, is to provide more detail about what they do, how they to it and why this means they are reducing their environmental impact. this detail does not necessarily prove, however, the firm employed these initiatives. this is where thirdparty substantiation might help, as explored next. third-party evidence substantiating the firm’s green claims the third theme related to the inclusion of details about third-parties on sme websites which can substantiate the green claims made by the firm. this theme is consistent with du and vieira (2012) who found in their study of large firm websites that a legitimacy tactic used by some firms was to provide details about third-party awards and certifications. our findings suggest that sme websites can use various types of third-party substantiation. one example noted earlier was using third-party logos (epromo2, epromoprint2, webdev, webhost) to substantiate, for instance, that firms’ use of renewable energy or participation in a program was legitimate. the perception of credibility depends, however, on whether visitors recognise and trust the logos (see echegaray, 2014) and/or are prepared to click the logos to hyperlink to an external website australasian journal of information systems volume 18 number 3 2014 152 to learn more. it also depends on whether the third-parties police the logos and/or list firms on their own website which use the products-services or have participated so that visitors have information to verify the firm’s claims. for example, the website of the government green program in which epromoprint2 participated included this sme in the list of participants and verified it was a certified “partner”. other third-parties such as earth hour did not provide a list of all firms which registered as participants, perhaps because participation cannot be verified and due to the numbers of firms taking part globally. some websites supplemented third-party logos by providing additional specificity about the third-party so that visitors could verify the firm’s green claims. for example, epromoprint2 provided a long description of the government program they participated in including: “[the program] assists businesses to identify efficiencies in waste, water and energy for financial and environmental benefits. [the program] has allowed us to record and register our improved environmental practise in the office…”. as noted earlier, the owner also had a photo of herself holding her certificate from the program. a second example was epromo2 which stated the name of the renewable energy plan they use: “our offices and [name] data centre are now using [plan name], a greenpower accredited option provided by [supplier name]”. the latter example is worth noting because, while a visitor cannot verify if the sme is actually using the plan, it does show that a few specific words can be sufficient for a visitor to verify on the provider’s website, if they wish, that the plan uses renewable energy. a third, somewhat interesting finding, was from epromoprint1 because it provided details about thirdparty verification of the green credentials of their supplier’s recycled paper: “[supplier] operate a closed loop energy system, meaning all by-products and wastes are reused as fuel for the paper mill. [supplier] take pride in the fact that no waste leaves their premises. remaining low levels of co2 emissions, caused during manufacture, have been offset through the independently audited and government certified organisation [climate company].” this finding is interesting because it suggests that a legitimacy tactic smes can use is to include information from its supplier on their own website. as with the previous example from epromo2, if supplier names are provided visitors can verify the accuracy of the details. a final example was an award for green activities which was listed among various other awards displayed on epromoprint2 website. these awards give a greater sense of legitimacy because they imply that the firm’s claims have been verified by third-party judges. visitors could, if they wish, check the website of the third-party which makes the award to verify that the sme is a recipient and to determine how the award is judged, for example whether the award was made on the basis of sme self-reporting or verifications. conclusion contribution to knowledge table 4 summarises the paper and its contribution to knowledge. the first two columns present the framework (comprising three dimensions—location, presentation and specificity—and their components). they also answer the research question “what theory helps conceptualise how smes embed green content in their websites?” by summarising the framework components based on framing and legitimacy theories. framing theory suggests the location and presentation components can be used to emphasise green content or frame the website as green. legitimacy theory implies the components of the specificity dimension can be used as tactics to increase the credibility of green content by including, for instance, statistics and third-party substantiation relating to the firm’s green activities. the third column highlights the main contribution to knowledge by showing the paper confirms australasian journal of information systems volume 18 number 3 2014 153 (indicated by “ ”) or extends (indicated by “+”) the use of framing and legitimacy theories by du and vieira (2012) to conceptualise new themes relating to how sme websites can embed green content. we noted earlier that few studies have investigated how specific content types are incorporated into sme websites in general, and this paper has addressed this gap in knowledge in the context of green content. the fourth column summarises the findings of this study using the framework. location framing theory findings summary using the framework clicks from the homepage the more clicks needed from homepages to reach the content, the less a website will be framed by the content (du & vieira, 2012) some sme websites were framed green by locating green content on the homepage (zero clicks). others deemphasised it by locating it one or two clicks from the homepage. dedicated versus subset webpages the more webpages are dedicated to the content, the more a website will be framed by the content + some sme websites were framed green by dedicating 1+ webpages to green content. others located green content on webpages with other content and thus deemphasised it. titles of subset or dedicated webpages the more dedicated and subset webpages which have titles relating to the content, the more a website will be framed by the content + green titles framed all dedicated and some subset webpages as green content. most subset webpages had non-green titles which deemphasised green content. titles contextualised or framed webpages (e.g. as product-service, philosophy, etc). many sme dedicated and subset webpages which were two clicks from the homepage were deemphasised by being located or framed under “about us” main-menus. navigation bars the more (top) navigation bars include links to the content, the more a website will be framed by the content + some sme websites were framed green by locating (links to) green content in navigation bars so it was accessible from webpages using these bars. the (links to) green content were deemphasised on bottom bars or emphasised on top bars. presentation framing theory findings summary using the framework lists/ paragraphs entirely or partially devoted to the content the more subset webpages use lists/ paragraphs devoted to the content, the more a website will be framed by the content + some sme subset webpages were framed green to some extent with entirely green paragraphs/bullet lists. others deemphasised green content by embedding it in paragraphs/bullet lists on other topics. section headings the more subset webpages use headings devoted to the content, the more a website will be framed by the content + some sme subset webpages were framed green to some extent with green headings. others deemphasised green content by not using green headings. australasian journal of information systems volume 18 number 3 2014 154 coloured/ bold text the more subset webpages bold/colour the content, the more a website will be framed by the content + some sme subset webpages were framed green to some extent by bolding/colouring green words/ phrases to stand out in paragraphs. others deemphasised green content by not using these types of text formatting. hyperlinks the more subset webpages have hyperlinks (with bold and/or colour) relating to the content, the more a website will be framed by the content + some sme subset webpages were framed green to some extent by using hyperlinks on green words/phrases which stood out by being bold/ coloured. others deemphasised green content by not using hyperlinks to green content. multimedia the more subset webpages use images/logos relating to the content (du & vieira, 2012) which are large and located toward the top of the webpage, the more a website will be framed by the content some sme homepages were framed green using images (logos) only and some dedicated/subset webpages were framed green using images (logos, icons, photos); often complementing the text. framing was increased or decreased depending on image size/location relative to non-green content on subset webpages. others deemphasised green content by not using images at all. specificity legitimacy theory findings summary using the framework statistics statistics are legitimising tactics suggesting credibility by quantifying claims relating to the content (du & vieira, 2012) some sme websites used statistics on reductions of their practices on the environment, green third-party products or their industry’s green impact as legitimacy tactics, giving a sense of credibility compared to websites not using them. detail on firm’s productsservices, actions or processes details are legitimising tactics suggesting credibility by explaining or qualifying how/why claims relating to the content are justifiable + some sme websites stated details of the green qualities of their products-services, processes and/or actions as legitimacy tactics which gave a greater sense of credibility compared to websites which stated only brief/vague details. third-party substantiation third-party substantiations are legitimising tactics suggesting disinterested parties have verified claims relating to the content (du & vieira, 2012) some sme websites used third-party logos, descriptions of third-party programs verifying their green credentials, details of third-party products-services used (including third-party verification of these supplier products-services) and/or awards as legitimacy tactics. these gave a greater sense of credibility compared to websites not using these tactics. table 4: framework: how sme websites embed green content ( confirmed, + new theme) while this paper focused on green content, we anticipate the framework could be used by researchers to examine how sme websites embed other optional content types such as: 1) social responsibility content (e.g. charity donations and/or free products-services to disadvantaged groups); and 2) content relating to the sme’s identity (e.g. size, history, staff details, family/sole-proprietor ownership). this is why the third column of table 4 states “the content” generically rather than “green content”. australasian journal of information systems volume 18 number 3 2014 155 scholars can use the location and presentation dimensions, informed by framing theory, to evaluate the degree to which smes frame their website using a selected content type and to develop a continuum model by which the content is embedded. using the example of sme identity content, at one end of the continuum some owner-managers may see competitive advantage in promoting a small (family) business identity. our framework could be used to define strong framing of this identity, such as displaying content relating to this identity prominently on the homepage, in navigation titles and in dedicated webpages. at the other extreme some may wish to hide their size. our framework could be used to define subtle or limited framing of this identity, such as embedding such identity-related content in subset webpages, multiple clicks from the homepage, and/or using presentation techniques which de-emphasise it. other sme websites will be between these extremes depending on which combinations of location and presentation components are used, and how they are used, to (de-)emphasise the small (family) business identity. our framework could therefore be used to identify patterns of location/presentation dimension combinations for each “stage” in the continuum regarding how this content is embedded in sme websites. researchers can use the specificity dimension, informed by legitimacy theory, to evaluate the credibility of the selected content type based on how it is described in sme websites. this could include using this dimension’s components to develop a continuum model of low through to highly credible forms of, for instance, social responsibility content. the framework could be used by scholars for theoretically-informed evaluation of sme websites over time to see how embedding of specific content changes. for example, green is researchers could determine whether a sample of sme websites: (de-)emphasise green content over time (based on the location and presentation dimensions) and in what ways; and increases/decreases the credibility of the green content (based on the specificity dimension). the framework could also be used to determine the basis on which sme websites in a particular industry sector evolve to become similar (or quite disparate) with regards to how they embed content on their green activities in their websites. the framework can potentially be used by scholars to develop theoretically informed interview and survey instruments to compare what sme owner-managers are trying to achieve when embedding specific content in their websites, with what our framework suggests is the case. finally, researchers could use the framework to develop theoretically informed experiments with (sme) customers. for example, they could design different websites, based on combinations of the three dimensions, and survey consumers about the degree to which each design influenced their ability to find a specifically content type (e.g. social responsibility content) and/or their perception of the credibility of the content. contribution to practice the presented framework suggests that proactive smes wishing to gain marketing advantage from their website should seriously consider framing and legitimacy design tactics. the concept of framing suggests these sme should identify the different types of content for the website and then, in order of importance, how prominent each type should be. the framing design tactics can then be used to prioritise the framing of each type of content appearing on the homepage, and then subsequently for what webpage titles should appear in the top navigation bar and so on. for example, if owner-managers believe all customers will like their green activities, social responsibility or small/family firm identity, the framing concept implies this content should not be delegated to the “about us” page but instead should be prominent on the homepage. the framing design tactics in the framework give ideas on how to achieve this. importantly, the framework provides options for decreasing prominence of specific content, not just increasing it. for example, smes wishing to balance the needs of non-green and green customers could consider segregating green content from product-service webpages, and reducing the green framing (e.g. using partial green paragraph/bullet lists) to suit non-green customers. finally, the australasian journal of information systems volume 18 number 3 2014 156 legitimacy concept emphasises the need for specific details and preferably third-party substantiation of this content. the framework also has potential policy implications. for example, we used the framework as a starting point for developing a new tool which governments, legal practitioners and industry associations can use to measure the quality of sme communication of their green activities online (see parker et al. 2014). this includes governments using the tool to measure or evaluate the success of their interventions aimed at reducing the degree to which smes (due to their lack of marketing and legal skills) inadvertently mislead customers when promoting their green activities on their websites. misleading promotions often result when firms, including smes, fail to provide evidence (e.g. statistics, third-party substantiation, detailed information about their green activities) of their marketing claims (accc, 2011). limitations of the study and future research the study, leading to the framework in table 4, has a number of limitations which can be addressed in future research. first, we concentrated on sme websites from the imt sector, which were typically website developers, so the framework may be more indicative of imt smes rather than sme with less website design experience. while the 12 websites identified through our research process were sufficient for the purposes of this study, future research might explore sme websites from other sectors to evaluate our framework. second, the framework may reflect how green content is embedded in sme websites more so than other content types. future research can explore how smes from other industry sectors and countries incorporate green, and other specific types of content, into their website designs. this work may identify further themes (or dimensions) which can be added to our framework and interpreted using framing and legitimacy theories. references accc (2011) green marketing and the australian consumer law, australian competition and consumer commission, canberra. alonso-mendo, f., fitzgerald, g. & frias-martinez, e. (2009) "understanding web site redesigns in smalland medium-sized enterprises (smes): a u.k.-based study on the applicability of ecommerce stage models", european journal of information systems, vol 18 no 3, pp 264-279. alonso mendo, f. & fitzgerald, g. (2005) "a multidimensional framework for sme e-business progression", journal of enterprise information management, vol 18 no 6, pp 678-696. andreopoulou, z.s., koutroumanidis, t. & manos, b. (2009) "the adoption of e-commerce for wood enterprises", international journal of business information systems, vol 4 no 4, pp 440-459. aragon-correa, j.a., hurtado-torres, n., sharma, s. & garcia-morales, v.j. (2008) "environmental strategy and performance in small firms: a resource-based perspective", journal of environmental management, vol 86 no 1, pp 88-103. beck, a.c., campbell, d. & shrives, p.j. (2010) "content analysis in environmental reporting research: enrichment and rehearsal of the method in a british–german context", the british accounting review, vol 42 no 3, pp 207-222. berente, n., hansen, s., pike, j.c. & bateman, p.j. (2011) "arguing the value of virtual worlds: patterns of discursive sensemaking of an innovative technology", mis quarterly, vol 35 no 3, pp 685709. bingley, s., burgess, s., sellitto, c., cox, c. & buultjens, j. (2010) "a classification scheme for analysing web 2.0 tourism websites", journal of electronic commerce research, vol 11 no 4, pp 281-298. australasian journal of information systems volume 18 number 3 2014 157 brammer, s., hoejmose, s. & marchant, k. (2012) "environmental management in smes in the uk: practices, pressures and perceived benefits", business strategy and the environment, vol 21 no 7, pp 423-434. burgess, l., parrish, b. & alcock, c. (2011) "to what extent are regional tourism organisations (rtos) in australia leveraging the benefits of web technology for destination marketing and ecommerce?", electronic commerce research, vol 11 no 3, pp 341-355. burgess, s. (2008) "determining website content for small businesses: assisting the planning of owner/managers", international journal of knowledge management studies, vol 2 no 1, pp 143-161. burgess, s. (2011) "the use of third party website providers by australian small businesses", international journal of digital society, vol 2 no 1, pp 328-336. carey, j. (2008) "role misconceptions and negotiations in small business owner/web developer relationships", journal of management and organization, vol 14 no 1, pp 85-99. cho, c.h. & roberts, r.w. (2010) "environmental reporting on the internet by america's toxic 100: legitimacy and self-presentation ", international journal of accounting information systems, vol 11 no 1, pp 1-16. chong, d. & druckman, j.n. (2007) "framing theory", annual review of political science, vol 10, pp 103-126. clarke-sather, a.r., hutchins, m.j., zhang, q. & gershenson, j.k. (2011) "development of social, environmental, and economic indicators for a small/medium enterprise", international journal of accounting and information management, vol 19 no 3, pp 247-266. coffey, p., tate, m. & toland, j. (2013) "small business in a small country: attitudes to “green” it", information systems frontiers, vol 15 no 5, pp 761-778. cooper, j. & burgess, l. (2000) "a model of internet commerce adoption (mica)", in: s.m. rahman & m.s. raisinghani (eds) electronic commerce: opportunity and challenges, igi global, hershey, pennsylvania, pp 189-201. de vreese, c.h. (2005) "news framing: theory and typology", information design journal + document design, vol 13 no 1, pp 51-62. dick, g.n. & burns, m. (2011) "green it in small business: an exploratory study", 14th southern association for information systems conference, 25-26 march, atlanta, georgia, usa. dincer, c. & dincer, b. (2010) "an investigation of turkish small and medium-sized enterprises online csr communication", social responsibility journal, vol 6 no 2, pp 197-207. du, s. & vieira, e.t., jr. (2012) "striving for legitimacy through corporate social responsibility: insights from oil companies", journal of business ethics, vol 110 no 4, pp 413-427. echegaray, f. (2014) "understanding stakeholders' views and support for solar energy in brazil", journal of cleaner production, vol 63, pp 125-133. elliot, s. (2009) "developing organizational capabilities in smes: enabling environmentally sustainable ict", 22nd bled econference, 14-17 june, bled, slovenia. fisher, j., craig, a. & bentley, j. (2007) "moving from a web presence to e-commerce: the importance of a business web strategy for small-business owners", electronic markets, vol 17 no 4, pp 253-262. goulding, c. (2005) grounded theory: a practical guide for management, business and market researchers, sage, london, thousand oaks. australasian journal of information systems volume 18 number 3 2014 158 gueben, c. & skerratt, r.g. (2007) "smes and environmental communications: motivations and barriers to environmental reporting", international journal of environment and sustainable development, vol 6 no 1, pp 1-16. herzig, c. & godemann, j. (2011) "internet-supported sustainability reporting--expectations and reality: empirical findings from the german dax30", in: m.a. quaddus & m.a.b. siddique (eds) handbook of corporate sustainability: frameworks, strategies and tools, edward elgar, cheltenham, uk, pp 249-281. hicks, k.e., wogalter, m.s. & vigilante, w.j., jr. (2005) "placement of benefits and risks in prescription drug manufacturers' websites and information source expectations", therapeutic innovation & regulatory science, vol 39 no 3, pp 267-278. hofmann, k.h., theyel, g. & wood, c.h. (2012) "identifying firm capabilities as drivers of environmental management and sustainability practices – evidence from small and mediumsized manufacturers", business strategy and the environment, vol 21 no 8, pp 495-566. hsieh, y.-c.j. (2012) "hotel companies' environmental policies and practices: a content analysis of their web pages", international journal of contemporary hospitality management, vol 24 no 1, pp 97-121. jiang, y., raghupathi, v. & raghupathi, w. (2009) "content and design of corporate governance web sites", information systems management, vol 26 no 1, pp 13-27. jones, p., simmons, g.j., packham, g., beynon-davies, p. & pickernell, d. (2014) "an exploration of the attitudes and strategic responses of soleproprietor micro-enterprises in adopting information and communication technology", international small business journal, vol 32 no 3, pp 285306. keller, s. & coulthard, d. (2013) "charting diversity and change in is publications: a tri-continental journal analysis", australasian journal of information systems, vol 18 no 1, pp 5-35. larose, r. & rifon, n. (2006) "your privacy is assured of being disturbed: websites with and without privacy seals", new media & society, vol 8 no 6, pp 1009-1029. lei, c.f. & ngai, e.w.t. (2013) "green it adoption: an academic review of literature", 17th pacific asia conference on information systems, 18-22 june, jeju island, korea. lodhia, s. (2012) "web based social and environmental communication in the australian minerals industry: an application of media richness framework", journal of cleaner production, vol 25 no 1, pp 73-85. macgregor, r.c. & kartiwi, m. (2010) "perception of barriers to e-commerce adoption in smes in a developed and developing country: a comparison between australia and indonesia", journal of electronic commerce in organizations, vol 8 no 1, pp 61-82. matthes, j. & kohring, m. (2008) "the content analysis of media frames: toward improving reliability and validity", journal of communication, vol 58 no 2, pp 258-279. melville, n.p. (2010) "information systems innovation for environmental sustainability", mis quarterly, vol 34 no 1, pp 1-21. mestre, l.s. (2011) "visibility of diversity within association of research libraries websites", the journal of academic librarianship, vol 37 no 2, pp 101-108. neilson, l.c., madill, j. & haines, g.h., jr. (2010) "the development of e-business in wine industry smes: an international perspective", international journal of electronic business, vol 8 no 2, pp 126-147. nielsen, a.e. & thomsen, c. (2009) "csr communication in small and medium-sized enterprises: a study of the attitudes and beliefs of middle managers", corporate communications: an international journal, vol 14 no 2, pp 176-189. australasian journal of information systems volume 18 number 3 2014 159 nowduri, s. & al-dossary, s. (2012) "management information systems and its support to sustainable small and medium enterprises", international journal of business and management, vol 7 no 19, pp 125-131. park, y.j. (2011) "provision of internet privacy and market conditions: an empirical analysis", telecommunications policy, vol 35 no 7, pp 650-662. parker, c.m., bellucci, e., torlina, l., fraunholz, b. & zutshi, a. (2014) "towards a method for measuring the quality of environmental improvement communications on sme websites", knowledge and process management, vol 21 no 3, pp 187-197. parker, c.m. & castleman, t. (2007) "new directions for research on sme-ebusiness: insights from an analysis of journal articles from 2003 to 2006", journal of information systems and small business, vol 1 no 1-2, pp 21-40. parker, c.m., fraunholz, b., zutshi, a. & crofts, m. (2010) "how do australian small and medium enterprises communicate their environmental improvement activities online?", australasian journal of information systems, vol 17 no 1, pp 5-21. parker, c.m., redmond, j. & simpson, m. (2009) "a review of interventions to encourage smes to make environmental improvements", environment and planning c: government and policy, vol 27 no 2, pp 279-301. parker, c.m. & scheepers, r. (2012) "applying king et al.’s taxonomy to frame the is discipline’s engagement in green is discourse", 23rd australasian conference on information systems, 3-5 december, geelong, australia. parker, c.m., zutshi, a., fraunholz, b. & crofts, m.r. (2013) "a method for examining sme descriptions of environmental sustainability online", in: p. ordonez de pablos (ed.) green technologies and business practices: an it approach, igi global, hershey, usa, pp 15-35. ramo, h. (2011) "visualizing the phronetic organization: the case of photographs in csr reports", journal of business ethics, vol 104 no 3, pp 371-387. resnick, m.l. & montania, r. (2003) "perceptions of customer service, information privacy, and product quality from semiotic design features in an online web store", international journal of human-computer interaction, vol 16 no 2, pp 211-234. revell, a., stokes, d. & chen, h. (2010) "small businesses and the environment: turning over a new leaf?", business strategy and the environment, vol 19 no 5, pp 273-288. saffu, k., walker, j.h. & mazurek, m. (2012) "perceived strategic value and e-commerce adoption among smes in slovakia", journal of internet commerce, vol 11 no 1, pp 1-23. sanders, j. & galloway, l. (2013) "rural small firms website quality in transition and market economies", journal of small business and enterprise development, vol 20 no 4, pp 788-806. sensis (2013) "sensis e-business report: the online experience of small and medium enterprises", sensis. http://about.sensis.com.au/downloaddocument.ashx?documentid=484 (accessed 5 november 2013). spence, m., boubaker gherib, j.b. & biwole, v.o. (2011) "sustainable entrepreneurship: is entrepreneurial will enough? a north–south comparison", journal of business ethics, vol 99 no 3, pp 335-367. strauss, a. & corbin, j.m. (1998) basics of qualitative research: procedures and techniques for developing grounded theory, sage, thousands oaks, california. tankard, j.w. (2001) "the empirical approach to the study of media framing", in: s.d. reese, o.h. gandy & a.e. grant (eds) framing public life, lawrence erlbaum, mahwah, new jersey, pp 95-106. australasian journal of information systems volume 18 number 3 2014 160 tsai, j.y., egelman, s., cranor, l. & acquisti, a. (2011) "the effect of online privacy information on purchasing behavior: an experimental study", information systems research, vol 22 no 2, pp 254-268. watson, r.t., boudreau, m.-c. & chen, a.j. (2010) "information systems and environmental sustainable development: energy informatics and new directions for the is community", mis quarterly, vol 34 no 1, pp 23-38. williams, s. & schaefer, a. (2013) "small and medium-sized enterprises and sustainability: managers' values and engagement with environmental and climate change issues", business strategy and the environment, vol 22 no 3, pp 173-186. wymer, s. & regan, e. (2011) "influential factors in the adoption and use of e-business and ecommerce information technology (eeit) by small & medium businesses", journal of electronic commerce in organizations, vol 9 no 1, pp 56-82. yahoo small business directory (2010) "small business information from kochie's business builders". http://smallbusiness.yahoo.com.au/smallbusinessdirectory/searchresults.aspx (accessed 31 december 2010). yu, j., coulson, k.r., zhou, j.x., wen, h.j. & zhao, q. (2011) "communicating corporate environmental citizenship: an examination of fortune 500 web sites", journal of internet commerce, vol 10 no 3, pp 193-207. zhu, y., basil, d.z. & hunter, m.g. (2009) "the extended website stage model: a study of canadian winery websites", canadian journal of administrative sciences, vol 26 no 4, pp 286-300. paper title (paper title style) australasian journal of information systems hoon, tan, vasa et al 2015, vol. 19, selected papers from acis ict-enabled time-critical clinical practices ict-enabled time-critical clinical practices: examining the affordances of an information processing solution leonard hoon swinburne university of technology lhoon@swin.edu.au felix ter chian tan university of new south wales f.tan@unsw.edu.au rajesh vasa swinburne university of technology rvasa@swin.edu.au kon mouzakis swinburne university of technology kmouzakis@swin.edu.au mark fitzgerald alfred hospital, melbourne m.fitzgerald@alfred.org.au abstract in this paper, we present a case study of a decision-support system deployment at the alfred hospital, in melbourne, australia. this work outlines information and communications technology (ict) affordances and their actualisations in time-critical clinical practices to enable better information processing. from our study findings, we present a stage-wise model describing the role played by ict in the context of the trauma centre practices. this addresses a knowledge gap surrounding the role and impact of ict in the delivery of quality improvements to processes and culture in time-critical environments, amid increasing expenditure on ict globally. our model has implications for research and practice, such that we observe for the first time how information standards, synergy and renewal are developed between the system and its users in order to reduce error rates in the healthcare context. through the study findings, we demonstrate that healthcare quality can be further refined as ict allows for knowledge dissemination and informs existing practices. keywords: time-critical clinical practice; ict affordances; information processing 1 introduction increased expenditure and the desire to improve healthcare quality are driving the adoption and research of information and communications technology (ict) in healthcare on a global scale (braa, monteiro & sahay 2004; goldschmidt 2005). the global health observatory and diagnostics and laboratory technology are key programs in the world health organization that recognise the key role played by health statistics and ict in contemporary healthcare practices. the health informatics forum, for instance, is a network of over 28 health informatics associations and 3,500 practitioners from around the world that is generating robust discussions on the role of technology in supporting clinical practices. whilst the implications of it in healthcare are likely to be pervasive in the coming years, the challenge is to better understand how it adoption in healthcare restructures the delivery system, reengineers care processes and recreates the culture in which healthcare occurs (goldschmidt 2005). one emergent area of research in this field is the diagnostic pathway in time-critical environments, which is a process supported through the use of clinical decision-support systems (berner 2007). the use of decision-support systems in hospitals to enact clinical procedures and diagnostics is widespread, but there is still no consensus on how the benefits 1 australasian journal of information systems hoon, tan, vasa et al 2015, vol. 19, selected papers from acis ict-enabled time-critical clinical practices are achieved or developed. this is important in order to better understand how decisionsupport technology can improve all types of healthcare decisions and transform healthcare services (goldschmidt 2005). given this knowledge gap, the following research question is posed in this study: how does ict enable information processing in time-critical clinical practices? we present a case study of a decision-support system deployment at the alfred hospital (the alfred), in melbourne, australia. in this study, we examine the application of information processing capabilities involving both computer hardware and software that deal with the storage, retrieval and use of healthcare information, data and knowledge for communication and decision making in a time-critical environment. our research has implications for both research and practice. we observed the impacts of ict affordance on clinicians in time-critical clinical practices. toward this goal, data collection was conducted to determine the actualisation of ict affordances in order to explain user satisfaction, clinical efficacy and the impact on the preventable error rate in procedures. we find that by affording growth in the ict knowledge base created by the end-users, the ict is able to evolve and incorporate contemporary medical practices and procedures. beyond the impact afforded by the introduction of the decision-support technology, we co-developed the algorithm designer, a component of the ict that incorporates the clinician expertise to extend the processes supported by the system. in sum, this work outlines ict affordances and their actualisations in time-critical clinical practices to enable better information processing. 2 health it and information processing health information technologies are “applications of information processing” involving both hardware and software that deal with the storage, retrieval, sharing and use of healthcare information, data and knowledge for communication and decision making (goldschmidt 2005, p. 71). they can include applications ranging from electronic health record, personal health record, decision-support and telemedicine systems. this notion of information processing posits that such organisations and individuals need quality information and the capability to gather, interpret, synthesise and disseminate information properly in order to cope with uncertainties (e.g. in technology, demand and supply) and improve decision making (tushman & nadler 1978). information processing constitutes needs, capabilities and the fit between the two in order to obtain desired outcomes (premkumar, ramamurthy & saunders 2005). the literature around information processing focuses on: (1) developing buffers to reduce the effect of uncertainty, and (2) implementing structural mechanisms and information processing capabilities to enhance the information flow and thereby reduce uncertainty. according to the literature, an information processing network and an organisation’s control mechanisms are central to developing these information processing capabilities. the structure of an information processing network describes the patterns through which communication is expedited and information is processed (ahuja & carley 1999). information processing networks are dynamic network-based information processing structures which operate as a coordination mechanism (kwon, oh & jeon 2007). over time, continual exposure to a specific type of organisational structure will propel individuals toward proficiency in processing information and will bolster their ability and confidence to solve problems, through ongoing learning, in a manner consistent with this structure (turner & makhija 2012). in other words, structures direct or adapt the behaviour of individuals by facilitating the individuals’ ability to obtain and derive meaning from the key information related to their work. this view of the individual interacting with the system to achieve information processing capabilities is reflected in the theoretical construct described below. 3 ict affordances in this study, ict affordances refer to action potential, that is, to what individuals or organisations with particular purposes can do with technology or information systems (majchrzak & markus 2012; strong, volkoff, johnson, pelletier, tulu, bar-on, trudel & garber 2014). various scholars have advocated the use of the theory of affordance to inform the study 2 australasian journal of information systems hoon, tan, vasa et al 2015, vol. 19, selected papers from acis ict-enabled time-critical clinical practices of it-associated organisational change processes (volkoff & strong 2013), to theorise how technologies offer action possibilities to work teams and organisational units (gaver 1991; robey, anderson & raymond 2013) and to create new organisational forms (leonardi 2011; zammuto, griffith, majchrzak, dougherty & faraj 2007). the theory of affordance arose primarily from the field of ecological psychology (gibson 1977). it explains how the inherent values and meanings of things in the environment are perceived, and how this information can be used to define the possibilities and limits for action that a material object offers to an actor or how a material becomes implicated in human activity. a number of formalisations of the concept of affordance have been advanced, most notably as dispositional properties of the environment (turvey 1992), relations between the abilities of organisms and features of the environment (chemero 2003), and opportunities for action (stoffregen 2003). the focused nature of the affordance concept is useful to examine the effects of introducing technology to organisations, such that it allows us to explain organisational-level affordances in a manner consistent with critical realism (volkoff et al. 2013). health it, ranging from electronic health record systems to decision-support systems and telemedicine systems, affords clinicians the potential to access patient data and medical history electronically and independently of their institution or location. this allows patient records to be entered into decision-support systems to inform pertinent healthcare decisions through the retrieval and use of healthcare information, data and knowledge. a key benefit of ict adoption in healthcare recordkeeping is the affordance of information dissemination across medical facilities (chad 2011; goldschmidt 2005). on the other hand, scholars have flagged a number of considerations to be taken into account when applying the affordance lens. according to volkoff et al. (2013), affordances are a type or subset of generative mechanisms. just as generative mechanisms are non-deterministic, different actors may actualise affordances differently. just as a mechanism exists whether or not it is exercised, some affordances may never be actualised (or even perceived) in a realworld domain unless there exists someone who—in addition to having the necessary capability—has an intention to actualise the affordance or is motivated by goals that are served by actualising the affordance. in is, efficacy focuses on individuals’ capabilities to competently use computers (computer self-efficacy) in measurements of computer use (compeau & higgins 1995). efficacy in general refers to the level of control wielded by an individual (e.g. the role of an expert) or by a community. collective efficacy is high in cohesive communities with mutual trust and is low in communities that are not cohesive and that do not have mutual trust. in addition, it is noteworthy that in a real-world domain, multiple affordances are present at the same time such that it-associated organisational change can be viewed as interacting strands of affordances spanning time. therefore, in addition to uncovering these affordances, researchers must pay attention to the nature of the relationships between affordances in order to examine the different structural levels from which they emerge from their constituent parts (volkoff et al. 2013). hence, researchers must account for the way in which affordances – when actualised – unfold temporally. 4 case study: trauma centre, the alfred hospital for this study, we use a case study method to exemplify how the application of ict-enabled information processing solutions aids decision making in time-critical clinical practices. the case study research methodology is particularly appropriate for this study for a number of reasons. first, case study research addresses ‘how’ research questions (walsham 1995) whilst examining processes (gephart 2004), and our research question delves into the role of ict affordances in enabling time-critical clinical practices. second, because we established that time-critical clinical practices form an inherently complex and multi-dimensional phenomenon, an objective approach to research is difficult (koch & schultze 2011), making it more appropriate to examine the phenomenon by interpreting the shared understanding of the relevant stakeholders (klein & myers 1999) who, in this case, are clinicians. 3 australasian journal of information systems hoon, tan, vasa et al 2015, vol. 19, selected papers from acis ict-enabled time-critical clinical practices hence, we selected and examined the trauma centre at the alfred—a site of a major computerised approach to trauma management—as our case study. the case selection was based on the following criteria: (1) the case must be a widely recognised time-critical clinical practice in order to fulfil our examination of activities, (2) the processes of the clinical practice must be reasonably complex in order for the underlying mechanisms to be studied, which means that information processing networks and activities exist, and (3) the case study must present opportunities for the ethnographic research of clinicians, in order to study the efficacy of the actors responsible for clinical practices. we narrowed the focus of our inquiry and data analysis to three pertinent themes: (1) the structure and environment of the trauma centre, (2) the actions of the clinicians and the actions of the centre’s technology partners, and (3) the development of time-critical information processing capabilities achieved through healthcare ict affordances. to this end, the authors conducted site visits to the alfred. we conducted semi-structured interviews (taylor & bogdan 1998) with doctors, nurses and other clinicians, and software developers, and collected secondary data from various documents related to the project. we also drew on secondary data such as newspaper articles and the hospital’s media releases to triangulate our mappings. in addition, three of the authors of this paper were actively involved in the design and implementation phases of the trauma centre’s decision-support system project, specifically in the algorithm design component (discussed in detail in section 5.1). table 1 lists the breakdown of the participants by role and experience. participant(s) role # of participants years of experience questions, topics discussed < 1 1-5 6-10 > 10 airway nurse 7 4 1 2 airway procedures, preand post-procedure checklists in trr anaesthetist/ registrar 6 2 1 3 anaesthetist duties and procedures, patient recovery checklist in trr circulation scout nurse 7 2 2 2 1 circulation procedures, pre and post-procedure checklists emergency physician/registrar 4 1 3 range of responsibilities in trr radiographer 6 1 3 2 frequency of scans, preand post-scan process trauma nurse leader 6 1 2 1 2 trr data entry and communication processes trauma medical team leader 3 1 1 1 team and expertise sizes, team communication, system use trauma orderly 2 1 1 role and responsibilities in trr process trauma surgeon/registrar 7 1 2 4 role and responsibilities in trr process table 1: participant demographics forty-eight professionals participated in this study, some of whom opted to not share their demographic information. the participants were volunteers from among the alfred’s trauma staff who had been rostered into trauma teams. the size of the trauma team for each operating scenario ranged from six to twenty people, based on the expertise required to treat the trauma. as multiple scenarios were run with trauma teams of differing sizes, deliberate effort was made in our participant grouping to ensure that each participant would only participate in, at most, two different scenarios. generally, all the participants were queried about medical treatments, patient reception procedures, treatment protocol, typical responsibilities, patient arrival 4 australasian journal of information systems hoon, tan, vasa et al 2015, vol. 19, selected papers from acis ict-enabled time-critical clinical practices information, and post-trauma recovery. questions specific to the participants’ roles are also listed in table 1. 5 trauma reception and resuscitation at the alfred the trauma centre at the alfred is the site of a major investigation into the impact of a computerised and algorithmic approach to trauma management, through the trauma reception and resuscitation (trr) project.1 the objective of this project is to assist trauma teams by guiding diagnoses with precedents and to facilitate the review processes of the procedures that are performed for compliance. previous studies in australia, europe and the united states demonstrate that the methodical approaches drawn from other high-risk domains should be incorporated into healthcare practices in order to reduce preventable errors and minimise patient harm and risk (the joint commission 2009); however, there is some ambiguity in the results associated with measuring compliance with algorithms in real time. therefore, a major aim of the trr project is to establish a standardised environment. meeting this aim involves testing whether compliance with evidence-based medical approaches will indeed improve outcomes in major trauma management with a measurable reduction in errors. 5.1 the ict artefact: the trauma reception and resuscitation system at the heart of the trr project is the trr system that guides medical staff during the crucial first 30 minutes of trauma reception and resuscitation, as this time period has been observed to have a significant impact on preventable morbidity and mortality. patient data including vital signs, confirmed and/or unconfirmed diagnoses and treatments are entered into the trr system directly via a touchscreen monitor by the trauma nurse leader (tnl). based on these data, computerised algorithms prompt the trauma team in real time to confirm the state of the patient, perform procedures and administer drugs as well as assist in diagnosing injuries. the trr system displays prompts and patient data for the trauma team on a large overhead 40” monitor mounted on the wall of the trauma bay. compliance with the algorithms is guided by real-time computer-generated prompts linked to real-time data collection. a computerised video audit is used to verify compliance and to assess error rates. the medical algorithms at the core of the trr system serve to guide and support the trauma team. the algorithms are not prescriptive, nor do they replace the expertise of the medical staff. the algorithms have been, and will continue to be, subject to close scrutiny, discussion and evaluation by trauma specialists. figure 1 illustrates the setting of the trr system, including the monitor and the interface. successfully integrating a computer system into the specialised and highly pressured trauma care environment required close collaboration between the software developers (see footnote 1) and the medical staff at the alfred. the other key stakeholders in the trr project consist of over 10 domestic and international hospitals, research institutes and trauma centres. the names of the stakeholders and collaborating organisations are removed due to the review process. these details are available from the authors on request. we focused on the relationships and interactions between the subject matter experts (smes) at the alfred, namely, the clinicians at the trauma centre, and the software developers in using the trr system and developing user-centred algorithms. the result of this interaction included the development of an innovative feature of the system, called the algorithm designer, which enables the medical staff themselves to create medical algorithms in an electronic form that is compatible with the system. the enrichment process supported by the algorithm designer allows a trauma team to commune outside of a time-critical environment to develop therapeutic and diagnostic courses of action based on patient conditions. the benefits of the algorithm designer include: 1) the removal of time pressure, 2) the absence of 1 the trr project is a joint effort of the alfred hospital and the swinburne software innovation lab. for details, visit: http://trrproject.com/ 5 australasian journal of information systems hoon, tan, vasa et al 2015, vol. 19, selected papers from acis ict-enabled time-critical clinical practices significant training required for the smes to enrich the system knowledge base, 3) the affordance of a self-sustaining knowledge base available for professionals to employ for review and training purposes, and 4) the medical staff have control over the reference data upon which the medical algorithms depend. preparation  treatment and diagnostics  maintenance figure 1: setting of the trauma reception and resuscitation system2 6 ict affordances in trauma reception and resuscitation this section presents the preliminary findings from the data analysis of the interview notes and documents gathered during the data collection. the discussion focuses on the three crucial phases in integrating the trr system into trauma reception and resuscitation process. for each phase, we discuss the themes, sub-themes and constructs, and representative evidence from our study that pertains to the ict affordances, its actualization and finally impact on the reference process. reiterating our analytical lens, affordances here refer to the potential function of the software and facilities in enabling the clinicians. 6.1 preparation phase the first thirty minutes of trauma reception and resuscitation have a significant impact on patient outcomes due to the often serious condition in which the patients arrive at the hospital (kramer 2009). it falls to the prerogative of the trauma team to assess and determine the therapeutic and diagnostic pathways based on the individual patient’s condition. we identify this as the preparation phase and discuss the potential it affordances and impacts on the medical staff at the alfred and the developers when creating algorithms for the trr system. correctly recognising afflictions and injuries and dealing with them is critical, as errors invite preventable morbidity and mortality. a 28 member multi-disciplinary medical team comprising nurses, emergency physicians, anaesthetists and trauma surgeons served as smes in the development of the algorithms for the preparation phase. the smes were convened to acquire, collate and curate the most recently available and relevant evidence related to trauma management. the smes took nine months to construct the algorithms, which are placed in the category of early management of severe trauma (civil 1999). the sme team was provided with several standard guidelines for translating their recommendations into medical algorithms and their contingencies. for instance, the sme teams were provided with instructions to: i) identify key ‘decision points’ that require a response from the trauma team, ii) limit the options, and iii) define a reasonable time for task performance. the process of converting medical procedures into computer-based algorithms created lively discussions among the medical staff. fortunately, some of the medical staff understood the constraints of binary logic and were able to assist in identifying the criteria needed for algorithm development. trauma specialists added a timeline to the best-practice algorithms to 2 images taken with permission of the trr project. 6 australasian journal of information systems hoon, tan, vasa et al 2015, vol. 19, selected papers from acis ict-enabled time-critical clinical practices enable computerisation and real-time prompts. informal rapid prototyping involving close collaboration between developers and clinicians was a successful methodology for developing the algorithm designer and its testing tool, called the algorithm tester. table 2 summarises the it affordances and impacts in the preparation phase. theme construct representative evidence it affordances in preparation standardised information “team members do not arrive at the same time, whenever a new member arrives, one of us will have to stop and bring the new arrival up to date with the patient and the treatments performed thus far.” – circulation scout nurse process contingencies “our systems [processes] are contingencies, called upon when needed, normally suggested by a nurse if they aren't busy doing another procedure.” – trauma medical team leader actualisation and it processing capabilities develop informed efficacy “when i walk into the bay and look up, i can see the status of the patient as well as procedures performed thus far, bringing me up to speed.” – radiographer develop process compliance “it is always the small things, like a tetanus booster shot post intubation that gets missed. the symptoms arising from this misstep only appear during recovery leading to extended stay for the patient and an investigative process for us to identify possible causes.” – emergency physician impact on preparation accurate diagnosis and scanning of patient needs “the fast [focused assessment with sonography for trauma] test, which scans the abdomen for internal bleeding, only takes 30 seconds to perform, but if there are major external injuries, it gets missed amongst the chaos.” – emergency registrar table 2. it affordances and impacts in the preparation phase the trr algorithms represent an advance in medical algorithm design because a suggested time frame is allocated to each algorithm step. this enables the generation of real-time prompts to guide effective trauma management. it also provides an evidence-based guide for the time of intervention and duration of procedure. the algorithms cover the majority of presentations to the trauma centre, and improving compliance. the majority of evidence used to inform the algorithms was based on clinical opinion or consensus because the sme team found that well-established evidence to support trauma management was scarce. the algorithms prompt trauma staff to perform accurate actions based on the patient’s vital signs and guide staff on the procedures to be performed, drugs to be administered and the diagnosis of injuries, and are accompanied by written evidence published in a trauma manual. 6.2 diagnosis and treatment phase when the trr system is operating, it compares the externally inputted data with the reference data in order to generate the outputs, which include visual prompts suggesting diagnoses and treatments. we identify this as the diagnosis and treatment phase and summarise the it affordances and impacts of implementing and using the trr systems. some algorithms are triggered immediately after the patient arrives in the trauma bay. other algorithms are triggered by the input of patient data captured directly from the vital signs monitor or via manual input, and others are triggered by particular diagnoses, treatments, vital signs or particular combinations of these. since the number of prompts on the screen at any one time is restricted to four, each prompt is allocated a level of urgency to ensure that medical staff are prompted to perform current and critical procedures before non-critical procedures. to successfully link the medical algorithms created by the clinicians with the trr system, it was essential to enforce a standard format. these algorithms, although created individually, are combined into a single file before being uploaded into the trr system. the trr system 7 australasian journal of information systems hoon, tan, vasa et al 2015, vol. 19, selected papers from acis ict-enabled time-critical clinical practices provides hospital trauma teams with access to computerised decision support for the first 30 minutes of trauma management. the system depends on evidence-based medical algorithms that are prepared as explained earlier. theme construct representative evidence it affordances in diagnosis real-time access to known (patient) records “why don’t we just make available what we wanted to measure real time? it wouldn’t be very helpful if it only raised the issue after the fact.” – airway nurse information currency in situation “we now know chest drains take eleven minutes, from making the decision to getting someone to scrub up and put in the chest drain. now, with chest injuries, we will have someone already sterile, reducing the procedure time by seven minutes.” – trauma orderly actualisation and it processing capabilities enabled continuous information stream “we now have a universal measurement tool that records and tracks the trauma, something we didn’t have before. the information recorded, with context, is so rich, especially when looked at during our weekly audits.” – trauma surgeon ensure compliant monitoring “now that we have time-stamping for our treatments as they happen, we can trace causality within the context of patient vitals during the fact, supported by video evidence.” – anaesthetist impact on diagnosis revised checklist of procedures and protocols “i wanted to implement a checklist for pre-intubation that had to be vocalised and followed so as to minimise loss of the airway. i had no way of objectively coming up with it till i had access to preand post-checklist trauma transcripts to review if having such a checklist would be fruitful.” – emergency physician “our blood transfusion rates have almost halved, because we were now paying attention to all these other things that might have caused the blood loss to begin with. however, this data in isolation was considered an error, as people were not getting any blood, so we had to revise our protocols to account for what we now call accepted variations.” – emergency registrar table 3. it affordances and impacts in the diagnosis and treatment phase the trr system in the trauma bay is set up with two screens. a large overhead screen enables the trauma team to view the patient data and action prompts continuously. a touchscreen is managed by the tnl who is responsible for monitoring the automated data entry and for entering the manual data. preparation prompts appear on both the touchscreen and the overhead screen to enable team members to carry out the necessary action(s) while communicating with the tnl who enters the data into the system. ‘vital signs check’ prompts only appear on the touchscreen because the data are usually captured from the vital signs monitor and are readily accessible to the tnl who is in a position to confirm the accuracy. table 3 presents a summary of the it affordances and impacts in the diagnosis and treatment phase. 6.3 system maintenance phase in practice, all the medical algorithms are subject to an ongoing process of evaluation and amendment. all parties in the project came to realise that a custom-built tool was needed to enable the medical staff to continue to create and maintain the algorithms themselves whenever new evidence becomes available and when protocols or practices change. we identify this as the system maintenance phase, and we outline the it affordances and impacts of implementing and using auxiliary systems observed. several months into the design process, a design tool was added to the clinician’s software requirements. an alternative to providing the clinicians with a design tool was for the developers to code all the algorithms themselves. this option was rejected for both the period of system development and the period of system maintenance following deployment as being 8 australasian journal of information systems hoon, tan, vasa et al 2015, vol. 19, selected papers from acis ict-enabled time-critical clinical practices neither time-efficient for the developers nor cost-effective for the clinicians. the software developers had a constrained timeline for development and they realised that converting the medical algorithms, already created in visio, into a format compatible with their custom-built software would take a considerable amount of their development time. initially, the medical staff provided about a dozen algorithms; however, this number was expected to increase over time. some of the algorithms had a relatively simple logic structure, but others were complex and required medical knowledge to fully appreciate the structure. theme construct representative evidence it affordances in maintenance an objective system, (eliminating subjectivity) “time to interventions was also difficult to record, it was challenging to stop and note all this information. additionally, having to record aseptic techniques, complications was difficult as i would be busy treating the complication.” – trauma medical team leader “it’s easy to record a surgery and have two professors watch it and gooh should have done this, should have done that, etc. but how can we take the subjectivity out of it? how can we standardise what was expected so that we can check for compliance?” – anaesthetist communal and committed users “people now have to commit, if someone is listening to the chest to hear if the tube is in the right place, they have to commit and vocalise – air entry equal? there is no remaining silent. is there a fractured femur? maybe? okay, we now have an unconfirmed fractured femur on the record.” – trauma nurse leader actualisation and itenabled processing capabilities build continuous sustainable knowledge base “we review everything weekly to assess and identify areas of improvement, in terms of patient morbidity and mortality, massive transfusions (of blood), major complications, to identify why these events happen, and if they were preventable. we had to rely on paper documentation made during the timecritical treatment, which may have missing notes.” – emergency registrar build inclusiveness “now that we have time-stamping for our treatments as they happen, we can trace causality within the context of patient vitals during the fact, supported by video evidence.” – radiographer impact on maintenance team proficiency “on a voluntary basis, junior consultants have the option to use the trr to track them, provided the results are confidential, so that they may self assess for performance appraisals. furthermore, they can review their transcript with a more experienced team member to identify areas of improvement.” – trauma medical team leader “teamwork has improved; everyone is on the same page, all the time.” – circulation scout nurse table 4. it affordances and impacts in the system maintenance phase the algorithm designer became a versatile tool that enables the clinicians to modify and update the trr system after deployment. eventually, the tool was developed simultaneously with, but separately from, the main trr system. a rapid prototyping technique was used to build and successively modify the algorithm designer during a close, informal collaboration between the developers and the trauma specialists. the algorithm designer and algorithm tester provide intuitive, easy-to-use interfaces enabling trained medical staff to translate the medical algorithms into an electronic format that is compatible with the trr system. the tools enable the committed clinicians to both design and test the medical algorithms. the algorithm designer serves two purposes: it allows the user to create and edit all the medical reference data used by the system and, as the name suggests, it allows the user to design medical algorithms. the reference data consist of all treatments, diagnoses, injury mechanisms and other communal data used by the trr system, developing into a knowledge base. the 9 australasian journal of information systems hoon, tan, vasa et al 2015, vol. 19, selected papers from acis ict-enabled time-critical clinical practices algorithms rely on the reference data to describe the various medical procedures which may need to be performed on a trauma patient. the application also performs numerous automated checks and displays appropriate warning and information dialogs as the user adds and removes symbols from the algorithm. table 4 presents a summary of the it affordances and impacts in the system maintenance phase. in view of the critical importance of algorithm accuracy to patient safety, it is essential that each algorithm be thoroughly tested before it is integrated into the trr system. the algorithm tester incorporates the same algorithm processing engine component used by the trr system to process algorithms and generate prompts. the algorithm tester is designed to test the operation of the algorithms by accepting inputs (e.g. treatments, diagnoses and vital signs), process the logical flow of the algorithm nodes, and output appropriate prompts for action by the user. the algorithm tester simulates the operation of the algorithm as it would be in the live system and allows the designer to check the accuracy of all the prompts that will be generated in the live system. it enables the user to input test data and to confirm that the required action prompts appear at the specified times and with the specified urgency rating (high, medium or low). once tested and considered to be satisfactory from the medical perspective, the algorithms are saved and uploaded by the clinician into the trr system. 7 discussion and contribution our study findings demonstrate the impact of ict affordances on clinicians in a time-critical clinical practice through a case study of the deployment of a decision-support system at the alfred hospital in melbourne, australia. from our study findings, we derive and present a summary stage-wise model of how clinical ict affordances are offered. figure 2: summary stage-wise model of how clinical ict affordance are offered in the trauma reception and resuscitation process the trr system is designed to provide trauma teams with access to real-time best practice in trauma management by providing decision-support software for the first 30 minutes of trauma management. one of the major factors contributing to the successful acceptance of the system was the provision of a facility for clinicians to create and maintain the medical algorithms on which the system is based. the algorithm designer is an auxiliary software tool that enables the trauma specialists to adapt the system to meet their own particular needs. using it, clinicians can add, upgrade and delete the algorithms and the reference data upon which the algorithms are based. close collaboration between the developers and the clinicians combined 10 australasian journal of information systems hoon, tan, vasa et al 2015, vol. 19, selected papers from acis ict-enabled time-critical clinical practices with the rapid prototyping methodology for development of the software were critical factors in the system’s successful implementation. although initially there were minor problems as a result of giving the clinicians control over the maintenance of the medical algorithms, the benefits are overwhelming because the medical aspects of the deployed system are under the ongoing control of the trauma specialists. figure 2 illustrates how clinical it affordances are offered in the process of trauma reception and resuscitation in the stage-wise model. the remainder of this section discusses the development of information standards, information synergy and information sustainability in the trr system. 7.1 development of information standards the first step towards creating a standardised environment for trauma management is to develop computerised medical algorithms. in order to do so, the information in the system must conform to a well-understood syntax and vernacular with which the trauma teams are familiar. the main objective of these algorithms is to identify and document standards for clinical trauma management derived from empirical evidence and best-practice trauma management approaches. in addition, the algorithms afford error detection and measurement, where error is defined as deviation from the algorithms as applied in a standard physical environment. we observed that the errors of omission were reduced by “at least 20%” [trauma medical team leader] due to the establishment of standardised best practices. an error of omission describes non-compliance with a process when administering a treatment that would lead to consequences of low severity if the error occurs in isolation. for instance, when intubating a patient, a tetanus booster shot should be administered. should the tetanus shot not be administered, it would be considered an error of omission. the consequences of not administering the shot could potentially lead to a longer recovery process for the patient, which directly translates to higher overheads of care and resource expenditure. by comprehensively tracking all procedures and treatments, prompts can be issued to avoid such errors. by reducing the cognitive overhead of stopping to brief any team member who arrives in the trauma bay to update them on the patient status and sequence of treatment thus far, the trauma team is afforded better focus on the task at hand. the display readout of the patient status and treatment administered is also consistent, which means that all members of the team, whether the newly arrived team members or the initial responders, are aware of the same precise information. the precision of data input also leads to concise and committed recordkeeping. as patients undergo treatment, each team member brings their skills to the fore to diagnose and address the injuries they see. additionally, as they treat the immediately apparent injuries, postulations on causal or coupled injuries (e.g. “it might be a broken rib cage causing this” or “this may have caused some abdominal bleeding, can someone check?”) can be vocalised and verified systematically, without being forgotten. verifying and committing to follow-through with such actions results in less activation of contingency processes as a result of mistakes and omissions. the algorithm designer is not only innovative for the purpose for which it is designed, it also opens up possibilities for use in other medical settings. using the algorithm designer, medical staff in a different environment can amend or replace the algorithms to meet their particular situations. for example, the algorithms designed by the trauma specialists at the alfred deal with approximately 90% of all trauma cases treated at that hospital. victims of gunshot wounds are rare. hospitals in other parts of the world that encounter such wounds with higher frequency would be able to develop new algorithms to deal with these cases. the algorithm designer has the necessary flexibility to be adapted to any situation. the software also has great potential for use as a training tool for inexperienced trauma staff. in addition, standardisation serves to bridge the gaps in information localisation and globalisation, most notably in conversions between imperial and metric systems when administering drugs. another opportunity for standards is between vernaculars used in different countries, e.g. such as “o2 sat” (in australia) vs “oxygen saturation” (in the us); this would reduce the delay from seeking clarification or reduce the margin of error from 11 australasian journal of information systems hoon, tan, vasa et al 2015, vol. 19, selected papers from acis ict-enabled time-critical clinical practices misunderstandings. through the data entry mechanisms in the algorithm designer, such discrepancies can be avoided by enforcing data input formats, whereas the data output could still be displayed with localisation to the relevant region. this translates to a system module that guides user input so as to achieve standardised output that is understandable to all the job roles in the trauma room. the by-product of this also ensures that if a well-established course of action is the standard response to specific patient conditions, the team can respond by applying their training immediately. 7.2 development of information synergy informal rapid prototyping involving close collaboration between developers and clinicians was a successful methodology for developing the algorithm designer and the algorithm tester. the trauma specialists added a timeline to best-practice algorithms, which enabled computerisation and real-time prompts. the algorithm designer and the trr system were not networked. the software developers were well aware of this issue. although some problems arose when the main trr system was not being developed concurrently with the algorithm designer, the onus was on the user to ensure consistency between versions of the trr system and the algorithm designer. to overcome the potential problem of uploading mismatched algorithms and reference data into the trr system, the same automated checks that were used by the algorithm designer to test the algorithms were performed whenever the trr system was started, warning the user appropriately if mismatched algorithm and reference data conditions were detected. data from sources such as monitoring equipment, patient records and medical practices need to be processed, consolidated and reported by the trr system. this relies on the natural synergy and preand post-conditions of patient vitals and history in order to determine appropriate procedures and courses of action. as the algorithms are aggregated from precedent, the algorithms displayed in the trauma bay give all the members of the team access to experience in treating wounds and injuries in real time. this is in contrast to relying only on the recollection of the more experienced members in the team and the leadership in the bay. patient history is incorporated in the algorithms if available, and conditions can be verified by the trr prior to advocating a diagnostic pathway. furthermore, subjectivity is removed from the treatment of patients by the clinicians’ observation of the standardised data and their use of it to decompose processes into modular treatments. in the trauma centre, this led to new findings during audits of instances of the trauma team’s performance that, without the contextual information such as time-stamping, video recordings and transcripts of procedures, would have been considered poor treatment. for instance, an interviewee was noted to mention: “a new team member would come in and say, ‘oh, patient was in shock, blood pressure was 90’. whereas now, the record reflects the patient blood pressure was normal for the past 15 minutes, but morphine was just administered, lowering the blood pressure just before the team member arrived; this information is critical in informing his next action.” additional findings were that blood transfusions were halved. typically, the audit team would have considered this statistic to indicate that the patients were not given enough blood. however, with the contextual information, it was observed that the trr system was enforcing due diligence and compliance by reminding physicians to follow through with their treatments; hence, patients needed fewer blood transfusions. this also led to the discovery and proposal of acceptable variations in compliance for the alfred. this information synergy, however, relies on having information standards in order to afford opportunities to use the algorithms developed by other hospitals. the compliance introduced by information standards and synergy effectively allows the trauma clinicians to incorporate the collective and empirical experience of trauma clinicians and use it when assessing patient conditions to determine diagnostic pathways – a step that was not possible prior to the it adoption. 12 australasian journal of information systems hoon, tan, vasa et al 2015, vol. 19, selected papers from acis ict-enabled time-critical clinical practices 7.3 development of information sustainability auxiliary systems afford clinicians the ability to add, upgrade and delete the algorithms and the reference data upon which the algorithms are based. we describe the continual capacity to manipulate and update the reference data upon which the systems are built as the it-enabled development of information sustainability. in our case study, the algorithm designer and algorithm tester are auxiliary software tools developed to afford trauma specialists the management of medical algorithms using a custom-built graphical user interface. the tools can be easily adapted to different clinical environments. the algorithm designer and algorithm tester are innovative tools that provide trauma specialists with an effective way of maintaining the trr system and keeping it at the cutting edge of medical knowledge. in such a mission-critical and time-critical domain, the knowledge base of the system must adapt to the evolving needs of users, and hence, a means to enrich the system’s knowledge base must be provided. by allowing the end-users who are also smes to enrich the knowledge base, the system can grow in afforded diagnostic pathways in parallel with practitioner experience. this allows the system to conform to and incorporate contemporary medical practices. a nonproprietary input format reduces the barrier for entry to enrich and update the system’s knowledge base. furthermore, it allows practitioners—both internal and external to the institution—to reference the diagnostic pathways entered by clinicians who are well versed in the execution of treatments. as more junior consultants join the trauma team, the trr system is offered as a selfassessment and critical analysis tool for personal development. the use of the system in that capacity is voluntary, and once confidentiality is assured to the consultants, the trr system is employed as a tracking and monitoring tool for their performance. the transcripts are then offered to them to self-assess their performance, also affording them the option to go through the transcripts with more senior staff to identify areas of improvement. when “non-typical actions” are performed by the new staff, the trauma team leaders can either incorporate them into the existing processes or add system prompts as contingencies against such actions. information sustainability requires information standards and information synergy in order to be applicable. should compliance to standards not be met, and affordance for synergy not be offered, the enrichment of system data cannot take place. 8 study implications we observed the impacts of ict affordances on clinical information processing and clinicians in a time-critical environment. our study has implications for both research and practice. for research, our study extends the existing knowledge on ict affordances in the is literature, demonstrating that human variables in a clinical setting afford both predictable and unintended occurrences in ict use, even among the most established systems and professional clinicians. this is particularly the case in it-enabled solutions that simultaneously lead to new or improved capabilities, relationships and better use of assets and resources (i.e. enhanced capacity to act). in our model and consistent with prior studies, ict affordances depict the relationships between the abilities of an individual and the features of the ict within the context of the environment in which they function (chad 2011). according to chad (2011), there is a range of affordances in a clinical environment; his study focused on understanding the affordance threshold as theoretical constructs in the nomological network of affordances that help to explain the use of ict as a function of the difficulty of acting on ict affordances. we show that affordances and technology change are invariably linked; recognising that it is increasingly building on the knowledge and skills of professionals and the day-to-day practices of sub-cultures. the processes used at the alfred prior to the deployment of the trr system were team-based, focusing on decomposing complex traumas into actionable steps using crisis management principles and communication. given the linearity of human thinking, and the variations in the number of operators, backgrounds and expertise, errors of omission are a constant risk in such an approach. hence, a system that affords feedback outside of linear 13 australasian journal of information systems hoon, tan, vasa et al 2015, vol. 19, selected papers from acis ict-enabled time-critical clinical practices thinking processes would offer support and real-time error avoidance. this eventuated in the design of a decision-support system that could be deployed in the trauma bay. in such a system, the trauma team should be able to serve as the smes to enrich the system’s pathways as they evolve and refine their practices. the use of affordances to analyse the design of technologies explicitly can help suggest ways to improve their usability in practice (gaver 1991). our study discuss the development of information standards, synergy and sustainability pertaining to the actualisation of the affordance of an ict tool that allows the organisation to evolve the ict without having to engage software engineers. when used in conjunction with electronic patient history and records, the ict displays potential applicability for teaching and training, as well as for audits and reviews. in addition, the location-independence of the system is a step towards better knowledge consolidation and dissemination among clinicians. the trr system requires the manual data input to be performed by the tnl in the trauma bay. hence, to ensure that this input process does not bottleneck the capacity of the system to support the trauma management, we performed further usability testing with the tnls to refine the effectiveness and efficiency of the input interface. the testing took place in a usability lab environment with eighteen participants, of whom seventeen were tnls working at the alfred. beyond the data input, the human variables introduced in a multi-disciplinary team, coupled with fluctuating patient conditions, resist the modelling of a standardised environment. these human variables result in the occurrence of errors even among established and experienced trauma centres and professionals. furthermore, we developed and discussed the impact of the algorithm designer, a component of the ict that allows clinicians to enrich and refine the processes supported by the ict. based on the analysis of the preliminary and secondary data, the pragmatic results of the ict use indicated a reduction in the errors made by emergency staff by over 20%. 9 conclusion in this paper, we present a case study of the trauma reception and resuscitation project at the trauma centre of the alfred hospital in melbourne, australia. in this study, we observe the actualisation of ict affordances in a decision support system that is designed for better information processing. specifically, our stage-wise model addresses a knowledge gap surrounding how ict afford the development of information standards, synergy and renewal in order to reduce error rates in time-critical clinical practices. we show that ict affordance focuses on the strengths and weaknesses of technologies with respect to the possibilities they offer to the people who might use them. this study represents the first step to better understanding how ict affordances enable information processing in time-critical clinical practices; we are working towards the conduct of future studies on adapting the ict for multinational surgeon training and evaluations, and as a means to track and monitor procedures for accountability in cases of morbidity and mortality. 10 acknowledgements we would like to thank the team at the trauma centre, the alfred hospital, melbourne, and all other participating agencies for their kind support for our research. references ahuja, m. k. and carley, k. m. (1999). network structure in virtual organizations. organization science, 10 (6), 741-757. berner, e. s. (2007). clinical decision support systems. springer. braa, j., monteiro, e. and sahay, s. (2004). networks of action: sustainable health information systems across developing countries. mis quarterly, 28 (3), 337-362. chad, a. (2011). health information systems affordances: how the materiality of information technology enables and constrains the work practices of clinicians. in: computer information systems, georgia state university. 14 australasian journal of information systems hoon, tan, vasa et al 2015, vol. 19, selected papers from acis ict-enabled time-critical clinical practices chemero, a. (2003). an outline of a theory of affordances. ecological psychology, 15 (2), 181195. civil, i. (1999). trauma '98 emst: past, present, future. the history of the emst course anz journal of surgery, 69 (8), 551. compeau, d. r. and higgins, c. a. (1995). computer self-efficacy: development of a measure and initial test. mis quarterly 19 (2), 189-211. gaver, w. w. (1991). technology affordances. in proceedings of the sigchi conference on human factors in computing systems, acm, pp. 79-84. gephart, r. p., jr (2004). from the editors: qualitative research and the academy of management journal. academy of management journal, 47 (4), 454-462. gibson, j. (1977). the theory of affordances. in r. shaw & j. bransford (eds.). perceiving, acting, and knowing: toward an ecological psychology. hillsdale, nj: lawrence erlbaum. pp. 67-82. goldschmidt, p. g. (2005). hit and mis: implications of health information technology and medical information systems. commun. acm, 48 (10), 68-74. klein, h. k. and myers, m. d. (1999). a set of principles for conducting and evaluating interpretive field studies in information systems. mis quarterly, 23 (1), 67-93. koch, h. and schultze, u. (2011). stuck in the conflicted middle: a roletheoretic perspective on b2b e-marketplaces. mis quarterly, 35 (1), 123-146. kwon, d., oh, w. and jeon, s. (2007). broken ties: the impact of organizational restructuring on the stability of information-processing networks. journal of management information systems, 24 (1), 201-231. leonardi, p. m. (2011). when flexible routines meet flexible technologies: affordance, constraint, and the imbrication of human and material agencies. mis quarterly, 35 (1), 147-167. majchrzak, a. and markus, m. l. (2012). technology affordances and constraints in management information systems (mis). encyclopedia of management theory, (ed: e. kessler), sage publications, forthcoming, premkumar, g., ramamurthy, k. and saunders, c. s. (2005). information processing view of organizations: an exploratory examination of fit in the context of interorganizational relationships. journal of management information systems, 22 (1), 257-294. robey, d., anderson, c. and raymond, b. (2013). information technology, materiality, and organizational change: a professional odyssey. journal of the association for information systems, 14 (7), 379-398. stoffregen, t. a. (2003). affordances as properties of the animal-environment system. ecological psychology, 15 (2), 115-134. strong, d. m., volkoff, o., johnson, s. a., pelletier, l. r., tulu, b., bar-on, i., trudel, j. and garber, l. (2014). a theory of organization-ehr affordance actualization. journal of the association for information systems, 15 (2), 53-85. taylor, s. j. and bogdan, r. (1998). introduction to qualitative research methods, 3rd ed. john wiley, new york. turner, k. l. and makhija, m. v. (2012). the role of individuals in the information processing perspective. strategic management journal, 33 (6), 661-680. turvey, m. t. (1992). affordances and prospective control: an outline of the ontology. ecological psychology, 4 (3), 173-187. 15 australasian journal of information systems hoon, tan, vasa et al 2015, vol. 19, selected papers from acis ict-enabled time-critical clinical practices tushman, m. l. and nadler, d. a. (1978). information processing as an integrating concept in organizational design. academy of management review, 3 (3), 613-624. volkoff, o. and strong, d. m. (2013). critical realism and affordances: theorizing it-associated organizational change processes. mis quarterly, 37 (3), 819-834. walsham, g. (1995). interpretive case studies in is research: nature and method european journal of information systems, 4 (2), 74-81. zammuto, r. f., griffith, t. l., majchrzak, a., dougherty, d. j. and faraj, s. (2007). information technology and the changing fabric of organization. organization science, 18 (5), 749-762. an earlier version of this paper was presented at the australasian conference on information systems (acis) 2014 in auckland, new zealand. copyright: © 2015 hoon, tan, vasa, mouzakis, fitzgerald. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. 16 http://creativecommons.org/licenses/by-nc/3.0/au/ http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 health it and information processing 3 ict affordances 4 case study: trauma centre, the alfred hospital 5 trauma reception and resuscitation at the alfred 5.1 the ict artefact: the trauma reception and resuscitation system 6 ict affordances in trauma reception and resuscitation 6.1 preparation phase 6.2 diagnosis and treatment phase 6.3 system maintenance phase 7 discussion and contribution 7.1 development of information standards 7.2 development of information synergy 7.3 development of information sustainability 8 study implications 9 conclusion 10 acknowledgements paper title (paper title style) australasian journal of information systems hasan & dawson 2015, vol. 19, selected papers from acis discipline diversity appreciating, measuring and incentivising discipline diversity: meaningful indicators of collaboration in research helen hasan australian health services research institute university of wollongong hasan@uow.edu.au linda dawson faculty of social science university of wollongong lindad@uow.edu.au abstract inter-disciplinary collaborative research is generally believed to lead to innovative outcomes in areas that may be missed in research studies based in a single discipline. however, currently available research performance indicators, based on scholarly peer-reviewed publications and citations from a single discipline, do little to recognise the merits of collaborative and interdisciplinary research. this paper presents an empirical study of members of a research unit and their publication and grant profiles. from analysis of this data a set of profile categories emerged together with the relevant indicators which provide a framework from which a deeper understanding of how different research behaviours contribute to the differences in researchers’ individual profiles. these profiles could be used to provide a richer environment for the evaluation of research performance, both in terms of outputs and potential funding opportunities, and indicators of ‘good research’ in inter-disciplinary projects. keywords: collaboration; diversity: research performance; indicators; inter-disciplinary 1 introduction although widely used and generally found beneficial or convenient, currently available research performance indicators, based on scholarly peer-reviewed publications and citations from the same in a single discipline, do little to recognise the merits of collaborative and interdisciplinary research. several scholars have suggested a broader range of academic research kpis than those that are already used, which are often restricted to publication counts in “quality” journals and citations to these publications (takeda et al 2010). goring et al (2014) suggest that scholarly performance evaluation should include research outputs other than publications, including educational outcomes, dataset creation, outreach products (eg blogs or social media), and the application of scientific results to policy or management activities, similar to altmetrics (priem, 2011). iivari (2008) recommends a combination of expert evaluation and bibliometric evaluation. in evaluating the influence of respected is scholar heinz klein, truex et al (2011) propose the “focus should be shifted from the venue of publication of the research to the uptake of the ideas contained in it, thus increasing the openness of the discourse, participation in the discourse, truthfulness, and reduction of the inequities in power distribution within academia” (truex et al 2011 p 422). li et al (2013) suggest that by using social capital indicators “a scholar can identify his or her structural position in the network and formulate a co-authorship strategy according to the future position where he or she wants to be.” … [however] … “collaborating with too many different scholars might put a researcher at risk of being distrusted by prolific scholars and losing chances to coauthor with them.” p 1528. these proposed means of assessing research performance are valuable but introduce a subjective element that would be difficult to implement. they also do not directly address issues of disciplinary diversity. the study that is presented in this paper provides a pragmatic 1 australasian journal of information systems hasan & dawson 2015, vol. 19, selected papers from acis discipline diversity way of measuring the extent of collaboration and inter-disciplinarity of a scholars work. although these measures in themselves say little about research outcomes, they do facilitate the creation of different researcher profiles that could determine on what grounds researchers with these different profiles could be assessed for relevant quality outcomes. 1.1 strength from diversity studies indicate that diverse teams that consisting of heterogeneous participants have the ability to be more innovative and more creative than teams consisting of very homogeneous members. beekhuyzen j. stein a. (2014) the above assertion in the call for papers is widely accepted and describes the motivation for this paper. there is an abundance of evidence that diversity in team membership is beneficial although it is not a simple matter with many types of diversity and finding a correlation between diversity and performance can be a challenge (crawford & hasan 2009, liao 2011, podestá et al 2013, kearney and gebert 2009, rafols and meyer 2009). in a study of teams playing weekly strategy games, crawford and hasan (2009) found that heterogeneous teams were able to perform complex activities better than homogeneous ones after an initial learning period of 2 or 3 weeks. they observed that the heterogeneous teams required a few games played together to learn cooperative team skills and effective ways of communicating. however, once team members appreciated the abilities of others, they could effectively leverage their diverse capabilities to perform more effectively overtaking the homogeneous teams whose performances did not greatly improve over time. podestá et al 2013 argue that “a self-reflective process to identify and intervene on factors that foster or impede cooperative production of knowledge should be an essential component of integrated assessments involving scientists, practitioners and stakeholders.” p 40. research that crosses disciplinary boundaries or falls between or outside traditional disciplines is generally believed to lead to innovative outcomes in areas that may be missed in single discipline research studies (liao 2011). according to liao (2011) diversity leads to better research quality which not only inspires scholars to continue their research, but also increases the possibility of higher research budgets from sponsors. it is therefore to be encouraged through appropriate incentives and rewards. however, recognition of collaboration and interdisciplinarity are rarely included in the key performance indications (kpi), on which researchers in academic institutions are assessed for promotion and funding, what is more, inter-disciplinary projects invariably involve collaborations between researchers from different disciplines. a consequence of this is that along with the potential benefits of interdisciplinary research, there are challenges of communicating across varying disciplinary content, language and paradigms leading to challenges in getting inter-disciplinary work published in respected, discipline-focussed journals. to many scholars, these costs are a disincentive to appreciating the benefits of working in diverse teams. this is important for both individual scholars and research managers as “current reward structures are problematic for scientists engaged in interdisciplinary research, particularly early career researchers, because academic culture tends to value only some research outputs, such as primary-authored publications.” (goring et al 2014, p39). the focus of this paper is on the extent, or lack of, appreciation, measurement and incentivising of discipline diversity in research teams. we propose that a new approach to the assessment of research quality is needed that will expand the scope of research performance indicators beyond individual single-disciplinary research. as a first step in gaining recognition for collaborative, inter-disciplinary research we take a critical look at research performance indicators in the next section, and in following sections describe an empirical study into suitable indicators relevant to their strengths and specific attributes of all researchers. the results of this study suggest ways that these indicators can be used to profile researchers so that their outputs and contributions can be assessed appropriately. 2 australasian journal of information systems hasan & dawson 2015, vol. 19, selected papers from acis discipline diversity 2 research performance indicators the reliance on currently available research performance indicators, based on scholarly peerreviewed publications and citations from the same in a single discipline, is concerning as according to rafols and meyer (2009), “interdisciplinary research is seen as more successful in achieving breakthroughs and relevant outcomes, be it in terms of innovation for economic growth or for social needs.” p 263. in order to gain appropriate recognition for those who take up the challenges of collaborative research (within or across disciplines) and inter-disciplinary research (by an individual or group), we investigate potential research performance indicators that could be used by researchers to create their collaboration and inter-disciplinary profile. thus, where researchers engage in inter-disciplinary research projects in order to achieve specific breakthroughs and innovation, they can be recognised as taking up additional challenges and their output could be judged accordingly. in the business of research, managers are required to make decisions on the allocation of funds and resources; on the appointment and promotion of researchers; on the formation of research units; and on whom to reward and encourage. these decisions take into account the quality of previous research performance and future research potential. there is, therefore an imperative to have reliable and meaningful indicators of research quality and potential. a traditional indicator of past performance is a researcher’s scholarly publication record and its impact. the quality of this record can be measured by the quantity of published work weighted by journal rankings. impact is usually determined by including citations to those publications using indicators, such as the h-index (hirsch 2005) where someone with an index of h has published h papers, which have been cited by others at least h times and, in some disciplines, published impact factors of the journals themselves which have been shown to be problematic when considering research quality (lamp et al, 2007). research potential is more difficult to determine but can be based on expert scoring of a project proposal, the stage of the researcher’s career, the research environment or the strength of partnerships with others. it is well accepted that there are limitations of using metrics to judge quality. donovan (2011) notes that: “metrics-only approaches employing economic data and science, technology and innovation indicators were found to be behind the times: best practice combines narratives with relevant qualitative and quantitative indicators to gauge broader social, environmental, cultural and economic public value. limited consultation between policy-makers and the research evaluation community has led to a lack of policy-learning from international developments.” further limitations include, performance indicators based on scholarly publications do not cover all attributes of what is considered “good research” in terms of its value, productivity, national benefit and wider impact. as scholarly publications are not the only output of research projects, reliance on the impact associated with citations in the same set of scholarly publications restricts the judgement of quality to the community of scholars in the researcher’s academic discipline. even then there is no guarantee that such citations confirm the quality of the research. they may refer to the method or theory underpinning the study rather than the study itself. indeed, a citation may be quite critical or damning of the work. the peer-review process has an inherent conservative bias (luukkonen 2012.) this impacts on both the researcher’s publication record and on the scoring of project proposals by expert panels. this bias also mitigates against the recognition of breakthrough ideas or innovative research proposals, which, as stated above, are the likely outcomes of collaborative, interdisciplinary research. it may even discourage researchers from engaging in inter-disciplinary innovative projects with potential to achieve breakthroughs in new areas. 3 australasian journal of information systems hasan & dawson 2015, vol. 19, selected papers from acis discipline diversity 3 indicators of collaboration and cross-disciplinarity 3.1 background in our investigation of potential indicators that could be used by researchers to create their collaborative and inter-disciplinary profile, a literature review outlining approaches to measuring interdisciplinary scientific research (wagner et al, 2011) provided inspiration and encouragement. these authors found that “key among the broader aspects are (a) characterizing the concept of knowledge integration, and (b) recognizing that it can occur within a single mind or as the result of team dynamics. output measures alone cannot adequately capture this process. among the quantitative measures considered, bibliometrics (co-authorships, collaborations, references, citations and co-citations) are the most developed, but leave considerable gaps in understanding.” the investigation reported here takes advantage of existing data within a university database (resdb) on publications and grant applications of the university’s researchers. as part of this data set, each researcher nominates their own 4 digit field of research (for) code from the list provided by the australian research council (arc). researchers can nominate more than one code but must allocate the relative percentage of each. publication outlets such as journals and conferences are also allocated one or more for codes. while most researchers publish in outlets with their own nominated for, we are interested in those that also publish in other for codes and/or with co-authors from other for codes. 3.2 the investigation thirty-five members of a research unit participated in an exercise to examine their own publication and grant records. the university’s research data management unit (rdmu) were contacted and asked to provide data from resdb for each researcher on the following: 3.3 collaboration 1. number of co-authors on papers/ co-investigators on grant a. total number co-authors/co-investigators b. average number of authors/ investigators per publication/grant application c. number of distinct co-authors/co-investigators and how many times they have collaborated d. number of co-authors/co-investigators internal/external to faculty e. number of co-authors/co-investigators internal/external to university 3.4 multidisciplinary 2. number and percentages of different for codes of publications / grants 3. number and percentages of different for codes among co-authors/coinvestigators 4. possibility of weighting these by quality (eg publication ranking, grant amount) the rdmu extracted publication and grant records for the 35 members of the research unit. in addition to this raw publication data, the researchers provided aggregated data on publication records which they itemised as follows: • total number of co-authors – is the total of all co-author collaborations that occurred on a publication for all the publications of the researcher. the researcher does not contribute towards the count. so in essence, for each publication, its total authors on pub – 1. 4 australasian journal of information systems hasan & dawson 2015, vol. 19, selected papers from acis discipline diversity • total number of co-authored publications – is the number of publications where there was a co-author collaborating on it. it does not include those publications which have been singularly authored by the researcher. • average number of co-authors per publication – is total number of coauthored publications / total number of co-authors. • number of distinct co-authors – is the number of authors who have collaborated with the researcher. • average distinct co-author per publication – total number of co-authored pub / number of distinct co-authors • total number of university authors – the university authors are determined using the affiliation tick placed by the library on the publication records in resdb. • number of external authors – every author that does not have the affiliation tick next to their name on the publication record in resdb is deemed to be external. • number of co-authors with different for codes – the number of co-authors who have a different for code as compared to the researcher. • total number of different for codes for co-authors – the number of for codes which the co-authors have that are not the same as those of the researcher. • number of different for-codes of publications and relative numbers of each looking at the raw data as lists of publications in spread sheets, participant researchers, noted there was some missing or incorrect entries but not to any great extent. the for codes allocation to publications was obviously heavily skewed by the australian era process with many being allocated values in line with one of the researcher’s nominated for codes giving a distorted tendency to within-discipline alignment. however, the participants agreed that the aggregated data was meaningful as an indication of how they would classify themselves as researchers. each researcher was given their own data and asked to report on which of these items were meaningful to them; if there were any other items that could be included; and what personal research profile was revealed by these indicators. a workshop was held with the participant researchers and the rdmu team to discuss the data, its potential meaning and implications. 4 findings the participants agreed that it was important to emphasise that the proposed indicators are not measures of quality research in themselves, i.e. high scores are not inherently better than low ones. rather, they would contribute to a researcher’s holistic profile, providing evidence for whether they work alone or with others and whether their work lies within or across disciplines. the participants were asked to use their own indicators to create profiles in order to support the story the researcher could tell of their research when, for example, they are applying for grants or promotion. more significantly, the profiles could also be used to reassess how the value of their research can be determined and, in particular, whether existing indicators of research performance are appropriate or whether others should be used. in the appendix are tables of the aggregated publication data for 29 participants (6 participants with less than 10 publications were removed). each row represents data from one researcher participant. tables 1-3 contain the same data but are sorted on three relevant indicators. the first column contains the number of the most appropriate profile (1-6) for the researcher whose data is contained in that row [nb: some are still unclassified] the following are the set of six profiles that emerged together with the relevant indicators and range of scoring on those indicators [in brackets] on which they would be based: 5 australasian journal of information systems hasan & dawson 2015, vol. 19, selected papers from acis discipline diversity 1. a lone researcher in a single discipline [few/no co-authors or co-investigators, publishes in their own for code i.e. scores low on all tables] 2. a lone inter-disciplinary researcher [few/no co-authors or co-investigators, publishes in many for codes i.e. scores low on tables 1 and 2 but high on tables 3 and 4] 3. a researcher who collaborates with the same set of co-authors within a single discipline [many, but few distinct, co-authors and co-investigators in the same for code, publishes in the common for code i.e. scores high on table 1 but low on tables 2, 3 and 4] 4. a researcher who collaborates with the same set of co-authors from other disciplines [many, but few distinct, co-authors and co-investigators with different for codes, publishes in different for codes i.e. scores high on table 1, 3 and 4 but low on table 2] 5. a researcher who collaborates with many different co-authors within a single discipline [many distinct co-authors and co-investigators with the same for code, publishes in this for code i.e. scores high on tables 1 and 2 but low on table 3 and 4], 6. a researcher who collaborates with many different co-authors across many disciplines [many distinct co-authors and co-investigators in many for codes, publishes in many for codes i.e. scores high on all tables] 5 analysis although the various scores are only indicative of how to allocate researchers to the six categories, all participants that have so far been categorised agreed that their profile made sense to them. from discussion with the participants, the six profiles are now designated as follows: 1. a lone researcher in a single discipline 2. a lone inter-disciplinary researcher 3. a researcher who works in a single discipline team 4. a researcher who works in an inter-disciplinary team. 5. a researcher who works with different researchers from their discipline 6. a researcher who works with different researchers from many disciplines as a general trend we can say that existing research performance indicators are increasingly less appropriate as we move from profile 1 to profile 6. the lone researchers (profiles 1 and 2) were viewed as being those most appropriately evaluated by existing research performance indicators. they published in standard academic outlets although those in profile 2 who were inter-disciplinary found it harder to get published. among those with a collaborative profile (profiles 3-6) there was a clear distinction between those who collaborated with a regular team of co-authors and co-investigators (profiles 3 and 4) and those who collaborated with a large number of different co-authors and co-investigators (profiles 5 and 6). the former were more focussed and fitted the existing research performance culture well. the latter tended to supervise large numbers of research students and/or were mentors to early career researchers. this meant that they had more publications in b and c ranked journal or conferences than the a and a* journals as would be expected of their status as in the excellence in research in australia (era) ranking as published by the arc in 2010. note that rankings were removed from era in 2012. another aspect of the distinction between collaborating in a regular team (profiles 3 and 4) and collaborating with many different other researchers, is expressed in the findings of 6 australasian journal of information systems hasan & dawson 2015, vol. 19, selected papers from acis discipline diversity research by abbasi and altmann (2011) who say that “scholars with strong ties (i.e., repeated co-authorships) show a better research performance than those with low ties (e.g., single coauthorships with many different scholars). the results related to efficiency show that scholars, who maintain a strong co-authorship relationship to only one co-author of a group of linked co-authors (i.e., co-authors that have joint publications), perform better than those researchers with many relationships to the same group of linked co-authors”. an inherent problem with collaborative researchers is that, while their outputs come from a team, they are mostly judged as individuals for awards, appointments, promotion and even funding. this is often in competition with team members or other collaborators. researchers are often required to allocate percentage contributions to all joint publications. funding formulae based on publications are reduced by this contribution making collaboration less attractive. those with profiles 2, 4 and 6 scoring highly on cross-disciplinary indicators have challenges getting their work published in outlets that are ranked highly on existing research performance indicators. these are predominantly embedded in the paradigm, methodology and language of a single discipline. this presents particular challenges for a lone inter-disciplinary researcher (profile 2). those with profiles 4 and 6 have the advantage that journals in which they want to publish belong to the discipline of one research team. indeed, the same basic paper can be legitimately published in more than one outlet in different disciplines. this raises the issue of the motivation for publication which can be both to reach an intended audience and to improve the researchers’ publication records in line with existing research performance indicators. the literature introduced earlier in the paper describes the particular benefits of collaborative interdisciplinary research in terms of innovation, new ideas and breakthroughs. these are often well ahead of accepted topics covered by conservative highly ranked journals. there may be a need to protect the innovation, new ideas and breakthroughs with patents but these are not always applicable or affordable. for most research it is enough to put the work in the public domain attributed to the researchers as authors and originators, and therefore owners of the idea, innovation or breakthrough. the researcher needs to ensure that these are not ‘stolen’ before they can be transformed into products that will reap some reward, and this has been known to happen during the peer-review process. for this reason, the work may be published first, somewhere other than in scholarly publications, often online, because of the need to publish quickly. there were cases among the participants who had published articles in outlets not ranked by the formal performance assessment process and these articles had been highly cited because they were ahead of the pack on a new idea or innovation. 6 implications the motivation for this paper is to give researchers some evidence for new ways in describing their research to others, stories that vary from “i am a researcher who works alone and publishes in a single discipline; the output of my research consists of publications in highlyranked scholarly journals so that its quality and values should be judged by my h-index” to “i am a researcher who works with an inter-disciplinary team of collaborators, who supervised research students and mentors early career researchers the output of these research activities consists of publications in highly-ranked scholarly journals in several disciplines, reports of break-though ideas in online quick-release journals and papers in b and c journals and conferences co-authored with students and mentees”. as a result of this study the questions we pose for future research are: • do current measures (h-index etc.) ignore, hinder or promote innovations, new ideas and breakthroughs? • what are indicators of ‘good research’ in inter-disciplinary projects – i.e. how do you measure progress towards innovation and ‘breakthrough’? 7 australasian journal of information systems hasan & dawson 2015, vol. 19, selected papers from acis discipline diversity • how do we allow for the dichotomy that collaboration is inherently teamnot individual based yet it is often individuals who are assessed? 7 conclusion this paper began with the general challenges of appreciating, measuring and incentivising diversity in order to reap the benefits of workforce diversity in organisations, particularly academic institutions. we have examined the particular case of appreciating, measuring and incentivising diversity in research teams looking at ways that individual researchers may be prepared to put in the extra efforts needed when working in multi-disciplinary teams in order to produce innovative outcomes in areas where this is greatly needed. ultimately, expanded measures incorporating a wider range of indicators should better reflect the important work of both disciplinary and interdisciplinary teams at all career stages; assist evaluators of research to understand wider dimensions and impacts of research; and help sustain and stimulate a collaborative culture of inter-disciplinary research. we assume that this will be of interest to conference participants who are themselves academic researchers. we suggest also that the results of this investigation may be relevant to nonacademic contexts where further research can be undertaken. references abbasi a., altmann j. 2011. “on the correlation between research performance and social network analysis measures applied to research collaboration networks”, proceeding of hicss 2011 beekhuyzen, j. stein, a. 2014. “discovering the potential of demographic and cultural diversity” track description acis2014 crawford, k. hasan, h. m. (2009). a gaming system experience for work in heterogeneous, self-directed teams. proceedings of simtect adelaide. donovan, c. 2011. “state of the art in assessing research impact: introduction to a special issue,” research evaluation (20:3), september, pp 175–179. goring, s., weathers, k., dodd w., soranno p., sweets l., cheruvelil k., jon s kominoski j., rüegg j., thorn a., utz r. 2014 “improving the culture of interdisciplinary collaboration in ecology by expanding measures of success,” frontier ecological environments (12:1), pp 39–47. hirsch, j. 2005. “an index to quantify an individual's scientific research output,” proceedings of the national academy of sciences, 102/46, pp 16569-16572. iivari, j. 2008. “expert evaluation vs bibliometric evaluation: experiences from finland,” european journal of information systems 17, pp 169–173. kearney, e., gebert, d. 2009 “managing diversity and enhancing team outcomes: the promise of transformational leadership,” journal of applied psychology, (94:1), pp 77-89. li, e.y., liao, c.h., and yen, h.r. 2013. “co-authorship networks and research impact: a social capital perspective,” research policy (42:9), 11//, pp. 1515-1530. liao, c. 2011. “how to improve research quality? examining the impacts of collaboration intensity and member diversity in collaboration networks,” scientometrics 86, pp 747– 761. luukkonen, t. 2012. “conservatism and risk-taking in peer review: emerging erc practices”, research evaluation, 21, pp 48–60 lamp, j., milton, s., dawson, l. and fisher, j. 2007. “rqf publication quality measures: methodological issues”, in proceedings of 18th australasian conference on information systems, toowoomba, australia. 8 australasian journal of information systems hasan & dawson 2015, vol. 19, selected papers from acis discipline diversity podestá, g.p., natenzon, c.e., hidalgo, c., and ruiz toranzo, f. 2013. “interdisciplinary production of knowledge with participation of stakeholders: a case study of a collaborative project on climate variability, human decisions and agricultural ecosystems in the argentine pampas,” environmental science & policy (26:0), 2//, pp. 40-48. priem, j. 2011. “altmetrics: a manifesto”, url: http://altmetrics.org/manifesto/, retrieved october 2014. rafols i., meyer m 2009. “diversity and network coherence as indicators of interdisciplinarity: case studies in bionanoscience,” scientometrics (82:2) pp 263-287. takeda h., truex d., cuellar m. 2010. “evaluating scholarly influence through social network analysis: the next step in evaluating scholarly influence,” amcis 2010 proceedings. paper 573. truex d., cuellar m., takeda h., vidgen r. 2011. “the scholarly influence of heinz klein: ideational and social measures of his impact on is research and is scholars,” european journal of information systems 20, pp 422–439. wagner, c. s., j. d. roessner, k. bobb, j. t. klein, k. w. boyack, j. keyton, i. rafols and k. börner 2011. “approaches to understanding and measuring interdisciplinary scientific research (idr): a review of the literature.” journal of informetrics 5(1): 14-26. 8 acknowledgements substantial contributions to the research behind this paper were made by members of the theori research group in the faculty of business at the university of wollongong. the authors particularly acknowledge the work of drs kate crawford, alanah kazlauskas and carole alcock. 9 australasian journal of information systems hasan & dawson 2015, vol. 19, selected papers from acis discipline diversity appendix profile total coauthors on pub total coauthored pubs avg coauthors / pub num distinct coauthors avg distinct co-authors / pub num of authors other for codes number of other for codes spread of for codes in pubs 6 19 6 3.17 12 2 7 5 5 5 11 4 2.75 11 2.75 2 2 2 4 170 68 2.5 33 0.49 6 3 6 6 101 42 2.4 49 1.17 3 3 5 4 85 37 2.3 19 0.51 4 3 7 131 59 2.22 45 0.76 8 6 3 21 10 2.1 10 1 1 2 7 27 13 2.08 20 1.54 15 7 1 5 55 28 1.96 21 0.75 0 0 1 3 19 10 1.9 3 0.3 0 0 1 5 58 31 1.87 32 1.03 0 0 1 3 26 14 1.86 14 1 1 2 1 6 257 142 1.81 107 0.75 25 14 9 5 163 92 1.77 61 0.66 12 9 4 6 45 26 1.73 21 0.81 11 8 4 6 50 29 1.72 27 0.93 10 8 5 5 10 6 1.67 5 0.83 0 0 1 5 39 24 1.63 16 0.67 1 1 1 6 75 47 1.6 33 0.7 7 7 7 3 41 26 1.58 16 0.62 0 0 1 6 32 21 1.52 17 0.81 1 2 3 82 56 1.46 40 0.71 4 3 0 5 31 23 1.35 18 0.78 2 1 4 64 48 1.33 29 0.6 3 2 7 2 41 31 1.32 13 0.42 5 10 6 39 31 1.26 23 0.74 3 4 3 1 11 10 1.1 4 0.4 1 2 2 1 20 19 1.05 7 0.37 0 0 0 2 24 23 1.04 10 0.43 1 2 6 table 1 publication data ordered by avg co-authors/publication 10 australasian journal of information systems hasan & dawson 2015, vol. 19, selected papers from acis discipline diversity profile total coauthors on pub total coauthored pubs avg coauthors/pub num distinct coauthors avg distinct coauthors/pub num of authors other for codes number of other for codes spread of for codes in pubs 5 11 4 2.75 11 2.75 2 2 2 6 19 6 3.17 12 2 7 5 5 27 13 2.08 20 1.54 15 7 1 6 101 42 2.4 49 1.17 3 3 5 5 58 31 1.87 32 1.03 0 0 1 21 10 2.1 10 1 1 2 7 3 26 14 1.86 14 1 1 2 1 6 50 29 1.72 27 0.93 10 8 5 5 10 6 1.67 5 0.83 0 0 1 6 45 26 1.73 21 0.81 11 8 4 6 32 21 1.52 17 0.81 1 2 3 5 31 23 1.35 18 0.78 2 1 4 131 59 2.22 45 0.76 8 6 3 5 55 28 1.96 21 0.75 0 0 1 6 257 142 1.81 107 0.75 25 14 9 39 31 1.26 23 0.74 3 4 3 82 56 1.46 40 0.71 4 3 0 6 75 47 1.6 33 0.7 7 7 7 5 39 24 1.63 16 0.67 1 1 1 5 163 92 1.77 61 0.66 12 9 4 3 41 26 1.58 16 0.62 0 0 1 64 48 1.33 29 0.6 3 2 7 4 85 37 2.3 19 0.51 4 3 7 4 170 68 2.5 33 0.49 6 3 6 2 24 23 1.04 10 0.43 1 2 6 2 41 31 1.32 13 0.42 5 10 6 1 11 10 1.1 4 0.4 1 2 2 1 20 19 1.05 7 0.37 0 0 0 3 19 10 1.9 3 0.3 0 0 1 table 2 publication data ordered by avg distinct co-authors/publication 11 australasian journal of information systems hasan & dawson 2015, vol. 19, selected papers from acis discipline diversity profile total coauthors on pub total coauthore d pubs avg coauthors / pub num distinct coauthors avg distinct co-authors / pub num of author s other for code numbe r of other for codes spread of for codes in pubs 6 257 142 1.81 107 0.75 25 14 9 27 13 2.08 20 1.54 15 7 1 5 163 92 1.77 61 0.66 12 9 4 6 45 26 1.73 21 0.81 11 8 4 6 50 29 1.72 27 0.93 10 8 5 131 59 2.22 45 0.76 8 6 3 6 75 47 1.6 33 0.7 7 7 7 6 19 6 3.17 12 2 7 5 5 4 170 68 2.5 33 0.49 6 3 6 2 41 31 1.32 13 0.42 5 10 6 82 56 1.46 40 0.71 4 3 0 4 85 37 2.3 19 0.51 4 3 7 6 39 31 1.26 23 0.74 3 4 3 101 42 2.4 49 1.17 3 3 5 64 48 1.33 29 0.6 3 2 7 5 11 4 2.75 11 2.75 2 2 2 5 31 23 1.35 18 0.78 2 1 4 21 10 2.1 10 1 1 2 7 3 26 14 1.86 14 1 1 2 1 6 32 21 1.52 17 0.81 1 2 3 5 24 23 1.04 10 0.43 1 2 6 2 11 10 1.1 4 0.4 1 2 2 1 39 24 1.63 16 0.67 1 1 1 5 58 31 1.87 32 1.03 0 0 1 5 10 6 1.67 5 0.83 0 0 1 5 55 28 1.96 21 0.75 0 0 1 3 41 26 1.58 16 0.62 0 0 1 1 20 19 1.05 7 0.37 0 0 0 3 19 10 1.9 3 0.3 0 0 1 table 3 publication data ordered by num of co-authors other for code an earlier version of this paper was presented at the australasian conference on information systems (acis) 2014 in auckland, new zealand. copyright: © 2015 hasan & dawson. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. 12 http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems hasan & dawson 2015, vol. 19, selected papers from acis discipline diversity 13 1 introduction 1.1 strength from diversity 2 research performance indicators 3 indicators of collaboration and cross-disciplinarity 3.1 background 3.2 the investigation 3.3 collaboration 3.4 multidisciplinary 4 findings 5 analysis 6 implications 7 conclusion 8 acknowledgements paper title (paper title style) australasian journal of information systems wang & ranasinghe 2017, vol 21, research on health information systems patient decision making in prostate cancer 1 the decision-making role of the patient in localised prostate cancer treatment luke wang eastern health drlukewang@gmail.com weranja ranasinghe eastern health abstract our objective was to review the current literature on patient participation and decision-making in the treatment selection process for localised prostate cancer, and to evaluate capacity for improvement. methods: 42 articles from our literature search were deemed eligible and relevant for review. we reviewed studies on all facets of the treatment decision-making process with most number of articles (16) on treatment preferences. results: the majority of the patients prefer an active or collaborative role in decision-making. patients are seeking information from a myriad of sources but the recommendation from their treating physician is often the most influential on the final decision. radical prostatectomy is more likely to be selected in patients who view a cure for cancer as being of the utmost importance and radiation therapy is preferred in patients who are concerned about treatment side effects. conclusion: currently no ideal tool exists to assist patients in making informed treatment decisions that also takes into account patients’ values and preferences. we encourage collaborative partnership in a multidisciplinary setting to optimise this process and individualised riskbased decision-making tools may provide a better pathway to assist patients reach decisions. keywords: prostate cancer; patient participation; decisions; treatment 1 introduction prostate cancer (pc) is the most common non-skin malignancy in australia, accounting for 21,808 new diagnoses in 2009 (aihw 2013). it is the fourth leading cause of mortality among australian males, resulting in almost 3,300 deaths in 2011 (aihw 2013). approximately 90% of men have disease confined to the prostate gland or clinically localized disease (wilt, macdonald et al. 2008). while the incidence of pc in australia is increasing, there is a decrease in the rates of cancer specific mortality (wilt, macdonald et al. 2008). while campaigns for greater public awareness, along with early detection and greater rates interventions have attributed to the improved pc outcomes, another significant contributor is the detection of low risk, low volume localised disease due to screening of asymptomatic patients (jemal, siegel et al. 2006, aihw 2013). thus, while screening for prostate cancer has shown some benefit in reducing pc specific deaths, this opened up the paradigm of over-diagnosis and over-treatment calling for uncoupling of screening from treatment (heijnsdijk, wever et al. 2012, schroder, hugosson et al. 2012). as such, therapeutic management of localised prostate cancer has become complex, owing to comparable cure rates and varying risk profiles of each treatment option. common treatment options include active surveillance (as), radical prostatectomy (rp) and radiation therapy (by external beam or by radioactive seed implantation into the prostate, known as brachytherapy). newly diagnosed patients are often faced with a complex decision in selecting a treatment. they must consider a number of factors such as evidence for treatment efficacy, differing profiles of short and long-term side effects, inputs from physicians, family members, or friends, economic consequences, prior medical care experiences, as well as other influencing factors (zeliadt, ramsey et al. 2006). three types of decision-making are usually distinguished: 1) active or autonomous decision making, when the patient is solely responsible for the decision; 2) collaborative or shared australasian journal of information systems wang & ranasinghe 2017, vol 21, research on health information systems patient decision making in prostate cancer 2 decision making, defined as a process of engagement in which physician and the patient work together, with available information, to understand the clinical issues and determine the best management options; 3) a passive role of the patient implies that the physician reviews the options and makes the decision on behalf of the patient (fischer, visser et al. 2006). in clinical setting, the decision making process is often affected by time and resource constraints, severity of illness and limitation of treatment options (o’grady, jadad 2010). in this review we aim to compile the information from the studies that examine the different aspect of the decision making process in men with localised prostate cancer. by identifying and reviewing the current patient participation, pattern of information seeking, and different influences in treatment selection, we may recognise areas of deficiency and improve the decision making process for these patients. 2 diagnosis and treatment options for localised prostate cancer 2.1 diagnosis the diagnosis of prostate cancer involves digital rectal examination (dre), serum concentration of prostate specific antigen (psa) and prostate biopsy (heidenreich, bastian et al. 2014). the need for prostate biopsies should be determined on the basis of the psa level, a suspicious dre, and the patient’s biologic age and comorbidities. single elevated psa should not prompt immediate biopsy, except for high values of >20ng/ml (heidenreich, bastian et al. 2014). a persistently elevated or steadily rising psa is more indicative of underlying malignancy. ultrasound-guided transrectal or transperineal prostate biopsy has become the standard way to obtain histopathologic diagnosis. for each biopsy site, the proportion or length of tumour involvement of each biopsy core and the grading according to the gleason score system should be reported (heidenreich, bastian et al. 2014). further diagnostic and staging work-up is dependent on tumour grading and treatment options available to the patient. 2.2 active surveillance aim of as is to reduce overtreatment and the side effects of treatment such as incontinence and erectile dysfunction in patients with clinically confined low-risk pc. low-risk prostate cancer has been defined as gleason score of 6 or less, psa less than 10 mg/ml, and a tumour that is either non-palpable or only palpable in less than half of one lobe of the prostate (d'amico, whittington et al. 1995). in these patients with low risk pc, radical treatment is delayed but are followed regularly with screening tests and examination, and treated with curative intent if progression or the threat of progression is detected during follow-up (heidenreich, bastian et al. 2014). klotz et al (klotz, zhang et al. 2010) reported a cohort of 450 men with localised pc on active surveillance, and found disease-specific survival of 97.2% at 10-year follow-up and 30% of patients requiring curative treatment. several other studies have now been published confirming a low rate of progression and cancer-specific death in well-selected patients with very-low-risk disease, making as a reasonable approach in these patients (klotz 2010, klotz, zhang et al. 2010). however there are a number of negative aspects of as such as the anxiety associated with a diagnosis of pc, the need for regular psa testing and the complications associated with rebiopsy also place a significant burden on the healthcare system (crawford, black et al. 2010) while small scale studies have shown no increase in reported anxiety and good satisfaction rates amongst patients on as (burnet, parker et al. 2007, davison and goldenberg 2011), as well as reduced sepsis rates owing to the transition to transperineal biopsy (grummet, weerakoon et al. 2014), the decision for as is largely made on an individual basis. 2.3 radical prostatectomy in high risk localised or locally advanced pc radical prostatectomy still offers the best cancer specific survival benefit up to 92% at 10 years (hsu, joniau et al. 2007) and is advocated in men with intermediate or high risk localised or locally advanced pc life expectancy of >10 australasian journal of information systems wang & ranasinghe 2017, vol 21, research on health information systems patient decision making in prostate cancer 3 years. the increasingly popular nerve-sparing technique and robotic-assisted laparoscopic rp have seen improved urinary continence and erectile function in patients post-operatively (ficarra, novara et al. 2012, ficarra, novara et al. 2012). there are still, however, limited evidence to suggest an advantage of rp over other treatment options with regard to overall survival and metastasis-specific survival (heidenreich, bastian et al. 2014). 2.4 external beam radiation therapy external beam radiation therapy offers an alternative curative modality for those with localised pc who do not want to undergo surgery with comparable results to a surgery (potters, klein et al. 2004, nguyen, aizer et al. 2014). delivery of radiation treatment is much improved over the years in terms of accuracy, limiting exposure of surrounding tissue and rate long-term complications. three-dimensional conformal radiation therapy (3d-crt) is now the gold standard in external-beam radiation therapy (ebrt) in many institutions, with biochemical disease-free survival of 63% (heidenreich, bastian et al. 2014). 2.5 brachytherapy brachytherapy, which involves implanting radioactive seeds in the prostate, also reports good 5 and 10-year survival rates of up to 93% and 85% respectively, with low incidence of genitourinary and gastrointestinal toxicities (heidenreich, bastian et al. 2014). 2.6 morbidity comparison studies found that urinary dysfunction was more common in men treated with radical prostatectomy than in men treated with radiation therapy (wilt, shamliyan et al. 2008). investigators reported incontinence rates for brachytherapy (2% to 32%), radical prostatectomy (5% to 35%), and radiation therapy (2% to 6%) (wilt, macdonald et al. 2008). sexual dysfunction was common after all treatments. impotence rates ranged from less than 5% to approximately 60% in the few studies reporting on men undergoing nerve-sparing radical prostatectomy (wilt, macdonald et al. 2008). urethral stricture and haematuria were more frequent with radiation therapy. bowel complications such as diarrhoea, faecal incontinence, and rectal bleeding occurred less frequently in men treated with radiation therapy (15% to 30%) (wilt, macdonald et al. 2008). at 15-year follow-up, no significant relative differences in disease-specific functional outcomes were observed among men undergoing prostatectomy or radiotherapy (resnick, koyama et al. 2013). 3 literature search we conducted an english language review of published reports concerning patient participation in the decision-making process for treatment of localized prostate cancer. databases used for the search included pubmed and scopus® (100% medline coverage). the search was conducted up to year 2015. key words included ‘prostate cancer’, ‘patient participation’, ‘prostate cancer treatment’ and ‘decision making’. we restricted our review to original, peer-reviewed studies that focused on patient participation in the decision-making process. we excluded studies of decision making for prostate-specific antigen (psa) screening, studies of men with advanced prostate cancer and studies that focused on outcomes after treatment. as a result, we identified 42 articles that met our criteria. (figure 1) these include, 1 study examining influences of partners (zeliadt, penson et al. 2011), 2 studies examining influence of physicians (fowler, mcnaughton collins et al. 2000, stalmeier, van tol-geerdink et al. 2007), 16 studies comparing treatment preferences (mazur and hickam 1996, hall, boyd et al. 2003, auvinen, hakama et al. 2004, berry, ellis et al. 2006, zeliadt, ramsey et al. 2006, block, erickson et al. 2007, cox and amling 2008, van den bergh, van vugt et al. 2010, davison and goldenberg 2011, ihrig, keller et al. 2011, davison and breckon 2012, davison and breckon 2012, hamilton, wu et al. 2012, penson 2012, sidana, hernandez et al. 2012, van tolgeerdink, willem leer et al. 2013), 4 studies on decision aids (lin, aaronson et al. 2009, adsul, wray et al. 2015, violette, agoritsas et al. 2015, van tol-geerdink, leer et al. 2016), 7 studies australasian journal of information systems wang & ranasinghe 2017, vol 21, research on health information systems patient decision making in prostate cancer 4 examining role preferences (cohen and britten 2003, davison, goldenberg et al. 2003, davison, parker et al. 2004, fischer, visser et al. 2006, deber, kraetschmer et al. 2007, pieterse, henselmans et al. 2011, song, chen et al. 2013), 4 studies related to psychosocial issues of decision making (wong, stewart et al. 2000, kraetschmer, sharpe et al. 2004, steginga, turner et al. 2008, birnie and robinson 2010), 3 studies on demographic differences in decision making (meyer, talbot et al. 2007, xu, dailey et al. 2011, palmer, tooze et al. 2013), 2 studies examining decision making tools (berry, ellis et al. 2006, henrikson, davison et al. 2011), 1 cohort study on medical decision in general(fowler, gerstein et al. 2013), and 2 studies on multidisciplinary care (strebel, sulser et al. 2013, hurwitz, cullen et al. 2015). figure 1: flow chart of the recruitment of studies for review. studies of decision making for psa screening, studies of men with advanced prostate cancer and studies that focused on outcomes after treatment were excluded. 4 results 4.1 participation in treatment decision-making majority of the patients played either an active or a collaborative role in deciding the mode of treatment. only a minority of the patients preferred a passive role in the decision-making process. figures for men seeking active role in the decision making process range from 6-71%; shared decision making is in the range of 19 – 86; and 4-32% of the patients assumed a passive role in the decision making process (davison and goldenberg 2011, hurwitz, cullen et al. 2015). (table 1) meyer et al (meyer, talbot et al. 2007) has identified that younger men (<60 years) were more likely to be an active decision maker compared to men older than 70 years of age. however, older adults were more likely to make immediate decisions than younger adults (davison and goldenberg 2011). frequently family members have been reported to actively participate in the discussion of treatment options and this greatly facilitates shared decision-making among physicians, patient and partners (zeliadt, penson et al. 2011). the desire of the family members to be australasian journal of information systems wang & ranasinghe 2017, vol 21, research on health information systems patient decision making in prostate cancer 5 involved did not, however, indicate a desire to make the final treatment decision; and most partners were determined that the final decision should be made by the patient (zeliadt, penson et al. 2011). 4.2 information preference berry et al (berry, wang et al. 2012) identified that patients with diagnoses of pc used friends, the internet, and pamphlets from the specialist’s office as frequent information sources. similarly, xu et al (xu, dailey et al. 2011) conducted a small study of 21 men reported important information sources were professional advice from physicians, and family or friends, especially those with personal experience with prostate cancer. a number of articles, however, have identified urologists as the most important source of information and the major factor influencing the treatment decision (hall, boyd et al. 2003). (table 1) urologists are often the specialists who diagnose prostate cancer and provide the initial treatment consultation. therefore, it is not unusual that patients place great importance on recommendations offered at the first treatment discussion (davison and goldenberg 2011). palmer et al further identified that even though most patients sought information about prostate cancer from physicians, active participants were significantly more likely to also seek information from the internet (palmer, tooze et al. 2013). surprisingly, spouses and fellow patients were relatively minor influences in this process (hall, boyd et al. 2003). decision aids (da) are literature material developed to facilitate shared decision making by providing an understanding of the risks and benefits associated with treatment choices (violette, agoritsas et al. 2015). there has not been adequate evidence to suggest significant impact on patients’ decision making (violette, agoritsas et al. 2015). however, patient participation was found to be increased in the patient population provided with decision aids, as well as the likelihood of patients choosing brachytherapy (lin, aaronson et al. 2009, van tol-geerdink, willem leer et al. 2013). 4.3 reasons for selecting a treatment majorities of the studies investigated treatment options of active surveillance, surgery, brachytherapy and external beam radiotherapy (table 1). percentage of patients choosing radical prostatectomy have been quoted to be between 29-81.4%; 10.7 – 42% of patients elected to have radiation treatment and rate of active surveillance has been reported up to 42% (mazur and hickam 1996, hall, boyd et al. 2003, fischer, visser et al. 2006, ihrig, keller et al. 2011, sidana, hernandez et al. 2012). a descriptive pilot study conducted by hall et al (hall, boyd et al. 2003) described two major motivations underlying treatment decisions – best procedure to cure the cancer and treatment with best side effect profile. over 97% of the patients who elected to undergo rp believed that it was the best procedure to cure their cancer, compared to 59.7% of the patient who chose ebrt (hall, boyd et al. 2003). in contrast, only 1.2% of the patients chose rp for its side effect profiles, which was the predominate reason given in patient who chose brachytherapy (40%) (hall, boyd et al. 2003). patients undergoing surgery were younger and more likely to report a family history of prostate cancer (hurwitz, cullen et al. 2015). concerns over cancer control tend to drive patients towards surgical intervention and is a key barrier to acceptance of active surveillance (penson 2012). patients choosing ebrt tended to be older, african american, and have a major comorbidity (hurwitz, cullen et al. 2015). they also believed it offers equal efficacy of cure but fewer side effects than surgery (xu, dailey et al. 2011). patients who chose brachytherapy are more concerned with convenience of the procedure and less concerned with risk of sexual dysfunction (van tol-geerdink, willem leer et al. 2013). urinary incontinence and sexual dysfunction were most commonly voiced concerns in all treatment types. this is especially of concern in younger patient cohorts. these side effects of treatment, however, are not the most influential factor when it comes to choosing treatment – australasian journal of information systems wang & ranasinghe 2017, vol 21, research on health information systems patient decision making in prostate cancer 6 physicians’ recommendation and the treatment option that was considered to offer the best survival were most influential (table 1). 5 discussion currently there is no ‘gold standard’ of therapy for treatment of localised prostate cancer with the contemporary treatment options demonstrating similar survival rates with variable side effect profiles. treatment recommendation often differ according tumour grading, size and patient factors (heidenreich, bastian et al. 2014). an ideal decision-making process should involve providing patients with access to all relevant information in an unbiased manner, which encourages patients to make a decision according to their true preference. in reality, patient’s decision making are often skewed by both the content and the method in which the information was delivered. review of the literature has shown a greater percentage of patients actively or shared a collaborative role in the treatment decision-making process. active decision-making are seen more often in the younger men, who are more likely to choose the surgical pathway as they believe it offers the best chance of cure. in contrast, elderly patients often take up a passive role and are likely to choose non-surgical options. it is encouraging to see that patients are taking increasing interest in their cancer diagnosis and treatment, which may have contributed to the rising proportion of patients taking an active approach in the decision making process. however, the collaborative and the passive process are still often depended upon the clinicians’ values and preferences. this is evident in a qualitative study conducted by cohen et al (cohen and britten 2003) examining the decision-making models in nhs patients with localised prostate cancer respondents’ treatment plans were mostly decided on their behalf by their clinicians and that the patients’ barriers to shared decision making included fear of appearing disrespectful to their doctors and of taking responsibility for the outcome of treatment. similarly, a national survey of 2718 adults by fowler et al (fowler, gerstein et al. 2013) investigating patient autonomy in the decision making for 10 common medical decision showed that there are still considerable paternalism in this process. participants in the survey reported much more discussion of the pros than the cons of all tests or treatments and there are still clinical decisions being made without physician seeking patients’ input (fowler, gerstein et al. 2013). active or shared decision making, however, should be encouraged, as stalmeier et al (stalmeier, van tol-geerdink et al. 2007) pointed out that physicians often underestimated patients’ preferences for active participation as well as misjudged patient’s preference in treatment option. decision aids (da) are a popular tool increasingly adopted by various centres designed to promote patient involvement in decisions by providing a common understanding of the risks and benefits associated with treatment choices (violette, agoritsas et al. 2015). most da’s use multiple media to convey information; this includes videos, booklets, audio recordings and computer-based tools (violette, agoritsas et al. 2015). however, according to a systematic review by adsul et al (adsul, wray et al. 2015), majority of the decision aids reviewed did not meet the international criteria – they often lacked thorough description of risks, benefits and outcome, lacked encouragement for partner involvement, or lacked readability owing to its complexity in content and language. violette et al’s (violette, agoritsas et al. 2015) systematic review and meta-analysis of da from 14 trials revealed inconsistent effects on decision outcomes and demonstrated no effect on choice. this is was thought to be due to the fact that da were used to purely provide information rather than to facilitate shared decision making through use in a clinical encounter (violette, agoritsas et al. 2015). lin et al (lin, aaronson et al. 2009) reported, however, some non-randomized trials showed that das appear to increase the proportion of patients wanting and assuming a more active role in the treatment decisionmaking process. there is a general lack of information regarding how a patient’s personal values influence decision-making. a number of individualised risk calculators based on large datasets have been developed to aid treatment taking into account the risk of each patient. development of similar australasian journal of information systems wang & ranasinghe 2017, vol 21, research on health information systems patient decision making in prostate cancer 7 risk calculators with decision-making aids based on datasets maybe useful to tailor treatments for individuals. a recent article suggested that there are often multiple approaches for patients to process data, occasionally focusing only on a subset of the available information, or systematically analysing individual information to make a decision (polednak 1998). however, more research is needed to determine how personal values affect perceptions, biases, and the integration of information to eventually forming a decision on treatment. equipped with such data, an information system or software can potentially be developed, not only providing patients with the essential baseline information on treatment options, but also catering to patients’ diverse values and beliefs. such system may also be modified in terms of treatment outcomes and morbidity rates of each institution and clinician to give patients more accurate outcome expectations. in the current landscape, we believe that these patients may benefit most from a collaborative partnership with their clinicians in the process of deciding treatment options for localised prostate cancer. it would be difficult for any type of literature material in existence currently to cover all aspect of the process, tailored individual patient’s values and needs, as well as providing comprehensive information. therefore, a collaborative multidisciplinary approach is essential. basic requirement is the obligation to present and discuss all newly diagnosed patients with prostate cancer at the multidisciplinary tumour board (strebel, sulser et al. 2013). what may make this process even more effective is for patients to be counselled in a clinic following the tumour board meeting where the patient has the opportunity to communicate with specialists of different fields to gain better insight into the different treatment options and to voice their concerns and preferences. this may help to eliminate the bias of a single consultation with clinician who is likely to promote treatment of his field of expertise. limitation of current review is that the literature mostly consists of patient cohorts from european and north american populations. this hence is not an accurate representation of the australian cohort. further studies are required in australian centres on patient participation as well as current multidisciplinary practice for prostate cancer. 6 conclusion in the absence of a ‘gold standard’ therapy for localised prostate cancer, treatment recommendations to pursue active surveillance, radical prostatectomy, radiation therapy or brachytherapy are based on disease burden and patient preference. this process involves active participation from both the medical practitioner and the patient, and the decision making should be based on providing patients with relevant information in an unbiased fashion, allowing the patient to make an informed decision without being skewed by the content or the method of delivery. while studies have shown little benefit in generalised decision aids, a personalised decision making system based on datasets can provide increasing active participation of patients in the decision making process for treatment of localised prostate cancer. as the current evidence suggest that the personal aspects of the process and its influence on the final decision remain inadequately explored, future research is this area should be combined with ongoing study of multidisciplinary care and individualised risk based decision making tools which may provide a better pathway to assist patients reach decisions. australasian journal of information systems wang & ranasinghe 2017, vol 21, research on health information systems patient decision making in prostate cancer 8 author year no. patient role treatment demographics side effects of concern influential factor mazur et al(mazur and hickam 1996) 1996 140 96% active or shared 53% rp, 42% as >70 y.o. more likely to prefer as (59%) urinary incontinence complete tumour removal hall et al(hall, boyd et al. 2003) 2003 262 30.9% rp, 23.7% rt, 40.1% bt no significant association found urinary incontinence and erectile dysfunction best procedure to cure cancer fischer et al(fischer, visser et al. 2006) 2006 126 18% active, 60% shared, 22% passive 29% rp, 18% rt younger age associated with active decision making berry et al(berry, ellis et al. 2006) 2006 260 71% active, 19% shared, 10% passive for men >70yo less likely surgery; <70yo surgery and bt equally likely bladder and sexual symptoms longevity sidana et al(sidana, hernandez et al. 2012) 2011 488 52.3% shared, 45.8% active 81.4% rp, 10.7% rt, 5.3% as all pt under 50y.o. more married men chose rp or rt 90% sexual function, 84% urinary function, 66% bowel function doctor's recommendation xu et al(xu, dailey et al. 2011) 2011 21 90% rp or rt younger men and men with good health more likely to choose active treatment impotence survival ihrig et al(ihrig, keller et al. 2011) 2011 31 6% active, 86% shared, 6% passive 58% rp, 42% rt surgical pts are younger se associated with ebrt physician's advice davison et al(davison and goldenberg 2011) 2011 25 27% active, 41% shared, 32% passive men older than 70 prefer to play a passive role impact of treatment on urinary function urologists' recommendation davison et al(davison and breckon 2012) 2012 258 35% active, 38% shared, 27% passive 44% of men under 60 assume active role; 44% men older than 70 assume passive role urinary and sexual function urologists' recommendation davison et al(davison and breckon 2012) 2012 150 38% active, 52.7% shared, 9.3% passive 58.7% rp, 20% rt, 16% as, 0.7% cryosurgery younger men prefer to have active role in decision making urinary function survival palmer et al(palmer, tooze et al. 2013) 2012 181 44.2% active, 38.1 shared, 14.4% passive 63% rp, 17% rt 100% african-american population urinary incontinence and sexual dysfunction best chance of cure australasian journal of information systems wang & ranasinghe 2017, vol 21, research on health information systems patient decision making in prostate cancer 9 van tol-geerdink et al(van tolgeerdink, willem leer et al. 2013) 2013 240 71% rp no association with treatment received bowel and urinary side effects survival song et al(song, chen et al. 2013) 2013 788 41% active, 45% shared, 13% passive 67% rp younger men more likely to be active in the decision making process general physical impact of treatment pt's age hurwitz et al(hurwitz, cullen et al. 2015) 2015 925 68% active, 29% shared, 4% passive 54% rp, 20% rt, 12% as patients undergoing surgery were younger and more likely to report a family history of pc doctor's recommendation radical prostatectomy (rp); active surveillance (as); radiation therapy (rt); brachytherapy (bt); prostate cancer (pc) table 1: studies summarising patient participation and factors in decision-making. australasian journal of information systems wang & ranasinghe 2017, vol 21, research on health information systems patient decision making in prostate cancer 10 references adsul, p., r. wray, k. spradling, o. darwish, n. weaver and s. siddiqui (2015). "systematic review of decision aids for newly diagnosed patients with prostate cancer making treatment decisions." j urol 194(5): 1247-1252. aihw (2013). "prostate cancer in australia." cancer series 79. cat. no. can 76. canberra: aihw. auvinen, a., m. hakama, m. ala-opas, t. vornanen, m. leppilahti, p. salminen and t. l. tammela (2004). "a randomized trial of choice of treatment in prostate cancer: the effect of intervention on the treatment chosen." bju int 93(1): 52-56; discussion 56. berry, d. l., w. j. ellis, k. j. russell, j. c. blasko, n. bush, b. blumenstein and p. h. lange (2006). "factors that predict treatment choice and satisfaction with the decision in men with localized prostate cancer." clin genitourin cancer 5(3): 219-226. berry, d. l., q. wang, b. halpenny and f. hong (2012). "decision preparation, satisfaction and regret in a multi-center sample of men with newly diagnosed localized prostate cancer." patient educ couns 88(2): 262-267. birnie, k. and j. robinson (2010). "helping patients with localized prostate cancer reach treatment decisions." can fam physician 56(2): 137-141. block, c. a., b. erickson, c. carney-doebbling, s. gordon, b. fallon and b. r. konety (2007). "personality, treatment choice and satisfaction in patients with localized prostate cancer." int j urol 14(11): 1013-1018. burnet, k. l., c. parker, d. dearnaley, c. r. brewin and m. watson (2007). "does active surveillance for men with localized prostate cancer carry psychological morbidity?" bju int 100(3): 540-543. cohen, h. and n. britten (2003). "who decides about prostate cancer treatment? a qualitative study." fam pract 20(6): 724-729. cox, j. and c. l. amling (2008). "current decision-making in prostate cancer therapy." curr opin urol 18(3): 275-278. crawford, e. d., l. black, m. eaddy and e. j. kruep (2010). "a retrospective analysis illustrating the substantial clinical and economic burden of prostate cancer." prostate cancer prostatic dis 13(2): 162-167. d'amico av, whittington r, malkowicz sb, schultz d, schnall m, tomaszewski je, wein a (1995). a multivariate analysis of clinical and pathological factors that predict for prostate specific antigen failure after radical prostatectomy for prostate cancer. j urol 154:131–8. doi: 10.1016/s0022-5347(01)67248-3. davison, b. j. and e. breckon (2012). "factors influencing treatment decision making and information preferences of prostate cancer patients on active surveillance." patient educ couns 87(3): 369-374. davison, b. j. and e. n. breckon (2012). "impact of health information-seeking behavior and personal factors on preferred role in treatment decision making in men with newly diagnosed prostate cancer." cancer nurs 35(6): 411-418. davison, b. j. and s. l. goldenberg (2011). "patient acceptance of active surveillance as a treatment option for low-risk prostate cancer." bju int 108(11): 1787-1793. davison, b. j., s. l. goldenberg, m. e. gleave and l. f. degner (2003). "provision of individualized information to men and their partners to facilitate treatment decision making in prostate cancer." oncol nurs forum 30(1): 107-114. australasian journal of information systems wang & ranasinghe 2017, vol 21, research on health information systems patient decision making in prostate cancer 11 davison, b. j., p. a. parker and s. l. goldenberg (2004). "patients' preferences for communicating a prostate cancer diagnosis and participating in medical decisionmaking." bju int 93(1): 47-51. deber, r. b., n. kraetschmer, s. urowitz and n. sharpe (2007). "do people want to be autonomous patients? preferred roles in treatment decision-making in several patient populations." health expect 10(3): 248-258. ficarra, v., g. novara, t. e. ahlering, a. costello, j. a. eastham, m. graefen, g. guazzoni, m. menon, a. mottrie, v. r. patel, h. van der poel, r. c. rosen, a. k. tewari, t. g. wilson, f. zattoni and f. montorsi (2012). "systematic review and meta-analysis of studies reporting potency rates after robot-assisted radical prostatectomy." eur urol 62(3): 418430. ficarra, v., g. novara, r. c. rosen, w. artibani, p. r. carroll, a. costello, m. menon, f. montorsi, v. r. patel, j. u. stolzenburg, h. van der poel, t. g. wilson, f. zattoni and a. mottrie (2012). "systematic review and meta-analysis of studies reporting urinary continence recovery after robot-assisted radical prostatectomy." eur urol 62(3): 405417. fischer, m., a. visser, b. voerman, b. garssen, g. van andel and j. bensing (2006). "treatment decision making in prostate cancer: patients' participation in complex decisions." patient educ couns 63(3): 308-313. fowler, f. j., jr., b. s. gerstein and m. j. barry (2013). "how patient centered are medical decisions?: results of a national survey." jama intern med 173(13): 1215-1221. fowler, f. j., jr., m. mcnaughton collins, p. c. albertsen, a. zietman, d. b. elliott and m. j. barry (2000). "comparison of recommendations by urologists and radiation oncologists for treatment of clinically localized prostate cancer." jama 283(24): 3217-3222. grummet, j. p., m. weerakoon, s. huang, n. lawrentschuk, m. frydenberg, d. a. moon, m. o'reilly and d. murphy (2014). "sepsis and 'superbugs': should we favour the transperineal over the transrectal approach for prostate biopsy?" bju int 114(3): 384388. hall, j. d., j. c. boyd, m. c. lippert and d. theodorescu (2003). "why patients choose prostatectomy or brachytherapy for localized prostate cancer: results of a descriptive survey." urology 61(2): 402-407. hamilton, a. s., x. c. wu, j. lipscomb, s. t. fleming, m. lo, d. wang, m. goodman, a. ho, j. b. owen, c. rao and r. r. german (2012). "regional, provider, and economic factors associated with the choice of active surveillance in the treatment of men with localized prostate cancer." j natl cancer inst monogr 2012(45): 213-220. heidenreich, a., p. j. bastian, j. bellmunt, m. bolla, s. joniau, t. van der kwast, m. mason, v. matveev, t. wiegel, f. zattoni, n. mottet and u. european association of (2014). "eau guidelines on prostate cancer. part 1: screening, diagnosis, and local treatment with curative intent-update 2013." eur urol 65(1): 124-137. heijnsdijk, e. a., e. m. wever, a. auvinen, j. hugosson, s. ciatto, v. nelen, m. kwiatkowski, a. villers, a. paez, s. m. moss, m. zappa, t. l. tammela, t. makinen, s. carlsson, i. j. korfage, m. l. essink-bot, s. j. otto, g. draisma, c. h. bangma, m. j. roobol, f. h. schroder and h. j. de koning (2012). "quality-of-life effects of prostate-specific antigen screening." n engl j med 367(7): 595-605. henrikson, n. b., b. j. davison and d. l. berry (2011). "measuring decisional control preferences in men newly diagnosed with prostate cancer." j psychosoc oncol 29(6): 606-618. australasian journal of information systems wang & ranasinghe 2017, vol 21, research on health information systems patient decision making in prostate cancer 12 hsu, c. y., s. joniau, r. oyen, t. roskams and h. van poppel (2007). "outcome of surgery for clinical unilateral t3a prostate cancer: a single-institution experience." eur urol 51(1): 121-128; discussion 128-129. hurwitz, l. m., j. cullen, s. elsamanoudi, d. j. kim, j. hudak, m. colston, j. travis, h. c. kuo, c. r. porter and i. l. rosner (2015). "a prospective cohort study of treatment decision-making for prostate cancer following participation in a multidisciplinary clinic." urol oncol. ihrig, a., m. keller, m. hartmann, j. debus, j. pfitzenmaier, b. hadaschik, m. hohenfellner, w. herzog and j. huber (2011). "treatment decision-making in localized prostate cancer: why patients chose either radical prostatectomy or external beam radiation therapy." bju int 108(8): 1274-1278. jemal, a., r. siegel, e. ward, t. murray, j. xu, c. smigal and m. j. thun (2006). "cancer statistics, 2006." ca cancer j clin 56(2): 106-130. klotz, l. (2010). "active surveillance for prostate cancer: a review." curr urol rep 11(3): 165171. klotz, l., l. zhang, a. lam, r. nam, a. mamedov and a. loblaw (2010). "clinical results of long-term follow-up of a large, active surveillance cohort with localized prostate cancer." j clin oncol 28(1): 126-131. kraetschmer, n., n. sharpe, s. urowitz and r. b. deber (2004). "how does trust affect patient preferences for participation in decision-making?" health expect 7(4): 317-326. lin, g. a., d. s. aaronson, s. j. knight, p. r. carroll and r. a. dudley (2009). "patient decision aids for prostate cancer treatment: a systematic review of the literature." ca cancer j clin 59(6): 379-390. mazur, d. j. and d. h. hickam (1996). "patient preferences for management of localized prostate cancer." west j med 165(1-2): 26-30. meyer, b. j., a. p. talbot and c. ranalli (2007). "why older adults make more immediate treatment decisions about cancer than younger adults." psychol aging 22(3): 505-524. nguyen, p. l., a. aizer, d. g. assimos, a. v. d'amico, s. j. frank, a. r. gottschalk, g. s. gustafson, i. c. hsu, p. w. mclaughlin, g. merrick, s. a. rosenthal, t. n. showalter, a. v. taira, n. vapiwala, y. yamada, b. j. davis and e. p. o. r. o. american college of radiology (2014). "acr appropriateness criteria(r) definitive external-beam irradiation in stage t1 and t2 prostate cancer." am j clin oncol 37(3): 278-288. o'grady, l and jadad a (2010). "shifting from shared to collaborative decision making: a change in thinking and doing." j participat med 2:e13. palmer, n. r., j. a. tooze, a. r. turner, j. xu and n. e. avis (2013). "african american prostate cancer survivors' treatment decision-making and quality of life." patient educ couns 90(1): 61-68. penson, d. f. (2012). "factors influencing patients' acceptance and adherence to active surveillance." j natl cancer inst monogr 2012(45): 207-212. pieterse, a. h., i. henselmans, h. c. de haes, c. c. koning, e. d. geijsen and e. m. smets (2011). "shared decision making: prostate cancer patients' appraisal of treatment alternatives and oncologists' eliciting and responding behavior, an explorative study." patient educ couns 85(3): e251-259. polednak, a. p. (1998). "prostate cancer treatment in black and white men: the need to consider both stage at diagnosis and socioeconomic status." j natl med assoc 90(2): 101104. australasian journal of information systems wang & ranasinghe 2017, vol 21, research on health information systems patient decision making in prostate cancer 13 potters, l., e. a. klein, m. w. kattan, c. a. reddy, j. p. ciezki, a. m. reuther and p. a. kupelian (2004). "monotherapy for stage t1-t2 prostate cancer: radical prostatectomy, external beam radiotherapy, or permanent seed implantation." radiother oncol 71(1): 29-33. resnick, m. j., t. koyama, k. h. fan, p. c. albertsen, m. goodman, a. s. hamilton, r. m. hoffman, a. l. potosky, j. l. stanford, a. m. stroup, r. l. van horn and d. f. penson (2013). "long-term functional outcomes after treatment for localized prostate cancer." n engl j med 368(5): 436-445. schroder, f. h., j. hugosson, m. j. roobol, t. l. tammela, s. ciatto, v. nelen, m. kwiatkowski, m. lujan, h. lilja, m. zappa, l. j. denis, f. recker, a. paez, l. maattanen, c. h. bangma, g. aus, s. carlsson, a. villers, x. rebillard, t. van der kwast, p. m. kujala, b. g. blijenberg, u. h. stenman, a. huber, k. taari, m. hakama, s. m. moss, h. j. de koning and a. auvinen (2012). "prostate-cancer mortality at 11 years of follow-up." n engl j med 366(11): 981-990. sidana, a., d. j. hernandez, z. feng, a. w. partin, b. j. trock, s. saha and j. i. epstein (2012). "treatment decision-making for localized prostate cancer: what younger men choose and why." prostate 72(1): 58-64. song, l., r. c. chen, j. t. bensen, g. j. knafl, m. e. nielsen, l. farnan, e. m. wallen, m. mishel, r. s. pruthi, j. l. mohler and p. a. godley (2013). "who makes the decision regarding the treatment of clinically localized prostate cancer--the patient or physician?: results from a population-based study." cancer 119(2): 421-428. stalmeier, p. f., j. j. van tol-geerdink, e. n. van lin, e. schimmel, h. huizenga, w. a. van daal and j. w. leer (2007). "doctors' and patients' preferences for participation and treatment in curative prostate cancer radiotherapy." j clin oncol 25(21): 3096-3100. steginga, s. k., e. turner and j. donovan (2008). "the decision-related psychosocial concerns of men with localised prostate cancer: targets for intervention and research." world j urol 26(5): 469-474. strebel, r. t., t. sulser, h. p. schmid, s. gillessen, m. fehr, u. huber, m. pless, r. morant, r. winterhalder and r. cathomas (2013). "multidisciplinary care in patients with prostate cancer: room for improvement." support care cancer 21(8): 2327-2333. van den bergh, r. c., h. a. van vugt, i. j. korfage, e. w. steyerberg, m. j. roobol, f. h. schroder and m. l. essink-bot (2010). "disease insight and treatment perception of men on active surveillance for early prostate cancer." bju int 105(3): 322-328. van tol-geerdink, j. j., j. w. leer, c. j. wijburg, i. m. van oort, h. vergunst, e. j. van lin, j. a. witjes and p. f. stalmeier (2016). "does a decision aid for prostate cancer affect different aspects of decisional regret, assessed with new regret scales? a randomized, controlled trial." health expect 19(2): 459-470. van tol-geerdink, j. j., j. willem leer, p. c. weijerman, i. m. van oort, h. vergunst, e. n. van lin, j. alfred witjes and p. f. stalmeier (2013). "choice between prostatectomy and radiotherapy when men are eligible for both: a randomized controlled trial of usual care vs decision aid." bju int 111(4): 564-573. violette, p. d., t. agoritsas, p. alexander, j. riikonen, h. santti, a. agarwal, n. bhatnagar, p. dahm, v. montori, g. h. guyatt and k. a. tikkinen (2015). "decision aids for localized prostate cancer treatment choice: systematic review and meta-analysis." ca cancer j clin 65(3): 239-251. wilt, t. j., r. macdonald, i. rutks, t. a. shamliyan, b. c. taylor and r. l. kane (2008). "systematic review: comparative effectiveness and harms of treatments for clinically localized prostate cancer." ann intern med 148(6): 435-448. australasian journal of information systems wang & ranasinghe 2017, vol 21, research on health information systems patient decision making in prostate cancer 14 wilt, t. j., t. shamliyan, b. taylor, r. macdonald, j. tacklind, i. rutks, k. koeneman, c. s. cho and r. l. kane (2008). comparative effectiveness of therapies for clinically localized prostate cancer. rockville (md). wong, f., d. e. stewart, j. dancey, m. meana, m. p. mcandrews, t. bunston and a. m. cheung (2000). "men with prostate cancer: influence of psychological factors on informational needs and decision making." j psychosom res 49(1): 13-19. xu, j., r. k. dailey, s. eggly, a. v. neale and k. l. schwartz (2011). "men's perspectives on selecting their prostate cancer treatment." j natl med assoc 103(6): 468-478. zeliadt, s. b., d. f. penson, c. m. moinpour, d. k. blough, c. r. fedorenko, i. j. hall, j. l. smith, d. u. ekwueme, i. m. thompson, t. e. keane and s. d. ramsey (2011). "provider and partner interactions in the treatment decision-making process for newly diagnosed localized prostate cancer." bju int 108(6): 851-856; discussion 856-857. zeliadt, s. b., s. d. ramsey, d. f. penson, i. j. hall, d. u. ekwueme, l. stroud and j. w. lee (2006). "why do men choose one treatment over another?: a review of patient decision making for localized prostate cancer." cancer 106(9): 1865-1874. copyright: © 2017 wang & ranasinghe. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ the australasian journal of information systems (ajis) began in 1993 and was based on the simple realisation that there was a need for an academic ‘outlet’ for is research in australia australasian journal of information systems volume 18 number 2 2014 111 agility in consumer retail: sense-response alignment through the eyes of customers maura atapattu queensland university of technology m.atapattu@qut.edu.au darshana sedera queensland university of technology t. ravichandran lally school of management abstract in hyper competition, firms that are agile: sensing and responding better to customer requirements tend to be more successful and achieve supernormal profits. in spite of the widely accepted importance of customer agility, research is limited on this construct. the limited research also has predominantly focussed on the firm’s perspective of agility. however, we propose that the customers are better positioned to determine how well a firm is responding to their requirements (aka a firm’s customer agility). taking the customers’ stand point, we address the issue of sense and respond alignment in two perspectives-matching and mediating. based on data collected from customers in a field study, we tested hypothesis pertaining to the two methods of alignment using polynomial regression and response surface methodology. the results provide a good explanation for the role of both forms of alignment on customer satisfaction. implication for research and practice are discussed. keywords: agility, agility alignment, sense and response, u-crm, polynomial regression introduction agility is emerging as an important determinant of success in contemporary competitive business environments, where achieving sustained competitive advantage is said to be difficult and elusive (roberts and grover 2012a; tallon and pinsonneault 2011). firms nowadays are striving towards constant and effective sensing of customer needs and requirements in order to respond to customers in a tailored manner (atapattu and sedera 2013b). in other words, sensing essentially provides customer intelligence which can help a firm’s responsive actions on unique customer requirements. hence, firms are investing heavily in a multitude of technological tools and platforms – mobile apps being one of the prominent examples of this. firms would like their customers to mimic their daily routines of purchasing and purchase considerations through such mobile apps (lamarre et al. 2012) as with smart functions of smart mobile technology (shankar et al. 2010), retailers have the potential to dynamically create and launch customer-focused activities with a high degree of precision. as narayanaswami et al. (2011) noted, smart devices and applications are heavily influencing the retailing landscape, making global shifts towards ‘everywhere – ubiquitous – retailing’, and ‘everywhere ubiquitous sensing and responding’. as per a recent study of key it and management trends in 2011-2012 customer intelligence gained through smart devices appeared among the top five most influential management technologies australasian journal of information systems volume 18 number 2 2014 112 worldwide (luftman et al. 2012). in australia, the two major retailers (woolworths with 41.1% of market share and $48.56 billion revenue, and coles with 31% market share and $34.1 billion revenue in 2012) both launched mobile shopping apps in 2011 with the intent of sensing customer requirements better, by connecting ubiquitously to access their daily routines 24x7 (atapattu and sedera 2013a). thus far, over 3 million customers (1.8 million woolworths customers and over 1.5 million coles customers) have downloaded the apps from android® and apple® markets which is about 14% of the australian population. consequently, when woolworths acquired a business intelligence company “quantium” in 2013 citing the importance of better analyzing the shopping habits of australians outside its stores (kohler 2013), coles recently relaunched its customer loyalty scheme “flybuys” with the help of an expert analytics team brought from the uk (robin 2013). on the other hand, customers are aware of the ‘sensing’ in which retailers engage through mobile apps. in return, they expect that the retailers will respond swiftly to their unique requirements (atapattu and sedera 2014). so, they expect highly personalized company-to-consumer responses (kaplan 2012). in other words the customers evaluate how agile a firm towards their unique requirements based on their actual experiences. whilst there is diversity in consumer mobile apps use; thus, the extent and wider use of the app and adoption of it is of great importance to the retailer. however, failing to provide adequate, efficient and effective responses could lead to low customer satisfaction and disengagement (choi and mattila 2008) from the mobile app. as this innovative customer engagement discussed here allows firms to understand customer requirements better (sensing), whilst customers expect that their ‘unique’ requirements are identified by the retailer (responding). this focused scenario of customer sensing through mobile apps in consumer retail, we argue that customers could effectively judge the responding capacity of the firm. it provides the customers’ perspective of firm’s agility which is missing in the current notion of agility literature. this view is unique in a number of different ways. most studies have observed agility from the firm’s perspective, where prior research elucidates two forms of sense-response alignments, namely, matching and mediation, on firm performance from the firm’s perspective (roberts and grover 2012a). also, past studies in the management discipline have alluded to the importance of the customer’s perspective in firm performance (kaplan and norton 1996). despite the importance of customer perceptions to contemporary firms in dynamic markets, little is known about the firm’s customer agility from the customer’s viewpoint. in addition, we argue akin to several past studies in marketing and consumer behavior (gao et al. 2010; kaplan 2012; lamarre et al. 2012; rohm et al. 2012; shankar et al. 2010) who demonstrated that customers as the users of a mobile app would have a reasonable knowledge of the sensing capabilities of such apps. therefore, we position that the customers could effectively judge the responding capacity of the firm through their perception of the firm’s responsiveness. the primary objective of this research is to demonstrate the importance of alignment between sensing and responding capabilities in the firm’s agility for achieving superior customer satisfaction. in other words, our main premise in this study here is not just to look at the sensing and/or responding capabilities per se but the alignment between the two from the customers’ perspective. while past studies have conceptualized agility in terms of structures and processes that enable firms to sense and respond, we take a different viewpoint in this research and seek to examine agility from the customers’ perspective. this paper investigates the notion of sense-response misalignment from the customers’ point of view in the context ubiquitous customer relationship management through smart mobile apps, which we label as ubiquitous crm (u-crm). we introduce the term u-crm for mobile apps here simply because, the mobile apps we refer in this discussion are a key component of the firm’s customer relationship management system which they use to connect with their customers ubiquitously to sense and respond to their customer unique requirements. additionally, the agility alignment has never been viewed from the customer’s angle. we argue here that, this discussion of customers’ perspective of firm’s agility is now long overdue. possibly, there australasian journal of information systems volume 18 number 2 2014 113 could be nuanced ways in which customers experience the firm’s ability in aligning sense-response capabilities, which in turn could impact on firm performance. thus, our second objective is to examine how a firm’s customer agility impacts on firm performance. more specifically, how does the alignment between a firm’s sensing and responding capabilities impact its performance (through customer satisfaction)? the paper first defines agility before invoking the importance of understanding the characteristics of firm agility from the customer’s standpoint. we then discuss customer agility in light of nomological discussion around other related concepts before developing our theoretical model. building on this discussion, we then formulate and present our research hypothesis. then, we describe our empirical research design and the testing of our research model. finally, we discuss our findings and limitations, implications for both research and practice, and potential future research directions. theoretical background studies of organizational agility (haeckel 1999; nazir and pinsonneault 2012; roberts and grover 2012a; zaheer and zaheer 1997) provide the theoretical foundation to this study. overby et al. (2006) have defined enterprise agility in its broadest sense as the firm’s ability to sense environmental change and respond rapidly, with several others defining agility in a similar manner (e.g. roberts and grover 2012a), with two common attributes describing it – sense and response (overby et al. 2006; roberts and grover 2012a; roberts and grover 2012b; sambamurthy et al. 2003). sensing represents the knowledge component of agility (overby et al. 2006) and reflects the firm’s ability to find appropriate opportunities and/or threats to act upon (dove 2001). responding describes the firm’s ability to act quickly and accurately on opportunities and/or threats (dove 2001; overby et al. 2006). sense and response explain how firms respond to opportunities and threats with speed, ease and dexterity (overby et al. 2006). a firm may display its agility in many different areas such as customer-based processes, supply chain interactions, and day-to-day operations (huang et al. 2012; roberts and grover 2012b; yusuf et al. 1999a). alignment of sensing and responding researchers argue that sense and response should be aligned (overby et al. 2006; roberts and grover 2012b). these two capabilities (overby et al. 2006; roberts and grover 2012a; roberts and grover 2012b) are complementary, yet distinctively different (roberts and grover 2012a). as previous studies suggest (overby et al. 2006; roberts and grover 2012a), simply possessing the capabilities of sensing and responding themselves do not lead to competitive advantage. sensing and responding alignment provides a well synchronized agile capability whereby a firm is able to detect a breadth of customer-based opportunities that they are able to respond to in a timely and effective manner (overby et al. 2006). on the other hand, sense-response misalignment can bring about severe consequences (strandholm et al. 2004), as described in the well-documented woolworths case study in the 1990s (roberts and grover 2012a). woolworths in the us initially failed to sense the growth of suburbs and the migration of shopping away from the urban centers in which most of the woolworths stores were located. they also had a mismatch between their merchandise and what their customers, who continued to shop downtown, wanted. their ultimate attempt to rebrand themselves was too late and ineffective, leading to the demise of all of their us stores by 1997. in another example, the bookstore borders failed to sense the shift to online retailing in the late 1990s and didn’t have the ability to respond once it sensed the shift (business week, 3 apr, 2000, p. 75). the point here is that if a firm is unable to sense in a timely manner, it might be too difficult for them to respond when they need to. quintessentially, the limitations in responding have to be mitigated through effective sensing. australasian journal of information systems volume 18 number 2 2014 114 the notion of alignment can be represented as one that is “matching” or “mediated” (overby et al. 2006; roberts and grover 2012a; venkatraman 1989). the matching perspective specifies the theoretically defined match between two related variables (in our discussion, these are sense and response) (venkatraman 1989). hence, if the match between the two (sense and response) variables is stronger, and both variables (sensing and responding) score “highly”, the greater the effect of customer agility on a relevant criterion variable (roberts and grover 2012a). consequently, the mediation perspective embraces the presence of noticeable intervening mechanisms between an antecedent variable and a dependent variable (venkatraman 1989). hence, the mediation perspective provides insights into the sense-response-performance process(roberts and grover 2012a). capturing the mediation perspective from the customer’s angle would provide insights into the process sense experience (customers’)-satisfaction. this perspective of viewing the firm’s agility through the customer’s lens can provide deeper insights into the relationship between agility and firm performance from a different yet important angle. agility from customer’s standpoint there are several studies in information systems that focus on firms’ agility to sense and respond to customers (as opposed to competitors). for example, roberts and grover (2012b) defined a firm’s customer agility as the “degree to which a firm is able to sense and respond quickly to customer-based opportunities for innovation and competitive action”. as mentioned earlier, traditional agility studies in information systems research observe agility of the firm by asking the management about the firm’s sensing and responding capabilities, that is, by asking managers what they think are their capabilities of sensing and responding. for example, in a study on market orientation (narver et al. 2004) and customer agility (roberts and grover 2012a; roberts and grover 2012b), the questions were directed to the firm’s middle management to comment on the sensing abilities of the company. similarly, the responding capabilities were measured by asking the responding managers about whether they thought the organization was responding adequately to customer demands. in studies like narver et al. (2004) and (roberts and grover 2012b), customer-centricity is implied – yet not measured from the customer point of view. for example, respondents guess the perceived responsiveness of the firm from the point of view of customers. in general, firms are in a better position to understand the amount of sensing, and customers are in a better position to inform us whether the firm is responding to their needs. in the present research, we observe the amount of sensing done through customers’ mobile devices. this phenomenon is directly linked with the use of mobile apps (discussed in the ‘model development’ section of this paper in more detail). in this discussion, we look at this issue from the customers’ point of view, where we define firm’s agility as the “degree to which a customer perceives a firm’s ability to sense and respond quickly to his/her changing needs and wants”. this notion of the customer’s perspective of agility is described below in detail. sensing customer agility includes scanning, learning and interpreting shifting customer requirements and customer-based opportunities (rapp et al. 2010; roberts and grover 2012a; teece 2007a), where responsiveness is related to the execution of customer-focused competitive actions through an assembly of a firm’s existing processes or services (jayachandran et al. 2004; roberts and grover 2012b; teece 2007a). following which, the customers can recognize the firm’s responding capabilities to their needs. in other words, a customer’s perception about the firm’s responsiveness is a valid indicator of the firm’s responsiveness. based on this reasoning, we argue that the customers are well positioned to evaluate firms’ customer responsiveness (firms’ customer agility). hence, we suggest that the customerperceived firm responsiveness is an important component in understanding agility. however, the existing research predominantly makes use of the managers of the individual firm to estimate the firm’s australasian journal of information systems volume 18 number 2 2014 115 responsiveness (roberts and grover 2012b). our intention here is not to discount the managers’ role in assessing the firm’s responsiveness, but to highlight the importance of customers’ participation in a firm’s responsiveness audit. we argue that the understanding of firms’ customer agility is incomplete without comprehending firm agility from the customers’ standpoint as the organizational perspective provides the internal view of the firm’s customer agility, and the proposed customer perspective could provide the much required external view of the firm’s customer agility. in the is in new paradigm (ubiquitous-is), customers are an important group of users. techno-savvy customers are increasingly making their shopping interactions online in digitized environments using contemporary applications such as u-crm. drawing from our example of the use of u-crm in two of the largest retailers in australia, we posit that they are constantly and continuously sensing the customer requirements through their engagement with u-crm. when customers create shopping lists, check fuel prices, and search recipes, products and specials using u-crm, the organizations are able to get rich insights about individual customer preferences as such activities leave digital information footprints (chi et al. 2010). the more the customers use the u-crm, the more opportunities are created for organizations to learn about individual customers. as increased use of u-crm provides more opportunities for customer sensing, the extent of u-crm use becomes a proxy for a firm’s sensing capability. however, the firm cannot sense their customer needs unless the customers use the application. hence, in this discussion, we surrogate firm’s customer sensing to the degree to which the customers use the u-crm. following the customer’s use of u-crm, firms are getting to know more about individual customer preferences, and then the firm is able to respond to the needs of individual customers with ease, speed and deftness, when the firm is agile (nazir and pinsonneault 2012; overby et al. 2006; roberts and grover 2012a; roberts and grover 2012b). thus, the customers are able to recognize that the firm is responsive to their individual needs. in other words, customers are able to perceive firm’s responding capabilities based on what they actually experience. customer-perceived firm’s responsiveness then influences customer satisfaction. research model, measurement and hypotheses development following the earlier notions of agility alignment (overby et al. 2006; roberts and grover 2012a), we propose a research model (figure 1) of sensing, responding and customer satisfaction. underlying the model is the assumption that ‘customer satisfaction’ will be at its highest when a firm’s sensing and responding capabilities are aligned. figure 1: conceptual model: customer’s use of ubiquitous mobile-crm, perceived responsiveness, and customer satisfaction based on the argument that increased u-crm use by customers increases the opportunities for an organization to carry out customer sensing, we conceptualize that the extent to which a customer uses the u-crm app is a surrogate measure for the firm’s sensing in this research. following the notions of agility alignment (roberts and grover 2012a), a firm is now able to execute customer-focused competitive actions based on what they have sensed about their customers through the customers’ use of u-crm. as the firm’s responsive actions are based on the intelligence obtained by the firm on what sensing (customer's use of u-crms) responding (customer perceived responsiveness) customer satisfaction agility alignment australasian journal of information systems volume 18 number 2 2014 116 customers want through their use of u-crm, customers are now able to recognize that the firm is responding to their specific requirements through the actual experience they perceive. in other words, a firm’s responsiveness is reflected in customer-perceived firm’s responsiveness. the actual experience of the customers then determines the level of customer satisfaction, since customers perceive that the firm is responding to their unique requirements. thus, we propose our conceptual model as above ( figure 1). we followed the guidelines recommended by churchill (1979) for developing the measures of our constructs. we first conducted a literature search on firms’ customer agility to specify the domain of the construct (see appendix a). following the definition of firms’ customer agility, we further reviewed the literature to develop sub-measures and measurement items for each construct of firms’ customer agility based on our conception. following this, we generated sample items based on this literature review and validated measures of similar constructs (e.g..barki et al. 2007; bhattacherjee 2001; jayachandran et al. 2004). where possible, the existing measures of constructs were adapted to the context of this study. for new measures and those that required significant changes, we followed the standard scale development procedures stipulated in mackenzie et al. (2011). we then conducted a pretest with a sample of ten fellow researchers some of who were an actual users of mobile apps for shopping and a pilot study comprising of 30 customers who actually uses the mobile shopping apps to assess the reliability and validity of our measures. our pilot analysis and subsequent follow-up discussion with a subset of respondents provided sufficient confidence in the scales to proceed with the full-scale survey administration of the target sample frame. our measures of customer satisfaction and responsiveness have been adopted from (bhattacherjee 2001) and (roberts and grover 2012a), respectively. although measures for sensing are available from past studies, all of those measures assessed sensing from the organization’s point of view. given our rationale for employing use measures as surrogates of sensing, appropriate measures were identified from barki et al. (2007); burton-jones and straub (2006) and venkatesh et al. (2008) where they have either used similar constructs or provided guidelines for developing new measures for similar constructs. appendix b presents the discussion of the formative/reflective nature of the measures, and appendix c lists the constructs, items and the sources for all construct measures. agility alignment as illustrated in overby et al. (2006), the agility alignment in general answers the question on whether a firm senses opportunities in areas where it possesses the capabilities to respond, or whether it senses opportunities beyond which the firm is able to respond. here, the firm is aligned if they seldom waste their capabilities, either by possessing responding capabilities that lie unused or by sensing opportunities that cannot be seized (overby et al. 2006). in other words, the firm only senses those opportunities to which it can respond, or the firm develops capabilities that are useful only for those opportunities it could sense. in contrast, non-aligned firms may hold responding capabilities that do not apply to the opportunities they sense, or they do not own responding capabilities to support the breadth of opportunities they sense. as discussed above, a firm can sense more, but may choose to only respond to a subset of what it has sensed. sensing essentially provides intelligence which can help a firm’s responsive actions. in this study, we define sensing in terms of the features that firms have provided in the apps. this is a case where a firm might have built capabilities to acquire customer preferences but may not have the capacity to act upon this information. in fact, sensing more could be problematic for firms as customers may raise their expectations and ask for more from the firm. prior research on agility alignment (roberts and grover 2012a) proposes that a firm’s customer agility impacts its performance; but, to gain optimum effect of customer agility on firm performance, the firm needs to align its sensing and responding capabilities (haeckel 1999; overby et al. 2006; teece 2007a). overby et al. (2006) suggested that the enterprise agility should be measured as a function of its sensing australasian journal of information systems volume 18 number 2 2014 117 and responding capabilities, and they oppose measuring enterprise agility as a direct measurement. in other words, to create an overall measurement, the two capabilities of sensing and responding should be measured individually and separately and then combined to get the actual agility assessment. therefore, the overall agility score will be dependent on the functional relationship linking the two sensing and responding sub-scores (or the alignment) (overby et al. 2006). further, they stated that agility alignment is better described by the degree to which the two sense-response capabilities are aligned in a continuous, rather than a binary scale. the core idea of alignment is not that the firm is merely aligned or non-aligned, but that the alignment is attained somewhere in a continuum between the two. alignment in general refers to “the degree to which the needs, demands, goals, objectives, and/or structures of one component are consistent with the needs, demands, goals, objectives, and/or structures of another component” (nadler and tushman 1983). as roberts and grover (2012a) discussed, in the context of customer agility, alignment refers to the degree to which the structures and objectives of a firm’s customer sensing capability should be consistent with the structures and objectives of its customer responding capability. researchers recommend considering multiple specifications as competing theories or models suggest, when considering the most appropriate perspective of alignment for a given research question (venkatraman 1989). roberts and grover (2012a), for example, studied sense-response alignment with matching and mediation perspectives from firms’ perspective with the sense, response and firm performance data obtained from the marketing managers of firms. in this paper, we investigate the alignment between firms’ customer sensing capability and customer responding capability in terms of two perspectives, namely, matching and mediation, from the customers’ perspective of agility. alignment as matching matching refers to the theoretical match between two related variables wherein the basic tenet is that the stronger the match between customer sensing capability and customer responding capability, the greater the effect of customer agility on an appropriate criterion variable (overby et al. 2006; roberts and grover 2012a). accordingly, firms are likely to extract greater value from their customer agility when they are aligned in their sensing and responding activities. this is clearly evident in the welldocumented case study of bmw (roberts and grover 2012a), as they were able to sense emerging customer needs by involving lead users in the idea generation towards the product innovation activities, and they responded quickly to the new ideas by implementing them in upcoming products. in the context of this discussion, the firm is able to sense emerging customer needs through their use of ucrm, and respond to the changing customer requirements through the targeted customer-based actions. as proposed by overby et al. (2006), the agility score for agile firms can be expressed as the minimum of the sensing and responding scores when the two capabilities are in synch (or aligned 100%). this supports the notion that, while a firm can neither sense nor respond to all opportunities, it is capable of responding to those that it senses (or vice versa): enterprise agility score aligned = min (sensing score, responding score) on the contrary, for non-agile firms, the score is calculated as the product of the sensing and responding scores as its sensing and responding capabilities are not in synch. what it shows is that, what the firm senses and what the firm is able to respond to, do not always match up; thereby, limiting the number of opportunities the firm can seize: enterprise agility score non-aligned = sensing score x responding score at one extreme, misalignment can cause severe negative consequences (strandholm et al. 2004). as reported in (roberts and grover 2012a), in the case of woolworths in the us the company initially failed to sense the growth of suburbs and, as a result, its target customer base shifted their shopping australasian journal of information systems volume 18 number 2 2014 118 activities away from the urban centers where most of the company’s stores were located. consequently, the merchandise and the needs of the customers who continued to shop ‘downtown’ had a mismatch. the subsequent attempts they made towards rebranding the “woolworths stores” appeared too late and were ineffective at the end, ultimately leading to the closure of all its us-based stores by 1997. in some cases as documented in the xerox case (overby et al. 2006; roberts and grover 2012a), some firms are equipped with higher sensing capabilities and are able to sense changes relevant to their line of business, yet lack the responding capability and fail to respond to the opportunities in an agile manner. in the 1970s, xerox managed to sense the impending changes in the computing industry to develop multiple computing innovations, yet failed to market them. xerox managed to sense the shifting customer demands, but nonetheless was unable to respond quickly or adequately. similarly, a firm may have under-developed sensing capabilities yet have well-honed responding capabilities as in the case of apple’s introduction of the newton (roberts and grover 2012a). apple’s decision to position the newton as a mass-market product was a failure as it was too early in its development to be available as a mass-market product. it lost its audience and never gained momentum in the pda market in the early 200os. however, apple subsequently did become the leader in the pda market with the recent introduction of the iphone and ipad. in such cases, the firms either sense the wrong opportunities or simply fail to sense the correct opportunities. all these practical cases share the common notion that the consequences of misalignment and alignment of a firm’s sensing and responding capabilities could be decisive and extremely critical for their competitive position in the market. as the extant research affirms (overby et al. 2006; roberts and grover 2012a), conceptually, firms that score highly on both sensing and responding capabilities and are well-aligned are more likely to respond to the right customer-based market opportunities at the right time than their non-aligned, less agile counterparts with low scores on either or both sensing and responding capabilities. as a result, such agile firms will be able to provide positive customer experiences and build positive customer perceptions and customer satisfaction. so, we propose customer agility impacts customer satisfaction; however, the full effect of customer agility on customer satisfaction will take place when a firm’s sensing (i.e., identifying changing customer needs and preferences) and responding (i.e., fulfilling changing customer needs and preferences) capabilities are aligned (haeckel 1999; overby et al. 2006; roberts and grover 2012a; teece 2007b). thus, we hypothesize: h1: agility alignment is positively related to customer satisfaction, such that firms with high (low) sensing capabilities and high (low) responding capabilities will have more (less) satisfied customers. alignment as mediation as explained by roberts and grover (2012a), the existence of an intervening mechanism between an antecedent variable and the dependent variable is specifies as the mediation perspective. the main difference visible in the mediation perspective, compared to the matching perspective, is that the mediation view of alignment is anchored to a particular criterion variable (venkatraman 1989). hence, while the matching perspective provides insights regarding the combinations of different levels of sense and response capabilities, the mediation perspective provides insights into the sense-responseperformance process (roberts and grover 2012a). in the context of this discussion, taking the customers’ view on agility into account, the mediation perspective provides insights into the senseresponse-satisfaction process. conceptually, the mediation perspective entails looking into how the conception of agility works from a process perspective (roberts and grover 2012a). referring to the dynamic capabilities view, focusing on competitor dynamics, teece (2007b) elucidated that “a firm’s ability to manage competitor threats and to reconfigure itself is dependent upon its investment activity, which is in turn dependent on its ability to sense an opportunity”. further, as roberts and grover (2012a) mentioned in their recent study on the sense-response-performance context, a firm’s performance is primarily dependent on its ability australasian journal of information systems volume 18 number 2 2014 119 to respond to market opportunities (hult et al. 2005), which in turn is inherently dependent on the firm’s ability to sense the opportunities. however, the superior sensing ability cannot be effectively leveraged for value creation if a firm is weak in its responding capabilities (overby et al. 2006; roberts and grover 2012a). when applied to our study, the ability to sense the continuously shifting customer needs via customers’ use of u-crm cannot be effectively leveraged for more satisfied customers unless the firm is able to create positive customer perceptions through superior customer experiences. likewise, a stronger customer response capability cannot be effectively leveraged if the firm fails to sense customer-based market opportunities (roberts and grover 2012a). when applied to our study, a firm’s capability of delivering superior customer experience cannot be effectively leveraged unless the firm is able to sense customer-based opportunities through their use of u-crm in their shopping engagements. roberts and grover (2012a) argued that the firm’s customer responding capability mediates the relationship between customer sensing capability and firm performance. when applying this notion to our study context, we argue that the customer-perceived responsiveness mediates the relationship between the firm’s ability to sense changing customer preferences (via customers’ use of u-crm) and customer satisfaction. so, we hypothesize: h2: firms’ customer responding capability mediates the impact of customer sensing capability on customer satisfaction. research method sampling frame and characteristics we operationalized the study model using data on the customers of two market leading australian retailers who actually use the u-crm app in their regular shopping engagements, in order to measure the constructs in our research model. we specifically selected the actual users of u-crm from either of the two largest australian retailers, as our objective is to understand the sense-response alignment through the degree to which customers use the u-crm app and the customer-perceived firm’s responsiveness. we administered our anonymous surveys online, with the invitations sent to multiple sources in order to capture a sufficient number of respondents who actually use the u-crm app in their shopping. alternatively, an online survey was posted on the user community pages on the social media sites of the two case organizations, in addition to the invitation emails sent to a selected sample of current and potential users of a u-crm app, comprising of students, faculty members and other staff in one of the leading universities in australia. our exercise yielded a total of 128 respondents who actually use the system, for a response rate of 38.9% (from 329 who actually accessed the url). our subsequent screening for missing data left us with 118 usable respondents, as one of the respondents left out data. the demographics of our sample revealed that mostly they were below the age of 45 (44%), and were male (64%). a sample size of 118 seems adequate to evaluate our research model using pls (barclay et al. 1995a; chin and newsted 1999; falk and miller 1992; goodhue et al. 2012). we used wave analysis (armstrong and overton 1977) to assess the impact of non-respondent bias, whereby the respondents were grouped into early and late respondents, and comparisons were made among the respondents’ age and gender. our analysis revealed no significant differences between early and late respondents. based on our findings, non-response bias does not appear to impact our study. testing the research model barclay et al. (1995b) stated that an assessment of a measurement model should examine: a) individual measurement item reliability, b) internal consistency, and c) discriminant validity. we used the partial least square (pls) technique of structural equation modeling in smartpls 2.0 software (ringle et al. 2005) to evaluate our research model and the measurement properties of the customer sensing australasian journal of information systems volume 18 number 2 2014 120 (customer’s use of u-crm app) and customer responding (perceived responsiveness) constructs of our model. pls allows a researcher to test the psychometric properties of the scales used to measure a variable in a measurement model, and to carry out an estimation of the structural model based on the strength and direction of the relationships between the variables, simultaneously (xu et al. 2011). to support individual item reliability, we checked the loadings of the individual measurement items on their intended constructs and compared them against the recommended tolerances of 0.60 or, ideally 0.70, as stipulated in extant research (barclay et al. 1995b; chin 1998). all of the measurement items met the recommended tolerance limit of 0.60, while 86% met the ideal threshold of 0.70 (see appendix d). moreover, as seen in appendix d, the loadings of a given construct’s indicators were higher than the loadings of any other, and the same indicators loaded more highly on their intended construct than on any other, lending support to the discriminant validly. however, as seen in appendix e, the average variance extracted (ave) of the latent variable “perceived responsiveness” was slightly lower than the correlation between this and the other latent variable “sensing / use”. overall, the discriminant and convergent validity of the model were acceptable. further, we calculated the composite reliability and cronbach’s alpha for each construct to examine the internal consistency of the constructs, and all met the suggested tolerances of >0.70 (fornell and larcker 1981) (see appendix e). we next analyzed the structural model to see the significance and strength of the relationships between the constructs using a pls model and smartpls software. we examined the standardized path coefficients, path significance and variance explained (r2) to test the predictive power of the model. the path between sensing (customer’s use of u-crm) and responsiveness (customer-perceived firm’s responsiveness) showed a strong and significant positive relationship (β=0.763, p<0.001), with customer’s use of u-crm (sensing) explaining 58.2% of the variance (r2) in customer-perceived firm’s responsiveness (responding). hypotheses testing agility alignment as matching we used polynomial regression to test the hypothesis involving agility alignment as matching (h1). our decision to use polynomial regression was mainly influenced by the non-linear and asymmetric nature of the customer satisfaction variable as documented in the marketing literature (anderson and sullivan 1993; cheung and lee 2009). further, the use of response surface methodology in the interpretation of the surface created by polynomial regression was influenced by its suitability to interpret the three-dimensional relationship between two independent variables (sensing and responding) with one outcome variable (customer satisfaction). with polynomial regression equations, a researcher is able to perform direct tests of theoretical models relevant to the study of alignment (edwards 1994; edwards and parry 1993; roberts and grover 2012a). following which, we used the polynomial regression equation below to test our first hypothesis, “agility alignment is positively related to customer satisfaction, such that firms with high (low) sensing capabilities and high (low) responding capabilities will have more (less) satisfied customers”: customer satisfaction = ƒ (customer sensing*, customer responding**) z = β0 + β1cs* + β2 cr** + β3 cs2 + β4 (cs x cr) + β5 cr2 + e where, in this study *cs = extent to which customer’s use of u-crm, **cr = customer-perceived firm responsiveness. table 1 summarizes our results. australasian journal of information systems volume 18 number 2 2014 121 beta coefficient / unstandardized regression coefficients intercept / constant 4.954 customer sensing capability (cs) (we surrogate this to customer’s use of u-crm) 0.124** customer responding capability (cr) (we use perceived responsiveness here) 0.698* cs2 -0.026 cs*cr 0.093** cr2 0.027 * p<0.0001, **p<0.05 table 1. results of polynomial regression analysis the response surface methodology (khuri and cornell 1987) provides the basis required for testing and interpreting the features of surfaces corresponding to polynomial quadratic regression equations. the combination provides the sophisticated statistical nuance required to examine the extent to which combination of two predictor variables relate to an outcome variable, in particular when the discrepancy (or match) between the two predictor variables is a fundamental consideration (shanock et al. 2010). figure 2 depicts a response surface for agility alignment; that is, alignment between sensing (using the surrogate measure of u-crm use) and responding (through the customer-perceived firm’s responsiveness) as it relates to customer satisfaction. first, we examined the effect of the agreement between two predictor variables – sensing and responding – on the outcome variable – customer satisfaction. as displayed in figure 2, customer satisfaction is significantly higher and changes in a curvilinear manner when they are aligned (along the line of perfect agreement between sensing and responding; see point a through to point b along the solid line on the floor of the graph where x=y and table 2). also, the customer satisfaction decreases when the discrepancy between sensing and responding increases (when one moves away to the left or right from the center of the graph along the dotted line; x= -y) suggesting that the customer satisfaction is peaking when sensing and responding are aligned compared to when they are misaligned. further, the customer satisfaction is higher when both sensing and responding values are both high (point a) than when they are both low (point b). australasian journal of information systems volume 18 number 2 2014 122 figure 2: agility alignment as it relates to customer satisfaction next, we examined the two curves; sensing curve (line along b-c) and responding curve (line along bd) against the outcome variable, namely, customer satisfaction. the response curve suggests that the customer satisfaction is greatest when the responding capability reaches medium to higher levels (point c). however, the sensing curve suggests that the customer satisfaction tapers off when a firm is sensing highly but is deficient in responsiveness (point d). we then checked the relationship between the direction of discrepancy and customer satisfaction along the line of discrepancy (x= y). as the figure displays, the direction of discrepancy is important for customer satisfaction. satisfaction levels do not suffer as much when the discrepancy is such that perceived responsiveness is higher than customer sensing than vice versa. effect coefficient standard error test statistics (t) p-value significance a1: slope along x = y (as related to z) 0.82 0.05 18.060 0.000 significant a2: curvature on x = y (as related to z) 0.09 0.03 3.159 0.002 significant a3: slope along x = y (as related to z) -0.57 0.09 -6.278 0.000 significant a4: curvature on x = -y (as related to z) -0.09 0.05 -1.824 0.071 table 2: testing slopes and curves agility alignment as mediation having established the overall level of consensus as regards to the sense-response alignment in agility as a match between the two capabilities, the analysis now focuses on the mediating effect of alignment in customer satisfaction. as roberts and grover (2012a) stated, the notion of agility alignment as mediation in general entails that the effect of customer sensing capability on firm performance is mediated by firms’ customer responding capability. agility alignment as mediation in this discussion, a b c d australasian journal of information systems volume 18 number 2 2014 123 taking the customer’s perspective into account, essentially refers to the effect of firms’ customer sensing capability (through the customer’s use of u-crm) on customer satisfaction being mediated by customer-perceived firm’s responsiveness. we first followed the several regression analysis approach suggested by baron and kenny (1986). here we test the effect of the independent variable (sensing) on the dependent variable (satisfaction) with and without the mediating variable (responsiveness) and compare the significance of the coefficients at each step. as depicted in table 3, the results of the three regressions support the idea of agility alignment as mediation in delivering customer satisfaction. test r2 adjusted r2 std. error significance of change sensing-responsiveness 0.516 0.512 0.94073 0.000 sensing-satisfaction 0.585 0.581 0.88148 0.000 responsivenesssatisfaction 0.795 0.793 0.61963 0.000 table 3: results of the regression analysis however, the examination of coefficients approach has one limitation as it never really tests the significance of the indirect pathway (roberts and grover 2012a). to address this limitation, we performed mediation analysis following sobel’s (1982) product of coefficients approach using smartpls 2.0 software (ringle et al. 2005). this approach calculates the indirect effect and tests for its significance (roberts and grover 2012a). in other words, the regression coefficient for the indirect effect signifies the change in the outcome variable (customer satisfaction) for every unit change in the independent variable (sensing) that is mediated by the intervening variable (perceived responsiveness). our results demonstrate that customer-perceived firm’s responsiveness mediates the relationship between firm’s customer sensing capability and customer satisfaction (sobel test statistic = 10.644, p<0.001). the direct effect between the independent variable and dependent variable decreases from 0.765 to 0.257 yet with t-statistics greater than 1.96 (t=4.676); hence, the mediation is described as a partial mediation. conclusion in this study we conceptualized and empirically tested a research model that views a firm’s customer agility in relation to customer satisfaction. in particular, our conceptualization of firms’ agility takes the view of the customers where we investigate the relationship of a firm’s agility to its customer satisfaction (as an indicator of firm performance). broadly speaking, our empirical analysis finds that a firm’s agility is significantly related to customer satisfaction. specifically, the alignment between firms’ ability to sense their customers’ needs and firms’ responding capability (in the form of perceived responsiveness) impacts customer satisfaction, in two distinct ways. in the conceptualization of agility alignment as matching and mediating, both show significant relationships with customer satisfaction. when taking the matching perspective, our study highlights that customer satisfaction is higher when the customer sensing capability and customer-perceived firm’s responsiveness (responding capability) are aligned than when they are misaligned. additionally, the customer satisfaction is higher when firm’s sensing and responding capabilities are higher than when they both are low. when considering the independent effects, 1) customer satisfaction is greatest when customer-perceived responsiveness (responding capability) is at its maximum, whilst 2) customer satisfaction is higher when sensing is low as it tapers off when a firm is sensing highly but is deficient australasian journal of information systems volume 18 number 2 2014 124 in responsiveness. this implies that a combination of high customer sensing capability and high customer responding capability is the way to deliver customer satisfaction and achieve firm performance whilst there is no value in just sensing the customer-based opportunities unless the firm is able to respond to them. further, our analysis reveals that the direction of discrepancy between sensing and responding capabilities (we refer to the customer-perceived firm’s responsiveness here) is important for customer satisfaction where the discrepancy is such that perceived responsiveness is higher than customer sensing than vice versa. additionally, our analysis of sense-response alignment as mediation reveals that indeed the customerperceived firm’s responsiveness partially mediates the relationship between sensing and customer satisfaction. our finding supports the prior research on the sense-response-performance relationship view (hult et al. 2005; roberts and grover 2012a; roberts and grover 2012b). to put it differently, a firm may sense customer requirements better with superior sensing capabilities yet it needs to respond to such requirements quickly, effectively and adequately in order to make customers happy and convert the firm’s superior customer sensing capability into firm performance. hence, we argue that at least minimal customer responsiveness is still a key to firm performance, despite having a superior customer sensing capability. limitations of the study our use of u-crm as the context limited our ability to fully understand how a firm develops and leverages its overall ability to sense and respond to its advantage in delivering customer satisfaction and achieving superior firm performance. as the constant advancements of mobile technology, the nature of mobile applications, their interactivity and their use in the organizational context poses many challenges for developing meticulous measures, future research should develop more rigorous measures of how firms leverage such technological advancements and tools to achieve superior customer agility via sensing and responding capabilities. implications for research and practice our study has a number of implications for both research and practice. first, we isolated key attributes and characteristics discussed in previous conceptions of agility and then conceptualized the notion of agility from the customers’ point of view. hence, we suggest that future research could consider the attributes and characteristics we summarized when conceptualizing firms’ agility from the customers’ perspective. additionally, we took two perspectives of agility alignment, namely, matching and mediation, and tested them from the customers’ perspective, but no comprehensive investigation was made on the ways in which the sensing and responding capabilities could or should be aligned. following which, we suggest future researchers could take multiple perspectives into account when investigating agility-related phenomena. regarding the implications for practice, our empirical investigation suggests that firms should align their sensing and responding capabilities in order to achieve business benefits and sustained competitive advantage. we suggest organizations place equal emphasis on nurturing sensing capabilities and on nurturing responding capabilities, in addition to configuring and organizing themselves in order to align the two capabilities. contributions this study addressed the issue of agility alignment from customers’ standpoint to add more detail to the agility construct by introducing the missing customer perspective to the extant body of literature. our approach reflects the current management thinking as it highlights the importance of customer perceptions in today’s hypercompetitive business environment. as the customers’ perspective of agility was absent in the extant body of agility literature this study provide a foundation for the nomological network around customers’ perspectives of agility. also, we hope that this study contributes to the cumulative progress of agility related phenomena. in addition we made methodological contributions australasian journal of information systems volume 18 number 2 2014 125 for research, by relaxing the traditional linearity assumptions and demonstrating that in doing so it is possible to uncover complex interactions between the constructs in a research model. future research we believe that this study adds more detail to the agility construct by introducing the missing customer perspective to the extant body of literature. it reflects the importance of customer perceptions in today’s hypercompetitive business environment. however, in order to understand this critical, yet underinvestigated perspective, and to build on this work, far more research is required on the nomological network around customers’ perspectives of agility. now that we have investigated firms’ agility from the customers’ perspective using the contemporary u-crm context, future research may extend this work to the other relevant contexts such as supply chain agility and operational agility in order to further investigate customers’ view of organizational agility. additionally, as we argue that the customers could effectively judge the firm’s responding capacity through their perception of the firm’s responsiveness, similar approaches are possible in number of other contexts. for example, evaluation of both online and off-line course delivery by university professors can be performed through students’ perceptions as they are better positioned to make valid and reliable judgments. as we have relaxed the traditional linearity assumptions and demonstrated that in doing so it is possible to uncover complex interactions between the constructs in a research model, we suggest future researchers to relax the linearity assumptions to consider non-linear analytical methods such as three dimensional modeling techniques, polynomial regression and response surface methodology, specifically when they use theoretical viewpoints that suggest non-linear relationships. references 2000. "can borders turn the page?," in: business week. p. 75. anderson, e.w., and sullivan, m.w. 1993. "the antecents and consequences of customer satisfaction for firms," marketing science (12), pp. 125-143. armstrong, j.s., and overton, t.s. 1977. "estimating nonresponse bias in mail surveys," journal of marketing research (14:3), pp. 396-402. atapattu, m., and sedera, d. 2013a. "agility of the firm: customers' perspective," european conference on information systems (ecis), utrecht, netherlands ais. atapattu, m., and sedera, d. 2013b. "agility: customers' perspective," pacific asia conference on information systems (pacis), jeju island, south korea: ais. atapattu, m., and sedera, d. 2014. "mobile digital connectedness: conceptualization, measurement and application," the second ais-journals joint author workshop in pacis., chengdu, china. barclay, d., higgins, c., and thompson, r. 1995a. "the partial least squares (pls) approach to causal modeling: personal computer adoption and use as an illustration," technology studies (2:2), pp. 285-309. barclay, d., higgins, c., and thompson, r. 1995b. "the partial least squares (pls) approach to causal modelling: personal computer adoption and use as an illustration," technology stusies (2:2), pp. 285-324. barki, h., titah, r., and boffo, c. 2007. "information system use-related activity: an expanded behavioral conceptualization of individual-level information system use," information systems research (18:2), pp. 173-192. australasian journal of information systems volume 18 number 2 2014 126 barkin, s.r., and dickson, g.w. 1977. "an investigation of information systems utilization," information & management (1:1), pp. 35-45. baron, r.m., and kenny, d.a. 1986. "the moderator–mediator variable distinction in social psychological research: conceptual, strategic, and statistical considerations," journal of personality and social psychology (51:6), pp. 1173-1182. bhattacherjee, a. 2001. "understanding information systems continuance: an expectationconfirmation model," mis quarterly (28:2), pp. 229-254. burton-jones, a., and gallivan, m.j. 2007. "toward a deeper understanding of system usage in organizations: a multilevel perspective," mis quarterly (31:4), pp. 657-679. burton-jones, a., and straub, d.w. 2006. "reconceptualizing system usage: an approach and empirical test," information systems research (17:3), pp. 228-246. cheung, c.m.k., and lee, m.k.o. 2009. "user satisfaction with an internet-based portal: an asymmetric and nonlinear approach," journal of the american society for information science and technology (60:1), pp. 111-122. chi, l., ravichandran, t., and andrevski, g. 2010. "information technology, network structure, and competitive action," information systems research (21:3), pp. 543-570,655,657. chin, w.w. 1998. "the partial least squares approach for structural equation modelling," in modern methods for business research, g.a. marcoulides (ed.). mahwah, nj: lawrence erlbaum, pp. 295-336. chin, w.w., and newsted, p.r. 1999. "structural equation modeling analysis with small samples using partial least squares," in statistical strategies for small sample research, r. hoyle (ed.). newbury park, ca: sage publications, pp. 307-341. choi, s., and mattila, a.s. 2008. "perceived controllability and service expectations: influences on customer reactions following service failure," journal of business research (61:1), 1//, pp. 24-30. churchill, g.a. 1979. "a paradigm for developing better measures of marketing constructs," journal of marketing research (16:1), pp. 64-73. dove, r. 2001. response ability: the language, structure, and culture of the agile enterprise. wiley. edwards, j.r. 1994. "the study of congruance in organizational behavior research: critique and a proposed alternative," organizational behavior and human decision processes (58:1), pp. 51100. edwards, j.r., and parry, m.e. 1993. "on the use of polynomial regression equations as an alternative to difference scores in organizational research," academy of management journal (36:6), pp. 1577-1613. falk, r.f., and miller, n.b. 1992. a primer for soft modeling. akron, oh: university of akron press. fornell, c., and larcker, d.f. 1981. "evaluating structural equation models with unobservable variables and measurement error," journal of marketing research), pp. 39-50. gao, t.t., sultan, f., and rohm, a.j. 2010. "factors influencing chinese youth consumers' acceptance of mobile marketing," journal of consumer marketing (27:7), pp. 574-583. goodhue, d.l., lewis, w., and thompson, r. 2012. "does pls have advantages for small sample size or non-normal data?," mis quarterly (36:3), pp. 981-a916. haeckel, s.h. 1999. adaptive enterprise: creating and leading sense-and-respond organizations. boston: harvard business school press. australasian journal of information systems volume 18 number 2 2014 127 huang, p.-y., ouyang, t.h., pan, s.l., and chou, t.-c. 2012. "the role of it in achieving operational agility: a case study of haier, china," international journal of information management (32), pp. 294-298. hult, g.t.m., ketchen, d.j., and slater, s.f. 2005. "market orientation and performance: an integration of disparate approaches " strategic management journal (26 12), pp. 1173-1181. jayachandran, s., hewett, k., and kaufman, p. 2004. "customer response capability in a sense-andrespond era: the role of customer knowledge process," journal of academy of marketing science (32:3), pp. 219-233. kaplan, a.m. 2012. "if you love something, let it go mobile: mobile marketing and mobile social media 4x4," business horizons (55:2), 3//, pp. 129-139. kaplan, r.s., and norton, d.p. 1996. "the balanced score card: translating strategu into action." boston, mass: harvard business school press. khuri, a.i., and cornell, j.a. 1987. response surfaces: designs and analyses. new york: marcel dekker. kohler, a. 2013. "woolworths invests $20m in data analytics." retrieved 17/7/2013, from http://www.businessspectator.com.au/news/2013/5/2/information-technology/woolworthsinvests-20m-data-analytics kohli, a.k., jaworski, b.j., and kumar, a. 1993. "markor: a measure of market orientation," journal of marketing research (4:30), pp. 467-477. lamarre, a., galarneau, s., and boeck, h. 2012. "mobile marketing and consumer behaviour current research trend," international journal of latest trends in computing (3:1). luftman, j., zadeh, h., s., derksen, b., santana, m., rigoni, e.h., and huang, z. 2012. "key information technology and management issues 2011-2012: an international study," journal of information technology (27:3), p. 198. mackenzie, s.b., podsakoff, p.m., and podsakoff, n.p. 2011. "construct measurement and validation procedures in mis and behavioral research: integrating new and existing techniques," mis quarterly (35:2), pp. 293-334. mclean, e., and sedera, d. 2010. "the measurement of information system use: preliminary considerations," amcis 2010 proceedings (255). nadler, d., and tushman, m. 1983. "a general diagnostic model for organizational behavior: applying a congruence perspective," in editors perspectives on behavior in organizations, j.r. hackman, e.e. lawler and l.w. porter (eds.). new york: mcgraw-hill, pp. 112-124. narayanaswami, c., kruger, a., and marmasse, n. 2011. "pervasive retail," pervasive computing (11), pp. 1536-1268. narver, j.c., slater, s.f., and maclachlan, d.l. 2004. "responsive and proactive market orientation and new-product success," journal of production innovation management (21:5), pp. 334-347. nazir, s., and pinsonneault, a. 2012. "it and firm agility: an electronic integration perspective," journal of the association for information systems (13:3), pp. 150-171. overby, e., bharadwaj, a., and sambamurthy, v. 2006. "enterprise agility and the enabling role of information technology," european journal of information systems (15:2), pp. 120-131. po-an hsieh, j.j., rai, a., and xin xu, s. 2011. "extracting business value from it: a sensemaking perspective of post-adoptive use," management science (57:11), pp. 2018-2039. rapp, a., trainor, k.j., and agnohotri, r. 2010. "performance implications of customer-linking capabilities: examining the complementary role of customer orientation and crm technology," journal of business research (63:11), pp. 1229-1236. australasian journal of information systems volume 18 number 2 2014 128 ringle, c., marc/wende, and sven/will, a. 2005. "smartpls," in: 2.0 (beta). hamburg, germany: http://www.smartpls.de. roberts, n., and grover, v. 2012a. "investigating firm's customer agility and firm performance: the importance of aligning sense and respond capabilities," journal of business research (65:5), pp. 579-585. roberts, n., and grover, v. 2012b. "leveraging information technology infrastructure to facilitate a firm's customer agility and competitive activity: an empirical investigation," journal of management information systems (28:4), pp. 231-270. robin, m. 2013. "australian companies “starting to get” big data as woolworths buys stake in data analytics firm quantium " retrieved 17/07/2013, 2013, from http://www.smartcompany.com.au/retail/055358-australian-companies-starting-to-get-bigdata-as-woolworths-buys-stake-in-data-analytics-firm-quantium.html rohm, a.j., gao, t., sultan, f., and pagani, m. 2012. "brand in the hand: a cross-market investigation of consumer acceptance of mobile marketing," business horizons (55:5), 9//, pp. 485-493. sambamurthy, v., bharadwaj, a., and grover, v. 2003. "shaping agility through digital options: reconceptualizing the role of information technology in contemporary firms," mis quarterly (2:27), pp. 237-263. shankar, v., venkatesh, a., hofacker, c., and naik, p. 2010. "mobile marketing in the retailing environment: current insights and future research avenues," journal of interactive marketing (24:2), 5//, pp. 111-120. shanock, l., baran, b., gentry, w., pattison, s., and heggestad, e. 2010. "polynomial regression with response surface analysis: a powerful approach for examining moderation and overcoming limitations of difference scores," journal of business and psychology (25:4), 2010/12/01, pp. 543-554. slater, s.f., and narver, j.c. 2000. "intelligence generation and superior customer value," journal of academy of marketing science (28:1), pp. 120-127. sobel, m.e. 1982. "asymptotic confidence intervals for indirect effects in structural equation models," sociological methodology (13), pp. 290-312. strandholm, k., kumar, k., and subramanian, r. 2004. "examining the interrelationships among perceived environmental change, strategic response, managerial characteristics, and organizational performance. ," journal of business research (57:1), pp. 58-68. tallon, p.p., and pinsonneault, a. 2011. "competing perspectives on the link between strategic information technology alignment and organizational agility: insights from a mediation model," mis quarterly (35:2), pp. 463-486. teece, d. 2007a. "explicating dynamic capabilities: the nature and microfoundations of (sustainable) enterprise performance," strategic management journal (28:13), pp. 1319-1350. teece, d.j. 2007b. "explicating dynamic capabilities: the nature and microfoundations of (sustainable) enterprise performance," strategic management journal (28:13), pp. 1319-1350. venkatesh, v., brown, s.a., maruping, l.m., and bala, h. 2008. "predicting different conceptualizations of system use: the competing roles of behavioral intention, facilitating conditions, and behavioral expectation," mis quarterly (32:3), pp. 483-502. venkatesh, v., morris, m.g., davis, g.b., and davis, f.d. 2003. "user acceptance of information technology: toward a unified theory view," mis quarterly (27:3), pp. 425-478. venkatraman, n. 1989. "the concept of fit in strategy research: towards verbal and statistical correspondence," academy of management review (14:3), pp. 423-444. australasian journal of information systems volume 18 number 2 2014 129 vodanovich, s., sundaram, d., and myers, m. 2010. "digital natives and ubiquitous information systems," information systems research (21:4), p. 711. xu, j.d., benbasat, i., and cenfetelli, r. 2011. "the effects of service and consumer product knowledge on online customer loyalty," journal of the association for information systems (12:11), p. 1. yusuf, y.y., sarhadi, m., and gunasekaran, a. 1999b. "agile manufacturing:: the drivers, concepts and attributes," international journal of production economics (62:1), pp. 33-43. zaheer, a., and zaheer, s. 1997. "catching the wave: alertness, responsiveness and market influence in global electronic networks," management science (43:11), pp. 1493-1509. australasian journal of information systems volume 18 number 2 2014 130 appendix a: definitions of organizational agility definition source firm’s profitable comprehensive response to the business challenges in faced in rapidly changing, continually fragmenting, global markets for customerconfigured, high-quality goods and services. goldman et al. (2007) the business’ ability to change and adapt quickly in response to rapidly changing environmental conditions. bititci et al. (2000) firm’s ability to cope up unexpected external changes, ability to convert and take them as opportunities, and survival ability on unprecedented threats posed by the environment. sharifi and zhang (2004) business’ ability to grow and respond quickly in a continuously and unexpectedly changing, competitive market where the value of products and services is driven by customers. yusuf et al. (2004) organizational ability to thrive in a constantly changing, unpredictable environment. day (2011) firm’s ability to sense opportunities and seizing them by assembling requisite knowledge, assets, and relationships with speed and dexterity. sambamurthy et al. (2003) firm’s ability to sense environmental changes and respond readily. overby et al. (2006) organization’s ability to discover new opportunities for competitive advantage, tie together the assets, knowledge and relationships to seize them, and adopt abrupt changes in business environment. setia et al. (1977) the ability to sense and respond to opportunities and threats with ease, speed, and dexterity. nazir and pinsonneault (2012) ability of the firm to sense and respond quickly to customer-based opportunities for innovation and competitive action. roberts and grover (2010) appendix b: formative and reflective nature of the constructs c on st ru ct a re th e in di ca to rs d ef in in g ch ar ac te ris tic s o f t he co ns tru ct s? d o ch an ge s i n in di ca to rs ca us es c ha ng es in th e co ns tru ct ? d o ch an ge s i n th e co ns tru ct ca us e ch an ge s i n th e in di ca to rs ? d o th e in di ca to rs ne ce ss ar ily sh ar e a co m m on th em e? d oe s e lim in at in g an in di ca to r a lte r t he co nc ep tu al d om ai n of th e co ns tru ct ? is a c ha ng e in o ne o f t he in di ca to rs n ec es sa ril y as so ci at ed w ith a c ha ng e in al l t he o th er in di ca to rs ? d o th e in di ca to rs h av e th e sa m e an te ce de nt s a nd co ns eq ue nc es ? sc al e ty pe customers’ use of u-crm (sense) yes yes no yes yes no no formative perceived (customer) responsiveness no no yes yes no yes no reflective customer satisfaction no no yes yes no yes no reflective australasian journal of information systems volume 18 number 2 2014 131 appendix c: construct measures construct items a measure source b customers’ use of u-crm (surrogate measure for firms’ customer sensing capability) i frequently use this mobile app to....... find products prepare my regular grocery shopping list place orders online provide comments and feedback find a store more convenient (barki et al. 2007; burton-jones and straub 2006; mclean and sedera 2010; venkatesh et al. 2008; venkatesh et al. 2003) perceived responsiveness (customers’ perspective of firms’ responding capability) woolworths/coles quickly react to the fundamental changes in my product requirements by providing me with relevant customized information when i browse recipes using the mobile app, woolworths/coles is quick to provide relevant promotional information for the products required to make that recipe when i continue to purchase a new product (e.g. baby nappies) repetitively, woolworths/coles quick to respond to it by providing other associated product information (e.g. other baby products) woolworths/coles, is fast to provide information about discounts and promotions based on the products i purchase regularly woolworths/coles, is quick to provide information on discounts and promotions when i change my preferred store based on the products i created in my shopping list when i pass a nearby woolworths/coles store, the mobile app is able to recognize my physical location to prompt discounts and promotions on my usual purchases for the particular store i can easily find the products required to satisfy my changing needs using the woolworths/coles mobile app woolworths/coles can easily satisfy my new and changing needs the product displayed in the specials section of the mobile app reflects my specific requirements overall the information i regularly receive from woolworths/coles is useful and matches my preferences (jayachandran et al. 2004; kohli et al. 1993; roberts and grover 2012a; roberts and grover 2012b) customer satisfaction i am satisfied with... the purchasing products functionality provided by the app my overall shopping experience with woolworths/coles (bhattacherjee 2001) note: a 1= strongly disagree, 7= strongly agree. b although we draw our survey items from the well-established measurement constructs from marketing and is, we only adapt items that correspond to the conceptual definitions of our constructs. australasian journal of information systems volume 18 number 2 2014 132 appendix d: loading and cross loading of measures perceived responsiveness firm’s sensing / customer’s use of u-crm perres1 0.8614 0.7134 perres2 0.8432 0.6739 perres3 0.7976 0.5912 perres4 0.8629 0.6807 perres6 0.7393 0.6036 perres7 0.8634 0.7853 perres8 0.6726 0.6726 perres9 0.8943 0.7375 perres10 0.8769 0.7822 use1 0.6893 0.8376 use2 0.5091 0.7017 use3 0.6541 0.7909 use4 0.6218 0.7838 use5 0.7659 0.7645 appendix e: internal consistency and discriminant validity of constructs composite reliability cronbach’s alpha pr s/u perceived responsiveness 0.9556 0.9480 0.6849 sense/use 0.8661 0.8093 0.7630 0.5648 the australasian journal of information systems (ajis) began in 1993 and was based on the simple realisation that there was a need for an academic ‘outlet’ for is research in australia australasian journal of information systems volume 17 number 2 2012 109 theorizing in information systems research using focus groups france belanger virginia tech belanger@vt.edu abstract information systems researchers have embraced a number of qualitative research approaches and methodologies, including interviews, observations, and even action research. one research method gaining visibility in is research is the focus group research method. focus groups have the potential to provide great insights into phenomena of interest to is researchers as they allow researchers to get deeper into the topic of interest by providing more background information about the circumstances of the subject’s answers or opinions. this paper presents a review of focus group research in the information systems literature, and provides a discussion of how and when the focus group research method can be the most appropriate method to use for is theorizing. the discussion highlights the idea that the focus group research method is particularly useful for exploratory research on topics where concepts normally emerge through interactions among individuals or where concepts are initially unclear to participants, and as part of a multi-method research program for theory development. examples of focus groups used in theory development are provided, together with a discussion of the limitations of the research method. keywords: focus group, qualitative research, information systems research, theory development introduction “a research process can be viewed as a series of interlocking choices, in which we try simultaneously to maximize several conflicting desiderata”(mcgrath 1982) information systems researchers have increasingly embraced new research approaches, methodologies, and epistemologies over the years. each approach or perspective provides its own set of advantages and disadvantages. qualitative techniques, for one, allow researchers to describe and explore an area of study in a rich context, in order to gain an understanding of the subjective experiences of the participants (bryman 1988). today, one can often find studies using qualitative methodologies published in leading is journals. among qualitative research methods, one often reads of case studies used for grounded theory or model testing, where interviews and observations have dominant roles to play. one research method gaining visibility in is research is the focus group research method (sobreperez 2008). rooted in market and social research, focus groups represent one qualitative research method where participants are selected and brought together to explore and discuss a specific topic, often in great detail (morgan 1997). importantly, in focus groups, participants are able to interact with, and react to, other participants in responding to questions and prompts of the moderator (krueger and casey 2000). it is argued in this paper that these interactions can be extremely useful in information systems theory development for phenomena where constructs exist in or emerge from individuals’ interactions with one another, such as team software development or impacts of telecommuting on groups. mailto:belanger@vt.edu australasian journal of information systems volume 17 number 2 2012 110 focus groups not only allow researchers to collect a concentrated set of interactions in a given time period, they also enable researchers to obtain more background information about the circumstances of the answers provided by the subjects (krueger 1994). as a result, focus groups have the potential to provide significant insights about phenomena of interest to is researchers in several situations: first, when information systems researchers are interested in new topics to be studied via exploratory research (e.g., what are the critical success factors for implementation of cloud computing); second, focus groups can also support research where constructs emerge from group interactions as opposed to individuals situations (e.g., how do software development teams benefit from (or not) mandatory use of new standards) (krueger and casey 2000; le rouge and niederman 2006); and finally, a review of the is literature reveals that focus groups can play an important role in developing theoretical models of is phenomena to be tested. the purpose of this paper is therefore to explore the use of the focus group method in the information systems literature, discuss its advantages and limitations, and discuss the applicability of this research method for the development of theories in information systems. more specifically, the discussion is guided by the following question: under which circumstances is the focus group research method most appropriate for is theorizing, and how can the method best be used for is theorizing? this paper provides two main contributions: first, it offers a review and categorization of how the focus group research method has been used in information systems research, and second and more importantly, it discusses how and when the focus group research method can be used for theory development in is research. the remainder of the manuscript is organized as follows. the background section briefly discusses the focus group research method followed by a review and analysis of focus group research in the information systems literature. then, a discussion of the applicability of the focus group research method to the field of information systems and the potential role of focus groups in is theorizing are provided and illustrated by research examples. finally, limitations of this work are offered before concluding comments are provided. the focus group research method the focus group research method has long been used in studies of consumer attitudes, perceptions, and opinions in the marketing and business communications literature (fern 1982; hartman 2004). as a result, many researchers describe the method in substantial details (edmunds 1999; fern 2001; krueger 1994; krueger and casey 2000). this section provides a brief overview of the method, with more details provided in appendix a. the focus group method is a qualitative data gathering technique where focus group sessions involve several participants assembled for a planned discussion to explore a specific topic of interest to researchers in a permissive, non-threatening environment (krueger 1994). the focus group research method benefits from the interaction among participants, which can reveal shared ideas, reactions, and opinions on the topic of the study. morgan (1997) refers to this as collaborative construction instead of consensus or negotiation. the participants are therefore influenced by each other when responding to ideas and comments in the discussion. in reality, they can also be influenced by the moderator, the setting, or any tools used in conducting the focus group. as a result, focus group research often better fits in the interpretivist epistemology (myers 1997), where participants’ interactions influence each other and might influence the interpretation of the researcher, allowing that participants might be influenced by their interactions with the researcher/moderator. without the interaction among participants, findings represent individual perceptions, i.e., ideas in isolation, as opposed to shared perceptions or individual perceptions embedded in a social or problem context. indeed, morgan (1997) suggests that the synergistic effects of focused interactions, can provide greater insights than the sum of individual interviews. australasian journal of information systems volume 17 number 2 2012 111 typically, focus group sessions are conducted by a moderator with the responsibility of running the session. a very high level of involvement is necessary to ensure that all participants voice their ideas, attitudes, perceptions or opinions, and that emerging concepts of interest are explored through further probing when appropriate (sobreperez 2008). the focus group participants are selected because they are knowledgeable about the topic investigated. the sessions are recorded with audio and/or video equipment, and are then transcribed so that they can be analyzed. in that aspect, focus groups follow similar procedures as other qualitative data collection approaches such as interviews and observations (strauss and corbin 1998). the focus group transcripts can be analyzed using standard qualitative analysis software and techniques (miles and huberman 1994). the results can be used to develop grounded theories (eisenhardt 1989), provide contextual data for case studies (yin 1994), or more frequently, to provide exploratory constructs and relationships that will be tested at a later stage of research (e.g., dickinger et al. 2008; lee et al. 1995). interestingly, the data gathered from focus groups are not limited to transcripts obtained from the discussions in the sessions, but can also include the moderator (or assigned scribe) notes, body language if video is used, results from debriefing sessions (if conducted), and any pre-session questionnaires that can be distributed at the beginning of focus groups (edmunds 1999). theory building is important to the development of the is field, and as straub (2009) suggests, having strong theoretical foundations and contributions is a major factor in publishing in top journals. indeed, tams and grover (2010) find that factors related to theoretical development, rather than research methodology or implications, lead to more highly cited, i.e., influential, articles in is literature. given the possibility of eliciting constructs from collaborative construction (constructs emerging from group discussions) (morgan 1997) and the ability to triangulate verbal data with other data sources such as gestures and reactions, this research method could be uniquely useful for theory building in many is contexts. for example, when the domain of interest is exploratory and the focus groups are used to generate information or ideas that can be later tested, when trying to obtain a deeper understanding of constructs (i.e. their dimensions) in established theory, or, when the constructs of interest exist in or emerge from the interactions of individuals with one another. this is particularly important to the information systems field because many topics of interest in the domain, from knowledge management, to telecommuting, to software development are affected by individuals’ interactions. using an individual unit of analysis to study group performance might lead to less complete and realistic theories. in order to explore this possible issue, and before further exploring how and when the focus group research method can be used in information systems theory development, the is literature is reviewed to identify how focus group research has been used in the is field. literature review: focus group research in is kruger (1994) suggests that focus groups can be very useful in gaining insights into a research domain in two particular cases: when there is limited research previously published, or when the focus group is part of a larger scale research effort. others suggest that focus groups have broad appeal for various types of evaluations of products, concepts or services, or to generate ideas for questionnaires (edmunds 1999). consistent with recent searches of the literature in top is research journals (baskerville and myers 2009; bélanger and crossler 2011; wang 2010), two well-known databases of articles in business, abi/inform and business source complete, were used to search for peer-reviewed journal articles in information systems that use the focus group research method as part of their research design between 1990 and 2011, using the keywords “information system” and “focus group”. a total of 94 articles were identified in abi/inform and 72 in business source complete. after screening for is journals australasian journal of information systems volume 17 number 2 2012 112 (broadly defined to include information science, computer science and related fields), there were 58 articles from abi/inform and 51 from business source complete. taking into account articles found in both databases, 75 individual articles were downloaded for further review. in the second round of review, only papers that study an is phenomena using focus groups as part of the research design were retained. papers removed from the list included papers that suggest future work should include focus groups, that were not is-related, or papers that discussed focus groups as a method to be adapted for other purposes such as online market research (clapper and massey 1996; markham et al. 2006; montoya-weiss et al. 1998) or for applicability checks (glass 2009; rosemann and vessey 2008), discussed focus group as a research method (sobreperez 2008; stahl et al. 2011), discussed focus group as a method for requirements determination in is development (leifer et al. 1994; o'heocha et al. 2010; peffers and tuunanen 2005) or to inform design research (tremblay et al. 2010), or explored is education issues using focus groups (dunning et al. 2001; dwyer and knapp 2004; maheshwari et al. 2009; shuk ying et al. 2008). the resulting sample of 58 papers shows that the oldest papers date to 1995 and 1996 in mis quarterly (lee et al. 1995; smith et al. 1996), information systems research (manning 1996), and information technology and libraries (tillotson et al. 1995). the sampled papers are summarized in table 1. following the types of focus groups identified by kruger (1994), information systems papers were classified along the dimensions of sole method or multi-method (also called mixed-method). sole method papers are those where the entire research findings of the paper are based on the use of the focus group research method. the multi-method papers represent those studies where the focus group method is used in conjunction with other research methods. extending the work of kruger (1994), the classification is further broken down into exploratory vs. explanatory research. exploratory research focuses on “what is” the state of some is phenomenon, whereas explanatory research focuses on “how” and “why” some is phenomenon is happening. the distinction between the use of interviews and the use of focus groups (sometimes called group interviews) is important and sometimes misunderstood. in conducting the review of the literature, one study was found (martz et al. 2007) where the researchers use the terminology of focus group in the description of the study (in the abstract) without apparently using focus groups as their research method. the authors conducted 18 interviews of practitioners and academics (in addition to observations) with the purpose of designing a master of healthcare informatics curriculum. however, the interviews appear to be individual, which would not constitute a focus group (the focus group terminology is not used except in the abstract). focus group type citation topic sample and other methods (for multimethod papers) focus group purpose sole method: exploratory (campbell et al. 2005) – journal of enterprise information management alignment of business and is 3 focus groups with 6 is/it managers in each group explore how managers view the alignment between business and it. participants develop a causal loop diagram. (grant and fitzgerald 2005) electronic journal of business research methods nexus between teaching and research in is 2 focus groups with staff and educators at universities explore whether a nexus exists between teaching and research at universities. also, evaluate focus group as method to explore this. (huang and lin 2009) – journal of computer information systems email usage 1 focus group with 11 participants explore email usage of knowledge workers. (jarvenpaa and lang 2005) information systems management mobile technology use 33 focus groups with 222 participants in 4 countries (finland, japan, china, usa) identify paradoxes and behaviours of mobile technology users. develop a user-technology interaction australasian journal of information systems volume 17 number 2 2012 113 focus group type citation topic sample and other methods (for multimethod papers) focus group purpose process framework. (jarvenpaa et al. 2003) communications of the acm mobile technology use 32 focus groups with 200 participants in 4 countries (finland, japan, china, usa) identify services used by mobile technology users, and issues related to use of mobile it. (le rouge and niederman 2006) communications of the ais public health knowledge management 6 focus groups with higher education administrators, educators and students use focus groups for requirement elicitation to develop public health knowledge base system. (mckeen et al. 2005) communications of the ais digital dashboards 1 focus group with it managers (# of participants unknown) to explore the use and types of digital dashboards in use by it managers. (mckeen and smith 2006) communications of the ais technology roadmap 1 focus group with it managers (# of participants unknown) explore managers’ practices in terms of creating and evolving a technology roadmap. (mckeen and smith 2007) communications of the ais it delivery options 1 focus group with it managers (# of participants unknown) identify decision criteria for it delivery to build a decision framework to guide selection of delivery options for it. sole method: exploratory (continued) (mckeen et al. 2009) communications of the ais it staffing 1 focus group with it managers (# of participants unknown) to explore the current it staffing challenges and issues. (mckeen and smith 2010b) communications of the ais total cost of ownership 1 focus group with it managers (# of participants unknown) identify firms’ use of total cost of ownership approach in it. (mckeen and smith 2010a) communications of the ais application portfolio management (apm) 1 focus group with it managers (# of participants unknown) to understand managers’ use of apm strategies and practices. (payton and zahay 2003) – journal of database marketing crm implementation success 5 focus groups with 21 marketing, is, and strategic managers in a healthcare firm identify factors that lead to crm implementation success for the focal firm (healthcare payer). (raitoharju et al. 2009) – electronic journal of business research methods health information systems focus groups (# unknown) with 23 healthcare professionals. examine usage of health information exchange systems in 2 kinds of healthcare sectors (i.e. finland and usa) (reid et al. 2010) european journal of information systems challenges for women in it 6 same-sex focus groups (3 each) with 45 it professionals in 3 usa companies identify how men and women describe the unique challenges that women face in the is workplace. (resatsch et al. 2008) – electronic markets point-of-sale rfid 2 focus groups with 10 consumers and 10 sales assistants identify factors determining intentions to use rfid-based smart products (scott et al. 2009) communications of the ais information systems as a career 6 focus groups with 31 undergraduate students identify factors that influence students to select is as a career. (smith and mckeen 2003) communications of the ais knowledge management usage 1 focus group with knowledge managers (# of participants unknown) from 25 companies explore the state of knowledge management usage and its likely future development. (smith and mckeen 2004) communications of the ais knowledge management marketing 1 focus group with practicing knowledge managers (# of participants unknown) identify how managers market knowledge management (km) in organizations to develop a framework for marketing of km within the organization. australasian journal of information systems volume 17 number 2 2012 114 focus group type citation topic sample and other methods (for multimethod papers) focus group purpose sole method: exploratory (continued) (smith and mckeen 2007) communications of the ais information management implementation 1 focus group with it managers (# of participants unknown) identify issues and challenges in information management. (smith et al. 2007) communications of the ais perceptions of is in organizations 1 focus group with it managers from 15 companies identify how is is perceived in companies – compared with similar results from 1996 study. (smith and mckeen 2010a) communications of the ais business it relationship 1 focus group with senior it managers (# of participants unknown) explores the relationship between it and business and how it can be characterized. (smith and mckeen 2010b) communications of the ais business and interpersonal competencies of it staff 1 focus group with senior it managers (# of participants unknown) to explore the business and interpersonal competencies that it staff will need in order to do their jobs effectively over the next five to seven years and what companies should be doing to help develop them. (smith and mckeen 2011) communications of the ais collaboration and it 1 focus group with it managers (# of participants unknown) identify the role of it in enabling collaboration within organizations. sole method: explanatory (sutton et al. 2008) journal of the association for information systems critical risk factors in b2b e-commerce relationships 5 focus groups: 3 internal groups (is security, internal it audit, e-commerce development managers) and 2 external groups (ecommerce consultants, external it auditors) identify critical risk factors used to assess impact of b2b ecommerce on overall enterprise risk. tests performed between groups. multi-method: exploratory (bishop et al. 2000) – journal of the american society for information science and technology use of digital libraries initiative 3 focus groups with 17 participants to supplement information from interviews, observations, usability testing, logs, and surveys. explore the use of the digital libraries initiative at the university of illinois (chesney et al. 2009) information systems journal grieving behaviours in virtual worlds 4 online focus group with 14 participants. combined with 15 hours of non-participant observations identify grieving behaviours in virtual worlds, their impact, why they occur, who the likely targets are, and who perpetrators are. multi-method: exploratory (continued) (cooper and heinze 2007) – journal of information systems education multi-year projects in is education programs (# unknown) focus groups with students and staff, combined with interviews and survey identify best ways to evaluate multi-year projects in is education programs. (hill et al. 1998) – journal of global information management information technology transfer and culture (# unknown) focus groups to supplement interviews and a field study in five arab countries explore linkages between information technology transfer and sociocultural factors that support or impede a successful transfer. (jones et al. 2008) communications of the ais post-installation use of erp 5 focus groups in 1 organization to supplement 104 survey responses from five other organizations determine level and effectiveness of erp training and usage several years after implementation. (kelly et al. 2007) – analytical 2 focus groups with 4 identify evaluation criteria for australasian journal of information systems volume 17 number 2 2012 115 focus group type citation topic sample and other methods (for multimethod papers) focus group purpose journal of the american society for information science and technology questionanswering systems analysts, followed by individual interviews interactive, analytical questionanswering (qa) systems. (kesner 2008) – communication of the ais competencies required of students 4 groups and 14 interviews with approximately 17 employees (15% of survey population), combined with survey of 111 employees identify competencies it students need as perceived by potential employers. (klaus and blanton 2010) european journal of information systems user resistance 1 focus group of 7 it professionals and 34 interviews in 3 organizations identify the determinants of user resistance to it implementation. (lehaney et al. 1999) – journal of end user computing database marketing 1 focus group to supplement survey identify the success factors for technology-enabled database marketing in the tourism industry. (manning 1996) information systems research use of cellular technology in police context (# unknown) focus groups of police officers and their leaders, combined with interviews and observations explore use of it in police context. (nahar et al. 2006) – information & management information technology transfer 16 (or more?) focus groups of 4 to 6 participants (complemented by interviews) of it managers, general managers, and executives in each in 4 finnish companies identify and rank the various factors affecting it-supported international technology transfers. rankings of factors quantitatively evaluated between firms. multi-method: exploratory (continued) (otondo et al. 2009) – european journal of information systems adoption of rfid 2 focus groups with 21 participants, complemented with data from published interviews explores how managers evaluate rfid and participate in its adoption. (pettigrew et al. 2002) – journal of the american society for information science and technology use of library community network systems (# unknown) focus groups of public library staff, human service providers and users, complement surveys and interviews identify how users use public networked community information systems, as well as the barriers and benefits of use. (ramesh and jarke 2001) – ieee transactions on software engineering reference models for requirement traceability 37 focus groups (several phases) in 26 software development firms, combined with interviews identify reference models for requirement traceability and discuss requirements for future traceability mechanisms. (srivastava and teo 2009) – communications of the ais trust in egovernment 5 focus groups with 45 participants in singapore, followed by 27 interviews, and use of secondary data. identify factors determining used trust in e-government. (tillotson et al. 1995) information technology and libraries use of library for internet usage 2 focus groups of 17 undergraduates to supplement online survey and logs of telnet explore use of the internet via the university of toronto library online catalog and information system australasian journal of information systems volume 17 number 2 2012 116 focus group type citation topic sample and other methods (for multimethod papers) focus group purpose connections . (van velsen et al. 2008) – international journal of enterprise information systems requirements elicitation for an erp system 1 focus group with 6 users of erp, compared (findings) with heuristic evaluation of the interface by experts and developers identify most important functions and tasks for erp users. multi-method: concept generation for testing (explanatory) (dickinger et al. 2008) – european journal of information systems adoption of push to talk (ptt) technology 3 focus groups with 23 experts, preceded by 10 interview with experts, followed by survey to test identify factors that affect user intentions to use ptt technology. (galup et al. 2008) journal of computer information systems impact of job characteristics on satisfaction 6 focus groups in a governmental agency (county) in florida followed by survey and interviews generate constructs for model of satisfaction (stage 1 of 3). (goh et al. 2010) – journal of computer information systems mobile tourism 1 focus group with 8 participants to generate items for a survey administered to 210 respondents identify mobile services of importance to tourists. (khalifa and limayem 2003) communications of the acm internet shopping (# unknown) focus groups with 177 internet consumers (undefined) followed by survey identify factors that affect adoption and use of internet shopping. multi-method: concept generation for testing (explanatory) (continued) (krasnova et al. 2010) – journal of information technology online social networks 2 focus groups with 16 students, followed by survey of 259 students generate items to measure students’ determinants of intentions to disclose information. (lee et al. 1995) mis quarterly skills and knowledge requirements for is professionals 50 is practitioners and academics (unknown # of focus groups), followed by survey generate items for subsequent survey of is trends. (smith et al. 1996) – mis quarterly concern for information privacy (# unknown) focus groups with 18 consumers (some interviewed ; some in focus group) and (# unknown) focus groups with 25 employees. combined with interviews; followed by survey. generate and validate items for development of cfip instrument. (torkzadeh et al. 2006) – decision support systems customer relationship management 6 focus groups of 6 customer service representatives, followed by survey with 1460 responses identify key barriers to success of customer relationship management. (wang et al. 2004) – information systems frontiers service quality, customer value, satisfaction (# unknown) focus groups (# unknown participants) identify and refine items to measure service quality beyond the servqual instrument. (weidong and lee 2005) – journal of management information systems information systems development project complexity (# unknown) focus groups with 45 is managers followed by survey generate items to measure isd project complexity (phase 1 of 4). multi-method: (fitch 2007) – evaluation of a 8 focus groups with 65 evaluate usefulness of australasian journal of information systems volume 17 number 2 2012 117 focus group type citation topic sample and other methods (for multimethod papers) focus group purpose concept generation after testing or framework validation (explanatory) decision support systems risk assessment instrument practitioners to supplement survey data (focus groups conducted with subset of survey respondents) information obtained from a risk assessment decision support tool developed by the authors. (price and shanks 2005) – journal of information technology information quality 3 focus groups with it practitioners, it academics, and end users to validate framework develop a framework for information quality then validated with focus group results. (xia and lee 2004) – communications of the acm is development projects focus groups and interviews with 74 development managers, followed by survey of 549 managers to validate framework develop a framework to explain is development project complexity, which is then validated by focus groups, interviews and a survey. multi-method: concept generation after testing or framework validation (continued) (xu et al. 2011) – information processing and management intelligent agent systems for executive information 3 focus groups with 31 middle and top-level managers, followed by 25 interviews to test hypotheses proposes 4 hypotheses that are validated with focus groups and interviews. multi-method: others (chang lee and kwon 2008) – decision support systems map-driven avatar design decision support system 1 focus group with 9 avatar experts (together with brainstorming sessions) uses focus group to extract the cognitive maps of experts and create a collective cognitive map. (smith et al. 2010) – mis quarterly mandated compliance to information security standard 22 focus groups (unknown # participants) as interventions between 8 online surveys (with interview, forums and discussions as other interventions) studies adoption of information systems security (iss) compliance in government. focus groups viewed as action research. table 1. information systems journal articles using the focus group research method using the focus group research method for is theorizing the number of studies and the quality of journals publishing research using focus group research methods in the sample clearly shows that focus group research has a substantive role in the repertoire of research methods for is researchers. the following discussion provides an analysis of the findings from table 1, and further explores how and when the focus group research method can be appropriate in is theorizing. table 1 reveals a number of interesting findings regarding the use of the focus group research method in information systems. first, it is clear that there are quite a few articles published that use this research method in general. second, one can notice that most studies using the focus group research method have been published in recent years: 49 of the 58 studies were published between 2003 and 2011. this suggests that the method is gaining acceptance as an appropriate research method in information systems research. more importantly, the categorization of papers along types of focus group research reveals some interesting trends, which are discussed below, together with the role of each category of focus group research method in is theorizing. before the findings from table 1 are discussed within the context of is theorizing, it is important to define the concepts related to theory. straub (2009) suggests that research rarely develops new australasian journal of information systems volume 17 number 2 2012 118 theory, but instead focuses on adapting or refining existing theories. yet, theory development is core to the growth of a research discipline with many and various phenomena of interest. in discussing the role of theories in information systems, gregor (2006) identified fives types of theories: analyzing, explaining, predicting, explaining and predicting, and design and action. importantly, different types of theories can provide different insights into an information systems domain. table 2 summarizes the different types of theories, and suggests how focus groups inform theory development for each type of theory. the subsequent discussion of each category of focus group studies will offer further examples of how they can be used in is theorizing. theory type definition sample use of focus groups examples using focus groups in is analyzing describe the state of information systems phenomena. elicit knowledge about a research topic from domain experts in a series of focus groups. jarvenpaa and lang (2005) identify services used by mobile technology users, and issues related to use of mobile it. explaining explain what is occurring but do not provide testable predictions. identify and rank important factors explaining usage of an it via focus groups of managers. nahar et al (2006) identify critical success factors for it transfers and rank them across firms. predicting provide testable predictions without well-developed causal relationships. use focus groups to elicit possible outcomes from an is. campbell et al (2005) explore how managers view the alignment between business and it. explaining and predicting explain what is occurring and provide testable predictions with causal explanations. use focus groups to elicit items for further testing and/or to develop a testable model. smith et al (1996) generate and validate items for development of the concern for information privacy instrument. design and action design an it artifact or a framework to evaluate such artifacts. elicit requirements for the development of an artifact; for example, showing a prototype to focus groups for feedback. lerouge and niederman (2006) identify architectural requirements for a public health knowledge system. table 2. focus groups and theory types (adapted from (gregor 2006)) as stated in the prior discussion, the focus group research method can be useful is is theorizing when ideas can be generated from group interactions or when constructs exist in or emerge from the interactions of individuals with one another. different from interview data, focus group data allow australasian journal of information systems volume 17 number 2 2012 119 researchers to understand the extent to which factors are perceived similarly and/or the extent of shared meaning and relative value of the constructs. focus group as sole method exploratory an important finding from table 1 is that the majority of papers using the focus group research method in information systems fall in the sole method – exploratory category (25 of 58). this is interesting not only because it is consistent with the type of research for which focus group research methods are recommended (krueger 1994), but also because it suggests that many more research projects could consider the focus group research method since new phenomena regularly become of interest to is researchers. for example, emerging technologies offer many topics: what are the new rules of social interactions with the increased use of social networking? or, what are the potential alterations to group culture when individuals interact substantially in virtual worlds? in analyzing the list of “sole method – exploratory” papers in table 1, it can be noted that most topics, with some exceptions, attempt to identify usage or managerial issues related to technology, systems, and it management. in fact, several studies investigate how individuals use email, mobile technology, dashboards, crm, health information systems, rfid, knowledge management, databases, application portfolio management, total cost of ownership, or digital libraries. in each of these studies, the main purpose of the focus group is to identify usage levels, determinants of use, or ways in which the system/technology/approach is used. as will be argued below, these exploratory studies are important in the research process towards future theory development. the second main group of studies investigate is managers’ perceptions of various concepts related to it management, such as it staffing issues, is-business alignment, teaching-research alignment, is delivery options, is careers, and general perceptions of is 1 . these studies often use one large session where several 1 it should be noted that several articles in the sole method – exploratory category are from the “developments in practice” series published in the communications of the ais journal by mckeen and smith (and other colleagues) mckeen, j. d., and smith, h. "developments in practice xxvii: delivery it functions: a decision framework," communications of the association for information systems (19) 2007, p 9, mckeen, j. d., and smith, h. "developments in practice xxxiv: application portfolio management," communications of the association for information systems (26) 2010a, p 87, mckeen, j. d., and smith, h. "developments in practice xxxvii: total cost of ownership," communications of the association for information systems (27) 2010b, p 627, mckeen, j. d., smith, h., and jin, j. "developments in practice xxxii: successful strategies for it staffing," ibid. (24:1) 2009, p 32, mckeen, j. d., and smith, h. a. "developments in practice xxiii: creating and evolving a technology roadmap," communications of ais (2006:18) 2006, pp. 451-463, mckeen, j. d., smith, h. a., and singh, s. "developments in practice xx: digital dashboards: keep your eyes on the road," communications of the association for information systems (16) 2005, p 1, smith, h. a., and mckeen, j. d. "developments in practice ix: the evolution of the km function," communications of ais (2003:12) 2003, pp. 69-79, ibid., smith, h. a., and mckeen, j. d. "developments in practice xiv: marketing km to the organization," communications of the association for information systems (14) 2004, p 1, smith, h. a., and mckeen, j. d. "developments in practice xxiv: information management: the nexus of business and it," communications of ais (2007:19) 2007, pp. 34-46, smith, h. a., mckeen, j. d., and singh, s. "developments in practice xxviii: managing perceptions of is," ibid.:20), pp. 760-773.. australasian journal of information systems volume 17 number 2 2012 120 managers share their ideas on a general topic. this does not follow the typical rules of focus groups (4 to 6 participants; several focus groups to allow for comparisons), and as a result should be considered mainly as informative. however, there could still be value in these as idea generation for research questions, as will be discussed below. sole method focus groups for is theorizing because the focus group research method allows concepts that would only emerge through interactions among individuals to be identified (grant and fitzgerald 2005; hartman 2004), it is especially useful in is theorizing when researchers are interested in group level phenomena (e.g., group cohesion in software development teams) or in phenomena that exist in the relationships among people (e.g., team satisfaction with technology used in teams within telecommuting environments). sopreperez’ (2008) study of a uk fire and rescue service using focus groups highlights the value of group interaction. the researcher concludes, “some details only emerged when those in attendance were given the opportunity to discuss the incident as a group. individual interviews would not have given this opportunity and observations would not have revealed attitudes and beliefs.” (p. 187). in information systems research, lerouge and niederman (2006) provide an example of how this type of focus groups can be useful in their study of architectural requirements for a public health knowledge system. they highlight the benefits of the focus group research method for identifying requirements for public health knowledge management systems: 1. “…stakeholder perspectives of the scope and potential of knowledge management systems were limited as the study began. by focusing on conceptual issues, the focus group method highlighted individual stakeholders’ reflections in building these conceptual insights… 2. … requirements gathering of systems such as knowledge bases may involve complex or vague topics as stakeholders look at new technologies or multifaceted constructs. when soliciting data from the field, it is sometimes difficult to achieve full explanation and understanding in surveys, especially in initial stages of research. focus groups allow the gaps to be filled in by fellow participants. limited moderator prompts may help ensure the topics and issues are understood, yet avoid excessive intervention by project directors, sponsors, or researchers… 3. …because the focus group draws participants from the full set of involved stakeholders, and because the participants are exposed to a wide range of ideas regarding the overall knowledge management program, these participants are themselves positioned to more easily continue diffusing knowledge about the program to nonparticipating stakeholders....” (pp. 18-19). the sole method exploratory focus groups can also be useful in generating research questions since the method allows researchers to probe subjects further when interesting concepts emerge from the “discussion” (edmunds 1999) and allows concepts unclear to participants at the beginning of a session to become clarified through their interactions (grant and fitzgerald 2005; hartman 2004). these would not result in analyzing theories as defined by gregor, but can serve to identify research questions that are driven by the views and needs of participants for future work. in table 1, this category is by far the largest, but as previously stated, one has to account for the “developments in practice” series by mckeen and smith published in communications of the ais. many phenomena of interest to information systems researchers could fall in the categories of exploratory concepts (e.g., understanding usage of emerging technologies, exploring new concerns of cios, etc.) or concepts initially unclear to participants (e.g., critical success factors of new it strategies). for example, asynchronous but moderated focus groups could be used to elicit knowledge from domain experts. building on market research where focus groups are used extensively to assess and provide feedback on products, services, or concepts, some is researchers have explored the use of the internet to conduct focus groups for eliciting information from groups of consumers online (clapper and massey 1996; markham et al. 2006; montoya-weiss et al. 1998). others have developed an internet tool to australasian journal of information systems volume 17 number 2 2012 121 run online focus groups, which are said to reduce costs, enable participation of more dispersed subjects, and offer anonymity (montoya-weiss et al. 1998). as discussed by the authors, technology can help mitigate some of the limitations of face-to-face focus groups. another potential use of this category of exploratory focus group studies is for requirements determination and design science (leifer et al. 1994; o'heocha et al. 2010; peffers and tuunanen 2005; tremblay et al. 2010). potential issues with using single method focus groups for is theorizing while the focus group research method as sole method – exploratory can be valuable in is theorizing, there are some important pitfalls, many of which are found in the studies presented in table 1. in particular, one has to be concerned with the lack of rigor regarding the sampling and research designs for this category of focus group research in the is literature. the first issue is that several studies fail to disclose necessary information to evaluate the validity of the findings, such as the number of focus groups conducted, the number of individuals per focus group, or the nature of participants. all future is studies using focus groups should clearly identify these important research design elements. without proper information on participants, it is impossible to discuss the generalizability of the findings, an issue further discussed in the “limitations of focus groups” section presented later in this paper. a second major issue is that many of the focus group-based studies in table 1 (as single method) use only one focus group. as a result, these studies do not allow for comparisons across groups. while this may not be an issue for exploratory studies seeking to simply elicit information (generating lists of concepts), it is of major concern when one tries to develop a theory based on the one focus group. depending on the composition of the group (homogenous or heterogeneous strangers), the researcher may find completely different findings. for example, a researcher interested in the success of it implementations in a firm can assemble participants for a focus group a with all participants from the it department, and arrive at a particular conclusion based on their inputs. the researcher could also assemble for focus group b with only participants from one user department, and will likely obtain different conclusions. finally, the researcher can create a focus group c, with a mix of it and non-it participants. 2 each of the groups would provide a different perspective on the success of the it implementation in the firm. each perspective could be valuable, depending on the interests or research question of the researcher. if the researcher wants to see how biased the view of the it team is, then using groups a and b would provide valuable insights (using more than one group of each to achieve saturation). an example from table 1 where this approach was well done is the study by reid et al (2010) who purposely designed the study with several same sex focus groups to identify the challenges women face in the is workplace were perceived similarly or differently by men and women. the use of several homogenous groups allowed the researchers to “validate” their findings across groups. in summary, the research design should identify which population is of interest to the particular research question, and then include enough focus groups and participants to achieve saturation, i.e., enough members from all stakeholder groups, relevant to the question and the intended references (what are we trying to say that can be obtained from the focus group findings?). other researchers suggest that focus groups should only be used when part of a multi-method methodology. for example, in discussing the nexus between teaching and research, grant and fitzgerald (2005) conclude that “the issue of measurement bias is a major problem for users of focus groups” (p. 50) and that “focus groups are not beneficial as standalone research methods” (p. 52). it 2 an anonymous reviewer is to be thanked for inspiring this example. australasian journal of information systems volume 17 number 2 2012 122 should be noted, however, that some of the decisions involving the design of their focus groups could have impacted their limited findings. first, the authors ran only two focus groups, not seeking saturation in their findings (carlsen and glenton 2011). more importantly, the authors did not select homogeneous strangers (subjects relatively “compatible” with each other) for their focus groups. for example, their second focus group included senior faculty, including a head of a school and a professor with an established research program, two faculty with established research but one new to the department, and two non-research active faculty, both long-standing and one responsible for the largest undergraduate program. academics active on committees in various academic institutions worldwide would probably recognize the potential issues of power plays, coalitions, and the like in such a group of individuals. it is therefore possible that the lack of “appropriate” results identified with the focus groups could be the result of the design of the study itself as opposed to weaknesses in the method. the authors suggest that other researchers may have had biased findings when using focus groups because of the “power and political interplay being exercised to give the illusion of a consensus.” however, as previously stated, it is this interplay that provides strength to focus group findings in theory development related to is phenomena where conducting individual interviews or surveys may not reveal important constructs. the important implication for is theorizing is that researchers must recognize whether the concepts of interest are meant to emerge from group discussions or are embedded in the social context of groups or communities (e.g., satisfaction with technology used in teams within telecommuting environments); or, whether they are clearly of individual-only concern (e.g., perceptions of mobile complexity). for example, a study that requires an understanding of individual perceptions or attitudes on their own, not embedded in social interactions, would benefit more from individual interviews or quantitative methods (e.g. an understanding of self-efficacy). in summary, and as suggested by morgan (1997), the focus group research method as sole method can provide greater insights than the sum of individual interviews through its synergistic effects. this is consistent with the advantages of the focus group research method presented by lerouge and niederman (2006), which could apply to other is studies when concepts would only emerge through interactions among individuals (e.g., cultural effects in software development team interactions). in other words, when researchers are interested in phenomena where interactions of individuals are important, the focus group research method can be a better research method than individual interviews. if, however, the researcher is interested in analyzing individuals’ intra-personal cognitions or perceptions (e.g., self-efficacy), the interview research method (or other methods like surveys) might be more useful (although focus groups could still be used in the early stages of the research to elicit dimensions). focus group as sole method explanatory there was only one study where the focus group research method was used as sole method to provide explanations (test) of concepts. in this study, sutton et al (2008) conduct five focus groups with various stakeholders representing internal constituency groups (is security, internal it audit, and ecommerce development managers) and external constituency groups (ecommerce consultants, and external it auditors) to identify critical risk factors to assess the impact of b2b e-commerce on overall enterprise risk. while the main effort is in the elicitation of the risk factors, the authors go beyond simple enumeration and quantification of concepts to testing the differences in the concepts identified between constituency groups, thereby providing a test of their findings. many studies identified in the sole method category could have taken these further steps to compare results across groups. for example, the authors could have conducted between group analyses for their data across countries in jarvenpaa and lang (2005) and jarvenpaa et al (2003), or between groups of administrators, educators, and students in lerouge and niederman (2006). this does not take away australasian journal of information systems volume 17 number 2 2012 123 from what has been accomplished in terms of knowledge elicitation in these studies since the goal in focus group research is usually saturation, where repeated focus group sessions should result in similar findings to a point where no new insight is gained from additional focus groups (carlsen and glenton 2011). group comparisons in this case would not provide any differences. yet, the possibility of comparing findings across diverse groups can provide further insights into the exploratory findings previously obtained (assuming saturation is reached with homogenous groups first and that there is availability of different stakeholder groups). focus group as part of multi-method: exploratory published journal articles using focus groups that fall in the multi-method category can also be exploratory or explanatory. exploratory papers tend to combine findings from focus groups with surveys and/or interview data to obtain a more global picture of the phenomenon of interest. in fact, findings from table 1 suggest that the most often used complementary research method to focus groups is interviews (found in 10 of the multi-method exploratory studies) and surveys (found in nine of the multi-method exploratory studies). other methods used in combination with focus groups include observations (3), logs (2), secondary data (2), and an expert evaluation (van velsen et al. 2008). an analysis of the studies in this category in table 1 reveals some interesting patterns, namely that the multi-method exploratory research studies explore similar topics as the single method exploratory focus groups, and suffer from some of the same issues. as can be seen from the table, the main topics continue to be on usage (e.g., of analytical question-answering systems, digital libraries, erp, database marketing, cellular technology, rfid) or identification (e.g., behaviours in virtual worlds, evaluation factors, it transfers, competencies, resistance factors, trust factors). more importantly, there remain issues of lack of information with four of the 16 studies not specifying the number of focus groups and several studies not indicating the number of participants or the nature of the participants. three of the studies were also conducted with only one focus group. while it might be less of a concern since the focus group is usually used only for eliciting concepts, it remains that using only one focus group might result in some missing concepts, as explained previously. multi-method (exploratory) focus groups for is theorizing the use of multi-method exploratory focus groups is consistent with the case study method to theory building (eisenhardt 1989; orlikowski and baroudi 1991; strauss and corbin 1990; yin 1994) where multiple sources of data must be used (myers 1997; yin 1994). a good example of this type of study is the work of manning (1996) who spent considerable time and effort studying the use of technologies by police officers. combining individual interviews with focus group data and observations allowed the researcher to gain a deep understanding of such use. there was no stated research model to be tested in the paper, and no particular grounded theory was presented as a result of the analyses in the paper. yet, the findings provide for an in-depth discussion of organizational use of technology, and the utility of the dramaturgical perspective used by the author for the analysis of technological innovation in organizational contexts. as used in later studies, the technology-inpractice perspective suggests that rules regulating technology use are developed through recurrent use of the technology (orlikowski 2000). eisenhardt (1989) proposes several steps in the process of theory building: 1. getting started (defining the research question and any a priori constructs), 2. selecting cases (not having hypotheses but selecting a specified population), 3. crafting instruments and protocols (making sure that multiple data collection methods are used, both qualitative and quantitative, and using multiple investigators), 4. entering the field (where data collection and analysis overlap, and data collection can be flexible and opportunistic), 5. analyzing data (both within case and cross-case), 6. shaping hypotheses (with iterative tabulation of evidence, using replication and/or logic across cases; searching for evidence for australasian journal of information systems volume 17 number 2 2012 124 why behind the relationships identified), 7. enfolding literature (comparing with conflicting and similar literature), and 8. reaching closure (with theoretical saturation when possible). while her work was more specifically targeted at case studies, the concepts to the multi-method focus group research can be applied as follows. step 1 remains the definition of the problem or research question of interest. step 2 involves selecting the appropriate population for the focus groups that is relevant to the research question. step 3 involves developing the focus group protocol (see appendix a for more details) and selecting the complementary research methods. step 4 involves conducting the focus groups while allowing new insights to be incorporated into subsequent focus groups or other data collection methods (like interviews). steps 5 to 8 remain similar (analysing the collected data, shaping hypotheses, comparing to the literature, and reaching closure), with the objective of developing theory. an example of the use of focus groups for grounded theory work comes from the author’s (and colleagues) study of online children privacy where the researchers were trying to identify what factors would make parents use an online privacy protection tool (bélanger et al. 2009). most researchers would use long-standing technology adoption models as theoretical foundations for their work. using the technology acceptance model (davis 1989) or the unified theory of acceptance and use of technology (venkatesh et al. 2003), researchers would likely create a model with perceived usefulness (or performance expectancy), perceived ease of use (or effort expectancy), and a variety of other factors such as social influence or facilitating conditions, as impacting likely intentions to use the software. instead, we conducted focus groups and interviews with actual parents to build an emergent model, which did result in the inclusion of ease of use and usefulness, but also awareness of laws and risks, trust, and behavioural control, among others. the focus groups allowed parents to discuss what was important to them, and to build on one another’s comments. in the real world (which they were in), parents often discuss with one another the best ways to protect children, and this emerged from the focus groups. more importantly, we found that intentions to use the privacy protection tool was not the ultimate dependent variable, but that various techniques (such as keeping a home computer in view) were as important if not more important for parents in protecting their children’s online privacy. in this case, parents were considered experts since their level of knowledge was part of the information the study was seeking. the grounded theory model derived from the focus group data was then tested using a large scale survey. in summary, while the findings from table 1 reveal that several issues remain in many of the multimethod exploratory focus group research studies in terms of sampling, the examples show that the exploratory focus group used in conjunction with other research methods can be useful in developing grounded is theories. focus group as part of multi-method: explanatory the second type of multi-method focus group research papers in information systems involve some form of testing using one of the “other” methods besides the focus group research method, providing a positivist view of research. most of them (10) from the sampled papers in table 1 use the focus group research method to generate constructs or items for model or scale development, and then use survey questionnaires (and sometimes interviews) for model testing. a few others (4) use focus groups after survey data have been collected and analysed to provide further insights into the findings, or to validate developed frameworks. only one paper uses focus group data to test hypotheses (xu et al. 2011). using focus groups for concept generation followed by testing is consistent with the research cycle of exploratory to explanatory studies since the use of focus groups as part of a larger research program offers triangulation for validating research, as called for by mcgrath (1982). australasian journal of information systems volume 17 number 2 2012 125 in table 1, it can be seen that research papers in the explanatory multi-method focus group research category differ from the exploratory papers in many ways. first, topics of interest move beyond usage and identification to explanation of a variety of is phenomena. for example, there are studies of impacts (e.g., job characteristics on is employee satisfaction (galup et al. 2008)) or adoption (e.g., push-to-talk technologies (dickinger et al. 2008)), or development of new is constructs such as concern for information privacy (smith et al. 1996) or is development project complexity (weidong and lee 2005). a second major difference is that most of the studies provide detailed information on the focus group sample, quantity, and the nature of participants, and most of them include many focus groups (only one study uses one focus group). the fact that more planning appears to have been done for those studies as opposed to the previously discussed ones can be due to 1. the more advanced state of knowledge of the topics of interest (as opposed to exploratory topics), or 2. the fact that many of these studies are published in leading is journals, which might be more demanding regarding details to be provided. multi-method (explanatory) focus groups for is theorizing in analysing the papers in table 1, it is clear that the multi-method explanatory focus group research method has been used for theory development, more specifically for the “explaining and predicting” types of theories proposed by gregor (2006). for these theories, where researchers seek to explain what is occurring and provide testable predictions with causal explanations, focus groups can be used to elicit items for further testing and/or to develop the testable model (similar to what was done with exploratory focus groups presented before). this is most beneficial when is researchers are interested in concepts that would only emerge through interactions among individuals. in these cases, the focus group research method can provide greater insights than the sum of individual interviews through its synergistic effects, allowing researchers to obtain a more realistic emergent model for later testing. if these interactions are important to the concepts of interest, a model built from interview data only might be missing some key constructs or relationships. a good example of using focus groups in developing explaining and predicting theories (gregor 2006) is the work of dickinger et al. (2008) who employ the focus group research method to identify factors that affect user intentions to use push to talk technology (ptt). more specifically, their methodology combines literature review and 10 interviews to define an initial model, followed by three focus groups to develop initial hypotheses to be tested with survey data. as shown in figure 1, their use of multiple methods follows a clear evolution from exploratory methodologies (literature, interviews and focus groups) to confirmative (explanatory) methodologies (survey and analyses). australasian journal of information systems volume 17 number 2 2012 126 literature review expert interviews focus groups survey analysis & presentation determination of key success factors definition of acceptance model formulation of hypotheses empirical testing presentation of results e x p lo ra ti v e r e s e a rc h p ro c e s s c o n fi rm a ti v e r e s e a rc h p ro c e s s figure 1. research process to study adoption of push to talk technology (from dickinger et al. 2008) another example from a study of attraction and retention of minorities in computing majors conducted by the author and colleagues (lewis et al. 2007) highlights the value of focus groups in unearthing key concepts. in the study, the researchers conducted a survey based on the literature to explain why minorities stay into or leave computing majors. the survey included a number of intrapersonal factors identified from the literature, such as coping styles, ethnic identity, affinity with the major, and even emotional intelligence. some findings from the survey data were surprising, so subsequent to conducting the survey a series of seven focus groups were conducted with minority computing students. the focus group findings confirmed some of the previously identified factors, such as coping strategies, and helped identify additional factors such as stressors. interestingly, the visual data helped identify a number of additional factors related to stereotypes in the major that helped explain prior findings. for example, many of the male students reacted with rolled eyes when the female member of one focus group discussed the issue of unbalanced life; or body reactions indicated that even students who denied that lack of confidence was a major stressor, actually thought it was important (but would never admit it). these findings were very helpful to the researchers in better understanding the retention issues they were studying. multi-method: others finally, there were two studies in the sample of papers that were classified as “others” because they combined focus groups with other methods in non-standard ways. in one of the “other” studies, smith and colleagues (2010) use 22 focus groups (as well as interview, forums and discussions) as interventions between a series of online surveys to study information systems security standard compliance in government. in this case, focus groups are viewed as action research. in the other study, chang lee and kwon (2008) used a focus group to extract the cognitive maps of experts and australasian journal of information systems volume 17 number 2 2012 127 create a collective cognitive map to be used in the development of a decision support system using avatars. because the interest of the researchers was on defining the collective cognitive map, the use of focus groups was very appropriate. in fact, it is likely that other group support system research use focus groups to create cognitive mapping systems. this type of research would be considered the development of a design and action theory, an area that many researchers suggest need further work (bélanger and crossler 2011; gregor 2006; hevner et al. 2004). for this type of theory building, focus groups can be used prior to the it artefact design as knowledge elicitation (as done in the above mentioned study) or after the it artefact is designed for testing of the artefact (e.g., usability testing). limitations of the focus group research method the focus group research method is not without its limitations. a good summary of the benefits and limitations of focus group research is provided by morgan and spanish (1984): “… the strengths of focus groups come from a compromise between the strengths found in other qualitative methods. like participant observation, they allow access to a process that qualitative researchers are often centrally interested in: interaction. like in-depth interviewing, they allow access to the content that we are often interested in: the attitudes and experiences of our informants. as a compromise, focus groups are neither as strong as participant observation on the naturalistic observation of interaction, nor as strong as interviewing on the direct probing of informant knowledge, but they do a better job of combining these two goals than either of the other two techniques." (p. 260). some criticisms of the focus group research methods are that they are too dependent on the skills of the moderator, they have an unclear level of analysis, and have measurement bias (grant and fitzgerald 2005). others have stated how focus groups represent collective thinking only; however, the main criticism is often validity. having conducted several research projects with focus groups (as part of large research programs), i know that reviewers often question whether the findings of a study that uses the focus group method are valid. clearly, positivist quantitative approaches to research have the favour of many researchers because of their ability to offer statistical validity (cook and campbell 1979) and to be “verifiable” through future studies (in reality, as a percentage, few published is studies have actually been replicated in research). although one can obtain quantifiable results using focus groups (as with interviews), focus groups are not designed for statistical validity (or internal validity as described below). yet, they do meet other validity objectives as identified by mcgrath (1982). the quote at the beginning of this paper is representative of the issues faced by researchers when making research design choices. mcgrath (1982) proposes that there are several types of validity that must be balanced: internal validity, external validity, and relevance. mcgrath also suggests that these validities represent mutually conflicting objectives: “there is no way […] to maximize all three conflicting desiderata of the research strategy domain.” table 3 summarizes how focus groups can address these three validities. in terms of generalizability, focus groups deal with a selected subset of a population, which is typically not randomly selected. in conducting research with non-probability sampling, whether qualitative or quantitative, researchers must clearly define the populations to which the findings apply (seddon and scheepers 2011). in discussing case studies, yin (1994) suggests there are two types of generalizability: statistical and analytical, and argues that for qualitative research analytical generalizability is necessary. analytical generalizability occurs when the results of the study provide an analytical understanding of individuals’ behaviours or motivations. when focus groups are properly conducted and documented, they provide an understanding of why individuals think or behave in a certain way, as can be seen from the examples presented in the paper. furthermore, since australasian journal of information systems volume 17 number 2 2012 128 subjects are considered experts in the topic of interest, their collective voice is representative of what others like them think of the subject. as discussed in lee and baskerville (2003), qualitative research can provide claims of generalizability when the context of the study and the boundaries are clearly delineated. in their recent discussion on generalizability, seddon and scheepers (2011) highlight the need for qualitative studies (or as relevant to this discussion, focus groups as sole method or as part of a multi-method effort) to clearly disclose and discuss the boundary conditions to which their findings apply. if researchers do provide their protocol as part of their publications, and clearly describe the population of interest, other researchers should be able to replicate the findings, just as survey instruments can be validated in multiple studies. unfortunately, as demonstrated by studies on generalizability (lee and baskerville 2003; seddon and scheepers 2011), and consistent with the findings in table 1, researchers often fail to disclose this important information. validity description focus group research requirement external validity (generalizability) extent to which the presumed finding can be generalized to and across different types of persons, settings, and times (populations). focus groups provide analytical generalizability, which is an analytical understanding of individuals’ behaviors or motivations. internal validity precision in the control and measurement of variables. focus group researchers need to provide a thorough description of the complete focus group process and data interpretation. relevance (realism) whether the phenomenon of concern is studied in a realistic setting. focus group participants are considered experts for the particular domain of interest. table 3. validity and focus groups in terms of internal validity, or reliability, it is clear that this cannot be accomplished through controls and precision of the measurement method and its ability to reproduce the same results repeatedly. however, sykes (1991) discusses how in qualitative research, reliability refers to the role of the researcher in the data collection and in her interpretation. she recommends the qualitative researcher provide a thorough description of the complete process used, including preparation, data collection, and analysis. in the context of focus groups, researchers should clearly indicate the research purpose, the selection criteria, and method used to solicit subjects, the focus group protocol, the capabilities and/or training of the moderator, the settings and tools used for the focus group sessions, the number of sessions held and the time spent at each session, and an actual description of the focus group sessions themselves (including a description of the transcriptions, analyses performed, etc.). clearly, based on the findings from table 1, is researchers should be more careful in this regard when using the focus group research method for their studies. finally, an additional potential limitation of the focus group research method is groupthink, a psychological phenomenon where individuals’ thinking tends to be overridden by the group’s australasian journal of information systems volume 17 number 2 2012 129 thinking as individuals seek harmony and reduced conflict in group conversations. groupthink has been identified as an issue in other research methods such as the delphi research method 3 . it is the role of the focus group moderator to avoid such a phenomenon from occurring when conducting the focus group sessions. conclusion the focus group research method is a qualitative research method of interest to information systems researchers. while it has some limitations with respect to statistical validity, if properly planned, conducted, and documented, the focus group research method can offer valuable insights to is researchers for a plethora of information systems phenomena where individuals are embedded in collectives (morgeson and hofmann 1999) that impact the emergence of their perceptions, attitudes and behaviors. more importantly, when is researchers are interested in constructs that emerge from or exist in group interactions, when they are exploring new is concepts to be tested later, or when they seek a deeper understanding of constructs in established theories, the focus group research method can provide valuable insights for is theorizing. acknowledgements the paper greatly benefited from comments from dr. steve sheetz, kathy enget, as well as the editors and reviewers of ajis. references baskerville, r. l., and myers, m. d. "fashion waves in information systems research and practice," mis quarterly (33:4) 2009, pp. 647-662. bélanger, f., and crossler, r. e. "privacy in the digital age: a review of information privacy in is literature," mis quarterly (35:4), december 2011, pp. 1017-1041. bélanger, f., crossler, r. e., hiller, j. s., park, j.-m., and hsiao, m. "children online privacy: issues with parental awareness and control," in: annals of emerging research in information assurance, security and privacy services, h.r. rao and s. upadhyaya (eds.), reed elsevier, 2009, pp. 311-333. bishop, a. p., neumann, l. j., star, s. l., and merkel, c. "digital libraries: situating use in changing information infrastructure," journal of the american society for information science and technology (51:4) 2000, p 394. bryman, a. quantity and quality in social research unwin hyman, london, 1988. campbell, b., kay, r., and avison, d. "strategic alignment: a practitioner's perspective," journal of enterprise information management (18:6) 2005, pp. 653-664. carlsen, b., and glenton, c. "what about n? a methodological study of sample-size reporting in focus group studies," bmc medical research methodology (11:26) 2011, pp. 2-10. chang lee, k., and kwon, s. "a cognitive map-driven avatar design recommendation dss and its empirical validity," decision support systems (45:3) 2008, pp. 461-472. 3 a reviewer is to be thanked for this insightful addition to the limitations. australasian journal of information systems volume 17 number 2 2012 130 chesney, t., coyne, i., logan, b., and madden, n. "griefing in virtual worlds: causes, casualties and coping strategies," information systems journal (19:6) 2009, pp. 525-548. clapper, d. l., and massey, a. p. "electronic focus groups: a framework for exploration," information & management (30:1) 1996, p 43. cook, t. d., and campbell, d. t. quasi-experimental design: design and analysis issues for field settings rand mcnally, skokie, il, 1979, pp. 223-336. cooper, g., and heinze, a. "centralisation of assessment: meeting the challenges of multi-year team projects in information systems education," journal of information systems education (18:3), fall2007 2007, pp. 345-355. davis, f. d. "perceived usefulness, perceived ease of use and user acceptance of information technology," mis quarterly (13:3) 1989, pp. 319-340. dickinger, a., arami, m., and meyer, d. "the role of perceived enjoyment and social norm in the adoption of technology with network externalities," european journal of information systems (17:1) 2008, pp. 4-11. dunning, k. a., vijayaraman, b. s., krovi, r., and kahai, p. s. "graduate e-business program design and evaluation," journal of computer information systems (42:1), fall2001 2001, p 58. dwyer, c., and knapp, c. a. "how useful is is 2002? a case study applying the model curriculum," journal of information systems education (15:4) 2004, pp. 409-416. edmunds, h. the focus group research handbook ntc contemporary, lincolnwood, il, 1999. eisenhardt, k. m. "building theory from case study research," academy of management review (14:4) 1989, pp. 532-550. fern, e. f. "the use of focus groups for idea generation: the effects of group size, acquaintanceship, and moderator on response quantity and quality," journal of marketing research (19:1) 1982, pp. 1-13. fern, e. f. advanced focus group research sage publications, thousand oaks, ca, 2001. fitch, d. "structural equation modeling the use of a risk assessment instrument in child protective services," decision support systems (42:4) 2007, pp. 2137-2152. galup, s. d., klein, g., and jiang, j. j. "the impacts of job characteristics on is employee satisfaction: a comparison between permanent and temporary employees," journal of computer information systems (48:4), summer2008 2008, pp. 58-68. glass, r. "making research more relevant while not diminishing its rigor," ieee software (26:2) 2009, p 96. goh, d., ang, r., and lee, c. "determining services for the mobile tourist," the journal of computer information systems (51:1) 2010, pp. 31-40. grant, k., and fitzgerald, s. "the nexus between teaching and research: a qualitative study using two focus group on academic information systems teachers," electronic journal of business research methods (3:1) 2005, pp. 37-56. gregor, s. "the nature of theory in information systems," mis quarterly (30:3), september 2006, pp. 611-642. hartman, j. "using focus groups to conduct business communciations research," the journal of business communications (41:4) 2004, pp. 402-410. hevner, a. r., march, s. t., park, j., and ram, s. "design science in information systems research," mis quarterly (28:1) 2004, pp. 75-105. hill, c. e., loch, k. d., straub, d. w., and el-sheshai, k. "a qualitative assessment of arab culture and information technology transfer," journal of global information management (6:3) 1998, pp. 29-38. huang, e., and lin, s. "do knowledge workers use e-mail wisely?," the journal of computer information systems (50:1) 2009, pp. 65-73. australasian journal of information systems volume 17 number 2 2012 131 jarvenpaa, s. l., and lang, k. r. "managing the paradoxes of mobile technology," information systems management (22:4), fall2005 2005, pp. 7-23. jarvenpaa, s. l., lang, k. r., takeda, y., and tuunainen, v. k. "mobile commerce at crossroads," communications of the acm (46:12) 2003, pp. 41-44. jones, m., zmud, r., and clark, t., jr. "erp in practice: a snapshot of post-installation perception and behaviors," communications of the association for information systems (23:article 25) 2008, pp. 437-462. kelly, d., wacholder, n., rittman, r., sun, y., kantor, p., small, s., and strzalkowski, t. "using interview data to identify evaluation criteria for interactive, analytical questionanswering systems," journal of the american society for information science & technology (58:7) 2007, pp. 1032-1043. kesner, r. m. "business school undergraduate information management competencies: a study of employer expectations and associated curricular recommendations," communications of ais (2008:23) 2008, pp. 633-654. khalifa, m., and limayem, m. "drivers of internet shopping," communications of the acm (46:12) 2003, pp. 233-239. klaus, t., and blanton, j. "user resistance determinants and the psychological contract in enterprise system implementations," european journal of information systems (19:6) 2010, pp. 625-636. krasnova, h., spiekermann, s., koroleva, k., and hildebrand, t. "online social networks: why we disclose," journal of information technology (palgrave macmillan) (25:2) 2010, pp. 109125. krueger, r. a. focus groups: a practical guide for applied research sage publications, thousand oaks, ca, 1994. krueger, r. a., and casey, m. a. focus groups: a practical guide for applied research sage, thousand oaks, ca, 2000. le rouge, c., and niederman, f. "information systems and health care xi: public health knowledge management architecture design: a case study," communications of ais (2006:18) 2006, pp. 2-54. lee, a. s., and baskerville, r. l. "generalizing generalizability in information systems research," information systems research (14:3) 2003, pp. 221-243. lee, d. m. s., trauth, e. m., and farwell, d. "critical skills and knowledge requirements of is professionals: a joint academic/industry investigation," mis quarterly (19:3) 1995, pp. 313-340. lehaney, b., clarke, s., kimberlee, v., and spencer-matthews, s. "the human side of information development: a case of an intervention at a british visitor attraction," journal of end user computing (11:4) 1999, pp. 33-39. leifer, r., lee, s., and durgee, j. "deep structures: real information requirements determination," information & management (27:5) 1994, pp. 275-285. lewis, t., smith, w. j., bélanger, f., and harrington, k. v. "a qualitative analysis of computing students’ professional identity and its relationship to strategies for coping with stressors in the computing disciplines," america’s conference on information systems (amcis), keystone, colorado, 2007. maheshwari, s., pierce, a., and zapatero, e. "understanding the lack of minority representation in graduate programs in computer science and information technology: a focus group study of student perceptions," academy of information and management sciences journal (12:1/2) 2009, pp. 71-90. manning, p. k. "information technology in the police context: the "sailor" phone," information systems research (7:1) 1996, pp. 52-62. australasian journal of information systems volume 17 number 2 2012 132 markham, s., gatlin-watts, r., and cangelosi, j. "the internet and its relationship to buyer behavior theory," journal of internet commerce (5:1) 2006, pp. 69-86. martz, b., zhang, x., and ozanich, g. "information systems and healthcare xix: developing an integrative health informatics graduate curriculum," communications of ais (2007:19) 2007, pp. 645-654. mcgrath "dilemmatics: the study of research choices and dilemmas," in: judgment calls in research j.e. mcgrath, j. martin and r.a. kulka (eds.), sage, newbury park, ca, 1982, pp. 69-102. mckeen, j. d., and smith, h. "developments in practice xxvii: delivery it functions: a decision framework," communications of the association for information systems (19) 2007, p 9. mckeen, j. d., and smith, h. "developments in practice xxxiv: application portfolio management," communications of the association for information systems (26) 2010a, p 87. mckeen, j. d., and smith, h. "developments in practice xxxvii: total cost of ownership," communications of the association for information systems (27) 2010b, p 627. mckeen, j. d., smith, h., and jin, j. "developments in practice xxxii: successful strategies for it staffing," communications of the association for information systems (24:1) 2009, p 32. mckeen, j. d., and smith, h. a. "developments in practice xxiii: creating and evolving a technology roadmap," communications of ais (2006:18) 2006, pp. 451-463. mckeen, j. d., smith, h. a., and singh, s. "developments in practice xx: digital dashboards: keep your eyes on the road," communications of the association for information systems (16) 2005, p 1. miles, m. b., and huberman, a. m. qualitative data analysis: an expanded sourcebook sage publications, thousand oaks, ca, 1994. montoya-weiss, m. m., massey, a. p., and qapper, d. l. "on-line focus groups: conceptual issues and a research tool," european journal of marketing (32:7/8) 1998, p 713. morgan, d. l. focus groups as qualitative research, (2 ed.) sage thousand oaks, ca, 1997, p. 80. morgan, d. l., and spanish, m. t. "focus groups: a new tool for qualitative research," qualitative sociology (7:3) 1984, pp. 253-270. morgeson, f. p., and hofmann, d. a. "the structure and function of collective constructs: implications for multilevel research and theory development," academy of management review (24:2), apr 1999, pp. 249-265. myers, m. d. "qualitative research in information systems," in: misq discovery, 1997. nahar, n., lyytinen, k., huda, n., and muravyov, s. v. "success factors for information technology supported international technology transfer: finding expert consensus," information & management (43:5) 2006, pp. 663-677. o'heocha, c., conboy, k., and xiaofeng, w. "using focus groups in studies of isd team behaviour," electronic journal of business research methods (8:2) 2010, pp. 119-131. orlikowski, w. j. "using technology and constituting structures: a practice lens for studying technology in organizations," organization science (11), july-august 2000, pp. 404-428. orlikowski, w. j., and baroudi, j. j. "studying information technology in organizations: research approaches and assumptions," information systems research (2) 1991, pp. 1-28. otondo, r. f., pearson, a. w., pearson, r. a., shaw, j. c., and shim, j. p. "managerial problemsolving in the adoption of radio frequency identification technologies," european journal of information systems (18) 2009, pp. 553-569. payton, f. c., and zahay, d. "understanding why marketing does not use the corporate data warehouse for crm applications," journal of database marketing (10:4) 2003, pp. 315326. australasian journal of information systems volume 17 number 2 2012 133 peffers, k., and tuunanen, t. "planning for is applications: a practical, information theoretical method and case study in mobile financial services," information & management (42:3) 2005, pp. 483-501. pettigrew, k. e., durrance, j. c., and unruh, k. t. "facilitating community information seeking using the internet: findings from three public library-community network systems," journal of the american society for information science and technology (53:11) 2002, pp. 894-903. price, r., and shanks, g. "a semiotic information quality framework: development and comparative analysis," journal of information technology (20:2) 2005, pp. 88-102. raitoharju, r., heiro, e., kini, r., and d'cruz, m. "challenges of multicultural data collection and analysis: experiences from the health information system research," electronic journal of business research methods (7:1) 2009, pp. 75-82. ramesh, b., and jarke, m. "toward reference models for requirements traceability," ieee transactions on software engineering (27:1) 2001, pp. 58-93. reid, m., allen, m., armstrong, d., and riemenschneider, c. "perspectives on challenges facing women in is: the cognitive gender gap," european journal of information systems (19:5) 2010, p 526. resatsch, f., sandner, u., leimeister, j., and krcmar, h. "do point of sale rfid-based information services make a difference? analyzing consumer perceptions for designing smart product information services in retail business," electronic markets (18:3) 2008, p 216. rosemann, m., and vessey, i. "toward improving the relevance of information systems research to practice: the role of applicability checks," mis quarterly (32:1) 2008, pp. 1-22. scott, c. k., fuller, m. a., macindoe, k. m., and joshi, k. d. "more than a bumper sticker: the factors influencing information systems career choices," communications of ais (2009:24) 2009, pp. 7-26. seddon, p. b., and scheepers, r. "towards the improved treatment of generalization of knowledge claims in is research: drawing general conclusions from samples," european journal of information systems ) 2011, pp. 1-16. shuk ying, h., pan, g., and ferguson, c. "the information systems-accounting nexus: lessons from an australian institution," communications of ais (2008:22) 2008, pp. 197-210. smith, h., and mckeen, j. "developments in practice xxxv: building a strong relationship with the business," communications of the association for information systems (26) 2010a, p 91. smith, h., and mckeen, j. "developments in practice xxxvi: how to talk so business will listen ... and listen so business will talk," communications of the association for information systems (27:-) 2010b, p 207. smith, h., and mckeen, j. "enabling collaboration with it," communications of the association for information systems (28) 2011, p 1. smith, h. a., and mckeen, j. d. "developments in practice ix: the evolution of the km function," communications of ais (2003:12) 2003, pp. 69-79. smith, h. a., and mckeen, j. d. "developments in practice xiv: marketing km to the organization," communications of the association for information systems (14) 2004, p 1. smith, h. a., and mckeen, j. d. "developments in practice xxiv: information management: the nexus of business and it," communications of ais (2007:19) 2007, pp. 34-46. smith, h. a., mckeen, j. d., and singh, s. "developments in practice xxviii: managing perceptions of is," communications of ais (2007:20) 2007, pp. 760-773. smith, h. j., milberg, s. j., and burke, s. j. "information privacy: measuring individuals' concerns about organizational practices," mis quarterly (20:2) 1996, p 167. smith, s., winchester, d., bunker, d., and jamieson, r. "circuits of power: a study of mandated compliance to an information systems security de jure standard in a government organization," mis quarterly (34:3) 2010, p 463. australasian journal of information systems volume 17 number 2 2012 134 sobreperez, p. "using plenary focus groups in information systems research: more than a collection of interviews," electronic journal of business research methods (6:2) 2008, pp. 181-188. srivastava, s., and teo, t. "citizen trust development for e-government adoption and usage: insights from young adults in singapore," communications of the association for information systems (25:1) 2009, p 20. stahl, b., tremblay, m., and lerouge, c. "focus groups and critical social is research: how the choice of method can promote emancipation of respondents and researchers," european journal of information systems (20:4) 2011, p 378. straub, d. w. "why journals accept your paper," mis quarterly (33:3) 2009, pp. iii-x. strauss, a., and corbin, j. basics of qualitative research: grounded theory procedure and techniques, thousand oaks, california, 1990. strauss, a., and corbin, j. basics of qualitative research, techniques and procedures for developing grounded theory, (2nd ed.), thousand oaks, california, 1998. sutton, s. g., khazanchi, d., hampton, c., and arnold, v. "risk analysis in extended enterprise environments: identification of critical risk factors in b2b e-commerce relationships," journal of the association for information systems (9:4) 2008, pp. 151-174. sykes, w. "taking stock: issues from the literature on validity and reliability in qualitative research," journal of the market research society (33:1) 1991, pp. 3-12. tams, s., and grover, v. "the effect of an is article’s structure on its impact," communications of the ais (27) 2010, p aricle 10. tillotson, j., cherry, j., and clinton, m. "internet use through the university of toronto library: demographics, destinations, and users' reactions," information technology and libraries (14:3) 1995, pp. 190-198. torkzadeh, g., chang, j. c.-j., and hansen, g. w. "identifying issues in customer relationship management at merck-medco," decision support systems (42:2) 2006, p 1116. tremblay, m., hevner, a., and berndt, d. "focus groups for artifact refinement and evaluation in design research," communications of the association for information systems (26) 2010, p 83. van velsen, l., huijs, c., and van der geest, t. "eliciting user input for requirements on personalization: the case of a dutch erp system," international journal of enterprise information systems (4:4) 2008, p 34. venkatesh, v., morris, m., davis, g., and davis, f. "user acceptance of information technology: toward a unified view," mis quarterly (27:3) 2003, pp. 425-478. wang, p. "chasing the hottest it: effects of information technology fashion on organizations," mis quarterly (34:1) 2010, pp. 63-85. wang, y., lo, h.-p., and yang, y. "an integrated framework for service quality, customer value, satisfaction: evidence from china's telecommunication industry," information systems frontiers (6:4) 2004, p 325. weidong, and lee "complexity of information systems development projects: conceptualization and measurement development," journal of management information systems (22:1), summer2005 2005, pp. 45-83. xia, w., and lee, g. "grasping the complexity of is development projects," association for computing machinery. communications of the acm (47:5) 2004, p 68. xu, m., ong, v., duan, y., and mathews, b. "intelligent agent systems for executive information scanning, filtering and interpretation: perceptions and challenges," information processing & management (47:2) 2011, p 186. yin, r. k. case study research: design and methods sage publications, thousand oaks, ca, 1994. australasian journal of information systems volume 17 number 2 2012 135 appendix a. basic guidelines for the focus group research method the table below briefly summarizes guidelines for focus group research in information systems adapted from (fern 2001; krueger 1994; morgan 1997), which are focused on selection of participants, how to run the sessions, the role of the moderator, and the physical setting. substantial research has been conducted in the field of marketing analyzing the effects of variations in these guidelines (e.g. smaller or large group size, personality of the moderator, etc.). for example, it has been found that eight-member groups generate more ideas than four-member groups (fern 1982). category guideline description participants homogeneous strangers to achieve high levels of interactions and participation, subjects must be relatively “compatible” with each other. job hierarchy, age, knowledge base, or other visible differences can hinder open discussions and relatively equal participation necessary to reap the benefits of focus groups. anonymity (no demographics collected except observed) can increase participation. knowledgeable about topic (“experts”) since the purpose of the focus groups is to gather insights from participants, it is important that they are able to offer their views in the discussion. it is important that the expertise of the participants is relevant to the research question of interest. this is needed for validity. three to 10 participants per group while some suggest that expert moderators can handle groups up to 15 participants, the most common number of participants in research is five to seven per group. smaller groups can present challenges since the interactions are limited, interactions being an integral part of the focus group process. number of sessions is function of saturation when no further insight is gained from additional data collection, saturation is reached. moderator high involvement and discussion leadership the moderator must be highly involved in the discussion, asking questions but also probing for further insights when appropriate. ensures everyone participates the moderator must try to facilitate productive discussions among group members. it can help to place potential “dominators” close to the moderator. physical setting round or rectangle tables participants should be able to see each other and have a sense that they are all on an equal footing in the discussion. familiar and/or relaxed setting a relaxed setting is needed for participants to become fully engaged in the conversation without being distracted by their environment. ground rules one person speaking at a time and no side conversations the moderator must ensure that these rules are known in advance and well respected. this ensures that all discussions are captured and that all potential findings are explored. session and protocol topic introduced in general terms the moderator must set the stage on the topic to get everyone in the right mindset. icebreaker or easy question first the moderator must ensure that everyone becomes quickly comfortable participating, and an icebreaker question allows him or her to bring in less vocal participants right away. start with broad and open ended questions; then relatively structured when probing further open-ended questions help gain a deeper understanding of respondents’ attitudes and opinions and avoid biasing participants toward specific issues researchers deem more important. structured protocol: ask the same questions, in a similar order, across groups protocol must be tested and modified several times after initial development. asking the same questions allows comparisons across groups as a validation of findings, although new questions can emerge after the first few groups. session is recorded. a scribe can take notes based on observations of the session. researchers should have backup devices! recorded sessions are transcribed into text files, which can then be used with qualitative analysis software such as atlas.ti or nvivo. video sessions offer the additional possibility of coding for non-verbal cues. one to two hours maximum fatigue can become an issue if longer sessions are held, for both the moderator and the participants. paper title (paper title style) australasian journal of information systems lim 2018, vol 22, preface revisiting concepts and theories in information systems and technology 1 revisiting concepts and theories in information systems and technology weng marc lim swinburne university of technology lim@wengmarc.com / marclim@swin.edu.au / wlim@swinburne.edu.my information system is both a product and a discipline. as a product, information system consists of a set of interrelated components—for example, hardware, software, people, process, and data—that work together to provide value to firms, such as useful information for informed decision-making. as a discipline, information system is the study of entities and events characterizing its product form. in revisiting the disciplinary perspective, it is important to note that most concepts and theories in information systems and technology (e.g. information systems success model, technology acceptance model) are not only interrelated but also conceived when mainstream society received initial exposure to digital technologies (e.g. computers, the internet) in applied settings (e.g. e-procurement, e-commerce). given the rapid proliferation in the technology advancement and usage of, as well as access and exposure to, information systems over time, a research notes special section in the australasian journal of information systems was commissioned to encourage scholars in the field to revisit existing concepts and theories in information systems and technology. specifically, research notes in the special section offer fresh insights on how the conceptual boundaries of and methodological approaches to utilize these concepts and theories can be expanded and/or reconfigured so as to maintain currency and relevance in the pursuit of advancing theory and practice in the area. all accepted research notes in the special section, which had undergone a series of doubleblind peer reviews, introduce an existing concept or theory in information systems and technology, explain its limitations with respect to understanding recent trends in information systems and technology, and offer well-articulated solutions to address or mitigate those limitations so as to facilitate greater theoretical and practical development and understanding of information systems and technology in the contemporary era. the article by gupta et al. (2018) reveals that there are two resource-based views (rbvs) of information systems: conventional and contemporary. the conventional rbv of information systems suggests that firms use information systems as a firm resource to establish sustainable competitive advantages and drive firm performance, particularly in stable business environments. meanwhile, the contemporary rbv of information systems takes into account the peculiarities of agile and dynamic business environments, thus encouraging firms to use information systems as a firm resource to establish transient competitive advantages and drive firm performance. the rejoinder by lim (2018) addresses some of the most pertinent issues confronting the technology acceptance model (tam) by reviewing the model’s critiques and responding to those critiques with dialectic antidotes in the form of conceptual, methodological, and replication treatments. these treatments, which are useful and relevant for behavioural modelling, support the continued use of the tam to understand the peculiarities of user interactions with technology in technology-mediated environments. australasian journal of information systems lim 2018, vol 22, preface revisiting concepts and theories in information systems and technology 2 the manuscript by namvar et al. (2018) introduces sensemaking as an approach to deal with complicated, complex, and chaotic data. the authors acknowledge and consolidate the essence of diverse sensemaking perspectives in the extant literature into a simplified blueprint. this blueprint is positioned as a reference point that can be used by scholars to gain an understanding of the concept, process, strengths, and shortcomings of sensemaking, as well as the relevant ways forward for its use with regard to information systems in contemporary business environments. the research note by samhan (2018) spells out extant conceptualization, contextualization, antecedents, interventions, and consequences of current and future work on technology resistance. the author makes the case that technology resistance can assume multiple forms, and thus necessitates greater research using multiple theoretical lenses and perspectives. these contentions and underpinnings are explained in the main text and depicted in a succinct and sharp illustration by the author. the paper by taxén (2018) examines sociomateriality, a niche but important concept in the field of information systems. the author elucidates the limitations of the predominant strand of sociomateriality—that is, strong sociomateriality—and proposes a reconfiguration of sociomateriality from a neurobiological perspective, which encapsulates assumptions pertaining to the dialectical ontology, the substantialist aspect (or the relata), and the processual aspect (or the dialectics between relata). the author argues that the extreme relational ontology of strong sociomateriality is an untenable foundation for information systems research and that the product of a neurobiological reconfiguration, including alternative variants such as weak sociomateriality and imbrication, should be considered, where individuals should be included as prime constituents, alongside the social and the material, in information systems. special thanks to everyone who have contributed to the publication of the special issue, namely: 1. the editor-in-chief for accepting the proposal for the special section as well as his ongoing support toward its publication, including leading a separate independent review process for the section editor’s submission: john lamp, deakin university. 2. the authors who have contributed intellectual insights and endured the lengthy but rigorous review process for the special section: jacob cybulski, deakin university; yaw seng ee, swinburne university of technology; gaurav gupta, indian institute of management; morteza namvar, university of wollongong; cynthia su chen phang, swinburne university of technology; bahae samhan, illinois state university; kevin tee liang tan, swinburne university of technology; and lars taxén, linköping university. 3. the reviewers who have tirelessly contributed to reviewing and re-reviewing the submissions for this special section: nabeel al-qirim, united arab emirates university; rui bi, charles sturt university; chin chen, national university of taipei; ranko cosic, university of melbourne; helen hasan, university of wollongong; yong jin park, howard university; kai riemer, university of sydney; stephen smith, macquarie university; siri wassrin, linköping university; yancong xie, queensland university of technology; and orkun yildiz, gazi üniversitesi. australasian journal of information systems lim 2018, vol 22, preface revisiting concepts and theories in information systems and technology 3 lastly, it is hoped that the insights from the special section will accelerate the advancement and impact of information systems and technology theory and practice. further extensions and post-publication reviews of papers in this special section are encouraged and welcomed on the basis of academic collegiality and freedom for knowledge advancement. weng marc lim head of school, school of business, swinburne university of technology section editor, australasian journal of information systems ranked “a” by the australian business deans’ council and the australian council of professors and heads of information systems references gupta, g., tan, k. t. l., ee, y. s., & phang, c. s. c. (2018). resource-based view of information systems: sustainable and transient competitive advantage perspectives. australasian journal of information systems, 22. lim, w. m. (2018). dialectic antidotes to critics of the technology acceptance model: conceptual, methodological, and replication treatments for behavioral modeling in technology-mediated environments. australasian journal of information systems, 22. namvar, m., cybulski, j. l., phang, c. s. c., ee, y. s., & tan, k. t. l. (2018). simplifying sensemaking: concept, process, strengths, shortcomings, and ways forward for information systems in contemporary business environments. australasian journal of information systems, 22. samhan, b. (2018). revisiting technology resistance: current insights and future directions. australasian journal of information systems, 22. taxén, l. (2018). reconfiguring sociomateriality from a neurobiological perspective. australasian journal of information systems, 22. copyright: © 2018 lim. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems kanaparan, cullen & mason 2019, vol 23, selected papers from acis 2017 self efficacy & engagement on students 1 effect of self-efficacy and emotional engagement on introductory programming students geetha kanaparan xiamen university malaysia malaysia geetha.kanaparan@xmu.edu.my rowena cullen victoria university of wellington new zealand david mason victoria university of wellington new zealand abstract high failure rates appear to be a norm in introductory programming courses. many solutions have been proposed to improve the high failure rates. surprisingly, these solutions have not led to significant improvements in the performance of students in introductory programming courses. in this study, the relationship between self-efficacy, emotional engagement and the performance of students in introductory programming courses were examined. enjoyment, interest, and gratification were identified as three factors contributing to emotional engagement in introductory programming courses from a review of existing literature and from focus groups. an online survey of 433 students in introductory programming courses showed that the students’ programming self-efficacy beliefs had a strong positive effect on enjoyment, while gratification and interest had a negative effect on programming performance. these findings have implications for course instructors who design and deliver introductory programming courses. keywords: self-efficacy; emotional engagement; introductory programming courses; programming performance 1 introduction software developer jobs are in demand due to the rapid advances in technology that has enabled automation in many businesses today. a recent report by the australian government showed that as of november 2017, software and application programmers were amongst the top three most in demand occupations within the professional, scientific and technical services industry which itself was one of top five growing industries in australia (australian government department of jobs and small business, 2018). in yet another report, the u.s. department of labour reported that employment for software developers is expected to increase by 24% from 2016 to 2022 (bureau of labour statistics, 2018). higher educational institutions (heis) play an important role in attracting, retaining, and producing graduates who are qualified in information and communications technology (ict) related disciplines in order to meet the rising demand for software developers. however, failure rates of between 30% and 50% (bennedsen & caspersen, 2007; quille & bergin, 2016; australasian journal of information systems kanaparan, cullen & mason 2019, vol 23, selected papers from acis 2017 self efficacy & engagement on students 2 watson & li, 2014) in introductory programming courses poses a problem to heis in meeting the demands of a skilled workforce for the software industry. introduction to programming is a core course in the first year of an undergraduate ict related discipline, introducing students to the fundamental concepts of computer programming and equips them with the skills to code solutions to programming problems. success in the introductory programming course is crucial so that students are able to understand and write programming code, and it is a mandatory course for students who wish to obtain a qualification in an ict related discipline and pursue a career as a software developer. kanaparan, g et al (2013) argued that high failure rates may be due to the demanding cognitive load of programming languages and the behaviour of the students. although a large number of solutions have focused on reducing the demanding cognitive load of programming languages, there has been little to no success in reducing the high failure and attrition rates. by contrast, the behaviour of the students has not been examined extensively although the literature on learning theory suggests that there is a strong link between the behaviour of the student and their performance (bartimote-aufflick, bridgeman, walker, sharma, & smith, 2016; schunk & mullen 2012). despite a dearth of research on the behaviour of students in introductory programming courses, the study by wiedenbeck, xiaoning, and chintakovid (2007) provided motivation for this research. the authors found that a student’s self-efficacy in computer programming affects their computer interest, and that their computer interest then affects their programming performance. computer interest was conceptualised as “end users who have interest in computers, programming, and related topics, even if they have low experience” (wiedenbeck et al., 2007, p. 70). interest is one of several indicators of engagement and is discussed in the next section. despite the encouraging findings, no further research could be found in the literature on computer programming that examines the link between self-efficacy, engagement, and performance. by contrast, the literature on learning theory suggests that self-efficacy is one contextual factor that affects engagement, and engagement then affects the performance of the student (appleton, christenson, & reschly, 2006; linnenbrink & pintrich, 2003; schunk & mullen, 2012). although self-efficacy is necessary for success in learning, the student may not necessarily be engaged in learning (appleton et al., 2006). instead, students who engage in learning are self-efficacious learners, strive to achieve their goals, and are ultimately successful (schunk & mullen, 2012). recognising the importance of self-efficacy, several researchers have produced frameworks to explain the relationship between self-efficacy, student engagement and learning, a discussion of which is provided in the next section. due to the compelling evidence presented in the literature on learning theory, this research examines two behavioural factors: self-efficacy and emotional engagement, and the effect of these two behavioural factors on the performance of students in introductory programming courses. this research asks the following questions: research question 1: what is the effect of programming self-efficacy on the emotional engagement of students in introductory programming courses? australasian journal of information systems kanaparan, cullen & mason 2019, vol 23, selected papers from acis 2017 self efficacy & engagement on students 3 research question 2: what is the effect of emotional engagement on the programming performance of students in introductory programming courses? 2 theoretical framework and research hypotheses 2.1 programming self-efficacy and social cognitive theory social cognitive theory (sct) is a widely accepted and empirically validated framework that explains human behaviour. the sct framework is based on the premise that human beings have the ability to control their thoughts, feelings, motivation, and actions (bandura, 1986). bandura refers to this ability as a self-system which allows human behaviour to be understood, predicted, and altered within a social context (bandura, 1977a; bandura, 1986). sct explains human behaviour based on three interacting determinants which form a triadic reciprocality (figure 1). the three interacting determinants are: personal factors, behaviour, and the environment. the personal factors determinant is shaped by expectations, beliefs, goals, selfperceptions and intentions; while the environmental determinant refers to factors such as social norms, access in the community and influence on others, and the behavioural determinant refers to skills such as practise and self-efficacy (bandura, 1986; pajares, 1997). figure 1: three-way relationship in triadic reciprocality (bandura, 1986, p. 24) as an example, a programming student who encounters an error while working on a programming project might seek help from peers or a teacher (personal factors influences behaviour). the teacher or peer proceeds to explain or debug the error with the student (behaviour influences environment). the student then reviews the error and understands how the error was resolved (environment influences personal factors). this suggests that the interaction between personal factors, behaviour, and the environment influences an individual’s self-belief, which then affects the environment and subsequently affects the individual’s performance (pajares, 1997). the theory of self-efficacy forms part of bandura’s sct and was proposed to predict and explain behavioural change in individuals. self-efficacy is defined as “people’s judgments of their capabilities to organize and execute courses of action required to attain designated types of performances” (bandura, 1986, p. 391). the definition of self-efficacy suggests that the individual’s perception of their ability may influence how well they carry out a task or activity, and that their ability then determines the degree of success in completing the task or activity. high self-efficacy in a task implies that the individual believes that they have the ability to accomplish that task successfully (walker, greene, & mansell, 2006) and that self-efficacy influences an individual’s choice and action (schunk, 1996). further, an individual with high behaviour personal factors environment australasian journal of information systems kanaparan, cullen & mason 2019, vol 23, selected papers from acis 2017 self efficacy & engagement on students 4 self-efficacy beliefs is likely to engage in tasks or activities that they feel they are competent in and are confident in accomplishing. conversely, an individual with low self-efficacy beliefs avoids tasks that they feel they are not able to do, perceiving them as difficult to accomplish (schunk, 1996). students with high self-efficacy demonstrate a positive attitude towards learning by seeking challenges, become intrinsically interested, have set goals, persist in the face of challenge, adopt effective strategies to resolve challenges, and are able to easily recover from failures or setbacks (bandura, 1997; schunk, 1996). existing research on learning theory largely examined the effect of self-efficacy on student motivation, learning and performance, and found that self-efficacy beliefs affects performance, perseverance to learn, whilst also having the ability to alter behaviour (phan, 2011; yip, 2012; zimmerman, 2000). by contrast, research on the effect of self-efficacy on programming students appears to be limited. one such research identified the factors that influence the selfefficacy beliefs of students in introductory programming courses (zingaro, 2014) while two other researchers reported positive outcomes when examining the relationship between the student’s self-efficacy beliefs and their programming performance (kinnunen & simon, 2011; ramalingam, labelle, & wiedenbeck, 2004). self-efficacy is proposed as an independent variable in this research due to the strong evidence presented in the literature on learning theory about the positive influences of self-efficacy on learning and performance. this study proposes to use the term programming self-efficacy in reference specifically to the student’s judgment of their ability to learn programming. 2.2 emotional engagement and student engagement student engagement has emerged as a theoretical model for understanding which students are likely to fail, dropout from school, and for improving student motivation and achievement (appleton, christenson, & furlong, 2008; christenson, reschly, & wylie, 2012; fredricks, blumenfeld, & paris, 2004). engagement is a multi-dimensional construct and is defined as “the student’s active participation in academic and co-curricular or school-related activities, and commitment to educational goals and learning.... student engagement drives learning; requires energy and effort; is affected by multiple contextual influences; and can be achieved for all learners” (christenson et al., 2012, pp. 816-817). over the years, several student engagement models and frameworks have been proposed. there is an on-going debate for a commonly accepted model for student engagement (christenson et al., 2012) due to the large body of research on the dimensions of student engagement (fredricks et al., 2004) and the lack of agreement between researchers on the three schools of thought that make up the literature on student engagement (reschly & christenson, 2012). one school of thought arose as a result of dropout prevention theory and intervention, the second from general school reform, and the third school of thought arose from motivational literature. within these schools of thought, sub-disciplines were formed with interests in various aspects of engagement. for example, educational psychologists are interested in examining engagement within an educational context while development psychologists are interested in engagement from the perspective of motivational theory (reschly & christenson, 2012). despite the lack of a commonly accepted engagement model, the more frequently discussed student engagement models include linnenbrink and pintrich’s framework (linnenbrink & pintrich, 2003), appleton et al.’s student engagement model (appleton et al., 2006), and australasian journal of information systems kanaparan, cullen & mason 2019, vol 23, selected papers from acis 2017 self efficacy & engagement on students 5 reschly and christenson’s student engagement model (reschly & christenson, 2012). these models and framework vary in the dimensions of engagement, indicators used to measure student engagement, the range of contextual factors that affect student engagement, and the outcomes of student engagement. however, there is general agreement among researchers that there are three dimensions of engagement. two of the engagement dimensions are behavioural engagement and cognitive engagement (appleton et al., 2006; fredricks et al., 2004; linnenbrink & pintrich, 2003; reschly & christenson, 2012), and the third dimension proposed was motivational engagement (linnenbrink & pintrich, 2003), emotional engagement (fredricks et al., 2004), psychological engagement (appleton et al., 2006), or affective engagement (betts, appleton, reschly, christenson, & huebner, 2010). despite a lack of consensus on the third engagement dimension, emotional engagement appears to have been more commonly discussed, examined, and clearly conceptualised in the literature on learning theory. emotional engagement is a dependent variable in this research since wiedenbeck et al. (2007) argued that computer interest (an indicator of emotional engagement) is an important engagement factor when learning programming. emotional engagement refers to the student’s feeling, attitude, perception towards learning, and the learning environment (sheard, carbone, & hurst, 2010; yazzie-mintz & mccormick, 2012). in this study, enjoyment and interest were identified as indicators of emotional engagement after an extensive review of the literature on learning theory (appleton et al., 2006; linnenbrink & pintrich, 2003; reschly & christenson, 2012) and from the literature on computer programming. enjoyment and interest were then validated as proposed indicators of emotional engagement in introductory programming courses through focus groups, while gratification emerged as a third indicator of emotional engagement during the focus groups. other factors that were identified and considered as potential indicators of emotional engagement include: affect, value, belonging, and identification with school membership/connectedness. affect did not appear to be a suitable indicator because the conceptualisation of affect had appeared to collectively measure several indicators of emotional engagement that had been identified in this study. similarly, value, belonging, and identification with school membership/connectedness did not appear to be suitable indicators in this study because these indicators were better measures of the student’s perception of the educational institution that they were attending instead of their performance in their introductory programming course. 2.2.1 focus groups – validating indicators of emotional engagement to validate the proposed indicators of emotional engagement in introductory programming courses, four (4) focus groups were conducted in new zealand and in malaysia. in total, 16 students who were mid-way through their introductory programming course participated in the focus groups. there were between 3 and 5 participants in each focus group. the duration for each focus group was between 45 and 60 minutes. the discussion in the third and fourth focus groups did not reveal any new findings that were not already identified in the first two focus groups. as such, it was felt that no further focus groups were required. the questions were reviewed by three academics who had more than 15 years of teaching experience at the tertiary level. the students were asked questions that encouraged them to australasian journal of information systems kanaparan, cullen & mason 2019, vol 23, selected papers from acis 2017 self efficacy & engagement on students 6 discuss how they felt when they were learning programming and when they were working on their programming assessments. the focus groups were transcribed and analysed. the analysis of the focus groups conforms to the key concepts analysis framework that identifies the key concepts or themes in the research by asking the participants to identify the core themes, as well as the important ideas and experiences that relate to these themes (krueger & casey, 2009). the data from the focus groups were analysed for evidence of the proposed emotional engagement indicators by examining the engagement behaviour of the students. next, an inductive and deductive approach was adopted to analyse the data (thomas, 2006). the deductive approach was used to examine the data for evidence of the indicators of emotional engagement, while the inductive approach was used to code the data, and to identify new emotional engagement indicators. the findings from the analysis of the focus groups confirmed that enjoyment and interest were indicators of emotional engagement in introductory programming courses, and interestingly, gratification emerged as a third indicator of emotional engagement in introductory programming courses. 2.2.2 enjoyment enjoyment “causes the subject to experience pleasure by causing occurrent beliefs which satisfy desires concerning the experience itself” (davis, 1982, p. 240). the following are quotes from the participants in the focus groups who appeared to enjoy learning programming: "because i really enjoy programming. maybe i go so far as to say i have fun while doing it." (cl, fg2) “…thoroughly enjoy programming... thoroughly hate doing essays” (cm, fg1) "i actually enjoy doing this subject [referring to programming course] and i want to like start on it [new programming assessment]" (jr, fg3) in the literature on learning theory, positive correlations were observed between self-efficacy and enjoyment (mills, pajares, & herron, 2007), and that high self-efficacy results in pleasant emotions such as enjoyment (putwain, sander, & larkin, 2013). therefore, the following hypothesis is proposed: h1: self-efficacy beliefs in learning programming will have a positive effect on enjoyment. the literature on computer programming suggests that enjoyment in learning programming increases as a result of using the alice programming environment (bishop-clark, courte, evans, & howard, 2007) and when engaging in pair programming (liebenberg, mentz, & breed, 2012; maguire, maguire, hyland, & marshall, 2014). however, the effect of enjoyment on the performance of the student was not examined in these studies. by contrast, frenzel, thrash, pekrun, & goetz (2007) found positive correlations between enjoyment and the academic performance of students. therefore, the following hypothesis is proposed: h2: enjoyment in learning programming will have a positive effect on programming performance. 2.2.3 interest interest is defined as “an emotion that arouses attention to, curiosity about, and concern with…” a discipline of study (akbulut & looney, 2007, p. 68). the following are quotes from the participants in the focus groups who appeared to have an interest in learning programming: australasian journal of information systems kanaparan, cullen & mason 2019, vol 23, selected papers from acis 2017 self efficacy & engagement on students 7 "i am interested because the code itself makes me very curious about what the outcome would be" (fa, fg4) “…programming is going to be my life…that’s what i want to be when i’m finished...” (mh, fg1) "i am one of those (who play games).…. super mario (game) doing certain things... start wondering what logic lies behind it... what codes... and who thought about it first? so, you start wondering, let’s poke something here and see what comes out." (mje, fg4) silvia (2003) suggested that self-efficacy affects interest indirectly, whereby “self-efficacy affects uncertainty about how the activity will resolve, which in turn affects interest” (p.239). in another study, linnenbrink and pintrich (2003) argued that as the student develops expertise in a given task, his or her self-efficacy beliefs and interest increases. additionally, bandura (1997) suggests that a moderate level of self-efficacy is essential for sustaining interest in a given task, while akbulut and looney (2007), and wiedenbeck et al. (2007) found a positive relationship between programming self-efficacy and interest. therefore, the following hypothesis is proposed: h3: self-efficacy beliefs in learning programming will have a positive effect on interest. mckinney and denton (2004) and wiedenbeck et al. (2007) found that interest is significantly correlated to programming performance, while sheard et al. (2010) reported that students have a high interest in topics such as programming and computer networks. furthermore, in the literature on learning theory, bye, pushkar, & conway (2007) found that interest is a strong predictor of motivation to learn. therefore, the following hypothesis is proposed: h4: interest in learning programming will have a positive effect on programming performance. 2.2.4 gratification gratification emerged as a new indicator of emotional engagement during the focus groups. in the literature on learning theory, research on gratification discusses “delay of gratification” and its effect on academic success. delay of gratification is the “voluntary postponement of immediate rewards and persistence in goal-directed behaviour for the sake of later outcomes” (mischel, schoda, & rodriguez, 1989, p. 933). however, during the focus groups, the participants felt a feeling of immediate gratification when they were able to debug the errors in their program and were able to see the output of their program. the following are quotes from the participants in the focus groups who appeared to have a feeling of immediate gratification when learning programming: "..just kind of smile thinking you’ve done it. it’s finished." (di, fg2) "...especially if i figure it out for myself that’s a really rewarding feeling." (rs, fg2) “…very satisfying feeling when it works… it doesn’t, it’s very frustrating...” (cm, fg1) "not frustration but definitely excitement when i finish [my programming assessment]." (rb, fg3) "triumph [when i find a solution to my programming problem]" (cl, fg2) for example, participant di from focus group 2 explained that programming gave him an immediate sense of achievement compared to other courses. similarly, participant rs from focus group 2 felt rewarded each time she achieved a milestone in her programming australasian journal of information systems kanaparan, cullen & mason 2019, vol 23, selected papers from acis 2017 self efficacy & engagement on students 8 assessment but added that she did not feel the same way after she had completed the assessments for other courses. participant cl from focus group 2 echoed similar feelings of gratification whereby he felt triumphant when he was able to solve a programming problem. therefore, for the purpose of this study, gratification is defined as students who experience “the feeling of receiving immediate rewards typically in the form of pleasure or satisfaction as a result of hard work”. the interpretation of gratification in this study focuses on immediate gratification as opposed to delay of gratification which is a common outcome of learning in the literature on learning theory. the following hypotheses are proposed despite a lack of evidence from the existing research that suggests a relationship between programming selfefficacy and gratification, and a relationship between gratification and programming performance: h5: self-efficacy beliefs in learning programming will have a positive effect on gratification. h6: gratification in learning programming will have a positive effect on programming performance. 2.3 programming performance: the dependent variable the dependent variable in this research is programming performance. the term programming performance has been used widely in the literature on computer programming. although no specific definition could be found in the literature, an analysis of the attribute(s) used to measure programming performance suggests that programming performance refers to an objective measure of how well the student has performed in their introductory programming course. the programming performance of the students were measured by the final grade that the students received at the end of their introductory programming course. grades are assigned to each piece of assessment that the student submits in the introductory programming course and these grades cumulatively contribute to the final grade of the student’s programming performance. two concerns arose when deciding on the use of the final grade as a measure for programming performance. firstly, the final grade may not reflect the true programming ability of the student, and secondly, the assessment design in some introductory programming courses may include non-programming related tasks or activities. in order to mitigate these concerns and to validate the use of programming grade to measure programming performance, the course outline was obtained from the instructors of each introductory programming course. the course outline was then examined for the method of assessment and the learning outcomes of the course. it was found that the assessments ranged from programming assignments, class tests, to lab-based exercises which were clearly linked to the learning outcomes of the introductory programming course. this served to allay the aforementioned concerns thereby giving confidence that the assessments were designed to test the programming ability of the students. as such, there was strong evidence that the final grade was able to provide an objective measure of the student’s level of performance in the introductory programming course. additionally, evidence from existing research on introductory programming courses suggests that course grade is a strong indicator of success and is a criterion for progression to the next level of study (debry, 2011; ford & venema, 2010). 2.4 confounding variables confounding variables are variables that may be “mixed up with the independent variable, making it impossible to determine which of the variables has produced changes in the dependent variable” (stangor, 2011, p. 231). prior programming experience and learning australasian journal of information systems kanaparan, cullen & mason 2019, vol 23, selected papers from acis 2017 self efficacy & engagement on students 9 ability were two confounding variables identified from the literature on computer programming, that were likely to influence the programming performance of students. learning ability was measured by the high school/pre-university results of the students while prior programming experience was measured by the participant’s evaluation of their programming experience. the results showed that the confounding variables did not significantly affect the performance of the students. 2.5 research model figure 2 shows the research model for self-efficacy, (indicators of) emotional engagement and programming performance, and their hypotheses. figure 2. research model for self-efficacy, emotional engagement, and programming performance the hypotheses on the left side of the research model (h1, h3 and h5) seeks to address research question 1, and the hypotheses on the right side of the research model (h2, h4 and h6) seeks to address research question 2. 3 methodology the data for programming self-efficacy, emotional engagement, prior programming experience (confounding variable) and the demographic information of the participants was collected using a cross-sectional survey. self-reporting questionnaires were administered to students in the final week of their introductory programming course. the programming grade and high school/pre-university results (confounding variable) of the survey participants were retrieved from their institution’s student records after obtaining approval from the participants. this was necessary to ensure the integrity of the data. the signed participant consent form was sent to the manager responsible for student records as evidence of consent to access the data. 3.1 survey instrument the items to measure programming self-efficacy was adapted from existing empirical research on programming self-efficacy (ramalingam & wiedenbeck, 1998), academic self-efficacy (bresó et al., 2011; sherer at al., 1982) and from bandura’s (1986) three dimensions of selfefficacy: magnitude, strength, and generality. these sources of item development conform to the recommendations made by mackenzie et al. (2011) for the development of measurement items. the reliability of the items to measure programming self-efficacy was initially reviewed by five introductory programming course instructors and were re-worded to make them australasian journal of information systems kanaparan, cullen & mason 2019, vol 23, selected papers from acis 2017 self efficacy & engagement on students 10 relevant to the context of programming and for this research. a 5-point likert scale was used, ranging from 1 (not confident at all) to 5 (very confident). the items to measure interest and gratification were developed from the findings of the focus groups and from the definition of the construct. at the time of the development of the instrument, no existing literature could be found with access to the items that measure interest and gratification. by contrast, the items to measure enjoyment were adapted from existing research that was conducted by bishop-clark et al. (2007) and from the focus groups. here again, a 5-point likert scale was used to measure emotional engagement, ranging from 1 (strongly disagree) to 5 (strongly agree). the survey instrument was further subjected to rigorous reliability and validity tests. a card sorting activity was performed to assess construct validity and to identify items that lacked clarity (moore & benbasat, 1991). a pilot study was then conducted, and finally an exploratory factor analysis (efa) was performed to assess the convergent and discriminant validity of the items in the survey. the survey items are listed in appendix a. 3.2 data collection the data was collected in two countries, malaysia and new zealand, which were able to offer representativeness of the sample and an international perspective to this study. a total of 10 tertiary level educational institutions participated in the survey, 6 of which were from new zealand, and 4 were from malaysia. the questionnaire was designed and administered online. table 1 shows the response rate by country. country sample population no. of participants no. of usable responses new zealand 750 (68.6%) 225 (47.9%) 214 (49.4%) malaysia 343 (31.4%) 245 (52.1%) 219 (50.6%) total 1093 470 433 table 1. response rate by country a total of 1093 students were invited to participate in the survey. the response rate was high: 43% (470) of the students from the sample population participated in the survey, of which 92.1% (433) of the responses were usable. out of the 433 usable responses, 72.7% were male participants and 33% (144) of the participants had prior programming experience. however, only 1.7% (14) of the participants had scored themselves as having good to extensive programming experience. the programming grades and high school/pre-university results were obtained between 6 and 10 weeks after the students completed their final programming assessment. table 2 shows the distribution of the final grades obtained by the participants in their introductory programming course. australasian journal of information systems kanaparan, cullen & mason 2019, vol 23, selected papers from acis 2017 self efficacy & engagement on students 11 grade malaysia new zealand no. of students (%) (n = 433) a (80% 100%) 45 117 162 (37.4%) b (65% 79%) 76 59 135 (31.2%) c (50% 64%) 69 29 98 (22.6%) d (40% 49%) 19 5 24 (5.6%) e (< 40%) 10 4 14 (3.2%) table 2. distribution of programming grade by country clearly, more than half of the participants (68.6%) had achieved a grade b or better in their introductory programming course, while only 8.8% of the participants had obtained a grade d or poorer. interestingly, a significantly higher percentage of the participants in new zealand had obtained a grade a (72.2%) compared to malaysia (27.8%). 4 results the data was analysed using the variance-based pls-sem smartpls 3 software (ringle, wende, & becker, 2015) since the objective of this research is to predict and explain the constructs, and pls-sem is able to handle single-item constructs (hair, hult, ringle, & sarstedt, 2014) such as programming performance and learning ability. the analysis of the data was performed in two stages. the first stage involved the analysis of the measurement model, and the second stage involved the analysis of the structural model (hair et al., 2014). the measurement model describes the relationships between the constructs and their corresponding indicators (hair et al., 2014) by establishing the reliability and validity of the constructs. the path connecting the items to the constructs were measured reflectively. the composite reliability (cr), indicator reliability (item loadings), and average variance extracted (ave) were established for the measurement model (hair et al., 2014). table 3 presents the convergent validity of the constructs in the research model. a cr value of 0.7, ave value of 0.5, and item loadings that are greater than 0.6 are acceptable to satisfy the convergent validity of the constructs (hair et al., 2014). construct no. of items item loadings ave cr programming selfefficacy 22 items .622 – .770 .511 .958 enjoyment 3 items .680 .866 .625 .832 interest 3 items .684 .890 .675 .860 gratification 4 items .618 .875 .590 .850 programming performance 1 item 1.000 table 3. convergent validity the discriminant validity of the measurement model was then examined. discriminant validity refers to how well a construct differs from the other constructs in the model through australasian journal of information systems kanaparan, cullen & mason 2019, vol 23, selected papers from acis 2017 self efficacy & engagement on students 12 statistical comparisons (hair et al., 2014). to satisfy the conditions for discriminant validity, firstly, the results of the fornell-larcker criterion and the cross-loadings of the items were examined. when evaluating the discriminant validity using the fornell-larcker criterion, the square root of a construct’s ave should be higher than any of the correlation with other constructs (fornell & larcker, 1981). table 4 presents the results for discriminant validity using the fornell-larcker criterion. construct programming self-efficacy enjoyment interest gratification programming performance programming self-efficacy .715 enjoyment .632 .790 interest .572 .753 .821 gratification .488 .542 .519 .768 programming performance .413 .314 .170 .142 1.000 table 4. discriminant validity the diagonal values that are highlighted in bold text represent the square root of the ave, while the off-diagonals represent the correlations. the results for the fornell-larcker criterion confirm that the square root of a construct’s ave is higher than the correlations of the other constructs. the second condition for discriminant validity requires that the items should load more strongly on their construct than on other constructs (hair et al., 2014). the cross-loadings of the items were examined and found to load strongly on their intended construct than on other constructs. thus, the conditions for discriminant validity have been met. the structural model tests the hypotheses in the research model and describes the relationships between the constructs (hair et al., 2014). figure 3 presents the structural model with the results of the path analysis. a bootstrapping procedure was used to determine the significance of the path coefficients (chin, 1998). five thousand (5000) bootstrap samples were used to estimate the pls path model (hair et al., 2014). the r2 values for the constructs determine the predictive accuracy of the model. the r2 values for enjoyment and interest constructs exceeded the moderate threshold predictive accuracy that was proposed by chin (1998), while the r2 values of the gratification construct and the programming performance construct were weak. the results of the path analysis in figure 3 shows that relationships h1 (programming selfefficacy -> enjoyment), h3 (programming self-efficacy -> interest), and h5 (programming selfefficacy -> gratification) were positive and significant. australasian journal of information systems kanaparan, cullen & mason 2019, vol 23, selected papers from acis 2017 self efficacy & engagement on students 13 note: t-values > 1.65* (p<0.1); t-values > 1.96** (p<0.05); t-values > 2.57*** (p<0.01) figure 3. structural model with results of path analysis a strong positive and significant relationship was found in path h2 (enjoyment -> programming performance), while the result of the path analysis for h4 (interest -> programming performance) showed a negative but statistically significant relationship. however, hypothesis h6 (gratification -> programming performance) showed a weak negative and not statistically significant relationship. therefore, hypothesis h6 was not supported. the research model was tested for common method bias since the data for the independent and the dependent variables was collected using a single method (podsakoff, mackenzie, lee, & podsakoff, 2003). an exploratory factor analysis (efa) was performed on all the measurement items in order to control for common method bias. common method bias may exist in a dataset if the majority (>50%) of the items load on one factor (podsakoff & organ, 1986). the results of the efa showed that the largest variance explained by an individual factor was 24.04%, suggesting that common method bias was not a problem in this research. 5 discussion this study proposed and tested a research model on programming self-efficacy, emotional engagement, and programming performance. enjoyment, interest, and gratification were identified as three indicators of emotional engagement in introductory programming courses. several key findings emerged from this research. our first finding suggests that the strong link between programming self-efficacy and emotional engagement validates the importance of self-efficacy beliefs on human behaviour, and the importance of bandura’s social cognitive theory within the context of students in introductory programming courses. second, the positive relationships between programming self-efficacy and enjoyment, and between programming self-efficacy and interest supports the findings from the published literature on learning theory. third, the positive relationship between programming self-efficacy and gratification is strong evidence that gratification is an indicator of emotional engagement and that it appears to be an important engagement factor in the programming discipline although no published research could be found to support this relationship. australasian journal of information systems kanaparan, cullen & mason 2019, vol 23, selected papers from acis 2017 self efficacy & engagement on students 14 fourth, the results showed a positive relationship between enjoyment and the programming performance of the students. this result supports the findings from the published literature on learning theory that found positive correlations between enjoyment and performance (frenzel et al., 2007), and that enjoyment is an important emotional engagement factor that leads to better programming performance in introductory programming courses. fifth, the negative relationship between interest and programming performance did not support our initial hypothesis and contradicts existing research which found positive correlations between interest and programming performance (mckinney & denton, 2004; wiedenbeck et al., 2007). o’keefe and linnenbrink-garcia (2014) argued that students who show interest in performing a task may be doing so due to affect-related interest, while students who place importance in performing a task well may be demonstrating value-related interest. in this study, it may be possible that the research participants had high affect-related interest but low value-related interest. as a result, the research participants may have placed more importance on being interested in learning to program and had not considered the criteria for performing well in their introductory programming course. lastly, despite a lack of evidence in the literature, the relationship between gratification and programming performance was hypothesised as a positive relationship. however, the findings in this research revealed a weak negative relationship between gratification and programming performance. this finding suggests that although students felt gratified upon seeing their program work without errors, their actual performance may not have been satisfactory. one plausible explanation could be that although the students were able to see the output of their program and were able to make their program work without errors, they may not have met all the requirements of the programming assessment that would lead to a better programming grade. these findings have implications for course instructors who are involved in the design and delivery of introductory programming courses. we argue that the cognitive demands of learning programming may be managed by building self-efficacy beliefs, by embedding enjoyable learning activities, and by encouraging affect-related interest while simultaneously developing the value-related interest of the students. table 5 summarises the implications and recommendations for course instructors. implication recommendation build self-efficacy beliefs to manage cognitive demands • weekly programming exercises to encourage practice • provide feedback on exercises • design programming tasks around threshold concepts embed enjoyable learning activities • encourage pair programming • use new programming tools • apply gamification encourage affect-related interest, develop value-related interest • provide clear guidelines and explain criteria for performing well table 5. implications and recommendations for course instructors social cognitive theorists argue that self-efficacy beliefs may be developed through enactive mastery, vicarious experiences, verbal persuasion, and/or physiological cues (bandura, 1997). one way to apply the enactive mastery approach is by introducing weekly programming australasian journal of information systems kanaparan, cullen & mason 2019, vol 23, selected papers from acis 2017 self efficacy & engagement on students 15 exercises that would enable students to practice writing programs so that they are able to improve their understanding. the feedback provided by the course instructor on the weekly programming exercises would enable the student to understand the source of the errors and how to overcome them, leading to increased self-efficacy beliefs to learn programming. in particular, course instructors should focus on designing programming tasks around threshold concepts, which is one factor that contributes to the difficulties of learning programming. threshold concepts are troublesome concepts that may hinder learning as the learner may become stuck, frustrated, lose interest in the subject, adopt a surface approach to learning, or even withdraw from the course (boustedt et al., 2007; sorva, 2010) which could result in low self-efficacy beliefs. threshold concepts are typically difficult to master, but once the programming concept is understood, students will be able to progress to the next programming concept. by focusing on programming tasks that require students to repeatedly master these threshold concepts, students may be able to increase their self-efficacy beliefs in programming. furthermore, students will be able to see immediate results upon completing their programming task and thus experience a feeling of gratification. the feeling of gratification is an indicator of emotional engagement unique to this study, and not found in other studies related to learning programming. evidence from the focus groups suggests that the challenge of learning programming creates a sense of enjoyment in students. in order to support students who may feel discouraged by the challenges of learning programming, course instructors could embed learning activities that are intended to invoke the feeling of enjoyment. using pair programming (liebenberg et al., 2012; maguire et al., 2014), introducing new programming tools (bishop-clark et al., 2007) and applying gamification techniques (fotaris, mastoras, leinfellner, & rosunally, 2016) are some examples from the extant research on computer programming that have successfully engaged students in enjoyable activities when learning programming. interest in programming clearly has an effect on the performance of the students. students who show affectrelated interest in programming should continue to be encouraged and should simultaneously be guided to develop their value-related interest. course instructors could develop the value-related interest of the students by providing clear guidelines and explaining the criteria for performing well in their programming assessment 6 conclusion this research examined the relationship between programming self-efficacy and emotional engagement, and the relationship between emotional engagement and the programming performance of students in introductory programming courses. we argue that the results of this study may be generalised to the larger population of students in introductory programming courses as data was collected from a fairly large (433 students) heterogeneous group of participants. additionally, data was collected from several institutions that offer tertiary level education in malaysia and in new zealand with students of various nationalities enrolled in their introductory programming course. gender imbalance and an uneven distribution in the programming grade obtained by the participants may have skewed the generalisability of our findings. since 72.7% of the participants in this study were male students, the findings in this study may be generalised to male students in introductory programming courses. further, the number of participants who performed poorly in their introductory programming course was small (8.8%). self-selection australasian journal of information systems kanaparan, cullen & mason 2019, vol 23, selected papers from acis 2017 self efficacy & engagement on students 16 bias is one factor which may have led to the small number of participants in the group that received a lower programming grade. thus, the findings of this study may be generalised to students who obtained a good or excellent programming grade in their introductory programming course. there was a strong positive relationship between programming self-efficacy and the emotional engagement factors. enjoyment was clearly an important emotional engagement factor for success in programming. by contrast, the negative relationship between interest and programming performance did not support our initial hypothesis. despite this, the negative relationship revealed interesting insights on how students perceived interest in their introductory programming course. although the relationship between gratification and programming performance was not statistically significant, we argue that gratification has the potential to be an indicator of emotional engagement in introductory programming courses and could benefit from further research. this study contributes to the design and delivery of introductory programming courses by making recommendations to introductory programming course instructors who wish to understand the behaviour of students and improve the performance of their students. recommendations made include: to improve self-efficacy beliefs, embed enjoyable learning activities, and to develop the value-related interest of students. additionally, this study revealed that the feeling of immediate gratification is an emotional engagement factor that appears to be unique to learning programming. for future research, experiments could be carried out to examine the outcome of implementing the recommendations made to course instructors to improve the design and delivery of introductory programming courses. studies could also be conducted in the next level of programming course that the student progress on to, in order to examine if the student’s performance in the introductory programming course had an effect on their programming self-efficacy beliefs. in conclusion, it is hoped that the findings and recommendations in this study would provide guidance to course instructors to develop and design courses that encourages introductory programming students to increase their self-efficacy beliefs and to be emotionally engaged with the course. in doing so, it is hoped that students may be able to improve their performance, thereby reducing the failure and attrition rates in introductory programming courses. references akbulut, a. y., & looney, c. a. (2007). inspiring students to pursue computing degrees. communications of the acm, 50(10), 67-71. appleton, j. j., christenson, s. l., & furlong, m. j. (2008). student engagement with school: critical conceptual and methodological issues of the construct. psychology in the schools, 45(5), 369386. appleton, j. j., christenson, s. l., kim, d., & reschly, a. l. (2006). measuring cognitive and psychological engagement: validation of the student engagement instrument. journal of school psychology, 44(5), 427-445. australasian journal of information systems kanaparan, cullen & mason 2019, vol 23, selected papers from acis 2017 self efficacy & engagement on students 17 australian government department of jobs and small business. (2018). australian jobs 2018 is now available. retrieved october 15, 2018, from https://www.jobs.gov.au/news/australian-jobs-2018-now-available bandura, a. (1977). self-efficacy: toward a unifying theory of behavioral change. psychological review, 84(2), 191-215. bandura, a. (1977a). social learning theory. englewood cliffs, new jersey: prentice hall. bandura, a. (1986). social foundations of thought and action. englewood cliffs, nj: prentice hall. bandura, a. (1997). self-efficacy: the exercise of control. new york: w h freeman and company. bartimote-aufflick, k., bridgeman, a., walker, r., sharma, m. & smith, l. (2016). the study, evaluation, and improvement of university student self-efficacy. studies in higher education, 41(11), 1918-1942, https://doi.org/10.1080/03075079.2014.999319 bennedsen, j., & caspersen, m. e. (2007). failure rates in introductory programming. sigcse bulletin, 39(2), 32-36. betts, j. e., appleton, j. j., reschly, a. l., christenson, s. l., & huebner, e. s. (2010). a study of the factorial invariance of the student engagement instrument (sei): results from middle and high school students. school psychology quarterly, 25(2), 84-93. bishop-clark, c., courte, j., evans, d., & howard, e. v. (2007). a quantitative and qualitative investigation of using alice programming to improve confidence, enjoyment and achievement among non-majors. journal of educational computing research, 37(2), 193207. boustedt, j., eckerdal, a., mccartney, r., mostrom, j. e., ratcliffe, m., sanders, k., & zander, c. (2007). threshold concepts in computer science: do they exist and are they useful? paper presented at the proceedings of the 38th sigcse technical symposium on computer science education, covington, kentucky, usa. retrieved from https://dl.acm.org/citation.cfm?doid=1227310.1227482 bureau of labour statistics. (2018). occupational outlook handbook. retrieved october 15, 2018, from https://www.bls.gov/ooh/computer-and-information-technology/softwaredevelopers.htm bye, d., pushkar, d., & conway, m. (2007). motivation, interest, and positive affect in traditional and nontraditional undergraduate students. adult education quarterly, 57(2), 141-158. chin, w. w. (1998). the partial least squares approach to structural equation modelling. in g. a. marcoulides (ed.), modern methods for business research (pp. 295–336). mahwah, nj: lawrence erlbaum associates. christenson, s. l., reschly, a. l., & wylie, c. (2012). handbook of research on student engagement. retrieved from http://vuw.eblib.com/patron/fullrecord.aspx?p=884351 davis, w. a. (1982). a causal theory of enjoyment. mind, 91(362), 240-256, doi: 10.2307/2253480 debry, r. (2011). a learning space for beginning programming students. journal of computing sciences in colleges, 27(2), 4-11. australasian journal of information systems kanaparan, cullen & mason 2019, vol 23, selected papers from acis 2017 self efficacy & engagement on students 18 ford, m., & venema, s. (2010). assessing the success of an introductory programming course. journal of information technology education, (9), 133-145. fornell, c., & larcker, d. f. (1981). evaluating structural equation models with unobservable variables and measurement error. journal of marketing research, 18(1), 39-50. fotaris, p., mastoras, t., leinfellner, r. & rosunally, y. (2016). climbing up the leaderboard: an empirical study of applying gamification techniques to a computer programming class. electronic journal of e-learning, 14(2), 94-110. fredricks, j. a., blumenfeld, p. c., & paris, a. h. (2004). school engagement: potential of the concept, state of the evidence. review of educational research, 74(1), 59-109. frenzel, a. c., thrash, t. m., pekrun, r., & goetz, t. (2007). achievement emotions in germany and china -a cross-cultural validation of the academic emotions questionnaire-mathematics. journal of cross-cultural psychology, 38(3), 302-309, doi: http://dx.doi.org/10.1177/0022022107300276 hair, j. f., hult, g. t. m., ringle, c. m., & sarstedt, m. (2014). a primer on partial least squares structural equation modeling (pls-sem). sage publications, inc. kanaparan, g., cullen, r., & mason, d. (2013). self-efficacy and engagement as predictors of student programming performance. paper presented at the proceedings of the 2013 pacific asia conference on information systems (pacis 2013). paper 282. https://aisel.aisnet.org/pacis2013/282 kinnunen, p., & simon, b. (2011). cs majors' self-efficacy perceptions in cs1: results in light of social cognitive theory. paper presented at the seventh international workshop on computing education research, providence, rhode island, usa. retrieved from https://dl.acm.org/citation.cfm?id=2016917 krueger, r. a., & casey, m. a. (2009). focus groups: a practical guide for applied research (4th ed.). thousand oaks, california: sage publications, inc. liebenberg, j., mentz, e., & breed, b. (2012). pair programming and secondary school girls’ enjoyment of programming and the subject information technology (it). computer science education, 22(3), 219-236, doi: 10.1080/08993408.2012.713180 linnenbrink, e. a., & pintrich, p. r. (2003). the role of self-efficacy beliefs in student engagement and learning in the classroom. reading & writing quarterly, 19(2), 119-137. mackenzie, s. b., podsakoff, p. m., & podsakoff, n. p. (2011). construct measurement and validation procedures in mis and behavioral research: integrating new and existing techniques. mis quarterly, 35(2), 293-334. maguire, p., maguire, r., hyland, p., & marshall, p. (2014). enhancing collaborative learning using pair programming: who benefits? aishe-j: the all ireland journal of teaching and learning in higher education, 6(2), 1411-14125. mckinney, d., & denton, l. f. (2004). houston, we have a problem: there's a leak in the cs1 affective oxygen tank. sigcse bulletin, 36(1), 236-239, doi: 10.1145/1028174.971386 mills, n., pajares, f., & herron, c. (2007). self-efficacy of college intermediate french students: relation to achievement and motivation. language learning, 57(3), 417-442, doi: 10.1111/j.1467-9922.2007.00421.x australasian journal of information systems kanaparan, cullen & mason 2019, vol 23, selected papers from acis 2017 self efficacy & engagement on students 19 mischel, w., shoda, y., & rodriguez, m. (1989). delay of gratification in children. science, 244(4907), 933-938. doi: 10.1126/science.2658056 moore, g. c., & benbasat, i. (1991). development of an instrument to measure the perceptions of adopting an information technology innovation. information systems research, 2(3), 192-222. o'keefe, p. a., & linnenbrink-garcia, l. (2014). the role of interest in optimizing performance and self-regulation. journal of experimental social psychology, 53(0), 70-78, doi: http://dx.doi.org/10.1016/j.jesp.2014.02.004 pajares, f. (1997). current directions in self-efficacy research. in m.maehr & p. r. pintrich (eds.), advances in motivation and achievement (vol. 10, pp. 1 49). greenwich, ct: jai press. phan, h. p. (2011). interrelations between self-efficacy and learning approaches: a developmental approach. educational psychology, 31(2), 225-246. podsakoff, p. m., mackenzie, s. b., lee, j. y., & podsakoff, n. p. (2003). common method biases in behavioral research: a critical review of the literature and recommended remedies. journal of applied psychology, 88(5), 879 903. podsakoff, p. m., & organ, d. w. (1986). self-reports in organizational research: problems and prospects. journal of management, 12, 531-544. putwain, d., sander, p., & larkin, d. (2013). academic self-efficacy in study-related skills and behaviours: relations with learning-related emotions and academic success. british journal of educational psychology, 83(4), 633-650. doi: 10.1111/j.2044-8279.2012.02084.x quille, k., & bergin, s. (2016). programming: further factors that influence success. paper presented at the twenty-seventh annual workshop of the psychology of programming interest group (ppig 2016), cambridge, uk. retrieved from http://www.ppig.org/library/paper/programming-further-factors-influence-success ramalingam, v., labelle, d., & wiedenbeck, s. (2004). self-efficacy and mental models in learning to program. sigcse bulletin, 36(3), 171-175. ramalingam, v., & wiedenbeck, s. (1998). development and validation of scores on a computer programming self-efficacy scale and group analyses of novice programmer self-efficacy. journal of educational computing research, 19(4), 367 381. reschly, a. l., & christenson, s. l. (2012). jingle, jangle, and conceptual haziness: evolution and future directions of the engagement construct. in s. l. christenson, a. l. reschly & c. r. f. wylie (eds.), handbook of research on student engagement. dordrecht: springer science+business media new york. retrieved from http://vuw.eblib.com/patron/fullrecord.aspx?p=884351. ringle, c. m., wende, s., & becker, j.-m. (2015). smartpls 3. boenningstedt: smartpls gmbh, http://www.smartpls.com. schunk, d. h. (1996). self-efficacy for learning and performance. paper presented at the annual meeting of the american educational research association, new york. retrieved from https://files.eric.ed.gov/fulltext/ed394663.pdf australasian journal of information systems kanaparan, cullen & mason 2019, vol 23, selected papers from acis 2017 self efficacy & engagement on students 20 schunk, d. h., & mullen, c. a. (2012). self-efficacy as an engaged learner. in s. l. christenson, a. l. reschly & c. wylie (eds.), handbook of research on student engagement. dordrecht: springer science+business media new york. retrieved from http://vuw.eblib.com/patron/fullrecord.aspx?p=884351.silvia, p. j. 2003. “self-efficacy and interest: experimental studies of optimal incompetence,” journal of vocational behavior (62:2), pp 237-249, doi: http://dx.doi.org/10.1016/s0001-8791(02)00013-1 sheard, j., carbone, a., & hurst, a. j. (2010). student engagement in first year of an ict degree: staff and student perceptions. computer science education, 20(1), 1-16, doi: 10.1080/08993400903484396 sherer, m., maddux, j. e., mercandante, b., prentice-dunn, s., jacobs, b., & rogers, r. w. (1982). the self-efficacy scale: construction and validation. psychological reports, 51(2), 663-671. doi: 10.2466/pr0.1982.51.2.663 sorva, j. (2010). reflections on threshold concepts in computer programming and beyond. paper presented at the proceedings of the 10th koli calling international conference on computing education research, koli, finland. retrieved from https://dl.acm.org/citation.cfm?doid=1930464.1930467 stangor, c. (2011). research methods for the behavioral sciences. belmont, ca, usa: wadsworth cengage learning. thomas, d. r. (2006). a general inductive approach for analyzing qualitative evaluation data. american journal of evaluation, 27(2), 237-246. doi: 10.1177/1098214005283748 walker, c. o., greene, b. a., & mansell, r. a. (2006). identification with academics, intrinsic/extrinsic motivation, and self-efficacy as predictors of cognitive engagement. learning and individual differences, 16(1), 1-12. https://doi.org/10.1016/j.lindif.2005.06.004 watson, c., li, f. w. b., & godwin, j. l. (2014). no tests required: comparing traditional and dynamic predictors of programming success. paper presented at the proceedings of the 45th acm technical symposium on computer science education, atlanta, georgia, usa. https://dl.acm.org/citation.cfm?doid=2538862.2538930 wiedenbeck, s., xiaoning, s., & chintakovid, t. (2007). antecedents to end users' success in learning to program in an introductory programming course. paper presented at the ieee symposium on visual languages and human-centric computing, 2007. vl/hcc 2007. retrieved from http://ieeexplore.ieee.org/document/4351328/ yazzie-mintz, e., & mccormick, k. (2012). finding the humanity in the data: understanding, measuring and strengthening student engagement. in s. l. christenson, a. l. reschly & c. wylie (eds.), handbook of research on student engagement (pp. 743-762). dordrecht: springer science+business media new york. retrieved from http://vuw.eblib.com/patron/fullrecord.aspx?p=884351.yip, m. c. w. 2012. “learning strategies and self-efficacy as predictors of academic performance: a preliminary study,” quality in higher education (18:1), pp 23-34. doi: 10.1080/13538322.2012.667263 yip, m. c. w. (2012). learning strategies and self-efficacy as predictors of academic performance: a preliminary study. quality in higher education, 18(1), 23-34. https://doi.org/10.1080/13538322.2012.667263 australasian journal of information systems kanaparan, cullen & mason 2019, vol 23, selected papers from acis 2017 self efficacy & engagement on students 21 zimmerman, b. j. (2000). self-efficacy: an essential motive to learn. contemporary educational psychology, 25(1), 82-91. zingaro, d. (2014). peer instruction contributes to self-efficacy in cs1. paper presented at the 45th acm technical symposium on computer science education, atlanta, georgia, usa. https://dl.acm.org/citation.cfm?doid=2538862.2538878 australasian journal of information systems kanaparan, cullen & mason 2019, vol 23, selected papers from acis 2017 self efficacy & engagement on students 22 appendices appendix a: survey items construct: programming self-efficacy item adapted from factor item note: all items start with “i am confident that…….” general selfefficacy i can complete a new task that is assigned to me if i keep trying. bresó et al. (2011) i can achieve the goals that i set for myself in this programming course. i have the capability to learn the contents of this programming course. when i learn something new, i will not give up easily if i am not successful initially. sherer at al. (1982) when unexpected problems occur i can handle them well. i can stick to completing a task even though it may be unpleasant. independence and persistence i can complete my programming assessment if i had a lot of time. ramalingam & wiedenbeck (1998) i can complete my programming project if i refer to resources such as the built-in help, programming reference manuals, or online programming forums. i can find ways of overcoming the problem if i get stuck at a point while working on a programming assessment. i can correct (debug) all the errors in a program that i have written, and make it work. complex programming tasks i can apply the right programming concepts to solve a problem given to me. i can understand and apply the fundamental object-oriented programming concepts used in my programming course. i can make use of a pre-written library in the programming integrated development environment (ide). i can write a program to solve any given problem as long as the specifications are clear. i can mentally trace through the execution of a complex program given to me. i can find a way to concentrate on my programming project, even when there are many distractions around me. i can find ways of motivating myself to program, even if the problem area was of no interest to me. simple programming tasks i can write programming code that runs without errors (no syntax errors). i can write programming code that is logical. i can write a small program for a small problem that is familiar to me. australasian journal of information systems kanaparan, cullen & mason 2019, vol 23, selected papers from acis 2017 self efficacy & engagement on students 23 construct: programming self-efficacy item adapted from factor item note: all items start with “i am confident that…….” i can understand the language structure and the usage of the reserved words /keywords in the programming language. i can write a reasonably sized program that can solve a problem that is only vaguely familiar to me. construct: enjoyment item adapted from item my programming course is stimulating compared to the other courses that i am enrolled in. bishopclark et. al (2007) i like writing programs. the challenge of coding solutions to programming problems appeals to me. construct: interest item adapted from item my programming assessments take priority over the assessments from other courses. focus groups i am passionate about programming. my programming course suits me better than the other courses that i am currently enrolled in. construct: gratification item adapted from item i feel a deep sense of satisfaction when i finally get my program to work. focus groups i feel relieved when i complete my programming assessment. i feel a strong sense of achievement when i complete my programming assessments. once i complete my programming assessment, i feel pleased that i have successfully completed a challenging piece of assessment. copyright: © 2019 kanaparan, cullen & mason. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems kanaparan, cullen & mason 2019, vol 23, selected papers from acis 2017 self efficacy & engagement on students 24 1 introduction 2 theoretical framework and research hypotheses 2.1 programming self-efficacy and social cognitive theory 2.2 emotional engagement and student engagement 2.2.1 focus groups – validating indicators of emotional engagement 2.2.2 enjoyment 2.2.3 interest 2.2.4 gratification 2.3 programming performance: the dependent variable 2.4 confounding variables 2.5 research model 3 methodology 3.1 survey instrument 3.2 data collection 4 results 5 discussion 6 conclusion appendix a: survey items the australasian journal of information systems (ajis) began in 1993 and was based on the simple realisation that there was a need for an academic ‘outlet’ for is research in australia volume 18 number 2 2014 australasian journal of information systems ajis issn 1326-2238 eissn 1449-8618 australasian journal of information systems volume 18 number 2 2014 1 australasian journal of information systems (issn 1326-2238 eissn 1449-8618) published by australasian chapter of the association for information systems (aais) website hosted by the australian computer society (acs) management board dr deborah bunker, the university of sydney (acphis president) dr glenn stewart, queensland university of technology (aais president) dr mark toleman, university of southern queensland (acphis past president) dr don kerr, university of the sunshine coast (aais past president) editor-in-chief dr john lamp, deakin university section editors in this issue amir talaei-khoei, don kerr, lorraine staehr, david wilson reviewers in this issue babak abedin, steven clark, justin debuse, jenny edwards, igor hawryszkiewycz, tony hookins, luke houghton, kenneth kendall, hossein khakbaz, ryan ko, elaine lawrence, robyn lawson, craig mcdonald, nitin naik, charles palmer, craig parker, lisa soon, suchada vichitvanichphong, liisa von hellens previous editors-in-chief dr kathy lynch, university of the sunshine coast (2012-2013) dr jon heales, university of queensland (2009-2012) dr craig mcdonald, university of queensland (2006-2009) dr rob macgregor, university of wollongong (founding editor: 1993 2006) submissions and subscriptions details of journal scope and instructions to authors can be found on the journal’s website: http://dl.acs.org.au/index.php/ajis/about/submissions#authorguidelines. the journal is free to access online, however, please contact the editor-in-chief for information regarding purchase of hard copies of the journal. enquiries and comments editorial enquires should be directed to ajis editor-in-chief (ajis.eic@gmail.com) paper title (paper title style) australasian journal of information systems sawas & watfa 2015, vol. 19, pp. 97-112 cloud computing and agility the impact of cloud computing on information systems agility mohamed s. sawas faculty of engineering and info. sciences university of wollongong in dubai s_sawas@hotmail.com mohamed k. watfa faculty of engineering and info. sciences university of wollongong in dubai mohamedwatfa @uowdubai.ac.ae abstract as businesses are encountering frequent harsh economic conditions, concepts such as outsourcing, agile and lean management, change management and cost reduction are constantly gaining more attention. this is because these concepts are all aimed at saving on budgets and facing unexpected changes. latest technologies like cloud computing promise to turn it, that has always been viewed as a cost centre, into a source of saving money and driving flexibility and agility to the business. the purpose of this paper is to first compile a set of attributes that govern the agility benefits added to information systems by cloud computing and then develop a survey-based instrument to measure these agility benefits. our research analysis employs non-probability sampling based on a combination of convenience and judgment. this approach was used to obtain a representative sample of participants from potential companies belonging to various industries such as oil & gas, banking, private, government and semi-governmental organizations. this research will enable decision makers to measure agility enhancements and hence compare the agility of information systems before and after deploying cloud computing. keywords: cloud computing; agility; information systems; agile is; agile business 1 introduction the notion of “doing more with less” had spread during the last few decades as a result of tough economic conditions facing businesses. examples of such conditions are: recurrent downturns and recessions. this difficult situation imposed fierce competition on companies, and obliged them to seek reduction of costs while at the same time maximize profit and customer value. businesses have to turn into agile enterprises, i.e. to be able to sense changes in the environment and respond quickly to these changes (overby et. al., 2006). the changes might happen at different levels: in the economy, in competitive environments or in customer needs, among others. agility is a characteristic of highly competitive organizations that can quickly make decisions in order to survive changes in uncertain, turbulent and complex environments. agile organizations drive corporate benefits by focusing on short time to market, continuous improvement, customer value and fast response to market changes. the agility concept originated from the manufacturing field in the early nineties. it comprised two attributes: leanness and flexibility. leanness is the elimination of non value-adding activities, whereas methods were used in software development projects for complex systems that require frequent changes, quick response and short release times. subsequently, agile methodology was applied to project management in general. moreover, agile became a management philosophy, and the concept of agile enterprise emerged (dove, 2001). research had shown that information system (is) agility is one of the most vital factors in sustaining a firm’s strategic alignment (jorfi et. al., 2011), therefore, it becomes clear that an agile enterprise necessitates an agile is (pessi et. al., 2009). it departments face serious agility 97 australasian journal of information systems sawas & watfa 2015, vol. 19, pp. 97-112 cloud computing and agility problems such as: outdated infrastructures, underutilized capacities, and rigid architectures. these problems hinder organizations’ flexibility and responsiveness, increase costs and reduce efficiency. 1.1 problem statement cloud computing is a new model for fulfilling corporate it needs. it is service based, flexible and cost effective. one of the important benefits of cloud computing (indicated by how much interest it received in the literature) is the agility it brings to a company’s is, and as a result, to the business itself. most of the research cites agility as one of the major attributes of cloud computing (carroll et. al. , 2011) and it is believed that cloud computing improves is agility. however, it is unclear how cloud computing, when integrated with existing components of an organization’s is, adds to the agility of that is as a whole, and furthermore, how can we measure or estimate the claimed agility improvement. while the adoption of cloud computing among businesses is growing, cloud computing has not come of age yet, and apart from cloud vendors' marketing hype, little research effort has been conducted to determine the impact of cloud computing on is agility (yang, et. al. 2013) using various agility dimensions identified in former research work. decision makers need to be able to realize the agility gains achieved by deploying cloud computing. such ability will allow them to view the situation before and after the change, and thus, make an informed decision about adding cloud computing to their it investment portfolios. 1.2 purpose of the study & research questions this study aims to build a framework for measuring agility changes, introduced by integrating cloud computing into the is of a given organization. also, it seeks to create a survey tool to measure those agility changes. the research question under study is: how to measure the impact of cloud computing on is agility? to answer that question, two sub-problems need to be addressed: 1. determine the factors that affect agility of is (attributes). 2. design a survey-based instrument to measure agility changes, brought to the is by cloud computing, based on the attributes determined in the first sub-problem. 1.3 research methodology in order to validate the hypothesis claiming that cloud computing improves is agility, a noncausal investigation is performed. it starts by conceptualizing the is agility construct through a literature review, and then operationalizes the construct when cloud computing is deployed into the is. operationalization is done using a survey which is based on a compiled group of agility-related attributes. the study verifies any pre-existing correlation between different cloud computing service models and different categories of agility. also, it makes and validates some claims in relation to the research question: to what extent, those who see agility improvement, exceed those who do not? the research is conducted in a non-contrived setting, using a survey on a longitudinal basis. 1.4 significance of the study agility has been described as the capability to effectively sense and respond to environmental change. much of the contemporary is literature focuses on defining agility and on whether information systems enable agility in the enterprise. also, due to the benefits that cloud computing offers businesses, many organizations have started building applications in the cloud, seeking enhanced business agility by using flexible and elastic cloud services. nevertheless, moving applications and/or data into the cloud is not straightforward. to leverage the full potential promised by cloud computing, numerous challenges have to be addressed. these challenges are often related to the fact that existing applications have specific requirements. this research takes up the unaddressed question of whether cloud computing is 98 australasian journal of information systems sawas & watfa 2015, vol. 19, pp. 97-112 cloud computing and agility really capable of enhancing is agility, and how this enhancement materializes. also, it enables decision makers to measure this agility enhancement and hence compare the agility of the is before and after cloud computing is deployed. 2 literature review 2.1 information systems agility an information system is often considered as the heart of any organization; hence, the performance of the enterprise depends on the efficiency of its information system (izza et al., 2008). the enterprise strategy is influenced by the socio economic, legislative and technology changes. moreover, globalization of the economy makes enterprise information systems more complex and makes competition increasingly fierce. therefore, to ensure survival and sustainability of the enterprise, it must be permanently agile. enterprise policies must rapidly adapt to the enterprise strategy, in other words, an enterprise must quickly drive important changes at all levels and dimensions, in order to align them to its strategy and vice versa. this can be achieved by relying on an appropriate set of best practices and/or standards. actually, the information system is becoming a tool of strategy for most organizations. earlier research on measuring agility of information systems surveyed different levels of it staff, using questionnaires, to assess various dimensions of agility in an is. maurer (maurer, 2010) provides a definition of is agility by combining many agility aspects from prior research. he defines is agility based on three second-order dimensions (technical infrastructure agility, is process agility and human characteristics); each one of them is broken down into 3-4 firstorder dimensions. then he provides an overview of a reliable survey-based scale that measures is agility. imache, izza and nacer (2012) propose a poire framework composed of five elements (process, organization, information, resource and environment) to assess enterprise is agility based on two main principles: urbanization and continuous improvement. they employ a survey instrument that consists of a group of criteria (questions) and a fuzzy logic method to perform evaluation for these five dimensions. terry and douglas (byrd et al., 2000) define the it infrastructure flexibility construct, operationalize it by developing a survey-based measurement instrument, which is created based on some predefined dimensions of it infrastructure flexibility, and then employ statistical processes to validate and verify reliability of the measurement. 2.2 cloud computing cloud computing has emerged as a paradigm to deliver on demand resources (e.g., infrastructure, platform, software, etc.) to customers similar to other utilities (e.g., water, electricity and gas). three main services are provided by a cloud computing architecture according to the needs of it customers (buyya et. al., 2009). firstly, software as a service (saas) provides access to complete applications as a service, such as customer relationship management (crm) (cusumano, 2010). secondly, platform as a service (paas) provides a platform for developing other applications on top of it, such as the google app engine (gae) (ciurana, 2009). finally, infrastructure as a service (iaas) provides an environment for deploying, running and managing virtual machines and storage. technically, iaas offers incremental scalability (scale up and down) of computing resources and on-demand storage (buyya et. al., 2009). carroll, van der merwe & kotze (2011) studied existing research on cloud computing, and conducted interviews with 15 senior managers in order to outline cloud computing benefits and security risks, and then recommended some controls to mitigate these risks. finally, they suggested creating a comprehensive framework of risk and control for cloud computing. the ‘cloud services measurement initiative consortium’ csmic developed a hierarchical framework that contains a number of key performance indicators (kpis) or metrics to measure and compare different cloud services (perdue et. al., 2011). garg, versteeg & buyya (2013), proposed a framework to quantitatively measure iaas cloud services based on the service measurement index (smi) metrics suggested by cloud service measurement index consortium (csmic). they developed a mechanism to discover, monitor, and calculate service 99 australasian journal of information systems sawas & watfa 2015, vol. 19, pp. 97-112 cloud computing and agility metrics for different cloud providers, and then rank them and select the best one meeting user’s essential and non-essential requirements. different metrics are weighted according to user’s priorities and the analytical hierarchy process (ahp) method is used to rank cloud services. finally, they applied the ahp calculation method in an example case to obtain the metrics for three cloud service providers and rank them accordingly. li, o'brien, zhang & cai (2013), aiming for evaluating and benchmarking commercial cloud services, compiled a catalog of de facto metrics using a systematic literature review (slr) of existing cloud services evaluation work. yang, huff & tate (2013) worked on the conceptualization of is agility based on prior research to evaluate the contribution of different cloud computing services to the is agility. cloud computing promises everything, from reduced complexity and unlimited scalability to capacity on demand and capex savings. although there are still many unanswered questions about cloud computing, many businesses are optimistic that it will be able to deliver on these promises. even skeptics are intrigued by cloud computing, if for no other reason than that they want to maintain a technological edge over their competitors, and the cloud is certainly the new technology trend. to whatever extent cloud computing delivers on these promises, one thing is certain: businesses are not willing to sacrifice security, visibility, and control to make the move to the cloud. they need to know what is happening in the cloud, how their applications are being delivered, and how traffic is being controlled and directed. what is emerging as a “must have” in cloud computing is agility: the quality that enables enterprises to respond quickly and precisely to unexpected and changing business demands. agile businesses, those that can provide it on demand under any workload conditions, can seize new opportunities and stay competitive. this is what led us to further pursue this research to see whether or not cloud computing is really capable of enhancing is agility and how much would companies be willing to sacrifice other cloud computing offerings. table 1 summarizes our literature review, highlighting the different methodologies and contributions. figure 1 presents a bibliographical tree of relevant prior research, detailing how various research work relates to or elaborates on each other and the gap in the research literature. 100 australasian journal of information systems sawas & watfa 2015, vol. 19, pp. 97-112 cloud computing and agility ref. research question methodology theoretical basis conclusion/extension (maurer, 2010). combine earlier definitions of is agility and outline steps for developing a reliable scale to measure it conceptual; synthesis of previous research works prior research focused on three main aspects of is agility: agility of the it technical infrastructure, the supporting processes, and the staff gives a brief overview of a measurement scale, the validation and finalization of it yet to be performed (imache et. al., 2012) apply the poire framework to is agility and measure it using fuzzy logic survey and case agility is assessed using fuzzy logic based on four dimensions: production infrastructure, market infrastructure, people infrastructure and information infrastructure applying the model to estimate the agility of a sample company clarified the connection and the consistency among the different components of the poire framework and its practicability. the model ignores the interactions between the dimensions, factors and criteria (byrd et al., 2000) define the flexibility construct, develop a measurement instrument, statistically analyze, validate and verify its reliability conceptual and survey it infrastructure flexibility construct further refinement of discovered factors, the relative contribution of each factor to the competitive advantage, relation between flexibility , its costs and its benefits (carroll et. al, 2011) outline cloud computing benefits and security risks and then recommend some controls to mitigate those risks qualitative approach, interview 15 senior managers. prior research on cloud computing benefits and risks suggests creation of a comprehensive framework of risk and control for cloud computing (garg et. al. , 2013) propose a framework to evaluate cloud offerings and rank them based on their ability to meet the user’s quality of service (qos) requirements designing metrics for each quantifiable qos attribute for measuring the service level of cloud providers attributes of services from service measurement index (smi) metrics by cloud service measurement index consortium (csmic ) and service ranking using analytical hierarchy process (ahp) extend the ranking algorithm to adapt variations in qos attributes using fuzzy sets, extend the quality model to nonquantifiable qos attributes, apply the smi framework on infrastructures provided by 2 cloud vendors, extend a cloud application platform to utilize services of the framework (li, et. al. 2013) identify, assess and synthesize the published primary studies of cloud services evaluation systematic literature review existing cloud services evaluation work evaluate elasticity andsecurity of commercial cloud services, develop more sophisticated metrics for better evaluation (yang, et. al. 2013) conceptualize is agility and how cloud computing might facilitate it cloud computing services are evaluated based on different dimensions of is agility prior research on cloud computing services iaas has the potential to deliver agility whereas paas, and saas agility gains depend on organizational and human factors table 1. summary of relevant literature including methodology used and corresponding theoretical basis. 101 australasian journal of information systems sawas & watfa 2015, vol. 19, pp. 97-112 cloud computing and agility cloud computing and is agility agility mitra 2006; cohen, lindvall & costa 2004; borjesson, martinsson & timmeras 2006 measuring is agility imache, izza & nacer 2012; terry & douglas 2000 is agility dimensions maurer 2010 organizational activities to improve is agility jandoš 2011; cloud computing wienman 2012; alford &morton 2009 technical infrastructur agility allen & boynton 1991; weill et al. 2002; sambamurthy et al 2003; human characteristics fink & neumann 2007; gebauer & schober 2006 is process agility boynton et al. 1994; wang et al. 2008; davis 2009; process, organization & environment tsourveloudis et al 2002;lui &piccoli 2007 business-related issues marston, zhi, bandyopadhyay, zhang & ghalsasi 2011 cloud computing advantages carroll, merwe, & kotze cloud computing metrics garg, versteeg & buyyaa 2012 li, o'brien, zhang, cai 2012 cloud computing for is agility yang, huff & tate 2013 figure 1. bibliography tree summarizing the gap in the literature. arrows indicate further research conducted on specific aspects. 3 data collection and analysis 3.1 sampling the unit of analysis is an organization that already has some kind of cloud computing as part of its is. sampling details regarding are listed below: target population: data were collected from it managers, cios, ctos, and senior it professionals from selected uae organizations that use any form of cloud computing. sampling method: non-probability sampling, based on a combination of convenience and judgment to obtain a representative sample. 102 australasian journal of information systems sawas & watfa 2015, vol. 19, pp. 97-112 cloud computing and agility selection procedure: select participants from potential companies belonging to various industries, such as oil & gas, banking, private, government and semigovernmental organizations. sample size: 78 companies. 3.2 operationalization the suggested framework to measure the different agility aspects is shown in figure 2. survey questions are related to different agility categories and subcategories mapped to our proposed framework of attributes. the result is a set of twenty eight different questions as summarized in table 2. table 3 is a matrix that maps attributes in the framework above to different types of survey questions. general information 1. demographic information 2. our organization uses the following cloud service models 3. our organization uses the following cloud deployment models technical infrastructure agility application agility 4. our organization is adding new products/services, with our cloud deployment in place, less time and effort is needed to add or modify the applications necessary to support those new products/services 5. in my opinion, using cloud computing enhanced our ability to create reusable applications information agility 6. if i need to share some data from one of our cloud applications with somebody outside the organization, i would be able to export that data using standard formats (such as xml) 7. i believe that deploying cloud applications improved information sharing within our organization and between our organization and external partners 8. if an employee wants to retrieve data from an application, he would experience faster access if the application resides in the cloud rather than on-premises compatibility & interoperability 9. if a need arises to migrate our cloud applications from one cloud service provider to another, i would see a smooth and easy migration process 10. a business requirement dictates connecting our on-premises environment to our cloud environment by linking a cloud application with an on-premises application, i believe it is possible to achieve that requirement easily and securely elasticity 11. when a customer-facing cloud application witnesses an increasing demand from our customers (because of seasonal or unexpected reasons), it can increase its resource consumption on the fly to meet that demand and then reverts to normal level when demand falls 12. our business is expanding and i want our cloud applications to cope with that expansion, all i need to do is to contact our cloud service provider and inform it about the required increase in allocated bandwidth and computing resources it processes agility maintenance process agility 13. in my opinion, compared to traditional applications, cloud applications require less time and effort for support and maintenance 14. a new branch of our business will be opened, i can tell that integrating the new branch into our enterprise has become less complex since we used the cloud 103 australasian journal of information systems sawas & watfa 2015, vol. 19, pp. 97-112 cloud computing and agility planning process agility 15. now that our it infrastructure contains both cloud and on-premises components, when i have to evaluate and prioritize proposed changes, i find it easier than before 16. since we started deploying systems and applications in the cloud and because of cloud elasticity features, i feel more relieved from complicated capacity planning monitoring and assessment process agility 17. compared to traditional it systems, gathering performance information for systems in the cloud seems easier 18. upon introducing cloud systems into our environment, i noticed that service management became less complex than before human characteristics training & staff 19. a new it staff member was hired and she is learning to work with different systems and applications managed by our department, it is obvious that the training required for cloud systems is less (compared to traditional it systems) 20. migrating some of our systems to the cloud reduced the number of it staff required to manage those systems business and technical skills 21. i think, as a result of utilizing the cloud, it staff’s task in translating business problems into technical solutions became simpler 22. i think that compared to traditional it systems, cloud systems are less prone to human error 23. utilizing cloud computing in our business did not require it staff to learn new it service delivery skills business aspects response 24. utilizing cloud computing made me feel more confident in our ability to counter unexpected changes (i.e. unexpected events such as corrections and reconfigurations) 25. after we added cloud technologies to our it investment portfolio, i feel more satisfied about our efficiency and effectiveness in seizing emerging opportunities customer service 26. after moving a customer-facing application to the cloud, our customers’ feedback indicates a notable improvement in customer service competitive status 27. from my experience with our business, i see a positive contribution of using the cloud to the alignment of it strategies with business strategies 28. cloud computing helped our organization to widen the array of products and/or services offered without increasing cost. table 2. interview questions using our proposed framework. 104 australasian journal of information systems sawas & watfa 2015, vol. 19, pp. 97-112 cloud computing and agility figure 2. framework for the impact of cloud computing on information systems agility attribute category attribute name behaviour opinion feeling knowledge sensory demographic 0 general information q 2,3 q 1 technical infrastructure agility 1 application agility q 5 q 4 2 information agility q 6 q 7 q 8 3 compatibility & interoperability q 10 q 9 4 elasticity q 12 q 11 it processes agility 5 maintenance process agility q 13 q 14 6 planning process agility q 16 q 15 7 monitoring & assessment process agility q 17,18 human characteristics 8 training & staff q 20 q 19 9 business and technical skills q 21,22 q 23 business aspects 10 response q 24,25 11 customer service q 26 12 competitive status q 28 q 27 table 3. attribute – question type matrix 3.3 data collection we created an internet survey using an online survey tool. this tool sends email invitations with a link to the survey to all participants. it allows the user to collect data and export it as a standard text file for further analysis. framework for the impact of cloud computing on information systems agility technical infrastructure agility application agility information agility compatibility & interoperability elasticity it processes agility maintenance process agility planning process agility monitoring & assessment process agility human characteristics training & staff business and technical skills business aspects response customer service competitive status 105 australasian journal of information systems sawas & watfa 2015, vol. 19, pp. 97-112 cloud computing and agility 3.4 data analysis data from the online survey tool were imported into ibm spss statistics software version 21 to perform cross tabulation, chi-square, and single population t-tests. detailed frequency tables of responses to the aggregated is agility categories are summarized as bar charts in the results section below. cross-tabulations of aggregated is agility categories and service models along with their chi-square tests were performed to find any associations between the cloud service model used and the different categories of is agility. also, different single population t-tests were performed to test several hypotheses related to the average number of population elements who see agility improvement in its different models. more specifically, the following sixteen hypotheses were formulated and tested at 95% confidence level. the first twelve hypotheses (h1-h12) include different association existence between the different agility categories and cloud computing models formulated as follows: h1-h12: there is no association between using a software model as a cloud service model and improving the is agility category of the is where the cloud model is either {iaas, paas, or saas} & the is agility category is either {technical infrastructure agility (tia), it processes agility (ipa), human characteristics (hc), or business aspects (ba)} the next four hypotheses (h13-h16) include different association existence between the different agility categories and cloud computing models formulated as follows: h13-h16: the average number of population elements who see agility improvement in an agility category exceeds the number of those who see otherwise by less than x % where the is agility category is either {technical infrastructure agility (tia), it processes agility (ipa), human characteristics (hc), or business aspects (ba)} and x is to be determined accordingly. 4 results this section summarizes the final results. tables 4 and 5 depict usage frequencies and percentages for various cloud computing service models and deployment models. figure 3. shows responses to the aggregated is agility categories. responses % percent of cases service modela iaas 52.4% 78.6% paas 19.0% 28.6% saas 28.6% 42.9% total 100.0% 150.0% a. dichotomy group tabulated at value 1. table 4. cloud computing service models usage frequencies and percentages responses % percent of cases deployment modela public cloud 26.1% 33.3% hybrid cloud 13.0% 16.7% private cloud 60.9% 77.8% total 100.0% 127.8% a. dichotomy group tabulated at value 1. table 5. cloud computing deployment models usage frequencies and percentages 106 australasian journal of information systems sawas & watfa 2015, vol. 19, pp. 97-112 cloud computing and agility figure 3. combined frequency distributions for responses to the aggregated is agility categories from a total of 162 respondents. as can be seen in table 4 and table 5, the infrastructure-as-a-service as well as the private cloud dominate the usage percentages which is quite expected when it comes to the it infrastructure in the uae. also, an interesting observation from figure 3, includes the high percentage of responses to the business aspects agility category including: more user confidence to counter unexpected changes (i.e. unexpected events such as corrections and reconfigurations), more satisfaction about the efficiency and effectiveness in seizing emerging opportunities, and a positive contribution of using the cloud to the alignment of it strategies with business strategies. for analysing the associations between the cloud service models used and the categories of is agility, figure 4 summarizes the cross tabulation results between the different agility categories and cloud models responses. figure 4. technical infrastructure agility (tia), it processes agility (ipa), human characteristics (hc), business aspects (ba) service model cross tabulation results. 107 australasian journal of information systems sawas & watfa 2015, vol. 19, pp. 97-112 cloud computing and agility for testing the first twelve hypotheses (h1-h12), table 6 summarizes the results from spss. technical infrastructure agility it processes agility human characteristics business aspects iaas paas saas iaas paas saas iaas paas saas iaas paas saas pearson chi-square .080 .235 .748 .082 .302 .413 .148 .026 .729 .128 .005 .370 likelihood ratio .079 .233 .742 .059 .305 .394 .133 .019 .724 .077 .004 .233 linear-by-linear assoc. .951 .249 .593 .181 .402 .630 .312 .559 .440 .567 .025 .236 no of valid cases 162 108 90 90 table 6. chi-square tests (asymp. sig. (2-sided) ) technical infrastructure agility (tia), it processes agility (ipa), human characteristics (hc), business aspects (ba) using different service models (infrastructure-as-a-service, platform-as-a-service, softwareas-a-service) figure 5. hypotheses test analysis using pearson chi-square asymp. sig. p-value (1-sided) of the different service models compared to a significanc level). four different associations. table 7 summarizes the results based on the chi-square tests with the null hypothesis being "there is no association" and the alternative hypothesis being that "an association exists". 108 australasian journal of information systems sawas & watfa 2015, vol. 19, pp. 97-112 cloud computing and agility iaas paas saas technical infrastructure agility reject null fail to reject null fail to reject null it processes agility reject null fail to reject null fail to reject null human characteristics fail to reject null reject null fail to reject null business aspects fail to reject null reject null fail to reject null table 7: hypotheses testing results for combinations of cloud service models and categories of is agility with null hypothesis h0 being no association and alternative hypothesis ha being association exists. the following conclusions can be summarized accordingly: conclusion 1: there is sufficient evidence to conclude that there is an association between using iaas as a cloud service model and improving the technical infrastructure agility or the it process agility of the is. there is no evident association between iaas and other agility categories. conclusion 2: there is sufficient evidence to conclude that there is an association between using paas as a cloud service model and improving the human characteristics or the business aspects of the is. there is no association between paas and other agility categories. conclusion 3: there is no association between saas and any agility category. this conclusion was a quite interesting given that saas would normally imply less it staff and fixed running it cost which would normally imply more business agility. this conclusion opens the door for more future research on this topic. the cloud is about providing services: infrastructure as a service (iaas), platform as a service (paas), or software as a service (saas). another way to look at it: the cloud is about providing a pool of computing resources that all operate together, effectively as a single computer. however as can be noted from the conclusions resulting from the statistical analysis, not all of these services have a direct association with agility as perceived by it professionals. more specifically, paas cloud model gives users the ability to respond to business demands more effectively and helps ensure employees have on demand access to critical business information, customers, partners, and each other, using nearly any device, from virtually anywhere. therefore, users can give priority to the most critical business tasks first, hence conclusion 2. however, iaas focuses more on decreasing the management burden of anticipating and building out excess capacity it infrastructure resulting in less management, maintenance, and deployment time, with the additional benefit of greater scalability to more easily handle peaks in demand; hence, conclusion 1. conclusion 3 shows the lack of association between saas and any agility category. this conclusion is supported by the fact that saas is more perceived as a cost saving option rather than an agility enhancement. in contrast to conventional financial models of software vendors who depend upon up front software license fees to support their p&ls, saas allows users to pay for only what is utilized during a given period but at the same time results in far less dependency on local it staff and services which would raise some limitations on agility in general. a right-tailed t-tests for the difference in numbers of those who see agility improvement, and those who do not, are given in table 8. 109 australasian journal of information systems sawas & watfa 2015, vol. 19, pp. 97-112 cloud computing and agility table 8. right-tailed t-tests for the difference in percentage between those who see agility improvement and those who do not (tests were performed for the various agility dimensions). ne sided p-value is less of t-tests regarding the consent among population elements pertaining to agility gains achieved. figure 6. hypotheses testing results for population elements who see agility improvement exceeding those who see otherwise. the following conclusion can be summarized: conclusion 4: there is sufficient evidence to say that the average number of population elements who see agility improvement in technical infrastructure agility, it one-sample test & statistics mean std. deviation std. error mean mean technical infra. agility 0.2901 0.35982 0.08481 mean it processes agility 0.3148 0.37438 0.08824 mean human characteristics 0.2111 0.41429 0.09765 mean business aspects 0.5278 0.41911 0.09879 t sig. (2-tailed) mean difference 95% conf. interval of the difference lower upper mean technical infra. agility (test value=0.14) 1.77 0.095 0.15012 -0.0288 0.3291 mean it processes agility (test value=0.16) 1.754 0.097 0.15481 -0.0314 0.341 mean human characteristics (test value=0.04) 1.752 0.098 0.17111 -0.0349 0.3771 mean business aspects (test value=0.35) 1.8 0.09 0.17778 -0.0306 0.3862 110 australasian journal of information systems sawas & watfa 2015, vol. 19, pp. 97-112 cloud computing and agility process agility, human characteristics, and business aspects exceeds the number of those who see otherwise by more than 14%, 16%, 4% and 35% respectively. from conclusion 4, it is evident that our initial findings from figure 3 are further confirmed with regards to the high percentage of responses to business aspects agility improvements. more specifically, this percentage difference is a result of cloud computing improvements in: user confidence to counter unexpected changes, efficiency and effectiveness in seizing emerging opportunities, and alignment of it strategies with business strategies. also, an evident observation from figure 6 is regarding the high percentage of acceptance on agility improvement with regards to process agility. more specifically, this percentage difference is a result of positive responses to cloud computing models implications. these implications include: decreasing time and effort for support and maintenance, simplifying new branch integration, decreasing the effort to evaluate and prioritize proposed changes, easier capacity planning and performance information gathering, and simplified service management. 5 discussion and conclusion the research question of this study was about measuring the impact of cloud computing on is agility. to evaluate how cloud computing changes is agility, it started by conducting a literature review and then compiled four groups of attributes in a proposed testing framework to be considered when looking at is agility. a survey was built based on these attributes and was distributed to senior it executives and professionals from companies who have deployed some sort of cloud computing. these companies were selected from different industries to make the sample more representative, participants responses were collected and statistically analyzed in order to find any associations between using cloud computing and improving agility. based on results of the research, we concluded that some cloud computing service models improve specific agility dimensions, for example, iaas improves technical infrastructure agility and paas improves human characteristics while saas does not associate with any category. we also concluded that agility improvements in the business aspects were the dominant agility category in the it industry. in general, it seems that cloud computing still needs a lot of improvement in order to convince businesses on the agility aspects it can provide. more precisely, additional work needs to be done on saas association with different categories of agility among others. references buyya, r., yeo, c., venugopal, s., broberg, j, brandic, i., (2009). cloud computing and emerging it platforms: vision, hype, and reality for delivering computing as the 5th utility, future generation computer systems 25 (6) 599– 616. byrd, t, and turner, d, (2000). "measuring the flexibility of information technology infrastructure: exploratory analysis of a construct," j. manage. inf. syst., vol. 17, pp. 167-208. carroll, m., van der merwe, a., and kotze, p. (2011). secure cloud computing: benefits, risks and controls, pp. 1-9. ciurana, e (2009). developing with google app engine, apress, berkeley, ca, usa. cusumano, m (2010). cloud computing and saas as new computing platforms, communications of the acm 53 (4) 27–29. dove, r. (2001). response ability: the language, structure and culture of agile enterprise, wiley publishers, isbn: 978-0-471-35018-7. 111 australasian journal of information systems sawas & watfa 2015, vol. 19, pp. 97-112 cloud computing and agility garg, s., versteeg, s., and buyya, r., (2013). "a framework for ranking of cloud computing services," future generation computer systems, vol. 29, p. 1-12. imache, r, izza, s and ahmed-nacer, m, (2012). "an enterprise information system agility assessment model," computer science and information systems, vol. 9, pp. 107-133. izza, s, imache, r, vincent, l, and lounis, y (2008). an approach for the evaluation of the agility in the context of enterprise interoperability, ed: springer, pp. 3-14. jorfi, s., md nor, k, and najjar, l (2011). "the relationships between it flexibility, itbusiness strategic alignment, and it capability," international journal of managing information technology, vol. 3, pp. 16-31. li, z., o'brien, l., zhang, h., and cai, r. (2013). on a catalogue of metrics for evaluating commercial cloud services, 1-10. maurer, c (2010). "measuring information systems agility: construct definition and scale development," sais 2010 proceedings. overby, e., bharadwaj, a. and sambamurthy, v. (2006). "enterprise agility and the enabling role of information technology," european journal of information systems, vol. 15, pp. 120-131. perdue, jeff, siegel, jane, (2011). cloud-services-measurement-initiative-consortium". csmic framework, v1.0, carnegie mellon university silicon valley. pessi, k., magoulas, t. and hugoson, m. (2009). "architectural principles for alignment within the context of agile enterprises," in proceedings of the 3rd european conference on information management and evaluation, sweden. -2010)," economics and management, pp. 892-899. yang, h., huff, s., and tate, m. (2013). "managing the cloud for information systems agility," in cloud computing service and deployment models: layers and management, ed: igi global, pp. 70-93. copyright: © 2015 sawas & watfa. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. 112 microsoft word ajis special 2002.doc ajis special issue december 2002 70 a critique of generalizability in interpretive research mike metcalfe & maureen lynch university of south australia mike.metcalfe@unisa.edu.au abstract this paper is about generalizability in interpretive systems research. the authors are concerned that, in its search for appropriate inquiry methods, the discipline of information systems (is) does not slip into the errors of other social disciplines and become a dismal science. many systems thinkers have repeatedly argued that the purpose of is research needs to be to produce what argyris calls actionable knowledge and ulrich, critique heuristics. that is, rules of thumb which managers can use to solve design problems, not too vague and not too detailed. this is a different concept from generalizability, which aligns with the scientific notion of seeking universal objective laws. the paper uses the argumentative inquiry in its reflexive capacity to critique knowledge claims on a chapter by baskerville and lee that discusses generalizability to bring out the differences between this cornerstone of universal truths and the more systems thinking concepts of actionable knowledge or critical heuristics. the problem baskerville and lee (1999) wrote a chapter entitled “distinguishing among different types of generalizing in information systems research” in which they refine the definition of “generalization” for the benefit of interpretive empirical inquiry. my9 concern is that they may have inadvertently led new researchers to think that generalizability is in some way central to interpretive systems research when this richer research tradition has a very different set of priorities. at least five different research perspectives provide examples of these priorities, notably in the now extensive argumentative (rehg, 1999), systems inquiry (churchman, 1971), perspectival thinking (linstone, 1999; haynes, 2000), actionable knowledge (argyris, 1996) and critical social theory (alvesson and skoldberg, 2000) literature. moreover, any discussion of generalizabilty might want to address popper’s (1963) critique of induction. the alternative priorities in interpretive systems research can be loosely clustered into one single claim that social inquiry is best seen as the justification of a perspective (insights, falsifiable conjectures). this justification may or may not use measured empirics, as empirics are understood to be actions to assist thinking, or merely illustrations intended to improve the human condition. generalization is not central to this view of empirics. so, more specifically, this paper will explore the conjecture that interpretive inquiry offers alternative priorities than generalizabity that are more appropriate for the “wicked” problem of information systems (is) related inquiry. rather than merely list the attributes of these alternatives, they will be used as the lens with which to critique baskerville and lee’s chapter constructively. it is hoped this will serve two purposes. the first is to reveal how the work of these five perspectives might be used systematically to critique any social inquiry and so reveal underlying assumptions. encouraging critique methods lies at the core of interpretive inquiry. the secondary purpose is to give further airing to the evaluation criteria these five perspectives suggest for interpretive inquiry, empirical or not. in line with most inquiry methods, some upfront working definitions may be helpful to establish common ground. my understanding of generalization is that it is the concept of being able to extrapolate from a sample onto the population from which it is drawn, as a whole. interpretive is inquiry, whether empirical or not, is the search for justified interpretations (perspectives) of events or things, maybe distilled from the multiple interpretations of others in order to improve the appreciation of the researcher (system designer). by argumentation is meant the process of making a one line claim, conjecture, conclusion or proposition that has to be justified (well reasoned) to a knowledgeable cynical audience using convincing evidence. it includes anticipating the counter evidence. objective knowledge is defined in terms of popper’s (1971) cultural world 2. it is knowledge agreed upon by the majority of a culture to the extent it has become independent of any one member of that culture, and science is a culture. objective-empirics are typically observations that are repeatable by skilled people at any time in any situation. a critique is more useful if the reader is familiar with the proposition, in this case baskerville and lee’s chapter. for those readers who are not familiar with this chapter, their “conclusion” is provided again here. 4. conclusion the existing concept of generalizability is fastened on one particular form of generalizing. the concept is further confounded in the is literature by its conflation with distinct concepts like generality and generalizing. untangling this confusion reveals a variety of obscured, and sometimes legitimate forms, of generalizing. many intensive research ventures could satisfy one of these forms, yet the authors self-flagellate over their inability to found their generalities on the one particular form that is researchers have mindlessly idealised. 9 for style reasons, this paper is written in the first person. ajis special issue december 2002 71 the self-flagellation is not merely annoying; it is harmful. when researchers unnecessarily divest their right to claim generality, their research audience is denied their analysis of the utility of their theories. when researchers generalize and claim generality, they encompass the larger scope of phenomena, beyond those directly captured by their research, to which their findings and understandings apply… by renouncing their right to generalize and claim generality, intensive researchers lose the latitude to explain the wide field of uses for their findings… i am not critical of what they say but rather of the centrality of “generalization” to interpretive research. argumentative inquiry and systems thinking popper’s work “conjectures and refutations” (1963) will need no introduction to scientific researchers. he solved the long induction versus deduction debate by calling for conjectures to be argued. to be “scientific”, these conjectures have to be risky or falsifiable rather than being inductions from empirics. rehg (1999) usefully summarizes the now extensive literature (e.g. perelman and olbrechts-tyteca, 1969; walton, 1998; van eemeren et al, 1987) on the use of argumentation as an interpretive inquiry method. he believes that argument, supported by convincing justification, provides a useful middle road between objectivism and relativism. it is the methodology of much of the critical social theory literature (alvesson and skoldberg, 2000) but one that asks that an argument has some surprise value or insight so as to emancipate. as a critique method, argumentative inquiry encourages questions such as, “what was the chapter’s argument? were terms well defined (common ground)? was the supporting evidence convincing? was the counter evidence dealt with well?” it also helps raise questions about the context of the argument such as, “why is the author presenting this argument and what are the implications of accepting the argument?” critical social theory prompts further questions such as, “what was their insight and how does that improve the future?” however, apart from relevance and possible counters, argumentative inquiry does not provide much advice on how to critique evidence beyond saying that it should be convincing. here systems thinking (ulrich, 1983) and linstone’s (1999) multiple perspectives can be of some use. systems thinking prompts questions like, “where is the boundary, the inter-connectivity, the purpose, the transformation and the stakeholders” perspective?” the perspective approach, as developed by churchman (1971), checkland (2000) and haynes (2000) can be used to complement argument in another way. this involves the explicit separation of the object from the perspective (lens, frame) of that object. so, a new system can be made the “object-under-study” and then alternative perspectives (efficiency, emancipation, interconnectivity) be used to bound the inquiry into that object. this perspectival thinking can be mapped onto an argument line that has an object and subject (metcalfe and lynch, 2002). for example, the argument, “generalizability is good” makes the object-under-study “generalization” and “goodness” the perspective. this separation then allows the two to be examined, defined, described separately prior to bringing them together to make up an argument that will need to be justified using convincing evidence. for numerous other perspectives on how to critique an article see metcalfe (2002). all these perspectives on human inquiry have been merged under argumentative inquiry into a pattern of six fairly broad questions (cues, themes) intended to assist in the critique of claims. these have been used to critique baskerville and lee’s chapter on generalizability. as the purpose of the questions is simply to prompt thought, the responses are intended to introduce new perspectives not to be generalizable. critique prompts 1) do the authors explain why they are making their argument? do they acknowledge any intent to persuade the audience of their conclusion? i think they do. they believe there is a problem with some interpretive researchers feeling defensive that their findings are not generalizable because they were not based on the statistical assumptions of sampling. they do this by identifying different types of generalizability. for example, they define “generality” as generalizing to some subset of “the entire class of empirical referents”. that their advice is important is assumed as generalization is assumed to be a very important measure of the usefulness of interpretive research. i think the authors did intend to persuade, but simply by presenting their own sub-definitions of generalizability. there are no empirics either of their own or of previous writers presented, nor are any alternative interpretations of the umbrella concept of generalizabilty considered. 2) what is the object or thing the chapter is studying? did they discuss, define this object in terms of its boundaries, relationships, how it changes over time and its purpose? due to the well-structured and clear style of writing, there is no doubt that the object-under-study is generalizability. the authors want to re-define this more carefully into four different sub-types. argumentation theory, starting from aristotle, would commend this as definitions of words to be used in an argument are best agreed up front so as to avoid semantic misunderstandings. put another way, common ground needs to be ajis special issue december 2002 72 established prior to presenting evidence. in social inquiry arguments, a lot of time has to be spent on defining (describing, constructing) terms. a classic example is the: “all men are mortal, socrates is a man, therefore socrates is mortal” argument of aristotle. as a social inquiry argument, this is very ambiguous, as “mortality” and “socrates” need careful definition. does mortality include in memory as well as the body, was socrates a person or a personification, and are ideas mortal? however, the authors present their sub-definitions of generalizability with little recourse to alternatives. it appears that they are not aiming to establish common ground prior to presenting evidence; rather it is their evidence. the authors do provide one phrase to define generalizabilty. they write “generalizability is a potential that a theory has, …” and draw a parallel with falsifiability10 as also being a potential that a theory has. there is no discussion of what a theory is or how essential it is to inquiry, methodologically speaking. they seemed to be simply telling the reader some unquestioned and unquestionable objective facts about generalizations. systems thinking encourages thought on interconnectivity of the object-under-study that makes me ask why “searching for objective facts” styles of inquiry place so much importance on generalizability. i think it is an outcome of that cornerstone of objectivity, repeatability. results that can be repeated across time and space are generalizable. clearly, repeatability is a problem with inquiry into human behaviour. this is because we, unlike physical objects like molecules, learn, making repeatable complex social situations difficult to achieve and study. it is because of this that social inquiry, such as is research, is designed to alter the future behaviour of the objects under their study, people. this underlines the important differences between objectivist and interpretive (seeking interpretations) research and calls into question the use of an objectivist evaluation tool for interpretive inquiry. if you heat up molecules three times without explaining your purpose, they respond in exactly the same way. if you do the same to people, they are likely to abuse you. thinking about interconnectivity also leads to thoughts about the connection between interpretive research and statistical sampling. the idea behind sampling is that, if a sample has external validity, then averages that are true of the sample can be said to be true of (are generalizable to) the population. averaging and sampling are two concepts that need to be used with care in interpretive inquiry. so, for example, with stakeholder analysis, which is so central to systems design, the averaging of the perspective of stakeholders would not be helpful. sampling, and averaging, is therefore a concept not really compatible with seeking powerful stakeholders” interpretations. if one powerful stakeholder has a perspective, it is relevant, and perspectives can spread as stakeholders talk to each other. interpretive research seeks perspectives, in particular unusual ones, not average perspectives. 3) what was the authors’ perspective (lens, intellectual frame…) on generalizabilty? did they discuss this and appreciate the need to seek a range of perspectives in order to undertake interpretive research into generalizabilty? the chapter authors seemed concerned that “intensive” is research be generalizable because, if it were, that would make it scientific and therefore the results would produce good knowledge. they are concerned to produce objective knowledge. this, coupled with their use of words like “theory” and “the class of empirical referents”, forces me to assume they think research can be defined only as the search for objective facts, therefore their perspective is an empirical objectivist one. their advice is all about how to add rigour to empirical measurement methods. they group what they call positivist, qualitative inquiry with interpretivist (qualitative or quantitative) inquiry. put another way, it seems predictable that an objectivist would give wellmeaning advice to interpretivists (as well as objectivists) that excludes attempts to explore alternative interpretations of the issue of the generalizabilty. the perspective of those numerous epistemologists who present a more inclusive definition of knowledge are not presented (see ulrich, 1983; perelman, 1969; van eemeren et al, 1987 and rehg, 2002), to say nothing of the postmodernists. for example, the issue of generalizabilty has long been an issue in historical inquiry. what do they have to say on the issue? how generalizable are the causes of the outbreak of the first world war? historians have long argued that to dismiss their inquiries as not producing good knowledge, because some interpretations of historical facts may not be generalizable, is a little simplistic. one of the seminal philosophers in is, churchman, as long ago as 1971 recommended and demonstrated that inquiry benefits from seeking these multiple perspectives (worldviews). his ideas have inspired two generations of is inquirers. these include mitroff, mason, argyris, linstone and checkland. these writers suggest that any inquiry into generalizabily would seek to be inclusive of some alternative perspectives on knowledge. this work is not mentioned in the chapter. my perspective, having been influenced by the works of the above writers, is that generalizability is not the most pressing performance measure that editors of interpretive inquiry should be looking for. more pressing is ensuring all stakeholders in an inquiry are given a voice; their justified perspectives need to be appreciated. moreover, argyris’ (1996) call to judge inquiry by its production of “actionable” knowledge is also very relevant. rehg’s (1999) justified argument or alvesson and skodberg’s (2000) “justified insights” seem similar 10 for a good critique of falisfiabilty see martin gardener (2001), skeptical inquirer 25(4), pp13-14 ajis special issue december 2002 73 calls. one of the consequential differences of undertaking inquiry into “objects” (subjects), often more intelligent and experienced than the inquirer, is that the unique opportunity exists to not only seek your interpretation of their behaviour but theirs also. to provide the researcher’s explanation and prediction confines inquiry into physical objects. however, that is about all you can do when studying inanimate objects (molecules), or people as inanimate objects. when studying people as intelligent and experienced equals, inquiry should seek their prediction of how they would like to do things in the future. the critical social theorists call this “social action”; argyris calls it a “theory for action”. using the systems design language for this, the inquiry should result in recommendations for the design of a better system. again, this is an option barred to inquiry into physical objects. we cannot design nature. 4) what does the chapter argue/conclude? neither the introduction of the chapter nor the conclusion is presented as an argument. rather, it states in the introduction that the purpose (a good systems term) is “to clarify the different processes of generalizing …” so, the chapter is perhaps a description of the objective concept of “generalizabilty”. however, using the argumentation perspective introduces some caution in accepting writings as neutral descriptions. i interpret the chapter as having an implicit argument that generalizability is an important performance measure for interpretive empirical research. moreover, if interpretive researchers try hard, their inquiries could be acceptable to the objectivist. indeed, this appears to be a “given” not worthy of any justification. their suggestions do not appear to be open to debate they are telling it like it really is, explaining it carefully to the “molecules”. 5) what evidence is brought to support the argument/conclusion? was this evidence convincing? was the counter argument fully considered? were there any empirics? if so why, if not why not? the chapter uses reasoning evidence to justify the authors’ argument that interpretive empirical results may be generalizable. no empirics are used to create this knowledge. knowledge is often created this way; indeed the rationalists demanded it and einstein provided an excellent example in his relativity theory. the reasoning presented in the chapter, as is often the case in social inquiry, comes in two parts: definitions of generalizability and definitions of standards of what constitutes good knowledge. first consider the provided definitions of generalizabilty. a quick check in the dictionary confirms that the authors’ definitions are not in common usage, and they explicitly say “we define” referring to their variants of the “general” concept. so, the only evidence provided is their un-argued sub-definitions of the possible different types of generalizability. put another way, no attempt is made to anticipate the counter evidence or multiple perspectives. supporting evidence for their four types of generalizability could have drawn on how it was used in the past, its roots in language or in its association with statistical testing. do their sub-definitions of the types of generalizability align with statistical testing theory? if something is generalizable from some subset of the sample, then is it necessary to demonstrate this statistically? also, “classification” is a large subject in library science, what theoretical justification is there for their particular sub-division of generalizability? what other types are possible? the second arm to the reasoning in the chapter is that practitioners have lower standards of what constitutes knowledge than scientists. i figure that someone can be both a scientist and a practitioner and have these two standards. i am a bit nervous of any suggestion of elitism with knowledge; popper’s language of knowing within a culture seems preferable. it seems just a power thing to assume practitioners have an impure version of knowledge that only scientists can purify. scientists get their respect from having detailed knowledge, usually from precise measurement (cohen, 1994). practitioners can spend their every working minute among some complex social problems, which are not measurable or repeatable. they acquire the most intimate experience of the detail, knowing more than scientists will ever be able to measure from a site visit. the definition of knowledge is something that has plagued philosophers for some time. popper (1963) presents the conjecture that some idea like the law of gravity or evolution is simply the best argument yet, waiting for a kuhnian revolution when a better argument is presented (kuhn, 1970). einstein’s relativity did this to newton’s attraction argument. this conjecture might be about an objective fact or a kuhnian perspective. perspectival knowledge includes how to “see” organisations (morgan, 1986), how we should treat other people, where best to invest and what defines scientific knowledge. the role of argument is to allow people who claim to know something (have an interpretation) to attempt to justify their position. if empirical objectivist knowledge is the best and only true form of knowledge, then it seems reasonable to use its standards back on the generalizability chapter. the chapter does not apply the empirical objectivist standard on itself so should we assume it has produced unacceptable knowledge? if objective empirics are the only source of good knowledge, how could the authors have tested their knowledge that interpretive results can be generalizable? rather than suggest some trivial and unending empirical program to test interpretive results for generality, an alternative might be to again turn the test on those who claim its usefulness. how generalizable are the results of social scientific research that has used the recommended statistical sampling? i suspect not at all. i struggle to think of any truly generalizable rule i have learnt from reading maybe thousands of so called social scientific articles. this includes research i have designed myself aimed at showing objectively and ajis special issue december 2002 74 empirically that small groups made up of self-selected members outperformed (statistically significant at 5%) those groups that had members allocated based on criteria like ability, cognitive style and inclusion needs. i could claim, as the tradition demands, that these results were generalizable, but my life experience stops me. concerns over external validity were created by the very demands for empirical rigour of experiments. it became a “catch 22”. if i made the experiment realistic, then i lost the independence of variables. if i increased independence, i lost external validity. either way, generalizability suffered. i do not believe that anything i could have done would have produced generalizable results, using the word as a practitioner would. moreover, by defining types of generalizability using only their own reasoning, baskerville and lee have silenced the perspective of interpretive researchers. this raises emancipatory concerns. is power to be the judge of good knowledge; are there to be “insightful experts” whose opinions are the truth? or rather, should there be a more democratic process for setting the criteria of good research? as an australian, supportive of a culture that is very wary of tall poppies, the democratic option sounds preferable. the apparent impartiality of repeatable actions that provide the same outcome sounds attractive. but there are problems, as popper (1963) points out. who says the repeated results are to be classified together, the results still need to be interpreted, and what if repeatable actions are not possible? popper’s solution is argumentation where everybody is given a voice to refute. moreover, by not treating the definition of generalizability as worthy of empirical evidence, the authors have missed the opportunity to learn from action – a key facet of systems thinking. rather than thinking of empirics as an exercise in precise measurement, it can be thought of as an opportunity to justify an insight (alvesson and skodberg, 2000). inquiry starts with an idea (perspective, intellectual frame) that can be learnt by actions that provide specific input into our senses. this may be done through experiments, measurements or by seeking the perspectives of those who have seen and heard things you cannot, by learning from them. in action research, this is extended to getting a group to experience a novel idea in actions and then reflect upon the idea. again this perspective does not see generalization as the most important criterion for empirics. counters so what are the counters to generalizability? as mentioned in the introduction, there are many. briefly, argyris (1996) argues that what should be provided by social inquiry is actionable knowledge. if managers are really to alter their actions as a result of an inquiry, they need to be given a rule of thumb to help make decisions. system design seeks solutions to problems, solutions that can be enacted. some solutions, like argumentation, are critique methods to find or seek or think through solutions to problems (ulrich, 1983; metcalfe, 2002). these need to be at the right level of conception. too vague, like general systems theory, and the manager will struggle to apply it meaningfully to the many problems that present themselves on a daily basis. too demanding, such as significance testing, and it cannot be applied quickly to all problem situations. the example argyris provides refers to gatekeeper roles. his actionable knowledge is that gatekeeper roles bestow power on the holders, so think carefully about whether a gatekeeper role is involved in a problem’s solution and how it is to be managed. ulrich (1983, 2001) calls for the creation of critical heuristics (rule of thumb critique methods) to explore the boundary, a very systems term, of a proposal. he advocates systems thinking, seeking churchman’s perspectives, and argumentation perspectives. checkland (2000) seems to support the first two, to improve the appreciation of the systems designer. argumentation (perelman, 1969; van eemeren, 1987; walton, 1998; rehg, 1999) sees the idea of providing a justified argument using convincing evidence as the equivalent to generalization. if the argument that systems x is best for situation y is convincing, then the boundary (generalizabilty?) is explicit. perspectival thinking (haynes, 2000) seems to have a slightly different agenda. it feels the purpose is to identify and separate the object under inquiry from the perspective. this allows for appreciation that the object can be inquired about from different perspectives. for example, a new system (the object) can be inquired about from an economic, environmental and emancipatory perspective. generalizability, if analogous, would be the search for new perspectives. last, critical social theory (alvesson and skoldberg, 2000) provides another counter to generalization being the dominant evaluation of inquiry. rather, this perspective sees the extent to which the inquiry improves society as the most relevant evaluation. 6) were the implications (the so-what) of accepting the authors’ argument explicit? accepting the authors’ explicit argument that interpretive empirics are generalizable to some extent rather lures systems interpretivists into being evaluated by objectivist criteria. this reinforces the impression that objective empirics borrowed from the physical sciences are the best and only way to produce useful knowledge. it also de-motivates discussion on alternative criteria to evaluate the search for knowledge to other than repeatable measurements. the interpretive literature has many suggestions about how to evaluate inquires these need more voice. examples such as convincing argument and actionable knowledge have been mentioned in the previous section. ajis special issue december 2002 75 conclusion i have argued against generalizabilty as a dominant measure of good interpretive system research, drawing on popper’s solution to the induction versus deduction debate that is to call for argumentative inquiry. this was elaborated using the system thinking problem solving concepts and of perspectival thinking as a critique lens on generalizabity. good knowledge is seen to be as much about appreciating justified perspectives as it is about seeking universal truths for human behaviour. rather than measuring research in terms of how well it can be generalized, it was suggested that preferable criteria might be how much it will improve the condition of mankind, or the problem appreciation by decision makers. purported knowledge has to compete with other claims of knowledge, so a “popperian” measure might simply be how well it out-argues opposition knowledge claims. a “churchmanian” measure is whether the research provided people with more alternatives, and whether it altered people’s actions. interpretive systems research needs to identify and have faith in its own measures of inquiry success and stop its rather sad pretence of “self flagellation” to appease journal gatekeepers. references alvesson, m. & skoldberg, k. (2000) reflexive methodology, california: sage arygris, c., (1996) “actionable knowledge”, journal of applied behavioural science, 32(4), pp 390-406 baskerville, r. & lee, t. (1999) “distinctions among different types of generalizing in information systems research,” in ngwenyama, o. & degross, j. i. (eds), new information technologies in organizational processes: field studies and theoretical reflections on the future of work, boston: kluwer academic, pp 33-54. checkland, p. (2000), “soft systems methodology: a thirty year retrospective”, systems research and behavioural science, vol 17, number 1, pp s11-s58 cohen, h.f. (1994) the scientific revolution, university of chicago press. churchman, c. w. (1971) the design of inquiring systems, wiley, new york haynes, j. (2000) perspectival thinking, nz: oneidea company kuhn, t. s. (1970) the structure of scientific revolutions, 2nd ed (enlarged), university of chicago press linstone, h. a. (1999) decision making for technology executives : using multiple perspectives, artech house publishing: boston metcalfe, m. & lynch, m. (2002) “arguing for is design”, ifip working group 8.2, barcelona, december metcalfe, m. (2002) “multiple methods for critiquing articles”, working paper university of south australia {mike.metcalfe@unisa.edu.au} morgan, g. (1986) images of organisations, calif.: sage publications perelman, ch. & olbrechts-tyteca, l. (1969) the new rhetoric: a treatise on argumentation, university of notre dame popper, k. r. (1963) conjectures and refutations, london: routledge and kegan paul popper, k. r. (1971) the open society and its enemies, new jersey: princeton university press. rehg, w. (1999) “argumentation theory and the philosophy of science since kuhn”, inquiry, summer. ulrich, w. (1983) critical heuristics of social planning: a new approach t practical philosophy, chichester: john wiley & sons ulrich w., (2001) “a philosophical staircase for information systems definition, design and development: a discursive approach to reflective practice in isd (part 1)”, jitta 3(3), pp 55-84 van eemeren, f.h., grootendorst, r. & kruiger, t., (1987) handbook of argumentation theory, dordrecht: foris publications. walton, d., (1998), the new dialectic, toronto university press paper title (paper title style) australasian journal of information systems gregor & richardson 2016, vol 20, selected papers from the information systems foundations conference (isf) editorial selected papers from the information systems foundations conference (isf) shirley gregor research school of management college of business and economics australian national university alex richardson research school of management college of business and economics australian national university editorial the papers in this section of volume 20 emerged from contributions to the 2014 information systems foundations (isf) workshop held at the australian national university on 2-3 october 2014. this workshop was the seventh in a series, begun in 1999 by kit dampney at macquarie university and continued biennially from 2002 at the anu, focussing on the theoretical foundations of the discipline of information systems. the theme of the workshop was “the value and evaluation of information and related theories and theorising in the discipline”. following their selection, the authors of the workshop papers developed them into full journal articles that were subjected to ajis’ usual double blind peer reviewing. the papers explore theoretical concerns as they relate to information systems. the use of systems theory and the principle of emergence are combined to examine the question of business value; and the use of facet theory to guide conceptualization to improve construct clarity. these are both fundamental issues in the justification and design of information systems and should make a useful contribution to further empirical research. thanks are due to all the isf workshop participants, who as usual engaged in lively discussion of all the nine papers presented at the workshop. the audience also enjoyed the keynote speech by ann steward, consultant and former australian government cio (2005-2012). the assistance of colleagues in the business information systems group at anu and our review teams are gratefully acknowledged. copyright: © 2016 gregor & richardson. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. 1 http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems gregor & richardson 2016, vol 20, selected papers from the information systems foundations conference (isf) editorial 2 paper title (paper title style) australasian journal of information systems duarte 2017, vol 21, research on indigenous ict connected activism 1 connected activism: indigenous uses of social media for shaping political change marisa duarte arizona state university marisa.duarte@asu.edu abstract prior studies describe digital tactics as specific strategies actors apply within broader repertoires of contention, specifically in social and political contexts. a comparison of ezln, idle no more, and the ongoing rio yaqui water rights movement reveals the kinds of community knowledge work that has to happen prior to and around activating digital tactics in indigenous rights movements, including choices in messaging and discourses of indigeneity, targeting of movement opponents, and selection of digital tools and techniques. activists harness these communicative affordances to practice a politics of visibility, cultivate solidarity, diffuse an indigenous consciousness, enforce dominant governments’ trust and treaty responsibilities, and remind many of the irrevocable injustice of colonialism. designing methodologies that account for specific indigenous social and political contexts as well as the affordances of various digital environments is part of the future work of indigenous media theorists. keywords: social media; indigenous peoples; methodology; activism; social movements 1 introduction indigenous peoples utilise social media for a number of reasons—from the utilitarian, to the political, to the politically transcendent. i am most interested in those studies that reveal the digital tactics and strategies that destabilise colonial power and hopefully, decolonise. discerning these strategies requires a distinct intellectual project. this project would, at an epistemic level, account for the real-time exigencies of indigenous peoples in specific geopolitical locations while also retaining sensitivity for the ways indigenous actors conscript networked digital systems, devices, and web platforms—especially sns (social networking sites)—to satisfy everyday life information behaviours. applying such a framework to three cases of indigenous social movements online—ejercito zapatista de liberación nacional (ezln) activism, idle no more, and rio yaqui water rights advocacy—reveals that while sns are far from being egalitarian participatory democratic spaces for indigenous peoples, they offer a compelling set of communicative affordances. activists harness these communicative affordances to practice a politics of visibility, cultivate solidarity, diffuse an indigenous consciousness, enforce dominant governments’ trust and treaty responsibilities, and remind many of the irrevocable injustice of colonialism. i argue that understanding indigenous digital repertoires of contention from an indigenous perspective allows us to not only decipher the significance of political and social connections forged via digital tools, but also helps us to understand how particular digital tactics relate to the needs and goals of particular indigenous movements. 2 framing indigenous uses of social media there is a thread in the literature around indigenous uses of digital technologies that asserts the following colonial logic: indigenous peoples as canaries in the cage of modernity suffer the onslaught of neoliberal technologies and therefore digital technologies are socially detrimental for both indigenous and non-indigenous peoples. these arguments hinge on one of four colonial and neoliberal desires: 1) the desire for indigenous ways of life to serve as a salvo for the pathos, alienation, and uncertainty of a contemporary networked social order; australasian journal of information systems duarte 2017, vol 21, research on indigenous ict connected activism 2 2) the desire for fresh stores of indigenous knowledge (ik), also coded through intellectual property institutions as traditional knowledge (tk) and traditional ecological knowledge (tek); 3) the desire to position indigenous peoples as a straw man for another unarticulated argument about the totalitarian (or genocidal) potential one (someone? anyone?) might be afforded through orchestrated conscription of digital networked technologies, and/or; 4) the fear of a viral indigenous cyber-terrorism. as indigenous scholars examining social media, we have a responsibility to interrogate and correct the colonial logics at play here. we conscientiously weave together insights garnered from general internet studies with insights from indigenous scholars. we choose which epistemologies to foreground and when. these epistemic approaches may come from the foundations of science, technology, and society studies, or they may be grounded in specific indigenous experiences, such as when subcomandante marcos (2002) describes building networks ‘from below’, two horses (1998) describes gathering around the electronic fire, or waitoa, scheyvens & warren (2015) explains e-whanaungatanga (de leon, 2002; two horses, 1998; waitoa, scheyvens & warren, 2015) ultimately i hope that we are able to discern the digital repertoires of contention (tarrow, 1998) that help us destabilise oppressive regimes, decolonise our own communities and practices, and create the conditions for healthier forms of governance and diplomacy. recent literature suggests that social movements online follow one of three approaches— collective action, collective action with connective capacity, and connective action. (earl and kimport, 2013) collective action movements are largely organised off-line, and may use digital tactics to support on-the-ground tactics, such as collecting signatures for petitions. organisers in these kinds of movements often do not leverage social media to mobilise support for campaigns, persuade publics, or disseminate key messages and discourses. when collective action organisers do leverage social media in these ways but still retain a brick-and-mortar institution, they increase the political capacity of their movement over all. distinct from the dynamics of brick-and-mortar institutions, many actors convening across many sns form the emergent and ephemeral dynamics of connective action movements. the digital tactics employed there may or may not map over to institutional participatory political processes, such as voting, voter registrations, and campaign fund donations, yet they provide spaces for collectively imagining alternative social and political visions, naming social phenomena, processing tragedy and trauma, and correlating personal experiences to social and political forces. indigenous peoples by definition regularly lack a direct means of political participation via local and state or provincial government agencies—for example, through voting, lobbying in competitive numbers, and campaign donations. sns become compelling mobilising grounds for indigenous connective activism. individuals who may not have imagined themselves as ‘political’ in the sense of being an elected official or direct action protestor post and re-post politicised images, phrases, narratives, and survival stories within their digital networks of belonging. these images and phrases contribute to the construction of specific activist causes such as #saveoakflat or #sosblakaustralia, as well as to broader imaginaries about both meaningful indigenous survival and quality-of-life amid the crush of neoliberal social and political rules. (mercado, 2015; sweet, pearson & dudgeon, 2013) in this sense indigenous activist uses of social media comprise one tool in the greater array of indigenous repertoires of contention. the digital tactics that indigenous activists employ to achieve specific political goals become part of the greater interplay of forces shaping placebased decolonisation efforts. prior studies note how marginalised groups’ social media discourses strengthen in-group solidarity, raise awareness of neoliberal forces—classism, types of racism, poverty, immigration processes, etc.--shaping the lives of marginalised peoples in different locations, and provide individuals with the speech tactics needed to navigate tricky political moments. (mercado, 2015; clark, 2014) crafting social media studies that frame the australasian journal of information systems duarte 2017, vol 21, research on indigenous ict connected activism 3 local indigenous political context can help us better understand the kinds of impacts that specific digital tactics have. this also raises the level of academic discourse about indigenous uses of digital technologies from one that romanticises notions of the anti-technological or premodern indigenous subject to one that positions indigenous connective activism as a decolonisation strategy. from there we can begin to conceptualise indigenous activist uses of social media as both digital tactics and also as a way of knowing. 2.1 three indigenous social movements online in the mid-1990s, castells (1997) predicted that the circulation of ideas and the material constraints of the network society would afford identity-based movements with the potential to raise awareness of their exigencies, and gather resources to stage local coups. castells was in part inspired by the zapatistas’ 1994-1995 savvy uses of the web to acquire support for their claims against the mexican state. since then, many indigenous movements have utilised web platforms for political causes. thinking about how to layer and interweave digital network effects into the existing land tenure, political, economic and social constraints shaping indigenous social movements requires a distinct intellectual project. it means identifying conceptual frameworks that allow for indigenous explanations of the particular geopolitical exigencies shaping local social movements, in addition to identifying sociotechnical frameworks that explain digital tactics within a greater repertoire of contention. it also means thinking about the impacts and effects of indigenous social movements in real-time but also across relevant webspheres, such as the blogsphere, facebook, vlogs, etc. figure 1. framing digital indigenous social movements indeed, taking this approach into account reveals the distinctive features of a range of digital indigenous social movements. a comparison of ezln, idle no more, and the ongoing rio yaqui water rights movement reveals the kinds of community knowledge work that has to happen prior to and around activating digital tactics. it reveals constraints around the kinds of messaging and discourses of indigeneity that become part of certain movements, but not others. it also reveals how the target of select movements in turn has an effect in shaping ascription to certain kinds of indigenous discourses. this is meaningful in that it helps us better appreciate the nuanced nature of indigenous activism and political participation as challenges to indigenous autonomy and political participation manifest through different social, political, and economic structures in different countries. 2.2 ejercito zapatista de liberacion nacional (ezln) in the late 1980s, a group of urban marxist intellectuals visited mayan communities in the chiapas highlands. their goals were to foment a land-based proletariat struggle. however, meeting and living with mayan community and frente de liberacion nacional (fln) guerrilla leaders helped them realise the conditions shaping political, social, and economic life in chiapas. the marxists had not taken into consideration the historicity behind mayan philosophies and goals, goals designed to strengthen resilience within the chiapas mountain homes and fields. over time, mayan leaders and area liberation theologians worked with the social movement geopolitical exigencies impacts and effects digital tactics australasian journal of information systems duarte 2017, vol 21, research on indigenous ict connected activism 4 marxists, exchanging ideas until they formulated the foundations of what would become the ezln. while the initial goals were to protest and through direct action challenge the unjust policies of the neoliberal and technocratic mexican government, in particular as these were manifesting under the aegis of the 1994 north american free trade agreement (nafta), within a year, after a short-lived violent series of battles and cease-fire, the ezln shifted the battle to a war of ‘ink-and-internet.’ (froehling, 1997) their digital discursive strategies proved so effective in such a short amount of time that movement opponents would for years thereafter identify the zapatistas with a challenging sociological neologism: cyber-terrorism. the ezln is widely cited as the first social movement to effectively use digital tactics for collective organising. at first, their digital tactics included use of a static website as well as electronic mailing lists, or listservs. the enigmatic spokesperson subcomandante insurgente marcos utilised these platforms to disseminate ezln communiqués. ezln supporters, inspired by the anti-neoliberal messaging, indigenous philosophical wit, and calls for a radical new hope promoted the site from their own webpages, and forwarded ezln emails. by the time myspace launched in 2003, the ezln had already shaped the networked websphere, with ngos connecting to each other by virtue of shared connections with the ezln. (garrido & havalais, 2003) organisations and individuals donated cash, volunteer time, foodstuffs and other material goods to the emerging caracoles and centros de gobierno that the ezln initiated after negotiating with the mexican government for a zapatista autonomous zone. in time, many ezln and ally groups popped up in myspace, facebook, via youtube and other sns. zapatistas used streaming radio, email, and other digital modes for communicating under duress. through the 2000s, allies in different countries—including groups as diverse as rage against the machine, wto protest organisers, and indymedia—referenced zapatista logics, discourses, and lessons ‘from below.’ scholars identified the paradox of the anti-neoliberal zapatistas conscripting the very globalising technologies of neoliberalism—social media circuits and consumer web platforms—to distribute powerful communiqués while acquiring in turn political support in their indigenous campesino “war against forgetting.” (wilson, 2007) scholars first approached the zapatistas from a range of vantage points that stretched disciplinary logics and explanatory principles. they described zapatista political phenomena as performative at times, adopting language from cultural studies to identify digital revolutionary aesthetics and mimesis. at other times scholars tested rational choice explanations of guerrilla politics against emerging post-structural explanations of social movements and political change, or alternatively used the case of zapatismo to ground theories about peasant rebellions. while chicana/o studies scholars recognised a familiar approach to mexican-state indigeneity, it took a while for some strictly us and canada-based native studies scholars to allow for this new brand of indigenous discourse to shape what would eventually become a stronger native and indigenous critique of neoliberal and neo-colonial approaches to globalisation and settlement. far from the physical site of violence, endurance and protest, many scholars overemphasised the network effects of this ‘cyber-warfare.’ zapatista comandantes theorised a direct relationship between mexican neoliberal trade policies, past and present colonial injustices, and indigenous campesino rights to self-govern. while ezln communiqués did reference specifically indigenous visions of global democracy, on-the-ground goals included the design of caracoles and centros de gobierno that often operated via a consensus based approach to governance rather than euro-american participatory democratic approaches. tactics of using masks and the guerrilla practice of switching bravos and pacíficos—or the front-line rebels and peaceful campesinos—also emerged from a need for hierarchical military tactics. a keen understanding of the corrupt nature of the mexican government’s presidential elections and enforcement of court rulings also prevented ezln authorities from embracing statist approaches. thus when we consider the real-life political, social, and economic exigencies of the zapatistas and neighbouring indigenous campesinos in chiapas, we can better understand how zapatista conscription of sns and other digital tactics perhaps did less to drive recognisable democratic participation mechanisms, such as voter registration, campaign fund donations, and lobbying to support widespread participation in a zapatista-led freirian australasian journal of information systems duarte 2017, vol 21, research on indigenous ict connected activism 5 concientizaçao, a transborder transnational grassroots anti-neoliberal agenda-setting, and a multi-point peer-to-peer digital transmission of an indigenous campesino vision of regional peace-making and economic stabilisation in spite of extreme poverty and disenfranchisement. 2.3 idle no more over fifteen years after the ezln shifted to a war of ‘ink-and-internet,’ first nations peoples at the canada-us border were experiencing the government condoned corporate natural resource extraction and border violence against native and indigenous women that now characterise indigenous experiences of nafta. in 2011, the then canadian prime minister stephen harper drafted a series of bills and amendments divesting first nations sovereign command of lands and waterways. the party’s goal was to support the extraction of oil from various tribal lands and border zones, including the environmentally detrimental installation of pipelines such as the keystonexl pipeline, highway expansion, and associated infrastructural projects. crafted into a single large omnibus bill, the assembly of first nations leadership had little time to respond. furthermore, many first nations environmental justice and indigenous rights activists had reason to be skeptical of the afn’s political and economic clout. like any elected officials, tribal elected officials can on occasion buy into corporate ventures that they hope will strengthen tribal enterprise, but that can at the same time divest tribes of their control over natural resources and legal rights to access those resources. (coulthard, 2014) this is precisely why indigenous participatory politics manifests in tactics and strategies that can sometimes culminate in the formulation of distributed social movements. in the autumn of 2011, three native women activists and a non-native feminist environmental justice activist crafted a series of teach-ins as well as a web campaign that would appeal to indigenous activists and environmental justice supporters. they named the movement idle no more. (mcmillan, young & peters, 2013) digital tactics included flash mob prayer rallies at locations defining neoliberal reach, including highways, busy city intersections in financial districts, shopping malls, and border crossings. they also developed image macros and supported the use of facebook groups and twitter hashtags, allowing activists in many different places around the world to re-post, share, re-tweet, and index idle no more. the rapid and widespread dissemination of information and summaries about happenings around idle no more-including the text of the c-series of bills, legislative changes, lobbying groups, and the status of chief theresa spence’s (attawapiskat) hunger strike--allowed educators and journalists in many places to host teach-ins and write independent media stories. while facebook newsfeeds and twitter accounts revealed a string of protests in both the us and canada through december and january of 2012 and 2013, the major media outlets such as cnn, the bbc, and the cbc did not pick up on spread and intensity of idle no more protests until well after activist groups had self-organised into diffuse action-ready coalitions. similar to the ezln, idle no more activists had a very clear message for the top executive branch leader of the country. while the ezln in the early days demanded a zapatista autonomous region, and an end to specific violent neoliberal trade policies and actions in chiapas, idle no more activists demanded recognition of first nations treaty rights in their lands and waterways, as well as the total repudiation of the proposed conservative party amendments to the indian act. however, idle no more activists made sure to distinguish between their goals and afn goals, precisely because of the likelihood of afn cooptation by the canadian government and private interest ventures. likewise, idle no more activists crafted calls for specific participatory actions—protests, ‘no’ votes, petitions, etc.—within a greater discourse of anti-neoliberal environmental justice, global indigenous rights, and most especially, justice for missing and murdered aboriginal women and the health and wellbeing of aboriginal women and children. these interlaced discourses contributed to the networked discourse of global indigenous solidarity perhaps initiated in the websphere by zapatista activists many years before. digital solidarity for idle no more emerged through maori facebook groups, uk-based environmental justice groups, indigenous feminist groups, and in many places far from the australasian journal of information systems duarte 2017, vol 21, research on indigenous ict connected activism 6 canadian border states and first nations homelands. from within first nations tribal circles, idle no more efforts revealed the distinctions between the modes of effective political participation afforded to aboriginal women and the mostly male elected officials of the afn. (john, 2015) while chief theresa spence would experience a great deal of criticism for waking sleeping giants, a few years after the first idle no more flash mobs president obama vetoed the construction of the keystonexl pipeline, and canadian prime minister harper definitively lost his second election run. the senate tabled the proposed amendments to the indian act in 2013. meanwhile, idle no more digital activism continues to bring awareness to issues around environmental justice, such as the recent court order for shell canada to pay over $800,000 to assist with the clean-up and restitution of a spill bordering aamjiwnaang first nation. as with the early efforts of the ezln, it has taken a while for scholars to decipher idle no more’s network effects in particular with regard to political participation not only in canada but also with regard to the afn’s parliamentary relationship. what is clear, however, is that idle no more represents a case of mediatised political participation across a diffuse and flexible feminist grassroots leadership not only in canada but also in the canada-us border states. idle no more reveals the limitations of participatory democracy as it plays out in tribal council chambers, through council meetings and elections, the afn, and the canadian parliamentary process. it also revealed how indigenous activists and allies in many different places and social positions orchestrated assemblages of sns, devices, various web platforms, and independent media channels to organise quickly and effectively, circulating messages, memes and actions that destabilised colonial efforts across first nations lands. 2.4 rio yaqui water justice idle no more and ezln reveal both the limitations and the stakes of activist uses of social media to further indigenous social movement goals. on the one hand, strategic uses of websites, independent media communiqués, listservs, sns, digital streaming radio, vlogging, image macros, and memes can contribute to raising awareness of the particular exigencies around indigenous social movement goals. conversely, as the zapatistas know, concientizaçao does not only occur in the world of ‘ink-and-internet.’ the very nature of indigenous decolonisation movements of all stripes requires violent upheaval, the taking up of arms, witnessing bloodshed, and willingness to destabilise oppressive government efforts at pacification. as idle no more feminist activists know, widespread awareness and understanding of social movement goals does not mean that tribal elected officials or federal elected officials will listen to or respond to activist demands. finally, there is the concern about how to shape solidarity for indigenous social movements through online spheres, in particular as on-the-ground political realities reduce opportunities for healthy and productive participation of allies. the ongoing rio yaqui water rights movement in northwest mexico reveals some of these dimensions as well. for centuries, the spanish agricultural and eventually mexican government industrial authorities appraised the rio yaqui valley and underlying aquifers as the key to national agricultural settlement and economic development. (radonic, 2015) surrounded by rocky desert highlands and saline coastlines, the yaqui river valley is the sacred homeland of the yaqui people. the yaqui people are the only indigenous people in mexico to have achieved something akin to reservation status by the mexican government, and have been rudely named the only indigenous people who were not conquered in mexico. such a dubious naming is not without the weight of history. in 1898, the mexican government initiated a genocidal campaign against all yaqui people, resulting in significant loss of life and leadership for the eight pueblos and the exile and enslavement of many yaqui families. the campaign laid the foundation for the mexican government’s regular practice of treaty-breaking, privileging of private corporate settlement and natural resource extraction in yaqui lands and waterways and similar to what is happening at the canada-us border, criminal lack of attention to the environmental and physical health and wellbeing of yaqui women, children, and families. while the governors of the eight yaqui pueblos and the sonoran governor frequently negotiate uneasy peace agreements, the mexican state continues to condone agricultural and industrial australasian journal of information systems duarte 2017, vol 21, research on indigenous ict connected activism 7 settlement of yaqui lands without abiding by proper consultation processes, and sometimes through military force. thus in 2010 when a group of young yaqui men posted a youtube video of mexican military personnel beating them for filling barrels of the only potable water in the rio yaqui for miles, the story caught fire. environmental justice activists, yaqui youth, anti-militarisation activists, anti-neoliberal activists, indigenous rights activists, human rights activists, and independent media journalists had been watching the unfolding story of the illegally constructed novillo dam, the independence aqueduct and its effects in rio yaqui. additional dams were nearing completion in spite of environmental impacts reports that clearly noted high quantities of pollution by private land developers, industrial engineering companies, and food companies such as bimbo breads. activists noted how the nestlé company was pumping hectares of water from dams—intended for purification and bottled water sales in cities--even though none of the eight pueblos had potable water, and local residents could not even eat the vegetables grown from non-potable water due to high industrial waste contaminants. (jaime leon, 2015; tortuga azul and herrera rivas, 2015) concerned activists developed coalitions between such campaigns as the no al novillo movement, the #justiciayaquis movement, and the# namakasia movement, all of which utilised digital tactics to address layers of injustices. over time, digital tactics included use of email listservs to organise and share news, use of facebook groups, youtube channels, and twitter feeds to distribute communiqués from the yaqui governors in sonora, development of static webpages, participation in independent media programs, and organisation of border and interstate highway blockade flash mobs and city protests similar to idle no more, with the distinction that the rio yaqui activists asked long-haul truckers and tractor drivers to park their vehicles as blockade posts, rather than using human chains and prayer rallies. documentary filmmakers began developing short projects, some of which are at the end of production and are being distributed through community viewings in the border states. digital tactics have been steadily mounting since 2010, garnering popular support as well as the attention of greenpeace activists and united nations investigators. two different groups in the yaqui autonomous region filed lawsuits against the state of sonora. one group filed on behalf of the tribe, noting how the state and private development partners began building dams and associated infrastructure without proper legal consultation with the eight pueblos. another group, concerned that that lawsuit would not result in an expedient impact, filed a separate suit in partnership with corporate land developers. meanwhile, mexican authorities took two of the lead activists into custody and detained them for an entire year, releasing evidence that one of the activists may or may not have been arrested not for civil disobedience but rather, for a series of unrelated criminal charges. activists began to be concerned that un investigations would privilege human rights solutions over indigenous rights approaches, leading to increasing theft of rio yaqui water from the peoples’ sacred and legal homelands. while the messaging via sns has shifted and spread, and with some activists fearful of being surveilled by mexican government authorities, the pressure to disseminate news about this latest round of low-intensity warfare in and around yaqui territories in sonora is increasing. while it is clear that neoliberal policy dynamics are shaping illegal rio yaqui water extraction and pollution in a manner similar to events surrounding the proposed canadian oil pipelines, rio yaqui activist uses of social media differ from those of idle no more activists. this is in part because the processes and techniques that afford indigenous peoples’ political participation in mexico differ from the processes and techniques in canada. while rio yaqui activists are working with larger indigenous activist networks in mexico, the governors of the eight yaqui pueblos initially signed peace accords with the governor of sonora, rather than at the federal level with mexico. this shifts the locations of contestation for rio yaqui activists. the spanish/english language divide also has an effect in the reach and dissemination of news and discourses about rio yaqui water rights. other than in arizona, few us and canada indigenous activists are aware of the rio yaqui justice efforts. the distinct but potentially coalitional #idlenomore and #namakasia movements are not referencing or indexing each other. australasian journal of information systems duarte 2017, vol 21, research on indigenous ict connected activism 8 the split is not only linguistic but also cultural as references to indigeneity differ in these border regions. for example, the yaqui people of sonora are known for distinguishing themselves politically from the zapatistas. in the early days of forming the zapatista autonomous zone, the ezln requested the support of yaqui guerrilla fighters from sonora and from the yaqui tribe in the united states. while some men did join the ezln, many chose not to under the advice of elders who cautioned them that the zapatistas would eventually give mayan land to rural campesinos who had no interest in supporting yaqui ways of life. elders reminded yaqui youth then, as they do now, that yaquis are only yaqui when speaking yaqui and living in connection with the yaqui sacred homeland. in that sense, rio yaqui water rights movement discourses of indigeneity are not always intended to be as encompassing as ezln discourses of global indigenous solidarity, nor are they as transferrable to associated causes as idle no more discourses around women’s rights. the set of goals and indigenous discourses surrounding the current #namakasia movement and associated rio yaqui justice movements are so precise and specific that in many ways, the activist uses of social media are likewise more pointed and oriented to address specific structural violence and sectors of the population within sonora. the goals are not like that of the ezln, wherein subcomandante insurgente marcos inscribed years of visions about a global indigenous solidarity, an indigenous space for a global participatory democracy, and anti-neoliberal practice through a series of cybercommuniques. the goals are more akin to idle no more, whose leadership were employing digital tactics to both spread awareness about a specific policy issue and set of practices while also distinguishing themselves from the political participation methods advocated through the afn. in rio yaqui, the rio yaqui justice movements digital tactics match the geopolitical terrain, where activists must on one hand support the yaqui-only leadership of the eight pueblos within the autonomous zone while also fostering regional solidarity among a diverse number of actors, from landowners, to activist mexican youth, proletariat workers and campesinos as well as the middle-class environmental justice activists, independent journalists and other fierce critics of the sonoran governor. 2.5 indigenous uses of social media: visibility, solidarity, consciousness, trust, and the reminder of irrevocable injustice by the time mexico, the u.s. and canada signed nafta, the ezln had already consolidated an internal political vision, developed modes of governance that fused mayan philosophy and know-how with urban marxist and liberation theology approaches, and practiced a land-based mode of sustenance in the mountains of chiapas that would ultimately ground the indigenous campesino autonomy movement for many years to come. the ways that zapatistas disseminated narratives and communiqués about their political exigency via the internet inspired leftist activists in many parts of the world (wolfson, 2012). speaking through the voice and public image of subcomandante marcos, many members of the ezln expressed the need for a global information network through which indigenous peoples and many others seeking liberation from the yoke of neocolonial and neoliberal government policies could connect with each other, establish solidarity, and continue through direct action the fight ‘from below’ (ponce de león, 2002; ruggiero, 1998). since that time the leaders of many leftist activist organisations, independent journalists, and media theorists have studied ezln media activism to learn about the ways network approaches amplify social movement opportunities for participatory action. a fascination with indigeneity and the relationship between social media and political organising began to shape arguments in social theory literature. lacking insight into the nuances of indigenous peoples movements and multiple forms of governance, and also assuming that more voice and visibility naturally lead to more opportunities for democratic participation, social movement theorists and media theorists alike conflate social media participation with democratic participation. (carpentier, 2011; waller, dreher & mccallum, 2015; wolfson, 2012) leftist activists also regularly conflate indigeneity with anti-neoliberalism and anti-racism among other agendas. many people presume that all indigenous peoples have political goals and visions akin to subcomandante marcos. australasian journal of information systems duarte 2017, vol 21, research on indigenous ict connected activism 9 however, activists, scholars and journalists who joined subcomandante marcos’ in the struggle ‘from below’ in chiapas learned that non-democratic forms of governance, subsistence living, mayan philosophies, campesino practical knowledge, limited capital means, and restricted political participatory mechanisms shaped the struggle in specific landand flesh-based ways. (wolfson, 2012) the indigenous aspects of online social movements became concrete, material, real, undeniable, muddy, bloody, uncomfortable, painful, lived, embodied, traumatic, and transformative. this is probably best exemplified by the experiences of those who survived the 1997 massacre of acteal, in which a paramilitary group with alleged connections to the ruling political partido revolucionario institucional, or pri, shot and brutalised 45 people, including women and children, who were praying in a catholic church in chiapas. those who died were members of las abejas, a pacifist group who supported the ezln and their use of nonviolence. beset by colonial and neoliberal logics, political influence, and lack of evidence, the court case continues to haunt the survivors, as well as the mexican justice system. thus when as social theorists we consider the relationship between indigenous uses of social media and indigenous peoples opportunities for political participation, we must centre that which is distinctively indigenous. in other words, we cannot safely presume an association between indigenous activist uses of digital media and fair and just governmental recognition of indigenous peoples’ rights. at present, indigenous peoples are indigenous not simply as a matter of neoliberal identity politics, but rather because they are the necessary ‘other’ to the formation and reproduction of modern nation-states, especially settler-states. the authorities of nation-states, even those most generous with regard to multicultural liberal philosophies, find it difficult to undo generations of social and legal policies that prevent indigenous peoples—as opposed to atomistic individuals--from direct meaningful engagement with nation-state authorities, dominant political parties, and state and provincial representatives. the kinds of political participation that the state affords indigenous peoples are already circumscribed in such a way as to prevent dismantling of the state’s economic neoliberal colonial apparatus. mass media and corporate news outlets may occasionally include or represent native voices in a turn to a politics of representation, but this is not equivalent to, nor does it inherently lead to, the consent of the governed in an informed democracy. (dreher, mccallum & waller, 2016) as researchers we can ask, what happens when indigenous people’s uses of social media allow specific indigenous political actors to move in, around, and through neoliberal power structures? additionally we have to foreground the very material challenges of indigenous life in particular because social media and internet studies scholars hailing from technologically advanced countries may have a very limited understanding of indigenous peoples’ political and social conditions. for example, at present activists working within the eight yaqui pueblos located in northwest mexico are organising a transnational indigenous water rights campaign through social media, and yet none of the pueblos have potable water. many of the women who are leaders of local idle no more efforts are working through various family and community crises as the increasing rates of missing and murdered indigenous women compound existing high rates of domestic violence and abuse. essentially we have to remind our non-indigenous colleagues that to nation-state authorities, forms of violence in indigenous communities is considered allowable and to be expected, a necessary sacrifice in the long march to neoliberal progress and “democracy for all.” we also have to continually conceptualise how indigenous peoples uses of social media are grounded in not just contemporary political exigencies, but also in the tribal philosophies, spiritualities, traditions, and historical legacies of peoples with memories reverberating through eras well before the founding of modern nation-states. on many occasions, indigenous leaders’ political goals may be in direct counterpoint to many tenets of contemporary eurocentric forms of democratic political participation. thus, as recent research in political uses of black twitter reveals that participants have to ‘be black’ to meaningfully engage in the black blogsphere (clark, 2013), i argue that if as social theorists we seek to understand indigenous online spheres, we have to be indigenous, able to meaningfully engage, decipher, and catch on australasian journal of information systems duarte 2017, vol 21, research on indigenous ict connected activism 10 to the coded language, turns-of-phrase, significance of political connections, and deeper philosophical meanings embedded in indigenous uses of social media. finally, we have to remember that indigenous people’s uses of social media will cohere to the specific political demands of specific indigenous groups in specific geopolitical locations. taking that perspective allows us to align with indigenous theory and research that discerns, envisions, and articulates the functionality of decolonising forms of governance, political economies that defy neoliberal exploitation, and customary laws grounded in the wisdom of time-tested tribal philosophies and worldviews. comparison of the abovementioned movements reveal how indigenous uses of social media support visibility of indigenous social movements and issues, promote solidarity for particular struggles and views, foment freirian processes of consciousness-raising, and enforce the government-to-government trust underlying peace agreements and treaties. indigenous uses of social media also intentionally disrupt and destabilise participatory processes of oppressive governments. messaging and discourses produce community unsettling and factionalization that in many ways is a reminder of the layers of irrevocable injustice that indigenous peoples have suffered in the making of neoliberal nation-states. indigenous uses of social media are thus inherently destabilising for dominant government processes. this is not a bad outcome; rather it reveals the further work we must do as scholars seeking to understand and respond with good heart to the social and political exigencies of indigenous peoples. 3 conclusion in 2001 garrido and havalais (2003) conducted a network analysis of the zapatista movement in southern mexico, looking for patterns around their working hypothesis: that the identitybased ezln was shaping flows of information around justice-based causes in the websphere. their analysis revealed the degrees of connectivity—nodes, distance, and association—between the ezln website and those of ngos affiliated within the ezln web network. at the time of the analysis, many network scientists had not yet embraced the ‘rough edges’ (munster, 2013) of network analysis and conceptualisations, where links may not be equivalent to two-way reciprocal flows of information, and where flows of information may or may not be equivalent to the real-life flow of material resources. beyond the clean linearity of network diagrams the messiness of life emerges: the human condition as it bleeds, so to speak. since the identity-based indigenous media studies of the late nineties, researchers devoted to understanding the relationship between indigenous uses of social media and indigenous political goals have noted what community leaders have known for a long time. the expression of strong mediatised indigenous voices does not ensure that dominant government authorities will accordingly listen or act in a just manner. effective political participation for indigenous peoples vis-à-vis nation-states still requires significant structural changes. meanwhile, indigenous activists, entrepreneurs, educators, and many other leaders must effectively and strategically push however they can from whatever digital/social/political position they hold, the embodiments of decolonisation and perpetual performers of radical change. references carpentier, n. (2011) media and participation: a site of ideological struggle, intellect ltd, bristol. clark, m. (2014) to tweet our own cause: a mixed-methods study of the online phenomena ‘black twitter,’ [doctoral thesis] university of north carolina at chapel hill. coulthard, g. (2014) red skin, white masks: rejecting the colonial politics of recognition, university of minnesota press, minneapolis. dreher, t., mccallum, k. & waller, l. (2016) “indigenous voices and mediatised policy-making in the digital age,” information, communication, and society, 19(1):23-39. australasian journal of information systems duarte 2017, vol 21, research on indigenous ict connected activism 11 earl, j. & kimport, k. (2013) digitally enabled social change: activism in the internet age, mit press, cambridge. froehling, o. (1997) “the cyberspace ‘war of ink-and-internet’ in chiapas, mexico,” the geographic review, 87(2): 291-307. garrido, m. & halavais, a. (2003) “mapping networks of support for the zapatista movement: applying social-networks analysis to study contemporary social movements.” in mccaughey and ayers, eds. cyberactivism: online activism in theory and practice. london: routledge. john, s. (2015) “idle no more: indigenous activism and feminism,” theory in action, 8(4): 38-54. jaime leon, j.s. (2015) “film discussion: mover un río,” yaqui pride community screening: mover un río, november 5, 2015, scottsdale, arizona. mccallum, k. (2012) the media and indigenous policy: how news media reporting and mediatised practice impact on indigenous policy, university of canberra, canberra. mcmillan, l.j., young, j. & peters, m. (2013) “commentary: the ‘idle no more’ movement in eastern canada,” canadian journal of law and society, 28(3): 429-431. mercado, a. (2015) “medios indígenas transnacionales: el fomento del cosmopolitanismo desde abajo,” comunicacion y sociedad, 23: 171-193. munster, a. (2013) an aesthesia of networks: conjunctive experience in art and technology, mit press, cambridge. ponce de león, j. (2002) our word is our weapon: selected writings of subcomandante insurgente marcos, seven stories press, new york. radonic, l. (2015) “environmental violence, water rights, and (un)due process in northwest mexico,” latin american perspectives, 42(5): 27-47. ruggiero, g. (1998) “second declaration of la realidad: closing words of the ezln at the intercontinental encuentro for humanity and against neoliberalism,” in ruggiero, ed. zapatista encuentro: documents from the first intercontinental encuentro for humanity and against neoliberalism, seven stories press, new york. sweet, m., pearson, l. & dudgeon, p. (2013) “@indigenousx: a case study of community-led innovation in digital media,” media international australia, 149: 104-111. tarrow, s. (1998) power in movement: social movements and contentious politics, cambridge university press, cambridge. tortuga azul (producer), rivas, a.h. (director). (2015). mover un río [motion picture], ittac films, mexico city. two horses, m. (1998) “gathering around the electronic fire: persistence and resistance in electronic formats,” wicazo sa review, 13(2): 29-43. waitoa, j., scheyvens, r. & warren, t.r. (2015) “e-whanaungatanga: the role of social media in maori political empowerment,” alternative 11(1): 45-58. waller, l., dreher, t. & mccallum, k. (2015) “the listening key: unlocking the democratic potential of indigenous participatory media,” media international australia, 154: 5766. wilson, j. (2007) “zapatista challenges: neoliberal governmentality and technologies of self”, paper presented at the national communication association 93rd convention, chicago, illinois. wolfson, t. (2012) “from the zapatistas to indymedia: dialectics and orthodoxy in contemporary social movements,” communication, culture, and critique, 5: 149-170. australasian journal of information systems duarte 2017, vol 21, research on indigenous ict connected activism 12 copyright: © 2017 duarte. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems mckinley, ritchie, burmeister, carson & frazer 2016, vol 20, post publication review tragedy: facebook & aboriginal people post publication review carlson, b.l., farrelly, t., frazer, r., borthwick, f. mediating (2015) tragedy: facebook, aboriginal peoples and suicide. australasian journal of information systems, 19. doi:http://dx.doi.org/10.3127/ajis.v19i0.1174. review this is a well-considered and informative article detailing the tragedy of premature, indigenous suicide and the effects of social media. the authors demonstrate a clear need for research in this area, showing for example that aboriginal suicide is among the highest in the world with children less than 14 years old being nearly eight times more likely to commit suicide than non-aboriginal australians. furthermore, their study identified that aboriginal people frequently use social media as a communicative tool to mutually support and care for friends and family. although the authors revealed that facebook as a platform broke down barriers for aboriginal people in accessing formal mental health services, it would have been helpful to readers had they more thoroughly discussed what might be appropriate responses. for instance, one study recently suggested a possible technical solution for rural and remote service providers, where indigenous populations are frequently higher, because privacy legislation in australia often hampers the communication of mental health data, hindering appropriate treatment for mental illness (burmeister, islam, dayhew, & crichton, 2015). thus whilst this study’s results demonstrate that facebook offers a further, potentially vital avenue for aboriginal people to seek help outside formal services, the way in which that can be done compliant with legislation needs further investigation.the use of facebook appears to be both positive and negative, in that it can be a platform for help-seeking or -giving by overcoming barriers for users, as human connectedness and community is a protective factor for suicide (mckinley, 2016). however, skeptics state that a ‘call for help’ via facebook can be an attention-seeking exercise and they lack confidence that facebook can actually assist. it will be interesting to see if other related initiatives shed further light on this. for instance, the young and well cooperative research centre’s initiative to build an internet portal for young people with mental illness, or the online service provided by headspace, or the nsw kids and families project ‘health system navigation in the digital age’, aimed at improving the way rural kids access health data. as the authors point out, a promising option in addressing aboriginal suicide is to engage in already existing effective practices. facebook’s potential to contribute to mental illness, through facilitating racial abuse and encouraging others to suicide, is of great concern. however development and evaluation of culturally appropriate suicide intervention programs, using facebook as a platform warrants further investigation. while the methodology has sought to understand the complexities of indigenous experience, age, gender and other attributes have not been used substantially in interpretation of findings. the authors point out that much of the social media focus is on young people and that their methodology involved 50 interviews with people aged 18-60. however, as they also point out repeatedly, the aboriginal suicide rates increased from 1970 onwards due to, amongst other causes, systematic oppression. furthermore, whilst they rightly point to the vulnerability of people exhibiting suicidal ideation, they do not explain why they limited the study to a maximum age of 60. suicide rates are high amongst older people in the wider population, and particularly older men. furthermore, older people are frequently seen as especially vulnerable, because unlike other age groups, they are unlikely to recover from declining functionality, due to the ageing process (bernoth, dietsch, burmeister, & schwartz, 2014). related to this is the fact that the authors highlight the high uptake of social media and mobile technology in aboriginal communities, but do not differentiate on the basis of age. for instance, psychosocial research in ageing has shown that over the age of 70 people increasingly experience learning difficulties (baltes & smith, 2002), and it has been shown that this is also the case for technology related learning (burmeister, 2010), which calls into question whether the touted 1 australasian journal of information systems mckinley, ritchie, burmeister, carson & frazer 2016, vol 20, post publication review tragedy: facebook & aboriginal people benefits the authors claim apply to aged groups they have not considered, namely people older than 60, including older aboriginal people who experienced the systematic oppression that led to the increasing suicide rates from 1970 onwards. further, the observations about social media usage were taken from other studies rather than the specific communities this study was based upon. whilst it is important to connect this research to that of others working in the area of apps, tablets and smartphones for suicide prevention (aguirre, mccoy, & roan, 2013), the way the authors have done this makes it difficult to compare their results to those of others. given the spread of communities between metropolitan, rural and remote communities, mapping communities to health service providers such as aboriginal medical services, might also have some influence on help-seeking and help-giving behaviours. the authors acknowledge that their participants were more often receiving requests for help and therefore, further work on why they were help-givers might assist in the better understanding of the helpseekers. in suggesting a lack of mental illness such as depression, and that males were more likely to act impulsively in the context of intoxication, the different contributing factors between adults and aboriginal children under 14, means the understanding about the use of social media is incomplete. social media is clearly important, as this study demonstrates, but further investigation is required to address its use in reducing the incidence of suicide in aboriginal communities. better understanding beliefs about health (boddington & raisanen, 2009), how they link to suicidal ideation, and therefore where appropriate supports might be accessed, would strengthen the important first step this study takes in examining the use of social media in aboriginal communities. the authors state that the work reported in the article is in its early stages. given that many of those most affected by the systematic oppression of past australian policies are now quite senior in age, not including older people in their sample seems to be an oversight that should be addressed as their study continues. overall this is an important contribution to the literature and a topic that needs to be explored in more detail. to successfully combat the alarming rates of suicide in our population more discussion is needed in various people groups: indigenous, military, law enforcement (mckinley, 2016), and others. the role of digital technologies in that process also needs to be considered. amber mckinley charles sturt university david ritchie charles sturt university oliver burmeister charles sturt university oburmeister@csu.edu.au references aguirre, r. t. p., mccoy, m. k., & roan, m. (2013). development guidelines from a study of suicide prevention mobile applications (apps). journal of technology in human services, 31(3), 269-293. doi: 10.1080/15228835.2013.814750 baltes, p. b., & smith, j. (2002). new frontiers in the future of ageing: from successful ageing of the young old to the dilemmas of the fourth age. paper presented at the valencia forum valencia, spain. bernoth, m., dietsch, e., burmeister, o. k., & schwartz, m. (2014). information management in aged care: cases of confidentiality and elder abuse. journal of business ethics, 122, 453-460. doi: 10.1007/s10551-013-1770-7 boddington, p. & raisanen, u. (2009) theoretical and practical issues in the definition of health: insights from aboriginal australia. j. of medicine and philosophy, 34(1), 49-67 2 australasian journal of information systems mckinley, ritchie, burmeister, carson & frazer 2016, vol 20, post publication review tragedy: facebook & aboriginal people burmeister, o. k. (2010). websites for seniors: cognitive accessibility. international journal of emerging technologies and society, 8(2), 99-113. burmeister, o. k., islam, m. z., dayhew, m., & crichton, m. (2015). enhancing client welfare through better communication of private mental health data between rural service providers. australasian journal of information systems, 19, 1-14. doi: http://dx.doi.org/10.3127/ajis.v19i0.1206 mckinley, a. (2016). solvability and applied victimology in new south wales 1996-2013 (phd), bond university, robina queensland. author response we would like to start by thanking the above authors for their thoughtful and well-informed response to our paper. their insights are eagerly welcomed—especially considering the subject matter, which is both complex and urgent. the authors identified a number of valid challenges and gaps within our research, which we hope to explore below. we thought it would be worth opening by explaining in more detail the context in which the paper emerged. the interview data on which the paper is based was collected for a larger research project that explored—broadly—indigenous australians’ use of social media. this larger project aimed towards developing insight into the diverse practices, meanings and politics of indigenous peoples’ engagement with social media technologies. by talking with approximately a hundred indigenous people from communities all across australia—from sydney to ceduna, brisbane to broome—we hoped to unpack indigenous social media usage in all its complexity. this qualitative study was primarily interview-based: we thus aimed at obtaining ‘thick descriptions’ of everyday lived experience rather than broadly representative samples. qualitative studies of this nature resist epistemological closure, allowing for the emergence of perhaps unexpected themes. for instance, at the outset of the project, we never imagined that sorry business (practices associated with death and loss in many indigenous cultures) would be such a key issue for indigenous peoples’ experience of social media. likewise, neither did we predict the extent of indigenous online activism. we have consequently been able to critically unpack these topics (see carlson & frazer 2015 and carlson & frazer in press, respectively), which had previously been neglected in the existing literature. adopting a narrower approach to the terms of the project would likely have precluded these important yet greatly under-explored topics from emerging. furthermore, this empirically open approach was, in part, a political decision. historically, research on indigenous australians has often been gravely misdirected—even while generally being well-intentioned. it has been argued that “australia’s indigenous peoples are the most researched people in the world” (fredericks 2008; see also smith 1999). too often, this research has not been for the benefit for indigenous peoples themselves. indeed, research has been directly implicated in increasing both colonial power and the efficacy of indigenous subjugation. early anthropological work on indigenous australians, for example, often simply confirmed the myth of white superiority. later, it was also complicit in justifying the forced removal of children from indigenous communities. in response, and understandably, there exists significant distrust of scientists and other researchers among many indigenous communities. this is particularly true of approaches that lean towards the ‘hard sciences’ and thus adopt primarily quantitative methodologies. these approaches have been criticised in some cases for propagating the myth of the distanced, ‘neutral’ observer, while in fact being influenced heavily by colonial power relations. for these reasons, we took cue from torres strait islander scholar martin nakata (2007), among others (smith 1999), who calls for the development and adoption of a research paradigm that is both culturally appropriate and relevant for indigenous australians—indeed, an adamantly de-colonial research paradigm. for this project, we drew inspiration from nakata’s indigenous standpoint theory, which grants epistemological privilege to indigenous 3 australasian journal of information systems mckinley, ritchie, burmeister, carson & frazer 2016, vol 20, post publication review tragedy: facebook & aboriginal people stories, meanings and world-views. in this way, fact and truth are not things that exist in some pure and transcendental form, but are instead continuously (re)produced at the ‘cultural interface’ of indigenous people’s lived everyday realities. however, as the above authors have identified, this approach also comes with its own set of limitations. in taking as a point of departure a heterogeneous epistemology, we circumscribe the extent to which any results may be generalisable or comparable across populations. particularly when it comes to such urgent topics as indigenous suicide, it does not generate the most rigorous or immediately practicable data. instead, we hoped this paper might offer insights into the diverse help-seeking/-giving practices in which indigenous australians are already engaged—a topic which has received little academic consideration. we contend that through paying attention to the range of online practices that already exist in indigenous communities—what we identified as a ‘continuum’ of practices, ranging from gentle online support to emergency intervention—we may begin to develop approaches to suicide prevention and intervention that are culturally appropriate and relevant. while in our paper we were able to offer initial insights into help-seeking/-giving, we were not able to take the crucial second step of propounding the efficacy of specific practices. we therefore agree wholeheartedly with the authors that this topic requires much greater attention than the modest insights our paper offered. to help approach this need for greater critical enquiry, the first author of the paper, associate professor bronwyn carlson, is leading a national study that will explore in detail indigenous help-seeking and help-giving practices online. funded by the australian research council for three years commencing this year, the project aims to unlock social media’s potential to create vital connections between help-seekers and help-givers. leveraging on the previous work on aboriginal and torres strait islander use of social media, the new research project will be the first comprehensive analysis of aboriginal and torres strait islander help-seeking and help-giving behaviours online. this larger project will thus aim to address the failings of our initial investigation. as the authors of the response point out, a truly rigorous research project needs to be able to account for differences in age, gender and various other participant attributes. further, if it wants to effectively reduce the alarming rates of indigenous suicide, the project’s results should offer practical and thoroughly evaluated interventions which can be promoted and implemented by communities, health services, and other interested parties. it is these hopes that we invest in this coming project. bronwyn l carson university of wollongong ryan frazer university of wollongong rfrazer@uow.edu.au references carlson, bronwyn, and ryan frazer. "“it’s like going to a cemetery and lighting a candle”: aboriginal australians, sorry business and social media."alternative: an international journal of indigenous peoples 11.3 (2015). carlson, bronwyn, and ryan frazer. indigenous activism and social media: the global response to #sosblakaustralia, in negotiating digital citizenship: control, contest and culture, rowman and littlefield (in press). fredericks, bronwyn. "so you want to do oral history with aboriginal australians." journal of the oral history association of australia 30 (2008): 22-24. nakata, martin n. disciplining the savages, savaging the disciplines. aboriginal studies press, 2007. 4 australasian journal of information systems mckinley, ritchie, burmeister, carson & frazer 2016, vol 20, post publication review tragedy: facebook & aboriginal people smith, linda tuhiwai. decolonizing methodologies: research and indigenous peoples. zed books, 1999. copyright: © 2016 mckinley, ritchie, burmeister, carson & frazer. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. 5 http://creativecommons.org/licenses/by-nc/3.0/au/ http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems ahmed, taskin, pauleen & parker 2017, vol 21, research article motivating information technology professionals 1 motivating information technology professionals: the case of new zealand shoaib ahmed eagle technology shoaib_ahmed@eagle.co.nz nazim taskin massey university david pauleen massey university jane parker massey university abstract it professionals play a critical role in organizations. research indicates that they may be unique in their attitudes toward motivation and job satisfaction. in new zealand, a shortage of skilled professionals may contribute to or impact on motivation. using a modified model of herzberg’s two-factor theory by smerek and peterson (2007), this research seeks to answer the question: what motivates new zealand it professionals? in response, an online questionnaire was distributed to a population of new zealand it professionals and the data analysed using partial least squares to understand the relationship between the various dimensions of job satisfaction, the impact of personal and job characteristics, and turnover intention. the findings show that the new zealand it professional is primarily motivated by the nature of his or her work, followed by perceptions of responsibility, and how supervisors encourage an environment for such. satisfaction with salary is a predictor to a lesser degree. perhaps somewhat surprisingly, professional growth opportunities, career advancement, and recognition do not have a statistically-significant positive association with motivation. we conclude that, to motivate their it workforce, organizations should: 1) focus on the nature of the jobs that it professionals undertake; 2) train supervisors to provide an empowering environment; 3) offer competitive salaries to retain top talent; 4) not hesitate to employ it professionals born outside new zealand; and 5) take account of the singularities of the new zealand labour market in seeking to attract, recruit and retain it professionals. implications for policy, practice and theory are discussed. keywords: it professionals; motivation; career advancement; job satisfaction; turnover 1 introduction managers are interested in understanding motivation to deliver better employee performance outcomes. psychologists have been interested in job satisfaction and motivation since the 1930s (locke & latham, 2002) and researchers have investigated the effects of supervision (roethlisberger & dickson, 1939), human cognitive growth needs (alderfer, 1969; herzberg, 2003; maslow, 1943; mcclelland & burnham, 1995) and psychological processes (adams, 1965; bandura, 1977; locke, 1977; vroom, 1964). while key schools of thought were formed decades ago, the subject remains relevant. amazon offers nearly 10,000 books on motivation. a typical new zealand (nz) university library contains over 200,000 academic journal articles on this topic. this mass of literature can hinder the choice of appropriate, context-sensitive recommendations for nz it managers. a complicating factor is that it professionals might somehow be distinct from others in their attitudes to motivation (ferratt & short, 1986), making motivation theories based on general populations less applicable. otherwise, managers may be susceptible to implementing australasian journal of information systems ahmed, taskin, pauleen & parker 2017, vol 21, research article motivating information technology professionals 2 recommendations that conform to prevalent management attitudes rather than those tailored to the motivations of it professionals. declining levels of job satisfaction levels in nz firms are anecdotally suggested but raise concerns about potential negative impacts on staff motivation and performance. this concern is valid given previous research on motivation and performance (e.g. judge, thoresen, bono, & patton, 1998; mcmurtrey, grover, teng, & lightner, 2002). a nz ministry of business innovation and employment (mbie) (2015) report acknowledged shortages in the skilled ict sector staff, which has consequences for workload, quality of life and stress. moore (2000) indicated that these features contribute to dissatisfaction among it professionals, causing them to look for alternative opportunities. this is couched within the wider trend in western nations of a loss of leisure time (schor, 2008). despite the sizeable and diverse literature on motivation, no one theory has gained universal acceptance. debate on work motivation is ongoing (locke & latham, 2004), as is that over whether it professionals share the same motivational characteristics. extant literature indicates that it professionals’ motivational drivers are somewhat unique. williamson (1996) identifies ‘expressively oriented’ workers, who may be influenced by factors other than pay, contrasting with most organizational literature which assumes workers are instrumentally oriented (smith & shields, 2013). little literature exists in this area in nz, and what does focuses on medicine (e.g. kluger & bryant, 2008). major studies on motivation have traditionally focused on non-it industries (ghazzawi, 2008), including in nz (e.g. boxall, macky, & rasmussen, 2003). previous analyses of nz technology firms have indicated indigenous approaches to compensate for geographical isolation and size (souder, buisson, & garrett, 2001). some work has viewed nz as a unique environment due to a relative lack of innovative human resources (hr) practices (e.g. guthrie, 2001) and more recent work still echoes the importance of organizational scale for hr development. the tech sector is a large contributor to nz’s economy, with over 28,000 companies employing almost 5% of workers, but calls to develop a sustainable local talent base have focused more on the education system (e.g. nz technology industry association (nztech), 2016) than workplace hr strategy. a lag persists between progress in hr approaches in nz workplaces and other developed economies due to a number of local features: small nz workplaces not having a full in-house hr function (ryan, 2012; parker, 2013) and limited hr outsourcing; hr not widely on par with other management functions (e.g. arrowsmith & parker, 2012); and a local cultural identity of ‘self-sufficiency’ (e.g. toulson & foster, 2013), locke and latham’s (2004) call for investigating general and situation-specific contributors requires the exploration of a theory that can accommodate both. this study assists in greater understanding of such in the nz context. to better understand it professionals’ motivational issues in nz, we address the question: what motivates the nz it professional? the results should be of conceptual and practical interest to it managers, firms employing it professionals, policy-makers intent on maximizing nz’s capability in this sector, academics, and it professionals. our review of relevant literature is followed by a description of the research methods used in this study, its findings, and finally, key conclusions and recommendations. 2 literature review academic interest in motivation began with the famous hawthorne experiments (roethlisberger & dickson, 1939) which found that the performance of professionals improved due to their knowledge of being observed, regardless of modifications to working conditions (e.g. changes to schedules, meal breaks, etc.). the study was criticised (sykes, 1965; wickström & bendix, 2000) but the hawthorne experiments led to a series of theories focusing on the cognitive growth needs of humans as a source of motivation. australasian journal of information systems ahmed, taskin, pauleen & parker 2017, vol 21, research article motivating information technology professionals 3 2.1 cognitive needs-based theories the most famous of cognitive needs theories is maslow’s (1943) hierarchy of needs: physiological, safety, love, esteem, and self-actualisation. as lower needs are satisfied, higher order needs appear. humans are motivated by the desire to attain or perpetuate the conditions that satisfy these needs. while actions can be driven by multiple needs, the more potent need is the primary motivator. maslow’s work has informed many fields, including psychology, management and education. the needs hierarchy has been challenged for various reasons (wahba & bridwell, 1976; haire, ghiselli, & porter, 1966), and alderfer (1969) sought to reframe maslow’s work beyond these criticisms through his existence, relatedness and growth (erg) theory, which presents the same needs without a hierarchy. mcclelland (1961) proposed an alternative to maslow by identifying achievement, affiliation and power as motivators of human behaviour. he asserted that these three needs are acquired over time through experiences and can be taught (mcclelland, 1965). herzberg first linked job satisfaction and motivation in his two-factor theory (herzberg, mausner, & snyderman, 1959). most of his subjects reported happiness with jobs when they were successful and saw potential for growth, leading him to term these motivating factors. in contrast, when these subjects were asked about unhappiness, they most commonly responded about conditions surrounding the job (hygiene factors). herzberg argued that sources of job satisfaction and dissatisfaction are different; addressing causes of job dissatisfaction lessens job dissatisfaction, rather than provides job satisfaction. he thus championed enriching the job itself to increase motivation. herzberg’s work faced criticism primarily related to his method of research (e.g., hardin, 1965; gordon, pryor, & harris, 1974). however, several studies replicated herzberg’s findings (myers, 1964; saleh, 1964). herzberg (1968) later substantiated his earlier conclusions with studies from italy, israel, japan, india, zambia and south africa. he also distinguished between motivation, an internal engine, and movement, where actions are due to motivation of a third party. lindsay, marks, and gorlow (1967) concluded that the theory has been both demonstrated and challenged in part. vroom was instrumental in adopting psychological processes as a key determinant of motivation through his work in developing expectancy theory, representing a move away from the prevailing needs-based thinking (locke & latham, 2004). his theory identified three psychological processes determining motivation: valence (the desire for attaining or avoiding an outcome); instrumentality (belief in how strongly an action is likely to lead to a desired outcome); and expectancy (belief in whether the desired outcome is feasible (vroom, 1964)). however, vroom did not discuss the origins of valence, instrumentality or expectancy. developing a theoretical model and testing it on a group of managers, porter and lawler (1968) identified motivation for effort as dependent on an individual’s perceived value of an outcome and his or her perception as to whether effort will lead to success. they noted that effort does not equate to performance, which is further dependent on the ability of the individual and role clarity. they further argued that the link between performance and satisfaction is tenuous and depends on rewards intrinsic more than extrinsic and their perception of equity relative to the outcome achieved (schuster, clark, & rogers, 1971). bandura (1977) took the role of expectancy further with his theory of self-efficacy. while efficacy itself cannot be considered the motivator, it affects motivation through its impact on an individual’s goals and aspirations. this is what sustains effort through adversity. individuals are only motivated to pursue a course of action they believe is within their capability, and routinely exclude options which they believe are beyond their means (bandura, 2000). however, his work is criticised for not controlling for the effects of behaviour approach tests in a social context (tryon, 1981). table 1 summarises the major theories and their proponents. australasian journal of information systems ahmed, taskin, pauleen & parker 2017, vol 21, research article motivating information technology professionals 4 basis theory researcher primary idea c o g n it iv e n ee d s hierarchy of needs (maslow, 1943) humans have five basic needs in order of priority. erg theory (alderfer, 1969) maslow’s theory, without the hierarchy. acquired needs theory (mcclelland, 1961) needs are acquired through experience and can be taught. two-factor theory (herzberg et al., 1959) satisfaction and dissatisfaction are separate scales. what dissatisfies, if addressed, only reduces dissatisfaction. it does not satisfy. p sy ch o lo g ic a l p ro ce ss expectancy theory (vroom, 1964) motivation is dependent on an individual’s perceived value of an outcome and his or her conviction on whether effort will lead to success. (porter & lawler, 1968) defines reasons for the expectancy theory. theory of self-efficacy (bandura, 1977) efficacy determines an individual’s goals and aspirations. goal-setting theory (locke & latham, 2002) faced with extreme goals, individuals are motivated to find new ways to achieve them. table 1: major theories on motivation 2.2 motivation in an it context research on motivation and job satisfaction in the it field is not new. goldstein and rockart (1984) investigated the motivation of software developers and analysts in the eighties. the prevailing motivation approaches consider rewards and recognition (beecham, baddoo, hall, robinson, & sharp, 2008). several experts have claimed that it professionals derive motivation from what they do, and technical challenges that they can overcome (ramachndran & rao, 2006; tanner, 2003). further support for differences in personalities between it and non-it professionals is found in wynekoop and walz (1998). expectation of market conditions compared to one’s present circumstances can play a role in job satisfaction (jiang & klein, 2002). this is strengthened by studies finding that access to the latest technology and training contributes to higher levels of satisfaction than remuneration (jyothi & ravindran, 2012; shuck, twyford, reio, & shuck, 2014). anjali and anand (2015) note specific characteristics of it professionals, such as their choice of employment due to high skill levels, unpredictable work schedules in the face of project deadlines, pressures of meeting service levels, often being based remotely, and sitting for long periods. they note that these lead to reduced efficiency and productivity as a result of boredom unless tasks can be kept exciting. rajagopal and abraham (2009) find that time pressure contributes to a different hierarchy of needs. it professionals show distinctly lower social and psychological needs. their needs progress from safety to self-esteem, with the authors concluding that higher-order needs are more relevant for them. wynekoop and walz (1998) found that those entering it professions were more creative than others. however, schambach and blanton (2002) found a dichotomy. organizations acknowledge the need for skilled it professionals and aggressively pursue them with attractive remunerations and stock options. yet many fail to create an environment where these skilled employees, especially those in mid-career, can maintain skills to produce necessary innovation. consequently, concern over continued employability plays a key role in reducing motivation. other researchers contest the proposition that it professionals are different in their attitude to motivation. ferratt and short (1986) asserted that studies claiming differences have not used similar occupational groups (i.e., it and non-it staff in the same company), causing misinterpretation (see also im and hartman (1990) and smith and speight (2006)). this study’s goal is to examine a) the motivation of it professionals b) in the nz context, which is considered to be unique on account of the nature and scale of its hr practices (guthrie, australasian journal of information systems ahmed, taskin, pauleen & parker 2017, vol 21, research article motivating information technology professionals 5 2001; toulson & foster, 2013). drawing on herzberg’s well-known two-factor theory, this focus aligns with the call from locke and latham (2004) to investigate situation-specific contributors to theory. 2.3 two-factor theory the two-factor theory was originally derived from the job attitudes of 203 accountants and engineers in pittsburgh. participants were asked to recall times when and why they were more satisfied and dissatisfied than usual (herzberg et al., 1959). the theory was refined over 12 further investigations. contrary to previous assumptions, job satisfaction (motivation) and dissatisfaction (hygiene) had distinctive contributions. herzberg related hygiene factors to the need to avoid pain. these include company policy and administration, supervision, relationship with supervisor, work conditions, relationships with peers and subordinates, salary, personal life, status, and security. in contrast, he compared the motivation factors to the unique human ability to succeed through achievement and resulting psychological growth (herzberg, 1968). motivation factors include recognition, the work itself, achievement, growth, responsibility and advancement. herzberg’s findings indicate that motivation factors have a four-fold impact on job satisfaction when compared with hygiene factors. the best summary of the two-factor theory is provided by herzberg et al. (1959: 114) where the authors declare: “(t)he factors that lead to positive job attitudes do so because they satisfy the individual’s need for self-actualization.” based on this theory, this study proposes the following hypotheses: h1: intrinsic motivation factors (motivators) are positively associated with job satisfaction for it professionals in nz; and h2: extrinsic motivation factors (hygiene factors) are positively associated with job satisfaction for it professionals in nz. 2.4 motivation and personal characteristics personal characteristics are cited as contributing to or moderating motivation among workers. these include age (allen & meyer, 1993), gender (deci, koestner, & ryan, 1999; magee, 2013) and even local community attitudes (shiu, hassan, & parry, 2015). the effect of education on job satisfaction appears less certain. some researchers contend that those with higher levels of education are likely to be less satisfied due to the routine nature of work tasks (bruce, bonjean, & williams jr., 1968; carrell & elbert, 1974). however, others argue that the higher extrinsic rewards achieved through higher education are likely to have the opposite effect (glenn, taylor, & weaver, 1977; lee & wilbur, 1985). accordingly, this study proposes that: h3: personal characteristics have a positive association with job satisfaction for it professionals in nz. 2.5 motivation and job characteristics academics have identified job characteristics as a key contributor to perceptions of job satisfaction and turnover intention (kristof-brown, zimmerman, & johnson, 2005; verquer, beehr, & wagner, 2003). the longer an employee remains with an employer, the less likely advancement becomes, leading to reduced job satisfaction (bartholomew, 1972; gibson & klein, 1970; stoikov, 1971). in some cases, experience has similar effects to tenure. fewer opportunities in leadership positions may mean reaching a plateau in one’s career, leading to lower job satisfaction (cohen, 1993). some researchers claim that money has the largest effect on motivation (tang & gilbert, 1995; wagner, rubin, & callahan, 1988). this is somewhat disputed by herzberg et al. (1959) who assert that hygiene factors can cause dissatisfaction rather than factors that have motivational power. herzberg, however, conceded that satisfaction with salary is a construct that could be part of the intrinsic factors. opportunities for training and professional memberships may provide higher motivation compared to salary and bonuses in certain it environments (jyothi & ravindran, 2012; shuck et al., 2014). based on this discussion, we propose that: australasian journal of information systems ahmed, taskin, pauleen & parker 2017, vol 21, research article motivating information technology professionals 6 h4: job characteristics have a positive association with job satisfaction for it professionals in nz. 2.6 job satisfaction and turnover intent research indicates a link between low job satisfaction and turnover intention. lambert, hogan, and barton (2001) posit that the work environment plays a greater role in shaping job satisfaction than an individual’s characteristics; this in turn influences turnover intent. this is supported by kanwar, singh, and kodwani (2012) who found that those with higher job satisfaction among indian it professionals have lower turnover intent. moreover, studies show that turnover reduces performance and can risk a period of low productivity (gersick, 1988; glebbeek & bax, 2004). some researchers maintain that not all turnover is detrimental for an organization and an optimal turnover rate exists (abelson & baysinger, 1984; wallace & gaylor, 2012). others point to alternative explanations such as age (bartholomew, 1972; gibson & klein, 1970; stoikov, 1971), concern about employability (direnzo & greenhaus, 2011), psychological reasons (lambert et al., 2001) and ‘fit’ with the organization (kristof-brown et al., 2005; verquer et al., 2003) as contributors to turnover intent. while research suggests that the source of turnover intention is multivariate in nature, motivation and job satisfaction play a significant role (tzeng, 2002). judge et al. (1998) contend that the relationship between job satisfaction and performance is more pronounced in complex professions, such as it. in their meta-analysis, griffeth, hom and gaertner (2000) identified that turnover actions, despite going through several withdrawal decisions and behaviours, are primarily initiated by job dissatisfaction (see also mak and sockel (2001)). they further point to work from mone (1994), indicating that it is the most talented staff who are at risk. this study extends the model (below) to incorporate turnover intent: h5: job satisfaction is negatively associated with turnover intent for it professionals in nz. 2.7 the smerek and peterson adjustment smerek and peterson (2007) developed and validated a survey instrument based on herzberg’s two-factor theory, which has been adopted here since it is comprehensive and takes account of respondents’ personal and job characteristics. this allows our study to observe both general traits and situation-specific factors, a key recommendation by locke and latham (2004). while the original study measured satisfaction among university employees, others have replicated the instrument among hospital employees (oladotun & öztüren, 2013) and social workers (kundrátová, 2011). the current study validates smerek and peterson’s (2007) updated model of herzberg’s theory by examining the relationship between motivation and personal and job characteristics, extends the model by testing the relationship between turnover intention and job satisfaction, and answers the call from locke and latham (2004) to investigate situation-specific contributors to theory by examining these factors with it professionals in nz. herzberg’s two-factor theory explores many dimensions of job satisfaction other than pay, a key consideration for skilled it professionals. smerek and peterson's (2007) study, using the two-factor theory as a basis, further allows for analysis of an individual’s general traits through their personal characteristics, and situation-specific contributors in the form of job characteristics, and is ideally suited for providing an overarching theory for this study. the model is extended to include turnover intention as this has been identified in the literature as having a strong negative association with job satisfaction. figure 1 summarises the hypotheses developed for this study. australasian journal of information systems ahmed, taskin, pauleen & parker 2017, vol 21, research article motivating information technology professionals 7 figure 1: conceptual model 3 methodology as this study aims to access a population who represent it professionals in nz, an online survey method was deemed the most efficient and effective approach (cobanoglu, warde, & moreo, 2001). the study used a modified model of herzberg’s two-factor study from smerek and peterson (2007). the items were adopted using a 10-point likert scale as in the original instrument. minor adjustments were made in wording the questions to provide the correct context (e.g. “the university” became “my organization”), place of birth and questions on minority status were replaced with place of birth (i.e. born in nz?), and work unit classifications were made relevant to the it profession. turnover intent is measured using singh and loncar's (2010) method, an extension of lum, kervin, clark, reid, and sirola's (1998) approach. 3.1 data collection immigration nz (2015) estimates that the it workforce in nz numbers approximately 46,000 and statistics nz (2013a) puts it at just under 54,000. the it profession does not require practitioners to belong to a licensing body, so the exact number and a definitive list of practitioners remain unknown. research studies collecting data from organizations may not be uniformly well received by all, resulting in an undesirable skew towards some organizations (bryman & bell, 2011). therefore, the population was accessed through linkedin, a professional social networking site. the population is actively engaged in their professional domain. as the questionnaire was administered online, it provided a convenient method for encouraging participation. social network sites represent a new but increasingly accepted context in academic research design. for example, robinson, sinar and winter (2014) used linkedin data to study organizational turnover. it studies have also used social media – barzilay and urquhart (2014) used linkedin forums to study perceptions of code sample reuse in software development, and benevenuto, rodrigues, cha, and almeida (2012) used data from social networks including linkedin to study user navigation on the web. while researchers recognise risks in using online survey data, especially around the selfreporting nature of social networks (patton, 2012), there is less concern here because the intention was to use social media to access a population in a random manner, rather than to australasian journal of information systems ahmed, taskin, pauleen & parker 2017, vol 21, research article motivating information technology professionals 8 seek to verify their claimed credentials. the risk of misrepresentation was therefore no worse than for any other method of accessing a population. 4 analysis and results the authors posted a survey invitation to the linkedin group entitled ‘nz it industry’ (nz it industry, 2015), in line with hewson and laurent (2008). this group serves as a common space for anyone employed as an it professional in nz. it had 8,390 members at the time of this study. among them, 250 group members were selected using the microsoft excel randomizer and invited to participate in the survey. a total of 157 responses were received (with a return rate of 62.8%). sixteen responses were removed due to some missing survey data, leading to an effective response rate of 56.4% (n=141). significantly, more males (n=107; 75.9%) responded than females (n=34; 24.1%). the number of women studying computer science and it is seen to be dropping, with 1 in 5 it students identifying as female (absoluteit, 2015). respondents were almost evenly split between those born in nz (n=69; 48.9%) and overseas (n=72; 51.1%). most were full-time employees (n=117; 83%). the majority of both nz-born (n=61; 88.4%) and overseas-born respondents (n=56; 77.8%) were employed full-time. the most widely represented age group was of 35-44 years (n=47; 40.4%), followed by 45-54 (n=42; 29.8%). more than three-quarters of respondents hold a bachelor’s degree (n=52; 36.9%) or higher (n=56; 39.7%). one-third has in excess of 20 years’ experience (n=47; 33.3%). more than one-third have been in their present role for less than two years (n=52; 36.9%), while a considerable number (n=17, 12.1%) have been there for a decade or longer. performance-based bonuses formed part of the remuneration package for 28.4% (n=40) of respondents, and a similar number belong to professional bodies (n=39, 27.7%). table 2 gives survey respondent details. gender n % age n % male 107 75.9 < 25 5 3.5 female 34 24.1 25 to 34 23 16.3 35 to 44 57 40.4 qualification n % 45 to 54 42 29.8 high school equivalent or less 5 3.5 55 to 64 13 9.2 industry certification (i.e. microsoft, oracle) 28 19.9 >=65 1 .7 bachelors degree 52 36.9 postgraduate degree or higher 56 39.7 role – overseas born n % self-employed or owner 4 5.6 role – nz born n % contract 10 13.9 self-employed or business owner 2 2.9 part time employed 2 2.8 contract 5 7.2 full time employed 56 77.8 part time employed 1 1.4 full time employed 61 88.4 tenure (years) n % <2 52 36.9 salary n % >=2 and <5 44 31.2 60,000 or less 12 8.5 >=5 and <10 28 19.9 60,001 to 80,000 25 17.7 >=10 17 12.1 80,001 to 100,000 28 19.9 100,001 to 120,000 22 15.6 experience (years) n % 120,001 to 150,000 29 20.6 <5 15 10.6 higher than 150,000 25 17.7 >=5 and <10 18 12.8 >=10 and <15 21 14.9 australasian journal of information systems ahmed, taskin, pauleen & parker 2017, vol 21, research article motivating information technology professionals 9 bonus as part of remuneration n % >=15 and <20 40 28.4 yes 40 28.4 >=20 47 33.3 no 101 71.6 prfs membership n % yes 39 27.7 no 102 72.3 table 2: descriptive statistics of the survey population analysis was undertaken using spss version 23, and warppls version 5.0, a structural equation modelling (sem)-based partial least squares (pls) application. unlike covariancebased sem approaches, pls is variance-based and has several advantages for this study (rigdon, 2012). it can provide strong statistical output without assumptions about a normal distribution of the data (hair, sarstedt, pieper, & ringle, 2012). shapiro-wilk and kolmogorov-smirnov tests of normality indicated that the study data were non-normal in their distribution. pls can also undertake sem with a smaller sample size as compared to what is deemed acceptable for covariance-based sem (n>150) (chin & newsted, 1999). this study used two orders of pls-sem, necessitated by its underlying model. the questionnaire from smerek and peterson (2007) allowed measurement of job satisfaction dimensions. a second abstraction was required to understand the concept of intrinsic and extrinsic motivation based on the dimensions from the observed variables. this was achieved through a second order construct (ciavolino, 2012; nitti & ciavolino, 2014; petter, straub, & rai, 2007). the first order comprised reflective indicators, measuring similar aspects of a construct, and was expected to correlate with other indicators of the same construct. abstract constructs such as intrinsic and extrinsic motivation were measured using the two-step modelling approach due to their formative nature, where the indicators within measured different aspects and are less likely to correlate highly (ciavolino, 2012; nitti & ciavolino, 2014). pls is ideal for analysing a combination of reflective and formative constructs (becker, klein, & wetzels, 2012; ringle, sarstedt, & straub, 2012). furthermore, it is an appropriate method and has been used for both developing and confirming theories (chin, 1998; urbach & ahlemann, 2010). 4.1 measurement model – reflective indicators prior to conducting confirmatory factor analysis (cfa) and testing the structural model, exploratory factor analysis (efa) was performed to test sampling adequacy. the principal component analysis (pca) method with direct oblimin rotation was used. keiser-meyerolkin’s (kmo) measure of sampling adequacy (kmo=.879) and bartlett’s test of sphericity (p<.001) indicated that data are suitable for factor analysis (nixon & saphores, 2007; wilson, rodgers, carpenter, hall, hardy, & fraser, 2004). the factors extracted explain 87% of the variation in the data (see appendices table a1). unidimensionality was retested after executing cfa, and required the removal of some indicators. the results of rohatgi-szúkely (rs) and klaassen-mokveld-van es (mkve) tests of unidimensionality (refer to table 3) further confirmed the identified factors can be relied upon to measure a single construct. cronbach’s alpha (ca) and composite reliability (cr) were used to measure the internal consistency of the questions that comprise each construct within the model. the ca score for the remaining factors all exceeded .8 which is considered good, and .9 which is considered excellent (devellis, 2012). however, the ca score for effective supervisor, in excess of .95, may indicate a possible common method bias (straub, boudreau, & gefen, 2004). a full collinearity test was undertaken for effective supervisor. however, it found a variance inflation factor (vif) score of 3.017, lower than 3.3, the most conservative indicator of collinearity in the latent variable (kock & lynn, 2012). therefore, the common method bias risk is considered low. cr scores for all constructs were above the acceptable threshold of .7 (nunnally & bernstein, 1994). table 3 shows the internal consistency reliability measures for each construct. australasian journal of information systems ahmed, taskin, pauleen & parker 2017, vol 21, research article motivating information technology professionals 10 construct in te r n a l c o n s is te n c y r e li a b il it y u n id im e n s io n a li ty c o n v e r g e n t v a li d it y d is c r im in a n t v a li d it y cr ca rs mkve ave vif recognition .883 .834 ✓ ✓ .603 1.873 work itself .965 .951 ✓ ✓ .874 2.564 opportunities for advancement .881 .819 ✓ ✓ .651 2.371 professional growth opportunities .935 .913 ✓ ✓ .744 2.439 responsibility .938 .918 ✓ ✓ .753 3.381 good feelings about organization .955 .941 ✓ ✓ .810 2.583 clarity of mission .936 .917 ✓ ✓ .711 2.939 effective senior management .973 .958 ✓ ✓ .922 2.243 effective supervisor .988 .987 ✓ ✓ .844 3.017 good relations with co-workers .937 .922 ✓ ✓ .651 1.895 satisfaction with salary .878 .807 ✓ ✓ .653 2.105 satisfaction with benefits .910 .865 ✓ ✓ .718 2.019 presence of core values .938 .901 ✓ ✓ .836 2.262 job satisfaction .958 .934 ✓ ✓ .884 3.922 turnover intent .831 .723 ✓ ✓ .559 1.947 table 3: reliability and validity of constructs indicator reliability is measured by the correlation between the latent variables and the square root of the average variance explained (ave). all indicators have a value of greater than .707 and have the highest ave value for their own construct, thereby explaining more than half of the indicator’s variance. as a result, indicator reliability has been established (refer to table 4) (chin, 1998). convergent validity is measured with the fornell and larcker (1981) criterion. it requires each latent variable to have an ave value of .500 or higher, explaining at a minimum half of the variances of its indicators. this condition is met by the analysis (refer to table 3). discriminant validity is established through examination of vif from a full collinearity test. the construct for job satisfaction scored the highest value of 3.922 (refer to table 3). vifs below 5 are generally accepted as reliable evidence of low collinearity, thereby confirming discriminant validity (hair, black, babin, & anderson, 2010). australasian journal of information systems ahmed, taskin, pauleen & parker 2017, vol 21, research article motivating information technology professionals 11 rec wrk adv gth rsp gfo mis smg spv cwr sal ben val sat trn rec (.777) wrk .460 (.935) adv .558 .494 (.807) gth .417 .523 .532 (.862) rsp .521 .606 .509 .565 (.868) gfo .474 .502 .574 .549 .637 (.900) mis .504 .601 .555 .541 .724 .636 (.843) smg .375 .356 .508 .449 .548 .516 .547 (.960) spv .441 .555 .543 .675 .679 .576 .581 .471 (.924) cwr .443 .481 .365 .458 .482 .518 .545 .406 .492 (.830) sal .484 .417 .448 .404 .564 .472 .549 .534 .420 .408 (.901) ben .242 .278 .418 .447 .417 .484 .433 .447 .305 .226 .538 (.847) val .356 .352 .529 .333 .458 .503 .506 .639 .447 .403 .407 .480 (.914) sat .442 .710 .499 .556 .713 .637 .607 .466 .656 .455 .560 .408 .495 (.940) trn -.310 -.336 -.487 -.497 -.423 -.487 -.352 -.433 -.592 -.293 -.398 -.276 -.370 -.613 (.809) notes: rec recognition wrk work itself adv opportunities for advancement gth professional growth opportunities gfo good feelings for organization rsp responsibility mis clarity of mission smg effective senior management spv effective supervisor cwr good relations with co-workers sal satisfaction with salary ben satisfaction with benefits val presence of core values sat job satisfaction trn turnover intent table 4: correlation between latent variables and square root of ave in diagonal australasian journal of information systems ahmed, taskin, pauleen & parker 2017, vol 21, research article motivating information technology professionals 12 4.2 measurement model – formative indicators indicator validity is measured by a combination of significance weight and vifs. each indicator for the formative constructs and control variables demonstrates a higher level of significance than the prescribed .05 (efron & tibshirani, 1993). additionally, all of the individual indicators for the formative constructs, overall constructs and control variables display vifs less than 5 as intrinsic motivation being 4.719 and extrinsic motivation being 3.470 (refer to appendices table a2). consequently, multicollinearity is not a major threat to indicator validity (diamantopoulos & siguaw, 2006). it can be concluded that the indicators for the second order pls-sem are valid. 4.3 structural model – analysis explained variance of constructs relative to its total measure is used to assess the first part of model validity. the model explains moderate to substantial variance for job satisfaction (r2=.61) and turnover intent (r2=.38) (chin, 1998). path coefficients between the latent variables further assess the model’s fit. the only sign with a negative valence is that of turnover intent, which is expected based on prior literature (kanwar et al., 2012; griffeth et al., 2000). the path from extrinsic motivation (β=.17, p=.02) and intrinsic motivation (β=.61, p<.001) to job satisfaction, and from job satisfaction to turnover intent (β =.61, p<.001) all indicate a significant impact in the model. further tests on robustness through using a holdout sample of around 70% of the original sample to retest the findings (hair, ringle, & sarstedt, 2013) did not contradict any major conclusions from the full sample. non-response bias was tested as a threat to external validity of the study by comparing the early responders with later ones. it is estimated that the later respondents (who answered after the final reminder was sent barclay, todd, finlay, grande, & wyatt, 2002) are closer to nonrespondents compared to those who responded early (armstrong & overton, 1977). the welch f test was measured as 3.123 (p<.05) for age and 3.804 (p<.05) for income while brownforsythe was measured as 2.903 for age and 3.130 for income (for both p<.05). the results indicated there were no significant differences in gender and qualifications but there are for age and income (refer to appendices table a3). later respondents were more likely to be younger (m=2.92, sd=1.02) and have a higher income (m=3.84, sd=1.94) compared to early respondents (m=3.37, sd=1.02 and m=3.70, sd=1.45 respectively). statistics nz (2013b) data indicate that the proportion of males (75.8%) to females (24.2%), and full-time employees (83.9%) align with industry averages. regardless of the survey design, an element of non-response bias cannot be avoided (dillman, 1991). based on the external validity tests and publicly-available information, the study participants can be claimed to be representative of the target population (barclay et al., 2002). our results should be read in the knowledge that younger and higher paid it professionals may be marginally under-represented in the sample. 4.4 structural model – results first order pls-sem was undertaken with 15 latent variables from 78 observed variables. all latent variables in the first order are reflective variables, measuring different aspects of the same construct (becker et al., 2012). the contribution of individual factors was analysed for their effect on job satisfaction and subsequently turnover intention. the model explained 87% of the variance in job satisfaction. second order pls-sem was undertaken with latent variables leading to job satisfaction grouped into two sub-categories to form reflective latent variables – intrinsic and extrinsic motivation. the latent variables of recognition, work itself, opportunity for advancement, professional growth opportunity, responsibility, good feeling about organization, and clarity of mission constituted intrinsic motivation, while effective senior management, effective supervisor, good relationship with co-workers, satisfaction with salary, satisfaction with australasian journal of information systems ahmed, taskin, pauleen & parker 2017, vol 21, research article motivating information technology professionals 13 benefits, and presence of core values comprised extrinsic motivation (smerek & peterson, 2007; volkwein & zhou, 2003; spencer, 1997). additionally, observed control variables for demographic and job characteristics were included as control variables. a two-step second order pls-sem model was used. figure 2 represents the conceptual model tested in the study. figure 2: conceptual model with second order pls-sem the model explains 61% of the variation in job satisfaction. extrinsic motivation factors have a marginal positive association (β=.17, p=.02) with job satisfaction. in comparison, intrinsic motivation factors have a much stronger positive association (β=.61, p<.001). the influence of demographic and job characteristics on motivation was statistically insignificant. the model explains 38% of the variation in turnover intent and has a strong negative association with job satisfaction (β=-.61, p<.001). 5 discussion the model used in the study explained 61% of the variation in job satisfaction, and 38% of the variation in turnover intention. this aligns with previous studies in the field of information systems research by moqbel, nevo, and kock (2013) and stuart, verville, and taskin (2012). the model r2 for job satisfaction was .60 without the presence of control variables. with the control variables added, there was a positive (r2=.61) although statistically insignificant change. contribution to turnover intent (r2=.38) was unchanged with or without the presence of the control variables in the model. this study, like previous ones, found a strong negative association between job satisfaction and turnover intent (β=-.61, p<.001) (alniaçik, alniaçik, erat, & akçin, 2013; kanwar et al., 2012). judge et al. (1998) concluded there is a significant correlation between job satisfaction and performance in complex roles, a relationship that fits the it profession. therefore, the need for management to ensure high levels of job satisfaction to retain top talent is obvious. the study found a marginal positive association of extrinsic factors towards job satisfaction (β=.17, p=.02). the association of intrinsic motivation factors with job satisfaction is strong (β=.61, p<.001), and statistically more significant. it is noteworthy that there was more than a three-fold difference in effect between association of extrinsic and intrinsic factors to job australasian journal of information systems ahmed, taskin, pauleen & parker 2017, vol 21, research article motivating information technology professionals 14 satisfaction based on path coefficients. these results are similar to those supporting the twofactor theory (herzberg et al., 1959). the study results did not support hypotheses regarding the positive effects of demographic or job characteristics on motivation. second order pls-sem indicated that the effects of gender, age, academic qualifications, tenure, experience, pay, bonuses, and professional membership were statistically insignificant. this result was further substantiated by a holdout sample of 70% of the total response showing near-identical results (hair et al., 2012). this contradicts previous findings from other researchers (e.g. glenn et al., 1977). this contradiction must be read in light of the recent trend whereby researchers discount or re-interpret associations of personal characteristics in particular. in their model for analysing job satisfaction and organizational commitment, collie, shapka, perry and martin (2015) did not include personal characteristics. lange (2012) showed that the reason for higher job satisfaction among the self-employed was due to autonomy and independence, rather than personal characteristics, as had been thought. the results indicated that organizations can be confident that initiatives to increase or sustain job satisfaction levels are likely to result in similar outcomes, regardless of personal or job characteristics of staff. analysis of the first order pls-sem indicated that satisfaction with salary has a positive association with job satisfaction (β=.14, p=.05). this is not a surprise, as salary was only classified as a marginal extrinsic factor by herzberg et al. (1959) although some previous studies have concluded that remuneration has a large effect on motivation (tang & gilbert, 1995). significantly, the nature of it professionals’ work appears to influence job satisfaction most. this is demonstrated by the work itself showing the highest positive association (β=.39, p<.001) alongside responsibility (β=.20, p<.01). when this is taken into consideration with the only other statistically-significant association with job satisfaction being effective supervision (β=.23, p<.01), a pattern emerges. interesting and challenging work, coupled with responsibility, and the belief that supervisors are willing to encourage such, appear to be the best predictors of job satisfaction, a view shared by others (judge et al., 1998; saari & judge, 2004). one surprising finding was the lack of positive association of advancement or professional growth opportunities. some previous studies indicate that it professionals may be more motivated by these factors than even salary (jyothi & ravindran, 2012; shuck et al., 2014). however, no such evidence was found here. how an it professional feels about the firm or its clarity of mission does not impact on individual motivation. taking into account lack of impact on good relations with co-workers, our findings support rajagopal and abraham (2009) who contend that the hierarchy of needs for it professionals goes from basic needs to higher order, and skips some of the social dimensions. responses based on the control variables on the dimensions of job satisfaction also indicated interesting patterns. females are happier than males on most intrinsic motivation factors. this can be explained by the expectation hypothesis which states that, although both men and women desire the same or similar features in their jobs, the lower expectations that women have make them more satisfied in their workplace (lambert, 1991). however, additional research is required to further clarify the relationship between gender and intrinsic motivation, and whether the expectation hypothesis or value hypothesis holds for it professionals in the nz context. notwithstanding this, higher satisfaction with salary for females was a surprising finding, contradicting some previous studies (clark, 1997). absolute and relative pay can provide a plausible explanation as two determinants of satisfaction with salary (taylor & vest, 1992). by comparing their salaries with other women who work in other departments or organizations, these women may appreciate their pay more (jung, moon, & hahm, 2007). additionally, women form a smaller portion of the it workforce (chabrow, 2007). belonging to a small, skilled and highly-paid workforce may be satisfactory to women, especially if they are earning equal to or more than they expected to earn (oshagbemi & hickson, 2003) in the australasian journal of information systems ahmed, taskin, pauleen & parker 2017, vol 21, research article motivating information technology professionals 15 nz it industry. another reason for salary satisfaction may concern the level of women’s seniority in their organization (oshagbemi & hickson, 2003). according to andrews and henry (1963), there is a positive association between organizational level and satisfaction with salary. satisfaction is observed to noticeably drop off along many dimensions as people age. however, this does not appear to translate into an increased turnover intent. there were no statistically significant differences in the responses based on qualification and whether people were born in nz or overseas. job characteristic factors – tenure, experience, salary, bonus and professional membership – demonstrated little or no differences in the response pattern for any of the factors. those with higher salary or professional membership are happier with their salary than others – this is unsurprising given previous research (tang & gilbert, 1995; wagner et al., 1988). the lack of statistically significant differences in job satisfaction based on country of birth was mildly surprising. in a recent study in australia, kifle, kler and shankar (2016) found significant differences in job satisfaction between those born in australia and others. this difference is even more pronounced when the immigrant comes from a non-english speaking background. johnson and johnson (2000) found similar results in the us and concluded that it reflects how new migrants have to restart their careers at a lower level than in their native environments. kifle et al. (2016) noted that this effect reduces with time. this study did not control for the duration that non-nz born it professionals had been in nz. 5.1 implications for practitioners what motivates it workers should be of interest to management of both it firms and other organizations that employ it professionals, as well as to policy-makers responsible for delivering improved outcomes for their country through, for example, strengthening tertiary education and attracting suitably-skilled migrants (mbie, 2015). the relative contributions of extrinsic and intrinsic factors to job satisfaction have implications for the results of job satisfaction surveys in the it sector. this study found that intrinsic motivation factors have far more influence on one’s job satisfaction than extrinsic factors. to undertake more accurate satisfaction surveys, organizations must invest significantly more in understanding the intrinsic motivation of their it professionals by ensuring that satisfaction surveys in particular explore the nature of the work undertaken by staff and the responsibility associated with their roles. the results showing that those born in nz and overseas did not differ statistically in their view of motivation have implications for employers. there is an acute skills shortage in the industry sector (nztech, 2016) that requires concerted action from policy-makers to improve tertiary education and to perhaps increase the number of international students studying in the information systems/technology discipline. in the short term, however, the shortage will have to be met by skilled migrants (mbie, 2015). within organizations, given our findings, consideration might be given to the following: i) firms should focus on developing and maintaining a motivated workforce to ensure top talent is retained. a lack of job satisfaction is what drives turnover intent; ii) firms should focus on the nature of the jobs that it professionals undertake if they wish to foster a motivated workforce. what they do and how much responsibility is afforded to them are the key predictors of job satisfaction; iii) firms should train supervisors to provide an empowering environment for it professionals. perceptions of how enabling supervisors are contributes significantly to job satisfaction; iv) firms should offer competitive salaries to retain top talent. while salary is not as strong a predictor of job satisfaction as the nature of the job, responsibility and satisfaction with supervisor, it still shows a positive association; and v) firms should not hesitate to employ it professionals born outside of nz. there are no significant differences between the two. these recommendations might be viewed in relation to the specificities of the nz it sector. the vast majority of nz firms, those in the it sector included, are small entities with relatively unsophisticated hr approaches. their development in terms of hr functions might be australasian journal of information systems ahmed, taskin, pauleen & parker 2017, vol 21, research article motivating information technology professionals 16 described as embryonic and in many cases uncharted (arrowsmith & parker, 2012); there is little to suggest that hr thinking has been deeply integrated into many nz firms’ strategic planning. whilst this partly reflects resourcing and structural issues, a recalibrated understanding of hr’s significance to employee outcomes such as turnover intention, productivity and long-term measures of corporate performance (e.g. huselid, 1995) would encourage tailored and more effective organizational and employment relations responses wherein managers encourage the development of satisfying job roles and of highly-motivated, autonomous staff. in nz, this might include greater utilisation of outsourced hr professional services and/or of shared hr services by small it firms. furthermore, nz’s it sector – using an oecd definition of ict and high-tech manufacturing – employs 5% of the national workforce. another one-fifth or 20,000 are tech workers in other sectors. hr strategies thus need to take account of the sector (and within-sector) location of it workers, and particularly, the characteristics of the specific jobs, career paths and opportunities in those areas. the nz tech sector produced nz$32.2 billion in total output and 9% of nz’s exports in 2015 (nztech, 2016). nz’s it sector is increasingly involved on the world stage, making effective people management in local organizations crucial. nz-based it professionals can look overseas if they are not sufficiently motivated by the work options available in nz; concomitantly, hr strategy in nz it firms needs to increasingly consider how engagement with overseas parties shapes their corporate mission and goals, and by extension, local work and job design and employment conditions. the world is arguably less concerned with nz’s size, location and organizational profile, and more with its innovation, quality and ease of doing business. cohesion on such, combined with sectoral agility on account of its relatively small size and ‘can do’ approach to innovation and work challenges, may combine with connectivity that negates geographical distance and nz’s neutrality in geopolitical terms, to emphasise competitive advantage. 5.2 implications for methodology and theory one criticism of two-factor theory concerns its method-bound nature. most successful replications have used the critical incident method, introducing natural ego defence into the equation (vroom, 1966). very few questionnaire-based studies have replicated similar results (house & wigdor, 1967). this study uses an online, self-completion questionnaire method, yet the results parallel the two-factor theory and therefore can add some weight in relieving its method-bound criticism. this study also contributes to an understanding of determinants of job satisfaction by examining it professionals who are unique in their motivational drivers in a country with some unique hr practices. the use of linkedin also extended the traditional means of data collection by reaching populations in a more convenient and cost effective way. with the changing nature of content consumption and communication, this is an important consideration for future studies. this mechanism can also potentially remove geographical barriers in studies and allow for larger samples. the use of social media profiles can also allow access to wider information about participants and their behaviour. 5.3 limitations while external validity tests indicated that the study participants are generally representative of it workers in nz, we note that the questionnaire was more suited to employees of traditionally-defined organizations, rather than individual contractors or start-ups, who may have different motivators. as this sector represents nearly 13% of the overall it workforce (statistics nz, 2013b), future studies will need to adjust the model to overcome this limitation. there were also some differences between early and late responders. later responders were younger in age and had a higher salary. as later responders were closer to non-responders (armstrong & overton, 1977), they may be under-represented among the respondents. diversity of employee characteristics and thus consumers in nz is one of the highest in the world (parker, 2013). yet, our study shows women are under-represented in the it sector australasian journal of information systems ahmed, taskin, pauleen & parker 2017, vol 21, research article motivating information technology professionals 17 despite considerable opportunities (e.g. nz human rights commission, 2012). following the 1986 immigration policy review, there has been sustained growth in the number of skilled asians entering highly-skilled occupations in nz in areas of labour shortage, including in the it sector (badkar & tuya, 2010). future studies might further examine the diversity of the sector’s workforce to better understand the context for developing meaningful diversity/inclusivity management and ‘best fit’ hr strategies that can progress competitive organizational behaviour. it has been argued that “the discourse of managing diversity, emerging from the us management literature, cannot be simply mapped on to organizations in other cultural contexts” (jones, pringle & shepherd, 2000: 364). diversity issues (including a sense of ‘self-sufficiency’ and innovative spirit) reflect cultural assumptions that are specific to nz, and require a multi-voiced discourse to focus attention on dynamic local demographics, cultural and political differences. 6 conclusion it firms wishing to remain competitive must prioritise the acquisition and retention of talent (dobbs, koller, & ramaswamy, 2015). this can only be done when they understand what motivates talent; otherwise they risk becoming strategically irrelevant (guttentag, 2015). even where strategic irrelevance is not an imminent threat, a lack of motivation compromises performance (kanwar et al., 2012). the rapidly changing nature of organizations and the role that it plays in shaping them, coupled with nz’s vision of becoming a knowledge economy, means the benefits of a motivated workforce are significant to both firms and policy-makers. the study findings indicate that the best predictor of motivation for the nz it professionals is the nature of the work they undertake, alongside their perception of responsibility afforded to them, and their supervisor’s commitment to ensuring such an environment. to a lesser extent, salary also plays a role in job satisfaction. surprisingly, there was no statistically significant evidence of opportunities for advancement, professional growth opportunities, or recognition playing a role in job satisfaction. it also emerged that females are more satisfied than males on most dimensions of job satisfaction. while satisfaction drops off as people age, it does not appear to lead to increased turnover intention. academic qualifications, whether a person is born in nz, tenure, experience, salary, bonuses, and professional membership did not play a significant role in the responses. figure 3 summarises the findings. figure 3: summary of findings australasian journal of information systems ahmed, taskin, pauleen & parker 2017, vol 21, research article motivating information technology professionals 18 the absence of studies on job motivation in the nz it sector helped to motivate this study. one plausible explanation for the lack of empirical work may concern the blurred definition of the it sector in this small nation. in nz, an extensive body of studies on the it sector hail from different economies. furthermore, the it sector in nz exhibits some unique attributes including many small and micro-entities; a relative absence of sophisticated, in-house hr functions/departments and limited use of outsourced hr capacity; and a somewhat uniquely strong sense of entrepreneurial spirit among nz workers generally. additional challenges are posed by the high level of diversity of the nz it workforce. furthermore, there may be a distinct set of motivators between the various different subgroups. our sample size did not allow for pls-sem to be repeated on sub-groups. a larger sample size may allow for the examination of tendencies within sub-groups so as to shed additional light on this. this may also lead to reduced risk of possible non-response bias from younger and higher-paid professionals. this study adopts and validates a modified version of herzberg’s well-known two-factor theory for the nz setting. future studies could examine the theory in other sectors or enhance the theoretical model by including other constructs such as performance. such studies can provide support to the universality or otherwise of herzberg’s theory. researchers could also test the model for global companies or virtual companies with distributed workforces to discover whether theory can explain the motivation for this type of workforce. an opportunity exists to undertake a comprehensive qualitative study to gain further insight into the results; quantitative analysis in this study did not find statistically significant effects for personal and job characteristics. references abelson, m. a., & baysinger, b. d. (1984). optimal and dysfunctional turnover: toward an organizational level model. academy of management review, 9(2), 331-341. doi: 10.5465/amr.1984.4277675 absoluteit. (2015). is nz's gender gap in tech as bad as we think? 29 july. retrieved on 7 february from https://www.absoluteit.co.nz/2015/07/nzs-gender-gap-in-tech-bad-aswe-think/ adams, j. s. (1965). inequity in social exchange. in l. berkowitz (ed.), advances in experimental social psychology (vol. 2, pp. 267-299). new york: academic press. alderfer, c. p. (1969). an empirical test of a new theory of human needs. organizational behavior & human performance, 4(2), 142-175. allen, n. j., & meyer, j. p. (1993). organizational commitment: evidence of career stage effect. journal of business research, 26(1), 49-61. alniaçik, e., alniaçik, ü., erat, s., & akçin, k. (2013). does person-organization fit moderate the effects of affective commitment and job satisfaction on turnover intentions? social and behavioral sciences, 99(6), 274-281. doi: 10.1016/j.sbspro.2013.10.495 andrews, i. r., & henry, m. m. (1963). management attitudes toward pay. industrial relations: a journal of economy and society, 3(1), 29-39. anjali, k. t., & anand, d. (2015). intellectual stimulation and job commitment: a study of it professionals. iup journal of organizational behavior, 14(2), 28-41. armstrong, j. s., & overton, t. s. (1977). estimating non response bias in mail surveys. journal of marketing research, 14(3), 396-402. arrowsmith, j., & parker, j. (2012). good work and good workplaces. in r. ryan and j. bryson (eds), critical perspectives on hrm: a primer (chapter 7, pp172-203). auckland: pearson educational. australasian journal of information systems ahmed, taskin, pauleen & parker 2017, vol 21, research article motivating information technology professionals 19 badkar, j. & tuya, c. (2010). the asian workforce in new zealand's economy. published paper in the labour, employment and work in new zealand conference proceedings, wellington. retrieved on 6 february 2017 from file:///c:/users/irruja/downloads/1721-2198-1-sm.pdf bandura, a. (1977). self-efficacy: toward a unifying theory of behavioral change. psychological review, 84(2), 191-215. doi: 10.1037/0033-295x.84.2.191 bandura, a. (2000). cultivate self-efficacy for personal and organizational effectiveness. in e. a. locke (ed.), the blackwell handbook of principles of organizational behavior (pp. 120-136). malden, ma: blackwell publishers ltd. barclay, s., todd, c., finlay, i., grande, g., & wyatt, p. (2002). not another questionnaire! maximizing the response rate, predicting non-response and assessing non-response bias in postal questionnaire studies of gps. family practice, 19(1), 105-111. bartholomew, d. j. (1972). the effect of changes in quits and hires on the length-of-service composition of employed workers: a comment on stoikov's paper. british journal of industrial relations, 10(1), 130-133. barzilay, o., & urquhart, c. (2014). understanding reuse of software examples: a case study of prejudice in a community of practice. information and software technology, 56(12), 16131628. doi: 10.1016/j.infsof.2014.02.013 becker, j.-m., klein, k., & wetzels, m. (2012). hierarchical latent variable models in pls-sem: guidelines for using reflective-formative type models. long range planning(45), 359394. doi: 10.1016/j.lrp.2012.10.001 beecham, s., baddoo, n., hall, t., robinson, h., & sharp, h. (2008). motivation in software engineering: a systematic literature review. information and software technology, 50, 860-878. doi: 10.1016/j.infsof.2007.09.004 benevenuto, f., rodrigues, t., cha, m., & almeida, v. (2012). characterizing user navigation and interactions in online social networks. information sciences, 195(july), 1-24. doi: 10.1016/j.ins.2011.12.009 boxall, p., macky, k., & rasmussen, e. (2003). labour turnover and retention in new zealand: the causes and consequences of leaving and staying with employers. asia pacific journal of human resources, 41(2), 196-214. bruce, g. d., bonjean, c. m., & williams jr., j. a. (1968). job satisfaction among independent businessmen. sociology and social research, 52(3), 195-202. bryman, a., & bell, e. (2011). business research methods (3rd ed.): oxford university press. carrell, m. r., & elbert, n. f. (1974). some personal and organizational deterinants of job satisfaction of postal clerks. academy of management journal, 17(2), 368-373. doi: 10.2307/254991 chabrow, e. (2007). numbers show big decline of women in it. cio insight. chin, w. w. (1998). the partial least squares approach to structural equation modeling. in g. a. marcoulides (ed.), modern methods for business research (pp. 1295-1336). mahwah, nj: lawrence erlbaum associates. chin, w. w., & newsted, p. r. (1999). structural equation modeling analysis with small samples using partial least squares. in r. hoyle (ed.), statistical strategies for small sample research (pp. 1307-1341). thousand oaks, ca: sage publications. ciavolino, e. (2012). general distress as second order latent variable estimated through plspm approach. electronic journal of applied statistical analysis, 5(3), 458-464. doi: 10.1285/i20705948v5n3p458 australasian journal of information systems ahmed, taskin, pauleen & parker 2017, vol 21, research article motivating information technology professionals 20 clark, a. e. (1997). job satisfaction and gender: why are women so happy at work? labour economics, 4(4), 341-372. cobanoglu, c., warde, b., & moreo, p. j. (2001). a comparison of mail, fax and web-based survey methods. international journal of market research, 43(4), 441. cohen, a. (1993). work commitment in relation to withdrawal intentions and union effectiveness. journal of business research, 26(1), 75-90. collie, r. j., shapka, j. d., perry, n. e., & martin, a. j. (2015). teachers’ psychological functioning in the workplace: exploring the roles of contextual beliefs, need satisfaction, and personal characteristics. journal of educational psychology. doi: 10.1037/edu0000088 deci, e. l., koestner, r., & ryan, r. m. (1999). a meta-analytic review of experiments examining effects of extrinsic rewards on intrinsic motivation. psychological bulletin, 125(6), 627-668. doi: 10.1037/0033-2909.125.6.627 devellis, r. f. (2012). scale development: theory and applications. los angeles, ca: sage. diamantopoulos, a., & siguaw. (2006). formative versus reflective indicators in organizational measure development: a comparison and empirical illustration. british journal of management, 17(4), 263-282. dillman, d. a. (1991). the design and administration of mail surveys. annual review of sociology. 17(1), 225-249 direnzo, m. s., & greenhaus, j. h. (2011). job search and voluntary turnover in a boundary less world: a control theory perspective. the academy of management review, 36(3), 567-589. dobbs, r., koller, t., & ramaswamy, s. (2015). the future and how to survive it: corporate profits are beginning a long slide. prepare for leaner times. harvard business review, 93(10), 48-62. efron, b., & tibshirani, r. (1993). an introduction to the bootstrap. new york: chapman hall. ferratt, t. w., & short, l. e. (1986). are information systems people different? an investigation of motivational differences. management information systems quarterly, 10(4), 377-387. fornell, c., & larcker, d. f. (1981). evaluating structural equation models with unobservable variables and measurement error. journal of marketing research(18), 39-50. gersick, c. j. g. (1988). time and transition in work teams: toward a new model of group development. academy of management journal, 31(1), 9-41. doi: 10.2307/256496 ghazzawi, i. a. (2008). job satisfaction among information technology professionals in the us: an empirical study. journal of american academy of business, cambridge, 13(1), 115. gibson, j. l., & klein, s. m. (1970). employee attitudes as a function of age and length of service: a reconceptualization. academy of management journal, 13(4), 411-425. doi: 10.2307/254831 glebbeek, a. c., & bax, e. h. (2004). is high employee turnover really harmful? an empirical test using company records. academy of management journal, 47(2), 277-286. glenn, n. d., taylor, p. a., & weaver, c. n. (1977). age and job satisfaction among males and females: a multivariate, multisurvey study. journal of applied psychology, 62(2), 189193. doi: 10.1037/0021-9010.62.2.189 goldstein, d. k., & rockart, j. f. (1984). an examination of work-related correlates of job satisfaction in programmer/analysts. mis quarterly, 8(2), 103-115. doi: 10.2307/249347 australasian journal of information systems ahmed, taskin, pauleen & parker 2017, vol 21, research article motivating information technology professionals 21 gordon, m. e., pryor, n. m., & harris, b. v. (1974). an examination of scaling bias in herzberg's theory of job satisfaction. organizational behavior & human performance, 11(1), 106-121. griffeth, r. w., hom, p. w., & gaertner, s. (2000). a meta-analysis of antecedents and correlates of employee turnover: update, moderator tests, and research implications for the next millennium. journal of management, 26(3), 463-488. guthrie, j. p. (2001). high-involvement work practices, turnover, and productivity: evidence from new zealand. academy of management journal, 44(1), 180-190. guttentag, d. (2015). airbnb: disruptive innovation and the rise of an informal tourism accommodation sector. current issues in tourism, 18(12), 1192-1217. doi: 10.1080/13683500.2013.827159 hair, j. f., black, w., babin, b., & anderson, r. (2010). multivariate data analysis. upper saddle river, nj: pearson. hair, j. f., ringle, c. m., & sarstedt, m. (2013). partial least squares structural equation modeling: rigorous applications, better results and higher acceptance. long range planning(46), 1-12. doi: 10.1016/j.lrp.2013.01.001 hair, j. f., sarstedt, m., pieper, t. m., & ringle, c. m. (2012). the use of partial least squares structural equation modeling in strategic management research: a review of past practices and recommendations for future applications. long range planning(45), 320340. doi: 10.1016/j.lrp.2012.09.008 haire, m., ghiselli, e. e., & porter, l. w. (1966). managerial thinking: an international study. new york: wiley. hardin, e. (1965). perceived and actual changes in job satisfaction. journal of applied psychology, 49(2), 363-367. herzberg, f. (1968). one more time: how do you motivate employees? harvard business review, 46(1), 53-62. herzberg, f. (2003). one more time: how do you motivate employees? harvard business review, 81(1), 87-96. herzberg, f., mausner, b., & snyderman, b. (1959). the motivation to work. new york: john wiley & sons, inc. hewson, c., & laurent, d. (2008). research design tools for internet research. in n. fielding, r. m. lee, & g. blank (eds.), the sage handbook of online research methods. london: sage. house, r. j., & wigdor, l. a. (1967). herzberg's dual-factor theory of job satisfaction and motivation: a review of the evidence and a criticism. personnel psychology, 20(4), 369389. huselid, m.a. (1995). the impact of human resource management pracgices on turnover, productivity, and corporate financial performance. the academy of management journal, 38(3), 635-672. im, j. h., & hartman, s. (1990). rethinking the issue of whether is people are different from non-is people. mis quarterly, 14(1), 1-2. immigration new zealand. (2015, january 14, 2015). job markets and key industries: information technology. retrieved july 11, 2015, from https://www.newzealandnow.govt.nz/work-in-nz/nz-jobs-industries/informationtechnology-jobs jiang, j. j., & klein, g. (2002). a discrepancy model of information system personnel turnover. journal of management information systems, 19(2), 249-272. australasian journal of information systems ahmed, taskin, pauleen & parker 2017, vol 21, research article motivating information technology professionals 22 johnson, g. j., & johnson, w. r. (2000). perceived overqualification and dimensions of job satisfaction: a longitudinal analysis. the journal of psychology: interdisciplinary and applied, 134(5), 537-555. doi: 10.1080/00223980009598235 jones, d., pringle, j., & shepherd, d. (2000). managing diversity meets aotearoa/new zealand. personnel review, 29(3), 364-380. judge, t. a., thoresen, c. j., bono, j. e., & patton, g. k. (1998). the job satisfaction job performance relationship. paper presented at the annual meeting of the academy of management, san diego, ca. jung, k., moon, m. j., & hahm, s. d. (2007). do age, gender, and sector affect job satisfaction? results from the korean labor and income panel data. review of public personnel administration, 27(2), 125-146. jyothi, b. s., & ravindran, p. t. (2012). employee job satisfaction in software and ites units in bangalore an empirical study. business management dynamics, 2(6), 46-55. kanwar, y. p. s., singh, a. k., & kodwani, a. d. (2012). a study of job satisfaction, organizational commitment and turnover intent among the it and ites sector employees. vision (09722629), 16(1), 27-35. doi: 10.1177/097226291201600103 kifle, t., kler, p., & shankar, s. (2016). immigrant job satisfaction: the australian experience. international journal of manpower, 37(1) 99-114. kluger, m. t., & bryant, j. (2008). job satisfaction, stress and burnout in anaesthetic technicians in new zealand. anaesthesia & intensive care, 36(2), 214-221. kock, n., & lynn, g. s. (2012). lateral collinearity and misleading results in variance-based sem: an illustration and recommendations. journal of the association for information systems, 13(7), 545-580. kristof-brown, a. l., zimmerman, r. d., & johnson, e. c. (2005). consequences of individuals' fit at work: a meta-analysis of person-job, person-organization, persongroup, and person-supervisor fit. personnel psychology, 58(2), 281-342. doi: 10.1111/j.1744-6570.2005.00672.x kundrátová, i. (2011). motivating social workers: the significant factors affecting practical social work. výzvy pre sociálne profesie v modernej spoločnosti, 94-106. lambert, s. j. (1991). the combined effects of job and family characteristics on the job satisfaction, job involvement, and intrinsic motivation of men and women workers. journal of organizational behavior, 12(4), 341-363. lambert, e. g., hogan, n. l., & barton, s. m. (2001). the impact of job satisfaction on turnover intent: a test of a structural measurement model using a national sample of workers. social science journal, 38(2), 233-250. lange, t. (2012). job satisfaction and self-employment: autonomy or personality? small business economics, 38(2), 165-177. doi: 10.1007/s11187-009-9249-8 lee, r., & wilbur, e. r. (1985). age, education, job tenure, salary, job characteristics, and job satisfaction: a multivariate analysis. human relations, 38(8), 781-791. doi: 10.1177/001872678503800806 lindsay, c. a., marks, e., & gorlow, l. (1967). the herzberg theory: a critique and reformulation. journal of applied psychology, 51(4), 330-339. locke, e. a. (1977). the myths of behavior mod in organizations. academy of management review, 2(4), 543-553. doi: 10.5465/amr.1977.4406724 locke, e. a., & latham, g. p. (2002). building a practically useful theory of goal setting and task motivation: a 35-year odyssey. american psychologist, 57(9), 705-717. doi: 10.1037/0003-066x.57.9.705 australasian journal of information systems ahmed, taskin, pauleen & parker 2017, vol 21, research article motivating information technology professionals 23 locke, e. a., & latham, g. p. (2004). what should we do about motivation theory? six recommendations for the twenty-first century. academy of management review, 29(3), 388-403. doi: 10.5465/amr.2004.13670974 lum, l., kervin, j., clark, k., reid, f., & sirola, w. (1998). explaining nursing turnover intent: job satisfaction, pay satisfaction, or organizational commitment? journal of organisational behaviour, 19(3), 305-320. magee, w. (2013). anxiety, demoralization, and the gender difference in job satisfaction. sex roles, 69(5-6), 308-322. doi: 10.1007/s11199-013-0297-9 mak, b. l., & sockel, h. (2001). a confirmatory factor analysis of is employee motivation and retention. information & management, 38(5), 265-276. doi: 10.1016/s03787206(00)00055-0 maslow, a. h. (1943). a theory of human motivation. psychological review, 50(4), 370-396. doi: 10.1037/h0054346 mcclelland, d. c. (1961). the achieving society. princeton, nj: van nostrand. mcclelland, d. c. (1965). achievement motivation can be developed. harvard business review, 43(6), 6-24. mcclelland, d. c., & burnham, d. h. (1995). power is the great motivator. harvard business review, 73(1), 126-139. mcmurtrey, m. e., grover, v., teng, j. t. c., & lightner, n. j. (2002). job satisfaction of information technology workers: the impact of career orientation and task automation in a case environment. journal of management information systems, 19(2), 273-302. doi: 10.1080/07421222.2002.11045719 ministry of business innovation and employment. (2015). ict report 2015. new zealand sectors report series. retrieved 11 february 2016, from http://www.mbie.govt.nz/infoservices/business/business-growth-agenda/sectors-reports-series/pdf-imagelibrary/information-and-communications-technologyreport/2015%20information%20and%20communication%20technology%20report.p df mone, m. a. (1994). relationships between self-concepts, aspirations, emotional responses, and intent to leave a downsizing organization. human resource management, 33(2), 281-298. moore, j. e. (2000). one road to turnover: an examination of work exhaustion among technology professionals. . mis quarterly, 24(1), 141-168. doi: 10.2307/3250982 moqbel, m., nevo, s., & kock, n. (2013). organization members' use of social networking sites and job performance. information technology & people, 26(3), 240-264. doi: 10.1108/itp-10-2012-0110 myers, m. s. (1964). who are your motivated workers? harvard business review, 42(1), 7388. nitti, m., & ciavolino, e. (2014). a deflated indicators approach for estimating second-order reflective models through pls-pm: an empirical illustration. journal of applied statistics, 41(10), 2222-2239. doi: 10.1080/02664763.2014.909786 nixon, h., & saphores, j.-d. m. (2007). financing electronic waste recycling californian households’ willingness to pay advanced recycling fees. journal of environmental management, 84(4), 547-559. doi: 10.1016/j.jenvman.2006.07.003 nunnally, j. c., & bernstein, i. h. (1994). psychometric theory. new york: mcgraw-hill. nz human rights commission. (2012). nz census of women's participation. november: wellington: nz human rights commission. australasian journal of information systems ahmed, taskin, pauleen & parker 2017, vol 21, research article motivating information technology professionals 24 nz it industry. (2015). nz it industry. retrieved july 11, 2015, from https://www.linkedin.com/grp/home?gid=1522397 nz technology industry association (nz tech). (2016). digital nation new zealand: from tech sector to digital. second edition. retrieved on 6 february, 2017, from http://www.epageflip.net/i/722403-from-tech-sector-to-digital-nation-2nd-editionebook oladotun, k. j., & öztüren, a. (2013). motivational factors of hospital employees: evidence from north cyprus. interdisciplinary journal of contemporary research in business. 4(12), 106-123. oshagbemi, t., & hickson, c. (2003). some aspects of overall job satisfaction: a binomial logit model. journal of managerial psychology, 18(4), 357-367. parker, j. (2013). defining new zealand’s workforce and workplace. in j. parker (general ed.) with j. arrowsmith, hrm and employment relations in new zealand: the big issues (chapter 3, pp. 37-56) auckland: cch. patton, s. (2012). where did your graduate students end up? linkedin knows. the chronicle of higher education, 58(39). petter, s., straub, d., & rai, a. (2007). specifying formative constructs in information systems research. mis quarterly, 31(4), 623-656. porter, l. w., & lawler, e. e. (1968). managerial attitudes and performance. homewood, il: irwin-dorsey. rajagopal, n., & abraham, s. (2009). prominence of higher order needs: an indian it sector experience. vilakshan: the ximb journal of management, 6(2), 15-28. ramachndran, s., & rao, s. v. (2006). an effort towards identifying occupational culture among information systems professionals. paper presented at the acm sigmis cpr conference on computer personnel research, claremont, ca. rigdon, e. e. (2012). rethinking partial least squares path modeling: in praise of simple methods. long range planning(45), 3341-3358. doi: 10.10.16/j.lrp.2012.09.010 ringle, c. m., sarstedt, m., & straub, d. w. (2012). editor's comments: a critical look at the use of pls-sem. mis quarterly, 36(1), iii-xiv. robinson, s. d., sinar, e., & winter, j. (2014). social media as a tool for research: a turnover application using linkedin. tip: the industrial-organizational psychologist, 52(1), 133-141. roethlisberger, f., & dickson, w. (1939). management and the worker. cambridge, ma: harvard university press. ryan, r. (2012). contextualising hrm in nz. in r. ryan and j. bryson (eds), critical perspectives on hrm: a primer (chapter 3). auckland: pearson educational. saari, l. m., & judge, t. a. (2004). employee attitudes and job satisfaction. human resource management, 43(4), 395-407. doi: 10.1002/hrm.20032 saleh, s. d. (1964). study of attitude change in the preretirement period. journal of applied psychology, 48, 310-312. doi: 10.1037/h0042932 schambach, t., & blanton, j. e. (2002). the professional development challenge for it professionals. communications of the acm, 45(4), 83-87. doi: 10.1145/505248.505250 schor, j. (2008). the overworked american: the unexpected decline of leisure. new york: basic books. schuster, j. r., clark, b., & rogers, m. (1971). testing portions of the porter and lawler model regarding the motivational role of pay. journal of applied psychology, 55(3), 187-195. australasian journal of information systems ahmed, taskin, pauleen & parker 2017, vol 21, research article motivating information technology professionals 25 shiu, e., hassan, l. m., & parry, s. (2015). the model effects of national age stereotyping on the relationship between job satsifaction and its determinants: a study of older workers across 26 countries. british journal of management, 26(1), 255-272. doi: 10.1111/14678551.12091 shuck, b., twyford, d., reio, t. g., jr., & shuck, a. (2014). human resource development practices and employee engagement: examining the connection with employee turnover intentions. human resource development quarterly, 25(2), 239-270. singh, p., & loncar, n. (2010). pay satisfaction, job satisfaction and turnover intent. relations industrielles / industrial relations, 65(3), 470-490. smerek, r., & peterson, m. (2007). examining herzberg’s theory: improving job satisfaction among non-academic employees at a university. research in higher education, 48(2), 229-250. doi: 10.1007/s11162-006-9042-3 smith, d. b., & shields, j. (2013). factors related to social service workers' job satisfaction: revisiting herzberg's motivation to work. administration in social work, 37(2), 189198. doi: 10.1080/03643107.2012.673217 smith, d. c., & speight, h. l. (2006). antecedents of turnover intention and actual turnover among information systems personnel in south africa. paper presented at the 2006 acm sigmis cpr conference on computer personnel research, germany. souder, w. e., buisson, d., & garrett, t. (1997). success through customer‐driven new product development: a comparison of us and new zealand small entrepreneurial high technology firms. journal of product innovation management, 14(6), 459-472. statistics nz. (2013a). new zealand it demographics by sex. (job-05804). statistics nz. (2013b). occupation (anzsco) (ict groups) and status in employment by age groups and sex. (job-07133). stoikov, v. (1971). the effect of changes in quits and hires on the length-of-service composition of employed workers. british journal of industrial relations, 9(2), 225-233. straub, d. w., boudreau, m.-c., & gefen, d. (2004). validation guidelines for is positivist research. communications of the ais(13), 380-427. stuart, f. i., verville, j., & taskin, n. (2012). trust in buyer-supplier relationships. journal of enterprise information management, 25(4), 392-412. sykes, a. j. m. (1965). economic interest and the hawthorne researches. human relations, 18(3), 253-263. tang, t. l.-p., & gilbert, p. r. (1995). attitudes toward money as related to intrinsic and extrinsic job satisfaction, stress and work-related attitudes. personality and individual differences, 19(3), 327-332. doi: 10.1016/0191-8869(95)00057-d tanner, f. r. (2003). managing technologically driven organizations, the human side of innovation and change. paper presented at the engineering management conference. taylor, g. s., & vest, m. j. (1992). pay comparisons and pay satisfaction among public sector employees. public personnel management, 21(4), 445-454. tietjen, m. a., & myers, r. m. (1998). motivation and job satisfaction. management decision, 36(4), 226-231. doi: 10.1108/00251749810211027 toulson, p., & foster, b. (2013). the relevance of historical context to hrm and employment relations. in j. parker (general ed.) with j. arrowsmith, hrm and employment relations in new zealand: the big issues (chapter 2, pp. 15-35) auckland: cch. tryon, w. w. (1981). a methodological critique of bandura's self-efficacy theory of behavior change. journal of behavior therapy and experimental psychiatry, 12(2), 113-114. doi: 10.1016/0005-7916(81)90003-3 australasian journal of information systems ahmed, taskin, pauleen & parker 2017, vol 21, research article motivating information technology professionals 26 tzeng, h.-m. (2002). the influence of nurses’ working motivation and job satisfaction on intention to quit: an empirical investigation in taiwan. international journal of nursing studies, 39, 867-878. doi: 10.1016/s0020-7489(02)00027-5 urbach, n., & ahlemann, f. (2010). structural equation modeling in information systems research using partial least squares. journal of information technology theory and application, 11(2), 5-40. verquer, m. l., beehr, t. a., & wagner, s. h. (2003). a meta-analysis of relations between person-prganization fit and work attitudes. journal of vocational behavior, 63(3), 473489. volkwein, j. f., & zhou, y. (2003). testing a model of administrative job satisfaction. research in higher education, 44(2), 149-171. vroom, v. h. (1964). work and motivation. new york: wiley. vroom, v. h. (1966). some observations regarding herzberg's two-factor theory. paper presented at the american psychological association convention, new york. wagner, j. a., rubin, p. a., & callahan, t. j. (1988). incentive payment and nonmanagerial productivity: an interrupted time series analysis of magnitude and trend. organizational behavior and human decision processes, 42(1), 47-74. doi: 10.1016/0749-5978(88)90019-2 wahba, m. a., & bridwell, l. g. (1976). maslow reconsidered: a review of research on the need hierarchy theory. organizational behavior & human performance, 15(2), 212-240. wallace, j., & gaylor, k. p. (2012). a study of the dysfunctional and functional aspects of voluntary employee turnover. sam advanced management journal (07497075), 77(3), 27-36. wickström, g., & bendix, t. (2000). the" hawthorne effect"—what did the original hawthorne studies actually show? scandinavian journal of work, environment & health, 363-367. williamson, d. a. (1996). job satisfaction in social services. new york: garland. wilson, p. m., rodgers, w. m., carpenter, p. j., hall, c., hardy, j., & fraser, s. n. (2004). the relationship between commitment and exercise behavior. psychology of sport and exercise(5), 405-421. doi: 10.1016/s1469-0292(03)00035-9 wynekoop, j. l., & walz, d. b. (1998). revisiting the perennial question: are is people different? the database for advances in information systems, 29(2), 67-72. copyright: © 2017 ahmed, taskin, pauleen & parker. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems ahmed, taskin, pauleen & parker 2017, vol 21, research article motivating information technology professionals 27 australasian journal of information systems ahmed, taskin, pauleen & parker 2017, vol 21, research article motivating information technology professionals 28 appendices rec wrk adv gth rsp gfo mis smg spv cwr sal ben val sat trn rec_1 .794 .406 .157 .212 .013 .016 .054 .232 .033 .036 .054 .113 .080 .340 .077 rec_2 .821 .411 .251 .098 .206 .068 .071 .177 .035 .010 .050 .099 .121 .451 .072 rec_3 .733 .085 .011 .037 .334 .150 .236 .039 .096 .127 .030 .093 .109 .090 .045 rec_4 .819 .347 .144 .206 .112 .045 .174 .138 .003 .109 .108 .007 .145 .372 .155 rec_5 .710 .441 .289 .076 .038 .007 .099 .264 .018 .058 .025 .137 .049 .379 .037 wrk_1 .139 .940 .146 .084 .062 .126 .049 .068 .028 .005 .002 .062 .093 .019 .070 wrk_2 .005 .963 .063 .086 .141 .021 .064 .009 .042 .007 .022 .019 .041 .074 .053 wrk_3 .028 .961 .074 .047 .026 .004 .053 .028 .031 .045 .039 .045 .010 .101 .043 wrk_4 .113 .872 .146 .133 .250 .110 .077 .095 .051 .036 .017 .039 .156 .214 .031 adv_1 .085 .214 .840 .267 .057 .109 .192 .168 .121 .038 .014 .150 .159 .278 .006 adv_2 .064 .096 .851 .018 .212 .163 .067 .001 .145 .129 .027 .047 .039 .015 .153 adv_3 .113 .016 .819 .033 .126 .137 .277 .144 .119 .137 .020 .085 .043 .135 .111 adv_4 .108 .120 .708 .258 .467 .167 .012 .031 .168 .042 .026 .220 .192 .469 .305 gth_1 .038 .068 .040 .884 .065 .110 .042 .108 .051 .015 .023 .025 .071 .104 .017 gth_2 .010 .052 .033 .883 .022 .176 .091 .116 .038 .008 .017 .011 .027 .128 .049 gth_3 .014 .044 .068 .868 .115 .007 .298 .065 .090 .023 .133 .085 .047 .020 .010 gth_4 .056 .027 .003 .879 .131 .178 .222 .080 .219 .024 .143 .010 .062 .058 .033 gth_5 .046 .115 .153 .793 .078 .130 .228 .089 .044 .059 .032 .063 .092 .171 .026 rsp_1 .101 .048 .101 .018 .891 .084 .019 .125 .060 .026 .070 .044 .044 .064 .059 rsp_2 .099 .064 .077 .039 .907 .001 .138 .041 .010 .043 .048 .102 .012 .191 .047 rsp_3 .010 .064 .046 .016 .897 .035 .205 .041 .055 .058 .060 .007 .070 .075 .114 rsp_4 .087 .024 .086 .013 .811 .039 .215 .014 .062 .004 .039 .083 .131 .304 .106 rsp_5 .074 .064 .109 .058 .829 .090 .182 .058 .045 .040 .024 .010 .018 .061 .008 gfo_1 .196 .155 .289 .055 .007 .881 .029 .065 .037 .100 .008 .108 .038 .266 .093 gfo_2 .016 .113 .225 .160 .193 .876 .071 .095 .000 .006 .072 .077 .163 .274 .104 gfo_3 .050 .091 .111 .012 .007 .933 .117 .042 .038 .040 .016 .061 .025 .123 .030 gfo_4 .037 .004 .109 .058 .006 .955 .019 .069 .079 .030 .051 .013 .088 .026 .018 gfo_5 .124 .183 .287 .143 .192 .849 .047 .042 .169 .020 .043 .110 .081 .453 .191 mis_1 .026 .120 .044 .017 .097 .047 .886 .201 .197 .151 .099 .014 .103 .303 .218 mis_2 .020 .096 .017 .010 .158 .027 .910 .181 .138 .040 .083 .012 .046 .196 .115 mis_3 .052 .080 .013 .023 .108 .024 .918 .180 .081 .021 .062 .038 .042 .151 .070 mis_4 .110 .063 .063 .105 .380 .155 .810 .108 .270 .022 .186 .014 .008 .197 .120 mis_5 .001 .260 .103 .094 .019 .056 .721 .476 .023 .155 .010 .019 .086 .361 .255 mis_6 .001 .037 .104 .018 .044 .004 .795 .317 .175 .076 .079 .073 .034 .208 .102 smg_1 .018 .023 .025 .013 .035 .055 .022 .964 .020 .018 .002 .039 .080 .016 .056 smg_2 .064 .027 .080 .025 .001 .017 .069 .967 .081 .030 .015 .046 .055 .093 .055 smg_3 .047 .051 .107 .039 .036 .038 .048 .950 .102 .048 .018 .007 .026 .079 .001 spv_1 .007 .050 .075 .058 .054 .083 .107 .124 .951 .035 .015 .023 .109 .025 .006 spv_2 .055 .022 .019 .161 .137 .068 .139 .099 .922 .083 .010 .003 .026 .131 .101 spv_3 .086 .021 .073 .078 .065 .098 .086 .162 .940 .046 .021 .030 .073 .030 .059 spv_4 .055 .001 .050 .039 .091 .119 .048 .035 .964 .009 .003 .005 .039 .013 .011 spv_5 .044 .025 .024 .017 .053 .017 .060 .030 .964 .073 .039 .020 .031 .063 .028 spv_6 .050 .019 .121 .054 .071 .059 .004 .102 .947 .049 .016 .061 .065 .059 .004 spv_7 .007 .066 .091 .022 .141 .100 .042 .029 .913 .021 .026 .024 .040 .094 .107 australasian journal of information systems ahmed, taskin, pauleen & parker 2017, vol 21, research article motivating information technology professionals 29 rec wrk adv gth rsp gfo mis smg spv cwr sal ben val sat trn spv_8 .050 .023 .017 .153 .029 .047 .006 .099 .911 .119 .054 .110 .083 .051 .066 spv_9 .096 .102 .116 .073 .045 .064 .088 .038 .938 .036 .080 .031 .004 .028 .019 spv_11 .041 .055 .023 .143 .098 .072 .048 .007 .937 .056 .042 .025 .020 .038 .049 spv_12 .220 .069 .083 .071 .093 .045 .045 .013 .868 .134 .013 .113 .022 .064 .156 spv_13 .039 .024 .101 .057 .112 .122 .132 .067 .919 .071 .004 .051 .027 .022 .037 spv_14 .006 .007 .054 .113 .168 .100 .179 .182 .910 .014 .020 .005 .021 .051 .020 spv_15 .055 .112 .114 .183 .032 .219 .110 .137 .842 .155 .068 .062 .069 .029 .025 cwr_1 .032 .227 .038 .057 .070 .026 .071 .116 .124 .886 .070 .026 .021 .056 .050 cwr_2 .003 .009 .103 .146 .147 .234 .043 .157 .092 .863 .074 .049 .056 .073 .038 cwr_3 .076 .020 .093 .178 .019 .037 .035 .034 .070 .870 .145 .075 .069 .087 .011 cwr_4 .047 .109 .131 .156 .045 .029 .163 .057 .035 .907 .059 .063 .006 .058 .029 cwr_5 .017 .102 .069 .140 .042 .203 .085 .106 .146 .825 .004 .003 .111 .030 .066 cwr_6 .031 .178 .258 .177 .098 .050 .036 .147 .174 .687 .073 .093 .025 .011 .088 cwr_7 .179 .114 .033 .021 .009 .070 .231 .227 .128 .746 .339 .169 .032 .111 .041 sal_1 .048 .090 .101 .113 .051 .005 .007 .045 .043 .062 .947 .088 .003 .109 .041 sal_2 .103 .090 .018 .033 .049 .008 .070 .081 .040 .034 .950 .021 .062 .123 .069 sal_3 .180 .215 .141 .094 .119 .004 .092 .150 .003 .114 .796 .079 .071 .277 .131 ben_1 .019 .050 .032 .034 .023 .021 .045 .009 .005 .064 .153 .924 .012 .112 .034 ben_2 .173 .033 .223 .086 .105 .006 .132 .073 .026 .108 .062 .886 .024 .096 .177 ben_3 .057 .037 .259 .039 .117 .029 .199 .076 .182 .017 .216 .708 .107 .065 .012 ben_4 .152 .058 .051 .085 .037 .052 .075 .023 .172 .194 .051 .855 .100 .076 .137 val_1 .094 .093 .130 .154 .015 .077 .165 .093 .074 .001 .090 .069 .895 .102 .038 val_2 .024 .055 .081 .111 .057 .062 .035 .051 .004 .028 .017 .109 .936 .155 .040 val_3 .067 .034 .044 .037 .043 .012 .127 .144 .069 .027 .107 .043 .910 .059 .004 sat_1 .022 .158 .110 .022 .072 .006 .007 .050 .043 .041 .007 .041 .112 .945 .039 sat_2 .071 .029 .055 .023 .072 .039 .051 .002 .033 .011 .077 .028 .153 .951 .037 sat_3 .096 .192 .056 .002 .148 .034 .046 .049 .011 .031 .073 .071 .043 .924 .078 trn_1 .005 .190 .005 .003 .049 .042 .009 .050 .113 .047 .016 .087 .008 .078 .895 trn_2 .078 .044 .120 .024 .261 .167 .157 .070 .128 .034 .012 .061 .087 .147 .853 trn_3 .095 .316 .162 .036 .271 .273 .192 .023 .013 .108 .007 .039 .102 .296 .659 legend: rec recognition wrk work itself adv opportunities for advancement gth professional growth opportunities rsp responsibility gfo good feelings for organization mis clarity of mission smg effective senior management spv effective supervisor cwr good relations with co-workers sal satisfaction with salary ben satisfaction with benefits val presence of core values sat job satisfaction trn turnover intent table a1: cfa first order factor loadings australasian journal of information systems ahmed, taskin, pauleen & parker 2017, vol 21, research article motivating information technology professionals 30 vifs factor indicator sig. indicator construct intrinsic motivation recognition * 1.670 4.719 work itself * 1.875 opportunities for advancement * 1.962 professional growth opportunities * 1.800 responsibility * 2.670 good feelings about organization * 2.145 clarity of mission * 2.610 extrinsic motivation effective senior management ** 2.070 3.470 effective supervisor * 1.560 good relations with co-workers * 1.477 satisfaction with salary * 1.800 satisfaction with benefits * 1.611 presence of core values * 1.956 control variables gender ** 1.319 age ** 2.972 qualifications ** 1.285 born in ** 1.275 tenure (years) ** 1.287 experience (years) ** 3.403 salary ** 2.123 bonus ** 2.197 professional membership ** 1.336 note: *=significant at p<.01, **=significant at p<.001 table a2: indicator validity control variable welch f brown-forsythe statistic significance statistic significance gender 2.500 .120 2.500 .120 age 3.123 .034 2.903 .033 qualification 1.595 .226 1.603 .218 income 3.804 .005 3.130 .012 table a3: non-response bias test paper title (paper title style) australasian journal of information systems alahakoon 2015, vol. 19, pp. s77-s97 interactive product retrieval model a new user model based interactive product retrieval process for improved ebuying oshadi alahakoon victoria university oshadi.alahakoon@vu.edu.au abstract when searching for items online there are three common problems that e-buyers may encounter; null retrieval, retrieving unmanageable number of items, and retrieving unsatisfactory items. in the past information retrieval systems or recommender systems were used as solutions. with information retrieval systems, too rigorous filtering based on the user query to reduce unmanageable number of items result in either null retrieval or filtering out the items users prefer. recommender systems on the other hand do not provide sufficient opportunity for users to communicate their needs. as a solution, this paper introduces a novel method combining a user model with an interactive product retrieval process. the new layered user model has the potential of being applied across multiple product and service domains and is able to adapt to changing user preferences. the new product retrieval algorithm is integrated with the user model and is able to successfully address null retrieval, retrieving unmanageable number of items, and retrieving unsatisfactory items. the process is demonstrated using a bench mark dataset and a case study. finally the product retrieval process is evaluated using a set of guidelines to illustrate its suitability to current ebuying environments. keywords: user modelling; e-commerce; customer analytics; personalization 1 introduction the internet and smart phones have revolutionized the way we shop. during the past decade the number of online shoppers have grown drastically, where statistics states that 40 percent of worldwide internet users have bought products or goods online (statistica 2014). however, for these buyers, navigating massive product bases to find the best suited products can be challenging. in the past either recommender systems or information retrieval systems were employed to facilitate online purchasing. both these techniques have inherent limitations; the information retrieval systems use product attributes specified by the user as selection criteria which may result in too stringent filtering resulting in the problem of null retrieval. recommender systems on the other hand do not provide the user with sufficient opportunity to express their needs but employ personalized retrievals based on past preferences. with the massive increase in the number of products and services available on line, with numerous variations and alternatives, the capturing of the user requirements has become very complex. at the same time the continuing huge increase in the online consumer population has resulted in more individuality being required to be expressed in ebuying. this environment has also resulted in more informed ebuyers who demand more and more customization. therefore as a result of this increase in both products, services, their variations as well as massive and demanding population of ebuyers, the traditional limitations of the information retrieval systems and recommender systems have become significant problems. user profiling or modelling has been used in a wide variety of applications to identify and capture user individuality. key problems in many user models is that they do not cater for different domains where users may behave differently, and the inability to learn and adapt as users continue to transact and communicate with a system thus providing further information about themselves. a layered user model (lum) was proposed to address these key limitations in (alahakoon, loke et al. 2007). in this paper we utilize the flexibility, adaptability and multi domain usability of the lum with a better product retrieval to cater for the growing need of efficient and customized product retrieval. s77 australasian journal of information systems alahakoon 2015, vol. 19, pp. s77-s97 interactive product retrieval model the rest of the paper is structured as follows. section 2 describes the background information on information retrieval systems and recommender systems while positioning the work presented. section 3 describes the new product retrieval algorithm in detail using a benchmark dataset. section 4 demonstrates the integrated product retrieval process and the value of the new user model with a comprehensive case study. a hypothetical user is introduced with a wide ranging set of requirements to enable the demonstration of various features, functionality and advantages of the new techniques. section 5 evaluates combined user model and the product selection according to criteria critical to requirements of the current ebuyer’s needs. finally, section 6 provides a discussion and concluding remarks. 2 background 2.1 information retrieval systems vs recommender systems in information retrieval, even powerful search engines such as google or yahoo still use onesize-fits-all strategy (smyth, coyle et al. 2011). therefore there is ongoing research to improve the performance of these search engines. most information retrieval systems employ two different retrieval techniques; filter based retrieval and similarity based retrieval. filter based retrieval is carried out either using keyword search or using parametric search. keyword search use keywords in item descriptions. parametric search assigns values to product attributes as parameters where the user query is represented as a set of constraints on item descriptions. generally the user is asked to fill in a form specifying values for product attributes. then, a database query is formed where the query consists of features or feature ranges. however, both these methods narrow down the search space according to set constraints and has a high chance of null retrieval. similarity based retrieval has its roots in case based reasoning (cbr) (burke 2002) and uses a similarity matrix. a similarity metric can be any function that takes two entities and returns a value reflecting their similarity with respect to a given goal. this results in less vocabulary mismatches and null retrievals. in product retrieval, forming and implementing a similarity metric is complicated, time consuming, and inefficient at runtime. however, similarity based filtering performs well in document retrieval due to comparison of words (ahn, brusilovsky et al. 2006). a recommender system will use past preferences rather compared to the direct user requirement at the time of product search. recommender systems mainly use two different techniques in recommender engines; either content based or collaborative techniques. there are systems which use pure collaborative recommendation techniques (schafer, frankowski et al. 2007) and ones that use pure content based techniques (pazzani and billsus 2007) while some use hybrid of these techniques (burke 2007). in a few hybrid systems demographic techniques and cased based recommendation techniques were used combined with either collaborative or content based techniques. conversational recommender systems can be considered as a combination of these two types of systems. notable work is presented in (shearin and lieberman 2001), (mahmood and ricci 2009) and (thompson, göker et al. 2002). these systems use a user model and also manipulate the user query to find suitable items. our work relates to this group of recommender system since it uses a user model to enhance performance of a product retrieval algorithm. the work described in this paper combines a user model with a product retrieval system. to address the limitations of traditional systems and accomplish the current needs our research has focused on the following: design and implement a new user model architecture (lum – layered user model) which can capture complex purchase behaviour. present users with a personalized interactive product retrieval process (piprp) making use of the above designed user model. s78 australasian journal of information systems alahakoon 2015, vol. 19, pp. s77-s97 interactive product retrieval model 2.2 the layered user model (lum) the lum was first introduced in (alahakoon, loke et al. 2007). lum allows modelling highly complex individual purchasing behaviour by capturing the elements which contribute to such behaviour: demographics, purchasing history and current requirements. each element is stored as an information layer. the three layers are as follows. i. user’s personal information (demographics) based general buying behaviour information (pi layer – personal information layer) ii. user’s buying preferences in specific domains (di layer – domain based information layer) iii. user’s transaction based needs for each interaction (ti layer – transaction based information layer). pi layer of the lum is the first layer to create. when seeking personalization in a given domain the corresponding di layer get generated based on the pi layer. once the pi layer and the di layers are available, user can start interacting with the system. since the slow changing demographics are used, pi layer becomes the most stable information layer as demographics depicts an individual’s ‘position and ability’. in addition to demographics, pi layer contains personal behaviour characteristics (pbc values) (alahakoon, loke et al. 2007). pbc values, exhibit an individual’s general trends and abilities that are valid in any purchasing domain. in lum pbc values are calculated combing demographics using multi attribute utility theory (maut) (schafer 2001). a sample of pi layer is shown in figure 1. figure 1: pi layer di layer links the domain attributes to user preferences. it contains, domain attributes under each domain feature. for each attribute, user’s current and initial relevance values are recorded. the relevance value of an attribute specifies the amount of relevance of that particular attribute to the user within a given domain. in figure 2, feature id 4 has five attributes (attribute id’s 73-77) and each attribute has a current and initial relevance. s79 australasian journal of information systems alahakoon 2015, vol. 19, pp. s77-s97 interactive product retrieval model at the creation of a new di layer, initial relevance values (figure 2 “init_relevance”) are calculated using the pi layer and a matrix called influence matrix. “relevance” is the updated current relevance of the attribute, after at least one transaction. some domain attributes are directly related to the information in the pi layer. such attributes are called personal information related attributes (pir-attributes). in the restaurants domain, some intuitions relating pi layer and domain attributes are; i. a price sensitive person would be more influenced by restaurants with lower costs, a quality conscious and a socializing person would be more influenced by better décor. figure 2: a section of the di layer an influence matrix (im) is a matrix which contains all such attributes along with their influence thresholds (table 1). the weight gives the influence of the pbc value on the attribute. for example, the feature ‘cost’ has four continuous attributes which becomes relevant to users (with a weight of 1). fe at ur e a tt ri bu te n am e pr ic e se ns it iv it y (p s) ps w ei gh t q ua lit y c on sc io us ne ss (q c ) q c w ei gh t so ci al is in g (s o) so w ei gh t cost below $15 0.75-1 1 0 0 0 0 $15-$30 0.5-0.75 1 0 0 0 0 $30-$50 0.25-0.5 1 0 0 0 0 over $50 00.25 1 0 0 0 0 décor fair décor 0 0 0.15-0.35 0.5 0.15-0.35 0.5 good décor 0 0 0.35-0.55 0.5 0.35-0.55 0.5 excellent décor 0 0 0.55-0.75 0.5 0.55-0.7 0.5 table 1: a section of the influence matrix for restaurants domain s80 australasian journal of information systems alahakoon 2015, vol. 19, pp. s77-s97 interactive product retrieval model for a given feature if more than one attribute value becomes relevant to an individual, a fuzzy approach (zadeh 1965) is employed. following is an example showing how fuzzy membership functions are used in relevance value calculation. figure 3: membership functions for cost attributes based on price sensitivity example: according to the membership functions in figure 6, for a person with price sensitivity value 0.67, out of the four attributes in ‘cost’, three becomes relevant; “less than $15”, 0.68, “$15-$30”, 1 and “$30-$50”, 0.32. since the user’s ps value is quite high (0.67), the highest price range is not relevant. the last information layer (ti layer) captures the short term needs of the consumer. information in the ti layer may not agree with the more general behaviour in the first layer or the expected domain centric behaviour in the di layer. the reason being, users may deviate from their usual behaviour due to impulsive purchases caused by mood changes or unusual circumstances. figure 4 depicts the layers of information in the lum architecture. each time a user interacts in a new domain a di layer for that domain gets created. figure 4: the user model with n-layers (the middle layer has sub layers) all the di layer components of the user model are attached to the pi layer of the user model. since a domain can be a specific subject area of knowledge such as restaurants, footwear, real estate, etc., each category of products presented in a hierarchical taxonomy becomes a domain. hence the domains in which users interact can be interconnected. in real life such categorization is observed in supermarket aisles. since the di layer components represent user information in a given domain, this second layer becomes n layered. the pi layer remains common to all di layers, as it is a user’s general s81 australasian journal of information systems alahakoon 2015, vol. 19, pp. s77-s97 interactive product retrieval model purchasing behavior irrespective of a specific domain. detailed descriptions of the lum with details of its initial creation, information contents, calculations and updates are presented in (alahakoon, loke et al. 2007). 3 personalized interactive product retrieval process – (piprp) figure 5 depicts piprp as a sequence of steps. figure 5: piprp steps each step is described using an example of restaurant search. step 1: personalized initial query specification the initial preference elicitation process provides the user with a flexible interface (figure 6). figure 6: screen – users initial query selection the products are categorized under features such as ‘cuisine’ and within a feature, sub features such as ‘italian’. sub-features may have sub-sub features until the final layer consists of atomic attributes. this allows the user to specify his/her need to very detailed level of granularity, with the further possibility of selecting more than one value for a given feature (italian and indian). step 2: personalized query expansion null retrieval occurs when the deployed technique does not return any results for a given search criteria. in the literature, to avoid null retrieval several approaches have been employed. s82 australasian journal of information systems alahakoon 2015, vol. 19, pp. s77-s97 interactive product retrieval model use of similarity matrices to identify the items with similar attributes (as in entrée (burke 2001)). inclusion of items preferred by like-minded users (user-user correlation – collaborative filtering (resnick, iacovou et al. 1994)). inclusion of items similar to previous user preference (item-item correlation as in (thompson, göker et al. 2002)). however, as explained in (burke 2001) two items maybe similar due to several different reasons where the underlying reason is not visible to the selection algorithm. if user-user correlation (resnick, iacovou et al. 1994) is used, two users may prefer the same item due to two different reasons (towle and quinn 2000). in (thompson, göker et al. 2002), all the attributes present in user’s preferred items (in the past) were included in the query. although user preferred the item, it doesn’t mean s/he preferred each and every attribute. in piprp, the idea of query expansion is not only to avoid null retrieval but also to include any possible items that may satisfy the user in the results set. therefore, piprp does not look for similar items. instead, if there are any other attributes (in the user model) that belongs to the same feature (as the selected one and has positive relevance values), such attributes are included in the query. in the user model, a positive relevance value for an attribute may result due to one or both of the following reasons. i. initially allocated based on the user’s pbc values ii. updated due to implicit user preferences during past product selections. for example, higher cost range can become highly relevant to a user due to one or both of the following; i. if the pbc values describe the user as a not so price sensitive person ii. or if the user explicitly expressed preference for higher cost ranges during previous interactions in the current query if the user selects the lower cost range as preferred value, still the higher range (from the user model) gets included in the personalized query expansion. this way piprp consider both user’s past preferences and his/her demographic preferences. step 3: further filtering based on the user model – personalized filtering to further filter out the unwanted results, there are two important facts to consider. i. what is the next most important feature of the item to set a constraint on? ii. what is the preferred value for that feature? doyle and cunningham (doyle and cunningham 2000) considered the information gain of a question to unobtrusively determine the next best attribute to ask. this method only focused on the least number of questions. if the user is asked to provide his/her preference for an unimportant feature, one of the following can happen. if the user answered the question and the records were filtered depending on the answer given, important records may be lost. if the user did not answer the question, then asking an unimportant question adds to the obtrusiveness of the system. for instance, feature ‘wheelchair access facility’ (in restaurant domain) was requested from the user. even though it is of least interest, the user may answer the question positively leading to removal of all the restaurants without ‘wheelchair access facility’. among the filtered restaurants, there may-be ones that offer more desired features. filtering records based on unimportant features leads to low quality recommendations. s83 australasian journal of information systems alahakoon 2015, vol. 19, pp. s77-s97 interactive product retrieval model the pir-attributes in the user model hold very important information to the user. therefore, personalized filtering is carried out using the existing values of pir-attributes in the corresponding di layer of the user model. since only the relevant pir-attributes are to be considered as preferred or applicable to the user, ones that are with a greater than zero relevance are used in the filtering. when personalized filtering is applied, nonzero pirattributes get added to the query in step 2. step 4: further querying the user to further filter the results set, the item features which were not constrained in the previous steps needs to be considered. this can be achieved by asking the preferred attributes values for such features from the user. however, as explained in the section under step 3, the most important item features need to be constrained first to avoid losing more suitable results. in lum, most important attributes are identified by its average total relevance (atr) value. the average total relevance (atr) of an item feature is the average of all relevance values of the attributes belonging to that feature. since each feature can have varying number of attributes, average is considered. step 5: calculate similarity between the query and the items after presenting all features with positive atr values as questions, if a further unmanageable set of records exists, the system needs to filter out the less important items. for this purpose, a novel similarity measure (pertaining to the lum) is used to rank the remaining list of items, and the first three items are displayed to the user. the novel similarity measure takes following information into consideration. i. user preferences provided in the current query. ii. user preferences provided during step4. iii. user preferences available in the user model. the similarity calculation is described next. each item in the product base are represented as an n-long one dimensional array, (where n is the number of attributes used to describe the products in a given domain note all the products are described under the same set of attributes). each element can assign either 1 or 0, indicating the existence (1) or absence (0) of each attribute. the user query is also represented in the same manner: each array element with ‘1’ indicating a preferred attribute. the similarity of an item to the user query is calculated based on the total relevance value of the attributes present in that item. each attribute present in an item, contributes a sum of two relevance values to the total similarity as follows: i. relevance of an attribute, according to the current query. (simiq) if the given attribute is among the constraints provided with the initial query (provided during step 1) assign a relevance value of r1. (simpsp) if the given attribute is among the user provided constraints during the product selection process (provided during step 4) – assign a relevance value of r2. ii. (simum) if the current attribute has a relevance value in the user model di layer – obtain the actual relevance value from the di layer. here, r1 > r2. the reason is; if the preferred attributes were provided in the initial query (in step1), that implies, that those are the most important attributes to the user. since, those are the features, that user is looking for. preferences provided during step4, was obviously less relevant than the step 1 explicitly provided preferences. hence, in the example, r1 and r2 are assigned 1 and 0.75 respectively. s84 australasian journal of information systems alahakoon 2015, vol. 19, pp. s77-s97 interactive product retrieval model the total similarity of an item i ( totsimi ) which is described using n attributes to the user query is given by; totsimi = [{simiq(i) or simpsp(i)} + simum(i)] using the above formula totsimi is calculated for each item remaining in the result set. 4 demonstration of the piprp process a benchmark dataset is used to demonstrate the process. the dataset (asuncion and newman 2007) which has been used in this work was intended to use as a knowledge base of a recommender system called entrée (burke 2002). the entrée dataset contains descriptive information about restaurants in a number of cities in the usa. data is available separately for each city and log files are available indicating user browsing sessions. each restaurant is described under 256 attributes indicating the presence or absence of each attribute. in our prototype implementation we modified the data according to our requirements. for example, the 256 attributes were grouped into 31 features (as ‘cost’, ‘décor’, etc) to form questions during the search. the experiments presented in this paper, are subject to the limitations of the datasets used. it is an obvious and known fact that the retrieval process greatly depends on the available data (the distribution of products in the search space). for example, if most of the available items belong to the same price range, then retrieving items belonging to other price ranges cannot be constrained as much as constraining the items belonging to the common price range, as this might lead to null retrievals. in addition there were missing values; out of the 1875 restaurants used in experiments only 295 had cuisine descriptions. such records are prone to null retrieval. each step is described in detail in the demonstration example below. example scenarios for the demonstration were constructed for a hypothetical user (called user41). information in pi layer (demographics and pbc values) of user41 is given in table 2. in the rest of the section the advantages of the new process is demonstrated with a bench mark dataset. age 26 family single/bachelor gender m income 30k 50k occupation trade person or related work hours 21-40 hrs adventurer 0.91 family person 0.05 fun 0.67 health conscious 0.21 price sensitive 0.53 quality conscious 0.29 socializing 0.35 time saver 0.42 table 2: pi layer information belonging to user41 according to the pi layer, user41 is a young bachelor with a low income. therefore, the user is ‘expected’ to be concerned about the price. however, realistically he may have other preferences that are not apparent from only the demographic information (such as interest in food, and a desire to taste different culinary disciplines). to facilitate demonstration of how the highlighted problems are addressed by the process several potential scenarios were constructed for user41 as shown in table 3. user41’s initial di layer for restaurant domain is created (based on the pi layer) to be used in the first run of the piprp. s85 australasian journal of information systems alahakoon 2015, vol. 19, pp. s77-s97 interactive product retrieval model scenario id description 41-a1 searching a restaurant to have lunch during the work break. he is interested in food, and would like to taste different culinary disciplines for an affordable price. 41-d1 to organize a gathering of young bachelor friends to celebrate his new promotion over dinner and a drink. price is still a concern but more attention to the friendly atmosphere and distance. 41-e1 to have dinner over a chat, with a friend. price is a concern as paying for both but the atmosphere and the quality of the restaurant should be adequately good. at the back of his mind plans for watching a movie before the meal. better if parking is available. table 3: possible scenarios of interactions for user41 steps in the piprp as detailed in the previous section are now demonstrated for user 41 for the above scenarios. step 1: personalized initial query specification: user41 selects cuisine as “italian” and food quality as “near perfect food” the resultant query is as follows; "search for the items offering cuisine type [‘italian (north & south)’ or ‘italian (north)’ or ‘italian (northern)’ or ‘italian (southern)’ or ‘italian’ or ‘italian nuova cucina’] and with food quality [‘near-perfect food‘]” this is parametric search resulting in few (only 3) restaurants being retrieved thus eliminating many restaurants which could still interest the user. step 2: personalized query expansion for capturing varying possibilities and minimizing null retrieval if only the constraints given in step 1 are used, there is a high chance of null retrieval. (as demonstrated, only 3 restaurants were retrieved). user41 is described as a person interested in food and different culinary disciplines. he may have selected ‘italian’ cuisine due to reasons such as, he not having ever experienced ‘italian’ food; having experienced italian food and liking the italian flavor, or maybe he preferring a certain italian dish. user41’s user model shows high relevance values for both ‘fair food’ and ‘good food’ which was allocated based on user’s pbc values. the query is expanded using all additional attribute values in the user model as follows. "search for the items that offer cuisine type [‘italian (north & south)’ or ‘italian (north)’ or ‘italian (northern)’ or ‘italian (southern)’ or ‘italian’ or ‘italian nuova cucina’] and with food quality [‘near-perfect food’ or ‘fair food’ or ‘good food’]” this resulted in an expanded 106 records. if this is an unmanageable number of records, for the user, then can be filtered in later steps. otherwise has the option to stop the search process in step 2 (see figure 5). step 3: further filtering based on the user model – personalized filtering "search for the items with cuisine type [‘italian(north & south)’ or ‘italian(north)’ or ‘italian(northern)’ or ‘italian (southern)’ or ‘italian’ or ‘italian nuova cucina’] and with food quality [‘near-perfect food’ or ‘fair food’ or ‘good food’] and the décor is from poor to good and the service is from fair to good” the outcome of step 2 (106) items was reduced to 85 items. after filtering, if still an unmanageable number of records are available, piprp prompts the user asking for further filtering. the user is allowed either to browse through the available result set (all 85 outcomes) or to provide constraints for further filtering. s86 australasian journal of information systems alahakoon 2015, vol. 19, pp. s77-s97 interactive product retrieval model step 4: further querying the user refer to di layer information in figure 2. feature number “4”, has five attributes. user 41’s atr for feature number 4 is calculated as follows; atrfeat4 = (0 + 0.2 +0.3+0.43+0.41)/5 = 0.265 figure 7, shows the atr values of features in the di layer of user41. feature “popularity” has the highest atr value, while the feature “restaurant category” has the least atr value. therefore, the first question in step 4 on ‘popularity’ will be followed by the question on ‘location’ which has the next highest atr value. figure 7: atr values calculated for user 41’s di layer for restaurants before the scenario 41-a1 the algorithm for ordering the item features according to atr values is shown in figure 8. each feature is represented as fj, j [1, 2, .., j], and j is the total number of features exploited to describe items in the current interacting domain di. now, starting from the feature with the highest atr value, each feature is directed to the user as a question. all the attribute values are provided as options for the question. fj constrained in step 2 ? fj constrained in step 3 ? read each fj from di layer of domain di belonging to the user stop find atrfj if atrfj > 0 features and atr values j = j -1if j > 0 no no yes yes yes yes no add avrfj to the list no figure 8: feature selection for further querying figure 9 shows the three possible attribute values as options to obtain a value for the feature ‘popularity’. the result set is filtered according to the user selected attribute values (in the example, ‘little known but well liked’ and ‘up and coming’). even though the user model indicates ‘popularity’ as the most important feature, there can be instances where the user is not interested in the feature for the current scenario. in such situations cancellation (“cancel”) of the query is possible. therefore, the user is able to control average total relevance values of features user41 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 cuisin e cost deco re food qulity rest auren t c ate gory speci al m eal tim es custo mer serv ice loca tio n atm osp her e park ing speci al ser vic es populari ty rese rvatio ns liqu or speci al m enu es speci al f ocus feature at r va lue s s87 australasian journal of information systems alahakoon 2015, vol. 19, pp. s77-s97 interactive product retrieval model the filtering during the interactive process. the “describe” button provides a description of any of the selected attributes, providing any sub features of the attribute or a simple description held by the system. figure 9: filtered according to ‘popularity’, constraint 28 records remains in the result set after filtering based on ‘popularity’ constraint the user can choose either to browse the results or further constrain the result set. if further filtering required, the question asking process continues until all the features with positive (greater than zero) atr values are presented to the user as a question. once the user shows interest to view the results, items are displayed as in figure 10. figure 10: item display screen the picture of the restaurant gives the user a more descriptive idea about what to expect. the presentation is personalized by showing the attributes explicitly requested by the user separately from the other attributes. user can explicitly see if his/her preferences are met by the result. after the results have been viewed, the user may decide to change preferences provided during step 4 system generated questions. therefore, at the end of the result display, the piprp provides the option for the user to return to the original set of records in step 3. the question asking process starts from the beginning of step 4 using the same item set which resulted after s88 australasian journal of information systems alahakoon 2015, vol. 19, pp. s77-s97 interactive product retrieval model step 3. as such, the user is allowed to change the preferences provided during system querying with minimal effort and time. step 5: calculate similarity between the query and the items for simplicity, assume a domain d with only fifteen attributes (a1-a15). a query qu and four items to compare are presented in figure 11. ia, ib, ic, and id, are four items each described using 15 binary-values to indicate the presence or absence of the fifteen attributes. in query qu, the user shows interest in attributes a1, a3, a5, a11 and a15. query qu is formed starting from the initial specification and then through the steps 2, 3, and 4 of the algorithm. according to the query, u requested attributes a1, a3 and a11 (circled) when specifying the initial query (step1, highest relevant attributes) and a5 and a15 (underlined) were provided as preferences during step 4 (which are less important than a1, a3 and a11). qu = 1 0 1 0 1 0 0 0 0 0 1 0 0 0 1 ic = 1 0 0 0 0 1 0 0 0 0 0 0 0 0 1 ib = 0 0 1 0 1 0 0 1 0 0 0 0 0 0 0 id = 0 0 0 1 0 0 0 1 0 0 0 1 1 1 0 ia = 1 0 1 0 1 1 0 1 0 0 1 0 0 0 1 a3 a11a1 a2 a5 a6 a7 a8 a9 a10 a12 a13 a14 a15a4 figure 11: comparison of four different items ia, ib, ic, and id to a query qu each attribute has a relevance value in u’s di layer for domain d. the relevance values of attributes a1-a15 in the di layer of a user u is as shown in figure 12. a1 0.69 a4 0 a7 0.46 a10 0 a13 0.24 a2 0 a5 0.18 a8 0.2 a11 0 a14 0 a3 0.42 a6 0.81 a9 0.02 a12 0.08 a15 0.27 figure 12: attribute relevance values for u based on the query and the user model, table 4, summarizes the details of the attributes of items ia-id. for example, item ia has seven attributes out of the fifteen attributes. item ia has all three attributes (a1, a3 and a11) requested in the initial query. in the user model a1, a3 and a11 has, 0.69, 0.42 and 0, relevance values respectively. ia also has both attributes that user showed an interest during step 4 (a5, a15). attributes a6 and a8 show a positive relevance value in the user model. when the total similarity to the query is calculated (table 4), ia shows the highest similarity. (note, r1 and r2 are assigned 1 and 0.75 respectively). ic scores the next highest similarity. it has one attribute from the initial query, one from the selection process and two highly relevant attributes from the user model. s89 australasian journal of information systems alahakoon 2015, vol. 19, pp. s77-s97 interactive product retrieval model it em in it ia l q ue ry d ur in g r et ri ev al fr om th e u se r m od el ia a1,a3, a11 a5, a15 a6, a8 ib a3 a5 a8 ic a1 a15 a6 id a4, a8, a12, a13, a14 table 4: summary of attributes and their calculated relevance values it em total similarity = n i s 1 umpspiq (i))sim(i)im(i)(sim ia (1+0.69)+(1+0.42)+(1+0)+(0.75+0.18)+(0.75+0.27)+(0+0.81)+(0+ 0.2) = 7.07 ib (1+0.42)+(0.75+0.18)+ (0+ 0.2) = 2.55 ic (1+0.69)+ (0.75+0.27)+(0+0.81) = 3.52 id (0+0)+(0+0.2)+(0+0.08)+(0+0.24)+(0+0)= 0.52 table 5: total similarity of each item ib, is similar to ic with regard to the number of attributes present. ib has a3 from initial query whereas ic, has a1; and ib has a5 during step 4, whereas ic has a15; they both have one more additional attribute present in them. if the relevance values from the user model are not considered, both items ib and ic ends up with identical similarities, which is not true. usually, when calculating similarity measures, methods such as cosine similarity1 or pearson correlation2 are required to avoid such errors. due to the use of relevance values from the user model, our approach of similarity calculation does not require such methods. as mentioned previously, similarity calculations are time consuming if used with a large dataset. in piprp, the similarity calculation is utilized only if required, after several filtering steps. therefore, in piprp similarity is calculated only for a smaller number of records that remain after previous filtering. 5 evaluation evaluation of the work requires combined evaluation of the user model (lum) and the interactive product search (piprp). as far as the user model is concerned, one of the most important issues is its accuracy in recommendations. although, there is no generally accepted methodology for evaluating the performance of a user model, there are few methodologies that are introduced to the user modelling community from other backgrounds such as artificial intelligence (zukerman and albrecht 2001). precision, recall and f1-measure, predicted probability and accuracy, and utility are a few of such methods. more often, the evaluation methods depend on the availability of data and testing environments. 1 cosine similarity wikipedia url: http://en.wikipedia.org/wiki/jaccard_index 2 pearson correlation wikipedia url: http://en.wikipedia.org/wiki/correlation#pearson.27s_product-moment_coefficient s90 australasian journal of information systems alahakoon 2015, vol. 19, pp. s77-s97 interactive product retrieval model with regard to interactive product selection processes the evaluation method varies according to the type of the system. in the literature, interactive systems are often evaluated based on the time spent to isolate manageable results set. or, by counting the number of interactions required to obtain the final recommended product list. in interactive decision guides (schmitt, dopichaj et al. 2002) the number of interactions is the number of features specified by the user. in critique based systems (burke 2002, viappiani, faltings et al. 2006) initial query and the number of critiques applied to direct the search becomes the total number of interactions. both these types do not use a model of the user. in user model based systems, such as adaptive place advisor (thompson, göker et al. 2002), the number of times explicit information acquired from the user during a transaction is considered as the number of interactions. since our work is a combination of a user model (lum) and an interactive product search (piprp) it is worth evaluating the model against criteria critical to requirements of the current ebuyer’s needs (pu, faltings et al. 2004). therefore, in next section the novel model is evaluated according to a set of product selection criteria in ebuying environments. 5.1 principles for evaluating online product selection work in (pu, faltings et al. 2004) investigated a broader spectrum of options to handle during an online interaction. these principles are based on the assumption that the user is not fully aware of his/her need to the minor detail. in (pu, faltings et al. 2004), to support user’s product selection process a three-way strategy has been employed: 1. provide users with domain knowledge provision of domain knowledge to help the user make the selections. 2. avoid means objectives capturing the user’s actual objectives. 3. convince the user – provide user with verification of system selections. the three-way strategy is enforced using a set of principles. as explained in (pu, faltings et al. 2004), figure 13, shows the three aspects and the six principles defined to achieve them. for a detailed explanations and examples please see (pu, faltings et al. 2004). provide users with domain knowledge principle 1: elicit preferences within context. a search tool should ask questions with reference to a complete and realistic context, not in an abstract way. principle 2: allow partial satisfaction of user preferences. when no solutions exists that satisfy all preferences, show solutions that satisfy a maximal subset. avoid means objectives principle 3: allow partial preference models. do not force the user to provide any specific preferences. principle 4: any preference allow users to state their preferences on any attribute rather than a fixed subset. principle 5: any order allow users to state their preferences in any order they choose. convincing the user principle 6: support tradeoff navigation the search tool should provide active tradeoff support for the user to compare examples shown. figure 13: interaction principles for online product navigation given in (pu, faltings et al. 2004) in (pu, faltings et al. 2004) the principles are implemented for smart client; a preference based search tool without a user model. in addition to fulfilling the above principles, piprp is capable of providing users with less obtrusive questioning. here we consider; i. asking questions in a meaningful sequence and ii. excluding personal questions whenever possible. s91 australasian journal of information systems alahakoon 2015, vol. 19, pp. s77-s97 interactive product retrieval model we declare two more new principles as in figure 14. principle 7: ask meaningful questions a search tool should ask questions with which are easy to understand and also in a meaningful sequence. principle 8: minimize personal questions a search tool should minimize the number of questions related to users’ personality. figure 13: additional principles 5.2 principle based evaluation using the proposed criteria in this section, we evaluate the new model and process against each of the above 8 principles. principle 1: elicit preferences within context piprp provides an easy to use flexible interface, facilitating the users to specify their changing needs in a dynamic and volatile environment. in step 1 of the algorithm, use of the interface where all the features and attributes are displayed (shown in figure 6) helps the user to concentrate on the context. this corresponds to the above principle 1, which request user inputs to be more specific to the context. since all the possible options are provided, specifying preferences needs less effort than providing inputs for an open-ended question. the items are thoroughly described and the options are provided to help specify the user need. principle 2: allow partial satisfaction of user preferences a search engine could often result in null retrievals: or it misses out on many good items that might suit a user. for example, strict constraints leave out closer outcomes such as slightly greater or lesser price ranges. since the relevance values are allocated as a fuzzy function, more than one attribute becomes relevant under the same feature (table 6). as explained in step 2 of the algorithm, personalized query expansion is carried out by expanding the initial query using the other relevant attributes in the user model. for example, if presented as a query, the initial attribute selection for scenario 41-e1 (see table 3) is as follows: “search for the items that has price in range $15-$30, has good décor and has the atmosphere as warm spots by the fire” at the time of the interaction the user model had more than one attribute value under each features. therefore, the following query resulted after the personalized query expansion using the user model information in table 6. “search for the items that has price in range ($15-$30) or ($15 and less) or ($30-$50) and has poor décor or fair décor or good décor and has the atmosphere as (warm spots by the fire) or (dining outdoors) or (place for singles) or (quiet for conversation)” s92 australasian journal of information systems alahakoon 2015, vol. 19, pp. s77-s97 interactive product retrieval model feature name attri-name init-rel curr-rel cost $50 and more 0 0 $15 and less 0.1 0.12 $30-$50 0.45 0.88 $15-$30 0.74 1 décor excellent décor 0 0 extraordinary décor 0 0 near-perfect décor 0 0 poor décor 0.08 0.15 good décor 0.7 0.85 fair décor 0.55 1 atmosphe re dining outdoors 0 0.2 place for singles 0 0.04 quiet for conversation 0 0.04 table 6: a summary of user 41’s user model content therefore, the expanded query is capable of retrieving more items than the items fulfilling exact constraints. the query expansion results in a larger number of outcomes reducing the chance of null retrieval. these items may not satisfy all the constraints the user specified in the initial search query, but the system still produces adequately matching outcomes. hence the algorithm allows partial satisfaction of user preferences. principle 3: allow partial preference models piprp does not force the user to provide any specific preferences. in step 1 the user can start product navigation by simply specifying the most important attributes that comes to the mind. although system generates questions (in step 4), users have the option to cancel any question that they are unsure about the answer or not interested in. therefore, the system does not force the user to provide any specific preference unless user willingly provides the most important preferences. hence the model complies with principle 3. principle 4: any preference in piprp users provide their inputs in two occasions; step1 and step4. in step1, the flexible interface allows input of any preference, where each individual’s personal choices can be entered. we believe, if a person is looking for presence of a particular attribute, they will provide such inputs in step1. in step4, user provides answers for system generated questions; if a feature seems important according to the user model such features are presented to the user as questions. again, such questions are personalized to each individual depending on the user model. however, if the presented feature is not important to the user, s/he has the ability to cancel such questions without providing an answer. therefore, piprp fulfils principle 4 by allowing users to state their preferences on any attribute rather than a fixed subset. principle 5: any order piprp allows users to state their preferences in any order they choose. in step 1, the user is free to select any of the features and in any order. in step4, the features are presented in a personalized order (according to the user model). however, since the user has the choice of entering his/her critically important features in step1, we believe personalized order provided in step4 suffices. s93 australasian journal of information systems alahakoon 2015, vol. 19, pp. s77-s97 interactive product retrieval model users are also allowed to select more than one preference from the given options in both step1 (figure 6) and step4 (figure 9), where the user is not particular about a single option. all selected attributes are included in the query. after each user selection, the results set is filtered retaining the records that fulfil the user request. principle 6: convincing the user and support trade-off navigation the search tool should provide active tradeoff support for the user to compare examples shown (pu, faltings et al. 2004). most prefer to compare the final results and find the true best answer by considering any affordable tradeoffs thus verifying the quality of the results. as previously mentioned, step 3 results are maintained as a record set throughout the process. therefore, if the user is not satisfied with the outcomes, it is possible to go back and invoke the same set of system questions. this allows modification of preferences to determine their effects on the results. when the attribute selection is changed, if the outcomes become different, user can compare and choose. for example, the filtering mechanism used in entrée, removes any un-matching items from the results set after each tweak (burke 2002). since the disagreeing items are removed from the result set, getting a different set of outcomes by changing the attribute values becomes very limited (apart from starting a fresh search). furthermore, the interface permits the user to go back and forth in the system selected result set (using ‘next’ and ‘back’ buttons, figure 10). as explained previously, the system output is in the highest to least relevant order. the user can use the navigation facility to traverse the list and compare the recommendations. for example, it is possible to verify if the recommended items in the top of the list are preferred than the items later in the list. principle 7: comprehensive questions comprehensive nature of a question depends on not only its simple and easy-to-understand aspect but also the sequence in which they were asked. for example, static surveys include related questions in a group box or in the same page for the interviewee to understand the questions which are related to the same issue. but in dynamic dialogues, the intention of the process is only to reduce the number of questions. these systems chose the most discriminating feature among the result set and present it as a question to the user (user provides the preferred attribute value as the answer). since the features are arbitrary this method often puzzles users with the order of their questions. in piprp, the questions are generated according to the average total relevance (atr) of a feature. therefore, the most relevant question is first directed to the user. for example if the atr values are 0.325 and 0.222 for cost and décor respectively, then price range is queried before asking the preference for décor. since each user has different atr values the question sequence is personalized; hence comprehensibility can be ensured. furthermore, in the design itself, we had grouped together the attributes corresponding to each feature. for example, feature ‘cost’ has four attributes. therefore the questions are meaningful with selection of options. principle 8: minimize personal questions a search tool should minimize the number of questions related to users’ personality’. work carried out on search costs in online environments claim that people are concerned about the type of facts that are requested (annacker, spiekermann et al. 2001). work has been carried out on measuring the intrusiveness of a question in an electronic sales interaction based on the personal information content of the questions (spiekermann, grossklags et al. 2001, spiekermann 2004). any questions that reveal the user’s personal information including the attributes that are related to user personality can be considered as personal questions (spiekermann, grossklags et al. 2001). for example, if the user does not indicate a concern about the price of the item, but certain other aspects, asking the user about his/her price range preference may appear to be intrusive. it may appear as intrusively trying to figure out the s94 australasian journal of information systems alahakoon 2015, vol. 19, pp. s77-s97 interactive product retrieval model user’s income. similarly system questions related to any pir-attribute may look intrusive trying to capture user’s personal information. therefore, we believe such questions should not be directed to the user unless the user specified them willingly as an initial preference. as a result piprp employs ‘personalized filtering’ as described in step 3 of the algorithm. such question reduction strategies further improves the quality of the interactions by indirectly encouraging a fewer number of questions. 6 conclusion we have illustrated the use of a layered user model (more specifically, lum) for providing personalization in all three phases of an interactive online product retrieval process: requirement elicitation, product search and product presentation. since the new technique introduced in the paper integrates a user model with a product retrieval process, evaluation requires transaction details with related user information such as demographics. there were no such benchmark datasets available and as such we have used the well-known entrée dataset which has detailed product information. by introducing a hypothetical user with a range of interests we have managed to demonstrate the features and functionality of the new product retrieval process. combination of the user model in interactive product retrieval has a number of benefits including: i. the ability to expand queries in an intuitive way (using the user model in addition to the current query) when there is possibly little or no items retrieved; ii. the ability to filter results set (using the user model) when the answer set to a query is too large; iii. the ability to increase the chance of returning a satisfactory results set (since we have more information from the user model); and iv. the ability to do the above three without increasing the burden of user interaction or obtrusiveness (by minimizing asking where possible by getting information from the user model, and avoiding questions with high personal information content). our scenario demonstrates (by example) that the method works, and we also provided a principles-based evaluation of our new process. our research, hence, provides a new way of using user models in the interactive product retrieval process. while our domain has been restaurants, interactive retrieval of other types of product information can be similarly considered. as future work we intend to test the system with a set of real users. also, even though we show that lum works in piprp, a similar process to piprp can be explored but replacing lum with other kinds of user models. acknowledgement author thankfully acknowledges the research discussions and comments contributed to this work by associate professor seng loke, la trobe university, australia and professor arkady zaslavsky, csiro, australia. references ahn, j.-w., p. brusilovsky and s. sosnovsky (2006 ). quizvibe: accessing educational objects with adaptive relevance-based visualization. proceedings of world conference on elearning, honolulu, hi, usa. alahakoon, o., s. loke and a. zaslavsky, eds. (2007). capturing buying behavior using a layered user model. proceedings of 8th international conference on e-commerce and web technology, lncs, spinger-verlag. s95 australasian journal of information systems alahakoon 2015, vol. 19, pp. s77-s97 interactive product retrieval model annacker, d., s. spiekermann and m. strobel (2001). e-privacy: a new search cost dimension in online environments. proceedings of 14th bled conference of electronic commerce, bled. slovakei. asuncion, a. and d. j. newman. (2007). {uci} machine learning repository. 2008, from http://www.ics.uci.edu/$\sim$mlearn/{mlr}epository.html. burke, r. (2001). ranking algorithms for costly similarity metrics. case-based reasoning research and development 4th international conference on case-based reasoning, new york, springer verlag. burke, r. (2002). "hybrid reccommender systems: survey and experiments." user modelling and user-adapted interaction 12: 331-370. burke, r. (2002). "interactive critiquing for catelog navigation in e-commmerce." artificial intelligence review 18(3-4): 245-267. burke, r. (2007). hybrid web recommender systems. the adaptive web. a. k. p. brusilovsky, and w. nejdl berlin heidelberg springer-verlag lncs 4321: 377 – 408. doyle, m. and p. cunningham (2000). a dynamic approach to reducing dialog in on-line decision guides. 5th european workshop on case-based reasoning (ewcbr 2000), springer verlag. mahmood, t. and f. ricci (2009). improving recommender systems with adaptive conversational strategies. hypertext. c. cattuto, g. ruffo and f. menczer, acm: 73–82. pazzani, m. j. and d. billsus (2007). content-based recommender systems. the adaptive web: methods and strategies of web personalization. p. brusilovsky, a. kobsa and w. nejdl. berlin heidelberg newyork, springer verlag. 4321/2007: 325-341. pu, p., b. faltings and m. torrens (2004). effective interaction principles for online product search environments. proceedings of the ieee/wic/acm international conference on web intelligence (wi'04). beijing, china. resnick, p., n. iacovou, m. suchak, p. bergstrom and j. riedl (1994). grouplens: an open architecture for collaborative filtering of netnews. conference on computer supported coorperative work. chapel hill, acm: 175-186. schafer, j. b., d. frankowski, j. herlocker and s. sen (2007). collaborative filtering recommender systems. the adaptive web:methods and strategies of web personalization. p. brusilovsky, a. kobsa and w. nejdl. berlin heidelberg newyork, springer verlag. 4321/2007: 291-324. schafer, r. (2001). rules for using multi-attribute utility theory for estimating a user's interests. 2008, from http://www.kbs.unihannover.de/%7ehenze/abis_workshop2001/abis_2001.html. schmitt, s., p. dopichaj and p. domínguez-marín (2002). entropy-based vs. similarityinfluenced: attribute selection methods for dialogs tested on different electronic commerce domains. proceedings of the 5th european confernce on case based reasoning (eccbr). aberdeen, scotland, uk: 380-394. shearin, s. and h. lieberman (2001). "intelligent profiling by example." proceedings of the international conference on intelligent user interfaces (iui 2001) ,santa fe, nm,: 145152. smyth, b., m. coyle and p. briggs (2011). communities, collaboration, and recommender systems in personalized web search. recommender systems handbook. f. ricci, l. rokach, b. shapira and p. b. kantor, springer us: 579-614. spiekermann, s. (2004). "ec websites and product context: how consumers' product focus drives their navigational needs." proceedings of acm conference of electronic commerce (ec'04), new york, mai. s96 australasian journal of information systems alahakoon 2015, vol. 19, pp. s77-s97 interactive product retrieval model spiekermann, s., j. grossklags and b. berendt (2001). e-privacy in 2nd generation ecommerce: privacy preferences versus actual behavior. proceedings of acm conference on electronic commerce (ec'01). tampa, florida. statistica. (2014). statistica digital buyer penetration worldwide from 2011 to 2018. from http://www.statista.com/statistics/261676/digital-buyer-penetration-worldwide/. thompson, c. a., m. göker and p. langley (2002). "a personalized system for conversational recommendations." journal of artificial intelligence research. 21: 393-428. towle, b. and c. quinn (2000). knowledge based recommender systems using explicit user models. knowledge-based electronic markets, papers from the aaai workshop, aaai technical report ws-00-04. menlo park, ca, aaai press: 74-77. viappiani, p., b. faltings and p. pu (2006). "preference-based search using example-critiquing with suggestions." journal of artificial intelligence research (jair) 27: 465-503. zadeh, l. a. (1965). "fuzzy sets." information and control 8: 338-353. zukerman, i. and d. albrecht (2001). "predictive statistical models for user modeling." user modeling and user-adapted interaction, invited paper 11(1-2): 5 -18. copyright: © 2015 alahakoon. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. s97 paper title (paper title style) australasian journal of information systems banerjee & gupta 2019, vol 23, research on the role of tech in workforce mgt talent attraction through online recruitment 1 talent attraction through online recruitment websites: application of web 2.0 technologies pratyush banerjee birla institute of technology & science (bits) pilani pratyush.banerjee@pilani.bits-pilani.ac.in ritu gupta t a pai management institute abstract talent attraction is one of the major challenges for hr managers. with the rise of online recruitment channels, the number of applicants for a given job vacancy have increased substantially. in addition, the time taken in finding the right talent from the huge pool of applications has increased significantly, adding up to the hiring cycle. in today’s competitive labour market, employers need to highlight their brand image to prospective job-seekers, so that there is higher chance of recruiting the best talent that fits their manpower requirement. in this paper, an attempt is made in investigating that to what extent web 2.0 technologies such as podcasts, blogs and online employee testimonials may enhance the employer's brand value in the eyes of employed professionals. a multi-group moderated mediation analysis is conducted with 361 working professionals who are active online recruitment platform users. the analysis helps establish the effect of the perceived quality and credibility of career websites on job-seekers’ perception about the employer and on their subsequent application intention. the findings reveal positive effect of video podcasts and realistic employee testimonials presented through third party blogs on job-seekers’ perceived quality and credibility of the job advertisement. this phenomenon is reflected in their heightened attraction for the employer and eventual intention to apply for jobs at the firm. keywords: e-recruitment; realistic job preview; blog; podcast; employer branding; talent attraction 1 introduction technology has completely changed the traditional human resource (hr) functions. one of the most affected functions is the process of employee recruitment. organizations today extensively use internet technology to invite applications in a cost and time efficient manner for searching and hiring talented individuals. this phenomenon is called electronic recruitment, also known as e-recruitment (maurer & liu, 2007; maurer & cook, 2011). there are several platforms of online recruitment, for example company’s career website, online newspaper’s career column, e-recruitment portals (also known as third party e-recruitment service provide or job boards) and professional job networking sites used for job postings such as linkedin (borstroff, marker & bennett, 2007; walker, field, giles, armenakis, & bernerth, 2009).a decade ago, an astounding number of almost 110 million jobs and 20 million resumes were posted on various job portals such as monster in united states of america (usa) alone (maurer & liu, 2007). over the years, these numbers have been exponentially growing. by 2017, there were 52 million registered job-seekers innaukri.com, the largest e-recruitment job search engine in india (naukri job speak, 2017; monster employment index, 2017). australasian journal of information systems banerjee & gupta 2019, vol 23, research on the role of tech in workforce mgt talent attraction through online recruitment 2 e-recruitment has emerged as a highly cost-effective solution compared to traditional recruitment channels such as print advertisement, walk-in interviews and campus placement. for example, dow chemical’s reduced its hiring cycle from 90 to 34 days and reduced its cost per hire by 26% by adopting e-recruitment practices (maurer & liu, 2007). majority of the fortune 500 organizations have moved to e-recruitment and selection processes (chapman & webster, 2003). with the advent of social media, there is further scope now to save more time and money through e-recruitment strategy (vicknair, elkersh, yansey, & budden, 2010). in india, third party e-recruitment service providers such as naukri and monster have created a strong presence in the job market (doh, stumpf, tymon, & haid, 2008; kumari, bhat, & pandey, 2010). there has been significant reduction in costs incurred by indian firms using erecruitment media, costing them approximately 80% less than traditional recruitment activities (tyagi, 2012). the credibility of online job advertisements has been questioned by several researchers, as the findings from empirical studies indicate that job-seekers view traditional media such as employee referrals and campus placements as more credible sources for job advertisements than modern channels such as job portals and career websites (cable, aiman-smith, mulvey & edwards, 2000; cable & yu 2006). one reason for this negative attitude may be due to the over utilization of employee testimonials in e-recruitment which specifically portray only the positive side of the work place (reeve, highhouse, & brooks, 2006). another reason behind the poor acceptance of e-recruitment channels may be attributed to lack of appeal of website designs. there is strong evidence that poorly designed websites may have a negative effect on applicant’s attitude towards the portal in turn affecting their will to apply for the jobs posted on such websites (cable & yu, 2006; maurer & liu, 2007). in the current study, the objective is to explore how the credibility of the e-recruitment advertisement and the quality of the e-recruitment website influence the working professional’s attitude towards e-recruitment websites, thus affecting their perceived attractiveness towards the employer and consequently their intention to apply for that job. specifically, the study explores whether the perceived quality of the e-recruitment advertisement, which in turn affects organizational attractiveness, can be influenced by providing innovative corporate preview of the firm such a streaming podcast, compared to a traditional textual description. this study also explores the impact of realistic employee testimonials in place of positively framed ones, to see if perceived credibility of the job advertisements through corporate career websites can be enhanced. additionally, this study probes the possibility of posting online job-related information of a corporate entity through a third-party blog platform to investigate whether this may further enhance the credibility of the information projected about the organization through online job advertisements. finally, this study tests the implication of such perceptions on the intention of job seekers to apply for a certain job advertisement belonging to a particular e-recruitment channel. 2 literature review: web 2.0 and e-recruitment majority of the studies in e-recruitment have focused on the issues surrounding the design adequacy of e-recruitment websites (cober, brown & levy, 2004). many researchers have explored the factors affecting perceptions of applicants (job-seekers) towards the credibility of messages propagated through e-recruitment websites (dineen, ash & noe, 2002; borstroff et al., 2007; allen, mahto, & otondo, 2007; van hoe & lievens, 2007). australasian journal of information systems banerjee & gupta 2019, vol 23, research on the role of tech in workforce mgt talent attraction through online recruitment 3 previous studies have found that source credibility is directly related with job offer acceptance (fisher, ilgen & hoyer, 1979). it has been found that realistic job previews (rjps) given to the job seekers at the time of recruitment enhance the commitment level of employees (barksdale, bellenger, boles, & brashear, 2003). rjps have potential to disseminate realistic information about a context, here applicants are shown both positive and negative aspects of the job in the organizational context, it increases their trust on the organization’s integrity and honesty (dugoni & ilgen, 1981; meglino, ravlin, & denisi, 1997; breaugh & starke, 2000). there are only a few studies exploring the effect of employee testimonials on applicant’s attitude in erecruitment context (van hoye & lievens, 2007). hence, there exists a lacuna in understanding how job-seekers perceive realistic employee testimonials put up in online career websites. it is suggested that rjps shared on web will make job information more credible than traditional text-based previews (meglino et al., 1997). as these reviews will be interactive and unbiased, they may be shared on neutral medium of communication. however, there is lack of empirical validation against such claim. there is a big dilemma about what may be the best way for enhancing credibility of job-related information to jobseekers. the dilemma is mainly centred around whether the online job advertisements on company owned websites like corporate career portals are sufficient or these advertisements should be posted on neutral cyber platforms (walker et al., 2009). in this regard the rise of unbiased end user driven platforms such as glassdoor (mallon, 2017) have opened scope of highlighting company image in a more objective manner. web 2.0 technology (gokhale & chandra, 2009), such as blogs, internet chat rooms and social networking websites have been observed to influence consumer decision to buy a certain product online (koufaris, 2002) or for adoption of online banking (adapa & cooksey. 2013). web 2.0 technology has found its application in e-recruitment as well, though the adoption is at a slower rate (russell, 2008). there is strong empirical evidence regarding the impact of website design features such as colour, layout, and page loading time and customized search options on perceptions about quality of the website (de goede, van vianen, & klehe, 2011). empirical tests in advertisement research have demonstrated that information delivered in audio-visual mode is considered more engaging and enriching than that delivered in text mode (walker et al., 2009; mcgowan, 2010). in e-recruitment context, banerjee (2016) conducted an experiment with college students about to enter the job market regarding the impact of various website manipulations such as rjps, blogs and use of podcasts. his study showed that job-seekers assessed career websites featuring rjps, blogs and podcasts with greater trust and more attention than the websites lacking in one or more of these features. however, the research was limited to only college graduates and could not be validated for working population. 3 hypothesis formulation 3.1 perceived quality of website → applicant’s attractiveness of the employer brand a website’s overall usefulness and user-friendliness encompasses the quality of the website (loiacano, watson, & goodhue, 2007; knight & burn, 2011). in a study conducted in the us, it was found that websites designed poorly cost $30 million per day to corporate career portals (maurer & liu, 2007). website design features may affect perceptions about quality of the australasian journal of information systems banerjee & gupta 2019, vol 23, research on the role of tech in workforce mgt talent attraction through online recruitment 4 website; these features include colour, layout, page loading time and customized search options amongst others (howardson & behrend, 2014). further, cognitive load theory (chandler & sweler, 1991) emphasizes that human beings can process information better if it is presented to them in a graphic manner. in e-recruitment research, the impact of audio-visual mode of information delivery as a means of enhancing employer attractiveness has not been explored much (banerjee, 2016). a website can make the employer or the job position really attractive, if the audio-visual tools such as podcasts are used to present the job as a desirable career opportunity for the applicant (mcgowan, 2010). using brand equity theory (aaker, 1997), it may be deduced that positive perceptions of quality of a firm’s e-recruitment website can positively influence job seeker attitude towards the firm as a prospective employer. learning a lesson from brand equity theory, it can be inferred that if job seekers receive cues about higher perception of website quality through audio-visual corporate previews, then job seekers will feel attracted towards the organization as a prospective employer. finally, the theory of planned behavior (tpb) (ajzen, 1991; lin, 2010), states that the decision to engage in a particular behaviour is the result of a rational process, which in the case of online job-seekers, involves the job seeker’s reasoning that a firm has a willingness to meet the diverse search demands of the job seeker through its website. as a result, job-seekers may develop a positive attitude towards the firm’s career website in the form of more positive perceptions of the employer brand (ambler & barrow, 1996). this eventually will be reflected in their actions such as feeling attracted towards the firm and in a higher intention to apply for jobs through that site (ajzen, 1991). therefore, we hypothesize h1: a positive perception of quality of e-recruitment website will significantly increase perceived attractiveness about the employer for online job-seekers. 3.2 perceived credibility → applicant’s attractiveness of the employer brand trust formation in an online platform requires the latter to project authentic and unbiased information to the users (bewsell, 2009). employee testimonials are being used increasingly to enhance credibility of the recruitment advertisements. it also helps in providing applicants an internal viewpoint about the work life at the organization (dineen et al., 2002; cable et al., 2000). applicants find employee testimonials informative and indicative of their expectations from the work place thus making them apply for the job. however, use of rjps in e-recruitment context has been relatively less. rjps have evolved from the realistic information hypothesis which indicates that, when applicants are shown both positive and negative aspects of the job in the organizational context, it increases their trust on organizations integrity and honesty (dugoni & ilgen, 1981; petty, casmer, haugtvedt, & cacioppo, 1987). it also follows from media credibility theory (cable & yu, 2006) and the yale persuasive communication model (oskamp, 1977) that if the source of any information is considered to be trustworthy, then the receivers would positively appraise the information. rjps provide information which breaks the conventions of employer branding by preaching about both the positive and negative aspects of a workplace. thus, on the overall level of trustworthiness, such testimonials may be considered as more credible than testimonials which appear to be fabricated to portray only good things about a firm (popovich & wanous, 1982). based on signalling theory (gregory, meade, & thompson, 2013), it may be further opined that the trust cues that job-seekers may receive due to realistic employee testimonials on australasian journal of information systems banerjee & gupta 2019, vol 23, research on the role of tech in workforce mgt talent attraction through online recruitment 5 neutral platforms (e.g., blogs) will increase their respect for the employer and make them more inclined towards considering the firm as a potential employer. therefore, it may be hypothesized that h2: positive perception about credibility of information shared through online career websites will significantly increase perceived attractiveness about the employer for online job-seekers. 3.3 mediating role of organizational attractiveness further, theory of reasoned action (tra) (ajzen, 1991) suggests that in case of systematic information processing, job-seekers may first make judgment about the credibility of the message and the quality of presentation of the message. after this, the next stage of cognition will involve perception regarding the attractiveness of the firm as a potential employer. following this, the applicants will form their overall intention to apply for the job (pfieffelmann, wagner, & libkuman, 2011). therefore, there is a possibility that organizational attractiveness will mediate the relationship between perceived quality and intention to apply and the relation between perceived credibility and intention to apply. h3a: organizational attractiveness will mediate the effect of perceived website quality on job-seekers’ intention to apply for jobs online. h3b: organizational attractiveness will mediate the effect of perceived credibility of online job advertisements on job-seekers’ intention to apply for jobs online. 3.4 moderating effects of website design manipulations signalling theory allows us to explore that depending on the nature of preview-mode, testimonial-type and testimonial-source, job-seekers will have different cues to process jobrelated information. hence, the treatment conditions can be considered as possible moderators in the linkage for the path between perceived quality and organizational attractiveness and the path between perceived credibility and organizational attractiveness. 3.4.1 moderating effect of preview-mode previous research has shown that audio-visual media is a better channel to persuade audience compared to oral or booklet-based channels (popovich & wanous 1982). baddeley’s (1992) cognitive load theory points in the same direction, where it is explained why information presented in different modes (some through audio, some through video mode) leads to easier interpretation of the message by the receiver. however, it has not been examined, whether use of video podcasts can help in increasing the appeal of online job advertisements. specifically, it is important to understand how using video-based information delivery styles increases the richness of the e-recruitment media thus enhancing credibility perceptions of the message. it has also been observed that the working memory of human beings consists of a video-spatial sketchpad and a phonological loop which can retain visual and audio stimuli independently (chandler & sweller, 1991; baddeley, 1992). thus, information distributed in parts through audio and visual channels such as in live streaming videos (podcasts and vodcasts) can be more helpful for processing of information when presented as a whole. communication channels which, permit timely feedback (e.g., real-time exchange of messages), convey multiple language cues (e.g., words, facial expressions, body posture), allow tailoring of messages to personal circumstances (e.g., customized to the needs of a recipient), and provide language variety (e.g., written text, charts and graphs, verbal expression) are referred to as rich media (daft & lengel, 1986). as per the media richness australasian journal of information systems banerjee & gupta 2019, vol 23, research on the role of tech in workforce mgt talent attraction through online recruitment 6 theory (daft & lengel, 1986; cable et al., 2000), synchronous media such as face-to-face oral communication, telephonic communications etc. are richer than asynchronous media such as mail correspondence or information in a webpage (cable & yu, 2006). in earlier studies, textual communication has been found to be the least rich medium of communication (zusman & landis, 2002; cober et al., 2004; cable & yu, 2006). previous studies have shown that pictures strongly enhance and reinforce information about an organization’s values (cober et al., 2004) and can impact memories more effectively than text. moreover, the use of pictures in addition to text increases the viewer’s cognitive awareness, leading to an increase in attention to the information provided (daft & lengel, 1986). presenting corporate previews through video podcasts/ vodcasts can have a more lasting effect on positive employer brand perceptions and consequently on better perceptions of website aesthetics. further, such audio-visual previews can be easier to understand compared to the current practice of textual previews. in one study by adams, morris and van scotter (1999), face-to-face communication was found to be the best attention grabber, followed by video with audio, then audio only and followed by text only. on the basis of the above argument, it is opined that the relationship between perceived quality and intention to apply will be moderated by the type of preview-mode used by the firm to describe its corporate profile to prospective applicants. an audio-visual profile will create a better perception regarding the website’s quality, and this may enhance the richness of the medium and make the message appear to be more believable (walker et al., 2009). consequently, it will trigger a positive perception of brand equity in the minds of the jobseeker thus, creating a higher sense of firm attractiveness. therefore, we posit that, h4a: the relationship between perceived quality and organizational attractiveness will be moderated by corporate preview mode such that the effect will be higher for audio-visual preview as compared to that for text-based preview. h4b: the relationship between perceived credibility and organizational attractiveness will be moderated by corporate preview mode such that the effect will be higher for audio-visual preview as compared to that for text-based preview. 3.4.2 moderating effect of testimonial-type online job ads may find it difficult to establish a sense of realism in their content to attract potential employees. to counter such effect, organizations have tried to use employee testimonials in erecruitment ads to give jobseekers a peep into the real work environment. however, such practices have been limited to using only positive feedback about the workplace from the employees (breaugh & starke 2000). rjps, which have been used extensively in traditional recruitment research, have been completely neglected in erecruitment research (strohmeyer, 2007; wolfswinkel, 2009). since the content of a webpage can be manipulated by the firm owning the website, therefore, if jobseekers find only good feedback given by employees about their organization in the testimonials, they may consider such feedback less credible and lose the desire to apply for job in that organization (young & foot 2005). it has also been found that people consider any information as trustworthy if they receive information that is different from what is expected by them (breaugh & starke 2000). in this regard, rjps definitely provide unexpected information about a job. similarly, it is being reasoned that if the testimonial is realistically framed, it will create a higher perception of quality and credibility as per technology australasian journal of information systems banerjee & gupta 2019, vol 23, research on the role of tech in workforce mgt talent attraction through online recruitment 7 acceptance model (tam) and yale persuasive communication model (ypcm) (oskamp, 1972; phillips, 1998). this may create a greater sense of appreciation regarding the honesty of the firm and indirectly make the firm more attractive (petty et al., 1987; cable & yu, 2006). therefore, it is proposed that, h5a: the relationship between perceived quality and organizational attractiveness will be moderated by testimonial-type such that the effect will be higher for realistically framed testimonial as compared to that for positively framed testimonial. h5b: the relationship between perceived credibility and organizational attractiveness will be moderated by testimonial-type such that the effect will be higher for realistically framed testimonial as compared to that for positively framed testimonial. 3.4.3 moderating effect of advertisement source due to the possibility of content manipulation by the website owner in case of e-recruitment, it has been time and again observed that job seekers have relied less on job related information posted online and have preferred more traditional media to seek such information. such user behaviour can be interpreted through the source credibility framework (van hoye & lievens 2007), which posits that more credible information sources are considered more persuasive in changing attitudes as well as behaviours. in e-recruitment context, this theory suggests that job seekers perceive credibility of different recruitment sources based on the level of neutrality of the sources. this is because, the more neutral a recruitment source, less is its motivation to explicitly promote the recruiting firm. therefore, company independent sources of erecruitment should be considered as more credible than company dependent sources (van hoye & lievens 2005; 2007). earlier e-recruitment was conducted primarily through company-dependent sources such as job boards (e.g., monster.com) that post job descriptions and specifications for organizations for a fee or via organizations’ official websites. in recent years, a new wave of internet revolution has taken place with the advent of web 2.0 technology (blogs, chat-rooms, social networking sites etc.), which has made the cyber experience of users more interactive and realtime (russell, 2008). nowadays, we have several web 2.0 enabled web-based platforms such as social networking websites (myspace, facebook etc.) being used by head-hunters to recruit passive job seekers and independent blogs such as glassdoor (www.glassdoor.com) where ex and current employees can post their testimonials about their previous and current workplaces respectively (russell, 2009). such platforms have the advantage over company owned websites for being considered as credible, because such media are free from any outside corporate influence and generally firms do not have control over the content of such websites (van hoye & lievens 2007). employee testimonials are an important means, which organizations can use in their website design features to portray their culture. fazio and zanna (1981) found that jobseekers like testimonials because they are first-hand personal accounts of the organizational culture and anecdotal evidence indicates that testimonials are highly persuasive in recruitment contexts (highhouse, hoffman, greve & collins, 2002). cober et al. (2004) also suggested the positive aspect of using employee testimonials citing the fact that such testimonials provide a human touch, enhancing feelings of identification with the hiring organization. web 2.0 platforms such as web blogs have opened a new avenue for communicating free thoughts to the masses in this digital age. also, if the firm does not provide the testimonials themselves but direct the australasian journal of information systems banerjee & gupta 2019, vol 23, research on the role of tech in workforce mgt talent attraction through online recruitment 8 job-seekers to third party blogs to view the testimonials, it may likewise create a greater sense of authenticity and trustworthiness for the message which again follows from tam and ypcm (petty et al., 1987; knight & burn, 2011). this leads to the following hypotheses, h6a: the relationship between perceived quality and organizational attractiveness will be moderated by advertisement-source such that the effect will be higher for company-independent source as compared to that for company dependent source of advertisement. h6b: the relationship between perceived credibility and organizational attractiveness will be moderated by advertisement-source such that the effect will be higher for company-independent source as compared to that for company dependent source of advertisement. (note: the moderation hypotheses are represented in a combined manner) figure 1: research model 4 research methodology an experimental design was adopted to test whether different types of corporate previews, testimonials and ad-sources can have different effect on the job-seekers’ desire to apply for job. for this, six different websites were created depicting a career webpage of a fictitious firm in six different styles with the help of a professional web-development agency. the authors used an open-source online website design service provider ‘webs’ (www.webs.com) to create a fictitious firm, ‘eco-tech concepts and solutions’ (henceforth referred to as ecs) website. the basic website development required an understanding of what a career website of an organization must look like to be considered good. for this purpose, the authors decided to take cue from both published studies on website content in recruitment context as well as through a content analysis of some of the best companies operating in india. a coding structure was developed by carefully scrutinizing earlier content australasian journal of information systems banerjee & gupta 2019, vol 23, research on the role of tech in workforce mgt talent attraction through online recruitment 9 analyses of job-related websites such as the studies from koong and colleagues (2002), cober and associates (2004), young and foot (2005), yoon kin tong and sivanand (2005), and banerjee and gupta (2016). these insights were used to design the prototype of a website having all possible features of one operating in real world so that the fictitious company looks real from similar industry context (banerjee & gupta, 2016). like any website may have multiple web-pages, this website also contained five separate web pages, namely homepage, about us, services, join us and contact us. the ‘homepage’ was used to describe the business aspect of the company. the ‘services’ page highlighted four different verticals of the firm. the ‘contact us’ page indicates the location of the head-quarter of the company and there was provision for customer query. the ‘join us’ page of the six different websites featured different manipulations. these manipulations include positive or realistic testimonials, a snapshot of which is provided in figure 2 which is showing rjp type testimonials (pros and cons) from the website 2), presence of testimonials across websites or through a fictitious external blog (named insideoffice) which was linked with the firm’s ‘join us’ page which is shown through figure 3 depicting an external blog in the website 4). the comments posted by employees about the company on this blog could be viewed by anyone and did not require any registration. this blog was created keeping in mind similar platforms like glassdoor and quora which are known for hosting anonymous feedback by ex-employees of firms. a manipulation was also made in the ‘about us’ page, where two different modes of corporate previewtext based and audio-visual have been created (figure 4: websites 5 and 6). the audio-visual manipulation was created using ‘windows moviemaker’ software. the content for these videos were picked from videos available online either on company websites or youtube, in order to maintain realistic feel of the company. it was ensured that none of the video clippings used could be used to identify any actual company from the video used through suitable pre-tests. the employee testimonials were designed to eliminate gender bias, by having one male and one female narrating their experience about the company in positive light for one website. in another version, employees were seen talking about the pros and cons of working at the ecs. three web development experts were also invited to give their opinion on the web development and to verify the website design. finally, the authors had six different websites, each with different combinations of preview-mode, testimonial type and testimonial source which were used as treatments in the following experiment. australasian journal of information systems banerjee & gupta 2019, vol 23, research on the role of tech in workforce mgt talent attraction through online recruitment 10 figure 2: realistic employee testimonials as depicted in website 2 figure 3: the website 4 showing an external blog where ecs employee testimonials can be accessed australasian journal of information systems banerjee & gupta 2019, vol 23, research on the role of tech in workforce mgt talent attraction through online recruitment 11 figure 4: website 5 showing audio-visual streaming video tour preview of organization 4.1 measurement of variables a survey questionnaire based on 5-point likert-type scale was used to collect data on the variables of interest for this study. the scale items were derived from existing valid psychometric instruments which have been repeatedly used by past researchers in related study contexts. the scale reliabilities were observed to be above 0.7 in all the cases (nunnally, 1978), thus giving evidence of good reliability of the indices. perceived job advertisement credibility is measured using a scale to measure perception of credibility of online information developed by johnson and kaye (2004). this is a 4 item likert type scale with feedback option ranging from 1=not at all believable and 5=very much believable. example of a scale item is ‘how believable did you find the job-related information presented in the join us section?’ perceived website quality is measured using the perceived design quality subscale of webqual 3.0 developed by barnes and vidgen (2001). this subscale measures generic perception regarding the design and layout of a website. it is an eight-item scale, which uses five-point likert 1=strongly disagree and 5=strongly agree, to capture responses. organizational attractiveness is measured using the 4 item 5-point likert scale created by highhouse et al., (2002). a sample item is “this company is attractive to me as a place for employment”. intention to apply is measured using a 3 item 5-pointscale developed by van birgelen, wetzels and van dolen (2008). a sample item is “i would like to accept a job offer from this company”. control and demographic variables: age, gender, and work experience have been controlled in the study. 4.2 data collection data was collected using a web-based survey with job-seekers who have already been working for the past 1-2 years and have registered their details in various popular third-party job australasian journal of information systems banerjee & gupta 2019, vol 23, research on the role of tech in workforce mgt talent attraction through online recruitment 12 portals in india such as naukri.com, monster.com etc. the criterion for selection was that the registered users should have been actively involved in online job search during the time of the survey (indicated through the frequency of visit/ updates made in their profile). job-seekers registered with either naukri or monster india or both with either or both engineering and mba backgrounds and with at least one year of experience were identified as the target population. this choice was justified by the fact that naukri and monster cater to the largest contingent of online recruitment service users in india. the email contact of the registered users in naukri and monster could be accessed through the co-operation of third-party recruiters who have employer login access to these websites. in order to maintain the confidentiality of the individuals, only a list of email-ids was provided by the third-party recruiters, who have either or both mba and engineering degrees and have worked for at most 2 years. the final list included 1425 email-ids of registered e-recruitment users. the respondents were then contacted through a formal email briefing them about the purpose of the study and then requesting them to take part in the survey. random allocation technique was used to ensure parametric estimation of results (forder, gebski, & keech, 2005). in total, 361 recipients responded to the invitation and agreed to take part in the study. then using random number table, the respondents were assigned to six different groups of approximate size of 60 members. the link to the six different websites along with a short online survey was then sent to the randomly selected groups to achieve randomized allocation. the online questionnaire was designed with a very simple interface so that participants can understand the necessary actions to be able to complete the survey. all questions were mandatory, i.e., unless respondents answered all the queries, the questionnaire could not be submitted. the completed responses which were submitted got recorded in an excel file. hence, there was no case of missing data. approximately 64 % respondents were male and 75 % were in the age group of 20-25 years. respondents were mostly graduate (44.5 %) and postgraduate (51%) with mba and engineering degrees accounting for 80 % of the educational qualification. over 60 % respondents reported work experience in the range of 1-2 years while another 30 % reported work experience of 2-3 years. nearly 68 % respondents replied that they have been actively involved in online job search process in the past 1-3 years. further, 98 % respondents replied that they have registered profiles in naukri.com while 90 %replied that they have profile in monster.com, followed by timesjobs.com at a distant third (52%). 4.3 data analysis overall, the descriptive statistics gave indications that the respondents in our sample had fairly high perceptions of quality of the website (mean= 3.515, sd=1.1082) and credibility (mean=3.368, sd=1.1352). perceived organizational attractiveness (mean=3.798, sd= 0.9071) and intention to apply (mean=3.471, sd=1.0464) were also high in the individuals (refer to table 1).the correlation matrix (table 1) gives indications that perceived credibility is positively correlated with perceived quality (r=.420), organizational attractiveness (r=.386) and intention to apply (r=.537). perceived quality is also positively correlated with attractiveness (r=.356) and intention to apply (r=.651). australasian journal of information systems banerjee & gupta 2019, vol 23, research on the role of tech in workforce mgt talent attraction through online recruitment 13 name of construct m sd cronbach alpha 1. 2. 3. 4. 1.perceived credibility 3.3684 1.13529 0.94 1 .820 .386 .937 2. perceived quality 3.5152 1.10826 0.96 .820 1 .356 .851 3. perceived organizational attractiveness 3.7978 .90713 0.94 .386 .356 1 .431 4. intention to apply 3.4709 1.04638 0.91 .937 .851 .431 1 * correlation is significant at the 0.01 level (one-tailed). table 1: descriptive statistics and correlation diagnostics organizational attractiveness is found to have a positive correlation with intention to apply (r=.431). cronbach alpha values are all above 0.7 (nunnally, 1978) indicating good reliability of the scales. a covariance based structural equation model (sem) is estimated to assess the hypothesized relationship (refer to figure 5). the fit of the structural model is acceptable, with χ2(77) =115.612 (p<.01), goodness of fit index (gfi) = 0.919, comparative fit index (cfi) = 0.989, incremental fit index (ifi) = 0.923 and root mean square error of approximation (rmsea) = 0.016. cmin/df is 1.501. (note: for each group, the β values are different; hence the coefficient values are not reported here) figure 5: sem based converged model for multi-group moderated mediation analysis based on these results, the measurement model indicates an acceptable model fit of the data (hair, anderson, tatham, & black, 2008). consistent with the different cut-off criteria provided in the literature (jöreskog, 1993; hu & bentler, 1999), it is concluded that the hypothesized causal model is within the acceptable range of all the fit statistics. the control variables, namely, gender (β=0.009, p>0.10), age (β=-0.10, p>0.10) and work experience (β=0.027, p>0.10) were all observed insignificant in affecting job-seekers’ intention to apply to australasian journal of information systems banerjee & gupta 2019, vol 23, research on the role of tech in workforce mgt talent attraction through online recruitment 14 online job advertisements. perceived quality of website (β= 0.331, p<0.001) and perceived credibility of online job advertisement (β=0.566, p<0.001) are observed to have positive effects on organizational attractiveness, thus providing support for hypotheses 1 and 2. for testing hypothesis 3, a mediation analysis was conducted. 4.4 results from mediation analysis a mediation analysis is conducted using bootstrapping technique (hayes, 2009; lowry & gaskin, 2014) in sem to test whether firm attractiveness mediates the relationship between perceived quality of website and intention to apply and between perceived credibility of online job advertisement and intention to apply. the direct effect of perceived quality of website is found to be significant on intention to apply (β=0.331, p<0.001). when the mediator is introduced, then the direct effect of perceived quality on intention to apply continues to be significant (β=0.143, p<0.001). perceived website quality also had a positive effect on firm attractiveness (β=0.241, p<0.001), while attractiveness significantly predicted intention to apply (β=0.784, p<0.001). therefore, a firm’s attractiveness acted as a partial mediator in this context, thus providing partial support for hypothesis 3a. perceived credibility of job advertisement also demonstrated direct effect (β=0.566, p<0.001) on intention to apply. when the mediator was introduced, the path from predictor variable to mediator was also significant (β=0.669, p<0.001), as was the path from mediator to predictor (β=0.784, p<0.001). but the path from predictor to outcome variable became insignificant (p>0.10). therefore, organizational attractiveness acted as a full mediator in this case, and thus hypothesis 3b received full support. 4.5 results from moderated mediation analysis a moderated-mediation analysis was conducted using the procedure outlined by preacher, rucker and hayes (2008) and by lowry and gaskin (2014) to test the hypotheses 4, 5 and 6. moderated mediation is defined as ‘a mediated effect that varies across levels of a moderator, either on the path from the independent variable to the mediator or from the mediator to the outcome variable’ (edwards & lambert, 2007; ng, ang, & chan, 2008).in this case, we expected that the path from perceived quality and perceived credibility to organizational attractiveness will be moderated due to different types of preview-mode, testimonial-type or testimonial-source. from table 2, it can be observed that depending on the type of corporate preview-mode, the nature of mediating effect of organizational attractiveness on the relationship between perceived quality of e-recruitment website and job-seekers’ intention to apply changes from partial to full mediation. therefore, preview-mode type moderates the mediating effect and the indirect effect is also more in case of audio-visual preview (indirect β=0.189, p<0.01), suggesting that this mode of preview has more impact on intention to apply than text preview (indirect β=0.148, p<0.05). therefore, hypothesis 4a received full support. with respect to the type of testimonial, we did not observe any change in mediation type, which indicated that testimonial type did not moderate the mediating effect of organizational attractiveness on the perceived quality-intention to apply relationship. australasian journal of information systems banerjee & gupta 2019, vol 23, research on the role of tech in workforce mgt talent attraction through online recruitment 15 design manipulations level of moderation perceived quality→organizational attractiveness→intention to apply pq→intention (without mediator oa) pq→intention (with mediator oa) total indirect effect (β) nature of mediation preview-mode text 0.331*** 0.143*** 0.148* partial mediation audio-visual 0.206*** 0.048 0.189** full mediation testimonialtype positive 0.238*** 0.116** 0.125* partial mediation realistic 0.285*** 0.099* 0.186** partial mediation advertisementsource company dependent 0.127* 0.056 0.075* full mediation company neutral 0.308*** 0.184*** 0.133* partial mediation note: ***p<0.001, **p<0.01, *p<0.05, (two tailed) pqperceived quality; intention – intention to apply; mediatororganizational attractiveness table 2: moderated mediation results for perceived organizational attractiveness across different design manipulations of online job advertisement on the association between perceived quality of online job advertisements and applicant’s intention to apply however, the indirect effect of the partial mediation with the realistic employee testimonials (indirect β=0.186, p<0.01) was greater than the website with positive testimonial (indirect β=0.125, p<0.05). however, hypothesis 5a did not receive support since there was no change in the type of mediation. hence, we got indication that the type of employee testimonial did not matter for website quality perception. as for the source of job advertisement, we received support for hypothesis6a since the nature of mediation changed from full to partial. the indirect effect for the website with company-dependent advertisement (indirect β=0.075, p<0.05) was less than the featuring company neutral blog-based advertisement (indirect β= 0.133, p<0.05), which indicated that blogs may be better suited for influencing job-seekers’ intention to apply. table 3 demonstrates the moderated mediation analysis with respect to perceived job advertisement credibility, organizational attractiveness and intention to apply. from table 3, it follows that all the three website modifications exert moderated mediation effect on the relationship between the above-mentioned variables. therefore, hypotheses 4b, 5b and 6b all received support. on closer inspection, the indirect effect for website with realistic employee testimonials (indirect β = 0.558, p<0.001) is distinctly higher than website with positive testimonial (indirect β=0.492, p<0.001), which indicates that rjp based employee testimonials were appraised more favourably than only positively framed ones. this provides support for our argument that realistic employee testimonials may be better suited to evoke credibility feeling in the job-seekers’ mind. it is also observed that in case of the website with advertisement featured through blog, the mediating effect (indirect β = 0.589, p<0.001) is considerably higher than the company website featured advertisement (indirect β=0.514, p<0.001). for the preview-mode manipulation, the difference in indirect effect is not encouraging. this indicates that preview-mode does not create any notable effect on credibility perception. australasian journal of information systems banerjee & gupta 2019, vol 23, research on the role of tech in workforce mgt talent attraction through online recruitment 16 design manipulations level of moderation perceived credibility→organizational attractiveness→intention to apply pc→intention (without mediator oa) pc→intention (with mediator oa) total indirect effect (β) nature of mediation preview-mode text 0.566*** 0.042 0.524** full mediation audio-visual 0.664*** 0.261*** 0.529** partial mediation testimonialtype positive 0.238*** 0.116** 0.492** partial mediation realistic 0.285*** 0.049 0.558** full mediation advertisementsource company dependent 0.719*** 0.169** 0.514** partial mediation company neutral 0.580*** 0.009 0.589** full mediation note: ***p<.001, **p<.01, *p<.05 (two-tailed) pcperceived credibility; intention – intention to apply; mediatororganizational attractiveness table 3: moderated mediation results for perceived organizational attractiveness acrossdifferent design manipulations of online job advertisement on the association between perceived credibility of online job advertisements and applicant’s intention to apply 5 discussion the findings provide strong support for our proposed relational model. it is observed that jobseekers’ perception about website quality is influenced by the use of attractive audio-visual rather than through plain text description. a glimpse of the internal work culture possibly increases their interest in the employer. this subsequently increases their intention to apply for jobs posted by firms which advertise their workplace environment innovatively. it is also observed that employee testimonials highlighting both pros and cons of a firm are viewed as more believable and such testimonials can enhance the perceived credibility of online job advertisements; since job-seekers get the signal that the potential employers are not sugarcoating the opportunity to work for them. finally, it is observed that online job advertisements linked with external employee review blogs increases the credibility of the advertisement and enhances the perceived attractiveness of the employer in the mind of the job-seekers. specifically, if corporate recruiters link their career web-pages with external employee review blogs such as glassdoor, then such an arrangement may be viewed as an honest effort by the employer to project what their organization is really about. 5.1 managerial and societal implication the future of labour market across countries is going to be skewed in such a manner that in certain countries, the working age population will continue to decline (e.g., japan) while in some others, there is going to be an abundance of youth (e.g., china and india) in the coming decades (united nations world population prospect, 2010). for instance, india is a country with a huge youth population as per the 2010 report by united nations world population prospects (undp), but, as of now, there is more supply of labour than it is required. on the other hand, countries like japan are going to face huge labour shortage in near future due to its ageing population (undp, 2010). this is going to posit a scenario, where countries facing population decline will rely on global workforce to fill up the future vacancies. in this endeavour, they will have to strongly rely on online recruitment as a medium of talent australasian journal of information systems banerjee & gupta 2019, vol 23, research on the role of tech in workforce mgt talent attraction through online recruitment 17 attraction, since the global workforce is geographically spread out across the world. under such circumstances, the websites of the firms of such countries will be the very first line of contact between the firms and potential candidates. another important development in near future is going to be the reshaping of traditional job role itself. the job market is already beginning to observe the phenomena such as crowd-sourcing, freelancing and jigs replacing more conventional direct payroll jobs. gradually, future jobs are going to shift to a more contractual type model, and in such a scenario, traditional recruitment channels will give way to social media recruiting, recruiting on the twitter and through virtual worlds such as second-life (russel, 2008). this increase in web-based recruitment is going to be a certainty as per industry experts (walker et al., 2010). hence, considering the changing dynamics of global workforce, our research in this area will add value to existing practitioner knowledge. 5.2 limitations and future directions this study provides empirical support regarding the impact that web 2.0 tools may have in addressing the problem of low credibility associated with e recruitment channels. however, since the study is experimental in nature, the lack of external validity is a limitation for this paper. further empirical validation of the same is required for strengthening the findings in this article. while the existing theoretical and empirical evidence do point towards the requirement for use of rjps and blogs in online career web-pages, it may not be absolutely mandatory since ‘job’ as product has its unique appeal that is missing in traditional products and services. often jobseekers may overlook such cues if they are in desperate need for employment and all they want is a job. it is only when a sustainable talent acquisition issue arises supplemented by shortage of adequate skilled workers that the issue of talent attraction and competition to hire the best will arise. hence the framework may not be applicable in a labour market, where job is a crisis. 6 conclusion this study highlights the role of web 2.0 technologies on attracting the right talent from the vast indian labour market which can reduce delay in sourcing candidates and also help in quality hiring. specifically, this study firms need to leverage on podcast technology to highlight their workplace, post their virtual workplace tours through their official youtube page and showcase the same through their website. the present study provides empirical support for incorporating this feature in corporate career websites. another point to note is that job-seekers who use e-recruitment extensively are mostly from the millennial generation (walker et al., 2009). this generation is more internet savvy than previous cohorts and they are more likely to leverage the potential of online recruitment. it is also more probable that online job-seekers will do their own background search about potential employers before applying for jobs. one likely source of getting such information can be through blogs and forums such as glassdoor and quora (mallon, 2017; thomas, 2017). glassdoor, an online platform for hosting job and company related information, features over 35 million employee reviews, while quora has a footfall of 190 million users per month. in our study, it is empirically shown that job-seekers value advertisements presented through blogs which do not have any apparent connection with the firm. information posted through such platforms trigger more trust in the minds of the job-seekers and this leads to increased inclination towards applying for jobs posted through company neutral platforms. australasian journal of information systems banerjee & gupta 2019, vol 23, research on the role of tech in workforce mgt talent attraction through online recruitment 18 this study further validates that rjps can also play a crucial role in enhancing the credibility of the recruitment message and this type of communication needs to be further explored. however, there may be issues regarding the extent to which neutral platforms can ensure the credibility of the information posted through such channels. for example, what if anonymous reviewers post malicious and fraudulent content out of some personal grudge against any employer? the solution to nullify such event lies in the degree to which third party job-blogs would go to authenticate identity of a user and in their readiness to take legal actions against those who misuse the forum. glassdoor provides a detailed list of its responsibilities and vigilance to keep any unhealthy discussion off their blog space through their privacy policies and terms and conditions. future business ventures should keep such intricacies in mind if they wish to appeal to credibility perceptions of job-seekers in proper light. references aaker, j. l. (1997). dimensions of brand personality. journal of marketing research, 34, 347–356. adapa, s., & cooksey, r. (2013). factors affecting consumer's continued use of internet banking: empirical evidence from australia. australasian journal of information systems, 18(1), 5-31. ajzen, i. (1991). the theory of planned behavior. organizational behavior and human decision processes, 50(2), 179-211. allen, d. g., mahto, r. v., & otondo, r. f. (2007). webbased recruitment: effects of information, organizational brand, and attitudes toward a web site on applicant attraction, journal of applied psychology, 92(6), 1696-1708. ambler, t., & barrow, s. (1996). the employer brand. journal of brand management, 4(3), 185206. banerjee, p. (2016). attracting job-seekers through online job advertisements: application of rjps, blogs and video podcasts. international journal of technology and human interaction, 12(3), 1-17. banerjee, p., & gupta, r. (2016). design deficiencies in corporate career web-sites in india: a content analysis. human systems management, 35(4), 291-300. barksdale jr, h. c., bellenger, d. n., boles, j. s., & brashear, t. g. (2003). the impact of realistic job previews and perceptions of training on sales force performance and continuance commitment: a longitudinal test. journal of personal selling & sales management, 23(2), 125-138. barnes, s. j., & vidgen, r. t. (2002). an integrative approach to the assessment of e-commerce quality. journal of electronic commerce research., 3(3), 114-127. bewsell, g. r. (2009). online trust mechanisms: a domain study of online auctions. australasian journal of information systems, 15(2), 149-167. borstorff, p. c., marker, m. b., & bennett, d. s. (2007). online recruitment: attitudes and behaviors of job seekers. journal of strategic e-commerce, 5(1/2), 1. breaugh, j. a., & starke, m. (2000). research on employee recruitment: so many studies, so many remaining questions. journal of management, 26(3), 405-434. australasian journal of information systems banerjee & gupta 2019, vol 23, research on the role of tech in workforce mgt talent attraction through online recruitment 19 cable, d. m., aiman-smith, l., mulvey, p. w., & edwards, j. r. (2000). the sources and accuracy of job applicants' beliefs about organizational culture. academy of management journal, 43(6), 1076-1085. cable, d. m., & yu, k. y. t. (2006). managing job seekers' organizational image beliefs: the role of media richness and media credibility. journal of applied psychology, 91(4), 828. cable, d. m., & yu, k. y. t. (2007). how selection and recruitment practices develop the beliefs used to assess fit. in perspectives on organizational fit, 155-182. chandler, p., & sweller, j. (1991). cognitive load theory and the format of instruction. cognition and instruction, 8(4), 293-332. chapman, d. s., & webster, j. (2003). the use of technologies in the recruiting, screening, and selection processes for job candidates. international journal of selection and assessment, 11(2‐3), 113-120. chatterjee, s. r. (2007). human resource management in india: ‘where from’ and ‘where to?’research and practice in human resource management, 15(2), 92-103. cober, r. t., brown, d. j., & levy, p. e. (2004). form, content, and function: an evaluative methodology for corporate employment web sites, human resource management, 43(2/3), 201−218. daft, r. l., & lengel, r. h. (1986). organizational information requirements, media richness and structural design. management science, 32(5), 554-571. de goede, m. e., van vianen, a. e., & klehe, u. c. (2011). attracting applicants on the web: po fit, industry culture stereotypes, and website design. international journal of selection and assessment, 19(1), 51-61. dineen, b. r., ash, s. r., & noe, r. a. (2002). a web of applicant attraction: personorganization fit in the context of web-based recruitment, journal of applied psychology, 87 (4), 723−734 doh, j.p., stumpf, s.a., tymon, w. & haid, m. (2008). how to retain talent in india? mit sloan review, 50 (1), 5-7. dugoni, b.l., & ilgen, d.r. (1981). realistic job previews and the adjustment of new employees. academy of management journal, 24(3), 579-591. edwards, j. r., & lambert, l. s. (2007). methods for integrating moderation and mediation: a general analytical framework using moderated path analysis. psychological methods, 12(1), 1. fisher, c.d., ilgen, d.r., & hoyer, w.d. (1979). source credibility, information favorability and job offer acceptance. academy of management journal, 22(1), 94-103. forder, p. m., gebski, v. j., & keech, a. c. (2005). allocation concealment and blinding: when ignorance is bliss. medical journal of australia, 182(2), 87-89. gokhale, p.a., & chandra, s. (2009). web 2.0 and e learning: the indian perspective. desidoc journal of library and information technology, 29 (1), 5-13. australasian journal of information systems banerjee & gupta 2019, vol 23, research on the role of tech in workforce mgt talent attraction through online recruitment 20 gregory, c. k., meade, a. w., & thompson, l. f. (2013). understanding internet recruitment via signaling theory and the elaboration likelihood model. computers in human behavior, 29(5), 1949-1959. hair, j.f., anderson, r.e., tatham, r.l., & black, w.c. (1998). multivariate data analysis (5th ed.). prentice hall. highhouse, s., hoffman, j. r., greve, e. m., & collins, a. e. (2002). persuasive impact of organizational value statements in a recruitment context. journal of applied social psychology, 32(8), 1737-1755. howardson, g. n., & behrend, t. s. (2014). using the internet to recruit employees: comparing the effects of usability expectations and objective technological characteristics on internet recruitment outcomes. computers in human behavior, 31, 334-342. hu, l. t., & bentler, p. m. (1999). cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. structural equation modeling: a multidisciplinary journal, 6(1), 1-55. india's best companies to work for: the complete list (2017). et bureau, 3rd july, 2017. available at: https://economictimes.indiatimes.com/news/company/corporatetrends/indias-best-companies-to-work-for-2017-the-completelist/articleshow/59419981.cms(accessed on: 3rd december, 2017, 12:45 pm) johnson, t. j., & kaye, b. k. (2004). wag the blog: how reliance on traditional media and the internet influence credibility perceptions of weblogs among blog users. journalism & mass communication quarterly, 81(3), 622-642. jöreskog, k. g. (1993). testing structural equation models. in k. a. bollen& j. s. lang (eds.)testing structural equation models (pp.294–316). newbury park, ca: sage. knight, s. a., & burn, j. m. (2011). a preliminary introduction to the otam: exploring users’ perceptions of their on-going interaction with adopted technologies. australasian journal of information systems, 17(1), 5-41. koong, k. s., liu, l. c., & williams, d. l. (2002). an identification of internet job board attributes. human systems management, 21(2), 129-135. koufaris, m. (2002). applying the technology acceptance model and flow theory to online consumer behavior. information systems research, 13(2), 205-223. kumari, g., bhat, j., & pandey, k. m. (2010). recruitment and selection process: a case study of hindustan coca-cola beverage pvt. ltd, gangyal, jammu, india. international journal of innovation, management and technology, 1(4), 441. lim, w. m. (2018). dialectic antidotes to critics of the technology acceptance model: conceptual, methodological, and replication treatments for behavioral modeling in technology-mediated environments. australasian journal of information systems, 22, 1-11. lin, h-f. (2010). applicability of the extended theory of planned behavior in predicting job seeker intentions to use job-search websites. international journal of selection and assessment, 18 (1), 64-74. loiacono, e. t., watson, r. t., & goodhue, d. l. (2007). webqual: an instrument for consumer evaluation of web sites. international journal of electronic commerce, 11(3), 51-87. australasian journal of information systems banerjee & gupta 2019, vol 23, research on the role of tech in workforce mgt talent attraction through online recruitment 21 lowry, p. b., & gaskin, j. (2014). partial least squares (pls) structural equation modeling (sem) for building and testing behavioral causal theory: when to choose it and how to use it. ieee transactions on professional communication, 57(2), 123-146. maurer, s.d., & liu, y. (2007). developing effective e-recruiting websites: insights for managers from marketers. business horizons, 50, 305314. maurer, s. d., & cook, d. p. (2011). using company web sites to e-recruit qualified applicants: a job marketing-based review of theory-based research. computers in human behavior, 27(1), 106-117. mcgowan, m.k. (2010). the unexplored new medium: recent trends in podcast advertising. the elon journal of undergraduate research in communications, 1 (2), 97-111. meglino, b.m., ravlin, e.c. & denisi, a.s. (1997). when does it hurt to tell the truth? the effect of realistic job previews on employee recruiting. public personnel management, 26 (3), 413422. mallon, d. (2017). thanks to the ‘glassdoor effect,’ companies will never be the same. available at: https://www.benefitnews.com/opinion/thanks-to-the-glassdoor-effectcompanies-will-never-be-the-same monster employment index (2017). available at: http://media.monsterindia.com/logos/mei/moin/mei_in_dec17.pdf naukri job speak (2017). available at: http://www.infoedge.in/pdfs/jobspeak-report-april2017.pdf ng, k. y., ang, s., & chan, k. y. (2008). personality and leader effectiveness: a moderated mediation model of leadership self-efficacy, job demands, and job autonomy. journal of applied psychology, 93(4), 733. nunnally, j. c. (1978). psychometric theory (2nd ed.). new york: mcgraw-hill. oskamp, s. (1977). attitudes and opinions. prentice hall, englewood cliffs, n.j. petty, r.e., kasmer, j., haugtvedt, c. & cacioppo, j. (1987). source and message factors in persuasion: a reply to stiff's critique of the elaboration likelihood model. communication monographs, 54 (3), 233-249. pfieffelmann, b., wagner, s. h., &libkuman, t. (2010). recruiting on corporate web sites: perceptions of fit and attraction. international journal of selection and assessment, 18(1), 4047. phillips, j. m. (1998). effects of realistic job previews on multiple organizational outcomes: a meta-analysis. academy of management journal, 41 (6), 673-690. popovich, p., & wanous, j. p. (1982). the realistic job preview as a persuasive communication. academy of management review, 7(4), 570-578. preacher, k. j., rucker, d. d., & hayes, a. f. (2007). addressing moderated mediation hypotheses: theory, methods, and prescriptions. multivariate behavioral research, 42(1), 185-227. reeve, c. l., highhouse, s., & brooks, m. e. (2006). a closer look at reactions to realistic recruitment messages. international journal of selection and assessment, 14(1), 1-15. australasian journal of information systems banerjee & gupta 2019, vol 23, research on the role of tech in workforce mgt talent attraction through online recruitment 22 russell, j. (2009). web 2.0 technology: how is it impacting your employer brand? nursing economics, 27(5), 335. saks, a. m., & cronshaw, s. f. (1990). a process investigation of realistic job previews: mediating variables and channels of communication. journal of organizational behavior, 11(3), 221-236. silverthorne, c. p., & mazmanian, l. (1975). the effects of heckling and media of presentation on the impact of a persuasive communication. the journal of social psychology, 96(2), 229236. thomas, a. (2017). quora raises $85 million, the q&a platform is now a unicorn. available at: https://economictimes.indiatimes.com/small-biz/money/quora-raises-85-million-theqa-platform-is-now-a-unicorn/articleshow/58313275.cms tyagi, a. (2012). effective talent acquisition through e recruitment: a study. excel international journal of multidisciplinary management studies, 2(1), 148-156. united nations world population prospect (2010), available at: http://esa.un.org/wpp/exceldata/population.htm (accessed on 14.01.2011). van hoye, g., & lievens, f. (2007). investigating web‐based recruitment sources: employee testimonials vs word‐of‐mouse. international journal of selection and assessment, 15(4), 372-382. van birgelen, m. j., wetzels, m. g., & van dolen, w. m. (2008). effectiveness of corporate employment web sites: how content and form influence intentions to apply. international journal of manpower, 29(8), 731-751. vicknair, j., elkersh, d., yancey, k., & budden, m. c. (2010). the use of social networking websites as a recruiting tool for employers. american journal of business education, 3(11), 7. walker, h. j., feild, h. s., giles, w. f., armenakis, a. a., & bernerth, j. b. (2009). displaying employee testimonials on recruitment web sites: effects of communication media, employee race, and job seeker race on organizational attraction and information credibility. journal of applied psychology, 94(5), 1354. yoon kin tong, d., & sivanand, c. n. (2005). e-recruitment service providers review: international and malaysian. employee relations, 27(1), 103-117. copyright: © 2019 banerjee & gupta. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems banerjee & gupta 2019, vol 23, research on the role of tech in workforce mgt talent attraction through online recruitment 23 paper title (paper title style) australasian journal of information systems mccormac, calic, butavicius et al 2017, vol 21, selected papers from acis 2016 information security awareness and bias 1 a reliable measure of information security awareness and the identification of bias in responses agata mccormac defence science and technology group edinburgh, south australia agata.mccormac@dst.defence.gov.au dragana calic defence science and technology group edinburgh, south australia marcus butavicius defence science and technology group edinburgh, south australia kathryn parsons defence science and technology group edinburgh, south australia tara zwaans school of psychology the university of adelaide adelaide, south australia malcolm pattinson adelaide business school the university of adelaide adelaide, south australia abstract the human aspects of information security questionnaire (hais-q) is designed to measure information security awareness. more specifically, the tool measures an individual’s knowledge, attitude, and self-reported behaviour relating to information security in the workplace. this paper reports on the reliability of the hais-q, including test-retest reliability and internal consistency. the paper also assesses the reliability of three preliminary overclaiming items, designed specifically to complement the hais-q, and identify those individuals who provide socially desirable responses. a total of 197 working australians completed two iterations of the hais-q and the over-claiming items, approximately 4 weeks apart. results of the analysis showed that the hais-q was externally reliable and internally consistent. therefore, the hais-q can be used to reliably measure information security awareness. reliability testing on the preliminary over-claiming items was not as robust and further development is required and recommended. the implications of these findings mean that organisations can confidently use the hais-q to not only measure the current state of employee information security awareness within their organisation, but they can also measure the effectiveness and impacts of training interventions, information security awareness programs and campaigns. the influence of cultural changes and the effect of security incidents can also be assessed. keywords: information security, information security awareness, cyber security, reliability, questionnaire design. 1 introduction employee information security awareness (isa) is critical in mitigating the risks associated with cyber security incidents (arachchilage & love, 2014; safa, von solms, & furnell, 2016). therefore, it is crucial for organisations to be able to measure their employees’ isa. through australasian journal of information systems mccormac, calic, butavicius et al 2017, vol 21, selected papers from acis 2016 information security awareness and bias 2 understanding employee isa, organisations can identify areas of strength and weakness, and use this information to tailor their training and awareness programs to improve their isa. it is increasingly acknowledged that security incidents cannot be mitigated through solely technical solutions (parsons et al., 2010; parsons, mccormac, butavicius, pattinson & jerram, 2014). human error plays a major role in information security breaches, with employees consistently identified as the main source of compromise (pricewaterhouse coopers (pwc), 2015; 2017). consequently, the consideration of the human element of information security has become increasingly important for organisations around the world. in this paper, we examine the test-retest reliability and internal consistency of the human aspects of information security questionnaire (hais-q) and report on the appropriateness of using the hais-q as a reliable measure of isa. previous research has validated the hais-q as a measure of isa and has demonstrated its internal consistency (parsons et al., 2017; parsons, mccormac, butavicius, pattinson, & jerram, 2014). in this paper we evaluate test-retest reliability to provide evidence about the extent to which the hais-q is reliable and stable. demonstrating that the hais-q is a reliable and stable measure of isa can enable organisations to confidently assess the effectiveness of information security training and intervention strategies, in conjunction with organisational changes. when developing any security assessment instrument it is important to consider the extent to which the results may be affected by social desirability bias. this form of bias can affect the reliability and validity of survey findings. hence, in this study participants also provided responses to three preliminary over-claiming items designed to ascertain if participants were responding in a socially desirable manner. the reliability of the hais-q over-claiming questions will also be assessed using test-retest correlations and cronbach’s alpha. 1.1 reliability reliability can be described as reputability of findings, and it serves to demonstrate the consistency of a measure or an instrument. essentially, if findings are able to be replicated consistently, they are considered to be reliable (portney & watkins, 2015). although reliability does not imply validity of a measure, without reliability, the validity of a measure is compromised (streiner, 2003). there are two types of reliability, internal reliability and external reliability. external reliability refers to the extent to which an instrument or a test varies from one administration to another. external reliability can be captured by assessing the test-retest of an instrument. test-retest requires the same participants to complete a test at two different times (portney & watkins, 2015). to determine if the hais-q is a reliable measure of isa, an individual should obtain a similar score if they are tested twice. if test-retest reliability can be demonstrated, it shows that the hais-q could also be used to assess the effectiveness of intervention strategies. there are a number of factors that may affect an individual’s results across multiple completions (allen & yen, 2001). for example, factors such as the completion of intervening information security training or changes in work or personal lives can influence responses. however, scores on a reliable test should still correlate highly. the time interval between the initial test (i.e., t1) and the retest (i.e., t2) should be sufficiently long to minimise practice effects, carry over effects and recall (allen & yen, 2001). other test-retest studies employed in organisational environments have used a two to eight week time interval period, therefore, a three to four week time delay between t1 and t2 was deemed to be sufficient (burch & anderson, 2004; griffiths, cox, karanika, khan, & tomas, 2006; traynor & wade, 1993). a measurement tool should have a test-retest coefficient of greater than .70 to illustrate external reliability (van saane, sluiter, verbeek, & frings-dresen, 2003). internal reliability is also referred to as internal consistency. it is the extent to which a measure is consistent within itself. cronbach’s alpha is used to measure the consistency of results, across items and within a measure (cronbach, 1951). an acceptable cronbach’s alpha should be over .70 (devellis, 2011). australasian journal of information systems mccormac, calic, butavicius et al 2017, vol 21, selected papers from acis 2016 information security awareness and bias 3 interpretation of alpha scores is important; this is because a high alpha level does not always reflect high internal consistency. when items within a measure are highly correlated, then the alpha level will also be high. lower alpha levels can be associated with low number of questions, poor correlations between items or heterogeneous constructs. these issues are important to keep in mind when interpreting findings and reliability assessments (tavakol & dennick, 2011). 1.2 information security awareness (isa): previous hais-q research isa can be defined as the extent to which a person understands the importance and implications of information security policies, rules and guidelines, as well as, the extent to which they are committed to and behave in accordance with these policies, rules and guidelines (kruger & kearney, 2006; siponen, 2000). this definition is consistent with the knowledgeattitude-behaviour (kab) model that the hais-q is founded upon. based on the kab model, as an employee’s level of knowledge of information security policy and procedures increases, their attitude towards information security policy and procedures improves, and this results in better information security behaviour (parsons, mccormac, butavicius, et al., 2014). the hais-q measures isa by examining seven focus areas, namely, password management, email use, internet use, social media use, mobile devices, information handling and incident reporting. the hais-q has been tested on diverse samples, using different methodologies. for example, content validity was assessed by pattinson, butavicius, parsons, mccormac, and jerram (2015) who used the repertory grid technique (rgt) interviews to obtain an in-depth understanding of student attitudes about the information security behaviours evaluated as part of the hais-q. content validity focuses on the extent to which the questions in an instrument really assess the construct of interest (burton & mazerolle, 2011; straub, boudreau, & gefen, 2004). also, more recently, parsons et al. (2017) report two further studies to establish construct validity of the instrument. construct validity is demonstrated when a measure correlates with other theoretically-related measures (westen & rosenthal, 2003). previous research has also demonstrated that the hais-q has high internal consistency. for example, parsons et al. (2015) used the hais-q to explore the relationship between information security and organisational security culture, and reported high cronbach’s alphas of above .80. most recently, mccormac et al. (2017) evaluated the extent to which individual differences (e.g., personality, age, gender) may be associated with hais-q scores, and also reported consistently high cronbach’s alpha scores. these evaluations and findings demonstrate the viability and internal consistency of the haisq as a useful measure, and have helped shape the current version of the hais-q. to date a total of 1,631 australians have completed the hais-q (parsons et al., 2017). this paper adds reliability evaluations and explores test-retest reliability and internal consistency. 1.3 social desirability bias social desirability bias is described as a form of response bias or cognitive bias, and it refers to the tendency of participants to respond to questions in a way that presents them in the best possible manner (fisher, 1993). when participants respond to survey questions in this way they are not providing truthful responses and this may lead to incorrect results that skew findings. therefore, surveys should incorporate questions to identify participants who are more likely to respond in a socially desirable way (grimm, 2010). essentially, when it comes to measures of isa, researchers need to be cognizant of individuals who may be assessed at higher levels of isa than they actually possess. such participants may be trying to present themselves as having greater knowledge; a more positive attitude; and report more secure behaviours than they actually exhibit in their workplace. experimental designs can incorporate several processes and techniques to minimise the effect of social desirability bias. for example, ensuring anonymity and confidentiality of respondents, has been shown to reduce bias (bowling, 2005). research has found that this assurance contributes to participants providing more truthful responses (szolnoki & hoffmann, 2013). australasian journal of information systems mccormac, calic, butavicius et al 2017, vol 21, selected papers from acis 2016 information security awareness and bias 4 allowing participants to complete a survey online also increases the likelihood of truthful answers because it is considered a neutral administration of the survey. participants are less likely to be affected by bias when providing responses online, as they may be less likely to feel judged (duffy, smith, terhanian, & bremer, 2005). researchers are also encouraged to include questions that can be used to determine if someone is providing socially desirable responses (nederhof, 1985). these are specifically constructed questions used for the sole purpose of separating credible responses from those who are providing socially biased answers. by incorporating these techniques into the experimental design it is expected that the impact of the social desirability bias could be reduced and also captured. for the purposes of our study, we designed a preliminary set of over-claiming questions for use within the hais-q. in this paper, we focus on an assessment of the reliability of the over-claiming items, as a measure of response bias. 2 method the present study involved the completion of two surveys using the same participant sample. this enabled a comparison of results obtained from the initial test (referred to as t1) and retest (referred to as t2). data collection involved an online survey, administered through the webbased survey platform, qualtrics. participants were required to meet the following inclusion criteria: they had to be currently employed and working in australia; be at least 18 years of age; spend at least 20% of their work time using a computer; and, work for an organisation with a formal or informal information security policy. also, upon completion of t2, participants were asked if they had completed any intervening information security training, if completing t1 had changed the way they use computers for work, and, if there were any other changes in their work or personal life that might have affected the way they use computers for work. these questions were asked as these aspects may have affected the participants’ survey responses. a total of 531 participants completed the hais-q, and, following a three to four week period, 207 of the participants in the initial sample completed the survey for a second time. ten outliers were identified from analysis. these participants had z scores more than two standard deviations from the mean. following recommendations made by meade and craig (2012), the data gathered from these ten participants were further examined to determine the quality of responses (e.g., whether they responded appropriately to questions or if there were signs of non-responsivity and careless responses, such as, only selecting the one response option). following this process, data from these 10 participants were excluded from analysis, leaving 197 participants who completed the online survey at both t1 and t2. all analyses reported in this paper focus on the 197 participants who completed the online survey at both t1 and t2. the 197 (105 females and 92 males) participants represented all age categories (12 between 18 and 29 years of age, 52 between 30 to 39, 49 between 40 to 49, 43 between 50 and 59, and 41 aged 60 and above), with most participants (94%) over the age of 30. level of completed education was also well represented among participants, with most participants having completed a bachelor degree (34%) or further post-graduate qualifications (20%). many participants had completed year 12 equivalent (14%) or had some post-secondary education (25%). participants represented over 13 employment sectors and eight job areas, including sales, labourers, professionals, management and technician/trade workers. 2.1 measures the online survey collected general demographic details and computer use information, including; gender, age, employment status, and the percentage of time at work spent using a computer. in addition to these questions, participants completed the human aspects of information security questionnaire (hais-q), a 63-item measure of isa (parsons, mccormac, butavicius, et al., 2014). the hais-q examines knowledge of information security policies and procedures, attitude towards policies and procedures, and self-reported information security behaviours. as mentioned previously, the hais-q focusses on seven areas of isa. respondents australasian journal of information systems mccormac, calic, butavicius et al 2017, vol 21, selected papers from acis 2016 information security awareness and bias 5 are asked to respond on a five-point likert-type scale, ranging from “strongly agree” to “strongly disagree”. three over-claiming items were developed, one a knowledge question, one an attitude question and one a behaviour related question. these questions were incorporated into the hais-q. the purpose of these questions was to identify individuals who were responding in a socially desirable manner. as with the hais-q, participants responded on a five-point likert scale. an example item was: “safe passwords must include the letter 'v'”. 3 results 3.1 internal consistency of the hais-q to measure the internal consistency of the hais-q, the cronbach’s alpha coefficients at t1 and t2 were compared. to assess the level of internal consistency, the cronbach’s alpha coefficient should be over .70 (devellis, 2011). table 1 presents cronbach’s alpha scores for knowledge, attitude, self-reported behaviour and overall isa at t1 and t2. it reveals minimal variation in estimated internal consistency between the two time intervals. t1 cronbach’s t2 cronbach’s knowledge .84 .86 attitude .93 .92 behaviour .90 .91 isa .96 .96 table 1. cronbach’s alpha scores for knowledge, attitude, behaviour and isa at t1 and t2 table 2, shows the cronbach’s alpha scores for the seven focus areas at t1 and t2. once again a similar pattern is observed, with little variation between t1 and t2 scores. these reported cronbach’s alpha coefficients reveal that the hais-q has high internal consistency as an overall measure of isa and also good internal consistency within its focus areas. focus area t1 cronbach’s t2 cronbach’s password management .83 .84 email use .77 .81 internet use .79 .80 social media use .75 .78 mobile devices .83 .82 information handling .76 .79 incident reporting .78 .78 table 2. cronbach’s alpha scores for focus areas at t1 and t2 3.2 test-retest reliability of the hais-q to evaluate the test-retest reliability of the hais-q, first we compared the knowledge, attitude and behaviour sub-scales, and the overall isa scores. table 3 shows the test (t1) and retest (t2) means, standard deviations and test-retest (t1/t2) correlations. the test-retest correlations for knowledge, attitude, self-reported behaviour and overall isa were all statistically significant, and were greater than .70 in all instances. it is generally accepted that a test-retest coefficient greater than .70 is required to illustrate external reliability (van saane et al., 2003). however, as shown in table 1, the scores for knowledge, attitude, self-reported behaviour and overall isa all increased from t1 to t2. to further assess this difference, raw scores were examined to identify the amount of variation between t1 and t2. for 93% of participants, there was less than 10% variation between t1 and t2. paired samples t-tests revealed that there were significant differences in t1 and t2 scores for knowledge, t(196) = -3.74, p = .000, d = .19, behaviour, t(196) = -2.73, p = .007, d = .11, and australasian journal of information systems mccormac, calic, butavicius et al 2017, vol 21, selected papers from acis 2016 information security awareness and bias 6 overall isa, t(196) = -3.44, p = .001, d = .12. however, as evident by the cohen’s d measures, these differences are all small sized effects (cohen, 1992a, 1992b). the findings for attitude were non-significant, t(196) = -9.95, p = .341, d = .04. this suggests that, overall, there was a high level of stability in hais-q scores. t1 mean(sd) t2 mean(sd) t1/t2 r correlation knowledge 80.64 (11.57) 82.84 (11.52) .75* attitude 86.54 (12.67) 87.09 (12.00) .79* behaviour 84.31 (12.31) 85.31 (11.66) .84* isa 251.50 (33.27) 255.55 (32.98) .88* table 3. correlations for knowledge, attitude, behaviour and isa at t1 and t2 (*p < .01, two-tailed) in table 4, we present the test-retest reliability of the seven focus areas, which provides evidence of the stability of the hais-q at the sub-scale level as well. the correlations between t1 and t2 were all significant, and, as shown in the table, the differences in means were very small. although four focus areas had statistically significant differences (i.e., email use, internet use, social media use and mobile devices), as the effect size was below .20, these differences were small (cohen, 1992a, 1992b). focus area t1 mean(sd) t2 mean(sd) t1/t2 r correlation password management 37.20 (5.82) 37.34 (5.76) .78* email use 34.50 (5.67) 35.56 (5.61) .73* internet use 33.86 (5.77) 34.50 (5.59) .72* social media use 36.08 (5.35) 36.76 (5.16) .74* mobile devices 36.88 (5.72) 37.72 (5.41) .77* information handling 36.84 (5.74) 37.10 (5.88) .82* incident reporting 36.13 (5.27) 36.56 (5.03) .75* table 4. correlations for focus areas at t1 and t2 (*p < .01, two-tailed) although these results provide sufficient evidence that the hais-q is a stable measure, we explored a number of other variables that may have affected responses. for example, only two participants indicated that they had received information security training in the intervening period, and a very small minority discussed any changes in their work or personal life that might have affected the way they use computers for work. when asked ‘did completing the initial survey change the way you use computers for work?’, approximately 40 participants stated that completing t1 affected their awareness of information security risks. a minority of respondents stated that they did not change their behaviour, for example, “have been following all the security rules for a long time”, and “i knew this already”. however, participants most commonly reported being more cautious in the use of computers. for example, “i thought more about it”, “i am more cautious”, and “it made me more aware of security risks both with information sources and my surroundings”. some reported being more cautious in relation to specific areas, such as password management, “i am more mindful of passwords being the same for personal and workrelated accounts”, and for the focus area of email use one participant said they were, “more careful with email links and attachments”. a small number of participants reported having taken more specific actions following t1, such as “[changing] their passwords” and “always [being] careful leaving things around”. these comments may account for the small increase in mean scores, between t1 and t2. 3.3 internal consistency and test-retest of the over-claiming items the same statistical analysis, conducted on the hais-q, was repeated using participants’ responses to the three over-claiming items to assess both internal consistency and test-retest reliability. to evaluate the internal consistency of the over-claiming questions the cronbach’s australasian journal of information systems mccormac, calic, butavicius et al 2017, vol 21, selected papers from acis 2016 information security awareness and bias 7 alpha coefficient scores at t1 and t2 were calculated. at t1 the cronbach’s alpha score was .53 and at t2 the score was .55 cronbach’s alpha scores were below the .70 mark used to determine internal consistency. table 5, summarises the test (t1) and retest (t2) means, standard deviations and test-retest (t1/t2) correlations for the individual knowledge, attitude and behaviour over-claiming items and the total over-claiming score. mean scores and standard deviations were stable from t1 and t2. all reported correlations were significant; however, they were below .70 for individual over-claiming items. given that there are only three items we correlated the total over-claiming score, between t1 and t2. this figure is .66, indicating that the scale is approaching the .70 cut off, which is indicative of test-retest reliability (van saane et al., 2003). t1 mean(sd) t2 mean(sd) t1/t2 r correlation knowledge over-claiming item 3.21 (1.16) 3.21 (1.16) .55* attitude over-claiming item 4.03 (1.00) 4.06 (.92) .51* behaviour over-claiming item 2.98 (1.29) 3.02 (1.27) .50* total over-claiming score 10.22 (2.49) 10.29 (2.45) .66* table 5. correlations, means and sd scores for the over –claiming individual knowledge, attitude, behaviour items and total over-claiming score at t1 and t2 (*p < .01, two-tailed) a paired samples t-test revealed that there were no significant differences in t1 and t2 scores for the total over claiming score which combined the three over-claiming items, t(196) = -.45, p = .65, d = -.03, suggesting stability. these results show that the preliminary over-claiming items are approaching test-retest reliability, and are stable; however, the items are not internally consistent. implications of these reliability findings are discussed in the following section. 4 discussion in this study, we examined the test-retest reliability and internal consistency of the hais-q, a measure of isa, and three preliminary over-claiming items, designed to measure response bias. current findings serve to further demonstrate that the hais-q is an accurate measure of isa, and, can be confidently used to assess interventions and training strategies. results show that the hais-q possesses both high internal and external reliability. there were small increases in scores between t1 and t2, which suggests that completing the hais-q may have prompted some participants to think more actively about information security, and this was demonstrated in participants’ qualitative responses. however, statistical analysis revealed that overall, these differences were not meaningful. test-retest coefficient values were over .70 across the overall measure and the three components that make up isa, namely, knowledge, attitude and self-reported behaviour. the seven focus areas of the hais-q followed the same pattern. similarly, the results of this study show the hais-q to be internally consistent, with cronbach’s alpha scores across all dimensions and focus areas above .70. this means that the hais-q is likely to be a reliable measurement tool. these findings have a number of practical implications. a reliable and valid tool that measures various aspects of isa is a valuable asset to any organisation. it provides an opportunity to reliably measure employee isa, and to potentially determine individual and organisational strengths and weaknesses. by administrating the hais-q to employees, an organisation can determine if, for example, password management is more of a weakness than social media use or mobile computing within their current work environment. australasian journal of information systems mccormac, calic, butavicius et al 2017, vol 21, selected papers from acis 2016 information security awareness and bias 8 the qualitative responses, although only a small component of the study, revealed that completing the hais-q may have affected employee awareness and made some individuals more cautious. in fact, some participants revealed that completing the hais-q, at t1, altered their behaviour in the intervening period. conversely, some participants reported no behavioural changes. these findings suggest that completing the hais-q, for certain individuals, may provide some training benefit. furthermore, both researchers and organisations can use the hais-q to measure the impact and effectiveness of interventions including training, isa programs, cultural changes and the impact of security incidents. for example, an organisation may initially administer the haisq to their employees in order to gather baseline data about their isa. using this information, they may identify certain areas of information security that require further targeted training campaigns. after this training is completed, the hais-q can be administered again to determine the success of the training intervention. if the training intervention was successful, improvements in scores across the knowledge, attitude and behaviour components of the hais-q, along with improvements in specific focus areas, should be evident. the three preliminary over-claiming items were designed to complement the hais-q, and their purpose was to identify individuals who were responding in a socially desirable manner. having the ability to identify these individuals gives organisations the opportunity to more rigorously understand the nature of isa in their environment. not only would they be able to assess if employees were being truthful in their responses but they would also be able to use this knowledge to investigate why this was occurring. for example, it may be the case that a high rate of socially desirable responses correlates with a lack of training opportunities. alternatively, in a work environment that is perceived as punitive, employees may be more likely to respond in a socially desirable manner to avoid the potential negative consequences associated with low information security behaviour that may potentially incur a penalty. although current findings indicate that the three preliminary over-claiming items were not internally consistent, they approached the test-retest reliability cut-off. these results are promising and it is suggested that the low alpha level could be due to the low number of questions used to assess bias which meant the measure lacked precision (tavakol & dennick, 2011). it is proposed that adding additional over-claiming items could improve both internal and test-retest reliability. an improved version would give organisations greater confidence in the over-claiming questions. assessing these responses in conjunction with hais-q findings may provide organisations with valuable information about their employees’ isa and the security culture in their workplace. specific suggestions as to how this can be achieved are provided in the following section. 4.1 limitations and future research in this study, we establish the test-retest reliability and internal consistency of the hais-q and measured the reliability of the over-claiming items. however, there are some limitations. for example, some authors recommend sample sizes between 200 to 400 participants for a testretest reliability study (charter, 2003; kline, 1986). this increases statistical precision, and improves generalisability of the findings. the sample size used in our reliability study (n = 197) is close to the minimum recommendation. while we do not envisage that a larger sample size would radically change the results, this study could be replicated with more participants. the qualitative responses provided insights that warrant further investigation into individual differences. it would be beneficial to explore why certain participants changed their behaviour after completing the hais-q and why others did not. it would undoubtedly help if we also knew more about participants, not only from an individual perspective, but also from an organisational one. participants in this study were all from unknown organisations. by completing the test-retest study using a known organisation, we would be better able to assess what happens between t1 and t2. any intervening training sessions or organisational changes could be controlled and accounted for. australasian journal of information systems mccormac, calic, butavicius et al 2017, vol 21, selected papers from acis 2016 information security awareness and bias 9 another limitation of this research is the generalisability of these results to other cultural settings and environments. the hais-q has been completed by a representative sample of working australians, and we have demonstrated that it is reliable within this australian context. however, further validity and reliability assessments of the hais-q should be conducted in different countries to measure the effects of any cultural differences. the overclaiming items are still in early stages of development but the effect of social desirability bias may certainly differ across cultural settings. once an improved reliable and validated measure has been developed it should also be further assessed in a variety of cultures and work settings. the findings relating to the over-claiming items were encouraging. however, its reliability may have been limited by the small number of questions, therefore, it would be prudent to increase the number of items in future research. it is recommended that rather than having a knowledge, attitude and behaviour over-claiming item there should be an over-claiming item for each of the focus areas. this would expand the number of items from three to seven. following the addition of five new items the reliability of the over-claiming questions would need to be re-assessed. this paper also only focused on the reliability of the hais-q and the preliminary over-claiming items, however, a measure needs to be both reliable and valid. validity testing and assessments have been conducted on the hais-q (parsons et al., 2017). however, further validation testing is recommended following the development of a more robust version of the over-claiming items. 5 conclusion this study has demonstrated that the hais-q, a measure of isa, is externally reliable and internally consistent. the reliability of the over-claiming items, used to measure social desirability bias, although encouraging require further development. in the current cyber environment, the ability to measure isa of employees, and having confidence in those results, is a valuable asset to organisations. the hais-q, as a reliable measure of isa, enables organisations to assess the effectiveness of any information security intervention strategies or other changes over time. by supplementing hais-q findings with reliable data on response bias, organisations will have greater insight into the state of isa in their organisation and a richer understanding of their organisational culture. therefore, research on the hais-q and over-claiming items provides a unique contribution to the information security literature and research field, as well a practical contribution to organisations. references allen, m. j., & yen, w. m. (2001). introduction to measurement theory. long grove, illinois: waveland press. arachchilage, n. a. g., & love, s. (2014). security awareness of computer users: a phishing threat avoidance perspective. computers in human behavior, 38, 304-312. bowling, a. (2005). mode of questionnaire administration can have serious effects on data quality. journal of public health, 27(3), 281-291. burch, g., & anderson, n. (2004). measuring person-team fit: development and validation of the team selection inventory. journal of managerial psychology, 19(4), 406-426. burton, l. j., & mazerolle, s. m. (2011). survey instrument validity part i: principles of survey instrument development and validation in athletic training education research. athletic training education journal, 6(1), 27-35. charter, r. a. (2003). study samples are too small to produce sufficiently precise reliability coefficients. journal of general psychology, 130(117-129). cohen, j. (1992a). a power primer. psychological bulletin, 112(1), 155. australasian journal of information systems mccormac, calic, butavicius et al 2017, vol 21, selected papers from acis 2016 information security awareness and bias 10 cohen, j. (1992b). statistical power analysis. current directions in psychological science, 1(3), 98-101. cronbach, l. j. (1951). coefficient alpha and the internal structure of tests. psychometrika, 16(3), 297-334. devellis, r. f. (2011). scale development: theory and applications (3rd ed.): sage publications. duffy, b., smith, k., terhanian, g., & bremer, j. (2005). comparing data from online and faceto-face surveys. international journal of market research, 47(6), 615. fisher, r. (1993). social desirability bias and the validity of indirect questioning. journal of consumer research, 20(2), 303-315. griffiths, a., cox, t., karanika, m., khan, s., & tomas, j. m. (2006). work design and management in the manufacturing sector: development and validation of the work organisation assessment questionnaire. occupational and environmental medicine, 63(10), 669-675. grimm, p. (2010). social desirability bias. wiley international encyclopedia of marketing. kline, p. (1986). a handbook of test construction: introduction to psychometric design. ny: methuen. : routledge. kruger, h., & kearney, w. (2006). a prototype for assessing information security awareness. computers & security, 25(4), 289-296. lewis, j., & baker, s. (2013). the economic impact of cybercrime and cyber espionage. retrieved from http://www.mcafee.com/au/resources/reports/rp-economic-impactcybercrime.pdf martins, a., & eloff, j. h. p. (2002). information security culture security in the information society (pp. 203-214). boston: ma: kluwer academic publishers. mccormac, a., zwaans, t., parsons, k., calic, d., butavicius, m., & pattinson, m. (2017). individual differences and information security awareness. computers in human behavior, 69, 151-156. meade, a. w., & craig, s. d. (2012). identifying careless responses in survey data. psychological methods, 17(3), 437-455. nederhof, a. j. (1985). methods of coping with social desirability bias: a review. european journal of social psychology, 15(3), 263-280. parsons, k., calic, d., pattinson, m., butavicius, m., mccormac, a., & zwaans, t. (2017). the human aspects of information security questionnaire (hais-q): two further validation studies. computers & security, 66, 40-51. parsons, k., mccormac, a., butavicius, m., pattinson, m., & jerram, c. (2014). determining employee awareness using the human aspects of information security questionnaire (hais-q). computers & security, 42, 165-176. parsons, k., mccormac, a., pattinson, m., butavicius, m., & jerram, c. (2013). an analysis of information security vulnerabilities at three australian government organisations proceedings of the european information security multi-conference (eismc 2013) (pp. 34-44). lisbon, portugal. parsons, k., mccormac, a., pattinson, m., butavicius, m., & jerram, c. (2014). a study of information security awareness in australian government organisations. information management & computer security, 22(4), 334-345. parsons, k., young, e., butavicius, m., mccormac, a., pattinson, m., & jerram, c. (2015). the influence of organisational information security culture on cybersecurity decision australasian journal of information systems mccormac, calic, butavicius et al 2017, vol 21, selected papers from acis 2016 information security awareness and bias 11 making. journal of cognitive engineering and decision making: special issue on cybersecurity decision making, 9(2), 117-129. pattinson, m., butavicius, m., parsons, k., mccormac, a., & jerram, c. (2015). examining attitudes toward information security behaviour using mixed methods. in s. furnell & n. clarke (eds.), proceedings of the ninth international symposium on human aspects of information security & assurance (haisa 2015) (pp. 57 70). mytilene, greece: centre for security, communications & network research. portney, l. g., & watkins, m. p. (2015). foundations of clinical research: applications to practice (3rd ed.). philadelphia: fa davis. pricewaterhouse coopers (pwc). (2015). turnaround and transformation in cybersecurity: key findings from the global state of information security survey 2016. retrieved from www.pwc.com/giss pricewaterhouse coopers (pwc). (2017). the global state of information security® survey 2018. retrieved from https://www.pwc.com/us/en/cybersecurity/informationsecurity-survey.html safa, n. s., von solms, r., & furnell, s. (2016). information security policy compliance model in organizations. computers & security, 56, 70-82. siponen, m. t. (2000). a conceptual foundation for organizational information security awareness. information management & computer security, 8(1), 31-41. siponen, m. t., pahnila, s., & mahmood, m. a. (2010). compliance with information security policies: an empirical investigation. computer, 43(2), 64-71. straub, d., boudreau, m.-c., & gefen, d. (2004). validation guidelines for is positivist research. the communications of the association for information systems, 13(1), 380427. streiner, d. l. (2003). starting at the beginning: an introduction to coefficient alpha and internal consistency. journal of personality assessment, 80(1), 99-103. szolnoki, g., & hoffmann, d. (2013). online, face-to-face and telephone surveys—comparing different sampling methods in wine consumer research. wine economics and policy, 2(2), 57-66. tavakol, m., & dennick, r. (2011). making sense of cronbach's alpha. international journal of medical education, 2, 53. tischer, m., durumeric, z., foster, s., duan, s., mori, a., bursztein, e., & bailey, m. (2016). users really do plug in usb drives they find. paper presented at the 37th ieee symposium on security and privacy, san jose, california. traynor, m., & wade, b. (1993). the development of a measure of job satisfaction for use in monitoring the morale of community nurses in four trusts. journal of advanced nursing, 18, 127-136. van saane, n., sluiter, j. k., verbeek, j. h. a. m., & frings-dresen, m. h. w. (2003). reliability and validity of instruments measuring job satisfaction—a systematic review. occupational medicine, 53(3), 191-200. westen, d., & rosenthal, r. (2003). quantifying construct validity: two simple measures. journal of personality and social psychology, 84(3), 608. acknowledgements this project was supported by a premier’s research and industry fund granted by the south australian government of state development. australasian journal of information systems mccormac, calic, butavicius et al 2017, vol 21, selected papers from acis 2016 information security awareness and bias 12 copyright: © 2017 mccormac, calic, butavicius, parsons, zwaans & pattinson. this is an open-access article distributed under the terms of the creative commons attributionnoncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ http://creativecommons.org/licenses/by-nc/3.0/au/ microsoft word ajis_18(3)_v10.docx australasian journal of information systems volume 18 number 3 2014 3 editorial by now i imagine many of you have noticed that the masthead announces this is issue 18(3) of ajis and are wondering what this is all about, as there has never been a third issue of any volume of ajis. this issue represents a major transition for ajis and heralds some major changes, which should significantly improve the service it delivers to the information systems community. i also wonder how many of you have wondered at the irony of us, as researchers in information systems, who have oftened trumpeted the fact that information technology produces disruptive rather than incremental change, have ourselves done little more than placing ajis on-line and adopting open access. we still release multiple articles in batches at six monthly intervals, we still have a print run for our subscribers, despite the fact that the number of subscriptions has fallen to a single digit number of institutions, and there are no individual subscribers. next year ajis will be released as a single volume per year. to make this a sane exercise, we have published this issue as 18(3), so that the change in volume numbers aligns with the change in years. those, however, are probably the most minor changes. the intent of these two changes is to allow progressive publication of articles. there are a huge number of arguments in favour of publishing articles as soon as they have been accepted and edited: maintaining the currency of the research; avoiding articles languishing six months in the “out basket” for no good reason; appeasing our managers who want fast throughput for assessment exercises, both of us and the schools they manage. from january 2015, general research articles and research notes will be published as soon as they are accepted and edited. themed sections will be published as soon as all their papers have been accepted and edited. this synchronising will be the last vestige of batch publishing in ajis. this issue contains a varied range of articles, which is as it should be, given the diverse nature of oiur academic community. it also contains selected papers from acis 2013. these papers represent much of the best work which was presented, and i look forward to acis 2014 next month and publishing selected papers from that conference. i have been approached regarding future themed sections by a number of people, and interest in managing themed sections is growing. future themes include: • selected papers from the foundations of information systems conference • small and medium sized enterprises • business analytics • indigenous information systems • security issues in social media/networking • intelligent decision making in information technology • the economic and societal impacts of broadband adoption in rural and regional communities • design research for decision support systems development again, my sincere thanks to my head of school, dineli mather, for her support of my role as editorin-chief, and to aais, acphis and phis-nz for the help and support that they have provided for this issue, and the acs for providing hosting services for the journal. dr john lamp ajis editor-in-chief deakin university november 2014 paper title (paper title style) australasian journal of information systems bakshi, talaei-khoei, ray & solvoll 2017, vol 21, research article podman: policy deviation management 1 podman: policy deviation management aishwarya bakshi university of new south wales amir talaei-khoei university of nevada, reno university of technology sydney atalaeikhoei@unr.edu pradeep ray university of michigan–shanghai jiao tong university joint institute, shanghai jiao tong university, china terje solvoll norwegian centre for integrated care and telemedicine university hospital of north norway abstract whenever an unexpected or exceptional situation occurs, complying with the existing policies may not be possible. the main objective of this work is to assist individuals and organizations to decide in the process of deviating from policies and performing a non-complying action. the paper proposes utilizing software agents as supportive tools to provide the best non-complying action while deviating from policies. the article also introduces a process in which the decision on the choice of non-complying action can be made. the work is motivated by a real scenario observed in a hospital in norway and demonstrated through the same settings. keywords: agent; policy; deviation 1 introduction while policy based systems have largely been successful in controlling and managing cooperative work environments, one of its weaknesses lies in the fact that there is always the risk of exceptions, where following the policy rule might result in non-optimal outcomes and at worst lead to completely devastating consequences. many studies have proposed the use of adaptive policies to counter the changing environmental circumstances, thereby reducing the risk of exception. as it is humanly not possible to envision all possible exceptions, there is a need to introduce procedures whereby the user is given the decision making support and flexibility to deviate from a policy when absolutely needed. while many decision support systems have been suggested for cscw systems (chau 2003, benbasat 1990,shum 2011), this paper aims to introduce a method which allows greater flexibility to cscw systems by assisting in the decision making process of deviating from policies, making sure that the deviations take place in a controlled manner when such exceptions arise. in order to address the problem, the study proposes a methodology called podman (policy deviation management) which uses software agents to detect policy violations and if required, assist users in the decision making process of deviating from the policy in a controlled manner. the rest of this paper is organized in the following way: section 2 illustrates the motivation behind this work by providing a real scenario observed in the wireless communication system in a hospital in norway. section 3 presents background and defines different concepts being used in this research. section 4 proposes the podman methodology that consists of a framework of definitions and semantics as well as a stepwise process for policy deviation deployed in the framework. section 5 demonstrates podman through the wireless communication scenario introduced in section 2. section 6 presents a simulation for evaluating podman. section 7 concludes the study and discusses the implications and limitations of the work. australasian journal of information systems bakshi, talaei-khoei, ray & solvoll 2017, vol 21, research article podman: policy deviation management 2 2 illustrating the motivation: wireless communication system at a hospital in this section, we illustrate the motivation behind this study using an example that authors have discussed (talaei-khoei 2011) where they were proposing a policy based communication system at st. olavs hospital in trondheim, norway. st. olavs hospital has implemented a single, converged ip network for all communications among the hospital staff. all medical personnel in the hospital communicate via ip phones and ip enabled devices where they have to log in at the start of the shift and can be reached by other personnel by a press of a button on their device. all communication amongst the personnel is routed via a communications center which continuously monitors the presence of users in hospital and their activity statuses such as busy, available, etc. the physicians at the hospital often complained that they were frequently distracted by calls from nurses while they were in the middle of a medical procedure which required them to stop what they were doing, remove their gloves, answer the call, wash their hands again and put on a new set of gloves. this reduced the overall efficiency with which they work. in order to prevent such distractions, the hospital laid down policy rules which forbid the nurses from calling physicians if they were unavailable or busy and instead recommended the nurses to get in contact with an alternate physician. while this works well under normal circumstances, it does not consider the case where the presence of a particular physician is critical to a patient’s health e.g. consider the case where only one specialist physician is present at the hospital but he is busy performing a non-critical procedure on a patient. at the same time another patient suddenly requires critical attention from the specialist. when the nurse tries to contact the specialist physician for advice, the system forbids the nurse from doing so as the doctor is busy with another procedure although it might not critical. in such a case, the provision to deviate from the policy and calling the doctor might have saved the life of the patient. hence, through this illustration, we have identified the following issues that have motivated us for this study:  users of collaborative policy based systems are often unaware of existing policies or changes made to existing policies  policies can often be too rigid and might lead to undesirable outcomes under exceptional circumstances  there is a need for controlled management of non-compliant actions when exceptional situations arise. however, the aforementioned scenario was relatively simple. the objective, here, was to illustrate the problem that motivates us for podman. the research in software agent has been confirmed, in different cases, that use of agents in order to address the sporadic contacts in distributed communications is beneficial (ulieru and worthington, 2006, ray et al., 2005a and ray et al., 2005b). therefore, here, we employ software agents to assist individuals managing the sporadic communication. we also propose a methodology to develop agent systems that assist individuals to deviate from policies. 3 background in this section, we provide the background to the study presented in this article. we set definitions and foundation on which the work has been grounded. 3.1 policy adaptation and policy deviation not surprisingly, in cooperative environments, complying with policies is required. however, these policies are built keeping the best possible future in mind. problems often arise when these policies are too rigid and designed without considering all possible situations, including australasian journal of information systems bakshi, talaei-khoei, ray & solvoll 2017, vol 21, research article podman: policy deviation management 3 exceptional situations and future uncertainties, called unexpected situations. when a cooperative role runs into unexpected or exceptional situation, any behavior that the role may perform may violate the policy constraints, and the behavior may have to be aborted. however, abortion may not be the best strategy (lewis 2010). another option is policy adaptation that provides two alternatives; (a) modifying the policy (dearle 2010) and (b) dealing with the consequences of performing a non-compliant action (migon 2010). these two alternatives result in two different definitions of policy adaptation.  adaptation by changing values of policy parameters depending on performance feedback (lotlikar 2006).  adaptation by relaxation (lewis 2010) which means monitoring consequences of nonconformance behavior against the desired behavior proposed by the policy to minimum incompliancy; that is policy deviation. verhagen (2001) describes policies from two perspectives, namely – policies from the social theory perspective and policies from legal theory perspective. social theory (verhagen, 2001) states that a cooperative role complies with policies because it is rational to do so and because that is what others expect from the role. therefore, policies in social theory can be described as a generalized expectation of behaviors. policies from this point of view are a set of social rules that are designed by the community of cooperative roles and can be changed on the go. therefore the first definition of policy adaptation can be applied to social policies. on the other hand, legal theory looks at policy as a sense of duty (verhagen, 2001) and that a cooperative role complies with a policy because of the authority issuing the policy. policy in this perspective is a set of constraints that limits behaviors. since these policies are designed by an authority, they cannot be modified on the go. therefore, instead of modifying the policies, when exceptional or unexpected situations occur, one has to decide what action must be performed to have minimum incompliancy from the specified legal policy. therefore, the definition of policy deviation can be applied. the comparison between two definitions of policy adaptation is presented in table 1. definition configurations outcome type of policy adaptation by changing values of policy parameters performance feedback modified policy social policies policy deviation: adaptation by relaxation comparison of consequences of non-conformance behavior against the desired behavior minimum incompliancy legal policies table 1: policy adaptation 3.2 policy adaptation framework: a systematic review on it-centric policy adaptation in run time independent on what type of policy adaptation is concerned, swanson et.al (swanson 2009) describe the adaptive policy cycle in three stages; (a) policy set-up, (b) adjustment, and (c) monitoring that involves continuously monitoring and evaluating the performance of the policy for further setup and adjustment. the main objective of the policy set-up phase is to determine the policy goals, constraints and performance factors in order to understand the policy vulnerabilities and in turn develop triggers, signposts, mitigating actions and contingency plans. using these outcomes from the policy setup phase, policy makers depending on the policy being social or legal choose to adjust the policy based on the findings of the previous step or adjust the behavior. once the policies are put in practice, their performance need to be continuously monitored and reviews must be carried out to learn from the outcomes of these policies to further improve them. depending on the outcomes of these reviews, another cycle of policy adaptation starting with policy setup will be conducted. australasian journal of information systems bakshi, talaei-khoei, ray & solvoll 2017, vol 21, research article podman: policy deviation management 4 having taken the swanson’s model into account, authors aimed to autotomize the adaptation cycle. to that end, they conducted a systematic review papers proposing it-centric cscw methods that can be used in each of the swanson’s model. the total number of paper reviewed was 1418. the result of the review was a framework for policy adaptation which provides detailed steps that needs to be taken for automated policy adaptation in run-time. for each step the methods that can be used are introduced. for more detailed information, please refer to (bakshi et.al 2013). it was found, in the systematic review, that it-centric cscw methods proposed to address the adjustment step in swanson’s policy adaptation model, have been heavily focused on lotikar’s (lotlikar 2001) perspective i.e. adaptation by changing values of policy parameters. while the literature has largely ignored that the lotikar’s perspective can only be applied in social policies, adaptation for legal policies requires further investigation. although lewis et al (2010) have presented an idea to solve this problem for policy deviations from a legal policies; the work lacks a sufficient level of details and implementation. having said that, this paper focuses on a definitive method on how it-centric cscw systems assist individuals to deviate from legal policies in an organization and perform nonconformance behavior while an unexpected or exceptional situation concurs. 4 policy deviation management (podman) in this section, we propose policy deviation management (podman), which aims to assist policy based collaborative systems in detecting potential policy violations in future and in return present alternative actions to the users to achieve the desired outcome whenever possible. in order to so, we shall first define a framework, including semantics that set a foundation of the different definitions under which podman is grounded. 4.1 podman framework 4.1.1 use of software agents to detect and assist for policy deviations the use of software agents has proved useful in many cscw application areas such as emergency management (yuan and detlor, 2005), risk management (ulieru and worthington, 2005), mobile health monitoring (ray et al., 2008), disaster management (ray and chattopadhyay, 2009), and wireless communication in hospitals (talaei-khoei 2011) .cscw tends to agree that the following characteristics of software agents make them useful in assisting individuals (woolridge and wooldridge, 2001): autonomy, an agent can operate without the intervention of individuals; social ability, an agent is able to interact with other agents and individuals; reactivity, computing power of agents makes it possible for them to react on the changes of the environment in a timely fashion (however, this is heavily dependent on the computing power); pro-activity, an agent tries to achieve the purpose for which it was initiated, which can be highly useful to assist individuals employing agents for particular purpose. therefore, podman has chosen software agents to present individual roles. the software agent acts as the mediator between an individual or organizational user and the system in which podman has been implemented. the agent communicates all requests on behalf of the user and delivers the responses from the application back to the user. it is also the responsibility of the agent to determine whether the requested user actions are compliant with the policies which govern the system. our system is composed of multiple such agents, one for every role. if required, these agents can communicate with other agents in the collaborative system in order to gather relevant information. 4.1.2 from decision trees to branching time models (btm) research in software agents has been interested in the natural semantics for the possible scenarios in a given situation as a mental attitude of agents describing internal features that must be interpreted subjectively from the agents’ point of view. for more about mental attitudes of agents, see (rao and georgeff, 1991; boella and van der torre, 2003). the classical approach is possible-world (or in short, world) in which a situation can be considered possible australasian journal of information systems bakshi, talaei-khoei, ray & solvoll 2017, vol 21, research article podman: policy deviation management 5 in addition to true or false (rao and georgeff, 1991). this can be modeled using a tree-like structure in which each path of the tree represents one possible scenario and each node depicts one possible world. this single past and multiple futures structure is called branching time model (btm), used in podman; see figure 3. as we discussed above, in the branching-time model of worlds, there may be more than one choice available to execute i.e. multiple future. here, we begin with the classical decision tree and show how we can generate a btm utilized for our problem. informally, a decision tree consists of decisions and chances. decisions represent points where the agent has to choose one alternative from a number of choices available. chances represent points where probability plays a dominant role and reflects alternatives over which the agent has no control. decision trees also include probability functions that map chances to real-value probabilities and a payoff function that maps decisions to a real number. we transform decision trees to an equivalent model (rao and georgeff, 1995) that represent decisions with optional actions and chances with events. this transformation provides an alternative basis for cases in we are not able to form pay-off or probability functions. we begin by considering a decision tree, in which every possible path (scenario), including those with zero pay-off, is represented. we start from the root node of the tree and traverse each arc. for each unique arc emanating from a chance node, i create a new decision tree that is identical to a tuple made of (1) the chance node, called event and (2) the decision node located before the chance node. the difference between the generated tree and the original decision tree is that (i) the chance node is removed and (ii) the arc coming to the chance node is connected to the arc emanating from the chance node. this process is carried out recursively until there is no chance node left. this will result in a set of trees with no chance node, while each corresponds to a different possible state of the environment; that is, from the traditional possible worlds perspective. indeed, each of these trees represents a possible world triggered by an event; see figure 3. any changes to the environment or the system are known as events. these events can trigger the agents to choose an action, modeled by arcs in btms (see figure 3) and transform from one situation to another by performing a specified action, although it is not mandatory that they do so. while one agent can react to a certain event in the environment, another agent can be completely apathetic towards it. therefore the agents treat each event differently depending on the cognitive state they are in. in podman, this cognition is presented by a set of btms programmed into the agent. if the agent has a btm trigged by an event, then the agent would react agents the event according to the btm; see figure 3. figure 1: an example of branching time model for an agent. figure 2: podman policy structure as an example, at t1 the possible paths that the agent can follow are: (a) 1-2-3, (b) 1-2-4-5, (c) 1-2-4-6, (d) 1-7-8-9 and (e) 1-7-8-10. in this example, when an event occurs at t1, the agent can transform situation 1 to either situation 2 or 7 by performing the specified action. on completion of the action, the agent marks the previous situation as ‘done’ and the new situation is marked as current. this process continues till the agent reaches the final situation in the path, signifying the end of a possible scenario. australasian journal of information systems bakshi, talaei-khoei, ray & solvoll 2017, vol 21, research article podman: policy deviation management 6 4.1.3 policy structure the policy structure for podman is borrowed from directory enabled networks – next generation (den-ng) (strassner et al., 2009), which provides a meta-model for the implementing business policies (strassner et al., 2008). according to the den-ng model, policies are defined as a composition of policy rules where each policy rule is a proposition with event-condition-action (eca) semantics. these rules are triggered by the events described earlier. based on the event and condition specified, an action may or may not be allowed. along with eca, podman also requires the inclusion of another attribute to the policy specifications; that is called legal terms and conditions. the legal conditions serve as guidelines for users in the decision making process of policy deviation. an example of such a guideline would be, “the following information is strictly confidential and sharing of this information with any unauthorized party may result in prosecution”. figure 4 illustrates the basic policy structure used in podman. in following, we may say policy in short while we refer to policy rule. sloman (1994) defines modality of actions by introducing four types of policy rules: permitting, forbidding, requiring and deterring. table 2 shows examples for each type of policy rules. type example code presentation of an action govern by these rules in btm permitting when an employee leaves a company, if he or she has been working in the customer service unit, it is permitted to remove his or her file from the archive. f f forbidding when an employee leaves a company, if he or she has ever worked in the accounting unit, it is forbidden to remove his or her file from the archive. d d requiring when an employee leaves a company, if he or she has ever worked in the accounting unit, it is required that his or her file be kept in the archive. p p deterring when an employee leaves a company, if he or she has been working in customer service unit, it is deterred to keep his or her file in the archive. r r table 2: examples of policy modality types twidle et al. (2009) and moffett & sloman (1991) state that amongst these, only forbidding and requiring modalities are usually in force. while both deterring and permitting policies recommend certain behaviors to the user, they do not enforce a user to abide by the rules i.e. the users can choose whether they want to follow the policy or not. we call these types of policies weak polices. on the other hand, forbidding and requiring policies do not offer any options to the users and they are forced to follow the rules irrespective of the situation. we call these types of policies strong policies. while strong policies are useful for operational control under known and expected conditions, problems due to the inflexibility of such policies might arise when unexpected or exceptional situations come to the fore and a policy prevents the user from performing a critical action. in such cases, some of the policies might need to be demoted to a weak modality, e.g. forbidden to deterring or requiring to permitting to temporarily allow the user to choose a nonaustralasian journal of information systems bakshi, talaei-khoei, ray & solvoll 2017, vol 21, research article podman: policy deviation management 7 compliant action under the circumstances. of course there are some policies which need to remain strong irrespective of the situation e.g. for security and auditing reasons, under no circumstances can a user perform an action without signing into the system. for these types of policies, the decision to deviate from these policies does not lie with the user but with a higher authority. 4.1.4 btm governed by policy rules actions in the btm are governed by policy rules and even though the same action might appear at different situations in the btm, the modalities can be different or stay the same depending on the situation the action emerges from; see table 2. in the example of wireless communication in the hospital, the action of a calling a doctor by a nurse might be prohibited if the doctor is in the operating theatre, but the same action of calling the doctor might be allowed if the doctor is performing clinic duties. the path which does not contain any action governed by a strong policy modality is called free path. other than free paths, whenever an agent comes across a strong policy, it utilizes the presence of multiple similar actions across the btm to determine whether the strong policies are conditional or absolute. this is done by checking if at least one of the instances of the action in the btm is governed by a weak policy, i.e. a permitting or deterring policy, and if such an action exists at any point in the btm, it is considered to be a conditional-strong policy. if no similar actions are found or none of the other similar actions are governed by weak policies, the action is treated as an absolute-strong policy by the agent. therefore, in our illustrative example, since the nurse is permitted to call the doctor in at least one situation, i.e. when the doctor is performing clinic duties, the forbidding action of calling the doctor is considered to be a conditional-strong policy by the agent. this implies that if the nurse considers the circumstances critical enough, and understands the implications of the decision, she is allowed to deviate from the policy and call the doctor even if he is in the operating theatre. it must be ensured that all the actions leading to the alternate action must be followed. figure 3: an example of branching time model governed by policy rules figure 4: an example to describe the podman process: btm governed by policy rules in this btm presented in figure 3, we noted that there are 5 complete paths to choose from at t1 [1-2-3], [1-2-4-5], [1-2-4-6], [1-7-8-9] and [1-7-8-10]. but after the policy rules are placed, we see that there are paths which contain actions forbidding the agent from completing the path and reaching the final situation: [2-4] and [8-9]. we also note that there is only one path which does not contain any action governed by a strong policy modality [1-2-3]. we can also observe that in this btm there are three situations from which policy deviations might be required situations 1, 2 and 8. this is because at least one of the actions emerging from these situations is governed by a strong policy. australasian journal of information systems bakshi, talaei-khoei, ray & solvoll 2017, vol 21, research article podman: policy deviation management 8 4.1.5 conditions in which policy deviation is required. policy deviation comes into the picture, only when a user wants to perform a forbidding action or not perform a requiring action, in other words only when a strong policy is encountered. when a user requests the agent to deviate from a strong policy, the agent calculates if the policy is an absolute-strong policy or a conditional-strong policy. it does so by trying to locate identical actions in the btm that are governed by a weak policy modality. if it finds at least one such action, it considers the policy to be of the conditional-strong type and allows the user to deviate from the policy once the legal terms and conditions have been signed. the agent also makes sure that the path and all the actions leading up to the alternate action are also performed by the user before proceeding. if more than one such action is found, the agent ranks these alternate paths for the user to choose from. a detailed explanation of calculating alternative paths is discussed in section 4.2.1. if no satisfactory alternative path can be found, the agent considers the policy to be an absolute strong policy and podman puts forward exception management procedures to temporarily permit the agent to perform the forbidden action. both techniques will be discussed in detail in section 4.2. 4.2 podman process: steps towards policy deviation having introduced the podman framework, we shall discuss the process in which policy deviations can be managed. we illustrate the process in a hypothetical example, in which we can describe and clarify the podman steps towards policy deviation; see figure 6. let us assume that in the btm shown in figure 6, action a1, a2 and a3 are identical and actions b1 and b2 are also identical. table 3 maps the situations and modality strength of actions. using our description of conditional and absolute strong policies, we can see that action a belongs to the conditional strong policy group as it has at least one instance where it is governed by weak policies (a2 and a3). on the other hand, both the instances of action b (b1 and b2) are governed by strong policies, and therefore belong to the absolute-strong policy group. action from situation to situation modality strength modality type modality subtype a a1 a2 a3 3 8 16 4 9 17 strong weak weak requiring deterring permitting conditional b b1 b2 2 20 14 21 strong strong forbidding forbidding absolute c c1 24 25 strong forbidding absolute table 3: an example to describe the podman process: situation mapping in the beginning of the scenario depicted in figure 6, at t1, the user agent performs a depthfirst traversal of the entire btm, allowing it to record every possible complete path ahead; in this case 12 of them. figure 7 illustrates the 7 paths that contain only the above mentioned actions. australasian journal of information systems bakshi, talaei-khoei, ray & solvoll 2017, vol 21, research article podman: policy deviation management 9 figure 7: an example to describe the podman process: possible paths from t1 contacting instances of action a, b, or c let us consider the following possible scenarios from figure 7: 1. the user follows a path containing no strong policies 2. the user follows a path containing conditional-strong policies 3. the user follows a path containing absolute-strong policies these present different possible settings that may happen. in following, we explain the podman process in each setting. i. free path: a path containing no strong policies: from figure 7, we can see that path 2 is a free path containing no strong policies and therefore can be chosen for this scenario:  at t1 the agent knows that amongst all the paths ahead, there is at least one free path i.e. a path that does not encounter any strong policies [path 2]. when the user requests the agent to transform situation 1 to 2, the agent proceeds to do so regardless of whether the user actually follows the policy of the action or not as the policy modality for the action is weak. it also marks situation 1 as ‘done’.  at t2, the agent recalculates the possible future paths from situation 2 and again finds that there is at least one path with no forbidden policies and thereby allows the user to transforms from situation 2 to 3.the process is repeated till the last action is performed, transforming situation 8 to 9 and the task is completed. this example illustrates a scenario where no deviations are required as all the policies encountered are weak and the user can freely choose whether or not they want to comply with the policy rules in order to complete the task i.e. no deviation requests are required. ii. path contains a conditional strong policy: a conditional strong policy is one which can be deviated from if required at the discretion of the users, as long as they accept the legal terms and conditions of doing so. in this scenario, we consider a path containing only conditional strong policies, which in our case is path 1.  in this path, the agent does not warn the user of any possible policy violation till t3 as it calculates the existence of at least one free path till that point, i.e. path 2.  at t3 if the user wants to deviate from the action that transforms situation 3 to 4, the agent detects a strong policy violation and warns the user. if the user still wants to persist with the decision, the agent first calculates the existence of alternate paths if available. from figure 4-5, we observe that paths 2, 4 and 5 contain identical actions (a2 and a3) which are governed by weak policies and therefore determines the strong policy governing a1 to be a conditional strong policy. a conditional strong policy signifies that under exceptional circumstances, the agent can deviate from the policy as long as an identical action with weak policy modality exists in the btm and the user understands and accepts the legal implications of performing the non-compliant action. the agent then builds alternative paths by replacing a1 with all the actions from the path containing a2 or a3, starting with australasian journal of information systems bakshi, talaei-khoei, ray & solvoll 2017, vol 21, research article podman: policy deviation management 10 the last common situation and ending at the situation where a2 or a3 is performed. it must however be noted that the agent ignores an alternate path if it contains even one forbidden policy. figure 8.a and figure 8.b illustrate the alternative paths calculated by the agent. a. choosing alternative actions from path 2 b. choosing alternative actions from path 5 figure 8: an example to describe the podman process: alternative paths to deviate from conditional strong policies once the alternative paths are calculated, the agent ranks the choices for the user to choose from. the process of ranking alternate paths is discussed in section 4.2.2. once the user chooses the preferred alternative, the agent proceeds to fetch the legal conditions associated with the policy and presents it to the user to sign and accept. the reasons for violation are logged for audit after which the agent is permitted to perform the requested action. figure 9 shows the podman methodology of handing conditional-strong policy deviations. figure 9: an example to describe the podman process: sequence what need to be done by the agent in conditional-strong policy deviation iii. path containing absolute-strong policies: finally, in this scenario, we consider a path containing absolute-strong policies. this means that even under exceptional circumstances, the user does not have the sole decision making power of deviating from the policy without permission from an authority. in our system, we consider an authority to be any role that is permitted to perform the required action, i.e. a role which has, in its btm, the permissions to perform an identical action. the reasoning behind this is that a role who is allowed to perform the action under normal circumstances has the ability to decide whether the action is required or not under other circumstances. path 3, 6 and 7 all contain at least one action with an absolute-strong policy and thus match this scenario. we will analyze path 3 and path 7 for this illustration.  if the user decides to follow path 3, as described in the previous scenarios, the agent allows the user’s requests and transforms up till situation 2 without any warning as it detects at least one free path till that point.  at t2 if the user chooses to follow path 3 and requests to perform the forbidden action b1 connecting situation 2 to 3, the agent detects a violation and immediately warns the user. the agent then performs a check to verify if the forbidden policies are absolute or australasian journal of information systems bakshi, talaei-khoei, ray & solvoll 2017, vol 21, research article podman: policy deviation management 11 conditional. since the other instance of action, b2, is also governed by a forbidden policy, the agent realizes that the policy is an absolute-strong policy.  if the user wants to persist with the action, the agent broadcasts a message requesting any agent who’s btm contains an identical action that is governed by a weak policy for permission to deviate. other agents in the system on receiving this message check to see if their policies permit them to perform the requested action and if it does, notifies its users of the request. any user belonging to the role with necessary privileges can respond to the request. this process is illustrated in figure 11. once the request is accepted by a user with the higher privileges and the legal conditions are co-signed, another message is broadcast to notify the agents that the last request can be dropped. only once both roles co-sign the terms and conditions is the agent allowed to deviate from the policy.  if the user decides to follow path 7, the moment the agent receives a request to transform situation 1 to 24, it detects that there are no free paths ahead and the path ahead contains a forbidding policy. thus it premeditates the policy violation and warns the user ahead of time. if the user still chooses to go ahead, the process described earlier is followed. figure 11 illustrates the complete process for absolute-strong policy deviation. figure 11: an example to describe the podman process: what need to be done by the agent in absolute-strong forbidden policy deviation 4.2.1 calculating the alternative paths as discussed in the previous section, whenever an agent receives a request to deviate from a conditional strong policy, its first response should be to notify the user of the imminent violation followed by calculating and presenting the user with alternate options to continue its path till the goal. it is illustrated below how the agent calculates the alternate routes depending on the location of the alternate actions. australasian journal of information systems bakshi, talaei-khoei, ray & solvoll 2017, vol 21, research article podman: policy deviation management 12 consider the paths in figure 12 to be the complete paths that contain available alternatives for the agent to choose from while deviating from a policy rule and the sub-paths marked in grey to be the actual steps that the agent must replace the original action with during the deviation process. these paths are used as examples to highlight how the agent ranks the alternative actions to assist the user in choosing a path. figure 12: an example to describe the podman process: alternate paths available to the agent (shaded grey) 4.2.2 ranking the alternative paths once the alternative paths are calculated by the agent, ranking these paths can assist the user in choosing the most convenient option. in our system, the agent ranks the alternative paths in two separate ways – by path length and path flexibility; see table 4. path total number of situations total forbidden policies total requiring policies total deterring policies total permitting policies a 3 0 1 0 1 b 3 0 0 1 2 c 5 0 1 0 3 d 5 1 0 0 3 e 5 0 0 0 4 table 4: an example to describe the podman process: total situations and policy distribution for each alternate path i. ranking by path length: ranking by path length involves sorting the paths based on the number of situations encountered in each of the paths. this provides the quickest path to reach the required objective but is not necessarily the most flexible. if two paths are of the same length, they are ranked by their path flexibility. while it can be argued that since weak policies are optional and can be ignored from the total count, it must be noted that policies are in place to improve organizational processes. policy rules should be ignored only if deemed absolutely necessary. hence it is included in the total count. from table 4, we can therefore rank the paths as follows [shorter] path b < path a < path e < path c [longer] ii. ranking by path flexibility: ranking by path flexibility involves calculating the number of policies of each type encountered by each alternate path. path flexibility can be said to be the amount of power the user or agent has to perform the task without being constrained by the policies. the path flexibility is ranked in the order of  the least number requiring policies in the path  the least number of deterring policies in the path  the most number of permitted policies in the path australasian journal of information systems bakshi, talaei-khoei, ray & solvoll 2017, vol 21, research article podman: policy deviation management 13 paths ranked by flexibility provide the easiest route to the required objective but are not necessarily the shortest. if two paths are equally easy, they are sorted by path length. based on table 4, we can therefore rank the ease of path as follows. [flexible] path e < path b < path a < path c [rigid] 5 illustrating podman: wireless communication system at a hospital keeping the scenario explained in section 2 in mind, let us consider a case where due to some unforeseen circumstances, only one neurologist is present at the hospital. as she prepares for a surgery, all of a sudden one of the other patients starts showing major neurological symptoms. at this point the nurse tries to call the attending neurologist to ask for advice but the policy forbids the call from going through, which leads to the patient suffering major nerve damage due to the lack of appropriate medical attention. a step-by-step illustration is presented of how this situation could be handled by podman. the branching time model of the nurse agent is shown in figure 13. figure 5: illustrating podman in wireless communication systems at a hospital: nurse agent’s btm figure 6: illustrating podman in wireless communication systems at a hospital: forbidden actions being replaced by alternative path to begin, we have created transformation table for this scenario; see table 5. transformation action modality s1-s2 nurse wants to call neurologist [doctor] permitted s2-s3 verify availability of neurologist required s3-s4 neurologist is available s4-s5 call doctor permitted s3-s6 neurologist not available s6-s7 call doctor forbidden s6-s8 call alternate doctor permitted table 5: illustrating podman in wireless communication systems at a hospital: transformation table initially, when the nurse decides to call the neurologist, the agent transforms s1 to s2. this is followed by a requiring policy to find out if the neurologist is available or not. on completing that action, the agent transforms s2 to s3. at s3, the agent is informed that the neurologist is unavailable and transforms to s6, from which the nurse is forbidden to call the doctor (s6-s7). at this point, the agent warns the nurse that calling the neurologist is against the policy rules and the nurse should call an alternate doctor. considering the situation to be an emergency where policy deviation is required, the nurse requests the agent to persist with the action of calling the neurologist even though she is unavailable. on receiving the request for deviation, the agent first tries to determine whether the policy is a conditional strong policy or an absolute strong policy. it does this by backtracking one situation at a time and checking each emerging branch from that situation for a path containing the same action but governed by a weak policy. australasian journal of information systems bakshi, talaei-khoei, ray & solvoll 2017, vol 21, research article podman: policy deviation management 14 thus from s6, the agent backtracks to s3 and observes that the path s3-s4-s5 contains the action [call doctor] but governed by a permitting policy (weak). this implies that the forbidding policy at s6 is a conditional strong policy and therefore the decision to deviate from the policy lies with the nurse. the agent then proceeds to fetch the legal conditions of deviating from the policy and once the conditions are accepted by the nurse, replaces the action (s6-s7) with actions (s3-s4, s4-s5), thereby permitting the nurse to contact the doctor. this is illustrated in figure 14. furthermore, the request for deviation is logged by the agent. this log can be used by the management in future to modify the agent’s branching time model in a way that the exception is handled. in case of absolute strong policy deviations, the request would be broadcast by the agent to all other agents with the necessary deviation rights. simulation in this section, we explain a simulation that was aimed to evaluate the effectiveness of the podman. it is important to first establish definition of effectiveness. in the context of policy deviation, we have defined effectiveness as the likelihood of policy deviation being allowed under exceptional or unexpected circumstances. in table 2, the various modalities of policy rules were introduced. it was determined that strong policies could be further classified into conditional-strong policies and absolute-strong policies. policy deviations from conditional strong policies can be performed by a user based on his or her own judgment and therefore the likelihood of the policy deviation being allowed lies completely with the user i.e. if the user chooses to deviate from a conditional policy, it will be permitted by the system each time. similarly, without podman, the deviation will not be permitted by the system each time. thus for conditional policies, the effectiveness of podman is accordingly confirmed. policy deviations from absolute strong policies require the permissions from other users who have the necessary authorizations to perform the action under normal circumstances. therefore the probability of being allowed to deviate from the policy depends on other users in the system. thus a simulation is performed to determine the probability of a deviation from absolute-strong policies being permitted in a cooperative environment under different parameters. 5.1 input and output parameters when a user requests deviation from an absolute-strong policy, in order to calculate the probability of absolute-strong policy deviation being allowed, first the parameters that affect this value need to be determined. these are the input parameters for the simulation; see table 6.  the number of agents assisting individuals in the system who have the right to allow deviation.  the probability of permission for policy deviation. since the decision to permit someone to deviate from an absolute-policy is eventually an individual decision in podman, it can be stated that the probability of a user permitting deviation is mutually exclusive amongst all the users with the rights to do so. thus to calculate the total probability of the permission to deviate being granted, the inclusion-exclusion principle (szpankowski 2001) was used, which states |𝐴1 ∪ 𝐴2 ∪ … ∪ 𝐴𝑝| = ∑ |𝐴𝑖|1≤𝑖≤𝑝 − ∑ |𝐴𝑖1 ∩ 𝐴𝑖2 |1≤𝑖1≤𝑖2≤𝑝 + ∑ |𝐴𝑖1 ∩ 𝐴𝑖2 ∩ 𝐴𝑖3 |1≤𝑖1≤𝑖2≤𝑖3≤𝑝 − ⋯ + (−1)𝑝−1 |𝐴𝑖1 ∩ 𝐴𝑖2 ∩ ⋯ ∩ 𝐴𝑖𝑝 | here ai is the probability of each user with the necessary permissions allowing another user to deviate from a policy while |𝐴1 ∪ 𝐴2 ∪ … ∪ 𝐴𝑝| is the total probability of the permission to deviate being granted. australasian journal of information systems bakshi, talaei-khoei, ray & solvoll 2017, vol 21, research article podman: policy deviation management 15 in this simulation, the number of users was set with the rights to allow deviations; that is n, and randomly assign each user agent with the probability of them granting the permission to deviate. furthermore, for each case of the simulation, a limit was set for the maximum probability (0% to 100%) of a user agent granting permission to deviate at intervals of 25%. this probability was generated randomly for each user agent and ranged between 0 and the maximum specified probability. the maximum probability parameter reflects the overall perception of policy deviation being the correct decision amongst the privileged users. this was the output parameter that we measured in this simulation; see table 6. input parameters output parameter number of users with deviation rights maximum probability of user permitting policy deviation* total probability of deviation being permitted. * individual probability for each user is randomly generated between 0 and max. table 6: inputs and output parameters for the simulation 5.2 results the simulation was run 100 times for each case and calculates the total probability of policy deviation being allowed each time. finally, the average probability of deviation being permitted under each case was calculated. . the results of the simulation are shown in figure 15. a maximum probability of 0% represents an environment without podman where no deviation is allowed. in this case, it can be seen that the overall probability stays at 0% regardless of the number of users in the environment as deviations are not possible under any circumstances. however, once the maximum probability of allowing deviation to 25% is increased, an increase is noted in the overall probability of deviation being permitted. moreover, it can clearly be seen that the total probability of deviation increases with the increase of maximum probability for each user in the environment. in fact, unless the maximum probability all users in the system is 0, the total probability of deviation being permitted is always greater than 0%. this not only shows the effectiveness of podman compared to a system without podman, it also illustrates that the overall perception amongst all users affects the chances of deviation being permitted. similarly, an increase is noted in the overall probability of deviation being allowed, i.e. an increase in effectiveness of podman, when the total number of users in the system is increased. figure 15: simulation results: probability of absolute policy deviation being allowed these results were obtained when the simulation was programmed in c++ and run on a 2.1 ghz quad core cpu with 4 gigabytes of memory and running ubuntu 13.04. 0 0.5 1 0 1 2 3 4 5 6 7 8 9 10 15 20 to ta l p ro b ab ili ty o f p o lic y d ev ia ti o n b ei n g al lo w ed number of agents with permissions to allow deviation from absolute strong policies probability of deviation from absolute-strong policy 0% 25% 50% maximum probability of any australasian journal of information systems bakshi, talaei-khoei, ray & solvoll 2017, vol 21, research article podman: policy deviation management 16 5.3 discussion the simulation was set out to test the effectiveness of podman. in order to make sure that that podman can be successful under different conditions, a simulation was designed to calculate the probability of a policy deviation while user agents are being permitted based on randomly assigned probabilities for their permissions. the simulation was run while the number of user agents was varying. these settings were deployed to perform an experiment in which different conditions could be tested. under all conditions, it was observed that the probability of policy deviation being permitted is always greater when podman is in use compared to when it is not. while this is of course expected, it is interesting to note that the overall perception of whether deviation should be permitted or not affects the total probability of deviation being permitted. e.g. if the maximum probability of an user permitting another user to deviate is 25%, the total probability of deviation being permitted remains much lower than when this maximum probability is higher. in other words, this shows that the total probability of deviation depends on the overall perception of the users and therefore the final decision of whether or not deviation is eventually permitted tallies with what the expected decision should be. while this simulation might seem simplistic, it covers a whole range of circumstances that might occur in a collaborative organization and gives a generic picture of the effectiveness of podman when used by multiple users. 6 summary and outlook the main objective of this work was to assist individuals and organizations when exceptional or unexpected situations enforce the decision of policy deviation. the paper proposed an information technology methodology, including a framework and a process, to help users performing non-complying action; that is policy deviation management (podman). the study was initially motivated by a real scenario observed in st olvas hospital in norway. while policies fail to manage a situation in which unexpected or exceptional conditions occurs, one can change the policy or perform a non-complying action. in real-time response, the first solution can only cover social policies that are changeable, however for legal policies changing would require authorities’ permission and organizational processes and would not be possible in real-time. therefore, the idea of policy deviation, or relaxing policies to perform noncomplying action, policy deviation, was introduced by (lewis 2010) for legal policies. however, the idea lacks of technical details and a definitive method. a systematic review conducted to identify methods that can assist in implementation of policy deviation. however, it was found that the literature has ignored such a method to help individuals or organizations to decide about policy deviations and the non-complying actions. the paper has introduced different types of deviation depending on the application of the policies in the scenario. the study has also presented a detailed method of how software agents can be used to detect the different types of policy deviations and how to compute and rank alternative actions when a policy deviation is detected. 6.1 implications for research the decision of choosing whether to deviate from a policy under exceptional circumstances is not a simple one, especially when users are unaware of all the policies that are in place and the legal implications behind them. thus it is important to develop a method that makes this decision making process simpler and more controlled. the existing research on policy deviation focuses on policy adaption where policy parameters are modified to match changing environmental conditions or exceptions. swanson provides a framework for policy adaption which involves setting up, adjusting and maintaining policies. the major drawback with this is that all the exceptions need to be considered before-hand or much after the event has already happened during the maintenance phase. podman on the other hand presents a methodology with allows policy deviations and exceptions to be managed at run time and involves multiple parties in the decision making process when required. along with this it opens up avenues for australasian journal of information systems bakshi, talaei-khoei, ray & solvoll 2017, vol 21, research article podman: policy deviation management 17 researchers of decision support systems and cscw to design and refine policy based systems in order to make them more flexible than they are currently. 6.2 implications for practice as more and more organizations move towards policy based collaborative systems, it becomes important to make sure that the right users have the right privileges to access information and perform tasks. while policies are designed keeping exceptions in mind, it is impossible for policy makers to consider every single exception. thus there is a definite need for a more flexible system that allows policy deviations in a controlled manner when absolutely necessary. podman provides a methodology to create such a system, and both the designers and users of policy based collaborative platforms can use podman to add a higher degree to flexibility to their systems as podman will not only allow deviations to take place under exceptional situations, it will also allow managers to monitor the exceptions and incorporate them into existing policies. 6.3 limitations the present work involves some limitations that we would like to point out. 6.3.1 determining the magnitude of deviation while podman provides a method of managing and handling policy deviations by determining whether the policy being violated is a conditional or absolute strong policy, it does not inherently know the actual impact the policy violation has on the environment. researchers in the future can refine the podman by incorporating the magnitude of deviation or the impact of deviation in the process e.g. though policy ranking. 6.3.2 managing policy updates podman provides managers and policy designers with the information containing all the deviation requests in the system using which they can modify existing policies to incorporate previously unforeseen exceptions. currently podman simply facilitates a manual process for policy maintenance where the information can be used to update the existing policies and branching time models. there is potential future work in developing methods to automatically update policies and btms when a previously unforeseen scenario is considered by the management. 6.3.3 automatic generation of branching time models from decision trees currently the process of generating btms from decision trees is a manual process and can be tedious to create. it might be interesting to design intelligent systems which can automatically generate btms based on the existing policy and procedure documents for each role and update these btms automatically whenever policy or procedures are changed in the environment. the instruction to implement this automation has been explained in section 6. 6.3.4 complex decision process while complex actions and events are involved. while btm is an effective way of representing role behaviors and actions, the events and actions in this representation are primitive. i.e. the events are accepted directly by the agents and the actions are performed directly by the agents as well. this is obviously an idealization. in real settings the events and the actions are usually very complex and made up different events and actions. in some cases this representation might be too simplistic and more complex behaviors might need to be represented to accurately describe the roles. future research can explore methods of representing complex events and actions through btms or develop a method to reduce complex actions into multiple primitive actions. having done that, the decision process would be possible to be deployed in real settings before transforming the complex events and actions to primitive ones. australasian journal of information systems bakshi, talaei-khoei, ray & solvoll 2017, vol 21, research article podman: policy deviation management 18 6.3.5 real life implementations while the current evaluation demonstrates how podman can be applied in real life scenarios to prove its applicability in real life, an actual organizational implementation should provide a greater insight to the strengths and weaknesses of this technique. in addition, although this thesis provides a generic methodology for policy deviation under exception using information systems, it would be interesting to study how it can be used to design complete systems in different domains such as banking and health care. it would also be of interest to note and compare how these systems would perform under actual emergencies such as natural disasters or terrorist attacks. acknowledgements the work was partly supported by the arc centre for excellence in population and ageing research (cepar), unsw and the smart services crc. the work was carried out at the asia pacific ubiquitous healthcare research centre (apuhc), unsw. references bakshi, a., talaei-khoei, a., & ray, p. (2013). adaptive policy framework: a systematic review. journal of network and computer applications, 36(4), 1261-1271. benbasat, i., & nault, b. r. (1990). an evaluation of empirical research in managerial support systems. decision support systems, 6(3), 203-226. boella, g., & van der torre, l. (2003, july). attributing mental attitudes to normative systems. in proceedings of the second international joint conference on autonomous agents and multiagent systems (pp. 942-943). acm. chau, m., zeng, d., chen, h., huang, m., & hendriawan, d. (2003). design and evaluation of a multi-agent collaborative web mining system. decision support systems, 35(1), 167183. czap, h. (2002). policy-agents for negotiation within virtual organizations. international journal of information technology & decision making, 1(03), 385-400. ellis, c. (1991). groupware: overview and perspectives. in verteilte künstliche intelligenz und kooperatives arbeiten (pp. 18-29). springer, berlin, heidelberg. fitzpatrick, g., & ellingsen, g. (2013). a review of 25 years of cscw research in healthcare: contributions, challenges and future agendas. computer supported cooperative work (cscw), 22(4-6), 609-665. lewis, l., parameswaran, n., & ray, p. (2010, july). fusion and policy relaxation in patient healthcare. in 2010 13th conference on information fusion (fusion), (pp. 1-5). ieee. lotlikar, r. m., & mohania, m. (2006, september). adaptive policies in information lifecycle management. in international conference on database and expert systems applications (pp. 612-621). springer berlin heidelberg. migon, l. b., borges, m. r., & campos, m. l. m. (2010, september). a method for identification and representation of business process deviations. in international conference on collaboration and technology (pp. 49-64). springer berlin heidelberg. moffett, j. d., & sloman, m. s. (1991, october). the representation of policies as system objects. in acm sigois bulletin (vol. 12, no. 2-3, pp. 171-184). acm. ouyang, s. (2006, may). integrate policy based management and process based management-a new approach for workflow management system. in 10th international conference on computer supported cooperative work in design, 2006. cscwd'06. (pp. 1-6). ieee. australasian journal of information systems bakshi, talaei-khoei, ray & solvoll 2017, vol 21, research article podman: policy deviation management 19 prakash, a., shim, h. s., & lee, j. h. (1999). data management issues and trade-offs in cscw systems. ieee transactions on knowledge and data engineering, 11(1), 213-227. rao, a. s., & georgeff, m. p. (1991). modeling rational agents within a bdi-architecture. kr, 91, 473-484. rao, a. s., & georgeff, m. p. (1995, june). bdi agents: from theory to practice. in icmas (vol. 95, pp. 312-319). ray, p., shahrestani, s. a., & daneshgar, f. (2005). the role of fuzzy awareness modelling in cooperative management. information systems frontiers, 7(3), 299-316. ray, p., parameswaran, n., chan, v., & yu, w. (2008). awareness modelling in collaborative mobile e-health. journal of telemedicine and telecare, 14(7), 381-385. ray, p., & chattopadhyay, s. (2009). fuzzy awareness model for disaster situations. intelligent decision technologies, 3(1), 75-82. shum, s. b., cannavacciuolo, l., de liddo, a., iandoli, l., & quinto, i. (2013). using social network analysis to support collective decision-making process. in engineering effective decision support technologies: new models and applications (pp. 87-103). igi global. sloman, m. (1994). policy driven management for distributed systems. journal of network and systems management, 2(4), 333-360. strassner, j., de souza, j. n., van der meer, s., davy, s., barrett, k., raymer, d., & samudrala, s. (2009). the design of a new policy model to support ontology-driven reasoning for autonomic networking. journal of network and systems management, 17(1-2), 5-32. strassner, j., liu, y., jiang, m., zhang, j., van der meer, s., foghlú, m. ó., ... & donnelly, w. (2008, april). modelling context for autonomic networking. in ieee network operations and management symposium workshops, 2008. (pp. 299-308). ieee. strassner, j., liu, y., & zhang, j. (2008, july). a context-aware policy model to support autonomic networking. in 32nd annual ieee international computer software and applications, compsac'08. (pp. 1097-1102). ieee. swanson, d. & bhadwal, s. (2009). creating adaptive policies: a guide for policymaking in an uncertain world, sage publications pvt. ltd. swanson, d., venema, h. d., rust, c., & medlock, j. (2000). understanding adaptive policy mechanisms through farm-level studies of adaptation to weather events in alberta, canada. international institute for sustainable development. szpankowski, w. (2001). inclusion‐exclusion principle. average case analysis of algorithms on sequences, 49-72. talaei-khoei, a., solvoll, t., ray, p., & parameshwaran, n. (2011). policy-based awareness management (pam): case study of a wireless communication system at a hospital. journal of systems and software, 84(10), 1791-1805. twidle, k., dulay, n., lupu, e., & sloman, m. (2009, april). ponder2: a policy system for autonomous pervasive environments. in fifth international conference on autonomic and autonomous systems, 2009. icas'09. (pp. 330-335). ieee. ulieru, m., & worthington, p. (2005, october). holonic risk management framework. in ieee international conference on systems, man and cybernetics, (vol. 1, pp. 209-214). ieee. ulieru, m., & worthington, p. (2006). autonomic risk management for critical infrastructure protection. integrated computer-aided engineering, 13(1), 63-80. verhagen, h. (2001, december). norms and artificial agents. in sixth meeting of the special interest group on agent-based social simulation, esprit network of excellence on agent-based computing. australasian journal of information systems bakshi, talaei-khoei, ray & solvoll 2017, vol 21, research article podman: policy deviation management 20 woolridge, m., & wooldridge, m. j. (2001). introduction to multiagent systems. new york, ny, usa: johnwiley & sons. yuan, y., & detlor, b. (2005). intelligent mobile crisis response systems. communications of the acm, 48(2), 95-98. copyright: © 2017 bakshi, talaei-khoei, ray & solvoll. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ http://creativecommons.org/licenses/by-nc/3.0/au/ ajis vol. 6 no. 1 september 1998 electronic commerce, the internet and travel cybermediarees paul f. o'brien information systems group business faculty sunshine coast university college locked bag no. 4, maroochydore south, qld, 4558, australia email pobrien@scuc.edu.au doctor of philosophy student department of computer science and electrical engineering university of queensland queensland 4072 australia and distributed systems technology centre pry ltd department of computer science and electrical engineering university of queensland queensland 4072 australia abstract the purchase of a very expensive, complex holiday requires a great deal of information and reliable expert advice. the risks involved in such a purchase are substantial. industry opinion leaders believe that consumers will continue to take advice from a human sales consultant before purchasing, with the exception of major corporate bookings and simple point to point bookings. this paper investigates the likely disintermediation effects of electronic commerce on retail travel agents. it provides the basis for the development of a strategic framework for intermediaries that can be used to enhance business performance. the findings of exploratory interviews with travel industry opinion leaders are presented, together with recommended strategic responses for retail travel agents. keywords tourism, information technology, electronic commerce, travel agent, internet introduction the purchase of a very expensive, complex product or service such as an extended international holiday requires a great deal of information and reliable expert advice. the risks involved in such a purchase are substantial. it is for this reason that opinion leaders in the travel and tourism industry believe that for the foreseeable future, most consumers will continue to take advice from a human sales consultant before purchasing, with the exception of major corporate bookings and simple point to point bookings. this research investigates the likely disintermediation effects of electronic commerce on the travel industry with a view to developing a strategic framework to guide intermediaries in the travel marketing channels on how to take advantage of any potential opportunities and how to minimise potential threats through the use of information technology. information technology in tourism and travel information technology has become progressively more important to the travel and tourism industry. the most dramatic change came in the 1980s with the introduction of central reservation systems (crss). crss increased the power of the airlines in the distribution chain and dramatically altered the balance of power in the wholesale travel market (poon, 1993). electronic commerce over the internet is the next major wave of technological change that will hit the industry. tourism is an information intensive industry. fast, efficient exchange of information between the players in the industry is essential for efficient distribution, sales and customer service. information and communication technologies currently play a vital role and are poised to play an increasingly important role as global, universal access to the internet, and its progeny, becomes a reality. these emerging new technologies will allow more effective marketing and distribution of services and will release existing staff from back-office tasks so that more time can be spent in "high touch" customer services (poon, 1993). information technology is having, and will continue to have, dramatic effects on the travel industry as it, • substantially alters the roles of intermediaries in the distribution chain (poon, 1993) 70 ajis vol. 6 no. 1 september 1998 • facilitates the production of "new, flexible travel products and services that are cost-competitive with mass, standardised and rigidly packed" products" (poon, 1993) • assists in transforming "tourism and travel into a more flexible, individual-oriented, sustainable and diagonally integrated industry" (poon, 1993) whoever controls the information, will control the channel. current trends in the travel industry in the past, distribution was closely tied to settlement. electronic ticketing and settlement will separate settlement from distribution, enabling new, flexible packaging structures (eastman, 1996). there is a worldwide trend towards independent travel (queensland tourist & travel corporation, 1997; poon, 1993). even the concept of a packaged holiday has changed dramatically over recent years. a typical package is now a fly-drive holiday with hire car and accommodation vouchers, or perhaps a semi-personalised package assembled from a number of mini-packages (travelworld, poon 1993), the size of which is getting smaller. the industry is facing increased competition through globalisation and de-regulation., changing customer demands for more specialised trips, increasing customer expectations, particularly in relation to customisation and increasing knowledge and sophistication of customers. (bloch & segev, 1996). disintermediation risks of new information technology very few empirical studies have been carried out to identify the intermediaries that are most likely to be affected by electronic commerce, or the likely extent of the disintermediation. there has been considerable discussion in the travel industry press about the inevitable "disintermediation" of travel agents as a result of the widespread availability of the internet and it in general. however, "from an academic point of view the "disintermediation" hypothesis is little documented, especially with respect to its underlying theoretical arguments and implications" (van der heijden, 1996). virtual information space virtual transaction space! virtual communication space virtual distribution space i angehrn, 1997 figure 1: the icdt model internet commerce has introduced new, virtual market spaces in addition to the traditional physical market spaces. angehrn (1997) introduced a generic framework, the icdt model (figure 1) to classify and illustrate significant business opportunities and threats generated by the internet. the model describes four new, virtual market spaces that are being created by the internet. they are the information, communication, transaction and distribution spaces. angehrn treats each space separately because they each require different strategies and technological approaches. the virtual information space is where company, product and price information is provided to die customer. it is a one-way communication channel. the virtual communication space is where interaction and relationship building with the customer occurs. the virtual distribution space is where products and services are delivered. since it is a 71 ajis vol. 6 no. 1 september 1998 virtual space, it is only suitable for delivery of digitised products and services. the virtual transaction space is where orders, invoices, and payments occur. figure 2 shows this model applied to the travel industry. virtual information space •product info •destinational info •availability info •rates info •package building virtual transaction space i virtual communication space! relationship building 'information assessment consulting advice 'bargain hunting •reservations •payment figure 2: the travel industry icdt model virtual distribution space •guides •itineraries •weather reports the risk of disintermediation is related to the ease with which technology can be used to replace the role of an intermediary in each of these virtual spaces. bloch and segev (1996) proposed that the positive impacts of electronic commerce for existing intermediaries would be: • raising of entry barriers for new entrants by increasing customer switching costs • faster introduction of new products • re-intermediation of new intermediaries acting as information brokers bloch and segev (1996) argue that electronic commerce will have most impact on industries where customer service and information play an important role in the buying process. this category would include industries such as banking, travel and retailing of commodities such as books, cds, and wine. when their methodology was applied to the travel industry, and travel agents specifically, the greatest impact was shown to be on the travel agent's roles as transaction processors (ticketing and settlement) and suppliers of raw unprocessed, product information. these roles will continue to be replaced by technology as suppliers provide standard product information, electronic ticketing and electronic funds transfer services directly to the consumer. the role of advisor, critically assessing product and destinational information and assisting the consumer to make a suitable choice based on their preferences, was not seen to be under any real threat. this indicates that travel agents need to re-position and focus on the latter role if they are to survive and grow. brenner, kolbe and hamm (1996) and lenz (1996) support bloch and segev's position. brenner, kolbe and hamm investigated the risk of disintermediation in the academic book publishing industry and concluded that the risk is related to the ability of information technology to replace the core competencies of the intermediaries. lenz concluded that the roles that are most easily replaced by technology, transaction processing and information provision, will be the first to be automated, as has already happened with the banks. the recent moves by airlines to cap or reduce travel agent's commission and continuing expansion of direct sales to consumers, highlights the vulnerability of the transaction space. similarly, focussing on recommending and selling preferred suppliers standard packaged products, presumably because of higher commissions or lower cost of sale, compromises the travel agents perceived independence and value added in the mind of the customer. poon (1993) asserts that "players closest to consumers (eg. travel agents, hotels) and those in control of the industry's information are expected to gain". since the travel agent is the closest intermediary to the customer at the point of sale (and usually the point of departure), the risk of disintermediation will be minimal if they are operating in an area that is difficult to replace with it. the market spaces where the greatest risk of disintermediation exists 72 ajis vol. 6 no. 1 september 1998 for travel agents are the transaction, information and distribution spaces. the space where least risk exists is the communication (or relationship) space. this may seem paradoxical since information and communications technologies are creating the risks. however, angehrn (1997) defines the communications space as the space where person to person communication takes place and is therefore minimally affected by communications technology itself. communications technology simply provides the infrastructure that allows the exchange of information between human participants at either end of the communications channel. it does not enhance the message, it simply delivers it. re-intermediation opportunities while information technology is designed to capture, store and process information, it is not capable (at this stage) of capturing, storing and analysing the "semantics" of that information to the same level of performance as a human expert. the internet is a huge repository of information but it still requires a human intermediary to assess the reliability, quality and appropriateness of the retrieved information. over time this situation will improve as more standardisation and controls are put in place, but the anarchic nature of the internet will make it extremely difficult to enforce. poon (1993) argues that if travel agents adopt an appropriate strategic response to the changes that are occurring, there will be enormous prospects for increased income generation. "it is to ensuring the satisfaction of the travel consumers that agencies must give priority in order to ensure their long-term survival and competitiveness. the ability of travel agents to acquire, provide and transmit unbiased information in a courteous, efficient and timely manner will be the key to their competitive success." (poon, 1993) more recent literature (bloch & segev, 1996; karcher, 1996; milne, 1996; ng & sussman, 1996), electronic ticketing initiatives by the airlines, and the opinions of the participants in this study indicate that ticketing is rapidly reducing in importance and the ability to create personalised packages is now vitally important. given that provision of advice is a core competency, then the risk of disintermediation will increase only if provision of advice can be easily replaced by technology. this is highly unlikely in the short term. at the same time, the quality of the advice provided by retail travel agents should be significantly improved by access to reliable, high quality destinational information (milne, 1996), through the internet, assisting them to reposition to more of a consulting role. real opportunities exist for the travel agent of the future to become a network information broker (lamp & keen, 1997) by finding, assessing and quality controlling information obtained from the internet and other electronic and physical sources and conveying it, together with appropriate advice, to the customer in a clearly understandable form. the changing role of retail travel agents exploratory interviews were carried out with travel industry opinion leaders, retail travel consultants and recent travellers to compare their views on the effects of electronic commerce on retail travel agents with those expressed in the literature.. the main findings were: • direct bookings via the internet are not having a significant impact • simple point to point bookings are expected to increasingly be made directly • customers are tending to do research on the internet, then book through a retail travel agent • customers are becoming more sophisticated and knowledgeable and are looking for more exotic, specialised holidays • niche marketing will become increasingly necessary • customers are requiring increasing package personalisation • the relationship with the customer is everything • fee for service is inevitable. it is just a matter of when and how • travel agents will increasingly become agents of the people who are travelling rather than agents of the principals, airlines, wholesalers and operators • a number of travel agent that exist today will not survive those that don't re-position 73 ajis vol. 6 no. 1 september 1998 current travel agent positioning figure 3 shows the current positioning of retail travel agents within angehrn's virtual market spaces. their roles are heavily weighted to the transaction, distribution and information spaces, the very spaces where technology is likely to have the greatest disintermediation effects. virtual information space i ̂ p-~^ [virtual communication space] & virtual distribution space j figure 3. current travel agent positioning the need to add value an intermediary will only survive in a distribution channel while it continues to add value (porter, 1985). this research has shown that the traditional roles of travel agents, transaction processing and information delivery, are being rapidly de-valued by electronic commerce. at the same time, the changing attitudes and preferences of customers and the explosion in available travel information on the internet have increased the value of their consulting (or relationship) and information broking roles. a continual shift towards the later two roles will be essential if they are to continue to add value. the importance of relationships the importance of building a relationship with the customer and clearly understanding his/her preferences is not a new concept. it was the theme of the keynote address by richard goeglein at the travel and tourism research association conference in memphis in 1986. goeglein stated that "the conventional wisdom was to offer what was convenient for the operator to provide, or what was better than the competition, rather than to define what the customer wanted and then be sure he/she received it....we can no longer afford the kind of thinking that puts industry assumptions about the customer ahead of a committed effort to gather facts about the customer " (goeglein, r., 1986). more recently david near, galileo international's senior vice-president subscriber marketing urged agents to " leverage the great strengths which you have and which everyone else wants. those strengths include your customer base, a deep knowledge of what it takes to satisfy those customers' needs and strong relationships with them." (near, 1997) the importance of access to information customers are demanding higher service levels, better quality information and personalised packages. information on the internet, is badly organised, distributed and unstructured. it is a time consuming nightmare (tschanz & klein, 1996) for a customer to access and assess the required information and assemble it into a personalised holiday. travel agents that have access to appropriate technological tools that can streamline this process, and can use them effectively and efficiently, can improve the quality of advice given considerably (milne,1996), thereby adding value. 74 ajis vol. 6 no. 1 september 1998 local tourist operators at most destinations are typically small to medium sized tourism enterprises (smtes), are normally organised on a regional or national level by means of public tourist boards (werthner, 1996) and are not typically accessible through crss. consequently the information required by the "new tourist" is not accessible to most travel agents who typically only have crs terminals (and no access to the internet). as a result, the advice given by these travel agents is based on incomplete information at best, or incorrect information at worst. research in the united kingdom has shown that an information system that integrates economic, social, cultural and environmental information at the destination level is required to ensure the long-term prosperity of smtes and destinations (buhalis, 1996). development of such systems is equally important to travel agents as they require this information to provide quality advice to their customers. enhancing retail travel agent performance internet commerce presents significant opportunities for travel agents to improve their performance by re-positioning themselves to roles that increase the value they add. future travel agent positioning virtual information space virtual transaction space virtual communication spacel virtual distributionion space i figure 4. future travel agent positioning the literature and the exploratory interviews with industry opinion leaders indicate that travel agents will need to re-position themselves from simple transaction processors, information providers and simple resellers to consultants, information brokers and developers of personalised products if they are to survive. figure 4 shows the required future positioning of retail travel agents. travel information broker direct contact between suppliers and consumers via the internet will result in consumer information overload. to address this problem, new intermediaries will emerge to help the consumer integrate and analyse information (bloch & segev,1996; lamp & keen,1997). in the short to medium term this role can be filled by travel agents using sophisticated software tools and agents. retail travel agents must become proficient at using these tools, and their own personal experience and knowledge, to access, integrate, analyse, interpret and present information from distributed, heterogeneous sources. this is a difficult task when the consultant has personal knowledge of the activity, product or destination but it is almost impossible when he/she has no personal knowledge or experience. in these situations the software agent must be able to search semantically for relevant products or destinations to meet the new demands of the "new tourist" (geyer et al, 1996). this view is supported by david near, galileo international's senior vice-president subscriber marketing who was recently quoted as saying that ".the proliferation of "info-junk" would require a new breed of info-navigators employing advanced information technologies to enhance and deliver their services" (near, 1997). 75 ajis vol. 6 no. 1 september 1998 a similar view was expressed by des paroz of southern cross galileo who said that" the internet was "an enormous repository of information" which could be used for background research and passed on to customers, adding value to travel agent's services (paroz, 1997). travel consultant the consulting role of the travel agent must move away from transaction processing to personalised consulting advice focussed on the needs of the customer rather than the needs of the airlines and wholesalers. a particular problem facing any travel consultant is matching the customer's requirements to available destinations and/or products. this is a complex operation that requires a high level of intelligence. the process is complicated by the fact there are no universal standards for the classification and description of destinations and products (baker et al, 1996). for example, hotel classifications are different in each country and in some cases in each state within a country. in addition, some features that are important to the customer are not rated at all, for example, friendliness of the staff. in many cases an exact match of customer preferences to available products or destinations will not be possible. in this situation, consultants can only do their best to get the closest match possible with the available information. to get an accurate match, the semantics of the customer's preferences must be mapped to the semantics of the features of the product or destination. baker et al (1996) recognised that "there is clearly a need for an intelligent system that incorporates local knowledge, regularly updated, and interpretation of the customers' needs, into the mapping process. intelligent agents, either obeying rules when they exist, or observing and recording in order to learn or infer the correspondence, seemed ideally suited." the travel agent, assisted by such an intelligent software agent, would create a very powerful travel cybermediary that could dramatically improve the quality of advice provided to the customer, and ultimately, improve customer satisfaction. while this should allow retail travel agents to survive in the short term, the rapidly increasing technology skills of their customers are likely to reduce the perceived added value of this information broking role. travel agents will need to "continually redefine themselves" (sheldon, 1997) and operate in increasingly specialised niche markets if they are to survive. this will require a change in role to relationship building and counselling so that they are able to clearly understand the customer's needs (ng & sussmann, 1996). personalised package developer the changing preference of customers for more individual, personalised holidays requires the travel agent to develop skills in integrating sub-products from different suppliers into a personalised package tailored to the customer's needs and preferences. this is a complex, time-consuming activity that must be highly automated to reduce the cost of sale and make it cost effective. intelligent itinerary builders that have access to a wide range of packages, minipackages and independent operators' products from a wide range of sources including the internet and crss will be required. some work has been done on the development of prototype intelligent agents that assist in the selection of suitable holiday packages (imtas/cabata) (lenz, 1996) however little work has been done on the development of intelligent agents that assist in the selection and assembly of components into personalised holiday packages. this is a rich area for future research. conclusion and further research "the experience of shopping is probably the only thing that electronic media will never really be able to fully replicate. it should therefore be fully exploited in the real world." (bloch & segev, (1996) to minimise the risk of disintermediation, and to improve business performance, travel agents should reposition themselves to the virtual spaces where technology will have least effect, and should make maximum use of technology in the other spaces to enhance their core competencies and value added. they will need to reduce their dependence on income from transactions and increase their income from intelligent, knowledge based activities such as counselling, information broking and package personalisation. fast, easy access to reliable product, destinational and personal experience information is shown to be critical to the survival of travel agents as consumer preference for personalised packages to more exotic destinations grows. 76 ajis vol. 6 no. 1 september 1998 the difficulty in finding, accessing and analysing a growing ocean of information indicates the need for an intelligent travel agent assistant. however, the limited abilities of current intelligent agents, and the extremely complex and "personal" nature of an extended holiday to an exotic destination mandate the continuing use of human travel consultants, working with intelligent agents. together, the travel agent and the intelligent agent will act as a travel network information broker, working on behalf of the customer, and hopefully for a consulting fee. the development of a suitable digital representation of the customer's desired experience and preferences, and the features of the product or destination, and an intelligent assistant that can interpret this information, search for matching products and destinations within all the sources of information available to it, assess the reliability and quality of the information accessed and make a range of recommendations to the human travel consultant is the subject of further research by the author. references australian federation of travel agents (1994) travel industry technology towards information highways, sydney, australian federation of travel agents. angehrn, a. (1997) the strategic implications of the internet, proceedings of the 1997 european conference on information systems baker, m., lockwood, a. & sussman, s. (1996) application standards for tourism products. proposals for a program of research, proceedings of the 3rd international conference on information and communications technologies in tourism, springer-verlag, vienna. brenner, w., kolbe, l. & hamm, v. (1997) the net: extinction or renaissance for intermediaries an analysis of core competencies in the book business, proceedings of the 1997 european conference on information systems. bloch, m. & segev, a. (1996) the impact of electronic commerce on the travel industry, the fisher centre for information technology & management, university of california, california. buhalis, d. (1996) information and telecommunication technologies as a strategic tool for tourism enhancement at destination regions, proceedings of the 3rd international conference on information and communications technologies in tourism, springer-verlag, vienna. geyer, g., kuhn, c. & schmid, b. (1996) an electronic catalog for distributed environments, proceedings of the 3rd international conference on information and communications technologies in tourism, springerverlag, vienna. goeglein, r. (1986) technology and tourism : a growing partnership, proceedings of the 17th annual conference of the travel and tourism research association conference, tourism and travel research association, salt lake city. karcher, k. (1996) re-engineering the package holiday business, proceedings of the 3rd international conference on information and communications technologies in tourism, springer-verlag, vienna. lenz, m. (1996) jmtas: intelligent multimedia travel agent system, proceedings of the 3rd international conference on information and communications technologies in tourism, springer-verlag, vienna. milne, s. (1996) tourism marketing and computer reservations systems in the pacific, tourism in the pacific: issues and cases, international thomson business press, london. near, d. (1997) travelweek. (30 july, p 16). open buying on the internet consortium (1997) open buying on the internet standard, release vi.0, the obi consortium, usa. paroz, d. (1997) travelweek. (30 july, p 17) poon, a. (1993) tourism, technology and competitive strategies, cab international, wallingford. porter, m. (1985) competitive advantage, free press, new york. queensland tourist & travel corporation (1997) international marketing briefs. sheldon, p. (1997) tourism information technology, cab international, new york. travel and tourism research association (1986) technology and tourism: a growing partnership, proceedings of the 17th annual conference of the travel and tourism research association, salt lake city: bureau of economic and business research. tschanz, n. & klein, s. (1996) internet strategy development in tourism concept and case study, sisnet research workshop 'internet and business', sisnet, lisbon. 77 ajis vol. 6 no. 1 september 1998 van der heijden, j. (1996) the changing value of travel agents in tourism networks: towards a network design perspective, proceedings of the 3rd international conference on information and communications technologies in tourism, springer-verlag, vienna. werthner, h. (1996) design principles of tourist information systems, proceedings of the 3rd international conference on information and communications technologies in tourism, springer-verlag, vienna. acknowledgements the author acknowledges the assistance provided by dr. robert colomb, university of queensland and jill dunn, sunshine coast university college in clarifying and focussing the concepts in this paper. the work reported in this paper has been funded in part by the cooperative research centres program through the department of prime minister and cabinet of the commonwealth government of australia 78 paper title (paper title style) australasian journal of information systems horne, maynard & ahmad 2017, vol 21, research article organisational information security strategy 1 organisational information security strategy: review, discussion and future research craig a. horne department of computing and information systems the university of melbourne chorne@student.unimelb.edu.au sean b. maynard department of computing and information systems the university of melbourne atif ahmad department of computing and information systems the university of melbourne abstract dependence on information, including for some of the world’s largest organisations such as governments and multi-national corporations, has grown rapidly in recent years. however, reports of information security breaches and their associated consequences indicate that attacks are escalating on organisations conducting these information-based activities. organisations need to formulate strategy to secure their information, however gaps exist in knowledge. through a thematic review of academic security literature, (1) we analyse the antecedent conditions that motivate the adoption of a comprehensive information security strategy, (2) the conceptual elements of strategy and (3) the benefits that are enjoyed postadoption. our contributions include a definition of information security strategy that moves from an internally-focussed protection of information towards a strategic view that considers the organisation, its resources and capabilities, and its external environment. our findings are then used to suggest future research directions.1 keywords: information security strategy; organisational strategy; security quality; strategic information systems; business management 1 introduction information resources play a critical role in sustaining business success by driving innovation and opportunities for the development of competitive advantage. as such, preservation of the confidentiality, integrity and availability of these information resources is a significant imperative, which begins with the development of an appropriate information security strategy in organisations (issio). the aim of this paper is to identify a strategic approach to securing information resources for the benefit of those decision-makers accountable for strategic-level organisational information security and ultimately, organisational success. the scope of the research is to examine the conceptual construct of issio. in particular, the authors of this paper are motivated by calls from other information systems researchers for the development of a comprehensive security strategic framework (baskerville et al. 2014), and for future research into the role that boards of directors may play in information security practices (mcfadzean et al. 2006). significantly, some of the world’s largest organisations, including governments and multinational corporations, have publicly suffered major information security incidents. by broadly 1 an earlier version of this article was published in proceedings at australasian conference on information systems 2015, see: horne, c.a., ahmad, a., and maynard, s.b. 2015. "information security strategy in organisations: review, discussion and future research directions," the 26th australasian conference on information systems, adelaide, australia. australasian journal of information systems horne, maynard & ahmad 2017, vol 21, research article organisational information security strategy 2 reviewing the extant literature, a perspective is established that can support the development of a comprehensive issio which might then be generalisable to all organisations. this paper is a critical literature review on the topic of issio. papers from various researchers were analysed and evaluated before being compared for depth of understanding and conclusions drawn. the paper commentary is explicative, interpretative and centres on the determination of the conceptual and theoretical nature of issio. the paper continues in four major sections. initially we introduce issio, discuss its origins and existing definitions whilst expanding on some of its more central properties. second, we review the construct space of issio to understand how extant research views the way issio is conceptualised, the level of analysis from which issio is approached, and propositions for measuring the distinct elements of an issio. third, we review the nomological network space to assess the environmental antecedents, conceptual elements, and possible yields from an issio. finally, we draw conclusions, construct a definition, consider limitations and provide suggestions for future research to advance understanding of information security strategy. 2 defining information security strategy definitions of issio are infrequent in the information systems literature so, in an indulgent departure from convention, the discussion in this section is largely author-centric rather than concept-centric. information security strategy is defined by beebe and rao (2010, pg. 330) as “the pattern or plan that integrates the organisation‘s major is security goals, policies, and action sequences into a cohesive whole”. these authors assert issio is a documented plan which matches an assessment of external cyber threats with a financially-informed set of internal countermeasures, including the required supporting policies and procedures. strategy is seen as the means to influence an organisation’s environment through the careful selection of internal controls. park and ruighaver (2008, pg. 27) define information security strategy as: “an art of deciding how to best utilize what appropriate defensive information security technologies and measures, and of deploying and applying them in a coordinated way to defence organisation’s information infrastructure(s) against internal and external threats by offering confidentiality, integrity and availability at the expense of least efforts and costs while to be effective”. these authors assert issio has been developed from the military literature and therefore tends to be focussed more on how to deploy strategies than focus on what goals the organisation is trying achieve. in terms of attempting to classify issio, their analysis of earlier literature leads them to the conclusion that issio balances three dimensions which are time, space and the decision-making process. ahmad et al. (2014b) and park and ruighaver (2008) believe issio can be used to incrementally improve the quality of the information security program, however there must be a strong link from the issio to the organisational strategic plan to support it. issio is necessary to prevent threats to an organisation’s information. issio can take the form of one of a number of areas which include deterrence, prevention, surveillance, detection, response, deception, perimeter defence, compartmentalisation and layering. senior business sponsorship of the security function is also required. hong et al. (2003) do not define issio per se but assert that it is a function of policy orientation, risk management orientation, control and auditing orientation, management systems orientation and contingency management. contingency management is assessed by the authors as a function of the organisational environment, management and technology. sveen et al. (2009) contend that an issio is like any other business strategy: it is the process of building up resources. by simply explaining what an issio is, sveen et al. (2009) describe australasian journal of information systems horne, maynard & ahmad 2017, vol 21, research article organisational information security strategy 3 the construct but have not provided a formal definition. their insights are still useful however in building up our cumulative understanding. these definitions give an insight into the difficulties with achieving unanimity on defining issio. using conceptualisation of issio as an example, beebe and rao (2010) explain it is a plan, sveen et al. (2009) assert it is a process and conceptualisations from park and ruighaver (2008), ahmad et al. (2014b) and hong et al. (2003) do not fit within either of these. there are many other researchers who have used the term ‘information security strategy’ in their literature however they have not provided an explicit definition. these various definitions universally describe information security strategy as being an organisational-level construct, as opposed to individual, group or inter-organisational. this is in spite of there being implications for other levels, for example the role of individuals in the governance of information security strategy. this is consistent with definitions of strategy from reference disciplines such as military and business. in military literature, strategy is the highest level of planning, with the next level lower down termed operational, and tactical level below that (bowdish 2013). the tactics, techniques and procedures at the lowest level support the achievement of operations, which then support a nation’s strategic goals at the highest level. at the strategic level, nations guide the facilitation and achievement of national goals and objectives, using critical infrastructure and other national resources to achieve them (bowdish 2013). from the business literature, views on strategy abound (drucker 1958; mintzberg 1987), however in one of the most highly-cited strategic management books of the last century, three generic competitive strategies were identified: 1. differentiation, 2. overall cost leadership and 3. focus (porter 1980). these generic competitive strategies form the basis of competitive advantage which drives profitability (widely defined as total revenue less total expenses) (porter 1980). these strategies provide insight into how strategy is defined and used within the business domain and an explanation follows. differentiation is achieved by creating the industry-wide perception that an organisation is unique, ideally in more than one way, which reduces buyer power due to less comparable alternatives, allowing margins to be increased (porter 1980). overall cost leadership is a focussed attempt to lower the cost of goods sold, relative to an organisation’s competitors, which is achieved through high market share or through favourable access to raw materials, resulting in a wider profit margin (porter 1980). focus concentrates sales efforts on a target market segment, rather than adopting an industry-wide approach, amplifying the effects of overall cost leadership through lower costs from servicing only a segment of the market, or differentiation by becoming known for servicing one particular market segment well (porter 1980). these descriptions reveal strategy as an organisational-wide construct, which affects the direction and activities of the board and all staff in response to the competitive environment, in pursuit of a goal (profit). 2.1 assumptions underlying strategy strategy must take into account a complex set of internal and external conditions and so in this section, we seek to understand what assumptions are used to clarify these. assumptions are a useful tool for mapping out causal relationships related to environmental and industry forces (schwenk 1988). in doing so however, it must be remembered that strategists and others who engage in the strategy-setting process hold preconceived biases, built on previous experiences, which serve to expedite decision-making constrained by bounded rationality. these biases must also be identified as they affect the assumptions that are identified (schwenk 1988). assumptions increase our shared understanding of why organisations engage in development of strategy, orientate collective efforts made in setting a strategic direction and link the external environment to internal organisational strategy (durand et al. 1996). assumptions are also made explicit in order to ensure appropriate use of the resulting strategic construct and ensure its validity (roberts et al. 2012). assumptions are required because decisions are made without australasian journal of information systems horne, maynard & ahmad 2017, vol 21, research article organisational information security strategy 4 complete certainty and so these gaps in knowledge are accounted for using assumptions (schwenk 1988). 2.2 information security strategy: plan or process? there are two main conceptualisations espoused by organisational scholars when describing issio. these include (1) a static plan, described as an artefact to be shared amongst stakeholders (beebe and rao 2010; bowen et al. 2006; von solms and von solms 2004), and (2) a dynamic process, to be followed by stakeholders concerned with protecting organisational information (booker 2006; brotby et al. 2006; flores et al. 2014; mcfadzean et al. 2006; sveen et al. 2009; van niekerk and von solms 2010). a profound comprehension of these interpretations will shed light on how issio should be understood. some information systems researchers view issio as a static plan: a central artefact to be developed that describes the linkages between various organisational concepts such as goals, policies and action sequences (baskerville and dhillon 2008; beebe and rao 2010). in a process orientation, issio involves using a strategy-setting process, whilst incorporating the organisational information systems security goals, such as regulatory compliance, as input. this strategy-setting process can group actions taken according to either the end product ultimately derived such as a strategic security plan, or the processes required such as aligning issio with organisational strategy (baskerville and dhillon 2008). finally some information systems scholars do not conceptualise issio at all or characterise it in abstract terms only (hong et al. 2003; park and ruighaver 2008). 3 information security strategy in information systems research a number of information systems researchers have made individual contributions towards understanding issio from various perspectives. the focus of these researchers was to address problems including adequate support for organisational strategic vision, information systemsbusiness cohesiveness and coordination of information security efforts. however, a complete and methodical evaluation of issio within the information systems literature has not been accomplished. therefore our research seeks to firstly examine what information systems researchers have analysed about the issio construct and secondly the issio nomological network describing its various elements. the issio construct denotes the theoretical domain of issio, specifically how it is conceptualised, at what levels of analysis it can be stratified, and measurement proposals to ensure unit specificity. the issio nomological network refers to our understanding of issio phenomena in the information systems domain, captured through the completion of a thematic analysis. 3.1 literature review our initial search for information security strategy was for manifestations of it in peerreviewed information systems journals and selected conference proceedings, found through searching institutional repositories, google scholar and a* information systems journals. our search consisted of articles that included the complete search string “information security strategy” in english. we searched backwards to discover prior articles and forwards for articles that cited seminal articles (webster and watson 2002). we did not restrict the search based on article age or grade of journal, preferring instead to examine each artefact found for nuances, no matter how small, which could shed light on our evolving understanding of the concept. we also included papers that referred to “information security” but included the word strategies (plural) instead, to facilitate an investigation for example into whether use of the singular ‘strategy’ or plural ‘strategies’ could indicate a shift in level of analysis within an organisation. finally, we included papers that centred on information security but discussed an implicit aspect of strategy. note that ‘organisation’ is a term used to denote private companies, public governments, not-for-profit societies and educational institutions. we included an international standard on information security (iso/iec 2013a), as we thought this could have important implications for motivating the use of an issio. however we did not include any practice-oriented literature such as vendor white papers due to issues australasian journal of information systems horne, maynard & ahmad 2017, vol 21, research article organisational information security strategy 5 with accessibility and peer-review process. in total, our literature review consisted of 45 papers. we then examined each paper to explore how issio relates to the article’s core paradigm. the following four classifications stratify how central issio is to each paper and is adapted from roberts et al. (2012): 1. implicit use of the term. information security forms the paper’s central theme and strategy is implicit only. information security strategy does not form the central argument of the paper, e.g. (van niekerk and von solms 2010). 2. provides conceptual support. papers use information security strategy to support the development of their concepts, e.g. (flores et al. 2014). 3. used in the research question or hypothesis. papers use information security strategy explicitly in their findings or analysis, e.g. (posthumus and von solms 2004). 4. forms the conceptual base for the paper. these papers are entirely consumed with the discussion of information security strategy, e.g. (baskerville and dhillon 2008). in summary, thirty five percent of articles (16 papers) that were collected implied some aspect of issio when discussing information security. twenty seven percent (12 articles) provided theoretical or conceptual support for developing the logic of issio. eighteen percent (8 articles) used issio in some part of their hypothesis, research question or proposition. one fifth of articles (9 papers) were focussed purely on discovery of aspects relating to issio. in the next section, we discuss the role of issio in information systems research in more detail. 3.2 the information security strategy construct from the previous sections, it could be perceived that issio has not been widely developed in the information systems literature so a more profound analysis is warranted. the following sections discuss in more detail issio’s (1) conceptualisation, (2) levels of analysis and (3) measurement domain. 3.2.1 conceptualisation we examined the distinctions that academics have made in their conceptualisation of the issio construct. the three conceptualisations used for this construct are firstly plan, secondly process, and thirdly neither of these. table 1 presents some conceptualisations (i.e. plans, processes, or neither conceptualisation) and the role of issio in the information systems literature. out of the forty five articles that were examined, twenty percent (9 papers) used issio as the core of the entire article. seventy eight percent (35 papers) gave neither explicit conceptualisation of issio. in terms of patterns, when issio is used in the research question (row 3) or forms the theoretical basis for the paper (row4), it becomes apparent that issio is largely viewed by information systems authors as neither plan nor process. plan process neither plan nor process total 1. implicit use of the term 1 1 14 16 2. provides conceptual support 1 3 8 12 3. used in research question or hypothesis 0 1 7 8 4. forms theoretical basis for paper 1 2 6 9 total 3 7 35 45 table 1. information security strategy conceptualisations and role in information systems research australasian journal of information systems horne, maynard & ahmad 2017, vol 21, research article organisational information security strategy 6 3.2.2 levels of analysis for the purposes of clarification, in this paper a group is a set of individuals who are responsible for some aspect of security within an organisation. also, in this section where a paper discusses aspects of responsibility for the application of issio at two different levels, the higher of the two was recorded for the purpose of this analysis. this is because the higher level is seen to be more complex, with greater relationship interdependencies. table 2 shows that while issio is acknowledged to be a multilevel construct, researchers (with only 3 from 45 papers, or 7 percent) do not typically characterise issio from an individual perspective. a significant 60 percent (27 from 45 papers) of the information systems literature examined contend that issio belongs at an organisational level. at an organisational or interorganisational level, it is apparent (with 35 from 45 papers, or 78 percent) that scholars believe issio is neither plan nor process. plan process neither plan nor process total 1. individual 0 0 3 3 2. group 0 1 7 8 3. organisation 3 5 19 27 4. inter-organisational 0 1 6 7 total 3 7 35 45 table 2. information security strategy conceptualisations and levels of analysis 3.2.3 measurement domain when operationalising issio, if conceptual elements cannot be measured, then their reliability cannot be known. there are eight papers in the information systems literature that use the term 'information security strategy’ and expand the theoretical base of issio. of these, seventy five percent (6 from 8 papers) contend that issio exists at an organisational level. half of these (4 from 8) papers hold that issio is neither a plan nor a process. a number of these papers confusingly use the word ‘measure’ as an abbreviation for ‘countermeasure’, which is a control installed to mitigate the risk arising from a threat to an asset (ahmad et al. 2014b; beebe and rao 2009; park and ruighaver 2008). two papers contained no mention of ‘measure’ at all (hong et al. 2003; kayworth and whitten 2010). of the three papers that addressed the measurement of some aspect of issio, the main areas which were measurable included risk management, goal achievement and quality. risk management can be measured by efficacy, efficiency or effectiveness (baskerville and dhillon 2008). time can be a primary measure of risk (baskerville et al. 2014) or alternatively an examination of a finite set of risk-reducing countermeasures can be measured (beebe and rao 2010). goal achievement is measured by the activities undertaken to achieve those goals (baskerville and dhillon 2008). quality improvement can be gained through the measuring of routine security tasks (baskerville et al. 2014). 3.3 the information security strategy nomological network in this section we undertake a thematic analysis within the information systems literature to conceptualise issio at various levels within an organisation and develop a nomological network map to explain the construct and its interrelationships. thematic analysis is a common technique that has been used by other researchers to examine large bodies of work within the information systems literature (leidner and kayworth 2006; roberts et al. 2012). thematic analysis is the process of conducting a qualitative content analysis on the literature of interest then listing meritorious ideas from each article before organising them into related groups (cline and jensen 2004). to conduct the thematic analysis, we first analysed forty five papers for their interpretation of issio and then grouped key constructs according to similarities of themes. this resulted in three distinct theme groups emerging from the analysis, which were antecedents, constituents and yields. australasian journal of information systems horne, maynard & ahmad 2017, vol 21, research article organisational information security strategy 7 antecedents are the precursor conditions that might prompt an organisation to consider the use of an issio. examples include governments with top secret files, pharmaceutical companies conducting extended clinical new drug trials and banks facilitating online trading. constituents are the elements that make up the core of an issio, to be adopted by an organisation seeking to protect its information. examples include risk management process to understand persistent common threats, security auditing to satisfy external regulators and governance activities to align organisational efforts. yields are the benefits that can be enjoyed after successfully adopting issio. examples include the confidentiality, integrity and availability of information, and protection of competitive advantage, reputation, and customer trust. based on the thematic analysis and discussion in preceding sections, a logical grouping of the conceptual elements of issio can be elicited from the literature and is shown in figure 1. figure 1. thematic map of information security strategy in organisations in is research the sections below discuss these themes in more detail, based on conceptual elements and an overall understanding of what each paper purports issio to be. 3.3.1 antecedents antecedents are the precursor conditions necessary to prompt the use of issio and emerged as a theme in the information systems literature after conducting a thematic analysis, as described in the previous section. at an individual level, there did not seem to be any antecedents apparent in the literature. it is impossible to make an exhaustive claim about this but perhaps this is an area that warrants further attention from researchers. at a group level, one issio antecedent is the requirement for global ubiquitous information availability and the necessity to distil incomprehensible threat intelligence complexity and volume in a timely fashion to management (booker 2006). at an organisational level, antecedents for issio apparent in the literature included gathering intelligence about the external environment. an organisation’s information security strategic posture involves a dependence on the external threat environment, not the continued successful achievement of organisational goals. the increasing complexity and sophistication of dynamic, targeted attacks over time naturally causes a general shift in posture balance from preventative towards a more response-oriented approach (baskerville et al. 2014). organisational ownership of valuable information assets is also a key driver towards the adoption of issio (kelly 1999). at the inter-organisational level, an issio must take into consideration an organisation’s regulatory compliance burden (banker et al. 2010; kayworth and whitten 2010; tutton 2010). this regulatory compliance-driven approach however only forms part of a holistic approach to security (anderson and choobineh 2008). regulatory and legal compliance along with australasian journal of information systems horne, maynard & ahmad 2017, vol 21, research article organisational information security strategy 8 adoption of standards and best practices is also required (iso/iec 2013a; posthumus and von solms 2004). examination of the industry in which the organisation competes and sufficient knowledge of industrial and economic considerations of an organisation’s competitive landscape are also required (baets 1992). the existence of a strategic information systems plan is notable, as it dictates the formulation of the information security policy by providing essential details of the business context or competitive landscape (doherty and fulford 2006). failure of political pressure and economic sanctions are important preconditions that may motivate the commencement of information warfare (baskerville 2010). issio is primarily based on prevention of incidents arising from advanced persistent threats (apt) using technical controls against external threats that are seen to be increasingly more frequent, novel and costly (beebe and rao 2009). environmental and organisational conditions, managerial understanding and actions, quality improvement initiatives and organisational achievement lead to use of issio (cline and jensen 2004). regulatory, political and legal compliance plus adoption of standards and best practices motivate the use of issio (kim et al. 2012; posthumus and von solms 2004). standards exist which detail management of information security which in turn could assist with issio development (brotby et al. 2006; iso/iec 2013b). 3.3.2 constituents constituents are the central conceptual elements of issio and emerged as a theme in the information systems literature after conducting a thematic analysis. individual level this section seeks to explore what role an individual has in contributing towards the overall success of the strategic use of information security. at an individual level, there were no constituent elements of issio however this seems unusual because it is widely accepted that overall security depends on the weakest link which is typically the individual. this may represent an opportunity for further research. group level this section examines the is literature to discover the dynamics of groups working to support the strategic use of information security. at a group level, the constituent components of the issio construct are varied and numerous. one is the identification and protection of knowledge assets, which can be resources forming a competitive advantage and can be either held in the human brain or in organisational documents, routines, procedures and practices. knowledge leakage is a security incident which can temporarily affect an organisation’s competitive advantage and affect its reputation, revenue streams, remediation costs and productivity. mitigation or protection of knowledge is achieved through initial classification of information assets, then compartmentalisation, development of technical solutions, policies, procedures, culture and legal support (ahmad et al. 2014a). issio should guide the overall security budget for an organisation, to enable the security staff group and their management to fund and implement security resources that optimise security outcomes based on expense versus benefits (anderson and choobineh 2008). issio includes the examination of stratified responsibility within an organisation that cohesively achieves overall information systems security. decisions made by one layer of responsible agents affect decisions made by agents in other layers and their communication is vital. issio success depends on action taken by responsible agents rather than technological controls. achievement of issio allows alignment with policies and regulatory compliance efforts (backhouse and dhillon 1996). an essential element of issio is a mix of technical, formal and informal controls to ensure regulatory compliance, protect the it infrastructure that the information resides on and deliver confidentiality, integrity and availability (cia) to users (beebe and rao 2009; posthumus and von solms 2004; sveen et al. 2009). security education, training, awareness and constant monitoring are required to ensure employees can use controls (taylor and robinson 2014; van niekerk and von solms 2010). issio includes the capability to respond to attacks effectively, which stems from supplementary forces creating a time buffer through the employment of defence-in-depth design to allow the responding forces enough to time to deploy to the breach australasian journal of information systems horne, maynard & ahmad 2017, vol 21, research article organisational information security strategy 9 from the central holding point (burnburg 2003). information systems solutions underpin business products and services and are therefore critical in maintaining an organisation’s competitive advantage. an issio must focus on how to maintain competitive advantage in the face of rapidly changing ict infrastructures (cegielski et al. 2013). organisational level the organisational level is where most influence can be exerted internally to achieve success in supporting an externally-focussed strategic application of information security and deserves special attention in our examination of the is literature. at an organisational level, issio can be used to incrementally improve the quality of the information security program. there must be a strong link from the issio to the business strategic plan to support it. issio is necessary to prevent threats to an organisation’s information (ahmad et al. 2014b). it supports incremental quality improvement, alignment with agency mission, and awareness and monitoring of external threats (bowen et al. 2006; johnson and goetz 2007). issio protects only the more valuable information assets in order to reduce expenditure. this is achieved through policies and communication structures, director-level sponsorship of security initiatives, measuring success and administering sanctions for security policy violations. identity and access management is important to overall success as is security incident detection and response activities (ahmad et al. 2012; kelly 1999). corporate knowledge assets can then be inventoried and values defined (baets 1992). if the labour involved with security functions is outsourced to other companies or individual contractors, then they need to equally adhere to the security policies and strategy adopted by the parent organisation (baskerville et al. 2014). issio can use situational crime prevention (scp) to introduce a deterrent option within the risk management section (beebe and rao 2010). it is centred in risk management, identifying controls to mitigate known threats (da veiga and eloff 2007). reducing risk lowers anticipated loss, which changes an organisation’s security posture. quantifying risk of anticipated loss requires recording of previous loss from security incidents (ryan and ryan 2006). conceptual constituents also include regulatory compliance, teleworkers, organisational agility, business justification requirements, reactive quality improvement and community cloud initiatives (booker 2006). the external environment places various demands on the organisation which changes in order to continue the achievement of the organisational objectives. the issio is contingent on the environment when changing to maintain focus on the organisational objectives (hong et al. 2003). issio uses governance to provide boundaries and procedures for employees along with their roles and responsibilities and considers the organisation’s risks and culture, performance and assurance, seta, suppliers and customers (brotby et al. 2006; hinde 2002). information security strategy is built on it products and solutions but extends to include the employees in the business. specifically issio integrates director-level security sponsorship and hierarchical structures that provide security governance (kayworth and whitten 2010). issio requires the attention and support of the board of directors and ceo because they are accountable for its outcomes. they affect issio by using corporate governance, specifically a corporate information security policy, as a tool to communicate with and direct management in the organisation. two-way communication is then required back from management to the board and executive in the form of regular progress reports (iso/iec 2013b; mcfadzean et al. 2006; posthumus and von solms 2004; vroom and von solms 2004). issio must consider corporate governance and provide those responsible for security with autonomy (von solms and von solms 2004). issio constituents include risk management components such as disaster recovery and business continuity, insurance, audits and new business units and groups (cline and jensen 2004). without a focus on business continuity, it is entirely possible that in the event of an ict infrastructure disaster a lack of business continuity translates directly into quantifiable revenue loss (van der haar and von solms 2003). information security strategy needs to focus on people and process not tools, as these are the main causes of security failure (da veiga and eloff 2010). issio is preventative in nature and australasian journal of information systems horne, maynard & ahmad 2017, vol 21, research article organisational information security strategy 10 seeks to protect against rational individuals perpetrating attacks rather than automated technical attacks. the preventative approach relies heavily on deterrence and advocates that effectiveness is derived from sanctions being believed to be swift, severe and certain (d'arcy and herath 2011). inter-organisational level the inter-organisational level of information security is where organisational benefits can potentially be mutually shared by contributing organisations for their individual success and factors that influence this are examined in the following section. at an inter-organisational level, compliance must be audited and a firm’s auditing costs, incurred through engagement with an external auditor, can be lowered through a focus on it assurance. this it assurance includes high-quality it documentation and an emphasis on systems security which lowers the cost because it makes the work of an auditor easier and quicker, therefore considerably lowering the time and materials auditing cost (banker et al. 2010). issio facilitates information warfare, which forms just one layer of a conflict with an adversary. the four layers of a nation attack are political, which then escalates to economic sanctions, then information warfare and finally full kinetic warfare (baskerville 2010). some information assets may be resources that create strategic competitive advantage for organisations. if these lose their confidentiality through a security incident, then their integrity may be lost forever, along with the value of the advantage. when a security incident of this nature is disclosed to the market, there are implications for the organisation’s share price (campbell et al. 2003). issio is the process of dynamically assessing customer perceptions of the organisation’s online transactions, with a view to increasing the security of transactions in order to prevent a decrease in brand trust in the marketplace. regulatory pressures have increased the requirement for this defensive process (datta and chatterjee 2008). issio must include an organisation’s business and policy cyber considerations and depends on the political environment in an organisation’s country of origin, which must synchronise with that of governments from other countries. the legal frameworks in various countries must harmonise globally to allow prosecution in the event of an attack. shouldering the responsibility for lowering attacks will involve constitutional examination for potential conflicts, a willingness to collaborate and a system for measuring attacks however the benefits are that the world will be a safer place (kim et al. 2012). 3.3.3 yields yields are the goals achieved from the successful use of issio and emerged as a theme in the information systems literature after conducting the thematic analysis described in section 3.3. at an individual level, there were no apparent benefits arising from issio, nor were there any apparent at a group level of analysis. at an organisational level, the security goals are to ensure knowledge assets’ confidentiality, integrity and availability (ahmad et al. 2014a). another yield is that high quality information is made readily available to managers as they require it (doherty and fulford 2006). prevention of potential losses is an objective but depends on the volume of organisational information assets, business continuity capabilities, profitability, threat intelligence and risk appetite. security budgets to achieve this prevention should be bounded by expected probable losses (anderson and choobineh 2008). loss prevention efforts should also guard against revenue loss (van der haar and von solms 2003). performance reporting is another goal but requires tracking of key kpis including systems, assigned assets, people, processes, compliance and auditing and customer service (booker 2006). finally, the protection of competitive advantage is an obvious goal (cegielski et al. 2013). at an inter-organisational level, issio yields can include the misdirection of an adversary’s attack assets, even from other nation-states, to protect information assets and physical critical infrastructure assets. yields can also include the disablement of adversary critical infrastructure, reduce foreign military abilities and impair foreign government operations australasian journal of information systems horne, maynard & ahmad 2017, vol 21, research article organisational information security strategy 11 (baskerville 2010). issio can also lower the risk of adverse litigation outcomes and achieve information confidentiality, integrity, availability, authenticity and non-repudiation (brotby et al. 2006). an important benefit is share price protection (campbell et al. 2003). regulatory compliance avoids adverse sanctions by ensuring external agencies are kept fully informed (banker et al. 2010). issio yields also include retaining customers, security incident prevention, improved business processes and public reputation (cline and jensen 2004). failure to implement an issio sensibly may result in estranged customers and tarnished reputation (datta and chatterjee 2008; oshri et al. 2007). 3.3.4 key findings of thematic analysis a number of gaps in knowledge have appeared through the conduct of this research. at an individual level of analysis, there appears to be very little research conducted into the role of an individual when supporting issio. there appear to be many contributors to various aspects of issio but there does not seem to be any one unified conceptualisation or theory. information security is managed at an organisational level but must include an assessment of inter-organisational level factors to take advantage of the potential yields. table 3 presents a thematic map of issio derived from the results of the literature review, as described in the previous sections, and summarises the key themes found. antecedents issio constituents yields inter-organisational regulatory compliance industrial and economic factors political and economic factors political and legal factors external threat environment standards inter-organisational regulatory compliance information warfare information asset protection environment scanning inter-organisational foreign adversary impairment litigation risk management share price protection regulatory compliance public reputation customer trust organisational valuable information intelligence organisational boardroom accountability quality improvement information asset management labour source risk management organisational agility governance business continuity people and process incident prevention policy organisational confidentiality, integrity and availability probable loss mitigation performance reporting competitive advantage protection group ubiquitous information availability group knowledge leakage prevention security budget responsibility controls incident response ict infrastructure group none individual none individual none individual none table 3. thematic map of results from literature review of issio the antecedent column can clustered into two main groups, which are internal and external to the organisation. internal antecedents include valuable information and its ubiquitous availability to management for use in daily operations. external antecedents at the interaustralasian journal of information systems horne, maynard & ahmad 2017, vol 21, research article organisational information security strategy 12 organisational level include all the constraints being placed on the organisation, such as regulatory compliance and the external threat environment. in the constituent column, there are a number of variables that can be altered to operationalise issio according to an organisation’s unique requirements. internal activities such as applying controls to protect information assets from threats are conducted from the board down, even including the activities that have external requirements at the inter-organisational level such as environment scanning. it is important to note that all activities in the constituent column are internally controlled, even the inter-organisation level ones. in the yields column, external benefits are largely intangible, such as the enhancement of customer trust, and internal benefits are largely tangible, such as protection of competitive advantage. this analysis is summarised in table 4: antecedents issio constituents yields external constraints internal information threats / information / resources / controls external intangible internal tangible table 4. analysis of results from literature review of issio 4 conclusion this literature review illustrates various aspects of issio and key themes were explored and grouped. yet, there is no single, well-developed conceptualisation apparent in the literature that comprehensively explains the issio construct and its relationships. additionally, information security is ostensibly lacking to a large extent from the strategic organisational literature and even from strategic information systems literature. a paradigm shift is required to move from internally-focussed protection of organisation-wide information towards a strategic view that considers an organisation’s environment. the following section offers suggestions to address these gaps through the conduct of future research, which could include positing a framework or model to allow information systems researchers to understand issio more fully. 4.1 contribution based on our review and a cumulative research tradition, we now construct a definition proposing the meaning of information security strategy in organisations: “information security strategy is an organisational-level model of conceptual elements, which is motivated by antecedent conditions that balance internal information needs and external environmental factors, to yield information security benefits to the organisation.” 4.2 limitations of research into information security strategy the issio construct developed so far is potentially of great benefit to organisations seeking to adopt an overall strategy for their information security. we understand firstly, the precursor conditions which when met, cause organisations to consider the use of issio; secondly, the constituent elements of an issio for operationalisation; and thirdly, the benefits that can be enjoyed by an organisation upon successful implementation. given that, we still have limitations impeding our understanding of issio. these are described in the next section. first, a significant amount of research conceptualises issio as a plan, which identifies the construct as a static document, bereft of dynamic processes to ensure its validity when responding to immediate changes in the external environment. this gives rise to construct validity issues as having a plan is important, but not a precondition for an organisation to vary its issio based on persistent incident detection and response (straub et al. 2004). second, the information systems literature contains analysis on issio from various levels within an organisation, largely focusing on the organisational perspective. this stratified perspective has its own properties and varies from an inter-organisational level, for example australasian journal of information systems horne, maynard & ahmad 2017, vol 21, research article organisational information security strategy 13 in terms of complexity and focus on external factors. therefore, the nomological network of terms will be different for each level. third, measurement issues arose in our study when we found that information systems researchers either did not adequately explain the dimensions with which to measure the elements of the issio construct at each level or defined theoretical measures for one level and then operationalised them at another (baskerville and dhillon 2008). additionally, tangible aspects of issio such as the use of technical controls were perceived to be very measurable through reporting but intangible aspects such as employee attitudes towards security less so. 4.3 future research directions in addition to conducting further research on the gaps identified in this paper, there are several prospects for information systems researchers to develop the body of knowledge that currently exists on issio. answers to these questions have implications for practice. this study provides the impetus hopefully for future research into issio, strategic information systems and organisational strategy. first, military strategy has influenced business management theory in many ways. how can military strategy contribute to our understanding of issio? what aspects of warfare, including embodying any supporting theory for example possibility theory, are pertinent to issio? second, given the strong links from issio to organisational strategic theory apparent in the literature, what lessons does business strategy have for issio? how can issio be integrated with business strategy? how does issio relate to it strategy? is there a dependence on issio to achieve organisational success, and if so, how is this success defined? what other preconditions would prompt an organisation to strategically consider the use of issio? are there avenues to generate additional competitive advantage through issio? are there differences in issio between public and private sectors? when organisations write an issio, do they have the capability to implement it? third, information systems researchers could generate a framework or model to explain the phenomena that collectively form the issio construct. what are the constituent elements of issio and how do these relate to each other? how would issio be operationalised within an organisation? to what extent will compliance culture influence the effectiveness of issio operationalisation (shedden et al. 2010; tan et al. 2010)? how does issio relate to strategic information systems? how does issio relate to organisational strategy? what is the role of the individual level in issio? how do levels of analysis apply in the digital realm? finally, there are a number of information systems scholars who have researched the theory underlying issio, including for example deterrence, prevention, surveillance, detection, response, deception, perimeter defence, compartmentalisation and layering (ahmad et al. 2014b; beebe and rao 2009; d'arcy and herath 2011). what would further analysis of these theories reveal about issio? what does systems theory have to offer issio? could a theory on information security be developed? what do the theory of information warfare or the theory of protection motivation offer issio (denning 1999; rogers 1975)? references ahmad, a., bosua, r., & scheepers, r. (2014). protecting organizational competitive advantage: a knowledge leakage perspective. computers & security, 42, 27-39. ahmad, a., hadgkiss, j., & ruighaver, a. b. (2012). incident response teams–challenges in supporting the organisational security function. computers & security, 31(5), 643-652. ahmad, a., maynard, s. b., & park, s. (2014). information security strategies: towards an organizational multi-strategy perspective. journal of intelligent manufacturing, 25(2), 357-370. anderson, e. e., & choobineh, j. (2008). enterprise information security strategies. computers & security, 27(1), 22-29. australasian journal of information systems horne, maynard & ahmad 2017, vol 21, research article organisational information security strategy 14 backhouse, j., & dhillon, g. (1996). structures of responsibility and security of information systems. european journal of information systems, 5(1), 2-9. baets, w. (1992). aligning information systems with business strategy. journal of strategic information systems, 1(4), 205-213. banker, r., chang, h., & kao, y.-c. (2010). evaluating cross-organizational impacts of information technology – an empirical analysis. european journal of information systems, 19(2), 153-167. baskerville, r. (2010). third-degree conflicts: information warfare. european journal of information systems, 19(1), 1-4. baskerville, r., & dhillon, g. (2008). information systems security strategy: a process view. in d. w. straub, s. e. goodman, & r. baskerville (eds.), information security: policy, processes, and practices. advances in management information systems (pp. 15-45). armonk, ny: m. e. sharpe. baskerville, r., spagnoletti, p., & kim, j. (2014). incident-centered information security: managing a strategic balance between prevention and response. information & management, 51(1), 138-151. beebe, n. l., & rao, v. s. (2009). examination of organizational information security strategy: a pilot study. paper presented at the amcis 2009 proceedings. beebe, n. l., & rao, v. s. (2010). improving organizational information security strategy via meso-level application of situational crime prevention to the risk management process. communications of the association for information systems, 26(17), 329-358. booker, r. (2006). re-engineering enterprise security. computers & security, 25(1), 13-17. bowdish, r. g. (2013). military strategy: theory and concepts. (doctor of philosophy phd), university of nebraska, lincoln, nebraska. (26) bowen, p., hash, j., & wilson, m. (2006). sp 800-100. information security handbook: a guide for managers. brotby, w., bayuk, j., & coleman, c. (2006). information security governance: guidance for boards of directors and executive management: illinois, it governance institute. burnburg, m. k. (2003). a proposed framework for business information security based on the concept of defense-in-depth. (master's thesis), university of illinois, springfield. campbell, k., gordon, l. a., loeb, m. p., & zhou, l. (2003). the economic cost of publicly announced information security breaches: empirical evidence from the stock market. journal of computer security, 11(3), 431-448. cegielski, c. g., bourrie, d. m., & hazen, b. t. (2013). evaluating adoption of emerging it for corporate it strategy: developing a model using a qualitative method. information systems management, 30(3), 235-249. cline, m., & jensen, b. (2004). information security: an organizational change perspective. paper presented at the amcis 2004 proceedings. d'arcy, j., & herath, t. (2011). a review and analysis of deterrence theory in the is security literature: making sense of the disparate findings. european journal of information systems, 20(6), 643-658. da veiga, a., & eloff, j. h. p. (2007). an information security governance framework. information systems management, 24(4), 361-372. da veiga, a., & eloff, j. h. p. (2010). a framework and assessment instrument for information security culture. computers & security, 29(2), 196-207. australasian journal of information systems horne, maynard & ahmad 2017, vol 21, research article organisational information security strategy 15 datta, p., & chatterjee, s. (2008). the economics and psychology of consumer trust in intermediaries in electronic markets: the em-trust framework. european journal of information systems, 17(1), 12-28. denning, d. e. r. (1999). information warfare and security (vol. 4): addison-wesley reading ma. doherty, n. f., & fulford, h. (2006). aligning the information security policy with the strategic information systems plan. computers & security, 25(1), 55-63. drucker, p. f. (1958). business objectives and survival needs: notes on a discipline of business enterprise. the journal of business, 31(2), 81-90. durand, t., mounoud, e., & ramanantsoa, b. (1996). uncovering strategic assumptions: understanding managers' ability to build representations. european management journal, 14(4), 389-398. flores, w. r., antonsen, e., & ekstedt, m. (2014). information security knowledge sharing in organizations: investigating the effect of behavioral information security governance and national culture. computers & security, 43, 90-110. hinde, s. (2002). security surveys spring crop. computers & security, 21(4), 310-321. hong, k.-s., chi, y.-p., chao, l., & tang, j.-h. (2003). an integrated system theory of information security management. information management & computer security, 11(5), 243-248. iso/iec. (2013a). iso/iec 27001:2013 information technology — security techniques — information security management systems — requirements. geneva, switzerland: international organization for standardization and international electrotechnical commission. iso/iec. (2013b). iso/iec 27014:2013 information technology — security techniques — governance of information security. geneva, switzerland: iso/iec. johnson, m. e., & goetz, e. (2007). embedding information security into the organization. 3, 16-24. kayworth, t., & whitten, d. (2010). effective information security requires a balance of social and technology factors. mis quarterly executive, 9(3), 163-175. kelly, b. j. (1999). preserve, protect, and defend. the journal of business strategy, 20(5), 2225. kim, s. h., wang, q.-h., & ullrich, j. b. (2012). a comparative study of cyberattacks. communications of the acm, 55(3), 66-73. leidner, d. e., & kayworth, t. (2006). review: a review of culture in information systems research: toward a theory of information technology culture conflict. mis quarterly, 30(2), 357-399. mcfadzean, e., ezingeard, j.-n., & birchall, d. (2006). anchoring information security governance research: sociological groundings and future directions. journal of information system security, 2(3), 3-48. mintzberg, h. (1987). the strategy concept i: five ps for strategy. california management review, 30(1), 11-24. oshri, i., kotlarsky, j., & hirsch, c. (2007). information security in networkable windowsbased operating system devices: challenges and solutions. computers & security, 26(2), 177-182. park, s., & ruighaver, t. (2008). strategic approach to information security in organizations. paper presented at the iciss. international conference on information science and security, 2008. australasian journal of information systems horne, maynard & ahmad 2017, vol 21, research article organisational information security strategy 16 porter, m. e. (1980). competitive strategy: techniques for analyzing industries and competitors: new york: free press. posthumus, s., & von solms, r. (2004). a framework for the governance of information security. computers & security, 23(8), 638-646. roberts, n., galluch, p. s., dinger, m., & grover, v. (2012). absorptive capacity and information systems research: review, synthesis, and directions for future research. mis quarterly, 36(2), 625-648. rogers, r. w. (1975). a protection motivation theory of fear appeals and attitude change1. the journal of psychology, 91(1), 93-114. ryan, j. j., & ryan, d. j. (2006). expected benefits of information security investments. computers & security, 25(8), 579-588. schwenk, c. r. (1988). the cognitive perspective on strategic decision making. journal of management studies, 25(1), 41-55. shedden, p., ruighaver, t., & ahmad, a. (2010). risk management standards – the perception of ease of use. journal of information systems security, 6(3), 23-41. straub, d., boudreau, m.-c., & gefen, d. (2004). validation guidelines for is positivist research. the communications of the association for information systems, 13(1), 63. sveen, f., torres, j., & sarriegi, j. (2009). blind information security strategy. international journal of critical infrastructure protection, 2(3), 95-109. tan, t., ruighaver, a. b., & ahmad, a. (2010). information security governance: when compliance becomes more important than security. paper presented at the the ifip tc11 24th international information security conference, brisbane, australia. taylor, r. g., & robinson, s. l. (2014). the roles of positive and negative exemplars in information security strategy. academy of information and management sciences journal, 17(2), 57-79. tutton, j. (2010). incident response and compliance: a case study of the recent attacks. information security technical report, 15(4), 145-149. van der haar, h., & von solms, r. (2003). a model for deriving information security control attribute profiles. computers & security, 22(3), 233-244. van niekerk, j. f., & von solms, r. (2010). information security culture: a management perspective. computers & security, 29(4), 476-486. von solms, b., & von solms, r. (2004). the 10 deadly sins of information security management. computers & security, 23(5), 371-376. vroom, c., & von solms, r. (2004). towards information security behavioural compliance. computers & security, 23(3), 191-198. webster, j., & watson, r. t. (2002). analyzing the past to prepare for the future: writing a literature review. mis quarterly, 26(2), xiii-xxiii. acknowledgements the authors would like to thank the reviewers for their valuable contributions to this paper. this research received funding support from australian government research training program scholarship. copyright: © 2017 horne, maynard & ahmad. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems horne, maynard & ahmad 2017, vol 21, research article organisational information security strategy 17 which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. paper title (paper title style) australasian journal of information systems hunter & radoll 2020, vol 24, research article dynamics of digital diffusion and disadoption 1 dynamics of digital diffusion and disadoption: a longitudinal analysis of indigenous and other australians boyd h hunter australian national university boyd.hunter@anu.edu.au peter j radoll university of canberra abstract the digital divide between indigenous and other australians describes the unequal access to information and communications technology (ict) between these groups. historically, researchers have focused on acquiring new technology, but we argue that it is important to understand all the dynamics of digital usage, including the loss of access to ict within a household. for long-lived technology such as internet access, it is particularly important to consider that retention of access to the technology. this paper conducts a longitudinal analysis of changes in internet usage for indigenous and non-indigenous australian households using the australian census longitudinal dataset, 2006–2016. while earlier work analyses the digital divide in terms of ‘diffusion’ or adoption of ict, this paper shows that the failure to retain internet access is also important in driving the digital divide. the dynamics of the digital divide have important and ongoing implications for addressing broader socioeconomic disadvantages experienced by indigenous australians. the covid-19 pandemic underscores the urgency of policy addressing the digital divide, given the renewed momentum for remote learning and telecommuting. keywords: information and communications technology, digital divide, diffusion, adoption, disadoption, internet access 1 introduction a digital divide is evolving in australian society, with some groups having better access to information and communications technology (ict) than others (davis, mcmaster and nowak, 2002, gurstein, 2004; daly, 2005). radoll (2010) identified that the use of ict is low for indigenous australians compared with non-indigenous australians. the 2006 census demonstrated that 43% of indigenous households had access to the internet, compared with 64% of other households (data accessed using the australian bureau of statistics product tablebuilder). given the social exclusion of indigenous australians, constraints affecting indigenous adoption of ict may have important implications for the ongoing gap in socioeconomic outcomes between indigenous and other australians. in the modern world, access to services and knowledge of opportunities to enhance wellbeing are dependent on access to ict, and hence it is crucial to address the digital divide if gaps in socioeconomic outcomes are to be addressed. while there are crossectional studies of the digital divide, we argue that it is important to understand how access to ict changes over time. ict is inherently dynamic in that the technology and its use vary substantially over time. furthermore, these manifold changes are australasian journal of information systems hunter & radoll 2020, vol 24, research article dynamics of digital diffusion and disadoption 2 driven by innovations that are not entirely predictable. rather than attempting to understand these potentially idiosyncratic factors, we focus on the basic infrastructure and overall access to the internet, which is crucial for most forms of ict. while specific technologies change rapidly, it is possible to conduct a sensible analysis of the dynamics of internet access over a five-year period. radoll and hunter (2017) identified the important role of the processes that lead to the loss of technology or access to technology, which is driven by the failure to reinvest in the face of depreciation, technological or social change, or simply changes in the preferences of people for various forms of technology. in this article we refer to this process as disadoption. note that disadoption does not necessarily imply some sort of agency from households as it could also involve structural (or infrastructural) issues that policy may have neglected in recent years, including public investment and depreciation. this study provides the first analysis of the dynamics of internet access of indigenous australians using a large-scale dataset that combines the responses from the last three censuses. radoll and hunter (2017) used the early release of the australian census longitudinal dataset (acld), which provided information on a substantial number of indigenous people over a five-year period—information from the 2006 census on 14 802 individuals who identified as being indigenous in 2006 was linked with 2011 census records for the same people (identified through probabilistic matching). the 2016 census data has since been added to the acld and this paper provides the first multivariate analysis of the dynamics of internet access for indigenous and non-indigenous households. the remainder of this paper is structured as follows. the next section provides an overview of the current literature on the digital divide and the adoption of ict, with a focus on issues for disadvantaged groups. the data and method used in this paper are then documented followed by a brief descriptive analysis of the dynamics of internet access by remoteness (defined in abs 2011). a multivariate analysis of adoption and disadoption is then conducted using the explanatory factors that are both available in census data and identified as being relevant in the literature review. the 2011–2016 acld release is used to estimate the factors associated with the change in ict access. a conclusion sums up the implications of the work that are particularly significant during, and in the aftermath of, the current covid-19 pandemic. 2 ict adoption and its opposite, disadoption a growing body of evidence demonstrates the benefits of ict adoption to communities, households and individuals. these include access to online services such as government services, educational institutions, electronic health and electronic banking, as well as increased income (curtin 2001; arocena and senker 2003; allyn and yun 2005; daly 2006). internet access is important in the context of the digital divide evident between indigenous and nonindigenous australians because it underpins the ability to adopt and use much of the latest ict. quality, access, coverage and use of ict are critical for participation in australian society. ict forms the basis of much economic activity, and not having access to ict has a clear detrimental economic and social impact. along these lines, radoll (2010) shows that some indigenous individuals and households may be excluded from ict access because of location, education, economic position or culture. australasian journal of information systems hunter & radoll 2020, vol 24, research article dynamics of digital diffusion and disadoption 3 the term ‘adoption’ is used in the information systems discipline to describe the uptake of ict. specifically, adoption pertains to the ‘decision to make full use of an innovation as the best course of action available’ (rogers 1995:21). one relevant contribution in the context of this paper is rice and katz (2003) who examined the digital divide in internet and mobile phone usage in terms of adoption and dropouts (also, see katz and rice 2002; rice and pearce 2015). another widely used term in the literature is ‘diffusion’: ‘the process by which an innovation is communicated through certain channels over time among the members of a social system’ (rogers 1995:5). more formally, rogers refers to the diffusion of innovations theory (doi). there are several other theories that may help explain the digital divide between indigenous and non-indigenous australians (radoll, 2010): theory of reasoned action (tra; ajzen and fishbein, 1980), the theory of planned behaviour (tpb; ajzen, 1991), the model of adoption of technology in households (math; venkatesh and brown, 2001), the technology acceptance model (tam; davis, 1989), and the unified theory of acceptance and use of technology (venkatesh, morris, davis and davis, 2003). radoll (2010) also argues that structuration theory of giddens (1984) has important implications for understanding the digital divide. theories such as doi and tam postulate that perceived ease of use and usefulness are key to adoption, while other theories such as tra and tpb rely on behaviour and beliefs, which are independent of the ‘perceived outcome’ of use of the technology (compeau, higgins and huff, 1999). while many factors are found to affect household adoption of ict in society (venkatesh and brown, 2001, venkatesh et al. 2003), research is relatively scarce in explaining the low ict adoption by indigenous australian households. there is also little research about how such adoption may change over time. the approach adopted in this paper is to take a step back from the specific use of a particular ict and focus on the more fundamental questions surrounding the access to its basic infrastructure. we do not deny the importance of evolving use of ict, but the internet has been around for almost 30 years and is no longer in itself a new technology. rather, access to the internet underpins the ability to adopt new technology that may be developed. while diffusion theory is broadly relevant to understanding connection to the internet, some socioeconomic issues need to be considered. the cost of the provision of internet infrastructure is likely to be higher where the cost of living is higher—for example, in remote areas and nonurban areas that are more distant from the major centres of population. any good or service that is not locally produced, including most ict installation and maintenance, will be more expensive because of high transport and fuel costs. in contrast, the ability to maintain internet infrastructure is likely to be associated with the resources available and the incentives to maintain the infrastructure in the face of technological change, changing community norms about adequate internet access and speed, depreciation, and natural wear and tear. economists believe that incentives to maintain infrastructure are largely driven by who provides infrastructure or owns the infrastructure (shilling, sirmans and dombrow, 1991). the parties who control decisions to maintain the internet are not necessarily the same people who derive benefit from the access. if people access the internet at work, the employer and worker costs and benefits of internet use need to be taken into account. if the internet is provided as part of a community resource funded by a local organisation or government agency, overuse, often associated with common property australasian journal of information systems hunter & radoll 2020, vol 24, research article dynamics of digital diffusion and disadoption 4 resources, needs to be considered. the key issue here is whether there is private or public ownership of infrastructure, and the incentives of users and providers to maintain the internet services in good working order. hence, internet access can diminish over time if the original funding agency does not adequately reinvest to maintain the infrastructure in working order or the users do not exercise due care in looking after the equipment provided. even if a political case is made that internet connectivity should be provided to the indigenous community at a particular point in time, policy needs to take into account who has the incentive to keep the infrastructure in good working order. the internet infrastructure is, by definition, very technical, so if it breaks down considerable expertise is required; it is probable that suitable expertise may not be locally available. we know from research that the prominent indicators where the digital divide gap is widening most includes those with employment, better incomes and higher education van dijk (2005). ethnicity also plays a role in determining the digital divide (van dijk 2005). the ict diffusion literature, and information systems literature more generally, tend to focus on the adoption phase of technology. this is understandable because they are attempting to analyse and explain the take-up of innovations that are, by definition, new. however, once innovations are adopted, they need to receive ongoing investment to maintain their usefulness, this concept is technology maintenance. technology maintenance predicts that “as the poor increasingly have initial in-home and public access to technology, the digital divide will begin to centre on differences in the ability to maintain that access” (gonzales, 2016, p235). hence, adoption is only one part of the story; in terms of the dynamics of the digital divide, we need to understand the outcomes and processes associated with disadoption, where households who had access to the internet lose their access to the internet over time. van dijk (2005) argues that when it comes to the digital divide there are broad consequences to consider, stating that we should consider other aspects of the digital divide beyond the more popular motivational aspects. his belief is that access to the internet is essential for people to participate in both the economy and society fully. van dijk (2005) highlights clearly that the digital divide is generally related to other societal gaps such a poverty and inequality in education. gonzales (2014) demonstrates the importance of the stratification model in terms of addressing the digital divide. van dijk (2005) stratification model states that physical access issues persist even when ownership challenges are addressed. understanding the reasons for disconnection is important as gonzales (2016) highlights, these reasons include broken hardware, difficulty in paying bills, and having no or limited access to public internet services. one interesting aspect that gonzales (2014) highlights is that poorer segments of the community are more likely to be “ill and live in fear of crime than wealthier people” (p. 235). having access to technology for this segment of the population can at times be critical. mobile phones or cell phones are also an important consideration in the discussion of the digital divide. there is an increasing reliance of small mobile devices on expensive mobile internet plans. additionally, small devices are often limited in function compared to having access to having access a large screen device in the home (gonzales, 2014). nevertheless; increasingly people living in higher income countries have been cancelling landline services and have moved to mobile-only services (gonzales, 2014). van dijk (2005) argues that the people adopting technology/internet at a higher rate tend to have higher incomes, better education, are younger and are white. gonzales (2017) also australasian journal of information systems hunter & radoll 2020, vol 24, research article dynamics of digital diffusion and disadoption 5 demonstrates that ethnicity plays are role in digital communications arguing that people of the same race and gender are likely to exchange messages whether that is via email or instant messaging. she also believes that in lower income communities this communication can in fact bring neighbourhoods closer together through building new social ties. the building of community ties through the internet may be particularly important for those who are socioeconomically challenged (gonzales 2017). for example, mesch (2012) argues that ‘minorities and immigrants will be more likely to use computer-mediated communication to compensate for their lack of cultural capital’ (p468). 3 data and method census questions are usually asked at a point in time and reported only as cross-sectional data. the acld is an important development for the australian bureau of statistics. a 5% random sample of the 2006 census was linked with the 2011 census using data linkage techniques to create the acld. the acld represents less than 5% of the indigenous population, but nonetheless forms the largest longitudinal dataset of indigenous australia currently in existence (abs, 2013). the 2016 census data has now been integrated into the acld. the 20062011 data is used to establish the stylised facts of the digital divide between indigenous and non-indigenous australians, which will be explained using the multivariate analysis of the 2011-2016 acld (abs 2018). the census household form is designed to be completed by one person on behalf of everyone in their household. census questions from 2006 and 2011 about internet access are identical and covers access via broadband, dial-up or other modes (including mobile phones, see abs, 2011). many researchers have pointed out the overall trend of increasing identification of indigenous people in recent statistical collections (taylor, 2009). however, at an individual level, it is possible that many people choose not to identify in a particular statistical collection. in this paper, we have defined indigenous status as measured by the 2006 and 2011 censuses. the analysis in this paper focuses on household-level data because this is the level at which internet access is measured in the censuses. for the purposes of this analysis, the measure of adoption is the percentage of households who did not have internet access in an earlier census but acquired it by the time of the next census. disadoption is measured as the percentage of households who had internet access in the earlier census, but for one reason or another lost that access by the time of the next census. we explore the main factors identified by radoll (2010) as being associated with ict diffusion or adoption for individuals, but we can also measure these factors at the household level as this is the level at which internet access is measured in this paper. employment is measured by the number of members in the household who are working. education is measured as the highest educational attainment attained by a member of the household. remoteness, and household composition and income are also measured at the household level by definition, and hence there is a consistency in the level of analysis throughout the analysis. household income is equivalised using the ‘modified oecd’ equivalence scale, to capture a measure of household resources available after household composition and structure are taken into account (de vos and zaidi, 1997). equivalising is a means of standardising household australasian journal of information systems hunter & radoll 2020, vol 24, research article dynamics of digital diffusion and disadoption 6 incomes in terms of household size and composition so that the relative material wellbeing of households of different sizes and compositions can be analysed. in practical terms, the process of equivalisation reflects that a larger household needs more income than a smaller household for the two households to have similar standards of living (all else being equal). it also means there are economies of scale as household size increases so that, as the sizes of households increase, the cost per person decreases. the ‘modified oecd’ scale assigns the first adult a cost value of 1.0, the second and subsequent adults a cost of 0.5, and each additional child a cost of 0.3 (or 30% of the first adult). it is not clear what the best equivalence scale is for indigenous australians (hunter, kennedy and biddle, 2004), but the oecd equivalence scales are widely used throughout the world and provide a sensible starting point for the analysis. radoll (2010) emphasises the role of the indigenous field, which can be defined as a domain of life over which the agency of indigenous people is paramount; it involves the interaction of people holding an indigenous world view motivated by a unique epistemology, ontology and axiology and rooted in an ancient culture and a shared experience of post-colonial australia. we attempt to encapsulate the concept of indigenous field is captured in the following analysis using household composition, where we compare households where there are only indigenous residents with other households (especially where there are only non-indigenous residents). note that another important category of indigenous households where indigenous and non-indigenous people live together; this is referred to as ‘mixed’ households for the sake of brevity (it is also a convention in the literature). the other explanatory factors included in the regression analysis are standard in socioeconomic analysis using census data (e.g., hunter, gray and crawford 2016). household composition variables can capture variation in demand for internet services in the dwelling. household mobility and home ownership and the housing stock variables capture resources available and investment in the household infrastructure. in order to provide a preliminary analysis that summarise the multivariate relationships we need a binary regression model to predict the marginal effect of various explanatory factors on the probability of experiencing adoption or disadoption (between 0 and 1). a linear regression model (i.e., the linear probability model estimated using ols) could be used but that is associated with heteroscedastic error variances. while a generalised linear model such as logistic model could be used to address the issue of heteroscedasticity, a probit regression specification can both address this issue and ensure that the errors are normally distributed. the probit model is easiest to think about in terms of a latent variable, y*: 𝑌∗ 𝑖 = 𝛽𝑋𝑖 + 𝜀𝑖 𝜀𝑖 ~ normal distribution [0,1] (1) if 𝑌∗ 𝑖>= 0, 𝑌𝑖 = 1 if 𝑌∗ 𝑖 < 0, 𝑌𝑖 = 0 where: then yi can be viewed as an indicator for whether this latent variable y*i is positive. yi is the dependent variable takes on a value of 0 or 1, if observed adoption and disadoption between 2011 and 2016. the model can be used to estimate the probability (between 0 and 1) of adoption and disadoption. australasian journal of information systems hunter & radoll 2020, vol 24, research article dynamics of digital diffusion and disadoption 7 we choose to report the model in terms of marginal effect of various explanatory factors on the probability of a household experiencing adoption or disadoption because it is relatively intuitive. for continuous data the standard deviation of the sample is calculated, and the marginal effects is estimated as the change in probability association with a one standard deviation change around the average. for dummy variables, the change in probability is measured as a one unit change from the omitted category. the next section describes some important fact about adoption and disadoption between 2006 and 2011 by examining the differing patterns by indigenous status of household and the disaggregated remoteness status (i.e., from major urban areas to very remote areas). section 5 then estimate marginal effects from the regression for the whole acld for 2011 to 2016 and the indigenous and non-indigenous samples (using a broad remoteness indicator). 2011 internet status indigenous and remoteness status 2006 internet status no internet (%) internet (%) total (%) population indigenous major urban no internet 34 66 100 70 515 internet 8 92 100 86 292 inner regional no internet 41 59 100 55 623 internet 12 88 100 50 776 outer regional no internet 50 50 100 64 737 internet 14 86 100 38 446 remote no internet 59 41 100 28 807 internet 20 80 100 10 175 very remote no internet 77 23 100 65 515 internet 46 54 100 7 258 all indigenous (by 2006 internet status) no internet 52 48 100 285 218 internet 12 88 100 192 936 all indigenous (unconditional) 36 64 100 478 186 non-indigenous major urban no internet 37 63 100 2 849 706 internet 4 96 100 9 297 533 inner regional no internet 41 59 100 1 007 134 internet 6 94 100 2 295 365 outer regional no internet 42 58 100 483 746 internet 6 94 100 1 003 017 remote no internet 40 60 100 59 995 internet 6 94 100 139 283 very remote no internet 38 62 100 16 377 internet 7 93 100 39 532 all non-indigenous no internet 39 61 100 4 416 882 internet 5 95 100 12 774 686 all non-indigenous (unconditional) 13 87 100 17 191 544 note: the population in the last column is the estimated residential population residing in indigenous and nonindigenous households in the 2006 census. table 1. changing internet use by indigenous status and remoteness, 2006–11 australasian journal of information systems hunter & radoll 2020, vol 24, research article dynamics of digital diffusion and disadoption 8 4 internet adoption and disadoption: changing access patterns of access to the internet by remoteness for indigenous and nonindigenous australians, 2006–2011 table 1 examines the role of remoteness in the prevalence of adoption and disadoption. we expect remoteness to be associated with these processes because lower levels of accessibility mean that access to information is more valuable, but the cost of providing internet services is likely to be substantially higher. unless the ict services are provided locally it will be more. table 1 is consistent with radoll’s (2010) observation that ict diffusion is higher in cities and urban areas than in remote areas. the increased access to the internet is highest in major urban areas, where 66% of indigenous households acquired internet access between 2006 and 2011. this percentage decreases gradually as the residence of the indigenous households becomes more remote, and only 23% of very remote indigenous households without internet access in 2006 acquired it by the time of the 2011 census. the rate of ict adoption among non-indigenous households is similar irrespective of remoteness: it is 63% in major urban areas; while it is slightly lower in regional areas, remote areas have a similar rate of non-indigenous adoption to that observed in major urban areas (62% of non-indigenous households in very remote areas acquired internet access between 2006 and 2011). in terms of adoption of internet access, indigenous households in remote areas are very different from non-indigenous households in remote areas in that they experience relatively low rates of adoption. one reason may be that indigenous people are more likely to be found in such areas, especially very remote areas, where the cost of provision is likely to be very high unless costs are completely offset by subsidies. as indicated above, it is possible that households lose access to the internet in what we call ‘disadoption’. indigenous people tended to have a particularly pronounced loss of internet access in this period, with 8% of indigenous households in major urban areas who had internet access in 2006 losing it by 2011. the analogous estimate for the non-indigenous population in major urban areas is only 4%. the major finding from table 1 is that indigenous households were much more likely to experience a loss of access to the internet between 2006 and 2011 as the residence becomes more remote, especially those indigenous households in remote and very remote areas. for example, 43% of indigenous people in very remote areas who had internet access in 2006 did not have internet access by the time of the 2011 census. while the indigenous subsample of the acld is relatively small, the indigenous rate of disadoption in remote areas is also high at 20%. in contrast, only 7% and 6% of non-indigenous households in very remote and remote areas lost internet connectivity over the same period. one explanation for the substantial change in internet connectivity in these areas for indigenous households is the need for reinvestment in household infrastructure over time, which is disproportionately concentrated in poorly maintained housing stock (memmott et al. 2012). in our opinion it is more likely to reflect a failure to invest in household ict infrastructure that may have a high rate of depreciation in certain circumstances. in large households embedded in complex kinship networks and communities, ‘permanent’ householders may look after infrastructure, but the large number of ‘visitors’ passing through households may not look after the technology as well as the people responsible for its maintenance or those who want to use the internet in the longer term. australasian journal of information systems hunter & radoll 2020, vol 24, research article dynamics of digital diffusion and disadoption 9 the failure to invest in infrastructure in remote areas may be the responsibility of individuals, communities or the government sector. householders may be personally responsible for access to the internet, and, if adequate resources are available, investment and reinvestment in the latest technological infrastructure is likely to be the individual's responsibility. it has been observed that the main reason for non-indigenous people living in remote areas is that they have a well-paid job that attracted them to live in the area in the first place (gray, howlett & hunter, 2014). if that is the case, such households will have more resources (wages) to invest in internet access. however, to the extent that employers are trying to attract good workers with the nonwage characteristics of the jobs advertised, access to the internet may also be a necessary part of the remuneration package. the job itself may involve access to the internet at work. in an indigenous household, the government and local community are more likely to have played a role in the initial investment in housing infrastructure, which is more communal in nature (memmott et al. 2012). if indigenous householders and communities feel less ownership and individual responsibility for that infrastructure, including ict infrastructure, they may be less inclined to maintain the infrastructure when it breaks down. disadoption is largely an indigenous phenomenon—the highest estimate of disadoption in the non-indigenous households is lower than the lowest indigenous estimate for disadoption in table 1. irrespective of the reason for the breakdown of the internet infrastructure, it is clearly a substantial concern in indigenous households, especially in very remote communities. we will return to this discussion in the concluding section. in the meantime, it is necessary to examine some of the factors that radoll (2010) identifies as being associated with ict diffusion. 5 multivariate analysis of adoption and disadoption, 2011 – 2016 the multivariate analysis of adoption and disadoption are presented as marginal effects of the main factors identified in the diffusion literature. the omitted categories for the dummy variables, are: living in major urban area, no household residents changing address in 5 years to 2011, no householders with year 11–12 completion, a diploma or degree, one family in household, a ‘standard’ dwelling (i.e., not improvised, caravan etc), no children in the household under 15 years old and the home is not owned by a resident. given the composition of indigenous households is potentially significant, the regression of all acld households, includes two dummies for indigenous-only and mixed households and the omitted category is non-indigenous households. for the regressions that focus on indigenous households, a dummy is included for mixed households and the omitted category is indigenous-only households. this informs the interpretation of the marginal effects as the probability of the adoption or disadoption for the reference household is defined by the omitted category (i.e., setting the dummies being set to zero) and assuming continuous data is set to the sample averages (descriptive statistics for the regression analysis are set out in appendix table a1). table 2 reports the marginal effect on probability of adoption over 2011–2016 associated with factors modelled using a probit specification. indigenous-only households has the largest effect on adoption of any explanatory factor (26.8 percentage points). even after controlling for observable information in the census, the gap between indigenous and non-indigenous adoption is large, indeed larger than the difference in the prevalence of adoption in the respective populations; 26.8 percentage points see appendix table a1). this implies that differences in endowments of indigenous and non-indigenous peoples does not explain the australasian journal of information systems hunter & radoll 2020, vol 24, research article dynamics of digital diffusion and disadoption 10 prevalence in the populations. mixed households have a 5.5 percentage point lower probability of adoption than non-indigenous households. all acld households indigenous households nonindigenous households indigenous-only household -0.268 mixed household -0.055 0.159 lives in regional area -0.015 -0.049 -0.014 lives in remote area -0.088 -0.205 -0.058 household mobility -0.017 0.008 -0.018 highest qualification is degree 0.160 0.159 0.159 highest qualification is diploma 0.123 0.134 0.122 highest qualification is year 11 or 12 0.094 0.103 0.093 multiple families in dwelling -0.097 -0.116 -0.092 non-standard dwelling location -0.064 -0.113 -0.062 children under 15 0.148 0.143 0.145 own home 0.112 0.149 0.110 equivalised household income§ 0.022 0.043 0.022 number of dependent students§ 0.015 0.017 0.014 number of employed in household§ 0.041 0.032 0.041 number of usual residents§ -0.017 -0.029 -0.016 number of bedrooms in house§ 0.014 0.022 0.014 seifa of local area deciles§ 0.007 0.015 0.007 pseudo r2 0.074 0.159 0.066 probability of reference household 0.688 0.431 0.696 concordance statistic 0.678 0.757 0.670 number of observations 859,730 28,962 830,768 notes: marginal effects of dummy variable are the change in probability of adoption associated a change in the variable from 0 to 1, while marginal effects for continuous variables (denoted with a §) are reported as the change in probability associated with a one standard deviation of the respective explanatory factor. all marginal effects in this table are significant at the conventional levels. the concordance statistics (or c-statistic) is a summary of the trade-off for the model between identifying true positives and false positives (model predicting an owner manager or other people to be an owner-manager). c-statistics with values over 0.7 is evidence that the model is adequate or even a good model (hosmer and lemeshow 2000: 162). according to these criteria, the goodness of fit of the reported model for indigenous households are more than adequate, but the non-indigenous and total regressions are slightly below the threshold for acceptable discrimination. table 2. marginal effects for adoption regression, 2011-16 given that indigenous households are very different from non-indigenous households in terms of adoption, we separately model the respective populations and focus on the two columns on the right of table 2. among indigenous households, mixed households are 15.9 percentage points more likely to experience adoption than indigenous-only households. after controlling for observable characteristics, living in remote areas is associated with less adoption among indigenous households compared to non-indigenous households living in major urban areas (marginal effect of -20.5% and -5.8%). the difference is also substantial for australasian journal of information systems hunter & radoll 2020, vol 24, research article dynamics of digital diffusion and disadoption 11 the marginal effect for regional areas (-4.9% and -1.4%). consistent with the results presented in table 1. note that this result controls for the socioeconomic status of the local area (through the seifa variable) and mobility of household members. after the effect of indigenous status and geography, the effect of education on adoption is the next most prominent factor. however, there is very little difference in the marginal effects for the indigenous and non-indigenous households, for example, the marginal effect on adoption of having at least one degree level qualification in the household was identical in the indigenous and non-indigenous regressions (15.9%). similarly having more than one family in the dwelling was associated with around 10 percentage points less adoption in both sub-populations. living in non-standard dwellings was associated with substantially lower probability of adoption among the indigenous households (-11.3% & -6.2%). this probably reflects on the poor quality of the housing stock that indigenous people live in. radoll (2010) argues that young people in households can facilitate the process of diffusion or acquiring internet access. young people tend to be more aware of technological developments, and this may assist in the installation and maintenance of relevant hardware and software. the presence of young people in households can drive higher demand for ict-related services because the internet provides educational resources, materials and even access to assessments. the presence of children and dependent students certainly have a large effect of the probability of diffusion in both indigenous and non-indigenous households. for example, households with children under 15 years old are over 14 percentage points more likely to experience adoption in both populations. if indigenous people manage to purchase or own their home the probability of adoption is even higher in the indigenous households than non-indigenous households. increasing the number of bedrooms in a dwelling is also associated with higher probability of adoption in both populations. after controlling for other aspects of the housing stock, increasing the number of usual residents in a dwelling reduces the probability of adoption. this may reflect the complexity of organising and maintaining large households. addressing housing shortfalls in indigenous communities is crucial for addressing the digital divide. adoption increases with equivalised household income for both indigenous and nonindigenous households. a one standard deviation increase in income increases adoption by more for indigenous households (4.3% & 2.2%). increasing the number of employed in the household increases the probability of adoption. this is true even after controlling for equivalised household income, which may indicate an increased demand for internet access associated with modern jobs to stay connected with work after hours. as noted above, the rate of disadoption is higher for indigenous households than nonindigenous households. appendix table a1 shows that in the acld twice as many indigenous households are twice as likely to lose internet access between 2011 and 2016 compared to non-indigenous households (7.7% versus 2.9%). after controlling for observable characteristics of the household in the acld, marginal effect of living in an indigenous-only households is 1.3 percentage point lower disadoption than for non-indigenous households (table 3). the marginal effect on disadoption of living in a mixed household is not significantly different from zero. hence differences in the size, composition and types of households australasian journal of information systems hunter & radoll 2020, vol 24, research article dynamics of digital diffusion and disadoption 12 explain some of the differences between indigenous and non-indigenous households. again, the regression models are estimated separately for indigenous and non-indigenous households. all acld households indigenous households nonindigenous households indigenous-only household 0.013 mixed household 0.002 -0.027 lives in regional area 0.001 0.016 0.000 lives in remote area 0.004 0.043 0.001 household mobility -0.004 -0.014 -0.003 highest qualification is degree -0.010 -0.033 -0.009 highest qualification is diploma -0.005 -0.018 -0.004 highest qualification is year 11 or 12 -0.003 -0.009 -0.003 multiple families in dwelling 0.006 0.010 0.005 non-standard dwelling location -0.002 ‡0.014 -0.001 children under 15 in household -0.012 -0.042 -0.010 own home 0.000 -0.011 0.000 equivalised household income§ -0.011 -0.026 -0.011 number of dependent students in household§ -0.013 -0.008 -0.014 number of employed in household§ -0.011 -0.009 -0.011 number of usual residents§ -0.002 ‡0.002 -0.003 number of bedrooms in house§ -0.003 -0.005 -0.003 seifa of local area deciles§ -0.005 -0.017 -0.005 pseudo r2 0.300 0.177 0.310 probability of reference household 0.006 0.038 0.005 concordance statistic 0.907 0.818 0.910 number of observations 859,730 28,962 830,768 notes. see notes for table 2 above. the probability of the disadoption for the reference household is defined by the omitted category (i.e. setting the dummies being set to zero) and assuming continuous data is set to the sample average. all marginal effects in this table are significant at the conventional levels unless otherwise indicated. ‡ denotes not significant at the 10% level. according to hosmer and lemeshow (2000: 162), c-statistics with values over 0.8 indicate an excellent model and values of ≥0.9 to show outstanding discrimination between observations at different levels of the outcome. table 3. marginal effects of disadoption, 2011-16 geographic variables are again a very significant factor explaining patters in disadoption among indigenous households, especially living in remote areas (4.3%). while disadoption is also higher for non-indigenous households outside major urban areas, the marginal effects are relatively small for that population (only 0.1%). disadoption is concentrated in households with a low level of resources. higher income and living in good neighbourhoods (i.e., with higher socioeconomic status measured by seifa) is associated with significantly lower probability of disadoption. this is consistent with the assertion that access to resources is a major reason that internet access is lost, particularly among indigenous households. australasian journal of information systems hunter & radoll 2020, vol 24, research article dynamics of digital diffusion and disadoption 13 like the results reported for income, there is a strong association between disadoption and education among indigenous households. indeed, the marginal effect of having someone in the household with a degree is associated with a -3.3 percentage point lower probability of disadoption, having young people in a household may increase the demand for internet access (i.e. for adoption), and the presence of dependents (both children under 15 and students) is associated with significantly lower rates of disadoption. the marginal effect on disadoption of having children under 15 in the households is particularly pronounced for indigenous households ( 4.2% versus 1.0%). other measures of housing stock and household composition have a relatively small impact on the probability of disadoption. for example, the effect of living in a non-standard dwelling has no significant effect on the probability of disadoption. one exception to this observation is the presence of multiple families in a dwelling, which is associated with a 1.0 percentage point higher probability of disadoption among indigenous households (& 0.5% for non-indigenous households). given that such households are relatively common in the indigenous population, this is likely to play a role in explaining the higher rate of disadoption among indigenous households, especially in more remote areas (memmott et al 2012). 6 the ongoing digital divide this paper has focused on the processes that underlie the ongoing digital divide between indigenous and non-indigenous australians: the processes of internet adoption and disadoption. however, we should also ask ourselves whether the ict adoption or disadoption documented above led to a systematic change in the digital divide. this research is motivated by 2006 census data that showed that 43% of indigenous households and 64% of nonindigenous households had access to the internet—a differential that implied that more than one-fifth of indigenous households needed to get access to the internet before there is digital equity in australia. this section examines how the dynamics of internet access described above have affected this digital divide. table 4 reports the access to the internet in the 2016 census by indigenous status and indigenous region. there is some good news in that there has been some convergence in internet access. access in indigenous households increased to 75%, whereas it increased to 86% for non-indigenous households. that is, the digital divide between indigenous and other australians fell from a differential of 21% in 2006 to only 11% in 2016. one reason for this is that it gets harder to increase the rate of internet access as that rate approaches 100%. the dwellings remaining without internet access may not want ict services or may be particularly difficult to provide these services for. this phenomenon is what economists call diminishing marginal returns from investment. indeed, as indigenous access to the internet improves, we should expect diminishing marginal returns to become more important (pearce 1986: 238). the relatively high rates of disadoption among indigenous households point to potential difficulty in achieving digital equity. unless the rate at which indigenous households lose ict services can be lowered substantially, the digital divide cannot be eliminated. australasian journal of information systems hunter & radoll 2020, vol 24, research article dynamics of digital diffusion and disadoption 14 indigenous region indigenous household (%) other households (%) digital divide (%) new south wales sydney–wollongong 82.3 88.1 5.8 dubbo 68.4 76.1 7.7 northeastern nsw 66.3 76.4 10.1 northwestern nsw 53.1 72.4 19.3 nsw central and north coast 78.8 81.7 2.9 riverina–orange 71.2 77.7 6.5 southeastern nsw 77.3 81.9 4.5 victoria melbourne 85.4 87.9 2.5 victoria excl. melbourne 76.6 80.8 4.2 queensland brisbane 84.6 88.5 3.9 cairns–atherton 64.9 84.0 19.1 cape york 67.1 81.8 14.7 mount isa 58.2 84.1 25.9 rockhampton 75.4 81.2 5.7 toowoomba–roma 71.3 79.9 8.7 torres strait 68.2 89.0 20.8 townsville–mackay 71.7 83.6 11.9 south australia adelaide 77.0 83.3 6.2 port augusta 52.3 74.5 22.2 port lincoln–ceduna 62.0 79.3 17.3 western australia perth 79.4 89.0 9.5 broome 61.3 88.6 27.3 geraldton 57.4 82.1 24.7 kalgoorlie 55.5 84.4 28.9 kununurra 39.6 84.8 45.2 south hedland 61.4 91.2 29.8 southwestern wa 69.6 83.2 13.5 west kimberley 47.3 85.0 37.7 tasmania tasmania 78.8 80.1 1.3 northern territory darwin 74.4 88.9 14.5 alice springs 63.2 87.8 24.7 apatula 27.5 78.2 50.7 jabiru–tiwi 53.2 81.6 28.4 katherine 47.8 83.9 36.0 nhulunbuy 55.7 90.5 34.8 tennant creek 45.5 83.8 38.3 australian capital territory act 88.1 91.9 3.8 total australia 75.3 85.8 10.5 table 4. internet access in 2016 by indigenous household status and indigenous regions australasian journal of information systems hunter & radoll 2020, vol 24, research article dynamics of digital diffusion and disadoption 15 table 4 illustrates that the dynamics of internet access lead to a larger digital divide in more remote regions. the table is grouped into regions within states and territories, with the first region in each group being the capital or most urban region. where a region is dominated by a city with more than 100 000 residents, the digital divide is less than 10 percentage points. as a region becomes more remote, the digital divide tends to increase. the largest differential between indigenous and non-indigenous internet access is in apatula, where the digital divide is more than 50 percentage points. it is not that the internet access of non-indigenous households is particularly high in these remote regions; rather it reflects the particularly low level of internet access in indigenous households in remote areas. 7 concluding remarks this paper pinpoints several factors associated with indigenous households being connected to the internet and hence having access to ict: remoteness, income or access to resources, employment, education, the housing stock and demography (including household composition). the internet is crucial for ensuring adequate connection to services and society. resources, such as household income, are identified as being important in ensuring that indigenous people have access to the internet and maintain access to ict over time. the analysis also identified disadoption as an important policy issue that will potentially lead to further social exclusion of indigenous australians. ensuring indigenous households have access to resources and an adequate housing stock are of paramount importance for dealing with the digital divide between indigenous and non-indigenous households. it is always salutatory to ask the ‘so what’ question: are these observations important in the long run? for example, increasing decentralised access to the internet through mobile services may be reducing the significance of relatively fixed household infrastructure. while there may be some truth in this observation, it would be a mistake to ignore the role of fixed infrastructure, because mobile devices generally have more limited functionality than desktop devices, and mobile internet connectivity can often be much slower and more strongly affected by the environment and surrounding infrastructure. notwithstanding these reflections, mobile internet access is arguably captured in the above census analysis, albeit at the household level. this paper identifies some information that can be further analysed using panel techniques to control for unobserved heterogeneity between households. unfortunately, census data are collected at five yearly intervals that makes is difficult to conduct a truly longitudinal analysis when there are so few waves of data (and where the technology is evolving rapidly over time). identification of the characteristics of households associated with ict adoption and disadoption would place policy makers in a better position to target their policies appropriately and bridge the digital divide. one important caveat to our analysis is that our analysis does not directly include communitybased access (libraries, internet cafés, etc.), where people without access in their household are nonetheless able to access the internet. however, the seifa variable may control for some local public goods, if community resources or access are associated with average local income and socioeconomic status. community-based access may be an important avenue for public investment, especially after social distancing injunctions associated with the covid-19 pandemic are no longer necessary. australasian journal of information systems hunter & radoll 2020, vol 24, research article dynamics of digital diffusion and disadoption 16 radoll (2010) identified ‘structure’ and ‘agency’ as key features of the indigenous household ict adoption process, especially in the intersection of the employment and education (structures) with indigenous agency. using these concepts of structure and agency, the theory asserts that the intersection of the indigenous field and other factors associated with ict adoption, along with the interactions between structures and agency, produces new practices by indigenous agents that lead to indigenous household ict adoption. most of the factors identified above are ‘structural’ in nature, with the possible exception of the household composition by indigenous status. it is difficult to analyse issues associated with agency of individuals and households using quantitative techniques, hence it is important for the regression analysis in this paper to be supplemented by qualitative analysis that can attempt to understand the reasons for decisions for adoption or perhaps even unpack the complex factors associated with the failure to make decisions that led to disadoption. the main contribution of this paper is to identify some factors contributing to the digital divide, but policy also needs to understand what determines household outcomes and characteristics. a comprehensive policy solution needs to be developed to stem the gap of the growing digital divide (dijk, 2005). technology maintenance strategies are required to ensure that the aboriginal and torres strait islander communities are able to maintain their internet connections. the covid-19 pandemic underscores the urgency of addressing the digital divide. the consequences of not taking immediate policy action will be to further exacerbate the indigenous disadvantage through constrained access to information and e-services, reduced remote learning opportunities for indigenous children and exclusion from the labour market that will be increasingly reliant on telecommuting. acknowledgement the authors are grateful to helpful comments provided by two anonymous referees. references abs (australian bureau of statistics) (2011). census dictionary, 2011. cat no. 2901.0, abs. canberra: —— (2013). information paper: australian census longitudinal dataset, methodology and quality assessment, 2006–2011, cat no. 2080.5 abs. canberra: —— (2018). microdata: australian census longitudinal dataset, 2011–2016, cat no. 2080.0, abs, canberra. ajzen, i. (1991). the theory of planned behaviour. organizational behaviour and human decision processes, 50, 179–211. https://doi.org/10.1016/0749-5978(91)90020-t —— & fishbein, m. (1980). understanding attitudes and predicting social behaviour. englewood cliffs: prentice hall. allyn, m. & yun, s. (2005). computers and human capital potentiation. journal of business and economic studies 11(2), 34–92. arocena, r. & senker, p. (2003). technology, inequality and underdevelopment: the case of latin america. science, technology, and human values, 28(1), 15–33. https://doi.org/10.1177/0162243902238493 australasian journal of information systems hunter & radoll 2020, vol 24, research article dynamics of digital diffusion and disadoption 17 compeau, d., higgins, c. & huff, s. (1999). social cognitive theory and individual reactions to computing technology: a longitudinal study. mis quarterly, 23(2), 145–158. https://doi.org/10.2307/249749 curtin, j. (2001). a digital divide in rural and regional australia? parliamentary library (ed.), canberra: australian government. daly, a. (2006). the diffusion of new technologies: community online access. in: grant, s., dyson, l. and hendriks, m. eds, information technology and indigenous people, information science publishing, london, 272–285. davis, f. (1989). perceived usefulness, perceived ease of use, and user acceptance of information technology. mis quarterly, 13(3), 319–340. https://doi.org/10.2307/249008 davis, c., mcmaster, j., & nowak, j. (2002). it-enabled services as development drivers in lowincome countries: the case of fiji. the electronic journal on information systems in developing countries, 9(4), 1–18. https://doi.org/10.1002/j.1681-4835.2002.tb00054.x de vos, k. & zaidi, m. (1997). equivalence scale sensitivity of poverty statistics for the member states of the european community. review of income and wealth, 43, 319–333. https://doi.org/10.1111/j.1475-4991.1997.tb00222.x giddens, a. (1984). the constitution of society: outline of the theory of structuration. berkeley: university of california press. gonzales, a. l. (2014). health benefits and barriers to cell phone use in low-income urban u.s. neighbourhoods: indications of technology maintenance. mobile media & communication, 2(3), 233–248. https://doi.org/10.1177/2050157914530297 gonzales, a. l. (2016). the contemporary us digital divide: from initial access to technology maintenance. information, communication & society, 19 (2), 234–248. https://doi.org/10.1080/1369118x.2015.1050438 gonzales, a., ems, l & suri v (2016). cell phone disconnection disrupts access to healthcare and health resources: a technology maintenance perspective. new media & society, 18(8), 1422–1438. https://doi.org/10.1177/1461444814558670 gonzales, a. l. (2017), disadvantaged minorities’ use of the internet to expand their social networks. communications research, vol. 44(4), 467–486. https://doi.org/10.1177/0093650214565925 gray m., howlett m. & hunter b. (2014). labour market outcomes for indigenous australians. the economic and labour relations review. 25(3), 497–517. https://doi.org/10.1177/1035304614545943 gurstein, m. (2004). effective use: a community informatics strategy beyond the digital divide. first monday, 8(12). https://doi:10.5210/fm.v8i12.1107 hosmer, d. and lemeshow, s. (2000). applied logistic regression (2nd edn). new york: john wiley & sons. hunter, b. (2004). indigenous australians in the contemporary labour market. canberra: australian bureau of statistics. australasian journal of information systems hunter & radoll 2020, vol 24, research article dynamics of digital diffusion and disadoption 18 hunter, b., gray, m. & crawford, h. (2016). who cares and does it matter for the labour market?: a longitudinal analysis of the labour force status of indigenous and nonindigenous carers. australian journal of labour economics. 19(1), 33–51. hunter, b., kennedy, s. & biddle, n. (2004). indigenous and other australian poverty: revisiting the importance of equivalence scales. economic record, 80(251), 411–422. https://doi.org/10.1111/j.1475-4932.2004.00198.x katz, j. e. & rice, r. e. (2002). social consequences of internet use: access, involvement and interaction. cambridge, ma: the mit press. memmott, p., greenop, k., clarke, a., go-sam, c., birdsall-jones, c., harvey-jones, w., corunna, v. & western, m. (2012). natsiss crowding data: what does it assume and how can we challenge the orthodoxy? in: hunter, b. and biddle, n. eds, survey analysis for indigenous policy in australia: social science perspectives. caepr research monograph 32, canberra: anu e-press. mesch, g. (2012). minority status and the use of computer-mediated communication: a test of the social diversification hypothesis. communication research, 39, 317–337. https://doi.org/10.1177/0093650211398865 pearce, d.w. (1986). macmillan dictionary of modern economics. 3rd ed, london: macmillan. radoll, p. (2010). stone chips to silicon chips: a grounded theory of information and communication technology adoption in australian indigenous households – rural, urban and remote. phd thesis, canberra: anu. doi: 10.25911/5d7a26719b054 radoll, p. & hunter, b. 2017. ‘the dynamics of the digital divide’, working paper 120, caepr, the anu, canberra. rice, r. e. & katz, j. e. (2003). comparing internet and mobile phone usage: digital divides of usage, adoption, and dropouts. telecommunications policy, 27(8/9), 597–623. https://doi:10.1016/s0308-5961(03)00068-5 rice, r. e., & pearce, k.e. (2015). divide and diffuse: comparing digital divide and diffusion of innovations perspectives on mobile phone adoption. mobile media & communication, 3(3), 401–424. https://doi.org/10.1177/2050157915590469 rogers, e. (1995). diffusion of innovations. 4th ed. new york: the free press. shilling, j., sirmans, c. & dombrow, j. (1991). measuring depreciation in single-family rental and owner-occupied housing. journal of housing economics, 1(4), 368–383. https://doi.org/10.1016/s1051-1377(05)80018-x taylor, j. (2009). indigenous demography and public policy in australia: population or peoples? journal of population research, 26(2), 115–130. https://doi:10.1007/s12546-0099010-9 van dijk, j.a. (2005). the deepening divide: inequality in the information society, southend oaks, ca, usa: sage publications. https://doi:10.4135/9781452229812 venkatesh, v. & brown, s. (2001). a longitudinal investigation of personal computers in homes: adoption determinants and emerging challenges. mis quarterly, 25(1), 71–98. https://doi.org/10.2307/3250959 australasian journal of information systems hunter & radoll 2020, vol 24, research article dynamics of digital diffusion and disadoption 19 venkatesh, v., morris, m., davis, g. & davis, f. (2003). user acceptance of information technology: toward a unified view. mis quarterly, 27(3), 425–478. https://doi:10.2307/30036540 australasian journal of information systems hunter & radoll 2020, vol 24, research article dynamics of digital diffusion and disadoption 20 appendix all acld indigenous-only non-indigenousonly mean standard deviation mean standard deviation mean standard deviation adoption 0.675 0.469 0.451 0.498 0.682 0.466 disadoption 0.031 0.173 0.077 0.266 0.029 0.169 indigenous-only household 0.012 0.109 0.359 0.480 mixed household 0.022 0.145 0.641 0.480 lives in regional area 0.270 0.444 0.441 0.496 0.264 0.441 lives in remote area 0.019 0.137 0.170 0.376 0.014 0.117 household mobility 0.472 0.499 0.554 0.497 0.469 0.499 highest qualification is degree 0.355 0.478 0.121 0.327 0.363 0.481 highest qualification is diploma 0.358 0.479 0.363 0.481 0.357 0.479 highest qualification is year 11 or 12 0.137 0.344 0.171 0.376 0.136 0.343 multiple families in dwelling 0.038 0.192 0.116 0.321 0.036 0.186 non-standard dwelling location 0.006 0.079 0.004 0.062 0.006 0.079 children under 15 0.520 0.500 0.724 0.447 0.513 0.500 own home 0.731 0.444 0.388 0.487 0.743 0.437 equivalised household income§ 1,002 696 660 543 1,014 698 number of dependent students§ 0.240 0.557 0.208 0.491 0.241 0.559 number of employed in household§ 1.650 1.041 1.334 1.116 1.661 1.037 number of usual residents§ 3.570 1.380 4.368 1.765 3.542 1.356 number of bedrooms in house§ 3.382 0.853 3.330 0.839 3.383 0.853 seifa of local area deciles§ 5.731 2.863 3.579 2.616 5.806 2.842 number of observations 859,730 28,962 830,768 notes. § denotes continuous data. table a1. summary statistics for regressions, acld 2011-2016 copyright: © 2020 hunter & radoll. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v24i0.1805 http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems hunter & radoll 2020, vol 24, research article dynamics of digital diffusion and disadoption 21 paper title (paper title style) australasian journal of information systems wang, alahakoon & de silva 2018, vol 22, research on recent advances in social media extended cognitive situation model 1 an extended cognitive situation model for capturing subjective dynamics of events from social media yujie wang la trobe university wang.y@students.latrobe.edu.au damminda alahakoon la trobe university daswin de silva la trobe university abstract the event-indexing situation models are introduced as event models derived from language to facilitate comprehension and memory retrieval. these models explain how fragmental information about events are collected, integrated and updated into a coherent set of views of what the text is about. the models are adopted as the basis of this study as an attempt to capture the event with contextual, dynamic, and social features, as conveyed by the vast volumes of online textual resources. information in social media is received through highly personalized channels and is reshaped and interpreted in a more individual, segmental and real-time manner. the reprocessed information is then spread at high speed to a wider range of receivers. therefore, the interpretation of mainstream media content is influenced by ongoing and dynamic contribution of opinions by users empowered by social media. this new phenomenon has not been examined so far from the perspective of the impact on conventional situation models. this paper explores how collaborative and sharing aspects of social media emphasize subjectivity of interpretation of mainstream media and proposes an extended cognitive situation model which better represents event-centric knowledge. this study investigates the mechanisms for constructing and updating the situation models with continuous textual information streamed from heterogeneous forms of media. it also empirically demonstrates how the proposed model can enhance the understanding of subjective aspects of events with dynamic social opinions. keywords: social media; natural language processing; text comprehension; collective social opinion 1 introduction streams of data are generated from every aspect of our daily life, especially unstructured data narrated in natural language. the streams of textual data often continuously evolve with time, which conveys rich information about events and provide the context of such events, such as temporal-spatial features, individuals involved, objects, causation and goals. the event knowledge embedded in the rapidly increasing volumes of textual data contains considerable potential value for many data-driven decision making applications, and as such has attracted increasing research interests from researchers focusing on natural language processing (nlp), machine learning, event extraction and knowledge representation. although existing research has made significant improvements, machine understanding of text remains inadequate considering the massive volume of data to be processed and the depth australasian journal of information systems wang, alahakoon & de silva 2018, vol 22, research on recent advances in social media extended cognitive situation model 2 of understanding required. the challenges stem from several aspects, including the complications of natural language, the complex and evolving nature of events, and the seemingly impenetrable interactive and dynamic information exchanging environment enabled by heterogeneous media and diverse platforms. firstly, knowledge extraction from text largely depends on machine understanding of human language, where most existing research applies either a structural or statistical approach, or a hybrid combination of both. the former stresses on examining the structural representation of text, and the latter focus on the word co-occurrence frequencies and the corresponding statistical distribution patterns. the acquired event knowledge is often static, segmental and isolated from context. secondly, there is still no consensus among researchers about the definition of an event and how eventbased knowledge should be structured and represented. focusing on event extraction and tracking, researchers model and represent events either in the form of pre-defined templates or as a list of representative words, terms or concepts depending on their statistical distribution in the texts. events constructed in these forms are unable to capture the essential situational settings where the events take place. consequently, it imposes greater challenges for researchers to establish associations between events and the evolutional behaviours of events overtime. in a word, the dynamic and evolving features of real life events are not represented in the knowledge bases built upon these approaches. lastly, most event extraction research stress on extracting factual knowledge about events from homogenous sources. however, events contain not only objective or factual knowledge such as time, space, individuals, objects, but also subjective information such as causality and intentionality which are subject to interpretation by individuals depending on their emotion, knowledge, understanding and experiences. traditionally the mainstream media plays an active role in framing the public opinion by viewpoints conveyed in reports. although the reader can construct an interpretation from such a report, the information flow is one way and do not facilitate a discussion or conversation. with the advent of social media platforms, the public are empowered with exposure to different forms of media such as news, blogs, question and answer (q&a) sites, social media platforms. online users engage in information and opinion creation in a more interactive and influential manner. how such social media data could be captured to augment event knowledge with the public opinion and its evolving dynamics is still an open question for researchers. even utilizing current extensive computing power and rapidly growing memory capacity, machine is still unable to achieve human capabilities such as extracting and disambiguating meaning from text, linking and merging fragmental information through situational context, and enhancing the understanding of events with pre-acquired knowledge and experiences. therefore, to reduce the cognitive gap between machine and human brain, we adopted a couple of psycholinguistic theories focussing on text comprehension, namely the situation models and the event indexing model. the former illustrates the mental representation that human brain deploys to decode text back into the micro-world described by the author (dijk, kintsch, & dijk, 1983; johnson-laird, 1983; r. a. zwaan, m. c. langston, & a. c. graesser, 1995). the latter emphasises on the construction mechanism of situation models (radvansky & zacks, 2014; r. a. zwaan et al., 1995). situation models and the event indexing model provide means to simulate the process that human brain builds up mental units from narrative texts. as ongoing output of comprehension, the layered and networked mental representation continuously directs reasoning processes and memory related operations (crowley, reignier, & barranquand, 2009; australasian journal of information systems wang, alahakoon & de silva 2018, vol 22, research on recent advances in social media extended cognitive situation model 3 radvansky & zacks, 2014). however, the models are ambiguous in term of guiding empirical study in computing. the theories also lack the explanation of how an information receiver’s pre-acquired knowledge impacts the interpretation of the events. additionally, the models are inapplicable to the interactive information exchanging environment where individual receivers’ opinions act as a collective influential force to facilitate the evolution of public opinion overtime. to address the limitations of event knowledge extraction from heterogeneous media of textual resources, we propose an extended situation model upon the re-evaluation of objective and subjective natures of situational dimensions of events. the revisited model emphasizes capturing events with situational contexts and knowledge enhancement, and elaboration with social opinion dynamics. this study identifies that the subjective features are liable to evolve with influential social opinions, hence we extend the existing model with a social view layer to capture the dynamic evolutions of the collective social opinions overtime. the rest of this paper is organised as follows: section (2) introduces the related work on natural language processing, event extraction and event knowledge construction with social opinions. section (3) presents the theoretical basis of this study focusing on the event-indexing situation models. to incorporate the dynamics of social opinions, we revisit the adopted models and propose an extended situation model. section (4) demonstrates the construction of the extended situation model with a sample dataset to examine the dynamic subjective factors of the constructed situation models. finally, we discuss the findings and conclude our study in section (5). 2 related work although an event is commonly defined as an occurrence that unfolds a segment of time and specific space (j. allan, carbonell, doddington, & yamron, 1998; chambers & jurafsky, 2008; hogenboom, frasincar, kaymak, de jong, & caron, 2016), the content of an event is more than the temporal and spatial aspects. an event is a collection of relations of the involved entities and concepts by essence (radvansky & zacks, 2014). a subset of the situational dimensions are especially important for event construction during text comprehension, including time, space, protagonist and object, causality and intentionality. these interrelated dimensions serve as the skeleton structure of events that supports the cognitive processes, such as relation inference, information analysis and integration, memory retrieval and updating, prediction and decision making (dijk, 1977; dijk et al., 1983; w. kintsch & dijk, 1978; radvansky & zacks, 2014; r. a. zwaan & g. a. radvansky, 1998). noticeably the dimensions of causality and intentionality are often not explicitly phrased in the original texts, and are always inferred and interpreted by the readers depending on their own emotions, knowledge and experiences (r. a. zwaan & g. a. radvansky, 1998), which can be subjective and liable to change with new information. therefore, we have investigated the existing work from related research fields including natural language processing, and event extraction. the aim is to find the research gaps in extracting event-based knowledge upon situational context awareness and to enrich the output with collective opinion dynamics in online global conversation environment. australasian journal of information systems wang, alahakoon & de silva 2018, vol 22, research on recent advances in social media extended cognitive situation model 4 2.1 natural language processing (nlp) the depth of machine understanding of text largely relies on the natural language processing (nlp) capabilities. nlp is defined as a theory-motivated range of computational technologies for automatic analysis and representation of human language (cambria & white, 2014). existing nlp approaches are adept at conducting lexical and syntactic analysis of text, which examines the structure of sentences and word co-occurrence frequencies (cambria & white, 2014). in knowledge extraction and modelling field, retrieving semantics or meanings and performing cognitive ‘thinking’ based on the extracted facts are still questionable. the challenges come from the complexity of human language. 1. displacement feature of language. the original sequential order of events is always broken into self-containing chunks and rearranged in accordance to the author’s interest. 2. missing background information. common-sense knowledge and some situational information are often left out since the information gaps are deemed to be filled up by referring to pre-acquired knowledge or context. 3. noisiness and redundancy of information. the ambiguous or even controversial information increases the difficulties for machines to resolve coreference, especially in cross-document cases. humans comprehend text through a series of much more complex processes taking place almost simultaneously. human mind can recognize entities mentioned in text and resolve the meanings of word and sentence, identify and merge associated incomplete information according to situational context to understand the events mentioned, and augment the understanding of event with background knowledge and experience to form more comprehensive episodic knowledge (garnham, 1987; graesser, millis, & zwaan, 1997). understanding is a holistic system which integrate different levels of analysing tasks, memory operations to store, retrieve and update extracted knowledge, and mechanisms to integrate local information from text with external knowledge bases. comparatively, most existing nlp approaches focus more on one or a few aspects of machine understanding of text, than investigating it in a systematic way(cambria & white, 2014). 2.2 event extraction techniques traditionally event extraction refers to the information extraction (ie) task, which detects and captures events from unstructured or semi-structured textual sources and generate structured information about the captured events. we studied the event extraction approaches with the focus on event representation and knowledge construction for text understanding. existing researches fall into two mainstreams: knowledge-driven approach and data-driven approach. knowledge-driven approaches emphasis on the richness of event knowledge. event is modelled as a collection of predefined templates in accordance with a set of information extraction rules defined by domain experts (aguilar et al., 2014; ahn, 2006; doddington et al., 2004; tanev, piskorski, & atkinson, 2008). data-driven approach stress on the efficiency of information retrieval and event trend identification from large amount of data. thus researchers conduct statistical analysis of the text structure and classify documents into eventcentric topical categories labelled as lists of representative words with timestamps (james allan, 2002; j. allan et al., 1998; brüggemann, hermey, orth, & schneider, 2016). a few researches also represent events in a network structure, such as a bank of predicates australasian journal of information systems wang, alahakoon & de silva 2018, vol 22, research on recent advances in social media extended cognitive situation model 5 (kingsbury & palmer, 2002), or semantic network of concepts (baker, fillmore, & lowe, 1998; gangemi, hassan, presutti, & reforgiato, 2013). majority of related work focus on extracting factual relationships between entities on sentence basis, or representation events as statistical calculation results. for example, the template based approaches fill the template slots with the static and fragmental information related to an event, such as person, location, type of event, number of casualties and so on; and the statistical approaches which model events as a list of topical terms can only represent events more ambiguously as words where the meanings of words are not considered (i.e., bank in ‘bank account’ and ‘lake bank’ are considered as the same). therefore, the acquired knowledge of events remains static, segmental and isolated from context. the above-mentioned limitations further restrain a system to develop sufficient understanding of events in several aspects. firstly, understanding of event depends on situational context, which is represented as relations of entities or concepts to events. secondly, event representation should support analysing operations such as distinguishing, comparison and updating of events. thirdly, various types of associations of events, such as temporal relation, spatial relation, causal relation, are necessary to capture the evolutional dynamics of events. lastly, understanding of events also depends on external knowledge, such as world knowledge, social knowledge, knowledge of psychological activities and so on (adolphs, 2009; langer, 1984; piaget & duckworth, 1970; van dijk, 1977). to sum up, although existing nlp and event extraction and tracking approaches demonstrate strengths in structured and domain-tailored event-related information extraction, or online capture of dynamic topical trend shifting from vast volume of data, these approaches lack two crucial factors to ensure effective text understanding: context and background knowledge. therefore, comprehension related cognitive processes are not well supported, such as situation reconstruction from text, association establishment of events, recognition of event evolution and knowledge enrichment with external knowledge. the insufficiency of event knowledge is further reflected in subjective dimensions of events, which largely depend on knowledge, experiences and psychological activities of both individuals and the society. the challenge of modelling subjectivity of event are intensified by the proliferation of social media. boundaries of conversations have been expanded such that people engage in discussions with broader communities in a more flexible and interactive manner. we propose this new form of information exchange being considered as a form of online global conversation. 3 theoretical basis 3.1 psychological events models of text understanding to improve machine understanding of textual data aiming at bridging the gap between machine and human processing, we examined the cognitive theories related to human comprehension of narrative texts. prevailing and experimented theories, including situation models (dijk, 1977; w. kintsch & dijk, 1978; r. a. zwaan et al., 1995) and event-indexing model (rolf a zwaan, 1999; r. a. zwaan et al., 1995) are commonly adopted by researchers of related areas (crowley et al., 2009; graesser et al., 1997; walter kintsch, 1998; sanford & emmott, 2012) and well support the objectives of this study. the situation models theory abstracts and represents the mental representation human construct during the comprehension processes (walter kintsch, 1998; w. kintsch & dijk, 1978; r. a. zwaan et al., australasian journal of information systems wang, alahakoon & de silva 2018, vol 22, research on recent advances in social media extended cognitive situation model 6 1995; r. a. zwaan & g. a. radvansky, 1998). it represents a spatially and temporally bounded event, as fundamental processing units to coordinate the comprehension processes and memory retrieval. the event-indexing model explains the mechanism of situation model construction on sentence basis and how event nodes are activated, updated and associated dynamically via situational dimensions while a reader processes a unit of narrative text (figure 1). figure 1: multi-layer and multi-dimensional mental representation of text comprehension three layers of multi-dimensional mental representation during human comprehension and the situation model (dijk et al., 1983; r. a. zwaan & radvansky, 1998) according these theories, when readers comprehend a story, they construct a multi-level and multi-dimensional mental representation to reflect the micro-world described by the author. the three levels of representations are surface level, propositional level or textbase, and situation model. surface level represents the environment, which is built upon examining words and syntax of the literal wordings of text (radvansky & zacks, 2014). identified objects and concepts are networked by extracting propositional relations from sentences, which forms the textbase layer (walter kintsch, 1998). the highest level of representation, situation model, represents an event model (radvansky & zacks, 2014). based on further analysis of the propositional relationship, and enhanced with prior acquired experience/knowledge, a coherent set of situation models are constructed in temporal and causal order as more comprehensive understanding and representation of the plot of the story (figure 1). 3.2 situation models: event based mental representation of text understanding the situation models are introduced as event models derived from language which are represented as components and relations of components. thus the spatial-temporal framework within which events takes place serves as an essential basis for situation model construction to restore the holistic view of events (radvansky & zacks, 2014; rolf a zwaan & gabriel a radvansky, 1998). in event cognition theory, a situation is the condition and status of a collection of events that occur within the same tempo-spatial bound (radvansky & zacks, 2014). a situation model is represented in a network structure with predicates or verbs as the focal points of the events to link the propositions coherently (rolf a zwaan, mark c langston, & arthur c graesser, 1995; rolf a zwaan & gabriel a radvansky, 1998). the five key situational dimensions serve as the supporting indices to assist comprehending, including temporality, spatiality, protagonist and object, and causality and intentionality. the construction of situation models follows a three-layer processes from surface code, textbase to situation model. australasian journal of information systems wang, alahakoon & de silva 2018, vol 22, research on recent advances in social media extended cognitive situation model 7 during texts comprehension processes, events serve as the critical focal units in the mental representation which not only connects the events and elements of events but also coordinates the inferencing instructions along situational indices (time, space, protagonist, causality, and intentionality)(crowley et al., 2009; r. a. zwaan, 2016; r. a. zwaan et al., 1995). the situation model theory presents the event-centric mental representation as the fundamental units for knowledge augmentation, inference and retrieval. 3.3 event-indexing model: mechanism of constructing situation models event-indexing model introduces the mechanisms and processes of how human mind continuously construct and update existing situation models in memory with incoming information (r. a. zwaan et al., 1995). while a reader comprehends a piece of text, the clauses are parsed into events as the focal units of the situation models. the semantic relations conveyed by the clauses are established through events with the five situational dimensions, time, space protagonist and object, causality and intentionality. the previously constructed situation models are subject to updating with incoming events depending on the continuity of situation settings. events are updated through a series of situation models. the current model is the model under construction at time tn extracted from clause cn. the integrated model of the situations from time t1 to tn-1 is the global model that is constructed by integrating clauses c1 to cn-1. finally, the completed model at tn is the integrated model after the complete set of clauses has been processed (rolf a zwaan, 1999; r. a. zwaan et al., 1995)(figure 2). figure 2: event-indexing model (updating existing situation model with new event model 3.4 new information exchange environment with social media before the emergence of web 2.0, information is predominantly generated by mainstream media such as newspapers, radio, television and publishing. the public receive information in a passive manner and there were limited channels for individuals to express opinions or communicate with others. the interactive web 2.0 based technologies have facilitated a multitude of digital media and technologies, which allow users to create and share content and to act collaboratively (gruber, 2008; schoder, gloor, & metaxas, 2013), and therefore is called social media. example social media platforms include micro-blogging sites such as twitter and sina weibo, businessand employment-oriented social networking service such as linkedin, social question and answer platforms such as quora and zhihu, photo-sharing applications such as instagram and tumblr, and video exchanging platforms such as youtube. the rapid growth of social media has immense impact on how people communicate and interact. research indicate that with online social activities becoming ubiquitous, a significant and growing number of people receive and spread mainstream media data through social media (hermida, fletcher, korell, & logan, 2012). social media has changed the way people consume and diffuse information. information is received though a set of highly personalized australasian journal of information systems wang, alahakoon & de silva 2018, vol 22, research on recent advances in social media extended cognitive situation model 8 channels (hermida et al., 2012), and is reshaped and interpreted in a more individual, segmental and real-time manner, and is spread at extremely high speed reaching a much wider range of receivers. furthermore, the social network structure of data users also affects the way how information is spread, accumulated and processed. therefore, the interpretation of mainstream media content is influenced by ongoing and dynamic contribution of opinions by involved users empowered by social media. to understand how various types of social media services have reshaped the interactions among online users, we refer to the seven functionalities of social media illustrated by the honeycomb framework (figure 3), including identity, presence, sharing, conversations, relationships, reputation and groups (kietzmann et al., 2011). these functionalities explain the varieties of strategic engagement of social media services. for instance, linkedin users are thought to care mostly about identity, reputation, and relationships, whereas twitter’s primary features are sharing, conversations, groups and reputation. figure 3: the honeycomb of social media (kietzmann, hermkens, mccarthy, & silvestre, 2011) the honeycomb framework was proposed to help business managers to understand the social media landscape so as to develop suitable strategies for their social media activities and content accordingly (kietzmann et al., 2011). it also portrays social media as the enabler of another layer of the information ecology, as called social layer in this study, which provides value for investigating how information is consumed, generated and diffused in the global online communication environment. ‘identity and reputation’ imply the influence of the users. ‘presence, relationships and groups’ highlight the pattern of interactions of online communities. ‘sharing and conversations’ focus on the dynamic and real-time features of the communications. this classification of the functionalities of social media indicates the significance of the roles users and the features of information transmissions. firstly, social media content has become an increasingly important information source in addition to the traditional mainstream data such as newspapers and publications. secondly, the interpretation of mainstream data is now influenced dynamically overtime due to the involvement of online communities, especially the influencer’s opinions. lastly, the social network structure and the influence factors of users have significant impact on the manner information is reprocessed and diffused. the accumulated information upon the connectivity of friends, subscriptions australasian journal of information systems wang, alahakoon & de silva 2018, vol 22, research on recent advances in social media extended cognitive situation model 9 and groups now has significant impact and reshapes the conventions of data consumption (figure 4). figure 4: comparison of traditional text processing and text processing with social media 3.5 rethinking of situation model theory with social media situation model theory was proposed for traditional discourse comprehension, especially narrative text understanding by the human mind (walter kintsch & van dijk, 1978; van dijk, 1977; rolf a zwaan & gabriel a radvansky, 1998). the strength of the situation models is to extract, integrate and model fragmental situational information from one or more texts into a coherent set of beliefs, which represents what the processed text(s) is about (walter kintsch & van dijk, 1978; rolf a zwaan et al., 1995; rolf a zwaan & gabriel a radvansky, 1998). notably the successful construction of situation models relies heavily on prior acquired knowledge and experiences to perform inferring operations and fill up the gaps of literal expressions (barclay, bransford, franks, mccarrell, & nitsch, 1974; graesser, gernsbacher, & goldman, 2003; graesser et al., 1997; walter kintsch, 1988). researchers claim that comprehension is always to some extent idiosyncratic because when readers elaborate the situational models they refer to the most appropriate and useful personal knowledge and experiences (anderson, 1984; langer, 1984; tannen, 1981). although situation model explains how human mind decode meanings sentence by sentence and construct holistic and coherent mental representations about the text, or even across a set of texts, the original model does not cater to fuse scattered, segmented, heterogeneous and subjective event related information created by large numbers of users. neither does it represent the dynamic augmentation of social views overtime. therefore, based on the investigation of information exchange behaviours in social media, we have adapted the existing cognitive models of text comprehension to cater to the new environment of globalized online conversations. to examine the influences of social media and how individuals and opinions are organized and interact dynamically, we examined the honeycomb framework of social web (figure 3) which illustrate seven key functionalities of social media in assisting individual content creation and diffusion. different social media platforms focus on various combinations of these functional areas depending on corresponding application requirements. the framework reveals the importance of both the structure of how people are connected and the role in the community (identity, relationships, presence, reputation, groups) and the way that users exchange comments about subject topics (sharing, conversations). considering the honeycomb framework, it could be seen that social media differs from previously described australasian journal of information systems wang, alahakoon & de silva 2018, vol 22, research on recent advances in social media extended cognitive situation model 10 narrative text. a key difference is that social media information is a ‘conversation’ which is one of the dimensions of the honeycomb framework. referring to another dimension ‘sharing’, we could see that the information is shared and such sharing is impacted by a) the identity of writer, (b) the groups the writer belongs to, (c) the reputation of the writer, (d) presence on the social network and the (e) ‘reputation’ on the social network as well as in society. each of these factors have been identified in the honeycomb framework as such we use the framework as a foundation to the claim that traditional narrative text develops into a conversation with dynamics dictated by the components of the honeycomb framework when considering social media information. the finding highlights the necessity of re-evaluating the dimensions of situation models. under the new information environment with social web, the understanding of events is not only coordinated by the direct protagonist involved in the events, but also indirect online observers with diverse forms of actions, such as viewing, commenting, sharing. although the indirect viewers are not necessary for situation model construction, their contribution has significant impact on the subject factors of events. 3.5.1 subjectivity and objectivity of situation models although situation model serves as the fundamental units for event construction, the five situational dimensions’ functions differently and are extracted from various information sources. temporality and spatiality in combination servers as the essential situational framework to distinguish events (rolf a zwaan, 1999; rolf a zwaan et al., 1995). protagonist is an important clue that lead the thread of events to understand the plot of the story (radvansky & zacks, 2014; zacks, speer, swallow, braver, & reynolds, 2007; rolf a zwaan, 1999). however, causality and intentionality are often not literally conveyed in the source text, which require the reader to interpreted with emotions, pre-acquired knowledge and experiences (rolf a zwaan & gabriel a radvansky, 1998), which can be subjective and liable to change with new information (van dijk, 1982). the interpretation outcomes vary amongst individuals and are largely influenced by psychological status and social opinions. van dijk (1982) defined subjective understanding of the situation conveyed in a text as the beliefs, opinions or attitudes that are formed or activated with reference to the referents of the discourse (i.e. an event, a situation or participants). therefore, we classify causality and intentionality as subjective dimensions, and the temporality, spatiality and protagonist as objective dimension, which is free of individual’s influence (figure 5). figure 5: subjectivity and objectivity of situation models australasian journal of information systems wang, alahakoon & de silva 2018, vol 22, research on recent advances in social media extended cognitive situation model 11 unlike objective dimensions which is always declarative and static in texts, subjective factors are not always directly presented in the source text and are constructed with background knowledge and experiences, psychological conditions and impact of social knowledge. moreover, in an information ecosystem impacted and influenced by social media data, event related information being interpreted is no longer purely dependent on the original source text. in many cases, the subjective factors related to an event described in a source article is subject to re-evaluation and interpretation based on social viewpoints related to the event and individual experiences and knowledge overtime. therefore, the collective and augmentation of the social view is subject to change overtime accordingly. figure 6: extended situation model australasian journal of information systems wang, alahakoon & de silva 2018, vol 22, research on recent advances in social media extended cognitive situation model 12 extended situation models to adapt the situation models into the global conversation environment and to model the knowledge ecosystem empowered by social media, we enhanced the existing model with the following features (figure 6): 1. the original situation models layer serves as the core framework of events for knowledge construction and enhancement. 2. the social view layer represents the collective opinion of the society. 3. individual opinions related to events are extracted from contents or activates generated by individual online users and anchored to corresponding events based on eventindexing theory. 4. the streams of individual opinions are analysed, categorized, and augmented as the collective social views, such sentiment, causation, intentionality to support analysis at more granular level. 5. the source comment, extracted individual opinions, categorized and augmented social opinions serve as the social opinion repository. 6. the collective social views are coordinated with related situation model of events, which provides an enriched layer of understanding of the subjective dimensions of the events. on the other hand, the evolution of social opinion also influence the outcome of individual interpretation of an event. 4 construction of the adapted situation models 4.1 sample data collection to demonstrate and validate the extended situation model, we gathered a sample dataset from twitter published by new york times’ official accounts (nytimes) about the topic “turkey coup” during 2016-07-15 to 2017-07-16. 33 tweets were collected, analysed and applied to the proposed model (appendix 1). 4.2 core situation models construction the core situation models construction follows the multi-layer processes from surface code, textbase to situation models. firstly, the explicit wording and syntactical rules were analysed to identify individuals, objects, concepts and the corresponding relations. then propositions are extracted based on the surface layer analysis to be networked as textbase via overlapping dimensions as illustrated in figure 7. australasian journal of information systems wang, alahakoon & de silva 2018, vol 22, research on recent advances in social media extended cognitive situation model 13 figure 7: textbase extraction example the situational dimensions including time, space, protagonist (people, organization), causality and intentionality were extracted based on the textbase, so that event and event-related dimensions are integrated and coordinated according to similarities of situational settings along the orderable tempo-spatial frameworks. in this experiment, significant situational dimensions were identified based on the frequencies of occurrence in the texts and plotted in a networked representation of situation models. two tempo-spatio situation frameworks are embedded in the investigated sample text:[time=15/07/16; space=turkey] and [time=16/07/16; space=turkey]. the top frequent protagonists include recep tayyip erdogan, fethullah gulen, the military and binali yildirm. given situation framework [time=15/07/16; space=turkey], the related events are connected via predicates or verbs as the central building blocks which links the propositional information and establishes coherence of understanding (figure 8). there are two subjective factors revealed in the illustrated model, as shown in the left section of figure 8. noticeably the causality factor [c: fethullah gulen has a causal role in the coup] demonstrates contradictory views from two tweets (table 1). australasian journal of information systems wang, alahakoon & de silva 2018, vol 22, research on recent advances in social media extended cognitive situation model 14 figure 8: integrated situation model [time=15/07/16; space=turkey] table 1: example tweets being examined for the subjective factor without external knowledge, the integrated situation model itself is insufficient to provide a clear view about the subjective factor. therefore, we introduce additional crowd opinions about the event to enhance the existing knowledge to clarify the ambiguous dimension with the adaptive situation models. 4.3 adaptive situation model extension to obtain more knowledge about the subjective causality dimension of “fethullah gulen has a causal role in the coup”, we extracted users’ 46 comments in english about two tweets (table 1) related to the stated causality of the events (appendix 2). comments were categorized as positive, negative or neutral opinions depending on relevance with the investigated subjective factor (appendix 2). for each comment(ci) in given comment set c, if the opinion support subjective factor in positive, negative or neutral australasian journal of information systems wang, alahakoon & de silva 2018, vol 22, research on recent advances in social media extended cognitive situation model 15 manner, we assign value 1 under corresponding category, and the rest categories with value 0. we define number of ‘likes’ for comment ci as li. and the augmented positive, negative and neutral opinions as opos, oneg and oneu: 𝑂𝑂𝑝𝑝𝑝𝑝𝑝𝑝 = ∑ (𝑝𝑝𝑝𝑝𝑝𝑝𝑖𝑖𝑖𝑖∈𝐶𝐶 × 𝑙𝑙𝑖𝑖) ( 1 ) 𝑂𝑂𝑛𝑛𝑒𝑒𝑔𝑔 = ∑ (𝑛𝑛𝑛𝑛𝑛𝑛𝑖𝑖𝑖𝑖∈𝐶𝐶 × 𝑙𝑙𝑖𝑖) ( 2 ) 𝑂𝑂𝑛𝑛𝑒𝑒𝑛𝑛 = ∑ (𝑛𝑛𝑛𝑛𝑛𝑛𝑖𝑖𝑖𝑖∈𝐶𝐶 × 𝑙𝑙𝑖𝑖) ( 3 ) we define confidence level of the augmented positive, negative and neutral opinion as: 𝐶𝐶𝑝𝑝𝑛𝑛𝑝𝑝𝑝𝑝𝑝𝑝 = 𝑂𝑂𝑝𝑝𝑝𝑝𝑝𝑝 ∑ (𝑙𝑙𝑖𝑖𝑖𝑖∈𝐶𝐶 +1) ( 4 ) 𝐶𝐶𝑝𝑝𝑛𝑛𝑛𝑛𝑒𝑒𝑔𝑔 = 𝑂𝑂𝑛𝑛𝑛𝑛𝑛𝑛 ∑ (𝑙𝑙𝑖𝑖𝑖𝑖∈𝐶𝐶 +1) ( 5 ) 𝐶𝐶𝑝𝑝𝑛𝑛𝑛𝑛𝑒𝑒𝑛𝑛 = 𝑂𝑂𝑛𝑛𝑛𝑛𝑛𝑛 ∑ (𝑙𝑙𝑖𝑖𝑖𝑖∈𝐶𝐶 +1) ( 6 ) the augmented confidence level for positive, negative or neural view about “fethulah gulen’s causal role” are 0.41, 0.21 and 0.38 respectively. the results indicate that 41% users hold a positive view of ‘fethulah gulen’s causal role’ in the turkey coup, and 21% and 38% hold negative and neural view respectively. it implies that although the official tweets from new york times present contradictory information and ambiguous judgement about the examined subjective factor, the involved social media community demonstrate strong bias that positively supports the view. 5 discussion this paper has (a) proposed a novel the representation of event-centric knowledge by extending situational and event extending models from psychology (b) outlined a mechanism of how events can be constructed and updated with continuous textual information streamed from heterogeneous forms of media especially magnified by social media, and (c) emphasised how the mechanism could result in enhancing the understanding of the subjective aspects of events with the dynamics of social opinions. we re-evaluated the definitions of events and borrowed ideas from psychological researches in text comprehension and event cognition fields. this study is based on the event-indexing situation model theory which construct events with five situational dimensions and utilize them as indices for event inference and updating. acknowledging the strength of situation model for capturing situational context to coordinate the integration of segmental text into coherent set of beliefs, we distinguished the subjective and objective natures of the situational dimensions and the different types of knowledge and the sources of information for such knowledge these dimensions are built upon. we also reiterate the importance of enhancing the understanding of subjective factors of events with social knowledge. therefore, we have extended the model with a social view layer to represent the interactive and evolutional features of social opinions from social media. the purpose of the adapted situation model is to 1) capture events with situational context, 2) establish associations of events with involved elements and associations across events, 3) revealing evolution of events with time features, 4) coordinate dynamic social opinions to corresponding events through situational dimensions and enrich the necessary social knowledge with the augmented social opinions over time, 5) enable fluid knowledge australasian journal of information systems wang, alahakoon & de silva 2018, vol 22, research on recent advances in social media extended cognitive situation model 16 augmentation through online construction and updating of events with textual sources of diverse formats streaming from heterogeneous media. we have experimented the model with a sample of real data gathered from new york times’s official twitter account. the case study demonstrated the construction of situation models via layered text analysis and representations. with situation model, we have re-established the events with corresponding situational settings back into sequential orders, i.e. temporal or causal order. it also reflected the contradictory views of subjective factors among different parties involved. relevant social opinions were extracted and analysed, and was anchored to the specific subjective dimensions of the related events to enrich the social knowledge aspect. we observed that the social opinion layer of the model indicates a noticeable biased view of the examined subjective factor, which could be quite dissimilar from the views conveyed by the investigated mainstream media. the extended social knowledge provides a means to investigate the influencing and recursive interactions between the core situation model layer and the social view layer. we did not elaborate the model with this feature as it was out of scope for the research presented in the paper. the dynamics of individual social view plays an increasingly important role in the evolution of public opinion as a collective impact factor. researches also demonstrate that emotions embedded in the contents (berger & milkman, 2012), types and size of media, interaction patterns of the information system are all contributive factors on the formation, trends and diffusion of the public opinions(quattrociocchi, caldarelli, & reports, 2014). how to recognize the behavioural patterns of information exchanging between the situation model layer and social view layer and how public and individual opinions evolves accordingly is still an open question and attracts continuous research interests in this area (bakshy, messing, & science, 2015; berger & milkman, 2012; quattrociocchi et al., 2014). references adolphs, r. (2009). the social brain: neural basis of social knowledge. annual review of psychology, 60, 693-716. aguilar, j., beller, c., mcnamee, p., van durme, b., strassel, s., song, z., & ellis, j. (2014). a comparison of the events and relations across ace, ere, tac-kbp, and framenet annotation standards. paper presented at the proceedings of the second workshop on events: definition, detection, coreference, and representation. ahn, d. (2006). the stages of event extraction. paper presented at the proceedings of the workshop on annotating and reasoning about time and events. allan, j. (2002). introduction to topic detection and tracking. topic detection and tracking, 1-16. allan, j., carbonell, j. g., doddington, g., & yamron, j. (1998). topic detection and tracking pilot study final report. topic detection and tracking pilot study final report. anderson, r. c. (1984). role of the reader’s schema in comprehension, learning, and memory. learning to read in american schools: basal readers and content texts, 29, 243-257. baker, c. f., fillmore, c. j., & lowe, j. b. (1998). the berkeley framenet project. paper presented at the proceedings of the 36th annual meeting of the association for computational linguistics and 17th international conference on computational linguistics-volume 1. australasian journal of information systems wang, alahakoon & de silva 2018, vol 22, research on recent advances in social media extended cognitive situation model 17 bakshy, e., messing, s., & science, a.-l. a. (2015). exposure to ideologically diverse news and opinion on facebook. science. barclay, j. r., bransford, j. d., franks, j. j., mccarrell, n. s., & nitsch, k. (1974). comprehension and semantic flexibility. journal of verbal learning and verbal behavior, 13(4), 471-481. berger, j., & milkman, k. l. (2012). what makes online content viral? journal of marketing research, 49(2), 192-205. brüggemann, d., hermey, y., orth, c., & schneider, d. (2016). topic detection and tracking system. topic detection and tracking system. cambria, e., & white, b. (2014). jumping nlp curves: a review of natural language processing research. ieee computational intelligence magazine, 9(2), 48-57. chambers, n., & jurafsky, d. (2008). unsupervised learning of narrative event chains. acl. crowley, j. l., reignier, p., & barranquand, r. (2009). situation models: a tool for observing and understanding activity. paper presented at the in workshop people detection and tracking, held in ieee international conference on robotics and automation, kobe, japan. dijk, v. t. a. (1977). semantic macro-structures and knowledge frames in discourse comprehension. cognitive processes in comprehension. dijk, v. t. a., kintsch, w., & dijk, v. t. a. (1983). strategies of discourse comprehension. strategies of discourse comprehension. doddington, g. r., mitchell, a., przybocki, m. a., ramshaw, l. a., strassel, s., & weischedel, r. m. (2004). the automatic content extraction (ace) program-tasks, data, and evaluation. paper presented at the lrec. gangemi, a., hassan, e., presutti, v., & reforgiato, d. (2013). fred as an event extraction tool. paper presented at the proceedings of the workshop on detection, representation, and exploitation of events in the semantic web. garnham, a. (1987). mental models as representations of discourse and text: ellis horwood. graesser, a. c., gernsbacher, m. a., & goldman, s. r. (2003). handbook of discourse processes: routledge. graesser, a. c., millis, k. k., & zwaan, r. a. (1997). discourse comprehension. annual review of psychology, 48(1), 163-189. gruber, t. (2008). collective knowledge systems: where the social web meets the semantic web. web semantics: science. hermida, a., fletcher, f., korell, d., & logan, d. (2012). share, like, recommend: decoding the social media news consumer. journalism studies, 13(5-6), 815-824. hogenboom, f., frasincar, f., kaymak, u., de jong, f., & caron, e. (2016). a survey of event extraction methods from text for decision support systems. decision support systems, 85, 12-22. johnson-laird, p. n. (1983). mental models: towards a cognitive science of language, inference, and consciousness: harvard university press. australasian journal of information systems wang, alahakoon & de silva 2018, vol 22, research on recent advances in social media extended cognitive situation model 18 kietzmann, j. h., hermkens, k., mccarthy, i. p., & silvestre, b. s. (2011). social media? get serious! understanding the functional building blocks of social media. business horizons, 54(3), 241-251. kingsbury, p., & palmer, m. (2002). from treebank to propbank. paper presented at the lrec. kintsch, w. (1988). the role of knowledge in discourse comprehension: a constructionintegration model. psychological review, 95(2), 163. kintsch, w. (1998). comprehension: a paradigm for cognition: cambridge university press. kintsch, w., & dijk, v. t. a. (1978). toward a model of text comprehension and production. psychological review. kintsch, w., & van dijk, t. a. (1978). toward a model of text comprehension and production. psychological review, 85(5), 363. langer, j. a. (1984). examining background knowledge and text comprehension. reading research quarterly, 468-481. piaget, j., & duckworth, e. (1970). genetic epistemology. american behavioral scientist, 13(3), 459-480. quattrociocchi, w., caldarelli, g., & reports, s.-a. (2014). opinion dynamics on interacting networks: media competition and social influence. scientific reports. doi:10.1038/srep04938 radvansky, g. a., & zacks, j. m. (2014). event cognition: oxford university press. sanford, a. j., & emmott, c. (2012). mind, brain and narrative: cambridge university press. schoder, d., gloor, p. a., & metaxas, p. t. (2013). social media and collective intelligence— ongoing and future research streams. ki-künstliche intelligenz. tanev, h., piskorski, j., & atkinson, m. (2008). real-time news event extraction for global crisis monitoring. natural language and information systems, 207-218. tannen, d. (1981). indirectness in discourse: ethnicity as conversational style. discourse processes, 4(3), 221-238. van dijk, t. a. (1977). semantic macro-structures and knowledge frames in discourse comprehension. cognitive processes in comprehension, 332. van dijk, t. a. (1982). opinions and attitudes in discourse comprehension. in advances in psychology (vol. 9, pp. 35-51): elsevier. zacks, j. m., speer, n. k., swallow, k. m., braver, t. s., & reynolds, j. r. (2007). event perception: a mind-brain perspective. psychological bulletin, 133(2), 273. zwaan, r. a. (1999). five dimensions of narrative comprehension: the event-indexing model. narrative comprehension, causality, and coherence: essays in honor of tom trabasso, 93-110. zwaan, r. a. (2016). situation models, mental simulations, and abstract concepts in discourse comprehension. psychonomic bulletin & review. zwaan, r. a., langston, m. c., & graesser, a. c. (1995). the construction of situation models in narrative comprehension: an event-indexing model. psychological science. doi:10.2307/40063035 australasian journal of information systems wang, alahakoon & de silva 2018, vol 22, research on recent advances in social media extended cognitive situation model 19 zwaan, r. a., langston, m. c., & graesser, a. c. (1995). the construction of situation models in narrative comprehension: an event-indexing model. psychological science, 6(5), 292297. zwaan, r. a., & radvansky, g. a. (1998). situation models in language comprehension and memory. psychological bulletin, 123(2), 162-185. doi:10.1037/0033-2909.123.2.162 appendices appendix 1: sample tweets of topic "turkey coup" (15/07/16-16/07/16)) australasian journal of information systems wang, alahakoon & de silva 2018, vol 22, research on recent advances in social media extended cognitive situation model 20 appendix 2: examine subjective dimension c(ausality) with example copyright: © 2018 wang, alahakoon & de silva. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems wang, alahakoon & de silva 2018, vol 22, research on recent advances in social media extended cognitive situation model 21 1 introduction 2 related work 2.1 natural language processing (nlp) 2.2 event extraction techniques 3 theoretical basis 3.1 psychological events models of text understanding 3.2 situation models: event based mental representation of text understanding 3.3 event-indexing model: mechanism of constructing situation models 3.4 new information exchange environment with social media 3.5 rethinking of situation model theory with social media 3.5.1 subjectivity and objectivity of situation models 4 construction of the adapted situation models 4.1 sample data collection 4.2 core situation models construction 4.3 adaptive situation model extension 5 discussion microsoft word ajis_18(3)_v10.docx australasian journal of information systems volume 18 number 3 2014 347 erp institutionalisationa quantitative data analysis using the integrative framework of is theories azadeh pishdad school of information technology and mathematical science university of south australia adelaide, australia email: azadeh.pishdad@mymail.unisa.edu.au andy koronios school of information technology and mathematical science university of south australia adelaide, australia email: andy.koronios@unisa.edu.au blaize horner reich beedie school of business simon fraser university vancouver, canada email: breich@sfu.ca gus geursen school of information technology and mathematical science university of south australia adelaide, australia email: gus.geursen@unisa.edu.au abstract there is a wide agreement that it projects have disappointing success rates and often generate less value than originally promised. in the context of erp systems, the same statistical reports exist which demonstrate an overwhelming number of failures in erp implementations. a thorough review of is literature, however, leads us to believe that organisations that broadly deploy and routinise it (in particular, erps) into their day-today work procedures realise the greatest productivity benefit and business values, and in return perceive to be more successful. the stage wherein erp is fully assimilated, widely accepted and routinised is also referred to as institutionalised erp in the extant is literature of institutional theory. as a result, the authors of this paper believe that studying the influence of various social, environmental, technological and organisational factors on erp institutionalisation has significant potential in improving the chance of successful erp projects. in doing so, this paper introduces an integrative framework of is theories based on an in-depth review of is literature. the survey instrument is developed to gather data on possible impacts of factors derived from each theory on erp institutionalisation. the gathered data is then analysed using quantitative data analysis methods to shape the final hypothetical inferences. finally, based on the data analysis results, this paper proposed valuable suggestions to business and it managers to improve the chance of erp success in their organisations. keywords: enterprise resource planning (erp) system, erp institutionalisation, information systems (is) theories, critical success factors australasian journal of information systems volume 18 number 3 2014 348 introduction enterprise resource planning (erp) system allows for streamlined processing of business data and cross-functional integration among a wide range of software products supporting day-to-day business operations and decision-making tasks (bajwa et al. 2004; dezdar and sulaiman 2009). erp systems have emerged as an enabling technology about two decades ago and still are very important elements for modern business organisations. nevertheless, the extent is literature shed light on the fact that despite the critical role of erp and its related activities in bringing value to the business, there is no doubt that many organisations are still struggling with it (chatterjee et al. 2002; xue et al. 2004; dezdar and sulaiman 2009; gartner 2014). this justifies the importance of studying various factors which are influencing successful erp implementation and use. at the same time, the extensive review of information system (is) literature suggests that researchers typically study the process of technology adoption, implementation and assimilation, and thus, little attention has been given to study technology institutionalisation process. this process is the result of fully assimilated and integrated technology which occurs when technology usage becomes stable, routinised and embedded within the organisation’s work processes and its value chain activities. studying technology institutionalisation, therefore, is value worthy for two reasons. first, it is a recent phenomenon in is domain. second, a thorough analysis of is literature suggests that the significant effects of information technologies (it) on productivity of organisations will only be realised if, and when, they are widely spread, utilised and institutionalised in the organisation (purvis et al. 2001; liang et al. 2007; baptista 2009). nevertheless, institutionalisation of complex technologies (like erps) is never easy and a myriad of environmental, social, organisational, and technological elements blend together to influence how potential adopters make sense out of technology, assimilate its use, and finally institutionalise it (soh et al. 2000; purvis et al. 2001; xue et al. 2004; pollock and williams 2008). this paper aims to study the effect of most cited theories in technology adoption, implementation, and assimilation on it/is (in particular, erp) institutionalisation. for this purpose, first, various theoretical/empirical literatures were searched and systematically organised. these activities lead us to propose an integrative framework of is theories for studying erp institutionalisation. the validity of this framework is then tested using quantitative data analysis methods (i.e., factor analysis and regression) on the gathered survey/questionnaire data. this analysis leads us to shape the final quantitative hypothetical inferences of this paper. literature review this section reviews the extant information system literature in order to develop an understanding of the conceptual basis and theoretical background of it/is implementation, assimilation and institutionalisation in contemporary business organisations. in so doing, the effort was made to engage critically with literature to realise, analyse and critique the current state of knowledge in finding the reasons which can be associated with high failure rate of erp implementations in today’s organisations. through this process, first, the erp assimilation process and its various stages are explained followed by reviewing the current state of theoretical knowledge of it/is planning, adoption and use. this review is both crucial and conductive, since each of the explained theoretical lenses opens up new ways of looking at erp assimilation process and its success/failure within organisation. the review of is theories has directed this research to the understanding of institutionalisation process as one of the core elements of institutional theory. this understanding further leads the authors to critique the extent is literature because of its disproportionate attention to studying it/is planning, implementation, and assimilation process comparing to its institutionalisation. the literature review section finally has concluded with the definition of the process of institutionalising it/is (in particular, erps). australasian journal of information systems volume 18 number 3 2014 349 erp assimilation most of the researchers to a large extent agree that technology assimilation process extends from initial awareness to full utilisation and usage of the technology within organisation (see for example, cooper and zmud 1990; chatterjee et al. 2002; rogers 2003; bajwa et al. 2004; zhu et al. 2006; liang et al. 2007). this research concurs with zhu et al. (2006) who suggest initiation, adoption, and routinisation as the core elements of it assimilation process. these three steps embody the pre-implementation, implementation, and post-implementation stages of erp assimilation, and thus, provide a more comprehensive foundation for this process. looking at the temporal relationship of these stages, implementation (physical installation) takes between one to three years (21 months on average), with benefits starting to accrue in an average of 31 months (post-implementation stage) (chatterjee et al. 2002; liang et al. 2007; saraf et al. 2013). later in the literature review section, erp institutionalisation is introduced as a result of fully assimilated erp system (it takes about five to six years for an erp module to become completely ingrained, routinised, and institutionalised). theoretical background used in development of the survey instrument many theories have been developed and used in is research (larsen et al. 2014). the theoretical focus of this research, however, is on theories about technology adoption, implementation, and assimilation. in this section, the most cited is theories such as social shaping of technology (mackenzie and wajcman 1985), socio-technical theory (latour 2005), diffusion of innovation (rogers 2003), task-technology fit theory (goodhue and thompson 1995), technology-organisation-environment framework (tornatzky and fleischer 1990), and institutional theory (dimaggio and powell 1983) are reviewed. these theories shape the main pillars of survey instrument developed for doing this research. in the elaboration of each theory, the details of predictors (or indicators) used to study the effect of that theory on erp institutionalisation also have been discussed. these predictors describe various initiation, adoption, and routinisation stages of erp assimilation. for example, the push from government, suppliers or customers of an organisation in decision to adopt erp is a coercive mechanism through initiation stage, however, the same pressures in using erp is related to routinisation stage. finally, the extensive review of is theories is concluded with a subsection on the definition of erp institutionalisation. according to socio-technical theory, information systems are social systems and their use and acceptance in the organisation is fashioned by the human interpretation of technology. assimilation and performance of technology can, therefore, only be attained if the social and technical subsystems are brought together and treated as interdependent aspects of a work system (orlikowski 2007). in fact, organisations need to find the fit between their technical and social sub-institutions in order to build an integrated harmonised organisation. in line with socio-technical theory, the theory of social shaping of technology (mackenzie and wajcman 1985; latour 2005) also explores the effects of social and cultural factors on the content of technology and the processes involved in introduction, implementation and use of technological innovations. it views it/is as a socially constructed product shaped by the social and cultural environment of its creation and use (soh et al. 2000). technological and social contexts, therefore, cannot be treated as separate phenomena; rather the definition of technology must become embedded within the social arrangements. for example, orlikowski (2007), viewed organisational practices as ‘sociomaterial’ and suggests the perspective of constitutive entanglement, i.e., inextricable relationship between humans and technology in organisations. she provided a practical example of information search in google to show how the resulting sociomaterial assemblage that delivers search results to a researcher (through temporarily binding a heterogeneous assembly of distributed agencies together) is both emergent and contingent. in this research, the influence of theories such as sociotechnical theory and social shaping of technology were assessed through three categories in the developed survey instrument comprising of user competency and support mechanism, cultural acceptance, and cultural barrier [refer to table 1]. australasian journal of information systems volume 18 number 3 2014 350 table 1. factors influencing erp institutionalisationtheory of social shaping of technology task-technology fit (ttf) theory was initially originated from the technology-to-performance chain (tpc) model suggested by goodhue and thompson (1995). it suggests that an explanation of is success can be recognised by both the task for which technology is used and the t between task and technology. this model gives a more accurate picture of the way in which technologies, user tasks, and utilisation relate to changes in performance and helps users and organisations to understand and make more effective use of information technologies. ttf theory, therefore, holds that if the capabilities and features of technology match the needs of tasks performed by users, the probability of achieving performance in technology adoption and assimilation is higher (goodhue and thompson 1995; zigurs and khazanchi 2008; seddon et al. 2010). ttf theory combines insights from research on user attitudes as predictors of use or utilisation with the notion of task-technology t as a predictor of performance. the literature related to this theory proposes that information systems (including e.g., technology, policies, and staffs) have a positive impact on performance only when there is correspondence between their functionality and the task requirements of users. moreover, at the organisational level the ‘fit’ between task characteristics and it innovations has been linked with is adoption, utilisation, and usage by various researchers. in this research, the influence of ttf theory on erp institutionalisation in developed survey instrument was measured through thirteen factors divided in to three categories, i.e., ttf initiation (e.g., identifying suitable technology options in alignment with strategic information needs and establishing processes for linking is strategy to business needs), ttf adoption difficulty australasian journal of information systems volume 18 number 3 2014 351 (e.g., configuration issues, functionality gaps, and lack of documentation), and ttf routinisation (e.g., strategic alignment between erp and each business unit and erp system’s alignment with organisation’s overall business strategy) [refer to table 2]. table 2. factors influencing erp institutionalisationtask-technology fit theory the literature on theory of diffusion of innovations (doi) has been broadly defined as the study of how, why, and at what rate technological innovations spread through a population of potential adopters over time (rogers 2003). doi theory sees it innovations as predominantly a process of communication through which potential adopters learn about it innovations and become persuaded to adopt them. according to rogers (2003), there are different categories of people with different levels of motivation to accept technology within the organisation, i.e., innovators, early adopters, early majority, late majority, laggards. the higher proportion of users, however, lies in the early and late majority area. the success of technology assimilation, therefore, depends upon how sooner individuals in the early majority and late majority accept technology and provide the same level of productivity as innovators and early adopters. various factors influence the willingness of adoption of a specific innovation across an entire population. rogers (2003) reviewed hundreds of diffusion studies and identified five generic attributes of an innovation that influence rates of adoption, i.e., relative advantage (like economic advantage, social prestige, and convenience), compatibility (e.g., with values, skills, work practices, and needs of users), complexity (the degree to which an innovation is perceived as difficult to understand and use), trialability (the degree to which an innovation can be experimented with a limited basis), and observability (the extent that the results of an innovation are visible to others). higher values of all these attributes (except for complexity) are favourable to easier implementation within a given organisation. rogers’s synthesis, however, is mainly based on studies of adoptions by individuals (e.g., australasian journal of information systems volume 18 number 3 2014 352 of consumer goods). nevertheless, innovation attributes also play an important role in adoption of is technologies by organisations (cooper and zmud 1990; zhu et al. 2006). in this research, the influence of doi theory on erp institutionalisation was measured through the survey instrument comprising of three attributes; automation capability, compatibility, and ease of use [refer to table 3]. table 3. factors influencing erp institutionalisationtheory of diffusion of innovation the technology-organisation-environment (toe) framework suggests that adoption, implementation, and assimilation of technological innovations are influenced by the technological, organisational, and environmental context of the organisation (tornatzky and fleischer 1990). this framework makes rogers’ innovation diffusion theory (2003) better able to explain intra-organisation innovation diffusion, since it emphasises individual characteristics along with other characteristics of organisation as drivers for organisational innovativeness. at the same time, it goes further and includes another critical component, i.e., environmental context comprising of external elements, constraints and opportunities such as industry norms, regulatory regimes, suppliers support, and customers’ needs. in this research, the influence of toe framework on erp institutionalisation was measured through two separated groups, i.e., technological and organisational. the technological aspect was quantified using eighteen predictors distributed in three categories, such as legacy system shortcomings (e.g., data redundancy, data reformatting, lack of decision quality, and lack of information ownership), technology pre-evaluation criteria (e.g., financial feasibility of technology and technology compatibility), adoption technical difficulties (e.g., the cost of software license and installation, data standardisation and conversion issues, lack of support documentation, data inconsistency, and unreliable hardware) [refer to table 4]. australasian journal of information systems volume 18 number 3 2014 353 table 4. factors influencing erp institutionalisationtechnological aspect of toe framework the organisational context of toe framework had been measured using twenty five predictors distributed in four categories [depicted in table 5], such as organisational adoption success driver (e.g., top management leadership and support, adequate training and education, financial and non-financial incentives to employee, and competency and expertise of it support), organisational pre-evaluation (e.g., organisational information need, cultural issues, and establishing appropriate deadlines/milestone), organisational routinisation success driver (e.g., employee’s involvement, regular operational budget and resource allocation plans, and suitable change management program), and organisational adoption difficulties (e.g. poor understanding of change management process and lack of required resources for maintenance and improvement of erp). australasian journal of information systems volume 18 number 3 2014 354 australasian journal of information systems volume 18 number 3 2014 355 table 5. factors influencing erp institutionalisationorganisational aspect of toe framework finally, the environmental dimension of toe framework was measured using institutional theory which comprises of two elements: institutional effects and institutionalisation process (mignerat and rivard 2009). institutional effects are processes in which institutions affect other institutions, organisations or organisational entities. according to institutional effects, the relationship of an organisation within its institutional environment causes some set of organisations to become more similar over time through resemblance of a focal organisation to other organisations in its environment (dimaggio and powell 1983; scott 2001; mignerat and rivard 2009). dimaggio and powell (1983) introduced three sources of institutional effects to explain how archetypes become similar over time, i.e., coercive, normative, and mimetic. coercive mechanisms arise from the legal environment of the organisation. it occurs through the formal and informal pressures which can be imposed by structures upon which the focal organisation is dependent such as governmental agencies or headquarters. normative mechanisms mostly concern the moral and pragmatic aspects of legitimacy by assessing whether the organisation plays its role correctly and in a desirable way. finally, mimetic mechanisms can be defined as rational or ritualistic imitation of a superior organisation. organisations pursue mimetic behaviour to achieve legitimacy, maintain competitive parity or limit rivalry. in this research, the influence of environmental factors on erp institutionalisation is measured through studying the impact of the institutional effects [as shown in table 6]. institutionalisation process as another element examined in institutional theory has directed this research to define erp institutionalisation phenomenon. the next section provides more detailed description of this process as well as the factors used in the survey instrument to measure it. the literature review section is then followed by an introduction of the primary research framework developed for doing this study. australasian journal of information systems volume 18 number 3 2014 356 table 6. factors influencing erp institutionalisationinstitutional theory (environmental aspect of toe) erp institutionalisation institutionalisation process argues that institutions (such as technological innovations) do not emerge in a vacuum; instead they are created, sustained, and reproduced by individuals through their everyday activities in various social settings (scott 2001; orlikowski 2007; mignerat and rivard 2009). most researchers agree that institutionalisation process occurs along some s-shaped curve that characterises most diffusion paths. this s-shaped curve is defined by a relatively fixed sequence that involves a period of time in which an innovation emerges and is diffused, and then a period in which the innovation australasian journal of information systems volume 18 number 3 2014 357 remains institutionalised (tolbert and zucker 1999; rogers 2003; jennings and greenwood 2003; baptista 2009). since researchers typically study the process of technology adoption, implementation and assimilation, technology institutionalisation is a recent phenomenon. it is the result of fully assimilated and integrated technology which occurs when technology usage becomes stable, routinised and embedded within the organisation’s work processes and its value chain activities. it is similar to institutionalisation stage of the institutionalisation process suggested by jennings and greenwood (2003) or sedimentation stage suggested by tolbert and zucker (1999). institutionalised technology becomes ingrained and absorbed into the work life of the organisation which influences the realisation of the higher productivity benefit, business values, and success (purvis et al. 2001; chatterjee et al. 2002; liang et al. 2007; peppard et al. 2007). to conclude, in the normal progression of events, erp technology is first implemented, then assimilated, and once its usage becomes routinised and embedded within the organisations’ work processes and value chain activities, its use will be taken for granted by various system stakeholders and users (tolbert and zucker 1999; rogers 2003; teo et al. 2003; baptista 2009; maheshwari et al. 2010; saraf et al. 2013). this taken-for-grantedness represents the institutionalisation of erp. in this research, erp institutionalisation was measured by the extent to which this technology is widely used (fully assimilated or widely adopted within the organisation) and become part of everyday life of its users (the extent of erp acceptance). these factors as illustrated in table 7. table 7. factors used to measure erp institutionalisation erp institutionalisation framework in the previous section, a thorough review of erp assimilation and institutionalisation process and the various indicators derived from is theories which may influence this process was presented. it has been argued that the main goal of this paper is to study the influence of discussed is theories on erp institutionalisation, since it has a significant potential in improving the chance of successful erp implementation projects. the literature review has directed this study to propose a research framework, i.e., illustrated in figure 1. the most inner layer of this framework is erp assimilation process. as explained before, the three-stage it/is assimilation process (comprising of initiation, adoption, and routinisation stages) proposed by zhu et al. (2006) is used here. the second layer of the suggested framework is the is theories used in erp assimilation. at this stage, organisations need to ascertain that erp is shaped properly through its interactions with the various social, organisational, cultural, australasian journal of information systems volume 18 number 3 2014 358 and technological contexts of the organisation (considering all different pre-implementation, implementation and post-implementation stages of erp assimilation process). figure 1. integrative erp institutionalisation framework diffusion of innovation, task-technology fit, social shaping of technology, technology-organisationenvironment framework, and institutional theory are the set of theories used to form this layer (the various constructs and indicators in the analysis of each theory were explicated thoroughly in the literature review section. as explained before, these indicators can be related to initiation, adoption, or routinisation stages of erp assimilation process). at this stage, the interactions between various technical, organisational, social, and cultural indicators derived from these theories influence erp system to become widely accepted and/or institutionalised in the organisation. erp institutionalisation is the third layer of the suggested research framework as well as one of the cornerstone elements of institutional theory. as discussed earlier, this process is one of the core elements of the institutional theory (dimaggio and powell 1983; mignerat and rivard 2009). when erp is institutionalised, it will be taken for granted by various organisational members, and therefore, the chance of its success will be increased. the main reason for this success is that erp users become comfortable with this technology and can employ its features effectively in their routine activities without requiring functional consultant or coach support. at this stage also acting in compliance with the institutional effects is viewed as logical by those who share erp technology (baptista 2009; seddon et al. 2010; maheshwari et al. 2010). in fact, coercive, normative and mimetic mechanisms make erp systems to be legally sanctioned, morally governed, and culturally supported (scott 2001; teo et al. 2003; saraf et al. 2013). in response to these institutional effects, however, the need for erp and its upgrading modules will be created (or recreated), which affects the successful assimilation and institutionalisation of this technology within the organisation. australasian journal of information systems volume 18 number 3 2014 359 research methodology this paper adopts a quantitative positivist approach. the research framework explained in previous section was validated using statistical techniques, i.e., factor analysis and regression methods. for this purpose, a survey instrument was initially developed and distributed (either electronically or in person) in four australian erp-adopting organisations who have embarked on erp implementation at least five to six years ago and have both success and failure stories to share (the organisational type includes private utility distributor, privatised public telecommunication, public energy provider, and governmental state based public health care organisation). the questionnaire/survey comprises of four sections which each section aims to study particular measures derived from is theories regarding erp institutionalisation. furthermore, the pilot study was mastered to ensure that content of questionnaire is understandable to target respondents prior to gathering data. a total of 78 surveys were distributed wherein only 30 respondents were returned answered. the response rate was approximately 40% and the sample size used for this study was thirty. various tests was conducted to make sure that despite the relatively small sample size utilised in this study, it is acceptable for drawing preliminary conclusions. the results confirmed that variables under study did not violate normality (the variables derived from factor analysis had acceptable skewness value of between positive and negative 1 or the result of dividing skewness value by std. error of skewness is between positive and negative 1.96), regression models were normally distributed (the histograms of relationship between dependent and independent variable were normally distributed), and factors retrieved through factor analysis had high level of communality (greater than 0.7). the respondents also were chosen from various system users. the demographic of interviewees are including chief/head information/technology officer (10%), it/business/knowledge manager/director/specialist (33%), enterprise/solution architect (20%), business/corporate strategy developer/consultant (10%), system/business support/service manager (10%), and project manager/director (17%). quantitative data analysis results this section aims to discuss the results of quantitative data analysis conducted in this study based on the suggested research framework. as discussed earlier, various is theories shape the main pillars of developed survey instrument for doing this research [the details of predictors/indicators have been explained in the study of each theory]. the goal of this section is to find emerging themes in explaining erp institutionalisation (i.e., dependent variable) using well known theories such as institutional theory, social shaping of technology, toe framework, diffusion of innovation, and task technology fit (as independent variables). towards this goal (quantitative data analysis), the following steps have been taken, 1. factor analysis to select the items (by reducing the number of variablesstatistical conclusion validity) 2. cronbach alpha to confirm inter-correlation among test items which shows internal consistency or reliabilityalso called internal validity. 3. average the items and regress the average against the applicable dependent variable. this helps us to understand how the typical value of the dependent variable (i.e., erp institutionalisation) varies when any one of the independent variables is changing. the results of step one and two for all is theories (independent variables) and erp institutionalisation (dependent variable) are demonstrated in table 8 and 9. australasian journal of information systems volume 18 number 3 2014 360 australasian journal of information systems volume 18 number 3 2014 361 table 8. loadings of key independent variables (indicators derived from is theories) table 9. loadings of key dependent variable (erp institutionalisation) as discussed before, step 3 of the process of quantitative data analysis involves applying regression method to understand how the typical value of the dependent variable (erp institutionalisation) varies when any one of the independent variables are changing (various is theories). for doing this step, a set of research hypotheses was initially defined. then, the regression was applied to find whether the research hypothesis is supported or not. the analysis of regression models also confirm that the sample under study is normally distributed. the rest of this section discusses the final results of quantitative data analysis for each theory. these quantitative inferences further can be utilised as managerial guidelines which aim at improving the organisational readiness over various stages of erp assimilation and its institutionalisation, i.e., when an erp introduces, starts to use, becomes dominant, maintained and improved within organisation. the next section concludes the key remarking points of this paper and suggests four major managerial principles for enhancing the chance of erp success in the modern business organisations. social shaping of technology the quantitative inferences in regards to this theory (shown in table 10) shed light to the fact that the competencies of users through initiation stage (e.g., previous expertise in using legacy systems) do not necessarily influence the success of erp institutionalisation positively, whereas preparing people through adoption stage (e.g., by training them how to use erp and familiarising them with how jobs and processes are going to be changed after implementation) and supporting them through erp routinisation (e.g., by providing users with adequate technical support or using competent it team in maintaining, updating and supporting erp needs) have positive influence on successful institutionalisation of this technology. the practical managerial guideline for this theory, therefore, suggests that organisations who invest in improving user competencies through adoption and routinisation stages have more chance of institutionalising erp than those who mainly rely on recruiting competent is users prior to erp implementation. moreover, the data analysis results regarding cultural aspects of erp implementation infer that some cultural barriers like employee’s resistance to new technological changes (like erp usage) are natural and inevitable, and hence, are not australasian journal of information systems volume 18 number 3 2014 362 critical determinants in assessing the success of erp project. on the other hands, cultural acceptance of this technology is required for successful erp institutionalisation. therefore, managers need to constantly assess the capabilities of erp users in using this technology without direct social control or sanctioning process as well as their capabilities in taking maximum advantage of erp’s suggested features in order to achieve more successful efficient results. table 10. hypothesis testing (social shaping of technology) technology-organisation-environment framework the results in regards to the toe framework (demonstrated in table 11) indicate that the success of erp institutionalisation is both a technological and organisational challenge. nevertheless, these challenges grow with the evolution of erp in the organisation. for example, our results shed light to the fact that considering technological (e.g., assessing technical compatibility of erp and putting in place strategies for preparing technical infrastructure like converting data and other technical pre-test activities) and organisational (e.g., managements’ beliefs in importance of it initiatives, planning for appropriate implementation strategy, and assessing organisational information needs) aspects through pre-implementation evaluation procedure do not necessarily result in experiencing successful outcomes through post-implementation and institutionalisation of erp system. instead these technological and organisational aspects become more an issue through erp adoption wherein physical implementation of this technology takes place. top management, therefore, need to master appropriate strategies for handling these technological (e.g., data inconsistencies issues, lack of support documentation, and unreliable hardware) and organisational (e.g., lack of understanding change requirements and defining suitable programs to manage it) adoption difficulties to reduce the possibility of erp institutionalisation failure. moreover, business and it managers need to consider an aggregated ongoing profile of these toe aspects through routinisation stage (e.g., top management leadership and support, expertise of it support team, user involvement, and mastering regular resource allocation plans for erp maintenance) as well in order to achieve successful fully institutionalised erp system. australasian journal of information systems volume 18 number 3 2014 363 table 11. hypothesis testing (toe framework) institutional theory the results in regards to the institutional theory (depicted in table 12) infer that coercive forces through initiation stage do not meaningfully influence successful erp institutionalisation whereas the same forces through routinisation stage have positive influence on this phenomenon. the gathered data do not show any meaningful relationship between normative forces (through erp initiation and routinisation stage) and erp institutionalisation. finally, mimetic mechanism through initiation stage as the last studied institutional effect has positive influence on erp institutionalisation. these quantitative inferences can be utilised as practical guideline for it/business managers. for example, an organisation cannot solely follow the professional industry norms and trends for technology planning, design and development and expect to have successful fully assimilated erp. there also exist other organisational and technological factors (for example, explained under toe framework) which need to be considered along this journey for achieving efficient results. at the same time, an organisation may initially decides to implement erp to be conformed to regulative forces, however, this does not means that it will achieve a successful legitimate erp. in fact, this conformance needs to be on an ongoing basis through other stages of erp assimilation (particularly, erp usage) in order to be conducive of success. finally, interestingly our results infer that organisations that mimic the best practices of other competitors and successful peers through initiation stage have more chance of achieving successful erp institutionalisation. australasian journal of information systems volume 18 number 3 2014 364 table 12. hypothesis testing (institutional theory) task-technology fit theory the results (shown in table 13) suggest that strategies for aligning business tasks and erp technology need to be considered constantly through all stages of erp assimilation (e.g., initiation: identifying suitable technology options in alignment with strategic information needs, adoption: reducing functionality gaps through rational choice of configuration and customisation strategies, and routinisation: an ongoing assessment of the fit between erp system and the organisation’s work design and business processes through erp usage) in order to achieve an institutionalised it/is infrastructure. in fact, ongoing ttf efforts through various stages of erp assimilation are required to guarantee that this technology is successfully institutionalised within the organisation. australasian journal of information systems volume 18 number 3 2014 365 table 13. hypothesis testing (task technology fit theory) theory of diffusion of innovation the quantitative inferences in regards to this theory (shown in table 14) lead us to believe that positive perceptions of technology attributes (such as relative advantage, ease of use, and compatibility) increase the chance of successful erp institutionalisation. this result re-emphasises the critical role of attributes of it innovations in the context of organisational adoption of complex technologies like erp system. this influence, however, is not only restricted to the initial decision to adopt this technology, but also the ease of going through later stages of adoption such as erp implementation, adaptation, and routinisation. table 14. hypothesis testing (diffusion of innovation) through this section, a discussion on quantitative inferences of this study and some of their managerial implications in improving the chance of successful efficient erp institutionalisation was provided. our results further emphasise that critical success factors as the key areas where ‘things must go right’ for the erp implementation to be successful should not only consider technical aspects but also other institutional and contextual factors, such as internal operational and managerial arrangements around erp implementation, social/cultural readiness and support, and the wider operating environmental within which erp is implemented and utilised. figure 2 depicts all the supported hypothesises of this research at the end of data analysis. australasian journal of information systems volume 18 number 3 2014 366 figure 2. diagram of supported hypotheses (dependent variable: erp institutionalisation) conclusions although successful implementation of erp systems can bring competitive advantage to organisations, the high failure rate reported in research and practice made significant concerns for organisations who either have already implemented this technology or desire to implement it. at the same time, the definition of erp success is controversial, since this technology is complex and touches many areas of the business. therefore, depending on different stakeholders’ viewpoint and the unique characteristics of projects, a wide range of factors can influence erp success (soh et al. 2000; seddon et al. 2010). for example, a business user may define success as the extent of alignment between erp system and tasks required for delivering a particular job or ease of use of working with user interfaces, whereas a project manager may define it through characteristics such as delivery of project within time and budget. research and practice, however, suggest that business organisations are more concerned about the physical implementation of erp system (more a technical challenge), rather than the factors and the cause and effects that help shape the use of this technology as an institutional challenge (peppard et al. 2007; pollock and williams 2008; baptista 2009). that is why it is believed that studying erp institutionalisation as a recent phenomenon is valuable and critical, since it helps organisational stakeholders and system users to make optimal advantage of erp and take its use for granted through the mutual interactions of various organisational sub-institutions such as internal culture, social structure, technical attributes, regulatory agencies, and external competitors. as a result, one of the australasian journal of information systems volume 18 number 3 2014 367 major goals of this research is to study various factors which are influencing efficient successful erp institutionalisation. this research is based in australian settings where there have been no previous studies of institutionalisation of technology. this paper contributes to the extant technology implementation literature by using a set of well-known is theories to construct the proposed erp institutionalisation framework. diffusion of innovation, task-technology fit, toe framework, social shaping of technology and institutional theory are the set of theories used to form the suggested research framework. it emphasises the character, shaping, and use of erp technology through continuous interfacing with technological, organisational, social/cultural, managerial, environmental, and institutional effects. moreover, this framework embraces the pre-implementation, implementation and post-implementation stages of erp assimilation into an integrated structure in order to achieve an institutionalised it infrastructure. using this framework, the authors studied the influence of various factors derived from is theories on erp institutionalisation. methodologically, this paper adopts a quantitative positivism approach to validate the research framework and to shape the final quantitative inferences/results of the study. these results further direct this paper to conclude a list of managerial principles which can guide them to fully assimilate erp into their daily working practices as well as to achieve a full realisation of erp benefits, as are listed below: erp success is more a post-live socio-technical challenge: the major benefits of erp are available when business managers and users are continuously trained and engaged to support its wide spread usage (mainly through physical implementation of erp and precisely when it is in use). besides, the various attributes of erp (like ease of use and compatibility) show off their major influence in post-live through their mutual interactions with system users. thus, technical advantages of erp are more likely to cause efficient results if business people and managers show positive behaviour towards their usage. some immediate negative outcomes of erp implementation are inevitable and should not be considered as hurdle to its success: erp success is a long-term goal, wherein some immediate negative outcomes (such as employee’s resistance) may appear. these negative outcomes, however, are not always a hurdle to success, since some of them are initially natural and unavoidable and some can be utilised in the process of improving organisational learnings. a comprehensive task-technology fit assessment plan need to be available on an on-going basis: ttf efforts need to be started right from initial project kick off (in development of business case) and continues forever to make sure all the initial expected and change benefits have been achieved. the various technological, organisational, and environmental challenges of erp implementation are growing with the evolution of this technology in the organisation: top management need to constantly master continuous standardised benefit realisation plans to control erp outcomes and to fully internalise the benefits of this technology. the best technological selection of the first day may finally end up with unmanageable solution which is difficult to control its use (precisely in post-adoption) predominantly because of underestimating the various post-influential aspects of such this decision. toward wrapping up the concluding remarks of this paper, the authors suggest that an integrative account of all technological, organisational (including various social, cultural, informational elements) and environmental (e.g., institutional effects) factors through various stages of erp assimilation (specifically through post-live stage) are required to be considered by business/it managers to fully assimilate erp into their daily working practices as well as facilitating making strategic business decisions. the authors also believe that acting in accordance with the suggested four managerial australasian journal of information systems volume 18 number 3 2014 368 principles will bring more tangible and intangible implications for all functional, managerial, strategic and organisational areas of contemporary business organisations. references bajwa, d. s., garcia, j. e., and mooney, t. (2004). an integrative framework for the assimilation of enterprise resource planning systems: phases, antecedents, and outcomes. journal of computer information systems, 44(3), 81-90. baptista john, j. (2009). institutionalisation as a process of interplay between technology and its organisational context of use, journal of information technology, 24(4), 305-319. chatterjee, d., grewal, r., and sambamurthy, v. (2002). shaping up for e-commerce: institutional enablers of the organisational assimilation of web technologies, mis quarterly, 26 (2), 65-89. cooper, r. b., and zmud, r. w. (1990). information technology implementation research: a technological diffusion approach. management science, 36(2), 123-139. dimaggio, p.j., and powell, w.w. (1983). the iron cage revisited: institutional isomorphism and collective rationality in organisational fields, american sociological review, 48(2), 147-160. dezdar, s. and sulaiman, a. (2009). successful enterprise resource planning implementation: taxonomy of critical factors, industrial management & data systems, 109 (8), 1037–1052. gartner (2014), working smarter, not harder, is the key to business success with erp, analyst : derek prior, carol hardcastle, heather keltz. goodhue, dale l. and thompson, ronald l. (1995). task-technology fit and individual performance. mis quarterly, 19(2), 213-236. jennings, p. d. and greenwood, r. (2003). constructing the iron cage: institutional theory and enactment, in r. westwood and s. clegg (eds.), debating organisation, malden, ma, blackwell. larsen, k. r., allen, g., vance, a., eargle, d. (eds.) (2014). theories used in is research wiki. from http://istheory.byu.edu latour, b. (2005). reassembling the social: an introduction to actor-network-theory. oxford, uk: oxford university press. liang, h., saraf, n., hu, q., xue, y. (2007). assimilation of enterprise systems: the effect of institutional pressures and the mediating role of top management. mis quarterly, 31(1), 59-87. mackenzie, d. and wajcman, j. (1985). the social shaping of technology: how the refrigerator got its hum (open university press, milton keynes). maheshwari, b., kumar,v. , and kumar. u. (2010). delineating the erp institutionalisation process: go-live to effectiveness. business process management journal, 16(4), 744-771. mignerat, m., and rivard, s. (2009). positioning the institutional perspective in information systems research. journal of information technology, 24(4), 369–391. orlikowski, w. j. (2007). sociomaterial practices: exploring technology at work. organisation studies, 28(9), 1435-1448. peppard, j., ward, j., and daniel, e. (2007). managing the realization of business benefits from it investments. mis quarterly executive, 6(1), 1-11. pollock, n., and williams, r. (2008). software and organisations: the biography of the enterprise-wide system or how sap conquered the world. new york: routledge australasian journal of information systems volume 18 number 3 2014 369 purvis, r. l., sambamurthy, v., and zmud, r. w. (2001). the assimilation of knowledge platforms in organisations: an empirical investigation. organisation science, 12(2), 117-135 rogers, e. m. (2003). diffusion of innovations, 5th ed. free press, new york. saraf, n., liang, h., xue, y., and hu, q. (2013). how does organisational absorptive capacity matter in the assimilation of enterprise information systems?, information systems journal, 23(3), 245267. scott, w. r. (2001). institutions and organisations (2nd ed.). thousand oaks, ca: sage. seddon, p. b., calvert, c., yang, s. (2010). a multi-project model of key factors affecting organisational benefits from enterprise systems. mis quarterly, 34(2), 305-328. soh, c., kien, s. s., & tay-yap, j. (2000). enterprise resource planning: cultural fits and misfits: is erp a universal solution?. communications of the acm, 43(4), 47-51. teo, h. h., wei,k. k., and benbasat, i. (2003). predicting intention to adopt interorganisational linkages: an institutional perspective. mis quarterly, 27(1), 19–49. tolbert, p. s., and zucker, l. g. (1999). the institutionalization of institutional theory. studying organisation. theory & method. london, thousand oaks, new delhi, 169-184. tornatzky, l.g., and fleischer, m. (1990).the processes of technological innovation. lexington books, lexington, massachusetts. xue, y., h. liang, boulton w.r., and snyder c.a. (2004).erp implementation failures in china: case studies with implications for erp vendors. international journal of production economics, 97(3), 279-295. zhu, k., kraemer, k. l., and xu, s. (2006). the process of innovation assimilation by firms in different countries: a technology diffusion perspective on e-business. management science, 52(10), 15571576. zigurs, i., and khazanchi, d. (2008). from profiles to patterns: a new view of task-technology fit. information systems management, 25(1), 8-13. an earlier version of this paper was presented at the australasian conference on information systems (acis) 2013 in melbourne, australia. paper title (paper title style) australasian journal of information systems lamp 2016, vol 20 editorial editorial john lamp ajis editor-in-chief deakin business school deakin university last year heralded some major changes to ajis and i begin this editorial by reviewing their impact. the move to wholly on-line publication on an annual basis has been well received, especially the move to progressive publishing of individual articles immediately on completion of their copyediting, rather than waiting for a six monthly issue. the dramatic reduction in the days to publication (table 1) makes the benefit of progressive publishing very clear. during 2015, the changes to some other areas, such as presentation of articles and indexing of articles also began to be felt. eight general research articles were published in 2015, plus special sections on • business analytics • indigenous use of information and communication technologies • small and medium sized enterprises; and • selected papers from acis one very innovative section introduced last year is the post-publication review (ppr) section. two contributions to the ppr section have been published. this section has been well received by authors and external commentators, and will remain a feature of ajis for the present. a substantial growth in readership has also been experienced (table 2). the more effective indexing of ajis articles lead to a five-fold increase in the number of article downloads, with the biggest increase in readership coming from the usa. a significant contributor to this increase is the better leverage of indexing systems. ajis is now fully integrated into the digital object identifier (doi) system, and doi errors have all but disappeared. article metadata is also uploaded and maintained on the directory of open access journals (doaj). this greater visibility of our articles has completely changed our referral profile. instead of most referrals coming from doaj, referrals are now dominated by google scholar. we look forward to a growth in citations to match the increase in article discovery. ajis is also active on twitter. while relatively new, this is growing as a means of keeping the wider community aware of our activities. we also have an rss/atom feed, which can be integrated into your email clients or other similar software to provide real time information on publication or articles. ajis is now integrated into the orcid system. orcid is an open, non-profit, communitydriven effort to create and maintain a registry of unique researcher identifiers and a transparent method of linking research activities and outputs to these identifiers. orcid is unique in its ability to reach across disciplines, research sectors and national boundaries. it is a hub that connects researchers and research through the embedding of orcid identifiers in key workflows, such as research profile maintenance, manuscript submissions, grant applications, and patent applications. i strongly recommend that you update your ajis registration to include your orcid profile (see http://orcid.org/ ) so that this registration system is automatically updated when an article is published. the five most popular articles in 2015 were: 1 australasian journal of information systems lamp 2016, vol 20 editorial mathrani, s., & viehland, d. (2009). business benefits from enterprise systems implementation in small and medium-sized enterprises. (567 downloads) lin, a., & parinyavuttichai, n. (2015). is project management and risk escalation: towards a dynamic model. (380 downloads) lauer, t., & tanniru, m. (2001). knowledge management audit a methodology and case study. (348 downloads) baskarada, s., & koronios, a. (2013). data, information, knowledge, wisdom (dikw): a semiotic theoretical and empirical exploration of the hierarchy and its quality dimension. (308 downloads) kendall, j., & kendall, k. (2012). storytelling as a qualitative method for is research: heralding the heroic and echoing the mythic. (281 downloads) in december, thomson-reuter contacted ajis to say that they were now indexing our content. this decision was taken without an approach by ajis, which is a very positive development. all of this activity indicates a strong move to the mainstream by ajis, and it is to be hoped that this continues and consolidates. this coming year should be a year of consolidation, and fine tuning our existing procedures. i already have received a number of proposals for special sections, and, very pleasingly some of these are repeats. please remember though, that ajis is australasian from two perspectives – publishing research by australasian researchers and research with a distinctive australasian focus. ajis is ranked “a” by both the australian council of professors and heads of information systems and by the australian business deans council. it is an excellent outlet for your research. this work has been guided by an advisory board drawn from the australasian association for information systems, the australian council of professors and heads of information systems and the professors and heads of information systems, new zealand (table 3). their advice has been invaluable. i would also like to thank all those who have donated their time to reviewing and to managing the review process on ajis submissions. this activity is at the core of producing an academic journal. it is indispensable work, without which ajis would fail. the year ahead is full of potential – the rest is up to you. 2 australasian journal of information systems lamp 2016, vol 20 editorial tables 2013 2014 2015 submissions 78 183 141 rejections 61% 78% * review duration 142 days 129 days 106 days days to publication 377 days 344 days 128 days * the small number of finalised papers received in 2015 makes this measure unreliable at this time. table 1: ajis processing statistics 2013-5 table 2: ajis geographic distribution of contacts aais deborah bunker, university of sydney don kerr, university of the sunshine coast acphis caroline chan, rmit university glenn stewart, queensland university of technology phisnz annette mills, university of canterbury felix tan, auckland university of technology table 3: ajis advisory board copyright: © 2016 lamp. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. 3 http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems lamp 2016, vol 20 editorial 4 paper title (paper title style) australasian journal of information systems sedera 2016, vol 20, research article does size matter? does size matter? the implications of firm size on enterprise systems success darshana sedera queensland university of technology d.sedera@qut.edu.au abstract with the increasing emphasis on globalization and consumer demands in highly competitive markets, organizations have attempted to integrate and standardize across their corporate subsidiaries by implementing enterprise systems (es). in such corporate-wide es implementations, system requirements are dictated by headquarters, favouring large subsidiaries, and thus potentially compromising the unique business requirements of smaller units. this research reports findings from an es initiative within a group of large, medium, and small organizations, where the impact of organization size is assessed in relation to the level of system success and knowledge-management competence. the results of this analysis suggest that (i) large organizations, in contrast to their smaller counterparts, receive greater benefits in terms of both individual and organizational impacts, (ii) while no such differences were observed in relation to system and information quality. similarly, (iii) no differences were observed in relation to knowledge-creation competencies amongst large, medium, and small organizations. however, (iv) differences were observed between the large and smaller firms on their knowledge-retention and knowledge-transfer competencies. keywords: enterprise system; organization size; knowledge management; cluster analysis; multi-group analysis 1 introduction with the increasing emphasis on globalization and highly competitive markets, corporations are looking to enterprise systems (es) as a way of integrating and standardizing across all their corporate subsidiaries (laukkanen, sarpola, & hallikainen, 2007; sedera, lokuge, grover, sarker, sarker 2016). such corporate-wide es implementations are widespread in multi-national companies, and the presence of such es implementations is growing in publicsector organizations. these es initiatives are motivated by the desire to create integrated, standardized, and efficient processes, providing predictable outputs across all subsidiaries (avison & malaurent, 2007). in order to reach this management control “dream,” companies promote organizational discipline both by requiring es users to follow prescribed corporate processes and by limiting their access to specific organizational roles. the substantial growth of these centralized es implementations in large corporations in the past two decades – more than 30,000 companies worldwide, according to mabert et al. (2003) – is a major testament to this quest for standardization, visibility, and control. in multidivisional corporations, enterprise systems are deployed across different geographical, cultural, or organizational boundaries, using what are commonly known as “global templates.” global templates in these es implementations typically omit specific requirements that may vary substantially from one subsidiary to the next, depending on the local standards, rules, norms, and values attaining simplicity and standardization of operational procedures across all organizational entities. global templates tend to favour the standards, structures, goals, and objectives of the parent company or its dominant subsidiaries. as a consequence, when companies fail to account for the specific needs of all their subsidiaries, these neglected subsidiaries are less likely to succeed in the long term (avison & malaurent, 2007). in relation to the ownership costs of full-scale enterprise systems, large companies and their large subsidiaries, due to their economies of scale, are more able to cover these costs, whereas small and medium enterprises (smes) struggle with finding a balance 1 australasian journal of information systems sedera 2016, vol 20, research article does size matter? between these costs and smaller (salim, sedera, sawang, alarifi, atapattu, 2015), easier-to-use systems that may be more suited to their needs (wang & hamerman, 2008). information systems researchers generally agree that, for the purposes of their research, there are three firm sizes: small, medium, and large. however, as goode and gregor (2009) point out that is research lacks a coherent argument on how firms should be grouped based on ‘size’. past is research has employed a myriad of features and heuristics to categorize organizations according to organization size. having studied nearly 2000 research papers over 11 years, goode and gregor (2009) identified 21 constructs underpinning the size construct and 25 heuristics of measuring firm size – with no clear definition or relationship among these measurement constructs. they observed that the most often employed criteria for determining organizational size – the number of employees – is fraught with inconsistencies when determining what is a small, medium, or a large organization1. this diversity of benchmarks has made it almost impossible to derive observations beyond a particular region or country. they also identified the number of the it staff and the size of the it budget as two criteria that closely align with is studies, but found similar inconsistencies to the number-of-employees metric. also, it is often difficult to obtain sensitive budget details in many circumstances. in conclusion, goode and gregor (2009) encouraged is researchers to develop is-specific criteria to group organizations according to “firm size.” reflecting this charge, a secondary motive of this study is the derivation of a repeatable is-specific size classification (described in the data analysis section). this profusion of arguments and diversity of results notwithstanding, all past is researchers agree that large, medium, and small organizations select, implement, use, and manage information systems differently. appendix a extends the discussion of goode and gregor (2009) and summarizes prior studies on firm size in relation to (i) system selection and implementation, and (ii) post-implementation management and outlines the differences between small, medium, and large organizations. the highlighted cells in appendix a show those characteristics that are considered favourable for es implementation and postimplementation success. these summarized characteristics suggest that large organizations have advantages in terms of financial resources, knowledge, and ability to make investments throughout the system lifecycle; and a tolerance of losses, are well-structured, have a sophistication in it, and they follow a formal is acquisition procedure. prior research concludes that such conditions are favourable to enterprise system success. small organizations, on the other hand, have other advantages, including the ease of managing change and experiencing less complexity, which contribute to high levels of es success. the primary objective of this study is to understand whether all organizations, regardless of their size, are capable of deriving similar benefits from corporate-mandated enterprise system initiatives. our study specifically observes (i) the level of enterprise system success (essuccess) throughout the lifecycle of the implementation and subsequent use at the onwards/upwards phase of the es (markus, axline, petrie, & tanis, 2003) and (ii) the organization’s competency in managing knowledge-management initiatives (km-competence) throughout the es lifecycle. the inclusion of km-competence – a widely cited critical success factor for es-success (sedera & gable, 2010) – enables us to study the extent to which different sized organizations are capable of managing the entire es lifecycle, not just the initial implementation phase. 1 for example, while the blemish companies with less than 90 employees are classified as small, in the u.s., a ”small” company must have less than 100 employees. according to european union guidelines, companies with less than 50 employees are classified as small, and companies with less than 500 are considered mid-sized. however, within the same euro-zone, germany defines a small organization as one with less than 250 employees. in the enterprise system context, sap positions its scaled-down es business one® application especially for ”small” organizations, which they define as less than 100 employees. 2 australasian journal of information systems sedera 2016, vol 20, research article does size matter? research on corporate-wide es implementations is of interest to both researchers and practitioners, especially at a time when there is strong growth of multinational companies in high-growth economies like brazil, russia, india, and china (the “bric” countries). for instance, our findings should benefit researchers and practitioners interested in observing mergers and acquisitions that are increasingly common in the global economy. organizationwide es initiatives are also of interest to public-sector organizations. they too, like the organizations in this study, engage in large-scale es initiatives in hopes of achieving efficiency gains across government departments. our analysis of the es literature for the period from 1995 to 2011 in the nine leading is research publication outlets (eight leading is journals2 and the icis proceedings) did not yield any study on multidivisional, corporate-wide es installations. the dearth of research in this important area can perhaps be attributed to the difficulties in gaining access to all the subsidiaries involved in a multidivisional es initiative. moreover, the effect of firm size on es-success is seldom studied, with most es studies concentrating on a single organization. the current study findings are derived from a large multidivisional sap initiative completed in the late 1990s, in which a group of large, medium, and small australian state government departments were brought together under a single sap installation. in this study, the characteristics of a corporate-wide es initiative are narrated using the phenomena of “panoptic gaze” of bentham (1971), cited in semple (1993), and its information system extensions by zuboff (1988). our research model is derived from sedera and gable (2010), who observed a significant positive relationship between km-competence and es-success. the sedera and gable model in the current study is tested with firm size as a moderating variable. the homogeneous context – where all organizational units implemented the same es at the same time, where all organizations were tracked throughout the entire es lifecycle, and where all organizational units were part of the same parent organization – minimized the possible influence of extraneous variables. 2 global templates in enterprise systems in this study, we conceptualize the centralized es implementation through the use of “global templates” using the notions of the “panoptic gaze” (bentham 1791; cited in semple (1993)) and of the “information panopticon” (zuboff, 1988). the panopticon concept, originally based on the work of bentham (1791), refers to a prison architecture that employs a central tower for the observation of the prisoners. the central tower allows the guards to observe, opticon, all, pan, the prisoners without their being able to tell whether they are being watched. the panoptic gaze is a continuous surveillance that is hierarchical; the observers can observe the prisoners and can themselves be observed by their superiors; and one-way, the prisoners cannot see the observer, cannot see each other, and cannot know whether they are being observed3. by applying the panopticon concept to it, zuboff (1988) proposed the term “information panopticon” to describe how computer technology makes work more visible to managers. sia et al. (2002) suggested that the information panopticon lens might produce a better understanding of the impact of es use in organizations, where es users are constantly being monitored by the tracking mechanisms embedded in the system. they argued that the transparency of the es allows management to monitor the activities and decisions of their subordinates. later, elmes et al. (2005) pointed out that an es provides greater control and enhanced management oversight of departments and subsidiaries. thus, enterprise systems, in many ways, inherently supports bureaucratic control and enforces organizational discipline by requiring organizations and their employees to follow prescribed processes with access to 2 european journal of information systems, information systems journal, information systems research, journal of information technology, journal of management information systems, journal of strategic information systems, journal of the association for information systems, and mis quarterly. 3 zuboff (1988) provides the closed-circuit television camera surveillance in public spaces as an example of how technology brings a panoptic gaze into the daily lives of citizens. 3 australasian journal of information systems sedera 2016, vol 20, research article does size matter? transactions restricted to their particular roles and responsibilities. elmes et al. further suggest that enterprise systems, in comparison to earlier technologies, reflect the view of panopticon control through the integrated database that captures real-time information for monitoring and tracking user actions. global templates in es initiatives help organizations achieve these objectives. such templates purposefully omit any specific local organizational requirements (elmes et al., 2005; sia et al., 2002). given the objectives of adopting a single es for an entire organization – replacing disparate local legacy systems, integrating multiple business processes, standardizing business process and reporting requirements, standardizing data inputs and outputs, storing data in a common database, and ensuring information accessibility on a real-time basis across organizational boundaries (laukkanen et al., 2007) – corporate headquarters or the lead organization demands minimal local customization at the subsidiary level (avison & malaurent, 2007). the extent to which the lead organization manages to achieve a single global template with no or little customization at the subsidiary level increases its management control. in our study, the phenomenon of management control attained through global templates administered by a lead organization is examined using the concepts in information panopticon. 3 the study context this study gathers data from a large-scale enterprise system initiative at a group of australian state government departments that implemented sap as a corporate-wide information system solution in the late 1990s. the es initiative brought twenty-seven (27) state government departments under a single sap implementation. the treasury department, one of the largest departments in the state, championed the initiative and played the central role of the “watch tower,” providing overall management, from initial user requirements analysis (ura), to implementation, to continuing management of the system. this es initiative is reported to be the first corporate-wide, public-sector sap initiative in the world. it is also the first time an australian state government has implemented a common management software platform statewide. the key events of the initiative are described below, in order to provide the study context and to demonstrate the role and influence that the treasury department played as the lead organization in this organization-wide es initiative. in 1993, the state treasury department conducted a user requirements analysis (ura) that revealed substantial differences among the individual systems within each department. the ura also identified the need for a statewide system to provide better decision-making; streamlined workflow automation; and integrated asset management, statewide project costing, sales and distribution, and procurement. then in 1994, the treasury department sent requests for information (rfi) to key es vendors with two mandatory requirements for all rfi submissions: (1) the ability to quickly and easily adapt to changes in organizational structures and business environments, through restructuring business units by merging or eliminating complete departments; and (2) the ability to standardize both cash and accrual accounting across all 27 departments. in october 1994, requests for offers (rfo) were sought from three short-listed es vendors; and the treasury department selected sap r/3 as their statewide enterprise system in december 1994. at the commencement of the implementation process, the treasury department stated seven specific strategic requirements for the es initiative. these requirements were communicated to the implementation partner (a consulting firm), the software vendor, and to all state departments. the requirements of the es initiative were as follows: (1) rationalize the legacy management systems with standardized enterprise software; (2) support statewide financial management through consolidation, integration, and standardization; (3) re-engineer business processes and encourage best practice resource management; (4) improve the quality of data for management purposes; (5) meet emerging whole-of-government needs; (6) achieve economies of scale in operations; and (7) enhance data collection and planning through system 4 australasian journal of information systems sedera 2016, vol 20, research article does size matter? integration. the sap implementation was to be completed at all of the installations by the end of 1995. as pointed out by madapusi et al. (2005), implementation of such strategies require high levels of control, which in this case were provided by the state treasury department, employing global templates to control the entire implementation process from ura, rfi, to rfo processes, and the final selection of the system. the es implementations were structured around five centralized functional “hubs,” with strategic decision-making centralized to the treasury department. these five hubs were created within and managed by the five largest departments within the state government, with some limited freedom to deviate from the global es templates, based on the needs, strategies, products, and common services within each of the hubs. however, any changes to the global templates at the hub level were strongly influenced by the treasury department. this demonstrates the influence of large organizations on global templates, similar to that reported in prior research (clemmons & simon, 2001; madapusi & d'souza, 2005; madapusi & d’souza, 2007; zuboff, 1988). in 1998, despite the anticipated benefits, one relatively small-sized department that provided corporate it services to a group of other small agencies expressed its dissatisfaction with the sap enterprise system. it argued that, despite its rich functionality, sap was too complex and costly to maintain and too rigid to make the modifications needed to suit the unique needs of the departments it was serving. after nearly three years of attempting to use the sap es, this small department was approved by the treasury department to replace their sap es with a locally-produced financeone enterprise system that is specifically designed for smes. figure 1 depicts the key historical events of the organization-wide es implementation. figure 1: the history of key events and characteristics in the state government es implementation 4 study model, instrument, and data collection the study model (figure 2), shows the two key constructs and their relationship. it also highlights the moderating effect of firm size on both key constructs and their relationship. the model observes a firm-size effect on the outcome, es-success, a key antecedent of es-success (km-competence) and the relationship between the constructs. this depiction of the study model allows us to investigate three key hypotheses: (h1) large organizations receive higher levels of es-success than their smaller counterparts in a organization-wide enterprise system initiative; (h2) large organizations are more competent in km than their smaller counterparts in a organization-wide enterprise system implementation of management services america as the common financial reporting system across all departments 1983 user requirements analysis (ura) for a new common system 1993 separate systems for each department, all systems maintained by a central authority, security, data and functionality different at each department significant functionality gaps between the system and requirements of the government 1993 1994 request for information (rfi) sent to key es vendors decision support system, workflow automation, integrated asset management, integrated project costing, integrated sales and distribution, integrated procurement ura identifies the key gaps for the new system two mandatory conditions of rfi (1) the ability to respond to restructuring of business units with amalgamation or devolution of complete departments and (2) the need to facilitate a standardized financial reporting. 1994 request for offer (rfo) three es vendors submit their offers 1995 commences sap implementation 1998 small departments call for separate systems the world’s first statewide public sector sap installation a group of small departments install financeone 5 australasian journal of information systems sedera 2016, vol 20, research article does size matter? initiative; and (h3) large organizations experience a stronger, more significant relationship between km-competence and the level of success of the enterprise system. figure 2: the study model the dependent variable – es-success – is conceptualized using the constructs of the is success model of delone and mclean (1992), as modified by gable sedera and chan. (gable, sedera, & chan, 2008). it is a formative, multidimensional index comprised of the four dimensions: two quantity dimensions, systems quality and information quality, and two impact dimensions, individual impact and organizational impact. in the gable et al. study (2008), they found evidence of the necessity, additivity, and completeness of these four dimensions, measured using 27 validated items to measure es-success (is-impact). the gable et al. (2008) is-impact evaluation approach suggests that the two quality dimensions are measured at one point in time and the impact dimensions are measured retrospectively at the same point in time at which the quality is assessed. in combination, impact and quality measures represent a complete measure of the information system success (its benefits). the antecedent of es-success – km-competence – is measured using the work of sedera and gable. (2010). they define km-competence using the four main phases of knowledge management, i.e., creation, transfer, retention, and application, that are distinct yet interrelated with competence in each phase, contributing to overall km-competence in an organization (see sverlinger, (2000) for a detailed summary of these knowledge-management phases). the knowledge creation phase corresponds primarily with the planning and implementation stages of the es lifecycle. while it is recognized that knowledge creation continues to occur beyond the es implementation stage, during implementation (and during major upgrades) there is an identifiable peak in new knowledge requirements and related knowledge creation. typically, the necessary expertise is brought to bear by three key players contributing to the es implementation and ongoing support: (1) the client organization, (2) the es software vendor, and (3) the implementation partner (gable, heever, erlank, & scott, 1997; soh, sia, & tay-yap, 2000). according to pan, newell et al. (2007), knowledge transfer channels can be informal or formal, where unscheduled meetings, informal gatherings, and coffee-break conversations are examples of the informal transfer of knowledge. although informal knowledge transfer promotes socialization and can be effective in small organizations, it precludes widespread dissemination (alavi & leidner, 2001) – thus the focus of this study. knowledge retention involves “embedding knowledge in a repository so that it exhibits some persistence over time” (argote, mcevily, & reagans, 2003, p.572). once the knowledge is created, transferred and retained, knowledge application occurs, where individuals apply the knowledge they have learned when interacting with the es. markus system quality information quality individual impact organization impact essuccess knowledge management competence knowledge creation knowledge transfer knowledge application as per gable et al., (2008)as per sedera et al., (2010) firm size knowledge retention h1h2 h3 6 australasian journal of information systems sedera 2016, vol 20, research article does size matter? (2001) suggests that the source of competitive advantage resides, not in the knowledge itself, but in the application of the knowledge. in summary, we argue that knowledge, created mainly during implementation, is subsequently, during post-implementation, transferred and then retained by the organization, and finally applied throughout the es lifecycle (pan & chen, 2005). since each phase of knowledge management makes a unique contribution to km-competence, the constructs are conceived of as dimensions “forming” km-competence as a hierarchical, multidimensional, and formative index. thus, the moderation effect of firm size on individual constructs and the relationship between the constructs, in addition to its moderating effect on the relationship, is appropriate, given that both es-success and km-competence are conceived as formative indexes. following the guidelines suggested by cenfetelli and bassellier (2009), petter et al. (2007), and diamantopoulos (2009), both constructs are conceived of as formative. thus, the 27 measures of es-success and the 10 measures of km-competence: (i) need not co-vary, (ii) are not interchangeable, (iii) cause the core-construct, as opposed to being caused by it (arrows point in), and (iv) may have different antecedents and consequences in potentially quite different nomological nets. in addition to the 37 measures of the two constructs, five criterion items were obtained through the referent studies. one criterion item is aimed at gauging overall km-competence, and the remaining four criteria measure the overall impact of essuccess. all survey items were measured on a seven-point likert scale with the end values “strongly disagree” (1) and “strongly agree,” (7) and the middle value “neutral” (4). the survey instrument also gathered detailed demographic data from the respondents regarding their job titles and descriptions of their daily involvement with the sap system. furthermore, objective information on each of the departments’ number of sap user licenses, hardware and software configurations of the sap systems, and other major systems in place was gathered from treasury department documents. simultaneous data collection from all 27 departments posed significant challenges. in order to ensure a high response rate and adequate representation of all departments, we sought endorsements from the highest officials in the treasury department4. the final survey instrument was then sent to 20 users for feedback on the question layout, clarity, and to determine the completion time. this process led to minor cosmetic changes to the format of the instrument. a respondent sample of 1200 active users was selected to participate in the study, with representation from all departments. the survey yielded a total of 319 responses. nine responses were removed from the data analysis due to incomplete data or perceived frivolity, leaving 310 valid records for data analysis (a response rate of 27%). 5 data analysis having collected data from all 27 organizations, the next step was to classify these organizations into the three groups (i.e., small, medium, large) employing appropriate determinates of firm size. our objective here was to develop a size-based criterion that we could use to group organizations. such criterion should be specific to information systems, yet generalizable beyond the context of the current study. we commenced our data analyses with cluster analysis aimed at exploring a natural scheme of grouping of organizations. once the classification method was established, the respondents were then assigned to one of the three sizes. next, we demonstrated descriptive and comparative statistics to compare the levels of es success (h1) and km-competence (h2) across large, medium, and small departments. hypothesis h3 was tested with a comparative partial least squares (pls) analysis of the three firm sizes using the multi-group analysis 4 the study received formal approval through the signing of a memorandum of understanding outlining responsibilities of both the university and the state government departments, ethics of research, confidentiality, and intellectual property of study findings. 7 australasian journal of information systems sedera 2016, vol 20, research article does size matter? techniques of plouffe et al. (2001). this enabled us to understand the moderating impact of firm size on the relationship between km-competence and es-success (h3). 5.1 cluster analysis: exploring firm size exploratory cluster analysis has the ability to identify “natural” classifications and to derive “common traits” of the respondents in an inductive manner (punj & stewart, 1983). specifically, we employed the two-step cluster procedure that has the ability to reveal natural groupings (or clusters) within a dataset that would otherwise not be apparent, with the loglikelihood as the distance measure (bacher, wenzig, & vogler, 2010). reflecting prior studies, and that “three” may not be the “natural” number of firm sizes, we decided not to include apriori number of clusters and leave the analysis to determine number of clusters automatically. the cluster results of the criterion item that evaluates the overall impact of the system on the organization (overall…the impact of sap on the organization has been positive.) did, however, revealed three natural clusters with spss indicating ‘good’ for cluster quality (bacher et al., 2010). this finding naturally follows the overall hypothesis of the study, that large organizations receive higher levels of es-success than their smaller counterparts in a corporate-wide enterprise system initiative. our argument is that, to the extent that respondents observe similarities in perceived benefits, their responses will then yield “clusters” based on firm size. having yielded three natural clusters categorizing every respondent, we then investigated possible explanations for these three clusters. not being sure of the reasons for the three classes (apart from knowing that they originated from the criterion item), and also to confirm that these three groups are not arbitrary and/or the result of a random outcome, we explored logical explanations using contextual information provided to us by the treasury department. the results of this analysis led to the identification of a direct relationship between the number of sap user licenses and the three clusters. it revealed that all respondents in cluster 1 represented organizations with more than 1000 sap user licenses, all respondents in cluster 2 represented organizations with 200-999 sap user licenses, and 98% of the respondents in cluster 3 represented organizations with less than 200 sap user licenses (except one respondent, whose data were later moved to cluster 3). interestingly, a focus group with the senior management at the treasury department designed to help with data interpretation revealed that they (the treasury department) too employ the same cut-offs as above in their internal management processes. the cut-off values for each group, based on the number of user licenses seem reasonable, with organizations with more than 1000 user licenses considered as large, less than 199 user licenses considered as small, and organizations with between 200-999 licenses considered as medium-sized. the respondent groups resulted through cluster analysis is depicted in table 1. cluster size # respondents # organizations # licenses 1 large 66 5 >1000 2 medium 196 16 200 – 999 3 small 48 6 <199 total 310 27 table 1: the cluster analysis results corresponding to sap user licenses the current classification has several advantages. unlike other commonly employed size classifications (e.g., number of employees, budget size, annual revenue, or profits), our classification and its cut-off points are more relevant to contemporary information systems. secondly, the cut-off points are universal and can be used in any type of business, industry type, or geographic location to determine firm size of es using organizations. furthermore, information on user licenses is less intrusive and sensitive to the organization and can be obtained from multiple sources (client organizations, consultants, or from software vendors). 8 australasian journal of information systems sedera 2016, vol 20, research article does size matter? 5.2 construct validity as depicted in the study model in figure 2, both constructs are specified as formative. although constructs validation is not a focus of this study, it is important that the validity of the two constructs be established before making descriptive and comparative observations. we employ the validation techniques suggested in recent studies of cenfetelli and bassellier (2009), diamantopoulos (2009), and petter et al. (2007). for both es-success and km-competence constructs, first the variance inflation factors (vif) were computed for each measure to assess the existence of multicollinearity between the formative measures. all measures (10 items in km-competence and 27 in es-success) were below the common vif threshold of 10.0 as recommended by kleinbaum et al. (1998). we next identified both constructs through measurement relations (hauser & goldberger, 1971; joreskog & goldberger, 1975) following the prescriptions of jarvis et al. (2003, p.214, figure 5, panel 3). the related validity test employs a multiple indicator multiple causes (mimic) model, using appropriate criterion measures as a reflective indicators, separately for each construct (diamantopoulos & winklhofer, 2001). the initial estimation of the mimic model for km-competence revealed a reasonably good fit, with chi-square = 321, d.f. = 20, rmsea = 0.28, gfi = 0.88, nnfi = 0.79, and cfi = 0.89, and all items having significant t-values. next, evaluating the absolute fit indicators of kmcompetence, the observed standardized rmr value of 0.077 represents a good fit. the mimic model of es-success was developed using the four separate criterion measures as reflective indicators of each of the four es-success dimensions. the statistics for the mimic model using the 27 items evidence good fit with the data (chi-square = 459.12, d.f. = 129, rmsea = 0.014, gfi = .89, nnfi = 0.87, cfi = 0.98, rmr = 0.10, srmr = .088, nfi= 0.97, nnfi = 0.87, ifi = 0.98, and cfi = 0.98). 5.3 es-success: descriptive and comparative statistics table 2 presents the descriptive statistics of the three firm sizes in relation to the four essuccess dimensions. large firms have received higher mean scores than their smaller counterparts for all four es-success dimensions. interestingly, the mean scores of small organisations are relatively higher than those of medium-sized organizations for all four success dimensions. small medium large mean st dev skewness kurtosis mean st dev skewness kurtosis mean st dev skewness kurtosis information quality 4.02 1.46 -0.07 -0.60 3.90 1.31 -0.23 -0.90 4.17 1.34 -0.37 -0.27 system quality 4.03 1.20 -0.41 -0.75 3.77 1.19 -0.21 -0.60 4.12 1.08 -0.42 0.42 individual impact 4.52 1.45 -0.59 0.00 4.47 1.46 -0.60 -0.50 5.10 1.14 -0.30 0.30 organisational impact 3.94 1.28 -0.69 0.21 3.91 1.17 -0.33 -0.45 4.29 0.95 -0.21 1.13 table 2: es-success descriptive statistics table 3 reports the results of paired comparisons of the four success dimensions for the three firm sizes using independent sample t-tests (with results significant at 0.05 levels). the t-test results reveal significant differences between large organizations and smes in relation to the two impact dimensions: individual impact and organizational impact. yet, the t-test results did not observe any differences among the three firm sizes for system quality and information quality. 9 australasian journal of information systems sedera 2016, vol 20, research article does size matter? quality impacts information system individual organization sig / df / t-value* sig / df / t-value* sig / df / t-value* sig / df / t-value* small 0.59 / 242 / 0.53 0.18 / 242 / 1.31 0.85 / 242 / 0.19 0.87 / 242 / .164 medium small 0.56 / 112 / -.58 0.66 / 112 / -.44 0.01 / 112 / -2.41 0.05 / 112 / -1.89 large large 0.15 / 260 / -1.43 0.30 / 260 / -2.09 0.00 / 142 / -3.59 0.00 / 136 / -2.36 medium table 3: results of the independent sample t-test of es-success (*significant at 0.005 level) the following observations are made for the es-success dimensions through findings reported in tables 2 and 3. • small and medium organizations (smes) do not demonstrate significant differences in relation to the four success dimensions. • large and smes do not demonstrate differences for system quality and information quality • large and smes demonstrate significant substantial differences in relation to individual impact and organization impact. 5.4 km-competence: descriptive and comparative statistics table 4 presents the descriptive statistics of the three firm sizes for the four phases of kmcompetence. for knowledge creation during es implementation, our results show similar mean scores for all three firm sizes. however, the mean scores of the two post-implementation knowledge-management competencies (i.e., knowledge transfer and knowledge retention) descend with the decreasing firm size. the knowledge application dimension assesses whether similar patterns were observed in relation to the post-implementation phase of kmcompetence, with large organizations receiving the highest mean score. small medium large mean st dev skewness kurtos is mean st dev skewness kurtos is mean st dev skewness kurtos is k-creation 4.07 0.88 0.58 0.91 3.95 0.90 -0.42 0.43 4.07 0.95 -1.35 2.30 k-retention 3.88 1.46 -0.32 -0.81 3.96 1.27 -0.43 -0.10 4.19 1.25 -0.50 0.02 k-transfer 3.67 1.60 -0.20 -1.02 3.88 1.68 -0.25 -0.88 4.03 1.65 -0.36 -0.80 k-application 4.00 1.70 -0.19 -0.97 3.97 1.47 -0.36 -0.44 4.06 1.60 -0.45 -0.44 table 4: km-competence descriptive statistics table 5 reports the results of the paired independent sample t-tests for the three organizational sizes (significant at the 0.05 level). 10 australasian journal of information systems sedera 2016, vol 20, research article does size matter? implementation post-implementation k-creation k-retention k-transfer k-application sig / df / t-value* sig / df / tvalue* sig / df / tvalue* sig / df / tvalue* small 0.40 / 242 / .84 0.69 / 242 / -.40 0.42 / 242 / -.81 0.91 / 242 / .10 medium small 0.98 / 112 / 0.27 0.01 / 112 / -2.44 0.01 / 112 / -1.78 0.85 / 112 / -.19 large large 0.37 / 260 / -.89 0.00 / 142 / -3.33 0.00 / 136 / -2.39 .688 / 260 / -.402 medium table 5: results of the independent sample t-test of km-competence (*significant at 0.005) following observations are made through table 5. • the independent sample t-tests did not demonstrate significant differences for any pairs of firm sizes in relation to knowledge creation. • significant differences were observed between large organizations and smes for knowledge transfer and knowledge retention dimensions. however, no differences were observed between small and medium organizations for knowledge retention and knowledge transfer. 5.5 implications of the organization-wide es initiative we make several observations related to organization-wide es initiatives, based on our case study, as reflected on the findings reported in tables 1-5. our observations are summarized in figure 3. in figure 3, panel a depicts changes to the es-success dimensions for the three firm sizes across the es lifecycle, while panel b represents the same timeline for km-competence phases. the x-axis of panels a and b denote the key phases of the es lifecycle, employing the three-phased es lifecycle of markus et al. (2003): implementation, stabilization, and shakedown. the x-axis of panel a commences with the go-live date, given that the data collected on es-success were made on the operational enterprise system; while the x-axis of panel b commences with the implementation phase. herein, knowledge creation competency involves generating knowledge with regards to business processes, organization, and software by the client, consultant, and vendor during the implementation phase, prior to the system going live. figure 3: implications of an organization-wide es initiative go-live t 1 t n shakedown onwards/upwards system quality low high large medium small smes smes smes information quality individual impact organisation impact smes panel a go-live t 1 t n shakedown onwards/upwardsimplementation knowledge creation knowledge retention knowledge transfer knowledge application low high large medium small smes smes smes smes panel b 11 australasian journal of information systems sedera 2016, vol 20, research article does size matter? for both es-success and km-competence, measurement constructs are positioned on the most relevant lifecycle phases, based on prior literature. based upon the theoretical foundations of the is success model of delone and mclean (1992), system quality and information quality lead to individual and organizational impacts. additional recent research (seddon, calvert, & yang, 2010; wang & sedera, 2011) suggests that organizations are more likely to observe system quality and information quality during the shakedown phase of the lifecycle (and that they are less likely to see full impacts to individuals and organization) and impacts during the follow-on onwards/upwards phase. for km-competence, created knowledge during the implementation phase will be retained and transferred at the shakedown phase. though employees apply knowledge in both the shakedown and onwards/upwards phases of the lifecycle, knowledge application is more intense in the onwards/upwards phase. using figure 3, panel a, we make the following observations on organization-wide es initiatives for es-success. • all organizations, regardless of their sizes, have attained similar levels of system and information quality. though there may be continuous minor improvements to the system, the features and functions of an enterprise system largely remains stable after the shakedown phase. similarly, information quality also peaks at the end of the shakedown phase. panel a demonstrates that system and information quality of all organizations remain similar. this, together with evidence that knowledge creation (in panel b) also remains similar across the three firm sizes perhaps alludes to the effectiveness of global templates. global templates, executed through the treasury department provided similar levels of funds, expertise and in general all resources necessary for the es initiative. furthermore, the global templates monitored the delivery of the required functionality of the system and made provisions for helpdesk support and technical handovers during the shakedown phase. • despite achieving similar levels of success on system and information quality and gaining competencies in knowledge creation, smes could not capitalize on their initial levels of success. figure 3 panel a shows that both individual and organization impact of smes drop substantially in the onwards/upwards phase. we attribute this to the lack of lifecycle wide resource availability in smes. past es studies suggest that, despite successful es implementations, organizations must make continuous and substantial investment to gain sustained positive impacts on the relevant individuals and organization units through (i) change management, (ii) appropriate use, (iii) knowledge management, (iv) appropriate training, and (v) empowering users to ensure high levels of positive impact organization (momoh, roy, & shehab, 2010; seddon et al., 2010; strong & volkoff, 2010). our results also provide a justifiable rationale for the smaller organizational agencies that changed their es from sap to the smaller, financeone, during the shakedown phase, citing resource constraints. we make the following observations, using the findings reported in figure 3, panel b, that focuses on km-competence. • similar to our observations on system and information qualities, similar levels of knowledge creation competency gained during system implementation highlight the effectiveness of global templates. the standardized structure of governance employed through the global templates provided a highly structured approach through which the interactions of both consultants and vendors (i.e., the main knowledge sources of es implementations) with the client organizations were managed carefully. • despite gaining similar levels of km-competence through knowledge creation, smes fail to exploit on knowledge transfer and knowledge retention competencies. most smes do not have the budgeting provisions to compete with well-resourced large organizations (both public and private sector). gable et al. (1998) observed both “staff poaching” and “knowledge drain,” especially in smes, due to the skills shortage in enterprise systems during the latter half of the 1990s, thereby highlighting the 12 australasian journal of information systems sedera 2016, vol 20, research article does size matter? importance of organizational knowledge retention strategies for es-success throughout the entire lifecycle. our findings suggest that smes find it difficult to generate value from es beyond what they receive during the implementation phase. furthermore, they fail to develop on-going esrelated knowledge management competencies necessary for wellbeing throughout the lifecycle. though further research is needed to conclusively identify how global templates help smes in managing es initiatives, our findings suggest that global templates have been effective in delivering similar level of success and competencies. however, our findings also suggest that the coverage of global templates should extend beyond its focus on the implementation phase and should include strategies and resources for the entire lifecycle. 5.6 a consolidated path model a consolidated path model is designed and tested to understand the implications for the third hypothesis. herein, we follow guidelines of plouffe et al. (2001), and conduct a multi-group pls analysis (wold, 1989) using smartpls software (ringle, 2005) to understand the influence of firm size on the causal relationship between km-competence and es-success. figure 4 demonstrates the results of the path model analysis. figure 4: path model of knowledge management for es-success several observations can be made from the path model results shown in figure 4: kmcompetence small firms es-success β= 0.646 system quality information quality individual impact organization impactknowledge creation knowledge retention knowledge transfer knowledge application implementation post-implementation kmcompetence medium firms es-success β= 0.678 system quality information quality individual impact organization impactknowledge creation knowledge retention knowledge transfer knowledge application implementation post-implementation kmcompetence large firms es-success β= 0.735 system quality information quality individual impact organization impactknowledge creation knowledge retention knowledge transfer knowledge application implementation post-implementation 0.328 0.584 r2 =0.466r2 =0.682 0.131 0.742 r2 =0.460r2 =0.704 0.020 0.899 r2 =0.541r2 =0.830 13 australasian journal of information systems sedera 2016, vol 20, research article does size matter? • all three models, regardless of the firm size, demonstrate large, statistically significant path coefficients between km-competence and es-success, with approximately 50% of the variance of es-success, is explained by km-competence (r-square values for essuccess range between 0.46 and 0.54). findings further strengthen km-competence as a critical antecedent of es-success. • we notice that r2 values of both constructs – es-success and km-competence – decline with the descending firm size, with large organizations reporting the highest values and smaller organizations reporting the lowest values. • similarly, path coefficient between km-competence and es-success also declined with the descending order of firm size (coefficient for small = 0.646; medium = 0.678; large = 0.735). • investigating the extent to which km-competence is determined by its individual constructs (i.e. creation, retention, transfer, and application), we observe the loadings of each construct across the three firm sizes. here, we observe that small firms have the highest loading for km-competence gained through knowledge creation, while the loadings of knowledge creation on km-competence decline with the increasing firm size. • however, large organizations demonstrate the highest loadings for km-competence gained through post-implementation knowledge management activities (retention, transfer, and application), while the loadings of post-implementation knowledge management competence is lowest in small organizations. consistent with the observations of sedera and gable (2010), our multi-group analysis demonstrates the positive, significant relationship between km-competence and es-success. the path coefficients and r-square values suggest that the data from large organizations fit better with our research model than with smaller organizations. the results also highlight a shift in emphasis for gaining km-competence across the three firm sizes. for example, large organizations place a greater emphasis on post-implementation activities (retention, transfer, and application), when defining km-competence. the results suggest that km-competence in small organizations is still based largely on knowledge created during the implementation period. 6 discussion the goal of this study was to test statistically the implied influence of firm size on system success and knowledge-management competence in a large-scale, multidivisional enterprise system initiative. the research insights were gained through an investigation of the first major public-sector sap installation in the world, which integrated 27 different departments into a single sap implementation. the study model investigated km-competence, as defined by sedera and gable (2010), and es-success using notions of the delone and mclean is success model. the study model was then tested with a data sample of 310 respondents, providing descriptive, comparative, and multi-group analyses, yielding several observations and interpretations. in summary, the three kinds of organizational units – small, medium, and large – did not demonstrate any statistically significant differences for knowledge creation and knowledge application constructs as the independent variables and system quality and information quality as the dependent variables. we attribute these characteristics to the similarities in the implementation approach followed by the lead agency, the treasury department. given that all 27 of the subsidiary organizations followed the same sap global template, using the same corporate-wide software platform, we would have expected these similarities in the beginning of the es lifecycle. however, we observed differences between large and smaller counterparts for individual impact, organizational impact, knowledge retention, and knowledge transfer. despite attaining similar levels of information and system quality and gaining similar levels of km-competence through knowledge creation, we found that smes, in comparison to larger 14 australasian journal of information systems sedera 2016, vol 20, research article does size matter? organizations, failed to carry their progress attained during the implementation/shakedown phase through to the latter onwards/upwards phase of the es lifecycle. thus, our results demonstrated significant differences between large organizations and smes for individual impacts, organization impacts, knowledge retention, and knowledge transfer. many factors could have contributed to the lack of progress of smes in the onwards/upwards phase. in general, it highlights the weaknesses in lifecycle-wide management capabilities of smes for large-scale business information system initiatives. the lack of preparation and attention to es lifecycle management in smes is further evidenced through the multi-group pls analyses, where km-competency of smes is primarily attributed to knowledge created during the implementation phase. in contrast, km-competence in large organizations is viewed more through the activities conducted during the onwards/upwards phase through knowledge retention and knowledge transfer. post-implementation knowledge transfer and knowledge retention showed substantially lower mean scores for both small and medium organizations. our research findings also provide insights into the role of global implementation templates in large-scale, multidivisional es implementations. for example, all organizations in our study managed to attain a similar level of success and competency at the go-live time. the use of global templates appeared to have helped the smes to devise a workable approach to software acquisition and communication strategies, to gaining a higher degree of it sophistication, and to increasing the level of risk tolerance throughout the organizations. finally, our study identified a repeatable and generalizable classification scheme and heuristics to categorize firm size, based on the number of user licenses. future research could use these heuristics in firm size studies to establish a cumulative tradition of research. 6.1 implications for research although studies of firm-size implications date back to the 1970s, this study is one of the few to have researched the firm-size implications in a large-scale enterprise system initiative, focusing on es-success and km-competence. there are several research implications of this study. first, answering calls by goode and gregor (2009) for establishing information-systemspecific classification and heuristics to determine firm size, this study established a repeatable and generalizable classification scheme for categorizing firms according to size using the number of software user licenses. this new classification method, with its corresponding cutoff values, will add uniformity to future firm-size research, where firm size can now be specified using a common, simple, yet consistent criterion. moreover, unlike prior classifications of firm size based on revenue, number of employees, or budgets, our classification has a direct, logical, relationship with information systems. moreover, it is expected that organizations will be less sensitive to revealing the number of user licenses, as opposed to information on it budgets, headcounts, or other financial information. however, this new firm-size classification scheme should be tested in future studies to further establish its validity as a classification method. second, this study contributes to the lack of knowledge of large-scale, corporate-wide enterprise system initiatives. to the best of our knowledge, we are unaware of any such studies that investigated a corporate-wide es initiative that gathered data from all of its organizational entities. in our study, we analysed the management of such corporate-wide initiatives through the use of the “panoptic gaze.” through a detailed study context, our study highlights how such management practices are accomplished through a lead department governing the es implementation process from the initial user requirements analysis (ura) to the go-live date. despite some studies suggesting that the use of global templates may be harmful to smaller subsidiaries (avison & malaurent, 2007), our study provided counter evidence, demonstrating that all organizations, regardless of the size, could attain similar levels of success and competence to the go-live date. 15 australasian journal of information systems sedera 2016, vol 20, research article does size matter? finally, our model test results re-validate the dimensions of km-competence (gable and sedera 2010) and the is-impact measurement model of gable, sedera, and chan (2008). furthermore, the results re-confirm the statistically significant impact of km-competence as an antecedent of es-success. 6.2 implications for practice at a time when enterprise-system vendors are aggressively moving towards scaled-down enterprise systems for smes (e.g., sap’s bydesign, microsoft’s nav), our study results provide some cautionary insights into the long-term viability of es in smes. despite attaining similar levels of system and information success, smes failed to reap similar levels of benefits in individual and organizational impacts. our results highlight the need for smes to develop an es lifecycle management plan, wherein adequate resources are dedicated, not only to the initial implementation process, but for the entire lifecycle. our results provide some degree of comfort for practitioners who undertake similar corporatewide es initiatives and who use management control through global templates. our study highlighted how a lead agency, in this case the state treasury department, used global templates for the entire es lifecycle, providing tight management control for all parts of the organization at every stage prior to implementation, thus achieving reasonable measures of success and km-competence. however, it highlights the weaknesses in the current approach, where the treasury department support and control seem to have diminished immediately following the go-live date. unlike large organizations where resources and motivations are adequate, smes require continuous support and guidance for their es initiatives to be successful. our study highlights the importance incorporating a lifecycle-long strategy for the use of global templates, including attention to the retention of knowledge within the organization and transferring it to the required staff through appropriate channels. this view has been echoed in chang et al. (2000), who argued that organizations require a lifecycle-long knowledge-sourcing strategy in enterprise system projects. low mean scores of smes in relation to the knowledge retention and transfer dimensions provide clear evidence of a “knowledge drain.” gable, scott, and davenport (1998) also recognized this occurrence of “staff poaching” and “knowledge drain” due to es skill shortages in the late 1990s, and these problems remain. having explained almost half of the variance in es-success across the 27 organizations studied, km-competence was identified as possibly the most important antecedent of success. given that many es installations around the world struggle to deliver expected benefits, we recommend a stronger emphasis be placed on related-knowledge management. the study’s results reinforce the early calls of researchers (davenport, de long, & beers, 1998; newell, huang, & tansley, 2006) to make an end-to-end lifecycle es knowledge-management strategy available to those organizations involved in undertaking the implementation of an es. 6.3 limitations and future research this study has several limitations. first, the data were gathered only from a public-sector organization, using a single sap application, suggesting that further testing in other industry sectors, with other enterprise systems, would contribute to the generalizability of the findings. the firm-size classification scheme also has its limitations. first, the sample only included organizations using sap applications. thus, the classification method should be tested with other types of information systems for its generalizability. also, the size cut-offs were determined in an inductive manner. further research should be conducted to generalize these firm-size dimensions as well. furthermore, the study’s results should be triangulated using alternative firm-size classifications (e.g., financial indicators). this study observed the outcomes employing global templates in a corporate-wide es initiative. however, the scope of the current study did not allow us to make deep observations on the role of global templates in corporate-wide es initiatives. this too is an important research direction. 16 australasian journal of information systems sedera 2016, vol 20, research article does size matter? references alavi, m., & leidner, d. e. (2001). review: knowledge management and knowledge management systems: conceptual foundations and research issues. mis quarterly, 25(1): 107-136. argote, l., mcevily, b., & reagans, r. (2003). managing knowledge in organizations: an integrative framework and review of emerging themes. management science, 49(4): 571-582. avison, d., & malaurent, j. (2007). enterprise resource planning software in china: a disaster story. intercultural management quaterly, 8(2). bacher, j., wenzig, k., & vogler, m. (2010). spss two-cluster a first evaluation available at http://www.statisticalinnovations.com/products/twostep.pdf bernroider, e. w. n., & koch, s. (2000). differences in characteristics of the erp system selection process between small or medium and large organizations sixth americas conference on information systems (amcis 2000): 1022-1028. long beach california: ais. bilili, s., & raymond, l. (1993). information technology: threats and opportunities for small and medium-sized enterprises. international journal of information management, 13(6): 439-448. cenfetelli, r. t., & bassellier, g. (2009). interpretation of formative measurement in information systems research. mis quarterly, 33(4): 689-707. chang, s.-i., gable, g., smythe, e., & timbrell, g. (2000). major issues with enterprise systems: a case study and survey of five government agencies. paper presented at the proceedings of the 21st international conference on information systems brisbane, australia. cheney, p. h. (1983). getting the most out of your first computer system. american journal of small business, 7(4): 476-485. clemmons, s., & simon, s. j. (2001). control and coordination in global erp configuration. business process management journal 7(3): 205-215. cooley, p. l., walz, d. t., & walz, d. b. (1987). a research agenda for computers and small business. american journal of small business, 11(3): 31-42. davenport, t. h., de long, d. w., & beers, m. c. (1998). successful knowledge management projects. , sloan management review, 39(2): 43-57. delone, w. h. (1981). firm size and the characteristics of computer use. mis quarterly, 5(4): 65-77. delone, w. h., & mclean, e. r. (1992). information systems success: the quest for the dependent variable. information systems research, 3(1): 60-95. diamantopoulos, a. (2009). reflective and formative metrics of relationship value: response to baxter's commentary essay. journal of business research doi:10.1016/j.jbusres.2009.03.001. diamantopoulos, a., & winklhofer, h. m. (2001). index construction with formative indicators: an alternative to scale development. journal of marketing research, 38(2): 269-273. doukidis, g. i., lybereas, p., & galliers, r. d. (1996). information systems planning in small businesses: a stage of growth analysis. journal of systems and software, 33 (2): 189201. ein-dor, p., & segev, e. (1978). organizational context and the success of management information systems. management science, 24(10): 1064-1077. 17 australasian journal of information systems sedera 2016, vol 20, research article does size matter? elmes, m. b., strong, d. m., & volkoff, o. (2005). panoptic empowerment and reflective conformity in enterprise systems-enabled organizations. information and organization, 15(1): 1-37. esteves, j., & pastor, j. (2005). a critical success factor's relevance model for sap implementation projects. in m. khosrow-pour (ed.), managing business with sap: 240-262. hershey, pa: idea group publishing. farhoomand, f., & hrycyk, g. p. (1985). the feasibility of computers in the small business environment. american journal of small business, 9(4): 15-22. gable, g., sedera, d., & chan, t. (2008). re-conceptualizing information system success: the is-impact measurement model. journal of association for information systems, 9(7): 377-408. gable, g. g., heever, r. v. d., erlank, s., & scott, j. (1997). large packaged software: the need for research. paper presented at the proceedings of the 3rd pacific asia conference on information systems brisbane, australia. gable, g. g., scott, j., & davenport, t. (1998). cooperative erp life cycle knowledge management. paper presented at the proceedings of the 9th australasian conference on information systems sydney, new south wales, australia. gable, g. g., & stewart, g. (1999). sap r/3 implementation issues for small to medium enterprises. paper presented at the proceedings of the 5th americas conference on information systems, milwaukee, wisconsin. goode, s., & gregor, s. (2009). rethinking organisational size: meaning, measurement and redevelopment. european journal of information systems, 18(1): 14-25. hage, j. (1980). theories of organization. new york: wiley publishers. hauser, r. m., & goldberger, a. s. (1971). the treatment of unobservable variables in path analysis. in h. l. costner (ed.), sociological methodology: 81-117. san francisco: jossey-bass. jarvis, c. b., mackenzie, s. b., & podsakoff, p. a. (2003). a critical review of construct indicators and measurement model misspecification in marketing and consumer research. journal of consumer research, 30(2): 199-216. joreskog, k. g., & goldberger, a. s. (1975). estimation of a model with multiple indicators and multiple causes of a single latent variable journal of the american statistical association, 70, (351 ): 631-639. kleinbaum, d. g., kupper, l. l., muller, k. e., & nizam, a. (1998). applied regression analysis and other multivariate methods. belmont, california: duxbury press. laukkanen, s., sarpola, s., & hallikainen, p. (2007). enterprise size matters: objectives and constraints of erp adoption. journal of enterprise information management, 20(3): 319-334. levy, m., & powell, p. (1998). sme flexibility and the role of information systems. small business economics, 11(2): 183-196. mabert, v. a., soni, a., & venkataramanan, m. a. (2003). the impact of organizationsize onen terprise resource planning (erp) implementations in the us manufacturing sector. omega, 31: 235-246. madapusi, a., & d'souza, d. (2005). aligning erp systems with international strategies. information systems management, 22(1): 7-17 madapusi, a., & d’souza, d. (2007). a framework for aligning the erp system with the international strategy of small firms. paper presented at the academy of management conference, philadelphia. 18 australasian journal of information systems sedera 2016, vol 20, research article does size matter? markus, l. (2001). toward a theory of knowledge reuse: types of knowledge reuse situations and factors in reuse success. journal of management information systems, 18(1): 57-93. markus, l., axline, s., petrie, d., & tanis, c. (2003). learning from adopters’ experiences with erp: problems encountered and success achieved. in g. shanks, p. seddon, & l. willcocks (eds.), second-wave enterprise resource planning systems. cambridge, uk: cambridge university press, . mintzberg, h. (1979). the structuring of organizations. englewood cliffs, new jersey: prentice-hall. mitev, n. n., & marsh, a. e. (1998). small businesses and information technology: risk, planning and change. journal of small business and enterprise development, 5(3): 228-245. momoh, a., roy, r., & shehab, e. (2010). challenges in enterprise resource planning implementation: state-of-the-art. business process management journal, 16(4): 537565. montazemi, a. r. (1988). factors affecting information satisfaction in the context of the small business environment. mis quarterly, 12(2): 238-256. newell, s., huang, j., & tansley, c. (2006). erp implementation: a knowledge integration challenge for the project team. knowledge and process management 13(4): 227-238. nord, w. r., & tucker, s. (1987). implementing routine and radical innovations. san francisco: new lexington press. pan, g., & chen, a. j. w. (2005). enterprise systems planning projects in china. in s. l. pan (ed.), managing emerging technologies and organizational transformation in asia, vol. 2. singapore: world scientific publishing co. pte. ltd. pan, s. l., newell, s., huang, j., & galliers, r. d. (2007). overcoming knowledge management challenges during erp implementation: the need to integrate and share different types of knowledge. journal of the american society for information science and technology, 58(3): 404-419. petter, s., straub, d., & rai, a. (2007). specifying formative constructs in information systems research. mis quarterly, 31(4): 623-656. plouffe, c. r., vandenbosch, m., & hulland, j. (2001). intermediating technologies and multigroup adoption: a comparison of consumer and merchant adoption intentions toward a new electronic payment system. the journal of product innovation management, 18: 65-81. proudlock, m. j., phelps, b., & gamble, p. (1999). it adoption strategies: best practice guidelines for professional smes. journal of small business and enterprise development, 6(4): 240-252. punj, g., & stewart, d. w. (1983). clustering in marketing research: a review and suggestions for application. journal of marketing research, 10(may): 134-148. raymond, l. (1985). organizational characteristics and mis success in the context of small business. mis quarterly, 9(1): 37-52. ringle, c., wende, s., and will, a. (2005). smartpls 2.0 (beta): university of hamburg. retrieved march 28, 2007 from http://www.smartpls.de. salim, s.a., sedera, d., sawang, s., alarifi, a., atapattu m., (2015) moving from evaluation to trial: how do smes start adopting cloud erp? australasian journal of information systems, 19 doi: http://dx.doi.org/10.3127/ajis.v19i0.1030 19 australasian journal of information systems sedera 2016, vol 20, research article does size matter? schultz, r. l., & slevin, d. p. (1975). implementation and organisational validity: an empirical investigation. in r. l. shultz, & d. p. slevin (eds.), implementing operational research / management science: 153-182. new york: elsevier, north-holland. seddon, p. b., calvert, c., & yang, s. (2010). a multi-project model of key factors affecting organizational benefits from enterprise systems. mis quarterly, 34(2): 305-a311. sedera, d., & gable, g. (2010). knowledge management competence for enterprise system success. journal of strategic information systems, 19(1): 296-306. sedera, d., lokuge, s. grover, v., sarker, s., sarker s., innovating with enterprise systems and digital platforms: a contingent resource-based theory view, information & management, 53(3), 297-408 semple, j. (1993). bentham's prison: a study of the panopticon penitentiary. oxford: clarendon press. sia, s. k., m. tang, soh, c., & boh, w. f. (2002). enterprise resource planning (erp) systems as a technology of power: empowerment or panoptic control? sigmis database, 33(1): 23-37. sistach, f., & pastor, j. a. (eds.). (2000). methodological acquisition of erp solutions with sherpa: tenhagenstam. sistach, f., pastor, j. a., & fernandez, l. f. (1999). towards the methodological acquisition of erp solutions for smes. paper presented at the emrps'99enterprise management and resource planning systems proceedings soh, c., sia, s. k., & tay-yap, j. (2000). cultural fits and misfits: is erp a universal solution? communications of the acm, 43(4): 47-51. soh, c. p. p., yap, c. s., & raman, k. s. (1992). impact of consultants on computerisation success in small businesses. information and management, 22: 309-319. strong, d. m., & volkoff, o. (2010). understanding organization--enterprise system fit: a path to theorizing the information technology artifact. mis quarterly, 34(4): 731756. sverlinger, p. o. m. (2000). managing knowledge in professional service organisations: technical consultants serving the construction industry phd thesis chalmers university of technology goteborg turner, j. s. (1992). personal dss success in small business. information and management, 22: 301-308. wang, r., & hamerman, p. d. (2008). topic overview: erp applications 2008: 19: forrester research, inc. wang, w., & sedera, d. (2011). a benefits expectation management framework for supply chain management systems. paper presented at the international conference on information systems, shanghai china. whisler, t. (1970). the impact of computers on organizations. new york, ny: praeger publishers. wold, h. (1989). introduction to the second generation of multivariate analysis. in h. wold (ed.), theoretical empiricism: vii-xl. new york: paragon house. zuboff, s. (1988). in the age of the smart machine: the future of work and power. new york: basic books. 20 australasian journal of information systems sedera 2016, vol 20, research article does size matter? appendix a – impact of firm size on es-lifecycle firm size implications: system selection and implementation whisler (1970), schultz and slevin (1975), and ein-dor and segev (1978) were among the first is researchers to suggest the importance of considering firm size in is implementations. in their early work, ein-dor and segev (1978) proposed a framework for is implementations, where they recognised the influence of firm size on resource availability, the requirements necessary for integration of professional units within an organization, the degree of formalisation of organizational systems, and the lead times for planning and implementation. similarly, they demonstrated that is projects are less likely to succeed in smaller organizations compared to their larger counterparts. bilili and raymond (1993) described the sme decisionmaking process in system selection as reactive, informal, and intuitive. cheney (1983) investigated various factors affecting small businesses in is implementations and found that they are prone to software and hardware problems during implementation. investigating is acquisition in medium and large organizations, turner (1992) recommended that smes develop applications in-house, rather than opting for packaged applications. similarly, raymond (1985) found that smes were better able to develop, implement, and administer their own applications in-house, compared to their larger counterparts. he also found that medium-sized organizations could maintain information systems with minimal financial, technical, and personnel requirements. moreover, historical studies suggest that smaller organizations can adapt to a higher degree of technological innovation than larger organizations (mintzberg, 1979; nord & tucker, 1987). nord and tucker (1987) argued that, although large organizations are more complex, they could easily adopt new technological innovations, using their resources and facilities such as financial strength, communication skills, research capabilities, and product development experience. they also found that the greater resources which exist in large organizations provide a certain tolerance for potential losses due to unsuccessful implementations (nord and tucker 1987). given the natural association of es with large organizations, only a handful of studies have investigated the prospects of es in smes (esteves & pastor, 2005). for example, gable and stewart (1999) analysed sme enterprise system implementations and found resource deficiency and lack of expertise as major barriers to es adoption. on the other hand, sistach et al. (1999) and sistach and pastor (2000) argued that smaller firm size could be beneficial to smes in es implementations. bernroider and koch (2000) detailed the results from an empirical study concerning differences in characteristics of the es selection process among small, medium, and large organizations. they found decision-making in the sme es selection process to be reactive, informal, and intuitive, re-confirming prior observations made in legacy applications (bilili & raymond, 1993; cheney, 1983). firm size implications: post-implementation management researchers assert that smes tend to have an opportunistic, day-to-day focus in relation to is benefits and seldom consider long-term benefits of a system (doukidis, lybereas, & galliers, 1996; proudlock, phelps, & gamble, 1999). whisler (1970) investigated 19 insurance companies and concluded that firm size was directly related to is performance, with smes receiving substantially fewer returns compared to their larger counterparts. montazemi (1988), investigating the aforementioned preposition, confirms the impact of organizational size on end-user satisfaction, claiming that users in large organizations are more satisfied with the is than users in smaller organizations. delone (1981) studied the relationship between the size of manufacturing firms and is usage. he concluded that firm size is: (1) directly related to the age of the firm’s computer operations, (2) inversely related to the amount of external support being used, and (3) directly related to the financial resources made available to the initiative. it is also noted that resource constraints lead to smes following an incremental approach to it investments, which, in turn, may result in isolated and incompatible systems, as well as limited flexibility (levy & powell, 1998). 21 australasian journal of information systems sedera 2016, vol 20, research article does size matter? a study by cooley et al. (1987) of mid-sized organizations identified user-friendly interfaces as the key factor of end-users’ satisfaction, while lower implementation costs were the most important for the management of smes. some scholars argue that smaller organizations have greater flexibility, a higher ability to adapt and improve, and find it easier to accept and implement change initiatives (hage, 1980). lifecycle-long dedication to km-competence of smes too has been discussed in prior academic studies, with most research suggesting a lack of resident knowledge and expertise on systems in smes (bilili & raymond, 1993; levy & powell, 1998; mitev & marsh, 1998). turner (1992) suggested that smes require assistance from external sources in creating knowledge in is adoption and management. similarly, soh et al. (1992) and gable et al. (1998) allude to the importance of seeking expert assistance from consultants when undertaking computerization in small businesses. they report better system usage in small businesses when consultants were employed. addressing the technical know-how within organizations, farhoomand and hrycyk (1985) reported that smallto medium-sized companies lack adequate in-house technical staff for managing is system lifecycle issues. smes, in general, are considered low in km-competence. research suggests that the resource constraints of smes hinder their ability to maintain knowledge within organizations and stay up-to-date with technology innovations, and that they are ill-prepared for managing long-term it investments (laukkanen et al., 2007; levy & powell, 1998). 22 australasian journal of information systems sedera 2016, vol 20, research article does size matter? firm sizes characteristics large medium small reference acquisition procedure rational, futuristic procedural ad-hoc, reactive bilili and raymond (1993); cheney (1983) change initiatives much effort, externally managed internally managed with external templates internally managed, relatively easy changes raj and bajwa (1997) communication and coordination much effort, greater need, highly structured less structured, high need fairly unstructured leifer (1988); rai and bajwa (1997) complexity very complex with many extraneous variables less complex simple, rule-based hall et al. (1967); gerhart et al. (1990) degree of structure structured rhythmic, procedural unstructured, irregular hart et al. (1994); brouthers et al. (2000) economies of scale greater some what low orlitzky (2001) flexibility rigid firm flexible fiegenbaum et al. (1991); hitt et al. (1997); kalantaridis (2004) is evaluations critical & futuristic short term optimistic & day-today focus doukidis et al. (1996); proudlock et al. (1999); whisler (1970); montazemi (1988) is sophistication high medium low yao et al. (2002); goss et al. (1994) it innovation low medium high turner (1992); raymond (1985); harrison (1997); stock et al. (2002) it investment high & continuous periodic incremental levy et al. (1998) knowledge / expertise high medium low nord and tucker (1987); delone (1981); bilili et al. (1993); levy et al. (1998); mitev et al. (1998); russell et al. (1998) need for control high moderate low blau (1970); rai et al. (1997) resource availability adequate restrained limited ein-dor and segev (1978); lang et al. (1997); yeo et al. (2002); delone et al. (1981); cooley et al. (1987); laukkanen et al. (2007) risk aversion less moderate more arrow (1962) risk tolerance greater scale moderate minimal hitt et al. (1997) slack resources more adequate less pearce et al. (1992); rogers (1995); bowen (2002) tolerance of losses high leeway moderate restrictive nord and tucker (1987) 23 australasian journal of information systems sedera 2016, vol 20, research article does size matter? appendix b – the survey instrument individual-impact 1. i have learnt much through the presence of (the is). 2. (the is) enhances my awareness and recall of job related information 3. (the is) enhances my effectiveness in the job 4. (the is) increases my productivity organizational-impact 5. (the is) is cost effective 6. (the is) has resulted in reduced staff costs 7. (the is) has resulted in cost reductions (e.g. inventory holding costs, administration expenses) 8. (the is) has resulted in overall productivity improvement 9. (the is) has resulted in improved outcomes or outputs 10. (the is) has resulted in an increased capacity to manage a growing volume of activity (e.g. transactions, population growth, etc.) 11. (the is) has resulted in improved business processes 12. (the is) has resulted in better positioning for e-government/business. information-quality 13. (the is) provides output that seems to be exactly what is needed 14. information needed from (the is) is always available 15. information from (the is) is in a form that is readily usable 16. information from (the is) is easy to understand 17. information from (the is) appears readable, clear and well formatted 18. information from (the is) is concise system-quality 19. (the is) is easy to use 20. (the is) is easy to learn 21. (the is) meets (the unit’s) requirements 22. (the is) includes necessary features and functions 23. (the is) always does what it should 24. the (the is) user interface can be easily adapted to one’s personal approach 25. (the is) requires only the minimum number of fields and screens to achieve a task 26. all data within (the is) is fully integrated and consistent 27. (the is) can be easily modified, corrected or improved knowledge creation 28. overall, sap knowledge possessed by the vendor (sap australia) has been appropriate 29. overall, sap knowledge possessed by the consultants has been appropriate 30. overall, sap knowledge possessed by the agency has been appropriate 31. overall, knowledge of the agency, possessed by the vendor (sap australia) has been appropriate 32. overall, knowledge of the agency, possessed by the consultants has been appropriate 33. overall, the agency knowledge of itself (e.g. business processes, internal policies, etc.) has been appropriate knowledge retention 34. overall, sap staff and knowledge retention strategies have been effective 35. the agency has retained the knowledge necessary to adapt the sap system when required formal knowledge transfer 36. training in sap has been appropriate knowledge application 37. overall, sap knowledge has been re-used effectively and efficiently by the agency 24 australasian journal of information systems sedera 2016, vol 20, research article does size matter? criterion items 38. overall, the impact of sap on me has been positive. 39. overall, the impact of sap on the agency has been positive. 40. overall, the sap system quality is satisfactory. 41. overall, the sap information quality is satisfactory. 42. overall, sap system related knowledge has been managed satisfactorily copyright: © 2016 sedera. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. 25 http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 global templates in enterprise systems 3 the study context 4 study model, instrument, and data collection 5 data analysis 5.1 cluster analysis: exploring firm size 5.2 construct validity 5.3 es-success: descriptive and comparative statistics 5.4 km-competence: descriptive and comparative statistics 5.5 implications of the organization-wide es initiative 5.6 a consolidated path model 6 discussion 6.1 implications for research 6.2 implications for practice 6.3 limitations and future research firm size implications: system selection and implementation firm size implications: post-implementation management paper title (paper title style) australasian journal of information systems bi, davison & smyrnios 2015, vol. 19, pp. s247-s266 it & fast growth sme performance it and fast growth small-to-medium enterprise performance: an empirical study in australia rui bi school of management and marketing charles sturt university rbi@csu.edu.au robert m. davison department of information systems city university of hong kong, hong kong isrobert@cityu.edu.hk kosmas x. smyrnios school of management rmit university kosmas.smyrnios@rmit.edu.au abstract information technology (it) is regarded as a facilitator for both small and large firms to speed up transactions between firms and their suppliers and customers, achieve real-time communication, lower transaction costs, and enhance speed and flexibility. however, understanding whether and how it helps small-to-medium enterprises (smes) to create business value still remains unclear. drawing upon resource-based view theory, we develop and test a theoretical model to explore the interrelationships between it resources (it expertise, it infrastructure), it capability (it integration), it-enabled inter-firm processes (activity integration, coordination, partnership enhancement), and organizational performance in the fast growth sme context. we propose that it business value depends on how smes employ it resources to develop it capability which facilitates inter-firm partnership processes along value chains. structural equation modelling is employed to test our theoretical conceptualization of 310 australian fast-growth smes across different industrial sectors. results show that it contributes to fast growth sme performance through the development of it capability and enhancement of inter-firm partnership activities. this research highlights the role of it in business value creation and the ways in which it is used by fast growth smes to foster core business competencies. keywords: it resources/capability, it-enabled inter-firm processes, resource-based view of firms, fast growth smes. 1 introduction information technology (it) provides new ways for firms to conduct business with their suppliers and customers, helping companies to achieve real-time communication, coordinate inter-firm processes, enhance speed and flexibility, and ultimately improve business performance. research into the use of it is gaining significant attention in the information system (is) area. particularly since carr (2003), it business value has become a growing thematic line of enquiry. today, more than ever, is researchers face mounting pressures to address questions concerning whether and how it investments lead to the creation of business value. debate has focused predominantly on large firms, with comparatively scant attention paid to small-to-medium enterprises (smes). it is widely acknowledged that smes are the driving engines of most economies. their survival and growth is imperative. for example, in australia, 94% 96% of businesses can be categorized as smes, employing approximately 3.5 million people and contributing to an estimated 30% of national gdp (oecd 2010). fast growth enterprises represent a significant powerhouse in the small business sector, contributing substantially to wealth, income, and job creation (gupta et al. 2013). fast growth smes are entrepreneurially oriented, willing to take risks, to be innovative and to initiate s247 australasian journal of information systems bi, davison & smyrnios 2015, vol. 19, pp. s247-s266 it & fast growth sme performance aggressive competitive actions, and grow faster than their industry sector average (upton et al. 2001). despite its importance, it appears that in the is research field, there is a dearth of literature focusing on smes and perhaps surprisingly, research on fast growth smes is almost nonexistent. therefore, understanding whether and how it helps fast growth smes to gain business advantage remains enigmatic. the present research attempts to address this gap in the literature. in this study, we define it as the application of computers and telecommunications equipment to the storage, retrieval, transmission, and manipulation of data in a business or enterprise context (lin & lin 2008). is refers to a system composed of people and computers that processes or interprets information (saraf et al. 2007). smes are defined as businesses in which personnel numbers fall below 250 (ghobadian & o'regan 2000). fast growth smes are those companies with an ability to take advantage of opportunities associated with unpredictable market, environmental, and economic conditions and thus achieving rapid sales turnover growth rates in a specific period (barringer et al. 2005). drawing upon the resource-based view of the firm (rbv) theory (barney 1991), we theorize and explore the interrelationships between it resources, it capability, it-enabled inter-firm partnership processes, and fast growth sme performance. rbv theory provides a theoretical platform upon which the indirect role of it resources/capability in value creation can be explored through the effects of intermediate-level processes (kim et al. 2011; mithas et al. 2011). we posit that fast growth companies achieve business growth by strategically utilizing it resources and it capability which enable these firms to conduct inter-firm processes effectively and efficiently. our research heeds the call from researchers that further studies are needed to investigate the imperative role of it in sme performance (cragg et al. 2011; cragg et al. 2013). we contribute further to the ongoing debate on it business value by understanding it value creation in the fast growth sme context and provide a solid theoretical framework to explain the interrelationships between variables. we examine the hypothesized linkages empirically based on data drawn from a survey of 310 fast growth australian smes. this paper is structured as follows. the literature review section synthesizes the extant it and sme literature, which constitutes research background of this study. the theoretical background section introduces the tenets of rbv, which forms the backbone of our conceptual model for hypothesis formulation. the research method section outlines the procedures used for data collection, validation of the measurement properties of the constructs, and the test of the proposed research model. next we present our findings and finally conclude with a discussion of findings, implications for research and practice, limitations and potential avenues for future research. 2 literature review 2.1 it and sme performance although several studies have investigated it value in the sme context, our understanding of the nature of it and how it impacts on sme performance still remains limited. for example, some studies (bharadwaj & soni 2007; butler & murphy 2008; cragg et al. 2011; cragg et al. 2013; johnston et al. 2007; oh et al. 2009; wynn 2008; zhang et al. 2008) provide empirical support for a positive relationship between it and sme performance through achieving business process competencies, enabling innovation, and increasing revenue and decreasing costs. others (cragg et al. 2002; hicks et al. 2006) suggest that the impact of it on sme performance is weak. to some extent, these mixed findings are not surprising as many smes have resource constraints with limited financial resources and it expertise and may be reluctant or unable to seek professional input (cragg et al. 2013). the mixed findings on it-performance linkage can be also attributed to the absence of a comprehensive understanding of the underlying mechanisms through which it improves sme performance. as it is an enabler which facilitates the development of relationships between firms and their business partners, examining how it impacts inter-firm competitive processes s248 australasian journal of information systems bi, davison & smyrnios 2015, vol. 19, pp. s247-s266 it & fast growth sme performance and enhance firms’ performance is imperative (rai & tang 2010). researchers (cragg et al. 2002; levy & powell 2005; nieto & fernández 2005) increasingly highlight that the value of it can be realized when it is embedded in a sme’s business processes through resource complementarities and co-specialization. for example, mithas et al. (2011, p. 238) indicated that “the role and articulation of the underlying mechanisms through which it capability improves firm performance remain unclear”. kim et al. (2011, p. 489) further proposed “that research on it business value should investigate the effects of it on business processes”. therefore, there is a need for fully exploring the relationship between it and firm performance to improve our understanding of the mechanisms through which it operates in the sme context. in addition, most studies have treated it resources and it capability as a unified concept, which limits our understanding on the relationship between them. therefore, unpacking the nature of it into the specific types of it resources and it capability will provide a more finegrained insight into the effects of it resources and it capability on sme performance and thus enrich the literature. 2.2 it and fast growth smes firm growth is defined as meeting or exceeding performance goals in the form of growth in financial (sales revenue, growth in profitability and financial goals) and non-financial (increasing customer satisfaction) terms (delmar & wiklund 2008). rapid growth is often regarded as an indication of market acceptance and firm success (storey & greene 2010). firm growth has significant managerial salience as growing companies generally exhibit better cash flow and higher profitability (anderson & eshima 2013). their ability to grow and establish themselves within their chosen markets in relatively brief periods makes fast growth enterprises an interesting target for academic research. a ground swell of interest in the strategic management literature have investigated the determinants of fast growth, identifying a range of internal (i.e., founder characteristics, resources, strategy & competencies) and external (i.e., alliances & access to capital) factors, as well as business practices related to their rapid growth (audretsch et al. 2014; gupta et al. 2013). in relation to it and sme growth, the literature suggests that fast growth companies differ from other smes and their larger counterparts in various ways. on the one hand, fast growth smes are entrepreneurially oriented and therefore are more likely than slow-growers to utilize new and advanced technologies as competitive tools (khazanchi 2005; raymond & bergeron 2008; storey & greene 2010). effective use of it innovation enables smes to implement new strategies to support their core competences, thereby competing and growing in rapidly changing business environments by insulating themselves from their competitors (raymond & bergeron 2008; uhlaner et al. 2013). on the other hand, compared with their larger counterparts, the flexible organizational structure and sound managerial skills in the growing smes guarantee the success of it use which in turn has positive effects of firm growth (hansen & hamilton 2011; mostafa et al. 2005; nieto & fernández 2005). however, limited research has been conducted to understand how it helps smes to achieve rapid growth. therefore, this study aims to address this significant gap in our current knowledge by exploring the role of it resources and it capability in helping smes to achieve growth through the improvements facilitated by it-enabled inter-firm partnership processes. 3 theoretical background firm performance depends on the availability of, or access to, valuable, rare, inimitable, nonsubstitutable and relatively immobile resources or resource bundles (barney 1991). the rbv theory suggests that organizations succeed and achieve competitive advantage through treatment of resources and capabilities as central considerations in strategy formulation and as primary sources of competitive advantage. according to the rbv, resources and capabilities represent two distinctive entities. while resources are used to create and produce products, capabilities are developed and emerge from utilization of resources in repeatable patterns (edelman et al. 2005). resources are generally regarded as inputs or outputs of organizational s249 australasian journal of information systems bi, davison & smyrnios 2015, vol. 19, pp. s247-s266 it & fast growth sme performance processes, and are typically not embedded within those processes. capabilities, however, are firm-specific and embedded in firm processes and routines, transforming inputs into outputs to generate value (newbert 2007). thus, capabilities are unique organizational processes developed to provide reliable services, create product innovations, generate operational flexibility, shorten product development cycles, and respond to evolving market trends (woldesenbet et al. 2012). makadok (2001, p. 387) posited that firms create value from two complementary, but distinct, mechanisms: “resource-picking” and “capability-building”. firms possessing bundles of advantage-generating resources and costly-to-imitate capabilities are regarded as fundamental drivers of superior performance. the rbv theory has been employed in the is field to explain how firms create value from it (dong et al. 2009; wade & hulland 2004; zhu & kraemer 2005). although it resources (e.g., hardware and software) are rarely drawn upon for the purpose of creating and sustaining competitive advantage, it capability as a firm’s ability to acquire, deploy, combine and configure it resources in order to support and enhance business strategies and processes is hard to imitate, helping firms to achieve superior performance (bharadwaj 2000; mithas et al. 2011). in this study, we examine the role of two types of it resources (i.e., it expertise, it infrastructure) in developing a specific it capability: it integration. it expertise is the skills and knowledge of a firm’s it personnel (cragg et al. 2013; wade & hulland 2004). it infrastructure refers to physical it assets including computers, communication facilities, shareable technical platforms, and databases (roberts & grover 2012). in the digitally-enabled business environment, it integration refers to the it-based linkages between the focal firm and its business partners which facilitate inter-firm business processes along the value chain (rai & tang 2010). unlike the commodity technologies, it integration is often associated with a firm’s strategic context and is woven into the organization’s fabric, which is not transparent to competitors (zhang et al. 2008). although competitors can easily mimic a firm’s it resources, the way companies effectively integrate it within an organizational strategy so as to develop an it capability is hard to acquire and difficult to imitate, thus providing firms with a source of competitive advantage. the rbv theory also provides a lens through which we can understand it value creation is “an processes which, in turn, lead to competitive advantage. therefore, -level dependent variables at the business process, department, or project level” (wade & hulland 2004, pp. 129-130). following this logic, we examine the interconnections between it resources, it capability, it-enabled inter-firm business processes, and sme performance. we posit that it resources and it capability provide fast growth smes with sources of advantage that help to differentiate these companies from competitors. inter-firm business processes involve inter-organizational relationships, are drawn upon it resources/capability, and are integrated with a firm’s strategies. these activities enable fast growth smes to coordinate and improve inter-dependent processes in order to respond to customer preference and market changes, provide high quality products and services to meet customer needs. thus these inter-organizational processes provide smes with core competences, helping firms to achieve growth. moreover, fast growth sme research (davidsson & wiklund 1999; ireland et al. 2003; obeng et al. 2012) has extensively employed rbv theory to explain the imperative role of internal resources and organizational capability in the achievement of firm growth. taken together, we argue that the rbv theory offers a solid theoretical foundation to explain how and why fast growth smes achieve competitive advantage not only from commonly available it resources but also from the integration of these it resources to form it capability which can be leveraged to develop it-enabled intermediated business routines residing in organizational skills and processes. 4 research model and hypothesis development figure 1 depicts a hypothesized model of it resources, it capability, it-enabled inter-firm partnership processes, and sme performance, and is followed by a discussion and formulation of testable hypotheses. s250 australasian journal of information systems bi, davison & smyrnios 2015, vol. 19, pp. s247-s266 it & fast growth sme performance figure 1. research model while it expertise refers to it personnel who have specialized intellectual technical skills to develop it applications that support business processes (lin & lin 2008), it infrastructure is physical it assets including computers, communication facilities, shareable technical platforms, and databases (zhang et al. 2008). it technical skills contain employees’ knowledge of programming, system analysis and design, and competencies in emerging technologies. compared with physical it assets, the technical skills of it employees are regarded as intangible and valuable firm resources which are more difficult to imitate by competitors and are likely to generate competitive advantage for firms (wade & hulland 2004). is literature highlights the prominent role of it human resources in successfully designing it applications in line with business strategy in order to sustain business advantage. for example, bhatt and grover (2005) posited that it employees with extensive business experience and skills in is development enable firms to integrate it strategy and business strategy, to develop reliable and cost-effective systems for businesses, and to anticipate business needs sooner than competitors. fink and neumann (2007) noted that firms with a high level of it expertise is able to establish efficient communication between it and business staff, to develop reliable and relevant business applications, and to align it and business processes effectively. it expertise is important for is success in smes (caldeira & ward 2003; cragg et al. 2013). recent studies (cragg et al. 2011; cragg et al. 2013) suggest that more and more smes are able to develop a high level of it expertise through gaining it project experience over the years and by employing internal it expertise. it expertise is essential in enabling smes to develop technically an appropriate it infrastructure platform on which to develop a technically superior it integration functionality for integrating multiple internal as well as external (i.e., business partners, suppliers) databases (cragg et al. 2011). thus, we hypothesize that: h1: it expertise is related positively to the development of appropriate it infrastructure platform. h2: it expertise is related positively to the development of it integration. it infrastructure provides a solid and flexible platform upon which firms can leverage it not only to conduct business activities but also to respond to customer demands and market changes for future business development (roberts & grover 2012). dong et al. (2009) argued that digital business is unlikely to become an integral part of the value chain when firms lack appropriate it infrastructure to readily and efficiently distribute necessary information for digital operations. in relation to the sme context, empirical research (bharadwaj & soni 2007; s251 australasian journal of information systems bi, davison & smyrnios 2015, vol. 19, pp. s247-s266 it & fast growth sme performance mostafa et al. 2005; nieto & fernández 2005) suggests that reliable and flexible it infrastructure fosters strong links between smes and their business partners, leading to the development of robust it integration. thus, we hypothesize that: h3: it infrastructure is related positively to it integration. activity integration is the extent to which firms collaborate with their value chains’ partners on strategic planning and forecasting activities (kim et al. 2006). coordination refers to firms’ ability to coordinate transactional related activities with their partners and suppliers (wu et al. 2006). a technically sound it integration with business partners and suppliers can increase transactional efficiencies, lower operation costs, and create business value for smes (butler & murphy 2008). smes with high levels of it integration with their business partners can improve inter-firm business processes by establishing integrative, collaborative, and coordinative connections among value chain members (arenius et al. 2005; levy & powell 2005). as an enabler for information processing, integration and coordination to facilitate cross-functional and multi-layer querying, it integration is expected to help smes to achieve business value by facilitating information flow (loane 2005), enhancing integrative business activities such as collaborative planning, forecasting, and replenishment (raymond & bergeron 2008), and improving coordination efficiency and reducing cost (dyerson et al. 2009). thus, we hypothesize that: h4: it integration is related positively to activity integration. h5: it integration is related positively to coordination. a high level of activity integration among value chain members facilitates joint inter-firm processes such as joint production planning and sales forecasting, joint resource planning and work scheduling (cao & zhang 2011). in the context of digital business operations, empirical studies (jansen et al. 2013; theyel 2013) suggest that smes tend to integrate strategic business activities effectively with their trading partners in the value chain in order to achieve high level of coordination efficiency. smes employing strategic integration with their business partners are likely to increase coordination efficiency of production through close integration of decisions and operations (voss & voss 2013). thus we hypothesize that: h6: activity integration is related positively to coordination. the relational view of competitive advantage (dyer & hatch 2006) suggests that critical resources often span firm boundaries and are embedded in inter-firm routines and processes. this type of resources has great potential to generate a relational rent. following this logic, we argue that it-enabled inter-firm partnership activities facilitate rent generation by means of inter-firm relation-specific assets and inter-firm knowledge-sharing routines. it can help firms to gain sustainable competitive advantage by facilitating communication, collaboration, and fostering relational capabilities (rai & tang 2010). firms can achieve business advantage when they understand how to strategically employ it to enhance collaboration and coordination activities, leading to the development of inter-firm complementary capabilities (tallon & pinsonneault 2011). in the context of relationship development, the sustainability of strategic advantage depends on how it is employed to foster inter-firm processes and to facilitate the development and use of complementary capabilities (raymond & bergeron 2008). recent research (jansen et al. 2013; wynn 2008) demonstrates that the inter-firm collaboration and coordination helps smes to increase the intensity of, and enrich the quality of, its interactions with partners and suppliers, thus enhancing business relationship development. smes sharing important product planning and inventory information with value chain members on a regular and realtime basis are more likely to develop productive relationships than those who do not (raymond & bergeron 2008; theyel 2013). in addition, an important precursor to effective and efficient inter-firm processes is the commitment of resources among value chain members to ensure that their business processes and systems are mutually compatible (bharadwaj & soni 2007). when the focal company’s systems and online information repositories are integrated with those of its partners and suppliers, these parties are able to exhibit a greater commitment to s252 australasian journal of information systems bi, davison & smyrnios 2015, vol. 19, pp. s247-s266 it & fast growth sme performance their mutual relationships (du et al. 2012). such commitment can, in turn, foster trusting, long-lasting relationships and can credibly signal the parties’ intentions to ensure the longterm success of their business relationships (eckhardt & shane 2011; jansen et al. 2013). thus, we hypothesize that: h7: activity integration is related positively to partnership enhancement. h8: coordination is related positively to partnership enhancement. inter-firm collaboration and coordination is necessary to ensure performance (kim & lee 2010). effective coordination not only reinforces a firm’s ability to maintain, advance, and strengthen its relationships with value chain partners (rai & tang 2010), but also enables partner organizations in the value chain to share information in a timely manner, schedule procurement, production, and distribution operations synchronously, and respond to market changes swiftly (kim & lee 2010). given that coordination is facilitated by the efficacy of a robust it integration and a repetitive collaboration process, smes endowed with superior coordination competency can outperform competitors through efficient order handling procedures and short delivery lead time, therefore achieving improved performance (eikebrokk & olsen 2007; levy & powell 2005; raymond & bergeron 2008). in the face of the increasing pace of change and complexity of business environments, it is imperative for smes to build strategic relationships in order to achieve sustainable value (voss & voss 2013). building strategic relationships with business partners and suppliers not only enables smes to access to new technologies, markets, and complementary resources, but also to increase their responsiveness to market changes, fosters great knowledge seeking, and achieve synergetic rents (carlo et al. 2012; raymond & bergeron 2008). studies (luo et al. 2008; zhang et al. 2008) show that the higher level of collaboration in a relationship, the better business performance smes can achieve. in relation to the fast growth sme context, empirical research (barringer et al. 2005; hansen & hamilton 2011; uhlaner et al. 2013) suggests rapid growth smes tend to engage proactively in inter-organizational partnerships to build resources which provide another avenue to high growth. thus, we hypothesize that: h9: coordination is related positively to sme performance. h10: partnership enhancement is related positively to sme performance. 5 research methodology 5.1 target population and survey sample the data used for testing our proposed model was collected through an online survey of australian fast-growth smes compiled by the present investigation for the business review weekly (brw). key inclusion or exclusion criteria are: previous year’s turnover must exceed aud$500,000; fewer than 200 full-time employees; not a subsidiary of an australian or overseas corporation; and must not receive more than 50% of their revenue from a single client. companies provide signed and audited turnover figures over four consecutive financial periods in order to calculate average growth rates for ranking purpose. growth is determined by average turnover over a four-year period using the fourth year as a baseline. we have chosen to test our proposed model using fast-growth smes because smes comprise a dominant part of the global economy (oecd 2010). smes, in general, are relatively unsuccessful in exploiting the potential of digital business (eikebrokk & olsen 2007). fastgrowth smes, in comparison, are more entrepreneurial in their business orientation, are less risk averse, and have achieved leadership positions by leveraging their it resources (tiessen, wright & turner 2001). 5.2 data collection procedures a personalized email highlighting the academic nature of the current study was sent to either the founder or ceo of all 1,335 fast-growth smes on the brw database. in our emails, we emphasized the importance of having respondents with a good understanding and overview of s253 australasian journal of information systems bi, davison & smyrnios 2015, vol. 19, pp. s247-s266 it & fast growth sme performance their firm’s e-business activities to participate in our survey, urging the founder or ceo to personally complete the online questionnaire, where possible. a follow-up email was sent three weeks after the initial one, and a second reminder email another two weeks later. respondents were assured of confidentiality. a total of 310 responses were obtained in 2011, which gave a gross response rate of 28.1%, after discounting 195 incorrect email addresses and 32 smes which declined to participate. all responses were filled by either the company founder or its ceo. we first tested the sample for non-response bias, using the approach suggested by armstrong and overton (1977). differences in responses to all the constructs between early respondents (i.e., those that completed the survey upon the first invitation) and late respondents (i.e., those who replied to follow-up emails) were compared. independent sample t-tests on each construct failed to reveal significant differences between early and late respondents (all ps>.05), suggesting that non-response bias was not an issue. the profile of the responding firms in our study (table 1) shows that our sample contains companies in all major industry sectors. there is also equal distribution of companies in terms of their age (or years of establishment). all responding firms had achieved a growth rate in excess of 20%. % (n=310) industry information technology 18.8 property & business services 18.1 personal & other services 9.6 finance & insurance 8.9 communications 6.6 others a 38 company age less than 5 years 49 more than 5 years 51 previous year growth rate 21.9-759.5 ceo/founder’s education level tertiary 53.9 mba 16.6 year 12 13.7 phd or doctorate 1.8 other 14.0 note. a other industry sectors include construction, retail trade, manufacturing, health & community services, wholesale trade, education, transport & storage, accommodation, café, restaurants, mining, cultural & recreational services. table 1. profile of responding firms 5.3 common method bias as our study used a self-administered questionnaire and respondents were in a senior management position qualified to assess firm performance, measurement was subject to cognitive biases due to participants “seeking to present themselves in a favourable manner” (thompson & phua 2005, p. 541). anticipating such a possibility, we incorporated marlowe and crowne’s (1961) social desirability scale in our online questionnaire, inviting participants to complete this section as part of the survey. the incorporation of marlowe and crowne’s s254 australasian journal of information systems bi, davison & smyrnios 2015, vol. 19, pp. s247-s266 it & fast growth sme performance (1961) social desirability scale in our questionnaire enabled us to assess all study items for social desirability response bias in order to address internal validity and psychometric aspects of instruments. marlowe and crowne’s (1961) social desirability scale has been used widely for checking cognitive biases (ballard 1992). in this study, we tested common method bias using sem procedures recommended by podsakoff et al. (2003). first, we conducted a harman one-factor test to estimate the extent of the bias. principal components analysis resulted in six components, accounting for 70.1% of the total variance. the first component explained only 37.1% of the variance, implying the absence of a dominant general factor that accounts for more than 50% of the variation. second, this study controlled for the effects of a directly measured social desirability factor (i.e., the social desirability factor was linked to all endogenous variables). results culminated in a poor fitting model entailing associations between social desirability and model parameters, with all path coefficients being close to zero and nonsignificant (all ps >.05). accordingly, social desirability is not an issue in our sample. based on these tests, we conclude that common methods bias is not a significant threat in our study. 5.4 constructs all the constructs and the corresponding measurement items were adapted from the literature (table 2). all constructs were assessed with seven-point likert scales ranging from strongly disagree (1) and strongly agree (7). specifically, three items measuring it expertise were adapted from lin and lin (2008). three items measuring it infrastructure were adapted from zhu (2004). three items measuring it integration were adapted from zhang et al. (2008). three items measuring activity integration were adapted from kim et al. (2006). three items measuring coordination were adapted from kim et al. (2006). three items measuring partnership enhancement were adapted from wu et al. (2003). three items measuring sme performance were adapted from dong et al. (2009). as control variables, we employed number of years since business start-up to measure firm age and used a series of industry dummies to control for exogenous factors at the industry level. development of respective measurement models incorporate successive stages of theoretical modelling, statistical testing, and refinement (straub 1989). constructs indicators 1. it expertise (itep) adapted from lin and lin (2008) our company hires highly specialized or knowledgeable it people. it people working for our company are well aware of the multifaceted functions of e-business. it people working for our company are adequately trained in ebusiness. 2. it infrastructure (itif) adapted from zhu (2004) our company has a good telecommunication infrastructure. our company has a good networking infrastructure. our company has a flexible it infrastructure for future development. 3. it integration (iti) adapted from zhang et al. (2008) our company has integrated web applications encompassing different functional areas. our company shares the databases for various applications. our company has it based links with external partners. 4. activity integration (ai) adapted from kim et al. (2006) our company collaborates actively in forecasting and planning with our business partners/suppliers. our company projects and plans future demand collaboratively with our business partners/suppliers. collaboration in demand forecasting and planning with our business partners/suppliers is something we always do. 5. coordination (cood) adapted from kim et al. (2006) our company conducts transaction follow-up activities more efficiently with our business partners/suppliers than do our competitors with theirs. our company spends less time on supply chain coordination transactions with our business partners/suppliers than our competitors with theirs. s255 australasian journal of information systems bi, davison & smyrnios 2015, vol. 19, pp. s247-s266 it & fast growth sme performance constructs indicators our company conducts coordination transactions at less cost than do our competitors with theirs. 6. partnership enhancement (pe) adapted from wu et al. (2003) our company is able to strengthen the existing business relationships with business partners/suppliers. compared with our competitors, the relationships between our company and business partners/suppliers are likely to last longer. our company is able to build up new business relationships with business partners/suppliers. 7. sme performance (sp) adapted from dong et al. (2009) compared with our competitors, the market share of our products has increased. compared with our competitors, the sales volume of our products has been increasing at a faster pace. compared with our competitors, our sales market has widened significantly. table 2. constructs and indicators 5.5 instrument validation data were analysed with amos 19.0, using confirmatory factor analysis (cfa) procedures with the maximum likelihood (ml) estimation method. prior to conducting the cfa, we ran an exploratory factor analysis (efa) on all indicators. principal axis factoring with direct oblimin rotation yielded consistent groupings with our hypothesized measurement models. all constructs were tested for reliability, validity, and fit. based on an assessment of cfa fit statistics, measurement models were further refined to obtain sound fit. respectively, tables 3 and 4 show correlations and descriptive statistics and measurement properties of constructs. as reported below, instrument validation proceeded through four steps: calculation of construct reliability; variance extracted estimates; and evaluation of convergent and discriminant validity. s256 australasian journal of information systems bi, davison & smyrnios 2015, vol. 19, pp. s247-s266 it & fast growth sme performance note. (1) *p<.05. **p<.01. (2) the diagonal elements are the square root of the ave. table 3. correlation matrix, mean scores and standardized deviations constructs construct reliability variance extraction range of standardized loadings range of indicator reliability 1. itep .86 .87 .69 .75-.94 .57-.90 2. itif .83 .89 .66 .74-.89 .54-.76 3. iti .75 .74 .50 .65-.77 .43-.60 4. ai .75 .74 .50 .65-.81 .42-.66 5. cood .87 .89 .73 .83-.93 .58-.82 6. pe .95 .89 .72 .71-.93 .58-.82 7. sp .86 .87 .69 .67-.80 .57-.84 note. all factor loadings are significant at p<.001 level table 4. confirmatory factor analysis: standardized loadings and reliability 5.5.1 construct reliability construct reliability, a measure of consistency, assesses the degree to which items are free from random error. indictor and composite reliability are two measures of construct reliability (fornell & larcker 1981). while indicator reliability represents the proportion of variation that is explained by a construct it purports to measures, composite reliability reflects the internal consistency of indicators (werts et al. 1974). in the present study, indicator reliability values range between .50 and .88, and composite reliability values exceed the recommended value of .70 (nunnally & bernstein 1994). 5.5.2 variance extracted estimate variance extracted estimate reflects the overall amount of variance in indicators accounted for by a latent construct (fornell & larcker 1981). in this study, all estimates exceed the recommended value of .50 (hair et al. 2006). 5.5.3 construct validity construct validity was established by measuring convergent and discriminant validity of measurement items (phillips & bagozzi 1986). convergent validity assesses the consistency across multiple operationalisations. values for t-statistics for all factor loadings were found to be significant (all ps<.001), indicating that measures satisfy convergent validity criteria (gefen et al. 2000). according to fornell and larcker (1981), average variance extracted for each construct should be greater than the squared correlation between constructs when assessing discriminant validity, the extent to which different constructs diverge from one another. in this mean sd 1 2 3 4 5 6 7 1. itep 4.95 1.69 .83 2. itif 5.53 1.08 .48** .82 3. iti 4.12 1.63 .52** .39** .70 4. ai 4.30 1.64 .28** .26** .34** .95 5. cood 4.40 1.24 .31** .38** .34** .45** .85 6. pe 5.42 1.11 .21** .24** .20** .37** .38** .87 7. sp 5.32 1.21 .27** .23** .23** .36** .51** .42** .86 s257 australasian journal of information systems bi, davison & smyrnios 2015, vol. 19, pp. s247-s266 it & fast growth sme performance case, the results suggest that items share more common variance with related than non-related constructs, with all constructs meeting this criterion. 5.6 data analysis confirmatory and full structural model fit were assessed using multiple indices (hair et al. 2006), including the normed chi-square ( /df), comparative fit index (cfi), tucker-lewis index (tli), standardized root mean-square residual (srmr), and root mean-square error of approximation (rmsea). all seven measurement models tested were found to meet the criteria set for these indices (i.e., /df ratio < 3; cfi and tli > .90; srmr < .08; and rmsea < .08). 6 results given the acceptable measurement models, we estimated a full latent variable structural model using same goodness of fit criteria to test our structural model and respective hypotheses. table 5 summarizes the results of hypotheses testing, revealing reliable and robust fit between our theoretical model and sample covariances: (179)=346.348, /df=1.935, cfi=.960, tli=.953, srmr=.062, rmsea=.055. these indices suggest a good model fit. the squared multiple correlation (smc) values, which are similar to r² in regression analysis, show that this model accounts for 32% of the variance in it infrastructure, 45% of the variance in it integration, 17% of the variance in activity integration, 31% of the variance in coordination, 24% of the variance in partnership enhancement, and 44% of the variance in sme performance. none of the control variables showed significant effects in the research model. table 5 shows that all hypothesized relationships are supported. hypothesis standardized paths estimates conclusion h1: it expertise it infrastructure .56*** supported h2: it expertise it integration .51*** supported h3: it infrastructure it integration .24** supported h4: it integration activity integration .41*** supported h5: it integration coordination .26*** supported h6: activity integration coordination .35*** supported h7: activity integration partnership enhancement .28*** supported h8: coordination partnership enhancement .25*** supported h9: coordination sme performance .23*** supported h10: partnership enhancement sme performance .53*** supported model fit indices ²(179)=346.348, ²/df=1.935, cfi=.960, tli=.953, srmr=.062, rmsea=.055 note. *p<.05. **p<.01. ***p<.001. table 5. proposed hypotheses and test results 7 discussion aiming to address understudied issues about it and fast growth sme performance, this research develops and empirically tests a hypothesized theoretical model integrating it resources, it capability, it-enabled inter-firm partnership activities, and organizational performance. the rbv theory underpins the present research. results show that all ten hypothesized relationships are supported, suggesting that it can help fast growth smes to achieve business success through the development of it integration capability and it-enabled s258 australasian journal of information systems bi, davison & smyrnios 2015, vol. 19, pp. s247-s266 it & fast growth sme performance inter-firm partnership processes. this study provides six insights significantly contributing to theory, is research, and business practices. first, this study develops, theorizes, and empirically validates an integrated theoretical model to investigate inter-relationships among it resources, it capability, it-enabled inter-firm processes, and fast growth sme performance. our results highlight that it resources and it capability are sources of advantage, helping fast growth smes to achieve business value through facilitating inter-firm business processes. because transaction-specific know how and inter-firm knowledge-sharing routines are socially complex advantages subject to considerable causal ambiguity and time-compression diseconomies, these inter-firm processes are difficult to replicate, providing positional advantage for firms to outperform their competitors (saraf et al. 2007). through exploring the entire net containing it resources, it capability, and implications of it resources/capability, this research empirically tests the application of rbv theory, thus offering a sharp theoretical lens to view the phenomena in it business value creation. second, this study contributes to is literature by synthesizing resource-picking and capabilitybuilding mechanisms to investigate how it resources help fast growth smes to build it capability which in turn leads to business value. although the rbv theory emphasizes the possession of rare, valuable, and inimitable resources as drivers of competitive advantage (barney 1991), recent theorizing argues that capability-building processes are more significant than resource-picking processes (kim et al. 2011). it is noteworthy, however, that ex ante studies of impact of it resources on it capability and sme performance largely overlook resource-picking and capability-building mechanisms. to bridge this apparent gap in the literature, we disaggregate it resources into it expertise and it infrastructure and explore the synergistic effects of these two it resources on the development of it integration capability. our results suggest that it expertise with superior knowledge about business strategy, competition, and opportunities are the origin of source of advantage, helping fast growth smes to renew and redevelop existing resources/capability and create strategic value. examinations of the direct relationship between it resources/capability and sme performance are unlikely to generate such insights. third, our results indicate that it integration has a substantial effect on inter-firm processlevel performance along the value chain. this finding highlights that it business value depends on how fast growth smes effectively employ technology to improve value chain operations. according to the rbv theory, in competitive environments, it is imperative for firms to develop their ability to integrate resources into bundles of organizational capabilities which are difficult to imitate and confer firms with superior performance (eisenhardt & martin 2000; teece et al. 1997). consistent with this logic, is researchers (kim et al. 2013; mithas et al. 2011) suggest that although commonly available it resources cannot by themselves create sustained performance gains for firms, it capability can provide firms with business advantage because it involves a long-term development process and requires firms to make a series of linked strategic decisions and moves related to it resources so as to blend them with organizational processes and knowledge resources. our results indicate that inter-firm partnership activities such as activity integration, coordination, and relationship enhancement processes are achieved through the deep embedding of integrated it capability as process enablers. the development of such itenabled inter-firm partnership capability that leverages it infrastructure integration requires significant time, substantial expertise spanning the business process domain, partnership context, and it, the perspective of which makes it hard for competitors to imitate. the fundamental role of it in value chain management is not only to provide speed and flexibility, but also to facilitate a close linkage in key processes among value chain members so as to create strategic value in extended organizations (kim et al. 2013; rai et al. 2012). although the role of it integration in enabling process efficiencies is well understood in is literature, its role in enhancing knowledge sharing and recombination across smes needs to be further explored using more granular constructs. therefore, our results provide an implication for is research, s259 australasian journal of information systems bi, davison & smyrnios 2015, vol. 19, pp. s247-s266 it & fast growth sme performance suggesting the usefulness to gauge intermediate firm performance and probe into the specific ways that it is used to improve business processes. fourth, our results suggest that it-enabled integration and coordination processes are positively related to partnership enhancement and fast growth sme performance. these findings confirm the relational view of competitive advantage which provides a valuable theoretical lens to investigate the ways in which inter-firm partnership activities influence business-related performance. recent research (bharadwaj et al. 2013) underscores that in the digitally enabled business environment, the emergence of a new model of competitive strategy stems from how firms effectively manage value chains through integration of inter-firm processes across partner organizations. our findings highlight that sustainability of strategic advantage depends upon how it is used to foster collaborative processes between fast growth smes and their business partners, and facilitate the development and use of complementary capabilities. fifth, this study contributes to is research by investigating the business value of it in the fast growth sme context. compared with their larger counterparts, research targeting fast growth smes is almost non-existent in the is field. fast growth smes are entrepreneurially-oriented, willing to seek competitive opportunities to add profitable value to their products or services (barringer et al. 2005). the seminal work of penrose (1959, p. 217) suggests that the “amount of resources administered by a firm has in itself a significant influence on the opportunities for expansion open to the firm”, highlighting that when a firm is entrepreneurial, the existence of resources promotes firm growth. recently, cragg et al. (2011) argued that technical it skills can be regarded as a source of competitive advantage in smes, helping firms to create competencies and thus achieve business goals. our findings lend support to these views, indicating that entrepreneurial smes adopt a proactive it stance, employ a high level of it expertise to experiment and explore new and available technologies in order to exploit existing competencies, address, and create new business opportunities. finally, this study contributes to practice by providing a useful integrative framework for fast growth sme managers to understand the ways in which it investments help firms to create strategic advantage and achieve business performance. sources of competitive advantage, be they technological or organizational, do not guarantee competitive advantage. it is the ways in which firms combine these qualities to develop unique organizational capabilities to achieve superior business advantage. sme managers are advised to examine external industry contexts, firm-specific it resources, and core processes that foster the development of organizational capabilities that form part of corporate strategies and growth. this study offers three important implications for fast growth sme managers. first, our model identifies that it human resources and it infrastructure are critical for fast growth smes to compete and succeed in the digitally enabled business environments. it is essential that fast growth sme managers take these it resources into account when transforming their physical supply chains into those based on digital connections and information flow. particularly, fast growth sme managers need to understand the important role of it integration in achieving seamless information flow among various information systems and databases both internally and externally across the production, supply, and distribution networks. this will become even more important as competition intensifies. second, this study highlights that resources/capabilities become sources of advantage only when they are exploited through business processes. fast growth sme managers should assign a high priority to the identification of resource competencies that have strong potential for developing specific it capability and focus on appropriate business processes where it capability is deployed. third, fast growth sme managers should bear in mind that establishing information-linked strategic alliances with business partners and suppliers is critical. in the current turbulent business environments, fast growth smes have to learn to build strategic partnerships with their value chain partners so as to leverage the external resource provided by business partners and thus gain relational benefits. s260 australasian journal of information systems bi, davison & smyrnios 2015, vol. 19, pp. s247-s266 it & fast growth sme performance 8 limitations and conclusion this study has four notable limitations. first, a cross-sectional research design was adopted with data collected at a single point in time. it-enabled business processes and competencies are dynamic. firms require time to reconfigure their resources to adapt to changes in the technological and business environments. future research might consider using longitudinal designs to address issues relating to the evolution of it-enabled processes and the development of core business competency. second, utilizing a single-informant (ceo and/or founder) in each responding company presents issues of data credibility. single informant studies are well-known for their susceptibility to reporting bias. future research might consider obtaining data from managers across the it, marketing, and operational functions. a third limitation relates to sample characteristics upon which the present hypotheses are tested. the current investigation is drawn from a relatively small proportion of self-selected fast-growth smes in a specific geographic region. while the present hypothesized model might be applicable to larger firms as well as firms in other geographic locales, further research is needed to expand the generalizability of the findings. finally, the present study only explores the impact of it resources/capability in enhancing sme performance. future research might consider extending the current investigation and explore other elements such as how culture, structure, and entrepreneurial leadership interact with it in enabling firm performance. including these variables could offer significant improvements over the current model, providing a more comprehensive understanding of the value of it complementary resources and capabilities. understanding whether and how it helps firms to create business value is an ongoing debate among both researchers and business practitioners. however, most studies target larger corporations with scant attention paid to smes. we sought to better understand this understudied link between it and business performance within the sme context. we examine the role of two specific it resources: it expertise and it infrastructure in business value creation. we argue that it resources and it capability, as sources of advantage, help fast growth smes to develop core competences over competitors by improving inter-firm partnership activities which in turn create business value. the current findings demonstrate that fast growth smes foster, nurture, and develop firm-wide it capability by successfully managing and leveraging their it resources in such a way as to achieve business success. in the light of the ongoing debate about the business value of it, we hope this study motivates further discussion and encourages the advancement of theory that helps us to improve our understanding of the dynamics of it and its relationship to organizational performance in the sme context. references anderson, b.s. & eshima, y. (2013) "the influence of firm age and intangible resources on the relationship between entrepreneurial orientation and firm growth among japanese smes", journal of business venturing, 28(3): 413-429. arenius, p., sasi, v. & gabrielsson, m. (2005) "rapid internationalisation enabled by the internet: the case of a knowledge intensive company", journal of international entrepreneurship, 3(4): 279-290. armstrong, j.s. & overton, t.s. (1977) "estimating nonresponse bias in mail surveys", journal of marketing research, 14(3): 396-402. audretsch, d., coad, a. & segarra, a. (2014) "firm growth and innovation", small business economics, 43(4): 743-749. barney, j.b. (1991) "firm resources and sustained competitive advantage", journal of managment, 17(1): 99-120. s261 australasian journal of information systems bi, davison & smyrnios 2015, vol. 19, pp. s247-s266 it & fast growth sme performance barringer, b.r., jones, f.f. & neubaum, d.o. (2005) "a quantitative content analysis of the characteristics of rapid-growth firms and their founders", journal of business venturing, 20(5): 663-687. bharadwaj, a., el sawy, o.a., pavlou, p.a. & venkatraman, n. (2013) "digital business strategy: toward a next generation of insights", mis quarterly, 37(2): 471-482. bharadwaj, a.s. (2000) "a resource-based perspective on information technology capability and firm performance: an empirical investigation", mis quarterly, 24(1): 169-196. bharadwaj, p.n. & soni, r.g. (2007) "e-commerce usage and perception of e-commerce issues among small firms: results and implications from an empirical study*", journal of small business management, 45(4): 501-521. bhatt, g.d. & grover, v. (2005) "types of information technology capabilities and their role in competitive advantage: an empirical study", journal of management information systems, 22(2): 253-277. butler, t. & murphy, c. (2008) "an exploratory study on is capabilities and assets in a smallto-medium software enterprise", journal of information technology, 23(4): 330-344. caldeira, m.m. & ward, j.m. (2003) "using resource-based theory to interpret the successful adoption and use of information systems and technology in manufacturing small and medium-sized enterprises", european journal of information systems, 12(2): 127-141. cao, m. & zhang, q. (2011) "supply chain collaboration: impact on collaborative advantage and firm performance", journal of operations management, 29(3): 163-180. carlo, j.l., lyytinen, k. & rose, g.m. (2012) "a knowledge-based model of radical innovation in small software firms", mis quarterly, 36(3): 865-a810. carr, n.g. (2003) "it doesn't matter", harvard business review, 81(5): 41-49. cragg, p., caldeira, m. & ward, j. (2011) "organizational information systems competences in small and medium-sized enterprises", information & management, 48(8): 353-363. cragg, p., king, m. & hussin, h. (2002) "it alignment and firm performance in small manufacturing firms", the journal of strategic information systems, 11(2): 109-132. cragg, p., mills, a. & suraweera, t. (2013) "the influence of it management sophistication and it support on it success in small and medium-sized enterprises", journal of small business management, 51(4): 617-636. davidsson, p. & wiklund, j. (1999) theoretical and methodological issues in the study of firm growth, jönköping international business school, sweden. delmar, f. & wiklund, j. (2008) "the effect of small business manager's growth motivation on firm growth: a longitudinal study", entrepreneurship: theory & practice, 32(3): 437-457. dong, s., xu, s.x. & zhu, k. (2009) "information technology in supply chain: the value of it-enabled resources under competition", information systems research, 20(1): 1832. du, t.c., lai, v.s., cheung, w. & cui, x. (2012) "willingness to share information in a supply chain: a partnership-data-process perspective", information & management, 49(2): 89-98. dyer, j.h. & hatch, n.w. (2006) "relation-specific capabilities and barriers to knowledge transfers: creating advantage through network relationships", strategic management journal, 27: 701-719. dyerson, r., harindranath, g. & barnes, d. (2009) "national survey of smes’ use of it in four sectors", the electronic journal information systems evaluation, 12(1): 39-50. s262 australasian journal of information systems bi, davison & smyrnios 2015, vol. 19, pp. s247-s266 it & fast growth sme performance eckhardt, j.t. & shane, s.a. (2011) "industry changes in technology and complementary assets and the creation of high-growth firms", journal of business venturing, 26(4): 412-430. edelman, l.f., brush, c.g. & manolova, t. (2005) "co-alignment in the resource-performance relationship: strategy as mediator", journal of business venturing, 20(3): 359-383. eikebrokk, t.r. & olsen, d.h. (2007) "an empirical investigation of competency factors affecting e-business success in european smes", information & management, 44(4): 364-383. eisenhardt, k.m. & martin, j.a. (2000) "dynamic capabilities: what are they?", strategic management journal, 21(10/11): 1105-1121. fink, l. & neumann, s. (2007) "gaining agility through it personnel capabilities: the mediating role of it infrastructure capabilities", journal of the association for information systems, 8(8): 440-462. fornell, c. & larcker, d.f. (1981) "evaluating structural equation models with unobservable variables and measurement error", journal of marketing research, 18(1): 39-50. gefen, d., straub, d.w. & bourdreau, m.c. (2000) "structural equation modeling and regression: guidelines for research practices", communications of the ais, 4(7): 1-78. gupta, p., guha, s. & krishnaswami, s. (2013) "firm growth and its determinants", journal of innovation and entrepreneurship, 2(1): 1-14. hair, j.f., black, w.c., babin, b.j., anderson, r.e. & tatham, r.l. (2006) multivariate data analysis, prentice hall, new jersey. hansen, b. & hamilton, r.t. (2011) "factors distinguishing small firm growers and nongrowers", international small business journal, 29(3): 278-294. hicks, b.j., culley, s.j. & mcmahon, c.a. (2006) "a study of issues relating to information management across engineering smes", international journal of information management, 26(4): 267-289. ireland, r.d., hitt, m.a. & sirmon, d. (2003) "a model of strategic entrepreneurship: the construct and its dimensions", journal of management, 29(6): 963-989. "information processing and strategic decision-making in small and medium-sized enterprises: the role of human and social capital in attaining decision effectiveness", international small business journal, 31(2): 192-216. johnston, d.a., wade, m. & mcclean, r. (2007) "does e-business matter to smes? a comparison of the financial impacts of internet business solutions on european and north american smes", journal of small business management, 45(3): 354-361. khazanchi, d. (2005) "information technology (it) appropriateness: the contingency theory of "fit" and it implementation in small and medium enterprise", journal of computer information systems, 45(3): 88-95. kim, d., cavusgil, s.t. & calantone, r.j. (2006) "information system innovations and supply chain management: channel relationships and firm performance", journal of the academy of marketing science, 34(1): 40-54. kim, d. & lee, r.p. (2010) "systems collaboration and strategic collaboration: their impacts on supply chain responsiveness and market performance", decision sciences, 41(4): 955-981. kim, g., shin, b., kim, k.k. & lee, h.g. (2011) "it capabilities, process-oriented dynamic capabilities, and firm financial performance", journal of the association for information systems, 12(7): 487-517. s263 australasian journal of information systems bi, davison & smyrnios 2015, vol. 19, pp. s247-s266 it & fast growth sme performance kim, g., shin, b. & kwon, o. (2013) "investigating the value of sociomaterialism in conceptualizing it capability of a firm", journal of management information systems, 29(3): 327-362. levy, m. & powell, p. (2005) strategies for growth in smes: the role of information and information systems, elsevir butterworth-heinemann, oxford. lin, h.f. & lin, s.m. (2008) "determinants of e-business diffusion: a test of the technology diffusion perspective", technovation, 28: 135-145. loane, s. (2005) "the role of the internet in the internationalisation of small and medium sized companies", journal of international entrepreneurship, 3(4): 263-277. luo, x., hsu, m.k. & liu, s.s. (2008) "the moderating role of institutional networking in the customer orientation-trust/commitment-performance causal chain in china", academy of marketing science. journal, 36(2): 202-214. makadok, r. (2001) "toward a synthesis of the resource-based and dynamic-capability views of rent creation", strategic management journal, 22(5): 387-401. marlowe, d. & crowne, d.p. (1961) "social desirability and response to perceived situational demands", journal of consulting psychology, 25: 109-115. mithas, s., ramasubbu, n. & sambamurthy, v. (2011) "how information management capability influences firm performance", mis quarterly, 35(1): 237-256. mostafa, r.a., wheeler, c. & jones, m. (2005) "entrepreneurial orientation, commitment to the internet and export performance in small and medium sized exporting firms", journal of international entrepreneurship, 3(4): 291-302. newbert, s.l. (2007) "empirical research on the resource-based view of the firm: an assessment and suggestions for future research", strategic management journal, 28(2): 121-146. nieto, m. & fernández, z. (2005) "the role of information technology in corporate strategy of small and medium enterprises", journal of international entrepreneurship, 3(4): 251262. nunnally, j.c. & bernstein, i.h. (1994) psychometric theory, mcgraw hill, new york. obeng, b.a., robson, p. & haugh, h. (2012) "strategic entrepreneurship and small firm growth in ghana", international small business journal, 32(5): 501-524. oecd (2010) smes, entrepreneurship and innovation, oecd publishing. available at: http://www.oecd.org/cfe/smesentrepreneurshipandinnovation.htm oh, k.-y., cruickshank, d. & anderson, a.r. (2009) "the adoption of e-trade innovations by korean small and medium sized firms", technovation, 29(2): 110-121. penrose, e.t. (1959) the theory of the growth of the firm, oxford university press, oxford. phillips, l. & bagozzi, r. (1986) "assessing measurement error in key informant reports: a methodological note on organizational analysis in marketing", journal of marketing research, 18: 395-415. podsakoff, p.m., mackenzie, s.b., lee, j.-y. & podsakoff, n.p. (2003) "common method biases in behavioral research: a critical review of the literature and recommended remedies", journal of applied psychology, 88(5): 879-903. rai, a., pavlou, p.a., im, g. & du, s. (2012) "interfirm it capability profiles and communications for cocreating relational value: evidence from the logistics industry", mis quarterly, 36(1): 233-a235. s264 australasian journal of information systems bi, davison & smyrnios 2015, vol. 19, pp. s247-s266 it & fast growth sme performance rai, a. & tang, x. (2010) "leveraging it capabilities and competitive process capabilities for the management of interorganizational relationship portfolios", information systems research, 21(3): 516-542. raymond, l. & bergeron, f. (2008) "enabling the business strategy of smes through ebusiness capabilities: a strategic alignment perspective", industrial management & data, 108(5): 577-595. roberts, n. & grover, v. (2012) "leveraging information technology infrastructure to facilitate a firm's customer agility and competitive activity: an empirical investigation", journal of management information systems, 28(4): 231-270. saraf, n., langdon, c.s. & gosain, s. (2007) "is application capabilities and relational value in interfirm partnerships", information systems research, 18(3): 320-339. storey, d.j. & greene, f.j. (2010) small business and entrepreneurship, pearson education, harlow. straub, d.w. (1989) "validating instruments in mis research", mis quarterly, 13(2): 147169. tallon, p.p. & pinsonneault, a. (2011) "competing perspectives on the link between strategic information technology alignment and organizational agility: insights from a mediation model", mis quarterly, 35(2): 463-486. teece, d.j., pisano, g. & shuen, a. (1997) "dynamic capabilities and strategic management", strategic management journal, 18(7): 509-533. theyel, n. (2013) "extending open innovation throughout the value chain by small and medium-sized manufacturers", international small business journal, 31(3): 256-274. thompson, e.r. & phua, f.t.t. (2005) "reliability among senior managers of the marlowecrowne short-form social desirability scale", journal of business and psychology, 19(4): 541-554. uhlaner, l., van stel, a., duplat, v. & zhou, h. (2013) "disentangling the effects of organizational capabilities, innovation and firm size on sme sales growth", small business economics, 41(3): 581-607. upton, n., teal, e.j. & felan, j.t. (2001) "strategic and business planning practices of fast growth family firms", journal of small business management, 39(1): 60-72. voss, g.b. & voss, z.g. (2013) "strategic ambidexterity in small and medium-sized enterprises: implementing exploration and exploitation in product and market domains", organization science, 24(5): 1459-1477. wade, m. & hulland, j. (2004) "review: the resource-based view and information systems research: review, extension, and suggestions for future research", mis quarterly, 28(1): 107-142. werts, c., linn, r. & jöreskog, k. (1974) "interclass reliability estimates: testing structural assumptions", educational and psychological measurement, 34(1): 25-33. woldesenbet, k., ram, m. & jones, t. (2012) "supplying large firms: the role of entrepreneurial and dynamic capabilities in small businesses.", international small business journal, 30(5): 493-512. wu, f., mahajan, v. & balasubramanian, s. (2003) "an analysis of e-business adoption and its impact on business performance", journal of the academy of marketing science, 31(4): 425-447. wu, f., yeniyurt, s., kim, d. & cavusgil, s.t. (2006) "the impact of information technology on supply chain capabilities and firm performance: a resource-based view", industrial marketing management, 35(4): 493-504. s265 australasian journal of information systems bi, davison & smyrnios 2015, vol. 19, pp. s247-s266 it & fast growth sme performance wynn, m. (2008) "information systems strategy development and implementation in smes", management research news, 32(1): 78-90. zhang, m., sarker, s. & sarker, s. (2008) "unpacking the effect of it capability on the performance of export-focused smes: a report from china", information systems journal, 18(4): 357-380. zhu, k. (2004) "the complementarity of information technology infrastructure and ecommerce capability: a resource-based assessment of their business value", journal of management information systems, 21(1): 167-202. zhu, k. & kraemer, k.l. (2005) "post-adoption variations in usage and value of e-business by organizations: cross-country evidence from the retail industry", information systems research, 16(1): 61-84. copyright: © 2015 bi, davison & smyrnios. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. s266 paper title (paper title style) australasian journal of information systems wang 2016, vol 20, research article social network motivation following a crisis people’s motivation to participate in social network sites, subsequent behaviours, and situation self-awareness following a crisis: evidence from the mh370 flight incident xuequn wang murdoch university xuequnwang1600@gmail.com abstract as people increasingly integrate social network sites (snss) into their daily lives, they also turn to these sites for timely information following crises. to date, few studies have examined the effects of different types of motivation on participatory behaviours within snss following crises. in this study, self-determination theory (sdt) is applied to examine how individuals are motivated to participate in snss following a crisis and how individuals’ participatory behaviours can better assist them understand a crisis. the malaysia airlines flight 370 (mh370) incident is selected as the context for this study, and data is collected following the incident. the results show that different types of motivation result in different participatory behaviours. controlled motivation is positively related to browsing content, commenting and sharing content. conversely, autonomous motivation is positively associated with browsing and sharing content. besides, browsing content and commenting are positively related to people’s situational self-awareness. the theoretical and practical implications of this study are discussed. keywords: social network sites; participatory behaviours; motivation; situational selfawareness; crisis; mh370. 1 introduction as people increasingly integrate social network sites (snss) into their daily life, they also turn to snss following crises (shklovski et al., 2008; vieweg et al., 2008). snss can increase the resilience of affected individuals (wang and li, in press) and enable the public to obtain timely information about a crisis (pang and ng, in press). for example, in the 2007 southern california wildfires, residents used blogs and other information and communication technologies (icts) to acquire valuable local information (e.g., whether to evacuate) (shklovski et al., 2008). during the 2013 little india riot, the public used twitter to access timely information and ascertain responses to the crisis (pang and ng, in press). spence et al. (2015) argue that individuals who use snss regularly in their daily lives probably turn to these sites for information in a crisis. despite the importance of snss during or following crises, several gaps remain in the literature. first, few studies have examined how the public participates in snss following crises (pan et al., 2012). recent studies have begun to examine how the public uses snss to responds to crises over time (pang and ng, in press; spence et al., 2015). however, little is known about how individuals are motivated to participate in snss to seek relevant information with different approaches. some may browse snss and comment on crisis-related content, while others may also share content within their social networks. thus, an important question arises: how are people motivated to participate in snss differently following crises? further, little is known about how different participatory behaviours in snss assist the public to understand crises. specifically, individuals may browse the crisis-related content posted by others, comment on others’ content, or share certain content within snss; so the question arises: how do these different participatory behaviours assist people to understand crises? 1 australasian journal of information systems wang 2016, vol 20, research article social network motivation following a crisis to answer these two questions and address gaps in the existing literature, this study selects the malaysia airlines flight 370 (mh370) incident as the context. specifically, this study examines how people’s different types of motivation within snss result in various participatory behaviours following the mh370 incident and how those participatory behaviours help individuals better understand the crisis. this study applies self-determination theory (sdt) to examine how different types of motivation lead to different participatory behaviours on snss. according to sdt, people are motivated to engage in behaviours either to obtain external rewards or because they integrate the rationale of the activities into their own values (ryan and deci, 2000). thus, sdt can provide valuable insights into why people are motivated to participate differently. this study has three main objectives. first, it seeks to examine how different types of motivation can result in different participatory behaviours within snss following a crisis. extending upon earlier research, this study examines browsing content, commenting and sharing content behaviours and how different types of motivation support those behaviours. the results can contribute to the literature by informing the reasons why people participate differently on snss following a crisis. the second objective is to assess how different social factors support different types of motivation. as different types of motivation may lead to different participatory behaviours, practitioners (such as emergency managers) may be interested in understanding how different types of motivation could be supported. according to sdt, people’s motivation can be controlled or autonomous. with controlled motivation, individuals tend to act for external reasons (e.g., monetary rewards) or internal pressures. conversely, autonomously motivated individuals are more likely to act because they integrate behavioural values with their own values (e.g., individuals may exercise for good health). such individuals understand the rationale of their behaviours. thus, this study assesses how social influence leads to controlled motivation and how trust can support autonomous motivation the effect of social influence on controlled motivation is examined, as people tend to engage in certain activities valued by their significant others (ryan and deci, 2000). for example, people may participate in a certain sns, even if they do not integrate participating in the sns into their own values, simply because their friends or relatives participate in the same sns (boyd and ellison, 2007). in such a scenario, people act out of controlled motivation. thus, the effect of social influence on controlled motivation should be examined. the role of trust on autonomous motivation is also examined. recent research has shown that trust can enhance people’s psychological needs (wang and li, 2014), which in turn can support their autonomous motivation (ryan and deci, 2000). thus, this study examines whether trust beliefs toward others directly enhance people’s autonomous motivation. by arguing that different types of motivation are supported by different social factors, our study can contribute to the literature by providing a deeper theoretical understanding of motivation and enable useful guidelines to be formulated that promote different types of motivation. finally, this study seeks to examine the effects of participatory behaviours. specifically, we argue that the public could obtain a better understanding of the crisis by participating in snss. this study introduces the concept of situational self-awareness to represent people’s understanding of a crisis and examines how people’s participatory behaviours enhance their situational self-awareness. here situational self-awareness refers to the consciousness of an individual’s internal state (e.g., thoughts and feelings) and surroundings (buss, 1980). thus, this study makes a significant contribution to the literature by determining how the public understands crises through various participatory behaviours. the rest of the paper is organized as follows: we first review the literature on crises and outlines various participatory behaviours in snss. then the theoretical foundation of the study is discussed and the hypotheses are developed. next, the data collected following the mh370 incident are discussed and the results of data analysis are presented. finally, the theoretical and practical implications of the study are discussed. 2 australasian journal of information systems wang 2016, vol 20, research article social network motivation following a crisis 2 literature review 2.1 crises crises refer to non-routine events that usually involve social disruption (fritz, 1961). examples of crises include natural disasters (e.g., earthquakes, floods) and human-made disasters (e.g., terrorist attacks, aircraft crashes). in addition to traditional media such as newspapers and television, snss have recently emerged as another important source of crisis-related information. previous studies have shown that information seeking is one of the main reasons individuals participate in snss (park et al., 2009; raacke and bonds-raacke, 2008). snss are more interactive and dialogic (schultz et al. 2011) than traditional forms of media, can reach a larger audience, and allow audience members to interact with one another. thus, snss can provide a higher level of participation, openness, conversation, and connectedness (mayfield, 2006) and more timely information. recent literature has recognized the importance of snss for both natural disasters and humanmade disasters (e.g., palen et al., 2009; shklovski et al. 2010; starbird and palen, 2010; taylor et al., 2012; yanay et al., 2011). for example, in relation to natural disasters, snss have been used to find missing people and offer various types of help such as offering food, water, and shelters following the 2005 hurricane katrina (palen et al., 2007). during the 2007 southern california wildfires, snss were used to distribute community-relevant information as to the areas affected by wildfires and evacuation warnings (shklovski et al., 2008). in the 2010 haiti earthquake, snss were used to offer needed medication (sarcevic et al., 2012). in relation to human-made disasters, snss were used to list victims of the 2007 virginia tech shooting (vieweg et al., 2008). snss also allowed people to voice dissent against institutionalized power during the 2011 egyptian uprising (al-ani et al., 2012). further, following the second gulf war, snss were used to build resilience and pursue daily routines (e.g., going to work or school, traveling, and contacting relatives and friends) (wang and li, in press).these studies show how snss can help affected people respond to and recover from crises. crises can cause the public to feel a high degree of uncertainty and anxiety (mitroff, 2004). in many instances, the public may be distant to the sites of crises and crises may damage telecommunication infrastructures (e.g., landline telephones and power stations have been destroyed in crises) (semaan and mark, 2011) resulting in information about crises not being broadcast. further, accurate information about crises may be difficult to obtain (hagar and haythornthwaite, 2005) even if the public wants to know more (boyle et al., 2004; procopio and procopio, 2007). accordingly, in addition to those directly affected by crises, the public also wants timely and relevant information about crises. however, few studies have examined how the public has used snss to gather timely and relevant information about crises (spence et al., 2015). one exception is from pang and ng (in press), who examine how the public uses twitter to respond to the 2013 little india riot. timely and relevant information can help the public better understand a crisis. this understanding can facilitate the public engaging in remedial activities (chen et al., 2007; spence et al., 2006) (e.g., donating money, clothes or other supplies to areas severely damaged by natural disasters) (pan et al., 2012)). the public may also become “digital volunteers” to help those affected by crises (e.g., to help translate information and move information between sources) (starbird and palen, 2011). in such circumstances, snss can be used to update the public with timely information in relation to the crisis and provide platforms where people can show their support to affected individuals (ahmed and sinnappan, 2013). thus, providing crisis-related information via snss can fulfil the public’s need for information in relation to the crisis, rescue efforts, government responses, and the assistance being provided (spence et al. 2015). on the other hand, few studies have differentiated people’s participatory behaviours within snss after crises. in summary, the importance of snss in distributing crisis-related information has been recognized in the literature; however, previous literature has mainly examined sns messages 3 australasian journal of information systems wang 2016, vol 20, research article social network motivation following a crisis related to crises, and few studies have examined how the public obtains timely information through snss to better understand crises (pan et al., 2012; pang and ng, in press). this study examines how different participatory behaviours within snss can help people better understand a crisis. because people can have different participatory behaviours in snss, below we clarify how people’s various participatory behaviours differ. 2.2 participatory behaviours in snss snss are defined as “web-based services that allow individuals to (1) construct a public or semipublic profile within a bounded system, (2) articulate a list of other users with whom they share a connection, and (3) view and traverse their list of connections and those made by others within the system” (boyd and ellison, 2008, p. 211). on snss, people can post new content, browse, comment on or share the content of others (boyd and ellison, 2008). different participatory behaviours above can require different levels of cognitive effort (wang, 2014). notably, although browsing content forms the foundation of many other behaviours, it requires relatively less effort and is more popular than other behaviours (benevenuto et al., 2009). commenting on content (wang, 2014) or sharing content requires slightly more effort, but remains cognitively easy. lastly, posting new content is more difficult and requires more effort and engagement (wang, 2014). this study focuses on browsing, commenting on and sharing content, as these behaviours are important mechanisms for exchanging information on snss (oh et al., 2013). it is important to clarify that different levels of cognitive effort are required for different participatory behaviours because people may participate in snss with different motivations (park et al., 2009; raacke and bonds-raacke, 2008). for example, when individuals are motivated by external rewards, they likely focus on how to maximize their rewards and minimize their efforts (ryan and deci, 2000). such individuals are likely to engage in participatory behaviours that require relatively little effort such as commenting (wang, 2014). conversely, when individuals are motivated by entertainment, they likely enjoy interacting with others on snss and thus spend more time and expend more effort on snss. accordingly, various behaviours are likely supported by different types of motivation. 3 theoretical background and hypotheses development sdt is applied as the theoretical foundation for this study. sdt differentiates between various types of motivation and clarifies social factors supporting different types of motivation (ryan and deci, 2000). sdt can thus be useful to examine the supporting conditions and outcomes of different types of motivation. 3.1 self-determination theory (sdt) sdt identifies intrinsic motivation and four types of extrinsic motivation (external regulation, introjected regulation, identified regulation, and integrated regulation). different types of extrinsic motivation have different levels of self-determination and can be divided into autonomous motivation (e.g., identified regulation or integrated regulation) and controlled motivation (e.g., external regulation or introjected regulation) (gagne and deci, 2005; reinholt et al., 2011).1 the distinction between autonomous motivation and controlled motivation is important because it implies that motivation not only differ quantitatively (i.e., the amount of motivation), but also qualitatively (i.e., the underlying goals or reasons) (ryan and deci, 2000). in the context of snss, individuals with controlled motivation may participate to obtain the external rewards that some snss use to encourage participatory behaviours and rank their users. for example, weibo.com, a chinese sns similar to twitter, provides medals to 1 intrinsic motivation is also autonomy-oriented. this study only focuses on people’s extrinsic motivations. 4 australasian journal of information systems wang 2016, vol 20, research article social network motivation following a crisis encourage participatory behaviours and rank users.2 with higher rankings, people obtain higher statuses and more privileges (e.g., they can search for others with the same or similar interests). thus, users of weibo.com with controlled motivation primarily focus on how to achieve higher rankings. conversely, users with autonomous motivation may participate because they enjoy communicating and interacting on snss and integrate those behaviours into their own values (wang, 2014). these users are likely to place less value on external rewards. in these two contexts, individuals may have the same levels of motivation, but different orientations. autonomous motivation takes time to develop and maintain (ryan and deci, 2000); however, crises usually only last for short periods. accordingly, it is more appropriate to examine how people’s general motivation in relation to snss influences their subsequent participatory behaviours within snss following a crisis (wang, 2014). spence et al. (2015) also suggest that people’s daily patterns of use and gratification processes within snss might influence their behaviours within snss during a crisis. the next section sets out the hypotheses developed based on sdt and other relevant research. 3.2 social influence and controlled motivation social influence is defined as “the degree to which an individual perceives that important others believe he or she should use the new system” (venkatesh et al., 2003, p. 451). social influence can affect individuals’ behaviours through three mechanisms: compliance, internalization, and identification (venkatesh and davis, 2000). this study focuses on the compliance mechanism to exclude the effect of a supportive environment (thompsonet al., 1991). the compliance mechanism alters individuals’ motivations in response to social pressure without changing their belief structures (venkatesh et al., 2003). thus, in relation to social influence, individuals may be motivated to perform a behaviour without holding a positive attitude toward the behaviour if they perceive that important social actors wish them to engage in the behaviour (warshaw, 1980). in the context of snss, if individuals’ important friends, classmates, or workmates want them to participate in specific snss and value high rankings within those snss, individuals may engage in the necessary behaviours to try to achieve higher rankings within those snss. by participating in those snss and achieving high rankings, individuals may feel that their images are enhanced (moore and benbasat, 1991). in such a context, individuals are motivated to participate to obtain higher rankings valued by others, but they may not integrate participation into their belief structures. thus, it is hypothesized that: h1: social influence (compliance) is positively related to controlled motivation in snss. 3.3 trust and autonomous motivation three types of trust beliefs have been identified: benevolence, competence, and integrity (bhattacherjee, 2002; mcknight et al., 2002). here competence trust beliefs refer to individuals’ perception of others’ knowledge in a certain area (bhattacherjee, 2002). this study examines how people’s trust toward others can support their autonomous motivation within snss. since people generally join snss to interact with others rather than to complete predefined tasks (e.g., finish business transactions), competence trust beliefs are not relevant to this study. accordingly, this study focuses on the trust beliefs of benevolence and integrity to determine people’s perceptions toward others. benevolence trust beliefs refer to trustees’ acting in trusters’ interests (mcknight et al. 2002) and can be used to determine whether trustees’ intentions are based on altruism (mayer et al. 1995). in the context of snss, benevolence trust beliefs between participants could reveal whether individuals perceive others as caring about their interests or how others show their willingness to assist in relation to specific needs. for example, if individuals post questions about a crisis on snss, others may attempt to answer these questions, resulting in a higher level of benevolence trust beliefs toward other participants. in this circumstance, individuals 2 http://help.weibo.com/faq/q/358 5 australasian journal of information systems wang 2016, vol 20, research article social network motivation following a crisis that make the posts are more likely to feel supported by others, which in turn support their autonomous motivation (ryan and deci, 2000). integrity trust beliefs refer to the degree to which trustees will adhere to a set of rules of interaction acceptable to trusters (mcknight et al. 2001). integrity trust reflects whether trustees keep their promises and behave reliably (mcknight et al. 2002). rules of integrity for participants within snss can include producing various types of content and friendly interactions among users. thus, integrity trust beliefs are used to refer to the degree to which other participants are friendly and keep their commitments. when individuals perceive that others are truthful in their sns interactions, their integrity trust beliefs are likely to increase. consequently, such individuals may feel that they can reliably participate in snss in the future and their autonomous motivation is thus supported (ryan and deci, 2000). previous literature has shown that integrity trust beliefs can enhance people’s willingness to engage in online transactions (bhattacherjee, 2002). to summarize, we argue that: h2a: benevolence trust beliefs are positively related to autonomous motivation in snss. h2b: integrity trust beliefs are positively related to autonomous motivation in snss. 3.4 controlled motivation and participatory behaviours previous sdt studies have shown that autonomous and controlled motivation may lead to different task outcomes (gagne and deci, 2005; wang, 2014). controlled motivation, such as external rewards, often leads to mundane and simple tasks (amabile et al., 1990; mcgraw, 1978; mcgraw and mccullers, 1979). for example, in the context of information technology (it) support, wang et al. (2015) find that external motivation, a specific type of controlled motivation, is positively related to mundane knowledge contribution (i.e., knowledge contribution via documents). in the context of snss, individuals with controlled motivation are likely to focus on external rewards (e.g., medals, high rankings) and tend to minimize their cognitive efforts whenever possible. thus, they may select easier participatory behaviours, such as commenting and sharing content. commenting is a simple task that requires relatively little cognitive effort, as people can comment on already created content by adding a few words (starbird and palen, 2012). sharing content is also an easy task; content can be shared just by clicking a “share” button. following a crisis, people with controlled motivations need timely information about the crisis (boyle et al., 2004) and tend to minimize the cognitive effort they expend during the process. consequently, they are more likely to comment or share content (starbird and palen, 2012) to maximize their rewards. wang (2014) also find that people’s controlled motivation is positively related to commenting on snss following a crisis. before commenting, people need to browse the content, at least cursorily, to ensure their comment makes sense. therefore, we hypothesize that: h3a: controlled motivation is positively related to browsing content on snss. h3b: controlled motivation is positively related to commenting on snss. h3c: controlled motivation is positively related to sharing content on snss. 3.5 autonomous motivation and participatory behaviours autonomous motivation often leads to people seeking out complex and challenging tasks (mcgraw, 1978; vansteenkiste et al., 2004), but may not influence simple or mundane tasks (gagne and deci, 2005). individuals with high levels of autonomous motivation do not engage in behaviours for external rewards, but act because they understand the rationale of those activities and integrate those activities into their own values (ryan and deci, 2000). for example, wang et al. (2015) find that the intrinsic motivation of system administrators is positively related to challenging knowledge contribution (i.e., knowledge contribution via tickets). following a crisis, individuals with autonomous motivation understand that they need to facilitate the distribution of crisis-related information within snss to ensure that others have 6 australasian journal of information systems wang 2016, vol 20, research article social network motivation following a crisis access to the information. these individuals may not wish to exert much effort in commenting (wang, 2014), as commenting is neither challenging nor helpful in distributing the information. conversely, sharing content can help distribute the information and inform more people about the crisis. thus, these individuals will probably engage in content sharing. again, before sharing content, these individuals need to browse the content to learn more about crises (boyle et al. 2004; procopio and procopio, 2007) and ensure that the information is relevant to others. thus, it was hypothesized that: h4a: autonomous motivation is positively related to browsing content on snss. h4b: autonomous motivation is negatively related to commenting on snss. h4c: autonomous motivation is positively related to sharing content on snss. 3.6 participatory behaviours and situational self-awareness situational self-awareness includes three sub-dimensions: private self-awareness, public selfawareness, and awareness of immediate surroundings (govern and marsch, 2001). public selfawareness includes paying attention to the features presented to the public (e.g., physical features); private self-awareness includes being attentive to internal, personal aspects (e.g., feelings of physical pleasure) (buss, 1980).these two dimensions reflect people’s self-focus; thus, they do not assist in examining how people understand their environment following a crisis. this study focuses on the third dimension—awareness of immediate surroundings; that is, the situation that arises when an individual’s attention is focused on something other than themselves. situational self-awareness can be helpful to reveal how people understand what is happening in relation a specific crisis. following a crisis, people seek up-to-date, relevant information about the crisis (boyle et al., 2004; procopio and procopio, 2007). by browsing, commenting on or sharing crisis-related content within snss, people can better understand the crisis and what is occurring, which in turn should lead to a higher level of situational selfawareness. thus, it is hypnotized that: h5a: browsing content on snss is positively related to situational self-awareness. h5b: commenting on snss is positively related to situational self-awareness. h5c: sharing content on snss is positively related to situational self-awareness. as discussed above, people with various types of motivation may participate differently in snss following a crisis. specifically, people may share crisis-related content within snss for different reasons (i.e., with different types of motivation). people with controlled motivation may share content to gain external rewards, while those with autonomous motivation may share content to facilitate information sharing. therefore, the effect of sharing content on situational self-awareness may be different in different scenarios. people with controlled motivation may simply increase the frequency of sharing the crisis-related content to maximize rewards without carefully browsing each piece of content; thus, such sharing may not assist them to better understand the crisis. conversely, people with autonomous motivations may carefully select and share certain content to help others better understand the crisis; thus, sharing that content can help them better understand the crisis. to summarize, we hypothesize that: h6: motivation and sharing content on snss will interact such that: (a) content sharing on snss by people with controlled motivations is not significantly related to situational selfawareness; (b) content sharing on snss by people with autonomous motivation is significantly related to situational self-awareness. our research model is presented in figure 1. 7 australasian journal of information systems wang 2016, vol 20, research article social network motivation following a crisis social influence trust (benevolence) trust (integrity) controlled motivation autonomous motivation browse content on snss comment on snss share content on snss situational selfawareness h1 h2a h2b h5a h5b h5c h3a h3b h3c h4a h4b h4c figure 1. research model 4 method this section describes the context, sample, measures, data analysis, and the results of the study. 4.1 context the mh370 incident was selected as the context for this study. mh370 was a flight scheduled to fly internationally from kuala lumpur, malaysia to beijing, china on march 8, 20143. after less than one hour in the air, it lost contact with air traffic control and disappeared at 01:20 myt. it was not until later that night that malaysia airlines reported the flight missing. a multinational search effort then started in the gulf of thailand and the south china sea and soon extended to the strait of malacca and andaman sea. later, the focus of the search turned to the southern part of the indian ocean. no confirmed flight debris was found or crash site identified. consequently, unofficial news items were published and rumours spread about the mh370 incident and people began continuously searching for news updates from various sources. this makes the mh370 incident an appropriate context to examine how the public participates on snss to learn more about crises. data was collected between march 24 and april 6 in 2014. these dates were selected as by this period, participants had already participated in snss in some way to learn more about the incident and might not have turned to other events or incidents. of the 227 passengers on the mh370 flight, 152 were chinese citizens. thus, the chinese public was particularly eager to learn more about the incident and was selected to participate in this study. 4.2 sample participants came from two sources. more than half of the participants (i.e. 64 percent, n = 130) were junior level business students at a large public university in the northeast of china. the remaining participants (i.e., 36 percent, n = 73) were recruited using a snowball procedure (i.e., the student participants were asked to refer non-student friends or relatives who often used snss and might be interested in the mh370 incident). screening questions were added to remove those who either did not use snss regularly or had not browsed, commented on or shared content of the mh370 incident on snss. approximately one percent course credit was awarded to students for participating and referring additional participants. table 1 lists the demographic information of the participants. 3 https://en.wikipedia.org/wiki/malaysia_airlines_flight_370 8 australasian journal of information systems wang 2016, vol 20, research article social network motivation following a crisis category student sample “snowball” sample (n = 130) (n = 73) gender female 81 (62.3%) 47 (64.4%) male 49 (37.7%) 26 (35.6%) age 21.6 (sd 1.2) 34.9 (sd 10.2) sns used renren 5 (3.8%) 7 (9.6%) wechat 100 (76.9%) 41 (56.2%) weibo 22 (16.9%) 16 (21.9%) other 3 (2.3%) 9 (12.3%) tenure (years) 2.5 (sd 1.1) 2.4 (sd 1.9) table 1. sample demographic information snowball sampling is an appropriate mode of data collection for this study. this method has been found to be effective in locating, accessing, and involving hard-to-reach populations (cohen and arieli, 2011; valdez and kaplan, 1999). the target population of this study is those who browsed, commented on, or shared content related to the mh370 incident in snss. as not everyone is interested in the mh370 incident, it is challenging to locate members of this population. thus, snowball sampling is suitable for this study. admittedly, representativity can be an issue of snowball sampling (cohen and arieli, 2011). thus, the age groups of participants and the industries they worked in were closely examined. the sample covered a variety of age groups and industries (see table 2). thus, the sample had an acceptable level of representativity across the whole population of sns users.4 category sample age below 29 81.28% 30-39 3.45% 40-49 13.79% above 50 1.48% industry student 48.77% banking/finance 11.82% education (k–12) 3.94% government 7.39% higher education 1.97% hospitality 0.49% manufacturing 4.43% transportation 1.48% health care 2.96% other 16.75% table 2. sample’s age and industry 4.3 measures items of our study were adapted from the literature. specifically, items for controlled motivation, autonomous motivation, browsing content, sharing content, and commenting on snss were adapted from ke and zhang (2010). items for social influence were adapted from moore and benbasat (1991) and taylor and todd (1995); items for trust were adapted from mcknight et al. (2002); items for situational self-awareness were adapted from govern and marsch (2001). each question was measured on a 7-point, likert-type scale, where “1” indicated “strongly disagree” and “7” indicated “strongly agree.” the final items used are shown in the appendix. 4 http://www.alibuybuy.com/posts/88202.html 9 australasian journal of information systems wang 2016, vol 20, research article social network motivation following a crisis 4.4 analysis and results our model was tested using partial least squares (pls). pls is appropriate for predictive models and theory building (barclay et al., 1995; chin, 2010). smartpls (ringle et al., 2005) was used in the analysis, and the bootstrap re-sampling method (using 5000 samples) was used to determine the significance of the paths in the structural model. pls is preferred as the analytical technique for this study for several reasons. first, as few similar studies have been conducted, this study is exploratory in nature and pls is preferred over covariance-based statistical tools in exploratory studies (gefen et al., 2011). second, pls works well with smallto-medium sized samples (chin, 2010). third, the model tested in this study predicts people’s situational self-awareness and pls is suited to predictive models (chin, 2010). first, common method variance (cmv) was examined (lindell and whitney, 2001; podsakoff et al. 2003). an exploratory factor analysis of all items showed that six factors explained 71.01 percent of the variance, with no single factor accounting for significant loading (at the p < .10 level) for any item. the second-smallest positive correlation among the manifest variables was then used as a conservative estimate of cmv (lindell and whitney, 2001). following adjustment, all the significant correlations remained significant. thus, cmv was probably not an issue in this study. next, data analysis was conducted in two stages: measurement model and structural model assessment. in the first stage, convergent validity was confirmed if it met the following criteria (gefen and straub, 2005): (i) each item loaded significantly on its respective construct and none of the loadings were below the cut-off value of 0.60 (see table 3); (ii) the composite reliabilities (crs) of all constructs were above 0.70 (see table 3); and (iii) the average variance extracted (ave) from all constructs was above the threshold value of 0.50 (tables 3). items mean s.d. loadings ave cr si1 5.49 1.15 0.70 0.61 0.86 si2 5.50 1.20 0.75 si3 3.97 1.54 0.83 si4 4.17 1.43 0.83 tb1 4.34 1.36 0.73 0.70 0.90 tb2 4.97 1.22 0.89 tb3 4.67 1.22 0.84 tb4 4.76 1.11 0.87 ti1 4.85 1.16 0.89 0.85 0.96 ti2 4.56 1.21 0.94 ti3 4.52 1.15 0.93 ti4 4.68 1.14 0.93 cm1 5.03 1.30 0.66 0.61 0.82 cm2 4.49 1.47 0.88 cm3 4.84 1.35 0.79 am1 5.57 0.96 0.83 0.66 0.86 am2 5.33 1.28 0.85 am3 4.94 1.24 0.76 vsns1 5.31 1.43 0.98 0.97 0.98 vsns2 5.18 1.49 0.98 fsns1 4.27 1.71 0.96 0.92 0.96 fsns2 4.13 1.76 0.96 csns1 3.83 1.77 0.97 0.95 0.97 csns2 3.88 1.72 0.98 sa1 5.29 1.35 0.86 0.81 0.93 sa2 5.33 1.15 0.92 sa3 5.34 1.11 0.93 table 3. items and descriptive statistics 10 australasian journal of information systems wang 2016, vol 20, research article social network motivation following a crisis discriminant validity was established by ensuring that the square root of ave for each construct exceeded all the correlations between that construct and any other construct (gefen and straub, 2005) (see table 4). overall, the measures demonstrated good psychometric properties. 1 2 3 4 5 6 7 8 9 1 social influence 0.78 2 trust (benevolence) 0.57 0.84 3 trust (integrity) 0.46 0.71 0.92 4 controlled motivation 0.49 0.52 0.38 0.78 5 autonomous motivation 0.51 0.62 0.54 0.59 0.81 6 browse content in snss 0.32 0.34 0.24 0.42 0.42 0.98 7 share content in snss 0.29 0.33 0.13 0.35 0.35 0.56 0.96 8 comment in snss 0.34 0.36 0.21 0.34 0.28 0.44 0.70 0.97 9 situational self-awareness 0.26 0.34 0.19 0.38 0.40 0.54 0.37 0.35 0.90 table 4. correlation between constructs and square-root of aves (on diagonal) in the second stage, the structural model was assessed using r2 measures and path coefficients. r2 shows the amount of variance in a dependent variable explained by independent variables (gefen et al., 2000) and path coefficients indicate the significance of the relationships between the constructs. h1, stating that social influence is positively associated with controlled motivation, was supported (b = 0.49, p < .001). h2a argues that benevolence trust beliefs are positively associated with autonomous motivation. this hypothesis was supported (b = 0.48, p < .001). h2b posits that integrity trust beliefs are positively related to autonomous motivation. this hypothesis was supported (b = 0.20, p < .05). h3a states that controlled motivation is positively associated with browsing content on snss. this hypothesis was supported (b = 0.26, p < .01). h3b states that controlled motivation is positively associated with commenting on snss. this hypothesis was supported (b = 0.27, p < .001). h3c argues that controlled motivation is positively associated with sharing content on snss. this hypothesis was supported (b = 0.23, p < .01). h4a states that autonomous motivation is positively associated with browsing content on snss. this hypothesis was supported (b = 0.27, p < .01). h4b states that autonomous motivation is negatively associated with commenting on snss; however, this hypothesis was not supported (b = 0.12, p > .05). h4c argues that autonomous motivation is positively associated with sharing content on snss. this hypothesis was supported (b = 0.21, p < .01). h5a, stating that browsing content is positively associated with situational selfawareness, was supported (b = 0.48, p < .001). h5b argues that commenting is positively related to situational self-awareness. this hypothesis was supported (b = 0.13, p < .05). h5c posits that sharing content is positively related to situational self-awareness; however, this hypothesis was not supported (b = 0.00, p > .05). these findings are presented in figure 2. 11 australasian journal of information systems wang 2016, vol 20, research article social network motivation following a crisis social influence trust (benevolence) trust (integrity) controlled motivation autonomous motivation browse content on snss comment on snss share content on snss situational selfawareness 0.49*** 0.48*** 0.20* 0.48*** 0.13* 0.01 0.26** 0.27*** 0.23** 0.27** 0.12 0.21** r2 = 0.237 r2 = 0.402 r2 = 0.310 r2 = 0.223 r2 = 0.124 r2 = 0.154 * p < .05 ** p < .01 *** p < .001 figure 2. model results the stone-geisser (q2) test was then conducted to assess the predictive quality of the model (geisser, 1975; stone, 1974). the model has estimation relevance with values of q2 above 0. the q2 for social influence was 0.61; the q2 for trust (benevolence) was 0.70; the q2 for trust (integrity) was 0.85; the q2 for controlled motivation was 0.61; the q2 for autonomous motivation was 0.66; the q2 for browsing content on snss was 0.97; the q2 for commenting on snss was 0.95; the q2 for sharing content on snss was 0.92; and the q2 for situational selfawareness was 0.75. thus, the model had good predictive relevance. 4.5 moderation analysis: motivation, participatory behaviours, and situational self-awareness to test h6, participants were classified as those with a relatively high level of controlled motivation and those with a relatively high level of autonomous motivation. the process of koestner and zuckerman (1994) was followed to classify participants and the z-scores of controlled motivation and autonomous motivation were calculated. a participant was classified as having controlled motivation if his/her z-score on the controlled motivation scale was higher than his/her z-score of the autonomous motivation scale; otherwise, he/she was classified as being autonomously motivated. the whole sample was thus divided into two subsamples. next, the relationship between sharing content and situational self-awareness was examined for participants with different types of motivation (see figure 3).the results showed that sharing content was not significantly related to situational self-awareness in participants with controlled motivation, supporting h6a. however, sharing content was only marginally significantly related to situational self-awareness in participants with autonomous motivation. thus, h6b was marginally supported. 12 australasian journal of information systems wang 2016, vol 20, research article social network motivation following a crisis browse content on snss comment on snss share content on snss situational selfawareness 0.50*** 0.35*** -0.13 r2 = 0.432 panel a: participants with controlled motivation (n=102) browse content on snss comment on snss share content on snss situational selfawareness 0.44*** -0.06 0.13# r2 = 0.244 # p < .10 * p < .05 ** p < .01 *** p < .001 panel b: participants with autonomous motivation (n=101) figure 3. participatory behaviours and situational self-awareness for participants with different types of motivation 5 discussion this study applies sdt to examine how different social factors support different types of motivation, which in turn result in different participatory behaviours. sdt is also used to examine how participatory behaviours contribute to people’s situational self-awareness. data was collected and analysed following the mh370 incident. the results show that while social influence is positively related to controlled motivation (h1), trust is positively associated with autonomous motivation (h2). different types of motivation are also found to lead to different participatory behaviours. controlled motivation is positively related to browsing content, commenting, and sharing content (h3); however, autonomous motivation is only positively associated with browsing and sharing content (h4). although autonomous motivation is not negatively related to commenting (h4b is rejected), the results still indicate that people with autonomous motivation tend not to comment on others’ content. overall, it is found that browsing content and commenting are positively related to people’s situational self-awareness (h5). further, the results show that sharing content has a marginal effect on situational self-awareness for people with autonomous motivation, but no effect on those with controlled motivation (h6) (see table 5). this study has important theoretical and practical implications. 13 australasian journal of information systems wang 2016, vol 20, research article social network motivation following a crisis hypothesis results h1 social influence (compliance) is positively related to controlled motivation. supported h2a benevolence trust beliefs are positively related to autonomous motivation. supported h2b integrity trust beliefs are positively related to autonomous motivation. supported h3a controlled motivation is positively related to browsing content on snss. supported h3b controlled motivation is positively related to commenting on snss. supported h3c controlled motivation is positively related to sharing content on snss. supported h4a autonomous motivation is positively related to browsing content on snss. supported h4b autonomous motivation is negatively related to commenting on snss. rejected h4c autonomous motivation is positively related to sharing content on snss. supported h5a browsing content in snss is positively related to situational selfawareness. supported h5b commenting in snss is positively related to situational selfawareness. supported h5c sharing content in snss is positively related to situational selfawareness. rejected h6 motivation and sharing content on snss will interact such that: (a) content sharing on snss by people with controlled motivations is not significantly related to situational self-awareness; (b) content sharing on snss by people with autonomous motivation is significantly related to situational self-awareness. h6a supported; h6b marginally supported table 5. hypotheses testing summery 5.1 theoretical implications this study makes three important theoretical contributions. first, it shows that different types of motivation lead to different participatory behaviours on snss following a crisis. commenting on and sharing content require relatively little cognitive effort; thus, people with controlled motivation probably browse, comment on and share relevant content on snss following the crisis. the significant relationship between controlled motivation and commenting in this study is also consistent with the results of wang (2014). conversely, people with autonomous motivation probably understand the rationale for participating in snss following a crisis (i.e., to help others understand the crisis better) and integrate this behaviour into their own values. consequently, they are more likely to browse and share relevant content. those results confirm sdt within the context of the mh370 incident. specifically, sdt argues that people’s motivation can be autonomous or controlled, which could lead to various behaviours. our results show that people’s various types of motivation indeed lead to various participatory behaviours. besides, this study also extends the literature (e.g., wang, 2014) by examining additional participatory behaviours (i.e., browsing and sharing content) and addresses the gap in the research in relation to how people participate in snss to gain information about crises (pan et al., 2012). second, the results show that controlled and autonomous motivation are supported by different social factors. social influence is positively related to controlled motivation and trust toward other participants is positively associated with autonomous motivation. these results provide valuable insight into how controlled motivation differs from autonomous motivation. our study also extends previous research (e.g., wang, 2014; wang and li, 2014) by examining the antecedent of controlled motivation and clarifying the role of trust on autonomous motivation. 14 australasian journal of information systems wang 2016, vol 20, research article social network motivation following a crisis third, our study shows that participatory behaviours can help enhance a person’s situational self-awareness and extends previous research (e.g., wang, 2014) by empirically examining the effects of participatory behaviours. the results show that participatory behaviours are indeed important in the public understanding about a crisis and require further examination. besides, participatory behaviours have various effects on people with different types of motivation. content sharing by those with controlled motivation is not related to situational self-awareness. thus, people with controlled motivation may simply share content without carefully browsing content to obtain external rewards. conversely, people with autonomous motivation are not motivated by external rewards, and probably browse the content before sharing. sharing content is found to have some effect on people with autonomous motivation; however, the effect is only marginally significant. one possible reason is that since people with autonomous motivation are trying to increase others’ understanding of a crisis, they may not share content in which they are not confident and thus limiting the frequency of sharing. clearly, more studies are needed to examine the activity of sharing content following crises. 5.2 practical implications this study also has important practical implications. first, it shows that snss can help people increase situational self-awareness and make sense of a crisis. specifically, our results show that overall browsing and commenting on crisis-related content can help people better understand a crisis. these results provide important insights for governments and relevant non-profit organizations as to how the public responds and understands a crisis through snss. thus, people should be encouraged to use snss to browse and comment on content related to crises on snss, and governments should try to take advantage of snss to distribute the crisisrelated information. for example, the government could set up an sns account and publish verified information that could be browsed and commented upon by sns users. the information posted could help the public better understand the crisis and plan and coordinate remedial activities. emergency managers could also be appointed to help the public interpret information in relation to a crisis. emergency managers need to re-post and update regularly to ensure that the updates appear in searchers related to the crisis (spence et al., 2015). second, the results show that different types of motivation lead to different participatory behaviours, which in turn have different effects on situational self-awareness. these results provide important guidelines to practitioners such as emergency managers. specifically, emergency managers can use different approaches to enhance people’s situational selfawareness and help them better understand a crisis. if people have controlled motivation, simply browsing content and commenting can increase their situational self-awareness. thus, practitioners should collaborate with operators of snss to provide extra rewards to people for commenting on crisis-related content to ensure that people spend more effort browsing content and commenting. when people have autonomous motivation, sharing content has a marginal effect on situational self-awareness. thus, content sharing should be encouraged and supported. practitioners can work with operators of snss to provide feedback as to how many people have browsed their shared content, and how the shared content benefits them (wang and clay, 2012). third, our results show that various types of motivation are supported differently. controlled motivation is enhanced by social influence; autonomous motivation is supported by people’s trust toward other participants. these results can help practitioners such as emergency managers better motivate people. for example, emergency managers may want to support different types of motivation at different stages of a crisis. when a crisis has just occurred and people are seeking information about the crisis, both types of motivation should be supported. in this context, emergency managers can support both social influence and trust such that people with different types of motivation share the content and facilitate the distribution of information. however, following a crisis, the government may wish to begin various initiatives to support people’s resilience and post information on the initiatives on snss. in such a context, controlled motivation should be supported to encourage people to comment on the 15 australasian journal of information systems wang 2016, vol 20, research article social network motivation following a crisis information and provide relevant feedback to the government. therefore, social influence among people should be supported to increase their controlled motivation. 5.3 limitations and opportunities for future studies this study has several limitations. first, the mh370 incident is selected as the context for this study. while the results provide valuable insights into the effect of people’s participatory behaviours on snss following the mh370 incident, these results may not be generalizable to other crises. future studies are needed to confirm people’s snss behaviours in other contexts. second, this study uses snowball sampling to recruit participants from different backgrounds. however, as the sample could be biased, the results must be interpreted cautiously. future studies could use other methods (e.g., posting a survey link on popular snss and in discussion forums related to crises) to assess whether the conclusions reached in this study still hold. notwithstanding these limitations, our findings suggest a number of interesting avenues for future research. first, other social factors supporting different types of motivation could be examined. future studies can also further divide autonomous motivation and controlled motivation and examine how different types of motivation can result in different participatory behaviours. this study focuses on the relationship between people’s participatory behaviours and their own situational self-awareness. future studies could examine how other participants’ behaviours help enhance people’s situational self-awareness following a crisis. 6 conclusion this study examines how people’s participatory behaviours on snss following a crisis are supported by different types of motivation and how participatory behaviours can help people better understand crises. using data collected following the mh370 incident, our study shows that different types of motivation can result in different participatory behaviours. further, browsing content and commenting are positively related to people’s situational self-awareness. future studies are needed to extend this research across other contexts and further examine the relationship between motivation and participatory behaviours. references ahmed, a., & sinnappan, s. (2013). “the role of social media during queensland floods: an empirical investigation on the existence of multiple communities of practice (mcops)," pacific asia journal of the association for information systems, 5(2): 1-22. al-ani, b., mark, g., chung, j., & jones, j. (2012). “the egyptian blogosphere: a counternarrative of the revolution.” proceedings of the acm 2012 conference on computer supported cooperative work, seattle, wa, 17-26. amabile, t. m., goldfarb, p., & brackfield, s. c. (1990). “social influences on creativity: evaluation, coaction, and surveillance.” creativity research journal, 3(1): 6-21. barclay, d., higgins, c. & thompson, r. (1995). "the partial least squares approach to causal modeling, personal computing adoption and use as an illustration," technology studies, 2(2): 285-309. benevenuto, f., rodrigues, t., cha, m., & almeida, v. (2009). “characterizing user behavior in online social networks,” proceedings of the 9th acm sigcomm conference on internet measurement, pp. 49-62. bhattacherjee, a. (2002). “individual trust in online firms: scale development and initial test,” journal of management information systems, 19(1): 211–241. boyd, d. m., & ellison, n. b. (2008). “social network sites: definition, history, and scholarship.” journal of computer-mediated communication, 13(1): 210-230. boyle, m., schmierbach, m., armstrong, c., & mcleod, d. (2004). “information seeking and emotional reactions to the september 11 terrorist attacks.” journalism and mass communication quarterly, 81(1): 155-167. 16 australasian journal of information systems wang 2016, vol 20, research article social network motivation following a crisis buss, a. h. (1980). self-consciousness and social anxiety. san francisco: freeman. chen, r., sharman, r., rao, h. r., & upadhyaya, s. (2007). “design principles for critical incident response systems.” information systems and e-business management, 5(3): 201-227. chin, w. w. (2010). "how to write up and report pls analyses," in vinzi, v. e., chin, w. w., henseler, j., and wang, h. (eds.), handbook of partial least squares: concepts, methods and applications, springer verlag, new york, ny, pp. 655-690. cohen, n., & arieli, t. (2011). “field research in conflict environments: methodological challenges and snowball sampling.” journal of peace research, 48(4): 423-435. fritz, c. e. (1961). “disaster,” in merton, r.k. and nisbet, r.a. (eds.), contemporary social problems, harcourt, brace & world, new york, ny, pp. 651-694. gagne, m. & deci, e. l. (2005). “self-determination theory and work motivation.” journal of organizational behavior, 26(4): 331-362. gefen, d., rigdon, e. e., & straub, d. (2011). “an update and extension to sem guidelines for administrative and social science research.” mis quarterly, 35(2): iii-xiv. gefen, d., & straub, d. (2005). “a practical guide to factorial validity using pls-graph: tutorial and annotated example.” communications of the association for information systems, 16: 91-109. gefen, d., straub, d., & boudreau, m. (2000). “structural equation modeling and regression: guidelines for research practice.” communications of the association for information systems, 4: 1-77. geisser, s. (1975). “the predictive sample reuse method with applications.” journal of the american statistical association, 70(350): 320–328. govern, j. m., & marsch, l. a. (2001). “development and validation of the situational selfawareness scale.” consciousness and cognition, 10(3): 366–378. hagar, c., & haythornthwaite, c. (2005). “crisis, farming and community.” journal of community informatics, 1(3): 41-52. ke, w., & zhang, p. (2010). “the effects of extrinsic motivations and satisfaction in open source software development,” journal of the association for information systems, 11(12): 784-808. koestner, r., & zuckerman, m. (1994).“causality orientations, failure, and achievement.” journal of personality, 62(3): 321–346. lindell, m. k., & whitney, d. j. (2001). “accounting for common method variance in crosssectional research designs.” journal of applied psychology, 86(1): 114-121. mayer, r. c., davis, j. h., & schoorman, f. d. (1995). “an integrative model of organizational trust,” academy of management review, 20(3): 709-734. mayfield, a. (2006). "what is social media?" spannerworks. mcgraw, k. o. (1978). “the detrimental effects of reward on performance: a literature review and a prediction model.” in lepper, m. r. and greene, d. (eds.), the hidden costs of reward, erlbaum, hillsdale, nj, pp. 33-60. mcgraw, k. o., & mccullers, j. c. (1979). “evidence of a detrimental effect of extrinsic incentives on breaking a mental set.” journal of experimental social psychology, 15(3): 285-294. mcknight, d. h., & chervany, n. l. (2001) “what trust means in e-commerce customer relationships: an interdisciplinary conceptual typology.” international journal of electronic commerce, 6(2): 35–39. 17 australasian journal of information systems wang 2016, vol 20, research article social network motivation following a crisis mcknight, d. h., choudhury, v., & kacmar, c. (2002). "developing and validating trust measures for e-commerce: an integrative typology," information systems research, 13(3): 334-359. mitroff, i. i. (2004). crisis leadership: planning for the unthinkable. john wiley & sons, hoboken, nj. moore, g. c., & benbasat, i. (1991)."development of an instrument to measure the perceptions of adopting an information technology innovation," information systems research, 2(3): 192-222. oh, o., agrawal, m., & rao, h. r. (2013). “community intelligence and social media services: a rumor theoretic analysis of tweets during social crises.” mis quarterly, 37(2): 407426. palen, l., hiltz, s. r., & liu, s. b. (2007). “online forums supporting grassroots participation in emergency preparedness and response.” communications of the acm, 50(3): 54-58. palen, l., vieweg, s., liu, s. b., & hughes, a. l. (2009). “crisis in a networked world features of computer-mediated communication in the april 16, 2007, virginia tech event.” social science computer review, 27(5): 1-14. pan, s. l., pan, g., & leidner, d. e. (2012). “crisis response information networks.” journal of the association for information systems, 13(1): 31-56. pang, n., & ng, j. (in press). “twittering the little india riot: audience responses, information behavior and the use of emotive uses”. computers in human behavior. park, n., kee, k. f., & valenzuela, s. (2009). “being immersed in social networking environment: facebook groups, users and gratifications, and social outcomes.” cyberpsychology & behavior, 12(6): 729-733. podsakoff, p. m., mackenzie, s., & lee, j. (2003). “common method biases in behavioral research: a critical review of the literature and recommended remedies.” journal of applied psychology, 88(5): 879-903. procopio, c. h., & procopio, s. t. (2007). “do you know what it means to miss new orleans? internet communication, geographic community, and social capital in crisis.” journal of applied communication research, 35(1): 67-87. raacke, j., & bonds-raacke, j. (2008). “myspace and facebook: applying the uses and gratifications theory to exploring friend-networking sites.” cyberpsychology & behavior, 11(2): 169-174. reinholt, m., pedersen, t., & foss, n. j. (2011). “why a central network position isn't enough: the role of motivation and ability for knowledge sharing in employee networks.” academy of management journal, 54(6): 1277-1297. ringle, c. m., wende, s., & will, s. (2005). smartpls 2.0 (m3) beta, hamburg, http://www.smartpls.de. ryan, r. m., & deci, e. l. (2000). “self-determination theory and the facilitation of intrinsic motivation, social development, and well-being.” american psychologist, 55(1): 68-78. sarcevic, a., palen, l., white, j., starbird, k., bagdouri, m., & anderson, k. (2012). “’beacons of hope’ in decentralized coordination: learning from on-the-ground medical twitterers during the 2010 haiti earthquake 2012”, proceedings of 2012 acm conference on computer supported cooperative work, seattle, wa. schultz, f., utz, s., & göritz, a. (2011). "is the medium the message? perceptions of and reactions to crisis communication via twitter, blogs and traditional media," public relations review, 37(1): 20-27. 18 australasian journal of information systems wang 2016, vol 20, research article social network motivation following a crisis semaan, b., & mark, g. (2011). “technology-mediated social arrangements to resolve breakdowns in infrastructure during ongoing disruption.” acm transactions on computer-human interaction, 18(4): 1-21. shklovski, i., burke, m., kiesler, s., & kraut, r. (2010). “technology adoption and us in the aftermath of hurricane katrina in new orleans.” american behavioral scientist, 53(8): 1128-1246. shklovski, i., palen, l., & sutton, j. (2008). “finding community through information and communication technology during disaster events.” proceedings of 2008 acm conference on computer supported cooperative work, san diego, vancouver, bc, canada, pp. 127-136. spence, p. r., lachlan, k. a., lin, x., & del greco, m. (2015). “variability in twitter content across the stages of a natural disaster: implications for crisis communication.” communication quarterly, 63(2): 171-186. spence, p. r., westerman, d., skalski, p., seeger, m., sellnow, t., & ulmer, r. r. (2006). “gender and age effects on information seeking after 9/11.” communication research reports, 23(3): 217-223. starbird, k., & palen, l. (2012). “(how) will the revolution be retweeted? information diffusion and the 2011 egyptian uprising.” proceedings of the acm 2012 conference on computer supported cooperative work, pp. 7-16. stone, m. (1974). “cross-validatory choice and assessment of statistical predictions.” journal of the royal statistical society: series b, 36(2): 111–147. taylor, s., & todd, p. a. (1995)."understanding information technology usage: a test of competing models," information systems research, 6(4): 144-176. taylor, m., wells, g., howell, g., & raphael, b. (2012). “the role of social media as psychological first aid as a support to community resilience building.” the australian journal of emergency management, 27(1) 20-26. thompson, r. l., higgins, c. a., & howell, j. m. (1991). "personal computing: toward a conceptual model of utilization," mis quarterly, 15(1): 124-143. valdez, a., & kaplan, c. d. (1999). “reducing selection bias in the use of focus groups to investigate hidden populations: the case of mexican-american gang members from south texas.” drugs and society, 14(1-2): 209-224. vansteenkiste, m., simons, j., lens, w., sheldon, k. m., & deci, e.l. (2004). “motivating learning, performance, and persistence: the synergistic effects of intrinsic goal contents and autonomy-supportive contexts.” journal of personality and social psychology, 87 (2): 246-260. venkatesh, v., & davis, f. d. (2000). "a theoretical extension of the technology acceptance model: four longitudinal field studies," management science, 45(2): 186-204. venkatesh, v., morris, m., davis, g.b., & davis, f. d. (2003). “user acceptance of information technology: toward a unified view.” mis quarterly, 27(3): 425-478. vieweg, s., palen, l., liu, s., hughes, a. & sutton, j. (2008). “collective intelligence in disaster: examination of the phenomenon in the aftermath of the 2007 virginia tech shootings.” proceedings of the 5th international iscram conference. washington, dc, u.s., pp. 44-54. wang, x. (2014). “how do people participate in social network sites after crises? a selfdetermination perspective.” social science computer review, 32(5): 662-677. wang, x., & clay, p. f. (2012). “beyond adoption intention: online communities and member motivation to contribute longitudinally.” journal of organizational computing and electronic commerce, 22(3): 215-236. 19 australasian journal of information systems wang 2016, vol 20, research article social network motivation following a crisis wang, x., clay, p. f., & forsgren, n. (2015)."encouraging knowledge contribution in it support: social context and the differential effects of motivation type", journal of knowledge management, 19(2): 315 333 wang, x., & li, y. (2014). “trust, psychological need, and motivation to produce user generated content: a self-determination perspective.” journal of electronic commerce research, 15(3): 241-253. wang, x., & li, y. (in press). “understanding collaborative resilience from continuous disruption: an actor-network perspective.” behaviour & information technology. warshaw, p. r. (1980). “a new model for predicting behavioral intentions: an alternative to fishbein.” journal of marketing research, 17(2): 153-172. yanay, u., benjamin, s., & yamin, h. g. (2011). “networking emergency teams in jerusalem.” disaster, 35(1): 183-199. 20 australasian journal of information systems wang 2016, vol 20, research article social network motivation following a crisis appendix: instrument5 social influence (moore and benbasat, 1991; taylor and todd, 1995): si1 si1: people who influence my behaviour think that i should use _. si2 si2: people who are important to me think that i should use _. si3 si3: using _ is a status symbol for people in my peer group. si4 si4: people in my peer group who use _ can thereby improve their image. trust (benevolence) (mcknight et al., 2002): tb1 i believe that people in _ would act in my best interest. tb2 if i required help, people in _ would do their best to help me. tb3 people in are interested in my well-being, not just their own. tb4 people in _ are open and receptive to my needs. trust (integrity) (mcknight et al., 2002): ti1 people in _ are truthful in their dealings with me. ti2 i would characterize people in _ as honest. ti3 people in _ would keep their commitments. ti4 people in _ are sincere and genuine. controlled motivation (ke and zhang, 2010): cm1 i am strongly motivated by various kinds of bonus (e.g., credit and rank) i can earn through my participation in _. cm2 i am keenly aware of the possible help for my life and work that may be brought by my participation in _. cm3 i am strongly motivated by the recognition i can earn from other people in _. autonomous motivation (ke and zhang, 2010): am1 i have a strong positive feeling toward _. am2 the reason i participate in _ is because of the way that people interact with each other. am3 my attachment to this group is primarily based on similarity of my desired interaction pattern and those represented by _. browse content in snss (ke and zhang, 2010): vsns1 i devoted a large number of hours to browse information about mh370 in _. vsns2 i devoted a lot of effort to browse information about mh370 in _. share content in snss (ke and zhang, 2010): fsns1 i devoted a large number of hours to share information about mh370 in _. fsns2 i devoted a lot of effort to share information about mh370 in _. comment in snss (ke and zhang, 2010): csns1 i devoted a large number of hours to comment on the information about mh370 in _. csns2 i devoted a lot of effort to comment on the information about mh370 in _. situational self-awareness (govern and marsch, 2001): sa1 right now, i am keenly aware of everything about mh370. sa2 right now, i am conscious of what is going on about mh370. sa3 right now, i am conscious of all objects about mh370. 5 the instrument was first written in english and was then translated into chinese by two bilingual chinese research assistants (ra). the first ra translated the instrument into chinese; the second ra translated the instrument back to english. any differences or inconsistencies were discussed and resolved. 21 australasian journal of information systems wang 2016, vol 20, research article social network motivation following a crisis copyright: © 2016 wang. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. 22 http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 literature review 2.1 crises 2.2 participatory behaviours in snss 3 theoretical background and hypotheses development 3.1 self-determination theory (sdt) 3.2 social influence and controlled motivation 3.3 trust and autonomous motivation 3.4 controlled motivation and participatory behaviours 3.5 autonomous motivation and participatory behaviours 3.6 participatory behaviours and situational self-awareness 4 method 4.1 context 4.2 sample 4.3 measures 4.4 analysis and results 4.5 moderation analysis: motivation, participatory behaviours, and situational self-awareness 5 discussion 5.1 theoretical implications 5.2 practical implications 5.3 limitations and opportunities for future studies 6 conclusion paper title (paper title style) australasian journal of information systems syed, bandara, french & stewart 2018, vol 22, research article csf of bpm in public sector 1 getting it right! critical success factors of bpm in the public sector: a systematic literature review rehan syed queensland university of technology s.abbaszaidi@hdr.qut.edu.au wasana bandara queensland university of technology erica french queensland university of technology glenn stewart queensland university of technology abstract the continuously evolving and dynamic social, economic, political environments and public pressures demand governments to deliver effective and efficient public services. in the quest for meeting these demands, governments respond by designing extensive reforms and performance objectives to improve the efficiency and responsiveness of their agencies. many governments have adopted business process management (bpm) as a strategy to achieve these reforms. however, the successful implementation of bpm initiatives has been a challenging task for agencies responsible for these initiatives, with many projects not reaching completion or not achieving the intended outcomes. this paper reports the results of a systematic literature review on critical success factors (csfs) related to bpm projects in the public sector. we analysed 31 papers in this review covering both the developed and developing country contexts, sourced from refereed and peer reviewed journals. a quasi-deductive approach was applied for the qualitative data analysis using nvivo 10 software. this resulted in the synthesis and identification of 14 critical success factors of bpm in the public sector, each defined and described in detail with specific attention to the sub-factors mentioned within the literature. any differences between developed and developing country contexts were sought for, and the observations critically analysed. we propose a series of research questions, designed to support the progression of bpm in the public sector of developed as well as developing countries. keywords: business process management; public sector organisations; developed countries; developing countries; critical success factors 1 introduction due to rapid changes in global and local socio-economic environments, public sector organisations of both developed and developing countries are faced with many challenges to meet the demands for better public services (thong, yap, & seah, 2000). the public sector is often characterised by archaic and rigid processes, red-tape and bureaucracies, and is often beset with sudden changes of policies (hutton, 1996) as a result of elections and changes in the machinery of government. organisations adopt business process management (bpm) as an approach and strategy to achieve efficiency, productivity, and improved performance (davenport, 1993; trkman, 2010). australasian journal of information systems syed, bandara, french & stewart 2018, vol 22, research article csf of bpm in public sector 2 recognising the value of bpm as an enabler for high performance and increased customer centricity, public sector organisations across the globe have shown keen interest in adopting bpm principles and practices (alves, valença, & santana, 2014; kassahun, molla, & sarkar, 2011; weerakkody, baire, & choudrie, 2006). dubey and bansal (2013) referred to bpm as the key solution to effectively handle citizens’ demand for better government services. weerakkody, janssen, and dwivedi (2011) observed the growing interest and use of bpm techniques to achieve the required organisational and process related changes for egovernment. the significance of and need for bpm was recognised by the world bank (bhatnagar, 2004) as a key factor for e-government success in developing countries. public sector organisations in developing countries in particular, have shown interest in business process management (bpm) since the 1980s (kassahun & molla, 2011; rajapakse, 2013). however, the successful implementation of bpm initiatives has been an ongoing challenge. weerakkody et al. (2006, p. 2) reported that problems related to business processes and ict are “multiplied in the public sector, where inefficient and bureaucratic business processes and disparate legacy is/it systems need to be integrated in an e-government environment”. gulledge and sommer (2002, p. 374) argued that government initiatives mainly focus on “plans and performance measures” with limited emphasis and “guidance on how to link the planning objectives to business processes and eventually align the organisational information systems is not included”. there have been numerous reports of failures of these projects, particularly in developing countries (dada, 2006; heeks, 2003; hunter, 2009). heeks (2003) reported an overall 85% failure rate. similarly, rainford (2006) reported that over 60% of e-government initiatives failed to achieve satisfactory results. heeks and stanforth (2007, p. 165) mentioned that in the last decade, governments in developing countries spent us$3 trillion on information technology (it) projects. rajapakse (2013) noted that developing countries are in the nascent stages of egovernment and struggling to perform in line with the developed countries. yet, despite these struggles and high failure rates, donors and funding bodies continue to fund these projects in developing countries in hope of achieving the millennium development goals (united nations, 2015). critical success factors (csfs) can be used to better systematically manage a program of work as they represent the focal areas that should receive constant and careful attention from management (dobbins & donnelly, 1998). a deep understanding of csfs enables an organisation to assess its threats, opportunities, weaknesses, and strengths in these dimensions, which is imperative in developing a sound strategy to achieve target project outcomes (leidecker & bruno, 1984). we acknowledge the prevalence of many csf studies on bpm that are specific to the private sector context (buh, kovačič, & indihar štemberger, 2015; trkman, 2010). however, a synthesis of the csfs of public sector bpm initiatives has not been updated since mcadam and donaghy’s paper published in 1999. we argue that a re-assessment and synthesis of the literature is warranted due to major changes in socio-technological environments since 1999. this paper addresses this reassessment. the paper is also driven by the research question: “what are the critical success factors for bpm in public sector organisations?” in recognition that government agencies in developing countries may have different contexts and challenges to developed countries, we also aim to identify the differences (if any) between the bpm csfs for developing and developed countries. australasian journal of information systems syed, bandara, french & stewart 2018, vol 22, research article csf of bpm in public sector 3 this systematic literature synthesis provides a detailed overview on the current status of research on bpm csfs in the public sector, and sets a concrete foundation for future research in this domain. the study findings will assist public sector organisations in general to define their strategies for successful bpm transformations, reducing the risk leading to bpm project failure, and can also be used as a springboard for future research for academics and practitioners interested in e-government and bpm in public sector organisations. in order to better position this work, the paper proceeds first with a discussion on the potential similarities and differences between public sector versus private sector, and developing versus developed country contexts. next, the adopted research method is presented together with a simple profiling overview of the selected papers. then, we present the csfs identified from the synthesis of literature defining and describing these. finally, the paper concludes with an overall discussion, making a call for action with a series of prospective research questions to guide further research in this space. 2 background 2.1 public versus private sector bpm differences and similarities organisational structures, culture, and bureaucratic norms in the public sector are different to private sector organisations (mcadam & donaghy, 1999). for example, the ‘value creation’ concept differs between public and private sector organisations, value creation is the core goal of bpm (hill, sinur, flint, & melenovsky, 2006) and in the public sector excludes cost and profitability concerns (halachmi & bovaird, 1997). tregear and jenkins (2007) discussed the key differences between public and private sectors across diverse facets (see table 1, column 1) and explain how these factors are vital for process oriented management approaches in public sector organisations. table 1 provides a summarised list of differences discussed by tregear and jenkins (2007) and fettke, zwicker, and loos (2014). tregear and jenkins (2007, p. 8) stated that “we have proposed a gestalt of differences between public and private sector organizations. these differences must be considered in developing approaches to achieving process-based management in public sector organizations. it is problematic to simply transfer the private sector bpm experience to the establishment and ongoing commitment to gpm”. public sector private sector mission & vision focus is on public interest elements, societal objectives and outcomes that can be difficult to quantify. focus is on maximising shareholder return on investment, profitability and stakeholder value. aim achievement of societal objectives. difficult to measure and quantify. public task fulfilment. achievement of key performance indicators, production target, financial targets. profit maximisation. process customers complex and diverse set of citizen groups. heterogeneous. easily identifiable user/consumer groups. mostly heterogeneous. due process requirements fully applicable. partially applicable. australasian journal of information systems syed, bandara, french & stewart 2018, vol 22, research article csf of bpm in public sector 4 public sector private sector public accountability comes under the purview and scrutiny of the parliament, various authorities and formal bodies (public). risk-taking behaviour is influenced by public scrutiny. risk tolerance depends on election cycle. comes under the purview of the relevant authorities, legal bodies (non-public). risk-taking behaviour is influenced by shareholders, corporate boards. risk tolerance depends on financial cycle. economic structure monopolistic. performance benchmarking is limited. political legitimisation (fettke et al., 2014). variety of economic structures. performance benchmarking is inevitable. economical market organization (fettke et al., 2014). political sensitivity highly sensitive, decisions/job security is subject to public opinion. plans/operations are influenced by the election results. changes in political leadership determines the structure and nature of public administration. governments can be “lobbied”. partially sensitive, job security is associated with kpis. plans/operations are influenced by the ceo/board of directors. organisational structure is impacted by the corporate strategy. organisations play the role of a lobby. ecosystems whole of government/egovernment. integrated agencies impacted by jurisdictional challenges and public scrutiny. large scale onestop operations. multi-level supply chains. little impact from jurisdictional issues. supported by contacts. budget cycle complexities governments are the largest spending entity in a country. budget process is highly complex, involves multiple agencies and activities governed by the law. budgets comes under heavy public scrutiny. budget process is relatively simple, involves functional managers. approved by the ceo/board of directors with little or no public scrutiny. information exchange citizens interact with government on multiple platforms and for a variety of citizens’ information requirements. information needs may involve multiple government agencies. provision of information is a prime responsibility. customers’ interaction with an organisation is limited to the nature of transaction. little or no focus on provision of information. regulatory society governments are the creators and enforcers of regulations. regulations influence the economic & social groups. organisations are the user & complainant of regulations. organisations’ operations are influenced by regulations, and processes are designed to comply with the regulations. machinery of government change administrative structure depends on the political manifesto of the political regime. impact of change is complex and larger due to multi-stakeholder, multi-agency inter-dependencies. administration and structural changes depend on the changes in the boards, and competitive environment. australasian journal of information systems syed, bandara, french & stewart 2018, vol 22, research article csf of bpm in public sector 5 public sector private sector culture driven by rules & regulations, hierarchical decision making processes, inflexible. influenced by political factors. little attention to innovation, highly risk-averse, nonperformance oriented, highly resistant to change. based on market factors. influenced by customers and social changes. rewards and performance oriented, flexible and less resistant to change. table 1: differences between public and private sector (constructed based on tregear and jenkins, 2007 & fettke et al., 2014) 2.2 developed versus developing country context differences and similarities chen, chen, huang, and ching (2006) stated that national e-government infrastructure, culture, and society factors are different between developed and developing countries and strongly argued that developing countries should design their own e-government strategies based on their unique factors instead of adopting the strategies of the developed countries. apart from restrictions on the technological infrastructure and financial capabilities needed for e-government, developing countries are constrained with competencies and know-how to develop robust strategies for developing and supporting government improvement initiatives (chen et al., 2006). bin taher, krotov, and silva (2015) argued that the models and experience of developed countries cannot be directly applied in developing countries due to various socio-economic and cultural differences. the key differences between developed and developing countries as summarised by chen et al. (2006, p. 27) are presented in table 2. these are broad, generic differences and the degree of the existence of these characteristics can vary from country to country within both the developed or developing groups. nevertheless, it does give a broad and summary synopsis on how the developing and developed countries may differ. developed countries developing countries history and culture • government and economy developed early, immediately after independence. • economy growing at a constant rate, productivity increasing, high standard of living. • relatively long history of democracy and more transparent government policies and rules. • government usually not specifically defined. • economy not growing or increasing in productivity; low standard of living. • relatively short history of democracy and less transparent government policies and rules. technical staff • has a current staff, needs to increase technical abilities and hire younger professionals. • has outsourcing abilities and financial resources to outsource; and staff are able to • does not have staff with dedicated technical skills, or has very limited inhouse staff. • does not have local outsourcing abilities and rarely has the financial ability to outsource; current staff may be unable to define specific requirements. australasian journal of information systems syed, bandara, french & stewart 2018, vol 22, research article csf of bpm in public sector 6 developed countries developing countries define requirements for development. infrastructure • good current infrastructure. • high internet access for employees and citizens. • current infrastructure is weak. • low internet access for employees and citizens. citizens • high internet access and computer literacy; still has digital divide and privacy issues. • relatively more experienced in democratic systems and more actively participate in governmental policy-making processes. • low internet access and citizens are reluctant to trust online services; few citizens know how to operate computers. • relatively less experienced in democratic systems and less actively participate in governmental policy-making processes. government officers • decent computer literacy and dedication of resources; many do not place e-government at a high priority. • low computer literature and dedication of resources; many do not place egovernment at a high priority due to lack of knowledge on the issue. table 2: key differences between developed and developing countries (extracted from chen et al. (2006, p. 27) ahmed and gregor (2007) argued the relevance of most information system theories to developing and less-developed country contexts but noted that it was important to identify when these theories were developed and that these theories were mainly tested and often only tested in the developed country context. similarly, chen et al. (2006), by referring to the key differences provided in table 2, argued that e-government strategies that emerged within the context of developed countries cannot be directly applied to developing countries. zarei, ghapanchi, and sattary (2008) instead called for new models, or the re-structuring of existing e-government models designed for developed countries, to suit the context of the developing countries due to differences in the technical, cultural, social, political and economic infrastructures. bpm efforts in developing countries often differ widely for various reasons. first, in developing countries the limited access to and availability of bpm expertise restrict the ability to enjoy full benefits of process efficiencies (bandara, syed, kapurubandra, & rupasinghe, 2012; huang & palvia, 2001). second, most developing countries face different levels of a digital divide and information poverty due to considerably underdeveloped technology infrastructure and technical know-how (chen et al., 2006; heeks, 2008b; united nations, 2016; walsham, robey, & sahay, 2007; walsham & sahay, 2006; world bank, 2014). third, and perhaps most important, with the exception of a few, most developing countries face volatile political, social, and economic macro-environments that may adversely affect the implementation and sustainability of process improvement initiatives (walsham & sahay, 2006; weerakkody, el-haddadeh, & al-shafi, 2011). australasian journal of information systems syed, bandara, french & stewart 2018, vol 22, research article csf of bpm in public sector 7 therefore, it can be deduced that bpm csfs in developing countries may have some unique contextual differences to those seen in developed contexts. the next section discusses the systematic literature review methodology to identify contextual differences in bpm csfs. 3 approach and methodology a number of researchers offer advice on literature reviews. bandara, furtmuller, gorbacheva, miskon, and beekhuyzen (2015) provide a detailed synthesis and harmonisation of existing literature review guidelines, and describe an end-to-end approach with tool support to assist with the rigour, and transparency of literature reviews. webster and watson (2002, p. xix) refer to a literature review as an important component of all scholarly efforts and explained how a review should "identify critical knowledge gaps" while alvesson and sandberg (2013) describe the different ways gap-spotting is done. though they critique gap-spotting as a sole approach to derive research problems, they conclude that gap-spotting is the most popular way of identifying and justifying research problems at present. müller-bloch and kranz (2015) provide guidelines to identify research gaps in literature reviews. this paper uses the bandara et al., (2015) guidelines to conduct a review, and the alvesson and sandberg’s (2013) gapspotting classification together with the muller-bloch and kranz (2015) guidelines for identifying future areas of research. we sought to build upon the existing work of mcadam and donaghy (1999) on public sector bpm csfs as the initial framework for analysis. the scope of our literature search extended from post mcadam and donaghy (1999) till december 2014. figure 1 illustrates the applied phases of the systematic literature review, and these phases are described further below. australasian journal of information systems syed, bandara, french & stewart 2018, vol 22, research article csf of bpm in public sector 8 figure 1. data collection and analysis approach (adopted from bandara et al., 2015) 3.1 phase 1: searching and extracting the relevant pool of papers the topic of this study falls broadly into the disciplines of management and information systems (is). therefore, both these domains were selected as the primary fields for literature search. the selection criteria used to ensure the relevance of articles to the research goals and scope is illustrated in table 3. rule # criterion description 1 the articles should be related to the public sector context. 2 the articles should be about bpm or other very closely related topics pertaining to process-centric improvements (such as business process re-engineering, business process improvement, lean six sigma etc.) in public sector organisations. 3 the articles related to the external, citizen perspective (i.e. adoption of e-government services, digital divide, policy frameworks etc.) were not considered relevant to the scope. 4 the articles focusing on technology frameworks/architectures, data standards, software development were not considered relevant to the scope as these areas represent specific technical aspects of design and development of e-government software applications. 5 only peer-reviewed full articles that are in english were taken into consideration. table 3: criteria used for literature search australasian journal of information systems syed, bandara, french & stewart 2018, vol 22, research article csf of bpm in public sector 9 the search for literature was performed in 2 iterations. in the first iteration, a preliminary search was conducted via google scholar using search strings [“business process management” and “critical success factors” and “public sector” not erp not software]. the boolean term ‘not’ was added in adherence to rule 4 above after early search results needed to be constrained to enhance relevance derived from keywords listed under ‘1st iteration’ in table 4. 1st iteration 2nd iteration keywords keywords + synonyms & other related terms public sector government state business process management e-government e-transformation organisational transformation business process reengineering business process change business process improvement change success factors critical success factors key success factors acceptance adoption table 4: keywords and synonyms used for literature search in the second iteration, literature was initially searched using the ‘basket of eight journals in is’ (see, https://aisnet.org/?seniorscholarbasket). however, our search found no articles related to the public sector. as a result, the search was widened through the ebsco, emerald insight, abi/inform, ieeexplore, acm, science direct, and proquest databases. the articles were extracted in endnote and checked for duplication. forty-three (43) articles that fitted the selection criteria were selected for analysis. finally, scopus and web of science (wos) were utilised for forward and backward searching. through this mechanism, another 5 new articles were included, resulting in a total of 48 relevant articles. 3.2 phase 2: preparation for the analysis next, all 48 articles were fully read to analyse and confirm the content relevancy and to further refine the dataset. it was found that the content of a few articles did not reflect their abstracts, titles and keywords. as an outcome, only thirty-one (31) articles qualified for the detailed analysis. the selected papers were first checked for text-recognition and converted using adobe ocr function to be readable in nvivo, then exported to nvivo (from endnote) following the guidelines of bandara et al. (2015). all papers were stored as documents with attributes (such as case study, literature review, survey, and publication outlet) maintained. papers were imported as ‘author-year’ format to facilitate the traceability of the papers in support of the write-up phase (bandara et al., 2015). a coding rulebook (following decuir-gunby, marshall, & mcculloch, 2011) was developed and maintained to guide the coding process. 3.3 phase 3 & 4: the data analysis procedures and literature profiling outcomes the literature analysis took place in multiple stages. first the literature was profiled (see results below) to obtain an overview on the current status of the existing literature. a quasiaustralasian journal of information systems syed, bandara, french & stewart 2018, vol 22, research article csf of bpm in public sector 10 deductive qualitative data analysis approach following bandara et al. (2015) was applied for the detailed analysis. mcadam and donaghy (1999)’s results were used as an a priori coding framework while allowing new themes to emerge from the data. the analysis of gaps and opportunities for future research was conducted as part of the overall literature analysis, following insights from alvesson and sandberg (2013) (to identify different types of gaps and opportunities) and the guidelines of muller-bloch and kranz (2015). 3.4 profile of the reviewed articles profiling of literature in a review-based analysis can provide useful insights about the overall status of research, assist with quality assurance (in support of selection bias) and provide input to future research agendas (gaffar, deshpande, bandara, & mathiesen, 2015). an analysis of the key outlets and geographical distribution of the studies are presented here for this purpose; the business process management journal (7 articles), the international journal of electronic government research (5 articles) and the journal of enterprise information management (3 articles) were three key publication outlets. the rest of the articles were distributed amongst other journals and sources. figure 2 illustrates the geographical spread of research on bpm in the public sector. the blue dots represent the studies conducted in developed countries, whereas the red dots represent developing countries. among the selected thirty-one (31) articles, 8 articles provided the developing country’s perspective. figure 2: articles by geography 4 study findings: the critical success factors of bpm the above approach resulted in fourteen (14) csfs, all are first clearly defined and then explained in further detail below. 4.1 identifying and defining the critical success factors the definitions for each of the fourteen (14) factors (adopted from the literature) as used in this study are presented in table 5 in the order of most discussed to least discussed. when deriving these definitions, any direct or indirect (implied) mentioning of each factor, together with the definitions and any additional insights the papers provided were coded, and checked by a second coder. furthermore, additional aspects that contributed towards a factor were also captured to enable a better understanding on what each factor was formed on. australasian journal of information systems syed, bandara, french & stewart 2018, vol 22, research article csf of bpm in public sector 11 csf example definitions gathered from the literature adopted definition to p m an ag em en t s up po rt -c om m it m en t a nd u nd er st an di ng o f b pm “willingness of top management to accept and implement the re-engineering team's recommendations'' (mcadam and donaghy, 1999, p.44); a proactive push to leverage potential opportunities enabled by change agents, ensured the funding to support the project, driven by local management with a strong business case; the delineation of a specific plan of action and then motivation of the entire organisation”. (kennedy et al, 201); “the existence of a single person within the organisation who is committed to introducing the innovative it initiative to the organisation”(kamal, 2006, p.213); “ clear leadership is essential to make this effort work” (al-kibsi, et al. 2001, p.69); “to be effective in implementing e-government projects, the cio should occupy a position in a senior central agency of government with an enterprise-wide view of government operations or budget.” (marilu, 2012, p.4). senior management’s active involvement in the project, with provisions of resources and funding, motivation, acceptance of risks, perseverance, and strong leadership qualities. c om m un ic at io n “to reduce the resistance to change it is necessary to develop internal and external communication strategies of the bpm initiative. these strategies shall disseminate information about the bpm key concepts, while communicating the results of the initiative” (alves, et al., 2014, p.285); “all stakeholders – users, suppliers, delivery partners elsewhere in the public, private and voluntary sector, politicians, the media etc. have a clear understanding of the programme and how they can engage with it.” (borras, 2012, p.46); “communicate with stakeholders about the system: sell the true benefits and address the true negative aspects” (heeks, 2003, p.13); “consolidation and communication of it-governance-related policies and guidelines. this can enhance the control, changes and enforcement of it-governance-related policies and guidelines for performing it-enabled functions.” (nfuka & rusu, 2011, p.1436). organisational strategies, channels, and methods to enable on-time, clear, and accurate exchange of relevant information amongst the stakeholders. pr ep ar ed ne ss fo r o rg an is at io na l c ha ng e “how a manager or leader manages the potential impact of change to make people accept it in order to implement change” (hartini et al., 2007, p.458); “breaking the organisational status quo and introducing new practices, new values and new structures” (mcadam & donaghy, 1999, p.46); “the influence of the social environment on the employees’ perception of the transformation” (meier, ben, & schuppan, 2013, p.320). monitoring, measurement, and mitigation strategies to effectively prepare and overcome potential impact of changes in internal and external environments on people and processes in an organisation. australasian journal of information systems syed, bandara, french & stewart 2018, vol 22, research article csf of bpm in public sector 12 csf example definitions gathered from the literature adopted definition en li st in g c us to m er a nd st ak eh ol de r s up po rt in vo lv em en t “customer focus is now emphasised more within the public sector, and customer friendliness and simplification of procedures is the imperative of the administration, in keeping with the stated aim of the organisation.” (kennedy, et al., 2012, p.19); “need for a greater awareness of the customers of public services” (mcadam & donaghy, 1999, p.47); “harnessing the full power of e-government requires reorganizing departmental processes around the needs of citizens and businesses” (al-kibsi, et al., 2001, p.69). a systematic approach incorporating active stakeholders and customer participation to continuously monitor and identify system and service needs for process improvement. c ho os in g th e bp m te am a core bpr group together with cross-functional specialist groups; experienced in variety of techniques. (al-mashari & zairi, 1999; mcadam & donaghy, 1999); “a cross-functional steering committee was created and the team teased out at a very early stage the essential enablers to the successful introduction of new technologies including issues related to people and process ” (kennedy, et al., 2012, p.16). a coherent cross-functional unit skilled with relevant technical and management techniques to lead the bpm implementation in an organisation. a ll ev ia ti on o f d ow ns iz in g fe ar s “maintenance of job security” (mcadam & donaghy, 1999, p.42); “an attempt to maintain the status quo, is commonly understood to be almost universal in change processes” (marilu, 2012, p.10); “bpm pilot projects shall increase managers fearing power loss. it means that managers may understand that these projects are affecting their own areas and threatening their control” (alves, et al., 2014, p.282) employees resistance to change from the effects of actual or expected outcomes related with bpm and it implementation. em po w er m en t involvement of staff, outside the reengineering team in the bpr exercise, incorporating their views and opinions (mcadam & donaghy, 1999); “organisational structure should enable bpr in terms of its encouraging creativity and innovativeness in the organisation, therefore the need for less bureaucracy, and more participation and empowerment in the organisation” (hartini, et al., 2007, p.459); “senior management interdepartmental coordinating committees with a mandate and reporting accountability to the agency head are necessary” (marilu, 2012, p.4). a managerial process to ensure provision of required delegation of authority, responsibility and accountability to the core bpm team. ic t in fr as tr uc tu re “the systems needed for effective communication, and whether organisation's personnel perceive information sharing and information flow using those systems as being difficult to understand and use” (kamal, 2006, p.210). physical hardware, system architectures, software systems, technology policies and standards, and management processes required for seamless collection and dissemination of information. australasian journal of information systems syed, bandara, french & stewart 2018, vol 22, research article csf of bpm in public sector 13 csf example definitions gathered from the literature adopted definition c ul tu re “a well-established culture of teamwork would make it easier for the organisation to achieve its goals” (hartini, arthur, & zairi, 2007, p.457); “revision of reward systems, communication, empowerment, people involvement, training and education, creating a culture for change, and stimulating receptivity of the organisation to change are the most important factors related to change management and culture” (almashari and zairi, 1999, p.88). organisational core value system, shared beliefs and norms that differentiate an organisation in an industry. pr oj ec t m an ag em en t a nd p ro je ct m an ag em en t s ki ll s “adequate plan execution as well as a flexible and adaptive methodology represent further important success factors as pertaining to bpr project management” (jurisch, 2012, p.2605); “the process by which projects are defined, planned, monitored and controlled to achieve the project objectives of timely delivery, adherence to budget, conformance to specifications and stakeholder satisfaction. it involves the management of an endeavor, with defined beginning and end periods, undertaken with tools and techniques, to create a unique product or service.” (kwamena, 2012, p.60); “in terms of project management, prioritisation of deliverables will ensure that the most strategically significant services are managed and delivered appropriately” (weerakkody, el-haddadeh, & al-shafi, 2011, p.176) a robust project management methodology and development of leadership, interpersonal, planning, financial, communication, negotiation and change management skills for staff involved in bpm projects. ic t a w ar en es s “appropriate training and development of new skills among staff ” (jurisch, et al., 2012, p.2606); proper training to develop adequate skills for employees to enable them to perform the assigned tasks and responsibilities (hartini, et al., 2007; heeks 2003). training and development initiatives to develop technical competence of public sector employees in order to capitalise on potential use of information communication technology systems. it b pm g ov er na nc e “a reference framework to guide organizational units to ensure responsibility and accountability” (alves, et al., 2014, p.274). systematic planning that includes the definition of specific roles and responsibilities, the description of policies and methodologies, and the selection of process-oriented software tools (valenca, alves, santana, de oliveira, & santos, 2013, p. 2) management practices and policies, span of management, and standard operating procedures to guide ict operations and bpm projects in a government organisation. australasian journal of information systems syed, bandara, french & stewart 2018, vol 22, research article csf of bpm in public sector 14 csf example definitions gathered from the literature adopted definition st ra te gi c c la ri ty & a lig nm en t “all-of-government view, establishing a clear vision, a strong business case, and a result-focused strategy” (borras, 2012, p.45); “interrelation between compatibility of the it system with the organisational need; organisational compatibility, which that can be thought of as the organisational fit of the system required for effective information sharing among different departments.” (kamal, 2011, p.211); “for egovernment initiatives to accomplish the stated goals, their strategic goals should be aligned with the government agency’s mission, strategic goals and business processes” (kwamena, 2012, p.46); “egovernment initiatives will be most effective and the impact of ict is most powerful when e-government is clearly defined as part of a broader public sector reform initiative” (marilu, 2012, p.4); “alignment of it with business strategy and enabling it structures” (nfuka & rusu, 2011, p.1436). ict strategic vision, mission, goals, and operational plans of a public sector organisation that are aligned with the political and social aims established by the government leadership. ex te rn al en vi ro n m en ta l “external forces or influences refers to the impact that external associations exercise on the organisation ” (kamal, 2006, p.214) global and local political, social, and economic macro factors that influence a public sector transformation endeavour. table 5: summary of critical success factor definitions an nvivo matrix query (see bazeley, 2007 for further details ) was used to provide the summary synopsis (see table 6 below), which illustrates the extent to which each critical success factor was derived and supported from the literature review (they are presented in order of most to least supported within the two categories of a priori and new factors). the numbers in each cell illustrate the frequency (number of separate coding references) a factor was discussed under each evidence-category which were definitions (a), direct supporting observations (b), indirect support observations (c) and other additional insights (d). note columns in the table denoted by the classification ‘a’, ‘b’, ‘c’ and ‘d’ for each paper which is presented in rows, in alphabetical order by first author. australasian journal of information systems syed, bandara, french & stewart 2018, vol 22, research article csf of bpm in public sector 15 table 6: cross tabulation matrix australasian journal of information systems syed, bandara, french & stewart 2018, vol 22, research article csf of bpm in public sector 16 5 a detailed review of the critical success factors the following section provides a discussion around each factor (in order of most prevalent to least prevalent), describing how the literature discussed the factor. sub-factors pertaining to each factor were extracted from the literature analysis in order to provide a deeper level of understanding of each factor. prospective research questions to shape future research were derived based on gaps and opportunities observed from this analysis. 5.1 top management support and understanding of bpm top management support was one of the main factors mentioned for a successful bpm implementation. key attributes of top management support entail the provisioning of project funding, motivating staff and being change agents (kennedy, joseph, & carol, 2012), and strategic alignment (alves et al., 2014). a different perspective was presented by doebeli, fisher, gapp, and sanzogni (2011) that includes corporate governance, leadership, bureaucracy and a formal structure within an organisation as the key aspects of top management support. alghamdi, goodwin, and rampersad (2014) also reported that corporate governance and the management’s ability to involve the right people were two key factors behind the success of public sector transformation projects. ravesteyn and batenburg (2010) gave strong emphasis on a top-down approach for bpm projects in an organisation. leadership is another key factor discussed within the context of top management support. alkibsi, de boer, mourshed, and rea (2001) identified a lack of leadership as a restricting factor for online integration and the need for it in redesigning processes. kamal (2006) further stressed the need for a committed champion to introduce novel it endeavours. from a developing country’s perspective, alghamdi et al. (2014) emphasised the creation of leadership steering committees while nfuka and rusu (2011) discussed the importance of articulating a vision for the it role by the chief information officer (cio) or equivalent in public sector projects. streib and navarro (2008) recommended the need for training and development in the public sector to develop much needed knowledge, skills, and leadership abilities. along similar lines, axelsson, melin, and söderström (2011) linked leadership skills with successful it-enabled transformation and risk management. furthermore, the crucial role of leadership in e-government projects to develop team cohesion was reported by marilu (2012) as well as. these sub-factors that seem to describe what was meant by or expected of top management support are documented in table 7. australasian journal of information systems syed, bandara, french & stewart 2018, vol 22, research article csf of bpm in public sector 17 (*) represents developing countries in no va ti ve -n es s m ob il is e/ a llo ca ti on of r es ou rc es c om m itm en t t o fu nd in g r is kta ki ng t en de nc y k no w le dg e of po te nt ia l u se o f i t le ad er sh ip a ct iv e in vo lv em en t in p la nn in g r ew ar ds a nd m ot iv at io n lo ng te rm c om m itm en t g ov er na nc e *alghamdi et al. (2014) √ √ *alves et al. (2014) √ √ √ *hartini, francis, and zairi (2007) √ √ *hisham, christopher, and hatem (2012) √ *nfuka and rusu (2011) √ √ *weerakkody, el-haddadeh, and al-shafi (2011) √ √ √ al-kibsi et al. (2001) √ axelsson et al. (2011) √ √ √ borman and janssen (2012) √ doebeli et al. (2011) √ jurisch, ikas, palka, wolf, and krcmar (2012) √ √ √ kamal (2006) √ √ √ √ √ kennedy et al. (2012) √ √ √ kwamena (2012) √ marilu (2012) √ tan, cater-steel, and toleman (2009) √ √ table 7: summary of attributes underpinning top management support when comparing the developed and developing country contexts (greyed rows used in table 7) it was revealed that the ‘innovativeness’, ‘knowledge of potential use of it’, and ‘long term commitment’ factors were not discussed in the literature related to developing countries. in contrast, ‘rewards and motivation’ was a distinctive factor mentioned in one study on developing countries. research into top management support for bpm in the public sector is limited. although leadership is identified as a key factor for bpm project success under top management support, the studies on the style, nature and attributes of leadership that contribute to bpm success are scarce (also stated in syed, bandara, french, & stewart, 2016). thus, we propose further research on the following: ‘how can top management support within public sector bpm initiatives be conceptualised?’; ‘how does leadership contribute to the success and sustainability of bpm in the public sector?’ furthermore, research on: ‘how to build leadership capabilities on process-centric innovation in public sector?’, and ‘how to obtain sustained long term top management commitment for bpm in public sector?’ is warranted. 5.2 communication communication of bpm strategies is vital to enable on-time, clear, and accurate exchanges of relevant information among stakeholders (chong & rosemann, 2010). alves et al. (2014, p. 283) discussed the need for communication as a strategy to overcome resistance to change, introduce the bpm approach, and to communicate the outcomes and achievements of the bpm project. the impact and effectiveness of communication on bpm project success has been widely acknowledged (alves et al., 2014; borman & janssen, 2012; kwamena, 2012; marilu, australasian journal of information systems syed, bandara, french & stewart 2018, vol 22, research article csf of bpm in public sector 18 2012; nfuka & rusu, 2011; ravesteyn & batenburg, 2010). studies by heeks (2003) and borras (2012) emphasised the provision of clear bpm project objectives for effective stakeholder involvement. hartini, francis, and zairi (2007) posited cross-functional integration and teamwork as key determinants of communication. meier, ben, and schuppan (2013) advocated the quality and sufficiency of information for controlling and enforcing itgovernance policies. it is noted that the communication strategy for bpm can be effectively used for team collaboration and coordination as well as for stakeholders engaged with the project. communication for collaboration and coordination is also referred to as an important technique to handle resistance to change. the sub-factors that seem to further describe communication (as a csf of bpm) are documented in table 8. (*) represents developing countries te am c ol la bo ra tio n te am c oo rd in at io n q ua lit y & s uf fi ci en cy of in fo rm at io n c om m un ic at io n fo r st ak eh ol de r en ga ge m en t d is se m in at io n of b pm c on ce pt s an d r es ul ts *alves et al. (2014) √ *hartini et al. (2007) √ √ *nfuka and rusu (2011) √ al-mashari and zairi (1999) √ borman and janssen (2012) √ √ √ borras (2012) √ heeks (2003) √ kwamena (2012) √ meier, ben, and schuppan (2013) √ table 8: summary of communication aspects in bpm the three articles covering developing countries did not mention ‘quality and sufficiency of information’, or ‘stakeholder engagement’ in the context of communication. the literature on developed countries emphasised the value of communication for better stakeholder engagement. these differences may be attributed to the different socio-cultural issues (chen et al., 2006) between the two groups. despite many discussions (30+ coding references, as shown in table 6) indicating the importance of communication for bpm success, important details such as ‘what are the essential elements of a public sector bpm communication plan’ are scarce. the importance of communication for effective stakeholder engagement in public sector bpm efforts, though mentioned widely, could be further studied – especially within the developing country context, where “bpm stakeholders”, including the non-traditional stakeholder groups (such as the various social networks that have the capacity to make a difference), need to be better understood to design and test innovative engagement techniques. hence, research is warranted on ‘who are public sector bpm stakeholders and what are their potential impacts on bpm efforts?’ australasian journal of information systems syed, bandara, french & stewart 2018, vol 22, research article csf of bpm in public sector 19 5.3 preparedness for organisational change sound change management practices are important to effectively prepare and overcome the potential impact of changes (coming from internal and external environments) on people and processes. mcadam and donaghy (1999) recognised rigid structures in government organisations as the main hindrance for bpm related change. hartini et al. (2007) construes change management as a strategy to manage the potential impact and people acceptance to implement change. kennedy et al. (2012) linked the causes of change with resource challenges. alves et al. (2014) posited that bpm activities performed in parallel with routine work can lead to resistance in the public sector. kwamena (2012) argued that the ability to adopt rapid technological change is restricted by the manner in which public administrators function. meier et al. (2013) reported positive association between social influences and change where staff were amenable to change when the initiative was supported by managers and colleagues. a few studies discussed the use of rewards and benefits (heeks, 2003) and staff involvement to handle resistance to change in public organisations (alghamdi et al., 2014). training and development programs to build team skills were advocated by jurisch, ikas, palka, wolf, and krcmar (2012), whereas the development of internal and external communication strategies was emphasised by alves et al. (2014). table 9 depicts the change management sub-factors as discussed in the literature. (*) represents developing countries m an ag em en t o f po te nt ia l i m pa ct of c ha ng e st af f in vo lv em en t u se o f e xt er na l c on su lta nt s jo b r ot at io n /r ede pl oy m en t d ep ar tm en ta l in te gr at io n so ci al po w er d is tr ib ut io n u se o f r ew ar ds & be ne fi ts t ra in in g & d ev el op m en t c om m un ic at io n st ra te gi es *alghamdi et al. (2014) √ √ *alves et al. (2014) √ *hartini et al. (2007) √ √ *weerakkody et al. (2011) √ √ axelsson et al. (2011). √ heeks (2003) √ jurisch et al. (2012) √ √ kennedy, et al. (2012) √ √ √ mcadam and donaghy (1999) √ meier, et al. (2013) √ table 9: summary of factors defined under preparedness for organisational change when the developed and developing country contexts were compared, factors such as ‘staff involvement’, ‘use of external consultants’, ‘rewards & benefits’ and ‘development of communication strategies’ for effective change management were discussed across both contexts; whereas ‘job rotation/re-deployment’, ‘social influence’, and ‘training & development’, were not mentioned in the literature on developing countries. on the other hand, ‘management of potential impact’, ‘departmental integration’ and ‘power distribution’ factors were not mentioned in the literature related to developed countries. socio-cultural, socio-technical, political and organisational differences between developed and developing countries suggest bpm related change will be experienced differently. questions proposed here include: ‘what constitutes a well-designed organisational change strategy in a public sector bpm initiative?’, ‘how can social and cultural factors in public sector organisations influence a australasian journal of information systems syed, bandara, french & stewart 2018, vol 22, research article csf of bpm in public sector 20 bpm initiative, in particular for better stakeholder engagement?’ and ‘how can training & development be designed to support effective bpm change in the public sector?’ 5.4 enlisting customer and other external stakeholder support/involvement the term customer-focused is no longer alien to public sector organisations. due to major policy changes and transformation initiatives, government agencies are becoming more citizen-centric (al-kibsi et al., 2001; almarabeh & abuali, 2010; borras, 2012; heeks, 2008a). al-kibsi et al. (2001) and mcadam and donaghy (1999) suggest the reorganisation of processes around the needs of citizens to achieve the full potential of bpm and e-governance and al-mashari and zairi (1999) have offered a broader perspective with emphasis on customer research, competitive analysis and benchmarking as critical elements of successful bpm efforts. in comparison with the above, borman and janssen (2012) emphasised incorporating the functional requirements of the stakeholders when designing interoperable system functionality, while borras (2012) emphasised the need for user focus, citizen-centric delivery, citizen empowerment, and cross sector partnership for successful e-government. furthermore, the literature also emphasised on the alignment of stakeholder goals as a key element of success in e-government projects (kwamena, 2012; nfuka & rusu, 2011; ravesteyn & batenburg, 2010). table 10 provides an overview of attributes discussed in the literature. (*) represents developing countries c us to m er fo cu s pr oc es s de si gn st ak eh ol de rs ’ g oa l al ig nm en t c us to m er f ri en dl in es s c us to m er r es ea rc h be nc hm ar ki ng u se r f oc us c iti ze n em po w er m en t *kennedy et al. (2012) √ √ *nfuka and rusu (2011) √ al-kibsi et al. (2001) √ √ al-mashari and zairi (1999) √ √ borman and janssen (2012) √ √ borras (2012) √ √ √ gulledge and sommer (2002) √ kwamena (2012) √ √ mcadam and donaghy (1999) √ √ √ ravesteyn and batenburg (2010) √ table 10: summary of factors defined under stakeholder support/involvement the analysis identified a stronger emphasis on the use of ‘customer focused process design’ within literature on developed countries. ‘stakeholders goal alignment’, and ‘customer friendliness’ factors were discussed across both developing and developed contexts. however, aspects such as ‘customer research’, ‘benchmarking’, ‘user focus (understanding of crossfunctional users’ technical requirements)’, and ‘citizen empowerment’ were not discussed in the literature on developing countries. with citizen focussed government services being the key objective in many public sector bpm projects, the questions ‘how to effectively engage the key stakeholders in public sector bpm initiatives?’ and ‘what are the innovative stakeholder engagement models that enable bpm success in public sector contexts?’ need further investigation. australasian journal of information systems syed, bandara, french & stewart 2018, vol 22, research article csf of bpm in public sector 21 5.5 choosing the bpm team a sound bpm team is known to be a coherent cross-functional group, skilled with relevant technical and management techniques to lead the bpm implementation (al-mashari & zairi, 1999). the appointment of core cross-functional specialist groups in public sector bpm initiatives was supported in the literature (al-mashari & zairi, 1999; kennedy et al., 2012; mcadam & donaghy, 1999). al-mashari and zairi (1999) suggested the role of consultants to overcome skill shortages in government organisations. kwamena (2012) linked the success of e-government projects with effective management and team skills & competencies. to overcome resource and skill shortages in public sector bpm projects, alves et al. (2014) proposed the establishment of a bpm office, a dedicated organisational unit to deliver bpm services. studies by nfuka and rusu (2011) and borras (2012) stressed human resource planning, recruitment, development and retention of qualified technical staff. dexter (2010) emphasised the optimal combination of individuals based on skill, roles, type, and willingness to learn as key factors for teams. the lack of required is/it capabilities and skills, and partnerships with industry experts in e-government projects were mentioned as constraining factors (alghamdi et al., 2014; weerakkody, el-haddadeh, et al., 2011). table 11 presents a summary of sub-factors. (*) represents developing countries c ro ss fu nc tio na l sp ec ia li st g ro up s u se o f ex te rn al c on su lta nt s sp ec ia li se d sk il ls a nd ex pe ri en ce se tti ng u p a ‘b pm o ff ic e’ *alghamdi et al. (2014) √ *alves et al. (2014) √ √ √ *weerakkody et al. (2011) √ √ al-mashari and zairi (1999) √ √ √ borras (2012) √ √ dexter (2010) √ kennedy et al. (2012) √ mcadam and donaghy (1999) √ √ table 11: aspects to consider for bpm team selection ‘specialised skills and experience’ was emphasised in the literature in both developing and developed country contexts, however, with most details predominantly discussing ict and technical skills. on the other hand, the need for ‘setting up a bpm office’ (or having a dedicated entity for bpm within the organisation) was mentioned only in a single study (in a developing country context). ‘cross-functional specialist group’, and the ‘use of external consultants’ were mentioned in both contexts. although the ‘skill set’ and the call for a ‘right mix’ was repeatedly discussed by various authors, the definitions and description of these concepts still need further clarification. furthermore, skills were mostly attributed towards ict and technical skills development, whereas based on the above discussion, it is observed that a majority of literature lacked details on bpm specific skills and competencies. this leaves the question ‘what constitutes an effective bpm team in public sector contexts?’ unanswered. bpm is still in its introductory stages in developing countries and due to the lack of resources and skills development opportunities (bandara et al., 2012), there is a need for robust human resource planning, training and development strategies. considering the above discussion, we australasian journal of information systems syed, bandara, french & stewart 2018, vol 22, research article csf of bpm in public sector 22 deduce that bpm knowledge and skill development in developing countries require attention. thus, ‘how to develop bpm capabilities (in general) in developing countries?’ can be a focus for future research. 5.6 alleviation of downsizing fears (resistance to change) resistance to change can influence the actual or expected outcomes related to bpm. bpm brings radical change to organisational structures, which is often perceived as a threat (alves et al., 2014; axelsson et al., 2011; mcadam & donaghy, 1999). employees who fear the loss of authority and choice due to the introduction of technology and ict-enabled processes show higher resistance to change (meier et al., 2013; weerakkody, el-haddadeh, et al., 2011). job security (mcadam & donaghy, 1999) was a key factor of concern for operational staff. marilu (2012), on the other hand, attributed the fear of job loss to the frontline and middle management staff in the government sector. borman and janssen (2012), suggested the need for union and staff relationship management to handle changes to jobs and reskilling needs and hartini et al. (2007) stressed the use of intrinsic and extrinsic motivational strategies to overcome resistance to change and fear of uncertainty. al-kibsi et al. (2001) on the other hand recommended government employees’ involvement in the process to overcome resistance, while kamal (2006) recommended that resistance to change can be effectively addressed by introducing the role of influential and tech-savvy champions. table 12 presents the identified sub-factors. (*) represents developing countries pr of es si on al a ut on om y jo b se cu ri ty t hr ea ts to a ut ho ri ty a nd po w er c ha ng es in jo b r ol e an d r es ki lli ng n ee ds *alves et al. (2014) √ √ *hartini et al. (2007) √ *weerakkody et al. (2011) √ axelsson et al. (2011). √ √ borman and janssen (2012) √ marilu (2012) √ mcadam and donaghy (1999) √ √ √ meier et al. (2013) √ table 12: summary of attributes underpinning downsizing fears ‘professional autonomy’, ‘job security’, and ‘threats to authority and power’ were the common sub-factors mentioned for resistance. whereas ‘changes in job roles and reskilling needs’ was not mentioned in studies on developing countries. considering the above discussion, ‘what are the key contributing factors that lead to resistance to change for bpm in public sector organisations?’ is an important aspect that requires further investigation. furthermore, mcadam and donaghy (1999) identified a higher level of resistance to change within operational staff, whereas marilu (2012) relates it to front-line and middle management staff. this anomaly warrants further understanding of which level of staff in a government organisation may exhibit higher resistance to change; thus questions to further investigate can be; ‘does resistance to change for bpm differ between staff at different levels in public sector contexts?’ and ‘how to identify the most vulnerable staff groups within government departments about to deploy bpm, and anticipate reactions?’. australasian journal of information systems syed, bandara, french & stewart 2018, vol 22, research article csf of bpm in public sector 23 5.7 empowerment empowerment can address the provision of delegation of authority, responsibility, and accountability issues to the bpm team. al-mashari and zairi (1999), reported that empowerment helps create collaborative teamwork cultures and as a result, staff at all levels take greater responsibility and accountability. the absence of empowerment will have negative consequences on e-government initiatives (weerakkody, el-haddadeh, et al., 2011). bpm often requires changing the organisational structure, and attempts to implement bpm within hierarchical organisational structures (i.e. public sector) increase the likelihood of failure (gulledge & sommer, 2002). mcadam and donaghy (1999) defined empowerment as the direct involvement of staff at all levels and the incorporation of their opinions in the process mapping exercise. in comparison, doebeli et al. (2011) linked empowerment with the directing and controlling aspects of management. hartini et al. (2007) emphasised a robust organisational structure to overcome bureaucracies and improvement of staff participation in decision making. in contrast, marilu (2012) and alghamdi et al. (2014) explained empowerment as the management mandate to delegate, budget, resource, control, and for the division of authority. table 13 provides a summary of various sub-factors discussed in the context of empowerment in public sector bpm projects. (*) represents developing countries in vo lv em en t o f s ta ff d ir ec tin g & m on it or in g r ob us t o rg an is at io n st ru ct ur e c on tr ol ov er bu dg et s/ r es ou rc es d iv is io n of a ut ho ri ty *alghamdi et al. (2014) √ √ *hartini et al. (2007) √ √ doebeli, et al. (2011) √ gulledge and sommer (2002) √ marilu (2012) √ √ mcadam and donaghy (1999) √ √ table 13: summary of attributes contributing to empowerment except for the ‘directing & monitoring’ factor (which was mentioned in one paper from the developing country context), all the sub-factors were identified in both developing and developed contexts. although empowerment is recognised as a key component for bpm initiatives, there is limited discussion on the description of the dynamics of empowerment and the manner and processes in which empowerment can be established. as likert (1961) argued, empowerment should not be equated with staff participation and involvement without proper delegation of authority, accountability and responsibility. public sector organisations are driven by hierarchical, top down approaches (hutton, 1996; tregear & jenkins, 2007), and these characteristics are different to organisational structures that are naturally more empowering, therefore raising the following questions; ‘how can public sector organisations be receptive to employee empowered bottom-up bpm approaches?’, and ‘how to create a sense of empowerment amongst public sector employees to support sustained bpm success?’ as areas for further investigation. furthermore, organisational structures and management approaches in developing countries towards empowerment of staff through delegation of power and decentralised decision making can be quite different compared to developed countries (chen et al., 2006). a deeper understanding of these differences and their impact on australasian journal of information systems syed, bandara, french & stewart 2018, vol 22, research article csf of bpm in public sector 24 public sector bpm is required, hence ‘what organisational structures would best provide accountability, responsibility, and authority to enable public sector bpm success? 5.8 ict infrastructure a robust ict infrastructure is vital for collection and dissemination of information. while kamal (2006) states technology systems are needed for effective communication, borras (2012) described the interoperability (open standards and support), web-centric delivery (service oriented architecture), agility (re-usability, flexibility, adaptability) and shared services (common data sets, authentication) as the key components of ict infrastructure for successful transformational government projects. weerakkody, el-haddadeh, et al. (2011) have taken the perspective of both implementers and users of e-government and identified security, privacy, and it standards as the critical concerns in e-government projects in developing countries. nfuka and rusu (2011) suggested the need to consolidate, standardise, and manage it infrastructure for cost optimisation and effective information flow. al-mashari and zairi (1999) explained the adequacy, reassessment and composition of it infrastructure as vital factors in successful bpr implementation. alves et al. (2014) reported that organisations with successful bpm experience had good technical infrastructure (e.g. software tools, supporting communication facilities) and positive synergy among units participating in bpm initiatives. table 14 illustrates the sub-factors related to ict infrastructure. (*) represents developing countries c om pa tib le t ec hn ol og y sy st em s o pe n st an da rd s se rv ic e o ri en te d a rc hi te ct ur es a gi li ty sh ar ed se rv ic es st an da rd sy st em s so ft w ar e t oo ls *alves et al. (2014) √ *nfuka and rusu (2011) √ borras (2012) √ √ √ √ kamal (2006) √ table 14: attributes underpinning ict infrastructure as ward and peppard (2002) stated, due to continuous technology evolution, defining it infrastructure and its components have become increasingly difficult. from the limited studies here, prior research on bpm in the public sector also faced similar it infrastructure challenges. although difficult to comment conclusively on any differences between developed and developing countries with the limited related papers on this csf as shown in table 14, literature on developing countries refers to ict infrastructure as software tools and standard systems to improve interoperability to enable seamless integration and sharing of data. on the other hand, literature on developed countries reflects the attention on interoperability, shared services and integrated aspects of ict infrastructure. according to current trends, developing countries are starting to move away from standalone systems to build more integrated ict infrastructure to provide one-stop-shop service platforms (united nations, 2016). further research on innovative, emerging ict infrastructure models to address ‘how can governments in developing countries best design and benefit from the use of emerging new ict options such as; shared services, cloud computing, open source, etc.?’, can be proposed. australasian journal of information systems syed, bandara, french & stewart 2018, vol 22, research article csf of bpm in public sector 25 5.9 project management and project management skills project management is an integral part of bpm projects in the public sector. hartini et al. (2007) observed that the practice of training staff in project management by deploying external consultants resulted in effective bpm implementations. marvine, martin, doaa, and salah (2012) compared the differences between public sector and private sector programmes and advocated the essential need to contextualise project management skills, knowledge, tools, and techniques to individual local contexts in order to meet project objectives. al-mashari and zairi (1999) confirmed planning, project management techniques, risk assessment, sufficiency of budgets, and human resource management as key factors in delivering a successful bpm project. weerakkody, el-haddadeh, et al. (2011) discuss prioritisation strategies demonstrating the value of narrowing the focus to first delivering the most strategically significant services. the consensus over the importance of project management is evident in the literature. most definitions and findings though narrowly describing the stages and techniques of project management, have acknowledged the need for project management as a vital feature for bpm projects. table 15 illustrates the related sub-factors as discussed in the literature. (*) represents developing countries pr oj ec t m gm t. t ra in in g pl an ni ng & im pl em en ta tio n w el l d ef in ed pr oj ec t p la n fl ex ib le a nd a da pt iv e pr oj ec t m et ho do lo gy r is k a ss es sm en t h um an r es ou rc es m gm t. pr io ri tis at io n of d el iv er ab le s bu dg et s *hartini et al. (2007) √ *marvine, et al. (2012) √ √ *weerakkody, et al. (2011) √ al-mashari and zairi (1999) √ √ √ √ √ jurisch et al. (2012) √ √ √ √ √ √ kwamena (2012) √ √ table 15: summary of factors defined under project management and project management skills the analysis indicated a considerable gap in project management and relevant skills in the developing countries. an observed key gap in the body of literature was the limited detail on how project management is embedded within standard bpm methodologies. the current literature discusses these as two discrete approaches, which should in reality be tightly integrated and complimenting each other. therefore, research on ‘how to integrate project management practices into bpm methodology?’ is recommended to strengthen the integration between project management and bpm methodology. some papers vividly discuss the need for (and the gaps in) project management training. one reason for high failure rates in public sector bpm is attributed to the project management practices, pointing to the need to investigate ‘how to ensure effective project management training and training transfer in public sector?’ and ‘how to build project management awareness in public sector bpm contexts?’ 5.10 ict – awareness ict awareness may be used to assess and develop the required technical and process competencies. a large body of literature emphasised the need for ict awareness and effective training (al-mashari & zairi, 1999; alghamdi et al., 2014; alves et al., 2014; borman & janssen, 2012; hartini et al., 2007; heeks, 2003). hisham et al. (2012) associated user satisfaction with the success of e-government and recommended the use of ict training for public sector australasian journal of information systems syed, bandara, french & stewart 2018, vol 22, research article csf of bpm in public sector 26 officers to achieve higher user satisfaction. alghamdi et al., (2014) emphasised the use of regulations and the need for technical support to improve ict awareness. on a different note, kamal (2006) linked technical competence and expertise of the it manager to gain top management’s support and trust. similar emphasis on technical competence for successful project execution in e-government projects was suggested by kwamena (2012). reskilling and training and development were the common factors discussed in the literature in both the developed and developing contexts. literature on developed countries further discussed ‘interoperability’, ‘it manager’s expertise’ and ‘user-friendliness’ aspects, whereas ‘technical support’ and use of ‘regulations’ is given much attention by the literature on developing countries. sub-factors related to ict awareness are presented in table 16. (*) represents developing countries r es ki lli ng /t ra in in g & d ev el op m en t u se o f r eg ul at io ns u se rfr ie nd lin es s of sy st em in te rop er ab il ity it m an ag er ’s ex pe rt is e t ec hn ic al su pp or t *alghamdi et al. (2014) √ √ √ *alves, et al. (2014) √ *hartini et al. (2007) √ *hisham et al. (2012) √ borman & janssen (2012) √ borras (2012) √ heeks (2003) √ jurisch et al. (2012) √ kamal (2006) √ √ table 16: summary of factors defined under ict awareness the analysis illustrated the lack of awareness on bpm capabilities, confirming the findings of bandara et al. (2012), who highlights the need to develop much needed bpm skills and competencies in public sector organisations. the bpm training needs analysis, design and conduct options (the methods, types, and durations of these training and development programmes), and training evaluations seem to be an obvious gap requiring further investigation. hence, an investigation on; ‘how to ensure effective bpm training and training transfer in public sector organisations?’ is warranted. bpm and ict have a recursive relationship (attaran, 2004; davenport & short, 1990), hence, development of organisational bpm skills are as important as ict skills; however they were rarely discussed in the current literature. we propose that an investigation on the following; ‘how does training in bpm skills improve bpm success in public sector organisations?’ and ‘how can public sector organisations proactively use ict for bpm efforts?’ can help progress bpm in the public sector. 5.11 culture as the culture factor represents the organisational core value system, shared beliefs and norms, it appears to interact and shape other organisational factors. hartini et al. (2007), emphasised the development of a strong culture in the organisation by the innovative incorporation of core values. maintaining a process-oriented culture is easier when systems are aligned with the processes (gulledge & sommer, 2002). tan, cater-steel, and toleman (2009) emphasised that public sector organisations should shift their culture from the technological to a service-focused culture. alves et al. (2014) construed process orientation as a key aspect of culture. weerakkody et al. (2011) recognised the importance of culture and its australasian journal of information systems syed, bandara, french & stewart 2018, vol 22, research article csf of bpm in public sector 27 influences on the implementation and dissemination of e-government. table 17 illustrates the sub-factors discussed in the literature. (*) represents developing countries sh ar ed v al ue s an d be lie fs c or e va lu es c ul tu re o f ch an ge pr oc es s o ri en ta ti on pe op le in vo lv em en t t ra in in g & d ev el op m en t g ro up pr og ra m m in g in te rd ep ar tm en ta l po li tic s se rv ic e fo cu s *alves et al. (2014) √ *hartini et al. (2007) √ √ √ *valenca, et.al (2013) √ *weerakkody et al. (2011) √ al-mashari and zairi (1999) √ √ √ √ al-kibsi (2001) √ gulledge and sommer (2002) √ wui-gee tan et al (2009) √ table 17: summary of factors defined under culture the identified sub-elements of organisational culture were parsimoniously touched upon by studies in both developed and developing contexts. most literature related culture with the resistance to change. we believe that the public sector organisations’ culture itself is a standalone research topic because of the unique socio-economic, ethnic, and political dynamics that exist both in developed as well as developing countries (chen et al., 2006; tregear & jenkins, 2007). therefore, the question ‘how to develop mechanisms to implement robust and innovative process-centric cultures in the public sector?’ can be posed. just as hofstede (1984) established distinct differences between western and eastern cultures that affect organisational planning and management, it can also be expected that organisational cultures in developing countries can have different complex layers (chen et al., 2006; weerakkody, el-haddadeh, et al., 2011). hence, it is vital to be able to assess ‘how organisational culture and its characteristics in developing countries uniquely affect bpm initiatives in public sector organisations?’ 5.12 it-bpm governance process governance provides a reference framework to guide organizational units to ensure responsibility and accountability to lead the bpm initiative (alves et al., 2014). while incorporating the general features of governance, the attention of bpm governance is on processes (doebeli et al., 2011). few papers discussed about the it governance in public sector bpm initiatives where it played a central role as the process enabler. we combined the two terms as it-bpm governance csf. doebeli et al. (2011, p. 199) stated that “the integration of bpm roles and responsibilities into the governance management system of the organisation ensures sustainability of the practice and business improvement efforts”. nfuka and rusu (2011) concur the importance of it governance and its awareness, training, and competency of professionals as an influencing factor for it governance to a sustainable, standardised and cost-effective it environment. furthermore, alves et al. (2014) suggested that bpm maturity in an organisation will improve by establishing a bpm office and reference framework. borman & janssen (2012) recommended the establishment of guidelines and processes, as well as unified organisational structure in which all stakeholder organisations should be grouped to reduce complexity leading to australasian journal of information systems syed, bandara, french & stewart 2018, vol 22, research article csf of bpm in public sector 28 effective governance. a summary of the sub-factors it-bpm governance is presented in table 18. (*) represents developing countries se t o f g ui de li ne s an d pr oc es se s bp m a w ar en es s bp m r ol es & r es po ns ib il it ie s t ra in in g c om pe te nt pr of es si on al s bp m o ff ic e r ef er en ce fr am ew or k u ni fi ed o rg an is at io na l st ru ct ur e *nfuka and rusu (2011) √ √ √ *alves et al. (2014) √ √ √ *valenca, et.al (2013) √ √ borman & janssen (2012) √ √ doebeli (2011) √ table 18: attributes underpinning it-bpm governance the analysis indicated a higher emphasis on it-bpm governance in the literature on developing countries. ‘bpm awareness’, ‘training’, ‘competent professionals’, ‘bpm office’, and ‘reference frameworks’ (in the context of governance) were not discussed in papers from developed countries. only two studies on developed countries on this important topic reflected a generic lack of literature in this area. while a diverse set of sub-factors pertaining to it-bpm governance in the public sector are discussed in the literature, clear normative guidelines on how to set effective, holistic bpm governance in public sector settings has been given little attention. in recognition that there are many bpm governance models out there, we propose future research that investigates; ‘how to best select, adopt and deploy bpm governance frameworks in public sector settings?’ we also see value in specifically investigating the roles and responsibilities aspect, and hence propose future research that investigates ‘how can bpm-related roles and responsibilities be effectively designed and managed in public sector settings?’ 5.13 strategic clarity & alignment the review of literature identified two different points of view related to strategic clarity and alignment. according to borras (2012), it is setting a clear vision, business case, and result focused strategy for e-government initiatives. more inclusive definitions demarcated strategic alignment as the interrelationship between strategic goals for an e-government project and the overall government’s vision and mission (kwamena, 2012; marilu, 2012). contrary to these, kamal (2006) defined it as the suitability of the system with the organisational need and organisational compatibility for cross-departmental information sharing. on similar lines, nfuka and rusu (2011, p. 1436) mentioned it as “alignment of it with business strategies and enabling it structures. this can make possible successful integration of it with effects in the public service delivery improvement in and across these widespread organizations” the above was strongly supported by weerakkody et al. (2011) who linked the lack of alignment of the e-government plan as the key factor for failure of service delivery. table 19 illustrates the sub-factors that were discussed by the literature. australasian journal of information systems syed, bandara, french & stewart 2018, vol 22, research article csf of bpm in public sector 29 (*) represents developing countries c le ar v is io n bu si ne ss c as e r es ul t fo cu se d st ra te gy a lig nm en t of v is io n w ith st ra te gi c in fo rm at io n sh ar in g *nfuka and rusu (2011) √ *weerakkody et al. (2011) √ borras (2012) √ √ √ gulledge and sommer (2002) √ kamal (2006) √ kwanmena (2012) √ √ marilu (2012) √ table 19: summary of attributes of strategic clarity and alignment studies related to both developed and developing countries showed the ‘alignment of vision with strategic goals’ as a common factor. literature on developing countries did not discuss the other factors as presented by the literature on developed countries. there is value in investigating and clearly articulating ‘how can bpm initiatives be best aligned with the strategic directions especially in public sector settings?’ which can guide public sector organisations to gain better strategic alignment of their bpm initiatives. 5.14 external environmental factors external environmental factors that include socio-economic factors (national income, political structure, literacy, population characteristics, etc.) can influence activities of a bpm enabled transformation. kamal (2006) lists a number of sub-factors as key influencing environmental factors that can affect adoption of innovations in government organisations. marvine et al. (2012) referred to controls and laws and regulations as such contributing external environmental sub-factors. sub-factors related to external environmental factors are presented in table 20. (*) represents developing countries in te rg ov er nm en ta l in fl ue nc e pr ov is io n of g ra nt s t ra ns fe rs t ec hn ic al a ss is ta nc e c on tr ol s la w s & r eg ul at io ns *marvine et al. (2012) √ √ kamal (2006) √ √ √ √ table 20: summary of attributes underpinning external environmental factors as highlighted by sadowsky (1996), the external environment of countries is influenced by many factors that warrant careful attention. as presented in table 20, current literature evidenced limited attention given to date to better identify and understand the influences of external environmental factors on bpm in the public sector (in both developed and developing countries). availability and access to specialised knowledge, resources (human & funds), political climate and administrative policies, and socio-economic/socio-ethnic compositions can significantly alter bpm enabled transformation endeavours, especially in a developing country (chen et al., 2006). the external environment is contextual, complex and diverse, and influences all other factors. hence, we argue that these factors need to be further analysed; and propose that future research investigate in more depth; ‘what contextual factors from the external environment influence bpm initiatives in the public sector?’, and also study more carefully ‘how australasian journal of information systems syed, bandara, french & stewart 2018, vol 22, research article csf of bpm in public sector 30 can these external environmental contextual factors influence other critical success factors within public sector bpm initiatives?’ 6 summary discussion the goal of this systematic literature review was to collate and analyse csfs of bpm efforts as discussed in public sector settings, post the last such synthesis which was reported by mcadam et al. in 1999. public sector bpm csfs as presented in papers post 1999 were collated, analysed, clearly defined (see table 6) and reported herein with special attention to elicit any mentioned sub-factors (see tables 720) to better understand each csf. given the differences between developing and developed contexts (as discussed above), attention was given in the analysis towards identifying any observable differences between the csfs reported within the developed and developing contexts. the overall analysis used a rigorous qualitative data analysis procedure following bandara et al. (2015). the mcadam and donaghy (1999) framework was used as the a-priori model for public sector bpm csfs. the analysis of 31 relevant papers, all derived post mcadam and donaghy (1999) resulted in the following: 1. confirmation of all the mcadam and donaghy (1999) factors (see table 6). 2. identification of seven new factors (see table 6) resulting in 14 csfs observed to be relevant to bpm in public sector organisations. top management support, communication, alleviation of downsizing fears (resistance to change), ict awareness, empowerment, and enlisting customer & stakeholder support were amongst the widely discussed factors by the researchers; it-bpm governance and culture were the least discussed critical success factors in the body of the literature. 3. capturing of sub-factors pertaining to each factor (with supporting literature evidence see table 7 to 20), and providing further insights to the csfs and contributing towards how to action them (for practice) and/or how to operationalise them (for future research). 4. deriving from the results a series of prospective research questions based on gaps and opportunities observed from this analysis that could shape the future research around bpm in public sector (see table 21) all identified csfs (14 of them) are common to both developing and developed countries. we did not discover any distinctly different csfs for the developing countries. this could be attributed to a number of reasons, such as: • the limited number of papers (8) representing the developing country context in the pool of papers found. • limited attention given to research specifically within the developing context (kennedy et al., 2012; walsham et al., 2007). this could be due to the resource and skill constraints in these countries to do effective research in these specific contexts, which could also explain the very limited number of papers of the developing country context found in high quality research outlets. • the limited research activities and practice happening in the developing country context to try to ‘learn’ and mimic from the strategies adopted by the developed countries with little attention given to better contextualise these to the specific australasian journal of information systems syed, bandara, french & stewart 2018, vol 22, research article csf of bpm in public sector 31 developing country context. we believe this can further explain why the current literature identified common factors amongst the two with little differentiation. however, sometimes a different emphasis on the sub-factors (within each main factor) was observed between developed and developing country contexts and this was discussed in detail as each csf was presented. as evident from this review, prior research is heavily skewed towards the developed context. the validity and suitability of these factors in a developing country context can be argued and provides an opportunity for further analysis and investigation. as posited earlier under the section titled; “public vs private sector bpm differences and similarities”, bpm efforts in developing countries could differ widely for various reasons thus supporting our proposition for deeper, contextually rich studies of bpm in the public sector of developing nations to gain contextually aligned insights. in addition, the application of existing frameworks and theories built in the developed country context may not be appropriate for developing countries for, “an interesting proposition was always the negation of an accepted one” (davis, 1971, p313), what seems to be the case is in fact not really the case when borrowing across diverse contexts. the international monetary fund (2015) classified 155 countries as developing nations. with 80% of the world population residing in developing countries, governments struggle to provide key public services to their citizens in an efficient and effective way. our search of literature identified only 8 papers constituting 26% of the studies with a focus on developing nations. given the few papers observed in top tier is and business journals on this important and under-researched area, we make a call for action to editors of leading journals of our disciplines to encourage research in this area by having special issues and showing more acceptance and interest to papers from the developing context, recognising it as an innovative (untapped) area to research. we also make a call for action to researchers interested in this area to make use of this body of knowledge of csfs pulled together in this review, and build on existing work to support cumulative knowledge creation and progression of developing country context research. as each factor was analysed, we elicited areas deemed to be overlooked, under-researched, or with lack of empirical support, and also sought ‘new ideas’ worthy and in need of future research (following alvesson and sandberg’s 2013). following muller-bloch and kranz’s (2015) guidelines, we summarise our views and present a list of potential future research questions in table 21. for ease of reading, the proposed questions (column 2) are grouped by the csfs (column 1) where the questions were originally proposed in the discussions above, but are not mutually exclusive to that csf. in other words, some of the questions proposed may directly or indirectly relate to multiple factors. most of the proposed questions are applicable to public sector bpm research in both developed and developing country contexts unless specifically mentioned otherwise. we also acknowledge that there can be a lot of diversity within the broader groupings of developed and developing countries (i.e. based on the region, per capita income and other diverse/ country specific needs), which could influence how and where the proposed research questions in table 21 can be adopted. we acknowledge the subjectivity of these suggested future research questions in recognition that they were derived from our interpretations which are potentially influenced by our world views, other literature we have read, and our own areas of research interests. however, such subjectivity is expected in any gap-spotting exercise (alvesson and sandberg, 2013). further australasian journal of information systems syed, bandara, french & stewart 2018, vol 22, research article csf of bpm in public sector 32 literature outside the scope of this review was referred to at times to confirm the gaps presented here. 1. top management support how can top management support within public sector bpm initiatives be conceptualised? how does leadership contribute to the success and sustainability of bpm in the public sector? how to build leadership capabilities on process-centric innovation in public sector? how to obtain sustained long term top management commitment for bpm in public sector? 2. communication what are the essential elements of a public sector bpm communication plan? who are public sector bpm stakeholders and what are their potential impacts on bpm efforts? 3. preparedness for organisational change what constitutes a well-designed organisational change strategy in a public sector bpm initiative? how can social and cultural factors in public sector organisations influence a bpm initiative, in particular for better stakeholder engagement? how can training & development be designed to support effective bpm change in the public sector? 4. enlisting customer and stakeholder support / involvement how to effectively engage the key stakeholders in public sector bpm initiatives? what are the innovative stakeholder engagement models that enable bpm success in public sector contexts? 5. choosing the bpm team what constitutes an effective bpm team in public sector contexts? how to develop bpm capabilities (in general) in developing countries? 6. alleviation of downsizing fears (resistance to change) what are the key contributing factors that lead to resistance to change for bpm in public sector organisations? does resistance to change for bpm differ between staff at different levels in public sector contexts? how to identify the most vulnerable staff groups within government departments about to deploy bpm, and anticipate reactions? 7. empowerment how can public sector organisations be receptive to employee empowered bottom-up bpm approaches? how to create a sense of empowerment amongst public sector employees to support sustained bpm success? what organisational structures would best provide accountability, responsibility, and authority to enable public sector bpm success? 8. ict infrastructure how can governments in developing countries best design and benefit from the use of emerging new ict options such as; shared services, cloud computing, open source, etc.? 9. project management and pm skills how to integrate project management practices into bpm methodology? how to ensure effective project management training and training transfer in public sector? how to build project management awareness in public sector bpm contexts? 10. ict-awareness how to ensure effective bpm training and training transfer in public sector organisations? how does training in bpm skills improve bpm success in public sector organisations? how can public sector organisations proactively use ict for bpm efforts? australasian journal of information systems syed, bandara, french & stewart 2018, vol 22, research article csf of bpm in public sector 33 11. culture how to develop mechanisms to implement robust and innovative process-centric cultures in the public sector? how organisational culture and its characteristics in developing countries uniquely affect bpm initiatives in public sector organisations? 12. it-bpm governance how to best select, adopt and deploy bpm governance frameworks in public sector settings? how can bpm-related roles and responsibilities be effectively designed and managed in public sector settings? 13. strategic clarity & alignment how can bpm initiatives be best aligned with the strategic directions especially in public sector settings? 14. external environmental factors what contextual factors from the external environment influence bpm initiatives in the public sector? how can these external environmental contextual factors influence other critical success factors within public sector bpm initiatives? table 21: prospective future research questions 7 conclusion governments across the world are still turning to ict-enabled bpm initiatives to transform their services to citizen-centric and sustainable. a sound understanding of csfs may provide a firm understanding on areas in need of attention to improve the effective design and implementation of ict-enabled bpm initiatives. public sector organisations have different characteristics to those in the private sector and likewise, the developing context is often different to the developed context. this study was driven by the goal to identify and understand csfs for bpm projects specific to the public sector (in recent researchpost the last reported synthesis in 1999), with an anticipation to observe differences between developed and developing contexts. the paper presents and details csfs pertaining to bpm initiatives that relate to both the developed and developing country contexts. the analysis confirms and expands prior work on csfs of public sector bpm, and discusses each factor, providing evidence on the composition of these factors. the paper closes by proposing a set of research questions and areas of investigation for the future as a forming research agenda and making a call for action. the study is constrained by the lack of literature specific to the public sector bpm initiatives and in particular to papers in developing countries. we acknowledge that this analysis was conducted based on a small pool of papers (31 in total, including 8 from the developing country context). this paper only focuses on identifying and exploring the critical factors, and does not include testing of the identified csfs with empirical results. however, the evidence-based csf synthesis provides opportunities for future research to explore the public sector bpm space to empirically test the validity of the identified factors. the fourteen (14) bpm csfs identified through this inductive analysis of bpm papers from the public sector context yielded factors deemed very similar to bpm csfs known from generic and/or private sector contexts. while the descriptions of each factor did present contextual specificities associated with the public sector, deeper analysis on how these factors manifest within different sectors (i.e. public vs private) and different institutional contexts would be a useful future investigation. australasian journal of information systems syed, bandara, french & stewart 2018, vol 22, research article csf of bpm in public sector 34 the proposed areas of investigation provide direction for areas of future research. we recommend further exploratory research in the context of public sector organisations, in particular the developing country context, to further understand the challenges and opportunities and to validate and respecify the research agenda with stronger evidence-based problematisation as recommended by alvesson and sandberg (2013). references ahmed, i., & gregor, s. (2007). a comparative analysis of strategies for egovernment in developing countries. journal of business systems, governance and ethics, 2(3), 89-99. al-kibsi, g., de boer, k., mourshed, m., & rea, n. p. (2001). putting citizens on-line, not in line. mckinsey quarterly, special ed(2), 64-73. al-mashari, m., & zairi, m. (1999). bpr implementation process: an analysis of key success and failure factors. business process management journal, 5(1), 87-112. doi:doi:10.1108/14637159910249108 alghamdi, i. a., goodwin, r., & rampersad, g. (2014). organizational e-government readiness: an investigation in saudi arabia. international journal of business and management, 9(5), 14-24. almarabeh, t., & abuali, a. (2010). a general framework for e-government: definition maturity challenges, opportunities, and success. european journal of scientific research, 39(1), 29-42. alves, c., valença, g., & santana, a. (2014). understanding the factors that influence the adoption of bpm in two brazilian public organizations. in i. bider, k. gaaloul, j. krogstie, s. nurcan, h. proper, r. schmidt, & p. soffer (eds.), enterprise, business-process and information systems modeling (vol. 175, pp. 272-286). berlin heidelberg: springer alvesson, m., & sandberg, j. (2013). constructing research questions: doing interesting research. london: sage publications ltd. retrieved from http://methods.sagepub.com/book/constructing-research-questions. doi:10.4135/9781446270035 attaran, m. (2004). exploring the relationship between information technology and business process reengineering. information & management, 41(5), 585-596. axelsson, k., melin, u., & söderström, f. (2011). analyzing best practice and critical success factors in a health information system case – are there any shortcuts to successful it implementation?". paper presented at the european conference on information systems (ecis), helsinki. bandara, w., furtmuller, e., gorbacheva, e., miskon, s., & beekhuyzen, j. (2015). achieving rigour in literature reviews: insights from qualitative data analysis and tool-support. communications of the association for information systems. special issue on the literature review in information systems research. bandara, w., syed, r., kapurubandra, m., & rupasinghe, l. (2012). building essential bpm capabilities to assist successful ict deployment in the developing context: observations and recommendations from sri lanka. paper presented at the 5th annual workshop of the ais special interest group for ict in global development, orlando. http://aisel.aisnet.org/globdev2012/14 australasian journal of information systems syed, bandara, french & stewart 2018, vol 22, research article csf of bpm in public sector 35 bazeley, p. (2007). qualitative data analysis with nvivo. los angeles london: sage. bhatnagar, s. (2004). building blocks of e-government: lessons from developing countries. world bank retrieved from http://www1.worldbank.org/prem/premnotes/premnote91.pdf. bin taher, n. a., krotov, v., & silva, l. (2015). a framework for leading change in the uae public sector. international journal of organizational analysis, 23(3), 348-363. doi:10.1108/ijoa-10-2014-0809 borman, m., & janssen, m. (2012). critical success factors for shared services: results from two case studies. paper presented at the 45th hawaii international conference on system sciences. borras, j. (2012). the oasis transformational government framework. european journal of epractice, 15, 26-51. buh, b., kovačič, a., & indihar štemberger, m. (2015). critical success factors for different stages of business process management adoption–a case study. ekonomska istraživanja, 28(1), 243-257. chen, y., chen, h., huang, w., & ching, r. k. (2006). e-government strategies in developed and developing countries: an implementation framework and case study. journal of global information management, 14(1), 23. chong, s., & rosemann, m. (2010). towards a framework for bpm communication. paper presented at the 7th itais conference naples dada, d. (2006). the failure of e-government in developing countries: a literature review. the electronic journal of information systems in developing countries, 26 davenport, t. h. (1993). process innovation: reengineering work through information technology. boston, mass: harvard business school press. davenport, t. h., & short, j. e. (1990). the new industrial engineering: information technology and business process redesign. sloan management review, 31(4), 11-27. decuir-gunby, j. t., marshall, p. l., & mcculloch, a. w. (2011). developing and using a codebook for the analysis of interview data: an example from a professional development research project. field methods, 23(2), 136-155. doi:10.1177/1525822x10388468 dexter, b. (2010). critical success factors for developmental team projects. team performance management, 16(7/8), 343-358. doi:http://dx.doi.org/10.1108/13527591011090637 dobbins, j. h., & donnelly, r. g. (1998). summary research report on critical success factors in federal government program management. retrieved from fort belvoir, va: doebeli, g., fisher, r., gapp, r., & sanzogni, l. (2011). using bpm governance to align systems and practice. business process management journal, 17(2), 184-202. doi:doi:10.1108/14637151111122310 dubey, s. k., & bansal, s. (2013). critical success factors in implementing bpr in a government manufacturing unit an empirical study. international journal of business and management, 8(2), 107-124. australasian journal of information systems syed, bandara, french & stewart 2018, vol 22, research article csf of bpm in public sector 36 fettke, p., zwicker, j., & loos, p. (2014). business process maturity in public administrations. in j. vom brocke & m. rosemann (eds.), handbook on business process management 2 : strategic alignment, governance, people and culture (vol. 2;2nd;2nd 2015;). berlin, heidelberg: springer. gaffar, a., deshpande, a., bandara, w., & mathiesen, p. (2015). importance of literature profiling: an archival analysis with illustrative examples for is researchers. paper presented at the pacis 2015 proceedings: pacific asia conference on information systems. gulledge, t. r., & sommer, r. a. (2002). business process management: public sector implications. business process management journal, 8(4), 364-376. doi:doi:10.1108/14637150210435017 halachmi, a., & bovaird, t. (1997). process reengineering in the public sector: learning some private sector lessons. technovation, 17(5), 227-235. doi:http://dx.doi.org/10.1016/s01664972(96)00123-x hartini, a., francis, a., & zairi, m. (2007). business process reengineering: critical success factors in higher education. business process management journal, 13(3), 451-469. doi:doi:10.1108/14637150710752344 heeks, r. (2003). most egovernment-for-development projects fail: how can risks be reduced? igovernment-information systems, technology and governement: working papers. heeks, r. (2008a). benchmarking e-government: improving the national and international measurement, evaluation and comparison of e-government. in z. irani & p. love (eds.), evaluating information systems (pp. 257-301). oxford: butterworth-heinemann. heeks, r. (2008b). ict4d 2.0: the next phase of applying ict for international development. computer, 41(6), 26-33. doi:10.1109/mc.2008.192 heeks, r., & stanforth, c. (2007). understanding e-government project trajectories from an actor-network perspective. european journal of information systems, 16(2), 165-177. hill, j. b., sinur, j., flint, d., & melenovsky, m. j. (2006). gartner's position on business process management. gartner research g, 136533. hisham, m. a., christopher, g. r., & hatem, a. e. k. (2012). success and failure of local egovernment projects: lessons learned from egypt managing e-government projects: concepts, issues, and best practices (pp. 242-261). hershey, pa, usa: igi global. hofstede, g. (1984). cultural dimensions in management and planning. asia pacific journal of management, 1(2), 81-99. huang, z., & palvia, p. (2001). erp implementation issues in advanced and developing countries. business process management journal, 7(3), 276-284. hunter, m. g. (2009). selected readings on strategic information systems (vol. illustratition). us: igi global. hutton, g. (1996). business process re-engineering–a public sector view. in c. armistead & p. rowland (eds.), managing business processes–bpr and beyond. chichester: john wiley and sons. international monetary fund. (2015). world economic outlook :uneven growth shortand longterm factors. washington, dc. australasian journal of information systems syed, bandara, french & stewart 2018, vol 22, research article csf of bpm in public sector 37 jurisch, m. c., ikas, c., palka, w., wolf, p., & krcmar, h. (2012). a review of success factors and challenges of public sector bpr implementations. paper presented at the 45th hawaii international conference on system science. kamal, m. m. (2006). it innovation adoption in the government sector: identifying the critical success factors. journal of enterprise information management, 19(1/2), 192-222. kassahun, a. e., & molla, a. (2011). bpr complementary competence for developing economy public sector: a construct and measurement instrument. paper presented at the pacis. kassahun, a. e., molla, a., & sarkar, p. (2011). government process reengineering: what we know and what we need to know. in c. susheel & k. muneesh (eds.), strategic enterprise resource planning models for e-government: applications and methodologies (pp. 1-24). hershey pa kennedy, a., joseph, p. c., & carol, k. (2012). business process change in e-government projects: the case of the irish land registry technology enabled transformation of the public sector: advances in e-government (pp. 9-22). hershey, pa, usa: igi global. kwamena, s. a. (2012). improving e-government project management: best practices and critical success factors managing e-government projects: concepts, issues, and best practices (pp. 42-60). hershey, pa, usa: igi global. leidecker, j. k., & bruno, a. v. (1984). identifying and using critical success factors. long range planning, 17(1), 23-32. doi:http://dx.doi.org/10.1016/0024-6301(84)90163-8 likert, r. (1961). new patterns of management. marilu, g. (2012). organizational change contributions to e-government project transitions managing e-government projects: concepts, issues, and best practices (pp. 1-21). hershey, pa, usa: igi global. marvine, h., martin, a. n., doaa, t., & salah, b. (2012). e-government implementation in a developing country: a case study managing e-government projects: concepts, issues, and best practices (pp. 262-289). hershey, pa, usa: igi global. mcadam, r., & donaghy, j. (1999). business process re-engineering in the public sector. business process management journal, 5(1), 33-52. doi:doi:10.1108/14637159910249135 meier, r., ben, e. r., & schuppan, t. (2013). ict-enabled public sector organisational transformation: factors constituting resistance to change. information polity: the international journal of government & democracy in the information age, 18(4), 315-329. doi:10.3233/ip-130315 nfuka, e. n., & rusu, l. (2011). the effect of critical success factors on it governance performance. industrial management & data systems, 111(9), 1418-1448. doi:doi:10.1108/02635571111182773 rainford, s. (2006). e-sri lanka: an integrated approach to e-government case study. regional development dialogue, 27(2), 209-218. rajapakse, j. (2013). e-government adoptions in developing countries: a sri lankan case study. international journal of electronic government research, 9(4), 38. doi:http://dx.doi.org/10.1108/14637150710823110 australasian journal of information systems syed, bandara, french & stewart 2018, vol 22, research article csf of bpm in public sector 38 ravesteyn, p., & batenburg, r. (2010). surveying the critical success factors of bpm-systems implementation. business process management journal, 16(3), 492-507. doi:doi:10.1108/14637151011049467 sadowsky, g. (1996). the internet society and developing countries. e-on the internet. retrieved from https://www.isoc.org/oti/articles/1196/sadowsky.html streib, g., & navarro, i. (2008). city managers and e-government development: assessing technology literacy and leadership needs. international journal of electronic government research, 4(4), 37-53. syed, r., bandara, w., french, e., & stewart, g. (2016). the status of research on leadership in business process management: a call for action. paper presented at the anzam 2016: 30th australian and new zealand academy of management conference: under new management: innovating for sustainable and just futures, brisbane. http://eprints.qut.edu.au/98771/ tan, w.-g., cater-steel, a., & toleman, m. (2009). implementing it service management: a case study focussing on critical success factors. journal of computer information systems, 50(2), 1. thong, j. y. l., yap, c.-s., & seah, k.-l. (2000). business process reengineering in the public sector: the case of the housing development board in singapore. journal of management information systems, 17(1), 245-270. tregear, r., & jenkins, t. (2007). government process management: a review of key differences between the public and private sectors and their influence on the achievement of public sector process management. . bptrends. trkman, p. (2010). the critical success factors of business process management. international journal of information management, 30(2), 125-134. doi:10.1016/j.ijinfomgt.2009.07.003 united nations. (2015). millennium development goals. retrieved from http://www.un.org/millenniumgoals/ united nations. (2016). united nations e-government survey 2016: e-government in support of sustainable development. retrieved from new york http://workspace.unpan.org/sites/internet/documents/unpan96407.pdf valenca, g., alves, c. f., santana, a. f. l., de oliveira, j. a. p., & santos, h. r. m. (2013). understanding the adoption of bpm governance in brazilian public sector. paper presented at the proceedings of the 21st european conference on information systems. walsham, g., robey, d., & sahay, s. (2007). foreword: special issue on information systems in developing countries. mis quarterly, 317-326. walsham, g., & sahay, s. (2006). research on information systems in developing countries: current landscape and future prospects ramiro montealegre was the accepting associate editor for this paper. information technology for development, 12(1), 7-24. doi:10.1002/itdj.20020 ward, j., & peppard, j. (2002). strategic planning for information systems (3rd ed.). west sussex: john wiley & sons, ltd. australasian journal of information systems syed, bandara, french & stewart 2018, vol 22, research article csf of bpm in public sector 39 webster, j., & watson, r. t. (2002). analyzing the past to prepare for the future: writing a literature review. mis quarterly, 26(2), xiii-xxiii. weerakkody, v., baire, s., & choudrie, j. (2006). e-government: the need for effective process management in the public sector. paper presented at the proceedings of the 39th annual hawaii international conference on system sciences, hawaii. weerakkody, v., el-haddadeh, r., & al-shafi, s. (2011). exploring the complexities of egovernment implementation and diffusion in a developing country. journal of enterprise information management, 24(2), 172-196. doi:doi:10.1108/17410391111106293 weerakkody, v., janssen, m., & dwivedi, y. k. (2011). transformational change and business process reengineering (bpr): lessons from the british and dutch public sector. government information quarterly, 28(3), 320-328. doi:http://dx.doi.org/10.1016/j.giq.2010.07.010 world bank. (2014). implementation completion and results report (icr-0000939). retrieved from zarei, b., ghapanchi, a., & sattary, b. (2008). toward national e-government development models for developing countries: a nine-stage model. the international information & library review, 40(3), 199-207. doi:http://dx.doi.org/10.1016/j.iilr.2008.04.001 copyright: © 2018 syed, bandara, french & stewart. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 background 2.1 public versus private sector bpm differences and similarities 2.2 developed versus developing country context differences and similarities 3 approach and methodology 3.1 phase 1: searching and extracting the relevant pool of papers 3.2 phase 2: preparation for the analysis 3.3 phase 3 & 4: the data analysis procedures and literature profiling outcomes 3.4 profile of the reviewed articles 4 study findings: the critical success factors of bpm 4.1 identifying and defining the critical success factors 5 a detailed review of the critical success factors 5.1 top management support and understanding of bpm 5.2 communication 5.3 preparedness for organisational change 5.4 enlisting customer and other external stakeholder support/involvement 5.5 choosing the bpm team 5.6 alleviation of downsizing fears (resistance to change) 5.7 empowerment 5.8 ict infrastructure 5.9 project management and project management skills 5.10 ict – awareness 5.11 culture 5.12 it-bpm governance 5.13 strategic clarity & alignment 5.14 external environmental factors 6 summary discussion 7 conclusion paper title (paper title style) australasian journal of information systems harvie, eustace, kerr, scott-parker 2016, vol 20, post publication review good old gamers, good drivers post publication review vichitvanichphong, s., talaei-khoei, a., kerr, d., ghapanchi, a. h., & scott-parker, b. (2016). good old gamers, good drivers: results from a correlational experiment among older drivers. australasian journal of information systems, 20. doi: 10.3127/ajis.v20i0.1110 review the recently published paper by vichitvanichphong, talaei-khoei, kerr, ghapanchi, and scottparker (2016), is interesting because their findings are contrary to previous research which reported that playing video games skills did not transfer to driving. their review of the literature showed that functional declines related to driving ability in many cases result in accidents that could be avoided if particular video games were used to test the abilities of elderly drivers. one thing that was not clear from their findings was the effect of training. vichitvanichphong et al cite literature to show that training can have positive effects on game play and driving ability, and also that functional declines can be counted. but they do not appear to investigate whether the close association between video game skills and driving ability mean that functional declines can be countered in older drivers. recent work in functional declines, not related to driving, have shown that improvements can be achieved, even for the very old (bernoth et al., 2016). furthermore, training in computing skills has been shown to have positive effects in quality of life outcome for community-dwelling older adults, many of whom are still driving (burmeister, bernoth, dietsch, & cleary, 2016). although we understand the reasons for choosing to research the young-old, our interest in this review particularly concerns the oldest-old, which is the focus of our research (eustace & burmeister, 2013; harvie, burmeister, & eustace, 2014, 2016). it might be interesting to perform the same research on the older age group, as this generation are less familiar with technology. many of your “early elderly” (young-old) participants would not be fazed by the introduction of the xbox, while the older generation, many of whom are still quite good drivers, may perform differently with the unfamiliar technology. furthermore, it would also be good to see more recent data, rather than the dated references to 2012 and the 2010 claim that “older drivers are overrepresented in crash statistics”. recent abs data suggests instead that males in the 20-24 age group and males in the 85+ age group which are in fact over-represented. drivers aged 65-74 seem to be relatively safe with quite low crash statistics (http://www.abs.gov.au/ausstats/abs@.nsf/0/1cd2b1952afc5e7aca257298000f2e76?o pendocument). the functional declines, particularly the cognitive ones, will be most prominent in the oldest drivers and hence doing similar, further research with that age group would seem worthwhile. in other areas a great deal of literature has been published on the cognitive declines affecting such people, including in relation to technology use (pakrasi, burmeister, mccallum, coppola, & loeb, 2015; teipel et al., 2016). however, that requires addressing their needs more directly, including privacy (more important to the oldest-old), accessibility (as vichitvanichphong et al point out, hearing and vision loss increase with increasing age) and greater professionalism will be required in working with such older people (bowern, burmeister, gotterbarn, & weckert, 2006; burmeister, islam, dayhew, & crichton, 2015; burmeister & weckert, 2003; carlson, farrelly, frazer, & borthwick, 2015; cockcroft, 2006). in conclusion, the aim of reducing driving related misadventure for older drivers is laudable. it is encouraging to know that video game use represents a low risk means to discovering potential driving difficulties. the research conducted by vichitvanichphong et al suggests that interventions to overcome/address functional declines and to increase training in driving could mean that older drivers can safely navigate the road for longer, with appropriate help, if through game play the problems of increasing age are discovered early enough. 1 australasian journal of information systems harvie, eustace, kerr, scott-parker 2016, vol 20, post publication review good old gamers, good drivers gillian harvie project officer, information technology services, courts administration authority, adelaide, sa dr kenneth eustace, adjunct senior lecturer, school of computing and mathematics, charles sturt university, wagga wagga, nsw. author response we are pleased to provide a response to the post-publication review of harvie and eustace regarding our recent publication, good old gamers, good drivers: results from a correlational experiment among older drivers. harvie and eustace note that our research is particularly interesting as some of the findings, viz a viz that video game skills transfer to real world driving, are contrary to earlier findings in which video game skills did not transfer to real world driving. harvie and eustace note that the effect of training, as reported, was unclear however. in relation to the statement about older drivers being overrepresented in crash statistics. we suggest that while it is not uncommon for delays to be experienced in the publication of complete crash fatality data (particularly as the definition for road crash fatality in australia pertains to death within 30 days as a result of injuries sustained in a road traffic crash), peerreviewed publication of research can and frequently does happen via a long and winding path. such circumstances are evidenced by our publication of some older statistics regarding older driver crash rates, as noted by harvie and eustace. to provide some context regarding the involvement of older drivers in car crashes, it is noteworthy that drivers aged 60-74 years comprise 14.5%, and drivers aged >74 years comprise 4.2%, of the licensed population in queensland in 2015 (dtmr, 2015). in queensland in 2015 (the latest figures currently available), senior adult road users aged 60-74 years comprised 10.7% of road crash fatalities, and senior adult road users aged >74 years comprised 14.4% of road crash fatalities. in this same period, senior adults road users aged 60-74 years were involved in 14.8% of road crash fatalities, and senior adults road users aged >74 years were involved in 12.8% of road crash fatalities. regarding hospitalised casualties, in queensland in 2013 (the latest figures currently available), senior adult road users aged 60-74 years comprised 10.7% of hospitalised casualties, with 17.0% of all hospitalised casualties across the state in queensland in 2013 arising from crashes involving a senior adult road user aged 60-74 years. during this same period, 4.4% of hospitalised casualties were senior adult road users aged >74 years, and 5.9% of hospitalised casualties arose from a road traffic crash involving a senior adult road user aged >74 years (dtmr, 2016). while it is apparent that the ‘older old’ are at greater risk of road traffic crash fatality and injury, it is noteworthy that regulation of driving exposure has been found amongst older drivers such that they drive shorter distances and durations, avoiding driving times of increased risk such as peak hour and at night (e.g.,king & scott-parker, in press; meng & siren, 2012). accordingly, the true crash fatality and injury risk of senior adult road users may be a simple underestimation at best, and a catastrophic underestimation at worst. we agree that the need to conduct further research is ever present, as reflected by harvie and euestace’s comments pertaining to the ‘oldest old’ drivers. we acknowledge the sensitivity of this research in relation to privacy issues. for example, what happens if the driving test is considered so bad that the older person should not be on the road or what happens if a serious accident occurs during the driving test? in our ethics application, this was covered through insurance policies and the option for the tester to terminate the test at any time if the tester felt unsafe. however, we do accept that more thought has to be put into this very real area of concern. perhaps the use of more sophisticated driving simulators could 2 australasian journal of information systems harvie, eustace, kerr, scott-parker 2016, vol 20, post publication review good old gamers, good drivers be an answer. also the provision of counselling in the case of a bad driving experience that could lead to a crisis in confidence for the older driver may be appropriate. this problem is especially concerning for the older old driver. thank-you for your statement on the positive contribution this research may bring. it is the intention of the research to help reduce the road toll through introducing safer ways of helping people maintain their driving skills and the associated mobility which affords access to such amenities as healthcare, entertainment, goods and other services. we hope that this research can be expanded upon and can lead to games developers creating new and more entertaining video games to help people in this area. donald kerr university of the sunshine coast dkerr@usc.edu.au bridie scott-parker university of the sunshine coast bscottpa@usc.edu.au references bernoth, m., burmeister, o. k., morrison, m., islam, m. z., onslow, f., & cleary, m. (2016). the impact of a participatory care model on work satisfaction of care workers and the functionality, connectedness and mental health of community-dwelling older people. issues in mental health nursing, 1-7. doi: 10.3109/01612840.2016.1149260 bowern, m., burmeister, o. k., gotterbarn, d., & weckert, j. (2006). ict integrity: bringing the acs code of ethics up to date. australasian journal of information systems, 13(2), 168-181. burmeister, o. k., bernoth, m., dietsch, e., & cleary, m. (2016). enhancing connectedness through peer training for community-dwelling older people: a person centred approach. issues in mental health nursing, 1-6. doi: 10.3109/01612840.2016.1142623 burmeister, o. k., islam, m. z., dayhew, m., & crichton, m. (2015). enhancing client welfare through better communication of private mental health data between rural service providers. australasian journal of information systems, 19, 1-14. doi: http://dx.doi.org/10.3127/ajis.v19i0.1206 burmeister, o. k., & weckert, j. (2003). applying the new software engineering code of ethics to usability engineering: a study of 4 cases. journal of information, communication & ethics in society, 3(3), 119-132. carlson, b. l., farrelly, t., frazer, r., & borthwick, f. (2015). mediating tragedy: facebook, aboriginal peoples and suicide. australasian journal of information systems, 19. doi: 10.3127/ajis.v19i0.1174 cockcroft, s. (2006). information privacy: culture, legislation and user attitudes. australasian journal of information systems, 14(1). doi: 10.3127/ajis.v14i1.7 department of transport and main roads (dtmr). (2016). queensland road crash weekly report. report no 954. year to date to monday, 2 may 2016. available at http://www.tmr.qld.gov.au/safety/transport-and-road-statistics/road-safetystatistics.aspx department of transport and main roads (dtmr). (2015b). current queensland driver’s licences, as at 30 june 2015. available at http://www.tmr.qld.gov.au/safety/transportand-road-statistics/licensing-statistics eustace, k., & burmeister, o. (2013, december). ethics and governance of ict-based social engagement in institutional aged care. paper presented at the seventh australasian institute of computer ethics conference, burwood. 3 australasian journal of information systems harvie, eustace, kerr, scott-parker 2016, vol 20, post publication review good old gamers, good drivers harvie, g., burmeister, o. k., & eustace, k. (2014, november 24-25). bringing the oldest-old into the digital age: overcoming challenges of mobility, literacy, and learning. paper presented at the 13th national conference of emerging researchers in ageing, flinders university, victoria square, adelaide, south australia. harvie, g., burmeister, o. k., & eustace, k. (2016). assistive technology devices for the oldest old: maintaining independence for the fourth age. paper presented at the 12th ifip tc9 human choice and computers conference, salford, uk. king, m. j., & scott-parker, b. (2016). older male and female drivers in car-dependent settings: how much do they use other modes, and do they compensate for reduced driving to maintain mobility? ageing and society, in press. meng, a. and siren, a. 2012. older drivers’ reasons for reducing the overall amount of their driving and for avoiding selected driving situations. journal of applied gerontology, 5. pakrasi, s., burmeister, o. k., mccallum, t. j., coppola, j. f., & loeb, g. (2015). ethical telehealth design for users with dementia. gerontechnology, 13(4), 383-387. doi: 10.4017/gt.2015.13.4.002.00 teipel, s., babiloni, c., hoey, j., kaye, j., kirste, t., & burmeister, o. k. (2016). information and communication technology solutions for outdoor navigation in dementia. alzheimer's & dementia: the journal of the alzheimer's association, 1-13. doi: 10.1016/j.jalz.2015.11.003 vichitvanichphong, s., talaei-khoei, a., kerr, d., ghapanchi, a. h., & scott-parker, b. (2016). good old gamers, good drivers: results from a correlational experiment among older drivers. australasian journal of information systems, 20. doi: 10.3127/ajis.v20i0.1110 copyright: © 2016 harvie, eustace, kerr, scott-parker. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. 4 http://creativecommons.org/licenses/by-nc/3.0/au/ http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems ahmad & campbell 2015, vol. 19, research on indigenous ict citizen perceptions of e-government citizen perceptions of e-government in the kurdistan region of iraq khosro mohammad ahmad university of canberra khosro_araki@yahoo.com john campbell university of canberra john.campbell@canberra.edu.au abstract despite increasing investment by governments globally, citizens in some countries are less willing to engage with e-government innovations. while previous studies have examined what might encourage citizens to adopt e-government services more broadly, further research is required to understand those factors that impede or enable e-government adoption in countries that have, or are in the process of, transitioning to democratic self-rule. this study is the first to investigate citizen attitudes to e-government adoption in the kurdistan region of iraq and thus provides insight into the factors affecting the adoption of e-government within an emerging democratic state. this study also distinguishes between two types of egovernment services; informational and transactional. thus a research model with two elements is developed and tested; one for intention to use informational e-government and one for intention to use transactional e-government. data were collected through a survey administered to university students and non-academic employees in arbil the capital city of kurdistan, and analysed using structural equation modelling. perceived usefulness, social influence, trust in government and perceived risk were found to be significantly and directly associated with the intention to use both kinds of e-government services. however perceptions of ease of use, information quality and trust in the internet were not significant. keywords: e-government adoption; kurdistan region; informational e-government; transactional e-government; intention to adopt 1 introduction the use of the internet to communicate and transact with customers has not been limited to commercial endeavours only. public sector organizations worldwide have also gained significant advantages by employing internet technology to support government activities, and to communicate with citizens and other stakeholders (kumar et al., 2007). e-government is defined as the “use of information technology to enable and improve the efficiency with which government services are provided to citizens, employees, businesses and agencies” (carter & belanger, 2005, p.5). it has been argued that e-government can generate substantial benefits for governments and citizens alike, and provide important infrastructure for the improvement of both societal and economic conditions in developing countries (gregor et al., 2013). through e-government initiatives government agencies in developing countries can improve service delivery and reduce costs, while citizens also benefit from greater transparency and increased access options to key government services (belanger & carter, 2008; featherman & pavlou, 2003; lean et al., 2009; ndou, 2004; wang & shih, 2009). the success of e-government initiatives is highly dependent upon citizens’ willingness to switch from traditional methods of engaging with government to online channels. this is particularly important because the use of e-government services requires voluntary support, and the introduction of online services does not usually bring traditional methods of government/citizen engagement to an end (alsaghier et al., 2009; lu et al., 2011; tung & rieck, 2005). despite the availability of e-government services, citizens may continue using the traditional engagement methods which may diminish the overall success of e-government 1 australasian journal of information systems ahmad & campbell 2015, vol. 19, research on indigenous ict citizen perceptions of e-government initiatives and even lead to their failure (guha & chakrabarti, 2014). further, the success of it developments in developing countries more broadly is reliant on contextual factors relating to culture, region and country (avgerou & walsham, 2000). thus, understanding citizens’ perceptions towards e-government and identifying the factors that motivate their acceptance of this technology is an important step towards successful e-government implementation in particular cultural and country contexts. there are a number of barriers that may inhibit e-government initiatives from reaching their potential in both developed and developing countries including a lack of infrastructure, skills, strategies, policies, and financial resources (basu, 2004; dada, 2006; ebrahim & irani, 2005; lam, 2005; ndou, 2004). these challenges bring our focus to the building blocks of egovernment which are those elements necessary to prepare a nation and its citizens for egovernment implementation particularly for governments in developing economies (belanger & carter, 2008; bwalya & mutula, 2014; gupta & gupta, 2008; kanat, 2009; kumar et al., 2007). further, progress towards e-government services in developing countries has been slower and less effective than in developed countries (kamal et al., 2013). like any other form of new technology, no matter how well-designed e-government services are, they will not succeed in realizing promised benefits if citizens are unwilling to adopt them and shift from offline to online engagement. in this regard, gauld et al. (2010, p.178) stated that “even apparently successful projects in terms of technological and project development may face ‘user failure’ if potential customers, employees, or citizens simply do not use the system, something that is surprisingly common, suggesting that it is at least sensible to give some regard to intended end users’ views on technology development and use”. this study is the first to explore citizens’ behavioural intentions towards the adoption of egovernment services in the kurdistan region of iraq. in addition to its contextual significance, we also seek to make theoretical and methodological contributions to the extant e-government literature by distinguishing between citizens’ intentions to use e-government for accessing information and conducting transactions. this distinction is lacking from many e-government studies despite empirical evidence showing that these two kinds of services have different implications with transactional e-government considered more complicated and more risky for citizens since it usually requires the exchange of personal and financial information with government agencies (kumar et al., 2007; vassilakis et al., 2005). 2 study context kurdistan is an autonomous region located in the north of the republic of iraq. the region borders iran to the east, turkey to the north, syria to the west and the rest of iraq to the south (see figure 1). the population is estimated to be approximately 5 million with almost 50% aged below 20 years of age. kurdistan occupies approximately 40,643 square kilometres which is four times the area of lebanon and larger than the netherlands (www.krg.org). there are three main governorates in the region namely duhok, arbil, and sulaymaniyah with arbil being the capital city of the region which is also called hawler by local people. the current government consists of a coalition of several political parties that reflect the diversity of the region’s people, who are chaldeans, assyrians, turkmen, yazidis and kurds who live together in peace. the main language in kurdistan is kurdish which comes from the indo-european family of languages, and has two main dialects, sorani and kurmanji. the sorani kurdish dialect uses arabic script while the kurmanji kurdish dialect is written in latin script. sorani is spoken in the cities of arbil and sulaymaniyah, while kurmanji is spoken in duhok. there are also other languages in the region such as turkmani, arabic, armenian, and assyrian which are spoken by their respective communities. 2 australasian journal of information systems ahmad & campbell 2015, vol. 19, research on indigenous ict citizen perceptions of e-government figure 1: geographical map of kurdistan (source: www.institutkurde.org). due to years of internal conflict with the previous iraq regime led by saddam hussein and the more recent civil war, the kurdistan region remained underdeveloped economically and isolated from the rest of the world. however, since the end of saddam hussein’s regime in 2003, the region has undergone substantial reconstruction and has experienced significant economic growth. the kurdistan government (krg) is determined to promote democracy in the region and to develop the quality of life of its inhabitants (www.krg.or). one area that the krg is particularly interested in is the implementation of e-government to establish modernized government with the aim of fostering transparent governance, building a stronger relationship with its citizens, and more effective delivery of basic services (krg it academy 2009). to achieve this, the krg developed a five-year plan (2011-2015) to prepare the region for e-government implementation. however, citizens’ perception about e-government has not yet been explored in the region and no study has yet been conducted to examine whether citizens are willing to adopt e-government services, and what factors might impact their decision. 3 australasian journal of information systems ahmad & campbell 2015, vol. 19, research on indigenous ict citizen perceptions of e-government 3 theoretical framework 3.1 information and transaction based e-government there are two key service types provided by e-government; informational and transactional (reddick, 2005; carter & belanger., 2004). that is, government websites can be used to either obtain information or conduct transactions online. informational e-government is the use of government websites to obtain information such as opening hours, download forms, or to obtain government contact details. in using this type of e-government the interaction between users and government agencies is a one-way interaction, and users do not have to provide any personal or financial information to receive the service (almahamid et al., 2010; chatfield & alhujran, 2009; horst et al., 2007). transactional e-government, in comparison, is the use of government websites to conduct transactions with government online (irani et al., 2006; reddick, 2005). this type of online service involves a two-way interaction and typically allows users to conduct an entire task online such as renewing a licence or car registration, applying for government jobs, or paying taxes (chatfield & alhujran, 2009). reddick (2005, p.41) argued that “this stage will empower citizens to deal with government online anytime, saving hours of paperwork, the inconvenience of travelling to a government office, and time spent waiting in line”. unlike informational e-government, transactional e-government often requires that users provide personal or financial information to the government department thus making these services more complex and potentially more risky (chatfield & alhujran, 2009; horst et al., 2007; irani et al., 2006). in this regard horst et al. (2007, p.1838) stated that “the risks of electronic services on the transactional level are more profound than at the informational level”. it is due to these differences that e-government researchers have suggested that citizens are more likely to adopt e-government for informational purposes only (gauld et al., 2010; kumar et al., 2007; vassilakis et al., 2005). therefore, it is important to distinguish between informational and transactional e-government when investigating citizens’ acceptance behaviour. this distinction enables researchers and government authorities to better understand the broad types of services particular citizens are willing to adopt, and to specifically identify the factors that are associated with intention to adopt either of these e-government service types. despite this importance, a review of e-government literature revealed that this distinction has been widely neglected by e-government researchers. in this regard, carter and belanger (2004) argued that future studies must distinguish between citizens’ intention to use egovernment systems to gather information as well as their intention to conduct online transactions. consequently, the current study distinguishes between intention to use both informational and transactional e-government services. 3.2 intention to adopt e-government services the concept of e-government adoption by citizens has been defined differently by various researchers (kumar et al., 2007). some have described it as the ‘intention’ (carter & belanger, 2005), and others have described it as the ‘willingness’ (gilbert et al., 2004) to use egovernment services. kumar et al. (2007, p.69) described e-government adoption as “a simple decision of using, or not using, online services”. consistently, e-government adoption, for the purpose of the current study, is defined as citizens’ intention/willingness to use government websites for informational and transactional purposes. note that willingness and intentions are used interchangeably in this study. the reason for measuring intention and not actual usage was because e-government in kurdistan is an emerging phenomenon and kurdistan’s citizens have little to no experience with accessing e-government services. therefore, this study was conducted to discover whether citizens in kurdistan are willing to adopt e-government services when they become available (mkude & wimmer, 2015). previous studies have shown that intention to use is a strong predictor of actual usage (ajzen, 1991; belanger & carter, 2008; chau & hu, 2001; chu et al., 2004; venkatesh et al., 2003). 4 australasian journal of information systems ahmad & campbell 2015, vol. 19, research on indigenous ict citizen perceptions of e-government 3.3 technology adoption models various models have been developed by researchers and practitioners to understand users’ acceptance of new technologies across various disciplines such as sociology, psychology and human behaviour (venkatesh et al., 2003). innovation diffusion theory was developed by rogers (1962) and originated from sociology. the theory of reasoned action (fishbein & ajzen, 1975) and its extensions; theory of planned behaviour (ajzen, 1991) and decomposed theory of planned behaviour (taylor & todd, 1995), and the motivational model (deci, 1972) originated from psychology. the technology acceptance model was developed by davis (1989) to explain the adoption of information technology in the workplace and was later extended to the technology acceptance model 2 (venkatesh & davis 2000). the model of pc utilization (triandis, 1971) and social cognitive theory (bandura, 1986) were both contributed from the human behaviour discipline. finally, the unified theory of technology acceptance and use (venkatesh et al., 2003) was developed by integrating the most important elements of these previous models. due to contextual differences there is no general model that can be used to explain egovernment adoption behaviour (kumar et al., 2007). consequently, e-government researchers have tried to develop their own models by adapting important constructs from existing technology adoption models. moreover, there are some factors that are thought to be important for determining e-government adoption behaviour which are not adequately considered by existing technology adoption models. for instance, risk and trust are perceived to be significant factors in explaining citizens’ e-government adoption behaviour, however they are both notably absent from the models discussed above (belanger & carter, 2008; carter & belanger, 2004; delafrooz et al., 2011; featherman & pavlou, 2003; gefen, 2003; hung et al., 2006; lu et al., 2011; mayer et al., 1995; mcknight & chervany, 2002; schaub & carter, 2005). 4 research model considering the characteristics of the study particularly the contextual newness of the phenomenon for kurdistan citizens, several variables were adapted from existing studies. as discussed above, this study distinguished between intention to use informational egovernment and intention to use transactional e-government. therefore, the research model also reflects these two elements (see figure 2). the first model element was designed to investigate citizens’ behavioural intentions towards informational e-government while the second section was designed to investigate citizens’ behavioural intentions towards transactional e-government. perceived usefulness (pu), perceived ease of use (peou), facilitating conditions (fc), social influence (si), perceived information quality (piq), compatibility (compt), trust in government (tig), trust in the internet (tii), and perceived risk (pr) were adapted as independent variables and behavioural intention (bi) was adapted as the dependent variable for both e-government service elements. 5 australasian journal of information systems ahmad & campbell 2015, vol. 19, research on indigenous ict citizen perceptions of e-government figure 2. hypothesized research model 4.1 hypothesis development 4.1.1 perceived usefulness this variable was adapted from the technology acceptance model (tam) and was defined as “the prospective user’s subjective probability that using a specific application system will increase his or her job performance within an organizational context” (davis et al., 1989, p. 985). tam posits that pu is associated with the intention to use new technology both directly and indirectly through its positive effect on attitude (davis, 1989). studies in the e-government domain have also found pu to be a significant determinant of intention to use e-government services (chu et al., 2004; featherman & pavlou, 2003; kumar et al., 2007; lin et al., 2011; schaupp & carter, 2005; wangpipatwong et al., 2008). pu in the context of e-government and this study is defined as the degree to which citizens believe using government websites to access information and conduct transactions will be useful. pu was hypothesized to have a direct and positive relationship with intention to use both informational and transactional e-government services (hypotheses 1 and 11). 4.1.2 perceived ease of use this construct was also adapted from tam and was defined as “the degree to which the prospective user expects the target system to be free of effort” (davis et al., 1989, p.985). tam posits that peou only indirectly predicts intention to use through its positive association with attitude and perceived usefulness. however, e-government studies have suggested that in addition to its indirect relationship with intention to use, peou can directly predict intention to use e-government services (chu et al., 2004; featherman & pavlou, 2003; fu et al., 2006; kumar et al., 2007; lin et al., 2011; schaupp & carter, 2005; wang, 2002; wangpipatwong et al., 2008). due to empirical evidence regarding its importance in predicting intentions to use, this variable was adapted in this study and was designed to measure participants’ perception regarding the ease of using government websites to access information and conduct transactions. peou was hypothesized to have a significant positive association with intention to use both informational and transactional e-government, directly as well as indirectly through its positive relation with perceived usefulness (hypotheses 2, 9, 10 and 20). 4.1.3 facilitating conditions this construct was adapted from the unified theory of acceptance and use of technology (utaut) and was defined as the degree to which participants believe they have the means such as the hardware and software necessary to access government websites to obtain information 6 australasian journal of information systems ahmad & campbell 2015, vol. 19, research on indigenous ict citizen perceptions of e-government and conduct transactions. utaut posits that fc is directly associated with the actual usage (venkatesh et al., 2003). in this study, fc was hypothesized to have a positive relationship with intention to use both informational and transactional e-government services (hypotheses 3 and 13). 4.1.4 social influence this variable was also adapted from the utaut model which defined si as “the degree to which an individual perceives that important others believe he or she should use the system” (venkatesh et al., 2003, p.451). social influence has been found to be directly associated with the intention to use a new system (al awadhi & morris, 2008; al-shafi & weerakkody, 2010; chu et al., 2004; lee, 2009; schaupp & carter, 2005; suki & ramayah, 2010; yaghoubi et al., 2010). si was defined, in the context of this study, as the degree to which participants believe that people who are important to them, such as friends and families, will have an influence on their decision to use e-government services for informational and transactional purposes. si was hypothesized to have a significant positive relationship with intention to use both informational and transactional e-government services (hypotheses 4 and 14). 4.1.5 compatibility compatibility was one of the five characteristics of an innovation that rogers (1983) suggested would have a positive effect on individuals’ decisions to adopt an innovation. compatibility reflects an individual’s perception about the congruence of a technology with his/her values, beliefs and lifestyle. support has been found in the e-government literature regarding the importance of this variable in influencing users’ intention to adopt e-government (carter & belanger, 2004; carter & belanger, 2005; fu et al., 2006; schaupp & carter, 2005; suki & ramayah, 2010). compatibility was deemed to be more relevant to transactional e-government because this type of e-government requires a significant change in the way people interact with government compared to informational e-government. thus compatibility was hypothesized to have a significant positive relationship with participants’ intention to use transactional egovernment (hypothesis 12). 4.1.6 perceived information quality this variable was adapted from delone and mclean (1992) and measures citizens’ perception about the quality of the information that is provided by state government agencies on their websites. according to delone and mclean (1992) the quality of the information that is provided on government websites can be assessed in terms of accuracy, relevance, precision, reliability, completeness, and currency. the e-government literature suggests that users’ perceptions about the quality of information they receive can have a significant impact on their behavioural intention to use government websites as a source of information (almahamid et al., 2010; al-shibly, 2011; lin et al., 2011; prybutok et al., 2008). therefore, this construct was integrated in the informational part of the model only and was hypothesized to have a direct and positive relationship with citizens’ intention to use government websites to seek information (hypothesis 5). 4.1.7 trust in the government and the internet mayer et al. (1995, p.712) defined trust as “the willingness of a party to be vulnerable to the actions of another party based on the expectations that the other will perform a particular action important to the trustor, irrespective of the ability to monitor or control that other party”. empirical support has been found for trust (both in the government and the internet) as a determinant of citizen adoption of e-government services (belanger & carter, 2008; carter & belanger, 2005; horst et al., 2007; schaupp & carter, 2005). trust in the government is defined as the extent to which citizens believe government agencies can be trusted in carrying out online transactions with them (carter & belanger, 2005; pavlou, 2003). therefore, the current study hypothesized that trust in the government will have a positive and direct association with citizens intention to adopt informational and transactional e-government (hypotheses 6 and 15). moreover, trust in the government was also hypothesized to indirectly 7 australasian journal of information systems ahmad & campbell 2015, vol. 19, research on indigenous ict citizen perceptions of e-government associate with the intention to use through its negative relation with perceived risk (hypotheses 8 and18). trust in the internet is more relevant to transactional e-government because informational egovernment does not require users to provide personal or financial information and thus it is unlikely that users would be concerned about the safety of the internet. for transactional egovernment, horst et al. (2007) argued that citizens must not only trust the government but also the infrastructure through which online transactions are performed: the internet. thus, trust in the internet was hypothesized to associate with the intention to use transactional egovernment both directly and indirectly through its negative association with perceived risk (hypotheses 16 and 19). 4.1.8 perceived risk risk is generally defined in terms of an individual’s belief about the possibility of gaining or losing something as a result of a particular action (horst et al., 2007; mayer et al., 1995; pavlou, 2003). studies have shown that perceived risk plays an important role in determining citizens’ intention to use e-government services (kumar et al., 2007; lu et al., 2011; schaupp & carter, 2009). in the context of e-government, featherman and pavlou (2003, p.454) defined perceived risk as “the potential for loss in the pursuit of a desired outcome of using an eservice”. perceived risk was integrated in both e-government service elements of our research model, and was defined as the extent to which participants believed using government websites to collect information and to conduct transactions would be risky. therefore, this study hypothesized that perceived risk would have a negative and direct association with the intention to use e-government services for both informational and transactional purposes (hypotheses 7 and 17). 4.1.9 behavioural intention as discussed earlier, this study was conducted on a perceptual basis due to the newness of egovernment concepts in kurdistan and its citizens’ comparative lack of prior experience. thus, behavioural intentions to use informational and transactional e-government were used as dependent variables. this study also hypothesized that there would be a positive relationship between intention to use informational e-government with intention to use transactional egovernment (hypothesis 21). the rationale for this was because informational services are typically introduced before transactional services. therefore, it was decided to examine this relationship more closely to see whether the intention to use government websites for informational access leads the intention to use government websites for transactional purposes as well. 5 research methodology the survey instrument incorporating the constructs discussed above was developed initially in english and then translated into kurdish by an independent translator. this kurdish version was then translated back to english by another translator. the two versions were then compared in order to resolve any differences in meaning or expression. the final kurdish version was then used for the main study (see appendix 1 for the english version of the questionnaire. the kurdish version can be provided on request). this research was conducted in accordance with the ethical research procedures and guidelines provided by the committee for ethics in human research at the university of canberra (approved project 11-100). university students and academic and non-academic staff have been widely used by egovernment researchers in prior studies (al awadhi & morris, 2008; belanger & carter, 2008; carter & belanger, 2004a; featherman & pavlou, 2003; huang et al., 2002; lu et al., 2011; schaub & carter, 2005; schaub & carter, 2009; suki & ramaya, 2010). thus data were collected through a survey questionnaire administered by the first author to undergraduate university students and non-academic university staff. these participants were targeted primarily because of their educated backgrounds and the newness of e-government concepts to ordinary citizens of kurdistan (zhao et al., 2014). although university students and staff 8 australasian journal of information systems ahmad & campbell 2015, vol. 19, research on indigenous ict citizen perceptions of e-government were also believed to be inexperienced with regard to e-government services, previous studies have reported that educated citizens are more likely to become early adopters of e-government services (huang et al., 2002; lu et al., 2011; suki & ramayah, 2010; wangpipatwong et al., 2008). subsequently questionnaires were distributed to students enrolled in engineering, education, administration and law faculties located within salahaddin university. while non-academic staff were recruited from all departments across the university. a total of 670 questionnaires were distributed of which 548 completed and usable questionnaires were obtained; a response rate of 81%. the data were analysed using path analysis in structural equation modelling. 6 results 6.1 participant demographics demographic characteristics of the participants are summarized in table 1. as demonstrated in the table, there were no significant difference in the number of male and female respondents; male 52% and female 48%. since the majority of participants in this study were undergraduate university students almost 80% were younger than 24 years. in terms of experience with computers, surprisingly more than 50% indicated having less than three years of experience. this number was even higher for internet experience (72%). when asked whether they had visited a government website prior to participating in this study, more than half answered ‘no’ and 43% indicated that they had. overall it can be concluded that participants in this study were young with little experience with computers and the internet. demographics frequency percent sex male 285 52 female 263 48 age 15-19 63 11.5 20-24 358 65.3 25-29 83 15.1 30-34 22 4 35-39 12 2.2 40 and above 10 1.8 computer experience less than 1 year 111 20.3 1 to 3 years 209 38.1 4 to 6 years 148 27 7 to 9 years 58 10.6 more than 10 years 22 4 internet experience less than 1 year 272 49.6 1 to 3 years 178 32.5 4 to 6 years 67 12.2 7 to 9 years 21 3.8 more than 10 years 10 1.8 prior website visit yes 236 43.1 no 312 56.9 table 1. demographic profile of participants 6.2 data analysis and results prior to performing statistical analysis, appropriate measures were undertaken to clean and prepare the collected data. the results revealed that the data was both reliable and valid. however, the data violated an important assumption to which the majority of statistical techniques are sensitive; normality. that is, for the majority of variables in this study the data were bimodal. to overcome this problem the data were transformed into dichotomized nominal variables with only two categories (‘0’ or ‘1’, ‘yes’ or ‘no’) using a median split approach. following this, frequencies were produced for responses and the results revealed 9 australasian journal of information systems ahmad & campbell 2015, vol. 19, research on indigenous ict citizen perceptions of e-government that more than 70% of participants indicated having the intention to adopt e-government services (73% informational and 79% transactional). path analysis was conducted to test the overall model; once for informational (figure 3) and once for transactional (figure 4). when performing path analysis sometimes the input model does not fit the data well, therefore modifications need to be applied to the model (berry & welsh, 2010; gefen et al., 2000). this was the case in the current study. thus, modifications were applied to the model until it fit the data which resulted in the removal or addition of some relationships. it is important to mention that demographic characteristics were also included in the path analysis to test their direct relationship with the intention to use, although initially they were not hypothesized to be related. the results indicated that previous experience with government websites (prior visit) was the only important demographic to have a relationship with the intention to use both e-government services. for informational e-government it was both direct and indirect through its positive association with facilitating conditions and trust in government and negative association with perceived risk. for transactional e-government, however, it was only indirect through its positive relationship with compatibility and trust in government. this suggests that experience with government websites is likely play a role in determining users’ e-government adoption behaviour. age of participants was also shown to be positively associated with the intention to adopt transactional e-government suggesting that older participants in this study were more likely to adopt transactional e-government. figure 3 shows that the model containing the significant pathways explained 23% (r²= .23) of the variance in the intention to use informational e-government. facilitating conditions (β= 0.25, p< 0.001), perceived usefulness (β= 0.17, p< .001), trust in government (β= 0.12, p< 0.001), perceived risk (β= -0.12, p< 0.01), and social influence (β= 0.11, p< 0.01) were all directly and significantly associated with the intention to use informational e-government. perceived ease of use was found to be only indirectly associating with the intention to use through its positive and significant relationship with perceived usefulness (β = .37, p< 0.001). perceived information quality was found to have no direct association with the intention to use and its relationship was positively mediated by trust in government (β= 0.35, p< 0.001) and perceived usefulness (β= 0.14, p< 0.01). in addition to its direct association, facilitating conditions was found to also indirectly encourage citizens to visit government websites through its positive association with perceived ease of use (β= 0.31, p< 0.001). thus hypotheses 1, 3, 4, 6, 7, 8 and 9 were supported and hypotheses 2 and 5 were rejected. as for the transactional model, figure 4 shows that the final fitted model explained 24% (r²= .24) of the variance in the intention to adopt transactional e-government. compatibility (β= 0.22, p< 0.001), perceived usefulness (β= 0.20, p< 0.001), social influence (β= 0.15, p< 0.001), trust in government (β= 0.13, p< 0.001) and perceived risk (β= -0.09, p< 0.05) were all directly and significantly associated with the intention to use transactional e-government. facilitating conditions was found insignificant, both directly and indirectly, in associating with the intention to use. trust in the internet was found to only indirectly associate with the intention to use through its positive relationship with trust in the internet and negative relation with perceived risk. in addition to its direct association with the intention to use, compatibility was found to be positively associated with perceived usefulness (β= 0.17, p< 0.001), perceived ease of use (β= 0.33, p< 0.001), trust in the government (β= 0.22, p< 0.001), and trust in the internet (β= .30, p< 0.001). surprisingly the results also suggested that age of the participants could have a positive but weak association with their intention to adopt transactional egovernment services (β= 0.09, p< 0.05). thus, hypotheses 11, 12, 14, 15, 17, 18, 19, 20 and 21 were supported and hypotheses 10, 13 and 16 were rejected. 10 australasian journal of information systems ahmad & campbell 2015, vol. 19, research on indigenous ict citizen perceptions of e-government figure 3. path model for informational e-government figure 4. path model for transactional e-government the final step was to test the relation between intentions to use informational e-government with intentions to use transactional e-government. to do so, the two fitted models were 11 australasian journal of information systems ahmad & campbell 2015, vol. 19, research on indigenous ict citizen perceptions of e-government combined and a path was drawn from intention to obtain information to intention to conduct transactions (figure 5). figure 5. path analysis testing the relationship between intention to seek information with the intention to conduct transactions the analysis was performed and the modification indices revealed that the model did not fit the data well. the modification indices indicated that those variables that were duplicated on both sides of the model could be related. this was due to high collinearity between these repeated items on the two elements of the model. that is, these variables on each side were virtually measuring the same underlying concept. for example, perceived ease of use for both informational and transactional e-government measured the perception about the ease of using government websites to search for information and to conduct transactions. since the objective was to see how much of the variance in the intention to use transactional egovernment was explained by the intention to use informational e-government, the predictor variables for the transactional model were removed and the analysis re-run. in order to fit the model the predictor variables were removed from the transactional model because they measured the same underlying concept within the informational model. consequently the model fit the data and the results revealed that the relationship was significant (β= 0.37, p< 0.001). thus hypothesis 21 was supported suggesting that individuals who have a positive intention to use government websites to obtain information are more likely to also use govenment websites to conduct transactions with government. a summary of the results of hypotheses testing is provided in table 2. 7 discussion this study investigated citizen attitudes to e-government adoption in the kurdistan region of iraq with the aim of providing insight into the factors affecting the adoption of e-government within an emerging democratic state. in doing so, distinction was made between two types of e-government services; informational and transactional. in the conduct of this study it was recognised that not every individual in the region was likely to adopt e-government because of 12 australasian journal of information systems ahmad & campbell 2015, vol. 19, research on indigenous ict citizen perceptions of e-government the reliance on systems that require users to be computer and internet literate. not all individuals in kurdistan have these characteristics. due to internal conflict within the region many people in kurdistan have not had access to the necessary education and technology. thus, in order to generate reliable results, the study focused on those who were most likely to become users of e-government services. to avoid having non-adopters participating in this study, it was decided to focus on university students and non-academic university staff to provide the sample for this study. informational e-government support transactional e-government support h1: perceived usefulness has a positive association. yes h10: perceived ease of use has a positive association. no h2: perceived ease of use has a positive association. no h11: perceived usefulness has a positive association. yes h3: facilitating conditions has a positive association. yes h12: compatibility has a positive association. yes h4: social influence has a positive association. yes h13: facilitating conditions has a positive association. no h5: perceived information quality has a positive association. no h14: social influence has a positive association. yes h6: trust in government has a positive association. yes h15: trust in government has a positive association. yes h7: perceived risk has a negative association. yes h16: trust in the internet has a positive association. no h8: trust in government would indirectly associate with the intention to use informational egovernment through its negative relation with perceived risk. yes h17: perceived risk has a negative association. yes h9: perceived ease of use would indirectly associate with the intention to use informational egovernment through its positive association with perceived usefulness. yes h18: trust in government would indirectly associate with the intention to use transactional egovernment through its negative relation with perceived risk. yes h19: trust in the internet would indirectly associate with the intention to use transactional egovernment through its negative relation with perceived risk. yes h20: perceived ease of use would indirectly associate with the intention to use transactional egovernment through its positive relation with perceived usefulness. yes h21: positive association between intention to use informational e-government and the intention to use transactional e-government. yes table 2. summary of findings given this study background, our findings suggest that the same variables are likely to be important in determining citizens’ intentions to adopt e-government services for both informational and transactional purposes. however, the magnitude of their importance may not be similar across both services, and it is likely to be stronger for transactional egovernment. the results of this study revealed that, except for perceived risk, the magnitude 13 australasian journal of information systems ahmad & campbell 2015, vol. 19, research on indigenous ict citizen perceptions of e-government of the relationships been the independent variables and intention to use was stronger for transactional e-government than for informational e-government. this result supports the argument that transactional e-government is more complex and could be perceived by citizens to involve more risk in comparison to transactional e-government. thus independent variables such as perceived usefulness, perceived ease of use, social influence and trust in government play a stronger role in predicting citizens’ willingness to adopt transactional e-government compared to informational e-government. 8 conclusion the findings from this study have both theoretical and practical importance. theoretically the findings confirm the contention that informational and transactional e-government services must be distinguished in order to more accurately identify the factors associated with the intention to adopt these different types of e-government services and so to make sound conclusions. in addition, this study was the first to investigate e-government adoption in the kurdistan region of iraq which provides an important first step for future research in this region and contributes to the existing e-government literature concerning emerging democratic states. in addition to these theoretical implications, the findings from this study also have implications for the kurdistan regional government. the findings from this current study will assist the authorities in the region to predict whether its citizens are likely to adopt egovernment services, and what types of services they are likely to be more willing to adopt. in addition, the findings provide authorities with important insights into the factors that citizens are likely to consider when deciding to adopt this technology. this will enable the authorities to design and promote more effective e-government programs that align with citizens’ expectations in order to increase citizen uptake. finally, the positive relationship between the intentions to use informational e-government with the intentions to use transactional e-government also provides practical value for authorities in the region. e-government in kurdistan is still in the informational stage. therefore, this finding suggests that designing and promoting the adoption of informational e-government services is likely to lead to greater rates of adoption of transactional egovernment services once available. this study has several limitations which need to be mentioned. the first and probably the most important limitation was that sampling was limited to university students and staff and this group is not entirely representative of the general population in kurdistan. however, at the time of this study, these participants were deemed to have the appropriate level of skill and access to consider using e-government services. future research, however, may include a broader sample. a second limitation was that this study was conducted using a cross sectional approach. future studies thus may consider a longitudinal approach in order to investigate citizens’ perceptions before and after using e-government services. finally, this study was conducted in arbil; the capital city of the region. therefore, including other cities in the region may provide further insights of practical and theoretical importance. references ajzen, i. (1991). the theory of planned behaviour. organizational behaviour and human decision processes, 50, 179-211. al awadhi, s. & morris, a. (2008). the use of the utaut model in the adoption of e-government services in kuwait. proceedings of the 41st hawaii international conference on system sciences, 1-10. almahamid, s., mcadams, a. c., al kalaldeh, t. & al-saeed, m. (2010). the relationship between perceived usefulness, perceived ease of use, perceived information quality, and intention to use e-government. journal of theoretical and applied information technology, 11(1), 30-44. 14 australasian journal of information systems ahmad & campbell 2015, vol. 19, research on indigenous ict citizen perceptions of e-government al-shafi, s. & weerakkody, v. (2010). factors affecting e-government adoption in the state of qatar. european and mediterranean conference on information systems. retrieved from http://v-scheiner.brunel.ac.uk/bitstream/2438/4395/1/c101.pdf. alsaghier, h., ford, m., nguyen, a. and hexel, r. (2009). conceptualising citizen’s trust in e-government: application of q methodology. electronic journal of e-government, 7(4), 295-310. avgerou, c. & walsham, g. (eds.) (2000). information technology in context: studies from the perspective of developing countries. ashgate publishing: aldershot, uk. bandura, a. (1986). social foundations of thought and action: a social cognitive theory. new jersey: prentice-hall, inc. basu, s. (2004). e-government and developing countries: an overview. international review of law, computers & technology, 18(1), 109-132. belanger, f. & carter, l. (2008). trust and risk in e-government adoption. journal of strategic information systems, 17, 165-176. berry, h. l. & welsh, j. a. (2010). social capital and health in australia: an overview from the household, income and labour dynamics in australia survey. social science & medicine, 70, 588-596. bwalya, k. j. & mutula, s. m. (2014). e-government: implementation, adoption and synthesis in developing countries. walter de gruyter: berlin, germany. carter, l. & belanger, f. (2004). citizen adoption of electronic government initiatives. proceedings of the 37th hawaii international conference on system sciences, 1-10. carter, l. & belanger, f. (2005). the utilization of e-government services: citizen trust, innovation and acceptance factors. journal of information systems, 15, 5-25. chatfield, a. t. & alhujran, o. (2009). a cross-country comparative analysis of e-government service delivery among arab countries. information technology for development, 15(3), 151-170. chu, p. y., hsiao, n., lee, f. w. & chen, c. w. (2004). exploring success factors for taiwan's government electronic tendering system: behavioural perspectives from end users. government information quarterly, 21, 219-234. dada, d. (2006). the failure of e-government in developing countries: a literature review. the electronic journal of information systems in developing countries, 26(7), 1-10. davis, f. d. (1989). perceived usefulness, perceived ease of use, and user acceptance of information technology. mis quarterly, 13(3), 319-340. davis, f. d., bagozzi, r. p. & warshaw, p. r. (1989). user acceptance of computer technology: a comparison of two theoretical models. management science, 35(8), 982-1003. deci, e. l. (1972). intrinsic motivation, extrinsic reinforcement, and inequity. journal of personality and social psychology, 22(1), 113-120. delafrooz, n., paim, l. h. j. & khatibi, a. (2011). a research modeling to understand online shopping intention. australian journal of basic and applied sciences, 5(5), 70-77. delone, w. h. & mclean, e. r. (1992). information systems success: the quest for the dependent variable. information systems research, 3(1), 60-95. ebrahim, z. & irani, z. (2005). e-government adoption: architecture and barriers. business process management journal, 11(5), 589-611. featherman, s. m. & pavlou, p. a. (2003). predicting e-services adoption: a perceived risk facets perspective. international journal of human-computer studies, 59, 451-474. 15 australasian journal of information systems ahmad & campbell 2015, vol. 19, research on indigenous ict citizen perceptions of e-government fishbein, m. & ajzen, i. (1975). belief, attitude, intention and behaviour: an introduction to theory and research. reading, ma: addison-wesley. fu, j. r., farn, c. k. & chao, w. p. (2006). acceptance of electronic tax filing: a study of taxpayer intentions. information & management, 43, 109-126. gauld, r., goldfinch, s. & horsburgh, s. (2010). do they want it? do they use it? the demand-side of e-government in australia and new zealand. government information quarterly, 27, 177-186. gefen, d., karahanna, e. & straub, d. w. (2003). trust and tam in online shopping: an integrated model. mis quarterly, 27(1), 51-90. gefen, d., straub, d. w. & boudreau, m.-c. (2000). structural equation modelling and regression: guidelines for research practice. communications of the association for information systems, 4(7), 1-78. gregor, s., imran, a. & turner, t. (2013). a ‘sweet spot’ change strategy for a least developed country: leveraging e-government in bangladesh. european journal of information systems advance online publication 9 july 2013; doi: 10.1057/ejis.2013.14 guha, j. & chakrabarti, b. (2014). making e-government work: adopting the network approach. government information quarterly, 31(2), 327-336. gupta, b., dasgupta, s. & gupta, a. (2008). adoption of ict in a government organization in a developing country: an empirical study. journal of strategic information systems, 17, 140-154. heeks, r. (2002). information systems and developing countries: failure, success, and local improvisations. information society, 18(2), 101-112. horst, m., kuttschreuter, m. & gutteling, j. m. (2007). perceived usefulness, personal experience, risk perception and trust as determinants of adoption of e-government services in the netherlands. computers in human behaviour, 23, 1838-1852. kanat, i. e. & ozkan, s. (2009). explaining citizen adoption of government to citizen services: a model based on theory of planned behaviour (tpb). european and mediterranean conference on information systems. krg it academy (2009). krg prime minister nechirvan barzani’s speech at launch of it academy, 10 september 2009, http://www.krg.org/a/d.aspx?a=31432&l=12&r=268&s=010000, [accessed 12 august 2014]. kamal, m. m., hackney, r. & sarwar, k. (2013). investigating factors inhibiting e-government adoption in developing countries: the context of pakistan. journal of global information management, 21(4), pp. 77-102. kumar, v., mukerji, b., butt, i. & persaud, a. (2007). factors for successful e-government adoption: a conceptual framework. the electronic journal of e-government, 5(1), 6376. lam, w. (2005). barriers to e-government integration. the journal of enterprise information management, 18(5), 511-530. lean, o. k., zailani, s., ramayah, t. & fernando, y. (2009). factors influencing intention to use e-government services among citizens in malaysia. international journal of information management, 29, 458-475. lee, m. c. (2009). factors influencing the adoption of internet banking: an integration of tam and tpb with perceived risk and perceived benefit. electronic commerce research and applications, 8, 130-141. 16 australasian journal of information systems ahmad & campbell 2015, vol. 19, research on indigenous ict citizen perceptions of e-government lin, f., fofanah, s. s. & liang, d. (2011). assessing citizen adoption of e-government initiatives in gambia: a validation of the technology acceptance model in information systems success. government information quarterly, 28, 271-279. lu, y., cao, y., wang, b. & yang, s. (2011). a study on factors that affect users' behavioural intention to transfer usage from the offline to the online channel. computers in human behaviour, 27, 355-364. mayer, r. c., davis, j. h. & schoorman, f. d. (1995). an integrative model of organizational trust, academy of management review, 20(3), 709-734. mcknight, d. h., choudhury, v. & kacmar, c. (2002). developing and validating trust measures for e-commerce: an integrative typology. information systems research, 13(3), 334-359. mkude, c. g. & wimmer, m. a. (2015). studying interdependencies of e-government challenges in tanzania along a pestel analysis. european conference on information systems. completed research papers. paper 135. ndou, v. (2004). e-government for developing countries: opportunities and challenges. the electronic journal of information systems in developing countries, 18(1), 1-24. reddick, c. g. (2005). citizen interaction with e-government: from the streets to servers? government information quarterly, 22(1), 38-57. pavlou, p. a. (2003). consumer acceptance of electronic commerce: integrating trust and risk with the technology acceptance model. international journal of electronic commerce, 7(3), 101-134. prybutok, v. r., zhang, x. & ryan, s. d. (2008). evaluating leadership, it quality, and net benefits in an e-government environment. information & management, 45, 143-152. schaupp, l. c. & carter, l. (2005). e-voting: from apathy to adoption. the journal of enterprise information management, 18(5), 586-601. suki, n. m. & ramayah, t. (2010). user acceptance of the e-government services in malaysia: structural equation modelling approach. interdisciplinary journal of information, knowledge, and management, 5, 395-413. taylor, s. & todd, p. (1995). understanding information technology usage: a test of competing models. information systems research, 6(2), 144-176. tung, l. l. & rieck, o. (2005). adoption of electronic government services among business organizations in singapore. journal of strategic information systems, 14, 417-440. triandis, h. c. (1971). attitude and attitude change. new york: john wiley & sons, inc. vassilakis, c., lepouras, g., fraser, j., haston, s. & georgiadis, p. (2005). barriers to electronic service development. e-service journal, 4(1), 41-63. venkatesh, v. & davis, f. d. (2000). a theoretical extension of the technology acceptance model: four longitudinal field studies. management science, 45(2), 186-204. venkatesh, v., morris, m. g., davis, g. b. & davis, f. d. (2003). user acceptance of information technology: toward a unified view. mis quarterly, 27(3), 425-478. wangpipatwong, s., chutimaskul, w. & papasratorn, b. (2008). understanding citizens continuance intention to use e-government website: a composite view of technology acceptance model and computer self-efficacy. the electronic journal of e-government, 6(1), 55-64. yaghoubi, n. m., kord, b. & shakeri, r. (2010). e-government services and user acceptance: the unified models' perspective. european journal of economics, finance and administrative sciences, (4), 36-49. 17 australasian journal of information systems ahmad & campbell 2015, vol. 19, research on indigenous ict citizen perceptions of e-government wang, y. s. & shih, y. w. (2009). why do people use information kiosks? a validation of the unified theory of acceptance and use of technology. government information quarterly, 26, 158-165. zhao, f., collier, a. & deng, h. (2014). a multidimensional and integrative approach to study global digital divide and e-government development. information technology & people, 27(1), 38-62 18 australasian journal of information systems ahmad & campbell 2015, vol. 19, research on indigenous ict citizen perceptions of e-government appendix 1: survey instrument (english version) e-government description e-government: e-government the usage of information and communication technologies (ict) such as the internet to deliver government services to citizens. types of e-government services: there are two basic types of services in e-government; informational e-government and transactional e-government. informational e-government: is the use of government websites by citizens to obtain information that they need or to download forms to apply for a service. for example, information about weather forecast, education related information, job notifications, the location of a government agency, the contact details for government employees, etc. this type of e-government services does not require the users to provide personal or financial information to the government agency. transactional e-government: is the use of the internet to conduct transactions with government agencies online. for example, including online submission of hajj applications, registering child birth, registering death, renewing driver license, paying bills online, paying fines, applying for jobs etc. in this type of e-government services, in order to receive the service users must provide either personal information or financial information or both such as name, date of birth; address, bank details, phone number etc, and this information will be accessible by all government agencies. informational e-government already exists in kurdistan as the majority of government agencies have launched their websites which allow users to obtain information, and in some instances to download forms. transactional e-government, on the other hand, does not yet exist in kurdistan. however, some agencies have introduced some limited transactional services. for example, the ministry of higher education allows users to access information, to apply for higher degree courses and to apply for jobs electronically. the government has planned to develop its e-government services according to a 5-year plan (2011-2015). this questionnaire consists of 3 parts. in the first part seeks demographic questions about yourself such as age, gender etc. in the second part, questions are asked regarding the use of e-government for information access only. in the third part, similar questions are asked regarding transactional e-government. note that questions may sound similar, but they are different in wording so please do not be confused. some examples of government websites available in kurdistan http://www.hawlergov.org/kur/subject.aspx?id=468 (parliament) http://www.mojkurdistan.com/kurdish_malper.aspx (ministry of justice) http://www.zhenga.net/english/index.php (ministry of environment) http://www.krgelectric.org (ministry of electricity) http://www.krp.org/english/default.aspx (kurdistan region presidency) http://www.krg.org/?lngnr=12&smap=01010000 (kurdistan regional government) http://www.moc-krg.com/english (ministry of transportation & communication) http://www.mhe-krg.org (ministry of higher education & scientific research) 19 australasian journal of information systems ahmad & campbell 2015, vol. 19, research on indigenous ict citizen perceptions of e-government section 1: please tick the most appropriate box for the following items. 1. gender: male female 2. what age group are you in? (a). 15-19 (b). 20-24 (c). 25-29 (d). 30-34 (e). 35-39 (f). 40 and above 3. what course are you enrolled in (if you are student)? or what is your educational level (if you are university employee)? (a). primary school qualification (b). high school qualification (c). undergraduate degree (d). post-graduate certificate or diploma (e). masters qualification (f). doctorate (h).other (please specify) 4. how many years of experience do you have with computers? (a). less than 1 year (b). 1 to 3 years (c). 4 to 6 years (d). 7 to 9 years (e). 10 and more 5. how many years of experience do you have with the internet? (a). less than 1year (b). 1 to 3 years (c). 4 to 6 years (d). 7 to 9 years (e). 10 and more 20 australasian journal of information systems ahmad & campbell 2015, vol. 19, research on indigenous ict citizen perceptions of e-government 6. have you ever visited a government (ministries) website prior to this survey? yes no 7. if yes, please explain what was the purpose of visit? ........................................................................................................................................................ ........................................................................................................................................................ ........................................................................................................................................................ section 2: in this section we would like to know your opinion about informational egovernment which is using government (ministries) websites to obtain information only. note that in this stage e-government can be used only to obtain information and to download forms, which does not require you to provide any personal and financial information. if you have visited government websites please answer the questions based on your experience. please circle the degree to which you agree with the following statements: perceived usefulness strongly disagree disagree neutral agree strongly agree 1 i find government websites useful in my information access 1 2 3 4 5 2 using government websites enable me to obtain information i need more quickly 1 2 3 4 5 3 using government websites increase my ability to get timely information 1 2 3 4 5 4 government websites enable me to access information i need any time anywhere 1 2 3 4 5 5 overall, i believe that government websites are useful in my information access 1 2 3 4 5 21 australasian journal of information systems ahmad & campbell 2015, vol. 19, research on indigenous ict citizen perceptions of e-government perceived ease of use strongly disagree disagree neutral agree strongly agree 1 my interaction with government websites for information access would be clear and understandable 1 2 3 4 5 2 it would be easy for me to navigate through government websites and find the information i need 1 2 3 4 5 3 it would be easy for me to become skilful at using government websites to find information and download forms 1 2 3 4 5 4 i would find government websites to be flexible to interact with 1 2 3 4 5 5 overall, i believe that government websites are easy to use for obtaining information 1 2 3 4 5 perceived information quality strongly disagree disagree neutral agree strongly agree 1 the information provided on government websites are accurate 1 2 3 4 5 2 the information provided on government websites are complete 1 2 3 4 5 3 the information provided on government websites are precise 1 2 3 4 5 4 government websites provide relevant information that i need 1 2 3 4 5 5 government websites provide timely information on their websites 1 2 3 4 5 6 overall, i believe that government websites provide information according to citizens need 1 2 3 4 5 22 australasian journal of information systems ahmad & campbell 2015, vol. 19, research on indigenous ict citizen perceptions of e-government facilitating conditions strongly disagree disagree neutral agree strongly agree 1 i have the resources (hardware & software) necessary to use government websites to obtain information 1 2 3 4 5 2 i have the skills necessary to use government websites to obtain information i need 1 2 3 4 5 3 i have the knowledge necessary to access government websites to obtain information i need 1 2 3 4 5 4 overall, i believe that i have the conditions necessary to access government websites and obtain information 1 2 3 4 5 social influence strongly disagree disagree neutral agree strongly agree 1 people who influence my behaviour will have an impact on my decision to use government websites to access information 1 2 3 4 5 2 people who are important to me will influence my decision to use informational e-government 1 2 3 4 5 3 my friends and colleagues will have an influence on my decision to use informational egovernment 1 2 3 4 5 4 using government websites to access information by a large proportion of people around me will influence my intention to use it 1 2 3 4 5 23 australasian journal of information systems ahmad & campbell 2015, vol. 19, research on indigenous ict citizen perceptions of e-government trust in government strongly disagree disagree neutral agree strongly agree 1 in my opinion, government agencies can be trusted in providing good information on their websites 1 2 3 4 5 2 i believe that government agencies are honest in providing information on their websites 1 2 3 4 5 3 i think that government agencies in kurdistan are trustworthy 1 2 3 4 5 4 overall, i think that i can trust government agencies as the provider of governmental information 1 2 3 4 5 perceived risk strongly disagree disagree neutral agree strongly agree 1 in my opinion the decision to use government websites to obtain information is a risky decision 1 2 3 4 5 2 i believe that using government websites to gather information is risky 1 2 3 4 5 3 overall, i believe using egovernment for informational purposes is risky 1 2 3 4 5 behavioural intention to get information strongly disagree disagree neutral agree strongly agree 1 i have the intention to use government websites to obtain information 1 2 3 4 5 2 i predict that i will continue using government websites to obtain information 1 2 3 4 5 3 i expect that i will use and continue to use government websites to obtain information 1 2 3 4 5 24 australasian journal of information systems ahmad & campbell 2015, vol. 19, research on indigenous ict citizen perceptions of e-government section 3: in this section we would like to understand about your opinion regarding transactional e-government. please rate the following questions on a scale of 1 to 5 that best describes your opinion about e-services (conducting transaction with government online). please note that this section is concerned with using transactional e-government, which is using the internet to transact with government where you must provide personal or financial information or both to government agencies in order to receive the service. for example, applying for jobs, renewing driver licence, applying for a passport, applying for marriage certificate, paying bills electronically. please circle the degree to which you agree with the following statements: perceived usefulness strongly disagree disagree neutral agree strongly agree 1 i believe that using transactional e-government will enable me to complete transactions with government more quickly 1 2 3 4 5 2 transactional egovernment will increase my effectiveness by lowering travelling and queuing time 1 2 3 4 5 3 using transactional egovernment will enable me to access more government services than would otherwise possible 1 2 3 4 5 4 using the internet to interact with government will enable me to do business with the government anytime not limited to regular business hours. 1 2 3 4 5 5 overall, i would find transactional e-government useful in my transaction with government 1 2 3 4 5 25 australasian journal of information systems ahmad & campbell 2015, vol. 19, research on indigenous ict citizen perceptions of e-government perceived ease of use strongly disagree disagree neutral agree strongly agree 1 learning to use e-government to transact with government will be easy for me 1 2 3 4 5 2 my interaction with government websites for transaction will be clear and understandable 1 2 3 4 5 3 it will be easy for me to become skilful at using government websites to conduct transaction with government 1 2 3 4 5 4 i will find transactional egovernment services to be flexible to interact with 1 2 3 4 5 5 overall, i believe that using egovernment services for transaction with government will be easy for me 1 2 3 4 5 social influence strongly disagree disagree neutral agree strongly agree 1 people who influence my behaviour will have an impact on my decision to use government websites to transact with government 1 2 3 4 5 2 people who are important to me will influence my decision to use e-government services to transact with government 1 2 3 4 5 3 my friends and colleagues will influence my decision to use egovernment services to transact with government 1 2 3 4 5 4 using e-government for doing transaction with government by a large proportion of people around me will influence my decision to use it 1 2 3 4 5 26 australasian journal of information systems ahmad & campbell 2015, vol. 19, research on indigenous ict citizen perceptions of e-government facilitating conditions strongly disagree disagree neutral agree strongly agree 1 i believe i have the resources (hardware & software) necessary to access government websites for conducting transactions with government 1 2 3 4 5 2 i think i have the knowledge necessary to use government websites for conducting transactions with government 1 2 3 4 5 3 i believe i have the skills necessary to use t e-government services for conducting transaction with government agencies 1 2 3 4 5 4 overall, i believe that i have the conditions necessary to use state government websites to conduct transactions with government agencies online 1 2 3 4 5 compatibility strongly disagree disagree neutral agree strongly agree 1 i think using the internet will fit with the way that i like to transact with government 1 2 3 4 5 2 using the internet to transact with government agencies is what i like to do 1 2 3 4 5 3 using the internet to transact with government agencies will fit into my lifestyle 1 2 3 4 5 4 i prefer face-to-face transaction with government agencies 1 2 3 4 5 27 australasian journal of information systems ahmad & campbell 2015, vol. 19, research on indigenous ict citizen perceptions of e-government trust in the internet strongly disagree disagree neutral agree strongly agree 1 i think the internet has enough safeguards to make me feel comfortable using it to transact with state government agencies online 1 2 3 4 5 2 i feel assured that legal and technological structures adequately protect me from problems on the internet 1 2 3 4 5 3 in general, the internet is now a robust and safe environment in which to transact with state government agencies 1 2 3 4 5 trust in government strongly disagree disagree neutral agree strongly agree 1 i think i can trust state government agencies in dealing with my personal and financial information 1 2 3 4 5 2 state government agencies can be trusted to carry out online transactions faithfully 1 2 3 4 5 3 in my opinion, state government agencies are trustworthy 1 2 3 4 5 4 overall, i trust state government agencies to keep my best interest in mind when doing online transactions 1 2 3 4 5 perceived risk strongly disagree disagree neutral agree strongly agree 1 using e-government to transact with government agencies may cause my personal and financial information to be stolen 1 2 3 4 5 2 the decision of whether to use transactional e-government services is risky 1 2 3 4 5 3 in general, i believe conducting transactions with government online is risky 1 2 3 4 5 28 australasian journal of information systems ahmad & campbell 2015, vol. 19, research on indigenous ict citizen perceptions of e-government behavioural intention to do transactions strongly disagree disagree neutral agree strongly agree 1 i intend to use e-government to do transaction with government agencies when it becomes available 1 2 3 4 5 2 i plan to use e-government to transact with government when it becomes available 1 2 3 4 5 3 using e-government services over the web to transact with government is something that i would do 1 2 3 4 5 thank you for your participation in this survey copyright: © 2015 ahmad & campbell. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. 29 http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 study context 3 theoretical framework 3.1 information and transaction based e-government 3.2 intention to adopt e-government services 3.3 technology adoption models 4 research model 4.1 hypothesis development 4.1.1 perceived usefulness 4.1.2 perceived ease of use 4.1.3 facilitating conditions 4.1.4 social influence 4.1.5 compatibility 4.1.6 perceived information quality 4.1.7 trust in the government and the internet 4.1.8 perceived risk 4.1.9 behavioural intention 5 research methodology 6 results 6.1 participant demographics 6.2 data analysis and results 7 discussion 8 conclusion paper title (paper title style) australasian journal of information systems radoll & campbell 2015, vol. 19, research on indigenous ict editorial editorial for the indigenous use of information and communication technologies section peter radoll university of newcastle peter.radoll@newcastle.edu.au john campbell university of canberra john.campbell@canberra.edu.au the rapid rise in the use of information and communication technologies (icts) by indigenous people and communities is an intriguing development with implications for indigenous cultures. icts have a critical role to play in supporting indigenous selfdetermination, strengthening indigenous identity, and providing a strong indigenous political voice. icts also have potential to address indigenous disadvantage through connecting business and government to indigenous communities. however, potential negatives also exist. ict use can give rise to negative consequences by diminishing cultural meanings and exposing individuals to new kinds of online-related risks. the initial idea for this special section arose during a panel discussion on indigenous peoples and information systems at the 2014 australasian conference on information systems. from that discussion it emerged that the use of icts had become an everyday, typical activity, utilised by indigenous people and communities for a range of purposes. in so doing, icts provide indigenous people with ways to circumvent limitations imposed by governments and mass media. widespread ict usage has allowed indigenous peoples not only to disseminate their views, but also to build grassroots alliances with similarly minded groups. ict has the ability to bring together indigenous groups globally in an online community that constitutes an informal nation, which exists outside of government regulation and controls, and builds solidarity. for this special section of ajis, we encouraged submissions that focused on aspects of indigenous use of icts in workplaces, educational settings, community, and specific personal and cultural applications. in so doing we wanted to examine important properties of information systems use unique to indigenous peoples, and to highlight the ways new technologies enable creative engagement and a means for indigenous people to represent themselves with less regulatory interference. in this special issue we present three papers that address a variety of issues related to icts and indigenous peoples. the first paper represents a social media study in which carson, farrelly and borthwick examine how aboriginal people utilise and interact on social media, and how these technologies can assist with suicide prevention strategies. they found that aboriginal people are engaging with facebook to both seek and offer help for issues relating to suicide and selfharm. these findings have significant implications for the design and implementation of effective suicide prevention programs for indigenous australians. the second paper by ashraf, grunfeld, and quazi examines how the use of icts affects the livelihoods of an indigenous community in bangladesh. the findings demonstrate that icts have a positive impact by improving the livelihoods of indigenous individuals and communities. the third paper examines attitudes to e-government adoption in the kurdistan region of iraq and thus provides insight into the factors affecting the adoption of e-government by an indigenous ethnic minority in an emerging democratic state. ahmed and campbell distinguish between informational and transactional e-government services and their findings have implications for the implementation of e-government services for indigenous peoples of the middle east. research on the use of icts by indigenous peoples is growing in prominence, yet the area overall is still very under theorised and much more research is required to better understand both the challenges and the opportunities that icts brings to the indigenous space. 1 australasian journal of information systems radoll & campbell 2015, vol. 19, research on indigenous ict editorial copyright: © 2015 radoll & campbell. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. 2 http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems kumar, sachan & kumar 2020, vol 24, research article service delivery and perceived value in internet banking adoption 1 impact of service delivery system process and moderating effect of perceived value in internet banking adoption rajiv kumar amrita school of business coimbatore india 1331.rajiv@gmail.com amit sachan indian institute of management ranchi india ritu kumar mukhyamantri laghu evam kutir udyam vikas board jharkhand, india abstract this study identifies the impact of internet banking service delivery system (ibsds) process on consumer behaviour; the mediating effect of customer satisfaction on the relationship between ibsds process and behavioural intention; and the moderating effect of the customer’s perceived value. the study uses a quantitative method using the data collected from 416 respondents. mediated regression is used to examine the mediating effect of customer satisfaction. spss process v2.16.3 is employed to analyse the moderating effect of the perceived value. the results indicate that ibsds process impacts customer satisfaction and behavioural intention. further, customer satisfaction plays a partial mediating role between the association of ibsds process and behavioural intention. this study provides evidence that the technological capabilities embedded in the bank website processes are an important factor in determining customer satisfaction and ultimately behavioural intention. the study also reveals that perceived value moderates the relationship between ibsds process and customer satisfaction, and between customer satisfaction, and, behavioural intention. it also adds to the existing knowledge on the adoption of internet banking literature. academicians and information systems researchers may use these findings for further research. financial service providers or related agencies may consider the attributes of the ibsds process (e.g., navigation, searching, transacting, etc.) while designing their internet banking services to provide enhanced customer experiences. the paper also highlights the research limitations and scope for future research. keywords internet banking, process, satisfaction, perceived value, behavioural intention, mediation, moderation, india 1 introduction advances in information and communication technology (ict) have radically changed the service delivery channels used by the financial services industry (al-somali, gholami, & clegg, 2009). internet banking has revolutionized banking services (kesharwani & bisht, 2012) by providing customers round-the-clock availability and ease of transactions, and thus, ensuring that they are no longer limited to branch operating hours for services (al-somali et australasian journal of information systems kumar, sachan & kumar 2020, vol 24, research article service delivery and perceived value in internet banking adoption 2 al., 2009). internet banking, also known as online banking or electronic banking (e-banking), refers to the use of internet as a remote delivery channel for providing customers services, such as opening a deposit account or transferring funds from one account to another (dhananjay & chandra, 2015; szopiński, 2016). the internet banking system typically connects to its customers and is part of the core banking system. information and communication technologies (icts) have influenced the way customers interact with banks. unlike traditional banking services, internet banking services do not require customer visits to the branch. internet banking services also have a relative advantage over brick-and-mortar banks in terms of “timeliness and accuracy of information flow” that minimize information latency in an intense decision-making environment (kesharwani & bisht, 2012). banks can leverage the internet to attract new customers, retain their existing ones and provide enhanced services, leading to higher customer satisfaction (cs). the banks also gain in terms of higher market share and are able to reduce their administrative and operational costs. more importantly, they can improve their competitive positions (al-somali et al., 2009; khalfan, alrefaei, & alhajery, 2006). internet banking was incepted in the 80s (dhananjay & chandra, 2015), however, it marked its presence in india in the mid-nineties (kesharwani & bisht, 2012). currently, most brickand-mortar banks have evolved to incorporate internet banking capabilities (kesharwani & bisht, 2012) in their bouquet of services. a report by facebook and the boston consulting group (encashing on digital: financial service in 2020, 2017), finds that the demonetization by the government of india in the year 2016 provided a major boost to online banking services, motivating people to adopt cashless channels to conduct their transactions. another report (encashing on digital: financial service in 2020, 2017), estimated that the total urban banking users in india were expected to grow from 45 million to more than 150 million by 2020. despite the various benefits offered by internet banking and the policy initiatives taken by the government and financial service providers, it is seen that financial industry in india has yet to harness the new channel to their advantage (encashing on digital: financial service in 2020, 2017). a report drafted by facebook and the boston consulting group (encashing on digital: financial service in 2020, 2017) asserts that “although digital influence is high during research and selection, adoption of internet banking remains low.” according to the report, only 12% of banking users actively use internet banking facilities. more than half of the consumers are largely dissatisfied with their internet banking and mobile apps experience (encashing on digital: financial service in 2020, 2017). people are hesitant/ reluctant to conduct banking/financial transactions through their banks’ websites (kesharwani & bisht, 2012). therefore, it is important to understand the factors that affect consumer behaviour while adopting internet banking. it becomes necessary to study the acceptance or intention to use internet banking through a user interface and glean insights that would help banks to review and fine-tune their internet banking initiatives and achieve higher customer acceptability. the web interface has several characteristics that are uncommon to brick-and-mortar banks. service providers can differentiate themselves and create value for customers to enact superior service experiences during the service delivery process (zeithaml, berry, & parasuraman, 1991). in general, the objective of a process strategy is to create a process that can produce an offering that meets customer requirements within cost and managerial constraints (jay, barry, chuck, & amit, 2017). interaction with the customer often affects process performance adversely. but a service by its very nature implies that some interaction and customization is australasian journal of information systems kumar, sachan & kumar 2020, vol 24, research article service delivery and perceived value in internet banking adoption 3 needed (jay et al., 2017). given that the customer’s unique desires tend to play havoc with a process, the more the manager designs the process to accommodate these special requirements, the more effective and efficient it will be (jay et al., 2017). technology adoption model (tam) by davis (1989) posits that the effect of external influences, such as system design characteristics and individual differences on the user intention is mediated by their perception of the ease-of-use and usefulness of the new system (kesharwani & bisht, 2012). although academic research (e.g., khalfan, alrefaei and al-hajery, 2006; al-somali, gholami and clegg, 2009; kaura, 2013; bashir and madhavaiah, 2015; singh and malhotra, 2015; szopiński, 2016) has examined the dynamics of user intention to adopt internet banking services, the subject of user service encounters while using technological interfaces, such as searching for information or conducting transactions through banking websites has not received adequate attention. while some research has been conducted on website design to improve user satisfaction, most studies, however, remain focused on factors such as information content, download speed, and information presentation (alawneh, al-refai, & batiha, 2013; papadomichelaki & mentzas, 2012). surveys of online users consistently indicate that a big percentage of the population is not satisfied with the interaction with the financial institutions (encashing on digital: financial service in 2020, 2017). therefore, whether the service can achieve what users want in a bank website constitutes an important area of study. more research is needed to better understand how services delivered through technological interfaces such as the web affect user evaluation of the service value. besides, there is a need to understand how banks can manage their technology-based service processes for higher user satisfaction and ultimately to influence user behavioural intention (bi). extant literature identifies that service process plays a critical role in influencing service quality and customer satisfaction (ba & johansson, 2008; luu, hau, ngo, bucic, & cuong, 2016; sachan, kumar, & kumar, 2018). however, to the best of our knowledge, no study has attempted to examine the effect of the internet banking service delivery system (ibsds) process on the adoption of internet banking. specifically, a service process is conceptualized as a configuration of technological capabilities through which service providers respond to user needs (ba & johansson, 2008), and perceived ibsds process refers to the user’s view of how the banking website’s technological capabilities deliver service processes. hence, a strong theoretical gap exists in the literature. this study aims to fill this gap by exploring the impact of the ibsds process on the adoption of internet banking. we build on theories from service management and propose viewing the interface between user and service provider (i.e., bank website) to identify and explain a possible determinant of internet banking user adoption. studies in the context of internet banking have focused on the relationship between customer satisfaction and behavioural intention (bashir & madhavaiah, 2015; khalfan et al., 2006; laukkanen, 2007; sharma & govindaluri, 2014; singh & malhotra, 2015; tan & teo, 2000). the relationship seems almost intuitive. however, the strength of the relationship between satisfaction and behavioural intention in internet banking may vary significantly under different conditions. e-commerce literature highlights the different conditions in which the relationship between customer satisfaction and behavioural intention varies (anderson & srinivasan, 2003; chang & wang, 2011; chang, wang, & yang, 2009). one such condition is the customer perceived value (pv) which has gained much attention (chang & wang, 2011; chang et al., 2009). it is found to play an important role in predicting purchase behaviour and enabling banks to achieve sustainable competitive advantage (zeithaml, 1988; bolton and drew, 1991; mcdougall and levesque, 2000; mishra, 2009). as a website is an important australasian journal of information systems kumar, sachan & kumar 2020, vol 24, research article service delivery and perceived value in internet banking adoption 4 interface in internet banking, it would be interesting to determine if perceived value has the potential to influence the relationship between customer satisfaction and behavioural intention. we also find that existing literature has not adequately examined the moderating effect of perceived value on the relationship between the ibsds process and customer satisfaction. examining these relationships will not only strengthen the theoretical foundations of the proposed research model in measuring the demand-side perspective of customer satisfaction and behavioural intention but also allow policymakers to understand the extended facets of customer satisfaction and their behavioural intention based on the delivery process of the internet banking services provided by the financial service providers. summarizing, the study has three objectives: first, to develop and test the research hypotheses linking together ibsds process, customer satisfaction, and customer behavioural intention; second, to examine the mediating effect of customer satisfaction on the relationship between ibsds process and consumer behavioural intention; and third, to examine the moderating effect of consumer perceived value on the relationship between ibsds process and customer value, and the relationship between customer satisfaction and behavioural intention. 2 theoretical framework a banking website is an integrated system of banking service delivery and is critical for its value creation strategy (gupta & suri, 2017). it is argued that the more value a customer receives during the service delivery process, the more positive emotions it evokes toward the service provider, thereby strengthening the emotional attachment between the two parties (luu et al., 2016; zeithaml et al., 1991). in view of the lack of literature on the ibsds process and its relationship with internet banking adoption, this study attempts to develop insights into these aspects by reviewing relevant literature. this research is based on the reformation of attitude theory by bagozzi (1992). it further introduces the ibsds process and also incorporates customer satisfaction and behavioural intention as the research constructs. bagozzi's (1992) model suggests that an initial service evaluation (i.e., appraisal) leads to an emotional reaction which, in turn, drives behaviour. we also adopt customer perceived value as our fourth research construct to test the moderating effect. the model is presented in figure 1. each construct is elaborated as follows and the model’s conceptual development and research hypotheses are presented herein. 2.1 ibsds process as discussed in the previous section, perceived ibsds process refers to customer views on how a banking website’s technological capabilities deliver service processes. service management literature identifies service process as a critical factor influencing service quality (ba & johansson, 2008). roth and jackson (1995) in their research provide evidence that the processes of a service delivery system have a more significant impact on service quality than people capabilities. people capabilities represent the stock of individual knowledge and skills possessed by the employees who interact with the customers (roth & jackson, 1995). a study by sachan, kumar and kumar (2018) in the e-government context shows that service delivery system process significantly affects user satisfaction. they argue that as the service delivery system process improves, it allows the citizen to make more decisions and choices in the way they interact with the bank and conduct their transactions and related activities. in an internet banking environment, the main channel of service delivery is the bank’s website. therefore, australasian journal of information systems kumar, sachan & kumar 2020, vol 24, research article service delivery and perceived value in internet banking adoption 5 we focus on ibsds from the web process point of view. although the online service delivery system process in the banking environment is of crucial importance, previous studies have omitted using it as a variable; hence, it is necessary to understand its role in internet banking behaviour. 2.2 customer satisfaction customer satisfaction refers to the overall positive or negative feeling about the net value of services received from a supplier (woodruff, 1997). customer satisfaction has been consistently used over the time in literature and conceptualized as an evaluation of emotion (chang et al., 2009; hunt, 1977). it is today the primary goal for most service firms (jones & sasser, 1995; mcdougall & levesque, 2000). improved profits, lower marketing expenditures, and positive word-of-mouth are the outcomes of increasing consumer satisfaction and retention (heskett, jones, loveman, sasser, & schlesinger, 1994; mcdougall & levesque, 2000). a dissatisfied customer is more likely to search for information on alternatives and yield to competitor overtures than a satisfied one (anderson & srinivasan, 2003). anderson and srinivasan (2003) argue that “a dissatisfied customer is more likely to resist attempts by his or her current retailer to develop a closer relationship and more likely to take steps to reduce dependence on that retailer.” meuter et al. (2000) find that process failure and poor process design are two major factors that lead to customer dissatisfaction in a technology-based service setting. 2.3 behavioural intention behavioural intention measures the future propensity of a customer to access a banking website (chang et al., 2009). consumers and organizations engage in behaviours that build relationships before, during and after the service encounters (chang & polonsky, 2012). such relationships are important for repeat consumption (chang & polonsky, 2012). service marketers often focus on activities that have the potential to build long-term relationships with consumers (youngdahl & kellogg, 1997). a number of studies have included behavioural intention as an outcome in service quality models (e.g., cronin, brady and hult, 2000; chang and polonsky, 2012; bressolles, durrieu and deans, 2015). 2.4 customer perceived value perceived value refers to “the consumers’ overall assessment of the utility of a product based on perceptions of what is received and what is given” (valarie a zeithaml, 1988). perceived value has attracted interest in the literature because it is seen to be a key predictor of customer satisfaction and loyalty (bressolles et al., 2015; chang & wang, 2011; mcdougall & levesque, 2000; wang, 2014; yang & peterson, 2004). most studies have examined customer value in the context of offline rather than online consumer behaviour (bressolles et al., 2015). although customer value in the internet banking environment is of crucial importance, existing studies have not given it attention in the internet banking context; hence, it becomes necessary to understand the role of customer perceived value in internet banking behaviour. 2.5 proposed hypotheses 2.5.1 ibsds process and customer satisfaction the importance of service delivery process aspects and outcome is clearly evident in the ongoing scholarly interest devoted to them (ba & johansson, 2008; grönroos, 1982; luu et al., 2016; sachan et al., 2018). it is argued that service process plays a greater role in determining australasian journal of information systems kumar, sachan & kumar 2020, vol 24, research article service delivery and perceived value in internet banking adoption 6 the overall customer satisfaction. studies (e.g., danaher & mattsson, 1994; luu et al., 2016) highlight that customers compare the time, cost and effort spent on the service during the service delivery process. it is also highlighted that positive experiences with the service delivery process enhance the cognitive strength of the relationship of the customer with the service provider (luu et al., 2016). ba and johansson (2008) state that an improved electronic service delivery system process allows the customer to make more decisions and choices in the service process, which leads to increased web interactivity. they also find that the electronic service delivery system process has a positive influence on customer satisfaction. the above arguments lead us to think that it is possible that technology and the banking processes embedded in the ibsds may have a significant impact on the adoption of internet banking services. sachan, kumar and kumar (2018) add service delivery system process as a construct in their adoption model in the e-government context and illustrate that it is one of the service dimensions factored into the consumer’s satisfaction judgment. luu et al. (2016) illustrate that a customer relationship can be affectively strengthened through positive experiences and interactions during a service process. these arguments suggest that ibsds process is likely to affect customer satisfaction, and in turn, serve as a better predictor of their behavioural intention. this leads us to formulate the first research hypothesis: h1: the ibsds process is positively associated with customer satisfaction. 2.5.2 customer satisfaction and behavioural intention many researchers (e.g., chang and polonsky, 2012; lekhawipat, 2014; bressolles, durrieu and deans, 2015) have focused on the relationship between customer satisfaction and behavioural intention. customer satisfaction is of crucial importance in the online environment. with satisfied customers, online stores can achieve higher customer retention rates, higher profitability and long-term growth (chen, ling, ying, & meng, 2012). anderson and srinivasan (2003) illustrate the impact of customer satisfaction on loyalty in the context of ecommerce. they define e-satisfaction in the e-commerce context as “the contentment of the customer with respect to his or her prior purchasing experience with a given e-retailer.” a large number of marketing researchers have incorporated satisfaction and behavioural intention as important variables in their theoretical framework and illustrated that service quality leads to satisfaction (bressolles et al., 2015; caruana, money, & berthon, 1998; chang et al., 2009; dabholkar, 2000), which, in turn, influences behaviour intention (chang et al., 2009; chang & polonsky, 2012; zeithaml, berry, & parasuraman, 1996). the above arguments highlight that satisfaction positively affects customer behavioural intention to use internet banking, and hence, we propose the following hypothesis: h2: customer satisfaction is positively associated with customer behavioural intention. 2.5.3 customer satisfaction, ibps process and behavioural intention customer satisfaction has also been found to be a mediator of the association between service quality and behavioural intention (bloemer, ruyter, & peeters, 1998). in taiwanese leisure setting, chang & polonsky (2012) demonstrate the indirect and positive association between consumer perceptions of the convenience dimensions (benefit and post-benefit) and their behavioural intention towards the service provider, mediated by customer satisfaction. as discussed, the ibsds process affects customer satisfaction, which, in turn, affects customer behavioural intention, thus, indicating that customer satisfaction may have a mediating effect in their relationship. therefore, based on past literature, mediating effect of customer australasian journal of information systems kumar, sachan & kumar 2020, vol 24, research article service delivery and perceived value in internet banking adoption 7 satisfaction would be expected between the ibsds process and behavioural intention in the internet banking context. hence, we formulate the following hypothesis: h3. customer satisfaction mediates the effect of ibsds process on behavioural intention. 2.5.4 perceived value, ibsds process and customer satisfaction extant literature (e.g., cronin, brady and hult, 2000) has tested the indirect relationship that exists between service quality and behavioural intentions via both service value and satisfaction. literature (e.g., caruana, money and berthon, 1998) also illustrates that perceived value has a moderation effect on the link between service quality and satisfaction. it is argued that high satisfaction can be achieved with a lower quality if value received is high (caruana et al., 1998). as discussed, service process influences service quality. it therefore becomes important to explore if perceived value moderates the link between ibsds process and customer satisfaction. extant literature (e.g., ba and johansson, 2008; sachan, kumar and kumar, 2018) provides support for a strong direct link between service delivery process and satisfaction but makes no such claim for the moderating effect of perceived value. therefore, this research seeks to explore whether value moderates the relationship between ibsds process and customer satisfaction. hence, our next hypothesis is: h4: customer perceived value has a significant moderating effect on the relationship between ibsds process and customer satisfaction. 2.5.5 perceived value, customer satisfaction, and behavioural intention researchers have established a positive relationship between perceived value and behavioural intention (bressolles et al., 2015; chang et al., 2009; cronin et al., 2000). extant literature (cronin et al., 2000; kim, 2014) has examined the indirect relationship between service value and behavioural intentions via satisfaction. past studies demonstrate that perceived value contributes to customers’ loyalty for an e-business and reduces their need to seek alternative service providers or alternative modes for accessing services. when the perceived value is low, customers will be more inclined to switch to an alternate mode or competing businesses in order to increase perceived value, thus leading to a decline in loyalty (chang et al., 2009). even satisfied customers are unlikely to patronize an e-business if they feel that they are not getting the best value (chang et al., 2009). instead, they will seek out other sellers in an ongoing effort to find a better value (anderson & srinivasan, 2003). they may also switch to alternate modes of accessing services such as visiting the bank branch to find better value. the relationship between customer satisfaction and loyalty appears the strongest when customers feel that their current e-business vendor provides higher overall value than that offered by competitors (chang et al., 2009). chang et al. (2009) illustrate the moderating effect of customer perceived value on customer satisfaction and loyalty in an e-commerce context and show that low satisfied customers with high perceived value would have higher customer loyalty than high satisfied customers with low perceived value. based on this argument, this study proposes that customer perceived value may have a significant moderating effect on the relationship between customer satisfaction and their behavioural intention in the internet banking context. this leads to the fifth research hypothesis: h5: customer perceived value has a significant moderating effect on the relationship between customer satisfaction and behavioural intention. australasian journal of information systems kumar, sachan & kumar 2020, vol 24, research article service delivery and perceived value in internet banking adoption 8 fig. 1. research model and hypotheses 3 methodology for this study, which focuses on internet banking adoption, we surveyed the customers who use internet banking services. a conceptual model (see fig. 1) has been proposed based on the detailed literature review. most of the literature was reviewed at the initial stage. however, the activity was continued throughout the study for refinement at various stages. the research was conducted between december 2017 and may 2018 in india. spss 20 was used to analyse the data. 3.1 measurement scale to test the model in the most realistic way possible, the study was conducted through a survey (a self-administered questionnaire) of a broad diversity of internet banking customers in several communities and regions. to test the proposed model, validated items were adapted from previous studies. the measure for behavioural intention was drawn from yang and peterson (2004). customer behavioural intention has three survey items that are common in measuring future intention as indicators of actual behaviour. the three-item measure of customer satisfaction was adapted from bressolles, durrieu and deans (2015) study. the sixitem measure for customers’ perceived value was adapted from bressolles, durrieu and deans (2015) study. finally, the measure for ibsds process with six items was adopted from ba and johansson (2008). the items are modified (minor changes) for the indian banking context. the items were revalidated before conducting the survey for this study. five-point likert scale with a rating from (1) “strongly disagree” to (5) “strongly agree.” has been used to facilitate measurement. the list of items used in the survey is given in appendix a. australasian journal of information systems kumar, sachan & kumar 2020, vol 24, research article service delivery and perceived value in internet banking adoption 9 3.2 sample profile and data collection the participants selected for the survey were the bank customers with prior experience (minimum of six months) of internet banking. the purpose of using these samples was to capture their internet banking service experience and also their future intentions. a paperbased survey was conducted in india, and a total of 600 questionnaires were distributed. the respondents participated voluntarily, and the purpose of the survey was disclosed to them while distributing the questionnaires. we have received 439 responses out of which only 416 (response rate 69.33%) responses were found eligible for inclusion in the study process. of the eligible respondents, 136 are female. as women have lesser representation compared to the mean in internet banking, their numbers are low in the sample (sanghera, 2018). the majority of the respondents were younger than 35 years of age. the majority of the respondents were highly educated, with 61.5 percent holding a graduate degree and 26.2 percent holding a post-graduate degree. the primary rationale for selecting these age group participants was that they were more tech savvy and likely to use internet-based services. table 1 illustrates the sample profile details. demographic variables category frequency percentage gender male 280 67.3 female 136 32.7 age 18-24 174 41.8 25-34 207 49.8 35-44 11 2.6 45-55 24 5.8 education high school 4 1.0 undergraduate 20 4.8 graduate 256 61.5 post-graduate 109 26.2 others 27 6.5 income (annual) <inr 5l 231 55.5 inr 5l-10l 93 22.4 inr 10l-15l 57 13.7 inr 15l-20l 22 5.3 >inr 20l 13 3.1 internet experience less than 2 year 28 6.7 between 2 and 4 years 51 12.3 between 4 and 6 years 105 25.2 greater than 6 years 232 55.8 internet banking experience less than 2 year 47 11.3 between 2 and 4 years 131 31.5 between 4 and 6 years 200 48.1 greater than 6 years 38 9.1 table 1: sample profile 3.3 reliability and construct validity reliability of the constructs in the model is tested using cronbach’s alpha (see table 2) and composite reliability. an exploratory factor analysis (efa) was conducted to validate the measurement model (see table 3). two items of ibsds process (ibsdsp2 & ibsdsp5) and two items of perceived value (pv3 & pv5) were dropped due to low factor loading (loading <0.5). australasian journal of information systems kumar, sachan & kumar 2020, vol 24, research article service delivery and perceived value in internet banking adoption 10 convergent validity of the constructs was estimated by the average variance extracted (see table 4). the cronbach’s alpha and composite reliability for all constructs were found to be above the recommended 0.70 level (hair, black, babin, anderson, & tatham, 2006). aves of all constructs were found to be above the recommended 0.50 level (fornell & larcker, 1981), which meant that more than half of the variances observed in the items are accounted for by their hypothesized constructs (chang et al., 2009). convergent validity can also be assessed by factor loading. all the factor loadings of the items in the research model were greater than 0.70, which also confirms the convergent validity. therefore, all constructs in the model have adequate reliability and convergent validity. dimensions cronbach’s alpha composite reliability (cr) ibsds process 0.800 0.856 cs 0.899 0.826 pv 0.900 0.826 bi 0.780 0.865 table 2: reliability test of the constructs rotated component matrixa component 1 2 3 4 pv1 .606 pv2 .752 pv5 .810 pv6 .772 cs1 .321 .745 cs3 .809 cs4 .794 ibsdsp1 .839 ibsdsp3 .860 ibsdsp4 .421 .690 ibsdsp6 .376 .694 bi1 .815 bi2 .874 bi3 .335 .786 extraction method: principal component analysis. rotation method: varimax with kaiser normalization. a. rotation converged in 5 iterations. table 3: exploratory factor analysis table 4 shows the correlations between all constructs in the model. none of the correlation values exceed the 0.80 cut-offs proposed by hair et al. (2006). therefore, it passes the multicollinearity test. the diagonal elements represent the square root of ave, and all are found to be greater than the correlation values of the corresponding column and row, which confirms discriminant validity. the reliability and validity test suggest that the constructs are suitable for further analysis (butcher, sparks, & o’callaghan, 2002). australasian journal of information systems kumar, sachan & kumar 2020, vol 24, research article service delivery and perceived value in internet banking adoption 11 constructs ibsds process pv cs bi ibsds process 0.775 pv .519** 0.739 cs .395** .516** 0.783 bi .389** .526** .498** 0.826 table 4: constructs correlation matrix 3.4 data analysis procedure and methods the study uses mediated regression to examine the direct effects of the ibsds process on customer satisfaction and behavioural intention, and the mediating effect of overall customer satisfaction on the association between ibsds process and behavioural intention. as suggested by baron & kenny (1986) and used in by extant study chang & polonsky (2012), to test for mediation, we have estimated following four regression equations: 1) regressing the mediator (customer satisfaction) on the independent variable (ibsds process); 2) regressing the dependent variable (behavioural intention) on mediator (customer satisfaction); 3) regressing the dependent variable (behavioural intention) on the independent variable (ibsds process); and 4) regressing the dependent variable (behavioural intention) on both the independent variable (ibsds process) and the mediator (customer satisfaction). we have estimated and tested separate coefficients for each equation. to test for mediation, the following conditions need to be met (baron & kenny, 1986; chang & polonsky, 2012): condition 1: there is a statistically significant relationship between independent variables (ibsds process) and the mediating variable (customer satisfaction), i.e., testing h1; condition 2: there is a statistically significant relationship between the mediating variable (customer satisfaction) and the outcome variable (behavioural intention), i.e., testing h2; condition 3: there is a statistically significant relationship between independent variables and the outcome variable (behavioural intention); condition 4: the statistically significant relationship between the independent variables (ibsds process, as per condition 1) and the outcome variable (behavioural intention) changes as a result of including the mediating variable (customer satisfaction), i.e., testing h3. based on the above conditions, we find the outcomes described below: • fully mediated: if the independent variables become statistically non-significant in condition 4 (whereas they were significant in condition 3) (baron & kenny, 1986; chang & polonsky, 2012). • partially mediated: if the independent variables remain statistically significant, but their influence is reduced, they are partially mediated (olkin & finn, 1995), and some of the effects of independent variables are transferred to the mediator (i.e., satisfaction) (baron & kenny, 1986; chang & polonsky, 2012). australasian journal of information systems kumar, sachan & kumar 2020, vol 24, research article service delivery and perceived value in internet banking adoption 12 to determine the moderation effect, process v2.16.3 by andrew f. hayes (hayes, 2012) was employed to help to analyse the moderating effect of the perceived value on the relationship between ibsds process and customer satisfaction and the relationship between customer satisfaction and behavioural intention. 4 results 4.1. mediation: table 5 shows the results of the mediation regression analysis. it can be seen that in the first condition (condition 1), we found that the ibsds process has a statistically significant influence on overall customer satisfaction. the second phase of the process examines the relationships between overall customer satisfaction and behavioural intention, and finds it to be significant. this satisfies the second mediating condition (condition 2). the third phase of analysis tests whether ibsds process influences consumer behavioural intention and this also found to be significant. the fourth phase of analysis (condition 4) determines whether customer satisfaction mediates the association between the ibsds process and consumer behavioural intention. full mediation exists when the direct path (i.e., beta coefficients) from the ibsds process becomes non-significant (i.e., full mediation). partial mediation exists if there is a reduction in the beta coefficients of the ibsds process after adding the overall customer satisfaction (i.e., mediator) to the regression (i.e., the difference in beta coefficients between condition 2 and 4 indicates partial mediation). the result suggests that beta coefficients of both ibsds process and overall customer satisfaction are found to be significant, and the beta coefficient of ibsds process is reduced. this indicates that ibsds process partially mediates the relationship. conditions standard coefficients independent variable dependent variable beta sig (p<0.05) results 1 ibsds process 𝑅𝑅𝑎𝑎𝑎𝑎𝑎𝑎. 2 = 0.154 cs 0.395 0.00 h1: supported 2 cs 𝑅𝑅𝑎𝑎𝑎𝑎𝑎𝑎. 2 = 0.223 bi 0.498 0.00 h2: supported 3 ibsds process 𝑅𝑅𝑎𝑎𝑎𝑎𝑎𝑎. 2 = .150 bi 0.389 0.00 supported 4 ibsds process, cs 𝑅𝑅𝑎𝑎𝑎𝑎𝑎𝑎. 2 = 0.292 bi ibsds process 0.229 cs-0.408 ibsds -0.00, cs-0.00 h3: partially mediated table 5: regression estimates of the conditions 4.1 moderation in spss process v2.16.3 by andrew f. hayes (hayes, 2012), we have selected model 1 to test moderation for both the cases and the output is shown in table 6 and table 7. the model in both cases indicates that zero doesn’t lie between the lower limit of confidential interval (llci) and upper limit of confidential interval (ulci) of “int_1” (see table 6 and 7), which indicates perceived value has a moderating effect on the relationship between customer satisfaction and australasian journal of information systems kumar, sachan & kumar 2020, vol 24, research article service delivery and perceived value in internet banking adoption 13 behavioural intention, and the relationship between ibsds process and customer satisfaction (supporting h4 and h5). model summary r r-sq mse f df1 df2 p .5475 .2998 .3738 58.7969 3.0000 412.0000 .0000 model coeff se t p llci ulci constant .9465 .4754 1.9912 .0471 .0121 1.8810 pv .7277 .1319 5.5156 .0000 .4684 .9871 ibsds .5235 .1469 3.5638 .0004 .2347 .8123 int_1 -.0997 .0378 -2.6361 .0087 -.1740 -.0254 product terms key: int_1 ibsdsp x pv y = cs x = ibsdsp m = pv conditional effect of x on y at values of the moderator(s): value effect se t p llci ulci 2.8798 .2364 .0529 4.4651 .0000 .1323 .3405 3.6514 .1595 .0425 3.7562 .0002 .0760 .2429 4.4231 .0825 .0501 1.6492 .0999 -.0158 .1809 table 6: moderation analysis output (perceived value on the relationship between ibsds process and customer satisfaction) model summary r r-sq mse f df1 df2 p .5943 .3532 .3334 74.9815 3.0000 412.0000 .0000 model coeff se t p llci ulci constant .5251 .5906 .8891 .3744 -.6358 1.6860 pv .6986 .1788 3.9065 .0001 .3470 1.0501 cs .5885 .1453 4.0499 .0001 .3028 .8741 int_1 -.0860 .0416 -2.0661 .0394 -.1678 -.0042 product terms key: int_1 cs x pv y = bi x = cs m = pv conditional effect of x on y at values of the moderator(s): value effect se t p llci ulci 2.8798 .3409 .0489 6.9653 .0000 .2447 .4371 3.6514 .2745 .0475 5.7766 .0000 .1811 .3680 4.4231 .2082 .0647 3.2186 .0014 .0810 .3354 table 7: moderation analysis output (present value on the relationship between customer satisfaction and behavioural intention) in order to test how the different levels of customer perceived value influence the relationships, we plot graphs (fig. 2 and 3) from the spss process output (data for visualizing conditional effect of x on y). according to fig. 2 and table 6, we can confirm that when ibsds process quality is low, a consumer with low perceived value is significantly less satisfied than the consumer with high perceived value. this supports h4, that is, customer perceived value has a significant moderating effect on the relationship between ibsds process and customer satisfaction. also from fig. 3 and table 7, we can confirm that customers with low perceived value have significantly lower behavioural intention to continuance use of internet banking than customers with high perceived value. this supports h5, that is, customer perceived value australasian journal of information systems kumar, sachan & kumar 2020, vol 24, research article service delivery and perceived value in internet banking adoption 14 has a significant moderating effect on the relationship between customer satisfaction and customer behavioural intention. fig. 2. moderating effect of perceived value on the relationship between ibsds process and customer satisfaction. fig. 3. moderating effect of perceived value on the relationship between customer satisfaction and behavioural intention. 5 discussion and conclusion we build on theories from service management and propose viewing the interface between user and service provider (i.e., bank website) to identify and explain the possible determinants of internet banking user adoption. we introduce a new variable, ibsds process, to the internet banking adoption literature. our study aims to address the following three questions: first, what role does the ibsds process play in determining consumer behavioural intention in using internet banking services?; second, does customer satisfaction mediate the relationship between the ibsds process and consumer behavioural intention?; and finally, does consumer perceived value moderate the relationships between the ibsds process and customer satisfaction, and between customer satisfaction and behavioural intention? 0 1 2 3 4 5 low medium high cu st om er s at is fa ct io n ibsds process the moderating effect of perceived value on the relationship between ibsds process and customer satisfaction low perceived value med perceived value high perceived value 0 1 2 3 4 5 low medium high be ha vi or al in te nt io n customer satisfaction moderating effect of perceived value on the relationship between customer satisfaction and behavioral intention low perceived value med perceived value high perceived value australasian journal of information systems kumar, sachan & kumar 2020, vol 24, research article service delivery and perceived value in internet banking adoption 15 our results are encouraging and provide theoretical and practical insights into the adoption of internet banking beyond those specified in existing literature ( e.g., tan and teo, 2000; bauer, hammerschmidt and falk, 2005; khalfan, alrefaei and al-hajery, 2006; al-somali, gholami and clegg, 2009; kesharwani and bisht, 2012). overall, the results support the significance of ibsds process, which exhibits reasonable accuracy and statistical significance. based on the mediated regression and spss process for moderating effect, as described in this study, three primary conclusions are drawn from this research as discussed below. the first aim of this study is to examine the effect of ibsds process on customer satisfaction. the result of the mediated regression analysis is consistent with our hypothesis. besides, it is also consistent with past studies (e.g., ba & johansson, 2008; sachan et al., 2018) which state that the service delivery process acts as an antecedent of customer satisfaction. our research demonstrates that as the ibsds process improves, it provides consumers more control and choices in the service process, leading to greater satisfaction. the second conclusion is that customer satisfaction mediates the relationship between the ibsds process and consumer behavioural intention. this is consistent with previous findings (e.g., chang & wang, 2011; chang et al., 2009). the results illustrate that pleasant experience with the ibps process translates into enhanced customer satisfaction, which in turn, motivates customers to adopt internet banking. the results are encouraging and provide theoretical and practical insights into the acceptance and use of internet banking services beyond those specified in the existing literature. the third conclusion of this study is that consumer perceived value moderates the relationships between ibsds process and customer satisfaction, and between customer satisfaction and behavioural intention. the results are consistent with our hypotheses. extant research (e.g., caruana, money and berthon, 1998) has examined the moderating effect of perceived value on the relationship between service quality and customer satisfaction. the results provide a basis for understanding the role of ibsds process and perceived value on customer satisfaction. it can be argued that while the technical capability may not be high in terms of a superior service delivery process, the fact remains that high perceived value contributes to a good level of customer satisfaction. the results indicate that customers may be more satisfied with a basic level of quality provided if they receive a higher value from online banking services. hence, it can be inferred that even if the perceived quality of the service delivery process is somewhat lower, a good level of satisfaction can still be achieved with the high received value. extant research (e.g., anderson & srinivasan, 2003; chang & wang, 2011; chang et al., 2009) also shows that consumer perceived value has a moderating effect on the relationship between customer satisfaction and behavioural intention. our results also support this relationship in the context of internet banking. besides, they also highlight that low satisfied customers who have low perceived value will have a low behavioural intention to continuance use of internet banking compared to low satisfied customers who have high perceived value. the results indicate that even if the consumers are less satisfied with internet banking, they may continue its use if the perceived value is high. the established relationships not only strengthen the theoretical foundations of the proposed research model in measuring the demand-side perspective but also allow policymakers to understand the extended facets of user satisfaction and their behavioural intention based on the delivery process of internet banking. this study provides a new direction for consumer australasian journal of information systems kumar, sachan & kumar 2020, vol 24, research article service delivery and perceived value in internet banking adoption 16 behavioural research by considering the service delivery system process characteristics of the internet banking environment. it also contributes to the extant literature by considering ibsds process in the proposed model, a feature not considered in the erstwhile studies, and, thus, advances the e-service theories related to the adoption of internet banking. the study also presents an integrated model of the determinants of internet banking adoption and use. the model develops a new theory by identifying a new variable, the ibsds process, in the integration of customer satisfaction and behavioural intention, and applying it to the internet banking context. it is worthy to note that this research is based on the reformation of attitude theory by bagozzi (1992) and the new variable, ibsds process, which is specific to the online environment. the findings suggest that it is crucial for consumer behaviour researchers to examine the role of the service delivery system process and consumer’s perceived value, which is likely to affect its use. excellent electronic service (e-service) quality is essential for successful internet banking. from a managerial standpoint, this study provides website designers and financial service providers a new perspective which is that they should ensure that their website is well-designed and that it incorporates all the necessary features and tools to provide their customers an easy-touse, secure, reliable and enhanced experience. our findings also demonstrate the impact of ibsds process on user satisfaction and behavioural intention. hence, it may be true that the quality of the online interaction process affects users’ access decisions, satisfaction, and behavioural intention in the context of internet banking services. moreover, consumers’ perceived quality of an internet banking service is reflective of not only the service quality but also the quality of the online interaction process. hence, financial service providers or related agencies must consider the ibsds process to enhance customer satisfaction and ultimately the adoption of internet banking while implementing their online banking strategy. customer service also needs to be given due attention. it is in the interest of the banks to offer their customers diverse contact channels for communication. the findings are significant and cannot be overlooked in practice. 6 limitations and suggestions for future research the study has some limitations, which provide a direction for future research. our research does not consider several other factors, such as the effect of involvement, and convenience orientation, which could moderate the relationships under study. future research could also study the effect of demographic characteristics like age, gender, and experience on the adoption of internet banking services. besides, the data sample used in this study, which is 416, is limited and cannot be used to represent the entire population of a country as diverse as india. a larger sample may strengthen the results obtained. another limitation is that this study considers perceived value as a one-dimensional variable. however, literature (e.g., sheth, newman and gross, 1991; sweeney and soutar, 2001) shows this value to be a multidimensional element. different dimensions of perceived value may have a different impact on the relationships. hence, a study considering different value dimensions may give more detailed insights. this work is only a first step in trying to understand the ibsds process and its impact on customer satisfaction and behavioural intention. we believe that this is a promising research area for researchers in the management of technology domain. to generalize the study to the other countries, research must be validated in other countries. australasian journal of information systems kumar, sachan & kumar 2020, vol 24, research article service delivery and perceived value in internet banking adoption 17 references al-somali, s. a., gholami, r., & clegg, b. (2009). an investigation into the acceptance of online banking in saudi arabia. technovation, 29(2), 130–141. https://doi.org/10.1016/j.technovation.2008.07.004 alawneh, a., al-refai, h., & batiha, k. (2013). measuring user satisfaction from e-government services : lessons from jordan. government information quarterly, 30(3), 277–288. https://doi.org/10.1016/j.giq.2013.03.001 anderson, r. e., & srinivasan, s. s. (2003). e-satisfaction and e-loyalty : a contingency framework. psychology & marketing, 20(2), 123–138. https://doi.org/10.1002/mar.10063 ba, s., & johansson, w. c. (2008). an exploratory study of the impact of e-service process on online customer satisfaction. production and operations management, 17(1), 107–119. https://doi.org/10.3401/poms.1070.0006 bagozzi, r. p. . (1992). the self-regulation of attitudes , intentions , and behavior. ocial psychology quarterly, 55(2), 178–204. baron, r. m., & kenny, d. a. (1986). the moderator-mediator variable distinction in social psychological research: conceptual, strategic, and statistical considerations. journal of personality and social psychology, 51(6), 1173–1182. https://doi.org/10.1037/00223514.51.6.1173 bashir, i., & madhavaiah, c. (2015). consumer attitude and behavioural intention towards internet banking adoption in india. journal of indian business research, 7(1), 67–102. https://doi.org/10.1108/el-01-2017-0019 bauer, h. h., hammerschmidt, m., & falk, t. (2005). measuring the quality of e-banking portals. international journal of bank marketing, 23(2), 153–175. https://doi.org/10.1108/02652320510584395 bloemer, j., de ruyter, k., & peeters, p. (1998). investigating drivers of bank loyalty: the complex relationship between image, service quality and satisfaction. international journal of bank marketing, 16(7), 276–286. https://doi.org/10.1108/02652329810245984 bolton, r. n., & drew, j. h. (1991). a multistage model of customers’ assessments of service quality and value. journal of consumer research, 17(4), 375. https://doi.org/10.1086/208564 bressolles, g., durrieu, f., & deans, k. r. (2015). an examination of the online service-profit chain. international journal of retail & distribution management, 43(8), 727–751. butcher, k., sparks, b., & o’callaghan, f. (2002). effect of social influence on repurchase intentions. journal of services marketing, 16(6), 503–514. https://doi.org/10.1108/08876040210443382 caruana, a., money, a. h., & berthon, p. r. (1998). service quality and satisfaction the moderating role of value. european journal of marketing, 34(11/12), 1338–1353. chang, h. h., & wang, h.-w. (2011). the moderating effect of customer perceived value on online shopping behaviour. online information review, 35(3), 333–359. chang, h. h., wang, y., & yang, w. (2009). the impact of e-service quality, customer satisfaction and loyalty on e-marketing : moderating effect of perceived value. total australasian journal of information systems kumar, sachan & kumar 2020, vol 24, research article service delivery and perceived value in internet banking adoption 18 quality management & business excellence, 20(4), 423–443. https://doi.org/10.1080/14783360902781923 chang, y., & polonsky, j. m. (2012). the influence of multiple types of service convenience on behavioral intentions : the mediating role of consumer satisfaction in a taiwanese leisure setting. international journal of hospitality management, 31(1), 107–118. https://doi.org/10.1016/j.ijhm.2011.05.003 chen, z., ling, k. c., ying, g. x., & meng, t. c. (2012). antecedents of online customer satisfaction in china. international business management. https://doi.org/10.3923/ibm.2012.168.175 cronin, j. j., brady, m. k., & hult, g. t. m. (2000). assessing the effects of quality, value, and customer satisfaction on consumer behavioral intentions in service environments. journal of retailing, 76(2), 193–218. https://doi.org/10.1016/s0022-4359(00)00028-2 dabholkar, p. a. (2000). technology in service delivery: implications for self-service and service e-support. in handbook of service marketing and management. beverly hills, ca.: sage. danaher, p. j., & mattsson, j. (1994). customer satisfaction during the service delivery process. european journal of marketing, 28(5), 5–16. davis, f. d. (1989). perceived usefulness, perceived ease of use and user acceptance of information technology. mis quarterly, 13(3), 319–340. dhananjay, b., & chandra, s. b. (2015). the electronic banking revolution in india. journal of internet banking and commerce, 20(2), 1. https://doi.org/10.1525/sp.2007.54.1.23. encashing on digital: financial service in 2020. (2017). retrieved from http://imagesrc.bcg.com/images/bcg-facebook-encashing-on-digital-jun-2017_tcm21-163357.pdf fornell, c., & larcker, d. f. (1981). structural equation models with unobservable variables and measurement error: algebra and statistics. journal of marketing research, 382–388. grönroos, c. (1982). an applied service marketing theory. european journal of marketing, 16(9), 30–41. gupta, p. j., & suri, p. (2017). measuring public value of e-governance projects in india: citizens’ perspective. transforming government : people , process and policy, 11(2), 236–261. hair, j. e., black, w. c., babin, b. j., anderson, r. e., & tatham, r. l. (2006). multivariate data analysis (6th ed.). upper saddle river, nj.: pearson prentice-hall. hayes, a. f. (2012). process: a versatile computational tool for observed variable mediation, moderation, and conditional process modeling. white paper, 1–39. https://doi.org/978-160918-230-4 heskett, b. y. j. l., jones, t. o., loveman, g. w., sasser, w. e., & schlesinger, l. a. (1994). putting the service-profi t chain to work. harvard business review, 72(2), 164–174. https://doi.org/10.1145/781031.781034 hunt, h. keith, e. (1977). conceptualization and measurement of consumer satisfaction and dissatisfaction. marketing science institut2, 77–103. australasian journal of information systems kumar, sachan & kumar 2020, vol 24, research article service delivery and perceived value in internet banking adoption 19 jay, h., barry, r., chuck, m., & amit, s. (2017). operations management sustainability and supply chain management. (12th, ed.). pearson india education. jones, t. o., & sasser, w. e. (1995). why satisfied customers defect. harvard business review, 73(6), 88. kaura, v. (2013). antecedents of customer satisfaction: a study of indian public and private sector banks. international journal of bank marketing, 31(3), 167–186. kesharwani, a., & bisht, s. s. (2012). the impact of trust and perceived risk on internet banking adoption in india: an extension of technology acceptance model. international journal of bank marketing, 30(4), 303–322. https://doi.org/10.1108/02652321211236923 khalfan, a. m. s., alrefaei, y. s., & al-hajery, m. (2006). the impact of trust and perceived risk on internet banking adoption in india: an extension of technology acceptance model. international journal of financial services management, 1(2–3), 155–172. kim, g. j. (2014). applying service profit chain model to the korean restaurant industry. international journal of hospitality management, 36, 1–13. https://doi.org/10.1016/j.ijhm.2013.07.008 laukkanen, t. (2007). internet vs mobile banking: comparing customer value perceptions. business process management journal, 13(6), 788–797. https://doi.org/10.1108/14637150710834550 lekhawipat, c. l. w. (2014). factors affecting online repurchase intention. industrial management & data systems, 114(2), 597–611. https://doi.org/10.1108/imds-10-2013-0432 luu, n., hau, l. n., ngo, l. v., bucic, t., & cuong, p. h. (2016). outcome versus process value in service delivery. journal of services marketing, 30(6), 630–642. https://doi.org/10.1108/jsm-12-2014-0410 mcdougall, g. h. g., & levesque, t. (2000). customer satisfaction with services: putting perceived value into the equation. journal of services marketing, 14(5), 392–410. https://doi.org/10.1108/08876040010340937 meuter, m. l., ostrom, a. l., roundtree, r. i., & bitner, m. j. (2000). self-service technologies : satisfaction with technology-based service encounters. journal of marketing, 64(3), 50– 64. mishra, s. (2009). a conceptual framework for creating customer value in e-retailing in india. south asian journal of management, 16(4), 127–147. olkin, i., & finn, j. d. (1995). correlations redux. psychological bulletin, 118(1), 155–164. https://doi.org/10.1037/0033-2909.118.1.155 papadomichelaki, x., & mentzas, g. (2012). e-govqual : a multiple-item scale for assessing egovernment service quality. government information quarterly, 29(1), 98–109. https://doi.org/10.1016/j.giq.2011.08.011 roth, a. v, & jackson, w. e. i. (1995). strategic determinants of service quality and performance : evidence from the banking industry. management science, 41(11), 1720– 1733. australasian journal of information systems kumar, sachan & kumar 2020, vol 24, research article service delivery and perceived value in internet banking adoption 20 sachan, a., kumar, r., & kumar, r. (2018). examining the impact of e-government service delivery process on user satisfaction. journal of global operations and strategic outsourcing, 11(3). sanghera, t. (2018). 80% of indians now have a bank account. so why is financial inclusion low? retrieved june 4, 2019, from https://www.businessstandard.com/article/finance/80-of-indians-now-have-a-bank-account-so-why-isfinancial-inclusion-low-118051700150_1.html sharma, s. k., & govindaluri, s. m. (2014). internet banking adoption in india: structural equation modeling approach. journal of indian business research, 6(2), 155–169. https://doi.org/10.1108/el-01-2017-0019 sheth, j. n., newman, b. i., & gross, b. l. (1991). why we buy what we buy: a theory of consumption values. journal of business research, 22(2), 159–171. singh, b., & malhotra, p. (2015). adoption of internet banking: an empirical investigation of indian banking sector. the journal of internet banking and commerce, 2004. sweeney, j., & soutar, g. (2001). consumer perceived value: the development of a multiple item scale. journal of retailing, 77(2), 203–220. https://doi.org/10.1016/s00224359(01)00041-0 szopiński, t. s. (2016). factors affecting the adoption of online banking in poland. journal of business research, 69(11), 4763–4768. https://doi.org/10.1016/j.jbusres.2016.04.027 tan, m., & teo, t. s. h. (2000). factors influencing the adoption of internet banking. journal of the association for information systems, 1(1), 1–44. https://doi.org/10.1016/j.elerap.2008.11.006 wang, c. (2014). antecedents and consequences of perceived value in mobile government continuance use: an empirical research in china. computers in human behavior, 34, 140– 147. https://doi.org/10.1016/j.chb.2014.01.034 woodruff, r. b. (1997). customer value: the next source for competitive advantage. journal of the academy of marketing science, 25(2), 139–153. https://doi.org/10.1007/bf02894350 yang, z., & peterson, r. t. (2004). customer perceived value, satisfaction, and loyalty: the role of switching costs. psychology and marketing, 21(10), 799–822. https://doi.org/10.1002/mar.20030 youngdahl, w. e., & kellogg, d. l. (1997). the relationship between service customers ’ quality assurance behaviors , satisfaction , and effort : a cost of quality perspective. journal of operations management, 15, 19–32. zeithaml, v. a. (1988). perceptions of price, quality, and value: a means-end model and synthesis of evidence. journal of marketing, 52(3), 2–22. https://doi.org/10.2307/1251446 zeithaml, v. a., berry, l. l., & parasuraman, a. . (1996). the behavioral consequences of service quality. journal of marketing, 60, 31–46. zeithaml, v. a., berry, l., & parasuraman, a. (1991). understanding customer expectations of service. sloan management review, 32(3), 42. australasian journal of information systems kumar, sachan & kumar 2020, vol 24, research article service delivery and perceived value in internet banking adoption 21 appendix a: measurement scales constructs scale items source perceived ibsds process ibsdsp1: the internet banking site is difficult to navigate through. (ba & johansson, 2008)adopted and modified ebsdsp2: the number of choices at each step of the internet banking service access process doesn’t need to be changed. ibsdsp3: the internet banking website ordering process wasn’t complicated. ibsdsp4: i did not experience any errors (e.g., web pages that did not load the first time) while accessing internet banking services. ibsdsp5: i had trouble finding what i was looking for on the internet banking website. ibsdsp6: the entire process of searching and transacting in internet banking takes a reasonable amount of time. perceived value pv1: using internet banking is a waste of my time. (r) (bressolles et al., 2015)adopted and modified pv2: the service provided through the bank website is very efficient. pv3: the bank website requires a lot of effort to use. (r) pv4: i am treated fairly by the bank website. pv5: very little thought is required to use this bank website. pv6: the bank website doesn’t provide value. (r) customer satisfaction cs1: i am satisfied with my decision to use the internet banking service. (bressolles et al., 2015)adopted and modified cs2: my choice to access internet banking service is a good one cs3: i am confident it is the right thing to access banking services from banking web site. behavioral intention bi1: i intend to continue to do online business with my present bank also in the future (yang & peterson, 2004) adopted and modified bi: i say positive things about the internet banking to other people. bi3: i encourage friends and relatives to use internet banking. australasian journal of information systems kumar, sachan & kumar 2020, vol 24, research article service delivery and perceived value in internet banking adoption 22 copyright: © 2020 kumar, sachan & kumar. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: http://doi.org/10.3127/ajis.v24i0.1923 http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 theoretical framework 2.1 ibsds process 2.2 customer satisfaction 2.3 behavioural intention 2.4 customer perceived value 2.5 proposed hypotheses 2.5.1 ibsds process and customer satisfaction 2.5.2 customer satisfaction and behavioural intention 2.5.3 customer satisfaction, ibps process and behavioural intention 2.5.4 perceived value, ibsds process and customer satisfaction 2.5.5 perceived value, customer satisfaction, and behavioural intention 3 methodology 3.1 measurement scale 3.2 sample profile and data collection 3.3 reliability and construct validity 3.4 data analysis procedure and methods 4 results 4.1 moderation 5 discussion and conclusion 6 limitations and suggestions for future research microsoft word ajis_18(3)_v10.docx australasian journal of information systems volume 18 number 3 2014 75 assistive technologies for aged care: supportive or empowering? suchada vichitvanichphong university of the sunshine coast suchada.vichitvanichphong@research.usc.edu.au don kerr university of the sunshine coast dkerr@usc.edu.au amir talaei-khoei university of technology sydney amir.talaei-khoei@uts.edu.au amir hossein ghapanchi griffith university a.ghapanchi@griffith.edu.au abstract the main objective of this paper is to explore the potentials of assistive technologies to support seniors’ independent living. the work looks at two salient aspects of utilizing technologies for elderly, namely direct support and empowering technologies. the research undertakes a comprehensive analysis of attempts that have been made through investigation of the literature. for this purpose, a realist review of relevant papers published since 2000 has been conducted. the paper concludes that although much research in this area targets the direct support for older adults, the effective use of technologies to maintain seniors’ physical and cognitive abilities requires further investigations. this can provide avenues of opportunities that would empower seniors for their independent living. introduction owing to a substantial decline in the age-specific mortality, the world’s population is ageing rapidly. in 2012, 6.9% of the world population were more than 65 years old, and this is estimated to increase to around 20% by 2050 (oecd, 2012). what was before a small number of exceptional old individuals is rapidly becoming a whole new generation for families and the society? the elderly have lived long enough to have acquired, on average, several medical conditions, and a range of age-associated impairments of bodily and mental functions. this has led to concern as to whether existing care provision will cope with the predicted future demand (australian nursing federation, 2012). also, it is commonly believed that seniors would like to live independently at their home (porteus and brownse, 2000). however, the issue is the fact that whether their home is compatible with their physical and cognitive conditions (leeson et al., 2004). in this perspective, assistive technologies provide aid for elderly, but are underutilized to offer effective training of skills that are required for independent living. marshall, (1997) (marshall, 1997b) has defined assistive technology in the context of aiding seniors with disabilities as “any item, piece of equipment, product or system, whether acquired commercially, off-the-shelf, modified or customized, that is used to increase, maintain or improve functional capabilities of individuals with cognitive, physical or communication disabilities”. the above definition has an emphasis on disabilities in seniors. a more recent definition has been given by the australian dementia resources guide (doha, 2008). this guide defines “assistive technologies as a product, equipment or device, usually electronic or mechanical in nature, which helps people with disabilities to maintain their independence or improve their quality of life”. this definition has extended the use of australasian journal of information systems volume 18 number 3 2014 76 assistive technologies from devices to help older adults with disabilities to products facilitating seniors’ with their daily life activities. our definition of assistive technology is most closely related to the dementia resources guide definition, although we look at assistive technologies in a boarder sense than only being used for dementia patients. while recent advancements in information technology (it) have resulted in cheap off-the-shelf products that can have potential to assist older people in their daily life activities at their home (miskelly, 2001), less is known on how these technologies can assist seniors. in other words, despite the existing body of literature in technologies that can assist elderly, the bigger picture of research on what are the real potentials for these technologies is missing. our exploratory search on application of assistive technologies in aged care (vichitvanichphong et al., 2013) has shown that there are very limited studies that aim to empowering seniors by training and maintaining the skills that are necessary for independent lifestyle. instead, work around this topic has been based more on introducing technologies that directly support elderly in their daily living. in response to the above mentioned concerns, this paper aims to look at the technologies that have been used for aged care and identifies the potentials of these technologies. the ultimate aim of this work is to better understand how technologies can be utilized in this setting. therefore, researchers and professionals would be aware of salient approaches that assistive technologies can be effectively utilized for seniors. this paper seeks to answer the following research questions: what are the potentials of assistive technologies for seniors’ independent living? for this to be answered, we seek to identify the available technologies. we also look at the attributes of these technologies and how they open up potential opportunities for effective approaches in utilising technologies in aged care. in order to answer the question, the present research has taken a realist review approach as suggested by (pawson et al., 2005). we have systematically searched papers and extracted relevant data to the questions. from these papers, we have also conducted some characteristics of the research in this area e.g. research method, or geographical distribution etc. method we customized the guidelines for realist reviews laid down by (pawson et al., 2005). the reason why we chose to follow the realist approach was that it has been proven (macaulay et al., 2011) as an effective way of reviewing complex interventions with social aspects, which implies to the matter of technology adoption among seniors. the realist review proposes six steps to carry out a literature review; (1) exploratory background search to identify the relevant keywords and databases, (2) searching for the initial list of studies, (3) relevance appraisal, (4) extracting data, (5) analysis of data and finally, (6) evaluation. in the following sections, we explain the process that we have conducted in our realist review of literature associated with the adoption of assistive technologies among seniors. exploratory background search: identifying keywords and databases the first step towards searching the articles was to identify the relevant keywords and online databases. we have conducted a background survey (vichitvanichphong et al., 2013) in two topic areas; medical informatics and information systems. these areas have been selected due to the focus of this review to the assistive technologies among seniors. an initial list of journals has been chosen from the journal classification list proposed by excellence in research for australia (era, 2012). the relevant journals were selected and the journals with an impact factor of less than one were omitted. as such, we came up with 14 journals regarded as popular outlets of publication in this area. we have found 31 relevant papers published in these journals, since 2009. australasian journal of information systems volume 18 number 3 2014 77 in order to identify the relevant keywords and related online databases, we have implemented the experimental method proposed by (dieste et al., 2009). the method was applied in the above-mentioned 31 papers and identified most popular online databases indexing these 31 papers, also the most popular keywords relevant to “aged care” and “adoption” used with “technology”. searching the initial list of papers following the search instructions given by each database and using the most popular relevant keywords identified in the background search, we required that the relevant articles contain the word “technology” along with any of “aged care”, “aged”, “aging”, “senior”, “old”, “elderly”, “elder” or “older” in their titles, keywords, abstracts or full texts. eight popular databases were searched using the above keywords. the search considered titles, keywords, abstracts and full texts of papers published since 2000, inclusive and returned 723,944 articles. the distribution of papers in each database is presented in table 1. relevance appraisal in this step, the objective was to filter relevant papers from the initial list and exclude the ones which are not related to our study. this process was carried out by excluding papers based on titles, keywords, abstracts and full texts; see figure 1. articles that have one of the following exclusion criteria were removed: • did not focus on assistive technologies for aged care. • did not have any empirical evidence. • were in languages other than english. • were not in the relevant fields or could not be applied to relevant fields. • were not peer reviewed. • were not available online. australasian journal of information systems volume 18 number 3 2014 78 filtered by titles filtered by abstract filtered by text name of database initial list of papers no. of found articles no. of duplicate articles no. of found articles no. of duplicate articles no. of found articles no.of duplicated articles springer 16,539 213 22 31 11 18 6 wiley interscience 453,537 99 15 17 8 11 5 science direct 11,442 49 18 38 16 32 10 ieeexplor 69,690 30 13 22 12 16 6 acm digital library 560 21 6 14 5 10 3 scirus 94,487 27 7 18 6 14 4 pubmed 10,989 33 9 25 8 18 6 google scholar 66,700 160 122 48 16 39 14 total 723,944 632 212 213 75 158 54 relevant papers by deducting the duplicated articles 420 138 104 table 1 distribution of papers in each online database. among 723,944 papers indexed in the databases and searched by the keywords, 420 papers were remained after title filtering, 138 articles were remained after abstract filtering and 104 papers were identified as final list of relevant papers after reading the full texts. we also found some of the papers were indexed by multiple databases, see table 1. it is notable that we used zotero (http://www.zotero.org/) for reference management. figure 1 relevance appraisal process extracting data in the data extraction stage, key details from the selected papers were obtained. three types of data were collected from each paper: • technology options: we identified what technologies have been used in each paper to assist seniors. australasian journal of information systems volume 18 number 3 2014 79 • demographics: in order to analyze the characteristics of the research in this area, we have collected demographic details such as research methods, fields of publication and geographical distribution each relevant paper. in order to ensure the reliability of the results, we used standard critical review forms for quantitative (law et al., 1998a) and qualitative studies (law et al., 1998b). for mix-method studies, we completed both forms. analysis of data in order to answer the research question, we identified the classification of technologies that have been used to assist elderly i.e. technology options. we also identified the main research areas. in these analyses, we used the process proposed by (ghapanchi and aurum, 2011). in order to answer to the research question, and identifying the technology options, the process involved extracting the terms and definitions used in the final list of selected papers, and eventually forming the primary list of technologies. it broadly categorized the factors as well as the technologies. the process is depicted in figure 2. figure 2 data analysis process. peer evaluation and reliability testing a peer review of the findings for the interventions and factors impacting the adoption of technologies among seniors was undertaken. four experts from north america, australia, europe and asia were invited to code both interventions and impacting factors of adoption into categories. the experts were chosen based on their academic knowledge and practical experience in the topic. the experts were given an instruction sheet including the description of all the items as well as an answer sheet in which they were asked to map the items into the proposed categories. a reliability test was then undertaken based on proportional reduction in loss (prl) reliability indicator introduced by (rust and cooil, 1994). prl is used to assess the consensus between judges who are invited to code a number of elements into exclusive categories. in addition to the four experts, we have considered the proposed contributions of this work as one coding. therefore, the total number of judges was five. inter-judge agreement was measured by dividing the total pair-wise agreements by the total pair-wise decisions. having five judges onboard; in assigning each item into a category, a total number of 10 pair-wise decisions were made. the consensus of assigning an item to a category was the most frequently selected choice by the five judges. the reliability for each contribution was calculated based on the value of inter-judge agreement. the number of pair-wise agreements was 169 out of the total australasian journal of information systems volume 18 number 3 2014 80 270 pair-wise decisions, which calculates 0.62 inter-judge agreements and 78% reliability of the results. therefore, based on (rust and cooil, 1994), the reliabilities of contribution in this paper are acceptable. findings in this section, we present the results of our investigation on the technology options, and demographics of the research in assistive technologies for aged care. technology options as described in this section, we have categorized technologies that have been adopted in 104 relevant papers into 7 categories; general purpose ict, social media, games, robots, online information services, smart home and remote care, and supportive devices. it was found that general purpose ict and smart home and remote care were the technologies that most of the interventions have been focused, see figure 3. figure 3 distribution of papers over different technology options the first category is ict in general purpose and this includes technologies such as mobile phones, the internet, email, etc. seniors have been using these technologies to cope with the transition period or life style change after retirement or to communicate with family, friends and caregivers; see table 3. these technologies are well established and not considered innovative any more. we consider the use and adoption of these well-established technologies to be relatively straightforward and requiring more of an educational training approach as community pressure will play an important part in older people adopting and using these technologies. these devices are becoming increasingly mainstream and part of everyday society much like television or the home phone. australasian journal of information systems volume 18 number 3 2014 81 technology sample of adoption email “…elderly can enhance their self-esteem and decrease loneliness by remaining in contact with the world via e-mail. the internet and e-mail are means to keep in touch with the world… (kiel, 2005)” other(s): (hiroyuki umemuro, 2004); (haase et al., 2012); (adams et al., 2005); (selwyn et al., 2003); (melenhorst et al., 2006); (rosenberg et al., 2009); (singh et al., 2009) mobile/smart phone “…using the mobile health services/ smart phone will improve seniors’ life quality and make seniors more effective in their lives. for example, seniors use mobile phone to communicate with family, friends, and carers any time they need... (guo et al., 2013)” other(s): (conci et al., 2009); (xue et al., 2012); (ahn et al., 2008); (barnard et al., 2013); (renaud and van biljon, 2008); (häikiö et al., 2007); (xue et al., 2012); (mallenius et al., 2007); (walsh and callan, 2011); (melenhorst et al., 2001); (haase et al., 2012); (boontarig et al., 2012); (kubik, 2009); ((boni) li and perkins, 2007); (neves, 2012); (rosenberg et al., 2009); (hardill and olphert, 2012); (salovaara et al., 2010); (copolillo and prohaska, 2001); (biljon and renaud, 2008); digital camera “…older adults is able to increase their memories as well as enhance their quality of life through using digital camera…(salovaara et al., 2010)” mp3 “…listening to music on mp3 player instead of an old radio help seniors to perform task better and enhance independence and quality of life…(salovaara et al., 2010)” gps “…the global positioning system (gps) provides seniors advanced means of communication and support the use of different modalities which makes seniors’ lives a lot easier…(salovaara et al., 2010)” personal digital assistant (pda) “…pda helps seniors to execute day-to-day duties as well as storage information, which is an important tool for managing their lives… (wilkowska and ziefle, 2009)” computer “…using computer makes older adults independent. computers can fulfill both types of activities—acting for some people as a means to ‘keep the brain ticking’ and ‘filling a void in seniors’ lives’…(selwyn, 2004)” other(s): (aula, 2005); (cameron et al., 2001); (ahn et al., 2008); (mitzner et al., 2010); (abdullah et al., 2011); (czaja et al., 2008); (giuliani et al., 2005); (lam and lee, 2006); (walsh and callan, 2011); (kiel, 2005); (melenhorst et al., 2001); (burnett et al., 2011); (czaja et al., 2006); (haase et al., 2012); (hernández-encuentra et al., 2009); (morris et al., 2007); ((boni) li and perkins, 2007); (neves, 2012); (selwyn et al., 2003); (rosenberg et al., 2009); (wood et al., 2005); (karavidas et al., 2005); (sayago et al., 2011); (carpenter and buday, 2007); (salovaara et al., 2010); (h. umemuro, 2004) internet “…functional abilities in daily activities of older adults can be boosted by using the internet. older adults use the internet for communication, seeking information, commercial purposes, etc. eventually, using the internet can enhance seniors’ wellbeing…(sum et al., 2008) other(s): (nayak et al., 2010); (aula, 2005); (cameron et al., 2001); (sum et al., 2009); (hanson, 2010); (pan and jordan-marsh, 2010); (mitzner et al., 2010); (abdullah et al., 2011); (el-attar et al., 2005); (czaja et al., 2008) ;(lam and lee, 2006); (kiel, 2005); (melenhorst et al., 2001); (czaja et al., 2006); (haase et al., 2012); (hernández-encuentra et al., 2009); (morris et al., 2007); (adams et al., 2005); (eastman and iyer, 2004); (neves, 2012); (melenhorst and bouwhuis, 2004); (rosenberg et al., 2009); (sum et al., 2008); (choi and dinitto, 2013) table 3 technology options: general purpose ict australasian journal of information systems volume 18 number 3 2014 82 the second category is social media. increasingly, we see seniors becoming isolated from social activities, usually after the loss of a partner and this, combined with the desire to live independently in their own home, leads to a loss of previously established social networks. researchers have used social media to engage seniors in the community and to increase their social interactions by staying in touch with people, establishing friendships and gaining information, even sharing thoughts and experiences; see table 4. social media technology is relatively new and requires a mindset of ubiquitous access and this usually entails the use of newer technologies such as tablets, smartphones or laptop computers to be truly effective. the adoption of these newer technologies could be more problematic for older people and there needs to be well thought out processes in place to incentivize adoption for this older generation. technology sample of adoption online social network “…to decrease loneliness as well as enhance (health) communication for older adults, social network or social media is a great tool used by them. using online social network, particularly comprehensive social network platform like facebook, allows seniors to keep up in the day to day lives of important others, for example…(braun, 2013)” other(s): (heinz et al., 2013) online community “…online community is increasingly being incorporated into the everyday routines of older adults. engaging in online community can improve seniors’ quality of life and decrease loneliness…(chung et al., 2010)” other(s): (ryu et al., 2009); (wong et al., 2012) table 4 technology options: social media another category is games; either video games or mobile games. many older adults have a connection with traditional games such as card games like bridge and board games like monopoly. this could easily be extended to the electronic media and research has shown that older adults can use video or mobile games as a therapy tool and for entertainment, relaxation and socialization; see table 5. we found games, because of their interactive nature can also be used to improve seniors’ cognitive skills and this can help them in their functioning for their daily lives. cognitive research has found that this could be a very significant feature in improving daily living for seniors and that gaming can provide a proactive approach by improving seniors’ living skills. in fact, this approach could provide brain training before the elderly start to need assistance and thus reduce the resources needed to facilitate daily living further down the track. we will discuss this aspect later. australasian journal of information systems volume 18 number 3 2014 83 technology sample of adoption video game “…video games increasingly used as a therapeutic tool in health and aged care settings. the use of interactive video games as a therapy tool for addressing both physical and cognitive function is a growing trend in the health and aged care sector …(laver et al., 2011)” other(s): (mckay and maki, 2010); (mclaughlin et al., 2012); (heinz et al., 2013); (wood et al., 2005) mobile game “…mobile games can benefit seniors in terms of entertainment and relaxation, socialization, mental challenges and physical fitness…(chu yew yee et al., 2010)” table 5 technology options: games robots are also a well-known technology option in this research area. social robots have been used in health-related settings (see table 6) and research has found that for seniors to adopt robots in their daily lives, they need to build a long term relationship with the robot. intelligent, interactive social software can assist seniors in an interactive way and help them with daily living chores. social robots are hardware robots that physically assist the elderly. this area of research looks promising; however there are major obstacles with respect to adoption that needs to be overcome. for example many older people would have problems with the concept of building a long term relationship with a machine (robot) and this concept may need to be modified to allow for human wants and emotional needs. this is very culturally and country specific however, the level of acceptance of long term machine/people relations could vary according to cultural norms. technology sample of adoption interactive social software “…using interactive social software at home as assisted therapy and physical and cognitive assistant can help seniors to improve quality of life, execute day-to-day activities as well as mental health…(heerink et al., 2010)” other(s): (bickmore et al., 2005b) social robot “…social robots are looked upon as potential aids to improve the quality of life of older adults, help them live independently and help relieve the stress of their caregivers…(klamer and ben allouch, 2010)” other(s): (m. heerink et al., 2008); (heerink, 2011); (heerink et al., 2006); (ng et al., 2012); (smarr et al., 2012); (marcel heerink et al., 2008); (mitzner et al., 2011); (neven, 2010) table 6 technology options: robots as seen in table 7, it was also found that online information services have been popularly used in this cohort e.g. using online health service to seek and update health information, electronic health records and to make health information more accessible to patients for the management of chronic conditions. the acceptance of these services would be very much associated with context. for example, a healthy person would find little need to access information on diabetes, unless they receive a diagnosis of the disease so we would expect high adoption rates for people suffering chronic disease, especially if they are having difficulty in managing the condition. australasian journal of information systems volume 18 number 3 2014 84 technology sample of adoption online health services “…online health services aid older adults to maintain their independence and quality of life…(ryu et al., 2009)” other(s): (steele et al., 2009a), (heinz et al., 2013) electronic health records “…electronic personal health records (phrs) have the potential to both make health information more accessible to patients and function as a decision-support system for patients managing chronic conditions. elder people tend to use phrs to help them to manage in health related information, which is convenient, beneficial, and productive in comparison to the paper based…(price et al., 2013)” online daily services “… older adults use eservice to shop, check prices, choose shipping methods and then pay and complete the transaction online which help them to increase self-actualization and independence…(mccloskey, 2006)” other(s): (phang et al., 2006) online learning “…online learning is a great tool to empower adult learner. seniors use online learning to improve cognitive abilities, maintain independence and enhance their quality of life…(chu, 2010)” table 7 technology options: online information services the smart home and remote care technologies are also used to help seniors balance safety and independence through remote monitoring, motion detecting sensors and fall alarms technology. furthermore, telecare has been used for balancing safety and independence purposes and like the smart home products it includes the provision of support for seniors who have dementia; see table 8. one of the interesting interventions is to use motion detection sensors to monitor seniors’ movement around the house, while living alone, for two purposes: (a) to monitor elderly and automatically detect their falls, and inform the authority, family or friend for primary aid (sarkisian et al., 2003), (b) to monitor seniors and automatically notice any radical changes in their functional or movement abilities, which may need serious and further treatments (goins et al., 2010). these technologies provide direct assistance to the elderly and are usually adopted with the thought that there is no other option but to adopt. usually these devices are installed with the ultimatum (from family and friends or health authorities) that if it is not adopted the aged person will have to leave their home and go into a residential care facility. this can force the individual into acceptance depending on how much they value their independence. australasian journal of information systems volume 18 number 3 2014 85 technology sample of adoption remote monitoring “…remote monitoring technology install and operate in seniors’ homes with the purpose of improving their quality of life and monitoring their health status…(demiris et al., 2004)” other(s): (ahn et al., 2008); (courtney, 2008); (mitzner et al., 2010); (conci et al., 2010); (wong et al., 2012); (mccreadie and tinker, 2005); (van hoof et al., 2011); (courtney et al., 2008); (mynatt et al., 2004); (goins et al., 2010); (steele et al., 2009b); (ahn et al., 2008); (steele et al., 2006); (beer and takayama, 2011); (wong et al., 2012); (or et al., 2011); (bickmore et al., 2005a); (selwyn et al., 2003) motion detecting sensor “…elderly people perceive an intelligent system, motion detecting sensor, embedded in their home, which surround humans in their daily lives and activities and should enable them to live independently longer…(zaad and allouch, 2008)” other(s): (sarkisian et al., 2003); (goins et al., 2010) surveillance camera “…using surveillance cameras at seniors’ home will assist to prevent them from accident, injury, illness or other ailment that effects mobility…(sarkisian et al., 2003) ” recording device “…installing recording devices at seniors’ home will assist to prevent them from accident, injury, illness or other ailment that effects mobility…(sarkisian et al., 2003) ” flood alarm “…elderly people install flood alarm at their home to enhance quality of life and safety purposes…(ahn et al., 2008)” telecare “…seniors use home telecare as an instrumental in improving independence and safety…(peeters et al., 2012)” other(s): (mahoney, 2010); (huang, 2011); (lai et al., 2010); (walsh and callan, 2011); (peeters et al., 2012); (demiris et al., 2013); (heinz et al., 2013) table 8 technology options: smart home and remote care we have also found that older adults have been using supportive devices for their functional ability; see table 9. medicine reminder, hearing assistance technologies and rehabilitation assistive devices are some examples of these devices. these devices may be more applicable for older people living at home, for example a medicine reminder may not be as useful in a residential care setting with health care professionals providing constant reminders and maybe even direct medicine dispensing to individual patients. again these are direct assistance technologies and may not be needed as early in a patient’s life if indirect methods such as brain training through games are used to improve memory and daily living skills. australasian journal of information systems volume 18 number 3 2014 86 technology sample of adoption medication reminder device “…older adults with multiple chronic conditions face the complex task of medication management involving multiple medications of varying doses at different times. using medication reminder devices aid older adults to manage their medications….(reeder et al., 2013)” other(s): (stojmenova et al., 2013); (wong et al., 2012) hearing assistance devices “…hearing assistance devices help elderly people hear better in order to maintain independence and enhance their quality of life…(southall et al., 2006)’ rehabilitation assistance devices “…seniors appreciate using rehabilitation assistance devices due to benefits for mobility, confidence, safety and independence... (smith et al., 2002)” other(s): (mitzner et al., 2010); (sintonen and immonen, 2013); (goins et al., 2010) table 9 technology options: supportive devices discussions and outlook the world’s population is ageing rapidly and the cost of caring for older people is also rising. an important feature of population aging is the progressive aging of the older population itself. by 2050, the world will have almost 400 million people aged 80 years or older. as such, the age group of 80 and older has become the fastest growing age segment in the global population (oecd, 2012). age is a good proxy for high support needs; 40 per cent of those currently aged 85 years and over are estimated to have a sever disability, as measured by their ability to perform their daily activities (leeson et al., 2004). this means the group of older people with high support needs is growing, as the prevalence of some physical and cognitive conditions such as dementia and movement difficulties is popular among this group. one approach to above mentioned concern is to maintain the older adults’ abilities and skills for their everyday life for a longer period of time, which will decrease the caring costs and increase the quality of life of elderly. in this regard, technologies can play a significant role. although the research in this matter has been growing and there have been different empirical evidences on contribution of assistive technologies for improvement of elderly’s quality of life, the effective potentials of technology in maintaining seniors’ abilities need further investigations. in response to the research question, “what are the potentials of assistive technologies for seniors”we found assistive technology can provide aid for elderly, but are underutilized to offer effective training for skills that may empower seniors for independent living. having reviewed the literature in assistive technologies for aged care, this paper has identified two basic categories; direct supportive and indirect empowering technologies. supportive technologies provide aid to seniors for their everyday life. a primary goal of assistive technologies is to maintain independent lifestyle of older adults. these technologies are usually designed to aid elderly for their daily life. we have found general purpose ict, robots; online information services, smart home and remote care and supporting devices have been utilized to this end. empowering technologies take a proactive approach to assist elderly for their independent living. these technologies highlight the fact that we need to not only aid seniors for their daily living activities, but also we should train them to improve their skills over the time. these group utilizing technologies such as games or social media provide facilities for elderly to maintain or improve their physical, cognitive or social skills. australasian journal of information systems volume 18 number 3 2014 87 despite all the concerns and attempts reported in literature for independent living, less is known on the differentiation of technologies that provide direct support for seniors’ daily activities and indirect support through empowering technologies that help the elderly through skills training for independent living. in this paper we consider direct supportive technologies differently to indirect, empowering technologies, for example using brain training through games to improve an older person’s resilience so they become less dependent. another example of empowering technologies is to use social media to empower seniors using internet for expand the circle of friends and avoid the social isolation issues. the term “empowerment” is a contested notion. it is a very complex term that is difficult to define. (silva, 1997) refers to empowerment as any process whereby people can gain increased control over their lives. (narayan et al., 2000) further defines empowerment as any process that enables “selfconfidence, self-direction, autonomy, and self-worth”. for the purpose of this paper, empowerment is the process of maintaining a concrete, new capability to perform some specific action (cornish, 2006). here empowerment is not seen as an increase in self-confidence or control, rather the ability to take an action. actions refer to the seniors’ daily activities. this view of empowerment implies enabling seniors to develop a range of capabilities (gigler, 2004), and refers to capability approach (sen, 1999), that can be utilized in adoption as the process of expanding a choice that lead people to the lives they have reason to value. the implication of this approach in our study is to adopt assistive technologies that build capabilities for seniors to help themselves in their daily activities as opposed to being supported by technologies. this will lead them to maintain their independent living, which they value highly. in this respect, capability theory (sen, 1999) confirms that technology adoption by seniors empowers them to change and improve their well-being which in turn leads to more independent living. however for adoption to be effective, empowerment of some skills is required. using capability theory, we can conclude that for seniors to be capable to live independently we must provide them technologies that train and empower their skills related to their everyday life activities. however, as proposed by capability theory, individual, environmental and psychological factors significantly influence this adoption. from the technology options identified earlier in this paper, we believe games and social media, because of their interactive nature, empowers elderly for their daily living. however, looking at figure 3, we can see games and social media have been given very limited attention in the research community. we therefore encourage researchers to investigate the adoption of games to empower elderly and help maintain their skills for independent living. we also suggest that the use of capability theory can help in this endeavor. games offer complex scenarios that can stimulate discussion, collaboration and imagination, or train skills such as hand-eye coordination, strategic abilities and problem solving. cognitive training is a set of procedures to help people maintain a useful level of performance in everyday tasks for different functions such as attention, memory, and reasoning, which can be also achieved by games. we believe games can provide cheap and enjoyable off-the-shelf platforms to support cognitive training. while games can empower elderly for their everyday life, physical exercises available through the game platforms e.g. xbox kinect can potentially support movement difficulties in older adults. we can think of several ways that social media can empower seniors to live on their own. a common seniors’ refrain is that they would like to stay in touch with their family and friends. social media can be used as a means by which seniors can use to stay connect with their friends and family while they might suffer from movement or driving difficulties. the importance of socializing as part of a community cannot be overstated, particularly for seniors spending much of their time living isolated at home. it can be critical for those unable to get out of the house to be with others. social media provides the opportunity to have and be a friend, to congregate without leaving the house, to never be alone even when you are the only one in the house. australasian journal of information systems volume 18 number 3 2014 88 the article concludes that much research has been conducted to directly support elderly in their everyday life while there are avenues of opportunities for empowering elderly and training them to maintain their abilities using assistive technologies that need to be discovered by researchers. limitations as is the case with almost any review, this paper has a few limitations that must be kept in mind and these are discussed below. quality dependency on the choice of keywords this review, like any other literature review, is highly dependent on the keywords that have been chosen and the databases that have been selected for the search. however, in this study, we have conducted a background and exploratory search and keywords and databases were selected according to an experimental method proposed by (dieste et al., 2009). limited resources because realist reviews embrace complexity and seeks to map out the operation of multiple theories in multiple contexts, they can easily grow quite large. compared to systematic reviews, a realist review, therefore, can only cover a limited number of papers. focusing only on publication in 2000 – 2013 the resources that have been selected in this review were only papers published during 2000 and 2013. this decision was made in order to focus of the study on recent developments in the field, it is still arguable that authors may have missed valuable technologies and adoption approaches that are applicable to today’s aged care settings. nature of the studies that have been reviewed in addition to the limitations on resources for this review, there are limitations to the nature of information that could be retrieved. most studies on the adoption of assistive technologies for aged care are not done from a realist perspective and therefore important details for a realist review about the nature of the intervention and its context may be omitted. reliability of the results in order to ensure the reliability of the results and avoid bias, we have deployed the proportional reduction in loss (prl) method. we asked four experts from north america, australia, europe and asia to review both interventions and impacting factors of adoption. however, it is still arguable that the results might have been impacted by subjectivity of the authors as well as the judges. references (boni) li, y., perkins, a., 2007. the impact of technological developments on the daily life of the elderly. technol. soc. 29, 361–368. doi:10.1016/j.techsoc.2007.04.004 abdullah, m.y., salman, a., razak, n.a., noor, n.f.m., malek, j.a., 2011. issues affecting the use of information and communication technology among the elderly: a case study on jenii, in: 2011 ieee 10th malaysia international conference on communications (micc). pp. 29–32. doi:10.1109/micc.2011.6150294 adams, n., stubbs, d., woods, v., 2005. psychological barriers to internet usage among older adults in the uk. inform. health soc. care 30, 3–17. doi:10.1080/14639230500066876 australasian journal of information systems volume 18 number 3 2014 89 ahn, m., beamish, j.o., goss, r.c., 2008. understanding older adults’ attitudes and adoption of residential technologies. fam. consum. sci. res. j. 36, 243–260. doi:10.1177/1077727x07311504 aula, a., 2005. user study on older adults’ use of the web and search engines. univers. access inf. soc. 4, 67–81. doi:10.1007/s10209-004-0097-7 australian nursing federation, 2012. aged care can’t wait report. australian nursing federation. barnard, y., bradley, m.d., hodgson, f., lloyd, a.d., 2013. learning to use new technologies by older adults: perceived difficulties, experimentation behaviour and usability. comput. hum. behav. 29, 1715–1724. doi:10.1016/j.chb.2013.02.006 beer, j.m., takayama, l., 2011. mobile remote presence systems for older adults: acceptance, benefits, and concerns, in: 2011 6th acm/ieee international conference on human-robot interaction (hri)., pp. 19–26. bickmore, t.w., caruso, l., clough-gorr, k., 2005a. acceptance and usability of a relational agent interface by urban older adults, in: chi ’05 extended abstracts on human factors in computing systems, chi ea ’05. acm, new york, ny, usa, pp. 1212–1215. doi:10.1145/1056808.1056879 bickmore, t.w., caruso, l., clough-gorr, k., heeren, t., 2005b. “it”s just like you talk to a friend’ relational agents for older adults. interact. comput. 17, 711–735. doi:10.1016/j.intcom.2005.09.002 biljon, j. van, renaud, k., 2008. a qualitative study of the applicability of technology acceptance models to senior mobile phone users, in: song, i.-y., piattini, m., chen, y.-p.p., hartmann, s., grandi, f., trujillo, j., opdahl, a.l., ferri, f., grifoni, p., caschera, m.c., rolland, c., woo, c., salinesi, c., zimányi, e., claramunt, c., frasincar, f., houben, g.-j., thiran, p. (eds.), advances in conceptual modeling – challenges and opportunities, lecture notes in computer science. springer berlin heidelberg, pp. 228–237. boontarig, w., chutimaskul, w., chongsuphajaisiddhi, v., papasratorn, b., 2012. factors influencing the thai elderly intention to use smartphone for e-health services, in: 2012 ieee symposium on humanities, science and engineering research (shuser). pp. 479–483. doi:10.1109/shuser.2012.6268881 braun, m.t., 2013. obstacles to social networking website use among older adults. comput. hum. behav. 29, 673–680. doi:10.1016/j.chb.2012.12.004 burnett, j.s., mitzner, t.l., charness, n., rogers, w.a., 2011. understanding predictors of computer communication technology use by older adults. proc. hum. factors ergon. soc. annu. meet. 55, 172–176. doi:10.1177/1071181311551036 cameron, d.d., marquis, d.r., webster, b., 2001. older adults perceptions, experiences and anxieties with emerging technologies. australas. j. ageing 20, 50–56. doi:10.1111/j.17416612.2001.tb00399.x carpenter, b.d., buday, s., 2007. computer use among older adults in a naturally occurring retirement community. comput. hum. behav. 23, 3012–3024. doi:10.1016/j.chb.2006.08.015 choi, n.g., dinitto, d.m., 2013. internet use among older adults: association with health needs, psychological capital, and social capital. j. med. internet res. 15, e97. doi:10.2196/jmir.2333 chu, r.j., 2010. how family support and internet self-efficacy influence the effects of e-learning among higher aged adults – analyses of gender and age differences. comput. educ. 55, 255–264. doi:10.1016/j.compedu.2010.01.011 australasian journal of information systems volume 18 number 3 2014 90 chu yew yee, s.l., duh, h.b.-l., quek, f., 2010. investigating narrative in mobile games for seniors, in: proceedings of the sigchi conference on human factors in computing systems, chi ’10. acm, new york, ny, usa, pp. 669–672. doi:10.1145/1753326.1753424 chung, j.e., park, n., wang, h., fulk, j., mclaughlin, m., 2010. age differences in perceptions of online community participation among non-users: an extension of the technology acceptance model. comput. hum. behav. 26, 1674–1684. doi:10.1016/j.chb.2010.06.016 conci, m., pianesi, f., zancanaro, m., 2009. useful, social and enjoyable: mobile phone adoption by older people, in: gross, t., gulliksen, j., kotzé, p., oestreicher, l., palanque, p., prates, r.o., winckler, m. (eds.), human-computer interaction – interact 2009, lecture notes in computer science. springer berlin heidelberg, pp. 63–76. conci, m., pianesi, f., zancanaro, m., 2010. older adults’ attitude towards a monitoring technology, in: proceedings of the 28th annual european conference on cognitive ergonomics, ecce ’10. acm, new york, ny, usa, pp. 143–146. doi:10.1145/1962300.1962328 copolillo, a., prohaska, t.r., 2001. older adults’ mobility device use after in-patient rehabilitation: intention, actual use, and need. phys. occup. ther. geriatr. 19, 35–48. doi:10.1300/j148v19v04_03 cornish, f., 2006. empowerment to participate: a case study of participation by indian sex workers in hiv prevention. j. community appl. soc. psychol. 16, 301–315. doi:10.1002/casp.866 courtney, k.l., 2008. privacy and senior willingness to adopt smart home information technology in residential care facilities. methods inf. med. doi:10.3414/me9104 courtney, k.l., demiris, g., rantz, m., skubic, m., 2008. needing smart home technologies: the perspectives of older adults in continuing care retirement communities. inform. prim. care 16, 195–201. czaja, s.j., charness, n., fisk, a.d., hertzog, c., nair, s.n., rogers, w.a., sharit, j., 2006. factors predicting the use of technology: findings from the center for research and education on aging and technology enhancement (create). psychol. aging 21, 333–352. doi:10.1037/08827974.21.2.333 czaja, s.j., lee, c.c., nair, s.n., sharit, j., 2008. older adults and technology adoption. proc. hum. factors ergon. soc. annu. meet. 52, 139–143. doi:10.1177/154193120805200201 demiris, g., rantz, m.j., aud, m.a., marek, k.d., tyrer, h.w., skubic, m., hussam, a.a., 2004. older adults’ attitudes towards and perceptions of “smart home” technologies: a pilot study. med inf. 29, 87–94. demiris, g., thompson, h., boquet, j., le, t., chaudhuri, s., chung, j., 2013. older adults’ acceptance of a community-based telehealth wellness system. inform. health soc. care 38, 27–36. doi:10.3109/17538157.2011.647938 dieste, o., grimán, a., juristo, n., 2009. developing search strategies for detecting relevant experiments. empir. softw. eng. 14, 513–539. doi:10.1007/s10664-008-9091-7 doha, 2008. australian dementia resources guide. australia. eastman, j.k., iyer, r., 2004. the elderly’s uses and attitudes towards the internet. j. consum. mark. 21, 208–220. doi:10.1108/07363760410534759 el-attar, t., gray, j., nair, s.n., ownby, r., czaja, s.j., 2005. older adults and internet health information seeking. proc. hum. factors ergon. soc. annu. meet. 49, 163–166. doi:10.1177/154193120504900203 era, 2012. the excellence in research for australia (era) initiative. ghapanchi, a.h., aurum, a., 2011. antecedents to it personnel’s intentions to leave: a systematic literature review. j. syst. softw. 84, 238–249. doi:10.1016/j.jss.2010.09.022 australasian journal of information systems volume 18 number 3 2014 91 gigler, b.-s., 2004. including the excludedcan icts empower poor communities? towards an alternative evaluation framework based on the capability approach. presented at the 4 th international conference on the capability approach, pavia, italy. giuliani, m.v., scopelliti, m., fornara, f., 2005. elderly people at home: technological help in everyday activities, in: ieee international workshop on robot and human interactive communication, 2005. roman 2005, pp. 365–370. doi:10.1109/roman.2005.1513806 goins, r.t., spencer, s.m., goli, s., rogers, j.c., 2010. assistive technology use of older american indians in a southeastern tribe: the native elder care study. j. am. geriatr. soc. 58, 2185– 2190. doi:10.1111/j.1532-5415.2010.03140.x guo, x., sun, y., wang, n., peng, z., yan, z., 2013. the dark side of elderly acceptance of preventive mobile health services in china. electron. mark. 23, 49–61. doi:10.1007/s12525-012-0112-4 haase, r., schultheiss, t., kempcke, r., thomas, k., ziemssen, t., 2012. use and acceptance of electronic communication by patients with multiple sclerosis: a multicenter questionnaire study. j. med. internet res. 14, e135. doi:10.2196/jmir.2133 häikiö, j., wallin, a., isomursu, m., ailisto, h., matinmikko, t., huomo, t., 2007. touch-based user interface for elderly users, in: proceedings of the 9th international conference on human computer interaction with mobile devices and services, mobilehci ’07. acm, new york, ny, usa, pp. 289–296. doi:10.1145/1377999.1378021 hanson, v.l., 2010. influencing technology adoption by older adults. interact. comput. 22, 502–509. doi:10.1016/j.intcom.2010.09.001 hardill, i., olphert, c.w., 2012. staying connected: exploring mobile phone use amongst older adults in the uk. geoforum 43, 1306–1312. doi:10.1016/j.geoforum.2012.03.016 heerink, m., 2011. exploring the influence of age, gender, education and computer experience on robot acceptance by older adults, in: proceedings of the 6th international conference on humanrobot interaction, hri ’11. acm, new york, ny, usa, pp. 147–148. doi:10.1145/1957656.1957704 heerink, m., ben, k., evers, v., wielinga, b., 2008. the influence of social presence on acceptance of a companion robot by older people. j. phys. agents 2, 33–40. heerink, m., krose, b., evers, v., wielinga, b., 2006. the influence of a robot’s social abilities on acceptance by elderly users, in: the 15th ieee international symposium on robot and human interactive communication, 2006. roman 2006., pp. 521–526. doi:10.1109/roman.2006.314442 heerink, m., kröse, b., evers, v., wielinga, b., 2010. assessing acceptance of assistive social agent technology by older adults: the almere model. int. j. soc. robot. 2, 361–375. doi:10.1007/s12369-010-0068-5 heerink, m., krose, b., wielinga, b., evers, v., 2008. enjoyment, intention to use and actual use of a conversational robot by elderly people, in: 2008 3rd acm/ieee international conference on human-robot interaction (hri). pp. 113–119. heinz, m., martin, p., margrett, j.a., yearns, m., franke, w., yang, h.-i., wong, j., chang, c.k., 2013. perceptions of technology among older adults. j. gerontol. nurs. 39, 42–51. doi:10.3928/00989134-20121204-04 hernández-encuentra, e., pousada, m., gómez-zúñiga, b., 2009. ict and older people: beyond usability. educ. gerontol. 35, 226–245. doi:10.1080/03601270802466934 huang, j.-c., 2011. exploring the acceptance of telecare among senior citizens: an application of backpropagation network. telemed. j. e-health off. j. am. telemed. assoc. 17, 111–117. doi:10.1089/tmj.2010.0118 australasian journal of information systems volume 18 number 3 2014 92 karavidas, m., lim, n.k., katsikas, s.l., 2005. the effects of computers on older adult users. comput. hum. behav. 21, 697–711. doi:10.1016/j.chb.2004.03.012 kiel, j.m., 2005. the digital divide: internet and e-mail use by the elderly. med. inform. internet med. 30, 19–23. doi:10.1080/14639230500066900 klamer, t., ben allouch, s., 2010. acceptance and use of a social robot by elderly users in a domestic environment, in: pervasive computing technologies for healthcare (pervasivehealth), pp. 1– 8. doi:10.4108/icst.pervasivehealth2010.8892 kubik, s., 2009. motivations for cell phone use by older americans. gerontechnology 8. doi:10.4017/gt.2009.08.03.007.00 lai, c.k.y., chung, j.c.c., leung, n.k.l., wong, j.c.t., mak, d.p.s., 2010. a survey of older hong kong people’s perceptions of telecommunication technologies and telecare devices. j. telemed. telecare 16, 441–446. doi:10.1258/jtt.2010.090905 lam, j.c.y., lee, m.k.o., 2006. digital inclusiveness--longitudinal study of internet adoption by older adults. j. manag. inf. syst. 22, 177–206. doi:10.2753/mis0742-1222220407 laver, k., ratcliffe, j., george, s., burgess, l., crotty, m., 2011. is the nintendo wii fit really acceptable to older people?: a discrete choice experiment. bmc geriatr. 11, 1–6. doi:10.1186/1471-2318-11-64 law, m., stewart, d., letts, l., pollock, n., bosch, j., westmorland, m., 1998a. guidelines for critical review formquantitative studies. law, m., stewart, d., letts, l., pollock, n., bosch, j., westmorland, m., 1998b. guidelines for critical review formqualitative studies. leeson, g.w., harper, s., levin, s., britain, g., 2004. independent living in later life: literature review. dwp. macaulay, a.c., jagosh, j., seller, r., henderson, j., cargo, m., greenhalgh, t., wong, g., salsberg, j., green, l.w., herbert, c.p., pluye, p., 2011. assessing the benefits of participatory research: a rationale for a realist review. glob. health promot. 18, 45–48. doi:10.1177/1757975910383936 mahoney, d.f., 2010. an evidence-based adoption of technology model for remote monitoring of elders’ daily activities. ageing int. 36, 66–81. doi:10.1007/s12126-010-9073-0 mallenius, s., rossi, m., tuunainen, v.k., 2007. factors affecting the adoption and use of mobile devices and services by elderly people – results from a pilot study. marshall, m., 1997a. state of the art in dementia care london. u.k. marshall, m., 1997b. state of the art in dementia care london. u.k. mccloskey, d.w., 2006. the importance of ease of use, usefulness, and trust to online consumers. j. organ. end user comput. 18, 47–65. doi:10.4018/joeuc.2006070103 mccreadie, c., tinker, a., 2005. the acceptability of assistive technology to older people. ageing soc. 25, 91–110. doi:10.1017/s0144686x0400248x mckay, s.m., maki, b.e., 2010. attitudes of older adults toward shooter video games: an initial study to select an acceptable game for training visual processing. gerontechnology int. j. fundam. asp. technol. serve ageing soc. 9, 5–17. doi:10.4017/gt.2010.09.01.001.00 mclaughlin, a., gandy, m., allaire, j., whitlock, l., 2012. putting fun into video games for older adults. ergon. des. q. hum. factors appl. 20, 13–22. doi:10.1177/1064804611435654 melenhorst, a.s., bouwhuis, d.g., 2004. when do older adults consider the internet? an exploratory study of benefit perception. gerontechnology 3. doi:10.4017/gt.2004.03.02.004.00 australasian journal of information systems volume 18 number 3 2014 93 melenhorst, a.-s., rogers, w.a., bouwhuis, d.g., 2006. older adults’ motivated choice for technological innovation: evidence for benefit-driven selectivity. psychol. aging 21, 190–195. doi:10.1037/0882-7974.21.1.190 melenhorst, a.-s., rogers, w.a., caylor, e.c., 2001. the use of communication technologies by older adults: exploring the benefits from the user’s perspective. proc. hum. factors ergon. soc. annu. meet. 45, 221–225. doi:10.1177/154193120104500305 miskelly, f.g., 2001. assistive technology in elderly care. age ageing 30, 455–458. mitzner, t.l., boron, j.b., fausset, c.b., adams, a.e., charness, n., czaja, s.j., dijkstra, k., fisk, a.d., rogers, w.a., sharit, j., 2010. older adults talk technology: technology usage and attitudes. comput. hum. behav. 26, 1710–1721. doi:10.1016/j.chb.2010.06.020 mitzner, t.l., smarr, c.-a., beer, j.m., chen, t.l., springman, j.m., prakash, a., kemp, c.c., rogers, w.a., 2011. older adults’ acceptance of assistive robots for the home. morris, a., goodman, j., brading, h., 2007. internet use and non-use: views of older users. univers. access inf. soc. 6, 43–57. doi:10.1007/s10209-006-0057-5 mynatt, e.d., melenhorst, a.-s., fisk, a.d., rogers, w.a., 2004. aware technologies for aging in place: understanding user needs and attitudes. ieee pervasive comput. 3, 36–41. doi:10.1109/mprv.2004.1316816 narayan, d., patel, r., schafft, k., rademacher, a., koch-schulte, s., 2000. voices of the poor: can anyone hear us? oxford university press for the world bank, new york. nayak, l.u.s., priest, l., white, a.p., 2010. an application of the technology acceptance model to the level of internet usage by older adults. univers. access inf. soc. 9, 367–374. doi:10.1007/s10209-009-0178-8 neven, l., 2010. “but obviously not for me”: robots, laboratories and the defiant identity of elder test users. sociol. health illn. 32, 335–347. doi:10.1111/j.1467-9566.2009.01218.x neves, b.b., 2012. too old for technology? how the elderly of lisbon use and perceive ict. j. community inform. 8. ng, j., tan, o., wong, a., kiat, k.w., 2012. older adults’ attitudes toward homes service robots, in: proceedings of the workshop at siggraph asia, wasa ’12. acm, new york, ny, usa, pp. 87–90. doi:10.1145/2425296.2425312 oecd, 2012. world population ageing: 1959-2050. paris. or, c.k.l., karsh, b.-t., severtson, d.j., burke, l.j., brown, r.l., brennan, p.f., 2011. factors affecting home care patients’ acceptance of a web-based interactive self-management technology. j. am. med. inform. assoc. 18, 51–59. doi:10.1136/jamia.2010.007336 pan, s., jordan-marsh, m., 2010. internet use intention and adoption among chinese older adults: from the expanded technology acceptance model perspective. comput. hum. behav. 26, 1111–1119. doi:10.1016/j.chb.2010.03.015 pawson, r., greenhalgh, t., harvey, g., walshe, k., 2005. realist review – a new method of systematic review designed for complex policy interventions. j. health serv. res. policy 10, 21–34. doi:10.1258/1355819054308530 peeters, j.m., de veer, a.j.e., van der hoek, l., francke, a.l., 2012. factors influencing the adoption of home telecare by elderly or chronically ill people: a national survey. j. clin. nurs. 21, 3183– 3193. doi:10.1111/j.1365-2702.2012.04173.x phang, c.w., sutanto, j., kankanhalli, a., li, y., tan, b.c.y., teo, h.-h., 2006. senior citizens’ acceptance of information systems: a study in the context of e-government services. ieee trans. eng. manag. 53, 555–569. doi:10.1109/tem.2006.883710 australasian journal of information systems volume 18 number 3 2014 94 porteus, j., brownse, s.j., 2000. exploring technologies for independent living for older people. anchor trust. price, m.m., pak, r., müller, h., stronge, a., 2013. older adults’ perceptions of usefulness of personal health records. univers. access inf. soc. 12, 191–204. doi:10.1007/s10209-012-0275-y reeder, b., demiris, g., marek, k.d., 2013. older adults’ satisfaction with a medication dispensing device in home care. inform. health soc. care. doi:10.3109/17538157.2012.741084 renaud, k., van biljon, j., 2008. predicting technology acceptance and adoption by the elderly: a qualitative study, in: proceedings of the 2008 annual research conference of the south african institute of computer scientists and information technologists on it research in developing countries: riding the wave of technology, saicsit ’08. acm, new york, ny, usa, pp. 210– 219. doi:10.1145/1456659.1456684 rosenberg, l., kottorp, a., winblad, b., nygård, l., 2009. perceived difficulty in everyday technology use among older adults with or without cognitive deficits [www document]. url http://informahealthcare.com/doi/abs/10.3109/11038120802684299 (accessed 6.21.13). rust, r.t., cooil, b., 1994. reliability measures for qualitative data: theory and implications. j. mark. res. 31, 1–14. doi:10.2307/3151942 ryu, m.-h., kim, s., lee, e., 2009. understanding the factors affecting online elderly user’s participation in video ucc services. comput. hum. behav. 25, 619–632. doi:10.1016/j.chb.2008.08.013 salovaara, a., lehmuskallio, a., hedman, l., valkonen, p., näsänen, j., 2010. information technologies and transitions in the lives of 55–65-year-olds: the case of colliding life interests. int. j. hum.-comput. stud. 68, 803–821. doi:10.1016/j.ijhcs.2010.06.007 sarkisian, g.p., melenhorst, a.-s., rogers, w.a., fisk, a.d., 2003. older adults’ opinions of a technology-rich home environment: conditional and unconditional device acceptance. proc. hum. factors ergon. soc. annu. meet. 47, 1800–1804. doi:10.1177/154193120304701505 sayago, s., sloan, d., blat, j., 2011. everyday use of computer-mediated communication tools and its evolution over time: an ethnographical study with older people. interact. comput. 23, 543–554. doi:10.1016/j.intcom.2011.06.001 selwyn, n., 2004. the information aged: a qualitative study of older adults’ use of information and communications technology. j. aging stud. 18, 369–384. doi:10.1016/j.jaging.2004.06.008 selwyn, n., gorard, s., furlong, j., madden, l., 2003. older adults’ use of information and communications technology in everyday life. ageing soc. 23, 561–582. doi:10.1017/s0144686x03001302 sen, a., 1999. commodities and capabilities (oup catalogue). oxford university press. silva, l., 1997. power and politics in the adoption of information systems by organisations: the case of a research centre in latin america (phd). the london school of economics and political science (lse). singh, h., fox, s.a., petersen, n.j., shethia, a., street, r.l., 2009. older patients’ enthusiasm to use electronic mail to communicate with their physicians: cross-sectional survey. j. med. internet res. 11. doi:10.2196/jmir.1143 sintonen, s., immonen, m., 2013. telecare services for aging people: assessment of critical factors influencing the adoption intention. comput. hum. behav. 29, 1307–1317. doi:10.1016/j.chb.2013.01.037 australasian journal of information systems volume 18 number 3 2014 95 smarr, c.-a., prakash, a., beer, j.m., mitzner, t.l., kemp, c.c., rogers, w.a., 2012. older adults’ preferences for and acceptance of robot assistance for everyday living tasks. proc. hum. factors ergon. soc. annu. meet. 56, 153–157. doi:10.1177/1071181312561009 smith, r., quine, s., anderson, j., black, k., 2002. assistive devices: self-reported use by older people in victoria. aust. health rev. 25, 169–177. southall, k., gagné, j.-p., leroux, t., 2006. factors that influence the use of assistance technologies by older adults who have a hearing loss. int. j. audiol. 45, 252–259. doi:10.1080/14992020500258586 steele, r., lo, a., secombe, c., wong, y.k., 2009a. elderly persons’ perception and acceptance of using wireless sensor networks to assist healthcare. int. j. med. inf. 78, 788–801. doi:10.1016/j.ijmedinf.2009.08.001 steele, r., lo, a., secombe, c., wong, y.k., 2009b. elderly persons’ perception and acceptance of using wireless sensor networks to assist healthcare. int. j. med. inf. 78, 788–801. doi:10.1016/j.ijmedinf.2009.08.001 steele, r., secombe, c., brookes, w., 2006. using wireless sensor networks for aged care: the patient’s perspective, in: pervasive health conference and workshops, 2006. pp. 1–10. doi:10.1109/pcthealth.2006.361661 stojmenova, e., debevc, m., zebec, l., imperl, b., 2013. assisted living solutions for the elderly through interactive tv. multimed. tools appl. 1–15. doi:10.1007/s11042-011-0972-1 sum, s., mathews, r.m., hughes, i., 2009. participation of older adults in cyberspace: how australian older adults use the internet. australas. j. ageing 28, 189–193. doi:10.1111/j.17416612.2009.00374.x sum, s., mathews, r.m., hughes, i., campbell, a., 2008. internet use and loneliness in older adults. cyberpsychol. behav. 11, 208–211. doi:10.1089/cpb.2007.0010 umemuro, h., 2004. lowering elderly japanese users’ resistance towards computers by using touchscreen technology. univers. access inf. soc. 3, 276–288. doi:10.1007/s10209-004-0098-6 umemuro, h., 2004. computer attitudes, cognitive abilities, and technology usage among older japanese adults. gerontechnology 3. doi:10.4017/gt.2004.03.02.002.00 van hoof, j., kort, h.s.m., rutten, p.g.s., duijnstee, m.s.h., 2011. ageing-in-place with the use of ambient intelligence technology: perspectives of older users. int. j. med. inf. 80, 310–331. doi:10.1016/j.ijmedinf.2011.02.010 vichitvanichphong, s., talaei-khoei, a., kerr, d., ghapanchi, a.h., 2013. literature survey “challenges in assistive technologies for aged care”. hawaiian conference of system sciences (hicss-47). walsh, k., callan, a., 2011. perceptions, preferences, and acceptance of information and communication technologies in older-adult community care settings in ireland: a casestudy and ranked-care program analysis. ageing int. 36, 102–122. doi:10.1007/s12126-0109075-y wilkowska, w., ziefle, m., 2009. which factors form older adults’ acceptance of mobile information and communication technologies?, in: holzinger, a., miesenberger, k. (eds.), hci and usability for e-inclusion, lecture notes in computer science. springer berlin heidelberg, pp. 81–101. wong, a.m.k., chang, w.-h., ke, p.-c., huang, c.-k., tsai, t.-h., chang, h.-t., shieh, w.-y., chan, h.-l., chen, c.-k., pei, y.-c., 2012. technology acceptance for an intelligent comprehensive interactive care (icic) system for care of the elderly: a survey-questionnaire study. plos one 7, e40591. doi:10.1371/journal.pone.0040591 australasian journal of information systems volume 18 number 3 2014 96 wood, e., willoughby, t., rushing, a., bechtel, l., gilbert, j., 2005. use of computer input devices by older adults. j. appl. gerontol. 24, 419–438. doi:10.1177/0733464805278378 xue, l., yen, c.c., chang, l., chan, h.c., tai, b.c., tan, s.b., duh, h.b.l., choolani, m., 2012. an exploratory study of ageing women’s perception on access to health informatics via a mobile phone-based intervention. int. j. med. inf. 81, 637–648. doi:10.1016/j.ijmedinf.2012.04.008 zaad, l., allouch, s.b., 2008. the influence of control on the acceptance of ambient intelligence by elderly people: an explorative study, in: aarts, e., crowley, j.l., ruyter, b. de, gerhäuser, h., pflaum, a., schmidt, j., wichert, r. (eds.), ambient intelligence, lecture notes in computer science. springer berlin heidelberg, pp. 58–74. paper title (paper title style) australasian journal of information systems salim, sedera et al 2015, vol. 19, pp. s219-s254 moving from evaluation to trial moving from evaluation to trial: how do smes start adopting cloud erp? siti aisyah salim queensland university of technology sitiaisyah.salim@hdr.qut.edu.au darshana sedera queensland university of technology sukanlaya sawang queensland university of technology abdulrahman h. e. alarifi queensland university of technology maura atapattu queensland university of technology abstract the advent of cloud technology involving low subscription overheads cost has provided small and medium-sized enterprises (smes) with the opportunity to adopt new cloud-based corporate-wide systems (i.e., cloud erp). this technology, operating through subscriptionbased services, has now provided smes with a complete range of it applications that were once restricted to large organisations. as anecdotal evidences suggest, smes are increasingly adopting cloud-based erp software. the selection of an erp is a complex process involving multiple stages and stakeholders, suggesting the importance of closer examination of cloud erp adoption in smes. yet, prior studies have predominantly treated technology adoption as a single activity and largely ignored the issue of erp adoption in smes. understanding of the process nature of the adoption and the factors that are important in each stage of the adoption potentially may result in guiding smes to make well-informed decisions throughout the erp selection process. thus, our study proposes that the adoption of cloud erp should be examined as a multi-stage process. using the theory of planned behaviour (tpb) and ettlie’s adoption stages, as well as employing data gathered from 162 owners of smes, our findings show that the factors that influence the intention to adopt cloud erp vary significantly across adoptive stages. keywords: sme decision-makers; cloud erp; tpb; multi-stage 1 introduction the employment of corporate-wide systems, such as enterprise resource planning (erp) systems, appears to be the most significant development in business use (sedera and gable 2010) and has been discussed heavily within information systems (is) research since it was first developed in the 1990s (klaus et al. 2000). interestingly, is researchers have predominantly looked at the adoption of erp by large organisations (e.g., soh et al. 2000, sykes et al. 2014) with much less attention being paid to small and medium-sized enterprise (sme) adoption (eden et al. 2014). the advent of software-as-a-service (saas) models through cloud technology (sharma and sood 2011) has given smes the opportunity to embrace corporate-wide systems (sedera et al. 2014, walther et al. 2015). such engagement of technology through subscription–based technology access and minimal infrastructure has given smes a complete range of it applications that were once restricted to their larger counterparts. a recent survey by gartner group on the future of corporate-wide system adoption found that 47% of the firms planned to move to cloud-based systems within the next five years and the majority of those were smes (rayner 2014). according to fox et al. (2009), the increase in adoption rates among smes can s219 australasian journal of information systems salim, sedera et al 2015, vol. 19, pp. s219-s254 moving from evaluation to trial be attributed to the benefits of deploying erp systems through cloud technology (i.e., cloud erp) that have enabled lower subscription costs (forrest and barthold 2009) while maintaining the same functionality as on-premise erp systems (koslowski and strüker 2011). further, given the typical setting of an sme that does not have an it department, the benefit of consulting services provided by the cloud service provider could be obtained. for the purposes of this study, the definition of cloud erp is derived through an amalgam of the national institute of standards and technology (nist) (2011) conceptualisation of cloud computing and markus et al.’s (2003) definition of erp. herein, cloud erp is defined as commercial software packages that enable the integration of business processes and transaction-oriented data throughout the organisation using a model that enables ubiquitous, convenient, on-demand network access within minimal management effort or service provider reaction. parallel with the increasing number of cloud erp adoptions among smes, there has been a growing recognition of the importance of revealing technology adoption as a process. a number of scholars have presented their studies through a process concept (e.g., campbell et al. 2013, choudhury and karahanna 2008, karahanna et al. 1999, pavlou and fygenson 2006) by demonstrating that the factors influencing technology adoption fluctuate as the adoption process progresses. however, the focus of existing studies of technology adoption have mainly been limited to the following aspects, namely: (i) simple technology adoption (e.g., website adoption and transactional information systems); and (ii) technology adoption within a single broad stage (e.g., pre-adoption, adoption and post-adoption). as damanpour and schneider (2006) alluded, the current approaches of technology adoption studies are neither able to explain the complex nature of corporate-wide systems adoption nor differentiate the changes in the importance level of each factor in different stages of the adoption process. although there are a few studies (e.g., del aguila-obra and padilla-melendez 2006, dholakia and kshetri 2004) that have looked on organisational technology adoption, these studies are now dated and focus on simple technology rather than complex systems such as cloud erp. the criticality of clear comprehension of the factors that influence corporate-wide systems adoption (i.e., pre-use) for is research is highlighted by a number of reasons including: (i) the possibility of making the wrong decision during the adoption process (winters et al. 2008); (ii) the inability to observe the change in critical adoption factors during the adoption process (law and ngai 2007, salim 2013); (iii) the inability to achieve adequate vendor involvement during the adoption process (willcocks and sykes 2000); (iv) the possibility for the firm to switch to an alternative technology from another vendor when not receiving adequate information during the early stage of the adoption process (dubey and wagle 2007); and (v) from the vendor’s side, the inability to understand the reasons why some firms choose to drop out of the adoption process (muscatello et al. 2003). however, studies investigating the comprehensive view of corporate-wide system adoption are still scant. table 1 (appendix a) shows the studies that specifically investigate corporate-wide system adoption in organisations, with most of the studies focusing on large firms and a single stage view of adoption factors (see columns b and c). motivated by the paucity of research explaining corporate-wide system adoption as a process view and the increasing number of smes abandoning the adoption half way through, this study investigates the factors that are critical across the adoption process in the context of smes. while there is no conceptual framework resulting from this research, one of the contributions of this paper is to combine two complementary theories. first, the critical adoption factors are identified through the lens of the theory of planned behaviour (tpb) (ajzen 1991) and the changes in the level of importance of adoption factors are observed through ettlie’s (1980) multi-stage adoption model. from the five stages that ettlie (1980) has introduced, only two stages, namely, the “evaluation” and “trial” were selected to be investigated further in this study.the selection of the evaluation and trial stages was based on the following reasons: (i) the evaluation and trial stages are the most critical in the process of adoption (howard and sheth 1969); (ii) empirical studies have concluded that a large number of firms drop out of the adoption process as a result of receiving limited information on the system to be adopted (e.g., s220 australasian journal of information systems salim, sedera et al 2015, vol. 19, pp. s219-s254 moving from evaluation to trial arthur 1989, au and kauffman 2003); (iii) the evaluation and trial stages have apparent differences (i.e., before and after experiencing the use of cloud erp), which enables fluctuations in the significance of the adoption factors to be observed in two different conditions; and (iv) completion of the evaluation and trial stages will lead smes to the final adoption stages (i.e., prior to use) where they are expected to decide to either go ahead or drop out of the process. this leads to the main research question posed in the present study: what are the factors that influence smes to adopt cloud erp during the evaluation and trial stages? the study applied tpb (i.e., individual-level theory) as the adoption decisions in smes are made by a single person (i.e., the owner) who represents the firm’s voice wholly. thus, the unit of analysis of the present study is the individual level of adoption. this study helps to understand corporate technology adoption of smes through three specific theoretical contributions. first, it provides a better understanding of how smes adopt cloud erp by evidencing that each of the factors in each of the adoption stages has a different level of significance. second, it presents the justification as to why certain factors are either superior or inferior in certain adoption stages. third, the study demonstrates the use of a new theoretical lens of viewing cloud erp adoption as a multi-stage process, rather than focusing on a single phase. this paper proceeds as follows. first, the prior research that discusses adoption stages in several contexts, including corporate-wide systems (e.g., verville and halingten 2003), is outlined. second, the methodology used for developing the study hypotheses is presented. third, the analytical methods used to validate the scales and test the research model are presented, followed by the results of the study. the paper concludes with a discussion of the theoretical and practical implications and the limitations of the study, together with a discussion of future research directions and conclusions reached. 2 theoretical development 2.1 studies related to technology adoption although there is a substantial body of research on technology adoption, in-depth discussion on technology adoption as a process is still limited. presenting technology adoption as a process commenced in the 1960s when rogers (1995) introduced a model comprising five adoption stages. later, zaltman et al. (1973) reviewed and updated rogers’ model making it more consistent. the updated model then appeared in a few technology adoption studies (e.g., ettlie 1980, fichman and kemerer 1997, salim et al. 2014). as discussed in fichman and kemerer (2012), the term “technology adoption” explains a broader spectrum of activities starting from awareness of the technology through to the widespread deployment of the technology in the organisation.this view is in line with the broader stages of technology adoption, including pre-adoption, adoption and post-adoption, that have been discussed in prior literature (e.g., aguirre-urreta and marakas 2012, schwarz et al. 2014). conversely, some studies refer to these stages as initiation, adoption decision and implementation, respectively (e.g., pierce and delbecq 1977, rogers 1995, zmud 1982). however, this study restricts its focus to the adoption stages that happen within the adoption process (which falls in the “prior-touse” period). these include: initial awareness of the existence of the technology, interest in the benefits of the technology, evaluation of the technology, trial of the technology and, finally, commitment to adopt. this study uses the definition of adoption as proposed by frambach and schillewaert (2002). in this particular definition, adoption refers to the sequence of stages through which an innovation (i.e., new technology) passes before the new product, service or idea will be accepted by a potential adopter prior to being used. further, in this study, the terms adoption and adoption process will be used interchangeably to refer to the definition proposed by frambach and schillewaert (2002). similarly, adoption stages refer to the sequence of stages that happen within the adoption process. there are several studies which have discussed the stages of adoption (e.g., ettlie 1980, guo and barnes 2011, verville and halingten 2003) where the number of stages range from five (e.g., shoham 1992) to seven (e.g., mintzberg et al. 1976). most studies agree on five common s221 australasian journal of information systems salim, sedera et al 2015, vol. 19, pp. s219-s254 moving from evaluation to trial stages spanning the technology adoption process, specifically: (1) awareness / need identification / knowledge, (2) interest / information search / product brokering, (3) evaluation / selection / negotiation, (4) trial / choice / decision, and (5) commitment / purchase / implementation / adoption. more concisely, they have discussed the following stages: awareness interest evaluation trial commitment, where these represent the full adoption process. for the purposes of this study, the five adoption stages, namely, awareness, interest, evaluation, trial and commitment, in the model proposed by ettlie (1980) are the best choice due to the following reasons: (i) it provides a detailed breakdown of the decision-making process yet it is not exhaustive; and (ii) it matches the industry roadmap of technology adoption stages (e.g., sap value). fichman and kemerer (1997) have made two changes to ettlie’s (1980) technology adoption stages model. first, ettlie’s sixth stage was not included in the new model because the present study focuses on the stages before the decision is made (adoption). second, the fifth stage of ettlie’s technology adoption stage has been changed from adoption to commitment .the new term can be deemed as more appropriate to fit the definition that ettlie provides. accordingly, this study follows the stages used in fichman and kemerer (1997). the details of ettlie’s adoption stages are described as follows. 2.1.1 awareness awareness corresponds to the first adoption stage and commences when one or more individuals in an organisation identify an important problem and seek a solution (rogers 1995). in many situations, problems can be solved by finding a suitable technology. the state of being aware could also happen fortuitously, that is, without realising either the problem and/or the need. for smes, the stage of being aware could come about from a diversity of sources including exposure to network peers, business partners and government regulations. thus, it is suggested that being aware (at the awareness stage) can occur due to the influence of external factors (rogers 1995). according to ettlie (1980), awareness is the stage at which the organisation might be aware of the existence of the potential technology to be adopted in the market, however, the relevant information has not yet been obtained. being aware will then lead to the next stage of adoption which comprises searching for and finding information. 2.1.2 interest in the second stage of adoption, individuals within the organisation start to gather as much information as possible on the particular technology which is the subject of interest (ettlie 1980). ettlie (1980) defined this stage as interest, whilst engel et al. (1978), together with verville and halingten (2003), referred to it as search and information search, respectively. at this stage, the firm’s representative (who in this context is the decision-maker) will start to, specifically; familiarise himself/herself with the technology (de bruyn and lilien 2008), pay attention to the advertisements (butler and peppard 1998), and eventually find the most appropriate vendor of the technology (gopalakrishna and lilien 1995). 2.1.3 evaluation once all the information has been gathered, the relative advantages and disadvantages (ettlie 1980) of cloud erp will be compared. ettlie (1980) referred to the action of comparing and contrasting as evaluation. engel et al. (1978) labelled evaluation as an alternative evaluation purchase, while guttman et al. (1998) branded it merchant brokering. further, robinson et al. (1967) divided evaluation into two sub-stages, namely, the acquisition and analysis of proposals, followed by the evaluation of proposals and selection of suppliers. according to ettlie’s study, evaluation signifies that “the new technology is being compared with the existing or future situation as to its relative advantages and disadvantages” (ettlie 1980, p. 992). according to verville and alannah (2003), three distinct areas need to be evaluated when considering a purchase of corporate-wide software systems. these include the vendor and any functional and technical issues. 2.1.4 trial trial denotes the stage where the firm has a chance to use the technology on a limited basis in order to determine its utility in a full-scale implementation (ettlie 1980). the term ‘trial’ is s222 australasian journal of information systems salim, sedera et al 2015, vol. 19, pp. s219-s254 moving from evaluation to trial specifically referred to in ettlie’s (1980) research as being the “adequacy of stage models for decisions on adoption of innovation”. a similar term is also used by shoham (1992). for this stage, this study employs ettlie’s definition that “the new technology is presently being used on a limited basis in order to determine its utility in a full-scale implementation” (ettlie 1980, p.992). with a cloud erp package, adopters (firms) are offered free trial services of the product. with this free service, the adopters are then able to use the technology and understand how the system would integrate into the firm’s business process. hence, it would be easier to make a decision r consequences of the system and thus, in turn, change the prevailing behavioural beliefs of the organisation. in fact, in most cases, firms will not adopt new technology without first trialling it on a probationary basis in order to determine its usefulness in their own situation (rogers 1995, p.172). this kind of opportunity could also assist in reducing any perceived uncertainty in the firm concerning the new technology (rogers 1995). 2.1.5 commitment according to dwyer et al. (1987, p.19), commitment is “an implicit or explicit pledge of relational continuity between exchange partners” (e.g., firm and vendor). during this stage, both parties will have achieved a certain level of satisfaction (dwyer et al. 1987) from the previous steps of the adoption process. in this stage, the experience and input gathered from the trial stage are considered in order to determine whether or not the technology will be adopted on a sustained and full-use basis (ettlie 1980). from these five stages, only evaluation and trial were selected for this research discussion. both stages are recognised as being the focus of new adopters (i.e., smes) as many activities that lead to acceptance happen in these two stages (aguirre-urreta and marakas 2012). further, as pointed out by howard and sheth (1969), evaluation and trial are recognised as being the most critical stages in the decisionmaking process, thus making these the most important periods for the firm to be able to move to the next level of adoption (i.e., acceptance stage). the selection of both stages is further supported by the fact that there is the possibility a firm will not purchase the technology after passing through the evaluation or trial stages. this is generally due to not receiving satisfactory follow-up actions from the consultant or vendor (cisco 2012). therefore, investigating the evaluation and trial stages would assist vendors to understand the factors that are important for firms to be able to proceed to the next stage and commit to the adoption of cloud erp. from the firm’s side, it promotes a perception of being able to reduce the possibility of making an incorrect decision when evaluating cloud erp systems. in this study, the respondents are from smes where, in most cases (as per our survey sample), the owner is the person who makes the final decision as to whether or not the technology will be adopted. smes were selected as the subject of investigation since the characteristics of this firm are less complex compared to large organisations. thus, the movement through the adoption stages is much quicker and the ability to produce evidence of the adoption progresses occurs over a shorter time period. cloud erp was selected as it provides the most suitable and affordable corporate-wide system for smes while maintaining almost all the functionalities of on-premise erp systems. during the process of understanding the nature of the adoption factors (henceforth referred to as “determinants”) and demonstrating how the adoption decision happens, a few of the behavioural theories regarding adoption have been reviewed. fishbein and ajzen’s (1975) theory of reasoned action and ajzen’s (1991) tpb are well-researched behavioural intention (henceforth called “intention”) models that integrate grounded concepts and principles (harrison et al. 1997). both theories have successfully been used in predicting important behavioural patterns in several research domains. as tpb is an extended version of the theory of reasoned action, additional variables are included in tpb. in contrast to other theories that have been used in technology adoption (burda and teuteberg 2013) such as the technology acceptance model [tam] proposed by davis (1989) and the unified theory of acceptance and use of technology proposed by venkatesh et al. (2003), tpb covers a range of other determinants. these are related equally to individual factors (e.g., owner’s attitude) as well as other factors that are beyond the control of the individual, including social influences (i.e., s223 australasian journal of information systems salim, sedera et al 2015, vol. 19, pp. s219-s254 moving from evaluation to trial subjective norms) and the facilitating conditions (i.e., perceived behavioural control). given the flexibility of tpb determinants to account for conditions over which individuals do not have complete control (taylor and todd 1995), this theoretical lens was selected in the present study to predict sme owners’ decisions concerning the adoption of the cloud erp systems. although smes and cloud erp are the subject of interest, the adoption decision stages discussed in this paper do not differ from the stages experienced in large organisations. however, given that this study applies tpb as the theoretical lens in which the unit of analysis is at the individual level, it would be difficult to see how the adoption decisions are operationalised for large organisations. for large organisations, the important decisionmaking is typically distributed among the management team and not by a single person as in the sme. 2.2 determinants of behavioural decisions through the lens of tpb the conceptual model proposed in this study (as shown in figure 1) is based on tpb, which has been employed in several research contexts to provide understanding and prediction of an individual’s intention to adopt new technology (ajzen 2011). further, tpb suggests that an individual’s intention to perform various kinds of behaviours can be predicted by: (1) the high precision of attitudes towards the behaviour; (2) subjective norms; and (3) perceived behavioural control (ajzen 1985, ajzen 1991, phang et al. 2006, sawang et al. 2014). the theory also suggests that behaviour can be explained by the following: behavioural belief; normative belief and control belief as the antecedents of attitudes, subjective norms and perceived behavioural control respectively (ajzen 1991, bulgurcu et al. 2010). however, the large majority of the existing literature in the technology adoption field has focused on three determinants (namely, attitude, subjective norms, and perceived behavioural control) rather than including the three antecedents of the determinants. figure 1: conceptual model of cloud erp adoption the literature provides a rich understanding of the manner in which the aforementioned determinants can lead to intention and adoption in smes. harrison et al. (1997) depicted how these three determinants work and also explored the process resulting from the decision of 162 small businesses from a broad range of industries to adopt it. using a structured survey, harrison et al. found strong support for the theory that the process of adoption is influenced by the following determinants: attitude (e.g., perceived positive and negative attitudes), subjective norms (e.g., social expectations), and perceived behavioural control (e.g., resources to overcome obstacles). pavlou and fygenson (2006) employed tpb to predict the process of e-commerce adoption at the individual level. they also extended the capabilities of tpb in an attempt to predict two prevalent online behaviours, namely, acquiring information and purchasing products from web vendors. by looking at these two exemplars from the literature, it can be seen that tpb could possibly be extended to smes and multi-stage contexts. s224 australasian journal of information systems salim, sedera et al 2015, vol. 19, pp. s219-s254 moving from evaluation to trial generally, the adoption of a new technology system involves a series of cognitive activities (attitude) within the individual’s mind. further, when making a decision, the individual may be influenced by issues related to social pressure (nuwangi et al. 2013a) such as competitors, government compliance, customers, vendors, or employees (subjective norms). in addition, individual conditions such as perceived ease or difficulty (perceived behavioural control) help to facilitate the adoption of new technology (e.g., alarifi and sedera 2014, bulgurcu et al. 2010, grandon and pearson 2004). thus, this study employs these core concepts as the underlying theory, as explained by tpb. therefore, these three determinants (attitude, subjective norms and perceived behavioural control) rooted in tpb are relevant to a wide variety of complex and subjective factors associated with cloud erp adoption. in addition to examining the adoption determinants, these determinants are also polarised into the two most critical stages in the adoption process, namely, the evaluation and trial. this enables the fluctuation of the significance of the determinants in two different stages to be observed. there are a large number of studies in technology adoption including: erp adoption (e.g., law and ngai 2007, ng and gable 2010), individual technology adoption (e.g., hernández et al. 2010, pavlou and fygenson 2006) and sme adoption (e.g., grandon and pearson 2004, li et al. 2011, meyer 2011, quaddus and hofmeyer 2007, riemenschneider et al. 2003). however, none of those studies appear to discuss corporate-wide system adoption in a multi-stage view. this led to the building of five hypotheses as discussed next. 3 hypotheses development this section explains the motivation behind sme owners’ intention toward the adoption of cloud erp. applying the relationship of tpb constructs to the context of this research, it is posited that the sme owners’ intention to adopt cloud erp is determined by the attitude of the owner him/herself (li et al. 2011). in this study, the attitude towards the intention to adopt cloud erp is conceptualised as the “individual’s overall positive or negative evaluation of the behaviour” (ajzen 2011). attitude is considered as one of the most significant predictors of behaviour (kraus 1995). in addition, the majority of prior research confirms a positive effect of attitude on intention towards various contexts. a number of adoption studies have demonstrated that positive attitudes are more likely to lead to the adoption of technology (e.g., childers et al. 2002, curran and meuter 2005, thong and yap 1995). accordingly, this study posits that a favourable attitude towards cloud erp is expected to cultivate an intention of adoption. based on the above arguments, it is hypothesised that: h1: the sme owner’s positive (negative) attitude towards the adoption of new cloud erp positively (negatively) affects his/her intention to adopt the system. subjective norms reflect an individual’s intention to adopt through the referent of others’ actions or thoughts (burda and teuteberg 2013, pavlou and fygenson 2006). thus, it is suggested that when technology is relatively new, the owner of the sme may have insufficient knowledge (nuwangi et al. 2012) or information by which to form his/her feelings towards the new technology. therefore, behavioural intention can be influenced greatly by the opinions expressed by others (alarifi and sedera 2013, thompson et al. 1994). subjective norms can be described as the degree to which an individual perceives the opinion of others that he/she should adopt the technology (venkatesh et al. 2003). in the context of smes where the owner makes the majority of the critical decisions, a strong influence could arise from external pressures such as large customers, network peers, vendors or government (quaddus and hofmeyer 2007). pressure could also come from inside the organisation itself, for example, from the needs of employees and the firm. based on the above arguments, it is hypothesised that: h2: the subjective norms that support (discourage) cloud erp adoption positively (negatively) affect the sme owner’s intention to adopt the system. further drawing on the attributes of tpb, it is expected that the intention to adopt cloud erp is determined by the level of perceived behavioural control. in this study, perceived s225 australasian journal of information systems salim, sedera et al 2015, vol. 19, pp. s219-s254 moving from evaluation to trial behavioural control is described as a person’s intention to adopt new technology based on the extent to which the person believes that he or she has control over personal or external factors that may facilitate or restrain the behavioural performance (ajzen 1991). perceived behavioural control could also be explained as indicating a person’s perception as to the respective ease or difficulty of the technology proposed to be adopted (ajzen 1991, riemenschneider et al. 2003). prior studies have verified that the ability to provide adequate resources can facilitate the adoption of new technology. for example, perceived behavioural control (defined as the level of controllability and self efficacy) was the second most salient predictor (after attitude) of ecommerce adoption among internet users (pavlou and fygenson 2006). the characteristics of smes (e.g., smaller size, less complexity) could make the implementation of cloud erp less complex, thereby encouraging the owner to continue with their intention to adopt the system. based on the above arguments, it is hypothesised that: h3: the sme owner’s perceived behavioural control over the adoption of cloud erp positively (negatively) influences the sme owner’s intentions towards cloud erp adoption. 3.1 extending and polarising tpb determinants into evaluation and trial stages past studies into technology adoption have rarely attempted to provide an understanding of how the relationships between tpb constructs fluctuate at different stages of the adoption process. it appears that only pavlou and fygenson (2006) have applied tpb to study behavioural intention in two distinct stages. their focus, however, was restricted to a single determinant (i.e., perceived behavioural control) with less discussion on the other two determinants (i.e., attitude and subjective norms). limiting the discussion prevents a holistic understanding of how tpb determinants behave differently in different stages of the adoption process. this gap led the present study to investigate tpb determinants in two distinct stages (i.e., evaluation and trial). the justifications for investigating tpb in different stages are based on the following scenarios. first, during the evaluation stage, owners form perceptions as to the benefits of the technology that they are planning to adopt (ettlie 1980). these perceptions are formed through different perspectives; however, according to tpb and the adoption context of smes, the perception is typically formed through the opinion or pressure (grandon and pearson 2004) given by the stakeholders (e.g., vendors, clients or government). using this scenario as an example, social influence (i.e., subjective norms) provides a greater impact on the intention of adoption as compared to other determinants. meanwhile, in the trial stage, the owner is given the opportunity to use the system for a limited period. by doing this, the owner will receive a hands-on experience and hence gain a better sense of the technology that is going to be adopted. having the experience and better sense will then lead to the attitude (i.e., perception and belief developed as a result of experiencing use of the system) being critical for the adoption, particularly in this stage (i.e., trial). the model tested in both the evaluation and trial stages yields hypotheses four and five as follows: h4: the subjective norms of the sme owner relating to the adoption of cloud erp are more significant than attitude and perceived behavioural control in the evaluation stage of the adoption. h5: the sme owner’s attitude towards adoption of cloud erp is more significant than subjective norms and perceived behavioural control in the trial stage of the adoption. 4 methodology 4.1 instrument development in developing measures, this study followed the guidelines stipulated by churchill (1979) and mackenzie et al. (2011). herein, the study aimed to develop a good formative index that completely exhausts the entire domain of the construct. as such, the study attempted to s226 australasian journal of information systems salim, sedera et al 2015, vol. 19, pp. s219-s254 moving from evaluation to trial develop the constructs that collectively represent all the relevant aspects of the variable of interest (bagozzi and fornell 1982, bagozzi and phillips 1982, fornell and bookstein 1982). this exercise is well-supported by the wealth of literature on technology adoption which allows a detailed account of past studies to be brought into the current study. the existing measures were first adapted to the context of this study. the standard scale development procedures stipulated in mackenzie et al. (2011) were then followed for new measures and those that required significant changes. this study operationalises normative belief, subjective norms and control belief as formative. the identification of normative belief, subjective norms and control belief as formative is ensured by adhering to the guidelines for identifying formative variables proposed by jarvis et al. (2003). as per the guidelines for identifying formative variables, the constructs of subjective norms, normative beliefs and control beliefs: (i) need not covary; (ii) are not interchangeable; (iii) cause the core-construct as opposed to being caused by it; and (iv) may have different antecedents and consequences in potentially quite different nomological nets (cenfetelli and bassellier 2009, jarvis et al. 2003, petter et al. 2007). moreover, the use of formative constructs in this case provides the specific and actionable attributes of a concept (mathieson et al. 2001).this is particularly interesting from a practical viewpoint as the weight of the construct can be used to draw practical implications on the importance of specific details. it can therefore guide practical enforcement on the characteristics (see details in furneaux and wade 2011). it is noted that the formative conceptualisation contradicts past tpb studies, where the measurements are implied as reflective. however, given the aforementioned characteristics, this study argues that tpb constructs and measures may be considered as formative. gable and sedera (2009), who explored the extent of problems in the potential mis-specification of past is research, highlighted that most formative indexes were mis-specified as being reflective in past is studies. once the instruments were ready, the individual items for reliability and validity were assessed with a sample comprising 30 respondents. as soon as sufficient confidence was gained through this exercise, the study then proceeded with the full-scale survey administration. the measurement items (refer to appendices b and c) were drawn from the literature and then adapted using standard psychometric instrument development procedures (boudreau et al. 2001). the survey instruments were adapted from three different sources. for validating questions related to technology adoption determinants, findings from a study by harrison et al. (1997) were adapted. seven-point likert scales, ranging from “strongly disagree” to “strongly agree”, were used for these types of questions. the instruments suggested by harrison et al. (1997) were selected since the group of respondents that they investigated was similar to the present study’s respondents (i.e., owners of smes), in that they were using the same theoretical lens (i.e., tpb) and focusing on the same firm size. studies by ettlie (1980) and fichman and kemerer (1997) were personalised in order to develop questions pertaining to technology adoption stages. for this particular question, respondents were asked to tick the relevant option indicating the stage at which their firm was currently positioned in relation to cloud erp adoption (each stage has an appropriate definition). the answers provided through this question enabled the total sample to be divided into different stages. the recommendations by fichman and kemerer were followed as their study operationalised and empirically validated the adoption stages. ettlie’s (1980) study is nevertheless among the earlier founding works on the adoption stages and has also been adapted by fichman and kemerer. thus, throughout this discussion, the original adoption stages from ettlie are cited. the final instrument comprises six components, namely: intention, attitude, subjective norms, normative belief, control belief, and perceived behavioural controls. 4.2 data collection and sample a sample of smes was selected in a southeast asian country (malaysia) for the data collection. since there is no generally-accepted definition for smes, this study adopts the malaysia national sme development council definition based on the following criteria: (i) sales turnover not exceeding rm50 million or full-time employees not exceeding 200 workers for the manufacturing sector, and (ii) sales turnover not exceeding rm20 million or full-time s227 australasian journal of information systems salim, sedera et al 2015, vol. 19, pp. s219-s254 moving from evaluation to trial employees not exceeding 75 workers for services and other sectors.the structures of smes are generally centralised, with the owner making most of the critical decisions. also, the same owner tends to regularly make decisions at divergent levels (salles 2006). in most cases, the same individual who makes the adoption decision will be the primary user of the technology to be adopted; hence, individualand firm-level decisions are related (li et al. 2011). to ensure that individual respondents reported accurately on behalf of their organisation, the key informant approach was applied following the guidelines of segars and grover (1998). in this study, the key informants were the owners of the respective smes where their opinions represented the voice of the entire firm. a paper-based survey was administered for the data collection. a total of 162 respondents (i.e., a response rate of more than 80 percent) drawn from sme decision-makers or owners participated in and answered the survey. the survey received a high response rate as the questionnaires were distributed and collected during an event at which the speaker was one of the study’s research members. the event was a voluntary one initiated by an email sent out to a list of smes which had been made available by the malaysian government. further, the attendance was voluntary and no incentives were provided to attend the conference or to take part in the survey. two hundred and ten (210) surveys were distributed to those key stakeholders representing each organisation. the industries represented in the sample were: electrical (36%), financial services (21%), design consultancy (16%), manufacturing (12%), construction (10%), automotive (2%) and others (4%). about 68% of the sample are from firms where the number of employees range from 20 to 40 and 32% of the sample are from firms whose number of employees extend between 40 to 130. from this results of this distribution, it can be seen that the sample are from small and medium-sized enterprises. the sample for this study does not include micro-organisations as the use of technology, especially cloud erp, is much less prevalent in micro-companies. further, it is beyond the research scope to differentiate the level of significance of each determinant between different sizes of firms. 4.3 data analysis the partial least square (pls) technique of structural equation modelling in smartpls 2.0 (ringle et al. 2005) software was used to evaluate the research model and the measurement properties of the constructs and individual items. pls also allows a researcher to simultaneously test the psychometric properties of the scales used to measure the variables in a measurement model (henseler et al. 2015), as well as the estimation of the structural model on the strength and direction of the relationships between the variables (xu et al. 2011). this study used pls as it supports a small sample size well (chin et al. 2003, hulland 1999), thereby providing parameter estimates for relatively low sample sizes. the recommended “rule of ten” with a minimum sample size of 10 times the maximum numbers of arrows pointing towards a construct was also met (hair et al. 2011) in the analysis conducted in the present research. pls is also well-suited for predictive applications due to its variance-based nature (hair et al. 2011). further, pls was chosen to accommodate the presence of formative factors. 5 results to evaluate and report the pls estimates, the recommendations by hair et al. (2011) were followed in a two-step approach suggested by chin (2010). 5.1 reflective measurement model the assessment of the measurement model included the estimation of the internal consistency, discriminant and convergent validity. the measurement instrument for reflective constructs demonstrated sufficient reliability with all the factor loadings above 0.70 which is over the proposed threshold level of 0.5 (nunnally and bernstein 1991). the average variance extracted (ave) values of all the reflective latent constructs were above the recommended threshold level of 0.5 (fornell and larcker 1981), showing sufficient convergent validity. further, the composite reliability for each reflective construct was calculated in order to examine the internal consistency of all constructs and all met the suggested tolerances of above 0.70 s228 australasian journal of information systems salim, sedera et al 2015, vol. 19, pp. s219-s254 moving from evaluation to trial (fornell and larcker 1981). discriminant validity of all latent constructs was given as the square root of each construct’s ave being greater than the latent–variable correlation between each construct and its comparing construct (hair et al. 2011) (table 2). table 2: test for discriminant validity 5.2 formative measurement model differing from the original tpb, the research model proposed in the present study has a few formative constructs by which the measurement provides specific and actionable attributes of a concept (mathieson et al. 2001). following the guidelines of diamantopoulos and siguaw (2006) as well as diamantopoulos and winklhofer (2001), the first step was to check for multicollinearity among the measures. formative measurement models are essentially based on regression (the formative construct against its measures) (gable et al. 2008). as a result, the stability of the coefficients of the measures can be influenced by the strength of the respective intercorrelations and sample size (gable et al. 2008). the variance inflation factor result from the multi-collinearity test was below 3; this falls under the proposed threshold of below 5. at the same time, the result for the measurement instrument for the formative construct showed sufficient reliability of factor weights. 5.3 testing the hypotheses first, -level (0.05) is adjusted by dividing with the number of tests (shaffer 1995) (in this case, three) where the test statistic will be significant if the associated p-value is less than 0.017. here, tests are referred to as three different datasets that have been tested, including: full dataset (162 respondents), evaluation dataset (47 respondents) and trial dataset (115 respondents). from the adjustment (i.e., bonferroni correction), the results show that significance levels of all the three main determinants (attitude, subjective norms and perceived behavioural control) have met the p>0.017 on the full dataset, the result of which can be found in appendix d. by achieving the recommended significance levels, it indicates that the cumulative result of the full dataset does not have any global type i error. this study has also tested h1 to h3 on the full dataset. the results showed a slight difference from the divided dataset but did not change the original result of the research. details of the results can be found in appendix d. further, as the study aimed to examine the manner in which smes adopt cloud erp by observing the different significance and strength of the relationships of three determinants in the evaluation and trial stages, the structural model was analysed by using two different types of data with two different sample conditions. the sample conditions were: (1) respondents who were in the evaluation stage, and (2) respondents who were in the trial stage. both data were analysed using the pls methodology (hair et al. 2011) using smartpls software (ringle et al. 2005). with the help of pls, an examination was conducted of the standardised path coefficients, path significances and variance explained (r²) to test the predictive power of the structural model, as well as the relationships between the determinants using the two types of samples. the results of all these tests are illustrated in figure 2. s229 australasian journal of information systems salim, sedera et al 2015, vol. 19, pp. s219-s254 moving from evaluation to trial while the possibility of a moderating effect of the adoption stages on the relationship between the three determinants is acknowledged, the main focus in this discussion is centred solely on the variances in intention determinants towards cloud erp adoption decision. figure 2: assessment of structural model for (a) evaluation and trial, and (b) stages of the adoption *p < 0.05; **p < 0.01; ***p < 0.001. 5.3.1 testing hypothesis 1: the first hypothesised relationship – h1: the sme owner’s positive (negative) attitude towards the adoption of new cloud erp positively (negatively) affects the sme owner’s intention to adopt the system – is tested by reference to the structural model illustrated in figures 2a and 2b. these structural models support the hypothesised relationship where the models concerning evaluation and trial (figures 2a and 2b) also affirm significant positive relationships between attitude– p<0.001), thereby lending support to the first hypothesis. 5.3.2 testing hypothesis 2: the second hypothesised relationship – h2: the subjective norms that support (discourage) cloud erp adoption positively (negatively) affect the sme owner’s intention to adopt the system – is tested by reference to the structural models presented in figures 2a and 2b. both structural models showed a significant positive relationship between subjective norms and lending support to the second hypothesised relationship. s230 australasian journal of information systems salim, sedera et al 2015, vol. 19, pp. s219-s254 moving from evaluation to trial 5.3.3 testing hypothesis 3: the third hypothesised relationship – h3: the sme owner’s perceived behavioural control over the adoption of cloud erp positively (negatively) influences the sme owner’s intention towards the adoption – is tested by reference to the path coefficients and the significance of the relationship between perceived behavioural control and intention, as shown in figures 2a and 2b. a positive relationship between perceived behavioural control and intention was predicted; however, the results show a negative relationship for the trial stage. similarly, even though the relationship of perceived behavioural control in the evaluation stage is positive, the 0.166, p<0.001) which challenges the relationship that was hypothesised. 5.3.4 testing hypotheses 4 and 5: the fourth hypothesised relationship was tested by making reference to the two structural models illustrated in figure 2. as shown in figure 2a (i.e., evaluation stage), the path coefficient between subjective norms and intention shows a much more powerful and significant relationship compared to the other two predictor determinants (i.e., attitude and perceived behavioural control) (attitude– – – <0.001)). this comprises three predictor determinants, namely, attitudes, subjective norms and perceived behavioural control, explaining 33.5% of the variance in behavioural intention (r²). next, figure 2b (i.e., trial stage) is referred to for the fifth hypothesised relationship. as can be seen therein, the attitude displays a much stronger relationship to intention than subjective norms and perceived behavioural control in the trial stage (attitude– subjective norm– 0.319, p<0.001, perceived control– 0.166, p<0.001), with the three predictor determinants (attitudes, subjective norms and perceived behavioural control) explaining 31.2% of the variance in behavioural intention (r²). accordingly, the result of hypotheses 4 and 5 suggests that the level of significance of attitude, subjective norms and perceived behavioural control for intention will vary as the condition of the adoption process changes (i.e., progressing from one stage to another), thereby lending support to the fourth and fifth hypothesised relationships. 6 summary and implications for research this study investigates how smes adopt the cloud erp system using multiple theoretical lenses, namely, the tpb (ajzen 1991) and technology adoption stages (ettlie 1980). using these theoretical lenses, the study examined the relationship between an sme owner’s attitude, subjective norms and perceived behavioural control towards the intention to adopt cloud erp in two distinct stages: the evaluation and trial. as hypothesised, attitude and subjective norms displayed a significant positive relationship towards the intention to adopt cloud erp by smes. despite predicting a positive relationship towards intention, the findings showed a negative relationship for perceived behavioural control. a possible reason for this discrepancy is the enormous pressure exerted on smes by the government and other regulatory bodies to implement cloud erp (i.e., compliance), in the current context. as such, even though the sme owner may believe that the firm does not have sufficient capability and resources (nuwangi et al. 2013b) to facilitate the adoption of cloud erp, their intention to adopt is high due to the overriding influence of subjective norms. next, in line with the second research objective, the data were analysed for variances in intention determinants towards cloud erp adoption. the findings showed that attitude, subjective norms and perceived behavioural control perform differently during the evaluation and trial stages (refer to figures 2a and 2b). the relationship of subjective norms towards intention is greater in the evaluation stage than in the trial stage. this occurs as a result of the owner receiving pressure from an external source (e.g., vendors, clients or government). on the other hand, the results demonstrated that an owner’s attitude can supersede subjective norms and perceived behavioural control in the trial stage. the use of a cloud erp system s231 australasian journal of information systems salim, sedera et al 2015, vol. 19, pp. s219-s254 moving from evaluation to trial through a limited time period gives the owner an opportunity to become familiar with the system. accordingly, this leads to a superiority of attitude in this stage. 6.1 implications this study makes theoretical contributions to the body of technology adoption research specifically in the domain of technology adoption by smes. there is no doubt that a considerable number of past studies have assumed that technology adoption is a single stage process. however, the decision to adopt new technology (especially in the case of corporatewide systems) comprises several activities such as: searching for information, comparing, evaluating, trialling and, finally, committing. treating corporate-wide system adoption as a snapshot can lead to erroneous adoption decisions (e.g., forcing firms to deal with unsuitable applications for a period of time). these issues call for the inclusion of multi-stages in technology adoption models. viewing corporate-wide system adoption as a multi-stage process not only leads to a more complete understanding of owners’ behaviour patterns, but could also improve the predictive power of complex technology adoption models. accordingly, this study presents an intention model by which to explain and predict owners’ behaviour patterns regarding cloud erp adoption. as the trend towards cloud erp adoption has been predicted to result in revenues of about $33 billion by 2016 (anderson et al. 2013), understanding the important determinants that influence these key stakeholders (e.g., decision-makers) could assist vendors as well as consultants in prioritising their strategies regarding the provision of more detailed information. further, this paper extends the scope of tpb in four different ways. first, through this work, tpb is extended into the field of corporate-wide system adoption. although pavlou and fygenson (2006) extended tpb by examining two different behaviours (i.e., stages), their work is still limited to individuals (consumers). in this study, the notions of tpb are used to gain inferences of the corporate-wide system adoption. though the participation is at individual level, it allowed the researchers to make observations about corporate technology adoption. second, the study succeeded in changing the normal perception of tpb. earlier tpb studies have treated the adoption determinants as positive or significant. however, in the research context of the present study, the result showed that perceived behavioural control has a negative relationship towards intention. in other words, the level of significance for perceived behavioural control is very low as compared to the attitude and subjective norm determinants. third, as this study modelled two distinct stages of technology adoption in parallel, the dependency of the evaluation and trial stages on the final stage (i.e., commitment) were observed. providing two related stages (activities) to be concurrently modelled could open a new avenue of future research. fourth, the present study shows theoretically and empirically that subjective norms, normative belief and control belief are formative constructs respectively. these findings are unusual in comparison to prior tpb studies which have mostly treated all tpb constructs as reflective. although the majority of researchers assume that the measurement model is reflective, there are in fact many instances in which this assumption may not be theoretically or empirically justified. from the practitioners’ point of view, this study contributes to the industry by providing guidance to the erp ecosystem (vendors, consultants and communities) in an attempt to understand their potential buyer’s behaviour and perception toward the adoption of cloud erp. it shows the role of external agencies (such as government or business partners) in triggering the continuity of cloud erp adoption especially in the early stages of the process. however, the owner’s attitude supersedes other determinants once the firm is using cloud erp on a trial basis. 7 limitations, future work and conclusions this research was not without some limitations. first, this research only examined the evaluation and trial stages in order to understand the corporate-wide systems adoption process. however, it is not possible to fully explain the entire corporate-wide systems adoption process without considering other stages of the adoption process. therefore, it is proposed that s232 australasian journal of information systems salim, sedera et al 2015, vol. 19, pp. s219-s254 moving from evaluation to trial future researchers consider other stages in the adoption process, such as awareness, interest and commitment in order to more fully explain the process of corporate-wide systems adoption. second, this study only used intention as the dependent variable, thus limiting the ability to predict how intention transforms into actual behaviour. third, in reference to tpb, all the constructs in the proposed model reflect the assessment of cloud erp adoption. consequently, this prevents the generalisation of the findings to other types of complex technology adoption. therefore, additional research that could capture a general construct pertaining to other types of corporate-wide systems could be undertaken in the future. fourth, the use of tpb (individual adoption theory) for organisation technology adoption only applied for organisations where the important decision is made by an individual and his/her decision represents the voice of the entire firm. however, for organisations with multiple decisionmakers, the result from the finding cannot be generalised. in conclusion, this study demonstrates that the predictors or determinants of cloud erp adoption have different effects at different stages of the adoption process. by using tpb, in addition to ettlie’s (1980) stages, as the theoretical lens, the study has shown that these determinants (i.e., attitude, subjective norms and perceived behavioural control) possess and provide different levels of significance at different stages. among all these determinants, subjective norms provide the most significant impact in the evaluation stage, while the owner’s attitude towards technology provides the most significant impact in the trial stage. additionally, this study complements existing technology adoption research (e.g., harrison et al. 1997) by integrating two stages and testing them simultaneously. the context of the malaysian study which provided data for this research may be perceived as a limitation. as mentioned earlier in the discussion, each country yields to a specific definition of what an sme might be. further, governmental and industry / customer pressure are two variables that might be of interest for a future study. references adams, d.a., nelson, r.r. & todd, p.a. (1992) "perceived usefulness, ease of use, and usage of information technology: a replication", mis quartely, 16(2): 227-247. aguirre-urreta, m.i. & marakas, g.m. (2012) "exploring choice as an antecedent to behavior: incorporating alternatives into the technology acceptance process", journal of organizational and end user computing (joeuc), 24(1): 82-107. ajzen, i. (1985) from intentions to actions : a theory of planned behavior, in action-control: from cognition to behaviour (khul j and beckman j, edition, j. kuhl, j. beckmann, eds. action control: from cognition to behaviour springer verlag, new york. ajzen, i. (1991) "the theory of planned behavior", organizational behavior and human decision processes, 50(2): 179-211. ajzen, i. (2011) "the theory of planned behaviour: reactions and reflections", psychology & health, 26(9): 1113-1127. alarifi, a. & sedera, d. (2013) "enhancing enterprise social network use: a control theory study", proceedings of the 24th australasian conference on information systems, melbourne, australia. alarifi, a. & sedera, d. (2014) "peripheral, central and coercive routes for promoting enterprise social networks", proceedings of the 25th australasian conference on information systems, auckland, new zealand. amoako-gyampah, k. & salam, a.f. (2004) "an extension of the technology acceptance model in an erp implementation environment", information & management, 41(6): 731-745. anderson, e., lam, l.-l., eschinger, c., cournoyer, s., correia, j.m., wurster, l.f., contu, r., biscotti, f., liu, v.k., pang, t.e.c., swinehart, h.h., yeates, m., petri, g. & bell, w. (2013) forecast overview: public cloud services, worldwide, 2011-2016, 4q12 update, gartner group, inc. , connecticut, united states, s233 australasian journal of information systems salim, sedera et al 2015, vol. 19, pp. s219-s254 moving from evaluation to trial http://my.gartner.com/portal/server.pt?open=512&objid=202&&pageid=5553&mod e=2&in_hi_userid=2&cached=true&resid=2332215&ref=analystprofile. arthur, w.b. (1989) "competing technologies, increasing returns, and lock-in by historical events", the economic journal, 99(394): 116-131. au, y.a. & kauffman, r.j. (2003) "what do you know? rational expectations in information technology adoption and investment", journal of management information systems, 20(2): 49-76. bagozzi, r.p. & fornell, c. (1982) theoretical concepts, measurements and meaning a second generation of multivariate analysis, c. fornell, editor, new york. bagozzi, r.p. & phillips, l.w. (1982) "representing and testing organizational theories: a holistic construal", administrative science quarterly, 27(3): 459-489. boudreau, m.-c., gefen, d. & straub, d.w. (2001) "validation in information systems research: a state-of-the-art assessment", mis quarterly, 25(1): 1-16. sized enterprises", issues of business and law, 4(1): 119-130. bulgurcu, b., cavusoglu, h. & benbasat, i. (2010) "information security policy compliance: an empirical study of rationality-based beliefs and information security awareness", mis quarterly, 34(3): 523-548. burda, d. & teuteberg, f. (2013) proceeding of 34th international conference on information systems milan, italy. butler, p. & peppard, j. (1998) "consumer purchasing on the internet:: processes and prospects", european management journal, 16(5): 600-610. campbell, d.e., wells, j.d. & valacich, j.s. (2013) "breaking the ice in b2c relationships: understanding pre-adoption e-commerce attraction", information systems research, 24(2): 219-238. cenfetelli, r.t. & bassellier, g. (2009) "interpretation of formative measurement in information systems research", mis quarterly, 33(4): 7. childers, t.l., carr, c.l., peck, j. & carson, s. (2002) "hedonic and utilitarian motivations for online retail shopping behavior", journal of retailing, 77(4): 511-535. chin, w.w. (2010) "how to write p and report pls analyses", in: e. in v. esposito vinzi et al. (ed.) handbook of partial least squares, springer, berlin heidelberg, pp. 655-690. chin, w.w., marcolin, b.l. & newsted, p.r. (2003) "a partial least squares latent variable modeling approach for measuring interaction effects: results from a monte carlo simulation study and an electronic-mail emotion/adoption study", information systems research, 14(2): 189-217. choudhury, v. & karahanna, e. (2008) "the relative advantage of electronic channels: a multidimensional view", mis quarterly, 32(1): 179. churchill , g.a. (1979) "a paradigm for developing better measures of marketing constructs", journal of marketing research, 16(1): 64-73. cisco (2012) collaborating in the cloud : why it is important, and what to look for when evaluating a cloud-based collaboration solution, cisco public information. available from http://www.cisco.com/en/us/prod/collateral/ps10352/ps10362/ps12584/white_pape r_c11-720970.html#wp9000042 [accessed sept, 1 2013]. curran, j.m. & meuter, m.l. (2005) "self-service technology adoption: comparing three technologies", journal of services marketing, 19(2): 103-113. s234 australasian journal of information systems salim, sedera et al 2015, vol. 19, pp. s219-s254 moving from evaluation to trial damanpour, f. & schneider, m. (2006) "phases of the adoption of innovation in organizations: effects of environment, organization and top managers", british journal of management, 17(3): 215-236. davis, f.d. (1989) "perceived usefulness, perceived ease of use, and user acceptance of information technology", mis quarterly, 13(3): 319-340. de bruyn, a. & lilien, g.l. (2008) "a multi-stage model of word-of-mouth influence through viral marketing", international journal of research in marketing, 25(3): 151-163. del aguila-obra, a.r. & padilla-melendez, a. (2006) "organizational factors affecting internet technology adoption", internet research: electronic networking applications and policy, 16(1): 94-110. dholakia, r.r. & kshetri, n. (2004) "factors impacting the adoption of the internet among smes", small business economics, 23(4): 311-322. diamantopoulos, a. & siguaw, j.a. (2006) "formative versus reflective indicators in organizational measure development: a comparison and edmpirical illustration", british journal of management, 17(4): 263-282. diamantopoulos, a. & winklhofer, h.m. (2001) "index construction with formative indicators: an alternative to scale development", journal of marketing research, 38(2): 269-277. dishaw, m.t. & strong, d.m. (1999) "extending the technology acceptance model with task– technology fit constructs", information & management, 36(1): 9-21. dubey, a. & wagle, d. (2007) "delivering software as a service", the mckinsey quarterly, 6(2007): 2007. dwyer, f.r., schurr, p.h. & oh, s. (1987) "developing buyer-seller relationships", the journal of marketing, 51(2): 11-27. eden, r., sedera, d. & tan, f. (2014) "sustaining the momentum: archival analysis of enterprise resource planning systems (2006–2012)", communications of the association for information systems, 35(3): 39-82. engel, j.f., blackwell, r.d. & miniard, p.w. (1978) consumer behavior, dryden press, hinsdale, il: dryden. ettlie, j.e. (1980) "adequacy of stage models for decisions on adoption of innovation", psychological reports, 46(3): 991-995. fichman, r.g. & kemerer, c.f. (1997) "the assimilation of software process innovations: an organizational learning perspective", management science, 43(10): 1345-1363. fichman, r.g. & kemerer, c.f. (2012) "adoption of software engineering process innovations: the case of object-orientation", sloan management review, 34(2). fishbein, m. & ajzen, i. (1975) belief, attitude, intention and behavior: an introduction to theory and research, addison-wesley, reading, ma. fornell, c. & bookstein, f.l. (1982) "two structural equation models: lisrel and pls applied to consumer exit-voice theory", journal of marketing research, 19(4): 440-452. fornell, c. & larcker, d.f. (1981) "evaluating structural equation models with unobservable variables and measurement error", journal of marketing research, 18(1): 39-50. forrest, w. & barthold, c. (2009) clearing the air on cloud computing, mckinsey & company. available from http://www.cloudmagazine.fr/dotclear/public/clearing_the_air_on_cloud_computing .pdf. s235 australasian journal of information systems salim, sedera et al 2015, vol. 19, pp. s219-s254 moving from evaluation to trial fox, a., griffith, r., joseph, a., katz, r., konwinski, a., lee, g., patterson, d., rabkin, a. & stoica, i. (2009) above the clouds: a berkeley view of cloud computing, dept. electrical eng. and comput. sciences, university of california, berkeley, rep. ucb/eecs. frambach, r.t. & schillewaert, n. (2002) "organizational innovation adoption: a multi-level framework of determinants and opportunities for future research", journal of business research, 55(2): 163-176. furneaux, b. & wade, m. (2011) "an exploration of organizational level information systems discontinuance intentions", mis quarterly, 35(3): 573-598. gable, g.g. & sedera, d. (2009) proceeding of 30th international conference on information systems milwaukee. gable, g.g., sedera, d. & chan, t. (2008) "re-conceptualizing information system success: the is-impact measurement model", journal of the association for information systems, 9(7): 377-408. gopalakrishna, s. & lilien, g.l. (1995) "a three-stage model of industrial trade show performance", marketing science, 14(1): 22-42. grandon, e.e. & pearson, j.m. (2004) "electronic commerce adoption: an empirical study of small and medium us businesses", information & management, 42(1): 197-216. guo, y. & barnes, s. (2011) "purchase behavior in virtual worlds: an empirical investigation in second life", information & management, 48(7): 303-312. guttman, r.h., moukas, a.g. & maes, p. (1998) "agent-mediated electronic commerce: a survey", knowledge engineering review, 13(2): 147-159. hair, j.f., ringle, c.m. & sarstedt, m. (2011) "pls-sem: indeed a silver bullet", the journal of marketing theory and practice, 19(2): 139-152. harrison, d.a., mykytyn jr, p.p. & riemenschneider, c.k. (1997) "executive decisions about adoption of information technology in small business: theory and empirical tests", information systems research, 8(2): 171-195. henseler, j., ringle, c.m. & sarstedt, m. (2015) "testing measurement invariance of composites using partial least squares", international marketing review, forthcoming hernández, b., jiménez, j. & martín, m.j. (2010) "customer behavior in electronic commerce: the moderating effect of e-purchasing experience", journal of business research, 63(9): 964-971. howard, j.a. & sheth, j.n. (1969) the theory of buyer behavior, wiley new york. hulland, j. (1999) "use of partial least squares (pls) in strategic management research: a review of four recent studies", strategic management journal, 20(2): 195-204. jarvis, c.b., mackenzie, s.b. & podsakoff, p.m. (2003) "a critical review of construct indicators and measurement model misspecification in marketing and consumer research", journal of consumer research, 30(2): 199-218. karahanna, e., straub, d.w. & chervany, n.l. (1999) "information technology adoption across time: a cross-sectional comparison of pre-adoption and post-adoption beliefs", mis quarterly, 23(2): 183-213. klaus, h., rosemann, m. & gable, g.g. (2000) "what is erp?", information systems frontiers, 2(2): 141-162. koslowski, d.-k.t. & strüker, j. (2011) "erp on demand platform", business & information systems engineering, 3(6): 359-367. s236 australasian journal of information systems salim, sedera et al 2015, vol. 19, pp. s219-s254 moving from evaluation to trial kraus, s.j. (1995) "attitudes and the prediction of behavior: a meta-analysis of the empirical literature", personality and social psychology bulletin, 21(1): 58-75. law, c.c. & ngai, e.w. (2007) "erp systems adoption: an exploratory study of the organizational factors and impacts of erp success", information & management, 44(4): 418-432. li, x., troutt, m.d., brandyberry, a. & wang, t. (2011) "decision factors for the adoption and continued use of online direct sales channels among smes", journal of the association for information systems, 12(1): 1-31. liang, h., saraf, n., hu, q. & xue, y. (2007) "assimilation of enterprise systems: the effect of institutional pressures and the mediating role of top management", mis quarterly, 31(1): 59-87. mackenzie, s.b., podsakoff, p.m. & podsakoff, n.p. (2011) "construct measurement and validation procedures in mis and behavioral research: integrating new and existing techniques", mis quarterly, 35(2): 293-334. markus, m.l., axline, s., petrie, d. & tanis, s.c. (2003) "learning from adopters' experiences with erp: problems encountered and success achieved", in: in second-wave enterprise resource planning systems (shanks g, seddon pb and willcocks lp, eds) cambridge university press, cambridge. mathieson, k., peacock, e. & chin, w.w. (2001) "extending the technology acceptance model: the influence of perceived user resources", acm sigmis database, 32(3): 86-112. mell, p. & grance, t. (2011) the nist definition of cloud computing, national institute of standards and technology, gaithersburg, md. available from http://csrc.nist.gov/publications/nistpubs/800-145/sp800-145.pdf. meyer, j. (2011) "workforce age and technology adoption in small and medium-sized service firms", small business economics, 37(3): 305-324. mintzberg, h., raisinghani, d. & theoret, a. (1976) "the structure of 'unstructured'decision processes", administrative science quarterly, 21(2): 246-275. muscatello, j.r., small, m.h. & chen, i. j. (2003) "implementing enterprise resource planning (erp) systems in small and midsize manufacturing firms", international journal of operations & production management, 23(8): 850-871. ng, c.s.-p. & gable, g.g. (2010) "maintaining erp packaged software–a revelatory case study", journal of information technology, 25(1): 65-90. nunnally, j.c. & bernstein, i.h. (1991) psychometric theory, mcgraw-hill, new york. nuwangi, s., sedera, d. & srivastava, s.c. (2013a) "information systems development outsourcing: the role of control configurations. ", proceeding of 34th international conference on information systems milan, italy. nuwangi, s.m., sedera, d. & murphy, g. (2012) "multi-level knowledge transfer in software development outsourcing projects: the agency theory view", proceeding of 33rd international conference on information systems orlando, fl. nuwangi, s.m., sedera, d., srivastava, s.c. & murphy, g. (2013b) "intra-organizational information asymmetry in offshore isd outsourcing", vine: the journal of information and knowledge management systems, 44(1): 94-120. pavlou, p.a. & fygenson, m. (2006) "understanding and predicting electronic commerce adoption: an extension of the theory of planned behavior", mis quarterly, 30(1): 115143. petter, s., straub, d. & rai, a. (2007) "specifying formative constructs in information systems research", mis quartely, 31(4): 623-656. s237 australasian journal of information systems salim, sedera et al 2015, vol. 19, pp. s219-s254 moving from evaluation to trial phang, c.w., sutanto, j., kankanhalli, a., li, y., tan, b.c. & teo, h.-h. (2006) "senior citizens' acceptance of information systems: a study in the context of e-government services", ieee transactions on engineering management, 53(4): 555-569. pierce, j.l. & delbecq, a. l. (1977) "organization structure, individual attitudes and innovation", academy of management review, 2(1): 27-37. quaddus, m. & hofmeyer, g. (2007) "an investigation into the factors influencing the adoption of b2b trading exchanges in small businesses", european journal of information systems, 16(3): 202-215. rayner, n. (2014) survey analysis: adoption of cloud erp, 2013 through 2023 gartner group, inc, connecticut, united states. riemenschneider, c.k., harrison, d.a. & mykytyn, p.p. (2003) "understanding it adoption decisions in small business: integrating current theories", information & management, 40(4): 269-285. ringle, c.m., wende, s. & will, s. (2005) smartpls 2.0 (m3) beta, hamburg, http://www.smartpls.de . robey, d., ross, j.w. & boudreau, m.-c. (2002) "learning to implement enterprise systems: an exploratory study of the dialectics of change", journal of management information systems, 19(1): 17-46. robinson, p.j., faris, c.w. & wind, y. (1967) industrial buying and creative marketing, allyn & bacon boston. rogers, e.m. (1995) diffusion of innovations, free press. salim, s.a. (2013) proceedings of the 17th pacific asia conference on information systems, jeju island, south korea. salim, s.a., sedera, d., sawang, s. & alarifi, a. (2014) proceedings of the 25th australasian conference on information systems, auckland, new zealand. salles, m. (2006) "decision making in smes and information requirements for competitive intelligence", production planning & control, 17(3): 229-237. sawang, s., sun, y. & salim, s.a. (2014) "it's not only what i think but what they think! the moderating effect of social norms", computers & education, 76: 182-189. schwarz, a., chin, w.w., hirschheim, r. & schwarz, c. (2014) "toward a process-based view of information technology acceptance", journal of information technology, 29(1): 7396. sedera, d. & gable, g.g. (2010) "knowledge management competence for enterprise system success", the journal of strategic information systems, 19(4): 296-306. sedera, d., lokuge, s., krcmar, h., srivastava, s.c. & ravishankar, m.n. (2014) "the future of outsourcing in the asia-pacific region: implications for research and practice—panel report from pacis 2014.", communications of the association for information systems, 35(1): 317-331. segars, a.h. & grover, v. (1998) "strategic information systems planning success: an investigation of the construct and its measurement", mis quarterly, 22(2): 139-163. shaffer, j.p. (1995) "multiple hypothesis testing", annual review of psychology, 46(1): 561584. sharma, r. & sood, m. (2011) "enhancing cloud saas development with model driven architecture", international journal on cloud computing: services and architecture (ijccsa), 1(3): 89-102. s238 australasian journal of information systems salim, sedera et al 2015, vol. 19, pp. s219-s254 moving from evaluation to trial shoham, a. (1992) "selecting and evaluating trade shows", industrial marketing management, 21(4): 335-341. soh, c., kien, s.s. & tay-yap, j. (2000) "enterprise resource planning: cultural fits and misfits: is erp a universal solution?", communications of the acm, 43(4): 47-51. sykes, t.a., venkatesh, v. & johnson, j.l. (2014) "enterprise system implementation and employee job performance: understanding the role of advice networks", mis quarterly, 38(1): 51-72. taylor, s. & todd, p.a. (1995) "understanding information technology usage: a test of competing models", information systems research, 6(2): 144-176. teo, h.h., wei, k.k. & benbasat, i. (2003) "predicting intention to adopt interorganizational linkages: an institutional perspective", mis quarterly, 27(1): 19-49. thompson, r.l., higgins, c.a. & howell, j.m. (1994) "influence of experience on personal computer utilization: testing a conceptual model", journal of management information systems, 11(1): 167-187. thong, j.y.l. & yap, c.s. (1995) "ceo characteristics, organizational characteristics and information technology adoption in small businesses", omega, 23(4): 429-442. venkatesh, v. & davis, f.d. (2000) "a theoretical extension of the technology acceptance model: four longitudinal field studies", management science, 46(2): 186-204. venkatesh, v., morris, m.g., davis, g.b. & davis, f.d. (2003) "user acceptance of information technology: toward a unified view", mis quarterly, 27(3): 425-478. verville, j. & halingten, a. (2003) "a six-stage model of the buying process for erp software", industrial marketing management, 32(7): 585-594. walther, s., eden, r., phadke, g. & torsten, e. (2015) "the role of past experience with onpremise on the confirmation of the actual system quality of on-demand enterprise systems. in enterprise systems. strategic, organizational, and technological dimensions ", in: springer international publishing, switzerland, pp. 238-246. doi: 10.1007/978-3-319-17587-4_15 willcocks, l.p. & sykes, r. (2000) "enterprise resource planning: the role of the cio and it function in erp", communications of the acm, 43(4): 32-38. winters, d.b., lindley, j.t., topping, s. & lindley, l.t. (2008) "the hidden financial costs of erp software", managerial finance, 34(2): 78-90. wixom, b.h. & todd, p.a. (2005) "a theoretical integration of user satisfaction and technology acceptance", information systems research, 16(1): 85-102. xu, j.d., benbasat, i. & cenfetelli, r. (2011) "the effects of service and consumer product knowledge on online customer loyalty", journal of the association for information systems, 12(11): 1. zaltman, g., duncan, r. & holbek, j. (1973) innovations and organizations, wiley, new york. zmud, r.w. (1982) "diffusion of modern software practices: influence of centralization and formalization", management science, 28(12): 1421-1431. s239 australasian journal of information systems salim, sedera et al 2015, vol. 19, pp. s219-s254 moving from evaluation to trial appendices appendix a – ten most cited studies on corporate-wide systems adoption references (a) study focus (b) findings (c) liang et al. (2007) this study illustrates how top management can mediate the impact of external institutional pressures on the usage of enterprise systems assimilation within large organisations. the study suggests that mimetic pressure and coercive pressures positively affect top management beliefs, which then positively affect top management participation in the erp assimilation process. on the other hand, normative pressures directly affect erp usage without requiring the participation of a mediator from among top management. wixom and todd (2005) this study develops an integrated research model that differentiates beliefs and attitudes about the system. it achieves this by examining beliefs and attitudes concerning the use of the systems through a sample of 465 users in seven large organisations. findings from the study supported the hypothesized model in that user satisfaction and technology acceptance can and should be integrated. amoako-gyampah and salam (2004) this study presents an extension of the technology acceptance model (tam) by empirical testing using a field survey from a large global organisation that was in the process of implementing an erp system. further, the study aims to examine how shared beliefs impact upon the core tam variables in the context of erp. the study found that managerial interventions, such as training and communication, influence the acceptance of technology while perceived usefulness and ease of use contribute to the behavioural intention to use the technology. teo et al. (2003) this study investigates institutional pressures that facilitate the adoption of financial electronic data interchange (fedi) in an institutionalized environment. it provides feedback from 222 respondents ranging from ceos to cfos and cios. the result shows the way in which mimetic, coercive and normative pressures had a significant influence upon the intention to adopt fedi. venkatesh et al. (2003) this study designed a unified technology acceptance model using longitudinal field studies data at four large organisations that were being introduced to this new technology in the workplace. from the eight intention and usage models discussed in the study, it can be seen that performance expectancy, effort expectancy, social influence, and facilitating conditions provide very significant or direct effect determinants. meanwhile, attitudes towards using technology, combined with selfefficacy and anxiety give less or no direct effect towards intention and usage. robey et al. (2002) this study presents a comparative case study involving 13 large firms that implemented an enterprise resource planning (erp) system by comparing their dialectic learning process. the study found seven distinct motivations for pursuing erp including: y2k compliance, legacy system replacement, process re-engineering initiatives, integration of multiple sites, support growth, improved reporting and decision-making, as well as regulatory compliance. s240 australasian journal of information systems salim, sedera et al 2015, vol. 19, pp. s219-s254 moving from evaluation to trial soh et al. (2000) this study explains the unanticipated “misfits” of erp adoption in the context of asian countries which typically reflect a bias towards western practices. few misfit types and solutions have been discussed as a way to anticipate the uniqueness of erp adoption in an asian context. overall, there is a need for vendors to explain the embedded data requirements and processes of erp to organisations. venkatesh and davis (2000) this study discusses an extension of the technology acceptance model (tam) by measuring three different points in time in each organisation using longitudinal data collected from four different large organisations. social influence processes (subjective norms, voluntariness and image) and cognitive instrumental processes (job relevance, output quality, result demonstrability and perceived ease of use) significantly influence user acceptance. dishaw and strong (1999) this study focuses on the theoretical foundation of the technology acceptance model (tam) and the task-technology fit model (ttf) that can be integrated. the integration model was tested using data collected from programmers at three different organisations all using large management information systems. the integrated path model shows an acceptable fit to the data with the amount of variance in the dependent variable explained by this model as being higher than the variance accounted for in either tam or ttf alone. adams et al. (1992) using research by davis (1989) as the basis of their discussion, this study focuses on evaluating psychometric properties of ease of use and usefulness. this is performed by examining the relationship of both constructs using survey data collected from 118 respondents from 10 different organisations using voice and electronic mail. the study has proved that the extended setting of davis (1989) is reliable and valid. further, it indicates that usefulness is related to usage, but ease of use is relatively less important in determining overall use. table 1: ten most-cited studies on corporate-wide systems adoption – references, focus of study and study findings. s241 au str ala sia n j ou rn al of inf or ma tio n s ys tem s sa lim , s ed er a e t a l 20 15 , v ol. 19 , p p. s2 19 -s 25 4 m ov ing fr om ev alu at ion to tr ial a p p e n d ix b m e a su re m e n t in st ru m e n t fo r e va lu a ti o n s ta g e re fle ct iv e m ea su re s o ut er lo ad in gs tva lu e co m po si te re lia bi lit y av e fo rm at iv e m ea su re s o ut er w ei gh ts tva lu e id ite m id ite m in te nt io n (a da pt ed fr om h ar ris on e t a l. 19 97 ) 0. 84 22 0. 64 1 n or m at iv e be lie f ( ad ap te d fr om h ar ris on e t a l. 19 97 ) in 1 0. 86 26 31 .0 76 2 n b1 0. 86 88 11 .5 56 2 in 2 0. 74 48 9. 65 3 n b2 0. 31 08 2. 18 82 0. 79 01 13 .2 86 2 su bj ec tiv e n or m (a da pt ed fr om h ar ris on e t a l. 19 97 ) pe rc ei ve d be ha vi or al c on tr ol (a da pt ed fr om h ar ris on e t a .l 19 97 ) 0. 87 86 0. 85 65 sn 1 0. 88 58 10 .0 79 3 pb c1 o ve ra ll , i nt ro du ci ng c lo ud e rp w ou ld b e ea sy fo r o ur fi rm 0. 91 44 9. 26 14 sn 2 0. 17 7 1. 27 92 pb c2 o ve ra ll , i nt ro du ci ng c lo ud e rp w ou ld b e un de r t he fi rm 's c on tr ol 0. 93 64 29 .9 66 co nt ro l b el ie f ( ad ap te d fr om h ar ris on e t a l. 19 97 ) at tit ud e (a da pt ed fr om h ar ris on e t a l. 19 97 ) 0. 92 27 0. 54 52 cb 4 0. 52 78 2. 32 28 at 2 o ve ra ll , i nt ro du ci ng c lo ud e rp w ou ld b e po si ti ve fo r o ur fi rm 0. 80 32 14 .1 14 9 cb 5 0. 54 44 2. 30 44 at 3 0. 84 69 18 .6 03 5 at 4 o ve ra ll , i nt ro du ci ng c lo ud e rp w ou ld b e w is e fo r o ur fi rm 0. 54 27 4. 88 37 at 5 0. 72 38 9. 13 15 o ur c om pa ny w il l c er ta in ly m ak e an e ff or t t o im pl em en t c lo ud er p in th e ne xt 6 -1 2 m on th s o ur c om pa ny w il l d ef in it el y im pl em en t c lo ud e rp in th e ne xt 6 12 m on th s o ur c om pa ny is p la nn in g to im pl em en t c lo ud e rp in th e ne xt 6 12 m on th s to w ha t e xt en t d o yo u be li ev e th at c us to m er s/ cl ie nt s th in k yo u sh ou ld in tr od uc e cl ou d er p to w ha t e xt en t d o yo u be li ev e th at s up pl ie rs /v en do rs th in k yo u sh ou ld in tr od uc e cl ou d er p in 3 o ve ra ll , i nt ro du ci ng c lo ud e rp w ou ld b e hi gh ly e ff ec ti ve fo r o ur fi rm o ve ra ll , i nt ro du ci ng c lo ud e rp w ou ld re du ce c os ts fo r o ur fi rm h ow li ke ly is it th at y ou r f ir m h as th e ab il it y to o bt ai n ad di ti on al e m pl oy ee s if n ee de d to e na bl e us e of cl ou d er p in th e ne xt 6 -1 2 m on th s to w ha t e xt en t d o yo u va lu e th e op in io n of cu st om er s/ cl ie nt s in re la ti on to in tr od uc in g cl ou d er p to w ha t e xt en t d o yo u va lu e th e op in io n of su pp li er s/ ve nd or s in re la ti on to in tr od uc in g cl ou d er p h ow li ke ly is it th at y ou r f ir m h as th e re so ur ce s to tr ai n yo ur c ur re nt e m pl oy ee s in th e us e of c lo ud e rp in th e ne xt 6 -1 2 m on th s s2 42 au str ala sia n j ou rn al of inf or ma tio n s ys tem s sa lim , s ed er a e t a l 20 15 , v ol. 19 , p p. s2 19 -s 25 4 m ov ing fr om ev alu at ion to tr ial a p p e n d ix c m e a su re m e n t in st ru m e n t fo r tr ia l s ta g e re fle ct iv e m ea su re s o ut er lo ad in gs tva lu e co m po si te r el ia bi lit y a v e fo rm at iv e m ea su re s o ut er w ei gh ts tva lu e id it em id it em in te nt io n (a da pt ed fr om h ar ri so n et a l. 19 97 ) 0. 87 41 0. 69 87 n or m at iv e b el ie f (a da pt ed f ro m h ar ri so n et a l. 19 97 ) in 1 0. 87 43 23 .9 40 7 n b 2 0. 54 96 2. 36 52 in 2 0. 84 71 15 .3 93 6 n b 3 0. 47 15 1. 78 74 in 3 0. 78 37 14 .5 45 n b 4 0. 45 82 1. 41 96 pe rc ei ve d be ha vi or al c on tr ol (a da pt ed fr om h ar ri so n et a l. 19 97 ) 0. 89 26 0. 73 59 su bj ec ti ve n or m ( a da pt ed f ro m h ar ri so n et a l. 19 97 ) pb c1 o ve ra ll , i n tr o d u ci n g cl o u d e r p w o u ld b e e a sy f o r o u r fi rm 0. 90 95 10 .9 14 6 sn 1 0. 16 5 0. 65 79 pb c2 o ve ra ll , i n tr o d u ci n g cl o u d e r p w o u ld b e u n d e r th e f ir m 's c o n tr o l 0. 89 28 9. 85 51 sn 2 0. 41 06 1. 61 46 pb c3 o ve ra ll , i n tr o d u ci n g cl o u d e r p w o u ld b e s im p le t o a rr a n ge 0. 76 39 4. 15 09 sn 3 0. 32 01 1. 51 26 a tt it ud e (a da pt ed fr om h ar ri so n et a l. 19 97 ) 0. 92 53 0. 71 27 sn 4 0. 24 79 0. 76 51 at 1 o ve ra ll , i n tr o d u ci n g cl o u d e r p w o u ld b e h e lp fu l fo r o u r fi rm 0. 81 33 22 .5 51 2 sn 5 0. 30 21 0. 99 2 at 2 o ve ra ll , i n tr o d u ci n g cl o u d e r p w o u ld b e p o si ti ve f o r o u r fi rm 0. 87 5 40 .9 75 4 at 3 0. 83 43 21 .0 04 1 co nt ro l b el ie f (a da pt ed f ro m h ar ri so n et a l. 19 97 ) at 4 o ve ra ll , i n tr o d u ci n g cl o u d e r p w o u ld b e w is e f o r o u r fi rm 0. 86 02 29 .5 45 1 cb 1 0. 36 31 0. 74 17 at 5 0. 83 69 21 .9 78 3 cb 2 0. 32 81 0. 76 09 cb 5 0. 70 37 1. 60 54 o u r co m p a n y w il l d e fi n it e ly i m p le m e n t cl o u d e r p in t h e n e xt 6 -1 2 m o n th s o u r co m p a n y w il l ce rt a in ly m a ke a n e ff o rt t o i m p le m e n t cl o u d e r p in t h e n e xt 6 -1 2 m o n th s o u r co m p a n y is p la n n in g to i m p le m e n t cl o u d e r p in t h e n e xt 6 -1 2 m o n th s o ve ra ll , i n tr o d u ci n g cl o u d e r p w o u ld re d u ce c o st s fo r o u r fi rm o ve ra ll , i n tr o d u ci n g cl o u d e r p w o u ld b e h ig h ly e ff e ct iv e f o r o u r fi rm to w h a t e xt e n t d o y o u b e li e ve t h a t su p p li e rs /v e n d o rs th in k yo u s h o u ld i n tr o d u ce c lo u d e r p to w h a t e xt e n t d o y o u b e li e ve t h a t go ve rn m e n t th in ks yo u s h o u ld i n tr o d u ce c lo u d e r p b y m e a n s o f in ce n ti ve s/ ta x e xe m p ti o n s? to w h a t e xt e n t d o y o u b e li e ve t h a t te ch n ic a l st a ff t h in k yo u s h o u ld i n tr o d u ce c lo u d e r p to w h a t e xt e n t d o y o u v a lu e t h e o p in io n o f cu st o m e rs /c li e n ts i n r e la ti o n t o i n tr o d u ci n g cl o u d e r p to w h a t e xt e n t d o y o u v a lu e t h e o p in io n o f su p p li e rs /v e n d o rs i n r e la ti o n t o i n tr o d u ci n g cl o u d e r p h o w l ik e ly i s it t h a t yo u r fi rm h a s th e a b il it y to o b ta in a d d it io n a l e m p lo ye e s if n e e d e d t h a t co u ld e n a b le y o u r fi rm t o u se c lo u d e r p in t h e n e xt 6 -1 2 m o n th s to w h a t e xt e n t d o y o u v a lu e t h e o p in io n o f go ve rn m e n t th ro u gh i n ce n ti ve s/ ta x e xe m p ti o n i n r e la ti o n t o in tr o d u ci n g cl o u d e r p to w h a t e xt e n t d o y o u v a lu e t h e o p in io n o f te ch n ic a l st a ff i n r e la ti o n t o i n tr o d u ci n g cl o u d e r p to w h a t e xt e n t d o y o u v a lu e t h e o p in io n o f e m p lo ye e s in r e la ti o n t o i n tr o d u ci n g cl o u d e r p h o w l ik e ly i s it t h a t yo u r fi rm h a s fi n a n ci a l a ss e ts t h a t co u ld e n a b le y o u r fi rm t o u se c lo u d e r p in t h e n e xt 6 -1 2 h o w l ik e ly i s it t h a t yo u r fi rm h a s ti m e t o u se c lo u d e r p in t h e n e xt 6 -1 2 m o n th s s2 43 australasian journal of information systems salim, sedera et al 2015, vol. 19, pp. s219-s254 moving from evaluation to trial appendix d – full dataset result with 162 number of respondents copyright: © 2015 salim, sedera, sawang, alarifi, atapattu. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. s245 paper title (paper title style) australasian journal of information systems burmeister, islam, dayhew & crichton 2015, vol. 19, selected papers from acis enhancing client welfare through better communication enhancing client welfare through better communication of private mental health data between rural service providers oliver burmeister charles sturt university oburmeister@csu.edu.au md zahidul islam charles sturt university zislam@csu.edu.au miriam dayhew charles sturt university mdayhew@csu.edu.au merrilyn crichton charles sturt university mcrichton@csu.edu.au abstract client welfare is detrimentally affected by poor communication of data between rural service providers, which in part is complicated by privacy legislation. a study of service provision involving interviews with mental health professionals, found challenges in communicative processes between agencies were exacerbated by the heavy workloads. dependence on individual interpretations of legislation, and on manual handling, led to delays that detrimentally affected client welfare. the main recommendation arising from this article is the creation of an ehealth system that is able to negotiate differing levels of access to client data through centralised controls, where the administration of that system ensures that it stays current with changing legislative requirements. the main contribution of the proposed model is to combine two well-known concepts: data integration and generalisation. people with mental illness are amongst the most vulnerable members of society, and current ehealth systems that provide access to medical records inadequately cater to their needs. keywords: confidentiality; privacy; ehealth; rurality; trust models 1 introduction in australia there are local, state and federal laws that govern what public and private mental health providers are able to do. the study reported here found that many providers, fearing possible litigation, overly err on the side of caution about sharing client data with other services. that, in turn, frequently disadvantages their clients because in mental health, a client is rarely served by a single professional. more frequently they will be served by a case worker, a general practitioner (gp), a clinical psychologist, a psychiatrist and/or other professionals, all with different information needs. while privacy legislation is part of a mental health professional’s training, changes to legislation as well as heavy workloads in rural areas, mean that it is difficult for service providers to keep up. as a result, information sharing between agencies becomes complicated. there is therefore no question of the importance of designing usable processes and technologies for interagency communication, an argument that has been documented previously (asan & montague, 2013; friberger & falkman, 2011). what is unique to this article is that it uses qualitative interviews with mental health service users and professionals to inform a model of communication that integrates privacy legislative changes into designs for communication between rural service providers. such communication has to navigate complex legislative restrictions, yet the burden of interpreting such restrictions falls to individual professionals in each service. helpful to 1 australasian journal of information systems burmeister, islam, dayhew & crichton 2015, vol. 19, selected papers from acis enhancing client welfare through better communication interagency communication would be a means of systematising the complex negotiations involved, in a way that removes the onus from individuals and places it upon a centrally controlled, automated ehealth system. the model proposed here involves a central server control information that is classified, through encryption, in order to only make available relevant information to those who need it, while also complying with the legislation as it is revised. the model also ensures fast access to accurate data related to a mental health patient. this article discusses privacy, confidentiality and trust within the current australian legislative framework and how that relates to mental health service users and providers. it then discusses the methodology that influenced the development of the computer system model. in the final section, the proposed model is described. 2 the principle of ‘keeping confidence’ privacy and confidentiality are closely related concepts and significantly affect the development of ehealth systems. generally the difference can be described as that privacy is about people, whereas confidentiality is about data. confidentiality, arguably the more significant of the two for ehealth systems, has been viewed as informational privacy, that is, it is about the storing of private data. allen (2011) claimed that privacy is about controlling the access of health information that is required by governments, institutions and individuals, whereas “many medical professionals, hospitals, insurers and other entities with access to health information regard maintaining the confidentiality of medical communications and the security of medical records as paramount professional responsibilities”. there are justifiably instances when breaches ought to occur and therefore not all breaches are unethical. one instance occurs regularly, when a client in a hospital voluntarily waives their right to confidentiality, for third party access, such as care givers other than the medical personnel treating them. other ethical issues can be involved, such as when a person does not have the mental capacity to make such a decision for themselves. in such a situation the surrogate decision maker needs to be fully informed about the situation, so that they can make an informed decision as to whether or not to waive the client’s right to confidentiality. yet another instance occurs when an institution or care giver is compelled through a court order to release confidential information. confidentiality thus concerns issues of restricting the flow of information. professional codes of ethics contain statements that explicitly deal with confidentiality in computing (bowern, burmeister, gotterbarn, & weckert, 2006; burmeister, 2013; burmeister & weckert, 2003) and in professions providing patient care (bernoth, dietsch, burmeister, & schwartz, 2014). crichton (2008) analysed professional codes of conduct and found that ethical service phrases, such as “shall avoid misleading clients”, “due care”, and “accurate information” were indicators of the professional values of honesty, integrity and expertise. keeping confidence is an important ethical principle because it promotes an environment of trust between a mentally ill client and the professional who is caring for them. as noted above, there are exceptions in which it is recognised that the professional has an obligation to breach confidentiality, or in other words, an obligation to betray the trust of their client. such situations are typically for the attainment of a greater good or for the prevention of harm, such as in an instance of suicidal behaviours or some other situation which contravenes their duty to protect the client, or to protect colleagues. in a mental health context, confidentiality is about managing the protection of private client data that has been disclosed in a situation of trust. 3 privacy legislation the privacy of health information is a difficult matter in law as well as in professional practice. the issue has even been the subject of civil court cases with the most notable matter being the ownership of patient records determined in breen v williams (breen v williams, 1996) where the high court of australia held that a patient had no general right to copy, nor to inspect medical records relating to them and held by a health care provider. magnusson and opie 2 australasian journal of information systems burmeister, islam, dayhew & crichton 2015, vol. 19, selected papers from acis enhancing client welfare through better communication (1998) identified that the high court finding upheld the nsw court of appeal finding and was characterised by the unanimous agreement and significant consistency of the reasoning of the judicial members of the court, consequently making this a powerful authority. it is well recognised that within the existing legislation there is the potential for inconsistency and for difficulties in interpretation and application. the final report from the nsw law reform commission (2010) makes 104 recommendations to simplify the law, improve the consistency and ensure adequate coverage and protection for individuals. recognising this level of complexity at the level of the law reform commission would suggest that the average practitioner might have difficulty remaining current in their knowledge and understanding of their statutory obligations under state and commonwealth legislation. for mental health care professionals in nsw, a state jurisdiction within the commonwealth of australia, the significant pieces of legislation are the privacy act 1988 (cth), the privacy and personal information protection act 1998 (nsw) known as the ‘ppipa’, and the health records and information privacy act 2002 (nsw) known as the ‘hripa’. there is also the ‘gipa’ which is the government information (public access) act 2009 (nsw) to make up the legislative framework concerned with information privacy protection. changes to the commonwealth privacy act were brought into play in mid-2014 and there are now far greater requirements for transparency in the handling of personal information. the individual whose information is held by a private practitioner, community agency or public sector agency can expect a greater level of understanding regarding the disclosure of their personal information. all entities must have a clear privacy management policy that details how they handle personal information and this must be available for their clientele. the changes are designed to enhance protection of personal information and are linked to strengthened powers for the australian information commissioner to investigate and resolve complaints and enforce compliance. a significant issue for mental health providers would be the receipt of unsolicited private information, which must now be destroyed or deidentified. another major issue for mental health practitioners supporting their mobile clients is the impact of sharing information across borders and it becomes the responsibility of the information holder to confirm for themselves that the recipient of information will manage that information in accordance with the australian privacy laws. each health practitioner will also be guided by the codes of conduct specific to their professional accreditation bodies. the ppipa covers personal information, that information from which a person’s identity may be confirmed, but the ppipa doesn’t cover ‘health information’ (ppipa section 4a), this belongs under the auspices of the hripa. the hripa applies to: “… every organisation that is a health service provider or that collects, holds or uses health information”. the term "organisation" means a public sector agency or a private sector person. (health records and information privacy act 2002 section 11). furthermore, the privacy act 1988 (cth) identifies that: a permitted general situation exists in relation to the collection, use or disclosure by an app entity of personal information about and individual if specific conditions are met (section 16a). the most likely of these specific conditions for a mental health professional, would be where there is a threat to the life, health or safety of an individual or the public. in the privacy act quoted above, an ’app entity’ is identified as an agency or organisation. there are also statutory guidelines released by the nsw privacy commissioner (http://www.ipc.nsw.gov.au/privacy/ipc_index.html#15) that support the use and disclosure of health information. these address the need to release personal details and health information for the purposes of management of a health service, research, and training purposes, where consent may not have been obtained from an individual, and the use of deidentified data is insufficient to fulfil the information requirements. there is a further guideline for the use of disclosure of health information obtained from a third party. all health information is considered sensitive and private and so mental health information is not specifically addressed as a discrete type of information under the hripa, but there are further impacts on the sharing of mental health information addressed by the mental health act 2007 (nsw) and the mental health (forensic provisions) act 1990 (nsw). the disclosure 3 australasian journal of information systems burmeister, islam, dayhew & crichton 2015, vol. 19, selected papers from acis enhancing client welfare through better communication of information is protected under both these acts. no information is to be disclosed without the consent of the individual except where the information is necessary for the administration or execution of the acts or other lawful purposes such as legal proceedings. there are special provisions under section 76j of the mental health (forensic provisions) act 1990 that allow the sharing of information between corrective services institutions and health service institutions, through the development of information sharing protocols that support the transfer of information, without the express consent of the individual where that information is necessary for the care and treatment of the individual as prescribed under the act. a person who accesses the services of their gp and is referred to a counsellor can expect that there will be information shared between these two practitioners and there is an implied consent to sharing the information as part of an ongoing care program. if, however, the person then seeks voluntary admission to the local public hospital mental health facility then the hospital is going to have to seek consent from the patient to request access to material held by the gp or counsellor. on discharge the gp may receive a copy of a discharge summary, provided that the patient nominated that gp as part of their health record. however, if they have not been nominated, then no information regarding the admission will be able to be made available. if the person has travelled interstate then the state-based privacy legislations will need to be considered, and although there is a relative sameness and the principles are relatively constant, there will be discrete differences. the primary key to transferring information between practitioners is written consent from the client. 4 methodology in order to understand the nature of mental health services in rural australia, an interpretive study utilising qualitative techniques to interpret meaning and to analyse participant understandings was undertaken. the methodology was informed by schwandt (2003) who argued that such a process helps to explore how people in a particular social context interpret mental health services and reveal meanings that constitute those perceptions. interviews were conducted with mental health service users and practitioners in the western murray darling basin (mdb) region of nsw between november 2011 to july 2012. the key question for this study was “what mental ehealth services will best meet the future needs of the western murray darling basin?” 4.1 sampling the interviewees were chosen on the basis of criterion sampling based on age, location, and length of experience, in order to give an understanding of professional experience across the region. the sample size of practitioners was limited to 27 participants spread geographically across the area of the western mdb. sampling of mental health service users reached saturation after only thirteen interviews when it became evident a limited range of servicing patterns were present, with no new data emerging very early in the interview stages of the project. ten of the thirteen participants recorded their mental health servicing experiences over a period of one month (30 days), which was used to support and provide depth to consumer perceptions and analysis. the study was conducted in an area of australia where service providers cover large geographical territories. additionally, australian government funding has meant that there are limited numbers of mental health practitioners and only one psychiatrist living in the area. figure 1 shows service provision that is based in one location, and reaches into other areas. for instance, the region is poorly serviced by psychiatrists and therefore there are two psychiatrists that service the region, who fly in from sydney. another example is that of a gp who is based in wagga wagga, but who also services the needs of prisoners at the junee detention facility. still another example is that of griffith based mental health case workers who regularly service clients in the hillston local government area (lga). although attempts were made to accurately depict all mental health services in the region, it is possible that some services have been omitted. 4 australasian journal of information systems burmeister, islam, dayhew & crichton 2015, vol. 19, selected papers from acis enhancing client welfare through better communication figure 1: mental health services available within the 13 riverina local government areas (map produced by spatial data analysis network, charles sturt university, 2012). 5 results regional mental health service providers deal with high case loads as well as large distances. this leads to difficulties in attracting and retaining staff, and to inadequate professional development opportunities, including training in changes to privacy legislation, knowledge of which directly impacted upon what professionals perceived they could or could not transfer about a client to another agency. the research revealed that clients at times experienced escalating problems because of the unavailability of their medical records, when transferring between rural services. in part this was due to the manual handling of records that could have been communicated electronically. these results are restricted to situations exemplifying issues of privacy in the western mdb. to protect participants the identifying details of people in these vignettes have been altered. the following examples illustrate the challenges of the distances involved. first there are exemplary quotations from rural service providers, then there is a vignette from a recipient of mental health care. i have a client in hay, hillston, goolgowi … yanco … barellan. i have had clients in west wyalong … narrandera … ardlethan. … hay is like a grey area, the case worker in deniliquin normally services hay, but if he's a bit overwhelmed then i pick it up. (case worker) … way down on the murray river, 40ks from swan hill. so nearly in victoria. … we have such a huge volume of emails that come through our system and we expect clinicians to be able to follow policies and those policies are provided by email and they just cannot cope with the volume of work, when they're driving 4 hours they take, that takes a day out, sets them behind, you know 60 emails might come in that day. (service manager) jemma is a woman in her early 40s. she has four children, two are grown up and two are in primary school. her early life was coloured by sexual and physical abuse. jemma became a mother for the first time at 17. jemma’s diagnosis has been changed from borderline personality disorder to that of bipolar disorder. she sees a psychiatrist in canberra, but lives in wagga wagga. with two small children, an active community life, and a limited capacity to work and therefore tight financial circumstances (she and her partner work as artists) travelling for treatment is a difficult process: i drive over every 3 months to see my psychiatrist in canberra … initially it was every 2 weeks … i drive for 2 and a half hours to get there, i have a 10 minute appointment with him, all he does is look at my bloods, order more blood tests “how are you going, everything okay?” “good” “yep see you later” that’s it. 5 australasian journal of information systems burmeister, islam, dayhew & crichton 2015, vol. 19, selected papers from acis enhancing client welfare through better communication 5.1 coordination between rural service providers lack of coordination is due to multiple reasons. for example, jennie is a young, vibrant university student. her mother (a mental health nurse) noticed that she was struggling to make friends and manage near the end of primary school. jennie was put in touch with mental health services and has spent many years with the headspace team (headspace is an organisation providing treatment for people aged 12 to 25, who are experiencing mental illness) as well as community youth mental health. in the process of turning 18, jennie was moved to the adult mental health system. she talked about the lack of communication between the organisations and having to repeat things many times to different people. a mental health professional, addressed the difficulties on interagency coordination: we've got a plethora of different software providers for different software but because they're only protecting their own little patch of territory they won't develop something that would interface into something else. … even with this building here we've got a headspace program and our national contract mandates we must use one form of software, but we've got a contract with another government department which mandates we must use this form of software. so you’ve constantly got staff juggling their way through two software systems and losing time there because the two software systems don’t talk to each other. a mental health case manager with a government agency: … if there was ways where we could access doctors a lot easier, or have the doctors understand the centrelink process, and the information that we needed, we could probably cut down, a lot of red tape … to have a look at the clients patient record … see like the specialist letters or, just a specific part of it, not have full blown access maybe, but just a specific part where you can go in, and go okay, i need a bit more information about this, without having then, to contact the doctor. because i’m busy, they’re busy, often you play phone tag, there’s a lot of doctors who don’t want to talk to you … if you know that a client has seen a particular specialist, like a neurologist for example, if you can get access to their letters, because all specialists write a letter back to the gp ...i had a situation a couple of months ago, where i was trying to get, a psychiatrist letter … with this particular case i found out that the psychiatrist had written the report. it was sitting on the customers file for 3 weeks, while the case manager went on holidays, and nobody bothered to deal with it. 6 a privacy preserving data sharing model the proposed model arises from the findings. it is apparent that a mental health client usually interacts with a number of service providers (parties). each of these parties collects data, which can be useful for other rural service providers as well. furthermore, a client often needs to provide a party with necessary information collected and stored by another party. however, due to various privacy issues, generally a hospital or a medical centre may not want to supply a client’s sensitive information to a social worker. therefore, for clients having serious mental health issues such as dementia, it can be very difficult to organize the transfer of his/her data from one party to another party. electronically storing all information (about a client) in a password protected central server may not be an adequate solution given that mental health clients, such as those with dementia, may forget their password. even if a mental health client can recollect the password, he/she may not be able to take the appropriate decision (due to their illness) on what information they need to release to a health service provider. different health service providers need to have access to different levels of personal health related information. that is, the level of health information required by a gp can be different to the level of information required by a case worker. for example, while a gp may need the exact disease of a patient (i.e. whether it is dementia or alzheimer’s disease) a case worker may only need a higher level of information such as mental illness. often there are also legal restrictions on the level of information that can be accessed by a party. a solution is a privacy 6 australasian journal of information systems burmeister, islam, dayhew & crichton 2015, vol. 19, selected papers from acis enhancing client welfare through better communication preserving data sharing model (see figure 2), based on data integration and generalization/aggregation. data integration is a well-known technique (batini, lenzerini, & navathe, 1986; hass et al., 1999; xu, zhang, & dong, 2006) that integrates data from heterogeneous sources into one single system. data integration is commonly applied when two or more organisations merge into one due to various reasons such as a government decision and one organisation acquiring another. generalisation is another well-known approach that is used for a completely different purpose: to preserve individual privacy (aggarwal & yu, 2008; verykios, 2004). for example, the age of a patient, say 34, is often generalised into an age category, say 30-40. a main goal of the generalisation is to prevent record re-identification and breaches of individual privacy. while data integration and generalisation are two well-known concepts they are typically used in completely different contexts and purposes. the main contribution of our proposed data sharing model is combining them into a single model to allow communication between rural service providers, for solving the issues (as described before) of a mental health client. the proposed model allows all parties that are involved with a mental health patient to store their data in a trusted central server (instead of only storing them in their individual computers separately) and access the data in the appropriate permission level. unlike the traditional generalisation techniques (aggarwal & yu, 2008; verykios, 2004) our model applies different levels of generalisation for different parties to suit their access permission levels. a problem in the interagency communication of the competing organisations is the absence of a trusted third party (a central server) that can make sure that all privacy sensitive data and logic rules (patterns) are appropriately handled. however, in the context of this study we propose a government run model where an advantage is the presence of a trusted third party, such as the ministry of health, that can protect the interests of all parties and follow the law of the land. a key issue of the proposed model is the dynamic data integration in the central server and automatic adaptation of the changes of privacy legislation in the data generalisation process. a truly dynamic data integration process allowing us to integrate the data from various sources that are unknown at the design time can be very challenging (wang, yu, & zhang, 2009). if the data schemas of the various sources are unknown at the design time then it can be difficult to integrate them completely automatically. for example, in one schema the diagnosis of a disease can be labelled as “disease” while in another schema the same attribute can be “diagnosis”. the unit of the attribute “weight” can be kg in one schema whereas the unit of the same attribute can be pound in another schema. an attribute value can be called differently (for example, coke and coca cola) in different schemas. sometimes different schemas can use different attributes where one attribute is a function of the other attribute. the integration of the schemas therefore may require human intervention and domain knowledge. additionally, the automation of the generalisation process to adapt the changes in privacy legislation can be a difficult task since the type of changes are unpredictable. therefore, in order to maintain a high accuracy and quality our proposed model uses a semi-automated data integration approach where the integration of the data can be done automatically for any additional data as long as the schemas of the sources are not changed. however, any existing dynamic data integration technique can be used if that is found suitable. although designing a suitable database schema for data integration and generalisation will require the involvement of a database administrator, it is required to be done only at the server level. moreover, it needs to be re-designed only if a new party joins, an existing party changes its database schema or privacy legislation changes. in such cases typically a minor adjustment at the server end is sufficient. 6.1 a model to improve client welfare outcomes and legislative compliance in the proposed model we have a central trusted server which stores all original information collected from the parties involved. the parties can be different health service providers such as hospitals, gps and age care centres. our model first analyses the properties (such as access permission levels) of the parties and then groups them into categories, where in each category the parties have exactly the same properties. it is not unlikely to have some categories having 7 australasian journal of information systems burmeister, islam, dayhew & crichton 2015, vol. 19, selected papers from acis enhancing client welfare through better communication only a single party where its property does not match with any other parties. each party regularly uploads its data to the central trusted server. the data are first encrypted using an encryption filter as shown in figure 2 before sending them to the server through the internet. in figure 2 there are three parties (party a, party b and party c) with exactly the same properties and grouped as category 1. party a is uploading its new data in the trusted central server. it first encrypts its data and then sends it to the server through the internet. the encrypted message contains the identification information of the sender party along with the actual data. any suitable asymmetric encryption technique can be used (rafaeli & hutchison, 2003; simmons, 1979). note that each party only communicates with the trusted server directly and therefore, only needs to know the key of the server. the server of course needs to know the keys of all parties. collected data are then decrypted, integrated and pre-processed through the decryption, data integration and data pre-processing filter, respectively as shown in figure 2. data integration is required at the server since different parties can have different database schema. for example, as seen in the findings section above, public and private mental health providers frequently use differing software systems to record client data, and therefore, as shown in figure 3, different parties can name the same table and the same attribute differently. moreover, different parties can use different units for the same attribute. some parties can have tables that are not stored by other parties. figure 3 shows an example where parties belonging to category 1 have two tables, namely patient and doctor, whereas parties belonging to category 2 have two tables called client and social worker. the same patient table is called differently as patient and client by the categories. moreover, the same attribute diagnosis is also called differently as diagnosis and disease by the categories. in this example (figure 3), both categories have a table that is not stored by the other category. the server takes all these facts into account and integrates the data into one consistent database schema. for example, the server creates a database from the tables of the two categories, where it finally contains three tables. it combines the attributes of the patient and client table and calls the table as patient. it also calls the attributes diagnosis and disease as diagnosis in its patient table. in the client table the identifier attribute is called c_id (client id), whereas the same attribute is called p_id (patient id) in the patient table of the server. records from different tables belonging to different categories can be matched in the combined table of the server through a global id, like the driver license number or social security number. figure 2: overall block diagram of the privacy preserving data sharing model. 8 australasian journal of information systems burmeister, islam, dayhew & crichton 2015, vol. 19, selected papers from acis enhancing client welfare through better communication figure 3: data integration in the trusted central server. note that for each category, the server stores the necessary metadata describing the mappings (for the tables and attributes such as disease to diagnosis), and access permission levels for each attribute of the tables. different types of data and database schema are often transformed and mapped into another type, for various reasons including building a data warehouse using heterogeneous data sources and generating a dynamic webpage using an underlying database (batini et al., 1986; hass et al., 1999; xu et al., 2006). in our privacy preserving data sharing model we convert the data (uploaded in the server by a party) in order to suit the database schema of the server. for example, when party d (figure 3) uploads a record in the client table it either inserts a new record or updates an existing record in the patient table of the server. the c_id attribute of the client table and the p_id attribute of the patient table can be matched through an external id. the metadata stored in the server for category 2, helps the server to map the client table to the patient table, and the disease attribute to the diagnosis attribute. through the data pre-processing filter, our model then performs data pre-processing for data cleansing and missing value imputation (m. g. rahman & islam, 2014). data cleansing techniques automatically identify any incorrect record or incorrect value in a table. the identified incorrect value is then typically amended by first considering it as missing and then making an educated guess using an missing value imputation technique. generally the accuracy of the incorrect data identification and missing value imputation is very high. however, instead of automatically changing an uploaded data, our model reports an incorrect value with a suggested possible correction. the party then requires re-uploading of the data after necessary correction or ignoring the warning/suggestion. the pre-processing step aims to ensure a high quality of the data, which are stored in the central server. figure 4: an example of generalization of data. 9 australasian journal of information systems burmeister, islam, dayhew & crichton 2015, vol. 19, selected papers from acis enhancing client welfare through better communication figure 5: different responses for a similar query once the data are stored in the trusted central server they can be used by any party (figure 2 and figure 4) in many different ways, including simple query response for providing information as requested by a party, and privacy preserving data mining through various data mining tasks such as clustering and classification for knowledge discovery. for query response, the server first uses the data smoothening filter (see figure 2) that changes a query made by a party into an appropriate query for the execution at the server. for example, a party may send a query as shown in figure 5 where it asks for the disease name of the client having id = 1 from the client table. however, the client table is called the patient table, and the attribute disease is called the attribute diagnosis in the server. the data smoothening filter changes the table name from client to patient and the attribute name from disease to diagnosis knowing that the query is made by a party from, say category 2, as reported by the query response filter in figure 2. while responding to a query the server applies necessary generalization functions, through the data generalisation filter, based on the access permission level of the party generating the query, as shown in figure 2, figure 4 and figure 5. for example, based on the access permission level of a party the exact diagnosis (“alzheimer’s disease”) of a patient can be generalized as “mental disease”. the level of generalization is related to the level of access permission. the more access permission results in the less aggregation/generalization. each level of access permission will correspond to a level of generalization, where the lowest (minimum) level of access permission will have the maximum generalization. for example, a diagnosis “alzheimer’s disease” can be generalized as “dementia”, “mental disease” or “disease” depending on the access permission level of a party asking for the information. the party having the maximum access permission to the attribute can see the query result as “alzheimer’s disease”, whereas the party having a lower access permission can get a query response as “mental disease” (figure 4 and figure 5). moreover, through the formatting filter a query response is reformatted according to the data format of the party generating the query. for example, the weight of a patient can be returned as 50 kg or 110 pounds depending on the data format of the party. the whole process of data integrations, query smoothening, data generalization, query response based on access level and response reformatting remains transparent to a party. the parties can also apply different data mining algorithms including decision tree (quinlan, 1996), decision forest (abellan & masegosa, 2009) and clustering (ji, pang, zhou, han, & wang, 2012; kashef & kamel, 2009; m. a. rahman & islam, 2011) on the dataset stored in the trusted server. while a party may not have full access to all data stored in the server it has access to complete results produced by the data mining algorithms. we consider the output of data mining algorithms as general patterns, which are not sensitive to individual privacy, and are public knowledge. for example, a decision tree algorithm (islam, 2012) returns a tree containing a set of logic rules that explain the rules leading to a class value, say disease type. 10 australasian journal of information systems burmeister, islam, dayhew & crichton 2015, vol. 19, selected papers from acis enhancing client welfare through better communication an example of a logic rule can be “diabetics = yes & blood pressure = yes & education = phd & head injury = yes  alzheimer’s (yes: 550, no: 20)” suggesting that people having diabetics, high blood pressure, previous history of head injury and a doctorate degree (i.e. people fulfilling the pre-condition) have high chance of alzheimer’s disease. moreover, there are altogether (550+20) 570 patients, in the dataset, satisfying the pre-condition and 550 patients out of them have alzheimer’s disease. while disclosure of the diagnosis of a particular patient is considered as a breach of individual privacy, the discovery of the logic rule as explained above is not considered as a breach of privacy. the information revealed by the logic rule is generally considered as public knowledge involving a group of people (islam & brankovic, 2011). for example, the fact that smokers have higher risk of cancer is a public knowledge, and is not considered as a breach of individual privacy. in order to facilitate data mining, a number of datasets can be manually or automatically prepared from the tables stored on the server. a party can only view the metadata (such as the names of the attributes and their domain values) of the datasets, and cannot view the actual records. however, a party can apply data mining algorithms on the datasets using the tools such as weka provided by the server. the algorithms are run on the datasets in the server, and the datasets are not required to be downloaded to the party’s local machine. a party is also allowed to choose necessary parameters such as the attributes to use, any attribute as the class attribute (such as diagnosis) for classification tasks, and the algorithm-specific parameters such as the minimum number of records per leaf of a tree. several necessary restrictions such as the minimum number of records per cluster and the minimum number of records per leaf of a tree, are enforced in order to preserve individual privacy. therefore, a user cannot learn a logic rule or cluster having a single patient (or a minimum number of patients), resulting in a protection of sensitive individual information from unauthorized parties. an advantage of such a centralized data mining model is that it allows a party to extract general knowledge from the datasets without allowing them to learn sensitive information about an individual. therefore, the proposed model capitalises on the advantages of privacy preserving data mining (islam & brankovic, 2011). there are also many other advantages of the proposed privacy preserving data sharing model. it allows the parties to collaborate and share data among themselves so that a party can access all the necessary information in order to provide an accurate and high quality service, without being required to ask a patient to arrange the transfer of his/her data from one service provider to another. without the model it could be difficult for a party (service provider) to collect all data for a patient. for example, a party may not know which other party to talk to or how many parties to talk to in order to collect all data for a patient. moreover, as seen in the example of the mental health nurse above, the party may not know what types of data are available about a patient. similarly, the party may not have an idea on the number of available attributes and their names in the centralized dataset. however, when the data from all parties are stored in the central server, a party can get all information of a patient by using a simple query like select * from patient where patient.id=1. such a query produces a response informing all attribute values stored in the patient table for the patient, following the access permission level of the party. if a party has high access permission it gets low generalized data. moreover, a party can explore the database schema for the central server to learn more about the available data. another advantage of the model is the simplicity of the encryption/decryption model. each party only needs to talk to the central trusted server instead of all other parties. therefore, each party only needs to know the key of the server resulting in a simple encryption/decryption process. it is more secure to communicate with a single trusted server (in a client server model) than to communicate with the peers in a peer to peer model (rafaeli & hutchison, 2003; simmons, 1979). still another advantage of a centralised model is that the onus of complying with legislative changes to privacy does not rely on individual, over-worked mental health professionals. instead compliance can be centrally administered and controlled. finally, from a managerial perspective, another advantage of data mining on a combined data set is that it may reveal interesting patterns that a party otherwise might not be able to extract. various 11 australasian journal of information systems burmeister, islam, dayhew & crichton 2015, vol. 19, selected papers from acis enhancing client welfare through better communication data mining tasks can be performed resulting in useful pattern discovery involving all records and attributes – which otherwise would not be possible for a party to perform. 7 conclusion the study reported here involved qualitative, in-depth interviews with mental health professionals and recipients of mental health care, in the western mdb. one of the findings of the study was a need for better management of client records, when they are communicated between providers of services. furthermore, it was found that the challenges experienced in the region were in part unique to the rurality of the services, exemplified in the excessive workloads of staff, and the long distances they needed to travel to service clients. a complicating factor is that privacy legislation is constantly being updated and it is different for the public and private mental health service providers and different again between states. therefore it is unreasonable to expect rural health workers to also keep current with privacy legislation, which is not core business for them. instead what is needed is a system that removes that responsibility from the health worker in a way that ensures compliance with legislative changes. the privacy preserving data sharing model solves these challenges and can be transferred to other medical contexts, outside mental health and outside the rural australian setting. it is possible to encrypt confidential data, smooth it, aggregate it or in other ways restrict access to it, such that centrally controlled data can be widely accessed by professionals servicing a particular client. the model facilitates accuracy and timely data access, whilst preserving confidential data and restricting access as appropriate to private information about the client. this speeds access to client data, improves the servicing of the client, and reduces lengthy visits in which client histories need to be repeated. the model allows all parties to access data on a patient (irrespective of who collected the data) according to the access permission level (following the privacy legislation) of a party. the model facilitates better professional practice in regards to compliance with privacy legislation, whilst simultaneously enhancing the welfare of clients through more efficient communication of their data between service providers. 8 acknowledgements this project was funded by a grant from the strategic projects branch, department of regional australia, regional development and local government, canberra. the team included the now retired dr muenstermann. references abellan, j., & masegosa, a. (2009). an ensemble method using credal decision trees. european journal of operational research, 205(1), 218-226. aggarwal, c., & yu, p. s. (2008). a general survey of privacy preserving data mining models and algorithms. the kluwer international series on advances in database systems, 34, 11-52. allen, a. (2011). privacy and medicine. in e. n. zalta (ed.), the stanford encyclopedia of philosophy. asan, o., & montague, e. (2013). technology-mediated information sharing between patients and clinicians in primary care encounters. behaviour & information technology, 1-12. doi: 10.1080/0144929x.2013.780636 batini, c., lenzerini, m., & navathe, s. b. (1986). a comparative analysis of methodologies for database schema integration. acm computing surveys, 18(4), 323-364. bernoth, m., dietsch, e., burmeister, o. k., & schwartz, m. (2014). information management in aged care: cases of confidentiality and elder abuse. journal of business ethics, 122, 453-460. doi: 10.1007/s10551-013-1770-7 12 australasian journal of information systems burmeister, islam, dayhew & crichton 2015, vol. 19, selected papers from acis enhancing client welfare through better communication bowern, m., burmeister, o. k., gotterbarn, d., & weckert, j. (2006). ict integrity: bringing the acs code of ethics up to date. australasian journal of information systems, 13(2), 168-181. medical records access case, hca 57 c.f.r. (1996). burmeister, o. k. (2013). achieving the goal of a global computing code of ethics through an international-localisation hybrid. ethical space: the international journal of communication ethics, 10(4), 25-32. burmeister, o. k., & weckert, j. (2003). applying the new software engineering code of ethics to usability engineering: a study of 4 cases. journal of information, communication & ethics in society, 3(3), 119-132. crichton, m. (2008). the praxis of voluntary service: an investigation of the logic of service in rotary and zonta. (phd phd), queensland university of technology, brisbane. friberger, m. g., & falkman, g. (2011). collaboration processes, outcomes, challenges and enablers of distributed clinical communities of practice. behaviour & information technology, 32(6), 519-531. doi: 10.1080/0144929x.2011.602426 hass, l. m., miller, r. j., niswonger, b., tork roth, m., schwarz, p. m., & wimmers, e. l. (1999). transforming heterogeneous data with database middleware: beyond integration. bulletin of the ieee computer society technical committee on data engineering, 22(1), 31-36. islam, m. z. (2012). explore: a novel decision tree classification algorithm. paper presented at the 27th british national conference on databases (bncod 2010), dundee, scotland. islam, m. z., & brankovic, l. (2011). privacy preserving data mining: a noise addition framework using a novel clustering technique. knowledge-based systems, 24(8), 12141223. ji, j., pang, w., zhou, c., han, x., & wang, z. (2012). a fuzzy k-prototype clustering algorithm for mixed numeric and categorical data. journal of knowledge-based systems, 30, 129135. kashef, r., & kamel, m. s. (2009). enhanced bisecting k-means clustering using intermediate cooperation. journal of pattern recognition, 42(11), 2557-2569. magnusson, r., & opie, h. (1998). patient access to medical records: fiduciary duties and other issues a classroom interactive. university of tasmania law review 99, 17(2). nsw law reform commission. (2010). protecting privacy in new south wales. nsw law reform commission. quinlan, j. (1996). improved use of continuous attributes in c4.5. journal of artificial intelligence research, 4, 77-90. rafaeli, s., & hutchison, d. (2003). a survey of key management for secure group communication. acm computing surveys, 35(3), 309-329. rahman, m. a., & islam, m. z. (2011). seed-detective: a novel clustering technique using high quality seed for k-means on categorical and numerical attributes. paper presented at the ninth australasian data mining conference (ausdm 11). rahman, m. g., & islam, m. z. (2014). fimus: a framework for imputing missing values using co-appearance, correlation and similarity analysis. knowledge-based systems, 56, 311-327. doi: 10.1016/j.knosys.2013.12.005 schwandt, t. a. (2003). three epistemological stances for qualitative inquiry: interpretivism, hermeneutics, and social constructionism. in n. k. denzin & y. s. lincoln (eds.), the landscape of qualitative research (2 ed., pp. 292-331). thousand oaks, ca: sage. 13 australasian journal of information systems burmeister, islam, dayhew & crichton 2015, vol. 19, selected papers from acis enhancing client welfare through better communication simmons, g. j. (1979). symmetric and asymmetric encryption. acm computing surveys, 11(4), 305-330. verykios, v. s. (2004). state of the art in privacy preserving data mining. acm sigmod record, 33(1), 50-57. wang, j., yu, a., & zhang, l. q. (2009, 8-9 august). a dynamic data integration model based on soa. paper presented at the isecs international colloquium on computing, communication, control, and management (cccm 2009), sanya, china. xu, z., zhang, s., & dong, y. (2006). mapping between relational database schema and owl ontology for deep annotation. paper presented at the ieee/wic/acm international conference on web intelligence, washington, dc, usa. an earlier version of this paper was presented at the australasian conference on information systems (acis) 2014 in auckland, new zealand. copyright: © 2015 burmeister, islam, dayhew, crichton. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. 14 http://creativecommons.org/licenses/by-nc/3.0/au/ http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 the principle of ‘keeping confidence’ 3 privacy legislation 4 methodology 4.1 sampling 5 results 5.1 coordination between rural service providers 6 a privacy preserving data sharing model 6.1 a model to improve client welfare outcomes and legislative compliance 7 conclusion 8 acknowledgements paper title (paper title style) australasian journal of information systems clarke 2016, vol 20, research article practical backup for small organisations & individuals 1 practicable backup arrangements for small organisations and individuals roger clarke australian national university roger.clarke@anu.com.au abstract the last thirty years of computing has resulted in many people being heavily dependent on digital data. meanwhile, there has been a significant change in the patterns of data storage and of processing. despite the many risks involved in data management, there is a dearth of guidance and support for individuals and small organisations. a generic risk assessment is presented, resulting in practicable backup plans that are applicable to the needs of those categories of it user. keywords: recovery; security risk assessment; micro-organisation; prosumer 1 introduction large and medium-sized organisations manage their backup and recovery mechanisms within the context of broader disaster recovery and business continuity planning. so do some proportion of small and micro-organisations and even individuals, where they recognise the importance of their data, and have access to sufficient technical and professional expertise. however, in the fourth decade of widespread computer usage, in australia alone, more than half-a-million small organisations and some millions of consumers are highly dependent upon data, and remain at risk if their data suffers harm. yet straightforward guidance on how micro-organisations and individuals should address those risks is surprisingly difficult to find, with multiple searches in such venues as ais elibrary and google scholar delivering relatively few sources of significant relevance. only a minority of text-books contain segments, in most cases brief, e.g. slay & koronios (2006, p.23), boyle & panko (2013, pp. 487-502). backup appears to be too prosaic a topic to attract attention from is researchers. for example, in over 500 articles published in ajis during its first 23 years, not one has the word 'backup' in the title and only 17 (3.5%) even contain the word. in the entire ais elibrary, whose collections date back in many cases for 20 years, and in the case of misq 40 years, 3 articles have the word in the title, and a further 7 have it in the abstract, in a corpus of over 30,000 entries. moreover, none of those papers offered any important contribution to the research reported in this paper. relevant sources found during the course of the research are cited in the particular sections in which their contributions are relevant to the analysis. a possible reason for the absence of a visible literature is that academics have assumed that backup and recovery are old problems, long since solved. that is not a tenable proposition, however, because the now three decades of ‘personal’ computing have involved significant changes in a variety of areas. organisational hierarchies have been giving way to networks of smaller entities, with a great deal of activity outsourced. many workforces have been subject to casualisation. desktops and laptops have been giving way to handheld devices. organisation-provided devices have been complemented and then to some extent replaced by bring your own device (byod) arrangements. the location of data and application software has switched from the desktop and nearby servers to distant service-providers, initially in a defined physical place but now 'in the cloud' (clarke 2011). these changes have brought with them an increased range of threats (e.g. phishing, ransomware) and increased intensity of existing threats. the low quality of software has brought with it an increased range of vulnerabilities. devices are more opaque than before, particularly smartphones and tablets, where apple has driven the industry away from generalaustralasian journal of information systems clarke 2016, vol 20, research article practical backup for small organisations & individuals 2 purpose computing devices and towards supplier-controlled and -limited 'appliances'. most users have little interest in understanding the complexities involved, and limited capacity to comprehend them. human-computer interfaces increasingly reflect a strong emphasis on hedonism, with convenience and excitement as the primary objectives, and little attention paid to risks. in the new context, a re-visit to the topic of backup is essential. the problem addressed in this paper is the absence of up-to-date guidance for microorganisations and individuals in relation to the design and conduct of backup and recovery. large and medium-sized organisations have access to specialist expertise. a proportion of small and micro-organisations also employ information technology (it) professionals or contract them on an intensive basis. the focus of this research, however, is on individuals, and on micro-organisations and small organisations, that (rightly or wrongly) do not perceive it and computer-readable data as being central to their work, and that have modest or very limited competence in it matters. a particular focus is on individuals who make relatively sophisticated use of computing facilities for such purposes as the management of personal finance, tax and pension fund, and correspondence, databases of images, videos or audio, or family-trees. the notion of the 'prosumer', coined by toffler (1970, 1980), has progressively matured (tapscott & williams 2006, clarke 2008). a prosumer is a consumer who is proactive (e.g. is demanding, and expects interactivity with the producer) and/or a producer as well. in the context of computer usage, a third attribute of relevance is professionalism, to some extent of the person themselves but also in relation to their expectation of the quality of the facilities and services they use. a second focus in this work is on very small organisations that involve one or two individuals. such micro-organisations are within-scope whether they are incorporated or not, whether their activities are stimulated by economic or social motivations, and whether they are forprofit or otherwise. some categories of small organisation with up to 20 staff or contractors have similar characteristics and needs to micro-organisations. the relevance of this work extends further, however. during the last two centuries, workers were mostly engaged full-time by organisations under 'contracts of service', and a large body of employment law developed. the last few decades have seen increasing casualisation of workforces, with large numbers of individuals engaged through 'contracts for services'. this requires them to take a far greater degree of self-responsibility. to the extent that large organisations depend on sub-contractors' use of computing and management of data, the subcontractors' security risks impact upon the organisations that engage them. the scope of the work is defined as 'backup and recovery'. this excludes broader security issues such as firewalls and the encryption of communications, except to the extent that they have a potential impact on the values that the individual perceives in data. the prosaic-sounding and somewhat dated terms 'backup and recovery' have been used intentionally. larger organisations may benefit from applying the broader concepts of business continuity planning and disaster recovery strategies, whereas the horizon of the majority of small organisations and individuals is unlikely to extend that far. the purpose of the research was defined as: to develop practical guidance on how small organisations and individuals can use backup techniques to address data risks reflecting that purpose, this paper is not addressed exclusively to researchers. the emphasis is on practicality, and the expression is intended to be accessible to professionals as well, with unnecessary intellectualisation of the issues avoided. the following section examines a key determinant of appropriate backup strategy, namely the pattern of data-storage and processing that the prosumer or micro-organisation adopts. a description is provided of how the design science approach was applied in order to address the problem. there is considerable diversity among the categories of users targeted by the research, so it is necessary to define a test-case to which the method can be applied. the main australasian journal of information systems clarke 2016, vol 20, research article practical backup for small organisations & individuals 3 body of the paper summarises the outcomes of the risk assessment process. this enables three backup plans to be presented, corresponding to three different patterns of it use. 2 patterns of data-storage and processing considerable changes have occurred during the last three or four decades in the locations in which data is stored, and in which it is processed. in order to provide guidance in relation to backup arrangements, it is therefore necessary to first identify the key distinctions among the alternative patterns. as table 1 depicts, the original pattern was for processing to be performed locally, with the data stored within the device, or on a nearby computing device. soon afterwards, the alternative emerged of using a storage-device attached to the local area network and commonly referred to as networked-attached storage (nas). within-device and nas approaches are categorised here as ‘self-sufficiency’, in order to convey the substantially independent nature of the person or organisation. the need to maintain ‘off-site’ or ‘fire’ backups is one particular challenge arising from the selfsufficiency model. a second pattern emerged, whereby another party provides support for the functions of creating and managing off-site backups. this is identified in table 1 using the term ‘backup service’. short description indicative timeframe location of the primary copy location of the backup copy user experience 1 self-sufficiency 1980local local demanding 2 backup service 1990local remote a little easier 3 file-hosting 2000remote local easier still 4 saas 2010remote remote easiest table 1: alternative data-storage patterns a third pattern became common, referred to in table 1 as ‘file-hosting’. this arose as the capacity of wide area networks increased, and transmission costs decreased. another important factor was the proliferation of device-types, with desktops and laptops (variously personal and/or employer-provided) complemented by pcs in internet cafés, airport lounges and customers' premises; and more recently the user's own smartphone and tablet. increasingly, individuals used multiple devices, and in multiple locations. the use of filehosting services represents outsourcing of the operation of the nas. users draw down copies of files when they need them, onto whichever device they are using at the time. changes made to the user's copy need to be uploaded to the file-host. where copies of files are maintained permanently on users' devices (e.g. address-books and diaries / calendars), processes variously referred to as replication, mirroring and synchronisation need to be implemented. a fourth pattern has since become increasingly common, indicatively from about 2010 onwards. whereas, with the 'file-hosting' approach, processing continues to be performed locally to the user, the fourth pattern involves the service-provider not only hosting the user's data, but also performing much of and even all of the processing. currently conventional terms for this include cloud computing and (application) software as a service (saas – armbrust et al. 2009, clarke 2011). the scope of the research conducted extends to all four of these patterns. this paper, however, reports on the outcomes of the research in relation to only the first three patterns. a companion paper considers appropriate backup approaches in the context of saas (clarke (2016). the three patterns of use identified in table 1 need to be articulated further, in order to establish a firm foundation for the analysis and design of backup plans. a preliminary decision australasian journal of information systems clarke 2016, vol 20, research article practical backup for small organisations & individuals 4 of relevance is the means that the individual uses to achieve sufficient synchronisation among their multiple platforms. in a multi-platform environment, there are likely to be multiple copies of files. clarity is needed as to which file is the primary copy. a basic arrangement involves the designation of one computing device as the master typically the desktop with the others managed as slaves. the primary copy of the data is on the masterdevice, and this is mirrored forward from the master to the slaves at convenient times. this is generally done when they have a suitable connection to the master, and there is little or no conflict with other activities, e.g. when all are on the local network overnight. where a slave device is used to create a new file or amend an existing file (e.g. while the user is away from the office), special care is needed to achieve disciplined backloading to the master-device. a second arrangement utilises as the primary storage-medium a nas on the individual's local area network. any of the computing devices may create new files and amend the primary data, with a file-locking mechanism used to ensure that at any time only one of the computing devices has the capability to amend each file. where a new or amended file is stored on one of the devices (e.g. because a network connection to the nas cannot be achieved at the time), special care is needed to achieve disciplined backloading to it. a step beyond nas is a redundant array of independent disks level 1 (raid), of which raid1 is the form relevant to the present discussion. this is essentially a nas containing two disks, with all write activities occurring on both disks. raid1 addresses the specific risk that one disk will be lost (in particular, because of a disk crash). all of the master-slave, nas and raid approaches are part of the first, self-sufficiency pattern. a further development is to arrange for someone else to host the nas or raid. this enables access from multiple devices in multiple locations, and delegates the device management to a specialist organisation, but it means that all of the individual’s platforms become slaves to the server operated by the outsourced service-provider, and it brings with it a heavy dependency on reliable telecommunications links. care is still needed to deal with the risks of multiple, inconsistent copies of files. additional data risks arise in the forms of exposure to second-party access (i.e. by the service-provider), and of more circumstances in which third-party access to or corruption of the data may occur, in particular because of the increased extent of filetransmission over external networks, and the greater attractiveness of service-providers to hackers (the ‘honeypot effect’). a master-slave arrangement may be easier for a small organisation to understand and manage, whereas storing the primary copy on a nas or raid1 device requires additional understanding and infrastructure. the use of a remote file-hosting service requires further understanding, may increase costs, and creates a dependency on a provider, but if the service is well-designed and communicated, reliable and cost-effective, it can relieve the user of a considerable amount of effort. the choice of file-synchronisation approach influences the decision as to which pattern to use for backups. this leads to the following further articulation of the first three patterns outlined in table 1: 1. self-sufficiency the primary copy is held locally. file-synchronisation is achieved using master-slave arrangements, or local nas or raid. backups are managed locally. because both the primary and the first-level backup are subject to similar threats and vulnerabilities, it is essential that a second-level backup (popularly referred to as 'fire backup'), be established at a remote location. 2. backup service the primary copy is held locally. file-synchronisation is achieved using master-slave arrangements, or local nas or raid. backups use a service provided by another party. australasian journal of information systems clarke 2016, vol 20, research article practical backup for small organisations & individuals 5 the primary and first-level backup copies are in different places, and hence a secondlevel backup might be considered superfluous. on the other hand, additional risks arise, such as the service-provider defaulting on their undertakings. hence a second-level backup, in particular one stored locally, remains highly advisable. 3. file-hosting service the primary copy is held remotely. file-synchronisation is achieved by means of the remote server. the supplier may offer a backup service as part of the arrangements or additionally, in which case reliance might be placed on that backup service. on the other hand, further risks arise, such as service-provider default, and storage of the first-level backup colocated with the primary copy and hence subject to the same threats. hence a second-level backup, in particular one stored locally, remains highly advisable. 3 method and test-case the work reported in this paper adopted the design science approach to research (hevner et al. 2004, hevner 2007). in terms of the research method described by peffers et al. (2007), the research’s entry-point is ‘problem-centred’. the process commences by applying risk assessment techniques in order to develop an articulated definition of the problem and of the objectives. an artefact is then designed – in this case a set of backup plans applicable to three patterns of it use. in terms of hevner (2007), the article’s important contributions are to the requirements phase of the relevance cycle, and the design cycle, drawing on existing theory in the areas of risk assessment, data management, and data security. the paper makes more modest contributions to the evaluation phase of the relevance cycle, but lays firm foundations for application and field testing. in this section, the existing literature is applied in order to define terms and specify a risk assessment process to support the analysis. within the scope declared above, a target-segment is defined that is both realistic and sufficiently rich to provide both a test of the method and an outcome that is useful in its own right. the risk assessment process is then applied to the testcase, to produce a sufficiently deep understanding of the needs of that category of users. practicable backup plans are presented, for each of the three patterns of it use. the assessment of risk, and the development of guidance for backups, needed to be based on a model of security, including a set of terms with sufficiently clear definitions. a substantial literature provides the framework for the analysis. this includes oecd (2002), firesmith (2004), iso 27002:2005, ietf (2007), cc (2012, pp. 38-39) and clarke (2015). appendix 1 provides a depiction of the conventional computer security model, and a glossary of terms. a brief summary of the model is that: • threatening events impinge on vulnerabilities, resulting in security incidents that may give rise to harm to assets • safeguards provide protections against threatening events, vulnerabilities and harm; and • security is a condition in which harm is in part prevented, and in part mitigated, because adequate safeguards are in place a suitable backup and recovery plan can only be established if alternative designs are outlined and compared with a set of requirements. a process is needed, comprising a series of steps that apply the conventional security model in order to unfold an understanding of the needs of the entities within the project’s scope. the refereed literature provides references on the discipline and practice of risk assessment and risk management, with particular reference to iso 27001:2005, nist (2012), iasme (2013), clarke (2013) and clarke (2015). there are, however, relatively few sources that formally articulate risk assessment and risk management processes in particular contexts relevant to the present analysis. an exception is shedden et al. (2011), which provides a case australasian journal of information systems clarke 2016, vol 20, research article practical backup for small organisations & individuals 6 study of a complex data backup process. the above literature was drawn on in order to establish the process declared in table 2. its purpose is to avoid subtleties and complexities in order to scale the process to the contexts of the target organisations and individuals. the testcase excludes individuals likely to be subject to targeted attacks, so the analysis pays little attention to countermeasures that may be adopted by attackers to circumvent the individual's safeguards. in principle, this process needs to be applied to the specific context facing a particular organisation or individual. in practice, most of the intended clientele would still find the process far too demanding. a practical compromise is to define a small set of client segments, apply the process to each of these segments, test the resulting recommendations in the field, and publish the outcomes to organisations and individuals in those segments. stratification of the broad domains of organisations and individual users could be performed in a variety of ways. industry sectors have varying needs, and hence analyses could be undertaken for a garden designer, a ceramic artist, a technical writer, a structural engineer, a motor vehicle repairer, a marriage-celebrant, a genealogist, and a cause-based advocate. however, many of these have similar assets and values, face similar threats, are afflicted with similar vulnerabilities, and trade off various factors in similar ways. it therefore appears feasible to define a smaller number of categories than would arise with a sector-based analysis, by focussing on the nature and intensity of the risks that the client faces. analyse 1. define the objectives and constraints 2. identify the relevant stakeholders, assets, values and categories of harm 3. analyse threats and vulnerabilities 4. identify existing safeguards 5. identify and prioritise the residual risks design 1. identify alternative backup and recovery designs 2. evaluate the alternatives against the objectives and constraints 3. select a design (or adapt / refine to achieve an acceptable design) do 1. plan the implementation 2. implement 3. review the implementation table 2: the process this paper presents a single such test-case. the criteria used in devising it were: • simplicity, in order to keep the scale of the work and its presentation within bounds • representative value, i.e. including a moderate proportion of the elements that arise in more complex cases, such that it can reasonably be used as a proxy for prosumers more generally, and for micro-organisations; and • familiarity to the author, in order to take advantage of existing knowledge and obviate the need for field-work during this phase of the project the selected case is a person who is a moderately sophisticated user of computing devices, but has limited professional expertise in information technology matters. they use their australasian journal of information systems clarke 2016, vol 20, research article practical backup for small organisations & individuals 7 computing devices for personal activities and/or in support of one or more organisations. the functions performed are primarily: • preparation and amendment of documents • maintenance of data-sets and databases in a variety of formats, including text, structured data, image, sound and video • exchange of communications with other people • access to web-sites • maintenance of their own web-sites • the use of internet banking and ecommerce, but only as a purchaser, not as a merchant the person operates out of a home-office that is equipped with a desktop device. when travelling, the person carries a portable / laptop / clam-shell device. the person has a handheld device, and uses this to access messages and send messages using a variety of channels (voice, sms, email, im), and to access web-sites. the laptop and handheld device may also be used within the home-office. the person copies files between the desktop and the other devices as needed. many of the files that the person creates are sent to other people, so if a file is accidentally deleted or damaged, it may be possible to rescue a copy from somewhere else. but the person has experienced several instances in which important files were simply lost, and needs to avoid that happening. use of services offered by third parties (internet service providers, isps) is within-scope for such mainstream activities as the hosting of email and web-sites. use of cloud computing, on the other hand, has been excluded from the case, because of the many additional factors this gives rise to. for example, the person might use cloud services for messaging, for storage of the primary copies of photographs, for storage of their address-book and appointments diary, for their log of past dealings with each contact in their address-book, or for their accounting records. analyses of the risks involved in consumer uses of cloud computing, and approaches to dealing with them, are in clarke (2011, 2013, 2016). the test-case excludes circumstances in which the individual is likely to be a specific target for attackers, as distinct from being just another entity subjected to random unguided attacks by malware and social engineering techniques. hence the trade-offs selected during this analysis are not appropriate to, for example, private detectives, and social and political activists who are likely to be directly targeted by opponents and by government agencies. 4 application of the process to the test case this section applies the process outlined in table 2 to the test-case defined immediately above. the discussion in the sections below reflects relevant sources on risk assessment and risk management. references are not cited in a single, consolidated exposition of existing theory. it is more appropriate to the progressive analysis presented in this paper to provide references in the particular sections to which they make contributions. 4.1 analysis this section follows the steps specified in the first segment of table 2. 4.1.1 objectives and constraints as a reference-point, the following definition is proposed of the individual's purpose and the constraints within which the design needs to work (clarke 2015): to avoid, prevent or minimise harm arising from environmental incidents, attacks and accidents, avoiding harm where practicable, and coping with harm when it arises, by balancing the reasonably predictable financial costs and other disbenefits australasian journal of information systems clarke 2016, vol 20, research article practical backup for small organisations & individuals 8 of safeguards against the less predictable and contingent financial costs and other disbenefits arising from security incidents on the other hand, the target audience needs a simpler formulation, such as: to achieve reasonable levels of security for reasonable cost 4.1.2 stakeholders, assets, value, harm literature searches found no refereed sources or standards documents that articulate the process of stakeholder analysis. in the case of an individual, the stakeholders comprise the individual themselves, the individual's family, any employees and sub-contractors, and any clients, whether of an economic or a social nature. in the case of micro-organisations, there may be additional stakeholders, such as employees, customers, suppliers, and perhaps an advisory committee. also within-scope are some categories of associations with membership and committee structures and small, multi-member enterprises such as investment clubs. in some contexts, regulatory agencies may loom at the level of stakeholder, e.g. for accountants, financial planners, marriage celebrants and health care professionals. the assets on which this study is focussed are data. a useful resource in this area is iso 27005 (2012, annex b). it equipment and services on which the individual depends are only withinscope to the extent that they play a role in the protection of data assets. relevant categories of assets are listed in table 3. • business-related content such as databases, transaction files, reports, work-in-progress, sources of data and information, customer information, details of outstanding debts • financial data – such as records of assets and transactions, insurance details • payment authenticators – such as pins, credit-card details • identity authenticators – such as passwords, passport, driver's licence details • funds – such as bitcoin wallets • personal data, in some cases of a sensitive nature:  of an individual – e.g. diaries, address-books, music collections, health-related data  of the individual's family – e.g. family albums, family history, tax return data  of other people – e.g. if the individual performs counselling, mentoring or coaching • infrastructure configuration data such as settings, parameters and scripts that support computing operations table 3: relevant data assets the values that stakeholders attribute to assets derive from a variety of sources (clarke 2013), in particular: • intrinsic value, as arises from debtors ledgers, share registers and bitcoins • operational value, as arises from the data's usefulness in performing a function, such as publishing images and scheduling meetings • competitive value, as arises both from the data's usefulness to the individual and its potential usefulness to other parties, including economic competitors and strategic competitors • reputational value, as arises from the data's capacity to influence the perceptions that other parties have of the individual or of some other party • compliance value, as arises from obligations, e.g. in relation to the protection of the data australasian journal of information systems clarke 2016, vol 20, research article practical backup for small organisations & individuals 9 • psychic value, as arises from personal associations, particularly with image, video and music collections, and family trees • privacy value, as arises from the concerns of a person to whom data relates (particularly health and financial data), whether or not the concerns are rational, and whether the value is associated with economic, social, psychological, political, religious or philosophical factors values associated with data involve a considerable set of attributes referred to in the literature using various terms, such as 'properties'. one concept of long standing is the 'cia' list, which stands for confidentiality, integrity and availability (saltzer & schroeder 1975). this convention is much-criticised, and many alternatives and adjuncts have been offered. for example, parker (1998) added possession, authenticity and utility; and cherdantseva & hilton (2013) add instead accountability, auditability, authenticity, trustworthiness, nonrepudiation and privacy. however, such lists lack clarity because they confound properties of data with properties of the infrastructure used to achieve access to the data. particularly for such purposes as backup and recovery strategies, the following areas are only indirectly relevant: • the availability of infrastructure, including computing and networking hardware and systems software • the availability of services, including application software, data and supporting services, whether it is full availability, degraded availability (as occurs with a business continuity plan that involves fallback arrangements), or complete unavailability (during interruptions to communications and while disaster recovery procedures are being undertaken) • the integrity of services • the robustness of services, i.e. their ability to withstand shocks • the resilience of services, i.e. their ability to quickly recover and resume service, following shocks this analysis is concerned specifically with the value attached by stakeholders to data. it accordingly applies the set of factors in table 4. the primary three values encompass the relevant aspects of the lists referred to in the previous paragraph, but separate out the confusing effects of multiple purposes. the third value is then disaggregrated into its constituents. this reflects sources on data quality and integrity in the information systems and related literatures, including oecd (1980), huh et al. (1990), van der pijl (1994), clarke (1995, pp. 601-605), wang & strong (1996), müller & freytag (2003, pp. 8-10), english (2006), piprani & ernst (2008) and the iso 8000 series emergent since 2009. harm to values in data needs to be considered at two levels. a useful resource in this area is iso 27005 (2012, annex b, pp. 39-40). categories of harm to data itself are listed in table 5, and the forms of consequential harm to stakeholders' values are listed in table 6. australasian journal of information systems clarke 2016, vol 20, research article practical backup for small organisations & individuals 10 • accessibility the data is accessible to appropriate entities in appropriate circumstances • inaccessibility the date is otherwise not accessible • quality the data adequately satisfies all dimensions of data integrity: o accuracy the degree of correspondence of the data with the real-world phenomenon that it is intended to represent, typically measured by a confidence interval, such as 'accurate to within 1 degree celsius' o precision the level of detail at which the data is captured, reflecting the domain on which valid contents for that data-item are defined, such as 'whole numbers of degrees celsius' o timeliness, which comprises distinct elements:  currency the absence of a material lag between a real-world occurrence and the recording of the corresponding data  temporal applicability the absence of ambiguity about the date and time when, or the period of time during which, the data represents or represented a real-world phenomenon. this is important in the case of volatile data-items such as total rainfall for the last 12 months, marital status, fitness for work, age, and the period during which an income-figure was earned or a licence was applicable • completeness the availability of sufficient contextual information that the data is not liable to be misinterpreted. of particular concern are provenance, and the data's syntax and semantics table 4: relevant values associated with data australasian journal of information systems clarke 2016, vol 20, research article practical backup for small organisations & individuals 11 • accessibility the data is not accessible to appropriate entities in appropriate circumstances o data loss  data in volatile memory is dependent on continuous functioning of the cpu and electrical power  data in non-volatile memory is at risk of being over-written, in many cases at file-level and in some cases at record-level within databases  data storage-media and data storage-devices containing storage-media are subject to theft, destruction and malfunction o data unavailability at a relevant time, e.g. due to shortfalls in infrastructure performance • inaccessibility the data is otherwise accessible. this takes several forms: o data access, whereby data in storage is accessed by an inappropriate person, or for an inappropriate purpose o data disclosure, whereby data in storage is communicated to an inappropriate person, or for an inappropriate purpose o data interception, whereby data in transit is accessed by an inappropriate person, or for an inappropriate purpose • quality the data does not adequately satisfy all dimensions of data integrity: o data quality is low at the time of collection o data quality is low at the time of use, due to data modification or loss of data integrity • completeness the data does not adequately satisfy the requirement, in particular the availability of the data’s provenance and the definitions of the data's syntax and semantics table 5: harm to values associated with data australasian journal of information systems clarke 2016, vol 20, research article practical backup for small organisations & individuals 12 • reduced asset value e.g. loss of a debtors ledger or prospects database with intrinsic value • degraded operational capacity tasks cannot be performed • degraded service quality tasks cannot be performed well • reduced revenue or amenity (depending on whether the purpose is economic or social) • cost, time, effort and economic loss incurred during recovery incl. the acquisition of backup data, the performance of recovery procedures, transport and communications, and the replacement of payment or identity authenticators • damaged reputation incl. the confidence of family, employees, customers, investors or regulators • negative privacy impact on individuals e.g. through unauthorised access to personal data, or disclosure or interception of personal data • non-compliance with obligations or commitments e.g. through loss of tax records table 6: harm to stakeholder values arising from harm to values associated with data 4.1.3 threats and vulnerabilities this section follows convention by identifying lists of threats and of vulnerabilities, although the distinctions can be challenging to make, and hence it is often more practicable to consider threat-vulnerability combinations. catalogues of threats and vulnerabilities are available from a variety of sources, most usefully iso 27005 (2012, pp. 42-49) and nist (2012, pp. 65-76). these are reflected in table 7 and table 8. all elements within these tables can of course be analysed in greater detail. for example, a deeper treatment of malware is in clarke (2009), and of social engineering in mitnick & simon (2003). a means is needed to invoke the range of threats but in a simplified form that is easier for the target-audience to grasp and to remember. a convenient approach is to adopt a single instance of each category of threats as being representative of the category, and to contrive the first letter of each to build a word. one such mnemonic is ‘fate’: f fire (for environmental events) a attack t training (for accidents caused by humans) e equipment (for accidents within infrastructure) the term ‘fate’ is in no sense a theoretical construct. its purpose is merely to provide prosumers and people in micro-organisations with a readily-remembered basis for imposing a degree of organisation on the wide variety of threats that may result in harm to their interests. 4.1.4 existing safeguards before conclusions can be reached about what risks need to be addressed, it is necessary to take into account factors that already exist and that intentionally or incidentally mitigate risks. common patterns of human behaviour such as habit, caution and loyalty have the effect of protecting stakeholders’ interests, and can be reinforced through training and reminders. longstanding practices in relation to physical security help as well, such as locks and smoke australasian journal of information systems clarke 2016, vol 20, research article practical backup for small organisations & individuals 13 alarms. aspects of infrastructure assist, such as those resulting from contractual terms and 'fitness for use' conditions imposed by the laws of contract and consumer rights. suppliers have a self-interest in delivering goods and services of reasonable quality, in order to sustain their reputation. logical security precautions are widely used, particularly in the form of accounting controls. insurance provides monetary recompense for financial losses, but also imposes requirements for some level of safeguards to be established and maintained. environmental event • electrical event (interruption, surge) • fire event • water event • impact event attack • on data storage o sabotage o theft o seizure • on traffic o interception • on a business process o abuse of privilege, e.g. unauthorised disclosure by an insider o masquerade, e.g. access by adopting an authorised identity o social engineering • on a computer-based process o 'hacking' / cracking o malware, incl. ransomware accident, i.e. unintentional error • by a human o business process design error o inadequate training in a business process o business process performance error • within infrastructure o equipment failure (processor, storage device, infrastructure, power) o storage-medium failure o network malfunction o data incompatibility table 7: threats to data australasian journal of information systems clarke 2016, vol 20, research article practical backup for small organisations & individuals 14 infrastructural vulnerabilities • dependence on the availability, reliability and integrity of: o power supply, subject to the threats of blackouts, brownouts, voltage variability, ups failure o computing facilities, subject to the threats of planned and unplanned downtime, unavailability of a storage-device that can read a particular storagemedium, seizure powers o networking facilities, subject to the threats of outages, congestion, dos attack o storage-media, subject to the threats of disk crash, corruption, encryption/hostage/ransom, loss, online accessibility of live and backup data at the same time, seizure powers, unreadability due to humidity, dust, magnetic disturbance, corrosion, etc. o ancillary services, e.g. air-conditioning, fire equipment, subject to the threats of outages and malfunctions o automated processes, subject to the threats of design and coding errors, malware, wrong versions of software or data, erroneous recovery of software or data, overwrite of valid backups with corrupted backups • dependence on the effectiveness of access controls over: o authenticators o software execution o remote access o message transmission o encryption and decryption human vulnerabilities • dependence on the availability, reliability and integrity of individuals, subject to the threats of: o inadequate performance o inadequate training o inadequate loyalty o insufficient wariness and scepticism table 8: data vulnerabilities 4.1.5 residual risks the final step in the assessment process is the identification and prioritisation of the 'residual risks', i.e. those that are not satisfactorily addressed by existing safeguards. the conventional approach to prioritisation is to assign to each residual risk a severity rating and a probability rating, and to then sort the residual risks into descending order, showing extreme ratings in either category first, followed by high ratings in both, etc. this is most comprehensively presented in nist (2012, appendices g, h and i). in principle, these are context-specific judgements that need to be made by the responsible individual, or by someone closely familiar with the individual's needs and circumstances. the analysis conducted here, however, assigns severity and probability ratings on the basis of the test-case described earlier. the results are summarised in table 9. australasian journal of information systems clarke 2016, vol 20, research article practical backup for small organisations & individuals 15 risk severity rating (e, h, m, l) probability rating (h, m, l) storage-media failure denying access to all files extreme high environmental event, destruction, theft or seizure denying access to the storage-medium extreme medium malware or hacking attack denying access to all of the data extreme medium malware or hacking attack resulting in inability to access a file high medium mistaken amendment, deletion or overwriting of a file high medium individual file-corruption: • discovered within-cycle • discovered after more backups have been run high high medium medium environmental event resulting in inability to access a file high medium software error resulting in inability to access a file high medium unavailability of networking facilities resulting in inability to access a file medium medium technological change causing a storage-medium to be unreadable low low table 9: priority threat-vulnerability combinations 4.2 design and implementation this section applies the remaining steps defined in table 2. a range of alternative approaches to backups exists. drawing on the literature, most usefully chervenak et al. (1998), plus lennon (2001), gallagher (2002), preston (2007), de guise (2008), strom (2010), tob (2012) and cole (2013), appendix 2 identifies relevant characteristics of backup data, and of backup processes, and appendix 3 describes each of the various categories of backup procedure. key considerations in designing a backup regime include the frequency with which full backups are performed, whether incremental backups are performed and if so how frequently, whether copies are kept online or offline, and whether second-level archives are kept and if so whether they are later over-written or archived. one of the most critical choices, however, is whether the first-level backup is stored locally or remotely. in appendix 4, a summary is provided of the extent to which different backup techniques address the various risks that afflict both individual files and the primary storage-medium as a whole. the information generated by the preceding sections enables a judgement to be made about what combination of approaches to platform-management and backups is most appropriate. a scheme that would cover every possible eventuality is highly likely to be too complex and too costly for a micro-organisation or a prosumer. important factors that need to be considered are identified in table 10. commonly, cost and complexity, on the one hand, are traded off against protections against the lower-priority threat-vulnerability combinations. an individual or small organisation may be able to directly utilise the outcomes from a generic analysis and design process such as that presented above. the responsibility for converting a risk management strategy to a reality, on the other hand, rests on the individual or organisation concerned. broadly, the following steps are necessary: australasian journal of information systems clarke 2016, vol 20, research article practical backup for small organisations & individuals 16 • plan the implementation establish policy, procedures, controls and training • implement the plan rehearse the procedures, run the procedures, check the results • review the implementation reconsider the procedures and outcomes after 1 month and 6 months the discussion of analysis, design and implementation in this section has been framed in a sufficiently general manner that individuals and small organisations confronted by a wide variety of circumstances could apply it. the following section presents the outcome applicable to the test-case defined earlier in the paper. risk management • the risks that are safeguarded against equipment • operational storage size • backup storage size • processor, bus and local network capacity • external network connection and capacity operation • batch backup run-times • (and consequential service unavailability or qualified availability) • recovery run-time • speed of recovery from a security incident • complexity of strategy, plan, policies and procedures • concentration and effort needed to implement the plan cost • one-time costs of safeguards • recurrent costs of safeguards • costs of each kind of security incident table 10: factors to be traded off 5 practicable backup plans in table 1, three patterns of use were distinguished. this section applies the assessment conducted above to each of those three patterns. it presents three backup plans that address all of the high-priority threat-vulnerability combinations identified in table 9, and do so in a manner that is not unduly complex or expensive. 5.1 self-sufficiency the first backup plan applies to circumstances in which the storage-medium on which the backup is stored is on the premises, and hence co-located with the primary copy. this may be by direct connection to the master-device, typically the desktop, or over a local area network. a review of wintel-oriented backup software is in mendelson & muchmore (2013), and another for mac osx environments is in preece (2016). examples of products that satisfy a significant australasian journal of information systems clarke 2016, vol 20, research article practical backup for small organisations & individuals 17 proportion of the requirements are acronis (wintel, mac and linux) and chronosynch (mac only). to address on-site risks (typified earlier as fate – fire, attack, training and equipment), it is necessary that a second-level backup be maintained at a sufficiently remote location. this backup plan is outlined in the first column of table 11. because the test-case encompasses some diversity of needs, a list of essential elements is provided, supplemented by a further set of recommended actions. the actions are further sub-divided into infrastructure features, file-precautions, backup runs and business processes. full detail of the plan is provided in appendix 5, in a form designed as a checklist for micro-organisations and prosumers who want to apply the technique. 5.2 use of a backup service the second backup plan is for circumstances in which the storage-medium to which the backup is performed is located remotely from the primary copy, and the transfer occurs over an internet connection. it is desirable to authenticate the remote device and to use channel encryption. the process can be driven either by a device on the local network – typically the desktop – or by the remote device that has direct access to the backup storage-medium. this backup plan is outlined in the second column of table 11. full detail of the plan is provided in appendix 6, in a form designed as a checklist for micro-organisations and prosumers who want to apply the technique. the plans outlined in table 11 for the self-sufficiency and backup service patterns are substantially the same. the small, but critical, differences are the logistical arrangements for first-level and second-level backups, and safeguards for the content of the remote backup. the remote backup device may be hosted by someone the individual or organisation has associations with (e.g. a business colleague or a relative). alternatively, the hosting may be performed by a service-provider, such as an accountant, a local provider of internet services, a specialist backup provider, or a cloud operator. a commercial catalogue of offerings is in muchmore (2013). a service that appears to score well on many aspects of the requirements is spideroak. it is also possible to use major service-providers as a backup service. evaluating their offerings against these requirements is difficult, however, because reliable information may be difficult to find. for example, postings in mac user fora make clear that apple’s time machine offering is unclear to many users. australasian journal of information systems clarke 2016, vol 20, research article practical backup for small organisations & individuals 18 elements of the backup plan l r essential infrastructure features a. install power-surge protection and an 'uninterruptible' power supply with battery backup (ups) y y b. use the desktop as master, and the laptop and handheld as slaves or maintain the primary-copy on networked attached storage (nas) y y file precautions c. when creating and amending files, perform continual saves y y d. when making significant amendments, first create a new file-version y y e. run malware detection and eradication software a. on each storage-device at the time it is connected to any working-device b. on all incoming files arriving via email, fetches using a web-browser, etc. y y backup runs f. a. perform 3-monthly full backup to a separate local storage-medium y y b. perform daily incremental backup to the same storage-medium y y g. a. perform weekly, fortnightly or monthly full backup to a rotating set of 2, 3 or 4 storage-media, as second-level backup y y b. promptly transport second-level backup storage-media to a remote location y c. store the second-level backup storage-media locally y d. store the second-level backup storage-media offline y y e. fetch the relevant second-level backup storage-media shortly before each backup run y h. a. annually, and after each significant upgrade to software, perform a complete disk-image backup of all working-devices , including all software and parameterfiles y y b. store the resulting disk-images remotely and offline y y business processes i. document and periodically rehearse backup procedures y y j. document and periodically rehearse recovery procedures y y recommended infrastructure features – additional measures k. implement a virtual private network (vpn) connection from laptop and handheld back to the local network y y file precautions – additional measures l. weekly, run malware detection and eradication software on all stored files y y backup runs – additional measures m. ensure that: • the remote online first-level backup is encrypted • the decryption key is stored locally • the decryption key is not accessible by the service-provider y n. a. half-yearly, retire a full backup to archive y y b. store successive archive copies locally and remotely, and possibly also on a third site y y o. annually, spool 3-year-old archives to new media y y p. 5-yearly, spool all archives to a new media-type y y table 11: backup plans using local (l) and remote (r) backup 5.3 dependence on a file-hosting service the third backup plan applies where the primary copy of the files is held by another party. this has some similarities to the use of a backup service, addressed in the previous section. key differences are, however, that the use of a backup service, by its nature: australasian journal of information systems clarke 2016, vol 20, research article practical backup for small organisations & individuals 19 • ensures that two copies exist – because the backup is in addition to the primary copy; and • maintains physical separation between the local primary copy and the remote firstlevel backup. in the case of a file-hosting service, on the other hand: • there may or may not be a backup copy; and • any backup copy may or may not be subject to the same risks as the primary copy – and will be if they are in the same location or both are accessible online. this backup plan is outlined in table 12. full detail of the plan is provided in appendix 7, in fuller form, as a checklist for micro-organisations and prosumers who want to apply the technique. much of the plan in table 12 is the same as that for the self-sufficiency and backup service patterns. the differences are as follows: • because the location of the primary copy is different, the technique for achieving appropriate synchronisation among multiple copies is different • malware-checking needs to be performed within the file-hosting services site on incoming files; • the daily (or more frequent) first-level backup run is a responsibility of the filehosting service; • the daily (or more frequent) second-level backup run is a responsibility of the filehosting service; • malware-checking needs to be performed within the file-hosting services site on stored files; • the primary copy of the data at the file-hosting service should be encrypted; • at least one recent backup should be in the possession of the individual or organisation. file-hosting services have gone through several generations. initially services were offered by internet access providers, as a form of value-add. then came consumer-oriented products typified by dropbox (since c. 2007). a further round has been cloud-based services, typified by apple icloud (since 2011) and google drive (since 2012). some are primarily outsourced data-storage services. others focus on providing their customers with access to their files from multiple devices and from any location, and are sometimes described as 'file-synchronisation' services. others are primarily to enable files to be provided by one user and made available to others. yet others are intended to support documents developed collaboratively by multiple people. some support files generally, and are agnostic about what formats the files are in. some, however, may use proprietary file-formats, a feature which is hostile to the purpose considered here. australasian journal of information systems clarke 2016, vol 20, research article practical backup for small organisations & individuals 20 essential infrastructure features 1. install power-surge protection and an 'uninterruptible' power supply with battery backup (ups) 2. a. store the primary copies of all files with a file-hosting service b. process the data using software on the desktop, laptop and handheld, working on local copies for as long as needed c. synchronise the copies of files on the individual's devices to the remote primary copy whenever each device has a suitable connection to the file-hosting service file-precautions 3. when creating and amending files, perform continual saves 4. when making significant amendments, first create a new file-version 5. run malware detection and eradication software a. on each storage-device at the time it is connected to any working-device b. on all incoming files arriving via email, fetches using a web-browser, etc. 6. ensure that the file-hosting service runs malware detection and eradication software on all incoming files backup runs 7. ensure that the file-hosting service maintains a full backup to a separate storage-medium no less frequently than daily 8. a. ensure that weekly, fortnightly or monthly full backup is performed by the file-hosting service to a rotating set of 2, 3 or 4 storage-media, as second-level backup b. ensure the second-level backup storage-media are stored at a different location from the first-level backup c. ensure that the second-level backup storage-media are stored offline 9. a. annually, and after each significant upgrade to software, perform a complete disk-image backup of all working-devices , including all software and parameter-files b. store the resulting disk-images remotely and offline business processes 10. document and periodically rehearse backup procedures 11. document and periodically rehearse recovery procedures recommended infrastructure features – additional measures 1. implement a vpn connection from all of the individual's devices to the service-provider file-precautions – additional measures 2. weekly, run malware detection and eradication software on all locally stored files 3. ensure that the service-provider runs weekly malware detection and eradication software on all stored files backup runs – additional measures 4. ensure that: • the primary copy of the data stored at the file-hosting service is encrypted • the decryption key is not accessible by the file-hosting service • the decryption key is stored locally 5. a. perform a monthly or quarterly full backup of all working-devices to a separate local storagemedium or b. ensure that an equivalent arrangement is in place using a saas data escrow service or backup as a service (baas) 6. a. ensure that, half-yearly, the file-hosting service retires a full backup to archive b. ensure that the file-hosting service stores successive archive copies locally and remotely 7. ensure that the file-hosting service, annually, spools 3-year-old archives to new media 8. ensure that the file-hosting service, 5-yearly, spools all archives to a new media-type table 12: backup plan using file-hosting services australasian journal of information systems clarke 2016, vol 20, research article practical backup for small organisations & individuals 21 it is a matter of serious concern that large corporations that offer file-hosting services generally make very little information available, which makes it very difficult to perform a satisfactory evaluation against the requirements expressed in table 12. even large organisations generally have far less market power than apple, microsoft and google, and hence small organisations and prosumers that value their data, and that use file-hosting services provided by such corporations, are subject to unmanaged risk exposures. 6 conclusions this paper has presented an analysis of the backup requirements of micro-organisations and prosumers. it has focussed on a test-case, in order to not merely provide general guidance, but also deliver a specification that fulfils the declared objective of developing “guidance on how small organisations and individuals can use backup techniques to address data risks”, which balances among multiple, inherently conflicting needs. the peffers et al. (2007) research method was applied, commencing with ‘problem-centred initiation’, through the problem definition, objectives formulation and articulation phases, and into the design phase, resulting in three sets of specifications. tables 3–9, which declare assets, forms of harm, data threats, vulnerabilities and priority threat-vulnerability combinations, all represent templates or exemplars that can be applied to similar studies of somewhat different contexts. the research has contributed to is theory in the areas of data management and data security, in particular by articulating risk assessment and risk management procedures. the project’s contributions in relation to the evaluation phase are less substantial. a limited evaluation has been conducted of one of the three backup plans. the author’s profile has a reasonably close correspondence with the test-case defined in table 2. in addition, the processes applied by the author for the last decade have been very similar to that derived for the self-sufficiency pattern in column 1 of table 11 and appendix 5. the backup procedures have been exercised hundreds of times, and the recovery procedures on a modest number of occasions. the author has suffered very few losses of datafiles. the rare exceptions are of two kinds. a few very old files have been discovered to be corrupted (by runs of disk utility software) only after all still-readable backups were similarly corrupted. a somewhat larger number of files (from the period 1984-1992) are no longer accessible because no device is available that can read the storage-medium and/or because no application software is available that can read the data-format. a review of the procedures in light of the analysis reported here highlighted the need for refinements to the author's procedures, and for more assiduous application of them, particularly relating to the periodic rehearsal of recovery processes, and the migration of copies forward from obsolescent to contemporary storage-media. further, the ease with which the self-sufficiency backup plan could be applied the author’s circumstances represents a useful, if weak, form of evaluation of the plan’s ease of application to the target-segment’s needs. the research has laid a firm foundation for is professionals to better address the needs of micro-organisations and prosumers. the specific backup plans proposed above can be used as a basis for evaluating the capabilities of software products that support local backup management, and for evaluating backup services offered by isps. they, and variants of and successors to them, are capable of being productised by providers. these include corporations that sell hardware, that sell operating systems, that sell pre-configured hardware and software, that sell value-added hardware and software installations, that sell storage-devices, and that sell storage services. a further opportunity is for guidance based on the plans to be distributed by industry associations, user associations and clubs, to assist those organisations' members. in order for the outcomes to be exploited, it is necessary for the analysis to be subjected to review by peers, and the feedback reflected in the three plans through the publication of revised versions. the analysis may require adaptation, at least of terminology, in order to be readily applied to specific contexts, in particular the many variants of microsoft windows, apple osx and ios, and linux and android operating environments, and particularly where the australasian journal of information systems clarke 2016, vol 20, research article practical backup for small organisations & individuals 22 individual uses multiple such platforms. the analysis needs to be applied to additional testcases, reflecting the needs of small organisations and individuals whose characteristics are materially different from those addressed by this paper. beyond analytical review, the three specific backup plans derived from the analysis need to be applied, and their effectiveness and practicality evaluated empirically. the analysis also needs to be applied in circumstances in which the individual accepts (but manages) the additional risks involved in relying entirely on networks and remote services – with all the uncertainties of format-compatibility and geographical and jurisdictional location that the cloud entails. those circumstances are addressed in a companion paper (clarke 2016). both analyses may require further adaptation if and when the target market-segments’ usage of general-purpose computing devices (such as desktops and laptops) declines, and datafile creation and amendment comes to be undertaken almost entirely on locked-down appliances (such as smartphones and handheld devices). as individuals increasingly act as prosumers, they become more demanding, and more aware of the benefits of effective but practical backup arrangements. meanwhile, many large organisations are becoming concerned about importing subcontractors' security risks. they can be expected to bring pressure to bear on small organisations and individuals to demonstrate the appropriateness of their backup plans, and to provide warranties and indemnities in relation to them. the work reported here accordingly lays a foundation for significant improvements in key aspects of the data security not only of individuals and small organisations, but also of the larger organisations that depend on them. references armbrust m., fox a., griffith r., joseph a.d., katz r., konwinski a., lee h., patterson d., rabkin a., stoica i. & zaharia m. (2009) 'above the clouds: a berkeley view of cloud computing' technical report no. ucb/eecs-2009-28, uc berkeley reliable adaptive distributed systems laboratory, february, 2009, at http://www.eecs.berkeley.edu/pubs/techrpts/2009/eecs-2009-28.pdf boyle r.j. & panko r.r. (2013) 'corporate computer security' pearson, 3rd ed., 2013 cc (2012) 'common criteria for information technology security evaluation – part 1: introduction and general model' common criteria, ccmb-2012-09-001, version 3.1, revision 4, september 2012, at http://www.commoncriteriaportal.org/files/ccfiles/ccpart1v3.1r4.pdf cherdantseva y. & hilton j. (2012) 'a reference model of information assurance & security' proc. ieee ares 2013 secont workshop, 2-6 september, 2013, regensburg, at http://users.cs.cf.ac.uk/y.v.cherdantseva/rmias.pdf chervenak a. l., vellanki v. & kurmas z. (1998) 'protecting file systems: a survey of backup techniques' proc. joint nasa and ieee mass storage conference, march 1998, at http://www.storageconference.us/1998/papers/a1-2-cherve.pdf clarke r. (1995) 'a normative regulatory framework for computer matching' j. of computer & info.l. 13,3 (june 1995), preprint at http://www.rogerclarke.com/dv/matchframe.html clarke r. (2008) 'b2c distrust factors in the prosumer era' invited keynote, proc. collecter iberoamerica, madrid, 25-28 june 2008, pp. 1-12, at http://www.rogerclarke.com/ec/collecter08.html clarke r. (2009) 'categories of malware' xamax consultancy pty ltd, september 2009, at http://www.rogerclarke.com/ii/malcat-0909.html clarke r. (2011) 'the cloudy future of consumer computing' proc. 24th bled econference, june 2011, preprint at http://www.rogerclarke.com/ec/ccc.html australasian journal of information systems clarke 2016, vol 20, research article practical backup for small organisations & individuals 23 clarke r. (2013) 'data risks in the cloud' journal of theoretical and applied electronic commerce research (jtaer) 8, 3 (december 2013) 59-73, preprint at http://www.rogerclarke.com/ii/drc.html clarke r. (2015) 'the prospects of easier security for smes and consumers' computer law & security review 31, 4 (august 2015) 538-552, preprint at http://www.rogerclarke.com/ec/ssacs.html clarke r. (2016) 'backup and the cloud: survival strategies for users dependent on service-providers' xamax consultancy pty ltd, february 2016, at http://www.rogerclarke.com/ec/pbar-sp.html cole e. (2013) 'personal backup and recovery' sans institute, september 2013, at http://www.securingthehuman.org/newsletters/ouch/issues/ouch-201309_en.pdf english l.p. (2006) 'to a high iq! information content quality: assessing the quality of the information product' idq newsletter 2, 3, july 2006, at http://iaidq.org/publications/doc2/english-2006-07.shtml firesmith d. (2004) 'specifying reusable security requirements' journal of object technology 3, 1 (jan-feb 2004) 61-75, at http://www.jot.fm/issues/issue_2004_01/column6 gallagher m.j. (2002) 'centralized backups' sans institute, july 2001, at http://www.sans.org/reading-room/whitepapers/backup/centralized-backups-513 de guise p. (2008) 'enterprise systems backup and recovery: a corporate insurance policy' auerbach, 2008 hevner a.r. (2007) 'a three cycle view of design science research' scandinavian journal of information systems, 2007, 19(2):87-92 hevner a.r., march s.t. & park, j. (2004) 'design research in information systems research' mis quarterly, 28, 1 (2004), 75–105 huh y.u., keller f.r., redman t.c. & watkins a.r. (1990) 'data quality' information and software technology 32, 8 (1990) 559-565 iasme (2013) 'information assurance for small and medium sized enterprises' iasme standard v. 2.3, march 2013, at https://www.iasme.co.uk/images/docs/iasme%20standard%202.3.pdfhttps://www.i asme.co.uk/images/docs/iasme%20standard%202.3.pdf ietf (2007) 'internet security glossary' internet engineering task force, rfc 4949, version 2, august 2007, at https://tools.ietf.org/html/rfc4949 iso 27005 (2012) 'information technology security techniques information security risk management' international standards organisation, 2012 lennon s. (2001) 'backup rotations – a final defense' sans institute, august 2001, at http://www.sans.org/reading-room/whitepapers/sysadmin/backup-rotations-finaldefense-305 mendelson e. & muchmore m. (2013) 'the best backup software' pcmag australia, 28 march 2013, at http://au.pcmag.com/backup-products/9607/feature/the-bestbackup-software mitnick k.d. & simon w.l. (2003) 'the art of deception: controlling the human element of security' wiley, 2003 muchmore m. (2013) 'disaster-proof your data with online backup' pcmag australia, 30 march 2013, at http://au.pcmag.com/backup-products-1/9603/feature/disaster-proofyour-data-with-online-backup australasian journal of information systems clarke 2016, vol 20, research article practical backup for small organisations & individuals 24 müller h. & freytag j.-c. (2003) 'problems, methods and challenges in comprehensive data cleansing' technical report hub-ib-164, humboldt-universität zu berlin, institut für informatik, 2003, at http://www.informatik.unijena.de/dbis/lehre/ss2005/sem_dwh/lit/mufr03.pdf nist (2012) 'guide for conducting risk assessments' national institute of standards and technology, special publication sp 800-30 rev. 1, september 2012, at http://csrc.nist.gov/publications/nistpubs/800-30-rev1/sp800_30_r1.pdf oecd (1980) 'guidelines on the protection of privacy and transborder flows of personal data' oecd, paris, 1980 oecd (2002) 'oecd guidelines for the security of information systems and networks: towards a culture of security' organisation for economic co-operation and development, july 2002, at http://www.oecd.org/dataoecd/16/22/15582260.pdf parker d.b. (1998) 'fighting computer crime' john wiley & sons, 1998 peffers k., tuunanen t., rothenberger m.a. & chatterjee s. (2007) 'a design science research methodology for information systems research' journal of management information systems 24, 3 (winter 2007–8) 45–77 van der pijl g. (1994) 'measuring the strategic dimensions of the quality of information' journal of strategic information systems 3, 3 (1994) 179-190 piprani b. & ernst d. (2008) 'a model for data quality assessment' proc. otm workshops (5333) 2008, pp 750-759 preece j. (2016) ‘mac backup software review’ top ten reviews, 2016, at http://macbackup-software-review.toptenreviews.com/ preston w.c. (2007) 'backup & recovery' o'reilly media, 2007 slay j. & koronios a. (2006) ‘information technology security & risk management’ wiley, 3rd ed., 2006 saltzer j. & schroeder m. (1975) 'the protection of information in computer systems' proc. ieee 63, 9 (1975), pp. 1278-1308 shedden p., scheepers r., smith w. & ahmad a. (2011) ‘incorporating a knowledge perspective into security risk assessments’ vine 41,2 (2011) 152 strom s. (2010) 'online backup: worth the risk?' sans institute, may 2010, at http://www.sans.org/reading-room/whitepapers/backup/online-backup-worth-risk33363 tapscott d. & williams a.d. (2006) 'wikinomics: how mass collaboration changes everything' portfolio, 2006 tob (2012) 'types of backup' typesofbackup.com, june 2012, at typesofbackup.com toffler a. (1970) 'future shock' pan, 1970 toffler a. (1980) 'the third wave' pan, 1980 wang r.y. & strong d.m. (1996) 'beyond accuracy: what data quality means to data consumers' journal of management information systems 12, 4 (spring, 1996) 5-33 australasian journal of information systems clarke 2016, vol 20, research article practical backup for small organisations & individuals 25 supplementary materials app. 1: the conventional security model http://www.rogerclarke.com/ec/pbar.html#app1 app. 2: backup characteristics http://www.rogerclarke.com/ec/pbar.html#app2 app. 3: backup procedures http://www.rogerclarke.com/ec/pbar.html#app3 app. 4: how backup mechanisms address threat-vulnerability combinations http://www.rogerclarke.com/ec/pbar.html#app4 acknowledgements the assistance of russell clarke is gratefully acknowledged, in relation to conception, detailed design and implementation of backup and recovery arrangements for the author's business and personal needs, and for review of a draft of this paper. the comments of a reviewer and the section editor were of material assistance in clarifying aspects of the analysis and the presentation copyright: © 2016 clarke. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems burmeister, ritchie, devitt, chia, dresser & roberts 2019, vol 23, research on health information systems telehealth and user perceptions 1 the impact of telehealth technology on user perception of wellbeing and social functioning, and the implications for service providers oliver burmeister school of computing and mathematics charles sturt university oburmeister@csu.edu.au david ritchie health services management charles sturt university alison devitt telehealth registered nurse livebetter eevon chia research and evaluation livebetter gregory dresser research and evaluation livebetter russell roberts leadership and management charles sturt university abstract the aim of the project was to evaluate the use of telehealth equipment in the homes of older community-dwelling people, and to review its social and economic impact. a mixed methods approach was adopted, involving interviews, observation and depression anxiety stress scales. overall, the greatest benefit was apparent in those participants with a low familiarity with technology and low digital literacy, where changes in behaviours to prevent an exacerbation of their condition was possible. the user interface design reduced concern about using the technology. changes achieved were through better compliance with medication and associated understanding of the impact on their vital signs and hence daily activities. this represented an improved health literacy and the economic benefits appear to be linked to that. less benefit was observed by those who had been self-monitoring previously. a greater focus on specific conditions and improved self-management could strengthen the evidence for targeted economic benefits. keywords older people, disability, rural, regional, elderly. 1 introduction telehealth can reduce socio-economic costs by reducing travelling for face-to-face visits. the social and financial costs, as well as the associated detrimental effects on well-being for people in rural and regional areas, have been well documented in information systems (is) literature, australasian journal of information systems burmeister, ritchie, devitt, chia, dresser & roberts 2019, vol 23, research on health information systems telehealth and user perceptions 2 with various technological solutions explored, not only telehealth (boonstra & van offenbeek, 2010; burmeister, islam, dayhew, & crichton, 2015; carlson, farrelly, frazer, & borthwick, 2015; scott, richards, & adhikari, 2015). technology can be successfully applied in community-based patient care to enhance the capacity of nurses to deliver medications management for clients, resulting in increased efficiency of service delivery as well as staff and patient satisfaction (georgantzi & gheno, 2012; towers & tyler, 2014). this article reports on a study of an installation of tunstall telehealth monitoring equipment, with a small selection of livebetter services (formerly carewest; hereafter called livebetter) clients in the rural and regional area about orange, nsw. the study was undertaken to evaluate the use of telehealth equipment in the homes of older community-dwelling people, and to review its social and economic impact. it contributes to our understanding of the particular access needs of telehealth users for these regional and rural community-dwelling older people; following the literature that access to telehealth is a basic human right (bradford, caffery, & smith, 2016; morgan, 2008). with the technical dimensions of the information technology residing in the “background”, the “foreground” focus was placed on the social perspectives associated with the use of this technology. the article begins with describing the context of livebetter and the telehealth equipment utilised by their clients. it then describes the methodology and findings, before discussing the results. the conclusion looks at the model features across the domains of accessibility, availability, acceptability, affordability, and adequacy and compares this new program with an established telehealth program (saurman, lyle, kirby, & roberts, 2014a) across these domains. 2 livebetter services context livebetter has offices in 25 locations across new south wales including bathurst, bourke, broken hill, dubbo, griffith, mudgee, orange and wagga wagga. a not-for-profit organisation, livebetter offers support and advice to individuals, families, and communities across a broad range of services. livebetter’s 2015-2020 strategic plan clearly identifies its purpose as ‘enabling people in regional australia to live their best lives’, and a key goal within this mission is to ensure that ‘citizens in regional and rural communities have access to the services they require’. a significant issue for the organisation in driving this agenda is managing the large distances involved. the organisation’s footprint covers roughly 70% of the state of new south wales, a similar size to kenya, and larger than either sweden, germany, or italy. one of livebetter’s strategies in managing both the large distances and sparse population densities is to turn to technologies that will allow patients and practitioners to access each other, exchange information, and to develop appropriate care plans. information systems which support these efforts have been shown to improve not only the health outcomes for patients, but the financial and time commitments made by individuals, their families, and the wider community (boonstra & van offenbeek, 2010; burmeister et al., 2015). 2.1 this study in the context of other telehealth studies overseas studies involving telehealth and related technologies have been shown efficacious in various ways (burmeister, 2016; jenkins & draper, 2015; niemeijer, depla, frederiks, francke, & hertogh, 2014; pakrasi, burmeister, mccallum, coppola, & loeb, 2015; ritchie, 1997; teipel et al., 2016). similarly, within australia, telehealth and other interventions for australasian journal of information systems burmeister, ritchie, devitt, chia, dresser & roberts 2019, vol 23, research on health information systems telehealth and user perceptions 3 community-dwelling older people and other age groups have been shown to be effective in supplementing delivery of health care, with positive impacts on patients’ care and clinical staff (bernoth et al., 2016; burmeister, bernoth, dietsch, & cleary, 2016; burmeister & marks, 2016; hungerford & fox, 2014; soar, capamagian, denaro, prentice, & skinner-smith, 2015; warburton, cowan, & bathgate, 2013). figure 1: telehealth sites and livebetter offices are located within western and far western nsw. remote healthcare can supplement the delivery of care, providing support to clients and healthcare staff. the use of telehealth is well-established in western nsw (figure 1). for instance, the mental health emergency care (mhec) program has been operating since early 2008, recording over 55,000 clinical contacts in the period between 2008 and 2013 (saurman, lyle, kirby, & roberts, 2014b). serving a population of over 300,000, dedicated telehealth infrastructure is located at hospitals and health services in centres across 445,000 square kilometres of western nsw (saurman, lyle, perkins, & roberts, 2014). in 2011, 46 of 48 rural emergency departments (ed) in western nsw used the mhec program (saurman et al., 2011). this program has been found to be effective in reducing unnecessary transport and unnecessary admission to hospital (saurman et al., 2011). since its establishment, care protocols have become well-established and clinical outcomes are also well-researched (saurman, kirby, & lyle, 2015). in addition to direct care to patients and families, the mhec program provides support to general practitioners (gp), ed staff and other generalist government and non-government health staff in rural and remote areas. it has high patient and clinician acceptability and satisfaction levels, and is now viewed as a routine part of clinical care (saurman et al., 2015; saurman et al., 2011). tunstall telehealth monitoring equipment was installed for at least 2 months in the homes of clients who consented to participate in the study. the monitoring equipment assessed a core set of measurements (such as blood pressure, heart rate, and weight) and obtained custom measurements depending on each client’s health condition e.g. heart failure, chronic obstructive pulmonary disorder, hypertension and diabetes. the clinical/triage team from australasian journal of information systems burmeister, ritchie, devitt, chia, dresser & roberts 2019, vol 23, research on health information systems telehealth and user perceptions 4 livebetter ascertained the custom measurements for each client. clients who consented to installation of the telehealth monitoring equipment were further given the choice to participate in the research. equipment description model brand manufacture location myclinic@home touch screen device ep121 asus korea digital blood pressure monitor ua-767 plus bt a&d medical japan omron bluetooth blood pressure monitor bp792it omron kyoto, japan uc-321pbt precision health scale 9030600 and japan glucometer accu-chek performa ce0088 roche badan-wurttemberg, germany glucometer myglucohealthwireless meter mghbt1 myglucohealth usa onyx® ii model 9560 finger pulse oximeter 9560 nonin plymouth, united states of america ecg remosekg 100bt vitaphone chemnitz, germany forehead thermometer td-1261 taidoc taipei county 248, taiwan lung monitor with bluetooth 40750 vitalograph ennis co clare, ireland table 1: tunstall telehealth equipment utilised during the livebetter study tunstall ‘data sheets’ from www.tunstallhealthcare.com.au provide the following information about this equipment. for instance, in table 1 the myclinic@home is an individual user, wireless touch screen communication device designed to manage health interviews for up to 4 family members in a home environment. a range of bluetooth-enabled vital sign peripherals interface with the system providing, for example, video conferencing capability. this system uses mobile or an internet connection to securely transmit data to the integrated care platform triage manager. the two types of blood pressure monitors facilitate daily blood pressure readings. the pulse oximeter, which activates automatically when slipped onto a finger, measures blood oxygen saturation and pulse rate. the weight scales (uc-321pbt precision health scale) are specifically designed for integration with the telehealth is. the thermometer measures the patient’s temperature, using infrared heat scanning skin over the user’s forehead. the two glucometers enabled participants with diabetes to monitor and manage their blood glucose levels using very small blood samples. 3 methodology 3.1 research aims the main research question addressed in this study was: what is the impact of the tunstall telehealth systems on user perception of well-being and social functioning? closely associated australasian journal of information systems burmeister, ritchie, devitt, chia, dresser & roberts 2019, vol 23, research on health information systems telehealth and user perceptions 5 with that was a secondary research question: what is the economic impact of the use of these systems? these questions were answered in a mixed methods approach that focussed on an interpretivist, constructivist methodology. user perceptions were captured through interviews, and through the observations of the telehealth nurse who worked with each client during the study. user perceptions were also evaluated through the use of a national standard instrument, the depression anxiety stress scales (dass), which is based on user perceptions about their well-being in the immediate past seven days prior to the administration of the dass. the short-form version was used, which assesses the emotional and mental well-being of each participant, through 21 items to measure subscales for depression, anxiety and stress. the purpose of the study of the tunstall telehealth systems was to determine the social and economic values of this program to older community-dwelling australians in the orange region. this study focussed on the situation before the installation of the equipment and subsequent to the removal of the equipment. for this reason it took place over two phases. approval to conduct the study was granted by the university’s human research ethics committee. data collected using semi-structured interviews during phase 1 allowed the researchers to understand the needs, attitudes, and skills of the clients using these systems. each interview began with the administration of the dass. the focus of the phase 2 interviews was to find out how well the telehealth program was helping clients meet the needs identified during phase 1. interviews in phase 2 were conducted immediately prior to the removal of the telehealth equipment. as with phase 1 interviews, each interview began with the administration of the dass. the interviews were conducted face-to-face and followed a semi-structured interviewing format utilising open-end questions and further probing questions. the semi-structured, indepth interview format was selected to allow rich accounts of participants’ experiences to be obtained. the interviews were conducted from september 2015 to july 2016 in orange and the surrounding region. all the interviews were recorded and transcribed verbatim. this study of telehealth monitoring was not limited to technological monitoring alone. in addition to the technological monitoring, livebetter employed a nurse clinician to be a telehealth nurse during the study, whose responsibility it was to assist participants to use the equipment and who had regular face-to-face and telephone contact with participants during the study. she recorded her observations with clients during the study. 3.2 recruitment livebetter identified and recruited clients by convenience sampling. there were 18 participants for phase 1 and 11 for phase 2. pre and post data reported here only relates to 11 of the 18 participants. three couples, where both parties in the couple participated, chose to not always do separate interviews. for instance, one woman was not well enough to be interviewed separately for phase 2, but she was present at her husband’s post-interview and contributed to that discussion. hence we consider we have data from 11 preand postinterviews, even though there are only 10 post-interview transcripts. of the seven preparticipants who did not participate in the post-evaluation, one died and his wife chose to discontinue in the study, one was a couple and when the male entered a care facility both dropped out of the study, and three others gave no reason for discontinuing. finally, it should australasian journal of information systems burmeister, ritchie, devitt, chia, dresser & roberts 2019, vol 23, research on health information systems telehealth and user perceptions 6 be noted that two participants had their spouses present during their interviews, but those spouses did not participate in the study themselves. for instance, in one case the person being interviewed was also the carer for her husband, thus requiring him nearby, to see to his care, if necessary. participant demographics were not a specific matter of data collection. however, 4 couples participated in which both members of the couple were also part of the study. 10 females and 8 males participated. ages ranged from the low 60s to the low 80s, with the exception of one 51-year-old. multicultural data was also not specifically captured, but participants included one indigenous person, two from the middle east, descendants from various cultures, but with all participants having been long term residents of rural and regional nsw. at the time of the study the majority lived in orange, a few in the immediate region near orange and the furthest participant lived 120 km outside orange. 3.3 analysis the transcribed interviews were analysed using thematic analysis with qsr nvivo 10, a software package for managing data. the analysis of all interviews was carried out by the team leader. early during the analysis process another member of the team independently analysed four interview transcripts, and near the conclusion of the phase 2 analysis that same member independently analysed another 3 interview transcripts. the latter was for inter-rater reliability, with the two team members discussing all discrepancies in analysis, to ensure the reliability of interpretation of the results. as regards the dass analysis, in addition to comparing phase 1 and 2 results to each other, the results were also compared to the australian norms (crawford, cayley, lovibond, wilson, & hartley, 2011). 4 findings three themes emerged from the analysis of the interviews. they were service delivery, social impact, and technology. the themes and categories are summarised in table 2. exemplary quotations addressing the research questions are given below, to reveal participant perceptions about this telehealth study. theme category service delivery self-monitoring health support assisted monitoring economic impact social impact health status reassurance socialising in the home socialising out of the home treatment australasian journal of information systems burmeister, ritchie, devitt, chia, dresser & roberts 2019, vol 23, research on health information systems telehealth and user perceptions 7 technology prior use passive attitude post use equipment hardware interface table 2: themes and the categories within them arising from the livebetter study 4.1 theme 1: service delivery categories within service delivery included self-monitoring, health support, assisted monitoring, and economic impact. most participants valued the ability to have independence and control, but not all. hence whilst there are many exemplars of self-monitoring, assisted monitoring was also needed. it good service because daily, ten o’clock in the morning, eleven o’clock between ten and eleven i got a chance to check my data because sometime it is different. (male) when i first had the equipment i thought, oh dear i’m not that ancient that i need that much help. even though i’d had heart failure, and i thought well it won’t be much use to me. but in the last few months i’ve found it's been excellent, because i’ve been able to monitor a bit more and (nurse clinician) said “if you can, take your blood pressure twice a day.” but i’ve been taking it three times now, if i didn’t have that equipment i wouldn’t be able to do that, and they wouldn’t be able to compare whether my blood pressure was dropping, or whether it was too high. so i’ve found that very good. and i’ve sort of changed my opinion about the whole thing, because i think it could be really very, very helpful as personally it gives you a bit more security. (female) several participants complained about the difficulties and costs of changing batteries. one lady, who bought groceries weekly, had to forego groceries one week, because after paying rent, utilities and the green slip for her car, she only had 5 cents left. clearly for someone in such a position, even the cost of replacement batteries, though small, is economically hard to manage. 4.2 theme 2: social impact categories within social impact included health status, reassurance, socialising in the home, socialising out of the home, and treatment. the health status of participants and/or their spouses impacted the ability to engage in social activities in and out of their residence. health status was distinguished from treatment in that treatment was the reason for the telehealth monitoring, but in addition to that, e.g. diabetes, the person might also have other health conditions, such as impaired mobility that required the use of a walking frame, which might also affect their ability to socialise or necessitate medical appointments. reassurance is different from self-monitoring within the theme of service delivery. selfmonitoring gave participants the ability to feel they had a measure of control over their australasian journal of information systems burmeister, ritchie, devitt, chia, dresser & roberts 2019, vol 23, research on health information systems telehealth and user perceptions 8 condition. reassurance is about the peace of mind that comes with knowing that when things go wrong, they can receive the help they require. for instance, one participant who died during the study had experienced a fall in his kitchen prior to the study. his wife was away and it was many hours before he could drag himself to the phone to ring for help. even though his perception of telehealth monitoring providing alarm monitoring was incorrect, it still provided reassurance that if something untoward happened again, the telehealth monitoring might speed assistance getting to him. other participants were indeed reassured and benefitted from the monitoring of their measurements to gain interventions before they might have become aware of the need for assistance. examples of socialising in the home and out of it follow. noticeably for many participants, social activities revolve around medical support activities, and/or care package services, such as cleaning, cooking and mowing. first, in the home, including around the home, such as in the garden or on the farm. (name) the heart failure clinic nurse comes around once a week. (male) neighbours come down at least twice a day to see how i’m going and then if i’m doing something they’ll give me a hand to do it and all this type of business. so i go and help them too which i think that’s part of life. (male) well sometimes i go in the ute with him around the farm. i drive and he looks and opens the gate. and i prefer that, to be with him. oh, we do it every day, every second day. (female) a couple of days a week we have some friends up the road that pop in. my daughter or my son in law at least every second day. (female) next, socialising out of the home: i’m a member of, like i said, an important men’s group. that’s – the idea of that is to get out of house. i have to go the doctors and go to mobility school which gets me out. the biggest problem with me was not being able to drive, because once you have a – your defib goes off you’re not allowed to drive for six months. and mines gone off that many times i’ll probably never get a licence back. (male) i don’t have much in the way of friends or social activities, or anything like that. i feel too bloody useless like i can't do anything because i can't walk properly, i can't use my hands the way i used to because they shake all the time. i want to try and do some walking up and down in the pool, but that's a bit expensive. (male) well we leave the house virtually every day for medical matters. socially maybe once or twice a week for church. (male) the last was from a man who went into a care facility during the study period and therefore did not participate in the post-interview. some participants were very immobile. one man had started dental treatment immediately prior to a fall, and given that dentists do not do home visits, was unable to complete his dental treatment. another example from a phase 1 interview in september 2015 was that of a man who had only left his house once in 12 months. australasian journal of information systems burmeister, ritchie, devitt, chia, dresser & roberts 2019, vol 23, research on health information systems telehealth and user perceptions 9 4.3 theme 3: technology categories within technology included prior use, passive, attitude, post use, equipment, hardware, and interface. generally the interface was easy to use, even for participants with little prior use of telehealth systems. attitudes towards technology use varied, but most participants welcomed the telehealth equipment and were willing to put in effort to learn to use it properly. a minority preferred a passive, minimalist approach, which did not involve much input from them. exemplary quotations of participant views in this theme include the following. it said “please take your ecg reading now”, you take it and you press the button, and then you go onto the next one, and you finish the four. and then when that’s all finished you just press the finish button, and then you just watch for the – i always watch for the signal little round piece – red piece that goes round and round to make sure it's gone to brisbane. (female) there’s the blood pressure, then weight, and then ask me about salt, and how i feel today. but see it’s only got better, worse, or something on the thing. well it doesn’t have in-between, so every time i press it i’ve got to press – same, same is the word. i just put the same because some days i feel in between it and i can’t put worse down, but i could say not as well or something like that. (female) that’s the ecg … it will ring and you just hold it on your chest, and it does its business. it’s pretty simple. (male) the rough summary of that is the blood pressure one has worked probably ninety-nine percent of the time, at least all the time. the thermometer one has hardly worked at all and the little one over the finger (pulse oximeter). it has been very poor; it hasn’t probably worked for the last month. (male) very good, because i don’t know – i would never know me blood pressure was up without it, because you can’t go up the doctor every day. i hadn’t been up the street for about probably three years. so i find that very handy – very good. (female) as stated above, each phase 1 and 2 interview was preceded by the administration of the dass, and those findings are presented next. that is followed by the findings from the observations of the telehealth nurse. 4.4 depression anxiety stress scales the depression anxiety stress scales (dass) have been an accepted measure of populations similar to those in this study for over two decades, including in rural settings (ciobanu, brodard, antonietti, genoud, & brandner, 2018; crawford et al., 2011; zhang et al., 2018). the dass was administered with each interview (figure 2). the depression subscale was observed to decrease from 9.56 ± 12.01 pre-telehealth to 6.33 ± 4.53 post-telehealth. the anxiety subscale decreased from 9.56 ± 5.75 pre-telehealth to 6.78 ± 4.66 post-telehealth. the stress subscale decreased from 9.22 ± 6.92 pre-telehealth to 6.11 ± 5.13 post-telehealth. the changes observed in the subscales were not significant. the participants of this study scored higher in the depression and anxiety subscales preand post-telehealth compared with the australian general adult population, which had a score of 5.02 ± 7.54 and 3.36 ± 5.07 for depression and anxiety, respectively (crawford et al., 2011). for the stress subscale, the australian general adult population scored 8.10 ± 8.40. compared to australasian journal of information systems burmeister, ritchie, devitt, chia, dresser & roberts 2019, vol 23, research on health information systems telehealth and user perceptions 10 the australian general adult population, participants of this study scored higher in the stress subscale before the telehealth intervention and lower after the telehealth intervention. figure 2. preand post-telehealth dass scores. 4.5 nurse clinician observations four main themes that emerged from the nurse clinician’s observations: chronic disease management; social connectedness; reduction in anxiety; technical difficulties and opportunities. firstly, the nurse clinician observed a favourable effect of telehealth monitoring on chronic disease management, likely associated with telehealth monitoring supporting selfmanagement of symptoms associated with chronic diseases. telehealth monitoring of vital signs combined with phone support and education from the nurse, improved participants’ ability to recognise and appropriately manage symptoms and medical exacerbations. this was also associated with better medication compliance, resulting in improved vital signs during the study and an observed participant self-reported reduction in unplanned hospitalisations. these benefits were most noticeable in participants with multiple chronic conditions and/or higher severity of disease state. the participants expressed to the nurse clinician that they valued their partnership with the nurse, who helped set health goals and provided advice for health-related decisions. most importantly, this relationship provided access to a trusted nurse clinician. the participants would initiate conversations with the nurse clinician to clarify health information they had received from medical specialists and seek help to solve health-related issues. one client with heart failure stated that “the telehealth nurse has become one of the pillars in my health along with my general practitioner (gp) and my cardiac nurse”. telehealth played an important role in chronic disease management by facilitating interdisciplinary care. the nurse clinician shared information with and collaborated with gps, nurse practitioners, registered nurses, pharmacists and home-care workers. data obtained from telehealth monitoring influenced medical management decisions related to medications and identified the need for further investigations. in one case, data from the ecg telehealth peripheral contributed to investigations which led to a more invasive corrective procedure, significantly improving the participant’s quality of life. secondly, telehealth monitoring also provided social connectivity, with 46% of the participants living alone. the nurse clinician called all the participants at least once a fortnight, including participants with stable vital signs and well-managed conditions. participants expressed their 0 5 10 15 20 25 30 35 40 depression anxiety stress pre post australasian journal of information systems burmeister, ritchie, devitt, chia, dresser & roberts 2019, vol 23, research on health information systems telehealth and user perceptions 11 appreciation of these conversations to the nurse clinician. these phone calls provided social connection and reassurance of remote monitoring for older participants. thirdly, at the start of the telehealth monitoring study, the nurse clinician noticed that anxiety impacted the quality of life of most participants with chronic and complex conditions. the presence of multiple chronic conditions seemingly increased the participants’ likelihood of psychological distress. as the study progressed, the nurse clinician observed a reduction in anxiety-related symptoms. this may be related to the participants’ ability to monitor their vital signs, enabling them to self-assess their own physical state and thus decreasing anxiety associated with fear of physical deterioration or a health emergency. it may also relate to improved disease self-management and reduction in symptoms. however, ongoing reassurance from the nurse clinician was the most obvious influence on reducing anxiety. finally, the fourth theme that emerged from the nurse clinician’s observations, revealed that there was diversity in the ability of the participants to use the telehealth equipment, with some requiring more technical support. age did not seem to be a factor in determining ability, whilst previous technological experience may be a positive factor. for example, an 81-year-old participant with tablet and email experience found the telehealth technology intuitive. in comparison, another participant of similar age felt overwhelmed by the technology and withdrew participation during installation of the equipment. the nurse clinician seemingly provided the most technical phone support during the first three weeks after installation. after this period, the participants felt more confident using the technology. they were also better able to troubleshoot technical difficulties independently or in consultation with it support who could access their telehealth computer remotely to fix system issues. conditions associated with older age, including osteoarthritis and poor vision, seemingly restricted some of the participants’ ability to independently replace the telehealth equipment batteries. this impacted on the role of the nurse clinician, who was at times required to attend the participant’s residence in order to replace the batteries. the nurse clinician suggested that using usb or electricity charged equipment may provide a solution to this issue. 5 discussion of results having presented the findings, this section relates them to the research questions this study sought to answer, and compares this new program with an established telehealth programs, such as that of saurman, lyle, et al. (2014a). the study sought to understand the impact of the tunstall telehealth systems on user perception of well-being and social functioning, as well as economic impacts. this is somewhat different from many of the telehealth studies that precede this one, which are frequently focused on the tyranny of distance in rural and remote areas, or on cost savings, or on other factors, such as the effectiveness of accessing services (bradford et al., 2016; kruse et al., 2017; saurman et al., 2015; saurman, lyle, perkins, et al., 2014). 5.1 perception of technology in the initial interviews, several participants reported playing electronic games, using consoles such as xbox or playstation, that younger family members might use, or other electronic devices such as an ipad, but did not regard these as using technology in the same way they viewed the telehealth monitoring equipment. such disconnect between different types of technologies, and not relating learning about one type of technology to another, has also been australasian journal of information systems burmeister, ritchie, devitt, chia, dresser & roberts 2019, vol 23, research on health information systems telehealth and user perceptions 12 seen in other studies involving people in this age group (morris et al., 2014; pakrasi et al., 2015). one participant would join in games to interact with younger family members, but was not really good at it or necessarily interested in those types of action-oriented games. they reported not having clear expectations of the telehealth monitoring equipment because it was unfamiliar. other participants, using electronic devices such as an ipad, were interested in hand-eye coordination and would play for that purpose, but were then drawn into successful play and advancement into higher levels of complexity. they would return to the device with that motivation as a distraction from other health related concerns, yet would still discount their understanding of technology. entertainment was separated from monitoring health in regard to the importance of the technology. the commencing level of health literacy of participants understanding the relationship between measurement of vital signs and the interaction required with the equipment was something that changed substantially. confusion about how to work with technology was expressed by some participants as being uncertain or disinterested in knowing how to use equipment such as a personal content recorder or player. a television could be switched on or off, and channels changed, but several had not bothered to become familiar with video players when tapes were used, let alone digital entertainment. many kitchens had micro-wave ovens, washing machines that had programmed settings, but these were regarded as household appliances and were not recognised as being technology, a finding supported in the literature (kurniawan, 2008; pakrasi et al., 2015). thus there were frequent doubts expressed about the complexity of the proposed telehealth monitoring equipment, because participants did not see how learning about technology use in other areas would benefit them in learning about the use of the telehealth systems installed in their homes. a number of participants also had mobile phones, but these were used for basic communications, such as text or voice calls, and were not smart phones with more advanced features. some participants, who had ipads, could also communicate with family members using skype or facetime, having been assisted with the setup of those programs by family members for easy use. the different reactions in part to technology depended on who had provided assistance in encouraging the use of the technology in the first instance, and the purpose it was used for. communicating with family, and then friends, meant that a video call was not much of a technological leap from making a mobile phone call, or from using a landline. when a family member had programmed the calling numbers and all that was required was a few simple steps, easily remembered or written down. how participants had been moved from what they were familiar with to more advanced forms of technology seemed to influence perception and initial apprehension. 5.2 adapting to self-managed health in the initial interviews, several participants reported a relatively low level of awareness about self-management or interpretation of basic measures such as blood pressure, pulse rate, and blood oxygen saturation. when feeling unwell participants would consult a health professional. understanding of medication, and its impact, was at the lower end of health literacy. this changed with the follow-up interviews. several participants had been recording their telehealth measures – with a medium they were comfortable with, i.e. pen and paper – and had become comfortable in linking their measures to how they felt. despite expressing their initial ambivalence with telehealth equipment during the pre-interview, most were converts and wished to keep the equipment in the post-interview. exceptions were participants with diabetes who were self-monitoring blood sugar levels using their own australasian journal of information systems burmeister, ritchie, devitt, chia, dresser & roberts 2019, vol 23, research on health information systems telehealth and user perceptions 13 instruments even before this study. they reported little change in confidence in using technology as a consequence of the use of the telehealth intervention. 5.3 economic impacts the secondary research question sought clarity around the economic impacts of the telehealth study on participants. a number of participants who expressed some initial scepticism about the telehealth equipment, were very accepting of the equipment by the follow-up interview. this is in line with other studies that have introduced information systems to older people (burmeister, 2010, 2012; kurniawan, 2008; morris et al., 2014; vichitvanichphong, talaeikhoei, kerr, ghapanchi, & scott-parker, 2016). there seemed to be relatively few technical problems with reliability, but some follow-up reminders to measure and report according to the agreed schedule had occurred. the development of health literacy about the relationship between the vital sign measurements, medication management, and perceptions of wellbeing, meant that it was possible that less expensive equipment, and less frequent reporting, might enable better self-management after the study had concluded. there seemed to be a development of both confidence and pride in the self-awareness and management of their conditions. a perceptible increase in the social interaction and positiveness on the follow-up interview was acknowledged by the interviewers. it seems possible that less expensive personal monitoring equipment, such as fitness or activity trackers, could be used in conjunction with digital scales and automatic blood pressure monitors. indicative prices for purchase of omron blood pressure upper arm monitors range from $99.95-249.00, digital scales soehnle pino digital personal scale $28-35, fitness trackers e.g. fitbit charge $169-230. 5.4 preventable admissions/reduced emergent medical and health consultations several participants reported a stabilisation of their vital signs, such as blood pressure, with improved compliance with medications and the ability to interpret “good” from “poor” health days. when blood pressure and/or weight was up adjustments to activity ensued, with increased activity and socialisation on good health days, and adjusted activity levels on poor health days. participants generally reported better awareness about how they felt based on the measurements, and that meant overall they felt that they were doing more. participants reported taking ad hoc measurements, outside of designated reporting times, just to “check” how they were measuring up against their perceptions. this suggests an improved health literacy and an ability to self-monitor and better manage their daily living. participants were more likely to discuss trends and/or examples of good/poor days and what they felt were different about them. medications had been reviewed and adjusted, and several participants indicated that ad hoc medical and health consultations were not as frequent. they generally expressed a greater degree of comfort with their ongoing management of their condition than the health professional directed management previously. to what extent preventable admissions have been avoided could not be ascertained, nor could frequency of contact with health professionals be verified without access to personal health records, but participants were more confident in reduced reliance on direct visits to monitor their health status than previously. 6 conclusion caution should be exercised in making definitive conclusions from this study based on the dass measures used. the size of the sample and nature of the participants recruited into the australasian journal of information systems burmeister, ritchie, devitt, chia, dresser & roberts 2019, vol 23, research on health information systems telehealth and user perceptions 14 study make statistical inferences problematic. what can be observed is an improvement between the measures preand post-study, and a reduction in variation in the dass subscales between measures, despite the lack of statistical significance. that could suggest some value in further investigation in a study where aiming for statistical inference was a design consideration. the reduction in anecdotal self-reported stress was a marked improvement compared to the generally higher depression and anxiety scales, and this seems to link to the reassurance that the tele-monitoring offered through greater health literacy and the empowerment associated with self-monitoring. the participants from the age demographic in this study made it clear that comfort with technology was an issue in recruitment and participation, and the efforts of the livebetter nurse clinician to address these was important. understanding the concerns of this group of participants could change how they might be introduced to the potential benefits of the monitoring and support offered by the telehealth systems. the role of the livebetter nurse clinician in supporting and assisting participants to develop their health literacy was significant and clearly remains one of the integral benefits many participants acknowledged. how future participants might be introduced to the technology, and the benefits of improved self-management, could facilitate a better fit between the health condition experienced and the life-style improvements that could result. the interface with the telehealth monitoring equipment was regarded by all participants as being very straight forward, easy to follow, and with few problems. slight changes could result in fewer equipment maintenance issues such as battery performance and replacement. the positive experience of the participants is evident in a number of ways, despite the numerical data from the dass instrument. those participants who were unstable in their selfmanagement of their health condition previously, were assisted in the development of their ability to better self-manage medications and activity. this affected their mood and socialisation, both within their house and outside it. that participants, who previously had not been interested in the use of technology, could now see benefits in improving their daily living should not be underestimated. several were now prepared to advocate for the use of telehealth equipment to others in similar circumstances who might be initially resistant. considering the use of lower cost equipment to support self-management after the study is something that might be considered. it could also enable an expansion of the use of the resources to improve health status over a longer period of time to maintain the independent living of these ageing individuals. in regards to the aims of the study, the telehealth systems introduced were successful for most of the community-dwelling older people who completed the study. less benefit was observed by those who had been self-monitoring previously. the greatest benefit was apparent in those participants where changes in behaviours to prevent an exacerbation of their condition were possible, through either better compliance with medication, or better understanding the impact of medication on their vital signs and what that meant to their daily activities. regarding the social and economic impacts, again, the overall impression from participants was an enhancement to the self-perceived quality of daily functioning. higher levels of selfreported socialisation, activity, and self-assurance seemed common. the economic benefits appear to be linked to that social improvement, but access to data held by others would be required to determine just how significant the improved self-management would be, to the range of health professionals the participants came into contact with in the lead up to the study australasian journal of information systems burmeister, ritchie, devitt, chia, dresser & roberts 2019, vol 23, research on health information systems telehealth and user perceptions 15 and beyond. the difference between frequency of personal visits and phone contact is another consideration. a greater focus on specific conditions and the ability to achieve stability in selfmanagement could strengthen the evidence for targeted economic benefits. comparing the results of this study with a mental health emergency care-rural access program (mhec-rap) (saurman, lyle, et al., 2014a) provides further insights into how technology can provide common benefits from both recipient and provider perspectives. the mhec-rap was designed to assist services provided to patients in locations where they did not have access to specialist services in a timely manner. the technology was designed to assist health professionals to extend their scope of support and to reduce higher costs being imposed on both the health care system and the individual. this purpose is somewhat different to the livebetter investigation of impact on the independence of the recipients, with the cost and impact being felt differently by a range of health service providers other than livebetter. this is where the study framework of accessibility, availability, acceptability, affordability and adequacy is useful in identifying who bears the costs and is advantaged by the benefits. accessibility, designing for improved access and reduction of barriers, applies to both but in different aspects. the tunstall telemonitoring equipment was rated by users as being easy to use and hence accessible, and the mhec-rap similarly by other health professionals. availability, the ease of access, was limited in the livebetter study primarily by the cost of the equipment to both livebetter and the recipients, while the mhec-rap had greater accessibility, because the direct costs and benefits were received by the provider, while the recipients had improvements in the timeliness and quality of care. acceptability, the meeting of needs, was apparent in both studies. affordability, was different and arguably mhec-rap had a simpler but stronger business case for the same reasons as accessibility. the livebetter arguments are more complex and would require a larger scale study and broader cooperation to become viable to more recipients. the adequacy of the technology, the fit for purpose, was common to both studies. what was apparent with the livebetter study, would be the potential to stepdown to less expensive technology once health and digital literacy had been developed to levels enabling more independent self-management. references bernoth, m., burmeister, o. k., morrison, m., islam, m. z., onslow, f., & cleary, m. (2016). the impact of a participatory care model on work satisfaction of care workers and the functionality, connectedness and mental health of community-dwelling older people. issues in mental health nursing, 1-7. doi:10.3109/01612840.2016.1149260 boonstra, a., & van offenbeek, m. (2010). towards consistent modes of e-health implementation: structurational analysis of a telecare programme's limited success. information systems journal, 20(6), 537-561. doi:10.1111/j.1365-2575.2010.00358.x bradford, n., caffery, l., & smith, a. (2016). telehealth services in rural and remote australia: a systematic review of models of care and factors influencing success and sustainability. rural & remote health, 16, 3808. burmeister, o. k. (2010). websites for seniors: cognitive accessibility. international journal of emerging technologies and society, 8(2), 99-113. burmeister, o. k. (2012). what seniors value about online community. journal of community informatics, 8(1). australasian journal of information systems burmeister, ritchie, devitt, chia, dresser & roberts 2019, vol 23, research on health information systems telehealth and user perceptions 16 burmeister, o. k. (2016). the development of assistive dementia technology that accounts for the values of those affected by its use. ethics and information technology, 18(3), 185-198. doi:10.1007/s10676-016-9404-2 burmeister, o. k., bernoth, m., dietsch, e., & cleary, m. (2016). enhancing connectedness through peer training for community-dwelling older people: a person centred approach. issues in mental health nursing, 1-6. doi:10.3109/01612840.2016.1142623 burmeister, o. k., islam, m. z., dayhew, m., & crichton, m. (2015). enhancing client welfare through better communication of private mental health data between rural service providers. australasian journal of information systems, 19, 1-14. doi:10.3127/ajis.v19i0.1206 burmeister, o. k., & marks, e. (2016). rural and remote communities, technology and mental health recovery. journal of information, communication and ethics in society, 14(2), 170-181. doi:doi:10.1108/jices-10-2015-0033 carlson, b. l., farrelly, t., frazer, r., & borthwick, f. (2015). mediating tragedy: facebook, aboriginal peoples and suicide. australasian journal of information systems, 19. doi:10.3127/ajis.v19i0.1174 ciobanu, t., brodard, f., antonietti, j., genoud, p. a., & brandner, c. (2018). screening negative affectivity in young adults: validation and psychometric evaluation of the french version of the depression anxiety stress scales. canadian journal of behavioural science / revue canadienne des sciences du comportement, 50(4), 238-247. doi:10.1037/cbs0000110 crawford, j. r., cayley, c., lovibond, p. f., wilson, p. h., & hartley, c. (2011). percentile norms and accompanying interval estimates from an australian general adult population sample for self-report mood scales (bai, bdi, crsd, ces-d, dass, dass21, stai-x, stai-y, srds, and sras). australian psychologist, 46, 3-14. georgantzi, n., & gheno, i. (2012). ict psp – accessibility, ageing and social integration programme. retrieved from bruxelles, belgium: http://www.ageplatform.eu/images/stories/draft_synthetised_dr_ab_feedback_short_version_fina l.pdf?%22%22 hungerford, c. l., & fox, c. (2014). consumer's perceptions of recovery-oriented mental health services: an australian case-study analysis. nursing & health sciences, 16(2), 209215. doi:10.1111/nhs.12088 jenkins, s., & draper, h. (2015). care, monitoring, and companionship: views on care robots from older people and their carers. international journal of social robotics, 1-11. doi:10.1007/s12369-015-0322-y kruse, c. s., krowski, n., rodriguez, b., tran, l., vela, j., & brooks, m. (2017). telehealth and patient satisfaction: a systematic review and narrative analysis. bmj open, 7(8). doi:10.1136/bmjopen-2017-016242 kurniawan, s. (2008). older people and mobile phones: a multi-method investigation. international journal of human-computer studies, 66, 889-901. morgan, m. (2008). what is the goal of access to health care? journal of legal medicine, 15, 742751. australasian journal of information systems burmeister, ritchie, devitt, chia, dresser & roberts 2019, vol 23, research on health information systems telehealth and user perceptions 17 morris, m. e., adair, b., ozanne, e., kurowski, w., miller, k. j., pearce, a. j., . . . said, c. m. (2014). smart technologies to enhance social connectedness in older people who live at home. australasian journal on ageing, 33(3), 142-152. doi:10.1111/ajag.12154 niemeijer, a. r., depla, m., frederiks, b., francke, a. l., & hertogh, c. (2014). the use of surveillance technology in residential facilities for people with dementia or intellectual disabilities: a study among nurses and support staff. american journal of nursing, 114(12), 28-37. pakrasi, s., burmeister, o. k., mccallum, t. j., coppola, j. f., & loeb, g. (2015). ethical telehealth design for users with dementia. gerontechnology, 13(4), 383-387. doi:10.4017/gt.2015.13.4.002.00 ritchie, d. (1997). managing the gap: balancing advances in technology with advances in management practice. australian health review, 20(1), 53-63. doi:http://dx.doi.org/10.1071/ah970053 saurman, e., kirby, s., & lyle, d. (2015). no longer ‘flying blind’: how access has changed emergency mental health care in rural and remote emergency departments, a qualitative study. bmc health services research, 15(1), 156. saurman, e., lyle, d., kirby, s., & roberts, r. (2014a). assessing program efficiency a time and motion study of the mental health emergency care-rural access program in nsw australia. international journal of environmental research and public health, 11(8), 76787689. saurman, e., lyle, d., kirby, s., & roberts, r. (2014b). use of a mental health emergency carerural access programme in emergency departments. journal of telemedicine & telecare, 20(6), 324-329. saurman, e., lyle, d., perkins, d., & roberts, r. (2014). the successful provision of emergency mental health care to rural and remote nsw an evaluation of the mental health emergency care-rural access program. australian health review, 38(1), 58-64. saurman, e., perkins, d., roberts, r., roberts, a., patfield, m., & lyle, d. (2011). responding to mental health emergencies: implementation of an innovative telehealth service in rural and remote new south wales, australia. journal of emergency nursing, 37(5), 453459. scott, k., richards, d., & adhikari, r. (2015). a review and comparative analysis of security risks and safety measures of mobile health apps. australasian journal of information systems, 19. doi:10.3127/ajis.v19i0.1210 soar, j., capamagian, l., denaro, c., prentice, l., & skinner-smith, r. (2015). research note: small pilot of home tele-health in a hospital heart failure outreach service. electronic journal for health informatics, 9(1). teipel, s., babiloni, c., hoey, j., kaye, j., kirste, t., & burmeister, o. k. (2016). information and communication technology solutions for outdoor navigation in dementia. alzheimer's & dementia: the journal of the alzheimer's association, 12(6), 695-707. doi:10.1016/j.jalz.2015.11.003 australasian journal of information systems burmeister, ritchie, devitt, chia, dresser & roberts 2019, vol 23, research on health information systems telehealth and user perceptions 18 towers, c., & tyler, m. (2014). the broadband-enabled innovation program: a working demonstration of the effective use of technology in community-based patient care. australian family physician, 43, 848-851. vichitvanichphong, s., talaei-khoei, a., kerr, d., ghapanchi, a. h., & scott-parker, b. (2016). good old gamers, good drivers: results from a correlational experiment among older drivers. australasian journal of information systems, 20. doi:10.3127/ajis.v20i0.1110 warburton, j., cowan, s., & bathgate, t. (2013). building social capital among rural, older australians through information and communication technologies: a review article. australasian journal on ageing, 32(1), 8-14. doi:10.1111/j.1741-6612.2012.00634.x zhang, s., dang, r., yang, n., bai, y., wang, l., abbey, c., & rozelle, s. (2018). effect of caregiver’s mental health on early childhood development across different rural communities in china. international journal of environmental research and public health, 15(11), 2341. copyright: © 2019 burmeister, ritchie, devitt, chia, dresser & roberts. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ http://creativecommons.org/licenses/by-nc/3.0/au/ microsoft word 1656-5316-2-le.docx australasian journal of information systems grottke, hacker & durst 2018, vol 22, research on recent advances in social media factors determining online social capital? 1 which factors determine our online social capital? an analysis based on structural equation modelling michael grottke gfk se and friedrich-alexander-universität erlangen-nürnberg michael.grottke@fau.de janine hacker friedrich-alexander-universität erlangen-nürnberg carolin durst friedrich-alexander-universität erlangen-nürnberg abstract the relationship between social network sites and social capital has received much research attention. however, two research gaps can be identified in the existing literature. first, only few studies have examined online social capital as a resource in online social networks. in this regard, it is not clear how to validly measure online social capital. second, while the factors influencing social capital, among them properties of an individual’s social network, have been investigated in offline settings, such factors have not yet been investigated in terms of online social capital. addressing these gaps, we asked 1000 facebook users to provide information on their facebook usage and online friendship network. employing structural equation modelling for analysing the survey data, we show that williams’ internet social capital scales, which are commonly used to assess social capital in offline settings, can be used to validly measure online social capital. moreover, we find that some of the variables influencing offline social capital, among them similarity in terms of sociodemographic attributes, seem less important in an online setting. keywords: social capital, social networks, structural modelling/structural equation modelling (sem) 1 introduction our social network influences the way we think, feel, and interact within the society (christakis & fowler, 2013). people we are connected to provide us with information and advice, they support us emotionally in times of crisis, or they lend us funds when we face temporary financial problems (rostila, 2011). the resources that we can access through our social network to obtain these benefits are referred to as social capital (lin, 2001b). our social networks carry different types of social capital. bonding social capital is closely related to thick trust (anheier & kendall, 2002), and it provides emotional support. bridging social capital, on the other hand, connects heterogeneous people across social groups and provides access to sets of non-redundant resources. the impact of properties of an individual’s social network (such as the extent to which someone interacts with people highly similar to him-/herself) on social capital has received a lot of attention in “purely” offline settings. with the rise of social network sites (sns), social capital has also become a prominent theme in the fields of computer-mediated communication and online social media research. against this background, the internet social capital scales (iscs) were developed (williams, 2006). although designed for measuring social capital in australasian journal of information systems grottke, hacker & durst 2018, vol 22, research on recent advances in social media factors determining online social capital? 2 online as well as offline contexts, the iscs have mostly been applied to investigate offline social capital. in fact, they are the predominantly-used survey tool in studies examining the influence of the use of sns on social capital in localised offline settings (burke, kraut, & marlow, 2011; burke, marlow, & lento, 2010; ellison, steinfield, & lampe, 2007; steinfield, ellison, & lampe, 2008; yoder & stutzman, 2011), e.g., campus life. hence, it remains unclear whether williams’ iscs can actually be applied for validly measuring online social capital. furthermore, a gap exists regarding the antecedents of online social capital. it is not clear how different attributes of our online friendship network, e.g., on facebook, relate to the formation of bonding and bridging social capital in this online setting. while the connections may mirror the ones of our offline network, sns enable different and more varied means of communication, e.g., for maintaining weaker connections with acquaintances. moreover, we are confronted with various information about our friends’ friends, which may lead to the conversion of “latent” connections into explicit linkages (e.g., ellison, steinfield, & lampe, 2011). besides testing the validity of williams’ iscs in terms of assessing online social capital, our article hence focuses on the following research question: what are the factors that influence online social capital? to answer our research question, a sample of 1000 facebook users was retrieved from the online panel of the gesellschaft für konsumforschung (gfk, germany’s largest market research institute). in an online survey, we collected information on the survey participants’ facebook usage, network structure, network ties, and network friends. in addition, the participants were required to give us access to their facebook data, including information on their own gender and age as well as the number of facebook friends, by installing a facebook application. employing structural equation modelling for analysing the survey data, we found that williams’ iscs can be used to validly measure online social capital. moreover, our analysis indicates some of the variables influencing offline social capital, among them similarity in terms of sociodemographic attributes, to be less important in an online setting. with this article, we contribute to online social media research by validating the iscs in an online setting and identifying a set of factors influencing online social capital. our study design addresses further methodological shortcomings of the existing literature. firstly, as compared to previous investigations (burke et al., 2010; ellison, lampe, steinfield, & vitak, 2010; steinfield et al., 2008) that relied on samples of university students, our study uses a more diverse and hence more representative selection. secondly, other researchers (e.g., ellison et al., 2007) employed reduced variants of the bonding and bridging social capital scales, while we apply the full 20 questions from williams’ original iscs. thirdly, our structural equation model combines aspects of both confirmatory factor analysis and regression analysis. doing so leads to more accurate results concerning online social capital and the variables affecting it as compared to previous studies that rely on a two-step approach to investigate the influence of online social networks on (offline) social capital. the remainder of the article is structured as follows. after discussing the theoretical background concerning the concept of social capital, we present related work on the relationships between sns and social capital. based on this foundation, we develop our research model, describe how the data were collected and analysed, and discuss our findings. the article closes with concluding remarks, pointing to areas of our future research. australasian journal of information systems grottke, hacker & durst 2018, vol 22, research on recent advances in social media factors determining online social capital? 3 2 social capital: conceptualisation and measurement social capital can be defined as the resources that are embedded in one’s social network and that can be accessed or mobilised through ties in the network (lin, 2001b, p. 29). rooted deeply in the social sciences (hyyppä, 2010), social capital theory has been employed to explain different societal phenomena. for instance, it has been studied as a determinant of public health in medical sociology (marmot & wilkinson, 2005), as a predictor of civil engagement in political science (putnam, 2001) and as a factor influencing team cohesiveness and project success in organisational science (chua, lim, & soh, 2012). social capital is associated with different types of resources, which can be classified into four categories (rostila, 2011):  informational support refers to information, knowledge and advice circulating in the network.  emotional support ensures that persons feel loved, admired, and respected.  instrumental support describes help with tangible needs, e.g., money.  appraisal support refers to assistance with decision-making or giving feedback. enabling a greater sense of belonging, access to information and affective support, social capital is often associated with positive outcomes. however, it has been suggested that specific network structures could contribute to an unequal distribution of social resources across networks (rostila, 2011). drawing on the work on the potential downsides of social capital (portes, 1998), it has further been argued that closed networks might limit an individual’s access to socially-valued resources, and might contribute to the exchange of “bad social capital” by producing and maintaining negative norms and behaviours within networks (rostila, 2011). social capital is commonly distinguished into bonding and bridging social capital (putnam, leonardi, & nanetti, 1994). bonding social capital overlaps with the idea of strong ties. being inward-looking, it is based on networks of very similar individuals with strong connections among each other. usually, bonding social capital is provided by close friends, family members, or neighbours (stone & hughes, 2002). on the other hand, bridging social capital is outward-looking, and it is based on heterogeneous networks with weak ties (granovetter, 1973). while bonding social capital supports individuals in “getting by” in life, bridging social capital helps individuals to “get ahead”, providing better opportunities for the achievement of informational and instrumental resources. yet, not all bridging links are equally useful (lin, 2001a). in the case of job seeking, for instance, a weak-tie contact may be particularly valuable if the contact person is in a more influential and powerful position than the job seeker (avenarius, 2010). individual social capital in offline contexts is commonly measured using one of the following three survey instruments (van der gaag & webber, 2008):  the name generator estimates the network size and identifies the respondent's social network structure and content (van der gaag & webber, 2008).  the position generator is based on the assumption that occupations of network members reflect social resources and that highly-ranked occupations correspond with large returns (van der gaag & webber, 2008). as such, the position generator measures the australasian journal of information systems grottke, hacker & durst 2018, vol 22, research on recent advances in social media factors determining online social capital? 4 highest accessed prestige, range in accessed prestige as well as the number of different positions accessed.  when using the resource generator, the respondent's access to social resources is checked against a list of useful and concrete resources (van der gaag & webber, 2008). afterwards, the respondent provides information on relationships to people having these resources (family members, friends, and acquaintances). focussing on the measurement of bridging and bonding social capital, the iscs (williams, 2006) can be applied to online as well as offline contexts. the measurement tool consists of a bridging and a bonding social capital sub-scale based on the criteria for the two social capital types (putnam et al., 1994). williams’ iscs are the predominantly-used survey tool in studies examining the influence of the use of sns on social capital in localised offline settings (burke et al., 2011, 2010; ellison et al., 2007; steinfield et al., 2008; yoder & stutzman, 2011). in this article, we do not aim to identify specific types of social resources, e.g., informational or emotional support. rather, this article analyses whether williams’ iscs can be used to validly measure the bonding and bridging social capital carried by online social networks. it further investigates the aspects of the online social network (as well as the individual attributes of the user) that influence the amount of bonding and bridging social capital attained. the focus of our research is depicted in figure 1. figure 1. focus of this research 3 social network sites and social capital more than fifteen years ago, researchers were wondering whether or not “cybernetworks carry social capital” (lin, 2001b, p. 214). it was suggested that the connections in the virtual world could lead to a rise of social capital due to the improved access to information for everyone (in theory, at least), as well as due to the increased ability of people to connect with fewer time and space constraints. since these expectations were formulated in 2001, there has been a debate among scholars on how online interactions influence social capital. summarising the body of literature on the subject, three major directions can be identified (ellison et al., 2010). 1. internet use allows people to generate new social capital, e.g., by triggering interactions with previously-unknown people. 2. internet use decreases people’s stock of social capital, e.g., by detracting people from face-to-face interaction. bonding bridging online social capital outcomes informational support emotional support instrumental support appraisal support online social networks individual attributes australasian journal of information systems grottke, hacker & durst 2018, vol 22, research on recent advances in social media factors determining online social capital? 5 3. internet use strengthens people’s offline social network and supports social capital development, e.g., by supplementing traditional means of communication. sns play an important role in keeping in touch with the existing offline social network, and they are also useful for the development of new ties. yet, most online ties are based on some kind of previous offline connection (antoci, sabatini, & sodini, 2011; brandtzæg & nov, 2010; ellison et al., 2007). sns are regarded as particularly advantageous in that they allow users to create and maintain large and diffuse networks in a very convenient way, enabling cheap and easy communication with friends, even those in distant locations (antoci et al., 2011). as social capital does not necessitate close friendships, even weak ties in sns are likely to have a positive impact on social capital (ellison et al., 2010). a number of studies, primarily conducted using the sns facebook, have sought to identify the association between demographic factors, the intensity of sns usage and the amount of bonding or bridging social capital. as mentioned before, most of them exclusively considered the impact of sns use on social capital in a localised offline setting, such as campus life (ellison et al., 2007; steinfield et al., 2008; yoder & stutzman, 2011). as for the research design employed, most of the considered studies were cross-sectional in nature, and were based on questionnaires or surveys using student populations. also, most studies adapted the iscs (williams, 2006) to assess bonding and bridging social capital, for instance at a university (ellison et al., 2007; steinfield et al., 2008; yoder & stutzman, 2011), in one’s general offline social network (burke et al., 2011, 2010) or in the facebook network (jung, gray, lampe, & ellison, 2013). only two studies (brooks, hogan, ellison, lampe, & vitak, 2014; ellison, vitak, gray, & lampe, 2014) considered both general social capital, which is created through interactions with one’s entire network, and facebook-specific social capital, which is created through interactions with one’s facebook network. with respect to demographic factors, older users of sns were found to have lower levels of bonding social capital, but age did not impact bridging social capital (burke et al., 2010; steinfield, dimicco, ellison, & lampe, 2009). according to one study, women tended to experience more bonding social capital than men (burke et al., 2010). regarding the relationship between facebook usage and bonding social capital, studies have led to mixed results. a positive relationship between items related to facebook use (i.e., the time spent on facebook, the attitude towards facebook, and the number of facebook friends) and bonding social capital was identified based on a student sample (ellison et al., 2007) as well as for employees using a corporate sns (steinfield et al., 2009). however, the existence of a significant relationship with bonding social capital could not be confirmed in later studies (kwon, d’angelo, & mcleod, 2013; vitak, ellison, & steinfield, 2011). in addition to the frequency of sns usage and structural properties of the network of facebook friends, the way in which individuals use the sns seems to play an important role. for instance, an increase in directed communication (such as wall posts and comments) is associated with a higher perceived bonding social capital (burke et al., 2010). researchers in the field of computermediated communication and online social media thus found facebook use to have a limited impact on the creation and maintenance of bonding social capital (burke et al., 2011; ellison et al., 2007; steinfield et al., 2008). yet, the online social network is regarded as useful in that it provides another way for the interaction with strong ties, especially when face-to-face interaction is not feasible because close friends and family are geographically dispersed (vitak et al., 2011). in a corporate context, bonding social capital has further been found to be australasian journal of information systems grottke, hacker & durst 2018, vol 22, research on recent advances in social media factors determining online social capital? 6 predictable by the extent to which the internal sns is used to connect with existing contacts (steinfield et al., 2009). bonding social capital turned out to be lower if the site is highly used to connect with new contacts. with respect to bridging social capital, the literature agrees that facebook use has a positive impact on the formation and enhancement of this kind of social capital, e.g., by allowing contact maintenance with a larger and more diverse group of acquaintances (young, 2011). a strong association between facebook usage intensity and bridging social capital could be identified (ellison et al., 2007). the results of this cross-sectional study were confirmed by a follow-up project using longitudinal data (steinfield et al., 2008), in other cross-sectional studies using student samples (kwon et al., 2013; vitak et al., 2011) as well as by a study in a corporate setting (steinfield et al., 2009). due to the identity information provided on a user’s profile page, facebook may be able to connect users with shared interests and encourage the conversion of latent ties into weak ties, which in turn increases bridging social capital (ellison et al., 2011). specifically, perceived bridging social capital appeared to be related to the number of facebook friends (burke et al., 2010). however, later empirical studies have shown that the total number of friends on facebook does not predict bridging social capital, whereas the number of facebook friends considered “actual” friends does (ellison et al., 2011). in this regard, it is not enough to simply add connections; rather, it is necessary to engage in so-called facebook relationship maintenance behaviours, i.e., activities that signal attention, create expectations of reciprocal attention or build trust (brooks et al., 2014; ellison et al., 2014). furthermore, changes in particular communication elements on facebook, such as the increased use of wall posts (lee, kim, & ahn, 2014; yoder & stutzman, 2011) and more frequent receipt of messages from friends (burke et al., 2011), are related to increases in bridging social capital. besides these empirical findings, other researchers have sought to identify network structure indicators to study and measure offline social capital (australian bureau of statistics, 2004; policy research initiative, 2005). in this regard, characteristics of the members were considered to study network diversity. in line with the conceptualisation of social capital (putnam et al., 1994) and the idea behind survey instruments such as the position and resource generator (van der gaag & webber, 2008), networks including members with similar profiles are thought to be associated with bonding relations. on the other hand, bridging relations emerge between individuals who are different from each other and are relatively more socially distant (policy research initiative, 2005). network diversity is measured by considering the social homogeneity of the network members, and is operationalised in terms of different dimensions of socioeconomic status, such as having the same first language, a similar ethnic background, roughly the same level of education, or a similar family income level (australian bureau of statistics, 2004). an overview of the empirical findings in the existing literature and the assumptions regarding the relationships between network diversity and the two forms of offline social capital is given in table 1. significantly positive, insignificant and significantly negative associations are denoted by “+”, “o” and “-”, respectively. assumptions (rather than empirical findings) are indicated by brackets. australasian journal of information systems grottke, hacker & durst 2018, vol 22, research on recent advances in social media factors determining online social capital? 7 explanatory construct bonding social capital bridging social capital gender (0 = “male”, 1 = “female”) + : burke et al. (2010) age : burke et al. (2010); steinfield et al. (2009) sns usage intensity. (*daily time spent on the sns) (*attitude towards the sns) + : ellison et al. (2007); steinfield et al. (2009) o : kwon et al. (2013); vitak et al. (2011) + : ellison et al. (2007); kwon et al. (2013); steinfield et al. (2008); steinfield et al. (2009); vitak et al. (2011) (* number of friends on the sns) + : burke et al. (2010) o : ellison et al. (2011) extent to which sns friends were initially met on the sns : steinfield et al. (2009) extent to which real-life are also sns friends + : steinfield et al. (2009) common mother tongue (+): australian bureau of statistics (2004); policy research initiative (2005) (-): australian bureau of statistics (2004); policy research initiative (2005) similarity of ethnic background similarity of educational background similarity of household income table 1. findings and assumptions concerning the factors determining (offline) social capital addressing the gaps and the directions for further research, most of the reviewed papers suggest that future studies should look at more diverse populations and use longitudinal data to allow the determination of causal relationships (bohn, buchta, hornik, & mair, 2014; brooks et al., 2014; ellison et al., 2014). as the measurement of social capital is still considered an issue, future research should also develop new metrics and validate the existing ones (ellison et al., 2014). 4 model development as shown in the last section, the existing literature has investigated the influence of various factors, such as the intensity of sns usage, on offline bonding and bridging social capital. we will now draw on this previous work to explain which constructs were included in our study of the social capital inherent in the online social network maintained via facebook, and how we operationalised them into variables. 4.1 online social capital we have already mentioned that other authors employed adaptations of the iscs (williams, 2006). for example, they used only five of the questions related to bonding social capital, and nine of the bridging-social-capital-related ones (ellison et al., 2007). in contrast to this, we decided to make use of all ten questions for both the bonding and the bridging social capital scale, listed in table 1 of the article introducing the iscs (williams, 2006), to find out whether these scales are actually appropriate for a purely online setting. to this end, we dropped the word “offline” in each question, and replaced “online” by an explicit reference to facebook. as our study was conducted in germany, all questions were translated into german. (please refer to the appendix for the original german questionnaire employed by us as well as an english version.) in our questionnaire, the 20 questions were grouped in five blocks; for each respondent the questions within a group were randomly ordered. similar to the original article (williams, 2006), all questions were formulated as statements, and the answers were measured australasian journal of information systems grottke, hacker & durst 2018, vol 22, research on recent advances in social media factors determining online social capital? 8 on a five-point likert scale from “strongly disagree” to “strongly agree”. we also followed williams in considering these variables to be metric. as for the constructs potentially able to explain online social capital, we distinguish between individual attributes and aspects of the online social network. an overview of our discussion is given in table 2, where constructs not included in previous studies are printed in italics. group explanatory construct variable name individual gender gender attributes age age duration of sns membership fbmem daily time spent on the sns fbtim attitude towards the sns fbatt online social network number of friends on the sns #fbf extent to which sns friends were initially met on the sns onfbf extent to which real-life friends are also sns friends rlfbf extent to which sns friends are considered close friends clofbf common mother tongue commt similarity of ethnic background simeb similarity of educational background simed similarity of household income simhi table 2. explanatory constructs and their operationalisation in this study 4.2 individual attributes similar to the existing literature, we hypothesise that the gender and the age of a person might affect his or her online social capital. we therefore included the variables gender (with 0 = “male” and 1 = “female”) and age (measured in years) in our model. with respect to the constructs explaining bonding and bridging social capital, literature has focused on the sns usage intensity, defined to be composed of various aspects (ellison et al., 2007): the daily time spent on the sns, the attitude towards the sns, and the number of friends on the sns. in fact, the first two aspects can be regarded as related to the individual him-/ herself. the influence of the number of friends on the sns on bridging social capital was analysed separately only once (burke et al., 2010). for our study, we decided to go a step further, and to distinguish between all aspects lumped together in the sns usage intensity. the variable fbtim in our model is based on the (translated) question referring to the average daily time spent on facebook (ellison et al., 2007), using the same six response categories. similar to a previous study (steinfield et al., 2008), we transformed these categories into a metric scale by assigning the score of the mid-point of each category. for example, the category “1–2 hours” was assigned the value 1.5 [hours]; for the category “more than 3 hours” we chose the value 7.5 [hours], implying a maximum of 12 hours spent on facebook per day on the average. in addition to the daily time spent on facebook, we also investigate the duration of the facebook membership as a possible explanatory variable for online bonding and bridging social capital. in the questionnaire, we made use of the question and the response categories defined in another german study (kneidinger, 2010). however, we added a sixth category representing a facebook membership exceeding four years. by assigning the midpoint to each australasian journal of information systems grottke, hacker & durst 2018, vol 22, research on recent advances in social media factors determining online social capital? 9 category (using the value 5 [years] for the last category), we obtained the metric variable fbmem. in our model, we include the latent variable fbatt, representing the attitude towards facebook, which was measured using the german translation (kneidinger, 2010) of six attitudinal questions (ellison et al., 2007). again, a five-point likert scale ranging from “strongly disagree” to “strongly agree” was employed for each response. 4.3 online social network all respondents were required to give us access to their facebook network data. therefore, we were able to easily measure the number of facebook friends using a tool. unlike other studies (such as steinfield et al., 2009) we thus did not have to rely on self-reported and grouped data resulting from categories representing intervals (e.g., “51–100 facebook friends”). the variable measuring the exact number of facebook friends is named #fbf in our model. the above-mentioned study (steinfield et al., 2009) also investigated whether or not the extent to which sns friends were initially met on the sns and the extent to which real-life friends are also sns friends had any influence on the social capital. there, both constructs were measured on an ordinal scale from 1 (= “none”) to 5 (= “a lot”), but they were treated as metric variables. for assessing our variables onfbf (fraction of facebook friends met online) and rlfbf (fraction of real-life friends who are also facebook friends) via our questionnaire, we chose the categories “none”, “few”, “about half”, “most” and “all”. based on these descriptions, it seems a natural choice to assign the values 0, 0.25, 0.5, 0.75 and 1 to the categories. this is equivalent to using the values 1 to 5, which can be derived via a simple linear transformation. we feel that our names for the categories make it more justifiable to consider the variables to be metric. employing the same five categories from “none” to “all”, we added five further variables to our analysis: the fraction of facebook friends considered close friends (clofbf), the fraction of facebook friends with the same mother tongue (commt), the fraction of facebook friends with a similar ethnic background (simeb), the fraction of facebook friends with a similar educational background (simed), and the fraction of facebook friends with a similar household income (simhi). the latter four variables have been derived based on the abovementioned publications arguing that networks with a higher degree of similarity tend to be associated with a higher bonding social capital but a lower bridging social capital (australian bureau of statistics, 2004; policy research initiative, 2005). the overall structural equation model based on all described variables is depicted in figure 2. it should be noted that our model includes both observable variables (drawn as rectangles) and latent variables (drawn as ovals). previous work (e.g., steinfield et al., 2009) proceeded as follows. first, an exploratory or confirmatory factor analysis was conducted to investigate the association between a latent construct (e.g., bond) and the observable variables (e.g., bond1– bond10). the latent construct was then estimated, for instance by simply averaging the scores of the related observable items. in a second step, these estimates were used to investigate the influence of the explanatory variables on the dependent variables with the help of a separate regression analysis. our structural equation model does not necessitate such simplifying approximations, and it therefore leads to more accurate results. before trying to estimate the model parameters from real data, we need to make sure that none of the equations is underidentified. it is easily seen that for each of the 28 equations in our australasian journal of information systems grottke, hacker & durst 2018, vol 22, research on recent advances in social media factors determining online social capital? 10 model the number of omitted system-exogenous variables is not smaller than the number of explanatory system-endogenous variables. the order criterion (von auer, 2013, p. 589) thus indicates that all equations are exactly identified or overidentified, which is a necessary condition for the identifiability of the entire model. it is also possible to show that the sufficient rank criterion (von auer, 2013, p. 603) is satisfied; therefore, all parameters can be estimated. our model structure implies that bond and brid could both be affected by all the explanatory constructs listed in table 2. technically, we will in each case test the null hypothesis that there is no influence of the respective explanatory variable on brid (or bond) to see whether this hypothesis can be rejected, indicating a significant relationship. figure 2. structural equation model 5 data collection and statistical analysis the participants of our study were drawn from the gfk online panel via an online survey fielded from may 5 until may 22, 2012. all participants were invited via an e-mail containing a short description of the study, some information about confidentiality and incentives, as well as a link to the online survey. each participant was required to have a facebook account and had to be willing to install a facebook application giving us access to his/her network data. we applied a stratified sampling approach based on gender and age to guarantee a representative sample from the facebook population. however, due to age restrictions in the gfk online panel, the age group “17 and younger” was underrepresented in the sample, while australasian journal of information systems grottke, hacker & durst 2018, vol 22, research on recent advances in social media factors determining online social capital? 11 the age groups “18–24” and “25–34” were slightly overrepresented. the study was closed once 1000 persons had participated. to ensure validity, not all 1000 observations were used in our analyses. we dropped those observations for which the credibility of responses was in doubt. as shown in figure 3, the data cleaning process involved the exclusion of observations from our sample based on six criteria. figure 3. data cleaning process four observations were excluded due to unusual patterns and an extraordinarily high variance in the responses. for nine observations the time taken by the respondents for filling in the questionnaire was less than four minutes and 30 seconds, which was considered too fast for thoroughly reading and answering the questions; therefore, these observations were dropped. moreover, comparing each participant’s responses on gender and age with information stored in the gfk online panel profile, we excluded four observations which showed a difference in gender and 22 data sets with a deviation in age exceeding two years. furthermore, we calculated the relative deviation of the number of facebook friends reported in the questionnaire based on the actual number of facebook friends retrieved from the participant’s facebook network; we excluded 57 additional observations which showed a relative deviation of more than 70%. finally, we omitted 16 observations related to participants who turned out to have more than one gfk online panel profile. after this data cleaning process, 888 observations remained. in each case, we made use of the age given in the gfk online panel profile as well as of the actual number of facebook friends extracted from the facebook profile, to rely on the most exact information available. table 3 compares the absolute and relative distribution of age and gender in our resulting data set with the one in the overall facebook community (czech, 2012). 1000 original sample size 996 987 983 961 904 888 sample size after data cleaning process unusual patterns and extraordinarily high variances in responses time for filling in questionnaire below 4.5 minutes discrepancy in actual vs. reported gender deviation in actual vs. reported age exceeding two years relative deviation of the number of facebook friends reported compared to actual number of friends is more than 70% participants with more than one gfk online panel profile -4 -9 -4 -22 -57 -16 australasian journal of information systems grottke, hacker & durst 2018, vol 22, research on recent advances in social media factors determining online social capital? 12 relative frequencies on facebook relative frequencies in our sample absolute frequencies in our sample age (years) 17 and younger 16% 6.1% 54 18–24 27% 30.5% 271 25–34 27% 31.5% 280 35–44 15% 15.9% 141 45–54 10% 10.4% 92 55–64 3% 3.6% 32 65 and older 2% 2.0% 18 total 100% 100% 888 gender female 48% 48.1% 427 male 52% 51.9% 461 total 100% 100% 888 table 3. age and gender stratification before continuing our analysis, we need to answer the question whether the n = 888 observations available suffice for properly and accurately estimating the model shown in figure 2 or whether this sample size should be considered excessive, leading to significant results even for tiny effects. one important aspect to consider is r, the ratio of items to latent variables. while both a higher value of r and a larger sample size n improve estimation, there is a compensatory relation between r and n (marsh, hau, balla, & grayson, 1998). from monte carlo simulations studying the robustness of estimating structural equation models in the context of confirmatory factor analysis (boomsma, 1982), it can be inferred that for r = 3 at least n = 200 observations are required, while for r = 4 a sample size of n = 100 may suffice. these minimum sample sizes for r = 3 and r = 4 are also suggested by the results of a similar simulation study (marsh et al., 1998), while for r = 2 at least n = 400 and for r = 12 at least n = 50 observations seem advisable. consolidating such findings, the following equation for determining the minimum sample size required has been suggested (westland, 2010): n ≥ 50r2 − 450r + 1100. while this formula works nicely for 2 ≤ r ≤ 4, interpolating the empirical results, the quadratic expression reaches its minimum at r = 4.5 and then rises when further increasing r. for r = 12, the minimum sample size computed amounts to 2900, which is completely at odds with the above-mentioned findings. we therefore propose the following continuous and differentiable function for approximating the sample size requirement: 𝑁 ≥ 50𝑟2 450r + 1100 for 2 ≤ r ≤ 4.45, 109.875 5r for 4.45 < r ≤ 12. (1) considering that there is a total of 26 items for the 3 latent variables in our model, the minimum number of observations required is thus calculated as 67. australasian journal of information systems grottke, hacker & durst 2018, vol 22, research on recent advances in social media factors determining online social capital? 13 however, the ratio r is only one aspect on which the necessary sample size depends. the minimum effect to be identified in the analysis as well as the desired significance level and the power of the test should also be taken into consideration. assuming that all observed variables follow a normal distribution, it has been shown (westland, 2010, 2012) that the sample size required for detecting a minimum gini correlation of size δ among k latent variables in a test with significance level α and power 1 − β can be calculated as 𝑁 ≥ 1 2𝐻 𝐴 𝜋 6 − 𝐵 + 𝐷 + 𝐻 + 𝐴 𝜋 6 − 𝐵 + 𝐷 + 𝐻 + 4 𝐴𝐻 𝜋 6 + √𝐴 + 2𝐵 − 𝐶 − 2𝐷 . (2) in this equation, a = 1 – δ², b = δ arcsin (δ/2), c = δ arcsin (δ), d = √ , h = ᾶ/ , where λq denotes the quantile of the standard normal distribution at level q, and ᾶ = 1 – (1 – α)2/k(k-1) is the significance level after šidak correction. for correlations, δ values of 0.1, 0.3 and 0.5 represent a small, medium and large effect, respectively (ellis, 2010, p. 41). based on a standard significance level of 5% and a power of 80%, we would thus need minimum sample sizes of 290, 28 and 8 to detect small, medium and high correlations between our 3 latent variables according to equation (2). (of course, considering the ratio of items to latent variables, n should never be smaller than 67, as derived from equation (1).) it should be noted that all calculations assume normally-distributed variables. for likert-scale data, as in our study, the sample size requirement may be two to three orders of magnitude larger due to non-normalcy (westland, 2010). nevertheless, this analysis suggests that our 888 observations are definitely not excessive and may suffice for identifying large effects. based on the available data, we estimated our structural equation model with the statistical tool r (r core team, 2018) and its sem package (fox, 2006). we started out with a basic model containing parameters for all relationships indicated in figure 2, as well as variance parameters for all disturbances. the variances of the latent variables fbatt, bond and brid were set to one, to ensure identifiability. following the modification indices calculated (sörbom, 1989), we subsequently added a parameter for the covariance between bond and brid, 19 covariance parameters between pairs of items, as well as three covariances between an item and the related latent construct. the parameter estimation was conducted based on the variance-covariance matrix of the observable variables. to check whether or not the results are influenced by the different scales used for the various variables, we also carried out parameter estimation using the correlation matrix; however, the results proved to be stable. also, the results were hardly affected when adding further covariance parameters. drawing on the visualisation of our model (figure 2), figure 4 illustrates the detected relationships between the investigated variables. in this regard, non-significant relationships are omitted, and those variables considered in our study for which none of the relationships with the dependent variables turned out to be significant are shown as white boxes with dashed frames. significant relationships with a negative value of the estimated parameter are indicated by a minus sign above the directed edge. those directed edges without a minus sign thus represent significant relationships with a positive estimated parameter. australasian journal of information systems grottke, hacker & durst 2018, vol 22, research on recent advances in social media factors determining online social capital? 14 providing more detailed insights, table 4 lists the significant parameter estimates, omitting the (co-)variance parameters, and their related p values. these p values have been derived using the asymptotic distributions of the test statistics, which seems appropriate considering the large sample size of 888. moreover, the fifth entry in each row is the beta coefficient, showing by how many standard deviations the respective dependent variable is expected to increase or decrease if the related explanatory variable is increased by one standard deviation. for the two equations explaining bond and brid, unlike the original estimates these beta coefficients allow us to rank the explanatory variable according to their relative effect, which is the greater the larger the absolute value of the beta coefficient is. these ranks are listed in the sixth column of the table. the overall goodness of fit of our model in terms of the root mean square error of approximation (rmsea) amounts to 0.0586. literature agrees that an rmsea exceeding 0.1 is not acceptable (browne & cudeck, 1992; schlittgen, 2009). while one of the two researchers who originally suggested the rmsea metric considers the model fit to be good for values below 0.1 and very good for values below 0.05 (steiger, 1989, p. 81), other authors think that an rmsea value between 0.08 and 0.05 indicates a reasonable error of approximation (browne & cudeck, 1992) or an adequate model fit (schlittgen, 2009, p. 476). the overall ability of our model to represent the data can thus be considered satisfactory. figure 4. significant relationships in the estimated structural equation model australasian journal of information systems grottke, hacker & durst 2018, vol 22, research on recent advances in social media factors determining online social capital? 15 group (expl. var.) expl. var. → dep. var. estimate p value beta coeff. rank online social capital bond → bond1 0.7805 8.0728 ∙ 10-136 1.0244 bond → bond2 0.7758 3.4317 ∙ 10-119 0.9004 bond → bond3 -0.3474 1.9594 ∙ 10-17 -0.2575 bond → bond4 0.8346 1.1627 ∙ 10-148 1.1745 bond → bond5 0.4197 1.6184 ∙ 10-23 0.3062 bond → bond6 0.6051 4.4996 ∙ 10-56 0.6924 bond → bond7 0.5561 1.0948 ∙ 10-71 0.6119 bond → bond8 0.5101 2.2098 ∙ 10-63 0.5918 bond → bond9 -0.2147 2.4067 ∙ 10-10 -0.1869 bond → bond10 0.5072 1.0821 ∙ 10-55 0.5143 online social capital brid → brid1 0.5091 1.1006 ∙ 10-98 0.6581 brid → brid2 0.5409 5.3701 ∙ 10-118 0.8278 brid → brid3 0.5011 1.8901 ∙ 10-100 0.6714 brid → brid4 0.4941 1.0905 ∙ 10-87 0.5756 brid → brid5 0.5125 4.2903 ∙ 10-110 0.7125 brid → brid6 0.5412 1.2860 ∙ 10-115 0.8045 brid → brid7 0.4592 2.5636 ∙ 10-77 0.5117 brid → brid8 0.4875 6.1073 ∙ 10-67 0.5937 brid → brid9 0.4482 6.5238 ∙ 10-80 0.5329 brid → brid10 0.4297 1.9892 ∙ 10-58 0.4913 individual attributes fbatt → fbatt1 0.9175 2.4797 ∙ 10-125 1.0433 fbatt → fbatt2 0.6844 1.3061 ∙ 10-88 0.7938 fbatt → fbatt3 0.9560 1.1073 ∙ 10-142 1.1550 fbatt → fbatt4 0.7660 2.2672 ∙ 10-95 0.8696 fbatt → fbatt5 0.9279 7.5468 ∙ 10-177 1.3873 fbatt → fbatt6 0.9025 5.1982 ∙ 10-128 1.0737 individual attributes age → bond -0.0171 5.7868 ∙ 10-7 -0.2115 3 fbtim→ bond 0.0944 3.4035 ∙ 10-6 0.1829 4 fbatt → bond 0.4889 2.4958 ∙ 10-29 0.4889 1 online social network onfbf → bond -0.3883 0.0398 -0.0797 6 clofbf → bond 1.1661 4.9037 ∙ 10-8 0.2131 2 simed → bond 0.8647 7.5531 10-5 0.1619 5 individual attributes gender → brid -0.2188 0.0055 -0.1092 6 age → brid 0.0166 4.3625 ∙ 10-6 0.2054 3 fbatt → brid 1.2849 1.2326 ∙ 10-67 1.2849 1 online social network #fbf → brid 0.0008 0.0026 0.1336 4 onfbf → brid 1.1426 1.9235 ∙ 10-8 0.2346 2 clofbf → brid 0.6024 0.0069 0.1101 5 commt → brid -0.5097 0.0231 -0.1049 8 simed → brid 0.5645 0.0138 0.1057 7 table 4. significant parameter estimates 6 discussion our study was aimed at answering the question. what are the factors that influence online social capital? in order to assess bonding and bridging social capital in an online setting, we used williams’ iscs. before we analysed the influencing factors, we validated those scales. table 4 shows that all items supposed to measure a latent construct are loading significantly on the respective factor. this suggests that all items within each group are indeed measuring the same construct. australasian journal of information systems grottke, hacker & durst 2018, vol 22, research on recent advances in social media factors determining online social capital? 16 the items bond3 and bond9 were designed to be reversed-scaled (williams, 2006); this is correctly reflected in the negative loadings. however, the fraction of the overall variation within the items fbatt1–fbatt6 explained by fbatt amounts to 53.7%, while only 22.6% of the total variance of bond1–bond10 and 19.1% of the variation in brid1– brid10 are explained by the related latent construct. this might indicate that there are further factors influencing the answers to the questions bond1–bond10 and brid1–brid10. regarding the factors influencing online social capital, the last 14 lines of table 4 list the variables that were found to significantly affect online bonding social capital and online bridging social capital, respectively, as well as their ranks according to the strength of the relative effects. as already depicted in figure 4, our analysis indicates significant influences of the variables fbatt, age, onfbf, clofbf and simed on both types of online social capital. while fbtim only affects online bonding social capital, gender, #fbf and commt solely influence online bridging social capital. no significant relationships with online social capital could be found for fbmem, rlfbf, simeb and simhi. as mentioned before, early investigations had found a positive relationship between the facebook usage intensity and the two forms of social capital perceived in offline settings (ellison et al., 2007; steinfield et al., 2008), while follow-up studies (kwon et al., 2013; vitak et al., 2011) did not find an association with bonding social capital. instead of lumping the daily time spent one the sns, the attitude towards the sns, and the number of friends on the sns within one concept, we considered the influence of each variable individually. facebook attitude (as in considering facebook a part of one’s daily life and being emotionally attached to the sns) has the strongest effect on both online bonding and online bridging social capital, and in each case it is a positive one. regarding online bonding social capital, it seems reasonable that users with a positive attitude towards facebook find the sns suitable to communicate with their close friends. as for bridging social capital, the feeling that one belongs to the facebook community may reinforce a person’s willingness to share updates and to keep in touch with people who are emotionally more distant. disclosing personal details makes other people aware of a user, and it is one way to maintain and develop weaker ties and thus reinforces online bridging social capital (ellison et al., 2007). for both types of online social capital, the variable with the second strongest effect within the category of individual attributes is age. our findings indicate that older sns users will have more online bridging social capital. moreover, confirming the results of previous studies (burke et al., 2010; steinfield et al., 2009), older users tend to experience less online bonding social capital than younger users. indeed, in 2012 more than half of the facebook users in germany were aged between 18 and 34 years (see table 3). hence, older users may be more reluctant or less able to communicate with close friends using the sns, for example because many of their close friends do not have a facebook account. on the other hand, older users may have a more diverse network in terms of socioeconomic background or age, which could explain their larger online bridging social capital. we further find the daily time spent on the sns to be positively related to online bonding, but unrelated to online bridging social capital. amount of time being one indicator of tie strength (granovetter, 1973), individuals may use the sns to maintain closer ties via direct communication, e.g., by writing private messages, which tends to be more time-consuming than liking a status update (burke et al., 2010). australasian journal of information systems grottke, hacker & durst 2018, vol 22, research on recent advances in social media factors determining online social capital? 17 while we could not detect a significant relationship between gender and online bonding social capital, our analysis shows that female participants tend to have lower levels of online bridging social capital. this relationship, however, is relatively weak compared to the previously-discussed effects. women might select their sns friends more cautiously or might not see a benefit in a high number of weak ties in their network, and may hence realise less bridging connections than men. as can be seen in table 4, all the constructs from the online social network category significantly related to online bonding social capital are significantly related to online bridging social capital, too. in addition, significant relationships can be established between #fbf and commt and online bridging social capital. in line with the conceptualisation of (offline) social capital (putnam et al., 1994) and the theory of tie strength (granovetter, 1973), we find the fraction of close friends on facebook to be positively related to online bonding social capital. in contrast to the conceptualisation, it is also (but much more weakly) positively related to online bridging social capital. in this regard, two explanations seem reasonable. first, close friends in the offline world are oftentimes individuals whom we meet and talk to frequently. frequent contact is often related to living at the same place or to having a similar background, as in being related. facebook, however, enables an easy and fast way to stay in touch with close friends who may for example live in a distant location or have a very different background. we can thus hypothesise that facebook helps maintain ties with close friends who are different and who thus allow for access to non-redundant sets of resources and online bridging social capital. second, the communication mechanisms on facebook facilitate the easy creation of new contacts. for instance, people who have only met a few times and who have very different backgrounds may add each other on facebook, follow each other’s updates, and eventually become close friends. our analysis further shows that the fraction of sns friends initially met on the sns increases online bridging social capital but decreases online bonding social capital. in fact, such connections often result from playing facebook games together, which is the main way to initiate new friendships on the sns. as these games attract a heterogeneous community, the connections may be a source of online bridging social capital. confirming the findings of one of the papers discussed in our literature review (steinfield et al., 2009), we have been able to establish a negative relationship between the fraction of sns friends initially met on the sns and online bonding social capital. representing a relatively small part of an individual’s friendship network, the absolute number of close friends is likely to remain constant even if the overall size of the network increases due to new weak ties created through gaming, etc. in accordance with the findings in an offline setting (burke et al., 2010), a higher number of facebook friends is related to increases in online bridging social capital, as a larger network bears the chance of having access to more diverse resources. however, network size does not have an influence on online bonding social capital. we further investigated the relationships between the composition of an individual’s facebook friendship network in terms of different variables related to socioeconomic status and online social capital (australian bureau of statistics, 2004; policy research initiative, 2005). according to the definition of bonding and bridging social capital (putnam et al., 1994), the underlying assumption is that networks composed of homogeneous people are rich in bonding social capital, whereas networks of dissimilar people are a source of bridging social capital. we did not find similarity in terms of ethnic background or household income to be australasian journal of information systems grottke, hacker & durst 2018, vol 22, research on recent advances in social media factors determining online social capital? 18 related to any of the two forms of social capital. these variables may be less important in online than in offline settings. against our expectations, similarity in terms of education is positively related to both online bonding and bridging social capital. on the one hand, people may have close relationships with friends from school or university and use facebook to communicate with them. on the other hand, similarity in educational background, as in a similar level of education, and online bridging social are not mutually exclusive. for instance, even if an individual’s friendship network mostly includes people who have completed undergraduate studies, these friends can still have degrees in very different fields, and hence generate online bridging social capital. as expected, a higher proportion of facebook friends with a different mother tongue is associated with increases in online bridging social capital. facebook friends with other mother tongues are likely to be made when travelling or spending time abroad. while often not close, these connections can easily be maintained on facebook. the findings of this study need to be weighed against its potential limitations. we used an online panel, which comes with certain disadvantages (göritz & moser, 2000). first, panel members are aware of being observed and have a great routine in participating in surveys, which may affect their way of answering questions. however, we have omitted those observations where the credibility was doubtful, e.g., due to patterns in the answers, or deviations of the answers from previously-known information. moreover, all invited panel members decided themselves whether or not to answer our questionnaire. as a consequence, frequent facebook users might be overrepresented in our sample. however, such selfselection effects are not a problem if they are related to the observed explanatory variables, such as fbtim. furthermore, while our stratified sampling approach did not consider attributes such as the educational background of the participants, we made sure that the sample is representative for the facebook users in terms of age and gender. 7 conclusions and outlook in this article, we have analysed factors influencing online social capital, the validation of williams’ iscs being a prerequisite for our investigation. based on our statistical analysis we have found that all items in each of the two catalogues of items in williams’ iscs indeed seem to measure the same latent construct. this suggests that williams’ iscs can be used to validly measure online social capital. similar to previous studies in offline settings, researchers could hence investigate the relationships between online bonding or bridging social capital, access to informational (and other types of) support, and further outcomes such as professional success or a healthy lifestyle. as for the factors influencing online social capital, we have found significant influences of the variables age, fbatt, onfbf, clofbf and simed on both types of online social capital. while fbtim only affects online bonding social capital, gender, #fbf and commt solely influence online bridging social capital. no significant relationships with online social capital could be found for fbmem, rlfbf, simeb and simhi. our study has important implications for research and practice. in terms of research, we propose the following implications:  in online settings, birds of a feature do not necessarily flock together. the similarity or dissimilarity of the members in one’s online friendship network in terms of different sociodemographic attributes is not a very strong predictor of the different types of online social capital. specifically, the influence of similarity in terms of ethnic australasian journal of information systems grottke, hacker & durst 2018, vol 22, research on recent advances in social media factors determining online social capital? 19 background and household income is less pronounced than in offline settings. furthermore, in contrast to previous studies performed in offline settings (australian bureau of statistics, 2004; policy research initiative, 2005), we found the similarity of the educational background to be positively related to bridging social capital.  the antecedents of online bridging social capital differ from the ones of offline bridging social capital. while the factors influencing online bonding social capital seem to be similar to the ones identified in offline settings, online bridging social capital is influenced by other factors as compared to offline settings. in this regard, we found that four variables (i.e., gender, age, onfbf, and rlfbf), which are not indicated to impact offline bridging social capital (table 1), have a significant effect on online bridging social capital. additionally, we found that the new variable clofbf is also positively related to both types of online social capital, with a stronger impact on bonding social capital.  different types of online social capital are likely related to different outcomes. like offline social capital, online bonding and bridging social capital may be associated with different types of resources and outcomes. while out of the scope of this paper, it is highly relevant to identify, characterise, and measure these resources, as well as to compare resources obtained based on different types of social capital offline and online. for instance, the association between online bonding social capital with informational support might be higher as compared to offline bonding social capital depending on the network of an individual’s friends, e.g., the number of weak ties in their networks. furthermore, it would be interesting to determine the “right” ratio between bonding and bridging social capital for different groups of individuals in order to achieve positive outcomes, e.g., concerning an individual’s health (durst, viol, & wickramasinghe, 2013).  studies investigating online social capital can focus on fewer variables. as some of our variables turned out to not be related to online social capital, we will be able to reduce the number of variables in future studies, decreasing the burden of assessing these pieces of information. furthermore, we suggest the following practical implications:  creating and maintaining a positive sns attitude is important. we found that fbatt is the most influential predictor of both bonding and bridging online social capital. hence, providers of sns need to assure that the platform becomes embedded into the user’s daily lives in order to achieve a positive attitude. exemplary implications concern the design of the platform, such as providing a pleasant interface that is easy and fun to use. moreover, platform providers should protect their users’ privacy in order to create a trustful environment in which people are willing to share personal information. they also should have further measures in place, for example to prevent bullying and the spread of fake news, as well as to give users control over comments on their posted content.  online social platforms could contribute to balancing different types of resources accessible to users. provided that it will be possible to measure resources accessible to users in the future, online social platforms could help users gain access to additional resources that they currently lack. for instance, users with high ratios of bonding social capital over australasian journal of information systems grottke, hacker & durst 2018, vol 22, research on recent advances in social media factors determining online social capital? 20 bridging social capital could be increasingly prompted with information regarding events that are different from what they would normally see in their newsfeed. besides contributing to breaking up echo chambers, platform vendors could address various societal issues (durst et al., 2013), among them public health and education. for example, persons showing unhealthy eating habits could be provided with information regarding healthy food or sports activities in their newsfeed (hacker, wickramasinghe, & durst, 2017). likewise, individuals with limited access to education could be prompted with information on online courses.  self-diagnostics may be a useful tool for interventions driven by individuals themselves. being able to measure online social capital, platform vendors could implement tests that allow individuals to assess their own online social capital and the related resources that they can access. as a tool for self-reflection and benchmarking, such tests might help individuals to recognise the absence of certain types of resources in their networks and motivate them to do something about it. for instance, individuals with a lack of online bridging social capital could join additional groups or subscribe to additional updates to access more diverse information. in terms of future research, we will investigate to what extent the process of data collection can be automated. some of the variables (e.g., gender, age, and #fbf) can directly be retrieved via the facebook application as users log into the application via facebook login (facebook, 2018b). for other variables, such as the duration of the facebook membership and the daily time spent on facebook, it is relatively easy to infer the values based on the data available. in terms of additional variables, data about a user’s “likes” or events a user attended, which can be collected based on an application (facebook, 2018a), could be used to determine a user’s level of access to informational resources, for instance. moreover, we will develop and apply data mining techniques that can be used to calculate values for the constructs that cannot simply be retrieved. for example, analysing tie strength, e.g., by identifying frequent communication partners, could help us determine the number of close friends, and certain usage patterns could allow conclusions regarding the attitude towards the sns. moreover, measures of homogeneity in terms of socioeconomic status and background can be determined if access to the data is granted. our ultimate goal is to estimate online social capital based on a small set of related variables that can be collected automatically. being able to do that would facilitate further analyses, concerning for example the dynamics of online social capital over time. acknowledgements we would like to thank the gfk verein (http://www.gfk-verein.org/) for supporting this study with regard to data collection via the gfk online panel. references anheier, h., & kendall, j. (2002). interpersonal trust and voluntary associations: examining three approaches. the british journal of sociology, 53(53), 343–362. antoci, a., sabatini, f., & sodini, m. (2011). see you on facebook! a framework for analyzing the role of computer-mediated interaction in the evolution of social capital. munich. retrieved 2015-04-30, from http://mpra.ub.uni-muenchen.de/29998/1/mprapaper29998.pdf australasian journal of information systems grottke, hacker & durst 2018, vol 22, research on recent advances in social media factors determining online social capital? 21 australian bureau of stat istics. (2004). measuring social capital : an australian framework and indicators (abs information paper no. 1378.0). australian bureau of statistics . canberra, australia. retrieved 2018-01-24, from http://www.ausstats.abs.gov.au/ausstats/free.nsf/lookup/13c0688f6b98dd45ca256e 360077d526/$file/13780_2004.pdf avenarius, c. b. (2010). starke und schwache beziehungen. in c. stegbauer & r. häußling (eds.), handbuch netzwerkforschung (pp. 99–111). wiesbaden. vs verlag für sozialwissenschaften. bohn, a., buchta, c., hornik, k., & mair, p. (2014). making friends and communicating on facebook. implications for the access to social capital. social networks, 37, 29–41. boomsma, a. (1982). the robustness of lisrel against small sample sizes in factor analysis models. in k. g. jöreskog & h. wold (eds.), systems under indirect observation. causality, structure, prediction. part i (pp. 149–173). amsterdam. north-holland. brandtzæg, p. b., & nov, o. (2010). facebook use and social capital – a longitudinal study. in proceedings of the fifth international aaai conference on weblogs and social media (pp. 454– 457). brooks, b., hogan, b., ellison, n., lampe, c., & vitak, j. (2014). assessing structural correlates to social capital in facebook ego networks. social networks, 38, 1–15. browne, m. w., & cudeck, r. (1992). alternative ways of assessing model fit. sociological methods & research, 21 (2), 230–258. burke, m., kraut, r., & marlow, c. (2011). social capital on facebook. differentiating uses and users. in proceedings of the 29th international conference on human factors in computing systems chi ’11 (pp. 571–580). burke, m., marlow, c., & lento, t. (2010). social network activity and social well-being. in proceedings of the 28th international conference on human factors in computing systems chi ’10 (pp. 1909–1912). christakis, n., & fowler, j. h. (2013). social contagion theory: examining dynamic social networks and human behavior. statistics in medicine, 32 (4), 556–77. chua, c., lim, w., & soh, c. (2012). enacting clan control in complex it projects. a social capital perspective. mis quarterly, 36 (2), 577–600. czech, c. (2012). facebook-statistiken für deutschland. retrieved 2012-04-15, from http://neunzehn80.de/facebook-statistik/ durst, c., viol, j., & wickramasinghe, n. (2013). online social networks, social capital and health-related behaviours: a state-of-the-art analysis. communications of the association for information systems, 32, 134–158. ellis, p. d. (2010). the essential guide to effect sizes. statistical power, meta-analysis, and the interpretation of research results. cambridge. cambridge university press. ellison, n., lampe, c., steinfield, c., & vitak, j. (2010). with a little help from my friends. social network sites and social capital. in z. papacharissi (ed.), a networked self. identity, community and culture on social network sites (pp. 124–145). new york. routledge. australasian journal of information systems grottke, hacker & durst 2018, vol 22, research on recent advances in social media factors determining online social capital? 22 ellison, n., steinfield, c., & lampe, c. (2007). the benefits of facebook “friends.” social capital and college students’ use of online social network sites. journal of computer-mediated communication, 12 (4), 1143–1168. ellison, n., steinfield, c., & lampe, c. (2011). connection strategies. social capital implications of facebook-enabled communication practices. new media & society, 13 (6), 873–892. ellison, n., vitak, j., gray, r., & lampe, c. (2014). cultivating social resources on social network sites. facebook relationship maintenance behaviors and their role in social capital processes. journal of computer-mediated communication, 19 (4), 855–870. facebook. (2018a). permissions reference facebook login. retrieved 2018-01-24, from https://developers.facebook.com/docs/facebook-login/permissions facebook. (2018b). permissions with facebook login. retrieved 2018-01-24, from https://developers.facebook.com/docs/facebook-login/permissions/overview fox, j. (2006). structural equation modeling with the sem package in r. structural equation modeling, 13 (3), 465–486. göritz, a. s., & moser, k. (2000). repräsentativität im online panel. der markt, 39 (4), 156–162. granovetter, m. s. (1973). the strength of weak ties. the american journal of sociology, 78 (6), 1360–1380. hacker, j., wickramasinghe, n., and durst, c. 2017. “can health 2.0 address critical healthcare challenges? insights from the case of how online social networks can assist in combatting the obesity epidemic,” australasian journal of information systems (21). hyyppä, m. t. (2010). healthy ties. dordrecht. springer netherlands. jung, y., gray, r., lampe, c., & ellison, n. (2013). favors from facebook friends. unpacking dimensions of social capital. in proceedings of the 31st international conference on human factors in computing systems chi ’13 (pp. 11–20). kneidinger, b. (2010). facebook und co. wiesbaden. vs verlag für sozialwissenschaften. kwon, m.-w., d’angelo, j., & mcleod, d. m. (2013). facebook use and social capital. to bond, to bridge, or to escape. bulletin of science, technology & society, 33 (1-2), 35–43. lee, e., kim, y. j., & ahn, j. (2014). how do people use facebook features to manage social capital? computers in human behavior, 36, 440–445. lin, n. (2001a). building a network theory of social capital. in n. lin, k. s. cook, & r. burt (eds.), social capital. theory and research (pp. 3–29). new brunswick. transaction publishers. lin, n. (2001b). social capital. a theory of social structure and action. cambridge. cambridge university press. marmot, m., & wilkinson, r. g. (eds.). (2005). social determinants of health (2nd ed.). oxford university press. marsh, h. w., hau, k.-t., balla, j. r., & grayson, d. (1998). is more ever too much? the number of indicators per factor in confirmatory factor analysis. multivariate behavioral research, 33 (2), 181–220. australasian journal of information systems grottke, hacker & durst 2018, vol 22, research on recent advances in social media factors determining online social capital? 23 policy research initiative. (2005). measurement of social capital. retrieved 2018-01-24, from http://www.ocol-clo.gc.ca/sites/default/files/measurement_of_social_capital.pdf portes, a. (1998). social capital. its origins and applications in modern sociology. annual review of sociology, 24 (1), 1–24. putnam, r. d. (2001). bowling alone. new york. simon & schuster. putnam, r. d., leonardi, r., & nanetti, r. y. (1994). making democracy work. civic traditions in modern italy. princeton. princeton university press. r core team. (2018). r. a language and environment for statistical computing. vienna. retrieved 2018-01-24, from http://www.r-project.org/ rostila, m. (2011). a resource-based theory of social capital for health research. can it help us bridge the individual and collective facets of the concept? social theory & health, 9 (2), 1– 21. schlittgen, r. (2009). multivariate statistik. münchen. oldenbourg verlag. sörbom, d. (1989). model modification. psychometrika, 54 (3), 371–384. steiger, j. h. (1989). ezpath. casual modeling [computer software manual]. evanston. steinfield, c., dimicco, j. m., ellison, n. b., & lampe, c. (2009). bowling online. in proceedings of the fourth international conference on communities and technologies c&t ’09 (pp. 245–254). steinfield, c., ellison, n., & lampe, c. (2008). social capital, self-esteem, and use of online social network sites. a longitudinal analysis. journal of applied developmental psychology, 29 (6), 434–445. stone, w., & hughes, j . (2002). measuring social capital. towards a standardised approach. in 2002 australasian evaluation society international conference . wollongong, sydney. retrieved 2018-01-24, from http://evaluationcanada.ca/distribution/20021030_stone_wendy_hughes_jody.pdf van der gaag, m., & webber, m. (2008). measurement of individual social capital. in i. kawachi, s. subramanian, & d. kim (eds.), social capital and health (pp. 29–49). new york. springer. vitak, j., ellison, n., & steinfield, c. (2011). the ties that bond. re-examining the relationship between facebook use and bonding social capital. in proceedings of the 44th hawaii international conference on system sciences (pp. 1–10). von auer, l. (2013). ökonometrie (6th ed.). berlin. springer verlag. westland, j. c. (2010). lower bounds on sample size in structural equation modeling. electronic commerce research and applications, 9 (6), 476–487. westland, j. c. (2012). erratum to ‘lower bounds on sample size in structural equation modeling’. electronic commerce research and applications, 11 (4), 445. williams, d. (2006). on and off the ’net. scales for social capital in an online era. journal of computer-mediated communication, 11 (2), 593–628. yoder, c., & stutzman, f. (2011). identifying social capital in the facebook interface. in proc. sigchi conference on human factors in computing systems (chi 2011) (pp. 585–588). australasian journal of information systems grottke, hacker & durst 2018, vol 22, research on recent advances in social media factors determining online social capital? 24 young, k. (2011). social ties, social networks and the facebook experience. international journal of emerging technologies and society, 9 (1), 20 – 34. australasian journal of information systems grottke, hacker & durst 2018, vol 22, research on recent advances in social media factors determining online social capital? 25 appendices appendix 1. german version of the online questionnaire a soziodemographie i s1 quotenfrage zunächst möchten wir sie bitten, einige angaben zu ihrer person zu machen. sind sie männlich oder weiblich?  männlich  weiblich s2 quotenfrage wie alt sind sie?  (bitte eintragen) |__|__| jahre s3 screenerfrage besitzen sie ein facebook-konto?  ja  nein s4 momentaufnahme des facebook-netzwerkes für unsere studie benötigen wir eine momentaufnahme ihres facebook-netzwerkes. klicken sie hierzu bitte auf [weiter] und akzeptieren sie die genehmigungsanfragen der facebookanwendung. b internetnutzung und facebook 1. allgemeine informationen i1 bitte machen sie im nächsten abschnitt angaben zu ihrer internetund facebook-nutzung. wie viel zeit verbringen sie durchschnittlich pro tag im internet?  weniger als 10 min  11-30 min  31-60 min  1-2 h  2-3 h  mehr als 3 h australasian journal of information systems grottke, hacker & durst 2018, vol 22, research on recent advances in social media factors determining online social capital? 26 i2 wie lange sind sie schon mitglied auf facebook?  6 monate oder weniger  6 monate bis 1 jahr  1-2 jahre  2-3 jahre  3-4 jahre  mehr als 4 jahre i3 wie viel zeit verbringen sie durchschnittlich pro tag auf facebook?  weniger als 10 min  11-30 min  31-60 min  1-2 h  2-3 h  mehr als 3 h i4 im folgenden interessieren wir uns für ihre einstellung zu facebook. bitte wählen sie jeweils die für sie am besten passende antwortkategorie aus. 1 bedeutet „trifft überhaupt nicht zu“ und 5 bedeutet „trifft voll und ganz zu“. mit den werten dazwischen können sie ihre meinung abstufen.  facebook ist teil meines täglichen lebens.  ich bin stolz, wenn ich jemandem sagen kann, dass ich auf facebook bin.  facebook ist zu einem fixen bestandteil meines täglichen lebens geworden.  ich fühle mich von der realität abgeschnitten, wenn ich längere zeit nicht auf facebook eingeloggt war.  ich fühle mich als teil der facebook-gemeinschaft.  ich fände es schade, wenn es facebook nicht mehr gäbe. 2. freundesnetzwerk auf facebook i5 mit den nachfolgenden fragen möchten wir nun etwas mehr über ihre facebook-freunde erfahren. wie viele facebook-freunde haben sie im moment:  __ freunde australasian journal of information systems grottke, hacker & durst 2018, vol 22, research on recent advances in social media factors determining online social capital? 27 i6 denken sie an ihr freundesnetzwerk auf facebook. wie viele ihrer facebook-freunde haben sie online kennengelernt?  keinen  wenige  etwa die hälfte  die meisten  alle  keine angabe i7 denken sie nun an ihre freunde aus dem realen leben. mit wie vielen dieser freunde sind sie auch auf facebook verbunden?  keinem  wenigen  etwa der hälfte  den meisten  allen  keine angabe i8 denken sie an ihr freundesnetzwerk auf facebook. wie viele ihrer facebook-freunde würden sie als engere freunde bezeichnen?  keinen  wenige  etwa die hälfte  die meisten  alle  keine angabe i9 denken sie an die personen, mit denen sie auf facebook verbunden sind. wie viele ihrer facebook-freunde sprechen die gleiche muttersprache wie sie?  keiner  wenige  etwa die hälfte  die meisten  alle  keine angabe australasian journal of information systems grottke, hacker & durst 2018, vol 22, research on recent advances in social media factors determining online social capital? 28 i10 wie viele ihrer facebook-freunde haben einen ähnlichen ethnischen hintergrund wie sie?  keiner  wenige  etwa die hälfte  die meisten  alle  keine angabe i11 wie viele ihrer facebook-freunde haben ein ähnliches bildungsniveau wie sie?  keiner  wenige  etwa die hälfte  die meisten  alle  keine angabe i12 wie viele ihrer facebook-freunde haben ein ähnlich hohes haushaltseinkommen wie sie?  keiner  wenige  etwa die hälfte  die meisten  alle  keine angabe i13 bitte beantworten sie im nächsten abschnitt einige allgemeine fragen zu ihrem freundesnetzwerk auf facebook. bitte wählen sie die jeweils auf sie am besten passende antwortkategorie aus. 1 bedeutet „trifft überhaupt nicht zu“ und 5 bedeutet „trifft voll und ganz zu“. mit den werten dazwischen können sie ihre meinung abstufen. block 1  das kommunizieren mit personen auf facebook weckt meine neugier für andere orte in der welt.  ich verbringe meine zeit gern damit, gemeinschaftliche aktivitäten auf facebook zu unterstützen.  auf facebook lerne ich ständig neue leute kennen. australasian journal of information systems grottke, hacker & durst 2018, vol 22, research on recent advances in social media factors determining online social capital? 29 block 2 auf facebook gibt es …  … mehrere personen, denen ich meine probleme anvertraue und die mir weiterhelfen können.  …jemanden, den ich um rat fragen kann, wenn ich sehr wichtige entscheidungen treffen muss.  … niemanden, mit dem ich bereitwillig über intime persönliche probleme spreche.  … mehrere personen, mit denen ich reden kann, wenn ich mich einsam fühle. block 3 durch den austausch mit personen auf facebook…  … fühle ich mich als teil einer größeren gemeinschaft.  … fühle ich mich mit dem großen ganzen verbunden.  … wird mir bewusst, dass alle menschen auf der welt miteinander verbunden sind.  … interessiere ich mich für dinge, die außerhalb meiner stadt geschehen.  … möchte ich neue sachen ausprobieren.  … interessiere ich mich dafür, was leute, die anders als ich sind, denken.  … lerne ich neue leute kennen, mit denen ich mich unterhalten kann block 4 die personen, mit denen ich mich auf facebook austausche, würden…  … ihr letztes geld mit mir teilen.  … mir dabei helfen, ein unrecht zu bekämpfen.  … ihren ruf für mich aufs spiel setzen.  … mir als referenz für einen potenziellen neuen arbeitgeber dienen. block 5 ich kenne auf facebook…  … jemanden, an den ich mich im notfall wenden könnte, um mir 500 euro zu leihen.  …niemanden so gut, dass ich ihn etwas für mich wichtiges machen lassen würde. c soziodemographie ii sa1 welches ist der höchste von ihnen erreichte schulabschluss?  bin noch in der schulausbildung  hauptschule/volksschule ohne abgeschlossene lehre (8-klassige schule)  hauptschule/volksschule mit abgeschlossener lehre (8-klassige schule)  mittel-/real-/fach-/handelsschule ohne abitur (10-klassige polytechnische oberschule)  abitur/hochschulreife (12-klassige erweiterte oberschule)  abgeschlossenes studium (universität, fachhochschule, akademie) australasian journal of information systems grottke, hacker & durst 2018, vol 22, research on recent advances in social media factors determining online social capital? 30 sa2 welche der folgenden aussagen trifft auf ihre derzeitige berufstätigkeit/ ausbildung zu? ich bin...  voll berufstätig  teilweise berufstätig  vorübergehend nicht berufstätig/ arbeitslos  nicht (mehr) berufstätig (in rente/ pension/ hausfrau/ hausmann)  in berufsausbildung/ lehre/ wehrpflicht/ zivildienst/ in schulausbildung (schüler)  in hochschulausbildung (student)  nicht berufstätig und nie berufstätig gewesen australasian journal of information systems grottke, hacker & durst 2018, vol 22, research on recent advances in social media factors determining online social capital? 31 appendix 2. english version of the online questionnaire a sociodemographic information i s1 first, we would like to ask you to give some information about yourself. are you male or female?  male  female s2 how old are you? (please fill in your age in years)  __ years s3 do you have a facebook account?  yes  no s4 snapshot of your facebook network for this study, we need a snapshot of your facebook network. to give us access to your facebook network, please click [next] and accept the permission requests of the application. b internet usage and facebook 1. general information i1 in this section, we will ask you to provide information about your internet and facebook usage. on average, how much time, approximately, do you spend on the internet per day?  less than 10 minutes  11-30 minutes  31-60 minutes  1-2 hours  2-3 hours  more than 3 hours australasian journal of information systems grottke, hacker & durst 2018, vol 22, research on recent advances in social media factors determining online social capital? 32 i2 for how long have you been a member of the facebook community?  6 months or less  6 months – 1 year  1-2 years  2-3 years  3-4 years  more than 4 years i3 on average, how much time, approximately, do you spend on facebook per day?  less than 10 minutes  11-30 minutes  31-60 minutes  1-2 hours  2-3 hours  more than 3 hours i4 in the following section, we are interested in your attitude towards facebook. for each statement, please select the most suitable answer from your point of view. answers range from “strongly disagree” (1) to “strongly agree” (5).  facebook is part of my everyday activity.  i am proud to tell people i’m on facebook.  facebook has become part of my daily routine.  i feel out of touch when i haven’t logged onto facebook for a while.  i feel i am part of the facebook community.  i would be sorry if facebook shut down. 2. network of friends on facebook i5 with the following questions, we would like to find out a bit more about your facebook friends. how many facebook friends do you currently have?  __ facebook friends australasian journal of information systems grottke, hacker & durst 2018, vol 22, research on recent advances in social media factors determining online social capital? 33 i6 think of your network of facebook friends. how many of your facebook friends did you meet online?  none  a few  about half  most  all  (no answer) i7 now, please think of your real-life friends. with how many of your real-life friends are you connected on facebook?  none  a few  about half  most  all  (no answer) i8 think of your network of facebook friends. how many of your facebook friends do you consider close friends?  none  a few  about half  most  all  (no answer) i9 think of the persons you are connected with on facebook. how many of your facebook friends share your mother tongue?  none  a few  about half  most  all  (no answer) australasian journal of information systems grottke, hacker & durst 2018, vol 22, research on recent advances in social media factors determining online social capital? 34 i10 how many of your facebook friends have an ethnic background that is similar to yours?  none  a few  about half  most  all  (no answer) i11 how many of your facebook friends have an educational background that is similar to yours?  none  a few  about half  most  all  (no answer) i12 how many of your facebook friends have a household income that is similar to yours?  none  a few  about half  most  all  (no answer) i13 in the following section, we will ask you questions about your network of facebook friends. answers range from “strongly disagree” (1) to “strongly agree” (5). for each question, please select the most appropriate answer. part 1  talking with people on facebook makes me curious about other places in the world.  i am willing to spend time to support general community activities on facebook.  on facebook, i come in contact with new people all the time. part 2 on facebook, there …  … are several people i trust to help solve my problems.  … is someone i can turn to for advice about making very important decisions.  … is no one that i feel comfortable talking to about intimate personal problems. australasian journal of information systems grottke, hacker & durst 2018, vol 22, research on recent advances in social media factors determining online social capital? 35  … are several people i can talk to when i feel lonely. part 3 interacting with people on facebook …  … makes me feel like part of a larger community.  … makes me feel connected to the bigger picture.  … reminds me that everyone in the world is connected.  … makes me interested in things that happen outside of my town.  … makes me want to try new things.  … makes me interested in what people unlike me are thinking.  … gives me new people to talk to. part 4 the people i interact with on facebook would …  … share their last dollar with me.  … help me fight an injustice.  … put their reputation on the line for me.  … be good job references for me. part 5  if i needed an emergency loan of $500, i know someone on facebook i can turn to.  i do not know people on facebook well enough to get them to do anything important. c sociodemographic information ii sa1 finally, we would like to know a bit about your educational background. what is your highest level of educational achievement?  i am still attending school.  secondary modern school (year 5 to year 9) without completed apprenticeship training  secondary modern school (year 5 to year 9) with completed apprenticeship training  secondary school (year 5 to year 10) without general qualification for university entrance  general qualification for university entrance  completed university degree (university, university of applied sciences, university of cooperative education) sa2 which of the following statements describes best your current occupation? i am …  … employed full-time  … employed part-time australasian journal of information systems grottke, hacker & durst 2018, vol 22, research on recent advances in social media factors determining online social capital? 36  … temporarily not employed / unemployed  … not employed anymore (e.g., retired)  … doing a vocational training / apprenticeship / military service / civilian service / attending school  … a university student  … not employed, and i have never been employed copyright: © 2018 grottke, hacker & durst. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. paper title (paper title style) australasian journal of information systems al-saggaf, burmeister & schwartz 2017, vol 21, research on applied ethics and ict qualifications and ethics education 1 qualifications and ethics education: the views of ict professionals yeslam al-saggaf charles sturt university yalsaggaf@csu.edu.au oliver k. burmeister charles sturt university michael schwartz rmit university abstract do information and communications technology (ict) professionals who have ict qualifications believe that the ethics education they received as part of their ict degrees helped them recognise ethical problems in the workplace and address them? if they do, are they also influenced by their personal ethics? what else helps them recognise ethical problems in the workplace and address them? and what are their views in relation to the impact of ethics education on professionalism in the ict workplace? a quantitative survey of 2,315 australian ict professionals revealed that participants who reported having various levels of qualifications found ethics education or training, to a small degree, helpful for recognising ethical problems and addressing them; although it is those with non-ict qualifications, not those with ict degrees, who were influenced more by ethics education or training. this suggests that educators need to consider how to better prepare ict graduates for the workplace challenges and the types of situations they subsequently experience. the survey also found that participants who reported having various levels of qualifications were not influenced by their personal ethics or indeed any other factor making ethics education or training important for developing professionalism. the quantitative survey was followed by qualitative interviews with 43 australian ict professionals in six australian capital cities. these interviews provided further empirical evidence that ethics education is crucial for enabling ict professionals to recognise ethical problems and resolve them and that educators need to consider how to better prepare ict graduates for the types of moral dilemmas that they are likely going to face in the workforce. keywords: professional ethics, personal ethics, it courses, industry certification and ethical awareness. 1 introduction professional ethics in business have been explored from various perspectives. often the failures are highlighted (bernoth, dietsch, burmeister, & schwartz, 2014; bull & mclean, 2007; gotterbarn & miller, 2009), although some literature also shows positive exemplars of professionalism (hedström, 2007; rogerson, weckert, & simpson, 2000). within ict there have been attempts to improve professional ethics, including a focus on values (burmeister, 2010, 2012; manders-huits, 2010; venable et al., 2011), codes of professional conduct (berleur et al., 2004; bowern, burmeister, gotterbarn, & weckert, 2006; burmeister, 2000; fawkes, 2014), as well as wider work on licensing, certification, adherence to international standards, and more (dodig-crnkovic & horniak, 2005; wsis+10, 2014). professional ethics in ict is part of a wider focus on business ethics which since its inception in the early 1980s has been plagued by disputes as to its soundness (drucker, 1981). indeed, some researchers have insisted that with regard to business ethics nothing “meaningful” (hill, 1995, p. 585) could ever be said due to a fundamental problem. namely, that we need to be able to first discuss ethics in order to be able to discuss business ethics: and discussing ethics required a meaningful moral discourse which remains highly problematic. following hill this australasian journal of information systems al-saggaf, burmeister & schwartz 2017, vol 21, research on applied ethics and ict qualifications and ethics education 2 was so as for many “moral words and propositions make no sense, or are meaningless” (1995, p. 588). this, hill argued, made it close to impossible “to talk about ethics” (1995, p. 589). in fact not only did it render such talk impossible. it also, he insisted, had a further consequence: it made any agreement as to the moral truth impossible. this underlying reality is revealed by the responses of our ict interviewees who often agitated for different initiatives in countering unprofessional behaviour. this is certainly heightened by the difference between what is taught to future ict professionals and the reality of what they ultimately experience in their workplaces. hill however identifies an even larger problem. moral truths he argued could not exist because there was no acceptance of any moral meaning. nonetheless, he argued that even if there was the possibility of a moral meaning, moral truth would still be difficult as in the normal course of events the verification of a truth relies on empirical evidence: but moral truths do not. moral truths rely on moral intuition and such “intuitions resist proof” (hill, 1995, p. 589). as a result different people can disagree – as our ict interviewees sometimes did – on a “moral issue precisely because they have opposing intuitions” (hill (1995, p. 589) italics in the original). this, hill argued, exposes the crux of our problem. the problem is that we live in an age in which there is little consensus. consequently both “moral meaning and truth are very problematic” (hill, 1995, p. 590). following hill business ethicists might engage in endless discussions about ethics, but regardless of that it is exceedingly difficult “to communicate a moral proposition that can be understood” (1995, p. 590). contemplating those circumstances hill speculated as to socrates being correct in arguing that “morality cannot be taught” (1995, p. 590) unless those studying it approached it with the correct “attitude” (1995, p. 591). how such an attitude might be engendered with regard to australian ict professionals does concern us in our paper. if we fail to do so we cannot help them identify ethical problems let alone solve them. that, however, returns us to the discrepancy we discussed earlier: that gap between what is taught and what is experienced. in such circumstances there is very little opportunity for any consensus to emerge. hill (1995) is not alone in having such misgivings as to the possibility of teaching business ethics. phillips and margolis (1999) also had major reservations as to using ethics to teach business ethics. however, for many the problem is not ethics per se but the actual variant of ethics with rule-based ethics the culprit. as a result alasdair macintyre is promoted by many business ethicists (amongst others see brewer (1997); horvath (1995); moore (2003)) who favour his championing of virtue ethics over utilitarian or deontological theories. that, however, might not be as simple as it sounds. the australian ict industry contains diverse bodies with some, as we discuss with regard to our interviewees, having very different priorities to others. identifying a commonly accepted virtue in such a diverse body might be every bit as difficult as agreeing on hill’s understandable moral proposition. macintyre sees virtues as “significantly defined by local social or cultural perspectives” (arthur & carr, 2013, p. 29). but australian ict professionals experience cultural differences, as we discuss in the paper, and macintyre himself has argued that given such circumstances “there can be no common moral education” (arthur & carr, 2013, p. 29). if that is so teaching business ethics to ict professionals as it is usually taught by virtue ethicists is increasingly futile. instead we should be seeking a way to help ict professionals identify ethical problems despite those circumstances. such an argument has conceivably been accepted by our leading educators (glanzer & ream, 2008). whilst past generations of educators thought the university should provide a moral education this is no longer the case which is why ict professionals increasingly grapple with this in their workplace (glanzer & ream, 2008). glanzer and ream (2008) quote contemporary leading educators as, contrary to the expectation of their predecessors, insisting that when it came to moral questions the faculty left it to the students to find their own answers as the faculty could not provide them. in making that argument they too invoke macintyre’s observation as to contemporary universities having less agreement than ever regarding “what it means to be fully human” (glanzer & ream, 2008, p. 114). as to why this was not so in the past, they explain in that prior period the existence and acceptance of a “metanarrative (by australasian journal of information systems al-saggaf, burmeister & schwartz 2017, vol 21, research on applied ethics and ict qualifications and ethics education 3 which) we mean a story” (glanzer & ream, 2008, p. 114). that story provided both understanding and guidance and, furthermore, knowledge as to those virtues which should be acquired. without such an acceptable metanarrative for ict professionals there can be no general consensus as to any of that. if we are serious about helping ict professionals identify ethical problems despite their circumstances much of our effort must be in creating for them a suitable metanarrative. in doing so we need to be most cognisant of the dissimilarity between what they are taught as to ethics and those ethical dilemmas ict professionals actually encounter in their workplaces. glanzer and ream (2008) do not mention hill (1995) but their arguments endorse his as to moral truths not being reliant on empirical evidence with the result that there is little consensus regarding them. hence “ethics became marginalised as a field of knowledge” (glanzer & ream, 2008, p. 115). such circumstances do not however negate morality. on the contrary all they do is stress the necessity of finding “some other basis for moral commonality” (glanzer & ream, 2008, p. 115). that is increasingly necessary for those in the ict profession. glanzer and hill (2013, p. 293) explain how the desire of the universities to provide practical courses led to “ethics largely disappear(ing) from the curriculum”. later it reappeared in the guise of “narrow professional ethics courses” (2013, p. 293) such as computer ethics but for the reasons explained by hill (1995) and also by arthur and carr (2013) such courses cannot readily be taught by means of utilitarian, deontological or virtue ethics as is unfortunately usually the case. for a discussion on why these three fail to do so, the reader is encouraged to read comer & schwartz (2015) careful explanation of the reason for their failure. criticisms of utilitarian, deontological and virtue ethics are not a recent development. ellerman (1998, p. 192) arguing that at times “moral aspiration is absurd” cites approvingly “nietzsche’s suspicion that morality is eminently dangerous” (1998, p. 191). friedrich nietzsche (1844-1900) described kant as “a moral fanatic” (nietzsche as quoted by russell (1972, p. 761)) and dismissed mill as a “blockhead” (nietzsche as quoted by russell (1972, p. 762)). but nietzsche, whilst he might have helped instigate a growing disenchantment with all those philosophies, lived in a time when most accepted them. as such for the vast majority the prevailing metanarrative held. it is the decline of any dominant metanarrative which is our major concern regarding the ability to improve that ethical awareness which ict professionals require to identify ethical problems. for an extremely long period whilst everything changed everything, paradoxically, remained much the same. the christians closed plato’s academy (rosenstand, 2006) but nonetheless appropriated much of greek philosophy for their own purposes (neusner & chilton, 1997). as such despite these changes, or perhaps regardless of them, a dominant story held sway and provided society with a metanarrative it could subscribe to and around which some consensus could be acquired. brown (1994, p. 37) has argued that our search for “objective knowledge, universal laws and absolute truths was a myth, legitimized by a high-level story-line, or metanarrative”. most significantly this metanarrative could vary but what was essential was that it be accepted. however, according to brown (1994, p. 37) this “potency of metanarratives has declined”: indeed, he disputes that there is any longer the acceptance of a metanarrative. that has dire implications for the teaching of morality to ict professionals for the very reasons which hill (1995) and arthur and carr (2013) explained. without the widespread acceptance of a metanarrative any ethical discussions are bound to be meaningless as there can be no underlying acceptance of any moral propositions. this is the fate of ethics subjects being taught in ict courses. and yet, whilst it is, that gap between what is taught in such subjects and the reality of ethical dilemmas experienced by ict professionals employed in the workplace provides the opportunity to create anew for such subjects a valid metanarrative: one that is firmly grounded in the ethical reality experienced by ict professionals. what is therefore essential is a study which highlights those ethical challenges that ict professionals experience in their workplaces. such ethical challenges are not going to be encountered by ict students while they are at university. until they experience the reality of their future workplaces this will merely be a story. but it is essential for that story to become the dominant story and in australasian journal of information systems al-saggaf, burmeister & schwartz 2017, vol 21, research on applied ethics and ict qualifications and ethics education 4 doing so to provide a valid metanarrative. without it regarding ethics nothing “meaningful” (hill, 1995, p. 585) could ever be discussed by them: and seeking to educate ict students as to the ethical dilemmas they will face in the workplace will be a largely hopeless endeavour. there is a long acknowledged link between the professions and ethical conduct. professionals have specific privileges and “as a corresponding item of such privileges, professionals are subject to restrictive ethical duties” (mullerat, 2013, p. 180). high profile failures of professionalism in business obfuscate an underlying change in society toward greater ethical behaviour in the workplace. cases like enron in the usa, phone tapping of celebrities and politicians by newspaper journalists in the uk, and more recently the 2014 systemic fraud amongst commonwealth bank financial advisors in australia, are magnets for the media and suggest that unprofessional behaviour continues unabated in society. the main contribution of this article is it highlights the inadequacy of what is taught to ict graduates prior to entering the workplace revealing the changes that need to take place in tertiary education, to better prepare ict graduates. having introduced ict professional ethics within the wider context of professionalism, this article next reviews prior work relevant to qualifications and ethics education. it then describes the mixed methods approach of the present study, which involved a quantitative survey of 2,315 australian ict professionals and qualitative interviews with 43 of them. the findings are discussed alongside each approach separately. 2 qualifications and ethics education most ict professionals work in organizations. far too often such organizations do not foster the morality of those within the organization but frustrate any such behaviour. comer and vega (2011) have argued that is so as within the organization employees face two different types of consequences. the one set of consequences are those that affect an employee personally. the other is when there are consequences not for that employee, but for others. comer and vega do not dispute that individuals are moral. but they argue that they will most probably discard their morality in two situations. the first situation is when the organization imposes situational pressures upon the employees so that if they heed their own morality they will incur major personal costs. as comer and vega explain “we all have a breaking point at which we jettison our morals” (2011, p. 28 – italics in the original). extreme situational pressures within the organization can easily push employees over that breaking point. the second situation is where “the moral intensity of the circumstances is relatively low” (comer and vega, 2011, p. 26 – italics in the original). this is those situations where should an employee breach their own morality the consequences for others will be minimal. again, comer and vega argue that in such circumstances employees will easily abandon their morality. conversely where the moral intensity is relatively high they will not readily do so. in support of their arguments comer and vega provide the case of an it specialist in a hospital who refuses “to change a patient’s computerised records” (2011, p. 28) although this it specialist had been dishonest earlier about things which she thought were of very minor consequence for others. comer and vega thus present two situations an employee will face. the first has to do entirely with the employee. heightened situational pressures affect only the wellbeing of that employee. but the second situation depends on the employee’s “perceptions of its collective importance” (2011, p. 28). they argue that this latter aspect impacts directly upon the former as employees “are more likely to do the right thing, even at personal expense, to the extent that (their) behaviour affects others” (2011, p. 29). but they caution that employees can overtime avoid doing what is morally right. they can do so by reducing dissonance by lowering their moral standards or by “reinterpreting the issue as less morally intense” (comer and vega, 2011, p. 31). furthermore, they can become “desensitized to the intensity of moral encounters” (comer and vega, 2011, p.31) by continuing to engage in immoral acts “of incrementally greater australasian journal of information systems al-saggaf, burmeister & schwartz 2017, vol 21, research on applied ethics and ict qualifications and ethics education 5 intensity” (comer and vega, 2011, p. 31). earlier we mentioned enron. for comer and vega enron is an example of a process of increasing desensitization where enron employees “grow accustomed to acting in ways (they) once thought (they) never would or could” (2011, p. 31). for those very reasons how students are educated as to their moral behaviour before entering the workplace is crucial. ict professionals were once students and regarding that there is evidence that a mismatch exists between the way in which tertiary students are prepared for the workplace challenges and the types of situations they subsequently experience (al-saggaf, burmeister, & weckert, 2015). as comer and vega (2011) argue their perceived moral intensity of such situations has ramifications for those involved. and their perceptions of that moral intensity will be influenced by the type of moral education they had. although there have been many attempts to improve ict ethics education (al-saggaf & burmeister, 2012; fleischmann, 2010; simpson, nevile, & burmeister, 2003), it appears that it has only had limited effect. a study of the perceptions of recent graduates, who had been working for at least 18 months, revealed that despite their course having a dedicated final year ethics subject, nothing they had studied had prepared them for the types of ethical dilemmas that they faced in their employment (burmeister & sharma, 2005). according to comer and vega whilst they might have been taught about ethics they had not been taught how in those circumstances “to become less vulnerable to organizational factors, and to act in concert with (their) moral values” (2011, p. 34) which they need to do if they are “to take ownership of (their) ethical choices” (2011, p. 34). an industry survey in 2006 involving 351 ict professionals, 30 of whom were then interviewed, found that the ethics components of ict courses had minimal effect on the professional behaviour of ict practitioners (lucas & weckert, 2008). the literature, of which there is relatively little on the specific topic of efficacy in ict (buchko & buchko, 2009) and in wider business education (hun-joon, 1998), generally supports this. there is some evidence too that formal qualifications in ict do not have a positive effect ethically. lucas and weckert (2008) found that those practitioners with ict qualifications were more likely to do what they were asked to do, even if they considered it ethically dubious. one possible reason for this is that ict education may be very narrow. perhaps there is a technical emphasis in ict qualifications that reflects a lack of a professional attitude. lucas and weckert (2008) did not distinguish between university, tafe or other qualifications. despite the fact that many (though not all) of those interviewed for that project expressed strong support for ethics education, the 2006 survey suggested that ethics education as part of an ict course appeared to have little positive effect on the ethical decision making or ethical behaviour of those working in the industry. one explanation for this is that a significant difference exists between the content of typical ethics courses and what practitioners see as the main ethical problems. a survey of the material covered in university courses showed an intersecting but substantially different set of issues (lucas & mason, 2008). the common topics covered were the standard ones of privacy, security, cybercrime, intellectual property, regulating commerce and free speech and a few others, with professional ethics being one topic but frequently left unspecified. a weakness of that research was that it was based on online descriptions of courses and contents of commonly used textbooks. the main ethical concerns of practitioners as expressed in surveys differed from the typical content of accredited university courses as found in a web examination of those courses and the contents of the most common texts used (lucas & mason, 2008). the 2006 survey of practitioners ranked privacy and intellectual property issues prominently but what could generally be described as professional issues, that is, issues where practitioners must make work-related decisions, dominated. high on the list were compromising quality to meet deadlines, unprofessional behaviour, making false promises and conflicts of interest. also significant were compromising functionality and requirements to meet deadlines. unprofessional behaviour also figured large in the extra comments section. typical concerns were blaming others for one’s own mistakes, poor team contributions, awarding contracts without due process, overpricing and under quoting of time, and overstating of skills. the issue of professionalism was also of major concern in the interviews and most of the same worries australasian journal of information systems al-saggaf, burmeister & schwartz 2017, vol 21, research on applied ethics and ict qualifications and ethics education 6 surfaced. a number of related new ones were also mentioned; of particular significance were responsibility and informed consent. building on lucas and weckert (2008) and lucas and mason (2008) 2006 survey, al-saggaf et al. (2015) conducted another survey in 2013 in which they asked australian ict professionals about the most common ethical problems that they faced in their workplaces and what are the most effective strategies for solving them. unlike the 2006 survey which had only 351 responses, and for which responses were gathered over a six month period, from july 10, 2006 to january 31, 2007, the 2013 survey involved 2,315 ict professionals and was only open for 2 months, from 12/9/2013 to 6/11/2013. that is, the 2013 result of 12.4% responses from professional members of the australian computer society (acs) compared to a 1.9% response rate in 2006, and over a significantly shorter time frame, suggests that there is much greater awareness and interest in the ict society about professionalism and ethics, now, compared to seven years ago. however, what has not changed is the type of ethical challenges faced in the ict workplace. indeed, the top ten problems across the seven year period were virtually identical (table 2 below provides more information). this raises the question: is it possible that seven years on that ethics education as part of ict courses continued to have little positive effect on the ethical decision making or ethical behaviour of those working in the industry? this article investigates using data from this 2013 survey, which ethical problems are actually experienced in the ict workplace, so that educators can be better informed about what needs to be taught to future ict students. the 2006 survey did not distinguish between university, tafe or other qualifications. the 2013 survey allowed participants to select their qualifications from a list of 10 options. the article then addresses the following questions: do information and communications technology (ict) professionals who have ict qualifications believe that the ethics education they received as part of their ict degrees helped them recognise ethical problems in the workplace and address them? if they do, are they also influenced by their personal ethics? what else helps them recognise ethical problems in the workplace and address them? and what are their views in relation to the impact of ethics education on professionalism in the ict workplace? 3 methodology and findings 3.1 the quantitative survey a mixed methods approach has been adopted for the project. three phases were involved, the first two of which are addressed herein. the first phase involved a quantitative survey of members of the acs, administered using surveymonkey.com, to allow the participants to fill the questionnaire and return it over the internet. the survey was informed by the results of a 2006 survey conducted by lucas and mason (2008) and also by the instrument they used. all acs members were invited to participate in the web-based survey by direct email sent to them by the acs once on 12 september 2013, and the survey was closed on 6 november. the online questionnaire was prefaced by the ethics consent statement and a description of the study. the questions comprised both closed and open ended questions. this article reports only on the closed ended questions. 3.1.1 sample (quantitative survey) a total of 2,315 participants completed the questionnaire. the average number of years of experience for all respondents was 19 years. table 1 shows a summary of the demographic information of all participants. it should be noted that with the exception of the qualifications variable, the independent variable in this study, for which all groups are shown and all percentages add up to 100, not all percentages add up to 100 in the case of the other variables as respondents’ choices with smaller percentages are not shown. australasian journal of information systems al-saggaf, burmeister & schwartz 2017, vol 21, research on applied ethics and ict qualifications and ethics education 7 demographic information participants gender male 84.5% (n=1940) female 15.5% (n=356) age 31 and 35 years 11% (n= 254) 41 and 45 years 12.4% (n=287) 51 and 55 years 14.1% (n=325) 61 years and above 12.2% (n= 282) state act 10% (n=247) nsw 30.4% (n=696) qld 12.2% (n=279) sa 5.5% (n=120) vic 25.4% (n=581) wa 9.5% (n=218) occupational category managers 33.8% (n=698) developers 14.8% (n=307) consultants 24.3% (n=502) technical support 13.3% (n=277) geographical location capital city or metropolitan 90.6% (n=2069) regional areas 9.4% (n=215) industry type education 12.3% (n= 244) finance 11.9% (n=235) government 15.4% (n=304) ict 34.2% (n=676) industry sector private sector 57.4% (n=1,282) public sector 27.5% (n=614) job classification permanent full-time employees 65.4% (n=1,388) independent consultants 5.2% (n= 110) self-employed 5.3% (n=112) qualifications non-ict degree or diploma 5.2% (n=114) experience 3.9% (n=85) industry or profession certification 6% (n=132) diploma 8.3% (n=181) degree 36.4% (n=794) certificate 2.7% (n=58) self-educated 0.8% (n=17) trade qualification 0.5% (n=10) higher degree 34.6% (n=756) no qualifications 1.6% (n=35) table 1: a summary of the demographic information of all participants 3.1.2 statistical analysis the 2013 survey asked participants to address a total of 57 problems, which extended the 2006 survey of 16 problems. the additional categories in the 2013 survey were based on the additional comments in the ‘other’ section of the open-ended questions in the 2006 survey (lucas & mason, 2008; lucas & weckert, 2008). table 2 lists the top 10 ethical problems along with the number of responses and their percentages. australasian journal of information systems al-saggaf, burmeister & schwartz 2017, vol 21, research on applied ethics and ict qualifications and ethics education 8 ethical problem number of responses percent of cases compromising quality to meet deadlines 1104 47.7 blaming others for own mistakes 957 41.4 compromising functionality to meet deadlines 846 36.6 overworking staff 762 32.9 incompetence 750 32.4 conflict of interest 682 29.5 unprofessional behaviour 633 27.4 compromising user requirements to meet deadlines 632 27.3 bullying 630 27.2 compromising security to meet deadlines or make things work 611 26.4 table 2: the top 10 ethical problems when asked to indicate their feeling towards the statement “my ethics education or training help me in identifying the ethical problems and solving them” 66.1% (n=1268) of the participants in the survey indicated that they agree or strongly agree with this statement and only 6.1% (n=117) disagree or strongly disagree with this statement. this suggests that the majority of the survey participants thought that ethics education is helpful for recognising ethical problems and addressing them. we wanted to determine if there were significant differences in agreement with the statement “my ethics education or training help me in identifying the ethical problems and solving them” among the participants who reported having various levels of qualifications. the qualifications variable was the independent variable and the statement about ethics education or training was the dependent variable. the independent variable was of type categorical comprising 10 levels ranging from ‘no qualifications’ to ‘higher degree’ but including a nonict degree or diploma; while agreement with the statement, the dependent variable, was assessed using a five-point likert scale ranging from strongly agree to strongly disagree. it should be noted that it is possible the participants in this study understood ethics education to a larger extent in terms of the curriculum and to a less extent in terms of the pedagogy and the roles of both the student and teacher in the learning process. given the independent variable is categorical and the dependent variable is assessed using a likert scale, an analysis of variance using a one-way between groups anova would have been the most appropriate test, especially that the homogeneity of variances assumption was tested and verified. however since the dependent variable is significantly positively skewed (i.e. did not display a normal distribution), this statistical tool was judged not to be suitable for the analysis; therefore we used kruskal wallis, the corresponding procedure for non-parametric dependent variable, to test the differences in agreement with this statement among the participants. prior to conducting the kruskal wallis, the assumption of homogeneity of variances for nonparametric tests was verified by: first, ranking the values of all the dependent variable responses; second, computing the ranks means; and then obtaining the absolute difference between the ranks and the ranks means (i.e. the two newly created variables). finally, an analysis of variance was performed on these absolute difference scores (in the context of this analysis, this procedure is called a non-parametric levene test). since levene’s p value was not significant, f(9, 1858)=1.01, p=.425, the assumption of homogeneity of variances was satisfied. 3.1.3 findings (quantitative survey) a significant difference in agreement with the statement was found among the participants who reported having different levels of qualifications, χ² (9, n = 1868) = 22.004, p = .009, ηp2 = .0118. that said, since only 1.18% of the variance in agreement with the statement was accounted for by the qualifications variable, the effect of this variable is very small. australasian journal of information systems al-saggaf, burmeister & schwartz 2017, vol 21, research on applied ethics and ict qualifications and ethics education 9 the mean ranks indicated that those with non-ict degree or diploma had the highest ranks (mean rank= 1116.07), with those with an ict degree ranked fifth form the top (mean rank = 944.87); while those without any qualifications were ranked tenth (lowest) (mean rank = 839.58). see table 3 for the mean ranks of all the qualification groups. please indicate your highest ict qualifications. n mean rank non-ict degree or diploma 99 1116.07 experience 68 1009.31 industry or profession certification 113 955.02 diploma 152 955.01 degree 685 944.87 certificate 47 941.89 self-educated 14 897.04 trade qualification 8 885.69 higher degree 656 884.86 no qualifications 26 839.58 total 1868 table 3: the mean ranks of all the qualification groups in order to evaluate if the groups with the highest mean ranks are statistically different from those with lowest mean ranks, the statistically significant kruskal wallis was followed-up with kruskal wallis pairwise analysis, while selecting only the cases (pairs) of interest. a mannwhitney would have been the preferred test for this situation, but since it does not show eta square it was not selected. a significant difference in agreement with the statement about ethics education was found between the participants who reported having non-ict degree or diploma and those who reported having no qualifications, χ² (1, n = 125) = 5.325, p = .021, ηp 2 = .0429. similarly, a significant difference in agreement with the statement was found between the participants who reported having non-ict degree or diploma and an ict degree, χ² (1, n = 784) = 9.837, p = .002, ηp 2 = .0125. this suggests that those with non-ict degree may have benefited more from their degree in terms of raising their ethics awareness than those with an ict degree or those who reported having no qualifications. it is not clear why this is the case but it could be because non-ict students have more exposure to the humanities than otherwise which allows them to engage with the ethics curriculum better than students who have only been exposed to technical-related curriculum. further research is needed to shed light on this unexpected finding. also a significant difference in agreement with the statement was found between the participants who reported having a non-ict degree or diploma and those who reported having an industry or professional certification, χ² (1, n = 212) = 5.073, p = .024, ηp 2 = 0.24. further research into the varieties of ict degrees and diplomas is required to discover which ones are more helpful in recognising ethical problems and addressing them than an industry or profession certification, in order to help all educators better understand how to prepare ict graduates for the workplace. but a significant difference in agreement with the statement was found between the participants who reported having an ict degree and those having a higher degree, χ² (1, n = 1341) = 4.730, p = .030, ηp 2 = 003, suggesting an ict degree could be more effective in this regard than a higher ict degree. again, further research into the varieties of ict degrees and diplomas is required to discover which ones are more helpful. interestingly, there is no significant difference in agreement with the statement between the participants who reported having an ict degree and those who reported not having any qualifications, χ² (1, n = 711) = 1.096, p = .295; and no significant difference in agreement with the statement was found australasian journal of information systems al-saggaf, burmeister & schwartz 2017, vol 21, research on applied ethics and ict qualifications and ethics education 10 between the participants who reported having an ict degree and those who reported having an industry or profession certification, χ² (1, n = 798) = .041, p = .840. this again hints at the inadequacy of the ict courses in developing ethics awareness among graduands. table 4 summarises these findings. qualifications a significant difference non-ict degree or diploma* no qualifications yes non-ict degree or diploma* ict degree yes non-ict degree or diploma* industry or profession certification yes ict degree* higher ict degree yes ict degree no qualifications no ict degree industry or profession certification no * more helpful table 4: summary of findings relating to agreement with ethics education or training of particular interest to this study is the statement “my personal ethics helps me in identifying the ethical problems and solving them” (see table 5 for the list of the nine statements assessed in the survey). the reason for this interest is because if those with qualifications believed that their personal ethics was sufficient in identifying the ethical problems and solving them, it may mean that their ethics education made little difference to their professional lives. to investigate whether or not there is a difference in agreement with this statement among the participants with various qualification levels, a kruskal wallis analysis was employed. prior to running the kruskal wallis tests, the assumption of homogeneity of variances was verified and tested using the non-parametric levene test. the analysis revealed that there is no significant difference in agreement with the statement among the participants, χ² (9, n = 1879) = 9.995, p = .351. this indicates that qualifications did not account for any variance in agreement with the statement about personal ethics suggesting personal ethics did not influence participants with qualifications as ethics education did. this suggests that those with qualifications did not believe that their personal ethics was sufficient in identifying the ethical problems and solving them. to investigate if the difference in agreement with ethics education or training that was found among the participants who reported having various levels of qualifications is not unique to this dependent variable, additional kruskal wallis analyses were performed on the remaining seven statements. prior to running the kruskal wallis tests, the assumption of homogeneity of variances was verified and tested using the non-parametric levene test. the series of kruskal wallis analyses found no significant difference in agreement with the remaining statements among the participants who reported having different levels of qualifications. this indicates that other than ethics education, qualifications did not account for any variance in agreement within the remaining statements. this suggests that only ethics education was found helpful for recognising ethical problems and addressing them among the participants who reported having qualifications. but it was non-ict qualifications that made the bigger difference; not the ict qualifications. australasian journal of information systems al-saggaf, burmeister & schwartz 2017, vol 21, research on applied ethics and ict qualifications and ethics education 11 # statement 1 the code of ethics of my professional body, such as the acs and sage, helps me in identifying the ethical problems and solving them. 2 my personal ethics help me in identifying the ethical problems and solving them. 3 my company policies help me in identifying the ethical problems and solving them. 4 my company’s culture helps me in identifying the ethical problems and solving them. 5 my company’s ethics committee helps me in identifying the ethical problems and solving them. 6 my company’s mission/ value statement helps me in identifying the ethical problems and solving them. 7 my ethics education or training help me in identifying the ethical problems and solving them. 8 my industry licences or certification help me in identifying the ethical problems and solving them. 9 the industry standard i am adopting, such as cobit and itil, helps me in identifying the ethical problems and solving them. table 5 the nine statements assessed in the survey 3.2 the qualitative interviews the second phase of the study involved a set of semi-structured, in-depth interviews with 43 participants selected from those who responded to the first phase. the interviews were conducted during the month of february 2014 and took place in six australian capital cities. the purpose of these follow-up interviews was to discover participants’ perceptions in regards to the nature of the ethical challenges experienced in the ict workplace and how exactly these problems are often solved. purposive sampling was adopted to select the participants from those who had indicated a willingness to be interviewed. purposive sampling allowed the researchers to choose cases that were representative of all sub-groups and personal characteristics which might be of interest to the study (patton, 2002). the sample drawn included professionals from a range of ict organisations, both large and small, government and private sector, representing different geographic locations, ages, gender, types of jobs, and employment experience. all the interviews were tape recorded and transcribed verbatim. the transcribed interviews were analysed using thematic analyses. data analysis was completed using qsr nvivo 10. while the first author completed the data analysis, the second author reviewed the outcome of this analysis. this was to address bias that may arise from the involvement of a single researcher. free nodes, those nodes not organised or grouped, were created based on keywords in the interview documents. similar text within the interview documents was located and assigned to these nodes. these nodes then acted as 'buckets' in the sense that they held all the data related to a specific node. at the end of the creation of the free nodes theses free nodes were further divided into tree nodes. that is, broader categories were developed to group the free nodes. this was to create a hierarchy that made it easy to make sense of the data and facilitate interpretation. 3.2.1 findings (qualitative interviews) when asked during the semi-structured interviews for their views about effective strategies for dealing with unprofessional behaviour, 22 interviewees out of 43 suggested ethics education or incorporating ethics in the it courses. the following comments typify these semi-structured interview participants’ views: “so going back to say the acs and the universities, there ought to be an ethics segment in education” (43, consultant, sydney); “maybe the only thing that will, is better education that might stop people from doing ...” (69, professional salesman, sydney); “ethics being taught as part of it courses … some ethics component in every course and not just done as a oh and by the way here’s this ...” (49, public servant, canberra); “it’s education and training in ethics at schools and universities” (48, company director, sydney); “there will always need to be an education component because ethics is not something we’re really touching in our, in the higher education” (32, software tester, canberra); “we’re always checking to see that the universities actually teach ict ethics in some way that’s acceptable, that’s reasonable” (50, course accreditation advisor, canberra); “if the uni had had this e-learning thing from the acs or whomever and included it in one or more of the finishing units, i think it would’ve been really good” (24, business analyst, sydney); “at the australasian journal of information systems al-saggaf, burmeister & schwartz 2017, vol 21, research on applied ethics and ict qualifications and ethics education 12 bottom level i would say that ethics and humanities need to be included in our courseware” (54, consultant, perth). the findings from the interviews are consistent with the findings from the survey in two ways: first, that ethics education is crucial for enabling ict professionals to recognise ethical problems and resolve them. second, that educators need to consider how to better prepare ict graduates for the types of moral dilemmas that they are likely going to face in the workforce. the following quotation from interviewee 29 (48, company director, sydney) hints to this mismatch: you know when you do your it degree the technology that they teach you at the beginning is redundant by the time you finish your degree. what you learn is the soft skills, the professional skills, how to write quality code, how to apply technology to business, how to do analysis, how to do design, how to architect. those principles don’t change yeah? the technology you use is redundant by the time you finish your degree but degrees are structured around teaching technology and all these things come secondary; no, it’s the other way around. interviewee 29 (48, company director, sydney) also adds: jobs that are going to be created in three years’ time don’t exist in now – app developers didn’t exist four years ago – i mean there was like 10 guys on the planet did it; now there’s thousands of them. they – yeah okay they need a little bit of applecoding language skills, that sort of thing. but what they need is they need the ability to recognise a problem ... so doing courses that do teach you ethics, but teach you generic capacities and capabilities that you can transfer into multiple areas of society is what’s needed and a lot of the courses aren’t structured that way. indeed, given the pace at which technology advances, it is not unusual for a particular technology to be obsolete by the time a student completes a course. thus it is important, as this interviewee argues, that it courses prepare students adequately for the unknown and uncertain future and also teach them to be ethical. the problem is that some university academics are not serious about ethics education or that they pay lip service to this area. interviewee 24 (43, it manager, melbourne), for example, said “my it course granted long time ago right – didn’t teach me much at all about ethics and it really didn’t”. interviewee 25 (31, it manager, melbourne) concurs adding that “when i did my degree years ago, i don’t recall there being anything in the degree about ethics in the it world”. interviewee 41 (50, course accreditation, canberra) provides evidence of this careless attitude towards ethics in it courses: it depends on how they do it – if they do it in the unit outline and it’s against the acs … then you can probably pick it up. but quite often they do it separately from that, and they claim all this stuff. you go back and look at the unit and there’s not a trace of it. and so, they’re pretty cheeky. … they just don’t know what’s going on, or they gave it to someone who doesn’t care or ... according to interviewee 41 one reason for this problem is that there is no one qualified to teach it ethics: i think one of the problems is that the teaching of ethics in ict courses – well, there’s no one there qualified to do it – or it’s very unusual, that is, because if you do an ict degree … hardly say that when you graduated you are now qualified to teach ethics. and when you’ve finished your phd you’re no better qualified ... to do it – to teach it. the exemplary quotations above provide further empirical evidence for the inadequacy of the current ict courses in preparing the ict graduates prior to entering the workplace. from this evidence it is clear that changes need to take place in tertiary education, to better prepare ict graduates. australasian journal of information systems al-saggaf, burmeister & schwartz 2017, vol 21, research on applied ethics and ict qualifications and ethics education 13 4 conclusion findings from both, the survey and the interviews, found ethics education to be the way forward if ict professionals are to recognise ethical problems in the workplace and solve them. but the findings also suggested that educators need to consider how to better prepare ict graduates for the workplace challenges and the types of situations they subsequently experience. further work is needed to explore how tertiary educators can develop ict courses that better prepare graduates for the kind of challenges they will confront in the workplace. lucas and weckert (2008) and lucas and mason (2008) found that there is a mismatch between what is taught and what is experienced in the workplace and that ethics education only marginally assisted ict employees with making ethical decisions. they also found that those with formal ethics education were only slightly less likely to rely on their personal ethics when making ethical decisions in the workplace. our findings are consistent with their findings regarding ict courses making little difference to the ict employees’ professional development and that ethics education only marginally assisted ict employees with making ethical decisions. but our findings contradict theirs regarding personal ethics. personal ethics was not invoked by those who reported having various levels of qualifications and ethics education was found to be helpful in identifying ethical challenges and addressing them. further research could shed light on the effect of qualification on the adoption of ethics education for institutions. similarly, the implication for tertiary educators also extend to reconsideration of graduate attributes. further work is needed to highlight which aspects of professionalism should be embedded in ict courses. lucas and weckert (2008) and lucas and mason (2008) have shown that there is mismatch between what is taught and what is experienced by ict professionals. this study has highlighted the real ethical challenges that face ict professionals in the ict workplace. therefore the work of the present study to reveal the common challenges and strategies for solving them is an important contribution to the literature, because it informs educators about the types of situations for which they need to prepare their students. given that 2,315 survey respondents provided input about these challenges, making this study the largest of its kind in australia, educators may want to consult the findings of this study when updating the contents of their textbooks and educational materials. in a rapidly changing environment textbooks and educational materials should be robust. the 12.4% survey response rate for the present study contrasts significantly with the 1.9% achieved in 2006, showing that over the intervening seven years there has been a significant shift within the ict society towards greater professionalism and ethical behaviour. that shift is part of a wider shift taking place in business, see ferrero and sison (2014), and reflects that society as a whole, not just in australia, but around the world is more concerned nowadays with issues around public trust, reliability of consumer products and service quality. however, that shift has to date not been reflected in the education of students preparing to enter the workforce, but this may be changing as discussed by may, luth and schwoerer (2014). acknowledgments the research reported here is supported by an australian research council linkage grant (lp130100808), for which the industry partner is the australian computer society. other team investigators were professor john weckert and mr john ridge. references al-saggaf, y., & burmeister, o. k. (2012). improving skill development: an exploratory study comparing a philosophical and an applied ethical analysis technique. computer science education, 22(3), 1-19. al-saggaf, y., burmeister, o. k., & weckert, j. (2015). reasons behind unethical behaviour in the australian ict workplace: an empirical investigation. journal of information, communication & ethics in society, 13(3/4), 235-255. australasian journal of information systems al-saggaf, burmeister & schwartz 2017, vol 21, research on applied ethics and ict qualifications and ethics education 14 arthur, j., & carr, d. (2013). character in learning for life: a virtue-ethical rationale for recent research on moral and values education. journal of beliefs & values, 34(1), 26-35. doi: 10.1080/13617672.2013.759343 berleur, j., duquenoy, p., holvast, j., jones, m., kimppa, k., sizer, r., & whitehouse, d. (2004). criteria and procedures for developing codes of ethics or of conduct (to promote discussion inside the ifip national societies), on behalf of ifip-sig9.2.2. laxenburg, austria. bernoth, m., dietsch, e., burmeister, o. k., & schwartz, m. (2014). information management in aged care: cases of confidentiality and elder abuse. journal of business ethics, 122, 453-460. doi: 10.1007/s10551-013-1770-7 bowern, m., burmeister, o. k., gotterbarn, d., & weckert, j. (2006). ict integrity: bringing the acs code of ethics up to date. australasian journal of information systems, 13(2), 168-181. brewer, k. b. (1997). management as a practice: a response to alasdair macintyre. journal of business ethics, 16(8), 825-833. doi: 10.1023/a:1017997200200 brown, s. (1994). marketing as multiplex: screening postmodernism. european journal of marketing, 28(8/9), 27-51. doi: doi:10.1108/03090569410067631 buchko, a. a., & buchko, k. j. (2009). so we teach business ethics—do they learn? journal of business ethics education, 6, 119-146. bull, c. m., & mclean, r. (2007). computer and information ethics: the neglect of moral issues associated with e-commerce. burmeister, o. k. (2000). applying the acs code of ethics. journal of research and practice in information technology, 32(2), 107-120. burmeister, o. k. (2010). websites for seniors: cognitive accessibility. international journal of emerging technologies and society, 8(2), 99-113. burmeister, o. k. (2012). what seniors value about online community. journal of community informatics, 8(1). burmeister, o. k., & sharma, a. (2005). professionalism in ict: meeting the challenge of ethical dilemmas in the workplace. paper presented at the fourth australian institute of computer ethics conference, deakin university, geelong. comer d. r. & schwartz, m. (2015) highlighting moral courage in the business ethics course. journal of business ethics, doi: 10.1007/s10551-015-2919-3 comer, d.r. and vega, g. (2011). moral courage in organizations: doing the right thing at work. m.e. sharpe, inc., armonk, ny. dodig-crnkovic, g., & horniak, v. (2005). good to have someone watching us from a distance? privacy vs. security at the workplace. paper presented at the proceedings of the sixth international conference of computer ethics: philosophical enquiry, cepe 200, july 17-19, 2005, university of twente, enschede, the netherlands conference retrieved from drucker, p. f. (1981). what is ‘business ethics’? the public interest, 63, 18-36. ellerman, c. p. (1998). a talent for life. the journal of value inquiry, 32(2), 191-204. doi: 10.1023/a:1004269911054 fawkes, j. (2014). trust me, i’m a professional public relations ethics and professionalism: the shadow of excellence (pp. 51-76). london & new york: routledge. ferrero, i., & sison, a. j. g. (2014). a quantitative analysis of authors, schools and themes in virtue ethics articles in business ethics and management journals (1980–2011). business ethics: a european review, 23(4), 375-400. doi: 10.1111/beer.12057 australasian journal of information systems al-saggaf, burmeister & schwartz 2017, vol 21, research on applied ethics and ict qualifications and ethics education 15 fleischmann, k. r. (2010). preaching what we practice: teaching ethical decision-making to computer security professionals. lecture notes in computer science, 6054, 197-202. glanzer, p. l., & hill, j. p. (2013). why most american universities have given up on human purpose and meaning: a critical exploration of the historical story. journal of beliefs & values, 34(3), 289-299. doi: 10.1080/13617672.2013.828951 glanzer, p. l., & ream, t. c. (2008). addressing the moral quandary of contemporary universities: rejecting a less than human moral education. journal of beliefs & values, 29(2), 113-123. doi: 10.1080/13617670802289494 gotterbarn, d., & miller, k. (2009). the public is the priority: making decisions using the software engineering code of ethics. ieee computer, 42(6), 66-73. hedström, k. (2007). the values of it in elderly care. information technology and people, 20(1). hill, j. (1995). can we talk about ethics anymore? journal of business ethics, 14(8), 585592. horvath, c. m. (1995). excellence v. effectiveness: macintyre's critique of business. business ethics quarterly, 5(3), 499-532. hun-joon, p. (1998). can business ethics be taught?: a new model of business ethics education. journal of business ethics, 17(9/10), 965-977. lucas, r., & mason, n. (2008). a survey of ethics and regulation within the ict industry in australia: ethics education. journal of information, communication and ethics in society, 6(4), 349-363. doi: doi:10.1108/14779960810921141 lucas, r., & weckert, j. (2008). regulation in the ict industry. canberra: centre for applied philosophy and public ethics, australian national university. manders-huits, n. (2010). what values in design? a critical analysis of value-sensitive design. science and engineering ethics, 17(2), 271–287. macintyre, m. (1984). after virtue, university of notre dame press, notre dame, ind. may, d. r., luth, m. t., & schwoerer, c. e. (2014). the influence of business ethics education on moral efficacy, moral meaningfulness, and moral courage: a quasi-experimental study. journal of business ethics, 124(1), 67-80. doi: 10.1007/s10551-013-1860-6 moore, g. (2003). hives and horseshoes, mintzberg or macintyre: what future for corporate social responsibility? business ethics: a european review, 12(1), 41-53. doi: 10.1111/1467-8608.00304 mullerat, r. (2013). professional ethics, what for . . . ? ramon llull journal of applied ethics, 4, 173-199. neusner, j., & chilton, b. (1997). the intellectual foundations of christian and jewish discourse: the philosophy of religious argument. london: routledge. patton, m. q. (2002). qualitative research and evaluation methods (3 ed.). thousand oaks, ca: sage. phillips, r. a., & margolis, j. d. (1999). toward an ethics of organizations. business ethics quarterly, 9(4), 619-638. rogerson, s., weckert, j., & simpson, c. (2000). an ethical review of information systems development – the australian computer society’s code of ethics and ssadm. information technology and people, 13(2), 121-136. doi: doi:10.1108/09593840010339853 rosenstand, n. (2006). the moral of the story: an introduction to ethics. new york: mcgraw-hill. australasian journal of information systems al-saggaf, burmeister & schwartz 2017, vol 21, research on applied ethics and ict qualifications and ethics education 16 russell, b. (1972). a history of western philosophy. new york: simon & schuster. simpson, c., nevile, l., & burmeister, o. k. (2003). doing ethics: a universal technique in an accessibility context. australasian journal of information systems, 10(2). venable, j. r., pries‐heje, j., bunker, d., russo, n. l., johri, a., & nair, s. (2011). the role of design values in information system development for human benefit. information technology and people, 24(3), 281-302. doi: doi:10.1108/09593841111158383 wsis+10. (2014). wsis+10 high-level event: open consultation process multistakeholder preparatory platform, draft final report. copyright: © 2017 al-saggaf, burmeister & schwartz. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ http://creativecommons.org/licenses/by-nc/3.0/au/ microsoft word ajis_18(3)_v10.docx australasian journal of information systems volume 18 number 3 2014 289 ict interventions for girls: factors influencing ict career intentions elena gorbacheva european research center for information systems, university of muenster, germany elena.gorbacheva@ercis.uni-muenster.de annemieke craig school of information and business analytics deakin university geelong, australia annemieke.craig@deakin.edu.au jenine beekhuyzen deakin university, australia jenine.beekhuyzen@deakin.edu.au jo coldwell-neilson school of information technology deakin university geelong, australia jo.neilson@deakin.edu.au abstract intervention programs aimed at promoting study and work opportunities in the information and communications technology (ict) field to schoolgirls have been encouraged to combat a decline in the interest among girls to study ict at school. the goal of our study is to investigate the influence of such interventions on schoolgirls’ intentions to choose a career in the ict field by analysing comprehensive survey data (n = 3711), collected during four interventions in australia, using the partial least squares method. our study is also aimed at identifying other factors influencing ict career intentions. we found that the attitude towards interventions has an indirect influence on ict career intentions by affecting interest in ict. our results also challenge several existing theoretical studies by showing that factors that had previously been suggested as influencers were found to have little or no impact in this study, these being same-sex education and computer usage. keywords: intervention, girls and ict, gender diversity. introduction information and communication technology (ict) is an umbrella term encompassing computing technology in a wide ranging continuum from engineering to business. ict incorporates the disciplines of computer science (cs) and information systems (is). according to the united nations commission on the status of women, mainstreaming a gender perspective in technology and innovation enhances social and economic equity (united nations 2011). they argue while efforts are being made to expand access to ict “far less attention is being paid to the extent to which women and gender concerns are shaping the regulatory and policy environments that will ultimately determine the utility and relevance of these technologies. the strategic challenge today is to ensure not only that both women and men benefit from the opportunities presented by new ict, but also that new ict are used to support greater socioeconomic, scientific and political equality” (unesco 2007). australasian journal of information systems volume 18 number 3 2014 290 practically, fewer than one-third of the vacant computing jobs expected by 2018 in the united states (u.s.) will be filled by u.s. graduates with computing degrees, which is partly due to the decreasing number of girls studying an ict-related subject at school (blithe 2010). even with unprecedented demand for qualified computing workers, there has not been a significant uptake of females to fill this need. from an academic perspective, it is also argued that recruitment and retention of women in the ict industry is particularly challenging (trauth et al. 2009). the call from the united nations related to the lack of female participation in ict in most developed countries is difficult to ignore by educators and researchers. to combat the decline in interest among girls in studying ict at school, and ultimately at university, intervention programs for girls in ict have been encouraged on a number of levels. the goal of our study is to investigate the influence of interventions on schoolgirls’ intentions to choose a career in the ict field and to understand other motivating factors. this leads to the following research questions (rqs): rq1: do ict intervention programs have an influence on ict career intentions? rq2: what other factors influence schoolgirls’ ict career intentions? in order to answer these questions a quantitative research study was carried out using comprehensive survey data (n = 3711). we could not find a well-established theoretical framework on the factors influencing girls’ ict career intentions, but there are several conceptual studies assuming a strong influence of career intentions on the actual career choice (e.g. heinze and hu 2009). as a starting point the model on the factors influencing girls’ career choices by adya and kaiser (2005) was used. the data was collected during four “go girl, go for it” events in australia in 2006, 2008, 2010 and 2012 (events). a model showing the influence of interventions on ict career intentions was developed and tested using partial least squares (pls) regression (marcoulides et al. 2009). this paper begins with a synthesis of previous research on successful interventions and an overview of studies on ict career intentions and career choice. introduction of the developed model showing the role of interventions in ict career intentions is followed by the description of the applied research methodology and presentation of the results. in the concluding sections the study contribution, limitations and propositions for future research are discussed. background although the trend across tertiary education level shows a general increase in the number of highly educated females, in specific sectors such as cs and is, female graduates are significantly outnumbered by male graduates (european commission 2008). the average percentage of female graduates in the science and technology field in europe is 33% (european commission 2012). similar trends are evident in australia. not only is female participation in computing education dropping, participation generally is also reducing. the australian computer society indicates that the number of students completing ict post-secondary qualifications has dropped by almost 30% from approximately 19,000 completions in 2003 to just under 13,500 in 2010 (australian computer society 2012). adding to the concern are the number of females graduating with computing skills from secondary education; this number in victoria, australia was down dramatically from 7,315 in 2001 to 790 in 2011 (vcaa statistics 2013). only 96 female students (and 1208 male students) successfully completed software development at senior secondary level (vcaa statistics 2013). as mentioned in the previous section, interventions are one possible means to tackle this challenge. while there are intervention programs aimed at women (panteli 2012), this paper focuses on intervention programs aimed at secondary school girls. there are strong claims that middle school is a critical period for getting girls interested in computing. according to quesenberry and trauth (2012), “in creating interventions to increase the representation of women in the ict profession, it is crucial to australasian journal of information systems volume 18 number 3 2014 291 develop those that influence young girls’ perceptions of careers in the ict workforce.” however, denner (2011) argues that there is little research to guide the development of interventions. craig (2010) suggests that there is little evidence in the literature of “which interventions work best for whom” as many are implemented by volunteers who are more concerned with conducting the actual event rather than with formally evaluating the program or disseminating findings. bravo and colleagues (2003) provide some relevant insights into successful interventions, identifying a number of essential requirements based on their aim to change computing classroom behaviour to be a more welcoming environment for girls. to complement this list, allison and cossette (2007) present a summary of stem (science, technology, engineering, and mathematics) recruitment strategies discussed in the literature and there are many similarities with the conceptualisation by bravo and colleagues (2003). in doing a comparison of what makes a successful intervention there are important differences to highlight. allison and cossette (2007) argue that there is a need to create a positive and non-aggressive environment, to emphasize family and parental roles in building confidence, and encouraging mentoring relationships; these elements are not in the conceptualization by bravo et al. (2003) who, however, identify one additional element which relates to deconstructing the essentialist view of a “natural” attraction to computers. this paper proposes that all of these elements can help to create a successful intervention and they are purposefully planned into the design of the interventions discussed in this study (events) in a variety of ways. thus, the ongoing design of the events is continually guided by theory and practice relevant to gender and technology. a number of successful interventions have been reported in the academic literature. in the u.s. the interventions based at harvey mudd college and georgia tech university are well-known among gender and computing researchers. a small liberal-arts college focusing on science and engineering, harvey mudd college has increased the percentage of women students in the cs major from an average near 12% to around 40% through a variety of interventions. part of their strategy was to encourage students to attend summer research experiences (alvarado et al. 2012). the “georgia computes!” intervention program targets females at different stages of the “pipeline”; they also invite secondary school students to attend summer computing camps. they report that these activities have been “overwhelmingly positive” (bruckman et al. 2009). on evaluating the program, they found statistically significant improvements in students’ attitudes about computing in response to survey questions like “computer jobs are boring”, “girls can do computing,” and “programming is hard.” their intervention consisted of 13 events with both pre-surveys and post-surveys, and 7 of the workshops had a statistically significant improvement in participants’ attitudes about computing. intervention strategies to increase the participation of girls in computing in australia have also been documented in the academic literature. one example is the digital divas program which “aims to scaffold positive perceptions around computing in the early years of secondary school by involving female students in upbeat computing experiences over a semester” (fisher et al. 2009; lang et al. 2010). the curriculum for the intervention was designed to promote positive experiences in regard to selfefficacy and persistence with computing courses and career decisions. positive outcomes were reported related to the female-only environment, the presence of role models, and the encouragement of creativity in using technology. reports of successful interventions however may not necessarily be published in the academic literature even though a thorough evaluation may have been completed. one example of this is the intervention conducted by the national center for women & information technology (ncwit) in the u.s. which reported that their “computer mania day” positively influenced students’ general feelings toward technology, their feelings regarding the usefulness and utility of computers and technology, and their feelings regarding women’s involvement in technology. the study also found that students were more likely to take a computer course and consider a career in ict after participating in “computer mania day” (morrell et al. 2004). australasian journal of information systems volume 18 number 3 2014 292 as our study concentrates on schoolgirls who haven’t made their career choice yet, we can investigate only their intentions towards a future career in the ict field. several studies having a similar focus and also discussing the relationship between intentions and actual career choice were found in the literature: in the australian context one study identified the factors influencing girls in their decisions regarding computing and future careers. it was reported that interventions have a positive effect in a way that participating girls were more positive about careers in computing and were more likely to consider enrolling in university cs courses (anderson et al. 2008). outside of australia, a comparison study of african americans and anglo americans examining ict career intentions revealed that ict selfefficacy and occupational stereotypes were related to attitudes toward ict jobs, and that these attitudes were positively related to career intentions (johnson et al. 2008). from a psychological perspective, barnir et al. (2011) found that role models have a significant and positive impact on career intentions, and that gender moderated the effects and self-efficacy mediated them. another study of greek high school students’ intentions and motivation towards and against pursuing academic studies in cs found that girls are less likely than boys to pursue a cs degree, and when they do so, it is mainly because of extrinsic reasons rather than personal interest in cs. computer self-efficacy was found to be related to intention to study cs. the study found that a considerable proportion of greek students opt for studying cs, however girls constitute a minority giving credence to the phenomenon of the “shrinking pipeline” (camp 1997) according to the authors (papastergiou 2008). this study uses as a starting point the adya and kaiser's (2005) well cited model for girls’ career choices in technology fields in order to better understand girls’ intentions to undertake a career in ict. the model is based on literature to hypothesise about the future of the ict workforce. the literature stems from a variety of disciplines, including education, psychology, sociology, cs, is and business. the model identifies relevant factors influencing girls’ career choices, which include: • social factors: role models and gender stereotypes. • structural factors: technology resources and same-sex education. • individual differences (based on trauth's (2002) individual differences theory of gender and is). an early version of the “go girl, go for it” intervention discussed in this study was conducted in queensland, australia in 2000 (craig et al. 2008) with many such events conducted in various states since then. the event fits courtney et al.'s (2005) classification of interventions, but not neatly. the event lasts two days and is historically organised by an ict industry group, which is driven by government policy. the event is hosted and partially funded (mostly in-kind) by a local university, and volunteers participate as part of the organizing committee and as presenters on the day. the event is aimed at building awareness about ict career requirements and expectations, and the types of education, knowledge and skills required to work in the industry. the overall goal is to encourage participation of females in ict studies and careers. most of the sessions are presentations by industry professionals with powerpoint slides and video content. the event is attended by secondary schoolgirls between the ages of 13 and 17. girls attend the event either as a year-level group, a group from a computing class invited by their ict teacher (usually female), or they volunteer to attend as individuals with interest in the event. in any of these cases there needs to be interest in ict within the school environment for the desire to attend an event to be actioned. we know, from the victorian secondary school statistics cited above, that this is not the case with declining interest in ict in schools. therefore, if less computing classes are being offered at secondary school, the pool of possible participants to attend the events becomes smaller, which, unfortunately, seems to be the case and can be seen in the statistics of the events’ participants (please see the method and discussion sections for more details). australasian journal of information systems volume 18 number 3 2014 293 research model the data used in this study originates from a two-page questionnaire handed out to the girls attending the event10. it has to be mentioned that the main aim of the questionnaire was to collect feedback and evaluate the event. therefore, the questionnaire was not intended to test a certain model or build a theory. nevertheless, besides the attitude towards the event, the participants were asked several additional questions, e.g. on their intentions to learn more about ict and to work in the ict field in the future, on the frequency and purposes of computer usage, some background information including age and school attended etc. after having a closer look at the data available, we found enough questions to analyse ict career intention (e.g. whether ict was considered as a career option before the event and how the event affected it). as it was mentioned in the previous section, career intention has a strong relationship to career choice. therefore, we decided to use the adya and kaiser (2005) model on the factors influencing girls’ career choices as a starting point to map the information available to the model’s factors: attitude towards the event (event attitude) can be related to the model’s social factors: • one of the interventions’ main goals is to tackle gender stereotypes. • people (women or men) who give presentations at such interventions can act as role models for participating girls. information from responses to other questions can be mapped to structural factors: • the technology resources factor can be partially tested using the questions on frequency (amount of hours per week) and purposes of computer usage (computer usage). • the same-sex education factor can be tested using the information about the schools attended by the respondents. based on the propositions made in the adya and kaiser (2005) model and the information available in the questionnaire, some initial hypotheses were built on possible factors influencing ict career intention: h1: favourable event attitude has a positive influence on ict career intention h2: frequency of computer usage (computer usage) has a positive influence on ict career intention h3: usage of computers for building web sites or programming (computer usage) has a positive influence on ict career intention h4: sex-segregation of school environment (same-sex education) has a positive influence on ict career intention h5: interest in ict (ict interest) has a positive influence on ict career intention h6: intention to find out more about ict after the event (ict intention to learn) has a positive influence on ict career intention 10 please contact the authors for more information on the questionnaire used in the study. australasian journal of information systems volume 18 number 3 2014 294 in order to test whether any dependencies between these constructs exist at all, associations between all the variables were tested and, depending on a variable type, several association coefficients were calculated. phi coefficients ( ) were calculated for binary nominal variables (those derived from ‘yes/no’ questions) and the significance of their associations was checked with chi-square tests (cramér 1946). for ordinal variables (those having a clear ordering) spearman's correlation coefficients ( ) were calculated (e.g. lehman and rourke 2005). finally, in order to reveal an existence of association between nominal and ordinal variables kruskal–wallis method was used (kruskal and wallis 1952) and cramér's v ( c) measure of association was calculated (cramér 1946). we found that ict career intention has strong positive association with ict interest and ict intention to learn and moderate positive relationship with event attitude. moreover, we found that ict interest has strong significant associations with both event attitude and ict intention to learn (please see the results section for more details). no strong relationships were found between ict career intention and computer usage or same-sex education, which contradicts the propositions made in the adya and kaiser (2005) model. thus, hypotheses h2, h3, and h4 had to be rejected and not taken into consideration for building a research model. based on the results, we suggest that both ict interest and ict intention to learn influence ict career intention. we also assume that event attitude has an influence on ict interest, which, in turn affects ict intention to learn. our research model for further testing is presented in figure 1. ict career intentionevent attitude ict intention to learn ict interest h5 h6 h1 figure 1: research model method the questionnaire was adapted from a similar one used at another intervention and pre-tested by three people. in order to motivate girls to fill in a questionnaire, a show bag full of gifts provided by participating organizations was given in return for a completed questionnaire. this was a highly successful strategy and the absolute majority of the event participants took part in the survey. as a result, in total 4168 responses were received: 1668 in 2006, 1070 in 2008, 792 in 2010, and 638 in 2012. such a decrease in the number of responses after 2006 occurred since fewer girls were participating in the events. the two main reasons here are administrative issues (there was little chance to schedule the events in 2008-2012 to fit the schools calendar) and a decrease in the number of ict teachers who usually organised the trips (please see the discussion section for more details). in the questionnaire we asked both open-ended and closed-ended questions, but for this study we concentrated on the closed-ended questions in order to test the proposed hypotheses. australasian journal of information systems volume 18 number 3 2014 295 once initial responses were coded using ms excel, we first introduced a construct categorizing girls according to their ict career intention using spss 21 software. initially we considered using digital natives, digital immigrants and digital neglecters to represent the categories but realized that at least the first two terms are very well established in meaning and use. according to prensky's (2001) seminal work, digital natives are those born in the digital era and digital immigrants are those who have adopted technology. since the girls participating in the events are all digital natives (due to their age) this terminology was not useful. further investigation revealed the anderson et al. (2008) model which differentiates between those who take technology subjects at school and those who do not and called them takers and non takers respectively. the anderson model suggests that being a taker or a non taker directly affects future participation in ict professions. this proved to be a useful starting point. however, studying a subject at school does not necessarily imply intent to have a career in the area nor does the lack of subject imply a lack of interest in a future career in the area. for example, if a student is studying mathematics there is only a slim chance that they would also be considering a career as a mathematician. however, there are no school subjects which naturally lead or imply, in a similar way, an interest in a career in teaching. based on this initial analysis, we developed a new model based on intent to consider ict as a career option rather than on current study habits. the three categories are based on those proposed by anderson et al. (2008) and reflect the girls’ intent to consider an ict career before and after the event. these three categories are: 1. ict career current non takers (non takers): girls who did not consider ict as a career option neither before nor after the event. 2. ict career newcomers (newcomers): girls who did not consider ict as a career option before the event, but changed their minds during the event. 3. ict career takers (takers): girls who considered ict as a career option both before and after the event. we consider this group to be more robust in their ict career intention than newcomers. we then cleansed the data: the 320 cases containing missing data to any of the questions related to our target variable ict career intention were removed. a category of cases with positive responses on ict as a career option before the event, but negative – after the event (meaning that the event had a negative influence on ict career intention) was very low in comparison to the other three above-mentioned categories (n = 136). therefore, it was not taken into consideration in the current study and left for future investigation. moreover, inconsistent responses to several control questions were marked as missing and then for each case the total share of missing information was calculated. one case contained more than 35% of missing responses and was removed. the resulting study sample containing 3711 cases is examined in table 1. as mentioned in the previous section, the initial hypotheses were analysed employing association coefficients and spss 21 software. the developed model was further tested using the partial least squares (pls) method using smartpls 2.0 (ringle et al. 2005). the pls algorithm was chosen because it has no distribution assumptions for the measured variables and is a relatively flexible forecasting method. all constructs were modelled using reflective indicators. the analysed data (involved in the developed model) contains 13.2% missing values, which were treated using the mean replacement algorithm. results newcomers is the largest category in the sub-samples for each event and in the resulting sample; non takers is consistently the smallest category (see table 1). two conclusions arise from this statistics: australasian journal of information systems volume 18 number 3 2014 296 first, among secondary schoolgirls (at least those participating in interventions) there are very few who neglect ict, the absolute majority consider this field as favourable for future studies and career. second, the event motivated 40% of participating girls to start considering ict as a career option, which was not the case before the event. non takers newcomers takers total (year) year n % n % n % n % 2006 462 30.9% 540 36.1% 495 33.0% 1497 100 2008 251 26.9% 366 39.2% 317 33.9% 934 100 2010 124 17.3% 333 46.4% 260 36.3% 717 100 2012 108 19.2% 247 43.9% 208 36.9% 563 100 total 945 25.5% 1486 40.0% 1280 34.5% 3711 100 table 1. study sample and distribution of ict career intention categories the above-mentioned analysis of association coefficients between all variables has led to the following results: 1. event attitude has positive strong association with the ability of the event to bring a positive feeling about careers in ict (1a) ( c1 = 0.466) and a moderate association with the attitude towards presenters speaking about their experiences in ict (1b) ( c2 = 0.244). these constructs are also strongly positively correlated with each other ( 3 = 0.420). 2. the ict interest construct is formed by two items: the interest in ict after attending the event (2a) and an interest in ict subjects at school (2b), which have strong significant correlation with each other ( 4 = 0.607) and with event attitude ( 5 = 0.587 and 6 = 0.415 respectively). there is also a strong positive association between 2a and a ( c7 = 0.531). 3. the ict intention to learn construct is again formed by two items, namely an intention to find out more about ict (3a) and an intention to select ict based subject at school next year (3b), which have strong significant associations between each other ( 8 = 0.609), 2a ( c9 = 0.698 and c10 = 0.622 respectively), as well as 2b ( c11 = 0.445 and c12 = 0.484 respectively). event attitude has strong positive associations with 3a ( c13 = 0.482) and 3b ( c14 = 0.379). there is also a strong positive association between 3a and a ( 15 = 0.351). 4. ict career intention (our target dependent construct) has strong positive associations with both the ict interest and the ict intention to learn constructs, namely with 2a ( c16 = 0.614, c17 = 0.448), 2b ( c18 = 0.364, c19 = 0.374), 3a ( 20 = 0.609, 21 = 0.390), and 3b ( 22 = 0.528, 23 = 0.390). ict career intention also has a moderate positive association with event attitude ( c24 = 0.381, c25 = 0.212). in order to test the developed model (see figure 1), we exploited the pls path modelling algorithm (marcoulides et al. 2009), which includes the following steps: 1. significance of the relationships between the items and their corresponding constructs, namely between ict career intention and its two items, ict interest and 2a, 2b, as well as ict intention to learn and 3a, 3b, were tested. the event attitude construct has only one item, so the test is not relevant for it. the test was performed by bootstrapping the original dataset with 500 samples. the results show australasian journal of information systems volume 18 number 3 2014 297 that the t-values were far above 1.96, which indicated their significance at p-value less than 0.01 (see table 2). t-statistics path coefficients ict interest --> ict career intention 3.059 0.33 ict intention to learn --> ict career intention 3.341 0.39 ict interest --> ict intention to learn 12.337 0.65 event attitude --> ict interest 7.960 0.57 table 2. t-statistics and path coefficients 2. the strength of influence of items on corresponding constructs (again, relevant only for multiple item constructs ict career intention, ict interest and ict intention to learn), as well as the strength of influence of the related constructs on each other were checked. all relevant factor loadings in the outer model are above 0.71 and, therefore, satisfy the requirement of discriminant validity (fornell and larcker 1981). all path coefficients in the inner model are also quite strong, especially between ict interest and ict intention to learn (0.65), as well as between event attitude and ict interest (0.57) showing very strong direct influence between these constructs (see table 2). 3. the coefficient of determination (r2) of our target variable ict career intention, i.e. the proportion of variability in a data set that is explained by the statistical model (steel and torrie 1980), is quite high (42.3%) (see table 3). as event attitude is not predicted by any other variable in our model, its r2 equals zero. cross loadings r2 cronbach's alpha ict career intention ict interest ict intention to learn event attitude 1a 1b 2a 2b 3a 3b ict career intention 0.42 0.60 0.89 0.79 0.60 0.30 0.50 0.54 0.35 ict interest 0.32 0.60 0.52 0.45 0.94 0.72 0.55 0.58 0.57 ict intention to learn 0.42 0.68 0.59 0.40 0.68 0.34 0.86 0.88 0.44 event attitude 0 1 0.37 0.21 0.59 0.30 0.34 0.42 1 table 3. coefficients of determination, cronbach's alpha and cross loadings 4. the internal consistency reliability (icr) of each construct was assessed by cronbach's alpha test, which is 0.6 or above and, therefore, can be considered as acceptable (hinton et al. 2005, see table 3). 5. the cross loadings coefficients test checks whether the items load the most to the related latent variables, which is fulfilled in our model: the items 1a and 1b load the most to ict career intention, 2a and 2b to ict interest and 3a and 3b – to ict intention to learn. as event attitude is the singleaustralasian journal of information systems volume 18 number 3 2014 298 item construct, its loading equals 1. it is important that the difference between an item loading to a related latent variable is at least 0.2 more than loadings to any unrelated latent variable, which is true in our case (see table 3). thus, all relationship in the model are statistically significant and strong, all the constructs built have high icr. therefore, our hypotheses h5 and h6 that ict interest and ict intention to learn have a positive influence on ict career intention are validated (see table 4). the hypothesis h1, that the event attitude has a positive influence on ict career intention is partially validated, as an indirect influence via ict interest was observed. validated rejected h1 (indirect influence), h5, h6 h2, h3, h4 table 4. hypotheses testing results based on the results achieved, we can argue that interventions have an indirect positive influence on ict career intentions (rq1) by directly influencing the general interest in ict, which, in turn, has a direct positive influence on ict career intentions (rq2). another factor derived from the data available, which positively influences ict career intentions is ict intention to learn, the intention to find out more about ict after the event (rq2). limitations and future research our study has a number of limitations, which are also the subject for future research. first of all, it is data-driven, not model-driven. the analysed questionnaire was initially developed to evaluate the event, enabling organisers to improve the running of the program; not to test our research model. therefore, for future events we will perform a major revision of the questionnaire and will include more questions to test other factors, which might have a significant influence on ict career intentions. the new set of questions will be informed by other comparable studies and theoretical propositions in the literature (such as the model of adya and kaiser, 2005): e.g. more attention would be given to the influence of role models and gender stereotypes coming from family, peer group, media and teachers. another important set of questions should address the newcomers category why they did not consider ict as a career option before the event. the updated questionnaire should also contain more closed questions and fewer open-ended questions. the reason for doing this is that some of the schoolgirls fill in the open questions very reluctantly – more than a quarter of responses to the open-ended questions are missing (25.4%) compared to 9.5% missing values for the closed-ended questions. we have already discussed that some open questions can easily be transferred to closed ones to avoid biased interpretation of the responses, e.g. a tick box for indicating a favourite session instead of free text. the improved questionnaire can then be used to evaluate similar interventions for girls in other countries forming a cross-cultural comparative study. second, more empirical evidence is required to confirm our assumption that career intentions lead to career choice. up to now we could only find conceptual propositions in the literature. a longitudinal study of the girls who participated in one of the events would be very valuable. here we also assume that an intention to choose a career in ict is a necessary, but not sufficient, factor for making a final choice in favour of this field for studies and work. we call for empirical testing of other possible factors influencing ict career choice taking for example the adya and kaiser (2005) model as a basis. such a survey could be enhanced by involving females who already study or work in the ict field. australasian journal of information systems volume 18 number 3 2014 299 conclusion there are strong arguments that having diversity in business and technology decision-making in organizations enables new kinds of innovation and is significantly beneficial, both economically and through increased productivity (e.g. craig et al. 2013). according to ncwit, gender diversity is of importance to the computing industry as diversity: “expands the employee pool; improves the bottom line, enhances innovation; [and] promotes equality” (dubow 2011). however in the ict field women continue to be underrepresented (e.g. european commission 2012). this article also highlights the lack of skilled personnel that will be available to fill the needs in the ict industry in the next decade (e.g. ashcraft and blithe 2010). one possible strategy to tackle these challenges is to invite schoolgirls to attend events aimed at promoting ict education and career opportunities. this paper critically reflects and evaluates the contribution of one such event, namely “go girl, go for it”, and investigates its influence on ict career intentions using quantitative analysis of survey data with pls method. the results of our survey confirm the success of the event with more than 90% of the respondents reporting a positive feeling about careers in ict after the event. at the same time, we would like to raise a concern that we observed a constant decrease in the amount of event participants in 2006-2012. as mentioned earlier, the girls are usually invited by their ict teacher, but the trend is that each year there are fewer teachers who can do it, as less computing classes are being offered at secondary school. we call for action to improve this situation, as our results show that interest in ict subjects at school is crucial in forming intentions to choose ict as a future career. in our study we introduced three categories of girls’ attitude towards careers in ict (non takers, newcomers and takers). based on the findings, we argue that if we can influence newcomers to become takers, then we may shift the cycle of non-participation in ict. this is an iterative cycle in that we propose that newcomers can become takers through continuous inspiring interventions and many positive messages about women in ict from a range of sources and within a range of environments. we also show that interventions have significant influence on interest in ict which, in turn, affects ict career intentions. contradictory to the adya and kaiser (2005) model, we uncovered that same-sex education environments, frequency of computer usage, and the purpose of usage (e.g. for building websites or programming) do not have an influence on ict career intentions. we need many different intervention programs for schoolgirls to engage in at different times in their young lives to give them many positive experiences and a wide variety of information about working with technology. fisher and colleagues (2009) identify impacts on the sphere of influence and the event discussed in this study is just one such sphere. we would like to further understand ict career intentions and plan to test the developed model at future events and other interventions. we also argue that in part due to the voluntary nature of these events, some limitations are evident. it is important to note that organisers and presenters are not paid directly for their efforts and are often not financially supported for their involvement in the event. this is in part due to the lack of continual support from one-well funded organization like the national science foundation or ncwit in the u.s. such events rely on industry funding and the volunteer time of those involved. as such, it is argued that more consistent and substantial funding could aid the better organisation, planning and implementation of these interventions. references adya, m., and kaiser, k. m. 2005. “early determinants of women in the it workforce: a model of girls’ career choices,” information technology & people (18:3), pp. 230–259. australasian journal of information systems volume 18 number 3 2014 300 allison, c., and cossette, i. 2007. “theory and practice in recruiting women for stem careers,” in proceedings of the wepan 2007 conference, copyright 2007, wepan-women in engineering programs and advocates network, lake buena vista, florida. alvarado, c., dodds, z., and libeskind-hadas, r. 2012. “increasing women’s participation in computing at harvey mudd college,” acm inroads (3:4)new york, ny, usa: acm, pp. 55– 64. anderson, n., lankshear, c., timms, c., and courtney, l. 2008. “‘because it’s boring, irrelevant and i don’t like computers’: why high school girls avoid professionally-oriented ict subjects,” computers & education (50:4), pp. 1304–1318. australian computer society. 2012. “2012 australian ict statistical compendium,” canberra. barnir, a., watson, w. e., and hutchins, h. m. 2011. “mediation and moderated mediation in the relationship among role models, self-efficacy, entrepreneurial career intention, and gender,” journal of applied social psychology (41:2), pp. 270–297. blithe, s. 2010. “women in it: the facts,” national center for women & information technology ( ncwit ), boulder, co. bravo, m. j., gilbert, l. a., and kearney, l. k. 2003. “interventions for promoting gender equitable technology use in classrooms,” teacher education quarterly (30:4), pp. 95–110. bruckman, a., biggers, m., ericson, b., mcklin, t., dimond, j., disalvo, b., hewner, m., ni, l., and yardi, s. 2009. “‘georgia computes !’: improving the computing education pipeline,” in proceedings of the special interest group on computer science education (sigcse’09), , pp. 86–90. camp, t. 1997. “the incredible shrinking pipeline,” communication of the acm (40:10), pp. 103–110. coldwell, j., fisher, j., and lang, c. 2013. “the silicon ceiling: women managers and leaders in ict in australia,” in women and mangement: global issues and promising solutions, (vol. 2) , pp. 177–200. courtney, l., timms, c., lankshear, c., and anderson, n. 2005. “establishing pathways for girls in ict: the search for strategies to achieve balance in queensland,” journal of the association of women educators (14:3), pp. 14–19. craig, a. 2010. attracting women to computing: a framework for evaluating intervention programmes, saarbrücken, germany: vdm verlag dr muller aktiengesellschaft & co. kg. craig, a., lang, c., and fisher, j. 2008. “twenty years of girls into computing days: has it been worth the effort?,” journal of information technology education (7). cramér, h. 1946. mathematical methods of statistics, princeton: princeton university press. denner, j. 2011. “what predicts middle school girls ’ interest in computing ? what predicts middle school girls ’ interest in computing ?,” international journal of gender, science and technology (3:1), pp. 54–69. dubow, w. 2011. “ncwit scorecard: a report on the status of women in information technology,” boulder. european commission. 2008. “women in ict: status and the way ahead,” brussels. european commission. 2012. “she figures 2012: gender in research and innovation statistics and indicators,” brussels. fisher, j., lang, c., forgasz, h., and craig, a. 2009. “digital divas: working to change students’ perceptions about ict courses and careers,” curriculum leadership (7:31). australasian journal of information systems volume 18 number 3 2014 301 fornell, c., and larcker, d. f. 1981. “structural equation models with unobservable variables and measurement error: algebra and statistics,” journal of marketing research (18:3), pp. 382– 388. heinze, n., and hu, q. 2009. “why college undergraduates choose it: a multi-theoretical perspective,” european journal of information systems (18:5)palgrave macmillan, pp. 462–475. hinton, p., brownlow, c., and mcmurray, i. 2005. spss explained 2nd edition, routledge chapman & hall. johnson, r. d., stone, d., and nichole phillips, t. 2008. “relations among ethnicity, gender, beliefs, attitudes, and intention to pursue a career in information technology,” journal of applied social psychology (38:4), pp. 999–1022. kruskal, w. h., and wallis, w. a. 1952. “use of ranks in one-criterion variance analysis,” journal of the american statistical association (47:260), pp. 583–621. lang, c., craig, a., fisher, j., and forgasz, h. 2010. “creating digital divas – scaffolding perception change through secondary school and university alliances,” in iticse’10 proceedings of the fifteenth annual conference on innovation and technology in computer science education, new york, ny, pp. 38–42. lehman, a., and rourke, n. o. 2005. jmp for basic univariate and multivariate statistics a step-bystep guide, analysis, , p. 481. marcoulides, g., chin, w., and saunders, c. 2009. “a critical look at partial least squares modeling,” mis quarterly (33:1), pp. 171–175. morrell, c., cotten, s., sparks, a., spurgas, a., and umbc graduate. 2004. “computer mania day: an effective intervention for increasing youth’s interest in technology.,” university of maryland/baltimore county. panteli, n. 2012. “a community of practice view of intervention programmes: the case of women returning to it,” information systems journal (22:5), pp. 391–405. papastergiou, m. 2008. “are computer science and information technology still masculine fields? high school students’ perceptions and career choices,” computers & education (51:2), pp. 594– 608. prensky, m. 2001. “digital natives, digital immigrants part 1,” on the horizon, , pp. 1–6. quesenberry, j. l., and trauth, e. m. 2012. “the (dis)placement of women in the it workforce: an investigation of individual career values and organisational interventions,” information systems journal (22:6), pp. 457–473. ringle, c. m., wende, s., and will, s. 2005. “smartpls 2. hamburg: smartpls, available at http://www.smartpls.com,” . steel, r. g. d., and torrie, j. h. 1980. principles and procedures of statistics: a biometrical approach, (second ed.) new york: mcgraw-hill kogakusha. trauth, e. m. 2002. “odd girl out: an individual differences perspective on women in the it profession,” information technology & people (15:2), pp. 98–118. trauth, e. m., quesenberry, j. l., and huang, h. 2009. “retaining women in the u.s. it workforce: theorizing the influence of organizational factors,” european journal of information systems (18:5)palgrave macmillan, pp. 476–497. unesco. 2007. “science, technology and gender: an international report,” paris, france. united nations. 2011. “mainstreaming a gender perspective in science, technology and innovation policy,” . australasian journal of information systems volume 18 number 3 2014 302 vcaa statistics. 2013. “senior secondary certificate statistical information,” victorian curriculum and assessment authority. retrieved 5/8/2013, from http://www.vcaa.vic.edu.au/pages/vce/statistics/subjectstats.aspx . acknowledgements we would like to thank all the volunteers who organized the “go girl, go for it” events and, consequently, all the girls who attended and made this study possible. we also gratefully acknowledge the support of the sponsors, speakers and the ict teachers who motivated schoolgirls to participate in the “go girl, go for it” events. we also gratefully acknowledge dr armin stein for his ongoing support of the project. an earlier version of this paper was presented at the australasian conference on information systems (acis) 2013 in melbourne, australia. towards the development of effective and flexible information systems aphrodite tsalgatidou, dimitris gouscos, constantin halatsis dept. of informatics, university of athens, typa buildings, panepistimiopolis, ilisia, gr-157 71 athens, greece email: {afrodite, gouscos, halatsis} @ uranus.di.uoa.ariadne-t.gr abstract the development of information systems which are effective (i.e. satisfy users' needs) and flexible enough to incorporate future changes of user requirements basically depends on the importance given to the requirements analysis stage and to the techniques employed for their development this paper presents a new approach to requirements analysis through an environment called venus. the venus environment provides a set of tools to support the requirements capture and analysis process so that design and development is more effective and flexible. the construction of user requirements starts by first constructing an entity-relationship model which is then transformed to an object-oriented rule-based model (orm) where rules are used for the definition of the behaviour of each object class. orm is then mapped to a graphical petri-net based model and is validated by employing graphical animation and exploiting formal properties of the underlying petri-net formalism. introduction two desirable characteristics of information systems are effectiveness and flexibility. effectiveness refers to the satisfaction of users' needs and is primarily related to the production of valid and complete requirements specifications. flexibility refers to the ability of information systems to easily incorporate any future changes of user requirements; flexibility depends on the techniques employed for the development of information systems starting from the initial requirements analysis phase to the final implementation and integration stage. the importance of requirements specifications in the information systems development process is not a recently identified need (endres, 1975). boehm (1975) has denoted that incorrect requirements specifications result in serious errors in the information system development process, where the cost to fix them increases as the development process progresses. however, there is still a lack of methodologies and tools which would focus on the capture and validation of requirements specifications and the work reported by reubenstein and waters (1991), zave (1991) and fickas and finkelstein (1993) are among the few examples that can be found in this area. the large number of existing methodologies for systems analysis and design produce specifications which are inherently static. furthermore, case tools play a passive rather than active role in requirements specifications. a developer, in order to validate the captured requirements, tries to derive a system's behaviour from a static description. what is needed is an approach and a set of tools that enables a developer to obtain a 'working' model of a system at the early requirements phase, so that the system's behaviour can be better understood and appreciated by end users. this may be achieved if the produced requirements specifications are executable. an executable specification is a formal model of the system with the ability to simulate the system's behaviour when executed by a suitable interpreter, thus, such a specification can be thought of as a prototype of a proposed information system and therefore it should be comprehensible, modifiable, easy to check for internal consistency and free from bias toward particular implementation strategies. to this end, this paper presents an integrated environment called venus which offers a number of tools to support the requirements engineering process. the tools offered by venus assist a system analyst to elicit, specify, analyse and validate executable requirements specifications within an objectoriented framework. petti nets (murata, 1989) are used as the underlying formalism for expressing object-oriented requirements specifications in a graphical manner. prototyping, animation and execution are some of the mechanisms used by venus for validating object-oriented executable requirements specifications. previous studies (tsalgatidou et al, 1994) found the object-oriented approach appropriate for developing requirements specifications in the venus system for a number of reasons such as: the uniform modelling of various real world objects the provision of useful abstractions, such as generalisation, aggregation and specialization, which facilitate the extension of the model and the incorporation of new types of information may 1994 74 the easy construction of executable specifications. this paper concentrates on the production of models, in particular of the entity-relationship model and the object-oriented static model and the validation of the produced specifications through graphical animation, and begins by presenting the venus environment together with its various components. the stepwise construction of requirements specifications is discussed. these specifications may firstly be expressed in an entity-relationship model (erm) which is subsequently automatically mapped to an object-oriented static model (osm). the osm is further enhanced so as to incorporate the dynamic behaviour of the system under study and produces an object-oriented rule-based model (orm). the orm is validated by being either translated to c++ executable specifications or mapped to a graphical petri-net based model, called an rbn, which is then executed and graphically animated. the rbn model is formally defined and briefly introduced. validation of the orm by graphically animating the resultant rbn and showing that, given an initial marking of the rbn, certain markings are reachable, is also discussed. finally, the venus approach is compared to related work and conclusions are drawn. the 'venus' environment the tools offered by venus, the interaction of the analyst with these tools, as well as the models used as input or produced by these tools are shown in fig. 1, which depicts the sequence of steps usually followed by an analyst during the requirements development process. the tools provided by venus are: (a) a graphical editor & text generator for entity-relationship models (erms), called germtg. using germ-tg, an analyst can develop a graphical erm which captures details about the data of the application under study. a textual description of the developed graphical erm is automatically generated by germ-tg (avdis & karpodinis, 1993). (b) a mapping tool, called meros, which takes as input erm specifications and produces objectoriented specifications, hence constructing an object-oriented static model (osm); the osm specifications are static since they are produced from erm specifications which are static by definition. thus, the osm needs to be enhanced in order to incorporate the dynamic behaviour of the identified object classes. this is accomplished with the use of the next tool. (c) a syntax directed editor, called sdorm, for enhancing osm specifications with dynamic aspects expressed in a rule-based manner and thus producing specifications expressed in an object-oriented rule-based model (orm). (d) a c++ generator, which takes as input orm specifications and produces executable specifications in c++. the execution of these specifications greatly facilitates the process of validating the requirements described by an orm. (e) a mapping tool, called morbn, which takes as input orm specifications and produces graphical rbn (petri-net based) specifications. (f) a graphical editor/animator for the rbn model, called gea-rbn. this tool may be used by an analyst to edit and animate graphical rbn specifications in order to validate them against certain validation criteria. this validation process is discussed in the relevant section of this paper. depending on the analyst's preferences and experience, the development of specifications may start at different points of the venus-based development process. furthermore, the venus architecture allows more than one analysts to be involved in the venus development chain, each one working on a different phase. the entity-relationship model (erm) many analysts are used to constructing erms as one of the first steps of the requirements analysis stage. thus, although venus aims at producing object-oriented requirements specifications for the various reasons mentioned in the introduction, it also provides germ-tg, thus giving to analysts the opportunity to construct a graphical erm. fig. 2 shows an example of a graphical erm and its corresponding textual description. this textual description is automatically produced by germ-tg after the graphical erm is constructed. the bnf definition of the language for textual description of erms can be found in (gouscos & tsalgatidou, 1992). 75 ajis m« imfe fig 1 the venus environment fig. 2 an example of an erm (graphical & textual representation) may 1994 76 an erm is a binary entity-relationship model enriched with inheritance and constructed on the basis of a linguistic approach (tsalgatidou & loucopoulos, 1991). the main concepts of an erm are: entity types, entity subtypes, label types and relationship types. entity types describe entities with the same properties (e.g. assistant, essay). label types describe labels (i.e. values) of a certain type. relationship types can be of two kinds: they can either associate entity types to label types, in which case they are called reference type relationships (e.g. course has name) or they may be confined to associating entity types, in which case they are called fact type relationships (e.g. assistant assists professor). a fact type relationship may be expressed in more than one ways, showing the various roles of the involved entity types. for example: 'assistant assists professor' and professor assisted_by assistant, where 'assists' and 'assisted_by' are the roles of the entity types assistant and professor respectively in this relationship. cardinality constraints are also modelled by erm, for example 'assistant assists 1:1 professor1 and professor assisted_by 1:n assistant. the object-oriented model the object-oriented static model (osm) osm is an object-oriented model (gouscos, 1992) which encapsulates only the static aspects of requirements and is automatically derived from erm using the meros tool. fig. 3 contains part of the osm specifications which were generated by applying the meros tool to the erm of fig. 2. it can be seen that entity types have been mapped to object classes, whereas relationship types have been mapped to properties of the corresponding object classes. object schema essays value classes essaystatus values [submitted, underjnitjeview for.revision] end value classes object classes object class professor subclass of person instanrp properties teaches 1:1 course assistedby 1:n assistant end instance properties end object class professor end object classes end object schema essays fig. 3 an example of an osm specification this is a first-cut object-oriented model which must be subsequently enchanced by a systems analyst so as to incorporate the dynamic aspects of the identified object classes (reflecting the business policy) and any new object classes that may be needed. the sdorm tool provided by venus is a syntaxdirected editor which assists the analyst in this enhancement of osm. the output of this process is an object-oriented rule-based model (orm) which is briefly described in the following. the object-oriented rule-based model (orm) v orm is an extension of osm incorporating the dynamic aspects of the identified object classes using the concepts of behaviour units (bus), signals and rules. the concept of bus is used to partition the behaviour of an object class. each bu belongs to one class of the object-oriented model and is associated with a specific triggering signal type. the receipt of individual signals of this type activates the behaviour described in this bu. a bu may be either class bu or instance bu and has the following general structure: class i instance bu class_name.bu_name triggered by signal_name(signal_parameters) bu body end bu bu_name 77 ajis the body of a bu is a set of dynamic rules having the form: [if preconditions then] actions the preconditions of a rule are expressed by a boolean formula and have to be satisfied before the actions described in the rule's then part can be executed. the receipt of a triggering signal by a bu activates all the dynamic rules of that bu. the preconditions of rules of the same bu are mutually exclusive, so that exactly one of the rules will always fire. a rule with no preconditions executes its actions every time an appropriate triggering signal is received. actions in a rule may modify/create/delete object instances and/or produce some signals sent to other bus or to the external environment of the modelled system. a rule cannot be part of more than one bu and a bu belongs to exactly one class; therefore the dynamic behaviour of each object class is modelled as a collection of rules grouped in the bus specified for that class and triggered by specific signals. object class professor subclass of person instance properties teaches 1:1 course assistedby 1:n assistant end instance properties instance behaviour instance bu professor.decideforessay triggered by decideforessay (essay.review) nile rl hac professor lac {professor} if signal.review.mark in {aplus, a, b} then produce signal setstatus(accepted) for signal.essay; produce signal newpass(signal.essay.writtenby, signal.essay.writtenfor, signal.review.mark) for pass; produce signal essayaccepted(signal.essay, signal.review) for signal.essay.writtenby; rule r2 hac professor lac {professor} if signal.review.mark in {d.e} then produce signal setstatus(rejected) for signal.essay; produce signal essayrejected(signalessay, signal.review) for signal.essay .writtenby; rule r3 hac professor lac {professor} if signal.review.mark = c then execute definedeacfline(signalessay, signal-review .deadline); produce signal setstatus(waiting_for_revision) for signalessay; produce signal setdeadline(deadline) for signal.review; produce signal essayforrevision(signal.essay, signal.review) for signalessay.writtenby; end bu decideforessay end instance behaviour end object class professor fig. 4 an orm specification orm supports single inheritance as follows: object classes inherit all the static properties of their superclasses and may introduce new ones. they also inherit the bus of their superclasses without any changes and/or redefine (some of) them by changing (some of) their existing rules or by adding new ones. object classes may also introduce new bus describing behaviour not specified by their superclasses. for each rule r, the rule model provides two fields: highest acceptor class (hac(r)) containing the name of the class where rule r is defined and lowest acceptor classes (lac(r)) containing the names of the lowest subclasses of hac(r) that still inherit rule r. fig. 4 contains part of the orm that resulted from the enhancement of the osm of fig. 3. more specifically, fig. 4 depicts the part of object class professor which specifies what an instance of this class has to do in order to decide for an essay that was reviewed. this is described in terms of a bu (called professor-decideforessay) triggered by a signal (called decideforessay(essay,review)). the body of this bu is a group of three rules which, depending on the satisfaction of certain preconditions, execute appropriate actions. the hac and lac fields of all three rules in this example are set to professor and {professor} respectively, meaning that these rules have been defined in this class and apply only in this class, i.e. they are not inherited by any other classes. the rules which have been inherited by class professor from class person (not may 1994 78 shown in this figure) have their hoc and lac fields set to person and {professor} respectively. in case that a rule of class person is also inherited by classes assistant and student, then its hoc field is again set to person, whereas its lac field is set to {professor, assistant, student}. therefore, the dynamic behaviour of an information system is modelled as a sequence of firings of rules, grouped in bus and triggered by specific signals. all the rules of a bu are triggered by the same signal. the firing of each rule, may result in the activation of other bus by generating appropriate signals. the interaction of an information system with its external environment is modelled by the receipt and sending of signals by bus. thus, signals serve as the means of communication between the various parts of an information system and between the system and its external environment more about orm may be found in (tsalgatidou et al, 1993). the dynamic behaviour of an orm can be validated by executing the resultant specifications. venus offers a c++ generator tool which is used for transforming orm specifications to c++ executable specifications (tsalgatidou et al, 1994). furthermore, venus offers a tool called morbn, which produces a graphical petri-net representation of an orm, called rbn. rbn is a formal, graphical and executable model. the dynamic behaviour of a rbn can be demonstrated by employing the graphical animation and execution facilities provided by the gea-rbn tool of venus. furthermore, system behaviour can be observed at a number of abstraction levels. the mapping of orm to rbn and the formal definition of the latter are presented in the following section. the rule-based net (rbn) model a rule-based net is a petri net-based model which is used for a graphical modelling of the orm. an rbn resembles a predicate-transition (prt) net (genrirch & lautenbach, 1981), augmented with additional information. the basic constituents of an rbn are places, transitions and arcs connecting places and transitions. each place is inscribed with a signal type and may hold tokens which represent signals of this signal type. each transition has a unique input place and is inscribed with rules which are triggered when the corresponding input place holds a token, thus denoting that a signal is present when the if part of a triggered rule is satisfied, the rule executes its actions. each arrow is inscribed with the number and type of signals that can flow through it in order to activate a rule or to collect signals into a place. an rbn also contains an underlying textual structure which describes information used in the various inscriptions. an analyst may interactively produce rbns at different abstraction levels using the morbn tool. five distinct abstraction levels may be produced: 1st abstraction level -> 'context rbn'; this rbn contains only one transition corresponding to the behaviour of the entire modelled system; places represent signals sent to and from the external environment 2nd abstraction level -> 'high-level rbn'; here, the unique transition of the 'context rbn' is decomposed in other transitions and places corresponding respectively to the object classes of orm and to the signals exchanged between these classes. 3rd abstraction level -> 'medium-level rbn'; each 'class transition' of the 'high-level rbn' is further decomposed here into other transitions and places, corresponding respectively to the bus of this object class and to the signals exchanged between these bus. 4th abstraction level -> 'low-level rbn'; in this rbn, each 'bu transition' is decomposed into transitions corresponding to the rules contained in this bu. these new transitions are inscribed with the corresponding rule names. 5th abstraction level -> 'detailed rbn'; this rbn does not contain more transitions or places than the 'low-level rbn'; the difference is in the inscription of transitions which now contain the 'body' of rules (namely the hoc, lac, preconditions and actions of each rule). this 'detailed rbn' is always accompanied by its structure s. a rbn of the lowest abstraction level, i.e. a 'detailed rbn' is formally defined as a tuple r = < p, t, f, k, n, _ , sig, insc, hac, lac, mo > 79 ajis where p, t and f are sets corresponding respectively to rbn places, transitions and arcs; k and n are functions which respectively map each place and each arc to an integer number denoting the capacity of the place or the multiplicity of the arc; _ is the underlying structure of an rbn which denotes the domain of signal parameters, of the properties of object instances, of objects of the rbn and the signal domain of the rbn; sig, insc, hac and lac are functions which relate the various rbn elements to appropriate elements of orm and finally mq is an initial marking that assigns token to places. a detailed definition of an rbn as well as examples for rbns of various levels may be found in (tsalgatidou et al, 1994). fig. 5 depicts an example of a 'low-level rbn', i.e. an rbn where transitions are inscribed with rule names and places are inscribed with appropriate signals which are produced by and/or trigger rules. fig. 5 a low-level'rbn thus, an rbn is a simple formal model high-lighting control flow within a system and hiding at the same time, in the corresponding transitions, information concerning what is happening in the system. one of the advantages of using rbns for modelling the dynamic behaviour of information systems is that the resulting graphical representation facilitates validation, as well as analytical study of system behaviour by exploiting certain properties of the nets; this topic is discussed in the next section. may 1994 80 validation of requirements specifications validation of user requirements is a hard process. it refers to the production of specifications which are complete and consistent without ambiguities and vagueness. one way of validating the orm requirements is by executing the generated c++ prototype, as has already been mentioned. another way of orm validation is through the constructed rbn model due to the executable nature and the potential for graphical animation of the latter. animation has proven a valuable validation tool, and the results of early animation are very promising way for reducing errors during requirements specifications and ensuring that the intended system behaviour has indeed been properly captured and modelled. fig. 6 is an example of validation through graphical animation of a 'high-level' rbn, i.e. an rbn where transitions correspond to object classes of the modelled application and places represent the signals exchanged between object classes. in this example, a student creates a new essay and this is shown by a token in the signal place called newessay (t,s,c,b) (see fig. 6(a)). this signal is directed to an appropriate instance of class essay, triggers the corresponding bu and the rule of this bu which has its if part satisfied will fire, i.e. execute its actions. in the next snapshot (6(b)), there are two tokens assigned to places setreviewer(a) and reviewessay(e); this means that the appropriate rule of class essay has already fired, and a signal of type assignessay(e) has been generated and sent to an instance of class professor triggering the appropriate bu. subsequently, an appropriate rule of the triggered bu has fired and has generated two signals of type setreviewer(a) and reviewessay(e) which are represented by the two tokens shown in fig. 6(b).tbe signal of the setreviewer(a) type is directed to the appropriate instance of class essay whereas the signal of the reviewessay(e) type is directed to the appropriate instance of class assistant and triggers the rules that deal with the process of reviewing an essay by an assistant the snapshot of fig. 6(c) shows two tokens at the places setreview(r) and essayreviewed (e,r) which are output places for the essayreview class. this means that the appropriate assistant has reviewed the essay and sent a signal of type newessay review(e,m,c) to the object class essayreview in order to activate the rule which generates a new instance of this class and subsequently produces the two signals shown in this snapshot (fig. 6(c)). at the last snapshot (fig. 6(d)), depicts three tokens, in places essayrejected(e,r), setstatus(s) and setreviewdata(m,c) respectively. this marking means that a professor, taking into account the comments and the marks assigned to the essay by his/her assistant, decided to reject the essay. thus, a signal of type essayrejected(e,r) is sent back to the student in order to inform him/her about the results of essay revision; furthermore, the appropriate instances of classes essay and essayreview receive a signal of type setstatus(s) and setreviewdata(m,c), respectively, in order to update the information held about the status of the essay and its review data. this is the final marking of the review process of an essay submitted by a student 81 ajis (a) llfll) rfcp*" ****"" tflp fig. 6 (a,b) (shaded transitions fire to produce next marking) may 1994 82 (c) llffl) fig. 6 (c,d) (shaded transitions fire to produce next marking) the graphical editor and animator for rbns (gea-rbn) (currently under development on unix workstations using osf/morif, c++ & prolog) provided by the venus environment, enables animation in two ways: a) user-driven (guided by an analyst working together with end-users of die modelled application) and b) automatic. in the former case, the user decides which rules will fire and which will not, and comments on results. in the latter case, gea-rbn is given an initial signal to 83 ajis evaluate all alternative paths (i.e. preconditions of all triggered rules), execute the corresponding actions and produce various alternative markings. thus, it is checked whether various markings are reachable, i.e. whether all prescribed states can be reached and under which conditions. furthermore, missing rules, dead-end rules (i.e. rules which can never fire), wrong rules (i.e. rules producing wrong results) or circular rules can also be identified. animation feedback can be used to 'weed-out' any errors discovered, thus helping in the interactive design of the rule base. therefore, using gea-rbn one can prove that the produced rbn specification is valid, i.e. that it is deterministic, complete, consistent and correct. an rbn specification is considered deterministic when the behaviour of the modelled system is always predictable; this follows from the premise that information systems are considered to be deterministic, i.e., certain things happen under certain conditions. this means that whenever a signal arrives from the external environment or is generated by the system, we always know what happens next. the rbn specification is complete, when there is always something happening, whenever a signal arrives. the rule-based specification is consistent, when there are no rules with the same triggering signal and the same preconditions but with contradictory actions. finally, the resulting requirements specifications are correct, when they do what end-users want their system to do. related work requirements specifications are produced in the venus environment within a rule-based objectoriented framework. there are many examples of the use of the object-oriented approach in systems analysis and development in van baelen et al (1992), booch (1991), coad & yourdon (1991), coleman et al (1992), rumbaugh et al (1991). furthermore, examples of the use of rules in requirements specifications may be found in dhaenens et al (1991) and loucopoulos et al (1991). the venus environment combines both the object-oriented and rule-based approaches within a uniform modelling framework for all system aspects. venus also places a lot of emphasis on the production of executable specifications. the importance of executable specifications has been realised by researchers and developers within the software engineering community and a number of executable system models, executable specification languages and tools contributing in this direction have appeared lately. for example paisley (zave, 1991) is a language designed for real-time distributed systems which uses an operational approach for building a requirements specification, emphasizing the definition of processes as the building blocks of the system. venus differs from paisley in the sense that the requirements specifications are objectoriented and the basic building blocks of the system are objects rather than processes. graphical animation and petri nets are among the mechanisms and tools used by venus for modelling and validation requirements specifications. animation, on the one hand, has proven to be a valuable validation technique; see, for example, the work reported by finkelstein and kramer (1992), dahler and his colleagues (1987) and shand et al (1988). on the other hand, petri nets are a powerful formalism (murata, 1989) and have been used as a modelling tool in various applications. for example, in lakos & keen (1993) a language for object-oriented petri nets (loopn) is used for modelling a door controller protocol. other examples may be found in brinkkemper & der hofstede (1990) and kappel & schrefl (1991), where extensions of petri nets are used for bridging the gap between informal requirements engineering activities and the more formal system development activities. venus introduces a rule-based extension of petri nets for modelling and animating objectoriented requirements specifications for validation purposes. therefore, the venus approach is related to object-oriented development, produces executable specifications and uses petri nets as the underlying formalism and graphical animation for modelling and validating requirements specifications. the major advantage of venus over other approaches is the integration of a number of paradigms into a uniform modelling framework. summary and conclusions the development of effective and flexible information system starts from the production of complete and valid requirements specifications. this paper presented the venus uniform framework which demonstrates the combination of object-oriented and rule-based approaches, integrating all information system aspects. requirements specifications produced by venus preserve explicit representation of application domain policy in terms of dynamic rules. these object-oriented rulemay 1994 84 kappel, g. & schrefl, m. (1991) "using an object-oriented diagram technique for the design of information systems", in (sol & van hee, 1991). lakos, c.a. & keen, c.d. (1993) "modelling a door controller protocol in loopn, proceedings of the 10th international conference tools europe 1993, versailles, france, prenticehall, pp. 31-44. loucopoulos, p., mcbrien, p., schumacker, r, theodoulidis, c., kopanas, v. & wangler, b. (1991). "integrating database technology, rule-based systems and temporal reasoning for effective information systems: the tempora paradigm", information systems, vol. 1, no. 1. loucopoulos, p. & zicari, r. (eds.) (1992) conceptual modeling, databases, and case, john wiley & sons, inc. murata, t. (1989) "petri nets: properties, analysis and applications", proceedings of the ieee, april, vol. 77, no. 4. reubenstein, h. b. & waters, r. c. (1991) 'the requirements apprentice: automated assistance for requirements acquisition", ieee transactions on software engineering, march, vol. 17, no.3, pp. 226-240. rolland, c. (1992). "trends and perspectives in conceptual modelling", in (loucopoulos & zicari, 1992), pp. 27-48. rumbaugh, j., blaha, m., premerlani, w., eddy, f., & lorensen, w. (1991) object-oriented modelling and design, prentice-hall int. shand, j et al. (1988) "an enviroment for the execution and graphical animation of jsd specifications", in procs. of international workshop of kbs in s/w engineering, umist, manchester. sol, h.g. & van hee, km. (eds.) (1991) dynamic modelling, north-holland. tsalgatidou, a. & loucopoulos, p. (1991) "an object-oriented rule-based approach to the dynamic modelling of information systems", in (sol & van hee, 1991). tsalgatidou, a., gouscos, d. & halatsis, c. (1993) "rule-based behaviour modelling of information systems", proceedings of the 26th hawaiian conference on systems sciences (hicss-26), vol. iv, january, ieee computer society press, pp. 409-418. tsalgatidou, a., gouscos, d. & halatsis, c. (1994) "specifying and validating requirements: the venus system", to appear in the proceedings of the llth international conference on technology of object-oriented languages and systems tools europe 1994, march, paris, prentice-hall, int. ltd., pp. 89-102. zave, p. (1991) "an insider's evaluation of paisley" ieee transactions on software engineering, vol. 17, no. 3, pp. 212-225. may 1994 86 view publication statsview publication stats https://www.researchgate.net/publication/30063059 paper title (paper title style) australasian journal of information systems kautz & bjerknes 2015, vol. 19, pp. 71-95 tales of it consultants tales of it consultants: understanding psychological contract maintenance and employment termination karlheinz kautz faculty of business university of wollongong kautz@uow.edu.au gro bjerknes information technology services university of wollongong gro.bjerknes@gmail.com abstract this research investigates the question why dedicated information technology (it) consultants quit their jobs and voluntarily terminate their employment contracts. a research approach inspired by grounded theory is used and as a theoretical contribution a novel, cyclic process model for the maintenance or neglect of the psychological contract between employees and employers is derived from the empirical data. the model is subsequently substantiated through the integration of various concepts and theories identified in the literature, combining the concept of psychological contract with social influence theory and a theory of the relation between intrinsic motivation and management practices. the research is based on a revelatory case study which provides tales from four it consultants in an it consulting firm. the model helps to explain it consultants’ behaviour of voluntarily terminating their employment contracts. additionally, it should assist managers in it consulting firms to avoid practices which might lead to their employees’ loss of intrinsic motivation and result in the loss of valuable employees for the organization. keywords: psychological contract; voluntary contract termination; it consultants 1 introduction why do dedicated it consultants quit their job and leave their employing it consulting firm during a current labour market climate with job shortages, downsizing and salary cuts, where they are among the privileged who could keep their jobs during several rounds of farewell packages and layoffs? this was the starting point for the work presented in this paper. it reports the results of research which answers this question by providing the tales and stories of four it consultants who worked in the same (scandinavian) it consulting firm – in the following referred to as the company due to a confidentiality agreement. the second author of this article and primary empirical investigator of the work described here was for many years employed in the company confronted with this phenomenon. using an innovative and rigorous research approach inspired by grounded theory we develop a model of the process which leads to the it consultants’ voluntary employment termination and subsequently substantiate the model through the integration of various concepts and theories identified in the literature. we draw on the concept of psychological contract (rousseau 1995, morrison and robinson 1997, carr 2001, guest 2004), integrating it with social influence theory (cialdini 2001a, b) and a theory of the relation between intrinsic motivation and management practices (amabile 1998) into a cyclic model of ‘the maintenance or neglect of the psychological contract’ between employees and their employing organization. through the analysis of the four stories we derive theoretical and practical insights about the phenomenon under investigation, both with regard to the voluntary termination of employment contracts during a crisis situation within the it industry, and also in a wider context. 71 australasian journal of information systems kautz & bjerknes 2015, vol. 19, pp. 71-95 tales of it consultants the concept of psychological contract describes the informal aspects of an employment relationship (svedberg 2000, carr 2001). the first appearance of the concept is commonly attributed to argyris (1960) who called the relationship between employees and their foremen a ‘psychological work contract.’ schein (1970) used the concept to discuss the informal relationship between consultants and clients. in information systems (is) research sabherwal (1999) used the concept in this context and analysed the informal relationship between a customer organization and an it supplier in a case study where the supplier developed a computerized information system based on customer-defined specifications. employment is a long term relationship that is governed by a formal contract. however, according to transaction cost theory resources are required to maintain long-term contracts and in this context, informal, relational contracts are often established in addition to the formal ones to reduce the possibility of opportunism (barney 1996). a psychological contract helps to regulate informal work situations and the expectations between the involved parties and what they provide to each other (svedberg 2000). in addition to salary employees want to derive meaning and satisfaction from work (morse 2003), and in this regard a psychological contract plays a significant role as well. psychological contracts are however obscure, they often only exist unconsciously (carr 2001). as its content is vague, an important aspect of a psychological contract is that the expectations in relation to the contract are met. the remainder of the paper is structured as follows. in the two following sections we introduce our research approach and present the stories of the four it consultants. we then describe the theoretical consideration and the resulting model in more detail. subsequently, we revisit and analyse these stories in light of the theoretical model and discuss our results together with our theoretical and practical insights and the limitations of our work. finally we summarize our contributions and provide avenues for further research. 2 research approach and method given the background of the second author the purpose of the study was to investigate in an unbiased manner why dedicated it consultants quit their job; the purpose was not to examine whether the consultants behaved in a specific preconceived and hypothesised way. the study was thus explorative and qualitative in nature and based on open-ended interviews (creswell 2003). its emergent strategy with no specific hypothesis suggested the adoption of a method that allowed for whichever explanatory variables and concepts. without hypotheses to be verified or invalidated, the data collection had to be more open than a search for the impact of already known factors. we therefore opted for a case study or, more precisely case studies based on storytelling with the stories documented in the form of narratives (gabriel 2000) and an analysis using an approach inspired by grounded theory (glaser and strauss 1967, strauss and corbin 1998, locke 2001). storytelling in combination with narratives is acknowledged as a valuable research approach in is (kendall and kendall 2012). letting respondents speak for themselves and give their own explanations through the provided data reveals the immediate reasons and reasoning behind conclusions and adds a more direct and valid touch to the results. in line with some of our earlier work (hansen and kautz 2005, kjærgaard and kautz 2008) we deduced relevant issues directly from the data and examined these issues and their interrelationships more closely, avoiding unnecessary interpretation of the interviewees’ statements where possible. in other words, the results informing the cyclic process model were ‘grounded’ directly in the observed data instead of being produced e.g. by deduction or other ‘intellectual experiments’ (hansen and kautz 2005). in spite of our open and emergent strategy and glaser’s (1992) advice that there is no need to review any literature in the area under study so as not to ‘contaminate’ the researchers’ interpretation of data, we followed walsham’s (1995) warning against ignoring existing theory completely and strauss and corbin’s (1998) suggestion to build up a certain theoretical sensitivity. thus, we subsequently allowed the literature to inform us about the subject, which is in line with interpretive case studies (walsham 1995) and we have been inspired by existing literature within the area benefiting from previous results achieved by others. 72 australasian journal of information systems kautz & bjerknes 2015, vol. 19, pp. 71-95 tales of it consultants however as stated above the aim of the study was to solicit and understand the interviewees’ own explanations; we thus did not explore and consider any other influences and drivers not stated by the consultants during the interviews for their employment and its subsequent termination. the primary reason for going back to existing literature was the extent of our study with its challenge of gathering sufficient data to examine all relevant aspects of the subject. we wished to achieve an interaction between the existing literature and our observations from the case organization to explain interrelationships and contribute to theory with new insight from practice. although the purpose of the study was not to verify or refine existing theory we in particular used the literature to substantiate the derived process model and to apply it to understand and theorize about our findings from practice as well as to generalise this understanding into an explanatory model that might be useful for scholars and practitioners. the ontological and epistemological assumptions of the study are informed by the interpretive paradigm, and accordingly the role of those involved in the investigated phenomenon is understood as that of active constructors of meaning as well as active interpreters of reality. their meaning construction process is central and forms the basis for understanding their actions. our study took place in a single case organization and we selected a revelatory case, or rather cases, if we consider the individual informants (yin 2009), which has not been described in the is literature before. such cases are considered to be prototypical or paradigmatic of the phenomenon of interest (gerring 2007) and by studying the case organization and the informants we were able to illuminate key aspects of the process, which led to it consultants’ voluntary employment termination, and to develop a model of what we term a cyclic process of the maintenance or neglect of the psychological contract. in contrast to a representative case selection our objective is to introduce ideal types, which are formed by the one-sided accentuation of one or more points of view (weber 1949). as such, the identified constructs and the subsequently presented model exhibit in their current form an instance of theorizing through idealization (lopreato and alston 1970, ghazawneh and henfridsson 2013), which needs to be further validated. this approach might be criticised for initially only generating a local empirical theory which might not be generalisable, but as argued by hughes and jones (2003) it contributes to the existing body of knowledge by providing a detailed account of empirical findings. there are two reasons why the company is suitable for this endeavour. first, the company’s history (see next section) involves a number of alterations in its strategy, which allowed us to segregate specific episodes for tracing underlying patterns and enabled temporal decomposition, which is important in process studies (langley 1999, ghazawneh and henfridsson 2013). second, as an employee of the company herself and a long-time colleague of the study participants, the second author who performed the data collection had the role of an insider researcher and shared an identity, language, and experiential base with them (asselin 2003). being an insider researcher enhances the depth and breadth of understanding a population that may not be accessible to a non-native researcher, but it also raises questions about objectivity, reflexivity, and the authenticity of the research project, because the insider might know too much or is too close to the project and may be too similar to those being studied (kanuha 2000). the study is based on in-depth interviews with four consultants, who had finished in the company. they were selected because they all had a high regard among their colleagues in the company and they all continued to perform the same kind of jobs and assignments in their new careers; job content as such had not been a reason for their employment and employment termination. they also had in common that they (a) ceased their employment by their own choice, they did not receive a farewell package or were laid-off; (b) had been employed for a long time, at least seven years; (c) they had been active in the professional development in their areas of expertise and demonstrated their commitment by giving more to the company than they were obliged to. each consultant was interviewed intensively once with each interview lasting about two hours. 73 australasian journal of information systems kautz & bjerknes 2015, vol. 19, pp. 71-95 tales of it consultants the interviews were based on a vitagram, i.e. a graph, which the interviewees produced before their interview showing their employment and working life satisfaction in the company over time. vitagrams have been used in medicine to capture quality of life since the 1970s (see armstrong et al. 2007). to support the interviewees during the report of their stories we used an interview guide and open questions, which took their starting point in the vitagram. once triggered by an overall question vitagrams are very useful because they let the interviewees tell their own story and develop their own narrative independent of the interviewer’s preconception or any questions developed on a specific theoretical foundation. in that sense the chosen research approach was similar to a grounded theory approach and supported the uncontaminated collection and initial analysis of the empirical data. during and immediately after the interviews interview reports were produced which the interviewees read, commented upon and approved, which built the background for the subsequent analysis. in line with the story telling approach and with the research question as the point of departure the data was openly coded and thematically analysed for similarities and patterns, as well as for disparities and idiosyncrasies and for notable episodes and incidents, but no extensive axial and selective coding was performed. to meet the mentioned challenges, especially to avoid an insider’s bias and to avoid influencing the respondents two further countermeasures were taken: to be very conscious about them, the inside researcher wrote down her own expectations of what she would find through the investigation before she started the interviews and very open questions not based on any of these assumptions, but rather on each individual's vitagram, were chosen. still, we found that the respondents took some truths for granted they did not elaborate on them, as they assumed the interviewer to know these based on their shared history. in spite of that, the differences between the stories that were told, were much larger than expected and they were different from the story the inside researcher had written down herself. the same is valid with regard to the consultants’ motives to quit their jobs. none of the expected references to the general crisis in the it industry, the company’s changing environment through mergers and acquisitions, de-mergers and separations with the resulting exposure to new managers were mentioned as influences for their employment termination and were for that reason not considered in the further analysis. hence, although the inside researcher might have influenced some details of the interviews by pointing to examples and incidents, we can conclude that this has had little or no bearing on the major features of the stories, and therefore it has been justifiable to use the collected empirical data to develop the model. 3 the tales of the it consultants the company, which builds the background for the four tales of the interviewed consultants, is a management and it consulting firm. it works with innovative restructuring processes in which it is a key instrument for achieving business goals and for helping its customers to add value. for anonymity purposes we named the four consultants art, bee, curt and di. the company was founded in the late 1980s by about 40 it consultants, who had left another organization. after initial problems, during the next 12 years the company went very well and grew steadily through mergers and acquisitions to about 1200 employees, among others through a major merger with another larger and more technology-oriented firm. it was also listed on the stock exchange. to stimulate the employees, incentive programmes including stock options and immediate cash rewards were established. however, in line with other it consultancies the company had grown too fast and too large and when an economic crisis hit the it sector around the year 2000, too many companies competed on a small market with too few customers. thus in the following three years parts of the company were sold again and in several rounds more than 100 employees accepted farewell packages while approximately another 50 were laid off and the salary system was changed into a variable pay scheme. at the end of the period, which also demarcates the end of this research, the company had shrunk to 600 employees again. each consultant’s story includes their vitagram (figures 1 4) which was produced as part of the interview. the vertical line on the vitagram marks a common point in time, and from this 74 australasian journal of information systems kautz & bjerknes 2015, vol. 19, pp. 71-95 tales of it consultants point the stories take place in the same time period. each story focuses on what is special for each consultant. common features of the stories, i.e. things that are mentioned by two or more consultants, are collected at the end of this section in a separate subsection. 3.1 art’s story when art started, he was very pleased with the organization. a good social environment prevailed and he soon got assignments to work for customers. although he at one point withdrew a little from the general social life in the organization, he still participated in the company’s internal life as an active member of one of the company’s professional core experience groups. satisfaction time figure 1: art’s vitagram the first time he became unhappy and felt let down was when he was assigned to a project with another consultant, who was not competent to perform the task. art informed his manager about this, but nothing happened. he had expected to either get an additional competent person or a competent replacement for the one who was on the project, so that the company would have a real chance to deliver a good result. this did not happen, but art considered it a one-off event. but when art was assigned to the next project and the same happened again, his motivation decreased dramatically which he expressed through the dashed line in his vitagram. nevertheless, art was still content, but that was mostly because the project gave him an opportunity for personal growth and that was satisfactory. his commitment to the project was strong, but the commitment to the company became less and less. the final ‘blow’ which led to art’s decision to resign was one of the last lay-off rounds. art experienced the lay-offs as deeply unfair and felt that they struck blindly. when asked about the organizational culture, art replied that the company had no clear understanding anymore of what the business should be and what the organization should specialize in. in his view this made it difficult to make personal choices that would increase the likelihood that one would keep one’s job in times of cutbacks and layoffs. 3.2 bee’s story bee started as a very satisfied employee. the first major downturn came about 1990, when the original company had two rounds of layoffs before it was itself acquired by another organization – the vitagram shows this as the first trough. bee felt that it was very hard to stay behind when her social network disintegrated and good colleagues disappeared. the reason why she did not leave herself at the time was that there was much uncertainty in her private life that it would have been too unsafe to switch jobs at the same time. the situation improved again and she was happy and thought there was a good balance between a focus on earnings, short-term gains and competence development, the opportunity for long term gains. eventually, however the weight tipped over and there was less focus on competence 75 australasian journal of information systems kautz & bjerknes 2015, vol. 19, pp. 71-95 tales of it consultants development and she grew more and more frustrated as there was no longer a correlation between the tasks she was assigned to and the time she had available and she felt quite alone. so she handed in her notice – the second, slightly smaller trough in her vitagram but when the organization decided to focus on something she was interested in, she changed her decision and stayed. her joy however proved short-lived. the new unit she had joined consisted of people from many of the newly merged entities and it turned out that there was no real consensus on what the unit would work with. as the unit did not produce financial results fast enough, it was closed down. the company’s management after all did not wish to focus on this area. at this point her motivation decreased rapidly (the last great wave valley in the vitagram). bee's personnel manager recognized this and got her quickly out in a project with a customer believing that her discontent would go over. but it did not. satisfaction time figure 2: bee’s vitagram she felt obligated to the project, but when the time came to finalize the project, she had thought of looking for a new job. this decision was further strengthened by the last rounds of lay-offs. when management decided to use such means, which she disagreed with very much, she lost faith in management. therefore it was not hard to say "yes" when a new job offer came. bee described the changes in the organizational culture as a process, in which different values influenced the culture. at the start of the company the focus was on creativity, cohesion and well-being. then after the acquisition the first slump came with an engineering culture and focus on rationality and numbers. then came a period of growth and focus on the stock exchange float the solid vertical line in the vitagram represents this period, followed by a period of balance between short and long term gains and goals, and finally a period, where management had not been willing to pay ‘the price for the growth of the company’ by supporting its employees. 76 australasian journal of information systems kautz & bjerknes 2015, vol. 19, pp. 71-95 tales of it consultants 3.3 curt’s story satisfaction time figure 3: curt’s vitagram curt was also very happy at the start of his employment, but gradually his well-being deteriorated. he defined well-being as related to enthusiasm, commitment, motivation, and loyalty, which he believed had to be in place to be able to do a good job, and which has to be shared by co-workers and staff for an organization to be a good place to work. there were incidents that affected his well-being, such as when promises were not kept; for example it was said that professional development would count positively for salary and bonus determination, but it did not. there were also disappointments with regard to his expectations of management: he perceived leadership as clumsy, lacking reasonable thought and sometimes being unethical. as the organization’s growth increased, simultaneously the workforce’s general discontent rose and this also affected his well-being. when the redundancy packages were initiated, in addition to the layoffs and the introduction of variable pay, i.e. a 15% decline in salary, curt thought that the job situation and his fixed income became too uncertain (first trough in the vitagram). however, when management withdrew the proposal for the variable pay model, he felt that his income was safe enough and determined to stay (next peak in the vitagram). but when the variable pay model was introduced again, he decided to leave and found himself a new job. curt thought that the company changed from being an organization with great openness, co-determination, and participation, where individual differences were seen as strength, and governance was executed according to individual needs, to an organization, which was completely closed, without co-determination and participation, where individual needs were ignored, and where governance was entirely executed according to economic principles. those who managed and rewarded only saw numbers, not people. 3.4 di’s story di started as a consultant, but eventually became a manager. she was mostly content, right up to the point where a reorganization of the company in terms of market areas meant that she got a new superior. there her motivation and enthusiasm sank dramatically (the steep downward curve in her vitagram) as her new boss was not a collaborator; he merely gave orders. at the time di was head of a department with 70 80 employees and she had three submanagers who reported to her and she divided the direct responsibility for the staff between them. cooperation was important for her and she felt that the collaboration between her and the three managers in the department worked very well. she also thought it was part of her job to work together with other managers, both the top management team, and managers across the individual units, who took care of the different subjects and market segments to further develop and prepare the company for the future. but she was not allowed to do so. as a consequence she felt overlooked by the management team and felt that the group did not work well. 77 australasian journal of information systems kautz & bjerknes 2015, vol. 19, pp. 71-95 tales of it consultants satisfaction time figure 4: di’s vitagram these feelings of suboptimization, not being allowed to contribute with her competence, and inconsistent, unequal treatment of employees were confirmed when the redundancy packages were to be awarded. management did not look at the employees’ collective skill set, but focused on departmental performance, although they knew that the company would be reorganized again in the following year, this time back along subject fields and not according to market segments. she resigned as a manager and became a consultant again. at this point it was also decided that the company would physically move to another location, which would mean a significantly longer travelling distance for her. thus she contacted someone she knew in the headhunting industry and soon got job offers. when she stopped her unit manager was surprised, as she believed that di was happy because she was out on a customer assignment. di described the biggest change in the company as going from being a company, where people cared about and stood up for each other to an organization, where people did not care, nor stood up for each other, i.e. a company where solidarity and social cohesion disappeared; among others the social environment had changed a lot. she thought that this probably was a consequence of the rapid growth and the many mergers. 3.5 similarities in the consultants' stories the consultants all have different and personal reasons why they chose to quit, yet there are the following similarities in their stories: (1) all four had emotional reasons for quitting: job insecurity (art and curt), not being appreciated as a human being, but only be considered a billing machine (art, curt and di), a feeling that management was not to be relied on (bee and curt). (2) for three of the four it was one event, the lay-offs, which meant that enough was enough, but mentally they had quit before this event.1 (3) it took time from when the decision to find a new job was taken until this decision was actually implemented. three of the four said that in this period they had withdrawn from the company, but they felt committed to the projects they were in and to the customers they were working for. (4) none of the four said anything about whether their direct managers had done something to persuade them to stay; admittedly, bee and di reported that their leaders 1 those who were laid off were either without an assignment or not key personnel in a tender at that time; being without a contract or not being key personnel in an offer affected everyone every now and then, no matter how good they were, but it varied as to how long a consultant would be without a contract. to lay employees off on the basis of such a short time frame was seen as random, deeply unfair and as very disloyal to the employees. 78 australasian journal of information systems kautz & bjerknes 2015, vol. 19, pp. 71-95 tales of it consultants had worked to get them onto customer assignments and thought that this was sufficient; but this was not perceived as doing something serious to change the overall situation. (5) two of the four were explicitly disappointed with the way their employment was terminated as no one at the company cared about the fact that they stopped at all or expressed any regret. (6) all appreciated their colleagues at the company, who were mainly described as clever, willing to provide advice and help when they were asked as well as being interested in doing a good job for the customers. (7) the four vitagram graphs all show a correlation with respect to time; all four consultants were very happy and content for a long time, with discontent emerging during the period of strong growth, the focus on shareholders and the stock exchange float and the ‘big merger’. (8) all four went to new employers where the content of their job was similar to the job content they had as employees of the company; therefore it was not boring assignments or too little challenging tasks which made them quit. (9) all four were glad that they had stopped and that they had drawn the consequences of their personal grievances; some of them also said that it was time to change job, but this was not stated as a reason why they started looking for a new job. all four consultants explicitly referred to their loss of motivation in their tales. the concept of motivation has in the literature been related to the concept of psychological contract (frey 2002), we therefore introduce in the following these concepts in more detail to inform our study and reflect upon our findings. 4 theoretical considerations rousseau (1989, 1995) who is credited with the key development of the concept of psychological contract in its current form (guest 2004) has defined it as “an individual’s belief in mutual obligations between that person and another party such as an employer” (p. 544 in guest 2004); morrison and robinson (1997, p. 229) broaden this definition by asserting that “these obligations are based on perceived promises and are not necessarily recognized by agents of the organization.” to emphasize that the metaphor of a contract involves at least two parties and following morrison and robinson (1997) who point out that although prior researchers have noted that reciprocity is a central aspect of psychological contracts, but that its role has been largely absent from research on psychological contract violation, guest (2004, p. 545) extends the meaning of the term as follows: “the perception of both parties to the employment relationship, organization and individual, of the reciprocal promises and obligations implied in that relationship.” guest (2004) argues that the primary emphasis of the psychological contract in an employment relationship is on the individual level between the employee and employer and puts forward that such an understanding provides an opportunity to explore the processes and content of the employment relationship through a focus on more or less explicit deals; these deals are likely to be re-negotiated or modified over time, to be influenced by contextual factors, and to have a variety of consequences. guest (2004) asserts that beyond the organizational context, the wider economic, social, and legislative context is essential for contemporary employment relations and presents an extensive number of factors that might influence the employee-employer relationship: at the individual level these are related to personal circumstances such as age, gender, education, ethnicity, tenure, income, type of work, hours worked, employment contract, work values and career anchors; at the organizational level he suggests factors such as industry sector, company size, ownership, trade union recognition, hr policies and practices, employment relations, and organizational culture. he also contends that beyond analysing the reciprocal promises, inducements, and obligations, there is a need to explore the employee’s and employer’s understanding of the nature of the promises and obligations and the extent to which they have 79 australasian journal of information systems kautz & bjerknes 2015, vol. 19, pp. 71-95 tales of it consultants been delivered and the circumstances under which there is positive fulfilment of the psychological contract. this includes the necessity to address the state of the psychological contract, comprising the relationship between fairness, trust, and delivery of the deal. for the latter, but not the other factors, he poses the question whether they can be conceived in either a virtuous or vicious circle to investigate their effect on the attitudes and behaviour of all parties involved in the relationship. a last crucial element of psychological contract research according to guest (2004) concerns the outcomes of perceived contract breach and the need to know more about when they begin to matter to an extent that has significant attitudinal and behavioural consequences in the form of organizational commitment, work satisfaction, worklife balance, job security, motivation, stress, work attendance, job performance, organizational citizen behaviour and intentions to stay or quit. 4.1 influences impacting on the psychological contract as a psychological contract is exposed to a number of influences, its state might change in the course of an employment and it therefore needs to be maintained. cialdini's (2001a, b) theory of social influence and persuasion provides a starting point for understanding and identifying the influences a psychological contract is exposed to and how to uphold an acceptable psychological contract. according to cialdini (2001a, b) people are influenced and persuaded to (accept requests to) change attitudes, beliefs and behaviours without further thinking about them through ‘automatically’ initiated influence processes. he presents six principles that initiate these processes and mechanisms for how they can be applied: (1) reciprocity: people repay in kind give what you want to receive; (2) commitment and consistency: people like to be consistent and align with clear commitments make their commitments active, public, and voluntary; (3) social proof: people follow the lead of similar others use peer power when possible; (4) liking: people like those who like them uncover real similarities and offer genuine praise; (5) authority: people defer to experts expose your expertise; do not assume that it is selfevident; (6) scarcity: people want more of what they can have less of highlight unique benefits and exclusive information. reciprocity and consistency evolve over time; the other four principles can occur instantaneously, but may still affect a long-term development (carr 2001). the principles are now explained in more detail and related to the concept of the psychological contract. 4.2 reciprocity reciprocity is fundamental for the maintenance of a psychological contract (carr 2001). a psychological contract involves mutual influence, interaction and negotiation between the employee and the employing organization (carr 2001). carr (2001) argues that the relational processes between the employee and the organization are “deep-seated; largely unconscious; intimately connected to the development of identity; and, have emotional content. indeed, perhaps the term seduction is more appropriate than notions of mutual influence and negotiation, which are more often associated with high degrees of consciousness and rationality” (p. 429). two key terms relating to reciprocity in a working relationship are identity and emotionality. according to carr (2001) employees are encouraged by their employers to identify themselves with the organization’s features, values and culture. identification involves developing emotional ties to the organization, and that employees surrender some of their own identity to the organization. the ties are strengthened when employees achieve recognition and rewards through their work. such ties create a feeling or sense of belonging, which is related to the concept of cohesion. cohesion is about how much an employee as an individual identifies with and/or is bound to a group. cohesion is the social force that causes individuals to join a group, the strength of a group to define itself as a unit, 80 australasian journal of information systems kautz & bjerknes 2015, vol. 19, pp. 71-95 tales of it consultants and the strength to remain a unit (forsyth 1999, svedberg 2000). the difference between belonging and cohesion is that belonging is an emotional power while cohesion is a social force. cast and burke (2002) discuss the relationship between identity, emotionality and self-esteem. identity is tied to individuals’ expectations of their own and of others' behaviour. when expectations are not met, negative emotions like anxiety and anger are built up. individuals address this by adjusting their own behaviour or their expectations of others' behaviour. if the gap between expectations and their fulfilment is too big, the individuals can either remove themselves from the situation or shed their identity. when expectations are met, a confirmation of the relationship to the others is achieved and self-esteem rises. a strong selfesteem serves as an emotional reservoir if expectations turn out not to be met, and this can contribute to stability in periods of change and negative emotions. furthermore, people with high self-esteem have greater cognitive resources that enable them to deal with unsatisfactory situations in a more effective way than people with low self-esteem. those who get most confirmation that they do a good job will have the greatest self-esteem. thus, they have the best basis for dealing with an emotional conflict, and it is reasonable to assume that they are the first to quit their jobs if the emotional conflict is too big. this is in line with britt (2003) who has shown that in conflict situations those who care most about a job become those who are most demotivated and find a new job; those who stay are often the ones who do not care, a finding consistent with social exchange theory (blau 1964, forsyth 1999). a psychological contract contains the criteria that must be met such that employees feel that they get a sufficient return on the investment which they make in relation to the organization. if over time an employee gets less back than what they have invested (according to their own opinion), the employee will leave the relationship and the organization. 4.3 commitment and consistency consistency in connection with a psychological contract is the conformity between the contract's content and its actual fulfilment, i.e. whether the commitments entered into are experienced as being met. people relate to clear commitments (cialdini 2001b). since the commitments in a psychological contract are largely implicit and not overt, they concern the individuals’ expectations about an organization. if expectations are met, an employee will fulfil the obligations they feel they have in relation to the contract, thus giving back in accordance with what has been given (see above, the reciprocity principle). if an employee gets less than expected, they will correspondingly give less than they initially felt obliged to give and if the employee gets more, they feel obliged to give more. however, inconsistency creates anxiety and anger. employees’ reaction on inconsistency, i.e. lack of behavioural integrity, is that trust and loyalty are weakened (simons 2002). reichheld (2001) argues that business management cannot expect employees to be loyal if they are not perceived themselves as loyal towards their customers and employees and concludes: “outstanding loyalty is the direct result of the words and deeds the decisions and practices of committed top executives who have personal integrity” (p. 76). simons’ (2002) work in the hospitality industry confirms this; it shows that organizations where employees perceive that managers act consistently in relation to their obligations – they do what they said they will do were substantially more profitable than others, and concludes that lack of trust comes at a high expense. kim and mauborgne (2003) relate lack of trust and loyalty to inadequate processes in organizations. they argue that it is more important to have a fair process than to achieve a fair result, because a fair process provides emotional support i.e. it reduces negative emotions. a fair process has three characteristics: it engages, it explains and it sets clear expectations. a fair result perhaps yields what one thinks one deserves, but nothing more. according to the reciprocity principle there is thus no reason to give something extra back to the organization. kim and mauborgne (2003) argue that the difference between an expected and a good result corresponds exactly to the effort, which is put in voluntarily as part of a reciprocal relationship. commitment can contribute to this in that employees remain in a relationship or an organization for some more time after negative feelings have arisen (cast and burke 2002). the period during which the employees have negative emotions, but wait and see, presents an 81 australasian journal of information systems kautz & bjerknes 2015, vol. 19, pp. 71-95 tales of it consultants opportunity for an organization to change behaviour by doing something in relation to the negative emotions, if it recognizes the situation, or for the employees to adjust their expectations. this provides the basis for a revised contract, where obligations and commitments are renegotiated. 4.4 the other principles of influence as influence is exerted through the exercise of power (bråten 1981), it makes sense to relate the principles of social proof, liking and authority to the kinds of power which exist. forsyth (1999) distinguishes between: (1) legitimate power power that comes from a ruler’s legitimate right to ask for and demand obedience; (2) reward power the capacity to promise or control the allocation of a reward; (3) punishment power the mandate to threaten and punish those who do not conform to a request or requirement; (4) reference power influence based on identification with, attraction to or respect for those in power; (5) expert power influence based on the belief that those in power have superior abilities and skills; (6) information power influence based on the potential use of information resources, including rational arguments, persuasion or actual data. bråten (1981) has coined the notion of model power as someone’s power to determine ways of understanding and talking about a phenomenon. influence in the form of social proof occurs via ‘the others’ and their perceptions of whether their own psychological contract is fulfilled. if the others are satisfied and happy, this affects one’s own well-being positive and if the others feel distressed, this affected one’s own well-being negatively. this influence principle can best be controlled by the exercise of power of information, based on the rule ’what people do not know, does not hurt them’. liking relates to ‘someone one identifies with and/or looks up to,’ as distinct from social proof, which relates to ‘someone else’, the others. when someone one knows and likes has an opinion about something, this has a stronger effect than when some unspecified others express this opinion. liking strongly relates to reference power. authority can be understood in two ways, as expertise (cialdini 2001b), or as based on the mandate provided through a specific role. experts influence through expert power and informational power, they are trusted and their advice is followed because they know best. designated, formal leaders are important authorities in an organization. managers have legitimate power by virtue of their roles. they also have reward power, punishment power and information power. a manager can affect the process of upholding a psychological contract by influencing the formal contract. terminations, layoffs, farewell packages and salary cuts are examples of elements that affect the formal contract, but which also affect the psychological contract because such measures are often associated with negative emotions (carr 2001). such measures will also be experienced as the exercise of punishment power. furthermore, a leader can influence a psychological contract by making requirements concerning results, which in practice are new expectations for behaviour or by exercising reward power in the form of, for example, provision of training, bonus and/or salary increases. in addition, managers have the opportunity to practice various forms of information power such as control of the agenda (bachrach and baratz 1962) by determining what is informed and communicated about or model power (bråten 1981) by creating strategies and structures for interaction. the exercise of power by a manager can thus have effects which lead an employee to believe that the organization does not appear to be consistent and thus does not fulfil its part of the psychological contract. it is not the exercise of leadership itself that affects whether the psychological contract is considered as fulfilled or not, but the perception of the exercised leadership. 82 australasian journal of information systems kautz & bjerknes 2015, vol. 19, pp. 71-95 tales of it consultants finally, scarcity can affect the psychological contract as it affects the scope of action when an employee is experiencing emotional conflicts. if many jobs are on the market, it is easier or more tempting to try to find a new job than if there are few jobs. as discussed with regard to reciprocity, it is not certain that everyone will react alike to scarcity. consultants who do not get their identity confirmed through their work have lower self-esteem than those who get confirmation that they do a good job. those with lower self-esteem may not seek other jobs in times of job scarcity, even though they might like to do so. thus, scarcity can be an influence, which does not affect those with high self-esteem, but which leads those with less self-esteem to stay. 4.5 the content and state of the psychological contract morrison and robinson (1997) as stated earlier emphasise the significance of reciprocity for the content and state of a psychological contract and put forward that “the psychological contract held by an employee consists of beliefs about the reciprocal obligations between that employee and his or her organization.” (p. 226) the content and state of the psychological contract is related to finding meaning and being satisfied in one’s job and with one’s work (morse 2003), to be allowed to do one’s best (britt 2003), to feel enough certainty to grow with a task, and enough uncertainty that it becomes necessary to grow with a task (moxnes 2000). it is also related to emotional support and belonging (forsyth 1999), to engagement, to clear expectations and to explanations why decisions are taken (kim and mauborgne 2003). people engage in their work for the challenge and the enjoyment of it, the work itself is motivating; amabile (1998) calls this intrinsic motivation. as such intrinsic motivation is an important part of a psychological contract. in refining the general relationship between the psychological contract and motivation, frey (2002) supports this by stating that a psychological contract includes a reciprocal appreciation of intrinsic motivation. de lange et al. (2011) have examined the direct relationship between psychological contract and intrinsic motivation in a quantitative study and found that relational contract violation predicts lower work motivation. amabile (1998) has earlier identified six categories of management practice, which when utilized, promote intrinsic motivation, and when neglected inhibit it. each category also includes some pitfalls and management practices which inhibit intrinsic motivation. the categories and pitfalls which contribute to the content and state of a psychological contract, according to amabile (1998), are: (1) challenge: is about assigning the right task to the right person, so the employees have something to strive for; if a task is too little challenging it becomes boring, but if it is too challenging it can create fear of losing control; the pitfall here is to give a task to someone, who incidentally has no actual assignment. (2) freedom: is about setting unambiguous and clear goals, but to give the employees influence concerning the way to reach the goal; it is important that the goals are stable over a relevant time period; pitfalls here are a) to change goals too often or not to define the goals properly, and b) to provide the illusion of free choice in a process that is predetermined. (3) resources: is about allocating time and money; insufficient resources give no space for challenges, or make it impossible to get a task done; too many resources will also affect the results negatively; the pitfall here is to miscalculate the actual need for time and money. (4) work-group features: is about putting together a working group that consists of people with sufficiently diverse backgrounds and of people, who can form a group whose members are mutually supportive; the pitfall here is to put together groups of employees, who are too similar, their members will easily cooperate, but they will hardly be able to create something new. (5) supervisory encouragement: is about the feeling to mean something and about providing the feeling that an employee does matter to the organization or to an 83 australasian journal of information systems kautz & bjerknes 2015, vol. 19, pp. 71-95 tales of it consultants important group of people; pitfalls here are a) not to show attention and b) to meet employees’ new ideas with too much criticism. (6) organizational support: is about the procedures and mechanisms to promote and reward creative solutions, particularly by facilitating information sharing and collaboration; pitfalls here are a) to reward in the form of money that gives the employees the feeling of being bribed and b) to install policies and reward systems, which inhibit information sharing and collaboration. 4.6 a cyclic process model of the maintenance or neglect of the psychological contract the empirical findings of our study uncover elements that can be consolidated into a cyclic process model of the maintenance or neglect of a psychological contract. employees enter a psychological contract with their employing organization when they start an employment relationship (rousseau 1995, morrison and robinson 1997, carr 2001, guest 2004). the data documents the relationship between the psychological contract and intrinsic motivation (amabile 1998, frey 2002) and the literature confirms that maintaining and upholding intrinsic motivation is a vital part of the content of the psychological contract (frey 2002). the four interviewed consultants had expectations concerning the company’s behaviour, in particular in regard to mutual obligations and commitments. the literature on social influences, which emphasizes the role of reciprocity, commitment and consistency and associated expectations for a psychological contract and its fulfilment, reinforces this finding (cialdini 2001a, 2001b, carr 2001). cialdini (2001a, b) also puts forward that an employee's perception of whether the organization acts consistent with the psychological contract is influenced by the principles of social proof and liking, and that revisions and adjustments of the psychological contract can be affected by the principle of scarcity, especially if emotional conflicts arise. the consultants explained that they compare their expectations with their experiences and revealed positive and negative feelings such as confirmation of their status and their relation, trust and loyalty to the organization, but also their diminished trust, abated loyalty, anger and anxiety as a consequence of that comparison. authors such as cast and burke (2002) and kim and mauborgne (2003) back up these experiences and the described expectation gaps as they relate issues of identity and emotionality, but also of self-esteem, to trust and loyalty. as an effect of the expectation gaps and the accompanying feelings and emotions the consultants also expressed how their motivation, which the literature qualifies as intrinsic motivation (amabile 1998, frey 2002) in form of their sensations of exclusion, belonging, closeness and distance to the organization, either remained stable or increased or decreased. this has been discussed in the literature by forsyth (1999), svedberg (2000), carr (2001) as well as cast and burke (2002) who refer to the concepts of sense of belonging and cohesion. the current status, including any change, of their intrinsic motivation led the consultants to recurrently assess their actual relation to the company that is to revisit their psychological contract and to either maintain or adjust its content by altering their expectations, but also their commitments and obligations. subsequently these expectations were regularly compared with the organizational behaviour shown by the company and the consultants mentioned a number of managerial practices and the lack hereof which can be linked to the practices of challenge, freedom, resources, working group features, supervisory encouragement, and organizational support as identified by amabile (1998). these practices which are also related to the aforementioned social influences of reciprocity, commitment and consistency, but also to management’s authority (cialdini 2001a, b, carr 2001) and their exercise of power (bråten 1981, forsyth 1999) impacted their ensuing experiences. 84 australasian journal of information systems kautz & bjerknes 2015, vol. 19, pp. 71-95 tales of it consultants in the course of time the consultants went through several of these cycles of managerial practices impacting social influences and creating expectations and expectation gaps with their positive and negative effects on intrinsic motivation and ultimately their psychological contract. their commitments followed an immanent need to give more back when they received more than expected (cialdini 2001a, b) and less when they received less than expected (kim and mauborgne 2003). the consultants, who initially all were content, i.e. had a strong sense of belonging to the company, all had an emotional surplus called an emotional reservoir by cast and burke (2002). even in bad times and after a bad cycle they did not lose all their intrinsic motivation nor did they terminate their psychological contract and employment relationship at once, but hoped for improvements, waited and observed. these were attempts to re-establish their psychological contracts to become again consistent with the commitments, which they had made through their original contracts. they experienced both good cycles and bad cycles. a good cycle in the process of maintaining the psychological contract was characterized by a situation where their expectations were more than fulfilled, they got more back from the organization than what they expected and as a consequence their sense of belonging to the company either kept level or was strengthened. the consultants adjusted their contracts and altered their commitments and obligations to the company as well as their expectations and in line with the principle of reciprocity responded with an aspiration to give even more back to the organization than what they were obliged to. the consultants however more often got less than expected, their sense of belonging to the company was weakened and they decided to give less back than what they initially felt obliged to these were bad cycles in the process of maintaining the psychological contract. thus, when no intrinsic motivation remained due to repetitive non-fulfilment of expectations and negative evaluations, they terminated both the psychological and their formal contract and left the company. this continuous process which we derived from the empirical data and which is substantiated by different segments of the literature integrated here into a coherent whole can be synthesized and depicted in a cyclic process model of the maintenance or neglect of the psychological contract (see figure 1). 85 australasian journal of information systems kautz & bjerknes 2015, vol. 19, pp. 71-95 tales of it consultants figure 1: a process model for the maintenance or neglect of the psychological contract the model follows the logic of a process theory, which means that causation consists of necessary conditions in a sequence (markus and robey 1988) where the absence of one such required condition will impede the outcome from occurring. 5 the tales and the psychological contract: an analysis we can now revisit the consultants' tales and relate them more closely to the cyclic process model for the maintenance or neglect of the psychological contract. first, we focus on the relation between the consultants’ actual expectation gap, the social influences, their intrinsic motivation and their psychological contracts. secondly, we concentrate on the relationship between the content and state of the psychological contract and the identified management practices. finally, we relate the tales to bad cycles captured in the model which result in the total loss of intrinsic motivation, the end of the psychological contract maintenance process, as well as the psychological contract and the consultants’ voluntary job termination. 5.1 the tales, expectations, influences, intrinsic motivation and the psychological contract from the consultants’ tales we recall that: art first experienced a disappointment with a poorly composed team of consultants, scaled his expectations down and withdrew mentally. when the same letdown returned, he decided to stop. this decision was however not implemented before the lay-off came, then he handed in the notice without having a new job to go to. bee had a downturn at the beginning of her career in the company. then her motivation went up; her expectations were met, and they actually rose. when the 86 australasian journal of information systems kautz & bjerknes 2015, vol. 19, pp. 71-95 tales of it consultants company focused too much on short-term profits and too little on long-term survival, her motivation went down, but went up again when the organization focused on something she was interested in and finally straight down when the company nevertheless didn’t focus on that area. she left when she was offered a new job. curt had many setbacks over a long period. his expectations of the company were met over time and towards the end of his employment he felt that the company management was stingy and uninterested in its employees. his attitude became ‘why give something to the company when the company has nothing to give?’ the economic insecurity caused by the variable pay model resulted in his final decision to go, although he had had a small resurrection when he thought that the company would not introduce variable pay after all. di had a number of setbacks over a very short period. in her role as a manager she was disappointed about the lack of cooperation, recognition and mandate for her work which did not meet her expectations. she also disagreed with the way the redundancy packages were determined, but persevered as a consultant. however her sense of belonging to a company which she described as careless, unfair and showing a lack of solidary had weakened too much, thus when a proposition for a new job came, she accepted the offer and left the company. the consultants’ tales exhibit some common features in terms of maintaining their psychological contract: (1) there as a relation between identity and emotionality, specifically, negative emotions, and sense of belonging. all four consultants stated emotional reasons for terminating their jobs. negative emotions, such as resentment, disappointment, mistrust, but also angst led to a decrease in their sense of belonging to the company, but the loyalty to the customers did not drop, neither in the company in general nor by the consultants in particular. they felt committed to their customers and they also perceived that their colleagues were striving to do a good job for the customers. the consultants’ self-esteem did not fall either; none of the four consultants had experienced problems in relation to being without a contract for a long time period and had survived several rounds of cutbacks and layoffs. in a consulting firm many contracts give status salary and bonus as contracts mean billed hours. contracts also mean more contracts because it is easier to get a new assignment when a consultant has been on assignments before; here the customers are affected by social proof. to put it forcefully: contracts and assignments are a basis for expert power, reference power and information power. all of the four consultants went on to new jobs with similar content to what they had in their positions at the company. (2) social proof, liking and scarcity had an impact on perceptions of consistency, but were not decisive in relation to making the decision to leave their job. none of the four consultants ceased their employment because someone in particular had also done so; the principles of social proof and liking did not influence them in these respects, although both bee and curt stated that when many other employees left and others started, something happened to the identity of the employees and their sense of belonging in general. yet this was not their major reason to terminate their employment contracts. the consultants’ self-esteem with regard to their assignments and contracts was large enough that a scarcity of jobs did not prevent them from quitting either. however they did the only rational thing from their perspective, they moved from a business which they no longer wished to be part of and continued with what they were good at somewhere else. (3) the consultants went through several, mostly negative cycles in the model. 87 australasian journal of information systems kautz & bjerknes 2015, vol. 19, pp. 71-95 tales of it consultants all consultants had several rounds of negative cycles. each round in a negative cycle affected the consultants’ commitment to their psychological contract as they became more and more dissatisfied. as their vitagrams indicate they probably expected less from the organization, but they did not quit. in this way they gave the company a chance to change its behaviour. the vitagrams also show that it is possible to reverse a negative trend. this is perhaps most evident in bee’s case, where the curve went from way down up to high again, but also in curt’s case where a small upwards turn occurred. the reversal of the negative trends happened when the consultants perceived that the company changed its behaviour, but the taken measures were not sufficient. 5.2 the tales, the management practices and the consultants’ psychological contracts the consultants’ tales are strongly related to management practices (amabile 1998), which impacted the state of their psychological contracts and their intrinsic motivation as part of the content of these contracts. an analysis of these management practices contributes to an explanation of why the consultants left their jobs. the reason why art resigned was a combination of little challenge, wrong work-group features, lack of organizational support, and too much freedom: he did not get the right job assignments with regard to his expertise, he was assigned to project groups which were composed of people with the wrong skills, and he felt that no one in management did anything with these issues. this happened twice. art also thought that the organization’s objectives and goals were too vague and very unclear to the employees. the first time bee resigned can be attributed to a lack of supervisor encouragement from management and a lack of resources as her tasks did not fit with the time that was available. however she stayed because she got a new challenge. when she finally resigned, the reason was that management did change goals again and did not hold on to those goals she worked towards long enough that reaching any results was possible, i.e. her reasons were a combination of lack of freedom and resources. curt finally quit because of a lack of supervisor encouragement from management; he provided several examples of that it did not matter when the consultants put in an extra effort; it was just not seen. in addition, there was a lack of organizational support, where the reward system for managers led to many management activities that were related to their goals, not the organization's goals. curt was the most outspoken with regard to the fact that his intrinsic motivation disappeared, which he related to his well-being, enthusiasm, commitment, motivation, and loyalty and considered absolutely necessary for a good work place for oneself and others. di felt a lack of supervisory encouragement and lack of organizational support; the ultimate reasons why she stopped were primarily related to the characteristics of her work group. the management group did not have the qualities of a good group where “first, the members must share excitement over the team's goal. second, members must display a willingness to help their teammates through difficult periods and setbacks. and third, every member must recognize the unique knowledge and perspective that other members bring to the table.” (amabile 1998, p. 83) some of the similarities in the stories are also related to the management practices. one of the managers at the company believed that it was enough to get consultants out into customer projects to get them to stay, but that did not help. we can speculate that these assignments were not challenging enough to increase the consultants’ intrinsic motivation. based on the considerations that the sense of belonging to the company was already weakened, it is reasonable to assume that more supervisor encouragement from management, i.e. interest and commitment from the managers’ side would have been necessary to convince the consultants to stay. 88 australasian journal of information systems kautz & bjerknes 2015, vol. 19, pp. 71-95 tales of it consultants three consultants said that the lay-offs, which were a dramatic change in some colleagues’ formal contracts, led them to lose all trust in management and as such were key events with regard to deciding not to go more cycles in the process of maintaining their psychological contracts. the lay-offs were perceived as a fundamental lack of managerial supervisor encouragement and organizational support. the employment terminations, where employees were ‘thrown out’ of the organization without so much as ‘thank you and good luck’ admittedly did not explicitly influence the consultants’ intrinsic motivation, but confirmed that it had been the right decision to leave the organization. consultant managerial practice art bee curt di challenge freedom resources working group features supervisor encouragement organisational support table 1: managerial practices neglecting the psychological contract in summary, all the consultants stated management practices, which inhibited intrinsic motivation as their reasons for terminating their employment (see table 1). 5.3 the tales, bad cycles and termination of the psychological contract the consultants all experienced mostly negative cycles of the maintenance of their contracts and ultimately left the company because their psychological contracts were all poorly maintained or more precisely neglected as they went through several cycles. the cycles were bad because management did not deploy management practices that provide the conditions to uphold intrinsic motivation. we have described the gap between all employees’ expectations and their experiences which resulted in negative emotions including reduced trust and loyalty and led to a situation where the consultants resigned mentally long before they physically left. as nothing was done to increase their intrinsic motivation to work for the company they left, but eventually continued in quite similar jobs at different employers. the process started with a psychological contract being entered into between each of the four consultants and the company. a bad cycle then commenced when the company did not meet the employees’ expectations concerning social influences, in particular reciprocity and commitment and consistency, and less so authority including exercise of power, social proof and liking. scarcity of jobs on the market did not play any role. this expectation gap with the associated negative emotions resulted in the consultants' intrinsic motivation, in particular their sense of belonging to the company, being weakened. the weakened relationship with the company subsequently led to the consultants’ revision of their psychological contracts with the outcome that their faith in and loyalty to the company compared to the trust and devotion, which they initially had, decreased. their loyalty to customers however was sustained. both the original and the revised psychological contract were exposed to management practices such as a shortage of challenges, too much freedom, dearth of resources, paucity of supervisory encouragement, and lack of organizational support which continuously impacted the consultants’ experiences of social influences and created a widened expectation gap which lead to further adjustments of the psychological contract with regard to expectations. the consultants went through several bad cycles with some of them experiencing small improvements in between cycles, and all of them waiting for change for the better until at one point there was no intrinsic motivation left and they terminated their employment. the eventual lack of organizational support in form of the layoffs which were perceived as unfair, and in one case the variable pay model which was considered as unjust, provided the ultimate 89 australasian journal of information systems kautz & bjerknes 2015, vol. 19, pp. 71-95 tales of it consultants cause for the loss of motivation and the termination of the psychological and the formal contract. 6 discussion in this study we have argued that employees in an it consulting firm terminated their employment with the organisation because their psychological contracts were poorly maintained throughout several cycles. the poor maintenance consisted mainly of management practices, which destroyed the consultants' intrinsic motivation. by relating and combining the empirical findings of our study with insights concerning the psychological contract (rousseau 1995, morrison and robinson 1997, carr 2001, guest 2004), the principles of social influence (cialdini 2001a, b), theories of power (forsyth 1999, bråten 1981) and of the relation between self-esteem, identity and emotionality (cast and burke 2002) with the principles of social influence (cialdini 2001a, b) and categories of management practices(amabiles 1998), which promote intrinsic motivation, we established a general process model of how psychological contracts are maintained or neglected. we conceptualize and demonstrate the maintenance or neglect of the psychological contract as a continuous, cyclic process, where the cycles can be ‘good’ or ‘bad’, as seen from an employee's perspective. the tales of the four consultants can be understood in terms of the model. the primary reasons why they were less satisfied with working in the case organization could be related to the pitfalls of management practices (amanile 1998), which damage intrinsic motivation. previous research has dealt with the concept of psychological contracts differently. morrison and robinson (1997) present a linear causal logic model that focuses on the development of violation of the psychological contract. it outlines the sense-making processes preceding an employee's experience of a contract violation based on two key factors: reneging, organizational agents knowingly breaking a promise to an employee, and incongruence of the employee’s and agents’ understandings of a promise. a perceived unmet promise creates a discrepancy between an employee's understanding of what was promised and her perception of what she actually has received. the perception of this discrepancy triggers a comparison process, whereby the employee considers how well each of the parties has upheld its respective promises, but the model pays no attention to the consequences, nor does it address possible feedback loops or cycles. out of his extensive list of identified factors (see the ‘theoretical considerations’ section) guest (2004) develops a factor-based, uni-directional causal framework for applying the psychological contract to the employment relationship. he takes contextual, background, policy and practice factors as well as characteristics of the psychological contract and its actual state into account and deduces attitudinal and behavioural consequences. his model considers the intention to stay or quit as a possible outcome, but does not go further. based on the insight that perceptions of fairness evolve over time jones and skarlicki (2013) present a dynamic model of organizational justice in which they describe the processes through which perceptions of fairness change. the model describes a cyclical process whereby individuals’ cognitive processing and judgments about the fairness of an event are guided by their perceptions about the organizational entity involved. it does however not focus on psychological contract as such and does not deal with the consequences of unchanged and changed perceptions. as with the other presented models their model has not been empirically validated. our work is based on the existing definitions of the psychological contract and like morrison and robinson (1997) and guest (2004) we emphasize the crucial role of reciprocity in a psychological contract. as our main contribution the model we present builds on this prior work and extends it with a focus on maintenance, good and bad, of the psychological contract in a cyclic process and we derive and thus validate our model empirically. 90 australasian journal of information systems kautz & bjerknes 2015, vol. 19, pp. 71-95 tales of it consultants until now we have looked at the process of change from engaged employee to volunteer job seeker from the individual employee’s perspective. but the individual consultants are employed and part of a business organization, an it consulting firm. thus in the following, we will look at the consultants' stories in a different context from the organization’s perspective and discuss what it means for the organization that employees leave, and how it could happen that the company was no longer able to maintain the consultants' intrinsic motivation. does it matter for the organization that experienced employees leave in frustration? for those who have left, it is certainly a relief to have made the decision and for them it is positive that they did something with a work situation which they did not like. for the organization this may have various consequences. of course when someone leaves, it means new opportunities for those who stay. but like in the company, it is often the best and most engaged (britt 2003) who leave first. the best people leaving reduces the sense of belonging and loyalty of those who stay as one of their colleagues, someone like themselves, did not think that the organization was a good enough place to work and social support and expertise disappear with those who used to provide it. for the organization the loss of experienced consultants may lead to the loss of contracts and work assignments, and thus loss of revenue, especially in times of great competition for these contracts. in itself it is neither good nor bad that someone ceases their employment. but if many employees leave because their intrinsic motivation is destroyed due to poor management practice, this means that the employees have found individual solutions for a problem, which actually is located at the organizational level (schein 1999). how could this happen at the company and what has led to the situation where the company’s managers who were good at motivating their employees until the late 1990s were no longer able to do so? during this period the company experienced rapid growth, first organic and then by mergers. shortly after the mergers were completed, the it consulting market hit rock bottom and this resulted in several rounds of staff reductions in the company. the consultants’ statements about the development of the company's organizational culture documents how they felt about what happened in the company. bee was disappointed that no one took responsibility for the ‘price’ the organization had to pay for the growth, that no one handled the problems that mergers bring with them in the form of different organizational cultures, different understanding of goals and different practices. curt put forward that the company changed from being a business with great openness and participation, where individual differences were seen as a strength, and which was also managed according to individual needs, to an organization that was completely closed, an organization without co-determination, where the ‘individuals have vanished’, and where governance was addressed by purely economic principles. those who managed and rewarded, saw only numbers, not people. finally, di said that the biggest difference was that people did not care anymore and would not engage and help each other out anymore. the descriptions of the cultural changes correspond to the mechanisms to avoid organizational anxiety as described by moxnes (2000): (1) removing emotions: by keeping distance and not getting involved as described by curt and di; (2) removing the problem: by not taking problems in connection with change serious – supported by bee’s statement about management’s unwillingness to pay the price for growth; (3) removing people: by reducing people to objects, see curt’s statements; (4) institutionalising silence: by either not talking about something or talking about something completely different – supported again by curt’s observations; (5) removing responsibility: by blaming the consultants for the company’s poor results because according to management the consultants did not sell enough; none of the consultants had mentioned this issue explicitly in the interviews, but that management had failed in this respect was an opinion, which was implicitly shared by many employees and observed during the data collection. 91 australasian journal of information systems kautz & bjerknes 2015, vol. 19, pp. 71-95 tales of it consultants according to mao tsetung (1937) the fact that the number of employees is changing rapidly over a short time brings with it anxiety and imbalance, which in turn enforces major changes, although these are not intended. major changes lead to organizational anxiety because basic structures such as goals and rules no longer are perceived in a shared way, roles become unclear and informal social structures break up (moxnes 2000). the organization falls into many different groups, which hold dissimilar basic assumptions and disintegrates as power struggles develop when new structures are established to deal with organizational anxiety (svedberg 2000). when organizational anxiety has grown to an extent that avert mechanisms occur, the organization develops characteristics which are inconsistent with the management processes that promote intrinsic motivation. the major changes in a short period of time apparently resulted in organizational anxiety in the company. this anxiety was met with averting mechanisms that made it difficult to exercise management practices that promote intrinsic motivation with the consequences described above. 7 conclusion the presented model of the maintenance or neglect of the psychological contract as a continuous, cyclic process presents a fresh perspective on a phenomenon which has long been studied in the behavioural and organizational sciences and which is important for is research and practice as well. derived from empirical data and substantiated through literature, the model has only been validated and specified based on a limited number of revelatory case studies in a single case organization. we acknowledge that this approach initially only generates a local empirical theory, but it contributes to the existing body of knowledge by providing a detailed account of empirical findings. the model exhibits in its current form an instance of theorizing through idealization, which needs to be further validated through further cases. although we were not interested in generalization as such, with its focus on the concept of psychological contract the model and its theoretical background are still useful and help explain it consultants’ behaviour of voluntarily terminating their employment contracts beyond the crisis situation, which the it industry experienced some years ago and which motivated this research originally. it should assist managers also in a more general context in avoiding practices which might lead to their employees’ loss of intrinsic motivation and as a consequence to the loss of valuable employees for their organization. references amabile, t. 1998. “how to kill creativity,” harvard business review (76:5), pp. 76-87. argyris, c. 1960. understanding organizational behavior. homewood, il, usa: dorsey press. armstrong, d., lilford, r., ogden j., and wessely s. 2007. “health-related quality of life and the transformation of symptoms,” sociology of health and illness (29:4),pp.570-583. asselin, m. e. 2003. “insider research: issues to consider when doing qualitative research in your own setting,” journal for nurses in staff development (19:2), pp.99-103. bachrach, p., and baratz. m. 1962. “two faces of power,” the american political science review (56:4), pp.947-952. barney, j. 1996. gaining and sustaining competitive advantage, reading: addison-wesley. blau, p. 1964. exchange and power in social life. new york, ny, usa: wiley. britt, t. w. 2003. “black hawk down at work,” harvard business review (81:1), pp. 16-17. bråten, s. 1981. “quality of interaction and participation on model power in industrial democracy and computer networks,” in applied systems and cybernetics, g. e. lasker (ed.), vol. i, pp. 191-200. 92 australasian journal of information systems kautz & bjerknes 2015, vol. 19, pp. 71-95 tales of it consultants carr, a. 2001. “understanding emotion and emotionality in a process of change,” journal of organizational change management (14:5), pp. 421-434. cast, a., and burke, p. 2002. “a theory of self-esteem,” social forces (80:3), pp.1041-1068. cialdini, r. 2001a. influence, science and practice, 4th ed., boston: allyn and bacon. cialdini, r. 2001b. “harnessing the science of persuasion,” harvard business review (79:9), pp. 72-79. creswell, j. w. 2003. research design qualitative, quantitative and mixed methods approaches, thousand oak, ca: sage publications. de lange, a. h., bal, p. m., van der heijden, b. i. j. m., de jong, n., schaufeli, w. b. 2011. “when i'm 64: psychological contract breach, work motivation and the moderating roles of future time perspective and regulatory focus”, work & stress: an international journal of work, health & organisations (25:4), pp. 338-354. forsyth, d. 1999. group dynamics, 3rd edition, belmont: wadsworth publishing company. frey, b. s. 2002. inspiring economics: human motivation in political economy. cheltenham, uk : edward elgar publishing. gabriel, y. 2000. storytelling in organizations, oxford, uk: oxford university press. gerring, j. 2007. case study research: principles and practices, cambridge: cambridge university press. ghazawneh, a., and henfridsson, o. 2013. “balancing platform control and external contribution in third-party development: the boundary resources model,” information systems journal (23:2), pp.173-192. glaser, b. g. 1992. basis of grounded theory analysis: emergence vs forcing, sociology press, mill valley, ca, usa: sociology press. glaser, b.g., and strauss, a.l.1967. the discovery of grounded theory strategies for qualitative research. new york, usa: aldine de gruyter. guest, d. e. 2004. “the psychology of the employment relationship: an analysis based on the psychological contract,” applied psychology: an international review (53:4), pp. 541–555. hansen, b., and kautz, k. 2005. “grounded theory applied – studying information systems development methodologies in practice,” in proceedings of the 38th hawaii international conference on system sciences, big island, usa, january 3-6, 2005. hughes, j., and jones, s. 2003. “reflections on the use of grounded theory in interpretive information systems research,” in proceedings of the ecis 2003 conference, naples, italy. jones, d. a., and skarlicki, d. p. 2013. “how perceptions of fairness can change: a dynamic model of organizational justice,” organizational psychology review (3:2), pp. 138–160. kanuha, v. k. 2000. “being” native versus “going native”: conducting social work research as an insider,” social work (45:5), pp. 439-447. kendall j. e., and kendall k.e. 2012. “storytelling as a qualitative method for is research: heralding the heroic and echoing the mythic,” australasian journal of information systems (17:2), pp. 161-187. kim, w. c., and mauborgne, r. 2003. “fair process: managing in the knowledge economy”, harvard business review (81:1), pp.127-136. kjærgaard, a., and, kautz, k. 2008. “a process model of establishing knowledge management: insights from a longitudinal field study,” omega – the international journal of management science (36:2), pp. 282-297. 93 australasian journal of information systems kautz & bjerknes 2015, vol. 19, pp. 71-95 tales of it consultants langley, a. 1999. “strategies for theorizing from process data,” academy of management review (24), pp. 691-710. locke, k. 2001. grounded theory in management research, sage publications, london, uk: sage publications. lopreato, j., and alston, l. 1970. “ideal types and the idealization strategy,” american sociological review (35), pp. 88-96. markus, m.l., and robey, d. 1988. “information technology and organizational change: causal structure in theory and research,” management science (34), pp. 583-598. morrison, e.w., and robinson, s.l. 1997. “when employees feel betrayed: a model of how contract violation develops,” academy of management review (22), pp. 226–256. morse, g. 2003. “why we misread motives,” harvard business review (81:1), p. 18. moxnes, p. 2000. positive anxiety of the individual, a group and organization (in norwegian), 3rd ed., oslo, norway: paul moxnes publishing. reichheld, f. 2001. “lead for loyalty,” harvard business review (79:8), pp. 76-84. rousseau, d. m. 1989. “psychological and implied contracts in organizations,” employee responsibilities and rights journal (2), pp. 121-139. rousseau, d. m. 1995. psychological contracts in organizations. thousand oaks, ca, usa: sage. sabherwal, r.1999. “the role of trust in outsourced is development projects,” communications of the acm (42:2), pp. 80-86. schein, e. 1970. organizational psychology, 2nd edition, englewood cliffs, nj, usa: prentice-hall. schein, e. 1999. process consultation revisited. building the helping relationship, reading, ma, usa: addison-wesley. simons, t. 2002. “the high cost of lost trust,” harvard business review (80:9), pp.2-3. strauss, a., and corbin, j. 1998. basics of qualitative research. london, uk: sage publication. svedberg, l. 2000. group psychology about groups, organizations and leadership (in swedish), 2nd edition, lund, sweden: studentlitteratur. tsetung, mao 1937. “on contradiction,” in tsetung, mao (1978): selected works (in norwegian), vol. 1, pp. 311-348, oslo, norway: october publishers. walsham, g., “interpretive case studies in is research: nature and method”, european journal of information systems (4:2), pp. 74-81, 1995. weber, m. 1949. the methodology of the social sciences. glencoe, il, usa: free press. yin, r.k. 2009. case study research: design and methods, 4th ed., thousand oaks, ca, usa: sage publications. copyright: © 2015 kautz & bjerknes. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. 94 australasian journal of information systems kautz & bjerknes 2015, vol. 19, pp. 71-95 tales of it consultants 95 paper title (paper title style) australasian journal of information systems islam, d’alessandro, furner, johnson, gray & carter 2016, vol 20, research article brand switching pattern discovery by data mining techniques 1 brand switching pattern discovery by data mining techniques for the telecommunication industry in australia md zahidul islam charles sturt university school of computing and mathematics zislam@csu.edu.au steven d’alessandro school of management and marketing charles sturt university michael furner school of computing and mathematics charles sturt university lester johnson department of management and marketing swinburne university of technology david gray department of marketing and management macquarie university leanne carter department of marketing and management macquarie university abstract there is more than one mobile-phone subscription per member of the australian population. the number of complaints against the mobile-phone-service providers is also high. therefore, the mobile service providers are facing a huge challenge in retaining their customers. there are a number of existing models to analyse customer behaviour and switching patterns. a number of switching models may also exist within a large market. these models are often not useful due to the heterogeneous nature of the market. therefore, in this study we use data mining techniques to let the data talk to help us discover switching patterns without requiring us to use any models and domain knowledge. we use a variety of decision tree and decision forest techniques on a real mobile-phone-usage dataset in order to demonstrate the effectiveness of data mining techniques in knowledge discovery. we report many interesting patterns, and discuss them from a brand-switching and marketing perspective, through which they are found to be very sensible and interesting. keywords: decision tree, decision forest, ensemble of decision trees, data mining, brand switching, switching behaviour 1 introduction 1.1 mobile phone brand switching australians love and hate their mobile phones. according to euromonitor, in 2011 there were some 25.54 million mobile phone subscriptions, more than one for each member of the australian population (mobile phone statistics n.d.). the telecommunications industry ombudsman (tio) received a record 197,682 complaints in 2011, which was 18% higher than the number of complaints received in the previous year. (annual report of the telecommunications industry ombudsman n.d.). the challenge for mobile service providers australasian journal of information systems islam, d’alessandro, furner, johnson, gray & carter 2016, vol 20, research article brand switching pattern discovery by data mining techniques 2 then is to retain their customers by gaining their trust and satisfaction. in order to do so they need to assess the dynamics of brand switching behaviour. a number of models have been developed to analyse the customer switching process (bansal & taylor 1999, colgate & lang 2001, keaveney 1995, lees, garland & wright 2007). the most general, the keaveney model (1995, p79) identified eight major factors behind customer exit (or push factors) including: pricing, inconvenience, core service failure, service encounter failure, response to service failure, competition, ethical problems and involuntary switching. 1.2 the problem with existing models bansal & taylor (1999) investigated the impact of other factors (on switching behaviour) including switching costs, and service quality. actual switching behaviour was also examined. the problem with many existing consumer behaviour models, is that they apply a standard model, to what is a heterogeneous market. there is also the possibility that a number of switching models may exist within a large market. to counter this issue researchers sometimes use latent class modelling (vinzi, trinchera, squillacciotti & tenenhaus 2008). however, even in this approach there is always an issue of model misspecification for different clusters. 1.3 a data mining approach an alternative to existing models is to let the data speak for itself. in other words, we can apply sophisticated data mining techniques to discover brand switching patterns, which can then be explained in a theory post-hoc manner. such an approach may be more attractive to practitioners, who may be more interested in prediction and then explanation, than testing existing theories of explanation by evidence. in other words, data-mining techniques can discover previously hidden, non-obvious, interesting and valid patterns (i.e., logic rules) from a dataset without requiring any pre-assumed and domain specific knowledge. hung, yen & wang (2006) showed the usefulness of data-mining models in brand switching prediction, without exploring the knowledge these models discovered. instead of discovering knowledge and understanding the patterns, they focussed on achieving a high quality prediction accuracy with the dataset they used. wei & chiu (2002) used data mining techniques to discover some knowledge, but they used only contractual data available with the service provider and did not carry out any data collection through survey. that is because knowledge discovery was not their primary objective. instead, they focussed on building prediction models based on data mining techniques. additionally, they suggested that future models make use of geographical and complaint information from customers. we use this information in the dataset used in our study. ahn, han & lee (2006) also discovered some knowledge by creating models for brand switching based on statistical methods for the south korean mobile industry. they found a variety of interesting statistics relating to demographics and phone usage. these however, did not use a data-driven approach, instead relying on hypotheses developed by the researchers. this paper presents how a data mining approach can be successfully applied to a marketing context to reveal patterns of brand switching. we discover valid, interesting and previously unknown brand switching patterns and other customer behaviour patterns using a variety of decision tree and decision forest algorithms, showing the power of a data-driven approach. 1.4 dataset, context and pre-processing the dataset used in this study has been adapted from the results of a survey conducted for amaysim in conjunction with macquarie university in the report “state of the mobile nation" (gray et al. 2012). the survey asked mobile phone users, who had either changed their mobile phone service provider in the last 12 months or had considered doing so, a variety of questions regarding their phone usage, contentment with a variety of factors relating to their provider, and their demographics. twelve months after the initial survey, the participants were again asked a variety of questions through a follow-up survey, and it was determined if they did indeed switch. australasian journal of information systems islam, d’alessandro, furner, johnson, gray & carter 2016, vol 20, research article brand switching pattern discovery by data mining techniques 3 the initial survey contained 9 questions regarding the respondent’s demographics, 21 questions regarding the nature of their phone plan, around 50 questions regarding the relationship they have with their current and previous providers, and some other miscellaneous questions. demographic questions included “what is your age?”, “what is the highest level of education you have completed?” and questions regarding relationship and familial status all of which are categorical. questions regarding the phone plan include “who is your current mobile phone provider?” and “what kind of mobile service do you have?”; they are a mix of numerical and categorical answers. relationship questions include “in general, to what extent do you trust mobile phone carriers to respect the rules and regulations protecting consumers, with 0 being little/no trust and 10 being very trustworthy?” and are largely numerical ratings provided by the respondent. the follow-up survey conducted 12 months after the original survey was considerably smaller, and included some demographics questions, and questions related to the respondent’s experience over the last year. the period of time between the two surveys allows us to analyse confirmed switchers as well as confirmed non-switchers (those who had thought about switching in the last survey, but still didn’t switch). the dataset with follow-up survey attributes appended was considered separate to the original dataset, but had the same preprocessing applied. the first step of pre-processing the dataset involves merging some sets of questions into single attributes. for example, a set of multiple-choice questions are often merged into a single attribute having domain values covering all multiple choice answers. after this, records with missing values were removed from the dataset. care was taken so that only records with missing values that affected the analysis were removed, as some missing values were naturally accounted for by the answers to previous questions. we did not use missing value imputation techniques (rahman & islam 2013, rahman & islam 2014) since the fundamental assumption of these techniques is that the missing values appear at random and do not depend on any other factor/s, which is not the case for our data set. in order to explore a wide variety of patterns we prepare a number of sub-datasets mostly by selecting different subsets of attributes and then applying six (6) data mining algorithms on the sub-datasets. the process undertaken for this and descriptions of the datasets created are discussed in section 3. 1.5 major contributions of the study the major contributions of this study are as follows: • the use of a real life dataset on mobile phone usage and brand-switching focussing in particular on the australian market – as far as we know this is the only study of its kind on an australian market dataset. • application of six (6) well known data mining techniques on the real life dataset in order to explore the effectiveness of the techniques in discovering valid patterns (section 3) different data mining techniques use various methods for extracting rules. in order to discover useful knowledge, it is important to know which algorithms are best for extracting interesting rules. • comparison of two groups of data mining techniques, decision forests and decision trees, in discovering valid patterns from the dataset. (section 4) this provides insights as to how and when to use these, and demonstrates the different amounts of knowledge in the form of rules accessible to these groups. • identification and presentation of a number of interesting patterns/trends (section 5). • evaluation of the discovered trends from a marketing perspective (section 6). • discussion of the use of data mining techniques in collaboration with traditional statistical techniques in marketing (section 6). australasian journal of information systems islam, d’alessandro, furner, johnson, gray & carter 2016, vol 20, research article brand switching pattern discovery by data mining techniques 4 section 2 briefly introduces some data-mining techniques relevant to this study. section 3 presents our data analysis approach. section 4 discusses the effectiveness of various datamining techniques in discovering valid rules (i.e., patterns/trends), and section 5 presents some interesting trends discovered by the data-mining techniques. finally, concluding remarks and discussion of the study’s relevance to the marketing industry are presented in section 6. 2 data mining techniques a brief review data-mining is used over a wide variety of disciplines (fayyad, piatetsky-shapiro, smyth & uthurusamy 1996, schultz, eskin, zadok & stolfo 2001, berry & linoff 2004). the term encompasses everything from the collection of data, to preprocessing and analysis through the supervised and unsupervised learning. supervised learning uses labelled data in order to make predictions, and classification. decision tree and decision forest algorithms are examples of classification algorithms. 2.1 what is a classifier? a dataset d contains a set of records r = {r1, r2, …, r|d|}, where |d| is the total number of records. each record of d has |a| attributes a = {a1, a2, …, am}. aj ∈ a is an attribute that can be either numerical or categorical. the domain of a numerical attribute aj = [l,u] varies between a lower limit l and an upper limit u. the domain size of the attribute is |aj| = [u – l + 1] for integer values. similarly, the domain of a categorical attribute can be presented as aj = {aj1, aj2, …, ajd}, where the domain size is |aj| = d. ri is the i-th record and ri,j is the j-th attribute value of the i-th record. ri,j takes a value from the domain of aj; that is ri,j ∈ aj. one of the categorical attributes aj (e.g. the attribute brand-switching with two possible values switched and didn’t switch) can be considered as the class attribute, the values of which are used for labelling the records. a record ri; ∀i has a class value switched or didn’t switch. the classification task aims to discover the logic rules that determine whether a record should have the class value switched or didn’t switch. an example of a logic rule can be “if gender = male and age ≤ 40 then brand-switching = switched" meaning that if any record ri is a male aged 40 years or younger then he is likely to switch his service provider. a classifier discovers and uses the rules to predict new, unlabelled data in the future. a black-box technique such as artificial neural networks does not provide any logic rules that are human understandable, whereas decision trees (quinlan 1993) avoid this issue of a black box algorithm. 2.2 decision trees, forests and logic rules a decision tree (quinlan 1986) discovers a set of logic rules, where each record ri of a training dataset (i.e., the dataset which the decision tree is built from) obeys/falls in one and only one logic rule. an example of a decision tree is shown in figure 1. this decision tree is assumed to be trained on an imaginary mobile-phone-user dataset, which has a class attribute with two possible classes: “switched" and “didn’t switch". australasian journal of information systems islam, d’alessandro, furner, johnson, gray & carter 2016, vol 20, research article brand switching pattern discovery by data mining techniques 5 figure 1: example of a decision tree. there are three leaves and each of them represents a logic rule. for example, leaf 1 indicates a logic rule: if gender = female then brand-switching = didn’t switch. the validity of a rule is measured by two terms: support and confidence (han, kamber & pei 2006). support is defined as the ratio of the number of records falling into the leaf to the total number of records in the dataset. confidence of a rule is defined as the ratio of the dominant class in a leaf to the other classes. the higher the support and confidence of a logic rule, the stronger (i.e., more valid) the rule is. a decision tree algorithm (quinlan 1993) chooses the test attributes and corresponding splitting points (such as age ≤ 40) automatically without requiring any input/assumption from a user/data miner. a decision forest (breiman 2001) being an ensemble of decision trees can typically discover higher number of valid and interesting patterns than a tree. it is important to discover those additional logic rules since based on them a service provider can get a better understanding on the reasons/trends of various important issues. in this study we make use of the random forest (breiman 2001), bagging (breiman 1996) and sysfor (islam & giggins 2011) forest algorithms as implemented by us. we also make use of the individual decision tree algorithms j48 (quinlan 1993), simplecart (breiman, friedman, olshen & stone 1984) and reptree (witten & frank 2005), all of which have implementations available in weka (hall, frank, holmes, pfahringer, reutemann & witten 2009). we classify logic rules/patterns into two main categories: valid and interesting. by valid rules we mean the rules that have high support and confidence. how high is high? we argue that this depends on the consideration of a domain expert or user. by interesting rules (freitas 1999) we mean the rules that reveal interesting, previously unknown and non-obvious information. a number of interesting rules have been presented in section 5. 3 our approach in analysing the dataset often an intelligent use of data mining techniques on a dataset can discover more (valid and interesting) rules/patterns than a straight forward application of data mining techniques. therefore, we apply several data mining techniques on various sub-datasets prepared from the pre-processed dataset discussed in section 1.4. sub-datasets are prepared by selecting different class attributes and subsets of attributes. we identify a set of attributes, each of which qualifies to be a class attribute in the sense that they have the ability to label the records. a class attribute is chosen to explore the causes of specific phenomena, such as switching. by selecting different class attributes for different subaustralasian journal of information systems islam, d’alessandro, furner, johnson, gray & carter 2016, vol 20, research article brand switching pattern discovery by data mining techniques 6 datasets and applying a classifier on a sub-dataset, we force the classifier to choose logic rules in order to identify trends relating to the chosen class attribute. in this study we select the following class attributes: switching status, mobile plan type and underspending. let us call the original pre-processed dataset (without follow-up questions) d, and the dataset with follow-up questions d'. the sub-datasets having switching status, mobile plan type and underspending as the class attributes are denoted by ds, dp and du, respectively. we next use a variety of attribute subsets ax ⊂ a, where ax;∀x relates to a common question set, for example, demographics. the attribute subsets that we use in this study are based on demographics, relationship with their telephone provider, and type of phone usage. additionally, we also select subsets of attributes (let us call them class specific subsets) that are related and specific to class attributes. sub-datasets made of subsets of attributes relating demographics, relationship with their telephone provider, type of phone usage and class specific subsets are denoted by dd, dr, dt and da, respectively. finally, we prepare sub-datasets using a subset of attributes and a class attribute for data analysis and knowledge discovery. for example, we prepare a sub-dataset with the demographic subset of attributes and switching status class attribute and denote the dataset by dds. for each class attribute we choose a number of subsets of attributes and thus prepare a number of sub-datasets. for the switching status class attribute we create 4 sub-datasets: dds, drs, dts and das. similarly, for the mobile plan type class attribute we create 2 sub-datasets: ddp and dap. for the underspending class attribute we create 1 sub-dataset ddu. the design and selection of sub-datasets are our subjective decision. for each sub-dataset we apply six (6) classification algorithms (i.e., data mining techniques) namely sysfor, random forest, bagging, j48, simplecart, and reptree. the code for sysfor is freely available online at http://csusap.csu.edu.au/~zislam/. based on the use of data mining algorithms on the sub-datasets and the whole dataset, we discover a huge set of logic rules from which we then select valid and/or interesting rules. our definitions of valid and interesting rules are provided in section 5. additionally, we take advantage of the follow up survey (see section 1.4) for a more useful data analysis. let us denote the dataset containing follow up survey questions (in addition to the initial survey questions) by d'. we also prepare a dataset d'fu, which only includes attributes from the follow up survey. we then prepare sub-datasets d'as, d'ds, and d'rs, as well as d'fus. all six data mining techniques are applied on these sub-datasets in order to explore valid and/or interesting rules. we also take advantage of d' in identifying the customers who have definitely switched within the last 12 months (i.e., confirmed switchers) by carefully checking their mobile service providers in the initial survey and follow up survey. the class values of d'as, d'ds, d'rs and d'fus are then determined accordingly. 4 data mining performance in the process of conducting this study, a wide variety of logic rules regarding brand-switching and a variety of other key issues have been generated. from these rules, those which provide meaningful, non-trivial information are extracted. rules that have high support and confidence are used to extract the meaningful information from the dataset. in this study, the criteria of valid rules are user-defined, following the suggestions of previous literature (freitas 1999). rules with both high support and high confidence are considered to be valid. we use two different confidence thresholds: 70% and 90%, and two different support thresholds: 5% and 10%. table 1 shows the model performance over the generated sub-datasets from section 3, grouped by ds (switching), dp (plan type), du (underspend) and d's (confirmed switchers). each section of the table has the data-mining techniques in one of two groups, forest or tree. the average classification accuracy column shows the average classification accuracy over all subaustralasian journal of information systems islam, d’alessandro, furner, johnson, gray & carter 2016, vol 20, research article brand switching pattern discovery by data mining techniques 7 datasets for a particular class attribute. for example, the average classification accuracy column for sysfor in the switching section represents the average classification accuracy on dds, drs, dts and das. similarly, the no. valid rules found column represents the sum of valid rules (unique) found over these sub-datasets. two identical valid rules are counted as one unique valid rule. from table 1, we can infer some interesting points. first, data mining techniques can discover a big number of valid and unique rules/patterns from the dataset. these rules can be used to classify the records against the class attributes, but are also the key in determining trends in the dataset. second, forests can discover considerably more (valid and unique) rules/patterns than trees, according to our settings of valid rules. third, classification accuracies of trees are similar to forests. so classification accuracies are not necessarily related to number of valid rules. fourth, it appears (somewhat misleadingly) that randomforest (and to a lesser extent bagging) generate larger number of valid rules than sysfor. this does not necessarily indicate the superiority of these techniques for knowledge discovery over sysfor for the following reason. randomforest and bagging use bootstrap samples of the dataset on which they build trees. since the bootstrap samples are often very similar to each other, they frequently discover very similar rules which have only slight differences in the splitting points of the attributes (despite the same attributes being tested). while technically “unique" rules, these are not necessarily useful more than once. this is in contrast to sysfor, for which the rules are diverse as the technique does not use bootstrap sampling. rules presented in table 1 for each algorithm (such as sysfor and bagging) are unique. that is, for underspend sysfor discovers 62 unique and valid rules, but some of these rules can also be discovered by another algorithm. therefore, table 2 presents the total number of valid and unique rules found by all six algorithms for each class attribute. we can see that data mining techniques can discover a huge number of valid and unique rules from the dataset. the table also gives an idea on how quickly the number of valid and unique rules decreases with the increase of support and confidence. table 2 provides an overview of the success of finding rules that fit our criteria of interestingness using all of the data-mining algorithms that are part of our methodology. 5 observations and findings using data mining in section 4 we reported on the number of valid and unique rules selected from a huge set of rules that is obtained by the data mining techniques. in this section we present some interesting rules that are again selected from the huge set of rules. we group together similar rules into trends, which explain interesting components/observations of the data analysis. the findings reported in the following subsections are discovered automatically through data mining and not based on any hypothesis/assumption. the survey was not originally designed by us. it only collected data on the customers who either switched their service providers within the last 12 months or considered switching seriously within the same time period. hence, we only have information on the customers who considered switching and then switched, and the customers who considered switching but did not switch. we do not have any data on customers who did not consider switching their providers within last 12 months. the names of all telecommunication companies have been replaced with pseudonyms to maintain confidentiality. in this study we use the data collected through the survey and extract knowledge from whatever data have already been collected. one of our goals is to demonstrate that data mining techniques can still discover valid and/or interesting rules from the dataset. 5.1 demographics based analysis for brand switching the goal of this section of the study is to test a variety of class attributes against only those attributes which represent demographic information, in order to explore how these class attributes (such as switching status) are dependent on the demographic information such as age, gender and place of residence. that is, we use the dds dataset (see section 3). australasian journal of information systems islam, d’alessandro, furner, johnson, gray & carter 2016, vol 20, research article brand switching pattern discovery by data mining techniques 8 5.1.1 trend: age, gender, and brand-switching out of 1392 records representing those surveyed who had either switched or considered switching, there were 149 participants (10.70% of the total participants) who were male and within the age bracket of 35 to 44. interestingly, 73.83% of these respondents (i.e., 110 respondents) had switched in the last 12 months. that is, the male customers aged between 35 to 44 years have 73.83% probability of switching their provider once they consider switching. a similar trend of switching is observed among the younger male participants. male participants within the age bracket 18-24, 25-34 and 35-44 have 75.51%, 81.31% and 73.83% chance of switching respectively, once they consider it. male participants aged between 18 and 44 have an overall 76.72% chance of switching once they consider it. however, the trend of switching is significantly lower among the older male customers. male customers within the age bracket 45-54, 55-64 and 65-above have only 58.27%, 52.87% and 52% chance of switching respectively, even when they consider doing it. the overall chance of switching among the male customers aged between 18 and 44 is 76.72%, while the switching chance for male customers aged between 45 and above is only 54.23% (once they consider switching). in the dataset, there were 426 older (aged between 45 and above) male participants and 305 younger (aged between 18 and 44) participants. from these results it is evident that males between the age of 18 and 44 are at considerable risk of switching. the providers may take different strategies for them as they are very vulnerable to switching. older male customers are more stable than younger (less than 45 years old) male customers. younger male customers being more switch-prone (vulnerable) may need more attention in order to increase retention. the switching tendency among the women is significantly lower at only 37.97%, regardless of their age, compared to the young male customers (aged between 18 and 44) who have 76.72% chance of switching. even within specific age brackets the switching tendency of women is lower than that of men. for example, once they consider switching the possibility of actually switching among men and women within the age bracket 25-34 are 81.31% and 40%, respectively. similarly, within the age bracket 35-44 the actual switching possibility among men and women are 73.83% and 35.21%, respectively. women that are married and with children have an even lower switching rate at only 25%. it would seem that regardless of their age women are a lot more stable than young men in terms of the switching tendency. perhaps the providers can have specific strategies for different groups to increase retention. the observations may help in building such strategies. 5.1.2 trend: professional communities and brand-switching the gender based difference of the switching pattern is also clearly evident within some professional communities. once they have considered switching, the male respondents working in finance and insurance have 88.24% chance of actually switching, whereas female respondents working in the same sector have only 26.67% chance of actually switching. there are 17 male and 15 female respondents working in finance and insurance in the dataset. a similar gender based difference in switching possibility exists among the students. while the possibility of actually switching within the male students is 62.5%, the actually switching possibility within the female students is 45%. once they consider switching, the overall actual switching possibility among the students is only 52.78%. there are 32 male students and 40 female students. interestingly, the gender based difference is not observed among the information media and telecommunication professionals. within this community the actual switching possibilities for male and female (once they consider of switching) are 76% and 75%, respectively. while the switching tendency among the women is low overall, it is not the case for the women in information media and telecommunication. it should be noted though that there were only australasian journal of information systems islam, d’alessandro, furner, johnson, gray & carter 2016, vol 20, research article brand switching pattern discovery by data mining techniques 9 four women from the information media and telecommunication profession surveyed, but the rules enjoy high confidence. 5.1.3 trend: marital and familial status, and brand-switching interestingly, it is observed that if male respondents aged between 55 and 64 are divorced then their actual switching possibility is 85%. in our dataset, there are 21 respondents within this category. interestingly, 18 of them considered switching and actually switched in the 12 months prior to being surveyed. on the other hand, men aged between 55 and 64 years who are married with their youngest child older than 5 years have an actual switching possibility of only 40%, even when they consider switching. there were 65 respondents within this group and 26 of them had switched in the last 12 months. this is even lower than the average for older men, which was only 54.23%. although generally older male customers are less likely to switch, their risk of switching increases dramatically if they are divorced (from 54.23% to 85%), and decreases dramatically if they are married with the youngest child aged 5 years or more (from 54.23% to 40%). 5.1.4 trend: income ranges and brand-switching surprisingly, the analysis undertaken shows no patterns within different income ranges. while it has been previously observed that the patterns for switching or non-switching have been mainly driven by gender, this is not the case for income ranges something which one would intuit differently. an interesting trend discovered however, is that women with an annual income of $40,000 and $49,999 have a lower switching possibility than most women at 21.93% compared to an overall average switching tendency of 37.97%. despite this being the case for almost all income ranges, a high non-switching tendency (87.5% possibility) is observed within the respondents having annual income between $100,000 and $109,000, and age between 55 and 64. there are 8 such participants, 7 of which did not switch in the last 12 months although they considered switching. when one compares this to the rest of men within this age bracket, who have a 47.13% non-switching tendency, this is a significant difference. 5.1.5 trend: education and brand-switching once they think of switching, the overall rate of switching for the university drop-outs is 31.51%. those respondents with bachelor’s degree as their highest education level who do not have children have an even higher actual switching rate at 37.25%. from this, that people who have been to university for a reasonable amount of time tend to switch a lot can be inferred. this finding suggests that advertising at universities and targeting university students to foster a sense of brand-loyalty may be a possible approach for the service providers. 5.2 demographics based analysis for properties other than brand switching in this subsection we use the same demographic based attributes used in the previous study, but explore patterns other than just brand switching tendency. therefore, we use some other attributes from the dataset as our class attribute in this section. this allows us to discover if there are any strong rules that predict other factors such as underspending and phone service type. these strong rules can be adapted into knowledge about these factors, just as has been done previously with brand-switching. 5.2.1 trend: relation between underspending and gender a demographic analysis of the 1077 respondents who are not on a pre-paid plan shows that males are highly likely to underspend the “included value" on their plan. males in this category have an underspending likelihood of 75.54%. this is opposed to females who have only 67% possibility of underspending. from this, it appears that men are slightly more likely to underspend the included value on their plans than women. australasian journal of information systems islam, d’alessandro, furner, johnson, gray & carter 2016, vol 20, research article brand switching pattern discovery by data mining techniques 10 5.2.2 trend: mobile service type and familial status a demographic analysis of 1540 survey-respondents pertaining to the type of mobile phone service shows that of all those customers with a child under 5 years old, a substantial amount of these (64.26%) are on a cap plan with a phone. the next largest service type for these customers is prepaid with 20.53% of respondents, a substantial difference. as a significant quantity (64.26%) of these customers are on a cap plan with a phone, it would make sense to directly market to people with young families for these services. 5.3 relationship-based analysis for brand-switching this section aims to explore interesting patterns of brand switching and the relationship a customer has with their phone company (and phone companies in general). the independent variables that are chosen in this section include those that represent various satisfaction metrics, number of times a customer has complained or had problems, and channels of complaint. the dependent variable (i.e., the class attribute) is brand-switching. 5.3.1 trend: perceived industry ethics and brand-switching interestingly those who believe that telecommunication service providers act ethically have a high switching tendency. once they consider switching they have a switching probability of 65.73%. that is, 94 of these respondents switched after having considered switching while 49 of them did not ultimately switch. this finding makes it seem likely that despite customers being strongly convinced that their service providers are ethical, they may still be prone to switch. that is, just functioning ethically may not be good enough to retain customers. conversely it seems those who generally think that phone providers are ethical and trust that carriers follow the industry rules and regulations, and who haven’t used their phone less frequently than usual in the past 12 months are very unlikely to switch (92.86% did not switch). 5.3.2 trend: brand-switching and decreased plan usage a high quantity (48.48%) of confirmed switchers don’t have a problem with their service, but claim that they use their service less. there are 31 customers who most strongly agreed that they use the service of the current provider less than the previous provider and 24 (77.42%) of them actually switched in the past 12 months. 5.3.3 trend: employment and service type a very high quantity (70.11%) of customers surveyed who work in retail have a cap plan with phone. also, a majority (83.39%) of respondents who claim to not be able to afford buying a phone outright are on cap plans with a phone. very surprisingly, even if they do not consider phone companies to be extremely ethical, and cannot afford to buy a phone outright, but are happy with their monthly cost, they will most likely be on a cap plan with a phone (84% likely). 5.4 other relationship-based analysis this study uses the relationship-based attributes to explore customer satisfaction and plan cost in relation to where plans are purchased. 5.4.1 trend: customer satisfaction, purchase channel, and plan-cost of those respondents that purchased their service online and pay less than $6 a week for the service (75 customers), they are generally happy with the service and the length of the contract. this seems mostly due to the fact they are prepaid customers (52 of 75, 69.33%). businesses probably need not focus resources on keeping these customers happy as they already largely satisfied, due to the nature of the service they have purchased. 5.5 overall preliminary survey analysis for brand-switching instead of analysing brand switching patterns based on groups of attributes such as demography and relationship related attributes, in this section we use a wide variety of independent variables. australasian journal of information systems islam, d’alessandro, furner, johnson, gray & carter 2016, vol 20, research article brand switching pattern discovery by data mining techniques 11 5.5.1 trend: lifelong switching tendency one of the strongest patterns is that although they have considered switching in the past 12 months, if a customer has never switched in the past then the possibility of his/her switching is only 1.15% which is extremely low. there are 174 such respondents out of which only 2 switched in the last 12 months. this is reinforced by the fact that of the people surveyed in this category who were customers of one of the major established brands, none switched. another strong pattern is if customers have switched more than 12 times in the past then the probability of their switching is 100%. there are 9 such customers all of them have also switched in the last 12 months. however, in comparison to this, those who have switched between 1 and 12 times in the past have a switching probability of 58.31% (705 switched and 504 did not switch). some customers have a strong predisposition to switching. if someone has switched more than 12 times in the past then he/she will switch again. similarly those who have never switched remain unlikely to do so. previously it has been noted that male customers have a higher switching tendency. however, it is also possible to observe that even male customers who have never switched before have a low switching probability of only 1.75% (1 switched and 56 did not switch in the last 12 months). a similar trend is found among the female customers as well. those who never switched before have a low switching tendency of 0.85% (1 switched and 116 did not switch). the male customers who have switched before have a switching probability 68.84% (464 switched and 201 did not switch). in comparison, those female customers who switched before have a switching tendency of 45.96% (250 switched and 294 did not switch). 5.5.2 trend: company size and switching probability our study shows that the switching probabilities for the major established telecommunication service providers are considerably lower than the smaller, less established providers. for the larger companies, listed here as bigtelco# there is an overall switching probability of only 27.29% if a respondent has considered switching in the last 12 months. this is especially low compared to the combined switching probability for the smaller providers, which was 85.2%. while in our survey dataset there are less customers for less established brands, the rate of customer attrition as a percentage of survey respondents is considerably higher in each case. for one group of the very smallest companies, this was close to 100% (these have been grouped together as nichecompetitors). other telecommunication companies have been listed as othercompetitor#, indicating that they are mid-sized. the individual results for each provider are as follows: • for bigtelco1, the switching probability is 0% (0 switched and 4 did not switch). • for bigtelco2, the switching probability is 23.14% (28 switched and 93 did not switch). • for bigtelco3, the switching probability is 39.47% (150 switched and 230 did not switch). • for bigtelco4, the switching probability is 44.13% (109 switched and 138 did not switch). • for bigtelco5, the switching probability is 54.74% (150 switched and 124 did not switch). • for bigtelco6, the switching probability is 56.98% (49 switched and 37 did not switch). • for othercompetitor1, the switching probability is 58.62% (17 switched and 12 did not switch). • for othercompetitor2, the switching probability is 71.43% (20 switched and 8 did not switch). australasian journal of information systems islam, d’alessandro, furner, johnson, gray & carter 2016, vol 20, research article brand switching pattern discovery by data mining techniques 12 • for othercompetitor3, the switching probability is 75% (6 switched and 2 did not switch). • for othercompetitor4, the switching probability is 80% (32 switched and 8 did not switch). • for the nichecompetitors group, the switching probability is 97.41% (113 switched and 3 did not switch). one of the mid-sized competitors has discontinued their brand, and its services were taken over by one of the major established providers since the survey was conducted. 5.6 overall preliminary survey analysis for other factors in this section we use the same set of attributes as in the previous section, but we change the class attributes to find patterns on other factors such as the method of plan purchase and contract-length. 5.6.1 trend: method of purchase customers who purchase services from supermarkets are overwhelmingly on “prepaid services” (93.24%). of these prepaid customers, most have not upgraded or downgraded their service in the last 12 months (89.86%). customers who report that they would not buy a handset outright as they could not afford it are overwhelmingly on “cap plans with phones” (81.25%). these customers account for 36.62% of all of those on “cap plans with phones”. there are less purchases (23) of “broadband bundles” “over the phone” than there are “cap plans with phones” (116). however, “over the phone” sales account for 26.7% of all bundle sales. another 36.05% of broadband bundles are sold in telecommunication carrier stores. “over the phone” sales seem to be one of the most useful methods to obtain broadband bundle customers. inside telecommunication carrier stores is also a great place to obtain these customers. the majority (63.04%) of non-prepaid services purchased online who pay more than $6 a week are for “cap plans with phones”. most “over the phone” sales of services are for “cap plans with phones”. of 163 customers that purchased their plans this way, 116 purchased “cap plans with phones” (71.17%). this accounts for 16.34% of all “cap plan with phone” purchases in the dataset used which contains 1540 records. 5.6.2 trend: contract length and customer happiness 45% of customers that pay between $3 and $10 a week for their cap plan with phone are unhappy with the length of their contract. 42.5% of all customers are unhappy with the length of their contract. customer retention may be increased by allowing more flexible contract lengths in order to increase customer satisfaction. almost half of these low cost cap plans are unhappy due to their contract length. 25.71% of prepaid customers that pay between $3 and $10 a week are unhappy or ambivalent with the quality of the phone, although are willing to pay for a phone outright to avoid entering into a contract. finally, the overwhelming majority (70%) of people happy with their contract length who make more than 10 calls a week are on cap plans with phones. of this group most (95.4%) are happy or ambivalent to their sms rates. australasian journal of information systems islam, d’alessandro, furner, johnson, gray & carter 2016, vol 20, research article brand switching pattern discovery by data mining techniques 13 class attribute algorithm type technique avg. classification accuracy no. valid and unique rules found switching forest sysfor 60.19 12 randomforest 61.34 6 bagging 61.14 4 tree j48 61.17 0 reptree 72.97 3 simplecart 70.69 0 plan type forest sysfor 71.54 53 randomforest 68.89 70 bagging 69.04 38 tree j48 74.21 0 reptree 69.57 3 simplecart 71.19 0 underspend forest sysfor 69.01 62 randomforest 68.83 59 bagging 68.88 33 tree j48 69.47 0 reptree 70.26 2 simplecart 69.47 0 confirmed switchers forest sysfor 80.24 124 randomforest 76.21 197 bagging 76.39 82 tree j48 80.19 0 reptree 77.02 1 simplecart 76.2 0 table 1: analysis of data mining algorithm performance class attribute c=0.7 s=0.05 c=0.7 s=0.1 c=0.9 s=0.05 c=0.9 s=0.1 switching 17 3 7 2 plan type 155 76 73 30 underspend 71 24 0 0 confirmed switchers 244 132 14 3 table 2: sums of unique rules found for each class attribute 5.6.3 trend: low plan usage and customer service the majority (76.84%) of customers that only make 1 or 0 calls on average per week are prepaid customers. 97.89% of these customers indicate that customer service is important to them when changing mobile phone providers. australasian journal of information systems islam, d’alessandro, furner, johnson, gray & carter 2016, vol 20, research article brand switching pattern discovery by data mining techniques 14 5.7 follow-up study based analysis the original customers surveyed were surveyed once again a year later, and data was collected based on this. in this study, only the data available from the questions asked in this follow-up survey was used. 5.7.1 trend: issues surrounding confirmed switchers people who claim to be happy with their current service provider, but who say that mobile service providers did not live up to their expectations at all over the last 12 months, are all confirmed switchers. 40% of confirmed switchers claimed to have had network issues that lead to the switch. it would appear that network issues are a major component of why people switch. 5.7.2 trend: high plan usage and switching those pre-paid customers that make 13 or more calls a week were shown to be 70.87% likely to switch. 5.7.3 trend: fixed line bundles most people (90.41%) with a fixed line bundle did not switch. 6 conclusion in this study we analyse the brand-switching and customer-behaviour patterns through data mining techniques without using any other switching models and domain knowledge. we find that the discovered patterns are interesting, valid and useful when we evaluate them from a marketing point of view. we also compare this to highly used statistical methods from the marketing community, and find that data-mining techniques, especially decision forests, are useful for finding “deep” patterns within the data and provide a compliment to existing analyses. the lower rates of switching by women compared to men, can be explained by organisational commitment theory, which suggests women are more likely to be loyal to an organisation than men (wahn 1998). mobile phone switching research also suggests that men are more likely to consider alternative offers than women, and therefore would be more likely to switch (wang & acar 2006). the pattern that suggests that older (45 years and over) male customers have lower switching tendency can be explained by relationship age with the provider (ranaweera & menon 2008). older males are more likely to have longer relationships with a provider and are less likely to switch. we discover in this paper that better educated consumers are more likely to switch. a possible reason behind the pattern is the fact that educated people have better capacity to compare alternative offers. the research clearly shows the importance of service failures in switching. around 40% of consumers switch because of network issues. interestingly customer service becomes more of an issue for consumers who have a low usage levels. for consumers who make no more than one call per week, poor service is almost the only reason to switch providers. the relationship here may also be the other way round, as research suggests that consumers who encounter poor service, are less satisfied and will therefore use the service less prior to switching (bolton & lemon 1999). therefore, providers should focus on consumers who make fewer calls, as they are quite likely to be dissatisfied and are also likely to switch. our study also discovers that switching history of a customer can be important in analysing future switching behaviour. it was found in the previous research that frequent switchers in the past are likely to change providers again (roos & gustafsson 2007). this is because such consumers feel empowered to do so and act in more active fashion when selecting their australasian journal of information systems islam, d’alessandro, furner, johnson, gray & carter 2016, vol 20, research article brand switching pattern discovery by data mining techniques 15 providers. it is important for telecommunications marketers to know previous switching history, particularly if the consumer had switched in the 12 months prior to making the change. this indicates that such consumers are problematic, in terms of repeat purchases. the last and perhaps most interesting finding of our study is the lower switching rates (ranging between 0% and 39.47%) among the big three telco providers. we also find that the switching rates among small telco providers are high ranging between 56.98% and 100%. this pattern reflects the notion of double jeopardy�, whereby brands with low share, such as these small telcos, are associated with lower brand loyalty, than companies with larger market shares (bongers & hofmeyr 2010, labeaga-azcona, lado-couste & martos-partal 2010). overall this research shows that patterns of provider switching are not uniform in the market and that reasons for switching are heterogeneous depending on customer background and company type. we believe that unlike models of switching which aim to fit an overall explanation to a market or dataset (bansal & taylor 1999, colgate & hedge 2001, colgate & lang 2001, keaveney 1995, lees, garland & wright 2007, levesque & mcdougall 1993), this study provides a fruitful avenue for the development of a number of explanations of consumer behaviour in this context. importantly, our results show consistency of empirical generalisations of “double jeopardy”, found in other brand switching studies, which suggests that data mining approach can provide consistent results with that which use a more complex theoretical explanation. references ahn, j. h., han, s. p., & lee, y. s. (2006). customer churn analysis: churn determinants and mediation effects of partial defection in the korean mobile telecommunications service industry. telecommunications policy, 30(10), 552-568. annual report of the telecommunications industry ombudsman (n.d.), http://www.tio.com.au/. accessed: 2015-04-24. bansal, h. s. & taylor, s. f. (1999), ‘the service provider switching model (spsm)’, journal of service research 2(2), 200 – 218. bernard, s., adam, s. & heutte, l. (2012), ‘dynamic random forest’, pattern recognition letter 33(12), 1580 – 1586. berry, m. j. & linoff, g. s. (2004), data mining techniques: for marketing, sales, and customer relationship management, john wiley & sons. bolton, r. & lemon, k. (1999), ‘a dynamic model of customers’ usage of services: usage as an antecedent and consequence of satisfaction’, journal of marketing research 36, 171 – 186. bongers, m. & hofmeyr, j. a. n. (2010), ‘why modeling averages is not good enough’, journal of advertising research 50(3), 323 – 333. breiman, l. (1996), ‘bagging predictors’, machine learning 24(2), 123–140. breiman, l. (2001), ‘random forests’, machine learning 45(1), 5–32. breiman, l., friedman, j. h., olshen, r. a. & stone, c. j. (1984), classification and regression trees, wadsworth international group, belmont, california. colgate, m. & hedge, r. (2001), ‘an investigation into the switching process in retail banking services’, international journal of bank marketing 19(5), 201–212. colgate, m. & lang, b. (2001), ‘switching barriers in consumer markets: an investigation of the financial services industry’, journal of consumer marketing 18(4), 332 – 347. fayyad, u. m., piatetsky-shapiro, g., smyth, p. & uthurusamy, r. (1996), advances in knowledge discovery and data mining, the mit press. australasian journal of information systems islam, d’alessandro, furner, johnson, gray & carter 2016, vol 20, research article brand switching pattern discovery by data mining techniques 16 freitas, a. a. (1999), ‘on rule interestingness measures’, knowledge-based systems 12(5), 309–315. geurts, p., ernst, d. & wehenkel, l. (2006), ‘extremely randomized trees’, machine learning 63, 3 – 42. gray, d., d’alessandro, s. & carter, l. (2012), the state of the mobile nation, technical report, faculty of business and economics, macquarie university. hall, m., frank, e., holmes, g., pfahringer, b., reutemann, p. & witten, i. h. (2009), ‘the weka data mining software: an update’, acm sigkdd explorations newsletter 11(1), 10–18. han, j., kamber, m. & pei, j. (2006), data mining: concepts and techniques, morgan kaufmann. ho, t. k. (1998), ‘the random subspace method for constructing decision forests’, ieee transaction on pattern analysis and machine intelligence 20(1), 832 – 844. hung, s. y., yen, d. c., & wang, h. y. (2006). applying data mining to telecom churn management. expert systems with applications, 31(3), 515-524. islam, m. z. & giggins, h. (2011), knowledge discovery through sysfor: a systematically developed forest of multiple decision trees, in ‘proceedings of the ninth australasian data mining conference-volume 121’, australian computer society, inc., pp. 195–204. keaveney, s. m. (1995), ‘customer switching behavior in service industries: an exploratory study’, journal of marketing 59(2), 71 – 82. labeaga-azcona, j. m., lado-couste, n. & martos-partal, m. (2010), ‘the double jeopardy loyalty effect using discrete choice models’, international journal of market research 52(5), 633 – 652. lees, g., garland, r. & wright, m. (2007), ‘switching banks: old bank gone but not forgotten’, journal of financial services marketing 12(2), 146 – 156. levesque, t. j. & mcdougall, g. h. (1993), ‘managing customer satisfaction: the nature of service problems and customer exit, voice and loyalty’, asia pacific journal of quality management 2(2), 40–58. mobile phone statistics (n.d.), http://www.portal.euromonitor.com/portal/pages/statistics/statistics.aspx. accessed: 2015-04-24. quinlan, j. r. (1993), c4.5: programs for machine learning, morgan kaufmann publishers, san mateo, ca. rahman, m. g. & islam, m. z. (2013), ‘kdmi: a novel method for missing values imputation using two levels of horizontal partitioning in a data set’, in proc. of the 9th international conference on advanced data mining and applications (adma 13), hangzhou, china, 14-16 december 2013, pg. 250-263. (eds.). rahman, m. g. & islam, m. z. (2014), ‘fimus: a framework for imputing missing values using co-appearance, correlation and similarity analysis’, knowledge-based systems 56, 311– 327. ranaweera, c. & menon, k. (2008), ‘for better or for worse: moderating effects of relationship age and continuance commitment on the service satisfaction-word-of-mouth relationship’, in advances in consumer research latin american conference, vol. 2. roos, i. & gustafsson, a. (2007), ‘understanding frequent switching patterns’, journal of service research 10(1), 93 – 108. australasian journal of information systems islam, d’alessandro, furner, johnson, gray & carter 2016, vol 20, research article brand switching pattern discovery by data mining techniques 17 schultz, m. g., eskin, e., zadok, e. & stolfo, s. j. (2001), ‘data mining methods for detection of new malicious executables’, in proceedings of the ieee symposium on ‘security and privacy, 2001. ieee, pp. 38–49. vinzi, v. e., trinchera, l., squillacciotti, s. & tenenhaus, m. (2008), ‘rebus-pls: a responsebased procedure for detecting unit segments in pls path modelling’, applied stochastic models in business and industry 24(5), 439 – 458. wahn, j. c. (1998), ‘sex differences in the continuance component of organizational commitment’, group and organization management 23(3), 256 – 266. wang, a. & acar, s. a. (2006), ‘information search and mobile promotions’, international journal of mobile marketing 1(2), 80 – 87. wei, c. p., & chiu, i. t. (2002). turning telecommunications call details to churn prediction: a data mining approach. expert systems with applications, 23(2), 103-112. witten, i. h. & frank, e. (2005), data mining: practical machine learning tools and techniques, morgan kaufmann. copyright: © 2016 islam, d’alessandro, furner, johnson, gray & carter. this is an openaccess article distributed under the terms of the creative commons attributionnoncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 1.1 mobile phone brand switching 1.2 the problem with existing models 1.3 a data mining approach 1.4 dataset, context and pre-processing 1.5 major contributions of the study 2 data mining techniques a brief review 2.1 what is a classifier? 2.2 decision trees, forests and logic rules 3 our approach in analysing the dataset 4 data mining performance 5 observations and findings using data mining 5.1 demographics based analysis for brand switching 5.1.1 trend: age, gender, and brand-switching 5.1.2 trend: professional communities and brand-switching 5.1.3 trend: marital and familial status, and brand-switching 5.1.4 trend: income ranges and brand-switching 5.1.5 trend: education and brand-switching 5.2 demographics based analysis for properties other than brand switching 5.2.1 trend: relation between underspending and gender 5.2.2 trend: mobile service type and familial status 5.3 relationship-based analysis for brand-switching 5.3.1 trend: perceived industry ethics and brand-switching 5.3.2 trend: brand-switching and decreased plan usage 5.3.3 trend: employment and service type 5.4 other relationship-based analysis 5.4.1 trend: customer satisfaction, purchase channel, and plan-cost 5.5 overall preliminary survey analysis for brand-switching 5.5.1 trend: lifelong switching tendency 5.5.2 trend: company size and switching probability 5.6 overall preliminary survey analysis for other factors 5.6.1 trend: method of purchase 5.6.2 trend: contract length and customer happiness 5.6.3 trend: low plan usage and customer service 5.7 follow-up study based analysis 5.7.1 trend: issues surrounding confirmed switchers 5.7.2 trend: high plan usage and switching 5.7.3 trend: fixed line bundles 6 conclusion paper title (paper title style) australasian journal of information systems chong & olesen 2017, vol 21, research article toe perspective on eco-effectiveness 1 a technology-organization-environment perspective on eco-effectiveness: a meta-analysis josephine l.l.chong auckland university of technology auckland, new zealand josephine.chong@aut.ac.nz karin olesen the university of auckland auckland, new zealand abstract in this research, we perform a meta-analysis to explain how organizations are deploying technologies to enforce organizational sustainability by meeting the goal of eco-effectiveness. prior studies have studied the influences on the adoption of technologies using the technologyorganisation-environment (toe) model that incorporate some aspects of technological, organizational or environmental factors. we collected prior research to test the factors of the toe model to ascertain their relative impact and strength. our meta-analysis found eight additional technological and organizational factors. we found strong support for it infrastructure, perceived direct benefits, top management support, and competitive pressure. moderate support for compatibility, technological readiness, perceived indirect benefits, knowledge (human resources), organizational size, attitudes towards innovation, learning culture, pressure from trade partners (industry characteristics) and regulatory support. lastly, weak support was found for relative advantage, complexity, perceived risks and information learning culture. only two dimensions, financial resources and environmental uncertainty failed to reach statistical significance. keywords: green-it; toe; meta-analysis; eco-effectiveness 1 introduction in recent years, research on environmental sustainability has gained considerable prominence (melville, 2010; butler, 2011; dao et al., 2011). given a growing awareness that information and communications technology (ict) is accountable for 2% of the world’s total carbon dioxide (gartner, 2007), thus “green information technology (it)/information systems” (is) has pervasively emerged as an important concept in the world of is. more significantly, the adoption of green it/is benefits enables organizations to reap competitive advantages such as better operational performance, cost reduction, and gaining a positive image as a “green” organization (mithas et al. 2010; rusinko, 2007; thambusamy and salam 2010). to understand the concept of green it/is comprehensively, it is vital to distinguish between green it and green is. green it is defined as “the study and practice of designing, manufacturing, using, and disposing of computers, servers, and associated subsystems (monitors, printers, storage devices, etc.) efficiently and effectively with minimal or no impact on the environment” (murugesan, 2008, p. 25-26). while green is “refers to the design and implementation of information systems that contribute to sustainable business processes” (watson et al., 2008, p.2). along the same vein, molla and abareshi (2011) define green it as the first-order effects which are associated with the environmental impact of it production, use, and disposal. whereas they define green is as the second-order effect which is associated with the positive impact of using it on the environmental sustainability of business and economic processes. in sum, we define green it to involve activities related to the adoption of green it practices, and green is plays a crucial role in greening organizational processes so that a more sustainable environment can be created. this research emphasizes green it as we are examining the implementation of green initiatives. australasian journal of information systems chong & olesen 2017, vol 21, research article toe perspective on eco-effectiveness 2 several theories have been proposed to explain the factors influencing green it adoption. the common theories adopted are: resource-based view theory (benitez-amado and walczuch, 2012; dao et al., 2011), transaction cost theory (nedbal and stieninger, 2014), institutional theory (butler, 2011; cai et al., 2013; gholami et al., 2013) and technology-organizationenvironment (toe) (bose and luo, 2011). these theories provide different lenses through which to analyse the adoption of green it. for instance, benitez-amado and walczuch (2012) provide evidence that it capability enables the development of environmental strategy in enhancing organizational performance. similarly, dao and his colleagues argue that long-term adoption of green it can contribute to environmental performance. nedbal and stieninger (2014) use transaction cost theory to explain how green technologies can reduce operational costs and enable cleaner production thus leading to higher organizational performance. institutional theory has been widely used to explain how coercive, mimetic and normative forces can influence organizations to adopt green it in achieving environmental and economic performance. given the theories mentioned above, this research is employing toe to explain the adoption of green it. toe is a generic framework that identifies various factors that influence organizational adoption of technologies. there are many factors that would drive an organization to implement green it. according to lei and ngai (2012), toe is the only theoretical framework that encompasses all the drivers that can influence green it initiatives. typically, these antecedents of adopting green it can be broadly categorized into three contexts: technological, organizational and environmental. the resource-based view theory emphasizes on antecedents from the context of technological and organizational. in particular, the theory provides a framework for explaining the contribution of firm-specific technological/organizational resources and capabilities to influence green it initiatives (barney, 1991; eisenhardt and martin, 2000). transactional cost theory is used to analyse the co-ordination of transaction costs in the context of the green supply chain. the argument of transaction cost theory is that organizations are adopting it to support green supply chain management practices such as green procurement and green manufacturing so as to improve their environmental performance (tate et al., 2011). the three forces of institutional theory are concerned with the environmental antecedents that influence green it initiatives. coercive forces are enforcement mechanisms that legally regulate organizations to adopt green it initiatives (dimaggio and powell, 1983). such coercive institutional factors encompass legal obligations, legally enforceable sanctions and, laws and regulations. mimetic forces are mechanisms which support legitimacy through imitating the characteristics of other organizations. examples of mimetic environmental factors are frequency-based imitation and outcome-based imitation (chen et al., 2010). although there is a vast body of research utilizing toe in examining the factors contributing to the adoption of technological innovations, the findings are fragmented (sophonthummapharn, 2008). therefore, the first objective of this research is to address the issue of inconclusive research on the antecedents that would motivate an organization’s intention to adopt it by conducting a meta-analysis of the literature. by identifying all potential factors that can influence organizational it adoptions leads to our first contribution to the existing literature. the second objective is to extend the identified technological, organizational and environmental antecedents and apply them in the context of green it adoption. this meta-analytical review is a first attempt to identify empirically important antecedents of green it adoption. thus, this study makes an important contribution by presenting a meta-analysis of empirical evidence to assess these antecedents. to our best knowledge, it is the first study to perform meta-analysis in green it research. this research presents a meta-analytic review of primary quantitative studies that have investigated the relationship between green it adoption and the antecedents. meta-analysis has proven to be a useful research method for statistically aggregating results across previous empirical individual results on a set of related hypotheses. it allows for better estimates of the true effect size than an individual empirical study. it is useful in correcting for statistical artefacts such as sampling error and measurement error. our findings provide valuable insight australasian journal of information systems chong & olesen 2017, vol 21, research article toe perspective on eco-effectiveness 3 into the effect of antecedents on green it adoption. future research can use this study to understand the antecedent factors that influence green it initiatives. the remainder of the paper is organized into six sections. section two reviews background literature on green it and toe. section three develops the research framework and hypotheses. section four describes the research method. section five presents the results. section six presents the discussion of the research findings. section seven concludes with the main contributions, the limitations of the research and the implications for future research. 2 theoretical background 2.1 green it sustainability is a broad concept that comprises three dimensions: economic, social and environmental (ecological). extending the concept of sustainability to an organizational context, organizational sustainability can be understood as an organization’s capability to achieve positive organizational performance, while at the same time managing corporate environmental issues and engaging in corporate social responsibility practices (wikstrom, 2010). the economic dimension emphasizes on organizational activities that generate profits while reducing energy consumption and resource use (watson et al., 2010). the social dimension focuses on organizational activities that promote social responsibility that future generations should have equal rights over the consumption of scarce environmental resources (gary and bebbington, 2000). the environmental dimension relates to organizational activities that help to reduce negative environmental externalities, thus creating a more sustainable society (murugesan, 2008). the above mentioned three main pillars of sustainability are known as the “triple bottom line” concept (elkington, 1997). more importantly, it provides a blueprint for organizations to plan and facilitate their corporate social responsibility activities. organizations have established a set of eco-goals to evaluate their triple bottom line performance. the eco-efficiency, eco-equity, and eco-effectiveness goals are concerned about delivering business activities under economic pressure, about the equity of all generations in access to natural resources and about doing the right things. according to watson et al. (2010), • eco-efficiency refers to organizations producing competitive products and services, and on the other hand reducing costs using efficient use of resources to minimize negative impacts on the environment. • eco-equity refers to the equal distribution of scarce environmental resources between people and generations respectively. • eco-effectiveness refers to organizations designing and producing the right sustainable goods and services from the beginning. more importantly, eco-effectiveness seeks a permanent solution to restore and enhance the natural environment (young and tilley, 2006). the phenomenon of green it is becoming more prominent this decade. this is because organizations are under pressure to create sustainable business practices (olson, 2008). it has been argued that green it can be exploited to generate economic profits by creating a competitive advantage, support corporate social responsibility goals and manage environmental issues (bose and luo, 2011; dao et al., 2011; murugesan, 2008; watson et al., 2008). more specifically, green it plays an important role in enforcing organizational sustainability by meeting the three eco-goals. for instance, green it enables organizations to achieve eco-efficiency using greening the use of hardware and software to reduce energy consumption and waste, and designing data centres to reduce power consumption and carbon emissions (watson et al., 2008). deploying virtualization software allows organizations to maximize natural resource utilization thus achieving eco-equity (murugesan, 2008). organizations can meet the eco-effectiveness goal by substituting travel and physical meetings with telecommuting, telepresence or video-conferencing that helps to contribute to australasian journal of information systems chong & olesen 2017, vol 21, research article toe perspective on eco-effectiveness 4 environmental sustainability (watson et al., 2008). we summarized the three eco-goals and their respective types of green it in table 1. eco-goals types of green it eco-efficiency energy-efficient hardware, software, data centres. eco-equity server virtualization and storage, cloud computing, climate data sensors, remote monitoring systems. eco-effectiveness video conferencing, radio frequency identification, social networking, intelligent transport system. table 1: eco-goals and types of green it 2.2 toe it is conducive to examine the mainstream technological innovation literature to understand organizational motives for implementing green it initiatives. diffusion of innovation theory is an established theoretical framework for understanding technology adoption (roger, 1995). according to rogers, technological factors such as relative advantage, compatibility and complexity can influence the adoption of technological innovations. however, it is insufficient to evaluate an organization’s decision to adopt or not to adopt it innovations by assessing technological factors alone. as such, researchers extend the diffusion of innovation theory into their empirical research by incorporating organizational and environmental factors. prior literature has identified determinants of organizational innovation adoption which can be categorized into four contexts: technological, organizational, managerial action and institutional (hameed et al., 2012, tsai et al., 2013). for example, technological factors are associated with relative advantage, complexity, compatibility, it capability and technology competence (low et al., 2011; seethamraju, 2015; zhu and kraemer, 2005). organizational factors refer to organizational size, organizational readiness, and employees’ attitude towards technology and ownership type (jackson et al., 2013; sila, 2010, tsai et al., 2013). environmental factors encompass competitive pressure, trading partner pressure, support from the government and environmental dynamism (low et al., 2011; sila, 2010). though the motives for green it adoption are similar to other types of innovation adoptions, there are some differences between green initiatives and traditional initiatives (olson, 2008). for instance, the start-up costs for green technologies are more expensive. therefore, organizations need a longer time to reach a breakeven. furthermore, organizations have to develop new organizational skills and transform business processes to adapt to the fresh green technologies. we draw upon the theory of toe to explain the organizational adoption of green it. it is a useful theoretical framework for studying factors that influence the adoption of green technological innovations. toe was developed by tornatzky and fleischer in 1990 which emphasize on the adoption of technological innovation. the theoretical framework identifies three dimensions of an organization's context that influence technological innovation adoption and implementation: technological context, organizational context, and environmental context. the technological context relates to both the internal and external technologies relevant to the organization. organizational context refers to descriptive measures about the organization such as size, scope, managerial structure and internal resources. the environmental context refers to an organization’s industry, competitors and government policy on intention. 3 research model and hypotheses figure 1 presents the conceptual model of our study. our model suggests that specific factors within the technological, organizational and environmental contexts can underpin green initiatives implementation. we begin unpacking the research model by first reviewing the australasian journal of information systems chong & olesen 2017, vol 21, research article toe perspective on eco-effectiveness 5 different determinants within each context that influence adoption of technological innovation. then, we discuss the hypothesized relationships between these key factors and organizational adoption of green it. figure 1: conceptual model 3.1 technological context in a meta-analytic review of the studies of innovation attributes, tornatzky and fleischer (1990) identified ten innovation factors that have been widely used to study technology adoption in organizations. they are a relative advantage, complexity, communicability, divisibility, cost, profitability, compatibility, social approval, trial-ability and observability. prior research has suggested that relative advantage, compatibility and complexity are consistently related to innovation adoption (agarwal and prasa, 1998; cooper and zmud, 1990; premkumar and ramamurthy, 1995). the hypothesis in this study will focus only upon on the most commonly studied factors in the technological innovation literature. the toe framework is consistent with the innovation theory of rogers (1995, p. 376-383). relative advantage is the “the degree to which an innovation is perceived as being better than the idea it supersedes” (rogers, 1995, p. 213). compatibility is the “degree to which an innovation is perceived as consistent with the existing socio-cultural values and beliefs, past experiences, and needs of potential adopters” (rogers, 1995, p. 223). complexity is the degree to which an innovation is perceived as relatively difficult to understand and use” (rogers, 1995, p. 230). green technological adoptions share similar fundamental characteristics that tradition technological innovation implementations do. that is to say, technological factors such as relative advantage, compatibility and complexity have significant relationships to organizational green it adoption. organizations are more likely to implement green technological innovations if these green technologies can bring about perceived organizational benefits such as better organizational performance and higher economic gains. studies (nedbal and stieninger, 2014; oliveira et al., 2014) show that economic advantages are an important technological characteristic that influences the adoption of cloud computing. furthermore, green initiatives implementation can be driven by green technologies that are more compatible with an organization’s technologies, processes and work application systems (ho and lin, 2012; weng and lin, 2011). therefore, the following hypotheses are proposed: hypothesis 1: relative advantage has a positive influence on green it adoption. technological context relative advantage compatibility complexity organizational context quality of human resources top management support organizational size environmental context  industry characteristics regulatory support green it adoption australasian journal of information systems chong & olesen 2017, vol 21, research article toe perspective on eco-effectiveness 6 hypothesis 2: compatibility has a positive influence on green it adoption. the literature suggests that more complex green it initiatives tend to have a negative influence on the adoption of new green innovations. green it initiatives implementation can be complex because it requires an organization to disseminate explicit knowledge about the initiatives to their employees. subsequently, employees have to codify the explicit knowledge into tacit knowledge which then is shared (nonaka and takeuchi, 1995). for more complex green it initiatives, employees may take a longer time to understand and adapt to the new green technology as they need more effort to learn and share the tacit knowledge about both technological and administrative adjustments (sila, 2010). therefore, the following hypothesis is proposed: hypothesis 3: complexity has a negative influence on green it adoption. additional factors in the toe model such as technological readiness, it infrastructure, perceived direct benefits and perceived indirect benefits were hypothesized as a positive influence on adoption of green it from prior literature studies shown in appendix 1. technological readiness was suggested as positive as the higher amount of structural aspects such as installed network technologies as well as specialized human resources an organization has, the more positive the adoption of green it (oliveira et al., 2014; pan and jang, 2008). it infrastructure also relates to higher amount of internet related resources in it infrastructure, the more positive the adoption of green it (soares-aguiar and palma-dos-reis, 2008; pan and jang, 2008). perceived direct benefits relate to a reduction in transaction errors and improved accuracy, and indirect benefits relate to customer relationship. both types of benefits are positive to the green it adoption (teo et al., 2009). perceived risks were hypothesized as a negative influence on adoption of green it from prior literature studies shown in appendix 1. risks in the online environment such as private data disclosure and hacking could bring about drastic setbacks to business. therefore, businesses are cautious about their investment and perceived risks with the negative influence on the adoption of green it (ghobakhloo and tang, 2011). these additional factors were also tested to see if their relationship strength is stronger than the widely used factors in the conceptual model. 3.2 organizational context the organizational context refers to the characteristics, structures, processes and resources that constrain or facilitate technological innovation adoption. tornatzky and fleischer discuss the influences of a variety of organizational factors such as the role of informal linkages and communication among employees, the quality of human resources, top management’s leadership behaviour, the amount of internal slack resources, and organizational size. among all these factors, the quality of human resources, top management's leadership, slack resources and size, are the most frequently discussed factors with the organizational context that affect innovation adoption. the hypothesis in this study will focus only upon on the most commonly studied factors in the organizational context innovation literature. the quality of human resources refers to the extent to which technical know-how knowledge is available within an organization. the study of cooper and molla (2014) provides empirical evidence that the greater the employees’ absorptive capacity to generate green it knowledge, it is more likely for the organization to adopt successfully green initiatives. top management’s leadership describes the role of executive leadership in encouraging and facilitating innovation within the organization’s overall strategy. in particular, top management commitment plays an important role in driving various initiatives for green it during the implementation stage. cooper and molla (2014) in their findings identified top management commitment as an essential factor in steering the organization towards the strategic importance of green it and realizing business value from green it adoption. similarly, the empirical study of lin et al. (2011) found that top management support has a positive influence on the adoption of cloud computing. therefore, the following hypotheses are proposed: hypothesis 4: quality of human resources has a positive influence on green it adoption. australasian journal of information systems chong & olesen 2017, vol 21, research article toe perspective on eco-effectiveness 7 hypothesis 5: top management support has a positive influence on green it adoption. prior research has been inconclusive about the effect of both organizational size and slack resources on innovation adoption. tornatzky and fleischer (1990) contend that larger organizations are more likely to adopt innovations as they have adequate slack resources and strong infrastructures to facilitate the process of technological implementation. however, larger organizations are associated with bureaucratic inertia and less flexible structure, therefore; they have more difficulty in accepting and implementing technological change (damanpour, 2010). while smaller organizations are more likely to facilitate innovation usage as they have less bureaucratic inertia and more flexible structure, allowing them to benefit from lesser communication, coordination and influence to get support to implement the technological change (nord and tucker 1987). the literature suggests that larger companies tend to be more active in adopting green initiatives (gonzález et al., 2008; kolk, 2010). therefore, we argue that bigger organizations are more likely to undertake the implementation of green initiatives and the concept of organizational inertia leads us to hypothesize: hypothesis 6: organizational size has a negative influence on green it adoption. additional factors in the toe model in the organizational context such as attitudes towards innovation, information sharing culture, learning culture and financial resources were hypothesized as a positive influence on adoption of green it from prior literature studies shown in appendix 1. these factors were tested to see if their relationship strength was stronger than the main hypothesized factors on organizational context. attitudes towards innovation are argued as positive as the attitude by top management will result in a more positive implementation also if they perceive that the benefits of adopting a new system will outweigh its risks and expenses they would be more willing to implement the system (ghobakhloo and tang, 2011). information sharing culture relates to implementing systems such as e-procurement that require a business to share data with its business partners; this means that the process is quicker and has lower costs and lower inventory. the sharing of the data is of benefit and will have a positive influence on green it adoption (teo et al., 2009). learning culture is reasoned as the more an organization can learn from external consultants and from accumulating knowledge on the it innovations within the firm the more positive the influence on the innovation will be (henderson et al., 2012). financial resources relate to the amount of resource the organization has to carry out the implementation. so more financial resources are positively relating to the adoption of green it (kuan and chau, 2001). 3.3 environmental context tornatzky and fleischer (1990) identify three categories of environmental factors: industry characteristics, government regulation, and technology support infrastructure. this study will focus on industry characteristics and government regulations that are the most commonly studied factors. the industry characteristics are comprised of competitive pressure and pressure from trading partners. this study defines competitive pressure as the pressure that arises from a threat to losing or maintaining competitive advantage, forces organizations to search for new technological adoption as alternatives to their current strategies (lin and lin, 2008). as more and more organizations are adopting green initiatives, this pervasive phenomenon has been pressuring organizations to adopt green technologies so as they can stay competitive in the industry or out-perform their competitors. the empirical study of oliveira et al. (2014) contends that organizations are increasingly adopting cloud computing as they face competitive pressure from their competitors, and consequently follow their competitors’ adoption of green technologies. furthermore, the adoption of green it can also be driven by the organizations’ trading partners. for instance, when walmart adopted radio frequency identification, the organization made it a mandatory for its top 100 suppliers to adopt the new technology for the shipping container and case identification (wang et al., 2006). therefore, the following hypothesis is proposed: hypothesis 7: industry characteristics have a positive influence on green it adoption. australasian journal of information systems chong & olesen 2017, vol 21, research article toe perspective on eco-effectiveness 8 regulatory support has been identified as a critical environmental factor influencing technological adoption within the toe framework (kraemer et al., 2002). government regulations can either motivate or discourage organizations from adopting technological innovations. when governments have specific mandates such as pollution control devices for energy firms and stringent safety and testing requirements on industries, these government regulations are more likely to constrain green initiative adoption (baker, 2011). while regulatory support such as the provision of financial support, essential resources and tax breaks can encourage organizations to adopt a technological innovation (tornatzky and fleischer, 1990; zhu and kraemer, 2005). the taiwanese government imposes green constraints on the electronics industry, and such a regulatory environment consequently motivates those organizations to adopt green it (chen and chang, 2014). therefore, the following hypothesis is proposed: hypothesis 8: regulatory support has a positive influence on green it adoption. an additional factor in the toe model in the environmental context named environmental uncertainty was hypothesized as a negative influence on adoption of green it from prior literature studies shown in appendix 1. environmental uncertainty relates to the fact that environmental uncertainty occurs when there are complex and rapid changes and information technology is necessary to be adopted to cope with this. however, they are unlikely to adopt these technologies without the support of mature infrastructure and institutions which is required for the operation of e-business. so that environmental uncertainty has a negative influence on influence of green it adoption (li et al., 2010). this factor was tested to see if their relationship strength was stronger than the main hypothesized factors on environmental context. 4 research methodology a meta-analysis approach (hunter and schmidt, 1990) was adopted to test the proposed hypotheses. a meta-analysis is a quantitative approach that aggregates findings across individual studies (see hunter and schmidt, 1990). the advantage of meta-analysis is to reconcile conflicting results among the research findings so as to study the strength of the variables underlying relations and causalities. this meta-analysis reviewed on empirical studies that examine how technological, organizational and environmental factors can determine technological adoption. meta-analysis is a technique that evaluates prior statistical findings by accumulating effect sizes across all empirical studies. in this study, effect sizes are described as correlation coefficients. more significantly, meta-analysis enables the researchers to discern whether relationships exist and provide overall estimates regarding the magnitude and direction of those relationships. we followed the meta-analysis process recommended by hunter and schmidt (1990) to aggregate and analyse correlations reported by prior research to draw valid conclusions. the meta-analysis procedures are described below (see figure 2). figure 2: research process 4.1 identifying relevant studies we performed a comprehensive search to identify any published journal articles of technological adoption that have studied in the context of toe. we began by conducting a computer search with databases such as acm digital library, computer source [ebsco], ieee xplore, igi global, proquest computing, sciencedirect, scopus, springerlink, and web australasian journal of information systems chong & olesen 2017, vol 21, research article toe perspective on eco-effectiveness 9 of knowledge. keywords searched included ‘technology-organizational-environment’, ‘adoption, ‘technology’, and ‘innovations’. a total of 122 published empirical studies were found in the initial search. after gathering the journal articles, we then applied three criteria to remove any unrelated to the studied phenomenon. first, we excluded studies that were purely theoretical and or review. second, we excluded studies without quantitative empirical data. third, we included studies that focused on toe framework to study it adoption. fourth, we included studies that reported a pearson correlation between dependent and independent variables. applying these criteria, we ended up with 28 studies forming the basis for this meta-analytic investigation. the summary of the studies and samples used in the meta-analysis is found in table two. 4.2 variable coding the 28 empirical studies were coded based on the toe framework. we coded technological adoption as the dependent variable while the technological, organizational and environmental factors as the independent variables. the empirical studies were coded independently by two of the authors. the first and second authors conducted the coding independently based on the variable names, construct definition and measures used in the empirical studies. overall, there was a clear consensus about the coding of the variables as most of the initial codes were not ambiguous. the two authors resolved any discrepancies via discussion. most of the 28 empirical studies refined the toe framework to include more technological, organizational and environmental variables. a total of 19 variables were obtained. we will discuss how we coded the variables that were used in each article. technological factors: the technological variables that examined in the technological adoption research are a relative advantage, compatibility, complexity, technological readiness, it infrastructure, perceived direct benefits, perceived indirect benefits and perceived risks. the coding scheme is depicted in appendix 1. organizational factors: the organizational variables that examined in the technological adoption research are attitudes towards innovation, financial resources, organizational size, knowledge, information sharing culture, learning culture and top management support. the coding scheme is depicted in appendix 1. environmental factors: the environmental variables that were examined in the technological adoption research are competitive pressure, environmental uncertainty, regulatory support and trading partner readiness. the coding scheme is depicted in appendix 1. technological adoption: in existing technological adoption research, different it adoptions that have been examined encompass: collaborative commerce (chong et al., 2009); cloud computing (oliveira et al., 2014) enterprise application (gu et al., 2010; henderson et al., 2012; ramdani et al., 2013); electronic business (alawneh et al., 2010; eze, 2008; oliveira and martins, 2010); electronic commerce (ekong et al., 2012; ghobakhloo et al., 2011; ghobakhloo and tang, 2011; hong and zhu, 2006; le et al., 2012; li et al., 2010; sila, 2013), electronic data interchange (kuan and chau, 2001; seyal et al., 2007) electronic procurement (cui et al., 2008; li , 2008; soares-aguiar and palma-dos-reis, 2008; teo et al., 2009); enterprise resource planning (pan and jang, 2008); electronic supply chain management system (lin, 2014); mobile business (picoto et al., 2014); radio frequency identification (paydar and endut, 2013; thiesse et al., 2011; wang et al., 2010; wu and subramaniam, 2011). 4.3 analysis we used the hunter and schmidt (1990) method of calculating statistically corrected effect size estimates. this study measured an estimate of population correlation ρ by using pearson’s correlation coefficient r. we performed the weighted mean effect size (ř) to correct for sampling error, which ř considers the sample size of each study and thus creates a weighted average of correlations. based on hunter and schmidt (1990), the estimate of the population correlation is given by: ř = ∑niri / ∑ni, where ni is the sample size of each study and ri is the observed australasian journal of information systems chong & olesen 2017, vol 21, research article toe perspective on eco-effectiveness 10 correlation value of each study. all estimates were calculated using comprehensive metaanalysis software (borenstein et al., 2005). for each study, we obtained the following information: sample size, the reliability of constructs (as reported using cronbach’s alpha or if not available the reported composite reliability or internal consistency scores) and correlation coefficients for each pair of the relationship. for studies that did not provide an average reliability, we used a conservative standard of 0.80 (bommer et al., 1995). some of the studies used logistic regression to calculate the odd ratios as the effect size estimates. we converted these effect size estimates into pearson’s r to facilitate interpretation (see borenstein et al., 2009). we reported the weighted mean correlation coefficients for each bivariate relationship. confidence intervals were constructed around the weighted mean correlation coefficients to facilitate hypothesis testing. we corrected for measurement errors due to unreliability in measurement (hunter and schmidt, 1990). we calculated the correlation between the variables to correct for measurement error. according to hunter and schmidt (1990), the estimate of the true score correlation is given by: rc = rxy/(√rxx)(√ryy), where rc is the effect size corrected for measurement error, rxy is the reported correlation between the variables, rxx is the reliability estimate for the independent variable and ryy is the reliability estimate for the dependent variable. 5 results table 2 summarizes the results of our hypothesis tests. examining the 95% confidence interval of each hypothesis, we found that the majority of the hypotheses were supported. contrary to expectations, we did not find a significant negative relationship between organizational size and green it adoption. all technological factors were found to be significantly related to green it adoption: relative advantage (ρ=0.255, p=0.000), compatibility (ρ=0.401, p=0.003), complexity (ρ=-0.099, p=0.027), technological readiness (ρ=0.406, p=0.000), it infrastructure (ρ=0.585, p=0.000), perceived direct benefits (ρ=0.591, p=0.000), perceived indirect benefits (ρ=0.303, p=0.000), and perceived risks (ρ=-0.306, p =0.000). of the 7 organizational factors, 6 were found to be positive significantly related to green it adoption: attitudes toward innovation (ρ=0.424, p=0.008) knowledge (ρ=0.403, p=0.000), organizational size (ρ=0.429, p=0.000), top management support (ρ=0.511, p=0.000), information sharing culture (ρ=0.153, p=0.003), and learning culture (ρ=0.482, p=0.000). the relationship with financial resources (ρ=0.085, p =n.s.) was not significant. other than environmental uncertainty (ρ= -0.127, p=n.s.), all of the other environmental factors were found to be significantly related to green it adoption: competitive pressure (ρ=0.548, p=0.000), pressure from a trading partner (ρ=0.300, p=0.009), and regulatory support (ρ=0.342, p =0.000). in total, we found significant relationships for 17 of our 19 variables. only two variables, financial resources and environmental uncertainty failed to reach statistical significance. all of the variables were significant in the predicted directions. while organizational size (ρ=0.429, p=0.000) was found to be positively significantly related to green it adoption. the results are presented in table 2 and 3. australasian journal of information systems chong & olesen 2017, vol 21, research article toe perspective on eco-effectiveness 11 hypothesis total sample size for the given metaanalysis (k) number of studies included in the meta-analysis (n) corrected population correlation (ρ) 95% confidence interval low upper effect size (p) result technological factors – green it adoption h1: relative advantage 8 2090 0.255* 0.133 0.370 0.000 supported h2: compatibility 8 2062 0.401* 0.142 0.142 0.003 supported h3: complexity 9 2184 -0.099* -0.185 -0.011 0.027 supported organizational factors – green it adoption h4: knowledge (human resources) 2 1917 0.403* 0.207 0.567 0.000 supported h5: top management support 11 2029 0.511* 0.389 0.615 0.000 supported h6: organizational size 12 5280 0.429* 0.227 0.596 0.000 not supported environmental factors – green it adoption h7: competitive pressure (industry characteristics) 16 7286 0.548* 0.404 0.666 0.000 supported h7: pressure from trading partner (industry characteristics) 3 607 0.300* 0.078 0.495 0.009 supported h8: regulatory support 9 4118 0.342* 0.184 0.483 0.000 supported table 2: summary of hypothesis testing additional factors in the toe model total sample size for the given meta-analysis (k) number of studies included in the meta-analysis (n) corrected population correlation (ρ) 95% confidence interval low upper effect size (p) technological factors – green it adoption technological readiness 6 1466 0.406* 0.362 0.448 0.000 it infrastructure 3 1879 0.585* 0.355 0.749 0.000 perceived direct benefits 2 719 0.591* 0.541 0.636 0.000 perceived indirect benefits 2 719 0.303* 0.019 0.542 0.000 perceived risks 2 1194 -0.254* -0.306 -0.200 0.000 organizational factors – green it adoption attitudes towards innovation 2 1161 0.424* 0.117 0.657 0.008 information sharing culture 2 376 0.153* 0.052 0.250 0.003 learning culture 2 372 0.482* 0.400 0.557 0.000 table 3: meta-analysis results australasian journal of information systems chong & olesen 2017, vol 21, research article toe perspective on eco-effectiveness 12 6 discussion research on green it started in 2007 in the is field and many papers have provided a literature review of the existing knowledge and gaps thereof as well as future research directions (tushi et al., 2014; loeser, 2013; dedrick, 2010; brocke et al., 2013; molla et al., 2011). these papers on literature reviews of green is/it have listed the theories or frameworks used to study green is and have noted most of these theories/frameworks have only been used once. the issue with this is the research in green it is immature and as such there is not consistent knowledge being consolidated on the field and few empirical studies on green it being performed (wang et al., 2015a; wang et al., 2015b; sahu and srivastava, 2011; brooks et al, 2010). the varying theories and frameworks have also been used to study the green it at different parts of its lifecycle from the initiation to strategy, to adoption then use. thus, papers on green it/is have many suggestions on research gaps and programs of research that should be undertaken (wang et al., 2015). given the absence of a theoretical framework as a major gap in the green it literature and a limited number of papers have presented empirical research papers in green it. one of the theoretical frameworks that have been reasonably consistently applied is toe. toe is a reasonable framework in that it indicates what the antecedents of green it adoption success would be. it also covers the different areas that researchers would want to examine in adoption – the technological, the environmental and the organizational. most antecedents would be able to be classified under one of these three areas. there have been 28 papers in the adoption area of green it systems which does provide us with a basis for performing a meta-analysis concentrating on green it and adoption. we found that the majority of hypotheses implied by toe were supported. table 2 summarizes the results of our hypothesis tests. h1 relative advantage and h3 complexity were supported but not as strongly as expected. h2 compatibility, h4 human resource knowledge, h7 pressure from trading partner and h8 regulatory support were moderately strong. h5 top management support and h7 competitive pressure we strongly supported. one unsupported hypothesis h6 was associated with organisational size. the prior research on this factor was inconclusive, and we hypothesized that organizational size has a negative influence on green it adoption. the effect was found to moderately positive. one of the benefits of a meta-analysis is its ability to examine the strength of the relationships between two constructs. additional factors that were in the toe model were also coded from prior studies during data analysis, and this enabled additional factors to be considered. table 3 shows that the additional attributes technological factors (technological readiness, it infrastructure, perceived direct benefits, perceived indirect benefits and perceived risks) and for organizational factors (attitudes towards innovation, information sharing culture and learning culture) were also significant. on the other hand, financial resources and environmental uncertainty failed to reach statistical significance. a plausible reason for this result might be these organizations perceive it adoption as a critical organizational strategy. therefore, despite insufficient financial resources and environment uncertainty, organizations would still persist with their adoption. the meta-analysis quantifies the strength and magnitude of the relationship between two constructs. the general guideline to judge the effect sizes is by looking at the values of the correlation coefficient. correlation coefficients are not precise but are generally classified as weak, moderate and strong. in our study, we assumed 0.00-0.09 to be insignificant, 0.10-0.29 to be weakly significant, 0.30-0.49 to be moderately significant, 0.50-0.69 to be strongly significant (cohen and cohen, 1983). table 4 shows the magnitude of these relationships with technological factor complexity at -0.990 on the border line of weakly significant. however, several variables only had two data sets. while this is sufficient to perform a meta-analysis, more studies would be useful for this variable. for example, more studies would also be useful in performing moderator analysis on size, type of green it and industry type. australasian journal of information systems chong & olesen 2017, vol 21, research article toe perspective on eco-effectiveness 13 relationship strength total sample size for the given meta-analysis (k) number of studies included in the meta-analysis (n) corrected population correlation (ρ) 95% confidence interval low upper effect size (p) strong technological factors – green it adoption it infrastructure 3 1879 0.585* 0.355 0.749 0.000 perceived direct benefits 2 719 0.591* 0.541 0.636 0.000 organizational factors – green it adoption h5: top management support 11 2029 0.511* 0.389 0.615 0.000 environmental factors – green it adoption h7: competitive pressure (industry characteristics) 16 7286 0.548* 0.404 0.666 0.000 moderate technological factors – green it adoption h2: compatibility 8 2062 0.401* 0.142 0.142 0.003 technological readiness 6 1466 0.406* 0.362 0.448 0.000 perceived indirect benefits 2 719 0.303* 0.019 0.542 0.000 organizational factors – green it adoption h4: knowledge (human resources) 2 1917 0.403* 0.207 0.567 0.000 h6: organizational size 12 5280 0.429* 0.227 0.596 0.000 attitudes towards innovation 2 1161 0.424* 0.117 0.657 0.008 learning culture 2 372 0.482* 0.400 0.557 0.000 environmental factors – green it adoption h7: pressure from trading partner (industry characteristics) 3 607 0.300* 0.078 0.495 0.009 h8: regulatory support 9 4118 0.342* 0.184 0.483 0.000 weak technological factors – green it adoption h1: relative advantage 8 2090 0.255* 0.133 0.370 0.000 h3: complexity 9 2184 -0.099* -0.185 -0.011 0.027 perceived risks 2 1194 -0.254* -0.306 -0.200 0.000 organizational factors – green it adoption information sharing culture 2 376 0.153* 0.052 0.250 0.003 table 4: magnitude of relationships australasian journal of information systems chong & olesen 2017, vol 21, research article toe perspective on eco-effectiveness 14 overall, given the original hypotheses, as well as the additional factors coded from the prior research were empirically supported. a model showing both the hypothesized factors and the additional factors is shown in figure 3. technological adoption of green it (particularly eco – effectiveness)organizational environmental h1: relative advantage h2: compatibility h3: complexity technological readiness it infrastructure h4: knowledge (human resources) perceived risks perceived indirect benefits perceived direct benefits h5: top management support h6: organizational size (not supported negative, but supported positive) h7: competitive pressure (industry characteristics) learning culture information sharing culture attitudes towards innovation h7: pressure from trading partner (industry characteristics) h8: regulatory support +ve +ve +ve +ve +ve +ve +ve +ve +ve +ve +ve +ve -ve -ve +ve +ve +ve figure 3: toe model for adoption of green it particularly eco-effectiveness original studies have shown green it as a single technology or practice. table 1 provided the types of green it and classified them as eco-efficient, eco-equity and eco-effectiveness. when we categorize the papers in this meta-analysis, all papers bar one fit in the eco-effectiveness category (refer to table 5). transcending this result into the green it context, when organizations making decisions about their it/is adoption, they are more conscious to implement technologies that restore and enhance the natural environment (young and tilley, 2006). a plausible reason why there is one paper in eco-equity category is that cloud computing research is still at an early stage thus it has many adoption issues (jing et al., 2013). also, it is costly to implement cloud computing as it needs a large-scale computing infrastructure (lee et al., 2014). no papers were categorized in the eco-efficiency category. given the findings relate to the eco-effectiveness domain of green it we can show the model in figure 3 using the hypothesis and the additional factors from the toe model that have moderate and strong strength. the magnitude of relationships in table 4 indicates our conceptual model in figure 1 was not correct. empirically factors that we had not hypothesized turned out to be important. the rationale for their importance also needs to be investigated in future research. figure 3 shows that the conceptual model for adoption of green it (ecoeffectiveness) is in reality closer to the general it adoption model. it would be interesting to see if this is also found with eco-efficiency and eco-equity domains of green it. australasian journal of information systems chong & olesen 2017, vol 21, research article toe perspective on eco-effectiveness 15 papers in the meta-analysis classified by type of green eco-efficiency eco-equity cloud computing (oliveira et al., 2014) eco-effectiveness collaborative commerce (chong et al., 2009); enterprise application (gu et al., 2010; henderson et al., 2012; ramdani et al., 2013); electronic business (alawneh et al., 2010; eze, 2008; oliveira and martins, 2010); electronic commerce (ekong et al., 2012; ghobakhloo et al., 2011; ghobakhloo and tang, 2011; hong and zhu, 2006; le et al., 2012; li et al., 2010; sila, 2013), electronic data interchange (kuan and chau, 2001; seyal et al., 2007); electronic procurement (cui et al., 2008; li, 2008; soares-aguiar and palma-dos-reis, 2008; teo et al., 2009); enterprise resource planning (pan and jang, 2008); electronic supply chain management system (lin, 2014); mobile business (picoto et al., 2014);radio frequency identification (paydar and endut, 2013; thiesse et al., 2011; wang et al., 2010; wu and subramaniam, 2011). table 5: papers in meta-analysis classified by type of green it the literature on green it has discussed the stages of green it adoption. in particular, the earlier papers raising awareness about environmental issues and particularly the relationships between information systems and sustainability (el idrissi and corbett, 2016). next literature moved on to a firm's motivations and their readiness to adopt green it/is (deng et al., 2015). after this later papers moved onto ways to assess the antecedents of green it adoption and their level of adoption. this paper presents first attempt to identify empirically the important antecedents from all prior studies influencing the adoption of green it in particularly the area of eco-effectiveness. originally, this was conceptualized as being different than the general it adoption model, however, figure 4 shows us that the green it adoption model for ecoeffectiveness has emerged more like a general it adoption model than a distinctive green it adoption model. a limitation of this research is that the papers used in the meta-analysis only cover the ecoeffectiveness area of green it. future research into the toe model would be interesting if additional papers were published on eco-equity and eco-efficiency areas that included sufficient information for them to be used in a meta-analysis and these areas constructs could be tested. also, this paper concentrated on adoption of green it, if there were sufficient empirical papers other parts of the green it life cycle such as initiation, strategy and use could be investigated as well. australasian journal of information systems chong & olesen 2017, vol 21, research article toe perspective on eco-effectiveness 16 figure 4: final toe model for adoption of green it particularly eco-effectiveness australasian journal of information systems chong & olesen 2017, vol 21, research article toe perspective on eco-effectiveness 17 7 conclusion the general literature on green it has been fragmented and not cohesive (tushi et al., 2014; wang et al., 2015a; wang et al., 2015a ). while past studies have looked at antecedents to green it – the frameworks used have been fragmented. however, the prior literature does include 28 studies that have explicitly examined antecedents of adoption of green it using the toe framework. this is enables the comprehensive examination of environmental, technological and organizational factors using a consistent theory. our work assesses the toe framework for the adoption of green it. we have examined aspects of the toe model – technological, organizational and environment that relate to green it adoption using meta-analysis. by using meta-analysis we have been able to get a clear understanding of the relative impact and strength of the antecedents of green it adoption. we found that the green it antecedents did not significantly differ from those of general it adoption. the findings of the meta-analysis allowed us to draw more definitive conclusions on the relationship between the toe model and adoption of green it. we have empirically examined the relationships using meta-analysis and found the majority of the relationships to be supported. however, the empirical papers used in the meta-analysis did support the ecoeffectiveness type of green it. this is particularly important as the current empirical papers cover the eco-effectiveness area bar one. this model is a significant contribution to the area of adoption of green it. however, some limitations have to be considered. we included only studies that performed correlation analysis. for some variables, the number of data sets available were not adequate to perform the meta-analysis. similarly, the study did not evaluate the moderator factor effect for variables due to the lack of data. we used studies that provided correlation values for the relationship between toe variables and green it adoption. our findings have considerable significance in understanding the determinants of green it adoption relating to eco-effectiveness. the study provides researchers and practitioners with a set of factors that affect the adoption of green it in organizations. we found strong support for it infrastructure, perceived direct benefits, top management support, and competitive pressure. moderate support for compatibility, technological readiness, perceived indirect benefits, knowledge (human resources), organizational size, attitudes towards innovation, learning culture, pressure from trade partners (industry characteristics) and regulatory support. lastly, weak support was found for relative advantage, complexity, perceived risks and information learning culture. only two dimensions, financial resources and environmental uncertainty failed to reach statistical significance. organizations do not have a clear understanding of what factors are important in green it adoption. managers need to consider these issues when embarking on green it adoption for eco-effectiveness. these may not be appropriate for eco-equity and eco-efficiency types of green it adoption. more empirical research is needed on these areas. we hope this framework will point researchers in useful directions in engaging in future studies with green it. future studies utilizing papers that cover green it from the eco-equity and eco-efficiency would be interesting to see if the same toe model as developed in this article were supported for these other types of green it. different models of green it adoption could be developed based on the type of green it this offers organizations more understanding of the issues related to green it adoption. given the small number of studies currently, moderator analysis was not examined. however, future research with a larger number of empirical papers could provide analysis size, industry type, information system type and type of green it as moderators. references agarwal r, prasa j. (1998) “a conceptual and operational definition of personal innovativeness in the domain of information technology”, information systems research 9(2): 204– 301. australasian journal of information systems chong & olesen 2017, vol 21, research article toe perspective on eco-effectiveness 18 alawneh a, hasan a, khaldoun b (2010) e-business adoption by jordanian banks: an exploratory study of the key. in actor-network theory and technology innovation: advancements and new concepts. baker j (2011) the technology-organization-environment framework. in information systems theory: explaining and predicting our digital society. springer, new york, pp. 231-246. benitez-amado j, walczuch rm (2012) “information technology, the organizational capability of proactive corporate environmental strategy and firm performance: a resource-based analysis”. european journal of information systems 21:664-679. bommer wh, johnson jl, rich ga, podaskoff pm, mackenzie sb (1995). on the interchangeability of objective and subjective measures of employee performance: a metaanalysis. personnel psychology 48:587–605. borenstein mj, hedges lv, higgins jpt, rothstein hr (2009) converting among effect sizes. in introduction to meta-analysis. chichester, uk: wiley. borenstein mj, hedges lv, higgins jpt, rothstein hr (2005) comprehensive meta-analysis [computer software]. englewood, nj; biostat. bose r., luo x (2011) integrative framework for assessing firms’ potential to undertake green it initiatives via virtualization a theoretical perspective. journal of strategic information systems 20:38–54. vom brocke j, watson rt, dwyer c, elliot s, melville n (2013) green information systems: directives for the is discipline. communications of the association for information systems 33:30 http://aisle.aisnet.org/cais/vol33/iss1/30. brooks s, wang x, sarker s (2010) unpacking green it: a review of the existing literature. amcis 2010 proceedings paper 398. http://aisel.aisnet.org/amcis2010/398. butler t, hackney r (2012) breaking the iron law: implementing cost effective, green ict in the uk public sector. in proceedings of european conference on information systems. cai s, de souza r, goh m, li w, lu q, sundarakani b (2008). the adoption of green supply chain strategy: an institutional perspective. in proceedings of ieee international conference on management of innovation and technology. chen aj, watson rt, bondreau mc (2010) an institutional perspective on the adoption of green is. australasian journal of information systems 17(1):23–45. chen hg, chang j (2014) exploring affecting factors on green it adoption. in knowledge management in organizations. springer, international publishing, pp.205-218. chong ayl, ooi kb, lin bs, raman m (2009) factors affecting the adoption level of ccommerce: an empirical study. journal of computer information systems 50(2): 13-22. cohen j, cohen p (1983) applied multiple regression/correlation analysis for the behavioral sciences, erlbaum, hillsdale, nj. cooper rb, zmud rw (1990) information technology implementation research: a technological diffusion approach. management science 36(2):123-139. cooper va, molla a (2014) absorptive capacity and contextual factors that influence green it assimilation." australasian journal of information systems 18(3): 271-288. cui l, zhang c, zhang c, huang l (2008) exploring it adoption process in shanghai firms: an empirical study. journal of global information management 16(2):1-17. dao v, langella i, carbo, j (2011) from green to sustainability: information technology and an integrated sustainability framework. journal of strategic information systems 20(1):63-79. australasian journal of information systems chong & olesen 2017, vol 21, research article toe perspective on eco-effectiveness 19 damanpour f (2010) an integration of research findings of effects of firm size and market competition on product and process innovations. british journal of management 21:996–1010 deng q, wang y, ji s (2015) why organisations adopt green it: a comprehensive review. conf-irm 2015 proceedings. paper 40. http://aisel.aisnet.org/confirm2015/40. dimaggio pj, powell ww (1983) the iron cage revisited: institutional isomorphism and collective rationality in organizational fields. american sociological review 48(2):147– 160. dedrick j (2010) green is: concepts and issues for information systems research. communications of the association for information systems, 27:11, http://aisel.aisnet.org/cais/vol37/iss1/11. eisenhardt k, martin ja (2000) dynamic capabilities: what are they? strategic management journal 21(10/15): 1105-1121. ekong uo, ifinedo p, ayo ck (2012) e-commerce adoption in nigerian businesses: an analysis using the technology-organization-environmental framework. in: a. usoro, g. majewski and p. ifinedo et al. leveraging developing economies with the use of information technology: trends and tools. hershey: igi global, pp 156–178. elkington j (1997) cannibals with forks: the triple bottom line of 21st-century business. capstone, oxford. eze uc (2008) e-business deployment in nigerian financial firms: an empirical analysis of key factors. international journal of electronic business research 4(2):29–47. gartner research (2007) gartner symposium/itxpo 2007: emerging trends. gartner inc. accessed 26 april 2015 ghobakhloo m, arias-aranda d, benitez-amado j (2011) adoption of e-commerce applications in smes. industrial management and data systems 111(8):1238-1269. ghobakhloo m, tang sh (2011) barriers to electronic commerce adoption among small businesses in iran. journal of electronic commerce in organizations, 9(4):48-89. gholami r, sulaiman ab, ramayah t, molla a (2013) senior managers’ perception on green information systems (is) adoption and environmental performance: results from a field survey. information and management 50(7):431-438. gonzález p, sarkis j, diaz ba (2008) environmental management system certification and its influence on corporate practices: evidence from the automotive industry. international journal of operations and production management 28(11):1021-1041. gray r, bebbington k (2000) environmental accounting, managerialism and sustainability: is the planet safe in the hands of business and accounting? advances in environmental accounting and management 1(1):1-44. gu vc, cao q, duan w (2012) unified modelling language (uml) it adoption – a holistic model of organizational capabilities perspective decision support system 54(1):257– 269. hameed ma, counsell s, swift s (2012) a meta-analysis of relationships between organizational characteristics and it innovation adoption in organizations. information and management 49(5):218–232. henderson d, sheetz sd, trinkle bs (2012) the determinants of inter-organizational and internal in-house adoption of xbrl: a structural equation model. international journal of accounting information systems 13(2): 109-140. australasian journal of information systems chong & olesen 2017, vol 21, research article toe perspective on eco-effectiveness 20 ho yh, lin cy (2012) an empirical study on taiwanese logistics companies' attitudes toward environmental management practices. advances in management and applied economics 2(4): 223–241 hong wy, zhu k (2006) migrating to internet-based e-commerce: factors affecting ecommerce adoption and migration at the firm level. information and management 43(2):204-221. hunter j.e, schmidt fl (1990) methods of meta-analysis: correcting error and bias in research findings. newbury park, ca: sage. el idrissi sc, corbett j (2016) green is research: a modernity perspective. communications of the association for information system s 38(30) http://aisel.aisnet.org/cais/vol38/iss1/30. jackson jd, yi my, park js (2013) an empirical test of three mediation models for the relationship between personal innovativeness and user acceptance of technology. information and management 50(4):154-161. jing, s, ali, s, she, k, zhong, y (2013) state-of-the-art research study for green cloud computing. journal of supercomputing 65(1):445-468. joseph d, ng k, koh c, ang s (2007) turnover of it professionals: a narrative review, metaanalytic structural equation modelling, and model development. mis quarterly 31(3):1– 31. kraemer kl, gibbs j, dedrick j (2002) environment and policy facilitators shaping ecommerce diffusion: a cross-country comparison. in proceedings of the twenty-third international conference on information system. kolk a (2010). trajectories of sustainability reporting by mncs. journal of world business 45(4):367–374. kuan kky, chau pyk (2001) a perception-based model for edi adoption in small businesses using a technology-organization-environment framework. information and management 38(8):507-521. le vh, rowe f, truex d, huynh mq. (2012) an empirical study of determinants of ecommerce adoption in smes in vietnam: an economy in transition. journal of global information management 20(3):23-54. lee, h, jeong, y, jang, h (2014) 'performance analysis based resource allocation for green cloud computing. journal of supercomputing 69(3):1013-1026. lei cf, ngai ewt (2012) green is assimilation: a theoretical framework and research agenda. in proceedings of american conference on information systems. li dh, lai fj, wang j (2010) e-business assimilation in china’s international trade firms: the technology-organization environment framework. journal of global information technology 18(1):39-65. li yh (2008) an empirical investigation on the determinants of e-procurement adoption in chinese manufacturing enterprises. in proceedings of international conference on management science and engineering. lin hf (2014) understanding the determinants of electronic supply chain management system adoption: using the technology-organization-environment framework. technological forecasting and social change 86(1):80-92. lin hf, lin sm (2008) determinants of e-business diffusion: a test of the technology diffusion perspective. technovation 28(3):135-45. australasian journal of information systems chong & olesen 2017, vol 21, research article toe perspective on eco-effectiveness 21 loeser f (2011) green it and green is: definition of constructs and overview of current practices. proceedings of nineteenth americas conference on information systems, chicago, illinois, august 15-17. low c, chen y, wu m (2011) understanding the determinants of cloud computing adoption industrial management and data systems 111(7):1006-1023. melville np (2010). information systems innovation for environmental sustainability. mis quarterly 34(1):1-21. mithas s, kuntia j, roy pk (2010) green information technology, energy efficiency, and profits: evidence from an emerging economy. in proceedings of international conference on information systems. molla a, abareshi a (2012). exploring the extent of green it adoption and its driving and inhibiting factors. journal of information science and technology 6(4):3 –21 molla a, abareshi a (2011) green it adoption: a motivational perspective. in proceedings of pacific asia conference on information systems. molla a, cooper v, pittayachawan s (2011) the green it readiness (g-readiness) of organizations: an exploratory analysis or a construct and instrument. communications of the association for information systems 29:4. http://aisel.aisnet.org/cais/vol29/iss1/4. murugesan s (2008). harnessing green it: principles and practices. it professional 10(1): 2433. nedbal d, stieninger m (2014) exploring the economic value of a cloud computing solution and its contribution to green it. international journal of business process integration and management 7(1):62-72. nonaka i, takeuchi h (1995) the knowledge-creating company. london: oxford university press. nord w, tucker s. (1987) implementing routine and radical innovations. new york: lexington books. oliveira t, thomas m, espadanal m (2014) assessing the determinants of cloud computing adoption: an analysis of the manufacturing and services sectors. information and management 51(5):497–510. oliveira t, martins mf (2010) understanding e-business adoption across industries in european countries. industrial management and data systems 110(9):1337-1354. olson eg (2008) creating an enterprise-level "green" strategy, the journal of business strategy 29(2):22-30. pan mj, jang wy (2008) determinants of the adoption of enterprise resource planning within the technology-organization-environment framework: taiwan’s communications industry journal of computer information systems 48(3):94-102. paydar s, endut ir (2013) key drivers of rfid adoption in malaysian retail industry, a theoretical model. in proceeding of the 2013 ieee international conference on rfid technologies and applications. picoto w, bélanger f, palma-dos-reis a (2014) a technology–organization–environment (toe): based m-business value instrument. international journal of mobile communications 12(1):78–101. premkumar g, ramamurthy k (1995) the role of inter-organizational and organizational factors on the decision mode for adoption of inter-organizational systems. decision sciences 26(3):303-336. australasian journal of information systems chong & olesen 2017, vol 21, research article toe perspective on eco-effectiveness 22 ramdani b, chevers d, williams da (2013) smes' adoption of enterprise applications: a technology-organisation-environment model. journal of small business and enterprise development 20(4):735-753. rogers e (1995) diffusion of innovations. the free press, new york, ny. rusink ca (2007) green manufacturing: an evaluation of environmentally sustainable manufacturing practices and their impact on competitive outcomes. ieee transactions on engineering management 54:445-454. sahu gp, srivastava r (2011) a literature review and classification of green computing research, amcis 2011 proceeding – all submissions. paper 399. http://aiel.aisnet.org/amcis2011_submissions/399. seyal ah, rahman mn, mohamad ha (2007). a quantitative analysis of factors contributing electronic data interchange adoption among bruneian smes. business process management journal 13(5): 728−746. sila i (2013) factors affecting the adoption of b2b e-commerce technologies. electronic commerce research 13(2):199–236. sila i (2010) do organizational and environmental factors moderate the effects of internetbased inter-organisational systems on firm performance? european journal of information systems 19(5): 581–600. seethamraju r. (2015) adoption of software as a service (saas) enterprise resource planning (erp) systems in small and medium sized enterprises. information systems frontiers 17(3). soares-aguiar a, palma-dos-reis a (2008) why do firms adopt e-procurement systems? using logistic regression to empirically test a conceptual model. ieee transactions on engineering management 55(1):120-133. sophonthummapharn, k. (2008) a comprehensive framework for the adoption of technorelationship innovations : empirical evidence from ecrm in manufacturing smes, umeå : umeå school of business, umeå university. tate wl, dooley kj., ellram lm (2011) transaction cost and institutional drivers of supplier adoption of environmental practices. journal of business logistics 32(1):6–16. teo tsh, lin s, lai k (2009) adopters and non-adopters of e-procurement in singapore: an empirical study. omega 37(5):972-987. thambusamy r, and salam, af (2010) corporate ecological responsiveness, environmental ambidexterity and it-enabled environmental sustainability strategy. in proceedings of international conference on information systems. thiesse f, staake t, schmitt p, fleisch e (2011) the rise of the “next-generation bar code”: an international rfid adoption study. supply chain management: an international journal 16(5):328–345. tornatzky lg, fleischer m (1990) the processes of technological innovation. lexington books, lexington ma. tsai mc, lai kh, hsu wc (2013) a study of the institutional forces influencing the adoption intention of rfid by suppliers. information and management 50(1):59–65. tushi bt, sedera d, recker j (2014) green it segment analysis: an academic literature review. twentieth americas conference on information systems, savannah, 1-15. wang x, brooks s, sarker s (2015a) understanding green is initiatives: a multi-theoretical framework. communications of the association for information systems 37:32 http://aisel.aisnet.org/cais/vol37/iss1/32. australasian journal of information systems chong & olesen 2017, vol 21, research article toe perspective on eco-effectiveness 23 wang x, brooks s, sarker s (2015b) a review of green is research and directions for future studies. communications of the association for information systems 37:21 http://aisel.aisnet.org/cais/vol37/iss1/21. wang ym, wang ys, yang yf (2010) understanding the determinants of rfid adoption in the manufacturing industry. technological forecasting and social change 77:803-15. wang wj, yuan yf, wang xf, archer n (2006) rfid implementation issues in china: shanghai port case study. journal of internet commerce 5(4):89–103. watson rt, boudreau mc, chen ajw, sepulveda hh (2011) green projects: an information drives analysis of four cases. journal of strategic information systems 20(1):55-62. watson rt, boudreau mc, chen aj (2010) information systems and environmentally sustainable development: energy informatics and new directions for the is community,” mis quarterly 34(1): 23-38. watson rt, boudreau mc, chen a, huber mh (2008) green is: building sustainable business practices. in information systems, r. t. watson (ed.), athens, ga: global text project. weng mh, lin cy (2011). determinants of green innovation adoption for small and mediumsize enterprises. african journal of business management 5(22):9154-9163. wikstrom pa (2010) sustainability and organizational activities three approaches. sustainable development 18(2):99–107. wu xr, subramaniam c (2011) understanding and predicting radio frequency identification (rfid) adoption in supply chains. journal of organizational computing and electronic commerce 21(4):348-367. young w, tilley f (2006) can business move beyond efficiency? the shift toward effectiveness and equity in the corporate sustainability debate. business strategy and the environment 15:402-415. zhu k, kraemer kl (2005) post-adoption variations in usage and value of e-business by organizations: cross-country evidence from the retail industry. information systems research 16(1):61-84. australasian journal of information systems chong & olesen 2017, vol 21, research article toe perspective on eco-effectiveness 24 appendix 1 list of independent variables examined in technological adoption research independent variables definition expected association data sources relative advantage the degree to which new technological adoption provides positive organizational performances and economic gains over existing technologies positive ekong et al. (2012); eze (2008); ghobakhloo et al. (2011); henderson et al. (2012); le et al. (2012); li (2008); li et al. (2010); wang et al. (2010) compatibility the degree to which new technological adoption is compatible with the existing it infrastructure positive ekong et al. (2012); ghobakhloo et al. (2011); ghobakhloo and tang (2011); henderson et al. (2012); le et al. (2012); li (2008); thiesse et al. (2011); wang et al. (2010) complexity the degree to which new technological innovation is perceived as difficult to use negative ekong et al. (2012); gu et al. (2010); henderson et al. (2012); le et al. ( 2012); li (2008); sila, (2013); thiesse et al (2011); wang et al. (2010); wu and subramaniam (2011) technological readiness refers to an organization’s capability in adopting new technologies. it can be assessed by an organization’s available technological infrastructure and it knowledge and skill to implement a specific new technology positive alawneh et al. (2010); eze (2008); li et al. (2010); oliveira et al. (2014); pan and jang (2008); picoto et al. (2014) it infrastructure refers to the number of it hardware and software and the networked status positive cui et al. (2008); pan and jang (2008) soares-aguiar and palma-dos-reis (2008) perceived direct benefits direct benefits include reduction in transaction errors and transaction costs, improved data accuracy and information quality, and faster application process positive kuan and chau (2001); teo et al. (2009) perceived indirect benefits indirect benefits include better customer services and improved relationship with business partners positive kuan and chau (2001); teo et al. (2009) perceived risks refers to the cost of implementing the specific new technology negative ghobakhloo and tang (2011); le et al. (2012) australasian journal of information systems chong & olesen 2017, vol 21, research article toe perspective on eco-effectiveness 25 independent variables definition expected association data sources attitudes towards innovation the degree to which top management’s attitude towards a specific new technology adoption positive ghobakhloo and tang (2011); le et al. (2012) financial resources refers to financial resources to invest in the technological adoption positive alawneh et al. (2010); kuan and chau (2001); li (2008) organizational size refers to an organization’s annual revenue, total number of employees, number of it staff, employees, positive alawneh et al. (2010); ghobakhloo et al. ( 2011); le et al. ( 2012); lin (2014); oliveira et al. (2014); oliveira et al. (2010); pan and jang (2008); picoto et al. (2014); soares-aguiar and palma-dos-reis (2008); teo et al. (2009); thiesse et al. (2011); wang et al. (2010) information sharing culture extent of information is being shared within the organization for the adoption process positive ghobakhloo et al. ( 2011); teo et al., (2011) learning culture the degree to which an organization’s emphasis on learning about specific new technology adoption positive henderson et al. (2012); li et al. (2010) top management support extent of top management’s engagement, commitment and allocation of resources to support the specific new technology adoption positive ekong et al. (2012); eze (2008); li (2008); lin (2014); oliveira et al. (2014); picoto et al. (2014); seyal et al., 2007; sila (2013); teo et al. (2009); thiesse et al (2011); wang et al. (2010); wu and subramaniam (2011) knowledge refer to the requisite expertise to adopt the specific new technology positive henderson et al. (2012); le et al. ( 2012) competitive pressure the degree to which an organization is affected by competition in the market alawneh et al. (2010); ekong et al. (2012); eze (2008); ghobakhloo et al. (2011); gu et al. (2010); hong and zhu (2006); oliveira et al. (2014); oliveira et al. (2010); le et al. ( 2012); li (2008); pan and jang (2008); picoto et al. (2014); sila (2013); soaresaguiar and palma-dos-reis (2008); wang et al. (2010); wu and subramaniam (2011) australasian journal of information systems chong & olesen 2017, vol 21, research article toe perspective on eco-effectiveness 26 independent variables definition expected association data sources environmental uncertainty the degree to which an organization is affected by the behaviours of its suppliers, customers and competitors negative eze (2008); li et al. (2010) regulatory support extent of support from the government positive alawneh et al. (2010); cui et al. (2008); eze 2008); ghobakhloo and tang (2011); kuan and chau (2001); le et al. (2014); oliveira et al. (2014); pan and jang (2008); seyal et al. (2007) trading partner readiness refer to trading partners’ readiness in adopting the specific new technology positive ghobakhloo and tang (2011); soares-aguiar and palma-dos-reis (2008); wu and subramaniam (2011) copyright: © 2017 chong & olesen. this is an open-access article distributed under the terms of the creative commons attributionnoncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ http://creativecommons.org/licenses/by-nc/3.0/au/ microsoft word ajis_18(3)_v10.docx australasian journal of information systems volume 18 number 3 2014 371 small and medium enterprises using software as a service: exploring the different roles of intermediaries nagarajan venkatachalam information systems school queensland university of technology brisbane, australia venkat.venkatachalam@qut.edu.au erwin fielt information systems school queensland university of technology brisbane, australia e.fielt@qut.edu.au michael rosemann information systems school queensland university of technology brisbane, australia m.rosemann@qut.edu.au shane mathews business school queensland university of technology brisbane, australia sw.mathews@qut.edu.au abstract software as a service (saas) can provide significant benefits to small and medium enterprises (smes) due to advantages like ease of access, 7*24 availability, and utility pricing. however, underlying the saas delivery model is often the assumption that smes will directly interact with the saas vendor and use a self-service approach. in practice, we see the rise of saas intermediaries who can support smes with sourcing and leveraging saas. this paper reports on the roles of intermediaries and how they support smes with using saas. we conducted an empirical study of two saas intermediaries and analysed their business models, in particular their value propositions. we identified orientation (technology or customer) and alignment (operational or strategic) as themes for understanding their roles. the contributions of this paper include: (1) the identification and description of saas intermediaries for smes based on an empirical study and (2) understanding the different roles of saas intermediaries, in particular a more basic role based on technology orientation and operational alignment and a more value adding role based on customer orientation and strategic alignment. we propose that saas intermediaries can address saas adoption and implementation challenges of smes by playing a basic role and can also aim to support smes in creating business value with saas based solutions by playing an added value role. keywords: saas, software as a service, smes, saas intermediaries, saas vendors, saas customers, saas adoption, saas implementation, saas use. australasian journal of information systems volume 18 number 3 2014 372 introduction software as a service (saas) is a software licensing and delivery model where the application or service is deployed from a centralised data centre across a network, providing access and use often on a recurring fee basis (hoch et al. 2001). saas is projected to be a disruptive force at the heart of the operating principles of it industry such as large capital software and infrastructure investments, centralized procurement and governance, and high service/software deployment cost ratios (gartner 2012). saas makes professional and innovative software applications easily accessible at attractive prices. some of the often cited benefits from saas are the reduction of the total cost of ownership, speed of deployment, reliability and risk mitigation through insulation from the continuous technology upgrades (waters 2005). small and medium enterprises (smes) are projected to be main beneficiaries of saas due to its utility pricing model with no or limited upfront capital investments and the ease of access with web browser access, 7 * 24 availability, 30 day free trials, direct subscription and automated upgrade cycles. though there are significant benefits to be realized from saas, smes face a set of challenges while sourcing and leveraging saas. for example, a study of saas sourcing related to german smes reported that there are challenges when integrating saas with other applications, there is loss of control over access to data and data migration complexities for smes (haselmann et al. 2011). further, an adoption study of saas explained that the saas model shifts all customization investments to the consumer side and the consumers are responsible for maintaining all customized components (xin et al. 2008). the implications are that smes may need development and support skills, if they need to customize saas to fit into their environments. these findings are in line with past studies related to the use of is by smes which highlight that resources and skills constraints prevent the full leveraging of benefits from new technologies (scupola 2009; palvia 2008; poon et al. 1997). though saas is described as an important technological and business model innovation that brings in new opportunities for small and medium enterprises (hong et al. 2009), smes may encounter significant challenges in terms of leveraging the opportunities from saas. because of the self-service approach and the easy access to saas applications, there seems to be an implicit assumption that smes can directly use these applications. however, as discussed above, we foresee that smes are still faced with significant challenges when sourcing and leveraging saas. traditionally, smes have used consultants and vendor support to overcome challenges associated with new technologies (yap et al. 1992). saas applications are often provided by saas vendors that operate globally and are located abroad and that may only provide limited support to smaller customers (e.g. salesforce). therefore, we expect that smes will to some extent still rely on trusted providers that act as intermediaries between the sme as a local customer and the global saas vendor. this imputes our research question “what is the role of intermediaries for smes using saas?” based on our review of existing is literature on smes and saas, we observe that there is no study yet that examines the role of intermediaries for smes using saas. this research aims to bridge this gap with an explorative case study of two australian saas intermediaries. the remainder of this paper is organized as follows. we begin with a literature review of saas, smes and saas intermediaries. then we explain the research design, which uses a multiple case study methodology. next the cases are described and analysed based on a business model framework. then the findings are presented through a discussion of the different roles of saas intermediaries. we conclude the paper with contributions, limitations and topics for future research. literature review in this section, we briefly address the specific nature of smes and discuss saas in more detail, in particular the benefits and challenges for smes using saas. we end with addressing the literature australasian journal of information systems volume 18 number 3 2014 373 related to saas intermediaries, including intermediaries in general and intermediaries in cloud ecosystems. small and medium enterprises in australia, small business is defined as an actively trading business with 0-19 employees; mediumsized businesses comprise 20-199 employees and large businesses as actively trading with 200 or more employees (abs. 2012) smes contributed to 57% of private industry value added in addition to employing 4.8 million people during 2010-11 in australia (abs 2012). the characteristics of sme firms are classified in two major categories, namely, external environment oriented (customer and market) and internal environment oriented (management, human and financial resources) (cocca et al. 2010). traditionally, the uncertainty factors from the external environment are high for the smes as they normally are price takers and only have a limited customer and product base in which they operate (storey 2000). in terms of the internal environment, empirical analysis of the role and significance of ceo, firm strategy and structure revealed that the relative growth and profitability of small firms correlated strongly with an innovative product-market position and a more aggressive and analytic mode of decision making guided by an explicitly codified strategy (miller et al. 1986). however, compared with the large enterprises, smes have often implied and undocumented business strategies that are mostly focused on cost efficiencies (ballantine et al. 1998; hagmann et al. 1993). though the importance of smes and their economic contributions are well established, traditionally resource and capability limitations have restricted these firms from capitalizing the opportunities from is/it innovations (mehrtens et al. 2001; ramdani et al. 2009). cost, lack of time, skills and knowledge are identified as uppermost barriers for smes’ adoption of new technologies (heenetigala et al. 2009). also, sme information technology expertise is limited in relation to the operationalization of implicit strategies (ballantine et al. 1998; levy et al. 2000). an e-commerce adoption study identified financial, technical and managerial resource constraints as the main barriers for smes (lin et al. 2007). lack of awareness of the potential of ict into their business operations, the perception of unresolved security and privacy issues associated with the use of internet, high-set up costs and limited skills were also reported as barriers for smes’ adoption of e-business technologies (taylor et al. 2004). software as a service the services offered through cloud-based technologies are classified under three categories namely, infrastructure as a service (iaas), i.e. storage, computing and bandwidth capacity; platform as a service (paas), i.e. custom development environment to utilise cloud services; and software as a service (saas), i.e. suite of applications subscribed from cloud (mcafee 2011). the saas model can be described as a delivery and subscription model where “the application, or service, is deployed from a centralised data centre across a network, providing access and use on a recurring fee basis” (hoch et al. 2001). saas may be conceptualized as (a) an open market place for software services, (b) the dynamic provision of software for changing user demands, (c) services supply network where service vendor may sub contract to provide their services, and (d) delivery transparency to its users whose main interest is its use (gold et al. 2004). the saas model solves the provider side customisation deficiencies of the application service provider (asp) model with the multi-tenant architecture and shifts the customization effort to the customer side of saas (xin et al. 2008). total cost of ownership reduction, speed of deployment, reliability, data security, data safety and disaster recovery, risk mitigation through insulation from the continuous technology upgrades are cited as some of the key benefits of the saas model (waters 2005). a mckinsey study on saas highlights that more frequent and less painful software upgrades, a lower cost of ownership and higher service levels are the key advantages of saas (dubey et al. 2007). research identified the virtualization, service ecosystems and elasticity of cloud infrastructure as the drivers for and significance of saas (barros et al. 2006; sushil et al. 2010). australasian journal of information systems volume 18 number 3 2014 374 saas is also discussed as an important technological and business model innovation that brings new opportunities for smes (hong et al. 2009) and can help them to overcome traditional resource and capability constraints. saas can potentially address technology related challenges of smes as the saas delivery model shifts the development, deployment and maintenance of applications to saas vendors (hong et al. 2009). the saas subscription model may overcome some of the financial barriers of smes in relation to it. however, there will still be challenges associated with the use of saas by smes. a recent german study on the saas adoption by smes concluded that saas is not expected to be the revolution in business for smes in the near future but declared that saas is an important concept in the is/it architecture of these enterprises (haselmann et al. 2011). gartner (2012) observed that only a few saas applications have reached maturity and most saas applications still remain far away from mainstream adoption. saas intermediaries traditionally, intermediaries are conceived as part of the marketing channel between producers and consumers, with wholesalers, retailers and specialized intermediaries – such as finance firms, logistics firms and advertising agencies – as their main forms (stern et al. 1996). intermediaries bring customers and suppliers together and facilitate demand and supply activities. intermediaries solve customers’ problems and, as a result, suppliers’ problems. their position on the high ground between both groups enables them to create value and charge for it (anderson et al. 2002). intermediaries provide valueadded services such as aggregation and distribution of products and product information, quality checks and warranties (chircu et al. 2000). intermediaries need to make choices with respect to their positioning in relation to the other actors in the business network. these role choices relate to aspects like functional scope (i.e., targeted business functions) and activity focus (i.e., supported business activities) (fielt et al. 2008). intermediaries can perform different roles, for example, they can perform basic market functions, provide management support for sourcing or serve as technology adapters (dai et al. 2002). the servitization of the software industry via saas and other cloud services results in services-oriented systems. intermediaries can be seen as part of a value chain service system and as separate service systems themselves (alt et al. 2010). bardhan et al. (2010) outlined how services-oriented systems will act as a disruptive technological innovation and emphasized the different roles of service producers (e.g., salesforce, google), consumers (e.g., sme), service intermediaries (resellers) and service monitors (governments and standard bodies). intermediaries have been discussed in the wider context of the cloud ecosystem. recent literature identifies a number of generic roles as part of the cloud service ecosystem: application provider, (technical) platform provider, market platform, infrastructure provider, consultant, aggregator, integrator and consumer (böhm et al. 2010). leimeister et al. (2010) identified integrators, consultants and brokers as value adding participants within a cloud ecosystem. overeem and vreeken (2014) argue that while cloud providers focus on standardization, cloud brokers focus on customization for each customer and differentiate between configurator, assembler, trust builder and integrator roles for cloud brokers. however, to the best of our knowledge, there has been hardly any attention for saas intermediaries so far, in particular in relation to smes. the only other study addressing saas intermediaries focused on their internal, supply-side capabilities and concluded that business facing skills, customer development and process re-engineering capabilities will become more important (overeem et al. 2014). while related, our study is more focussed on the external role of saas intermediaries in relation to their support for sme customers. research design saas is a relatively a new phenomenon with little empirical research on smes using saas and the role of intermediaries. notable exceptions are a few studies which address saas adoption factors, benefits and challenges (haselmann et al. 2011; xin et al. 2008). moreover, to the best of our knowledge, this australasian journal of information systems volume 18 number 3 2014 375 is the first empirical study of intermediaries for smes using saas. therefore, we opted for an explorative research approach. we used a multiple case study design to investigate the role of saas intermediaries for smes using saas. the case study methodology is proposed as one of the best options for conducting explorative studies when there is scant prior research on the topic (yin 1994). also multiple case studies are preferred over the single case study due to the vulnerability of single case study and associated criticism with generalisation of the results as well as for extension of the existing theories (benbasat et al. 1987; yin 1994). we selected two saas intermediaries, alpha and beta, based on (a) their focus on sme customers, (b) their support for market leading saas products, (c) their credentials in the market place, and (d) the availability of key stakeholders for the interviews and subsequent clarifications. table 1 presents an overview of the two saas intermediary firms in this study. semi-structured, face to face interviews were conducted with representatives from both firms. two interviews were conducted with the owner of alpha. two interviews were conducted with the managing director and one interview with the customer manager of beta. the duration of the interviews was between 60 to 90 minutes. the interview transcriptions were sent to participants to review for verification of their comments. contact summary forms (miles et al. 1984) were completed to record the first observations as well as capture main themes from these interviews. the same form was also used by the researcher to verify that all required data was collected. in addition to this, the information from both companies’ websites was used to gather additional data on partnerships and details of the services provided by these intermediaries. when required, specific clarifications were requested via emails and phone calls (for example, saas release documentations). the researcher also attended three local salesforce user group meetings conducted by the owner of intermediary alpha. this allowed the researcher to observe the interactions between saas intermediary alpha and the sme saas customers in the brisbane market. the nvivo qualitative data management tool was used for the coding and analysis of collected data through the open coding as well as descriptive, interpretive coding and analytical processes. firm type of firm no. of sme customers interviewees no. of interviews data sources alpha salesforce intermediary 100+ owner (director) 2 interviews, website, salesforce user group beta cloud intermediary 70+ managing director, customer manager 3 interviews, website, saas roadmap documents table 1. overview of data collection at saas intermediaries. based on a first summary of the data from both cases, we concluded that the best way to holistically understand the role of the saas intermediary would be by means of describing and analysing their business model. a business model describes and analyses how organizations create (and capture) customer value (chesbrough et al. 2002). more specifically, we adopted the business model canvas (osterwalder et al. 2010) as this has become one of the most widely applied frameworks by both academics and practitioners and because it covers the core dimensions of a business model (fielt 2013). the business model canvas consists of nine elements: (1) an organization serves one or several customer segments, (2) it seeks to solve customer problems and satisfy customer needs with value propositions, (3) value propositions are delivered to customers through communication, distribution, and sales channels, (4) customer relationships are established and maintained with each customer segment, (5) revenue streams result from value propositions successfully offered to customer segments, (6) key resources are the assets required to offer and deliver the previously described australasian journal of information systems volume 18 number 3 2014 376 elements, (7) performing of a number of key activities, (8) some activities are outsourced and some resources are acquired outside the enterprise via key partnerships, and (9) the business model elements result in the cost structure (osterwalder et al. 2010). case descriptions in this section, we describe both saas intermediaries in detail by means of their business model. we highlight some specific details in the text and provide a summarized overview of the business model elements in table 2. saas intermediary alpha saas intermediary alpha is based in brisbane and provides a selected set of salesforce related services, in particular consulting, integration and support services, to sme customers that are part of the salesforce ‘ecosystem.’ their salesforce based value propositions are based on increasing customer awareness of saas based opportunities and using best practices for implementation. the intermediary’s customers are smes customers are often firms that are moving from micro to small size. the firm operates project-based with short duration contracts for customer-specific requirements. the owner explained that “as an organisation, we are a small team, and we believe that large customers need a large team that can concentrate 100% on that customer. i would be winning in one hand but then at the detriment of the business as a whole. so, definitely looking for smaller, profitable shorter engagements rather than less profitable longer.” the firm had been in business for 3.5 years and serviced around 100+ sme customers so far. the firm’s potential customers originate through referrals and call with specific requests like establishing a contact database. project duration varies from 10 days for more straightforward configuration and getting salesforce up and running, to longer engagements involving customization and development. the owner goes through requirement-gathering discussions with the prospects. as part of this process they increase the customer’s awareness regarding the opportunities with salesforce crm and associated service offerings. the owner of saas intermediary alpha is also the facilitator of brisbane salesforce user group meetings where he and his team share the knowledge about best practices in using salesforce. these knowledge sharing sessions cover a range of topics including salesforce upgrade idiosyncrasies, question and answer sessions and demonstrations of complementary services that are part of salesforce eco system. these forums and knowledge sharing sessions can also identify prospective customers for this intermediary. the key resources of the firm are their authentic salesforce expertise held in the matured age workforce and the provision of the local access to global knowledge for their sme customers. as stated by the owner: “our unique selling point is our experience, our qualifications; we are certified. we definitely promote ourselves as being local to brisbane.” the firm employs three certified salesforce consultants with access to global resources via development centres abroad. this intermediary has established partnerships with other saas service providers of the salesforce ecosystem, for example, nimbus and radianscore. these partnerships enable access to a wide range of salesforce related knowledge and solutions that are relevant to their customers’ specific situations and are complementary to their own service offering. saas intermediary beta saas intermediary beta offers a ‘full service cloud’ product which essentially gives an integrated package of all three cloud based services namely iaas, paas and saas. this full service offering is aimed at smes with no or very limited is capabilities. for firms with in-house is capabilities, this intermediary offers specific advisory services towards their saas selection and implementation activities. their saas based value propositions are (a) providing advisory services on saas product australasian journal of information systems volume 18 number 3 2014 377 selection, (b) customization of standard saas applications to align with the customers’ specific environment situations, and (c) provide best fit solutions by combining a specific set of service options to match their customers’ process specific implementation. the managing director classified the potential customers into two categories by stating “i think, there is really two camps, the ones that go direct to internet and download and play with it, trial with it and they end up with silos of saas. they will have silo finance saas, silo procurement saas and erp, then they need to this professional to come in to make all talk together. the other ones ask us give us a proposal for everything.” in 2013, this intermediary had a total of 79 customers out of which 67 belong to the sme category. in terms of customer engagement, customer focussed consultants conduct a one day or two day workshops with all key stakeholders of their customer firm to get a complete understanding of the customers’ current position in the market place, their strategic goals and financial capabilities. the managing director stressed that they always start with the workshop: “everything stems from the workshop. we always do the workshops with stakeholders before we implement anything.” the intermediary provides a ‘saas roadmap’ for their customers along with implementation options to choose after a structured analysis of the information collected in the workshops. the saas roadmap gives the options for the sme customers in terms of the choice of saas products, customized components with saas platform and/or off-the-shelf packages based on the customer’s specific needs. the key resources of the firm are the saas programming and implementation skills, locally managed hardware infrastructure, existing customer relationships originating from infrastructure service offerings and the past experiences of managing director. the firm has established service partnerships with global vendors for hardware (amazon web services), network (cisco) and applications (salesforce, google and microsoft). it is owned and managed by an entrepreneurial managing director who has a track record of creating multiple it services firms. the contractual relationships tend to be subscription based and long term oriented with 2 or 3 years duration. australasian journal of information systems volume 18 number 3 2014 378 business model element saas intermediary alpha saas intermediary beta customer segments micro, small and medium size firms that are already using salesforce focus target market (salesforce customers only) small, medium and large enterprises as its customers. customers from all three cloud based services market customer relationships mostly short term relationship with limited project scope community services through coordination and facilitation of the local user group forums long term relationship focussed on repeat customers cross selling of services to existing customers. community engagements through vendor conferences customer channels providing most of the services in person at the customer site customers from direct as well as referrals from supply-side partnerships knowledge sharing and user group facilitations for identifying prospective customers providing some of the services in person at the customer site (e.g., workshops) direct and partnership based channels value propositions focus on salesforce related service offering increase customer awareness of saas based opportunities knowledge sharing for best practices with implementation information sharing of customer specific add-on services cloud based service offering (as separate services and as integrated offering) provide advisory role to support the customers saas purchasing decisions troubleshooting of technical issues from customer’s own implementations data sovereignty with local storage activities development, customization, integration, data migration and back-up related activities for salesforce and add-on services salesforce administration trainings to end users development, customization, and integration related activities for all three cloud services conduct business analysis workshops special data related activities for data cleansing, integration and localization provide help desk services for it support key resources the skills and automation experience of the owner consultants with certified knowledge in salesforce access to global software development resources limited infrastructure the skills and entrepreneurial experiences of the managing director consultants with technology expertise for all three cloud services access to local and global software development resources physical and it infrastructure customer specific knowledge from the existing service subscriptions partners partnerships are with salesforce eco system solution providers only use cloud based infrastructure services from external providers the partnerships are broader and with all three cloud service vendors revenue streams project based revenues (depend on project duration and services offered to individual customers) two or three year services subscription model provides revenue predictability costs key cost contributors are from human resources, office space and global development resources human resource costs, physical office space, local and global development resources costs are also from hardware infrastructure and redundant network connectivity related costs physical space related costs table 2. the business models of the saas intermediaries alpha and beta. australasian journal of information systems volume 18 number 3 2014 379 case analysis the two case studies provide insights into how saas intermediaries address saas opportunities and challenges for smes related to (1) increasing the awareness of saas based opportunities, (2), getting saas applications up and running, (3) integrating saas into new and existing business activities, (4) addressing the need for customization of standard saas offerings, and (5) leveraging new and innovative features available through automated upgrade cycles. to provide a deeper understanding of the role of the saas intermediaries, we analysed the business models of alpha and beta from an insideout perspective (orientation) based on reasoning from activities, resources, partners and costs towards value proposition and customer elements and from an outside-in perspective (alignment) based on reasoning from customer elements, revenues and value proposition towards activities, resources, partners and cost. for orientation, the case analysis resulted in differentiating between technology and customer orientation. technology orientation is related to the generic features of the vendor supplied saas offerings and getting the saas applications up and running in a straightforward way. customer orientation is related to how intermediaries offer and deliver engagements that address the individual, customer-specific requirements for using saas. for alignment, the case analysis resulted in differentiating between operational and strategic alignment. operational alignment relates to how intermediaries purposefully fit the vendor-offered saas solutions with the processes of the smes. strategic alignment relates to how the intermediary’s scope of solutions and contractual arrangements go beyond the operational needs of their sme customers. please note that we differentiate between technology and customer orientation and between operational and strategic alignment as perspectives for analytical purposes. in practice, saas intermediaries will be a mix of these perspectives, as also is the case for alpha and beta. however, on the whole, beta demonstrates more customer orientation and strategic alignment than alpha. this may be explained by the fact that beta has more physical, technological and human resources and has global partnerships with service providers for all three cloud services (i.e., iaas, paas, and saas). technology and customer orientation of saas both saas intermediaries demonstrate technology and customer orientation within their sme engagement process (table 3). technological orientation is related to the generic features of the saas offerings and getting the saas applications up and running in a straightforward way. customer orientation is related to how intermediaries offer and deliver engagements that address the individual customer specific requirements using saas. the technology orientation of the two saas intermediaries varies in terms of solutions offered to their customers. intermediary alpha offers only saas solutions that are based upon salesforce and employs certified salesforce consultants. the owner of alpha stated that “we’re only interested in people that have already recognised that salesforce is the platform of choice for them”. intermediary beta targets customer segments which are in need of all three types cloud-based services, namely iaas, paas and saas. beta offers technology solutions that are based upon multiple saas (salesforce, google) and other cloud services (iaas and paas) vendors (amazon & cisco). beta has technology integration expertise to integrate the different types of saas and saas-related infrastructure needs. in addition to the technology orientation, these intermediaries also demonstrate a customer orientation during the requirements gathering and solution proposal processes with the individual sme customers. during the first step of engaging with their customers and prospects, both intermediaries conduct specific activities to identify the scope of the required saas related services for a specific customer or prospect. australasian journal of information systems volume 18 number 3 2014 380 in the case of the intermediary alpha, the owner educates the customer and increases the customer awareness of potential solutions from the salesforce eco system that can enhance the customer’s value with saas use. the owner described an example related to a call from a prospect enquiring about setting up a contact database for their micro business. the owner identified the requirement as that micro sme could not track or keep any information related to the received emails during the conversation. he described it with, “an email comes in to an email inbox or an outlook inbox. well, it’s relying on somebody responding to it. how many did we get? we don’t know. how many were about, were genuine complaints as opposed to just an enquiry? don’t know because there’s no analytics on that.” he subsequently elaborated about his proposed solution for this prospective customer during the site visit “but then, whilst there, i was talking about customer services. so well, how do you deal with issues and complaints and email enquiries, and things like that, and that’s why we think that customer services in the service cloud kicks in nicely.” in this example, the owner is proposing the ‘service cloud’ product of salesforce as a solution to address an underlying problem related to managing customer relationships that goes beyond keeping track of their contacts and customers. in case of intermediary beta, the requirements gathering and solution proposal process is much longer and more extensive. such an extended process results in very specific outcomes that can enable the sme to plan for ongoing leveraging of saas. this firm conducts one or two day workshops with their customers before starting any engagement. subsequently, this intermediary offers a best fit solution to address specific needs of their customers. the managing director explained that his firm aims to offer the optimal solution for their customers irrespective of any particular vendor: “we are vendor agonistic in terms of helping them decide, this is where we do the stakeholder workshop, because we have done this with couple of clients full workshops and we have realized that hang on a sec, we have been thinking that you need a customized application using force.com but you actually don’t. there is an app in microsoft which you can plug in and use it.” moreover, beta can address specific customer needs by offering a combination solution that uses both saas and client-server products. australasian journal of information systems volume 18 number 3 2014 381 business model element technology orientation customer orientation customer segments size of target customer market (number of saas products) customer relationships short term and limited to the project scope long term and relationship based customer channels referrals via technology partners existing customers asking for additional services value propositions awareness of saas and cloud service solution options technology skills for initial implementation data migration, conversions, clean up and back up services for remotely hosted saas data small to large scope of customization local access to expertise vendor agnostic, integrated, best fit solutions additional data services for specific customer needs (data localization and improved quality) activities technology related activities (development, testing, minor extensions, configuration) understanding sme business direction customer problem resolutions best fit solution identification for customer problems resources automation knowledge salesforce certified skills consultant’s knowledge of custom fit solutions locally accessible skills and knowledge infrastructure for data partnerships global technology vendor partnerships vendor agnostic customer focussed solutions revenues short term long term costs global technology development resources customer interactions table 3. orientation of saas intermediaries. operational and strategic alignment of saas both saas intermediaries demonstrate both operational and strategic alignment during the execution of their solutions for their customers (table 4). through operational alignment we explain how intermediaries implement the vendor-offered saas solutions as part of the sme’s processes. with strategic alignment, we explain how the intermediary’s scope of solutions and contractual arrangements go beyond the operational needs of their sme customers. both intermediaries support their sme customers with the full operational implementation of saas as part of their particular processes. as intermediary alpha engages with their customers after their choice for salesforce, they support the sme to implement salesforce for their sales and marketing processes. alpha provides the smes with the knowledge so they can ‘self-sustain’ their saas oriented operations after initial set-up, including administration activities. intermediary beta minimizes the operational reporting issues with saas by data cleansing and security issues with saas by data sovereignty solutions. the managing director of intermediary beta observed that smes “do not cleanse it australasian journal of information systems volume 18 number 3 2014 382 thoroughly and reports do not run correctly and then we go and investigate it to find that the field is numeric field and they have alpha characters in it. it is a very common issue; we sort of mandate that (i.e. data conversions)”. what tends to happen is that people try to do it themselves, and ends up very very messy. we say to them, you have to let us do that properly. because rubbish in rubbish out”. these observations explain how intermediary beta provides ongoing operational support to smes, when they start using saas. for strategic alignment, the saas intermediaries go beyond the operational needs of their sme customers. due to their connections within the salesforce ecosystem, intermediary alpha provides strategic value for their customers and prospects via the salesforce user group. in these forums, the owner of alpha shares knowledge about the important features of new releases from salesforce (for example, the use of chatter for collaboration or mobile features that are anticipated in the new release). he also organizes presentations by salesforce platform vendors that are his supply-side partners. these partners explain the specific features offered via the salesforce application exchange. the intermediary beta completes an extended analysis of customer’s requirements through workshops. the managing director described the activities and outcomes of the workshops as follows: “we basically drill down the entire business with the questions are like how do you actually market your product? how do you sell your product? how do you deliver the product? or if it is a service, we use product and service interchangeably. how do you make sure that there is no churn? what is your client loyalty program? we have a set template that covers we ask all these questions. there are four fundamental platforms for any business, it is all about identifying the strategy, you have the funding to execute on that, where the funding is going to come from, you have the people to implement and the most important thing is what your execution plan is? we cover all four pillars of a business. we are more like a business coach”. such investigations enable this intermediary to deliver a saas roadmap that is aligned with that customer’s strategic “aspirations”. these workshops also enable intermediary beta to provide best fit solution options aligned with their customers’ strategic objectives. australasian journal of information systems volume 18 number 3 2014 383 business model element operational alignment strategic alignment customer segments focus on current need (with selfsustainment after set-up) long term with a strategic focus customer relationships customer channels onsite for set-up services onsite for set-up and enhancement services remote location for ongoing support value propositions services for initial set-up only best practices for efficient implementation self-sustainment of saas after initial set-up error free operational reporting through data cleansing services also ongoing support and enhancement services saas roadmap aligned with customer business objectives trusted advisory role for long term solutions activities investigations for aligning saas for a defined scope of requirements develop custom extensions and integration of components for a specific need conduct workshops for understanding business objectives of customers ongoing maintenance of customized extensions resources partnerships revenues project fees recurring revenue model costs physical and human resources for set-up physical and human resources for set-up, enhancement, and support table 4. alignment of saas intermediaries. case discussion based on the findings from our case studies in terms of orientation and alignment, we propose that saas intermediaries can address the saas adoption and implementation challenges of smes by playing a basic role based on technology orientation and operational alignment. saas intermediaries can also aim to support smes in creating business value with saas-based solutions by playing an added value role based on customer orientation and strategic alignment. figure 1 displays both roles of saas intermediaries. below we discuss each role for saas intermediaries in more detail and relate our findings to the is and saas literature on smes and intermediaries. australasian journal of information systems volume 18 number 3 2014 384 figure 1. saas intermediaries: basic and added value roles. basic role: technology orientation and operational alignment the basic role of saas intermediaries is characterized by technology orientation and operational alignment. in this basic role, the intermediary supports the adoption and implementation of saas by smes. this means getting the saas application up and running in a straightforward way and making it part of the sme’s processes. previous research of saas discusses the technology uncertainties of the saas market, data migration complexities and saas-specific skills required for basic configurations as some of technology challenges specific to getting a saas application operational in the sme environment (haselmann et al. 2011; xin et al. 2008). saas intermediaries assist smes with technology and operational challenges originating from saas adoption and implementation. they conduct the requirements analysis, solution design and implementation activities for their saas customers. this includes the configuration of saas and may sometimes require intermediaries to educate the sme’s admin users to use the controls offered by saas vendors to make the application ready for all users in the sme. these activities enable a fully configured saas application ready for all users of saas within the particular sme’s operating environment. a basic role for an agreed scope of saas technology and operational activities comes mostly with short-term, project fees based contracts in the case of intermediary alpha, data integrity, locality, data access and data segregation are identified as issues associated with the use compared in-house based data of traditional applications (subashini et al. 2011). both firms also offer specific data-related services through data migration, cleansing, integration, with other applications and back-up activities that aim to address the data-related challenges associated with saas use. the data related services have the potential to provide smes with cleaned up data, keeping the data in local infrastructure and error-free reporting benefits. added value role: customer orientation and strategic alignment the added value role of saas intermediaries is characterized by customer orientation and strategic alignment. in this added value role, the focus shifts to the individual, customer-specific requirements and the bigger, more holistic picture of saas and its impact on business value. due to the depth of knowledge and experience of intermediary alpha with salesforce, they address the sme customers’ future needs for leveraging saas-based solutions. with such practices, alpha provided an added value role to overcome the knowledge constraints of smes with is (heenetigala et al. 2009). the upfront workshops by intermediary beta enable them to provide their customers and prospects with best fit solutions with specific options based upon the holistic understanding of their customers’ strategic aspirations. by delivering a saas roadmap at the end of the workshop, beta explicitly documents the sme’s business strategy and alignment of technology options with that explicit strategy. thus, the intermediaries enable smes to overcome some of the traditional is adoption challenges related to align information technology with the sme’s implicit strategies (levy et al. 2000). australasian journal of information systems volume 18 number 3 2014 385 intermediary alpha has supply-side partnerships with other complementary service providers of salesforce eco system, which enables them to increase the customer awareness of these saas extension options and offer an integrated value proposition from the network of salesforce vendors. in case of intermediary beta, supply-side partnerships are established with all three cloud service providers. by such offerings, intermediaries enables the sme to overcome challenge of lack of awareness of how ict can enhance their operations (taylor and murphy 2004). this results in the integrated value propositions through the full cloud service offerings that includes saas, paas and iaas. intermediary beta delivers a saas road map with applications from a choice of vendors (salesforce or microsoft) as well as alternative implementation processes (customization or standard plug-ins) due to their extended engagement during the evaluation stage of their customers’ purchasing process. while the saas model can overcome some of the problems and challenges of smes with it, for example, the need for capital expenses related to hardware and software licenses (waters 2005), it leaves other it problems and challenges unaddressed, for example, the perception of unresolved security and privacy issues associated with the use of the internet (taylor et al. 2004), and also introduces new saas related problems and challenges, for example, skills and knowledge required for customization needs (xin et al. 2008) and data management in saas applications and the integration of saas with other applications (haselmann et al. 2011). however, the saas model often focusses on the role of the saas vendor and does not address the role of the saas intermediary, in particular for smes. the basic role of saas intermediaries for smes can provide smes with easier access to software applications by addressing technology-related problems, for example, data cleaning data migration, application integration (subashini et al. 2011), and operational challenges, for example, configure and implement saas within their operations (haselmann et al. 2011). the added value role of saas intermediaries can support smes with creating business value by customer orientation (e.g. understand deep understanding of business needs of the customers, providing vendor agnostic solutions) and strategic alignment (e.g. a saas roadmap aligned with business strategy, awareness of new solutions). overall, we argue that it is the combination of the saas model and saas intermediaries for smes that can provide smes with easier access to software applications and support smes in creating business value with it. our identification of a basic role and value added role of saas intermediaries for smes is in line with the findings on internal, supply-side capabilities of overeem and vreeken (2014). overeem and vreeken conclude that business facing skills, customer development and process re-engineering capabilities will become more important and this is in line with our understanding of the value added role of saas intermediaries for smes. looking more generally at cloud intermediaries, we see similarities between our differentiation between technology and customer orientation and the integrator and aggregator roles on the one hand and the consultant role on the other hand of böhm et al. (2010). our understanding of strategic alignment adds to the more operational perspective on cloud roles as provided by the value network of böhm et al. (2010). more broadly, our research shows that even in the electronic world of saas and cloud, which is often predicted to be disintermediated (malone et al. 1987), there is still a need for intermediaries that solve customers’ problems and, as a result, suppliers’ problems (anderson et al. 2002). conclusion in this paper, we explored the role of saas intermediaries that mediate between sme customers and saas vendors. while saas offers potential advantages to smes, it can still result in adoption and implementation challenges due to the self-service model of saas vendors and the complexity of software applications. therefore, we conducted an empirical study that took a closer look at two saas intermediaries to determine the roles they can play to address the sme problems and challenges with saas. australasian journal of information systems volume 18 number 3 2014 386 we identified different approaches for saas intermediaries with respect to their orientation (technology or customer) and alignment (operational or strategic). technology orientation is related to the generic features of the vendor-supplied saas offerings and getting the saas applications up and running in a straightforward way. customer orientation is related to how intermediaries offer and deliver engagements that address the individual, customer-specific requirements for using saas. operational alignment relates to how intermediaries purposefully fit the vendor offered saas solutions with the processes of the smes. strategic alignment relates to how the intermediary’s scope of solutions and contractual arrangements go beyond the operational needs of their sme customers. we propose that saas intermediaries can address the saas adoption and implementation challenges of smes by playing a basic role based on technology orientation and operational alignment. saas intermediaries can also aim to support smes in creating business value with saas based solutions by playing an added value role based on customer orientation and strategic alignment. as far as we know, this is the first paper that describes saas intermediaries in more detail and also identifies the different roles that saas intermediaries can play. the contributions of this paper are: (1) the identification and description of saas intermediaries for smes based on an empirical study and (2) understanding the different roles of saas intermediaries, in particular a more basic role based on technology orientation and a more value adding role based on customer orientation and strategic alignment. as this is an explorative study, there are a number of limitations that need to be taken into account. the empirical study has a limited number of intermediary cases and is only based on australian intermediaries. moreover, while the cases focussed on saas, one of the cases also included cloud offerings. this makes the cases slightly different and less comparable. we propose some directions for future research. future research can gather data from the saas vendor’s perspective and saas sme consumers’ perspectives based upon their experiences with saas intermediaries to expand on the interactive nature of intermediary roles. further, future research can also apply this saas-related intermediary knowledge to a cloud service intermediary context. references abs 2012. "australian small business key statistics and anlaysis," c. o. australia (ed.), department of industry, innovation, science, research and tertiary education. alt, r., abramowicz, w., and demirkan, h. 2010. "service-orientation in electronic markets," electronic markets (20:3-4), pp 177-180. anderson, p., and anderson, e. 2002. "the new e-commerce intermediaries," mit sloan management review (43:4), pp 53-62. ballantine, j., levy, m., and powell, p. 1998. "evaluating information systems in small and mediumsized enterprises: issues and evidence," european journal of information systems (7:4), pp 241251. bardhan, i., demirkan, h., kannan, p., kauffman, r., and sougstad, r. 2010. "an interdisciplinary perspective on it services management and service science," journal of management information systems (26:4), pp 13-64. barros, a., and dumas, m. 2006. "the rise of web service ecosystems," it professional (8:5), pp 3137. benbasat, i., goldstein, d. k., and mead, m. 1987. "the case research strategy in studies of information systems," mis quarterly (11:3), pp 369-386. böhm, m., koleva, g., leimeister, s., riedl, c., and krcmar, h. 2010. "towards a generic value network for cloud computing," in economics of grids, clouds, systems, and services, j. altmann and o. rana (eds.), springer berlin heidelberg, pp. 129-140. australasian journal of information systems volume 18 number 3 2014 387 chesbrough, h., and rosenbloom, r. s. 2002. "the role of the business model in capturing value from innovation: evidence from xerox corporation's technology spin off companies," industrial and corporate change (11:3), pp 529-555. chircu, a. m., and kauffman, r. j. 2000. "reintermediation strategies in business-to-business electronic commerce," international journal of electronic commerce (4(4)), pp 7-42. clark, l., desisto, r., holincheck, j., white, a., kyte, a., sarner, a., lheureux, b., gassman, b., klappich, c., and kolsky, e. 2012. "hype cycle for software as a service," gartner research, id:g00141122). cocca, p., and alberti, m. 2010. "a framework to assess performance measurement systems in smes," international journal of productivity and performance management (59:2), pp 186-200. dai, q., and kauffman, r. j. 2002. "business models for internet-based b2b electronic markets," international journal of electronic commerce (6), pp 41-72. dubey, a., and wagle, d. 2007. "delivering software as a service," the mckinsey quarterly (6), pp 112. fielt, e. 2013. "conceptualising business models: definitions, frameworks and classifications," journal of business models (1:1). fielt, e., janssen, w., faber, e., and wagenaar, r. 2008. "design trade offs for electronic intermediaries," electronic markets (18:4), pp 362-374. gartner 2012. "agenda for software as a service, 2012," r. p. desisto (ed.), gartner, p. 7. gold, n., mohan, a., knight, c., and munro, m. 2004. "understanding service-oriented software," ieee software (21:2), pp 71-77. hagmann, c., and mccahon, c. s. 1993. "strategic information systems and competitiveness: are firms ready for an ist-driven competitive challenge?," information & management (25:4), pp 183-192. haselmann, t., and vossen, g. 2011. "software-as-a-service in small and medium enterprises: an empirical attitude assessment," in web information system engineering – wise 2011, a. bouguettaya, m. hauswirth and l. liu (eds.), springer berlin / heidelberg, pp. 43-56. heenetigala, k., and armstrong, a. 2009. "the use of internet reporting for small business 23," journal of business systems, governance and ethics (4:4). hoch, f., kerr, m., and griffith, a. 2001. "software as a service: strategic backgrounder," software & information industry association (siia)). hong, c., ke, z., ming jun, z., wei, g., jun jie, c., and xin sheng, m. year. "an end-to-end methodology and toolkit for fine granularity saas-ization," cloud computing, 2009. cloud '09. ieee international conference on2009, pp. 101-108. leimeister, s., böhm, m., riedl, c., and krcmar, h. 2010. "the business perspective of cloud computing: actors, roles and value networks," in ecis 2010 proceedings, paper 56. levy, m., and powell, p. 2000. "information systems strategy for small and medium sized enterprises: an organisational perspective," the journal of strategic information systems (9:1), pp 63-84. lin, c., huang, y. a., and tseng, s. w. 2007. "a study of planning and implementation stages in electronic commerce adoption and evaluation: the case of australian smes," contemporary management research (3:1), pp 83-100. malone, t. w., yates, j., and benjamin, r. i. 1987. "electronic markets and electronic hierarchies," communications of the acm (30:6), pp 484-497. australasian journal of information systems volume 18 number 3 2014 388 mcafee, a. 2011. "what every ceo needs to know about the cloud-cloud computing represents a genuine sea change in how organizations use it. the time has come for companies to overcome their fears and embrace it," harvard business review, pp 124-132. mehrtens, j., cragg, p. b., and mills, a. m. 2001. "a model of internet adoption by smes," information & management (39:3), pp 165-176. miles, m. b., and huberman, a. m. 1984. qualitative data analysis: a sourcebook of new methods, sage publicaions: beverly hills, calif. miller, d., and toulouse, j.-m. 1986. "strategy, structure, ceo personality and performance in small firms," american journal of small business (10:3) winter86, pp 47-62. osterwalder, a., pigneur, y., and clark, t. 2010. business model generation: a handbook for visionaries, game changers, and challengers, john wiley & sons: hoboken, nj. overeem, g., and vreeken, a. 2014. "implications of saas on competencies of itbrokerages," in twenty second european confererence on information systems: tel aviv palvia, s. c. j. 2008. "challenges for small enterprises in the sourcing life cycle: evidence from offshoring to india," journal of information technology case and application research (10:4), p 75. poon, s., and swatman, p. m. c. 1997. "small business use of the internet," international marketing review (14:5), pp 385-402. ramdani, b., kawalek, p., and lorenzo, o. 2009. "predicting smes' adoption of enterprise systems," journal of enterprise information management (22:1/2), pp 10-24. scuploa, s. 2009. "smes' e-commerce adoption: perspectives from denmark and australia," journal of enterprise information management (22:1/2), pp 152-166. stern, l. w., el-ansary, a. i., and coughlan, a. t. 1996. marketing channels, (prentice hall upper saddle river, nj. storey, d. j. 2000. understanding the small business sector, thomson learning: london. subashini, s., and kavitha, v. 2011. "a survey on security issues in service delivery models of cloud computing," journal of network and computer applications (34:1), pp 1-11. sushil, b., leena, j., and sandeep, j. 2010. "an approach for investigating perspective of cloud software-as-a-service (saas)," international journal of computer applications (10:2), pp 4447. taylor, m., and murphy, a. 2004. "smes and e-business," journal of small business and enterprise development (11:3), pp 280-289. waters, b. 2005. "software as a service: a look at the customer benefits," journal of digital asset management (1:1), pp 32-39. xin, m., and levina, n. year. "software-as-a service model: elaborating client-side adoption factors," proceedings of the 29th international conference on information systems, paris, france, 2008. yap, c., soh, c., and raman, k. 1992. "information systems success factors in small business," omega (20:5–6), pp 597-609. yin, r. k. 1994. case study research: design and methods, (second ed.) sage: thousand oaks, ca. acknowledgements this research was carried out as part of the activities of, and funded by, the smart services cooperative research centre (crc) through the australian government’s crc programme (department of innovation, industry, science and research). australasian journal of information systems volume 18 number 3 2014 389 an earlier version of this paper was presented at the australasian conference on information systems (acis) 2013 in melbourne, australia. paper title (paper title style) australasian journal of information systems weerasinghe, pauleen, scahill & taskin 2018, vol 22, research article a theoretical framework of big data in healthcare 1 development of a theoretical framework to investigate alignment of big data in healthcare through a social representation lens kasuni weerasinghe massey university new zealand w.m.k.g.weerasinghe@massey.ac.nz david pauleen massey university new zealand shane scahill massey university new zealand nazim taskin massey university new zealand abstract the aim of this paper is to develop a theoretical framework grounded in the literature, which can be used to explore the influence of big data on business-it alignment in the healthcare context. increasingly the availability of information systems in healthcare delivery and service management results in massive amounts of complex data that have the 3v characteristics of big data (i.e. volume, variety, velocity). use of big-healthcare-data has been identified as bringing significant benefits to the healthcare sector from improved decision making through to population health analysis. although the technical dynamics around big data such as analytics and infrastructure requirements are extensively researched, less attention has been given to social dynamics such as peoples’ experience, understanding and perceived usefulness of this data. to address this gap, the paper uses social representation theory as a methodological lens to develop a theoretical framework to study the social dynamics around big data and its use in the healthcare context. the selected case for this development is the new zealand healthcare sector and an approach using multi-level macro, meso, and micro analysis is taken. use of social representation theory as a methodological lens to develop a theoretical framework is a novel approach. such a theoretical framework will be useful as a foundation for carrying out on-going empirical research on big data to understand its influence on business-it alignment in the healthcare context. keywords: big data, healthcare, business-it alignment, social representation theory, new zealand healthcare, healthcare information systems. 1 introduction the growing use of information systems (is) in the healthcare sector, alongside increasing patient populations, diseases and medication, is generating enormous amounts of unstructured and complex data that have the characteristics of ‘big data’ (ward et al. 2014, australasian journal of information systems weerasinghe, pauleen, scahill & taskin 2018, vol 22, research article a theoretical framework of big data in healthcare 2 wyber et al. 2015). big data is commonly known for its ‘3v’ characteristics: volume (large in size), variety (many different types of data), and velocity (availability of data in near real-time) (mcafee and brynjolfsson 2012). the analysis of big data, also known as ‘big data analytics’ is central to a revolutionary change in the business world (davenport 2013). until recent times data driven approaches in healthcare to make use of large volumes of complex data were considered difficult, if not impossible, because available technology was not mature enough to handle such data (wyber et al. 2015). however, recent technological developments have opened promising avenues for healthcare to make use of big-healthcare-data for more effective healthcare management and delivery (mace 2014). as opposed to companies born in the digital era, traditional businesses face a greater challenge in integrating big data into their existing information technology (it) ecosystems (davenport and dyché 2013). implementing big data capabilities in traditional businesses like healthcare requires the management of change in the socio-technical aspects of an organisation such as: analytics platforms, it architecture, it infrastructure, security measures, required expertise and organisational structure. within the healthcare sector, change is identified as a key factor that influences business-it alignment (bush et al. 2009). the importance of aligning uses of big data with clear business objectives has been acknowledged (bean and kiron 2013, watson 2014). however, no alignment studies could be found in the literature which investigate the influence of big data analytics on business-it alignment in either the business or wider healthcare literatures. furthermore, as a recent it phenomenon, big data research shows a bias towards understanding technical dynamics and as such social dynamics around big data use have been largely ignored, and are not adequately researched (shin 2015). business-it alignment can be examined through four dimensions: (i) strategic, (ii) structural (formal and informal), (iii) social, and (iv) cultural (chan and reich 2007). as the least studied and most suitable to investigate, social dynamics – the social dimension of alignment – is posited in this paper to be the ideal platform to support the investigation of the social dynamics associated with the big data construct. healthcare systems differ from country to country; the selected case context for the development of the framework discussed in this paper is the new zealand healthcare system. when studying complex systems such as healthcare, which are composed of different components, structural divisions, organisations, and actors it is useful to categorise the system into macro, meso and micro (mmm) levels (dopfer et al. 2004). for this study, we use the mmm conceptualisation discussed for the new zealand healthcare system by scahill (2012): macro – policy setting organisations, meso – funders and planners, and micro – service providers. based on the findings from the literature on big data, business-it alignment and healthcare, this paper discusses development of a framework that can be used to investigate the influence of big data on business-it alignment in this context. social representation theory (srt) (moscovici 1963) is used as a methodological lens that guides the development of the theoretical framework (see figure 1). srt provides a holistic stance allowing us to understand how individuals co-construct representations within a social group when new situations emerge (andersén and andersén 2014). representations are influenced by the pressure, opinions, social negotiation and collective sense-making of a group (dulipovici and robey 2013). studying social representations brings methodological direction to a study and is therefore appropriate to be applied to is/it research (gal and berente 2008). australasian journal of information systems weerasinghe, pauleen, scahill & taskin 2018, vol 22, research article a theoretical framework of big data in healthcare 3 based on srt, mmm levels are identified as social groups, which could have different representations of big data and thus impact alignment in different ways. sub-groups such as organisations, departments or project teams within these levels are anticipated. figure 1: use of methodological lens for the development of a theoretical framework the term ‘methodological lens’ refers to the use of srt as the methodology that guides the development of a theoretical framework (see figure 1). the use of srt allows not only the study of social dynamics but it also informs the structure of the theoretical framework in terms of what is to be explored and how this needs to be studied. in this manner srt acts as a methodological lens as opposed to a theoretical lens. srt has been used as a theoretical lens in past research (e.g. dulipovici and robey 2013, gal and berente 2008) but the authors are not aware of literature outlining the use of srt as a methodological lens in developing a theoretical framework. this paper discusses the development of a theoretical framework to study the influence of big data on business-it alignment based on the new zealand healthcare context. the process of developing such a framework could be applied to healthcare systems in other countries. moreover, such frameworks can then be used as the basis for empirical studies which aim to develop theory. in the following sections, we discuss the literature around big data, business-it alignment and healthcare, highlighting the identified gaps, while srt is discussed separately. a description of the nz healthcare system is provided as the case under study. the aim of this paper is to posit a framework grounded in the appropriate literature, which will later be used to explore the influence of big data within the healthcare sector. 2 literature review in this section we review the relevant literature on big data, the healthcare sector, and business-it alignment to identify the gaps and build a foundation for the theoretical framework. 2.1 big data based on past literature we define big data as enormous amounts of structured, unstructured and complex data produced by a wide range of computer applications (groves et al. 2013, kacfah emani et al. 2015, shin 2015, wang and huang 2015). phrases such as “massive amounts of data”, “enormous growth of data” and “large data sets” are typically seen within the literature as defining big data (chen et al. 2014, eynon 2013, shin 2015). australasian journal of information systems weerasinghe, pauleen, scahill & taskin 2018, vol 22, research article a theoretical framework of big data in healthcare 4 three characteristics, known as the 3v’s – volume, variety and velocity – are generally used to define big data and distinguish it from standard data (mcafee and brynjolfsson 2012, russom 2011). according to gartner (2013, para1). big data is “high-volume, high-velocity and/or highvariety information assets that demand cost-effective, innovative forms of information processing that enable enhanced insight, decision making, and process automation”. watson (2014), mcafee and brynjolfsson (2012), and russom (2011) also characterise big data using these 3v’s. two additional v’s – value and veracity – have also been identified, extending the characteristics of big data to 5v’s (kacfah emani et al. 2015, saporito 2013, sathi 2012). based on the 5v characteristics kacfah emani et al. (2015, p72) state “dealing effectively with big data requires one to create value against the volume, variety and veracity of data while it is still in motion (velocity), not just after it is at rest”. “analytics” is an umbrella term for all data analysis applications (watson 2014, p1250) and refers to the use of tools to analyse data, not necessarily big data. in the business context, big data and analytics are often discussed together; they are sometimes even confused with each other with the terms being used interchangeably (mcafee and brynjolfsson 2012). simply collecting and storing big data creates no value unless analytics is performed to make sense of this data to improve decision making within a business (watson 2014). however traditional analytic capabilities are not sufficient to process big data. much more advanced infrastructure and analytical techniques are needed to glean insight from data that is high in volume, variety and velocity (kacfah emani et al. 2015). big data or big data analytics “is not a single out-of-the-box product” (loshin 2013, p21). making effective use of big data demands a specific combination of tools, techniques, and skills. companies that were born in the internet era, such as google, facebook and ebay were built around big data (davenport and dyché 2013), thus these companies possess the capabilities to manage and make use of it. with technological advancements and the commercialisation of the internet, companies that existed before the internet era (deemed traditional businesses) are also looking into opportunities to develop their businesses by effectively using big data (bholat 2015, chawla and davis 2013, davenport and dyché 2013, dhawan et al. 2014). to integrate big data, traditional businesses will need to consider making changes to their existing it ecosystem. they will not only be working with big data but also with standard small datasets. their hadoop1 clusters may run along with their ibm mainframes; big data analytics will be used to complement traditional analytics; their data scientists will be working together with quantitative analysts (davenport and dyché 2013). therefore, it is likely to be a challenge for traditional businesses to integrate the new (implementation of big data analytics) with the known (traditional data technologies in the it ecosystem) (bean and kiron 2013, davenport and dyché 2013). in a traditional business it is expected that the use of big data would be associated with a wide range of social and technical aspects. technical dynamics (technology requirements of big data, challenges and opportunities of big data analytics and so forth) towards big data implementations have been extensively researched (e.g. chen et al. 2014, davenport 2013, dhawan et al. 2014, jagadish et al. 2014). as a technology revolution itself, it is fair to say that big data research often shows a bias toward 1 hadoop is an open source software framework for distributed storage and distributed processing of large data sets australasian journal of information systems weerasinghe, pauleen, scahill & taskin 2018, vol 22, research article a theoretical framework of big data in healthcare 5 technical dynamics. due to this bias adequate research has not yet investigated the associated social dynamics surrounding big data and its use (shin 2015, shin and choi, 2015). in this paper, social dynamics refer to the users’ understanding, commitment, and perceived value of big data, within a given context. among the scarce research that has been found exploring these humanistic factors in relation to big data analytics implementations, shin (2015) extended the unified technology acceptance and usage theory (utaut) with characteristics such as perceived usefulness and perceived ease of use extracted from the technology acceptance model (tam). shin used this model to test the normalisation process of big data in korean society. in his research shin (2015) discovered that big data implementations should be user centric and driven by the requirements of end-users. however, more research is needed to investigate the influence of social dynamics when implementing big data analytics in the traditional business context, because utilisation and management of big data is equally complex. traditional businesses should not implement big data just to be “trendy”, but rather clear goals must drive the strategy and process (loshin 2013). a study undertaken by newvantage partners (2012) with fortune 500 companies and federal agency leaders identified that business-it alignment is crucial for the success of big data implementation. business-it alignment is achieved through business and technology (big data analytics) working together in harmony with proper understanding of business objectives and big data capabilities (bean and kiron 2013, loshin 2013). although the existing literature highlights the importance of business-it alignment, studies investigating the influence of big data on business-it alignment have not been found. 2.2 business-it alignment for the past 30 years, alignment has been a major concern for it practitioners and company executives (kappelman et al. 2013). similarly, for the past few decades many researchers have explored the importance of business-it alignment in various business domains (e.g. drazin and van de ven 1985, dulipovici and robey 2013, henderson and venkatraman 1992, luftman 1996, sabherwal and chan 2001). and therefore, alignment has remained one of the dominant fields of is research through the years (chan and reich 2007, sousa and machado 2014). cognates for “alignment” include terms such as fit, coherence, harmony, match, integration, congruence, relationship, gestalt, synergy and linkage. these all refer to the degree of fit between business strategy, organisational structure, it strategy and it infrastructure (chan and reich 2007, henderson and venkatraman 1993, luftman 1996). researchers who study alignment typically study the fit between two or more of these four domains (henderson and venkatraman 1992). grounded in the literature, our definition of alignment refers to how well technology is utilised to bring value to a business. in the context of big data, this definition can be extended to describe how well technology is realised to make sense out of big data to create value. creating value in the business context signifies achievement of business goals and objectives. over 30 years of research on alignment has led to conceptualising it in numerous ways. as such, henderson and venkatraman (1992) proposed the strategic alignment model (sam) to study this phenomenon. their paper explores alignment between the domains of business strategy, business structure, it strategy and it structure. they identify three types of australasian journal of information systems weerasinghe, pauleen, scahill & taskin 2018, vol 22, research article a theoretical framework of big data in healthcare 6 alignment: bivariate fit, cross-domain alignment and strategic alignment. bivariate fit in the sam model refers to investigation of alignment at any two domains of the model. crossdomain alignment explains alignment across three of these domains. the third and most complex type of alignment defined in the sam model is strategic alignment, which refers to giving “simultaneous or concurrent attention to all four domains” (henderson & venkatraman, 1992, p. 20). reich and benbasat (1996) conceptualise studying business-it alignment through two dimensions: social and intellectual. chan and reich (2007), grounded in past alignment literature, identify a number of different dimensions of business-it alignment including: strategic/intellectual, structural (formal/informal), social and cultural. strategic/ intellectual dimension of alignment looks at fit between business strategy and it strategy. structural dimension investigates the relationship between an organisation’s structure and its it. the social dimension is applied to understand how it is perceived by different players within the organisation or the unit under study. the cultural dimension is the degree of agreement between the it approach and the organisational culture. chan and reich (2007) also identified different levels of alignment as: organisational, operational, project, and individual. although different, these conceptualisations seem to share similar characteristics, i.e. the social dimension has strong ties with the individual level (chan and reich 2007). as discussed above, research on big data shows a lack of empirical studies on the social dynamics associated with big data implementations. the ideal dimension to study social dynamics to understand alignment is the social dimension of alignment as it explores how the it is perceived by different players at different levels of an organisation. according to reich and benbasat (1996, p58) the social dimension can be defined as “the level of mutual understanding of commitment to the business and it mission, objectives and plans”. unpredictable social aspects may influence the business-it alignment. the social dimension of alignment explores how users perceive and understand the it implementation and how the technology is actually used (dulipovici and robey 2013). compared to the strategic/intellectual and structural dimensions of alignment, fewer studies have been carried out to investigate the social dimension of alignment. tan and gallupe (2006) used a cognitive approach to examine the shared understanding between business and it executives. they studied both the commonalities2 and individualities3 among people that contributed to shared cognition in an organisation. dulipovici and robey (2013) applied social representation theory to explore how a knowledge management system (kms) is perceived and embraced by different groups of people within an organisation and discussed how the kms influenced business-it alignment of the organisation. to address the identified gaps in the literature, this paper incorporates the social dimension of alignment and examines how social dynamics may influence the alignment of big data within the traditional business setting. healthcare is selected as the context to carry out the research as it fits well with the notion of traditional business and it is also one of the fields in which big data has been identified as being beneficial (groves et al. 2013). this provides the basis to investigate alignment in terms of how implementation and technologies around big 2 commonalities: similarities in individuals’ cognition. 3 individualities: differences in individuals’ cognition. this reflects the diversity in values and beliefs of individuals in a team. australasian journal of information systems weerasinghe, pauleen, scahill & taskin 2018, vol 22, research article a theoretical framework of big data in healthcare 7 data are perceived to be creating value and how knowledgeable actors are of the business objectives of such implementations, across a country’s healthcare sector. 2.3 transformation of healthcare with information technology until recently the quality of healthcare delivery has been largely dependent on the intuition and intelligence of clinical practitioners. healthcare services have been successfully delivered because the clinical practitioners within these systems “are bright, hard-working and wellintentioned – not because of good system designs or systematic use of data” (celi et al. 2013, p1157). however, recent research shows that there is a growing interest in using data to aid clinical practitioners in healthcare delivery and service management (mace 2014, patil et al. 2014, tormay 2015). the global healthcare industry faces significant challenges as populations grow and age – with more chronic disease, resources are constrained and systems are under significant pressure to perform (gauld 2009). as a result, internationally health systems have undergone major changes in the past few decades to address the call for enhanced patient care and improved outcomes (anderson 2007, bush et al. 2009, pare et al. 2008, patil et al. 2014, sicotte et al. 2006). it and deployment of information systems (is) is identified as central to this transformation of healthcare systems (bush et al. 2009). these information systems focus on improving patient care, service quality, operational efficiency and patient satisfaction (peng et al. 2014). such targets are achieved by reducing medical errors, streamlining clinical processes, increasing productivity and controlling healthcare costs (anderson 2007, kannry 2011). the information systems used across health are key in creating healthcare data. a wide range of clinical and operational information systems have been introduced (ward et al. 2014) and used effectively within healthcare in many developed countries (e.g. usa, new zealand, and canada). based on their use, these information systems are classified into two types: (i) clinical is (cis), and (ii) administrative is (ais) (menon et al. 2009). in their classification menon et al. (2009) identify is assisting primary value chain activities of healthcare as clinical is; thus, these can be identified as information systems used in healthcare delivery. these systems capture, store and analyse clinical data to provide improved services in healthcare delivery (pare et al. 2008). electronic health records (ehr), laboratory information management systems, picture archiving and communication systems are some examples of clinical is (menon et al. 2009, ward et al. 2014). in addition to aiding professionals in clinical practice, cis also provide information for strategic planning (glandon et al. 2008). also identified as operational management systems by glandon et al. (2008), administrative is are used for healthcare administration and service management. these information systems are used to facilitate the secondary value chain activities (support activities) of healthcare (menon et al. 2009). thus, they support non-patient care activities of healthcare organisations (glandon et al. 2008). some examples of these information systems are human resource management systems, supply chain management systems and payroll systems, similar to those in any other organisation (glandon et al. 2008, menon et al. 2009, ward et al. 2014). in addition to these stand-alone discrete information systems, complex integrated systems that combine a variety of cis and ais can be found across healthcare sectors. some examples of these integrated information systems are hospital information systems (his) (ahmadian et al. 2014) and general practice information systems (gpis) (yusof et al. 2008). australasian journal of information systems weerasinghe, pauleen, scahill & taskin 2018, vol 22, research article a theoretical framework of big data in healthcare 8 2.3.1 big data in healthcare the increasing use of ehr and other it deployments in healthcare is contributing to the rapid growth of healthcare data (bates et al. 2014, patil et al. 2014). given population growth and the rising numbers of diseases and medications, large amounts of complex data are being generated in the healthcare sector (wyber et al. 2015). due to the complex nature of this sector, data generated by information systems typically have characteristics (i.e. the 3v’s) of big data. although data with the 3v characteristics of big data have been generated within the healthcare sector for some time, making use of this data in healthcare has been considered complex, if not impossible, because the available technology was not mature enough to handle such data (wyber et al. 2015). recent developments of technology around big data analytics are opening up promising avenues for the healthcare sector to make use of big-healthcare-data for improved healthcare delivery (mace 2014, nash 2014, tormay 2015, wyber et al. 2015). for example, hadoop clusters introduced as a result of the big data phenomenon can be used to store massive amounts of data in an economic fashion which was not previously possible. additionally, the development of data science skills has produced people who are capable of making sense of large and complex datasets generated in near real time. hence, with the recent improvements to technology, the healthcare sector is now capable of deriving accurate data (veracity) to create value through big data analytics for improved healthcare delivery (wyber et al. 2015). consequently, although big data is not new for healthcare, making use of big data and creating value (through big data analytics) for improved healthcare delivery and management is an innovation that healthcare sectors globally are grappling with. nonetheless, compared to other industries such as retail merchandising and banking, the uptake of big data in the healthcare sector has been slow and limited (bates et al. 2014, groves et al. 2013). on top of the complex nature of the healthcare system, resistance to change by healthcare practitioners, uncertainty of returns on capital investment, and privacy concerns are identified as possible reasons for this lag (groves et al. 2013). however, due to increasing it expenditure and the enormous amounts of under-utilised and complex data, the healthcare sector needs more efficient practices, research and tools to analyse big data and optimise its use (chawla and davis 2013, groves et al. 2013). recently, developed countries have recognised the importance of big data analytics for healthcare (prewitt 2014). an estimate by mckinsey & company reports that with the use of big data analytic tools and technologies for healthcare, the united states can save an extraordinary $300 billion to $450 billion per year (groves et al. 2013). according to experts, harnessing big data for knowledge could have significant implications for the healthcare sector. predicting disease outbreaks, detecting gaps in care delivery, discovering the most effective treatments, identifying patterns related to medication side effects and hospital readmissions, improving pharmaceutical research, and personalised medicine are some of the identified benefits of big data analytics for healthcare (groves et al. 2013, nash 2014, tormay 2015). because the healthcare system falls under the category of traditional business, implementing big data analytics could transform the existing it ecosystem of the healthcare sector. as discussed previously, research outside of the healthcare sector has recognised that big data initiatives to succeed they need to be aligned with business objectives. thus, although the healthcare sector is interested in using big data, for such initiatives to succeed, the basis for big australasian journal of information systems weerasinghe, pauleen, scahill & taskin 2018, vol 22, research article a theoretical framework of big data in healthcare 9 data implementation should be driven by clinical and/or administrative healthcare goals and objectives. 3 social representation theory as a methodological lens social representation theory (srt) is used as a methodological lens for guiding and structuring the proposed theoretical framework of this paper. by methodological lens, we mean a lens that methodologically guides development of a framework. dulipovici and robey (2013) applied srt to frame the investigation of how social representation of a new knowledge management system (kms) in a government agency influenced business-it alignment. our paper adopts their approach but extends it with the aim to develop a theoretical framework using srt to frame the study of social dynamics associated with implementing big data analytics in the healthcare sector, in order to investigate how it may influence business-it alignment. use of srt in this manner forces structure and direction upon the theoretical framework, hence acting as a methodological lens. adopting srt as a methodological lens allows the humanistic aspects of business-it alignment to be considered in depth. srt is a theory from social psychology developed by serge moscovici in 1961, which provides a holistic stance to understand meaning making within social groups. over the past 50 years, srt has been extensively used in many different fields including social sciences, media research, organisational change, healthcare, it implementation, and information security (e.g. andersén and andersén 2014, breakwell 1993, dulipovici and robey 2013, gal and berente 2008, nichols 1981, vaast 2007, wagner et al. 1999). representation of a phenomenon (concept, object or a situation) is the central idea of the srt. as moscovici (1963, p251) defines it, social representation is “the elaborating of a social object by the community for the purpose of behaving and communicating”. this definition was later refined to mean that objects or concepts are constituted within a social group upon thoughts, feelings and behaviours of the actors (wagner et al. 1999). therefore, a representation can be characterised using three elements: (i) the object which is represented; (ii) the individual who builds the understanding; and (iii) the group to which the individual belongs (dulipovici and robey 2013). gal and berente (2008, p134) outline srt “as a socio-cognitive framework used to study the social production of common-sense knowledge. it offers a set of concrete conceptual tools for addressing the social context from which shared meanings emerge, and for capturing the temporal nature of socio-cognitive activity”. fundamentally, srt denotes how individuals coconstruct representations based on common understanding of an object, idea or a concept within a social group when new situations emerge (andersén and andersén 2014). the representation is therefore influenced by the pressure, opinions, social negotiation and collective sense-making of the group (dulipovici and robey 2013). due to these continuous social interactions, representation of the object is constantly developing. as it provides methodological direction (dulipovici and robey 2013) srt is ideal for use as a methodological lens. reviewing studies in past is research, gal and berente (2008) illustrated how such studies could make a more significant contribution if studied through srt. studying social representations brings not only methodological direction but also conceptual richness; therefore, it is favourable to be used for is/it research (dulipovici and robey 2013, gal and berente 2008). additionally andersén and andersén (2014) point out that srt is much more useful when examining situations concerning organisational change. as explained, big data australasian journal of information systems weerasinghe, pauleen, scahill & taskin 2018, vol 22, research article a theoretical framework of big data in healthcare 10 has the potential to change many aspects of a traditional business. hence, srt is identified as an appropriate methodology to guide an alignment study of big data around social dynamics. according to moscovici (1984) the social representation process emerges through two component processes: (i) anchoring, and (ii) objectification (dulipovici and robey 2013). anchoring is the symbolic classification of a new phenomenon based on past experience, common background and aspirations (gal and berente 2008). anchoring will develop a common understanding of the phenomenon within the group. through anchoring the group classifies the unfamiliar and represents it in a known arrangement (wagner et al. 1999). objectification supports the classification (anchoring) by mapping it with examples, images, models, methods or metaphors. it is the individual interpretation of the novel concept by each individual member of the group (dulipovici and robey 2013, gal and berente 2008). therefore these two processes complement each other, as anchoring being a social process promotes stability, and objectification being a cognitive process prompts change (dulipovici and robey 2013). consequently, social representations are continuously shaped within the social group. figure 2 illustrates how we conceptualise this understanding of anchoring, objectification and the formation of a representation within a social group. figure 2: conceptual illustration of srt based on this, it is understood that the concept of big data and its use may be perceived within a socially constructed group through anchoring and objectification. objectification of the notion of big data and its potential by an individual will be influenced by the individual’s background, knowledge, past experience and understanding. objectification will then influence anchoring of big data analytics within the group and result in constructing a representation. the boundaries of a social group were broadly explained in the early definitions of srt (moscovici, 1963). for example, society, the community or the public was seen as the social setting that influences a social representation (andersén and andersén 2014). but the most recent uses of srt divide the population into much smaller entities such as organisations (gal and berente 2008), and departments within organisations (dulipovici and robey 2013). a group may consist of two or up to an infinite number of members (wagner et al. 1999). moscovici (1988) defines three types of representations based on formation of social groups: (i) hegemonic, (ii) emancipated, and (iii) polemic (see table 1). thus as a theoretical underpinning to an empirical study of social groups, the representation to be studied can guide the selection of the group. australasian journal of information systems weerasinghe, pauleen, scahill & taskin 2018, vol 22, research article a theoretical framework of big data in healthcare 11 hegemonic representation a straightforward representation shared usually by a highly structured group (e.g. city). this type of representation is already established and is not produced by the group. emancipated representation a representation formed within a group. it could be influenced by several representations of sub-groups within the group; collectively the representations of sub-groups influence the representation of the phenomenon within the group. polemic representation mutually exclusive representations within a group due to conflict and social controversy. for example marxism is a polemic representation that requires studying groups with contrasting perceptions of it. table 1: types of representations based on formation of social groups (moscovici 1988, p221) the phenomenon of big data analytics is not a known representation in the healthcare context; therefore, it is not a hegemonic representation. nor is it known to have a conflicting identity with a polemic representation. thus, based on moscovici’s definition of types of representations, we suggest that implementing big data analytics falls under the classification of an emancipated representation. it is anticipated that big data analytics could be socially constructed within a group, which may have sub-groups that contribute to the representation. 4 the case: new zealand healthcare sector the healthcare systems of countries across the world operate in different ways. we have selected new zealand (nz) as a case to develop the theoretical framework. it is our expectation that the process of analysis of the influence of big data on alignment developed here will be applicable to other countries with similar health systems. it is estimated that nz spends about 10.3% of its gross domestic product (gdp) on healthcare, with 31% of this spent on acute in-patient care, 34% on out-patient care, 15% on long term care, 11% on medical goods and 10% on collective services4 (oecd 2013). these services are provided to new zealanders through a multifaceted system governed by the ministry of health (moh). the health and disability system of nz is funded nationally, planned regionally and delivered locally (pollock 2012). the moh “provides whole-of-sector leadership” to the nz healthcare system (moh 2014, p1). high level health policy development is undertaken by the office of the minister of health with input from cabinet and the government, to set strategic direction for the healthcare sector. although the moh has a greater influence in healthcare policy development, the national health board, health workforce new zealand, the national health committee, and other ministerial advisory committees also support and advise the minister (moh 2011, 2014). organisations under the moh are divided into 2 categories: (i) organisations that support healthcare delivery, and (ii) business units. the key organisations for healthcare delivery include the district health boards (dhbs), primary health organisations (phos), crown entities and agencies, national ambulance sector office, non-governmental organisations (ngos), public health units, and professional and regulatory bodies. apart from the organisations supporting healthcare delivery, several business units also support the moh 4 collective services include health education, training of health professionals, administration services and food, hygiene and water control australasian journal of information systems weerasinghe, pauleen, scahill & taskin 2018, vol 22, research article a theoretical framework of big data in healthcare 12 focusing on a variety of functions and areas. these business units are composed of: (i) client insights and analytics, (ii) strategy and policy, (iii) service commissioning, (iv) protection, regulation and assurance, (v) technology and digital services, (vi) finance and performance, (vii) people and transformation, (viii) office of the director-general, (ix) maori leadership, (x) chief nursing officer, (xi) chief medical officer and chief pharmacy advisor as well as (xii) critical projects (moh 2016). due to the inter-relationships between many different organisations, actors, and structural divisions of the nz healthcare system this can be identified as a complex health system. biological, socio-natural or socio-technical systems with more than three coupled components are likely to demonstrate chaotic behaviour under certain circumstances and are then identified as complex systems (liljenström and svedin 2005). when studying such complex systems, it is best to take an approach through the macro-meso-micro perspective of the system to compartmentalise, reduce complexity and obtain a holistic understanding (dopfer et al. 2004). the macro-meso-micro (mmm) model can be used to conceptualise in a variety of ways dependent on the purpose of the study. additionally, mmm levels will provide an ideal basis upon which groups can be segmented in order to study social representations. within the nz healthcare sector several authors propose mmm models with slightly different but related conceptualisations. cumming (2011) conceptualises macro as a single organisation, or a body that oversees organisation to organisation collaboration, meso as activities that promote work between organisations – e.g. clinical partnerships (mays 2013), and micro as individual practitioners. from a slightly different view scahill (2012) conceptualises policy setting organisations as macro, funders and planners as meso, and service provider organisations and the individuals within them as micro. following scahill (2012), and based on preliminary interviews, we define mmm around the use of big data as: • macro – government bodies who set the strategy direction and policies that govern it implementation, particularly implementations utilising big data fall under the macro level. therefore, the business units as well as the moh are conceptualised as the macro level bodies. in the nz context apart from providing strategy direction the moh and its business units are currently interested in initiating implementations around big data. • meso – the planners and funders meso level symbolises organisations that follow the guidelines of the macro bodies and plan to initiate (or have initiated) big data analytics. the organisations that support healthcare delivery can be mapped to the meso level (e.g. dhbs, phos). in the nz context these organisations are likely to work with the government (e.g. dhbs) or have their own plans and initiate projects around big data (e.g. phos). • micro – service provider organisations (e.g. hospitals, general practices) and individuals (e.g. general physicians, clinicians) within those organisations are identified as the micro level. big data is typically generated at this level and has considerable potential to be utilised in this environment. however, it is observed that the use of big data does not solely lie with service provider organisations and their individual members. organisations that fall under the macro and meso umbrellas are keen to make use of big-healthcare-data generated at the micro level for better planning and service delivery within the nz health system. australasian journal of information systems weerasinghe, pauleen, scahill & taskin 2018, vol 22, research article a theoretical framework of big data in healthcare 13 figure 3 provides a conceptualisation of the mmm levels in nz healthcare and how the notion of big data fits within each level. figure 3: conceptualisation of mmm in nz healthcare table 2 presents the elements of srt mapped to the nz setting. big data is the construct of interest for the study, however depending on the group (mmm level) that is being studied at that instance, big data analytics sit within planning, implementation or use stages. the groups are mapped to the identified levels of healthcare (mmm). individuals vary depending on the level (group) under study. object big data group macro, meso, micro levels (each level is considered as one social group) individual macro – directors, strategy level roles of moh and business units; meso – managers of healthcare planning and funding bodies; micro – the users of systems that generate data and analytics outcomes table 2: elements of social representation theory for the proposed research although based on srt mmm are identified as social groups to study, it may be that there are unseen sub-groups within them. these sub-groups could be influencing representation of big data within each level (group) which can only be identified through investigation. however, within each of these socially constructed groups (mmm), a social representation of big data will be uniquely constructed with or without the influence of sub-groups. new zealand’s application of it within the healthcare sector is among the highest in the developed world (protti and bowden 2010). in particular, the primary healthcare sector makes extensive use of information systems (atalag et al. 2013). these information systems aid the clinicians in many tasks varying from administrative to management of patient care delivery as well as the associated clinical activity required to achieve this. administrative is are used for appointment scheduling, billing and financial administration. clinical is such as ehrs, eprescriptions, ereferrals, and lims are used by clinicians to monitor patient history, obtain the latest medicine updates, refer patients to specialists, receive test results and so forth (atalag et al. 2013, pollock 2012). therefore, cis and ais are effectively used across meso and micro levels of the nz health sector. additionally, information systems are used in hospitals to aid clinicians in healthcare delivery and hospital management. the is applications that are used in nz hospitals include clinician portals, patient management systems, systems for admission management, systems for management of transfers and discharges, bed management, outpatient management, laboratory ordering and result reporting systems, digital radiology reporting systems and systems to manage specific departments such as the intensive care unit, emergency department and operating theatres (pollock 2012). use of such information systems rapidly generates different types of healthcare data in large volumes; therefore, it is likely to have the australasian journal of information systems weerasinghe, pauleen, scahill & taskin 2018, vol 22, research article a theoretical framework of big data in healthcare 14 characteristics of big data (andreu-perez et al. 2015). a recent study undertaken by the national institute for health innovation (nihi) found that most datasets generated by information systems across the nz healthcare sector are able to be linked through the national health index (nhi) and thus can be used together for large-scale data analysis (atalag et al. 2013). this provides some evidence that nz has taken a step towards implementing big data analytics and there is significant opportunity to further perform analyses within and across these multiple large data sets. in addition to this, the recently revised nz health strategy (minister of health 2016) identifies the “smart system” as a key action area. the concept of a smart system is to collect wellorganised data from across the healthcare system as well as from other sources, to be able to achieve better health outcomes as well as to share information with other government bodies in order to achieve inter-sectorial government-wide goals (minister of health 2016). although some work around the use of big data analytics is available within the global healthcare sector, published research is very limited in the nz context. at the time of writing tormay (2015) and atalag et al. (2013) are the only publications that could be found in the nz setting which directly discuss big data in the nz healthcare context. neither of these two publications addresses social dynamics nor investigates business-it alignment. 5 discussion: the theoretical framework 5.1 the framework this section describes the theoretical framework, big data alignment using srt (ba-srt) (see figure 4), that has been developed from the findings in the literature on business-it alignment, big data and the healthcare sector. srt as the methodological lens guided the development of this framework5. as the framework was developed by considering the structure of the nz healthcare sector it is ideal for use in that setting. the theoretical framework guides investigation of internal alignment by looking at government/business objectives and big data using the social dimension as a lens at each sector level (macro, meso and micro). srt enforces studying social representations of big data at each level to understand alignment through a social dimension lens. using the developed framework in a different country context may require the framework to be adapted dependent on whether it is a similar health system. as shown in the framework (see figure 4), although big data is generated at the micro level by the clinical interface, big data can be used across the healthcare sector for strategy and policy making, planning and funding, as well as for clinical decision making. both macro and meso level organisations can be identified for implementing strategies and plans for successful use of big data, along with their involvement in the implementation of big data-related tools. big data can be used for strategy and planning decisions at both macro and meso levels (e.g. population health). therefore, the framework can be used to guide researchers looking into alignment, strategy, implementation and use of big data at the macro and meso levels. big data generation and use can also be studied simultaneously at the micro level using this framework. 5 formation of groups and their functions are informed by the preliminary interviews conducted. australasian journal of information systems weerasinghe, pauleen, scahill & taskin 2018, vol 22, research article a theoretical framework of big data in healthcare 15 figure 4: big data alignment using srt (ba-srt) 5.2 advantages of applying this framework this framework can be used as a guide to explore the influence of big data on business-it alignment using the social dimension of alignment. as shin (2015) suggests, the use of big data involves social practices and so there needs to be more focus on integration with the social setting. moreover, investigating the social dimension will facilitate understanding of how big data is integrated amongst the social elements of healthcare planning and delivery. the framework that has been formulated in this synthesis provides clear guidance through understanding: (i) what to study (big data strategy, implementation, generation and use at respective levels), (ii) where to study (mmm levels), and (iii) whom (policy makers, planners and funders and clinicians) to study when researching big data in the nz context. achieving and sustaining alignment is not as simple as ensuring effective formulation of big data plans and strategies and maintaining consistency with documentation, but rather the execution of is strategies, which includes user interactions (dulipovici and robey 2013). using a social dimension lens to explore business-it alignment, we can study how strategies, plans, projects and implementations around big data are perceived by different players at different levels within the sector. this brings a focus to the human interaction with technology from a sector perspective. it is important to know how plans around big data are interpreted by different players in organisations and bodies throughout the sector. although documented is strategy (big data plans) could be properly aligned with the documented business strategies, the situated use could influence alignment differently. the situated use of is involves the subjective understanding of a technological concept (i.e. big data analytics) (dulipovici and robey 2013). thus it is important to look at the level of understanding and commitment to big data across healthcare organisations by key stakeholders. in order to do so, we propose that it is appropriate to look at alignment through a social dimension lens, to gain the greatest insights on the influence of big data on business-it alignment. our theoretical framework recommends studying social representations of big data analytics, allowing a more humanistic approach. australasian journal of information systems weerasinghe, pauleen, scahill & taskin 2018, vol 22, research article a theoretical framework of big data in healthcare 16 the developed framework will enable researchers studying business-it alignment to engage with all levels of the healthcare sector, from policy makers, planners, funders to practitioners. the framework therefore specifically encourages sector wide alignment, ensuring a holistic understanding of the situation of big data within the nz healthcare sector. it is also important that alignment is present internally (within the healthcare organisations) as well as externally aligning to the government (e.g. nz health strategy) and other funders’ plans. as such we propose alignment should be studied internally within healthcare organisations as well as externally across the sector. given the strategic refocus of the nz health strategy (minister of health 2016) on refining the health strategy to set new directions for the next 10 years, it will be beneficial to understand how the nz healthcare sector is reacting to the notion of big data analytics and its developments. as our framework takes a holistic view of alignment we believe it provides a sound platform for studying business-it alignment. given that big data is a high profile phenomenon within the healthcare context in nz and internationally, it is important to examine the perceived effect of these big data initiatives on business-it alignment. 5.3 limitations of the framework an identified limitation of our theoretical framework is that it has been formulated to meet the needs of a big data study in the new zealand healthcare context. based on the interpretative nature of the study and the fact that social constructions will inform the findings, it is expected that this framework is not generalizable to other healthcare contexts in its current form, and this is identified as a pitfall. countries with similar healthcare settings may wish to adopt this framework with minimal change, but for the large majority of nations this framework will require modification based on the structure of the system and how much their system differs from the new zealand context. however, it is expected that within most developed countries where there is a structured health system in place the macro, meso and micro levels should be able to be identified. another identified pitfall of the developed theoretical framework is that although it encourages studying health sector wide alignment, it has not engaged the technology vendors that work within the system and alignment with these groups must be investigated in future research. in such cases, researcher will need to analyse the similarity of the external party (such as it vendor) with an identified level (i.e. meso) and amend the framework as required. 6 conclusion: implications, recommendations and reflections there is much interest in the phenomenon of big data analytics in the modern business world. the big data construct has been discussed and researched largely through technical dynamics such as analytics, security and technological infrastructure requirements. much less research has been reported in the area of social dynamics and the influence of these dynamics on big data strategies and use: i.e. the human side of is implementation. therefore, research on social dynamics such as experience, perceived value and usefulness is an important contribution for contemporary big data literature and especially in the inherently complex healthcare sector. in addition to the context, the study of social dynamics is complex, particularly in the case of a construct such as big data. as such, sound planning and a theoretical foundation is required before entering the field in order to carry out an empirical study. australasian journal of information systems weerasinghe, pauleen, scahill & taskin 2018, vol 22, research article a theoretical framework of big data in healthcare 17 in addition, in this paper we discuss the development of a theoretical framework that is founded on and grounded in the literature, which will be used to guide this research on social dynamics associated with the big data construct. srt was used as a methodological lens to guide the development of a theoretical framework. big data research suggests that alignment of big data to business goals and objectives is key to the success of big data initiatives. using srt as the methodological lens to study social dynamics, the developed framework identifies the groups to be studied and sub-groups within the groups to study. this will act as a guide to an empirical study which aims to investigate the complex nature of business-it alignment around big data use in the nz healthcare context. the development of a theoretical framework to study the influence of big data on business-it alignment through social representations has two implications: on the one hand, it forces structure and direction to study the identified gaps in the literature. on the other, it aims to investigate how social dynamics around on-going and planned big data projects influence alignment between business and it in the healthcare context. to date, such a framework is not available and such studies have not been conducted in the new zealand or global context. a study using the developed theoretical framework will capture a broader view of the nz healthcare sector and its players’ understandings and perceptions that will lead to actions around big data use. therefore, this framework and subsequent study findings has the potential to influence policy and practice. additionally, understanding social dynamics around big data at each level outlined in the framework (macro, meso, micro) will facilitate investigation of alignment of these social dynamics across the sector levels. because big data is a phenomenon that runs through all the sector levels (i.e. generated at the clinical interface but can be used by policy makers, planners and funders) it is important that optimal intersector alignment is achieved across the health sector. research studies using this theoretical framework will identify the current level of inter-sector alignment between levels (macro, meso, micro) that will better inform policy makers, planners, funders as well as the practitioners of the current status of big data and its possible implications. the use of srt as a methodological lens is a contribution of this paper. using theory to explain and simplify phenomena is common practice (mintzberg 2005). in this paper, we have used theory to focus and simplify the literature into a theoretical framework, which can then be used to explain phenomena. we believe this is the first time srt has been used in this fashion. we recommend that theories like srt can provide methodological guidance to studies and are capable of providing a basis for the development of theoretical frameworks, which can then be used in empirical studies. this review of the literature provides an understanding of the gaps to be filled and the direction our future study will take. however, to go a step beyond, having a methodological lens to pull literature and context together to provide a solid foundation in the form of a preliminary theoretical framework is desirable for this research. our theoretical framework was developed in the context of the nz healthcare sector and generalisability will be limited. varying degrees of modification may be required if applying this framework to other countries’ healthcare contexts, dependent on how similar the systems are. despite this, the applicability of srt as a methodological lens to investigate social dynamics around big data in a healthcare context is a novel approach which any healthcare sector should be able to benefit from. we recommend the use of srt to study social dynamics around big data, beyond the healthcare sector. other government sectors, such as education australasian journal of information systems weerasinghe, pauleen, scahill & taskin 2018, vol 22, research article a theoretical framework of big data in healthcare 18 and transportation, may be able to adopt our framework, however considering sectorial differences there may be a need for amendments. additionally, the healthcare sector can use the developed framework to investigate the introduction of novel technological concepts with minimal change to the developed theoretical framework. references ahmadian, l., khajouei, r., nejad, s. s., ebrahimzadeh, m. & nikkar, s. e. (2014) “prioritizing barriers to successful implementation of hospital information systems”, journal of medical systems, 38(12):151. doi: 10.1007/s10916-014-0151-9 andersén, j. & andersén, a. (2014) “deconstructing resistance to organizational change – a social representation theory approach”, international journal of organizational analysis, 22(3): 342 355. anderson, j. g. (2007) “social, ethical and legal barriers to e-health. international journal of medical informatics, 76(5-6): 480-483. doi: dx.doi.org/10.1016/j.ijmedinf.2006.09.016 andreu-perez, j., poon, c. c., merrifield, r. d., wong, s. t. & yang, g.-z. (2015) “big data for health”, ieee journal of biomedical and health informatics, 19(4):1193-1208. atalag, k., gu, y. & pollock, m. (2013) a stocktake of new zealand’s healthcare datasets. health informatics new zealand conference, rotorua. bates, d. w., saria, s., ohno-machado, l., shah, a. & escobar, g. (2014) “big data in health care: using analytics to identify and manage high-risk and high-cost patients”, health affairs, 33(7):1123-1131. doi: 10.1377/hlthaff.2014.0041 bean, r. & kiron, d. (2013) “organizational alignment is key to big data success”, mit sloan management review big idea: data & analytics, 54(3). bholat, d. (2015) “big data and central banks”, quarterly bulletin 2015 q1, 55(1): 86-93. breakwell, g. m. (1993) “social representations and social identity”, papers on social representations, 2(3):198-217. bush, m., lederer, a. l., li, x., palmisano, j. & rao, s. (2009) “the alignment of information systems with organizational objectives and strategies in health care”, international journal of medical informatics, 78(7):446-456. celi, l. a., mark, r. g., stone, d. j. & montgomery, r. a. (2013) “‘big data’ in the intensive care unit. closing the data loop”, american journal of respiratory and critical care medicine, 187(11):1157-1160. doi: 10.1164/rccm.201212-2311ed chan, y. e. & reich, b. h. (2007) “it alignment: what have we learned?”, journal of information technology, 22(4):297-315. chawla, n. & davis, d. (2013) “bringing big data to personalized healthcare: a patientcentered framework”, jgim: journal of general internal medicine, 28(suppl 3):660-665. doi: 10.1007/s11606-013-2455-8 chen, m., mao, s. & liu, y. (2014) “big data: a survey”, mobile networks and applications, 19(2):171-209. cumming, j. (2011) “integrated care in new zealand”, international journal of integrated care, 11(special 10th anniversary edition):e138. australasian journal of information systems weerasinghe, pauleen, scahill & taskin 2018, vol 22, research article a theoretical framework of big data in healthcare 19 davenport, t. h. (2013) “analytics 3.0: in the new era, big data will power consumer products and services”, harvard business review. available at: https://hbr.org/2013/12/analytics-30 [accessed 10 sep 2015] davenport, t. h. & dyché, j. (2013) “big data in big companies”, international institute for analytics, 3. dhawan, r., singh, k. & tuteja, a. (2014) “when big data goes lean”, mckinsey quarterly, 24(2):97-102. dopfer, k., foster, j. & potts, j. (2004) “micro-meso-macro”, journal of evolutionary economics, 14(3):263-279. drazin, r. & van de ven, a. h. (1985) “alternative forms of fit in contingency theory”, administrative science quarterly, 30(4):514-539. dulipovici, a. & robey, d. (2013) “strategic alignment and misalignment of knowledge management systems: a social representation perspective”, journal of management information systems, 29(4):103-126. eynon, r. (2013) “the rise of big data: what does it mean for education, technology, and media research?”, learning, media and technology, 38(3):237-240. gal, u. & berente, n. (2008) “a social representations perspective on information systems implementation: rethinking the concept of ‘frames’”, information technology & people, 21(2):133-154. gartner. (2013) it glossary. available at: http://www.gartner.com/it-glossary/big-data/ [accessed 25 jun 2016] gauld, r. (2009) the new health policy. open university press, maidenhead. gerow, j. e., thatcher, j. b. & grover, v. (2014) “six types of it-business strategic alignment: an investigation of the constructs and their measurement”, european journal of information systems, 24(5):1-27. doi: 10.1057/ejis.2014.6 glandon, g. l., austin, c. j., boxerman, s. b., smaltz, d. h., & slovensky, d. j. (2008) austin and boxerman's information systems for healthcare management. health administration press, chicago. groves, p., kayyali, b., knott, d. & van kuiken, s. (2013) “the 'big data' revolution in healthcare: accelerating value and innovation”, mckinsey quarterly. henderson, j. c. & venkatraman, n. (1992) “strategic alignment: a model for organizational transformation through information technology”, in t. a. kochan & m. useem (eds.), transforming organizations (pp. 97-116). oxford university press, new york. henderson, j. c. & venkatraman, n. (1993) “strategic alignment: leveraging information technology for transforming organizations”, ibm systems journal, 32(1):472-484. jagadish, h. v., gehrke, j., labrinidis, a., papakonstantinou, y., patel, j. m., ramakrishnan, r. & shahabi, c. (2014) “big data and its technical challenges”, communications of the acm, 57(7):86-94. kacfah emani, c., cullot, n. & nicolle, c. (2015) “understandable big data: a survey”, computer science review, 17. doi: dx.doi.org/10.1016/j.cosrev.2015.05.002 australasian journal of information systems weerasinghe, pauleen, scahill & taskin 2018, vol 22, research article a theoretical framework of big data in healthcare 20 kannry, j. (2011) “effect of e-prescribing systems on patient safety”, the mount sinai journal of medicine, 78(6):827-833. doi: 10.1002/msj.20298 kappelman, l. a., mcleon, e., luftman, j. & johnson, v. (2013) “key issues of it organizations and their leadership: the 2013 sim it trends study”, mis quarterly executive, 12(4):227-240. liljenström, h. & svedin, u. (2005) micro, meso, macro: addressing complex systems couplings, world scientific publishing company. loshin, d. (2013) big data analytics : from strategic planning to enterprise integration with tools, techniques, nosql, and graph. amsterdam: morgan kaufmann. luftman, j. n. (1996) competing in the information age : strategic alignment in practice, oxford university press, new york. mace, s. (2014) “big data and the analytics imperative”, healthleaders magazine, 17(9): 12-24. mays, n. (2013) reorienting the new zealand health care system to meet the challenge of longterm conditions in a fiscally constrained environment. new zealand treasury conference, affording our future, wellington. mcafee, a. & brynjolfsson, e. (2012) “big data: the management revolution”, harvard business review, 90(10):60-68. menon, n. m., yaylacicegi, u. & cezar, a. (2009) “differential effects of the two types of information systems: a hospital-based study”, journal of management information systems, 26(1):297-316. minister of health (2016) new zealand health strategy: future direction, ministry of health, wellington. mintzberg, h. (2005) “developing theory about the development of theory”, great minds in management: the process of theory development, 355-372. moh (2011) overview of the health system. available at: http://www.health.govt.nz/newzealand-health-system/overview-health-system [accessed 20 may 2015]. moh (2014) briefing to the incoming minister 2014, wellington. moh (2016) ministry business units. available at: http://www.health.govt.nz/aboutministry/ministry-business-units [accessed 19 january 2017]. moscovici, s. (1963) “attitudes and opinions”, annual review of psychology, 14(1):231-260. moscovici, s. (1984) “the phenomenon of social representation”, in r. m. farr & s. moscovici (eds.), social representations: european studies in social psychology, cambridge university press, cambridge. moscovici, s. (1988) “notes towards a description of social representations”, european journal of social psychology, 18(3):211-250. nash, d. b. (2014) “harnessing the power of big data in healthcare”, american health & drug benefits, 7(2):69-70. australasian journal of information systems weerasinghe, pauleen, scahill & taskin 2018, vol 22, research article a theoretical framework of big data in healthcare 21 newvantage partners (2012) big data executive survey: themes & trends. available at: http://newvantage.com/wp-content/uploads/2012/12/nvp-big-data-surveythemes-trends.pdf [accessed 20 november 2015] nichols, b. (1981) ideology and the image: social representation in the cinema and other media (vol. 256), indiana university press, bloomington. oecd (2013) health at a glance. available at: http://www.oecd-ilibrary.org/ [accessed 20 may 2016] pare, g., sicotte, c., jaana, m. & girouard, d. (2008) prioritizing clinical information system project risk factors: a delphi study. 41st hawaii international conference on system sciences (hicss 2008). patil, p., raul, r., shroff, r. & maurya, m. (2014) “big data in healthcare”, international journal of research in information technology, 2(2):202-208. peng, g., dey, d. & lahiri, a. (2014) “healthcare it adoption: an analysis of knowledge transfer in socioeconomic networks”, journal of management information systems, 31(3):7-34. doi: 10.1080/07421222.2014.994672 pollock, m. (2012) better information for better care new zealand's approach to efficient and affordable healthcare (white paper), national institute for health innovation (nihi). available at: http://www.ehealthnews.eu/download/white-papers/3028-betterinformation-for-better-care-new-zealands-approach-to-efficient-and-affordablecare [accessed 10 november 2015]. prewitt, e. (2014, 1 november) “addressing the big challenges”, healthleaders magazine, 17:66. protti, d. & bowden, t. (2010) “electronic medical record adoption in new zealand primary care physician offices”, commonwealth fund: issues in international health policy, 96. reich, b. h. & benbasat, i. (1996) “measuring the linkage between business and information technology objectives”, mis quarterly, 20(1):55-81. russom, p. (2011) “big data analytics”, tdwi best practices report, fourth quarter. available at: ftp://ftp.software.ibm.com/software/tw/defining_big_data_through_3v_v.pdf [accessed 10 oct 2015] sabherwal, r. & chan, y. e. (2001) “alignment between business and is strategies: a study of prospectors, analyzers, and defenders”, information systems research, 12(1):11-33. saporito, p. (2013) “the 5 v's of big data: value and veracity join three more crucial attributes that carriers should consider when developing a big data vision”, best's review, 38. available at: https://www.thefreelibrary.com/the+5+v%27s+of+big+data%3a+value+and+ve racity+join+three+more+crucial...-a0350676739 [accessed 10 november 2015] sathi, a. (2012) big data analytics: disruptive technologies for changing the game: mc press, boise. australasian journal of information systems weerasinghe, pauleen, scahill & taskin 2018, vol 22, research article a theoretical framework of big data in healthcare 22 scahill, s. l. (2012) “the 'way things are around here': organisational culture is a concept missing from new zealand healthcare policy, development, implementation, and research”, new zealand medical journal (online). available at: http://www.nzma.org.nz/journal/read-the-journal/all-issues/2010-2019/2012/vol125-no-1348/view-scahill [accessed 10 october 2015] shin, d.-h. (2015) “demystifying big data: anatomy of big data developmental process”, telecommunications policy, 40(9):837-854. shin, d.-h. & choi, m. j. (2015) “ecological views of big data: perspectives and issues”, telematics and informatics, 32(2):311-320. doi: http://dx.doi.org/10.1016/j.tele.2014.09.006 sicotte, c., pare, g., moreault, m.-p. & paccioni, a. (2006) “a risk assessment of two interorganizational clinical information systems”, journal of the american medical informatics association, 13(5):557-566. sousa, j. l. r. & machado, r. j. (2014) “sociomaterial enactment drive of business/it alignment: from small data to big impact”, procedia technology, 16:569-582. doi: 10.1016/j.protcy.2014.10.005 tan, f. b. & gallupe, r. b. (2006) “aligning business and information systems thinking: a cognitive approach”, ieee transactions on engineering management, 53(2):223-237. doi: 10.1109/tem.2006.872243 tormay, p. (2015) “big data in pharmaceutical r&d: creating a sustainable r&d engine”, pharmaceutical medicine new zealand, 29(2):87-92. doi: 10.1007/s40290-015-0090-x vaast, e. (2007) “danger is in the eye of the beholders: social representations of information systems security in healthcare”, the journal of strategic information systems, 16(2):130152. wagner, w., duveen, g., farr, r., jovchelovitch, s., lorenzi-cioldi, f., markova, i. & rose, d. (1999) “theory and method of social representations”, asian journal of social psychology, 2(1):95-125. wang, x. & huang, j. z. (2015) “editorial: uncertainty in learning from big data”, fuzzy sets and systems, 258(1):1-4. doi: 10.1016/j.fss.2014.10.010 ward, m. j., marsolo, k. a. & froehle, c. m. (2014) “applications of business analytics in healthcare”, business horizons, 57(5):571-582. doi: 10.1016/j.bushor.2014.06.003 watson, h. j. (2014) “tutorial: big data analytics: concepts, technologies, and applications”, communications of the association for information systems, 34(1):1247-1268. wyber, r., vaillancourt, s., perry, w., mannava, p., folaranmi, t. & celi, l. a. (2015) “big data in global health: improving health in lowand middle-income countries”, bulletin of the world health organization, 93(3):203-208. doi: 10.2471/blt.14.139022 yusof, m. m., kuljis, j., papazafeiropoulou, a. & stergioulas, l. k. (2008) “an evaluation framework for health information systems: human, organization and technology-fit factors (hot-fit)”, international journal of medical informatics, 77(6):386-398. australasian journal of information systems weerasinghe, pauleen, scahill & taskin 2018, vol 22, research article a theoretical framework of big data in healthcare 23 copyright: © 2018 weerasinghe, pauleen, scahill & taskin. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 literature review 2.1 big data 2.2 business-it alignment 2.3 transformation of healthcare with information technology 2.3.1 big data in healthcare 3 social representation theory as a methodological lens 4 the case: new zealand healthcare sector 5 discussion: the theoretical framework 5.1 the framework 5.2 advantages of applying this framework 5.3 limitations of the framework 6 conclusion: implications, recommendations and reflections microsoft word ajis_18(3)_v10.docx australasian journal of information systems volume 18 number 3 2014 243 ‘i just saw this on facebook, i need it now’: exploring small business use of facebook naomi augar college of business victoria university melbourne, australia naomi.augar@vu.edu.au john zeleznikow college of business victoria university melbourne, australia john.zeleznikow@vu.edu.au abstract this paper reports on a study exploring social media adoption and use by twelve small businesses. results indicate that facebook is the dominant platform used and it can provide small businesses with a virtually cost free and easy way to reach customers. businesses reported immediate results from using facebook; customers arrived in store after seeing facebook posts. participants felt there were minimal risks involved and none had experienced negative feedback or inappropriate posts. most felt that using facebook increased their competitive advantage due to their enhanced online profile. despite some effort to engage customers, business to consumer transfer of information is the primary function of the facebook pages studied. page owners report limited training for and analysis of their facebook activities. this indicates a need for further research in the small business context, focused on approaches to boosting customer interaction, to realize the full potential of facebook engagement. keywords: facebook, small business, adoption, use, benefits. introduction social media allows users to generate content (posts) and respond to the posts of other individuals and organizations (kaplan & haenlein, 2010) in an informal, social online setting. in recent years, businesses have recognized the opportunity to join the conversation and have moved into the social media domain (harris & rae, 2009; hopkins, 2012; peters & salazar, 2010; seo & rietsema, 2010; singh, davison, & wickramasinghe, 2010). many businesses now have social media pages and customers are able to gather information about and review products and services and engage with businesses online. the use of social media is second only to email in terms of online communication trends in australia (neilsen, 2012). facebook is the dominant social media site in australia, boasting almost half the total population as members (cowling, 2012). recent nielsen (2012) market research indicates that there is an upward trend in consumer engagement with business via social media platforms in australia. in 2010, 46 per cent of users had ‘liked’ a brand online; in 2011 this figure rose to 57 per cent (neilsen, 2012). in australia 96 per cent of businesses are categorized as small businesses (australian bureau of statistics, 2012), having 20 or less employees (australian bureau of statistics, 2011). this equates to just over 2 million small businesses (australian bureau of statistics, 2012). only 700,000 of these businesses employ staff (australian bureau of statistics, 2012). in recent times, the australian retail australasian journal of information systems volume 18 number 3 2014 244 sector has experienced poor to negative growth (national retail association, 2012) (relative to historical performance), which has challenged all retailers, large and small, to try to enhance their competitive advantage. some are turning to social media. to date, the majority of social media research exploring business adoption and consumer engagement has focused on medium to large organisations, e.g. (jarvenpaa & tuunainen, 2012; saundage & lee, 2011; senadheera, warren, & leitch, 2011; seo & rietsema, 2010; singh, et al., 2010). there is some research, e.g. (hopkins, 2012; rupert hills & cairncross, 2011), focusing on why and how small businesses are using social media at present, but there is little work exploring what small business perceive as success in terms of social media engagement. some international work focuses on return on investment (roi) e.g. (hoffman & fodor, 2010; kaske, kugler, & smolnik, 2012; powell, groves, & dimos, 2011), business value e.g. (derham, cragg, & morrish, 2011; hopkins, 2012) and ‘how-to’ guides exist for businesses wanting to create a facebook presence e.g. (holzner, 2009). this research augments the existing literature and contrasts this work with results exploring the adoption and use of facebook in the small business context. whilst the scope of the research initially included all social media, in identifying cases, it was found that the majority of small businesses who were potential participants only used facebook. hence the focus was narrowed. the paper first presents the theory and literature that framed the research. the research methodology is explained and a description of the twelve qualitative cases explored is provided. the paper then presents the results and an analysis of research findings before concluding with implications and opportunities for future work. background business adoption and use of social media gallaugher and ransbotham (2010, pp.199-200) characterize business and consumer interaction through social media via their 3-m framework: megaphone (business sharing of information), magnet (consumer interactions with business) and monitor (consumer based interactions). sharing information (megaphone) with customers raises awareness about products and services within a customer base. however, the power of social media lies in the networked nature of the tool and its ability to reach far beyond the initial contact group, potentially broadening market share (magnet and monitor). to harness this potential network, businesses need to promote active customer participation via social media. that is, active posting techniques and interactions with their online customer base to ensure that customers understand the brand, product or service, the values and behaviours of the company (jarvenpaa & tuunainen, 2012). active customer engagement can be developed through techniques including personalized feedback, contests and visioning scenarios (jarvenpaa & tuunainen, 2012). kaplan and haenlein (2010, pp.6667) provide five useful points on being social. they identify the need for businesses to be active, interesting, humble, unprofessional and importantly, honest. these points posit that businesses should not simply use social media to pass on information to customers. the social nature of the platform requires businesses to adjust their communication style accordingly – social media communications need to be accessible and less formal than traditional business communication so as to engage users. this takes resources, time and knowledge to implement. in addition, a key ingredient to active engagement is achieving critical mass. having a large number of consumers engaging with the site can enhance the chances of active discussion and engagement (senadheera, et al., 2011). understanding how established businesses promote their social media presence can be instructive. while obvious engagement metrics including the number of followers are important, the promotion of social media by a business is critical. key facets to consider include the visibility of links to social network pages and the use of social network features (sharing or liking content) on organizational web sites (peters & salazar, 2010). australasian journal of information systems volume 18 number 3 2014 245 an engaged customer group have the potential to become brand ambassadors for a business through the promotion of products and services based on their own experience. word of mouth (wom) marketing, the recommendation or promotion of a product through interpersonal exchanges within a network of consumers (brooks jr, 1957; cao, knotts, xu, & chau, 2009) characterizes this phenomenon. electronic word of mouth (ewom) marketing, describes the extension of wom marketing from physical social networks to those facilitated online (nakayama, wan, & sutcliffe, 2010) by web 2.0 platforms, including social media. word of mouth recommendations play an important role in influencing consumer decision making (brown & reingen, 1987; kaske, et al., 2012). it is clear that companies, large and small are adopting social media to enhance their marketing capability and reach their customers. finnair have used social media to support the renewal of their brand (jarvenpaa & tuunainen, 2012), small craftspeople are using it to raise their profile (holmes, mclean, & green, 2012) and restaurants are using social media to support marketing and relationship management and to respond to negative reviews on other sites (derham, et al., 2011). medium to large australian based organizations are using social media for marketing, advertising, raising brand awareness, customer engagement and reducing advertising costs (singh, et al., 2010). these themes are echoed in studies from new zealand and england where small retailers used social media for customer engagement citing market penetration, and the absence of upfront costs as major factors in adoption (harris & rae, 2009; hopkins, 2012). some companies believe that social media engagement increases their competitive advantage (singh, et al., 2010) and creates business value (derham, et al., 2011; hopkins, 2012). in a recent case, an online retailer quantified business value in terms of impact on sales, which increased dramatically as a result of engagement via facebook (hopkins, 2012). however, large organizations do not always find that social media reduces marketing costs. in fact, a key challenge for large organizations is the financial cost of resourcing social media engagement (fueller, schroll, dennhardt, & hutter, 2012; kuikka & äkkinen, 2011). additionally, while small craftspeople appreciate that social media has no financial cost, they feel it costs them valuable time that they would otherwise dedicate to production (holmes, et al., 2012). recent media coverage, e.g. (cook, 2012; moses, 2011), has highlighted the potential for negative or inappropriate posts that can harm business reputations. this is echoed by gallaugher and ransbotham (2010) who offer recommendations on managing risks including dissent and negative feedback via social media. models of technology adoption and use seminal work by davis (1989, p.320) identified that there are two major influencing factors in terms of an organizations adoption of technology: perceived usefulness and perceived ease of use. the oecd (1999) noted the need for a model to move beyond technology adoption, to chart the technology lifecycle. their model mapped ecommerce readiness (motivations for and barriers to adoption), intensity (documenting the technology in action through observation and metrics on technology usage) and impact (understanding the benefits that technology adoption brings to the organization and beyond) as three key areas that chart the adoption and usage life cycle (oecd, 1999). fuchs et al. (2010, p.168) adapted the original oecd (1999, pp.6-7) model and zhu and kraemer’s (2005, p.66) model of ebusiness adoption to present a detailed framework mapping readiness, intensity and impact in the ebusiness context. fuchs et al.’s (2010, p.168) framework identifies several facets that impact business readiness to adopt technology: organizational context – it resources required to get started, potential costs and benefits environmental context – the pressure the business perceives from competitors and customers australasian journal of information systems volume 18 number 3 2014 246 firm related context – the size and type of the business decision maker’s context – the demographic, background and risk propensity of the decision maker (fuchs, et al., 2010, p.168) the framework maps the use of it from adoption through day to day business to intensity and aspects of it value are explored through impact on sales, efficiency, business relationships and customer satisfaction which are mapped to impact (fuchs, et al., 2010, p.168). these models and framework provide a useful lens for exploring business facebook adoption and use. this research adopted the oecd readiness, intensity and impact model (1999) and the fuchs et al. (2010) framework to guide exploration of the following key research questions: readiness why do small businesses adopt facebook and how do they resource their facebook presence? do they encounter barriers and risks when adopting facebook? intensity how are small businesses using facebook and how are their customers responding to their facebook presence? impact what do small businesses believe are the benefits of using facebook? do they believe it provides their business with a competitive advantage and a good return on investment? methodology the research used an exploratory multiple-case study approach (yin, 2009) collecting data through a series of semi-structured interviews. a case study approach is considered appropriate when exploring information systems (is) (benbasat, goldstein, & mead, 1987) and their use within organisations. thirty minute interviews were conducted with representatives of twelve small businesses with 20 or less employees (australian bureau of statistics, 2011) drawn from the retail, health and beauty sectors in melbourne, australia. cases were identified through a purposive sampling approach involving internet searches focusing on small businesses with facebook pages where the businesses were physically located in the inner east and west regions of melbourne. the retail sector was selected as the primary group for study due to the recent growth challenges experienced by the sector in australia (national retail association, 2012). several cases from the health and beauty sectors were also selected. while location and business type was not believed to influence facebook adoption, use or perceptions of impact, different areas and businesses were selected for contrast. the sample was further restricted to cases where the facebook page had been established for more than a year with either 200+ or less than 80 likes. these limitations were designed to identify diverse and polar cases (eisenhardt, 1989) that might extend understanding of what business representatives perceive as beneficial in terms of facebook engagement for small business. businesses were contacted in person to discuss the project and their ability to participate. the size of the final sample was limited to between 10-15 businesses. this limit aimed to provide enough scope to identify emergent themes while still retaining a manageable number of cases (miles & huberman, 1994). australasian journal of information systems volume 18 number 3 2014 247 interviews were conducted with the retail store owner/manager or manager6. participants were selfnominated by the businesses that volunteered to join the study. due to the limited number of staff and the need to protect brand and reputation in small businesses, cases de-identified to protect the privacy of participants. additional data was collected through de-identified observation of participating small business web and social media presences7 (peters & salazar, 2010; senadheera, et al., 2011). interviews were recorded and transcribed via a third party service to facilitate text based cross-case analysis (yin, 2009). transcriptions were reviewed and classified using a coding in detail approach using nvivo to cluster codes into tree nodes (bazley, 2007, p.69) that allowed for identification of emerging themes relating to facebook adoption, usage, risks and benefits, roi and competitive advantage; a cross-case synthesis approach (yin, 2009). the interview results were augmented with results of site observations, which were classified to identify and confirm usage trends. this study focuses on a limited sample of small retail, health and beauty businesses and the results are presented to provide insights into the themes around adoption, use and the value of social media perceived by small businesses. however, due to the sample size and restrictions the results are not representative, nor generalizable. results the final sample comprised six cases drawn from the east (e) of melbourne and six from the west (w) of melbourne, as noted in table 1. three quarters of the businesses were retail (r) focused, with two hairdressers (beauty-b) and one osteopathy clinic (health h) rounding out the sample. participants were asked to describe the demographic of their typical customer. all responded that the vast majority were female. seven participants could identify a broad age range from 15-60 years of age, with the majority citing 30-55. readiness participants were invited to discuss their business, their background and motivation for adopting facebook, and their experience of getting it established. several common themes emerged across the cases. the interview participants were either the owner or a staff member within the business who was responsible for the social media presence. the only commonality within the participant group in terms of education and experience was that they had no technology background or formal technology or marketing qualification; every story was unique. prior to becoming small business owners or managers, participants held a variety of roles from zoo keeper, publisher to scientist. all participants reported some sort of experiential learning when adopting social media for their business. all but one participant had used facebook in their personal lives prior to adopting it for business. a few participants reported learning more about tools and techniques informally through staff members, family or friends. only two participants had attended specialist facebook training. one had attended two one hour sessions of training which was provided free of charge to members of the local chamber of commerce. they found the training beneficial and noted that it was non-technical and focused on increasing dialogue with customers via facebook. 6 mid-2012 7 from mid-2012 through mid-20 australasian journal of information systems volume 18 number 3 2014 248 participants were asked if they had a documented business or marketing plan. this question aimed to understand the degree of planning behind the decision to adopt social media. only two companies had a marketing plan and two others had a business plan but no marketing plan. social media use did not feature in any of the plans. the vast majority of businesses operated without any documented plans. as one participant put it, “no we don’t [have a business plan], it is all in our heads (case 10).” as noted in table 1, a common feature of all participants was their use of a facebook page to promote their business. all but one of the businesses had established their facebook site over a year ago, with most establishing their site in 2011. facebook is the dominant social media platform in australia, and this guided some participants when selecting the best platform to focus their social media activities. “at that time... there was myspace and facebook i think, and facebook was the one that was starting to take over. we have never had anything to do with myspace, all of the talk was surrounding the facebook phenomenon at that stage (case 12).” however, as is noted in table 1, a variety of other social media tools were adopted by a minority of businesses, ranging from a couple using instagram and flickr for photo sharing and management to twitter, which was used by three participants for sharing information, but not to the same degree as facebook (in one case twitter had been dormant for some time). pinterest, a relatively new image based social media platform was being used experimentally by two businesses to share images of products. finally, three participants had a blog that they used for extended articles, which were not suited to facebook. the dominant platform for all businesses was facebook, hence the focus of the feedback gathered in interviews is the business experience with that platform. case 1 2 3 4 5 6 7 8 9 10 11 12 business type b r r b r h r r r r r r no. staff 8 4 4 4 5 9 2 5 4 8 2 4 location w e e w w w e e w e w e facebook y y y y y y y y y y y y web site y y y y y y y y y y y ecommerce y y y twitter y y y pinterest y y instagram y flickr y blog y y table 1 small business participant overview all interview participants noted that setting up their business facebook page was relatively quick and easy. one participant noted “the reason i chose facebook was because i knew how to use it and it was free (case 7).” another echoed this sentiment, “because it is free, it is a free marketing tool (case 1).” the absence of entry costs and the relative ease of use of the tool were cited consistently as enabling their use of it for business purposes. a few noted that their knowledge of the tool based on their personal australasian journal of information systems volume 18 number 3 2014 249 use helped with the adoption decision, “i had already used facebook as a personal user and it was all fairly intuitive, click here to make a business account… (case 7).” participants indicated that very few, if any resources were needed to start and maintain a social media presence. most established a presence with no help in a matter of minutes. many used their own mobile phone to take photos and a personal or work computer to setup and manage the page; no specialist equipment or other resources were required. seven participants did not identify any barriers to the adoption of social media. one participant noted that some customers appeared to be opposed to the increasing use of technology in the business, but that this had not stopped adoption of social media, “…we have been doing a service for 15 years that we think we have done pretty well and it is just a time, like everything, that we have to move on (case 3).” interestingly, the two businesses who had adopted the largest range of social media tools, noted that time was the only barrier to use. one said, “…i just don’t have time that is my main issue with social media is the time it takes (case 10).” another owner initially needed to be convinced by a staff member proposing facebook adoption that the time spent would bring benefits to the business. in identifying the key motivations for engaging with facebook the participants cited sharing information and raising awareness about their business and products; brand awareness. they also felt it was important to maintain the immediate connection with their customers and to be visible online. a few participants used the word community when describing their motivation for establishing a facebook page. they tried to provide value beyond promoting products and services by linking to articles and events related to their business, which might be of interest to customers. one third of the participants identified a desire to expand their customer base as a motivator for adopting facebook. almost half of the participants described what can be summarised as competitive pressure as a motivator for adopting facebook. the perceived pressure comes from observing other businesses; as one participant put it “i had been reading a lot about how businesses were reaching out in social media and it seemed to me that it was really important to be on that early on and getting to those people as soon as possible. you can get left behind very quickly in social media i find, so we jumped on board really quickly (case 10).” the pressure also came from a need to be where their clients were, “well once i got on facebook and realised the whole world was on facebook and facebook is people’s life, i realised that i had to get [my business] on facebook (case 4).” as part of the interview participants were invited to discuss their perception of risk with relation to their social media presence. all participants were aware of the potential damage that could be caused to business reputation through inappropriate posts and negative reviews. however, all participants noted that they had not had any negative or inappropriate posts via facebook. one business mentioned recent media coverage (cooper, 2012) to indicate awareness of the pitfalls. another felt that if that were to happen in the future they would deal with it as they might an in store or email complaint – by focusing on their “the customer is always right attitude (case 5).” this sentiment was echoed by most participants. intensity participants were invited to discuss how they used facebook to support their business. they described their posting habits and the customer engagement they received on facebook. table 2 summarizes the types of posts that businesses are placing on their facebook page. observation indicated that the majority of business posts were informational in nature with businesses describing and displaying (through embedded photos) new products in stock or services on offer. one participant noted the value of pictures (featured in posts by all businesses) and videos, “you know videos and pictures speak a lot louder than just words, so… [we]… try and have a visual which grabs the eye and makes the person want to actually click on it (case 8).” australasian journal of information systems volume 18 number 3 2014 250 case 1 2 3 4 5 6 7 8 9 10 11 12 likes 44 72 72 246 248 410 430 491 715 727 1433 6242 manager8 c o o o+ o o o c+ o+ o o o bus post freq.9 1pw 1pq 2pm 1pw 23pw 1pw 2pw 3pw 2pw 2pm 7pw 3pw product info. y y y y y y y y y y y y research info. y y y y humorous y y y y y y staff y y y y y seasonal y y y y y y y y event y y y y competitions y y y y y y y y y pictures y y y y y y y y y y y y videos y table 2 small business facebook page data a few businesses asked occasional questions of their followers to try and stimulate interaction. however, the majority of posts were ‘closed in nature’ (statements not explicitly inviting a response). two thirds of businesses used facebook to let customers know about competitions, special offers or promotions. over half the businesses regularly featured humorous or light hearted, off topic posts. one participant noted the value of an informal, humorous post, “…at easter i just randomly put up an album that was nothing to do with the shop which was just forty pictures of bunnies sitting in tea cups and when you see my user stats… it was ten times my usual access that weekend… (case 7)” this post also reflects the trend for seasonal posts relating to events like mother’s day, christmas, national events or public holidays. two thirds of businesses featured posts relating to such events. businesses estimate that a very small proportion (less than 10% of their customer base) had liked their facebook page, and of those who had, very few responded to business posts with a like or a comment. “i would say probably 80% use facebook and how many engage? 5%, not even (case 4).” when discussing customer interaction one participant noted, “they might leave a comment or they might ‘like’ a photo on facebook but generally i do not get a lot of interaction. i look at other peoples’ and think they have got 500 people like that site or post, why doesn’t anyone like mine? (case 5)” another participant noted, “if you don’t put the price up it’s always ‘how much?’ that is just your standard [comment] (case 12).” others felt that customers only responded if there was a competition or prize on offer. observation of the business facebook pages indicated that very few customers shared business 8 the person who manages the facebook page was either the owner (o) or staff member, known as a champion (c). in cases where staff assisted the primary administrator a + appears. 9 per week (pw), per month (pm) and per quarter (pq) – every three months. australasian journal of information systems volume 18 number 3 2014 251 posts and few commented in response to business posts. the majority of interactions were a modest number of likes (relative to the number of page likes) for individual business posts. impact participants were invited to reflect on the benefits that using facebook brought to their business. they were also asked their thoughts about whether facebook provided a competitive advantage, return on investment, led to an increase in sales and increased customer satisfaction. many common themes emerged across cases. when asked about the impact that facebook had on sales, several participants noted the virtually instantaneous results they received from posting a new product via facebook. one commented that “…sometimes people come in 10 minutes after i have posted, so it is really immediate (case 7),” noting that customers have come into the store and said, “i just saw this on facebook, i need it now (case 7).” this is echoed by another who found that facebook “… definitely generates traffic, physical traffic. people come into look at something they have seen on facebook, and that is good, or they will ring and ask about it (case 9).” this trend went beyond the retail sector, to health services “sometimes we will get people who we haven’t seen for a while and they say ‘i saw it on facebook, i really need to come in (case 6).’” all of the businesses used other mediums to promote their businesses, several had used flyers and placed or continue to place advertisements in local newspapers. most cross promote facebook in these advertisements. however, many participants noted that they didn’t feel that they received value from paid advertisements when compared with the value they gained from their facebook page postings which they didn’t have to pay to use. in contrast, one business noted that due to their modest following on facebook, they felt that they got more value from targeted text message marketing to vip customers on a regular basis. all of the businesses promoted their social media presence to some degree. all businesses with web sites featured social media logos on their site. the majority of businesses also promoted their social media presence in store through in store signs, while a minority also encouraged sales staff to promote it direct to customers making a purchase. one company put stickers advertising their social media presence on every purchase as it is wrapped at the point of sale. when they established their facebook page, they also purchased targeted facebook advertisements to help boost their following. eleven businesses perceived their use of facebook as providing them with a competitive advantage. specifically, they felt that they had an advantage over their competitors who were not on facebook. when asked if they felt facebook provided a competitive advantage, one responded “yes, i think so, compared to another gift shop 500 meters down the road, if they are not using facebook then i just have that benefit of, that my logo pops up on peoples screen a couple of times a week when hopefully it will enter their mind to go and have a look (case 7).” the business who did not agree felt that facebook engagement had the potential to enhance competitive advantage, but given their modest following on facebook, they felt that potential was yet to be realized. when queried about the benefits versus the risks, all believed that facebook engagement was worthwhile. although few reviewed site analytics to inform their posting and management approach, and none had formally analyzed return on investment. as one participant put it “i don’t really. the only way i evaluate the value of it is when i look at what they send you which is an update on how many potential people you have reached and i think it’s worth it just to know that you have reached that many people, that they have looked (case 4).” another noted, “the only investment is time… it is definitely a good return for the time you put into it (case 6).” a few of the participants felt that customer satisfaction was enhanced by their social media presence. some were enthusiastic about an increase in customer satisfaction, “yes i think so, because i feel like i australasian journal of information systems volume 18 number 3 2014 252 have become friends with some of my customers, that we have a better rapport because they have commented on something and i will say, ‘thanks for your feedback idea, nice to see you in store the other day, how is the new job going?’ and we just have a continued conversation (case 7).” another was more measured, “to a small degree, yes; we still do not have enough of our customers on there to really measure that (case 10).” while another did not agree, “customer satisfaction?…no i wouldn’t think so (case 12).” discussion fuchs et al.’s (2010), framework highlights the role that the decision maker’s context plays in technology adoption or readiness. the results of this research indicate that the business owner or manager, the decision maker, plays a key role in facebook adoption or readiness. however, their demographic, education and risk propensity, key attributes of a decision maker in fuchs et al.’s (2010) framework, do not appear to influence readiness. indeed all participants viewed facebook engagement as a low or no risk activity. rather, their experience with facebook, and its perceived ease of use and usefulness (davis, 1989) appeared to have a greater influence on readiness, and the decision to adopt social media. familiarity, ease of use and usefulness were cited as key motivators for adoption. however, the environmental context (fuchs, et al., 2010) does appear to be significant, an area that does not appear to be emphasised in the current social media literature. most participants cited competitive pressure from similar businesses or their customers as motivating social media adoption. the organizational context appears to be very strongly related to the decision maker’s context in the small business setting. participants found that limited it resources and skill were required to establish and maintain their facebook presence. only two had engaged in training, and while their endeavours had a larger audience and more engagement than some, others with no training had a bigger following. however, the absence of cost was a major motivator for small business adoption of facebook. the perception of reduction in marketing costs aligns with other small and medium business findings (hopkins, 2012; singh, et al., 2010) but contradicts findings from large business that indicate social media adoption and management can incur significant costs (fueller, et al., 2012; kuikka & äkkinen, 2011). however, in one case where a cost effective alternative method (text messaging) with a larger audience was available, the perception of the value and impact of facebook engagement was dramatically reduced as was the intensity of engagement. by contrast two other stores with similar facebook followings perceived their facebook presence to be beneficial. a focused approach to adopting facebook, limiting the range of tools used, publicising them both in store and on the business web site and posting on social media regularly (several times a week) seemed to achieve a desirable and manageable intensity as perceived by the majority of participants. businesses have moved beyond pure product information and imagery to personalise posts through humour, seasonal themes, events, staff and competitions showing that many businesses have adopted the varied and informal posting style espoused by kaplan and haenlein (2010). in doing so businesses achieved the megaphone facet of gallaugher and ransbotham’s (2010) 3-m framework. however, the limited customer engagement noted by participants indicates that further research into understanding how to support magnet and monitor interactions is required to realize the benefits of ewom for small business. while most participants felt that their current practice provided a competitive advantage and roi, further gains (e.g. ewom/brand ambassadors) may be achieved by implementing further engagement and socialization techniques to facilitate greater customer interaction. it appears that a critical mass of followers may not have been reached by participants, which may also impact their ability to move toward magnet and monitor (gallaugher & ransbotham, 2010) facebook engagement. further, promotion of facebook to customers to engage more of their customer base, and analysis of their efforts using facebook and sales data may also assist with meeting this objective. australasian journal of information systems volume 18 number 3 2014 253 however, the immediate nature of customer response to postings (e.g. arriving in store to purchase or ringing up after seeing a product on facebook) was also identified as an additional benefit or positive outcome by several participants. they perceived this to be an indicator that facebook engagement had a positive impact on sales, a driver of value in fuchs et al.’s (2010) framework. finally, perhaps due to the nature of their facebook use, businesses did not highlight impact on efficiency or impact on business relationships (fuchs, et al., 2010) as drivers of value. however, several participants did feel that their facebook engagement did have a positive impact on customer satisfaction, although this perception was not universal. conclusion and future work several themes that augment and extend the current literature have emerged from this study into small retail business use of facebook. the ease of use and usability of tools like facebook provide small businesses with a virtually cost free channel for advertising their business and connecting with existing and potential customers. while customer interaction via facebook is lacking for many of the small businesses studied, several perceive their facebook presence as having a positive impact on sales due to the immediate reaction they receive from postings through customers visiting their store and mentioning facebook posts. the modest customer interaction observed indicates that some small businesses may benefit from training, further advertising and analysis of facebook and sales data. such activity may enhance customer engagement through creating dialogue and building participation numbers (likes or followers) to expand the reach of their facebook presence. however, the relatively small facebook audience and control over the customer service experience that the small businesses studied have, appears to diminish the risks perceived by business in terms of their engagement with facebook. this study indicates that facebook is perceived to have many benefits and limited risks for the small retail, health and beauty businesses featured in this sample, warranting further research. future survey based research will contrast the small businesses perspectives on facebook adoption presented here, with a broader sample drawn from small businesses across australia. it is envisaged that the national survey will enhance generalizability, providing results which have relevance for small businesses across australia and beyond. references australian bureau of statistics. (2011). 8155.0 australian industry retrieved 8 june, 2011, from http://www.abs.gov.au/ausstats/abs@.nsf/lookup/8155.0glossary12009-10 australian bureau of statistics. (2012). 8165.0 counts of australian businesses, including entries and exits, jun 2007 to jun 2011 retrieved june 1, 2012, from http://www.abs.gov.au/ausstats/abs@.nsf/mediareleasesbytitle/950ec94db899312eca2573b 00017b8f4?opendocument bazley, p. (2007). qualitative data analysis with nvivo. california: sage publications inc. benbasat, i., goldstein, d. k., & mead, m. (1987). the case research strategy in studies of information systems. mis quarterly, 11(3), 369-386. brooks jr, r. c. (1957). "word-of-mouth" advertising in selling new products. journal of marketing, 22(2), 154-161. brown, j. j., & reingen, p. h. (1987). social ties and word-of-mouth referral behavior. journal of consumer research, 14(3), 350-362. australasian journal of information systems volume 18 number 3 2014 254 cao, j., knotts, t., xu, j., & chau, m. (2009). word of mouth marketing through online social networks. paper presented at the americas conference on information systems, amcis 2009 proceedings, lima, peru. cook, h. (2012). coles twitter campaign goes down, down gurgler retrieved march 7, 2012, from http://www.theage.com.au/business/coles-twitter-campaign-goes-down-down-gurgler20120307-1uj4c.html cooper, m. (2012). what a whopper: spat rages over halal burger claims retrieved june 4, 2012, from http://www.theage.com.au/entertainment/restaurants-and-bars/what-a-whopper-spat-ragesover-halal-burger-claims-20120202-1qugs.html cowling, d. (2012). social media statistics australia march 2012 retrieved june 1, 2012, from http://www.socialmedianews.com.au/social-media-statistics-australia-march-2012/ davis, f. d. (1989). perceived usefulness, perceived ease of use, and user acceptance of information technology. mis quarterly, 13(3), 319-340. derham, r., cragg, p., & morrish, s. (2011). creating value: an sme and social media. paper presented at the pacific asia conference on information systems, pacis 2011 proceedings, brisbane. eisenhardt, k. m. (1989). building theories from case study research. the academy of management review, 14(4), 532-550. fuchs, m., höpken, w., föger, a., & kunz, m. (2010). e-business readiness, intensity, and impact: an austrian destination management organization study. journal of travel research, 49(2), 165-178. doi: 10.1177/0047287509336469 fueller, j., schroll, r., dennhardt, s., & hutter, k. (2012). social brand value and the value enhancing role of social media relationships for brands. paper presented at the 2012 45th hawaii international conference on system science (hicss). gallaugher, j., & ransbotham, s. (2010). social media and customer dialog management at starbucks. mis quarterly executive, 9(4), 197-212. harris, l., & rae, a. (2009). social networks: the future of marketing for small business. journal of business strategy, 30(5), 24-31. hoffman, d. l., & fodor, m. (2010). can you measure the roi of your social media marketing? mit sloan management review, 52(1), 41-49. holmes, k., mclean, r., & green, g. (2012). crafting a future online: a study of how independent craftspeople adopt social media and web technologies. journal of systems and information technology, 14(2), 142-154. holzner, s. (2009). facebook marketing: leverage social media to grow your business indianapolis: que. hopkins, j. (2012). can facebook be an effective mechanism for generating growth and vallue in small business? journal of systems and information technology, 14(2), 131-140. jarvenpaa, s. l., & tuunainen, v. k. (2012). company tactics for customer socialization with social media technologies: finnair's rethink quality and quality hunters initiatives. paper presented at the 2012 45th hawaii international conference on system science (hicss). kaplan, a. m., & haenlein, m. (2010). users of the world, unite! the challenges and opportunities of social media. business horizons, 53(1), 59-68. doi: 10.1016/j.bushor.2009.09.003 kaske, f., kugler, m., & smolnik, s. (2012). return on investment in social media--does the hype pay off? towards an assessment of the profitability of social media in organizations. paper presented at the 2012 45th hawaii international conference on system science (hicss). australasian journal of information systems volume 18 number 3 2014 255 kuikka, m., & äkkinen, m. (2011). determining the challenges of organizational social media adoption and use. paper presented at the european conference on information systems, ecis 2011 proceedings, helsinki. miles, m. b., & huberman, a. m. (1994). qualitative data analysis: an expanded sourcebook (2nd ed.). thousand oaks: sage publications, inc. moses, a. (2011). reputation wrecking: social media alarm sounds retrieved april 5, 2011, from http://www.theage.com.au/technology/biz-tech/reputation-wrecking-social-media-alarmsounds-20110405-1d17z.html nakayama, m., wan, y., & sutcliffe, n. (2010). wom or ewom or something else: how does the web affect our dependence on shopping information sources? paper presented at the americas conference on information systems, amcis 2010 proceedings, lima, peru. national retail association. (2012). retail sector struggles with recessionary conditions retrieved 1 june, 2012, from http://www.nra.net.au/images/120229-nramediareleasejanuaryretailtrade-approved.pdf neilsen. (2012). media release nielsen australian online report 2011-12 retrieved june 6, 2012, from http://au.nielsen.com/site/documents/theaustralianonlineconsumerreport-2012info.pdf oecd. (1999). defining and measuring e-commerce: a status report (working party on indicators for the information society, trans.) (pp. 19). paris: oecd. peters, a., & salazar, d. (2010). globalization in marketing: an empirical analysis of business adoption and use of social network sites. paper presented at the americas conference on information systems, amcis 2010 proceedings, lima, peru. powell, g., groves, s., & dimos, j. (2011). roi of $ocial media : how to improve the return on your social marketing investment. singapore: john wiley & sons. rupert hills, j., & cairncross, g. (2011). small accommodation providers and ugc web sites: perceptions and practices. international journal of contemporary hospitality management, 23(1), 26-43. saundage, d., & lee, c. y. (2011). social commerce activities – a taxonomy. paper presented at the acis 2011 proceedings, sydney, australia. senadheera, v., warren, m., & leitch, s. (2011). a study into how australian banks use social media. paper presented at the pacific asia conference on information systems, pacis 2011 proceedings, brisbane. seo, d., & rietsema, a. (2010). a way to become enterprise 2.0: beyond web 2.0 tools. paper presented at the international conference on information systems, icis 2010 proceedings, saint louis, mi. singh, m., davison, c., & wickramasinghe, n. (2010). organisational use of web 2.0 technologies: an australian perspective. paper presented at the americas conference on information systems, amcis 2010 proceedings, lima, peru. yin, r. k. (2009). case study research: design and methods (vol. 5). thousand oaks: sage. zhu, k., & kraemer, k. (2005). post-adoption variations in usage and value of e-business by organizations: cross-country evidence from the retail industry. information systems research, 16(1), 61-84. an earlier version of this paper was presented at the australasian conference on information systems (acis) 2013 in melbourne, australia. paper title (paper title style) australasian journal of information systems huang, yang & chen 2018, vol 22, research article motivational factors, user experience &social networking 1 when do motivational factors lead to negative user experience on social networking applications? shin-yi huang national taiwan university of science and technology miles m. yang macquarie university miles.yang@mgsm.edu.au chien-hsiung chen national taiwan university of science and technology abstract we investigate why some innovative user interface (ui) designs (e.g., facebook home), although initially expected to encourage users to accept or adopt social networking applications (apps), actually discouraged users from using those applications over time. using two experimental studies, we explore the relationship between new design factors and user behavioural intentions. in study 1, we adopted two-factor theory to discover motivational factors related to using a social networking app. we then designed and created a virtual social networking app (called unicorn) using these identified motivational factors. in study 2, we investigated how motivational factors de-motivate user behavioural intentions. we unpacked this relationship using the technology acceptance model (tam). further, we explored how involvement level may alleviate this negative relationship. our results show that users with a high involvement level exhibit a relatively high level of perceived usefulness and perceived playfulness. therefore, involvement level can alleviate the negative relationship that exists between motivational factors and behavioural intentions. this study provides both theoretical implications to the literature and practical insights for ui designers. keywords: social networking applications, technology acceptance model, two-factor theory, involvement level, facebook 1 introduction one of the critical challenges for user interface (ui) designers has been the question of how to create a desirable ui that can both attract and retain users (chou, lai, & liu, 2013; kim, kim, & han, 2013; mouakket, 2015; park, roman, lee, & chung, 2009; wang, 2016). a new, innovative ui for social networking apps may have a better opportunity of attracting user attention and subsequently increase the rate of revisiting. however, some innovative designs fail to achieve these desired results over the long term—even though, initially, those same designs received considerable attention from users. indeed, over time, some new designs produce the opposite effect from that intended, actually discouraging users from adopting or accepting the renovated application. the facebook home app is an example of a new, disruptive and innovative design that failed to succeed in attracting users to that application due to its ui design and functionality (carlson, 2013). in 2013, taiwanese smartphone and tablet manufacturer htc allied with facebook to launch a new mobile app named facebook home on their android mobile phone, the htc first. the goal was to create a strong connection between mobile devices and facebook by providing a australasian journal of information systems huang, yang & chen 2018, vol 22, research article motivational factors, user experience &social networking 2 drop-in replacement for the existing home screen (“launcher”) on an android device (pogue, 2013). at the time, industry experts noted that facebook home’s new function in its ui design, which allows users to place their facebook friends in front of their phone’s home screen directly, results in other apps fading into the background (stern, 2013). this new function is highly appreciated, according to it experts (pogue, 2013). however, upon facebook home’s launch, the feedback from users was overwhelmingly negative, especially this new function (cheng, 2013). the app’s poor reception was evidenced by the fact that within a few hours after its release, facebook home received an average rating of 2.3 out of 5 stars on google play (grove, 2013). a business case like this provokes an interesting question: why is it that some innovative ui designs or functions counterintuitively demotivate users’ behavioural intentions? recently, researchers have begun to study why people reuse social networking services (sns) through the research lens of various perspectives such as socio-technical systems (chai & kim, 2012) and social influence theory (e.g., lin & lu, 2015; tao, 2011) (and see a review of relevant work in the next section). using two experimental studies, we investigate this question from an organisational psychology perspective, using two-factor theory, the technology acceptance model (tam), and involvement theory (il). in study 1, we adopted two-factor theory to discover motivational factors related to using a social networking app. we then designed and created a virtual social networking app using the identified motivational factors. in study 2, we investigated how motivational factors de-motivate user behavioural intentions. we advance our understanding of sns app ui designs and provide new insights into how and why some innovative designs fail to attract and retain users. 2 literature review and theoretical development our study focuses on how ui design on functionalities encourage or discourage use of sns apps. next, we discuss the relevant work, and then in the following section we introduce the theoretical background of our work by introducing two-factor theory, followed by involvement theory and tam. recently, researchers have begun to study why people reuse social network sites (snss) through the research lenses of social influence theory (lin & lu, 2015; tao, 2011), social networking theory (luarn & chiu, 2015), the technology acceptance model (rauniar, rawski, yang, & johnson, 2014), gratifications theory (wei & lu, 2014), social cognitive theory (chiang & hsiao, 2015), and innovation diffusion theory (chiang, 2013). table 1 summarises prior related studies from 2012 to 2016, showing that the use and design of snss has received considerable attention. in particular, research using gratifications theory and innovation diffusion theory has explained why people employ continuous use of snss (chiang, 2013). they found that attitude toward snss had the strongest main effect on continuous intention, while the impact of social norms has no effect on the different stages of the innovation diffusion process. however, to the best of our knowledge, there are limited studies investigating the continuous use of sns apps. our study differs from prior studies in three ways. first, we are interested in sns apps which have received little attention thus far (hacker, wickramasinghe, & durst, 2017; wang, 2016). second, in order to understand the actual impact of design factors (nguyen, johnson, & hackett, 2003), we not only conducted a traditional survey to find out the critical design factors, we also implemented those important factors in a virtual, new sns app called australasian journal of information systems huang, yang & chen 2018, vol 22, research article motivational factors, user experience &social networking 3 study (year) base-line theory target online services/samples key findings tao (2011) social influence theory data were collected from 450 students in china. all participants had to complete the survey questionnaire based on their favourite online community usage experience. the results show that both social identity and group norms have significant effects on user participation. in addition, group norms affect social identity. it was not possible to find the effect of subjective norms on participation intention. chai and kim (2012) socio-technical system data were collected from 211 snss users at a large university in the u.s. this study investigated the factors which influence knowledge contribution behaviours of sns users by sharing through user-created contents with one another. this study discussed the roles of social system factors, such as ethical culture, social ties, and a sense of belonging in online social network. further, this study examined technical systems factors, such as structural assurance of service providers and structural assurance of the internet. chiang (2013) gratification theory, innovation diffusion theory data were collected from 348 members of facebook in taiwan, using an online questionnaire. the study investigates how innovation diffusion perspective and uses and gratifications perspective influence the continuous use of snss. in particular, attitude toward snss had the strongest direct effect on continuous intention, while the impact of social norms was not significant in the different innovation diffusion stages. wei and lu (2014) network externalities theory, gratification theory data were collected from 237 students and office workers who were using bahamut (www.gamer.com.tw), gamebase (www.gamebase.com.tw), and taiwan’s game-related bulletin board systems. the results reveal that both network externalities and individual gratifications significantly influence the intention to play social games on mobile devices. time flexibility, however, which is one of the mobile device features, appears to contribute relatively little to the intention to play mobile social games. rauniar et al. (2014) technology acceptance model data were collected from 389 full-time students from two business schools in the usa. the results demonstrate that the revised social media tam model proposed in this study supports all the hypotheses of social media usage behaviour. the results of this study provide evidence for the importance of additional key variables to tam in considering user engagement on social media sites and other social-media-related business strategies. australasian journal of information systems huang, yang & chen 2018, vol 22, research article motivational factors, user experience &social networking 4 study (year) base-line theory target online services/samples key findings chiang and hsiao (2015) gratification theory, social cognitive theory data were collected from 265 students and office workers who were mostly from taiwan, with a few from hong kong and china. the results indicated that continuance motivation and sharing behaviour were important antecedents of youtube ‘stickiness’ and mediated the influence of need, personal and environmental factors. lin and lu (2015) social influence theory data were collected from 116 students and office workers who were using or had used mobile sns services in taiwan. the results revealed that mobile convenience, service compatibility, security risk and cognitive effort have a significant and indirect impact on the user acceptance of mobile snss through their respective paths on hedonic value and utility value. luarn and chiu (2015) social network theory data were collected from 145 undergraduates who had using facebook experience. the results developed an algorithm (predictive model) that quantifies and measures tie strength continuously to bridge the gap between theory and practice. the results found that the variables in the dimension of emotional intensity had stronger effects than other interaction variables. lim (2015) gratification theory the study tests 320 usable responses collected from e-shoppers against the integrated model using structural equation modelling. the findings show the significance of perceived value, social factors, perceived ease of use, perceived usefulness, entertainment gratification, web irritation, emotional state, and web atmospherics in the process of eshopping. lin and lu (2015) social influence theory data were collected from 116 students and office workers who were using or had used mobile sns services in taiwan. the results revealed that mobile convenience, service compatibility, security risk and cognitive effort have a significant and indirect impact on the user acceptance of mobile snss through their respective paths on hedonic value and utility value. table 1. selected empirical studies related to sns from 2012 to 2015 australasian journal of information systems huang, yang & chen 2018, vol 22, research article motivational factors, user experience & social networking 5 ‘unicorn’ in order to discover the actual impact of such factors. third, we examine the relationship between design factors and adoption/acceptance of sns apps using three baseline theories: herzberg’s two-factor theory, involvement theory and tam, which provides us with a solid understanding of user behaviours from an organisational psychology perspective. we selected these three baseline theories due to our summary of the current literature in table 1. particularly, it delivers three key messages. first, it shows motivation play an important role in users’ adoption/acceptance of sns apps (i.e., tao (2011), chiang (2013), chiang & hsiao (2015)). thus, we consider herzberg’s two-factor theory in our study (see section 2.1). second, recent studies such as luarn and chiu (2015), and chai and kim (2012) implicitly show that users’ perceived level of interests in using a sns app has significant impact on their adoption attitude. therefore, we consider involvement theory (see section 2.2) in developing our hypotheses. third, tam (see section 2.3) is often utilized as mediators for examining the relationship between web design and users’ behaviours. relevant studies in table1 include lim (2015), lin and lu (2015), wei and lu (2014), and rauniar et al. (2014). 2.1 two-factor theory herzberg’s (1959) two-factor theory shows that neither satisfaction nor dissatisfaction are impervious to change, and that the development of either state is influenced by two primary components: hygiene factors (hfs) and motivational factors (mfs). hfs refer to the basic elements that quantify one’s satisfaction, including aspects of the general task environment involved. when lacking hfs, individuals demonstrate high dissatisfaction. in contrast, mfs refer to the characteristics of the work itself, such as challenges and a sense of fulfilment and personal growth that stimulate individuals’ satisfaction regarding their task environment. while the presence of mfs is sufficient to motivate individuals, those factors, however, have little or no effect on one’s dissatisfaction (herzberg, 1987). this dual nature of motivation theory has attracted a great deal of attention from researchers. subsequent studies have continually supported the two-factor model (maidani, 1991; murphy & fraser, 1978; schwab & cummings, 1970; soliman, 1970; whitsett & winslow, 1967). consequently, two-factor theory promotes the idea of job redesign in order to increase the quality of work performance and individuals’ satisfaction levels (hackman & oldham, 1975; kopelman, 1986). the theory has been widely used by practitioners and managers to build mfs into the work environment (rantz, scott, & porter, 1996). further research into the two-factor theory in organisational psychology has shown that the theory can be applied to ‘motivation’ in other disciplines and research areas, such as education (poppleton, 1988), consumer motivation (maddox, 1981; tuten & august, 1998) and knowledge sharing (hendriks, 1999). in particular, the two-factor theory has been applied to understanding users’ motivation for using web services. for example, zhang and von dran (2000) argue that hfs are critical to ensuring that websites are functional, whereas mfs can contribute to users’ satisfaction levels by adding value to websites. a follow-up study by wu, chuang and chen (2008) shows that, in an application of search engine websites, while hfs (e.g., basic query process) provide general and functional operations and reliable service, mfs add value to applications of search engine websites in order to enhance consumer satisfaction. for example, search engine developers need to provide stability in a search engine as a hf to support users in their information seeking process. moreover, search engine developers can provide a ranking function (e.g., the degree that the relevance ranking of search results meets user needs) to increase user satisfaction levels by helping users navigate and comprehend the australasian journal of information systems huang, yang & chen 2018, vol 22, research article motivational factors, user experience & social networking 6 retrieved information. while the use of two-factor theory provides insights to website development (chuang & chen, 2008), online software services (lee, chae, & cho, 2013), and mobile data services (lee, shin, & lee, 2009), these insights receive less attention in the development of sns applications. following this line of research, in the present work we adopted two-factor theory to investigate users’ potential needs in using sns app uis in study 1. then we examined the effects of mfs on sns users’ behavioural intentions (bi) in study 2. prior research suggests that, when designing websites, interface designers should use hfs as prerequisites for planning mfs (zhang & von dran, 2000). the theory suggests that, if the ui design contains mfs, users will be attracted to visit a new sns. therefore, ui designers should include mfs in their design. we empirically tested this idea to advance our understanding about how to effectively design a sns app. 2.2 involvement theory krugman (1965) first introduced the construct of involvement level (il) in investigating consumer behaviour. an individual’s involvement level is based on inherent needs, values and interests that motivate one toward the object (zaichkowsky, 1994): that is, an individual’s ‘level of involvement with an object, situation, or action is determined by the degree to which they perceive that concept to be personally relevant’ (p.211). il is regarded as one of the critical factors influencing an individual’s behaviours. for example, shiau and luo (2013) conducted research on blogs and found that users’ il has a positive impact on the tendency toward continuous use of blogs. users’ il also affects their satisfaction with searching the internet for information (santosa, wei, & chan, 2005). while the use of involvement theory provides insights to website development such as blogs (e.g., santosa, wei, & chan, 2005), we contend that the interaction effect of mf and il has been less addressed. we argue that mf and il should be considered jointly for their impact of bi. while mfs can lead to an ‘internal process’ that increases an individual’s satisfaction level, il represents an ‘external process’ that is an individual’s willingness to spend more time on the objects involved. considering these two processes together, we argue that user behavioural intentions on sns apps will be influenced by mfs and il jointly through a well-established model, tam. the details of tam are discussed in the section below. in this study, we built on prior research by defining high/low sns il as being possessed by people who show a high/low level of care about snss. we expected that users with high/low il are willing to spend more/less time on the snss they attach to. consequently, the amount of time attached to snss moderates the relationship between mfs and users’ perceptions (i.e., tam) of the sns app. 2.3 technology acceptance model (tam) in this section, we first explain tam and how it relates to users’ bi. second, we explain why we expect mfs will have an impact on tam. the tam is an information systems theory that models how users come to accept and use a technology. the model suggests that, when users are presented with a new technology, a number of factors influence their decision about how and when they will use it, including perceived ease of use (peou), perceived usefulness (pu) and perceived playfulness (pp). the tam is frequently used to explain user willingness to adopt certain technologies (davis, 1989). in this model, both peou and pu affect users’ bi when one considers a website as a new piece of information technology; peou represents ease of use when users interact with an interface; australasian journal of information systems huang, yang & chen 2018, vol 22, research article motivational factors, user experience & social networking 7 pu refers to a website system’s functionality; and bi represents the individual willingness to use the website (lin & lu, 2000). of these, pu plays an important role in determining whether users accept certain information technologies (lu, zhou, & wang, 2009; sledgianowski & kulviwat, 2009; yen, wu, cheng, & huang, 2010). when it comes to snss in particular, pu has a direct impact on bi (kang & lee, 2010; kwon & wen, 2010; sledgianowski & kulviwat, 2009). another important construct in tam is peou (davis, 1989). before accepting and using any type of technological product, users must first overcome the initial barriers of ease of use. thus, peou comprises the fundamental decision factor in users’ consideration of whether they should continue to use a particular sns app (choi & chung, 2013). in their research study on the acceptance level of the world wide web (www), moon and kim (2001) adopted the pp factor and applied it to tam. they argue that technological products which are not easy to use will reduce users’ pp. at the same time, technological products with high ease of use reduce users’ sense of threat during operations. research has shown that pp has a positive impact on bi (moon & kim, 2001). thus, playfulness is a key factor affecting user intentions to visit specific websites. in other words, if users ‘feel good’ when using a system or performing a task, they are encouraged to engage more with that system or activity. lin, wu, and tsai (2005) pointed out that, when users participate in website activities with feelings of pleasure and enjoyment, their preferences for those activities increase, as does their willingness to revisit the website. in addition, sns app interface designers add many entertainment functions, in line with their goal of stimulating users’ perceived interest levels in using the app. this is done with the hope of increasing users’ pleasure in using the app and of enhancing users’ bi. in the context of a sns app, we believe mfs play a part in explaining users’ acceptance and usage of the sns app. incorporating the mfs into the tam provides better prediction of user acceptance and usage of a sns app. according to the two-factor theory, mfs exert crucial influence on users’ satisfaction level and perceptions on achievement expectations (vansteenkiste, simons, lens, sheldon, & deci, 2004). adding mfs into a sns app will stimulate users to appropriate various testing strategies to use the sns app in order to reach the increased satisfaction level and achievement expectations based on their current knowledge, skills and experience. during the learning process stimulated by mfs, users are inclined to believe that their satisfaction level is increased by using the particular sns app. then, given the time and effort they allocated to the sns app, their perceived ease-of-use will be increased accordingly. also, given the increased level of satisfaction driven by mfs, user intention to reuse the sns app increases, because they feel good when performing tasks in the sns app. based on this reasoning, we suggest that il and mfs jointly influence users’ bi through tam (i.e., pu, peou, and pp). figure 1 shows this theoretical model. australasian journal of information systems huang, yang & chen 2018, vol 22, research article motivational factors, user experience & social networking 8 figure 1. theoretical model 3 research design and methodology we conducted two experimental studies. study 1 investigated mfs, and study 2 examined the impact of these mfs using a pseudo-sns app (unicorn). we recruited participants from a large university in taiwan. student participants from taiwan were particularly suitable for this study given the fact that the population were active social media users. a research report shows that, in the fourth quarter 2016, 81% of the population were active social media users, and the most popular social network was fb, with an 83% penetration rate (statista, 2017). to be more specific, in study 1, two-factor theory is applied to unpack potential users’ demands in order to identify the hfs and mfs of a sns app’s ui. in study 2, we then examine the effect of those mfs on bi. the aim of study 2 is to determine whether the ui-integrated mfs can stimulate a user’s bi, as predicted by two-factor theory (tuch & hornbæk, 2015), or if they actually demotivate a user’s bi due to the demand for additional hfs or the inherent delay of perception or confidence in using the new ui. we conducted a 2x2 mixed-factorial experimental design, manipulating for two factors as the repeated tests: the between-subject factor (the il, both high and low) and the within-subject factor (the sns app ui, with and without mfs). we manipulate for il because users’ il with snss and their awareness of interface design are not always the same. we adopted krugman’s (1965) involvement theory to identify participants’ il with the sns. we also measured participants’ pu, peou, and pp of the sns app, using tam as the mediator. in study 1, we incorporated three main dimensions from popular interface design evaluation principles. these are: ‘visuals,’ ‘usability’ and ‘functionality.’ we used an online survey to investigate user perception of the hfs and mfs of snss. for this survey, we used a list with 41 core features (based on the literature review from zhang and von dran 2000, and wu et al., 2008) and allocated each of these to one of three categories: visuals, usability, and functionality. we asked participants to identify whether specific features were an hf or an mf. to do so, we adopted zhang and von dran’s (2000) procedure to ask participants to classify each feature to one of four possible options: hygiene (basic function: if not present, users will complain), motivation (additional function: if present, will satisfy users), unclear h/m (unclear whether basic or additional function: a “useless factor”), or unclear wording (the description is difficult for a participant to understand: a “useless factor”). we adopted these four options for australasian journal of information systems huang, yang & chen 2018, vol 22, research article motivational factors, user experience & social networking 9 participants to use in identifying whether specific features of the sns app belong to the category of hf or mf. we adopted questionnaire items from zhang and von dran (2000) and wu et al. (2008), and revised these based on the characteristics of our task (sns apps), which resulted in the items being slightly different from zhang and von dran (2000) and wu et al. (2008). in addition, we referred to the social media report 2012 (nielsen, 2012) to help us filter the top five sns apps in 2012. these were determined to be facebook, twitter, foursquare, google+, and pinterest. consequently, the questionnaire consists of 11 items in the visuals category, and 15 items each in the usability and functionality categories. these items are listed in appendix 1. then we calculated the percentage of each feature that was selected from the questionnaires. more details about how we decided hfs and mfs is in section 3.3. based on the result of the questionnaires, we can determine which item is hf and which item is mf. then, to test our theoretical model, we conducted another study. the first step was to develop a pseudo sns app. we designed a pseudo sns app which incorporated both hfs and mfs in its design. in study 2, we unpacked the causal relationship between mfs and bis in relation to sns app. employing online surveys as the experiment’s method, we manipulated for two factors: il (high and low) and mfs (with and without). the levels of each factor are described below. the sns app with mfs is the one that we added mfs to—the virtual app, which we called unicorn. the mfs introduced here were those retrieved from our study 1. in this experimental task, we used the facebook app as the sns app without any mfs. thus, a 2x2 experimental design was adopted in study 2. the participants were asked to complete five tasks (see appendix 2) to experience mfs in the use of the unicorn app. after completing those five tasks, all participants had to complete questionnaires, which included measures for both il and tam. we used the personal involvement inventory (pii) from zaichkowsky (1994) as the measure for participants’ il. to measure participants’ pu, peou, pp and bi, we used the questionnaire from moon and kim (2001), which includes four sections that measure for pu, peou, pp and bi. all items were measured using a 5-point likert scale, with anchors ranging from ‘strongly disagree’ to ‘strongly agree’ (see appendix 3). as recommended by jiang and benbasat (2007), we used a two-step test. in the first step, we employed a two-way mixed design anova to examine whether mfs moderated the users’ il, as well as their pu, peou and pp. in the second step, a multiple regression was used to test the structural model proposed on the right-hand side of figure 1. in that step, we tested whether pu, peou and pp have the power to explain the variance of bi. 3.1 study 1 – data collection procedure we distributed our online survey to 54 participants who were selected by a convenience sampling method from a large university, for a total of 46 usable survey responses over six days (from 18 to 24 april, 2013). we recruited these participants to complete the survey in order to investigate user perception of the hfs and mfs of snss. the sample consisted of an equal ratio of genders: 50% male and 50% female. participants ranged in age from 20-35 years old; 18 participants were aged 20 to 24 years old, 23 participants were aged 25 to 29 years old, and the remainder were 30 to 35 years old. a significant number of respondents (28%) reported regularly spending between two and four hours a day surfing the internet via smartphones. australasian journal of information systems huang, yang & chen 2018, vol 22, research article motivational factors, user experience & social networking 10 the primary reason reported for time spent surfing online was addiction to sns apps such as facebook, with approximately 87% of all responses citing this reason. 3.2 study 1 – analysis once we collected empirical data using the survey instrument, we could determine whether one item (see appendix 1) was a mf or hf. we used a left-tailed z test and adapted the alpha value as 0.05, arriving at a confidence level of under 95% to separately process factor tests for hfs and mfs. finally, we compared the results of these two tests and listed the final factor for each feature. for the hf test, we tested whether the hfs’ percentage was significantly greater than 0.5, which is equal to the mfs’ percentage of less than 0.5. similarly, for the mf test, we tested whether the mfs’ percentage is significantly greater than 0.5, which equals to the hfs’ percentage p-value less than 0.5. note that the zα = z0.05 = -1.645 and the decision rule was to reject alternative hypotheses if the z value was less than -1.645. 3.3 study 1 – results the results of our studies show that of those items tested eight were not classified as either being an hf or an mf after our two-factor tests had been conducted, as illustrated in table 2. these items are: item 2 (arrange layout on timelines page), item 3 (background customization), item 22 (colourful visual design), item 30 (edit photos), item 31 (save photos), item 33 (save message), item 35 (sort home by categories) and item 38 (reminder alarm). the z value of these eight items was in the twilight zone of z testing; therefore, those eight items were not identified as hf or mf. thus, we used ‘n/a’ to represent them and we did not include them in study 2 for the mf sns app (unicorn) design. in terms of mfs, the results showed no mfs to be located in the usability category, and only one mf (dynamic font) in the visuals category. however, there were four mfs in the functionality category (i.e., items 28, 36, 39 and 40, or virtual gifts, video call, my favourite, and incentive mechanisms, respectively). we found that these five mfs are prone to be considered ‘interaction functions’. this implies that users looked for the ability to have multiple interactive functions to use in connecting with their friends and peers. 3.4 study 2 – method we describe the details of how we designed and developed unicorn app in this section. the development of unicorn has four steps within three stages: implementation, deployment and testing. implementation. the unicorn app prototype (see figure 2) is designed by using the ui design software tool rightware kanzi 2.5.2. we have two implementations of this app: one was using this software’s graphical library to design unicorn’s layout and animation. the other was based on our finding from study 1 to generate a user interface-flow diagram to help us acquire a full picture of this app’s structure. the user interface-flow diagram is also called a storyboard. the purpose of using a storyboard is to enable a designer to model the relationship between major user interface elements and fundamental usability. moreover, a storyboard can put the designer in a position where he/she can validate the overall flow of app ui (user interface flow diagrams, 2017). deployment. the following are the main development tools that we used: australasian journal of information systems huang, yang & chen 2018, vol 22, research article motivational factors, user experience & social networking 11 • rightware kanzi 2.5.2. was utilized to implement a ui framework of information and run the mfs that were implemented in the application application layout, for example, the visual design; application animation, for example, the click response of ui, mfs implementation. • user interface-flow diagram was used to design each tasks’ operating processes for participators to perform this application. testing. we chose a 4.3-inch screen (i.e., htc sensation smartphone) as the test platform. the range of screen sizes among smartphones was between 3.5 and 5.5 inches in width during our experimental period. we selected 4.3 inches because this is the average screen size. a final prototype was rolled out to all 71 participating users from a large university in taiwan. they were asked to complete five tasks to experience. these tasks include ‘dynamic font’, ‘video call’, ‘my favourite’, ‘incentive mechanisms’, and ‘virtual gifts’. for example, in the video-call task, participants were asked to look for specific contacts from the friend list, and select the video call button to make the call. after finishing the call, they pressed the end button to hang up the call, and go back to home page. appendix 2 provides more details about these tasks. australasian journal of information systems huang, yang & chen 2018, vol 22, research article motivational factors, user experience & social networking 12 cat items hygiene factor test result motivation factor test result final factor freq % z h1 factor freq % z h1 factor u sa bi lit y 1. see information immediately 34 94.44 5.33 a hygiene 2 5.56 -5.33 r hygiene hygiene 2. arrange layout on timelines page 26 60.47 1.37 a hygiene 17 39.53 -1.37 a motivation n/a 3. background customization 19 61.29 1.26 a hygiene 12 38.71 -1.26 a motivation n/a 4. user guide support 30 78.95 3.57 a hygiene 8 21.05 -3.57 r hygiene hygiene 5. swipe pages 44 97.78 6.41 a hygiene 1 2.22 -6.41 r hygiene hygiene 6. the latest articles on the top 41 93.18 5.73 a hygiene 3 6.82 -5.73 r hygiene hygiene 7. go back 39 97.50 6.01 a hygiene 1 2.50 -6.01 r hygiene hygiene 8. go back home 43 95.56 6.11 a hygiene 2 4.44 -6.11 r hygiene hygiene 9. loading status icon 32 88.89 4.67 a hygiene 4 11.11 -4.67 r hygiene hygiene 10. stability while in operation 34 91.89 5.10 a hygiene 3 8.11 -5.10 r hygiene hygiene 11. multiple languages selection 28 68.29 2.34 a hygiene 13 31.71 -2.34 r hygiene hygiene 12. save information 35 83.33 4.32 a hygiene 7 16.67 -4.32 r hygiene hygiene 13. click link accurately 38 86.36 4.82 a hygiene 6 13.64 -4.82 r hygiene hygiene 14. privacy setting 43 93.48 5.90 a hygiene 3 6.52 -5.90 r hygiene hygiene 15. authority for restricting reading articles 39 86.67 4.92 a hygiene 6 13.33 -4.92 r hygiene hygiene v is ua l 16. consistency of layout 36 81.82 4.22 a hygiene 8 18.18 -4.22 r hygiene hygiene 17. consistency of icons 35 77.78 3.73 a hygiene 10 22.22 -3.73 r hygiene hygiene 18. dynamic font 1 6.25 -3.50 r motivation 15 93.75 3.50 a motivation motivation 19. consistency of font 37 92.50 5.38 a hygiene 3 7.50 -5.38 r hygiene hygiene 20. consistency of space 39 88.64 5.13 a hygiene 5 11.36 -5.13 r hygiene hygiene 21. consistency visual 39 88.64 5.13 a hygiene 5 11.36 -5.13 r hygiene hygiene 22. colourful visual design 8 42.11 -0.69 a hygiene 11 57.89 0.69 a motivation n/a 23. main functions on top of home 36 97.30 5.75 a hygiene 1 2.70 -5.75 r hygiene hygiene 24. main functions on bottom of home 23 76.67 2.92 a hygiene 7 23.33 -2.92 r hygiene hygiene 25. swipe while reading albums 30 75.00 3.16 a hygiene 10 25.00 -3.16 r hygiene hygiene 26. clear icons to identify 44 95.65 6.19 a hygiene 2 4.35 -6.19 r hygiene hygiene australasian journal of information systems huang, yang & chen 2018, vol 22, research article motivational factors, user experience & social networking 13 cat items hygiene factor test result motivation factor test result final factor freq % z h1 factor freq % z h1 factor fu nc tio na lit y 27. share location 33 78.57 3.70 a hygiene 9 21.43 -3.70 r hygiene hygiene 28. virtual gifts 3 18.75 -2.50 r motivation 13 81.25 2.50 a motivation motivation 29. photography 29 76.32 3.24 a hygiene 9 23.68 -3.24 r hygiene hygiene 30. edit photos 14 37.84 -1.48 a hygiene 23 62.16 1.48 a motivation n/a 31. save photos 28 62.22 1.64 a hygiene 17 37.78 -1.63 a motivation n/a 32. tag friends 33 78.57 3.70 a hygiene 9 21.43 -3.70 r hygiene hygiene 33. save message 13 40.63 -1.06 a hygiene 19 59.38 1.06 a motivation n/a 34. like function 40 95.24 5.86 a hygiene 2 4.76 -5.86 r hygiene hygiene 35. sort home by categories 19 59.38 1.06 a hygiene 13 40.63 -1.06 a motivation n/a 36. video call 9 28.13 -2.47 r motivation 23 71.88 2.47 a motivation motivation 37. emotional icon for texting 28 70.00 2.53 a hygiene 12 30.00 -2.53 r hygiene hygiene 38. reminder alarm 24 61.54 1.44 a hygiene 15 38.46 -1.44 a motivation n/a 39. my favourite 3 15.00 -3.13 r motivation 17 85.00 3.13 a motivation motivation 40. incentive mechanisms 1 6.25 -3.50 r motivation 15 93.75 3.50 a motivation motivation 41. search function 38 90.48 5.25 a hygiene 4 9.52 -5.25 r hygiene hygiene table 2. two-factor test result australasian journal of information systems huang, yang & chen 2018, vol 22, research article motivational factors, user experience & social networking 14 figure 2. the ui of unicorn virtual gifts video call incentive mechanisms australasian journal of information systems huang, yang & chen 2018, vol 22, research article motivational factors, user experience & social networking 15 3.5 study 2 – data collection procedure in study 2, 71 students were selected through a convenience sampling method from a large university in taiwan. each participant had to complete the experimental task (i.e., operationalise the facebook app and the unicorn app) and fill out the online survey. meanwhile, we observed participants’ behaviour on using the unicorn app by an unobtrusive naturalistic observation method when participants executed the experimental task. in order to get deep insights about participants’ perspectives and feelings about this app, we employed a short semi-structured interview after they completed the experimental task. the purpose is to gather more descriptive data on their personal experiences of the experimental task. due to this purpose, we asked the participants the following questions: how do you feel after executing the experimental task? which motivation feature does you prefer the most? what is your willingness to continue using this app (unicorn)? finally, do you have any comments/ suggestions that you would like to provide to us? we concluded with 66 valid survey responses over six days (from 16 to 21 june, 2013). there were 35 male and 31 female participants, with average ages of 20-24 years old (18.27%), 25-29 years old (23.35%) and 30-34 years old (25.38%). note that there is no overlapping in samples between study 1 and study 2. in addition, 66% reported that they spent between two and four hours a day surfing the facebook app via smartphone devices. we used the facebook app as a base because the mfs, which we developed in study 1, had not yet been implemented in the facebook app while we were conducting this study. in addition, all participants had experience in using the facebook app. therefore, the participants were able to compare the differences between the facebook app and unicorn app. 3.6 study 2 – analysis table 3 presents the correlations and descriptive statistics for all study 2 variables. in our reliability analysis (shown in appendix 4), since the cronbach’s alpha values of all the constructs—with one minor exception in the spss results—were over 0.8, we determined that all of the variables are reliable. note: *p<0.05; **p<0.01 table 3. table of descriptive statistic and correlations for pilot study variables the average il for snss was found to be 3.57 out of 5. we adopted a common practice in social science – median split – to turn the continuous variable (e.g., il) into a categorical one. essentially, the idea was to categorise one participant as high il if his/her il value was above the median, which shows an increasing trend, or categorise one as low il if his/her il value was below the median, which shows a decreasing trend. as shown in figure 3, our results variable 1.pu 2. peou 3.pp 4.bi 5.il descriptive statistics mean 3.66 3.94 3.39 3.78 3.57 median 3.67 3.89 3.33 3.33 2.10 sd 0.46 0.56 0.53 0.71 0.46 range 2.11-5.00 2.22-5.00 1.89-5.00 1.00-5.00 2.30-4.40 n 132 132 132 132 66 correlations 1.pu 2.peou 0.24** 3.pp 0.62** 0.20* 4.bi 0.57** 0.25** 0.64** 5.il 0.38** 0.13 0.48** 0.40** australasian journal of information systems huang, yang & chen 2018, vol 22, research article motivational factors, user experience & social networking 16 showed that 36 respondents fell into the category of high involvement subjects, while 30 were low il participants. figure 3. involvement level distribution based on the theoretical model shown in figure 1, we theorized il as the moderator between mfs and pu, peou and pp. therefore, we employed anova on pu, pp and peou (each alone) to examine whether il, mf and the interaction between il and mf yielded a significant effect on those three dependent variables. these test results are provided below. 3.7 anova results on pu we found that the facebook app facilitated high il participants to have a higher physical experience in relation to pu than in regard to the other tam construct variables. as the descriptive statistics summary in table 4 shows, the mean of high il participants for this variable is significantly higher than the mean of low il participants, especially for the unicorn app. the significant value of this interaction effect between il and mfs suggests that the effects of il are moderated by mfs (see table 5). specifically, il and mfs had significant effects on pu (p < 0.01 and p < 0.05, respectively). figure 4 shows the interaction effect of mfs and il on bi. according to the figure, the participants with high il perceived higher pu than those with low il in both types of sns apps (with and without mfs). interestingly, participants with high il perceived lower pu when using the unicorn app than when using facebook app, while participants with low il did not perceive the difference in pu between the unicorn and facebook apps. thus, there is a significant interaction effect between mfs and il on bi (p < 0.05). motivation factor (mf) involvement level (il) mean sd n yes low 3.46 0.30 30 high 3.67 0.47 36 total 3.57 0.41 66 no low 3.49 0.45 30 high 3.98 0.43 36 total 3.75 0.50 66 table 4. descriptive statistics table for pu source ss df ms f p-value australasian journal of information systems huang, yang & chen 2018, vol 22, research article motivational factors, user experience & social networking 17 involvement level (il) 4.02 1 4.02 19.58 0.00* motivation factor (mf) 0.94 1 0.94 0.00 0.01* il x mf 0.60 1 0.60 4.08 0.05* within-subjects 22.60 128 block (if) 13.13 64 0.21 error (mf) 9.47 64 0.15 total 28.16 131 note: *p<0.05 table 5. anova summary table for pu figure 4. interaction graph of mf and il on pu 3.8 anova results on peou we did not find a significant interaction effect of mfs and ils on peou (p = 0.82). 3.9 anova results on pp we found that there is a significant difference in pp between high il and low il (p < 0.01), while mf and il x mf had no significant effect on pp (see table 6). this result indicates that participants’ different levels of il have an impact on their sense of pp in relation to both sns apps (with and without mfs). in addition, table 7 shows that high il participants have a higher sense of pp than those with low il on both sns apps. source ss df ms f p-value involvement level (il) 8.46 1 8.46 29.64 0.00* motivation factor (mf) 0.02 1 0.02 0.14 0.71 il x mf 0.15 1 0.15 0.96 0.33 within-subjects 28.25 128 block (if) 18.26 64 0.29 error (mf) 9.99 64 0.16 total 36.87 131 note: *p<0.05 table 6. anova summary table for pp motivation factor (mf) involvement level (il) mean sd n yes low 3.13 0.41 30 high 3.57 0.49 36 australasian journal of information systems huang, yang & chen 2018, vol 22, research article motivational factors, user experience & social networking 18 total 3.37 0.50 66 no low 3.09 0.50 30 high 3.66 0.47 36 total 3.40 0.56 66 table 7. descriptive statistics table for pp during the second step, we employed a multiple regression test to help identify which variables made significant contributions to predicting users’ bi. the test results (see table 8) show that the tam had a significant effect on the dependent variable of bi (p < 0.00), for which the explained variance was 45%; r2= 0.45, f-value = 36.35, p < 0.01). these results indicate that the participants’ pu and pp had a significant effect on bi. by following this with the application of a multiple regression test on each independent variable, we found that pu and pp had a significant effect on bi, as β (coefficient) = 0.24 (t = 2.83, p < 0.01) for pu and β= 0.47 (t = 5.67, p < 0.01) for pp (see table 9). interestingly, peou is not shown to have a direct influence on bi as β= 0.10 (t = 1.51, p > 0.05). notably, the effect of pp on participants’ bi to use sns app was very strong, as shown by the path coefficient of 0.47, which was higher than the pu of 0.24 (also see figure 5). the results from this regression analysis imply that pp played an important role in influencing participants’ bi to use sns app, and that pu was the second most significant key factor. furthermore, it indicates that users’ pp and pu experiences determined whether they decided to keep using sns app or not. we also evaluated whether there was a multi-collinearity issue within our model (shown in table 9). we found that all variance inflation factors (vif) were less than 5 and the conditional index (ci) was less than 30. thus, the degree of multi-collinearity was slight in this model and could be ignored. dependence variable adjusted r2 df f-value p-value behavioural intentions (bi) 0.45 3 36.35 0.00** note: **p<0.01 table 8. the summary descriptive statistics table on the tam of behavioural intentions figure 5. path coefficient summary variable b se beta t p-value vif ci perceived usefulness (pu) 0.36 0.13 0.24 2.83 0.01** 1.67 14.48 perceived ease of use (peou) 0.13 0.09 0.10 1.51 0.14 1.07 21.04 perceived playfulness (pp) 0.63 0.11 0.47 5.67 0.00** 1.64 25.09 note: **p<0.01 r2 = 0.45 perceived usefulness (pu) perceived ease of use (peou) perceived playfulness (pp) behavioral intentions (bi) 0.24** 0.10 0.47** australasian journal of information systems huang, yang & chen 2018, vol 22, research article motivational factors, user experience & social networking 19 table 9. the summary table of regression coefficients in conclusion, based on this two-step test, we found that mfs negatively influenced users’ bi. this meant that users’ bi for revisiting the unicorn app decreased, especially for addicted sns users. however, the facebook app attracted users’ physical experience of pu and pp by enhancing their bi to revisit the app. 4 discussion in this study, we address an interesting empirical question: why is it that some innovative ui designs effectively (and counter-intuitively) demotivate users’ behavioural intentions? based on our findings, we advance the understanding of ui design in sns app accordingly. to the authors’ knowledge, this is the first empirical study to explore the effect of the mfs on users’ bi in the context of sns apps. we aim to investigate why innovative designs may demotivate user intention to adopt or accept a new sns app due to its ui design or functions. although the theory suggests that mfs can enhance user satisfaction level on a particular object, our study suggests that, in a context of sns apps, mfs may have a negative impact on pu, and consequently, on users’ bi. moreover, consistent with prior research, our results show that il has a positive impact on pp, and consequently on bi; and users’ il strengthens the negative relationship between mfs and bi through pu. this set of results advances our understanding of users’ behaviours on the acceptance or adoption of a new sns app from an organisational psychology perspective (chiang, 2013). we discuss theoretical implications below, and followed by managerial implications, limitations and future research directions. 4.1 theoretical implications the first theoretical contribution is that we investigate how to design sns apps based on organisational psychology theory. to be more specific, we applied herzberg’s two-factor theory and il theory to advance our understanding about the impact of design factors on the tam and, consequently, on bis. this line of inquiry is significantly different from prior studies in this stream of research. thus our study provides unique insights and a different perspective on the design of sns apps, based on the theories we have applied (e.g., aladwani, 2015). in addition, our finding constitutes a novel and intriguing contribution to two-factor theory. while scholars and practitioners from motivational theory suggest that mfs could stimulate users’ visiting intention due to an increase of satisfaction level, we argue that mfs could decrease a user’s behaviour intention in accepting or adopting a new or renovated sns app. for example, in data drawn from interviews and a survey, aladwani (2015) argued that both hfs and mfs facilitate the use of twitter. however, unlike prior researchers, who have approached the facilitators of the use of sns apps from an inductive perspective, we favour the deductive approach (e.g., experiments), because it provides a better opportunity for building a generalizable theory of the ui design of sns apps that is not simply a reflection of characteristics idiosyncratic to just a single sns context (e.g., twitter). we believe that taking the deductive approach allows for the development of finer-grained models than those which can be derived from inductive approaches (wiseman, cuevas-rodríguez, & gomez-mejia, 2012). our study makes a unique contribution to two-factor theory by extending its application to the adoption or acceptance of a new sns app as a unique context, while prior applications of the theory focus on job satisfaction in workplaces (herzberg, 1959; herzberg, 1987). australasian journal of information systems huang, yang & chen 2018, vol 22, research article motivational factors, user experience & social networking 20 the second contribution is our finding that different design elements have unequal not equal importance. we found that usability, visuals and functionality, which are assumed to have equal importance in the literature, actually have varying levels of importance in sns app. according to study 1, we argue that elements in usability are hfs, whereas most of the elements in visuals and functionality are hfs, but five are comprised of mfs: ‘dynamic font’, ‘virtual gifts’, ‘video call’, ‘my favourite’, and ‘incentive mechanisms’. to further explore the findings in study 2, we also observed how participants completed tasks 1 to 5 and then conducted a short interview to explore how they felt about their experience of the unicorn app. based on the reciprocal evidence received, we suggest two reasons to explain our findings. first, there might be a significant time delay between understanding the interface and using the interface with confidence, especially for heavy users of sns apps. one participant from study 2 reported that it was difficult to keep his attention on the new design, because he wanted to be able to perform multiple tasks at the same time. in other words, he was expecting to be able to complete some tasks faster, but he was hindered by the delay involved in using the new design. as a result, his pu significantly decreased. thus, designers should consider this delay and how to make it shorter when planning future designs. second, in two-factor theory, some scholars argue that, before a mf starts to perform as expected, the mechanism first needs to satisfy the need for hfs such as base payment. therefore, there is a chance that the baseline app did not completely satisfy user demands regarding hfs. hence, the advanced app, which incorporates mfs, did not succeed. for this reason, ui designers should pay extra attention to meeting users’ basic demands before adding any mfs. 4.2 managerial implications this study also has important practical implications. first, our results provide clear guidance to ui designers regarding whether they should continue investing time and effort to add mfs into the design of sns apps. while the theory suggests that mfs can enhance people’s satisfaction level on particular objects, our results show that in a context of sns apps mfs may have a negative impact users’ bi. thus, ui designers should be careful when developing the ui for a sns app with a mf. second, ui designers should also consider users’ il when developing the ui for a sns app. while users with high il exhibit high bi in accepting or adopting a new sns app, users’ il further enhances the negative relationship between mfs and bi. these findings provide important guidelines to ui designers. specifically, ui designers can improve users’ bi in a new sns app by appropriately adding mfs when users show a low il. 4.3 limitations and future research directions four limitations of the studies reported here are worth noting, as they point to directions for future research. first, although we designed a new, pseudo-app in study 2, we acknowledge the possibility that some participants may have felt familiar or unfamiliar with the newly developed app. future research should consider perceived familiarity as a potential moderator as an extension of the study. second, regarding the pu, peou and pp, participants may have different impacts on their bi depending on the genres of the app. in other words, participants or users may have different priorities for each type of app, when they make the decision to adopt and continue using an app. for example, game or entertainment, news or information, education, commerce, and health care applications may have different utility and appealing aspects. future research is needed to examine the impact of priority and value on users’ adoption decisions. australasian journal of information systems huang, yang & chen 2018, vol 22, research article motivational factors, user experience & social networking 21 third, in this study we use students as the study sample. a student sample was used because it is relatively homogenous, which is important to experiment research (calder, phillips, & tybout, 1981). students subscribe to and use internet services often. therefore this population may better fit the purpose of the study (pizzutti dos santos & basso, 2012). however, a student sample somewhat restricts the generalizability of the results for a broader population. our study also used non-random and relatively small samples from a large taiwanese university. there is a concern regarding the sampling biases in this study. future research can enlarge the sample size with a random sample, and expand the sample demographics to users of varying ages, working experience and backgrounds to further validate and replicate the findings from this study. fourth, in this study we mainly investigate how mfs and il affect users’ bi through tam. further research should consider other factors possibly influencing users’ bi. future research may investigate different forms of enabling factors to improve users’ pu, peou and pp. one possibility is to consider the enabling factors suggested in the recent literature, such as web atmospherics and entertainment gratification (lim, 2015). fifth, in this study we only examined the mfs jointly, but did not consider the potential effect of each individual mf can have on users’ behaviours (nguyen et al., 2003). future research can extend this study by investigating both the individual effect and joint effect of mfs on users’ perceptions and behaviour on sns apps. for example, we may expect that there is a difference on the effect of users’ bi between an sns app’s functionality and its outlook design. 5 conclusion using data collected from two experimental studies, this study examines how mfs and il independently and jointly influence users’ bi on accepting or adopting a new or renovated sns app through the perception of ease of use, playfulness and usefulness (i.e., tam). our study shows that mfs have negative impact on users’ bi through pu, but users’ il has a positive impact on users’ bi through pp. further, users’ il enhances the negative relationship between mfs and bi through pu. future studies are needed to extend this research by examining other cognitive factors, across other study samples, and different genres of sns apps. references aladwani, a. m. (2015). facilitators, characteristics, and impacts of twitter use: theoretical analysis and empirical illustration. international journal of information management, 35(1), 15-25. carlson, n. (2013). facebook's new mobile product is a huge flop because it was built by iphone users. business insider. retrieved 12 december 2017, from http://www.businessinsider.com/facebook-made-a-simple-huge-mistake-buildinghome-which-looks-like-a-bigger-flop-by-the-second-2013-5?ir=t calder, b. j., phillips, l. w., & tybout, a. m. (1981). designing research for application. journal of consumer research, 8(2), 197-207. chai, s., & kim, m. (2012). a socio-technical approach to knowledge contribution behavior: an empirical investigation of social networking sites users. international journal of information management, 32(2), 118-126. australasian journal of information systems huang, yang & chen 2018, vol 22, research article motivational factors, user experience & social networking 22 cheng, r. (2013). here's why the facebook phone flopped: cnet. retrieved may 5, 2013, from http://www.cnet.com/news/heres-why-the-facebook-phone-flopped/ chiang, h. (2013). continuous usage of social networking sites: the effect of innovation and gratification attributes. online information review, 37(6), 851-871. chiang, h., & hsiao, k. (2015). youtube stickiness: the needs, personal, and environment perspective. internet research, 25(1), 85-106. choi, g., & chung, h. (2013). applying the technology acceptance model to social networking sites (sns): impact of subjective norm and social capital on the acceptance of sns. international journal of human-computer interaction, 29(10), 619-628. chou, w. h., lai, y. t., & liu, k. h. (2013). user requirements of social media for the elderly: a case study in taiwan. behaviour & information technology, 32(9), 920-937. davis, f. (1989). perceived usefulness, perceived ease of use, and user acceptance of information technology. mis quarterly, 13(3), 319-340. grove, v. (2013). facebook home arrives in google play store: cnet. retrieved april 12, 2015, from http://www.cnet.com/news/facebook-home-arrives-in-google-play-store/ hackman, j. r. & oldham, g. r. (1975). development of the job diagnostic survey. journal of applied psychology, 60(2), 159-170. hacker, j., wickramasinghe, n., & durst, c. (2017). can health 2.0 address critical healthcare challenges? insights from the case of how online social networks can assist in combatting the obesity epidemic. australasian journal of information systems, 21, 1-17. hendriks, p. (1999). why share knowledge? the influence of ict on the motivation for knowledge sharing. knowledge and process management, 6(2), 91-100. herzberg, f. (1959). the motivation to work. new jersey: wiley. herzberg, f. (1987). one more time: how do you motivate employees? harvard business review, 65(5), 5-16. jiang, z., & benbasat, i. (2007). the effects of presentation formats and task complexity on online consumers' product understanding. mis quarterly, 31(3), 475-500. kang, y., & lee, h. (2010). understanding the role of an it artifact in online service continuance: an extended perspective of user satisfaction. computers in human behavior, 26(3), 353-364. kopelman, r. e. (1986). a point of view. global business and organizational excellence, 5(3), 209. kim, s., kim, h., & han, s. (2013). a development of learning widget on m-learning and elearning environments. behaviour & information technology, 32(2), 190-202. krugman, h. (1965). the impact of television advertising: learning without involvement. public opinion quarterly, 29(3), 349-356. kwon, o., & wen, y. (2010). an empirical study of the factors affecting social network service use. computers in human behavior, 26(2), 254-263. lee, s. g., chae, s. h., & cho, k. m. (2013). drivers and inhibitors of saas adoption in korea. international journal of information management, 33(3), 429-440. australasian journal of information systems huang, yang & chen 2018, vol 22, research article motivational factors, user experience & social networking 23 lee, s., shin, b., & lee, h. g. (2009). understanding post-adoption usage of mobile data services: the role of supplier-side variables. journal of the association for information systems, 10(12), 860-888. lim, w. m. (2015). antecedents and consequences of e-shopping: an integrated model. internet research, 25(2), 184-217. lin, c., wu, s., & tsai, r. (2005). integrating perceived playfulness into expectationconfirmation model for web portal context. information & management, 42(5), 683-693. lin, j., & lu, h. (2000). towards an understanding of the behavioral intention to use a website. international journal of information management, 20(3), 197-208. lin, k., & lu, h. (2015). predicting mobile social network acceptance based on mobile value and social influence. internet research, 25(1), 107-130. lu, y., zhou, t., & wang, b. (2009). exploring chinese users' acceptance of instant messaging using the theory of planned behavior, the technology acceptance model, and the flow theory. computers in human behavior, 25(1), 29-39. luarn, p., & chiu, y. (2015). key variables to predict tie strength on social network sites. internet research, 25(2), 218-238. maidani, e. a. (1991). comparative study of herzberg’s two-factor theory of job satisfaction among public and private sectors. public personnel management, 20(4), 441-448. maddox, r. n. (1981). two-factor theory and consumer satisfaction: replication and extension. journal of consumer research, 8(1), 97-102. moon, j., & kim, y. (2001). extending the tam for a world-wide-web context. information & management, 38(4), 217-230. mouakket, s. (2015). factors influencing continuance intention to use social network sites: the facebook case. computers in human behavior, 53, 102-110. murphy, g.c. & fraser, b.j. (1978). intuitive-theoretical scales of content and context satisfaction. personnel psychology, 31(3), 485-494. nguyen, h. q., johnson, b., & hackett, m. (2003). testing applications on the web: test planning for mobile and internet-based systems paperback. john wiley & sons, uk, london. nielsen. (2012). social media report 2012: social media comes of age. nielsen report. retrieved april 20, 2015, from http://www.nielsen.com/us/en/insights/news/2012/social-mediareport-2012-social-media-comes-of-age.html park, n., roman, r., lee, s., & chung, j. e. (2009). user acceptance of a digital library system in developing countries: an application of the technology acceptance model. international journal of information management, 29(3), 196-209. pizzutti dos santos, c., & basso, k. (2012). price unfairness: the indirect effect on switching and negative word-of-mouth. journal of product & brand management, 21(7), 547-557. pogue, d. (2013). facebook grabs for your phone. what gives? the new york times. retrieved april 9, 2015, from http://www.nytimes.com/2013/04/11/technology/personaltech/facebooksgrab-for-your-phone-what-gives.html?_r=0 australasian journal of information systems huang, yang & chen 2018, vol 22, research article motivational factors, user experience & social networking 24 poppleton, p. (1998). teacher professional satisfaction: its implications for secondary education and teacher education. cambridge journal of education, 18(1), 5-16. rantz, m. j., scott, j., & porter, r. (1996). employee motivation: new perspectives of the ageold challenge of work motivation. nursing forum, 31(3), 29-36. rauniar, r., rawski, g., yang, j., & johnson, b. (2014). technology acceptance model (tam) and social media usage: an empirical study on facebook. journal of enterprise information management, 27(1), 6-30. santosa, p., wei, k., & chan, h. (2005). user involvement and user satisfaction with information-seeking activity. european journal of information systems, 14(4), 361-370. schwab, d. p., & cummings, l. l. (1970). theories of performance and satisfaction: a review. industrial relations a journal of economy and society, 9(4), 408-430. shiau, w., & luo, m. (2013). continuance intention of blog users: the impact of perceived enjoyment, habit, user involvement and blog time. behavior & information technology, 32(6), 570-583. sledgianowski, d., & kulviwat, s. (2009). using social network sites: the effects of playfulness, critical mass and trust in a hedonic context. journal of computer information systems, 49(4), 74-83. soliman, h. m. (1970). motivation-hygiene theory of job attitudes: an empirical investigation and attempt to reconcile both the one and two-factor theories of job attitudes. journal of applied psychology, 54, 452-461. statista. (2017). penetration of leading social networks in taiwan as of 4th quarter 2016. retrieved december 12, 2017, from https://www.statista.com/statistics/295611/taiwansocial-network-penetration/ stern, j . (2013). facebook home and htc first review: a nice place to visit, but not quite home, abc news . retrieved april 9, 2013, from http://abcnews.go.com/technology/facebook-home-htc-review-facebooktakes-face-phone/story?id=18916810 tao, z. (2011). understanding online community user participation: a social influence perspective. internet research, 21(1), 67-81. tuten, t.l. & august, r.a. (1998). understanding consumer satisfaction in services settings:a bidimensional model of service strategies. journal of social behavior and personality, 13(3), 553. tuch, a. n., & hornbæk, k. (2015). does herzberg's notion of hygienes and motivators apply to user experience? acm transactions on computer-human interaction (tochi), 22(4), 16 user interface flow diagrams (ui storyboards) (2017). an agile introduction, retrieved december 8, 2017, from http://agilemodeling.com/artifacts/uiflowdiagram.htm vansteenkiste, m., simons, j., lens, w., sheldon, k. m., & deci, e. l. (2004). motivating learning, performance, and persistence: the synergistic effects of intrinsic goal contents and autonomy-supportive contexts. journal of personality and social psychology, 87(2), 246260. australasian journal of information systems huang, yang & chen 2018, vol 22, research article motivational factors, user experience & social networking 25 wang, x. (2016). people’s motivation to participate in social network sites, subsequent behaviours, and situation self-awareness following a crisis: evidence from the mh370 flight incident. australasian journal of information systems, 20, 1-22. wei, p., & lu, h. (2014). why do people play mobile social games? an examination of network externalities and of uses and gratifications. internet research, 24(3), 313-331. wiseman, r. m., cuevas-rodríguez, g., & gomez-mejia, l. r. (2012). towards a social theory of agency. journal of management studies, 49(1), 202-222. whitsett, d. a. & winslow, e. k. (1967). an analysis of studies critical of the motivator-hygiene theory. personnel psychology, 20(4), 391-416. wu, l. l., chuang, y. l., & chen, p. y. (2008). motivation for using search engines: a twofactor model, journal of the american society for information science and technology, 59(11), 1829-1840. yen, d., wu, c., cheng, f., & huang, y. (2010). determinants of users' intention to adopt wireless technology: an empirical study by integrating ttf with tam. computers in human behavior, 26(5), 906-915. zaichkowsky, j. (1994). research notes: the personal involvement inventory: reduction, revision, and application to advertising. journal of advertising, 23(4), 59-70. zhang, p., & von dran, g. (2000). satisfiers and dissatisfiers: a two-factor model for website design and evaluation. journal of the american society for information science and technology, 51(14), 1253-1268. australasian journal of information systems huang, yang & chen 2018, vol 22, research article motivational factors, user experience & social networking 26 appendix 1. study 1 question lists category questionnaires name u sability 1. the information can be seen immediately see information immediately 2. users can arrange the layout of information context on their timelines page. (e.g. based on the updating status, best nearby, the friends nearby me) arrange layout on timelines page 3. users can customize their background pages background customization 4. have a user guide while first using the application user guide support 5. swipe pages smoothly swipe pages 6. the latest article could be placed on the top of timeline the latest articles on the top 7. go back the previous page go back 8. go back to home page go back home 9. have a loading status icon loading status icon 10. the system is stable while operation (e.g. click the link button will not jump back to home without warning) stability while in operation 11. have diverse languages version (e.g. chinese, english, japanese) multiple languages selection 12. the information could be saved on the timeline save information 13. have a link function for user to click accurately click link accurately 14. privacy setting function privacy setting 15. have permission to restrict the reading authority based on groups authority for restricting reading articles v isual 16. have a consistency design layout consistency of layout 17. have a consistency icon design consistency of icons 18. dynamic font (e.g. twinkle text) dynamic font 19. have a consistency font for wording consistency of font 20. have the same space consistency of space 21. have a consistency visual style consistency visual 22. have a colourful visual design colourful visual design 23. the main function buttons are designed on the top of home page main functions on top of home 24. the main function buttons are designed on the bottom of home page main functions on bottom of home 25. the interface of albums can be swiped for reading swipe while reading albums 26. have clear icons to identify (e.g., camera icon means photography or take a picture) clear icons to identify functionality 27. share location share location 28. giving virtual gifts virtual gifts 29. photography photography 30. edit photos edit photos 31. save photos (e.g., users can save the favourite photos to their folders) save photos 32. tag friends tag friends 33. save favourite message save message 34. like, +1 function like function 35. the home page can be sorted by diverse categories (e.g. food, night market, coffee) sort home by categories 36. have a video call video call 37. have emotional icons while typing text emotional icon for texting 38. activities remind for birthday parties or events reminder alarm 39. my favourite my favourite 40. incentive mechanisms (e.g. have a visual reward while sharing location) incentive mechanisms 41. search function search function australasian journal of information systems huang, yang & chen 2018, vol 22, research article motivational factors, user experience & social networking 27 appendix 2. task lists function task description purpose dynamic font type ‘great’ then click dynamic font button, select the colour as red, and add a food photo before posting. finally, go back to home page. to investigate users’ visual feeling with dynamic font interface. video call looking for andy liu and judy lu from friend lists first then select the video call button. after finishing video call, press the end button to hang up the phone. finally, go back to home page. to investigate if the video call could enhance users’ intention to revisit the sns app. my favourite scroll up and down to review friends’ posts and find a post from wang hsiao ming. add this post into my favourite. then go back to home page to find the post which you just saved it, then share this post to your friends as ben yang, candy and linda. finally, go back to home page. to extend the function of to do list idea for investigating whether the multiple interaction functions could simulate users’ behavioural intention to revisit the sns app. incentive mechanisms click ‘share location’ button to share the current location as the northeast coast coffee shop. follow by typing ‘great’ and pick up a coffee photography from phone library before posting. when finish posting, the sns app system will send a trophy to you, then click this trophy directly to link to your personal award record page. finally, go back to home page. to investigate if users can be simulated by sending reward message for improving their feeling of pp for enhancing their willing to share information and post articles with their peers. virtual gifts scroll up and down for reading friends’ posts and press ‘awesome’ button for luca wu’s post, and leave a message as ‘great’ then select a gift as a cake to her. finally, go back to the home page. to enhance the usefulness function for users for increasing their bi for revisiting this sns app. note: the above questioners all includes four options for participates to choose, they are basic function (if not have, will complain), additional function (if have, will satisfy), unclear basic or additional function and unclear wording. australasian journal of information systems huang, yang & chen 2018, vol 22, research article motivational factors, user experience & social networking 28 appendix 3. study 2 question lists involvement level question lists questions: 1. facebook is important to me 2. it is interesting to use facebook 3. all social activities on facebook are relevant to my life 4. i am excited about using facebook 5. facebook means a lot to me 6. i think facebook appealing to me 7. i feel fascinating with facebook 8. the relationship with my friends on facebook is valuable 9. i am involving on facebook and would like to share my status with friends 10. i need facebook tam question lists1 questions: 1. using this social networking app improves my relationship quality with my friends 2. using this social networking app improves the performance of my social life 3. using this social networking app helps me to improve the critical part of interaction with my friends in physical life 4. using this social networking app improves my efficiency in sharing information and connecting with others 5. using this social networking app enables me to acquire more information and know more people 6. using this social networking app enables me to have more useful information 7. using this social networking app enables me to access a lot of updated information from my friends 8. using this social networking app enables me to access a lot of the latest information 9. using this social networking app enables me to acquire high quality information 10. it will be impossible to use this social networking app without expert help 11. learning to operate this social networking app is easy for me 12. it is difficult to learn how to use this social networking app 13. i find it easy to get this social networking app to do what i want it to do 14. it takes too long a time to learn how to use this social networking app 15. it is easy to remember how to use this social networking app 16. using this social networking app requires a lot of mental effort 1 we first asked participants' perception about the use of fb app, then after participants completed the tasks with unicorn app, we ask their perception about the use of unicorn app. australasian journal of information systems huang, yang & chen 2018, vol 22, research article motivational factors, user experience & social networking 29 17. my interaction with this social networking app is clear and understandable 18. it is easy for me to become skillful at using this social networking app 19. when interacting with this social networking app, i do not realize the time elapsed 20. when interacting with this social networking app, i am not aware of any noise 21. when interacting with this social networking app, i often forget the work i must do 22. using this social networking app gives enjoyment to me for my task 23. using this social networking app gives fun to me for my task 24. using this social networking app keeps me happy with my task 25. using this social networking app stimulates my curiosity 26. using this social networking app leads to my exploration 27. using this social networking app arouses my imagination 28. i will use this social networking app on a regular basis in the future 29. i will frequently use this social networking app in the future 30. i will strongly recommend others to use this social networking app australasian journal of information systems huang, yang & chen 2018, vol 22, research article motivational factors, user experience & social networking 30 appendix 4. reliability analysis category item cronbach’s alpha mf social networking app pu 9 0.80 peou 9 0.91 pp 9 0.85 bi 3 0.89 none mf social networking app pu 9 0.85 peou 9 0.88 pp 9 0.88 bi 3 0.87 the involvement level of sns 10 0.86 copyright: © 2018 huang, yang & chen. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 literature review and theoretical development 2.1 two-factor theory 2.2 involvement theory 2.3 technology acceptance model (tam) 3 research design and methodology 3.1 study 1 – data collection procedure 3.2 study 1 – analysis 3.3 study 1 – results 3.4 study 2 – method 3.5 study 2 – data collection procedure 3.6 study 2 – analysis 3.7 anova results on pu 3.8 anova results on peou 3.9 anova results on pp 4 discussion 4.1 theoretical implications 4.2 managerial implications 4.3 limitations and future research directions 5 conclusion paper title (paper title style) australasian journal of information systems sun & stirling 2017, vol 21, selected papers from ausdm event group based classification framework 1 an event group based classification framework for multivariate sequential data chao sun the university of sydney chao.sun@sydney.edu.au david stirling university of wollongong abstract decision tree algorithms were not traditionally considered for sequential data classification, mostly because feature generation needs to be integrated with the modelling procedure in order to avoid a localisation problem. this paper presents an event group based classification (egbc) framework that utilises an x-of-n (xon) decision tree algorithm to avoid the feature generation issue during the classification on sequential data. in this method, features are generated independently based on the characteristics of the sequential data. subsequently an xon decision tree is utilised to select and aggregate useful features from various temporal and other dimensions (as event groups) for optimised classification. this leads the egbc framework to be adaptive to sequential data of differing dimensions, robust to missing data and accommodating to either numeric or nominal data types. the comparatively improved outcomes from applying this method are demonstrated on two distinct areas – a text based language identification task, as well as a honeybee dance behaviour classification problem. a further motivating industrial problem – hot metal temperature prediction, is further considered with the egbc framework in order to address significant real-world demands. keywords: multi-variate time series; symbolic data mining; pattern search; sax motifs; xof-n decision trees 1 introduction time series data mining (tsdm) is a challenging task, which has attracted enormous attention in the recent years. tsdm is of particular interest in many industry areas where improvements and knowledge extensions are expected from utilising data science technology over massive existing operational records. the research work described in this paper was initiated and motivated by the real industrial needs from the iron-making industry, where operational data was continuously collected from a blast furnace (bf) for a number of years. through building predictive models on the bf data, the authors have identified a major problem that the data attributes, due to the variation of type or the spatial difference at collection, are shifted along the time axis with different extent. as the results may occur and be detect prior to the detection of the causes, the inter-variable granger causality relationship (granger 1969) are broken. the situation in which attributes are not aligned in time is in fact a quite common problem in tsdm, however this is often overlooked in practice. temporal misalignment is the name we attribute to this scenario where time delays exist between the generation and detection of various data attribution, and therefore break the causal-resultant relationship between input and output attributes. the time shifts on differing attributes are generally not uniform; these in turn make synchronisation very hard if not possible. for example, figure 1 illustrates a typical example of temporal misalignment when lightning and thunder are detected at different times with different time spans due to the speed difference between light and sound. australasian journal of information systems sun & stirling 2017, vol 21, selected papers from ausdm event group based classification framework 2 figure 1: temporal misalignment in the lightning detection. the heights of the cloud signs (on y-axis) represent the distances between the lightning events and the observer. and the x-axis represents the time when flashes and the thunder are detected. assuming the flashes are detected almost immediately, the delays of the associated thunder (marked with the same colour) are determined by the associated distance. therefore, proximally close flash and thunder pairs may in-fact not be associated to each other, due to the occurrence of temporal misalignment. in this example, when multiple lightning events (marked as dark/light green and red clouds) occur at different distances (y-axis) from the observer, the corresponding flashes and thunder (marked by the same colours as the source clouds) may be received in a mixed ordering that are different than the ordering of the source lightning. without prior knowledge of the distance of each lightning event, the time shift as a function of distance cannot be recovered, therefore it is impossible to correctly associate the corresponding flash and thunder observations. the lightning example illustrates that even in such a simple system with only two attributes, without some awareness of additional contextual information, such as the distance, height and direction between the actual lightning and the observer, correctly synchronising the light and sound signals is very difficult. due to the complex and stochastic nature of the industrial processes in the bf, it is near impossible to analyse this huge industrial plant using the fixed attribute-value model data mining techniques. this paper proposes a new approach, the event group based classification (egbc) framework in order to address this temporal misalignment issue. in the remainder of this paper, section 2 provides background and relevant research work. section 3 focuses on the methodology and experimental settings. section 4 describes a number of experimental results by applying the egbc framework to various time series classification tasks and compares the outcomes with other existing techniques. lastly, in section 5 we discuss the advantages of this approach and the possibility to further develop this as a universal, time series data mining framework for solving other real-world problems. 2 background and literature review a bf, as illustrated in figure 2, is a massive industrial plant that functions as both a chemical reactor and a heat exchanger continuously for many years. iron oxide pellets, or sintered ore, along with coke and limestone, are charged at the top into layers, which gradually over some eight hours decent in the blast furnace undergoing a series of chemical reductions to produce molten iron. australasian journal of information systems sun & stirling 2017, vol 21, selected papers from ausdm event group based classification framework 3 figure 2: a diagram of the blast furnace in the iron-making industry. the bf is a typical non-stationary system; almost all attributes in the bf dataset are subject to uncertain time shifts. for instance, domain experts explain that newly produced molten iron may require one to two hours before it is sufficiently tapped out of the bf. other production related data, such as the contents of the exhaust gases, metallurgy lab test results and the quantity of materials fed from the top of the bf, are generally acquired at a variety of sampling rates, such as minutes, possibly even hour or up to 8 hours in relation to the actual chemical and metallurgical processes. large variances in time shifts between these different attributes lead to the temperature of the actual tapped iron, namely the hot metal temperature (hmt), being contextually related to a range of process attributes variously distributed backwards in time. various researchers have previously attempted to utilise artificial neural networks (anns), because of their computational efficiency and little to no requirement of prior domain knowledge (rumelhart et al. 1988), for modelling the complex inter-variable relationships, such as predicting the silicon content of molten iron (banks 1999, sarma 2000). bhattacharjee et al. (1999) proposed traditional feed-forward neural networks to predict a number of quality parameters of the molten iron, including a categorised hmt. this work indicated that the various multi-layer perception networks were able to predict daily hmt trends utilizing only 15 inputs from the blast furnace. the extreme learning machine (elm) algorithm was also used for generating dynamic modelling for a predictive model that targeted the silicon content level in the molten iron (zhou et al. 2015). in addition to ann algorithms, other techniques have also been evaluated in the iron-making industry for various purposes. kommenda et al. (2011) used an unguided symbolic regression approach for variable selection and knowledge extraction from bf dataset. in this work, genetic programming (gp) (koza 1992) was executed multiple times for reducing the stochastic effects and for identifying important variables through a variable interaction network. in a recent work proposed by harvey and gheribi (2014), a direct search algorithm called mads was combined with the classical thermodynamics modelling of for optimising the process parameters of a bf. promising outcomes were produced in the bf simulation, and the model indicated improved precision in subsequent quantitative analytic task, such as the degradation of the bf refractory materials and liquid metal quality control. a kalman filter and minimum description length (mdl) algorithm was also employed by waller et al. in a australasian journal of information systems sun & stirling 2017, vol 21, selected papers from ausdm event group based classification framework 4 series of works (2000, 2002), in order to improve the linear arma and fir models for predicting the silicon content in the hot metal produced by a bf. in considering general tsdm work, much of the assessed research focuses on producing appropriate time series (ts) representations in order to retain the order of the data. however, these representations, considered as ts features, are further analysed by a limited number of simpler methods. some tsdm approaches rely on the indexability of their features (shieh and keogh 2008, agrawal et al. 1993), whereas others focused on similarity based features (möllerlevet et al. 2003, morrill 1998). beyond the level of representations, orders among nominal features can be represented by allen’s interval algebra (allen 1983), in which 13 basic relations between two intervals are defined, or various extensions of allen’s definition (freksa 1992, roddick & mooney 2005). however, with numerous potential ordering types between intervals, the problem becomes significantly more complex in a real time series dataset with tens or hundreds of representative features. inference of the cause-effect relationships is commonly based on the granger causality (granger 1969, 1980), which evokes two fundamental principles: (1) the effect does not precede its cause in time; (2) the casual series contains unique information about the series being caused that is not available otherwise. although granger causality was initially proposed for solving economics problems, it has been widely utilized for mining time series data. qiu et al. (2012) used granger graphical models to compute the correlation anomaly of each variable in various industrial time series. mohammad and nishida (2012) utilised the granger causality for discovering the casual structure of interesting recurring events in multi-dimensional time series data. the same method was also used for determining the size of sliding windows and feature selection in multivariate time series with lagged values (sun et al. 2015). episode mining (mannila et al. 1995) is a specialised approach for analysing temporal event data with numerous event combinations that occur within a given window. this technique can be extended to general time series data if nominal ts features are defined as events. the concept of an episode emphasises together with the importance of combinations of various events, that this conforms to the manner in which humans often perceive temporal events (batyrshin & sheremetov 2008). despite this extensive research on episode mining considers the discovery of frequent serial episodes as the fundamental problem (mannila et al. 1997, laxman et al. 2007), as these frequent episodes are believed to have a higher importance than infrequent episodes. however, in the real world, this is not always true. for example, a combination that repeatedly occurs may not be interesting to the observer. a similar situation arises with decision tree modelling. typically, every node in a tree represents a conditional test out of many others which divides the data into segments with a purer mixture of classes at each descending level. therefore, the meaningfulness of a condition node is measured by the information gain obtained in dividing the dataset. if all event combinations in an event sequence are labelled, then the meaningfulness of these can be measured in a similar way. in this paper, a new sequential data classification approach is presented where the meaningful event combinations are the key factors. the meaningfulness of a combination is established by an xon (x-of-n) decision tree (zheng 2000) rather than by the statistical measure, therefore this approach is based on information theory. multiple event groups are constructed and selected in order to form compact xon features that are used in the tree model. an xon representation contains special features that cover multiple possible combinations of given conditions. this is formed by two parts, x and n, and each is a non-empty unsorted set. the n component consists of several traditional conditions (ts events in this case), and the x component is a list of non-negative integers, denoting the exact numbers of conditions in n to be satisfied. australasian journal of information systems sun & stirling 2017, vol 21, selected papers from ausdm event group based classification framework 5 figure 3: a full expansion of an xon feature. a typical xon feature can be decomposed into a number of combinations of various unordered events, as seen in figure 3. the simple xon representation covers seven different situations, where the a, b, c and d denote conditions that may occur concurrently in the dataset. although the xon algorithm was not developed for time series analysis, we understand there are similarities between the decomposed event combinations and episodes. as an integrated feature, an xon node practically gathers a series of episodes and combines their classifying capacities in order to obtain a better performance. 3 methodology the method is designed to process sequential data with either numeric or nominal values, and is based on an idea that any real-world event can be observed and identified as a combination of other observable “events” over various temporal attributes. the word “event” here is defined as any distinguishable feature derived locally from a sequential data, such that, it may refer to temporal shapes, different value zones, transformations of the real data, symbol segments or any other features that reflect the characteristics of any temporal segments. whilst not all contextual features are tied to targeted real-world events, some feature combinations across various attributes however, may be exclusively associated with such an event. this is similar to the manner in which a doctor may diagnose patients: one symptom alone may not be sufficient evidence of a certain disease, however a set of relevant symptoms could align the diagnosis to a specific disease with considerable confidence. in this paper, an egbc framework, the “events” hold an equivalent meaning to the “symptoms” in a medical diagnosing system. the basic assumption can be described as follows: if a (detectable) process occurs at some time point t, its causal factors or resultant responses can be represented as time series features and in turn transformed into nominal events that occur around the time point t. the novelty of this work is that all casual and consequent events within a given time frame are indiscriminately analysed as unsorted combinations, and the type of the target process can be classified based on these combinations, so that the potential localised temporal misalignments are eliminated within the given time frame. therefore, in situations where specific classes are clearly labelled on each stage of such sequences, the goal is to build a model that classifies these segments based on certain important event combinations they contain. most time series data are not originally presented in the form of nominal events, thus an appropriate transformation is required to represent the time series in a nominal event form. event groups are extracted with a sliding window over the sequence, and meaningful combinations of these are automatically constructed and selected by an adapted xon algorithm. the flow chart in figure 4 briefly explains the overall procedure with a simplified example. assuming there are three different working stages in a process1, and the task is then set to 1 for instance, the hmt may be at higher, lower or normal states from the expected temperature level during different periods, and these periods can be labelled as three classes (high, low, normal) which are associated to different bf working stages respectively. australasian journal of information systems sun & stirling 2017, vol 21, selected papers from ausdm event group based classification framework 6 identify these stages (classes) from two associated time series ts-a and ts-b. the numeric time series are then transformed into two event sequences respectively in which synchronisation is not compulsory, named seq-a and seq-b, and events are labelled as an and bn accordingly (n being an event numbers). event groups are then formed across both sequences by selecting all events that fall within a sliding window, and labelled by the classes as the training dataset. the xon decision tree model is then trained in order to find the optimised event combinations for classifying the various process stages. during this procedure, an understanding of the expected event combinations and their correct labelling are essential for verifying whether this approach is effectively finding useful combinations and classifying the data accurately. figure 4: the procedure of event group based ts classification approach. in order to evaluate the feasibility of our approach, we employed our event-group based approach for two different sequential data classification tasks in the following experiment section, described as follows: 1. language identification. the goal of this task is to recognise the languages from three articles written in different languages. our approach is used to identify what language a word is written in without any dictionary. this task demonstrates how our approach performs on real event based sequences (non-time series) rather than artificially generated data. the outcomes are compared with the text mining algorithm – “textcat” (hornik et al. 2013). 2. honeybee dance behaviour recognition. in order to identify the dancing behaviours of a honey bee, genuine honeybee motion data are studied by employing our event group based approach. the sequential real-valued bee motion data are transformed and clustered into events before being analysed by the xon decision trees. this task extends the online classification work to a real-world scenario, involving multi-dimensional time series data. 3. hot metal temperature prediction. the aim of this task is to model the blast furnace process based on the abundant historical operational data in order to predict if the future hmt falls outside a normal range. multiple ts attributes are included in the dataset with various connotations and sampling rates, among which the temporal misalignment issue exist. the events in the bf data are based on the basic shapes, and ensemble of xon models are generated and updated regularly. extract e ven t g ro u p s fro m seq u en ce xon decision tree algorithm [2]-of-[a2,b2] [1]-of[a6,b4,b5] class 2 class 3 [0]-of-[b3] class 1 class 2 time series data with different stages transformed temporal event sequences temporal event groups with class labels decision tree model showing important event combinations similarity clustering symbolic time series to temporal events segmenting australasian journal of information systems sun & stirling 2017, vol 21, selected papers from ausdm event group based classification framework 7 4 experiment and results in this section, the method proposed in section 3 will initially be applied on two sequential classification tasks, and subsequently for modelling a more significantly complex industrial problem. a language identification (lid task) is conducted in order to model and distinguish three different languages (english, italian and dutch) without the benefit of any prior linguistic or dictionary based knowledge. with naturally occurring events and groups within the text, i.e. letters and words, the aim of this task is to verify the feasibility of our event group based methodology. in the second, the honey bee dancing task, we evaluate the method on realvalued time series. in both tasks, online classification along continuous sequences is accomplished, i.e. different classes in different states, and in a progressive manner in terms of the complexity. finally, the evaluated full egbc framework is utilised for modelling the ironmaking dataset in order to predict abnormal states of the blast furnace. 4.1 text mining – language identification the automatic identification of a language from text is an important and well-studied problem, which was considered as a solved problem (mcnamee 2005). lid is a typical multi-class classification task, however, in the past, decision tree techniques were not considered in this domain. the general lid methods are based on either statistical language modelling or the frequency of common word usage. both the statistical and frequency of common words usage methods work better on sentences that consist of more than 15 words (mcnamee 2005). the accuracy of various traditional lid methods approaches some 98% if the decision is based on sentences or short paragraphs (takcı & soğukpınar 2004). for the general lid task, identifying the language of single words is not practically necessary, because it is not common to mix different languages within individual sentences and paragraphs. however, in this work, in order to evaluate the proposed event group based classification approach, all articles are analysed and classified word by word. 4.1.1 dataset six public domain e-books in the three specific languages are obtained from the project gutenberg website (project gutenberg. n.d.), two in each language2. in order to ensure that only letter combinations are used as a valid basis for the classification process, all symbols and non-english characters are removed. words with three or fewer letters are ignored and the texts are pre-processed with the porter stemming algorithm (porter 1980) to reduce the word form variances for better information retrieval. the sizes of the training and testing datasets are 118,192 words (eng: 31,387; ita: 50,282; dut: 36,523) and 76,143 words (eng: 25,644; ita: 34,068; dut: 16,431) respectively. because words are the basic meaningful elements in languages, the event groups are naturally defined as single words. as the spelling is sensitive to the order of characters, in order to partially retain ordering information, every pair of adjacent characters in a word are defined as events rather than individual letters. preliminary experiments indicate that the xon models using 2-letter events have a noticeable improvement (5%) on classification accuracy compared to models using 1-letter events, if the same modelling parameters are used. 4.1.2 experiment setting and results following the pseudocode in zheng’s paper, the xon algorithm was implemented using python 2.7. in order to check the stability of the method, a number of xon models were generated based on a random 60% of the training text, and all performed similarly in terms of the amount 2 training texts:
 english: http://www.gutenberg.org/files/1999/1999.txt italian: http://www.gutenberg.org/cache/epub/1012/pg1012.txt dutch: http://www.gutenberg.org/files/18066/18066-8.txt testing texts:
 english: http://www.gutenberg.org/ebooks/43230 italian: http://www.gutenberg.org/ebooks/43226 dutch: http://www.gutenberg.org/ebooks/11500 australasian journal of information systems sun & stirling 2017, vol 21, selected papers from ausdm event group based classification framework 8 of pruning used and their accuracy. in the following experiments, the model with the highest classification accuracy and moderate pruning was chosen for further analysis. in this experiment, the xon tree model is not forced to make a decision on every event group (word), as some words may exist in different languages, and these cross-class groups are not classifiable. for the aim of either, classifying languages or obtaining linguistic knowledge, the classification of every word is not necessary. in fact, ignoring some common words is seen to help the model to focus on the more important linguistic characteristics. in order to avoid vague decisions, in this experiment, a minimum confidence parameter is used to control the output behaviour of the xon model. when the confidence from a decision is lower than a given threshold “minconf”, the classifier output is altered to “unknown” rather than the most likely class. introducing this “unknown” class and a minimum confidence reduces the number of overall mis-classifications and increases the overall classification accuracy. another tree pruning parameter is the general concept of “mincase”, minimum number of instances in a node before stopping further splitting. figure 5 illustrates how the accuracies and unknown ratios (the number of unknown cases divided by the size of the test dataset) are affected when the model is being pruned with various mincase and minconf. figure 5: classification performance on models with different mincase and minconf. a) accuracy; b) unknown ratio. in figure 5, as the minimum confidence varies from 0% to 90%, the classification accuracy on the testing text increases. however, a high confidence requirement also renders the model as ineffective when the tree is heavily pruned, and the unknown ratio becomes too high. it is reasonable to assume that an optimal model could be expected to maintain a relatively high accuracy whilst keeping the unknown ratio below an acceptable level. the yellow line (minconf = 50%) in figure 5(a) reaches its peak (accuracy = 89.6%) when the value of mincase is 40, and the unknown ratio for the same model is 9.8% when classifying the testing dataset. considering the highest possible accuracy model has 93.79% accuracy and a 34.09% unknown ratio, the yellow line model is considered to be an optimised tree model for the task. note that the training and testing texts are sourced from different e-books with different authors, styles and expressions. it is also expected that this approach would also improve if the training and testing data were sourced from the same article or author context. 4.1.3 performance comparison text-cat (van noord n.d.), a lid implementation of the n-gram-based text categorization (cavnar et al. 1994), is applied on the testing dataset as a comparison. the text-cat is limited to the selected three languages in this experiment although it supports up to 69 different languages. text-cat may make multiple decisions on a word, and these are post-processed to either “unknown” or a false decision. for example, if an english word is identified as “english australasian journal of information systems sun & stirling 2017, vol 21, selected papers from ausdm event group based classification framework 9 or italian”, then the result is converted to an “unknown”; and if an english word is identified as “italian or dutch”, because both are incorrect, the first incorrect decision will be kept. table 1: identification on testing texts with xon and text-cat models. based on different pruning options, the same fully grown xon tree model has various sizes (node number), accuracies and unknown ratios, listed as model 1, 2 and 3 respectively. table 1: identification on testing texts with xon and text-cat models. based on different pruning options, the same fully grown xon tree model has various sizes (node number), accuracies and unknown ratios, listed as model 1, 2 and 3 respectively. the text-cat models used in this work are publicly available (van noord n.d.), and each language model file contains 400 high frequency terms. a comparison between text-cat and the xon tree models (with three different pruning options) can be viewed in table 1. the first xon decision tree model is lightly pruned, containing some 1222 nodes and achieves 89.6% word-by-word classification accuracy on the testing texts (9.8% unknown words of the testing texts). as a comparison, the text-cat has 81.7% in accuracy on the same testing text (32.4% unknown words of the testing texts). confusion matrices expressed in percentages, are also included in table 1 for easy comparison between all models. one advantage of the xon decision tree model is that it can be easily pruned to suit different requirements. for instance, if the accuracy is of higher priority than the unknown ratio, the model can be pruned as model 2 in table 1, which provides a similar unknown ratio to what text-cat provides, but with a significantly higher accuracy at 91.9%. however, if a smaller, less complex form is preferred, the xon model can also be further pruned for less nodes. with a mincase=460 and minconf=50%, such as model 3 which contains only 292 nodes yet also produces a similar unknown ratio (32.6%) compared to text-cat. this model still provides a marginally higher identification accuracy at 83% on the test dataset. figure 6: word-by-word language identification by xon and text-cat models. font colours indicate true class, background colours indicate identified class. meaning of colours: blue-english, red-italian, green-dutch, black-unknown. (a,b,c) results from xon model; (d,e,f) results from textcat model. table 1: ident ificat ion on test ing texts with xon and text -cat models xon model 1 xon model 2 xon model 3 cat model model mincase= 40 minconf= 50% mincase= 40 minconf= 98% mincase= 460 minconf= 50% or iginal size 1222 nodes 1342 nodes 292 nodes 1200 terms accuracy 89.6% 91.9% 83% 81.7% unknown 9.9% 25% 32.6% 32.4% confusion matrix (%) 0 b @ pr ed ! e ng i ta d ut e ng 88.9 7.3 3.8 i ta 6.5 90.9 2.6 d ut 6.6 5.0 88.4 1 c a 0 b @ pr ed ! e ng i ta d ut e ng 90.1 6.5 3.4 i ta 5.0 93.5 1.5 d ut 5.5 3.8 90.7 1 c a 0 b @ pr ed ! e ng i ta d ut e ng 70.4 23.4 6.2 i ta 6.6 89.6 3.8 d ut 9.3 5.9 84.8 1 c a 0 b @ pr ed ! e ng i ta d ut e ng 60.1 19.4 20.5 i ta 6.9 89.1 4.0 d ut 6.1 1.8 92.1 1 c a (a) (b) (c) (d) (e) ( f ) figure 4: word-by-word language ident ificat ion by xon and text -cat models. font colours indicate true class, background colours indicate ident ified class. meaning of colours: blue-english, red-italian, greendutch, black-unknown. (a,b,c) results from xon model; (d,e,f ) results from text -cat model. clustering methods (zhou et al. 2013). in this sect ion, the same classificat ion task is performed to show how our event based decision tree method performs on this same dataset . 3.2.1 d at a pr e-pr ocessing in order to t ransform thereal-valued mot ion data into nominal events, we define pat terns of movements and consequent ial movements at any given t ime point t as the event for describing the dancing behaviours. for example, it is safe to assume when the behaviour is left -turn, the bee’s posit ion moves towards the left relat ive to the previous t rajectory. for the waggle behaviour, the bee may move both left and right in an alternat ing manner. therefore, the basic movement pat terns for this task are closely associated with the direct ions of movements. the sequences of raw posit ions and head angles are t ransformed into the following six features: • x-o↵ : the x o↵set of current posit ion relat ive to the previous frame. • y-o↵ : the y o↵set of current posit ion relat ive to the previous frame. • cosang: cosine of the head angle. • sinang: sine of the head angle. • angdi↵ : the di↵erence between head angle and previous t rajectory. • dist : the distance the bee travels since last frame. in the above list , the cosang and sinang features can be considered as an est imat ion of the x-o↵ and y-o↵ for the following moment , if the bee maintains its current heading direct ion and speed. the angdi↵ indicates the di↵erence between the current heading and that of the previous movement . from every frame, a vector of all six features is viewed as an unique mot ion status of the dancing bee, and every status contains informat ion derived from both the current and previous frame. if every such mot ion status is t reated as a nominal event , there will be a massive number of events. the curse of dimensionality will a↵ect our decision t ree algorithm that would seek to find the opt imised combinat ions. the feature vectors are therefore preclustered using a minimum message length (mml) algorithm(wallace & boulton 1968). subsequent ly, each cluster is t reated asan event represent ing a series of similar mot ion states. the mml clustering also reduces thesix dimensional featuresequences into a single dimensional event sequence with a limited number of events. 3.2.2 par amet er det ai ls unlike the textual lid event sequences, the honeybee dancing events have uniform t ime intervals, therefore a fixed number of events are selected with a fixedwidth sliding window. the behavioural label of an event group is determined by the majority of the ground t ruth within that window. in order to obtain an opt imised model reflect ing the inherent t ruth within the honey bee dancing mot ion data, a series of t raining processes were executed with various australasian journal of information systems sun & stirling 2017, vol 21, selected papers from ausdm event group based classification framework 10 the actual xon tree models in table 1 are too large to be included in this paper. figure 6 illustrates a number of the classification results where three sections of texts and their classification results are visually presented. in figure 6, the english, italian and dutch are printed in blue, red and green fonts. if a word is mis-classified, the background colour of that word changes to the colour of the wrong decision accordingly. further, when an unknown identification is made, the word is presented in black. 4.2 honeybee dancing behaviour classification in this task, classification on genuine, real-valued time series data is involved. the honey bee dancing dataset (oh et al. 2008) includes six videos of honeybee dancing, where the trajectory of a signalling bee is automatically tracked and converted into quantitative sequential motion data, including sequences of position (x and y) and the head angle of the bee. this dataset contains ground truth labels of all data records according to which behaviour of three possible behavioural patterns it contains: waggle, right-turn or left-turn. a number of researchers have previously attempted to classify the type of dancing behaviour based on these motion features, using extended hmm, segmentation (fox et al. 2008) and clustering methods (zhou et al. 2013). in this section, the same classification task is performed to show how our event based decision tree method performs on this same dataset. 4.2.1 data pre-processing in order to transform the real-valued motion data into nominal events, we define patterns of movements and consequential movements at any given time point t as the event for describing the dancing behaviours. for example, it is safe to assume when the behaviour is left-turn, the bee’s position moves towards the left relative to the previous trajectory. for the waggle behaviour, the bee may move both left and right in an alternating manner. therefore, the basic movement patterns for this task are closely associated with the directions of movements. the sequences of raw positions and head angles are transformed into the following six features:  x-off: the x offset of current position relative to the previous frame.  y-off: the y offset of current position relative to the previous frame.  cosang: cosine of the head angle.
  sinang: sine of the head angle.
  angdiff: the difference between head angle and previous trajectory.  dist: the distance the bee travels since last frame. in the above list, the cosang and sinang features can be considered as an estimation of the xoff and y-off for the following moment, if the bee maintains its current heading direction and speed. the angdiff indicates the difference between the current heading and that of the previous movement. from every frame, a vector of all six features is viewed as a unique motion status of the dancing bee, and every status contains information derived from both the current and previous frame.
 if every such motion status is treated as a nominal event, there will be a massive number of events. the curse of dimensionality will affect our decision tree algorithm that would seek to find the optimised combinations. the feature vectors are therefore pre-clustered using a minimum message length (mml) algorithm (wallace & boulton 1968). subsequently, each cluster is treated as an event representing a series of similar motion states. the mml clustering also reduces the six-dimensional feature sequences into a single dimensional event sequence with a limited number of events. 4.2.2 parameter details unlike the textual lid event sequences, the honeybee dancing events have uniform time intervals, therefore a fixed number of events are selected with a fixed-width
sliding window. the behavioural label of an event group is determined by the majority of the ground truth australasian journal of information systems sun & stirling 2017, vol 21, selected papers from ausdm event group based classification framework 11 within that window. in order to obtain an optimised model reflecting the inherent truth within the honey bee dancing motion data, a series of training processes were executed with various parameters, such as different initialisations, training datasets, numbers of mml clusters, sliding window sizes and the minimum case numbers required before splitting a node. the parameters used in this experiment are listed below:  number of mml clusters: this parameter was pre-set to be 8, 10 or 12 clusters.  sliding window size (winsize): 5,7,9,11. these are also the sizes of event groups, i.e. the number of continuous video frames used for classifying a behaviour. only an odd number of events are used for simple majority labelling.  xon seed: 100, 200, 300, 400. this seed controls all random functions in the xon training procedure, such as initialisation, dataset division etc.  training ratio: 70% of the sequential data are used for training process and the rest are for testing purpose. because the continuity is important in the time series data, the testing data is always sourced from a continuous section of the sequence.  maximum x (max-x): varies from the number four (4) up to the number (winsize-1). because the event group size is winsize, any x greater than winsize is meaningless, therefore the maximum integer in the x part is limited by the number (winsize -1).  minimum case number for splitting a node (mincase): 2,5,8,10,15,20,40,70. these are traditional parameter values used in many decision trees, and indicates the minimum size of data before a node can be further split, to extend the model. 4.2.3 model selection the training process of the xon algorithm exhaustively uses all parameters listed in previous section, and the models are evaluated on all three datasets (training, testing and whole) for error rates.
 id cno seed winsize max-x mincas e slideacc u errdiff 1 12 400 11 5 15 85.3% 0.4% 2 8 300 11 8 5 93.0% 0.7% 3 8 100 9 7 20 87.04% 9.5% 4 14 400 11 5 10 90.62% 7.7% 5 16 200 11 8 15 88.92% 11.5% 6 14 200 9 5 15 88.17% 12.1% table 2: selected models and sliding prediction accuracy. the classification error rates are calculated through two techniques: a raw prediction error rate and a sliding prediction error rate. the raw error rate is the case by case error rate showing the accuracy of event group classification. however, because each frame of data is contained in multiple event groups, the classification on a single frame should also be further determined using the majority of classifications it receives as the sliding window passes. the sliding error rate in general is about 5-10% better than the raw prediction error rate, thus all error rates or accuracies in the rest of this section are based on the sliding method. in most cases, a fully grown decision tree model would have higher classification accuracy on the training data than on the testing data, this is of course a symptom of over training as the model performs worse on previously unseen data. pruning with larger values of mincase normally reduces this performance difference, however it also lowers the overall accuracy. in order to select an optimised tree model from all the trained models, the models are ascendingly sorted based on a score = allerr + abs(trainerr − testerr). a good model is expected to have a low score indicating that both the overall error rate and the performance difference between testing and training datasets are low. australasian journal of information systems sun & stirling 2017, vol 21, selected papers from ausdm event group based classification framework 12 4.2.4 experimental results figure 7: comparison with classification result from other researches; where the motions are waggle (green), right-turn (red), left-turn (blue). (a) – (f) show outcomes for 6 individual honeybee motion sequences. the exhaustive training process selects the following model parameters for classifying each honeybee dancing data as shown in table 2.
table 2 illustrates that the best fully grown xon tree models discovered during the exhaustive training process. due to the limited number of models that were generated, these are not reflective of the best performance our approach could achieve. even though, the performances of these models are comparable and even exceed several of other techniques which were employed for the same task.
zhou et al. (zhou et al. 2013) summarised a table to compare some state of the art techniques for classifying the honeybee dancing dataset, and the accuracies are further compared with our method in figure 7. 4.2.5 discussion the accuracy comparison in table 3 indicates that selected event group based decision tree models have comparable classification performance to other existing techniques. our method outperforms all other techniques in three of the six sequences, and it also has the highest average accuracy in all of the compared methods. australasian journal of information systems sun & stirling 2017, vol 21, selected papers from ausdm event group based classification framework 13 table 3: classification accuracy comparison with other techniques, all numbers in percentage. bold fonts stand for the best accuracy on predicting the full sequences among methods. the last row includes the accuracies of the xon-mml approach. figure 8(a) illustrates the mml cluster pre-processing utilised in the no.2 honeybee dancing dataset model as an example. figure 8(a) illustrates the common bee motion states as eight mml clusters, where the red arrows represent the movements from the previous frame, contrastingly, the blue arrows represent the current head direction. figure 8(b) also includes histograms that indicate how the signatures or profiles of these eight mml cluster (events) associate with the three behaviour categories. a number of key facets can be understood in figure 8(b): firstly, the mml cluster events are a biased distribution across the three behaviours, which indicates the possibility to identify the honeybee behaviours based on these mml events. secondly, the distribution histograms of “left-turn” and “right-turn” are similar to each other, therefore judging the type of behaviour with a single cluster/event is not feasible. for example, when new motion data is identified as cluster 0, it has little chance of it being a “waggle”, however the chance for being either a “leftturn” or “right-turn” are both very high. considering that the final xon model provides an accuracy of some 93% on classifying sequence-2, this provides evidence of the effectiveness of this extended decision tree algorithm. australasian journal of information systems sun & stirling 2017, vol 21, selected papers from ausdm event group based classification framework 14 figure 8: (a) presentation of mml clusters and (b) the histogram of clusters on each behaviour. the very similar histograms of leftand right-turn indicate that the individual mml clusters are not the distinguishing features for these motion behaviours, therefore, the reason behind the successful classification lies in utilising the localised feature combination with the egbc method. 4.3 hot metal temperature prediction as stated in section 1, this research was motivated by a demanding industrial problem, i.e. to utilise data mining techniques to model and predict the iron-making process. in this section, the egbc framework is employed for making predictions of the hmt, in order to evaluate how this method performs in a stochastic industrial environment. 4.3.1 problem definition the hmt temperature of molten iron, is one of the most important indicators of product quality in the iron making process. however, due to the complexity of the associated non-linear australasian journal of information systems sun & stirling 2017, vol 21, selected papers from ausdm event group based classification framework 15 processes and systems, predicting and stabilising this temperature is extremely difficult using standard statistical techniques or even mass-transfer equations (biswas 1981). the hmt is recognised as one of the most important indicators because it is directly associated with the product quality (banks 1999), which is expected to be stabilised at an ideal value of 1,500°c with a tolerable ±20°c variance. therefore, three classes, “high”, “norm” and “low”, are used to label every ‘cast’ (the process of tapping the molten iron) in the bf operational data. the bf is continuously monitored by various sensors that are installed all over its structure. these sensors report a variety of operational data, such as the wall temperatures (via a complete array of water cooled blocks or staves), internal pressures near the wall, the contents of exhaust gases at the top, the blast temperature, plus the volume and humidity etc. other information such as the rate of fuel, the volume or mass of raw feed materials being charged, laboratory test results and real-time simulation based indices are also collected as part of the monitoring log. the data was collected regularly from each source, and the complete bf operational data was finally logged as a multi-dimensional time series. as the egbc framework is employed here for predicting the possible changes of future hmt based on current and historical observations, the associated parameter settings (similar from the previous sections) will not be repeated in details in the remainder of this section. 4.3.2 data cleansing and event generation the attributes in the original bf dataset were collected at differing sampling rates, some manually measured attributes are also entered at irregular time intervals. within many data mining methods, this is often considered as a missing data scenario for the low frequency attributes. however, within the egbc framework, events are constructed based on individual attributes, according to the temporal characteristic of each attribute, customised methods can be designed for feature generation from attributes with specific constrains, therefore the interpolation is not necessary on the original dataset because of this flexibility. although the temporal misalignment among attributes is inevitable, the egbc framework is designed to accommodate such attributes with known time shifts, these are approximately aligned with other attributes in order to reduce the number of irrelevant events being included. by doing so, the grouping window for this task was consequently set at three hours, instead of more than eight hours as in the raw data. due to a lack of specific knowledge on all of the individual attributes, events in this case were selected to be the intuitively shape-based. an extended symbolic aggregate approximation (sax) algorithm, variance-wise sax (sun et al. 2013), is utilised for dynamically converting all attributes in to sax motifs. within every training period, the mean and standard deviation values are recalculated based on the up-to-date history so that the equiprobable zones reflect the true historical information. variance-wise sax produces varying numbers of motifs within a selected period. active attributes are automatically transformed into more sax motifs with finer details of the sequence, and on the contrary, less active attributes incur fewer motifs that abstract larger time spans with fewer details. the sax motifs are further converted into events using a k-medoids clustering algorithm. in each training period, all sax motifs from each individual attribute are gathered and clustered into a fixed number of clusters, and these clusters are in-turn used as the shape-based events. because the number of event combinations generated during the xon training process is exponential to the number of attributes and the number of individual features per attribute, and the capacity and performance of the test platform and the number of attributes in the bf dataset (over 30), the number of clusters for each attribute is limited to seven or nine clusters per attribute, such that the overall number of events are under control in the following xon modelling stage. australasian journal of information systems sun & stirling 2017, vol 21, selected papers from ausdm event group based classification framework 16 4.3.3 ensemble learning with xon models the xon decision tree classifier becomes the central modelling function. however, because of the stochastic nature of the bf, it is unlikely that any individual model would fully describe the problem and produce reliable predictions. in order to improve the overall performance, multiple best performing base-xon models are employed for building an ensemble model to vote for a final prediction of hmt in the test dataset. additionally, as the process modality of the bf changes over time, models are retrained periodically by sliding through the whole ten months of data (in the step of 7 days) and segmenting these into 36 overlapped sets, each includes 40 days of training data and subsequently utilising the following 7 days as test data. from each of these data segments, 48 xon trees are generated by adjusting such parameters as: groupwin, clusterno etc. every tree is further expanded into 15 variations by altering the pruning parameters. seven of the best performing historical models, from a prior (ts) segment plus two newly trained models form a pool of models in each new data segment – subsequent predictions are then made over the test data (remaining 7 days of a data segment) for evaluation. all xon trees were fully grown and pruned back differently, as the following (subsequent) testing data is not available before the training finished, the newly added models are then selected based on evaluations over the training dataset, thus over-fitting is expected. however, over-fitted models can also be useful for ensemble models (sollich and krogh 1996). further, all other models were evaluated and selected (and deselected) based on the historical testing data, therefore the possible occurrence of over-fitting during the training process is controlled and contained. 4.3.4 outcomes and comparison because the bf data is an imbalanced 3-class dataset, in order to objectively evaluate the performance, the adjusted geometric mean (agm) (batuwita & palade 2012) is selected as the main metric for evaluating the prediction performance. the agm is an extension of the gm (geometric mean) metric, which in turn is calculated from the four basic statistical measures (true positive (tp), false positive (fp), true negative (tn) and false negative (fn)) with the following formulas: 𝐴𝐺𝑀 = { 𝐺𝑀 + 𝑇𝑁𝑟𝑎𝑡𝑒 × (𝐹𝑃 + 𝑇𝑁) 1 + 𝐹𝑃 + 𝑇𝑁 , 𝑖𝑓 𝑇𝑃𝑟𝑎𝑡𝑒 > 0 0, 𝑖𝑓 𝑇𝑃𝑟𝑎𝑡𝑒 = 0 𝑊ℎ𝑒𝑟𝑒, 𝑇𝑁𝑟𝑎𝑡𝑒 = 𝑇𝑁 𝐹𝑃 + 𝑇𝑁 𝑇𝑃𝑟𝑎𝑡𝑒 = 𝑇𝑃 𝑇𝑃 + 𝐹𝑁 𝐺𝑀 = √𝑇𝑃𝑟𝑎𝑡𝑒 × 𝑇𝑁𝑟𝑎𝑡𝑒 the average agm of the two abnormal hmt classes are listed in table 4, together with a number of other performance indicators such as the true positive rates (sensitivity) for both ‘low’ and ‘high’ classes, and the overall prediction accuracy in all bf test periods. in general, for the low-class there are less hits compared to the other classes, and the sensitivities of lowclass are often zero or other small values. however, because the low-class cases are often rare in the various test datasets, this lack of hits on several occasions amongst some two hundred cases is somewhat understandable. in the total 36 periods, the average test data size is approximately 170 cases per period. however, within these, there are some 26 periods that contain five or less instances of the lowclass, and the sensitivities of low-class in these periods are all zero except period-22. on the other hand, in the seven periods with more than ten low-class cases, the low-class sensitivities are 63.4, 91.7, 90.9, 21.4, 60, 38.7 and 0 (%) respectively. as for the other abnormal hmt class (high), the xon tree models successfully predicted more than 60% of these high-class cases in half of the test periods, and only seven periods have sensitivities of less than 50%. australasian journal of information systems sun & stirling 2017, vol 21, selected papers from ausdm event group based classification framework 17 table 4: summarised hmt prediction performances of the egbc framework over all test periods. the avg agm column shows the average agm (adjusted geometric mean of the true cases) on both abnormal classes (low and high), which is selected as the indicator of the prediction performance. columns se-* list the sensitivities of the predicting outcomes on all three classes, followed by the number of cases belonging to the given class and the number of total cases in the corresponding testing period. the last column shows the overall predicting accuracy on the test datasets, which is less significant than the sensitivities on abnormal classes. in addition to the prediction summary table, the prediction outcomes are visualised and three examples are given as in figure 9. by examination of these visualisations, it is obvious that the egbc models raise a number of false alarms. however, when and before a real abnormal hmt class happens, the ensemble model generally corrects this within a small-time proximity, thus such visual results may still be useful for bf control in reality. australasian journal of information systems sun & stirling 2017, vol 21, selected papers from ausdm event group based classification framework 18 figure 9: a few visual examples of hmt predictions at period 4, 10 and 26. the bars at the bottom of pictures indicate real and predicted classes. upper bars stand for real hmt classes, and lower bars are the outcome of predictions. pink colour means the class is high and cyan colour means low class. the line plot in the middle represent the measured hmt records, where each valid hmt sample is marked with ‘x’. different colours indicate different casts from the bf in a test period. x-axis represents days; y-axis represents temperatures (°c). by comparing the prediction of egbc framework to a previous study (sun et al. 2011) on the same bf dataset employing c5.0 and cubist decision tree models, the egbc framework shows significant improvements. although the overall prediction accuracies of the c5.0 approach are occasionally comparable or even better in certain cases than the egbc approach, it is achieved by sacrificing the success or ‘hit’ rate for both abnormal hmt classes. in most of the data segments, the c5.0 predictions have almost zero sensitivity on the “low” class, and the hit rate on the “high” class is also much poorer. australasian journal of information systems sun & stirling 2017, vol 21, selected papers from ausdm event group based classification framework 19 5 conclusion and future work this paper presents a novel method to utilise an xon decision tree technique for classifying generic streams of data sequences. provided that numeric time series data can always be transformed into a sequence of nominal events with an appropriate abstraction method, the xon approach can be readily expanded to other time series data mining areas. the language identification test indicates that the xon based approach has similar or better performance than the traditional lid algorithm, especially when classification is made on isolated words. the success on the honeybee behaviour classification demonstrates that our approach has the capacity to model and classify genuine real-valued time series data, with appropriate transformation of temporal events, and its stable performance, which is significant when compared with other techniques. however, the temporal misalignment does not exist in the two tasks above. in order to evaluate the merit of event group classification, plus to validate that it actually works as expected when dealing with the misalignment issue, real-world industrial data encompassing severe time shifting problems was studied. here, the outcomes indicate a significant improvement in correctly predicting abnormal process states or modalities. there are a number of advantages for the methodology presented in this paper. firstly, the method looks for causally related signs for the classification, which naturally allows time variances between various attributes of the input data. secondly, the method is based on an abstracted layer (beyond numeric values), the differences in attributes’ physical meanings are ignored on this layer. thirdly, temporal information and local ordering of a single attribute can be embedded into individual events, depending on the event transformation, and the sliding window ensures decision are made over a controllable period. however most importantly, unordered event combination allows temporal variance within the period, which simplifies the feature space and provides tolerance to temporal misalignment among attributes. it has been illustrated in this work that the event group based decision tree approach demonstrates significant capability for classifying the sequential data. this event group based method now has the potential to be used as a generic methodology for modelling more complex time series data with missing or misaligned variates. however, users need to obtain considerable knowledge of the target problem in order to appropriately select the techniques for event generation, and the parameters for the egbc modelling procedure, e.g. the size of sliding window, duration of events and size of the xon nodes. we also find the event groups generated by xon have certain similarities to the concept of topic modelling (steyvers & griffiths 2007), however these are derived from supervised learning based on the information gain rather than probabilities. future research will include utilising topic modelling for minimising or scaling down the feature space, in order to extend this method to more challenging real world time series data mining with multi-dimensional forms and potentially complicated intra-sequential relationships.
 references: agrawal, r., faloutsos, c. & swami, a. n. (1993), efficient similarity search in sequence databases, in ‘fodo’, pp. 69–84. allen, j. f. (1983), ‘maintaining knowledge about temporal intervals’, commun. acm 26(11), 832–843. banks, b. s. (1999), neural network based modeling and data mining of blast furnace operations, phd thesis, massachusetts institute of technology. bhattacharjee, s. d. & das, a. (1999), ‘application of artificial intelligence in tata steel,’ tata search, tech. rep. batuwita, r. & palade, v. (2012), ‘adjusted geometric-mean: a novel performance measure for imbalanced bioinformatics datasets learning.’, j. bioinformatics and computational biology 10(4). australasian journal of information systems sun & stirling 2017, vol 21, selected papers from ausdm event group based classification framework 20 batyrshin, i. z. & sheremetov, l. (2008), ‘perception-based approach to time series data mining’, appl. soft comput. 8(3), 1211–1221. biswas, a. (1981), principles of blast furnace ironmaking: theory and practice, cootha. url: https://books.google.com.au/books?id=tbdxqgaacaaj cavnar, w. b., trenkle, j. m. et al. (1994), ‘n-gram-based text categorization’, ann arbor mi 48113(2), 161–175. fox, e. b., sudderth, e. b., jordan, m. i. & willsky, a. s. (2008), nonparametric bayesian learning of switching linear dynamical systems, in ‘nips’, pp. 457–464. freksa, c. (1992), an event group based classification framework for multi-variate sequential data, ‘temporal reasoning based on semi-intervals’, artif. intell. 54(1), 199– 227. granger, c. w. j. (1969), ‘investigating causal relations by econometric models and crossspectral methods’, econometrica 37, 424–438. granger, c. (1980), ‘testing for causality: a personal viewpoint journal of economic’, dynamics and control, 1980, 2, 329-352. harvey, j.-p. & gheribi, a. e. (2014), ‘process simulation and control optimization of a blast furnace using classical thermodynamics combined to a direct search algorithm,’ metallurgical and materials transactions b, vol. 45, no. 1, pp. 307–327, 2014. hornik, k., mair, p., rauch, j., geiger, w., buchta, c. & feinerer, i. (2013), ‘the textcat package for n-gram based text categorization in r’, journal of statistical software 52(6), 1–17. kommenda, m., kronberger, g., feilmayr, c., & affenzeller, m. (2011), ‘data mining using unguided symbolic regression on a blast furnace dataset,’ in applications of evolutionary computation. springer, 2011, pp. 274–283. koza, j.r. (1992), ‘genetic programming: on the programming of computers by means of natural selection,’ mit press, cambridge, ma, usa. laxman, s., sastry, p. & unnikrishnan, k. (2007), a fast algorithm for finding frequent episodes in event streams, in ‘proceedings of the 13th acm sigkdd international conference on knowledge discovery and data mining’, acm, pp. 410–419. mannila, h., toivonen, h. & verkamo, a. i. (1995), discovering frequent episodes in sequences extended abstract, in ‘1st conference on knowledge discovery and data mining, montreal, ca’. mannila, h., toivonen, h. & verkamo, a. i. (1997), ‘discovery of frequent episodes in event sequences’, data mining and knowledge discovery 1(3), 259–289. mcnamee, p. (2005), ‘language identification: a solved problem suitable for undergraduate instruction’, journal of computing sciences in colleges 20(3), 94–101. mohammad, y. & nishida, t. (2010), ‘mining causal relationships in multidimensional time series’, smart information and knowledge management: advances, challenges, and critical issues, springer berlin heidelberg, 309-338 möller-levet, c. s., klawonn, f., cho, k.-h. & wolkenhauer, o. (2003), fuzzy clustering of short time-series and unevenly distributed sampling points, in ‘ida’, pp. 330–340. morrill, j. p. (1998), ‘distributed recognition of patterns in time series data’, commun. acm 41(5), 45–51. oh, s. m., rehg, j. m., balch, t. r. & dellaert, f. (2008), ‘learning and inferring motion patterns using parametric segmental switching linear dynamic systems’, international journal of computer vision 77(1-3), 103–124. australasian journal of information systems sun & stirling 2017, vol 21, selected papers from ausdm event group based classification framework 21 porter, m. f. (1980), ‘an algorithm for suffix stripping’, program: electronic library and information systems 14(3), 130–137. project gutenberg. (n.d.), http://www.gutenberg.org. (accessed on 15/aug/2013). qiu, h.; liu, y.; subrahmanya, n. a. & li, w. (2012), ‘granger causality for time-series anomaly detection’, proceedings of the 2012 ieee 12th international conference on data mining, ieee computer society, 1074-1079. roddick, j. f. & mooney, c. (2005), ‘linear temporal sequences and their interpretation using midpoint relationships’, ieee trans. knowl. data eng. 17(1), 133–135. rumelhart, d. e., hinton g. e., & williams, r. j. (1988), ‘neurocomputing: foundations of research,’ j. a. anderson and e. rosenfeld, eds. cambridge, ma, usa: mit press, ch. learning representations by back-propagating errors, pp. 696–699. sarma, r. k. (2000), ‘neural network based prediction and input saliency determination in a blast furnace environment,’ ph.d. dissertation, massachusetts institute of technology, 2000. shieh, j. & keogh, e. j. (2008), isax: indexing and mining terabyte sized time series, in ‘kdd’, pp. 623–631. sollich, p. and krogh, a. (1996), ‘learning with ensembles: how overfitting can be useful’, advances in neural information processing systems, volume 8, pp. 190-196. steyvers, m. & griffiths, t. (2007), ‘probabilistic topic models’, handbook of latent semantic analysis 427(7), 424–440. sun, y.; li, j.; liu, j.; chow, c.; sun, b. & wang, r. (2015), ‘using causal discovery for feature selection in multivariate numerical time series’, mach. learn., kluwer academic publishers, 101, 377-395. sun, c., stirling, d., ritz, c. & sammut, c. (2013), ‘variance-wise segmentation for a temporaladaptive sax’, journal of research and practice in information technology, si. sun, c., stirling, d., wright, b., zulli, p. & ritz, c. (2011), ‘a hot metal based on hybrid decision tree techniques.’, international journal of technology, knowledge & society 7(2), 37 – 48. takcı, h. & soğukpınar, ̇i. (2004), centroid-based language identification using letter feature set, in ‘computational linguistics and intelligent text processing’, springer, pp. 640– 648. van noord, g. (n.d.), ‘textcat’, http://odur.let.rug.nl/vannoord/textcat/index.html. (accessed on 10/aug/2015). wallace, c. s. & boulton, d. m. (1968), ‘an information measure for classification’, the computer journal 11(2), 185–194. waller, m. & saxén h. (2000), ‘applying nonlinear time series methods to blast furnace tap variables,’ in presented at cybernetics & informatics eurodays, marianska, czech reublic. waller, m. & saxén h. (2002), ‘ontheuseofladle-wiseanalysestopredicthotmetal silicon content,’ in proceedings of the 61th ironmaking conference, nashville, tn, pp. 135–143. zheng, z. (2000), ‘constructing x-of-n attributes for decision tree learning’, machine learning 40(1), 35–75. zhou, f., la torre, f. d. & hodgins, j. k. (2013), ‘hierarchical aligned cluster analysis for temporal clustering of human motion’, ieee trans. pattern anal. mach. intell. 35(3), 582–596. australasian journal of information systems sun & stirling 2017, vol 21, selected papers from ausdm event group based classification framework 22 zhou, p., yuan, m., wang h. & chai t. (2015), ‘data-driven dynamic modeling for prediction of molten iron silicon content using elm with self-feedback,’ mathematical problems in engineering, vol. 2015, article id 326160, 11 pages, 2015. doi:10.1155/2015/326160. copyright: © 2017 sun & stirling. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ the australasian journal of information systems (ajis) began in 1993 and was based on the simple realisation that there was a need for an academic ‘outlet’ for is research in australia australasian journal of information systems volume 18 number1 2013 5 charting diversity and change in is publications: a tri-continental journal analysis susan keller deakin university susank@deakin.edu.au darryl coulthard deakin university darryl.coulthard@deakin.edu.au abstract the past decade has seen enormous changes in the availability, use, and application of information systems. information systems are no longer the preserve of business and the corporation but have become a routine part of everyday life. the purpose of this paper is to explore the extent to which the increased diversity in the application and use of information systems in the past decade has been reflected in increased diversity of the publications of three key journals of three continents – north america, europe, and australasia. the paper contributes to the debate on research diversity in information systems in three ways. firstly, it describes four key underpinning dimensions for understanding information systems (is) research diversity in the information age. secondly, it details the results of a content analysis which examines the trends in research and methodological diversity within the journals mis quarterly (misq), european journal of information systems (ejis) and the australasian journal of information systems (ajis) during the years 2001-2010. finally, the paper reflects on diversity within these journals and discusses the findings in the context of both exogenous and endogenous pressures on the discipline more broadly. keywords: research pluralism, journal content analysis, research methods, discipline reflection. introduction it is almost trite to observe that over the past decade or so we have seen major changes to information systems and their use. quite possibly, the most clear and most novel change over this period has been the rise in ubiquitous information systems (uis) and the rise of the digital native (vodanovich et al. 2010). uis refers to the rise of eeverything and the penetration of information systems not only in our working lives and business operations but into our lives and the way in which we shop, socialise, entertain, learn, coordinate meetings, seek information and find news. within the western world, people are becoming ‘digital natives’ (vodanovich et al. 2010). this does not mean that everyone is adept at web2.0, excel, jabber or tweeting and texting but that we all come across and interact with some forms of electronic information systems on a daily basis and that most people have, at the very least, become resigned to the fact that they do have to learn the rudiments to mailto:susank@deakin.edu.au mailto:darryl.coulthard@deakin.edu.au australasian journal of information systems volume 18 number1 2013 6 participate to some extent in the world. as hillis (2011) among others has suggested we are now irretrievably entangled in the internet and information systems whether we realise it or not. reliable technology uptake statistics are difficult to obtain but the growth in internet based information systems appears is extraordinary. one estimate suggests that internet users world-wide have increased by around 480% in the period 2000 and 2011(internetworldstats 2011). other studies suggest that during the decade to 2010, the percentage of adults in the us who access the internet via broadband has grown from 3% to 66% (smith 2010), and in the four years to 2011, mobile-broadband subscriptions globally have increased 45% annually (itutelecom 2011). incredibly, there are now twice as many mobile broadband subscriptions as fixed broadband (itutelecom 2011). by any account, social media use has exploded in the past five years. a study of us adult internet users in 2011 found that sixty-five per cent now say they use a social networking site like facebook or linkedin. coupled with mobile broadband, social media is increasingly being consumed and produced via smart phones. in 2011, a nielson poll showed that close to 40% of social media users in the us accessed content from a mobile phone (nielson 2011), and the market research company comscore, reported a 44% increase in the number of people accessing social networking via a mobile in france, germany, italy, spain and the u.k during 2011 (comscore 2011). there are also now hundreds of thousands of mobile and other applications that are changing the way people interact with business and government, and that are blurring the boundaries between work life and social life. this can be seen in the growing trend of virtual work where professionals work remotely from home, client locations, or on the road (busch 2011; mulki et al. 2009). already some fortune 500 companies including accenture, at & t and ibm have totally or partially eliminated traditional offices, and the increasing globalisation of work is expected to accelerate this trend in the coming decades (mulki et al. 2009). by contrast, information systems within businesses have arguably matured rather than changed dramatically. more efficient systems, better alignment to strategic goals and the tantalising prospects of data mining beckon but one would be hard pressed to suggest that the real story of the first decade of the 21st century was a revolution in business processes in comparison with this penetration of the internet and ict generally into everyday life. most of the largest growing companies google, apple, samsung, facebook etc. have grown in tandem with, and as a result of, the rise of the digital native and the penetration of information systems into everyday life. the challenge for business is less operational efficiency but how to exploit the rise of ubiquitous systems, big data, social media and the digital natives for competitive advantage. business and corporations may then no longer be the sole or even main catalyst for change in information systems and their impact on the world. the rise of the digital native and the extensive use of information systems outside of business and the corporation has meant that information systems have become a routine part of everyday life, often changing how we live and work (brockman 2011; yoo 2010). as simmons (2011 p.589) suggests: i cannot think of anything that has the potential to change social and political relationships as much as the production, dissemination, and use of new information technologies and new ways to communicate globally. the locus of change in information systems as a phenomenon appears to be moving from primarily the business sphere to the social sphere. this massive change prompts the questions of how indeed australasian journal of information systems volume 18 number1 2013 7 does uis change the way we live, think, work and do business? the fundamental changes in technology use in the past decade challenge the discipline with an expanded research agenda and we suggest that the rise of uis over the past decade frames a raft of new research questions for the discipline. on prima facie grounds, one could quite easily suggest that the discipline of information systems should be at the forefront of these changes. however, it is a moot point, as to whether information systems has kept up with, let alone foreshadowed or led these changes from business and laboratory to ‘esociety’ or indeed ‘eeverything’. our initial and provisional forays into the is literature did not identify a strong sense of following, let alone leading, the development of ict in the everyday and it was only at the very end of the decade that papers such as yoo (2010), identifying these developments, were being published in the most highly esteemed journals. this led us to consider the diversity of our most highly prized journals and also their ability to reflect and lead intellectual development of the discipline. for this paper, our question was whether this profound change in emphasis from business to social use of information systems has been reflected in changes to the academic is literature. has the focus of is research study changed to reflect these broader changes, or does it remain essentially concerned with the application of technology in the business setting? is is essentially about the development of more effective and efficient information systems for business applications or has it followed the computer out into the social world? whether or not we have overstated the penetration of is into the everyday, we would expect, that the academic journals would at the very least reflect something of this development. while the concept of an academic discipline is not a straightforward one, a discipline is defined at least in part by the academic journals in which research is published. thus, to explore these dimensions of diversity in information systems we decided to focus on what was being published rather than what was being argued for or against. as siderova et al. (2008:469) have observed “the body of published is research can be considered to be the stakeholders’ reflection on the is field identity” (emphasis in original). this is further supported by benbasat and zmud (2003), who see what is researches and thus publishes as signalling its intellectual core. to this end, we selected three english language journals of general information systems across three continents that are key journals and highly influential in each respective continent: mis quarterly (misq) in north america, the european journal of information systems (ejis) in europe, and the australasian journal of information systems (ajis) in australasia. each is amongst the top ranked generalist journals of their continent in the ais journal rankings list (ais journal rankings). the choice of continents was to reflect the nature of information systems as being at least to some degree context specific – research topics in one country may not be considered as important or as generalizable to other countries. ajis is, by various measures, rated as being lower than either mis or ejis in international rankings. the effect of this difference we believe is however muted by the influence of the journals themselves within their respective continents and the vagaries and politics of the determination of rankings. a ten year period 2001-2010 was chosen to identify changes to is following the widespread adoption of high-speed internet and the emergence of social networking and mobile computing. australasian journal of information systems volume 18 number1 2013 8 previous studies have suggested that there exists differences between north american and european journals (galliers et al. 2007; gallivan et al. 2007). specifically, north american journals are seen as more methodologically conservative, leaning heavily to positivist paradigms and favouring quantitative and survey methods. european journals on the other hand have been found to be more likely to favour new approaches most notably interpretive approaches, case studies and qualitative work (mingers 2003;benbasat & weber, 1996). to date, no australian journal content analyses have been conducted although the nature of is research in australia has been discussed in the literature (see for example gregor et al. 2007). our attention and focus of this paper is twofold: mapping diversity, and mapping the changing nature of research in the selected journals. diversity concerns itself with pluralism and providing the opportunity for the expression of a range of viewpoints and types of research to be published. to use a simile, diversity in research journals is like genetic mutation in evolution, it forms the spark for disciplinary growth and ‘evolution’: one might expect that more diverse journals are by their very nature more responsive to emerging trends. diversity can refer to two central elements: diversity of the source of research – the characteristics of those who are published and diversity of outcome – what is published. diversity of the source of research can be readily identified by the characteristics of the authors but diversity of outcome – the things being published – is a far more difficult to operationalize. what is published then becomes not only a measure of diversity but also of change. our research purpose is to explore whether (1) there is diversity of the sources and outcome of research (2) to chart the change in the types of research published in three key journals since 2001 and (3) to consider the findings in the light of the rise of ubiquitous information systems. to accomplish these aims, the paper is organised as follows. we first discuss the operationalization of diversity and measurement of change, and develop a set of research questions. we do this by reviewing previous studies aimed at understanding diversity in is research. in particular, we review the classification schemes previously used to capture dimensions of diversity in is publications and explain why they may be inadequate for tracking changes in the new world of the digital native and ubiquitous computing. next, we explain our research approach and method, and present the results of the content analysis of the three journals (misq, ejis and ajis) over the ten year period from 2001 to 2010. we conclude with a reflection on diversity within the three journals and consider the findings in relation to both exogenous and endogenous pressures on the discipline. the dimensions of the is field over the past twenty-odd years, many publication content and citation analyses have been undertaken with a variety of aims, including determining diversity and change within the discipline. evaluating diversity in publications depends on determining what the key characteristics of diversity are. benbasat and weber (1996) characterised diversity in is research as: diversity in the problems addressed; diversity in the theoretical foundations and reference disciplines used to account for is phenomena; and diversity in the methods used to collect, analyse, and interpret data. benbasat and weber are primarily concerned with diversity of publication outcome –that is, what is published. australasian journal of information systems volume 18 number1 2013 9 however in addition to this, a second dimension on diversity is the source of publications, such as the gender and country of the author or even originating discipline. previous diversity studies to develop our diversity criteria we analysed the past information system journal publication content analyses of other diversity studies (see appendix a for list and rationale for inclusion). from this it became apparent that most researchers have classified research in the same way as benbasat and weber (1996). that is, the ‘what’ and ‘how’ of published outcomes. some work has also looked in a limited way at the source of research, that is, who publishes and who makes publishing decisions (see appendix a). encapsulating the ‘what’ that has been published provides a particular problem of granularity. on the one hand categories can be drawn so broadly that fluctuations and change in the publication record cannot be realistically observed, whereas fine distinctions between concepts and problems tackled leads to a false impression of diversity of published outcome. fortunately, as a result of the ‘methodology wars’ of the 1990s (orlikowski et al. 1991; weber 2004) methodological differences or ‘research paradigms’ have been well established and more or less agreed upon levels of granularity and distinction are easily identified. a key challenge was to develop a framework of similar robustness for research areas or subject matter. on the other hand, diversity measures relating to the source of research appeared more straightforward. these measures could include who was undertaking the research as identified by gender, region and faculty. diversity in subject matter of investigation our first major task was to operationalise ‘what’ is published in is such that the measure was sufficiently sensitive to identify whether is remains primarily focussed on business imperatives or there is a shift to exploring the increasingly social drivers and use of information systems while not being swamped by other changes. although researchers have used various measures of ‘subject matter’ diversity including the theories and reference disciplines used in is, the table in appendix a shows that researchers have most frequently examined the subject matter of research by analysing topics and themes. these studies have applied a variety of classification schemes, some fine-grained and some at higher levels of abstraction. the scheme devised by barki, rivard, and talbot (1993) has informed many of the studies. in this work, barki et al. (1993) analysed the keywords of approximately 2,000 papers in six journals during 1987 to 1992 to arrive at a very detailed classification scheme for is research under the following nine high-level categories: reference disciplines; external environment; information technology; organisational environment; is management; is development; is usage; information systems; and is education and research. other is classification schemes have been developed since this time (see for example larsen et al. 2008; sidorova et al. 2008) and also lamp, j (2011 pp30-40) for discussion of the different attempts to categorise and classify the is domain. however, all these schemes are fundamentally alike in that they classify the topics and subjects of interest in is using an inductive approach. our approach was to use a content analysis but on the dimensions of interest to the study. the problem is that how we identify and define a phenomenon limits what we will discover about that phenomenon (sansone et al. 2004:6). while previous topic analyses have provided a surface picture australasian journal of information systems volume 18 number1 2013 10 of topics, they have not revealed the deeper values or beliefs that “circumscribe definitions of ‘worthwhile problems’” (chua 1986:602). so while schemes that categorise what we research by topic are useful for tracking themes and the emergence of new areas they do not develop our understanding of the is research domain significantly past “it and …” as more and more situations are added where it plays a role. what is needed is a view of the research domain that is at a higher semantic level than previous topic category schemas. what is required is a categorisation of topic similar in granularity and distinction as that of research methodology. to this end, we have developed the concept of research domain with initially three basic dimensions. our key interest was to investigate whether there was any change from a business to a social focus and these two dimensions were developed to identify research with either as primary aims. for the business domain, we identified articles where is is a handmaiden of business, identifying competitive advantage and operational effectiveness (porter 2001) of it and is systems and their management. more generally, this business dimension, which may include government business and provision of services, can be seen as an instrumental dimension: information systems research is a means to more efficient business and organisational ends. the social dimension attempted to delineate and identify the emerging use of ict by society outside business utility and technical development. whereas the third dimension concerns the development of technical efficacy and innovation of the is artefact, historically an important concern of information systems. here the principal concern is not with instrumentality but with technical development, irrespective of its application. three further dimensions were developed empirically as a result of our pilot content analysis: ‘education’, ‘disciplinary reflection’ and ‘other’. disciplinary reflection, such as this study, may be categorised as a form of ‘meta-information system’. ‘education’ included training and education of is professionals or the use of digital methods and information systems for education. ‘other’ comprised a residual category for any studies that were unable to be placed in the other dimensions during the content analysis. only eight articles were coded into the 'other' category. diversity in research approach as shown in appendix a, the most frequent way of investigating how is research is conducted is by examining research methods, and differentiating between empirical vs. non-empirical research. an analysis of the studies in appendix a showed that the three most frequently used research methods in is are case studies, surveys, and laboratory experiments. therefore, we coded for these three methods, and since our aim was to determine the diversity of research method, especially those methods which may be associated with the study of social phenomena, we also initially coded for action research, grounded theory, ethnography, interview methods, field experiments as well as multiple methods, content analysis and meta-analysis. low numbers of papers in most of these categories meant we eventually collapsed several categories into an ‘other’ category. since orlikowski & baroudi (1991) unsettling discovery that 96.8% of articles published in the us journals they examined were based on the positivist tradition, with only 3.2% interpretive studies, and no critical studies at all, other researchers have conducted analyses of research paradigm looking for evidence of these same three paradigms (see appendix a). we followed this lead but added postmodern studies since these are important in is’s reference disciplines. australasian journal of information systems volume 18 number1 2013 11 like previous researchers we also analysed research articles in terms of empirical vs non-empirical and qualitative vs quantitative research. diversity in research setting the third major question concerns the ‘where’ of research; to what extent has the discipline embraced diversity in geographical regions of studies, cross-country collaboration (hutton et al. 2011) and different research settings. research setting might be in a business setting, a research laboratory or university or with a select group of the general public outside of these settings. since information systems have broken out of the bounds of business we were interested to see if research setting has also diversified. the location of research has not received attention in past analyses of is research diversity (see appendix a). if what is published in is journals is following trends of globalisation then we would expect to see diversity in study region and setting. thus, in our analysis we devised two measures to examine where research is conducted. firstly, the geographical region where the study is undertaken, and secondly research setting which refers to the location of the study, in terms of whether it is in a business, a laboratory or a social setting. diversity of who is published our final question concerns the ‘who’ of research; this dimension of research concerns diversity of authors. that is, who gets published: what countries, gender, ethnic backgrounds and so on are overrepresented in publications. the internet and associated technologies offer the potential for increased cross-institute e-research collaboration, and the increased diversity in the use and application of information systems suggests we might expect to see more diversity in who publishes in our journals. in this research, we report on the gender and geographical region of authors. research procedure for the content analysis we examined all research articles in misq, ejis and ajis from 2001 to 2010, and following dwivedi & kuljis (2008) and gallivan & benbunan-fich (2007) we excluded editorials, guest editorials and book reviews. we also excluded short responses to research articles on the basis that these are more akin to editorials than full research papers, being typically only a few pages in length. we employed and trained two research assistants to code the articles who had cataloguing and taxonomic experience as librarians. we developed a ten page coding manual to provide detailed descriptions of the types of paper we expected to see in each category. figure 1, from the coding manual, shows diagrammatically the coded variables. these are explained next. australasian journal of information systems volume 18 number1 2013 12 figure 1: coded variables coding for subject matter research domain in analysing articles for research domain, we were interested in the underlying focus of the research rather than the topic, although in some cases there was an overlap of these constructs. for instance, ecommerce articles, while sharing the same topic, could have very different domains. in coding for research domain, articles were examined for the main focus or intent, and each article was coded as one of the following. • social: the principal aim of these studies is to examine the use and consequences of ict in society and people’s relationship to technology. studies with a social focus may aim to answer questions such as ‘how do people work with is/it?’ or, ‘what is the impact or effect of is/it on people or societies?’ some examples of indicative topics include: technology acceptance and perception, status of women in it, cross cultural software production and use. topics such as social media do not necessarily make the research ‘social’ for our purposes. studies of social media aimed at increasing organisational profits would be classified as instrumental while studies aimed at developing new social media systems would be classified as ‘technical’. • instrumental: the principal research aim of these studies is to increase business efficiency, effectiveness, security and risk minimisation or other outcomes in business and business organisations. it is about making work and information systems more efficient – better management, better monitoring, better systems to ‘get the job done’. this includes the use of ict to increase the efficiency and effectiveness of the state, business or service. australasian journal of information systems volume 18 number1 2013 13 • technical: the principal research aim of technical studies is to design and develop an is artefact or to test a theoretical construct, practical model or algorithm where the context and application is largely irrelevant. technical refers to the primary interest of systems information artefacts or processing where people or business is incidental to the process. example of ‘technical’ topics include is development, methods, and tools, databases, networks/telecommunications, hardware, security and internet/computer based communication systems if technically focused. • education: the focus here is on studies covering some aspect of is/it education. this may include the examination or evaluation of ict on education. • discipline reflection: discipline reflection covers those articles that aimed to reflect on aspects of the current state, progress, or future of the discipline. • other: this category was used when the main focus of the article could not be identified as any of the above. some examples of articles placed in this category include: thinking styles in science, international information systems security standards, standardizing and scaling in health is and organisational identity & boundary objects. all articles, whether empirical or non-empirical were coded for research domain. coding for location research setting empirical articles were also coded for research setting or location. here we aimed to understand the context of the study. this was operationalised as determining where the study participants or representatives were from. this dimension required less interpretation than research domain as most authors declared the study context. categories in this dimension include the following: • large business • small/medium enterprise • government agency • mixed studies e.g. government agency and business, or large business and sme • not-for-profit/community organisation • university/educational setting • general public • other • not applicable e.g. content analyses geographical location of research the geographical location in which the research was undertaken was also coded for. namely: • north america • united kingdom (uk) • europe not inc. uk • australia • new zealand (nz) • africa australasian journal of information systems volume 18 number1 2013 14 • asia (incl middle east, india) • south and central america • unknown • multi-region • not applicable to aid analysis, we grouped papers from nz with australia creating a category called australasia. similarly, because of the small numbers of papers from africa, south and central america we merged these regions into a category called ‘other’ for analysis purposes. coding for research approach research paradigm while there exists considerable debate on the differences and subtleties of the various paradigms we take positivism to be the view that nomothetic laws underpin and explain human behaviour and these laws can best be identified through objective methods that attempt to avoid the views of the researchers that might bias the results. interpretivists on the other hand argue that humans act according to their understandings of the world, not according to some underpinning law. the key task of the interpretivist is to identify and tease out what these often contradictory, complex and contextual understandings are. postmodernists question not only the objective nature of our social world but apply their scepticism to the ontological status of human agent. for them the human agent is not in itself a coherent subjectivity but is itself constructed. underpinning critical theory is the view expressed by marx that the point of research is not to understand the world but to change it (marx et al. 1974). for practical purposes we also included a neutral category for the rare study that attempts to survey the academic field and draws on all these perspectives. our coders were therefore instructed to code each paper using the following guide: • positivist: often identified by the presence of hypothesis testing, deductive reasoning, variables, causal relationships and the use of quantitative methods especially inferences drawn from samples to populations. we also followed orlikowski and baroudi’s (1991) lead and included 'descriptive' papers where researchers present what they believe are objective or factual accounts of events. these 'descriptive' studies sometimes contained case studies and simple descriptive statistics. • interpretive: in these studies, participants’ perspectives are used as the primary source of understanding of phenomena. the researcher is principally trying to discover the participants understanding of the phenomena and the meanings attached to these understandings and the actions they take as a consequence. also phenomena are examined with respect to cultural or contextual circumstances. interpretive studies are principally qualitative but some quantitative material may be presented. • critical: these studies take a critical stance towards the status-quo, often look at issues of inequality (e.g. unequal or unfair outcomes) and power and have a perspective of underpinning structural conflict between social agents. australasian journal of information systems volume 18 number1 2013 15 • postmodern: postmodern studies not only question the objective nature of our social world but the ontological status of the human agent. that is, postmodernists believe truth, totality (society) and the human subject to be fabrications or social constructions and the key focus of the postmodern study is to disclose this fabrication. research design, method and type each article was coded as either empirical or non-empirical, and for each empirical study the research method and whether the study was qualitative, quantitative or mixed. the research methods coded for were: • case study • surveys • interviews • lab experiments • field experiments • ethnography • meta-analysis • grounded theory • action research • content analysis • mixed methods • other owing to the small numbers of papers in the categories action research, grounded theory, ethnography, field experiments, and meta-analysis we collapsed these methods into the other category during analysis. coding for who is published we coded both gender and region for all authors. following galliers and meadows (2003) we used institutional affiliation as a proxy for the actual region of the authors. coding procedure in addition to the coding manual, we also developed an excel template to facilitate coding and asked both our research assistants to code eighteen randomly selected papers six papers from each journal. coders read the keywords and abstract of each paper and skimmed the introduction, conclusions, method section and references of each paper. some articles however did require more extensive reading. we also coded the same selection of papers and then compared results to identify areas of disagreement. some discrepancies were due to different interpretations of terms used in the coding manual and some highlighted shortcomings of the coding schema. differences in initial coding decisions were resolved and we updated the coding manual and amended categories as appropriate. our research assistants worked closely with each other to code the remaining articles, checking with us when in doubt. minor amendments to the coding manual were made as necessary. data were imported from excel into spss for data analysis. since we coded all research articles from the three journals during the period 2001 to 2010, the collected data were population data and analysed using descriptive rather than inferential statistics. australasian journal of information systems volume 18 number1 2013 16 findings and discussion we organise the findings and discussion around the four dimensions of research pluralism we are investigating: what subject matter was published, where studies were conducted, how studies were conducted and who is published. to facilitate the flow of discussion, percentages are rounded to the nearest percentage point but the tables show exact figures. what subject matter was published? our first research question aimed at discovering if what is published in the three journals is primarily focused on business imperatives or if researchers are exploring the increasingly social context of information systems. the findings indicate that while instrumental studies still dominate in all journals, studies with a social focus are increasing. table 1 shows that over the study period, instrumental studies made up the largest group accounting for 45% of papers overall, with misq publishing a larger proportion (55%) than the other two journals (both 41%). papers within the social domain were the next largest group at 38% overall. ejis published more socially focused papers (45%) than the other two journals which were around 33% each. overall, research focused on discipline reflection accounted for 9% of papers, while technically focused papers accounted for 7%. ajis published a much higher proportion of technically focused research than the other two journals. looking at trends over time, table 1 shows that overall instrumental papers have decreased from 51% (2001-2005) to 42% (2005-2010). social papers on the other hand have increased from 33% (20012005) to 42% (2006-2010). this suggests that these mainstream journals at least are ‘following the computer out of business’. over the study period, discipline reflection papers have trended upwards while technical papers have trended downward such that in the last five year study period they make up only 4% of the total papers published in the three journals. educational papers make up a very insignificant proportion of the three journals surveyed. ajis ejis misq 2001-2005 2006-2010 count 27 25 25 20 57 77 % within journal 10.6% 7.6% 10.1% 5.6% 12.0% 9.3% count 5 2 1 3 5 8 % within journal 2.0% .6% .4% .8% 1.1% 1.0% count 106 136 135 183 194 377 % within journal 41.6% 41.3% 54.7% 51.3% 40.9% 45.4% count 85 149 81 118 197 315 % within journal 33.3% 45.3% 32.8% 33.1% 41.6% 37.9% count 32 17 5 33 21 54 % within journal 12.5% 5.2% 2.0% 9.2% 4.4% 6.5% count 255 329 247 357 474 831 % within journal 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% domain journal period total total discipline reflection educational instrumental social technical table 1: research domain overall and by journal (excludes 8 articles coded as ‘other’) australasian journal of information systems volume 18 number1 2013 17 we also analysed each journal to see if individually they followed the same trends over time. surprisingly, figure 2 clearly shows that from 2005-2006 onward the proportion of misq’s publication of instrumental papers has been declining and at the same time social papers have increased, actually overtaking ejis in the last period. indeed, ejis is showing an upward trend in instrumental papers during the 2007-2010 (see figure 2) seemingly defying the commonly held view that that ejis is more progressive or future oriented than misq. 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 01-02 03-04 05-06 07-08 09-10 two year intervals ajis instrumental ajis social ejis instrumental ejis social misq instrumental misq social figure 2: trends of research domain within journal going against the trends of misq and ejis, ajis has, during the study period, consistently published slightly more instrumental papers than social papers. overall, comparing 2001-2005 and 2006-2010, we can say that the proportion of socially focused papers has increased, while instrumental papers have decreased and there are more discipline reflection papers suggesting that these journals are engaging with the expanded context of use of information systems and trying to make sense of it. the continued decline in technical papers suggests that is leaving behind its technical roots. where were studies conducted? research setting since information systems have broken out of the bounds of business we postulated that research setting, that is the context in which empirical research is conducted, would also show increased diversification. overall, we found that business settings dominate in the publications. nearly 30% of empirical studies were set in large business (see table 2) although there appears to be considerable difference between the journals. of the studies published in ejis, 37% were set in large business; considerably more than misq at 25% and ajis at 20%. australasian journal of information systems volume 18 number1 2013 18 ajis ejis misq 2001-2005 2006-2010 count 10 16 24 14 36 50 % within journal 8.3% 6.3% 13.7% 6.6% 10.7% 9.1% count 13 24 7 20 24 44 % within journal 10.7% 9.4% 4.0% 9.4% 7.1% 8.0% count 24 93 43 52 108 160 % within journal 19.8% 36.6% 24.6% 24.5% 32.0% 29.1% count 7 46 39 38 54 92 % within journal 5.8% 18.1% 22.3% 17.9% 16.0% 16.7% count 6 17 8 13 18 31 % within journal 5.0% 6.7% 4.6% 6.1% 5.3% 5.6% count 30 24 15 41 28 69 % within journal 24.8% 9.4% 8.6% 19.3% 8.3% 12.5% count 31 34 39 34 70 104 % within journal 25.6% 13.4% 22.3% 16.0% 20.7% 18.9% count 121 254 175 212 338 550 % within journal 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% sme university general public government agency large business mixed nfp/community setting journal period total table 2: research setting overall by journal and period (excludes 65 ‘not applicable’ and 30 ‘other’ papers). in addition, most of the ‘mixed’ studies, which accounted for 17% of studies overall, also included a large business (either large business and smes, or large business and government department). misq published 22% of these mixed studies, ejis 18% and ajis only 6%. smes were the setting of only 13% of studies overall, with a large difference between ajis (25%) and the other two journals both at around 9%. in contrast, research set in government agencies, not-for-profit or community organisations, and the general public together account for 23% overall in the publications. university based studies were the second largest single group with 19% of studies overall. this may be because of the convenience factor; researchers often use student cohorts for data collection purposes and universities also provide venues for lab studies. however, there were differences between the journals. ejis published less research set in universities (13%) compared to ajis (26%) and misq (22%). we were also interested in understanding how study setting has changed overtime. the most noteworthy trends are those involving large business and sme settings. both these settings show considerable difference between the first period (2001-2005) and the second time period (2006-2010). during 2001-2005, research set in large business accounted for only 25% of studies overall but by 2006-2010 the figure is 32%. it would appear that these gains have been at the expense of research set in smes since during 2001-2005 smes made up 19% of the studies but by 2006-2010 the figure had declined to 8% (see table 2). this decline is consistent with the findings of powell and woerndl (2008) regarding the falling research interest in smes. geographical region of study we also analysed the region in which each study was located since we conjectured that if information systems is following trends of globalisation then we would expect to see some evidence of increased diversity in study regions. as table 3 shows, overall north american studies make up the largest proportion with over a third of all studies, followed by european studies (including the uk) at 23%. australasian journal of information systems volume 18 number1 2013 19 australasian studies make third place with 19% of studies. south & central america and africa have only been the research setting of 6 papers in ten years (.04% each) so we combined them into the category ‘other/unknown’. looking at the individual journals, ejis has the greatest diversity of study regions with 38% focused on uk and europe, 33% focused on north america, 16% focused on asia and australasia. both misq and ajis appear largely parochial with around 70% of studies conducted locally and on this basis one may question their international scope. ajis ejis misq count 9 29 14 52 % within journal 6.7% 10.9% 7.4% 8.9% count 95 12 3 110 % within journal 70.9% 4.5% 1.6% 18.7% count 10 55 16 81 % within journal 7.5% 20.8% 8.5% 13.8% count 5 45 2 52 % within journal 3.7% 17.0% 1.1% 8.9% count 4 86 130 220 % within journal 3.0% 32.5% 69.1% 37.5% count 9 30 17 56 % within journal 6.7% 11.3% 9.0% 9.5% count 2 8 6 16 % within journal 1.5% 3.0% 3.2% 2.7% count 134 265 188 587 % within journal 100.0% 100.0% 100.0% 100.0% journal total study region as ia (incl middle east, india) austra las ia europe not inc. uk uk north america other/unknown multi -region total table 3: study region by journal we also examined the trends over time for each journal and plotted the major changes in figure 3 below. this analysis shows that there has been a large increase of north american based studies published in ejis (21% during 2001-2005 up to 38% during 2006-2010) and a corresponding dip in studies set in north america published in misq (80% down to 64%). the other noteworthy trend in ejis is a decrease in publication of studies undertaken in the uk, down from 27% 2001-2005 to 12% in 2006-2010, while at the same time there is an increase in studies located in continental europe (15% to 24%). 0% 20% 40% 60% 80% europe not inc. uk north america uk north america ej is m is q 2001-2005 2006-2010 figure 3: study regions in misq and ejis showing key changes only australasian journal of information systems volume 18 number1 2013 20 how were studies conducted? research paradigm if information systems is to claim research pluralism we would expect to see a good mix of papers using critical, interpretivist, positivist and postmodern paradigms within its journals. however, table 4 shows that overall positivist approaches, at 61% of all studies, still dominate while interpretivist studies make up only 30% overall. critical and postmodern studies have made hardly any inroads into the publications we studied. comparing the journals, misq published significantly more positivist research (81%) compared to ejis (51%) and ajis (56%). ejis leads the way with interpretivist research at 41% compared to misq’s at 12% and ajis’s at 32%. notably misq published no critical papers during the study period. ajis ejis misq 2001-2005 2006-2010 count 8 12 0 12 8 20 % within journal 3.1% 3.6% .0% 3.3% 1.7% 2.4% count 83 136 30 123 126 249 % within journal 32.2% 41.0% 12.0% 34.3% 26.3% 29.7% count 20 14 17 22 29 51 % within journal 7.8% 4.2% 6.8% 6.1% 6.0% 6.1% count 145 169 201 199 316 515 % within journal 56.2% 50.9% 80.7% 55.4% 65.8% 61.4% count 2 1 1 3 1 4 % within journal .8% .3% .4% .8% .2% .5% count 258 332 249 359 480 839 % within journal 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% critical interpretivist neutral period total paradigm journal positivist postmodern total table 4: paradigm by journal and period during the last five-year period, table 4 also shows that positivism on average has increased by 10% in the journals while there has been a decrease in interpretive research of around eight percentage points. analysing trends by journal however, reveals that most of the change has occurred in ejis and ajis. ejis averaged 50% interpretive research in the first period but this decreased to 36% during the second period (see figure 4). at the same time, ejis increased the publication of positivist studies from 42% to 55%. ajis showed a similar increase in positivist studies and a corresponding decrease in interpretive studies during the same period (see figure 4). so while misq has maintained a fairly constant proportion of positivist to interpretive studies over the two study periods, both ejis and ajis appear to be moving in a more conservative direction, publishing more positivist and less interpretive studies over the last five year period. research design, type, and method each article was also coded as either empirical or non-empirical. non-empirical studies include work such as theory development and literature reviews. overall, empirical research at 77% outstrips nonempirical research at 23% by a wide margin. there is also a marked difference between ajis and the other two journals. during the period 2001-2010, ajis published 59% empirical papers compared to ejis and misq at approximately 85% (see figure 5). if non-empirical studies serve to develop australasian journal of information systems volume 18 number1 2013 21 theory and new concepts as suggested by chen & hirschheim (2004) and avison et al (2008) we would expect to see an increase in non-empirical work as researchers grapple with the new research problems in the age of ubiquitous information systems. however, trends over the two 5 year periods show the proportion of empirical to non-empirical research remained fairly constant. 64.6% 55.4% 80.9% 51.2% 41.8% 80.5% ajis ejis misq postitivist studies 2001-2005 2005-2010 27.1% 35.6% 11.7% 35.2% 50.0% 12.6% ajis ejis misq interpretive studies 2001-2005 2005-2010 figure 4: paradigm trends by journal – positivist and interpretive only 58.9% 84.6% 85.1% 41.1% 15.4% 14.9% ajis ejis misq empirical non-empirical 7.2% 10.7% 9.9% 9.6% 48.0% 54.1% 18.4% 40.9% 44.7% 35.2% 71.7% 49.5% ajis ejis misq all journals mixed qualitative quantitative figure 5: empirical vs non-empirical figure 6: study type by journal and in total for each empirical study we also analysed whether the study used primarily qualitative, quantitative or mixed methods. we found that overall during 2001-2010 quantitative approaches where used in 50% of studies, outnumbering qualitative approaches at 41% and mixed methods at 10% (see figure 6). however, looking at the individual journals it is clear that misq publishes a much greater proportion of quantitative papers (72%) than ejis (35%) and ajis (45%) (see figure 6). conversely, ejis (54%) published more qualitative work than ajis (48%) and much more than misq (18%). in terms of trends, figure 7 misq shows a small upward trend in quantitative work (from 70% in 2001-2005 to 73% in 2006-2010) and a corresponding small decrease in qualitative studies (from 22% in 2001-2005 to 17% in 2006-2010). this is particularly interesting in light of misq’s increase in publishing research within the social domain. although ejis has published more qualitative australasian journal of information systems volume 18 number1 2013 22 papers than either misq or ajis it is actually trending downward from 62% in 2001-2005 to 50% in 2006-2010 while showing a corresponding increase in quantitative work during the same period (28% in 2001-2005 to 39% in 2006-2010). ajis has maintained a fairly constant proportion of qualitative to quantitative papers over the two time periods. % 10% 20% 30% 40% 50% 60% 70% 80% 2 0 0 1 -2 0 0 5 2 0 0 6 -2 0 1 0 2 0 0 1 -2 0 0 5 2 0 0 6 -2 0 1 0 2 0 0 1 -2 0 0 5 2 0 0 6 -2 0 1 0 ajis ejis misq mixed qualitative quantitative figure 7: proportion of research types over time in terms of research method, our findings, show that case studies (29.8%), surveys (27.6%) and lab experiments (9.8%) are the top three methods used although the proportion varies across the three journals. surveys were the most used method in misq papers (34% of studies) and ajis (29% of studies), while case studies were the most popular method in ejis (42% overall). as explained earlier, we initially coded for a wide range of research methods looking particularly for methods associated with the study of social phenomena (see section 3.3). however, since the categories action research, grounded theory, ethnography, field experiments, and meta-analysis yielded only a small number of papers (less than 5% each of the total) we collapsed these methods into a category called other. examining trends of the top three methods in each journal (see figure 8) shows that both ejis and misq show an upward trend in publishing survey-based research and ejis a corresponding drop in case study research. misq also published more lab experiments in the second study period while ejis and ajis published less. apart from the small rise in case study research evidenced in ajis, there is no increase in methods we would associate with the study of social phenomena or a new research area since ‘other’ methods accounted for 13.7% of total methods during 2001-2005 and 16.5% during 2006-2010. australasian journal of information systems volume 18 number1 2013 23 37.6% 50.5% 16.5% 19.2% 26.6% 22.7% 24.7% 16.8% 36.0% 30.1% 29.7% 28.4% 4.3% 8.4% 18.0% 11.0% 4.7% 12.5% 2006-2010 2001-2005 2006-2010 2001-2005 2006-2010 2001-2005 ej is m is q a ji s trends in top 3 methods lab experiments survey case study figure 8: top three research methods trends in summary, misq is still dominated by positivist, quantitative, survey research. ejis leads the way in interpretive, qualitative and case study research although it appears to be trending towards more conservative empirical methods. ajis is more eclectic, publishing more non-empirical work and a more even proportion of qualitative and quantitative work than the other two journals. socially focused research has increased over the past decade and as of 2010 is showing around the same proportion as instrumental studies. however, socially focused studies are being researched using the same type of research methods as instrumental studies. further analysis showed for instance, that overall case studies were used in approximately 31% of both types of studies, and surveys in around 30% of cases. lab experiments were the preferred method of 7.6% of instrumental studies and 8.6% of social studies. comparing research domain with research paradigm we find that interpretivism comprises 30% of instrumentalist studies and 42% of social studies. positivism comprises the balance (70%) of instrumentalist studies and over half of social focussed studies. finally, comparing research domain with research type shows that only 45% of social research is approached using qualitative methods. source of research: who publishes? our final question explored who is published. in particular, we analysed the gender, geographical region (as indicated by affiliation) and academic home department of authors. author gender the findings show a significant gender imbalance amongst authors. overall 28% of the total 1836 authors are women and a slightly higher proportion (32%) of first authors are women. misq averaged 25% female authorship, compared with ejis at 26% and ajis at 37% (see figure 9). avison et al (2008) report an even greater gender imbalance in the information systems journal where the percentage of female authors in publications between 2003 to 2007 was around 20%. in light of avison et al’s (2008) findings we also analysed the trends in the gender of authors over time. australasian journal of information systems volume 18 number1 2013 24 we found that overall the proportion of female authors was almost identical across the two time periods 2001-2005 and 2006-2010. 37.2% 25.7% 24.6% 62.8% 74.3% 75.4% ajis ejis misq author gender by journal female male figure 9: author gender by journal (excludes 89 authors where gender could not be determined) next, we looked at whether women publish differently to men. differences in research focus and paradigm were evident. men were more likely to publish positivist papers (66% men compared with 52% women) and instrumental papers (47% men compared with 40% women). author region the findings for the geographical region of the authors for the three journals show that approximately 42% of authors overall are from north america (see table 5 below). ajis ejis misq count 28 71 48 147 % within journal 5.2% 9.1% 7.9% 7.6% count 394 56 24 474 % within journal 73.2% 7.2% 0 24.6% count 44 195 51 290 % within journal 8.2% 24.9% 8.4% 15.1% count 40 293 468 801 % within journal 7.4% 37.4% 77.5% 41.6% count 7 3 5 15 % within journal 1.3% .4% .8% .8% count 25 165 8 198 % within journal 4.6% 21.1% 1.3% 10.3% count 538 783 604 1925 % within journal 100.0% 100.0% 100.0% 100.0% other/unknown uk journal total asia (incl middle east, india) australasia europe not inc. uk north america table 5: author regions australasian journal of information systems volume 18 number1 2013 25 further, 78% of misq authors are north american and surprisingly 37% of ejis authors are also from north america (see table 5). further exploration of the ejis data for regions of first author reveals the greatest change has occurred in the last 4 years, with north american authors increasing from around 15% in 2001 and 2002 to nearly half of all publications (47.6%) by 2009 and 2010. the only other straight-line trend in the data is again in ejis which has seen a steady decrease in first author publications from uk authors falling, from nearly half all publications in 2001 and 2002 (48.7%), to less than ten per cent (8.5%) by 2009 and 2010. the changes in region-authored publications may help to explain the changes in the research focus and research paradigm of ejis and misq respectively. with respect to ejis, the decline in uk authored publications and the rise in north american authored publications may explain the increase in positivist research in ejis. seventy per cent of north american first authored publications published in ejis were positivist, compared with only 13% of uk first authored publications. on the other hand, in misq, 42% of interpretivist publications had european first authors, compared with only 9% of north american first authors. fifty three per cent of social research domain papers had european first authors compared with 30% of north american first author publications. further reflections taken as a whole, the journals’ record is not a ringing endorsement for change and diversity of research outcomes. positivist and instrumentalist research predominates across the decade. indeed, positivist studies have trended upward over the ten-year period from around 55% in 2001-2002 to nearly 74% in 2009-2010. all three journals have published very little postmodern or critical theory research (see table 4). significantly, interpretive studies, while far better represented in is than postmodern and critical theories, actually fell over the period (see table 4). on the other hand, our analysis shows that overall socially focused research has increased over the past decade from 33% in 2001-2005 to 42% in 2006-2010 and as of 2010 is showing around the same proportion as instrumental studies. over the period we have seen a small rise in reflective studies at the expense of a similar fall in technical studies. there was little change over the period in the proportion of empirical to non-empirical studies. with respect to research settings the picture is not especially clear-cut. there was a small increase in the research setting of the general public but this was offset by a similar decline in the studies taking place in community settings. the two major changes in research setting were seen in the business arena with a large decline in the study of small business and a corresponding increase in the study of large businesses. with respect to methods, the most popular research method, quantitative surveys, increased its popularity in publication. this suggests that there has been a modest and most likely conservative response in the publications to the changing nature and penetration of information systems into the social world. it is modest to the extent that the rapid rise of ubiquitous computing and the digital native in the past decade represents a new and significant social phenomenon, in contrast to the relative stability in the strategic direction of business and is. the conservatism is suggested by the decline of diversity in methodology. interpretivism has fallen and the publication of postmodernism and critical theory is negligible. social focus has increased but the methodological paradigm remains conservative. at face value this is surprising as interpretivism australasian journal of information systems volume 18 number1 2013 26 is well represented in the social science research literature and postmodern and critical theory enjoy considerable status and popularity in the social sciences. one would expect that as information systems research grapples with the penetration of information systems into the social world that the discipline would draw upon a greater breadth of current social sciences methodologies. to sum up regarding who publishes, we see little change in the balance of gender over the ten year period with a little over one in four publications including a woman author. there is evidence to suggest that this reflects the number of women academics (lamp 2007). it is nevertheless a little disheartening not to see any improvement in this number. with respect to author region, it can be argued that of the three journals surveyed, only one is international in perspective ejis. ajis and misq predominantly publish authors of their own region. ejis published around 54% of non-european authors while ajis published 25% of non australasian and misq published around 20% non-north american authors. the question becomes one of whether this is a reflection of submissions from regions or some form of editorial gatekeeping. the question must remain open and we suspect different answers for each of these journals. with respect to ejis, there was a major increase in north american authors, more or less at the expense of british authors following the appointment of a north american editor and a strong policy of internationalising the journal. certainly, it would appear that ajis and misq have a long way to go should they wish to become truly international in input, if not perspective. it is nevertheless a moot point as to the degree to which is journals can and should be international. this is at least partly due to the nature of the discipline itself. unless one takes an extreme nomothetic view of is, one can expect contextual and regional matters to play a significant role. indeed, even should one consider that is is or can be highly nomothetic, the regional context of the study significantly weakens whatever nomothetic lessons that can be drawn from the study and the interest of the regional readership. for example, a tam study in the us is likely to be of greater general interest in the us than one in new zealand, taiwan, india or zimbabwe. van halen and henken (2012) in their international survey of demographers found that researchers focussed less on regional and policy matters in order to increase their publications in high ranking journals. this is supported by the study of journal titles by jaques and sabire (2010) who found that, among other things, the mention of a region in the title lessened citations in medical journals. what might interest the general reader of such studies outside of his or her region is the cultural or contextual proximity of the region to their own or the novelty of methods employed in the study. this matter of regional context appears to be completely overlooked in the various rankings of journal quality. the problem of regional context may also be compounded by research in the social world and the methodologies of non-positivist research and the cross fertilisation of ideas and approaches across the continents will remain a challenge. finally we come to consider the reasons for the tepid changes we have described. we propose three principal and interlocking reasons: australasian journal of information systems volume 18 number1 2013 27 the nature of leading journals to begin, it may be argued that we are looking in the wrong place for innovation in information systems. it may be the case that the top journals are unsuited to innovation and picking up rapid change. there has been an explosion of new journals in information systems over past decade (lamp 2011 p.4) and it may be these new journals that are publishing innovation. for as much as we might wish that these top journals reflect the acme of research in the discipline, they probably more accurately reflect exemplary research at the centre of the discipline. that is to say, these flagship journals and other highly ranked journals may define an orthodox, mainstream view of what to research and how to research it (adler et al. 2009). this position is quite consistent with kuhn’s (1970) view of the development of normal science and is probably inevitable in any ongoing community of practice (latour et al. 1986; wenger 1999). it is possible however that the slowness seen in publishing in the social area may be due to a necessary lag between identifying a research project, undertaking it and getting it published. this is undoubtedly true and brings into question the appropriateness of journals in rapidly changing areas such as information systems. our concern is that if journals prove to be averse and slow to change, this may influence the discipline, particularly with the rise of the importance of top ranked journals rather than conferences or social media publications. the influence of north american information systems on the basis of our study, one cannot but be impressed by the sheer quantity of publications by north american authors. this may well reflect the industriousness, quality, funding and number of north american scholars. however, it may well have an unintended homogenising effect on is that tends to undermine ideographic and contextual research as well as different research cultures and approaches. such a point rests on the notion that there are legitimate cultural differences in is across the globe in terms of interests, writing styles, traditions and approaches. our study does suggest that different regions do differ in the use of different methodologies, possibly the most well-established of the indicators of diversity. sheer numbers of publications may lead researchers to the view that there is an orthodox position which they need to emulate (van halen and henken, 2012). in the case of is there is a real possibility that this orthodoxy is increasingly becoming a north american one. our evidence suggests that the weight of numbers of north american research may be affecting ejis. rather than ejis diversifying in response to the changes over the past decade, this acclaimed diverse and progressive is publication (wilcocks et al. 2008), appears to be moving towards positivism, instrumentalism and big business and if anything beginning to converge with the profile of misq (see figure 4 and table 1). an explanation for the convergence of research domain and the movement towards the north american positivist and instrumentalist position is that (a) north american scholars are increasingly targeting ejis, possibly in response to the ais senior scholars forum 2006 recommending the inclusion of ejis in its list of a journals. (b) the emulation of successful publication strategies and the adoption of the cultural research practices and interests of north american information systems by authors and the underplaying of the particular circumstances of is in their own region or (c) some combination. australasian journal of information systems volume 18 number1 2013 28 north american dominance may unintentionally represent a form of academic globalisation and homogenisation and the major journals may become less interested in publishing contextual matters, especially outside the interests of the dominant research and readership region, north america. the conservative nature of the discipline itself finally, the discipline itself was borne out of the problems of business with information technology. this led to is identifying itself with business and its instrumental needs and as a discipline it has trained and attracted people to these ends. critical theory and postmodernism are unlikely to feature prominently in this mindset. moreover, universities themselves are becoming increasingly instrumental in character and impact is often considered in terms of business interest. these three factors: the nature of flagship journals, the influence of north america and the conservatism of the discipline may provide a significant brake on diversity and the response in is to a changing world and an isolation from real world problems. however, it seems that while we may have identified possible reasons for what we describe as a tepid diversity in the journals and response to change, what may drive this process and have greater implications is the journal ranking process itself and the heightened need of the is researcher to publish in the higher ranked journals. this may be the key driver of this process. as we have described above, the top journals have a natural tendency towards the mainstream and orthodox. to get published in such journals, the researcher with an eye to his or her career and survival as an academic will be tempted to study in the mainstream, using the orthodox repertoire of methods and if we are correct, studying such issues and with approaches that are popular in north america. one may suggest that research has been ever thus with researchers looking to the centre (in our case north america) for guidance. however, previously the non-mainstream journals provided an outlet for alternatives. now, they are by definition of ‘lower quality’ and publications in such journals are discouraged or discounted. orthodoxy is reinforced. there is now the real fear that important problems will be neglected for safe problems investigated in an orthodox manner as a result of an orthodoxy and increased emphasis on publishing in the top journals (powell & woerndl, 2008; lincoln, 2012; özbilgin, 2009). essentially, researchers will be taking the advice they give their phd students: to pick a topic that is likely to succeed and at the very least won’t threaten the examiners if it doesn’t expressly flatter them. powell and woerndl (2008) in their provocatively titled article, ‘time to stop researching the important things?’ have shown that there are real consequences for working in new or small areas of research. such research garners fewer citations than those working in a more fashionable and orthodox area. they show that articles that cite the technology acceptance model (tam) have a higher citation index than those that cite small medium enterprises (smes). this is consistent with our findings of the decline in sme research in our flagship journals. it is also consistent with van dalen and henken’s (2012) findings that more context free theoretical work such as tam is being published at the expense more applied and thus localised work such as sme research. özbilgin (2009) goes further in the potential or actual effects of the journal ranking system on publications. referring to the field of business and management studies, özbilgin supports the view that such systems stifle innovative scholarship and research and forcefully argues “the current hegemonic order in journal publishing encourages cloning and inbreeding. even more worrisome is australasian journal of information systems volume 18 number1 2013 29 an engagement with questions of marginal relevance. this trend presents a tyrannical challenge against ethical, purposeful, meaningful, and engaged scholarship.” within our own discipline, chen and hirschheim (2004) identified similar pressures at work to discourage interpretivist work, observing that: while under the pressure of tenure and promotion, researchers would tend to choose a less time consuming approach – positivism – to avoid perishing. this might help explain why interpretive influences are still marginal and likely to continue to be this way. as such, for a pluralist research tradition to be established, both the publication and tenure and promotion systems might need some modifications. the reason that most researchers employ a less timeconsuming approach is because for tenure and promotion, the quantity of publications counts. the less time consuming the approach, the more productions are possible; the more productions, the better the chance for tenure and promotion. while the tenure and promotion system values the quality of publications as well, it is difficult to get tenure at a good school without suitable numbers of publications. in the existing system, researchers are more concerned about how to get a research project published rather than asking significant research questions. many research papers fail to address meaningful issues as they are too difficult and take too long to publish (chen and hirschheim, 2004:225). our findings are consistent with this view. chen and hirschheim’s view is supported by one of the doyennes of qualitative research, yvonne lincoln (2012) where she argues that qualitative research and research that threatens the status quo is being silenced through lack of funding and lack of publishing outlets into the high ranking areas. conclusion we found at best modest change in the three leading journals we studied in the face of what we consider to be a massive change and penetration of is into the social sphere. while modest change may be expected in the top journals it does not auger well for the discipline in the light of the increased emphasis on journal rankings that in all likelihood reinforce orthodoxy. we suggest that cultural or regional diversity in terms of research interests; context and methodology may be under threat, as may be research that challenges the status quo. if this is correct, then the growth of the discipline is stifled right at the very time it could well become the central discipline of information in society and business. this rather gloomy conclusion for the discipline contrasts with the sunny disposition of is in the social world. however, at the very least, seeing this issue, even somewhat tentatively suggests firstly further research to determine the existence, extent and power of this homogenising and conservative force. it also provides the opportunity for government and university policy makers, particularly in non-dominant regions, to consider the need to adjust their journal rankings to ensure diversity and inclusion. it would be remiss at this point were we not to suggest some recommendations for the development of diversity in these and other journals. it is here that the editorial boards of journals may well have the most important role. this has been previously recognised. for instance, allen lee’s work as senior australasian journal of information systems volume 18 number1 2013 30 editor and later misq editor-in-chief was influential in promoting methodological pluralism in the discipline (chen et al. 2004). the influence of the chief editor may also be seen where in 2005, carol saunders became editor misq and brought a new “diamond cutter” approach to reviewing papers and called for a dramatic change in approach at misq. significant in this was an opening up of the reviewing pool and a focus on assisting authors, rather than being gatekeepers. it is possible that these changes ushered in by saunders led to the change in research content in misq post 2005. similarly, we note the recent call by ejis editor-in-chief franz rowe (2012) for an increase in the diversity of research genres. rowe hopes the explicit encouragement of the call will work against what he describes as the pressure of researchers to produce ‘incremental papers, with a quantitative standard methodology and a theory testing inclination’. these calls and acts by individual, influential editors are encouraging. three specific proposals could be included:  establishment, or in the case of misq the continuation, of a more diverse and larger review panel of academics;  an increase in special issues that may cover research in different research domains and paradigms, and;  the introduction of special issues or sections on regional areas such as special focus on information systems research in various countries for journals that wish to reflect an international perspective. more radically, the journals must consider their role in the dissemination of research and the identification and support of quality in the digital age. journals and academic presses act as the ‘de facto arbiters of hiring, tenure, and promotion’ (dawson, 2010:2). as such there is a danger that conformity will triumph over diversity. however, there is an opportunity to increase diversity as well. open source reduces the distribution costs of journals to allow more issues and larger, more diverse review panels can review and constructively criticise and support a larger number of quality papers. that may be the next step for journals. references "mis journal rankings," in association of information systems. adler, n., and harzing, a. 2009. "when knowledge wins: transcending the sense and nonsense of academic rankings," academy of management learning and education (8:1), pp 72-95. alavi, m., and carlson, p. 1992. " a review of mis research and disciplinary development," journal of management information systems (8:4), pp 45-62. avison, d., dwivedi, y. k., fitzgerald, g., and powell, p. 2008. "the beginnings of a new era: time to reflect on 17 years of the isj," information systems journal (18:1), pp 5-21. ayanso, a., lertwachara, k., and vachon, f. 2007. "diversity or identity crisis? an examination of leading is journals," communications of the association for information systems (20), pp 660680. barki, h., rivard, s., and talbot, j. 1993. "a keyword classification scheme for is research literature: an update," mis quarterly (17:2), pp 209-226. benbasat, i., and weber, r. 1996. "research commentary: rethinking "diversity" in information systems research," information systems research (7:4) december 1, 1996, pp 389-399. australasian journal of information systems volume 18 number1 2013 31 benbasat, i., and zmud, r. w. 2003. "the identity crisis within the is discipline: defining and communicating the discipline's core properties," misq (27:2), pp 183-194. brockman, j. (ed.) is the internet changing the way you think?: the net's impact on our minds and future. atlantic books, london, 2011. busch, e., nash, j., & bell, b. s. 2011. "remote work: an examination of current trends and emerging issues.," center for advanced human resource studies, cornell university, ithaca, ny. chen, w., and hirschheim, r. 2004. "a paradigmatic and methodological examination of information systems research from 1991 to 2001," information systems journal (14), pp pp 197-235. chua, w. f. 1986. "radical developments in accounting thought," the accounting review (61:4), pp 601-632. claver, e., gonzalez, r., and llopis, j. 2000. "an analysis of research in information systems (19811997)," information & management (37), pp pp 181-195. comscore 2011. "mobile social networking audience grew 44 percent over past year in eu5." dwivedi, y., and kuljis, j. 2008. "profile of is research published in the european journal of information systems," european journal of information systems (17:6), pp 678-693. farhoomand, a. f. 1987. "scientific progress of management information systems," data base (18:4), pp 48-56. farhoomand, a. f., and drury, d. h. 1999. "a historiographical examination of information systems," communications of the association for information systems (1), pp 1-27. galliers, r. d., and meadows, m. 2003. "a discipline divided: globalization and parochialism in information systems research," communications of the association for information systems (11:5). galliers, r. d., and whitley, e. a. 2007. "vive les differences? developing a profile of european information systems research as a basis for international comparisons " european journal of information systems ( 16, 20 – 35 1), pp 20-35. gallivan, m. j., and benbunan-fich, r. 2007. "analyzing is research productivity: an inclusive approach to global is scholarship," european journal of information systems (16:1), pp 36-53. gregor, s., bunker, d., cecez-kecmanovic, d., metcalfe, m., and underwood, j. 2007. "australian eclecticism and theorizing in information systems research," scandinavian journal of information systems (19:1), pp 11-38. hillis, w. d. 2011. "introduction: the dawn of entanglement," in is the internet changing the way you think?: the net's impact on our minds and future, j. brockman (ed.), atlantic books: london. hutton, g., and fosdick, m. 2011. "the globalization of social media," journal of advertising research (51:4), pp 564-570. internetworldstats 2011. "world internet usage and population statistics," in internet world stats, miniwatts marketing group. itutelecom 2011. "ict facts and figures," itu telecom. australasian journal of information systems volume 18 number1 2013 32 jacques, t. s., and sebire, n. j. 2010. "the impact of article titles on citation hits: an analysis of general and specialist medical journals," journal of the royal society of medicine (1:2). kuhn, t. s. 1970. the structure of scientific revolutions, (2nd ed.) university of chicago press: chicago, ill. lamp, j. w. 2007. "perceptions of gender balance of is journal editorial positions," communications of the ais (20), pp 124-133. lamp, j. w. 2011. information categorisation: an emergent approach, phd, university of melbourne. larsen, k. r., monarchi, d. e., hovorka, d. s., and bailey, c. n. 2008. "analyzing unstructured text data: using latent categorization to identify intellectual communities in information systems," decision support systems (45:4), pp 884-896. latour, b., and woolgar, s. 1986. laboratory life : the construction of scientific facts, (princeton university press: princeton, n.j. lending, d., and wetherbe, j. c. 1992. "update on mis research: a profile of leading journals and u.s. universities," data base (23:2), pp 5-11. lincoln, y. s. 2012. "the political economy of publication: marketing, commodification, and qualitative scholarly work," qualitative health research) august 30, 2012. marx, k., and engels, f. 1974. the german ideology. part one, with selections from parts two and three ; together with marx's introduction to a critique of political economy and theses on feuerbach, (2nd ed.) lawrence & wishart: london. mingers, j. 2003. "the paucity of multi-method research: a review of the information systems literature," information systems journal (13), pp 233-249. mulki, j., bardhi, f., lassk, f., and nanavaty-dahl, j. 2009. "set up remote workers to thrive," mit sloan management review (51:1), pp 63-69. nielson 2011. "the social media report," in state of the media, nielsen. orlikowski, w. j., and baroudi, j. j. 1991. "studying information technology in organizations: research approaches and assumptions.," information systems research (2:1), pp 1-28. özbilgin, m. f. 2009. "from journal rankings to making sense of the world.," academy of management learning & education. (8:1), pp 113-121. palvia, p., mao, e., salam, a. f., and soliman, k. s. 2003. "management information systems research: what's there in a methodology?," communications of the association for information systems (11), pp 289-309. palvia, p., pinjani, p., and sibley, e. h. 2007. "a profile of information systems research published in information & management," information & management (44:1), pp 1-11. porter, m. e. 2001. "strategy and the internet," harvard business review:march), pp 63-78. powell, p., and woerndl, m. 2008. "time to stop researching the important things? ," european journal of information systems (17), pp 174-178. sansone, c., morf, c. c., and panter, a. t. 2004. "the research process: of big pictures, little details, and the social psychological road in between," in the sage handbook of methods in social psychology, c. sansone, c. c. morf and a. t. panter (eds.), sage. australasian journal of information systems volume 18 number1 2013 33 sidorova, a., evangelopoulos, n., valacich, j. s., and ramakrishnan, t. 2008. "uncovering the intellectual core of the information systems discipline," mis quarterly (32:3), pp pp.467482. simmons, b. a. "international studies in the global information age," international studies quarterly (55:3), pp 589-599. simmons, b. a. 2011. "international studies in the global information age," international studies quarterly (55:3), pp 589-599. smith, a. 2010. "home broadband 2010," pew research center’s internet & american life project. van dalen, h. p., and henkens, k. 2012. "intended and unintended consequences of a publish-orperish culture: a worldwide survey " journal of the american society for information science and technology (63:7), pp 1282–1293. vodanovich, s., sundaram, d., and myers, m. 2010. "digital natives and ubiquitous information systems," information systems journal (21:4), pp 711-723. vogel, d. r., and wetherbe, j. c. 1984. "mis research: a profile of leading journals and universities," data base (16:1), pp 3-14. weber, r. 2004. "the rhetoric of positivism versus interpretivism: a personal view (editor's comments)." mis quarterly (28:1), pp iii-xii. wenger, e. 1999. communities of practice : learning, meaning, and identity, (cambridge university press: cambridge. wilcocks, l., whitley, e. a., and avgerou, c. 2008. "the ranking of top is journals: a perspective from the london school of economics," european journal of information systems (17), pp 163-168. yoo, y. 2010. "computing in everyday life: a call for research on experiential computing " mis quarterly (34:2), pp 213-231. australasian journal of information systems volume 18 number1 2013 34 appendix a a la v i & c ar ls o n ( 1 9 9 2 ) a v is o n e t al .( 2 0 0 8 ) a y an so e t al .( 2 0 0 7 ) c h en & h ir sc h h ei m ( 2 0 0 4 ) c la v er e t al . (2 0 0 0 ) d w iv ed i & k u lj is ( 2 0 0 8 ) f ar h o o m an d & d ru ry ( 1 9 9 9 ) f ar h o o m an d ( 1 9 8 7 ) g al li er s & m ea d o w s (2 0 0 3 ) l en d in g & w et h er b e (1 9 9 2 ) m in g er s (2 0 0 3 ) o rl ik o w sk i & b ar o u d i 1 9 9 1 ) (1 9 9 1 ) p al v ia e t al . (2 0 0 3 ) p al v ia et a l. ( 2 0 0 4 ) p al v ia e t al .( 2 0 0 7 ) v es se y e t al . (2 0 0 3 ) v o g el a n d w et h er b e (1 9 8 4 ) t h is s tu d y what treatment of it artefact consideration of it artifact x topic / themes x x x x x x x x x x x x theories used x reference discipline x x research domain x how empirical vs non-empirical x x x x x x x x x quantitative/qualitative x x x x research methods x x x x x x x x x x x x x x x x x research paradigm x x x x x x x unit/level of analysis x x longitudinal vs snapshot x x who: authors individual researchers x x academic/practitioner x x x x gender x x home department x x university/ institute x x x x region x x x x number authors x x :editors region x x gender x where (research conducted): study setting x study region x dimensions of diversity coded for in previous content analyses (this study final column) australasian journal of information systems volume 18 number1 2013 35 rationale for studies included in our analysis. content analyses that were included: (i) had a clear focus on the is discipline; (ii) employed a content analysis method (ie. papers were classified using a predefined classification scheme); (iii) investigated primarily journal articles (iv) investigated the adoption of at least one dimension of diversity, for instance, research methods or topics/themes. however excluded studies that focused on only a single aspect within that dimension such as only case studies within research methods; and (v) investigated articles published no earlier than the 1980s since arguably the discipline was changed considerably since that time. we also excluded research aimed at primarily analysing citation indexes and individual researcher productivity. paper title (paper title style) australasian journal of information systems shahparvari, chhetri, abareshi, abbasi & alahakoon 2015, vol. 19, pp. s133-s151 multi-objective decision analytics multi-objective decision analytics for short-notice bushfire evacuation: an australian case study shahrooz shahparvari school of business it & logistics rmit university shahrooz.shahparvari@rmit.edu.au prem chhetri school of business it & logistics rmit university ahmad abareshi school of business it & logistics rmit university babak abbasi school of business it & logistics rmit university damminda alahakoon la trobe business school la trobe university abstract late evacuation in a bushfire is a crucial stage of emergency response, which requires a quick response under life-threatening conditions. lack of operational intelligence and decision support system to execute evacuation of residents within a 10 minutes time window were among key factors, which contributed to the loss of 119 lives (68 percent of total fatalities) during the black saturday 2009 bushfire in victoria. hard constraints of the limited time window, the uncertainty of bushfire spread, road disruptions and elderly and disabled late evacuees pose multiple challenges for fire agencies. an application of analytics and decision support system, capable of computing just-in-time allocation of resources, can enhance the capacity of fire services agencies. using the multi-objective analytics, this paper therefore develops optimal resource allocation models to enhance emergency response to improve the efficiency of late evacuation response. three key operational challenges are tackled including timely evacuation, shelter assignment and routing. three bushfire scenarios are constructed to incorporate constraints of restricted time-window and potential road disruptions. capacity and number of rescue vehicles and shelters are other constraints. this mathematical model is solved by the application of the constraint approach. objective functions are simultaneously optimised to maximise the total number of evacuees while minimise the number of assigned rescue vehicles and shelters. we argue that this model provides a scenario-based decision-making tool to aid maximise the resource utilisation and coverage of late evacuees. the analytics based insights drawn from various disruption scenarios could help emergency services agencies in identifying appropriate strategies to improve the efficiency of late evacuation response. keywords: bushfire, late evacuation, multi-objective optimisation, emergency management 1 introduction bushfire is one of the most prevalent natural hazards in australia. a bushfire is “a freely burning, uncontrolled and unplanned fire, which needs to be extinguished” (cfa definitions 2012). in recent years, there has been a substantial increase in the number of bushfires around the world (cameron et al. 2009). global warming and climate change have potentially s133 australasian journal of information systems shahparvari, chhetri, abareshi, abbasi & alahakoon 2015, vol. 19, pp. s133-s151 multi-objective decision analytics increased the risk of bushfire, particularly in peak seasons in many countries including australia (teague et al. 2009), usa (westerling et al. 2006), canada (podur et al. 2002) and russia (kharuk et al. 2007). bushfire is a threat to communities, particularly those who live in bushfire-prone areas. every year, bushfires (also referred as wildfire) cause significant financial and human losses (butry et al. 2001). although direct and indirect costs are difficult to estimate, there are three cost components: loss of human life and injury; economic costs; and damage to the ecosystem. the bushfires royal commission has estimated the cost of the black saturday disaster to be $4.4 billion, along with 173 lives lost (teague et al. 2009). due to the significance of bushfire hazards to australia, much research is conducted in a variety of research areas including environment impact assessment, risk mitigation, community resilience and cost estimation (cameron et al. 2009; church and cova 2000; ellis et al. 2004; lindell and perry 2003). however, the late evacuation of residents from bushfire-affected areas has attracted much less research attention in australia until recent years. applications of innovative optimisation techniques and analytics would help fire agencies in improving the operational response to bushfire emergency. this will in turn help saving lives and prevent injuries as well as reducing economic costs through a better utilisation of scarce resources. one of the key challenges in bushfire events is the operational capacity to design an efficient and effective evacuation plan from bushfire-prone areas to shelters within a restricted time windows with finite resources. bushfire tends to disrupt the emergency supplies and transportation networks, which in turn makes the transfer of late evacuees difficult. for example, bushfire propagation rate increases twice with every 10-degree increase in slope and the bushfire spread doubles with a small change in wind speed. the temperature reaches to 800-degree celsius during a bushfire (whittaker et al. 2009), and the flames crown attained a height of about 30 meters (victorian bushfires royal commission report 2009). as in the case of the black saturday bushfire in victoria, late evacuation has proven to be extremely hazardous and lethal (victorian bushfires royal commission report 2009). late emergency evacuation in dynamic bushfire conditions, therefore, requires a robust modelling and analytic capability that consider the simultaneous optimisation of multiple objectives and constraints. such decision support tool should be capable of dynamically selecting a shelter, assigning each evacuee to a shelter, maximise spatial coverage, and determine the optimal routing. thus, it involves solving four key problems: allocation, assignment, spatial coverage and routing. designing of a robust system is dependent on human behaviour and response to a threat. thus, the decision to stay or leave early in a bushfire is a critical to community safety. people those expose to fire risk may decide to leave early, or take shelter-in-home or shelter-in-refuge (cova et al. 2011). obviously, early evacuation is the safest option to protect life. nevertheless, there are some residents who prefer to stay and protect their properties. there are also those who leave at the last minute (victorian bushfires royal commission report 2009). the late evacuation exposes the evacuees to radiant heat, which has been found a key factor in human fatalities (teague et al. 2009). late evacuation often increases the risk of injury or death. studies that have contextualised the bushfire emergency evacuation considering australian bushfire emergency policy context to tackle the key operational challenges including timely evacuation, shelter assignment and routing are rather rare (shahparvari et al. 2015a; shahparvari et al. 2015b). this research bridges the gaps by developing a reliable mathematical model, which aims to maximise the number of evacuated late evacuees within the restricted time windows (clearance time) via shortest risk routes. the proposed model also is capable of allocating the optimal utilisation of the available resources such as accessible shelters and vehicles. this model has been implemented as a two-step framework with the application of the -constraint method. the main contributions of the model are summarised as follows: alignment to provide evidence to assess the “stay or go recommendation” policy, which has been developed based on many years of past experience. s134 australasian journal of information systems shahparvari, chhetri, abareshi, abbasi & alahakoon 2015, vol. 19, pp. s133-s151 multi-objective decision analytics the formulation of a novel capacitated mixed-integer multi-objective optimisation problem with time windows which is capable of generating and enhancing the requirement of quick responses and prioritising shipments during emergency evacuations. the ability of the model to handle complex capacitated multi-location, multi-vehicles types under multi-time windows problems using a multi-objective optimisation modelling framework. the application of the analytical tool to a real-world case study a bushfire. the remainder of the paper is organised as follows. the next section reviews relevant literature in the area of bushfire evacuation planning. then, the problem statement is described followed by problem assumptions. the developed mathematical formulation and the solution approach are presented in the subsequent sections. the effectiveness of the model will be assessed and discussed by implementing a range of bushfire scenarios. the paper concludes with presenting a summary of the key findings. 2 literature review evacuation is the process of “relocation from areas at risk to areas of greater safety” (southworth 1991; zel transferring people from assembly points to safe shelters within the close vicinity. evacuation is a complex process, which entails operations required across different phases (figure 1). evacuation process during a disaster can be categorised as pre-disaster, during-disaster and post-disaster phase. the pre-disaster is the initial phase of detecting an incident and make the decision to stay or leave before the disaster begins to threaten people’s lives. during-disaster phase includes five stages that should be carried out. it in this phase when risks and potential threats in specific areas are estimated by decision makers prior to issuing an evacuation order (lindell 1995). in phase iii, the communities at risk are alerted with early warning. in phase iv, a decision to stay or evacuate is made by residents, which leads to the preparation for evacuation. in the next phase, transfer of people to safe areas or shelter occurs. this phase implies clearing of affected people from hazardous zones. in the penultimate phase, people at risk are transferred to shelters outside of hazardous areas. as shown in figure 1, the response time (clearance time), which includes phase five and six, is one of the main critical factors that plays a key role in evacuation processes. this study therefore focuses on these phases to enhance the evacuation response time. at last, and in the third category as post-disaster processes and at the final phase, verification and auditing are carried out to ensure that evacuees are evacuated to safe places. evacuation order issued deliver order/ message to public via transmitters incident detection preparation for evacuation movement through evacuation network arrival at safety zone verification phase i phase ii phase iii phase iv phase v phase vi phase vii decision time notification time preparation time response time (clearance time) evacuation time pre-disaster during-disaster (self-evacuees tend to by-pass these steps) post-disaster figure 1 evacuation phases (source: adapted from lindell, 1995) in the event of bushfire, residents’ responses can be categorised into three groups; those who leave early; those who stay at their properties (shelter in refuge) and those who decide to shelter in the refuge. cova et al. (2011) referred these decision-choices as the ‘protective actions’ (figure 2). bushfire evacuation can be mandatory, recommended, or voluntary. in most countries, evacuation during a bushfire is compulsory. in australia and southern parts of s135 australasian journal of information systems shahparvari, chhetri, abareshi, abbasi & alahakoon 2015, vol. 19, pp. s133-s151 multi-objective decision analytics france a stay or go choice is given to residents except in the case of a severe bushfire (arnol 2007). people have the choice to leave early, or stay at home and protect their properties (shelter-in-place). some residents with disability, elderly, and parents with children, however, require additional support; whilst others might not have access to a personal vehicle. evacuate 100% (e) shelter in refuge 100% (r) shelter in place 100% (p) e+ r + p =100% 0% 0% 0% figure 2 protective actions ternary diagram (source: cova et al., 2011) in australia, the “stay or go” policy transfer the responsibility and decision of staying and protecting the property or leave early to individuals. the current australian ‘stay or go’ policy, which is a replacement of previous plan ‘prepare’, stay and defend, or leave early’, has strategically design to reduce risk to government agencies by making people responsible for their actions (paveglio et al. 2008). one key limitation of this policy in victoria is the lack of consideration for late evacuees who are not well integrated into the evacuation procedures (teague et al. 2009). the policy emphasizes the importance of timely evacuation and condemned the policy of prepare and protect. an early evacuation is desirable and is most preferred option to minimise risk in a bushfire. similarly in the united states, the “ready, set, go” (rsg) policy is in use, which provides information on when and how to prepare and evacuate or take refuge in a shelter. people who decided to stay and protect their properties in an extreme weather required an efficient and effective emergency response. hence, planning and resourcing are the key issues in emergency management. the evacuation problem has traditionally been solved as an optimization problem that minimises/maximises a certain objective function subject to supply, demand and time constraints. numerous objective functions have been formulated in the literature with the goal of expediting the evacuation process. in comprehensive research, cohon (2004) identified some objectives, such as clearance time, the number of an evacuee, and resources, which should be considered in decision making. while the early evacuation occurs by car (pel et al. 2012), however, the late evacuation has to be carried out using high-capacity vehicles such as buses to transfer people to shelters (vuchic 2005). although, real-time location-allocation and routing of late evacuees to shelters are subject to a range of other objectives, including capacities, distances, and susceptibility or vulnerability to the disaster, which should also be simultaneously optimised while operating within a range of stringent constraints. (li et al. 2012; negreiros and palhano 2006). nevertheless, the most challenging objective of emergency situation management is cited as the quick and safe transportation of late evacuees to shelters within a very tight time window. to deal with such a problem, the key questions required to be tackled including when, where, and how to safely transfer the late evacuees from assembly points within a short time window under stringent conditions? real-time assignment of evacuees to a shelter is affected by a range of factors, including its capacity, distance, and susceptibility or vulnerability to the hazard (li et al. 2012; negreiros and palhano 2006). furthermore, other objectives should also be simultaneously optimised while operating within a range of constraints. allocation of shelters, in terms of optimal location, number and capacity, is critical to emergency planning (alexander 2000). a number of studies have developed mathematical formulations to discuss transit-based evacuation under both unpredictable (e.g., earthquake; see (sayyady 2007)) and predictable (e.g., cyclone; (chan 2010; margulis et al. 2006)) disasters. in addition, several studies consolidated location-allocation problem with routing problem (chan 2010; he et al. 2009; mastrogiannidou et al. 2009). concerns over unplanned evacuation resources such as lack of s136 australasian journal of information systems shahparvari, chhetri, abareshi, abbasi & alahakoon 2015, vol. 19, pp. s133-s151 multi-objective decision analytics rescue vehicles, shelters availability that may be a reflection of uncoordinated pre-emergency planning. furthermore, sufficient attention has not been given to the integration of transitbased systems with the emergency evacuation location-allocation in the frame of or/ms (operations research/management science). the focus of this research is therefore on analysing the operational aspects of the late evacuation process, that is, the transfer of people from assembly points to the designated shelters. in conclusion, there are only a few studies that have employed multi-objective optimisation models for late evacuation problems (shahparvari et al. 2015a; stepanov and smith 2009). the majority of studies have focused on minimising the total evacuation time without considering other objectives such as resource utilisation or area coverage. therefore, this study aims to apply an optimisation approach to deal with the multi-objective evacuation problem to generate the possible solutions to improve emergency evacuation planning in case of bushfires. 3 mathematical formulation a typical evacuation problem contains various objective functions, uncertainties and constraints, which makes the multi-objective programming method much more appropriate. hence, in this paper, a multi-objective (mp) method is utilised to formulate the problem. 3.1 modelling assumptions for initialising model parameters, the following assumptions are set out to represent bushfire emergency evacuation conditions. shelters are pre-designated by country fire authority. number and capacity of shelters and rescue vehicles are finite. the late evacuee population in each assembly point are known from real population data. evacuees can be transferred to more than one shelter. access to some routes is restricted by bushfire. the transfer time between assembly points and shelters is known and computable. 3.2 notations i index for assembly points (townships and origins); j index for candidate shelter areas (safe places or destinations); k index for vehicle types; i set of assembly points; j set of candidate places of shelters; 3.2.1 parameters capj capacity of shelter j; di population of people (late evacuees) in assembly point i to be evacuated; j binary variable; 1 if shelter j is accessible, 0 otherwise; ij binary variable; 1 if road is not disrupted between assembly point i and shelter j, 0 otherwise. tij estimated transportation time between assembly point (township) i and shelter j; p total maximum number of required shelters; tvk total maximum number of vehicle type k; vck capacity of vehicle type k; s137 australasian journal of information systems shahparvari, chhetri, abareshi, abbasi & alahakoon 2015, vol. 19, pp. s133-s151 multi-objective decision analytics twi time window for evacuation people from assembly point i; qk usage cost of vehicle type k; w weighted sum coefficient 3.2.2 decision variables xij denotes the number of affected people transferred from the assembly point i to the designated shelter j denotes the number of rescue vehicles type k that is required to be utilised in order to transfer the evacuees from assembly point i to shelter j. 3.2.3 auxiliary decision variables yj (binary variable) if it is possible to transfer the evacuees to the designated shelter j it takes 1 and 0 otherwise. sj (binary variable) if a candidate shelter j is assigned to serve the evacuees, it takes 1 and 0 otherwise. 3.3 objective functions the following two objective functions are optimised simultaneously subject to constraints on resources and network structure: to maximise the number of people transferred to a designated shelter within the defined time windows to minimise usage of number of available resources (i.e. the number of assigned shelters and rescue vehicles). 3.4 mathematical model the developed formulation as a multi-objective model comprises two objective functions and a range of constraints as follows: = (1) the first objective function shown in (1), maximises the number of evacuees assembled in assembly point i which must be transferred toward shelter j in minimum time and across the shortest route. = + (2) the second objective function (2) minimises total number of designated shelters and rescue vehicles. this goal is set to decrease expenditures of allocating new facilities and to distribute evacuees by utilisation of minimum available shelters and rescue vehicles. w1 and w2 as auxiliary coefficients, are utilised as a weighted sum method and are selected in the range of (0, 1) to represent weight of impression for each parameter (i.e. shelter and vehicle). 3.5 constraints the objective functions are subject to the following constraints: (3) constraint (3) ensures that the number of allocated shelters is less than number available shelters. j (4) constraint (4) determines if it is possible to assign a shelter (yj) among all possible candidate places (sj). j (5) s138 australasian journal of information systems shahparvari, chhetri, abareshi, abbasi & alahakoon 2015, vol. 19, pp. s133-s151 multi-objective decision analytics constraint (5) ensures that people at assembly point i would be transferred to shelter j, only if the shelter j is accessible and available. also, this constraint measures that transferred people to shelter j does not exceed the shelter capacity. i,j (6) constraint (6) defines that people at assembly point i can be transferred to shelter j if the connective road between i and j is accessible and available. (2 × ) i,j (7) constraint (7), as time window constraints expresses that total evacuation time (2×tij) of each assembly point i should not be more than its available time window considering available rescue vehicles and their limited capacities. k (8) constraint (8) measures the numbers of allocated rescue vehicle type k to transfer people from each assembly point i to shelter j to not exceed the total number of available vehicles. i (9) constraint (9) restricts the number of transferred peopled from assembly point i to shelters j to their capacities. , 0 i,j,k (10) constraint (10) expresses the negative numbers could not be considered for the integer variables., = 0 1 j (11) constraint (11) restricts the assignment of shelters and transferring issues to binary values, as sj, yj are either allocated or not. 4 case study: lake eildon national park the problem is formulated and applied to a real case study of a bushfire prone area in northeast of victoria in australia. the area of study contains six townships (eildon, thornton, mainton, alexandra, acheron, and bonnie), which are located near the lake eildon national park. with a total population of about 1036 people, this hilly region is thus not densely populated. based on the analysis carried out by cfa, approximate one-third (33%) of residents were affected by the black saturday bushfires, majority of which could be deemed late evacuees (teague et al. 2009). four potential safer townships (i.e. taggerty, merton, yarck, and yea) are nominated by cfa to shelter late evacuees during an emergency evacuation situation. shelters are assumed to have a finite capacity to accommodate the evacuees (table 1) (e.g. the merton public cricket ground oval can shelters 450 persons). real transportation network and travel time between the townships are computed. therefore, the travel time between any two nodes the network are calculated based on real distances and travel speed zones. shelters availability and roads connectivity are affected by the direction and path of bushfires. hence, time windows are defined to prioritise the bushfire arrival time. time windows are calculated on the basis of wind direction and distance between bushfire ignition point and townships using the average bushfire spread rate, which is approximately 20 kilometres per hour (teague et al. 2009). also, two different types of vehicles (bus can carry 40 and van 10 people) are incorporated. for the base case, 20 buses and 30 vans are available to be assigned to gain the optimal routing process. also, to balance vehicles assignment, a usage cost for each type of vehicle is considered as 100 financial unit for bus and 40 for the van. s139 australasian journal of information systems shahparvari, chhetri, abareshi, abbasi & alahakoon 2015, vol. 19, pp. s133-s151 multi-objective decision analytics townships [i] (origins) population [ ] (numbers) time windows [ ] (mins) shelters [j] (destinations) capacity [ ] (numbers) traversal time [ ] (mins) bonnie 171 150 taggerty 350 28.8 14.4 24.6 31.2 mainton 29 90 merton 450 18 27.6 17.4 18 alexandra 398 105 yarck 500 10.8 20.4 10.2 10.8 acheron 96 120 yea 500 4.8 26.4 16.2 16.8 thornton 121 75 7.2 28.2 18 18.6 eildon 221 30 15.6 34.8 24.6 25.2 table 1 summary of inputs and assumptions 4.1 development of bushfire scenarios three comprehensive what-if bushfire scenarios are considered. it ranges from a simple through to more complex bushfire scenarios, as described in table 2. bushfire scenario intensity wind direction and road conditions road disruptions bushfire scenario a low south-eastern to north-western wind direction spreads the bushfire and disrupts 2 roads (northern mainton rd) (back eildon rd) bushfire scenario b medium wind direction changes to east – west and restricts 4 main roads (northern mainton rd) (southern mainton rd) (goulburn valley hwy) (back eildon rd) bushfire scenario c high bushfire spotted in three different points and as a result of a north-eastern to south-western wind direction, 7 roads are disconnected (northern mainton rd) (maroondah hwy) (southern mainton rd) (mainton rd) (taggerty-thornton rd) (goulburn valley hwy) (back eildon rd) table 2 bushfire what-if scenarios also, figure 3 illustrates the bushfire scenarios accompanied by bushfire-spread isochrones. s140 australasian journal of information systems shahparvari, chhetri, abareshi, abbasi & alahakoon 2015, vol. 19, pp. s133-s151 multi-objective decision analytics bushfire scenario b medium intensity bushfire scenario a low intensity bushfire scenario c high intensity 31.2' 24.6' 24' 16.8' 18' 28.8' 7.2 18' 27.6' 17.4' 10.2' 20.4' 10.8' j2 10.8' 4.8' 16.2' 26.4' 18.6' 15.6' 24.6 25.2' 18 28.2 31.2' 24.6' 24' 16.8' 18' 28.8' 7.2 18' 27.6' 17.4' 10.2' 20.4' 10.8' 10.8' 16.2' 26.4' 18.6' 15.6' 24.6 34.8' 25.2' 18 28.2 31.2'' 17.77 4'' 4''' '' 28.88 2 31.2' 24.6' 24' 16.8' 18' 28.8' 7.2 18' 27.6' 17.4' 10.2' 20.4' 10.8' 10.8' 4.8' 16.2' 26.4' 18.6' 15.6' 24.6 34.8' 25.2' 18 28.2 31.2'' 24 17.444444444'''''''''' '' 28888888888888888888888888888.88 2 34.8' j3 i1 bonnie i2 mainton i6 eildon i5 thornton i4 acheron i3 alexandra j4 j1 merton yarck yea taggerty j2 merton taggerty j1 j4 yea j3 yarck i6 eildon i2 mainton i4 acheron i5 thornton i3 i1 bonnie i4 acheron i1 bonnie i6 eildon i2 mainton i5 thornton i3 alexandra taggerty j1 j2 merton j4 yea j3 yarck 4.8' alexandra figure 3 bushfire scenarios 5 solution approach the multi-objective mixed-integer problem solutions are not typically straightforward. in other words, there is not generally a feasible absolute optimal solution enables to optimise all objective functions concurrently (ehrgott 2006). to take such a problem, pareto-optimal solutions have attracted much more attentions that typically referred to as ‘‘a posteriori’’ or ‘‘non-dominated solution generation’’. pareto front optimal solutions are sort of common moips (multi-objective integer problems) solution approaches that attempt to optimise the main objective by degrading at least one of the other objectives. the epsilon constraint method is sort of common pareto front solution techniques, which has been widely employed to solve similar mixed integer multi-objective problems (vira and haimes 1983). in this approach, the main objective of problem out of n is chosen as to be optimised while remaining objectives are converted into form of inequality constraints by assigning allowable levels of epsilon as an upper or lower bound given target values. all other objective functions will be minimised while are constrained to the first objective function value. therefore, this method provides a vast range of flexibility for the decision makers by varying s141 australasian journal of information systems shahparvari, chhetri, abareshi, abbasi & alahakoon 2015, vol. 19, pp. s133-s151 multi-objective decision analytics the lower or mathematical terms, decision maker let fn(x) be the objective function selected among m objective functions to be optimised, the multi-objective problem is transformed as follows: ( ) ( ) p {1,2,…,m} (12) where s is feasible solution space. therefore and given all of these, the epsilon constraint approach was chosen as an efficient method to solve the research problem. in this paper model, the main objective is set to save more human lives and evacuate more people within the time windows. therefore, the first objective f1 is chosen as the main objective to be optimised, while, f2 is converted into a hard constraint by the application of the constraint method. respectively in the next step, the second objective function will be minimised. step 1: this step aims to evacuate the entire evacuee population by minimising the number of assigned shelters and rescue vehicles within the bushfire-prone areas. hence, the model will solve considering the objective function f2 as hard constraint of covering the entire evacuee population. the output of this step determines preliminary minimised values for parameters of sj as number of required shelters and as number of required rescue vehicles that required evacuating all the evacuees. respectively, these initial values will be used as an upper bond in the next step to calculate the optimal values. the model formulation will be converted as follows: ( ) (13) subject to constraints (3, 4, 5, 6, 7, 8, 10, 11) = i (14) step 2: this step optimises the initial values for allocated resources while maximising the number of evacuated people. hence, values for number of required shelters (sj) and rescue vehicles ( ), which were obtained from the previous step will be used as a upper bond for total number of shelters (p) and total number of available vehicle type k ( ). in reality the value of p and are usually known and given by the fire agencies based on maximum available budget, resources and capacities. it is obvious that by given the preliminary values for these parameters, the first step is not necessarily required. so the preliminary values of the aforementioned parameters are set in the constraints. also, penalty value is considered to adjust the constraints (equation 12). consequently, the proposed multi-objective model is converted into the following single-objective problem: (x) subject to ( ) (15) the value of is determined as follows:= . (16) here [0,1] indicates the maximum percentage of total penalty. in this problem, the value is selected randomly in the range of (0.4, 0.9) at the discretion of decision makers. besides, value of p is set by multiplying by the initial value of sj. respectively, is calculated by multiplying of by the initial value of . 6 results and analysis the main aim of this section is to demonstrate how the proposed methodical formulation can be utilised to improve the performance of the evacuation process in an emergency situation to s142 australasian journal of information systems shahparvari, chhetri, abareshi, abbasi & alahakoon 2015, vol. 19, pp. s133-s151 multi-objective decision analytics save lives of more human beings. therefore, a real case study in three bushfire scenarios was comprehensively examined and presented to demonstrate the problem methodology. also, sensitivity analysis against changing the key parameters such as number of required vehicles and number of assigned shelters is provided. furthermore, the shelter capacity usage in each scenario is investigated is this section. the model was implemented using the cplex solver 12.6. according to the proposed solution approach, the outputs are as follows: step 1: preliminary number of available resources (shelters and rescue vehicles) the objective value for the first step solution is 87.9 that represents the minimised cumulative value of assigning facilities and resources to evacuate the evacuees entirely. also, the preliminary output of the solving model in this step indicates that all 4 available shelters (j1, j2, j3, and j4) should be utilised to cover the evacuation demand of late evacuee population within the time windows. besides, results show that at least 23 rescue vehicles including 13 buses and 10 vans are required to transfer all the evacuees. in this case study, the optimal output was obtained by assuming 0.95 for parameter w1 and 0.05 for w2. obviously the higher assigned value for w lead to stronger impact of the associated parameter in the model. step 2: optimal number of shelters, vehicles and routing 6.1 optimal evacuees’ transportation to the assigned shelters in this step, the model is solved while is constrained to the preliminary values of required resources values that were achieved in the first step. table 3 represents the optimal distribution of evacuees within the accessible routes in transportation network in each scenario. respectively, table 3 represents the optimal assignment of the required vehicles followed by the number of required trips to evacuate the evacuee population. bushfire scenario a low intensity bushfire scenario b medium intensity bushfire scenario c high intensity (evacuated people) (evacuated people) (evacuated people) 0 100 71 0 0 100 0 71 0 0 100 71 0 29 0 0 29 0 0 0 29 0 0 0 100 100 100 98 100 100 100 98 100 100 100 98 96 0 0 0 0 0 96 0 0 0 96 0 52 0 0 69 100 0 0 21 0 0 41 80 76 0 97 48 100 0 21 100 100 0 97 24 u se d c ap 324 229 268 215 u se d c ap 329 200 217 290 u se d c ap 229 100 434 273 (shelters assignment) (shelters assignment) (shelters assignment) 1 1 1 1 1 1 1 1 1 1 1 1 table 3 optimal evacuation plan to the assigned shelters as it is shown in table 3, eildon (i6) has the minimum time window and people should be evacuated within 30 minutes before bushfire reaches there. therefore, in bushfire scenario a, merton (j2) is not accessible and there are no transportation to there. the optimal plan for this township is to transfer 76 people to taggerty (j1), 97 people to yarck (j3) and the rest of 48 evacuees to yea (j4). in scenario b, bushfire intensity is medium and goulbourn valley highway and mainton road also are blocked. regarding to the new road disruptions and scenario’s constraints as time windows, shelter capacities and vehicles availability, the optimal emergency evacuation plan is to evacuate 100 people to shelter j1, 21 people to j3 and rest of 100 people to shelter (j4). respectively, the optimal evacuation routing plans for other hazardous townships are presented in table 3. also, figure 4 visualises the optimal emergency evacuation routing and distribution of late evacuees in each scenario. s143 australasian journal of information systems shahparvari, chhetri, abareshi, abbasi & alahakoon 2015, vol. 19, pp. s133-s151 multi-objective decision analytics bushfire scenario c high intensity bushfire scenario b medium intensity bushfire scenario a low intensity i6 71 100 52 29 100 100 98 100 96 69 76 97 48 100 71 29 98 100 100 100 96 80 100 96 24 41 71 100 29 100 100 100 98 j3 100 21 100 21 100 j2 j4 j1 j4 j3 j2 j1 j1 j4 j3 j2 i1 i2 i3 i5 i4 i1 i2 i6 i4 i5 i3 i5 i4 i6 i3 i1 i2 acheron alexandra bonnie mainton eildon thornton merton yarck yea taggerty acheron bonnie mainton eildon thornton merton yarck yea taggerty acheron alexandra bonnie mainton eildon thornton merton yarck yea taggerty 96 alexandra figure 4 optimal evacuation routing plan 6.2 optimal number of assigned rescue vehicles and trips respectively, table 5 represents the optimal assignment of vehicle and number of trips that are required to evacuate the late evacuee population. for example, in scenario a, eildon has the shortest time window. therefore, 2 buses are assigned to transfer 76 evacuees from eildon (i6) to taggerty (j1). accordingly, 97 people are transported to yarck (j3) by 4 buses in one-way trip and the rest of 48 evacuees are evacuated by assignment of 2 buses and 1 van traveling 1 and 5 times between eildon (i6) and yea (j4) simultaneously to transfer eildon population within the 45 minutes predefined time window. as it can be realised from the results, due to the predefined route disruption in the northern mainton road, evacuees could not be transferred to merton (j2). in the scenario b, 100 people are evacuated to taggerty (j1) by assignment of 3 buses trip 3 times while 1 bus is assigned to serve evacuation process from i6 to j3. s144 australasian journal of information systems shahparvari, chhetri, abareshi, abbasi & alahakoon 2015, vol. 19, pp. s133-s151 multi-objective decision analytics bushfire scenario a low intensity bushfire scenario b medium intensity bushfire scenario c high intensity (number of vehicles) (number of vehicles) (number of vehicles) 0 2v 1b 0 0 2v 0 1b 0 0 1b 1b 0 2v 0 0 2v 0 0 0 2v 0 0 0 1b 1b 2v 1b 1b 1b 2v 1b 1b 1b 2v 1b 1v 0 0 0 0 0 1b 0 0 0 1b 0 1v 0 0 1b 2v 0 0 2v 0 0 2v 1b 2b 0 4b 2b 3b 0 1b 4b 2b 0 4b 1b 1v 1v 1v number of vehicle trips number of vehicle trips number of vehicle trips 0 10 2 0 0 10 0 2 0 0 3 2 0 4 0 0 2 0 0 0 4 0 0 0 3 3 10 3 3 3 10 3 3 3 10 3 10 0 0 0 0 0 3 0 0 0 3 0 6 0 0 1 10 0 0 4 0 0 6 2 2 0 4 2b 3 0 1 1b 2 0 4 1b 5v 10v 3v table 4 optimal assignment of rescue vehicles and number of trips (v= van, b = bus) finally, due to limitation in number of available resources and route disruptions, combination of 4 buses and 1 van are assigned to evacuate the remaining 100 evacuees to yea (j4). in the same way for eildon at the scenario c and due to severe road disruptions and longer traversal times to reach the accessible shelters, the optimal vehicle assignment is to transfer population by 6 buses and 1 van towards the defined shelters. slightly further, thornton (i5) evacuees must be evacuated within 75 minutes to the safest and closest shelters. in the scenario a, this process is planned to be fulfilled by designation of one van trips 6 times transferring 52 evacuees to shelter (j1) plus 1 bus transferring 69 people to shelter (j4) trips 2 times. while in the scenario b, the optimal vehicles assignment is to consider 2 vans travel 10 times between i5 and j1 and 2 vans travel 4 times between i5 and j4. interestingly, in the scenario c, all the evacuees are evacuated to shelter j3 and j4 by the application of 2 vans travel 6 times and 1 bus travels 2 times to optimise the number of the utilised vehicles within the limited time windows. the results indicate that mainton evacuees have more time and are evacuated by assigning fewer vehicles that travel more. figure 6 illustrates a better representation for the optimal emergency evacuation vehicle assignment and number of trips in each scenario. s145 australasian journal of information systems shahparvari, chhetri, abareshi, abbasi & alahakoon 2015, vol. 19, pp. s133-s151 multi-objective decision analytics bushfire scenario c high intensity bushfire scenario a low intensity i6 2 10 6 4 10 3 3 3 10 1 2 4 2 bus 5 van j4 j3 j2 j1 i1 i2 i3 i5 i4 33 1 bushfire scenario b medium intensity 2 10 4 10 10 3 3 j3 3 4 3 2 1 bus 10 van j2 j4 j1 i1 i2 i6 i4 i5 i3 acheron 101100000001000100000000 n alexandra 2 3 2 10 3 3 3 3 2 2 4 1 bus 2 van 6 j1 j4 j3 j2 i5 i4 i6 i3 i1 i2 bonnie mainton eildon thornton merton yarck yea taggerty drdrdrdrddrddddrddddrdddrd aaadd aaa3 acheron bonnie mainton eildon thornton merton yarck yea taggerty acheron bonnie mainton eildon thornton merton yarck yea taggerty alexandra alexandra figure 5 number of vehicles and trips for evacuation 6.3 objective function (number of evacuated people) as the results show (table 5), based on predefined bushfire scenarios, all the evacuees could safely be transferred to the safe shelters. solving the model in the first step, results 88.8 for the objective function in scenario a. in scenario b, the first objective function value equals to 91.7 while due to bushfire conditions, the objective function value in scenario c is increased to 95.8. the increase in the preliminary objective values is caused by decreasing the transportation network size aftermath of bushfire propagation. in all the scenarios, however, the final objective value is 1036 and indicates that all the 1036 late evacuees in hazardous townships are safely routed and sheltered by optimal assignment of the available resources. 6.4 number of required vehicles the results of the first step solution of ‘scenario a’ indicates that minimum 13 buses and 10 vans are required to transfer the entire evacuee’ population. however, the proposed figures are not necessarily optimal. hence, in the second step and by the application of the -constraint method, the number of required vans is decreased to 9 vans to optimise the number of available s146 australasian journal of information systems shahparvari, chhetri, abareshi, abbasi & alahakoon 2015, vol. 19, pp. s133-s151 multi-objective decision analytics resources. in the same way in scenario b and c, the total numbers of assigned vehicles are optimised. for example, the total optimal number of required buses in ‘scenario b’ is 13 buses while there is an increase in number of vans to mitigate the impact of extra road disruptions and cover the evacuation demands. accordingly, in scenario c, more vehicles are assigned to speed up the evacuation process. despite eildon (i6) is ranked as the second township in terms of the number of evacuees, the highest number of the vehicles in all scenarios are assigned to eildon (i6) to evacuate people regarding to its short time window and long distance to nearest accessible shelters. therefore, the model has assigned more vehicles to accelerate the evacuation process in eildon. bushfire scenario a bushfire scenario b bushfire scenario c first step objective value 88.8 91.7 95.8 first step tv 13 bus, 10 van 14 bus, 10 van 15 bus, 12 van optimum objective value (evacuated people) 1036 1036 1036 uncovered people 0 0 0 tvk optimum number of required vehicles 13 bus, 9 van 13 bus, 11 van 14 bus, 7 van table 5 objective functions results 6.5 number of required shelters figure 6 provides the results of sensitivity analyses. results indicate that due to predetermined evacuation time windows, road blockages, distances and capacities, at least 4 shelters are required to be assigned to evacuate all evacuees in all the scenarios. obviously, assigning more shelters increases the total objective function value. figure 6 number of required shelters figure 7 depicts the percentage of usage capacity of the designated shelters for each scenario. in scenario a, due to the close distance to the townships, most evacuees are sheltered at taggerty (j1). however, considering the increase in number of road disruptions in scenario b, the usage percentage of this shelter has decreased to 49.43% and evacuees are transferred to other shelters. finally, in the scenario c, most evacuees are evacuated to yarck (j3). the capacity usage of other shelters is shown in figure 7. 0 100 200 300 400 500 600 700 800 900 1000 1100 0 1 2 3 4 5 6 7 8 …n um be r of e va cu at ed p eo pl e number of allocated shelters number of required shelters bushfire scenario a bushfire scenario b bushfire scenario c 1036 s147 australasian journal of information systems shahparvari, chhetri, abareshi, abbasi & alahakoon 2015, vol. 19, pp. s133-s151 multi-objective decision analytics figure 7 shelters capacity usage 6.6 distribution of evacuees to assigned shelters in the scenario a, people are relocated to the assigned shelters normally, however, due to distance, limited time windows of assembly points and the northern-mainton road disruption, most evacuees (31%) are evacuated to taggerty (j1). in the scenario b, goulburn valley highway disruption has impacted the transportation distribution and 353 people (34%) are planned to be transferred to yea (j4) to maximise evacuated people in evacuation process (figure 8). finally in scenario c, severe bushfire conditions have heavily disrupted the transportation network. beside disruptions in the two arterial highways (maroondah and goulburn valley), taggerty-thornton road as the only direct linkage to taggerty (j1) is disrupted. consequently, most late evacuees (42%) are evacuated to yarck (j3) instead of yea (j4). nevertheless, regarding limited capacity of yarck (j3) rest of the evacuees was planned to be transported to the closest available shelters such as yea (j4) and taggerty (j1). due to maroondah highway blockage and the shortage of time, only 10% of population could be evacuated to merton (j2). figure 8 distribution of evacuees to the assigned shelters 7 conclusion this paper developed a novel optimisation model to enhance the capacity of the emergency agencies to response to a bushfire. the model incorporated the key parameters and taggerty merton yarck yea bushfire scenario a low intensity 92.57% 50.89% 53.60% 43.00% bushfire scenario b medium intensity 49.43% 50.89% 55.60% 71.20% bushfire scenario c high intensity 65.43% 22.22% 86.80% 54.60% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% u sa ge p er ce nt ag e shelter capacity usage tagerty 31% merton 22% yarck 26% yea 21% bushfier scenario a low intensity tagerty 17% merton 22% yarck 27% yea 34% bushfire scenario b medium intensity tagerty 22% merton 10% yarck 42% yea 26% bushfire scenario c high intensity s148 australasian journal of information systems shahparvari, chhetri, abareshi, abbasi & alahakoon 2015, vol. 19, pp. s133-s151 multi-objective decision analytics assumptions to adjust the potential impact of a range of constraints such as time window, road disruption and resource capacity. using a real case study in victoria, this model has dynamically optimised emergency evacuation routines including evacuation of late evacuees, dynamic selection of routing and, vehicle assignment and shelter allocation. timely evacuation, distance to the nearest accessible shelters, and the number and capacity of required rescue vehicles are the key decision criteria to formulate the problem. the problem is formulated as a mixed-integer multi-objective linear programming model, which maximises the number of evacuees from towns at a higher risk to safe shelters by minimising available shelters and vehicles. the -constraint method is applied using a three-step solution framework to solve the model. the solution method firstly prioritised the objective functions and then converted the main model to two sub-models to hierarchically optimise the objective functions. using three bushfire scenarios, the model optimises the number transferred evacuees, vehicle utilisation, shelter allocation and optimal routing. these scenarios include different wind directions influence bushfire propagation direction, speed and intensity to disrupt egress links. the results show that all late evacuees (a total of 1036) can be evacuated within the time window assigned to different townships and available resources. however, a minimum of four operating shelters are needed at all times to absorb the late evacuees in the area. with three shelters, 91 out of 1,036 people could not be evacuated. five shelters, on the other hand, will add an additional capacity but will require additional financial resources whilst remaining underutilised. in all the scenarios, yarck plays a key role in providing refuge to evacuees because of its centrality to serve the region. it is worth nothing that beside the evacuee’s protective decisions, evacuation time limitations and available resources, there are other factors that may affect the evacuation process. among them, bushfire propagation risks, multiple roads and traffic congestions are most critical ones, which need to be considered in future research. references alexander, d. e. 2000. "confronting catastrophe: new perspectives on natural disasters,"). arnol, c. 2007. "firefighting operations on the urban interface. fellowship report 20081," the winston churchill memorial trust of australia). butry, d. t., mercer, e., prestemon, j. p., pye, j. m., and holmes, t. p. 2001. "what is the price of catastrophic wildfire?," journal of forestry (99:11), pp. 9-17. cameron, p. a., mitra, b., fitzgerald, m., scheinkestel, c. d., stripp, a., batey, c., niggemeyer, l., truesdale, m., holman, p., and mehra, r. 2009. "black saturday: the immediate impact of the february 2009 bushfires in victoria, australia," med j aust (191:1), pp. 1116. cfa definitions. 2012. "definitions." 2013, from http://www.cfa.vic.gov.au/warningsrestrictions/fire-definitions/ chan, c. p. 2010. "large scale evacuation of carless people during short-and long-notice emergency,"). church, r. l., and cova, t. j. 2000. "mapping evacuation risk on transportation networks using a spatial optimization model," transportation research part c: emerging technologies (8:1–6), pp. 321-336. cohon, j. l. 2004. multiobjective programming and planning. doverpublications. cova, t. j., dennison, p. e., and drews, f. a. 2011. "modeling evacuate versus shelter-in-place decisions in wildfires," sustainability (3:10), pp. 1662-1687. ehrgott, m. 2006. "a discussion of scalarization techniques for multiple objective integer programming," annals of operations research (147:1), pp. 343-360. s149 australasian journal of information systems shahparvari, chhetri, abareshi, abbasi & alahakoon 2015, vol. 19, pp. s133-s151 multi-objective decision analytics ellis, s., kanowski, p., and whelan, r. 2004. national inquiry on bushfire mitigation and management. he, s., zhang, l., song, r., wen, y., and wu, d. 2009. "optimal transit routing problem for emergency evacuations," transportation research board 88th annual meeting. kharuk, v., kasischke, e., and yakubailik, o. 2007. "the spatial and temporal distribution of fires on sakhalin island, russia," international journal of wildland fire (16:5), pp. 556-562. li, a. c. y., nozick, l., xu, n., and davidson, r. 2012. "shelter location and transportation planning under hurricane conditions," transportation research part e: logistics and transportation review (48:4), pp. 715-729. lindell, m. k. 1995. "assessing emergency preparedness in support of hazardous facility risk analyses: application to siting a us hazardous waste incinerator," journal of hazardous materials (40:3), pp. 297-319. lindell, m. k., and perry, r. w. 2003. communicating environmental risk in multiethnic communities. sage publications. margulis, l., charosky, p., fernandez, j., and centeno, m. a. 2006. "hurricane evacuation decision-support model for bus dispatch," fourth laccei international latin american and caribbean conference for engineering and technology (laccet ‘2006), “breaking frontiers and barriers in engineering: education, research, and practice, pp. 21-23. mastrogiannidou, c., boile, m., golias, m., theofanis, s., and ziliaskopoulos, a. 2009. "using transit to evacuate facilities in urban areas: a micro-simulation based integrated tool," transportation research record: journal of the transportation research board (3439), pp. 1-15. negreiros, m., and palhano, a. 2006. "the capacitated centred clustering problem," computers & operations research (33:6), pp. 1639-1663. paveglio, t., carroll, m. s., and jakes, p. j. 2008. "alternatives to evacuation protecting public safety during wildland fire," journal of forestry (106:2), pp. 65-70. pel, a. j., bliemer, m. c., and hoogendoorn, s. p. 2012. "a review on travel behaviour modelling in dynamic traffic simulation models for evacuations," transportation (39:1), pp. 97-123. podur, j., martell, d. l., and knight, k. 2002. "statistical quality control analysis of forest fire activity in canada," canadian journal of forest research (32:2), pp. 195-205. sayyady, f. 2007. "optimizing the use of public transit system in no-notice evacuations in urban areas." mississippi state university. shahparvari, s., chhetri, p., abareshi, a., and abbasi, b. 2015a. "enhancement of short-notice emergency evacuation response during bushfire: a multi-objective epsilon– constraint optimisation approach," international symposium on logistics bologna, italy: isl pp. 10-17. shahparvari, s., chhetri, p., abareshi, a., and abbasi, b. 2015b. "enhancing late evacuation emergency response in bushfire-prone areas: a multi-objective optimization approach," transportation research board washington d.c, usa: trb. southworth, f. 1991. "regional evacuation modeling: a state-of-the-art review". stepanov, a., and smith, j. m. 2009. "multi-objective evacuation routing in transportation networks," european journal of operational research (198:2), pp. 435-446. teague, b., mcleod, r., and pascoe, s. 2009. "victorian bushfires royal commission interim report," parliament of victoria, melbourne). s150 australasian journal of information systems shahparvari, chhetri, abareshi, abbasi & alahakoon 2015, vol. 19, pp. s133-s151 multi-objective decision analytics victorian bushfires royal commission report. 2009. july 2009. interim report, section (7). vira, c., and haimes, y. y. 1983. multiobjective decision making: theory and methodology. north-holland. vuchic, v. r. 2005. urban transit: operations, planning, and economics. westerling, a. l., hidalgo, h. g., cayan, d. r., and swetnam, t. w. 2006. "warming and earlier spring increase western us forest wildfire activity," science (313:5789), pp. 940-943. whittaker, j., mclennan, j., elliott, g., gilbert, j., handmer, j., haynes, k., and cowlishaw, s. 2009. "victorian 2009 bushfire research response: final report," bushfire crc post-fire research program in human behaviour. bushfire crc: melbourne available at http://www.bushfirecrc.com/managed/resource. the emergency evacuation of cities: a cross-national historical and geographical study. rowman & littlefield publishers. zitzler, e., and thiele, l. 1998. "multiobjective optimization using evolutionary algorithms— a comparative case study," parallel problem solving from nature—ppsn v: springer, pp. 292-301. copyright: © 2015 shahparvari, chhetri, abareshi, abbasi & alahakoon. this is an openaccess article distributed under the terms of the creative commons attributionnoncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. s151 microsoft word 1647-5206-1-le.docx australasian journal of information systems duane & corcoran 2018, vol 22, research on recent advances in social media social media & knowledge sharing in heis 1 using social media to enable staff knowledge sharing in higher education institutions niall corcoran limerick institute of technology (lit) niall.corcoran@lit.ie aidan duane waterford institute of technology (wit) abstract higher education institutions (heis) are knowledge intensive environments by nature. however, the management of organisational knowledge and the promotion of staff knowledge sharing is largely neglected in these institutions. this study examines how enterprise social networks can enable staff knowledge sharing in communities of practice in that context. the study is framed as an action research project, covering three cycles over a 12 month period. a conceptual model was developed for empirical testing and data was collected through focus groups and interviews, supplemented by reflective journaling and content analysis. the findings support the conceptual model and provide insight into the antecedents necessary for the creation of an enterprise social network enabled knowledge sharing environment, the motivators for and barriers to participation, and the perceived organisational and individual benefits of increased staff knowledge sharing activity. the findings indicate that the barriers to participation are influenced by the prevalent organisation structure and culture, and a divide between faculty and other staff. however, individual benefits that accrue from participation may influence greater participation, and organisational benefits that accrue may influence organisational strategies that drive change in structure and culture to promote the development of the knowledge sharing environment. a number of findings have practical implications for the management of higher education institutions, such as the evidence of a divide between faculty and other staff, and the perceived existence of an organisational culture that inhibits staff communication, interaction and collaboration. in general, the study findings provide an opportunity for educationalists to better understand the scope and impact of employing social media platforms for knowledge sharing. this study adds to the growing body of work on organisational implementations of social media, and should be of interest to practitioners and researchers undertaking similar projects. 1 introduction the importance of knowledge as a strategic asset to any organisation is increasingly recognised in tandem with a growing awareness that economies and societies have become more information and knowledge intensive (hislop, 2013). this view of the importance of knowledge to contemporary economies and societies is typified by stewart (1997, p. 9), who wrote that knowledge has become the “primary ingredient of what we make, do, buy and sell. as a result, managing it finding and growing intellectual capital, storing it, selling it, sharing it has become the most important economic task of individuals, business and nations.” goffee and jones (2015) assert that the need for organisations to develop their human capital, or knowledge assets, has long been recognised by knowledge intensive businesses, and that there is a long-term correlation between organisational performance and investment in human australasian journal of information systems duane & corcoran 2018, vol 22, research on recent advances in social media social media & knowledge sharing in heis 2 capital. intellectual capital is the sum of everything that everybody in an organisation knows that gives it a competitive advantage, and, according to rao (2010), the intellectual capital of an organisation can only be maximised through the application of knowledge management (km) practices. the importance of knowledge sharing to organisational success is also highlighted by liebowitz (2001), who argues that organisations can attain competitive advantages through encouraging and promoting knowledge sharing. edvinsson (2002) maintains that the nature of competitive advantage is changing and is no longer based on market position, size and power, but on the incorporation of knowledge into all of an organisation’s activities. in general, although public higher education institutions (heis) would not be considered in the same light as profit-driven organisations, neither are they isolated from such contemporary views of the role of knowledge in society. heis are, by their very nature, knowledge intensive environments and their primary raison d’être is to create and disseminate knowledge through teaching, learning and research activities. for a number of economic and social reasons, and driven by national policy, public heis are increasingly being operated and managed as businesses, with growing requirements to drive down costs, improve efficiencies and generate their own income (rowley, 2000; wohlmuther, 2008). heis are required to compete against each other for students in an increasingly competitive, global marketplace, and in order to be successful they need to maximise their competitive advantage (bloom, 2005; cranfield & taylor, 2008). according to swart and kinnie (2003), the need for knowledge sharing is even more important in knowledge-intensive organisations such as public heis, and they must share knowledge held by staff if they are to gain the most from their intellectual capital and compete effectively in the ever expanding, global marketplace. therefore, it would seem logical that km would be a core business strategy of any hei through which they could recognise, manage and use their knowledge assets. for example, kidwell, vander linde, and johnson (2000) believe that heis can derive tremendous value from developing initiatives to share knowledge for the achievement of business objectives, arguing that, if done effectively, km can lead to better decision making capabilities, reduced development cycle time for curriculum and research, improved academic and administrative services and reduced costs. however, studies have shown that, although the external transfer of knowledge is effectively managed by most heis (kok, 2007; pinto, 2012), the management of organisational knowledge and the promotion of staff knowledge sharing is largely neglected, with low levels of km implementation and knowledge sharing evident in these organisations (fullwood, rowley, & delbridge, 2013; ramachandran, chong, & ismail, 2009). a number of reasons for the lack of km implementation, and knowledge sharing in general, in heis can be garnered from the literature, among them: organisational structures (collinson & cook, 2003; j. j. lee, 2007; rowley, 2000); organisational culture (cronin, 2001; j. j. lee, 2007; rowley, 2000); the bureaucratic character of some types of heis (taylor, 2006; tippins, 2003), and a divide and mistrust between academic staff and administration staff (castleman & allen, 1995; conway, 2000; dobson, 2011; szekeres, 2004). this lack of km implementation and knowledge sharing has significant negative impacts on the intellectual capital and consequently the operational performance of heis, similar to any type of organisation, commercial or otherwise. according to ramakrishnan and yasin (2012), speed of curriculum revision and updating, and quality of administrative and support services are particularly impacted. australasian journal of information systems duane & corcoran 2018, vol 22, research on recent advances in social media social media & knowledge sharing in heis 3 1.1 enterprise social networks as a knowledge management tool the importance of social media as a tool for knowledge sharing in organisations is gaining increasing recognition (levy, 2009; von krogh, 2012), where social media are described as computer-mediated tools that allow people to create, share or exchange information, ideas and media in virtual communities and networks (kaplan & haenlein, 2010). according to alberghini, cricelli, and grimaldi (2014), the informality of social media can help employees fulfil their knowledge tasks and objectives, and in many ways, organisational applications of social media are close to some of the ideal principles of km, which include the unrestricted sharing of knowledge, information and data (razmerita, kirchner, & nabeth, 2014). the application of these technologies within the workplaces of organisations to facilitate workrelated communication and collaboration is increasingly referred to as enterprise social networking (friedman, burns, & cao, 2014; richter & riemer, 2013). their use is gaining in popularity and they have been identified as beneficial to the performance and competitive advantage of organisations (leftheriotis & giannakos, 2014; leonardi, huysman, & steinfield, 2013). social media and enterprise social networking (esn) tools bear many, similar characteristics to a km technique known as communities of practice (cop), described by wenger and snyder (2000) as groups of individuals linked together by their enthusiasm for sharing and expanding their knowledge, typically in informal settings and arrangements. according to ardichvili, page, and wentling (2003), cop are efficient tools for knowledge generation and sharing because the majority of an organisation’s intellectual capital is embedded in the intangible, tacit knowledge of its people, and new knowledge can be produced and disseminated in conversations and networking activities. as social media provide many opportunities for the exchange of ideas, information and knowledge (lewis & rush, 2013), this technology has particular relevance to cop and can be used to create what is termed a virtual community of practice (vcop). in vcop, the communication and coordination of work takes place online, facilitated by social media tools, meaning that the community is relational, without reference to a specific location (gammelgaard, 2010). vcop are interactive environments that give their members the chance to engage with other members through a series of tools such as chats, document postings and community discussions at any time and from any place. according to wenger (1999), cop within organisations have been identified as a mechanism to facilitate knowledge exchange between individuals by connecting isolated and distributed pockets of expertise. l. lee, reinicke, sarkar, and anderson (2015) see vcop as offering a way of interacting with a wider range of colleagues, and vcop in business have been shown to be effective in improving knowledge sharing and reducing professional and structural isolation (barnett, jones, bennett, iverson, & bonney, 2012). according to annabi and mcgann (2013, p. 14), the use of esn in vcop has “tremendous strategic potential, as they inherently emphasize strong relationships, encourage social interactions and promote streamlined, widespread communication between community members”. esn tools can offer improvements to the manner in which vcop function and can increase the level of real-time collaboration across organisations. hoffman (2009) finds that social networks can be useful mechanisms for creating communities, and are able to support social learning. a combination of esn applications can create an effective and convenient environment for communication and learning and can help to build community through dialogue and conversation (gunawardena et al., 2009). according to levy (2009), the rise of web 2.0, and social media in particular, has brought a new wave of technologies that should be adopted in km practice. essentially web 2.0 is now australasian journal of information systems duane & corcoran 2018, vol 22, research on recent advances in social media social media & knowledge sharing in heis 4 a collection of social software that supports group interaction towards establishing communities, and creating and exchanging content (von krogh, 2012), and is sometimes referred to as the “social web”. a number of researchers feel that web 2.0 technologies have fundamentally altered the approach to km and it has undergone a paradigm shift from a static knowledge-warehouse approach to a more dynamic communication-based or network approach (faraj, jarvenpaa, & majchrzak, 2011; haefliger, monteiro, foray, & von krogh, 2011). however, using social media, or esn, for km is at an early stage and has some way to go before it gains mainstream acceptance and recognition of its value for the management and sharing of knowledge in organisational settings (yates & paquette, 2011). according to schneckenberg (2009), esn support dynamic knowledge exchange, representing the genuine interests and competence domains of employees, leading to the emergence of corporate information structures through a bottom-up, almost organic approach, which has the potential to bypass the rigid bureaucracy that exists in the majority of heis. ng and pemberton (2013) state that the concept of cop has been extensively examined within the corporate context and can produce many benefits for both individuals and organisations, and, according to wiig (1999), these benefits may also apply to heis. however, the majority of research into using cop in heis has been done in the context of teaching and learning, and consequently very little is known about how cop can benefit the wider hei organisation (kimble, hildreth, & bourdon, 2008). heis are rarely to the fore in the implementation of information systems for either their teaching or corporate practices and, according to leidner and jarvenpaa (1995), academic institutions typically lag behind businesses by about ten years in the adoption of new technologies. this is in contrast to academics themselves, who are known to be early adopters of social technologies (eysenbach, 2011) and research community software (lin, 2012). given that esn are only recently becoming commonplace in organisations, it is hardly surprising then that they have not yet gained a significant foothold in heis. accordingly, there has been little research into how esns might be used to enable knowledge sharing in heis (ortbach & recker, 2014). 1.2 research objective to improve the current knowledge sharing and km levels in heis, the use of esn in vcop is seen to have some promise. in seeking to find ways to improve staff communication, collaboration and knowledge sharing, this study sets about exploring the links between esn and cop, and how they can be used to create a virtual knowledge sharing environment for staff. the formal objective of this research is to examine how enterprise social networks can enable staff knowledge sharing in virtual communities of practice in higher education. the research questions examine the antecedents for a knowledge sharing environment, the dominant problems associated with staff using esn and participating in vcop, the key motivations for participating, and the perceived organisational and individual benefits of a vibrant knowledge sharing environment. a conceptual model was developed from the research questions and a comprehensive review of the literature. the model focuses on how a knowledge sharing environment can be created by determining the antecedents for it and the motivators for staff to engage with it, whilst identifying the dominant problems from both an organisational and individual perspective so that they can be managed. the organisational and individual benefits are also listed as outputs from the model, which is shown in figure 1. the conceptual model adds a level of detail to allow the research questions to be investigated in the contextual setting. the model suggests that the implementation of an esn, and the australasian journal of information systems duane & corcoran 2018, vol 22, research on recent advances in social media social media & knowledge sharing in heis 5 promotion and support of its use in vcop, will enable staff knowledge sharing activities, providing a number of individual and organisational benefits. figure 1. conceptual model 2 research design and implementation the study was undertaken as an ar project, in a public, multi-campus hei in ireland with approximately 6,500 students and 600 staff, over a 12 month period from september 2015 to august 2016. as the research focuses on staff knowledge sharing, the student population was not part of study. the practical aspect of the project involved the implementation of esn tools in the organisation, specifically microsoft’s social networking tool called yammer, and the promotion and support of these to facilitate the establishment of vcop. yammer is a social network that’s entirely focused on a business. it facilitates group conversation and collaboration and has many similarities to familiar social media tools such as facebook and twitter. the lead researcher was part of the host organisation and the approach to the study embraced the researcher’s place within it, fully recognising that a priori knowledge and existing values would invariably intrude upon the observation. from this understanding, it was considered that action research (ar) would be the most appropriate research strategy to adopt for the study. ar involves the active participation of the researcher and seeks to bring about change within the organisation in which it is conducted. it is an iterative process constructed with a longitudinal design to allow time to examine changes as iterations of the research progress (baum, macdougall, & smith, 2006). the ar project engaged three cycles that followed a process of diagnosing, action planning, action taking, evaluating and specifying learning, illustrated in figure 2, and adapted from a model developed by susman and evered australasian journal of information systems duane & corcoran 2018, vol 22, research on recent advances in social media social media & knowledge sharing in heis 6 (1978). the project involved the creation of a communities portal and the use of esn to facilitate the establishment and operation of vcop. the portal acts as a collection point for all of the vcop in the organisation, and allows users to see what communities are active, join communities or create new ones. cycle 1 of the ar project was concerned with the technical establishment of the knowledge sharing environment and setup of a number of vcop. yammer has a feature called groups that directly facilitates the hosting of communities online and provides an environment where they can have conversations, share files, post comments, etc. these features make it a suitable tool to support vcop and it was selected for this project on that basis. following an exploratory process, a number of vcop were setup as yammer groups in conjunction with community leaders staff members who had expressed interest in establishing communities. training on cop and yammer was provided for communities, including specialist training for community leaders. as the number of users and activity on yammer increased, a number of esn champions were identified. these are typically highly active users who are immediately comfortable with using such systems and are seen as crucial to stimulate its growth and attract more users. these users were engaged in order to create a more formal recognition for their role and to empower them to promote the esn in the organisation. figure 2. phases of the ar design cycles 2 and 3 of the ar project produced further packages of interventions which were based on the evaluations of the interventions carried out in the previous cycles. both of these cycles focused on growing the user base on the esn, fostering the development of the established vcop and promoting the establishment of additional vcop. a number of initiatives to increase the number of yammer users were introduced, such as providing additional functionality external to vcop, such as support groups, working groups, department groups, and australasian journal of information systems duane & corcoran 2018, vol 22, research on recent advances in social media social media & knowledge sharing in heis 7 information feeds. this led to a number of groups being established on yammer that might not be considered as cop, such as department groups and academic course groups, with a number of others established for particular purposes, such as organising events and conferences, which could be considered as project groups. the establishment of this type of group is considered important to the long-term viability of the esn as they engage staff who might not otherwise have had a reason to use the platform. awareness campaigns were conducted during both cycles, and these included mass emails, advertisements on information portals and digital signage; webinars and live training sessions. 2.1 data collection and analysis the study itself is qualitative in nature and the primary data collection methods used were focus groups for cycle 1 and semi-structured interviews for cycles 2 and 3. the strategy employed for the data collection was to use the focus groups to prove the validity of the key themes for the data collection (table 1), and to use the data from the focus groups to develop a refined set of questions for the semi-structured interviews. focus group meetings were combined with training for new vcop and the groups were therefore representative of the membership of those communities. this proved a useful way to engage with a significant portion of the yammer user base at an early stage, and the interaction between group members provided rich data. table 1. focus group and interview topics the strategy worked well in practice and the themes were managed through a logical progression from the focus groups, and improved on for the interviews, so that they became close to a natural conversation, providing for further sources of rich data. in selecting 30 subjects for interview, a deliberate effort was made to achieve a sample that was representative of the target population, whilst including all stakeholders. this was achieved by using a combination of stratified and purposeful sampling to ensure a valid mix of faculty and staff, management and non-management subjects, and yammer users and non-users. according to marshall (1996), the most common sampling technique for qualitative studies is judgement or purposeful sampling, where the researcher actively selects the most productive sample to answer the research questions. reflective journaling was used extensively throughout the ar cycles in order to capture interpretations of the interventions for each cycle, and also to capture informal conversations, observations, and anything else to do with the project. the content of a number of conversation threads from the esn was also analysed to determine the depth of engagement of staff with particular communities. australasian journal of information systems duane & corcoran 2018, vol 22, research on recent advances in social media social media & knowledge sharing in heis 8 3 findings in the analysis of the overall data set, the conceptual model and associated research questions were used as a focus, and the findings are presented in the context of these (table 2). every aspect of the conceptual model presented in the data, and the findings confirm the model. table 2. summary of key findings 3.1 antecedents the findings indicate that the willingness of staff to share knowledge and participate in esn and vcop is influenced by a number of factors, almost all of which are inexorably linked to the culture of the organisation. this organisational culture in turn is influenced by a number of factors over time such as the structure and hierarchy, the attitudes and actions of management, and the organisational strategy. the findings indicate that management has an important role to play in helping to change the culture through developing and implementing organisational strategies that stimulate an active knowledge sharing environment. however, attempts by management to gain control of communities that are viewed as successful was described as a potential problem, and any such efforts should be resisted. this indicates that all cop should have a clear terms of reference that defines what the community is for and what the expected outcomes and deliverables are, and that this is fully understood and acknowledged by all stakeholders, including management. structured training should be available for communities during a facilitated start-up phase, and this phase should include face-to-face kick-off meetings. this highlights the need for the development of a blueprint for the creation of cop, detailing the steps involved for community leaders and members, including things like how to organise kick-off meetings. 3.2 organisational and individual problems the structure of the organisation was identified as a major impediment to the development of a knowledge sharing environment, and the size and geographical dispersion of the organisation was also highlighted as problematic. the strongest individual barrier that presented in the data was time, the most cited reason for lack of participation. this applies both to academics, who face an ever increasing teaching workload, and support staff who feel that they are under resourced to adequately provide the required levels of service. fear factors australasian journal of information systems duane & corcoran 2018, vol 22, research on recent advances in social media social media & knowledge sharing in heis 9 are also prevalent, highlighted by a number of interviewees discussing fear and trust issues as inhibitors to participation. some described a lack of understanding of the nature of social media tools by their managers as an inhibitor, fearing that management would not understand why a member of staff would want to participate in a cop that was not part of their cognitive domain: the attitudes of staff towards social media and the terminology used to describe the tools, such as esn, also presented as barriers to use. many staff see social media as something that should only be used outside of work and could not see any application for it in the workplace. the terms used to describe it, such as ‘social media’ and ‘enterprise social networks’ are problematic, either suggesting applications for frivolously chatting in the first instance, or meaning nothing very much at all in the second. 3.3 key motivators although most staff are motivated to participate and recognise both the individual and organisational benefits, in many cases they are either unable or unwilling to break free of the boundaries that the organisational culture places around them. the study findings indicate that in some cases, staff will participate in the knowledge sharing environment and share knowledge willingly and freely if a convenient (online) and meaningful environment is created for them to do so, and they are suitably motivated. the presence of vcop in this environment is crucial as a motivation for participation as these can help to break down the structural and cultural boundaries that inhibit knowledge sharing. in order to sustain the development of vcop and build vibrant communities that will appeal to members and attract new participants, the roles of community leaders and esn champions are vital. a number of interviewees felt that staff of a certain personality type are more inclined to openly share knowledge with, or to seek knowledge from others, and that the activities of these people have a positive effect on the development of cop. this supports the idea of those that see knowledge as a public good (de long & fahey, 2000; hislop, 2013; mclure & faraj, 2000), and are motivated to share knowledge with others. therefore, identifying people in the organisation who have these personality traits, and may have not yet joined the esn, but who might be willing to participate and act as esn ambassadors, is an important future intervention to stimulate the growth of cop. staff are motivated to use esn and participate in vcop if they either find their engagement to be enjoyable, interesting or stimulating, or if they can derive other benefits from participation, such as making their working lives easier or deriving some rewards in terms of recognition or career progression. the possibility of reducing workload is a strong motivational factor and many interviewees spoke about the opportunities that cop presented for not having to reinvent the wheel. 3.4 organisational and individual benefits all of the interviewees were of the opinion that both the use of esn and a cop model would be of significant benefit, both to themselves in their jobs and personal development, and to the organisation. much of the commentary was aspirational in nature and spoke about a desire for change in the culture that might arise from the establishment of an active and vibrant knowledge sharing environment. however, a number of more tangible organisational benefits were also elucidated, such as the possibility for cop to break down the social divides and eliminate the siloes in the organisation. many interviewees had practical examples of cop that they might like to form that may provide considerable organisational and individual benefits. for example, it appears that new staff find it particularly difficult to engage with the organisation, and limited training opportunities are provided, suggesting a need for a human australasian journal of information systems duane & corcoran 2018, vol 22, research on recent advances in social media social media & knowledge sharing in heis 10 resources backed cop to provide an on-boarding and mentoring environment for new staff. a perceived benefit that was particularly prevalent amongst support staff was cop participation helping them to feel more a part of the organisation. many of them feel a sense of detachment from the organisation and this is highlighted by the commentary on the divide between academic and support staff. the benefits of having ready access to knowledge both through vcop and the esn also drew some commentary. yammer is seen as a significant way in which to enhance communications, providing an opportunity to communicate with people who would not normally meet on a day to day basis, and this is a particular benefit for multicampus heis which may have departments and faculties spread across multiple sites. a further theme that emerged from the data was the sense of being able to use cop as a vehicle to get things done within the organisation. it was felt that both individual and group initiatives could be achievable with the support and promotion by communities. 4 discussion the findings are highly interdependent and heavily influence each other (figure 3). for example, according to schein (2010), organisational culture and leadership are two sides of the same coin and neither can be really understood by itself. therefore, although the themes of the key findings are discussed independently, there is a considerable amount of overlap between them. the findings are critically assessed in light of the earlier reviews of the source literature using the connections and interdependencies between them to develop the study’s conclusions. figure 3. interdependencies of findings 4.1 organisational culture and structure the study findings indicate that organisational culture has a complex and crucial relationship with the knowledge sharing activities of the organisation. the link between organisational culture and knowledge sharing, and specifically the willingness of staff to participate, has been the subject of much research in the km field (e.g. annabi & mcgann, 2013; dimicco et al., 2008). it is generally argued that organisational culture can significantly influence knowledge australasian journal of information systems duane & corcoran 2018, vol 22, research on recent advances in social media social media & knowledge sharing in heis 11 sharing activities and according to hislop (2013), most of these studies suggest that it can have a positive influence on knowledge sharing. however, a number of studies highlight the negative impacts that certain organisational cultures can have on knowledge sharing activities (lam, 2005; rai, 2011; suppiah & singh sandhu, 2011). alavi, kayworth, and leidner (2005) suggest that positive cultural values such as openness, sharing and trust will lead to positive behaviours, and negative values will lead to dysfunctional behaviours. therefore, if an organisation wishes to develop an active and useful knowledge sharing environment in which staff openly and freely collaborate and share knowledge, it is necessary for it to understand what type of an organisational culture it has. although this may sound like a straightforward process, in practice it can be difficult to achieve, and ogbonna and harris (1998) conclude that management’s view of what the culture is may not be shared by other members of the organisation. there are a number of organisational culture types identified in the literature, such as cameron and quinn’s (2005) widely cited classification of clan, adhocracy, market and hierarchy cultures, and in examining the links between leadership and organisational culture, bass and avolio (1993) usefully classify organisational culture into two distinct types: transactional and transformational. a transactional organisational culture can be closely linked to a hierarchical organisational culture type and these are rooted in formal organisational structures, processes and procedures, and tend to dominate public sector organisations and heis. according to bass and avolio (1993), a transactional culture focuses on everything in terms of explicit and implicit contractual relationships, and is characterised by staff with short-term interests working as independently as possible from colleagues, and having little identification with the mission and vision of the organisation. this results in a mechanistic organisation with much attention to controls, directions and standard operating procedures. little gets done that is not as a consequence of formal agreements, and risk-taking is avoided, resulting in little or slow progress and change. according to denison and mishra (1995), this leads to a high degree of separation between management and staff, and where open and frank discussions between levels in the hierarchy are discouraged. de long and fahey (2000) maintain that this helps to create a context for communication that undermines effective knowledge sharing. the findings from this study clearly demonstrate that the organisation is firmly in the hierarchy quadrant of the cameron and quinn (2005) model with a wholly transactional organisational culture. the conceptual model highlighted the necessity of having an appropriate organisational culture as an antecedent of a successful and active knowledge sharing environment. annabi and mcgann (2013) conclude that organisations must adopt a knowledge sharing culture that recognizes the value of informal knowledge sharing in vcop using social media tools, and zboralski (2009) highlights the need for having an encouraging environment and culture where knowledge sharing is promoted. the existence of a transactional culture as the dominant culture in the organisation suggests that the development of a successful knowledge sharing environment is not possible. although the ar project led to the creation of a number of vcop and some growth and sustained use of the esn, this could not be extrapolated as a completely successful implementation across the organisation. for this to happen, the organisational culture needs to move towards what bass and avolio (1993) describe as a transformational culture, where employees go beyond their self-interests or expected rewards for the good of the organisation. such organisations are flexible, adaptive, dynamic, and australasian journal of information systems duane & corcoran 2018, vol 22, research on recent advances in social media social media & knowledge sharing in heis 12 informal, and place much emphasis on the potential of its individual employees to grow and improve. rigid organisational structures that promote transactional cultures tend to be the norm in public sector heis and have been in existence in the same format with little change since their foundation, and encourage the creation of silos and lead to staff isolation (bannister, 2001; tippins, 2003). organisations that can create an open and transparent culture help to make employees feel empowered and have a voice, making them feel more connected and loyal to the organisation (lok & crawford, 2004; trice & beyer, 1993). much hei strategic vision and rhetoric is given to the desire of such organisations to be dynamic in operation and responsive to the needs of their markets and customers. however, it is not feasible for large public sector heis to suddenly develop loose, decentralized, flat structures that are common in innovative and creative private sector firms, not least because such organisational structures are largely mandated by government policy and applied at a national level (mccarthy, 2014). therefore, in accepting that organisational structures cannot be readily changed to create the conditions that would help the development of a transformational culture, we must look for other ways within the existing structures for this to happen. for example, a number of authors argue that managing cultural change is difficult and that a natural change of culture is more likely, taking place through the socialization of new staff over time (pascale, 1985; sathe, 1983), and ogbonna and harris (1998) argue that attempts at culture change should focus on means of perpetuations such as communications. it is also possible for culture change to be led through a bottom-up approach, where pockets of excellence and influence can have a significant impact on the overall behaviour of the organisation, through individuals described by pascale and sternin (2005, p. 72) as “positive deviants”, and these can be likened to the roles of community leaders and esn champions. 4.2 academic staff divide the academic staff divide contributes to the fragmentation of the organisation and exacerbates a culture that limits opportunities for staff interaction, collaboration and knowledge sharing. however, the study found that a number of vcop have both faculty and staff membership, and both groups seem quite happy to freely share knowledge and collaborate within their vcop. this indicates that the knowledge sharing environment can provide a means for faculty and staff to collaborate on areas of common interest and help to bridge the divide that exists between them. for this to happen in a meaningful way, and on a scale that can influence a shift towards a transformational culture, it requires management to recognize the existence of the academic staff divide and its impact on knowledge sharing and collaborative activities. according to florenthal and tolstikov-mast (2012), the culture of a hei has a significant influence on the overall education experience of its students, and a strong sense of community amongst staff is critical to achieving a positive organisational culture. this sense of community can only be created in the presence of strong working relations between faculty and staff. the choice then for management is to find opportunities to get faculty and staff to work together in meaningful ways or to perpetuate a divide that simply does not help students or the organisation in any way. the study findings indicate that when the right collaboration opportunities are presented to faculty and staff, they will collaborate and communicate with each other. this suggests that having a knowledge sharing environment complete with vcop as a central knowledge management technique, and supported by esn, should be at the heart of organisational strategies that seek to improve staff communication and knowledge sharing in order to drive organisational efficiencies. australasian journal of information systems duane & corcoran 2018, vol 22, research on recent advances in social media social media & knowledge sharing in heis 13 4.3 the role of management the attitudes, actions and behaviours of leaders and managers have an important role to play in the context of knowledge sharing in heis. in organisations in general, bircham-connolly, corner, and bowden (2005) suggest that leadership has a pivotal role to play in promoting and cultivating knowledge sharing behaviour, mainly by providing opportunities and managing the processes for staff to share and transfer their knowledge. according to s. wang and noe (2010), when management is supportive of knowledge sharing, employees perceive that a knowledge sharing culture is prevalent, indicating that leadership is highly significant in this regard. according to wi (2015), management participation is crucial for successful collaboration to happen within organisations. leaders are aware that they should engage with employees, and particularly through social and digital channels, but they tend not to. there are a number of reasons for this including fear that such engagement would result in a weakening of power relations, reducing their ability to control and command. wi (2015) maintains that collaboration depends on trust and leaders must learn how to trust their staff on platforms such as an esn, although the tools themselves are not as important as management’s understanding of the purpose and nature of the tools. using platforms such as esn requires organisational change, and that change generally needs to be led by management. this requires visionary leadership, defined strategic objectives, and a commitment to lead the organisation through the necessary change. kezar and eckel (2002, p. 457) found a strong relationship between organisational change and culture in the context of higher education, and that leaders who are acting as organisational change agents must become “cultural outsiders” in order to understand their own cultures and their impact on change initiatives, through examination and reflection. fidelman (2012, p. 3.6), refers to the change in culture as organisations becoming “social businesses”, and this requires a new strategy, which takes “time, persuasion, planning, teamwork, and measurable goals”. it is suggested that this process is quite difficult for bureaucratic and hierarchical organisations, of which heis are typical examples. however, once the value to the organisation has been recognized, a vision for a knowledge sharing enterprise and the expected outcomes can be provided. following this, a community management plan can be established where the people, processes, resources and technology are provided to support the vision. lastly, the execution of the plan requires the identification of leaders and champions within the organisation who will help to promote the vision and the changes associated with it. according to zboralski (2009, p. 94), establishing a “knowledge management friendly atmosphere”, which includes the active promotion of cop, will increase employees’ awareness of the need to share knowledge and will encourage cop participation. the findings imply that middle management has a pivotal role to play in the development of knowledge sharing initiatives and their attitudes and behaviour will have a direct impact on staff participation. this is supported by huy (2002), who investigated the emotional commitment to organisational change by middle managers, and ogbonna and wilkinson (2003), who investigated the impact of cultural change initiatives on middle managers. it is suggested that managers with creative abilities are more likely to recognise the potential of esn and vcop and promote their use within their own departments and amongst their staff. in contrast, strictly operational managers are more likely to see the esn as another it tool that has to be negotiated and could potentially hinder participation through discouragement and negativity. having an active executive sponsor can go some way towards mitigating against australasian journal of information systems duane & corcoran 2018, vol 22, research on recent advances in social media social media & knowledge sharing in heis 14 this situation, and increasing awareness through training and information sessions for middle managers can also help in this regard. 4.4 virtual communities of practice according to bolisani and scarso (2014), the concept of cop as a km technique has been widely applied by companies in practice in order to promote knowledge sharing amongst employees to improve business performance. the study findings indicate that vcop can and do emerge from the bottom-up, through the efforts of individuals who are interested in particular subjects and who seek out colleagues with similar interests in order to share their knowledge and experiences and learn from each other. vcop are also more likely to emerge, and in greater numbers, because of the relative simplicity of their creation, facilitated by the esn platform, compared to traditional cop. the findings also suggest that for genuine social interaction to take place in vcop, they need to be relevant, purposeful and appealing in order to stimulate a real desire or need to engage. the knowledge sharing environment also needs to be in tune with the ways people interact on the social web, which embody the underlying open ethos that people enjoy, rather than be a forced environment for conversations. to facilitate this, the selection of the esn is important, and it must provide an interface and functionality that people expect from a social media platform. the provision of a familiar tool has dual benefits of making the system more attractive to users, and reducing the need for extensive training on the platform itself. it does not, however, reduce the need for training in vcop, which is essential to provide members with a complete understanding of what communities are, how they operate, and to provide a set of guidelines or a framework to work within. hislop (2013) states that cop provide staff with a sense of collective identity and this is supported by the study findings where staff who expressed feeling a sense of detachment from the organisation felt that participation in vcop would give them a better sense of engagement and identification with it. as cop are based on shared professional interest and commitment rather than on projects or organisational position (alavi et al., 2005), they provide opportunities for staff members from different departments and disciplines to interact on areas of common interest, and importantly, provide an environment for faculty and staff to interact in ways that would otherwise most likely not arise. 4.5 community leaders and champions according to borzillo, aznar, and schmitt (2011), community leaders are specific people who undertake organising roles with the objective of developing and sustaining the community. in many cases, they are the founders of particular cop and are generally the driving force behind them. the vitality of cop are very dependent on the interest and commitment of their leaders, and communities that do not have dedicated leaders are bound to fail. zboralski (2009) states that community leaders are responsible for motivating other members to participate, coaching new members, and organising and advertising community events, and the more attention that is paid to these tasks, the more intensive the activity of the cop will be. however, it is the most demanding and time-intensive role in a community and community leaders must be prepared to nurture the development of a cop, particularly if they are instrumental in establishing them. the study findings show that during the early growth phase of the esn, the conversations were dominated by a number of individuals who use the technology freely and enthusiastically, and are generally comfortable using social media. this is supported by hart (2015) and consistent with rogers (1995) diffusion of innovations theory. most of them became members of more than one vcop on the esn and participated in discussions in all of the australasian journal of information systems duane & corcoran 2018, vol 22, research on recent advances in social media social media & knowledge sharing in heis 15 communities that they were in. the findings also suggest that esn champions have a distinct personality type that inclines them to openly share knowledge with, or to seek knowledge from others, and that the activities of these people have a positive effect on the development of cop. they take a professional pride in helping people through sharing knowledge and this finding supports the idea that some individuals see knowledge as a public good and will freely share it (hislop, 2013; mclure wasko & faraj, 2005). the identification of these individuals is central to opening up the esn to everyone in the organisation as they, in conjunction with community leaders, keep conversations and activity at levels that are necessary to attract other users and reach a critical mass for sustainability. the role of champions should be clearly identified in an esn strategy and they should be given adequate supports to fulfil their roles, which may simply be providing them with adequate equipment and connectivity to ensure that they have ready access to the esn, and training in their role as an esn champion. 4.6 staff motivation in order to successfully manage the development and growth of the knowledge sharing environment, it is necessary to understand both what motivates the majority of the staff population to participate and what prevents them from participating. therefore, motivations for participation must be examined in conjunction with barriers to participation. the study findings indicate that staff are motivated to participate if they either find their engagement to be enjoyable, interesting or stimulating, or if they can derive other benefits from participation, such as making their working lives easier or gaining some rewards in terms of recognition or career progression. in addition to the cultural barriers that inhibit many staff from participating, there are also a number of individual barriers that emerged from the study, some of which were predicted in the conceptual model and some not. the main individual barriers to participation that present in the study, such as fear factors, lack of time and the age profile of staff, have already been explored to some degree in the literature. one unexpected finding is that the terminology used to describe the online tools (esn and social media) can be problematic and puts many staff off participation because they associate social media with frivolity and activities that should not be associated with work. indeed, many staff see social media as something that should only be used outside of work and do not see any application for it in the workplace. many participants also had no concept of what enterprise social networking actually is, much less envisage it as a collaboration platform that could be used in the workplace. addressing the problem requires organisations who are developing knowledge sharing platforms based on esn technology, to consider a re-branding of the esn with terminology that is more closely associated with community activity. staff should also be educated about the purpose and use of esn in the organisation through awareness and training programmes that emphasise the professional aspects of these applications. 4.7 perceived benefits the study findings show that there is a general perception that participating in the knowledge sharing environment will result in benefits to both the individual and the organisation. the findings also indicate that there is a perception that a successful knowledge sharing environment can positively influence the organisational culture and lead to a more open, creative and sharing organisation, with an increased sense of belonging and loyalty engendered through participation. however, it should be noted that these are perceived benefits and have not been measured as actual benefits that have been derived from participation. although, individual benefits can be empirically explored, organisational australasian journal of information systems duane & corcoran 2018, vol 22, research on recent advances in social media social media & knowledge sharing in heis 16 benefits, such as increased intellectual capital and improved performance, can be difficult to quantify (lópez-nicolás & meroño-cerdán, 2011; z. wang & wang, 2012). cop can deliver different benefits to different types of organisation. for example, lesser and storck (2001) suggest that they have the potential to overcome many of the inherent problems for slow-moving, hierarchical organisations that have to exist in a fast-moving, virtual economy. they are also an effective way for organisations to share knowledge outside of traditional structural boundaries. this suggests that cop would be a beneficial km technique for public sector organisations such as heis to adopt. however, because communities don’t appear on organisational charts and balance sheets, they can only be considered as a hidden asset, and this presents a difficulty in determining how exactly they deliver value. this may also present a problem for highly risk-averse organisations, such as public sector bodies, who generally need to be able to quantify a return before making an investment. rather than attempting to quantify the benefits of a cop model, a better approach may be to for the organisation to develop an understanding of how cop can create value. lesser and storck (2001) suggest that thinking of communities as engines for the development of social capital would be helpful, and argue that the development of social capital in cop leads to behavioural change, resulting in greater knowledge sharing, and this in turn can positively influence organisational performance. similar to cop, the benefits of esn implementation and use for organisations can be difficult to quantify in terms of specific deliverables and direct value. however, there is a growing body of research that argues that esn can bring many and significant benefits to the organisation through increased communication and knowledge sharing, and increased social capital (davison, ou, martinsons, zhao, & du, 2014; leonardi & meyer, 2015). some recent empirical research furthers this by making positive associations between esn use and employee performance (riemer, finke, & hovorka, 2015), and finding that esn can help to overcome some of the barriers to organisational knowledge sharing, such as motivation, and developing and maintaining social ties (fulk & yuan, 2013). for heis, the use of esn is seen to have the potential to promote communication amongst staff and encourage interaction across functional areas, and between faculty and staff (schneckenberg, 2009; zhao & kemp, 2013). the study found considerable evidence to support this in a number of vcop that have both faculty and staff members, where there was open discussion that was independent of boundary and reporting lines. a potential benefit that emerged from the study was the opportunity to use esn and vcop as on-boarding tools to support new staff. a number of participants discussed the lack of support available for new staff, especially faculty, and felt that participating in relevant online communities would allow them to assimilate into the organisation more efficiently. in addition, the availability of the esn as a communications platform would provide them with a space to seek information to help them in their new roles, allowing them a means to tap into the existing organisational knowledge base. the use of esn in this fashion has been investigated by leidner, koch, and gonzalez (2010, p. 229), who found that they can immediately increase the sense of cultural belonging to the organisation, make the environment of entry-level workers exciting, and increase morale amongst a “generation y” workforce. australasian journal of information systems duane & corcoran 2018, vol 22, research on recent advances in social media social media & knowledge sharing in heis 17 5 conclusion there is a growing awareness amongst practitioners and researchers that the implementation of social media in organisations has given a new impetus to km. however, limited research has been conducted on applications of social media for organisational knowledge sharing, and especially in higher education contexts. in examining how esn tools can enable staff knowledge sharing in vcop in a hei, this research has produced a number of findings informing both theory and practice, and which can be used as a basis for further research. one of the outcomes from the ar project was the development of a package of interventions to continue to grow the esn user base and encourage the establishment of more communities, which is necessary to ensure the continuation of the practical element of the project (table 3). implications for practice are presented in table 4, and suggestions for further research in table 5. table 3. package of interventions for esn/vcop management the main limitation of this study was the time frame available for the ar project. according to holtzblatt, drury, weiss, damianos, and cuomo (2013), adoption of new social software in organisations can be very slow, with interactions between users, changes in work practices, and the impact on business outcomes, all taking time to emerge. in order for these ‘long-tail effects’ to be realised, social communities must reach a critical mass, and the impacts are only seen in large populations over long periods of time. the problem is further exacerbated in this context by the nature of the academic year, with significant periods when faculty are largely absent from campus, including a three to four week period in december/january and an eight to twelve week period from june to august. notwithstanding the validity of the empirical testing of the conceptual model, the ability to conduct a similar ar study over a period of two, or even three, academic years, would significantly enhance the results, and would allow for the introduction of more quantitative measures, informing a mixed methods approach. australasian journal of information systems duane & corcoran 2018, vol 22, research on recent advances in social media social media & knowledge sharing in heis 18 table 4. implications for practice. a further limitation of this research may be the generalisability of the findings outside of the subject organisation. however, the size and standing of the hei in question, indicate that it is a valid representative sample to apply generalisability to the conclusions drawn from this study to the wider, national higher education sector. irish heis in general bear many similarities to heis in other jurisdictions, and indeed to other public sector organisations, and the findings may be of interest to the wider higher education community and other elements of the public sector. in addition, many aspects of the conceptual model may be applied to any organisation, including in the private sector, and a number of the related findings may be of interest to practitioners undertaking similar projects. these include the development of clear terms of reference and blueprints for cop, and the development of structured training for both cop and esn. some of the barriers to participation such as lack of time and fear of using social media are also not unique to heis, and the findings should be of interest to others who are using social media in a similar fashion. the use of esn as a means of improving general communications within the organisation should be of practical interest to management, as should the suggestion that esn and vcop could be used as on-boarding and mentoring tools for new staff. the management of heis in particular should be concerned with the strong australasian journal of information systems duane & corcoran 2018, vol 22, research on recent advances in social media social media & knowledge sharing in heis 19 evidence uncovered for the existence of a divide between academic and other staff. the identification of the importance of management support for the establishment and use of esn for vcop and knowledge sharing also has implications for management in heis who wish to improve their km capabilities. table 5. opportunities for further research notwithstanding the evidence for organisational structures and geographical dispersion in a multi-campus environment presenting as barriers to knowledge sharing, there is a perception that the development of a strong knowledge sharing environment can have a positive impact on organisational culture, helping to break down social divides and eliminate silos. studying the further development and the ongoing activities of the esn and vcop will help to see how this can be achieved in practice. in addition, given the extended time-frame required for achieving critical mass with the implementation of any knowledge sharing environment using esn tools, long-term studies are required to fully understand the implications of achieving strategic goals for knowledge sharing, both in terms of the derived benefits to both the organisation and individuals, and also for the culture of the organisation. references alavi, m., kayworth, t. r., & leidner, d. e. (2005). an empirical examination of the influence of organizational culture on knowledge management practices. journal of management information systems, 22(3), 191-224. alberghini, e., cricelli, l., & grimaldi, m. (2014). a methodology to manage and monitor social media inside a company: a case study. journal of knowledge management, 18(2), 255277. annabi, h., & mcgann, s. t. (2013). social media as the missing link: connecting communities of practice to business strategy. journal of organizational computing and electronic commerce, 23(1-2), 56-83. australasian journal of information systems duane & corcoran 2018, vol 22, research on recent advances in social media social media & knowledge sharing in heis 20 ardichvili, a., page, v., & wentling, t. (2003). motivation and barriers to participation in virtual knowledge-sharing communities of practice. journal of knowledge management, 7(1), 64-77. bannister, f. (2001). dismantling the silos: extracting new value from it investments in public administration. information systems journal, 11(1), 65-84. barnett, s., jones, s. c., bennett, s., iverson, d., & bonney, a. (2012). general practice training and virtual communities of practice-a review of the literature. bmc family practice, 13(1), 87. bass, b. m., & avolio, b. j. (1993). transformational leadership and organizational culture. public administration quarterly, 1993(spring), 112-121. baum, f., macdougall, c., & smith, d. (2006). participatory action research. journal of epidemiology and community health, 60(10), 854-857. bircham-connolly, h., corner, j., & bowden, s. (2005). an empirical study of the impact of question structure on recipient attitude during knowledge sharing. electronic journal of knowledge management, 32(1), 1-10. bloom, d. (2005). raising the pressure: globalisation and the need for higher education reform. in g. jones, p. mccarney, & m. skolnik (eds.), creating knowledge, strengthening nations. toronto: university of toronto press. bolisani, e., & scarso, e. (2014). the place of communities of practice in knowledge management studies: a critical review. journal of knowledge management, 18(2), 366-381. borzillo, s., aznar, s., & schmitt, a. (2011). a journey through communities of practice: how and why members move from the periphery to the core. european management journal, 29(1), 25-42. cameron, k. s., & quinn, r. e. (2005). diagnosing and changing organizational culture: based on the competing values framework. new jersey: john wiley & sons. castleman, t., & allen, m. (1995). the forgotten workforce: female general staff in higher education. australian universities' review, 38(2), 65-69. collinson, v., & cook, t. f. (2003). learning to share, sharing to learn: fostering organizational learning through teachers' dissemination of knowledge. paper presented at the annual meeting of the american educational research association, chicago, il, usa. conway, m. (2000). what's in a name? issues for atem and administrators. journal of higher education policy and management, 22(2), 199-201. cranfield, d. j., & taylor, j. (2008). knowledge management and higher education: a uk case study. electronic journal of knowledge management, 6(2), 85-99. cronin, b. (2001). knowledge management, organizational culture and anglo-american higher education. journal of information science, 27(3), 129-137. davison, r. m., ou, c. x., martinsons, m. g., zhao, a. y., & du, r. (2014). the communicative ecology of web 2.0 at work: social networking in the workspace. journal of the association for information science and technology, 65(10), 2035-2047. australasian journal of information systems duane & corcoran 2018, vol 22, research on recent advances in social media social media & knowledge sharing in heis 21 de long, d., w, & fahey, l. (2000). diagnosing cultural barriers to knowledge management. the academy of management executive, 14(4), 113-127. denison, d. r., & mishra, a. k. (1995). toward a theory of organizational culture and effectiveness. organization science, 6(2), 204-223. dimicco, j., millen, d. r., geyer, w., dugan, c., brownholtz, b., & muller, m. (2008). motivations for social networking at work. paper presented at the acm conference on computer supported cooperative work, san diego, ca, usa. dobson, i. r. (2011). how the other half lives: university general staff today and tomorrow. campus review, 2012(3), 1-7. edvinsson, l. (2002). corporate longitude: what you need to know to navigate the knowledge economy. london: pearson education. eysenbach, g. (2011). can tweets predict citations? metrics of social impact based on twitter and correlation with traditional metrics of scientific impact. journal of medical internet research, 13(4), e123. faraj, s., jarvenpaa, s. l., & majchrzak, a. (2011). knowledge collaboration in online communities. organization science, 22(5), 1224-1239. fidelman, m. (2012). socialized!: how the most successful businesses harness the power of social. brookline, ma, usa: bibliomotion, inc. florenthal, b., & tolstikov-mast, y. (2012). organizational culture: comparing faculty and staff perspectives. journal of higher education theory and practice, 12(6), 81. friedman, b. d., burns, m. j., & cao, j. (2014). enterprise social networking data analytics within alcatel-lucent. bell labs technical journal, 18(4), 89-109. fulk, j., & yuan, y. c. (2013). location, motivation, and social capitalization via enterprise social networking. journal of computer‐mediated communication, 19(1), 20-37. fullwood, r., rowley, j., & delbridge, r. (2013). knowledge sharing amongst academics in uk universities. journal of knowledge management, 17(1), 123-136. gammelgaard, j. (2010). knowledge retrieval through virtual communities of practice. behaviour & information technology, 29(4), 349-362. gibbs, g. (1988). learning by doing: a guide to teaching and learning methods. london: further education unit. goffee, r., & jones, g. (2015). why should anyone work here? – what it takes to create an authentic organisation. boston, ma, usa: harvard business press. gunawardena, c. n., hermans, m. b., sanchez, d., richmond, c., bohley, m., & tuttle, r. (2009). a theoretical framework for building online communities of practice with social networking tools. educational media international, 46(1), 3-16. haefliger, s., monteiro, e., foray, d., & von krogh, g. (2011). social software and strategy. long range planning, 44(5), 297-316. hart, j. (2015). modern workplace learning: a resource book for l&d. self-published: centre for learning & performance technologies. australasian journal of information systems duane & corcoran 2018, vol 22, research on recent advances in social media social media & knowledge sharing in heis 22 hislop, d. (2013). knowledge management in organizations: a critical introduction. oxford: oxford university press. hoffman, e. (2009). evaluating social networking tools for distance learning. paper presented at the tcc worldwide online conference, hawaii, usa. holtzblatt, l., drury, j. l., weiss, d., damianos, l. e., & cuomo, d. (2013). evaluating the uses and benefits of an enterprise social media platform. journal of social media for organizations, 1(1), 1. huy, q. n. (2002). emotional balancing of organizational continuity and radical change: the contribution of middle managers. administrative science quarterly, 47(1), 31-69. jasper, m. a. (2005). using reflective writing within research. journal of research in nursing, 10(3), 247-260. kaplan, a. m., & haenlein, m. (2010). users of the world, unite! the challenges and opportunities of social media. business horizons, 53(1), 59-68. kezar, a., & eckel, p. d. (2002). the effect of institutional culture on change strategies in higher education: universal principles or culturally responsive concepts? the journal of higher education, 73(4), 435-460. kidwell, j. j., vander linde, k. m., & johnson, s. l. (2000). knowledge management practices applying corporate in higher education. educause quarterly(4), 28-33. kimble, c., hildreth, p. m., & bourdon, i. (2008). communities of practice: creating learning environments for educators (vol. 1). charlotte, nc, usa: information age publishing. kok, a. (2007). intellectual capital management as part of knowledge management initiatives at institutions of higher learning. the electronic journal of knowledge management, 5(2), 181-192. lam, w. (2005). successful knowledge management requires a knowledge culture: a case study. knowledge management research & practice, 3(4), 206-217. lave, j. & wenger, e. (1991). situated learning: legitimate peripheral participation, london: cambridge university press. lee, j. j. (2007). the shaping of the departmental culture: measuring the relative influences of the institution and discipline. journal of higher education policy and management, 29(1), 41-55. lee, l., reinicke, b., sarkar, r., & anderson, r. (2015). learning through interactions: improving project management through communities of practice. project management journal, 46(1), 40-52. leftheriotis, i., & giannakos, m. n. (2014). using social media for work: losing your time or improving your work? computers in human behavior, 31(1), 134-142. leidner, d., & jarvenpaa, s. l. (1995). the use of information technology to enhance management school education: a theoretical view. mis quarterly, 19(3), 265-291. leidner, d., koch, h., & gonzalez, e. (2010). assimilating generation y it new hires into usaa's workforce: the role of an enterprise 2.0 system. mis quarterly executive, 9(4). australasian journal of information systems duane & corcoran 2018, vol 22, research on recent advances in social media social media & knowledge sharing in heis 23 leonardi, p. m., huysman, m., & steinfield, c. (2013). enterprise social media: definition, history, and prospects for the study of social technologies in organizations. journal of computer‐mediated communication, 19(1), 1-19. leonardi, p. m., & meyer, s. r. (2015). social media as social lubricant: how ambient awareness eases knowledge transfer. american behavioral scientist, 59(1), 10-34. lesser, e. l., & storck, j. (2001). communities of practice and organizational performance. ibm systems journal, 40(4), 831-841. levy, m. (2009). web 2.0 implications on knowledge management. journal of knowledge management, 13(1), 120-134. lewis, b., & rush, d. (2013). experience of developing twitter-based communities of practice in higher education. research in learning technology, 21(1), 1-13. liebowitz, j. (2001). knowledge management and its link to artificial intelligence. expert systems with applications, 20(1), 1-6. lin, t. (2012, 16 january). cracking open the scientific process. new york times, pp. 1-5. lok, p., & crawford, j. (2004). the effect of organisational culture and leadership style on job satisfaction and organisational commitment: a cross-national comparison. journal of management development, 23(4), 321-338. lópez-nicolás, c., & meroño-cerdán, á. l. (2011). strategic knowledge management, innovation and performance. international journal of information management, 31(6), 502509. mäntymäki, m., & riemer, k. (2016). enterprise social networking: a knowledge management perspective. international journal of information management, 36(6), 1042-1052. marshall, m. n. (1996). sampling for qualitative research. family practice, 13(6), 522-526. mccarthy, a. (2014). leading during uncertainty and economic turbulence: an investigation of leadership strengths and development needs in the senior irish public sector. advances in developing human resources, 16(1), 54-73. mclure, m., & faraj, s. (2000). “it is what one does”: why people participate and help others in electronic communities of practice. the journal of strategic information systems, 9(2), 155-173. mclure wasko, m., & faraj, s. (2005). why should i share? examining social capital and knowledge contribution in electronic networks of practice. mis quarterly, 35-57. ng, l. l., & pemberton, j. (2013). research-based communities of practice in uk higher education. studies in higher education, 38(10), 1522-1539. ogbonna, e., & harris, l. c. (1998). managing organizational culture: compliance or genuine change? british journal of management, 9(4), 273-288. ogbonna, e., & wilkinson, b. (2003). the false promise of organizational culture change: a case study of middle managers in grocery retailing. journal of management studies, 40(5), 1151-1178. ortbach, k., & recker, j. c. (2014). do good things and talk about them: a theory of academics usage of enterprise social networks for impression management tactics. paper presented at the australasian journal of information systems duane & corcoran 2018, vol 22, research on recent advances in social media social media & knowledge sharing in heis 24 proceedings of the 35th international conference on information systems, auckland, new zealand. pascale, r. t. (1985). the paradox of" corporate culture": reconciling ourselves to socialization. california management review, 27(2), 26-41. pascale, r. t., & sternin, j. (2005). your company’s secret change agents. harvard business review, 83(5), 72-81. pinto, m. (2012). a framework for knowledge managemt systems implementation in higher education. paper presented at the proceedings in arsa-advanced research in scientific areas, virtual. rai, r. k. (2011). knowledge management and organizational culture: a theoretical integrative framework. journal of knowledge management, 15(5), 779-801. ramachandran, s. d., chong, s. c., & ismail, h. (2009). the practice of knowledge management processes: a comparative study of public and private higher education institutions in malaysia. the journal of information and knowledge management systems, 39(3), 203-222. ramakrishnan, k., & yasin, n. m. (2012). knowledge management system and higher education institutions. international proceedings of computer science and information technology, 37(1), 67-71. rao, m. (2010). km and pm: case studies and learnings from the infotech sector. in t. kanti, m. e. srikantaiah, d. koenig, & s. hawamdeh (eds.), convergence of project management and knowledge management (pp. 308-317). lanham, md, usa: scarecrow press, inc. razmerita, l., kirchner, k., & nabeth, t. (2014). social media in organizations: leveraging personal and collective knowledge processes. journal of organizational computing and electronic commerce, 24(1), 74-93. richter, a., & riemer, k. (2013). the contextual nature of enterprise social networking: a multi case study comparison. paper presented at the ecis completed research, utrecht, netherlands. riemer, k., finke, j., & hovorka, d. (2015). bridging or bonding: do individuals gain social capital from participation in enterprise social networks? paper presented at the thirty sixth international conference on information systems, fort worth, texas, usa. rogers, e. m. (1995). diffusion of innovations (fourth ed.). new york: the free press. rowley, j. (2000). is higher education ready for knowledge management? international journal of educational management, 14(7), 325-333. sathe, v. (1983). implications of corporate culture: a manager's guide to action. organizational dynamics, 12(2), 5-23. scanlan, j. m., care, w. d., & udod, s. (2002). unravelling the unknowns of reflection in classroom teaching. journal of advanced nursing, 38(2), 136-143. schein, e. h. (2010). organizational culture and leadership (vol. 2). san francisco: jossey-bass. schneckenberg, d. (2009). web 2.0 and the empowerment of the knowledge worker. journal of knowledge management, 13(6), 509-520. doi:doi:10.1108/13673270910997150 australasian journal of information systems duane & corcoran 2018, vol 22, research on recent advances in social media social media & knowledge sharing in heis 25 spendlove, m. (2007). competencies for effective leadership in higher education. international journal of educational management, 21(5), 407-417. stewart, t. a. (1997). intellectual capital: the new wealth of organizations. new york: doubleday. suppiah, v., & singh sandhu, m. (2011). organisational culture's influence on tacit knowledgesharing behaviour. journal of knowledge management, 15(3), 462-477. susman, g. i., & evered, r. d. (1978). an assessment of the scientific merits of action research. administrative science quarterly, 23(4), 582-603. swart, j., & kinnie, n. (2003). sharing knowledge in knowledge-intensive firms. human resource management journal, 13(2), 60-75. szekeres, j. (2004). the invisible workers. journal of higher education policy and management, 26(1), 7-22. taylor, j. (2006). “big is beautiful.” organisational change in universities in the united kingdom: new models of institutional management and the changing role of academic staff. higher education in europe, 31(3), 251-273. tierney, w. g. (1988). organizational culture in higher education: defining the essentials. the journal of higher education, 59(1), 2-21. tippins, m. j. (2003). implementing knowledge management in academia: teaching the teachers. international journal of educational management, 17(7), 339-345. trice, h. m., & beyer, j. m. (1993). the cultures of work organizations. englewood cliffs, nj, usa: prentice-hall, inc. von krogh, g. (2012). how does social software change knowledge management? toward a strategic research agenda. the journal of strategic information systems, 21(2), 154-164. wang, s., & noe, r. a. (2010). knowledge sharing: a review and directions for future research. human resource management review, 20(2), 115-131. wang, z., & wang, n. (2012). knowledge sharing, innovation and firm performance. expert systems with applications, 39(10), 8899-8908. wenger, e. (1999). communities of practice: learning, meaning, and identity. cambridge: cambridge university press. wenger, e., & snyder, w. m. (2000). communities of practice: the organizational frontier. harvard business review, 78(1), 139-146. wi, c. (2015). the engaged leader: a strategy for your digital transformation. pa, usa: wharton digital press. wiig, k. m. (1999). what future knowledge management users may expect. journal of knowledge management, 3(2), 155-166. willmott, h. (1993). strength is ignorance; slavery is freedom: managing culture in modern organizations. journal of management studies, 30(4), 515-552. australasian journal of information systems duane & corcoran 2018, vol 22, research on recent advances in social media social media & knowledge sharing in heis 26 wohlmuther, s. (2008). ‘sleeping with the enemy’: how far are you prepared to go to make a difference? a look at the divide between academic and allied staff. journal of higher education policy and management, 30(4), 325-337. yates, d., & paquette, s. (2011). emergency knowledge management and social media technologies: a case study of the 2010 haitian earthquake. international journal of information management, 31(1), 6-13. zboralski, k. (2009). antecedents of knowledge sharing in communities of practice. journal of knowledge management, 13(3), 90-101. zhao, f., & kemp, l. (2013). exploring individual, social and organisational effects on web 2.0-based workplace learning: a research agenda for a systematic approach. research in learning technology, 21(1), 1-15. copyright: © 2018 duane & corcoran. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. microsoft word ajis_18(3)_v10.docx australasian journal of information systems volume 18 number 3 2014 227 promises and pitfalls of using social media in public e-procurement: an appraisal sharif as-saber school of management rmit university, melbourne sharif.as-saber@rmit.edu.au shams rahman school of business it and logistics rmit university, melbourne shams.rahman@rmit.edu.au md mahbubur rahim faculty of information technology monash university, melbourne md.mahbubur.rahim@monash.edu george cairns school of management rmit university, melbourne george.cairns@rmit.edu.au charmine härtel uq business school the university of queensland, queensland c.hartel@business.uq.edu.au wahed waheduzzaman faculty of business and enterprise swinburne university of technology, melbourne wwaheduzzaman@swin.edu.au abstract social media (sm), in recent years, is emerging as a common platform for low cost information exchange, and has attracted a critical mass of users both at corporate and retail levels. theoretically, sm can thus be used as a tool to strengthen e-procurement in the public sector. towards this end, we have prepared a conceptual model drawing on literature reviews and some examples while identifying a set of expected benefits and challenges within four stages of e-procurement. using the framework, a case study has been conducted involving australian public procurement initiatives and 15 federal government senior officials engaged in e-procurement. they have been interviewed to shed light on the possibilities and challenges of using sm in the public e-procurement context. the findings of the study suggest a limited scope for sm usage in the australian public sector e-procurement process. the implications of the findings are discussed and some recommendations offered. keywords: e-procurement, social media, public sector, potentials, challenges australasian journal of information systems volume 18 number 3 2014 228 introduction procurement of goods/services by an organisation via the internet (commonly known as eprocurement) has received considerable popularity worldwide since the 1990s (gunasekaran, mcgaughey, ngai, & rai, 2009). despite many benefits that organisations may experience from the implementation of e-procurement initiatives (e.g. reduced administration costs, shortened procurement cycle, greater data accuracy, enhanced buyer image), they are not without criticisms. in many cases, eprocurement is costly (raymond, 2008), inflexible (devaraj, vaidyanathan, & mishra, 2012), not fully transparent (puschmann and alt, 2005), and even inefficient (australian national audit office, 2010). in recent years, social media (sm) technologies such as facebook, twitter, youtube, linkedin are increasingly being used for low cost information transfer and wide circulation of information promptly (bertot, jaeger, & grimes, 2012; neiger et al., 2012; zavattaro, 2013). it could, therefore, be argued that sm can be considered as an emerging technological innovation which, if carefully manipulated, can help strengthen e-procurement initiatives. sm could help organisations to broadcast about tenders more efficiently and quickly reaching a wider audience, facilitate stakeholder feedback on procurement related issues, and, at the same time, contribute to establishing transparency in the process. this in turn could improve efficiency and effectiveness of the e-procurement process. however, little attempts in this regard have so far been reported in the scholarly literature which thus necessitates further research to investigate the potential usage of sm within the public sector e-procurement context and to identify the challenges that could hinder the integration of sm in e-procurement. arguably, public procurement officials may benefit from a better understanding on how sm could be effectively used in public eprocurement initiatives which may prompt them to integrate sm into the existing e-procurement setting. based on the extant literature and experiences, we have initially developed an initial conceptual model that identifies a set of challenges within the four stages of public sector e-procurement process and link them to the key procurement issues. using a case study approach and the conceptual model, 15 senior executives from a selection of major australian federal government agencies involved in public procurement function were interviewed in order to identify the potential use and challenges of introducing sm to complement e-procurement initiatives. the findings helped refine the model, present several possibilities, and identify a number of challenges for the public sector organisations to adopt sm within their e-procurement initiatives. the paper provides a brief synthesis of the relevant literature on e-procurement and sm which is followed by an introduction of an initial literature driven model. then, the research approach is presented and the empirical findings are described and discussed in light of the model. finally, some concluding remarks are made highlighting the contributions of the paper and identifying future directions of sm-driven e-procurement research. public e-procurement and the use of social media e-procurement defining e-procurement is a challenging task as scholars have defined this concept using different perspectives. one group of authors describes e-procurement as an internet-based technological solution for corporate purchases (e.g. alaniz and roberts, 1999). in contrast, mccue & roman, (2012) adopt a process oriented view and define e-procurement to be an automated purchasing process employing webenabled information technologies. in agreement with the process oriented view, wu, zsidisin, & ross, (2007) conceptualise e-procurement as managing the information flow between an organisation and its suppliers while obtaining materials. tatsis et al. (2006, p.63) further extend the process oriented notion of e-procurement and describe it as, ‘‘the integration, management, automation, optimization and enablement of an organisation’s procurement process, using electronic tools and technologies, and web-based applications’’. these varied views about e-procurement indicate that it is not a one-off event australasian journal of information systems volume 18 number 3 2014 229 of electronically purchasing goods/services and that it includes an idea of exchanging product/service related information among various stakeholders. this view further involves execution of a series of activities which can be oragnised into several stages from an intention of obtaining goods/services to the actual obtaining of a product or service. regardless of how e-procurement is defined, there exists empirical evidence which suggest that many organisations across the world have moved away from the traditional paper-based procurement processes and embraced new technologies supporting e-procurement process (bertot et al., 2012). adopters of e-procurement have reported gaining significant benefits (croom & brandon-jones, 2007). according to caniato et al. (2011), these benefits can be grouped into two types: i) process benefits, such as reduced cost involved for processing orders due to error reduction (ageshin, 2001, nergiz and rahim, 2011), shortened sourcing cycle time (presutti, 2003), greater simplicity due to process streamlining, maverick buying reduction (subramaniam & shaw, 2004), and ii) organisational benefits, such as reduction in inventory costs (ageshin, 2001), reduction in contract compliance (subramaniam & shaw, 2004), enhanced organisational image perceived by suppliers (nergiz and rahim, 2011; rahim and bintawal, 2012), greater decision support, and increased transparency (rahim and as-saber, 2011). despite many benefits associated with e-procurement, progress in adoption and subsequent diffusion of these initiatives is still less than satisfactory across many countries including in australia. total internet commerce for business-to-business [including e-procurement], and business-to-consumers still only represents 2.2 per cent of sales turnover in australia, 1.3 per cent in canada and ranges from 0.01 to 17 per cent for european countries (walker & harland, 2008, p. 835). several challenges prevail around existing e-procurement initiatives which inhibit organisations to implement these initiatives in fast pace (rahim and as-saber, 2011; bertot et al, 2012). commonly reported challenges include: high cost of implementation (angeles and nath, 2007), increased cost in electronic interface for communication (subremenium and shaw, 2004), high training cost for learning e-procurement software (ageshin, 2001), inability to integrate back-end systems to the e-procurement process (hawking et al 2004), high cost of restructuring organisation (ageshin, 2001) or organsational resistance (mccue and roman, 2012), unsuitability of software platforms, lack of strategic systems integration and failure to involve public procurement professionals in the design of e-procurement systems (mccue and roman, 2012), and lack of standard interchange formats for e-procurement (angeles and nath, 2007). in addition, buying organisations tend to become more dependent on a small number of suppliers due to limited registration of suppliers in any e-procurement initiative. this in turn leads buyers to dodge accountability and transparency (ageshin, 2001). furthermore, many of the current generation eprocurement initiatives are less responsive to medium and small suppliers, as not all small suppliers can provide sufficient catalogue information to the buyers (vaidya, sajeev and callender 2006). therefore, there is a need to consider bringing an improvement in e-procurement process to help mitigate some of these challenges. in this regard, we believe that sm technologies may bear considerable potentials to address some of the existing challenges associated with public e-procurement process. social media social media is designed for many-to-many interaction (porter, 2008). it is often used as a proxy for face-to-face interactions, and has a strong social connotation within a virtual platform. sm represents a collection of web-based tools (e.g. blogs, wikis, information sharing services and social networking services) that enable individuals and businesses to share content, communicate and interact in a social environment (bartot, 2012). the sm enabled networking primarily consists of user generated contents shared either openly or across specific groups of individuals (neiger et al, 2012). its popularity together with the convenience and user-friendliness have already been capitalised by businesses around the world to promote their products and to interact with various stakeholders including customers and australasian journal of information systems volume 18 number 3 2014 230 suppliers (bertot et al, 2011, robert, 2009, zavattaro, 2013). research indicates that 80% of physicians, who are involved in online consultation with patients, now use sm to share medical contents (neiger et al 2012). sm further facilitates collaboration and deliberation among stakeholders which in turn helps make public functions more transparent and accountable (bryer and zavattaro, 2011, zavattaro, 2013). the use of social media in the public e-procurement the public sector has begun extracting value from sm in optimising the use of scarce public resources (zavattaro, 2013). furthermore, owing to its many sharing features, sms have seen applications in many e-government efforts (bartot, 2012, zavattaro, 2013). from an e-procurement perspective, the integration of sm appears to be fairly new (bartot, 2012). despite this newness, public officials strongly believe that business values could be added to the existing e-procurement process by creating a link between sm and e-procurement process (robert, 2009). in keeping with this belief, we too expect that the potential use of sm in e-procurement process can offer some benefits but realisation of these benefits is not without challenges. however, no work has yet been reported in the scholarly literature that investigates the role of sm within public e-procurement context. a conceptual model drawing on the relevant background literature (discussed in the previous section), we acknowledge that a range of activities are involved in the public procurement processes which are supported by eprocurement solutions. this view is in agreement with that of morris et al. (2000), where they observe that a number of activities are involved to complete a purchase event. there is however no agreement in the literature on the number of specific activities involved in e-procurement process. furthermore, the number of activities may not be same for all types of businesses (presutti, 2003). we have accepted major activities involved within e-procurement process suggested by subramaniam and shaw (2004) and grouped them in 4 stages: searching, order processing, monitoring and control and coordination. based on our reading of the e-procurement literature, we have further identified several key activities involved in each of these stages. this is clearly indicated in our proposed cyclic model (figure 1). in the outer ring of the cyclic model, we identify possible impact (benefits) that could be brought in by the use of sm for that stage. the inner most ring highlights the 4 stages of e-procurement process and the intermediate ring shows the key activities within each of these stage. we now briefly describe each of these stages and explain how sm can influence the benefits for the activities representing those stages. australasian journal of information systems volume 18 number 3 2014 231 stag e 1 : searc hing stag e 4 : coo rdi na tio n stage 2: order processing stage 3: monitoring & control n ee d as se ss m en t dev elo p de sig n fe ed ba ck & re po rti ng e-cataloguing expression of interests call for tenders tender r eceipt & evaluation tender c onsideration & negotiation circulation of tender outcomes receive & verify order arrange payment rev iew of ten de r com mon pl atf orm fo r in fo. ex ch an ge rea dil y a va ila ble in for mati on lo w co st for in fo. co lle ctio n continuous monitoring transparent reporting reliable feedback quick circulation trust in suppliers prompt response ver ify w ith up da ted pr od uc ts/ se rvi ce s le ss tim e f or pa ym en t p roc es sin g social media in e-procurement figure 1: a conceptual model describing use of social media in e-procurement process stage 1 (search) during the early need assessment stage of the e-procurement process, online networking (via sm) with citizens, community groups and prospective suppliers might provide feedback and ideas with respect to the actual need for the product or service including any requirements of special features. the size of the procurement and the urgency of procurement could also be determined based on such feedback. once the need is assessed, social networking could even provide with out-of-box ideas with respect to the design or concept of the product or services. using these inputs, management and procurement officials within the buying agencies might come up with an appropriate set of tender documents and ecatalogues. e-catalogues and interactive forms could be made available through sm enabled networking outlets in order to allow stakeholders to better understand the procurement process and to provide comments on these documents. stage 2 (order processing) the call for expressions of interests could then be circulated via sm networks together with usual outlets (e.g. newspapers). the formal call for tenders could follow as the next step. use of sm would help agencies to draw attention of wider group of suppliers across the world businesses. once formal tenders are received, updates on the numbers of total bids and the outcome of the initial screening without referring to any name of suppliers may be disseminated via the social network. however, the actual tender consideration process would ideally remain confidential until the final decision is made. australasian journal of information systems volume 18 number 3 2014 232 stage 3 (monitoring & control) once the successful bidder is selected by the buying agency, documents and the reasons for such decision could be made public through sm. in addition to reporting to the regulatory agencies and sending individual letters to all participating bidders, this information could be posted to the sm network as well. feedback and comments received through the network could be useful for the organisation to further improve the process. stage 4 (coordination) sm would be able to provide advantages of coordination of information between buyers and suppliers. because of the availability of soft information, buyers could easily be able to review the whole process and provide feedback, which may be used for future tendering processes. we argue that all stages of e-procurement process would be benefited from the possible use of sm in e-procurement. for example, a rich body of information can be searched within short time for developing product design activity. tender order can be circulated within wide range of suppliers. these possible benefits are all outlined in the outer ring of the model. research approach selection of research method this research has selected australian government e-procurement system as a case study. in order to address the research concerns, an exploratory qualitative approach is undertaken. as the case study is suitable for exploratory kind of research (neuman, 2003), this research adopted interview techniques to gather in-depth opinions about the motives behind considering (or not considering) sm as an eprocurement tool from key informants. the ‘key informant’ approach is well suited to gathering qualitative and descriptive data that are difficult or time intensive to collect through structured quantitative data-collection techniques (tremblay, 1957). a semi-structured interview protocol was used to guide the interview process addressing the key questions. organisations selected for interviews focus group and face-to-face interviews were organised for officials / key informants from four federal government institutions who are directly involved in procurements in australia. altogether seven interviews (in the form of focus group and face-to-face discussion) were conducted during septembernovember 2012, as mentioned in table 1. australasian journal of information systems volume 18 number 3 2014 233 brief description of selected public sector entity (pse)5 position of interviewees interviewee number data collection method pse-1 is the central body for publication of australian government business opportunities, annual procurement plans and contracts awarded. it is also known as the central procurement authority for public purchases in australia. suppliers register with this pse to receive free information if any tender notice is circulated for procurement. incumbent public organisations those want to buy something over au$10,000, posts a request to the pse-1. registered suppliers can download the request and submit bids electronically. there are around 86,000 registered suppliers at present. it evaluates the bids, negotiates online, and places the order with the lowest responsible bidder. director/assistant secretary level executives 3 focus group pse-2 is responsible for publishing data related to government procurement and to help australian business community in matters relating to government procurement activities. it is also involved in the development of procurement policy, managing etendering system, designing and implementing the tender processes, and establishing the contracts, for relevant government offices. assistant secretary 1 face to face assistant director 1 face to face financial officer and officials responsible for procurement coordination 3 focus group director level officials, 1 face to face pse-3 has a program delivery role in supporting markets and business, and providing commonwealth payments to the state and territory governments. regarding procurement, it publishes information about significant procurements and its future trends in an annual procurement plan. executives at the level of financial officer/director 2 focus group pse-4 is primarily responsible for auditing the activities of most commonwealth public sector entities. director level executives 4 focus group table 1: organisations and respondents collecting and organising data a semi-structured questionnaire was used for interviews. the recorded interviews were transcribed into text manually. the interview texts were then organized under different broad descriptive codes. texts 5 names of actual pses are not listed because of privacy and ethical considerations. australasian journal of information systems volume 18 number 3 2014 234 referring to the same argument, particularly for potentials or challenges, were then taken out from each group of themes for further analysis. the developed themes were then compared and contrasted for thematic analysis (welsh, 2002). data from interviews, documents, and theories were finally triangulated to avoid any bias in analysis. findings drawing on the thematic analysis of the interview data, we have identified three possible usage potentials of sm within public e-procurement context. we have further identified nine challenges that may jeopardise sm usage. these are briefly discussed below: potential 1 (limited usage potential of sm) during literature review, based on the theories, this research assumed that the sms have potentiality to be used in different stages of e-procurements, which consequently would make the system more efficient. however, the data analysis shows that most of the interviewees believe in the use of sm in eprocurement processes differently. some of the interviewees claimed that they are already using a kind of sm, and that is the end of using sm in procurement. the social media is not possible to use across all the stages of e-procurement. one manager from pse-1 mentioned: “i think e-procurement and social media are two different things...still don’t see where social media fits into any of that....i think there is a distinct difference between social media and e-procurement..., we have yet to understand, we have yet to think that there is a role for social media in our procurements base. that’s not to say there never will be but in this current environment... we’re trying to understand what we could possibly achieve via social media that we don’t already...”. though sm can be used in e-procurement, it cannot be possible in all stages; it could be useful for only limited stages. one of the directors working in pse-1 suggested: “i guess the more that i’ve thought about it i think that focusing on the expression stage and the reporting stage is certainly doable. for mine, what you’d need to be able to address are the probity control requirements that are inherent in the other phases. potential 2 (quick updating of suppliers) some interviewees support the use of sm. they acknowledge the potential use of sm like twitter. sm can be used in big and lengthy procurement (takes many years or decades sometime) in particular where tenders are required to be revised; it is necessary to use such tools to quickly and cost-effectively appraise the suppliers of the changes made in the tender. this sentiment is reflected when a senior manager remarked: “and often in my memory they would go out with a tender, a request for tender, and then the tender itself would change constantly as they discovered more as they found out what the potential solutions were and they would then have to send out the revised tender to all prospective candidates who might bid for the work. and it struck me, just on my quick walk down here, that that’s where something like twitter would be useful, saying to everyone’s who’s registered for a bidding process, the tender has now changed or these requests for clarification have now been published, or whatever. so a general updating of the process over a long and complex tender i think would be certainly useful.” potential 3 (informal collection of supplier feedback on complex tenders) during focus group interview most of the interviewees acknowledge that sm can be used either in small or large scale. use of sm is suitable for doing research to prepare document at the stage of ‘calling for tender’. a perfect communication is mandatory to make e-procurement successful, and sm can help in australasian journal of information systems volume 18 number 3 2014 235 this regard. one respondent from pse-3 mentioned during interview that sm might help to streamline the process and reduce the complexities in tender documents. “i’ve discovered this, you can do as much work on a tender, yeah, you can spend weeks on it, get it together, yeah statement of requirements, you think it’s perfect, you get it out there and the saddest thing is when no-one responds to it.... because they can’t understand what you’re asking for or you’ve made it too complex or too… the amount of… i had a tender that fell over ‘cause they couldn’t actually understand what we wanted... you spent thousands of man hours on something and you’re going to get no response, what a waste of time.... social media would help, just might have made that more efficient... i see that’s where it’s in, especially when you’re building very complex type procurement arrangements or community…if you’re advanced of the tender and you’ve got a draft of a tender out, you’re trying to attract as many people as you can to look at it so they can, i guess, they can share their views on the tender...” challenge 1 (sm adds additional costs) though sm is free to use and therefore perceived to be cost effective, the interview data contradicts this view. interviewees pointed that the extraction of appropriate information from sm for procurement will be costly. during interview, one manager posited that the present e-procurement system is very cost effective, and use of sm will make e-procurement enabled procurement process more costly because it will require hiring of new employees to extract raw data. more information while is expected in procurement process but it may increase cost to analyse and find out the desired information. according to a respondent from pse: “so there’s, in terms of social media, there’s probably, there’d probably be, i mean yes, the technology’s free, but there is a cost for managing a twitter account, for example, that doesn’t exist at the moment because we don’t have one. but in terms of further automating our processes, yes there probably is scope for us to do that, but that’s quite different than necessarily e-procurement and necessarily using social media....for example just theoretically, we start having different ways of approaching the market and generating responses to requirements, all of a sudden you need a lot more people on-board to manage that. where does the commonwealth get its money from? taxes; but all of a sudden if we start putting more opportunities out there or advertising those opportunities in a different way, it’s costing time for people who respond to those, so they build that into their process to cover those costs, the cost of doing business and the cost of what we’re buying goes up, the cost of doing business goes up and it’s… are we creating something we don’t want to create there?” challenge 2 (difficulty to locate information relevant to procurement) the present structure of sm while has seen suitable for sharing huge information, it may also bring trouble to find the correct information. one manager from pse during interview finds existing sm to be unstructured in nature and suggests that sm needs to be more structured to be cost effective: “but the problem again is structure. if you’re going to do something like that, particularly if it’s a ultimately going to end in procurement, right, you want to put it out in a structured way. so people are notified about it in accordance with their business profile so they response formally, rather than ad hoc commentary. you know it’s one of the great challenges i think for social media is getting some structure into it so that we’re not just rounding in, as yvette said, in 95% of the information we get, we’re not interested in...and again, there’s a cost both ways to do that type of business. so the reality is it takes your time, it costs money regardless of whose side of the fence you’re sitting on, so the commonwealth government, well people mightn’t see it, but we try and run as lean as we can, just like the private sector, just that we have a few more hoops that we need to jump through with regards to accountability and transparency and due process. so certain things that we have to do, so if we were to start to introduce additional requirements, the cost of doing business just for all, just increases dramatically australasian journal of information systems volume 18 number 3 2014 236 and that has to be funded from somewhere...it’s unstructured, so how do you find what you’re looking for. and i mean we’ve all trawled through blogs and so on, i mean there’s so much, you know information that’s really not very useful. how do you ensure you don’t miss something important if you’re not in a structured database environment?” challenge 3 (compromising trust between suppliers and procuring agencies) moreover, trust in each other is very important to develop a responsive relation (croom & brandonjones, 2007). sm is very open and wide, which is not suitable to develop a trust based relation. this would be an issue to create a network with stakeholders. suppliers may not respond in sm when they find a lacking of trust in their private information that they need to provide in procurement systems. one high official from pse mention that: “the requests for tender, all of that information from the government’s perspective is in the public domain. what we have is a great deal of trust from our suppliers that the information they give us is secure and isn’t going to be in the public domain in any way, shape or form.” challenge 4 (lack of legal binding of suppliers to respond) a legal binding is required in order to build a responsive relation between all stakeholders in an eprocurement system. sm is far away in this regard. these are no guarantee that suppliers will participate or respond even though they were invited to participate through sm. sm is not a legally binding tool that could ensure response of targeted suppliers. one interviewee expressed concern about the consequences of irresponsiveness to the buyers’ requirement in sm because no one is legally bound to respond in sm. according to a senior executive from pse-3: “what if no-one ever answered? ...i just don’t, i… the challenge i think around this there are a couple of issues.” challenge 5 (transparency in e-procurement enabled process) transparency, which is openness of tendering process to all stakeholders, is not possible to maintain in all stages of e-procurement. analysis of interview data shows that sm would be helpful in terms of making procurement more transparent because more people would be able to follow the process, but in many stages sm is unsuitable to use because it may uncover confidential information. thus sm cannot be used in e-procurement for the sake of maintaining transparency. one manager from pse-3 in the interview mentioned that sm could be used to make the procurement process transparent to a greater number of people. “transparency does not mean that procurement process should be visible to everyone. it is about maintaining integrity and fairness in the process. moreover the process can be examined or reviewed. the definition of transparency needs to be made public for appropriate and uniform understanding.” challenge 6 (data security) security of data is the main concern in using sm because sm is commonly open to all. maintaining confidentially of data interchanged between buyers and suppliers are very important function in eprocurement process. one respondent from pse maintained that security would be a major concern in using sm: “... it’s a major requirement from us is data security. so we’ve got requirements that were placed upon us by government that we have certain security arrangements in place for data; and i don’t know that these other platforms would be able to meet the current defensive or strategy requirements. [033]...if the department of defence puts something out there we’ve got to… we want to procure something here, and it might be non-military, but it’s department of defence… how inviting that is to hackers to get on australasian journal of information systems volume 18 number 3 2014 237 there and manipulate that data.]...commonwealth procurement, there’s a range of elements there; there is confidentiality requirements. so if we’re issuing tenders for the asio building that’s just been built over here for example, we don’t want hackers or foreign nationalities to be able to access that to look at it. we don’t want people to get in there and have a little bit of fun and manipulate so all of a sudden instead of buying some toilet paper, the department of defence, all of a sudden they’re buying lollypops, i don’t know, instead of toilet brushes they’re buying chuppa-chups because someone jumped in and, for fun, manipulated a government website. ... why would you do that when you’ve got a system that handles that in a secure environment now?” challenge 7 (accountability) accountability to all suppliers needs to be ensured in the e-procurement process, which may not be possible if sm is placed in the procurement system. sm make all information open, which may not ensure accountability of buyer to a particular supplier and vice versa. a buyer may overlook an important question from a supplier and a supplier may overlook an important question from a buyer. thus, sm may undermine the norms of accountability. one director from pse-1, claimed for control over information which will be easily available through sm during e-procurement. “...you are in a procurement process the way you interact with your suppliers has to be very controlled. certainly in the lead up to that, we mail our potential suppliers for information and ideas in exactly the same way as there’s an interaction there. but the minute we actually go out to a process all contact has to be completely controlled; it has to be monitored, records of it have to be maintained and all that sort of thing. so, again, we use austender; if you’re in a procurement process, a question’s asked, all suppliers have to receive the question and the answer at the same time, so it’s done through the austender system and they’re notified that there is...” challenge 8 (not suitable for usage for all activities in e-procurement) not all sm is equally applicable in procurement process. they cautiously approved about twitter and blog, but have concerns about facebook. they want to say facebook could be useful for human procurement, such as, hiring, but not for material/service procurement. one manager from pse-3 outlined that: “...outside twitter and the blog, i haven’t seen like facebook used in government at all for that [eprocurement] purpose.” sm, as data analysis reveals, may not be effective as it was predicted initially. most of the participants were found in dilemma with the use of sm in a larger scale, rather they press that the present eprocurement system for public purchase is adequately effective. they also mentioned that they are already using sm like tweeter and blogs along with the existing e-procurement systems. they do not see any further benefit from using sm across all the stages of e-procurement. they emphasised to focus on better outcomes rather not only focusing on transparency and collaboration in terms of using sm in e-procurement process. challenge 9 (outcomes vs transparency – less priority given to transparency) the managers suggested that sm would help to make e-procurement more participatory and accountable, but they are concerned with over-transparent and less efficient. particularly they concern about handling of unstructured data through sm. no doubt the implementation and use of sm is free and thus huge information can be gathered within short time. more information while is expected in procurement process but it may increase cost to analyse and find out the desired information. unstructured information available in sm may not be suitable for both the buyers and sellers. however, for the initial stage of documentation and at the stage of reporting sm may bring some advantages. the overall potential and challenges, unravelled through interviewees, can be depicted in figure 2. australasian journal of information systems volume 18 number 3 2014 238 social media in e-procurement figure 2: potentials and challenges of social media in using for e-procurement based on interviewee responses, the second-last outer ring of the model lists issues, possible potentials, and challenges of using sm in public e-procurement process. discussion we now reflect on the nature of the challenges identified in the previous section. most of them are internal in nature, while a few involve external players. for example, except challenges 3 and 4, the remaining challenges are primarily concerned with intra-organisational issues. challenges 3 and 4 involve trust and legal issues with suppliers (external players). we further observe the dominance of the managerial challenges. only two challenges (e.g. challenges 2 and 6) involve technological concerns in terms of ensuring data security and difficulty of locating appropriate information. these challenges can be addressed using technological innovations. we thus suggest that successful implementation of sm within public e-procurement would require more managerial considerations. conclusion given the increasing use and popularity of social media as a means of communication, its integration to public e-procurement process may provide some advantages in making the process more dynamic, transparent and fast. however, little research has so far been done or reported linking social media to public e-procurement. to address this apparent gap in the literature, we have proposed a conceptual model that suggests several possibilities and challenges of using social media within the public eprocurement process. using this model, a case study involving several australian federal government australasian journal of information systems volume 18 number 3 2014 239 agencies has been conducted which, however, paints a different picture about the potential use of social media in its existing forms. according to government officials involved in australian public procurement, limited improvement in tender circulation and transparency is possible with the use of social media. in its current form, social media may not be considered suitable to use across all stages of e-procurement process. however, there appears to be limited scope of using sm in stage 1 (searching), stage 2 (order processing) and stage 3 (monitoring and control), especially to assess the need, e-cataloguing, submission of the expression of interests, call for tenders and disseminating the tender outcomes. the challenges constraining sm usage are primarily managerial and intraorganisational in nature. however, these challenges should not undermine sm’s needs and potential in enhancing efficiency in government procurements. therefore, our research contributes to knowledge by identifying a number of critical issues (e.g. challenges) which serve as a springboard to initiate further detailed investigations. the findings reported in this paper however cannot be generalised due to limited focus on the buyer side and because of its reliance on data from the public sector only. hence, future research is needed to solicit the viewpoints expressed by suppliers as well as the potential role and challenges associated with sm in e-procurement context. references adfd (australian department of finance and deregulation), 2012, strategic guide to e-procurement, http://www.finance.gov.au/publications/strategic-guide-to-e-procurement/executivesummary.html, retrieved on 9 february 2012. agmio (australian government information management office). 2005, review of the eprocurement demonstration projects, canberra: commonwealth of australia. agimo, 2013, (http://agimo.gov.au/about-agimo/#procurement_management_branch) ageshin, e. a. (2001). e-procurement at work: a case study. production and inventory management journal, (49:3), pp 48-53. anao, 2013, (http://www.anao.gov.au/procurement-and-tenders) anthony, s. and evans, s. 2010. reforming public procurement in australia via cost sharing incentives, background briefing 2010/01, canberra: macroeconomics. austender, 2013, (https://www.tenders.gov.au/?event=public.document.list) australian national audit office (2010). audit report no. 6: 2009–2010, canberra: anao. bertot, j.c., jaeger, p.t. and grimes, j.m. 2010. using icts to create a culture of transparency: egovernment and social media as openness and anti-corruption tools for societies, government information quarterly, (27:3), pp 264–271 bertot, j. c., jaeger, p. t., & grimes, j. m. (2012). promoting transparency and accountability through icts, social media, and collaborative e-government. transforming government: people, process and policy, (6:1), pp 78 91. brodsky, r. (2009, november 6). better buying through social media? national journal, 1. croom, s., & brandon-jones, a. (2007). impact of e-procurement: experiences from implementation in the uk public sector, journal of purchasing & supply management, (13:4), pp 294-303. davila, a., gupta, m., & palmer, r. (2003). moving procurement systems to the internet, the adoption and use of e-procurement technology models. european management journal, (21:1), pp 1123. devaraj, s., vaidyanathan, g., & mishra, a. n. (2012). effect of purchase volume flexibility and purchase mix flexibility on e-procurement performance: an analysis of two perspectives. journal of operations management, (30:7-8), pp 509-520. australasian journal of information systems volume 18 number 3 2014 240 gunasekaran, a., mcgaughey, r. e., ngai, e. w. t., & rai, b. k. (2009). e-procurement adoption in the southcoast smes. international journal of production economics, (122:1), pp161-175. hawking, p., stein, a., wyld, d.c. and foster, s. 2004. e-procurement: is the ugly duckling actually a swan down under?, asia pacific journal of marketing and logistics, (16:1), pp 3-26. lindlof, t. r., & taylor, b. c. (2002), qualitative communication research methods, 2nd edition. thousand oaks, ca: sage. lawlor, m, 2001. capitalizing on e-procurement saves dollars makes sense, signal online, http://www.afcea.org/signal/articles/templates/signal_article_template.asp?articleid=593& zoneid=118, retrieved on 9 february 2012. mccue, c., & roman, a. v. (2012). e-procurement: myth or reality? journal of public procurement, (12:2), pp 221-248. neiger, b. l., thackeray, r., wagenen, s. a. v., hanson, c. l., west, j. h., barnes, m. d., & fagen, m. c. (2012). use of social media in health promotion : purposes, key performance indicators, and evaluation metrics. health promotion practice, (13:2), pp 159-164. nergiz, i. and rahim, m.m. (2011) “predicting e-procurement systems implementation benefits: development of a model and implications for management”, 16th international business information management association (ibima ) conference, kuala lumpur, malaysia 29-30 june. neuman, w. l. 2003. social research methods: qualitative and quantitative approaches. boston: pearson education inc. noie (the national office for the information economy). 2003, transforming government: achievements in e-government, canberra: noie. presutti, w. d. (2003). supply management and e-procurement: creating value added in the supply chain. industrial marketing management (32:3), pp 219-226. puschmann, t., & alt, r. (2005). successful use of e-procurement in supply chains. supply chain management: an international journal, (10:2), pp 122 133. raymond, j. (2008). benchmarking in public procurement, benchmarking: an international journal, (15:6), pp 782-793. rahim, m.m. and as-saber, s.n. (2011). e-procurement adoption process: an arduous journey for an australian city council, international journal of electronic finance, (5:2), pp 150–168. rahim, m. m. and bantwal, a. p. (2012) “perceived benefits from a local government public procurement initiative: a diffusion of innovation perspective”, chapter 16, in interorganizational information systems and business management: theories for researchers, pp.288-308, (editor: vaida, k.), igi global publishers, usa. schepers, j. and wetzels, m., 2007. a meta-analysis of the technology acceptance model: investigating subjective norm and moderation effects, information and management, (44:1), pp 90–103. subramaniam, c., & shaw, m. j. (2004). the effects of process characteristics on the value of b2b e-procurement. information technology and management, (5:1-2), pp 161-180. treasury, 2013, (http://www.treasury.gov.au/about-treasury/ourdepartment). also find, http://www.treasury.gov.au/publicationsandmedia/publications/2011/treasury-annualreport-2010-11/report/part-3-management-and-accountability/purchasing tatsis, v., mena, c., van wassenhove, l., & whicker, l. (2006). e-procurement in the greek food and drink industry. journal of purchasing and supply management, (12:2), pp 63-74. australasian journal of information systems volume 18 number 3 2014 241 tindal, s. 2011. nsw agencies ignore e-procurement rules, zdnet.com.au, http://www.zdnet.com.au/nsw-agencies-ignore-e-procurement-rules-339310057.htm, retrieved on 9 february 2012. tremblay, m. a. 1957. the key informant technique: a non-ethnographic application. american anthropologist, (59:1), pp 688-701. walker, h., & harland, c. (2008). e-procurement in the united nations: influences, issues and impact. international journal of operations & production management, (28:9), pp 831-857. wu, f., zsidisin, g. a., & ross, a. d. (2007). engineering management antecedents and outcomes of e-procurement adoption: an integrative model. ieee transactions on engineering management, (54:3), pp 576 587. zavattaro, s. m. (2013). social media in public administration’s future: a response to farazmand. administration & society, online first (xx:x), pp 1-14. an earlier version of this paper was presented at the australasian conference on information systems (acis) 2013 in melbourne, australia. paper title (paper title style) australasian journal of information systems keruzer, born & bernius 2015, vol. 19, pp. s175-s196 who needs to be informed? who needs to be informed? – empirical results from a field experiment on the adoption of iois among smes stanislav kreuzer goethe-university frankfurt kreuzer@wiwi.uni-frankfurt friedrich born goethe-university frankfurt fborn@wiwi.uni-frankfurt steffen bernius goethe-university frankfurt bernius@wiwi.uni-frankfurt abstract inter-organizational information systems (iois) play a critical role in today’s organizations and their relationships with business partners. while large organizations already began utilizing iois at the outset, small and medium-sized enterprises (sme) have subsequently been reluctant to adopt and use iois. as such systems are subject to high network effects, a firm thus has to reach out especially to its sme partners to achieve a critical mass of adopters among them. prior research agrees that the provision of support in terms of circumstantial information and expertise can influence organizational adoption decisions. however, research in this direction has remained inconclusive. this study conducts a controlled field experiment at the organizational level to investigate the provision of support as a non-coercive persuasion strategy to foster the adoption of iois among 203 sme business partners of a large german organization. a cluster analysis is further conducted to identify distinct clusters of iois adopters showing significantly different adoption rates that result from informing them as a strategy. the results first offer evidence for the importance of informing smes as a viable strategy to foster iois adoption among them. furthermore, the results provide empirical evidence for the presence of particular arrangements of characteristics describing the strategy and structure of analysed organizations that ultimately interact with the effect of the provision of support as a persuasion strategy. keywords: inter-organizational information systems; organizational adoption; small and medium-sized enterprises; configuration analysis; persuasion strategy 1 introduction the adoption and use of inter-organizational information systems (iois) to conduct electronic business transactions across organizational borders is largely acknowledged to positively influence the financial and operational performance of today’s organizations (ke et al. 2013; rai et al. 2006), as well as to generate considerable business value through is integration and interoperability with business partners (loukis and charalabidis 2012). iois have existed for several decades (robey et al. 2008), and have been successfully utilized in numerous contexts and industries such as electronic data exchange (edi) (massetti and zmud 1996), electronic business-to-business (b2b) marketplaces (howard et al. 2006), or internet-based ordering platforms (reimers et al. 2014). while large organizations began utilizing iois from their beginning, small and medium-sized enterprises (sme) have been largely reluctant about their use in the past (iacovou et al. 1995), mainly due to the high setup and running costs of such systems, while presenting only limited benefits to smes (kauffman and mohtadi 2004). even today, notwithstanding the fact that iois technology has vastly evolved from custom-built systems based on proprietary standards to internetand open standard-based systems (zhu, kraemer, gurbaxani, et al. 2006), lowering the financial entry hurdle and thus making iois particularly valuable for smes, adoption and use of such systems among smes has remained low, with an average rate below 23% (raymond et al. 2013). for organizations considering s175 australasian journal of information systems keruzer, born & bernius 2015, vol. 19, pp. s175-s196 who needs to be informed? adopting or which already have adopted a particular iois, this is especially troublesome as such systems are subject to high network effects, meaning that their value for each adopter largely depends on the total number of adopters using them (zhu, kraemer, gurbaxani, et al. 2006). to realize the full potential of an iois implementation, a firm thus has to reach out especially to its sme business partners and achieve a critical mass of adopters among them (au and kauffman 2001). numerous studies have been conducted to understand the organizational iois adoption decisions of smes, and to present decision makers with persuasion strategies to foster adoption among sme business partners (robey et al. 2008). extant research introduced coercive as well as non-coercive strategies that organizations can follow to foster adoption among their business partners (hart and saunders 1997). for instance, barua and lee (1997) and iacovou et al. (1995) identified the exertion of bargaining power (e.g. sanctioning undesired behaviour) as a strategy to coerce sme suppliers to adopt. in a similar vein, iacovou et al. (1995) and ramamurthy et al. (1999) introduced the provision of support (e.g. offering information and expertise, or financial and technical support) as a non-coercive strategy to increase awareness and technology readiness of sme suppliers and thus foster adoption among them. however, while the exerting of coercive power has been largely acknowledged as an effective strategy in this context (teo et al. 2003), the strategy of providing support to sme business partners has received less attention from prior research (son et al. 2008). as a review of that literature shows, investigations of the provision of support to smes have remained inconclusive. while several studies do report the provision of support to be a driver of organizational adoption in this context (son et al. 2008), there are also studies arguing that the provision of support does not influence organizational decisions to adopt iois (ulu and smith 2009). likewise, the review shows that experimental investigations of the provision of support have ranged from scarce to non-existent to date, leaving its causal relationship to organizational adoption of iois among smes experimentally unconfirmed. finally, as the iois technology has been evolving dramatically over the last decade (reimers et al. 2014), prior research can only profit from a critical re-examination in the light of the modern technology that is available for adoption by today’s smes (robey et al. 2008). this study sets out to experimentally investigate the effects of providing support in terms of information and expertise to sme business partners as a non-coercive strategy to foster adoption of iois among them. therefore, the study conducts a controlled field experiment among the sme business partners of a large german organization in its attempt to approach its partners and to foster the adoption of a particular iois through the provision of support in terms of circumstantial information and expertise. thereby, only business partners categorized as smes according to the definition given by the european union – namely, having less than 250 employees and an annual balance of less than 44 million euro – are investigated and analysed (european commission 2005). acknowledging the need for a view beyond a single organization and towards groups of organizations operating in configurations of structure and strategy in research on the adoption of iois (lyytinen and damsgaard 2011), this study furthermore incorporates a configurational view of the organization in conducting its investigation that is also the predominant view in the domains of management and organizational research (meyer et al. 1993; miller 1986). a cluster analysis is applied to the experimental results as a suitable method for the analysis of configurations (lyytinen and damsgaard 2011), with the aim of identifying among the partners of the focal firm distinct groups or clusters of smes operating in different adopter configurations. results of the experiment are then analysed in the light of the identified clusters. in so doing, this study addresses the following primary research questions: rq1: how does the provision of support in terms of circumstantial information and expertise by a focal firm to its sme business partners affect their organizational decisions towards the adoption of iois? rq2: how do differing adopter configurations, in which adopting smes are embedded and operate, affect an sme’s susceptibility to the provision of support as a persuasion strategy? s176 australasian journal of information systems keruzer, born & bernius 2015, vol. 19, pp. s175-s196 who needs to be informed? this study initially aims to observe an actual iois adoption in its natural environment. by drawing on prior research, a set of key characteristics for the adoption of sme business partners is further derived and utilized to identify distinct adopter configurations among the participating smes (meyer et al. 1993). a cluster analysis is conducted as a suitable approach for the purpose of identifying distinct clusters pertaining to different adopter configurations among the set of derived key characteristics (balijepally et al. 2011). an analysis of adoption among the identified adopter configurations then aims to uncover the influence that differing adopter configurations as such can have on the provision of support as a strategy to foster the adoption of iois among smes. thereby, the study likewise aims to provide a possible explanation for varying results in prior research on the provision of support to smes. finally, the results of this study inform decision makers on how to efficiently and effectively utilize the strategy of providing sme business partners with support by offering them information and expertise. 1.1 the need for a focus on configurations of adopting organizations in research on iois adoption by smes prior research on organizational iois adoption generally defines iois as information systems that are built around computer and communications technologies (e.g. personal computers, email), and are shared by at least two or more companies (johnston and vitale 1988). such information systems can be based on various technologies and standards (zhu, kraemer, gurbaxani, et al. 2006). while in the past iois were largely based on proprietary systems (e.g. proprietary software), and proprietary standards such as edi (venkatraman and zaheer 1990), modern iois however shift from using proprietary systems and standards to the use of open and internet-based systems that are based on open standards such as xml (reimers et al. 2014). prior investigations established an ample body of knowledge explaining organizational decisions towards adopting iois, and identified various adoption factors that describe aspects of the iois technology, the organization, as well as an organization’s position in a particular business environment (robey et al. 2008). these prior investigations often relied on rogers' (1962) diffusion of innovations (doi) theory to explain iois adoption decisions as optional innovation-decisions from a purely rationalistic perspective on adoption (robey et al. 2008). in the context of iois adoption, doi theory and the rationalistic perspective have therefore been criticized with regard to their ability to explain organizational adoption decisions in this context (chau and tam 1997). specifically, lyytinen and damsgaard (2001) and fichman (2004) criticize doi’s ignorance of the complex business environment in which organizations are embedded, as doi’s primary focus lies in examining singular technologies being adopted autonomously by individual adopters that are not embedded in complex networks. this limitation is likewise reported by lee and cheung (2004) in the context of iois adoption by smes, describing doi as applying an individualist approach while ignoring the influence of organizational and environment factors. nevertheless, the domains of organizational and management research in particular value a focus on the organization and its environment, as the unit of analysis – the organization – is itself regarded as a set of tightly interconnected structures and practices (fiss et al. 2013), continuously operating in a complex business environment and ever responding to its various characteristics (bandiera et al. 2011). management and organizational research thereby broadly acknowledge a configurational perspective in research of organizations that assumes that organizations operate in particular arrangements of organizational structure and strategy within their external business environment (miller 1986). organization theory further argues that organizations are best understood as organized clusters of interconnected factors rather than by analysing distinct components in isolation (fiss 2007; fiss et al. 2013), as one central observation in organizational analysis is that characteristics are often interdependent in essential ways (meyer et al. 1993). such organized clusters thereby can be described by configurations of conceptually distinct characteristics that commonly occur together and form an integrative and meaningful whole (meyer et al. 1993). this observation is particularly valid s177 australasian journal of information systems keruzer, born & bernius 2015, vol. 19, pp. s175-s196 who needs to be informed? in the case of iois adoption, as organizations that adopt iois are likewise seen to align along a limited set of coherent patterns of strategy and structure called adopter configurations according to characteristics describing their particular structural and strategic arrangements (lyytinen and damsgaard 2011). thus, investigations encompassing this interrelatedness can lead to more detailed insights into the mechanisms of organizational adoption of iois among smes and larger firms alike. for example, reported synergies between organizational and technological characteristics among business partners are able to provide an explanation for particular adoption decisions when benefits such as compatibility are perceived on both sides of a trading relationship (venkatesh and bala 2012). similarly, reported moderation effects of organizational characteristics provide an explanation for variations in iois adoption decisions (sila 2010). therefore, it is argued that the particular adoption configuration an individual organization operates in is crucial in order to understand its adoption decision, for as a whole it may interact significantly with organizational adoption decisions and thereby provide valuable insights into why apparently structurally similar adopters make different decisions concerning iois adoption (lyytinen and damsgaard 2011). 1.2 informing sme partners as a key strategy to foster iois adoption among them an organization’s decision to adopt a technological innovation is in most cases preceded by a high level of uncertainty (fichman 2004). to counter this uncertainty, organizations tend to acquire circumstantial information about the innovation prior to the decision to adopt it (rogers 1962). smes, however, differ from large firms in various ways (thong 1999). smes reportedly suffer from limited information search and processing capabilities (thong and yap 1995), a lower level of awareness and a lack of is knowledge and technical skills (premkumar and roberts 1999; thong 1999), as well as limited financial resources and workforce (thong 1999). thus, a number of primary challenges have to be overcome by smes in order for them to be able to successfully adopt and use it technologies such as iois. the provision of support has been investigated in prior research as a strategy to overcome the aforementioned challenges of smes in the context of iois adoption, however these have yielded differing results in two primary research streams. one research stream relied on economic analysis and simulation to explain the effects of the provision of support to (nondominant) suppliers of a focal (dominant) firm. riggins et al. (1994, 1995) investigated iois adoption among suppliers of a focal firm, suggesting that the provision of financial incentives to suppliers may foster adoption among them. similarly, nault et al. (1997) stress that besides financial incentives, the provision of information and expertise can contribute to successful iois adoption and implementation. on the contrary, barua and lee (1997) argue that the adoption decisions of suppliers to join an iois network are largely attributed to strategic necessity, based on the degree of their dependence on the focal firm. according to their analysis, neither financial incentives nor penalties influenced suppliers’ adoption decisions. mamer and mccardle (1987) and ulu and smith (2009) further argue that the provision of information does not necessarily lead to an increased intention to adopt an iois but can indeed lead to a more rapid decision on whether to adopt or reject it. the other research stream relied on empirical investigations to study the influence of the provision of support on smes decisions to adopt iois, acknowledging that informing business partners about benefits and providing assistance in implementing the technology do present viable forms of persuasion strategies to foster adoption among them (hart and saunders 1997). conducting an empirical study among seven companies, iacovou et al. (1995) conclude that the provision of financial and technical assistance (e.g. offering information and technical expertise besides financial incentives) is able to foster adoption among smes. similarly, thong et al. (1996) present the results of a structural equation modelling (sem) based analysis of survey results among 114 smes, and report that the provision of is expertise from external consultants and technology vendors to smes is of critical importance as a means of fostering adoption of iois among smes. by analysing survey data from 194 smes utilizing sem s178 australasian journal of information systems keruzer, born & bernius 2015, vol. 19, pp. s175-s196 who needs to be informed? techniques, raymond and bergeron (1996) likewise report that cooperation received from the various business partners involved significantly influenced decisions of smes to adopt iois. ramamurthy et al. (1999) analyse the responses from executives of 83 smes utilizing sem techniques, and conclude that the provision of information and expertise is a key variable for influencing the iois adoption decisions of sme business partners, and thus suggest that firms provide greater support to partners who may not have the necessary expertise and knowledge. khalifa and davison (2006) analyse the data gathered from a survey among 114 smes utilizing sem techniques, and find that the provision of support is among the significant predictors of organizational adoption in this context. son et al. (2005) and son et al. (2008) further provide empirical evidence for the influence of providing support to, and the degree of cooperation with, smaller suppliers on their adoption and use of iois. according to their investigations, this strategy is even more effective when long term relationships with a higher degree of cooperation are desired by business partners. in summary, we make several key observations from reviewing prior investigations of the provision of support to smes. first, the review shows a fair amount of variability in results concerning the effect of the provision of support on organizational adoption decisions of smes. while economic investigations do not necessarily suggest the existence of a significant effect, empirical investigations nevertheless provide evidence for a significant influence of the provision of support on smes’ adoption decisions in this context. second, prior investigations have largely been ignorant of the possible interrelatedness of analysed characteristics that can be present in research on organizations (fiss et al. 2013), primarily investigating key predictors as independent variables within statistical and economic models. therefore, it is reasonable to suggest that the effect of the provision of support on smes’ decisions to adopt iois could be dependent on the particular adopter configuration in which adoption decisions are made. finally, experimental evidence for the causal relationship between the provision of support and smes’ adoption of iois has remained scarce to non-existent to date. nevertheless, particularly experimental investigations are essential to test and confirm causal relationships derived from theory and observation, as well as to assess the size of effects (wade and tingling 2005). empirical investigations based on survey data are, however, limited in their ability to provide such results (field and hole 2003). in conclusion, we acknowledge the results of prior empirical analyses among smes and likewise expect the provision of support in terms of circumstantial information and expertise as a treatment within our experimental design to lead to a higher rate of adoption among participating smes. by increasing smes’ awareness of iois (thong 1999), counteracting their uncertainty connected with the adoption decision (edelmann and sintonen 2006; thong et al. 1996), and further by positively affecting their technology readiness (iacovou et al. 1995), we therefore propose that: h1: providing circumstantial information and expertise to sme business partners will lead to a higher rate of adoption among them 1.3 identifying key characteristics to describe the adopter configurations of smes a first step in the analysis of configurations and their influence on smes’ adoption decisions is the identification of a set of key characteristics that are able to provide an integrative and meaningful picture of sme adopter configurations in the context of organizational iois adoption (lyytinen and damsgaard 2011). this set is likewise crucial for the cluster analysis, as the chosen variables define the structure of the dataset to be analysed (balijepally et al. 2011). prior research has investigated various characteristics describing organizational structure, as well as the business environment in the context of iois adoption (robey et al. 2008), with several key characteristics being consistently reported to significantly influence organizational adoption decisions in this context. we focus on these characteristics in the following, assuming them to be the primary components likewise describing smes’ configurations of organizational strategy and structure. s179 australasian journal of information systems keruzer, born & bernius 2015, vol. 19, pp. s175-s196 who needs to be informed? 1.3.1 characteristics describing organizational structure organizational size is a key characteristic that has consistently been recognized as a driver of organizational innovation in general (damanpour 1992), as well as in smes in particular (thong and yap 1995). studies on iois adoption largely acknowledge its influence (kauffman and mohtadi 2004; zhu, kraemer, and xu 2006). similarly, organizational structure is likewise reported to affect organizational decisions in this context, as changes to organizational structure have to be made when adopting iois (teo et al. 2006). organizational structure thereby reflects the resource and routine rigidity of a firm and can ultimately prevent it from adopting an iois (bala and venkatesh 2007). technology readiness, described as the extent to which technological capabilities and knowledge are present within the respective organization (chwelos et al. 2001; iacovou et al. 1995), is another key characteristic reported to significantly influence the adoption decisions of smes and larger firms alike (thong 1999; zhu, kraemer, and xu 2006). 1.3.2 characteristics describing organizational interaction with the external environment characteristics describing environment related influence can generally be divided into five distinct types which are inherently distinct in the type of influence they have on organizational iois adoption (kreuzer et al. 2014). describing the influence of the business community on organizational decision making in this context, the type of industry of an organization has been reported to be of primary significance (markus and loebbecke 2013; zhu, kraemer, gurbaxani, et al. 2006), as it is able to provide an adequate proxy for the particular strategic arrangement of an organization, mirroring the business environment in which it is embedded (lyytinen and damsgaard 2011). the influence of business partners on organizational decision making is further described by the degree of interdependence between business partners as a primary characteristic (hart and saunders 1997). the differentiation of tiers has been investigated in this context to account for differences in interdependence (iskandar et al. 2001). furthermore, regulatory requirements can vary considerably between different regulatory environments (e.g. different countries), resulting in different decisions being made by otherwise similar organizations towards the adoption of iois (teo et al. 2006). therefore, the regulatory environment and particularly the regulatory scope of organizations are regarded as key characteristics in this context (zhu, kraemer, and xu 2006). relationship related influence further reflects the type of relationships that exist between a focal firm and its business partners through the degree of asset specificity as well as the duration of those relationships (bala and venkatesh 2007), with the latter regarded as a key characteristic in this context (venkatesh and bala 2012). finally, the particular institutional environment of a firm resulting from institutional forces that are present in its business environment can influence its decisions to adopt iois (dimaggio and powell 1983). teo et al. (2003) describe such institutional forces to be primarily characterized by the types of a firm’s competitors, customers, and suppliers. 1.4 identifying types of adopter configurations and their susceptibility to the strategy of informing prior research on organizational iois adoption acknowledges the existence of frequently occurring structural arrangements in investigations of organizational iois adoption (hart and saunders 1997; iacovou et al. 1995), as well as the ability of particular arrangements of structure and strategy to affect organizational decision making in this context (lyytinen and damsgaard 2011). management research likewise acknowledges the existence of particular strategic arrangements – market-based and hierarchical setups – inside an organizational business landscape (malone et al. 1987). lyytinen and damsgaard (2011) therefore identify four distinct types of organizational adopter configurations – dyadic, hub and spoke, industry, and community wide adopter configurations – as particular arrangements of structure and strategy that are able to affect organizational decision making. this configurational view is further supported by albrecht et al. (2005), presenting a typology of distinct iois architectures, who argue that there is no single technology that provides a complete solution, s180 australasian journal of information systems keruzer, born & bernius 2015, vol. 19, pp. s175-s196 who needs to be informed? as each is tailored to a particular strategy and purpose. in the following, we thus take a configurational view on organizations as forming organized clusters by operating in particular arrangements of organizational structure and strategy with their external environment (meyer et al. 1993). we further argue that distinct types of such configurations are present in the business landscape (lyytinen and damsgaard 2011), and that these arrangements of inherently interrelated strategic and structural components as wholes are able to significantly influence the iois adoption decisions of smes (meyer et al. 1993). dyadic configurations emerge when two autonomous organizations adopt an iois to create an electronic link for the exchange of business information (chwelos et al. 2001). configurations of this type generally mirror hierarchical strategic arrangements in supply chain contexts (malone et al. 1987), with decision making likely to be largely defined by aspects of strategic necessity (barua and lee 1997), as well as interdependence (pfeffer and salancik 1978). dyadic configurations are therefore likely to be employed primarily by sme firms manufacturing parts for higher tiers inside a supply chain context as a part of a hierarchical strategic arrangement (iacovou et al. 1995). furthermore, while smes in general are likely to have a lower level of awareness of the technology and a lack of is knowledge and technical skills (thong 1999), highly customized dyadic links and high asset specific investments can lead to modest routine (process) and resource (technical systems) rigidities in these smes that additionally impede information search and processing (bala and venkatesh 2007; thong and yap 1995). similarly, such smes are likely to be oriented on long-term relationships with only a limited set of suppliers and customers (klein and rai 2009). for smes primarily operating in configurations of a dyadic type, we therefore propose that the provision of information will have a positive effect, as these firms are likely to have a lower degree of awareness concerning new technologies and also limited information search and processing capabilities. these firms should therefore be eager to learn about them to increase competitiveness and optimize processes (thong et al. 1996). the provision of information is thus able to increase smes’ awareness of iois technology (thong 1999), reduce the organizational uncertainty of smes connected with the adoption decision (edelmann and sintonen 2006; thong et al. 1996), and increase the technology readiness of these firms (iacovou et al. 1995), ultimately leading to a higher adoption rate among them. in conclusion, we hypothesize that: h2: providing circumstantial information and expertise to sme business partners primarily operating in adopter configurations of a dyadic type will lead to a higher rate of adoption among them hub and spoke configurations span particular industries (lyytinen and damsgaard 2011), and involve at least three adopters – two or more organizations that create an electronic link to a central hub by utilizing iois technologies (koch and schultze 2011). such configurations are likely to be employed by final product assemblers and distributors that are either situated as a hub in a market-based, rather than a hierarchical, structure with their suppliers, or provide products to retail customers or other resellers on a similar basis (howard et al. 2006; reimers et al. 2014). thus, smes operating in hub and spoke configurations can be characterized as the dominant buyers or suppliers in a market-type trading relationship that are less dependent on their business partners (hart and saunders 1997). likewise, as they are situated in a marketbased structure with their partners, these firms are more interested in spreading their own iois (howard et al. 2006), rather than adopting (eventually vastly different and incompatible) systems proposed by eventually less dominant partners (markus and loebbecke 2013). for smes primarily operating in hub and spoke configurations, we therefore propose that the provision of information will have only a minimal influence on their decision making as a result of them being only marginally interested in information about iois other than their own. furthermore, rather than needing to be informed, these firms are more likely to provide information and expertise to their business partners themselves (lyytinen and damsgaard 2011). concluding, we hypothesize that: h3: providing circumstantial information and expertise to sme business partners primarily operating in adopter configurations of a hub and spoke type will not lead to a higher rate of adoption among them s181 australasian journal of information systems keruzer, born & bernius 2015, vol. 19, pp. s175-s196 who needs to be informed? industry and community-wide configurations are described as emerging due to the concept of a common good, and to be based on mixes of adopter configurations of different types (lyytinen and damsgaard 2011). while industry configurations emerge inside an industry and are initiated by particular industry associations (reimers et al. 2014), community-wide configurations emerge within a regional or national scope as a result of regional or national governmental iois initiatives (penttinen et al. 2008). such configurations largely depend on institutional involvement and therefore exhibit a more latent character (lyytinen and damsgaard 2011). furthermore, they are likely to be populated by smes that operate within various and multiple business communities and industries, and are not bound to a supply chain context (markus and loebbecke 2013). smes situated in such configurations are therefore likely to adopt a “wait and see” approach to the adoption of iois (edelmann and sintonen 2006). this behaviour is based on the high organizational uncertainty resulting from smes’ presence in multiple business communities, and of fears of failing to achieve a critical mass of business partners (teo et al. 2006), or of adopting the wrong technology (venkatesh and bala 2012). the provision of circumstantial information and expertise is, however, not likely to reduce this uncertainty, as in this case the uncertainty largely stems from the multitude of divergent information sources available to smes in these configurations rather than from the absence of relevant information itself (markus and loebbecke 2013). this can even lead to an information overload when organizational information processing capabilities are low (thong and yap 1995), resulting in a perception of lower compatibility and higher complexity of iois (kuan and chau 2001). concluding, we hypothesize that: h4: providing circumstantial information and expertise to sme business partners primarily operating in adopter configurations of an industry or community type will not lead to a higher rate of adoption among them in summary, we expect that smes’ decisions to adopt iois will be positively influenced overall by the provision of support to them in terms of circumstantial information and expertise. nevertheless, the effect of the provision of support is expected to be significantly different among particular types of adopter configurations as a result of differences in strategic and structural arrangements. 1.5 empirical study the use of experiments is a widely acknowledged method to build, refine, accumulate, and apply knowledge in the social sciences (jackson and cox 2013). experiments allow for the testing of causal relationships between independent and dependent variables, as well as assessing the size of effects (wade and tingling 2005). laboratory experiments allow the researcher to create settings similar to their real-world counterparts through accurate simulation of real-world conditions in the laboratory (grant and wall 2009). however, the high degree of control does have a strong negative impact on the generalizability (external validity) of results (kim et al. 2002; winkler and murphy 1973). on the other hand, field experiments are conducted within actual real-world settings under realistic conditions, and in so doing allow a balance between experimental control and generalizability (harrison and list 2004). furthermore, the field experiment has additional advantages when organizations are the unit of analysis. decision-making in organizations is dependent on the particular external environment in which organizations are embedded and its characteristics which they have to respond and adapt to (bandiera et al. 2011). likewise, in most cases organizational decisionmaking is a cascaded process within a tightly interconnected corporate structure and is not subject to a single person (meyer et al. 1993). therefore, such organization-wide decisions are not testable in artificial laboratory conditions, even if the same participants would have been involved (harrison and list 2004). this study focuses the firm as its unit of analysis (e.g. the business partner firms of a focal organization). thus, the field experiment was chosen as a suitable approach to analyse the posed research questions as it allows for the testing of the influence of a treatment in relation to a control group in a real-world scenario, which is not feasible by handing out questionnaires (field and hole 2003). furthermore, the field experiment allows the observation and s182 australasian journal of information systems keruzer, born & bernius 2015, vol. 19, pp. s175-s196 who needs to be informed? investigation of participants in their real-world environment, and likewise allows investigations of participants that are completely unaware that the experiment is taking place (harrison and list 2004). the field experiment is thereby able to mitigate participant bias and achieve a much higher external validity of results (field and hole 2003; jackson and cox 2013). nevertheless, there are also some downsides to conducting field experiments. field experiments suffer from a limited control over key variables, difficulties in random assignment of participants to the different groups, as well as from difficulties in organizing and planning, as well as the high costs of such experimental designs for the field (grant and wall 2009; kim et al. 2002). however, the downsides are addressed in the design of the conducted field experiment and are discussed in the following. 1.5.1 experimental setting as part of a long-term innovation strategy towards electronic document management and exchange, an automated electronic invoice processing system based on open-standards was adopted by the focal organization to optimize the invoice document flow from its business partners into the organization’s internal erp system. invoice reception by the organization has been by paper only in the past for all of its business partners. invoices were collected and periodically sent to a digitizing services provider where they were digitized to be integrated into the organization’s erp system. the new invoice processing system was able to handle electronic documents in an open-standard based portable document format (pdf) directly by extracting and restructuring the unstructured data of the invoices into the necessary form for integration into the target erp system, allowing for a faster invoice processing. thus, business partners, previously accustomed to a paper-based invoice process, were given the possibility of switching to an electronic means of direct invoice exchange by sending pdf invoices to the focal organization. the experimental design presented in the following was integrated into an address to the sme business partners of the focal organization. the aim of the focal organization thereby was to persuade business partners to switch to this new form of invoice exchange with the focal organization by applying a non-coercive persuasion strategy. 1.5.2 experimental design adequate control of variables is a crucial concern particularly for field experimental designs (grant and wall 2009). simple designs are therefore recommended when conducted within less controllable settings in the field (jackson and cox 2013). with this in mind, the design of the field experiment has been constructed to be as simple as possible. thereby, we aimed to achieve a better control of the treatment, to minimize disruptions to the daily operations of the focal organization, and to minimize administrative complexity and costs. furthermore, by designing a natural field experiment – a field experiment where the phenomenon of interest is investigated in its natural environment while participants are not aware of the experiment – we aimed to maximize external validity (harrison and list 2004). therefore, the fieldexperimental design was placed within an ongoing iois implementation project of the focal organization, allowing the data collection to be conducted at the particular point in time when the focal firm was about to address its business partners. the questioned business partners thereby did not have to decide whether they could imagine adopting in the near future, but instead whether they actually would adopt or not. furthermore, it was accurately ensured that the experiment would not be identified as such by the participating organizations. we made sure that only a small circle of people knew about the experiment while it was prepared and conducted. the group only involved several responsible managers of the focal organization, two consultants, and the authors. the field-experimental design consisted of one control group and one treatment group, with the treatment being the exogenous variable (bandiera et al. 2011). the experimental design was integrated into an address of the focal organization to its business partners in the form of a cover letter. in this cover letter, the focal organization informed participants among its business partners of its novel ability to exchange electronic invoices. the cover letter further required the participating organizations to decide whether they would adopt it or not, and to reply to the cover letter stating their decision. replies could either be sent by mail or fax, and a prepared reply sheet was provided in the cover letter. the reply sheet asked for a simple “yes” or “no” as a response regarding the adoption decision, as s183 australasian journal of information systems keruzer, born & bernius 2015, vol. 19, pp. s175-s196 who needs to be informed? well as for a signature of a responsible contact person within the respective organization. the signature guaranteed that the replied decision was not a formless intention and the issue at hand had undergone a decision process within the respective organization, culminating in a signed answer in reply to the focal organization. as shown in figure 1, our control group received a cover letter from the focal organization, which did not have a high degree of information. in fact, the participation was stated as being completely voluntary and the degree of information can be understood as no different from a normal newsletter offering information about recent developments in a summarized form. no coercive pressure was applied within the cover letters of the respective groups. for our treatment group, we varied the exogenous variable to be circumstantial. figure 1: treatment group, control group, and corresponding apparatus the treatment group received a high degree of information by receiving a detailed cover letter, an additional brochure, and an additional link to a dedicated website giving specific and very detailed information about the technical solution and the potential benefits of the e-invoicing iois. while we could not control further research conducted by the subjects on the general topic of electronic invoicing, detailed information on this particular electronic invoicing system was not publicly available. in order to prevent participants of the control group from finding the dedicated website, we prevented the usual search engine crawlers, like google’s, from indexing the homepage of the focal organization weeks in advance. after mailing the cover letter, participating organizations were given a timespan of four weeks to respond. incoming electronic invoices without a preceding explicit reply were further accepted as a positive response. after the data collection period of four weeks, all participating organizations were debriefed with another cover letter including detailed information about the specific electronic invoicing system. the focal firm also distributed this information on their website. the participants did not face any repercussions by the focal organization after the end of the experiment. 1.5.3 participants in our experiment, 203 german smes were involved. all of them were business partners of the focal firm and could be categorized into what is often called b-class partners, defining them as important but not the most important ones. the focal organization is a firm with a longstanding tradition, established over 60 years ago, and employing over 1,500 people around the world. it is a private corporation with global operations, including more than a dozen manufacturing plants, and more than two dozen international offices, while doing business in 90 countries. we chose only business partners with at least a shallow relationship to the focal firm, which was assessed through the amount of invoices exchanged per year. the minimum of received invoices per year has been set to three, while the upper limit was set to a 100 invoices per year. on the one hand, the selected group of smes accounted for a third of total annually received invoices, while it still offered a wide range of relationship strength through the frequency of annual invoice volume as a proxy. most of the subjects were suppliers of raw materials, manufactured goods, or miscellaneous services. all subjects in the group were further categorized as smes according to the definition given by the european commission, having less than 250 employees and an annual balance of less than 44 million euros (european commission 2005). this approach is acknowledged by prior research, with studies providing definitions of smes primarily relying on employee counts and annual balance information as s184 australasian journal of information systems keruzer, born & bernius 2015, vol. 19, pp. s175-s196 who needs to be informed? limiting criteria, and applying country-specific limits (thong and yap 1995; chwelos et al. 2001). in the european case, criteria provided by the european commission (2005) thus are regarded as a suitable approach. however, we had to relax the definition to assert at least one requirement to be fulfilled (using or conjunction), as annual balance information was not always available. larger business partners have been addressed separately and also personally during the migration to electronic invoicing. to ensure an adequate random assignment of participating organizations among the experimental groups in the field experiment, retained organizations were randomly distributed among the treatment and control groups. nevertheless, we introduced several constraining rules to avoid an uneven distribution of either large or small firms, and an uneven distribution of related industries. likewise, to ensure that the dependence relationship with the focal firm would not bias experimental results, the random assignment was controlled with respect to the amount of invoices exchanged with the focal firm per year as an adequate proxy for the level of dependence in the relationship (chwelos et al. 2001). after the random assignment, we checked the resulting group structures and compared them to the overall sample structure. this resulted in two almost equal groups as can be seen in table 1. information control total sample size 105 99 203 average invoice frequency per year 15 15 15 median of invoice frequency per year 8 8 8 range of invoice frequency per year 3-96 3-100 3-100 table 1: sample profile we made sure, that both groups were heterogeneous within while being homogenously structured compared to each other. the probability of receiving a certain treatment or no treatment at all was equal for each subject, which resulted in a randomized sample. with almost equal structural groups, it could then be assumed that the exogenous treatments were responsible for the difference in outcomes (bandiera et al. 2011). 1.5.4 operationalization of the key characteristics for the analysis of adopter configurations to be able to identify and analyse different adopter configurations, the identified key characteristics were operationalized as categorical variables, and objective observational data was collected for all participating organizations accordingly. all variables describing the characteristics were transformed onto a categorical scale, if needed, to mitigate otherwise present scale differences across clustering variables (balijepally et al. 2011). • the organizational size of smes was measured through a categorical variable describing the number of employees (thong 1999). smes were described as having “1 to 9”, “10 to 49”, “50 to 199” and “200 and more” employees. • organizational structure was measured by a dichotomous variable describing an organization either as having a “centralized” organization structure or a “distributed” organization structure, based on the availability and focus on subsidiaries. • technology readiness was measured by a categorical variable describing the availability and the capabilities of the corporate website (hong and zhu 2006). this approach provided a suitable proxy for the assessment of technological capabilities and thus technological readiness of firms in the internet age (zhu and kraemer 2002). technology readiness was described as “low” when no or only a basic website was available, “moderate” when a timely and up-to-date website was available, and “high” when a timely website was available which also offered dynamic interaction capabilities s185 australasian journal of information systems keruzer, born & bernius 2015, vol. 19, pp. s175-s196 who needs to be informed? (e.g. web-shop or a dynamic communication platform such as chat-based customer support). • business community influence was measured by a categorical variable describing the type of industry in which an organization operates, differentiating between “parts manufacturing”, “final assembly / wholesale”, and “service provision” (hong and zhu 2006). • business partner-related influence was measured by a categorical variable describing the degree of interdependence between the organization and its business partners. interdependence was “high” for organizations situated within a supply chain context (e.g. organizations manufacturing parts for partners of higher tiers), “moderate” at the end of a supply chain context (e.g. final assemblers or resellers), and “low” when an organization was not situated in a supply chain context (e.g. service provision, banks) (iskandar et al. 2001). • the influence of the regulatory environment was measured through the scope of operation of an organization, being either “regional”, “national”, or “international” in nature (zhu, kraemer, and xu 2006). • relationship-related influence was measured by a categorical variable describing the degree of importance of the relationship for a particular organization as having a focus on “long term relationships”, a focus on “occasional customers”, or “no focus” (venkatesh and bala 2012). • institutional forces were measured through two categorical variables describing the primary industry type of suppliers and customers of an organization (teo et al. 2003). suppliers were described as being of type “resource supply”, “parts supply”, or “no primary type”. customers were described as being of type “manufacturing”, “reseller”, or “no primary type”. data for the operationalized variables was collected from openly available data sources such as commercial and trade registers, the participating companies’ webpages, industry associations, and recent news. the data was evaluated by a four-headed research team and used for the subsequent cluster analysis. 1.5.5 cluster analysis of adopter configurations cluster analysis is regarded as a powerful approach for extracting natural configurations among data that otherwise cannot be easily seen by the naked eye (hair et al. 2013). in the context of iois adoption, lyytinen and damsgaard (2011) therefore likewise argue that, when taking a configurational perspective, cluster analysis is an approach well-suited to uncovering natural configurations among adopting organizations according to a set of key characteristics that describe the particular configurations of structure and strategy in which organizations are embedded. thus, by drawing from current theoretical conceptions on the appropriate use of cluster analysis as a method in is research (balijepally et al. 2011), we utilize cluster analysis to analyse smes’ adopter configurations according to previously derived key characteristics. the process of cluster analysis can generally be conducted according to several critical steps (hair et al. 2013). the first step of the process is the selection of objectives and the selection of clustering variables. in our case, the objective of the analysis was to identify iois adopter configurations as clusters in our data, and attribute each of them to one of the generic iois adopter configuration types. to this end, we derived a set of key characteristics from prior research on iois adoption. the next step of the process was the standardization of variables. as the identified key characteristics consisted of metric, nominal and ordinal attributes, we operationalized all key characteristics on the nominal and ordinal scale. furthermore, to mitigate any scale differences between different ordinal scales, we defined the ordinal characteristics to be treated in the same way as nominal scale characteristics (disregarding order information), thus further standardizing all key characteristics to be treated equally. the hamming distance (macwilliams and sloane 1977), a measure for similarity or dissimilarity s186 australasian journal of information systems keruzer, born & bernius 2015, vol. 19, pp. s175-s196 who needs to be informed? between two data objects with categorical attributes, was chosen as a suitable distance measure in the next step. to adequately handle the purely categorical space and produce proper results utilizing the hamming distance, we further selected a clustering algorithm developed for the purpose of clustering categorical spaces (zhang et al. 2006). the selected algorithm does not require any specific statistical models or assumptions about particular underlying statistical distributions of cluster attributes, as it utilizes a pearson chi-squared-type statistic to identify clusters among the datasets based on hamming distance vectors. the partition of the produced dataset is further insensitive to the input order of the datasets, making it also invariable in relation to particular starting solutions (hair et al. 2013). the algorithm further identifies one cluster per iteration, excluding it from the dataset for the next iteration, and thus is able to determine the number of clusters automatically. 2 results the field experiment was started in the last week of april 2013. we collected the data over a course of four weeks after we sent out the different treatments by mail. table 2 shows the responses of our participants. we gathered the positive and negative responses of each group, which have been the dichotomous variables. our results show that between the treatment group and the control group, there is no significant effect of the treatment on the rate of could be found. treatment control yes 37 32 no 31 30 no answer 37 36 p value 0.862 effect size 0.020 table 2: overview of the treatment effects in the full dataset 2.1 results of the cluster analysis participating smes were clustered among the set of identified key characteristics to address the remaining hypotheses. the cluster analysis resulted in four distinct clusters presented in table 3. the cluster centre of each cluster thereby represents the most characteristic configuration for values of the respective characteristics in this cluster. in so doing, the cluster centres describe the characteristic adopter configuration of smes in their respective cluster among the set of key characteristics. based on the characteristic configuration of key characteristics, cluster 1 thereby represents an adoption configuration of a hub and spoke type, comprising primarily small (characteristic count of 9<x<50 employees for firms in cluster 1 according to the sme definition of the european commission (2005)) smes that focus long term relationships and operate as hubs for international suppliers or customers. accordingly, cluster 2 can be attributed to the dyadic adopter configuration type, comprising primarily medium-sized (characteristic count of 49<x<250 employees for firms in cluster 2 according to the sme definition of the european commission (2005)) manufacturing smes that primarily operate within highly interdependent dyadic supply chains that cross national borders. cluster 3 can be attributed to a community-based adopter configuration type, comprising primarily microsized (based on the characteristic count of 0<x<10 employees for firms in cluster 3 according to the sme definition of the european commission (2005)) sme service providers operating on national level and providing services within various configurations and therefore not having a primary supplier or customer type. one dataset was further grouped into a fourth cluster. s187 australasian journal of information systems keruzer, born & bernius 2015, vol. 19, pp. s175-s196 who needs to be informed? however, as one dataset is not enough to identify a particular adopter configuration type, the dataset is marked as an outlier. cluster 1 cluster 2 cluster 3 outliers dataset count 66 75 61 1 industry type final assembly / wholesale parts manufacturing service provision organizational size 10 to 49 50 to 199 1 to 9 regulatory scope international international national it readiness moderate moderate moderate organizational structure centralized centralized centralized relationship importance long term relationships long term relationships long term relationships partner interdependence moderate high low primary supplier type parts supply resource supply no primary type primary customer type reseller manufacturing no primary type table 3: overview of the clusters in the dataset, and their respective cluster centres to test hypotheses h2-h4, we conducted fisher’s exact test for count data to analyse the significance of the association between the administration of our treatment and the rate of adoption among smes in the treatment and the control groups of the respective clusters of our treatment (cramer’s v) (cramér 1946). the effect size can thereby be described as small, 2003). the resulting contingency tables for the control and treatment groups of the three clusters, each counting the replies of smes in respect to their decision to adopt or reject the iois, are shown in the following table 4. cluster 1 cluster 2 cluster 3 treatment control treatment control treatment control yes 10 12 23 9 4 11 no 10 11 8 12 13 7 no response 12 11 10 13 14 12 p value (two-tailed) 1.0 0.0408 * 0.0409 * 2 statistic 0.020 5.194 5.042 effect size 0.022 0.316 0.380 table 4: overview of the treatment effects in the three clusters our results show that in the case of cluster 1, representing a hub and spoke adopter configuration, there is no significant effect of the treatment on the rate of adoption among could be found. for cluster 2, representing a dyadic adopter configuration, our results show a significant positive effect of medium size for the treatment on the rate of adoption among smes of cluster 3, our results further show a significant negative effect of medium size for the thereby supporting hypothesis h4. s188 australasian journal of information systems keruzer, born & bernius 2015, vol. 19, pp. s175-s196 who needs to be informed? 3 discussion according to the results, when treating all participants equally, the provision of support in terms of circumstantial information and expertise as a persuasion strategy resulted in no significant change in smes’ adoption rates (no support for h1). furthermore, adoption seemed to be modest at best, with nearly half of the responding smes refusing to adopt. however, significant effects of moderate size could be uncovered for the strategy of providing support among particular adopter configurations of participating smes. in the case of the dyadic-type configuration, the provision of support in our field experiment led to a significant, medium-sized positive effect regarding the adoption rate among sme business partners (support for h2). this result suggests that the provision of circumstantial information and expertise indeed increases the low awareness of smes in this group that results from only limited information search and processing capabilities by relieving them from the actual search task (thong and yap 1995). furthermore, present organizational uncertainty is reduced through the provision of circumstantial information (thong et al. 1996). finally, technology readiness of smes is increased by enhancing their understanding of and expertise in the technology (iacovou et al. 1995). the result is then a higher adoption rate of iois among the respective smes. this explanation is further supported by the only moderate degree of it readiness and the high degree of interdependence with suppliers and customers present among smes in this group, suggesting that the provision of information compensates the only limited information search and processing capabilities. smes in this group, however, are eager to learn about new technologies to increase competitiveness and to optimize processes (dyadic electronic exchanges) with business partners. in the case of the identified configuration of a hub and spoke type, no significant effect could be observed regarding the adoption rate in this group (no support for h3). this result nevertheless suggests that organizations in this configuration do not seem to be significantly affected by the provision of support in terms of circumstantial information and expertise. a possible explanation for this result could be that smes operating as hubs for suppliers and customers already possess an adequate level of awareness and technology readiness, and are not affected by the provision of already known (redundant) information and expertise (markus and loebbecke 2013). furthermore, smes operating as hubs may be more interested in convincing their business partners to adopt their own solution, rather than otherwise (howard et al. 2006). the only moderate degree of interdependence in this group likewise suggests that smes in this group are less dependent on their business partners and may be more interested in promoting their own iois, as well as informing business partners themselves (lyytinen and damsgaard 2011). in the case of the identified community-type configuration, the provision of information led to a significant negative effect of medium size regarding the adoption rate among firms in this group (support for h4). this result not only shows that smes in this group display a “wait and see” approach towards the adoption of iois that is not significantly mitigated through the provision of information and expertise (edelmann and sintonen 2006), but more importantly also suggests that the provision of information does indeed lead to an intensified perception of the iois as being more complex and less compatible by smes in this group, ultimately leading to the opposite effect of a lower adoption rate. this perception is thereby unlikely to be a result of an absence of awareness or information, but rather a result of the amount and diversity of information available to smes in this group (markus and loebbecke 2013). accordingly, in this group the low organizational information processing capabilities of smes lead to an overload of information (thong and yap 1995), resulting in the iois being perceived as more complex and less compatible (kuan and chau 2001). the low degree of interdependence and the absence of a primary supplier and customer types likewise suggest that smes in this group are operating in highly heterogeneous business environments. thus, smes may not be willing to bind themselves to a particular iois due to a fear of failing to achieve a critical mass of business partners (teo et al. 2006), or adopting the wrong technology (venkatesh and bala 2012). s189 australasian journal of information systems keruzer, born & bernius 2015, vol. 19, pp. s175-s196 who needs to be informed? in summary, the results of the conducted field experiment did not provide empirical support for an overall positive effect of the provision of support in terms of circumstantial information and expertise as a persuasion strategy to foster smes’ adoption of iois. a cluster analysis, however, was able to uncover several adopter configurations of differing types that represent differing arrangements of strategy and structure in which adopting smes operate. these differing types of adopter configurations exhibit a significant variation in adoption rates, and provide a possible explanation for variations in prior investigations of the influence of the provision of support on smes’ adoption decisions in this context. 4 conclusion and further research our objective in this study was to examine, how the provision of circumstantial information and expertise can be used as an effective persuasion strategy to foster the adoption of iois among sme business partners of a focal firm. by conducting a controlled field experiment, we were able to observe an actual iois adoption taking place in its natural environment. a cluster analysis of a set of identified key characteristics of participating smes and their environment in our field experiment further was able to explain how the provision of support as a persuasion strategy is able to affect decisions to adopt iois of smes operating in particular adoption configurations, thus answering our research question rq1. by analysing the results of our experiment in the light of the identified adopter configurations, we were further able to uncover and explain differing susceptibilities of structurally similar smes situated in different adopter configurations to the strategy of the provision of support, thus answering our research question rq2. our results show that the strategy of informing sme business partners about an iois to foster the adoption among them is not necessarily beneficial, but can be a vital strategy to foster adoption among smes operating in particular adoption configurations. however, it can also hinder the adoption in smes operating in particular others. theoretical and practical contribution this study uncovers contrary decisions of structurally similar smes by analysing their decision to adopt in the light of their operation in different configurations of strategy and structure. these differences would have been overlooked by a traditional analysis without a focus on adoption configurations. the analysis that was carried out to test h1 likewise supports this, as it did not find any significant effect of the treatment in the experimental setup. nevertheless, significant effects are indeed present (h2, h4) when focusing on differing adoption configurations within the analysis. thus, this study contributes to the existing body of knowledge by first providing experimental evidence for the causal relationship between the provision of support to, and the organizational iois adoption by, smes. second, the study provides empirical evidence for the importance of the analysis of adopter configurations to enable better explanations of organizational iois adoption by smes. in particular, this study provides an important insight for investigations on this factor of influence, suggesting that future research pay attention to the presence of arrangements of characteristics describing the strategy and structure of analysed organizations, as these are able to significantly interact with the effect of the provision of support as a persuasion strategy. such multilateral interactions thus provide a possible explanation for varying results in prior studies, and furthermore might provide even better explanations of organizational iois adoption when focused in future investigations. finally, this study provides future studies with a set of key characteristics derived from prior research that are able to provide a meaningful picture of configurations of organizational structure and strategy in the context of iois adoption by smes. besides the theoretical contribution, the results of this study are also of interest to organizations that plan to or already have adopted an iois, and now are attempting to foster its adoption among business partners through the strategy of providing support in terms of information and expertise. this study uncovers several distinct clusters of organizations that decide differently when information is provided as a persuasion strategy. while business partners operating within dyadic adopter configurations are quite susceptible to the strategy of informing, business partners operating in community-based adopter configurations will s190 australasian journal of information systems keruzer, born & bernius 2015, vol. 19, pp. s175-s196 who needs to be informed? exhibit a quite contrary behaviour, when confronted with circumstantial information. business partners operating within adopter configurations of a hub and spoke type will furthermore not be significantly affected by the strategy of informing. the results of this study suggest that, particularly in supply chain contexts, the innovativeness of smes is significantly suppressed through the high levels of customization present in dyadic links. nevertheless, the provision of information and expertise is particularly effective for smes operating in supply chain contexts, as these smes are eager to innovate, while not being overwhelmed by the information provided. the results also provide decision makers with an approach to identify business partners that are in need of information and expertise to be able to adopt a particular iois. decision makers can utilize this approach to provide support only where it is needed, and likewise are able to increase the effectiveness of this strategy by not overloading the wrong partners with redundant information. 4.1 limitations and further research this study is subject to several limitations that have to be acknowledged. first, based on the relatively short time span of the field experiment of four weeks, we had to use a proxy for adoption in our experimental design, since not every business partner had sent an invoice during this period (this can also be due to the fact that roughly half of participating smes might send only eight or fewer invoices per year to the focal organization, meaning that the mean period between invoices would be six weeks). adoption was therefore measured through formal replies that had to be signed. we thereby assumed that business partners of the focal firm would take the signed agreement seriously. furthermore, key characteristics identified in the course of this study were operationalized as categorical, and were all analysed on the nominal scale. thereby, information was lost in several characteristics to a certain degree. the rationale for this approach was to achieve a standardization of all variables prior to conducting a cluster analysis (hair et al. 2013). however, future research should investigate alternate methods of identifying adopter configurations to avoid losing information for the sake of standardization. similarly, we acknowledge that the identified key characteristics in our study do not present an exhaustive set of attributes for the identification of adopter configurations. therefore, subsequent research should focus on uncovering additional key characteristics in this context. furthermore, by investigating only non-coercive persuasion, and analysing only smes that have not been forced to adopt iois, this study focused on only one of the two primary persuasion strategies identified in prior research (son et al. 2008). future investigations in this direction should analyse both strategies – providing support and exerting coercive force – in conjunction so as to provide richer insights and implications for research and practice. questions arising from the results of this study in the context of organizational iois adoption could for example investigate which particular adopter configurations are susceptible to which particular strategy, or analyse if the application of both strategies will lead to better results. references albrecht, c. c., dean, d. l. & hansen, j. v. (2005) “marketplace and technology standards for b2b e-commerce: progress, challenges, and the state of the art”, information & management, 42(6):865-875. au, y. a. & kauffman, r. j. (2001) “should we wait? network externalities, compatibility, and electronic billing adoption”, journal of management information systems, 18(2):47-63. bala, h. & venkatesh, v. (2007) “assimilation of interorganizational business process standards”, information systems research, 18(3):340-362. balijepally, v., mangalaraj, g. & iyengar, k. (2011) “are we wielding this hammer correctly? a reflective review of the application of cluster analysis in information systems research”, journal of the association for information systems, 12(5):375-413. s191 australasian journal of information systems keruzer, born & bernius 2015, vol. 19, pp. s175-s196 who needs to be informed? bandiera, o., barankay, i. & rasul, i. (2011) “field experiments with firms”, the journal of economic perspectives, 25(3):63-82. barua, a. & lee, b. (1997) “an economic analysis of the introduction of an electronic data interchange system”, information systems research, 8(4):398-422. chau, p. y. k. & tam, k. y. (1997) “factors affecting the adoption of open systems: an exploratory study”, management information systems quarterly, 21(1):1-24. chwelos, p., benbasat, i. & dexter, a. s. (2001) “research report: empirical test of an edi adoption model”, information systems research, 12(3):304-321. cramér, h. (1946) mathematical methods of statistics, princeton university press, princeton. damanpour, f. (1992) “organizational size and innovation”, organization studies, 13(3):375402. dimaggio, p. j. & powell, w. w. (1983) “the iron cage revisited: institutional isomorphism and collective rationality in organizational fields”, american sociological review, 48(2):147-160. edelmann, j. & sintonen, s. (2006) “adoption of electronic invoicing in finnish smes: two complementary perspectives”, international journal of enterprise network management, 1(1):79-98. european commission (2005) the new sme definition – user guide and model declaration, european commission, brussels. fichman, r. g. (2004) “going beyond the dominant paradigm for information technology innovation research: emerging concepts and methods”, journal of the association for information systems, 5(8):314-355. field, a. p. & hole, g. (2003) how to design and report experiments, sage publications, london; thousand oaks. fisher, r. a. (1925) statistical methods for research workers, oliver & boyd, edinburgh, london. fiss, p. c. (2007) “a set-theoretic approach to organizational configurations”, the academy of management review, 32(4):1180-1198. fiss, p. c., marx, a. & cambré, b. (2013) “configurational theory and methods in organizational research: introduction”, in configurational theory and methods in organizational research, research in the sociology of organizations, p. c. fiss, b. cambré & a. marx (eds.), 38:1-22, emerald group publishing limited, bingley. grant, a. m. & wall, t. d. (2009) “the neglected science and art of quasi-experimentation: why-to, when-to, and how-to advice for organizational researchers”, organizational research methods, 12(4):653-686. hair, j. f., black, w. c., babin, b. j. & anderson, r. e. (2013) multivariate data analysis, pearson education limited, harlow. harrison, g. w. & list, j. a. (2004) “field experiments”, journal of economic literature, 42(4):1009-1055. hart, p. & saunders, c. (1997) “power and trust: critical factors in the adoption and use of electronic data interchange”, organization science, 8(1):23-42. hong, w. & zhu, k. (2006) “migrating to internet-based e-commerce: factors affecting ecommerce adoption and migration at the firm level”, information & management, 43(2):204-221. s192 australasian journal of information systems keruzer, born & bernius 2015, vol. 19, pp. s175-s196 who needs to be informed? howard, m., vidgen, r. & powell, p. (2006) “automotive e-hubs: exploring motivations and barriers to collaboration and interaction”, journal of strategic information systems, 15(1):51-75. iacovou, c. l., benbasat, i. & dexter, a. s. (1995) “electronic data interchange and small organizations: adoption and impact of technology”, management information systems quarterly, 19(4):465-485. iskandar, b. y., kurokawa, s. & leblanc, l. j. (2001) “adoption of electronic data interchange: the role of buyer-supplier relationships”, ieee transactions on engineering management 48(4):505-517. jackson, m. & cox, d. r. (2013) “the principles of experimental design and their application in sociology”, annual review of sociology, 39(1):27-49. johnston, h. & vitale, m. (1988) “creating competitive advantage with interorganizational information systems”, management information systems quarterly, 12(2):153-165. kauffman, r. j. & mohtadi, h. (2004) “proprietary and open systems adoption in eprocurement: a risk-augmented transaction cost perspective”, journal of management information systems, 21(1):137-166. ke, w., liu, h., wei, k. & hua, z. (2013) “supply chain exploitation, exploration, and firm performance: effects of top management and information technology capabilities”, in proceedings of the 34th international conference on information systems (icis), december 15-18, milano, italy. khalifa, m. & davison, r. n. (2006) “sme adoption of it: the case of electronic trading systems”, ieee transactions on engineering management, 53(2):275-284. kim, b., barua, a. & whinston, a. b. (2002) “virtual field experiments for a digital economy: a new research methodology for exploring an information economy”, decision support systems, 32(3):215-231. klein, r. & rai, a. (2009) “interfirm strategic information flows in logistics supply chain relationships”, management information systems quarterly, 33(4):735-762. koch, h. & schultze, u. (2011) “stuck in the conflicted middle: a role-theoretic perspective on b2b e-marketplaces”, management information systems quarterly, 35(1):123-146. kreuzer, s., krönung, j. & bernius, s. (2014) “dismantling the environmental context the role of environmental characteristics in the organizational adoption of open standard-based inter-organizational information systems”, in proceedings of the 22nd european conference on information systems (ecis), june 5-13, tel aviv, israel. kuan, k. k. y. & chau, p. y. k. (2001) “a perception-based model for edi adoption in small businesses using a technology–organization–environment framework”, information & management, 38(8):507-521. lee, m. k. o. & cheung, c. m. k. (2004) “internet retailing adoption by small-to-medium sized enterprises (smes): a multiple-case study”, information systems frontiers, 6(4):385-397. loukis, e. & charalabidis, y. (2012) “business value of information systems interoperability – a balanced scorecard approach”, in proceedings of the 20th european conference on information systems (ecis), june 10-13, barcelona, spain. lyytinen, k. & damsgaard, j. (2001) “what’s wrong with the diffusion of innovation theory?”, in diffusing software product and process innovations, ifip – the international federation for information processing, m. a. ardis and b. l. marcolin (eds.), 173-190, springer, berlin; heidelberg. s193 australasian journal of information systems keruzer, born & bernius 2015, vol. 19, pp. s175-s196 who needs to be informed? lyytinen, k. & damsgaard, j. (2011) “inter-organizational information systems adoption – a configuration analysis approach”, european journal of information systems, 20(5):496-509. macwilliams, f. j. & sloane, n. j. a. (1977) the theory of error-correcting codes, elsevier, amsterdam. malone, t. w., yates, j. & benjamin, r. i. (1987) “electronic markets and electronic hierarchies”, communications of the acm, 30(6):484-497. mamer, j. w. & mccardle, k. f. (1987) “uncertainty, competition, and the adoption of new technology”, management science, 33(2):161-177. markus, m. l. & loebbecke, c. (2013) “commoditized digital processes and business community platforms: new opportunities and challenges for digital business strategies”, management information systems quarterly, 37(2):649-654. massetti, b. & zmud, r. w. (1996) “measuring the extent of edi usage in complex organizations: strategies and illustrative examples”, management information systems quarterly, 20(3):331-345. meyer, a. d., tsui, a. s. & hinings, c. r. (1993) “configurational approaches to organizational analysis”, the academy of management journal, 36(6):1175-1195. miller, d. 1986. “configurations of strategy and structure: towards a synthesis”, strategic management journal, 7(3):233-249. nault, b. r., wolfe, r. a. & dexter, a. s. (1997) “support strategies to foster adoption of interorganizational innovations”, ieee transactions on engineering management, 44(4):378-389. penttinen, e., dorota, k., harald, b., hyytiäinen, m., kuivalahti, t., poteri, j. & turunen, t. (2008) electronic invoicing initiatives in finland and in the european union – taking the steps towards the real-time economy, helsinki school of economics, helsinki. pfeffer, j. & salancik, g. r. (1978) the external control of organizations: a resource dependence perspective, harper & row, new york. premkumar, g. & roberts, m. (1999) “adoption of new information technologies in rural small businesses”, omega, 27(4):467-484. rai, a., patnayakuni, r. & seth, n. (2006) “firm performance impacts of digitally enabled supply chain integration capabilities”, management information systems quarterly, 30(2):225-246. ramamurthy, k., premkumar, g. & crum, m. r. (1999) “organizational and interorganizational determinants of edi diffusion and organizational performance: a causal model”, journal of organizational computing and electronic commerce, 9(4):253-285. raymond, l. & bergeron, f. (1996) “edi success in small and medium-sized enterprises: a field study”, journal of organizational computing and electronic commerce, 6(2):161172. raymond, l., bergeron, f. & croteau, a. m. (2013) “innovation capability and performance of manufacturing smes: the paradoxical effect of it integration”, journal of organizational computing and electronic commerce, 23(3):249-272. reimers, k., johnston, r. b. & klein, s. (2014) “an empirical evaluation of existing is change theories for the case of iois evolution”, european journal of information systems, 23(4):373-399. s194 australasian journal of information systems keruzer, born & bernius 2015, vol. 19, pp. s175-s196 who needs to be informed? riggins, f. j., kriebel, c. h. & mukhopadhyay, t. (1994) “the growth of interorganizational systems in the presence of network externalities”, management science, 40(8):984998. riggins, f. j., mukhopadhyay, t. & kriebel, c. h. (1995) “optimal policies for subsidizing supplier interorganizational system adoption”, journal of organizational computing, 5(3):295-325. robey, d., im, g. & wareham, j. (2008) “theoretical foundations of empirical research on interorganizational systems: assessing past contributions and guiding future directions”, journal of the association for information systems, 9(9):497-518. rogers, e. m. (1962) diffusion of innovations, free press, new york. sila, i. (2010) “do organisational and environmental factors moderate the effects of internetbased interorganisational systems on firm performance?”, european journal of information systems, 19(5):581-600. son, j.-y., narasimhan, s. & riggins, f. j. (2005) “effects of relational factors and channel climate on edi usage in the customer-supplier relationship”, journal of management information systems, 22(1):321-353. son, j.-y., narasimhan, s., riggins, f. j. & kim, n. (2008) “understanding the development of ios-based trading partner relationships: a structural model with empirical validation”, journal of organizational computing and electronic commerce, 18(1):3460. teo, h. h., wei, k. k. & benbasat, i. (2003) “predicting intention to adopt interorganizational linkages: an institutional perspective”, management information systems quarterly, 27(1):19-49. teo, t. s. h., ranganathan, c. & dhaliwal, j. (2006) “key dimensions of inhibitors for the deployment of web-based business-to-business electronic commerce”, ieee transactions on engineering management, 53(3):395-411. thong, j. y. l. (1999) “an integrated model of information systems adoption in small businesses”, journal of management information systems, 15(4):187-214. thong, j. y. l., yap, c.-s. & raman, k. s. (1996) “top management support, external expertise and information systems implementation in small businesses”, information systems research, 7(2):248-267. thong, j. y. & yap, c.-s. (1995) “ceo characteristics, organizational characteristics and information technology adoption in small businesses”, omega, 23(4):429-442. ulu, c. & smith, j. e. (2009) “uncertainty, information acquisition, and technology adoption”, operations research, 57(3):740-752. venkatesh, v. & bala, h. (2012) “adoption and impacts of interorganizational business process standards: role of partnering synergy”, information systems research, 23(4):1131-1157. venkatraman, n. & zaheer, a. (1990) “electronic integration and strategic advantage: a quasi-experimental study in the insurance industry”, information systems research, 1(4):377-393. wade, m. r. & tingling, p. (2005) “a new twist on an old method: a guide to the applicability and use of web experiments in information systems research”, database, 36(3):6988. winkler, r. l. & murphy, a. h. (1973) “experiments in the laboratory and the real world”, organizational behavior and human performance, 10(2):252-270. s195 australasian journal of information systems keruzer, born & bernius 2015, vol. 19, pp. s175-s196 who needs to be informed? zhang, p., wang, x. & song, p. x.-k. (2006) “clustering categorical data based on distance vectors”, journal of the american statistical association, 101(473):355-367. zhu, k. & kraemer, k. l. (2002) “e-commerce metrics for net-enhanced organizations: assessing the value of e-commerce to firm performance in the manufacturing sector”, information systems research, 13(3):275-295. zhu, k., kraemer, k. l., gurbaxani, v. & xu, s. x. (2006) “migration to open-standard interorganizational systems: network effects, switching costs, and path dependency”, management information systems quarterly, 30(si):515-539. zhu, k., kraemer, k. l. & xu, s. (2006) “the process of innovation assimilation by firms in different countries: a technology diffusion perspective on e-business”, management science, 52(10):1557-1576. copyright: © 2015 kreuzer, born, bernius. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. s196 paper title (paper title style) australasian journal of information systems lamp 2015, vol. 19, selected papers from acis editorial editorial for the acis selected papers section associate professor john lamp ajis editor-in-chief deakin business school the australasian conference on information systems (acis) is the premiere information systems conference in the region. in 2014 it was hosted by the department of business information systems at the school of business, auckland university of technology. they keynote speakers were professor robert m davison, department of information systems, city university of hong kong and dr. jürg von känel, associate director of the “ibm research – australia” lab. the conference was extremely well attended as it was run back to back with the international conference on information systems at the university of auckland. the track chairs were asked to nominate papers for inclusion in this special section of ajis, and the authors of those papers were contacted and offered a place in this section. some declined as they already had acceptances in other journals, or because of other commitments. authors were given the opportunity to update their papers for this section and most have done so. acis 2015 will be held at the university of south australia, and we look forward to publishing selected papers from that conference in 2016. copyright: © 2015 lamp. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. 1 http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems shokouhyar, noorbakhsh & aalirezaei 2017, vol 21, selected papers from ausdm evaluation of green it services 1 evaluation of green it services with fuzzy screening approach sajjad shokouhyar shahid beheshti university iran alireza noorbakhsh politecnico di milano italy armin aalirezaei semnan university a.aalirezaei@semnan.ac.ir abstract regarding development of information technology, the world of industry has inordinately benefited, albeit that has some losses. unless the losses are considered, advanced losses will be seen after progress with which is more difficult to cope. neglecting the future and the risk involved in the industry, not to mention the lack of knowledge in dealing with sudden alterations, compel irrecoverable loss. in this context, information technology services in organizations are aimed to be cost-effective and have minimum environmental impact, according to green information technology strategies. concerning significance of the issue, purpose of this research is assessment of information technology services with respect to greenness level in a general contractor organization by combination of fuzzy analytic hierarchy process and fuzzy screening procedure to enhance the greenness level of it services. the effectiveness of using this approach is including qualitative, quantitative, and uncertainty nature of the problem. in this paper, to consider the green it services criteria, literatures have been studied by meta-synthesis method, then the importance of the criteria has been determined by questionnaires so as to rank green it criteria. eventually, the organization level has been concluded in terms of the greenness level of it services. as a case study, it experts and managers of kayson inc. organization are considered as statistical population of this research. the reduction had the highest weight among other criteria recycling and reusing in kayson inc. organization. finally, the organization greenness level was determined moderate in terms of it services. keywords: green information technology, fuzzy screening, fuzzy analytic hierarchy process 1 introduction regarding the fact that every industry which enters the economy cycle of country has inordinate advantages, if the disadvantages of this industry are not considered, after its considerable spread, the damages and harms will undesirably increase. in this case, information technology (it) has a wide array of negative impacts on vital global systems such as energy and economic systems (moyer & hughes, 2012). hilty et al. (2006) also support this view by stating that the production, use, and disposal of it have a direct effect on the natural environment and eco-sustainability. the adverse impact of it infrastructure is reflected by the escalating size of carbon footprints. estimates indicate that the it industry accounts for 2% of global co2 emissions, which is equivalent to the amount generated by the aviation industry (goasduff & forsling, 2007). this phenomenon translates to the rise of global temperature and may affect rapid loss of biodiversity, due to poorly designed systems, obsolete technologies and uninformed behaviour (jenkin et al., 2011a). correspondingly, the operating expenditures of it equipment rise as much as 20 per cent annually, significantly outpacing the annual increase of overall it spending, which is 6 per cent australasian journal of information systems shokouhyar, noorbakhsh & aalirezaei 2017, vol 21, selected papers from ausdm evaluation of green it services 2 (gordon et al., 2009). green it refers to the using of it resources in an energy-efficient and cost-effective manner (bose & luo., 2011), and it is foreseen as the most important strategic technology in the imminent future. organizations are now actively pursuing green it solutions for a multitude of reasons and benefits, including lower power consumption, lower costs, lower carbon emissions, less environmental impact, improved system performance, increased collaboration and interaction amid constituents, space savings, and an agile workforce (bose & luo., 2011). according to siegler and gaughan (2008), green it is attributed to the initiatives and programs that address environmental sustainability. it also addresses energy consumption as well as waste associated with the use of hardware and software which tends to have a direct and positive impact (jenkin et al., 2011b). using green it services is applicable on condition that the managers of different parts of the organization have a fundamental planning so that the organization would be equipped with it equipment which consume minimum energy. not only should plan use such it equipment, but it also coordinates both human resources and other parts of the organization to optimally use mentioned equipment. thus, the organization can move towards playing the managerial role of green it services by optimal energy consumption and cost reduction which in turns enhance industry, economy, environment, and the competition in global markets (ruth, 2009). now, the question is how managers can comprehend the greenness level of it services in their company. it is under discussion that which criteria of green it are suitable to achieve a fundamental planning for each part of the organization based on the equipment and human resources and which methods are used to manage them. when the importance of criteria is determined, managers utilizing a practical model can conceive the greenness level of the organization it services and make the right decisions in order to save energy and reduce energy costs. the proposed framework in this research develops a useful model for making decisions to address discussed issue. the objective of this paper is measuring the greenness level of it services in a general contractor company by the approach of rating the effective criteria as to green it in an organization. the rest of the paper is organized in five sections. the following section reviews background of literature on green it and its criteria. section three develops the research model and questions. the case study, along with data and empirical findings are described in the fourth and the fifth sections respectively. after discussing the main findings in the fifth section, the sixth section highlights the main conclusion and charts avenue for future research. 2 literature review the first time the term “green it” appeared was 2007 in cio magazine (overby, 2007). green it is a systematic application of environmental sustainability criteria to the design, production, sourcing, use and disposal of the it technical infrastructure as well as within the human and managerial components of the it infrastructure in order to reduce it, business process and supply chain related emissions and waste and improve energy efficiency (molla et al., 2009). although practitioners have begun to focus on ‘green it’, there is little research in this area. in the face of ever-increasing energy costs, researchers, practitioners, and governments are now seeking effectively initiatives to regulate inefficient energy use by global business enterprises. for example, the uk government aims at decreasing greenhouse gas by 20% before 2020 (capra and merlo, 2009). the combination of the eco-sustainability and the it infrastructure perspectives can offer useful insight in conceptualizing green it. as more organizations become concerned with green it, there is a need to understand their capability for implementing and sustaining green it strategies and polices. (mines. 2008). green it policy encompasses the frameworks an organization puts in place to apply environmental criteria in its it related activities. it measures the extent to which green issues are encapsulated in organizational procedures guiding the sourcing, use and disposal of the it technical infrastructure and the activities of the it human infrastructure (goasduff & forsling, 2007; info~tech 2007a, 2007b, 2008; olson, 2008). australasian journal of information systems shokouhyar, noorbakhsh & aalirezaei 2017, vol 21, selected papers from ausdm evaluation of green it services 3 bose & luo (2011) attempt to develop a theoretical framework that illustrate how business transformation via virtualization may assist organizations assess their potential to undertake green it initiatives. they develop an integrative framework by synthesizing three information system (is) theories: (a) technology-organization-environment, (b) process-virtualization, and (c) diffusion of innovation. using the resource-based view as the theoretical foundation, dao et al (2011) develop an integrated sustainability framework, illustrating the integration of human, supply chain, and it resources to enable firms develop sustainability capabilities, which help them deliver sustainable values to relevant stakeholders and gain sustained competitive advantage. particularly, the role of automate, informate, transform, and infrastructure it resources are examined in the development of sustainability capabilities. jenkin et al (2011) develop a multilevel green it/s research framework by placing the fragmented green it/s literature in conversation with environmental sustainability literatures in management, environmental psychology, and social marketing. molla et al (2011) report a green it readiness framework to capture the input, transformational and output capabilities that organizations need to nurture in sustainable management of it. it identifies five components of green it readiness (g-readiness) and provides an exploratory framework and a research-ready instrument. uddin et al (2012) propose a green it framework using virtualization technology to achieve power and energy efficiency in data centres. the framework highlights the importance of implementing green metrics like power usage effectiveness (pue) and data centre effectiveness, and carbon emission calculator to measure the efficiency of data centre in terms of energy utilization and carbon dioxide (co2) emissions. uddin and rahman (2012) introduce a framework uses latest energy saving techniques like virtualization, cloud computing and green metrics to achieve greener data centres. green metrics like power usage effectiveness, data center effectiveness and carbon emission calculator to measure performance of individual components have been applied to the proposed framework so that benchmarking values can be achieved and set as standard to be followed. bai and sarkis (2013) present grey-based fuzzy topsis as a generalized method to strategically evaluate and justify, based on multiple sustainability factors, a green and sustainable information technology, specifically a green data centre design. wang et al (2015) develop a multi theoretical framework to provide a holistic understanding of the process of implementing and adopting green is initiatives. the proposed framework examines how these initiatives’ structures, organizational attributes, and environment may influence this process. thomas et al (2015) find explaining the adoption of green it. they evaluate the adoption stages of green it. they validate that the pre-adoption stage (green it initialization) influences the formal stage of adoption (green it integration) and, its integration into the firm’s value chain activities (green it maturation). dolci et al (2015) also provides empirical experiences and characteristics of organizations that adopt green it practices. their adopted theoretical framework emphasized it teams' role in institutionalizing the environmental dimension of sustainability in organizations. the framework tried to interrelate technology, people, and organizations' institutional properties. they show the adoption of more flexible green technologies depends on the users' acceptance so as to become institutionalized. addition to above study, literatures are scrutinized on green it criteria by meta-synthesis method (jensen & allen, 1996) with the aim of developing a comprehensive list of green it criteria. each mentioned model in literature has endeavoured to assess the it greenness level of an organization separately in either qualitative or quantitative point of view, whereas to grasp greenness level of it services it is required to assess in both qualitative and quantitative aspects. as regards above literature, a framework measuring both qualitative and quantitative green it criteria based on their influence on greenness of it services has not been yet provided. the contribution of this paper, given all presented criteria in the literature, is now providing a comprehensive model containing the criteria. with such a businesslike framework, general australasian journal of information systems shokouhyar, noorbakhsh & aalirezaei 2017, vol 21, selected papers from ausdm evaluation of green it services 4 contractor organizations can determine their current it performance with respect to green it initiatives (greenness level of it services). presented objectives will be achieved by means of combining fuzzy analytic hierarchy process (fahp) and fuzzy screening procedure (fsp) since the former ranks multiple criteria and the latter determines the level of greenness. finally, according to obtained greenness level of it services, conscious decisions will be made with respect to related criteria with aim of enhancing level of sustainability. based on the literature, a comprehensive list of green it services criteria was compiled in table 1. criteria reference reduction (dao et al.,2011 ; mithas et al.,2010 ; akman & mishra,2015) recycling (olson.,2008; mithas et al.,2010) reuse (bose & luo., 2011 ; fernando,2015) the use of backup power system (gan et al., 2007 ; kant,2009) the use of data center model (kipp et al., 2012 ; peng,2013) designing appropriate cooling system in data centers (sharma.s & sharma.g, 2016) virtualizing server and desktop (tomlinson ,2010 ; olzak et al.,2010 ; bose & luo., 2011) composing server's structure (norton.,1996) the use of more internet access (tomlinson ,2010 ; manju et al., 2014) buying environmentally compatible technologies (tomlinson ,2010 ; höjer& wintzell., 2012 ; zelenika & pearce, 2014) buying modern information technologies (mansell & when.,1998 ; tomlinson ,2010 ; davidson et al.,2011) the use of office automation software system (zhang et al., 2003; uddin et al.,2015) the use of video conferencing (molla & abareshi,2012 ; hedman & henningsson.,2016) energy management of personal computers (affanni et al., 2005 ; tomlinson ,2010 ; ozturk et al., 2011) implementing telework (anthony & majid ,2016 ; hynes , 2013) asset disposal to include it equipment recycling (bose & luo., 2011) management of recycling system (simmonds & bhattacherjee ,2012 ; fitzsimmons et al., 2013) the recycle information technology equipment (choi et al., 2006 ; sinha,2011) sustainable packaging (tomlinson ,2010 ; frehe & teuteberg, 2014 ; dubey et al,2015) product redesign (porter & van der linde., 1996 ; wolf et al.,2012 ; mysore,2012) cost of new product assembly (bramorski et al., 2000 ; kalpakjian & schmid., 2006 ) sustainable design (tomlinson ,2010 ; jenkin et al., 2011 ; garbie, 2016) remanufacture (putri & hudirarto.,2015) table 1comprehensive list of green it criteria 3 method the research method used literature review, questionnaire assessment and case study. firstly, literature of green it criteria and their impacts on it services sector have been studied by australasian journal of information systems shokouhyar, noorbakhsh & aalirezaei 2017, vol 21, selected papers from ausdm evaluation of green it services 5 meta-synthesis method (erwin et al., 2011). considering keywords “green information technology” and “criteria” 518 articles was found through online data sources, then the search was narrowed down in accordance with relevant abstract, content, title. consequently, 42 papers were found by the end of year 2016. secondly, after extracting criteria from the final stage of meta-synthesis method, questions have been designed according to gathered criteria (table 1) by the authors, then questionnaires were distributed between experts of kayson inc. as a general contractor. at the end, by utilizing a combination method of fuzzy ahp and screening the greenness level of it services in the general contractor organization was defined. figure 1 shows the schematic form of the research methodology. figure 1the schematic of research methodology 3.1 hierarchical structure of the conceptual model in the hierarchical structure of the conceptual model, based on criteria relationships obtained from meta-synthesis method, all of criteria mentioned in table 1 were categorized into three major criteriareduction, recycling, and reuseas shown in figure 2. australasian journal of information systems shokouhyar, noorbakhsh & aalirezaei 2017, vol 21, selected papers from ausdm evaluation of green it services 6 green it reduction reuserecycling product redesign sustainable design cost of new product assembly remanufacture management of recycling system the recycle information technology equipment sustainable packaging asset disposal to include it equipment recycling the use of video conferencing designing appropriate cooling system in data centers virtualizing server and desktop energy management of personal computers implementing telework the use of data center model the use of backup power system composing server's structure the use of office automation software system the use of more internet access buying environmentally compatible technologies buying modern information technologies figure 2hierarchical structure of the conceptual model in the hierarchical structure of the proposed model, the purpose is located in the first level. in the second level and the third level, the main criteria and sub criteria are located respectively. regarding significance of the issue, the purpose of the research model is assessment of it services by fuzzy screening to enhance greenness level of a general contractor organization. in this plan, the problem is solved by means of a two-phased methodology. in the first phase, ahp approach is used so that evaluating organization in respect of green it criteria and subcriteria. to do this, two questionnaires determined the importance of criteria and weight of them were token to reporters of the it department. an ahp method is used for the aims of approaching the priority set of criteria concerning green it services objectives. in the second phase, assessment of it services is applied by fuzzy screening to calculate and enhance the greenness level of organization. using the developed methodology will eventually result in a ranking of it services criteria that makes it possible for decision makers to select the most suitable service(s) concerning green objectives in organizations. 3.2 fuzzy ahp the fuzzy ahp method is to combine the hierarchy analysis with the fuzzy set theory, and add the triangular fuzzy function into the pair-wise comparison matrix to obtain the fuzzy weight australasian journal of information systems shokouhyar, noorbakhsh & aalirezaei 2017, vol 21, selected papers from ausdm evaluation of green it services 7 values of various objects (buckley, 1985). it will avoid having extreme subjective perception and uncertainty. regarding fuzzy approach, linguistic variables and fuzzy numbers have been used as shown in table 2. the steps of fuzzy ahp method follow (chang, 2004). first, through the survey of experts, the relative importance level of each pair of factors can be obtained with pair-wise comparison matrix. to integrate the experts’ opinions, saaty (1980) suggested taking the ‘geometric mean’ as the integration value. the pair-wise comparison matrix �̃�𝑖 is shown below. �̃�𝑖 = [ 1 �̃�21 �̃�12 1 … … �̃�1𝑛 �̃�2𝑛 ⋮ ⋮ ⋱ ⋮ �̃�𝑛1 �̃�𝑛2 … 1 ] �̃�𝑖𝑗 = (𝐿𝑖𝑗 ‚ 𝑀𝑖𝑗 ‚ 𝑈𝑖𝑗) �̃�𝑖𝑗 * �̃�𝑗𝑖 ≈ 1, ∀i, j = 1, 2, ..., n where 𝐿𝑖𝑗 ‚ 𝑀𝑖𝑗 ‚ 𝑈𝑖𝑗 are the left value, middle value, right value of the triangular fuzzy function that represents the relative importance level of factor i to factor j, respectively. the second step is to verify the consistency for pair-wise comparison matrix and delete those that do not fit the qualification. the third step is to calculate the fuzzy weight on every row of the fuzzy positive reciprocal matrix, �̃�𝑖 as follows: �̃�𝑖 = [ �̃�𝑖1 ⨂ … ⨂ �̃�𝑖𝑛 ] 1 𝑛, ∀i = 1, 2,..., n (1) �̃�𝑖 = �̃�𝑖 ⨂(�̃�1 ⨁ �̃�2 ⨁ … ⨁ �̃�𝑛 ) −1 (2) where �̃�𝑖 is the geometric mean of triangular fuzzy number; �̃�𝑖𝑗 is the triangular fuzzy number located on row i and column j of the fuzzy positive reciprocal matrix. the fourth step is to use weighted method to defuzzy the triangular fuzzy number, and obtain the best non-fuzzy value or crisp value. the calculation is shown below. 𝐷𝐹𝑊𝑖 = [ (𝑈𝑖 −𝐿𝑖 )+(𝑀𝑖 − 𝐿𝑖 )] 3 + 𝐿𝑖 (3) the fifth step is to standardize the weight value obtained for every factor, and make it a unique crisp value. 𝑊𝑖 = 𝐷𝐹𝑊𝑖 ∑ 𝐷𝐹𝑊𝑖 (4) finally, if 𝑓𝑗 = (𝑓𝑗1‚ … ‚𝑓𝑗𝑘) is the fuzzy weight of every factor 𝐴𝑗, and �̃� = (�̃�1 ‚ … . . ‚ �̃�𝑘) is the fuzzy weight of the upper level e, then the weight of every factor is shown as follows: �̃�𝑗 = (𝑓𝑗1 ⨂ �̃�1 ) ⨁ … ⨁ (𝑓𝑗𝑘 ⨂ �̃�𝑘 ) (5) fuzzy triangular scale linguistic scale code (1,1,1) just equal 1 (0.5,1,1.5) equal important 2 (1,1.5,2) weakly more important 3 (1.5,2,2.5) strongly more important 4 (2,2.5,3) very strongly more important 5 (2.5,3,3.5) absolutely more important 6 (3,3.5,4) extremely important 7 table 2linguistic variables and fuzzy numbers australasian journal of information systems shokouhyar, noorbakhsh & aalirezaei 2017, vol 21, selected papers from ausdm evaluation of green it services 8 3.3 fuzzy screening the fsp is useful for the selection of a small subset of alternatives from much larger set of alternatives (carlsson & fullér, 1995). each alternative is described by minimal information. the process involves multiple criteria attributes used for alternative description, as well as multiple experts whose opinion must be considered. therefore, fuzzy screening procedure (fsp) is usually considered as a multi expert – multi criteria procedure. the fsp is of following steps: (stojanov et al., 2013) step 1: formation of matrix (π); each expert for each alternative gives his/her opinion (score) how well that alternative satisfies each of the criteria. for this process, the scale s of m elements is used. formation of matrix (i); independently, each expert should provide an opinion on how important is each criteria attribute; this is done by using the same scale s. step 2: very important is the definition of the negation for the scale s according to the equation (6): 𝑁𝑒𝑔(𝑆𝑖) = 𝑆𝑚−𝑖+1 and i=1, 2, …, m (6) step 3: according to the equation (7), the integration of neg(i) and π is achieved and then its minimum is taken for each expert. 𝑈𝑖𝑘 = 𝑚𝑖𝑛𝑗{𝑁𝑒𝑔(𝐼𝑘𝑗)𝑣 𝜋𝑖𝑗𝑘} , and k=1, 2, …, r , j=1,2, … , h (7) k: experts, j: sub-criteria step 4: sorting elements obtained in step 3 according descending order. step 5: after this stage of the process, each expert evaluates each alternative. if there are r experts, then for each alternative is associated r scores. further process is an aggregation of scores in order to obtain overall score for each alternative. there are many aggregation operators, but in this case, ordered weighted averaging operator (owa) proposed by yager (yager, 1994; detyniecki, 2001) is used. the step 5 in the process is definition of the aggregation function q. as in carlsson and fullér, (1995) “this function can be seen as a generalization of the idea of how many experts it feels need to agree on an alternative for it to be acceptable to pass the screening process”. besides some trivial, as well as more complex definitions of q, the function which emulates the average is denoted as q (k): q (k) = 𝑆𝑏(𝑘) (8) 𝑏(𝑘) = 𝑖𝑛𝑡[1 + 𝑘(𝑞−1) 𝑟 ] (9) in eq. (9) int [a] returns a value that is the closest to a, q is number of points on the scale and r is the number of experts. step 6: an owa operator of dimension n is a mapping f: 𝑅𝑛 → 𝑅 that has associated vector: w = (𝑤1 ‚ … ‚𝑤𝑛 ) 𝑇 , where 𝑤𝑖 ∈ [0,1] , 1≤ 𝑖 ≤ 𝑛 . furthermore: ∑ 𝑤𝑖 𝑛 𝑖=1 =1, and f (𝑎1‚ … ‚𝑎𝑛 ) = ∑ 𝑤𝑗 𝑛 𝑖=1 𝑏𝑗 , where 𝑏𝑗 is the j-th largest element of the bag <𝑎1,..., 𝑎𝑛>. after appropriately selected q, it is possible to use the owa method for aggregating experts’ opinions. the overall score a for an alternative is calculated by eq. (10): a = 𝑚𝑎𝑥𝑗{𝑄𝑗˄𝐵𝑗} (10) in eq. (10) 𝑏𝑗 is the worst of the j-th top scores. australasian journal of information systems shokouhyar, noorbakhsh & aalirezaei 2017, vol 21, selected papers from ausdm evaluation of green it services 9 4 case study: it experts and managers of kayson inc. organization founded in 1975, kayson inc. is an international general contractor company providing premier management, engineering, procurement, construction, financing and investment services worldwide. the organizational structure of kayson inc. has been designed that every project enjoys a unified chain of project management steps. a network of specialized technical, engineering, logistics and supporting units feed the projects. (kayson inc., 2016) coordinating these units, it plays a key role in kayson inc. not only is kayson inc. -general contractor-committed to safeguarding the health and safety of its staff and other stakeholders, but it is also committed to protect the environment. to achieve the goal of augmenting the positive environmental impact, green it as a practical means should be considered. therefore, much needs to be done by kayson inc. with the aim of adjusting its organizational units to applying green it services. as known, testing the validity of a proposed or developed model is an extremely important issue. in this study, the issue of validity of the proposed ahp model is considered. the model’s validity, reliability, and generalizability have assessed as following: a) variables have extracted from literature validated from pervious researches, b) application of the model is developed with the aim of assessing greenness level of it services in general contractor organizations. in addition, the model is implemented and approved as an applicable model in kayson. inc. as a general contractor by its executives expertise. therefore, the model has validity, reliability, and generalizability for assessing similar general contradictors. the developed model is applied so as to enhance the greenness level of it services. it experts and managers of kayson inc. organization are population of this research. 4.1 result and discussion the next step of the paper is to conduct surveys with the five experts from it sector of kayson inc. inc. in order to verify the model. three types of questioners designed in order to obtain weights, importance, and presently state of the criteria have been designed for the survey. detailed results are presented as follows: 4.1.1 the hierarchy of the assessment the whole hierarchy of the assessment can be easily visualized in figure 3. it involves three levels. the goal is to evaluate greenness it service level of the kayson inc. in the second level, there are three assessment areas: reduction, recycling, and reuse. eventually, there are a number of sub-criteria under these three major criteria. goal a a6 cb a7 a8 a9 a10 a11a5a4a3a2a1 a12 b1 b2 b3 b4 c1 c2 c3 c4 figure 3 the schematic hierarchy of the assessment the relative importance of each green it criteria and sub-criteria at a specific level are measured by a procedure of pairwise comparison (see appendix). the decision makers provide fuzzy weights for the priority of each criteria by using a rating scale (table 2). the process is repeated for each level of the hierarchy until a decision is finally reached by overall composite weights. the last step is to use equations (3, 4, and 5) to obtain the non-fuzzy value. table 3 presents the overall weighting information for each element in the ahp model. by synthesizing the experts' opinions with the priorities, the ratings of the green sub-criteria in respect of the australasian journal of information systems shokouhyar, noorbakhsh & aalirezaei 2017, vol 21, selected papers from ausdm evaluation of green it services 10 end and synthesized priorities for each sub-criterion have been obtained. the final ranking of criteria and sub-criteria are depicted respectively in figures 4 and 5. criteria symbol criteria name normalized fuzzy weight nonfuzzy weight a reduction (0.379,0.536,0.728) 0.545 a.1 the use of backup power system (0.065,0.125,0.219) 0.08 a.2 the use of data center model (0.062,0.122,0.216) 0.078 a.3 designing appropriate cooling system in data centers (0.045,0.092,0.17) 0.06 a.4 virtualizing server and desktop (0.05,0.096,0.174) 0.062 a.5 composing server's structure (0.048,0.092,0.175) 0.061 a.6 the use of more internet access (0.04,0.078,0.15) 0.052 a.7 buying environmentally compatible technologies (0.039,0.073,0.134) 0.048 a.8 buying modern information technologies (0.043,0.076,0.139) 0.05 a.9 the use of office automation software system (0.033,0.06,0.115) 0.04 a.10 the use of video conferencing (0.032,0.058,0.112) 0.039 a.11 energy management of personal computers (0.03,0.054,0.108) 0.037 a.12 implementing telework (0.038,0.075,0.187) 0.058 b recycling (0.222,0.306,0.44) 0.318 b.1 asset disposal to include it equipment recycling (0.227,0.354,0.516) 0.123 b.2 management of recycling system (0.244,0.332,0.465) 0.116 b.3 the recycle information technology equipment (0.126,0.174,0.245) 0.061 b.4 sustainable packaging (0.102,0.14,0.204) 0.049 c reuse (0.12,0.158,0.218) 0.164 c.1 product redesign (0.255,0.369,0.514) 0.065 c.2 cost of new product assembly (0.174,0.255,0.366) 0.045 c.3 sustainable design (0.184,0.254,0.366) 0.046 c.4 remanufacture (0.089,0.122,0.178) 0.022 table 3 the overall weighting information for each element in the ahp model figure 4final ranking of main criteria in accordance with table 3, figure 4 and 5 in kayson inc., main criteria of reduction of costs and energy has the first priority from the point of experts’ view. asset disposal including it equipment recycling had the highest priority and remanufacture had the lowest priority among the sub-criteria. among sub-criteria, the criterion of composing server's structure and the recycle it equipment have the same priority. 0.545 0.318 0.164 0 0.1 0.2 0.3 0.4 0.5 0.6 criterion a criterion b criterion c non-fuzzy weight australasian journal of information systems shokouhyar, noorbakhsh & aalirezaei 2017, vol 21, selected papers from ausdm evaluation of green it services 11 figure 5final ranking of sub criteria 4.1.2 results of fuzzy screening five men from it experts and managers of kayson inc. were asked to assign values to assess the importance of attributes selected for judging about how well that alternative satisfies each of the criteria and sub-criteria and how important is each of them attribute. as a sample in point, tables 4-10 show the result of fuzzy screening process of the reduction criteria (a). subsequently, greenness level of it services in kayson organization from viewpoint of reduction in cost, energy, and environmental impact has been calculated according to eq. (6)(10). fuzzy screening of the reduction criteria (a): to form matrix (i), independently, each expert has provided a judgment on how important is each criteria attribute according to scales of table 2. matrix i a.1 a.2 a.3 a.5 a.6 a.8 a.9 a.10 a.11 a.12 expert 1 6 5 5 5 4 7 4 3 7 4 expert 2 6 6 6 5 6 7 5 4 7 4 expert 3 5 6 5 5 5 6 5 5 6 5 expert 4 6 5 5 5 6 5 6 5 6 1 expert 5 5 4 4 5 5 6 4 3 7 5 table 4matrix (i), the importance of reduction sub-criteria according to the experts 0.123 0.116 0.08 0.078 0.065 0.062 0.061 0.061 0.06 0.058 0.052 0.05 0.049 0.048 0.046 0.045 0.04 0.039 0.037 0.022 0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 b.1 b.2 a.1 a.2 c.1 a.4 a.5 b.3 a.3 a.12 a.6 a.8 b.4 a.7 c.3 c.2 a.9 a.10 a.11 c.4 1 2 3 4 5 6 7 8 9 1 0 1 1 1 2 1 3 1 4 1 5 1 6 1 7 1 8 1 9 2 0 non-fuzzy weight su b cr it er ia r an k in g australasian journal of information systems shokouhyar, noorbakhsh & aalirezaei 2017, vol 21, selected papers from ausdm evaluation of green it services 12 then, to form matrix (π), each expert for each criteria gives his/her score (opinion) of how much that criterion is important as presently it is in kayson inc. for this process, the scale s of table 2 is used. matrix π a.1 a.2 a.3 a.5 a.6 a.8 a.9 a.10 a.11 a.12 expert 1 6 5 6 5 2 4 6 4 6 1 expert 2 6 5 5 4 6 5 5 4 6 1 expert 3 6 6 6 5 6 5 5 5 7 1 expert 4 5 5 5 4 6 5 5 4 6 1 expert 5 5 4 6 4 5 5 5 4 6 1 table 5matrix (π), the score of reduction sub-criteria according to the expert as presently it is according to the equation (6) in step 2 of fuzzy screening, negation for the matrix (i) is calculated as follow: 𝑵𝒆𝒈(𝑰) a.1 a.2 a.3 a.5 a.6 a.8 a.9 a.10 a.11 a.12 expert 1 2 3 3 3 4 1 4 5 1 4 expert 2 2 2 2 3 2 1 3 4 1 4 expert 3 3 2 3 3 3 2 3 3 2 3 expert 4 2 3 3 3 2 3 2 3 2 7 expert 5 3 4 4 3 3 2 4 5 1 3 table 6negation for the scale s of matrix (i) in the next step, according to the equation (7), the integration of neg(i) and π is achieved (table 7) and then its minimum is taken for each expert (table 8). matrix 𝑵𝒆𝒈(𝑰) 𝒗 𝛑 a.1 a.2 a.3 a.5 a.6 a.8 a.9 a.10 a.11 a.12 expert 1 6 5 6 5 4 4 6 5 6 4 expert 2 6 5 5 4 6 5 5 4 6 4 expert 3 6 6 6 5 6 5 5 5 7 3 expert 4 5 5 5 4 6 5 5 4 6 7 expert 5 5 4 6 4 5 5 5 5 6 3 table 7𝑁𝑒𝑔(𝐼) 𝑣 𝜋 b criteria b11 4 b12 4 b13 4 b14 3 b15 3 table 8assortment of 𝑚𝑖𝑛 [𝑁𝑒𝑔(𝐼) 𝑣 𝜋] australasian journal of information systems shokouhyar, noorbakhsh & aalirezaei 2017, vol 21, selected papers from ausdm evaluation of green it services 13 next step is to aggregate the scores in order to obtain overall score for each criteria. in step 5 of fuzzy screening, based on equation (8) and (9), the results of aggregation function is shown in table 9: b k=1 q (1) =2 k=2 q (2) =3 k=3 q (3) =4 k=4 q (4) =5 k=5 q (5) =7 table 9the aggregation function (q) the overall score a for a criterion (here reduction) is calculated by eq. (10). the result is shown in table 10: 𝑸(𝒋)˄𝑩𝒊𝒋 2 3 4 3 3 table 10q˄b a=max {2, 3, 4, 3, 3} =4 similarly, above steps are repeated for criteria “b” and “c”. eventually, the results of ranking and evaluating of greenness level of it services in the kayson inc. organization have been presented in table 11. criterion c criterion b criterion a company 4 4 4 kayson inc. organization table 11the results of greenness level of it services in the kayson inc. organization according to the fuzzy scales shown in table 2 and results in table 11, all three major criteriareduction, recycling, and reuse-are equally “strongly more important”. in other words, the greenness level of kayson inc. is average in terms of utilizing it services. 5 conclusion there is resistance and misunderstanding surrounding green it among some managers resulting from a general misunderstanding of the benefits of green it. green it helps the reduction of environmental impacts resulted from the activity of it in it-benefited organizations. furthermore, such enhanced impacts will be accompanied with lower total costs for companies. the equipment used in this technology are green on condition that they can be either disposed or reused after the end of their lives with minimum possible side effects on environment, in other words, the equipment should be stroke a right balance between probable adverse environmental impacts for used technology and that for available new technology. australasian journal of information systems shokouhyar, noorbakhsh & aalirezaei 2017, vol 21, selected papers from ausdm evaluation of green it services 14 after realizing the importance of green it in it-oriented organizations, then the question is how to assist managers of general contractor organizations to comprehend the greenness level of it services in their company and to make right decisions to save energy and reduce energy costs. this research proposed a framework to address this issue: firstly, literature of green it criteria and their impacts on it services sector have been studied by meta-synthesis method. a comprehensive list of criteria is created. secondly, the priority of green it criteria has been studied by using fuzzy ahp approach and then the organization level has been examined in term of the greenness of it services through fuzzy screening. finally, the proposed model has been received its’ validity, reliability, and generalizability by implementing it in kayson. inc. as a case study. the relative ease with which multiple criteria can be handled is one of the main strength of fuzzy ahp. in addition to this, ahp is easier to understand and it can effectively handle both qualitative and quantitative data (kahraman & ulukan 2003). as the focus of this study is on the indices which measure the greenness level of it services in general contractor organizations and these quantitative and qualitative indices cannot be exactly measured, fuzzy ahp method have been used to evaluate them. furthermore, highly cited studies have proposed modelling with triangular fuzzy numbers as an effective method to formulate decision-making problems where the available information is subjective and inaccurate (chang & yeh, 2002). in this paper, combination of fuzzy ahp and fuzzy screening is preferred in the ranking of organizational green indicators as this method is the only one utilizing a hierarchical structure. usage of pair-wise comparisons is another advantage of this method that provide precise information about the preferences of decision makers. furthermore, since the decision-makers are usually unable to explicit about their preferences owing to the fuzzy nature of the decision process, this method helps them providing an ability of giving interval judgments instead of point judgments (bozbura & beskese2007). this could inform decision makers of crucial factors so as to reduce the environmental impacts, not to mention reducing the organization total costs. as a case study, the model is implemented in kayson inc. which is one of the biggest general contractors in iran. the major factors of the green it services in kayson inc., namely reduction, recovery and reuse, were extracted from the research model, then the questionnaires were completed by it experts of the company. the reduction criterion refers to the reduction of energy, cost, and environmental impacts which can be achieved as the result of using the green it equipment. the recycling criterion also brings about the reduction of environmental impacts and the reuse criterion is repairing and the fundamental product correction leading to the cost reduction. the results show that in kayson inc., based upon both fuzzy ahp analysis and order of weights, reduction is the most preferential option, then recycling and eventually reusing are so. furthermore, among sub-criteria the highest priority is asset disposal including it equipment recycling and the lowest priority is remanufacturing. in addition, after fuzzy screening processes, according to the fuzzy scales given in table 2 and results in table 11, all three major criteria-reduction, recycling, and reuse-are equally “strongly more important”. in other words, the greenness level of kayson inc. is moderate in terms of utilizing it services. the kayson inc. managers should consider the following solutions according to highest weights in figure 5, in order to enhance the greenness level of it services: asset disposal to include it equipment recycling: kayson inc. does disposal it equipment with none recycling plan. choosing a waste disposal organization that follows the waste hierarchy to donate kayson inc.’s working equipment and recycle the non-working equipment, in other words, planning and executing enterprise it asset disposition programs. in the meantime, kayson inc. should store waste equipment safely. management of recycling system: kayson inc. does not environmentally take any liability to its it equipment end of life. therefore, managing the recycling system from the reduction step, storing and processing, collection and transportation, recycling, dispelling is recommended. subsequently, consideration and attention after the disposal should be taken into account. australasian journal of information systems shokouhyar, noorbakhsh & aalirezaei 2017, vol 21, selected papers from ausdm evaluation of green it services 15 the use of backup power system: the backup power systems being used in kayson inc. does not possess an efficient system to support scalability and to provide expected back up timing. the system should have the capability to start with the smaller system and to add another system later on to double up on the available peak power output. furthermore, recommended system should have the option to select the right amount of power for its needs. to have more green backup system, hydrogen fuel cell power generators is also recommended in comparison with mechanical power generation or batteries. the use of data centre model: a major concern for kayson inc. it department is slow responsiveness to business demands inasmuch as a simple request takes inordinate time to fulfil. as a forward looking company, kayson inc. should use a data center model which facilitate virtualization, simplification, automation, and accelerated delivery of cloud applications and services to provide a sustainable business advantage. to resolve the current data center model, the cisco unified data center platform was designed specifically to overcome the current data centre constraints and provide agile, simplified, and efficient it service delivery and cloud computing. cisco unified fabric, computing, and management. when it equipment reach end of their lives, kayson inc. should consider decision models with which the optimal mixture of decisions for end of life among various market segment over different it equipment life cycles can be determined. (noorbakhsh et al., 2015 ; shokouhyar & aalirezaei., 2017) 5.1 future research 1. this paper has presented a framework aiming at ranking effective green it criteria which contain mostly environmental aspects; however, it is suggested ranking sustainable it factors which is more extensive aspects containing economic and social perspectives. 2. so as to ranking green it factors, criteria have been considered independent in this study; applying other ranking methods such as anp is suggested concerning existed correlation between criteria. 5.2 limitations fuzzy ahp is a complex approach which needs too many numerical calculations in analysing combined preferences compared with the classic ahp. in addition, in ahp assumption there is no correlation between criteria which might be a drawback of ahp. in ahp, although various approaches are utilized for defuzzification, each approach extracts different levels of information from the fuzzy numbers, therefore might give different ranking output, in this case, aggregation of preferences is not easily interpretable. consequently, an alternative ranked the best may be ranked differently upon changing the defuzzification approach. this is called rank reversal or fuzzy reciprocals, which is a controversial debate in ahp method. this problem is further exacerbated with the fuzzy outputs and using different defuzzification techniques. there are two general limitations regarding conducting qualitative meta-synthesis. first, there is a limitation of discerning or measuring the quality of qualitative research. it is important to recognize that qualitative research as a body of research is not as unified as quantitative research. there are significant differences in theoretical perspectives as well as differences in methodologies. a second issue of concern in conducting qualitative meta-synthesis is the critical context may be stripped away and the rich, thick description will be sacrificed. references affanni, a., bellini, a., franceschini, g., guglielmi, p., & tassoni, c. (2005). battery choice and management for new-generation electric vehicles. ieee transactions on industrial electronics, 52(5), 1343-1349. australasian journal of information systems shokouhyar, noorbakhsh & aalirezaei 2017, vol 21, selected papers from ausdm evaluation of green it services 16 akman, i., & mishra, a. (2015). sector diversity in green information technology practices: technology acceptance model perspective. computers in human behavior, 49, 477486. anthony, b. j., & majid, m. a. (2016). green is for sustainable decision making in software management. journal of soft computing and decision support systems, 3(3), 20-34. bai, c., & sarkis, j. (2013). green information technology strategic justification and evaluation. information systems frontiers, 15(5), 831-847. bose, r., & luo, x. (2011). integrative framework for assessing firms’ potential to undertake green it initiatives via virtualization–a theoretical perspective. the journal of strategic information systems, 20(1), 38-54. bozbura, f. t., & beskese, a. (2007). prioritization of organizational capital measurement indicators using fuzzy ahp. international journal of approximate reasoning, 44(2), 124-147. bramorski, t., madan, m., motwani, j., & sundarraj, r. p. (2000). improving competitiveness of ready-to-assemble manufacturers through information technology. logistics information management, 13(4), 201-209. buckley, j. j. (1985). fuzzy hierarchical analysis. fuzzy sets and systems, 17(3), 23 capra, e., & merlo, f. (2009). green it: everything starts from the software. ecis 2009 proceedings. 453. carlsson, c., & fullér, r. (1995, august). on fuzzy screening systems. in proc. 3rd european congress on intelligent technologies and soft computing, aachen (pp. 1261-1264). chang, m. j. (2004). a study on software project risk management using fuzzy analytic hierarchy process. master thesis, national pingtung university of science and technology, department of industrial management (chinese edition). chang, y. h., & yeh, c. h. (2002). a survey analysis of service quality for domestic airlines. european journal of operational research, 139(1), 166-177. choi, b. c., shin, h. s., lee, s. y., & hur, t. (2006). life cycle assessment of a personal computer and its effective recycling rate (7 pp). the international journal of life cycle assessment, 11(2), 122-128. chowdhury, g. g. (2015). how to improve the sustainability of digital libraries and information services. journal of the association for information science and technology. dao, v., langella, i., & carbo, j. (2011). from green to sustainability: information technology and an integrated sustainability framework. the journal of strategic information systems, 20(1), 63-79. davidson, e., vaast, e., & wang, p. (2011, september). the greening of it: how discourse informs it sustainability innovation. in ieee 13th conference on commerce and enterprise computing (cec), 2011 (pp. 421-427). ieee. dolci, d. b., lunardi, g. l., salles, a. c., & alves, a. p. f. (2015). implementation of green it in organizations: a structurational view. revista de administração de empresas, 55(5), 486-497. dubey, r., gunasekaran, a., papadopoulos, t., & childe, s. j. (2015). green supply chain management enablers: mixed methods research. sustainable production and consumption, 4, 72-88. erwin, e. j., brotherson, m. j., & summers, j. a. (2011). understanding qualitative metasynthesis: issues and opportunities in early childhood intervention research. journal of early intervention, 33(3), 186-200. australasian journal of information systems shokouhyar, noorbakhsh & aalirezaei 2017, vol 21, selected papers from ausdm evaluation of green it services 17 fernando, m. s. d. (2015). green information technology managerial capabilities of it organizations in sri lanka. scientific research journal (scirj), volume iii. fitzsimmons, j., & fitzsimmons, m. (2013). service management: operations, strategy, information technology. mcgraw-hill higher education. frehe, v., & teuteberg, f. (2014). the role of ict in green logistics: a systematic literature review. in information technology in environmental engineering (pp. 53-65). springer berlin heidelberg. gan, l., eskeland, g. s., & kolshus, h. h. (2007). green electricity market development: lessons from europe and the us. energy policy, 35(1), 144-155. garbie, i. (2016). sustainability awareness. in sustainability in manufacturing enterprises (pp. 131-150). springer international publishing. goasduff, l., & forsling, c. (2007). gartner says 50 percent of mid and large sized western european it organizations will develop a green strategy by the end of 2008. gartner, surrey. gordon, j., skinner, a., & button, a. (2009). green technology: driving economic and environmental benefits from ict. in world economic forum. hedman, j., & henningsson, s. (2016). developing ecological sustainability: a green is response model. information systems journal. hilty, l. m., arnfalk, p., erdmann, l., goodman, j., lehmann, m., & wäger, p. a. (2006). the relevance of information and communication technologies for environmental sustainability–a prospective simulation study. environmental modelling & software, 21(11), 1618-1629. höjer, m., larsen, k., & wintzell, h. (2012). sustainable communications and innovation: different types of effects from collaborative research including university and companies in the ict-sector. in ict critical infrastructures and society (pp. 170-182). springer berlin heidelberg. hynes, m. (2013). what's smart about working from home? telework and the sustainable consumption of distance in ireland. in: sylvie thouësny, cathy fowley, claire english (eds.). internet research, theory, and practice: perspectives from ireland. dublin: research-publishing. info~tech, (2007b). exercise environmentally info~tech research group, october 22, pp. 19. info~tech, (2008). “use epeat to decrease equipment costs and help the environment”, info~tech research group, february pp.1-3. info~tech., (2007a). “pc power saving plans reduce costs and environmental impact”, info~tech research group, december 19, pp.1-10. jenkin, t. a., mcshane, l., & webster, j. (2011b). green information technologies and systems: employees’ perceptions of organizational practices. business & society, 50(2), 266-314. jenkin, t. a., webster, j., & mcshane, l. (2011). an agenda for ‘green’ information technology and systems research. information and organization, 21(1), 17-40. jenkin, t. a., webster, j., & mcshane, l. (2011a). an agenda for ‘green’ information technology and systems research. information and organization, 21(1), 17-40. jensen, l. a., & allen, m. n. (1996). meta-synthesis of qualitative findings. qualitative health research, 6(4), 553-560. kahraman, c., cebeci, u., & ulukan, z. (2003). multi-criteria supplier selection using fuzzy ahp. logistics information management, 16(6), 382-394. australasian journal of information systems shokouhyar, noorbakhsh & aalirezaei 2017, vol 21, selected papers from ausdm evaluation of green it services 18 kalpakjian, s., & schmid, s. (2006). manufacturing, engineering and technology si 6th edition-serope kalpakjian and stephen schmid: manufacturing, engineering and technology. digital designs. kant, k. (2009). data center evolution: a tutorial on state of the art, issues, and challenges. computer networks, 53(17), 2939-2965. kayson inc., (2016) ‘rank 99 in top 250 international contractor (enr)’, retrieved 26 july 2016 [online]. http://www.kayson-ir.com/ kipp, a., jiang, t., liu, j., fugini, m., vitali, m., pernici, b., & salomie, i. (2012). applying green metrics to optimize the energy-consumption footprint of it service centers. international journal of space-based and situated computing 5, 2(3), 158174. kipp, a., jiang, t., liu, j., fugini, m., vitali, m., pernici, b., & salomie, i. (2012). applying green metrics to optimize the energy-consumption footprint of it service centres. international journal of space-based and situated computing 5, 2(3), 158-174. kittlaus, h. b., & krahl, d. (1998). the siz banking data model. in handbook on architectures of information systems (pp. 723-743). springer berlin heidelberg. m. detyniecki, (2001). “fundamentals on aggregation operators”, berkeley initiative in soft computing computer science division university of california, berkeley united sates of america. manju, p., minu, g. m., jerusha, g. d., & banumathi, j. (2014). li-fi technology light source as a hub of data transmission and projection display. i-manager's journal on embedded systems, 3(4), 35. mansell, r., & wehn, u. (1998). knowledge societies: information technology for sustainable development. united nations publications. mines, c. (2008). the dawn of green it services. cambridge, ma: forrester research. mithas, s., khuntia, j., & roy, p. k. (2010, december). green information technology, energy efficiency, and profits: evidence from an emerging economy. in proceedings of the international conference on information systems (p. 11). molla, a., & abareshi, a. (2012). organizational green motivations for information technology: empirical study. journal of computer information systems, 52(3), 92-102. molla, a., cooper, v. a., & pittayachawan, s. (2009). it and eco-sustainability: developing and validating a green it readiness model. in proceedings of the international conference on information systems, 141. molla, a., cooper, v., & pittayachawan, s. (2011). the green it readiness (g-readiness) of organizations: an exploratory analysis of a construct and instrument. communications of the association for information systems, 29(1), 67-96. moyer, j. d., & hughes, b. b. (2012). icts: do they contribute to increased carbon emissions. technological forecasting and social change, 79(5), 919-931. mysore, alka, (2012). taxonomy of sustainable it values. graduate theses and dissertations. paper 12588. noorbakhsh a, shokohyar s and aalirezaei a, (2015). enhancing economical, environmental and social objectives through leasing services. research journal of applied sciences, engineering and technology, 11(10): 1124-1138. norton, m. j. (1996). local area networks in brief. introducing and managing academic library automation projects, 894(2986), 11. olson, e. g. (2008). creating an enterprise-level “green” strategy. journal of business strategy, 29(2), 22-30. australasian journal of information systems shokouhyar, noorbakhsh & aalirezaei 2017, vol 21, selected papers from ausdm evaluation of green it services 19 olzak, t., sabovik, j., boomer, j., & keefer, r. m. (2010). microsoft virtualization: master microsoft server, desktop, application, and presentation virtualization. syngress. overby, s. (2007) clean, green machines, cio, 20, 12, 36-44 ozturk, a., umit, k., medeni, i. t., ucuncu, b., caylan, m., akba, f., & medeni, t. d. (2011). green ict (information and communication technologies): a review of academic and practitioner perspectives. international journal of ebusiness and egovernment studies, 3(1), 1-16. peng, f. (2013). integration of tv white space and femtocell networks (doctoral dissertation, queen mary university of london). porter, m., & van der linde, c. (1996). green and competitive: ending the stalemate. business and the environment, 61-77. putri, n. k. s., & hudirarto, a. (2015). e-waste handling in dki jakarta private higher education institution. journal of theoretical and applied information technology, 74(2). r. r. yager, aggregation operators and fuzzy systems modeling, fuzzy sets and systems, 67 (1994), pp. 129-145. ruth, s. (2009). green it more than a three percent solution. ieee internet computing, 13(4), 74-78. saaty, t. l. (1980). the analytic hierarchy process. mcgraw-hill inc. sharma, s., & sharma, g. (2016). a review on secure and energy efficient approaches for green computing. international journal of computer applications, 138(11). siegler, k., & gaughan, b. (2008). a practical approach to green it. webinar. http://www.itmanagement.com/land/green-it-webinar/?tfso=2058 retrieved august 2016. simmonds, d., & bhattacherjee, a. (2012). environmental sustainability in organizations: the information technology role. amcis 2012 proceedings. 11. sinha, m. (2011). green information technology: a strategy to become socially responsible software organization. international journal of enterprise computing and business systems, 1(2). shokouhyar, s., & aalirezaei, a. (2017). designing a sustainable recovery network for waste from electrical and electronic equipment using a genetic algorithm. international journal of environment and sustainable development, 16(1), 60-79. stojanov, zeljko, vladimir brtka, and dalibor dobrilovic. "evaluation of the software maintenance tasks based on fuzzy screening." ieee 11th international symposium on intelligent systems and informatics (sisy), ieee, 2013. thomas, m., costa, d., & oliveira, t. (2015). assessing the role of it-enabled process virtualization on green it adoption. information systems frontiers, 1-18. tomlinson, b. (2010). greening through it: information technology for environmental sustainability (pp. 1-28). cambridge, ma: mit press. uddin, m., & rahman, a. a. (2012). energy efficiency and low carbon enabler green it framework for data centers considering green metrics. renewable and sustainable energy reviews, 16(6), 4078-4094. uddin, m., hindu, r. c., alsaqour, r., shah, a., abubakar, a., & saba, t. (2015). knowledge management framework using green it to implement sustainable entrepreneur ecosystem. applied mathematics & information sciences, 9(5), 2703. australasian journal of information systems shokouhyar, noorbakhsh & aalirezaei 2017, vol 21, selected papers from ausdm evaluation of green it services 20 uddin, m., talha, m., rahman, a. a., shah, a., ahmed, j., & memon, j. (2012). green information technology (it) framework for energy efficient data centers using virtualization. international journal of physical sciences, 7(13), 2052-2065. wang, x., brooks, s., & sarker, s. (2015). understanding green is initiatives: a multitheoretical framework. communications of the association for information systems, 37(1), 32. wolf, f., & mujtaba, b. g. (2012). sustainability in service operations. implementation and integration of information systems in the service sector, 257. yager, r. r. (1994). aggregation operators and fuzzy systems modeling. fuzzy sets and systems, 67(2), 129-145. zelenika, i., & pearce, j. m. (2014). innovation through collaboration: scaling up solutions for sustainable development. environment, development and sustainability, 16(6), 12991316. zhang, x., chi, t., chen, h., & zhao, h. (2003, july). research on electronic government oriented geographic information service system. in proceedings of the ieee international geoscience and remote sensing symposium, (vol. 6, pp. 3796-3798). ieee copyright: © 2017 shokouhyar, noorbakhsh & aalirezaei. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems shokouhyar, noorbakhsh & aalirezaei 2017, vol 21, selected papers from ausdm evaluation of green it services 21 appendix pair-wise comparison of each green it criteria and sub-criteria goal a b c geometric mean a (1,1,1) (1.413,2.056,2.631) (2.371,2.876,3.38) (1.496,1.808,2.072) b (0.38,0.486,0.708) (1,1,1) (1.76,2.268,2.773) (0.875,1.033,1.252) c (0.296,0.348,0.422) (0.361,0.441,0.568) (1,1,1) (0.474,0.535,0.621) total (2.845,3.377,3.945) table 12. pairwise comparison of green it main criteria b b.1 b.2 b.3 b.4 geometric mean b.1 (1,1,1) (1.11,1.644,2.159) (1.413,1.933,2.446) (1.176,1.719,2.239) (1.165,1.529,1.854) b.2 (0.463,0.608,0.901) (1,1,1) (2.322,2.642,2.952) (2.322,2.642,2.952) (1.257,1.435,1.674) b.3 (0.409,0.517,0.708) (0.339,0.379,0.431) (1,1,1) (1.275,1.644,1.991) (0.648,0.753,0.883) b.4 (0.447,0.582,0.85) (0.339,0.379,0.431) (0.502,0.608,0.784) (1,1,1) (0.525,0.605,0.732) total (3.596,4.322,5.142) table 13. pairwise comparison of sub – criteria for recycling c c.1 c.2 c.3 c.4 geometric mean c.1 (1,1,1) (1.431,1.815,2.169) (1.351,1.719,2.064) (1.496,2.041,2.569) (1.304,1.589,1.842) c.2 (0.461,0.551,0.699) (1,1,1) (0.944,1.351,1.719) (1.431,1.949,2.46) (0.888,1.098,1.311) c.3 (0.484,0.582,0.74) (0.582,0.74,1.059) (1,1,1) (2.789,3.291,3.792) (0.942,1.091,1.313) c.4 (0.389,0.49,0.668) (0.407,0.513,0.699) (0.264,0.304,0.359) (1,1,1) (0.452,0.526,0.64) total (3.586,4.303,5.106) table 14. pairwise comparison of sub – criteria for reuse australasian journal of information systems shokohyar, noorbakhsh & aalirezaei 2017, vol 21, selected papers from acdm evaluation of green it services 22 a a.1 a.2 a.3 a.4 a.5 a.6 a.7 a.8 a.9 a.10 a.11 a.12 geometric mean a.1 (1,1,1) (1.351,1.864,2.3 71) (1.496,2.021,2. 537) (1.465,1.838,2.1 87) (0.66,1.084,1.4 65) (0.944,1.465,1. 974) (1.176,1.552,1.9 04) (1.176,1.552,1.9 04) (1.351,1.922,2.4 6) (1.303,1.864,2. 395) (1.176,1.683,2. 187) (0.69,1.246,1.7 77) (1.114,1.554,1.95 7) a.2 (0.422,0. 536,0.74) (1,1,1) (1.552,2.15,2.7 02) (0.758,1.303,1. 825) (0.66,1.176,1.6 83) (0.91,1.496,2.0 41) (1.465,2.056,2. 605) (0.944,1.351,1. 719) (2.239,2.778,3. 301) (2.141,2.678,3. 201) (2.096,2.616,3. 129) (0.5,1,1.5) (1.063,1.512,1.9 37) a.3 (0.394,0. 495,0.66 8) (0.37,0.465,0.6 44) (1,1,1) (1.149,1.532,1.8 88) (0.715,1.246,1. 76) (0.871,1.443,1. 979) (1.024,1.552,2. 064) (1.024,1.552,2. 064) (0.66,1.176,1.6 83) (0.758,1.303,1. 825) (1.683,2.208,2. 724) (0.5,1,1.5) (0.774,1.147,1.51 9) a.4 (0.457,0. 544,0.68 3) (0.548,0.768,1. 32) (0.53,0.653,0.8 71) (1,1,1) (1.516,2.038,2. 551) (0.822,1.38,1.9 08) (1.024,1.431,1.7 97) (1.024,1.431,1.7 97) (1.201,1.76,2.2 9) (0.871,1.413,1. 933) (1.552,2.064,2. 572) (0.5,1,1.5) (0.849,1.19,1.56 2) a.5 (0.683,0. 922,1.516 ) (0.594,0.85,1.5 16) (0.568,0.803,1. 398) (0.392,0.491,0. 66) (1,1,1) (1.246,1.853,2. 408) (1.351,1.736,2.0 91) (1.176,1.431,1.6 57) (0.822,1.351,1. 864) (0.944,1.496,2 .021) (1.024,1.585,2. 114) (0.574,1.084,1. 589) (0.812,1.142,1.5 7) a.6 (0.506,0. 683,1.059 ) (0.49,0.668,1.0 99) (0.505,0.693,1. 149) (0.524,0.725,1. 217) (0.415,0.54,0.8 03) (1,1,1) (0.776,1.32,1.84 ) (1.024,1.431,1.7 97) (0.944,1.351,1.7 19) (1,1.413,1.783) (1.024,1.431,1. 797) (0.5,1,1.5) (0.684,0.963,1. 347) a.7 (0.525,0. 644,0.85) (0.384,0.486,0. 683) (0.484,0.644,0 .977) (0.556,0.699,0. 977) (0.478,0.576,0. 74) (0.543,0.758,1. 289) (1,1,1) (0.871,1.176,1.4 31) (1.351,1.79,2.16 9) (1.246,1.673,2. 048) (0.758,1.303,1. 825) (0.5,1,1.5) (0.668,0.9,1.20 3) a.8 (0.525,0. 644,0.85) (0.582,0.74,1.0 59) (0.484,0.644,0 .977) (0.556,0.699,0. 977) (0.603,0.699,0 .85) (0.556,0.699,0 .977) (0.699,0.85,1.1 49) (1,1,1) (1.552,1.941,2.2 97) (1.303,1.736,2. 112) (0.758,1.303,1. 825) (0.715,1.285,1. 825) (0.725,0.944,1.2 4) a.9 (0.407,0. 52,0.74) (0.303,0.36,0.4 47) (0.594,0.85,1.5 16) (0.437,0.568,0. 833) (0.536,0.74,1.2 17) (0.582,0.74,1.0 59) (0.461,0.559,0. 74) (0.435,0.515,0. 644) (1,1,1) (1.084,1.465,1. 821) (0.776,1.32,1.8 4) (0.574,1.084,1. 589) (0.561,0.746,1.0 26) a.10 (0.418,0. 536,0.768 ) (0.312,0.373,0. 467) (0.548,0.768,1. 32) (0.517,0.708,1.1 49) (0.495,0.668,1. 059) (0.561,0.708,1) (0.488,0.598,0. 803) (0.474,0.576,0. 768) (0.549,0.683,0 .922) (1,1,1) (0.891,1.431,1. 949) (0.574,1.084,1. 589) (0.544,0.718,1.0 01) a.11 (0.457,0. 594,0.85) (0.32,0.382,0.4 77) (0.367,0.453,0. 594) (0.389,0.484,0 .644) (0.473,0.631,0. 977) (0.556,0.699,0 .977) (0.548,0.768,1. 32) (0.548,0.768,1. 32) (0.543,0.758,1. 289) (0.513,0.699,1. 122) (1,1,1) (0.69,1.246,1.7 77) (0.511,0.672,0.9 66) a.12 (0.563,0. 803,1.45) (0.667,1,2) (0.667,1,2) (0.667,1,2) (0.629,0.922,1. 741) (0.667,1,2) (0.667,1,2) (0.548,0.778,1. 398) (0.629,0.922,1. 741) (0.629,0.922,1 .741) (0.563,0.803,1 .45) (1,1,1) (0.65,0.925,1.67 7) total (8.953,12.415,17 .006) table 15. pairwise comparison of sub – criteria for reduction microsoft word 1603-5454-1-le.docx australasian journal of information systems senarathna, wilkin, warren, yeoh & salzman 2018, vol 22, research article factors that influence adoption of cloud computing 1 factors that influence adoption of cloud computing: an empirical study of australian smes ishan senarathna deakin university ishan.senarathna@deakin.edu.au carla wilkin monash university matthew warren deakin university william yeoh deakin university scott salzman deakin university abstract cloud computing is a recent computing paradigm enabling organizations to have access to sophisticated computing services via the internet on a fee-for-service basis. it provides small and medium-sized enterprises (smes) with opportunities to become as technologically advanced as their larger counterparts, without significant financial outlays. this paper examined the important factors that influence smes’ adoption of cloud computing technology. drawing upon aspects of the technology, organization and environment framework and diffusion of innovation theory, we developed a research model of smes’ adoption of cloud computing and tested it through an online survey of 149 australian smes. data was analyzed using multiple regression methods, with results showing that smes were influenced by factors related to advantaging their organizational capability (i.e., relative advantage, quality of service and awareness) rather than risk-related factors (i.e., security, privacy and flexibility). the findings offer insights to smes owners, cloud service providers and government in establishing cloud computing adoption strategies for smes. keywords: cloud computing, adoption, small and medium-sized enterprises (smes) 1 introduction by delivering resources and services through a user-pay system via the internet, cloud computing offers a new business solution that enables customers to rent information technology (it) infrastructure, platforms and software through positioning their business applications and data storage in the cloud. in defining its scope and role, the national institute of standards and technology’s definition is widely accepted – “a model for enabling convenient, on demand network access to a shared pool of configurable computing resources (e.g., network, servers, storage, applications and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction” (mell & grance, 2011, 3). well-known cloud examples include amazon, google and microsoft (department of finance and deregulation [dfd], 2013). unsurprisingly opportunities and savings have attracted strong market growth in such services. “the cloud market ... [is] valued [at] $148 billion in 2016” (waterford technologies, 2017), with businesses expected to spend us$191 billion on cloud services by australasian journal of information systems senarathna, wilkin, warren, yeoh & salzman 2018, vol 22, research article factors that influence adoption of cloud computing 2 2020, compared to us$72 billion in 2014 (international trade administration, 2015). the australian market shows similar growth, with recent figures from the australian bureau of statistics [abs] showing that for 2015-2016, 31% of australian businesses have adopted cloud computing, up from 20% in 2013-2014 (coyne, 2017). significantly, abs’ figures show that the adoption rate varies by organizational size, with adoption of paid-for-cloud products in 20152016 at “25 percent for businesses with 0-4 employees, and 60 percent for companies with a 200+ headcount” (coyne, 2017). this is reasonably consistent with results from a microsoft survey, which show “40% of australian businesses are already using hybrid cloud technology which they predict to increase to 49% in the next 12-18 months. of these, 43% are only using a private cloud with 17% using purely public cloud solutions” (microsoft, 2016). yet studies into the factors that influence the adoption of cloud computing are limited (yang & tate, 2012) and have focused on larger organizations (venters & whitley, 2012; office of the chief economist, 2016). further relevance is evident from smes’ importance to national economies. globally smes comprise more than 95% of businesses, accounting for approximately 60% of private sector employment (ayyagari et al., 2011). in australia they comprise 95% of active businesses; employ 70% of the nation's workforce, and contribute over 57% of gdp (australian bureau of statistics [abs], 2015). yet australian smes’ adoption of cloud computing (44%) lags that of large organizations (86%) (australian communications and media authority [acma], 2014). in fact, “firms with four or fewer employees were the lowest users of cloud computing, compared to those with 200 or more employees” (office of the chief economist 2016, 94). this slower uptake of cloud computing warrants investigation, as cloud’s special characteristics are particularly advantageous for smes (alshamaila et al., 2013). firstly, cloud computing allows them to technologically compete with larger businesses without significant infrastructure investment (mudge, 2010). it eliminates up-front investment in hardware and software, replacing it with pay-by-use resources. consequently options, such as costly enterprise resource planning systems that permit business capabilities and accessibility from multiple locations, become affordable. secondly, businesses tend to estimate resource needs based upon peak demand (armbrust et al., 2010), whereas need for computing services varies over time. cloud computing offers advantages as businesses are charged for the actual use of such resources (marston et al., 2011) i.e., elasticity. thirdly, off-site data storage helps disaster protection (dfd, 2013). reports are unclear concerning the causes of smes’ slower uptake. one australian government survey (acma, 2014) shows that cloud computing’s principal benefits include convenient access to services (36%), including from multiple locations (15%), and disaster protection (11%). these smes attribute lagged uptake (acma, 2014) to its unsuitability for their business (48%), and their lack of knowledge/awareness (22%). alternatively research has identified issues such as security, legal and privacy concerns (mahmood, 2011; imo, 2013; tehrani & shirazi, 2014). acknowledging cloud computing’s global growth, its recognized benefits to smes, and its slow adoption rate in countries such as australia, this research investigates factors that influence smes’ decision-making concerning adoption. a theoretical model was derived by integrating the technology, organization and environment (toe) framework with the diffusion of innovation (doi) theory, and extending it to include specific factors that represent cloud computing’s unique characteristics. this research studied the applicability of this model australasian journal of information systems senarathna, wilkin, warren, yeoh & salzman 2018, vol 22, research article factors that influence adoption of cloud computing 3 to the adoption of cloud computing among micro, small and medium-sized organizations separately, which has received less attention in previous research. this provided the basis for design of a survey instrument to elicit insights from australian smes. findings from this study contributes new knowledge about the factors that affect cloud computing adoption. firstly, our research model (validated by a survey) addresses calls for empirical investigation of this topic (hsu et al., 2014) and development of a model that is relevant to smes (carcary et al., 2014), which can be referenced in future research. further, our findings show that australian smes regard relative advantage, quality of service and awareness of cloud computing as being influential in their adoption processes. prior studies identified relative advantage (i.e., oliviera et al., 2014; gangwar et al., 2015) and top management support (i.e., abubakar et al., 2014; rowe et al., 2012; gangwar et al., 2015), which may well be linked to cloud awareness/knowledge, as significant influences. when taken together our three identified factors, which explain 83.3% of the variance in our model, suggests smes were influenced by cloud computing’s value to build organizational and it capability. this contrasts with hsu et al. (2014) who found that perceived benefits and it capability were not significant. in contrast to prior research (i.e., gupta et al., 2013) and industry reports (idc, 2012; acma, 2014), our findings also show that risk-related factors such as security, privacy and flexibility were of lesser significance, although respondents did indicate some concerns about these issues and about lack of resources. additionally, we show that organizational size was statistically significant even within the sme construct (i.e., micro/small vs medium enterprises). this refines prior research that has consistently identified differences between large and smaller organizations’ adoption of cloud computing (i.e., brender & markov, 2013; low et al., 2011; guiterrez et al., 2015). alternatively, in contrast to prior research (i.e., alshamaila et al., 2013; oliviera et al., 2014), industry type was not found to be influential. the remainder of the paper is organized as follows. section 2 presents the theoretical foundation including an overview of prior research and application of the toe framework and doi theory to the study. the research model and hypotheses are presented in section 3, followed by a description of the methodology (section 4). results and findings are discussed in section 5. finally, section 6 outlines implication for research and practice, followed by conclusion and limitations of the research in section 7. 2 theoretical foundation 2.1 organizational size and cloud adoption it is important to acknowledge that research related to large organizations does not necessarily translate to smes (carcary et al., 2014). in comparison to larger organizations, smes face resource constraints that limit their competitiveness and capabilities in areas such as innovation (rosenbusch et al., 2011), knowledge management (durst & edvardsson, 2012), and it/is competency (cragg et al., 2011). two resource constraints may be particularly important. firstly, a lack of personnel with specialist it knowledge may cause reduced absorptive capacity (zahra & george, 2002), as there is less internal capacity to acquire and apply relevant knowledge (madrid-guijarro et al., 2009). secondly, reduced financial and managerial resources (georgiadis & pitelis, 2012) may limit smes in articulating the value of cloud computing. as such, one study related to cloud computing showed that larger australasian journal of information systems senarathna, wilkin, warren, yeoh & salzman 2018, vol 22, research article factors that influence adoption of cloud computing 4 businesses were better prepared for cloud adoption than smaller ones (brender & markov, 2013). despite such differences and the considerable benefits to smes, little research has directly investigated factors influencing smes’ adoption of cloud computing. in fact, yang and tate (2012), in their review of 221 relevant journal articles up to 2011, found only six articles related to cloud computing adoption and only one, a single case study, to smaller enterprises. similarly, gupta et al. (2013, 861) noted that “many studies were (and currently are being) conducted on the use of cloud computing by large scale enterprises primarily on their perceptions”. therefore, to position this research, we scanned peer-reviewed journal articles published after 2012 concerned with adoption of cloud computing. after eliminating literature reviews and opinionative pieces, twelve studies were identified (see table 1). these showed a range of influential factors, with the most consistent being top management support, organizational readiness, complexity of the innovation when linked with relative advantage, and compatibility with existing it, risk and security. of these, four studies involved both larger and smaller organizations with insufficient data to extrapolate smes’ differences. of the remaining eight, one indicated findings were limited as smes’ cloud computing adoption was embryonic (hsu et al., 2014); one studied it professionals (werfs et al., 2013); and another investigated use and adoption at the same time (ross & blumenstein, 2015). consequently, the need identified by carcary et al. (2014, 325) is yet to be addressed, namely “an sme-specific model and associated guidance to support cloud adoption would be of considerable benefit to the sme market in managing the adoption process and deriving resultant benefits.” we address this need through our review (see table 1), our related theoretical model and our related survey that investigates what influences smes’ adoption of cloud computing. primary focus authors context for adoption of cloud computing theory method findings smes gupta et al. (2013) smes’ perceptions literature review survey factors: ease of use; privacy; convenience; reduced cost; improved security. lesser: reliability; sharing; collaboration. oliviera et al. (2014) portuguese businesses [micro & small (36%), medium (46%), large (18%) enterprises] toe + doi survey factors: relative advantage; top management support; complexity; technology readiness; organizational size. carcary et al. (2014) irish smes literature review survey based informal management styles. factors: cost reduction, resource utilization, collaboration, complexity. abubakar et al. (2014) sub-saharan african smes grounded theory 10 case studies factors: top management australasian journal of information systems senarathna, wilkin, warren, yeoh & salzman 2018, vol 22, research article factors that influence adoption of cloud computing 5 support; computing resources; solving power supplies. lesser: security; privacy. triguerospreciado et al. (2013) barriers to smes’ empirical group meetings with managers + survey main barrier: culture. ross & blumenstein (2015) sme entrepreneurship schumpeterian creative destruction scenario interviews factors: better access to global markets; reducing opportunity costs; facilitating collaboration and innovation. rowe et al. (2012) vietnamese smes – e-commence toe survey factors: employees’ knowledge of ecommerce; enterprise size + innovation compatibility; managers’ attitude; competition; trading partners, industry/government support. inhibitors: complexity; risk. it professionals werfs et al. (2013) sme software vendors adaptive sts perspective longitudinal study using interviews factors: strategy; complexity; culture. large & small organizations studied hsu et al. (2014) taiwanese firms toe + doi tested on 200 smaller and larger firms – 65% < 200 employees, but 60% with > 3 employees in the it dept. i.e., atypical of smes factors: benefits; choosing private cloud providers; greater it capability. lesser: external pressure. garrison et al. (2015) mainly larger organizations resource-based theory on-site interviews, online participation, and telephone interviews factors: relational, managerial and technical it capabilities. gangwar et al. (2015) small and larger organizations’ tam1 + toe survey factors: relative advantage; compatibility; complexity; 1 technology acceptance model australasian journal of information systems senarathna, wilkin, warren, yeoh & salzman 2018, vol 22, research article factors that influence adoption of cloud computing 6 organizational readiness; top management support. gutierrez et al. (2015) managers (smaller and large organizations) toe survey factors: competitive pressure; complexity; technology readiness; trading partners’ pressure. this study australian smes toe + doi survey factors: cloud relative advantage; service quality; awareness of cloud computing. lesser: security; privacy; and flexibility. table 1: summary of the literature post 2012. in summary, despite cloud computing being “a fundamental change in the way it services are invented, developed, deployed, scaled, updated, maintained and paid for” (marston et al., 2011, 176), there are limited investigations into what influences smes’ decision-making regarding adoption of this technology. we aim to address this research gap through deriving a model and testing it by surveying australian smes. 2.2 the technology, environment and organization framework and diffusion of innovation theory both toe and doi are relevant to this study, and their use in prior research motivates comparative appraisal of their relevance (see table 1). doi theorizes two influential factors (innovation and organizational characteristics) as an innovation progresses from instantiation to use. an innovation’s attributes are influential on an individual’s decision-making (rogers, 2003) (i.e., relative advantage, compatibility, complexity, trialability and observability). other relevant variables include: the decision type (collective, authority etc.); communication strategies; organizational social systems (norms, interconnectedness); and effectiveness of change agents (rogers, 2003). when applied at an organizational level, where innovation decisions are usually less individualistic, doi construes organizational innovativeness as depending upon the leadership characteristics, as well as organizations’ internal structure and external characteristics (rogers, 1995). despite dois’ wide acceptance, concerns have been raised about its ability to take into account the particularities of complex is. issues relate to the interactions that occur, their social construction and related intensive learning (lyytinen & damsgaard, 2001); as well as accommodating environmental contexts (hsu et al., 2014). toe usefully extends doi. unlike doi’s focus on the individual, toe focuses on organizations’ adoption decisions. toe’s two categories (technology and organization) have obvious synergies with doi’s innovation and organization. toe’s technological context concerns perceived benefits, barriers and importance of compliance, interoperability; and interconnectivity (i.e., oliveira et al., 2014). organizational factors include: size; complexity; perceptions of existing it; cost; and management impacts (i.e., gutierrez et al., 2015). environmental factors include: market uncertainty; competitors; and regulatory pressure (i.e., australasian journal of information systems senarathna, wilkin, warren, yeoh & salzman 2018, vol 22, research article factors that influence adoption of cloud computing 7 low et al., 2011). thus, the two theories meaningfully complement each other (oliveira et al., 2014). we draw upon both doi and toe to derive our model and survey instrument (see appendix a) for studying smes’ decision-making regarding adoption of cloud computing. through linking this theoretical positioning to our literature review (i.e., table 1, section 3 and appendix a), we identify key attributes of cloud computing adoption in terms of technological, organizational and environmental factors. as discussed section 3, we construe the key attributes of our theorized model and instrument as: • technological: doi (relative advantage), toe (security, privacy and flexibility); • organizational: doi and toe (leadership awareness, flexibility, slack and interconnectedness). note: size is controlled (i.e., smes); • environmental: toe (competitors, market, regulations and service). 3 research model and hypotheses 3.1 cloud relative advantage relative advantage refers to the degree to which potential adopters perceive that innovations provide benefits and superiority to any predecessors (rogers, 2003). it has been widely investigated in previous studies concerned with technology adoption (salleh et al., 2012; gibbs & kraemer, 2004; ramdani & kawalek, 2007), with findings showing it as an important attribute (gangwar et al., 2015; oliviera et al., 2014). in this study cloud relative advantage (cra) is defined in terms of wider market coverage (including better communication with customers and suppliers) and lower business costs from related flexible cost structures and scalability (sultan, 2011; bharadwaj & lal, 2012). cra relates to smes reducing operating costs by avoiding capital expenditure in hardware, software and it support (george & shyam, 2010, marston et al., 2011) by outsourcing infrastructure, platforms and services (dfd, 2013). through reducing duplication and costs, buying bandwidth and paying according to their usage, organizations leverage economies of scale and increase savings through virtualization (sharif, 2010; dfd, 2013). hence, we propose: h1: the relative advantages of cloud computing positively influence smes’ decision to adopt cloud computing. 3.2 cloud flexibility it flexibility is defined as the fast deployment of technology components, facilitated through business/it infrastructure (garrison et al., 2015). it is particularly relevant to adoption of new it (zhao et al., 2014). flexibility involves the ability of an organization to rapidly and easily adapt to up-to-date services to support business processes (mell & grance, 2011; mccabe & hancock, 2009). for smes without it resources, such as dedicated staff, cloud-based hardware and software provides available, affordable and reliable alternatives to installing and maintaining on-site infrastructure (fairchild, 2014). as such, cloud-based flexibility (cf) has been found to foster more responsive, adaptive and competitive businesses (mccabe & hancock, 2009; mudge, 2010). indeed, strategic flexibility gained through cloud computing is shown to effect major operational improvements (armbrust et al., 2010; whitten et al., 2010). therefore, the following hypothesis is proposed: australasian journal of information systems senarathna, wilkin, warren, yeoh & salzman 2018, vol 22, research article factors that influence adoption of cloud computing 8 h2: the flexibility of cloud computing positively influences smes’ decision to adopt cloud computing. 3.3 quality of service quality of service (qos) is defined as the capability to meet specific requirements using specific metrics to analyze the level of quality i.e., response time and throughput (carroll et al., 2014). as an inherent feature of many clouds, qos is often delivered via service level agreements (slas) between the sme and the cloud service provider (dfd, 2013). for example, amazon has sought to provide assured qos through basic slas such as 99.9% infrastructure uptime (luis et al., 2009). high levels of qos are typically provided at competitive costs by dynamic resource scaling of it management and administration with commitment to service efficiency (busch et al., 2014). in providing qos, availability and reliability may be bundled with ongoing service updates (information industry innovation council [itiic], 2011; lippert & govindarajulu, 2006), which contributes to growing demand for cloud computing, including access to cloud services from various client devices (itiic, 2011). however, smes need to balance the trade-offs between costs and qos. therefore, the following hypothesis is proposed: h3: quality of service of cloud computing positively influences smes’ decision to adopt cloud computing. 3.4 cloud security current research has identified cloud security (cs) as an important issue, with specific reference to smes (i.e., abubakar et al., 2014; gupta et al., 2013). specifically, cloud computing’s architecture may generate new security issues such as data leakage, virtualization, vulnerability and hypervisor vulnerability (gonzalez et al., 2012). cs is complicated by the multi-tenancy of the virtualized resources (dfd, 2013), with data owners not necessarily knowing the location or reliability of the data hosts (sarwar & khan, 2013). other major issues relate to: transmission, and availability (mahmood, 2011); malicious insiders, outside attacks, and service disruptions (behl, 2011); data protection, disaster recovery, and business continuity (dfd, 2013). smes commonly cite cs concerns about use of cloud computing (sultan, 2011). therefore, the following hypothesis is proposed: h4: security in the cloud computing environment positively influences smes’ decision to adopt cloud computing. 3.5 cloud privacy given cloud’s technologically-driven interactions, cloud privacy (cp) and related risks are long-standing issues (gangwar et al., 2015; gupta et al., 2013). with information stored in the cloud, users may be concerned about it being accessed by others anywhere in the world (vanessa, 2014). moreover, cloud’s distributed physical location creates legal issues due to difficulties in determining jurisdiction. the australian government’s concern regarding the location of outsourced personal data storage is reflected in the preference for cloud services to be located within australia’s borders (hutley, 2012), meaning that data is protected by australian privacy laws. recommendations to manage cloud-related privacy issues include: minimizing the personal information stored, as well as specifying and limiting the purposes for its data usage (pearson, 2009). for smes, issues include: poor user control; trustworthiness; and lack of transparency. even with appropriate policies, risks remain apparent (ko et al., 2011). thus, cp remains a relevant issue. therefore, the following hypothesis is proposed: australasian journal of information systems senarathna, wilkin, warren, yeoh & salzman 2018, vol 22, research article factors that influence adoption of cloud computing 9 h5: privacy in the cloud computing environment positively influences smes’ decision to adopt cloud computing. 3.6 awareness of cloud knowledge and awareness of the cloud (awc) have been identified as significant constraints upon adoption for initial cloud adoption (vanessa, 2014) and its sustainable use (prasad et al., 2014). knowledge about cloud computing has been shown as a primary influence in smes’ decision to adopt cloud computing (tehrani & shirazi, 2014). equally lack of awareness about the benefits of options such as saas2 has demonstrably limited adoption (carcary et al., 2014). given limited awareness is reportedly an issue in the cloud market, especially within australian smes (hutley, 2012), the following hypothesis is proposed: h6: awareness of cloud computing positively influences smes’ decision to adopt cloud computing. two control variables were identified. firstly, the industry sector within which organizations operate has been shown to influence it adoption (alshamaila et al., 2013; oliviera et al., 2014; forman, 2005). different business needs in different industry sectors may affect innovation uptakes (levenburg et al., 2006; prasad et al., 2014; ifinedo, 2011). secondly, larger businesses arguably possess more resources, skills, experience and hence ability to adopt new innovation than smaller businesses. equally small organizations can be more innovative as their simpler organizational structures arguably permit greater business agility (jambekar & pelc, 2002). therefore industry sector and organization size (micro, small and medium) are considered control variables. based on this analysis we hypothesized a theoretical model (see figure 1) with six latent variables (cra, cf, qos, cs, cp and awc) as influences upon smes’ adoption of cloud computing. cloud adoption is the dependent variable. figure 1. conceptual model of the factors that affect smes’ adoption of cloud computing. 2 software as a service (saas) is a cloud model in which vendors host applications that customers access over a network i.e. the internet. australasian journal of information systems senarathna, wilkin, warren, yeoh & salzman 2018, vol 22, research article factors that influence adoption of cloud computing 10 4 methodology 4.1 context: small and medium-sized enterprises (smes) given no universal definition of smes, as this study was conducted in australia, we adopted the abs’ definition i.e., smes employ less than 200 people (specifically micro <4 people, small 5-19, and medium 20-199 people). despite their importance in national economies, their lagged uptake of internet innovation has been established (see section 1). for example, despite the australian market for cloud computing growing, and 94% of australian smes having internet access, only 44% use cloud computing compared to 86% of large organizations’. 4.2 survey development based on the identified factors (see section 3), a survey was developed with items adapted from previous research and/or industry reports (see appendix a). the instrument comprised three parts covering: (1) company profile; (2) actual use of cloud computing; and (3) 28 items related to the 6 factors and adoption (cra – 6 items; cf – 4 items; qos – 4 items; cs – 3 items; cp – 3 items; awc – 4 items; and adopt – 4 items). a 7-point likert scale (1 “strongly disagree” to 7 “strongly agree”) was used to capture perceptual data. pre-testing honed survey understandability, wherein items were reviewed and revised several times by the research team, then pre-tested on 10 smes from different industries. refinements at each stage were based on the feedback received. the survey instrument (see appendix b) was designed to capture respondents’ use of cloud computing. for example they were asked: (1) whether they understood the differentiation between saas, paas and iaas; (2) about their familiarity with private, public and hybrid cloud; and (3) to indicate what applications were used in their organization (i.e., dropbox, google docs, icloud, google drive, sky drive, office365, jolidrive, other). 4.3 survey application and testing an australian marketing research company was used to identify/invite a stratified random sample of 1,179 respondents (smes) to complete the online survey. it is not recognized that there is no method to evaluate the expert authority of the marketing research company. responses were collected in qualtrics and downloaded into ibm spss v22 for analysis. several analytical methods were used. for multivariate analysis, data were screened to check for missing data, outliers and normality (pallant, 2011). then descriptive summary analysis was performed using cronbach’s alpha to assess the reliability of the items (α >0.70 was used as the cut-off point). to verify each factor’s validity, confirmatory factor analysis, using principal axis factoring extraction, was used on all 28 survey items. to test hypotheses 1-6, results of the factor analysis were used as input values for the multiple regression analysis (mlr). the direct relationship between the six independent variables (cra, cf, qos, cp, cs and awc) and the dependent variable (adopt) was examined using pearson’s correlation and standard multiple regression. since all six variables were considered equally important, a simultaneous enter method was chosen when performing mlr (pallant, 2011). to test the effect of control variables (organizational size and type) on the dependent measure, an anova was conducted. after organizations were categorized into two groups (micro and small/medium smes), a one-way anova was used. australasian journal of information systems senarathna, wilkin, warren, yeoh & salzman 2018, vol 22, research article factors that influence adoption of cloud computing 11 5 results and discussion in total 191 responses were received. due to incompleteness, 39 were excluded, leaving 149 usable responses i.e., an effective response rate of 13%, which is regarded as acceptable for statistical purposes (i.e., harrigan et al., 2008), especially as there is consistency with rates achieved by prior research3 (denscombe, 2014). descriptive analysis (see table 2 below) showed that unsurprisingly states with larger populations provided higher response rates. the sampled smes comprised mainly micro enterprises (61.8%), with the remainder classified as small (20.8%) and medium (17.4%). 27.5% of the respondents indicated that they used some form of cloud computing. category no. of responses % no. of employees 0 to 4 (micro) 92 61.8% 5 to 19 (small) 31 20.8% 20 to 199 (medium) 26 17.4% state/territory victoria 29 19.5% new south wales 30 20.1% queensland 41 27.5% western australia 19 12.8% south australia 17 11.4% tasmania 10 6.7% northern territory 3 2.0% use of cloud computing yes 41 27.5% no 108 72.5% table 2: demographic characteristics of the responding organizations (n=149). cronbach’s alpha and factor analysis were used to determine whether there was adequate reliability and validity of the measures. the cronbach’s alpha score for each reflective indicator was >0.7, permitting confidence in accepting internal consistency (hair et al., 2005). all extracted factors had an eigenvalue of 1 (table 3) and each reflective indicator loaded strongly on its associated factor (loadings >0.50). as no reflective indicators cross-loaded (see table 4), discriminant validity was established. this implied that the instrument provided a reasonable measure of the theorized model’s predefined factors (see figure 1). orthogonal varimax rotation was used to produce factor structures that were uncorrelated. the rotated solution revealed unexpected reflective indicator groupings. therefore factor analysis was run several times and indicators that did not load were removed i.e., <0.50 (thompson, 2004) with the exception of cfa4. three items (2 cra items and 1 qos item) were removed due to assumptions associated with cook’s distance (pallant, 2011) (see appendix a). the final model provides a clear solution with 6 extracted factors accounting for 62% of variability in the original data. 3 the response rate is comparable with research reported by garrison et al. (2015), trigueros-preciado et al. (2013) and carcary et al. (2014). australasian journal of information systems senarathna, wilkin, warren, yeoh & salzman 2018, vol 22, research article factors that influence adoption of cloud computing 12 factor initial eigenvalues extraction sums of squared loadings rotation sums of squared loadings total % of variance cumulative % total % of variance cumulative % total awc 5.814 27.688 27.688 5.435 25.882 25.882 3.231 cf 3.089 14.711 42.399 2.838 13.514 39.396 2.435 cra 2.080 9.906 52.305 1.707 8.128 47.524 2.137 qos 1.799 8.568 60.873 1.329 6.330 53.854 1.850 cp 1.353 6.445 67.318 0.951 4.529 58.383 1.685 cs 0.987 4.698 72.016 0.583 2.777 61.160 1.506 table 3: total variance explained. note: extraction method is principal axis factoring. the pattern matrix (see table 4) reveals all reflective indicators (items) load significantly on their respective factors (cra, cf, qos, cs, cp and awc) with the exception of item cf4, although its loading (0.464) is above the 0.4 threshold (hair et al., 2005). further, the reflective indicators were not cross-loaded, supporting discriminant validity. thus, there is reason to accept that the instrument provides a reasonable measure of the theorized factors. items factor cra cf qos cs cp awc cra3 .704 .266 .119 .140 -.018 .062 cra1 .678 .189 .167 .160 -.017 .064 cra2 .617 .183 .100 -.117 -.062 .173 cra4 .588 .255 .153 .045 -.118 .044 cf1 .330 .733 .203 .075 -.081 .135 cf2 .313 .693 .139 .096 -.034 .135 cf3 .244 .564 .252 .081 -.133 -.123 cf4 .255 .464 .249 .064 -.004 .014 qos1 .264 .158 .728 .137 .083 .163 qos2 .142 .136 .740 .044 .053 .148 qos3 .105 .321 .627 .077 -.006 .133 cs3 .050 .078 .130 .722 -.013 .109 cs1 -.104 .112 -.001 .685 -.166 -.005 cs2 .266 .000 .072 .622 .032 -.064 cp2 .092 -.132 -.070 .001 .879 .057 cp3 -.019 .095 .175 -.085 .724 .023 cp1 -.215 -.159 -.008 -.060 .551 -.056 awc4 .112 -.011 .131 -.014 -.018 .966 awc2 .106 -.039 .127 .002 .013 .914 awc1 .121 .062 .160 .007 -.018 .846 awc3 -.012 .131 .028 .056 .040 .752 table 4: pattern matrix of factor analysis for the independent variables. note: extraction method (principal axis factoring); rotation method (varimax with kaiser normalization); rotation converged in 7 iterations. australasian journal of information systems senarathna, wilkin, warren, yeoh & salzman 2018, vol 22, research article factors that influence adoption of cloud computing 13 for the dependent variable, factor analysis yielded one factor (adopt) with an eigenvalue >1 explaining 97.4% of the variance, with four reflective items loading strongly onto this extracted factor. no rotated pattern matrix was imposed as only one factor was extracted. all item loadings were significant i.e., >0.95 and all factor loading scores were saved as variables. next, multiple regression analysis (mra) using the simultaneous enter method, was used to determine if the extracted factors could significantly explain smes’ perceptions related to adopting cloud computing. underlying assumptions of the regression analysis were assessed and met (anova p-value <0.001). parametric testing of our data, which was derived on a likert scale, is justified as testing by norman (2010) showed that (1) “parametric tests not only can be used with ordinal data, such as data from likert scales, but also that parametric tests are generally more robust than nonparametric tests “ and (2) “parametric tests are sufficiently robust to yield largely unbiased answers that are acceptably close to “the truth” when analyzing likert scale responses” (sullivan & artino, 2013, 541). significant factors contributing to these results were cra, awc and qos. most smes emphasized that their organization’s needs could be met by cloud computing and agreed/strongly agreed (mean 5.53 out of 7, sd 1.6458) with the statement that their organization would take steps to adopt cloud computing in the near future. hypotheses 1-6 could now be considered with mlr used to identify the relative contribution of each independent variable (extracted factor). this technique has been widely used in previous it adoption studies (e.g., alam et al., 2011; tan et al., 2010). the results are presented in table 5 below. as a result, hypotheses 1, 3 and 6 were accepted (see table 5 and figure 2). model b beta t sig. tolerance vif (constant) 0.038 1.157 0.249 cra 0.187 0.190 5.635 0.000 0.994 1.006 cf -0.002 -0.002 -0.047 0.962 0.991 1.009 qos 0.193 0.186 5.497 0.000 0.987 1.013 cs -0.018 -0.019 -0.564 0.574 0.999 1.001 cp 0.006 0.006 0.188 0.851 0.998 1.002 awc 0.853 0.873 25.969 0.000 0.998 1.002 table 5: coefficients of mlr note: r2 = 0.840; adj.r2 = 0.833; f (6, 142) = 19.834; p <0.001 in summary results indicate a significant and strong positive relationship between awc and cloud computing adoption (beta weight 0.873, p<0.001), with both qos and cra also indicating a significant positive influence (p <0.001). cs, cp and cf did not significantly contribute to modelled outcomes (p >0.05). anova was conducted to test whether the influence of the independent variables on the dependent variable (adopt) varied based on organizational size or industry type. australasian journal of information systems senarathna, wilkin, warren, yeoh & salzman 2018, vol 22, research article factors that influence adoption of cloud computing 14 sum of squares df mean square f sig. between groups within groups total 6.239 135.432 141.671 2 146 148 3.120 .928 3.363 .037 table 6: anova (organization size and cloud adoption) as can be seen in table 6, results indicate that adoption of cloud computing as a function of organizational size was statistically significantly different (p < 0.05), with small & medium enterprises more likely to adopt cloud computing than micro organizations. one possible explanation for this significant relationship is that small to medium organizations have greater resources to allocate to such initiatives than micro ones (see section 2.1). contrastingly, variances related to smes’ industry types were not different and statistically not significant (p >0.05). given contrary findings from prior research (ininedo, 2011; levenburg et al., 2006), this result is unexpected. however, whilst not statistically significant, results suggest that the following industries are more likely to adopt cloud computing: professional, scientific and technical services; information, media and telecommunication; education and training; health care and social assistance; administrative and support services; manufacturing; electricity, gas and water and waste services; and retail trade. as a result, the research model was revised to reflect the findings (see figure 2 below). the model explains 83.3% of the variance and is consistent with the toe/doi structure that framed the study. analysis shows reason to accept that the research model has statistical significance. figure 2. revised model showing the factors affecting smes’ adoption of cloud computing. 6 contributions and implications for research and practice cloud computing offers flexible and affordable access to it infrastructure, platforms and software. it has generated strong uptake by large organizations (ita, 2015), but less in smes (office of the chief economist, 2016). yet its special characteristics, including pay-by-use access to previously unaffordable resources, accessibility from multiple locations, and protection against disaster risks, are equally significant for smes (marston, 2011; dfd, 2013). as such, lagged uptake by smes (idc, 2012), including those in australia (acma, 2014), is sme’s intention to adopt cloud computing h1: cloud relative advantage (cra) h6: awareness of cloud computing (awc) h3: quality of service r2 = 0.84 key: ***: p < .001 .186*** .190*** result .873*** australasian journal of information systems senarathna, wilkin, warren, yeoh & salzman 2018, vol 22, research article factors that influence adoption of cloud computing 15 unexpected and a significant issue given their strategic contribution to national economies (ayyagari et al., 2011). rather than focusing on perceived barriers to smes' adoption of cloud computing (i.e., unsuitability for their business [idc, 2012] or their lack knowledge [acma, 2014]), this research sought to probe to determine the factors influence smes’ adoption. the contributions from the research are fourfold. firstly, it extends knowledge about adoption of cloud computing to smes where there is identified need for such a study (i.e., table 1; carcary et al., 2014; yang & tate, 2012). whilst prior research provides important knowledge about cloud computing, “empirical studies that rigorously examine the proposed factors that might affect the adoption of cloud computing [are] needed” (hsu et al., 2014, 475). rather than using the technology acceptance model and unified theory of technology acceptance, which focus on individuals and technology use, we drew upon the established use of toe and doi in our organizational context of cloud computing adoption (see table 1), linking them with the factors identified in prior research and practitioner sources (see appendix a). this aligns with hsu et al. (2014, 476) that whilst “the three toe dimensions influence adoption ... these dimensions must be uniquely operationalized in each specific innovation context”. our proposed research model comprised six factors (and their related indicators) with one dependent variable (adopt) and two control variables. by testing the models’ significance through a survey, our revised model (see figure 2) addresses calls to develop a model for smes’ adoption of cloud computing (carcary et al., 2014) that may be referenced in future research. secondly, as the first study to investigate australian smes’ decision-making regarding adoption of cloud computing, findings show that organizational size (micro/small vs medium) is significant even within the sme construct (see table 6), with small & medium enterprises more likely to adopt cloud computing than micro organizations. these findings refine research on this topic, as to date research has been limited to revealing differences between larger and smaller organizations’ adoption of cloud computing (i.e., brender & markov, 2013; low et al., 2011; guiterrez et al., 2015). in contrast to other research (i.e., alshamaila et al., 2013; oliviera et al., 2014), industry type was not found to be influential. thirdly, in the context of cloud computing, our findings reveal that the primary influence upon smes’ decision-making relates to the benefit in building organizational capability. this is reflected in the three primary influences being: relative advantage (increasing profits, reducing costs, creating business opportunities); quality of service (creating a flexible business environment with dynamic scaling and accessibility); and awareness of cloud (related to understanding the opportunities of various options). relative advantage has previously been identified as a significant influence (oliviera et al., 2014; gangwar et al., 2015), as has top management support (abubakar et al., 2014; rowe et al., 2012; gangwar et al., 2015), which may be linked to awareness/knowledge about cloud computing. our finding that the three significant influences are relative advantage, quality of service and awareness of cloud computing, suggests that smes are influenced by cloud computing’s value to build it and organizational capability. this contrasts with hsu et al. (2014) who found that perceived benefits and it capability were non-significant. fourthly, findings show that risk-related factors such as security (sensitive information); privacy (trust and issues with location of hosting); and flexibility (reliability, responsiveness and availability) were of lesser significance. this contrasts with prior research (i.e., gupta et australasian journal of information systems senarathna, wilkin, warren, yeoh & salzman 2018, vol 22, research article factors that influence adoption of cloud computing 16 al., 2013; idc, 2012; acma, 2014). one interesting concern raised by smes was lack of resources, a reflection of smes’ capabilities. finally, there are implications for practice. our findings provide guidelines for decision makers by revealing the factors smes consider important as they evaluate cloud adoption. for example, the importance attributed to relative advantage should encourage decision makers to evaluate potential benefits regarding organizational capability. the weighting afforded to awareness and service quality may encourage a focus on strategically evaluating potential benefits related to resource utilization, increased productivity, and cost/benefit analysis. the findings also help software vendors to devise a strategy to cater for this category, offering components of a system as a module instead of offering a system as a whole (e.g., the components of an erp system). similarly, the australian federal government can reflect upon the cloud computing framework when developing policies for smes to facilitate them to reduce the cost of their it operations and reach globalization. 7 conclusion and limitations this study examined the factors that influence cloud computing adoption by australian smes. our results reveal a significant and strong positive relationship between awareness of cloud computing and cloud computing adoption, with both quality of service and cloud relative advantage also having significant positive influences. cloud security, privacy and flexibility did not significantly contribute to modelled outcomes. findings also indicate that organizational size was significant even within the sme construct (i.e., micro/small vs medium enterprises). there are a number of limitations to the study. for example, the survey was restricted to australian smes, limiting generalizability. moreover the quantitative methodology suggests value in additional qualitative techniques to consolidate our derived understanding. inclusion of factors such as culture, different legal systems and cloud providers’ management styles would also be useful extensions. given the value that larger organizations perceive in cloud computing, as evidenced by their substantial investments, together with the contribution of smes to national economies and smes’ lagged uptake of this technology, it is important to understand why smes’ intentions are being so slowly translated into what is recognized by their larger counterparts, namely the business value that can be generated from adopting cloud computing. references abubakar, a.d., bass, j.m., & allison, i. (2014). cloud computing: adoption issues for subsaharan african smes. electronic journal of information systems in developing countries, 62(1): 1-17. acma. (2014). cloud computing in australia. communication report 2014, australian communications and media authority. alam, ss., ali, m.y., & jani, m. f. (2011). an empirical study of factors affecting electronic commerce adoption among smes in malaysia. journal of business economics & management, 12(2): 375-399. australasian journal of information systems senarathna, wilkin, warren, yeoh & salzman 2018, vol 22, research article factors that influence adoption of cloud computing 17 alshamaila, y, papagiannidis, s & li, f. 2013, cloud computing adoption by smes in the north east of england: a multi-perspective framework, journal of enterprise information management, 26(3): 250-75. armbrust, m., fox, a., griffith, r., joseph, a.d., katz, r., konwinski, a., lee, g., patterson, d., rabkin, a., stoica, i., zaharia, m. (2010). a view of cloud computing. communication of the acm, 53(4): 50-58. australian bureau of statistics (abs) key statistics and analysis. 2012. http://www.treasury.gov.au/~/media/treasury/publications and media/publications/2012/australian small abs. (2015). summary of it use and innovation in australian business.2013-14. australian bureau of statistics, www.abs.gov.au/ausstats/abs@.nsf/mf/8166.0. ayyagari, m., demirgüç-kunt, a., & maksimovic, v. (2011). small vs. young firms across the world: contribution to employment, job creation, and growth. world bank policy research working papers, (5631). behl, a. (2011). emerging security challenges in cloud computing. ieee international conference on information & communication technology. bharadwaj, s.s., & lal, p. (2012). exploring the impact of cloud computing adoption on organizational flexibility: a client perspective. international conference on cloud computing technologies, applications & management (iccctam2012). brender, n., & markov, i. (2013). risk perception and risk management in cloud computing: results from a case study of swiss companies. international journal of information management, 33(5): 726-733. busch, p., smith, s., gill, a., harris, p., fakieh, b., & blount, y. (2014). a study of government cloud adoption: the australian context. proc. australasian conference of information systems. carcary, m., doherty, e., conway, g., & mclaughlin, s. (2014). cloud computing adoption readiness and benefit realization in irish smes—an exploratory study. information systems management, 31(4): 313-327. carroll, n., helfert, m. & lynn, t. (2014). towards the development of a cloud service capability assessment framework. in continued rise of the cloud (289-336): springer. coyne, a. (2017). cloud computing adoption in australia is booming. https://www.itnews.com.au/news/cloud-computing-adoption-in-australia-is-booming468833 cragg p, caldeira m, & ward j. (2011) organizational information systems competences in small and medium-sized enterprises, information & management, 48:353–363. denscombe, m. (2014). the good research guide: for small-scale social research projects. mcgrawhill education (uk). dfd. (2013). cloud computing strategic direction paper. department of finance and deregulation. durst s, & edvardsson i.r. (2012) knowledge management in smes: a literature review, journal of knowledge management, 16(6): 879-903. australasian journal of information systems senarathna, wilkin, warren, yeoh & salzman 2018, vol 22, research article factors that influence adoption of cloud computing 18 fairchild, a.m. (2014). patterns of trust: role of certification for sme cloud adoption. in continued rise of the cloud (145-158): springer. forman, c. (2005). the corporate digital divide: determinants of internet adoption. management science, 51(4): 641-654. gangwar, h., date, h., & ramaswamy, r. (2015). understanding determinants of cloud computing adoption using an integrated tam-toe model. journal of enterprise information management, 28(1): 107-113. garrison, g., wakefield, r.l., & kim, s. (2015). the effects of it capabilities and delivery model on cloud computing success and firm performance for cloud supported processes and operations. international journal of information management, 35(4): 377-393. george, f., & shyam, g. (2010). impact of cloud computing: beyond a technology trend. system integration 262-269. georgiadis a., pitelis c.n. (2012) human resources and sme performance in services: empirical evidence from the uk. international journal of human resource management, 23(4):808-825. gibbs, j.l., & kraemer, k.l. (2004). a cross-country investigation of the determinants of scope of e-commerce use: an institutional approach. electronic markets, 14(2): 124-137. gonzalez, n., miers, c., redigolo, f., simplicio, m., carvalho, t., näslund, m., & pourzandi, m. (2012). a quantitative analysis of current security concerns and solutions for cloud computing. journal of cloud computing, 1(1): 1-18. gupta, p., seetharaman, a., & raj, j. r. (2013). the usage and adoption of cloud computing by small and medium businesses. international journal of information management, 33(5): 861874. gutierrez, a., boukrami, e., & lumsden, r. (2015). technological, organisational and environmental factors influencing managers’ decision to adopt cloud computing in the uk. journal of enterprise information management, 28(6): 788-807. hair, f, black, w, babin, b, anderson, r & tatham, r (2005). multivariate data analysis, 6th edn, pearson prentice hall, upper saddle river, new jersey. harrigan, j. a., rosenthal, r., & scherer, k. (2008). new handbook of methods in non-verbal behaviour research. new york, ny: oxford university press. hsu, p.f., ray, s., & li-hsieh, y.y. (2014). examining cloud computing adoption intention, pricing mechanism, and deployment model. international journal of information management, 34(4): 474-488. hutley, n. (2012). modelling the economic impact of cloud computing kpmg. idc (2012) quantitative estimates of the demand for cloud computing in europe and the likely barriers to up-take, international data centre, https://ec.europa.eu/digital-singlemarket/en/news/final-report-study-smart-20130043-uptake-cloud-europe idc. (2015). worldwide cloud it infrastructure market growth expected to accelerate to 21% in 2015, driven by public cloud datacenter expansion, international data centre, https://www.idc.com/getdoc.jsp?containerid=prus25576415 australasian journal of information systems senarathna, wilkin, warren, yeoh & salzman 2018, vol 22, research article factors that influence adoption of cloud computing 19 ifinedo, p. (2011). internet/e-business technologies acceptance in canada's smes: an exploratory investigation. internet research, 21(3): 255-281. imo. (2013). privacy and cloud computing for australian government agencies. version 1.1. information management office. itiic. (2011). cloud computing: opportunities and challenges, information technology industry innovation council. jambekar, a.b., & pelc, k.i. (2002). managing a manufacturing company in a wired world. international. journal of information technology & management, 1(1): 131-141. ko, r.k., jagadpramana, p., mowbray, m., pearson, s., kirchberg, m., liang, q., & lee, b. s. (2011, july). trustcloud: a framework for accountability and trust in cloud computing. in services 2011 ieee world congress on (584-588). levenburg, n., magal, s.r., & kosalge, p. (2006). an exploratory investigation of organizational factors and e-business motivations among smfoes in the us. electronic markets, 16(1): 70-84. lippert, s.k., & govindarajulu, c. (2006). technological, organizational, and environmental antecendents to web service adoption. communication of the iima, 6(1): 146-158. low, c., chen, y., & wu, m. (2011). understanding the determinants of cloud computing adoption. industrial management & data systems, 111(7): 1006-1023. luis, m.v., luis, r.m., juan, c. & maik, l. (2009). a break in the clouds: towards a cloud definition, sigcomm computer communication review, 39(1): 50-55. lyytinen, k., & damsgaard, j. (2001). what’s wrong with the diffusion of innovation theory? (pp. 173-190). springer us. mccabe, b., & hancock, i. (2009). cloud computing: australian lessons and experiences. kpmg it advisory,[http://www.kpmg.com/au/en/issuesandinsights/articlespublications/docu ments/cloudcomputing-australian-lessons-and-experiences. pdf]. madrid-guijarro a, garcia d, van auken h. (2009). barriers to innovation among spanish manufacturing smes. journal of small business management, 47(4): 465-488. mahmood, z. (2011). data location and security issues in cloud computing. ieee international conference of emerging intelligent data &web technology. marston, s., li, z., bandyopadhyay, s., zhang, j., & ghalsasi, a. (2011). cloud computing — the business perspective. decision support systems, 51(1): 176-189. doi: 10.1016/j.dss.2010.12.006 mell, p., & grance, t. (2011). the nist definition of cloud computing. microsoft. (2016). microsoft survey: it leaders in australia are prioritising hybrid cloud to transform it https://news.microsoft.com/en-au/2016/11/10/microsoft-survey-it-leadersin-australia-are-prioritising-hybrid-cloud-to-transformit/#sm.00019qlk6p5kldkypms1faolzr0r8 mudge, j.c. (2010). cloud computing opportunities and challenges for australia (pp. 1-34). melbourne, victoria: australian academy of technology sciences and engineering (atse). australasian journal of information systems senarathna, wilkin, warren, yeoh & salzman 2018, vol 22, research article factors that influence adoption of cloud computing 20 norman, g. (2010). likert scales, levels of measurement and the “laws” of statistics. advances in health sciences education, 15(5): 625-632. office of the chief economist. (2016). australian industry report. at https://industry.gov.au/office-of-the-chiefeconomist/publications/australianindustryreport/assets/australian-industry-report2016-chapter-6.pdfoliveira, t., thomas, m., & espadanal, m. (2014). assessing the determinants of cloud computing adoption: an analysis of the manufacturing and services sectors. information & management, 51(5): 497-510. pallant, j. (2011). multivariate analysis of variance. spss survival manual. crows nest: allen & unwin, 20(11): 283-96. pearson, s. (2009). taking account of privacy when designing cloud computing services. icse workshop in software engineering challenges of cloud computing, 2009. cloud'09. prasad, a., green, p., heales, j., & finau, g. (2014). towards a model of cloud computing services for smes. australasian conference of information systems, auckland, nz. ramdani, b., & kawalek, p. (2007). smes & is innovations adoption: a review & assessment of previous research. academia revista latinoamericana de administración (39): 47-70. rogers, e.m. (1995). diffusion of innovation theory. free press, new york. rogers, em. (2003). diffusion of innovations, 5th ed., free press, new york. rosenbusch, n., brinckmann, j., & bausch, a. (2011). is innovation always beneficial? a metaanalysis of the relationship between innovation and performance in smes. journal of business venturing, 26(4): 441-457. ross, p.k., & blumenstein, m. (2015). cloud computing as a facilitator of sme entrepreneurship. technology analysis & strategic management, 27(1): 87-101. rowe, f., truex, d., & huynh, m.q. (2012). an empirical study of determinants of e-commerce adoption in smes in vietnam: an economy in transition. journal of global information management, 20(3): 23-54. salleh, s.m., teoh, s.y., & chan, c. (2012). cloud enterprise systems: a review of literature and its adoption. pacis 2012 proceedings. sarwar, a., & khan, m.n. (2013). a review of trust aspects in cloud computing security. international journal of cloud computing & services sciences, 2(2): 116-122. sharif, a.m. (2010). it's written in the cloud: the hype and promise of cloud computing. journal of enterprise information management, 23(2): 131-134. sullivan, g.m., and artino jr, a.r. (2013). analyzing and interpreting data from likert-type scales. journal of graduate medical education, 5(4): 541-542. sultan, n.a. (2011). reaching for the “cloud”: how smes can manage. international journal of information management, 31(3): 272-278. tan, sk., chong, cs., lin, b., & eze, cu. (2010). internet-based ict adoption among smes: demographic versus benefits, barriers, and adoption intention. journal of enterprise information management, 23(1): 27-55. australasian journal of information systems senarathna, wilkin, warren, yeoh & salzman 2018, vol 22, research article factors that influence adoption of cloud computing 21 tehrani, s.r., & shirazi, f. (2014). factors influencing the adoption of cloud computing by small and medium size enterprises (smes). in human interface & management of information: information & knowledge in applications and services (631-642). springer international publishing. thompson b. (2004). exploratory and confirmatory factor analysis: understanding concepts and applications. washington, dc: american psychological association. trigueros-preciado, s., pérez-gonzález, d., & solana-gonzález, p. (2013). cloud computing in industrial smes: identification of the barriers to its adoption and effects of its application. electronic markets, 23(2): 105-114. vanessa, r. (2014). a us-china comparative study of cloud computing adoption behavior: the role of consumer innovativeness, performance expectations and social influence, journal of enterprise in emerging economies, 6(1): 53-71. venters, w., & whitley, e.a. (2012). a critical review of cloud computing: researching desires and realities. journal of information technology, 27(3): 179-197. waterford technologies. (2017) 2017 cloud facts and statistics. https://www.waterfordtechnologies.com/cloud-computing-stats-2017/ werfs, m., baxter, g., allison, i.k., & sommerville, i. (2013). migrating software products to the cloud: an adaptive sts perspective. journal of information technology & information management, 22(3). whitten, d., chakrabarty, s., & wakefield, r. (2010). the strategic choice to continue outsourcing, switch vendors, or backsource: do switching costs matter? information & management, 47(3): 167-175. yang, h., & tate, m. (2012). a descriptive literature review and classification of cloud computing research. communications of the association for information systems, 31(2): 3560. zahra s.a. & george g. (2002) absorptive capacity: a review, reconceptualization and extension, academy of management review 25(2):185-203. zhao, f., scheruhn, h.-j., & von rosing, m. (2014). the impact of culture differences on cloud computing adoption human-computer interaction. application & services (pp. 776-785): springer. australasian journal of information systems senarathna, wilkin, warren yeoh & salzman 2018, vol 22, research article factors that influence adoption of cloud computing 22 appendix a: sources of items for the survey instrument. factor academic government practitioner key points for 28 items cloud relative advantage (cra) oliveira et al. (2014); tehrani & shirazi (2014); vanessa (2014); gangwar et al. (2015) dbcde (2013); doc (2014) hutley (2012); ben (2014) 1. increase profits 2. reduce operating costs 3. allow organization to start new businesses 4. better communication with customers & suppliers 5. no upfront capital investment* 6. capability to extend/reduce access at any time* cloud flexibility (cf) oliveira et al. (2014); tehrani & shirazi (2014); li et al. (2015); itiic (2011); jansen & grance (2011); dbcde (2013); doc (2014) frost & sullivan (2011); hutley (2012); acca (2014); ben (2014); minifie (2014) 1. cloud computing (cc) is a reliable service 2. cc is highly available 3. cc is an up-to-date service 4. cc responds quickly to customers’ requests quality of service (qos) kauffman et al. (2014); mero & mwangoka (2014); itiic (2011); jansen & grance (2011); dbcde (2013); doc (2014) jansen & grance (2011); telstra (2011); herhalt & cochrane (2012); ren et al. (2012) 1. cc creates a flexible environment 2. cc permits dynamic resource scaling 3. cc means access to cloud from various client devices 4. cc enables agility re changing business environment* cloud security (cs) oliveira et al. (2014); tarmidi et al. (2014); tehrani & shirazi (2014); gangwar et al. (2015); li et al. (2015); safari et al. (2015); tang and liu (2015) dfd (2011a); itiic (2011); jansen & grance (2011); anthony (2012); dbcde (2013); imo (2013); doc (2014) telstra (2011); herhalt & cochrane (2012); pearson (2012); ren et al. (2012); ben (2014); minifie (2014) 1. security is a major concern with cc 2. cc is more secure than traditional computing australasian journal of information systems senarathna, wilkin, warren yeoh & salzman 2018, vol 22, research article factors that influence adoption of cloud computing 23 3. we are willing to host sensitive information on cc cloud privacy (cp) tehrani & shirazi (2014); vanessa (2014); gangwar et al. (2015); safari et al. (2015); tang & liu (2015) anthony (2012); dfd (2011a; 2011b); imo (2013) telstra (2011); herhalt & cochrane (2012); pearson (2012); ren et al. (2012); ben (2014); minifie (2014) 1. prefer australian storage 2. feels less privacy with storage in different country 3. feels cc cannot be trusted awareness of cloud (awc) tarmidi et al. (2014); tehrani & shirazi (2014); shetty & kumar (2015) itiic (2011); jansen & grance, (2011); anthony (2012) ko et al. (2011); pearson (2012); ning (2013); acca (2014); minifie (2014) 1. our organization aware of cc 2. difference between saas, paas and iaas understood 3. aware cc linked with other applications 4. familiar with public, private and hybrid cloud services adopt oliveira et al. (2014); tehrani & shirazi (2014); vanessa (2014); dfd (2011c) csa (2009); pearson (2012); dave (2012) 1. our organization intends to adopt cloud computing 2. our organization feels that the organization’s needs can be met by cloud computing 3. our organization will take steps to adopt cloud computing in the future 4. our organization will adopt cloud computing in the next twelve months *items deleted during factor analysis i.e., cra 5, cra 6, and qos 4 australasian journal of information systems senarathna, wilkin, warren yeoh & salzman 2018, vol 22, research article factors that influence adoption of cloud computing 24 appendix b: the survey instrument australasian journal of information systems senarathna, wilkin, warren yeoh & salzman 2018, vol 22, research article factors that influence adoption of cloud computing 25 australasian journal of information systems senarathna, wilkin, warren yeoh & salzman 2018, vol 22, research article factors that influence adoption of cloud computing 26 if cloud computing is being used, directed to: australasian journal of information systems senarathna, wilkin, warren yeoh & salzman 2018, vol 22, research article factors that influence adoption of cloud computing 27 if cloud computing is being used without aware, directed to: australasian journal of information systems senarathna, wilkin, warren yeoh & salzman 2018, vol 22, research article factors that influence adoption of cloud computing 28 if cloud computing is not being used, directed to: australasian journal of information systems senarathna, wilkin, warren yeoh & salzman 2018, vol 22, research article factors that influence adoption of cloud computing 29 australasian journal of information systems senarathna, wilkin, warren yeoh & salzman 2018, vol 22, research article factors that influence adoption of cloud computing 30 copyright: © 2018 senarathna, wilkin, warren, yeoh & salzman. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. australasian journal of information systems senarathna, wilkin, warren yeoh & salzman 2018, vol 22, research article factors that influence adoption of cloud computing 31 paper title (paper title style) australasian journal of information systems lamp 2017, vol 21, selected papers from acis 2016 preface to selected papers from acis 2016 1 preface to selected papers from acis 2016 john lamp editor in chief ajis is very pleased to present four papers selected from acis 2016, held at the university of wollongong, and expanded for journal publication janak adhikari, anuradha mathrani, chris scogings, “a longitudinal journey with byod classrooms: issues of access, capability and outcome divides” we have conducted a longitudinal study to investigate a byod initiative by a new zealand school. this study shares rich insights in the context of technology-mediated pedagogies and specifically byod classroom, as to how digital divides moved beyond access and skills to ensure inclusive learning outcomes. as a part of the five-year study of the technology-mediated teaching and learning initiative, we have been able to explain some of the unanswered questions around the issue of digital divides in the learning process. we investigated issues pertaining to digital divide in the context of byod classrooms to make the following revelations. first, the byod classroom initiative did not end up accentuating existing gaps in access to digital tools and technologies, despite earlier studies indicating towards increase in gaps. second, our analysis strongly indicated the changing nature of digital divides with the presence of gaps in terms of information literacy and critical thinking ability, as the byod classroom progressed to mature stage. this was eventually bridged in the later stage, as students slowly adjusted to the classroom curricular structures in the byod classroom. third, learner self-efficacy has been identified as a determinant of learning outcomes. in the earlier phase of ict adoption, learner self-efficacy is influenced by a combination of information literacy, critical thinking ability, and positive motivation; however subsequently, self-efficacy influences affordances in various aspects of social cognitive abilities related to individual’s learning activities affecting how learners engage and apply technology to achieve learning outcomes. philip dobson, paul jackson, “using critical realism and reflexivity to explain broadband nonadoption in rural australia” australia is conducting a substantial nationwide provision of broadband. it is primarily a fixed line network but includes wireless and satellite networks in more remote areas. the rollout is under the control of the nbn co, whose goal is ensuring access to fast broadband for all australians. the nbn co has recently recognized the importance of adoption by including premises activated as a kpi alongside service provision. coverage and adoption reflect the two faces of the nbn rollout – as a technical program and as a social program. adoption will be the ultimate measure of the success of the nbn as a social program. ubiquitous internet adoption across all sectors is necessary to maximize the promised benefits. the adoption of broadband is an important first step in achieving this. international experience suggests that broadband adoption is plateauing and this has prompted our proposal that there needs to be focus on nonadoption rather than adoption. we suggest that it is important to be able to explain the mechanisms by which individuals respond to the promise of the internet. only by so doing can we address issues. we contend that there needs to be more focus on those disenchanted or disinterested “non-users” who are never likely to adopt without specific targeted strategies. we argue for a critical realist perspective, more particularly reflexivity, to better represent the adoption context and to provide a grounding for explanations of the causes behind the decision not to adopt. we also propose possible common-sense strategies to reverse non-adoption. agata mccormac, dragana calic, marcus butavicius, kathryn parsons, tara zwaans, malcolm pattinson, “a reliable measure of information security awareness and the identification of bias in responses” australasian journal of information systems lamp 2017, vol 21, selected papers from acis 2016 preface to selected papers from acis 2016 2 the human aspects of information security questionnaire (hais-q) is designed to measure information security awareness. more specifically, the tool measures an individual’s knowledge, attitude, and self-reported behaviour relating to information security in the workplace. this paper reports on the reliability of the hais-q, including test-retest reliability and internal consistency. the paper also assesses the reliability of three preliminary over-claiming items, designed specifically to complement the haisq, and identify those individuals who provide socially desirable responses. a total of 197 working australians completed two iterations of the hais-q and the over-claiming items, approximately 4 weeks apart. results of the analysis showed that the hais-q was externally reliable and internally consistent. therefore, the hais-q can be used to reliably measure information security awareness. reliability testing on the preliminary over-claiming items was not as robust and further development is required and recommended. the implications of these findings mean that organisations can confidently use the hais-q to not only measure the current state of employee information security awareness within their organisation, but they can also measure the effectiveness and impacts of training interventions, information security awareness programs and campaigns. the influence of cultural changes and the effect of security incidents can also be assessed. meng xiao, rob meredith, shijia gao, “an exploratory study investigating how and why managers use tablets to support managerial decision-making” managers are often mobile and a large proportion of their work is dealing with decisions. although many managers currently use tablet computers in their work, there is little research on the use of tablets for managerial decision-support. this exploratory study aims to investigate the ways in which managers use tablets to support their decision-making and the reasons why they do so. using task-technology fit theory, semi-structured interviews were conducted with 20 managers, 17 of whom used tablets for their work-related decision-making. the study reveals managers’ tablet usage patterns in terms of location, tablet applications, decision activities and types. this study has also found that a range of tablet characteristics and decision-task characteristics affect managers’ use of tablets to support decision-making at work. this exploratory study contributes to both academia and industry by providing evidence on the tablet decision-support area, and affording organisations, tablet vendors and tablet application developers informative findings for further improvement in the provision of tablet-based decision support. being able to provide the authors of these papers with a wider audience is a service that this journal values as a means of promoting their research. copyright: © 2017 lamp. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems lamp 2017, vol 21, selected papers from acis 2016 preface to selected papers from acis 2016 3 microsoft word ajis vol 19.docx iii editorial march 2015 we are often reminded that we live in an era of rapid change, and the changes that we all have been instrumental in ushering in are often called disruptive changes as a result of their effect on the previous status quo. ajis is not exempt from these changes. since 1993, when ajis first appeared, changes have been made to its format, it began open access electronic publication including mounting all articles back to volume 1 issue 1. not many journals can say that. having said that, ajis still has some aspects in which it could improve. in 2015 we address several of those. firstly, the volume numbering has been brought into alignment with calendar years. previously volumes spanned two years – the first issue came in the second half of one year and the second issue came in the first half of the following year. the final issue for 2014 was 18(3), and from now volumes will match calendar years with this year being vol 19, 2016 vol 20 et seq. this now allows for a change which should greatly benefit our community of authors. from volume 19, ajis moves to progressive publication of articles. the practice of having separate issues in a volume was a compromise required in the days of hard copy publication. an annual volume makes sense as a convenient linear collation. having issues is a trade-off between releasing a single annual volume and the consequent delay in the appearance of an article, and breaking up the volume by having multiple issues to allow a more speedy appearance. electronic publication removes the need for issues, as it allows for the immediate appearance of an article once accepted and copyedited. while i have always advocated this, the need was dramatically brought home to me when releasing issue 18(2). the following week an article went into copyediting, and was consigned to the locker for six months to appear in issue 18(3)! that helps nobody. from vol 19 ordinary research articles will appear as soon as they are accepted and copyedited. special issues will become special sections of ajis, and all articles in a special section will appear simultaneously once the section editors have cleared them. given these changes, it is almost trivial to mention that ajis will revert to an a4 format, which is more convenient for electronic reading, machine readable copyright advice appears at the article level and qr codes now appear with articles for ease of locating the journal entry for the article. the latter changes move ajis towards best practice in the area of open access journals. a new style has also been adopted, as i hope you have noticed, to provide a more clearly defined and more easily readable presentation of ajis articles. i would like to acknowledge the considered input received from dr craig parker on the design of the new style. dr john lamp ajis editor-in-chief deakin university iv copyright: © 2015 john w lamp. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. microsoft word 1667-5447-2-le.docx australasian journal of information systems chattopadhyay, sen & gupta 2018, vol 22, research article meta-analysis of machine learning techniques 1 a comprehensive review and meta-analysis on applications of machine learning techniques in intrusion detection manojit chattopadhyay indian institute of management raipur india mchattopadhyay@iimraipur.ac.in rinku sen nshm college of management and technology kolkata india sumeet gupta indian institute of management raipur india abstract securing a machine from various cyber-attacks has been of serious concern for researchers, statutory bodies such as governments, business organizations and users in both wired and wireless media. however, during the last decade, the amount of data handling by any device, particularly servers, has increased exponentially and hence the security of these devices has become a matter of utmost concern. this paper attempts to examine the challenges in the application of machine learning techniques to intrusion detection. we review different inherent issues in defining and applying the machine learning techniques to intrusion detection. we also attempt to identify the best technological solution for the changing usage pattern by comparing the different machine learning techniques on different datasets and summarizing their performance using various performance metrics. this paper highlights the research challenges and future trends of intrusion detection in dynamic scenarios of intrusion detection problems in diverse network technologies. keywords: intrusion detection, machine learning, soft computing, dataset, performance metrics, cyber-infrastructure, mobile communications, mobile systems, security, wireless technology 1 introduction intrusion detection deals with detecting any malicious activities that may jeopardise the system. the attack can be an internal attack, modification of the important system files categorized as host-based intrusion detection or attack coming from the network system commonly known as the network intrusion-based detection system. apart from these types, any deviation of software performance from normal functioning is also termed as anomalybased intrusion detection. the intrusion detection has been a field of study for around four decades. since 1972, securing data has been a matter of concern for researchers. for example, anderson et al., (1995) discussed about the security problems faced by united states air force (usaf) operations and administration. in 1980, he prepared a detection system from an audit log file. between 1984 and 1986, denning and neumann (1985) developed the first real time australasian journal of information systems chattopadhyay, sen & gupta 2018, vol 22, research article meta-analysis of machine learning techniques 2 intrusion detection. since then various researchers applied different techniques to mitigate the more sophisticated form of attacks increasing day by day. the information security company – hold security – reported recovery of 360 million account credentials for web services from the black market (meyer, 2017). seals (2015) reported that the world wide distributed denial of service attack doubled from the 1st quarter of year 2014 to starting of year 2015. companies such as ebay suffered a massive attack where 233 million users’ personal databases were hacked. the heartbleed encryption bug affected about 17% of the internet's secure web servers by making passwords vulnerable which was protected by ssl/tls encryption (mccartney, 2014; mcgregor, 2014). the intrusion detection process encompasses various set of activities. the most difficult task is to demarcate a clear boundary between normal and malicious activities in a system. in the late 80s and early 90s, statistical approach, expert system, and time series models were capable enough of building a detection model. however, they failed to perform well in a complex highly correlated dataset and their performance degraded with noisy data. machine learning techniques with self-learning ability can effectively overcome the above limitation. research has been conducted on how data mining, soft computing (fuzzy logic), and statistical (bayesian network) approaches have been applied to solve the intrusion detection problem with discussion on the concept of honeypots to detect intrusion (lee et al., 200; kabiri et al., 2005). applications of computational intelligence techniques, datasets and performance evaluation has also been surveyed in literature (wu and banzhaf., 2010). however, there is a dearth of literature on the detailed quantitative analysis of the vast number of machine learning algorithms successfully applied for intrusion detection. the aim of this review is therefore to perform a comprehensive analysis of machine learning techniques, and in the process try to find out the applications of the most popular machine learning techniques on the most popular dataset. the effectiveness of these techniques on datasets in terms of performance metrics has been also evaluated. lastly, we assess different challenges prevailing in this field and identify possible research direction. the novelty of the present study is in performing the numerical analysis on the applied machine learning techniques to elucidate the success of intrusion detection from widely published literature and provide a trend for machine learning approaches to detect attack. we divide our study into the following sections. in section 2, an overview of intrusion detection technology is discussed. section 3 introduces current state-of-the-art techniques in intrusion detection. the results of the critical review are provided in section 4 and subsequently discussed in section 5. finally, the in-depth observation on surveyed outcomes concludes in section 6. 2 literature review 2.1 intrusion detection the basic function of an intrusion detection system is to monitor an activity taking place in a system and to generate an alarm report stating whether an attack is happening or whether everything is normal. figure 1 depicts this aspect of intrusion detection. australasian journal of information systems chattopadhyay, sen & gupta 2018, vol 22, research article meta-analysis of machine learning techniques 3 figure 1. steps in intrusion detection intrusion detection can be classified into two types, namely misuse and anomaly. misuse detection identifies intruders by comparing them with predefined description of events that denote attack. it matches an event which has already occurred with these predefined events, and any case of deviation is categorized as intrusion. however, this method cannot determine new or unknown attacks. on the other hand, anomaly detection builds models of normal events, and any deviations from these normal events are categorized as attacks. 2.2 technology types different technologies are applied to detect intruders, but each has its own limitations as summarized in table 1. the description in the table 1 reveals that a deeper analysis of the intrusion detection system is necessary. existing threats and their solutions still need in-depth analysis from the point of view of researchers. a computing system with high computational speed, fault tolerance, and ability to deal with dynamic real time data is necessary. several works reported successful intrusion detection system based on machine learning techniques which fit perfectly for the aspects like higher computation speed (rathore et al., 2016), fault tolerance (gao et al., 2015), and dealing real time big data (singh et al., 2014). different machine learning techniques bear close resemblance to computational statistics methods such as markov chain’s monte carlo method and kernel density function (julisch & dacier, 2002). due to these similarities of computational statistics, machine learning has attracted researchers’ attention to use them in the field of intrusion detection. on the other hand, neural networks become attractive for easy implementation, faster speed of learning, more generalizability and better dealing with nonlinear activation functions and kernels. due to increasing demand for symbolic representation of problems, symbolic artificial intelligence gains attention of the researchers. australasian journal of information systems chattopadhyay, sen & gupta 2018, vol 22, research article meta-analysis of machine learning techniques 4 techniques used to solve intrusion detection subsystem advantage disadvantage statistical based system (julisch & dacier, 2002) univariate quick and inexpensive to operate. extra attribute handling is difficult. multivariate robust relationship between variables are complex, hence very difficult in handling. helps establish the relationship between numbers of variables and find out the relationship between them. for giving the effective result large amount of data is required knowledge based or expert based system (mann & kaur, 2013; julisch & dacier, 2002) finite state machine it helps in solving the complex system using simple states. a number of states if large, can be unmanageable. time series model it forecasts by converting the nonlinear model to linear model. not all packet type especially real-time system packets can be converted to linear model, making the task difficult. expert system as databases prepared by experts, therefore, all point of view of attributes of dataset are considered. human interaction is needed to create the rule. data mining technique (sisodia et al., 2012) hierarchical less cost to join the clusters once the cluster is formed joining two or more clusters, decomposing the cluster is difficult. performance degrades with noise partitioning suitable for dataset where the relationship between attributes are less. performance degrades with noise. simple reliability on the user to create clusters grid based method performance does not degrade with noise. clustering is done on the summary of objects and not on an individual object. if any error occurs in an individual object, then the overall result becomes inaccurate. efficient handling of high dimensional data takes less time table 1. popular intrusion detection techniques and their advantages and disadvantages however, the approach of combining various techniques for improving the robustness of intrusion detection is not new (fossaceca et al., 2015). mukkamala et al., (2005) combined artificial intelligence-based methods, such as ensemble of neural networks, support vector machine, and multiple adaptive regressive splines achieved better results than individual method. however, it is challenging to apply the same on a big intrusion dataset. research (e.g. sabhnani & serpen, 2003) also suggest using combination of multiple classification algorithms that work efficiently against each intrusion type in future intrusion detection research australasian journal of information systems chattopadhyay, sen & gupta 2018, vol 22, research article meta-analysis of machine learning techniques 5 2.3 detection methodologies to overcome problems the limitations of various techniques as mentioned in table 1 in intrusion detection, it is important to identify the exact requirement of intrusion detection in the current environment characterized by rapidly changing data, large volume and variety. from the extensive study of articles mentioned in table 2, we found out that the researcher has effectively solved one or two perspectives of the problem. we feel that to keep pace with the changing pattern of attack, solving one or two approaches may not be sufficient. along with a good detection rate and accuracy, the following requirements (presented in the form of objectives – obj1, obj2 …) have to be addressed for a reliable intrusion detection. obj1: efficient clustering and classification, that is, the machine should not be over-trained and should give unbiased result the articles mentioned in table 2 basically classify and cluster the attacks on a particular system. henceforth, the techniques used by different authors is based on doing feature selection (which select the most relevant attributes contributing to the attack) and then training the system of the possible vulnerabilities. when a new set of attack is fed to the system, they can identify and classify new attack by comparing with the normal data. any machine learning technique has a tendency of overfitting, which leads to erroneous results on a new data set. due to overfitting, the technique fails to segregate a normal data from malicious attack and misleads the user by making proper prediction. obj2: less human interaction human beings, in the form of system administrator or programmers, are usually involved in intrusion detection for configuring the environmental settings, writing requisite code and analysing the results. however, attacks evolve every fortnight and demand improvement of the human personal in detecting attacks or anomalies. machine learning techniques make their task easier because of their self-learning capability, if these techniques are trained with certain set of patterns. the similar kind of patterns can be identified by them automatically without requiring human intervention. obj3: low computational overhead and cost the data required to be handled for intrusion detection is huge. handling this huge amount of data is a herculean task for any researcher. optimal selection of criteria which can identify the majority type of attacks is a challenging task for a researcher. tsang (2005) tried to select an optimal feature subset for dimensionality reduction. aslahi-shahri et al (2016) and chung and wahid (2012) also tried to reduce the data, so that the execution time is reduced, and space management becomes effective. hu et al (2014) also tried to reduce the communication cost of handling a distributed database. most of the studies have strived to achieve this through the feature selection mechanism, otherwise results obtained by analysing un-filtered data by compromising space, time and system performance will have no significance later, especially in case of a real time attack. obj4: identification of the new type of attacks the mode of attack is constantly changing as new types of virus and worms are invented every day. the software developed today for handling the attack may not be equipped to handle these new types of attack thus resulting in loss of vital data and consequent huge economic loss. to combat this problem the researcher strives to design a system capable of identifying australasian journal of information systems chattopadhyay, sen & gupta 2018, vol 22, research article meta-analysis of machine learning techniques 6 new types of attack by training the system with some predefined pattern. hu et al. (2014) tried to identify new types of attack by reducing the communication cost. stopel et al. (2009) proposed a system to detect new types of computer worms which will give good detection rate and accuracy and can be effectively used for identifying newly discovered attacks. obj5: robustnesscapability of handling large interrelated datasets and real type packets dataset used for intrusion detection is huge with interrelated data, especially when we are dealing with real time data. different researcher like bankovic et al (2007), denning and neumann (1985), and creech and (2014) has proposed a system for dealing with real time attacks. to classify a real time attack is a challenging task as they are interrelated with respect to time. for example, it is really difficult to accurately structure the behaviour of the system by extracting on an exact point, when two or more consecutive attacks are executed (chen et al., 2017). moreover, the user-content analysis demands accurate result within a fraction of second. if the system is not robust to handle this situation, it can be catastrophic and can jeopardize the system. 3 exploring current state–of-the-art techniques in intrusion detection 3.1 research methodology research articles related to several key words of the machine learning techniques and applications in intrusion detection were searched for and selected from science direct, taylor & francis, springer, scopus, google scholar, and the emerald online portals. we obtained more than 400 articles from these sources. the sample size considered was larger than that obtained in earlier studies. out of these 400, we considered the top 60 articles from reputed journals and analysed them. in all these 60 articles, author(s) obtained effective results by using the techniques such as svm, ann and fuzzy by one way or another. so we have included only these 60 articles in our paper and try to summarize the result based on them, so that from it we can give a new direction to the researcher. surveyed articles on intrusion detection are mentioned in table 6 in the appendix. sl. no. article ref. no. sl. no. article ref no. 1 abraham (2005) 1 35 mitrokotsa (2005) 49 2 aburomman (2016) 2 36 njogu (2013) 52 3 aslahi (2015) 5 37 orfila (2009) 54 4 ashfaq (2016) 4 38 ozyer (2007) 55 5 bankovic (2007) 6 39 pinzón (2013) 57 6 chen (2005) 8 40 powers (2007), 58 7 chung(2012) 10 41 peddabachigari (2007) 56 8 creech (2014) 11 42 ramasubramanian (2006) 59 9 damopoulos (2012) 12 43 revathi (2014) 61 10 de (2015) 13 44 sangkatsanee (2011) 63 11 elbasiony (2013) 15 45 sani (2015) 64 12 elhag (2014) 16 46 shamshirband (2014) 66 13 feng (2014) 17 47 sheikhan (2014) 67 australasian journal of information systems chattopadhyay, sen & gupta 2018, vol 22, research article meta-analysis of machine learning techniques 7 table 2: top works in the field of intrusion detection 4 data analysis and results the distribution of our studies on intrusion detection in the area of machine learning field from 2002 to 2015 is presented in figure 2. figure 2 reveals that the use of machine learning was at its peak in 2007, reduced in 2011, but again gained momentum during 2013-2016. figure 2. showing the distribution of research in the field of intrusion detection 14 fisch (2010) 18 48 shon (2007) 68 15 ganapathy (2012) 20 49 sindhu (2012), 69 16 grediaga (2006) 22 50 stopel (2009) 72 17 han (2006) 23 51 subbulakshmi (2010) 73 18 hoang (2009) 24 52 tajbakhsh (2009) 74 19 horng (2011) 25 53 tong (2009) 75 20 hu (2014 b) 26 54 toosi (2007) 76 21 hu (2008) 27 55 tsang (2007) 77 22 jazzar (2008) 28 56 wang (2010) 78 23 jiang (2009) 29 57 yin (2005) 80 24 jiang (2006) 30 58 zanero (2005) 81 25 khan (2006) 33 59 zhang (2007) 82 26 kim (2014) 34 60 zhang (2005) 83 27 kuang (2014) 35 28 kumar (2015) 36 29 kumar (2013) 38 30 lei (2012) 40 31 li (2009) 41 32 liao (2009) 42 33 luo (2014) 43 34 mabu (2011) 44 australasian journal of information systems chattopadhyay, sen & gupta 2018, vol 22, research article meta-analysis of machine learning techniques 8 we attempted to identify the most popular techniques through our survey and summarize the result in the form of a pie chart in figure 3. figure 3. popular machine learning techniques in intrusion detection figure 3 reveals that perceptron (single layer and multi-layer) 18%, fuzzy logic 14%, genetic algorithm 19% and support vector machine 18% are the most popular techniques for intrusion detection. we now attempt to sub-categorize these techniques with respect to the dataset it was capable of handling effectively. we obtained these data sets from the following sources: i. data packets from networks ii. data collected from snort or tcpdump, and iii. system call or cpu usage or memory usage. these datasets can be summarized as follows: • ids_bag1: this dataset comprises a collection of system call sequences collected at massachusetts institute of technology lincoln lab for one week. • bsm dataset: this dataset is collected from the victim’s solaris machine. bsm audit logs contain system calls produced by the program running on solaris machine. for each day, a ‘bsm list file’ is prepared and each line of the file indicates a session. the line contains information such as time, service, source ip, and destination ip. a ‘0’ at the end of the line indicates the session is normal and a ‘1’ indicates the session is intrusive. • kdd dataset: kdd training dataset comprises of single connection vectors (4,900,000). each of connection vector has 41 features and is categorized as either normal or an attack, with just one specific type of the attack. following are the four types of simulated attacks: viz. denial of service attack (dos), user to root attack (u2r), remote to local attack (r2l), and probing attack. australasian journal of information systems chattopadhyay, sen & gupta 2018, vol 22, research article meta-analysis of machine learning techniques 9 • darpa intrusion detection evaluation: darpa intrusion detection evaluation has two parts, one operates in offline evaluation mode, and another operates in real-time evaluation mode. the offline mode normally operates in the batch mode and uses network traffic and audit logs collected on the simulated network. the system tries to identify attacks in the middle of normal activities. in the online mode, which is the real time mode, these systems are inserted into the afrl network test bed and are used to identify attack sessions in the middle of normal activities, in real time. • nsl-kdd: is a dataset which has solved many problems found in kdd dataset, though it is not suitable for real time networks because of lack of public data sets for network-based idss. it does not have redundant or duplicate records. the number of training and testing data is reasonable. effectiveness of a technique on a particular dataset can only be judged by analysing the correctness or accuracy of the result obtained. several performance metrics or measures are taken into consideration for this purpose, namely, sensitivity or true positive rate (proportion of correctly classified positive cases), detection rate (ratio of true positive data to the total number of intrusion data present in the system), accuracy (total number of correct predictions) and roc curve (graph used to measure the performance of binary classifier). from the 60 articles surveyed, we have found that ga covers 19%, svm covers 18%, fuzzy + neuro fuzzy covers 18%, som covers 18% and perceptron covers 18% of the total articles (figure 3). we did not include perceptron because its effectiveness in handling real time data, as well as in other types of readily available data, is not promising as in case of other techniques. the performance of numerous techniques using these parameters on the abovementioned dataset is summarized in figures 4a-4d. support vector machines (svm) self organizing maps (som) figure 4a. percentage of different data set used by support vector machines (svm) figure 4b. percentage of different data set used by self-organizing maps (som) australasian journal of information systems chattopadhyay, sen & gupta 2018, vol 22, research article meta-analysis of machine learning techniques 10 genetic algorithms (ga) fuzzy logic figure 4c. percentage of different data set used by genetic algorithms (ga) figure 4d. percentage of different data set used by fuzzy logic we can see from figure 4 that fuzzy, self-organizing maps, and genetic algorithm are used extensively in detecting real time attacks, whereas svm is used extensively with publicly available dataset (snort, caida, etc.). however, among all datasets, most of the experiments were carried out on kdd dataset. to further justify our claim, we take the help of the graphs in figure 5. figure 5. showing the popular datasets used with various techniques figure 5 reveals that these techniques have used kdd dataset in nearly 40% of their intrusion detection analysis, while they have worked with real time data set for only 25% of their analysis. australasian journal of information systems chattopadhyay, sen & gupta 2018, vol 22, research article meta-analysis of machine learning techniques 11 after understanding the popular datasets, we now try to understand the most popular performance metrics used by the above-mentioned techniques, and we obtain figure 6 which provides the evaluation of the different performance measures of intrusion detection. figure 6. showing the different performance metrics used by our popular techniques from figure 6, we can see that accuracy and detection rate and false positive rate are the most popular performance metrics used with machine learning techniques. 5 discussion the machine learning techniques and soft computing techniques are special techniques for handling huge data sets. fuzzy logic techniques effectively cluster overlapping datasets. they are also used extensively in removing attributes which are a misfit for a particular cluster. the fuzzy rule-based system tries to match a pattern with a set of predefined patterns and thereby helps in removing misfit data of the cluster. genetic algorithm (ga) is also capable of handling unrelated data in clusters for using the process of selection, crossover and mutation. som (self organizing map) reveals the most important relationships between the features and hides unwanted details. because of abstraction, these techniques are expert in handling complex interrelated data and the suppression of unnecessary details helps in saving time and unwanted processing. svm (support vector machine) converts the highly complex data, especially the text, into a form suitable for classification. neural network is effective by being trained on a certain pattern of data. it reports deviation from this pattern, if any, in a new dataset and thus helps in identification of dissimilarity formed by each cluster. therefore, this technique in stand-alone mode or with other techniques can solve the security threats followed worldwide. we summarize our findings in table 3. australasian journal of information systems chattopadhyay, sen & gupta 2018, vol 22, research article meta-analysis of machine learning techniques 12 problems in existing technologies solved effectively with dissimilarity of the discovered pattern with existing pattern fuzzy clustering, bpnn dependencies among data and understanding dependency clustering fuzzy logic by the progressive reduction of cognitive dessonance, som, ga web personalization ga, fuzzy rule based system data summarization fuzzy set theory, som, bpnn, creation of association rules fuzzy acyclic directed graph, fuzzy rule based system difficulty in processing documents containing images fuzzy logic, som regression neural network, neuro fuzzy computation extra attribute handling k-nearest neighbour, ga complexity of relationship owing to a large number of variables svm, decision tree time consuming som difficulty in managing huge dataset genetic algorithm, som, svm reliability on the user to change cluster genetic algorithm table 3: machine learning techniques and their solution as identified in table 1 obj1 obj2 obj3 obj4 obj5 abrahim (2007) √ √ √ √ aburomman (2016) √ √ √ ashfaq (2016) √ √ √ √ aslahi (2015) √ √ √ √ bankovic (2007) √ √ √ √ chen (2005) √ √ √ chung (2012) √ √ √ √ creech (2014) √ √ √ √ damopoulos (2012) √ √ √ √ de (2015) √ √ √ √ elbasiony (2013) √ √ √ √ elhag (2014) √ √ √ √ feng (2014) √ √ √ √ √ fisch (2010) √ √ √ √ ganapathy (2012) √ √ √ √ √ grediaga (2006) √ √ √ √ han (2005) √ √ √ √ hoang (2009) √ √ √ √ horng (2011) √ √ √ √ hu (2008) √ √ √ √ australasian journal of information systems chattopadhyay, sen & gupta 2018, vol 22, research article meta-analysis of machine learning techniques 13 obj1 obj2 obj3 obj4 obj5 hu (2014 ) √ √ √ √ jazzar (2008) √ √ √ √ jiang (2009) √ √ √ jiang (2006) √ √ √ √ khan (2007) √ √ √ kim (2014) √ √ √ √ kuang (2014) √ √ √ √ kumar (2015) √ √ √ √ kumar (2013) √ √ √ √ lei (2012) √ √ √ √ li (2009) √ √ √ √ liao (2009) √ √ √ √ luo (2014) √ √ √ mabu (2011) √ √ √ mitrokotsa (2005) √ √ √ njogu (2013) √ √ √ √ orfila (2009) √ √ √ özyer (2007) √ √ √ pinzón (2013) √ √ √ √ powers (2008) √ √ √ ramasubramanian (2006) √ √ √ √ √ revathi (2014) √ √ √ sangkatsanee (2011) √ √ √ √ sani (2015) √ √ √ shamshirband (2014) √ √ √ √ sheikhan (2014) √ √ √ √ shon (2007) √ √ √ √ sindhu (2012) √ √ √ stopel (2009) √ √ √ subbulakshmi (2010) √ √ √ √ tajbakhsh (2009) √ √ √ √ tong (2009) √ √ √ toosi (2007) √ √ √ tsang (2007) √ √ √ wang (2010) √ √ √ peddabachigari (2007) √ √ √ yin (2005) √ √ zanero (2005) √ √ zhang (2005) √ √ zhang (2007) √ √ √ √ table 4: comparison of different work according to our taxonomy australasian journal of information systems chattopadhyay, sen & gupta 2018, vol 22, research article meta-analysis of machine learning techniques 14 • obj1: efficient clustering and classification that is the machine should not be over trained and give bias result. • obj2: less human interaction • obj3: low computational and cost overhead • obj4: identification of new type of attack • obj5: robustnesscapability of handling large interrelated datasets and real type packets in table 4 we map our objectives (obj1 to obj5) with the existing articles and we conclude that though most of the articles fulfil requirements of obj1, obj2 and obj4, they are unable to meet the requirements of obj3 and obj5. as already pointed out that real time data are interrelated and demands special mechanism for segregation and this calls for special attention. we can observe that machine learning techniques with their self-learning or supervisory mode are able to detect most of the attacks and have provided very good detection and accuracy rate. however, these metrics alone do not consider the hostility of the environmental condition. for this reason, certain other metrics such as cost and sensitivity need to be taken into consideration. if the environmental condition is not taken into consideration, detection of new types of attack will be a difficult task. moreover, reliability on kdd dataset is also not a good solution to judge the efficacy of the system regarding detection of attacks. the kdd dataset contains redundant records and has laid less stress on u2r and r2l attacks. in most of the training experiments, these attacks are likely to be missed, as the number of rows is less as compared to other types of attacks. moreover, 75% to 78% of the records are duplicate. based on the training of kdd dataset, the new dynamic real time attack may not be handled by the system. there is still requirement for a unified architecture or technique, which will provide a platform to identify real time attack, and also a standardized solution in handling wired and wireless attack, as shown in table 5. it is evident from table 5 that different environmental conditions favour different techniques. area of observation detail observation concluding remark percentage of total analyzed literature som and svm covers 13% and 18% respectively. fuzzy and ga covers 14% and 19% respectively while perceptron covers 17% according to our survey ga,perceptron and svm are most popular tool. most common approach fuzzy rule based system with ga covers nearly 11% of analyzed literature genetic algorithm used on knowledge base dataset containing fuzzy rule are popular techniques used for feature selection. most common performance metrics accuracy and detection rate. detection rate covers 49% of analyzed data, accuracy covers 28% of analyzed data detection is given more importance than analysis during performance evaluation table 5: observation based on machine learning techniques approach in intrusion detection current trends in network involves distributed computing with increasing demand for cloud computing (i.e. more involvement of internet) (kumar et al., 2010). in addition, ad-hoc and sensor networks have indicated possibility of new type of attacks. the performance of most of the above mentioned techniques in dealing with intrusion in cloud platform or determining black hole attack in sensor networks is questionable. intrusion detection fails to determine the australasian journal of information systems chattopadhyay, sen & gupta 2018, vol 22, research article meta-analysis of machine learning techniques 15 attacks at different levels of architecture of a cloud. meanwhile as stated by modi et al (2013), internal attacks are also increasing. there is a lack of suitable mechanism to handle them. sensor networks on the other hand are more sensitive to attack. mobile nodes with poor inbuilt security mechanism are easy to capture via wired networks. an attacker can listen to traffic, modify the traffic or can act as one of the legitimate users. as there is no such central architecture which can help in intrusion detection, proper cryptography via public or private key is difficult to implement in mobile adhoc or sensor network. our article has identified the impact of machine learning on intruded packets and at the same time has identified the issue of security concern that are left to be handled when dealing with real time data, sensitive data in mobile phones and sensor networks. 6 conclusion this study provides an insight into the progress of research on intrusion detection based on machine learning techniques. it has discussed the most popular machine learning techniques, and their advantages and disadvantages. as machine learning techniques are extensively used with soft computing techniques we have been analysed them separately. this review graphically provides a clear indication of the overall picture of research in the field of intrusion detection with changing usage pattern, in terms of popular performance metrics and most widely used techniques and datasets. most of the techniques perform well with kdd. fuzzy logic techniques perform well with real time datasets. the study also revealed that machine learnings approaches applied to intrusion detection are quite successful except in the matter of fulfilling the objectives of low computational and cost overheads and robustness (capability of handling large interrelated datasets and real type packets). therefore, the future research may be directed towards those machine learning tools that will achieve both of these objectives (i.e. low computational cost and robustness). more obvious gap is the labelled data application like kdd dataset on which majority of the techniques are applied and it would be more worthy if intrusion data is collected and labelled partially. as the machine learning techniques require training and testing data so they can be trained using partial labelled dataset for the known attack and tested on unknown data for measuring performance. therefore, the promising techniques may be further tested on these new data set for developing effective and efficient intrusion detection system for breakthrough performance. moreover since majority of the results are based on kdd dataset, approximation of the actual performance of the intrusion detection system on real time data is difficult to evaluate. the effectiveness of these techniques on real time data and effective performance metrics used for their evaluation is an open area for the future researchers. references aburomman, a. a., & ibne reaz, m. bin. (2016). a novel svm-knn-pso ensemble method for intrusion detection system. applied soft computing journal, 38, 360–372. https://doi.org/10.1016/j.asoc.2015.10.011 anderson, d., frivold, t., & valdes, a. (1995). next-generation intrusion detection expert system (nides): a summary. sri international, (may 1995), 47. https://doi.org/citeulikearticle-id:7898221 australasian journal of information systems chattopadhyay, sen & gupta 2018, vol 22, research article meta-analysis of machine learning techniques 16 arun raj kumar, p., & selvakumar, s. (2013). detection of distributed denial of service attacks using an ensemble of adaptive and hybrid neuro-fuzzy systems. computer communications, 36(3), 303–319. https://doi.org/10.1016/j.comcom.2012.09.010 ashfaq, r. a. r., wang, x. z., huang, j. z., abbas, h., & he, y. l. (2017). fuzziness based semisupervised learning approach for intrusion detection system. information sciences, 378, 484–497. https://doi.org/10.1016/j.ins.2016.04.019 aslahi-shahri, b. m., rahmani, r., chizari, m., maralani, a., eslami, m., golkar, m. j. , & ebrahimi, a. (2016). a hybrid method consisting of ga and svm for intrusion detection system. neural computing and applications , 27(6), 1669–1676. https://doi .org/10.1007/s00521-015-1964-2 banković, z., stepanović, d., bojanić, s., & nieto-taladriz, o. (2007). improving network security using genetic algorithm approach. computers and electrical engineering, 33(5–6), 438–451. https://doi.org/10.1016/j.compeleceng.2007.05.010 chen, c., ghassami, a., mohan, s., n. k. (2017), a reconnaissance attack mechanism for fixedpriority real-time systems. arxiv.org. chung, y. y., & wahid, n. (2012). a hybrid network intrusion detection system using simplified swarm optimization (sso). applied soft computing, 12(9), 3014–3022. https://doi.org/10.1016/j.asoc.2012.04.020 creech, g., & hu, j. (2014). a semantic approach to host-based intrusion detection systems using contiguous and discontiguous system call patterns. ieee transactions on computers, 63(4), 807–819. https://doi.org/10.1109/tc.2013.13 damopoulos, d., menesidou, s. a., kambourakis, g., papadaki, m., clarke, n., & gritzalis, s. (2012). evaluation of anomaly-based ids for mobile devices using machine learning classifiers. security and communication networks, 5(1), 3–14. https://doi.org/10.1002/sec.341 de la hoz, e., de la hoz, e., ortiz, a., ortega, j., & prieto, b. (2015). pca filtering and probabilistic som for network intrusion detection. neurocomputing, 164, 71–81. https://doi.org/10.1016/j.neucom.2014.09.083 denning, d. & neumann, p. (1985). requirements and model for ides—a real-time intrusion detection expert system. document a005, sri international, california. 1-74. retrieved from faculty.nps.edu/dedennin/publications/idesreportsri1985.pdf (accessed on 3105-2018) elbasiony, r. m., sallam, e. a., eltobely, t. e., & fahmy, m. m. (2013). a hybrid network intrusion detection framework based on random forests and weighted k-means. ain shams engineering journal, 4(4), 753–762. https://doi.org/10.1016/j.asej.2013.01.003 elhag, s., fernández, a., bawakid, a., alshomrani, s., & herrera, f. (2015). on the combination of genetic fuzzy systems and pairwise learning for improving detection rates on intrusion detection systems. expert systems with applications, 42(1), 193–202. https://doi.org/10.1016/j.eswa.2014.08.002 feng, w., zhang, q., hu, g., & huang, j. x. (2014). mining network data for intrusion detection through combining svms with ant colony networks. future generation computer systems, 37, 127–140. https://doi.org/10.1016/j.future.2013.06.027 australasian journal of information systems chattopadhyay, sen & gupta 2018, vol 22, research article meta-analysis of machine learning techniques 17 fisch, d., hofmann, a., & sick, b. (2010). on the versatility of radial basis function neural networks: a case study in the field of intrusion detection. information sciences, 180(12), 2421–2439. https://doi.org/10.1016/j.ins.2010.02.023 fossaceca, j. m., mazzuchi, t. a., & sarkani, s. (2015). mark-elm: application of a novel multiple kernel learning framework for improving the robustness of network intrusion detection. expert systems with applications, 42(8), 4062-4080. ganapathy, s., kulothungan, k., yogesh, p., & kannan, a. (2012). a novel weighted fuzzy c means clustering based on immune genetic algorithm for intrusion detection. in procedia engineering (vol. 38, pp. 1750–1757). https://doi.org/10.1016/j.proeng.2012.06.213 gao, z., member, s., cecati, c., ieee, f., & ding, s. x. (2015). a survey of fault diagnosis and fault tolerant techniques part ii : fault diagnosis with knowledge based and hybrid / active approaches. ieee transactions on industrial electronics, 62(6), 3768–3774. https://doi.org/10.1109/tie.2015.2419013 grediaga, á., ibarra, f., garcía, f., ledesma, b., & brotóns, f. (2006). application of neural networks in network control and information security. in advances in neural networks isnn 2006 (pp. 208–213). https://doi.org/10.1007/11760191_31 han, s.-j., & cho, s.-b. (2006). evolutionary neural networks for anomaly detection based on the behavior of a program. ieee transactions on systems, man, and cybernetics. part b, cybernetics : a publication of the ieee systems, man, and cybernetics society, 36(3), 559–570. https://doi.org/10.1109/tsmcb.2005.860136 hoang, x. d., hu, j., & bertok, p. (2009). a program-based anomaly intrusion detection scheme using multiple detection engines and fuzzy inference. journal of network and computer applications, 32(6), 1219–1228. https://doi.org/10.1016/j.jnca.2009.05.004 horng, s. j . , su, m. y., chen, y. h., kao, t. w., chen, r. j . , lai , j . l. , & perkasa, c. d. (2011). a novel intrusion detection system based on hierarchical clustering and support vector machines. expert systems with applications , 38(1) , 306–313. https://doi .org/10.1016/j .eswa.2010.06.066 hu, w., gao, j., wang, y., wu, o., & maybank, s. (2014). online ada boost-based parameterized methods for dynamic distributed network intrusion detection. ieee transactions on cybernetics, 44(1), 66–82. https://doi.org/10.1109/tcyb.2013.2247592 hu, w., hu, w., & maybank, s. (2008). adaboost-based algorithm for network intrusion detection. ieee transactions on systems, man, and cybernetics, 38(2), 577–583. https://doi.org/10.1109/tsmcb.2007.914695 jazzar, m., & jantan, a. (2008). an approach for anomaly intrusion detection based on causal knowledge-driven diagnosis and direction. studies in computational intelligence, 149, 39– 48. https://doi.org/10.1007/978-3-540-70560-4_4 jiang, h., & ruan, j. (2009). the application of genetic neural network in network intrusion detection. journal of computers, 4(12), 1223–1230. https://doi.org/10.4304/jcp.4.12.12231230 jiang, s., song, x., wang, h., han, j. j., & li, q. h. (2006). a clustering-based method for unsupervised intrusion detections. pattern recognition letters, 27(7), 802–810. https://doi.org/10.1016/j.patrec.2005.11.007 australasian journal of information systems chattopadhyay, sen & gupta 2018, vol 22, research article meta-analysis of machine learning techniques 18 julisch, k., & dacier, m. (2002). mining intrusion detection alarms for actionable knowledge. in proceedings of the eighth acm sigkdd international conference on knowledge discovery and data mining kdd ’02 (p. 366). https://doi.org/10.1145/775047.775101 kabiri, p., & ghorbani, a. a. (2005). research on intrusion detection and response: a survey. international journal of network security, 1(2), 84–102. https://doi.org/10.1.1.129.698 khan, l., awad, m., & thuraisingham, b. (2007). a new intrusion detection system using support vector machines and hierarchical clustering. the vldb journal, 16(4), 507–521. https://doi.org/10.1007/s00778-006-0002-5 kim, g., lee, s., & kim, s. (2014). a novel hybrid intrusion detection method integrating anomaly detection with misuse detection. expert systems with applications, 41(4 part 2), 1690–1700. https://doi.org/10.1016/j.eswa.2013.08.066 kuang, f. , xu, w., & zhang, s. (2014). a novel hybrid kpca and svm with ga model for intrusion detection. applied soft computing journal , 18, 178–184. https://doi .org/10.1016/j .asoc.2014.01.028 kumar, g., & kumar, k. (2015). a multi-objective genetic algorithm based approach for effective intrusion detection using neural networks. in intelligent methods for cyber warfare (pp. 173–200). https://doi.org/10.1007/978-3-319-08624-8_8 kumar, g., kumar, k., & sachdeva, m. (2010). the use of artificial intelligence based techniques for intrusion detection: a review. artificial intelligence review, 34(4), 369–387. https://doi.org/10.1007/s10462-010-9179-5 lee, w., stolfo, s., & mok, k. (2000). adaptive intrusion detection: a data mining approach. artificial intelligence review (vol. 14). https://doi.org/10.1023/a:1006624031083 lei, j. z., & ghorbani, a. a. (2012). improved competitive learning neural networks for network intrusion and fraud detection. neurocomputing, 75, 135–145. https://doi.org/10.1016/j.neucom.2011.02.021 li, y., wang, j. l., tian, z. h., lu, t. b., & young, c. (2009). building lightweight intrusion detection system using wrapper-based feature selection mechanisms. computers and security, 28(6), 466–475. https://doi.org/10.1016/j.cose.2009.01.001 liao, n., tian, s. , & wang, t. (2009). network forensics based on fuzzy logic and expert system. computer communications , 32(17), 1881–1892. https://doi .org/10.1016/j .comcom.2009.07.013 luo, b., & xia, j. (2014). a novel intrusion detection system based on feature generation with visualization strategy. expert systems with applications, 41(9), 4139–4147. https://doi.org/10.1016/j.eswa.2013.12.048 mabu, s., chen, c., lu, n., shimada, k., & hirasawa, k. (2011). an intrusion-detection model based on fuzzy class-association-rule mining using genetic network programming. ieee transactions on systems, man, and cybernetics, part c: applications and reviews, 41(1), 130–139. https://doi.org/10.1109/tsmcc.2010.2050685 mccartney, j . (2014). the top 10 data breaches of the past 12 months , http://www.in.techradar.com/news/software/security-software/the-topaustralasian journal of information systems chattopadhyay, sen & gupta 2018, vol 22, research article meta-analysis of machine learning techniques 19 10-data-breaches-of-the-past-12-months/articleshow/38472866.(accessed on 12-05-2016) . mcgregor, j . (2014). the top 5 most brutal cyber attacks of 2014 so far , http://www.forbes.com/sites/jaymcgregor/2014/07/28/the-top-5-mostbrutal-cyber-attacks-of-2014-so-far/#5722d80721a6.(accessed on 12-052016). meyer, c., (2017). data theft, http://www.infosecwriters.com/papers/cmeyer_datatheft.pdf (accessed on 27-03-2018) mitrokotsa, a., & douligeris, c. (2005). detecting denial of service attacks using emergent selforganizing maps. proceedings of the fifth ieee international symposium on signal processing and information technology, 2005., 375–380. https://doi.org/10.1109/isspit.2005.1577126 modi, c., patel, d., borisaniya, b., patel, h., patel, a., & rajarajan, m. (2013). a survey of intrusion detection techniques in cloud. journal of network and computer applications. https://doi.org/10.1016/j.jnca.2012.05.003 mukkamala, s., sung, a. h., & abraham, a. (2005). intrusion detection using an ensemble of intelligent paradigms. journal of network and computer applications, 28(2), 167–182. https://doi.org/10.1016/j.jnca.2004.01.003 njogu, h. w., jiawei, l., & kiere, j. n. (2013). network specific vulnerability based alert reduction approach. security and communication networks, 6(1), 15–27. https://doi.org/10.1002/sec.520 orfila, a., estevez-tapiador, j. m., & ribagorda, a. (2009). evolving high-speed, easy-tounderstand network intrusion detection rules with genetic programming. in lecture notes in computer science (including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics) (vol. 5484 lncs, pp. 93–98). https://doi.org/10.1007/9783-642-01129-0_11 özyer, t., alhajj, r., & barker, k. (2007). intrusion detection by integrating boosting genetic fuzzy classifier and data mining criteria for rule pre-screening. journal of network and computer applications, 30(1), 99–113. https://doi.org/10.1016/j.jnca.2005.06.002 peddabachigari, s., abraham, a., grosan, c., & thomas, j. (2007). modeling intrusion detection system using hybrid intelligent systems. journal of network and computer applications, 30(1), 114–132. https://doi.org/10.1016/j.jnca.2005.06.003 pinzón, c. i., de paz, j. f., herrero, á., corchado, e., bajo, j., & corchado, j. m. (2013). idmassql: intrusion detection based on mas to detect and block sql injection through data mining. information sciences, 231, 15–31. https://doi.org/10.1016/j.ins.2011.06.020 powers, s. t., & he, j. (2008). a hybrid artificial immune system and self organising map for network intrusion detection. information sciences, 178(15), 3024–3042. https://doi.org/10.1016/j.ins.2007.11.028 ramasubramanian, p., & kannan, a. (2006). a genetic-algorithm based neural network shortterm forecasting framework for database intrusion prediction system. soft computing, 10(8), 699–714. https://doi.org/10.1007/s00500-005-0513-9 australasian journal of information systems chattopadhyay, sen & gupta 2018, vol 22, research article meta-analysis of machine learning techniques 20 rathore, m. m., ahmad, a., & paul, a. (2016). real time intrusion detection system for ultrahigh-speed big data environments. journal of supercomputing, 72(9), 3489–3510. https://doi.org/10.1007/s11227-015-1615-5 revathi, s., & malathi, a. (2014). multi-tier framework using sugeno fuzzy inference system with swarm intelligence techniques for intrusion detection. indian journal of science and technology, 7(9), 1437–1443. sabhnani, m., serpen, g., & more, k. k. (2003). application of machine learning algorithms to kdd intrusion detection dataset within misuse detection context. proceedings of international conference on machine learning: models, technologies, and applications (mlmta), 209–215. https://doi.org/citeulike-article-id:9827151 sangkatsanee, p., wattanapongsakorn, n., & charnsripinyo, c. (2011). practical real-time intrusion detection using machine learning approaches. computer communications, 34(18), 2227–2235. https://doi.org/10.1016/j.comcom.2011.07.001 sani, r. a., & ghasemi, a. (2015). learning a new distance metric to improve an svmclustering based intrusion detection system. in proceedings of the international symposium on artificial intelligence and signal processing, aisp 2015 (pp. 284–289). https://doi.org/10.1109/aisp.2015.7123497 seals, t. (2015). ddos attacks more than double in 12 months. infosecurity. retrieved from http://www.infosecurity-magazine.com/news/ddos-attacks-more-than-double-in/ (accessed on 28-03-2018) shamshirband, s., patel, a., anuar, n. b., kiah, m. l. m., & abraham, a. (2014). cooperative game theoretic approach using fuzzy q-learning for detecting and preventing intrusions in wireless sensor networks. engineering applications of artificial intelligence, 32, 228–241. https://doi.org/10.1016/j.engappai.2014.02.001 sheikhan, m., & jadidi, z. (2014). flow-based anomaly detection in high-speed links using modified gsa-optimized neural network. neural computing and applications, 24(3–4), 599–611. https://doi.org/10.1007/s00521-012-1263-0 shon, t., & moon, j. (2007). a hybrid machine learning approach to network anomaly detection. information sciences , 177(18), 3799–3821. https://doi .org/10.1016/j . ins.2007.03.025 singh, k., guntuku, s. c., thakur, a., & hota, c. (2014). big data analytics framework for peer-to-peer botnet detection using random forests. information sciences, 278, 488–497. https://doi.org/10.1016/j.ins.2014.03.066 sisodia, d., singh, l., sisodia, s., & saxena, k. (2012). clustering techniques: a brief survey of different clustering algorithms. international journal of latest trends in engineering and technology (ijltet), 1(3), 82–87. sivatha sindhu, s. s., geetha, s., & kannan, a. (2012). decision tree based light weight intrusion detection using a wrapper approach. expert systems with applications, 39(1), 129–141. https://doi.org/10.1016/j.eswa.2011.06.013 stopel, d., moskovitch, r., boger, z., shahar, y., & elovici, y. (2009). using artificial neural networks to detect unknown computer worms. neural computing and applications, 18(7), 663–674. https://doi.org/10.1007/s00521-009-0238-2 australasian journal of information systems chattopadhyay, sen & gupta 2018, vol 22, research article meta-analysis of machine learning techniques 21 subbulakshmi, t., mercy shalinie, s., suneel reddy, c., & ramamoorthi, a. (2010). detection and classification of ddos attacks using fuzzy inference system. in communications in computer and information science (vol. 89 ccis, pp. 242–252). https://doi.org/10.1007/9783-642-14478-3_25 tajbakhsh, a., rahmati, m., & mirzaei, a. (2009). intrusion detection using fuzzy association rules. applied soft computing, 9(2), 462–469. https://doi.org/10.1016/j.asoc.2008.06.001 toosi, a. n., & kahani, m. (2007). a new approach to intrusion detection based on an evolutionary soft computing model using neuro-fuzzy classifiers. computer communications, 30(10), 2201–2212. https://doi.org/10.1016/j.comcom.2007.05.002 tsang, c. h., kwong, s., & wang, h. (2007). genetic-fuzzy rule mining approach and evaluation of feature selection techniques for anomaly intrusion detection. pattern recognition, 40(9), 2373–2391. https://doi.org/10.1016/j.patcog.2006.12.009 wang, g., hao, j., ma, j., & huang, l. (2010). a new approach to intrusion detection using artificial neural networks and fuzzy clustering. expert systems with applications, 37(9), 6225–6232. https://doi.org/10.1016/j.eswa.2010.02.102 wu, s. x., & banzhaf, w. (2010). the use of computational intelligence in intrusion detection systems: a review. applied soft computing journal , 10, 1–35. https://doi .org/10.1016/j .asoc .2009.06.019 yin, c., tian, s., huang, h., & he, j. (2005). applying genetic programming to evolve learned rules for network anomaly detection. advances in natural computation, first international conference, icnc 2005, proceedings, part iii, 3612, 323–331. https://doi.org/doi:10.1007/11539902_38 zanero, s. (2005). analyzing tcp traffic patterns using self organizing maps. image analysis and processing iciap 2005, proceedings, 3617, 83–90. https://doi.org/10.1007/11553595_10 zhang, c., jiang, j., & kamel, m. (2005). intrusion detection using hierarchical neural networks. pattern recognition letters, 26(6), 779–791. https://doi.org/10.1016/j.patrec.2004.09.045 zhang, j .-r., zhang, j ., lok, t.-m., & lyu, m. r. (2007). a hybrid particle swarm optimization–back-propagation algorithm for feedforward neural network training. applied mathematics and computation , 185(2) , 1026–1037. https://doi .org/10.1016/j .amc.2006.07.025 australasian journal of information systems chattopadhyay, sen & gupta 2018, vol 22, research article meta-analysis of machine learning techniques 22 appendix refer ence. no. reference details (selected sixty articles studied for meta-analysis in the research work) 1 abraham a. (2005). evolutionary computation in intelligent network management. in: ghosh a., jain l.c. (eds) evolutionary computation in data mining. studies in fuzziness and soft computing, vol 163. springer, berlin, heidelberg, https://doi.org/10.1007/3-540-32358-9_9 2 aburomman, a. a., & ibne reaz, m. bin. (2016). a novel svm-knn-pso ensemble method for intrusion detection system. applied soft computing journal, 38, 360–372. https://doi.org/10.1016/j.asoc.2015.10.011 3 ashfaq, r. a. r., wang, x. z., huang, j. z., abbas, h., & he, y. l. (2017). fuzziness based semi-supervised learning approach for intrusion detection system. information sciences, 4 aslahi-shahri, b. m., rahmani, r., chizari, m., maralani, a., eslami, m., golkar, m. j., & ebrahimi, a. (2016). a hybrid method consisting of ga and svm for intrusion detection system. neural computing and applications, 27(6), 1669–1676. https://doi.org/10.1007/s00521-015-1964-2 5 banković, z., stepanović, d., bojanić, s., & nieto-taladriz, o. (2007). improving network security using genetic algorithm approach. computers and electrical engineering, 33(5–6), 438–451. https://doi.org/10.1016/j.compeleceng.2007.05.010 6 chen, w. h., hsu, s. h., & shen, h. p. (2005). application of svm and ann for intrusion detection. computers & operations research, 32(10), 2617-2634. doi:10.1016/j.cor.2004.03.019. 7 chung, y. y., & wahid, n. (2012). a hybrid network intrusion detection system using simplified swarm optimization (sso). applied soft computing, 12(9), 3014– 3022. https://doi.org/10.1016/j.asoc.2012.04.020 8 creech, g., & hu, j. (2014). a semantic approach to host-based intrusion detection systems using contiguousand discontiguous system call patterns. ieee transactions on computers, 63(4), 807–819. https://doi.org/10.1109/tc.2013.13 9 damopoulos, d., menesidou, s. a., kambourakis, g., papadaki, m., clarke, n., & gritzalis, s. (2012). evaluation of anomaly-based ids for mobile devices using machine learning classifiers. security and communication networks, 5(1), 3–14. https://doi.org/10.1002/sec.341 10 de la hoz, e., de la hoz, e., ortiz, a., ortega, j., & prieto, b. (2015). pca filtering and probabilistic som for network intrusion detection. neurocomputing, 164, 71–81. https://doi.org/10.1016/j.neucom.2014.09.083 11 elbasiony, r. m., sallam, e. a., eltobely, t. e., & fahmy, m. m. (2013). a hybrid network intrusion detection framework based on random forests and weighted kmeans. ain shams engineering journal, 4(4), 753–762. https://doi.org/10.1016/j.asej.2013.01.003 12 elhag, s., fernández, a., bawakid, a., alshomrani, s., & herrera, f. (2015). on the combination of genetic fuzzy systems and pairwise learning for improving detection rates on intrusion detection systems. expert systems with applications, 42(1), 193–202. https://doi.org/10.1016/j.eswa.2014.08.002 australasian journal of information systems chattopadhyay, sen & gupta 2018, vol 22, research article meta-analysis of machine learning techniques 23 refer ence. no. reference details (selected sixty articles studied for meta-analysis in the research work) 13 feng, w., zhang, q., hu, g., & huang, j . x. (2014). mining network data for intrusion detection through combining svms with ant colony networks. future generation computer systems , 37, 127–140. https://doi .org/10.1016/j .future.2013.06.027 14 fisch, d., hofmann, a., & sick, b. (2010). on the versatility of radial basis function neural networks: a case study in the field of intrusion detection. information sciences, 180(12), 2421–2439. https://doi.org/10.1016/j.ins.2010.02.023 15 ganapathy, s., kulothungan, k., yogesh, p., & kannan, a. (2012). a novel weighted fuzzy c -means clustering based on immune genetic algorithm for intrusion detection. in procedia engineering (vol. 38, pp. 1750–1757). https://doi.org/10.1016/j.proeng.2012.06.213 16 grediaga, á., ibarra, f., garcía, f., ledesma, b., & brotóns, f. (2006). application of neural networks in network control and information security. in advances in neural networks isnn 2006 (pp. 208–213). https://doi.org/10.1007/11760191_31 17 han, s.-j., & cho, s.-b. (2006). evolutionary neural networks for anomaly detection based on the behavior of a program. ieee transactions on systems, man, and cybernetics. part b, cybernetics : a publication of the ieee systems, man, and cybernetics society, 36(3), 559–570. https://doi.org/10.1109/tsmcb.2005.860136 18 hoang, x. d. , hu, j. , & bertok, p. (2009). a program-based anomaly intrusion detection scheme using multiple detection engines and fuzzy inference. journal of network and computer applications , 32(6), 1219–1228. https://doi.org/10.1016/j . jnca.2009.05.004 19 horng, s. j., su, m. y., chen, y. h., kao, t. w., chen, r. j., lai, j. l., & perkasa, c. d. (2011). a novel intrusion detection system based on hierarchical clustering and support vector machines. expert systems with applications, 38(1), 306–313. https://doi.org/10.1016/j.eswa.2010.06.066 20 hu, w., hu, w., & maybank, s. (2008). adaboost-based algorithm for network intrusion detection. ieee transactions on systems, man, and cybernetics, 38(2), 577– 583. https://doi.org/10.1109/tsmcb.2007.914695 21 hu, w., hu, w., & maybank, s. (2008). adaboost-based algorithm for network intrusion detection. ieee transactions on systems, man, and cybernetics, 38(2), 577– 583. https://doi.org/10.1109/tsmcb.2007.914695 22 jazzar, m., & jantan, a. (2008). an approach for anomaly intrusion detection based on causal knowledge-driven diagnosis and direction. studies in computational intelligence, 149, 39–48. https://doi.org/10.1007/978-3-540-70560-4_4 23 jiang, h., & ruan, j. (2009). the application of genetic neural network in network intrusion detection. journal of computers, 4(12), 1223–1230. https://doi.org/10.4304/jcp.4.12.1223-1230 24 jiang, s., song, x., wang, h., han, j. j., & li, q. h. (2006). a clustering-based method for unsupervised intrusion detections. pattern recognition letters, 27(7), 802–810. https://doi.org/10.1016/j.patrec.2005.11.007 australasian journal of information systems chattopadhyay, sen & gupta 2018, vol 22, research article meta-analysis of machine learning techniques 24 refer ence. no. reference details (selected sixty articles studied for meta-analysis in the research work) 25 khan, l., awad, m., & thuraisingham, b. (2007). a new intrusion detection system using support vector machines and hierarchical clustering. the vldb journal, 16(4), 507–521. https://doi.org/10.1007/s00778-006-0002-5 26 kim, g., lee, s., & kim, s. (2014). a novel hybrid intrusion detection method integrating anomaly detection with misuse detection. expert systems with applications, 41(4 part 2), 1690–1700. https://doi.org/10.1016/j.eswa.2013.08.066 27 kuang, f., xu, w., & zhang, s. (2014). a novel hybrid kpca and svm with ga model for intrusion detection. applied soft computing journal, 18, 178–184. https://doi.org/10.1016/j.asoc.2014.01.028 28 kumar, g., & kumar, k. (2015). a multi-objective genetic algorithm based approach for effective intrusion detection using neural networks. in intelligent methods for cyber warfare (pp. 173–200). https://doi.org/10.1007/978-3-319-08624-8_8 29 arun raj kumar, p., & selvakumar, s. (2013). detection of distributed denial of service attacks using an ensemble of adaptive and hybrid neuro-fuzzy systems. computer communications , 36(3), 303–319. https://doi .org/10.1016/j .comcom.2012.09.010 30 lei, j. z., & ghorbani, a. a. (2012). improved competitive learning neural networks for network intrusion and fraud detection. neurocomputing, 75, 135–145. https://doi.org/10.1016/j.neucom.2011.02.021 31 li, y., wang, j. l., tian, z. h., lu, t. b., & young, c. (2009). building lightweight intrusion detection system using wrapper-based feature selection mechanisms. computers and security, 28(6), 466–475. https://doi.org/10.1016/j.cose.2009.01.001 32 liao, n., tian, s., & wang, t. (2009). network forensics based on fuzzy logic and expert system. computer communications, 32(17), 1881–1892. https://doi.org/10.1016/j.comcom.2009.07.013 33 luo, b., & xia, j. (2014). a novel intrusion detection system based on feature generation with visualization strategy. expert systems with applications, 41(9), 4139– 4147. https://doi.org/10.1016/j.eswa.2013.12.048 34 mabu, s., chen, c., lu, n., shimada, k., & hirasawa, k. (2011). an intrusion-detection model based on fuzzy class-association-rule mining using genetic network programming. ieee transactions on systems, man, and cybernetics, part c: applications and reviews , 41(1) , 130–139. https://doi.org/10.1109/tsmcc.2010.2050685 35 mitrokotsa, a., & douligeris, c. (2005). detecting denial of service attacks using emergent self-organizing maps. proceedings of the fifth ieee international symposium on signal processing and information technology, 2005., 375–380. https://doi.org/10.1109/isspit.2005.1577126 36 njogu, h. w., jiawei, l., & kiere, j. n. (2013). network specific vulnerability based alert reduction approach. security and communication networks, 6(1), 15–27. https://doi.org/10.1002/sec.520 australasian journal of information systems chattopadhyay, sen & gupta 2018, vol 22, research article meta-analysis of machine learning techniques 25 refer ence. no. reference details (selected sixty articles studied for meta-analysis in the research work) 37 orfila, a., estevez-tapiador, j. m., & ribagorda, a. (2009). evolving high-speed, easy-to-understand network intrusion detection rules with genetic programming. in lecture notes in computer science (including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics) (vol. 5484 lncs, pp. 93–98). https://doi.org/10.1007/978-3-642-01129-0_11 38 özyer, t., alhaj j , r., & barker, k. (2007). intrusion detection by integrating boosting genetic fuzzy classifier and data mining criteria for rule pre-screening. journal of network and computer applications , 30(1) , 99–113. https:/ /doi .org/10.1016/j . jnca.2005.06.002 39 pinzón, c. i., de paz, j. f., herrero, á., corchado, e., bajo, j., & corchado, j. m. (2013). idmas-sql: intrusion detection based on mas to detect and block sql injection through data mining. information sciences, 231, 15–31. https://doi.org/10.1016/j.ins.2011.06.020 40 powers, s. t., & he, j. (2008). a hybrid artificial immune system and self organising map for network intrusion detection. information sciences, 178(15), 3024–3042. https://doi.org/10.1016/j.ins.2007.11.028 41 peddabachigari, s., abraham, a., grosan, c., & thomas, j. (2007). modeling intrusion detection system using hybrid intelligent systems. journal of network and computer applications, 30(1), 114–132. https://doi.org/10.1016/j.jnca.2005.06.003 42 ramasubramanian, p., & kannan, a. (2006). a genetic-algorithm based neural network short-term forecasting framework for database intrusion prediction system. soft computing, 10(8), 699–714. https://doi.org/10.1007/s00500-005-0513-9 43 revathi, s., & malathi, a. (2014). multi-tier framework using sugeno fuzzy inference system with swarm intelligence techniques for intrusion detection. indian journal of science and technology, 7(9), 1437–1443. 44 sangkatsanee, p., wattanapongsakorn, n., & charnsripinyo, c. (2011). practical real-time intrusion detection using machine learning approaches. computer communications, 34(18), 2227–2235. https://doi.org/10.1016/j.comcom.2011.07.001 45 sani, r. a., & ghasemi, a. (2015). learning a new distance metric to improve an svm-clustering based intrusion detection system. in proceedings of the international symposium on artificial intelligence and signal processing, aisp 2015 (pp. 284–289). https://doi.org/10.1109/aisp.2015.7123497 46 shamshirband, s., patel, a., anuar, n. b., kiah, m. l. m., & abraham, a. (2014). cooperative game theoretic approach using fuzzy q-learning for detecting and preventing intrusions in wireless sensor networks. engineering applications of artificial intelligence, 32, 228–241. https://doi.org/10.1016/j.engappai.2014.02.001 47 sheikhan, m., & jadidi, z. (2014). flow-based anomaly detection in high-speed links using modified gsa-optimized neural network. neural computing and applications, 24(3–4), 599–611. https://doi.org/10.1007/s00521-012-1263-0 48 shon, t., & moon, j. (2007). a hybrid machine learning approach to network anomaly detection. information sciences, 177(18), 3799–3821. https://doi.org/10.1016/j.ins.2007.03.025 australasian journal of information systems chattopadhyay, sen & gupta 2018, vol 22, research article meta-analysis of machine learning techniques 26 refer ence. no. reference details (selected sixty articles studied for meta-analysis in the research work) 49 sivatha sindhu, s. s., geetha, s., & kannan, a. (2012). decision tree based light weight intrusion detection using a wrapper approach. expert systems with applications, 39(1), 129–141. https://doi.org/10.1016/j.eswa.2011.06.013 50 stopel, d., moskovitch, r., boger, z., shahar, y., & elovici, y. (2009). using artificial neural networks to detect unknown computer worms. neural computing and applications, 18(7), 663–674. https://doi.org/10.1007/s00521-009-0238-2 51 subbulakshmi, t., mercy shalinie, s., suneel reddy, c., & ramamoorthi, a. (2010). detection and classification of ddos attacks using fuzzy inference system. in communications in computer and information science (vol. 89 ccis, pp. 242–252). https://doi.org/10.1007/978-3-642-14478-3_25 52 tajbakhsh, a., rahmati, m., & mirzaei, a. (2009). intrusion detection using fuzzy association rules. applied soft computing, 9(2), 462–469. https://doi.org/10.1016/j.asoc.2008.06.001 53 tong, x., wang, z., & yu, h. (2009). a research using hybrid rbf/elman neural networks for intrusion detection system secure model. computer physics communications, 180(10), 1795-1801.doi:10.1016/j.cpc.2009.05.004. 54 toosi, a. n., & kahani, m. (2007). a new approach to intrusion detection based on an evolutionary soft computing model using neuro-fuzzy classifiers. computer communications, 30(10), 2201–2212. https://doi.org/10.1016/j.comcom.2007.05.002 55 tsang, c. h., kwong, s., & wang, h. (2007). genetic-fuzzy rule mining approach and evaluation of feature selection techniques for anomaly intrusion detection. pattern recognition, 40(9), 2373–2391. https://doi.org/10.1016/j.patcog.2006.12.009 56 wang, g., hao, j., ma, j., & huang, l. (2010). a new approach to intrusion detection using artificial neural networks and fuzzy clustering. expert systems with applications, 37(9), 6225–6232. https://doi.org/10.1016/j.eswa.2010.02.102 57 yin, c., tian, s., huang, h., & he, j. (2005). applying genetic programming to evolve learned rules for network anomaly detection. advances in natural computation, first international conference, icnc 2005, proceedings, part iii, 3612, 323– 331. https://doi.org/doi:10.1007/11539902_38 58 zanero, s. (2005). analyzing tcp traffic patterns using self organizing maps. image analysis and processing iciap 2005, proceedings, 3617, 83–90. https://doi.org/10.1007/11553595_10 59 zhang, c., jiang, j . , & kamel, m. (2005). intrusion detection using hierarchical neural networks. pattern recognition letters , 26(6), 779– 791. https://doi.org/10.1016/j .patrec.2004.09.045 60 zhang, j.-r., zhang, j., lok, t.-m., & lyu, m. r. (2007). a hybrid particle swarm optimization–back-propagation algorithm for feedforward neural network training. applied mathematics and computation, 185(2), 1026–1037. https://doi.org/10.1016/j.amc.2006.07.025 table 6: studied top works in the field of intrusion detection australasian journal of information systems chattopadhyay, sen & gupta 2018, vol 22, research article meta-analysis of machine learning techniques 27 copyright: © 2018 chattopadhyay, sen & gupta. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. paper title (paper title style) australasian journal of information systems ong, alahakhoon & shanks 2015, vol. 19, pp. s1-s3 business analytics editorial editorial for the special issue on business analytics applications kok-leong ong la trobe business school la trobe university kok-leong.ong@latrobe.edu.au damminda alahakhoon la trobe business school la trobe university d.alahakoon@latrobe.edu.au graeme shanks dept. computing and information systems university of melbourne gshanks@unimelb.edu.au the need for organisations to make sense of their data has never changed. data has always played a role in helping organisations improve their operational efficiencies, maintain customer relationships, and seek maximum returns through different targeted strategies. from the early days of aggregated data reports, to dashboards and business intelligence, the complexity in the volume of data to analyse and the need for complex insights to influence decision-making has organisations shifting their interest and focus on analytics. analytics refer to the use of a variety of computer technologies, both in terms of hardware and software, to analyse massive amount of data for complex data patterns and data characteristics. business analytics broadly refers to the application and customisation of these techniques to effect the positive outcomes sought by organisations. the underlying algorithms and techniques are based on data mining and machine learning but business analytics focus on the practical application of these. in addition to the technological aspects of analytics, “business analytics” also builds on the foundation of information systems theory that enables the raw technology to be meaningfully contextualised and applied to solve the real-world business problem. in this special issue, we are delighted to have a diverse set of papers covering a broad spectrum of business analytics as an emerging discipline. starting from the information systems perspective, we have two complementary papers. the first by cosic et. al., proposed a business analytics capability framework designed from a resource-based view and a thematic content analysis of the existing business analytics literature. the aim of the framework is to enable academics and practitioners understand an organisation’s business analytics initiatives and to determine the relative importance of business analytics within the organisation. the proposal is complemented by yeoh et. al., where a bi and analytics diagnostic framework was proposed to allow organisations evaluate the effectiveness of their bi and analytics initiatives. a key outcome from yeoh et. al. is an online toolkit that includes a comprehensive survey instrument to allow other organisations to understand their own bi and analytics readiness. the next two papers discussed business analytics from the computer science perspective, focusing more on the underlying algorithm that enable business analytics. alahakoon suggested at a “new user” model for interactive product retrieval aimed at solving issues in product recommendation on e-buying systems. the proposed solution allows the recommender to avoid null retrievals, over-retrievals and unsatisfactory retrieval of product recommendations thus, improving the overall e-buying experience. however to get customer experience right will require more than just good recommendations. it’s also about understanding what customers feel about your product and that’s where text analytics come into play. the paper by nathawitharana et. al. tackled this issue by a new algorithm that improves on term overlap detection. with improved accuracy in term detection, text analysis s1 australasian journal of information systems ong, alahakhoon & shanks 2015, vol. 19, pp. s1-s3 business analytics editorial such as clustering will be more effective. in the paper, nathawitharana et. al. showed that their gsom hierarchical clustering algorithm are able to achieve good intra-cluster difference for distant categories when the their new algorithm is applied. between these two perspectives, the remaining papers shared how various problems are solved through the effective use of analytics and along the way, present application exemplars of business analytics. asefi et. al. described how their optimisation model enables the cost of waste management at a municipal in nsw, australia to be minimised by reducing the total cost of transportation and facility establishment for different types of waste. singh and rumantir analysed eftpos data using both clustering and classification to derive a set of business rules that allows them to easily recognise different types of retailers and therefore, allow tailored strategies to be applied by the banks when marketing eftpos products to retailers. although minimising cost and maximising market opportunities is typical of any business, analytics could also be applied to save lives such as the work reported in shahparvari et. al. who used a multi-objective optimisation model to help create a decision support system for bush fire evacuation, shelter assignment and jit allocation of resources to enhance the capacity of fire services agencies. lastly, no discussion of business analytics would be complete without the mention of “big data”, which was presented in the paper by de silva. this case study paper unpacks the inertia facing the healthcare industry in tapping into the potential of its big data and suggests a solution to improve the big data analytics adoption rate by discussing the solution from the diabetes screening case. we like to end this editorial by mentioning the reviewers who worked tirelessly to the timeframes of this special issue, many whom have gone the extra mile to get reviews completed on very short notice. we are deeply appreciative of their contributions and they are formerly acknowledged here. • siew fan wong, sunway university • hassan hijazi, nicta, optimisation research group • greg richards, university of ottawa • michael gendron, central connecticut state university • prem chethri, rmit university • asanka foneska, equipsuper pty ltd • md zahidul islam, charles stuart university • masoud mohammadian, university of canberra • kali nepal, deakin university • delwar akbar, cquniversity australia • mahabub rahim, monash university • jayantha rajapaksa, monash university • don kerr, the university of the sunshine coast • sumith matharage, crown entertainment • yee ling boo, rmit university • qing huang, la trobe university • grace rumantir, monash university • ta-tao chuang, gonzaga university • maura atapattu, queensland university of technology • susan keller, la trobe university s2 australasian journal of information systems ong, alahakhoon & shanks 2015, vol. 19, pp. s1-s3 business analytics editorial • martin dick, rmit university • daswin de silva, la trobe university • ravi seethamraju, the university of sydney • tim turner, unsw canberra at the adfa lastly, we hope you enjoy the papers in this special issue. we had a wonderful time putting the papers together and none of these is possible without the on-going support of the editor in chief, associate professor john lamp, whom we sincerely thank for accepting this special issue proposal and the on-going support rendered throughout. copyright: © 2015 ong, alahakhoon & shanks. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. s3 paper title (paper title style) australasian journal of information systems cataldo & mcqueen 2015, vol. 19, selected papers from acis it alignment in smes it alignment in smes: should it be with strategy or process? alejandro cataldo school of business systems engineering – department of system engineering universidad de talca – universidad de atacama talca, chile email: acataldo@utalca.cl robert mcqueen department of management systems university of waikato hamilton, new zealand email: bmcqueen@waikato.ac.nz abstract for many years researchers have been concerned about the alignment of information technology (it) in businesses. most of the published research has been focused on understanding the alignment of it with strategy, but a more recent view is that it alignment with processes would be better, especially for smes. this research presents the two different approaches to measuring it alignment, then uses the case of a small dental clinic to contribute to understanding these two alternative approaches. the results showed that both strategic alignment and process alignment were important, but strategic guidelines set the framework for the use of it in the processes. these results contradict the claims of researchers who defend the process approach. this work has implications for practitioners because it again highlights the importance of it in a strategic context in smes. this research also contributes to the theory of it alignment in smes, and we hope this will encourage other researchers to study which approach is best suited for this type of business. keywords: it alignment, process alignment, strategic alignment, smes. 1 introduction a concern that researchers and practitioners have had for many years has been how companies can effectively align information technology (it) with business goals (johansson and sudzina 2010). there is strong evidence that a proper it alignment improves the performance of the company (chan and reich 2007), and researchers have sought different ways to tackle the problem of it alignment in business. one aspect that some researchers have studied is whether the it alignment theory is valid for small and medium enterprises (sme) (chan and reich 2007). this concern has been raised primarily because the study of it in business has had a bias towards larger firms (burgess 2002; cragg and tagliavini 2007; cragg et al. 2007). some researchers have tried validating existent theory in smes, while others have proposed new frameworks exclusively designed for the particular conditions of smaller companies (blili and raymond 1993; levy and powell 2005). in recent years, some researchers have suggested that in smes, it alignment should be studied with respect to processes, rather than strategy (cragg and tagliavini 2007; tagliavini et al. 2004). these researchers suggest that smes often lack a defined strategy against which it can be aligned and the operational focus of smes makes it likely to be aligned to the processes the company implements (tagliavini et al. 2004). recent debate questions whether it alignment in smes should continue to be studied with respect to strategy as has been traditionally been the case, or whether it alignment should be analysed against processes. to contribute to this debate, we studied the case of a dental clinic to determine how alignment among it, strategy and processes was reached. the approach used was qualitative. two 1 australasian journal of information systems cataldo & mcqueen 2015, vol. 19, selected papers from acis it alignment in smes researchers collected data through in-depth interviews to key informants and work observation. both participants and an external researcher validated the data. this article presents the case and reports the main findings of the study. the remainder of this paper is divided into five sections, which are the literature review, the methodology used, the main results of the study, the results compared to previous literature, and the conclusions and limitations. 2 literature review this section summarizes the most relevant published literature related to this study. this section is divided in four subsections, which are research about it alignment, the traditional approach of it alignment in smes, the new approach of it alignment based in processes, and the research question which guides the remainder of this study. 2.1 it alignment one of the main concerns in the field of technological alignment is to clearly define the meaning of it alignment. for example, chan and reich (2007) quoting reich and benbasat defined it alignment as the degree to which the mission, goals and plans contained in the business strategy is shared and supported by the it strategy. henderson and venkatraman (1999) stated that alignment is the fit and integration between business strategy, it strategy, business infrastructure and it infrastructure. campbell (2005) concluded that alignment is the business and it working together. different definitions have also resulted in different terms to refer to it alignment. for example, some authors refer to alignment as “fit” (chan and reich 2007; henderson and venkatraman 1999), while others use the term “connection” (reich 1993). in turn, other terms used are “integration”, “fusion” and “congruence” among others (chan and reich 2007). although they are used almost interchangeably, some cases represent small differences between themself. for example, the term “fit” is associated to a quantitative line of research about alignment and, under this point of view, is linked to mathematical models (venkatraman 1989a; venkatraman 1989b). currently, however, it seems not to be under discussion that the term most dominant is “alignment” (chan and reich 2007). researchers have also developed models of it alignment. these alignment models are intended to describe how different elements of an organization interact to achieve an effective alignment. as consequence, they try to represent the it alignment process from a holistic point of view. the first model of alignment was developed by mit and is known as mit90. this model argues that revolutionary changes involving it investment bring substantial rewards if and only if, the key elements: business strategy, technology, structure, management processes and individuals and roles, remain aligned (chan and reich 2007). inspired by the mit90 model, henderson and venkatraman (1999) developed the strategic alignment model (sam), which is perhaps the most widely cited of all models. sam is based on four key interrelated domains: (1) business strategy, (2) it strategy, (3) infrastructure and organizational processes and (4) is processes and infrastructure. each domain consists of three components. the components of the first two domains comprise scope, distinctive competencies and governance (in the domains of business and it strategy), and the components of the second two domains consist of: infrastructure, processes and skills (in the domains of administration and is). according to sam, alignment is achieved in a bivariate fashion: between business strategy and it and between organizational infrastructure and information systems (henderson and venkatraman 1999). however, despite its popularity, sam has some limitations. for example, it is not easy try to use sam to understand how a company can align its technological resources with their strategic goals (avison et al. 2004). in fact, several works have been presented to operationalize this model (avison et al. 2004; chan and reich 2007; coleman and papp 2006). 2 australasian journal of information systems cataldo & mcqueen 2015, vol. 19, selected papers from acis it alignment in smes that is one reason why other researchers have sought to expand sam or propose alternative models. for example, baets (1992) developed another model by arguing that in many organizations there is no explicit strategy or it is not well known to all members. kearns and lederer (2003) developed the factors that influence the alignment, and argued that it was necessary to distinguish between the alignment between business plan/is plan (bp-isp) and is plan/business plan (isp-bp). using the resource-based view, they determined that the main factor to achieve bp-isp alignment is the role played by the cio (chief information officer) in the organization. a significant contribution to the theory of alignment was the development of the strategic orientation of business enterprises (strobe) and strategic orientation of information system (strois) of venkatraman (1989b) and chan (chan et al. 1997a; chan et al. 1997b), respectively. strobe was developed to determine the strategic behaviour of an enterprise using six dimensions. later, using an improved strobe of eight dimensions, chan developed strois as a complementary tool to evaluate the strategic usage of it in a company. together these instruments have been widely used by researchers to quantify the alignment of it with business strategy (byrd et al. 2006; chan and reich 2007; hale and cragg 1996). 2.2 it alignment in smes: the traditional approach based on strategy since many of the models have been developed for large companies, several researchers have focused on determining validity in the context of small and medium enterprises. for example, hale and cragg (1996) adapted and applied the strobe/strois in new zealand smes. to measure the it alignment, they used bivariate analysis: matching and moderation. the authors applied the instruments to eight companies to validate them. on the other hand, chan et al. (2006) examined the factors affecting it alignment in a group of business and academic organizations. in their study the authors analysed both large and small organizations. one of their main conclusions was that the organizational size affected alignment it in business firms but not in academic institutions. more recently, johansson and sudzina (2010) examined whether alignment is a good or bad influence on company performance. to do this they surveyed a group of 112 slovak enterprises, both large firms and smes. the alignment was measured as a perception of the respondents. the main conclusion was there was a significant difference between high it alignment and companies with high performance. others have developed approaches just for smes, such as blili and raymond (1993). they argued smes have “specificities” that distinguish them from larger firms, so smaller companies need to be studied separately. they developed a framework to align it with the business proposing a topographic view of the application of it. their framework was based on porter’s competitive forces, the value chain and the critical success factors (blili and raymond 1993). another example of frameworks exclusively designed for smes is the work of levy and powell (2005). they proposed a new model based on the path of alignment depending on the type of company. their model classifies smes into four types of companies according to the scope of action: (1) efficiency, (2) co-ordination, (3) collaboration, and (4) repositioning. they defined two criteria to classify an sme: the strategic focus and the number of customers. the model helps to align it using different paths specified by the authors (levy and powell 2005; levy et al. 2001) 2.3 it alignment in smes: the new approach based on processes another group of researchers of it alignment in smes have indicated that in this type of organization alignment should be evaluated with respect to processes. several reasons are put forward to support this position. the first reason is the small numbers of studies of it alignment in smes compared to those made in large companies. for example, gutierrez et al. (2006) developed an instrument to measure alignment at the tactical and operational level. they suggested that much of the 3 australasian journal of information systems cataldo & mcqueen 2015, vol. 19, selected papers from acis it alignment in smes research on alignment was focused only on the strategic dimension, but it was not the only way to align an organization. based on the sam model, they constructed a questionnaire that evaluated the degree of maturity at strategic, tactical and operational levels in a sme. second, the informality of the roles and inter-functional nature of the tasks suggests that a process-based approach is best suited to the conditions of smes. in other words, the scarcity of resources, lack of expertise and focus on the operational questions the traditional view about it alignment in smes (ravarini et al. 2002; tagliavini et al. 2004). for example, tagliavini et al. (2004) say “in particular, a process-based approach seems to properly fit smes, where employees carry out inter-functional tasks and do not precisely define their roles”. this has led these researchers to develop instruments called “check-up” to assess the alignment of it with a set of standard processes (ravarini et al. 2002; tagliavini et al. 2004). thirdly, cragg et al. (2007) argued that a process approach probably is more important than a strategic one in smes, but that approach is less understood and researched. they claimed “this suggests that operational alignment in smes could be at least as important as strategic alignment, and possibly of greater importance. yet operational alignment in smes is poorly understood and an under-researched topic”. as a consequence, they measured the relationship between it alignment against processes and it success. the authors took the framework of pcf (process classification framework) of the apqc (apqc 2008). the apqc is an independent business model that allows industry to identify the activities of a process in a generic enterprise. in their study, they firstly measured the alignment between it and processes and then evaluated that alignment with it success. finally and most recently, cragg and mills (2011), continuing with the study of process-based alignment, assessed how well it supports business processes in a group of smes in new zealand. their main conclusion was that “the business process view provides a useful lens for studying it support. there seems to be much potential for smes to improve their it support for some business processes” however, recent evidence has emerged to show that the relationship between it alignment and processes in an sme may still be influenced by the strategy of the company. cataldo et al. (2012) studying a group of latin american companies, concluded that both the strategic orientation of smes and their processes influenced each other in relation to it success. their findings suggest that the relationship between process-level alignment and it success may be spurious and that strategic alignment may be the confounding variable and that both it success and process-level alignment could be driven by strategic alignment. as a possible explanation, they speculated that despite the lack of formal and explicit long term planning in smes, there always seems to be an implicit strategy that shapes the way that it is used in processes. similar explanations have been proposed by other researchers such as avison et al. (1998) and levy and powell (2005). 2.4 research question from the foregoing review of previous research, the following research question has been constructed to guide this research: rq: what is the driver of the alignment of it: strategy or processes? 3 methodology we studied a small dental clinic as a single exploratory study to better understand the scope of the research question, and lay the groundwork for further studies. the study of a sme like a dental clinic has the advantage that this type of business is very autonomous and almost all processes run independently, so they can be analysed as a completely autonomous and quasiisolated unit. other authors already have researched similar kind of business (davidson and heslinga 2006). 4 australasian journal of information systems cataldo & mcqueen 2015, vol. 19, selected papers from acis it alignment in smes the research strategy used was case studies. for data collection we used semi-structured interviews and work observation. both the strategy and techniques of data collection and analysis are part of what is known as qualitative research methods (bryman and bell 2007). the decision to use qualitative research methods depends on the questions raised. qualitative research can be effective for questions such as “what is happening?” and “how is it happening?” (bluhm et al. 2010; lee et al. 1999; lee 1999). qualitative research is unique in its ability to conduct problems of description, explanation and interpretation of a phenomenon. it is also essential to discover deeper processes in individuals, teams and organizations, as well as understand how these processes occur in time. additionally, qualitative research is critical to gain an understanding of the experience of individuals and how they interpret (bluhm et al. 2010). for reliability and validity of the study we followed the recommendations made by creswell (2009) and (yin 2009). the respondents were the owner of the clinic and his receptionistadministrator because both are key users of the systems. each interview lasted between two and three hours. all sessions were videotaped and digital audio recorded. also, interviewers took field notes. we used an interview protocol to ensure reliability. collected data were also triangulated with data collected through observation of work and documents. as an important part of the interviews were based on how the key informants executed their activities, some parts of the interviews were later transcribed and almost all data were analysed directly using qualitative data analysis software. the authors acknowledge a methodological limitation on this procedure. to ensure the validity of the findings, two people conducted the interviews: a principal researcher and an assistant. after each interview, interviewers asked for feedback from participants. additionally, a person outside the group of researchers who had access to interviews also validated the analysis. 4 results this section is divided into two subsections. the first subsection briefly describes the case. the second subsection presents the analysis performed. to maintain confidentiality of identities, the names used are fictitious. 4.1 brief description of the case oceandental is a dental clinic in chile which employs five staff. the clinic was founded 2010 by a dentist specializing in endodontic services, who we will call patrick. patrick has nearly 20 years of experience as a dentist, worked 19 years in a family dental clinic and also had an internet company that was forced to close after the dotcom crisis. despite its failure, this experience provided him with two fundamental lessons: firstly, he acknowledges that while he had his company, he learned everything he knows about strategic management, and secondly, during that period he became convinced that it could give him a competitive advantage over his competitors. after he paid all the debts of his internet business, patrick opened oceandental that only treats endodontic patients. currently, the clinic has a good reputation and attracts patients from the entire country. a feature of the business is that other dentists or large clinics send their patients to be treated. a key role is played by the administrative assistant and receptionist, who we will call soledad. she is a dental assistant with 15 years of experience. she has been in that position since the opening of the clinic but she had also previously worked with patrick. soledad is a skilled it worker who learned computing while working at an international electronics company where she worked six years. as assistant and receptionist, she is responsible to book and confirm the appointments’ patients, receive their payments, and make daily reports to patrick about income. in addition, she must do banking and administrative tasks and when a patient finishes 5 australasian journal of information systems cataldo & mcqueen 2015, vol. 19, selected papers from acis it alignment in smes the treatment, she has to print and deliver the report to the patient and send it via e-mail to her dentist. according to patrick, one of the characteristics of the clinic is that it makes extensive use of dental and information technologies. in fact, the company has two major systems: dentalvision, which is a dental management system (dms) used to store all the information of the clinic, and clinicsystem which is software to store the radiographs. clinicsystem is integrated with the digital dental x-ray equipment used in the clinic. the integration between the data generated by equipment and the software is achieved through ad-hoc applications developed by patrick and freelance programmers hired by him. 4.2 results almost all of oceandental’s internal processes are computerized, and the clinic uses only paper documents for its external transactions, aiming for a paperless environment. like any small dental clinic, oceandental’s processes are highly centralized in the receptionist (see figure 1). their information is stored on an external server, but soledad and patrick have access. only the daily income accountability process is manual. although dentalvision generates income reports, soledad prepares a daily report that is delivered to patrick. figure 1: diagram of main information flow in oceandental: processes are highly centralized in soledad. 4.3 aligning it with business processes according to lockamy and smith (1997), a process is a sequence of pre-defined activities executed to achieve a pre-specified type or range of results. consequently, it aligned with processes should effectively and efficiently support this sequence of activities. it was found that it was strongly aligned with the processes of the clinic. to illustrate this, we describe a part of the patient care process, specifically the sub-process to take x-rays and deliver reports of treatment. first, the x-rays are taken using digital equipment. this equipment is connected directly to the dentist's computer and he inserts the x-ray in a 6 australasian journal of information systems cataldo & mcqueen 2015, vol. 19, selected papers from acis it alignment in smes customized form. this form is filled using a word macro developed by patrick and freelance programmers who he hired. this form is saved in pdf format on the file server of the clinic and the clinicsystem. when the patient finishes his treatment, soledad accesses the report of treatment on her computer, prints it out and delivers it to the patient in a folder. in addition, she also e-mails the report to the doctor who referred the patient. several additional examples of it alignment with processes were found. for example, the use of macros for creating forms has reduced the processing time of reports of patients from thirty minutes to just four minutes. another example is the process of admission of new patients. soledad told us how they entered each new patient: “patient arrives with her x-ray then i scan it so that the doctor can see it on his computer. if i don’t do that, the doctor will not be able to work, then ... i mean, he can still work but he won’t have something important which is the x-ray” 4.4 strategic alignment there are several definitions of what is strategy, some of them complementary. for example, for porter (2001), strategy is the creation of a unique and valuable position, involving a different set of activities. in that sense, he describes a good strategy as having (a) to consider a long timeframe (10+ years), (b) to define one or more types of positions (based on variety, needs or access) and (c) to establish the trade-off between alternative directions, and what not to do. hax (2010) reported that the classic definition of strategy, “to achieve sustainable competitive advantage” is dangerous. for him a good strategy has to be focused on the customer rather than on competition as in porter's definition. so, for hax the strategy should create the conditions so that a company (a) achieves long-term sustainable financial returns, (b) creates activities not imitable, (c) avoids commoditizing their products or services, and (d) looks at the customer so as to be able to attract her, satisfy her and retain her. for hax (2010), the most sustainable strategy is one in which the company succeeds in creating the strongest bond with the customer. in that sense, this is achieved when the lock-in is obtained with the client, i.e., switching costs for the customer are so high that she decides to stay with the company. when a company creates a lock-in, its competitors are in a state of lockout, and the whole system begins to work for the company. although oceandental did not have a written strategy, during the interview with patrick we could identify many of the features that, according to hax and porter, must be part of a strategy. table 1 presents evidence about oceandental’s strategic characteristics identified during the interview with patrick. alignment between it and strategy requires that it supports one or more of the strategic features identified in table 1. we were able to confirm that several of these strategic features were supported by it (column 3 of table 1). for example, reports that are delivered to patients and their dentist are key for patrick, and therefore the format of the reports and their presentation must distinguish them from the traditional reports that are made by “98% of endodontists” who deliver an “x-ray in an small envelope”. table 1 shows that it plays a key role in the report provided to patients. according to patrick “this is a barrier to entry” for other specialists who would want to do the same. 4.5 strategic it alignment with processes: the chicken or the egg? oceandental is clearly a company whose it is aligned with both the strategy and the processes. however, the question remains what was the driver of the alignment of it: strategy or processes? for oceandental the strategic decisions taken by patrick were a key driver in the incorporation and use of it. for example, he said “that's what i offer: a specialty clinic with the latest technology”. clearly in this case the use of technologies (not just information but also dental 7 australasian journal of information systems cataldo & mcqueen 2015, vol. 19, selected papers from acis it alignment in smes treatment technologies) was a strategic choice taken by patrick, that is, he made that decision before identifying which technologies he would utilize or how it would be done. for patrick the use of technology brought several advantages: (1) improved the customer experience, giving them greater confidence about the quality of service, (2) created a barrier to entry for other dentists (he believes his colleagues still do not use it as intensively as he does), and (3) increased profitability per patient because he is able to charge a higher price than the market average for treatment. this was confirmed by soledad when she said “i think the price is fair…he has all the technology needed”. however, the response of what technologies to use or how to do it depended on the processes and their design. for example, patrick made the decision to create word macros to automate and to do more efficient process to prepare patient reports. another example is the decision of patrick to create a paperless clinic acquiring clinicsystem and dentalvision. this decision was related to design internal processes fully digitized in a paperless environment. soledad said “now everything is digital... i don’t have papers. there aren’t folders with paper, or anything of those things ... all is digital.” in summary, although oceandental did not design a formal strategy, the company showed a high level of strategic direction, which was an orientation where it played a key role. in turn, these guidelines acted as a platform on which the processes that the company would implement were subsequently designed. meanwhile, specific commercial systems and technologies were selected to support these processes. in sum, for patrick alignment between his strategy and it occured prior to the alignment between processes and it. 5 discussion researchers have reported that most smes do not have formal planning processes (cragg et al. 2007; hale and cragg 1996; levy and powell 2005). however, table 1 shows significant evidence that the company has a strategic direction that has guided many of the decisions made by patrick as owner of the clinic. cases like oceandental that have no formal strategy but execute a strategy have been reported in literature (chan and reich 2007; levy and powell 2005; venkatraman 1989b). on the other hand, our results show that there was an alignment between oceandental strategic and processes levels. both types of it alignment are consistent with a large body of published research. for example, chan and reich (2007) discuss the strategic alignment of it, while cragg and mills (2011) present evidence on the alignment of it with processes in smes. our results indicate that both strategic and process alignment of it occurred in oceandental, and that strategy alignment occurred before the process alignment. this is consistent with findings of other researchers. chan and reich (2007) claimed “clearly defined business goals and vision are often first steps in an alignment process.” the results are also consistent with cataldo et al. (2012) indicating a correlation between strategic alignment and processes. also, the relationship between strategy, processes and infrastructure manifested in oceandental is consistent with the sam model, which establishes that alignment is a tripartite relationship between three domains (coleman and papp 2006; henderson and venkatraman 1999). 8 australasian journal of information systems cataldo & mcqueen 2015, vol. 19, selected papers from acis it alignment in smes table 1. strategic features identified in oceandental and its alignment with it. however, the results contradict the findings of some researchers that process alignment is more important than strategic alignment in smes (cragg and tagliavini 2007; cragg et al. 2007; tagliavini et al. 2004). our results show both type of alignment were concurrent in oceandental. perhaps, it can be speculated that process alignment might be more relevant in companies where there is not a formal strategy or strategic guidance from top management. however, in smes where there is a clear strategic orientation (even if not made explicit), the role of strategic alignment may still be just as important as the alignment at the process level. strategic feature patrick’s comment it strategic alignment positions based on variety this kind of clinic doesn’t exist [anywhere else] in santiago. it’s just endodontic. not identified. trade off if someone asks me ‘can you make a dental filling?’ i answer no, you have to go to your dentist. if they ask 'will you do a filling?' i say no, i don’t care to do that. not identified. long-term returns it is attractive because i'm earning more, spending more, [but] the delta is the same as it was 10 months ago [when i started], but at 40% capacity. [if i] grow only 10% more… these amounts will become quite attractive. the visible and intensive use of technology creates in the patient a feeling of confidence that makes them willing to pay more money and not complain about that. non-imitation i can offer more [technology] than my competition. i can offer discounts because we are a group. use of word macros to automate reporting and reduce form filling time from 30 to 4 minutes. noncommoditization the reports are very important because [the patient] pays 400 or 600 dollars for a scan and they take their report and say ‘hey look super expensive but this is my scan!’ for the traditional 98% of endodontists, all the patient gets is an x-ray in an small envelope. delivery of digital information to patients and referring dentists. look-in i used to have only a chair. fully saturated! ... no chance to react. and the reference doctor called me and i said him ‘i can’t treat the patient, sorry’. an endodontist who can’t receive a patient is wrong, because the endodontist attends urgencies! use of website to build confidence in a patient with information about treatments, cv and clinic’s facilities strategic alliances i have a pool of [referring] dentists ... who aren’t endodontists, and when they need to do a endodontcy, ... everyone sends me five, six, ten or fifteen patients per month using e-mail to send digital reports to the patient’s dentist 9 australasian journal of information systems cataldo & mcqueen 2015, vol. 19, selected papers from acis it alignment in smes 6 conclusions there is a broad consensus that it alignment can produce significant improvements in business. however, in the context of smes, this consensus seems to have shades of agreement. while some researchers apply the theory of alignment to both large and small companies, other researchers argue that strategic it alignment theory does not fit with the conditions of smes, and therefore they advocate a process-based analysis rather than a strategic one. our study analysed the case of a small dental clinic with the aim of trying to explore the issue and contribute to the debate on it alignment in smes. our results showed that in the case studied, both strategic alignment and process alignment appeared concurrently and seemed to be interrelated. in the case of the dental clinic, there was a strong alignment of it with the strategy driven primarily by the strong visionary role that the owner has based on the intention to use the latest technologies, both dental and it, in his business. as a result, it strongly supports this vision of the owner of the company so that both strategy and processes have reached a high level of integration with it. the alignment of it with both strategy and processes has been reported in previous studies. as consequence, these results are also consistent with other studies that validate the theory of traditional alignment in smes. however, these results do seem to contradict some recent findings that measurement of it alignment in smes is better based on process than on strategy. in this regard, these findings extend the results of cataldo et al. (2012) who found a correlation between the alignment of it with both strategy and processes. the authors acknowledge two important limitations of this study. firstly, because this research was based on a single case, its external validity is limited, although it follows the recommendations of previous studies about case studies. secondly, the case study's business, geographical and cultural environments are very specialized, leaving open the question of whether the results would be replicable in other conditions. despite the above, this work may have implications for both practitioners and researchers. for practitioners, this study again highlights the importance that it has when it is organized in a strategic context in accordance with the processes of the company. for researchers, the study provides new information on the theory of it alignment in smes and the debate that has been built whether it is best analysed against strategy or processes. we think and hope that this work may encourage other researchers to try to provide additional insight on this problem in different contexts and environments. references apqc. 2008. "process classification framework," apqc, houston, usa. avison, d., jones, j., powell, p., and wilson, d. 2004. "using and validating the strategic alignment model," strategic information systems (13), pp. 223-246. avison, d.e., eardley, w.a., and powell, p. 1998. "suggestions for capturing corporate vision in strategic information systems," omega (26:4), pp. 443-459. baets, w. 1992. "aligning information systems with business strategy," journal of strategic information systems (1:4), pp. 205-213. blili, s., and raymond, l. 1993. "information technology: threats and opportunities for small and medium-sized enterprises," international journal of information management (13:6), pp. 439-448. bluhm, d.j., harman, w., lee, t.w., and mitchell, t.r. 2010. "qualitative research in management: a decade of progress," journal of management studies). bryman, a., and bell, e. 2007. business research methods, (second ed.). oxford university press, usa. 10 australasian journal of information systems cataldo & mcqueen 2015, vol. 19, selected papers from acis it alignment in smes burgess, s. 2002. "managing information technology in small business: challenges and solutions. introduction," in managing information technology in small business: challenges and solutions, s. burgess (ed.). hershey, pa: igi global. byrd, a., lewis, b.r., and bryan, r.w. 2006. "the leveraging influence of strategic alignment on it investment: an empirical examination," information & management (43:3), pp. 308-321. campbell, b. 2005. "alignment: resolving ambiguity within bounded choices," pacis 2005, bangkok, thailand, pp. 54. cataldo, a., mcqueen, r., and hardings, j. 2012. "comparing strategic it alignment versus process it alignment in smes," journal of research and practice in information technology (44:1), pp. 43 57. chan, y., huff, s., barclay, d., and copeland, d. 1997a. "business strategic orientation, information systems strategic orientation, and strategic alignment," information systems research (8:2), pp. 125-150. chan, y., huff, s., and copeland, d. 1997b. "assessing realized information systems strategy," journal of strategic information systems (6:4), pp. 273-298. chan, y., sabherwal, r., and thatcher, j. 2006. "antecedents and outcomes of strategic is alignment: an empirical investigation," ieee transactions on engineering management (51), pp. 27-47. chan, y.e., and reich, b. 2007. "it alignment: what have we learned?," journal of information technology (22:4), pp. 297-315. coleman, p., and papp, r. 2006. "strategic alignment: analysis of perspectives," proceedings of the 2006 southern association for information systems conference, jacksonville, florida, pp. 242-250. cragg, p., and mills, a. 2011. "it support for business processes in smes," business process management journal (17:5), pp. 697-710. cragg, p., and tagliavini, m. 2007. "evaluating information systems alignment in small firms." university of southern queensland, toowoomba, australia, pp. 38-48. cragg, p., tagliavini, m., and mills, a. 2007. "evaluating the alignment of it with business processes in smes," 18th australasian conference on information systems (acis 2007), toowoomba: university of southern queensland, pp. 38-48. creswell, j.w. 2009. research design: qualitative, quantitative, and mixed methods approaches, (third ed.). los angeles: sage publications, inc. davidson, d.e., and heslinga, d.d. 2006. "bridging the it adoption gap for small physician practices: an action research study on electronic health records," information systems management (24:1), pp. 15-28. gutierrez, a., orozco, j., and serrano, a. 2006. "using tactical and operational factors to assess strategic alignment: an sme study," in: european and mediterranean conference on information systems (emcis). costa blanca, alicante, spain. hale, a.j., and cragg, p.b. 1996. "measuring strategic alignment in small firms," information systems conference of new zealand (iscnz'96), palmerston north, new zealand: ieee computer society washington, dc, usa. hax, a. 2010. the delta model: reinventing your business strategy. cambridge: springer verlag. henderson, j.c., and venkatraman, n. 1999. "strategic alignment: leveraging information technology for transforming organizations," ibm systems journal (38:2-3), pp. 472484. 11 australasian journal of information systems cataldo & mcqueen 2015, vol. 19, selected papers from acis it alignment in smes johansson, b., and sudzina, f. 2010. "can both good and bad alignment of business and information strategies lead to high business performance?," 14th ibima conference, k. soliman (ed.), istanbul, turkey, pp. 1382 1387. kearns, g.s., and lederer, a.l. 2003. "a resource-based view of strategic it alignment: how knowledge sharing creates competitive advantage," decision sciences (34:1), pp. 1-29. lee, t., mitchell, t., and sablynski, c. 1999. "qualitative research in organizational and vocational psychology, 1979 1999," journal of vocational behavior (55:2), pp. 161-187. lee, t.w. 1999. using qualitative methods in organizational research. california: sage publications, inc. levy, m., and powell, p. 2005. strategies for growth in smes: the role of information and information systems. united kingdom: butterworth-heinemann. levy, m., powell, p., and yetton, p. 2001. "smes: aligning is and the strategic context," journal of information technology (16:3), pp. 133-144. lockamy, a., and smith, w.i. 1997. "a strategic alignment approach for effective business process reengineering: linking strategy, processes and customers for competitive advantage," international journal of production economics (50:2-3), pp. 141-153. porter, m. 2001. "strategy and the internet," harvard business review (79:3), pp. 62-79. ravarini, a., tagliavini, m., buonanno, g., and sciuto, d. 2002. "information system checkup as a leverage for sme development," in managing information technology in small business: challenges and solutions, s. burgess (ed.). hershey pa: idea group inc (igi), pp. 63-82. reich, b.h. 1993. "investigating the linkage between business and information technology objectives: a multiple case study in the insurance industry." tagliavini, m., faverio, p., ravarini, a., and sciuto, d. 2004. "monitoring the strategic effectiveness of sme information system," 49th icsb world conference, johannesburg: university of pretoria, south africa. venkatraman, n. 1989a. "the concept of fit in strategy research: toward verbal and statistical correspondence," academy of management review (14:3), pp. 423-444. venkatraman, n. 1989b. "strategic orientation of business enterprises: the construct, dimensionality, and measurement," management science (35:8), pp. 942-962. yin, r.k. 2009. case study research: design and methods, (fourth ed.). usa: sage publications, inc. an earlier version of this paper was presented at the australasian conference on information systems (acis) 2014 in auckland, new zealand. copyright: © 2015 cataldo & mcqueen. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. 12 http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems cataldo & mcqueen 2015, vol. 19, selected papers from acis it alignment in smes 13 1 introduction 2 literature review 2.1 it alignment 2.2 it alignment in smes: the traditional approach based on strategy 2.3 it alignment in smes: the new approach based on processes 2.4 research question 3 methodology 4 results 4.1 brief description of the case 4.2 results 4.3 aligning it with business processes 4.4 strategic alignment 4.5 strategic it alignment with processes: the chicken or the egg? 5 discussion 6 conclusions paper title (paper title style) australasian journal of information systems vichitvanichphong et al 2016, research article good old gamers, good drivers good old gamers, good drivers: results from a correlational experiment among older drivers suchada vichitvanichphong university of the sunshine coast suchada.vichitvanichphong@research.usc.edu.au amir talaei-khoei university of technology sydney amir.talaei@uts.edu.au donald kerr university of the sunshine coast dkerr@usc.edu.au amir hossein ghapanchi griffith university a.ghapanchi@griffith.edu.au bridie scott-parker university of the sunshine coast bscottpa@usc.edu.au abstract in many situations, driving is essential for senior citizens to maintain their independent lifestyle. a systematic literature review was conducted that summarized the age-related physical, visual and cognitive functional declines and their associated risk to driving. based on these findings, we explored whether the skills required in playing xbox kinect video games were correlated with measures of driving performance among older drivers. fifty-two participants, 65 years of age or older (mean = 72; sd = 3.84; range 65 – 85 years; 29 males) who have access to a car and drive frequently were invited to play just dance, table tennis (ping pong), bowling, and dr kawashima’s brain training exercises on an xbox kinect 360. participants also completed a 25-minute on-road driving task along a predetermined route to assess and identify critical driving errors using a similar instrument as that used by a driving license examiner. bivariate correlation examined the relationship between game scores and these objective driving skills. there was a significant correlation between the xbox kinect video games and on-road driving scores (r = 0.861, p < 0.001), indicating that ‘good gamers are good drivers’. this correlation was significant for the males (r = 0.864, p < 0.001) as well as for the females (r = 0.878, p < 0.001). we suggest that performance on xbox games may be a suitable, cost-effective and less-risky indicator of on-road driving skills for older drivers, particularly in jurisdictions in which mandatory testing of older citizens has been introduced or is being considered as a requirement in the driver licensing process. keywords: older drivers; xbox; gaming; road safety; driving behaviour 1 introduction the world’s population is ageing rapidly. in 2012, 6.9% of the world population were more than 65 years old, and this is estimated to increase to around 20% by 2050 (oecd, 2012). as the population is ageing, the number of older drivers is also increasing. unfortunately, older drivers are overrepresented in crash statistics (insurance institute for highway safety, 2010). in the united states, 17% of all traffic fatalities and 16% of all vehicle occupant fatalities in 2012 were caused by older drivers of age 65 and older. in australia, older drivers have the higher risk (per distance travelled) of being killed in a crash than any other age group (transport accident commission, 2013). 1 australasian journal of information systems vichitvanichphong et al 2016, research article good old gamers, good drivers ageing is associated with functional declines in abilities such as cognitive skills, physical skills and visual attention which negatively impact on driving of older adults (e.g., decline in information processing speed, cognitive inflexibility, decline in executive functions and motor skills) (anstey et al., 2012; bixby et al., 2007). however, the literature (etnier et al., 1997; hawkins et al., 1992; kramer et al., 2002; shay and roth, 1992) reveals that exercises or training appears to be positively correlated with improving perception, cognitive, physical and visual abilities. as a result, people who participate in exercise training perform better on tests such as attention capacity, visual-spatial processing, and information processing. in this paper we extend this to investigate the correlations between the skills and abilities required for playing xbox games and the skills and abilities required for driving. cognitive performance in particular has been of considerable interest and appears to be the factor most related to driving limitations of the elderly (vance et al., 2006). safe driving necessitates the coordination of cognitive abilities (e.g., speed of processing, perception, attention, memory). the cognitive abilities of older adults are able to be boosted through cognitive training (ball et al., 2002), (ball et al., 2007), (kramer and willis, 2002). unsurprisingly given that driving involves skills and abilities to handle complex tasks, a considerable proportion of older driver crashes take place while they are driving in complex situations such as negotiating intersections (mayhew et al., 2006), (mcgwin jr and brown, 1999). physical abilities are also necessary for safe driving by older adults. marottoli et al (marottoli et al., 2007) demonstrate a potential relationship between physical exercise training and physical abilities. they also reveal that there is possibility of maintaining older adults’ driving performance by providing physical exercise over a period of time. furthermore, ostrow et al (ostrow et al., 1992) reported that a randomized control trial of a group of elderly drivers on an eight-week physical exercise training regime resulted in improvements in older drivers’ shoulder flexibility and trunk rotation, and these improvements were positively related to specific driving skills such as lane changing and shoulder checking. visual difficulties associated with ageing (e.g. declining speed of visual search, glare sensitivity and acuity, and peripheral field loss) have been found to contribute on unsafe driving or involvement in crashes (wood et al., 2008). a 12-week exercise intervention with older drivers revealed that visual attention, behavioural speed and visuospatial abilities required for driving were significantly enhanced for the participants (hancock et al., 2002). exercise training has been extensively used to enhance the functional abilities of the elderly (i.e. cognitive, physical and visual abilities) resulting in better driving skills. while exercise training programs can be implemented through playing motion-controlled consoles such as xbox kinect, the relationship between the elderly’s functional abilities and their score in playing video games has not been examined. hence, the main objective of this paper is to evaluate the correlation between playing xbox kinect video games as an enjoyable and inexpensive way to combine functional abilities with driving performance among older drivers. the literature regarding the driving skills of the elderly and their associated age-related functional declines has been systematically reviewed. using the experts’ opinion, classifications of elderly’s risky driving skills and their associated age-related functional declines have been generated. this has led to test the following research questions with respect to xbox kinect video games addressing exercise combining functional abilities and how this is correlated to the driving skills of the elderly: rq: are good gamers, good drivers? therefore, this study does not aim to prove the effectiveness of the xbox kinect video games exercises as an effective intervention to improve driving skills, but it is a correlational study evaluating the game skills and the maintenance of driving skills amongst the elderly. conventionally, “elderly” has been defined as a chronological age of 65 years old or greater. orimo et al (orimo et al., 2006) have discussed the differences between those from 65 through 85 years old, referred to as “early elderly” and those over 85 years old as “late elderly” or “oldest old”. for the purpose of this work, discussing a technological solution that requires minimum 2 australasian journal of information systems vichitvanichphong et al 2016, research article good old gamers, good drivers functional abilities, we treat the term elderly as being in their early elderly stage; as suggested those who have begun the ageing process and are starting to experience decline in their functional abilities (orimo et al., 2006). the rest of this paper is organized in the following way: section 2 explains a systematic literature review on the relationship of age-related functional declines among the elderly and the associated impact upon their driving performance. this section also introduces xbox kinect as a way that can combine different functional exercises. section 3 presents an experiment that examines the correlation between the elderly’s score in playing xbox kinect games and their on-road driving performance. section 4 discusses the findings of the experiments, presents the limitations of the paper and opens new avenues of research for future studies. 2 xbox kinect video games: mapping the combination of physical, visual and cognitive engagement to driving performance of older adults in this section, we map skills required to play xbox kinect video games to age-related functional declines associated with driving. this would help us to hypothesize the correlation between xbox kinect playing skills and driving performance. the first step for doing so is to systematically review the papers in this area and elucidate the classifications of risky driving skills and their associated age-related functional declines. we customized the guidelines for systematic search suggested by etnier et al (etnier et al., 1997) which have been used extensively (e.g., (vichitvanichphong et al., 2013a), (vichitvanichphong et al., 2013b)). the systematic method proposes three main steps to carry out a literature search; (1) exploratory background search to identify the relevant keywords and databases, (2) searching for the initial list of studies and (3) relevance appraisal. in what follows, we explain the process that we use in order to conduct our systematic search of the literature for the issues associated with the driving performance of older adults. step 1: exploratory background search: identifying keywords and databases: the first step towards searching the articles was to identify the relevant keywords and online databases. we have conducted a background survey in google scholar. in order to identify the relevant keywords and related online databases, we have implemented the experimental method proposed by (dieste et al., 2009). the method was applied in the first 20 relevant papers found by searching in google scholar. we have identified the most popular online databases indexing these 20 papers, in addition to the most popular keywords relevant to “driving skill”, “functional ability”, and “senior”. based on the exploratory search, six databases have been chosen as listed in table 1. step 2: searching the initial list of papers: the six databases were searched for the terms “aged care”, “aged”, “aging”, “senior”, “old”, “elderly”, “elder”, “older”, “older driver”, “elderly driver” and any of “driving”, “driving assessment” and any of “ability”, “capability” or “performance” in titles, keywords, abstracts or full texts of papers published since 2000, inclusive. the search returned 71,486 articles. the distribution of papers in each database is presented in table 1. step 3: relevance appraisal: in this step, the objective was to filter relevant papers from the initial list and exclude the ones which were not related to our study. this process was carried out by excluding papers based on titles, keywords, abstracts, and full texts. among 71,486 papers indexed in the databases and searched by the keywords 6,942 papers were retained after title filtering, 518 articles remained after abstract filtering and 68 papers were identified as the final list of relevant papers after reading the full text. we also found some of the papers were indexed by multiple databases; see table 1. it is notable that zotero (http://www.zotero.org/) was used for reference management. 3 australasian journal of information systems vichitvanichphong et al 2016, research article good old gamers, good drivers filtered by titles filtered by abstract filtered by text name of database initial list of papers number of found articles number of duplicated articles number of found articles number of duplicated articles number of found articles number of duplicated articles springer 838 188 53 60 8 8 5 wiley interscience 777 193 36 161 23 30 10 sciencedirect 6,030 204 44 146 15 59 31 scirus 41,151 3,326 153 68 6 0 0 pubmed 3,190 922 71 66 8 23 13 google scholar 19,500 2,713 247 89 12 46 39 total 71,486 7,546 604 590 72 166 98 relevant papers by deducting the duplicated articles 6,942 518 68 table 1 distribution of papers in each online database figure 1 relevance appraisal process the final 68 papers were further categorized as follows: (1) driving skills that have been identified as risky among older drivers and (2) age-related functional declines that have been reported by the literature as contributors to driving risk amongst elderly people. in the following sections, we discuss these two findings. the detailed explanations of the results in this systematic review can be found in appendix 1 and 2. in addition, the relevant 68 papers are listed online1 2.1 what happens to our driving when we get older? the driving skills that have been identified as associated with increased risk for older drivers can be further categorized as (1) decision making, (2) direction and lane control, (3) lack of regulation compliance and awareness, (4) speed performance, (5) visual checking and physical control, (6) recognizing and responding to signs, (7) recognizing and responding to traffic lights, and (8) skills involved in turning and parking. the first category is decision making skills such as how elderly people determine, behave, and respond when they approach an intersections or headways (e.g., behaviours in relation to the use of mirrors, signalling, and obedience to signals, judgment, and turning procedures). not surprisingly, we found that when older drivers have made an error, it is difficult for them to recover the situation by subsequently making the right decisions. the second category is direction and lane control. this category has been reported as a critical issue for the elderly’s driving. direction and lane control relate to the ability to control the vehicle with respect to heading along the desired route as well as keeping to the lanes. we found large numbers of errors and unsafe behaviours being made by elderly drivers in relation to this. e.g., extreme positioning error (driving too close to lane markings), turning or changing lanes without looking, unsafe performance in steering, failure to observe nearby traffic when merging and changing lanes. elderly people tend to repeatedly produce errors in direction and lane control which leads to the greater risk and unsafe driving. the third category is a lack of regulation compliance and awareness. the findings suggest that older people who have better self-awareness and traffic awareness when driving have a reduced 1 https://onedrive.live.com/?cid=7d934cf0ac729f11&id=7d934cf0ac729f11!470 4 australasian journal of information systems vichitvanichphong et al 2016, research article good old gamers, good drivers injury risk, crash involvement, and collisions. ignoring or not complying with the traffic regulations leads to an increase in number of crash occurrence in this age group. another problem that has been reported in the literature is speed performance e.g., how elderly approach an intersection and whether they do it at an appropriate speed, confidence when driving at high speeds, and speed control; not too fast or not too slow. the research findings indicate that older road users are more likely to be involved in traffic accidents caused by being unable to approach an intersection at an appropriate speed. for instance, when elderly enter on to larger roads or even intersections, they are more likely to speed up before getting to the road or the intersections. visual checking and physical control have been highlighted by the literature as another risky issue associated with older drivers. the visual checking and physical control entails mirror and blind spot verification, steering wheel control, and brake reaction. these have been reported as one of the common failures related with ageing. another category that has been reported in the literature over the past thirteen years is recognizing and responding to signs. that is a failure to recognize and respond to signs. this is ranked as the third critical major mistake made by the elderly. recognizing and responding to signs include how elderly observe, approach, and react to traffic signs. as they age, perceiving and analysing the situation on the roads becomes difficult and complex for older drivers including in small roads or uncomplicated circumstance. for instance, the elderly fail to stop in the right place, ignore give way signs, and this includes being indecisive of when to proceed when the opportunity arises. it was also found that recognizing and responding to traffic lights has been reported as one of the significant issues associated with driving skills of elderly people. the findings indicate that older drivers commit more traffic light violations owing to failures to stop before the line at a red light and are involved in greater numbers of traffic accidents and collisions. in addition to this, older drivers sometimes fail to approach or even notice traffic lights whether they are yellow, red, or green. turns and parking are also one of the classic examples of risky driving amongst older people. turning or changing lanes without looking, for example, has been reported as a critical error made by the elderly. turning position errors (e.g., wide turns or cut turns), aggressive manoeuvres (e.g., risky turns), turning into the wrong lane, particularly in motorways, have been identified as common mistakes made in this age group. we also found that older drivers often fail to perform parallel parking well which in turn can lead to crashes. readers wishing more detailed explanations on these categories of risky driving skills among elderly and the citations of the relevant articles are recommended to refer to appendix 1. 2.2 age-related functional declines among older drivers the age-related functional impairments of elderly drivers which have been investigated in the 68 relevant papers into three main groups; namely (1) physical abilities, (2) cognitive abilities, and (3) visual abilities. we found that the following physical abilities were deemed necessary for skilful driving and these are associated with safe driving for the elderly. the physical abilities include the measures of proprioception, motor conditions, physical reaction time, hearing ability, and so forth. the findings indicate that since accident involvement tends to increase with age, physical declines could make older drivers more vulnerable to serious injuries or even fatalities. for instance, older drivers, who experience a range-of-motion disability such as neck rotation, limbs, shoulder flexion, elbow, hands, and foot abnormalities, will find difficulty in remaining safe on the roads. this range of motion plays an integral role to support the movement of joints. not surprisingly, driving is a complex task and it is driven basically by body movement. we found that the correlation between physical impairments and lower level driving performance is relatively high. 5 australasian journal of information systems vichitvanichphong et al 2016, research article good old gamers, good drivers the effect of cognitive impairment on driving ability has been also found in the literature. several studies have examined cognitive abilities as a predictor of the elderly’s driving performance and safe driving. safe driving necessitates the coordination of cognitive abilities such as attention, perception, memory, speed of information processing, and self-regulation. however, the literature has reported that a large number of older drivers have suffered from cognitive declines due to ageing as well as chronic diseases. these lead to an increase in the number of collisions. changing in visual abilities also impacts on elderly’s driving performance. visual abilities include the measures of visual acuity, useful field of view (ufov), glare sensitivity and acuity, eye movement, and speed of visual search. age-related changes in vision have been reported in the literature as a significant factor contributing to various errors or crash involvement while driving. the findings also indicate that older drivers with impaired contrast sensitivity have an increased risk of crashes especially when driving at night or on rainy days. one of the measurements for visual attention and processing speed is useful field of view. ufov is the area from which a person can extract visual information in a brief glance without head or eye movement which technically deteriorates with age. we found that reduction of ufov, age and driving skills of the elderly show significant correlation to older adults’ driving difficulty. readers wishing more detailed explanations on these categories of age-related functional declines among elderly and the citations of the relevant articles are recommended to refer to appendix 2. 2.3 mapping: experts’ opinion a peer review of the findings for the age-related functional declines and the risky driving abilities was undertaken. four experts from north america, australia, europe and asia were invited to code both interventions and impacting factors of adoption into categories. the experts were chosen based on their academic knowledge and practical experience in the topic. in order to recruit judges, 3 invitation emails to north american experts, 4 emails to australian experts 4 to european experts and 2 to asian experts were sent. these emails were sent to people who have known contributions in the field. out of these invitations, 6 experts showed their interest and one did not reply to the follow up email. therefore a total of five judges examined the reliability of the results, this meant that there was at least one from each geographical region. the experts were given an instruction sheet including the description of all the items as well as an answer sheet in which they were asked to map the age-related functional declines related to the risky behaviours of the elderly. a reliability test was then undertaken based on proportional reduction in loss (prl) reliability indicator introduced by (ortiz et al., 2013). prl is used to assess the consensus between judges who are invited to map a number of elements together. inter-judge agreement was measured by dividing the total pair-wise agreements by the total pair-wise decisions. with five judges, the mapping of each age-related functional decline into a risky driving skill, resulted in a total of 10 pair-wise decisions being made. the consensus of this mapping was the most frequently selected choice by the five judges. the reliability for each contribution was calculated based on the value of inter-judge agreement. therefore, the hypothetical mapping resulted in a reliability rating of 81%. based on measures presented in (ortiz et al., 2013), the proposed correlation is considered a reliable hypothesis. 3 correlational experiment 3.1 participants fifty-two individuals (range 65 – 85 years; mean = 72; sd = 3.84; 29 males) participated in this study. all the participants had a sufficient level of familiarity with the xbox kinect video games and the project. 6 australasian journal of information systems vichitvanichphong et al 2016, research article good old gamers, good drivers the recruitment process was as follows; senior citizens were emailed and asked to volunteer for the research. the selection criteria were: • must be 65 years or greater • hold a legal driver licence • have been driving regularly for over the last 12 months • have access to a car older people who wished then to participate in the study were required to give written informed consent stating that they have no medical conditions which would preclude their participation and they accepted the minimum chance of accidents and will carry its liabilities as their responsibilities. the university provided a $20 fuel voucher for any participant who completed the driving course to compensate them for the use of their car. of the participants who volunteered, a two-step process followed. in the first step the older driver was asked to participate in an easy low risk driving course. if the researcher then identified that the participant was a high risk driver, he/she was not included in the second (evaluation) course. 3.2 design the study collected data from two different assessments; (1) playing xbox kinect video games and (2) on-road driving assessment. the first assessment was to get the subjects to play the four games (just dance, table tennis (ping pong), bowling, and dr kawashima’s brain training exercises). the second experiment assessed driving performance of the older people. the experiment was a correlational study between two variables resulted in the above-mentioned assessments; (1) gaming score and (2) driving score. 3.3 the choice of games the choice of games was based on a small-scale self-observatory study by sue et al., (2014a, 2013) in which eleven xbox video games were played at least twice per week over a three month period. the player documented his/her experience by explaining how he/she was engaged with these games. the notes from the player were discussed in a focus group of five experts. experts were provided with an instruction sheet as well as an answer sheet in which they were asked to map the xbox kinect video games, functional abilities and driving skills. among the eleven games, only bowling, just dance, table tennis2 and dr kawashima’s brain exercises3 were chosen. this was based on the experts’ opinions on the suitability of these games with respect to combining visual, cognitive and physical exercises. 3.4 procedure the participants were invited to complete two assessments: playing xbox kinect video games: all participants were invited to play 1) just dance (a rhythm game in which players pick any track of music and attempt to dance with the on-screen dancer; the more the player dances similar to the dance the better score she/he will get), 2) table tennis (ping pong), 3) bowling, 2 http://marketplace.xbox.com/en-au/product/kinect-sports/66acd000-77fe-1000-9115d8024d5308c9 3 http://www.totalxbox.com/25651/reviews/dr-kawashimas-body-and-brain-exercises-review/ 7 australasian journal of information systems vichitvanichphong et al 2016, research article good old gamers, good drivers 4) dr kawashima’s brain training exercises (a set of memory performance exercises for the player). the participants played each game individually, in separate sessions. each session began with an introduction to the game explained by the assessor. after that, the player was asked to practice each game until he/she was comfortable with the game. each player was given five minutes’ rest after the practice session. the participant was then asked to play each game three times with actual scores being recorded by the assessor (one round). the player was given 3 minutes of rest between rounds. the average score was calculated as the player’s score for the game. this was repeated in separate sessions for each of the four games. on road driving assessment: in order to ensure the safety in the driving experiment, it was recommended by the ethics committee that dangerous drivers be identified in a basic, nonchallenging pre-test. any drivers identified as dangerous were excluded from the full test. therefore, this experiment consisted of two steps: step 1: the main objective of this step was to minimize driver and assessor risk. in this step, all participants were requested to drive for approximately 10 minutes in a basic nonchallenging road environment while the assessor was sitting next to them; see the driving course provided in appendix 3. at this stage, if the researcher felt that the participant had potentially unsafe driving habits (failing in any of the driving tasks in step 1), the participant would not been invited to proceed to step 2. the participants were given instruction and information regarding the commencement of actual driving, as well as during the driving as required. nonetheless, the experiment did not include any unusual activity that was more than what the participants do in their everyday driving. if the decision was made not to proceed to the second stage, the participant was informed in a polite and considered way (no indication was given to any perceived problems with the participant’s driving abilities). it is notable that no participants were excluded from step 2 due to being classified as a dangerous driver. step 2: main driving assessment. the on-road driving course took approximately 20-25 minutes to complete (the route is provided in appendices 3 & 4. the assessor was seated next to the participant who assessed his/her driving skills based on the driving assessment instrument as described in appendix 3. the participants were given instructions and information on the road prior to the actual driving as well as during the driving anytime they needed it. the objective of this step was to assess the driving skills of the participants. the instructions referred to the direction of driving according to the pre-defined path. once on the road, if the assessor felt unsafe, she would accordingly advise the participant of regarding her concern, as agreed to in the previously signed written consent form, in a polite manner. participants could withdraw from the study at any time. the traffic conditions of the on-road route reflected everyday normal driving (e.g., signalized intersections, pedestrian traffic). 3.5 instruments in order to assess the gaming score and driving score, as discussed above, the experiment employed two separate instruments: instrument for gaming assessment: the instrument recorded the scores given by xbox kinect 360 console for each of the four games for each participant. participants played each game three times, therefore the actual score that was calculated for each game for each participant was the average of his/her scores in each of the three game sessions. instrument for driving assessment: the driving assessment has been designed based on assessing risky driving skills reported in the literature and identified in the systematic literature review (appendix 3). the assessing method and the scoring system were borrowed from the guidelines of assessing driving from the transport department of the region where the experiment was conducted (australian state). the scoring system calculated the scores based on negative marking of driving errors that participants made. there were three types of errors as defined below: 8 australasian journal of information systems vichitvanichphong et al 2016, research article good old gamers, good drivers • critical driving error (cde) that indicates any error that ends up needing a physical or oral interference by the driving assessor, collision, dangerous action, disobeys an official direction by a police officer or an authorized traffic controller. (please note, although it was planned that if such an error occurs the driving test should be immediately stopped, there were no cde made by any of participants.) • non-critical driving error (ncde) which is driving errors that are not critical. • specific repeated driving error (srde) that is ncde associated with a specific assessment item for more than six times. in this scoring system, as followed by the guidelines of the transport department, the instrument is designed in such a way that for each assessment item (see appendix 3) the driving assessor only records an ncde three times and an srde once. for example, if a participant makes 4 or 5 ncde errors, he or she would be still subjected to deduction of only three error points. that is, if a participant repeatedly makes the same error, (more than three times), a maximum of 3 points will be deducted. however, if any errors cause a concern for safety reasons and ends up with assessor interference, then this will be considered to be a cde and will result in termination of the driving assessment. in calculating the total score, once an srde has been recorded, 6 error scores are deducted. therefore, the maximum error for each item is 6 and as such the maximum error that a participant could make during the test for each item can be calculated based on multiplying 6 to the number of assessment items/ risky driving skills among the elderly (see appendix 1). the guidelines requires that the actual score for each participant can be calculated based on how far he/she is from the maximum errors that she/he can make during the test. actual driving scorei = maximum error -error scorei 3.6 analysis after running the above-mentioned assessments, two sets of data were analysed 1) gaming scores and 2) driving score. in the analysis, the following steps were taken: step1 – calculating the relative scores: the objective of the experiment was to evaluate if good gamers are good drivers. therefore, being ‘good’ was defined as a relative term in which the participants were compared with others in the assessment. this was applied to gaming score as well as to the driving score. for each game where i is a participant, j is one of the games i.e. bowling, just dance, table tennis, dr kawashima’s brain training, actual score is what score she/he obtained in each game and the relative score is what is going to be used in the analysis. this also resolved the issues of games having different scoring system, as the relative gaming score is a comparative/ratio value. 𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅 𝐺𝐺𝑅𝑅𝐺𝐺𝑅𝑅𝐺𝐺𝐺𝐺 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑅𝑅𝑅𝑅 𝑗𝑗 = 𝐴𝐴𝑆𝑆𝑅𝑅𝐴𝐴𝑅𝑅𝑅𝑅 𝐺𝐺𝑅𝑅𝐺𝐺𝑅𝑅𝐺𝐺𝐺𝐺 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑅𝑅𝑗𝑗 𝑅𝑅 𝐴𝐴𝑅𝑅𝑅𝑅𝑆𝑆𝑅𝑅𝐺𝐺𝑅𝑅 𝐴𝐴𝑆𝑆𝑅𝑅𝐴𝐴𝑅𝑅𝑅𝑅 𝐺𝐺𝑅𝑅𝐺𝐺𝑅𝑅𝐺𝐺𝐺𝐺 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑅𝑅𝑆𝑆𝑗𝑗 𝑅𝑅 based on this equation, the average of these scores for all the four games was calculated and used in the analysis. in this study, the term game score refers to the relative gaming score. 𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅 𝐺𝐺𝑅𝑅𝐺𝐺𝑅𝑅𝐺𝐺𝐺𝐺 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑅𝑅𝑅𝑅 = 𝐴𝐴𝑅𝑅𝑅𝑅𝑆𝑆𝑅𝑅𝐺𝐺𝑅𝑅 (𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅 𝐺𝐺𝑅𝑅𝐺𝐺𝑅𝑅𝐺𝐺𝐺𝐺 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑅𝑅𝑅𝑅 𝑗𝑗) similarly, a relative score for driving was calculated to reflect the measurement of good driving in comparison with other drivers in the assessment. in this study, the term driving score refers to the relative driving score. 𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅 𝐷𝐷𝑆𝑆𝑅𝑅𝑅𝑅𝑅𝑅𝐺𝐺𝐺𝐺 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑅𝑅𝑖𝑖 = 𝐴𝐴𝑆𝑆𝑅𝑅𝐴𝐴𝑅𝑅𝑅𝑅 𝐷𝐷𝑆𝑆𝑅𝑅𝑅𝑅𝑅𝑅𝐺𝐺𝐺𝐺 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑅𝑅𝑖𝑖 𝐴𝐴𝑅𝑅𝑅𝑅𝑆𝑆𝑅𝑅𝐺𝐺𝑅𝑅 𝐴𝐴𝑆𝑆𝑅𝑅𝐴𝐴𝑅𝑅𝑅𝑅 𝐷𝐷𝑆𝑆𝑅𝑅𝑅𝑅𝑅𝑅𝐺𝐺𝐺𝐺 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑅𝑅𝑆𝑆 step 2 bivariate correlation analysis using statistical package for the social sciences (spss): in order to test the correlation between the two data sets of the relative gaming and 9 australasian journal of information systems vichitvanichphong et al 2016, research article good old gamers, good drivers driving scores, a bivariate correlation analysis was conducted in spss to calculate correlation coefficient (r). the significance level of this correlation was measured by p-value. 3.7 results the results indicated that the correlation between the participant’s score for playing xbox kinect video games and their driving performance score was significantly high (r = 0.861, p < 0.001) thus supporting the hypothesis that “good old gamers are good drivers”. it was found that the participants’ score for playing xbox kinect video games was positively correlated with their driving performance score for the males, r (29) = 0.864, p < 0.001, as well as for the females, r(23) = 0.878, p < 0.001. the difference between these correlations was statistically non-significant, z = -0.194, p < 0.846. this suggests that a good old gamer would be a good driver and this would be also the case for males as well as female with no significant statistical difference between the two. 4 discussion, limitations and future work recently a new phenomenon of motion-controlled consoles such as xbox kinect has entered into the market of video games. these devices have the potential to provide insight into not only the cognition of players but also the physical and visual states requiring interactivity by the use of gestures from the entire body. as such, this technology provides a relatively inexpensive platform upon which the different functional abilities associated with the driving performance of the elderly can be explored, a heretofore unrealised opportunity. previous work had suggested that when we get older, we experience functional declines that result in risky driving behaviour. the systematic literature review revealed that the ageing-related driving literature encompasses physical, cognitive and visual declines as these functional abilities are associated with risky driving. this study sought to investigate if an elderly driver’s skills in playing the xbox kinect video games which combines physical, visual and cognitive exercises, can correlate with their on-road driving skills. interestingly the study demonstrated that an elderly person who plays particular games on the xbox kinect also drives well on the road in typical driving conditions. the consistent finding, irrespective of participant gender, that good gaming performance was strongly correlated with good on-road driving performance suggests that this relationship is relatively robust. this can have important road safety implications. to illustrate, it is noteworthy that a considerable part of the road fatalities involves older drivers aged 65 years and older (transport accident commission, 2013; transport and main roads, 2014). consequentially some jurisdictions have introduced mandatory driver testing4, mandatory medical assessments5, or are considering introducing mandatory testing of older drivers6 as part of the licensing process. the research findings suggest that inexpensive and low-risk xbox kinect games may be an indicator of on-road driving performance for drivers who are familiar with such technology. as such, these devices may suitable for use as a screening tool which could identify drivers requiring on-road assessment to ascertain whether it is safe for them to retain their driving privileges. furthermore, the gaming scores may also indicate on-road driving performance of drivers with medical conditions such as parkinson’s disease (e.g., vaux et al., 2010) as such, xbox games may be used as an inexpensive screening tool to identify drivers who may need additional assessment irrespective of age-related or disease-related impairments, and in particular on-road driving assessment. the present results are consistent with earlier research in older adults which examined the relationships between functional abilities and driving performance of elderly people. however, the study findings are in stark contrast to the findings of lutz and klimmt (jäncke and klimmt, 4 e.g., http://www.rms.nsw.gov.au/licensing/downloads/olderdriverguide.pdf 5 e.g., see http://www.qld.gov.au/seniors/transport/safe-driving/ 6 e.g., see http://www.abc.net.au/radionational/programs/backgroundbriefing/compulsory-testingfor-senior-drivers-under-scrutiny/5117828 10 australasian journal of information systems vichitvanichphong et al 2016, research article good old gamers, good drivers 2011) who reported that expertise in playing video games cannot be transferred to driving measured by computerized tests. there are two possible explanations for this discrepancy. first, while most of the studies in this domain have evaluated the skills of drivers in simulators or computerized tests, the current work measures the on-road driving of participants. the differences between the results of real on-road driving and simulators have been also the topic of discussion in other contexts (helland et al., 2013). to illustrate, mullen et al (mullen et al., 2011) state that simulator approximates the relative driving but does not exactly replicate onroad driving behaviour. this can be one indicator of the conflicting results between (jäncke and klimmt, 2011) and the recent experiment. in this regard, the systematic literature review has come up with the list of functional declines among elderly that need to be taken into account in deploying any technological interventions including simulators. second, the work of lutz and klimmt (jäncke and klimmt, 2011) targets the whole population and does not specifically examine the on-road driving performance of the elderly. in this regard, the work presented here has developed and evaluated a driving assessment instrument for older drivers. in order to do so, a systematic literature review initially was conducted to identify the risky driving skills among older adults. the reliability of the final instrument was evaluated by a panel of experts while the validity of the instrument was tested in the on-road driving study. 4.1 limitations and future work the present work examined the relationship between the elderly’s skills in playing the xbox kinect video games, combining different functional abilities, and their driving performance. the statistical procedure – correlation used in this work does not evaluate possible casual relationships between these constructs. therefore, due to the different objective of this work, it cannot be concluded that training based on playing the xbox kinect video games would improve driving performance of the elderly and a recommendation for the widespread use of the xbox in order to improve driving performance cannot be made. however, it does open up an opportunity for future randomized longitudinal studies to investigate the causalities among these two constructs. following the recommendation of the ethics committee and to reduce the dangerous risks for the assessor, when assessing the driving performance of the elderly, the drivers first were asked to drive a basic and non-challenging route. if their driving was identified by the assessor as dangerous, then they would not be invited to the main driving test. this could have led to a bias in not assessing poor drivers. however, the assessor did not exclude any participants from the second on-road driving stage. in addition, it can be also argued that driving assessment is an observational exercise that can be vulnerable to the subjectivity of the assessor. however, the procedures and the guidelines, issued by the department of road safety and transportation of the region where the experiment was conducted, were used to maximise the reliability of the driving assessment. in this regards, further research is planned in which the findings of the current study are compared to the results of driving in a simulated driving environment. other objective measures such as g-force events (including sudden acceleration, deceleration, and sharp turns) can also be incorporated in these projects, further validating the subjective driving assessments. 5 concluding remarks given the ageing population around the world, the road safety of older drivers is likely to remain of considerable importance. in addition, numerous jurisdictions have mandated driver testing as a licensing requirement for older drivers. testing requirements (mandatory and other) raise issues regarding social and financial capital investments in measuring driver performance. the findings suggest that readily available and inexpensive technology options, such as xbox kinect devices, may be able to assist in identifying drivers in need of further assessment (including on-road driving assessment in specific driving circumstances). 11 australasian journal of information systems vichitvanichphong et al 2016, research article good old gamers, good drivers references anstey, k.j., horswill, m.s., wood, j.m., hatherly, c., 2012. the role of cognitive and visual abilities as predictors in the multifactorial model of driving safety. accid. anal. prev. 45, 766–774. doi:10.1016/j.aap.2011.10.006 ball k, berch db, helmers kf, et al, 2002. effects of cognitive training interventions with older adults: a randomized controlled trial. jama 288, 2271–2281. doi:10.1001/jama.288.18.2271 ball, k., edwards, j.d., ross, l.a., 2007. the impact of speed of processing training on cognitive and everyday functions. j. gerontol. b. psychol. sci. soc. sci. 62, 19–31. bixby, w.r., spalding, t.w., haufler, a.j., deeny, s.p., mahlow, p.t., zimmerman, j.b., hatfield, b.d., 2007. the unique relation of physical activity to executive function in older men and women. med. sci. sports exerc. 39, 1408–1416. dieste, o., grimán, a., juristo, n., 2009. developing search strategies for detecting relevant experiments. empir. softw. eng. 14, 513–539. doi:10.1007/s10664-008-9091-7 etnier, j.l., salazar, w., landers, d.m., petruzzello, s.j., han, m., nowell, p., 1997. the influence of physical fitness and exercise upon cognitive functioning: a metaanalysis. j. sport exerc. psychol. 19, 249–277. hancock, p., kane, m.j., scallen, s., albinson, c.b., 2002. effects of gender and athletic participation on driving capability. int. j. occup. saf. ergon. 8, 281–292. hawkins, h.l., kramer, a.f., capaldi, d., 1992. aging, exercise, and attention. psychol. aging 7, 643. helland, a., jenssen, g.d., lervåg, l.-e., westin, a.a., moen, t., sakshaug, k., lydersen, s., mørland, j., slørdal, l., 2013. comparison of driving simulator performance with real driving after alcohol intake: a randomised, single blind, placebo-controlled, cross-over trial. accid. anal. prev. 53, 9–16. , 2010. fatality facts 2009, older people. insurance institute for highway safety jäncke, l., klimmt, c., 2011. expertise in video gaming and driving skills. z. für neuropsychol. 22, 279–284. kramer, a.f., colcombe, s., erickson, k., belopolsky, a., mcauley, e., cohen, n.j., webb, a., jerome, g.j., marquez, d.x., wszalek, t.m., 2002. effects of aerobic fitness training on human cortical function. j. mol. neurosci. 19, 227–231. kramer, a.f., willis, s.l., 2002. enhancing the cognitive vitality of older adults. curr. dir. psychol. sci. 11, 173–177. marottoli, r.a., allore, h., araujo, k.l.b., iannone, l.p., acampora, d., gottschalk, m., charpentier, p., kasl, s., peduzzi, p., 2007. a randomized trial of a physical conditioning program to enhance the driving performance of older persons. j. gen. intern. med. 22, 590–597. doi:10.1007/s11606-007-0134-3 mayhew, d.r., simpson, h.m., ferguson, s.a., 2006. collisions involving senior drivers: highrisk conditions and locations. traffic inj. prev. 7, 117–124. mcgwin jr, g., brown, d.b., 1999. characteristics of traffic crashes among young, middleaged, and older drivers. accid. anal. prev. 31, 181–198. mullen, n., charlton, j., devlin, a., bedard, m., 2011. simulator validity: behaviors observed on the simulator and on the road. handbook of driving simulation for engineering, medicine and psychology pp13-1 – 13-18 crc press 6000 broken sound parkway, nw, suite 300 boca raton, fl 33487 usa oecd, 2012. world population ageing: 1959-2050. paris. 12 australasian journal of information systems vichitvanichphong et al 2016, research article good old gamers, good drivers orimo, h., ito, h., suzuki, t., araki, a., hosoi, t., sawabe, m., 2006. reviewing the definition of “elderly”. geriatr. gerontol. int. 6, 149–158. doi:10.1111/j.1447-0594.2006.00341.x ortiz, c., castro, j.j., alarcón, a., soler, m., anera, r.g., 2013. quantifying age-related differences in visual-discrimination capacity: drivers with and without visual impairment. appl. ergon. 44, 523–531. doi:10.1016/j.apergo.2012.11.006 ostrow, a.c., shaffron, p., mcpherson, k., 1992. the effects of a joint range-of-motion physical fitness training program on the automobile driving skills of older adults. j. safety res. 23, 207–219. doi:10.1016/0022-4375(92)90003-r shay, k.a., roth, d.l., 1992. association between aerobic fitness and visuospatial performance in healthy older adults. psychol. aging 7, 15. sue, d., ray, p., talaei-khoei, a., jonnagaddala, j., vichitvanichphong, s., 2014a. an initial proof of concept for xbox kinect as an alternative technology to improve driving performance of seniors. gerontechnology 2014; 13(2):283 doi: http://dx.doi.org/10.4017/gt.2014.13.02.417.00. sue, d., ray, p., talaei-khoei, a., jonnagaddala, j., vichitvanichphong, s., 2014b. assessing video games to improve driving skills. j. med. internet res. serious games. 2014;2(2):e5 doi: 10.2196/games.3274 sue, d., talaei-khoei, a., ray, p., jonnagaddala, j., 2013. assessing video games to improve driving skills. presented at the ieee international conference of communication 2014. transport accident commission, 2013. road safety for ageing road users [www document]. url http://www.tac.vic.gov.au/road-safety/safe-driving/older-drivers (accessed 2015/09/15). transport and main roads, 2014. queensland road toll report (no. 864). queensland, australia. vance, d.e., roenker, d.l., cissell, g.m., edwards, j.d., wadley, v.g., ball, k.k., 2006. predictors of driving exposure and avoidance in a field study of older drivers from the state of maryland. accid. anal. prev. 38, 823–831. vaux, l.m., ni, r., rizzo, m., uc, e.y., andersen, g.j., 2010. detection of imminent collisions by drivers with alzheimer’s disease and parkinson’s disease: a preliminary study. accid. anal. prev. 42, 852–858. vichitvanichphong, s., kerr, d., talaei-khoei, a., ghapanchi, a.h., 2013a. analysis of research in adoption of assistive technologies for aged care. presented at the 24th australasian conference on information systems. vichitvanichphong, s., talaei-khoei, a., kerr, d., ghapanchi, a.h., 2013b. adoption of assistive technologies for aged care: a realist review of recent studies. presented at the the hawaii international conference on system sciences. wood, j.m., anstey, k.j., kerr, g.k., lacherez, p.f., lord, s., 2008. a multidomain approach for predicting older driver safety under in-traffic road conditions. j. am. geriatr. soc. 56, 986–993. doi:10.1111/j.1532-5415.2008.01709.x 13 australasian journal of information systems vichitvanichphong et al 2016, research article good old gamers, good drivers appendix 1: risky driving skills among elderly detailed error d ec is io n m ak in g • approach to headway • approach to pedestrians • avoid taking unnecessary risks in traffic • be aware of cyclists • cautious when other erratically • communication with other road users • decision and judgement • error detection (awareness and sensitivity of participants in finding out they had made mistakes) • error recovery (driving behaviours when participants tried to rectify mistakes committed) • foresee dangerous situations in the traffic • intersection negotiation • judge when it is safe to enter a larger road from a smaller side street • judging distanced of oncoming cars • judging speed of oncoming cars • beware of railroad crossing • share road with vulnerable • stop for the pedestrians at zebra crossings d ir ec ti on a n d la n e co n tr ol • centre car in lane • centreline crossings • follow direction • forward view • proper lane selection • keep distance change lane • lane changes/ diverging • lane keeping and lane position • lateral lane position on the road • maintain lane when turn • use mirror for lane change • use of indicator • vehicle control l ac k of re gu la ti o n • comply with traffic regulations • awareness of driving of others • be patient with other road users even when they make mistakes • driving self-regulation e.g. judge whether you are too tired to drive • traffic awareness • use seat belt s p ee d p er fo rm an • approach intersection at appropriate speed • approach to speed limit • low confidence on high speed • speed control (too fast or too slow) 14 australasian journal of information systems vichitvanichphong et al 2016, research article good old gamers, good drivers v is u al c h ec ki n g an d p h ys ic al c on tr ol • appropriate steering recovery • mirror and blind spot verification: check blind spot before change, check mirror change lane, left-right side mirror check, scan to sides, scan to rear/ head check • brake reaction (smoothly and accurately) • car control • choice visual reaction time • head rotation towards the left • head rotation towards the right • mechanical operations (fluency and timeliness of steering and pedal control) steering control: steering torque, steering ability r ec og n iz in g an d re sp on d in g to s ig n s • observing signs: approach to stop signs, traffic sign compliance (give way, sign with caution, stopping in the right place, giving way as required, being decisive to proceed when opportunity arises and correct use of indicators) • changing traffic signals • comply with traffic signals and railway crossing signals • make complete stops • reaction time to popup stop signs • respond to signage r ec og n iz in g an d • approach to signal lights (green, yellow, or red) • colour choice reaction time • obey traffic lights s ki ll s of t u rn s an d p ar ki n g • directional turns and k turns • left turns • right turns • position car for turns • parallel parking • turn into wrong lane • postural sway 15 australasian journal of information systems vichitvanichphong et al 2016, research article good old gamers, good drivers appendix 2: risky driving skills among elderly physical abilities description proprioception the ability to sense the position, location, orientation and movement of the body and its part postural sway balance ability or the movement of the body in a still position sensorimotor performance abilities and skills required to acquire and process sensory information in order to produce a desired response executive control mechanisms the ability to regulate the actions of the body psychomotor functioning measures of strength and balance the measures of strength and balance of dexterity which is related to the ability to control vehicles or maintain lane position endurance the ability or the power to withstand a difficult process or situation wrist flexion and abduction the ability to bend or sway of wrist to the wanted direction range of motion (e.g. neck, limbs, shoulder, elbow, hands) range of flexion and extension of parts of body (e.g. neck, limbs, shoulder, elbow, hands) which joints can be moved eyesight the ability to see that directly impacts on driving hearing impairment the deterioration of sense of hearing which is significantly associated with driving cessation motor conditions (e.g. leg strength and mobility, head and neck flexibility) the flexion and rotation ability of motor/ somatosensory functions of elderly e.g. leg strength and mobility, head and neck rotation, shoulder and elbow flexion, finger curl, ankle plantar flexion and ankle dorsiflexion manual dexterity adroitness of using and controlling hands physical reaction time the interval time between the presentation of a stimulus and the initiation of the muscular response to that stimulus cognitive abilities description executive abilities ability to think or manage more likely related to decision making visuo-spatial perception it refers to ability to process and interpret visual information about where objects are in space working memory explains human beings’ ability to hold relevant information in mind and ignoring irrelevant information speed of information processing speed of process of information evaluation, which declines by age attention and concentration attention and concentration is a part of cognitive assessments. the results will identify how elderly are able to pay attention to what they are doing semantic fluency distinctions of between the denotations of different words and symbols mental flexibility the ability to cope with various circumstances in different ways, particularly how to respond effectively to new, complex and problematic situations cognitive-perceptual dysfunction e.g. abnormality in depth perception, sustained attention, and divided attention difficulties with orientation the relationship between physical position and direction 16 australasian journal of information systems vichitvanichphong et al 2016, research article good old gamers, good drivers reacting too slowly reaction of doing something which the performance is slower than expected self-regulation of driving the perception and decision of oneself to not to drive when he/ she doesn’t feel good or when he/ she realises that health condition is not in a good shape visuo-constructional ability the ability to organize and manually manipulate spatial information to make a design visual memory the ability to store and retrieve previously experienced visual sensations and perceptions when the stimuli that originally evoked them are no longer present visuospatial perception the ability to process and interpret visual information about where objects are in space visual abilities description visual acuity sharpness of vision which is measured by how well elderly can see at different distances e.g. far visual acuity, near visual acuity visual contrast sensitivity (vcs) the ability to distinguish the differences between objects and background visual field perception the ability to see objects in one direction useful field of view (ufov) the area from which one can extract visual information in a brief glance without head or eye movement which basically is measured by visual attention and processing speed peripheral vision the ability to see and identify objects at different degrees/ angles to left or right of the midline while looking straight ahead binocular vision the use of both eyes to produce a single image colour vision the ability to distinguish an object in order to see its colour glare sensitivity and acuity is defined as lowered contrasts in visual filed due to outside sources of light glaucoma an eye disease which damages the optic nerve and impairs vision visuospatial ability related to understand and conceptualizing visual representations and spatial relationships in learning and performing a task perception of motion real movement of objects through visual field eye movement the ability to move eyes to be able to catch an object passed speed of visual search the ability to seek and find something by consuming less time for searching 17 australasian journal of information systems vichitvanichphong et al 2016, research article good old gamers, good drivers appendix 3: driving assessment ncde: non-critical driving error cde: critical driving error: any error that ends up to: • physical or oral intervention by the driving examiner • collision • dangerous action • disobeys an official direction by a police officer or an authorised traffic controller. srde: specific repeated driving error: ncde associated with a specific assessment item for more than six times. driving skills assessment item errors ncde1 ncde2 ncde3 srde cde direction/ lane control centre car in lane centreline crossings forward view proper lane selection keep distance change lane lane changes/ diverging lane keeping maintain lane when turn use mirror for lane change use indicator visual checking and physical control mirror and blind spot verification brake reaction (smoothly and accurately) head rotation towards the left head rotation towards the right steering control speed control approach intersection at appropriate speed approach to speed limit control of speed (too fast or too slow) decision making intersection negotiation (entail assessment of behaviours related to mirror use, signalling, approach, obedience to sign/ signal, judgement and turning procedures) judge when it is safe to enter a larger road from a smaller side street stop/ start/ back (behaviours) recognizing and responding to signs observing signs respond to signage recognizing and responding to traffic lights approach to signal lights (green, yellow, or red) turns/ parking left turns right turns position car for turns turn into wrong lane parallel parking awareness use seat belt 18 australasian journal of information systems vichitvanichphong et al 2016, research article good old gamers, good drivers appendix 4: course of driving – step 1 1. begin: parking 5a – olympic park, sydney, australia 2. leave parking 5a to the right 3. drive to holker way. 4. go to parking 5b 5. leave the parking 5b to the right. 6. turn to the left and come back to holker way. 7. come back to parking 5a. 19 australasian journal of information systems vichitvanichphong et al 2016, research article good old gamers, good drivers appendix 4: course of driving – step 2 1. begin: parking 5.a – olympic park. 2. turn right toward turn right toward hill rd 3. take the first left onto hill road 4. continue on john ian wing parade 5. take perkins avenue and spitz avenue to charton avenue 6. head west on beaconsfield st towards wetherill st north 7. take the 1st right onto wetherill st north 8. turn left onto carnarvon st 9. head west on carnarvon st toward silverwater rd/a6 10. turn right onto silverwater rd/a6 11. turn right onto holker st 12. head east on holker st toward slough ave 13. turn left onto hill rd 14. head northeast on hill rd toward bennelong parkway 15. turn right onto bennelong parkway 16. head southwest on bennelong parkway toward marjorie jackson parkway 17. turn right onto marjorie jackson parkway 21. turn left onto hills road. 22. come back to p5a. 20 australasian journal of information systems vichitvanichphong et al 2016, research article good old gamers, good drivers copyright: © 2016 vichitvanichphong, talaei-khoei, kerr, ghapanchi & scott-parker. this is an open-access article distributed under the terms of the creative commons attributionnoncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. 21 http://creativecommons.org/licenses/by-nc/3.0/au/ http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 xbox kinect video games: mapping the combination of physical, visual and cognitive engagement to driving performance of older adults 2.1 what happens to our driving when we get older? 2.2 age-related functional declines among older drivers 2.3 mapping: experts’ opinion 3 correlational experiment 3.1 participants 3.2 design 3.3 the choice of games 3.4 procedure 3.5 instruments 3.6 analysis 3.7 results 4 discussion, limitations and future work 4.1 limitations and future work 5 concluding remarks microsoft word ajis vol 19.docx australasian journal of information systems cuellar, parinyavuttichai & lin 2015, vol. 19, pp. p1-p5 post publication review p1 post publication review lin, a., & parinyavuttichai, n. (2015). is project management and risk escalation: towards a dynamic model. australasian journal of information systems, 19. doi: http://dx.doi.org/10.3127/ajis.v19i0.929 review this paper seeks to advance project risk analysis beyond what it terms a static risk factor analysis into a process that deals with risks that emerge as the project progresses. the authors analyse the escalation literature and a case study of a system implementation and provide a model of how they believe risk escalation occurs. while i found the concept of the paper interesting, it ultimately disappoints. the main issue with this paper is that it confounds the concept of risk factors and issues, which causes its analysis to be skewed, and compromises its contribution. in terms of the model, it is based on an outdated typology of project risk and some of the logic involved is questionable. they define risks as “… the uncertainties that prevent a project team from delivering a planned system on time and within budget.” (p. 1). this definition is in line with the standard definitions that we see in the literature. e.g. as barki, rivard, talbot (1993) observe “… many definitions of risk comprise two dimensions: (1) the probability associated with an undesirable event and (2) the consequences … of this event occurring” (pp. 204-205). see also, parker and stanworth (2005), schmidt, lyytinen, keil and cule (2001), wallace, keil and rai (2004). the key point is that the concept of a risk factor deals with adverse events that have not yet occurred. this is a crucial distinction; once a risk event occurs it is no longer a risk factor but rather an issue or a problem that must be resolved. this definition while seemingly insignificant is a key importance. in the rest of the paper, what is seen is not risk factor analysis but rather what they call “pragmatic” responses to issues. there is no evidence that risk factor analysis resulting in response strategies to risk events has ever occurred in this project. thus when they discuss “emergent risks” what they are really discussing is issues or problems in the project. this has a key impact on the validity of their analysis. their confounding of risk and issues is seen throughout the paper. they state: “is project risks may emerge during a project as a result of unexpected factors; either within the project (e.g. changes in users’ expectations of a system or a change of project champion), or within the environment of the project.” (p. 2). correctly stated, it might say, “potential risk factors can become obvious” or “project issues emerge.” similarly, section 5 begins “this section develops an explanation of how the risks identified and managed contributed not to their alleviation but their escalation” (p. 10). it should read, “… how issues identified…” the confounding of risk factors and issues is further illustrated when the situations addressed by the project manager are called “problems” (p. 10). in the focal case, there is no discussion in the findings of how the risk factors were managed in the focal case. they do not discuss if there was any risk analysis or response planning. there is much discussion of how issues were handled or the processes used by the team to address issues. instead of risk management, what we have is the response of the project manager to the problems that occurred in the project and the unfortunate decisions he made that exacerbated existing problems or created new ones. while they organize the discussion around the various categories of risks to project success, the discussion is about issues that occurred. the paper seems to try to place the blame on risk factor analysis for the escalation of the problems when in fact they are the result of bad project management decision-making in the handling of issues. the problems related to confounding risks and issues cannot be resolved simply by replacing the word “risk” with “issue.” to do so makes the motivation of the paper illogical since it talks about the deficiencies of risk factor analysis. it also removes the contribution of the paper in australasian journal of information systems cuellar, parinyavuttichai & lin 2015, vol. 19, pp. p1-p5 post publication review p2 which they describe a proposed model of risk (issue?) escalation. by confounding risks with issues, they have invalidated the model that they develop in section 5. what is actually occurring is that the “pragmatic” responses to issues has resulted in the escalation of the issues to an issue that is more critical. the proposed model also has problems. first, the literature review on escalation on which the model is based is somewhat outdated. while keil 1995 is a good base, there are more recent papers that form a new stream dealing with communications related issues (e.g. mum and deaf effect or message distortion) as a cause of escalation (see cuellar 2009 ch 2 especially for a review up to date as of 2009; cuellar et al. 2006; lee et al. 2014; morrison and milliken 2000; park et al. 2009; smith et al. 2001; tan et al. 2003; tourish and robson 2006). some of these papers were cited in the paper, and included in table 3 as psychological issues. now while there is a psychological basis for these issues, they are more precisely classified as communications failures in the project. this new stream would need to be identified in the model. second, the evidence of project escalation is missing from the findings. it is discussed in section 5, but the proof that it occurred is missing from section 4. third, i’m not sure that the evidence shows the project escalation causes risk (issue?) escalation. it seems to me that the evidence shows that issue escalation leads to project escalation. finally, the model does not seem to be clear on how the antecedent factors, the management actions and the escalation factors are related. while an attempt has been made in table 3, the logical inference is not convincing that the risk areas cause the escalation factors, which cause the management response. for example, staff shortages and an unrealistic schedule are not causal factors of the mum effect (park et al. 2009; smith and keil 2003; tan et al. 2003). the evolution of risk over time in a project and how project responses affect the project risk profile are important understudied areas. it is unfortunate that this paper does not address them adequately. michael cuellar georgia southern university mcuellar@georgiasouthern.edu references barki, h., rivard, s., and talbot, j. 1993. “toward an assessment of software development risk,” journal of management information systems 10:2, pp. 203-225. cuellar, m. j. 2009. an examination of the deaf effect response to bad news reporting in information systems project, in: doctoral disseration. atlanta, ga: georgia state university. cuellar, m. j., keil, m., and johnson, r. d. 2006. “the deaf effect response to bad news reporting in information systems projects,” e-service journal 5:1, pp. 75-97. lee, j. s., cuellar, m. j., keil, m., and johnson, r. d. 2014. “the role of a bad news reporter in information technology project escalation: a deaf effect perspective,” the database for advances in information systems 45:3, pp. 8-29. morrison, e. w., and milliken, f. j. 2000. “organizational silence: a barrier to change and development in a pluralistic world,” academy of management review 25:4, pp. 706725. park, c., keil, m., and kim, j. w. 2009. “the effect of it failure impact and personal morality on it project management reporting behavior,” ieee transactions on engineering management 56:1, pp. 45-60. parker, j., and stanworth, h. 2005. “'go for it!' towards a critical realist approach to voluntary system risk,” health, risk and society 7:4, pp. 319-336. australasian journal of information systems cuellar, parinyavuttichai & lin 2015, vol. 19, pp. p1-p5 post publication review p3 schmidt, r., lyytinen, k., keil, m., and cule, p. 2001. “identifying software development risks: an international delphi study,” journal of management information systems 17:4, pp. 5-36. smith, h. j., and keil, m. 2003. “the reluctance to report bad news on troubled software projects: a theoretical model,” information systems journal 13, pp. 69-95. smith, h. j., keil, m., and depledge, g. 2001. “keeping mum as the project goes under: toward an explanatory model,” journal of management information systems 18:2, pp. 189-227. tan, b. c. y., smith, h. j., keil, m., and montealegre, r. 2003. “reporting bad news about software projects: impact of organizational climate and information asymmetry in an individualistic and a collectivistic culture,” ieee transactions on engineering management 50:1, pp. 64-77. tourish, d., and robson, p. 2006. “sensemaking and the distortion of critical upward communication in organizations,” journal of management studies 43:4, pp. 711-730. wallace, l., keil, m., and rai, a. 2004. “understanding software project risk: a cluster analysis,” information & management 42, pp. 115-125. author response we would like to thank the reviewer for not only taking interest in our paper but also spending time on giving us valuable feedback. the reviewer pointed out that the paper has confounded risk factors and issues and argued that the two, risk factors and issues, should be differentiated in the analysis and discussion. according to the reviewer “the concept of a risk factor deals with adverse events that have not yet occurred” while “once a risk event occurs it is no longer a risk factor but an issue or problem that must be resolved”. we agree that the importance of such distinction as it has implications for our arguments and analysis. we want to point out that there are two types of risk factors: perceived and unperceivable risk factors (pmi, 2004 and 2013); the former refers to those risks that can be known and anticipated by a project team while the latter refers to those that cannot be known or identified prior to the occurrence of such risks. if a risk factor is defined as uncertainty that an event may bring to a project then we can argue that an issue/problem (once a risk factor occurs) and even solution that is designed to solve the issue/problem in itself can potentially be a risk factor. it is assumed that risks can be eliminated if they can be identified in advance and contingency plans are in place nevertheless depending on the situations the contingency plans which are planned in advance can potentially introduce more risks (uncertainties) into the project. the paper is interested in the unperceivable risk factors and uncertainties that brought by a project team’s intentions of trying to manage the issues/problems and responding to unperceivable risks during the project. we agree with the reviewer that we need to make clearer distinction between risk factors and issue/problems in our writing. in this case study we observed that very little effort was put into risk identification and analysis prior to and during the project. this is mostly due to the lack of experience on the management part. we saw that most actions and decisions taken by the project managers and team were the results of pragmatic responses to issues/problems, and these responses induce more uncertainties further down the line in the project. the paper does not argue that risk factor analysis or lack of it was the cause for the escalation of the problems rather the paper intends to argue that the act of ‘managing’ (e.g. issue/problem) is indeed a source of further risks. pragmatic responses, which usually are knee-jerk reactions and lack of holistic view, can escalate the situation from bad to worse. situation here refers to more uncertainties. we are aware that in some projects communication failures are the sources of risk factors. the paper discussed requirement risks (misunderstanding requirements, partial understanding requirements) under the heading of psychological factor. yes, the requirement risks can easily be discussed under the heading of ‘communication’ but the paper regards that the uncertainties caused by requirements are due to information framing and processing bias rather than australasian journal of information systems cuellar, parinyavuttichai & lin 2015, vol. 19, pp. p1-p5 post publication review p4 communication or miscommunication in this case study. we argue that how a system and problem is framed and communicated depends on information framing and processing bias. therefore in order to understand why particular actions were taken we need to understand how the system and problem was framed and processed. in the case study we observed that there was ‘lack of communication’ between project manager and system analyst and among project team members. we discussed this under the heading of psychological factor because we considered that the lack of communication is an outcome rather than a cause. for example not reporting to the manager the progress of the project was interpreted as a deliberate act by system analyst for personal reason (e.g. personal ego); and the manager tolerated this behaviour because he did not want to upset the relationship by confronting with the system analyst. the reviewer pointed out that “the evidence of project escalation is missing from the findings. it is discussed in section 5 but the proof that it occurred is missing from section 4.” section 4 focuses on the risk factors which are induced by decisions taken to manage the issues/problems encountered and/or induce further uncertainties in the project. section 5 examines the impacts of these risk factors in a broader context: a project as a whole. we structure the writing in this way so that project escalation can be understood as a process as well as a result of a number of risk occurrences. in so doing we avoid placing the blame on any single risk factor that cause project escalation. we believe that escalation factors are closely linked to the management actions because the decisions taken are informed by the management’s perception of the issue/problem and expectation of what would happen and bounded by the context e.g. structural and social. the outcomes of their actions can re-enforce particular or contribute other escalation factors. we also believe that their decisions whether they are well informed or not to extent bear risks/uncertainties. we acknowledge the critique that the paper requires some clarifications and improvements. we will incorporate the feedback and acknowledge it in our future work. nipon parinyavuttichai rajabhat rajanagarindra university nparinya@gmail.com angela y. lin the university of sheffield a.lin@sheffield.ac.uk references project management institute (2004). a guide to the project management body of knowledge (pmbok guide). newtown square, pa: project management institute. project management institute. (2013). a guide to the project management body of knowledge (pmbok guide) fifth ed. newtown square, pa: project management institute. copyright: © 2015 cuellar, parinyavuttichai & lin. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. australasian journal of information systems cuellar, parinyavuttichai & lin 2015, vol. 19, pp. p1-p5 post publication review p5 paper title (paper title style) australasian journal of information systems deschine parkhurst 2017, vol 21, research on indigenous ict protecting oak flat 1 protecting oak flat: narratives of survivance as observed through digital activism nicholet deschine parkhurst arizona state university nicholet.deschine@asu.edu abstract american indians are increasingly using social media/social network platforms as a tool to influence policy through social change. the activist group apache stronghold represents a case of american indians utilising social media tools to protect oak flat and influence federal indian policy. oak flat is sacred apache land located in superior, arizona. united states legislators transferred oak flat to the mining company resolution copper as part of the omnibus national defense authorization act of 2015. qualitative analysis of social media content and advocacy tactics – specifically through use of timeline and digital ethnography – of apache stronghold from 2015-2016 reveal the interrelated nature of on-the-ground efforts, online efforts, solidarity efforts, and legislative support efforts. in sum, these efforts express narratives of survivance, healing, and a future orientation, as a unique dimension of social change. keywords: american indian, activism, decolonising methodologies, survivance, social media, internet studies 1 introduction contemporary american indian activism in the united states has been referred to as a resurgence of the american indian activism of the mid-20th century, particularly with regard in the tradition of the american indian movement (aim) and red power (risling baldy, 2016; amerman, 2003). but to suggest there is resurgence, or the restart, of activism minimises the efforts of american indians who continue to engage in present day long-term resistance, and suggests that injustices against american indians in the united states after the 1970s were non-existent, or were un-fought battles and of no consequence. the framing of contemporary american indian activist efforts as resurgence is perhaps because of the increased use of visibility via social media, such as twitter. american indian activist uses of social media have elevated national and international awareness about the ongoing struggles of injustice in indian country related to the protection of sacred sites, homelands, natural resources, and tribal sovereignty. this study reveals the digital activist efforts of apache stronghold to protect sacred land known as oak flat. apache stronghold is an activist group advocating alongside the san carlos apache tribe, which has been opposed to mining and mineral extraction in oak flat for over ten years (apache stronghold, 2016). while social media has been discussed in the broader literature as a tool that can be harnessed to raise awareness and bring about social change for oppressed and underrepresented groups of people (greenleigh, 2013), there is less literature exploring social media activism as an extension of contemporary grassroots activism, and as a narrative of american indian survivance, described as “an active sense of presence, the continuance of native stories” and the “renunciations of dominance, tragedy, and victimry” (vizenor, 1999, p.vii). this less explored area is important to consider especially due to the debates of whether or not social media networks are echo chambers (whether by selective exposure or ideological segregation) of like-minded individuals (barberá, jost, nagler, tucker, & bonneau, 2015) and that cross-ideology exposure especially of political views on twitter is unlikely (himelboim, mccreery, & smith, 2013). given the likelihood that social media users, particularly on twitter, are more likely to be ‘tweeting to the choir’, the use of social media by american indians does more than amplify minority voices, disburse information, and mobilise collective action but also shares the narrative of survivance among those individuals within their networks. australasian journal of information systems deschine parkhurst 2017, vol 21, research on indigenous ict protecting oak flat 2 survivance is a powerful way to understand that the content of social media posts are very well acts of resistance (smith, 2012) that celebrate every day stories of american indian culture, ties to land, and stories of survival. this research study combines a decolonising and digital methodological approach to a singlecase study investigation of how an american indian activist group utilises social media and onthe-ground organising to influence social change. the advocacy of apache stronghold represents an appropriate case for understanding how activists frame campaigns and mobilise action in spite of the limited opportunities afforded to american indians and tribes in the united states. apache stronghold not only uses social media as a tool to bring about social change but also as a tool to celebrate survival as a form of colonial resistance, and survivance as a means toward encouraging action. 2 literature review writing about american indian activist uses of social media requires an understanding of the intersection of at least three bodies of literature: activism and social media; decolonising methodologies in social media; and american indian activism. read together, these areas of literature show how the amplification of voice that occurs through strategic activist uses of social media relates to the history and commitments of american indian activism. 2.1 activism and social media 2.1.1 amplification of voice american indian tribes experience a unique situation when it comes to united states federal policy. the federal government in their role as trustee of indian affairs has a government-togovernment relationship with tribes, and mandates tribes be consulted when federal policies have tribal implications (richland & deer, 2010; rockwell, 2010; federal register, 2000). in the united states, 1.7 percent of the total population, that is 5.2 million people, identify as american indian or alaska native (ai/an) (u.s. census bureau, 2012). as less than two percent of the u.s. population, ai/ans lack substantive representation in congress, whether this is few congressional members being enrolled members of federally recognised tribes or too few congressional members who have a constituency base of ai/ans to advocate for (conner, 2014). in addition, congressional voting on federal indian policy is greatly influenced by the presence of a federally recognised american indian tribe within the district of elected officials (conner, 2014) versus a district having a substantial number of american indian constituents (turner, 2005). the lack of representation in congress and policy-decision making processes, or too few congressional members to advocate for ai/ans, leaves the native voice out of policy making discussions and ultimately forgotten. social media has been considered a tool to amplify the voices of oppressed and underrepresented groups of people to raise awareness about issues in the general public and to activate them to take action (greenleigh, 2013). however, concerns about whether policy decision-makers or mainstream media (dreher, mccallum, & waller, 2016) and even individuals with opposing ideological viewpoints (himelboim, mccreery, & smith, 2013) are listening raise questions about the ability of social media to effectively bring about social change. while there may be concerns about whether the right political actors are listening, grassroots activist efforts can leverage collective power to bypass media gatekeepers and to amplify issues that may otherwise be forgotten into the public conscious. for example, the trayvon martin murder case—in which a young black teenager was killed by police – highlighted one major aspect about the power of social media. initially, national mainstream media mostly ignored the case but after weeks of increasing social media attention the national media picked up the story (graeff, stempeck, & zuckerman, 2014). the public voiced the issue as a top-tier story requiring 24-hour coverage. the magnitude of people’s interest in the case and eventual national news coverage demonstrated the power that social media can have in influencing traditional media gatekeepers (greenleigh, 2013). while traditional media may help to legitimise social media movements, as was the case for #idlenomore in its first six weeks, the australasian journal of information systems deschine parkhurst 2017, vol 21, research on indigenous ict protecting oak flat 3 media’s framing of a movement may decenter indigenous peoples preference on how their issues are discussed (moscato, 2016). idle no more is a grassroots indigenous movement in canada that drew attention to the importance of indigenous sovereignty and the protection of treaty rights (john, 2015). 2.1.2 collective action social media is also a tool to mobilise collective action to bring about social change. the arab spring, the occupy wall street movement, and spain’s los indignados are examples of the organisation of collective action for mass political influence through social media. social media can mobilise collective action more quickly than through traditional methods (valenzuela, 2013) as well as mobilise collective actions across geographic areas that may have previously been a barrier for grassroots activist organising. the mobilisation of collective actions manifest in various forms, including wide-spread demonstrations and protests across multiple locales, the convergence of people in one location, and even the collective act of thousands of people each engaging in individual tasks. for example, the #standwithstandingrock and #nodapl activist movement highlights the manifestation of collective actions in various forms from solidarity demonstrations in cities across the united states and worldwide (democracy now!, 2016), the exponential growth of up to 10 thousand ‘water protectors’ camping near the pipeline’s crossing of the missouri river (medina & davis 2016; loeffelholz dunn, 2016), and individuals, tribes, and cities divesting from banks funding the dakota access pipeline (chappell, 2017; brainerd dispatch, 2017; grinberg, 2016). digital activism has been considered an important component to effective contemporary activism. on-the-ground activist efforts and digital activist efforts can both contribute to the end goal of raising awareness to a larger audience on a particular issue, and mobilising individuals on the other side of a screen. maile, executive director and founder of english at work, states that you need the people who chain themselves to the tree, the people who work 30 years to save the tree, and you need the people that use social media as a part of the overall awareness strategy; but you need all the elements together to be most effective (harlow & guo, 2014). this being the case, digital activist efforts in indian country benefit from the foundation of on-the-ground activist networks (e.g. american indian movement, indigenous environmental network) and the proliferation of indian country issues being raised online, which presses the demand for media coverage. 2.2 decolonising methodologies 2.2.1 decolonising social media studies by celebrating survival social media studies often utilise methods that quantify aspects of the social media experience by collecting data characteristics such as the exposure of social media posts, the sharing and diffusion of information, number of times specific hashtags are used, or the magnitude and analysis of social networks. these types of data analytics can be beneficial for understanding and potentially predicting social phenomena especially in situations of policy decisions or in the case of activism to further understand the influence and impact of social media activism. for example, researchers (vigil-hayes, duarte, deschine parkhurst, & belding, 2017) tracked the connective actions of native american advocates on twitter to gain an in-depth understanding of the ubiquitous and pressing political issues that can and should be addressed by political actors. consequently, the quantification of data naturally leads to measuring impact, influence, or importance, which, in the case of studying digital activism, draws challenges since events, news, and trends can shift quickly. social media studies in a way are about how the collection, measurement, and interpretation of data can be used to influence outsiders who then can make decisions about an issue or community. since decolonising methodologies aim to empower indigenous communities (smith, 2012; jacob, 2013) the decolonisation of social media studies should center indigenous perspectives and their experiences with social media, and, much like mass media, provide a counter perspective to colonial discourses (knopf, 2010, smith, 2012). for example, knopf (2010), in a study about decolonising media in canada, states that aboriginal peoples by australasian journal of information systems deschine parkhurst 2017, vol 21, research on indigenous ict protecting oak flat 4 controlling and creating their own media are focusing on the needs and priorities of their own people as well as offering opposing perspectives on current issues. furthermore, the decolonising methodology ‘celebrating survival,’ which focuses on how indigenous peoples celebrate “a sense of life, and diversity, and connectedness” (smith, 2012, p. 146), can empower individuals and communities who, in the case of indigenous struggle, express narratives of survival (or survivance) via social media. 2.3 american indian activism research on american indian activism in the united states has focused on american indian civil rights activists and ethnic pride particularly of the 1960s and 1970s through the efforts of american indians who occupied alcatraz island, the american indian movement (aim) and red power. yet, activism and advocacy in indian country has persisted. long-term resistance efforts include the navajo matriarchs at black mesa on the navajo nation who have resisted relocation for 30-years; the strong tribal and american indian opposition to the expansion and development of a ski resort on the san francisco peaks since 1969 and the resort’s use of reclaimed waster water and production of man-made snow; the 45+ year efforts to eliminate stereotypical native american images and team names in media, including the washington nfl team’s name the redskins; and the focus of this study, the 10+ year long-term efforts of the san carlos apache tribe to protect sacred land known as oak flat located in arizona from mineral extraction. further, in recent years there has been an increase in the use of social media platforms and digital tactics to raise awareness about a multitude of injustices american indians face such as the protection of sacred sites and the environment against mining or commercial development (e.g. keystone pipeline, san francisco peaks), violence against native women, the representation of natives in sports and media, the cultural appropriation and theft of cultural property and practices, and civil rights. in the united states, social media has been used to circulate internet memes related to the free leonard peltier movement (lenhardt, 2016); to express solidarity with idle no more and first nations people’s defense of indigenous sovereignty and treaty rights; to draw attention to missing and murdered indigenous women through the use of the hashtag #idlenomore and subsequent round dance flash mobs in malls and parks (john, 2015; walker, 2014); and to disseminate information and organise collective action through facebook and twitter account pages setup for specific causes and their applicable hashtags. hashtag use in american indian political discourse on twitter have been categorised in one study as representing topics of identity, civil rights, current events, and environmental issues (vigil-hayes, duarte, deschine parkhurst, & belding, 2017). the use of hashtags in american indian activist efforts not only can draw attention to specific causes but also can be used to support a greater movement of similar injustices across tribal groups, or even to connect to the global indigenous community. for example, #savethepeaks refers directly to the efforts to protect the san francisco peaks where as #protectsacredsites refers to a conglomerate of sacred sites protection efforts, which may include the san francisco peaks, standing rock, and oak flat. the hashtag #indigenous has been discussed as an overarching all encompassing label in which everything “relevant to indigenous, aboriginal, or native american peoples” may be attached (vigil-hayes, duarte, deschine parkhurst, & belding, 2017). furthermore, hashtags can also be indigenised to not only present alternative discourses but also to “support efforts for self-determination in a very modern and public context” (risling baldy, 2016, p.94). for example, the hashtag #notyourtonto was generated to protest and counter the film the lone ranger’s portrayal of american indian peoples in stereotypical characters and hollywood’s failure to promote the representation of american indian people in media (risling baldy, 2016). 3 methodology the proliferation of digital activist efforts related to american indian issues leads to the overarching question and focus of this study: “how does an american indian activist group australasian journal of information systems deschine parkhurst 2017, vol 21, research on indigenous ict protecting oak flat 5 utilise social media and on-the-ground organising to influence social change?” specifically, by studying the efforts of apache stronghold, this study reveals interrelated digital and on-theground aspects of oak flat activism, as well as the narratives of survivance portrayed by the activist group apache stronghold through social media. there were two steps to this qualitative single-case study. in the first step, a timeline of oak flat activist ‘critical events’ was developed based on online data sources such as websites, social network sites, and local and national news sources. ‘critical events’ refers to junctures in which the actors in the movement recognise they cannot go back to their sense of what happened before, that history is irrevocably changed due to changes in the social world around them (sewell, jr., 2005). this study is comprised of the critical events surrounding oak flat activism, which took place between december 2, 2014, the date the southeast arizona land exchange and conservation act of 2013 was introduced in the u.s. house of representatives, through december 31, 2015. the timeline was integral to the second step, which consisted of qualitative analysis of social media content associated with critical events. this qualitative analysis was conducted with a goal of centering the narratives, experiences, goals and social contexts of the american indian activists, in short, de-centering the white or non-native gaze by foregrounding the american indian activists’ goals. 3.1 method: data gathering, sampling, and analysis gathering and analysing data was a detailed and simultaneous process, which i describe in two steps. i first compiled a timeline of critical events relating to oak flat between december 2014 and december 2015 that included a variety of san carlos apache tribe efforts, legislative efforts, efforts of the activist group apache stronghold, and ally efforts. these events were compiled from online public data sources such as websites, facebook, twitter, instagram, local and national news sources, and civic engagement platforms. examples of critical events include the creation of a petition on president barack obama’s online civic engagement platform, ‘we the people’, apache stronghold’s creation of a tumblr account on march 1, 2015, congressional legislation introduced in the senate and house, statements made by the u.s. department of the interior secretary (jewell, 2014), the white house response to the ‘we the people’ online petition (gillette, 2015), the start of oak flat occupation, apache stronghold’s calls for action, spiritual runs and ceremonies, protests and demonstrations, and the uploading of unique digital content. simultaneously, as i was compiling the timeline i read social media statuses, articles, and websites; watched videos and listened to audio; reviewed photos; and clicked attached url links located within the digital content of identified critical events. the goal of this digital ethnographic process was to obtain a diversity of media content and emerging patterns and themes. during this process i reviewed, documented, wrote descriptions, and took notes as to the nature and themes of content, from a wide range of media, till i reached saturation of data or in other words when i started to identify repeated content; for example, a popular picture of apache stronghold activist naelyn pike is standing in times square holding a bow and arrow (see figure 1). i also reviewed all of apache stronghold’s social media accounts (e.g. facebook, tumblr, twitter, and instagram) specifically looking at the content and types of social media they shared. australasian journal of information systems deschine parkhurst 2017, vol 21, research on indigenous ict protecting oak flat 6 figure 1. naelyn pike holding a bow and arrow in new york times square, photo by standing fox (revkin, 2015) initially, my notes did not code the content based on a priori categories (corbin & strauss, 2015). as i reviewed each new piece of data i noticed patterns and themes appearing such as the inclusion of specific hashtags, memes, art, and the nature of text. i then back coded the content and future content based on those observed patterns. if a new pattern was observed that had not been previously seen and didn’t fit with an already existing code, a new code was created; i did not, however, go back and recode the previous data, as i chose to preserve the insights gathered through my ethnographic process. in this way, i began to observe the threads of narratives of survivance emerging. throughout these dual processes, i digitally archived data through screenshots; saving websites and news articles as pdfs; saving website and video urls, tweets, facebook statuses, and news articles. 4 results the results of this digital single-case study indicate aspects of social media content, such as the structure of tweets and facebook status, emergent themes of content, relevant hashtags, and the categorisation of activists’ efforts: on-the-ground, online, solidarity, and legislative efforts. 4.1 case background: apache stronghold since 2005, the san carlos apache tribe has stopped 13 congressional attempts to transfer sacred land known as oak flat, located near superior arizona, to the mining company resolution copper. however, on december 4, 2014 congressional action bypassed federaltribal government-to-government consultation policy and passed the southeast arizona land exchange and conservation act of 2013 (h.r. 687) as an amendment (i.e. additional provision) to the national defense authorization act for fiscal year 2015 (ndaa). the provision was made publicly known at 11:00pm on december 2, 2014, the night before the house of representatives vote. san carlos apache chairman terry rambler immediately criticized the last-minute provision (toensing, 2014), a white house petition on ‘we the people’ was created on december 4th, and the hashtag #apachelandgrab was first tweeted on december australasian journal of information systems deschine parkhurst 2017, vol 21, research on indigenous ict protecting oak flat 7 5th with a call to action and a url link to the white house petition. on december 19th, the ndaa was signed into law. in january 2015, the san carlos apache tribe announced a four-pronged approach to save oak flat that includes grassroots efforts, legislative actions, legal actions, and educational outreach campaigns (toensing, 2015; rambler, 2015). physical occupation of oak flat by grassroots activists started february 9th and by march these activists became known as the apache stronghold per the creation of a tumblr microblog and creation of accounts on other social network platforms. apache stronghold continues to pursue on the ground activist efforts coupled with digital activism to raise awareness about efforts to protect oak flat. although apache stronghold officially formed in 2015 on social network platforms, others primarily used twitter as early as 2011 to bring attention to the congressional activity about oak flat. 4.2 structure of apache stronghold social media posts the structure of apache stronghold’s social media posts such as facebook statuses and tweets could be generally described as having the following attributes: hashtags, url links, embedded visual/audio attachments, and text and symbols. social media posts at times included cross-platform url links to share content to and from multiple social network sites such as youtube, vimeo, instagram, facebook, twitter, and tumblr. the structure of social media content helps to understand how activists utilise a combination of social media tools to disseminate information, engage viewers in content, and to document on-the-ground activist efforts. reviewing the attributes of social media posts further gives insight into the interplay between activist efforts online, across social media platforms, and on the ground. figure 2 is an example of an apache stronghold twitter post, which includes both hashtags and a visual attachment. figure 2. screenshot of apache stronghold’s first twitter post from april 20, 2015. the post is a combination of four hashtags and an image macro (i.e. meme). the image macro depicts four hummingbirds, cross symbols, and the text “protect oak flat”. text posts are generally more informative in nature to disseminate statements, press releases, information about congressional efforts, challenging claims that minimise the importance of an issue, and may include calls to action. calls to action could be categorised into the following calls: supporting petitions, calling congressional leaders or local organisations, sharing of posts, fundraising requests, and shows of solidarity. table 1 lists four examples of apache stronghold’s social media text-only posts on facebook and twitter. australasian journal of information systems deschine parkhurst 2017, vol 21, research on indigenous ict protecting oak flat 8 @protectoakflat “the rain and thunder came after a weekend of singing! we are also joined by nihi gaal bee iina diné walkers #blessings #protectoakflat” (twitter, may 4, 2015). @protectoakflat: “thank you for sharing, the youth are a big reason of why we need to fight for the sacred land. #saveoakflat” (twitter, june 24, 2015). @protectoakflat: “#saveoakflat as #holyplace for #apache // sign #petition to make it a national monument. http://wh.gov/iicko #protectoakflat” (twitter, august 5, 2015). @apache-stronghold-802193869856079/ “++++urgent call to action++++ as we prepare to enter dc tonight. we got word from washington that we need you to call sen. heinrich from new mexico, sen. franken from minnesota, senator testor from montana. we need you to call them if they represent you! we need them to cosponsor the save oak flat act in the senate!! if so the repeal could be introduced by thursday!! ++++ thank you for your support!” (facebook, november 3, 2015) table 1: examples of apache stronghold’s social media text posts on facebook and twitter visuals such as photographs or videos are also included in apache stronghold’s social media posts, which engage and speak to viewers on a different level than text-only posts. videos may convey individual testaments of the cultural significance of land, connection to land, and spirituality, the landscape of oak flat, or document on-the-ground activist efforts. photographs may include moments of movement such as individuals on a walk, or drumming and singing, individual or group pictures, and pictures of oak flat landscape. figure 3 is an example that depicts the structure of an apache stronghold facebook post that includes text and a video attachment. australasian journal of information systems deschine parkhurst 2017, vol 21, research on indigenous ict protecting oak flat 9 figure 3. screenshot of apache stronghold’s facebook post from october 31, 2015. this is an example of a post with text and a video attachment. the video depicts on-the-ground activist efforts of the person filming and following another person into an elevator and asking the people in the elevator a question. the text depicts a call to action for people to sign a petition and to contact congressional leaders. memes are a variation on photo posts, which include text right on the image, which are also very effective at communicating a specific message. popularly known as memes, in the scholarly literature, memes are more accurately known as image macros. an example of a popular stylised image macro includes an image of an apache female youth in traditional regalia often worn in the apache sunrise dance, a coming-of-age ceremony, with the text “protect sacred oak flat” (see figure 4). lastly, there appears to be an increase in the inclusion of apache youth activist naelyn pike in photo posts. naelyn has appeared in almost all of the video attachments i sampled. australasian journal of information systems deschine parkhurst 2017, vol 21, research on indigenous ict protecting oak flat 10 figure 4. image macro from apache stronghold’s facebook post on march 15, 2015. the image macro depicts an apache female youth in traditional regalia often worn in the apache sunrise dance and is accompanied with the text “protect sacred oak flat”. 4.3 emerging themes. some of the emerging themes from social media content surrounding oak flat activism include the dissemination of information (e.g. negative consequences to the land, challenges to economic claims, history of legislative efforts to exchange the land), showcasing culture and spirituality (e.g. spiritual run, traditional practices), spiritual significance of the land, importance of youth, as well as contemporary acts of resistance (e.g. occupation of oak flat, navajo nation sit-in). emergent themes in the content of apache stronghold’s social media point to the overarching presence of culture and spirituality across themes. for example, the emerging theme ‘showcasing culture and spirituality’ came about because of social media content that depicted or discussed the active expression of cultural heritage or spirituality, such as photos of people holding items of cultural significance. however, the presence of culturally significant items did not exclude social media content from contributing to other emerging themes. for example, figure 1 of naelyn pike holding a bow and arrow in the new york times square was an example of the emerging theme ‘acts of resistance’ (see table 2). this photo depicts naelyn wearing moccasins and a camp skirt standing in what appears to be an archery stance with her hands gripping a bow and holding australasian journal of information systems deschine parkhurst 2017, vol 21, research on indigenous ict protecting oak flat 11 the arrow on the bowstring (the arrow points downward). the symbolism and active presence of naelyn standing in this manner with items of cultural significance and the skyscrapers in the background portrays naelyn as a protector ready for what comes her way. table 2 lists examples of emerging themes observed through the digital ethnographic process. dissemination of information showcasing culture and spirituality spiritual significance of land acts of resistance sharing the link to the ‘we the people’ petition titled “stop apache land grab” spiritual runs, ceremonies, and blessings image macro of hummingbirds and symbols of stars with the text “protect oak flat” occupation of oak flat and living off the land youtube video describing the specific type of mining to be conducted and the impact to the land image macro of an apache female youth in traditional regalia often worn in the apache sunrise dance, a coming-of-age ceremony descriptions about oak flat’s purpose in apache sunrise dance, a coming-of-age ceremony apache stronghold activist naelyn pike standing in times square holding a bow and arrow apache stronghold website documenting the history of the san carlos apache tribe’s efforts to save oak flat virtual calling out (i.e. shaming) as a cultural practice of social control and accountability photos and videos of prayers, dancing, singing, and gatherings while in oak flat women and youth prominently placed in the front row of marches and photo opportunities sharing event information and where apache stronghold or san carlos apache leaders will be speaking pictures of people with items of cultural significance stories about the san carlos apache people’s ties to the land connecting with and supporting other american indian activist groups table 2: examples of emerging themes from social media content related to oak flat activism 4.4 hashtags hashtags are keywords or a phrase preceded with the hash symbol (#) that people use in their social media posts either to connect the post to similar posts (small, 2011), or to express identification or membership in a group (cunha, magno, goncalves, cambraia, & almeida, 2014). the widespread use of hashtags originated on twitter but has extended to other social media platforms such as facebook, tumblr, and instagram. social media pertaining to oak flat activism were generally accompanied by hashtags. i identified four primary hashtags and four supplemental hashtags that were associated with oak flat activism (see table 3). according to a search on topsy.com, a social media analytics search engine, the first tweet using the movement’s common hashtag #saveoakflat was created in 2011 by earthworks, a nonprofit organisation dedicated to protecting the environment from mineral and energy development. the tweet included information about a congressional hearing taking place on the southeast arizona land exchange and conservation act. identifying the first use of the hashtag, #saveoakflat, further highlights that the issues surrounding the oak flat area were being raised on social media since 2011. other uses of the movement’s common twitter hashtags share information about critical events related to oak flat and to call on the audience to take action. the hashtag #apachelandgrab was first tweeted on december 5, 2014 with a call to action and a link to the ‘we the people’ petition. the hashtag #protectoakflat was first used on february 9, 2015 in conjunction with australasian journal of information systems deschine parkhurst 2017, vol 21, research on indigenous ict protecting oak flat 12 three other hashtag variants (#occupyoakflat, #saveoakflat, #oakflat), with no other text and a photo attached. the photo depicted land, trees, and a campsite assumed to be located within oak flat. posts may also be accompanied by the supplemental hashtags, which connect activism on oak flat to a larger phenomenon of native american digital activism, connecting this specific cause to other local, national, and international issues on the exploitation of land, desecration of other indigenous sacred sites, and a larger movement of solidarity in indigenous issues. table 3 lists examples of primary and supplemental hashtags observed through the digital ethnographic process. primary hashtags supplemental hashtags #apachelandgrab #occupyoakflat #saveoakflat #protectoakflat #nativepride #honorthetreaties #protectsacredland #respectexistenceorexpectresistance table 3: examples of hashtags associated with oak flat activism 4.5 types of activist efforts critical events surrounding oak flat activism could be categorised into the following types of activist efforts: on-the-ground activism, online efforts, solidarity efforts, and legislative support (see table 4 for examples). while i categorise certain critical events as a specific type of effort, critical events may be characteristic of more than one type of effort. for example, i categorised the navajo nation sit-in, where oak flat activists joined navajo activists to protest arizona senator john mccain, as a solidarity effort because the action took place on the navajo nation in coordination with navajo activists. this critical event could also be categorised as on-the-ground activism because of the in-person interaction of activists with each other, with the community, and with senator mccain. table 4 lists examples of activist efforts by category as observed through the digital ethnographic process. on-the-ground activism online efforts solidarity efforts legislative support physical occupation of oak flat creation of we the people petition titled “stop apache land grab” on the white house civic engagement platform walk to save oak flat – mother and daughter team who walked from oregon to oak flat in a stand of solidarity h.r.2811, save oak flat act – call of action to support the legislation, and activists meeting with the sponsoring arizona congressman raúl grijalva awareness event with activist speakers, art, music video titled “protect oak flat: saving apache sacred grounds” uploaded to vimeo navajo nation sit-in – oak flat activists joined navajo activists to protest arizona senator john mccain s.2242, save oak flat act – call to action to support the legislation, and activists meeting with the sponsoring vermont senator bernie sanders spiritual run and ceremony in front of the u.s. capital in washington dc (this is one type of event on apache stronghold’s journey to washington dc virtual calling out (i.e. shaming) of an arizona local native american non-profit organisation for accepting funding from resolution copper 18-day fast by allies in solidarity with the san carlos apache tribe and apache stronghold naelyn pike testifying in a house of representatives executive session forum on protecting native american sacred sites australasian journal of information systems deschine parkhurst 2017, vol 21, research on indigenous ict protecting oak flat 13 on-the-ground activism online efforts solidarity efforts legislative support naelyn pike speaking at a presidential candidate bernard sanders rally creation of online crowd funding page to support apache stronghold’s journey to washington dc table 4: examples of types of activist efforts associated with oak flat activism 5 discussion the findings of this study reveal narratives of survivance (vizenor, 1994; vizenor, 1999) and the celebration of survival (smith 2012) as framed by oak flat and apache stronghold activists as expressed via social media. smith (2012) describes that the celebration of survival is a way to document the active resistance of colonialism. celebrating survival can be further seen as celebrating both “survival and resistance”, or survivance (smith, 2012, p.146) and can appear as stories, arts, music, or events where there is celebration of life, all of which have been observed in social media surrounding oak flat activism. gerald vizenor (2008) describes multiple narratives of native survivance that reframe native existence from that of victimhood to activist resistance. in narratives of survivance, natives are not victims of colonisation but rather are resisters who have survived colonialism. and so, how might survivance be portrayed in digital activism to influence social change? considering this approach, the overall implication is that american indian digital activism blurs the concept of contemporary activism, not just in the interplay of online and on-the-ground activist efforts but also in reframing the dimensions of social change that are possible through digital activism. this is found in three ways: the first being that social media is more than just an activist tool for the amplification of voice, dissemination of information, and coordination of collective actions, but also as tool to showcase survivance. the second is that one intended social change of american indian digital activism is not just to influence individuals within social networks so they can influence political actors, or be more civically engaged, but also to influence survivance among individuals in their networks and communities, or, in other words to heal. the third is that to celebrate survival is also to understand the important role that american indian youth, such as naelyn pike, have in american indian activism; american indian youth are the ultimate form of colonial resistance as they represent life and a specific indigenous future orientation. these narratives can be observed through apache stronghold’s activist efforts as expressed through on-the-ground and online efforts. 5.1 american indian digital activism is survivance in this study, apache stronghold’s activist efforts as portrayed through social media include many aspects of contemporary activism. specifically, such on-the-ground activist efforts are identified as physical occupation of space such as sit-ins, civic activism such as demonstrations and petition signing, efforts to influence the legislative process, and efforts to increase awareness about the cause to save oak flat. while the findings indicate an interplay between online (e.g. the creation of petitions in an online civic engagement platform, information dissemination) and on-the-ground activist efforts, further examination of the themes of apache stronghold’s social media reveal another aspect of american indian activism that can be observed in both activist efforts on-the-ground and online, that of survivance. drawing from smith, survivance “accentuates the degree to which indigenous peoples and communities have retained cultural and spiritual values and authenticity in resisting colonialism” (2012, p.146). survivance has been defined as “survival and resistance” (smith, 2012, p.146), but is an interpretation of vizenor’s description of survivance, which is “an active sense of presence over absence, deracination, and oblivion; survivance is the continuance of stories, not a mere reaction, however pertinent” (vizenor, 2008, p.1). while vizenor discusses australasian journal of information systems deschine parkhurst 2017, vol 21, research on indigenous ict protecting oak flat 14 survivance as literary theory and practice, survivance was also observed in the social media content surrounding the efforts of oak flat activists. the findings of the emerging themes of apache stronghold’s social media content showcase aspects of cultural and spiritual survival and the continuance of native stories; it showcases activist efforts are an active presence. the role of culture in american indian activism is then perhaps more deep and complex than simply observing culture in activist efforts but rather the active presence of culture, its survival from colonial attempts to dominate and assimilate american indian peoples and the continuance of culture, is activism. for example, figure 1 depicts naelyn pike wearing moccasins, a camp skirt, and holding a bow and arrow. at the surface one could make an observation that the outfit is fashion, or a way to express cultural pride, but it is an active presence of culture compared to the display of cultural artifacts in a museum. the active presence of culture and spirituality were observed through apache stronghold social media content that depicted spiritual runs, the use of traditional objects in walks, traditional regalia worn by activists who are in photos and videos, speaking traditional language in videos, symbols of life in media content such as hummingbirds and stars, and the spiritual connection to place and land as described by activists. not only have cultural and spiritual values been retained but the active presence of culture also celebrates the survival of a group of people. 5.2 american indian digital activism is healing communities research on digital activism has focused on how the internet and social networking sites are used to coordinate activists’ efforts, build networks (juris, 2005), constrain the exposure of messages outside of ideological clusters (himelboim, mccreery, & smith, 2013), and influence social movement coverage (malinick, tindall, & diani, 2013). even if the use of social networks only increases the exposure of a cause within like-minded clusters, gaining access to such a network could prove beneficial to oak flat activists if it results in gaining supporters who may be willing to click a button, sign an online petition, or otherwise proactively contribute to the overall cause. the findings indicate that oak flat and apache stronghold digital activism and collective action mobilisation has received a fair amount of media attention from native american focused news outlets, local news outlets, and national news outlets (e.g. coverage in indian country today media network, phoenix new times, new york times). the movement has gained support from prominent political actors such as senator bernie sanders and congressman raúl grijalva, and even received a response from the ‘we the people’ white house petition from jodi gillette, special assistant to president barack obama for native american affairs, titled “we’ll work with tribes to protect sacred land” (gillette, 2015). the use of hashtags to connect apache stronghold’s network to larger indigenous movements and connect oak flat activists with other american indian activist groups such as the american indian movement, moccasins on the ground, indigenous environmental network, and tó bei nihi dziil (water gives us power), presents an interesting insight into how american indian digital activism influences social change within networks and beyond the desired influence on political actors. the findings of this study suggest that oak flat activism has made considerable contributions to influence political actors who have the means to implement policies that can save oak flat. however, approaching the findings from the standpoint of celebrating survival further suggests that macro social change, healing communities, can also come from influencing one’s own networks, including within social networks that center the survivance of indigenous peoples. for example, jacob (2013) discusses how strengthening yakama culture through revitalisation efforts is not only an act of bringing back into practice culture, traditions, and language, but also is an act of healing from the destructions of colonisation. apache stronghold showcasing the survival and resistance of their culture and people with their network of american indian activist groups is thus promoting the healing of indigenous peoples in their own cultural ways. it is empowering to realise that the collective action of many indigenous groups healing through activism represents the basis of a societal change. australasian journal of information systems deschine parkhurst 2017, vol 21, research on indigenous ict protecting oak flat 15 5.3 american indian digital activism is a future orientation this study brought into focus how apache stronghold celebrates survival through apache culture and how youth are the embodiment of life, which is the ultimate form of colonial resistance. the presence of american indian youth was an emerging theme in apache stronghold’s social media content. in particular, naelyn pike, an american indian youth activist of apache stronghold was a prominent and actively present figure in oak flat activism. naelyn was also observed as an emerging spokesperson and she appears in many photos where she appears to be speaking to audiences of many sizes, including the u.s. congress, and in videos talking about the significance of oak flat to the san carlos apache people. while naelyn is the most recognisable youth in social media content surrounding oak flat activism, other youth are featured at the start of spiritual runs and marches, and are photographed being present in oak flat. the significance of american indian youth in social media activism cannot be overstated. symbolically, youth in american indian cultures represent life, the continuance of people and their stories, and the future of a tribe. for example, the seven generations philosophy refers to the intergenerational teaching and planning of an indigenous community keeping in mind that the impacts of such actions today will reverberate and be felt by future generations (jacob, 2013). youth then are an important part in the transmission of cultural knowledge and survival of traditions and future communities. 6 conclusion in sum, this case study provides a snapshot of aspects shaping contemporary american indian activism in digital and on-the-ground moments, in particular with regard to a single environmental rights issue, #saveoakflat. this study provides insight into the content of social media that are shared by apache stronghold activists in their efforts to protect oak flat as well as different groups that activists are working with, possibly from different networks. oak flat activists are connected with local and national activist groups, organisations, and congressional members who are in some form supporting the efforts to save oak flat. activism around oak flat is also the combination of on-the-ground and online efforts and a demonstration of resistance. future studies may incorporate observations of on-the-ground efforts and look further into the relationships and connections between apache stronghold and their allies, which may provide a thorough and rich analysis of the phenomenon to protect oak flat. this study also highlights the importance of the timeline and the mapping of critical events in social media studies, and in particular with regard to indigenous activism. as of this writing, the current donald j. trump presidency in the united states has engaged in a number of acts, including the crafting of a series of executive orders and efforts to eliminate key policy and regulatory measures, that threaten various contemporary american indian activist and environmental justice movements in the u.s. thus, the timeline also provides context into the form of government shaping the digital activism of american indian activists in certain historical moments. overall, understanding the interplay between specific cases of contemporary american indian activism and the use of social media to express narratives of survivance, and culture and survival makes room for understanding the decolonisation of colonial spaces such as the internet and social media platforms (gaertner, 2015). centering the expressions, goals and experiences of american indian activists while analysing social media data is one way that social media researchers can better understand dimensions of social change through digital means. references amerman, s. k. (2003). “let’s get in and fight!”: american indian political activism in an urban public school system, 1973. the american indian quarterly, 27(3&4): 607-638. australasian journal of information systems deschine parkhurst 2017, vol 21, research on indigenous ict protecting oak flat 16 apache stronghold. (2016, february). once again, the fight for religious freedom in american begins. retrieved from http://apache-stronghold.com/about.html barberá, p., jost, j. t., nagler, j., tucker, j. a., & bonneau, r. (2015). tweeting from left to right: is online political communication more than an echo chamber? psychological science, 26(10): 1531-1542. brainerd dispatch. (2017, january 17). mille lacs band splits business from wells fargo. brainerd dispatch. retrieved from http://www.brainerddispatch.com/news/4200190mille-lacs-band-splits-business-wells-fargo chappell, b. (2017, february 8). 2 cities to pull more than $3 billion from wells fargo over dakota access pipeline. npr. retrieved from http://www.npr.org/sections/thetwoway/2017/02/08/514133514/two-cities-vote-to-pull-more-than-3-billion-from-wellsfargo-over-dakota-pipelin conner, t. w. (2014). exploring voting behavior on american indian legislation in the united states congress. the social science journal, 51: 159-166. corbin, j. & strauss, a. (2015). basics of qualitative research. techniques and procedures for developing grounded theory. fourth edition. los angeles: sage. cunha e., magno, g., goncalves, m. a., cambraia, c., & almeida, v. (2014). he votes or she votes? female and male discursive strategies in twitter political hashtags. plos one, 9(1). doi: 10.1371/journal.pone.0087041 democracy now!. (2016, november 16). worldwide protests demand u.s. army reject dakota access pipeline. democracy now!. retrieved from https://www.democracynow.org/2016/11/16/headlines/worldwide_protests_demand _us_army_reject_dakota_access_pipeline dreher, t., mccallum, k., & waller, l. (2015). indigenous voices and mediatized policymaking in the digital age. information, communication & society, 19(1): 23-39. federal register. (2000, november 6). executive order 13175 of november 6, 2000. consultation and coordination with indian tribal governments. vol 65 (218): 6724967252. washington, dc: federal register gaertner, d. (2015). indigenous in cyberspace: cyberpowwow, god’s lake narrows, and the contours of online indigenous territory. american indian culture and research journal, 39(4): 55-78. gillette, j. (2015, january 9). official the white house response to “stop apache land grab”, we’ll work with tribes to protect sacred land. retrieved from https://petitions.whitehouse.gov/response/well-work-tribes-protect-sacred-land graeff, e., stempeck, m. & zuckerman, e. (2014). the battle for ‘trayvon martin’: mapping a media controversy online and off-line. first monday, 19(2). doi:10.5210/fm.v19i2.4947 greenleigh, i. (2013). the social media side door: how to bypass the gatekeepers to gain greater access and influence. new york, ny: mcgraw-hill. grinberg, e. (2016, november 4). sarandon asks you to empty bank account to protest dakota pipeline. cnn. retrieved from http://www.cnn.com/2016/11/03/entertainment/susansarandon-dakota-pipeline-petition/ harlow, s., & guo, l. (2014) will the revolution be tweeted or facebooked? using digital communication tools in immigrant activism. journal of computer-mediated communication, 19(3): 463-478. himelboim, i., mccreery, s., & smith, m. (2013). birds of a feather tweet together: integrating network and content analyses to examine cross-ideology exposure on twitter. journal of computer-mediated communication, 18: 154-174. australasian journal of information systems deschine parkhurst 2017, vol 21, research on indigenous ict protecting oak flat 17 jacob, m. m. (2013). yakama rising: indigenous cultural revitalization, activism, and healing. tucson, az: the university of arizona press. jewell, s. (2014, december 19). statement by interior secretary sally jewell on the national defense authorization act for fiscal year 2015. [press release]. retrieved from http://www.doi.gov/news/pressreleases/statement-by-interior-secretary-sally-jewellon-the-national-defense-authorization-act-for-fiscal-year-2015.cfm john, s. (2015). idle no more – indigenous activism and feminism. theory in action, 8(4): 38-54. juris, j. s. (2005). the new digital media and activist networking within anti-corporate globalization movements. the annals of the american academy of political and social science, 597: 189-208. knopf, k. (2010). “sharing our stories with all canadians”: decolonizing aboriginal media and aboriginal media politics in canada. american indian culture and research journal, 34(1): 89-120. lenhardt, c. (2016). “free peltier now!” the use of internet memes in american indian activism. american indian culture and research journal, 40(3): 67-84. loeffelholz dunn, t. (2016, december 6). the numbers that tell the story of this standing rock victory. yes! magazine. retrieved from http://www.yesmagazine.org/peoplepower/numbers-tell-the-story-of-the-standing-rock-sioux2019s-victory-20161204 malinick, t. e., tindall, d. b., & diani, m. (2013). network centrality and social movement media coverage: a two-mode network analytic approach. social networks, 35: 148-158. medina, d. a., & davis, r. (2016, december 9). veteran finds forgiveness, peace in standing rock fight. nbc. retrieved from http://www.nbcnews.com/storyline/dakota-pipelineprotests/veteran-finds-forgiveness-peace-standing-rock-fight-n694226 moscato, d. (2016). media portrayals of hashtag activism: a framing analysis of canada’s #idlenomore movement. media and communication, 4(2): 3-12. rambler, t. (2015, january 27). all nations gathering. indian country today media network. retrieved from https://indiancountrytodaymedianetwork.com/2015/01/27/allnations-gathering-158879 revkin, a. c. (2015, july 17). from times square to the capitol, apache protestors fight u.s. land swap with mining company. the new york times. retrieved from https://mobile.nytimes.com/blogs/dotearth/2015/07/17/from-times-square-to-thecapitol-apache-protestors-fight-u-s-land-swap-with-mining-company/ richland, j. b. & deer, s. (eds.). (2010). introduction to tribal legal studies (2nd ed.). lanham, md: alta mira press. risling baldy, c. (2016). the new native intellectualism. #elizabethcook-lynn, social media movements, and the millennial native american studies scholar. wicazo sa review, 31(1): 90-110. rockwell, s. j. (2010). indian affairs and the administrative state in the nineteenth century. cambridge, ny: cambridge university press. small, t. a. (2011). what the hashtag? a content analysis of canadian politics on twitter. information, communication & society, 14(6): 872-895. smith, l. t. (2012). decolonizing methodologies. research and indigenous peoples (2nd ed.). united kingdom: zed books. sewell, jr., w. h. (2005). logics of history: social theory and social transformation. chicago, il: university of chicago press. australasian journal of information systems deschine parkhurst 2017, vol 21, research on indigenous ict protecting oak flat 18 toensing, g. c. (2014, december 8). san carlos apache leader seeks senate defeat of copper mine on sacred land. indian country today media network. retrieved from http://indiancountrytodaymedianetwork.com/2014/12/08/san-carlos-apache-leaderseeks-senate-defeat-copper-mine-sacred-land-158181 toensing, g. c. (2015, january 27). san carlos apache leader: ‘what was a struggle to protect our most sacred site is now a battle’. indian country today media network. retrieved from http://indiancountrytodaymedianetwork.com/2015/01/27/san-carlos-apacheleader-what-was-struggle-protect-our-most-sacred-site-now-battle-158878 turner, c. (2005). politics of minor concern. oxford, uk: university press of america. u.s. census bureau. (2012). the american indian population: 2010. washington, d.c.: u.s. census bureau. retrieved from http://www.census.gov/prod/cen2010/briefs/c2010br10.pdf valenzuela, s (2013). unpacking the use of social media for protest behavior: the roles of information, opinion expression, and activism. american behavioral scientist, 57(7): 920-942. vigil-hayes, m., duarte, m., deschine parkhurst, n., belding, e. (2017). #indigenous: tracking the connective actions of native american advocates on twitter. 20th acm conference on computer-supported cooperative work and social computing (cscw 2017), february 25-march 1, 2017, portland, oregon. vizenor, g. (1994). manifest manners: postindian warriors of survivance. hanover, n.h: wesleyan university press. vizenor, g. (1999). manifest manners: narratives of postindian survivance. lincoln, ne: university of nebraska press. vizenor, g. (2008). survivance: narratives of native presence. lincoln, ne: university of nebraska press. walker, t. (2014, november-december). 8 top social media conversations, starting with #rockyourmocs. retrieved from http://www.nativepeoples.com/nativepeoples/november-december-2014/going-social/ wasser, m. (2015, august 17). john mccain confronted by oak flat protesters and chased off navajo nation. phoenix new times. retrieved from http://www.phoenixnewtimes.com/news/john-mccain-confronted-by-oak-flatprotesters-and-chased-off-navajo-nation-7570607 copyright: © 2017 deschine parkhurst. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems miah, mcgrath & kerr 2016, vol 20, design research for dss development design science for dss: publication trends 1 design science research for decision support systems development: recent publication trends in the premier is journals shah jahan miah college of business victoria university shah.miah@vu.edu.au g. michael mcgrath college of business victoria university don kerr faculty of art and business university of the sunshine coast abstract this paper presents a contemporary literature review of design science research (dsr) studies in the domain of decision support systems (dss) development. the latest studies in the dss design domain claim that dsr methodologies are the most popular design approach, but many details are still yet to be revealed for supporting this claim. in particular, it is important to thoroughly investigate the trends in either the form or deeper insights in use of dsr in this field. the aim of this study is to analyse the existing dss design science studies to reveal insights into the use of dsr, so that we can outline research agenda for a special issue, based on findings of analysis. we selected articles (from 2005 to 2014) that were published in seven selected premier is journals (ranked as a* in the abdc journal ranking). the selected 57 sample articles are representative of dss design studies that used dsr in theorising, designing, implementing, and evaluating dss solutions. we discuss the theoretical positions of dsr for dss development through six categories: dss artefacts, dsr methods, dsr views, user involvement, dss design innovations and problem domains. the findings indicate that new studies are needed to fill the knowledge gap in dss design science, for more solid theoretical basis in near future. keywords: design science research; dss; construction centric design science; human centric design science; is research 1 introduction decision support systems (dss) development has been well-recognised research in information systems (is) projects within the areas of unstructured, semi-structured and wellstructured problem domains. vital explorations have been viewed in terms of technological developments in dss such as using ontologies, human-centred techniques, simulations, data mining and business intelligence or analytic techniques. these approaches are designed to enhance the elicitation of decision problems and for designing effective support strategies (haghighi, burstein, zaslavsky & arbon, 2013; meensel, lauwers, kempen, dessein & huylenbroeck, 2012; mackrell, kerr, & von helens, 2009; miah, kerr and gammack, 2009; miah, ahsan and msimangira, 2013). despite this, few studies on the development of innovative it systems (e.g. a dss artefact) have explored the knowledge and understanding related to research methods needed for the design of such it artefacts. in particular, existing research are still emergent activities to produce understanding and knowledge on how design methods such as design science research (dsr) could be used for an effective it artefact design by presenting useful guidelines addressing the combined requirements of technologies, people and organisations. this concern was raised in a recent study by gregor and hevner (2013) where it was argued that design research has yet to attain its full potential with respect australasian journal of information systems miah, mcgrath & kerr 2016, vol 20, design research for dss development design science for dss: publication trends 2 to the effective development of various is applications, as socio-technical systems. this is because there are gaps in the understanding and application of dsr knowledge and relevant concepts. we believe that this concern needs to be explored within the domain of dss development. studies of dss development have had a long history of utilising dsr patterns (arnott & pervan, 2014, 2012). in one of their articles, arnott and pervan (2012) reported that of 1167 dss research papers, 362 were classified as being concerned with dss dsr studies. more recently, out of their analysis on 1466 dss articles, arnott and pervan (2014) suggested that the seven years (from 2003 to 2010) of dss research indicates a significant increase in dss dsr to almost half of published articles, and that this represents a rapid transition in dss research from a field based on statistical hypothesis testing and conceptual studies related to dss solution design, into a new state, where dsr has evolved into a vital research method. this implies that the subject of dsr within dss design studies is developing towards its maturity. it is therefore of paramount importance to investigate on dsr movements in dss specifically on what forms of dsr methods have been developed and effectively utilised in guiding the design process of dss artefacts over past is studies. simon (1969) introduced the term design science by distinguishing it from natural sciences in his well-known book, the sciences of the artificial. simon’s work suggested that design sciences are established on how to design or construct is solution artefacts (hevner, march, park and ram, 2004). the work on the view of design science influenced is researchers for the first time when nunamaker and chen (1990) and nunamaker et al. (1991) outlined a new design methodology for developing effective is artefacts. subsequently, new ideas were brought together on how dsr could be theorised, redefined and actualised in the field of is in order to achieve more effective understanding and design knowledge (iivari 1991; walls et al. 1992; march & smith, 1995). building on existing work, hevner et al. (2004) published the most-cited dsr framework (over 7,500 citations in google scholar at june 2016) published in mis quarterly, in order to provide a new shape of methodological understanding for conducting is design research. peffers et al. (2008) also proposed a dsr artefact design methodology that consists of six activity steps for conducting the is design accomplishments. is research has positioned itself between technology, organisational and managerial viewpoints (carlsson, 2006; orlikowshi & lacono, 2001) and a growing interest in design research is now evident in the global is community (vaishnavi & kuechler, 2007) and in operations research more widely (manson, 2006). a new view of dsr which is related to more holistic conceptualization of design has been offered by mckay, marshall and hirchheim (2012). in this, both behavioural and technical design components were recognized as being better for integration and for achieving improved design knowledge and understanding. such conceptualization would identify beyond the need for dsr with the notions of product and function that well catered for in the guidelines of hevner et al. (2004). the view described that there was a need for a focus on more immaterial notions such as systems, processes, organizations, user knowledge and experiences, on-going interactions, relationships and the situated meaning of things (mckay et al. 2012; stewart 2011). mckay et al. (2012) suggested that there were problems with dsr orthodoxy and demonstrated the breadth of ways in which is design could be conceived and, hence, the type of research that could be conducted. an argument was articulated suggesting that both construction-centric (cc) design (focusing largely on technical build-related activities) and human centric (hc) design perspectives – which placed emphasis on organization, people and the context of use) might together be better to inform is design activities and achieve good result in research outcomes for more user-uptake. this suggests a clear requirement of practitioner oriented view of dsr that combines both views of cc and hc. this results in a collective design to promote active engagement of practitioners, for instance. such a combined approach has been termed an ensemble artefact design view for dss development (miah and gammack, 2014); it includes the content, context and processes (ccp) as defined by mackrell and mcdonald (2014). for instance, miah, gammack and kerr (2007) introduced requirements of context-sensitive dss design method that may better capture end-users’ australasian journal of information systems miah, mcgrath & kerr 2016, vol 20, design research for dss development design science for dss: publication trends 3 contextual realities. this reinforces us to explore on dsr in particularly for bringing more insights of dsr methods used in the existing dss design literature. in order to investigate the insights on use of dsr, we conducted a bibliometric content analysis adopted from arnott and pervan (2014). for the analysis we selected the seven premier is journals in which dss studies are most frequently published. the selected journals hold higher impact factors (we consider if >= 2.33) and were used for sample collection in the study by arnott and pervan (2014). from the journals, 57 sample papers were collected for our analysis. we attempted to answer the following questions. • what are the particular types of dsr methods used in dss design studies? • what are the types of dss artefacts designed in dsr studies for dss design? the results of this study promise to extend the use of dsr in dss design further, to justify selecting this as a theoretical basis for the use of dsr approaches, and to identify further research directions for outlining a call for paper submission on a special issue in the particular dss dsr research domain. this review paper is structured as follows: the next section describes the background of the study while the section 3 describes the methodological details. the section 4 presents the findings of the literature review and the section 5 describes the elected articles published in the special issue and further avenues of studies in the field. finally, the section 6 provides a discussion and summarises the paper. 2 study background dss is well known class of is applications that have a relatively long past as a major and rapidly growing research field within the is discipline. dss development research focuses on system development activities that help to identify, develop, innovate, implement, support and improve organisational, managerial, operational and other decision-making practices. the application of dss development techniques covers various aspect of organisations and businesses; including, for example, public health (walczak, pofahl & scorpio, 2002), agriculture (walker, 2002; kerr, 2004; miah, 2008), and finance (muntermann, 2009). in a comprehensive literature review, hosack, hall, paradice and courtney (2012) suggested that dss development research needed to shift its focus to the delivery of more “customer-centric” solutions. this implies that a significant research challenge still remains for dss design, especially with dss artefact design, to ensure that appropriate decision support features are available to provide adequate support for target stakeholders. the diversity of dss artefact has been defined in four dimensions: construct, model, method and instantiation (arnott & pervan, 2012). the analytical study by arnott and pervan (2012) called for immediate research attention to be directed towards dss research design and its relevance to theory development for supporting more-useful dss artefact designs. over the past recent decades, dss has changed from a novel, cutting-edge (and somewhat inaccessible) technology to a point where advanced decision-making tools are now considered by many organisations to be essential components of their is architectures (arnott & pervan, 2012). while dss has continued to grow as a significant area of is, many studies attempted to employ and develop various development methodologies for meeting specific design needs. classical dss development methods such as user-centred approaches (ayed et al. 2010), evolutionary prototyping and participatory approaches (meensel et al. 2012) provide little support to system developers, especially on how to proceed with formal problem analysis and its conceptualization for producing views about the solution design. arnott (2006) described the problem for a dss developer; explicitly, how to conceptualize aspects of the decision task that need improvement during iterations of the evolutionary development process. arnott and pervan (2012) suggested that although many of the early dss studies involved designing and implementing innovative it artefact through classical development methodologies (for example, keen & gambino, 1983), lack of user relevance and contextual reflections were found as common issues on their solution design. hevner et al. (2004) exemplified growing interests australasian journal of information systems miah, mcgrath & kerr 2016, vol 20, design research for dss development design science for dss: publication trends 4 on dss design studies that indicate the success of the continuation in identifying and describing dsr in dss research. although the dsr view shows promise over the classical approaches of is development, the analysis of dss dsr using hevner et al.’s (2004) guidelines provides a basis for considering how to improve the quality and impact of dss design studies(arnott & pervan, 2012). as suggested, arnott and pervan (2012) indicated major five issues related to the research method (75 % of previous dss articles as being “weak” with respect to research methods), evaluation (42.3 % 0f dss articles did not undertake any form of evaluation), theorizing (the type of solution artefacts such as constructs, models, and methods – are considered as theory or components of theory). one of the strongest findings by arnott and pervan (2012) was that 66% of dss artefacts are instantiations1, and have a strategic focus (problem relevance, showed that 75% dss articles have been focused on operational management problems). arnott and pervan (2012) also described how the fifth major issue, user relevance, cuts across the first four issues (research method, evaluation, theorizing, and strategic focus). the above issues identified and described by arnott and pervan (2012) indicated a major gap between dss design research and practitioners’ world of practices. as discussed above, the majority of dss design is limited to developing problem solving techniques with the lack of adequate devotion at the practitioner’s contextual demands; in most of the cases, the user involvement and their contextual relevance were ignored in order to develop new knowledge around the dss solution artefact. dss development research has indicated the need to develop and employ more-rigorous methods for dss solution development (arnott & pervan, 2005, 2008, 2012; miah, kerr and von helens, 2014; miah and gammack, 2014). this need can be addressed through a substantial exploration and improvement in design research. this may ensure for building understanding and knowledge around the quality and relevance in dss development, by accommodating the system and its users and acknowledging the context of its use (such as the key components of technologies, people and organisations) (miah et al., 2007, miah, 2009). we argue that it is imperative for us to explore the full impact of design research for dss development and reveal if there is a lack of proper utilisation, in “understanding of legitimate design research frameworks” and artefact design concepts when the dss studies are attempted to capture relevant organisational needs (hevner et al. 2004). to explore this issue we conducted a literature review to find the current status of the dss dsr (that may lead to outline effective research agenda for publishing a special issue) such as to identify current portions of dss studies that are using various forms of dsr such as cc, hc views, and contexts that have been highlighted. by doing this we should also have a better idea about why 75% of dss dsr is assessed as being weak with respect to the effective use of research methods. 3 methodology we utilised existing methodology for conducting our systematic literature review of studies on existing relevant dss design. kitchenham (2004) described the systematic literature review as a logical approach to identify, collect, evaluate and interpret information about current studies conducted in the target area of researchers’ interest. for conducting a literature review, is researchers have applied both qualitative and quantitative analyses by employing different methods. for example, al-debei and avison (2010) used content analysis in their examination of a business model in the field of is, while arnott and pervan (2012) explored the dsr for dss research articles employing a specific type of content analysis method called “bibliometric” content analysis. indulska and recker (2010) also conducted qualitative content analysis on dsr articles to establish the claims of these articles on the usage of the dsr paradigm in is research. in our study, we followed this same path for two purposes: 1) to 1 march and smith (1995) defined instantiation as a type of design artefact. this type of artefact represents full-functional application. the other types that are also described by many authors are model, method and construct (march and smith, 1995; hevner et al. 2004). australasian journal of information systems miah, mcgrath & kerr 2016, vol 20, design research for dss development design science for dss: publication trends 5 analyse dsr methods for dss development studies in previous research and 2) to identify dss artefacts and solution development strategies in previous research. we revisited published dss dsr studies and examined the progress of the dsr in the dss field from the year 2005 to 2014, for fulfilling our objectives specifically the purpose of developing agenda of special issue. figure 1 shows the overall procedure of our study reported in the paper. we found two main articles (such as arnott and pervan, 2014 and mckay et al. 2012) that provide us motivations for conducting the in-depth literature study in the area of dss dsr. after that, we select a suitable approach for conducting the literature study. the bibliometric content analysis (arnott & pervan 2014) was adopted to bring insights into our literature study, which comprised four phases: sample identification, sample study, sample analysis, and reporting outcomes. bibliometric content analysis is a type of analysis that involves “coding and analysis of a representative sample of research articles” (arnott & pervan, 2014, p.274). the approach provides a protocol for data capture that includes both quantitative and qualitative aspects. applying both qualitative and quantitative analyses, we report the findings of the analysis to outline schema of the special issue and further research direction. our analysis was focused on the following aspects: • does dss research use construction or human-centric or context-specific dsr types? • do design studies ensure user involvement in the design and evaluation activities? • what particular dsr method did they use? • what aspect did the studies target in their dss design innovation? • what application areas did they use to build the dss solution? the answers provide us to come up with possible topic areas that may have potentials for new research, but we also focused on other relevant topics that were not limited to the prime areas identified for publishing the special issue. figure 1: overall procedure of our study reported in the paper 3.1 sources of text sampling table 1 gives the list of journals from which we collected the sample articles (we refer as text sample) for our analysis. our approach is different from the arnott and pervan (2014) identify motivational requirements (arnott & pervan, 2014; mckay, marshall & hirchheim, 2012) conduct literature review (content analysis on published studies) report findings of analysis outline and organise special issue based on findings of analysis publish the special issue + reveal further research avenue australasian journal of information systems miah, mcgrath & kerr 2016, vol 20, design research for dss development design science for dss: publication trends 6 methodology as we used a different set of rules in the sample identification phase, mainly for qualitatively analysing dss development studies. we narrowed down the space for searching the text sample (articles), as we looked at published materials only in the premier is journals listed by arnott and pervan (2014) that are listed in the latest abdc (australian business deans council) journal ranking. we picked seven journals (out of 19 journals selected by arnott and pervan, 2014) that also hold higher impact factors – greater than 2.33 or above – and frequently published dss papers. the details of the selected journals are illustrated in table 1. we searched samples published in these journals with the following keywords: “dss and design science”, “design research” and “decision support”, “dss” and “design science research”, “design studies” and “dss”, “design theory” and “dss”, “decision support systems” and “design science”. the reason we chose journals from the list employed by arnott and pervan (2014) was to receive a validated and relevant sample of articles. however, more criteria were added to our process of selecting/collecting dss articles. these criteria were to focus on a) dss papers that are fully designed and evaluated, b) design studies in dss, c) the dss design for individual’s or managerial decision support rather than other forms of dss, and d) dss studies that discussed dss design innovations leading to design theories. the rationale for applying these criteria was to gather relevant and reliable findings to meet the specific demands of our study. we finally identified 57 sample articles out of 116 initially found relevant through our manual selection process (in our sample selection phase – table 2 illustrates the phases of text samples collection); this involved time consuming reading, so meaningful categories and/or concepts on the sample could be developed for their selection or rejection (in our sample analysis phase). journals acphis ranking if: impact factors arnott & pervan, (2014) dss design papers (1990 to 2010) our review: no of fullfunctional dss dsr papers (2005 to 2014) decision support systems (dss) a* 2.331 41 44 mis quarterly (misq) a* 7.497 7 2 decision sciences (ds) a* 3.146 8 2 management science (ms) a* 3.304 2 1 information and management (i&m) a* 3.796 10 1 information systems research (isr) a* 4.131 8 1 journal of information technology (jit) a* 3.000 5 6 total: 7 journals 81 57 table 1: selected journal details in which samples were collected australasian journal of information systems miah, mcgrath & kerr 2016, vol 20, design research for dss development design science for dss: publication trends 7 steps number of text samples collected number of text samples selected for the analysis activities details step 1: text sample identification 570 351 initial keyword searching and checking step 2: text sample selection 351 127 screening and reading activities step 3: text sample analysis 116 57 reading samples, and manual acceptance step 4: reporting on the text samples 57 57 reporting, sorting and generalising aspects that are of significance table 2: process of our literature review 3.2 text sample analysis after identifying the text sample, we selected the most appropriate labels for the most relevant samples that qualified with our criteria. we focused on each sample one by one (for reading and note taking) and assigned a code that defined the artefact type, method used and design theory etc. after finishing the labelling, we commenced reading each article and taking notes on relevant aspects as part of the analysis tasks in the second round. the process was time consuming and laborious; sometimes, re-reading was required to re-adjust our initial observation. however, this helped us to gain greater insights into each text sample and on the categories that are shown below. • dss artefact types: march and smith (1995) defined types of design artefacts as constructs, models, methods, and instantiations that are well-recognised in is design research. extending from this, gregor & hevner (2013) described an additional outcome of dsr studies, which is ‘design theory’ (as “an abstract, coherent body of prescriptive knowledge” (gregor & hevner, 2013, p. a3). a construct is a type of artefact that constitutes the problems specific language and solutions. a model is a type of artefact that uses the language to represent problems and solution. a method is a type of artefact that describes the processes and offers guidance on how to solve target problems. an instantiation is a type of artefact that explains problem-specific solution applications; that is, the aggregate of constructs, models, and methods (miah & gammack, 2014; hevner et al. 2004). the design theory is an abstract that explains prescriptive knowledge – describing the principles of form and function, methods, and justificatory theory that are used to develop an artefact (gregor and jones, 2007). • dsr methods: the methods (e.g. the guidelines) of design research address the requirements of innovative design development, solution modelling or problem solving, as well as the evaluation of artefact design (hevner et al. 2004; carlsson, 2007). • dsr views: we investigated on types of dsr views that were employed in design studies. mckay et al. (2012) described dsr views in terms of both construction-centric (cc) and human-centric (hc) design perspectives. the cc view is useful when the it artefact and its relevant technical qualities are of interest to the dsr researcher. these artefacts may become of interest to the is researcher working in the behavioural science paradigm. such artefacts are seen as “surrounded” by human (e.g. user-specific problem context) and their contingencies called the hc view. mckay et al. (2012) also suggested that such artefact design may also place emphasis not only on technology and people, but also on the context of use and content of the design artefact. such a australasian journal of information systems miah, mcgrath & kerr 2016, vol 20, design research for dss development design science for dss: publication trends 8 view, which can represent the content, context and process together, might better inform design activities and result in research outcome that will achieve more practical uptake (miah & mckay, 2016). • user involvement: “user participation” is often used interchangeably with “user involvement”. harris and weistroffer (2009) described from barki and hartwick (1989) that the term “user participation” should be employed when referring to the behaviours and activities of the target users in the systems development process. accordingly, “user involvement” has been referred to by harris and weistroffer (2009) as a psychological viewpoint of the individual user and has been defined as “the importance and personal relevance that users attach either to a particular system” (p.43) or to an is design in general. we assessed how the user involvement were ensured in the phases or cycles of the dss design studies. • dss design innovation aspect: hevner et al. (2004) described how dsr pursues the production of innovations representing ideas, practices, technical capabilities, and products through which the analysis, design, implementation, management, and use of is can be effectively and efficiently accomplished for addressing targeted issues. therefore, it was important for us to assess on how the dss design studies were transformed for offering value and quality into artefact design. • problem domain: problem domain refers to the set of task structures under consideration in designing a solution. some tasks of decision aids are people, process and product-focused issues. for instance, mckay et al. (2008) defined that it is important to describe organisational problems domain, in which dsr will concentrate on finding innovative and new solutions. hevner et al. (2004) described a construct as an artefact that enables the construction of models or full representations of the problem domain. such representation has a profound impact on the design of solution artefacts. we investigated different aspects of problem definitions that were defined in the dss design studies. 4 findings vital concerns of our text sample selection is that we found 77% of our samples of the dss design studies published in dss journal and rest of the 33% of our text samples were from the other six is journals. this indicates that decision support systems journal is the only vital outlet, which published complete form (full-functional design articles) of dss dsr over the past years. other journals such as journal of information technology also published some (lower proportions of) dss dsr articles. however, we show a comparison in the study by arnott and pervan (2014) for validating such a sampling predisposition. following table 3 shows the findings of our literature analysis on dss design studies. it suggested that about 65% of published dss design studies used the cc view and about 14% of studies used the hc view of dsr methodology for developing innovative dss artefacts. australasian journal of information systems miah, mcgrath & kerr 2016, vol 20, design research for dss development design science for dss: publication trends 9 aspects findings comments dss artefact types construct as artefact = 1.7% method as artefact = 49% model as artefact = 28% instantiation as artefact = 15% design theory as artefact = 14% the result represents the dss design artefact as a method of solution that was vital in the past, while other types of artefacts are still at embryonic stage in development. dsr methods hevner et al. 2004 = 47.61% march and smith, 1995 = 5.2% peffers et al., 2008 = 5.2% others action research based design studies = 30% the result shows that hevner’s seven design principles, as design method, were vital in use of dss design while other approaches such as the use of peffers’ method looks still at the developmental stage. some dss design studies (30%) used traditional approaches such as actionresearch-based design studies. dsr views cc = 65.52% context specific (hc + cc) = 19.20% hc = 14.02% this shows that dss design studies are mainly based on the cc view and, therefore, the practitioner’s perspective in the design may be ignored in most of the design cases. user involvement no: 61%; yes: 34.10% this shows that the majority of the studies did not engage the user in the development and evaluation processes. dss design innovation aspect higher = technical capabilities oriented innovations medium = product oriented innovations lower = ideas and practiceoriented innovations this shows that the majority of the studies did focus on technical improvement for innovating artefact design to offer value adding to user. problem domains people-focused issue =12 process-focused issue = 27 product-focused issue = 8 this shows that the majority of the studies did on artefact design by innovating the decision process improvement rather than addressing product and people (user) oriented issues table 3: findings of the literature review, with comments on each aspect. 5 special issues and further research avenues from the aforementioned six aspects, we generalise three key areas of interest in which more studies are needed to fill the knowledge gap in dss design science. the areas are dss artefact design theory & knowledge development (e.g. more hc related design theories), emerging & innovative dss artefact design (e.g. more problem and user specific dss solution design and its diversified innovations), and dss artefact evaluation (e.g. more theories on dss evaluation – enhancement of direct user involvement). by advocating these issues, we generated a call for paper submission in a special issue in the australasian journal of information systems (june 2015). in total, eight articles have been submitted to this special issue. two were declined as they were irrelevant to the main topic’s area of interest. therefore, only six articles qualified for the peer review process. after conducting three review cycles, only four articles were accepted for publication in the special issue. key similarities of these studies are that each study ensured direct user involvement throughout their design research processes. the details of the published articles are given below. article 1: an evaluation view of an ensemble artefact for decision support using action design research (dale mackrell & craig mcdonald) this article represented an example of contemporary studies in dss artefact evaluation research. the study investigated the integration of content, context and process (ccp) into the action design research (adr) framework to account for the interplay of organisational issues australasian journal of information systems miah, mcgrath & kerr 2016, vol 20, design research for dss development design science for dss: publication trends 10 in artefact design and development. the investigation was conducted through a case study in which successive ict student teams incrementally built, over several semesters, a tailored, low cost business intelligence system as an ensemble artefact for an organisation in the not-forprofit sector. during project development, ccp's human-centred approach to evaluation complemented adr's more prescribed technology-driven software testing. the integration of ccp into adr as an evaluation view offers a holistic approach to assessing an ensemble artefact. the resultant conceptual framework was presented as a model with an explication of unexpected design and research outcomes. article 2: construction and use of a ‘green growth’ tourism decision support system: a multi-model approach (g. michael mcgrath & geoffrey h. lipman) this article represented an example of innovative and emerging dss artefact design study. the article describes a dss artefact that was designed to support the development of “green growth” (gg) strategies for travelism (travel & tourism) destinations. a sound gg strategy is important: first, because tourism was a major contributor to the global economy – particularly for developing and island states; second, because it represented some 5% of greenhouse gas (ghg) emissions and these are increasing faster than the global norm; and third, because the environment is an essential element of destination attractiveness. thus, the problem domain is certainly non-trivial and important. the authors argued that the design of the dss artefact described was original and novel in the sense that i) it supported the entire gg strategy development process (which is actually cyclical), ii) it allowed for the sharing of data, functionality and knowledge between different dss applications and different strategy development exercises in a seamless, integrated manner, and iii) it will be deployed in a global community-based program in 2016. system design draws heavily on previous is, information management and software engineering research, particularly with regard to use of abstraction and interfaces in support of component sharing and reuse. article 3: development of a dss and online tools to support sleep disorder consultations using design science (jacqueline blake, don kerr & john gammack) this article represented another example of innovative and emerging dss artefact design study. it also describes the use of design science to develop and evaluate a dss incorporated into online tools to support a sleep physician’s diagnosis of sleep disorders. the design science approach offers guidance for developing a solution that ensures i) that the artefacts fit into their usage context, and ii) that development and evaluation principles that generalise to similar problem domains are identified. how this project fits in the design science guidelines was specified, thus demonstrating how this philosophy and methodology advance theory, and are particularly relevant to other, similar medical diagnostic domains. the usability and technical evaluation of the set of tools indicated provide clear practical benefits at patient, physician and organisational levels. article 4: a design science research methodology for expert systems development (shah j miah & hussein genemo) this article represented another example of innovative and emerging dss (in the form of an expert system) artefact design study. the aim of this study was to investigate the use of dsr for es design. first, we explored the es development literature to reveal the presence of dsr as a research methodology. for this, the study selected relevant literature criteria and applied a qualitative content analysis in order to generate themes inductively to match the dsr components. second, by utilising the findings of the comparison, the study determined a new dsr approach for designing a specific es that is guided by another result – the findings of a content analysis of examination scripts in mathematics. the specific es artefact for a case demonstration was designed for addressing the requirement of a “wicked” problem, in that the key purpose was to assist human assessors when evaluating multi-step question (msq) solutions. it is anticipated that the proposed design knowledge, in terms of both problem class and functions of es artefacts, will help es designers and researchers to address similar issues for designing information system solutions. australasian journal of information systems miah, mcgrath & kerr 2016, vol 20, design research for dss development design science for dss: publication trends 11 6 overall discussion we revealed three key areas of interest related to dss design science studies that are of paramount importance for the field. as mentioned earlier these areas are: dss artefact design knowledge, innovative dss artefact design, and dss artefact evaluation. these areas are emerging, but still limited for conducting more in-depth design studies on innovative and practitioners’ centric dss artefacts and their evaluation, as well as new knowledge and understanding growing on the emergent area of study. for example, future studies can explore how to build a dss meta-artefact as a general solution concept to address any particular class of problems, such as markus et al. (2002) developed a dss theory for supporting emergent knowledge processes in problem domain. this type of research may offer benefits within the field. although relevant dsr theories are introduced (e.g. iivari, 2015) for the exploration of a meta-artefact in is, appropriate investigation, application and modification of such approaches (only limited to iivari, 2015) are yet to be undertaken within the domain of dss design research. in addition, theories from previous empirical work in which dsr methods or strategies have been successfully applied can be an important aspect to develop in the form of new knowledge and that can be generalised as general solution concept (meta-artefact). in relation to research on dss artefact evaluation, although studies (phillips-wren et al. 2009; manyard, burstein and arnott, 2001) attempted to improve dss evaluation strategies, there is a clear need for conducting research investigation in many aspects to better address and accomplish organisational and decision makers’ behavioural issues that are vitally relevant for evaluating dss artefacts. based on this analysis of the existing dss design studies, we suggest that is researchers should explore and conduct more new studies in the three directions. for instance, in order to enhance theoretical knowledge, studies can focus on generalising understanding of specific problem areas in which decision support applications are developed. this will enable researchers to employ, extend, revisit, and develop holistic dsr methods and frameworks for dss. researchers could pay more attention and do investigation to understand human and construction-centric views, and to better incorporating them when addressing context-specific realities. coupled with the need for increasing technological flexibility in artefact design studies, and in order to re-adjust to rapidly changing user contextual and organisational factors, the range of dss design methods and approaches can be outlined to meet both the demand of socio-technical and behavioural factors for appropriate artefact design and relevant knowledge creation. the need is imperative for conducting new dss research that will enhance design knowledge and is design approaches in the field. further aims of research are articulated at supporting and recreating effective dss design knowledge. the exploration of design knowledge will further establish the dss field as a reference discipline. references al-debei, m. m. and avison, d. (2010). developing a unified framework of the business model concept, european journal of information systems, 19, 359-376. arnott, d. (2006). cognitive biases and decision support systems development: a design science approach, information systems journal, 16 (1), 55–78. arnott, d. and pervan, g. (2005). a critical analysis of decision support systems research, journal of information technology, 20 (2), 67–87. arnott, d. and pervan, g. (2008). eight key issues for the decision support systems discipline, decision support systems, 44, 657–672. arnott, d. and pervan, g. (2012). design science in decision support systems research: an assessment using the hevner, march, park, and ram guidelines, journal of the association for information systems, 13 (11), 923–949. arnott, d. and pervan, g. (2014). a critical analysis of decision support systems research revisited: the rise of design science, journal of information technology, 29, 269–293 australasian journal of information systems miah, mcgrath & kerr 2016, vol 20, design research for dss development design science for dss: publication trends 12 ayed, m., ltifi, h., kolski, c. and alimi, a. (2010). a user-centered approach for the design and implementation of kdd-based dss: a case study in the healthcare domain, decision support systems, 50, 64–78. barki, h. and j. hartwick (1989). rethinking the concept of user involvement, mis quarterly 13 (1), 53-63. carlsson, c. and turban, e. (2002). dss: directions for the next decade, decision support systems, 33, 05–110. carlsson, s.a. (2007). developing knowledge through is design science research: for whom, what type of knowledge, and how, scandinavian journal of information systems, 9(2), 75-86. creswell, j. w., hanson, w. e., clark plano, v. l., and morales, a. (2007). qualitative research designs: selection and implementation, the counselling psychologist, 35(2), 236-264 ellis, t. j., and levy, y. (2008). framework of problem-based research: a guide for novice researchers on the development of a research-worthy problem" informing science: international journal of an emerging transdiscipline, 11, 17-33. ercikan, k., and roth, w.m. (2006). what good is polarizing research into qualitative and quantitative?, educational researcher, 35(5), 14-23. fischer, a., greiff, s., and funke, j. (2012). the process of solving complex problems, the journal of problem solving, 4(1), 19-42 giarratano, j. c., and riley, g. (2005). expert systems: principles and programming, thomson course technology (4th ed), cambridge, mass. gregor, s. and hevner, a.r. (2013). positioning and representing design science research for maximum impact, mis quarterly, 37(2), 337-355 gregor, s., and jones, d. (2007). the anatomy of a design theory, journal of the association of information systems, 8(5), 312-335 haghighi, p. d., burstein, f., zaslavsky, a. and arbon, p. (2013). development and evaluation of ontology for intelligent decision support in medical emergency management for mass gatherings, decision support systems, 54, 1192–1204. harris, m.a. and weistroffer, h.r. (2009). a new look at the relationship between user involvement in systems development and system success," communications of the association for information systems , 24, 42. available at: http://aisel.aisnet.org/cais/vol24/iss1/42 hevner, a., march, s., park, j. and ram, s. (2004). design science in information systems research, mis quarterly, 28, 75–105. hosack, b., hall, d., paradice, d. and courtney, j.f. (2012). a look toward the future: decision support systems research is alive and well, journal of the association for information systems, 13, 315–340. iivari, j., (1991). a paradigmatic analysis of contemporary schools of is development, european journal of information systems, 1(4), 249272. iivari, j., (2015). distinguishing and contrasting two strategies for design science research, european journal of information systems, 24, 107–115 indulska, m. and recker, j. (2010). design science in is research: a literature analysis. information systems foundations, 285. keen p.g.w. and gambino t.j., (1983). building a decision support system: the mythical man month revisited, in j.l. bennett (ed.), building decision support systems, reading, ma: addisonwesley, 133-172. australasian journal of information systems miah, mcgrath & kerr 2016, vol 20, design research for dss development design science for dss: publication trends 13 kerr, d.v. (2004). factors influencing the development and adoption of knowledge based decision support systems for small, owner operated rural businesses, artificial intelligence review, 22 (2), 127-147. kitchenham, b., (2004). procedures for performing systematic reviews, joint technical report, keele university technical report tr/se-0401, national ict australia ltd, garden street, eversleigh, nsw 1430, australia available at: http://www.inf.ufsc.br/~aldo.vw/kitchenham.pdf mackrell, d., kerr, d.v. and von helens, l. (2009). a qualitative case study of the adoption and use of an agricultural decision support system in the australian cotton industry: the sociotechnical view, decision support systems, 47,143–153. mackrell, d. and mcdonald, c. (2014). incorporating content, context and process evaluation into action design research. information systems foundation workshop. the national centre for information systems research anu. manyard, s., burstein, f., and arnott, d., (2001). a multi-faceted decision support system evaluation approach, journal of decision systems 10 (3-4), 395–428. march, s.t. and smith, g.f., (1995). design and natural science research on information technology, decision support systems, 15, 251-266. markus, m. l, majchrzak, a. and gasser, l (2002). a design theory for systems that support emergent knowledge processes. mis quarterly, 26(3), 179–212. mckay, j., marshall, p. and hirchheim, r. (2012). the design construct in information systems design science, journal of information technology, 27, 125–139 meensel, j.v., lauwers, l., kempen, i., dessein, j. and huylenbroeck, g.v. (2012). effect of aparticipatory approach on the successful development of agricultural decision support systems: the case of pigs2win, decision support systems 54, 164–172. miah, s. j. (2008). an ontology based design environment for rural decision support, unpublished doctoral dissertation, griffith university, australia. miah, s. j. (2009). end user as application developer for decision support, proceedings of the americas conference on information systems (amcis), 142. retrieved 21 july, 2010 from: http://aisel.aisnet.org/amcis2009/142 miah, s.j. and gammack, j. (2014). ensemble artifact design for context sensitive decision support, australasian journal of information systems, 18 (2), 5-20. miah, s.j. and mckay, j. (2016). a new conceptualisation of design science research for dss development, proceedings of the 20th pacific asia conference on information systems (pacis 2016), taiwan. miah, s.j. kerr, d. and von-helens, l (2014). a collective artefact design of decision support systems: design science research perspective, information technology & people, 27 (3), 259-279. miah, s.j., ahsan, k., and msimangira, kab (2013). an approach of purchasing decision support in healthcare supply chain management, operations and supply chain management, 6 (2), 43-53 miah, s.j., gammack, j., and kerr, d. (2007). ontology development for context-sensitive decision support, proceedings of the third international conference on semantics, knowledge and grid (skg 2007), xian, china, 475-478 miah, s.j., kerr, d. and gammack, j. (2009). a methodology to allow rural extension professionals to build target specific expert systems for australian rural business operators, expert systems with applications, 36 (1), 735-744 australasian journal of information systems miah, mcgrath & kerr 2016, vol 20, design research for dss development design science for dss: publication trends 14 muntermann, j. (2009). towards ubiquitous information supply for individual investors: a decision support system design, decision support systems, 47 (2), 82-92. nunamaker, j.f. and chen, m. (1990). systems development in information systems research, ieee, 631-639. nunamaker, j.f., chen, m., and purdin, t.d.m. (1991). systems development in information systems research, journal of management information systems, 7(3), 89-106. orlikowski,w. j. and iacono c. s. (2001). desperately seeking the “it” in it research – a call to theorizing the it artefact, information systems research, 12 (2), 121-134 peffers, k., tuunanen, t., rothenberger, m., and chatterjee, s. (2008). a design science research methodology for information systems research, journal of management information systems, 24 (3), 45-77. phillips-wren, g., mora, m., forgionne, g.a. and gupta, j.n.d. (2009). an integrative evaluation framework for intelligent decision support systems, european journal of operational research 195, 642–652 simon, h. a. (1996). the sciences of the artificial (3rd ed.), mit press, cambridge, ma. stewart, s. (2011). interpreting design thinking, design studies, 32(6), 515–520 vaishnavi, v., and kuechler, w. (2008). design science research methods and patterns: innovating information and communication technology. auerbach publications, boston, ma. venable, j. and baskerville, r. (2012). eating our own cooking: toward a more rigorous design science of research methods, the electronic journal of business research methods, 10, 141-153. walczak, s., pofahl, w.e., and scorpio, r.j. (2002). a decision support tool for allocating hospital bed resources and determining required acuity of care, decision support systems, 34, 445–456. walker, h. (2002). decision support, learning and rural resource management, agricultural systems, 73, 113–127. walls j.g., widmeyer, g.r. and el sawy, o.a. (1992). building an information system design theory for vigilant eis. information systems research, 3(1), 36-59. copyright: © 2016 miah, mcgrath & kerr. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 study background 3 methodology 3.1 sources of text sampling 3.2 text sample analysis 4 findings 5 special issues and further research avenues 6 overall discussion paper title (paper title style) australasian journal of information systems oseni, foster, mahbubur, smith 2017, vol 21, research article framework for erp post-imp amendments 1 a framework for erp post-implementation amendments: a literature analysis taiwo oseni federation university australia t.oseni@federation.edu.au susan foster monash university mahbubur rahim monash university stephen patrick smith monash university abstract post-implementation amendments to erp systems (erp-pia) are of importance for advancing erp research, but more importantly essential if erp systems are to be used as a strategic and competitive business tool. for ease of clarity, we have adopted the term “amendments” to encompass the main forms of post implementation activities: maintenance, enhancements and upgrades. the term “amendments” is used to counteract one of the major findings from this research the inconsistency of terms used by many authors to explain post implementation activities. this paper presents a review of the erp post-implementation amendment literature in order to provide answers to two specific questions: first, what is the current state of research in the field of erp-pia; and second, what are the future research directions that need to be explored in the field of erp-pia. from the review, we develop a framework to identify: (a) major themes concerning erp post-implementation amendments, (b) inherent gaps in the post-implementation amendments literature, and (c) specific areas that require further research attention influencing the uptake of amendments. suggestions on empirical evaluation of research directions and their relevance in the extension of existing literature is presented. keywords: post-implementation modifications; erp; maintenance; enhancements; upgrades 1 introduction companies spend millions of dollars in maintenance and support activities involved with erp post-implementation. a gartner study (2003) identified that, on average, 11% of a company’s it budget allocation is spent on application support and maintenance. to put this figure into context, the authors state that, as a rough estimate, “organisations should plan to spend 40% to 60% of the initial implementation costs to maintain their erp system for every year that it is in full production”. this advice is still relevant today given that maintenance and upgrade costs are ever present in the yearly erp budget. client companies of some of the larger erp vendors, such as sap and oracle, can be charged hundreds of thousands if not millions of dollars in maintenance per year (olsaker, 2010). for example, a cfo research study involving 157 senior finance executives (cfo research services, 2009) confirms that the ongoing costs of maintaining, modifying, and updating an erp system is substantial, even for mid-sized organisations. in some cases, such cost run to about 17% to 22% of the initial implementation cost (olsaker, 2010). unfortunately, limited information is available to assist organisations in making informed decisions concerning the management of this cost after implementation. as a result, some organisations are finding that post-implementation amendments often do not justify the fees paid to vendors. for instance, dana, an $8 billion dollar auto parts supplier, was reported to have stopped making maintenance payments to its erp vendor and scaled back on erp australasian journal of information systems oseni, foster, mahbubur, smith 2017, vol 21, research article framework for erp post-imp amendments 2 maintenance projects in order to cut costs (nash, 2007). this type of behaviour is understandable; although post implementation amendments are required to exploit erp capabilities effectively, amendments are costly and time consuming to perform and may not produce any measurable benefit unless carefully managed. (peng & nunes, 2010). having a better understanding of post-implementation amendments is certainly a first step to ensuring successful uptake of amendments. in the academic and practitioner literature, the topic of erp post-implementation issues encompassing ongoing requirements, user support, maintenance and upgrades have received scant attention in comparison with adoption and implementation issues (law, chen, & wu, 2010). this is a considerable failing since erp implementation projects are almost invariably followed by maintenance, enhancement, and upgrade projects. post-implementation amendments are relevant and deserve adequate research attention, especially since future benefits obtainable from erp systems are moderated by post-implementation changes (cao, nicolaou, & bhattacharya, 2013). this paper presents a review of the erp-pia literature, a framework to analyse and synthesise prior research on post-implementation amendments, identifies the gaps in the literature and presents the major themes concerning erp post-implementation amendments. the outcomes offer a resource for researchers who may be interested in issues affecting erp systems postimplementation thus encouraging the undertaking of more rigorous research to further examine post-implementation issues. for the erp practitioner community, a review of the post-implementation amendments’ literature provides an in-depth knowledge of the critical issues surrounding such amendments. 2 a background to erp post-implementation amendments an enterprise resource planning (erp) system is a type of packaged application software that is capable of fully integrating and automating business processes across functional areas. in addition, erp systems are capable of centralising operational information so that it can be shared by an organisation’s key functional systems, and to facilitate rapid decision making, cost reductions, and greater managerial control (davenport, harris, & cantrell, 2004; galy & sauceda, 2014; klans, rosemann, & gable, 2000). these systems further offer provision for mobility with anywhere anytime access to an organisations’ resources as well as connectivity to more advanced enterprise systems such as: supply chain management (scm), customer relationship management (crm), advanced planning and optimisation (apo), and business intelligence (bi) (motiwalla & thompson, 2009). after the initial implementation of erp systems, companies are continually required to make post-implementation amendments. such amendments include maintenance, enhancement, and upgrade activities which are referred to in literature by a variety of terms such as: maintenance activities, maintenance practices, enhancements, changes, amendments revisions, and upgrades; making it difficult to disentangle the meanings with any clarity. to address the issue of inconsistent terminology, and reduce the terms to a common denominator we use the term amendments. in the following sections, we present the current state of research in the field of erp-pia. 3 a review of the erp-pia literature in this study, we have followed a three-step approach for reviewing the erp-pia literature: step 1 – identification of articles discussing amendments; step 2 development of a framework to guide the discussion on amendments; and step 3 discussion on amendments. this is presented in figure 1. australasian journal of information systems oseni, foster, mahbubur, smith 2017, vol 21, research article framework for erp post-imp amendments 3 future research directions for erp postimplementation amendments search for erp post-implementation amendments literature the development of a framework for analysing erp post-implementation amendments literature the discussion of erp post-implementation amendments literature step 3 step 1 step 2 led to led to produced figure 1. three-step approach to reviewing the erp-pia literature step 1 includes a wide search for publications documenting one or more forms of amendments. following on the realisation, from step 1, that as yet no review has been conducted for erp post-implementation amendments, we thus developed a framework for reviewing and analysing the literature: the framework is referred to as the erp-pia framework. frameworks have been used in the wider erp literature for clarifying and presenting a holistic view of the erp literature. the absence of such a framework to assess erp-pia can restrict the development of a richer understanding of erp-pia. these three steps eventually culminated in the identification of directions for further research on erp-pia. 3.1 step 1: a search for erp-pia literature to develop an overview of academic activity relating to erp post-implementation amendments, top level is journals were scanned for the period 2000 2016. the timeframe was chosen because erp systems became predominant in the late 1990s and early 2000 as a result of the y2k phenomenon. the search was made through the use of such key words as erp post-implementation, erp amendments, erp maintenance, erp upgrades, erp enhancement, and erp post-implementation changes. in particular, only articles with either ‘erp’, ‘enterprise resource planning’, or ‘enterprise systems’ together with either ‘maintenance’, ‘upgrade’, ‘enhancement’, ‘amendments’ or ‘post-implementation changes’ in their titles were selected these key words were chosen as they represent some of the terms used in the prior erp literature review articles to indicate those activities that concern erp systems after implementation. the key words were also informed by the definition of erp postimplementation amendments as inclusive of all forms of maintenance, enhancements and upgrades (information resources management association, 2011). the criteria for choosing articles discussing erp post-implementation amendments are as follows: first, the article must have been published in a peer-review journal; second, 31 october 2016 was selected as the cut-off date; hence, the articles must have been published before the cut-off date; and thirdly, only articles having one or more of the key words (mentioned in the previous paragraph) in their titles were selected. the process of selection included four rounds of searches that identified a total of 26 articles on erp post-implementation amendments. first round of search was restricted to those journals listed on the is journals ranked by the australasian journal of information systems oseni, foster, mahbubur, smith 2017, vol 21, research article framework for erp post-imp amendments 4 australian council of professors and heads of information systems (acphis) in 2013 (acphis, 2013). a complete list of the 2013 acphis journal ranking can be accessed via the following link: http://www.acphis.org.au/index.php/is-journal-ranking. this list comprises 181 journals, from which a total of 19 articles discussing erp post-implementation amendments were identified. second round of search was done using google scholar and scopus search engines, from which additional six articles were identified from journals that are not in the acphis list. third round of search was done by reviewing the citations for the articles identified in the first and second rounds as recommended by webster and watson (2002). one article, published in one of the acphis journals, was identified through this round. we had previously missed this article in the first round of search as its title and abstract does not include one of the main key words. instead, key words such as ‘erp’, ‘enterprise resource planning’ and ‘enterprise systems’ had been replaced with ‘packaged application software’. the fourth round of search, again as recommended by webster and watson (2002), was done to identify articles citing the key articles identified in the previous rounds. this search however identified no relevant journal article that could be included in our review. altogether, the 26 articles were identified by the lead researcher. however, each member of our research team individually read the title, key words, abstract, and introduction of each article. by carefully reading the details provided in the abstract and introduction, each of the 26 articles were considered by the team to be appropriate for inclusion in our analysis. the acphis list contained 13 a* journals, 39 a journals, 50 b journals, and 79 c journals. out of about 86 erp articles published across all 13 a* journals, only four articles were found discussing erp post-implementation amendments. in order to indicate this, a list of journals, in which articles on erp post-implementation amendments were published, along with the number of articles appearing in each journal is presented in table 1. level of journal journal name no. of erp-pia articles year of publication total no. of articles a* european journal of information systems 1 2007 11 information and management 1 2011 journal of information technology 2 2010; 2011 a communications of the acm 2 2000; 2006 enterprise information systems 2 2007;2010 journal of computer information systems 2 2006; 2007 journal of information systems 1 2013 b journal of systems and software 2 2002; 2010 9 c expert systems with applications 1 2001 journal of software maintenance and evolution 5 2001 journal of cases on information technology 1 2006 nonacphis is journals journal of accounting information systems 1 2006 6 computers in industry 1 2010 ieee transactions on software engineering 1 2012 computer standard and interfaces 1 2014 industrial management and data systems 1 2009 global journal of human social science 1 2013 table 1: literature search for erp post-implementation amendments from the table, it can be observed that 42% (11) of erp-pia articles were published in leading a* and a journals; while 35% (9) were published in b and c journals, and the remaining 23% (6) in non-acphis journals. this shows that erp post-implementation amendments are considered relevant, even for leading is journals. australasian journal of information systems oseni, foster, mahbubur, smith 2017, vol 21, research article framework for erp post-imp amendments 5 a key observation from table 2 reflects the decline in the number of published articles that focus on erp-pia. this is personified in fig 2, demonstrating the decline in articles being published, and especially the lack of erp-pia publications in periods 2003 to 2005 (0), 2008 (0) and 2015-2016 (0). one would expect that as organisations spend large sums of money in adapting their erp systems the reverse would be the case. figure 2. number of erp-pia articles per year, 2000-2016 in the following section, we discuss step 2, the research method for developing the framework and clarifying and analysing the erp-pia literature. 3.2 step 2 erp-pia framework the framework was developed based on a call for erp post implementation research to focus on a lifecycle-wide view of erp post-implementation activities (gable, chan, & tan, 2001). the importance of developing such a framework is its role in clarifying and providing an intensive view of particular aspects of the erp-pia literature (webster & watson, 2002). this is presented in figure 3 and consists of two sections: categories and phases of erppia. figure 3. the erp-pia framework this framework is valuable as it assists in the analysis of the erp-pia literature by: (a) providing a structure for the themes and key issues identified in the literature, (b) facilitating the classification of articles according to similarities and differences in categories and phases of erp-pia, (c) enabling a comparison of relative importance placed by various erp scholars on different categories, phases, and themes of erp-pia, and (d) informing the identification of specific gaps within categories and phases of erp-pia. this framework developed from the literature identifies three categories of amendments: maintenance, upgrades, and enhancement activities. as opposed to the available literature, we believe it is important to differentiate between upgrades, enhancements and maintenance as they produce varying effects on the erp system and ultimately the organisation. these effects australasian journal of information systems oseni, foster, mahbubur, smith 2017, vol 21, research article framework for erp post-imp amendments 6 can be in terms of the benefits derived by the organisation by undertaking the amendment, or in terms of the effort that the organisation is required to invest in undertaking the amendment. an example of the former can be seen when organisations are undertaking enhancements and major upgrades. such amendments are likely undertaken in response to post-implementation assessments and reviews (cao, nicolaou, & bhattacharya, 2010), with differences in the nature of the amendments impacting on the performance of the firm in relation to its use and enhancement of the erp system (nicolaou & bhattacharya, 2008). in other words, organisational effectiveness is often impacted by the decisions made regarding the extensions and continued use of erp systems. an example of the latter can be seen in the level of effort an organisation needs to invest in an upgrade as opposed to implementing a support stack. for instance, undertaking a major upgrade is a non-trivial activity, often creating organisational challenges that were unexpected and can be prolonged (khoo, chua, & robey, 2011; khoo, robey, & rao, 2011). considering the fragmented nature of post-implementation studies, the framework is relevant and necessary for clarifying the scope of documented work relating to erp postimplementation amendments. using constructs (such as those included in the framework the type and phases of amendments) is a pragmatic approach for deconstructing the existing literature and permits the identification of associations between constructs, if any exists. this is useful for identifying implicit aspects covered in the literature, especially because such associations are not always clearly mentioned in the literature. 3.2.1 three phases of erp-pia the erp-pia framework defines phases of amendments to include: phase 1 pre-amendment (before the amendment is undertaken); phase 2 – amendment-implementation (while the amendment is conducted), and phase 3 post-amendment (after the amendment has been conducted). in the development of our framework, key themes were extracted from the erp-pia literature based on the phase of amendment within which they were associated. examples of such themes include: critical factors, motivation, activities costs and risks. in addition, these themes have been discussed with respect to the three different categories of amendments; for example, we discuss the influence of motivation on maintenance, enhancements as well as upgrades. we now present (as shown in table 2) a list of erp-pia articles identified in our literature search (step 1), showing the categories of erp-pia [maintenance (m), upgrade (u), and enhancement (e)], and the themes identified across the three phases. the table also indicates the theoretical perspectives and methodology utilised in the erp-pia articles. australasian journal of information systems oseni, foster, mahbubur, smith 2017, vol 21, research article framework for erp post-imp amendments 7 reference erp-pia category themes across erp-pia phases theoretical perspective methodology pre-amendment amendment-implementation post-amendment motive for pia risks in pia method. for pia activities in pia critical factors for pia cost of pia activities in pia critical factors for pia benefit of pia impact of pia ng, gable, and chan (2002) m,u,e * * * * * case study salmeron and lopez (2012) m * * * * fuzzy cognitive maps expert panel kwon and lee (2001) m * multi-agent intelligent system salmeron and lopez (2010) m * * literature review and expert panel lópez and salmeron (2014) m * * augmented fuzzy cognitive maps modelling peng and nunes (2009) m * literature review hirt and swanson (2001) m, u * * case study nah, faja, and cata (2001) m, u * * multiple case study ng and gable (2010) m,u * case study law et al. (2010) m * * case study light (2001) m * * * * case study ng (2001) m,u,e * * case study fryling (2010) m,u,e * simulation modelling case study kremers and dissel (2000) u * min khoo and robey (2007) u * * realist case study khoo, chua, et al. (2011) u * relational foundation model case study khoo, robey, et al. (2011) u * communication framing theory inductive case study zarotsky, pliskin, and heart (2006) u * * case study dempsey, vance, and sheehan (2013) u * case study peslak, subramanian, and clayton (2007) u * * survey beatty and williams (2006) u * nah and delgado (2006) u * * multiple case study olson and zhao (2007) u * * nicolaou and bhattacharya (2006) u,e * industry survey cao et al. (2013) u,e * real options theory industry survey malihe, nor hidayati, davide, and mohammad akbarpour (2015) m,u,e content analysis and interviews table 2: analysis of erp-pia literature australasian journal of information systems oseni, foster, mahbubur, smith 2017, vol 21, research article framework for erp post-imp amendments 8 in the next section, we present findings from our analysis of the erp-pia literature. 3.3 step 3 – discussion of findings this section discusses findings from the erp-pia literature. one of the major findings and what we consider a major failing in the literature was the inconsistent use of the erp-post implementation terminology making it difficult to clarify the terms. many authors for example (law et al., 2010; ng, chan, & gable, 2001; ng, gable, & chan, 2003; ng et al., 2002) who referred to enhancements as well as upgrades literally placed them under the umbrella term ‘maintenance’. this is problematic as the literature referring to maintenance not only describes maintenance activities, but also includes elements that can be linked to upgrades and enhancements. the implication of this is that all amendments after implementation are sometimes seen as compulsory non-beneficial activities rather than as opportunities to uphold best business practices (edberg & olfman, 2001). as is the case with traditional in-house software literature, erp maintenance has been defined to encompass all post-implementation activities undertaken from the time the erp goes live through to its retirement (lópez & salmeron, 2011; ng et al., 2002). one author even considered maintenance as a phase conducted during an implementation (peslak et al., 2007). the concept of maintenance, in the traditional software maintenance literature as defined by the institute for electrical and electronics engineers (ieee), encompasses “amendment of a software product after delivery to correct faults, to improve performance or other attributes, or to adapt the product to a modified environment” (ieee, 1998). many researchers define maintenance as: “the process of identifying and performing changes required for improving or maintaining system usability and performance throughout the erp post-implementation stage” (lópez & salmeron, 2014; nah et al., 2001; ng et al., 2002). we argue that postimplementation amendments are generally classified as maintenance, and include the application of hot packs, import of new objects from erp vendor, version upgrade amongst others. some articles even document all kinds of post-implementation amendments (ng, 2001; ng et al., 2002), others simply focus on a single kind (khoo & robey, 2007; nah et al., 2001). consequently, we argue that dissimilar activities such as installation of upgrades, service packs and maintenance of custom codes, have been classified in some literature as maintenance (khoo, chua, et al., 2011), but should be called amendments and appropriately linked to maintenance, enhancements and/or upgrades for clarification purposes. following on from the review of the terminology, we now discuss the two major sections of our framework: categories and phases of erppia. 3.3.1 categories of post-implementation amendment the three erp-pia categories identified in the framework include maintenance, upgrade, and enhancement. maintenance: erp maintenance refers to regular post-implementation activities required to keep the erp system sustained. our argument is that any effort carried out in maintaining an erp system but which does not focus on improving or enhancing the system and which also is not a minor or major upgrade should be categorised as maintenance. therefore, maintenance includes requests for support package or patches, on-going system and help desk support, as well as bug fixes and is typically directed at minor corrections and further adjustments due to legal changes (ng, 2001; ng et al., 2002; worrell, 2007). consequently, we suggest that general maintenance activities are not typically expected to have significant business and technical change impacts. in the erp maintenance literature, maintenance activities are classified into many categories, most of which are adapted from traditional software maintenance literatures. for instance, nah et al. (2001) classifies erp maintenance into six categories (corrective, adaptive, perfective, preventive, user support, and external parties), three of which (corrective, adaptive, and perfective) are defined by swanson (1976). according to swanson (1976), the purpose of corrective maintenance is to correct faults; adaptive maintenance to accommodate australasian journal of information systems oseni, foster, mahbubur, smith 2017, vol 21, research article framework for erp post-imp amendments 9 changes in the data and processing environments; and perfective for performance improvement in form of enhancive maintenance (which include changes and additions to system functionality) and non-functional perfective maintenance. to accommodate erp software in particular, nah et al. (2001) also defined other dimensions such as preventive, user support and external parties. maintenance activities can also be divided into client-side and vendor-side maintenance (ng, 2001). client-side maintenance focuses on maintenance requests initiated by the client and refers to bug-fixes and help-desk requests; whereas vendor-side maintenance includes the application of hot-packs (system patches), enhancement packs and online service system (oss) notes that provide updates on patches for the system. it should be noted that the terminology varies by vendor for instance, sap refers to enhancement packages and support pack stacks for initiating maintenance activities. however enhancement packs may also be used to fulfil functional requirements such as activating updates to business processes that are initiated by the vendor. upgrade: upgrades are vendor-provided packages that contain substantial new features, and are weighted in terms of scope and version. (nah et al. 2001; ng 2001). the scope of an upgrade describes the extent of implementation and is defined in technical or functional terms: technical upgrades keep the system in a supported version, while functional upgrades refer to new software functionality which can lead to business process improvement. panaya solutions (technical-sap-upgrade, n.d.) define a technical upgrade as “an upgrade undertaken to move the already implemented system onto the latest technology platform, without the implementation of new functionality that are able to change user behaviour or business processes”. a technical upgrade is motivated by the need to maintain a technological infrastructure that is able to support required change and is usually not accompanied by any significant difference in the core product or functions (technical-sap-upgrade, n.d.). as such, technical upgrades are necessary in order to keep abreast with advancing technology (chorafas, 2001), and are likely to be accompanied by a high technical impact but low business impact. a functional upgrade on the other hand is undertaken solely for the purpose of gaining substantial enhancements as well as new functionality (fryling, 2010; khoo, robey, et al., 2011; kremers & dissel, 2000; ng, 2001). a functional upgrade is defined as “an upgrade undertaken to extend the business process functionality of an existing system (functionalsap-upgrade, n.d.), and it is more complex than a technical upgrade and involves the adoption of new business processes as well as automation of previously un-automated processes”. also, functional upgrades are largely initiated as part of a line-of-business initiative unlike the technical upgrades which are usually initiated by the it department. in fact, functional upgrades are usually expected to be preceded by technical upgrades and are expected to have both high business impact and high technical change impact (ng, 2001). upgrade version on the other hand describes an upgrade in terms of whether it reflects a move within the same series or to a different series, and can be a minor or a major upgrade. we consider an upgrade to be an improvement to the current software from an older version of the same software product. implementing a completely new system (even if it is similar) is considered to be outside the scope of this construct. enhancement: enhancements are undertaken to specifically increase the performance of the system by further adapting the system to meet organisational requirements (ng 2001; ng et al. 2002). we expect enhancements to bring about a high business impact. generally, enhancements encompasses maintenance activities targeted towards the improvement of software (hirt & swanson, 2001). in the context of erp systems, enhancements are considered to be add-ons and upgrades (nicolaou & bhattacharya, 2006), and aim to provide additional business functionality to the system. moreover, a maintenance request is considered to be an enhancement request based on the measure to which it contributes to business objectives (gable et al., 2001). enhancement efforts also include the design and implementation of customizations, as well as creation or amendment of user australasian journal of information systems oseni, foster, mahbubur, smith 2017, vol 21, research article framework for erp post-imp amendments 10 interfaces (worrell, 2007). considering the high business involvement in enhancement tasks, we consider that enhancement amendments should be treated and categorised differently to maintenance amendments. enhancements are associated with improvements initiated both from the client-side and vendor-side. some examples of client-side enhancements include configuration by setting parameters via the erp interface, add-on packages, tables and text enhancement, creation of customer-specific objects such as screens and reports, workflow programming to create non-standard workflows, and interface development to interface the erp system with various legacy systems (ng 2001; nicolaou and bhattacharya 2006). from the vendor driven side, enhancements may come in the form of hot-packs that initiate new functionality and capabilities. these packs are often developed for use in the current system and for which organisations may find more preferable to implement than outlaying the cost of a full-blown upgrade. having identified the three major categories of erp post-implementation amendments, we now explain the three major phases of erp post-implementation amendments. 3.3.2 phases of erp post-implementation amendments the phases of erp post-implementation amendments are identified as phase 1 -preamendment, phase 2 amendment, and phase 3 post-amendment. within each phase, several themes are identified. these themes represent major issues identified from the literature which have been shown to influence post-implementation amendments. we now discuss the erp-pia categories with respect to the phases and themes. we note that this discussion is presented as identified in the erp-pia literature. for instance, we find that in some cases discussion of themes do not cut across all categories of amendments, and as such impose a gap. as an example, nah and delgado (2006) discuss critical success factors for upgrades without touching on maintenance and enhancement. on the other hand, law et al. (2010) discusses csfs for maintenance without discussing on upgrades and enhancements. consequently enhancements are exempted from our discussion on the csf theme as no literature was found discussing this. phase 1 – pre-amendment in this phase, amendments are conceived and planned. five themes are identified for this phase: motivations, risks, methodologies, activities, and critical success factors. phase 1 theme 1 (motivation) motivation refers to high-level objectives of an organisation to initiate a particular project (smith, rahim, shanks, & johnston, 2008). motivation is discussed against the three erppia categories: maintenance, upgrades and enhancements. maintenance: motivations for conducting maintenance include requests for user-support, master-data-change, corrections, and patches (ng et al., 2002). user-support requests address system functionalities and difficulties associated with human-computer interaction; change requests are aimed at corrections and fixing bugs; while master-data change requests deal with updates to the master data file. patches are used to fix bugs in the existing system, adapt to external environmental changes such as government regulations, and maintain the installed system in its operational standard required by the vendor; thus ensuring the system meets the contractual arrangements of maintenance and support (ng, 2001; ng et al., 2002). upgrades: motivations for erp upgrades can be classified based on the source of the upgrade (khoo & robey, 2007; khoo, robey, et al., 2011), and the reason for the upgrade (kremers & dissel, 2000; ng et al., 2002). the motivation for erp upgrades originates from both internal as well as external forces. internal forces refer to internal requirements generated by the client organisation who has installed the erp system; for example the need for more benefits. other examples of internal motivating forces include the availability of resources to facilitate the upgrade, and the corporate and it policies to mitigate software risks. external forces refer to the external dependence placed squarely on the erp vendor; examples include the functionality of the software and the level of technical support provided by the vendor (dempsey et al., 2013). other motivating factors reflect business reasons, technical reasons and environmental reasons (kremers & dissel, 2000). business reasons reflect the need for australasian journal of information systems oseni, foster, mahbubur, smith 2017, vol 21, research article framework for erp post-imp amendments 11 added functionality such as that affecting a core method of carrying out a business activity. technical reasons may include compliance with government standards, expiration of vendor support for an installed version, and keeping the system up-to-date (dempsey et al., 2013; zarotsky et al., 2006). environmental reasons refer to pressure from the value chain, such as a business partner requiring a specific functionality in the system. enhancements: enhancements are motivated by a business need and are requested to seek improvement and optimise business processes; for instance, adjusting a particular module of the erp system to better suit the business needs of the organisation (ng 2001; ng et al. 2002). phase 1 theme 2 (risks) risks can threaten the positive outcomes of amendment projects (salmeron and lopez 2012). post-implementation erp risks are defined as an event that has consequences or impacts the use, maintenance, and enhancement of implemented erp systems (peng & nunes, 2009). in the erp-pia literature, only risks specifically related to maintenance activities have been documented; risks pertaining to erp upgrades and enhancement have not been reported. prior to undertaking an amendment, risks may be caused by continuous changes in scope/objectives of amendments or conflicting requirements in the design of the amendment. these risks strongly affect erp maintenance performance, specifically erp user satisfaction. thus, it is critical to control risks in the pre-amendment phase because risks, if not addressed early, can negatively influence the outcome in the postamendment phase. one recommended way to reduce risks is to develop risk strategies early in order to successfully identify and mitigate the risk (lópez & salmeron, 2011; salmeron & lopez, 2012). phase 1: theme 3 (methodology to guide amendments) ‘methodology’ as a theme in the erppia literature concerns the approaches that facilitate and support amendments. the erp-pia literature only focuses on methodologies for erp maintenance, excluding methodologies for upgrades and enhancements. only two methodologies were identified from the literature: problem solving algorithms by kwon and lee (2001) and fuzzy cognitive maps by salmeron and lopez (2012). these are discussed here. (i) to ensure the highest level of operation of the erp system in the organisation, kwon and lee (2001) suggest that the erp system must be combined with problem solving algorithms if effective erp system maintenance is to be achieved in the amendment phase. they propose a multi-agent intelligent system architecture to alleviate erp maintenance efforts. in addition, they propose that in the pre-amendment phase, plans are defined to establish for continuity and autonomy. (ii) salmeron and lopez (2012) propose a framework, referred to as the fuzzy cognitive map (fcm), for managing the process of undertaking erp maintenance. arguments for the framework follows that erp maintenance is ambiguous and that there are no clear frameworks to define how best erp professionals should manage the maintenance process. this framework is designed to address the need for a systematic methodology to predict the effects of risks on the maintenance and simulated distinct scenarios depicting how risks may affect the maintenance being undertaken. phase 1 theme 4 (activities in pre-amendment) activities related to pre-amendment have been identified by erp scholars in attempt to differentiate erp amendments from custombuilt it software amendments. although activities are often referred to as characteristics in the literature we argue that activities effectively characterise erp-pia (nah et al., 2001). hirt and swanson (2001); nah et al. (2001); peslak et al. (2007) identify activities peculiar to maintenance and upgrade type amendments only; activities in enhancement type amendment are not identified. firstly we will discuss maintenance activities, followed by upgrade activities. maintenance amendment: erp maintenance activities occurring in the pre-amendment phase are described by ng and gable (2010) as maintenance preparation. specific activities in maintenance preparation include: (i) defining maintenance, (ii) estimating maintenance resources, (iii) managing relationship with the vendor, and (iv), defining all maintenance management issues. these activities are designed to aid the maintenance procedure by australasian journal of information systems oseni, foster, mahbubur, smith 2017, vol 21, research article framework for erp post-imp amendments 12 outlining maintenance objective, the maintainer’s roles and responsibilities, resources required, and maintenance strategies. upgrade amendment: activities in pre-amendment where upgrades are involved focus on the justification and planning of new software versions (nah et al. 2001). such activities include: the design of a project methodology for the upgrade, researching upgrade options, developing a business case, and performing impact analysis between a new version and a previously installed version. phase 1 theme 5 (critical success factors) critical success factors (csf) are areas of influence that, if not addressed and managed at the pre-amendment phase, can impact on post-amendment activities. several studies identified critical success factors in erp upgrade activities, while one study addressed critical factors for erp maintenance. csfs will be discussed against maintenance and upgrade categories only as we found no studies reporting csfs for enhancement amendments. maintenance: law et al. (2010) developed a framework of critical success factors (csf) for erp maintenance and support and identified some very valuable csfs. a major observation in this framework is that the csfs proposed are not unique to erp maintenance, but can be extrapolated to a full blown erp implementation as well. they identified, critical factors contributing to effective performance of erp maintenance and support activities to include: (i) the implementation strategy – a poorly defined implementation approach not only leads to implementation issues but also makes maintenance and support difficult; (ii) the formulation of clear organisation structures and clearly defined roles and responsibilities contribute to the effective operation and execution of maintenance activities; (iii), client-vendor alignment – maintaining a strategy that aligns with the vendor’s services and product strategies; (iv) the ability to leverage erp expertise from multiple sources; (v) ensuring a level of support and participation by stakeholders; and (vi) communication and co-ordination among functional units at all levels of personnel. upgrade: for upgrade projects to be successfully achieved beatty and williams (2006) recommend companies build their business case on new functionality, treat the upgrade project as a new business project, and utilise the initial implementation project team, a practice not easily achieved. organisations are advised to integrate these factors into their upgrade plans to deliver quality systems on time and within budget. likewise, nah and delgado (2006) found that the factors critical to a successful upgrade project include having a business plan and vision, determining the type and amount of organisation change necessary to accommodate the upgrade, defining and evaluating the project scope to avoid scope creep, and planning for effective communication between project team members and stakeholders. phase 2 – amendment-implementation the amendment-implementation phase is described as the period where an amendment is in the process of being implemented. three major themes pertaining to implementing an amendment include cost, activities, and critical success factors. each theme is discussed: phase 2 -theme 1 (costs) costs as a theme in the amendment-implementation phase pertains to factors that influence the cost of implementing an amendment. this theme will be discussed against the erp-pia categories: maintenance, upgrades and enhancement. maintenance: the costs associated with erp maintenance are greatly impacted by the manner in which the application of patches are managed (ng 2001). for example, maintenance costs are determined by the estimated number of hours and cost of labour to implement the patch. additionally, the number of changes included as part of the maintenance is known to impact maintenance costs (fryling, 2010; ng, 2001). for instance, it may be necessary to revisit all customisations when applying new updates. upgrade: costs associated with an upgrade could be minimal or large scale, and are usually dependent on such costs as software license fees, support and maintenance costs, hardware, user-training, consultancy fees, and previous enhancement amendment re-application costs australasian journal of information systems oseni, foster, mahbubur, smith 2017, vol 21, research article framework for erp post-imp amendments 13 (ng 2001). when an upgrade is within the same series (e.g from 3.1h to 3.1i), it is much cheaper than upgrading to a higher software series (e.g from 3.1h to 4.6b). this is because an upgrade within the same series excludes licensing costs and maintenance and support costs, not to mention training and general organisational change approaches, and very often is freely provided. an upgrade to a higher series on the other hand is accompanied by numerous changes in system structure, business processes and possibly the architecture (fryling 2010; ng 2001). while testing associated with an upgrade also introduces cost as the help of external testing firms may be required (zarotsky et al., 2006). enhancement: enhancement costs usually comprise the labour cost multiplied by the numbers of hours per enhancement. if the enhancement requires users to be trained, then the cost of end user training must also be included (ng 2001). phase 2 theme 2 (activities involved in implementing an amendment) maintenance: activities involved in the implementation of maintenance amendments are described as maintenance procedure (ng & gable, 2010). these activities include the monitoring of patch introduction; receipt and classification of maintenance request, request for vendor support, problem analysis and solution design, and finally the application of vendor patch. upgrade: activities occurring in the implementation phase of an upgrade include: the installation and construction of new version in a dev system (the testing environment); conduct testing and trial upgrades between the dev system and quality assurance system (qas); and ultimately the conversion (or go live) in the prd (production system) (ng and gable 2010). enhancement: enhancement activities are aimed at obtaining benefits from the erp system, and are designed to improve business processes (ng et al. 2002). as such, activities that characterise the implementation of enhancements include development of new reports as well as new features in reports, generating new interface, creating new program or software functionality, improving existing program functionality, and updating business processes. phase 2 theme 3 (critical success factors) in this phase critical factors such as project management, change management, and effective communication between project team and stakeholders are identified (nah and delgado 2006). these factors are however no different from those identified in the pre-implementation phase. phase 3 – post-amendment major themes in the literature concerning the post implementation phase of an amendment mainly refer to the impact of the changes from the amendment. these themes include: business benefits and impacts. phase 3 theme 1 (business benefits) the erp-pia literature refers to benefits as business performance improvements after undergoing amendments. some of these benefits are related to new and improved functionality, increased accurate sales planning or forecasting relative to competitive advantage, better availability of information for managers to support globalisation, and reduction in the number of third-party legacy systems (ng 2001; ng et al. 2002). unlike maintenance activities which usually do not contribute to any business benefit, benefits to organisation performance are most often obtained from upgrades and enhancements, especially when the upgrades and enhancements are performed within the first few years of implementing a new erp version (cao et al. 2013; nicolaou and bhattacharya 2006). although post-implementation changes generally affect a firm’s performance; the timing of the changes also has a marked impact on performance improvements. for instance, erp-adopting firms that initiate enhancements in the form of either add-ons or upgrades soon after the implementation, tend to enjoy superior differential performance benefits when compared to their non-erp peers (cao et al. 2013; nicolaou and bhattacharya 2006). phase 3 theme 2 (impact of the amendment) – erp-pia are expected to impact both the erp system itself as well as the benefits obtainable from the system (galy & sauceda, 2014). australasian journal of information systems oseni, foster, mahbubur, smith 2017, vol 21, research article framework for erp post-imp amendments 14 the erp-pia literature documents some impacts resulting from amendments, especially with respect to erp upgrades. for instance, impacts from upgrades are explained in terms of the effect of the upgrade on the existing system, is staff, and key users of the system (khoo et al. 2011b; ng et al. 2002). however impacts from maintenance and enhancement activities are not documented. upgrade: erp upgrades are known to impact on the installed erp system, as well as on is staff. the impact of an erp upgrade on the existing erp system can be measured in terms of three relationships: relationship between the vendor and the systems, relationship on and between it support staff and vendors, and the relationship among systems (khoo and robey 2007). the impact of the upgrade on support staff is evidenced in the relationship between the is staff and end users in supporting and training users after upgrades have been implemented. as a result of changes in user interfaces due to an upgrade, the relationship among users is impacted as users find a need to increase their interaction to assist each other during upgrades. the above subsections have discussed the erp-pia framework and identified the major themes from the literature. a number of gaps in the literature were observed that could serve as future directions for erp research. these directions are discussed in the following section. 4 future research directions and recommendations in this section, we discuss the research gaps. this discussion is presented in accordance with the erp-pia framework in three steps: first, the gaps are presented in figure 4, and are discussed as future research directions (rd); second, we reflect upon how these directions could extend the existing literature; third, suggestions identifying how these directions can be empirically examined are presented. figure 4. gaps identified in the erp-pia literature 4.1 phase 1 (pre-amendment) research directions rd1: the influence of organisation type and size on motivations for amendments the studies addressing motivations for amendments primarily involve single case studies. to extrapolate the findings of single case studies, we recommend multiple case studies of different organisation types and sizes to enable comparisons across several organisations and provide a broader picture of the influence of differences in organisational motivations for amendments. rd2: principles for prioritising amendments to better understand how organisations prioritise amendments, it may be useful to investigate possible criteria that could be of value for choosing a particular amendment over another. for instance, it may be more cost effective to perform a particular kind of amendment at a particular point in time. it may however also be the case that a more extensive amendment may be capable of preventing further amendments. a study to understand what criteria underlie the amendment choices that organisations make would be an appropriate approach. australasian journal of information systems oseni, foster, mahbubur, smith 2017, vol 21, research article framework for erp post-imp amendments 15 rd3: critical factors for initiating amendments some contingency factors inhibiting or influencing the timing of amendments are suggested in the literature. for instance, the availability of resources is known to be a factor that determines whether or not upgrades are undertaken or delayed (khoo and robey 2007). however, there may be other factors that need consideration before a particular amendment is undertaken. as an example, the level of expertise of the organisation to effectively utilise the amendment may be an important factor. to investigate critical factors influencing how amendments are initiated, semi-structured surveys could be chosen, identifying several examples of amendments and asking what factors would ideally be considered by the management of an organisation before such amendments are undertaken. 4.2 phase 2 (amendment-implementation) research directions rd4: organisational learning during amendment-implementation activities involved in amendments have been documented in the literature (ng and gable 2010). it may be interesting to find out whether amendments actually demonstrate an organisation’s learning approach. for instance, an organisation could choose a particular learning style for one amendment and adopt another learning style for another amendment. hence, a variation in learning styles across several amendment instances may be observed. in a similar fashion, a variation in learning styles for similar amendments could be investigated across several organisations. in a case, where organisation a and organisation b undertake a similar kind of amendment, do both organisations adopt the same or different learning approaches? the influence of organisational learning is important because different organisational learning approaches are found to produce different outcomes (hunter, 2003). in particular, kraemmerand, møller, and boer (2003) suggest that post-implementation; activities reflect different learning approaches and in turn influence firm performance differently. further arguments follow that the use and maintenance of existing capabilities suggest exploitation, whereas the search for alternate capability suggests exploration (kraemmerand et al., 2003; tomblin, 2010). based on the different features that characterise each learning type, it can be argued that post-implementation amendments to erp systems can be classified using these learning concepts. such a classification is significant because amendments can be indicative of learning and at the same time hamper learning. for example, given the inherent tension between exploitation and exploration, the tendency to explore versus exploit is affected by several factors such as resources, organisational culture, and structure and as such a focus on exploitation can inhibit exploration (otieno, 2010). organisations’ behaviours may also be a relevant lens for studying amendments and may facilitate a better and complete framework in the future. rd5: business process changes during amendments noting that an organisation faces options about the types of amendments to undertake, an important investigation to make during amendment endeavours concerns organisations’ willingness or openness to accommodate changes to business processes. davenport et al. (2004) argue that an organisations’ configuration and implementation of erp systems should be accompanied by business process re-engineering, coupled with changes in managerial processes to exploit the new data. however, existing research shows that due to the amount of time and resources necessary for a successful implementation, business process changes during erp implementation in organisations are usually only minimal, if undertaken at all. therefore, business process changes occurring during post-implementation become the focus of value derived from the implemented systems. in fact, existing research demonstrates that significant value can only be achieved if organisations continue to optimise business and management processes (harris & davenport, 2006). it would be valuable to know why some organisations are open to making business process changes during one amendment and not doing the same for another. rd6: user training during amendment-implementation training accompanying initial erp implementation project is one of the critical success factors identified in the literature and is recognised as one of the factors that does not end after the project has been concluded (nah australasian journal of information systems oseni, foster, mahbubur, smith 2017, vol 21, research article framework for erp post-imp amendments 16 and delgado 2006). in fact, training is recommended to continue as organisations make further changes to their systems, and is considered as one of the key factors to successful erp maintenance, and upgrades (law et al. 2010). understanding how organisations manage the training process during the amendment-implementation phase thus appears necessary. for example, if business process changes are made alongside the amendment, then is there a need for training to accompany such amendment to facilitate a smooth transition to new business practices? if continuous changes are made to the erp system, do organisations have a process to cater for user training needs? is it possible that the level of user training required for a particular amendment is a factor that determines how the amendment is undertaken and how much business process changes are introduced during the amendment-implementation? rd7: critical success factors during amendment-implementation one of the observations from our literature analysis is that the erp literature documents several critical success factors for erp implementation but is very limited with regards to the post-implementation phase. the post-implementation csfs are however inherited from the erp implementation literature, and as such there is a need to investigate csfs that are specific to the postimplementation phase. 4.3 phase 3 (post erp-pia implementation) research directions rd8: capturing the outcomes of amendments post-implementation audit and reviews are recommended after erp implementations to assist organisations in capturing whether or not the benefits anticipated from implementation endeavours have been realised. it appears logical to reason that similar efforts be undertaken after amendments have been made to the erp system. a limited number of studies have been conducted to measure the impact of such amendments following initial implementations (nicolaou & bhattacharya, 2008; nicolaou, 2004). these studies have however been conducted using secondary data. while this is valuable for indicating performance outcomes from amendments, the impact of amendments may not always reflect on the overall organisational performance. as such, first-hand operational effects of amendments also need to be captured. do organisations investigate and measure operational benefits from amendment initiatives. if yes how are these measured? from these questions, researchers could propose methodologies that are able to assist organisations with such efforts providing support to the amendment-implementation phase. rd9: factors influencing the outcomes of amendments several factors may influence outcomes of amendment efforts. such factors may include motivation, learning, and training. for instance, do motivating forces for undertaking erp-pias contribute to the benefits derived from the amendment? does the organisational learning approach adopted for an amendment influence its benefits? or could training be one of the factors that influence what benefits are derived? an understanding of the factors that influence outcomes of amendments may indicate a relationship amongst different types of amendments and outcomes that organisations may experience. 5 conclusion and limitations erp post-implementation amendments are of practical importance as is the case with initial erp implementations. however, the attention given to erp-pia in the academic and practitioner literature appears to be relatively small compared to that of initial implementations. this is considered to be a major failing as organisations continually need to decide on how to manage and maintain the systems after implementation. understanding erp-pia is therefore an important subject to be included in academic and practitioner literature, especially since future benefits, obtainable from erp systems, are moderated by post-implementation changes. this paper has documented a review of the literature concerning erp post-implementation amendments in order to clarify terminologies and to present an overview of the current state of research in the field of erp-pia. while only a small number of articles discuss erp-pia, our review identified three categories and three phases of erp-pia. in addition, relevant australasian journal of information systems oseni, foster, mahbubur, smith 2017, vol 21, research article framework for erp post-imp amendments 17 themes such as costs, critical success factors, and motivation, amongst others were covered in the erp-pia literature. however, other themes like training and organisational learning have been identified as missing, but relevant to erp-pia. furthermore, with a goal to assist the formulation of a research agenda for future erp research, this paper presents a contribution to the erp literature by developing a framework to analyse erp-pias. the framework provided a logical sequence for the discussion of predominant amendment themes and aided the development of further research directions. practical implications of this paper concern the importance of erp-pia. erp managers must understand differences in erp-pias as well as possible outcomes, if effective outcomes are to be achieved. the framework is useful for exposing where managerial efforts and resources should be directed to foster successful uptake of erp-pia, thereby facilitating better decisions regarding how and when amendments should be undertaken. erp systems have been of interest over the last two decades and the number of publications has increased enormously over the last 10-17 years. however, the topic of post-implementation amendments is still developing. it is our recommendation that scholars pay attention to erppia as the era of new erp implementations is over and amendments are essential for the realisation of further benefits. the review presented in this paper is not without limitations. like other literature reviews, the paper is largely dependent on selected keywords and publication outlets. to eliminate such limiting factors, the review process for published articles covering post-implementation amendments was carefully conducted; first beginning with articles in the ahcpis list and then expanding the search to include other relevant journal outlets. references acphis. (2013). is journal ranking. retrieved from http://www.acphis.org.au/index.php/isjournal-ranking beatty, r. c., & williams, c. d. (2006). erp ii: best practices for successfully implementing an erp upgrade. communications of the acm, 49(3), 105-109. cao, j., nicolaou, a. i., & bhattacharya, s. (2010, june 28, 2010.). a longitudinal study of market and firm-level factors influencing erp systems adoption and postimplementation system enhancement options. paper presented at the 7th international conference on enterprise systems, accounting, and logistics (icesal), rhodes, greece. cao, j., nicolaou, a. i., & bhattacharya, s. (2013). a longitudinal examination of enterprise resource planning system post-implementation enhancements. journal of information systems, 27(1), 13-39. cfo research services. (2009). the high cost of change for erp: what does it cost to keep up to d ate? . retrieved from http://www.cfo.com/whitepapers/index.cfm/displaywhitepaper/13316056 chorafas, d. n. (2001). integrating erp, crm, supply chain management, and smart materials. boca raton: auerbach publications. davenport, t. h., harris, j. g., & cantrell, s. (2004). enterprise systems and ongoing process change. business process management journal, 10(1), 16 26. dempsey, s., vance, r., & sheehan, l. (2013). justification of an upgrade of an enterprise resource planning (erp) system the accountant's role. global journal of human social science, 13(1). edberg, d., & olfman, l. (2001, 3-6 jan. 2001). organizational learning through the process of enhancing information systems. paper presented at the system sciences, 2001. proceedings of the 34th annual hawaii international conference on information systems. australasian journal of information systems oseni, foster, mahbubur, smith 2017, vol 21, research article framework for erp post-imp amendments 18 fryling, m. (2010). estimating the impact of enterprise resource planning project management decisions on post-implementation maintenance costs: a case study using simulation modelling. enterprise information systems, 4(4), 391-421. functional-sap-upgrade. (n.d.). retrieved from http://www.panayainc.com/articles/thecase-for-the-functional-sap-upgrade.html gable, g. g., chan, t., & tan, w.-g. (2001). large packaged application software maintenance: a research framework. journal of software maintenance and evolution: research and practice, 13(6), 351-371. galy, e., & sauceda, m. j. (2014). post-implementation practices of erp systems and their relationship to financial performance. information & management, 51(3), 310-319. gartner. (2003). it spending: how do you stack up? retrieved from http://www.gartner.com/research/attributes/attr_47450_115.pdf harris, j. g., & davenport, t. h. (2006). new growth from enterprise systems: achieving high performance through distinctive capabilities. retrieved from hirt, s. g., & swanson, e. b. (2001). emergent maintenance of erp: new roles and relationships. journal of software maintenance and evolution: research and practice, 13(6), 373-387. hunter, s. d. (2003). information technology, organizational learning, and the market value of the firm. journal of information technology theory and application, 5(1), 1-28. ieee. (1998). ieee standard for software maintenance, ieee std 1219-1998. (pp. 47). new york: institute of electrical and electronic engineers. information resources management association. (2011). enterprise information systems: concepts, methodologies, tools and applications khoo, h. m., chua, c. e. h., & robey, d. (2011). how organizations motivate users to participate in support upgrades of customized packaged software. information & management, 48(8), 328-335. khoo, h. m., & robey, d. (2007). deciding to upgrade packaged software: a comparative case study of motives, contingencies and dependencies. european journal of information systems, 16(5), 555-567. khoo, h. m., robey, d., & rao, s. v. (2011). an exploratory study of the impacts of upgrading packaged software: a stakeholder perspective. journal of information technology, 26(3), 153-169. klans, h., rosemann, m., & gable, g. g. (2000). what is erp? information system frontiers, 2(2), 141-162. kraemmerand, p., møller, c., & boer, h. (2003). erp implementation: an integrated process of radical change and continuous learning. production planning & control, 14(4), 338348. kremers, m., & dissel, h. v. (2000). erp system migration: a provider's versus customer's perspective. communications of the acm, 43(4). kwon, o. b., & lee, j. j. (2001). a multi-agent intelligent system for efficient erp maintenance. expert systems with applications, 21(4), 191-202. law, c. c. h., chen, c. c., & wu, b. j. p. (2010). managing the full erp life-cycle: considerations of maintenance and support requirements and it governance practice as integral elements of the formula for successful erp adoption. computers in industry, 61(3), 297-308. australasian journal of information systems oseni, foster, mahbubur, smith 2017, vol 21, research article framework for erp post-imp amendments 19 light, b. (2001). the maintenance implications of the customization of erp software. journal of software maintenance and evolution: research and practice, 13(6), 415-429. lópez, c., & salmeron, j. (2011). exploring risk factors in erp maintenance projects. scientific journal of riga technical university. computer sciences, 43(-1), 50-56. lópez, c., & salmeron, j. l. (2014). modeling maintenance projects risk effects on erp performance. computer standards & interfaces, 36(3), 545-553. malihe, m., nor hidayati, z., davide, a., & mohammad akbarpour, s. (2015). developing instruments for enterprise resources planning (erp) post-implementation failure model. international journal of enterprise information systems (ijeis), 11(3), 68-83. min khoo, h., & robey, d. (2007). deciding to upgrade packaged software: a comparative case study of motives, contingencies and dependencies. european journal of information systems, 16(5), 555-567. motiwalla, l. f., & thompson, j. (2009). enterprise systems for management. new jersey: pearson education. nah, f. f.-h., & delgado, s. (2006). critical success factors for enterprise resource planning implementation and upgrade. the journal of computer information systems, 46(5), 99-113. nah, f. f.-h., faja, s., & cata, t. (2001). characteristics of erp software maintenance: a multiple case study. journal of software maintenance and evolution: research and practice, 13(6), 399-414. nash, k. s. (2007). erp: how and why you need to manage it differently. retrieved from http://www.cio.com/article/2421027/enterprise-software/erp--how-and-why-youneed-to-manage-it-differently.html ng, c. s.-p., & gable, g. g. (2010). maintaining erp packaged software a revelatory case study. journal of information technology, 25(1), 65-90. ng, c. s. p. (2001). a decision framework for enterprise resource planning maintenance and upgrade: a client perspective. journal of software maintenance and evolution: research and practice, 13(6), 431-468. ng, c. s. p., chan, t., & gable, g. g. (2001, 2001). a client-benefits oriented taxonomy of erp maintenance. paper presented at the ieee international conference on software maintenance (icsm 2001), 7-9 november 2001. . ng, c. s. p., gable, g., & chan, t. (2003). a revelatory case study into the adequacy of standard maintenance models in an erp context. paper presented at the 7th pacific asia conference on information systems, 10-13 july 2003, , adelaide, south australia. ng, c. s. p., gable, g. g., & chan, t. (2002). an erp -client benefit-oriented maintenance taxonomy. journal of systems and software, 64(2), 87-109. nicolaou, a., & bhattacharya, s. (2008). sustainability of erps performance outcomes: the role of post-implementation review quality. international journal of accounting information systems, 9(1), 43-60. nicolaou, a. i. (2004). quality of postimplementation review for enterprise resource planning systems. international journal of accounting information systems, 5(1), 25-49. nicolaou, a. i., & bhattacharya, s. (2006). organizational performance effects of erp systems usage: the impact of post-implementation changes. international journal of accounting information systems, 7(1), 18-35. olsaker, e. (2010). an unlikely place to find cost savings: erp software maintenance and support annual fees. government finance review. retrieved from australasian journal of information systems oseni, foster, mahbubur, smith 2017, vol 21, research article framework for erp post-imp amendments 20 http://www.gfoa.org/unlikely-place-find-cost-savings-erp-software-maintenance-andsupport-annual-fees olson, d. l., & zhao, f. (2007). cios' perspectives of critical success factors in erp upgrade projects. enterprise information systems, 1(1), 129-138. otieno, j. o. (2010). enterprise resource planning systems implementation and upgrade: (a kenyan study). (doctor of philosophy), middlesex university. peng, g. c., & nunes, m. b. (2009). surfacing erp exploitation risks through a risk ontology. industrial management & data systems, 109(7), 926-942. peng, g. c., & nunes, m. b. (2010). why erp post-implementation fails? lessons learned from a failure case in china. paper presented at the 14th pacific asia conference on information systems, taipei, taiwan. peslak, a. r., subramanian, g. h., & clayton, g. e. (2007). the phases of erp software implementation and maintenance: a model for predicting preferred erp use. journal of computer information systems, 48(2), 25-33. salmeron, j. l., & lopez, c. (2010). a multicriteria approach for risks assessment in erp maintenance. journal of systems and software, 83(10), 1941-1953. salmeron, j. l., & lopez, c. (2012). forecasting risk impact on erp maintenance with augmented fuzzy cognitive maps. software engineering, ieee transactions on, 38(2), 439-452. smith, s. p., rahim, m. m., shanks, g., & johnston, r. b. (2008). how organization goals affect interorganization system implementation projects: evidence and implications. asia pacific management review, 13(3), 567-581. swanson, e. b. (1976). the dimensions of maintenance. paper presented at the proceedings of the 2nd international conference on software engineering, san francisco, california, united states. technical-sap-upgrade. (n.d.). retrieved from http://www.panayainc.com/articles/thecase-for-the-technical-sap-upgrade-keeping-up-with-business-and-technologyevolution.html tomblin, m. s. (2010). theory development in enterprise systems and organizational learning. journal of organizational computing and electronic commerce, 20(4), 398-416. webster, j., & watson, r. t. (2002). analyzing the past to prepare for the future: writing a literature review. mis quarterly, 26(2). worrell, j. l. (2007). running the erp marathon: enhancing erp business fit in the postimplementation phase. florida state university. zarotsky, m., pliskin, n., & heart, t. (2006). the first erp upgrade project at dsw: lessons learned from disillusion with simplicity expectations. journal of cases on information technology, 8(4), 13-23. copyright: © 2017 oseni, foster, maburbhir, smith. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems oseni, foster, mahbubur, smith 2017, vol 21, research article framework for erp post-imp amendments 21 paper title (paper title style) australasian journal of information systems asefi, lim & maghrebi 2015, vol. 19, pp. s21-s35 model for location routing a mathematical model for the municipal solid waste location-routing problem with intermediate transfer stations hossein asefi school of civil and environmental engineering the university of new south wales hossein.asefi@student.unsw.edu.au samsung lim school of civil and environmental engineering the university of new south wales mojtaba maghrebi department of civil engineering ferdowsi university of mashhad mashhad, iran abstract municipal solid waste management is one of the challenging issues in mega cities due to various interrelated factors such as operational costs and environmental concerns. cost, as one of the most significant constraints of municipal solid waste management can be effectively economized by efficient planning approaches. considering diverse waste types in an integrated municipal solid waste system, a mathematical model of the location-routing problem is formulated and solved in this study in order to minimize the total cost of transportation and facility establishment. the main aim of the proposed model is to optimize the locations of the waste management system’s facilities including transfer stations, treatment centres, recycling centres and disposal centres, and identify the optimal routes to and from the facilities. a case study of the location-routing problem in new south wales, australia, is investigated and analysed. twenty five districts are selected to represent the whole state of new south wales for the analysis. twenty-four districts are also assumed to be candidate locations for each of the six types of the system’s facilities which leads to a network with 24*6 potential location nodes i.e. 24^6 possible solutions. the presented application in this study is by far the largest size of the network among the related studies in the literature. keywords: location-routing problem; municipal solid waste; hazardous waste; routing; facility location 1 introduction business analytics approaches have received growing attention in recent years as to how various computer technologies can be utilized to provide decision makers with ad hoc analysis capabilities. new business models based on analysis of cost and quality have been adopted in operational and strategic decision-making. apart from cost and quality, business models have also taken account of environmental issues such as emission and energy consumption of production facilities and pollution from logistics (dawson and spedding, 2009). the reduction of waste management costs and the increase of the benefits from recycling have been perceived as a new competitive business for many large-scale private sectors and government organizations. however, investigations on how business analytics can be applied to waste management are rare in the literature. municipal solid waste management (mswm) is considered as one of the most challenging issues in many populous cities. generally, mswm includes the processes associated with collection, transportation, treatment, recycling and disposal of waste in a safe, hygienic and cost-effective manner. a successful mswm requires the appropriate site selection of the waste management system’s facilities such as recycling and disposal facilities, and transportation of s21 australasian journal of information systems asefi, lim & maghrebi 2015, vol. 19, pp. s21-s35 model for location routing wastes among the facilities. the extensiveness and complexity of the factors affecting mswm (e.g. limited resources such as land, investment costs and operational costs) make it difficult to be properly implemented. there are different kinds of municipal solid waste (msw) such as rubbish, food waste, commercial waste, industrial waste, construction waste and sanitation waste. generally, msw includes recyclables (such as paper, glass and plastics), toxic substances (paint, pesticides, used batteries, medicines), compostable organic matter (fruit and vegetable peels, food waste) and soiled waste (blood stained cotton, disposable syringes, etc.) (sharholy et al., 2008). however, in a broader classification perspective we can divide msw into the three main categories: hazardous wastes, recyclable wastes and garbage. hazardous wastes are harmful to human health and animals, and can have destructive environmental impacts if they remain in the environment and residential areas. hazardous wastes can be classed as four main types: (i) ignitable wastes, (ii) wastes with the property of corrosiveness, (iii) wastes with the property of chemical reactivity, and (iv) toxic wastes. hazardous wastes must be treated under certain technologies such as incineration and chemical treatment, and the material which is recyclable after treatment can be recycled. the rest must be disposed in residues disposal sites after removing the hazardous properties (samanlioglu, 2013). hazardous wastes are often generated from particular types of smallscale businesses and households. some of the hazardous waste generating businesses are dry cleaners, auto repair workshops, hospitals, exterminators, and photo processing centres (alumur and kara, 2007). household hazardous waste (hhw) is also included in msw. hhw can be generated from a number of household products including paint, garden pesticides, pharmaceuticals, photographic chemicals, detergents, personal care products, fluorescent tubes, waste oil, heavy metal containing batteries, wood treated with dangerous substances, etc. (slack et al., 2005). recyclable materials of msw are those which can be fully or partly recycled at recycling centres and can be reused after treatment processes. this category of msw includes paper, plastics, glass and metals. the third category so called here as garbage includes any waste that is neither hazardous nor recyclable and goes directly to residues disposal sites to get disposed there. in a real-world scenario, sorting the collected wastes into the above-mentioned main categories is performed by intermediate facilities which are often called transfer stations or screening centres. these facilities play a significant role in economizing the costs of the waste management system. a transfer station is a processing site used for the temporary deposition of wastes by collection vehicles. prior to being loaded into larger vehicles, the wastes are sorted and balled into the different sorts (epa, u.s., 2014). concurrently taking multiple waste types as an input, while each has distinct processing flows among the mswm system’s facilities (e.g. transfer stations, recycling-, treatmentand disposal centres), forms a complex network of many interrelated components which requires critical decisions including where to establish these facilities and how to route wastes among the facilities. the mswm network can be even more complex when the types of hazardous wastes and their required treatment technologies are taken into consideration as a real world case. different types of hazardous wastes require distinctly different treatment processes and technologies such as incineration and chemical treatment. that is, a compatible treatment technology must be selected based on the waste characteristics (nema and gupta, 1999). in addition, an integrated mswm system (as the one proposed here) requires consideration of different disposing processes and therefore different disposal centres. in practice, disposal centres for hazardous wastes are different from other disposal centres because more strict regulations and controls such as leachate are applying to them (epa, nsw, 1996). the existing mathematical models for the location-routing problem (lrp) in waste management have been focused on hazardous wastes only (nema and gupta, 1999; list and mirchandani, 1991; jacobs and warmerdam, 1994; giannikos, 1998; alumur and kara, 2007; zhao and zhao, 2010 and samanlioglu, 2013). a comprehensive, integrated model for the optimization of the locations of the waste management system’s all components and routes s22 australasian journal of information systems asefi, lim & maghrebi 2015, vol. 19, pp. s21-s35 model for location routing among them is proposed in this paper. this is a novel approach because the existing approaches ignore the locations of recycling centres and do not include transfer stations even though these facilities are inseparable parts of a waste management system. optimization cannot be achieved without considering their interrelations with the other system’s components. mathematical models can be utilized to design the system by describing objectives, component interactions and possible management strategies. a comprehensive mathematical model can provide systemic means with which the decision-makers can make an optimal management plan (nema and gupta, 1999). this paper aims to provide a systemic mathematical model for mswm to optimize the locations of its components and provide the optimum routing plan for different types of waste transportation flows in order to minimize the total cost of transportation and facility establishment. 2 municipal solid waste management modelling of mswm has been widely studied in the past several decades. the early models on mswm dealt with specific aspects of the problem such as vehicle routing or transfer station siting while having practical shortcomings such as neglect of recycling centres and oversimplicity of the model with a single waste type only. the latest researches have focused on refinement of various optimization approaches for the development of more practical models of reliable mswm (sharholy et al., 2008). comprehensive reviews and classifications of the proposed models in waste management are presented in berger et al. (1991) and tanskanen (2000). among the developed models on different aspects of waste management, lrp has been particularly applied to hazardous waste management. in lrp, decisions should be made simultaneously for the establishment of a single or a set of facilities and the determination of a number of routes for each facility in order to minimize the fixed cost of opening facilities and the cost of shipment between the facilities via the determined routes (lin et al., 2014). development of mathematical models for the optimal site selection of treatment and disposal centres and for the efficient transportation routes from waste sources to these facilities have been addressed in many researches (list and mirchandani, 1991; revelle et al.,1991; jacobs and warmerdam, 1994; current and ratick,1995; giannikos, 1998; nema and gupta,2003; alumur and kara ,2007; zhao and zhao, 2010; samanlioglu, 2013 and boyer et al.,2013). minimization of the total cost consisting of the opening cost of the facilities and the cost of transportation in the network together with minimization of the risks which are measured by people’s exposure to the facilities and routes have been the main objective of their research. concurrent optimization of site selection and routing in a waste management system has been studied by some researchers. zografos and samara (1990) utilized a goal-programming approach to model a hazardous waste management system. their main objective is to minimize the total travelling time and risks. however, they considered only a single type of hazardous waste and allocation of only one treatment centre to each generation node. list and mirchandani (1991) proposed a model for a hazardous waste location-routing problem while considering only treatment and disposal centres. revelle et al. (1991) developed a mathematical model specifically for nuclear waste to locate storage facilities and select shipment routes while only one type of waste was taken into account. jacobs and warmerdam (1994) proposed a mathematical model for a hazardous waste lrp to minimize a linear combination of costs and risks in time while site selection of the storage and disposal facilities were included in their addressed problem. current and ratick (1995) included equity in addition to costs and risks in their developed model. their model aims to maximize the equity of the risks of the system’s components and to minimize other costs and risks. wyman and kuby (1995) also presented a model for a hazardous waste management system aiming to optimize the total cost, inequity and the associated risks and to choose the optimum treatment technology. giannikos (1998) utilized a goal-programming approach to a hazardous waste lrp. in addition to typical cost objective s23 australasian journal of information systems asefi, lim & maghrebi 2015, vol. 19, pp. s21-s35 model for location routing functions, an equitable distribution of disutility caused by the operation of treatment centres was also included in the proposed model. different types of hazardous wastes and compatibility between waste types and treatment technologies have been important real-world constraints in some researches. nema and gupta (2003) addressed a hazardous waste lrp considering different types of wastes and wastetechnology compatibility for treatment. they proposed a model to select the locations of treatment and disposal centres and to route wastes to these facilities without considering recycling centres in the network. alumur and kara (2007) developed a mathematical model for a hazardous waste lrp to minimize the total cost of transportation and facilities establishment and to minimize the transportation risks which are measured by the number of people who are exposed to the allocated routes. they studied site selection of disposal and treatment centres and the routing problem of different types of hazardous wastes from generation nodes to compatible treatment centres and from treatment centres to disposal facilities while recycling centres were not considered in their addressed problem. zhao and zhao (2010) proposed a goal-programming optimization approach to a hazardous waste management system to select the locations of treatment and disposal centres, and to route multiple hazardous wastes. they also studied the routing problem of hazardous wastes from generation nodes to compatible treatment centres and from treatment centres to disposal facilities. recycling centres have been factored in by a few researchers. samanlioglu (2013) developed a more comprehensive model based on the models introduced by alumur and kara (2007) and zhao and zhao (2010) by taking some additional real-world aspects into account, such as site selection of recycling centres and determination of waste routes to and from recycling centres. boyer et al. (2013) proposed a mathematical model for a single type hazardous waste locationrouting problem when site selection of recycling centres and direct routing of residues from generation nodes to disposal centres are applied without considering different waste types and waste-technology treatment compatibility constraints. regardless of waste types and objective functions, the fundamental principle of our defined problem and proposed model are close to those of alumur and kara (2007), samanlioglu (2013) and boyer et al. (2013). however, our model covers recycling centres and routing of wastes to and from these centres which were not considered by alumur and kara (2007). moreover, samanlioglu (2013) and alumur and kara (2007) did not study the ability of direct routing between generation nodes and disposal centres in their proposed models. beyond these differences, different types of waste and waste-technology treatment compatibility are also included in our model unlike similar approach by boyer et al. (2013). in addition, intermediate transfer stations and distinct disposal facilities for hazardous and non-hazardous waste residues have not been taken into consideration so far in the literature. in summary, a new location-routing problem for an integrated mswm system is formulated in this paper. considering real-world aspects, our mathematical model is presented to minimize the total cost of the system including the transportation costs and the opening costs of the system’s facilities. the formulation is also tested with real data acquired across new south wales, australia. 3 location-routing problem a schematic display of our defined problem is presented in figure 1. the diagram in figure 1 starts with transportation of wastes from generation nodes to transfer stations ( , ) where the wastes are sorted and balled into the recyclable, hazardous and garbage balls. after the sorting process, the balled wastes are sent to their distinct destinations by larger vehicles. recyclable wastes are transferred to recycling centres ( , ); hazardous wastes are sent to treatment centres with compatible technologies ( , , ) and garbage which are neither hazardous nor recyclable is transported to non-hazardous disposal centres ( , ). after the treatment process, a waste mass is reduced ( , ) and the parts which are recyclable are sent to recycling centres ( , ) s24 australasian journal of information systems asefi, lim & maghrebi 2015, vol. 19, pp. s21-s35 model for location routing and non-recyclable waste residues are transported to final hazardous disposal centres ( , ). at the recycling centres, after the recycling process, recyclable wastes are sent to the markets or other factories ( ) and the generated waste residues are sent to the final non-hazardous disposal centres ( , ). generation node (geni) recycling centre (hri) treatment centre (trw,q,i) non-hazardous disposal centre (d’ri) hazardous disposal centre (dri) xi,,jjj,,,xi,j pi,j,,pi,j li,j,,li,j ki,j,,ki,j vi,jj,,vi,j zi,j,,zi,j rwrr ,qrw,q ii yw,i,jj,,yw,i,j transfer station (kri) figure 1. schematic view of the defined problem the defined problem consists of concurrent site selection of the locations of the system’s facilities (e.g. transfer stations, treatment-, recyclingand disposal centres) from the candidate locations and the determination of routes and amounts of shipments among the selected locations to minimize the total cost of transportation and facility establishment. 4 mathematical model a mixed integer programming is proposed in this study to formulate the defined msw location-routing problem with respect to the minimization of the total cost of transportation and facilities establishment. the nodes of the transportation network consist of generation nodes, potential transfer stations, potential treatment facilities, potential recycling centres, potential hazardous disposal centres, potential non-hazardous disposal centres, and a combination of any of the above. our mathematical model is formulated to determine: (i) the locations of transfer stations and the routes for wastes to be transported to the transfer stations, (ii) the locations of treatment centres and their adopted technologies, and the routes for different types of hazardous wastes to be transported to compatible treatment centres, (iii) the locations of recycling centres and the routes for recyclable wastes and waste residues to be transported to the recycling centres, (iv) the locations of non-hazardous disposal centres and the routes for waste residues to be transported to these centres, and (v) the locations of hazardous disposal centres and the routes for hazardous waste residues to be transported to these centres. the defined problem can be proven to be np-hard for large scales (alumur and kara, 2007; samanlioglu, 2013). the sets, parameters and decision variables of our model are presented below. sets: n = (v,a) is a transportation network of nodes v and arcs a g = {1,…,g} is a set of waste generation nodes, g v k = {1,…,k} is a set of potential transfer station nodes, k v t = {1,…,t} is a set of potential treatment nodes, t v s25 australasian journal of information systems asefi, lim & maghrebi 2015, vol. 19, pp. s21-s35 model for location routing d = {1,…,d} is a set of potential hazardous disposal nodes, d v d' = {1,…,d' } is a set of potential non-hazardous disposal nodes, d' v h = {1,…,h} is a set of potential recycling nodes, h v w = {1,…,w} is a set of hazardous waste types q = {1,…,q} is a set of treatment technologies parameters: is a transportation cost per unit of waste on link (i,j) a, i g, j k is a transportation cost per unit of hazardous waste on link (i,j) a, i k, j t is a transportation cost per unit of hazardous waste residue on link (i,j) a, i t, j d is a transportation cost per unit of non-hazardous waste residue on link (i,j) a, i h, j d' is a transportation cost per unit of recyclable waste on link (i,j) a, i k, j h is a transportation cost per unit of recyclable waste residue on link (i,j) a, i t, j h is a transportation cost per unit of garbage waste on link (i,j) a, i k, j d' is a fixed cost of opening a transfer station at node i k, is a fixed cost of opening a treatment technology q q at node i t is a fixed cost of opening a hazardous disposal centre at node i d is a fixed cost of opening a non-hazardous disposal centre at node i d' is a fixed cost of opening a recycling centre at node i h is an amount of waste generated at generation node i g, is a proportion of hazardous waste type w w sorted at transfer station node i k is a proportion of recyclable waste sorted at transfer station node i k is a proportion of garbage waste sorted at transfer station node i k, is a proportion of mass reduction of hazardous waste type w w treated with technology q q, is a proportion of recycling of hazardous waste type w w treated with technology q q is a proportion of total waste recycled at node i h, is a capacity of treatment technology q q at node i t is a capacity of recycling centre at node i h is a capacity of hazardous disposal centre at node i d is a capacity of non-hazardous disposal centre at node i d' is a capacity of transfer station at node i k, is the minimum amount of hazardous waste required to establish treatment technology q q at node i t is the minimum amount of recyclable waste required to establish a recycling centre at node i h is the minimum amount of hazardous waste residue required to establish a hazardous disposal centre at node i d s26 australasian journal of information systems asefi, lim & maghrebi 2015, vol. 19, pp. s21-s35 model for location routing is the minimum amount of garbage and non-hazardous waste residue required to establish a nonhazardous disposal centre at node i d' is the minimum amount of waste required to establish a transfer station at node i k, is 1 if hazardous waste type w w is compatible with technology q q; or 0 otherwise decision variables:, is an amount of waste transported through link (i,j) a, i g, j k , , is an amount of hazardous waste type w w transported through link (i,j) a, i k, j t, is an amount of recyclable waste transported through link (i,j) a, i k, j h , is an amount of garbage waste transported through link (i,j) a, i k, j d' , is an amount of treated recyclable waste residue transported through link (i,j) a, i t, j h, is an amount of waste residue transported through link (i,j) a, i h, j d' , is an amount of hazardous waste residue transported through link (i,j) a, i t, j d is an amount of waste transferred at node i k, , is an amount of hazardous waste type w w treated at node i t with technology q q is an amount of hazardous waste residue disposed at node i d is an amount of non-hazardous waste residue disposed at node i d' is an amount of waste recycled at node i h, is 1 if treatment technology q q is established at node i t; or 0 otherwise is 1 if hazardous disposal centre is established at node i d; or 0 otherwise is 1 if non-hazardous disposal centre is established at node i d'; or 0 otherwise is 1 if recycling centre is established at node i h; or 0 otherwise is 1 if transfer station is established at node i k; or 0 otherwise. the main objective of the problem is to minimize the total cost under the given constraints as follows: ( ) = , , + , , , + , , + , , + , , + , , + , , + + , , + + + (1) = , (2) , = (3) s27 australasian journal of information systems asefi, lim & maghrebi 2015, vol. 19, pp. s21-s35 model for location routing , = , , (4) = , (5) (1 , ) = , (6) , , = , , , (7) , , 1 , 1 , = , (8) , , 1 , , = , (9) , + , = (10) (1 ) = , (11) , = (12) , + , = (13) (14) , , , , , (15) (16) (17) (18) (19) , , , , , (20) (21) (22) (23), , , , , , (24)( , , , , , , , , , , , , , , , , , , , , , ) { } (25) , , , , , {0,1} (26) the objective function given in equation (1) minimizes the total cost including the transportation cost of different waste types and waste residues and the fixed cost of opening transfer stations, treatment-, recyclingand disposal centres. the transportation cost is measured by the unit transportation cost times the amount of shipped wastes on a given link. s28 australasian journal of information systems asefi, lim & maghrebi 2015, vol. 19, pp. s21-s35 model for location routing equation (2) is the flow balance constraint of the flows from generation nodes to transfer stations. this constraint ensures that all the generated wastes are transported to transfer stations. equation (3) indicates the total amount of the transported wastes to transfer stations that have to be sorted and balled at these centres. equations (4)-(6) show the flows of hazardous wastes, recyclables and garbage regarding their proportions from transfer stations to treatment, recycling and non-hazardous disposal centres, respectively. equation (7) ensures that all hazardous wastes transported to treatment centres have to be treated. equations (8) and (9) provide the flow from treatment centres to hazardous disposal centres and recycling centres regarding the ratios of recycling and mass reduction associated with different treatment technologies at treatment centres, respectively. equation (10) presents the flow from transfer stations and treatment centres to recycling centres. equation (11) provides the flow of generated residues from recycling centres to non-hazardous disposal centres. equation (12) shows the flow of hazardous waste residues from treatment centres to hazardous disposal centres ensuring that all the transported hazardous residues to these centres have to be disposed at these centres. equation (13) is the flow of garbage and generated non-hazardous residues from screening and recycling centres to non-hazardous disposal centres and ensures that the total amount of transported residues to these centres has to be disposed at these centres. equations (14)-(18) determine the capacity limitation for transfer stations, treatment, recycling and disposal centres, respectively. equations (19)-(23) ensure that minimum amounts of different waste types and waste residues have to exist in order to open the related facilities i.e. transfer stations, treatment, recycling, disposal centres, respectively. equation (24) presents the compatibility limitation for treatment of different types of hazardous wastes with different treatment technologies. equations (25) and (26) are utilized for stating nonnegative and binary variables, respectively. 5 experimental results and discussion recently, the amount of waste generation in many countries has increased as a result of population growth, technological advances, increase in trade in chemical products and improved health care. australia has one of the highest rates of waste generation per capita in the world (abs, 2012). from 1997 to 2012 the rate of waste generation in australia has sharply increased by 145% compared with the moderate increase rates of 22% and 64% in the population and gross-value-added respectively. australia’s population is estimated to be 35.5 million by the year 2056 which will place increasing pressure on the natural environment and its resources (abs, 2013). new south wales (nsw) is australia’s most populous state with a population of 7.5 million (abs, 2014). the total amount of domestic wastes generated in nsw was 3.47 million tonnes between 2012 and 2013 with the total recycling ratio of 46.5% (epa, nsw, 2014). nsw’s hazardous waste tracking system also recorded 260,920 tonnes of hazardous wastes generated within the state in 2010-11 (kmh environmental, 2013). nsw is divided into 12 statistical divisions (sds) where each sd consists of some statistical subdivisions (sss). sydney sd is the most populous sd in nsw and includes two-thirds of the state's population. in order to cover the whole state in the model analysis and reach more accurate results, all of its 14 sss are selected so that 25 nodes represent the whole state with the other 11 sds. all of the 25 waste generation nodes (excluding inner sydney due to dense urban environment) are also assumed to be candidate sites for transfer station, treatment, recycling, and disposals centres, concurrently. the study area that has been considered as a case study is illustrated in figure 2 and the corresponding key information is summarized in table 1. s29 australasian journal of information systems asefi, lim & maghrebi 2015, vol. 19, pp. s21-s35 model for location routing figure 2. twenty-five nodes and their corresponding locations on the map of nsw node id district name classification population waste generation (ton/year) 01 inner sydney ss 334,634 143,236 02 eastern suburbs ss 247,533 109,894 03 st george-sutherland ss 445,532 229,229 04 canterbury-bankstown ss 317,412 155,188 05 fairfield-liverpool ss 360,294 173,993 06 outer south western sydney ss 241,433 125,148 07 inner western sydney ss 179,892 81,097 08 central western sydney ss 323,891 155,290 09 outer western sydney ss 315,985 176,145 10 blacktown ss 284,692 155,476 11 lower northern sydney ss 308,068 142,854 12 central northern sydney ss 433,118 228,938 13 northern beaches ss 237,514 132,865 14 gosford-wyong ss 306,376 194,638 15 hunter sd 624,296 397,438 16 illawarra sd 417,901 243,654 17 richmond-tweed sd 232,948 126,013 18 mid-north coast sd 300,006 187,875 19 northern sd 180,067 114,002 20 north western sd 115,419 59,151 21 central west sd 178,840 128,578 22 south eastern sd 209,270 120,650 23 murrumbidgee sd 154,663 84,297 24 murray sd 116,471 67,451 25 far west sd 22,817 18,056 table 1. features of the case study to assess our model, the data of recyclables and garbage and their proportions are directly derived from the reported figures from 152 councils within the state (epa, nsw, 2014). for hazardous wastes, due to the lack of distinct data on the amount of generation in each division, we assume that they are proportional to the populations of the defined nodes. similar to alumur and kara (2007) and samanlioglu (2013), two types of treatment technologies for hazardous wastes are considered: incineration and chemical treatments. three types of hazardous wastes are also considered: the first type is compatible with the incineration treatment technology such as clinical wastes; the second type includes wastes which are compatible with chemical treatment e.g. flammable hazardous wastes; and the third type s30 australasian journal of information systems asefi, lim & maghrebi 2015, vol. 19, pp. s21-s35 model for location routing involves wastes which can be treated with both incineration and chemical technologies such as organic hazardous wastes. the proportion of hazardous waste residues which are suitable for recycling after chemical and incineration treatments are also taken as 30% and 0% respectively; and, the percent of waste residues which are sent to disposal centres after the recycling process is assumed to be 5%. to calculate the total cost of transporting wastes, the amounts of shipment, the transportation distance and the cost of fuel are considered for each pair of the nodes in the network. a code was developed to utilize google map api to calculate actual distances derived from centroids of districts polygons using arcmap 10.2. here, it is assumed that the average fuel consumption for a truck is 0.3 litre per km, and the average cost of the fuel is $1.5 per litre in australia. in order to consider special care and equipment associated with transportation of hazardous wastes and hazardous residues, their unit cost of transportation in the network is assumed to be 43% higher than other types of wastes. (alumur and kara, 2007; samanlioglu, 2013). lastly, other costs of transportation include insurance, driver salary and truck depreciation. the constant factor of 2 is multiplied by the unit cost of transportation in order to include these costs (boyer et al. 2013). based on alumur and kara (2007) and samanlioglu (2013), and considering the current situations of the studied area, the other parameters are determined and listed in table 2. facility \ parameter establishment cost ($m) capacity (ton) minimum required to establish (ton) transfer station 35 1,500 250 treatment centre 87.5 1,500 50 recycling centre 35 700 250 hazardous disposal centre 35 1,500 100 non-hazardous disposal centre 35 1,500 500 table 2. parameters of mswm facilities the defined problem has been solved using the general algebraic modelling system (gams) software with cplex solver version 12.4.0.1 on a redhat ® centos ® 5.9 linux server with 8 3.60 ghz intel ® xeon ® cpus with a 198 gb physical memory. table 3 summarizes the results for the best obtained solution. at this solution, establishment of 7 transfer stations, 1 treatment centre for the each technology, 6 recycling centres, 1 hazardous disposal centre and 1 non-hazardous disposal centre are suggested. table 4 represents the suggested locations for these facilities. as can be seen, the model results in the minimum possible number of each of the facilities and utilizing the capacity of facilities at the maximum extent. the locations of different types of the facilities are also suggested to be the same in some nodes. figure 3 illustrates the resulted routing strategy for the defined mswm system. transportation cost ($m) total cost ($m) total cost lower bound ($m) cpu time (seconds) number of iterations 10.4 815 612 544 2,361,862 table 3. the optimal solution of the model transfer station chemical treatment centre incineration treatment centre recycling centres hazardous disposal centres nonhazardous disposal centres 2,3,4,8,14,15,21 7 7 2,3,4,14,15,21 7 2,4,15,21 table 4. suggested locations for opening facilities at optimal solution s31 australasian journal of information systems asefi, lim & maghrebi 2015, vol. 19, pp. s21-s35 model for location routing figure 3. the optimal routing plan in order to analyse the sensitivity of the cost of transportation and the total cost to the number of transfer stations we varied its optimal value from 7 (case 1) to 8 (case 2), 9 (case 3), 10 (case 4) and 11 (case 5). table 5 summarises the results for the above five cases. the results indicate that, while the total cost rises gradually as the number of transfer stations increases, the transportation cost fluctuates at values near the optimal result and the increase in the number of transfer stations does not reduce the cost of transportation in all cases (figure 4). problem number of transfer stations transportation cost ($m) total cost ($m) case 1 7 10.4 815 case 2 8 16.2 856 case 3 9 10.4 885 case 4 10 9.12 919 case 5 11 13.0 958 table 5. transportation and total cost values for different numbers of transfer stations 7 hazardous disposal node hazardous disposal node transfer stations transfer stations 4 15 21 non-hazardous disposal nodes non-hazardous disposal nodes 14 15 21 8 2 3 4 2 transfer stations transfer stations 4 14 15 21 8 2 3 7 treatment node treatment node 32 4 8 14 15 21 2 4 15 21 non-hazardous disposal nodesnon-hazardous disposal nodes 2,3,4,14 15,212 3 4 14 15 21nodes recycling nodes 22 22 33 44 3, 4 3, 4 44 2, 4 2, 4 88 88 88 22 4, 8, 14 4, 8, 14 22 1414 1515 33 14, 15 14, 15 1414 15, 21 15, 21 2121 2121 2121 2121 2121 2121 transfer stations transfer stations 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 generation nodes generation nodes s32 australasian journal of information systems asefi, lim & maghrebi 2015, vol. 19, pp. s21-s35 model for location routing figure 4. the effect of the number of transfer stations on the system’s costs 6 concluding remarks in this study, a new location-routing problem for mswm is proposed by taking applicable aspects of previously developed models in the area of waste management and new real-world features into account. the main contribution of this paper is a mathematical model of mswm where different waste types are simultaneously factored in, and transfer stations and distinct disposal facilities for hazardous and non-hazardous residues are included in a mixed integer mathematical model. taking these new aspects into consideration together with the other realworld features such as waste-treatment compatibility, locations of recycling centres, possibility of locating different facilities in the same node, and considering minimum requirement amount constraints, led us to develop a comprehensive and yet more practical model in waste management context. the main objective of the presented study is to find the optimal locations of the mswm system’s facilities consisting of transfer stations, treatment, recycling and disposal centres, and to determine the optimum routing strategy to and from these facilities in order to minimize the total cost of establishment and transportation. as a case study, the proposed model was tested with the data collected across nsw, australia. many real-world features of an integrated mswm system have been taken into consideration and realistically implemented in the model. the 25 districts based on statistical divisions and statistical subdivisions zoning approaches were selected to represent the whole state of nsw for the analysis. 24 districts out of the 25 generation nodes were also assumed to be candidate locations for each of the six types of the system’s facilities (transfer stations, two types of treatment technologies, recycling centres, hazardous disposal centres and non-hazardous disposal centres) leading to a network with 24×6 potential location nodes. in terms of the number of all the candidate location nodes in the network, the presented application in this study is by far the largest instance studied in the literature. as it was mentioned by alumur and kara (2007) and samanlioglu (2013) the problem cannot be solved for a large-scale instance even with available high performance computing facilities within a polynomial time. hence, in order to find a solution for mswm in a practical time, development of an efficient heuristic method even with a larger size would be worthwhile for future work. a multi-objective optimisation approach to the defined problem can be another future research direction. as the amount of generated wastes is not always deterministic, one can apply stochastic programming techniques to tackle the problem. finally, factoring in different types of recyclable wastes and waste-recycling technology constraint can improve the applicability of the proposed model in reality. 700 750 800 850 900 950 1000 0 2 4 6 8 10 12 14 16 18 case 1 case 2 case 3 case 4 case 5 transportation cost ($m) total cost ($m) s33 australasian journal of information systems asefi, lim & maghrebi 2015, vol. 19, pp. s21-s35 model for location routing references alumur, s., & kara, b. y. (2007). a new model for the hazardous waste location-routing problem. computers & operations research, 34(5), 1406-1423. australian bureau of statistics. (2008). regional population growth, australia, cat no. 3218.0. canberra: australian government. australian bureau of statistics. (2012).year book. australia, cat no. 1301.0. canberra: australian government. australian bureau of statistics. (2013).waste account, australia, experimental estimates, cat no. 4602.0.55.005. canberra: australian government. australian bureau of statistics. (2014). australian demographic statistics, cat. no. 3101.0. canberra: australian government. berger, c., savard, g., & wizere, a. (1999). eugene: an optimisation model for integrated regional solid waste management planning. international journal of environment and pollution, 12(2), 280-307. boyer, o., sai hong, t., pedram, a., mohd yusuff, r. b., & zulkifli, n. (2013). a mathematical model for the industrial hazardous waste location-routing problem. journal of applied mathematics, 2013. current, j., & ratick, s. (1995). a model to assess risk, equity and efficiency in facility location and transportation of hazardous materials. location science, 3(3), 187-201. dawson, p., & spedding, t. (2009). simulation modelling and strategic change: creating the sustainable enterprise. australian journal of information systems, 16(2), 71-80. environmental protection agency (epa). (2002). waste transfer stations: a manual for decision-making. epa530-r-02-002. united states. environmental protection authority. (1996). environmental guidelines: solid waste landfills, nsw. environmental protection authority. (2014). local government waste and resource recovery data report, nsw. giannikos, i. (1998). a multiobjective programming model for locating treatment sites and routing hazardous wastes. european journal of operational research, 104(2), 333-342. jacobs, t. l., & warmerdam, j. m. (1994). simultaneous routing and siting for hazardouswaste operations. journal of urban planning and development, 120(3), 115-131. kmh environmental. (2011). hazardous waste data assessment. vic lin, c., choy, k.l., ho, g.t.s., chung, s.h. & lam, h.y. (2014). survey of green vehicle routing problem: past and future trends. expert systems with applications, 41, 1118– 1138. list, g., & mirchandani, p. (1991). an integrated network/planar multiobjective model for routing and siting for hazardous materials and wastes. transportation science, 25(2), 146-156. nagy, g. & salhi, s., (2007). location-routing: issues, models and methods. european journal of operational research, 177, 649–672. nema, a. k., & gupta, s. k. (1999). optimization of regional hazardous waste management systems: an improved formulation. waste management, 19(7), 441-451. nema, a. k., & gupta, s. k. (2003). multiobjective risk analysis and optimization of regional hazardous waste management system. practice periodical of hazardous, toxic, and radioactive waste management, 7(2), 69-77. s34 australasian journal of information systems asefi, lim & maghrebi 2015, vol. 19, pp. s21-s35 model for location routing revelle, c., cohon, j., & shobrys, d. (1991). simultaneous siting and routing in the disposal of hazardous wastes. transportation science, 25(2), 138-145. samanlioglu, f. (2013). a multi-objective mathematical model for the industrial hazardous waste location-routing problem. european journal of operational research, 226(2), 332-340. sharholy, m., ahmad, k., mahmood, g., & trivedi, r. c. (2008). municipal solid waste management in indian cities–a review. waste management, 28(2), 459-467. slack, r. j., gronow, j. r., & voulvoulis, n. (2005). household hazardous waste in municipal landfills: contaminants in leachate. science of the total environment, 337(1), 119-137. tanskanen, j. h. (2000). strategic planning of municipal solid waste management. resources, conservation and recycling, 30(2), 111-133. wyman, m. m., & kuby, m. (1995). proactive optimization of toxic waste transportation, location and technology. location science, 3(3), 167-185. zhao, j., & zhao, j. (2010). model and algorithm for hazardous waste location-routing problem. in proceedings of the international conference of logistics engineering and management (iclem’10) (pp. 2843-2849). zografros, k. g., & samara, s. (1989). combined location-routing model for hazardous waste transportation and disposal. transportation research record, (1245). copyright: © 2015 asefi, lim and maghrebi. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. s35 microsoft word ajis_18(3)_v10.docx volume 18 number 3 2014 australasian journal of information systems issn 1449-8618 eissn 1326-2238 australasian journal of information systems volume 18 number 3 2014 1 australasian journal of information systems (issn 1326-2238 eissn 1449-8618) published by australasian chapter of the association for information systems (aais) website hosted by the australian computer society (acs) advisory board dr deborah bunker, the university of sydney (acphis president) dr glenn stewart, queensland university of technology (aais president) dr dennis viehland (phis-nz president) dr mark toleman, university of southern queensland (acphis past president) dr don kerr, university of the sunshine coast (aais past president) dr felix tan (phis-nz nominee) editor-in-chief dr john lamp, deakin university section editors in this issue nabeel al-qirim, stephen burgess, craig parker, john campbell, dale mackrell, anette mills, adele lee, lemai nguyen, erwin fielt, darryl coulthard, michael chilton, amir h. ghapanchi, helen hasan, michael lane, christine owen, mark freeman, eric chan, ryan ko reviewers in this issue heather gray, gerald goh, lundy lewis, lois burgess, brychan thomas, steven curnin, karyn bosomworth, matt halliwell, holly tootell, kenneth stevens, nipon parinyavuttichai, sai honig, vivek agrawal previous editors-in-chief dr kathy lynch, university of the sunshine coast (2012-2013) dr jon heales, university of queensland (2009-2012) dr craig mcdonald, university of queensland (2006-2009) dr rob macgregor, university of wollongong (founding editor: 1993 2006) submissions and subscriptions details of journal scope and instructions to authors can be found on the journal’s website: http://journal.acs.org.au/index.php/ajis/about/submissions#authorguidelines. the journal is free to access online, however, please contact the editor-in-chief for information regarding purchase of hard copies of the journal. enquiries and comments editorial enquires should be directed to ajis editor-in-chief (ajis.eic@gmail.com) ajis vol. 6 no. 1 september 1998 method points: towards a metric for method complexity graham mcleod, university of cape town, private bag, rondebosch, 7700 south africa mcleod @ iafrica.com abstract a metric for method complexity is proposed as an aid to choosing between competing methods, as well as in validating the effects of method integration or the products of method engineering work. it is based upon a generic method representation model previously developed by the author and adaptation of concepts used in the popular function point metric for system size. the proposed technique is illustrated by comparing two popular i.e. deliverables with counterparts in the object oriented unified modeling language (uml). the paper recommends ways to improve the practical adoption of new methods. key words methods, methods engineering, metrics, uml introduction methods engineering, as used in this paper, is the process whereby methodologists develop and enhance methods related to information systems in a disciplined way. as defined by james odell: "method engineering is the coordinated and systematic approach to establishing work methods" (brinkkemper, lyytinen, welke, 1996). methods engineers have several objectives when performing this activity. these may include: • providing a structured, repeatable process for practitioners to follow in producing a desirable result • integration of techniques to produce a comprehensive approach to address a broader problem • updating existing methods to include more sophisticated techniques or deal with new requirements • specification of capable model notation and representation forms which will be accessible to practitioners and enhance communications between users research by the author and others (bosman r, mcleod g, tanfield j, 1992; murphey, harvey, mattison, 1990) has shown that success in adoption and use of methods is, in part, determined by the degree of fit of the methods to the problem and the difficulty of becoming proficient in the use of such methods. the latter is directly influenced by the complexity of the method. as yet, there is no published technique to measure the complexity of a method so that quantitative comparisons can be made. this paper is a first attempt at establishing such a metric. modeling methods the author previously developed a generic method model which is capable of holding the knowledge describing a method (mcleod, 1992, 1993, 1995). the method description includes definitions of the tasks performed, the deliverables created and the resources required to carry out the tasks. these types of method components each occupy a facet within the method model. 48 ajis vol. 6 no. 1 september 1998 mcleod method model j» resources figure 1 method model facets within each facet, there is a hierarchy of nodes, representing the collation of lower level nodes or the decomposition of higher level nodes. the task (process) facet can thus hold a work breakdown structure, while the product facet may hold a configuration of deliverables and how these are collated into larger products. components within a facet may be linked by dependency relationships. these form a network similar to a critical path network. components can also be linked to others across the facets, for example: • a product may be linked to the task(s) which create it or use it as input • a task may be linked to the types of resources necessary to complete it • a resource could be linked to the products it may hold (e.g. a case repository which would hold a variety of deliverables identified in the product facet) there is no restriction on the level or number of these links and they may connect any type of component to any other. the components in each facet are further described by a set of attributes and behaviours, sufficient to fully describe the product, task or resource. attributes represent data items whose values will describe an instance of that type of component. 49 ajis vol. 6 no. 1 september 1998 product facet node structure product name parent children prerequisites subsequents representation data content purpose validation rules example quality standards associated tasks associated resources permitted states estimating method tool support figure 2 product facet structure the model has proven capable of describing a wide variety of methods in a consistent way, and is thus seen as a suitable basis for representation of methods with a view to the application of metrics. function points the function point metric was first described by albrecht (1979) at ibm canada to enable technology-independent estimating of the size (functional weight) of a software product. this was useful in the estimation of software project effort, and in determining relative productivity of development groups. function point (fp) counting methods were standardised by the international function point users group (ifpug), which updates and improves the technique on an ongoing basis (ifpug, 1994). many organisations have adopted the fp metric, and it has proved to be a useful and fairly consistent (though not perfect) technique. a fp count for a software product is determined by counting the number of: • inputs (that add data to the system) • outputs (that provide information to end users) • queries (online requests for information) • interfaces (exchange of data with other systems) • main files (persistent data maintained by the system) 50 ajis vol. 6 no. 1 september 1998 that the system will provide/maintain, categorising each of these as simple, average or complex, and multiplying the number of deliverables by a function point credit for the type of deliverable and its complexity (e.g. a simple input may be worth 3 fp while a complex input may be worth 5 fp). these scores are totalled to produce a raw fp count. a second process takes into account adjustment factors which can influence the relative difficulty (hence effort) of delivering the particular project. these include factors such as performance, number of target sites, data distribution, online processing, reusability and others. the influence factors are scored on a scale of 0 to 5. they are totalled and used to calculate a degree of influence, which, in turn, is used to adjust the raw fp count. the adjusted fp count can be used in conjunction with previously measured productivity figures (in fp per person month, or person days per fp) to calculate the likely effort required to create a proposed software product. for completed products, an accurate count can be produced. with data about actual effort expended on development, this can be used to determine the relative productivity of the development process. function point calculation determine rawfp countinputs, outputs, queries, interfaces entity or object model determine adjustment environmental factors ^calculate" adjusted fp count figure 3 calculating function points method points a method or method fragment can be regarded as behaving in much the same way as a software system: • it interacts with its environment • it has users (practitioners) who provide it with information and receive outputs (deliverables) from the processes it performs • it manages information to support the various processes and outputs 51 ajis vol. 6 no. 1 september 1998 • it interfaces with other methods (or method fragments) which may precede or follow it, or operate in parallel to it formal definition of the method in a standard way allows us to collect information for measurement in a similar way to which building a context diagram and entity or object model for a software system allows us to collect information for input to the fp technique. thus, a model can be built of a method which allows use of a similar process to determine its complexity to that used in function points for a system described in a model. we recommend that counting should proceed in the following way: 1. express the method to be counted in the method model in terms of tasks, resources and deliverables 2. determine the counts for each type of deliverable specified in the method. we distinguish three types of deliverables: • graphical deliverables such as diagrams and models. examples would be entity relationship diagrams, data flow diagrams, class hierarchy diagrams and project management network diagrams • tabular deliverables which can be expressed as columns and rows or as records in a relational table. examples would include the definition of the attributes of a data group, which may have columns for name, type, length and valid ranges; and a system consistency matrix • textual deliverables which include long descriptions and narratives as well as more structured reports and hypertext documents we will discuss the counting of each type of deliverable in the following sections. 3. determine and add the count for task complexity. this will normally not come into play, unless the tasks are more complex than would be apparent from the deliverables produced. note that for function point counting, an actual deliverable (instance of a description) is used. this yields a size. for method points, we consider the structure of a generically described deliverable (class). since the latter represents a meta deliverable, its variety of options effectively translates not to size, but complexity. as far as possible, a principle of equivalence should apply, whereby an equivalent deliverable expressed in a different notation should generate the same count, provided that its semantic content is the same. likewise, the same deliverable expressed as a graphic model or as text, should yield similar counts. counting graphical deliverable types we identify several components of a graphical model which are relevant to our purpose: • symbol types. these are unique shapes that represent something in the domain being modelled. examples include entity boxes in an entity relationship model, process boxes in a data flow diagram, and class symbols in an object model. • link types. these are unique types of connection between symbols. examples would include a data flow arrow on a data flow diagram, a relationship line on an entity relationship diagram, and inheritance relationships in an object model. • embelishments. these are any unique type of modifier which can be added to a symbol, a link or the model canvas. examples include: text label of a data flow, cardinality indicator on an entity to entity relationship, and indication of a key field identity on a data group symbol. further examples include: a duplicate marker added to a data store on a data flow diagram, a boundary around symbols indicating geographical location or mutual exclusion, and a business rule related to a symbol on an event model. • decomposition. it is common that a symbol on one model can be expanded into another model at a greater level of detail. an example would be a process box (representing a complete system) within a context diagram which may be decomposed to a data flow diagram expressed as a separate model. to determine the count for a graphical deliverable, proceed as follows: • for each unique symbol type, count 1 • for each unique connector type, count .5 • for each unique type of embellishment count .5 • for each type of symbol which can be expanded into another model (which is cross referenced from this one) count .5. if this diagram (model) can contain a reference to a parent model, count .5. 52 ajis vol. 6 no. 1 september 1998 counting tabular deliverable tabular deliverables are those which have a structure including labels and other information which can be expressed in tabular form (e.g as a spreadsheet, or as records in a relational table). typical examples would include: the definition of the attributes of a relation in a relational data model (attribute name, type, length), and the system consistency matrix(create, read, update, delete diagram) used in ie to check that data groups have corresponding processes and vice versa. tabular deliverables may also have embelishments. an example would be underlining attributes which comprise the key when defining a relational table. like graphic models, one tabular model can be expanded into a greater level of detail in another. for example, the value in a cell of one spreadsheet can be the result of a variety of complex factors which are expressed in another spreadsheet. to determine the count for a tabular deliverable, proceed as follows: • for each column, count .25 • for each unique type of embellishment, count .5 • for each type of expansion that this table can contain, count .5 • if this deliverable can hold a reference to a parent deliverable, count .5 counting textual deliverables textual deliverables include long descriptions, narratives and various structured and semi-structured textual documents. examples include a feasibility report, a process narrative for a data flow process symbol, and an elaboration of the legal requirements for a process on a process chart. text deliverables can include hyperdocuments which contain links to other sections of the same document, or to other documents, such as would be found in html documents designed for perusal on the world wide web or an intranet. normal textual (not hypertext) documents can contain references to other documents and deliverables. to determine the count for a textual deliverable, proceed as follows: • for each identified section which the deliverable should include, count .25 • for each page (a4) estimated to be included in an average deliverable of this type, count .25. this is based upon a normal size and font (the reference being times roman at 12 point). • if the document can contain hyperlinks, or references to other documents or deliverables, count .5 • if the document contains an "outline" or "indented list" structure indicative of a hierarchical arrangement, count .5 compensating for task complexity in most cases, the effort involved in tasks will be adequately incorporated in the count by the inclusion of the resultant deliverables. however, some methods may include tasks of considerable complexity, which result in relatively simple deliverables as measured by the above processes. in these cases, we will need to compensate for the effect of such tasks. this is analogous to using feature points, as opposed to function points for systems where there is considerable algorithmic complexity, but relatively little data input, output or storage (software composition technologies, inc., 1997). there is a danger that estimating task complexity could be very subjective. we thus stipulate that tasks should be expressed in the method model according to the following criteria: • tasks should be completely described, so that no external references are required to complete them • each task description should not exceed one a4 page of text. if more is required, the task should be further decomposed. • each task as described should be able to be performed by one person expending 5 days (or less) of effort. where more effort would be required, the task should be decomposed further once the tasks required are expressed in the form described above, we can add a count for task complexity as follows: • count .5 for each task in the hierarchy • subtract 1 for each deliverable produced 53 ajis vol. 6 no. 1 september 1998 • if the result is greater than 0, add it to the method point score; if not, discard it (this is to bring the task complexity factor into play only after a certain default level of effort associated with deliverables is exceeded) sample usage information engineering and uml deliverables to demonstrate the use of the technique, we examine and contrast two components of the information engineering approach (martin, mcclure, 1988) with two components of the newly published unified modeling language (uml) from the collaborators (booch, rumbaugh, jacobsen) at rational (rational, 1997a-d). uml has been submitted to the object management group (omg) for consideration as a possible standard object oriented method. it is thus useful to obtain a benchmark complexity measure for it (and its components) which can be used in comparison with other methods (and possibly other submissions for omg consideration). we have chosen two key deliverables of uml for analysis, viz: • the static structure diagram, which can represent many static aspects of the system being modelled including: classes (types), packages, relationships and instances. these will be contrasted with entity relationship models (erm). • use case diagrams, which are used to capture the interaction of users with facilities provided by the system. these will be contrasted with the i.e. context diagram. the above method components were chosen because they are frequently used, represent static and dynamic perspectives, and perform similar functions within the respective approaches. analysis of the static structure diagram uml allows for some variation in presentation of the static structure diagram. we have selected for analysis the option which shows only the names of classes included, not their expanded notation. the expanded notation could be treated separately as a tabular deliverable type. the static structure diagram can include the following unique symbol types: • class: a descriptor for a set of objects with similar structure, behaviour and relationships • type: a descriptor for objects with abstract state, concrete external operation specifications, and no operational implementations. • template: a parameterised class which describes common characteristics for a family of classes. it must be bound to a set of parameters to form a class. • ternary (and higher order) association: showing a relationship between more than two classes via a diamond symbol. these are counted at 1 point each, for a total of 4. the following unique link types are permitted: • interface: indicates externally visible behaviour of a class, component or other entity (e.g. package) • imports: indicates the dependency of one package upon a class definition in another • binary association: an association between two classes • generalisation: an inheritance relationship • dependency: a dependence of one model element upon another • refinement: indicating the expansion of an element to a greater level of detail these are counted at .5 each, for a total of 3. the following embellishments are identified: • bound element • utility modifier: an indicator added to a class to indicate that the attributes and operations of the class are globally visible. • metaclass modifier, an indicator added to a class to indicate that the instances of the class are classes. • pathname modifier: used to show that a class included in the model actually resides in another package. • on associations: 54 ajis vol. 6 no. 1 september 1998 • association name • association class symbol • association role (indicates multiplicitly, ordering, qualifier, navigability, aggregation, rolename) • or association indicator dashed line, indicating optional association « composition • details: an ellipsis indicating that not all items are shown on the current diagram • constraints: overlapping, disjoint, complete or incomplete modifiers on relationships between subclasses • derived element indicator: showing that an element is derived and need not be included in the implementation • navigation expression: indicating the navigation path in the class model these are counted at .5 each for a total of 7. all of the symbols can be decomposed, and the model itself can be a decomposition of a higher level model, so we add 4*.5 + .5 = 2.5 our basic method point score based on deliverables only is thus: 4 + 3 + 7 + 2.5 = 16.5 since uml currently does not specify a standard process for the creation of the deliverables, we cannot perform the task complexity augmentation. it will be instructive to perform this once variants of the uml are popularised and endowed with process elements. analysis of the entity relationship model based on the standard martin/mcclure (1988) notation without sub and super-types, we find that there is just one unique symbol type: the entity box. count 1 there is just one unique type of link, a relationship line. count .5 embellishments are present as follows: • labels on the relationships • cardinality indications on the ends of relationships as follows: • minimum number of associated instances (a 0 or i symbol) • maximum number of associated instances (a i or crow's foot symbol) • a numerated cardinality where numbers are used instead of the symbols at .5 each, we count 2. an entity symbol can be decomposed into another diagram, which could be referenced to the parent, so we count 1. our method point score for the er model is thus: l+.5+2+l=4.5 analysis of the use case diagram use cases (rational, 1997) show the relationship among actors and use cases within a system. they are frequently used in early analysis to identify user-system interactions and the transactions and functions which must be present in the system under consideration. unique symbols include: • actors: represent persons or organisational units in the environment who will interact with the system • use case: an ellipse containing the name of the use case we count 2. 55 ajis vol. 6 no. 1 september 1998 unique link types include: • communication: links an actor to a use case • extension: indicating that one use case may include behaviour of another • uses: indicates that one use case makes use of behaviour in another • refinement: indicating the expansion of an element to a greater level of detail count these at .5 each = 2. embellishments: • a boundary is drawn to indicate the bounds of the system. it contains the use cases, and does not include the actors. counts .5. hyperlinks/references: • a use case can be decomposed into another model (more detailed use case, state machine, collaboration) we count this at 1 (.5 for reference to a detailed model, and .5 for reference to a parent model) our total for the use case diagram is thus: 2 + 2 + .s + 1 = 5 . 5 analysis of the i.e. context diagram a context diagram contains the following unique symbols: • the actors in the environment which provide inputs to the system, or make use of outputs • a process box in the centre, representing the system itself at 1 each, count 2. connectors are limited to the arrows indicating communication of actors with the system. count .5 embellishments are confined to labels on the arrows and the boundary indication. count .5 each, total 1. the context diagram can be (and usually is) decomposed into further models. it does not have a parent diagram, since it is the highest level model. count .5 total for the context diagram is thus: 2+.5+l+.5 = 4 findings the uml ssd rates as considerably more complex than the ie erd. this bears out the experience of practitioners interviewed with respect to the difficulty of teaching, learning and applying each technique. there is much less difference between the uml use case and the ie context diagram, although the uml technique is rated slightly more complex. again, this is borne out by interviews with practitioners who found relatively little difficulty in moving to the use case approach. a large number of surveys and case studies have found significant resistance to the adoption of object oriented techniques (graham, 1994). these techniques are often richer and more comprehensive than the preceding ie techniques, however, ways will have to be found to teach and apply them in such a way that their increased complexity does not become a barrier to their adoption and successful use. one approach is the use of incremental model completion, where only certain aspects of the full model are completed during a phase. in this way, the comprehensiveness and cohesiveness of the full model are not compromised, while reducing the difficulty encountered by project staff in dealing with too many aspects concurrently (inspired, 1997). this approach is catered for in the notion of states in the method model. 56 ajis vol. 6 no. 1 september 1998 further work a complexity metric alone will be of little use. it is like knowing the fuel consumption of a vehicle without knowing its load carrying capacity or speed of travel. we thus propose that a comprehensiveness metric for methods is also required. this would measure the spread of capabilities of the method. the efficiency of competing methods could then be compared by determining the ratio between comprehensiveness and complexity. a comprehensiveness metric should embody: • lifecycle coverage (from and to where in the overall process does the method provide guidance) • dimensions (data, process, timing, quality, geographic distribution, technology, human interface) addressed by the method • integration (degree to which method is seamlessly integrated) we intend in future work to suggest such a metric. the practicality of a method in an industrial setting is also influenced by availability of the calibre and quantum of resources required for its execution. it may be useful to amend the calculated method point based upon the resource demands of the method. this would allow favouring of methods with lower requirements in terms of practitioner skill or number of discrete resource types needed. conclusion we have proposed a metric for method complexity and applied it to commonly used deliverables from two major schools of systems development. it appears to provide a useful measure of relative complexity which is consistent with the experience of practitioners interviewed. further experimental work will be necessary to accurately determine the weight of the factors used for each type of device in the method representation. we would encourage other researchers to undertake studies to facilitate this. the basic framework provides for incorporation of deliverable and process complexity. we have suggested that a resource intensity adjustment might also be introduced. the method point approach can provide methods engineers with a useful evaluation tool. we encourage the development of a comprehensiveness measure which will yield a score in the same range as the complexity measure for the current "benchmark" method set, viz ie. a method efficiency score of 1, (calculated by dividing the comprehensiveness measure by the complexity measure) would equate to the current ie practice. this factor could then be calculated for new approaches. scores higher than 1 would indicate progress towards more efficient methods, while those less than 1 would indicate a regress. the author is keen to collaborate with other researchers in the evolution of these techniques. references albrecht, a.j., 1979. measuring application development productivity, proceedings ibm share/guide symposium, guide international corp., chicago. bosnian r, mcleod g, tanfield j, 1992. the influence of project-method fit on the success of system development methodology usage. uct empirical project report, dept of information systems, university of cape town, 1992. brinkkemper s, lyytinen k & welk j (editors), 1996, method engineering: principles of method construction and tool support, chapman & hall, 1996. graham i, 1994. migrating to object technology, addison-wesley, wokingham, u.k. 1994. ifpug, 1994. function point counting practices manual, release 4.0, international function point users group. inspired, 1997. advanced systems engineering with objects, inspired, p o box 384, howard place, 7450, south africa, 1996a7. martin j, mcclure c, 1988. structured techniques, the basis for case, prentice hall, englewood cliffs, nj, 1988. mcleod g, 1992. a model for representation, integration and management of methods. south african computer journal. mcleod g, 1993. a generic method model for the representation, comparison, integration and management of methods, proceedings: 1st international conference on information systems, henley, u.k. mcleod g, 1995. metatool: a tool for the engineering and management of methods, proceedings: 1st international conference on meta-case. sunderland, u.k., jan 1995. 57 ajis vol. 6 no. 1 september 1998 murphy s, harvey s, mattison k, 1990. a status review of development methodologies in south africa. empirical project, uct information systems, nov 1990. software composition technologies, inc., 1997. frequently asked questions (and answers) regarding function point analysis. updated 28 feb 1997. http://ourworld.compuserve.com/homepages/softcomp/fpfaq.htm rational software corporation, 1997a. unified modeling language summary, vsn 1.0,13 january 1997. rational software corporation, 1997b. unified modeling language semantics, vsn 1.0,13 january 1997. rational software corporation, 1997c. unified modeling language notation guide, vsn 1.0,13 january 1997. rational software corporation, 1997d. unified modeling language glossary, vsn 1.0,13 january 1997. 58 paper title (paper title style) australasian journal of information systems lim 2018, vol 22, research note dialectic antidotes to critics of the technology acceptance model 1 dialectic antidotes to critics of the technology acceptance model: conceptual, methodological, and replication treatments for behavioural modelling in technology-mediated environments weng marc lim swinburne university of technology lim@wengmarc.com / marclim@swin.edu.au / wlim@swinburne.edu.my abstract the technology acceptance model (tam) is a prominent and parsimonious conceptual lens that is often applied for behavioural modelling in technology-mediated environments. however, tam has received a great deal of criticism in recent years. this article aims to address some of the most pertinent issues confronting tam through a rejoinder that offers dialectic antidotes—in the form of conceptual, methodological, and replication treatments— to support the continued use of tam to understand the peculiarities of user interactions with technology in technology-mediated environments. in doing so, this article offers a useful response to a common but often inadequately answered question about how tam can continue to be relevant for behavioural modelling in contemporary technology-mediated environments. keywords: technology acceptance model (tam); technology-mediated environment. 1 introduction the technology acceptance model (tam), which was proposed by davis (1986), refined by davis et al. (1989) and davis (1993), and finalized by venkatesh and davis (1996), is a popular theoretical lens from the information systems and technology discipline that is commonly applied to understand how users come to accept and use a particular technology (chuttur, 2009; see table 1). tam is generally intuitive and easy to use, and it has been cited more than 79,000 times on google scholar. however, many scholars who use tam in their investigations tend to (1) ignore the criticisms the model has received and/or (2) offer inadequate support for its continued use. to this end, this article aims to offer a succinct but useful response to the extant criticisms of tam and to support, through dialectic antidotes, the continued use of tam for behavioural modelling—that is, to understand user interactions with and, therefore, acceptance and usage of technology—in contemporary technology-mediated environments. 2 what is tam? tam, which is one of the most influential extensions of the theory of reasoned action (ajzen and fishbein, 1980; fishbein and ajzen, 1975), suggests that two factors influence the decision of users about a particular technology, namely perceived usefulness, or the degree to which a user believes that using a particular technology will enhance his or her performance, and perceived ease of use, or the degree to which a user believes that using a particular technology will be free from effort. a user’s perception of usefulness and ease of use of a particular australasian journal of information systems lim 2018, vol 22, research note dialectic antidotes to critics of the technology acceptance model 2 technology can (1) be influenced by external factors and (2) influence his or her intention toward that technology, which in turn influences his or her actual use of that technology (venkatesh and davis, 1996). more important, tam has undergone several revisions over the years (see table 1), and thus scholars interested in modelling user behaviour in technologymediated environments should be cognizant of the model’s development and base their investigations on the finalized (or latest) model (i.e. venkatesh and davis, 1996). version details citation proposed tam in 1986 model note: x = design feature; i = 1, 2 … n; n = total. number of citations note: google scholar search result on december 25, 2017. davis (1986, p. 24) cognitive response affective response design feature perceived usefulness perceived ease of use attitude toward using xi out of n actual system use behavioural response australasian journal of information systems lim 2018, vol 22, research note dialectic antidotes to critics of the technology acceptance model 3 version details citation revised tam + revised scale in 1989 model number of citations + note: google scholar search result on december 25, 2017. davis et al. (1989, p. 985) + davis (1989) actual system use perceived usefulness perceived ease of use attitude toward using external variables behavioural intention to use australasian journal of information systems lim 2018, vol 22, research note dialectic antidotes to critics of the technology acceptance model 4 version details citation revised tam in 1993 model note: linked hypothesized insignificant but found significant. number of citations note: google scholar search result on december 25, 2017. davis (1993, p. 481) finalized tam in 1996 model number of citations note: google scholar search result on december 25, 2017. venkatesh and davis (1996, p. 453) perceived usefulness perceived ease of use attitude toward using system actual system use perceived usefulness perceived ease of use behavioural intention external variables actual system use australasian journal of information systems lim 2018, vol 22, research note dialectic antidotes to critics of the technology acceptance model 5 version details citation tam2: extension of tam in 2000 model note: tam2 is an example of how the finalized tam can be conceptually contextualized (i.e. external variables in organizational settings) and employed to understand the peculiarities of user interactions with technology in specific technology-mediated environments, specifically employees’ mandatory and voluntary usage of proprietary systems for accounting and financial services and manufacturing activities during the systems’ preand post-implementation stages in the organization (venkatesh and davis, 2000). the same approach can be used to understand user interactions with technology (e.g. digital natives’ and digital immigrants’ interactions with artificial intelligence, blockchain, big data, cloud computing, and internet of things at the individual and organizational level) in contemporary technology-mediated environments (e.g. augmented reality, blended spaces, and sharing economy). number of citations note: google scholar search result on december 25, 2017. venkatesh and davis (2000, p. 188) table 1 evolution of tam 3 why was tam criticized? despite its frequent application, tam has been widely criticized for its supposed triviality (e.g. putative predictors of user behaviour, uncritically accepted assumption of user intention– perceived usefulness perceived ease of use behavioural intention subjective norm actual system use result demonstrability image job relevance output quality experience voluntariness finalized tam australasian journal of information systems lim 2018, vol 22, research note dialectic antidotes to critics of the technology acceptance model 6 behaviour linkage), limited explanatory and predictive power (e.g. neglect of self-regulatory and social aspects of user behaviour), and lack of novelty and practical value (e.g. limited discovery of predictors not subsumable under existing predictors of user behaviour, limited reconciliation or transformation of reasons for users’ technology resistance), among others (see bagozzi, 2007; benbasat and barki, 2007; chuttur, 2009). moreover, user adoption of technology in emerging domains with an array of new technological and technology-mediated products stimulated by the evolving socioeconomic environment (e.g. growing emerging markets, increasing number of consumers who have better economic wellbeing, higher levels of education, improved internet access, and greater desire for instancy) and technological proliferation (e.g. augmented and virtual environments, delegation to autonomous technology, enhanced consumer connection to the internet of things, facilitated information processing, and ubiquitous computing) has raised concerns about the relevance of tam to help understand the peculiarities of user interactions with technology in contemporary technology-mediated environments (lowe et al., 2016). 4 revisiting tam given the extant criticisms of tam, this article contends that it is necessary to revisit tam— in terms of its conceptual assumption, methodological application, and replication strategies— to find and offer greater support to demonstrate the relevance of tam to understand user acceptance and use of a particular technology, and thus its continued use for future investigations interested in behavioural modelling in contemporary technology-mediated environments. more specifically, the model’s only explicit assumption, to date, suggests that the intention concerning a particular technology is predicated on the perceived usefulness and ease of use of that technology (king and he, 2006; legris et al., 2003; turner et al., 2010), whereas the method for applying and testing the model, to date, has largely been regression-based methods, such as multiple linear regression analysis and covariance and partial least squares structural equation modelling, (e.g. ha and stoel, 2009; lim, 2015a; lim and ting, 2012). in that sense, the model is likely to suffer from overwhelming replication, with the possibility that endeavours to offer incremental contributions to the field may become extremely difficult, if not nearly impossible—an issue that resonates with many in the information systems and technology discipline (lim, 2016). further conceptual assumptions to extend the capability of tam to accommodate contextual peculiarities and to be applied beyond descriptive and exploratory settings should be useful to demonstrate greater utility and practical value of tam as a conceptual lens for behavioural modelling in contemporary technology-mediated environments. new forms of replication that demonstrate evidence of representing significant and relevant conceptual, methodological, and/or managerial contributions should be worthwhile. the sections that follow elaborate further on these treatment propositions to tam. 5 conceptual treatment to tam in order for tam to continue to be relevant for behavioural modelling of contemporary user interactions with technology, behavioural and technological researchers need to consider tam as a basic model that offers the benefit and flexibility of integrating extended and contextualized motivational influences and user behaviours based on emerging realities in australasian journal of information systems lim 2018, vol 22, research note dialectic antidotes to critics of the technology acceptance model 7 contemporary technology-mediated environments. in particular, perceived usefulness and ease of use should be considered as the fundamental tenets of tam, which have been and will arguably continue to be relevant for understanding user interactions with technology in technology-mediated environments—these motivational influences account for the basic evaluations of usability of a particular technology. more important, tam will require integration of extended and contextualized motivational influences that explain emerging realities around users and their interaction with technology, such as perceptions and evaluations of value (e.g. connectedness, enjoyment, entertainment, gratification, flexibility, instancy, irritation, impact, newness, recognition, scarcity, self-enhancement), behavioural control (e.g. accessibility, affordability), personal factors (e.g. ability to adapt, resistance to change, self-efficacy), social factors (e.g. social pressure, sense of belonging), and security factors (e.g. privacy, risk, trust), among others. moreover, the components of tam can assume multidimensionality to account for the complexities in contemporary realities of user behaviour in technology-mediated environments (e.g. affective and cognitive perceptions of ease of use and usefulness; attitude toward the corporate and product brand of competing technologies; initial and continued intention to use and recommend, in terms of its frequency and type, a particular technology; and contextualized conceptualizations of motivational influences [e.g. the perception of what constitutes as ‘useful’ or ‘easy to use’ for products using near-field communication technology may differ between young and older consumers]). while these propositions may, to a certain extent, on the surface level, reflect some of the criticisms of tam, on the deeper level, the act of flipping those concerns around and into conceptual assumptions for using and making tam relevant in contemporary technology-mediated environments demonstrates a logical and feasible approach to shift the focus of critiquing tam for its “inadequacies” to preparing future research to apply and extend tam more appropriately and relevantly for advancement in behavioural modelling in contemporary technology-mediated environments. thus, to put the articulation herein into perspective, tam should be considered as: a conceptual lens that provides the core tenets to user interactions—in the form of perceptions of ease of use and usefulness—with technology and that necessitates novel, meaningful extensions to develop a full fledge model that holistically and rigorously accommodates and purposefully explains contextual peculiarities and behavioural complexities in technologymediated environments. it should be noted that though some scholars may have already been engaging in this practice, this article contends that such conceptual treatments to tam are required in writing to provide greater clarity and support for its continued use for behavioural modelling in contemporary technology-mediated environments—which remains unavailable to date. 6 methodological treatment to tam while the predominant method of applying and testing tam is through regression-based methods in descriptive and exploratory investigations, this article contends that behavioural and technological researchers can consider engaging in causal investigations by applying and testing tam via experimentation over the shortand long-run. more specifically, the method of experimentation allows behavioural and technological researchers to test chronic dispositions and primed responses to manipulated technology-mediated scenarios (lim, 2015b). that is to say, through experimentation, behavioural and technological researchers australasian journal of information systems lim 2018, vol 22, research note dialectic antidotes to critics of the technology acceptance model 8 will be able to test for causal effects in the interactions between controllable technologymediated factors (e.g. product design and interface) and user behaviour (e.g. perception, intention, and usage) (e.g. teh et al., 2017). in doing so, they will be able to offer empirical and tested solutions for encouraging desired user behaviour, such as purchase and consumption of and satisfaction with a particular technology. indeed, testing for behavioural changes and effectiveness of user-centric behavioural strategies in the shortand long-run toward competing technologies by means of experimentation (as well as traditional regression-based methods) should help behavioural and technological researchers to obtain an up-to-date understanding about their target users as well as the effectiveness of their user-centric behavioural strategies in encouraging greater desired user behaviour (e.g. encouraging adoption, mitigating resistance) toward an offered technology among its target markets (including to identify and target behavioural change attempts). 7 replication treatment to tam for behavioural modelling in technology-mediated environments, ground-breaking contributions are often evaluated more favourably than incremental contributions delivered by replications (coulthard and keller, 2016; lim, 2016). direct replication of previously published empirical studies is challenging because of likely differences in study location, personnel, and time, as well as the threat of evolving socioeconomic changes and rapid technological proliferation, and thus the general understanding of replication herein encapsulates the action or process of repeating the test of similar propositions by using similar and dissimilar methodological procedures for behavioural modelling (e.g. epistemology, ontology, data collection, data analysis, measurement, instrument, sampling). replication using similar combinations of propositions and methodological procedures (e.g. same geographic location with the same sample characteristics and information systems and technology within a relatively short period) with no strong grounds (e.g. high impact) is discouraged because of insufficient novelty (i.e. what is new and so what) to warrant spending of scarce resources and unlikely publication at respectable outlets. corroborations using different combinations of propositions and methodological procedures (e.g. alternative methods of analysing data, different measurement and target population, selective inclusion or omission of constructs), especially if project findings had (i.e. many people relied on the result—e.g. highly cited, such as journal articles receiving more than 100 citations within five years) or will have (e.g. high risk project classification by human research ethics committee) high impact, are encouraged for their conceptual development and extension, generalizability, and rigor. 8 conclusions in short, tam should be viewed as a model that increases opportunities to understand the peculiarities of user interactions with technology in contemporary technology-mediated environments, not limiting them. despite the limitations of a thought piece (e.g. absence of empirical data and analysis), it is hoped that the articulation herein will help to clarify that tam can be appropriately and relevantly applied in theoretical and practical behavioural modelling endeavours in contemporary technology-mediated environments characterized with evolving socioeconomic changes and continued technological proliferation. more important, the discussion herein should stimulate further research that aims to examine and understand user relationships with new technologies to use and extend tam in ways that australasian journal of information systems lim 2018, vol 22, research note dialectic antidotes to critics of the technology acceptance model 9 advances theory and practice. among the potentially fruitful areas for further exploration include but not limited to user adoption, use, and disposal of technological and technologymediated products in new, emerging, and matured markets; user interactions with smart technologies and the internet of things; and user involvement in augmented and virtual environments, among others. references ajzen, i., & fishbein, m. (1980). understanding attitudes and predicting social behavior. englewood cliffs, nj: prentice-hall. bagozzi, r. p. (2007). the legacy of the technology acceptance model and a proposal for a paradigm shift. journal of the association for information systems, 8(4), 244-254. available at http://aisel.aisnet.org/jais/vol8/iss4/12 benbasat, i., & barki, h. (2007). quo vadis tam? journal of the association for information systems, 8(4), 211-218. available at http://aisel.aisnet.org/jais/vol8/iss4/12 coulthard, d., & keller, s. (2016) . publication anxiety, quality, and journal rankings: researcher views. australasian journal of information systems , 20. doi: https://doi.org/10.3127/ajis.v20i0.1262 chuttur, m. y. (2009). overview of the technology acceptance model: origins, developments and future directions. sprouts, 9(37). available at http://sprouts.aisnet.org/9-37 davis, f. d. (1986). a technology acceptance model for empirically testing new end-user information systems: theory and results. cambridge, ma: mit sloan school of management. davis, f. d. (1989). perceived usefulness, perceived ease of use, and user acceptance of information technology. mis quarterly , 13(3), 319-340. doi: https://doi.org/10.2307/249008 davis, f. d. (1993). user acceptance of information technology: system characteristics, user perceptions and behavioral impacts. international journal of man-machine studies, 38(3), 475-487. doi: https://doi.org/10.1006/imms.1993.1022 davis, f. d., bagozzi, r. p., & warshaw, p. r. (1989). user acceptance of computer technology: a comparison of two theoretical models. management science, 35(8), 9821003. doi: https://doi.org/10.1287/mnsc.35.8.982 fishbein, m., & ajzen, i. (1975). belief, attitude, intention, and behavior: an introduction to theory and research. reading, ma: addison-wesley. ha, s., & stoel, l. (2009). consumer e-shopping acceptance: antecedents in a technology acceptance model. journal of business research , 62(5), 565-571. doi: https://doi.org/10.1016/j.jbusres.2008.06.016 king, w. r., & he, j. (2006). a meta-analysis of the technology acceptance model. information and management, 43(6), 740-755. doi: https://doi.org/10.1016/j.im.2006.05.003 legris, p., ingham, j., & collerette, p. (2003). why do people use information technology? a critical review of the technology acceptance model. information and management, 40(3), 191-204. doi: https://doi.org/10.1016/s0378-7206(01)00143-4 australasian journal of information systems lim 2018, vol 22, research note dialectic antidotes to critics of the technology acceptance model 10 lim, w. m. (2016). a post publication review of “publication anxiety, quality, and journal rankings: researcher views”. australasian journal of information systems, 20. doi: https://doi.org/10.3127/ajis.v20i0.1487 lim, w. m. (2015a). antecedents and consequences of e-shopping: an integrated model. internet research, 25(2), 184-217. doi: https://doi.org/10.1108/intr-11-2013-0247 lim, w. m. (2015b). enriching information science research through chronic disposition and situational priming: a short note for future research. journal of information science, 41(3), 399-402. doi: https://doi.org/10.1177/0165551515577913 lim, w. m., & ting, d. h. (2012). e-shopping: an analysis of the technology acceptance model. modern applied science, 6(4), 49-62. doi: https://doi.org/10.5539/mas.v6n4p49 lowe, b. , dwivedi, y. , & d’alessandro, s. (2016) . consumers and technology in a changing world. european journal o f marketing . available at https://www.emeraldgrouppublishing.com/products/journals/call_for_papers.htm?id= 6762 teh, p.-l., lim, w. m., ahmed, p. k., chan, a. h. s., loo, j. m. y., cheong, s.-n., & yap, w.-j. (2017). does power posing affect gerontechnology adoption among older adults? behaviour and information technology, 36(1), 33-42. doi: https://doi.org/10.1080/0144929x.2016.1175508 turner, m., kitchenham, b., brereton, p., charters, s., & budgen, d. (2010). does the technology acceptance model predict actual use? a systematic literature review. information and software technology, 52(5), 463-479. doi: https://doi.org/10.1016/j.infsof.2009.11.005 venkatesh, v., & davis, f. d. (1996). a model of the antecedents of perceived ease of use: development and test. decision sciences, 27(3), 451-481. doi: https://doi.org/10.1111/j.1540-5915.1996.tb01822.x venkatesh, v., & davis, f. d. (2000). a theoretical extension of the technology acceptance model: four longitudinal field studies. management science, 46(2), 186-204. doi: https://doi.org/10.1287/mnsc.46.2.186.11926 copyright: © 2018 lim. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems lim 2018, vol 22, research note dialectic antidotes to critics of the technology acceptance model 11 1 introduction 2 what is tam? 3 why was tam criticized? 4 revisiting tam 5 conceptual treatment to tam 6 methodological treatment to tam 7 replication treatment to tam 8 conclusions microsoft word ajis vol 19.docx australasian journal of information systems gill & alam 2015, vol. 19, pp. 23-40 social architecture 23 social architecture: an emergency management case study asif qumer gill school of software university of technology sydney, australia asif.gill@uts.edu.au sultana lubna alam & jessica eustace discipline of is and accounting university of canberra canberra, australia {lubna.alam, jessica.r.eustace}@canberra.edu abstract emergency management agencies are progressively using social media for the sourcing and distribution of disaster information. emergency management agencies are often unsure as to how to best identify and assess social media concerns (e.g. information security, trust) which must be addressed to develop a social media-enabled disaster information management environment. this paper adopts the social architecture viewpoint assessment (sava) framework for identifying and assessing social media concerns from four different viewpoints: it, value, resource and management. this paper demonstrates the use of the sava framework in the context of an in-depth empirical case study of an australian emergency management agency. the results of this study indicate that the sava framework is useful for emergency information management managers in identifying and assessing social media concerns. keywords: crowd sourcing, disaster management, enterprise architecture, social architecture, social media 1 introduction disaster information management is a complex domain due to its diversity and dynamic characteristics (othman and beydoun 2013). traditional disaster information management systems come under pressure when an unpredictable emergency situation causes these existing systems to breakdown (semaan and mark 2011). social media, originated in the context of socialization (parameswaran et al. 2007; gill et al. 2011), has recently become very important in times of crisis (grabowski and roberts 2011; the advertiser 2011; eustace and alam 2012). social media provides new ways of supporting communities of people to source and share information during a crisis. it refers to systems that support the “activities, practices, and behaviours among communities of people who gather online to share information, knowledge, and opinions using conversational media” (safko and brake 2010). data directly contributed by citizens and data gathered from disaster bystanders have a strongly positive potential to give responders more accurate and timely information than is possible with traditional information gathering methods (tapia and moore 2014). recent examples of volunteer-gathered disaster information in emergency situations (e.g. the haiti earthquake, san diego fires, hurricane sandy, the queensland floods) have triggered policy makers to reconsider the role of social media tools in disaster response (majchrzak and more 2011). it has been observed that, throughout history, humans have demonstrated that, when faced with a disaster, they adapt whatever systems and skills offer help during a crisis (sutton et al. 2011). this idea has been largely proven correct, as, most recently, emergency management agencies have begun to realize that they need to be connected to each other and with the community through social media systems (foster 2013). australasian journal of information systems gill & alam 2015, vol. 19, pp. 23-40 social architecture 24 for decades, emergency management agencies have operated with a centralized command structure, standard operating procedures, and internal vetting standards to ascertain appropriate responses to disasters (tapia and moore 2011). a central aspect of these organizational mechanisms is complete control over the internal flow of information concerning the crisis from source to organizational decision-maker, respectively, to ensure accuracy, security, legitimacy, and, eventually, trust between the organization and information source (tapia and moore 2011). however, social media systems are cloud-based web 2.0 platforms that reside outside the emergency agency. moreover, there are numerous challenges to the adoption of social media data, including issues of reliability, the quantification of performance, deception, focus of attention, non-verifiable sources and information, and the translation of reported observations/inferences to respond to crises (tapia and moore 2011; starbird and palen 2011). however, findings from recent comparative studies (cf. tapia and moore 2011) argue that social media data in crisis response already claims pockets of use and acceptance among organizations. they found that social media data is useful to responders in situations where information is limited, such as at the beginning of an emergency response effort, and when the risks of ignoring an accurate response outweigh the risks of acting on an incorrect one. these studies demonstrate that the barriers used by responding organizations have gone beyond discussions of trustworthiness and data quality to that of more operational issues. there are a number of social media systems, such as sms, mashups, wikis, twitter, and facebook that can be adopted by the emergency management agencies for information management. however, due to the varied nature of different social media systems and a lack of available scientific literature and practical tool support (kerkhop 2012), emergency management agencies are often unsure how to best assess and use social media (majchrzak and more 2011). if emergency management agencies are already using social media, they need to understand and be able to evaluate how well they are using social media, and, further, how they should be using social media to effectively establish a social media-enabled disaster information management environment. the effective establishment of the social mediaenabled disaster information management environment involves the continuous identification and assessment of different social media concerns or challenges (terman 2011; kerkhof 2012). social media concerns must be addressed by the current or to-be developed social mediaenabled disaster information management environment. against this backdrop, the key research question, in the context of australian emergency management agencies, which is addressed in this paper, is as follows: how to best identify and assess different social media concerns to effectively establish a social media-enabled disaster information management environment for the timely sourcing and distribution of disaster information? in order to address the above research question, this paper adopts a social architecture viewpoint assessment (sava) framework. the sava (gill and bunker 2012) framework provides an assessment index and a conceptual model. the sava framework can be used to assist emergency management agencies to identify and assess social media concerns from four different viewpoints: it, value, resource and management. the framework is generic in nature and thus can be applied to all forms of social media platforms. this paper presents the use of the sava framework in the context of an australian emergency management agency case study. this paper is organized as follows. firstly, it discusses the theoretical background and research context. secondly, it provides an overview of the sava framework. thirdly, it discusses the use of the sava framework in an australian emergency management agency empirical case study. finally, the empirical case study results are discussed along with the question of how the findings of this research can be further used in our current research. australasian journal of information systems gill & alam 2015, vol. 19, pp. 23-40 social architecture 25 2 theoretical background and research context australia is a disaster-prone country and is frequently challenged by different types of emergency situations, such as floods and bush fires (e.g. royal commission report 2011). australian emergency management agencies are showing significant interest in the adoption of emerging social media (nsw ses report 2011). social media can be used by emergency management agencies for the sourcing of disaster information for planning and coordinating activities in response to disaster scenarios (quillinan et al. 2009; latonero and shkovski 2010). for example, twitter, as one of the social media tools, was used for disseminating tsunami and flood warnings (chatfield and brajawidagda 2012). it has been reported that social media is mostly used during disaster response for information sharing (horita et al. 2013). gill et al. (2014) highlight that social media has been mostly used for fire and floods. however, social media is not restricted to any specific type of crisis and can be used in different crisis scenarios. a number of case studies on the use of social media in crisis management have been reported, such as the 2011 canberra mitchell factory fire (eustace and alam 2012), the 2010-2011 queensland floods (royal commission report 2011), the 2009 oklahoma grassfires (hughes 2009), and the 2007 virginia tech shootings (vieweg et al. 2008). social media is a non-linear complex and distributed echo-system that involves a large number of independent parties who interact through different types of tools and channels. unlike traditional disaster communication and warning systems (e.g. radio, tv), it provides a real-time distributed collaborative platform to government and the community for problem solving through content co-creation, sharing and evaluation (chun and reyes 2012). social media can be used along with existing traditional disaster management systems to help the emergency management agencies interact with citizens and non-government organizations in the context of sourcing and disseminating disaster information. however, emergency management agencies operate in a complex and highly regulated enterprise architecture environment. enterprise architecture is the “organizing logic for business processes and it infrastructure reflecting the integration and standardization requirements of the firm’s operating model” (ross et al. 2006). enterprise architecture describes the building blocks of the formal business, information system and technology architecture domains of an enterprise (harrison 2011). the current influx of emerging social media systems (cross et al. 2005; starbird and stamberger 2010; starbird and palen 2011) provides both opportunities and challenges to the emergency management agencies who are used to linear and traditional enterprise architecture (anderson, 2012). emergency management is a multi-dimensional social system (foster 2013). the emergency management agencies require the establishment of contemporary social media-enabled social architecture as part of their traditional enterprise architecture that needs to be supported through the continuous strategic adoption of social media. the social architecture domain is important for emergency management agencies in effectively understanding the communities of people who want to share information and opinions during a crisis. social architecture (based on the iso/iec 42010 architecture definition) is defined as the “fundamental concepts or properties of a social system in its environment embodied in its elements, relationships, and in the principles of its design and evolution” (gill 2013). social architecture refers to the social structure, behaviour and culture of communities of people who share information and opinions that influence the desired behaviours within and outside the operating environment of an enterprise. the social architecture of a social system can be designed by using an enterprise architecture framework. there are a number of enterprise architecture frameworks, such as the zachman framework (1987), federal enterprise architecture framework (cio council 2001), and the open group architecture framework (togaf 9.1) (harrison 2011). togaf 9.1 is one of the most well-known and comprehensive architecture frameworks that claim to provide a complete architecture development method and guidelines for developing and managing the traditional business, system, and technologydomain architectures. although togaf 9.1 is perhaps the most comprehensive enterprise architecture framework, it does not provide any support for developing and managing the emerging social architecture (see figure 1). togaf 9.1, along with the other mentioned australasian journal of information systems gill & alam 2015, vol. 19, pp. 23-40 social architecture 26 mainstream enterprise architecture frameworks, lacks support for identifying and assessing social media concerns, which is a fundamental step in the development of a social architecture. most recently, a meta-framework for adaptive enterprise service systems (gill 2013) has been developed, which provides support for identifying and assessing social media concerns (assessment capability) to develop and improve the social architecture (operating capability). this meta-framework builds on multi-disciplinary action-design research in well-known enterprise requirements, strategy, architecture, service, and project management disciplines. this meta-framework uses the system of systems (maier 1998), agility (qumer and hendersonsellers 2008), and service science (spohrer and kwan 2009) theories to describe the modern enterprise as an “adaptive enterprise service system” (gill 2013). this meta-framework includes a novel sava framework that provides an assessment index and a conceptual model to assist emergency management agencies in identifying and assessing social media-related concerns. the sava (gill and bunker 2012) construct is developed based on the recent literature and two disaster cases (2010 haiti earthquake and 2010-11 queensland floods). the sava framework has both a theoretical and practical foundation and is not tied to any specific social media. the sava index and a conceptual model can be applied to different social media in different organisational contexts. this paper adopts the sava framework and demonstrates its use in the context of an australian emergency management agency case study. the next section discusses the sava framework. 3 the sava framework emergency management agencies operate in a complex and highly-regulated environment. they need a framework for assessing social media concerns before making any long term commitments and investments in the strategic adoption of social media for their local context. in order to provide a practical approach for assessing social media concerns, this section discusses the sava framework. the sava framework has two main components: the sava index (table 1), and the sava conceptual model (figure 1). 3.1 the sava index the sava index provides a list of 24 basic concern categories and assessment questions that can be used by emergency management agency information management managers to identify and assess social media concerns (table 1). these basic concern categories can be divided into 15 sub-categories which can be further organised into 4 key themes or viewpoints on social media (see table 1). these viewpoints are: social media value, social media resources, social media it (information technology) and social media management viewpoints. the viewpoint theory (harrison 2011) offers 5 key concepts: system, stakeholders, concerns, views and viewpoints. a system is a social media software system, platform or collection of components or tools (e.g. twitter, facebook) stakeholders are emergency management agency managers (e.g. digital media managers); concerns are the key social media challenges or interests (e.g. right information, right people, and information privacy) that are important to stakeholders a view represents the part of the system from the perspective of a related set of stakeholder’s concerns; and a viewpoint is similar to a template that can be used to create a number of views for different stakeholders. australasian journal of information systems gill & alam 2015, vol. 19, pp. 23-40 social architecture 27 ref. viewpoints major categories basic concern categories 01 social media value 01. right information 1. false information and rumours 2. bad information patterns 02. right people 3. changing demographics 4. information source trustworthiness 03. right time 5. information latency 02 social media resources 04. financial 6. information overload 05. human resources 7. additional resources and supervision 8. performance implications 03 social media it 06. infrastructure 9. information unavailability 07. integration 10.information integration 08. interoperability 11. information interoperability 09. privacy 12. crowd-sourced information commingling 10. security 13. malware 14. group attack 15. distributed denial-of-service attack 11. scalability 16. content constraint 12. resilience 17. social media “self-victims” 04 social media manageme nt 13. policy 18. lack of social media monitoring policy 14. process 19.verification and validation 20. overwhelming streams of crowdsourced information 15. authority 21. authority confusion 22. ownership confusion 23. unauthorised information sharing 24. multiple or too many sources of information table 1: the sava index (adopted from gill and bunker 2012) australasian journal of information systems gill & alam 2015, vol. 19, pp. 23-40 social architecture 28 figure 1: the sava conceptual model (adopted from gill and bunker 2012) 3.2 the sava conceptual model the sava conceptual model describes the grouping and relationships of the four key social media viewpoints of the sava index (see figure 1). it can be observed from figure 1 that the value for emergency management agency managers or stakeholders is in getting the right information from the right people to the right people at the right time in order to make quick, effective decisions during a crisis. the “value” viewpoint of the sava conceptual model (see figure 1) groups and represents the right information, right people, and right time majorconcern categories. in order to obtain and deliver “value”, emergency management agencies need resources and it. the “resource” viewpoint groups the financial and human resource major concern categories. the “it” viewpoint groups the infrastructure, integration, interoperability, privacy, security, scalability and resilience major-concern categories. the “management” viewpoint provides the overarching management support to other viewpoints (e.g. it management, resource management, and value management). the “management” viewpoint groups the policy, process and authority major concern categories. the concern categories, major categories, and related viewpoints may also consider another two important aspects, namely (1) the likelihood and (2) the impact of the concern (heath 1997). likelihood is the possibility of social media concern, whereas impact is how strongly the concern would affect emergency management agencies. emergency management agencies can assess each concern listed under the major categories and give a score from 1 to 10 (e.g. from low to high) for the two dimensions of likelihood and impact. in this research, we adopted a qualitative descriptive case study approach and purposely did not use the quantitative score. this is because the purpose of this paper is to qualitatively generate information while answering the key research question: how to best identify and assess social media concerns? the next section discusses the use of the sava framework in an in-depth case study of an australian emergency management agency. 4 emergency management case study 4.1 the case study organisation this section discusses the use of the sava framework in the naturalistic setting of an australian emergency management agency case study organisation in victoria, australia (codenamed emax). a naturalistic setting allows the exploration of the performance or utility of a solution artefact or construction in its real environment. it is always empirical and may be interpretivist, positivist, and/or critical (pries-heje et al 2008). this study presents a single empirical and interpretive in-depth analysis of the emax case study using sava. yin (1994) argued that a case study approach is appropriate when the available or existing knowledge is not yet evident and when the focus is on real-life context. a case study approach can be used to analyse the information system in a case study organisation. the objective of this empirical case study is to demonstrate a practical approach to explain how to best identify and assess (analyse) social media (information system) concerns in the real context of an emergency australasian journal of information systems gill & alam 2015, vol. 19, pp. 23-40 social architecture 29 management organisation (case study organisation as a unit of analysis). the 2009 victorian bushfires royal commission established the need for improved information flow between emergency management agencies and the community, calling on agencies to place as much value on warnings as they do on fire suppression. in 2011, the victorian flood warnings and response review recommended actively pursuing the use of social media as part of emergency warning and public information systems. they also urged further trials to explore the use of social media as a credible source of information to and from the public during an emergency. emax is one of the largest volunteer-based australian emergency management agencies. emax has over 50,000 volunteers and over 1000 support personnel. emax has a collaborative culture that values volunteerism and strong collaboration and communication between the government and citizens. emax provides state-wide emergency coordination, community education, and response to major incidents in australia and internationally such as the 2010-2011 queensland floods and the 2011 christchurch earthquake. emax is selected for this paper as a unit of analysis because it is a well-known agency within australia which has made significant advances in its use of social media for emergency warnings and the provision of advice during major incidents. it won the ict excellence in e-government award (i.e. gov 2.0 category) from the department of finance and deregulation in recognition of its excellence in institutionalizing social media use for disaster management. emax has developed its social media capabilities and has made significant advances in its use of social media for warnings and advice during major incidents. these advances are part of a strategic initiative to help deliver the recommendations of the royal commission and accomplish emax’s core mission of protecting lives and property. the initiative has resulted in emax‘s social media systems being integrated into the “one source and one message” system that delivers emergency warnings to agency websites/emergency broadcasters. emax and the state control centre are now engaging directly with, and gathering intelligence from, the online community during major emergencies. this has played a significant role in meeting the fire services commissioner’s commitment to provide timely, relevant, and tailored advice to the community. it has also allowed a much quicker response to communication issues and has provided valuable intelligence to official emergency management structures. 4.2 empirical analysis the case study organisation emax was engaged to participate in this empirical study. the sava framework was used as an analytical lens to collect and analyse qualitative data on social media in emax with an in-depth interview with emax’s digital media manger. the sava construct was explained to emax’s digital media manger. additional information was also provided to the emax manager in order to clarify the sava framework. the digital media manager was used as a single source of data for this case study. this was sufficient because the digital media manager was across the social media use in emax and represented the emax social media team. also, the questions did not relate to the digital media manager’s personal beliefs, rather the questions were specific to social media and its use in emax. the assessment questions listed (derived from major and basic concern categories) in the sava index provided a lens for analysing emax’s capability to effectively adopt, operate, and improve social mediaenabled social architecture for real-time crowd-sourcing and the distribution of disaster information. for each major category of sava, additional social media concerns (if any) were also sought during the analysis. the researchers were also provided with the anonymous statistics of the users from the emax facebook page and other extant documentation on emax (e.g. social media policy, web content) to provide additional context and support for the answers to relevant questions (e.g. questions relevant to social media policy). the next section discusses the empirical case study analysis results. 4.3 analysis results the sava framework was used in identifying and assessing social media concerns in the real context of the case study organisation emax. a review of emax was carried out to identify the key social media concerns by using the four viewpoints and social media concerns listed in australasian journal of information systems gill & alam 2015, vol. 19, pp. 23-40 social architecture 30 sava: social media value, social media resources, social media it and social media management. this section discusses the research results. 4.3.1 social media value the social media value viewpoint groups and represents the right information, right people, and right time major concern categories. these categories were used to review emax’s social media-enabled social architecture from the value perspective. the value assessment results indicate that emax’s current social media-enabled social architecture addresses most of the social media concerns listed under the social media value viewpoint (see table 2). it is also evident that the sava construct provided the essential criteria for facilitating the identification and assessment of social media concerns in the real context of emax. the results of this analysis are summarised below. major categories basic concern categories assessment questions assessment results and evidence 01. right information 1. false information and rumours does it have the ability to determine whether social media-sourced information is correct or not? emax uses the social media monitor officers and hootsuite system to determine the correctness of the social media sourced information. 2. bad information patterns does it have the ability to handle maliciously-generated patterns of information? emax uses the social media monitor officers and hootsuite system to identify the maliciouslygenerated patterns of information. 02. right people 3. changing demographics does it have the ability to determine whether the young people are living in the disasteraffected area and whether they know to use social media in order to receive information from emergency management agencies? emax perceives that “changing demographics” is not a concern for them. 4. information source trustworthiness does it have the ability to determine whether the information source is trustworthy or not? emax uses the social media monitor officers and hootsuite system to determine trustworthiness of the information source. 03. right time 5. information latency does it have the ability to handle or minimise delays in the activity of raw information translation from different tweets or re-tweets? emax uses the social media monitor officers and hootsuite system. table 2: social media value viewpoint assessment results when assessing emax’s current social media-enabled architecture using the sava criteria, it was found that emax uses the social media monitoring officers and hootsuite system to australasian journal of information systems gill & alam 2015, vol. 19, pp. 23-40 social architecture 31 address the concerns of false information, rumours, bad information patterns, information source trustworthiness, and information latency. emax has two specific roles in the state emergency control centre, one being the social media officer who ensures official notifications are sent out on facebook and twitter pages as well as maintaining an automated system that links to websites that have emergency warnings and information. the second role, an intelligence-gatherer, is activated to help with operations and decision-making during a crisis. the hootsuite system is a social media management platform that manages profiles and data across multiple social networks and channels. it creates and schedule reports based on social media data to inform decision making and it also allows teams to collaborate. these roles and the hootsuite system enable emax to search for certain keywords on social media sites, as well as monitor direct communications and direct activity, to maintain a correct, time-effective flow of information in order to make quick, effective decisions during a crisis. this helps emax to find misinformation (e.g. false information, rumours, and bad information patterns) quickly. this is critical because incorrect information could potentially put a community in danger. emax stresses the importance of having these roles as well as clear policies, processes and tools, in order to deal with misinformation. the emax digital media manager mentioned that social media has even assisted them to quell rumours. during recent emergencies, rumours were passed around that were incorrect and could have potentially put people in danger. emax would not have known or been able to monitor this. as people post these possible rumours on social media sites, emax can quickly verify or invalidate information. the review of emax indicated that there is a slight lag time (e.g. information latency) between the piece of possible information being sourced and posted. this is because the sourced information must be first verified. however, in order to deal with the information-latency concern, emax has two additional roles, along with the hootsuite system. the review of emax suggested that despite formal roles, policies, processes, and tools, emax has a policy of using informal language as much as possible in the context of social media during both emergency and non-emergency times. emax believes that “changing demographics” is not an issue for them, and they make their social media site pages appropriate for all ages and interests. it was evident from the facebook page demographic data that people of all ages were accessing the facebook page, with 8% in the 13-17 age group, 24.2% in the 18-24 age group, 28.6% in the 25-34 age group, 21.3% in the 35-44 age group, 11.5% in the 45-54 age group, and 5% in the 55+ age group. it is assumed that the demographics were more spread due to the emax facebook page being a government site. 4.3.2 social media resources the social media resources viewpoint groups and represents both financial and human resources’ major concern categories. these categories were used to review emax from the resources viewpoint. the resource viewpoint assessment results indicate that emax clearly addresses the human resource concerns of the “additional resources and supervision concern category” (see table 3). however, emax is still trying to determine how to address the information overload and performance concerns. it is also evident that sava provided the essential criteria (from the social media resource viewpoint) for facilitating social media assessment in the real context of emax. the results of this analysis are summarised below. australasian journal of information systems gill & alam 2015, vol. 19, pp. 23-40 social architecture 32 major categories basic categories assessment questions assessment results and evidence 04. financial 6. information overload does it have the ability to effectively handle overloaded bigdata coming via social media streams and to get real business value out of it (e.g., opportunity cost)? emax is still trying to determine how to deal with this challenge. 05. human resources 7. additional resources and supervision does it have the ability to manage and support the need and cost of additional staff or resources for monitoring, updating, correcting, and preventing the posting of inaccurate information via social media 24/7? emax has two additional roles to specifically address the challenge of additional resources and supervision. 8. performance implications does it have the ability to manage staff struggling to keep up with an overload of information, the impact on their performance, and work practices? emax is still trying to determine how to deal with this challenge. table 3: social media resources viewpoint assessment results emax has two additional roles specifically addressing the concerns of additional resources and supervision for monitoring, updating, correcting, and preventing the posting of inaccurate information via social media. emax stated that they have been discussing the information overload and performance-implication concerns in recent months. emax stated that there has been research into determining how to manage the noise-to-signal ratio. they are looking into research that is currently being conducted in natural-language processing to filter out the ‘white noise’ from social media-sourced information. 4.3.3 social media it the social media it viewpoint groups and represents the infrastructure, integration, interoperability, privacy, security, scalability, and resilience major concern categories. these categories were used to review emax from the it perspective. the assessment results indicate (see table 4) that emax clearly addresses all it-related concerns. however, in the case of it infrastructure and security-related concerns, emax mainly relies on social media service providers and other sources to distribute information. it is also evident that sava provided the essential criteria for facilitating the review of emax’s social architecture concerns from the social media it perspective. the results of this analysis are summarised below. australasian journal of information systems gill & alam 2015, vol. 19, pp. 23-40 social architecture 33 major categories basic categories assessment questions assessment results and evidence 06. infrastructure 9. information unavailability does it have the ability to handle information unavailability due to power failure or loss of mobile phone or internet connection or permanent/ temporary service shut down during a crisis? emax has a contingency plan. 07. integration 10. information integration does it have the ability to handle integration between the unstructured bigdata real-time streams via social media and the legacy system relational database? social media is integrated with the emax website, and warning and notification system. 08. interoperability 11. information interoperability does it have the ability to handle information interoperability issues between cloud-based social media and legacy systems? information is interoperable between social media and the emax warning and notification system. 09. privacy 12. crowdsourced information commingling does it have the ability to handle the possibility of the commingling of crowdsourced information on an entirely different crisis if the information for both crises is stored in a single cloud-based multi-tenant environment? emax refers to its website for more information on each tweet they publish. this information is time stamped on the emax website, thereby minimising the impact of commingling, as much as possible. 10. security 13. malware does it have the ability to block the access of malicious software that may view and modify the social media-sourced information packets in transit? emax relies on the social media service provider and other sources. 14. group attack does it have the ability to handle a brute force attack that may manipulate the encrypted user’s social media account information and their broadcasted messages or tweets? emax relies on the social media service provider and other sources. australasian journal of information systems gill & alam 2015, vol. 19, pp. 23-40 social architecture 34 major categories basic categories assessment questions assessment results and evidence 15. distributed denial-ofservice attack does it have the ability to handle an attack targeting specific social media websites by flooding a web server with repeated tweets or messages? emax relies on the social media service provider and other sources. 11. scalability 16. content constraint does it have the ability to effectively operate within the maximum 140character tweet content size? emax refers to its website for more information on each short tweet they publish. 12. resilience 17. social media “self-victims” does it have the ability to effectively operate if the social media service suffers due to a disaster (e.g., emergency management agencies dependent on social media may not be able to access the information posted on the social media site)? emax has a contingency plan to deal with social media service interruptions and issues. table 4: social media it viewpoint assessment results emax has well-established warning and notification systems, and the social media system is only a part of their overall enterprise architecture. all warnings and notifications are published on its website. emax makes use of the emergency alert system and official emergency broadcast stations like the australian broadcasting corporation (abc). the integrated emax website and the warning and notification system of their information-management architecture seem to address the concerns of information integration and interoperability. emax tries to distribute as much information as possible in one tweet; however, this sometimes might not be sufficient. therefore, in order to address this scalability concern, emax has found a way to link back the small tweet description to the emax website on twitter. emax refers to its website for more information on each tweet they publish. this information is also time-stamped on the emax website to minimise commingling as much as possible. with concerns such as malware, group attacks, distributed denial of service attacks, and the unavailability of social media sites such as twitter, emax mainly relies on other sources to distribute information. emax also has contingency plans in case they themselves end up in an emergency situation which may require them to relocate to an alternative location. emax emphasised that social media is a mobile medium and can be accessed from anywhere on any internet-enabled device, including a mobile phone. the social media officers are not tied to any computer system and can do their job from anywhere if needed; they are not necessarily reliant on only agency-based networks to do their work. emax, overall, seems well-placed in dealing with the it-related concerns outlined in the sava construct. 4.3.4 social media management the social media management viewpoint groups and represents the policy, process and authority major concern categories. these categories were used to review emax from the management perspective. the assessment results indicate (see table 5) that emax addresses most of the concerns listed under the management viewpoint. it is also evident that sava australasian journal of information systems gill & alam 2015, vol. 19, pp. 23-40 social architecture 35 provided the essential criteria for facilitating the review of emax’s architecture concerns from the social media management viewpoint. the results of this analysis are summarised below. major categories basic categories assessment questions assessment results and evidence 13. policy 18. lack of social media monitoring policy does it have a clear policy to use and monitor social media in a specific situation? emax has a social media policy. 14. process 19. verification and validation does it have the ability to verify and validate the information sourced via social media streams? emax has specified roles, processes and tools to verify and validate social mediasourced information. 20. overwhelming streams of crowdsourced information does it have clear processes that guide how, when, and how often the streaming of social mediasourced data between the social media and legacy systems should occur? emax does not seem to have a clear process to address this concern. 15. authority 21. authority confusion does it have the ability to handle a situation where social media providers change their terms and polices at their own will? emax has to rely on the third-party social media service provider at this stage. 22. ownership confusion does it has a clear understanding of who owns the information residing in the cloud-based social networks, whether that be the community, social media vendor or emergency services? emax has to rely on the third-party social media service provider at this stage. 23. unauthorised information sharing does it have a clear understanding as to who has rights to grant permissions to share and secure the information sourced via social media (e.g., who may create or close threads for communication in social media)? emax’s online channels, including social media platforms, must be authorised by the executive manager, strategic communications. 24. multiple or too many sources of information does it have the ability to control which media website or thread to follow, and manage the situation if someone shuts down a popular thread? emax authorised official representatives to disclose that they are emax members or contractors. emax also has multiple channels and could still operate if someone shut down a popular thread. table 5: social media management viewpoint assessment results emax has a social media policy that applies to all emax members and external contractors who use social media professionally and personally. emax has a process to support their policy that defines its obligations with regard to online comments and social media use and to raise awareness among members who make comments via social media platforms. the social media australasian journal of information systems gill & alam 2015, vol. 19, pp. 23-40 social architecture 36 process suggests that emax members cannot comment as official representatives of emax unless they are authorised to do so. when engaging with social media, the authorised official representative must disclose that they are an emax member or contractor. the authorised members must disclose and comment only on information classified as public domain information and take all reasonable measures to ensure that online content is accurate and not misleading. the official emax online channels have a moderation policy in place that outlines what content and conduct is considered acceptable and what action will be taken if policies are breached. all websites established as official emax online channels, including social media platforms, must be authorised by the executive manager, strategic communications. emax stated that, in order to gain credibility with over 100,000 followers on social media sites, emax has had to have a social media presence for a number of years as well as having a media unit to gain credibility offline as well. emax has social media officers and the hootsuite system to monitor social media sites to verify and validate information. in summary, the overall empirical case study analysis results suggest that emax has a well-established social mediaenabled social architecture that addresses most of the concerns listed in the sava index. 5 discussion: limitations and contribution a number of case studies regarding the use of social media during a crisis have been reported in the past. most of the existing research studies are focused around the use of social media and its benefits during a crisis (sutton et al. 2011). emergency management agencies are still learning about the ramifications of the use of social media and are not sure how to best assess and use it. this paper takes a critical viewpoint and discusses the social media concerns in the context of an australian emergency management agency. this paper adopted sava, as a systematic analytical framework, to identify and assess social media concerns in the context of the australian case study organisation, emax. the sava framework provided an analytical lens through tables (table 2-5) that contain key concern categories, major categories and viewpoints. the sava construct is focused on the social media concerns related to an emergency management agency’s social architecture. however, sava is not limited to any particular social media. the use of sava in the australian emergency management agency case study gave us a number of insights into social media adoption in emergency management agencies. it can be observed from the results that sava lays the foundation of how one might rationally approach the assessment of social media concerns, which are critical for developing or updating social architecture for the real-time sourcing and distribution of disaster information. the sava construct, by no means, is an exhaustive criteria list (e.g. categories) of social media concerns. since the literature and practice are dynamic, sava should be considered as an evolving construct; and it should be revised and extended by emergency management agencies to suit their local context. the use of a single case study could also be considered a limitation when generalizing the analysis results. however, the case study selected for this empirical analysis is a well-known organisation with advanced social media capabilities and architecture. the case study organization has been using social media for a number of years. the review of emax through the lens of sava indicated that emax has a mature architecture and approach to social media usage for the crowd-sourcing and distribution of disaster information. emax’s current architecture addressed most of the concerns outlined in sava. emax realises that some of the architecture concerns mentioned in sava require attention and that they need to address those concerns. through this analysis, it was found that social media systems like facebook and twitter are only one part of their public information strategy and social architecture. emax makes sure that it is not reliant on only one form of public information. it is interesting to note that emax has two additional roles within the emergency co-ordination centre dedicated to social media intelligence gathering and posting. from the review and observations made on the emax facebook page, the emax connect page, and the emax twitter account, emax has definitely moved away from the notion of being a gatekeeper to being that of a translator, as mentioned in the literature review and the social media tool (e.g. 5. information latency) and has a collaborative approach to engage with their followers on each page and australasian journal of information systems gill & alam 2015, vol. 19, pp. 23-40 social architecture 37 welcome discussion and inquiry to clear up incorrect information. it is also interesting to note that people of all ages follow emax on facebook, contradictory to the perceived notion that ‘young’ people would be more interested in using social media. the emax connect website is also interesting, as it is dedicated exclusively to emax-related materials with web 2.0 elements such as blogs, videos and photos, and live online forums to which members of emax can contribute. members of emax contribute photos and videos of training exercises and blog posts about fire brigade news. emax connect appears to be similar to a facebook for emergency services personnel and members of the public who are also interested in emax activities. it appears that emax is quite mature in relation to social media usage. governments, policy makers, and other emergency management agencies should consider the emax empirical case study analysis as an example of how to effectively assess and adopt social media. based on the analysis results presented in this paper, we can offer the following practical recommendations: • create the roles of social media intelligence gatherers and general social media officers and have these positions on standby in the event of an emergency • provide more support, resources, and encouragement to use social media within emergency management agencies • communicate and collaborate with social media providers such as facebook and twitter so these social media providers can help during an emergency • develop the ability to automatically post information from an official website to twitter and facebook for strong information integration and interoperability this research presents previously unavailable social media viewpoints and concerns. the identified social media viewpoints and concerns, if not all, could be considered a research theme. for example, consider the following concerns by category: changing demographics, information source trustworthiness, lack of information interoperability, crowd-sourced information commingling, information overload and social media “self-victims.” one may be interested in researching and proposing theories or constructs for checking the informationsource trustworthiness. others may be interested in researching information overload and its impact on human and financial resources. this paper purposely did not discuss social media hype and benefits in great detail; rather, it adopted a systemic sava approach in order to identify and assess social media concerns. in our future research, we intend to investigate the “information source trustworthiness” concern category to further explore trusted social architecture. in summary, social media is simply another system through which to source and disseminate disaster information. it is, essentially, making sure that communities stay informed during a disaster situation. based on the analysis presented in this paper, it can be suggested that social media systems should be considered a part of the whole solution for creating informed and resilient communities. 6 conclusion this paper presented an in-depth case study of an australian emergency management agency with the help of sava. the results of the case study indicate that sava seems useful in providing necessary criteria questions to identify and assess the social media concerns of the existing or to-be-developed emergency management social architecture. this case study provided a deeper insight into social media concerns, which must be carefully identified, assessed, and addressed by emergency management agencies. the case study results indicate that the case study organisation, emax, was able to identify and assess all of the social architecture concerns through the use of sava. social media concern assessment is, of course, not an easy task, and emergency management agencies may tailor and extend the sava construct and include their own criteria for the assessment of social media concerns. this paper aims to reduce the uncertainty of emergency management agencies and increase understanding of how to identify and assess social media concerns. on the basis of the empirical case study, it is concluded that sava is useful for identifying and assessing social australasian journal of information systems gill & alam 2015, vol. 19, pp. 23-40 social architecture 38 media concerns for developing or updating social architecture. however, sava needs to be considered with a view of its limitations, since the body of literature and practices are both dynamic in nature, and it should thus be considered an on-going construct to be revised and extended by emergency management agencies and researchers. in future, we intend to extend our research into the trusted social architecture viewpoints and concerns of sava. 7 acknowledgements we are thankful to people from the emergency management agencies who helped us with their valuable feedback and experience. references chatfield, a.t. and brajawidagda, u. (2012) “twitter tsunami early warning network: a social network analysis of twitter information flows”, twenty-third australasian conference on information systems, geelong australia, deakin university cio council (2001) a practical guide to federal enterprise architecture, version 1.0. us government accountability office (gao). available from: http://www.gao.gov/products/p00201 cross, r., liedtka, j. and weiss, l. (2005) a practical guide to social networks. harvard business review. available from: https://hbr.org/2005/03/a-practical-guide-to-socialnetworks eustace, j. and alam, su. (2012). "tweeting from the danger zone: the use of twitter by emergency agency during mitchell factory fire in canberra", mcis 2012 proceedings. paper 27. foster, h. (2013). interactive hazard preparation strategy efficacy: consideration for future community engagement programs. australian journal of emergency management, 28(1), 8-14. gill, a.q., bunker, d. and seltsikas, p. (2011) an empirical analysis of cloud, mobile, social and green computing: financial services it strategy and enterprise architecture. 2011 ieee ninth international conference on dependable, autonomic and secure computing, sydney, australia. gill a.q. and bunker, d. (2012) crowd sourcing challenges assessment index for disaster management. proceedings of the americas conference on information systems, seattle, united states. gill, a.q. (2013). towards the development of an adaptive enterprise service system model. proceedings of the americas conference on information systems, chicago, usa. gill, a.q., alam, s. and eustace, j. (2014) using social architecture to analyzing online social network use in emergency management". proceedings of the americas conference on information systems, savannah, usa. grabowski, m. and roberts., k. (2011) high reliability virtual organizations: co-adaptive technology and organizational structures in tsunami warning systems. acm trans. comput.-hum. interact. 18(4). iso/iec 42010. systems and software engineering — architecture description: defining architecture [online]. available from: http://www.iso-architecture.org/ieee1471/defining-architecture.html. harrison, r. (2011). togaf foundation. the open group. heath, l. r. (1998) new communication technologies: an issue management point of view, public relations review, 24(3), 273-288. australasian journal of information systems gill & alam 2015, vol. 19, pp. 23-40 social architecture 39 horita, f., degrossi, l., assis, l., zipf, a., albuquerque, j. (2013) “the use of volunteered geographic information and crowdsourcing in disaster management: a systematic literature review”, proceedings of the 19th amcis, chicago, illinois. hughes, a. (2009) twitter adoption and use in mass convergence and emergency events, university of colorado: bolder. kerkhof, j.v.d. (2012). towards a social media maturity model. master’s thesis. utrecht university. latonero, m.and shklovski, i. (2010). respectfully yours in safety and service emergency management & social media evangelism. ssrn: http://ssrn.com/abstract=1566423. maier, m.w. (1998) architecting principles for system of systems. systems engineering. 1(4), 267-284. majchrzak, a and more, p.h.b. (2011). emergency! web 2.0 to the rescue. communications of the acm, 54(4), 125-132. nsw ses report (2011) the capacity of communications networks and emergency warning systems to deal with emergencies and natural disasters. environment and communications references committee, canberra. othman. s.h., and beydoun, g. (2013). model-driven disaster management. journal of information and management. 50(5) 218-228. parameswaran, m., whinston, a.b. (2007) social computing: an overview, communications of the association for information systems, 19, 762-780. pries-heje, j., baskerville, r., and venable, j.r. (2008) strategies for design science research evaluation. the 16th european conference on information systems (ecis), galway, ireland. quillinan, t.b., brazier, f., aldewereld, h. dignum, f., dignum, v., pen serini, l., wijngaards, n. (2009) developing agent-based organizational models for crisis management, international conference on aamas, budapest, hungry, 45-52. qumer, a. and henderson-sellers, b. (2008). an evaluation of the degree of agility in six agile methods and its applicability for method engineering. journal of information and software technology , 50(4), 280-295. royal commission report. (2011) queensland floods commission inquiry – interim report [internet]. available from: www.floodcommission.qld.gov.au ross, j.w. weill, p, and robertson, d. (2006). enterprise architecture as strategy: creating a foundation for business execution. harvard business review press. semaan, b., and mark, g. (2011) technology-mediated social arrangements to resolve breakdowns in infrastructure during ongoing disruption. acm transactions on computer-human interaction (tochi), 18(4), article 21. spohrer, j. & kwan, s. k. (2009) service science, management, engineering, and design (ssmed): an emerging discipline outline & references. international journal of information systems in the service sector, 1(3), 1-31. starbird, k. & palen, l. (2011) “voluntweeters:” self-organizing by digital volunteers in times of crisis, proceedings of the acm 2011 conference on computer human interaction, vancouver, bc, canada. starbird, k. & stamberger, j.t. (2010) tweak the tweet: leveraging microblogging proliferation with a prescriptive syntax to support citizen reporting, proceedings of the information systems for crisis response and management, seattle, usa. safko, l., and brake, d. (2010). the social bible. hoboken (nj). john wiley & sons, inc. australasian journal of information systems gill & alam 2015, vol. 19, pp. 23-40 social architecture 40 sutton, j., hansard, b., and hewett, p. (2011). changing channels: communicating tsunami warning information in hawaii. the 3rd international joint topical meeting on emergency preparedness and response, robotics, and remote systems. knoxville, tennessee. tapia, a. & moore, k. (2014) good enough is good enough: overcoming disaster response organizations’ slow social media data adoption. proceedings of computer supported cooperative work. terman, e. (2011) five top challenges of integrating social media data with business applications [internet]. available from: http://www.ctoedge.com/content/five-topchallenges-integrating-social-media-data-business-applications the advertiser. (2011). cfs to use twitter and facebook for bushfire warnings [internet]. available from: http://www.adelaidenow.com.au/first-ban-of-fire-season-for-sa-northeast-pastoral-district/story-e6frea6u-1226140282012 vieweg, s,palen l,lui s,hughes, sutton j. (2008) collective intelligence in disaster: examination of the phenomenon in the aftermath of the 2007 virginia tech shootings, university of colorado: boulder yin, r.k. (1994) case study research: design and methods, sage publications. zachman, j.a. (1987) a framework for information systems architecture. ibm systems journal, 26, 276-292. copyright: © 2015 gill, alam & eustace. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. microsoft word ajis_18(3)_v10.docx australasian journal of information systems volume 18 number 3 2014 5 do self-service technologies affect interfirm relationships? a b2b perspective raechel johns university of canberra raechel.johns@canberra.edu.au abstract while considerable literature examines business-to business (b2b) relationships, the impact of technology on these relationships has lacked attention. it has impacted the way businesses operate in a b2b context as well as influencing services by altering the way services are delivered. to understand the way in which internet technology has impacted these b2b services, it is essential to examine its impact not only on business processes but on business relationships too. one technology, enabled by modern internet technologies, which is changing the nature of business relationships is the increased use of self-service technologies (ssts) or technology-enabled services, however, there is a shortage of research in the area in a b2b context. the discussion in this paper provides an overview of the impact of it on business relationships, using relationship marketing theory to provide a theoretical framework. a qualitative study in the australian banking industry provides findings relating to the theory. the findings of this study provide evidence that while traditional relationship marketing theory is still applicable for some business customers, new theory is required for business customers who do not seek relationships. with both practical and theoretical implications, the research detailed in this paper makes a useful contribution to the literature and indicates the necessity for further research to be developed which explores business customers who prefer to remain transaction-oriented, rather than develop interpersonal relationships. keywords: self-service; internet banking; interfirm relationships; b2b; qualitative introduction it is essential to understand the impact of technologies on the relationship between the marketer and their business customer, business processes and productivity, because it will impact on customer satisfaction in relationships. businesses are increasingly using it in their b2b operations (pujari 2004). in particular, the increasing use of self-service technologies (ssts) within b2b (b2b) relationships removes face-to-face contact traditionally believed to be important in relation to service delivery between two organisations (storper and venables 2004). the impact this has on the relationship has not been widely examined because of the focus on the interpersonal face-to-face relationship. it would be useful to know whether technology impacts on trust, loyalty and ultimately the relationship itself. from a theoretical perspective, the importance of developing and fostering relationships with customers has long been regarded as important within services marketing (berry 1983) and also within b2b relationships (ford 1990). in the 1980s and 90s, a shift in marketing focus has seen an increased emphasis on relationship marketing (morgan and hunt 1994). it is therefore important to consider relationship marketing in the context of this study in order to have a better understanding of the impact of it on business relationships. this paper examines the impact of ssts on interfirm relationships. the paper commences with an overview of literature; findings and implications from an exploratory study will then be discussed and the paper will conclude with further research streams, drawn from the research. australasian journal of information systems volume 18 number 3 2014 6 to explore the research question “what impact does the use of ssts have on interfirm relationships?” qualitative research was utilised. this was due to the exploratory nature of the study (gummesson 2008; johns 2008). first, the research propositions linked the literature will be provided. findings will then be discussed and an overall discussion, linked back to new literature will be provided. literature review understanding relationship marketing relationship marketing is a major theory in marketing, particularly within a b2b (interfirm) context (kandampully, 2003) and when services are offered rather than goods (berry, 2002). relationship marketing requires the establishment of trust and commitment (morgan and hunt, 1994) and marketers may strive to produce alliances (hunt, lambe and wittmann, 2002) through their relationships with their business customers. despite this, there is little understanding of how the use of it impacts on trust and commitment to the relationships. it is essential to understand what relationship marketing is, and while there are many published definitions, one definition commonly used is: relationship marketing includes tasks undertaken to identify and establish, maintain and enhance and, when necessary, terminate relationships with customers and other stakeholders, at a profit, so that the objectives of all parties involves are met; and this is done by mutual exchange the fulfilment of promises (grönroos, 1994: 9). relationship marketing focuses on increasing customer retention and loyalty (berry 1983) and the importance of relationship marketing in the literature has grown considerably in the past two decades. attracting new customers is quite costly, therefore organisations strive to retain satisfied customers. relationship marketing is particularly important in a b2b or interfirm context, where there are fewer customers, but transactions are generally of higher value (kotler, adam, denize and armstrong 2009). within banking, relationship marketing practices are essential a way to enhance customer retention (colgate et al. 1998; colgate and stewart 1998), foster trust, resulting in relationship commitment and satisfaction (morgan and hunt 1994). while many marketing activities focus on reaching out to a specific target market, relationship marketing focuses on the individual (i.e. looking at the single customer), rather than the ‘average’ customer (wolfe 1998). relationship marketing orientation is viewed as more important than a transactional orientation within a b2b context (anderson 1995), due to the cost of attracting a new customer rather than maintaining a relationship with an existing customer (gummesson 2008). without trust, loyalty and satisfaction it is impossible to build and maintain a relationship between organisations (morgan and hunt 1994); which is the main objective of b2b marketing. as a consequence, many researchers have explored trust (young, 2006) and commitment (cater and zabkar, 2009). relationship marketing is particularly important in a service setting due to the direct contact between customer and marketer and the interactive nature and process of service delivery. the intangible nature of services makes them difficult to evaluate prior to service because customers cannot feel or see service quality (berry 2002) and must purchase their service prior to experiencing it (berry and parasuraman 1991). if a customer trusts a service provider, they are likely to return to that service provider (berry 2002), increasing loyalty to the relationship and reducing perceived risk to the customer. with an understanding that relationships are particularly important in a service b2b context, the first research proposition is therefore: proposition 1: business customers want relationships with their service providers australasian journal of information systems volume 18 number 3 2014 7 benefits of ssts it is evident that it has altered the way business is conducted over the past fifteen years (meuter et al., 2005). with the introduction of ssts, consumers are required to carry out the transaction themselves; they are responsible for their own satisfaction (meuter and bitner, 1997; bendapudi and leone, 2003) and as a result organisations have had to effectively train customers to be co-producers. turning the customer into a co-producer of a service has become evident where the customer participates in value creation (vargo and lusch, 2004) such as checking out a library book, booking a hotel room online, undertaking online investment trading and internet banking. perhaps the most dramatic change is the increase in the use of the internet, allowing organisations the opportunity to introduce ssts. most new ssts are internet enabled and in many industry sectors, such as travel, corporate banking and professional services, proprietary self-service technology systems are developed (pujari, 2004) to most effectively meet the needs of customers and the organisation. it has already been shown in the literature that ssts can be used to enhance customer service (despite limited staff involvement), pay bills, track delivery times (bitner et al. 2002) and undertake other services which do not require staff involvement. organisations are introducing ssts rapidly for three major reasons: to reduce costs, increase customer satisfaction and loyalty and to reach new customer segments (bitner et al. 2002). ssts are increasingly being utilised in business (meuter, bitner, ostrom and brown 2005) due to people being time poor and as a drive towards cost savings for the organisation. while increasing customer loyalty is aligned with the objectives of relationship marketing, reducing costs is simply an organisational driven initiative and may not serve the interests of the customer. furthermore, although time savings lead to cost savings for organisations, this is only apparent if the ssts are adopted by the customer (meuter, bitner, ostrom and brown 2005). this means organisations must give customers no choice (potentially decreasing satisfaction) or encourage use of the technology by ensuring customers use the technology efficiently and having them feel there is support if anything goes wrong or presenting them with a compelling reason as to why they should use such a service. preference for humans although there are benefits to using ssts, customers often prefer to deal with humans for some services (marr and prendergast 1993). johns (2012) indicates that some business customers are more relationship oriented than others, supporting lovelock’s (1991) belief that the target market and industry should influence the choice of delivery system. as lovelock’s study was conducted prior to internet banking usage, it is important to consider the use of internet-enabled ssts. furthermore, the producer and consumer are often viewed as separate in marketing, however, when considering ssts, “…the consumer is always involved in the production of value” (vargo and lusch 2004: 11). this means the customer must use and repair the service themselves, based on their requirements (vargo and lusch 2004) and this impacts on perceptions of service quality (bitner et al. 2002). this is particularly an issue because most ssts do not have service recovery systems in place (bitner et al. 2002) increasing dissatisfaction with the technology and potentially the brand. face-to-face face contact has been widely regarded as necessary for the establishment and strengthening of relationships (jayawardhena, souchon, farrell and glanville, 2007; crosby, evans and cowles, 1990, winstead, 2000), however more recently it has been suggested that customers can be satisfied with an increased transaction focus, providing this is a choice for the customer (johns, 2012). this is particularly important in a b2b context. with the understanding that ssts provide benefits to customers, but that face-to-face contact is important in establishing and strengthening relationships, the second research proposition is: proposition 2: increased use of ssts reduces relationship closeness australasian journal of information systems volume 18 number 3 2014 8 co-creation and customer satisfaction with the introduction of ssts, consumers are required to carry out the transaction themselves; they are responsible for their own satisfaction (meuter and bitner 1997; bendapudi and leone 2003) and as a result, organisations have had to effectively train customers to be co-producers. in order to effectively engage in self-service, therefore, customers must have sufficient skills and core competencies (vargo and lusch 2004) or have undertaken training on service use to make this self-service possible. training, however, is often not available as most organisations implement ssts as a cost and staff cutting measure and cannot justify training customers. the most likely obstacle to getting customers to utilise ssts is getting them to change their existing behaviours (meuter et al. 2005) and from a b2b perspective there is limited understanding of ssts and how the use of these technologies impact on relationships between the marketer and customer (johns et al. 2009). in addition to potential issues with the use of ssts, criticisms regarding using the internet in b2b relationships indicate that the separation of buyers and sellers can be problematic (ratnasingam and pavlou 2003). this potentially impacts on trust and, ultimately, loyalty to the relationship, but there is little evidence of this in the literature. a theoretical overview: b2b marketing a broad perspective on b2b marketing is provided in the literature, drawing on theory from economics, psychology, anthropology, sociology and management (peters, pressey, vanharanta and johnston, 2013). cooperation is considered vital in establishing and strengthening any business relationship (wilkinson and young, 2002) and trust is a priority area of the b2b literature. despite a lot of research into trust, however, more conceptualization is needed in a b2b context (young, 2006). some researchers have explored the end of an interfirm (b2b) relationship (young and denize, 1995 and bogomolova and romaniuk, 2009) and others have focused on improving problematic interfirm relationships (pervan, bove and johnson, 2009). b2b marketing has a number of key themes: sales management, buyer behaviour, innovation, marketing management, distribution and finally buyer-seller relationships (laplaca and katrichi, 2009). this current study explores buyer-seller relationships and how management of these fragile and important relationships impact on marketing strategy. since the debate and discussion of service-dominant logic commenced (vargo and lusch, 2004), the concept of value has increased in the b2b literature, with researchers exploring co-production (chen, tsou and ching, 2011). with the understanding that trust is essential in b2b relationships and commitment is developed through trust, but cocreation is undertaken in a self-service context, the final research proposition suggests that: proposition 3: an increase in ssts decreases trust and ultimately commitment summary of the literature review this brief review of the literature has indicated the nature of interfirm relationships and the importance of these relationships. furthermore, current understanding regarding the impact of it on business relationships was discussed as this is the purpose of this research. the nature of trust in relationships and the requirement of trust to build loyalty is increasingly understood by marketers (morgan and hunt 1994) yet there is limited examination of how it impacts on important b2b relationships. table 1, below, provides an overview of the literature discussed. australasian journal of information systems volume 18 number 3 2014 9 dimension author method relevant findings commitment morgan and hunt, 1994 quantitative trust is necessary to build relationship commitment in an interfirm context trust young and wilkinson, (the role of trust and cooperation conceptual firms need a strong relationship with their interfirm partners in order to develop efficiency face-to-face contact jayawardhena, souchon, farrell and glanville, 2007 quantitative face-to-face contact is necessary to establish trust the impact of technology on relationships johns, 2012 qualitative although the literature states that relationships require face-to-face contact, there is a consideration that some customers prefer to transact at a distance separation of buyer and seller ratnasingam and pavlou 2003 quantitative the internet can be problematic to relationships due to the separation of buyers and sellers technology trust and trading partner trust ratnasingam and pavlou 2003 quantitative in business relationships facilitated by technology, there are two different kinds of trust, both important: trust of the trading partner and trust of the technology table 1: summary of the broad relationship literature while the literature suggests that complementary it services can add value to a relationship (stone and woodcock 1997; grimm 1999) there seems to be a requirement for face-to face contact to ensure trust and, subsequently, relationship commitment (rocco 1998). the major research question this study answers is “what impact does the use of ssts have on interfirm relationships?” although relationship marketing theory assists in providing a theoretical background to the study, the extant literature does not address the impact of self-service use on relationships in a b2b context. as the use of ssts has increased in b2b contexts, this paper explores this gap by working toward answering three research propositions: proposition 1: business customers want relationships with their service providers proposition 2: increased use of ssts reduces relationship closeness proposition 3: an increase in ssts decreases trust and ultimately commitment australasian journal of information systems volume 18 number 3 2014 10 methodology the industry context as a sample frame, this study used the banking industry; the reasons were two fold (a) this industry has been fundamentally changed by self-service technology, and (b) the issues of trust and loyalty being important to maintaining existing customers have been recognised in the banking industry. previous research in bank marketing has primarily focused on personal customer banking (pujari, 2004) however over recent years businesses have increasingly utilised technology in managing relationships (gummesson, 1996 and johns and perrot, 2008). it is therefore important to understand how the use of technology impacts on relationships between banks and commercial customers. a focus on banking is particularly important due to the importance of customer relationships in banking (barnes, 1997; colgate, auckland and alexander, 1998) and within a b2b context (kandampully, 2003; gummesson, 2008b). the banking industry is renowned for its innovation in service delivery and distribution and this has been a method of differentiation in the industry for decades. in order to ensure that banks remain competitive, alternative means of distributing banking products have been considered for many years (pikkarainen, pikkarainen, karjaluoto and pahnila, 2004). in recent years, banks have been promoting online banking as a convenience for customers and for cost savings for the banks. consequently, the research presented in this study is essential to understand the impact of ssts on business banking relationships. qualitative research methods are appropriate in this study due to the exploratory nature of the work. qualitative methods can contribute to understanding the way managers interact with other players in the market (o'donnell and cummins, 1999) and are appropriate for exploring relationships. with a gap in the existing research, it is necessary to consider the area from an exploratory perspective, calling for the use of qualitative research methods. semi-structured interviews were utilised to collect data because they give an insight into an individual’s behaviour and attitudes (tull and hawkins, 1990) and are effective for collecting data from business customers (curran and blackburn, 2001). table 2 indicates which methods have been utilised in the past in previous studies in similar areas: australasian journal of information systems volume 18 number 3 2014 11 authors focus of study method/s utilised pujari, 2004 self-service encounters, b2b satisfaction and dissatisfaction critical incident method joseph, mcclure and joseph, 1999 service quality in the banking sector focus groups lang and colgate, 1993 banking – personal bankers quantitative only – survey mulligan and gordon, 2002 the role it plays in supporting relationships between customers and marketers in the financial services industry personal bankers personal interviews of banks chan and lee, 2002 e-procurement adoption by smes case study interviews with small businesses; observation; document review walker, craig-lees, hecker and francis, 2002 adoption of technology enabled service delivery b2c focus groups durvasula, lysonski and mehta, 2000 b2b relationship marketing – ocean shipping quantitative only survey meuter, bitner, ostrom and brown, 2005 focus on trial of ssts innovation adoption focus quantitativesurvey of groups who have used the technology vs. those who have not table 2: review of previous studies’ methods as indicated in the table above, researchers have previously use a variety of methods for collecting data in similar studies. while there are advantages to various methods, it was deemed most appropriate to use semi-structured interviews, because of privacy and security concerns and to obtain a more indepth response. table 3 indicates a summary of the respondents. australasian journal of information systems volume 18 number 3 2014 12 respondent profile n= banks interviewed n = 6 business customers interviewed n = 25 decision makers n = 13 users n = 12 transaction-oriented customers n = 13 relationship-oriented customers n = 12 small organisations n = 13 medium organisations n = 7 large organisations n = 5 table 3: respondents for the study during the analysis process, the twenty-five business customer respondents were categorised by the business customers’ role with the banking institution into the role of ‘decision maker’ or ‘user’. this was particularly important due to the differences in the two categories of users (zaltman, duncan and holbeck 1973; leonard-barton and deschamps 1988; gallivan 2001). while a decision maker invariably utilised banking services, they were also the person in the organisation who selected the bank the business banked with and, in many cases, had more control over the tools used in banking. data was analysed both manually through thematic analysis and by the computer programs nvivo and leximancer. nvivo requires the researcher to conduct coding in the initial stages, while leximancer develops codes and maps based on these codes through its thesaurus of terms. both computer programs were used due to the advantages of both and also to allow for a comparison of data analysis and more thorough analysis of the data. (a table comparing the software is provided in appendix one). findings and implications for some respondents, loyalty to a company’s bank was established through the bank developing a thorough understanding of their business. this built up over time through negotiations with the bank. it was evident that the relationship exists between the business customer and the business banker. if either party changed, the loyalty may no longer exist. in general, decision makers were more favourable toward the relationship with the bank, compared with users. (an overview of representative quotations and summary is provided in appendix two.) the interpersonal relationship between banks and businesses was identified as being between two people and therefore relationship dynamics would differ if another person came into the role in the customer organisation, or another relationship manager took over. in addition to personality and authority, respondents stated that a good relationship manager had a good understanding of the customer’s business requirements. during the interviews it was discovered that two types of customers existed – those who were transaction-oriented and those who were relationship-oriented and size was found to not be an indicator of orientation toward ssts. it was found that both transaction and relationship-orientated customers behaved differently and had different issues. australasian journal of information systems volume 18 number 3 2014 13 relationships in b2b banking services proposition 1: business customers want relationships with their service providers the research uncovered a new contribution to the literature – while the literature states that business customers want relationships with their service providers, this proposition was deemed to be untrue in the context of ssts, for some customers. it was clear through the research that for relationship-oriented customers, transactions can be facilitated through self-service transactions, but relationships were essential for building loyalty. for transaction-oriented customers, however, interpersonal contact was seen as unnecessary as almost all transactions could be conducted at a distance and required no discussions with the bank staff unless a problem arose. many customers had to go to the branch, regardless of whether they were transaction or relationship-oriented, because they need to deposit cash. several customers explained that in the branch they were generally not known by the bank staff. according to the data, a relationship banker, willing to spend time to understand a unique business structure, could have considerable influence on how much an organisation was able to borrow and therefore had a huge impact on business growth. some respondents claimed their business structure allowed them to avoid any real relationship with the bank and focus purely on transactions. for example, one young business owner, in his twenties, stated that he fell into a customer category that was not catered for, as the business was not of a significant size. he explained that this was not completely based on his generation, but rather about the way certain people embrace it. the banks felt that relationships were shifting, and this was generational. younger people tend to be online more and, as a consequence, want to do all transactions online. this lowered loyalty to the bank brand because there was no real relationship. while there were divergent views relating to whether business customers had a preference for relationships with their service providers or not, the service providers (banks) had a different perception. the banks did not understand that transaction-oriented business customers existed. instead the representatives interviewed believed that relationships were essential for business customers. for example, one quote stated: oh (relationships are) critical. particularly for small business, you need to be there and available to them. it’s not like doing a home loan where you’ll only hear from your customer once or twice a year. with a business client you’re probably talking to them once or twice a week (bank 2, manager) there are some business customers still seeking and requiring a relationship. through this analysis, two distinct segments in the business banking market have been identified – those primarily seeking a transaction and maintain a relationship at a distance, and those requiring a relationship where they choose to work closely with their business banker. research proposition 1, therefore, is deemed to be only partially correct – not all business customers seek relationships with their service providers. proposition 2: increased use of ssts reduces relationship closeness the research indicated that the increased use of ssts does alter relationships and relationship closeness for transaction-oriented customers but not for relationship-oriented customers. the change in the nature of the relationship is simply a business evolution but it was something that was acknowledged by both the banks and the business customers interviewed. it has changed the nature of relationships, according to the banks interviewed, creating increased bank switching. aside from not physically seeing customers as regularly as they had previously, there was a perception that there was a reduced loyalty to the relationship due to the technology. a decision maker, boasting a thirty year relationship with their bank, stated that he was still aware of who the bank manager was, however he stated “i haven’t really spoken to them much and i’m probably australasian journal of information systems volume 18 number 3 2014 14 having less to do with them now than i used to” (r1, decision maker). he reported that this was not problematic, nor did it affect his loyalty provided the technology worked. he did comment, however, that it was essential to establish a relationship in the initial business set up stage. technology results in business customers having more control over their regular daily transactions; however, this can reduce the level of loyalty. the findings indicate that loyalty can be generational, or can be related to cash flow requirements for the business, however bank customers are less loyal than they were in the past. furthermore, with the use of technology in business banking, relationships are less valued than they were in the past. as the younger generation become decision makers in business there is the potential that they will prefer to transact at a distance, rather than through having a strong, interpersonal relationship. this research proposition is not completely accepted: increased use of ssts shifts some relational aspects for transaction-oriented customers, but for relationship-oriented customers, there is very little impact. loyalty and trust proposition 3: an increase in ssts decreases trust and ultimately commitment the research indicated that an increase in ssts decrease trust and commitment for transaction-oriented customers, but not for relationship-oriented customers. commitment was explored primarily due to its importance in the relationship marketing literature. there was no real pattern in terms of commitment, with a fairly even mix between ‘low’ and ‘strong’ levels of commitment. however, as indicated in table 4, there were differences with relationship-oriented customers: level of commitment transaction based customers relationship based customers low r5, r8, r13, r15, r16 r2 medium r20, r21, r23 r3 strong r9, r10 , r14, r17, r19 r1, r4, r6, r7, r11, r12, r18, r22, r24, r25 table 4 – commitment to bank as rated by interviewees although table 4 above indicates both relationship based customers and transaction-oriented customers can have a strong degree of commitment, there is more diversity in commitment for transaction-oriented customers while the relationship based customers were mainly found to have a high level of commitment to their banks. the results illustrated in the table show that it could be expected that relationship-oriented customers are loyal to the relationship. although not all relationship-oriented customers have a strong commitment to their bank, this was only apparent when there have been disruptions to the relationship and they would like to move but feel they cannot (in the case of r2) or when a user feels their boss is loyal, but has no particular reason to be loyal themselves (in the case of r3). in contrast, it is very difficult to predict how loyal a transaction-oriented customer is likely to be, due to the diversity of responses. the data indicated that commitment and trust was built on long term interactions, rather than through self-service delivery of technology. commitment and trust are outcomes of interactive relationships, while self-service delivery was perceived as merely fulfilling a transaction. therefore relationshiporiented customers were more likely to trust their business partner than transaction-oriented customers. however trust comes in several forms, and trust in the technology was present for the majority of australasian journal of information systems volume 18 number 3 2014 15 customers regardless of whether there was a relationship or transactional exchange. table 5 indicates factors leading to commitment for each segment. factors leading to loyalty transaction based customer relationship based customer trust of the bank trust of the banking industry switching cost perceptions organisation’s value offering carefully targeted promotion personality of individual people/ individual interactions table 5: factors leading to commitment for the customer segments both segments reported that carefully targeted promotion fostered commitment. this can be seen in the table above. switching cost perceptions, however, were the main reason to stay for the transaction segment while trust and interactions with their relationship banker and bank staff fostered loyalty for the relationship-oriented segment. the differences in the segments are illustrated in table 6: transaction customer: relationship customer: embraces it accepts it gets value from technology working tends to be a ‘user’ tends to be younger tends to be technologically savvy in banking – tends not to require funding tends to have a complex business structure tends to prefer relationships with other organisations views technology as separate from the interpersonal relationship in banking – tends to require funding table 6 customer segment profile characteristics while commitment and trust existed in varying degrees for all customers, there were different types of commitment evident. calculative commitment is a more complex form of relationship loyalty where the customer perceives it is difficult to replace their trading partner so stays in the relationship (kumar, hibbard and stern 1994). this was evident with some transaction-oriented customers where the switching cost perception saw them maintain a relationship that they were not particularly satisfied with. other respondents stated that they would switch if a cheaper partner was possible. in contrast, affective commitment is associated with positivity toward the relationship (kumar et al. 1994). for the relationship-oriented customers who felt a strong level of relationship loyalty, affective commitment was present. therefore, proposition three, that an increase in ssts decrease trust and ultimately commitment, is correct for the transaction-oriented customers, but not for relationship-oriented customers. australasian journal of information systems volume 18 number 3 2014 16 findings linked back to the literature i don’t know what the future is for internet banking. at the moment, it ticks a lot of boxes, but if any one of those boxes weren’t ticked, you’d have a problem with it. it doesn’t really exceed expectations, though, does it? (r19, decision maker). two customer segments were identified in the research – those primarily seeking to transact and those primarily seeking an interpersonal relationship with the bank. the findings in this study indicate that the use of it does not impact on relationships for relationshiporiented customers, due to their perception that the technology and relationship is separate. for transaction-oriented customers, both trust and relationship loyalty were altered, however, these customers still prefer the use of technology. technology is continually improved and utilised in organisations, this study has examined how technology, and its use as a transaction medium between two businesses, can impact relationships. relationship marketing requires building and maintaining long term relationships (gummesson 2008). previous literature acknowledges that not all customers require relationships and that ‘transaction marketing’ is the opposite of relationship marketing. despite the existence of transaction marketing in the literature, the findings in this study differ from those in the literature on relationship marketing and transaction marketing. transaction marketing in the literature is focused on what the customer is purchasing, generally taking a short term, discrete transaction perspective (grönroos 1995; brodie, coviello, brookes and little 1997; ivang and sørensen 2005). this means that some industries want relationships while other industries are prone to transactions. for example, purchasers of fast moving consumer goods (fmcg) will generally not seek a relationship with the marketer, while professional service customers will. it has also been argued that in a service context, relationships are essential (grönroos 1995) and that transaction marketing applies in a goods context (baker, buttery and richter-buttery 1998). the literature relating to transaction marketing is about personal customers rather than business customers (baker et al. 1998) and it is suggested that relationship marketing is more applicable for business customers. while this study does not contradict the extant literature, it contributes by finding that when ssts are introduced, some business customers will move to a transaction-based approach to their relationships. the study found that, on the basis of their relationship preference, business customers should be segmented. the belief in the literature that ‘commercial bank customers want an interactive relationship with their bank’ is no longer appropriate and an understanding of different customer relationships is necessary by banks. while this research found that these two distinct groups of relationship preferences exist, it also found that some business customers could not be completely categorised into these two relationship groups. this is because some customers will require relational elements but seek transactions that work, and other customers will prefer transactions because a relationship has been established in the past. the findings indicate, however, that customers with a pure transaction orientation will not develop the depth of trust and loyalty to the relationship that relationship-oriented customers will develop. while trust and loyalty still exists for transaction-oriented customers, the trust of the banking industry and the loyalty to their bank is usually caused through a perception of switching costs, rather than a real loyalty to work in an ongoing relationship with their business partner. another major contribution relates to the use of ssts. much of the literature on ssts indicates that technology forms the main basis of the relationship and has an impact on brand image. this was found to be true for the transaction-oriented respondents, however, for relationship-oriented respondents, technology was perceived merely as a means to conduct a transaction and was viewed as separate from the relationship. if the technology continually fails, the relationship would be reconsidered. on the other australasian journal of information systems volume 18 number 3 2014 17 hand, for most relationship-oriented customers, technology was perceived as just one component of transaction and relationships were based on interpersonal contact. value-adding through technology a perception of being forced to utilise technology in a relationship can drive a customer to resist the technology (godson 2009). egan (2004) believes that marketers should not attempt to replace human relationships with technology, particularly if technology is used for business efficiency rather than for value-adding. according to the findings in this study, all respondents saw the use of technology as value adding through improvement in their business efficiency. for some respondents (the transactionoriented segment), value is derived through the use of technology. this segment does not wish to have a regular, interpersonal contact with the marketer. in contrast, the ‘relationship’ segment perceives the relationship as adding value and the technology as merely a means of transactions. this finding deviates from egan (2004)’s recommendation that relationships should not be replaced with technology; and suggests that banks should provide both as a choice to ensure they are targeting both segments identified. this research indicates that when trust of the bank was established prior to the use of technology, this had an impact on the perception of technology. this was unexpected and significant. these results showed that businesses who perceive their relationship is unsatisfactory will also have an unsatisfactory opinion of the technology provided by their bank. furthermore, businesses that have a positive relationship with the bank will also have a positive impression of the technology provided. the research therefore shows that customers also seemed to perceive a separation between the technology and the relationship, viewing technology issues as separate from the bank particularly when there was a positive relationship with the bank and problems with the technology. this requires further exploration. the necessity of customer segmentation: transaction and relationship customers this discussion has highlighted the necessity of customer segmentation, because some business customers seek a interpersonal relationship with the bank while others do not. while the literature considers ‘transaction marketing’ to be making a sale (godson 2009), the description of the transactionoriented segment is a group of customers not seeking a traditional b2b relationship. the relationshiporiented segment of business customers identified in this research, in contrast, tends to mirror the behaviour described in the existing relationship marketing literature of business customers. the literature divides marketing into two types – relationship marketing and transactional marketing but in this literature the key distinction is in what the marketer offers rather than what the customer seeks (gummesson 1987). in banking, relationships are crucial to differentiate the offering (godson 2009) but the findings in this study indicated that some customers just wanted to be ‘left alone’. little empirical research exists to indicate what triggers a shift from transactional to relationship marketing (grönroos 2004). the findings from this study indicate that customers seeking a transactional relationship, rather than an interpersonal relationship, do so because they see no value to themselves or their business in having an interpersonal relationship with the marketer (or bank). our research shows that the likelihood of shifting toward a relationship is limited for these customers and therefore we suggest that research should not focus on what triggers a shift from a transactionorientation to a relationship-orientation but, rather, should consider what makes some customers value relationships and whether there are segmentation variables to consider. this study has explored both aspects and results indicate that customers with a complex business structure require face-to-face relationships. further research is needed to explore this is more detail. marketers must design their service offerings so that customers have the opportunity to establish the relationship they want with the marketer – at a distance, or with proximity but marketers should be australasian journal of information systems volume 18 number 3 2014 18 aware that customers with a more distant relationship tend to have lower loyalty and decision making develops into being more based on price of the offering than anything else. marketers must also be aware that the technology side of the relationship is transactional. this research found that only interpersonal contact establishes and builds interpersonal relationships. customers relying completely on technology do not build a relationship with the marketer – instead, they focus on the transaction elements and separate this from the interpersonal relationship. this has the possibility of reducing loyalty. for transaction-oriented customers, value is in technology that works and other factors such as lower prices (fees in a banking context). loyalty exists with transaction customers but if a superior offer becomes available they are more likely to consider switching than are relationship customers. relationship based customers, in contrast, seek a deeper relationship. their loyalty is focused on people and the organisation, rather than switching costs, and are therefore less likely to switch even if there is a better offer available. while they utilise the technology and find it essential for business efficiency (and therefore get value from the use of this technology) they place more value on their relationship with their business banker. this is due to a high level of trust and relationship loyalty and a long term focus of the relationship. technology assists all customers, regardless of which segment they belong to, in undertaking daily transactions but too much emphasis on technology can be detrimental to the relationship and ultimately reduce loyalty. wagner and boutellier (2002) argue that transaction marketing applies to a discrete transaction where there are limited, if any, future exchanges. the literature even identifies that in discrete transactions, relationships can occur (cousins 2002). there is, however, an absence of literature relating to a segment of business customers that appear in a typical relationship context, such as banking, but are transaction focused. this may be due to the introduction of ssts and this research suggests that this segment of customers will continue to increase. the findings of this research are consistent with this growth. figure 1 provides a graphical representation of the data. figure 1: relationships in a self-service context figure 1, above, indicates the differences between relationship-oriented and transaction-oriented customers. while relationship-oriented customers interact with the organisation through both technology and face to face interactions, commitment is built through trust and as a result of this, interpersonal relationships are preferred. for transaction-oriented customers, however, commitment is australasian journal of information systems volume 18 number 3 2014 19 a result of switching cost perceptions, rather than trust. these customers trust technology and therefore interact with their business partner through technology. implications for managers, theory and further research customer segmentation is a key requirement of b2b relationships. it is no longer sufficient to state that ‘business relationships require interpersonal relationships’. managers must therefore work toward an understanding of customer needs and utilise customer segmentation in implementing relationship marketing principles. from a theoretical perspective, a new theory of relationship marketing within a self-service technology context is called for. marketers must have a good understanding of the two customer segments this research has discovered. building trust and relationship loyalty is essential for the relationship segment; switching costs for the transaction segment. the existing theory on trust and relationship loyalty is still relevant in a self-service context for a group that wish to continue a relationship orientation however this does not apply to other customers such as the transaction-oriented group. the existing literature discussed indicates the relationship-orientation of business banking. the findings in this study, however, suggest that transaction-oriented customers also exist and utilise ssts. the findings of this research have shown that this segment of transaction-oriented customers is likely to grow. relationship-oriented customers, in the past, have primarily been the focus of customer interaction by the banking industry in a b2b context. this paper has highlighted that there is a growing transaction-oriented segment. due to changes in banking as well as the further advancement of ssts, it is expected that in the future the transaction segment will grow. the future direction of the transaction-oriented customers remains unknown. evidence suggests that even the most loyal, relationship-oriented customers are increasingly becoming transaction-oriented because of the pervasive nature of the self-service technology; however the implications from this arise when a distance arises between the customer and the marketer which impacts negatively on loyalty to the relationship. while some customers still seek proximity to the marketer, this is generally related to specific business process requirements. it is therefore feasible to accept that in the future the transaction-oriented customer base will grow considerably through expectations, time constraints, and satisfaction with the technology. the relationship-oriented customer base will shrink through the perception that the bank prefers them to engage at a distance and through a lack of requirement to engage in interpersonal relationships. loyalty is threatened through this however, as switching cost perception remains high, loyalty to the relationship will still remain. the three research propositions have been explored in the research. table 7 provides an overview of whether the propositions were supported or not: australasian journal of information systems volume 18 number 3 2014 20 proposition correct for relationshiporiented customers? correct for transaction-oriented customers? proposition 1: business customers want relationships with their service providers x proposition 2: increased use of ssts reduces relationship closeness x proposition 3: an increase in ssts decreases trust and ultimately commitment x table 7: support for the research propositions the existing literature shows no consensus on the impact of technology on business relationships (lang and colgate 2003; stone and woodcock 1997; joseph 1998). some authors believe too much technology in relationships can be problematic (lang and colgate 2003), while others believe technology is of benefit to the relationship (stone and woodcock 1997; joseph 1998). this research has argued that both schools of thought are partially correct. two distinct segments of businesscustomers have emerged through the research. one is a very technologically savvy segment, who would prefer to do most, if not all, of their business activity at a distance. they do not require a relationship with a human being, but do require someone they can call on if technology fails. the other segment is more traditional and prefers to maintain an interactive interpersonal relationship with the bank providing its facilities. this is demonstrated in figure 2. figure 2 outcomes of relationship orientation, versus the outcomes of a transaction orientation the diagram above indicates the traditional relationship marketing outcomes of trust and relationship loyalty (morgan and hunt 1994) for the relationship-oriented customer. in contrast, transactionoriented customers demonstrate trust for the banking industry. satisfaction may be an outcome of the relationship; however, it is not apparent in all cases. australasian journal of information systems volume 18 number 3 2014 21 all interviewees had an established relationship with their bank prior to their use of internet banking. the potential changes to how bank client relationships are established when new business owners are establishing businesses are yet to be determined. business relationships with banks might be set up completely online in the future which will have an impact on how trust and loyalty are established. further research could be conducted with new business owners, perhaps focusing specifically on generation y business people. two key segments have been identified, there is much uncertainty as to who will embrace technology as a basis of the relationship and who will prefer a relationship based on interpersonal interaction. organisational demographics are not an indicator, although some capital constraints of the new businesses may be. further research is needed to explore these aspects in detail. novelty of the research this research makes a contribution to an area of literature that has been neglected in the literature – that is, an understanding of how the use of ssts impact on fragile and highly valuable interfirm relationships. research in services marketing is emerging with greater importance, the market is expanding, and at the same time, the use of technology is increasing. as a consequence, it is essential that there is a greater understanding of the impact of technology on relationships. this paper has provided some surprising findings: that technology is more likely to impact on relationships, trust and commitment for transaction-oriented customers. this is likely due to the lack of commitment in the relationship, and the fact that technology is perceived to be the primary relationship facilitator for this customer segment. relationship-oriented customers, in contrast, value the human aspects of the relationship, therefore the technology provides little impact on trust and commitment. further research should explore this in more depth. references bendapudi, n. & leone, r. (2003) psychological implications of customer participation in coproduction, journal of marketing, 67 14-28. berry, l. (1983) relationship marketing, emerging perspectives on services marketing, american marketing association, chicago 25-8. berry, l. (2002) relationship marketing of services perspectives from 1983 and 2000, journal of relationship marketing 1, 59-77. berry, l. & parasuraman, a. (1991) marketing services: competing through quality, free press, new york. bitner, m. j., ostrom, a. l. & meuter, m. l. (2002) implementing successful self-service technologies, academy of management executive, academy of management 16, 96-119. carson, d., gilmore, a., perry, c. & gronhaug, k. (2001) qualitative research methods, sage, london. duncan, r. (1979) qualitative research methods in strategic management, strategic management: a new view of business policy and planning, little brown, boston. flick, u. (2002) an introduction to qualitative research, sage publications ltd. wiltshire. flick, u. (2009) an introduction to qualitative research, sage publications ltd., wiltshire. ford, d. (1990) understanding business markets: interaction, relationships and networks, academic press, london. gallivan, m. (2001) organisational adoption and assimilation of complex technological innovations: development and application of a new framework, acm sigmis database, 32, 85. glaser, b. (1992) basics of grounded theory analysis: emergence vs forcing, sociology press, mill valley. australasian journal of information systems volume 18 number 3 2014 22 glaser, b. & strauss, a. (1967) the discovery of grounded theory: strategies for qualitative research, aldine. grimm, m. (1999) ford connects: hatches massive relationship program via web, brandweek, 40 1920. grönroos, c. (2004) the relationship marketing process: communication, interaction, dialogue, value, journal of business & industrial marketing 19, 99-113. grönroos, c. (1995) relationship marketing: the strategy continuum, journal of the academy of marketing science 23 252-254. grönroos, c. (1994) from scientific management to service management, international journal of service industry management, 5, 5-20. gummesson, e. (2008) total relationship marketing, butterworth-heinemann, oxford. hunt, s., lambe, c. & wittmann, c. (2002) a theory and model of business alliance success, journal of relationship marketing, 1, 17-35. johns, r. & perrott, b. (2008) the impact of internet banking on business-customer relationships (are you being self-served?) international journal of bank marketing 26, 465-482. johns, r. (2008) towards an empirical understanding of service dominant logic, anzmac conference proceedings, university of western sydney. johns, r., blackman, d. & low, d. (2009) service dominant logic: is it simply repositioning relationship marketing? anzam conference proceedings. melbourne. joseph, r. (1998) customer orientation and the creation of new technology based services: electronic media and executive education, prentice hall, london. kandampully, j. (2003) b2b relationships and networks in the internet age, management decision, 41, 443-451. keller, k. (2001) building customer-based brand equity: a blueprint for creating strong brands, marketing science institute, cambridge massachusetts. lang, b. & colgate, m. (2003) relationship quality, on-line banking and the information technology gap, international journal of bank marketing 29, 37. leavy, b. (1994) the craft of case-based qualitative research, irish business and administrative research 15 105-18. leonard-barton, d. & deschamps, i. (1988) managerial influence in the implementation of new technology, management science, 34 1252-1265. lovelock, c. (1991) services marketing, prentice-hall international inc, new jersey. marr, n. & prendergast, g. (1993) consumer adoption of self-service technologies in retail banking: is expert opinion supported by consumer research? international journal of bank marketing 11, 3-10. meuter, m., bitner, m., ostrom, a. & brown, s. (2005) choosing among alternative service delivery modes: an investigation of customer trial of self-service technologies, journal of marketing, american marketing association, 69, 61-83. meuter, m. & bitner, m. (1997) the new service encounter: customer usage and satisfaction with self-service technologies, journal of marketing, 64, 50-64. morgan, r. & hunt, s. (1994) the commitment-trust theory of relationship marketing, the journal of marketing, 20-38. moisander, j. & valtonen, a. (2006) qualitative marketing research: a cultural approach, sage publications ltd, london. australasian journal of information systems volume 18 number 3 2014 23 pattinson, h. (2005) mapping marketing decision space: an exploration of emerging cognitive mapping tools for analysis of marketing scenarios and decision-making, anzmac conference proceedings. pujari, d. (2004) self-service with a smile? international journal of service industry management 15 200-219. ratnasingam, p. & pavlou, p. (2003) technology trust in internet-based interorganisational electronic commerce, journal of electronic commerce in organisations 1, jan-march 2003 17-41. reichheld, f. (1993) loyalty-based management, harvard business review. spiggle, s. (1994) analysis and interpretation of qualitative data in consumer research, journal of consumer research 21, 491-503. stone, m. & woodcock, n. (1997) winning new customers in financial services using relationship marketing and information technology in consumer financial services, trans-atlantic publishing, london. storper, m. & venables, a. (2004) buzz: face-to-face contact and the urban economy, journal of economic geography, 4, 351. strauss, a. & corbin, j. (1990) basics of qualitative research: grounded theory procedures and techniques, sage publications newbury park, ca. tax, s., brown, s. & chandrashekaran, m. (1998) customer evaluations of service complaint experiences: implications for relationship marketing, the journal of marketing, 62, 60-76. thomson, g. (2008) taxonomy of selected relationship marketing theories, online, available at: http://works.bepress.com/grace_thomson/22 vargo, s. & lusch, r. (2004) evolving to a new dominant logic for marketing, journal of marketing, 68 1-17. zaltman, g., duncan, r. & holbeck, j. (1973) innovations and organisations, new york. john wiley & sons. australasian journal of information systems volume 18 number 3 2014 24 appendix one: comparison of use of thematic analysis; nvivo and leximancer method features comments thematic analysis judgement by the researcher involved in all aspects researcher gets very familiar with the data qualitative research requires an interpretative stance by the researcher, making this an appropriate tool of analysis. researcher comes to a conclusion about the meaning nvivo storage of documents into project sets ranking of search-drive words coding of works, phrases, sentences as nodes coding across several documents mapping of coded nodes researcher judgement required for coding and map development developed out of nud*ist powerful analytical after coding completed leximancer storage of documents into project sets integrated package with word rankings, summarisation of documents, thesaurus features to enhance theme/ concept sources. automatically draws maps with links of association between theme and concepts can be set to purely automatic based on sets of bayesian conditional algorithms with capability to factor in wider searches based using thesaurus options powerful range of mapping capabilities very good pre-processer for netmap source: from flick‘s (2002) discussion on thematic analysis and pattinson’s comparison of leximancer and nvivo (2005) australasian journal of information systems volume 18 number 3 2014 25 appendix two: sample of quotations from the interviews respondent quote comment decision maker vs user r19, decision maker if i had of walked into the organisation and (my bank) were the bankers, i’d have no specific loyalty to them, but my loyalty comes from a point where we were at a real growth stage and a pivotal point for the business... that’s built a fair bit of loyalty.... at least while i’m here. i think it’d change if i left, someone else came in, because they’d just be like “oh it’s a functional relationship, just like any other business relationship”. so although you’re satisfied, you’re not necessarily loyal and that would happen if i were to walk into another business as well. there is a difference is trust for the organisation, based on the role within the customer organisation. for instance, decision makers were more likely to express a higher degree of trust and satisfaction than users were. this quote was representative of the views of all respondents. role of the relationship banker r 18, decision maker it’s 100% the little things. you don’t call on them for support for internet banking or something like that.... you only ever use it for the important stuff. you don’t use it to save $8 on a bank cheque this respondent reported demonstrated a belief that there was an advantage to a full relationship and acknowledged differences between transactions requiring interpersonal contact and transactions that did not. this was representative of the views of all relationship-oriented customers switching service providers r19, decision maker if you go through bank 5, although the relationship manager’s a good guy, he just had no authority. every time we’d have a meeting, he’d just go “oh, look, i’ll have to come back to you”.... (now) i could ring our relationship manager and say “we’ve got a proposal to spend $500 000... i’ll send through a business plan on it and she’ll come back within 24 hours with any questions and an approval. with bank 5, that’d have to go to sydney, they’d do all the ratio testing.... it shows a lack of understanding for a business like ours. this respondent explains why they switched banks. part of their reason for this switch was the limited approval authority of the bank manager and requirement for a greater understanding of their business it is evident through this statement that the respondent wanted to be clear that the relationship manager at the first bank was a ‘good guy’ and he stressed this several times throughout the interview. he stated that the understanding at the second bank was essential to their business banking the importance of face-toface contact r19, decision maker i think an understanding of the business (is essential) and that’s where it differs from someone on the end of the phone. there are certain elements of our business which are very different from others for some customers, it was important to have face to face contact, as indicated in the quote included. australasian journal of information systems volume 18 number 3 2014 26 relationships with the relationship banker r11, decision maker the human being is very important, but the bankers have got to the point that they change them every year. the customer is always the same and i think what happens is they change them purely because of promotions... bank 5, probably about five years ago... as soon as they felt your relationship banker was getting too close to you, they moved them because it concerned them... conflict of interest kind of thing, so it was like an oxymoron to the relationship banker. that policy is long changed at the b5. but that’s how they were, probably only about five years ago. b8, now... you get a new relationship banker, it’s mainly due to promotions. the training aspect of their next one is well and truly aligned with “well now this is the customer... you have to treat them really, really well”. so when they do move on, the next one is well and truly trained in terms of the importance of... customers i suppose this quote indicates a former relationship manager policy help by a bank. this was counterintuitive to the establishment and strengthening of the relationship the lack of face to face contact r2, decision maker they have business banking centres, but they’re not really set up for transactional banking from a business. you still have to go into a branch and stand in queue with everyone else who could be there for god knows what reason. the quote demonstrates that business customers are not given any priority over personal customers in a branch, despite the promises of ‘business banking centres’ which are meant to offer customers value added services. this impacted on customer satisfaction. the importance of face to face contact r19, decision maker in the early days, (our relationship banker) would just come out here. she’d come out here 5 o’clock, 6 o’clock at night, sit down here, have a chat, work through what we’re doing, that kind of thing. that built a lot of loyalty between the company and the bank.... just that at the time when the business needed the bank, they were there for us a unique business structure increased the need for organisations to have faceto-face contact with their business partner. this was a common perception, with a number of relationship-oriented customers indicating a need to explain their business structure carefully to their bank, as it impacted on their use of bank servies. australasian journal of information systems volume 18 number 3 2014 27 servicing a customer segment r8, decision maker customers like me at present are so insignificant (in numbers) that catering for us .... they’re not going to bother, but as soon as that shifts toward people like me, the demand is going to be huge. ... they’re paying people... they could pay less staff, make more money and therefore serve us better.... that’s a win win, for both the customer and the bank. they make more money and i get better service. how does anyone lose? and there’s a first mover advantage for anyone who picks up this field this customer explains the importance of banks fulfilling the needs of a “niche” customer segment. he explains that ‘customers like me’ were not completely based on his generation, but rather about the way certain people embrace it. this was a common perception among business customers with a transactionfocus customers embracing technology r8, decision maker it’s willingness to accept the technology... embrace it, really. it’s being comfortable with (the technology)... and not being scared of it. i don’t think you need to be savvy to actually get to that level. i think people can be comfortable and go “yes i trust it” without necessarily understanding it. now i understand it as well but i don’t think that’s a ... component necessarily of trusting and accepting it this respondent discusses a category of customer that prefers to transactonly, through technology importance of the interpersonal relationship r19, decision maker the business is in a very strong position now, but that’s only because of the support of this group of people back when we were in the building phase. that’s where this real loyalty has come from, i’d be very hesitant to change accountants, hesitant to change lawyers, bankers. these are people we’ve worked with and formed a relationship with one relationship-oriented customer stated that their relationship was due to an individual within the bank. this was a common perception between relationship-oriented customers paper title (paper title style) australasian journal of information systems burgess, bingley, paguio & woodley 2018, vol 22, research article interventions in entrepreneurial learning 1 interventions in entrepreneurial learning: blended learning and ict capacity stephen burgess centre for applied informatics victoria university stephen.burgess@vu.edu.au scott bingley college of business victoria university rafael paguio college of business victoria university carolyn woodley school of policing studies charles sturt university abstract the effective use of information and communications technologies (ict) can benefit most small businesses. this article examines the design, development and delivery of a pilot blended learning program for entrepreneurial learning that targets the use of ict by tradespeople. the study involves a case study of tradespeople in australia who participated in a program designed to build their capacity to identify, use and evaluate ict to enhance their business practices. the program combined traditional instructional approaches (through faceto-face and online sessions for delivering basic ict concepts to participants) with learnercentred approaches (through a combination of student mentor visits and practical exercises to identify applications that were context-specific for each participant's business). a blended learning assessment framework is employed throughout the article to inform and evaluate various aspects of the program. project participants rated the face-to-face sessions, student mentor visits and practical exercises as very effective. however, some participants could not free up the time from their employment to participate in all of the face-to-face classes. from a theoretical perspective, an amended blended learning assessment framework for the delivery of ict programs to entrepreneurs is presented and potential lessons for the delivery of ict concepts to entrepreneurs in the trades and similar vocations are proposed. keywords: blended learning; entrepreneurial learning; ict use; instructional approach; leaner-centred approach. 1 introduction the simple definition of ‘entrepreneur’ is that relates to an individual who commences or owns a business (either self-employed or as owner-manager) with the aim of expansion (van praag and versloot, 2007). however, an extended viewpoint contends that entrepreneurs tend to take more risks and be more innovative than regular small business owner-managers (fernandes, 2016). entrepreneurial learning refers to the learning process of entrepreneurs and the impact that this has on their own personal development and that of their businesses (cope australasian journal of information systems burgess, bingley, paguio & woodley 2018, vol 22, research article interventions in entrepreneurial learning 2 and watts, 2000). a search of the literature on entrepreneurial learning reveals two major themes: learning by the entrepreneur and learning by the organisation (erdélyi, 2010). this article addresses learning by entrepreneurs, particularly from the viewpoint of proactive behaviour by tradespeople to participate in a blended learning program developed to build their capacity to identify, use and evaluate ict to enhance their business practices. it is important to note that this article discusses the use of ict for two distinct purposes: 1. the use of ict by entrepreneurs to improve their businesses (which is the focus of the blended learning training program) 2. the use of ict by instructors to deliver the blended learning program. using a blended learning assessment framework (wong, tatnall and burgess, 2014) as a lens of analysis, the study examines the preparedness of instructors and tradespeople for the blended learning program, the choice of blended learning options provided for the tradespeople and the impact of the program. an updated version of the blended learning framework that is specifically tailored towards the training of entrepreneurs in the trades and similar vocations is then proposed. the article begins by discussing entrepreneurial learning in the context of tradespeople and moves on to discuss how blended learning could support the use of ict by tradespeople. the blended learning assessment framework is introduced as a means of outlining the different stages involved in implementing a blended learning program. a case study involving a blended learning program that was developed and delivered to enhance the ict capacity of tradespeople as enterprising business managers is then presented. 2 literature review this literature review comprises two main sections. the first relates to the notion of entrepreneurial learning, tradespeople as entrepreneurs and how they use ict. the second section relates to the use of ict (through blended learning) for delivering training to tradespeople. 2.1 entrepreneurial learning – the use of ict by tradespeople the theoretical approaches to describing entrepreneurial learning are divided into two major ‘camps’ – those that focus on the entrepreneur (and associated personal learning experiences and cognitive capabilities) and those that focus on the organisation (how learning occurs collectively within the business and via its external networks) (erdélyi, 2010). these approaches are not mutually exclusive. literature focussing on the entrepreneur is most prominent and broadly fits two categories (erdélyi, 2010): • entrepreneurial learning as a type of management learning which results in an entrepreneur who specialises in setting up new ventures; • entrepreneurial learning that occurs at different stages of the entrepreneur’s career. here, entrepreneurship is a behaviour that is learned through experience. erdélyi (2010) suggests that this area of the literature is the most developed of the two. entrepreneurial learning can occur in several ways. one approach is through training (such as executive and/or management training) that can take place in various locations such as universities, government agencies, large businesses, the small business workplace (erdélyi, 2010) and, more recently, online. another type of learning occurs through unplanned crises or australasian journal of information systems burgess, bingley, paguio & woodley 2018, vol 22, research article interventions in entrepreneurial learning 3 chance occurrences in the business (pittaway and thorpe, 2012) through which entrepreneurs adapt their behaviour to survive and learn from the experience. in this case, the learning process occurs through finding answers to problems that are encountered (fayolle and gailly, 2008). this article is concerned with planned learning by entrepreneurs who are running small businesses. for the purposes of this article a small business is considered to have between one (sole trader) to 20 regular employees (burgess et al., 2009) 2.1.1 how entrepreneurial businesses can use ict ict is “the use of computers and other electronic equipment and systems to collect, store, use, and send data electronically” (cambridge university press, 2013a, para. 1). effective ict use can benefit most small businesses in many ways, such as adding value to good and services, reducing the level of involvement of intermediaries, reducing the cost of searching for information, reducing the cost of conducting transactions and promoting external networks (martin and wright, 2005). however, small businesses that use ict need to have the skills to do so effectively. in these businesses, the responsibility of learning how to use ict effectively typically falls upon owner-managers as it is less likely that they will be able to recruit staff with ict skills into the business. 2.1.2 tradespeople and ict use niblock (2013, p.2) refers to a tradesperson as “a skilled manual worker whose status, economically and socially, is considered somewhere between a labourer and a professional. examples of tradespeople might be welders or electricians. the practitioners possess a high degree of both practical and theoretical knowledge of their trade”. training for such entrepreneurs has traditionally been the domain of vocational education, where the focus is on preparation for specific work in non-professional jobs and developing the skills needed to carry out specific tasks (kirkup, 2010). knowledge in the trades area might be predominantly described as procedural knowledge (biggs and tang, 2011) or knowing how to do things. in australia, the national training system for vocational educational training is responsible for delivering ‘workplace specific’ skills. the use of ict in small businesses is also influenced by the ‘digital divide’. this refers to the gap which separates those who have access to ict from those who do not. srinuan and bohlin’s (2011) literature review of digital divide research found that early researchers examined physical access to technology as the main cause of the digital divide. more recently, factors such as socio-economic status and ict knowledge and skills were also found to play a role in the adoption and effective use of ict. thus, the divide occurs where there are businesses who have both access (to the technology) and the skills to use it effectively and those who do not. it is important for entrepreneurs to possess both physical access to ict and the knowhow to use it effectively. many trades businesses (certainly those examined as part of this study) are either sole traders or have very few employees. as an example, in australia the construction sector (incorporating builders, electricians, plumbers and so forth) has the third highest proportion of small businesses of 20 industry sectors listed by the australian bureau of statistics (2012). sole trader businesses especially are often more interested in survival than growth and tend to lack a longer term strategic focus (jones et al., 2014). their primary motivation for existence is usually to retain autonomy and independence in operation. it may even be difficult to consider owner/managers of such businesses as entrepreneurs. however, there are some owner/managers who may be considered as being energetic and innovative (gray and australasian journal of information systems burgess, bingley, paguio & woodley 2018, vol 22, research article interventions in entrepreneurial learning 4 gonsalves, 2002). if they are motivated to learn, interested in different topics, ask excellent questions and so forth then such owner/managers exhibit the characteristics of entrepreneurs (löbler, 2006). the tradespeople participating in the study described in this article are referred to in this article as entrepreneurs as they exhibited an innovative attitude towards improving themselves and their business through the learning and adoption of ict tools and importantly, had taken the initiative to participate in the course. as mentioned earlier, this article examines a specific, planned instance of entrepreneurial learning. as such, it does not focus on learning that occurs through unplanned crises or chance occurrences (or critical incidents as described by cope and watts, 2000) – but rather through planned activities, such as training. when considering educating entrepreneurs about ict use, walker (2001) examines how one training program to improve ict skills was designed to ‘cascade’ from delivering generic (basic) ict skills (required by most participants) to specialised training that is context-specific to certain trades. in walker’s example, early skills provided are predominantly technical (for instance, basic internet use) to later ict use that is more contextual (how the skills could be applied in the varied conditions of different trades businesses). this scaffolding suggests an initial need to understand basic ict concepts that could then be extended to the use of ict to support work activities. 2.2 blended learning – ict as a tool for delivering training this section of the literature moves from how ict can be used by entrepreneurs in their business to how it can be used to deliver training for entrepreneurs. as part of a planned approach to entrepreneurship education, fayolle and gailly (2008) proposed a teaching model framework. at the ontological level, the framework examined such concepts as what education means in the context of entrepreneurship and outlines the roles of educators and participants. this article is concerned with the educational level, where considerations relate to identifying the target audience for the education (for whom?), the educational objectives (why?), program content (what?), the methods and pedagogies adopted (how?) and the evaluations of effectiveness (for which results?). perren (2003, p.517) suggested that “there is a growing consensus that entrepreneurs require flexible, learner-centred and informal mechanisms of support”. harrison and leitch (2005) discussed the important role that ict can play in such learning. even participating in training programs that use ict to assist in their delivery can raise the ict skill level of participants (elliott and clayton, 2007) and provide access to training for those whose work conditions and/or location make it difficult to attend face-to-face training (walker, 2001). using fayolle and gailly’s teaching model framework, the areas of interest of this article at the educational level are: • for whom: tradespeople • why: to assist them to use ict effectively within their businesses • what: concepts related to effective ict use within the business • how: blended learning training program (see following discussion) • for which results: effective use of ict in the businesses. australasian journal of information systems burgess, bingley, paguio & woodley 2018, vol 22, research article interventions in entrepreneurial learning 5 one term employed to represent combinations of ict-based and face-to-face learning is blended learning. bliuc, goodyear and ellis (2007) described blended learning as a mix of different methods, including face-to-face and online teaching, the latter often referred to as ‘elearning’ (wong et al., 2014). the use of ict to assist learning has been a focus within the information systems discipline. for instance, overby, slaughter and konsynski (2010) examined the role that virtual environments can play in numerous disciplines, including education, and discuss the important of developing theories and frameworks to assess their implications for society. gasson and waters (2013, p.95) examined the engagement of learners in online courses and found “a dearth of useful theory about the strategies employed by learners to manage their engagement with the course and about the effect that learner strategies have on course outcomes”. wong et al. (2014) introduced a ‘blended learning assessment’ (bla) framework to consider all aspects of blended learning (refer figure 1). figure 1: blended learning assessment (bla) framework: tailored for tradespeople (amended from wong, et al. 2014) the wong et al. (2014) framework was initially developed for use by higher education institutions and was meant to provide an overview of the blended learning experience with regards to readiness, intensity of adoption and impact. in figure 1, the terms ‘training provider’, ‘trainer’ and ‘trainee’ have replaced the higher education terms ‘higher education institution’, ‘staff’ and ‘student’ to reflect the emphasis on industry training in this article as opposed to higher education course delivery. the bla framework discusses three aspects that should be considered when delivering a blended learning program: readiness – of the training provider, trainers and trainees to adopt the blended learning program; intensity of adoption – the suite of blended learning options implemented and the selection of those options by students; and, impact – assessing the quality and extent of learning achieved. the bla framework has obvious overlap with fayolle and gailly’s (2008) teaching model framework where the learner readiness relates to ‘for whom?’ in the teaching framework, australasian journal of information systems burgess, bingley, paguio & woodley 2018, vol 22, research article interventions in entrepreneurial learning 6 selection of blended learning options relates to the ‘what?’ and ‘how?’ aspects of the teaching framework and impact relates to the ‘for which results?’ aspect of the teaching framework. the following sections discuss each aspect of the bla framework. 2.2.1 readiness readiness is a term that has been employed to determine how ‘ready’ a business is to adopt ict. parker (2000) described ereadiness as how prepared a business is to operate in an ebusiness marketplace. similarly, machado (2007) developed a framework for assessing ereadiness in higher education institutions. machado’s framework differentiates between the roles of administrators, instructors and students. this view of readiness was adopted by wong et al. (2014) in their bla framework. for them, readiness incorporates factors such as the availability of infrastructure and skilled workers, finance to introduce blended learning initiatives, the willingness to use ict within the organisation and the attitudes of senior management. from an organisational viewpoint (training provider readiness) it is important that the ict infrastructure used to support blended learning is in place (elliot and clayton, 2007). the institution needs to achieve ‘buy in’ from staff (who will be trainers) and provide suitable training for designers and users of blended learning where they are lacking in relevant skills (valentine, 2011). from the viewpoint of the trades entrepreneur, readiness refers to: their level of time commitment, basic technical proficiency, confidence and interest in the blended learning approach. additionally, participants must have access to the technology being used in the training program and adequate support in how to use it (valentine, 2011). 2.2.2 intensity of adoption for wong et al. (2014), intensity of adoption relates to the selections made by instructors when deciding what features to incorporate into their blended learning package and the subsequent choices made by learners as to which of these features to use. blended learning options wong et al. (2014) suggest that different approaches to delivery could be assembled as part of an overall blended learning package. these different approaches could be classified according to the level of ict support that they rely upon, using a continuum of blended learning that the authors extended from the work of jones, et al. (2009). the stages identified in the continuum range from face-to-face (no ict support) through to e-intensive (complete online delivery). in between lies a mix of face-to-face and online delivery, such as that which occurred in the course that is the focus of this article. approaches to teaching can also vary from an instructional to a more learner-centred approach. a learner-centred teaching approach requires learners to be more involved in determining their own learning experience and sometimes even negotiating how this might occur. such experiences can occur if learning activities are structured to pose questions or problems, and then students search for solutions themselves and assess the relevance of any information they have found in their own context (emes & cleveland-innes, 2003). this fits well with a blended learning environment where the participants select which program delivery features to use. with a traditional (instructional) approach, the teacher acts as ‘expert’, sourcing or developing and delivering resources and making judgements regarding content and assessment. löbler (2006) refers to this as the ‘transmission’ approach. the role of the teacher is that of a transmitter australasian journal of information systems burgess, bingley, paguio & woodley 2018, vol 22, research article interventions in entrepreneurial learning 7 of content and the role of the learner is that of a passive consumer. the predetermined curriculum forms the basis of transferring knowledge to the learner. interaction occurs between instructor and learners. typical activities include listening (to instructors), reading (assigned materials) and memorizing. when discussing the notion of education in the context of entrepreneurship, fayolle and gailly (2008) distinguished between teaching (such as imparting a knowledge or skill; instructing; conditioning an action) and educating (for instance, developing capacity; stimulating mental growth and so forth). the authors suggest that, since entrepreneurship refers to individual initiative, creativity and even innovation, the notion of educating seems more appropriate to entrepreneurs than teaching. this implies that entrepreneurs would be well-suited to a learner-centred approach. selection of options the selection of options by trainees can be affected by the approach to delivery. with a learnercentred approach, students are likely to have more freedom to select blended learning options that suit their own situation and search for their own solutions to problems raised within the course (emes & cleveland-innes, 2003). an instructional approach is more likely to prescribe the use of blended learning options (löbler, 2006). learner-centred approaches do not necessarily preclude the use of instructional approaches, but learners can choose from a set of learning elements that allow them to consider the instruction in their own context and transform the instruction into learning (de georgewalker & keeffe, 2010). typical activities in this approach involve experiential, ‘open-ended’ learning activities, where “neither students nor the instructor has prior knowledge of all of the results” (emes & cleveland-innes, 2003, p.60). for entrepreneurs, this can create an environment that supports “autonomy, self-reliance, independent thinking and in general the ability of self-governing” (löbler, 2006, p.31). such active learning is highly suited to an environment where the aim is to apply general knowledge gained in a specific context (theory to practice). from an entrepreneurial learning viewpoint, a learning-centred approach provides the type of environment that encourages double loop learning (gradual change in behaviour over time) rather than single loop learning, which relates to how to solve a specific problem or instance (pittaway and thorpe, 2012). other intensity of adoption issues the other key aspect of blended learning, not emphasised in wong et al’s (2014) bla framework, is the need for the ict infrastructure to remain operational throughout the delivery of the program (elliot and clayton, 2007). strategies need to be put in place to support the use of the technology during this time (valentine, 2011), which would include dealing with situations when all or part of the technology is not operational (either due to technical fault or lack of user skills). 2.2.3 impact impact can be assessed in several ways – such as the level of usage of different features, learner satisfaction levels and the effect on overall performance (wong et al., 2014). evaluation is also a key aspect of fayolle and gailly’s (2008) teaching framework as evaluation criteria are decided at the time of designing the program. the assessment of the impact of blended learning initiatives allows the effect of the separate blended learning options on australasian journal of information systems burgess, bingley, paguio & woodley 2018, vol 22, research article interventions in entrepreneurial learning 8 students to be determined. the different options can be assessed with regards to the aims of the program, the content and the approach taken by the deliverers. for instance, mullin (2013) examines the use of online learning to assist in the delivery of courses of varying lengths to hairdressing apprentices in england. impact was assessed through questions asked during separate focus groups of trainers and apprentices. it was generally regarded that the program made a positive contribution to learning. it was “fun and motivating” (mullin, 2013, pp.74-75) and had the potential to speed up course completion. an assessment of the success of a blended learning approach should include reference to any improvements in accessibility that the learners have to the various aspects of the program due to the nature of its delivery. for instance, one of the advantages that online learning can provide over traditional face-to-face delivery for organisations is increased flexibility in training options due to: flexibility in time (delivery at a time that suits); flexibility of place (selection of design and delivery options); and, flexibility of delivery (to varied numbers of learners concurrently) (elliott and clayton, 2007). this is particularly suited to tradespeople who are often time poor and ‘practically minded’ learners who prefer variety regarding the methods used to train them. a blended learning approach to trades training may be preferable (bruce, 2012). fayolle and gailly (2008, p.579) refer to the possibilities offered by ict: indeed in entrepreneurship as in almost all disciplines, the growing use of ict … tend[s] to develop and offer to a certain extent more autonomous ways of learning, outside of the classroom and far from the more traditional and dependent methods of learning. 3 research methodology this project was conducted at victoria university (an australian metropolitan university) and was funded by the telematics trust with the aim of developing ‘enabled tradie’, a blended learning program to support tradespeople in the adoption of ict tools appropriate to their businesses. 3.1 research questions this article adopts wong et al.’s (2014) blended learning framework as a lens to describe a research project examining a case study involving the use of blended learning to encourage entrepreneurial learning through building ict capacity. this also provided an opportunity to test the framework for its effectiveness outside of the environment in which it was developed, undergraduate higher education. in doing this, the researchers considered the following research questions (developed around sections of the framework): • what was the preparedness of the university, instructors and tradespeople for the blended learning program? (readiness) • what combination of blended learning options was provided for the tradespeople (intensity of adoption)? (jones et al., 2009) • how was the quality and extent of learning (impact) assessed for the program? • what were the advantages of blended learning delivery for participating tradespeople? (elliott and clayton, 2007; mullin, 2013). australasian journal of information systems burgess, bingley, paguio & woodley 2018, vol 22, research article interventions in entrepreneurial learning 9 • was the blended learning program able to provide suitable skills to tradespeople about basic and context relevant ict use? (walker, 2001). the article concludes by proposing a version of the blended learning framework that is specifically tailored for training entrepreneurs. 3.2 study approach this study involves an interpretivist case study of six tradespeople who were the owner/managers of their businesses and who were interested in building their ict awareness. as an interpretivist study, the aim is to develop the blended learning framework for these and similar entrepreneurs. the amended framework could then be tested and validated in later studies. as such, the research design is based on inductive reasoning and the research questions are interpreted in a more flexible manner than would occur in a positivist study (williamson, 2013) to build theory rather than test it. they not only investigate the blended learning program under review, but they also help to guide the development of the revised blended learning framework for entrepreneurs in trades and similar vocational areas. cope and watts (2000) conducted a case study involving six small business owners when investigating the role of critical incidents in entrepreneurial learning. their study confirmed the case study as an appropriate methodology to design learning programs, an approach also used by de george-walker and keefe (2010) in the design of a learner-centred blended learning program for undergraduate students. a case study allows for a phenomenon to be examined in its real-life context and typically has multiple sources of evidence and data collection techniques, which were evident here. it is also common in a case study approach for the research questions to have an applied orientation aimed at improving practice (darke and shanks, 2002). as indicated earlier in the literature review, improved practice is vital for effective entrepreneurial learning. in this instance, the unit of analysis (yin, 2003) was individual participants and, by implication, the businesses that they operated. another key approach used in this study was action research, where the researchers are involved in the topic and are part of the process of bringing about change. this occurs in a setting that is not ‘detached’ as would occur in more traditional research approaches (veal, 2005). action research involves an approach that is critically reflective. when an action occurs, its effect is assessed and modifications to the research approach are made if needed. a cycle of plan – action – results – reflection – plan is typical of the action research process (oosthuizen, 2002). as such, it is often difficult to plan ahead of time how a particular action research cycle will pan out. it is important to define and explain the role of the researcher in action research and, thus, the approaches to the evaluation of the research need to be carefully considered (oosthuizen, 2002). the specific role of the researchers in this project is described in section 3.4. additionally, the results and discussion sections outline how the researchers reflected upon (and altered) the modes of delivery and the content of the course as it was being delivered. 3.3 participant selection it was hoped to source 10-12 participants in a small number of trades occupations as a purposive sample to run the blended learning program as a trial of a larger roll-out of the program at a later date. general invitations for participation were sent out via a local government mailing list and the victoria university ‘global’ mailing list, asking for contacts to potential participants. this resulted in nine responses from tradespeople who were provided australasian journal of information systems burgess, bingley, paguio & woodley 2018, vol 22, research article interventions in entrepreneurial learning 10 with further details about the proposed program. of these, six tradespeople agreed to participate, with the non-participants explaining that the time requirements of the course were too demanding or that the course was too basic for their needs. the final grouping of tradespeople participating in the program was as follows: a handyman, an electrician (who had to drop out half way through the program due to work pressures); a fitter and turner who had moved into specialist maintenance; two hairdressers and a café proprietor who also provided external catering services (who had heard about the program and asked to participate.). the participants were all owner/ managers. ‘pre-participation’ surveys (to be explained further in section 3.5) suggested that they were keen to develop their businesses into sustainable and competitive small enterprises. it is reasonable to assume that the fact that the participants were proactive and motivated enough to engage with the program indicated that they were energetic and innovative fundamental entrepreneurial characteristics as described by gray and gonsalves (2002). however, the small number of participants means that it would be difficult to state that all small businesses would be suited to this form of study program. however, the study is well suited to identifying the main components of the blended learning framework for entrepreneurs in the trades and similar vocations. 3.4 design of the program a unique aspect of the program is that student mentors were introduced to support the learning process. the mentoring was provided by employing victoria university business students (selected, trained and managed by the research team). the aim was that mentors would conduct follow up sessions (face-to-face and via asynchronous technologies) with participants to assist in the initiation and integration of ict adoption. mentors are considered to provide the following benefits to the entrepreneur (perren, 2003): • one-to-one relationship; • the mentor is more experienced than the entrepreneur (in the relevant subject matter). in our program, it would be more accurate to say that the student mentors were more knowledgeable than the entrepreneurs in the subject matter.; • the focus is on growth and development of the individual, and • a confidential and protected relationship exists between mentor and mentee. the authors of this article (‘the researchers’) were involved in the design and delivery of the blended learning program. two student mentors were employed to liaise with the businesses and assist them to tailor aspects of the course material to their specific businesses (context). each student was assigned three businesses to mentor. whilst it was understood that the students were not experienced business practitioners, they were very well-performed in a range of business units and could use technologies likely to be trialled by the businesses. it was expected that they would also benefit from having exposure to different businesses and the ability to add the experience to their curricula vitae. the mentors were themselves mentored by the researchers with regards to how to interact with the businesses and when to offer advice. they were encouraged to seek advice from the researchers throughout the program and to report regularly on their progress (as described later). australasian journal of information systems burgess, bingley, paguio & woodley 2018, vol 22, research article interventions in entrepreneurial learning 11 the program ran for 11 weeks in early 2013. face-to-face sessions were initially held at the victoria university campus where the researchers were based. use of technology for program delivery was limited to that available within the university at the time. in the first session, participants were shown how to access the university network and to access the website of the blended learning program, which was set up using the learning management system, blackboard. figure 2 shows the opening blackboard screen for the program. the blackboard interface used by victoria university could not be described as ‘mobile friendly’, although it could be access by smartphones and tablets. figure 2: opening screen of the 'enabled tradie' blended learning program blackboard website. it was decided that the program should commence with some face-to-face sessions so that the participants could meet with the program instructors and mentors. these were held in professional development rooms of victoria university. these rooms combine computer projection facilities (via large screen televisions situated around the room) and computers on desktops. they provide a different experience from the traditional lecture room, tutorial room or computer laboratory. refer to figure 3 for an image of the professional development room used in the program. figure 3: professional development room used for face-to-face sessions returning to the program design, the aims of the early face-to-face sessions were to provide the opportunity for participants, instructors and mentors to meet each other and to familiarise themselves with the requirements of the program. a final face-to-face session was conducted australasian journal of information systems burgess, bingley, paguio & woodley 2018, vol 22, research article interventions in entrepreneurial learning 12 at the end of the program. afternoon tea was provided in all face-to-face sessions as an ‘ice breaker’. however, the researchers understood the challenges faced by participants regarding attending face-to-face sessions during regular work hours. it was difficult to find a common time where all participants were able to attend the sessions (a challenge noted by walker, 2001). not all of the participants were able to attend the two face-to-face sessions at the beginning of the program. in these instances, the mentors provided an overview of the session when they met up with participants during their business visits. after two introductory face-to-face sessions, a series of four online sessions was arranged. although the sessions required the participants to be available in real time, their travel time was significantly reduced by not having to attend the university campus. these sessions used an online conferencing tool linked to blackboard collaborate that allowed a presenter to deliver ‘live’ audio while microsoft powerpoint slides were viewed by participants. participants were provided with headsets (with microphones) and were assisted with setting these up on their home computers by their mentors. an option was available for the software to be used on tablets, but this was not selected by any of the participants. the online tool allowed them to ask questions via the microphone, comment via a ‘chat’ facility, participate in voting sessions and indicate certain ‘emotions’ (through ‘like’ and ‘smiley face’ emoticons). the sessions were generally well ‘attended’ and were recorded for participants to view later on the few occasions that they were not able to participate. participants who could not participate cited work reasons for their non-participation. a key aspect of the program was the interface between the researchers (as deliverers of the program), the mentors and the participants. vital to this was the relationship between the mentors and their associated businesses. these relationships were key to the ability of participants to be able to take the generic concepts presented in the face-to-face and online sessions and apply them to their specific businesses. the interaction between the mentors and their businesses was strong throughout the program. other blended learning activities were introduced as support activities for the program. online support materials were provided to allow participants the opportunity to access further resources about specific topics that were covered in the sessions, or find out further information about topics of interest. these resources were used sparingly by some participants and more enthusiastically by others. online tests were provided as revision for the face-to-face and online sessions. these were not used to any great extent and were thus discontinued during the program. regular practical exercises were provided to encourage participants to work by themselves or with their mentors to determine how generic concepts could be applied to their businesses. these exercises were embraced by the participants. 3.5 data collection and analysis data collection for the research occurred in several ways alongside (and after) the delivery of the blended learning program, as follows: • participants were requested to complete surveys in the first week of the program (‘preparticipation’ survey) and the last week of the program (‘post-participation’ survey). there was also a follow-up interview conducted by one of the authors with each participant some 12 months after the completion of the course. the surveys contained mostly open-ended questions. the initial survey helped to formulate the content of the australasian journal of information systems burgess, bingley, paguio & woodley 2018, vol 22, research article interventions in entrepreneurial learning 13 course, whilst the other survey and final interview were used to ascertain the views of participants regarding the success of the course. • student mentors were asked to provide regular reports (three each throughout the program) that outlined how participants were progressing, the challenges that the participants faced and the aspects of the course that the mentors thought had worked well. the mentors were also asked to complete a self-assessment at the end of the program to indicate the depth of their own self-development. • throughout the course, participants were asked to complete exercises related to the program, either independently or in conjunction with their mentors. the outputs of these exercises were not assessed, but they were discussed, and they provided some insights that were used in the development of the program and reporting of results in this article. • additionally, the reflections of the authors (as program deliverers) were used in the presentation of results in this article. all of the above (except for the author reflections) were either already in ms word format, were transcribed into ms word files or were documents that were scanned. these were all entered in the qualitative software analysis program nvivo. ms word documents and scanned files were imported as ‘sources’ into folders under the names of the participants and student mentors. separate nodes relating to the research questions were set up (as ‘themes’) and the results were analysed according to these themes. 4 results this section addresses the research questions from the perspective of the blended learning program under review. the research questions relate to the wong et al. (2014) bla framework and provide a lens via which the results can be presented. the following description of the bla program is based upon the main stages of the bla framework. section 4.1 presents an updated version of the bla framework, proposed for entrepreneurs in trades and similar vocational areas. 4.1 readiness the first research question was what was the preparedness of the university, instructors and tradespeople for the blended learning program? university readiness was discussed in section 3.4. the level of readiness of participants was partly known ahead of program delivery as it was requested that participants at least had access to a smart phone or a suitable device to access the internet. thus, the researchers anticipated that any problems would relate more to usage aspects of the digital divide rather than problems relating to access to ict. it was apparent at that early stage that the participants were, as anticipated, in a good position with regards to possessing ict devices to access the course materials. four of the six participants indicated little or no use of ict in the workplace, but there was recognition by all participants that ict could potentially improve their work practices. the areas that they regarded themselves as most competent in were the use of email and web browsers, as well as taking digital pictures via their smartphones. the readiness of staff (the researchers) was already known and meant that access to the facilities required for delivering the program was in place and the skills were available to australasian journal of information systems burgess, bingley, paguio & woodley 2018, vol 22, research article interventions in entrepreneurial learning 14 facilitate this. however, the researchers were not experienced in the use of this technology for external course delivery to this cohort and thus the delivery of the program provided some challenges for them that were not anticipated (such as expectations regarding how long it took for tradespeople to master the use of the remote technology). table 1 summarises the readiness assessment. bla framework stage blended learning approach readiness • ict available and already used by researchers – but they lacked experience in the use of the technology externally. • participant readiness assessed via ’pre-participation’ survey – they were well placed to access program material remotely table 1: blended learning program readiness (based on wong et al. 2014 bla framework) 4.2 intensity of adoption this section examines the question: what combination of blended learning options was provided for the tradespeople? table 2 shows the variety of options made available, classified according to wong et al.’s (2014) intensity of adoption framework stage. bla framework stage blended learning approach intensity of adoption blended learning options face-toface sessions mentor visits online support materials online tests online sessions practical exercises blended learning continuum basic ict use face-toface eenhanced efocussed efocussed varied perceived importance of selection of options by participants recommended vital recommended optional vital vital adoption by participants strong strong good weak strong strong table 2: representation of the intensity of adoption of the blended learning program for tradespeople (based on wong et al. 2014 bla framework) in relation to the ideal balance of instructional versus learning-centred delivery, consideration was first given to walker’s (2001) suggestion that the provision of basic technical skills early on in such a program can facilitate later ict use that is more contextual (applied specifically to the needs of the businesses). as discussed, generic material was covered in face-to-face and online sessions, and this was contextualised for each business via the mentor visits. the researchers made the decision to deliver basic ict skills at the start of the program and then move to more specific business-related ict topics later in the program (as per walker, 2001). this australasian journal of information systems burgess, bingley, paguio & woodley 2018, vol 22, research article interventions in entrepreneurial learning 15 combination of basic-to-complex and generic-to-context-specific delivery of ict concepts is shown in table 3. although the overall purpose of the program was to educate rather than teach (fayolle and gailly, 2008), it was necessary to set the foundations for this early in the program by using an instructional approach. time in program face-to-face or online session (instructional approach) mentor visit and practical exercises (learning-centred approach) early sessions generic (basic) ict concepts context-specific (basic) ict concepts later sessions generic (advanced) ict concepts context-specific (advanced) ict concepts table 3: delivery of ict concepts throughout the blended learning program the learner-centred approach allowed the tradespeople to select what combination of online support materials, tests and exercises that they wished to complete for their own learning experience, as well as the extent to which they interacted with their student mentor. this also addressed perren’s (2003, p.517) concern regarding entrepreneurs requiring “flexible, learnercentred” approaches. as an example of how participants applied generic ict concepts that they were introduced to in the instructional sessions in their own context, one of the exercises in an online class assisted the participants to identify and evaluate ‘apps’ for use in their businesses. participants were first introduced to the notion of an innovation or idea having specific characteristics that might influence its adoption (rogers, 2003). they were then asked to carry out an exercise for their business to identify and evaluate apps that might be useful for them (refer figure 4). this provided quite different results for the participants. the handyman tested and evaluated an app for forecasting the weather as much of his work was conducted outside. the hairdresser examined apps relating to accounts and record keeping for hairdressers and another app that suggested different hair styles for clients. australasian journal of information systems burgess, bingley, paguio & woodley 2018, vol 22, research article interventions in entrepreneurial learning 16 figure 4: sample participant exercise australasian journal of information systems burgess, bingley, paguio & woodley 2018, vol 22, research article interventions in entrepreneurial learning 17 week interaction initial program amended program 1 face-to-face session welcome; introduction welcome; introduction 2 mentor business visit; online revision 3 face-to-face session tablets & smartphones; customer interaction tablets & smartphones; security, privacy and storage (moved forward) 4 mentor business visit; online revision 5 online session security, privacy and storage; peripherals customer interaction (moved back) assessing ict investments (added) 6 mentor business visit; online revision 7 online session quotations; scheduling jobs; office software quotations; job scheduling; office software 8 online revision 9 mentor business visit; online session invoice and payment invoice and payment internet basics (added) 10 online session software as a service; setting up a website setting up a website portals and directories (added) 11 face-to-face workshop review software as a service (moved back); ict and work-life balance (added); future ict trends (added) table 4: blended learning program: initial and amended program table 4 shows the topics of the blended learning program from the initial program (as originally conceived) to the amended program (delivered). alterations were made after the initial face-to-face session in week 1 (when ict expertise of participants was assessed). this is an example of how the program fitted the action research approach, as well as demonstrating the learner-centred approach that was undertaken. the amended program shows topics that were moved from later in the program to earlier (‘moved forward’), topics delayed (‘moved back’) and new topics added to the program (‘added’). the content of the course was initially determined by the instructors, based upon many years’ experience in delivering basic ict concepts to business undergraduate students and research into the use of ict by small businesses. several changes in course content were initially made on the basis of the responses to the ‘pre-participation’ survey in week 1 of the course. for instance, after the initial week the researchers realised that participants lacked an understanding of basic ict concepts and thus would benefit from knowing about the basics of security, privacy and data storage of information on ict systems earlier, so this topic was moved forward from week 5 to week 3. subsequently, as deficiencies in ict knowledge were identified by instructors throughout the course, other changes were made. for example, a new topic, assessing ict investments, was added to week 5. australasian journal of information systems burgess, bingley, paguio & woodley 2018, vol 22, research article interventions in entrepreneurial learning 18 in other weeks, topics related to basic understanding of the internet and the use of business directories and portals were also added. these were examples of how the researchers identified during the course that the participants required more background about ict basics. the flexible nature of the program design allowed these topics to be incorporated. the next section discusses the evaluation of the blended learning program. 4.3 impact this section addresses the research question: how was the quality and extent of learning (impact) assessed for the program? it includes reference to the following sub-questions: • what were the advantages of blended learning delivery for participating tradespeople? • was the blended learning program able to provide suitable skills to tradespeople about basic and context relevant ict use? participants were asked to complete a ‘post-participation’ survey at the end of week 11. the survey asked for their overall views of the quality of the program and the different blended learning options that were provided. regarding those options, participants were asked to rate each option as great (very satisfactory); good (satisfactory); fair (neutral) or not satisfactory and were also requested to provide specific feedback on each approach. according to lozano, garcia-cueto and muniz (2007) the optimum number of alternatives in rating scales is four to seven. the even number of suggestions were selected to force respondents to lean to a positive or negative response. they were also interviewed one year after the conclusion of the program to ascertain the longer-term effects of the program or the extent of their learning. in addition, the mentors were interviewed after the program to determine their views of the success of the different options. an evaluation of each blended learning option will now be provided, followed by an evaluation of the overall program. 4.3.1 face-to-face sessions whilst attendance at the face-to-face sessions was satisfactory (all the participants attended at least one session), there were a few instances when some participants were not able to attend specific sessions. this was always because of their work commitments (as noted by walker, 2001). one mentor commented that this was a potential limitation of the sessions. all but one of the participants rated these sessions ‘good’ or ‘great’. the café owner rated the sessions as ‘fair’ and indicated that some aspects were a bit hard to understand as her english was “not great”, although this participant also suggested that the “demonstrations (in the sessions) helped with this”. the handyman suggested that the classes were good, but “could have had an even greater practical emphasis”. overall, it was considered that the sessions went well, but in future sessions the needs of participants for whom english is not a first language should be considered. the mentors indicated that the sessions were useful, although one mentor suggested that it might have been more useful to have more interactive activities. 4.3.2 online sessions figure 5 provides a sample screen of the online sessions as typically viewed by a participant. again, participation rates were good for these sessions. in the few instances where participants were not available, it was again because of work commitments – although these sessions were easier for participants to participate in due to the reduced travel requirements. australasian journal of information systems burgess, bingley, paguio & woodley 2018, vol 22, research article interventions in entrepreneurial learning 19 figure 5: sample screen from the online sessions (participant details removed) however, none of the participants rated these sessions as ‘great’. there were some technical problems with the sessions, especially early on. all participants were able to view the slides, hear the audio, offer text input via the ‘chat’ function and use the voting facility. however, some participants had difficulty with setting up audio and were not able to ask questions via their microphones in the early sessions. this is an example of a situation mentioned in the literature review where there could potentially be a problem with ict infrastructure. there was initially only basic information provided to participants with regards to setting up and participating in the online sessions. the unrealistic expectation was that this would be managed by mentors. one participant suggested that it “would be handy to have rehearsals before the online sessions”. another was not able to comment via microphone until the last online session. before that he was limited to text comments. he observed that “improved audio is advisable”. one mentor commented that “most [of his] participants had communication problems”. the café manager noted that it was once again difficult to understand aspects of the sessions when “complicated language was used” and that “this was harder without face-to-face demonstrations”. 4.3.3 mentor visits all participants interacted with their mentors during the allocated periods for mentor visits. this aspect of the blended learning program was rated equal highest by the participants (either ‘great’ or ‘good’). at different times throughout the program, mentors were provided with a ‘window’ of time to visit their businesses between the face-to-face and online sessions. each visit was aimed to reinforce the concepts from the previous face-to-face or online session and to provide any other assistance for participants – such as technical assistance (setting up ict), assisting/ advising on practical exercises (context) and providing clarification on aspects of the sessions that participants were unsure of. the handyman described the mentor visits as a “vital part of connective learning”. in support of this, a student mentor commented that the visits were “very necessary. a vital part of the participants’ learning. they would not have been able to cross the river without the bridge”. the café owner, who had previously commented on the difficulties of understanding aspects of the sessions, added the following feedback: when i didn’t understand the online and face-to-face session because of my english level, the student helped to explain it to me and i understood it better. she showed me how to use the internet and computer so that i can see for myself and learn. australasian journal of information systems burgess, bingley, paguio & woodley 2018, vol 22, research article interventions in entrepreneurial learning 20 the student mentor who had previously mentioned the difficulty for some participants to attend the face-to-face and online sessions observed: i think these were most useful and accessible given the flexibility of arranging visits. we were able to narrow the information in sessions to their specific industry, making it more relevant and practical. they [the participants] were able to ask questions and learn at their own pace. it was well received, particularly learning how to use the internet via practical demonstrations. overall the participant results indicated that the mentors played a vital role in the conduct of the program. 4.3.4 practical exercises as was mentioned earlier, the practical exercises were designed to provide a means by which participants could apply the generic concepts to their businesses. these were rated at the same level as mentor visits by participants (‘great’ or ‘good’). the participant with a maintenance business commented that there should be more practical exercises. the handyman suggested that the exercises allowed concepts to be explored in greater depth. one mentor commented that the exercises were “very important for the participants. this is where they were able to put the pieces to the puzzle together”. the café owner indicated that she specifically enjoyed the exercise that allowed her to examine other catering websites for ideas on how to design her own. however, the comments were not all positive. in one exercise, participants were asked to assess the suitability of different software packages for their business. this referred to the specific exercise that was highlighted earlier in the article. a hairdresser commented: i understand the (software/app) assessment form was to make me think first before i choose to buy, but when the student helped to fill it out for me i didn’t really understand the headings. my answers were all the same for every one that i filled out. the results suggest that the student mentors balanced the need to support the businesses by demonstrating aspects of the course against allowing participants to find things out for themselves quite well. the quoted example might have been one instance where this was not the case. two other participants indicated that this exercise made them think about evaluating their options in relation to their businesses. 4.3.5 online support materials all but one participant used the online support materials and rated them between ‘fair’ and ‘great’. only one participant, however, rated them as ‘great’. it appeared that these materials were accessed mainly when participants were searching for more information to help them understand the course content. an example of this was a comment from the café owner: “most of them [the materials] didn’t apply to my catering business but i did find useful the link to terms and guide of what to look for when choosing a computer”. the participant who rated the materials as ‘great’ indicated that they “enabled me to learn more”. one participant that rated the materials as ‘fair’ suggested some difficulties with locating the materials on blackboard at times. figure 6 provides an example of how the extra materials were made available to participants. australasian journal of information systems burgess, bingley, paguio & woodley 2018, vol 22, research article interventions in entrepreneurial learning 21 figure 6: week 5 of the course content. this shows the support materials for the week – a list of software examples and a guide to buying a laptop or tablet. the mentors generally supported the views of the participants. one mentor observed that the materials would have been helpful but that there needed to be greater focus placed upon them. the other mentor observed: the content provided useful information, but in all honesty, i think if we didn’t go through them during the mentor visits, i don’t think my participants would have explored the links … to some extent it could be argued that the support materials served their purpose. they were not actually meant to be key components as such – just to provide extra support where needed or selected by participants as part of the learner-centred approach. 4.3.6 online tests participants were introduced to the online tests in the first face-to-face session. the tests took the form of a series of multiple choice questions. the tests could be taken at any time by participants and feedback was provided immediately by the system upon completion. the aim of the tests was to reinforce the most basic concepts introduced in the face-to-face and online sessions. however, the online tests were discontinued early in the program due to lack of use by participants. two comments made by participants were that the tests “were too basic” and “i don’t have the time with other tasks i am performing for the class”. 4.3.7 evaluation of overall program in the ‘post-participation’ survey, participants were asked to rate the extent of their knowledge in relation to the key topic areas covered in the class. all participants indicated an increase in the extent of their knowledge about ict usage. when asked to elaborate, the following comments were typical: “excellent” [handyman] “i gained further knowledge” [maintenance]. table 5 summarises the findings of the evaluation of the various components of the blended learning package. three aspects of the program, face-to-face sessions, mentor visits and supporting practical exercises, were viewed as being vital to the program’s success. perhaps greater emphasis on the practical exercises could be considered in future courses as suggested by some participants. the online sessions were also an important part of the program once the technology problems were overcome. delivered on a wider scale, the ongoing success of such a program would rely on the technical problems involved in the online sessions being solved. australasian journal of information systems burgess, bingley, paguio & woodley 2018, vol 22, research article interventions in entrepreneurial learning 22 the online support materials were not viewed as a vital part of the program but did provide the means for those interested participants to further extend their knowledge if they desired. this enabled them to further tailor the program to their own needs. although the online tests did not prove to be successful in this program, they should not be ruled out for ongoing programs if some participants require the reinforcement of basic ict concepts. an interesting aspect of the evaluation occurred when participants were interviewed one year after the conclusion of the program to ascertain their views of the program and its longer-term effects. all of them indicated that they enjoyed the program and that it had changed their business behaviour in some manner (to varying degrees). bla framework stage blended learning approach face-toface sessions mentor visits online support materials online tests online sessions practical exercises im pa ct effectiveness assessed via: • participant feedback during course • intensity of adoption ‘fair’ to ‘great’ ‘good’ to ‘great’ ‘fair’ to ‘great’ not evaluated ‘fair’ to ‘good’ ‘good’ to ‘great’ observations effective could be difficult to attend resource intensive vital link not seen as mainstream activity too basic participants too busy technology challenges useful when technology worked bridge between sessions, mentors, business response vital to program continue vital to program continue served their purpose. continue not useful in this context continue, but need to sort out the technology problems vital to program continue, perhaps with greater emphasis overall impact assessed via: • ‘post-participation’ survey • interviews with student mentors • program was beneficial; worthwhile • benefits were evolutionary rather than revolutionary table 5: representation of the impact of the blended learning program for tradespeople (based on wong et al. 2014, bla framework) further, it could be argued that any alterations in behaviour were evolutionary rather than revolutionary, with increased use of the internet to search for information and mobile devices being the main outcomes. two of the businesses (handyman and café) had introduced websites due to participation in the program. other ‘new’ uses of ict revolved around use of the internet for transactions with online purchases being reported by one participant and online banking by another. all participants expressed that the program was worth the effort of participation. australasian journal of information systems burgess, bingley, paguio & woodley 2018, vol 22, research article interventions in entrepreneurial learning 23 overall, the mentors and the participants suggested that the program was a success. this was determined on the basis that all participants indicated that there was an increase in their practical knowledge in the last week of the course (via their ‘post-participation’ survey) and in the interviews 12 months down the track where they indicated that there had been a change in their business activities due to their participation in the course. obviously, there are some issues to be considered before the program could be delivered on a wider scale, but the combination of general sessions, mentor visits and practical exercises provided a useful way to firstly deliver general ict knowledge and then to cascade this to individual businesses to apply in their own contexts. the success of the program also supported the combination of the instructional approach (for delivery of basic concepts) and the learner-centred approach (for context-specific applications) in the program. 5 discussion in this section the results presented in section 4 are compared back to the literature review. when considering the readiness of tradespeople for the blended learning programs, they were well placed to access the course due to their prior use of smart mobile devices, although their usage was basic (email and surfing the web). with regards to the digital divide, participants had physical access to suitable technology but were limited regarding their use of ict (srinuan & bohlin, 2011). the fact that these businesses agreed to participate in the program indicated that they had an entrepreneurial attitude that lead to them wanting to improve themselves and/or their businesses (gray & gonsalves, 2002). regarding program delivery, whilst the instructors were experienced in the use of the technology, they had not delivered such programs to groups that were external to the university before. this resulted in some difficulties (such as some of the technical problems that were encountered) that could potentially have been anticipated earlier. the delivery of the program included activities that ranged between face-to-face and e-focussed on jones et al.’s (2009) continuum of blended learning. with regards to an appropriate combination of blended learning options for tradespeople (jones et al., 2009), the participants struggled at times to attend both the face-to-face and online sessions, although not having to travel made it easier for them to access the online sessions. this time restriction faced by tradespeople was also noted by walker (2001). even the student mentors had to reschedule their visits on a few occasions as participants’ work took first priority. this was the challenge of dealing with entrepreneurs who were self-reliant and did not necessarily work regular hours. the appropriate balance of instructional versus learning-centred approaches for the tradespeople was achieved. the course was delivered by initially covering the necessary generic ict skills before moving onto the instructional ict skills that allowed learning-centred learning to occur, as recommended by walker (2001). the way participants could work by themselves to apply the concepts delivered in the general sessions to their own unique situations (with the assistance of student mentors where required) allowed this type of learning to occur. the effectiveness the program was assessed via a number of means. there were certainly indications that the blended learning program allowed for all of the advantages of blended learning as identified by elliot and clayton (2007) (flexibility in time; flexibility of place and flexibility of delivery) to occur. however, as indicated earlier, even with the flexible australasian journal of information systems burgess, bingley, paguio & woodley 2018, vol 22, research article interventions in entrepreneurial learning 24 approaches, some of the participants struggled to participate at times due to work commitments. as suggested by fayolle and gailly (2008), a range of evaluations (designed around the program) were used to assess the success of the program. all of the participants indicated satisfaction with the program, although their progress could be described as evolutionary rather than revolutionary, with changes to business activities being an important outcome of the program. however, from an entrepreneurial learning viewpoint, they all indicated that they had developed new knowledge that assisted in them in the operation of their businesses. the results of the evaluation suggested that the key aspects of the program were the face-toface sessions, the involvement of the student mentors to visit and work with the businesses and assist them to apply the generic concepts to their business and the practical exercises that assisted them to do this. 5.1 limitations and implications the program was obviously limited in size due to the small number of participants. a similar program would need to be delivered with a larger group before the researchers could be confident that the program outcomes achieved in this instance could be translated to provide benefits to tradespeople or those in similar vocational areas. from an entrepreneurial learning viewpoint, the program targeted planned, personal (rather than collective) learning and, as such, conclusions are limited to that sphere. as such, the generalisability of the study is limited. however, the benefits of the program are promising and the researchers feel that some of the lessons learned during the delivery of this blended learning program reflect the issues that were identified in the literature and could potentially be applied to the delivery of ict concepts to entrepreneurs in the trades and similar vocations: • the notion of starting with general ict concepts and then branching out to more context-specific ict topics was effective (walker, 2001). • when selected carefully, student mentors can provide a vital and cost-effective means of converting generic ict concepts into business-specific applications (context). • a well-planned group of practical exercises to support these activities are vital. • entrepreneurs are often limited regarding the time they can devote to these types of courses (walker, 2001). the provision of different blended learning options can help to overcome these problems by offering flexibility of time, place and delivery (elliott and clayton, 2007). • any use of ict for blended learning should be accompanied by a level of technical support that provides effective access and use of that technology – otherwise participants may find themselves victims of a form of digital divide (srinuan and bohlin, 2011) regarding the technology being used to deliver the program. 5.2 amended version of framework for entrepreneurs in the trades and similar vocations whilst the wong et al. (2014) bla framework had been developed through the delivery of blended learning to undergraduate higher education students, it had not been applied outside of this environment. refer to figure 7 for a proposed update to the framework, specifically australasian journal of information systems burgess, bingley, paguio & woodley 2018, vol 22, research article interventions in entrepreneurial learning 25 tailored for entrepreneurs working in the trades or in similar vocational areas. this has been developed in an inductive manner based on results presented in section 4. when revisiting the framework, a few immediate issues can be considered. the overall ict infrastructure will typically be put in place by the training provider, but entrepreneurs will need to have access to ict devices (such as pcs, tablets and/or smartphones) to use the system. trainers will need to have the skills to develop learning materials for the blended learning package and a positive attitude towards doing so (as per wong et al. 2014). one of the obvious findings of the study was the enthusiasm (positive attitude) that participants had towards the program (as discussed in the methodology section). the skills to use the technology are also important for entrepreneurs. the need to consider training provider infrastructure; entrepreneurs’ access to ict devices, skill level and enthusiasm; and the skills and attitudes of trainers have been added to the readiness section of the bla framework. figure 7: revised blended learning assessment framework for entrepreneurs in the trades and similar vocational areas (amended from wong et al., 2014). note also there is now a direct link from training provider readiness to the selection of blended learning options by the trainer, as the ict infrastructure will influence the availability of options for trainers to select from. the intensity of adoption section has two important additions that have been added because of the study. the first relates to the provision of adequate training for trainers (to develop blended options and associated curricula) and entrepreneurs (to effectively access and use the system). there is also a recognition that technical support will likely need to be provided to both trainers and entrepreneurs throughout the conduct of the program. note that the framework does not stipulate the types of training and support provided for the system, just that it should be evident. the impact section of the framework remains unchanged, with the quality of learning remaining the key objective to be measured. it could potentially be argued that improvements to the small business could also be added here and might be considered in future implementations of the framework. academics assessing blended learning programs for similar groups should find the amended framework australasian journal of information systems burgess, bingley, paguio & woodley 2018, vol 22, research article interventions in entrepreneurial learning 26 useful as it maintains the readiness, intensity of adoption and impact aspects of the wong et al. (2014) framework, but introduces other key considerations as key aspects of such a blended learning program. 6 conclusion this article examined the development and delivery of a pilot blended learning program designed to assist a group of tradespeople to build their ict capacity as part of a proactive entrepreneurial learning exercise for planned, personal learning rather than collective learning. in doing this, wong et al.’s (2014) blended learning assessment framework was used to identify the processes that are key to the successful implementation of such programs: determining the readiness of instructors to deliver and enterprising business operators to take part in the program; considering the blended learning options available and how they are adopted by participants (intensity of adoption) and determining the program’s impact through an evaluation of the different blended learning options delivered and the program overall. consideration of all of these factors ensured a comprehensive evaluation of the program. whilst the combination of face-to-face and online sessions were effective for delivering basic ict concepts to motivated participants, some participants faced pressures regarding freeing up the time from their employment to participate. the combination of student mentor visits and practical exercises provided an effective way for business operators to apply the basic ict concepts into context specific applications for their businesses. the project also demonstrated how a combination of instructional approaches (for the delivery of basic ict concepts) and learning-centred approaches (for context-specific applications) facilitated the eventual successful outcomes of the program – highlighting the importance of educating rather than teaching entrepreneurs (fayolle and gailly, 2008) to achieve effective entrepreneurial learning. the experience of delivering the program informed the development of a revised blended learning framework for entrepreneurs working in the trades or in similar vocational areas (a theoretical contribution of this article), as well as the provision of potential lessons for the delivery of ict concepts to these entrepreneurs. 7 acknowledgement the authors acknowledge the important assistance of the telematics trust in providing the funds for the conduct of the program reported in this article. references australian bureau of statistics (2012). australian small business: key statistics and analysis december 2012, canberra, australia: commonwealth of australia. biggs, j., & tang, c. (2011). teaching for quality learning at university (4th ed.). berkshire, england: mcgraw-hill international. bliuc, a., goodyear, p., & ellis, r. a. (2007). research focus and methodological choices in studies into students' experiences of blended learning in higher education. the internet and higher education, 10(4): 231-244. bound, h. (2011). vocational education and training teacher professional development: tensions and context. studies in continuing education, 33(2): 107-119. bruce, j. (2012). discipline specific e-training for trades tutors. waikato, new zealand: ako aotearoa. australasian journal of information systems burgess, bingley, paguio & woodley 2018, vol 22, research article interventions in entrepreneurial learning 27 burgess, s., sellitto, c., & karanasios, s. (2009). effective web presence solutions for small businesses: strategies for successful implementation. hershey, pa: information science reference. cambridge university press. (2013a). english definition of ict. cambridge dictionary. retrieved from http://dictionary.cambridge.org/dictionary/business-english/ict, [accessed 3 december 2013]. cambridge university press. (2013b). english definition of tradesperson. cambridge dictionary. retrieved from http://dictionary.cambridge.org/dictionary/businessenglish/tradesperson?q=tradesperson; [accessed 3 december 2013]. cope, j., & watts, g. (2000). learning by doing–an exploration of experience, critical incidents and reflection in entrepreneurial learning. international journal of entrepreneurial behaviour & research, 6(3): 104-124. darke, p., & shanks, g. (2002). case study research. in k. williamson (ed.), research methods for students, academics and professionals: information management and systems (2nd ed). (pp.111-124). nsw, australia: centre for information studies, charles sturt university. de george-walker, l., & keeffe, m. (2010). self-determined blended learning: a case study of blended learning design. higher education research & development, 29(1), 1-13. deloitte. (2012) digital disruption—short fuse, big bang?, building the lucky country: business imperatives for a prosperous. stories from the digital frontline. australia. retrieved from http://www.deloitte.com/au/digitaldisruption, [accessed 4 april 2014]. elliott, r., & clayton, j. (2007). e-learning for new zealand industry training organisations: analysis of benefits and barriers. ict: providing choices for learners and learning. ascilite proceedings, singapore: ascilite. emes, c., & cleveland-innes, m. (2003). a journey toward learner-centred curriculum. the canadian journal of higher education, 33(3), 47-70. erdélyi, p. (2010). the matter of entrepreneurial learning: a literature review. international conference on organizational learning, knowledge and capabilities 2010, boston, us: olkc. fayolle, a., & gailly, b. (2008). from craft to science: teaching models and learning processes in entrepreneurship education. journal of european industrial training, 32(7): 569-593. fernandes, p. (2016). entrepreneurship defined: what it means to be an entrepreneur. business news daily. march 2, retrieved from https://www.businessnewsdaily.com/7275entrepreneurship-defined.html, [accessed 8 january 2018]. gasson, s, & waters, j. (2013). using a grounded theory approach to study online collaboration behaviors. european journal of information systems, 22(1): 95-118. gray, c., & gonsalves, e. (2002). “organizational learning and entrepreneurial strategy”. the international journal of entrepreneurship and innovation, 3(1): 27-33. harrison, r. t., & leitch, c. m. (2005). entrepreneurial learning: researching the interface between learning and the entrepreneurial context. entrepreneurship theory and practice, 29(4): 351-371. jones, p., simmons, g., packham, g., beynon-davies, p., & pickernell, d. (2014). an exploration of the attitudes and strategic responses of sole-proprietor micro-enterprises australasian journal of information systems burgess, bingley, paguio & woodley 2018, vol 22, research article interventions in entrepreneurial learning 28 in adopting information and communication technology. international small business journal, 32(3): 285-306. jones, n., chew, e., jones, c., & lau, a. (2009). over the worst or at the eye of the storm? education training, 51(1): 6-22. kirkup, g. (2010). vocational education and training (vet) for ict employment: preparing women for work. in women and icts through the lifecycle, barcelona, spain: uoc. löbler, h. (2006). learning entrepreneurship from a constructivist perspective. technology analysis & strategic management, 18(1): 19-38. lozano, l., garcia-cueto, e. & muniz, j. (2007). effect of the number of response categories on the reliability and validity of rating scales, methodology, 4(2): 73-79. machado, c. (2007). developing an e-readiness model for higher education institutions: results of a focus group study. british journal of educational technology, 38(1): 72-82. martin, l. m., & wright, l. t. (2005). no gender in cyberspace?: empowering entrepreneurship and innovation in female-run ict small firms. international journal of entrepreneurial behaviour & research, 11(2): 162-178. mullin, t. (2013). cutting edge or cut loose? an exploration of apprentices’ experiences of workplace e-learning. journal of vocational education & training, 65(1): 66-86. niblock, s. (2013). media professionalism and training (key concerns in media studies series ed.). uk: palgrave macmillan. oosthuizen, m. j. h. (2002). action research. in k. williamson (ed.), research methods for students, academics and professionals: information management and systems (2nd ed.). (pp. 159-176). wagga wagga, nsw, australia: centre for information studies, charles sturt university. overby,e., slaughter, s., & konsynski, b (2010). the design, use and consequences of virtual processes, information systems research, 21(4): 700-710. parker, s. (2000). a survey of small business in colorado. colarado, usa: colorado community college and occupational education system. perren, l. (2003). the role of e-mentoring in entrepreneurial education and support: a metareview of academic literature. education + training, 45(8/9): 517-525. pittaway, l., & thorpe, r. (2012). a framework for entrepreneurial learning: a tribute to jason cope. entrepreneurship & regional development, 24(9-10): 837-859. rogers, e. m. (2003). diffusion of innovations (5th ed.). new york: the free press. srinuan, c., & bohlin, e. (2011). understanding the digital divide: a literature survey and ways forward. 22nd european regional conference of the international telecommunications society, budapest, hungary: international telecommunications society. telstra corporation. (2013). sensis e:business report: the online experience of small and medium enterprises. australia: sensis. valentine, e. (2011). ict in vocational education and training: a view of information and communication technology in vocational education in new zealand. wellington, new zealand: tertiary education commission and service industry training alliance. australasian journal of information systems burgess, bingley, paguio & woodley 2018, vol 22, research article interventions in entrepreneurial learning 29 van praag, c.m. & versloot, p.h. (2007). what is the value of entrepreneurship? a review of recent research. small business economics. 29(4): 351-382. veal, a. j. (2005). business research methods: a managerial approach (2nd ed.), australia: pearson education australia/addison wesley. walker, s. (2001). icts and innovation in trade unions: internet training in european trade union confederations and worker education organizations. human resources global management conference, barcelona, spain: cranet and esade. willliamson, k. (2013). research concepts, in williamson, k. & johanson, g. (eds). research methods: information, systems and concepts. prahran, australia: tilde university press. wong, l., tatnall, a., & burgess, s. (2014). a framework for investigating blended learning effectiveness. education + training, 56(2/3): 233-251. yin, r. k. (2003) case study research: design and methods (3rd ed.). london, uk: sage. copyright: © 2018 burgess, bingley, paguio & woodley. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 literature review 2.1 entrepreneurial learning – the use of ict by tradespeople 2.1.1 how entrepreneurial businesses can use ict 2.1.2 tradespeople and ict use 2.2 blended learning – ict as a tool for delivering training 2.2.1 readiness 2.2.2 intensity of adoption 2.2.3 impact 3 research methodology 3.1 research questions 3.2 study approach 3.3 participant selection 3.4 design of the program 3.5 data collection and analysis 4 results 4.1 readiness 4.2 intensity of adoption 4.3 impact 4.3.1 face-to-face sessions 4.3.2 online sessions 4.3.3 mentor visits 4.3.4 practical exercises 4.3.5 online support materials 4.3.6 online tests 4.3.7 evaluation of overall program 5 discussion 5.1 limitations and implications 5.2 amended version of framework for entrepreneurs in the trades and similar vocations 6 conclusion 7 acknowledgement paper title (paper title style) australasian journal of information systems thuan, antunes & johnstone 2017, vol 21, research article a process model for establishing business process crowdsourcing 1 a process model for establishing business process crowdsourcing nguyen hoang thuan can tho university of technology vietnam nhthuan@ctuet.edu.vn pedro antunes victoria university of wellington david johnstone victoria university of wellington abstract crowdsourcing can be an organisational strategy to distribute work to internet users and harness innovation, information, capacities, and variety of business endeavours. as crowdsourcing is different from other business strategies, organisations are often unsure as to how to best structure different crowdsourcing activities and integrate them with other organisational business processes. to manage this problem, we design a process model guiding how to establish business process crowdsourcing. the model consists of seven components covering the main activities of crowdsourcing processes, which are drawn from a knowledge base incorporating diverse knowledge sources in the domain. the built model is evaluated using case studies, suggesting the adequateness and utility of the model. keywords: business process crowdsourcing, crowdsourcing, design science, process model 1 introduction organisations have been opening their boundaries to access external labour, knowledge, expertise, and innovation for some time, including the use of open software, open innovation and outsourcing. more recently, crowdsourcing, referring to the distribution of work to internet users via open calls, has emerged as a viable alternative (howe 2006). by adopting crowdsourcing, organisations can, for the first time, procure, assign and control work online independently from geography, time, and location. crowdsourcing can also tap into external expertise and creativity. such capability, combined with an elastic and round the clock workforce, increases organisational agility and resilience. all in all, the virtually limitless workforce makes crowdsourcing an attractive strategy to organisations (gill et al. 2015; rosen 2011; saxton et al. 2013). while bringing new business opportunities to organisations, crowdsourcing at the same time changes the way organisations conceive and manage work. through crowdsourcing, the organisational boundaries become hard to identify since organisations may access an unprecedented range of human resources from virtually everywhere (tranquillini et al. 2015; vukovic and bartolini 2010). furthermore, crowdsourcing has changed the way organisations design work structures. different from traditional structures, which establish a stable and topdown hierarchy of responsibilities and remunerations, crowdsourcing promotes a dynamic, loose and bottom-up structure where professional knowledge, decision making, problem solving, and supervisory control can be assigned to unknown members of the crowd (brabham 2013; kittur et al. 2013). these changes properly reflect a new, perhaps more turbulent, organisational business model fostered by the internet era. in this new business model, one important challenge is what the best way to establish a crowdsourcing process is. we stress here the etymology of the word ‘process’, which refers to a systematic and repeatable course of action to accomplish some deliberate results. wellplanned and dedicated processes are assumed not only to produce better crowdsourcing australasian journal of information systems thuan, antunes & johnstone 2017, vol 21, research article a process model for establishing business process crowdsourcing 2 results, but also to deploy the strategy faster on top of existing crowdsourcing platforms (tranquillini et al. 2015). in contrast, an ad hoc process affects organisations due to the need for re-planning and the waste of organisational resources (muhdi et al. 2011; rouse 2010). motivated by the challenge, coupled with the increasing popularity of crowdsourcing as a business practice, this paper aims at supporting organisations successfully establishing the crowdsourcing process. given the important role of the process view in crowdsourcing strategies, researchers have already begun to investigate the crowdsourcing process. although different studies have already investigated several parts of the crowdsourcing process, most of them tend to focus on its individual aspects (geiger and schader 2014; man-ching et al. 2011). we note in particular that a large number of studies regard crowdsourcing as a one-off venture, instead of being one among many business processes available to an organisation. this one-off view leads to scattered, ad hoc practices, which are often difficult to reproduce, support and optimise. recently, some research efforts have focussed on integrating and harmonising concepts of crowdsourcing processes (amrollahi 2015; hetmank 2013), yet further efforts are needed to validate and empirically test these propositions before they can be used in practice. for this reason, there is a current lack of a holistic crowdsourcing process model, which organisations can rely upon to establish crowdsourcing processes. against this gap, the key research question addressed in this paper is stated as follows: what is the best way to model the holistic crowdsourcing process, which identifies, structures and relates its fundamental components in a systematic way? this research will investigate crowdsourcing using a business process management (bpm) lens, essentially decomposing complex systems into a set of independent, yet coordinated, activities (van der aalst and hee 2004). this lens allows us to analyse independently crowdsourcing components and coordinate them into an integrated crowdsourcing process. we designate this particular view as business process crowdsourcing (bpc), which was first coined by la vecchia and cisternino (2010). based on the bpc view, we then seek to identify the salient activities of crowdsourcing processes and use them to build a process model that supports the establishment of crowdsourcing as an organisational business process. finally, we evaluate the adequateness and utility of the proposed model by examining two case study projects. balancing between building and evaluating, we adopt design science as the research paradigm (hevner and chatterjee 2010; hevner et al. 2004). design science comprises two important complementary activities: build and evaluate (hevner and chatterjee 2010). the build activity focuses on developing artefacts to address the defined problem and to meet the identified business needs. the evaluation activity then assesses the artefacts to form evidence for the artefacts’ utility (venable et al. 2012). following the tenets of design science, we first build a knowledge base using a scoping literature review. the model is designed using the raw materials provided by the scoping review, offering a systematic comprehensive approach to model construction. the evaluation is done using two case studies of crowdsourcing projects, where the proposed model becomes an analytical lens investigating the collected data drawn from multiple sources that include interviews with key members of the projects. as a result, the case studies provide empirical evidence about utility and adequateness of the proposed model. the expected contributions of the current study are threefold. first, the study provides holistic understanding of the crowdsourcing process with the bpc concept (geiger et al. 2011; manching et al. 2011). second, we propose a model for bpc that identifies the critical components that organisations should consider when integrating a crowdsourcing strategy. by adopting a process-centred perspective, the model addresses the current lack of a way to organise business processes based on crowdsourcing (khazankin et al. 2012a; satzger et al. 2011). third, through the case study approach, the study evaluates the adequateness and utility of the model in two crowdsourcing projects. the case studies provide empirical evidence that complements other australasian journal of information systems thuan, antunes & johnstone 2017, vol 21, research article a process model for establishing business process crowdsourcing 3 research efforts seeking to conceptualise the crowdsourcing process (amrollahi 2015; hetmank 2013; thuan et al. 2014). the remainder of this paper is structured as follows. the next section discusses the related work, providing an overview on crowdsourcing and defining the concept of bpc. we then present a detailed development of the proposed model, starting with a scoping review to identify the main activities of crowdsourcing processes. subsequently, we describe the two case studies and present results from using the model as an analytic lens. we finally discuss our findings and conclude the study, including suggestion for future research directions. 2 background 2.1 concept of crowdsourcing the concept of crowdsourcing has recently emerged when howe (2006) introduced a process utilising the crowd for fulfilling tasks through the internet. investigating this concept, researchers proposed several underpinnings behind the emergence of crowdsourcing. by and large, these underpinnings can be grouped into three categories: the wisdom of crowd, the dominance of web technology, and the organisational context. first, it is now widely accepted that the crowd’s wisdom plays a significant role in crowdsourcing (saxton et al. 2013; zhao and zhu 2014). the ‘wisdom of crowds’ was explored by surowiecki (2004), who claimed that “under the right circumstances, groups are remarkably intelligent, and are often smarter than the smartest people in them” (p. xiii). malone et al. (2010) have extended the underpinning by adding the idea of collective intelligence. different from the ‘wisdom of crowds’ that is based on the independence of individuals (surowiecki 2004), the idea of collective intelligence stresses the coordination of many individuals (bonabeau 2009; malone et al. 2010). this extension allows crowdsourcing to find solutions in not only individual capability but also coordinated ways. the second category of underpinnings concerns web-centric platforms, such as social media and community sites, which have changed the roles of internet users from passive receivers to more active contributors (brabham 2010). brabham (2013) notes that internet users seem keen to contribute their ideas, knowledge, skills, and labour into these platforms. such contributions are as well valuable for crowdsourcing (saxton et al. 2013). from an organisational perspective, the web empowers the open call, which allows reaching any given interested participants, which is a distinctive characteristic of crowdsourcing (doan et al. 2011; schenk and guittard 2011). with these characteristics, the web provides a medium for organisations to approach a large number of users who actively contribute to different work propositions. the last underpinning comes from an organisational standpoint. we have already mentioned that web-centric platforms enable the crowd to contribute with fewer barriers, e.g. regarding time and space. therefore, the next question is whether organisations need such contributions. in fact, they do. the needs for external agents to solve organisational problems have been clearly presented in both outsourcing (dibbern et al. 2004) and crowdsourcing literature (muntés-mulero et al. 2013). by adopting crowdsourcing, organisations may get benefits similar to outsourcing, such as cost savings and access to outside capabilities (rouse 2010; saxton et al. 2013), or even more, such as customer involvement and flexible, on-demand labour. given the aforementioned discussion, a combination of the crowd, web technology, combined with organisational demands, can explain the emergence of crowdsourcing. however, it seems the demands of organisations for crowdsourcing have recently changed. previously, organisations used crowdsourcing mainly for simple one-off tasks (zhao and zhu 2014). recently, they have started to adopt crowdsourcing for more complex organisational processes, such as product development and industrial processes (djelassi and decoopman 2013; muntés-mulero et al. 2013). this leads to the need for clear conceptualisation of business process crowdsourcing. australasian journal of information systems thuan, antunes & johnstone 2017, vol 21, research article a process model for establishing business process crowdsourcing 4 2.2 crowdsourcing process and business process crowdsourcing a crowdsourcing process structures a set of activities necessary to operationalise a crowdsourcing project. understanding the crowdsourcing process allows us to understand its activities, and thus possibly produce better crowdsourcing results (tranquillini et al. 2015). consequently, this process view has attracted some attention from researchers. by and large, research on crowdsourcing processes can be classified into two views: high and low level of granularity. with high granularity, a few studies adopt a holistic conceptualisation about the crowdsourcing phenomenon. trying to paint an overall picture of the crowdsourcing process (muhdi et al. 2011; stol and fitzgerald 2014), they tend to focus more on high level concepts, and thus face significant gaps explaining what is the best way to effectively establish the crowdsourcing process. in the other view, a larger number of studies tend to investigate only specific parts of a crowdsourcing process. the ad hoc nature of these studies has been recently highlighted in the literature (geiger and schader 2014; man-ching et al. 2011; zhao and zhu 2014). the issue is not about their usefulness due to lack of repeatability, but instead there is no collective cohesiveness. as a result, there is little scaffolding of the studies’ outcomes towards a structured, holistic framework. that is, this group of studies have suggested scattered sets of practices, which challenges organisations when trying to establish their crowdsourcing processes. as a result, the domain is still lacking “a comprehensive guideline through which practitioners can initiate and manage their crowdsourcing projects” (amrollahi 2015, p. 2). in our research, we attempt to reconcile the two views by providing a more integrated picture of the crowdsourcing process. more precisely, we describe this integration as business process crowdsourcing (bpc). the term bpc was coined by la vecchia and cisternino (2010), and further discussed by thuan at al. (2014) as a way to establish organisational business processes based on crowdsourcing. etymologically, bpc combines the phrase business process with the word crowdsourcing. this paper elevates the business process construct to be equally important to the crowdsourcing construct. according to aalst and hee (2004), a business process is defined as a collection of individual activities and a workflow coordinating them. it aims to achieve a particular goal with both effectiveness and efficiency. a business process is purely conceptual, yet it serves as a template for creating multiple, real life instances of the same process, which organisations may create repeatedly and concurrently. given that, we define bpc as a set of activities completed by crowdsourcing entities, in conjunction with a coordination of these activities, that collectively form the entire business process. our thesis is that bpc proposes an integrated approach for organisations to establish crowdsourcing processes both systematically and efficiently. more precisely, the bpc perspective allows analysing both individual aspects of crowdsourcing and coordinating them into an organisational workflow (la vecchia and cisternino 2010; lüttgens et al. 2014). furthermore, bpc also allows to standardise crowdsourcing processes. process standardisation can only be achieved by comprehending all related activities and their relationships. bpc that relies on both the individual and coordinated views provides a unique position for this comprehension. in that sense, bpc is expected to establish crowdsourcing as a repeatable organisational practice, and move crowdsourcing process to a more well-defined status. given this central role, there are many calls for further studying bpc, especially how to conceptualise it. vukovic et al. (2010) was among the earliest researchers that asked “how does crowdsourcing become an extension of the existing business process” (p. 7). khazankin et al. (2012a) echo the question and complained about “the lack of an integrated way to execute business processes based on a crowdsourcing [platform]” (p. 1). similarly, lüttgens et al. (2014) have recently emphasised the need to build a dedicated process for crowdsourcing. the need of bpc can also be implied from the fact that organisations have utilised crowdsourcing for some of their core, complex processes (djelassi and decoopman 2013; muntés-mulero et al. 2013), which should be necessarily linked with other internal business processes. australasian journal of information systems thuan, antunes & johnstone 2017, vol 21, research article a process model for establishing business process crowdsourcing 5 addressing these calls, we now move onto the bpc focus with a process model. only a few models/frameworks of crowdsourcing processes have been proposed in the field, but they focus more on technical aspects of crowdsourcing systems rather than the business processes operated on these systems (hetmank 2013). furthermore, most of the models proposed so far have not yet been empirically evaluated (amrollahi 2015). taking that into consideration, this study aims to propose and evaluate a process model establishing bpc. 3 a bpc process model the current study follows the design science paradigm to build and evaluate a process model for bpc (hevner and chatterjee 2010; hevner et al. 2004). laying out the foundations for this type of research, hevner et al. (2004) suggest that research should be based on a sound knowledge base, which can be drawn from three sources: scientific theories, existing artefacts, and experiences and expertise. yet, crowdsourcing is an emerging field lacking a strong theoretical background (zhao and zhu 2014), which makes it difficult, if not impossible at this stage, to rely on the first source. given that, the study must rely on the other sources of knowledge. adopting a design science method grounding existing artefacts and experience and expertise (thuan et al. 2015), the study begins with an inventory of existing artefacts and experience and expertise in the crowdsourcing field. for this purpose, we adopt a scoping method to extract domain knowledge from the existing crowdsourcing literature. we then synthesise the process model from the extracted domain knowledge. 3.1 scoping knowledge sources adopting a scoping review that enables a comprehensive view on the literature of a certain topic (paré et al. 2015), we analysed the crowdsourcing literature to identify the main components of the crowdsourcing process. the detailed steps are described in our conference paper, and interested readers are directed to (thuan et al. 2014). we summarise, here, its main outcomes. the scoping review started by systematically searching sources related to crowdsourcing from popular online bibliographic databases. we filtered out the irrelevant sources and sharpened the pool into 536 sources. the next step involved content analysis to identify relevant components of the crowdsourcing process, and to eliminate sources not contributing with at least one component. as a result, we ended up working with 238 reviewed sources. the reviewed sources were then analysed to identify the main components of crowdsourcing processes. in particular, we employed an analytical process based on coding (miles et al. 2014). we started with an abstract level of analysis, centred on three high-level activities: decision to crowdsource, design, and configuration. we then extracted, coded and analysed the main components and detailed features of crowdsourcing processes. coding was done iteratively, which required extracting, comparing, and merging data multiple times. the detailed coding process can be found in (thuan et al. 2014). 3.2 main stages and activities of crowdsourcing processes the end result of the analysis revealed a diversity of sub-activities and associated features, which were then aggregated into a set of logical components. we found more than 20 components, and identified the number of sources supporting each component. the ‘wisdom of researchers’ suggests taking into consideration the number of supporting sources to indicate the important components. when applying the ‘wisdom of researchers’, we chose a cut-off value where components with less number of supporting sources are filtered out. this choice faced a trade-off. if the cut-off value was low, many components would be selected and the model would become too complex, which is undesirable for a conceptual model (jonker and pennink 2010). in contrast, if the value was high, only a few components would be selected, which reduces the representation level of the model. testing several values, we finally chose 10 as the cut-off value that balances the aforementioned trade-off. consequently, we eliminated from the list components with less than 10 supporting sources. the final list is shown in table 1. details of these components are clarified in the next sections. australasian journal of information systems thuan, antunes & johnstone 2017, vol 21, research article a process model for establishing business process crowdsourcing 6 components no. of supporting sources (n>10) quality control 42 incentive mechanism 37 crowd management 32 task design 29 results aggregation 26 workflow design 25 capability & characteristics of crowdsourcing 23 task assignment 21 output 17 platform 16 technical configuration 16 circumstance to crowdsource & decision factors 16 table 1: main components of business process crowdsourcing 3.3 bpc model construction further analysis of the components identified in the previous sections lead us to construct the process model. we followed guidance from webster and watson (2002) for building conceptual models from extant literature. once again, the three high-level activities, i.e. the decision to crowdsource, design, and configuration, were used. more precisely, the components listed in table 1 were each allocated to one of the high-level activities. the allocations on the decision to crowdsource and configuration were straightforward, because they exhibit strong conceptual links. for instance, components ‘circumstance to crowdsource and decision factors’ and ‘characteristics of crowdsourcing’ are logically linked to the decision to crowdsource (thuan et al. 2016). ‘technical configuration’ is also clearly linked to the configuration activity. however, allocations to the design activity were more difficult since the links extracted from the reviewed sources are more diffuse and cover very different concerns. to help logically organise these components, we classified them into ‘what’ and ‘how’ categories, where the former focuses on identifying what work has to be done, and the latter focuses on how work is expected to be done. the ‘task design’ and ‘workflow design’ components are related to the ‘what’ category, while the remaining components, including ‘crowd management’, ‘quality control’ and ‘incentive mechanism’ relate to the ‘how’ category. in particular, crowd management includes profiling the members of the crowd, which, for instance, includes how to approach and manage the expected crowd; quality control includes mechanisms to control the quality; and the incentive mechanism concerns a decision on how to reward the crowd workers. using the classification described above, we allocated the ‘what’ components before the ‘how’ components, drawing from the conceptualisation of the outsourcing process described by dibber et al. (2004). these components were further expanded to comprise the activities that organisations go through as they progress through their crowdsourcing processes. this is depicted in figure 1 as the bpc process model. australasian journal of information systems thuan, antunes & johnstone 2017, vol 21, research article a process model for establishing business process crowdsourcing 7 1. decision to crowdsource decision factors (including capability of crowdsourcing) 3. technical configuration platform2d. quality control design-time run-time input output 2a. task design -task description (property definition) 2b. workflow design tasks decomposition results aggregation 2e. incentive mechanism intrinsic motivation extrinsic motivation 2c. crowd management profiling the crowd task assignment stage 1: decision to crowdsource stage 2: design stage 3: configuration figure 1: process model of business process crowdsourcing 3.4 bpc model description we now describe the process model in more detail. as seen in figure 1, the model adopts the input-process-output (pedersen et al. 2013) and stage-gate configurations (cooper 2008) that are typical of process models. more precisely, the model consists of seven components structured into three stages, which are described as follows. decision to crowdsource. the crowdsourcing process is triggered by an opportunity to crowdsource a piece of work, which starts with a decision to crowdsource (muhdi et al. 2011; wexler 2011). this component initially conceptualises the crowdsourcing strategy in order to “decide whether the crowdsourcing approach is appropriate to solve their internal problem/problems [tasks]” (muhdi et al. 2011, p. 322). the component is a logical antecedent to any crowdsourcing project, similar to the ‘make or buy’ decision in outsourcing projects (dibbern et al. 2004). by making it explicit in the model, we signal that the decision to crowdsource should be founded on a logical assessment of the project adequacy. to make a logical decision to crowdsource, organisations need to evaluate several contingency factors. thuan et al. (2016) have identified several factors influencing the decision to crowdsource. analysing 50 papers related to the decision, these authors suggest nine influencing factors, which have been structured into a comprehensive decisional framework considering the task, people, management, and environment. that study also derives a set of actionable guidelines for logically making the decision, which are completely applicable to our model. due to the limited space, we do not describe them here and point readers to the abovementioned reference. design. after the decision to crowdsource has been made, the stage 2 covers a set of design activities necessary to operationalise the decision. it includes five components: task design, workflow design, crowd management, quality control, and incentive mechanism. task design component transforms the conceptual ideas about the crowdsourcing tasks into concrete task descriptions (model component 2a). most of the reviewed sources suggest clearly defining what tasks are crowdsourced (malone et al. 2010; rosen 2011). the aim of this component is to designate a complete task description that can be given to the potential crowd members who may perform the tasks. to define these tasks, the task properties like significance, autonomy, variety, etc. suggested by tokarchuk et al. (2012) should be taken into account. the next component concerns the workflow design. this involves task decomposition and results aggregation (model component 2b). task decomposition divides the tasks into a set of smaller tasks. this activity has been suggested by several researchers to increase the potential number of workers interested in participating in the open call (afuah and tucci 2012; kulkarni et al. 2012). a counterpart to task decomposition is results aggregation. results aggregation describes how the outputs from the smaller tasks will be put together so that the objectives of australasian journal of information systems thuan, antunes & johnstone 2017, vol 21, research article a process model for establishing business process crowdsourcing 8 the overall task may be fulfilled (geiger et al. 2011). the results aggregation largely depends on task decomposition, as the function of the former is to reverse the results of the latter. kittur et al. (2013) explained this relationship that “facilitate[s] decomposing tasks into subtasks, managing the dependencies between subtasks, and assembling the results” (p. 5). crowd management is a design component that refers to how organisations manage the crowd members in order to accomplish the defined tasks (model component 2c). the reviewed sources suggest two sub-components of crowd management: profiling the crowd (allahbakhsh et al. 2012) and assigning tasks (khazankin et al. 2012b). first, organisations analyse the required capacity of crowd members for performing a task (allahbakhsh et al. 2012; kittur et al. 2013), and use this evaluation to build member profiles. based on these profiles, organisations can determine an overall picture of the crowd and may impose constraints to crowd recruitment (chandler and kapelner 2013; stewart et al. 2010). second, based on the crowd profiles, task assignment can be executed. that is, tasks can be assigned to crowd members who have appropriate profiles. examples of existing task assignment mechanisms include the auction-based mechanism (satzger et al. 2011) and the scheduled mechanism (khazankin et al. 2012b). according to table 1, quality control is the most frequently cited and thus should be regarded as a critical component (model component 2d). one distinctive characteristic of crowdsourcing is that tasks are performed by crowd members with very different backgrounds, skills and expertise (hirth et al. 2012). this sometimes leads to a number of low-quality contributions. thus quality control mechanisms are necessary to ensure the outputs meet the organisation’s quality goals (allahbakhsh et al. 2013; ipeirotis et al. 2010). by and large, quality control mechanisms can be classified into design-time and run-time (allahbakhsh et al. 2013). at design-time, organisations can design tasks and workflows in a robust way to increase the chances of receiving high-quality contributions (eickhoff and de vries 2013). at run-time, organisations can consider several active quality control mechanisms like expert reviews, peer reviews, gold standards, output agreements, and even majority voting (allahbakhsh et al. 2013). crowdsourcing relies on voluntary members of the crowd to perform tasks. thus, organisations need incentive mechanisms to attract and engage these voluntary members in their open calls (model component 2e). the reviewed sources suggest that incentive mechanisms should be developed based on two main types of motivation: intrinsic and extrinsic. for extrinsic motivation that focuses on workers’ external drives, most of the investigated sources have examined the adoption of financial incentives (kaufmann et al. 2011; mason and watts 2009). regarding intrinsic motivation that focuses on workers’ internal drives, a variety of factors have been suggested by the extant literature, such as fun (doan et al. 2011), meaningful tasks (chandler and kapelner 2013), and love of the community (kaufmann et al. 2011). configuration. the final stage focuses on how to configure a crowdsourcing process for instantiation in computational systems. since this activity mainly concerns an in-depth technical view (e.g. adopting specific architectures, technical frameworks, and computational platforms), the business perspective adopted by this study limits our considerations regarding this component to considering available crowdsourcing platforms. furthermore, we note that several tools supporting the technical aspects of the configuration process have already been proposed, including turkit (little et al. 2010), crowdforge (kittur et al. 2011), and bpmn4crowd (tranquillini et al. 2015). we expect that, in the near future, tools may be able to automatically transform a designed crowdsourcing process into an instantiation capable of running on specific crowdsourcing platforms. as a result, we regard the main output of this component as a configuration file containing a set of low-level details about the crowdsourcing process, which should be supported by existing crowdsourcing tools and platforms. australasian journal of information systems thuan, antunes & johnstone 2017, vol 21, research article a process model for establishing business process crowdsourcing 9 4 bpc model evaluation with case studies after the construction of the process model as a design science artefact, we now have to evaluate the proposed model. we decided to evaluate the model using case studies for three reasons. first, case studies enable artefacts to be evaluated in their practical environments. second, case studies can be used to explore the complex nature of crowdsourcing by providing in-depth, detailed explanations about their components and overall structure. yin (2013b) states that “for evaluations, the ability to address the complexity and contextual conditions nevertheless establishes case study methods as a viable alternative among the other methodological choices” (p. 322). third, case studies are considered appropriate for evaluating design science artefacts, as suggested by gill et al. (2015) regarding the evaluation of a social architecture framework. 4.1 approach for the case study evaluation, we considered the model regarding to two metrics: adequateness and utility. adequateness is defined as ‘the degree to which the components and their arrangement in the model align with the activities done in the studied crowdsourcing project’. we define utility as the ‘usefulness of the model as perceived by the study participants’. using these two metrics, we collected, analysed and compared data from two crowdsourcing projects. we followed the guidelines provided by yin (2013a; 2013b) for conducting case study evaluation research, including how to select cases, collect data, and analyse data. 4.2 case selection the selection of crowdsourcing projects was based on comparability and access to source material. first, we selected projects with a comparable team size, between 2 and 10 members. this range is sufficiently large to include multiple project roles, which the model aims to support, but not so large as to hold a diversity of settings that overshadow the evaluation purposes. second, we chose crowdsourcing projects where we had access to project participants and other data sources. as a result, two crowdsourcing projects were selected: crowd tagging (ct) and logo design contest (ldc). the ct project was part of a bigger project aiming to uncover the impact of new zealand predators on biodiversity in urban areas. this ct plan involved the installation of motiontriggered cameras in 40 locations in new zealand, which collected more than 65,000 pictures. ct aimed at identifying the animals captured in these pictures. because of the large number of pictures that needed to be analysed, the project launched a website with an open call to help tagging the pictures. project development involved a team of three members: project manager, web developer, and consultant. the call was live from june to december 2014. as a result, the project attracted over 300 users. about half of them tagged more than 20 pictures each. the other project ldc utilised the crowd for artistic design. a university in the mekong delta, vietnam was founded in 2013 from what began as a tertiary education centre. as a result of this transformation process, the university needed a new logo that would represent the spirit of the university. to design the logo, the university adopted a crowdsourcing approach that opened the logo design to designers from both inside and outside the university. it was in this spirit that the ldc project was created. the project started in may 2013 and finished in december 2013, when the winning logo was officially adopted by the university. when the project was launched, it received 68 logo designs from the crowd. three of them were selected and declared as the winning solutions: two were awarded for creative prizes and one was awarded for the final winning solution, which is the current logo of the university. 4.3 data collection we collected data from multiple sources, both primary and secondary. secondary sources included press releases, the open calls, meeting reports, and project websites, all of which provided materials necessary to clarify key project activities. the activities and their relationships were further detailed and validated in interviews. across the two case studies, we conducted three key informant interviews with project leaders and other participants, both australasian journal of information systems thuan, antunes & johnstone 2017, vol 21, research article a process model for establishing business process crowdsourcing 10 face-to-face and through skype. due to the small size of the project teams, these interviewees had to ‘wear many hats’ and therefore could provide insights into several perspectives of the crowdsourcing projects. besides being interviewed about the activities performed in the projects, the interviewees were asked to analyse a printed version of the model proposed in this paper and were asked to make a judgment and produce comments about the usefulness of the model. a summary of demographic information about the cases is presented in table 2. crowd tagging logo design contest number of project members 3 10 project duration 6 months 7 months project purpose to tag pictures about animals in new zealand to design a logo for the university interviews 1 2 roles of interviewees project leader project leader project coordinator other data sources press and media website, tutorial internal documents (e.g. example submissions) press and media internal documents (e.g. meeting reports, example submissions) table 2: demographic information 4.4 empirical analysis to prepare data for analysis, we first arranged a full description of each case, including details about the project, project team, and project activities. we then used the model proposed in this paper to map the project activities into the model components, while critically analysing the interviewees’ comments about the model. more precisely, this empirical analysis included the two following activities. adequateness analysis: this analysis followed a pattern matching technique (yin 2013a), which looked for major similarities, patterns, and notable differences between the model and the activities reported for each project. to begin, we examined each case description for phases indicating the activities of crowdsourcing processes. we marked phases or sentences describing the project activities, and matched them to the model components. success to match suggested a similarity, while failure to match suggested a potential difference. to maintain the independence, matching was undertaken by the first author and checked by the other authors. this process generated a list of tentative matching patterns. the matching patterns were then cross-checked using the interviews and supplementary materials, which allowed us to refine the existing patterns and to identify additional patterns that emerged from the interviews. as a result, the final list of matching patterns (both similarities and differences) was created, allowing us to finally map the project activities in the model for comparing between them (presented in figures 2 and 3 below). utility analysis: we gathered judgements and comments from the interviewees regarding the perceived utility of the model. during the interviews, we asked ‘what do you think about the model components?’ the interviews were analysed using template analysis, a simple and well-known procedure (king 2012). the analysis started with a few predefined codes related to utility, including ‘usefulness’, ‘future use’ and ‘future improvement’, which were then applied to the interview transcripts. we reviewed the transcripts and identified transcript snippets relevant to our codes. during this process, some emerging codes were created to further capture the judgements made by the interviewees. the codes were then aggregated and synthesised into salient templates, which will be discussed in the results section. australasian journal of information systems thuan, antunes & johnstone 2017, vol 21, research article a process model for establishing business process crowdsourcing 11 4.5 validity validity is critical factor in qualitative evaluation since it allows judging levels of accuracy and strengthening knowledge claims that the research represents. yin (2013a) shares similar views on determining validity in case studies when considering construct validity, internal validity, external validity, and reliability. he further suggests several useful techniques for establishing validity. the current study was already applied the techniques, summarised in table 3. construct validity  use multiple sources of evidence  case study reports reviewed by key project members external validity  use literal replications to choose the cases internal validity  insights from the crowdsourcing literature  pattern matching in data analysis reliability  use case study protocol  develop and maintain case a study database table 3: study validity 4.6 case study results the case study results are structured according to the two investigated metrics, adequateness and perceived usefulness. 4.6.1 adequateness of the model to report on model adequateness, we graphically represent the project activities of the two cases using the model as a baseline. this highlights not only the similarities but also the differences between our model and the investigated projects. figures 2 and 3 summarise the projects’ activities. differences with the model are represented in italic; sub-activities are presented in smaller font size and marked with ‘+’. 1. decision to crowdsource decision factors + limited employee for tasks + wider community + increase environmental awareness 3. technical configuration build website 2d. quality control run-time + expert evaluation + asking confidence level input output 2a. task design task description tutorials 2b. workflow design tasks decomposition + cluster of three pictures + three pools of pictures 2e. incentive mechanism instrinsic motivation + meaningfulness 2c. crowd management (partly perform) profiling the crowd + sign up figure 2: activities of crowd tagging (ct) australasian journal of information systems thuan, antunes & johnstone 2017, vol 21, research article a process model for establishing business process crowdsourcing 12 1. decision to crowdsource decision factors + outside experts + diverse solutions 3. technical configuration existing website to publish tasks, not perform tasks 2d. quality control run-time + expert evaluation + cheating detectioninput output 2a. task design task description 2b. workflow design 4-step workflow results aggregation + voted by committees 2e. incentive mechanism extrinsic motivation + monetary rewards + to-be-recognised 2c. crowd management (partly perform) profiling the crowd + only in the second round figure 3: activities of logo design contest (ldc) based on this graphical representation, we observe high adequateness of the model components. both representations show strong concordance between the model components and the projects’ activities. examples include the strong alignment on decision to crowdsource, task design, workflow design, incentive mechanism, quality control, and partial alignment on crowd management and technical configuration. several project sub-activities are also aligned with the model. however, both cases reveal several additional activities that were necessary to instantiate the components in practice. examples include developing a tutorial in the task design of the ct case, and aggregating results through voting in the workflow design of the ldc case. nevertheless, we find a strong alignment between the model components and the two projects, which suggests high adequateness. regarding specifically the interdependencies suggested by the model, the two investigated projects are also largely aligned, i.e. they generally adopt the sequence of steps from input, decision to crowdsource, several aspects of crowdsourcing design, configuration, and finally to output. this alignment is stronger in the ldc case where most components follow the model sequence. in the ct case, we find strong alignment in the first four components, but some differences in the relationships among the last three components. more precisely, the last three were developed in a more iterative way, rather than following a sequential relationship. more details about the activities’ interdependencies are presented below. crowd tagging (ct) the ct project started with an input consisting of a large number of pictures to be analysed. to process these pictures, the project manager decided to adopt crowdsourcing. he stated three supporting reasons: 1) limited human resources to process the vast amount of data; 2) allowing wider community to access the collected data; and 3) increasing environmental awareness of the community. the first reason, which was considered the most important decision factor by the project manager, examined the lack of internal employees to perform the task (afuah and tucci 2012; malone et al. 2010; thuan et al. 2016). reason 1 is consistent with the ‘decision to crowdsource’ model component, while reasons 2 and 3 were specific to the nature of ct as a citizen science project. after deciding to crowdsource, the project manager specified the crowdsourcing process itself, starting with task design. a task description was developed to promote the general aims of the project and explaining how the task could be fulfilled by the crowd: “this research aims to evaluate the use of remote cameras to estimate abundances of non-native predators in urban environments. you will be shown a series of images, taken earlier this year, from various cameras placed around the wellington city and asked to identify the animal in the photograph” [ct, website]. the task design is consistent with the model component 2a. besides, we note the project included a tutorial and a visual explanation of the task, which served to train the australasian journal of information systems thuan, antunes & johnstone 2017, vol 21, research article a process model for establishing business process crowdsourcing 13 crowd on how to perform the tagging. such focus on training seems appropriate for this type of task, and research suggests training the crowd may improve results (park et al. 2014). the ct project designed the crowdsourcing workflow through task decomposition. first, the whole activity was divided into sub-tasks of tagging three pictures, which the project alluded to as a cluster. this clustering was directly related to how data was collected in the project: “the camera takes three pictures every time they detect something. thus, the group of three pictures helps make the task easier to perform” [ct, project manager]. the project also divided the whole set of pictures into three pools: sign-up pool, working pool, and finished pool. the signup pool had 20 clusters (of three related pictures). the ideas was that a user who just signedup for tagging would start with this small pool. the tagging work on the sign-up pool would be scrapped as it served just for gaining experience. after a user finished ten clusters from the sign-up pool, the website would direct the user to the working pool. this pool includes the remaining pictures that need to be tagged, and thus is the main working zone. when a cluster had been tagged more than three times, it was considered finished and moved to the finished pool. repeated tagging (three times) would help increase task quality. while the task decomposition was quite well organised in the ct case, results aggregation was not clearly specified by the project and was considered future work by the interviewee. this model component was missing in the ct project. moving to the next component, crowd management included gathering users’ demographic information and task allocation. ct collected demographic information at sign-up. the project assessed the task performance based on the sign-up pool, by comparing the users’ tags with known correct answers. this allows the project making decision whether a user should work on the sign-up pool for training, or the working pool for actually performing the tasks. yet, crowd management in ct is quite preliminary since the project did not build complete user profiles, which could be done if the project had built a metric to assess user performance. since tagging was performed by voluntary users, there was no guarantee that the results would be of high quality. quality control is recommended by the model as necessary for projects similar to the ct case. in fact, two run-time quality control mechanisms were implemented in the ct project. the first mechanism asked the users directly how confident they were about tagging in order to identify the reliability level. the second mechanism was based on expert evaluation after all tags were received from the crowd. the choice of these mechanisms led to a concern that quality control “heavily depended on one person’s opinions” [ct, project manager]. as previously noted, the project did not address the concern and left it to future work. to attract the crowd, project manager considered both extrinsic and intrinsic incentive mechanisms. however, he finally decided not to use the former so as he believed the users would be intrinsically keen enough to contribute to the citizen science project. as a result, the project was mainly based on intrinsic incentives. similar to other citizen science projects (brabham 2012), ct attracted participants who wished to make an altruistic contribution to science; as stated in the website “every image you tag will help us to better understand the relationships between new zealand’s invasive mammals and native species” [ct, website]. regarding technical configuration, the project built a dedicated website to publicize the open call. this website also functioned as a crowdsourcing platform, managing pools and clusters, distributing pictures and collecting tags. the decision to develop a dedicated website was that the project members wanted to have full control over the entire set of crowdsourcing activities. logo design contest (ldc) in the ldc case, the decision to crowdsource was based on two main reasons. the first reason was the ability of the crowd to provide diverse and innovative solutions. “the university has decided to conduct the open contest to find ideas that are standard [i.e. meeting requirements] and creative” [ldc, project manager]. this is consistent with other crowdsourcing cases where open calls seek unique and innovative ideas (brabham 2010; leimeister et al. 2009). a second influence on the crowdsource decision was to call for designers from both inside and outside australasian journal of information systems thuan, antunes & johnstone 2017, vol 21, research article a process model for establishing business process crowdsourcing 14 the university. interestingly, reduced costs (compared to hiring a design expert) was not considered as an important factor in the decision to crowdsource. task design considered the defining of requirements for the logo, terms and conditions to join the contest, submission deadline, and prizes. within these elements, the requirements played a major role as they specified what the solution should look like (zheng et al. 2011). however, it seems that task description in ldc did not fully elucidate what had to be done, which led to an extension of the contest due to several requests to clarify requirements. the workflow design was an interesting activity in the ct project with two distinctive aspects. first, while our model suggests task decomposition, the ldc project did not decompose the task. this can be explained by the nature of the logo design task, which seems difficult to break down into smaller tasks. crowdsourcing a whole task has been already adopted in several design contests. examples include a bus stop shelter design (brabham 2012) and t-shirt design (howe 2006). second, ldc published its workflow description in the open call. according to the open call, the project workflow consisted of four steps: 1) the crowd submits their solutions (round 1); 2) a preliminarily evaluation is conducted by the board; 3) based on the board evaluation, a short-list of submissions is chosen, given feedback, and then re-submissions are allowed (round 2); and 4) the final submissions are evaluated, voted, ranked, and awarded, which is referred to as result aggregation in the model. this workflow description provided transparency to the process by explaining exactly what would happen during the project. crowd management was not a focus in the ldc case. the project did not assign tasks to any specific members or even profiled the crowd. the ldc project only profiled the participants in the second round, when a subset of submissions was already chosen. this was considered a project limitation: “the management of crowd information was limited, which may be because we did not specify rules about providing information” [ldc, project coordinator]. to control quality, the ldc project relied on expert evaluation (zhao and zhu 2014). in particular, the board that aggregated results was also the board that assessed quality and provided feedback to the participants. since the number of submissions was not large (68 in total), this revealed an effective mechanism. however, the project found a few cheating submissions that were likely copied from logos of other organisations. these cases were identified by external experts on the board, who have long experience with logo design. the project adopted extrinsic mechanisms to attract the participants, which consisted of monetary rewards and recognition by the organisation. like other contests, the monetary rewards were only provided for the best solutions, which included two creative prizes and one top winner prize. the creative and top winner prizes were quite valuable, relative to the living standard in the area, equivalent to one and five month’s salary of a typical office worker, respectively. another extrinsic motivation for the participants was that the project announced the winners on the university website, which is a recognition motivator (brabham 2012). both types of motivations were clearly aligned with our model. the technical configuration was rather poor in this project, as it only used the website as a channel to publish the open call of the contest. email was used to receive the submissions. this was because the project members were not aware of existing technology supporting crowdsourcing contests. overall, the two cases suggested the adequacy of the proposed model to describe the project activities. adequateness was indicated through the highly matching between the model components and the project activities. adequateness was further suggested through the participant interviews. when we showed graphical representations of the project using the model, the participants viewed the model to be aligned with their own representations. as one participant noted, “we may miss some of the points, but we touch all of them [all model components]” [ct, project manager]. australasian journal of information systems thuan, antunes & johnstone 2017, vol 21, research article a process model for establishing business process crowdsourcing 15 4.6.2 perceived utility of the model we analyse the utility of the model based on the interviewees’ perception of how useful it seems to help planning and running the project. this is mainly based on’ judgements and comments of project leaders and coordinators. in general, the result from the analysis suggest the model to be a useful tool for crowdsourcing. this was demonstrated by the following comments. “i think it will be nice to follow the model. […]. yes, i want to use the model, following this flow or at least have something to follow” [ct, project manager] “the model is very well constructed and all of its activities should be necessarily for the project” [ldc, coordinator] “as i said, i think this model is totally suitable” [ldc, project manager] finding the usefulness of the model, these participants were extremely enthusiastic about applying the model for the future crowdsourcing projects: “i think that any future crowdsourcing projects should apply strictly these steps, which will create better results” [ldc, coordinator] “from my opinion, the model can be suitable for many activities that need the resources from the crowd” [ldc, project manager] interestingly, we found slightly different views between the project manager and coordinator roles over what aspects of the model were most useful. for instance, in the ldc case, while the project manager viewed the model as a tool for making decisions, the project coordinator instead stressed the role of the model in supporting communication among project members and in achieving a consensus. these differences suggest the model’s utility was not confined to one particular role. in summary, we conducted two case studies evaluating the process model. the results of the case studies found evidence that the model can represent the key activities of crowdsourcing projects. within each project, the model components were demonstrated and instantiated in the project contexts, although small differences in the instantiation could be found. furthermore, we also obtained evidence of the perceived usefulness of the model, inspired by the reception of the key project informants. 5 discussion multiple efforts conceptualising the crowdsourcing process have been reported in the literature. most of them, however, view crowdsourcing processes as one-off activities, which are difficult to establish and reproduce (amrollahi 2015). the issue of establishing crowdsourcing as a repeatable process becomes more pressing, since the crowdsourcing strategy has recently been integrated with some core organisational processes. this paper introduces a bpc lens on conceptualising crowdsourcing processes. while the term bpc has been used previously (la vecchia and cisternino 2010), it is this paper that elevates and defines the bpc concept. we suggest that bpc is best conceptualised as balancing between the business process construct and the crowdsourcing construct. regarding the business process construct, bpc serves as a template for creating multiple, real life instances of the same crowdsourcing process. the bpc conceptualisation can only stand with the condition that there are common repeatable activities of existing crowdsourcing processes. in this research, the condition has been met. our scoping review found a set of common activities in crowdsourcing processes, repeatedly pointed out by multiple knowledge sources (table 1). these common activities, which have been supported by other recent reviews (amrollahi 2015; hosseini et al. 2015), enable us to confirm the condition backing the bpc concept. further, these common activities also suggest the basic building blocks of bpc. australasian journal of information systems thuan, antunes & johnstone 2017, vol 21, research article a process model for establishing business process crowdsourcing 16 using the building blocks identified by the scoping review, we then conceptualised bpc through a process model. the model, on the one hand, sharpens the bpc concept, using a process viewpoint that is well known to most organisations. on the other hand, the model highlights the core repeatable building blocks of bpc. with this focus, the model defines the abstract structure of bpc while treating new crowdsourcing projects as real life instances of the same core building blocks (figure 1). all in all, the proposed process model brings the bpc concept into a space that is quite distinct from the one-off crowdsourcing endeavours (lüttgens et al. 2014; stol and fitzgerald 2014). the evaluation of the bpc process model in two existing crowdsourcing projects allowed us to test the representation and utility of the proposed model in practical environments. the case study analysis suggested high adequateness and usefulness of the bpc model in identifying the main activities of crowdsourcing processes. it can be observed from the results that the bpc model lays an abstract foundation of how organisations might rationally analyse and manage crowdsourcing projects. furthermore, the case study results, to some extent, give insights into crowdsourcing project activities, which demonstrates how the model can be applied in practice. we note however some limitations in the current study. regarding the build activity, the model was constructed based on data gathered from a scoping review, and thus eventual limitations of these sources may also extend to this study (kitchenham 2007). we partially addressed this concern by applying the ‘wisdom of researchers’, which is further discussed here to clarify its role in the current study. founding on the ‘wisdom of the crowd’, the ‘wisdom of researchers’ holds the following four conditions: diversity, independence, decentralisation, and aggregation, which collectively allows the crowd to be wiser than an individual expert (surowiecki 2004). in particular, we ensured diversity of knowledge by searching sources from eight online bibliographic databases. our review was also based on independent sources of knowledge. we did not restrict the analysis to any particular theory or viewpoint; instead, we allowed the model to emerge from the extant literature. decentralisation is a distinctive characteristic of the crowdsourcing field, which is noted for mixing different research fields (geiger and schader 2014). surowiecki (2004) defined aggregation as a mechanism turning individual opinions into a final outcome. such view supports the empirical decision to only select model components that received support from at least 10 sources. because all four conditions hold, the application of the ‘wisdom of researchers’ enabled us to build a model faithfully representing bpc. regarding the evaluation by case study, the selection of two case studies could also be considered as a limitation, since generalisation to a broader context may be difficult. that applies to every case study. we note however that the generalisation for case studies is obviously not statistical but analytical one, where our model was used as an analytical framework for investigating the empirical results of crowdsourcing projects (rowley 2002). because of the characteristics inherent to the case study approach, the model by no means provides a generalised account of crowdsourcing projects. rather, it is abstract. this could be seen via the small differences among the detailed activities mapped in the two investigated projects. we note that on the other hand the abstract characteristic of the model is also its strength, as it can be applied to and adapted in different crowdsourcing project settings. 6 conclusions and future work we began our study with the observation that more and more organisations are adopting crowdsourcing within their business. thus, there is a strong need for a model to guide organisations to establish repeatable crowdsourcing processes (djelassi and decoopman 2013; lüttgens et al. 2014). addressing this gap, we developed a model allowing organisations to structure and manage the main building blocks necessary to establish the crowdsourcing strategy. we evaluated the model in two existing projects. results, to a certain extent of generalisation, indicate the model is useful in identifying and articulating the main crowdsourcing activities. australasian journal of information systems thuan, antunes & johnstone 2017, vol 21, research article a process model for establishing business process crowdsourcing 17 from an academic point of view, our study adopted a broad perspective on what activities need to be considered when planning, designing and instantiating crowdsourcing processes, thus overcoming the “excessive ad-hoc-ness” criticism found in the crowdsourcing literature (geiger and schader 2014; man-ching et al. 2011). although a few models and frameworks have already been proposed (amrollahi 2015; hetmank 2013), the current study is the first to both conceptualise and evaluate a bpc model. from a practical point of view, the proposed model suggests several components that need to be articulated when establishing crowdsourcing processes. this provides a blueprint for managers and process designers, guiding them in their crowdsourcing projects. from a research point of view, our work extends the existing research in three aspects. first, the model construction was based on a comprehensive review, in terms of number of reviewed sources. we analysed 238 papers, compared to 39 papers in a model construction by amrollahi (2015). second, we allowed the model components to freely emerge from the literature, which introduced some new important components: ‘the decision to crowdsource’, ‘workflow design’, and ‘task assignment’. finally, the model was empirically evaluated using a case study approach. while our case study findings were mainly indicative, they moved forward the theoretical efforts conceptualising the crowdsourcing process (amrollahi 2015; hetmank 2013; thuan et al. 2014). future work is needed to confirm the utility of the model across different crowdsourcing contexts. this confirmation can be done using a mix of evaluation techniques. we have also started developing a decision tool which will operationalise the model and provide further support to project managers and process designers. this development will contribute to a better understanding of the model interdependencies and their impact in crowdsourcing projects (miah and gammack 2014). another interesting future research direction is to consider the model in the context of crowdsourcing instantiation, and in particular integration with existing crowdsourcing platforms. although the model captures the main activities of crowdsourcing processes, it would be interesting to further study how the model could be used to automate process instantiations in specific crowdsourcing platforms. giachetti (2004), for example, suggests four levels of focus related to such integration, which comprehend business processes, applications, data, and networks. thus, future research should provide further detail about the model regarding information structures, data structures, and technical structures. references afuah, a., and tucci, c.l. 2012. "crowdsourcing as a solution to distant search" academy of management review,37 (3). 355-375. allahbakhsh, m., benatallah, b., ignjatovic, a., motahari-nezhad, h.r., bertino, e., and dustdar, s. 2013. "quality control in crowdsourcing systems: issues and directions" internet computing, ieee,17 (2). 76-81. allahbakhsh, m., ignjatovic, a., benatallah, b., beheshti, s., bertino, e., and foo, n. 2012. "reputation management in crowdsourcing systems," 8th international conference on collaborative computing: networking, applications and worksharing (collaboratecom), pp. 664-671. amrollahi, a. 2015. "a process model for crowdsourcing: insights from the literature on implementation" proceedings of the australasian conference on information systems 2015,paper 18. bonabeau, e. 2009. "decisions 2.0: the power of collective intelligence" mit sloan management review,50 (2). 45-52. brabham, d.c. 2010. "moving the crowd at threadless" information, communication & society,13 (8). 1122-1145. australasian journal of information systems thuan, antunes & johnstone 2017, vol 21, research article a process model for establishing business process crowdsourcing 18 brabham, d.c. 2012. "motivations for participation in a crowdsourcing application to improve public engagement in transit planning" journal of applied communication research,40 (3). 307-328. brabham, d.c. 2013. crowdsourcing. canbridge, ma: the mit press. chandler, d., and kapelner, a. 2013. "breaking monotony with meaning: motivation in crowdsourcing markets" journal of economic behavior & organization,90. 123-133. cooper, r.g. 2008. "perspective: the stage‐gate® idea‐to‐launch process—update, what's new, and nexgen systems*" journal of product innovation management,25 (3). 213232. dibbern, j., goles, t., hirschheim, r., and jayatilaka, b. 2004. "information systems outsourcing: a survey and analysis of the literature" acm sigmis database,35 (4). 6102. djelassi, s., and decoopman, i. 2013. "customers' participation in product development through crowdsourcing: issues and implications" industrial marketing management,42 (5). 683-692. doan, a., ramakrishnan, r., and halevy, a.y. 2011. "crowdsourcing systems on the worldwide web" communications of the acm,54 (4). 86-96. eickhoff, c., and de vries, a. 2013. "increasing cheat robustness of crowdsourcing tasks" information retrieval,16 (2). 121-137. geiger, d., and schader, m. 2014. "personalized task recommendation in crowdsourcing information systems—current state of the art" decision support systems,65. 3–16. geiger, d., seedorf, s., schulze, t., nickerson, r.c., and schader, m. 2011. "managing the crowd: towards a taxonomy of crowdsourcing processes" proceedings of the seventeenth americas conference on information systems. paper 430. giachetti, r.e. 2004. "a framework to review the information integration of the enterprise" international journal of production research,42 (6). 1147-1166. gill, a.q., alam, s.l., and eustace, j. 2015. "social architecture: an emergency management case study" australasian journal of information systems,19. 23-40. hetmank, l. 2013. "components and functions of crowdsourcing systems–a systematic literature review," in: 11th international conference on wirtschaftsinformatik. leipzig, germany. hevner, a., and chatterjee, s. 2010. design research in information systems: theory and practice. integrated series in information systems. springer berlin heidelberg. hevner, a., march, s.t., park, j., and ram, s. 2004. "design science in information systems research" mis quarterly,28 (1). 75-105. hirth, m., hoßfeld, t., and tran-gia, p. 2012. "analyzing costs and accuracy of validation mechanisms for crowdsourcing platforms" mathematical and computer modelling,57 (11-12). 2918-2932. hosseini, m., phalp, k., taylor, j., and ali, r. 2015. "on the configuration of crowdsourcing projects" international journal of information system modeling and design (ijismd),6 (3). 27-45. howe, j. 2006. "the rise of crowdsourcing," in: wired magazine 2006. dorsey press, pp. 14. ipeirotis, p.g., provost, f., and wang, j. 2010. "quality management on amazon mechanical turk" proceedings of the acm sigkdd workshop on human computation. 64-67. australasian journal of information systems thuan, antunes & johnstone 2017, vol 21, research article a process model for establishing business process crowdsourcing 19 jonker, j., and pennink, b.w. 2010. "conceptual models," in: the essence of research methodology. springer berlin heidelberg, pp. 43-63. kaufmann, n., schulze, t., and veit, d. 2011. "more than fun and money. worker motivation in crowdsourcing–a study on mechanical turk" proceedings of the seventeenth americas conference on information systems, detroit, mi. paper 340. khazankin, r., satzger, b., and dustdar, s. 2012a. "optimized execution of business processes on crowdsourcing platforms" ieee 8th international conference on collaborative computing: networking, applications and worksharing. 443 451. khazankin, r., satzger, b., and dustdar, s. 2012b. "predicting qos in scheduled crowdsourcing," in: advanced information systems engineering, j. ralyté, x. franch, s. brinkkemper and s. wrycza (eds.). springer berlin heidelberg, pp. 460-472. king, n. 2012. "doing template analysis" qualitative organizational research: core methods and current challenges. 426-450. kitchenham, b. 2007. "guidelines for performing systematic literature reviews in software engineering" ver. 2.3 ebse technical report. kittur, a., nickerson, j., bernstein, m., gerber, e., shaw, a., zimmerman, j., lease, m., and horton, j. 2013. "the future of crowd work" proceedings of the 2013 conference on computer supported cooperative work. kittur, a., smus, b., khamkar, s., and kraut, r.e. 2011. "crowdforge: crowdsourcing complex work" proceedings of the 24th annual acm symposium on user interface software and technology. 43-52. kulkarni, a., can, m., and hartmann, b. 2012. "collaboratively crowdsourcing workflows with turkomatic" proceedings of the acm 2012 conference on computer supported cooperative work. 1003-1012. la vecchia, g., and cisternino, a. 2010. "collaborative workforce, business process crowdsourcing as an alternative of bpo," in: icwe2010. lncs, f. daniel and f.m. facca (eds.). springer, pp. 425-430. leimeister, j.m., huber, m., bretschneider, u., and krcmar, h. 2009. "leveraging crowdsourcing: activation-supporting components for it-based ideas competition" journal of management information systems,26 (1). 197-224. little, g., chilton, l.b., goldman, m., and miller, r.c. 2010. "turkit: human computation algorithms on mechanical turk" proceedings of the 23nd annual acm symposium on user interface software and technology. 57-66. lüttgens, d., pollok, p., antons, d., and piller, f. 2014. "wisdom of the crowd and capabilities of a few: internal success factors of crowdsourcing for innovation" journal of business economics,84 (3). 339-374. malone, t.w., laubacher, r., and dellarocas, c. 2010. "the collective intelligence genome" ieee engineering management review,38 (3). 3852. man-ching, y., king, i., and kwong-sak, l. 2011. "a survey of crowdsourcing systems" 2011 ieee third international conference on privacy, security, risk and trust (passat), and 2011 ieee third international conference on social computing (socialcom). 766-773. mason, w., and watts, d.j. 2009. "financial incentives and the "performance of crowds"" proceedings of the acm sigkdd workshop on human computation. 77-85. miah, s.j., and gammack, j.g. 2014. "ensemble artifact design for context sensitive decision support" australasian journal of information systems,18 (2). miles, m.b., huberman, a.m., and saldaña, j. 2014. qualitative data analysis: a methods sourcebook. sage publications, incorporated. australasian journal of information systems thuan, antunes & johnstone 2017, vol 21, research article a process model for establishing business process crowdsourcing 20 muhdi, l., daiber, m., friesike, s., and boutellier, r. 2011. "the crowdsourcing process: an intermediary mediated idea generation approach in the early phase of innovation" international journal of entrepreneurship and innovation management,14 (4). 315332. muntés-mulero, v., paladini, p., manzoor, j., gritti, a., larriba-pey, j.l., and mijnhardt, f. 2013. "crowdsourcing for industrial problems," in: citizen in sensor networks. lncs, j. nin and d. villatoro (eds.). springer berlin heidelberg, pp. 6-18. paré, g., trudel, m.-c., jaana, m., and kitsiou, s. 2015. "synthesizing information systems knowledge: a typology of literature reviews" information & management,52. 183199. park, s., shoemark, p., and morency, l.-p. 2014. "toward crowdsourcing micro-level behavior annotations: the challenges of interface, training, and generalization" proceedings of the 19th international conference on intelligent user interfaces. 37-46. pedersen, j., kocsis, d., tripathi, a., tarrell, a., weerakoon, a., tahmasbi, n., jie, x., wei, d., onook, o., and de vreede, g.j. 2013. "conceptual foundations of crowdsourcing: a review of is research" 46th hawaii international conference on system sciences (hicss). 579-588. rosen, p.a. 2011. "crowdsourcing lessons for organizations" journal of decision systems,20 (3). 309-324. rouse, a.c. 2010. "a preliminary taxonomy of crowdsourcing" proceedings of the 21st australasian conference on information systems. paper 76. rowley, j. 2002. "using case studies in research" management research news,25 (1). 16-27. satzger, b., psaier, h., schall, d., and dustdar, s. 2011. "stimulating skill evolution in marketbased crowdsourcing," in: bpm 2011. lncs, s. rinderle-ma, f. toumani and k. wolf (eds.). springer berlin heidelberg, pp. 66-82. saxton, g.d., oh, o., and kishore, r. 2013. "rules of crowdsourcing: models, issues, and systems of control" information systems management,30 (1). 2-20. schenk, e., and guittard, c. 2011. "towards a characterization of crowdsourcing practices" journal of innovation economics,7 (1). 93-107. stewart, o., lubensky, d., and huerta, j.m. 2010. "crowdsourcing participation inequality: a scout model for the enterprise domain" proceedings of the acm sigkdd workshop on human computation. 30-33. stol, k.-j., and fitzgerald, b. 2014. "two's company, three's a crowd: a case study of crowdsourcing software development" proceedings of the 36th international conference on software engineering. 187-198. surowiecki, j. 2004. the wisdom of crowds: why the many are smarter than the few and how collective wisdom shapes business. new york: doubleday. thuan, n.h., antunes, p., and johnstone, d. 2014. "toward a nexus model supporting the establishment of business process crowdsourcing," in: fdse 2014. lncs, t.k. dang, r. wagner, e. neuhold, m. takizawa, j. küng and n. thoai (eds.). springer, heidelberg, pp. 136–150. thuan, n.h., antunes, p., and johnstone, d. 2015. "a design science method for emerging decision support environments" proceedings of the australasian conference on information systems 2015,paper 32. thuan, n.h., antunes, p., and johnstone, d. 2016. "factors influencing the decision to crowdsource: a systematic literature review" information systems frontiers,18 (1). 47-68. australasian journal of information systems thuan, antunes & johnstone 2017, vol 21, research article a process model for establishing business process crowdsourcing 21 tokarchuk, o., cuel, r., and zamarian, m. 2012. "analyzing crowd labor and designing incentives for humans in the loop" ieee internet computing magazine,16 (5). 45-51. tranquillini, s., daniel, f., kucherbaev, p., and casati, f. 2015. "modeling, enacting, and integrating custom crowdsourcing processes" acm transactions on the web (tweb),9 (2). article 7. van der aalst, w., and hee, k.m. 2004. workflow management: models, methods, and systems. cambridge, ma: the mit press. venable, j., pries-heje, j., and baskerville, r. 2012. "a comprehensive framework for evaluation in design science research" design science research in information systems. advances in theory and practice. 423-438. vukovic, m., and bartolini, c. 2010. "towards a research agenda for enterprise crowdsourcing," in: leveraging applications of formal methods, verification, and validation. lncs, t. margaria and b. steffen (eds.). springer berlin heidelberg, pp. 425434. vukovic, m., laredo, j., and rajagopal, s. 2010. "challenges and experiences in deploying enterprise crowdsourcing service," in: icwe 2010. lncs, b. benatallah, f. casati, g. kappel and g. rossi (eds.). springer berlin heidelberg, pp. 460-467. webster, j., and watson, r.t. 2002. "analyzing the past to prepare for the future: writing a literature review" mis quarterly,26 (2). xiii-xxiii. wexler, m.n. 2011. "reconfiguring the sociology of the crowd: exploring crowdsourcing" international journal of sociology and social policy,31 (1/2). 6-20. yin, r.k. 2013a. case study research: design and methods. sage publications. yin, r.k. 2013b. "validity and generalization in future case study evaluations" evaluation,19 (3). 321-332. zhao, y., and zhu, q. 2014. "evaluation on crowdsourcing research: current status and future direction" information systems frontiers,16 (3). 417-434. zheng, h., li, d., and hou, w. 2011. "task design, motivation, and participation in crowdsourcing contests" international journal of electronic commerce,15 (4). 57-88. copyright: © 2017 thuan, antunes & johnstone. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ microsoft word ajis_18(3)_v10.docx australasian journal of information systems volume 18 number 3 2014 53 university staff adoption of ipads: an empirical study using an extended technology acceptance model michael lane university of southern queensland michael.lane@usq.edu.au adrian stagg university of southern queensland adrian.stagg@usq.edu.au abstract the adoption of ipads in the university sector has been widespread, with attention predominantly being directed to students’ adoption and use of ipads. however, there is a lack of empirical research which has critically examined the key factors influencing university staff adoption of ipads. the main objective of this research is to empirically examine key factors influencing adoption of ipads by university staff. an online survey was used to collect quantitative data to validate the measurement model and test hypothesised relationships in an extended tam. the findings from the results of the data analyses show that university staff consider ipads to be easy to use and useful, with a high level of compatibility with their work. social status had no influence on the attitude of university staff to using an ipad in their workplace. however, older university staff and/or university staff with no previous experience in using a similar technology such as an iphone or smartphone found ipads less easy to use. furthermore, a lack of formal university ict support for end-users impacted negatively on the use of ipads by university staff. this research contributes to theory by validating and establishing support for an extended tam which is contextualized for ipad adoption by university staff. this model has practical application because it is generalisable for a range of mobile computing devices and can be used to evaluate the adoption and use of mobile computing devices in a range of industries beyond the university sector. keywords: technology acceptance model, ipad, social status, age, previous experience with similar technology, ict support introduction mobile computing and e-learning continues to drive the focus of higher education course design and delivery (caudill 2007; murphy et al. 2013). hence, there is an increasing need for university staff to adopt mobile devices such as ipads for experimentation, professional development and implementation within their work practices, including administration, research and teaching. thus, the current paradigm shift in both technology and mobile computing challenges higher education providers to effectively leverage the multi-functionality of mobile devices such as smart phones and ipads (murphy et al. 2013; wiebrands 2012). these mobile devices have the potential to extend educational experiences through touch, location-aware learning, embedded multimedia, and multiple channels of educational content delivery. support for university academic and professional staff to conduct their work more effectively by using mobile devices such as smart phones and tablets in their work will be critical. the factors leading to mainstream mobile device adoption in the university workplace is an important consideration for those in senior management roles responsible for developing, supporting and evaluating these activities. australasian journal of information systems volume 18 number 3 2014 54 this paper is structured as follows. firstly, a review of the relevant literature is conducted in relation to ipads and their adoption in higher education and the technology acceptance model (tam) to provide the theoretical and conceptual basis for our proposed model of ipad adoption by university staff. next, the hypothesised relationships in the proposed university staff ipad adoption model are discussed and justified in relation to the relevant literature. then the research methodology used in this research is described and justified. the results of the data analysis results are discussed in terms of the survey population and response rate, demographics of survey respondents and the reliability and validity of the measurement model. the next section discusses the results of the hypothesis testing in relation to the existing literature. finally, we present our conclusions and implications for theory and practice in relation to the use of ipads by university (academic and professional) staff in higher education. background and motivation for this study the use of ipads by university staff has increased steadily in response to the release of the ipad in 2010, with conservative estimates putting total sales of ipads based on apple’s quarterly financial report for october 2013 at 170 million worldwide (costello 2014). consistent anecdotal evidence suggests there is a need for university staff to be appropriately supported so that they can effectively make the transition to integrating these mobile devices into their daily work activities. some studies regarding the practical implications and impact of the ipad have been undertaken in the university sector; however, these studies have primarily focused on students (hoover & valencia 2011; pepperdine university 2012; wu, kuo & wu 2013). hence, there is a lack of empirical research which has critically assessed the factors determining the adoption of ipads by university staff. this research project seeks to understand, within the university environment, the factors leading to widespread adoption and the perceived usefulness of these devices by university staff in their work. the technology acceptance model (tam) provided the theoretical foundation for this research and the proposed model of university staff adoption of ipads. the main research question investigated in this project was: to what extent do factors such as work compatibility, previous experience in using a similar technology such as an iphone or smartphone, age and social status influence university staff adoption of ipads in their work? previous literature regarding ipad/tablet use in higher education the ipad achieved a fifty-seven percent share of the tablet pc market in the fourth quarter, 2011 (ong 2011). this trend, coupled with the 2012 launch of ibooks and the itunes u app, show apple corporation attempting to align itself with the education sector. universities such as stanford, university of california and university of central florida (shurtz, halling & mckay 2011), and australian counterparts such as the university of adelaide (cross 2010) and university of new england and, more recently, the university of western sydney, have taken a proactive stance by issuing ipads to commencing students. research into the acceptance and use of ipads in higher education has focused on ubiquitous and flexible learning (kukulska-hume 2012; maddux & johnson 2010). hence, ipads are viewed primarily as a presentation tool, and an e-text and e-learning content delivery mobile device (becker 2011; shen 2010)—which is limiting their potential for adding value to both the roles and activities of university staff and students. previous empirical research on the use of ipads by university staff two main studies were identified as directly relevant to this project as they focused on the adoption and continuing developmental needs of ipads by academic staff in higher education. oldfield and cochrane (2011) worked with twenty staff in a business and law faculty to map key staff development issues in relation to the adoption and use of ipads. they found two dimensions requiring support, namely, an initial ‘learning curve’ (primarily self-directed learning)—which matured to a second australasian journal of information systems volume 18 number 3 2014 55 communal learning phase (facilitated by a community of practice model). qualitative data was gathered via surveys and focus groups. kelly and schrape (2010) also undertook a primarily qualitative study of staff acceptance and use of ipads in teaching by segregating the staff learning curve into three, one-month blocks (familiarisation, integration and reflection). the focus of data collection was ipad usage in daily activities and the compatibility of the ipad with the learning management system (lms), as well as a list of the most used apps. the lack of empirical research that has been undertaken into the use of ipads by university staff provided the motivation for the development and testing of a university staff ipad adoption model. technology acceptance model as a theoretical framework for ipad adoption the tam, developed by davis (1989), seeks to understand the rationale for technology adoption among users. as a predictive model of technology adoption it is considered to be mature and has been extended to accommodate a wide range of technologies including ebook readers (read, robertson & mcquilken 2011), online learning communities (liu et al. 2010), mobile payment systems (lee, kozar & larsen 2003), mobile learning paradigms (park, nam & cha 2011) and tablet pcs (moran, hawkes & el gayar 2010) . the latter was of particular interest to this project, given there are some similarities between tablet pcs and ipads. the original tam (davis 1989) used predictive factors based upon users’ perceptions of ease of use and usefulness (nair & das 2011). however, the model has been extended by numerous researchers (notably to include intent to use) (davis, bagozzi & warshwa 1989). this has been imperative given the ubiquitous nature of the internet which has led to wide scale adoption of many different internet enabled technologies and the need for validated research concerning the adoption and use of internet enabled technologies. mathieson (1991) argued that the base tam over-simplified the adoption process, but variants of extended tam have been developed to accommodate variables such as mobile learning attitude (park 2009), and context for use (mcfarland & hamilton 2006). this research uses an extended tam to contextualize the assessment of adoption and use of ipads by university staff, as most of the previous literature only investigated student acceptance and uptake of ipads. hence, the technology acceptance model adapted for this research uses some specific and relevant external variables for the context of this study which have been shown to moderate the relationships in tam (gerpott, thomas & weichert 2013; lee, kozar & larsen 2003; straub, keil & brenner 1997). this research seeks to examine whether compatibility with university staff work, previous use of a device with a similar interface and operating system (in this case an iphone or android or windows smart phone), age and the level of ict end user support affects university staff adoption and use of ipads. the relatively new nature of the ipad as a touch screen and internet enabled mobile computing device and its recent adoption in higher education provides the justification for an extended variant of tam as an appropriate theoretical framework for evaluating the adoption and use of ipad/tablets by university staff (knight & burn 2011). hypothesised relationships in the proposed research model compatibility is defined as the degree to which a technological innovation such as ipad is perceived as being consistent with existing values, needs, and past experiences of potential adopters (i.e. university staff in this study) (moore & benbasat 1991). compatibility has been found to be strongly associated with the successful adoption and use of technological innovations (guiltinan 1999; holak & lehmann 1990; taylor & todd 1995; tornatzky & klein 1982). kotler (2003) states that compatibility refers to the degree of fit of an innovation with an individual in a social system (university work in this research) australasian journal of information systems volume 18 number 3 2014 56 and observed that compatibility can influence the adoption of innovative products such as personal computers. a number of previous empirical studies have found that compatibility is a key factor that influences the perceived ease of use (lai & chang 2011) and perceived usefulness (lai & chang 2011) of technological innovations. lai and chang (2011) found that compatibility directly impacts on the perceived ease of use and perceived useful of a technological innovation. therefore, the literature provides support for the following two hypotheses: h1: compatibility of an ipad with staff work is positively associated with the perceived usefulness of an ipad. h2: compatibility of an ipad with staff work is positively associated with the perceived ease of use of an ipad. social status is defined as the degree to which use of a technological innovation such as an ipad is perceived to enhance one's image or status in one's social system (moore & benbasat 1991). it is reasonable to assume that if adopting a new technology is considered to enhance one's social image, the attitude toward adopting the new technology will be positive (choi et al. 2003; karahanna, straub & chervany 1999). therefore, if a user perceives that adopting a new technology such as an ipad will showcase their good image or social status in an organisation, then the user will have a positive attitude towards an ipad. therefore, we propose the following hypothesis: h3: the social status of using an ipad in the workplace has a positive association with the attitude to using an ipad. perceived usefulness is defined as the extent to which a person perceives that using a new technology such as an ipad will enhance one’s job performance (chong et al. 2010). previous studies have found that perceived usefulness has a significant impact on the adoption of a technology, including the attitude towards a technology and the intention to use a technology (chong et al. 2010; lee, cheung & chen 2005). if university staff perceive an ipad will be useful in their work they are more likely to have a positive attitude and intention toward using an ipad. hence, the following hypotheses have been proposed: h4: perceived usefulness of an ipad has a positive association with the intention to use an ipad. h5: perceived usefulness of an ipad has a positive association with the attitude towards using an ipad. perceived ease of use is defined as the extent to which a person believes using a technology would be relatively effortless (davis 1989). a number of previous studies have found perceived ease of use is a key determinant of attitude towards using a new technology and the actual use of a new technology (davis 1989; lapczynski & calloway 2006; lu & viehland 2008; malhotra & galleta 1999). if university staff perceive an ipad to be easy to use it is likely to influence their attitude towards using an ipad and their actual use of an ipad. therefore, the following hypotheses are proposed: h6: perceived ease of use of an ipad has a positive association with the attitude towards using an ipad. h7: perceived ease of use of an ipad has a positive association with the frequency of actual use of an ipad. attitude to using a technological innovation has long been established by previous studies of technology adoption to influence intention to use which, in turn, influences actual use of a technological innovation (davis 1989; davis, bagozzi & warshwa 1989; horan et al. 2004; karahanna, straub & chervany 1999; lu & viehland 2008; straub, keil & brenner 1997). previous studies suggest that university staff’s’ attitude and subsequent intent to use a new technology such as an ipad ultimately determines their adoption and actual use in a workplace. therefore, the following hypotheses are proposed: australasian journal of information systems volume 18 number 3 2014 57 h8: attitude towards using an ipad has a positive association with the intention to use an ipad. h9: intention to use an ipad has a positive association with the frequency of actual use of an ipad. it has been found that those who are early adopters of new technologies in the workplace, such as the higher education sector, expected their ict support staff to continuously develop their knowledge, to understand user expectations and to maintain a high level of communication between support staff and users (kagaari, munene & ntayi 2010). a number of previous studies have established that computing support is positively associated with the adoption of a new technology in the workplace (igbaria et al. 2007; schaper & pervan 2007; thompson, higgins & howell 1991). hence, the existing literature suggests that an appropriate level of computing support provided at the organization level for the use of ipads by university staff will have a positive impact on the frequency of actual use. therefore, we propose the following hypothesis: h10: ict support has a positive association with the frequency of actual use of an ipad. the adoption of new computing devices has been positively linked to prior experience and use of existing similar platforms and/or similar devices (jackson, chow & leitch 1997; shih & venkatesh 2004). past experience with similar technologies is likely to affect one’s perceptions about the ease of use of a new technology such as an ipad (tan et al. 2012). the iphone, android and windows smartphones share similar or identical user interfaces and operating systems to an ipad. hence, it is argued that familiarity with aspects of a similar technology will be positively associated with the users’ perceptions of ease of adoption of the technology. therefore, we propose the following hypothesis: h11: users with previous experience in using a similar technology such as an iphone or smartphone will rate the perceived ease of use of an ipad higher than users with no previous experience of using a similar technology such as an iphone or smartphone. prior studies have examined the interaction between age and technology adoption (goldin & katz 2008; reisdorf 2011). empirical research into the adoption of mobile learning devices has indicated little significant difference in perceived usefulness and perceived ease of use due to individual characteristics such as gender or age (tan et al. 2012). however, age has been found to be indicative of risk-avoidance and conservativeness, and negatively associated with adoption of a new technological innovation (wang, dou & zhou 2008). therefore, we propose the following hypothesis: h12: age is negatively associated with the perceived ease of use of an ipad. the proposed research model in figure 1 depicts the hypothesized relationships tested in this study. australasian journal of information systems volume 18 number 3 2014 58 figure 1 research model – university staff adoption of ipads (source: this research) research methodology this research was guided by a positivist research design. we used a cross-sectional survey method to collect primarily quantitative data to test the hypothesised relationships in the proposed research model (figure 1). the survey was administered online after ethics clearance was obtained from the university ethics committee. respondents were assured that their responses were anonymous. the survey instrument was developed using existing validated survey instrument items which were adopted/adapted from a number of previous empirical studies (see table 1) (straub, boudreau & gefen 2004). face validity of the survey instrument was verified and refined through a pretest of the survey instrument with five university staff with substantial experience in survey research. australasian journal of information systems volume 18 number 3 2014 59 construct no of items (scale) adopted/adapted from previous empirical studies compatability with work (cw) 3 items 7 point likert-type compatibility (moore & benbasat 1991, p. 216) ict support (icts) 3 items 7 point likert-type facilitating conditions (thompson, higgins & howell 1991, p. 132) social status (ss) 3 items 7 point likert-type image / social status (moore & benbasat 1991, p. 216) perceived usefulness (pu) 5 items 7 point likert-type perceived usefulness (davis 1989, p. 331) perceived ease of use (peu) 6 items 7 point likert-type perceived ease of use (davis 1989, p. 331) attitude to use (atu) 3 items 7 point likert-type attitude (ajzen & fishbein 1980) (taylor & todd 1995, p. 173) intention to use (itu) 3 items 7 point likert-type intention to use (ajzen & fishbein 1980) (taylor & todd 1995, p. 174) frequency of actual use (foau) 1 items 7 point likert-type single item rating of frequency of use per week age 1 item ordinal-type single item ordinal scale of age categories previous experience with a similar technology (pest) 1 item nominal-type single item nominal variable for previous experience with similar technology converted to binary variable for analysis table 1 sources of survey questionnaire items measuring each construct results of data analysis study population and survey response rate the university employed approximately 1300 staff (including academic and professional positions)at the time when this research was conducted. four hundred ipads are estimated to be used by university staff at the time the survey was conducted according to the division of ict services. this number was estimated from the number of ipads purchased for university staff and ipad campus network usage statistics provided by university ict services. the survey received 87 usable responses from university staff using ipads in their work (an estimated 22 percent response rate within the identified ipad university staff user population of 400). the survey focused on the use of ipads in activities relating to learning, teaching and general duties directly pertaining to university business. the survey respondents were relatively evenly spread across the academic (53.5 percent) and professional (46.5 percent) employee groups. demographics of survey respondents university staff usage of ipads is dominated by the age categories 36 years and higher (table 2). these age categories represented 85 percent of respondents, with the 18 to 35 age category accounting for the remaining 15 percent. more importantly, almost half of the university staff who responded to this survey are in the 46-55 and 55 and over age categories. the distribution of age categories in the survey responses is representative of the distribution of the age categories of university staff overall. most, if not all, australian universities have an ageing workforce. australasian journal of information systems volume 18 number 3 2014 60 age category frequency percentage 18-25 2 2.3 26-35 11 12.6 36-45 33 37.9 46-55 29 33.3 over 55 12 13.8 total 87 100.0 missing 1 table 2 distribution of survey responses by age category university staff roles were evenly distributed between academic staff (46.5 percent) and professional staff (53.5 percent) in the survey responses, as shown in table 3. university staff role frequency percentage academic staff 40 46.5 professional staff 46 53.5 total 86 100 missing 2 table 3 university staff role (academic or professional) measurement model for university staff adoption of ipads/tablets the testing of the proposed relationships between the seven constructs and three single item measures in the proposed university staff adoption of ipads model (see figure 1) was conducted using partial least squares (pls) and the software package smart pls (ringle, wende & will 2005). the sample size of 87 valid responses from university staff using ipads was not large enough for covariance based sem to be used. pls is a second generation multivariate analysis technique for testing means based structural equation models (wold 1985). pls does not require a normal distribution, unlike covariance based sem, and will work with relatively small data sets (ghapanchi & aurum, 2012a, ghapanchi & aurum, 2012b, eikebrokk & olsen, 2007). as a rule of thumb, the minimum sample size for pls analysis should be 10 times the number of items present in the most complex construct, or 10 times the largest number of independent variables impacting on a dependent variable. in the proposed research model, the most complex construct is perceived usefulness, which has 6 items; and the largest number of independent variables which are estimated to impact on a dependent variable (intention to use) is 3 (chin 1998; karahanna, straub & chervany 1999). pls simultaneously evaluates the measurement model and the theoretical model by assessing the reliability and validity of the theoretical constructs and relationships between those constructs (barclay, thompson & higgins 1995). in this study the pls model is analysed and interpreted in a rigorous two phase approach: (1) an assessment of the reliability and validity of the measurement model describing the relationship between the latent constructs and their manifest indicators; and (2) an assessment of a structural model describing the relationships between the latent constructs (barclay, thompson & higgins 1995). this approach ensures that construct measures are valid and reliable before drawing conclusions about the predictive strength of relationships between the constructs in the theoretical model. assessment of the measurement model for university staff adoption of ipads the measurement model in pls is assessed for construct validity and reliability and the extent to which the underlying manifest variables accurately reflect and measure their constructs (hair et al. 2012; straub, boudreau & gefen 2004). an assessment of the measurement model evaluates construct australasian journal of information systems volume 18 number 3 2014 61 validity and the extent to which the underlying manifest variables accurately reflect their constructs. this assessment includes the individual item reliability, construct reliability, average variance extracted (ave) analysis, and convergent validity and discriminant validity (see table 4 and table 5) (straub, boudreau & gefen 2004). individual item reliability is considered adequate when an item has a factor loading greater than 0.7 on its intended construct, which implies more shared variance between the construct and its measures than error variance (carmines & zeller 1979). the factor loadings for the 27 items measuring the 7 constructs in the research model were all above the recommended 0.7 (chin 1998; fornell & larcker 1981) (see table 5). construct reliability was measured using composite reliability, cronbachs alpha and average variance extracted (ave). reliability and convergent validity was interpreted using 0.7 level, which has been widely suggested as the benchmark for moderate reliability (nunnally 1979). in this research, all the constructs demonstrate high reliability and convergent validity with values for composite reliability and cronbachs alpha—which are above the recommended 0.7 level. ave measure quantifies the amount of variance that a construct captures from its measurement items (indicators) (hair et al. 2012). all of the ave measures for the latent variables exceed the recommended 0.5 value (fornell & larcker 1981). ave composite reliability cronbachs alpha communality redundancy icts 0.82 0.93 0.90 0.82 0.00 peu 0.83 0.96 0.95 0.83 0.08 pu 0.86 0.97 0.97 0.86 0.55 atu 0.84 0.95 0.94 0.84 0.26 cw 0.85 0.94 0.91 0.85 0.00 itu 0.87 0.95 0.93 0.87 0.50 ss 0.85 0.94 0.92 0.85 0.00 legend cw = compatibility with work; ss = social status; peu = perceived ease of use; pu = perceived usefulness; atu = attitude to use; intu = intention to use; foau = frequency of actual use; age; pest = previous experience with similar technology; icts = ict support table 4 reliability of constructs in proposed research model (source: this research) to assess discriminant validity, ave should be greater than the variance shared between the construct and other constructs in the model (squared correlation between two constructs) (ghapanchi & aurum 2011; hair et al. 2012; henseler, ringle & sinkovics 2009). adequate discriminant validity is determined by comparing the square root of the ave value of each construct and its correlation with the other constructs in the model. all of the constructs in the model have square root ave values which are higher than their corresponding correlation values with the other constructs. the items retained for each factor were determined from the factor loadings and cross loadings produced by pls. all items were retained as all the factor loadings were above the recommended 0.7 level (see table 5). overall internal validity was established through ensuring that the constructs measuring dimensions of tam for university staff ipad use had adequate face and content validity and adequate convergent and discriminant validity. hence, the constructs measured in this study were found to be reliable and valid (see tables 4 and 5) and can be replicated in further studies involving the adoption and use of mobile devices. australasian journal of information systems volume 18 number 3 2014 62 construct items icts peu pu atu cw itu ss attitude1 0 0 0 0.89 0 0 0 attitude2 0 0 0 0.91 0 0 0 attitude3 0 0 0 0.93 0 0 0 attitude4 0 0 0 0.93 0 0 0 compatibility1 0 0 0 0 0.94 0 0 compatibility2 0 0 0 0 0.93 0 0 compatibility3 0 0 0 0 0.89 0 0 easeofuse1 0 0.92 0 0 0 0 0 easeofuse2 0 0.92 0 0 0 0 0 easeofuse3 0 0.86 0 0 0 0 0 easeofuse4 0 0.96 0 0 0 0 0 easeofuse5 0 0.91 0 0 0 0 0 futureuse1 0 0 0 0 0 0.93 0 futureuse2 0 0 0 0 0 0.96 0 futureuse3 0 0 0 0 0 0.92 0 socialstatus1 0 0 0 0 0 0 0.96 socialstatus2 0 0 0 0 0 0 0.92 socialstatus3 0 0 0 0 0 0 0.89 ictsupport1 0.84 0 0 0 0 0 0 ictsupport2 0.92 0 0 0 0 0 0 ictsupport3 0.95 0 0 0 0 0 0 useful1 0 0 0.94 0 0 0 0 useful2 0 0 0.95 0 0 0 0 useful3 0 0 0.94 0 0 0 0 useful4 0 0 0.95 0 0 0 0 useful5 0 0 0.87 0 0 0 0 useful6 0 0 0.92 0 0 0 0 legend: cw = compatibility with work; ss = social status; peu = perceived ease of use; pu = perceived usefulness; atu = attitude to use; intu = intention to use; foau = frequency of actual use note: age; pest = previous experience with similar technology; and icts = ict support were not included in this factor loadings table because these are single items which always have a factor loading of 1. table 5 factor loadings for construct items (source: this research) we also checked for common method bias in the measurement model which is the variance attributable to the measurement method, rather than to the measures that represent the constructs of interest in a study (podsakoff et al. 2003). common variance bias is a major systematic contributor to measurement error in survey research (bagozzi & yi 1991). to test for the extent of bias caused by common methods variance (cmv), harman’s single factor test was conducted using an exploratory factor analysis in australasian journal of information systems volume 18 number 3 2014 63 ibm spss version 19 (podsakoff et al. 2003). it is argued that if there is a detrimental level of common method bias, ‘(a) a single factor will emerge from exploratory factor analysis (unrotated) or (b) one general factor will account for the majority of the covariance among the measures’ (podsakoff et al. 2003, p. 889). as more than one factor emerged from an exploratory factor analysis to explain the variance in our analysis, we can infer that common methods bias in this case is not high. following the approach of podsakoff et al. (2003), saraf, langdon and gosain (2007) and siponen and vance (2010), a second test in pls examined a control for the effects of an unmeasured latent methods factor in our research model (podsakoff et al. 2003). in this analysis, of the 27 paths from methods factor to singleindicator constructs, only six were significant—indicating a relatively small amount of cmv. hence, common methods bias has been shown to have minimal effect and is not considered to be a concern in this study. assessment of structural model and goodness of fit university staff adoption of ipads figure 2 shows the hypothesised relationships in the pls structural model, beta scores and the statistical significance of each hypothesised relationship and the amount of variance (r square) explained in individual variables in the structural model. legend cw = compatibility with work; ss = social status; age; pu = perceived usefulness; peu = perceived ease of use; atu = attitude to use; itu = intention to use; foau = frequency of actual use; age; pest = previous experience with similar technology; icts = ict support; figure 2 final research model – university staff adoption of ipads [source: this research] an overall goodness-of-fit index for the research model cannot be reported because the objective of pls is prediction versus fit. while pls does not provide an index that is a global validation of a research model such as covariance based sem, x2 and related model fit indicators, a goodness of fit index can be calculated for a pls model. goodness of fit (gof) represents an operational solution for validating the pls model globally (guenzi, georges & pardo 2009). gof is determined by calculating the geometric mean of the average communality and average r2 (tenenhaus et al. 2005). by taking the square root of the product of the variance extracted of all constructs with multiple indicators and the australasian journal of information systems volume 18 number 3 2014 64 average r2 value of the endogenous constructs, we can calculate a fit measure ranging between 0 and 1. according to the categorization by fliess and cohen (1973) and using .50 as a cut-off value for communality (fornell & larcker 1981), the gof criteria for small, medium and large effect sizes are .10, .25, and .36 respectively. the blindfolding approach proposed by wold (1985) was used to calculate the cv-communality and cv-redundancy indexes. the cv communality index (h2) measures the quality of the measurement model; while the cv redundancy index (f2) measures the quality of the structural model. the mean of the cv communality indexes can be used to measure the global quality of the measurement model if they are positive for all blocks of variables. the global quality of the structural model can be measured by the mean of the cv-redundancy indexes related to the endogenous blocks if they are all positive (guenzi, georges & pardo 2009). table 6 shows that an overall large effect size is evident in the research model with gof index of 0.55. the mean cv-communality index (h2) of 0.68 indicates a large effect size for the measurement model and the mean cv-redundancy index (f2) of 0.44 indicates medium effect size for structural model. construct r2 cvcommunality cv-redundancy age na na na icts na 0.54 na peu 0.27 0.73 0.21 pu 0.63 0.80 0.52 pest na na na foau 0.42 na 0.30 atu 0.74 0.71 0.59 cw na 0.65 0.59 itu 0.71 0.69 na ss na 0.64 na ave r2 0.55 h2 index – ave cv-communality 0.68 f2 index ave cv-redundancy 0.44 goodness of fit index (gof) 0.55 legend cw = compatibility with work; ss = social status; peu = perceived ease of use; pu = perceived usefulness; atu = attitude to use; intu = intention to use; foau = frequency of actual use; age; pest = previous experience with similar technology; icts = ict support; table 6 global fit indexes for pls measurement model and structural model (source: this research) discussion of key findings in this section the key findings of this study in relation to each of the twelve hypothesized relationships tested in the research model are discussed. table 6 summarises the results of the hypothesis testing. the t value statistics column indicates the level of significance at which hypotheses have been supported. all hypotheses except hypothesis h3 are accepted at the 0.05 level of significance. australasian journal of information systems volume 18 number 3 2014 65 12 hypotheses tested in research model t values significance levels hi: compatibility of an ipad with staff work is positively associated with the perceived usefulness of an ipad (supported) 3.68*** h2: compatibility of an ipad with staff work is positively associated with the perceived ease of use of an ipad (supported) 19.66*** h3: the social status of using an ipad has a positive association with the attitude to using an ipad (not supported) 0.46 ns h4: perceived usefulness of an ipad has a positive association with the intention to use an ipad (supported) 3.98*** h5: perceived usefulness of an ipad/tablet has a positive association on the attitude towards using an ipad (supported) 4.30*** h6: perceived ease of use of an ipad has a positive association with the attitude towards using an ipad (supported) 8.67*** h7: perceived ease of use of an ipad has a positive association with the frequency of actual use of an ipad (supported) 2.17** h8: attitude towards using an ipad has a positive association with the intention to use an ipad (supported) 2.69** h9: intention to use an ipad has a positive association with the frequency of actual use of an ipad (supported) 4.64*** h10: ict support has positive association with the frequency of actual use of an ipad (supported) 3.47*** h11: users with previous experience of using a similar technology such as an iphone or smartphone will rate the perceived ease of use of an ipad higher than users with no previous experience of using a similar technology such as an iphone or smartphone (supported) 2.38** h12: age is negatively associated with the perceived ease of use of an ipad (supported) 1.67* legend cw = compatibility with work; ss = social status; peu = perceived ease of use; pu = perceived usefulness; atu = attitude to use; intu = intention to use; foau = frequency of actual use; age; pest = previous experience with similar technology; icts = ict support; table 6 summary of results of hypothesis testing (source: this research) h1: compatibility of an ipad with staff work is positively associated with the perceived usefulness of an ipad—supported with a beta score of 0.80 and a strong level of significance. h2: compatibility of an ipad with staff work is positively associated with the perceived ease of use of an ipad—supported with a beta score of 0.35 and a strong level of significance. the findings regarding h1 and h2 shows that university staff in general perceived an ipad to be highly compatible with their work and, therefore, strongly influenced their perception of the usefulness of an ipad; and, to a lesser extent, their perception of ease of use of an ipad. these findings in relation to h1 and h2 are consistent with the findings of a number of previous studies which found support for the relationship between compatibility and perceived usefulness and perceived ease of use (chau & hu 2001; lai & chang 2011; oh, ahn & kim 2003). indeed, the compatibility of a technological innovation such as an ipad is considered to be an antecedent of its perceived usefulness and perceived ease of use. australasian journal of information systems volume 18 number 3 2014 66 h3: the social status of using an ipad has a positive association with the attitude to using an ipad—not supported with a beta score of 0.03 and not significant. hypothesis h3 was based on the assumption that the social status associated with using a new technology innovation such as an ipad may be an influencing factor in technology adoption (karahanna, straub & chervany 1999; moore & benbasat 1991). hypothesis h3 was not supported with a beta score of 0.03, which was not significant at the 0.05 level. furthermore, an analysis of the survey responses for the three social status survey items showed that 65-75 percent of the survey responses displayed either a neutral or negative stance to the suggestion that any social status is bestowed by ownership of one of these devices, or that enhanced status is an influencing factor to adoption. hence, the finding in this study regarding h3 is contrary to previous studies which found the social status gained from using a new technological innovation was positively associated with the attitude to using a new technological innovation (in this study an ipad) (choi et al. 2003; lu & viehland 2008). however, this finding is supported by other studies which have found that social status or image inferred with the adoption and use of a technological innovation (in this study an ipad) to be a weak predictor of a user’s attitude to adoption of a new technology (karahanna, straub & chervany 1999; moore & benbasat 1991). h4: perceived usefulness of an ipad has a positive association with the intention to use an ipad— supported with a beta score of 0.66 and a strong level of significance. h5: perceived usefulness of an ipad/tablet has a positive association on the attitude towards using an ipad—supported with a beta score of 0.53 and a strong level of significance. the findings show that there is strong support for h4 and h5 and the perceived usefulness of a technological innovation such as an ipad is positively associated with university staff’s attitude and intent to using an ipad. hence, the perceived usefulness of an ipad in university staff’s work would appear to a key driver and determinant of the adoption and use of an ipad by university staff. these findings are similar to previous studies which found that perceived usefulness was positively associated with attitude towards adoption and the intention to use a technological innovation that is mobile and/or internet enabled (chong et al. 2010; lee, cheung & chen 2005; tan et al. 2012). h6: perceived ease of use of an ipad has a positive association with the attitude towards using an ipad— supported with a beta score of 0.24 and a strong level of significance. h7: perceived ease of use of an ipad has a positive association with the frequency of actual use of an ipad—supported with a beta score of 0.36 and a strong level of significance. the findings regarding h6 and h7 show perceived ease of use is moderately and positively associated with the attitude of university staff to using an ipad and the frequency of their actual use of an ipad in their work. these findings regarding h6 and h7 are consistent with previous studies that found perceived ease of use leads to an increased positive attitude to use of a technological innovation (in this study an ipad) and increased frequency of actual use (davis 1989; tan et al. 2012). h8: attitude towards using an ipad has a positive association with the intention to use an ipad— supported with a beta score of 0.36 and a strong level of significance. h9: intention to use an ipad has a positive association with the frequency of actual use of an ipad— supported with a beta score of 0.50 and a strong level of significance. the findings show that there is moderate to strong support for the positive relationship between university staff attitude to using an ipad and their intention to use an ipad and their actual use of an ipad. hence, the findings show that the attitude of university staff towards a technological innovation such as an ipad translates into intent to use and then actual use of an ipad. these findings are similar to a number of previous studies which found the attitude towards a technological innovation positively influenced the intention to use and the actual use of a technological innovation (karahanna, straub & australasian journal of information systems volume 18 number 3 2014 67 chervany 1999; lapczynski & calloway 2006; lu & viehland 2008; tan et al. 2012). indeed, three of these previous studies examined these relationships in a similar context of mobile computing and mobile learning (lapczynski & calloway 2006; lu & viehland 2008; tan et al. 2012). h10: ict support has a positive association with the frequency of actual use of an ipad—supported, but with a negative beta score of -0.20 and a strong level of significance. the findings show that ict support has a moderate association with the frequency of actual use of an ipad. however, the beta score for the hypothesised relationship is negative—which is the reverse of the positive relationship that was predicted. ict support was found to have a negative impact on the actual use of an ipad by university staff, with most respondents disagreeing that there was adequate ict support. this finding confers with previous studies using an extended tam that also found technology support at the organisation level was a critical factor for user adoption and use of a technology innovation (horan et al. 2004; schaper & pervan 2007). moreover, this finding can be explained in the context of this study because, currently, the university does not provide any formal, dedicated support for mobile devices such as ipads, android tablets and windows tablets. for ipads (and other touch mobile devices) there is lack of enterprise wide solutions for managing the use of apps by university staff in their work. an ict services support model for these mobile devices is challenging and, potentially, has significant human resourcing implications for university ict support services. h11: users with previous experience of using a similar technology such as an iphone or smartphone will rate the perceived ease of use of an ipad higher than users with no previous experience of using a similar technology such as an iphone or smartphone—supported with a beta score of 0.15 and a moderate level of significance. previous experience in using a similar technology such as an iphone/smartphone (nominal variable) was converted to a binary variable. this allowed us to compare the difference between university staff with previous experience in using similar technology such as an iphone or smartphone against those with no previous experience in percentage terms for the beta score of the hypothesised relationship. the finding regarding hypothesis h11 shows that university staff with previous experience in using a similar technology such as an iphone or smartphone rated perceived ease of use of an ipad 22 percent higher than university staff with no previous experience in using a similar technology such as an iphone or smartphone. this finding provides support for a number of previous studies that identified that previous experience in using a similar technology (in this study, iphones and smart phones based on similar interfaces and operating systems) will result in users rating the perceived ease of use of a technology innovation (in this study an ipad) much higher (lu & viehland 2008; tan et al. 2012). h12: age is negatively associated with the perceived ease of use of an ipad—supported with a negative beta score of -0.28 and a strong level of significance. the finding regarding the result of h12 hypothesis test shows that age is moderately negatively associated with the perceived ease of use of an ipad. in other words, university staff in the older age categories of 46-55 years and over 55 years rated perceived ease of use of ipads lower than university staff in the younger age categories. this finding is consistent with the findings of a number of previous studies which found that age is a barrier to the adoption of new technological innovations (goldin & katz 2008; reisdorf 2011; wang, dou & zhou 2008). conclusions and implications there has been little previous empirical research conducted on the adoption of ipads by university staff. this study provides support for a validated model for the adoption of ipads by university staff using an extended variant of tam contextualised to the higher education environment. australasian journal of information systems volume 18 number 3 2014 68 clearly our findings show there is a strong level of positive adoption of ipads across the academic and professional university staff workforce in one regional university; and anecdotal reports indicate a similar worldwide trend with potential for universities to leverage the affordances and ubiquity of these devices for educational and/or other work purposes. whilst the findings point to a strong perception of ipad compatibility with the work undertaken by university professional and academic staff, there are a number of strategic imperatives that universities need to consider as a result. as mobile devices such as ipads become mainstream in the work of university staff, ict departments will need to deliver more appropriate user-centred ict support models. in particular, our findings suggest that older staff and staff with no or little previous experience of using a similar technology such as an iphone or smartphone will require more end user support from ict services departments when using mobile devices such as ipads. as with all research there are some limitations in this study. the number of useable responses was not sufficient to use covariance based sem. future work could extend this research to other universities and other industry sectors and provide further validation of the extended tam for evaluating staff adoption of both ipads and android tablets in the workplace. additional work is also required to develop models which can longitudinally evaluate the effective ongoing use of mobile devices in general by university staff beyond the initial adoption phase of a technological innovation such as an ipad. acknowledgments the authors express their gratitude to our usq colleagues michael gardiner, dr mustafa ally, associate professor jane summers and justine fritisch for their assistance in pre-testing the face validity of our survey instrument. the financial and in-kind support received from the faculty of business and law and associate professor jane summers (associate dean learning and teaching) is also gratefully acknowledged. lastly, the generous time-release for mr adrian stagg granted by the australian digital futures institute at usq is acknowledged, as without it the data collection for this research would have been more difficult. references ajzen, i & fishbein, m (1980), understanding attitudes and predicting social behaviour, prenticehall, englewood cliffs, nj. bagozzi, rp & yi, y (1991), 'multitrait-multimethod matrices in consumer research, journal of consumer research, vol. 17, no. 4, pp. 426-39. barclay, d, thompson, r & higgins, c (1995), 'the partial least squares (pls) approach to causal modeling: personal computer adoption and use: an illustration', technology studies, vol. 2, no. 2, pp. 285-309. becker, b (2011), 'the e-book apocalypse: a survivor’s guide', behavioral and social sciences librarian, vol. 30, no. 3, pp. 181-4. carmines, e & zeller, r (1979), reliability and validity assessment, sage publications, beverly hills. caudill, jg (2007), 'the growth of m-learning and the growth of mobile computing: parallel developments', the international review of research in open and distance learning, vol. 8, no. 2, viewed 07 april 2014. chau, pyk & hu, pjh (2001), 'information technology acceptance by individual professionals: a model comparison approach', decision sciences, vol. 32, no. 4, pp. 699-719. chin, w (1998), 'the partial least squares approach to structural equation modeling', in g marcoulides (ed.), modern methods for business research, psychology press, london, pp. 295 336. australasian journal of information systems volume 18 number 3 2014 69 choi, h, choi, m, kim, j & yu, h (2003), 'an empirical study on the adoption of information appliances with a focus on interactive tv', telematics and informatics, vol. 20, pp. 161-83. chong, ayl, ooi, kb, darmawan, n & lee, vh (2010), 'determinants of 3g adoption in malaysia: a structural analysis', journal of computer information systems, vol. 51, no. 2, pp. 71-80. costello, s (2014), 'what are ipad sales all time?', about.com iphone / ipod, <http://ipod.about.com/od/ipadmodelsandterms/f/ipad-sales-to-date.htm> cross, k (2010), september 11, 'ipad replaces uni textbooks at university of adelaide science faculty', the advertiser, viewed 27th august 2012, <http://www.adelaidenow.com.au/news/ipad-replaces-uni-textbooks-at-university-of-adelaidescience-faculty/story-fn5jhv6y-1225918213032> davis, f (1989), 'perceived usefulness, perceived ease of use, and user acceptance of information technology', mis quarterly, vol. 13, no. 3, pp. 319-39. davis, f, bagozzi, r & warshwa, p (1989), 'user acceptance of computer technology: a comparison of two theoretical models', management science, vol. 35, no. 8, pp. 982-1003. eikebrokk, tr & olsen, dh (2007), 'an empirical investigation of competency factors affecting ebusiness success in european smes', information & management, vol. 44, no. 4, pp. 364-83. fleiss, j & cohen, j (1973), 'the equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability', educational and psychological measurement, vol. 33, pp. 613-9. fornell, c & larcker, d (1981), 'evaluating structural equation models with unobservable variables and measurement error', journal of marketing research, vol. 18, no. 1, pp. 39-50. gerpott, t, thomas, s & weichert, m (2013), 'personal characteristics and mobile internet use intensity of consumers with computer-centric communication devices: an exploratory empirical study of ipad and laptop users in germany', telematics and informatics, vol. 30, no. 2, pp. 87-99. ghapanchi, ah & aurum, a (2012a), 'competency rallying in electronic markets: implications for open source project success', electronic markets, vol. 22, no. 2, pp. 117-27. ghapanchi, ah & aurum, a (2012b), 'the impact of project capabilities on project performance: case of open source software projects', international journal of project management, vol. 30, no. 4, pp. 407-17. ghapanchi, ah & aurum, a (2011), 'measuring the effectiveness of the defect-fixing process in open source software projects', in: proceedings of the 44th hawaii international conference on systems sciences (hicss) pp. 1-11. goldin, c & katz, l (2008), the race between education and technology, belknap, cambridge. guenzi, p, georges, l & pardo, c (2009), 'the impact of strategic account managers' behaviors on relational outcomes: an empirical study', industrial marketing management, vol. 38, no. 3, pp. 300-11. guiltinan, j (1999), 'launch strategy, launch tactics, and demand outcomes', journal of product innovation management, vol. 16, no. 6, pp. 509-29. hair, jf, sarstedt, m, ringle, cm & mena, ja (2012), 'an assessment of the use of partial least squares structural equation modeling in marketing research', journal of academic marketing science, vol. 40, no. 3, pp. 414-33. henseler, j, ringle, cm & sinkovics, rr (2009), 'the use of partial least squares path modeling in international marketing', in advances in international marketing, emerald, bingley, vol. 20, pp. 277-319. australasian journal of information systems volume 18 number 3 2014 70 holak, s & lehmann, d (1990), 'purchase intentions and the dimensions of innovation: an exploratory model', journal of product innovation management, vol. 7, no. 1, pp. 59-73. hoover, d & valencia, j (2011), 'ipads in the classroom: use, learning outcomes, and the future. presentation', in proceedings of the 2011 educause annual conference philadelphia, pa, <http://www.educause.edu/sites/default/files/library/presentations/e11/sess081/ipads%2bin %2bthe%2bclassroom.pdf>. horan, ta, tulu, b, hilton, b & burton, j (2004), 'use of online systems in clinical medical assessments: an analysis of physician acceptance of online disability evaluation systems ', in proceedings of the 37th hawaii international conference on system sciences ieee, hawaii. igbaria, m, zinatelli, n, cragg, p & m, cal (2007), 'personal computing acceptance factors in small firms: a structural equation model', mis quarterly, vol. 21, no. 3, pp. 279-305. jackson, c, chow, s & leitch, r (1997), 'toward an understanding of the behavioral intention to use an information system', decision sciences, vol. 28, no. 2, pp. 357-89. kagaari, j, munene, j & ntayi, j (2010), 'performance management practices, information and communication technology (ict) adoption and managed performance', quality assurance in education, vol. 18, no. 2, pp. 106-25. karahanna, e, straub, dw & chervany, ml (1999), 'information technology adoption across time: a cross-sectional comparison of pre-adoption and post-adoption beliefs', mis quarterly, vol. 23, no. 2, pp. 183-213. kelly, j & schrape, j (2010), '100 days with an ipad: lessons learnt and apps acquired', in proceedings of the curriculum, technology and transformation for an unknown future, ch steel, et al. (eds.), ascilite, sydney, pp. 484-6. knight, s & burn, j (2011), 'introducing the otam: exploring users' end perceptions of their ongoing interaction with adopted technologies', australasian journal of information systems, vol. 17, no. 1, pp. 5-41. kotler, p (2003), marketing management: analysis, planning, implementation, and control, 11th edn, prentice hall, englewood cliffs, nj. kukulska-hume, a (2012), 'how should the higher education workforce adapt to the advancements in technology for teaching and learning?', internet and higher education, vol. 15, no. 4, pp. 247-54. lai, j-y & chang, c-y (2011), 'user attitudes toward dedicated e-book readers for reading: the effects of convenience, compatibility and media richness', online information review, vol. 35, no. 4, pp. 558-80. lapczynski, p & calloway, l (2006), 'a scheme of technology acceptance for mobile computing', in m khosrow-pour (ed.), emerging trends and challenges in information technology, idea group incorporated, vol. 1. lee, mko, cheung, cmk & chen, z (2005), 'acceptance of internet-based learning medium: the role of extrinsic and intrinsic motivation', information and management, vol. 42, no. 8, pp. 1095104. lee, y, kozar, k & larsen, k (2003), 'the technology acceptance model: past, present, and future', communications of the association for information systems, vol. 12, no. 50, pp. 752–80. liu, i, chen, m, sun, y, wible, d & kuo, c (2010), 'extending the tam model to explore the factors that affect intention to use an online learning community', computers & education, vol. 54, no. 2, pp. 600-10. australasian journal of information systems volume 18 number 3 2014 71 lu, x & viehland, d (2008), 'factors influencing the adoption of mobile learning', in proceedings of the australasian conference on information systems univerity of canterbury, christchurch, new zealand, <http://aisel.aisnet.org/acis2008/56>. maddux, c & johnson, d (2010), 'global trends and issues in information technology in education', computers in schools, vol. 27, no. 3/4, pp. 145-54. malhotra, y & galleta, d (1999), 'extending the technology acceptance model to account for social influence: theoretical bases and empirical validation', in proceedings of the 32nd hawaii international conference on system sciences ieee, hawaii. matthieson, k (1991), 'predicting user intentions: comparing the technology acceptance model with the theory of planned behaviour', information systems research, vol. 2, no. 3, pp. 173-91. mcfarland, d & hamilton, d (2006), 'adding contextual specificity to the technology acceptance model', computers in human behaviour, vol. 22, no. 3, pp. 427-47. moore, g & benbasat, i (1991), 'development of an instrument to measure the perceptions of adopting an information technology innovation', information systems research, vol. 2, no. 3, pp. 192222. moran, m, hawkes, m & el gayar, o (2010), 'tablet personal computer integration in higher education: applying the unified theory of acceptance and use technology model to understand supporting factors', journal of education computing research, vol. 42, no. 1, pp. 79-101. murphy, a, farley, h, lane, m, hafeezbaig, a & carter, b (2013), 'mobile learning anytime, anywhere: what are our students doing? (acphis kit dampney best paper award for an educational issue)', in proceedings of the 24th australasian conference of information systems rmit, melbourne, australia, <http://mo.bf.rmit.edu.au/acis2013/306.pdf>. nair, i & das, v (2011), 'analysis of recent studies undertaken for assessing acceptance of technology among teachers using tam', international journal of computer applications, vol. 32, no. 8, pp. 38-46. nunnally, j (1979), psychometric theory, mcgraw-hill, new york. oh, s, ahn, j & kim, b (2003), 'adoption of broadband internet in korea: the role of experience in building attitudes', journal of information technology theory and application, vol. 18, no. 4, pp. 267-80. oldfield, j & cochrane, t (2011), 'equipping lecturers for the irevolution', in proceedings of the ascilite conference, g williams, et al. (eds.), hobart, pp. 919-29. ong, j (2011), apple maintains dominance of global market with 58% share in q4 2011, viewed 10th october 2012, <http://www.appleinsider.com/articles/12/01/26/apple_maintains_dominance_of_global_tablet _market_with_58_share_in_q4_2011.html > park, s (2009), 'an analysis of the technology acceptance model in understanding university students' behavioural intention to use e-learning', educational technology and society, vol. 12, no. 3, pp. 150-62. park, s, nam, m & cha, s (2011), 'university students' behavioural intention to use mobile learning: evaluating the technology acceptance model', british journal of educational technology, vol. 43, no. 4, pp. 1-13. pepperdine university 2012, ipad research study, pepperdine university, viewed 10th october 2012, <https://community.pepperdine.edu/it/tools/ipad/research/default.htm>. australasian journal of information systems volume 18 number 3 2014 72 podsakoff, p, mackenzie, s, lee, j & podsakoff, n (2003), 'common method biases in behavioral research: a critical review of the literature and recommended remedies', journal of applied psychology, vol. 88, no. 5, pp. 879-903. read, w, robertson, n & mcquilken, l (2011), 'a novel romance: the technology acceptance model with emotional attachment', australasian marketing journal, vol. 19, no. 4, pp. 223-9. reisdorf, b (2011), 'non-adoption of the internet in great britain and sweden', information, communication and society, vol. 14, no. 3, pp. 400-20. ringle, cm, wende, s & will, a (2005), smartpls release: 2.0 (beta), university of hamburg, viewed 20th september 2012, <http://www.smartpls.de >. saraf, n, langdon, c & gosain, s (2007), 'is application capabilities and relational value in interfirm partnerships', information systems research, vol. 18, no. 3, pp. 320-39. schaper, l & pervan, g (2007), 'an investigation of factors affecting technology acceptance and use decisions by australian allied health therapists ', in proceedings of the 40th hawaii international conference on system sciences ieee, hawaii, <http://www.computer.org/csdl/proceedings/hicss/2007/2755/00/27550141c.pdf>. shen, j (2010), 'the ebook lifestyle: an academic library perspective', the reference librarian, vol. 52, no. 1-2, pp. 181-9. shih, c & venkatesh, a (2004), 'beyond adoption: development and application of a use-diffusion model', journal of marketing research, vol. 68, no. 1, pp. 59-72. shurtz, s, halling, t & mckay, b (2011), 'assessing user preferences to circulate ipads in an academic medical library', journal of electronic resources in medical libraries, vol. 8, no. 4, pp. 311-24. siponen, m & vance, a (2010), 'neutralisation: new insights into the problem of employee information systems security policy violations', mis quarterly, vol. 34, no. 3, pp. 487-502. straub, d, keil, m & brenner, w (1997), 'testing the technology acceptance model across cultures: a three country study', information and management, vol. 33, no. 1, pp. 1-11. straub, d, boudreau, m & gefen, d (2004), 'validation guidelines for is positivist research', communications of the association for information systems, vol. 13, no. 24, pp. 380-427. tan, gwh, sim, jj, ooi, kb & phusavat, k (2012), 'determinants of mobile learning adoption: an empirical analysis', journal of computer information systems, vol. 52, no. 3, pp. 81-92. taylor, s & todd, pa (1995), 'understanding information technology usage: a test of competing models', information systems research, vol. 6, no. 2, pp. 144-76. tenenhaus, m, vinzi, v, chatelin, y-m & lauro, c (2005), 'pls path modelling', computational statistics & data analysis, vol. 48, no. 1, pp. 159-205. thompson, r, higgins, c & howell, j (1991), 'personal computing: toward a conceptual model of utilization', mis quarterly, vol. 15, no. 1, pp. 124-43. tornatzky, l & klein, k (1982), 'innovation characteristics and innovation adoption-implementation: a meta-analysis of findings', ieee transactions on engineering management, vol. 29, no. 1, pp. 28-45. wang, g, dou, w & zhou, n (2008), 'consumption attitudes and adoption of new consumer products: a contingency approach', european journal of marketing, vol. 42, no. 1/2. wiebrands, m (2012), 'the coming paradigm shift in computing interfaces and how academic libraries need to adapt', in proceedings of the iatul 2009 conference purdue university, <http://docs.lib.purdue.edu/iatul/2012/papers/9>. australasian journal of information systems volume 18 number 3 2014 73 wold, h (1985), 'systems analysis by partial least squares', in p nijkamp, et al. (eds), measuring the unmeasurable, martinus nijhoff, boston, ma. wu, c, kuo, y & wu, s (2013), 'investigating the antecedents of university students’ behavioral intention to use ipad for learning ', international journal of e-education, e-business, emanagement and e-learning, vol. 3, no. 6, pp. 468-71. australasian journal of information systems volume 18 number 3 2014 74 appendix a university staff adoption of ipads model – survey instrument construct survey items compatibility with work (cw) compatibility1 an ipad/tablet is compatible with most aspects of my work. compatibility2 an ipad/tablet fits well with the way i like to work. compatibility3 an ipad/tablet is compatible with other systems i use. social status (ss) social status1 people in my organisation who use an ipad/tablet have more prestige than those who do not. social status2 people in my organisation who use an ipad/tablet have a high profile. social status3 having an ipad/tablet is a status symbol in my organisation. ict support (icts) ict support1 guidance was available to me in selection of suitable ipad/tablet apps ict support2 specialized instruction on the use of an ipad/tablet was available to me. ict support3 a specific person (or group) is available for assistance with ipad/tablet difficulties. perceived usefulness (pu) pu1 an ipad/tablet in my job enables me to accomplish some tasks more quickly. pu2 an ipad/tablet helps improve my job performance. pu3 an ipad/tablet in my job increases my productivity. pu4 an ipad/tablet enhances my effectiveness on the job. pu5 an ipad/tablet makes it easier to do some aspects of my job. pu6 an ipad/tablet is useful in my job. perceived ease of use (peu) peu1 learning to use an ipad/tablet is easy for me. peu2 it is easy to make an ipad/tablet do what i want it to. peu3 it is clear and understandable how i interact with ipad/tablet through touch. peu4 it has been easy for me to become skillful at using an ipad/tablet. peu5 it is easy to use an ipad/tablet. attitude to use (atu) attitude1 using an ipad/tablet is good experience. attitude2 using an ipad/tablet makes my work more interesting. attitude3 using an ipad/tablet is a fun experience. attitude4 i like using an ipad/tablet in my work. intention to use (itu) intention1 i intend to use an ipad/tablet as a routine part of my job over the next year. intention2 i intend to use an ipad/tablet more at every opportunity. intention3 i plan to increase my use of an ipad/tablet over the next year. frequency of actual use (foau) frequency of actual use in work; 7 point likert scale; less than once a week through to constantly using during a day age what is your age? 18-25; 26-35; 36-45; 46-55, over 55 previous experience using iphone or smartphone (pest) have you previously used an iphone or smart phone prior to using an ipad? table 8 construct survey items (source: adapted previous instruments for this research. see table 1 for specific references for construct survey items)] paper title (paper title style) australasian journal of information systems gupta, pandey, gaur & vohra 2019, vol 23, research on the role of tech in workforce mgt preface 1 preface to research on role of technology in workforce management manish gupta ibs hyderabad hyderabad india. manish.gupta.research@gmail.com jatin pandey indian institute of management indore india. jighyasu gaur t a pai management institute, manipal india. neharika vohra organizational behaviour indian institute of management ahmedabad india. technology has become omnipresent in our lives and workplace is no exception. recent advancements in human resource management (hrm) and organizational behaviour (ob) are predominantly technology driven that require techno-savvy human resource managers (lengnick-hall & lengnick-hall, 2018). in general, the quest for effective and efficient workforce management has led to embracing digitization (berber et al., 2018). it does affect the several stages of a typical human resource process which includes stages such as recruitment, selection, performance appraisal, demand forecasting, supply forecasting, job description, job specification, job analysis, job evaluation, training and development, career planning and development, and succession planning among others (tooranloo, et al., 2017). technology is a double edged sword with potential to make or break organizations with major consequences for employees and employers. we are experiencing the impact of industry 4.0 on hr functions. for example, chat-bots are assisting in screening candidates (sivathanu & pillai, 2018), analytics-powered employee demand and supply prediction (sommers, 1999), and smart offices connected through cloud and social networks (filos, 2008). crowdsourcing is on a record spree, with deloitte forecasting five billion people engaged in crowdsourcing projects by 2020 (howe, 2006; grewal-carr & bates, 2016), performance appraisal of employees can be done using a mobile app (samuel et al., 2014), and sophisticated human resource information systems (hris) manage all hr processes ranging from the simple ones such as leave management and salary slip generation to more complex processes such as succession planning (nagendra & deshpande, 2014). rapidly developing technology which is available to both employers as well as employees can become a source of sustained competitive advantage or a catalyst of australasian journal of information systems gupta, pandey, gaur & vohra 2019, vol 23, research on the role of tech in workforce mgt preface 2 ineffective utilization of organizational assets. this pervasiveness, peculiarity and potential of technology at workplace needs extensive exploration and examination. thus, the objectives of this special section are (1) to gain theoretical insights into the relationship between technology and employee level factors, (2) to capture the increasingly changing nature of workforce and its management, and (3) to examine role of external factors such as social, cultural, and market conditions among others in affecting the relationship between employee behaviour and technology. organizations want and would want the technology and the talent to work in tandem. organizations thus need to use newer web-based technologies to attract these talented professionals. it has been observed that the firms have not been able to attract the best talent due to several reasons including the reach of information to the job aspirants. however, with the advent of web 2.0 technologies such as podcasts, blogs and online employee testimonials, the employers can communicate and attract the talent far easier than ever before. the first article, talent attraction through online recruitment websites: application of web 2.0 technologies, investigates the impact of web 2.0 technologies such as podcasts, blogs and online employee testimonials talent attraction. the findings show a positive impact of video podcasts and realistic employee testimonials (presented through third-party blogs) on job-seekers’ perceived quality and credibility of the job advertisement. the add-on advantage of the web 2.0 technologies is increase in brand value of the employers. technology thus becomes an important facilitator of the absorption of talent in today’s team-based organizations. these teams now are now spread across geographical locations and global virtual teams (gvts) have become an indispensable part of organizations. organizations are receiving projects from across the globe. to complete these projects, organizations need virtual teams that have members from different cultural backgrounds and time zones, may (or may not) meet inperson to take complex decisions. the major challenge for the organizations to make gvts successful is engaging the employees who are not physically present. employee engagement requires employees’ cognitive, behavioural and physical state directed towards organizational outcomes. the second article, drivers of employee engagement in global virtual teams, conceptualizes five drivers of employee engagement namely cultural intelligence, communication (formal and informal), technology, trust, and individual maturity through interpretive analysis employees working in organizations that extensively use gvts for achieving its strategic goals. technology also facilitates behaviours that are deviant to organizational norms and this particular aspect is studied in the third article. this article brings into light the supervisor who is an important stakeholder that nurtures the acquired talent in organizations. supervisor plays a critical role in enhancing and deterring subordinate behaviours; the third article, impact of supervisors’ perceived communication style on subordinate’s psycap and cyberloafing, shows how the passive, aggressive, and assertive communication styles of the supervisor affect the psychological capital (psycap) and cyber-loafing. results from 680 full-time managerial employees show that supervisors assertive and aggressive communication styles impacted psychological capital which in turn impact cyber-loafing. it is essential for organizations to ensure that the acquired talent is retained in the organization and allowed to grow. information technology (it) industry provides a knowledge-based working environment to its employees in terms of the flexibility for progression of career and management of family along with work. the second article investigates specifically for it australasian journal of information systems gupta, pandey, gaur & vohra 2019, vol 23, research on the role of tech in workforce mgt preface 3 professionals how the fit/misfit impacts career satisfaction. results of the fourth study, expectations from technology and career satisfaction: a study among it professionals in india, show that the perceived fit as well as misfit between experienced and expected growth in career, work-life balance impacts career satisfaction. we conceptualize today’s organizations as talent repositories and based on the synthesis of these papers we propose technology-enhanced talent management for new age organizations. initially, the use of technology is essential for attracting potential talent followed by a fit of selected talent and the job. supportive supervisors and technology for virtual engagement can inhibit deviant behaviours and lead to a refined talent capital for an organization that bestows sustained competitive advantage. figure 1 illustrates how organizations that are essentially talent repositories can be benefitted by the three technological forces. figure 1: organizations as talent repositories the organizations can derive the aforesaid benefits (1) by technology-enabled attraction and acquisition, (2) through technology-enabled fit, engagement, satisfaction and balance between work and family life, and (3) through supportive existing talent in the organization in the form of supervisors who can prevent technology fostered deviant behaviours like cyber-loafing. direction for future research organizations are increasingly seen as talent repositories and technology is playing a vital role in recruiting, training, retaining, and assessing talent. it is essential that the future research focuses on identifying the issues, whether technological or technical, in making the data technology friendly. once these concerns are identified, the research on addressing these issues would be required. though this special section has taken the initiative of not only identifying three issues relating to technology enabled fit with the job attitudes, supervisor support, and technology enhanced attractions, the theme under investigation is still in its organization as talent repositories technology enhanced attraction and acquisition technology enabled fit, engagement, satisfaction and balance supportive supervisors to prevent technology fostered deviant behaviours australasian journal of information systems gupta, pandey, gaur & vohra 2019, vol 23, research on the role of tech in workforce mgt preface 4 nascent stage. thus, we urge the future studies to explore, capture, and examine the role of technology in managing a workforce at department, branch, subsidiary, organizational, country, and inter-country level. moreover, studies comparing different contexts such as interbranch, inter-industry, cross-country would help researchers and practitioners understand the dynamics of the relationship between technological and hr variables in different situations. references berber, n., đorđević, b., & milanović, s. (2018). electronic human resource management (ehrm): a new concept for digital age. strategic management, 23(2), 22-32. grewal-carr, v., & bates, c. (2016). the three billion: enterprise crowdsourcing and the growing fragmentation of work. deloitte consulting llp, available at: https://www2.deloitte.com/content/dam/deloitte/de/documents/innovation/us-consenterprise-crowdsourcing-and-growing-fragmentation-of-work%20(3).pdf howe, j. (2006). the rise of crowdsourcing. wired magazine, 14(6), 1-4. lengnick-hall, m. l., & lengnick-hall, c. a. (2018). strategic management approach to technology-enabled hrm. in e-hrm (pp. 45-61). routledge. nagendra, a., & deshpande, m. (2014). human resource information systems (hris) in hr planning and development in mid to large sized organizations. procedia-social and behavioral sciences, 133, 61-67. samuel, o. w., omisore, m. o., & atajeromavwo, e. j. (2014). online fuzzy based decision support system for human resource performance appraisal. measurement, 55, 452-461. sivathanu, b., & pillai, r. (2018). smart hr 4.0–how industry 4.0 is disrupting hr. human resource management international digest, 26(4), 7-11. somers, m. j. (1999). application of two neural network paradigms to the study of voluntary employee turnover. journal of applied psychology, 84(2), 177. tooranloo, h. s., azadi, m. h., & sayyahpoor, a. (2017). analyzing factors affecting implementation success of sustainable human resource management (shrm) using a hybrid approach of fahp and type-2 fuzzy dematel. journal of cleaner production, 162, 1252-1265. copyright: © 2019 gupta, pandey, gaur & vohra. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems gupta, pandey, gaur & vohra 2019, vol 23, research on the role of tech in workforce mgt preface 5 paper title (paper title style) australasian journal of information systems sedera & atapattu 2019, vol 23, research note polynomial regression and response surface methodology 1 polynomial regression and response surface methodology: theoretical non-linearity, tutorial and applications for information systems research darshana sedera monash university australia darshana.sedera@gmail.com maura atapattu university of queensland australia abstract information systems (is) studies regularly assume linearity of the variables and often disregard the potential non-linear theoretical interrelationships among the variables. the application of polynomial regression and response surface methodology can observe such non-linear theoretical assumptions among variables. this methodology enables to examine the extent to which two predictor variables relate to an outcome variable simultaneously. this paper utilizes the expectation confirmation theory as an example and provides a methodological commentary that illustrates a step-wise process for conducting a polynomial regression and response surface methodology. keywords: quantitative analysis, polynomial regression, response surface methodology, non-linearity 1 introduction non-linearity of variables is a key underlying assumption in most research conducted in information systems (is), where the researchers assume that there is no interplay between variables. this approach is particularly important in multi-source feedback research, where it allows a researcher to examine the extent to which the combinations of two predictor variables relate to an outcome variable (shanock et al. 2010). for an example, the technology acceptance stream of research, which have been developed using the theory of planned behaviour (ajzen 1991) and theory of reasoned action (ajzen and fishbein 1980), highlight non-linearity. both these theories and their derivations like unified theory of acceptance and use of technology, require the researcher to consider non-linearity of the constructs. similarly, the expectation confirmation theory (ect) (oliver 1977; oliver 1980; oliver et al. 1994) is also a widely employed theory in is research (lim et al. 2013), with non-linear assumptions. despite such theories requiring non-linearity, is research rarely treats them as linear theories. for example, venkatesh and goyal (2010, p. 282), citing technology acceptance model and expectation confirmation theory1, state that “prior technology acceptance research in general and edt research in is in particular is limited to linear models. linear models fail to reveal complexities that are anticipated in theories of congruence, such as edt in which attitudes and behaviors result from 1 a representative collection of past information systems (is) studies employing ect include (fan and suh 2014; lankton et al. 2014; lankton and mcknight 2012; mckinney et al. 2002) australasian journal of information systems sedera & atapattu 2019, vol 23, research note polynomial regression and response surface methodology 2 the congruence between expectations and experiences.” failure to adhere to the non-linear assumptions compromise research and practical explanations of the research, compromising the credibility and insightful findings. moreover, such linear models assume that there is a similar effect for multiple independent variables on the dependent variable (venkatesh and goyal 2010). therefore, even if the actual relationship between the component measures and the outcome measure is curvilinear, linear models would oversimplify the relationship and mask the true relationships among the variables (edwards 2002; edwards and cooper 1990). (fan and suh 2014; lankton et al. 2014; lankton and mcknight 2012; mckinney et al. 2002) polynomial regression (edwards and parry 1993) and response surface methodology (box and draper 1987; khuri and cornell 1987) have been acknowledged as techniques that allow researchers to engage in testing the non-linear assumptions, allowing greater exploratory and explanatory potential of the phenomenon under investigation. specifically, the polynomial regression (edwards and parry 1993), together with the response surface methodology (box and draper 1987; khuri and cornell 1987), provide the basis for testing and interpreting the features of surfaces corresponding to polynomial quadratic regression equations between two predictor variables and an outcome variable (shanock et al. 2010). the non-linear analysis provides an opportunity to investigate beyond the simple relationships between two (or more) variables and extend the understanding of the interaction effects of the variables. the theoretical simplification through linearity assumptions not only hinder the possible opportunities to better understand how complex relationships between variables unravel, but also carries the dangers of mis-interpretation and/or overstating the effects, ultimately leading to incomplete, partial or erroneous interpretations of the findings (see ping 2002; titah and barki 2009). naturally, the employment of non-linear postulations in analysis has the potential to uncover the complex and contingent relationship that the theory originally suggests between the constructs. despite the aforementioned advantages, only a handful of is researchers have employed the polynomial regression and surface response technique in their analyses (see brown et al. 2008; roberts and grover 2012b; titah and barki 2009). this paper attributes the paucity of studies that relax linearity assumptions to (i) the conceptual challenge of theorizing complex three dimensional non-linear relationships, (ii) interpretive difficulties of the results and (ii) the limited existence of discussions on the method. as such, the objective of this paper is to present: (i) the applicability of the polynomial regression and surface response technique to the main-stream is research, (ii) how new research questions and insights can be derived using the polynomial regression and surface response technique, (iii) guidelines for developing hypotheses corresponding to two related predictor variables and their dependent variable, (iv) the procedure to conduct a polynomial regression with its corresponding response surface graph, (vi) an application of the methodology in a field setting and (vii) the benefits of the polynomial regression and response surface methodology. the paper proceeds in the following manner. first, the paper provides an overview of polynomial regression and response surface method. therein, it introduces the assumptions and interpretations of the method. next, it provides results of an analyses demonstrates the assumptions and interpretations, employing the expectation confirmation theory. it provides a step-by-step application of the methodology, as well as how the results can be interpreted. finally, the paper concludes with the contribution to academia and practice. australasian journal of information systems sedera & atapattu 2019, vol 23, research note polynomial regression and response surface methodology 3 2 background the use of polynomial regression together with response surface methodology (henceforth referred to as ‘polynomial regression,’ since response surface method also follows the same regression principals) in is research is scant. to the best of our knowledge, only a small number of studies have even suggested the importance of polynomial regression and response surface method (see brown et al. 2008; roberts and grover 2012b; titah and barki 2009). however, a review of the literature reveals that polynomial regression could have been applied in most of the is studies to derive deeper and better theoretical explanations. for example, appendix a includes a sample of 14 articles from the two leading journals in the discipline, mis quarterly and information systems research, that could have employed polynomial regression to improve their insights. it is highlighted that the original intention of these studies was not to employ polynomial regression. for example, in the study of alternative sampling in digital age by steelman et al. (2014), inclusion of polynomial regression could have complemented the traditional structural model testing, further elaborating (i) how different combinations of perceived usefulness and perceived ease of use impact the behavioural intention? and (ii) how combinations of perceived usefulness and perceived ease of use impact the behavioural intention across different cohorts of samples. observing the new research questions, one might come to an invalid conclusion that the focus of polynomial regression is limited to observing the impact of two variables in combination on the dependent variable. in fact, the polynomial regression can assess the complete interaction effect of two variables on the dependent variable. this means, that the studies in appendix a could have observed (i) the optimal combinations and (ii) the effect of variables on a continuum. as such, it can be argued that the is researchers have missed opportunities to provide greater insights by not adhering to the notions of polynomial regression and response surface methodology. with the objective of providing the methodological awareness, the next section provides an overview of the method, its fundamental assumptions and details of how results are interpreted. 3 overview of polynomial regression and response surface methodology polynomial regression is a technique that can be used to model the relationship between multiple independent variables (x and y) in relation to a dependent variable (z) through a non-linear relationship (shanock et al. 2010). citing edwards (1994), venkatesh and goyal (2010, p. 286) state “based on a basic theoretical model, z = f (x, y), this technique allows the examination of curvilinear terms so that a more accurate picture of the relationship between component measures and an outcome variable can be detected.” polynomial modelling involves a hierarchical analysis of polynomial equations, which continues until the variance explained by the next higher-order equation is not statistically significant. for example, a quadratic polynomial equation is z = b0 +b1 x + b2 y + b3 x2 + b4 xy + b5 y2 + e here, component scores for x and y (first stage x1 and y1) are entered to test their linear relationship with z in hierarchical stages. during the second stage of analysis, higher-order terms (x2 and y2) are added into the equation along with the product term (xy) to test for the presence of curvilinear (here, quadratic) relationships. subsequently, cubic terms could be australasian journal of information systems sedera & atapattu 2019, vol 23, research note polynomial regression and response surface methodology 4 added into the equation in order to test for the presence of further higher-order curvatures (edwards 2002; edwards and van harrison 1993). similar to polynomial regression (edwards and parry 1993), response surface methodology (box and draper 1987; khuri and cornell 1987) is a common technique that analyse nonlinearities. moreover, the polynomial regression, together with the response surface methodology, provides the basis for testing and interpreting the features of surfaces corresponding to polynomial quadratic regression equations. especially, these techniques are popular in micro and macro organizational literature to investigate congruence and/or discrepancies between variables (shanock et al. 2010). moreover, polynomial modelling and response surface methodology have been used widely in organization behaviour (e.g., hecht and allen 2005; kristof 1996), marketing (e.g., kim and hsieh 2003), and personnel psychology (e.g., shaw and gupta 2004). these techniques allow a researcher to examine the extent to which the combinations of two predictor (independent) variables relate to an outcome (dependent) variable. further, this has been widely used in multi-source feedback research (shanock et al. 2010). the combination of the two methods provides the statistical insight required to examine the nuanced views of tripartite relationships by investigating the variables in a three-dimensional space where it provides the relationships between combinations of two predictor variables and an outcome variable (shanock et al. 2010). 3.1 assumptions of polynomial regression generally, the polynomial regression and response surface methodology can be used for any circumstances in which the phenomenon involves an investigation of how combinations of two predictor variables relate to an outcome. however, edwards (2002) outlines few assumptions that must be met in order to apply this analytical technique. first, the two predictor variables must be commensurate and thus should represent the same conceptual domain. as such, the relationship between the predictor variables can be interpreted in a meaningful way in relation to the dependent variable. for an example, if a researcher is trying to predict a customer’s satisfaction with the service provided in a hotel, it is more meaningful to look at customer satisfaction (outcome variable z) in relation to the discrepancy or agreement between the customer’s expectations (variable x) and the actual customer experience (variable y). second, the predictor variables (x and y) must be measured on the same numeric scale; or, when the scales are different, the scales should be transformed to a standardized scale so that the degree of correspondence between the variables can be determined in a more accurate manner. for example, both variables are required to be measured on a 5-point or 7-point likert scale (e.g., “strongly disagree” to “strongly agree” / “not at all” to “very frequently”). in case the two scales are different (e.g., 7-point likert and 5-point likert scale), the scales need to be standardized to a single scale in order to place both variables on a common metric (see harris et al. 2008). third, as with any regression technique, all the standard assumptions relating to the multiple regression technique as explained by tabachnick and fidell (2007) need to be met in order to apply this technique. moreover, polynomial regression could be used in the place of moderated regression to study the relationships between combinations of variables (x and y). the reason for this is that the polynomial regression has more explanatory power compared australasian journal of information systems sedera & atapattu 2019, vol 23, research note polynomial regression and response surface methodology 5 to the traditional moderated regression as it provides nuanced views on the relationships between different combinations of predictor variables (x and y) and a dependent variable (z) in a three-dimensional space (shanock et al. 2010). finally, the polynomial regression and response surface methodology can be used when the underlying theoretical assumptions suggest non-linearity between the independent and dependent variables. for example, the original theoretical assumptions of the ect suggest non-linearities in the relationships between expectations, experience and satisfaction (oliver 1977; oliver 1980). thus, studies that use ect as the underlying theoretical lens could employ polynomial regression together with response surface methodology to establish nuanced views of the relationships between the three variables of expectation, experience and satisfaction (brown et al. 2012; brown et al. 2014; brown et al. 2008). once it has been ensured that the above assumptions are met then the polynomial regression needs to be run to resolve the polynomial equation and to obtain the outputs. then, the polynomial outputs needs to be projected in a three-dimensional response surface as the higher order polynomial outputs often result from a polynomial model are difficult to interpret (edwards 2001). shanock et al. (2010) provides a detailed stepwise guidance for conducting polynomial regression and creating the response surface using polynomial outputs. 3.2 interpretation of the response surface the interpretation of the resultant response surface is accompanied by four properties (denoted using a1, a2, a3 and a4), that are relating to the slope and the curvature of response surface. the property a1 is related to the slope or the inclination of the surface along the agreement line (x=y) whereas the property a2 relates to the curvature of the x=y line. similarly, the properties a3 and a4 relate to the slope and the curvature of the surface along the disagreement line (x= negative y) respectively. as such the four indicators a1, a2, a3 and a4 relate to the inclination and the curvature of the surface along the two perpendicular axes x = y and x = negative y. several different research questions can be answered through the resultant response surface and the calculated surface values. for example, using a response surface, a researcher can examine: (i) the agreement or alignment between two predictor variables on an outcome variable, (ii) the discrepancy between two predictor variables on an outcome variable, (iii) the direction and magnitude of the discrepancy between two predictor variables on an outcome variable, (iv) the nature of the synergy between two predictor variables (positive or negative) and outcome variable, and (v) the mediation effect of one predictor variable on the other predictor variable (complete or partial) on an outcome variable. it is noted that, while individually useful, the aforementioned five types of analyses must be viewed in combination, rather than in isolation. moreover, as exemplified in table 2, each line must be assessed together with its significance and not just on the shape or slope of the surface. panel a in figure 1 provides the x, y and z axes and lines of congruence and incongruence, while panel b in figure 1 includes a sample of a response surface graph using two predictor variables (x and y) and one dependent variable (z). australasian journal of information systems sedera & atapattu 2019, vol 23, research note polynomial regression and response surface methodology 6 figure 1. a sample graph for a response surface of two predictor variables (x and y) and an outcome variable (z) 3.2.1 agreement between two predictor variables as related to an outcome variable a key mechanism of the response surface methodology allows a researcher to study the agreement between two predictor variables in relation to an outcome variable. in panel b of figure 1, the line between point a and b of the surface area of the graph (marked as abcd) depicts the perfect agreement or alignment between the two predictor variables (which means x is equal to y). whilst the surface property a1 (slope or the inclination of the surface along agreement line, x is equal to y) explains the slope of the surface along a to b (i.e. the agreement line), a2 (curvature of the x is equal to y) explains the curvature or the non-linearity of the surface along the a-b line (panel b). if the test value a1 is positive (negative), the outcome variable increases (decreases) as the two predictor variables increase. whilst the significant value of a2 indicates a non-linear (curvilinear) slope along the agreement axis, a significant positive a2 value suggests an upward convex curvature for the surface along the a-b line, whilst a significant negative a2 value suggests a downward concave curvature for the surface along the a-b line. if the agreement between two predictor variables shows a linear additive relationship to the outcome variable a1, the slope is positive but if it shows a linear additive relationship to the outcome variable, thus a2, the curvature of the surface along the line of agreement would not be significant. moving along the solid line from the front of the graph in panel b of figure 1 (point b) to the back (point a), the line of perfect agreement of two predictor variables as related to outcome variable (z) has a positive slope. further, the lowest level of the outcome variable is at the front corner of the graph on the a-b line (point b) where both predictor variables are low, and increasingly higher towards the back corner of the graph on the a-b line (point a) where both predictor variables are high. 3.2.2 disagreement between two predictor variables as related to an outcome variable disagreement, or incongruence, explains how the discrepancy between two predictor variables relates to an outcome variable. the line perpendicular to the line of perfect agreement (the a-b line on the abcd response surface) represents the line of incongruence. the dotted line on the floor of the graph in panel b of figure 1 depicts this line of incongruence (x= negative y), that is, when the two predictor variables are not in agreement. as the surface australasian journal of information systems sedera & atapattu 2019, vol 23, research note polynomial regression and response surface methodology 7 property a4 explains the curvature along the line x= negative y, a significant negative curvature depicts that the dependent variable drops as the difference between the two predictor variables widens (i.e. increasingly discrepant as moving away from the line of agreement). whilst a significant negative a4 indicates a concave surface, a significant positive a4 indicates a convex surface. the concave surface explains that the value of the outcome variable decreases more sharply as the degree of discrepancy between the two predictor variables increases; whilst the concave surface explains vice versa. the surface along the line of disagreement of the graph in panel b of figure 1 displays a concave surface and shows a sharp decrease in the outcome variable as the discrepancy between the predictor variables widens. 3.2.3 direction and magnitude of the discrepancy between two predictor variables as related to an outcome variable the slope along the line of incongruence between the two predictor variables (x = negative y) as related to the dependent variable explaining the extent to which the direction of the discrepancy matters to the outcome variable. as the surface property a3 relates to the slope of the line of incongruence as it relates to the outcome variable, a significant value for a3 shows that the direction of discrepancy indeed matters to the changes in the outcome variable. a significant positive a3 indicates that the outcome variable decreases when predictor variable 1 (x) is less than predictor variable 2 (y) and vice versa. moreover, the graph in panel b of figure 1 shows that moving to left or right of the ‘agreement line’ on the floor (i.e. towards points x and d of the abcd response surface on the graph respectively), where the two predictor variables become more and more discrepant one could observe the changes in the outcome variable as a result of the discrepancies between predictor (x and y) variables. such observations will be evident through the slope (a3) and curvature (a4) of the surface along c and d. moving away from the agreement line towards point c along the horizontal dotted line on the floor of the graph, the predictor variable 2 (y) increases and predictor variable 1 (x) decreases and the outcome variable z decreases. similarly, moving from the agreement line towards point d along the dotted line on the floor of the graph, the predictor variable 1 (x) increases as predictor variable 2 (y) decreases. this highlights the observations that can be made on the discrepancy and direction of assimilation between two predictor variables using the resultant response surface. 3.2.4 synergy between two predictor variables as related to an outcome variable synergy between two predictor variables has been described as the combined effect of two factors (x and y), where the combined effect could either create a negative synergy (making the combined effect of the two factors less than the sum of each factor’s separate effect) or a positive synergy (making the combined effect of the two factors greater than the sum of each factor’s separate effect) (titah and barki 2009). positive synergy explains the complementarity of two variables, whereby it explains that the increase in either factor increases the impact of the other (titah and barki 2009). similarly, negative synergy explains the substitutability of two variables, whereby it explains that the increase in one factor decreases the impact of the other. the lines of a to c and b to d of the abcd surface provide an example of positive synergy. the b-d line represents the relationship between predictor variable 1 and the outcome variable when predictor variable 2 is at its lowest. the a-c line represents the relationship between predictor variable 1 and the outcome variable when predictor variable 2 is at its highest. both australasian journal of information systems sedera & atapattu 2019, vol 23, research note polynomial regression and response surface methodology 8 lines show that the outcome variable increases with the increase of predictor variable 1 (points c to a and b to d). however, the rate of increase (i.e. slope of the curve) is higher in the c-a curve compared to the c-d curve. the in-between line on the response surface that runs parallel to the two a-c and b-d curves also shows that the slope of the curve increases when moving from the b-d line towards the a-c line. this shows that, as predictor variable 1 increases, it positively influences the relationship between predictor variable 1 and the outcome variable z, suggesting a positive synergy between the two predictor variables. alternatively, the e-c and bd lines of the ecbd response surfaces provide an example of negative synergy. the b-d line represents the relationship between predictor variable 1 and the outcome variable when predictor variable 2 is at its lowest. the e-c line represents a hypothetical relationship between predictor variable 1 and the outcome variable when predictor variable 2 is at its highest. both lines show that the outcome variable increases with the increase of predictor variable 1 (points c to e and b to d). however, the rate of increase (i.e. the slope of the curve) is less in the c-e curve compared to the c-d curve. similar to the earlier example, if lines were to be drawn on the hypothetical ecbd response surface, hypothetical lines that runs parallel to the e-c and b-d lines, it would show that the slope of the curve decreases moving from the b-d line towards the e-c line. this shows that, as predictor variable 1 increases, it negatively influences the relationship between predictor variable 1 and the outcome variable z and suggests a negative synergy between the two predictor variables. 3.2.5 test of mediation mediation in general entails the intervening effect of an antecedent variable on a dependent variable. thus, mediation refers to the intervening effect of one of the predictor variables on the relationship between the other predictor variable and the outcome variable. the discussion herein focuses on the mediation of predictor variable y (2) on the relationship between predictor variable x (1) and the outcome variable z. two types of mediation, namely, partial mediation and complete mediation, are discussed in the literature (see roberts and grover 2012a). to discuss the two types of mediation, this section uses the two response surfaces – abcd and abdf – of the graph in panel b of figure 1. however, the argument of mediation or moderation effect of a variable must be carefully made only through a theoretical grounding – rather than using polynomial equations as a tool of exploration. venkatesh and goyal (2010) highlight that advances in moderation and mediation (e.g., edwards and lambert 2007) have allowed researchers to incorporate the effects of moderators and mediators into polynomial models in organizational behaviour. we take the abcd response surface of the graph to discuss partial mediation. both a-c and b-d lines (or curves) of the graph explain the relationship between predictor variable x and the outcome variable z. the b-d line of the graph explains the relationship between predictor variable x and the outcome variable z when predictor variable x is absent or at its lowest, whilst the a-c line explains the relationship between predictor variable x and the outcome variable z when predictor variable x is at its highest. both the a-c and b-d lines in the graph show that as predictor variable x increases, the outcome variable z also increases as a result. when moving from the b-d line towards the a-c line on the response surface, it shows that when the magnitude of the predictor variable y increases, the relationship between predictor variable x and the outcome variable z (i.e. the slope or curvature between predictor variable x and the outcome variable z changes) changes slightly. it means that predictor variable y australasian journal of information systems sedera & atapattu 2019, vol 23, research note polynomial regression and response surface methodology 9 mediates the relationship between predictor variable x and the outcome variable z, but the impact of the predictor variable y is only partial. next, we take the abdf response surface of the graph to discuss complete mediation. similar to the above discussion, both the af and bd lines (or curves) of the graph explain the relationship between predictor variable x and the outcome variable z. whilst the b-d line of the graph explains the relationship between predictor variable x and the outcome variable z when predictor variable y is absent or at its lowest, the f-a line explains the relationship between predictor variable x and the outcome variable z when predictor variable y reaches its highest. whilst the b-d line shows that, as predictor variable x increases, the outcome variable z also increases consequently, the f-a line shows no change to the outcome variable as predictor variable x increases when predictor variable y is at its highest. when moving from the b-d line towards the f-a line on the response surface, it shows that when predictor variable y increases, the relationship between predictor variable x and the outcome variable z changes substantially. this explains that predictor variable y acts as a strong mediator in the relationship between predictor variable x and the outcome variable z. 4 applying polynomial regression to a study of ect having demonstrated the methodological approach of polynomial regression and response surface, the following section of the paper uses primary data collected by the authors to demonstrate the theoretical and practical applications of polynomial regression and surface response method. expectation confirmation theory (ect – also known as the expectation disconfirmation theory) is an ideal theoretical example to demonstrate the strengths of the polynomial regression and surface response method. the measurement of confirmation in ect involves comparing two distinct component measures: pre-exposure expectation and post-exposure experience (venkatesh and goyal 2010). therein, instead of testing the linear model, the method allows us to determine the difference in the absolute level of confirmation, when the values of both expectations and experiences are high (rather than when both are low, a distinction that is not possible to assess using direct measures) (venkatesh and goyal 2010). to test the ect purported relationships between expectations, experience and customer satisfaction, we gathered data from 171 australian consumers prior to their retail experience using a retail mobile app (with a response rate of 40%). prior to analysis, we conducted a wave analysis (armstrong and overton 1977) to assess the impact of non-respondent bias, whereby the respondents were grouped into early and late respondents and online and offline respondents and comparisons were made according to the respondents’ age (under 21 years = 19%, 22-32 = 37%, 33-43 = 27%, 44-55 = 11%, over 56 = 6%) and gender (46% male, 54% female). our analysis revealed no significant differences between early and late respondents or between the other combinations. based on our findings, non-response bias does not appear to impact on our study. moreover, the assessment of outliers of the sample found no abnormalities. in the linear model, the theoretical constructs of ‘expectation’ and ‘experience’ explained 63% of the variance in the dependent variable – ‘satisfaction’ (r2 = 0.63). the beta value of path between expectation and satisfaction was 0.002 significant at 0.5, while the beta of the path between experience and satisfaction was 0.645, significant at 0.001. australasian journal of information systems sedera & atapattu 2019, vol 23, research note polynomial regression and response surface methodology 10 4.1 theorizing for polynomial regression and response surface methodology considering the analytical potential of the polynomial regression (and response surface methodology) guidelines on theorizing tri-partite relationships for various research questions that have been discussed earlier would be of great value. theorizing a tri-partite interaction effect between two predictor variables with an outcome variable in a three-dimensional space and developing testable hypotheses is a challenging task. as such, the section below provides a practical guideline for theorizing the five views of the three variables taking ect (oliver 1977; oliver 1980) as an example. 4.1.1 agreement between two predictor variables as related to an outcome variable as the two predictor variables represent the two axes of x and y, the agreement between two predictor variables corresponds to the line representing x equals to y coordinates. thus, the agreement between two predictor variables and their relationships with the outcome variable corresponds to the response surface along the x equals to y axis. in other words, how the outcome variable is influenced by the alignment between two predictor variables when both variables are at their lowest to the highest is evident in the behaviour of the response surface corresponding to the x equal to y line. in the case of the example using ect, this paper examines the tri-partite relationship between expectation, experience and satisfaction. expectation and experience are the two predictor variables, while satisfaction is the outcome variable. also, observations suggest that it is more likely that one reaches a high level of satisfaction when high expectations are met with matching high experiences as compared with the level of satisfaction derived as a result of matching low expectations with low experience. whilst ect suggests the customer expectations need to be matched to satisfy the customers; it can be argued that, the alignment between customer expectations and their actual experience is positively related to customer satisfaction. 4.1.2 disagreement between two predictor variables as related to an outcome variable disagreement refers to the mismatch or the misalignment. thus, in the context of our discussion the term misalignment refers to the incongruity between the two predictor variables. as the alignment line represents the x=y coordinates, the misalignment refers to the line perpendicular to line of alignment, which is x= negative y line. as the z axis represents the outcome variable satisfaction, the surface corresponds to the x= negative y line and denotes how the misalignment between two predictor variables (in our case it is expectation and experience) relates to the satisfaction. also, the interception between x=y line and x= negative y depicts the point in which the two predictor variables having minimal mismatch, where moving away from the interception of the two lines to either direction (the left or right) shows the degree of discrepancy (mismatch) between the two predictor variables. herein the conceptual challenge is to make theoretical predictions about the mismatch between two predictor variables and its relationship to the outcome variable in a threedimensional space. in the example provided, as conceived in ect, expectations and experiences need to be matched for satisfaction and a mismatch between the two could create dissatisfaction. hence, it can be argued that, the incongruence between customer expectations and their experience is negatively related to customer satisfaction. australasian journal of information systems sedera & atapattu 2019, vol 23, research note polynomial regression and response surface methodology 11 4.1.3 direction of discrepancy between two predictor variables as related to an outcome variable another important consideration is the direction of the discrepancy between the two predictor variables and its effect on the outcome variable. conceiving this could be even more challenging as one should consider the individual relationships between the two predictor variables against the outcome variable in relation to the mismatch and the outcome. the inclination of the surface depends upon the nature of the relationship between individual predictor variable and the outcome variable. as discussed earlier, the response surface corresponding to the intersection between the agreement and disagreement lines provides a reference point to discuss the variations in outcome variable as it relates to the direction of discrepancy between the two predictor variables (moving towards either direction from that intersection). in the example, the aforementioned point shows the level of satisfaction in point which the two lines expectations-experience agreement and the expectation-experience disagreement intersect. moving towards experience dimension (the axis that denotes experience) from that intersection on the line of incongruence, the discrepancy assimilates towards experience (experience > expectations); and when moving towards the expectations dimension (towards other side, towards experience axis in which expectation > experience) from the intersection, the discrepancy assimilates towards expectations. the conceptual challenge here is to predict the outcome variable when moving either side of the intersection that we have mentioned earlier, hence one should carefully consider the one-on-one relationships between expectations and experience (independent variables) and satisfaction (the outcome variable) and the interaction effect between expectations and experience. 4.1.4 synergy between two predictor variables as related to an outcome variable synergy refers to the combined effect of two factors. thus, synergy is discussed here in relation to the combined effect of the two predictor variables on the outcome variable. as such, when theorizing the relationship, one should consider the relationship between the two predictor variables with the outcome variable, when one predictor variable is absent and until that variable reaches its maximum. the line where the value of x axis (or y axis) is at its minimum is the point which the predictor variable that represents x axis (or y axis) is at its minimum. the line where the value of x axis (or y axis) is at its maximum is the point which the predictor variable that represents x axis (or y axis) is at its maximum. the changes of the inclination of the surface corresponding to the line where the value of x axis (or y axis) is minimum and the line where the value of x axis (or y axis) is maximum (and the parallel lines in between) explain the type of synergy (whether that is positive or negative) between the two predictor variables. in the ect example, a synergistic relationship can exist between expectations and experience and their effect on the outcome variable, satisfaction. thus, to explain the relationship, the relationship between expectations and satisfaction when the experience is at its lowest was taken into consideration. next, the relationship between expectations and satisfaction when the experience is at its highest (and the lines parallel and in between the two lines) was taken into account. based on the observations, theoretical arguments of the earlier conceptions of the theory and the assumptions, a positive synergy between the two predictor variables in generating customer satisfaction was identified. australasian journal of information systems sedera & atapattu 2019, vol 23, research note polynomial regression and response surface methodology 12 4.1.5 testing the mediation effect of predictor variable 2 on the relationship between predictor variable 1 and an outcome variable mediation entails the intervening effect of an antecedent variable (predictor variable x) on an outcome variable. herein, the theorizing of the mediation effect could be done for two types of mediations – partial and complete. the mediation effect is partial, when the presence of mediation variable increases the relationship between predictor variable x and the outcome variable z alters partially / gradually. conversely, the mediation effect is complete when the relationship between predictor variable x and outcome variable z changes completely when the mediator is added to relationship. in the case of ect, the focus is on the mediation effect of experience on the relationship between expectations and satisfaction. as expectations increase, it is more likely that it has a negative relationship to satisfaction (since the expectations serves as the base line for evaluating experiences to form satisfaction). however, it is more likely that in the presence of experience, the relationship between expectations and satisfaction alters as experience is mediating the original relationship. it is more likely that experience makes substantial changes to the relationship between expectations and satisfaction. thus, it can be argued that the mediation effect of experience on expectation-satisfaction relationship is complete. 4.2 generating the polynomial outcomes and the response surfaces in a study that the authors conducted, the ect was employed to investigate the tripartite relationship between customer expectations, customer experience and customer satisfaction in the context of customers’ use of smart shopping apps in consumer retail. therein, it was argued that customer satisfaction is defined by expectations and actual experience, and where the level of satisfaction is determined by the degree to which the prior expectations are confirmed by the actual experience (oliver 1977; oliver 1980). therefore, the differences between expectations and experience are considered important for achieving satisfaction. in this example, customer expectations and customer experience are the two predictor variables, and customer satisfaction is the outcome variable. as such, customer expectations variable was identified as x variable, customer experience as the y variable, whilst customer satisfaction is labelled as the outcome variable z. the following polynomial equation was employed to test the tripartite relationship between customer expectations, customer experience and customer satisfaction: customer satisfaction = ƒ (customer expectations*, customer experience**) (1) z=β0 + β1cexpt* + β2 cexpr** + β3 cexpt2 + β4 (cexpt x cexpr) + β5 cexpr2 + e where, *cexpt = customer expectations, and **cexpr = customer experience the procedure outlined in shanock et al. (2010) was followed to run the polynomial regression analysis on the data collected from 428 respondents in a field study and to create the response surface for customer expectations, customer experience and customer satisfaction, as shown in figure 2. table 1 provides the output coefficients of polynomial regression by employing syntax similar to the table 2 in appendix 2 in shanock et al. (2010). table 2 in shanock et al. (2010) provides the four surface properties, a1, a2, a3 and a4, calculated using the microsoft excel worksheet. the test value a1 corresponds to the slope of the surface alone the line of perfect agreement between two predictor variables (pv1 = pv2 or x=y) where it is related to the dependent variable z. where a1= (b1+b2), and b1 and b2 are the non-standardized beta coefficients for the scale centred pv1 and pv2 (x and y), respectively. meanwhile the test value australasian journal of information systems sedera & atapattu 2019, vol 23, research note polynomial regression and response surface methodology 13 a2 corresponds to the curvature along the line of perfect agreement between two predictor variables (pv1 = pv2 or x=y) as related to the dependent variable z. it is given by a2=(b3+b4+b5), where b3, b4 and b5 are the non-standardized beta coefficients for the scale-centred dv1 squared (x2), cross-product (dv1 * dv2 or xy) and dv2 squared (y2), respectively to incorporate nonlinearity. similarly, the test value a3 corresponds to the slope of the line of incongruence between two predictor variables (pv1 = negative pv2 or x= negative y) as related to the dependent variable z. the value a3= (b1-b2), where b1 and b2 are the non-standardized beta coefficients for the scale-centred pv1 and pv2, respectively. this line explains the changes in dv (z) as related to the direction and magnitude of the discrepancy between two predictor variables (iv1 is higher than / lower than iv2). alternatively, the test value a4 corresponds to the curvature along the line of disagreement between two predictor variables (pv1 = negative pv2 or x= negative y) as related to the dependent variable z. it is given by a4= (b3-b4+b5), where b3, b4 and b5 are the non-standardized beta coefficients for the scale-centred dv1 squared (x2), cross-product (dv1 * dv2 or xy) and dv2 squared (y2), respectively. as such it is possible for a1, a2, a3 and a4 to be independently be significant or non-significant as they are related to the two different properties of the two different lines of the resultant response surface. next, the response surface for agreement, discrepancy, direction of discrepancy, synergy and mediation effects was tested, and the results are as follows. variable beta coefficient / non-standardized regression coefficients intercept / constant (β0) 4.693 expt = customer expectations (x) 0.055* expr = customer experience (y) 0.533** expt2 (x2) -0.033 expt*expr (xy) 0.018 expr2 (y2) 0.046 p<0.0001 **p<0.05 table 1. results of polynomial equation involving customer expectations, customer experience and customer satisfaction figure 2. response surface for two predictor variables – customer expectations (x), customer experience (y) and customer satisfaction (z) -4 0 4 1.00 2.00 3.00 4.00 5.00 4.00 2.00 0.00 -2.00 -4.00 p q r s z (customer satisfaction) y (customer experience) x (customer expectations) australasian journal of information systems sedera & atapattu 2019, vol 23, research note polynomial regression and response surface methodology 14 effect coefficient standard error test stat (t) p value significance a1: slope along x=y (as related to z) 0.59 0.11 5.507 0.000 significant a2: curvature on x=y (as related to z) 0.03 0.03 1.186 0.236 non-significant a3: slope along x=-y (as related to z) -0.48 0.03 -14.286 0.000 significant a4: curvature on x=-y (as related to z) 0.00 0.04 -0.119 0.905 non-significant table 2. properties of the response surface between customer expectations, customer experience and customer satisfaction 4.3 interpreting the response surfaces for five different research questions 4.3.1 agreement between customer expectations and customer experience as related to customer satisfaction the solid line on the floor of the graph represents the line p to r on the three-dimensional surface of figure 2, where it depicts the perfect agreement between the two predictor variables, customer expectations and customer experience (i.e. x=y). as suggested in ect, the customer expectations need to be matched to satisfy the customers. thus, the alignment between customer expectations and their actual experience is positively related to customer satisfaction where the line of alignment (x=y) has a positive slope through the line from r to p. hence, the agreement between the customers’ expectations arising from their use of smart shopping apps and the customers’ ultimate shopping experience matters to the customers’ final satisfaction. the level of customer satisfaction is lowest at the front corner of the graph along the line of agreement where customers’ expectations as well as customer experience both stay low. respectively, the satisfaction becomes increasingly higher towards the back of the graph as customer expectation and experience both reach higher levels. as seen in table 2, only the slope along the p-r line is significant (a1), but the curvature along the p-r line remains nonsignificant. 4.3.2 discrepancy between customer expectations and customer experience as related to customer satisfaction the line perpendicular to the solid line of agreement (x = y) on the floor of the graph depicts the line of disagreement. in contrast, the dashed disagreement line on the floor of the graph in figure 2 depicts the line of incongruence (the x and y variables are not in agreement, i.e. x= negative y) and it represents the surface along the line q to s. moving away from the interception of the two lines to either direction (the left or right) shows the degree of discrepancy (mismatch) between customer expectations and customer experience and how they relate to customer satisfaction. as conceptualized in ect, customer expectations need to be matched for customer satisfaction; thus, the mismatch between customers’ expectations and customers’ actual experience could create customer dissatisfaction. the graph in figure 2 shows that customers become satisfied when the difference between expectations and actual experience is at its lowest or when the actual experience exceeds their initial expectations. as presented in table 2, the slope along the line of incongruence (q-s) is significant (a1), whilst the curvature along the q-s line remains non-significant. australasian journal of information systems sedera & atapattu 2019, vol 23, research note polynomial regression and response surface methodology 15 4.3.3 direction of discrepancy between customer expectations and customer experience as related to customer satisfaction as discussed earlier, the slope along the line of incongruence explains the extent to which the direction of discrepancy between the two predictor variables matters to the outcome variable. whilst the point on the response surface corresponding to the intersection between the solid p-r and dotted q-s lines on the floor of figure 2 depicts the point of perfect agreement on the q-s line, moving towards either direction from that intersection explains the incongruence. moving towards point q from the intersection, the discrepancy assimilates towards customer experience; moving towards point s from the intersection, the discrepancy assimilates towards customer expectations. whilst the surface property a3 relates to the slope of the line of incongruence as it relates to the outcome variable, a significant value for a3 shows that the direction of discrepancy indeed matters to the changes in the outcome variable, and the test of the slope in figure 3 indeed provides a significant value for a3 as seen in table 2. a significant negative a3 therein indicates that the customer satisfaction outcome variable increases when the discrepancy between customer expectations (x) and customer experience (y) assimilates towards customer experience. 4.3.4 synergy between customer expectations and customer experience as related to customer satisfaction synergy refers to the combined effect of two factors; thus, synergy is discussed here in relation to the combined effect of customer expectations and customer experience on customer satisfaction. the p-q and r-s lines of the graph in figure 2 show a positive synergistic relationship between customer expectations and customer experience. the r-s line shows the relationship between customer expectations and customer satisfaction when customer experience is absent or at its lowest. the p-q line represents the relationship between customer expectations and customer satisfaction when customer satisfaction reaches its maximum. the lines running parallel to the p-q and r-s lines on the surface show that customer satisfaction increases with the increase of customer expectations (moving from points r to q and s to p). whilst the r-s line shows a decrease in customer satisfaction as it reaches higher customer expectations, the decrease of customer satisfaction corresponds to higher levels of customer expectations diminishes in the lines that are parallel to r-s but are run closer to the p-q line (lines that correspond to higher levels of customer experience). in addition, the slope of the curves that are parallel to the r-s line changes as customer experience increases, the slope of the lines diminishes, and customer satisfaction remains high as customer experience reaches its maximum levels. this shows that, as customer experience increases, it has a positive influence on the relationship between customer expectations and customer satisfaction, suggesting a positive synergy between the two predictor variables. 4.3.5 testing the mediation effect of customer experience on the relationship between customer expectations and customer satisfaction as mediation entails the intervening effect of an antecedent variable on a dependent variable, herein the focus is on the mediation of customer experience on the relationship between customer expectations and customer satisfaction. next, reference is made to the graph in figure 2 to test the mediation effect discussed above. the p-q and r-s lines (or curves) of the graph therein explain the relationship between customer expectations and customer satisfaction. the r-s line explains the relationship between customer expectations and customer satisfaction when customer experience is absent or at its lowest. the p-q line explains the relationship between customer expectations and customer satisfaction when australasian journal of information systems sedera & atapattu 2019, vol 23, research note polynomial regression and response surface methodology 16 customer experience reaches its highest. meanwhile, the r-s line shows that when customer expectations reaches its highest, customer satisfaction decreases consequently, but the p-q line shows no change to the outcome variable as the customer expectations increases from point q to p. further, a comparison of the r-s and p-q lines shows that, as customer experience increases, the original behaviour of the relationship between customer expectations and customer satisfaction is disturbed completely. this explains that customer experience completely mediates the relationship between customer expectations and customer satisfaction, by impacting the relationship completely. 5 conclusion this paper outlined the basic assumptions under which the polynomial regression can be performed and the response surface methodology can be used in an analysis. it also outlined the theorizing of procedures for developing hypotheses on different research questions and provided detailed step-by-step guidelines on conducting the analysis and creating response surfaces. next, the paper elaborated different research questions that can be investigated using this technique with the help of a hypothetical response surface for better comprehension of the technique and the interpretation of its results. further, the paper provided a field-based example of the application of the methodology and the interpretation of findings to better comprehend the technique. in the process, several advantages of the polynomial regression and response surface methodology were identified. the polynomial regression and response surface methodology is a solution for the problems associated with the use of absolute, algebraic or squared differences between two scores (difference scores) in analysing discrepancies (brown et al. 2008; edwards 1994; edwards 2002; shanock et al. 2010). the polynomial regression has a clear advantage over the traditional use of differential scores in analysing incongruence in rating scales as it addresses the confounding effect associated with the traditional differential scores (brown et al. 2008). for example, traditional difference scores confound the effects of two variables on the outcome by combining distinct measures into a single score. as such, the use of difference scores wouldn’t allow a researcher to investigate the extent to which the measure of each component contributes to the outcome variable. nor does it inform the researcher about which variable of the two is better for the outcome variable or vice versa. by using two component measures, polynomial regression can investigate the independent effects of the two variables on the outcome variable. thus, the method keeps the independent effect of each component’s measure intact. this enables a researcher to investigate the individual effects of each component measure on the outcome variable, the effects of the two component measures in combination on the outcome variable, and the effects of two component measures in different combinations/proportions on the outcome variable. the second issue related to using difference scores is that it reduces the model dimensions to two from three-dimensions. thus, the congruence between the two compound variables is denoted by a ‘point’ instead of showing it as a line (line of agreement (x=y), a-b line in figure 2) which hides information that is valuable for clear interpretation and limits the explanatory potential. further, the traditional regression analysis only provides a two-way interaction in a two-dimensional view compared to the three-dimensional view provided by the response surface methodology discussed herein. the polynomial regression together with response surface methodology facilitate a three-dimensional examination of the tripartite relationship between two predictor variables and an outcome variable, and thus improves the explanatory potential. the traditional model testing often carried out by is researchers only provides a australasian journal of information systems sedera & atapattu 2019, vol 23, research note polynomial regression and response surface methodology 17 snapshot view of the data (e.g., the structural model tests in pls and lisrel) and hence only provides information relating to the snapshot view and limits the nuanced views that the data could generate. for example, such structural models only provide a generalised relationship between variables taking the dataset but cannot provide information on how such relationships behave for the respondents who are at different levels of the measurement scale (unless the analysis is repeated using polarised samples). however, the three-dimensional response surface created in the polynomial regression and response surface methodology has the potential to provide nuanced views of the data in several different combinations. for example, in the response surface presented in figure 2, moving along the line of agreement, a researcher can interpret how the outcome variable behaves when the value of perfect agreement is low, moderate or high (regarding any point that the researcher wishes to investigate). in addition, with this technique a researcher can investigate how the outcome variable behaves when the discrepancy between two variables is inclined towards either variable (for combinations with different proportions – along the x= negative y line). further, the traditional methods assume linearities between the variables even when the underlying theories suggest non-linearities. this could possibly mask the true explanatory potential of the model and may lead to misleading interpretations. however, by relaxing the linearity assumption, the polynomial regression with response surface methodology allows a researcher to investigate possible non-linearities between the variables by staying true to the original theoretical assumptions. in addition, by employing the response surface technique, a researcher could test for the significance of such non-linearities with response surface statistics a2 and a4. further, the three-dimensional visualisations of the interactions among the three variables could provide nuanced views on the relationships when applied for comparison purposes (for two different samples, different time frames, different types of respondents etc.). this paper extends the works of shanock et al. (2010) by elaborating on the structured guidelines to perform polynomial regression and response surface methodology to investigate alignment, discrepancy and the direction of discrepancy between two predictor variables and an outcome variable. moreover, advantages of employing such an approach for making deeper and insightful research are highlighted by demonstrating possible research questions that could have been discussed in a sample of past is studies. the method has a major issue that researchers must be aware of. the method is susceptible to the presence of outliers, where one or two outliers in the data can seriously affect the results of the nonlinear analysis. as such, prior to analysis, the researcher must engage in removing the outliers. however, there are only fewer model validation tools for the detection of outliers in nonlinear regression than there are for linear regression. references ajzen, i. 1991. "the theory of planned behaviour," organizational behavior and human decision processes (50:2), pp 179-211. ajzen, i., and fishbein, m. 1980. understanding attitudes and predicting social behaviour. englewood cliffs, nj: prentice hall. armstrong, j.s., and overton, t.s. 1977. "estimating nonresponse bias in mail surveys," journal of marketing research (14:3), pp 396-402. australasian journal of information systems sedera & atapattu 2019, vol 23, research note polynomial regression and response surface methodology 18 balaji, s., and brown, c.v. 2014. "lateral coordination mechanisms and the moderating role of arrangement characteristics in information systems development outsourcing," information systems research (25:4), pp 747-760. box, g.e.p., and draper, n.r. 1987. empirical model-building and response surfaces. new york: wiley. brown, s.a., venkatesh, v., and goyal, s. 2012. "expectation confirmation in technology use," information systems research (23:2), pp 474-487. brown, s.a., venkatesh, v., and goyal, s. 2014. "expectation confirmation in information systems research: a test of six competing models," mis quarterly (38:3), pp 729-756. brown, s.a., venkatesh, v., kuruzovich, j., and massey, a.p. 2008. "expectation confirmation: an examination of three competing models," organizational behavior and human decision processes (105), pp 52-66. edwards, j.r. 1994. "the study of congruance in organizational behavior research: critique and a proposed alternative," organizational behavior and human decision processes (58:1), pp 51-100. edwards, j.r. 2001. "alternatives to difference scores: polynomial regression analysis and response surface methodology," in: measuring and analyzing behavior in organizations: advances in measurement and data analysis, f. drasgow and n. schmidt (eds.). san francisco, ca: jossey-bass?pfeiffer, pp. 350-400. edwards, j.r. 2002. "alternatives to difference scores: polynomial regeression analysis and response surface methodology," in: advances in measurement and data analysis, f. drasgow and n.w. schmidt (eds.). san francisco: jossey-bass, pp. 350-400. edwards, j.r., and cooper, c.l. 1990. "the person-environment fit approach to stress: recurring problems and some suggested solutions," journal of organizational behavior (11:4), pp 293-307. edwards, j.r., and lambert, l.s. 2007. "methods for integrating moderation and mediation: a general analytical framework using moderated path analysis," psychological methods (12:1), pp 1-22. edwards, j.r., and parry, m.e. 1993. "on the use of polynomial regression equations as an alternative to difference scores in organizational research," academy of management journal (36:6), pp 1577-1613. edwards, j.r., and van harrison, r. 1993. "job demands and worker health: threedimensional reexamination of the relationship between person-environment fit and strain," journal of applied psychology (78:4), pp 628-648. fan, l., and suh, y.-h. 2014. "why do users switch to a disruptive technology? an empirical study based on expectation-disconfirmation theory," information & management (51:2), pp 240-248. fang, y., qureshi, i., sun, h., mccole, p., ramsey, e., and lim, k.h. 2014. "trust, satisfaction, and online repurchase intention: the moderating role of perceived effectiveness of ecommerce institutional mechanisms," mis quarterly (38:2), pp 407-427. australasian journal of information systems sedera & atapattu 2019, vol 23, research note polynomial regression and response surface methodology 19 fredricks, a.j., and dossett, d.l. 1983. "attitude–behavior relations: a comparison of the fishbein-ajzen and the bentler-speckart models," journal of personality and social psychology (45:3), p 501. harris, m.m., ansaal, f., and lievens, f. 2008. "keeping up with the joneses: a field study of the relationships among upward, lateral, and downward comparisons and pay level satisfaction," journal of applied psychology (93), pp 665-673. hecht, t.d., and allen, n.j. 2005. "exploring links between polychronicity and well-being from the perspective of person–job fit: does it matter if you prefer to do only one thing at a time?," organizational behavior and human decision processes (98:2), pp 155178. hong, y., and pavlou, p.a. 2014. "product fit uncertainty in online markets: nature, effects, and antecedents," information systems research (25:2), pp 328-344. johnston, a.c., warkentin, m., and siponen, m. 2015. "an enhanced fear appeal rhetorical framework: leveraging threats to the human asset through sanctioning rhetoric," mis quarterly (39:1), pp 113-134. kahneman, d., and tversky, a. 1979. "prospect theory: an analysis of decision under risk," econometrica: journal of the econometric society), pp 263-291. khuri, a.i., and cornell, j.a. 1987. response surfaces: designs and analyses. new york: marcel dekker. kim, k., mithas, s., whitaker, j., and roy, p.k. 2014. "research note—industry-specific human capital and wages: evidence from the business process outsourcing industry," information systems research (25:3), pp 618-638. kim, s.k., and hsieh, p.-h. 2003. "interdependence and its consequences in distributorsupplier relationships: a distributor perspective through response surface approach," journal of marketing research (40:1), pp 101-112. kristof, a.l. 1996. "person-organization fit: an integrative review of its conceptualizations, measurement, and implications," personnel psychology (49:1), pp 1-49. lankton, n., mcknight, d.h., and thatcher, j.b. 2014. "incorporating trust-in-technology into expectation disconfirmation theory," the journal of strategic information systems (23:2), pp 128-145. lankton, n.k., and mcknight, h.d. 2012. "examining two expectation disconfirmation theory models: assimilation and asymmetry effects," journal of the association for information systems (13:2), pp 88-115. li, t., kauffman, r.j., van heck, e., vervest, p., and dellaert, b.g. 2014. "consumer informedness and firm information strategy," information systems research (25:2), pp 345-363. lim, s., saldanha, t.j., malladi, s., and melville, n.p. 2013. "theories used in information systems research: insights from complex network analysis," journal of information technology theory and application (jitta) (14:2), p 2. australasian journal of information systems sedera & atapattu 2019, vol 23, research note polynomial regression and response surface methodology 20 ma, x., kim, s.h., and kim, s.s. 2014. "online gambling behavior: the impacts of cumulative outcomes, recent outcomes, and prior use," information systems research (25:3), pp 511527. maruping, l.m., and magni, m. 2015. "motivating employees to explore collaboration technology in team contexts," mis quarterly (39:1), pp 1-16. mccullagh, p., nelder, j.a., and mccullagh, p. 1989. generalized linear models. chapman and hall london. mckinney, v., yoon, k., and zahedi, f.m. 2002. "the measurement of web-customer satisfaction: an expectation and disconfirmation approach," information systems research (13:3), september, pp 296-315. meservy, t.o., jensen, m.l., and fadel, k.j. 2013. "evaluation of competing candidate solutions in electronic networks of practice," information systems research (25:1), pp 1534. o'leary, m.b., wilson, j.m., and metiu, a. 2014. "beyond being there: the symbolic role of communication and identification in perceptions of proximity to geographically dispersed colleagues," mis quarterly (38:4), pp 1219-1243. oliver, r.l. 1977. "effect of expectation and disconfirmation on postexposure product evaluations: an alternative interpretation," journal of applied psychology (62:4), pp 480486. oliver, r.l. 1980. "a cognitive model of the antecedents and consequences of satisfaction decisions," jmr, journal of marketing research (17:4), pp 460-460. oliver, r.l., balakrishnan, p.v.s., and barry, b. 1994. "outcome satisfaction in negotiation: a test of expectancy disconfirmation," organizational behavior and human decision processes (60:2), pp 252-275. ping, r.a. 2002. "interpreting latent variable interactions," american marketing association (winter) educators' conference proceedings, austin, tx, pp. 213-220. roberts, n., and grover, v. 2012a. "investigating firm's customer agility and firm performance: the importance of aligning sense and respond capabilities," journal of business research (65:5), pp 579-585. roberts, n., and grover, v. 2012b. "leveraging information technology infrastructure to facilitate a firm's customer agility and competitive activity: an empirical investigation," journal of management information systems (28:4), pp 231-270. scherer, a., wünderlich, n.v., and von wangenheim, f. 2015. "the value of self-service: long-term effects of technology-based self-service usage on customer retention," mis quarterly (39:1), pp 177-200. shanock, l., baran, b., gentry, w., pattison, s., and heggestad, e. 2010. "polynomial regression with response surface analysis: a powerful approach for examining moderation and overcoming limitations of difference scores," journal of business and psychology (25:4), pp 543-554. shaw, j.d., and gupta, n. 2004. "job complexity, performance, and well-being: when does supplies-values fit matter?" personnel psychology (57:4), pp 847-879. australasian journal of information systems sedera & atapattu 2019, vol 23, research note polynomial regression and response surface methodology 21 steelman, z.r., hammer, b.i., and limayem, m. 2014. "data collection in the digital age: innovative alternatives to student samples," mis quarterly (38:2), pp 355-378. sykes, t.a., venkatesh, v., and johnson, j.l. 2014. "enterprise system implementation and employee job performance: understanding the role of advice networks," mis quarterly (38:1), pp 51-72. tabachnick, b.g., and fidell, l.s. 2007. using multivariate statistics. boston: allyn and bacon. titah, r., and barki, h. 2009. "nonlinearities between attitude and subjective norms in information technology acceptance: a negative synergy?," mis quarterly (33:4), pp 827-844. tversky, a., and kahneman, d. 1973. "availability: a heuristic for judging frequency and probability," cognitive psychology (5:2), pp 207-232. venkatesh, v., and goyal, s. 2010. "expectation disconfirmation and technology adoption: polynomial modeling and response surface analysis," mis quarterly (34:2), pp 281-303. xu, j.d., benbasat, i., and cenfetelli, r.t. 2014. "the nature and consequences of trade-off transparency in the context of recommendation agents1," mis quarterly (38:2), pp 379406. australasian journal of information systems sedera & atapattu 2019, vol 23, research note polynomial regression and response surface methodology 22 appendix a: possible applications of polynomial regression and response surface methodology on the papers published in misq and isr study 1: (ma et al. 2014) referent theory: gambling/prospect theory (kahneman and tversky 1979), availability heuristics (tversky and kahneman 1973) and repeated behaviour (fredricks and dossett 1983). study model possible applications of polynomial regression and response surface methodology the three theoretical propositions used for the conceptual model development suggest non-linearities. the response surfaces could complement the discussion by providing insights into how different combinations of predictor variables (e.g., two types of gains and losses) influence the online gambling behaviour. also, with two response surfaces for current online gambling, regular use (or extended use) and subsequent online gambling could provide deeper insights into the nature of interference that the two types of use has on the relationship between current online gambling and subsequent online gambling behaviour. polynomial regression and response surface methodology allows the investigation of how two sets of predictor variables, (i) cumulative gain and cumulative loss, and (ii) immediate gain and immediate loss individually as well as in combination influence the dependent variable subsequent online gambling behaviour. the method also facilitates the understanding of how different combinations of predictor variables influence the outcome variable. cumulative gain (cg) cumulative loss (cl) immediate gain (ig) immediate loss (il) current online gambling regular use extended use subsequent online gambling (sog) individual behaviour outcomeantecedents gambling theory availability heuristic repeated behaviour h1 h2 h3 h4 h5 h6 h7 cg cl sog ig il sog australasian journal of information systems sedera & atapattu 2019, vol 23, research note polynomial regression and response surface methodology 23 study 2: (kim et al. 2014) study model no illustration of the study model is presented in the paper possible applications of polynomial regression and response surface methodology growth rate of the wages (grw) = ƒ (industry experience (ie), firm experience (fe), total experience (te)) theoretical model suggests that the growth rate of the wages is a function of industry experience, firm specific experience and total experience. also, it depicts the rate of change is not constant as the experience increases, thus suggest non-linearity. polynomial regression can create three different response surfaces for different combinations of three types of experiences and growth rate of wages, thus could provide insights into what combinations of experience would yield better (worse) growth rates of wages. three different response surfaces for different combinations of three types of experiences and growth rate of wages can complement the discussion by what combinations of experience would yield better (worse) growth rates of wages. ie te grw ie fe grw fe te grw australasian journal of information systems sedera & atapattu 2019, vol 23, research note polynomial regression and response surface methodology 24 study 3: (hong and pavlou 2014) study model possible applications of polynomial regression and response surface methodology the theoretical propositions used in deriving the model conceive non-linear relationships. the study model suggests that the product returns is a function of product fit uncertainty and product quality uncertainty. the application of a response surface to test the interaction effects between product fit uncertainty (pfu) and product quality uncertainty (pqu) on product returns (pr) would have provided insights into how different combinations of pfu and pqu influence the pr and optimum combination of pfu and pqu that ensure the least amount of pr. also, the response surface methodology has the potential to provide information relating to what degree of pfu and pqu is most critical for pr. a response surface for product fit uncertainty, product quality uncertainty, and product returns would complement the discussion by providing insights into how different combinations influence the product returns and optimum combination. this ensures the least amount of product returns. pfu pqu pr australasian journal of information systems sedera & atapattu 2019, vol 23, research note polynomial regression and response surface methodology 25 study 4: (li et al. 2014) study model: the model is developed based on discrete choice model. however, no illustration of the research model was presented in the paper. the theoretical propositions used to theorize the consumer informedness the price and the product, and the two consumer behaviours – the trading out and trading down, do not conceive linear relationships between constructs. response surfaces for two types of informedness, on types of consumer behaviours for two segments of products would have complemented the discussion by providing visual representations thus making the comparisons easier to comprehend. in addition, the response surface methodology has the potential to provide insights into how different combinations of two predictor variables influence the two forms of consumer behaviours. possible applications of polynomial regression and response surface methodology: customers sensitivity (cs) = ƒ (price informedness (pri-i), product informedness (pro-i)) a response surface would complement the discussion by providing different combinations of product and price informedness that influence customer sensitivity. australasian journal of information systems sedera & atapattu 2019, vol 23, research note polynomial regression and response surface methodology 26 study 5: (meservy et al. 2013) study model: possible applications of polynomial regression and response surface methodology: the study evaluates how two types of knowledge evaluations – the central and peripheral influences the knowledge filtering decisions. the study utilizes a repeated generalized linear model that uses quasi likelihood function to model predictor variables effects (mccullagh et al. 1989). as the theoretical grounding hints non-linearities, by applying polynomial regression together with response surface methodology, the analysis would have unearthed deep insights into how the interactions between different processing cues would differently influence the knowledge filtering decisions. such analysis also could provide insights into how elaboration mediates/moderates the relationships between each type of processing cues and the knowledge filtering decisions. with six response surfaces comprising of five key constructs in the model could have aided a rich discussion on how different magnitudes of each element influence consumers choice set. content quality (cq) elaboration (e) retention in choice set (rics) central processing cues (cpc) source expertise (se) validation (v) peripheral processing cues (ppc) cq ppc rics cq e rics ppc e rics se v rics se e rics v e rics australasian journal of information systems sedera & atapattu 2019, vol 23, research note polynomial regression and response surface methodology 27 study 6: (maruping and magni 2015) study model: possible applications of polynomial regression and response surface methodology the theories used to develop the conceptual model involving two types of individual cognitions: intention and expectations, exploration all suggest nonlinear relationships between the constructs. also, the model conceived as exploration is a function of both intention and expectations. as such the model tested for both linear and non-linear mixed-effects yet polynomial regression and response surface methodology were overlooked. however, the inclusion of polynomial regression together with response surface methodology would have enriched the discussion by providing insights into (i) how the interaction effect of intention and expectations influences exploration, and (ii) how different combinations of intention and expectations differently influences the exploration behaviour. four different response surfaces for four tripartite relationships above between four key constructs in the research model can complement the overall discussion with providing insights into how varying levels of team empowerment influence two kinds of human cognitions thus the varying degree of two forms of cognitions mediates ct exploration behaviour. team empowerment (te) team environment team-level individual-level intention to continue exploring (ice) individual cognitions expectation to continue exploring (ece) ct exploration (cte) te ic cte ice ece cte te ice cte te ece cte australasian journal of information systems sedera & atapattu 2019, vol 23, research note polynomial regression and response surface methodology 28 study 7: (johnston et al. 2015) study model: possible applications of polynomial regression and response surface methodology: whilst the conceptual model predicts the behaviour of compliance intention, the theories used in development of conceptual model conceive non-linear relationships between the constructs. the behaviour of fear appeal is conceived as a function of the two types of fear appeal elements – conventional and enhanced, whilst the two types of fear appeal elements are consisting of number of different sub-components. thus, derivation of three-dimensional response surfaces for many different tripartite relationships between different components of the study model such as types of sanctions, severity, certainty and celerity of them, perceived threat severity and susceptibility, perceived selfefficacy and response efficacy are possible. such an exercise has a greater potential to unearth complex relationships, nuanced views of such relation4ships and deep insights into how different combinations of different elements and different magnitudes of two types of fear appeal elements in combination influences an individual’s compliance intention. twelve tripartite relationships between the different types of constructs in the study model may have complemented the discussion allowing the researchers to discuss how different magnitudes of two types of fear appeal elements in combination define an individual’s compliance intention. perceived threat severity (pts) perceived threat susceptibility (pts) conventional fear appeal elements (cfae) perceived self efficacy (pse) perceived response efficacy (pre) formal sanction certainty (fsc) informal sanction certainty (isc) formal sanction severity (fss) informal sanction severity (iss) compliance intention (ci) sanction celerity (sc) enhanced fear appeal elements (eae) pts pse ci pts ptsu ci pts pre ci ptsu pse ci cfae eae ci isc sc ci ss sc ci sc iss ci fsc isc ci fsc iss ci fsc fss ci isc iss ci australasian journal of information systems sedera & atapattu 2019, vol 23, research note polynomial regression and response surface methodology 29 study 8: (scherer et al. 2015) study model: possible applications of polynomial regression and response surface methodology: the objective of the study is to investigate how the ratio of self-service versus personal service use influence customer defection/retention over time. whilst the theories underlying the theoretical model supports non-linear relationships between the constructs, a response surface created through a polynomial regression would have complemented the discussion by providing insights into how different combinations of self service and personal service together influence customer defection/retention. a response surface created through a polynomial regression would have provided insights into how varying degree of value proposition and value-in-context in combination assure customer retention. in addition, the response surface has the potential to explain the nature of mediation that value-in-context play on the relationship between value proposition and customer retention. service channel (value proposition) (vp) customer characteristics (e.g. experience with the provider) channel expansion theory value-in-context (integration of market, public and private resources) (vic) customer retention (cr) task characteristics (e.g. complexity) media richness theory service-dominant logic of marketing note focus of empirical investigation theoretical underpinnings vp vic cr australasian journal of information systems sedera & atapattu 2019, vol 23, research note polynomial regression and response surface methodology 30 study 9: (o'leary et al. 2014) study model: possible applications of polynomial regression and response surface methodology: the paper investigates how perceived proximity-cognitive and affective sense of relational closeness and objective distancegeographical closeness, relate to relationship quality for collocated and geographically dispersed work colleagues. whilst the underlying theoretical propositions suggest non-linearities a response surface methodology has the potential to complement the analysis by providing further insights into how different combinations of perceived and objective proximities together would differently influences the quality of the overall relationship. further, such an analysis could provide deep insights in to how perceived proximity is mediating the relationship between objective distance and relationship quality (or contrariwise). also, establishing two different response surfaces for the tripartite relationships between (i) objective distance, shared identification, and perceived proximity, and (ii) objective distance, communication and perceived proximity have the ability to provide insights into how different degrees of communication and shared identification mediate the relationship between objective distance and perceived proximity thus its overall influence the relationship quality. two different response surfaces for the tripartite relationships between shared identification, communication, perceived proximity and relationship quality can complement the analysis by providing how different combinations of communication and shared identification influence perceived proximity and relationship quality. similarly, a response surface for objective distance, perceived proximity and relationship can explain how perceived proximity is mediating the relationship between objective distance and relationship quality. objective distance (od) perceived proximity (pp) relationship quality (rq) communication (c) shared identification (si) c si pp c si rqod pp rq australasian journal of information systems sedera & atapattu 2019, vol 23, research note polynomial regression and response surface methodology 31 study 10: (steelman et al. 2014) study model: possible applications of polynomial regression and response surface methodology: the aim of this paper is to examine the consistency of survey responses across different samples obtained from different cohorts of respondents across two different studies involving original tam model and edt. whilst both tam and edt in their original conceptions predict non-linear relationships between the constructs both models conceive that the respective dependent variables behavioural intention and satisfaction. use of response surfaces between the three key variables in tam and ect across five different samples in the study would have complemented the structural model comparisons using pls. this can be done by visually exhibiting the similarities and/or contrasting the differences much effectively across the samples used in the analysis. such response surfaces have the potential to provide further insights into how different proportions of two predictor variables are impacting the dependent variable similarly of differently across different cohorts of samples. same model can be tested across student sample, worldwide ocm sample, us ocm sample, non-us ocm sample and the consumer sample. response surfaces between the three key variables of the research model across five different samples would have complemented the structural model results using pls, as response surface methodology has the potential to visually exhibit the similarities and/or contrast differences better across the samples used in the analysis. perceived usefulness (pu) perceived ease of use (peou) behavioural intention (bi) pu peou bi australasian journal of information systems sedera & atapattu 2019, vol 23, research note polynomial regression and response surface methodology 32 study 11: (xu et al. 2014) study model: possible applications of polynomial regression and response surface methodology: the study investigates the complex relationships and interactions between trade-off transparency, perceived enjoyment and perceived product diagnosticity based on the theoretical notion of s-o-s in environmental psychology. response surfaces for different combinations of stimulus-trade-off transparency, two types of organismsperceived enjoyment and perceived product diagnosticity, and two types of responses-perceived decision quality and perceived decision effort have the ability to complement the discussion by providing in-depth visual representations of many different tripartite relationships between the constructs. such response surfaces have the potential to provide rich insights into how different ratios of the two predictor variables and the interaction effect of the two predictor variables impact the dependent variable rather than fixing the analysis to a twodimensional structural model. such discussions would have provided nuanced views on such tri-partite relationships that go beyond path coefficients, path significance and the explanatory power of the model. response surfaces would have complemented the data analysis by providing in depth visual representations of ten different tri-partite relationships between the constructs of the research model rather than restricting the analysis to a twodimensional structural model. such understanding would have provided nuances views on such tri partite relationships that goes beyond path coefficients, path significance and the explanatory power of the model. trade-off transparency (tot) perceived enjoyment (pe) perceive product diagnosticity (ppd) perceived decision quality (pdq) perceived decision effort (pde) intention to reuse ra (iru) tot pe pdq tot ppd pde tot pe pde tot ppd pdq pdq pde iru pe ppd iru pe pdq iru ppd pde iru pe pde iru ppd pdq iru australasian journal of information systems sedera & atapattu 2019, vol 23, research note polynomial regression and response surface methodology 33 study 12: (fang et al. 2014) study model: possible applications of polynomial regression and response surface methodology: based on the theory of organizational trust this study investigates how the perceived effectiveness of e-commerce institutional mechanisms moderate the trust, satisfaction and online purchase intention relationship. response surfaces among four key variables involving perceived effectiveness of e-commerce institutional mechanisms, trust, satisfaction and online purchase intention have the potential to provide further insights. for example, it can provide insights into how the magnitude of trust in vendor and perceived effectiveness of e-commerce institutional mechanisms mediate and moderate the relationship between satisfaction with vendor and repurchasing intentions that go beyond the traditional structural model testing. response surfaces of three tripartite relationships between four key variables of the research model would enable how the magnitude of trust in vendor and perceived effectiveness of e-commerce institutional mechanisms mediate and moderate the relationship between satisfaction with vendor and repurchasing intentions. satisfaction with vendor (sv) trust in vendor (tv) repurchasing intention (ri) perceived effectiveness of ecommerce institutional mechanisms (peecim) website quality (wq) reputation (r) familiarity with vendor (fv) satisfaction (internet) gender income education income education expertise product characteristics peecim tv ri sv peecim tvsv pp ri australasian journal of information systems sedera & atapattu 2019, vol 23, research note polynomial regression and response surface methodology 34 study 13: (sykes et al. 2014) study model: possible applications of polynomial regression and response surface methodology: key objectives of the study are to understand the interaction effects between two types of advices get-advice and give-advice on software and workflows on post implementation job performance. thus, the data was collected in four types of socio matrices: workflow get-advice, workflow give-advice, and software get-advice and software give-advice. the data was analysed using hierarchical regression. whilst the hierarchical regression can test the interaction effects between the different types of socio matrices on post implementation job performance it would provide detailed insights into how the combinations of different intensities of socio matrices differently influences the post implementation job performance. however, response surface methodology has the potential to add an additional perspective of how varying degrees of four different socio matrices influences the post implementation job performance. importantly the response surface methodology can provide insights into the best combination/s of the two predictor variables that could produce optimal post-implementation job performance. response surface methodology would add an additional perspective of how varying degrees of get-advice and give-advice influences the post implementation job performance. workflow software workflow software get-advice (gt-a) give-advice (gv-a) postimplementation job performance (pijp) control variables: pre-implementation job performance, advice networks, conscientiousness, demographics gt-a-s gv-a-s pijp gt-a wf gv-a wf pijpgt-a gv-a pijp australasian journal of information systems sedera & atapattu 2019, vol 23, research note polynomial regression and response surface methodology 35 study 14: (balaji and brown 2014) study model: taking the organization design theory, the paper investigates how two types of lateral coordination mechanisms – formal structural and non-structural informal mechanisms, relates to strategic it benefits to the client firm in is development outsourcing context. through the creation of a response surface for the tri-partite relationship between two forms of lateral coordination mechanisms and strategic it benefits, the discussion could be extended to discuss how different intensities of the two types of lateral coordination mechanisms work together to deliver different types and levels of strategic it benefits to the client organizations. further the response surface has the potential to inform the optimum combination of the two coordination mechanisms for different levels of resource provisioning and different value similarity groups. run the model for different samples of: (1) low resource provisioning, (2) high resource provisioning, (3) low value similarity, and (4) high value similarity. response surfaces created for the tri-partite relationship between the three main constructs of the paper across polarised samples can provide insights into how different degrees of the two types of governance mechanisms work together to deliver the strategic it benefits. further the response surface has the potential to inform the optimum combination of the two governance mechanisms that are useful in four distinct resource and value similarity groups. copyright: © 2019 sedera & atapattu. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. informal governance structural governance strategic it benefits http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 background 3 overview of polynomial regression and response surface methodology 3.1 assumptions of polynomial regression 3.2 interpretation of the response surface 3.2.1 agreement between two predictor variables as related to an outcome variable 3.2.2 disagreement between two predictor variables as related to an outcome variable 3.2.3 direction and magnitude of the discrepancy between two predictor variables as related to an outcome variable 3.2.4 synergy between two predictor variables as related to an outcome variable 3.2.5 test of mediation 4 applying polynomial regression to a study of ect 4.1 theorizing for polynomial regression and response surface methodology 4.1.1 agreement between two predictor variables as related to an outcome variable 4.1.2 disagreement between two predictor variables as related to an outcome variable 4.1.3 direction of discrepancy between two predictor variables as related to an outcome variable 4.1.4 synergy between two predictor variables as related to an outcome variable 4.1.5 testing the mediation effect of predictor variable 2 on the relationship between predictor variable 1 and an outcome variable 4.2 generating the polynomial outcomes and the response surfaces 4.3 interpreting the response surfaces for five different research questions 4.3.1 agreement between customer expectations and customer experience as related to customer satisfaction 4.3.2 discrepancy between customer expectations and customer experience as related to customer satisfaction 4.3.3 direction of discrepancy between customer expectations and customer experience as related to customer satisfaction 4.3.4 synergy between customer expectations and customer experience as related to customer satisfaction 4.3.5 testing the mediation effect of customer experience on the relationship between customer expectations and customer satisfaction 5 conclusion the australasian journal of information systems (ajis) began in 1993 and was based on the simple realisation that there was a need for an academic ‘outlet’ for is research in australia australasian journal of information systems volume 18 number1 2013 5 factors affecting consumers’ continued use of internet banking: empirical evidence from australia sujana adapa university of new england adapa2@une.edu.au ray cooksey university of new england rcooksey@une.edu.au abstract the goal of this study was to investigate the factors that influence consumer’s continued use of internet banking in australia. patterns of continued use of internet banking have been neglected as most of the existing studies focus on either consumer adoption or acceptance of internet banking. however, in comparison to new customer acquisition, measures of continued use of internet banking are important as it is related to a cost-effective marketing strategy aimed at retaining customers. the present study is a response to a gap in existing literature which requires the application of more integrated theory testing and the identification of factors that influence the continued use of internet banking in order of importance to consumers. the study sets out to develop a conceptual framework based on theoretical models related to the acceptance of technology and diffusion of innovations theory. the conceptual model developed encompasses technology, channel, social and value for money factors as predictors in the identification of influential factors for consumers and their continued use of internet banking. data were collected using a cross-sectional mall intercept survey in the western sydney region, comprising a sample of 372 internet banking users. after testing the data for accuracy, exploratory factor analysis was performed for factor clarification on five key scales related to the use of internet banking service delivery based on technology, channel, social, value for money and continued use factors. the final components that emerged were employed in hierarchical multiple regression analyses to test the proposed framework. value for money, channel and technology factors was found, in that order, to be influential in predicting consumers’ continued use of internet banking. keywords: continued use, internet banking, value for money factors, channel factors, technology factors introduction the traditional modes of conducting many businesses has been changed over a period of time due to the advent of the internet, rapid technological evolutions, globalisation, financial deregulations, liberalisation and consolidation of the financial markets (mia et al. 2007). several modern businesses mailto:adapa2@une.edu.au mailto:rcooksey@une.edu.au australasian journal of information systems volume 18 number1 2013 6 are also utilising technological advancements in order to make their services more accessible to consumers (reid & levy 2008). the retail banking sector is one such business entity that has been using a wider array of information systems to satisfy its customers (wresch & fraser 2006). the retail banking industry has also realised that relying exclusively on the traditional price-related factors for competition was a questionable way to increase revenue and market share in the industry (durkin et al. 2008). the two largest expenses incurred by banks are associated with the maintenance and overhead costs of branch networks and their associated human resources (durkin 2004). banks started to realise that relying on non-price factors provided alternative strategies for differentiation, gaining competitive advantage and cost cutting (daniel 1999). as a result, the financial services industry has become much more competitive over the last few years (thornton & white 2001). in an intensifying competitive environment, superior distribution strategies relating to how to communicate with, and deliver products to, the customer effectively provides a comprehensive advantage to the banking institutions (kerem et al. 2003). moreover, customers are also demanding greater convenience and accessibility, as reflected in longer branch opening hours and an increase in the choice of service delivery mechanisms. on a global scale, many banks have started to set in place more cost-effective alternative service delivery systems (shih & fang 2004). the trend has seen the proliferation of multiple service delivery channels through which consumers can interact with banks. therefore, modern banks provide their consumers with increased channel choice that reaches out to consumers through many different routes. alternative service delivery systems such as automated teller machines (atms), telephone, internet and wireless channels are now available to the consumers to perform their banking transactions in addition to the traditional branch banking (reid & levy 2008; akinci et al. 2004). the growing use of technologies in services has revolutionised the interactions between service providers and customers and increased standardisation of many services (liljander et al. 2006). the introduction of new service delivery channels increases the productivity and efficiency of the service providers apart from offering access to customers, thereby better meeting customer demand and increasing satisfaction (liljander et al. 2006; walker et al. 2002). internet banking facilitating electronic transactions is a key facilitator of communication in business activities and in consumer’s day-to-day activities (mccole & ramsay 2009). the decision to perform internet banking is at the consumers’ discretion, indicating the importance of the consumer in electronic interactions. it is critical for business success to understand how consumers make decisions to use internet banking and at the same time, how marketers try to understand consumers’ values, beliefs and preferences (lindgreen & wynstra 2005). existing customers represent a valuable asset to the service provider. from the bank’s perspective, maintaining and retaining their existing customer base is always more profitable than trying to acquire new customers (srinivasan et al. 2002). therefore, banks continuously scan and attempt to persuade customers not to switch to competitors in order to maximise their sales and profits (hammond & ehrenberg 1995). customer participation has always been an indispensable part of any service delivery process (karmarkar & pitbladdo 1997). the management of the service delivery channel performance presents an unusual problem in that the performance of the service system is affected by the actions of customers who may feel uncertain about their capabilities and goals. in the past decade, the objective of most research in internet banking research has been to analyse the acceptance and adoption of internet banking service delivery channels and to understand the factors, which lead to the adoption process (chen et al. 2005). there are multitudes of internet banking australasian journal of information systems volume 18 number1 2013 7 studies that have investigated factors that influence a consumer’s intention to adopt or their acceptance of internet banking (gerrard & cunningham 2003; lee et al. 2003). a persistent argument is that prior studies on internet banking adoption by consumers, and the factors responsible for their adoption of the internet, have produced inconclusive and sometimes conflicting results. this has culminated in a difficulty in articulating the precise findings of internet banking research (ndubisi & sinti 2006). thus, the findings from research on internet banking from the consumer’s perspective remains unclear and in many ways impractical as it is not applied to the business context. since its introduction, internet banking, due to its speed and convenience, has been widely accepted by consumers as a service innovation (klopping & mckinney 2004). in recent years, internet banking transactions increased to a greater extent, indicating the voluminous nature of transactions carried out by the internet banking consumers (kasheir et al. 2009). this phenomenal uptake and use of internet banking, together with the internet banking experience, modifies the behaviour of individuals towards the channel (gefen et al. 2003). change is evident in consumer’s initial perceptions and heightens the need to understand not only consumer’s pre-adoption behaviour, but also post-adoption behaviour (yu et al. 2005). it is only recently that researchers have started to understand the importance of consumers’ continued use of internet banking and its impact on the financial performance of the banking sector (kasheir et al. 2009; yousafzai et al. 2005). continued use of internet banking by customers is often neglected in the marketing literature, as most studies focus on either consumer adoption or acceptance of internet banking (eriksson & nilsson 2007). however, in practice, in comparison to new buyer acquisition, continued use is related to a cost-effective marketing strategy aimed at retaining customers. the goal of any business entity in the long term is to increase its productivity and maximisation of the profits. retention strategies foster customer relationship management and reduce customer switching behaviour (kasheir et al. 2009). in order to achieve this, it is vital to concentrate on the consumers’ continued use of a particular product or service rather than solely on their acceptance of the service (kim & malhotra 2005). with these considerations in mind, and the existing gaps in the knowledge surrounding consumers’ continued use of internet banking, the present study focuses on exploring the antecedents to consumers’ continued use of internet banking. literature synthesis methodologically speaking, it has become increasingly important for researchers to focus on integrating relevant theories and developing hybrid models for the development of theory in the academic context (kasheir et al. 2009; yousafzai et al. 2005). the present study uses theoretical models of technology acceptance and diffusion of innovations theory and integrates them in order to identify the potential factors that might have an impact on consumers’ continued usage of internet banking. the value of integrating models lies in the ability of the findings from such research to be more robust and less open to criticisms based on reliability and validity of the findings. nevertheless, in recent years, internet banking has shown increasing uptake by consumers and there are more and more consumers who have carried out internet banking transactions several times. this, together with the fact that when consumers have experience with internet banking their behaviour is modified, which produces a change in their initial perceptions about internet banking. such a change in consumer behaviour makes it necessary to study not only the pre-adoption behaviour of consumers, as in previous research, but also the post-adoption behaviour towards internet banking (hernandez et australasian journal of information systems volume 18 number1 2013 8 al. 2008). therefore, the objective of the present study is to analyse the post-adoption behaviour of experienced internet banking users in identifying what factors influence consumers to continue to use internet banking. thus, from the existing research, possible theoretical models that provide a comprehensive understanding of user acceptance of innovations come from disciplines such as information systems, psychology and sociology. according to bagozzi (2007) the theory of reasoned action (tra), theory of planned behaviour (tpb) and technology acceptance model (tam) models all seem to neglect group, social and cultural aspects of the consumer decision-making process. mostly these models depend on a purely deterministic framework and often self-regulation processes are not taken into consideration. the models also attempt to evaluate the consumers’ post-purchase decision-making process; however, in reality, these theories rely on consumers’ perceptions of innovation characteristics as a significant predictor of behavioural intention. however, kasheir et al. (2009) and many researchers believe that these theories are incomplete and that integrating aspects of these theories provides a better understanding and explanation of consumer evaluation of the post-purchase decision-making process rather than what is provided by each theory alone. consumers’ continued use of internet banking assessment may be more complex than current theories explain, due to the fact that consumers often need to abandon or minimise their current behaviour and usual concerns associated with technological advancements (falk et al. 2005). also, usage depends upon an individual’s capability and capacity to engage with these proliferated service delivery channels (walker & johnson 2006). often, continued use is related to the technology updating mechanisms (kim & malhotra 2005). consequently, existing models, whilst indicative of internet usage behaviour, are not complete in terms of all of the factors consumers refer to in their decisionmaking. technological advances in society have an impact on marketing practices. information and knowledge about markets, customers and competitors becomes more complex and available with the implementation of technological innovations (rust & espinoza 2006). as a result, better services can potentially be offered by companies to satisfy each customer’s specific needs. thus, a clear shift from traditional product orientation to customer orientation in marketing practice is evident as businesses vigorously compete in the application of technology to services (lovelock & wirtz 2004). therefore, service delivery has become the central focus of many businesses today, paving the way for a new paradigm described as ‘the service revolution’. existing research has revealed the multifaceted use of technology-enabled products and services (ye et al. 2008). the influence exerted by technological innovations in pre-adoption consumer behaviour has been acknowledged by researchers (jha et al. 2006). the simultaneous interactions of consumers with technology have caused them to experience positive and negative reactions about the use of technology (parasuraman 1997). an array of consumer reactions such as control, competence, efficiency and assimilation can be triggered when consumers experience technological interactions (mick & fournier 1998, p.126). in order to cope with technologies, consumers engage in different strategies that range from non-adoption and ignorance, to actual adoption and usage (parasuraman 2000). however, consumers continue to modify and innovate with technology artefacts, even after adoption of a technology-associated product or service (desouza et al. 2007). the role played by technology-related factors during consumers’ postadoption behaviour remains unexplored. as consumer’s level of experience and sophistication in using a particular technology becomes elevated, consumers’ expectation of the service suitability to their specific needs also increases (ye et al. 2008). australasian journal of information systems volume 18 number1 2013 9 the past decade has seen some of the most rapid and substantive changes in channels of distribution for goods and services in many economies (black et al. 2002). the advent of the internet challenged the traditional role played by many intermediaries and presented buyers with a choice in terms of their distribution channel usage. today’s consumers are exposed to several service delivery channels and researchers in this area tend to concentrate on understanding and analysing channel design strategies, instead of the issue of channel selection and usage which have received relatively little attention in the existing literature (yousafzai et al. 2005). such trends therefore highlight the growing need to understand the ways in which consumers may choose between alternate channels and the circumstances under which one channel may be chosen over another (black et al. 2002). the proliferation of service delivery channels is evident in the financial services context, specifically in the retail banking sector. much of the initial channel adoption literature in financial services marketing research focused attention on atms (rugimbana 1995). in the 1990s, there was a shift in interest towards telephone banking (lockett & littler 1997) and electronic banking (liao et al. 1999). subsequent studies focused specifically on internet banking (black et al. 2001), although the initial focus of attention was on the activities undertaken in organisations rather than on the behaviour of consumers (black et al. 2002). some later studies started realising the importance of consumers and their channel interactions and emphasised the degree of congruence between a particular service and a channel (morrison & roberts 1998). moreover, existing research also concentrated on identifying factors influencing a consumer’s decision to adopt a particular channel as well as multiple channels (bell & lyman 1999). the approach of the present study is different in two ways. firstly, it makes an attempt to focus on consumer channel interactions during their post-adoption behaviour. secondly, channel-related factors are identified as consisting of components such as consumers’ perceptions of their self-efficacy, risk, trust and personalisation. finally, the study attempts to examine the influence of these channel factors on consumers’ continued use of internet banking. social influence is identified extensively in the existing literature as normative pressure, while subjective norm refers to a person’s perception of the social pressures put on him/her to perform or not to perform the behaviour in question (ajzen & fishbein 1980). thus, social influence has been treated in the existing literature as analogous to the subjective norm construct incorporated in the tra and the tpb (moore & benbasat 1991). the role of social influence in an individual’s decision whether or not to use a technology is complex and subject to a wide range of contingent influences (venkatesh et al. 2003). internet banking as a service innovation creates uncertainty and individuals who are uncomfortable with uncertainty will tend to interact with their social network before making a decision (lu et al. 2005). mathieson (1991) reported that social variables could be important in influencing consumers’ perceptions if they capture variance that is not already explained by other variables. taylor & todd (1995) decomposed the sources of social influence into peers, superiors and subordinates. the influence from peers and superiors significantly relate to the subjective norm construct that leads to consumers’ behavioural intention and usage of technology. the role of social norms in information technology usage literature to date is somewhat unclear (yousafzai et al. 2003). the study of the influence exerted by the social factors construct on consumers’ adoption of internet banking and internet-based technologies produced mixed and inconclusive results (ma & liu 2004). the creation of value is key in marketing (lindgreen & wynstra 2005). the role of marketing is ‘to assist the firm to create value for its customers that is superior to competition’ (tzokas & saren 1999, p.53). it is essential to know what consumers actually value before one can truly understand the australasian journal of information systems volume 18 number1 2013 10 choice of a particular service delivery channel, and its continued usage. it is also of equal interest in identifying the attributes that are most important in consumers’ judgments of value. the present study focuses on the value that is derived from the disposition of products and services and extends it to the service delivery channels choice generally, and specifically to the internet banking context. the diversity associated with the meanings of the value concept indicated the difficulty of conceptualisation of the value construct. in this study, value factors relate to consumers’ perceptions and expectations of value. the concept of perceived value stems from equity theory. perceived value is identified as the ratio of the consumers’ outcome/input to that of the service providers’ outcome/input (oliver & desarbo 1988). in other words, the equity concept refers to customer’s evaluation of what is fair, right or deserved for the perceived cost of the offering (bolton & lemon 1999). the choice of a particular banking service delivery channel, its subsequent acceptance, and continued usage depends on consumers’ expectations of the value offered. it is important to understand what consumers’ value for and what the important dimensions are, that play a critical role in their judgments of value offering. perceived value represents a consumers’ overall assessment of the utility based on perceptions of what is received and what is given (zeithaml 1988). also, perceived value is identified in the literature as a trade-off between the perceived benefits and perceived costs derived by the consumers from using a particular channel. monetary payments and non-monetary sacrifices, such as time consumption and stress experienced by consumers, together include perceived costs (yang & peterson 2004), whereas, customer perceived value emerges from an evaluation of the relative rewards and sacrifices associated with the offering. the conceptual framework shown in figure i has a predictive intent. the demographic characteristics include gender, age, level of education, level of income, ethnicity and occupation. the specific order of entry of the predictor variables was based on the theoretical and logical considerations. demographic characteristics are not of research interest as their influence has been widely acknowledged in the existing research. therefore, demographic characteristics will be statistically controlled for in order to avoid any confounding effects they might exert on the predictor variables. this will be accomplished by entering the demographic set of variables first in the hierarchical predictive model. technology factors related to the context of internet banking will be entered into the model at the second step and are expected to significantly predict consumers’ continued usage of internet banking over and above the variability explained by the demographic set. technology-related factors were considered to be important, as consumers, during their post-adoption, tend to be more confident with the use of the technology-enabled internet banking service delivery channel. in the third step, channel factors will be entered to significantly predict consumers’ continued usage of internet banking over and above the influence exerted by the demographic characteristics set and technology factors. the order of entry of channel-related factors was based on the theoretical foundations and the role attributed to the consumers’ psychographics in the existing literature (black et al. 2002). in the fourth step, social factors will be entered in to the model and will be expected to significantly predict variability in the criterion variables over and above the previously entered demographics set, technology and channel factors. the order of entry of the social factors was of theoretical importance, asserted to the influence of the peers, friends and family members in influencing consumers’ behaviour, once the consumer encompasses contextual and psychological factors. finally, value for money factors will be entered into the model expected to significantly predict the criterion variables over and above those of previously entered predictor variables. australasian journal of information systems volume 18 number1 2013 11 technology factors channel factors social factors value for money factors continued use of internet banking figure 1: overall conceptual framework showing the order in which variables sets will be considered in the prediction of continued use of internet banking h1: identified technology factors will significantly predict continued usage of internet banking over and above the influence of the demographic control variables. h2: identified channel factors will significantly predict continued usage of internet banking over and above the influence of the demographic control variables and technology factors. h3: identified social factors will significantly predict continued usage of internet banking over and above the influence of the demographic control variables, technology and channel factors. h4: identified value for money factors will significantly predict continued usage of internet banking over and above the influence of the demographic control variables, technology, channel and social factors. research methodology the present study was conducted in the sydney region, as it is a financial centre in australia, as well as the location where a majority of the retail banks conduct their operations. the questionnaire was distributed to both the users and non-users of internet banking. out of the 1308 respondents intercepted in the foyer of the shopping mall, 698 respondents answered the survey questionnaire on the spot. of these questionnaires obtained, only 683 questionnaires were valid with complete responses, thus indicating 52.5 percent of the response rate. among the 683 usable questionnaires, 372 were completed by users and the remaining 311 by non-users of internet banking. for the purpose of this study, only questionnaires obtained from the internet banking users (n=372) has been taken into consideration for subsequent analysis. the survey instrument consisted of questions pertaining to respondent’s general banking habits and internet usage, demographic characteristics, scale items and open-ended questions, organised into logical sections. questions related to general banking behaviour, internet usage and demographic characteristics were incorporated in the questionnaire irrespective of the respondent’s internet banking australasian journal of information systems volume 18 number1 2013 12 usage pattern. to begin with, the questionnaire consisted of an open-ended question asking the nonusers of internet banking to give the reasons why they do not use internet banking. if respondents did not use internet banking they were thanked for their participation and then screened out of the survey. the questionnaire then goes on to obtain answers from internet banking users, which contained items on technology, channel, social, value for money and continued use factors. all the five scales representing the major variables were used by researchers in different contexts whose validity and reliability had been established in previous studies. furthermore, the survey also had an open-ended qualitative section included at the end in order to collect the respondent’s opinions about their internet banking patterns from the users of internet banking. technology factors included in the present study were operationalised in a manner consistent with previous internet banking studies in other research contexts (hernandez & mazzon 2007; chan & lu 2004; tan & teo 2000; moore & benbasat 1991). for this study, technology factors were measured using a 17 item scale consisting of five sub-dimensions such as relative advantage, compatibility, complexity, trialability and result demonstrability (moore & benbasat 1991). channel factors were operationalised in accordance with work by hernandez & mazzon (2007), chan & lu (2004), tan & teo (2000), rhee & riggins (1997), and lee (1996). channel factors were measured using the 15 item scale consisting of four sub-dimensions such as perceived self-efficacy, perceived risk, perceived trust and perceived personalisation (hernandez & mazzon 2007; tan & teo 2000). social factors were operationalised consistent with work by chan & lu (2004), venkatesh & davis (2000), and tan & teo (2000). in this study, social factors were measured using the six-item scale consisting of two sub-dimensions subjective norm and interpersonal influence. value for money factors were operationalised in accordance with work by thomas & sullivan (2005), keen et al. (2004), petrick (2002), zeithaml et al. (2002), and sweeney & soutar (2001). in this study, value for money factors was measured using the eight-item scale consisting of two sub-dimensions perceived benefits and perceived costs. continued use factors were operationalised consistent with work by hernandez & mazzon (2007), chan & lu (2004), and venkatesh & davis (2000). continued use factors were measured using a three item scale without any sub-dimensions. data were collected from the respondents who were current internet banking users, using the aforementioned scale items, which used a six-point likert scale defined as (1) strongly agree, (2) agree, (3) neither agree nor disagree, (4) disagree, (5) strongly disagree, and (6) unable to rate. initially all questionnaires obtained were checked for completeness. vetting of the questionnaires obtained was done with the objective of increasing accuracy and precision. vetting was done by screening the questionnaires to identify illegible, incomplete, inconsistent and ambiguous responses. a coding process was followed involving the assignment of a code to represent a specific response to a specific question along with the data record and column position that code will occupy. whilst performing data coding it was ensured that all the data were numeric, each variable for each respondent occupied the same column in the spss data editor, all the codes assigned for a variable were mutually exclusive, each variable was coded in order to obtain maximum information and coding rules were applied consistently for all respondents (cohen et al. 2003). coded data were entered from the questionnaires to spss 17.0 computer program (spss 2008). the spss data records were checked thoroughly and extensively for consistency and missing responses as part of the data cleaning process. australasian journal of information systems volume 18 number1 2013 13 results the demographic profile of the participants who responded to the survey as internet banking users and non-users of internet banking indicated that there were more males (53.8%) in the sample of internet banking users than females. among the non-users of internet banking there were more females (65.9%) than males. maximum numbers of the respondents identified were within the age range of 31 to 40 years as internet banking users and non-users represented as 28.2% and 22.5% respectively. the majority of the internet banking users had attained a higher education level, whilst for the majority of non-users of internet banking the highest level of education was at the vocational education and training level (45.7%). internet banking users (21.2%) were within the income range of $80 001 and over, and non-users of internet banking (28.6%) were within the income range of $50 001$65 000. most internet banking users were of asian ethnicity (23.7%) and most non-users were anglo-australians (22.8%) in the sample. maximum numbers of the respondents as internet banking users were derived from the managerial occupation (30.1%). the maximum number of respondents as internet banking non-users was reported from the managerial (18.6%) and administration (18.3%) categories. exploratory factor analysis was performed using principal components and promax rotation. certain scale items were deleted from entering further analysis as they did not meet the essential criteria such as low factor loading values, less eigenvalues or low item-to-total correlations (hair et al. 2006). the log transformed scale items ‘internet banking can be frustrating (ltd5)’, ‘internet banking is an easy way to conduct banking transactions (ltd11)’, and ‘i would have difficulty explaining to others why using internet banking may be beneficial (ltd17)’ of technology factors were deleted from the original scale due to the presence of factor loadings with an absolute value of less than 0.4. similarly, the scale item ‘performing internet banking transactions makes me feel psychologically uncomfortable (lcd8)’ was deleted from the channel factors due to the presence of low loading values. a refined exploratory factor analysis was performed excluding the scale items that did not satisfy the assumptions of the exploratory factor analysis. principal component analysis output for the technology factors the principal component analysis method and a promax rotation of the technology determinants constituting fourteen scale items was conducted on a random sample (n = 372) of australian internet banking users and is presented in table i. a two-component solution provided the clearest extraction, using the kaiser-guttman retention criterion of eigenvalues of more than one. the component correlation matrix indicates that the two components emerged from the technology factors to be positively correlated (0.42) at a moderate level, are fairly well discriminated and accounted for 38.23 percent of the total variance. table i presents the fourteen scale items, communality estimates and eigenvalues. the two technology factors components were labelled based on a review of the thematic content of the different items of their defining items. ‘perceived usability’ component consists of items relating to the consumer’s extent of ease and convenience with internet banking, compatibility with their personal and professional lifestyles and the extent to which they could communicate the benefits of internet banking usage to others. ‘perceived trialability’ component relates to the extent to which customers can actually try internet banking or any add on features with regard to internet banking on a limited basis before they actually consume the service. australasian journal of information systems volume 18 number1 2013 14 scale items factors 1 2 ltd3 internet banking allows me to manage my finances more efficiently 0.71 ltd2 internet banking gives me greater control over my finances 0.70 ltd6 internet banking is compatible with my lifestyle 0.70 ltd7 using internet banking fits well with the way i like to manage my finances 0.68 ltd4 internet banking is a convenient way to manage my finances 0.67 ltd8 using the internet to conduct banking transactions fits into my working style 0.65 ltd10 internet banking is useful for managing my financial resources 0.53 ltd1 internet banking makes it easier for me to conduct my banking transactions 0.51 ltd16 i believe i could communicate to others the consequences of using internet banking 0.46 ltd17 i would have no difficulty explaining to others why using internet banking may be beneficial 0.41 ltd9 internet banking does not require a lot of mental effort 0.45 ltd12 prior to the actual adoption, internet banking is available for me to use on a trial basis 0.84 ltd14 i have a great deal of opportunity to try internet banking before the actual adoption 0.79 ltd13 i am able to use internet banking to see what it can do for me prior to the actual adopti0on 0.76 eigenvalues kmo bartlett’s test of sphericity 3.93 0.81 p < 0.001 2.06 factor correlation matrix factor 1 factor 2 factor 1 1.00 0.42 factor 2 0.42 1.00 table 1: summary of principal component analysis for technology factors measure (n = 372) principal component analysis output for the channel factors the results of the principal component analysis output for the channel factors are presented in table ii. using the kaiser-guttman retention criterion of eigenvalues greater than one, a two-component solution provided the clearest extraction. these two components accounted for 36.59 percent of the total variance. the two component model was deemed to be the best solution because of its conceptual clarity and ease of interpretation. also, the component correlation matrix reveals that the two emerged components of the channel-related factors to be moderately positively correlated (0.34) and are well discriminated. australasian journal of information systems volume 18 number1 2013 15 scale items factors 1 2 lcd10 internet banking is trustworthy 0.70 lcd11 i trust in the ability of internet banking to protect my privacy 0.66 lcd12 using internet banking is financially secure 0.62 lcd1 i am confident of using internet banking even if there is no one around to show me how to use it 0.62 lcd3 i am confident of using internet banking even if i have only the online instructions for reference 0.58 lcd9 internet banking transactions does not lead to an inefficient use of my time 0.55 lcd6 i am confident that internet banking in australia is secure 0.53 lcd7 information concerning my internet banking transactions cannot be accessed by others 0.43 lcd5 i am confident of using internet banking if i could call someone for help if i got stuck 0.74 lcd4 i am confident of using internet banking if i see someone else using it before i try it myself 0.67 lcd14 internet banking websites provide information that is tailor-made for me 0.54 lcd2 i am confident of using internet banking if i have the built-in online “help” function for assistance 0.49 lcd13 performing internet banking makes me feel that i am a unique customer 0.44 lcd15 the promotional offers that i receive through internet banking transactions are attractive to me 0.40 eigenvalues kmo bartlett’s test of sphericity 3.42 0.77 p < 0.001 1.69 factor correlation matrix factor 1 factor 2 factor 1 1.00 0.34 factor 2 0.34 1.00 table 2: summary of principal component analysis for channel factors measure (n = 372) the two channel factors components were labelled based on a review of the thematic content of the different items of their defining items. ‘perceived safety’ component relates to the safety of internet banking with regard to the risk, security and trust associated by the consumers with internet banking as a service delivery channel. moreover, the component included items related to the internal component of self-efficacy, indicating that internet banking users were confident that they can perform their transactions without any help. ‘perceived specialty’ component is related to the personalisation component perceived by the consumers as unique and special customers due to their act of performing internet banking. perceived specialty component also consists of the external component of self-efficacy, indicating less confidence associated with the internet banking consumers without the presence of a role model or online help functions. australasian journal of information systems volume 18 number1 2013 16 principal component analysis output for the social factors the results obtained from the principal component analysis for the social factors are presented in table 3. the kaiser-guttman retention criterion of eigenvalues greater than one provided the clearest extraction of a one-component solution. the resultant single component accounted for 64.72 percent of the total variance. scale items factor lsd1 my decision to use internet banking is influenced by my friends 0.72 lsd2 my decision to use internet banking is influenced by my family 0.63 lsd3 my decision to perform internet banking is influenced by my colleagues 0.70 lsd4 i frequently gather information from friends about internet banking before i actually perform any transactions 0.73 lsd5 i often check with others to make sure that i am properly using internet banking 0.67 lsd6 i often observe how others conduct their banking transactions before i actually perform any transactions 0.41 eigenvalues kmo bartlett’s test of sphericity 3.88 0.89 p < 0.001 table 3: summary of principal component analysis for social factors measure (n = 372) principal component analysis for the social factors construct resulted in an extraction of a single solid component consisting of the scale items related to the influence exerted on the customer to use internet banking by the friends, family members and colleagues. principal component analysis output for the value for money factors table 4 shows the principal component analysis solution for the value for money factors construct. the kaiser-guttman retention criterion of eigenvalues greater than one provided the clearest extraction of one solid component solution. the resultant component accounted for 38.19 percent of the total variance. scale items factor lcpv1 internet banking is very reliable 0.46 lcpv2 internet banking is of outstanding quality 0.43 lcpv3 internet banking provides easy access to information 0.48 lcpv4 i am happy with the price charges for performing internet banking 0.42 lcpv5 internet banking has a good reputation in australia 0.58 lcpv6 internet banking transactions can be performed with very little effort 0.59 lcpv7 performing internet banking transactions is the right decision when price and other expenses are considered 0.47 lcpv8 internet banking transactions are economical 0.69 eigenvalues kmo bartlett’s test of sphericity 3.05 0.82 p < 0.001 table 4: summary of principal component analysis for value for money factors measure (n = 372) australasian journal of information systems volume 18 number1 2013 17 the principal component analysis solution for the value for money factors resulted in the extraction of a single component consisting of the scale items representing the trade-off between the benefits and costs associated by the customer with the use of internet banking as a service delivery channel. principal component analysis output for the continued use factors table 5 displays the summary of principal component analysis for continued use factors measure. the kaiser-guttman retention criterion of eigenvalues greater than one provided the clearest extraction of a single solid component solution. the resultant component accounted for 57.39 percent of the total variance. scale items factor lud1 internet banking usage is a positive experience for me 0.61 lud2 i intend to use internet banking regularly to perform my banking transactions 0.66 lud3 i intend to increase my use of internet banking transactions in future 0.45 eigenvalues kmo bartlett’s test of sphericity 1.72 0.61 p < 0.001 table 5: summary of principal component analysis for continued usage factors measure (n = 372) principal component analysis of the continued usage factor extracted a single component. the resultant component consists of scale items related to the customer’s associations with internet banking, intention to perform banking transactions through the internet regularly, and their intentions to increase the use of internet banking in future. reliability analysis cronbach’s alpha is the most common form of internal consistency reliability coefficient. by convention, a lenient cut-off of 0.60 is common in exploratory research; alpha should be at least 0.70 or higher to retain an item in an ‘adequate scale’, and a cut-off of 0.80 is required for a ‘good scale’ (graham 2006). in the present study, an approach guided by a lenient cut-off of 0.60 is considered to be appropriate. the reliability statistics for the scales identified via principal components analysis are presented in table 6. in all cases, cronbach’s alpha exceeded at least 0.60. for all subsequent analyses, respondents were given scores on each identified component or scale by averaging the scores on the items that defined that scale. variable sub-scale number of items cronbach’s alpha technology factors perceived usability 11 0.81 perceived trialability 3 0.75 channel factors perceived safety 8 0.73 perceived specialty 6 0.60 social factors 6 0.89 value for money factors 8 0.76 usage factors 3 0.62 table 6: reliability statistics australasian journal of information systems volume 18 number1 2013 18 the main aim of the study was to predict the power of the continued use of internet banking to influence a specific order of independent variable sets, thus assessing the unique contribution of each set of independent variables. such a task is ideally suited to the statistical analysis of hierarchical multiple regression (cooksey 2007). the specific order of entry of the independent variable sets was based both on logical and theoretical considerations (kasheir et al. 2009; black et al. 2002, 2001), resulting in a parsimonious model. demographic variables were entered at the beginning of the analyses based on the logical consideration to control for their influence. the order of entering the technology, channel, social and value for money factors was based on the theory as discussed in the literature synthesis. hierarchical multiple regression analyses was important as ‘semi-partialling’ or ‘the statistical removal of the influence of one or more variables from the correlational relationship between two variables’ (cooksey 1997, p. 78-79). thus, the partialling set of variables to be controlled for are considered first, followed by the actual predictor variables of interest which are added subsequently. table 7 summarises the different independent variable sets and their constituent predictor variables in the order in which the sets would be entered into the hierarchical regression model. data intended to be subjected to hierarchical regression analysis were obtained from computed average scores on the two confirmed components of technology factors, two confirmed components of channel factors, one solid component of social factor and one resultant component of value for money factor, as identified earlier. variable sets variables demographic characteristics gender age level of education level of income ethnicity occupation technology factors perceived usability perceived trialability channel factors perceived safety perceived specialty social factors social factors value for money factors value for money factors table 7 variable sets and their constituent predictors included in the hierarchical regression analyses the data pertaining to all the variables in table vii were examined through the spss 17 program for data accuracy, normality and assumptions of multivariate analyses prior to the actual hierarchical regression analyses. then list wise deletion method was considered to be the safest and most appropriate option from a statistical estimation perspective since there were minimal missing data. explanation of the model predicting the continued use of internet banking hierarchical multiple regression analysis was employed to test for the effects of all independent variable sets in predicting the continued use of internet banking as the dependent variable. table viii displays the different variable sets entered in their sequential order, the r 2 change and its associated degrees of freedom, the f change (partial f-test) and associated p-value. the resultant regression australasian journal of information systems volume 18 number1 2013 19 weights obtained from the hierarchical multiple regression analysis is based on the log transformations of the independent variables. the right-hand portion of table viii lists the p-values for the individual independent variable sets contributing to the predictor variable at that step, relating to the variables analysed, the part (semi-partial) correlation, partial f, and sig. p-value. the durbin-watson statistic was used to test for the presence of serial correlation among the residuals that is the assumption of independence of errors, which requires that the residuals or errors in prediction do not follow a pattern from case to case (meyers et al. 2006). the value of the durbinwatson statistic ranges from 0 to 4. as a general rule of thumb, the residuals are not correlated if the durbin-watson statistic is approximately two, and an acceptable range is 1.50-2.50 (wilcox 1997). for the present research, the durbin-watson statistic of 2.10 falls within the acceptable range. the tolerance values for all the independent variables were larger than 0.10, indicating that multicollinearity was not a problem in progressing with the proposed regression analysis (cleveland 1984). table viii displays a summary of the hierarchical multiple regression analysis with five models listed. at the end of each step model r 2 was significantly different from zero. in the final model, with all the independent variable sets in the equation, the overall regression model was significant (r 2 = 0.42, adjusted r 2 = 0.39, f = 20.34, p < 0.001). thus, with all variable sets entered, nearly 43 percent of the variance in continued use of internet banking was explained. in model 1, the demographic characteristics set of variables did not contribute significantly to the prediction of continued use of internet banking (r 2 change = 0.01, f change = 0.85, p = 0.52). at model 1, the demographic characteristics set were entered first in order to control for their influence. demographic characteristics were examined to make sure that the effects attributed to later independent variable sets entered into the hierarchical multiple regression analysis were not due to demographic differences, thus removing their potential contaminating influence, if any. all other variable sets (i.e., technology, channel, social and value factors) were entered from model 2 to model 5. in model 2, the technology factors contributed significantly to the prediction of continued use of internet banking over and above what the demographic characteristics set could predict (r 2 change = 0.23, f change = 55.87, p < 0.001). thus, the technology factors set uniquely accounted for 23 percent of the variance in continued use of internet banking over and above the previously entered set. among the technology factors set, the perceived usability component (sr 2 = 0.45, partial f = 10.04, p < 0.001) and perceived trialability component (sr 2 = 0.18, partial f = 2.38, p < 0.001) were significant in the explanation of the variance in continued use of internet banking at this step. increases in perceived usability and perceived trialability were predictive of an increase in consumers’ continued use of internet banking. in model 3, the channel factors contributed significantly to the explanation of variance in continued use of internet banking over and above what the demographic characteristics and technology factors set could predict (r 2 change = 0.13, f change = 38.31, p < 0.001). thus, the channel factors set uniquely accounted for 14 percent of the variance of continued use of internet banking over and above the previously entered sets. among the channel factors set, perceived safety (sr 2 = 0.23, partial f = 5.69, p < 0.001) and perceived specialty (sr 2 = 0.19, partial f = 4.79, p < 0.001) components were significant in the explanation of the variance in continued use of internet banking. increases in perceived safety and perceived specialty were predictive of an increase in consumers’ continued use australasian journal of information systems volume 18 number1 2013 20 of internet banking. thus, respondents who strongly agreed on perceived safety and perceived specialty components of the channel factors were associated with a significant increase in their continued use of internet banking. in model 4, the social factors set did not add significantly to the prediction of continued use of internet banking (r 2 change = 0.001, f change = 0.19, p < 0.001). thus, respondents did not consider social factors as significant predictors in determining their continued use of internet banking. in model 5, the value for money factors contributed significantly to the explanation of variance in continued use of internet banking over and above what the demographic characteristics, technology, channel and social factors could predict (r 2 change = 0.05, f change = 35.49, p < 0.001). thus, the value for money factors set uniquely accounted for 6 percent of the variance of continued use of internet banking over and above the previously entered sets. thus, respondents who strongly agreed on the value for money factors also tended to strongly agree on their significant increase in continued use of internet banking. model variable set r2 change df f change sig. f change variable sr2 partial f sig. 1 demographic characteristics 0.01 6 0.85 0.52 ns 2 technology factors 0.23 2 55.87 < 0.001* perceived usability 0.45 10.04 < 0.001* perceived trialability 0.18 2.38 < 0.001* 3 channel factors 0.13 2 38.31 < 0.001* perceived safety perceived specialty 0.23 0.19 5.69 4.79 < 0.001* < 0.001* 4 social factors 0.001 1 0.19 0.66 ns 5 value for money factors 0.05 1 35.49 < 0.001* value for money factors 0.23 35.49 < 0.001* overall model r 2 = 0.42, adjusted r 2 = 0.39, f = 20.34, < 0.001* *≤ 0.05 considered significant 0.05 < p ≤ 0.10 considered marginally significant table 8: hierarchical multiple regression model summary table ix provides the hypotheses testing outcomes of the hierarchical multiple regression analyses performed with continued use of internet banking as a dependent variable. australasian journal of information systems volume 18 number1 2013 21 hypothesis result h1: identified technology factors will significantly predict continued use of internet banking over and above the influence of the demographic control variables. h1a: identified perceived usability component of the technology factors will significantly predict continued use of internet banking over and above the influence of the demographic control variables. h1b: identified perceived trialability component of the technology factors will significantly predict continued use of internet banking over and above the influence of the demographic control variables. supported supported supported h2: identified channel factors will significantly predict continued use of internet banking over and above the influence of the demographic control variables and technology factors. h2a: identified perceived safety component of the channel factors will significantly predict continued use of internet banking over and above the influence of the demographic control variables and technology factors. h2b: identified perceived specialty component of the channel factors will significantly predict continued use of internet banking over and above the influence of the demographic control variables and technology factors. supported supported supported h3: identified social factors will significantly predict continued use of internet banking over and above the influence of the demographic control variables, technology and channel factors. not supported h4: identified value for money factors will significantly predict continued use of internet banking over and above the influence of the demographic control variables, technology, channel and social factors. supported table 9: hypotheses testing outcomes discussion the results obtained from the hierarchical multiple regression show the influence of value for money, channel and technology factors on consumers’ continued use of internet banking. as the objective of the study was to identify variables of theoretical importance, demographic variables were controlled. as such demographic variables did not exert any influence on consumers’ continued use of internet banking. similarly, social factors did not exert any significant influence on consumers’ continued use of internet banking as the comfort levels of the consumers’ increases from their pre-adoption and adoption processes. it had been expected that the more the influence of the salient referent groups such as friends, colleagues and family to the consumer, the more would be the usage of internet banking by the consumer on a continued basis. it was assumed that general banking transactions may have been seen to be too private and confidential an activity for respondents to discuss in surveys. therefore, consumers may not be interested in discussing their banking transactions with others within their social and family groups, thus reducing their potential influence on the usage of internet banking. maybe the referent group influence could be contended to apply to pre-purchase and purchase stages of the consumer decision-making process, during which, due to the general apprehension associated with a service innovation, consumers tend to discuss the behaviour under australasian journal of information systems volume 18 number1 2013 22 question with salient referent groups. during the post-purchase stage, the influence of the referent groups might drastically reduce as consumers become more accustomed to the use of internet-based transactions. further, as consumers use internet banking in the post-purchase stage more continuously and frequently, their familiarity with the service delivery channel and presence of the trialability option encourages them to use and try on their own, rather than involving social channels. in the pre-specified hierarchical order of importance, value for money, channel and technology factors were identified to exert significant impact on consumers’ continued use of internet banking. thus, during the consumers’ post-adoption process, consumers’ perceptions of the value for money component play a pivotal role by taking into consideration the economic value of the internet banking service delivery channel, its cost effectiveness and its service quality. moreover, consumers who experienced internet banking as safe, special, secure, convenient and easy tend to perform internet banking transactions on a continuous basis, thus indicating the importance of channel and technology factors. it was expected that the more consumers perceive the value components associated with the use of internet banking service delivery channel, the more they use it on a continued basis. the results showed that consumers clearly weigh up the benefits and costs associated with various banking service delivery channels and tend to use the most profitable one. the above discussion was consistent with similar prior existing studies relating to the value perceptions of the consumers and their subsequent online store choice (petrick 2002). understanding value perceptions from the customer perspective is relevant for managing and supporting both the bank’s and the customer’s value creation processes. further, understanding the dimensions of the value for money factors in this study complements previous theories on customer perceived value and helps in conceptualisation of the value specific to the electronic services. financial entities should provide service offers that generate value to their customers in a sustained way, in order to attain a competitive advantage and maintain it over time. thus, bank management, marketers and service providers should increasingly look at the possibilities of providing business customer value and enhance their profitability. it had been expected that the emerged components ‘perceived safety’ and ‘perceived specialty’ of the channel factors would enhance consumers’ usage of internet banking on a continued basis. the first component of research hypothesis two and its sub-components were supported, as perceived safety and perceived specialty exhibited a significant impact on consumers’ continued usage of internet banking. these results showed that the more consumers perceive their internet banking service channel as safe to perform their banking transactions, the more they use it on a continued basis. a further finding was that the more consumers perceive that they were treated uniquely whilst making their online banking transactions, the more they tend to use internet banking on a continued basis. the perceived specialty aspects shed new light on the emerging research regarding consumers’ usage of internet banking as existing research on this dimension is scarce. it had also been expected that the emerged components ‘perceived usability’ and ‘perceived trialability’ of the technology factors would enhance consumers’ usage of internet banking on a continued basis. results from this study confirm the positive and significant relationship between perceived usability and perceived trialability components and consumers’ continued use of internet banking. these results show that consumers often tend to evaluate various advantageous aspects of the different service delivery channels at the pre-purchase and purchase stages of their decision making process for internet banking products. the perceived usability and perceived trialability components of the technology-related factors still influence consumers’ decisions long after the initial australasian journal of information systems volume 18 number1 2013 23 adoption of internet banking. as consumers become comfortable with the use of the technologyenabled service delivery channels in the post-purchase decision-making process, consumers become more oriented towards the usability nature of a specific channel and therefore continue to use it on a frequent basis. perceived trialability could be related to the consumer’s underlying fear of making mistakes in processing their financial transactions. thus, consumers in an online environment tend to use the trialability feature to test whether transactions can be conducted safely and securely. thus it is evident that highly experienced internet banking service delivery channel users still desire more useful alternatives. theoretically, this research helps to expand the current literature and understanding on the topic and provides a number of approaches for future research. this study relates to the understanding of consumers’ post-adoption behaviour, unlike earlier studies that focused predominantly on consumers’ pre-adoption behaviour. the contribution of this study is the investigation of integrated theory testing based on technology acceptance models and diffusion of innovations theory. this study investigates the conceptual model developed from integrated theory testing, to assess consumer’s continued use of internet banking process by incorporating technology, channel, social and value for money factors. technology, channel, social and value for money factors were conceptualised to measure consumers’ continued use of internet banking to encompass diverse aspects of consumer behaviour, and not limiting its evaluation to consumer’s acceptance and adoption of internet banking. further, the distinct role of consumers’ continued use of internet banking was investigated in the dynamics of technology, channel, social and value for money factors and strengthens the existing literature by examining the different impact of each component entered in the model. furthermore, in order to limit the impact of confounding research findings and conclusions, the present findings showed the importance of controlling for the influence of demographic characteristics on criterion variables. the actual predictive power of the predictor variables would have resulted in erroneous estimations if demographic characteristics were not controlled for whilst undertaking hierarchical multiple regression analyses. the outcomes from the present study add value to marketing research and practice in a broader perspective to the australian financial sector and more specifically to the australian retail banking industry in several ways. the relevance of the technology-related factors emphasises that the traditional focus of banking management on bank employee-customer interaction issues needs to be revised. bank management essentially needs to reorient their focus on technology-customer interactions and effective management of the technology interface. consumers’ attitudes towards the internet banking service delivery channel would be different from those of traditional bank branches (curran et al. 2003). therefore, bank management may opt for the interconnectedness of all the service delivery channels as there is a potential possibility that positive attitudes of consumers towards an existing branch channel might create negative attitudes towards a proliferation of new electronic distribution channels. in order to enhance consumers’ positive attitudes towards internet banking, banks may need to consider publicising widely the various advantages associated with the technology-related internet banking interface. banks could further consider highlighting the importance of the user-friendly features associated with internet banking in order to enhance consumers’ perceptions towards the usability sub-dimension. the banks should also consider providing step-by-step instructions, opportunity to test-drive the technological interface, and possible demonstrations on how to use internet banking effectively and efficiently. although a majority of the banks are proactive in providing step-by-step instructions to australasian journal of information systems volume 18 number1 2013 24 their modern customers about the use of electronic service delivery channels, possible demonstrations and feasible test-drive opportunities of the technological interfaces are available only with a few big banks. banks may need to consider embarking on providing free training sessions to customers regarding the general use of computers and specific use of internet banking and educate them with the benefits associated with internet banking usage. thus, bank marketers strive continuously to incorporate new and useful capabilities to improve the user-friendliness of their service offerings to the customers. service providers of individual consumer-oriented applications should engage in constant efforts to positively influence consumers’ perceptions as improved user satisfaction; broader usage and positive consumer perceptions are better defences against competitors. as technology-enabled service delivery channels like internet banking constantly compete for user support, it is essential to understand what users take into consideration when choosing between available service delivery channel alternatives. thus channel technology designers should always be vigilant of consumers, changing needs and never take consumer’s satisfaction for granted. thus bank management and marketers always strive to improve their products and services and further enhance consumers’ satisfaction and loyalty, regardless of how successful they have been in the past. banks are also developing strategies that will improve the consumer’s trust in banking with the internet by development of secure policies and procedures, incorporation of secure mechanisms such as secure socket layers (ssl), embracing encryption and firewall technologies, virus detection and protection measures, protective mechanisms including backup servers, and working coherently with online security firms which might reduce the perceptions of the consumers that the internet-based service delivery channels are uncertain and unsafe. additionally, banks may need to focus on implementing effective mechanisms to address any violation of the consumer’s sensitive data by placing adequate controls in place to ensure security of personal data. modern banks are also implementing cryptography techniques in order to protect the confidentiality of the consumer needs. during and after exchange of information between the consumer and the service provider, information content should be unchanged and tamper free through encryption and digital signatures. information accessibility and availability should be delivered in as reliable and authorised a manner as possible. banks are striving to provide appropriate measures to protect consumers’ authentication and authorisation regarding their internet-based financial transactions through creation of passwords and access control policies. banks may need to ensure non-repudiation measures such as confidentiality associated to information passed over the network during communication and stored at different locations through digital signatures and certificates issued to the internet banking users. moreover, banks need to continue to protect consumers’ privacy through various policies and disclosures. banks in future should demonstrate the ability to send personalised messages to their prospective and potential internet banking consumers eliciting them as unique customers and, when possible, retrieve consumers’ feedback with regard to their service provision. installation of voice prompts by bank management may create a sort of online personal help and thus attract more consumers to use internet banking channel. at this juncture, banks need to take advantage of marketing communications in reaching their effective target group in order to further enhance their productivity and maximise their profits. brochures, seminars and online information content must be developed based on a clear understanding of the current knowledge level of the customers. bank management may need to consider exerting a great effort whilst selecting and training the online contact personnel. online australasian journal of information systems volume 18 number1 2013 25 contact personnel should also be monitored continuously over time regarding their appropriateness of their level of knowledge and their attitude towards customers. retail banking and marketing managers may need to have a coherent understanding of the relative importance of the various dimensions of value as perceived by customers during their evaluations and accordingly develop marketing strategies. thus, bank management and retail banking marketers try to control or minimise the losses and strive to enhance customer loyalty through customer satisfaction by offering a comprehensive service. the relevant importance and hierarchical impact of the various predictor variables on consumers’ continued use of internet banking would help bank marketers to effectively segment customers based on the motivations that drive their online banking behaviour, rather than on traditional methods. limitations and future directions one of the limitations associated with the current study is related to the reduced capacity to draw causal inferences that were inherent in the application of a survey type of research. the data for testing the conceptual model were obtained in a cross-sectional manner on a single occasion from a single source. therefore the data collection method employed in this study had the potential for single source bias. responses were obtained only from those respondents who agreed to answer the questionnaire by interception in a shopping mall. therefore the findings invite speculation with regard to causal relations concerns and further testing would be essential for verification. in order to avoid any intervention of confounding and spurious relationships, demographic characteristics were statistically controlled for whilst performing hierarchical multiple regression analysis. another limitation of this study is the use of self-report measures associated with consumers’ perceptions of internet banking. such measures have the potential to confound findings due to common method variance. although a limitation, nevertheless it was a convenient method that allowed the researcher to efficiently examine the large number of relevant variables from a wider sample. however, there was no sign of lack of discriminant validity, the usual sign of common method variance among the principal constructs. also self-report measures have been successfully used in prior research (yousafzai et al. 2005; straub et al. 1995). the issue of generalisability is a limitation in most of the consumer behaviour studies and the present study is no exception. the study has been conducted in australia and specifically targeted the retail banking consumers who were internet banking users and non-users. the resultant findings may vary with the application of the study to other banking contexts and other countries. prior internet banking research indicated variation in the findings obtained when conducted in cross-cultural contexts (unnithan & swatman 2001). this may possibly explain why some of the theorised relationships between certain factors and consumers’ continued use of internet banking did not materialise. therefore the present study could not either rule out or rule in the predicted effects in these areas unequivocally. this issue could be attributed partly to the complexities of the models tested. furthermore, this research did not investigate whether the predictor variables exhibit any differences as a function of consumers’ familiarity, nature and type of internet banking transactions and would result in any moderating or mediating effects. similarly, the study did not take into consideration soderlund’s intentions approach, commonly used in the consumers’ repurchase intentions as the aforementioned focuses on customer satisfaction and customer familiarity variables. however, it is hoped that findings from this study will stimulate additional research and will enhance our australasian journal of information systems volume 18 number1 2013 26 understanding of the role of various factors exerted as predictor variables on consumers’ continued use of internet banking outcomes. therefore, replication of similar research and samples would enhance our knowledge whether or not this absence of confirmation may be attributed to the industry-related sample. the findings obtained from the study relate to the financial services sector, retail banking industry in general and specifically address consumers’ usage patterns of internet banking. future research should focus on other related service sectors such as tourism; call centres and hospitality and consumers’ perceptions related to their continued use of the relevant service provider’s service. it is also imperative to study the influence of potential factors on consumers’ perceptions and usage of products and services of credit unions, building societies and non-banking financial institutions and compare the findings with those of the retail banking industry to see if this would result in any similarities and differences. future research could possibly encompass all the electronic service delivery channels in order to determine any variation in consumers’ usage patterns. a further extension of the present research to the other nations and subsequent comparative analysis would result in interesting findings. references ajzen, i. & fishbein, m. 1980, understanding attitudes and predicting social behaviour. prenticehall, upper saddle river, new jersey. akinci, s., akoy, s. & atilgan, e. 2004, adoption of internet banking among sophisticated consumer segments in an advanced developing country. international journal of bank marketing, 22 (3), 212-232. bagozzi, r. p. 2007, the legacy of the technology acceptance model and a proposal for a paradigm shift. journal of the association for information systems, 8 94), 243-254. bell, g. & lyman, m. m. 1999, buying financial services: a model including the channel effect. academy of marketing conference, stirling. black, n. j., lockett, a., ennew, c., winklhofer, h. & mckechnie, s. 2002, modelling consumer choice of distribution channels: an illustration from financial services. international journal of bank marketing, 20 (4), 161-173. black, n. j., lockett, a., winjklhofer, h. & ennew, c. 2001, the adoption of internet financial services: a qualitative research. international journal of retail and distribution management, 29 (8), 390-398. bolton, r. n. & lemon, k. n. 1999, a dynamic model of customers’ usage of services: usage as an antecedent and consequence of satisfaction. journal of marketing research, 36, 171-186. chan, s. & lu, m. 2004, understanding internet banking adoption and use behaviour: a hong kong perspective. journal of global information management, 12 (3), 21-43. chen, t. y., chang, p.l. & chang, h. s. 2005, price, brand cues, and banking customer value. international journal of bank marketing, 23 (3), 273-291. cleveland, w. s. 1984, graphical methods for data presentation: full scale breaks, dot charts, and multibased logging. the american statistician, 38(4), 270-280. australasian journal of information systems volume 18 number1 2013 27 cohen, j., cohen, p., west, s. & aiken, l. 2003, applied multiple regression/ correlation analysis for the behavioural sciences (3 rd ed.). lawrence erlbaum associates, mawah, new jersey. cooksey, r. w. 1997, statistics for behavioural and social research: a descriptive handbook. university of new england, armidale, australia. cooksey, r. w. 2007, illustrating statistical procedures: for business, behavioural & social science research (1 st ed.). tilde university press, australia. curran, j. m., meuter, l. m. & suprenant, c. f. 2003, intentions to use self service technologies: a confluence of multiple attitudes. journal of service research, 5 (3), 209-224. daniel, e. 1999, provision of electronic banking in the uk and the republic of ireland. international journal of bank marketing, 17 (2), 72-82. desouza, k. c., awazu, y. & ramaprasad, a. 2007, modifications and innovations to technology artefacts. technovation, 27, 204-220. durkin, m. 2004, in search of the internet banking customer, exploring the use of decision styles. international journal of bank marketing, 22(7), 484-523. durkin, m., jennings, d., mulholland, g. & worthington, s. 2008, key influencers and inhibitors on adoption of the internet for banking. journal of retailing and consumer services, 15, 348-357. eriksson, k. & nilsson, d. 2007, determinants of the continued use of self-service technology: the case of internet banking. technovation, 27 (4), 159-167. falk, t., schepers, j., hammerschmidt, m. & grossenbacher, s. 2005, should i stay or should i go the role of status quo for using new self-service technology. retrieved 18 th may, 2008, from www.ssrn.com/abstract=962242 gefen, d., karahanna, e. & straub, d. w. 2003, trust and tam in online shopping: an integrated model. mis quarterly, 27 (1), 51-90. gerrard, p. & cunningham, j. b. 2003, the diffusion of internet banking among singapore consumers. international journal of bank marketing, 21 (1), 16-28. graham, j. m. 2006, congeneric and (essentially) equivalent estimates of score reliability: what they are and how to use them? educational and psychological measurement, 66, 930-944. hair, j. f., black, b., babin, b., anderson, r. e. & tatham, r. l. 2006, multivariate data analysis (6 th ed.). prentice hall, nj. hammond, k. & ehrenberg, a. 1995, heavy buyers: how many do you have? how important are they? proceedings of the 24 th annual conference of the european marketing academy, 16511655. hernandez, j. m. c. & mazzon, j. a. 2007, adoption of internet banking: proposition and implementation of an integrated methodology approach. international journal of bank marketing, 25 (2), 72-88. hernandez, o. d., jimenez-martinez, j. & jose marti-de, h. m. 2008, b2c e-commerce acceptance: the moderating effect of gender. communication of the ibima, 6 (16), 104-112. http://www.ssrn.com/abstract=962242 australasian journal of information systems volume 18 number1 2013 28 jha, s., papagari, s., desouza, k. c., seo, d. & ye, c. 2006, technology failures and complaint management: antecedents to post-adoption discontinuance of technology usage. academy of management annual meeting, atlanta, georgia. karmarkar, u. s. & pitbladdo, r. c. 1997, quality, class, and competition. management science, 43 (1), 27-39. kasheir, d., ashour, a. s. & yacout, o. m. 2009, factors affecting continued usage of internet banking among egyptian customers. communication of the ibima, 9, issn: 1943-7765. keen, c., wetzels, m., de ruyter, k. & feinberg, r. 2004, e-tailers versus retailers: which factors determine consumer preferences? journal of business research, 57, 685-695. kerem, k., lustsik, o., sorg, m. & vensel, v. 2003, the development of e-banking in a eu candidate country: an estonian case. proceedings of international atlantic economic society conference, vienna, march 11-17. kim, s. s. & malhotra, n. k. 2005, a longitudinal model of continued is use: an integrative view of four mechanisms underlying post adoption phenomena. management science, 51 (5), 741-755. klopping, i. m. & mckinney, e. 2004, extending the technology acceptance model and the task technology fit model to consumer e-commerce. information technology, learning, and performance journal, 22 (1), 35-48. lee, e., lee, j. & eastwood, d. 2003, a two-step estimation of consumer adoption of technologybased service innovations. the journal of consumer affairs, 37 (2), 256-282. lee, p. 1996, the cutting edge: so far, online banking is mostly wishful thinking. los angeles times, september 30, 1. liao, s., shao, y. p., wang, h. & chen, a. 1999, the adoption of virtual banking: an empirical study. international journal of information management, 19, 63-74. liljander, v., gillberg, f., gunmerus, j. & riel, a. v. 2006, technology readiness and the evaluation and adoption of self-service technologies. journal of retailing and consumer services, 13, 177-191. lindgreen, a. & wynstra, f. 2005, value in business markets: what do we know? industrial marketing management, 34, 732-748. lockett, a. & littler, d. 1997, the adoption of direct banking services. journal of marketing management, 13, 791-811. lovelock, c. & wirtz, j. 2004, services marketing: people, technology, strategy (5 th ed.). prenticehall, upper saddle river, nj. lu, m. t., liu, c. h., jing, j. & huang, l. j. 2005, internet banking: strategic responses to the accession of wto by chinese banks. industrial management & data systems, 105 (4), 429442. ma, q. & liu, l. 2004, the technology acceptance model: a meta-analysis of empirical findings. journal of organisational and end user computing, 16 (1), 59-72. australasian journal of information systems volume 18 number1 2013 29 mathieson, k. 1991, predicting user intentions: comparing the technology acceptance model with the theory of planned behaviour. information systems research, 2 (3), 173-191. mccole, p. & ramsey, e. 2009, a profile of adopters and non-adopters of ecommerce in sme professional service firms. australasian marketing journal, 13, 36-45. mia, m. a. h., rahman, m. a. & uddin, m. m. 2007, e-banking evolution, status and prospects. the cost and management, 35 (1), 36-48. mick, d. g. & fournier, s. 1998, paradoxes of technology: consume cognizance, emotions, and coping strategies. journal of consumer research, 25 (2), 123-143. moore, g. c. & benbasat, i. 1991, development of an instrument to measure the perceptions of adopting an information technology. information systems research, 2 (3), 173-191. morrison, p. d. & roberts, j. h. 1998, matching electronic distribution channels to product characteristics: the role of congruence in consideration set formation. journal of business research, 41 (3), 223-229. ndubisi, n. o. & sinti, q. 2006, consumer attitudes, system’s characteristics and internet banking adoption in malaysia. management research news, 29 (1/2), 16-27. oliver, r. l & desarbo, w. s. 1988, response determinants in satisfaction judgments. journal of consumer research, 14, 495-508. parasuraman, a. 1997, reflections on gaining competitive advantage through customer value. journal of the academy of marketing science, 25 (2), 154-161. parasuraman, a. 2000, technology readiness index (tri): a multiple-item scale to measure readiness to embrace new technologies. journal of service research, 2 (4), 307-320. petrick, j. f. 2002, development of a multi-dimensional scale for measuring the perceived value of service. journal of leisure research, 34 (2), 119-134. reid, m. & levy, y. 2008, integrating trust and computer self-efficacy with technology acceptance model: an empirical assessment of customers’ acceptance of banking information systems (bis) in jamaica. journal of internet banking and commerce, 12 (3). retrieved 15 th april, 2009, from www.arraydev.com/commerce/jibc rhee, h. s. & riggins, f. 1999, gvu’s www user surveys: high level summary of internet banking survey. retrieved 10 th december, 2009, from www.gvu.gatech.edu/user-survey/survey-199704/graphs/banking/report.html rugimbana, r. 1995, predicting automated teller machine usage: the relative importance of perceptual and demographic factors. international journal of bank marketing, 13 (4), 26-32. rust, r. t. & espinoza, f. 2006, how technology advances influence business research and marketing strategy. journal of business research, 59, 1072-1078. shih, y. & fang, k. 2004, the use of decomposed theory of planned behaviour to study internet banking in taiwan. internet research, 14 (3), 213-223. spss. 2008, spss 17.0.1 for windows. spss inc., chicago, illinois. http://www.arraydev.com/commerce/jibc http://www.gvu.gatech.edu/user-survey/survey-1997-04/graphs/banking/report.html http://www.gvu.gatech.edu/user-survey/survey-1997-04/graphs/banking/report.html australasian journal of information systems volume 18 number1 2013 30 srinivasan, s. s., anderson, r. & ponnavolu, k. 2002, customer loyalty in e-commerce: an exploration of its antecedents and consequences. journal of retailing, 78 (1), 41-50. straub, d., limayem, m. & karahanna-evaristo, e. 1995, measuring system usage: implications for is theory testing. management science, 41, 1328-1342. sweeney, j. c. & soutar, g. 2001, consumer perceived value: the development of multiple item scale. journal of retailing, 77 (2), 203-220. tan, m. & teo, t. s. h. 2000, factors influencing the adoption of internet banking. journal of the association for information systems, 1, 1-42. taylor, s. & todd, p. a. 1995, understanding information technology usage: a test of competing models. information systems research, 6 (2), 144-176. thomas, j. & sullivan, u. 2005, managing marketing communications with multichannel customers. journal of marketing (october), 239-251. thornton, t. & white, l. 2001, customer orientations and usage of financial distribution channels. journal of services marketing, 15 (3), 168-184. tzokas, n. & saren, m. 1999, value transformation in relationship marketing. australasian marketing journal, 7, 52-62. unnithan, c. r. & swatman, p. m. c. 2001, ebanking adaptation and dot.com viability – a comparison of australian and indian experiences in the banking sector. proceedings of the 11 th bit 2001 conference, manchester, uk. venkatesh, v. & davis, f. d. 2000, a theoretical extension of the technology acceptance model: four longitudinal studies. management science, 46 (2), 186-204. venkatesh, v., morris, m. g., davis, g. b. & davis, f. d. 2003, user acceptance of information technology: toward a unified view. mis quarterly, 27 (3), 425-478. walker, r. h. & johnson, c. w. 2006, why consumers use and do not use technology-enabled services. the journal of services marketing, 20 (2), 125-135. walker, r. h., craig-lees, m., hecker, r. & francis, h. 2002, technology-enabled service delivery: an investigation of reasons affecting customer adoption and rejection. international journal of service industry management, 13 (1), 91-106. wilcox, r. r. 1997, some practical reasons for reconsidering the kolmorogov-smirnov test. british journal of mathematical and statistical psychology, 50(1), 9-20. wresch, w. & fraser, s. 2006, managerial strategies used to overcome technological hurdles: a review of e-commerce efforts used by innovative carribean managers. journal of global information management, 14 (3), 1-16. yang, z. & peterson, r. t. 2004, customer perceived value, satisfaction, and loyalty: the role of switching costs. psychology and marketing, 21 (10), 799-822. ye, c., seo, d. b., desouza, k. c., sangareddy, s. p. & jha, s. 2008, influence of information technology substitutes and user experience on post-adoption user switching: an empirical australasian journal of information systems volume 18 number1 2013 31 investigation. journal of the american society for information science and technology, 59 (13), 2116-2132. yousafzai, s. y., pallister, j. g. & foxall, g. r. 2003, a proposed model of e-trust for electronic banking. technovation, 23, 847-860. yousafzai, s. y., pallister, j. g. & foxall, g. r. 2005, strategies for building and communicating trust in electronic banking: a field experiment. psychology and marketing, 22 (2), 181-201. yu, j., ha, i., choi, m. & rho, j. 2005, extending the tam for a t-commerce. information and management, 42 (7), 965-976. zeithaml, v. a. 1988, consumer perceptions of price, quality, and value: a means-end model and synthesis of evidence. journal of marketing, 52 (7), 2-22. zeithaml, v. a., parasuraman, a. & malhotra, a. 2002, service quality delivery through web sites: a critical review of extant knowledge. journal of academy of marketing science, 30 (4), 362375. microsoft word ajis_18(3)_v10.docx australasian journal of information systems volume 18 number 3 2014 97 a knowledge management model to improve the development of bushfire communication products keith koon teng toh rmit university keith.toh@rmit.edu.au brian corbitt rmit university jenine beekhuyzen griffith university abstract this paper brings together two bodies of literature around knowledge management (km) as enterprise integration (ei) and organisational ontology and epistemology as philosophy, in order to develop an extended km approach to the development of bushfire preparedness material in the australian context. knowledge management (km) in enterprise integration (ei) practice manifests as process-centric electronic document and records management solutions. knowledge creation and organisational epistemology is viewed as a social process, but this is often left unrepresented by km processes. the body of literature on km tends to focus on organisational functionality and organisational km that is based on ei ontology tends to be restricted by organisational functionality and process models. we argue that developing the km-model using subjectivist epistemology has a significant role in km and organisational studies for emergency and disaster agencies. as part of the bushfire cooperative research centre (crc) ‘effective communication and communities’ project, bushfire communication materials were collected from all australian states and territories and analysed using nvivo, representing a knowledge base. data sources including semi-structured interviews with bushfire agency staff and residents in bushfire-prone localities. interview data was analysed using thematic analysis, and emergent themes were represented using uml as a platform independent representation of the extended knowledge domain that is capable of representation in a digital space. this work unites organisational ontology, organisational epistemology and ei; the different manifestations of km. we theorise about how organisational epistemology itself forms as part of the knowledge, where currently there is a lacking of a satisfactory end-to-end framework. the km lifecycle, therefore, is extended by incorporating the social research processes as part of organisational epistemology to include external audiences through the themes of locality types, place and roles of individuals as volunteers and agency staff. the practical implications are that qualitative methods and toolsets can be included as part of km to improve the development and deployment of bushfire preparedness material. keywords: community engagement, organisational ontology, organisational epistemology, emergency organisations, knowledge management introduction in australia, effective response to emergency and disaster incidents involves multiple state agencies, as well as humanitarian organisations. in this context, there is a strong tradition of volunteerism from within the community for emergencies, where people from diverse backgrounds and professions commit their time and energy to join emergency services, and then train and respond in emergencies australasian journal of information systems volume 18 number 3 2014 98 situations when required. bushfire emergency arrangements between agencies are complex and differ in individual australian states. for instance, in victoria there are fire services such as the metropolitan fire services (mfb), whose members focus on structural incidents. in bushfire events, the bushfire fire emergency services1 are the lead agency. due to the regularity and ferocity of bushfires (and grassfires) in australia, fire emergency services spend considerable resources on developing information packs to guide and facilitate residents to prepare and take actions to survive. while recognising the inputs from other services2, in the case of bushfires, information packs are developed primarily by the fire services, as lead agencies. this paper focuses on the development of bushfire preparedness knowledge by the fire services that respond to bushfire incidents (identified in footnote 1). as the peak body for australasian fire, land management and emergency services, the australasian fire and emergency services council (afac) plays a significant role in facilitating collaboration. for example, afac facilitated key fire and emergency service personnel from all australian states and territories and this interaction resulted in the development of common descriptors and key messages for agencies and for the public, resulting in a new national catch phrase and campaign ‘prepare. act. survive’ (national bush re warnings taskforce, 2009). there are important lessons in how the fire services develop this preparedness knowledge as organisational learning, and in how this information is made explicit, and then received by the public. the public react in a variety of ways to the information, some residents take action to prepare. public responses are complex and mcneill et al. (2014) examined personality traits and decision-related factors that lead to a delay in decision making. sims and baumann (1983) pointed to the subjective link between awareness and behaviour change (pp. 184). mcgee (2005) studied social bonds within groups of people and stated that this did not significantly improve the uptake of mitigation measures (pp. 153). by contrast, prior and eriksen (2013) found that social cohesion expressed as ‘sense of community’ was important for meaningful action. the availability of information and the variability of public responses highlights the need for a bushfire km-model (km-model) for the development of preparedness information. the purpose of this paper is to bridge the different manifestations of km and discuss how organisational epistemology itself forms as part of the knowledge, where the current process centric approach lacks a satisfactory end-toend framework. background the fields of qualitative research and knowledge management (km) are often viewed as separate disciplines with different ontological roots. km is traditionally information technology in origin, still 1 fire emergency agencies are the new south wales rural fire services (nswrfs), western australia fire emergency services authority (fesa) of western australia, victorian country fire authority (cfa), tasmania fire services (tfs), australian capital territory rural fire service (actrfs), northern territory fire and rescue service (ntfast), rural fire service queensland (rfsq). 2 these arrangements differ in each australian state, but in victoria, other support agencies are the state emergency service (ses), police and humanitarian organisations such as the australian red cross. australasian journal of information systems volume 18 number 3 2014 99 related to the computer-based system definition of alavi and leidner (2001). information technology (it) based km systems have been used since the early eighties (kamara et al., 2002). avgerou (2000) noted that the computer sciences and information systems research traditionally adopted a positivist epistemology, but interpretivist approaches have gradually become more accepted. bénel (2012) also stated that the km notion of ‘ontologies’ was not declared to be related with the philosophical concept, but this term had been co-opted as ‘jargon’. in view of this, the km approach of ontology development (verifiable and conclusive) lacked reflection upon creating, testing and revising, with no place for knowledge construction and refutation (bénel 2012). however, in this paper, we considered favourably bénel’s belief that even systems should also provide ‘digital spaces for debate between contradictory user-generated viewpoints’ and allow users of our systems to have the ‘hypothetical and intersubjective value of truth in mind’ (bénel 2012, p. 4). we noted that in the km context, ontology development has been restricted by a dominant focus on process models and artefacts (kanjanabootra et al. 2013). we adopt the position in this paper that km may be viewed as having a symbolic-interpretivist or constructivist ontology3. in taking this view, we examine the implications of organisational epistemology for emergency agencies by bridging different manifestations of km and discuss how the very choices of organisational epistemology, which relies on an organisation’s ability to be sufficiently self-reflexive, also form part of knowledge where currently there is a lacking of a satisfactory end-to-end framework. in the bushfire context, the knowledge boundary may be extended to incorporate the inter-subjective nature of knowledge. an approach to organisational epistemology has been proposed which shows how the concept of shared, or constructed, knowledge can be ‘deconstructed’ (as themes of people, locality, place, processes etc.) and applied to the development and deployment of bushfire safety publications. these themes may be subsequently enacted as part of a km model as digital spaces for knowledge capture. methodology as part of the bushfire cooperative research centre (crc) ‘effective communication and communities’ project, secondary data collection for the communication materials took place first in 2010 for the state of victoria. this was recorded and catalogued, and then work commenced gathering secondary data from other australian states and territories. one agency referred to their website as the source of all their communication material, which could be downloaded by the general public. data on bushfire preparedness was also collected via an environmental scan which identified material from other organisations, such as the australian red cross. between july and august 2011, this environmental scan was conducted using the websites of the lead bushfire agencies. searches of websites were also carried out to identify other sources (such as banks, newspapers, schools etc.). some of the secondary data collected was shown as examples during the face to face interviews and related fieldwork. all primary and secondary data was coded and queried using nvivo. to better understand how bushfire agencies communicated with the public, we conducted semistructured interviews with bushfire agency staff and residents in bushfire-prone localities (see table 1). key stakeholders were identified for interviews from new south wales, western australia, victoria and tasmania. at state level, we focused on key personnel, e.g. the people at cfa or tfs headquarters 3 for a definition of symbolic interpretative interpretivism please see (schwandt, 1994). australasian journal of information systems volume 18 number 3 2014 100 who had community or communication engagement responsibilities. participants were selected from a number of different locality types, namely urban-rural and holiday-maker. participants were also selected on the basis of being at state level, local government (councils) and then residents in the different locality types. when targeting different locality types, we identified a locality as urbanrural/holiday maker. at the local council level, we identified those in charge of enforcing local regulation. the residents interviewed included holiday makers, permanent residents, volunteers etc. in each of these locality types. we analysed the interview data, the research notes and the secondary data in combination to identify patterns/themes through thematic analysis (braun & clarke 2006), an approach compatible with constructivist ontology. these themes represented meaning within the data in relation to knowledge and development of bushfire communication material. the emergent themes were then represented as uml patterns using class diagrams in order to examine and represent links between themes and to develop the km model that was platform-independent. participant australian state m/f age education role crcvicst04 victoria f tertiary community development crcvicst03 victoria f manager, community development crcvicst08 victoria m commander/manager crcnswst03 new south wales m 35-44 post-graduate community safety crctasst03 tasmania f 55-64 tertiary community development crcvicang01 victoria f 35-44 post-graduate community development coordinator crcvicang05 victoria m 55-64 secondary schools presenter, volunteer, manager crcwadun05 western australia f 55-64 post-graduate volunteer, resident crcwabed13 victoria f 35-44 tertiary resident crcecpkor02 victoria f 55-64 tertiary resident crcvicang15 victoria f 55-64 tertiary resident table 1 interviewee demographics the knowledge management context km, as a practice, has been developed as a strategic tool, separating information and knowledge (hackett, 2000; lee & yang, 2000; lin, 2007; lejeune, 2011; schiuma, 2012). perhaps to confuse matters with social science discourse, computer-based knowledge bases, data models or schemas are also referred to as ontologies (farquhar et al., 1997; holsapple and joshi, 2003; maedche et al., 2003; noy and mcguinness, 2001; o'leary, 1998). madhavan and grover (1998) view knowledge creation as a social and cognitive team process and it is agreed that a number of phases such as knowledge identification, capture, storage and sharing are required (boh, 2007, bartholomew, 2009, booth, 2010). respective factors that influenced behaviours in efficiently sharing of organisational knowledge was highlighted by nesan (2012) who identified that trust, creativity, motivation, ability, and learning influenced employees’ knowledge sharing behaviour. within this context, knowledge is highly individualised and has been described as being socially australasian journal of information systems volume 18 number 3 2014 101 constructed (berger & luckmann, 1991). rubenstein-montano et al. (2001) argued that culture, business strategies and the technological infrastructure are part of a km framework. knowledge comprises experiences of people and is constructed within different contexts and with personal interpretation and reflection (polanyi, 1962; von krogh et al., 2000). meyer and rowan (1977) and mcdermott (2000) argue that knowledge is illusive and cannot be delivered solely by information technology. knowledge is scattered, and often unstructured, with tacit knowledge existing within peoples' heads, or it may be augmented or shared via interpersonal interaction and social relationships (nwilo & osanwuta, 2003; fergus et al. 2003). knowledge is recognised as a corporate asset and treated as with tangible assets (davenport & prusak, 1998; alavi & leidner, 2001; bartholomew, 2009). therefore, qualitative research provides the tools, methods and approaches to study and in some way capture this phenomenon (creswell, 2007). organisational ontology and epistemology relate to assumptions about truth and how knowledge may be accessed, captured, codified and re-used (lejeune, 2011; nonaka, 1994; schwartz, 2006), thus highlighting the dependent relationships between km and qualitative research. social methods are also well suited to the study of tacit knowledge in organisations as is evidenced in the works of von krogh et al. (1994), nonaka (1994), nonaka et al. (2000a), nonaka et al. (2000b), nonaka and toyama (2003), nonaka et al. (2006), von krogh et al. (2012), who have intrinsically linked km to organisational epistemology. organisational epistemology is the study of ways of knowing in the organisation and the process of codifying and amplifying knowledge created by individuals (nonaka, 1994; nonaka et al., 2006). filemon and uriarte (2008) included people's skills, competencies, ideas, intuitions, commitments and motivations. these attributes of people in organisations can be captured somewhat through interviews which facilitate access to the ‘knowing’ – tacit knowledge within organisations. there are implications on how knowledge is organised and the value of knowledge taxonomies (alavi & leidnerm, 2001). holsapple and singh (2012) underpinned the knowledge structure for electronic commerce with a definitional taxonomy including descriptive, procedural and reasoning knowledge. kakabadse et al. (2003) also discuss the incorporation of business processes, information technologies, knowledge repositories and individual behaviours. organisational knowledge domain similar to the product design workflows of catalano et al. (2008), the consideration of km can be essential to how bushfire safety material is developed. where the current process centric approach lacks a satisfactory end-to-end framework, this paper will now bridge the different manifestations of km and discuss how organisational epistemology itself forms as part of the knowledge. providing the ‘digital space’ for the inclusion of the public audience as part of organisational epistemology are key aspects of this work. two ideological fields exist, where organisational epistemology and ontology remain within knowledge philosophy and enterprise integration (ei) within the commercial domain. while organisational ontology remains in research, it is not well supported either by toolsets as enterprise integration. the development of toolsets such as nvivo (qsr international, 2013) and protégé (stanford university, 2009) may also be an indication of how the two fields remain separate, where nvivo caters to the qualitative research community and protégé for development of web ontology language (owl) compliant ontologies. if organisational ontology and organisational epistemology are considered to be integral concepts to km, then existing integration systems that only consider ontology as an end-result (excluding the intersubjective nature of knowledge creation) may yet fall short in support of the knowledge lifecycle. australasian journal of information systems volume 18 number 3 2014 102 the interviewing process is one of the many methods used in social research (kvale & brinkmann, 2009; ambrosini & bowman, 2001; desouza, 2003; frey and fontana, 1991; layder, 1998) and may be applied in support of organisational epistemology. nvivo allows researchers make sense out of “raw” semi-structured data, providing memo links, coding stripes and models, and arguably, its use in research is a form of km. constructs of nvivo such as source, nodes etc. are also different to the ontology of classes, instances and objects, which underpins the syntax of the web ontology language (owl). we propose a unifying framework for organisational knowledge which has been developed based on the research literature (figure 1). the conceptual framework proposed comprises three components. there is an overarching component of organisational ontology and two other components of organisational epistemology and enterprise integration (ei), where the latter refers to the involvement of the technology platforms that fulfil the role and functions of information integration as used by chen et al. (2008). figure 1. a unifying framework for organisational knowledge ei relates to technology and knowledge meta-theories relate to the meta-theory that underpin our assumptions about knowledge. this distinction may also be seen in the works of (davenport et al., 1998) and (saito et al., 2007). with ei, ‘ontology’ tends to be based upon enterprise functionality, in the form of service or process models and (event based) execution (bussler, 1996; scheer & nüttgens, 2000; chandrasekaran et al. 1999). therefore, this model of unifying framework (figure 1) affords a conceptualisation of the proposed km architecture. organisational ontology and epistemology an ontological position determines data collection and analysis and interpretation. in taking an interpretive view, knowledge is seen as socially constructed and this ontological position determines how data is collected and interpreted, such as through interviews, observations and document analysis. the way we use qualitative research interviews, seeks to describe meanings of central themes in the lives people and to understand how meaning is determined (kvale, 2006; britten, 1995; opdenakker, 2006; kvale & brinkmann, 2009). accepting fisher’s concept of knowledge as a form of social capital (fisher, 2013), we argue that representing organisational knowledge as solely functional models leads to a highly constrained view, which does not provide the inter-subjective space as discussed by bénel (2012). it is highly constrained because there exists richer, complex and sometimes subjective content to be accessed through qualitative methods. australasian journal of information systems volume 18 number 3 2014 103 in interviews, audio recordings provide a representation of what was said, but much of the conversation in terms of highly valuable contextual and personal information is often not captured, or forgotten in fields notes that many researchers are unsure what to do with (layder, 1998, pp. 59). after recording and transcribing the interviews, it is necessary for the interviewer to capture observations such as the environment, or attitude/persona of the interviewee (memon & higham, 1999) and possibly even mannerisms communicated through body language and sounds such as laughter or long pauses in response (creswell, 2007). summaries and additional contextual information captured brings a new dimension to the data, and the analyst one step closer to the data than is possible with a typed verbatim transcript alone (silverman, 2006). enterprise integration (ei) technologies assist in the process of data collection, storage, analysis and publication. examples of integration platforms are ibm’s websphere mq and other web-service based solutions to integrate multiple software applications so that they work together and exchange information (bieberstein et al., 2005). ei applications that focus on explicit and structured functional knowledge, such as document management or enterprise resources management, often do not focus on tacit knowledge or context. therefore we propose that ei ontologies are narrowly focused on a functional approach. the functional approach of enterprise integration (ei) ontologies organisational ontology is both complex and extensive, and ontologies developed in the fields of enterprise modelling tend to view business operations from a perspective of functionality and information integration (davenport & short, 2003). business functionality has also been described in terms of contracted services between an organisation and its employees or customers (weske, 2012; wegmann et al., 2008). ei results in a focus on business where ontology comprises processes and information schemas and ei epistemology is based on using modelling languages that record as many facts as possible from data models, function trees and organisation views. therefore, process and information models form the constructs of many enterprise modelling architectures (vernadat, 2002). the cimosa4 ontology captures enterprise operations using function, information, resource and organisation viewpoints (amice, 1991). the tove (toronto virtual enterprise) ontology (fox et al, 1996; grüninger et al., 2000) is based around defining semantics to enable data sharing. the architecture of integrated information systems (aris) ontology comprises function, organisation and resources (scheer & nüttgens, 2000). we build on these ontologies and reference models to develop a conceptual architecture of the organisational knowledge domain. developing the organisational knowledge domain people draw on knowledge to make sense of situations, to communicate, and to act. organisational knowledge is complex (o’dell & grayson, 1998; ardichvili et al., 2003) and we have argued so far that organisational ontology is far more extensive than just knowledge that is distilled as process and business process models. developing the organisational knowledge domain (see figure 1) and mapping of toolsets demonstrates how the existing km framework may be extended. the notion of organisational knowledge creation locates knowledge where humans interact, which can be considered as a cyclic process (blackler, 1995; nonaka, 1994; demarest, 1997; alvesson & 4 computer integrated manufacturing – open system architecture australasian journal of information systems volume 18 number 3 2014 104 kärreman, 2001). this occurs at different levels of the organisation (inkpen & dinur, 1998). fisher (2013) discussed the concept of social capital as several levels through which knowledge is acquired and transmitted between individuals and government organisations. in the bushfire context, organisational epistemology involves networks and interactions of people and this view of the knowledge domain is shown using the uml use-case notation (figure 2). creative processes in agencies lead to the production of the preparedness information and knowledge is generated through interactions with the public when materials are handed out. key to the knowledge domain is the ‘includes’ relationship which elements such as ‘public’ is represented by an aggregation of agency staff, volunteers, statutory officers and even researchers. figure 2. mapping of individuals involved in the knowledge lifecycle a ‘visualisation’ of organisational knowledge has offered a step beyond process centric (or reductionist) abstractions and organisational km needs go beyond codifying knowledge as workflows and as technical platforms. qualitative research practices should be drawn upon, as organisational epistemology, to ‘unpack’ the meanings of themes. thematic analysis guides how concepts may be interpreted and handled; this is discussed extensively by braun and clarke (2006), and ryan and bernard (2003). the two domains of km may also be illustrated through examples of their different toolsets adopted. in ei, organisational ontology may be defined in the unified modelling language (rumbaughet al. 1999) or in the object web language (owl) through applications such as protégé. in social research, nvivo is a software application that supports the processes of qualitative raw data handling (interview texts, audios etc.), coding and thematic analysis. although nvivo does not produce an ontology that is owl compliant, it features greater recognition of organisational epistemology through its support of social research. therefore we argue that socially-oriented processes that lead to the construction and interpretation and emergence of knowledge themes are not well served by contemporary ei ‘ontology’ solutions alone. the proposed organisational knowledge domain (figure 3) provides a place for qualitative approaches to capture richer, complex and sometimes subjective content in km practice. australasian journal of information systems volume 18 number 3 2014 105 figure 3. providing for richer, complex and sometimes subjective content in km practice australian bushfire communications: a case study in australia, the state emergency service (ses) provides preparation and urgent community assistance for storms and floods. for bushfires, public preparedness is underpinned by how the fire services, as lead agencies, develop information packs and facilitate residents to take action. as alluded to earlier, it has been reported that the public generally ignores the abundant amount of preparedness information available (rhodes et al., 2011; australia parliament, 1984, pp. 27) and there are lessons to be learnt from how safety publications are designed and disseminated. ordoñez and serrat (2009) argue that widespread dissemination is ‘a core responsibility of any organization tasked with generating and sharing knowledge products, especially of new kinds of unique (and uniquely valuable) content that are as usable and accessible as possible’. the need to produce and disseminate quality information is also recognised along with cultivating community knowledge and public engagement (o’dubhchair et al., 2001). mcdermott (2000 pp. 28) argued that ‘we need to focus on the community that owns it [knowledge] and the people who use it, not the knowledge itself’. o’dubhchair et al., (2001) argued that communities must develop capacities for learning by interfacing with local and regional agencies. this need to improve design and dissemination may be viewed as a km problem. whereas the main body of literature on internal km tends to focus on organisational knowledge (nonaka et al., 2006; dalkir, 2013), this work examines how external audiences, whom mcdermott (2000) described as users of the knowledge, may be included as part of organisational epistemology, in the development of preparedness material. the view of organisational knowledge proposed in figure 4 posits the shared conceptual models in the form of enterprise ontologies. this was developed with the ideas of maedche et al. (2003) in mind, emphasising the importance of integrating knowledge across databases, file systems, as well as providing the capacity to capture tacit knowledge held by individuals. therefore, the km architecture distinguishes both process models (as ei ontology) and knowledge models which extends towards tacit content, in our attempt to provide the digital spaces for the ‘hypothetical and inter-subjective value of truth in mind’ which bénel (2012, p. 4) called for. in figure 4, subjective knowledge is denoted as ‘knowledge models’, which is differentiated from ‘process models’. australasian journal of information systems volume 18 number 3 2014 106 a digitised collection of bushfire preparedness material, along with interview data, was organised and analysed in nvivo. the role of nvivo is located as the ‘knowledge information’ component of the km architecture, shown previously (figure 3), and the meta-model is captured in uml classes, documenting the knowledge structures that are relevant to the development of bushfire preparedness material. nvivo supported in-depth data analysis, by providing the ability to code images, audio and video sources, thereby reflecting complicated knowledge than solely plain text. figure 4. positioning knowledge models in km architecture the km architecture (figure 4) was initially developed out of underlying issues of document and version control. organisational boundaries later were identified when bushfire preparedness material was collected and the roles and activities of those responsible for knowledge in emergency organisations were explored. nvivo supported this process through its ability to manage a range of data including interviews and communication materials, as well as coding of bushfire safety themes as they emerged. the schema of the km-model (figure 5) gives an overview of the themes including people, roles, processes, locality, place and dissemination experiences. the schema now goes beyond the reductionism of process modelling. although functional representations as ‘processes’ are a component of knowledge, there are other concepts that develop further insights into knowledge, such as with individuals and their roles, the significance of locality and its link to risk, the meaning of place, the development and dissemination of key messages and originating agencies. figure 5 provides an overview of the themes and also indicates that these themes are related in particular ways as shown later using the uml class notation. australasian journal of information systems volume 18 number 3 2014 107 figure 5. the bushfire km-model in a more formal format, the schema of the bushfire km-model is presented using uml class diagram which enables integration between ‘hard’ ei functional concepts and ‘soft’ intersubjective themes. schema of the bushfire preparedness knowledge model the schema of the km-model is a layout of themes that were identified from the analysis of the communication material and interviews conducted. the model was also developed to accommodate processes and knowledge of bushfire agencies, as well as the knowledge contributed by residents, the combination of which captures aspects of community knowledge. themes, such as products, processes, roles and persons offer elements against which knowledge may be embedded. the following sections describe how uml classes are used as the ‘containers’ for instances of tacit knowledge. capturing tacit knowledge knowledge is represented in bushfire education material and is also inherent in social processes and interactions. there are examples where knowledge is exchanged or constructed through informal interactions between agency personnel, or formally facilitated through afac. there is a tradition of volunteering in australia and fire emergency agencies also comprise volunteer personnel who provide the vital capability to deal with emergencies. the boundary of knowledge is ‘elastic’ and external knowledge is implicitly involved because agencies comprise employed staff and volunteers. knowledge is exchanged as agencies distribute the material and it is inclusive of wider segments of the community. davenport and prusak (1998) and mcdermott (2000) argued that knowledge is embedded not only in documents or repositories, but in organisational routines, processes, practices and organisational norms. an example of how organisational knowledge may be codified as business processes may be found in (johannsen & fill, 2014). building upon the work of o'dubhchair, scott and johnson (2001), the kmmodel focuses on a different context, on community. this focus on community is achieved through the themes of agency, service, roles, person, process, products, place and locality. importantly, the kmmodel provided a means for managing the accumulated knowledge, to provide a tool for easier accounting for an agency’s knowledge assets and to support ways through which agencies may better target different segments of people. the focus was on how concepts of processes and people intersect as knowledge products are developed and distributed. australasian journal of information systems volume 18 number 3 2014 108 the km-model (figure 6) was developed in uml to reflect themes that emerged from analysis of key agency personnel and residents in different localities. knowledge is the combination of agency processes and knowledge contributed by residents which capture aspects of ‘community memory’. elements in the km-model underpin knowledge associated with products, processes, roles and persons. figure 6. the uml schema of the bushfire-km model originating agency the originating agency identified the source of the knowledge product. there is a ‘lead agency’ which depends on the nature of the emergency, but the actual situation is much more involved. ‘lead agencies’ in the context of bushfire/grassfire emergencies were the cfa, rfs, fesa, act rfs, etc. other agencies were the office of the premier of victoria, department of health, fire services commissioner, individual councils. therefore, the element ‘originating agency’ identifies the particular agency associated with a particular knowledge product. australasian journal of information systems volume 18 number 3 2014 109 agency ‘we have multiple agencies wanting to engage communities about fire at different times.... so we've been working with red cross this year to make sure that all fire planning's consistent... not selling two different kinds of planning approaches.’ crcvicst04 services ‘services’ are often described through agency websites as ‘what we do’. as an example, one agency provided wildfire suppression, structural fire suppression, and providing community awareness and education. these services may be formally listed as a catalogue that encapsulates service level agreements. the class ‘service’ caters to the themes of organisational changes or reconfiguration to focus on public engagement. service ‘[brigades] have seen some progress within their community and they are more willing to take and wanting to be resourced and trained to be able to do this [community education] within their communities. whereas when i first met with them, they kind of were, no we’re just operational, we don’t do that sort of stuff.’ crctasstate03 ‘…..a lot of the fire officers, they join thinking their job is to go out and put wet stuff on the red stuff, it's not. they spend 5 or 10 per cent of their time doing that. the other 95 per cent of their time, they've got to be out there in the community. to be talking up to prevent fires...’ crcvicst08 organisational processes organisational processes may be classified as operational, tactical and strategic processes and correspond with local, regional and state levels, where local operational processes are executed for example, by individual brigades. process knowledge is captured at a strategic level where one interviewee described the need to effectively manage the development of messages. there were also examples of informal collaboration processes, but knowledge sharing between agencies is facilitated by afac in the development of key messages. sub-processes were often identified such as consultation, public engagement, interactive forums and other forms of dissemination such as mass mail-outs. process ‘…..we like to joint badge our media releases for sure, and particularly our media campaigns and our launches. anytime we do a media launch we involve the minister and get that… everything's all in synergy that way….it's almost a case of let's go to a neutral corner….’ crcvicst08 the theme ‘campaign’ emerged where some interviewees related ‘campaign’ to a number of concepts, including development of key messages, public consultation and dissemination. ‘campaign’ was an abstract theme, such how interviewees described television and radio campaigns. as an overarching australasian journal of information systems volume 18 number 3 2014 110 element it contained attributes, such as campaign slogan. examples of campaign slogans were ‘fire ready’ or ‘prepareact-survive’. campaign ‘….we have a number of different levels of communication. they would involve media and as a broad range of call to action…then we have more interaction with communities through public meetings that we would run either in community halls…on street corners….’ crcvicst03 people and roles analysis of the fire emergency domain identified key personnel (modelled as persons) as those who have tacit knowledge. tacit knowledge can be captured through interviews with the key or domain experts and conceptually this was linked through roles. this theme captured individual roles, where interviewees may reveal they also have roles as volunteer fire-fighters, separate to their occupation. agency members act in different role(s), where some are subject matter experts, responsible for production and of the preparedness information, who also act as volunteers in their own local brigades. often, volunteer personnel performed communication roles external to the organisational boundaries through on personal networks. the public or residents also play a role during the processes of public consultation. the need for a qualitative interviewer to capture observations such as the environment, or attitude/persona of the interviewee and mannerisms was identified earlier (e.g. memon & higham, 1999; creswell, 2007). in relation to roles and environment, an interview was arranged with a local councillor, also a police officer. the researcher noted that the location was conducted at a police station (classified as place of work) and research field notes reflected the interviewee’s stilted and guarded responses that seemed to stem from the role of a police officer. such research notes are used in analysis and may inform readers of the researcher’s observations. the following illustrates another example of how roles may be described. person, role ‘i'm a mum, granny, teacher and also a member of the local fire brigade and a number of environmental groups ….in addition to being a regular volunteer bushfire firefighter, i'm what's called a facilitator of an organisation called bushfire ready…..i'm also caring for my elderly, retired, farmer uncle in my house…at home, i do i guess the best description is technical writing.’ crcwadun05 place and locality there was significance in the way in which interviewees spoke of place and locality. some interviewees spoke of places as a distinct concept to locality, for example, referring to a fire station, or to a meeting place, a place of work or a place of worship. the distinction between place and locality is also found in the work of day and murdoch (1993). the fire station may be mentioned as the only focus place for a small locality, or absence of a local shopping facility. the abstract class ‘place’ caters to this concept and other specific instance of sub-classes can be created, such as ‘station’, ‘headquarters’, or ‘council office’. australasian journal of information systems volume 18 number 3 2014 111 place ‘there's not a shopping centre as such. so there's not this bringing together of the population where you're going to buy your litre of milk and you see someone else getting their morning paper and you say g'day and this sort of stuff’. crcwabed13 closely linked to the concept of place, was locality, which related to spatial boundary such as a town or a suburb. in relation to this study, some interviewees described characteristics of living in a holiday maker locality. knowledge associated with this class may be related to peri-urban, sea change, and rural localities. the targeted strategies are developed for at-risk localities and knowledge segments give insight into barriers against acceptance and implementation of strategies that may have an impact upon local brigades. locality ‘so we're not just delivering a script that's developed in melbourne that might not be that interesting to some communities…. or how do we shape that [engagement strategy] so that we're more able to deliver it in a way that works for local communities, rather than just meeting targets.’ crcvicang01 risk profile associated with locality, is the component of risk, which is related to the locality’s risk profile. an example of how this knowledge may impact the relevance of bushfire communication was illustrated by one interviewee who said that residents largely ignored communication because of the perceived risk profile of the locality and a lack of consensus on terms used in communication. risk profile ‘i think that terminology bushfire awareness, too, is a bit of a misnomer for people. they just don’t think that actually them, because we're all grassfire area, and slightly residential, as well. they'd be more concerned of...a factory explosion fire…. so, they're don't think bushfire, that name bushfire, they'll think, oh that's not us.’ crcecpkor02 dissemination the value of the knowledge relies heavily on its effective dissemination to the target audience. the dissemination class denotes the processes of delivery, whether this involves the engagement of individuals or through television or radio campaigns. an effective dissemination strategy will continue to evolve and the segments of knowledge relating to the utilisation of the product can include reasons why particular pathways fail. australasian journal of information systems volume 18 number 3 2014 112 dissemination, pathway ‘... they’ve bombarded people with too much…these kits that they sent out to every house and admittedly some houses got four to five. every person that was an adult in that house got one. but there were four or five different books…..and people were scratching their heads……….’ crcvicang05 ‘i think just from handing this out and speaking with people, this is a document that needs to be explained....rather than just hand it to them…..you try to explain the different things that could happen…..some will change their mind by the end of the conversation.’ crcnswst03 ‘if you stand there with a handful of brochures, some balloons on a string and while people are walking past, you say, would you like a balloon, hello little child, whatever, they stop, they talk and before you know it, you’re having really meaningful conversations. we could engage in 200 in-depth conversations in a day and come home exhausted, or you can sit back on your chair and wait for people to come to you and have 20 conversations’ crcvicang 15 discussion the use of ‘ontology’ is contested from the perspectives of two different disciplines. bénel (2012) stated that ontology had been adopted ‘jargon’ in the computer sciences, and we also noted this use in contemporary ei and km applications. with this point in mind, we approached km from two perspectives; firstly we critically examined km theory and secondly the implications of organisational epistemology as a philosophy on extending the km lifecycle and framework. the review of literature showed that contemporary ei and km practice has been restricted by an epistemology that focuses on process models and process integration which can be reductionist. an approach to organisational epistemology has been proposed which accepts that knowledge can be intersubjective. this paper has presented a knowledge domain which develops ‘soft’ km as separate to ‘hard’ process integration (ei) and sought to develop the ‘digital spaces’ which bénel (2012, p. 4) alluded to. the extension of the km cycle to include qualitative approaches was illustrated by the mapping of a qualitative toolset such as nvivo, within the knowledge management domain. this mapping differentiated ‘knowledge models’ from ‘process models’, but showed how an organisation’s ability to be self-reflexive (e.g. triple-loop learning) may also be captured and integrated as part of the km lifecycle. the concept of knowledge models was applied to the case of bushfire safety materials for australian bushfires, where we develop a km-model was developed to address how themes of community knowledge can be applied to the development and deployment of bushfire safety publications. we found the factors influencing the public’s reaction to bushfire preparedness are complex, but we were influenced by ordoñez and serrat (2009), carlsson (2004) and alavi and leidner (2001) who stated that capacities for learning and action depends on people being genuinely interested in participating in a continuous learning and knowledge sharing process. knowledge around such capacities for learning will not be captured using process-centric ei approaches. the need to capture external sources was also alluded to by quintas et al. (1997), who argued that part of the aim was ‘supporting the acquisition of australasian journal of information systems volume 18 number 3 2014 113 knowledge from external sources, and developing the capability to assimilate and utilize that knowledge; to ensure that new knowledge is distributed to those who require it’. the km-model developed in this paper attempts to extended the boundary of knowledge wider to encapsulate public audiences through the themes of locality, place and different locality types. the focus was widened beyond organisational processes that produce the knowledge material and includes the view of the agency personnel as they use the preparedness material in their public engagement. the km-model also caters to capturing how recipients perceive the mode of delivery at the agency-public interface. qualitative research provided methods for data collection, analysis and presentation of information provided the means to extend the reductionist process-centric view of ei ontology to include subjective interpretations of bushfire themes. to define this ‘digital space’, the themes of people, locality, place, processes, or dissemination strategies were represented in uml. the km-model bridges themes and provides a step towards further translation into other km formalisms such as owl. further work further work should be carried out to extend the km approach across multiple agencies and multiple hazards and agencies in australia and even internationally, to include flood, storms, tropical cyclones and pandemics. oliveira et al. (2002) discussed the evolutionary process of km as initially document management systems. maturity models define a number of dimensions or process areas at several discrete stages or levels of maturity, with a description of performance at various levels of granularity (ahern et al., 2003, paulk et al., 1993). the idea of an organisational ‘maturity model’ for km has been presented for a number of industries including construction (robinson et al., 2006). further work should develop the concepts of organisational capability maturity and potentially adopt km capability frameworks for application to bushfire emergency organisations acknowledgements the authors gratefully acknowledge the funding of the bushfire crc. we are also grateful to the various bushfire agencies, particularly to damien killalea of the tasmanian fire services and john schauble of the office of the fire services commissioner, victoria, who facilitated access to bushfire preparedness material. we would also like to acknowledge the work and conversations with researchers of the centre of sustainable organisations and work (csow) and school of business it and logistics of rmit, and across other universities who all contribute to this valuable cross-disciplinary nature of research. references alavi, m. and leidner, d.e. (2001), “review: km and km systems: conceptual foundations and research issues”, mis quarterly, pp. 107-136. alvesson, m. and kärreman, d. (2001), “odd couple: making sense of the curious concept of km”, journal of management studies, vol. 38 no. 7, pp. 995-1018. ambrosini, v. and bowman, c. (2001), “tacit knowledge: some suggestions for operationalization”. journal of management studies, vol. 38 no. 6, pp. 811-829. amice, (1991), cimosa: open system architecture for cim, 2nd revised and extended edition, springer, london. ardichvili, a., page, v. and wentling, t. (2003), “motivation and barriers to participation in virtual knowledge-sharing communities of practice”, journal of km, vol. 7 no. 1, pp. 64-77. australasian journal of information systems volume 18 number 3 2014 114 australia parliament, house of representatives, standing committee on environment and conservation and milton, p. (1984), bushfires and the australian environment, australian government publishing service, canberra. avgerou, c. (2000), “information systems: what sort of science is it?”, omega – the international journal of management science, vol. 28, no. 5, pp. 567-579. bartholomew, d. (2009), building on knowledge: developing expertise, creativity and intellectual capital in the construction professions, wiley-blackwell. bénel, a. (2012), where do ‘ontologies’ come from? seeking for the missing link, in 3rd international symposium on the web and philosophy (philoweb), april 17th, 2012, lyon. berger, p.l. and luckmann, t. (1991), the social construction of reality: a treatise in the sociology of knowledge, penguin, london. bieberstein, n., bose, s., walker, l., and lynch, a. (2005), “impact of service-oriented architecture on enterprise systems, organizational structures, and individuals”, ibm systems journal, vol. 44 no. 4, pp. 691-708. blackler, f. (1995), “knowledge work and organizations: an overview and interpretation”, organization studies, vol. 16 no. 6, pp. 1021-1046. boh, w.f. (2007), “mechanisms for sharing knowledge in project-based organizations”, information and organization, vol. 17 no. 1, pp. 27-58. booth, c. (2010), “km for electric power utility companies”, in metaxiotis k. (ed.), intelligent information systems and km for energy: applications for decision support, usage, and environmental protection, igi global, new york. braun, v. and clarke, v. (2006), “using thematic analysis in psychology”, qualitative research in psychology, vol. 3 no. 2, pp. 77-101. britten, n. (1995), “qualitative interviews in medical research”, british medical journal, vol. 311 no. 6999, pp. 251-253. bussler, c. (1996), “specifying enterprise processes with workflow modelling languages”, concurrent engineering, vol. 4 no. 3, pp. 261-278. carlsson, s.a., (2004), “enhancing product development through ict based relationships with customers and consumers”, in proceedings of the european conference on information systems (ecis), paper 45. catalano, c.e., camossi, e., ferrandes, r., cheutet, v. and sevilmis, n., (2008), “a product design ontology for enhancing shape processing in design workflows”, journal of intelligent manufacturing, vol. 20 no. 5, pp. 553-67. chandrasekaran, b., josephson, j.r. and benjamins, v.r. (1999), “what are ontologies, and why do we need them?”, intelligent systems and their applications, ieee, vol. 14 no. 1, pp. 20-26. chen, d., doumeingts, g. and vernadat, f. (2008), “architectures for enterprise integration and interoperability: past, present and future”, computers in industry, vol. 59 no. 7, pp. 647-659. cook, s.d., and brown, j.s. (1999), “bridging epistemologies: the generative dance between organizational knowledge and organizational knowing”, organization science, vol. 10 no. 4, pp. 381-400. creswell, j.w. (2007), qualitative inquiry and research design: choosing among five approaches, sage publications, thousand oaks ca. dalkir, k. (2013), km in theory and practice, routledge, chicago. day, g. and murdoch, j. (1993), “locality and community: coming to terms with place”, the sociological review, vol. 41 no. 1, pp. 82-111. australasian journal of information systems volume 18 number 3 2014 115 davenport, t. and prusak, l. (1998), working knowledge: how organizations manage what they know, harvard business school press, boston. davenport, t.h. and short, j.e. (1990), “the new industrial engineering: information technology and business process redesign”, sloan management review, vol. 31 no. 4, pp. 1-31. davenport, t.h., david, w. and beers, m.c. (1998), “successful km projects”, sloan management review, vol. 39 no. 2, pp. 43-57. demarest, m. (1997), “understanding km”, long range planning, vol. 30 no. 3, pp. 374-384. desouza, k.c. (2003), “facilitating tacit knowledge exchange”, communications of the acm, vol. 46 no. 6, pp. 85-88. farquhar, a., fikes, r. and rice, j. (1997), “the ontolingua server: a tool for collaborative ontology construction”, international journal of human-computer studies, vol. 46 no. 6, pp. 707-727. fergus, p., mingkhwan, a., merabti, m. and hanneghan, m. (2003), “capturing tacit knowledge in p2p networks”, in 4th epsrc annual postgraduate symposium on the convergence of telecommunications, networking and broadcasting (pgnet2003), liverpool, uk. filemon a. and uriarte j., (2008), introduction to km, asean foundation, jakarta, indonesia. fisher, r. (2013), “a gentleman's handshake: the role of social capital and trust in transforming information into usable knowledge”, journal of rural studies, vol. 31, pp. 13-22. fox, m.s., barbuceanu, m., and gruninger, m. (1996), “an organisation ontology for enterprise modeling: preliminary concepts for linking structure and behaviour”, computers in industry, vol. 29 no. 1, pp. 123-134. frey, j.h. and fontana, a. (1991), “the group interview in social research”, the social science journal, vol. 28 no. 2, pp. 175-187. grüninger, m., atefi, k. and fox, m.s. (2000), “ontologies to support process integration in enterprise engineering”, computational and mathematical organization theory, vol. 6 no. 4, pp. 381394. hackett, b. (2000), beyond knowledge management: new ways to work and learn. the conference board inc., ny. holsapple, c.w. and singh, m. (2000), “electronic commerce: from a definitional taxonomy toward a knowledge-management view”, journal of organizational computing and electronic commerce, vol. 10 no. 3, pp. 149-170. holsapple, c.w. and joshi k.d. (2003), “a km ontology”, in holapple c.w. (ed.), handbook on km, vol. 1 – knowledge matters, pp. 89–128, springer-verlag, heidelberg. inkpen, a.c. and dinur, a. (1998), “km processes and international joint ventures”, organization science, vol. 9 no. 4, pp. 454-468. johannsen, f., and fill, h. g. (2014), codification of knowledge in business process improvement projects, in proceedings of the european conference on information systems (ecis), tel aviv, israel, june 9-11, 2014 kakabadse, n.k., kakabadse, a. and kouzmin, a. (2003), “reviewing the km literature: towards a taxonomy”, journal of km, vol. 7 no. 4, pp. 75-91. kamara, j.m., anumba, c.j. and carrillo, p.m. (2002), “a clever approach to selecting a km strategy”, international journal of project management, vol. 20 no. 3, pp. 205-211. kanjanabootra, s., corbitt, b. & nicholls, m. (2013), “evaluating knowledge management systems efficacy and effectiveness in a design science context”, journal of systems and information technology, vol. 15, no. 4, pp. 324-46 australasian journal of information systems volume 18 number 3 2014 116 kvale, s. and brinkmann, s. (2009), interviews: learning the craft of qualitative research interviewing, sage publications, incorporated. kvale, s. (2006), “dominance through interviews and dialogues”, qualitative inquiry, vol. 12 no. 3, pp. 480-500. layder, d. (1998), sociological practice: linking theory and social research, sage publications, incoporated. lee, c.c. and yang, j. (2000), “knowledge value chain”, journal of management development, vol. 19 no. 9, pp. 783-794. lejeune, m. (2011), “tacit knowledge: revisiting the epistemology of knowledge”, mcgill journal of education, vol. 46 no. 1, pp. 91-105. lin, h.f. (2007), “knowledge sharing and firm innovation capability: an empirical study”, international journal of manpower, vol. 28 no. 3/4, pp. 315-332. madhavan, r. and grover, r. (1998), “from embedded knowledge to embodied knowledge: new product development as km”, journal of marketing, vol. 62 no. 4, pp. 1-12. maedche, a., motik, b., stojanovic, l., studer, r. and volz, r. (2003), “ontologies for enterprise km”, intelligent systems, ieee, vol. 18 no. 2, pp. 26-33. mcdermott, r. (2000), “why information technology inspired but cannot deliver km”, knowledge and communities, vol. 41 no. 4, pp. 21-35. mcgee, t. k. (2005), “completion of recommended wui fire mitigation measures within urban households in edmonton, canada”, environmental hazards, vol. 6 no. 3, pp. 147-157 mcneill, i.m., dunlop, p.d., morrison, d.l., skinner, t.c. (2014), “predicting delay in residents’ decision on defending versus evacuating through antecedents of decision avoidance”, international journal of wildland fire. memon, a. and higham, p.a. (1999), “a review of the cognitive interview”, psychology, crime and law, vol. 5 no. 1-2, pp. 177-196. meyer, j.w. and rowan, b. (1977), “institutionalized organizations: formal structure as myth and ceremony”, american journal of sociology, pp. 340-363. national bush re warnings taskforce (2009), australia’s revised arrangements for bush re advice and alerts: 2009 –2010 fire season. nesan, j. (2012), “factors influencing tacit knowledge in construction”, australasian journal of construction economics and building, vol. 5 no. 1, pp. 48-57. nonaka, i. (1994), “a dynamic theory of organizational knowledge creation”, organization science, vol. 5 no. 1, pp. 14-37. nonaka i., von-krogh g. and voelpel s., (2006), “organizational knowledge creation theory: evolutionary paths and future advances”, organization studies, vol. 27, pp. 1179–1208. nonaka, i. and toyama, r. (2003), “the knowledge-creating theory revisited: knowledge creation as a synthesizing process”, km research and practice, vol. 1 no. 1, pp. 2-10. nonaka, i., toyama, r. and konno, n. (2000a), “seci, ba and leadership: a unified model of dynamic knowledge creation”, long range planning, vol. 33 no. 1, pp. 5-34. nonaka, i., toyama, r. and nagata, a. (2000b), “a firm as a knowledge-creating entity: a new perspective on the theory of the firm”, industrial and corporate change, vol. 9 no. 1, pp. 1-20. noy, n. and mcguinness, d. (2001), ontology development 101: a guide to creating your first ontology, technical report smi-2001-0880, stamford medical informatics. nwilo, p.c. and osanwuta, d.a. (2003), “knowledge sharing as a tool for professional development in surveying and geoinformatics in nigeria”, fig working week, april 2003, paris, pp. 13-17. australasian journal of information systems volume 18 number 3 2014 117 o’dell, c. and grayson, c.j. (1998), “if only we knew what we know”, california management review, vol. 40 no. 3, pp. 154-174. o'dubhchair, k., scott, j.k. and johnson, t.g. (2001), “building a knowledge infrastructure for learning communities”, the electronic journal of information systems in developing countries, no. 4, pp. 1-21 o'leary, d.e. (1998), “using ai in km: knowledge bases and ontologies”, ieee intelligent systems and their applications, vol. 13 no. 3, pp. 34–39. oliveira, m., caldeira, m. and batista romão, m.j. (2012), “km implementation: an evolutionary process in organizations”, knowledge and process management, vol. 19 no. 1, pp. 17-26. opdenakker, r. (2006), “advantages and disadvantages of four interview techniques in qualitative research”, forum: qualitative social research, vol. 7 no. 4, article 11. ordoñez, m. and serrat, o. (2009), disseminating knowledge products, asian development bank, washington, dc. polanyi, m. (1962), personal knowledge: towards a post-critical philosophy, university of chicago press, chicago, il. prior, t. and eriksen, c. (2013), “wildfire preparedness, community cohesion and social–ecological systems”, global environmental change, vol. 23 no. 6, pp. 1575-1586. qsr international (2013), nvivo 10, qsr international, available at http://www.qsrinternational.com (accessed november 2013). quintas, p., lefrere, p. and jones, g. (1997), “km: a strategic agenda”, long range planning, vol. 30 no. 3, pp. 385-391. rhodes, a., gilbert, j., nelsson, c., preece, e. (2011), evaluation report 2010-2011, cfa community safety directorate, melbourne, australia. robinson, h.s., anumba, c.j., carrillo, p.m. and al-ghassani, a.m. (2006), “steps: a km maturity roadmap for corporate sustainability”, business process management journal, vol. 12 no. 6, pp. 793-808. rubenstein-montano, b., liebowitz, j., buchwalter, j., mccaw, d., newman, b. and rebeck, k. (2001), “a systems thinking framework for km”, decision support systems, vol. 31 no. 1, pp. 5-16. rumbaugh, j., jacobson, i. and booch, g. (1999), the unified modeling language reference manual (2nd edition), the addison-wesley object technology series. ryan, g.w., and bernard, h.r. (2003), “techniques to identify themes”, field methods, vol. 15 no. 1, pp. 85-109. saito, a., umemoto, k. and ikeda, m. (2007), “a strategy-based ontology of km technologies”, journal of km, vol. 11 no. 1, pp. 97-114. scheer, a.w. and nüttgens, m. (2000), aris architecture and reference models for business process management, springer-verlag, heidelberg. schiuma, g. (2012), “managing knowledge for business performance improvement” journal of km, vol. 16 no. 4, pp. 515-522. schwartz, d.g. (2006), “an aristotelian view of km”, encyclopaedia of km, idea group reference, hershey, pa (usa), pp. 10-16. schwandt, t.a. (1994), “constructivist, interpretivist approaches to human inquiry”, in denzin, n. k. and lincoln, y. s. (eds.), handbook of qualitative research, sage, thousand oaks ca, pp. 118137. silverman, d. (2006), interpreting qualitative data, sage publications, thousand oaks ca. australasian journal of information systems volume 18 number 3 2014 118 sims, j.h. and baumann, d.d. (1983), educational programs and human response to natural hazards, environment and behavior, vol. 15 no. 2, pp. 165-189 stanford university (2009), welcome to protégé, stanford center for biomedical informatics research, stanford university, viewed 19 july 2013, retrieved from http://protege.stanford.edu/. vernadat, f. (2002), “ueml: towards a unified enterprise modelling language”, international journal of production research, vol. 40 no. 17, pp. 4309-4321. von krogh, g., ichijo, k. and nonaka, i. (2000), enabling knowledge creation: how to unlock the mystery of tacit knowledge and release the power of innovation, oxford university press. von krogh, g., nonaka, i. and rechsteiner, l. (2012), “leadership in organizational knowledge creation: a review and framework”, journal of management studies, vol. 49 no. 1, pp. 240277. von krogh, g., roos, j. and slocum, k. (1994), “an essay on corporate epistemology”, strategic management journal, vol. 15, pp. 53–71. wegmann, a., regev, g., garret, g.a. and maréchal, f. (2008, september), “specifying services for itil service management”, paper presented at the international workshop on service-oriented computing consequences for engineering requirements (soccer'08), in the 16th ieee international requirements engineering conference (re’08), barcleona. weske, m. (2012), business process management: concepts, languages, architectures, springerverlag, heidelberg. paper title (paper title style) australasian journal of information systems karagoz, korthaus & augar 2016, vol 20, research article managing project knowledge in the public sector 1 how do ict project managers manage project knowledge in the public sector? an empirical enquiry from the victorian public sector in australia yakub karagoz victoria university yakub.karagoz@live.vu.edu.au axel korthaus victoria university naomi augar rmit university abstract projects are temporal organisation forms that are highly knowledge-intensive and play an important role in modern public (and private) sector organisations. the effective and efficient creation, dissemination, application and conservation of relevant knowledge are a critical success factor in the management of projects. yet, project management (pm) and knowledge management (km) are two distinct disciplines. this paper explores the relationship between pm and km by analysing the literature at the intersection of those disciplines and presenting the empirical results of a case study of the victorian public sector (vps) in australia. a series of 14 interviews were conducted to explore how ict project managers manage project knowledge across the departments of the vps. findings show a strong preference among the participants for informal, face-to-face interactions and agile approaches to facilitate knowledge transfer and creation in ict project environments. keywords: ict; project manager; project management; projects; knowledge; knowledge management; public sector; case study. 1 introduction within academic and community circles, project management (pm) is understood to be a sub discipline or a specialised branch of management that has progressed to control modern day business ventures (karagoz et al et al., 2014). pm, according to the project management institute “…is the application of knowledge, skills, tools, and techniques to project activities to meet the project requirements” (pmi, 2013a, p. 5). put simply, a project is motivated by a unique idea to introduce change, whereas pm is the consulting aid to realise its purpose. turner (2009) refers to projects as temporary organisations. when a business endeavours to accomplish a vision of its future state, it creates a new organisation with a temporary existence, disbanded once the objectives are achieved. in many instances, information and communications technology (ict) initiatives are implemented via projects (cadle & yeates, 2004) and effective management of such projects is imperative in today’s turbulent and competitive climate. while the importance of investing in icts and their applications cannot be ignored, there is, however, high level of interest and concern when it comes to investing in such endeavours, particularly those funded by the taxpayer. rosacker and rosacker (2010) postulate that “…it projects are far too often…wasteful, inefficient, mismanaged, expensive and behind schedule” (p. 578). past and present projects across the australian public sector have resulted in similar outcomes (brouwer, 2011). pm is innately a knowledge-intensive activity and there is growing interest in how knowledge management (km) integrates with pm practices. like the terms project and pm, there is no unified or agreed definition of km (gasik, 2011). however, scholars do tend to agree it adds value to the organisation, for example, competitive advantage (argote & ingram, 2000; pmi, 2015). australasian journal of information systems karagoz, korthaus & augar 2016, vol 20, research article managing project knowledge in the public sector 2 sarayreh et al. (2012) suggest that km “… is about trying to harvest all the insights and experience that go into making an organisation function” (p. 45) while abdul rahman et al. (2008) conclude that km is a strategy to identify, create, acquire, transfer, share and exploit knowledge. gasik (2011) separated the definition of km across two camps, “…the first focuses on processing the single knowledge element and enumerates functions of its life cycle” (p. 23) and the second camp “…focuses on the whole knowledge possessed by individuals and organisations and the benefits of its application” (p. 24). this looks at how to generate and leverage knowledge that creates business value and sustains competitive advantage (zhang, 2007). research from the turn of the 21st century revealed considerable efforts to shed light on and promote the importance of km in australia. for example, in 2005, the australian standard (as5037-2005) for km was published offering a framework that was flexible for “…designing, planning, implementing and assessing policies and initiatives to improve knowledge management in an organisation” (linger et al., 2007, p. 6). the standard offers guidance for organisations and provides methodologies and tools to assess their readiness to adopt km concepts. the academic community is becoming increasingly interested in the study of knowledge sharing (boateng & agyemang, 2014; mansingh et al., 2014; sandhu et al., 2011; seba et al., 2012; yao et al., 2007; yusof et al., 2012). in addition, there appears to be demonstrable evidence linking effective km and in particular, knowledge sharing activities to creativity (wang & noe, 2010), critical decision-making (mohammed & jalal, 2011) and importantly, success in project delivery (ismail et al., 2009). however, the majority of km and knowledge sharing studies as they relate to pm are concerned with private enterprises and to date, there is a lack of research undertaken towards understanding km in project environments across the public sector (amayah, 2013). there is also relatively limited scholarly analysis geared towards examining and providing a descriptive analysis on how the project manager manages project knowledge. therefore, this paper contributes to the literature by examining how ict project managers manage project knowledge in the victorian public sector in australia. this paper is structured as follows. the next section reviews literature related to km and pm in the global and australian context and examines work to date that focuses on knowledge and the project manager. this is followed by the methodology and presentation of the results of the research. the paper concludes with a discussion and recommendations for further work. 2 background this section examines knowledge management in australia and in project environments. it highlights km models that have influenced this research and explores the relationship between knowledge and the project manager. 2.1 knowledge management in australia coyte et al. (2012) examined the methods adopted to control the management of knowledge resources in small and medium enterprises (smes) in australia. the authors found that certain mechanisms were in place, which governed the strategisation and management of knowledge resources. these included informal, intensive dialogue based processes, structured by an overriding management philosophy. various other authors have examined km within the context of australia including the dairy industry (soliman, 2000), law firms (khandelwal & gottschalk, 2003), intellectual capital (zhou & fink, 2003), construction companies (maqsood et al., 2006), software development (aurum et al., 2008) and not-for-profit organisations (downes, 2014). interestingly, zhou (2004) compared managers' perceptions between private and public sectors in australia to develop a greater understanding of current km practices. the results showed that public sector respondents in australia had a less developed understanding of km australasian journal of information systems karagoz, korthaus & augar 2016, vol 20, research article managing project knowledge in the public sector 3 and intellectual capital when compared to the respondents from private sectors. at the public sector level, zhou (2004) highlighted that respondents had little understanding of km practices and were unsure about the organisational structure. further, km initiatives were often ignored at the top level. zhou (2004) also noted that very few processes were in place with no incentive and reward measures established, concluding that the context of the organisation in which they work heavily impacts managers’ perceptions of km. 2.2 knowledge management models according to cong and pandya (2003), “the terms ‘information’ and ‘data’ are often used interchangeably with the term ‘knowledge’” (p. 26), whilst in fact these concepts have different meanings. understanding these differences is important if the aim is to manage knowledge successfully. data is generally considered as raw facts such as a set of abstract numbers or values. information is data put in a given context, endowed with meaning and purpose (wallace, 2007), which in turn supports decision making (cong & pandya, 2003). knowledge “…embodies cognition, insight, erudition” (blumentritt & johnston, 1999, p. 291) and by extension, enables efficient management of data and information (grant & grant, 2008). km is often seen as an organisation’s ability to learn or handle knowledge processes (process view) or its ability to handle knowledge (product view) (maier et al., 2005). from a process view, at least four core processes of km, notably creation, storage and retrieval, distribution/transfer and application of knowledge, are present in nearly all definitions of km (alavi & leidner, 2001). this is sometimes referred to as the km life cycle (dalkir, 2005). whilst very similar in essence, various authors describe different km life cycles, often built on existing frameworks (sağsan, 2006). as an example, the km life cycle developed by evans et al. (2014) has been derived from the model described in evans and ali (2013) and contains the seven phases: identify, store, share, use, learn, improve and create. similarly, sağsan (2006) developed a comprehensive model consisting of only five sequential steps. from the km literature, it is apparent that knowledge transfer and knowledge sharing have been used synonymously (renzl, 2006) and are recognised to have different meaning to different authors (paulin & suneson, 2015). for the purposes of this research, knowledge sharing is “the exchange of knowledge between and among individuals” (schwartz, 2005, p. 542) and knowledge transfer is the transmission of information from source to recipient (renzl, 2006). a seminal work in km-related literature, which has become the cornerstone of knowledge creation and transfer theory, is the seci model introduced by nonaka and takeuchi (1995). the authors distinguished between explicit knowledge (formal, codified knowledge) and tacit knowledge (knowledge in the minds of people), and drew attention to the way western firms tend to focus too much on the former. they proposed four ways that knowledge types can be combined and converted, showing how knowledge is shared and created in the organisation: socialisation, externalisation, combination and internalisation (seci). the process of socialisation occurs when tacit knowledge is passed on through practice, guidance, imitation and observation to create new tacit knowledge through shared experience, which also denotes the presence of a mutually agreed activity where individuals or groups of people interact in a uniform environment (martín-de-castro et al., 2008; nonaka et al., 2000). externalisation occurs when tacit knowledge is converted into explicit knowledge, i.e. codified into documents, manuals, etc. so that it can spread more easily through the organisation. this is particularly difficult since tacit knowledge can be virtually impossible to codify. combination refers to the simplest form of knowledge creation/conversion, where codified knowledge sources (e.g. documents) are combined to create new knowledge. lastly, internalisation describes the process of converting explicit into tacit knowledge, where a person’s existing tacit knowledge is modified and extended by internalising knowledge as explicit sources are used and learned in practice (nonaka & takeuchi, 1995). the knowledge creation process is widely acknowledged as a prominent vehicle for innovation through the application of novel ideas (swan & newell, 2000). from this notion, both tacit knowledge and explicit knowledge are harmonised and (new) knowledge is created through a continuous dialogue between the two knowledge types (nonaka, 1994). australasian journal of information systems karagoz, korthaus & augar 2016, vol 20, research article managing project knowledge in the public sector 4 taking the product view of km, it is useful to gain an understanding of different types of relevant knowledge in pm settings. based on qualitative research, reich (2007) has identified four knowledge categories that are vital to the success of projects: process knowledge, domain knowledge, institutional knowledge and cultural knowledge. process knowledge refers to knowledge that the project team and sponsors have about the project structure, methodology, tasks and timeframes. it allows a team to self-organise, since the team knows the outputs required and the time frames and can, if empowered, decide how the work should best be accomplished (reich, 2007). domain knowledge is knowledge about the environment or context within which the project operates. it includes the knowledge of the industry, firm, technologies, processes, business, products, current situation, problems/opportunities, risks and potential solutions (reich, 2007). institutional knowledge is a mix of history, power structure and values of the organisation (reich, 2007). cultural knowledge ties in with institutional knowledge and considers the cultural characteristics and backgrounds of the project team so as to understand how to manage it staff (reich, 2007). the theoretical models and frameworks introduced in this section informed the design of the questionnaires used in this research as well as the structure of parent and child nodes for the analysis of the interview data (see section 3). 2.3 knowledge management in project environments km and pm are two very distinct areas of study and a vast volume of publications exist across each domain. although pm consists of knowledge-intensive activities, the majority of the literature focuses on the project manager while the study of km in project environments only attracts a modest coverage (brookes et al., 2006; love et al., 2005). in more recent times, several scholars have progressed towards bridging the gap between km and pm, exploring diverse fields such as situated learning (sense, 2007), post project reviews (rezania & lingham, 2009) and issues in managing knowledge across project-based organisations (d'armagnac, 2014). the link between km and pm is often represented as km in project environments (hanisch et al., 2009; koskinen & pihlanto, 2008; lytras & pouloudi, 2003; polyaninova, 2011; pretorius & steyn, 2005). lessons learned are key project experiences that have relevance for future projects (schindler & eppler, 2003). in their case study of an engineering consulting organisation, owen and burstein (2005) acknowledged that lessons learned from previous projects were applied at project planning phases to avoid obvious mistakes, which was usually done through informal methods. it was also noted that lessons learned processes were seen as a form of knowledge creation, the first step sequence in the km life cycle. polyaninova (2011) develops this further and suggests that project knowledge is generated from two sources – internal and external. internal sources include risk logs, lessons learned and experience while external sources include seminars, benchmarking and competitor analysis. 2.4 knowledge and the project manager the role of the project manager has been discussed by various authors (gaddis, 1959; morris et al., 2006) and the demand for such practitioners is growing. for example, according to the pmi (2013b), “between 2010 and 2020, 15.7 million new project management roles will be added globally across seven project intensive industries” (p. 2). these include manufacturing, business services, finance and insurance, oil and gas, information services, constructions and utilities. however, there are significant voices opining that project managers fall into the profession by chance or coincidence (darrell et al., 2010; pinto & kharbanda, 1995). this is further amplified by richardson et al. (2015) who argue “…professional project managers do not intend to be project managers but ‘fall into’ the profession” (p.1). there is a growing body of empirical work dedicated to focusing on the project manager as the theme or the unit of analysis. such research focuses on the perspectives and opinions of the project manager as they relate to the topic of investigation. geraldi et al. (2010) explored the enabling conditions that support a project manager’s ability to cope with unexpected events australasian journal of information systems karagoz, korthaus & augar 2016, vol 20, research article managing project knowledge in the public sector 5 and questioned “…what organisational foundations are relevant to respond to unexpected events successfully?” (p. 548). creasy and anantatmula (2013) focused on personality traits and dimensions of project managers and how they impact project success. sewchurran and barron (2008) examined the relationship between project managers and project sponsors in the delivery of it projects. several other scholars have examined the project manager as part of their analysis towards understanding cognitive decision making (esa et al., 2014), competencies (hanif & tariq, 2014) and leadership (geoghegan & dulewicz, 2008). there is however, a lack of enquiry examining the role of and the relationship between knowledge and the project manager. therefore, the following research question underpins this study: how do ict project managers manage project knowledge in the public sector? a detailed description of the research methodology used to address this research question is presented in the next section followed by a discussion of the results. 3 methodology a qualitative method was used to pursue the objectives of this investigation as it is aligned with the interpretivist paradigm (weber, 2004). the interpretivist paradigm draws on meanings from social context (rowlands, 2005), in this case from the perceptions of ict project managers across several departments within the vps. it is designed to capture and interpret meanings and understandings, exploring how and why a particular phenomenon occurs in its contemporary context in real-world setting (weber, 2004). case study research often follows the interpretive custom of exploration whereby the phenomenon is explored through the lens of participants (cohen et al., 2007). shavelson and towne (2002) note that the case study approach is applicable when your research tackles questions of a descriptive nature such as what happens (or happened) or explanatory questions such as how or why something occurs (or occurred), which aim to produce a first-hand understanding of the issue. thus the case study approach allowed the researchers to explore empirical events and acquire data that provides insights from participants that would not otherwise be captured through other research methods. an embedded case study design has been adopted for the research (yin, 2009). the overarching case study is the vps, while each project manager represented their respective departments within the vps and thus served as embedded sub-cases within the overarching case (yin, 2009). in this design the project manager is the unit of analysis. this design supported a detailed level of enquiry and provided opportunities for extensive analysis. a semi-structured interview process was used since questions were based on existing theoretical models. the design of the questionnaire was informed by the km life cycle models presented in section 2.2. with the components knowledge creation, knowledge capture and storage, knowledge sharing and knowledge application/reuse. parent nodes such as knowledge required and knowledge creation were filtered through reich’s (2007) knowledge types required in it projects and nonaka and takeuchi’s (1995) seci model, respectively (cf. section 2.2.). such efforts were intended to enrich these existing models and further the understanding of the role of km in pm. open-ended questions were used to elicit information from respondents allowing for further dialogue and discussions to be generated (chen et al., 1995). the project manager was chosen as the subject of enquiry (or unit of analysis) for reasons elaborated on in the background section of the paper. as the research question suggests, there is a predetermined demographic target. for consistency, the study established a baseline set of standards in relation to the experience levels of participants. to support intra-case comparison and analysis it was important that project managers had managed similar projects in terms of budget and/or scope. in addition, it was important that participants have had some experience in project management that they could reflect on when responding to questions. therefore, participants selected for the study had managed ict projects with a minimum budget of $100,000, a project duration of at least one year (in any project managed at the time of australasian journal of information systems karagoz, korthaus & augar 2016, vol 20, research article managing project knowledge in the public sector 6 interview), were currently serving as an employee within their respective departments and had been employed with the vps for at least two years. two project managers were shortlisted and interviewed across each vps department, totalling a sample size of 14 project managers. the sample size is consistent within the single exploratory case study approach, as is evidenced by other research with similar subjects of enquiry (blackburn, 2002; mchugh & hogan, 2011; worthy, 2012). interviews were transcribed to facilitate data analysis via the nvivo software package. transcriptions were analysed via a coding in detail approach whereby codes were grouped into tree nodes (bazeley, 2007, p. 69) informed by the km models presented in section 2.2. this allowed for relationships amongst themes and issues to be assessed and for emergent phenomena to be examined. 4 results this section presents the data and related patterns emerging from the interviews. table 1 below represents the initial nodes (informed by the km models highlighted in section 2.2.) that were used to classify interview data. the remainder of this section elaborates on the findings from the interviews. parent node child nodes number of references knowledge required process knowledge 10 institutional knowledge 5 domain knowledge 4 cultural knowledge 3 knowledge creation socialisation 13 internalisation 4 combination 3 externalisation 2 knowledge capture and storage procedural 11 electronic systems 10 knowledge sharing informal 10 agile 6 project artefacts 6 social media 3 knowledge application and reuse lessons learned 10 personal experience 4 table 1: classification of interview data against nodes/themes informed by km models 4.1 knowledge required during the interview, participants were asked what types of knowledge they needed to successfully manage projects. their responses were categorised using reich’s (2007) types of knowledge. references in relation to knowledge under the process knowledge category were made ten times, whilst institutional knowledge was mentioned five times, domain knowledge four times and cultural knowledge was referred to only three times, respectively. interestingly, exploration of data provided evidence that 10 of the 14 participants referenced more than one knowledge type required for their project. however, no participant referenced more than two of reich’s (2007) four project knowledge types. australasian journal of information systems karagoz, korthaus & augar 2016, vol 20, research article managing project knowledge in the public sector 7 theme 1: process knowledge it was clear that the majority of the participants emphasised the need for process knowledge. two participants specifically highlighted the need for a business case, as one suggested “the business case, for a start, or anything that’s remotely close that tells me about the core elements about the projects” pm13. 4.2 knowledge creation participants were asked how they created the knowledge required for managing projects. results were classified against four nodes based on the seci model of knowledge creation (cf. section 2.2.): internalisation, combination, externalisation and socialisation. very few responses highlighted internalisation (4), combination (3) or externalisation (2) as a preferred knowledge creation approach. the vast majority, 13 participants, indicated that they adopted socialisation to create knowledge. theme 2: socialisation the socialisation process is achieved through a variety of techniques including workshops, training programs, (informal) meetings and the like. it was discovered that although some participants engaged in various forms of knowledge creation (i.e. internalisation, combination and externalisation), the majority most often employed the socialisation process to create new knowledge within their projects as illustrated by pm1: pm1 “i feel it's important to talk to various people and have that connection and conversation with project stakeholders…especially those who are experts, which helps gather vital information to support my project. with that social interaction…that face-to-face interaction, i think it's invaluable, it helps me identify the knowledge gaps that exist within my everyday projects and i’m able to bring in new perspectives, new dynamics into my projects”. interviews also demonstrated participants felt that they found this process to be the easiest and most efficient way to create knowledge for their projects. they alluded to the fact that emailing, telephoning or other methods of communication within their departments “slowed down” their knowledge creation capabilities, and thus their efforts were focused on socialisation. one participant (pm6) further justified this position as it allowed them to uncover knowledge that would not otherwise be captured through other forms of knowledge creation: pm6 “i feel face-to-face meetings are much better because i feel that you pick up the extra dimension and pick up cues on people’s voices, and what the severity levels are, if they have the confidence in their ability to solve problems and deal with issues they have. you know, i can have that ability to try to read between the lines that you would not normally get when dealing with emails or telephone conversations. so there’s that extra dimension added when, that social complexity you have, that face-to-face interaction and meeting people as opposed to non-face-to-face interaction”. 4.3 knowledge capture and storage interviewees were asked to discuss how they captured the knowledge required to manage projects. results were classified into two main themes: procedural and electronic systems with 11 and 10 references, respectively. theme 3: electronic systems and procedural methods participants adopted a combination of electronic systems accompanied by either a departmental or (project) methodical process to categorise and store project knowledge. a large proportion of participants confirmed the existence of knowledge management systems and exploited such tools as required to store specific project knowledge. of the participants who confirmed the usage of such systems, namely sharepoint and trim, most agreed that according to the best of their knowledge, there were no ‘official’ km systems or a km standard that was endorsed by their department or the vps. however, participants indicated that they australasian journal of information systems karagoz, korthaus & augar 2016, vol 20, research article managing project knowledge in the public sector 8 were highly encouraged to adopt the system (trim) and undergo training and development to improve user experience and project efficiency: pm10 “in our department, we have to have a four hour introduction before you get access to trim and every single thing that you do is stored in trim. so you have to do this course literally the first day you start. it’s very structured and much organised in that respect. there are documents and templates that are accessible through trim and i use quite to benefit my project. i also encourage my project team to use it as well. yet, there is not a strict process or compliance standards that tell us project managers…you know…you are required to use this for all your projects and if you don’t, then there will be consequences. so long as you stick to the known protocols within the department and follow a project methodology, you should be fine”. participants stored project information into their dedicated network drives. each of the participants confirmed that specific network drives were set up for project teams before projects even commenced. the participants would then create project folders and files that were consistent with a project methodology. for example, one of the participants would use a prince2 approach as a guide to systematically set up and structure the project in their respective network drives. this includes folders such as pre-project, initiation, delivery/control and closing a project. 4.4 knowledge sharing theme 4: informal structures an analysis of how participants engaged in sharing knowledge within their projects revealed an overwhelming consensus towards the adoption of the socialisation process. some participants would create informal knowledge sharing structures of their own accord in addition to leveraging various social events or activities outside formal processes within their departments, as was the case of pm2: pm2 “i find the best way to knowledge transfer or knowledge sharing is through an informal setting. i’d tried doing it as a lecture series or like a lunch session or something like that. and it really depends on people’s personality types and how they operate. i find the best way to knowledge sharing…i love melbourne and i enjoy my coffee and i’ll have a chat. i have two coffees a day and i’ll say “look let’s go for a coffee and have a quick chat”, where you’re just waiting for things just so that’s a little bit of transfer or sharing in that sort of instance. and it depends on the type of resource. i am an extrovert, so i find that doing things at a coffee shop or something like that with fellow extroverts is the best way to do transfer. if i am dealing with highly technical staff or subject matter experts, they tend to be very introverted so they really shut down in that sort of environment so it’s really formal and having a formal meeting. for example, it’ll be a one-on-one meeting, you know going through a structured agenda, with technical diagrams, and so i find that to be the best way in terms of sharing knowledge”. the phenomenon of using informal settings to share project knowledge seemed to be the dominant method. for most project managers, the idea of sharing project knowledge in an environment outside the “four walls” orchestrated by their departments brought about a degree of either establishing or enhancing an open and honest relationship within the project team. pm2 “this [informal knowledge sharing] i’m much more comfortable with…certainly has its merits and beats traditional meetings. i get what i need or near to what i need, avoid potential issues and most of all avoid politics”. it was further emphasised that the participants would generally find the need to adopt this method to bring about team collegiality within their projects, extract meaningful information that would not otherwise be captured through formal means and efficiently solve project issues with suppliers and other project stakeholders. australasian journal of information systems karagoz, korthaus & augar 2016, vol 20, research article managing project knowledge in the public sector 9 theme 5: using agile techniques to facilitate knowledge sharing agile pm techniques draw on the philosophy of the agile manifesto which values “individuals and interactions over processes and tools, working software over comprehensive documentation, customer collaboration over contract negotiation [and] responding to change over following a plan” (beck et al., 2001). the results of this study suggested that participants relied on using agile techniques to facilitate the knowledge sharing process within the project team. pm12 suggested this approach generated a drive to share knowledge as the environment encouraged the project team to organically discuss technical issues including project integration and inter-dependencies, raise questions, resolve problems, provide feedback, facilitate team collaboration, build trust and foster new relationships. pm12 “well we have a lot of “stand ups”. and this is part of the agile movement where the project team has a time boxed meeting between 5 to 15 minutes for a quick status update. we stand up to keep the meeting short every morning. the team asks for clarifications and makes brief statements about the project’s progress, such as "let's discuss this more after the meeting", so we avoid full-fledged discussions. the team leader asks if anyone else has anything to share and this is where the knowledge sharing really flourishes. so this is great for knowledge sharing and always works well. other approaches include your basic communication mediums – telephone, emails and meetings etcetera. pm8 “the key training there for us is to learn to communicate in that agile/scrum sort of way – stand up meetings, sprint planning and sprint reviews and all that sort of thing. that is a much more effective way of communicating the project than through a prince2 way, which is just documentation”. the free-flowing casual environment seems to yield a positive atmosphere for project managers and their teams, which demonstrated a level of equality, freedom of expression (including opinions) and allowing immediate access to members of the project team that would not otherwise be as approachable in traditional formal office structures. it was further revealed that knowledge sharing (within an agile environment) went beyond the dynamics of the project team and towards a tacit build up within clients/customers through face-to-face conversations during iterations. participants expressed that having this freedom meant they were able to work outside the constraints of formal structures that would impede on their ability to roam and perform their duties as a project manager and tackle the scope of the project. they further voiced that removing the hurdles of documentation and traditional processes meant they were generally able to interact with other project members including business analysts, developers and testers. all in all, it was apparent that the agile approach played a pivotal role in the success of facilitating knowledge sharing as perceived by the participants and further allowed project managers to occupy roles within the project team. theme 6: project artefacts another form of knowledge sharing within the project team was the use of project artefacts, namely project documentation. participants described that they would circulate project documents or materials such as technical diagrams, business/process mapping, project plans, status reports and risk and issues registers to relevant stakeholders to share knowledge about their projects. the frequency of project artefact circulation differed depending on the nature of the project and the applied pm framework. the project artefacts were used to examine the health and status of the overall project, determine operational quality, confirm progress, reiterate constraints and dependencies and identify risks and issues. an emailing system was used as a key channel to transfer and share project documentation. this allowed the team to review, critique and update relevant information throughout the duration of the project. theme 7: social media data provided evidence that there was to some extent continuous online knowledge sharing within projects. participants reported ‘yammer’ as the vehicle for collaboration, which was sometimes used with the stakeholders. in other words, it was not exclusively used for the australasian journal of information systems karagoz, korthaus & augar 2016, vol 20, research article managing project knowledge in the public sector 10 project team but was available to other colleagues including senior users, suppliers, product owners, suppliers and vendors. as pm12 put it, “depending on the piece of work, i am actually using yammer at the moment for collaboration stuff with an external supplier and that’s the easiest way to share information, it’s consistent and handy”. further, data collected from interviews suggested that colleagues, including management, endorsed the usage of such products across projects. this platform did bring about a number of advantages to the project such as allowing users to share live information (including creating, editing and evaluating documents), requesting support, removing everyday obstacles such as organising meetings, co-ordinating work and a whole host of activities from work or in a remote setting. 4.5 knowledge application and reuse theme 8: lessons learned a common strategy to apply and reuse knowledge from one project to another was the utilisation of a lessons learned process. this approach was widely practiced by the participants who saw this method as an effective medium to carry valuable knowledge for reuse. participants indicated that the lessons learned process was a valued activity and was seen as an integral part of propagating effective methods throughout projects by sharing and reusing knowledge. data from the interviews also indicated that there was not one single guiding principle, method or practice according to which the lessons learned process was conducted. at times, the process was done informally at the end of the project where participants drew on their own experiences from past projects or organisations and created their own process for collecting, storing and disseminating the lessons. participants would use existing templates either downloaded from their respective department’s records management systems or by simply browsing the intranet. to do this, participants would follow systematic steps where they would identify and document what worked well and what needed improvement, generate a case for methods of improvement and ensure the material was archived as required by departmental standards. other forms of lessons learned activities included post project reviews such as face-to-face meetings. this was considered to be a more formal approach to lessons learned as it was controlled or monitored by a nominated facilitator. one of the participants stated that they would implement an agile technique to documenting lessons learned. dubbed “retrospective”, project team members would meet at the end of each iteration (including after the completion of the project) and reflect/deliberate on what went well, what didn't and what could be improved. this approach allowed project managers to capture vital discussion points for future implementation. when probed about what mechanisms exist or how the organisation supported the lessons learned process, participants unanimously agreed that more could be done from “upper chambers” of the department. for example, the lessons learned process should be made consistent and compulsory for every project and the pmo should play a more collegial role throughout such activities. further, the lack of time available to undertake capturing lessons learned seemed to be a major hurdle for participants. although they unanimously agreed on its importance, they felt that if they had more time overall, they would be able to spend more quality time on the lessons learned process. one participant expressed that sometimes it is considered a “dump and run exercise” due to the need to move on to other projects. another challenge participants expressed was the notion of questioning or not relying on the originator of documents relating to lessons learned or post implementation reviews. “meaning” and “background” were the most frequently used words to describe how lessons learned were not readily understood within their context. in-depth narratives and contextual analysis were left out from most lessons learned documents. thus, many experienced difficulties in trying to make use of and apply such knowledge to their current or future projects. different circumstances and environments made it even more difficult to give meaning and usage. participants sensed that they had to play “dodge-ball” in order to avoid “responsibility” and damage “reputation” because there was a high risk that their “voice” would australasian journal of information systems karagoz, korthaus & augar 2016, vol 20, research article managing project knowledge in the public sector 11 not be heard. for example, “…in the past there has been plenty of covering up and not enough owning up” (pm4). however, this did not stop participants in documenting their project failures when it came to lessons learned. participants tended to focus on questioning information arising from lessons learned, post implementation reviews and the like. lastly, locating or having access to lessons learned proved to be another challenge for the participants. few recounted that they faced some difficulty locating the large database of documents and others mentioned that such documents were not made available to them. those who reported having access issues agreed that much more could be done in either making relevant lessons learned available to them or providing a better database management system for retrieval and usage. theme 9: personal experience to a lesser extent, interviews demonstrated that participants’ prior experiences lead them to reuse that knowledge. for example, the knowledge gained from previous projects, whether positive or negative, was applied to future projects. in other words, as a result of a series of events, behaviour was modified or new skills were acquired, which were then (tacitly) transported to new projects and applied in practice. participants further indicated that not all knowledge could be captured and put on paper and successively transferred to other projects “… my 15 years of project experience in the public sector can’t be expressed on paper…experience is what counts and managing project after project and learning what i’ve done in the past allows me to manage them [projects] more effectively” (pm7). those who relied on personal experiences as an effective means to apply or reuse knowledge agreed that this brought about several benefits to new projects that would not otherwise be realised from other means. 5 discussion this section introduces an analysis of the research findings in relation to the existing literature. it examines what the discoveries mean in light of the current theoretical body of knowledge and highlights practical implications arising from the analysis. 5.1 knowledge required classification of project manager responses related to the knowledge they require to manage their projects was made against reich’s (2007) model of the four knowledge types in it projects. almost all respondents indicated they needed process knowledge. less than half of the respondents indicated that institutional, domain and cultural knowledge were required. when probed for examples of process knowledge, many identified the business case as a critical document. the business case justifies the project and guides the development of early project planning (kloppenberg, 2012); it helps stakeholders to determine if the project is feasible for the organisation (axelos, 2009). flawed business cases can lead to project failure (whittaker, 1999) so a complete and accurate business case is essential for project managers to properly plan their projects (axelos, 2009; kloppenberg, 2012). 5.2 knowledge creation results from the investigation affirmed the importance of the socialisation process. knowledge creation relied on personal networks as participants would leverage a number of social activities and events including workshops, professional training and in particular, the use of informal meetings with colleagues. informal approaches, which were also predominantly adopted for knowledge sharing (see knowledge sharing section 4.4.), were found to be the preferred method in the creation of knowledge as it allowed participants to bring in new perspectives to their projects; beyond what they would normally unearth through the scanning of random project documents. the concept of creating knowledge through non face-to-face interaction such as emailing and telephoning, although present and embraced, appeared not to be the favoured method, as it did not offer effective and immediate solutions to the required problem or question. however, kao et al. (2011) argued that socialisation alone does not australasian journal of information systems karagoz, korthaus & augar 2016, vol 20, research article managing project knowledge in the public sector 12 necessarily create knowledge. rather, when combination and internalisation are executed and knowledge is internalised (i.e. put into action), knowledge can be generated. 5.3 knowledge capture and storage it was recognised that departments presented participants with relevant tools to be able to store project knowledge, and the process of capturing knowledge appeared to be tactical and systematic. these approaches resonate with dougherty (2004), who suggested “…capturing it [knowledge] requires the practices themselves to be organised somehow” (p. 35). additionally, participants often relied on it as a tool to structure project information conforming to a strategy or a particular method. for example, the prince2 method was used to set up project files and make knowledge explicit (i.e. tangible) for project use. in km terms, this process is closely tied to the codification process whereby once relevant knowledge is captured, content is categorised in a systematic manner (dalkir, 2005) and is accessible for further use (almeida & soares, 2014; boh, 2005) – an activity that was practiced in this study. from this perspective, boh (2005) argues that once knowledge is codified in various artefacts, they can be recognised as important means for knowledge sharing. moreover, to a lesser extent, the departmental electronic databases or record management systems were also used for knowledge capture and storage. making use of such tools provides several benefits to the accessibility and reusing of knowledge (kivrak et al., 2008). the literature linking it to knowledge capture is well established (alavi & leidner, 2001; lee & choi, 2003). it is considered to be a critical element in effectively managing intellectual capital (chou, 2005; dalkir, 2005; egbu & botterill, 2002). it was identified that knowledge captured from previous projects was stored and reused for future projects (kivrak et al., 2008). such findings correspond with the results from this investigation. however, gasik (2011) asserts that this process should be monitored at the organisational level to ensure appropriate classification, consistency and integrity of project knowledge. according to dalkir (2005), it is simply more than making use of technology and he argues “…it plays only a small part in ensuring that information is available to those who need it” (p. 78). 5.4 knowledge sharing findings indicated that many informal meetings occurred in settings outside of the office in spaces such as nearby parks, cafés, lounges, in cars while travelling to project sites and even at home. however, it was also noted that specific meeting spaces were selected dependent on the type of information required. davenport and prusak (2000) noted that such locations were common, which also gave rise to a sense of mutual trust outside the boundaries of the organisation with collaboration from stakeholders such as customers and suppliers (nonaka et al., 2000). participants enjoyed the idea of freely conversing with project team members through informal environments. according to karlsen et al. (2011), the agile paradigm places less emphasis on project documentation and partially replaces it by informal interactions within and between project teams and stakeholders. this view along with those of cockburn and highsmith (2001), chau and maurer (2004), dybå and dingsøyr (2008), boden et al. (2009) and stettina et al. (2012) are consistent with the findings of this investigation. a key factor that enabled participants to share knowledge was not only the establishment of a casual environment, but a set of workspaces that allowed speedy access to members of a project team. according to santos et al. (2013), such a climate enables project teams to be conscious of emerging organisational matters, network with little effort and have a sense of purpose beyond their project team. 5.5 knowledge application and reuse interviews revealed two main themes concerning the application and reuse of knowledge – lessons learned and personal experience. results uncovered two main approaches to lessons learned; formal and informal practices. this tallies with jugdev (2012) who argues that lessons learned processes “…involve formal and informal practices” (p. 14) and occur in “…selfaustralasian journal of information systems karagoz, korthaus & augar 2016, vol 20, research article managing project knowledge in the public sector 13 directed, collective and social ways” (p. 20). this is echoed by owen et al. (2004) who postulate that this process is critical in the creation, transfer and reuse of knowledge for projects. the results of this research build upon these findings, since participants took on the initiative with little assistance or direction, formulating their own practices to drive the lessons learned process. moreover, it was determined that once lessons were captured at the end of the project, they were archived in a server or a department database and reused for forthcoming projects. owen and burstein (2005) opined that once project material is stored in such systems, the content needs to be maintained or updated to ensure it’s relevant for future project use. although the method of storing lessons learned was consistent with owen and burstein’s (2005) proposition, little evidence was found of mechanisms that supported participants in updating the material once archived. furthermore, while not practicing lessons learned at the end of a project is not uncommon, it does result in low quality outcomes, and a lack of cross project learning (ajmal & koskinen, 2008; pemsel & wiewiora, 2013). the lack of time to capture lessons learned has been well documented in the literature (carillo et al., 2004; keegan & turner, 2001; von zedtwitz, 2002) and findings from this paper concur with literature across the pm and km fields (fadairo, 2016; paranagamage et al., 2012; pemsel & wiewiora, 2013; shokri-ghasabeh & chileshe, 2014). one major factor was the presence of contemporaneous and new projects, which posed a barrier to spending quality time identifying and capturing lessons learned. further efforts should focus on other factors that impede pm to capture lessons learned in these environments and pragmatic solutions should be made available for sustainable and long term competitiveness (anbari et al., 2008). 6 conclusion ict projects are temporal organisations accepted to be knowledge-intensive organisational forms that have to address complex, socio-technical business changes. to be successful, projects critically depend on utilising and creating the right mix of knowledge and experiences. moreover, due to the temporal nature of projects, acquired knowledge and experiences can become hard to access and leverage in future projects if not handled properly by a suitable km function. thus, as a first step towards improving existing practices in future, it is essential to increase our understanding of how pm activities are currently supplemented by km activities in practice. there is a particular gap of insight to date regarding public sector environments. this paper presents the findings of a descriptive case study that examines how a cohort of ict project managers in several departments of the vps manage project knowledge. several general conclusions may be drawn from the work, which has been presented in detail in the results and discussion sections of this paper. one particular theme stood out from the analysis of the interview data in the study, and that was the participants’ preference for informal structures and face-to-face interactions/exchanges. this preference falls under the umbrella of the socialisation category of the seci model and is best enabled by agile approaches to pm that shift the emphasis from formal codified project artefacts towards human interactions. process knowledge turned out to be the most relevant type of knowledge in the project context in the vps according to the participants. the importance of both personal experience and codified lessons learned was acknowledged by the interviewees. while both procedural methods and electronic systems (including enterprise social media) are actively being used in projects to facilitate capturing and sharing project artefacts. the empirical research presented in this paper contributes to and extends the existing body of knowledge regarding the role of knowledge management in ict project management contexts, an area that is currently characterised by a rather limited amount of prior research. one limitation of the research is the relatively small sample size of 14 ict project managers across seven departments in a single case study of the vps. therefore, the findings of this study should be considered as a driver and starting point for further research. more empirical research is needed in order to further increase understanding of how knowledge is managed across the australasian journal of information systems karagoz, korthaus & augar 2016, vol 20, research article managing project knowledge in the public sector 14 project life cycle and to derive actionable insights into how this process can be improved. future work will seek to capture a broader sample of participants via a survey based on the findings presented in this paper. as part of a larger research program, the authors are also investigating the barriers to knowledge sharing that are commonly experienced in pm settings. the results of this study make an academic contribution by adding to the empirical foundation of our knowledge about the topic area. they also contribute to practice by informing project managers and consultants carrying ict projects in public sector environments. references abdul rahman, h., yahya, i. a., beravi, m. a. & wah, l. w. (2008) "conceptual delay mitigation model using a project learning approach in practice", construction management and economics, 26(1): 15-27. ajmal, m. m. & koskinen, k. u. (2008) "knowledge transfer in project-based organizations: an organizational culture perspective", project management journal of information science, 39(1): 7-15. alavi, m. & leidner, d. e. (2001) "knowledge management and knowledge management systems: conceptual foundations and research issues", mis quarterly, 25(1): 107–136. almeida, m. v. & soares, a. l. (2014) "knowledge sharing in project-based organizations: overcoming the informational limbo", international journal of information management, 34(6): 770-779. amayah, a. t. (2013) "determinants of knowledge sharing in a public sector organization", journal of knowledge management, 17(3): 454-471. anbari, f. t., carayannis, e. g. & voetsch, r. j. (2008) "post-project reviews as a key project management competence", technovation, 28(1): 633-643. argote, l. & ingram, p. (2000) "knowledge transfer: a basis for competitive advantage in firms", organizational behavior and human decision processes, 82(1): 150-169. aurum, a., daneshgar, f. & ward, j. (2008) "investigating knowledge management practices in software development organisations–an australian experience", information and software technology, 50(6): 511-533. axelos. (2009) managing successful projects with prince2, the stationery office, belfast, northern ireland. bazeley, p. (2007) qualitative analysis with nvivo, sage, california, usa. beck, k., beedle, m., van bennekum, a., cockburn, a., cunningham, w., fowler, m., grenning, j., highsmith, j., hunt, a., jeffries, r., kern, j., marick, b., martin, r. c., mellor, s., schwaber, k., sutherland, j. & thomas, d. (2001) manifesto for agile software development. available at: http://www.agilemanifesto.org/ [accessed 30 august 2016] blackburn, s. (2002) "the project manager and the project-network", international journal of project management, 20(3): 199-204. blumentritt, r. & johnston, r. (1999) "towards a strategy for knowledge management", technology analysis & strategic management, 11(3): 287-300. boateng, h. & agyemang, f. g. (2014) "a qualitative insight into key determinants of knowledge sharing in a public sector institution in ghana", information development(april 17, 2014): 1-10. boden, a., avram, g., bannon, l. & wulf, v. (2009) knowledge management in distributed software development teams does culture matter?, paper presented at the fourth ieee international conference on global software engineering, limerick, ireland. boh, w. f. (2005) reuse of intellectual capital from repositories: a mixed methods study, paper presented at the academy of management meeting, honolulu, hawaii. australasian journal of information systems karagoz, korthaus & augar 2016, vol 20, research article managing project knowledge in the public sector 15 brookes, n., morton, s., dainty, a. & burns, n. (2006) "social processes, patterns and practices and project knowledge management: a theoretical framework and an empirical investigation", international journal of project management, 24(6): 474-482. brouwer, g. (2011). own motion investigation into ict-enabled projects: melbourne: victorian ombudsman, victorian government printer. cadle, j. & yeates, d. (2004) project management for information systems, pearson education. carillo, p., robinson, h., al-ghassani, a. & anumba, c. (2004) "knowledge management in uk construction: strategies, resources and barriers", project management journal, 35: 46-56. chau, t. & maurer, f. (2004). knowledge sharing in agile software teams. in m. m. richter & w. lensk (eds.), logic versus approximation (pp. 173-183). berlin: springer. chen, v., pearce, w. b. & leeds-hurwitz, w. (1995) "even if a thing of beauty, can a case study be a joy forever", social approaches to communication: 135-154. chou, s.-w. (2005) "knowledge creation: absorptive capacity, organizational mechanisms, and knowledge storage/retrieval capabilities", journal of information science, 31(6): 453-465. cockburn, a. & highsmith, j. (2001) "agile software development, the people factor", computer, 34(11): 131-133. cohen, l., manion, l. & morrison, k. (2007) research methods in education (6th ed.), routledge/taylor & francis group, new york, ny, us. cong, x. & pandya, k. v. (2003) "issues of knowledge management in the public sector", electronic journal of knowledge management, 1(2): 25-33. coyte, r., ricceri, f. & guthrie, j. (2012) "the management of knowledge resources in smes: an australian case study", journal of knowledge management, 16(5): 789-807. creasy, t. & anantatmula, v. s. (2013) "from every direction—how personality traits and dimensions of project managers can conceptually affect project success", project management journal, 44(6): 36-51. d'armagnac, s. (2015) "issues in the management of embedded knowledge in project-based organizations: the project actor’s role", knowledge management research & practice, 13(4): 446-462. dalkir, k. (2005) knowledge management in theory and practice, elsevier butterworthheinemann, burlington. darrell, v., baccarini, d. & love, p. e. d. (2010) "demystifying the folklore of the accidental project manager in the public sector", project management journal, 41(5): 56-63. davenport, t. & prusak, l. (2000) working knowledge, (2nd ed.), harvard business press, usa. dougherty, d. (2004) "organizing practices in services: capturing practice-based knowledge for innovation", strategic organization, 2(1): 35-64. downes, t. v. (2014) an evaluation of knowledge management practices in nonprofit community services organisations in australia. (doctor of business administration), southern cross university, lismore, nsw. dybå, t. & dingsøyr, t. (2008) "empirical studies of agile software development: a systematic review", information and software technology, 50(9): 833-859. egbu, c. o. & botterill, c. (2002) "information technologies for knowledge management: their usage and effectiveness", itcon, 7: 125-137. australasian journal of information systems karagoz, korthaus & augar 2016, vol 20, research article managing project knowledge in the public sector 16 esa, m., alias, a. & abdul samad, z. (2014) "project managers’ cognitive style in decision making: a perspective from construction industry", international journal of psychological studies, 6(2): 65-73. evans, m. & ali, n. (2013) bridging knowledge management life cycle theory and practice, paper presented at the international conference on intellectual capital, knowledge management and organisational learning, washington dc, usa. evans, m., dalkir, k. & bidian, c. (2014) "a holistic view of the knowledge life cycle: the knowledge management cycle (kmc) model", electronic journal of knowledge management, 12(2): 85-97. fadairo, o. (2016) benefits of conducting postproject reviews to capture lessons learned. (doctor of business administration), walden university, minneapolis, usa. gaddis, p. o. (1959) the project manager, harvard university. gasik, s. (2011) "a model of project knowledge management", project management journal, 42(3): 23-44. geoghegan, l. & dulewicz, v. (2008) "do project managers' leadership competencies contribute to project success?", project management journal, 39(4): 58-67. geraldi, j. g., lee-kelley, l. & kutsch, e. (2010) "the titanic sunk, so what? project manager response to unexpected events", international journal of project management, 28(6): 547–558. grant, k. & grant, c. (2008) the knowledge management capabilities of the major canadian financial institutions, paper presented at the the international conference on knowledge management. hanif, a. & tariq, s. (2014) an evaluation of personal and interpersonal competencies of project managers, paper presented at the 2014 international conference on emerging technologies (icet), islamabad, pakistan. hanisch, b., lindner, f., mueller, a. & wald, a. (2009) "knowledge management in project environments", journal of knowledge management, 13(4): 148-160. ismail, w., nor, k. & marjani, t. (2009) "the role of knowledge sharing practice in enhancing project success", institute of interdisciplinary business research, 1(7): 34-52. jugdev, k. (2012) "learning from lessons learned: project management research program", american journal of economics and business administration, 4(1): 13-22. kao, s., wu, c. & su, p. (2011) "which mode is better for knowledge creation?", management decision, 49(7): 1037-1060. karagoz, y., korthaus, a., & augar, n. (2014) barriers to knowledge sharing in ict project environments. paper presented at the 25th australasian conference on information systems, auckland, new zealand. karlsen, j. t., hagman, l. & pedersen, t. (2011) "intra-project transfer of knowledge in information systems development firms", journal of systems and information technology, 13(1): 66-80. keegan, a. & turner, j. r. (2001) "quantity versus quality in project-based learning practices", management learning, 32(1): 77-98. khandelwal, v. k. & gottschalk, p. (2003) "information technology support for interorganizational knowledge transfer: an empirical study of law firms in norway and australia", information resources management journal, 16(1): 14-23. kivrak, s., arslan, g., dikmen, i. & birgonul, m. t. (2008) "capturing knowledge in construction projects: knowledge platform for contractors", journal of management in engineering, 24(2): 87-95. australasian journal of information systems karagoz, korthaus & augar 2016, vol 20, research article managing project knowledge in the public sector 17 kloppenberg, t. j. (2012) contemporary project management, (2nd ed.), south western, cengage learning, mason, oh. koskinen, k. u. & pihlanto, p. (2008) knowledge management in project-based companies, palgrave macmillan. lee, h. & choi, b. (2003) "knowledge management enablers, processes, and organizational performance: an integrative view and empirical examination", journal of management information systems, 20(1): 179-228. linger, h., hasan, h. m. & burstein, f. (2007) "integrating doing and thinking in a work context: an australian knowledge management perspective", scandinavian journal of information systems, 19(1): 59-86. love, p. e., fong, p. s.-w. & irani, z. (2005) management of knowledge in project environments, routledge. lytras, m. d. & pouloudi, a. (2003) "project management as a knowledge management primer: the learning infrastructure in knowledge-intensive organizations: projects as knowledge transformations and beyond", the learning organization, 10(4): 237-250. maier, r., haedrich, t. & peinl, r. (2005) enterprise knowledge infrastructures, springer, berlin. mansingh, g., osei-bryson, k.-m. & reichgelt, h. (2014). knowledge sharing in the health sector in jamaica: the barriers and the enablers knowledge management for development (pp. 199-213): springer. maqsood, t., finegan, a. & walker, d. (2006) "applying project histories and project learning through knowledge management in an australian construction company", the learning organization, 13(1): 80-95. martín-de-castro, g., lópez-sáez, p. & navas-lópez, j. e. (2008) "processes of knowledge creation in knowledge-intensive firms: empirical evidence from boston's route 128 and spain", technovation, 28(4): 222-230. mchugh, o. & hogan, m. (2011) "investigating the rationale for adopting an internationallyrecognised project management methodology in ireland: the view of the project manager", international journal of project management, 29(5): 637–646. mohammed, w. & jalal, a. (2011) "the influence of knowledge management system (kms) on enhancing decision making process (dmp)", international journal of business and management, 6(8): 216-229. morris, p. w., jamieson, a. & shepherd, m. m. (2006) "research updating the apm body of knowledge 4th edition", international journal of project management, 24(6): 461-473. nonaka, i. (1994) "a dynamic theory of organizational knowledge creation", organization science, 5(1): 14-37. nonaka, i. & takeuchi, h. (1995) the knowledge-creating company: how japanese companies create the dynamics of innovation, oxford university press. nonaka, i., toyama, r. & konno, n. (2000) "seci, ba and leadership: a unified model of dynamic knowledge creation", long range planning, 33(1): 5-34. owen, j. & burstein, f. (2005). where knowledge management resides within project management. in m. jennex (ed.), case studies in knowledge management (pp. 138154). hershey, pa: idea group publishing. owen, j., burstein, f. & mitchell, s. (2004) "knowledge reuse and transfer in a project management environment", journal of information technology case and application research, 6(4): 21-35. australasian journal of information systems karagoz, korthaus & augar 2016, vol 20, research article managing project knowledge in the public sector 18 paranagamage, p., carrillo, p., ruikar, k. & fuller, p. (2012) "lessons learned practices in the uk construction sector: current practice and proposed improvements", engineering project organization journal, 2(4): 216-230. paulin, d. & suneson, k. (2015). knowledge transfer, knowledge sharing and knowledge barriers–three blurry terms in km. in k. grant & j. dumay (eds.), leading issues in knowledge management (vol. 2, pp. 73-94). pemsel, s. & wiewiora, a. (2013) "project management office a knowledge broker in projectbased organisations", international journal of project management, 31(1): 31-42. pinto, j. k. & kharbanda, o. p. (1995) "lessons for accidental profession", business horizons, 38(2): 41-50. pmi. (2013a) a guide to the project management body of knowledge (pmbok® guide), (5th ed.), project management institute, pennsylvania, usa. pmi. (2013b) pmi's industry growth forecast: project management between 2010 + 2020. , available at: http://www.pmi.org/~/media/pdf/businesssolutions/pmiprojectmanagementskillsgapreport.ashx [accessed 12 july 2015] pmi. (2015) pmi's pulse of the profession: capturing the value of project management through knowledge transfer . available at: http://www.pmi.org/~/media/pdf/learning/capturing-value-knowledge-transfer.ashx [accessed 12 july 2015] polyaninova, t. (2011) "knowledge management in a project environment: organisational ct and project influences", vine, 41(3): 1-16. pretorius, c. & steyn, h. (2005) "knowledge management in project environments", south african journal of business management, 36(3): 41-50. reich, b. h. (2007) "managing knowledge and learning in it projects: a conceptual framework and guidelines for practice", project management institute, 38(2): 5-17. renzl, b. (2006) "trust in management and knowledge sharing: the mediating effects of fear and knowledge documentation", omega, 36: 206-220. rezania, d. & lingham, t. (2009) "towards a method to disseminate knowledge from the post project review", knowledge management research & practice, 7(2): 172-177. richardson, t. m., earnhardt, m. p. & marion, j. w. (2015) "is project management still an accidental profession? a qualitative study of career trajectory", sage open, 5(1): 1-10. rosacker, k. m. & rosacker, r. e. (2010) "information technology project management within public sector organizations", journal of enterprise information management, 23(5): 587-594. rowlands, b. h. (2005) "grounded in practice: using interpretive research to build theory", the electronic journal of business research methodology, 3(1): 81-92. sağsan, m. (2006) a new life cycle model for processing of knowledge management, paper presented at the 2nd international conference on business, management and economics, izmir, turkey. sandhu, m. s., jain, k. k. & bte ahmad, i. u. k. (2011) "knowledge sharing among public sector employees: evidence from malaysia", international journal of public sector management, 24(3): 206-226. santos, v., goldman, a., guerra, e., de souza, c. & sharp, h. (2013) a pattern language for inter-team knowledge sharing in agile software development, paper presented at the 20th conference on pattern languages of programs, monticello, il. australasian journal of information systems karagoz, korthaus & augar 2016, vol 20, research article managing project knowledge in the public sector 19 sarayreh, b., mardawi, a. & dmour, r. (2012) "comparative study: the nonaka model of knowledge management", international journal of engineering and advanced technology, 1(6): 45-48. schindler, m. & eppler, m. j. (2003) "harvesting project knowledge: a review of project learning methods and success factors", international journal of project management, 21(3): 219-228. schwartz, d. (ed.). (2005). encyclopedia of knowledge management. hershey, pa: igi global. seba, i., rowley, j. & delbridge, r. (2012) "knowledge sharing in the dubai police force", journal of knowledge management, 16(1): 114-128. sense, a. j. (2007) "structuring the project environment for learning", international journal of project management, 25(4): 405-412. sewchurran, k. & barron, m. (2008) "an investigation into successfully managing and sustaining the project sponsor–project manager relationship using soft systems methodology", project management journal, 39(1): 56-68. shavelson, r. j. & towne, l. (eds.). (2002). scientific research in education. washington dc, usa: national academies press. shokri-ghasabeh, m. & chileshe, n. (2014) "knowledge management: barriers to capturing lessons learned from australian construction contractors perspective", construction innovation, 14(1): 108-134. soliman, f. (2000) "application of knowledge management for hazard analysis in the australian dairy industry", journal of knowledge management, 4(4): 287-294. stettina, c. j., heijstek, w. & fægri, t. e. (2012) documentation work in agile teams: the role of documentation formalism in achieving a sustainable practice, paper presented at the agile conference (agile), dallas, texas. swan, j. & newell, s. (2000) linking knowledge management and innovation, paper presented at the 8th european conference on information systems, vienna. turner, j. (2009). the handbook of project-based management: leading strategic change in organizations (vol. 452): mcgraw-hill. von zedtwitz, m. (2002) "organizational learning through post–project reviews in r&d", r&d management, 32(3): 255-268. wallace, d. (2007) knowledge management: historical and cross-disciplinary themes, (2nd ed.), libraries unlimited inc, usa. wang, s. & noe, r. a. (2010) "knowledge sharing: a review and directions for future research", human resource management review, 20(2): 115-131. weber, r. (2004) "editor's comments: the rhetoric of positivism versus interpretivism: a personal view", mis quarterly, 28(1): iii-xii. whittaker, b. (1999) "what went wrong? unsuccessful information technology projects", information management & computer security, 7(1): 23-30. doi: doi:10.1108/09685229910255160 worthy, t. (2012) knowledge sharing among a community of project managers: a descriptive case study examining the relationship between social structures and knowledge sharing among project managers in a global engineering firm. (doctor of education), the george washington university, ann arbor, mi, usa. yao, l. j., kam, t. h. y. & chan, s. h. (2007) "knowledge sharing in asian public administration sector: the case of hong kong", journal of enterprise information management, 20(1): 51 – 69. australasian journal of information systems karagoz, korthaus & augar 2016, vol 20, research article managing project knowledge in the public sector 20 yin, r. k. (2009) case study research: design and methods, sage publications. yusof, z. m., ismail, m. b., ahmad, k. & yusof, m. m. (2012) "knowledge sharing in the public sector in malaysia a proposed holistic model", information development, 28(1): 43-54. zhang, m. j. (2007) "an empirical assessment of the performance impacts of is support for knowledge transfer", international journal of knowledge management, 3(1): 66-85. zhou, a. z. (2004) "managing knowledge strategically: a comparison of managers' perceptions between the private and public sector in australia", journal of information and knowledge management, 3(3): 213-222. zhou, a. z. & fink, d. (2003) "the intellectual capital web: a systematic linking of intellectual capital and knowledge management", journal of intellectual capital, 4(1): 34-48. copyright: © 2016 karagoz, korthaus & augar. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems samhan 2018, vol 22, research note revisiting technology resistance 1 revisiting technology resistance: current insights and future directions bahae samhan illinois state university bmsamha@ilstu.edu abstract nuance conceptualization of technology resistance is needed due to the different conceptualizations that exist in the extant literature. this paper calls for information systems (is) researchers to begin exploring technology resistance using novel perspectives. the paper also provides a guideline for conceptualizing technology resistance away from the common existing conceptualization that is shaped through theories of adoption and resistance-tochange. the goal is to encourage future research to conceptualize technology resistance beyond the typical findings available in the literature and provide more insights in light of the provided arguments and suggestions. keywords: technology resistance, concept, current insights, future directions. 1 introduction the implementation of information systems (is) has been steadily increasing within different industries. however, one of the main issues related to new technology is the difficulty of identifying significant factors that influences potential users to accept technologies developed and implemented by others (lim & ting, 2012). nevertheless, there are evidence of technology resistance behaviours conducted by the same people who are expected to use the technology the most (samhan, 2016). technology resistance is found to be a prominent reason for system failure (kim & kankanhalli, 2009). an example, reported by bhattacherjee et al. (2013), was from the cedars-sinai medical center in los angeles where doctors resisted to use the newly installed computerized physician order entry system (cpoe), which caused the system to fail and resulted in a complete withdrawal after it has been already implemented in two-thirds of the 870-bed hospital. this highlights the importance of understanding technology resistance causes and possible remedies. for the past three decades, is research has extensively investigated concepts related to the technology use and adoption while giving less attention to technology resistance topics. despite the wide recognition of the technology resistance phenomenon, research in this field is still somewhat immature (laumer & eckhardt, 2012). prior research in is suggest a lack of research on factors that influence technology resistance (samhan & joshi, 2017). in general, major is studies that referred to technology resistance as an important issue can be categorized into two main research streams: (1) studies explaining resistance as the opposite of adoption (e.g. markus, 1983; martinko et al., 1996; cenfetelli, 2004) and (2) studies focusing on resistance to change (e.g. hirschheim and newman, 1988; joshi, 1991; kim & kankanhalli, 2009; marakas & hornik, 1996). laumer and eckhardt (2012) argue that resistance cannot be simply considered as “the reverse side of the acceptance coin.” therefore, studying acceptance alone will do little to provide insights into user resistance. caroll et al. (2003) argue that technology acceptance and rejection are poorly understood and need to be investigated independently. lapointe and rivard (2005) australasian journal of information systems samhan 2018, vol 22, research note revisiting technology resistance 2 found that is research has neglected the study of resistance phenomena. their literature analysis of is journals over the prior 25 years revealed that only 43 studies considered resistance as a key issue. these studies viewed resistance as a “black box” and only 9 of the 43 studies defined concepts of resistance. further, only 4 studies explored the black box of resistance and suggested theoretical explanation of resistance (i.e. joshi, 1991; marakas & hornik, 1996; markus, 1983; martinko et al., 1996). however, the work of venkatesh and brown (2001) broaden the technology adoption perspective by presenting preliminary evidence that non-adoption decisions are based on different critical barriers. additionally, in the second stream of research, we find that resistance to change studies focused on users’ perspectives of the organizational and environmental agents influencing their decision of complying or resisting the change. this includes concepts related to the change such as locus of control, difference in power, net-gain, etc. while these studies provide important insights about individuals’ perspective to it-induced change, they fall short from explaining users’ perceptions of the technology itself. therefore, this research note serves as a call for is researchers to direct focus on technology resistance behaviours. this includes exploring concepts related to the technology features, users’ beliefs and attitudes about the technology and the general effects of the tasks accomplished using the technology. 2 conceptualization of technology resistance according to the oxford english dictionary, resistance is defined as the action of resisting, which means withstanding an action or effect and trying to prevent by action or argument. prior is research defined technology resistance as an action or intentional inaction that opposes the implementation of new technology (laumer & eckhardt, 2012). lapointe and rivard (2005) suggest that users evaluate technology in terms of its features and existing conditions (as an individual and as an organization) and based on this evaluation users can make projections about the consequences of the potential use of the technology. this leads to an intention to accept or an intention to resist the technology based on perceived value and threat associated with the technology (lapointe and rivard, 2005). according to is literature, technology resistance is characterized by low levels of use, by a lack of use, or by dysfunctional (harmful) use (martinko et al. 1996). additionally, technology resistance may be manifested by an individual, a group or an entire organization (saga & zmud 1993). in general, is research does not consider resistance as dysfunctional. the definition of resistance specifically states that it may occur over time (laumer & eckhardt, 2012). however, knight and burn (2011) argue that technology adoption theories fail to capture users’ behaviour overtime—that is, the technology acceptance literature based on the intention to adopt suggests that a user may develop intentions to adopt or resist the technology based on the outcome of evaluating the acceptance and resistance arguments (joshi, 1991). moreover, resistance behaviours could be distinguished between rejection and non-adoption. rejection refers to the conscious decision made by the users to avoid the technology, as opposed to non-adoption, which implies potential future use (cenfetelli, 2004). after the adoption stage, users can develop different usage behaviours based on their experience and continuous evaluation of the technology, which may include the manifestation of resistance behaviour towards the technology (ferneley & sobreperez, 2006). australasian journal of information systems samhan 2018, vol 22, research note revisiting technology resistance 3 furthermore, it is important to recognize that resistance behaviour exists on a spectrum of various behaviours from the absence of the will to engage to the act of becoming physically destructive (carnall, 1986; marakas & hornik, 1996; lapointe & rivard, 2005). additionally, technology resistance may be manifested as a form of avoidance or workaround strategies (samhan & joshi, 2017). to put this understanding into perspective, the conceptualization of technology resistance can be summarized as follows: the conceptualization of technology resistance encapsulates the actions or intentional inactions that oppose adoption and usage of a particular technology, from the absence of willing to engage to becoming physically destructive, such as avoidance, rejection, and workaround strategies leading to lack, low, and sometimes dysfunctional levels of use, among others, which may be affected by the users’—as an individual, as a group, or as an organization—perceptions and evaluations of value and threat posed by that technology. 3 contextualization of technology resistance outcomes of interactions with the same technology will be different when tested in different settings (boiney, 1998). gopal and prasad (2000, p. 512) suggested that “technology cannot be studied outside its social context and that inconsistent results may be directly related to our lack of attention to this fact.” therefore, understanding the context of the study, in which resistance is being evaluated within, is a vital process. orlikowski and iacono (2001) argued that conceptualizing technology is unique to its context, and thus, what may be considered a resistance behaviour in one context may not be equally true when implementing similar technology in a different context. one reason may be related to how outcome demands and expectations may differ across contextual groups (hong et al., 2013). further, the context of the study’s discipline and the characteristics of the type of users are important attributes of is research (hevner et al., 2004). is researchers must also consider the different levels in which resistance may occur. in some cases, resistance may be manifested by individuals, which normally occurs when resistance is studied using a psychological lens (e.g. joshi, 1991; martinko et al., 1996; marakas & hornik, 1996). in other cases, technology resistance may be manifested by groups or organizations (e.g. lapointe & rivard, 2006; markus, 1983; ang & pavri, 1994). furthermore, the type of technology enforcement must be considered as well (i.e., voluntary vs. mandatory, utilitarian vs. hedonic systems, etc.) (laumer & eckhardt, 2012). to put this understanding into perspective, the contextualization considerations of technology resistance can be summarized as follows: the contextualization of technology resistance necessitates considerations pertaining to (1) the demands and expectations of contextual groups, such as the types of is users, (2) the levels of resistance, such as at the individual, group or organizational level, and (3) the type of technology enforcement, such as voluntary versus mandatory enforcement on utilitarian versus hedonic systems, among others. 4 antecedents of technology resistance one of the first antecedents of technology resistance (or resistance to change) was found by ang and pavri (1994). other prior is research explained technology resistance based on beliefs australasian journal of information systems samhan 2018, vol 22, research note revisiting technology resistance 4 and attitudes toward the technology (e.g. martinko et al., 1996; cenfetelli, 2004; lapointe & rivard, 2005). the main beliefs explored in is research include perceived threats, technology inhibitors, and loss of power. another important antecedent to resistance is social influence. prior studies in is that evaluated the impact of social influence on resistance have mostly focused on influence stemming from workplace peers only (e.g. kim & kankanhalli, 2009; eckhardt et al. 2009), and ignored other sources of influence, such as workplace superiors (managers) and/or private referents, such as friends (samhan, 2016). thus, further research that considers exploration and investigations into other referent groups is encouraged to provide a better understanding of social influence on technology resistance. moreover, limited research in is considered individual differences, such as age, gender and levels of education when exploring antecedents of resistance (laumer & eckhardt, 2012; samhan & joshi, 2014). individual differences can influence one’s attitude toward the technology (venkatesh & morris 2000). moreover, user resistance may be predicted through other personal characteristics, such as risk tolerance (judge et al. 1999), believed levels of control (lau and woodman 1995), self-esteem (wanberg & banas 2000) and the desire to achieve (miller et al. 1994), and thus further research in this direction should be potentially fruitful. 5 technology resistance theories and findings in is research while there are various perspectives on technology resistance in is research, there is a general agreement that understanding resistance is important (lapointe & rivard, 2005; kim & kankanhalli, 2009). table 1 provides an overview of technology resistance theories and models in is research. theory description resistance to change (joshi, 1991) this model uses the equity theory perspective to examine resistance to change. the equity-implementation model identifies a three-level process to evaluate change in terms of its effect on users’ equity status. to assess the change in equity, users evaluate the net-gain associated with changes in their inputs and outcomes and comparing their relative outcomes with that of other users and the employer. user resistance model (kim and kankanhalli, 2009) this theory develop the construct “user resistance” and use the theory of status quo bias to explain user resistance prior to a new is implementation. it explored resistance to change based on perceived value and its antecedents, organizational support, self-efficacy and social influence. revealed causal map of resistance (samhan & joshi, 2017) this model was revealed using qualitative analysis of interviewing healthcare providers about a newly implemented electronic health records system. the model suggest that resistance is shaped by perceived value, perceived threat and circumvention availability. multilevel model of resistance to it implementation (lapointe & rivard, 2005) in this model, perceived threat is the main antecedent to resistance behaviors. it results from the interaction between initial conditions and a given object. australasian journal of information systems samhan 2018, vol 22, research note revisiting technology resistance 5 theory description power, politics and mis implementation (markus, 1983) this model explains resistance in terms of interaction between the technology and its context. it suggests that a group of actors will be inclined to use the technology if they believe it will support their position of power. if a user thinks it might cause lose in power, he/she will manifest resistance. passive resistance misuse (marakas & hornik, 1996) this model explains resistance behaviors as passive–aggressive responses to threats that users associate with a new system. the introduction of new technology exposes the rigidity of users toward change, and coupling it with feelings of resentment will lead to resistance behaviors. an attributional explanation of resistance (martinko et al., 1996) this theory suggest that variables associated with technology resistance can be conceptualized using an attributional perspective of achievement motivation. this model draws on the attribution theory, which suggests that new technology, internal and external variables, and an individual’s experience evoke causal attributes. the theory suggests that the intensity of technology depends on the interaction of these factors. the dual factor model of technology usage (cenfetelli, 2004) this theory suggest that inhibitors discourage usage and enablers encourages usage. however, the absence of inhibitors does not encourage use. the inhibiting beliefs are independent from enabling beliefs and therefore can coexist and have consequent effects. physicians’ resistance toward hit (bhattacherjee & hikmet, 2007) this model incorporates the notion of resistance to change and its antecedents into a model of it usage to explain physicians’ reactions to healthcare information technology (hit). it mainly explains resistance through the perceived threat lens. table 1 is theories on technology resistance figure 1 illustrates the current and future is research insights derived from a review of prior research on technology resistance, including that which used is resistance theories. specifically, the figure depicts revealed reasons for technology resistance, methods to overcome technology resistance, current insights, extant gaps, and suggestions for future research. australasian journal of information systems samhan 2018, vol 22, research note revisiting technology resistance 6 figure 1 current and future is research on technology resistance the review of the extant research on technology resistance in the is literature reveals several implications for future research in this area. first, technology resistance is a multifaceted construct, and thus is scholars should aim for a more comprehensive characterization of technology resistance. second, a unified understanding of technology resistance is lacking in is research, and thus suggesting a need for theoretical advancements in this area. to start unification work in this area, a useful starting point is to examine and build upon the multilevel model of resistance (lapointe & rivard, 2005). third, the limited connections and explanations between technology resistance (rather than implementation) and organizational change points to an area that needs further exploration. fourth, fourth, existing studies on technology resistance have so far been concentrated on the beliefs and attitudes contributing to technology resistance, such as loss of power (martinko et al., 1996), technology inhibitors (cenfetelli, 2004), and perceived threat (bhattacherjee & hikmet, 2007), with limited work exploring the outcome technology resistance. given that most is researchers have only gone as far as conceptualizing user resistance as an outcome variable (kim and kankanhalli, 2009; laumer & eckhardt, 2012; oreg, 2006), future research on technology resistance in is should consider extending the investigated set of outcome-related constructs, such as continued commitment, job satisfaction and turnover intentions, among others. australasian journal of information systems samhan 2018, vol 22, research note revisiting technology resistance 7 fifth, existing research on technology resistance has focused on explaining technology resistance using theoretical lens pertaining to resistance to change and non-adoption (laumer and eckhardt, 2012; oreg, 2006). indeed, these are important aspects of technology resistance, but it is also important that future research consider and use alternative theoretical lenses to reveal novel explanations of why people reject technologies (see table 1 and figure 2, which builds on the review by laumer and eckhardt [2012], for alternative theoretical lenses and antecedent-consequence-intervention relationships, and psychology and sociology journals for new ideas). sixth, existing studies on technology resistance have shown that social influence plays an important role in shaping technology resistance behaviour (kim & kankanhalli, 2009; samhan & joshi, 2017; eckhardt et al., 2009). however, these studies have only considered workplace referents as the only source of social influence, and ignoring private referents, such as family or friends. thus, future studies in this area are encouraged to consider further referent groups when aiming to frame social influence and its impacts in technology resistance research. such studies should also account for potential differences in the context of the investigated technology (e.g. voluntary versus mandatory utilitarian versus hedonic technology), as well as individual differences, such as age, gender, tenure, and educational background, especially if there is a potential that these differences can influence the evaluation and attitude toward that investigated technology (venkatesh & morris, 2000). 6 conclusion a lack of unified conceptualization of technology resistance will impede scholarship in this area (samhan & joshi, 2015). this call for research highlighted the need for a deep and rich unified understanding of technology resistance. new perspectives of how and why technologies are being resisted may emerge when evaluating the resistance behaviour toward the technology itself rather than toward any other object of resistance such as change. future research needs to pay attention to the different forms of resistance manifestation, the context in which resistance is being evaluated within, the object of resistance, and the potential antecedents that may predict resistance. future researchers are encouraged to discuss how technology resistance may lead to different outcome variables by incorporating other potential outcomes, such as continuous commitment, turnover intentions, and job satisfaction, among others. references aggarwal, a. (1998). end user training – revisited. journal of end user computing, 10(3), 3233. ang, j., & pavri, f. (1994). a survey and critique of the impacts of information technology. international journal of information management, 14(2), 122-133. doi: https://doi.org/10.1016/0268-4012(94)90031-0 bhattacherjee, a., & hikmet, n. (2007). physicians' resistance toward healthcare information technology: a theoretical model and empirical test. european journal of information systems, 16(6), 725-737. doi: https://doi.org/10.1057/palgrave.ejis.3000717 bhattacherjee, a., davis, c., & hikmet n. (2013). physician reactions to healthcare it: an activity-theoretic analysis. 46th ieee hawaii international conference on system sciences (hicss), maui, hi. doi: https://doi.org/10.1109/hicss.2013.448 australasian journal of information systems samhan 2018, vol 22, research note revisiting technology resistance 8 boiney, l. g. (1998). reaping the benefits of information technology in organizations: a framework guiding appropriation of group support systems. the journal of applied behavioral science, 34(3), 327-346. doi: https://doi.org/10.1177/0021886398343006 carnall, c. a. (1986). toward a theory for the evaluation of organizational change. human relations, 39(8), 745-766. doi: https://doi.org/10.1177/001872678603900803 carroll, j., howard, s., peck, j., & murphy, j. (2003). from adoption to use: the process of appropriating a mobile phone. australasian journal of information systems, 10(2), 38-48. doi: https://doi.org/10.3127/ajis.v10i2.151 cenfetelli r.t. (2004). inhibitors and enablers and dual factor concepts in technology usage. journal of the association of information systems, 5(11), 472–492. de jager, p. (1994). communicating in times of change. journal of systems management, 45(6) 28-30. eckhardt, a., laumer, s., & weitzel, t. (2009). who influences whom? analyzing workplace referents' social influence on it adoption and non-adoption. journal of information technology, 24(1), 11-24. doi: https://doi.org/10.1057/jit.2008.31 ferneley, e., & sobreperez, p. (2006). resist, comply or workaround? an examination of different facets of user engagement with information systems. european journal of information systems, 15(4), 345-356. doi: https://doi.org/10.1057/palgrave.ejis.3000629 ginzberg, m. (1975). implementation as a process of change: a framework and empirical study. rept. cisr-13, center for information systems research, massachusetts institute of technology, cambridge, uk. gopal, a., & prasad, p. (2000). understanding gdss in symbolic context: shifting the focus from technology to interaction. mis quarterly, 24(3), 509-546. doi: https://doi.org/10.2307/3250972 hevner a., march s., & park, j. (2004). design science in information systems research. mis quarterly, 28(1), 75–105. doi: https://doi.org/10.2307/25148625 hirschheim, r., & newman, m. (1988). information systems and user resistance: theory and practice. computer journal, 31(5), 398-408. doi: https://doi.org/10.1093/comjnl/31.5.398 holmes, t., & holmes, t. (1970). short-term intrusions into lifestyle routines. journal of psychosomatic research, 14(1), 121-132. doi: https://doi.org/10.1016/0022-3999(70)90022-x hong, w., chan, f., thong, j., chasalow, l., & dhillon, g. (2013). a framework and guidelines for context-specific theorizing in information systems research. information systems research, 25(1), 111-136. doi: https://doi.org/10.1287/isre.2013.0501 hussain, d., & hussain k. (1984). information resource management, irwin, homewood, il. janson, m., woo, c., & smith, l. (1993). information systems development and communicative action theory. information & management, 25(1), 59-72. doi: https://doi.org/10.1016/0378-7206(93)90048-x joshi, k., (1991). a model of user perspective on change: the case of information systems technology implementation. mis quarterly, 15(2), 229-242. doi: https://doi.org/10.2307/249384 australasian journal of information systems samhan 2018, vol 22, research note revisiting technology resistance 9 judge, t., thoresen, c., pucik, v., & welbourne, t. (1999). managerial coping with organizational change: a dispositional perspective. journal of applied psychology, 84(1), 107-122. doi: https://doi.org/10.1037/0021-9010.84.1.107 keen, p. (1981). information systems and organizational change. communications of the acm, 24(1), 24-32. doi: https://doi.org/10.1145/358527.358543 klein, k., hall, r., and laliberte, m. (1990). training and the organizational consequences of technological change: a case study of computer-aided design and drafting. in u.e. gattiker, u. e. & larwood, l. (eds.), technological innovation and human resources: end-user training (pp. 7-36), de gruyter, new york. knight, s., & burn, j. (2011). a preliminary introduction to the otam: exploring users' perceptions of their on-going interaction with adopted technologies. australian journal of information systems, 17(1), 5-41. doi: https://doi.org/10.3127/ajis.v17i1.541 kim, h., & kankanhalli, a. (2009). investigating user resistance to information systems implementation: a status quo bias perspective. mis quarterly, 33(3), 567-582. doi: https://doi.org/10.2307/20650309 land, f. (1992). the management of change: guidelines for the successful implementation of information systems. in brown, a. (eds.), creating a business-based it strategy (pp. 145157), chapman & hall, london, uk. lawler, e., & mohrman, s. (1991). quality circles: after the honeymoon. in staw, b. m. (ed.), psychological dimensions of organizational behavior (pp. 144-173), macmillan, new york. lapointe, l., & rivard, s. (2005). a multilevel model of resistance to information technology implementation. mis quarterly, 29(3), 461-491. doi: https://doi.org/10.2307/25148692 lapointe, l., & rivard, s. (2006). getting physicians to accept new information technology: insights from case studies. canadian medical association journal, 174(11), 1573-1578. doi: https://doi.org/10.1503/cmaj.050281 lau, c., & woodman, r. (1995). understanding organizational change: a schematic perspective. academy of management journal, 38(2), 537-554. doi: https://doi.org/10.2307/256692 laumer, s., & eckhardt, a. (2012). why do people reject technologies: a review of user resistance theories. in information systems theory (pp. 63-86). springer, new york, ny. doi: https://doi.org/10.1007/978-1-4419-6108-2_4 liang, h., & xue, y. (2009). avoidance of information technology threats: a theoretical perspective. mis quarterly, 33(1), 71-90. doi: https://doi.org/10.2307/20650279 lim, w., & ting, d. (2012). e-shopping: an analysis of the technology acceptance model. modern applied science, 6(4), 49-62. doi: https://doi.org/10.5539/mas.v6n4p49 marakas, g., & hornik, s. (1996). passive resistance misuse: overt support and covert recalcitrance in is implementation. european journal of information systems, 5(3), 208-219. doi: https://doi.org/10.1057/ejis.1996.26 markus m. (1983). power, politics, and mis implementation. communications of the acm, 26(6), 430-444. doi: https://doi.org/10.1145/358141.358148 australasian journal of information systems samhan 2018, vol 22, research note revisiting technology resistance 10 martinko, m., henry, j., & zmud, r. (1996). an attributional explanation of individual resistance to the introduction of information technologies in the workplace. behavior and information technology, 15(5), 313-330. doi: https://doi.org/10.1080/014492996120085a martinsons, m., & chong, p. (1999). the influence of human factors and specialist involvement on information systems success. human relations, 52(1), 123-152. doi: https://doi.org/10.1177/001872679905200107 miller, v., johnson, j., & grau, j. (1994). antecedents to willingness to participate in a planned organizational change. journal of applied communication research, 22(1), 59 80. doi: https://doi.org/10.1080/00909889409365387 mumford, e. (1979). human values and the introduction of technological change. manchester business school review, 3(2), 13-17. nord, w., & tucker, s. (1987). implementing routine and radical innovation. free press, lexington books, lexington, ma. oreg, s. (2006). personality, context, and resistance to organizational change. european journal of work & organizational psychology, 15(1), 73-101. doi: https://doi.org/10.1080/13594320500451247 orlikowski, w., & iacono, c. (2001). research commentary: desperately seeking the "it" in it research—a call to theorizing the it artifact. information systems research, 12(2), 121134. doi: https://doi.org/10.1287/isre.12.2.121.9700 rousseau, d. (1989). managing the change to an automated office: lessons from five case studies. technology & people, 4(1), 31-52. doi: https://doi.org/10.1108/eb022653 rivard, s. (1984). successful implementation on end-user computing. interfaces, 17(3), 25-33. doi: https://doi.org/10.1287/inte.17.3.25 rivard, s., & lapointe, l. (2012). information technology implementers' responses to user resistance: nature and effects. mis quarterly, 36(3), 897-920. saga, v., & zmud, r. (1993). the nature and determinants of it acceptance, routinization, and infusion. in proceedings of the ifip tc8 working conference on diffusion, transfer and implementation of information technology (pp. 67-86), elsevier science, pittsburg, pa. samhan, b., & joshi, k.d. (2015). resistance of healthcare information technologies; literature review, analysis, and gaps. in 2015 48th hawaii ieee international conference on system sciences (hicss) (pp. 2992-3001). ieee, kauai, hi. samhan, b. (2017). why do people resist patient portal systems?: an application of the dual factor model of it usage. international journal of healthcare information systems and informatics, 12(4), 68-86. doi: https://doi.org/10.4018/ijhisi.2017100105 samhan, b., & joshi, k. d. (2017). understanding electronic health records resistance: a revealed causal mapping approach. international journal of electronic healthcare, 9(2), 100-128. doi: https://doi.org/10.1504/ijeh.2017.083163 samuelson, w., & zeckhauser, r. (1988). status quo bias in decision making. journal of risk and uncertainty, 1(1), 7-59. doi: https://doi.org/10.1007/bf00055564 smith, h., & mckeen j. (1992). computerization and management: a study of conflict and change. information & management, 22(1), 53-64. australasian journal of information systems samhan 2018, vol 22, research note revisiting technology resistance 11 swanson e. (1988). information systems implementation: bridging the gap between design and utilization. mcgraw hill, irwin, homewood, il. venkatesh, v., & morris, m. (2000). why don't men ever stop to ask for directions? gender, social influence, and their role in technology acceptance and usage behavior. mis quarterly, 24(1), 115-139. doi: https://doi.org/10.2307/3250981 venkatesh, v., & brown, s. (2001). a longitudinal investigation of personal computers in homes: adoption determinants and emerging challenges. mis quarterly, 25(1), 71-102. doi: https://doi.org/10.2307/3250959 venkatesh, v., morris, m., davis, g., davis, f. (2003). user acceptance of information technology: toward a unifying view. mis quarterly, 27(3), 425-478. doi: https://doi.org/10.2307/30036540 wanberg, c., & banas, j. (2000). predictors and outcomes of openness to changes in a reorganizing workplace. journal of applied psychology, 85(1), 132-142. doi: https://doi.org/10.1037/0021-9010.85.1.132 copyright: © 2018 samhan. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 conceptualization of technology resistance 3 contextualization of technology resistance 4 antecedents of technology resistance 5 technology resistance theories and findings in is research 6 conclusion paper title (paper title style) australasian journal of information systems dobson & jackson 2017, vol 21, selected papers from acis 2016 explaining broadband non-adoption in rural australia 1 using critical realism and reflexivity to explain broadband non-adoption in rural australia philip dobson centre for innovative practice school of business & law edith cowan university p.dobson@ecu.edu.au paul jackson centre for innovative practice school of business & law edith cowan university abstract australia is conducting a substantial nationwide provision of broadband. it is primarily a fixed line network but includes wireless and satellite networks in more remote areas. the rollout is under the control of the nbn co, whose goal is ensuring access to fast broadband for all australians. the nbn co has recently recognized the importance of adoption by including premises activated as a kpi alongside service provision. coverage and adoption reflect the two faces of the nbn rollout – as a technical program and as a social program. adoption will be the ultimate measure of the success of the nbn as a social program. ubiquitous internet adoption across all sectors is necessary to maximize the promised benefits. the adoption of broadband is an important first step in achieving this. international experience suggests that broadband adoption is plateauing and this has prompted our proposal that there needs to be focus on nonadoption rather than adoption. we suggest that it is important to be able to explain the mechanisms by which individuals respond to the promise of the internet. only by so doing can we address issues. we contend that there needs to be more focus on those disenchanted or disinterested “non-users” who are never likely to adopt without specific targeted strategies. we argue for a critical realist perspective, more particularly reflexivity, to better represent the adoption context and to provide a grounding for explanations of the causes behind the decision not to adopt. we also propose possible common-sense strategies to reverse non-adoption. keywords: broadband adoption, social realism, modes of reflexivity, technology acceptance model 1 introduction announced as a key policy by the labor party prior to the 2007 election in australia, the national broadband network (nbn) is being implemented across australia. whilst it is mainly a fixed line network using mixed copper and optical technology with fibre to the node, wireless and satellite networks are also used to supply services to remote areas. whilst being an instrument of industrial policy which will facilitate production and consumption, the original motivation also embodied a substantial equity motive, connecting those in sparsely populated remote and less affluent regions which would otherwise not be economically viable to connect (cave and martin, 2010). the nbn co was founded in 2009 to manage the rollout. their objective is to deliver broadband to all australians. the benefits of such a service are acknowledged, with many countries worldwide recognizing broadband service or internet service as a legal right, human right or citizen’s right (broadband commission 2015). the principle of a universal service is that all users should be provided with a range of basic but good quality services at affordable prices. the national australian telecommunications operator telstra has had a universal services obligation (uso) with respect to telephony provision for many years and more recently there has been a call to apply such a uso to broadband on completion of the current national rollout. this uso needs to australasian journal of information systems dobson & jackson 2017, vol 21, selected papers from acis 2016 explaining broadband non-adoption in rural australia 2 focus on what is provided rather than how – the provision of internet services can take many forms including fixed line, satellite, wifi and media not invented yet (cradduck 2016). a recognition of the benefits of universal provision, and perhaps a uso, is an important first step in achieving promised internet benefits. however, it is important to distinguish between the provision of internet services and the adoption/activation of such technologies. this has clearly been appreciated by the nbn co in their acceptance of two key performance indicators service provision (homes and businesses serviceable) and premises activated. it is suggested that activations are exceeding expectations with recent reports showing approximately 2.9 million premises made ready for service and 1.1 million activated (nbn co, 2016). horrigan and duggan (2015) examine the adoption of broadband in the home in the us and suggest that adoption levels are plateauing at around 70%; this plateauing a consequence of the increasing adoption of mobile broadband. perrin and duggan (2015) more generally analyse internet usage for american users over 2000-2015. they suggest that some sectors are reaching near saturation levels whilst other sectors are lagging. lagging sectors include rural regions, the elderly, lower income households and african-americans and hispanics. we suggest that this plateauing is a consequence of a core of “non-adopters” rather than so called “laggards”. we argue that in order to address non-adoption we need to focus on the characteristics of non-adopters and understand the individual motivations behind the decision not to adopt. in this paper we focus particularly on rural adoption where despite strong promotion, reasonable pricing and social and economic benefits, adoption of broadband and its alternatives have been disappointing and well behind that of urban regions. larose et al (2007, p. 360) suggests the rural benefits of broadband are substantial including the fostering of social interactions to increase attachment to rural communities and reduce out-migration; enhancing economic opportunities by stimulating the development of home businesses; and improving rural access to health care and education. using rural america as an example they suggest an important paradox in that the region most to benefit is lagging behind in adoption. we suggest that the rollout of large-scale technology platforms, such as the nbn, need to be considered as importantly a social program as well as a technical program. this different focus requires a careful contextual examination of the social reality, or ontology, of agents both in terms of the context within which the adoption decision is made and in the way that agents consider this context in relation to their personal “projects” and their own particular ways of behaving. as archer (2003) suggests, people always have the possibility to do otherwise than expected or predicted – this unexpected behaviour depends largely on their own reflexive deliberations – their “internal conversation”. it is this reflexive engagement of purposive agents with their existing social contexts that needs to be understood in order to explain the reasons behind non-adoption. non-adoption is a significant barrier to achieving many of the benefits of ubiquity (such as access to many government services, particularly medical or educational). it is important that the non-adopters be better understood. we suggest that particular modes of reflexivity are associated with particular types of non-use and propose particular strategies for these different categories of non-use. we suggest that only by understanding the social reality of agents and their reflexivity (impacted by their natal, experiential and social history) can we hope to be able to explain and possibly address this rejection of broadband and its alternatives. we first examine some types of non –use and suggest that reflexivity is an important mechanism for explaining non-adoption. in particular, we suggest that two modes of reflexivity will explain much of the non-adoption (as described later in the paper these are the “communicative” and the “fractured”). we critique the dominant tam based research models used to predict and evaluate adoption patterns, suggesting that the lack of a clear ontology for these models limits their application to the undoubted social complexity of internet adoption and usage. we then suggest ways for promoting broadband targeted at particular reflexive modalities. we consider the context of rural adoption to better highlight our arguments. australasian journal of information systems dobson & jackson 2017, vol 21, selected papers from acis 2016 explaining broadband non-adoption in rural australia 3 2 focusing on the non-user satchell and dourish (2009) describe a range of varieties of non-use including lagging adoption, active resistance, disenchantment, disenfranchisement, displacement and disinterest. the concept of lagging adopters can be seen to be prevalent in much of the internet adoption literature to date and in the statistics used to examine adoption. much of the literature discusses the benefits of universal adoption and the desirability of improving the factors supporting adoption they implicitly assume that 100% adoption is possible. yet the adoption of broadband or its alternatives is ultimately a voluntary choice for many, home use being perhaps more voluntarist than business in that businesses, through competitive threats, might generally be expected to adopt the internet promptly for commercial reasons. non-use is not all about laggards, active resistance is an option for some users in that they may actively reject the internet for various reasons such as concern for privacy. similarly disenchantment may be reflected by a limited or partial use of internet applications as a reflection of nostalgic regret for things changing. disenfranchisement or exclusion can take many forms as satchell and dourish (2009, p. 12) suggest: “interest in universal accessibility has largely focused on physical and cognitive impairments as sources of technological disenfranchisement, but it may also have its origins in economic, social, infrastructural, geographical, and other sources”. the investigation of disenfranchisement depends heavily on an understanding of contextual social and personal situations and the particular material and ideational structures in place. similarly displacement is an important concept in that many “users” will depend on different material and social intermediaries to achieve their internet connection. family members may depend on “local experts” (stewart 2007) to fulfil their internet requirements. such a variation of “non-use” is difficult to quantify, but by applying the notion of reflexivity, it can be explained by reference to the “communicative reflexive” who depends on “similars and familiars” to assist their daily obligations and desires. such degrees of “non-use” are difficult to get a grip on when the dominant model for understanding adoption is considering a person with their own computer and not some public provision or private arrangement. the concept of “user” is challenged by such use or “non-use”. the final category of disinterest is perhaps the most difficult challenge for researchers and those promoting broadband in that the non-user may have no interest in the things that you assume they are interested in. as will be described below this is the domain of the “fractured” reflexive – those who are not participants in society but rather victims. converting such a group is important since for many these persons have the most to gain, both economically and socially. the crucial constraint in current adoption models is that they cannot properly represent these non-adopters nor make clear the causes behind the non-adoption. such consideration often requires a careful deep analysis of the social and contextual: simple factors models cannot provide this depth. much more sophisticated models are needed to reflect the complexity of adoption and non-adoption. in particular current research models have no recognition of the different types of user and how their different life worlds affect the adoption decision. we suggest the consideration of reflexivity to better explain non-adoption. 3 the role of reflexivity we propose a focus on non-adopters and suggest the importance of reflexivity as a mechanism to help explain the non-adoption decision. the characteristics of the potential adopter need to be considered, as well as the context. potential users differ in their cultural and economic environments, personal characteristic, their capabilities and their life-concerns. as kontos and australasian journal of information systems dobson & jackson 2017, vol 21, selected papers from acis 2016 explaining broadband non-adoption in rural australia 4 poland (2009) suggest when examining social health care improvements, there is a need to reflect human choice along with context in examining government programs: in seeking to understand how mechanisms play out in a particular setting, with particular agents at a specific time, we must also take account of how reflexive agents perceive, negotiate, unwittingly reinforce or selectively resist the effects of these broader trends and influences in the context of their own life biographies, socialization, and the micro-social context of peer relations in the workplace (p6). recent research in the social sciences suggests that individual reflexivity (termed the “internal conversation” by archer (2003, 2007)), driven by personal biography, context and personal concerns, provides useful information about how individuals engage with information and decision-making. reflexivity is defined as the “regular exercise of the mental ability, shared by all normal people, to consider themselves in relation to their (social) contexts and vice versa (archer, 2007, p4). as garcia-ruiz and rodriguez-lluesma (2009) describe: human reflexivity becomes important as the linkage between concerns, projects and practices. we act in order to promote our concerns, and form projects to advance or to protect what we care about most. it is through our internal conversations that we reflexively define the courses of action conducive to the realization of our ultimate concerns in an appropriate modus vivendi [mode of living]. what people seek to do is reflexively defined by reference to the concerns they wish to realize. this means establishing practices, both satisfying to and sustainable by the subject, in an appropriate social environment (archer 2007, p. 88). hence, to understand the meaning of those practices it is necessary to grasp the life-projects in which they are embedded, as well as the ultimate concerns that underlie such projects (p. 223). archer (2010) asserts that most western social theorising has “regarded reflexivity as a more or less homogenous phenomenon” (2010, p5) which when applied in similar circumstances would lead to the same outcomes. this, of course, has induced analysts to search for the “silver bullet”, the “critical success factors” which explain what most people will do most of the time – thus for broadband we are tempted to propose important identifiable supportive adoption factors such as price, speed, availability, ease of use and so on. these are identifiable external factors and are important considerations in the adoption decision. but such determinants need to be considered alongside the individual’s internal reflexivity seen as an inner dialogue. individuals always have the possibility to do otherwise than deterministic logic dictates. archer (2000, 2003, 2007, 2012) describes the importance of this mechanism, largely seen to be a consequence of the natal context, in explaining agents’ life decisions. in a series of books she examines the role of reflexivity in today’s society and proposes a number of dominant reflexive modes that can help to explain people’s ultimate concerns and how these concerns impact their life choices. in particular, archer (2000) explores the dynamics of reflexivity and places the “ultimate concerns” of the individual within “three orders of reality” which shape the outcomes of their “internal conversations”: these are the natural, the practical and the social (p. 197). the natural world encompasses that which must be navigated to ensure physical well-being: avoiding hunger and stubbing your toe. the practical world demands that we take steps to get things done: to catch a bus or catch a deer, to use a monthly ticket or a bow and arrow. the social world provides normative directives and notions of self-worth and aspiration. the self (providing the “necessary anchorage” for the application of reflexivity) moves between these strata in making decisions or rationalising them away. it is this conception of individual agency that allows us to locate decisions to adopt broadband within a historical and dialectical context. unlike a computer or decision tree that will identify, weight and process criteria immediately prior to the event, the application of reflexivity is a mode of deciding which unfolds from a preceding timeline in which physicality, emotion, active practice and reason have been engaged. australasian journal of information systems dobson & jackson 2017, vol 21, selected papers from acis 2016 explaining broadband non-adoption in rural australia 5 3.1 modes of reflexivity the notion of reflexivity in explaining the behaviours of agents and actors as developed by archer is both simple and powerful (mutch, 2007). it characterises individuals as variously adopting communicative, autonomous, meta and fractured modes depending on their personal concerns, projects and practices: • the communicative reflexive inhabits a coherent, stable social world that is constantly reinforced by reiteration and external conversations with others, re-establishing and reinforcing the status quo. • the autonomous reflexive is less reliant on others, more dependent on internal conversation and shuns predictability through this self-reliance and limited dependence on the “similars and familiars” that support and shape the conservatism of the communicative reflexive. they are primarily self-motivated, upwardly mobile, innovative, and not risk averse. they tolerate and flourish in contextual discontinuity and change. • meta reflexives are inward looking, contemplative and ask questions about the questions themselves: why do i think like this, what caused me to be this way? they are idealistic, support worthy causes or the disadvantaged, and can become socially seditious as they observe contradictions and historical conspiracies. • finally, fractured reflexives are passive agents who are unable to reflexively examine life’s alternatives – they become frozen, unable to participate fully in reinforcing, changing or challenging the way of things – they are seen by archer (2003) as societies “victims”. archer suggests a relatively even division between each sector within today’s society. emirbayer and mische (1998) consider a more action-focused perspective and define three distinct capacities through which individuals engage with change: iterative, practical evaluative and projective. the iterative capacity is constrained to the continual reconstruction of the status quo, a highly conservative and static form of dealing with the world directed at reproduction of the past. the agent with practical evaluative capacity, whilst often conversational and therefore conservative, will adopt a technology if it fits existing purposes and structures in the present – a better kind of hammer, but still a hammer, so to speak. the projective capacity, focused on the future, and usually residing in autonomous reflexives, envisions new purposes and new structures for the realisation of aspirations. it is suggested that these three action-focused components of agency also play an important role in understanding the adoption decision by individuals. archer’s communicative reflexive re-enacts external conversation with “similar and familiars” to develop their internal conversation, thus reinforcing conventionalism and “the familiar over the novel”. as detailed in table 1 below, we propose that the brand of non-use particular to the communicative would be disenfranchisement, displacement and perhaps disinterest. for the iterative old ways of doing would do just as well and little motivation is possible unless their similar and familiars adopt. communicatives may tend to depend on other parties to fulfil their needs and may never completely adopt. the communicative reflexive would tend to adopt broadband if it was deemed to support iterative behaviours or if it were mandated in some way (for example for health services). subsequent enactment would only ensue if trusted advisors from their relational group confirmed and supported their initial interest. this reflects katz, matsaganis and ball-rokeach’s (2012) proposal to embed influence within local media and minority anchor groups as a means of helping the usa’s national broadband program gain traction amongst ethnic groups in that country. local media and anchor groups understand the context and life world of those groups and can frame the applications of broadband in a way which makes sense to those participants. the autonomous reflexive, with their dependence on their own reason and future orientation, would be practical or projective; they would be open to novel ideas or practical conceptions, australasian journal of information systems dobson & jackson 2017, vol 21, selected papers from acis 2016 explaining broadband non-adoption in rural australia 6 providing they were in line with their instrumental purposes. in general they would be enthusiastic adopters of broadband and non-adopters would be considered as laggards – the attraction of the internet for personal and social gain would ultimately be expected to lead to adoption. selling broadband to the autonomous reflexive sme owner-managers, society’s entrepreneurs, should not be difficult if they are familiar with internet possibilities, but external factors such as speed, cost, and availability would be important in discouraging laggard behaviour. perhaps in some cases they would need encouragement in self-efficacy and coaching to show how the broadband would directly improve utilitarian outcomes and their own life chances and business prospects. in fact archer (2007) suggests that the autonomous reflexive increasingly dominates today’s globalized society: “today, decreasing numbers of us live in the situation termed ‘contextual continuity’, which seems to be the necessary though not sufficient condition for the development of communicative reflexivity” (p. 320). archer goes on to propose that the proportional reductions in communicative reflexives within today’s society will lead to corresponding increases in autonomous reflexivity. as the situational logic of opportunity engendered by increasing globalization and contextual social discontinuity the autonomous reflexive will have increasing opportunities for personal advancement. such argument is good news for broadband adoption, however as dobson, jackson and gengatharen (2013) suggest, rural communities are perhaps more ensconced in a situational logic of protection, rather than opportunity. for rural communities the drift towards metropolitan cities, lower job opportunities and a desire to maintain the family encourages a focus on protection of opportunities and life chances in order to better fulfil their rural projects. for rural communities the predominant reflexivity mode would be the communicative reflexive who depends heavily on similars and familiars to help guide their life choices. this suggests a different approach is needed in rural communities – one focused on avoiding displacement and emphasizing the benefits for social groups. it can be similarly argued that the meta-reflexive would increasingly be observed in rural regions as people move to the rural regions in order to avoid or address the incongruities of modern society. the meta-reflexive would be most open to the projective ideas and novel choices provided by broadband providing they were in line with their moral aims; their actions would be expected to be targeted at the greater good and could be ambitious and impractical in their social aims, perhaps even to their own detriment. meta-reflexives may actively resist broadband for moral and altruistic reasons and may reflect a disenfranchisement in non-use. the ability of the internet as a change agent would need to be emphasized and the social benefits achievable highlighted. the hardest “nut to crack” would appear to be the fractured – their brand of non-use would largely be dis-interest and would perhaps be the most difficult to address. how can those disengaged from society as a whole be encouraged to adopt the social applications provided by broadband? the external factors suggested by traditional models would have little attraction for the fractured – broadband access would generally be used in public services such as libraries or government offices. adoption of broadband would be a consequence of mandated requirements and would be limited and short term. yet this sector of society is the most vulnerable to experiencing the disadvantages of non-use; the digital divide has serious implications in terms of accessing government services and information. direct focused intervention would need to be introduced to individually introduce broadband and its benefits. 3.2 broadband adoption research models we have argued that the notion of reflexivity provides a powerful lens through which to explain the non-adoption of broadband. in this section we discuss and evaluate other theories and approaches to understanding the uptake and use of technology in order to show where reflexivity offers a richer perspective, in particular for the conundrum of non-adoption. tsai and larose (2015) compare 5 models used by researchers to examine broadband adoption – the technology acceptance model (tam, davis, 1986; davis, bagozzi and warshaw, 1989), the unified theory of the acceptance and use of technology (utaut, venkatesh, morris, australasian journal of information systems dobson & jackson 2017, vol 21, selected papers from acis 2016 explaining broadband non-adoption in rural australia 7 davis, and davis, 2003), diffusion of innovations (doi, rogers, 2003), the model of adoption of technology in households (math, brown and venkatesh, 2005; venkatesh and brown, 2001), and social cognitive theory (sct, larose et al., 2007). they suggest that the first of these (doi, tam, utaut and math) work on similar foundations, each progressively extending previous models by adding particular variables. such development is useful and important, however, as bagozzi (2007) suggests, the parsimony or simplicity of the tam model is its major strength – the powerful core assumption being that intentions to use a technology influence adoption behaviour, and perceived usefulness (pu) and perceived ease of use (peu) determine intentions to use. nevertheless: “parsimony has also been an achilles’ heel for tam. it is unreasonable to expect that one model, and one so simple, would explain decisions and behavior fully across a wide range of technologies, adoption situations, and differences in decision making and decision makers. …in favoring a simple model, researchers have overlooked essential determinants of decisions and action, and turned a blind eye to inherent limitations in tam…almost no research has deepened tam in the sense of explaining pu and peu, reconceptualizing existing variables in the model, or introducing new variables explaining how the existing variables produce the effects they do” (baggozi p. 245). in line with this suggestion we contend that existing models tend to inadequately represent the social reality of the agent and also have an unclear grounding in terms of representing the social and material context within which adoption occurs. we suggest that critical realism can provide a useful grounding for examining broadband adoption and non-adoption. in particular, we suggest that an individual’s reflexivity is important in understanding how the existing identified factors will be less or more important for certain types of people and the personal projects they might be pursuing. the tam model is a deterministic model that assumes causes inevitably lead to stated outcomes (bagozzi 2007). there is no recognition of the mechanisms by which an agent may decide to do otherwise than expected. the tam model tends to implicitly assume that non-adopters are “laggards” and will inevitably adopt if the conditions are right. archer’s reflexivity approach is useful in that it describes the mechanisms by which agents might fallibly respond to structures and actions. individual reflexivity is an important mechanism for explanation of outcomes – in our case for explaining the surprising non-adoption in rural regions. utaut builds on original doi and tam models to propose four key constructs: “performance expectancy, effort expectancy, social influence, and facilitating conditions that influence behavioural intention to use a technology and/or technology” (venkatesh, thong and xu (2012, p. 159). the math model perhaps represents the most advanced of the models in that it builds on previous representations by suggesting that intention to adopt can be predicted by considering normative, attitudinal, and control beliefs: “normative beliefs refer to other people’s influence (including influence from friends and family, secondary (media) sources, and workplace referents) on an individual’s behavior (venkatesh & brown, 2001). attitudinal beliefs include applications for personal use, children, work, fun, and status gains. control beliefs entail fear of technological advances, fear of declining cost, cost of the product, perceived ease of use, and possession of the requisite knowledge to use the innovation”. yet we suggest that such additional constructs seem arbitrary in their inclusion and are not reflected in a clear theoretical grounding. as bagozzi (2007) suggests none of the models adequately reflect the group, cultural or social aspects of technology acceptance. our paper argues that a useful model for representing the social role of people is critical realism – an approach fundamentally focused on the actions of agents within a pre-existing social environment. such an approach seems more suitable in considering the adoption of a technology like broadband that has such wide social and personal implications. the tam is largely conceived as a model for the adoption of technology by individuals; where social influences are acknowledged they are represented as external constraints or enablers to the individual adoption decisions, as bagozzi (2007 p. 247) suggests: “when so-called “social influence processes” have been introduced into tam, the practice has been to treat social australasian journal of information systems dobson & jackson 2017, vol 21, selected papers from acis 2016 explaining broadband non-adoption in rural australia 8 influence in the limited senses of either a constraint or force on the decision maker and perceived as originating from “other people whose opinions are important to me” (e.g., venkatesh and davis, 2000) or as an attempt to “enhance one’s … status in one’s social system,” such as a reference group (e.g., moore and benbasat, 1991, p. 195)”. a model based around critical realism fundamentally reshapes the adoption argument in its acknowledgement that in such matters the individual cannot be separated from their social role, as archer (1995, p. 1-2) suggests “…for what we are and what we do as social beings are also affected by the society in which we live and by our very efforts to transform it… we are simultaneously free and constrained and we also have some awareness of it. the former derives from the nature of social reality; the latter from human nature's reflexivity”. depending on the reflexivity of the individual concerned, reactions to impacting factors will be different. for example we would expect that for communicatives the identified variable “social influence” from the utaut model and observational learning from the sct will have more relevance than for the autonomous or fractured. similarly the autonomous may be more responsive to perceptions as to perceived usefulness in the original tam model – their focus being selfdirected towards a focus on personal benefit. larose et al (2007) propose the use of social cognitive theory (sct) to examine adoption behaviours seeking to understand the inner reasoning leading one to accept or reject an innovation. the sct approach proposes that perceived outcomes are formed through direct experience with one’s own behaviour or through observation of the behaviour of others. the sct grounding is used to propose additional causal factors termed enactive learning and observational learning. this approach is used to address the criticism that the diffusion paradigm often neglects the individual’s capabilities and psychological factors and their role in technology acceptance or rejection. tam and earlier models are said to instead focus more on the characteristics and qualities of the innovation rather than the important user perceptions: “that distinction is a crucial one in the present context, since the attributes of broadband internet are more or less fixed while the perceptions of those attributes by consumers may still be malleable through promotional efforts” (p. 362). sct shares some elements with a critical realist approach in that both recognize the important role that cognition play in causal analysis. for both “reasons are causes”, but critical realism provides a more developed foundation in that it also suggests an emphasis on pre-existing, perhaps non-recognized, non-ideational structures and mechanisms as well. as johnson and duberley (2000, p. 165) suggest: while our knowledge of these structures is always interpretative, human agency draws upon extant structures as a condition of action. moreover it is through human agency that social structures come about, are reproduced and transformed — regardless of our intentions or awareness that this is so. so while human behaviour in, for instance, organizations may often lie in and be caused by the inner interpretative reasoning of actors: for the critical realist there may be causes that are not recognized by, nor accessible to, those actors. in particular, the work of archer (2000) highlights the role of emotions and embodied “active practice” in guiding “inner conversations”, reasoning and decisions. the richer model provided by critical realism can provide a better understanding of the non-adoption decision. this distinction between adoption and non-adoption is important. as described above, adoption is a positive action whereas non-adoption may be a consequence of deliberate action or even disinterest. forms of “non-use” will follow different causal chains and we argue will depend very much on the mode of reflexivity engaged by a person at the time. 3.3 modes of reflexivity and adoption programs if avoidance of the adoption plateau is our goal, we propose that understanding the reflexive modes of various agents’ and groups of agents has practical implications. our characterisation of individual agency for example has consequences for the provision of information upon which to base decisions to adopt broadband. although only a starting point, table 1 presents australasian journal of information systems dobson & jackson 2017, vol 21, selected papers from acis 2016 explaining broadband non-adoption in rural australia 9 the differentiated approaches that might be taken to encourage broadband adoption. it loosely proposes the application of aspects of the various tam based and sct models detailed above. the justification for so doing is that, whilst critical realism has fundamental issues with the underlying arguments of such models, they can provide suggestions as to likely tendencies of concerned agents. training programs to the general public, often communicative reflexives, must allow for patient and inclusive conversation which encourages re-evaluation of norms and collective assumptions to overcome their disinterest. social norms influence attitudes to computermediated interaction but informed conversation can change them. furthermore, information might be better delivered to cohesive social groups and in social environments to encourage ongoing conversation about broadband to take a positive and practical direction that gains momentum within a social group, rather than allowing the reinforcement of the status quo (which might be based upon a general ignorance of both their own life worlds and technology). another strategy would be to embed technologically savvy agents of changes into community groups to conduct conversations about broadband in the language and context of the group. whilst there is a palpable rural ideology, it is not clear how universal it is or indeed that everyone has bought into it. and yet as an “undiscussable” component of the rural social world, its presence can be felt in group defensiveness or rejection vis à vis broadband and the internet in general. the role of computer-mediated communication in enhancing existing practices and its possible practical applications may encourage adoption. education or advertising about the benefits and scope of the internet and broadband should be integrated into this fabric. it can be both discussed and demonstrated to strengthen and not diminish the country way of life. similarly, communicative reflexives are also responsible for and participate in small manufacturing, farming, medical services, country town historical societies or sporting clubs. the practical evaluative capacity of these agents must be engaged through understanding the social and business ontologies of their groups to identify and hook into the most salient aspects of those business and life worlds. these would lead to a picture of what is important in those groups and points of leverage for the nbn could be highlighted in education. information provision, whether in advertising, training or brochures, should be as specific as possible to life worlds, providing specific and common scenarios and use-cases. whilst it may be important for other reasons, non-specific advertising will probably not influence people in the decision to adopt. again, embedding of domain experts or change agents who also understand the opportunities of broadband within such groups could help adjust attitudes to broadband to become more positive. the autonomous reflexive seeks personal advancement and, if a non-adopter, will typically be a laggard, as this reflexive category otherwise tends to recognise and act upon the material benefits of new technologies. the cost-benefit equation is important here, and information provision should emphasise savings, efficiencies and functionality. the meta reflexives on the other hand tend to be deliberate non-adopters, rejecting the mantra of function and transactional efficiency in favour of humanistic values. the social potential of the internet to help others or enhance the lives of the excluded should thus be an area of focus. this leaves us with the fractured, where the reflexive and material barriers and marginalisation will tend to be the greatest, where getting through the day might be an achievement. reducing material obstacles by providing free access in public locations (such as libraries or social welfare agency offices) is a first strategy. making the technology directly useful, easy to navigate and robust are very important, with co-location of empathetic, helpful staff to prevent easy discouragement. table 1 summarises the differentiated approaches that might be taken to encourage broadband adoption. australasian journal of information systems dobson & jackson 2017, vol 21, selected papers from acis 2016 explaining broadband non-adoption in rural australia 10 agents’ reflexive mode type of nonuse agents’ reflexive capacity possible target groups promotion strategies communicative disenfranchisement, disinterest iterative older rural people, isolated communities, some community groups (historical society, museum, sport) embedding empathetic specialists within social groups, conducting dialogue and sensitive conversation in group contexts focus on normative beliefs (math model) appreciating personal capacities and self-determination. focus on group ontology (community activity), specific use cases and applications communicative disenfranchisement displacement practical evaluative farmers, small businesses, artisans focus on group ontology (business activity), specific use cases and applications focus on normative and attitudinal beliefs (math) encourage enactive and observational learning (sct) autonomous laggard projective and practical evaluative managers, community leaders, innovators enactive and observational learning (sct) improve external factors like pricing, cost, availability etc, focus on enactive learning address attitudinal beliefs meta active resistance disenchantment projective ngos, consultants, volunteers focus on control beliefs (math) emphasise social benefits and opportunities fractured disinterest not consistent marginalised groups training in reflexivity to understand personal capacities and self-determination. emphasise non-exclusional aspects of broadband (e.g. voicevoice, applications requiring low literacy) embedding empathetic specialists within social groups, conducting dialogue and sensitive conversation in group contexts table 1: summary of adoption strategies for modal groups 4 conclusion activations to broadband are exceeding expectations for the nbn rollout, however, as discussed international experience suggests that adoption of the nbn will plateau as the rollout completes. as the rollout moves towards full coverage a focus must be made on nonadopters in order to achieve the social benefits promised by universal internet adoption. we suggest that this universal adoption will be threatened as there will always be an element that will never adopt fixed or mobile broadband unless specifically targeted. understanding the physical, practical, social and cultural characteristics of this group is an essential prerequisite to addressing inequities. we believe using archer’s model of self and reflexivity allows a rich analysis of adoption trajectories and has profound implications for future research. in rural regions in particular, we suggest that promotion of broadband needs to be cognisant of the dominant communicative reflexive element evident in the region – the dependence on “similars and familiars” to confirm or deny decisions, the family focus and lifestyle orientation of many inhabitants. the difficulties of enrolling the fractured are particularly challenging and will require significant investment in time and resources. generally, our analysis suggests that there is the opportunity for a more nuanced and sophisticated provision of knowledge and information to businesses and communities in rural australasian journal of information systems dobson & jackson 2017, vol 21, selected papers from acis 2016 explaining broadband non-adoption in rural australia 11 and remote areas that will accelerate the uptake, application and innovative redesign of business and life worlds to exploit this landmark communications infrastructure. accepting the more dominant role for the autonomous reflexive within metropolitan regions would allow a similar conclusion but the dominant message from our study is that no mode can be ignored – each mode needs to be recognized in advertising and promotions since all targets have their own particular issues and contextual concerns. in particular strategies need to be developed to target the fractured reflexive and to break through the evident disinterest and lack of connection. traditional internet adoption models developed around the diffusion paradigm will be useful to develop strategies to convert non-adopters to adopters but the strategies need to focus on particular groups to be most effective. this paper suggests a social realist approach such as critical realism can provide useful grounding for examining the social, cultural and personal issues underlying non-adoption. references archer, m. s. (1995). realist social theory: the morphogenetic approach. cambridge: cambridge university press. archer, m. s. (2000). being human: the problem of agency. cambridge university press. archer, m. s. (2003). structure, agency and the internal conversation. cambridge university press. archer, m. s. (2007). making our way through the world: human reflexivity and social mobility. cambridge university press. archer, m. s. (2010). routine, reflexivity, and realism. sociological theory, 28(3), 272-303. archer, m. s. (2012). the reflexive imperative in late modernity. cambridge university press. bagozzi, r. p. (2007). the legacy of the technology acceptance model and a proposal for a paradigm shift. journal of the association for information systems, 8(4). broadband commission. (2015). the state of broadband 2015. achieving digital inclusion for all. brown, s. a., and venkatesh, v. (2005). model of adoption of technology in households: a baseline model test and extension incorporating household life cycle. mis quarterly, 29, 399–426. cave, m., and martin, i. (2010). motives and means for public investment in nationwide next gneration networks. telecommunications policy, 34, 505-512. cradduck, l. (2016). submission to the productivity commission issues paper june 2016telecommunications universal service obligation. davis, f. d. (1986). a technology acceptance model for empirically testing new end-user information systems: theory and results. doctoral dissertation. sloan school of management, massachusetts institute of technology. davis, f. d., bagozzi, r. p., and warshaw, p. r. (1989). user acceptance of computer technology: a comparison of two theoretical models. management science, 35, 982– 1003. dobson, p., jackson, p., and gengatharen, d. e. (2013). explaining broadband adoption in rural australia: modes of reflexivity and the morphogenetic approach, mis quarterly, 37(3), 965-992. emirbayer, m., and mische, a. 1998, what is agency?, american journal of sociology (103:4), pp 962–1023. australasian journal of information systems dobson & jackson 2017, vol 21, selected papers from acis 2016 explaining broadband non-adoption in rural australia 12 garcia-ruiz, p. and rodriguez-lluesma, c. (2009), reflexive consumers': a relational approach to consumption as a social practice , in archer, m. s. (ed.), conversations about reflexivity. routledge horrigan, j. b., and duggan, m. (2015). home broadband 2015. pew research center, december 21, 2015(4.) johnson, p., and duberley, j. (2000). understanding management research: an introduction to epistemology. sage. katz, v. s., matsaganis, m. d., and ball-rokeach, s. j. (2012). ethnic media as partners for increasing broadband adoption and social inclusion. journal of information policy, 2, 79-102. kontos, p. c., and poland, b. d. (2009). mapping new theoretical and methodological terrain for knowledge translation: contributions from critical realism and the arts. implementation science, 4(1), 1. larose, r., gregg, j. l., strover, s., straubhaar, j., and carpenter, s. (2007). closing the rural broadband gap: promoting adoption of the internet in rural america. telecommunications policy, 31(6), 359-373. moore, g.c. and i. benbasat (1991), development of an instrument to measure the perceptions of adopting an information technology innovation, information systems research, 2, 3 (september 1991), 192-222. mutch, a. (2007), reflexivity and the institutional entrepreneur: a historical exploration, organization studies (28:7), pp 1123–1140. nbn co. (2016). nbn weekly progress report as of 15 september 2016. from http://www.nbnco.com.au/corporate-information/about-nbn-co/corporateplan/weekly-progress-report.html perrin, a., and duggan, m. (2015). americans’ internet access: 2000-2015. pew research center, 26(6). rogers, e. m. (2003). diffusion of innovations (5th ed.). new york: free press. satchell, c., and dourish, p. (2009, november). beyond the user: use and non-use in hci. in proceedings of the 21st annual conference of the australian computer-human interaction special interest group: design: open 24/7 (pp. 9-16). acm. stewart, j. (2007) local experts in the domestication of information and communication technologies, information, communication & society, 10:4, 547-569. tsai, h. y. s., and larose, r. (2015). broadband internet adoption and utilization in the inner city: a comparison of competing theories. computers in human behavior, 51, 344-355. venkatesh, v., and davis, f. d. (2000). a theoretical extension of the technology acceptance model: four longitudinal field studies. management science, 46(2), 186-204. venkatesh, v., and brown, s. a. (2001). a longitudinal investigation of personal computers in homes: adoption determinants and emerging challenges. mis quarterly, 71–102. venkatesh, v., morris, m. g., davis, g. b., and davis, f. d. (2003). user acceptance of information technology: toward a unified view. mis quarterly, 27, 425–478. venkatesh, v., thong, j. y., & xu, x. (2012). consumer acceptance and use of information technology: extending the unified theory of acceptance and use of technology. copyright: © 2017 dobson & jackson. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems dobson & jackson 2017, vol 21, selected papers from acis 2016 explaining broadband non-adoption in rural australia 13 permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. paper title (paper title style) australasian journal of information systems de silva, burstein, jelinek & stranieri 2015, vol. 19, pp. s99-s115 complexities of big data addressing the complexities of big data analytics in healthcare: the diabetes screening case daswin de silva la trobe business school la trobe university d.desilva@latrobe.edu.au frada burstein centre for organisational and social informatics monash university frada.burstein@monash.edu herbet jelinek school of community health & centre for research in complex systems charles sturt university hjelinek@csu.edu.au andrew stranieri centre for informatics and applied optimization federation university a.stranieri@federation.edu.au abstract the healthcare industry generates a high throughput of medical, clinical and omics data of varying complexity and features. clinical decision-support is gaining widespread attention as medical institutions and governing bodies turn towards better management of this data for effective and efficient healthcare delivery and quality assured outcomes. amass of data across all stages, from disease diagnosis to palliative care, is further indication of the opportunities and challenges to effective data management, analysis, prediction and optimization techniques as parts of knowledge management in clinical environments. big data analytics (bda) presents the potential to advance this industry with reforms in clinical decision-support and translational research. however, adoption of big data analytics has been slow due to complexities posed by the nature of healthcare data. the success of these systems is hard to predict, so further research is needed to provide a robust framework to ensure investment in bda is justified. in this paper we investigate these complexities from the perspective of updated information systems (is) participation theory. we present a case study on a large diabetes screening project to integrate, converge and derive expedient insights from such an accumulation of data and make recommendations for a successful bda implementation grounded in a participatory framework and the specificities of big data in healthcare context. keywords: big data analytics; health informatics; clinical decision support; translational research; business analytics; information fusion 1 introduction a new demand, increasingly driven by cost pressures and evidence-based medicine, is pushing the healthcare sector to acquire, manage and disseminate relevant data to every stakeholder, from medical practitioners to patients and carers (groves et al. 2013). the type and complexity of medical condition(s) has a direct impact on the volume of data accumulated. groves et al. (2013) clearly identify this as a big data scenario. reduced expenditure and improved patient outcomes are means of value generation in healthcare. the five pathways to value generation; right living, right care, right provider, right value and right innovation are empowered by the use of big data analytics (groves et al. 2013). these value pathways are well-positioned with recent advances in clinical decision-support (cds). cds has evolved from medical data processing tools to complex decision support frameworks and infrastructure for clinical s99 australasian journal of information systems de silva, burstein, jelinek & stranieri 2015, vol. 19, pp. s99-s115 complexities of big data knowledge management. two prominent taxonomies for cds architectures are presented by (sen et al. 2012) and (wright and sittig 2008). in the former, the authors explore cds architectures within the context of underlying technologies of information management, data analytics and knowledge management. in the latter, the authors define architecture as the form of interaction between related systems, with four distinct cds phases; standalone, integrated, standards-based and the service model. further impetus for big data analytics arises from the role of clinical scientists in research into disease management. the declining role of clinical scientists in medical research has been identified as a potential reason for the critical gap (termed the ‘valley of death’ crisis) that lies between bench research and bedside treatment (butler 2008). (roberts et al. 2012) emphasise the increasingly important role of data and clinical scientists in translational research; research that converts laboratory discoveries into clinical interventions. ambitious projects such as (burton et al. 2007) aim to predict the likelihood of medical conditions based on molecular biomarkers derived from datasets of disease-related mutations as well as the simpler historybased approach proposed by (davis et al. 2008) are further indications of the potential of big data. we reflect on cds architectures from the perspective of the updated is participation theory (markus and mao, 2004), which allows to not just look at the necessary components of the bda system, but suggest the roles and the level of participation expected to lead to successful system implementation and use. the paper is organised as follows. the following section presents a theoretical underpinning for articulating the role of bda in advancing user participation in development and use of such systems from the updated is participation theory perspective. an account of big data analytics (bda) is reported next with examples from healthcare, relevant technology components, use and integration. we then review the context of analytics in healthcare, scrutinize the complexities of applying big data analytics to healthcare data and propose a solution to address these complexities. 2 value of bda platform from the is participation theory perspective (markus and mao 2004) revisited traditional is participation theory and proposed the key elements of a new theoretical framework that clearly articulates the key parameters for designing the systems “for users with users”. in their view, the updated theory of is participation, requires specific re-definition of system success, differentiation among actors’ roles in systems development and implementation, and refinement of the concept of participation between each actor accordingly (figure 1). figure 1: updated participation theory (adapted from markus and mao, 2004) s100 australasian journal of information systems de silva, burstein, jelinek & stranieri 2015, vol. 19, pp. s99-s115 complexities of big data the motivation of their work was to specify the effects of participation on various types of outcomes that are clustered together as system success. they identify three theories linking participation to system success (buy-in, system quality and emergent interactions) and determine conceptual gaps in these links that lead to the updated theory. foundations of the updated theory lie in: 1) the distinction of system success into two concepts: system development success and system implementation success, with emergent reciprocal relations between them; 2) the description of groups of actors including stakeholders where participants are a subgroup, and change agents where is specialists are a subgroup; 3) a reformulated behavioural concept of participation activities, characterized in terms of type and richness, methods and conditions; and 4) the hypothesis of emergent causal processes (markus and mao, 2004). following on from these foundations, (de silva et al. 2013) adopted other dimensions of participatory design processes as suggested by (bergvall-kåreborn et al. 2010), e.g. “designing for users” and “designing with users”. the latter provides an opportunity for closer engagement between the is developers and users. such participatory design process assumes the users’ “voice” to be fully appreciated and better understood, together with the new opportunities that flow from full articulation of their needs in an act of active engagement from the planning phase to implementation, and commercialisation of the final product. in this sense new tools and techniques can be employed for “tracing user needs” by continuously monitoring their behaviour both implicitly and through shared data management, as well as by making users express their feedback as part of the systems deployment process. having a bda platform increases stakeholder participation in healthcare. it is not only the domain experts and technology experts but also other stakeholders, the patients, carers, advocacy groups and regulatory bodies that can partake in healthcare delivery and management. to this end, the updated is participation theory proposed by de silva et al, (2013) and contextualised in a participatory information management framework to facilitate patient-centred care is even more relevant to the case of bda as it deals with even greater complexity of data and has to address the dynamic and diverse needs of multiple stakeholders. foundations of the updated participation theory lie in: 1) the distinction of system success into two concepts: system development success and system implementation success, with emergent reciprocal relations between them; 2) a broad engagement of the stakeholder groups where participants are a subgroup, and change agents where is specialists are a subgroup; 3) a reformulated behavioural concept of participation activities, characterized in terms of type and richness, methods and conditions; and 4) the hypothesis of emergent causal processes (markus and mao, 2004). the key participation activities, type, richness, methods and conditions are enriched by the bda platform, which leads overall to improved healthcare outcomes. 3 big data analytics this discipline originated as business intelligence (bi) in the 1990s but was later renamed to business analytics at the turn of the century to reflect the major contribution of analytics and a separation from information management. more recently, big data and bda have been introduced to comprehend the changing nature of data (davenport 2006). the formal definition by (gartner) clearly notes these changes, big data is high-volume, -velocity and variety information assets that demand cost-effective, innovative forms of information processing for enhanced insight and decision making (beyer and laney 2012). frequently known as the 3vs, volume (the large quantities of data), velocity (speed of input/output, rate of change) and variety (different types of data from varied sources) are also used as quantifiers for applying analytics to big data. bda aims to derive value from datasets quantified by these parameters. several other parameters have been introduced to account for further intricacies of big data. table 1 presents definitions of these concepts along with examples from healthcare. s101 australasian journal of information systems de silva, burstein, jelinek & stranieri 2015, vol. 19, pp. s99-s115 complexities of big data parameter description examples from healthcare volume size of data cohorts of patients, multiple conditions and treatment plans variety different formats and types (numbers, images, text) medical, clinical and omics data and images from patients with diverse conditions velocity the rate at which data arrives and changes (streams, batches, infrequent intervals) wearable sensors and diagnostics transmitting patient behaviour veracity unpredictability of innately imprecise data types patient feedback and clinician notes on patient’s state variability different interpretations of the same data clinical data on the same condition affecting a diverse group of patients value inherent value addition to the organisation against the costs to acquire/accumulate. extent of value addition to clinical decision-support and translational research sparseness low density of useful content (missing or null values) variability of patient feedback on symptoms and progress complexity hierarchies, linkages between entities and recurrent data structures multi-pharmacy and multi-morbidity table 1. characteristics of big data (chen et al. 2012) conducted a comprehensive review of business intelligence and analytics (bia) evolution from structured content (bia 1.0), unstructured content (bia 2.0) to mobile and sensor-based content (bia 3.0). they signify smart health and wellbeing as a promising and high-impact bia application, alongside ecommerce, e-government, science and technology and security and public safety. in healthcare, they identify patient data and genomics as the two main sources of big data. these two sources pair very well with the expectations of cds and translational research. privacy preservation and ethical research are highlighted as challenges to knowledge discovery from healthcare big data. interestingly, the nature of data largely determines phases of evolution while the analytics techniques remain fairly consistent. it is this nature of data that gives rise to complexities in healthcare bda. bda also presents a paradigm shift in computing with the innovation of independent and selfmanaged components that can be infinitely scaled to suit large volume computations, without the complication of shared resources. the computing platform most often used for bda is apache hadoop (https://hadoop.apache.org/); it consists of a distributed file-system (hadoop distributed file system; hdfs) and a robust programming model (mapreduce) that work together to distribute the algorithm to the data through task assignment (instead of the traditional approach of data into the algorithm) and to create schemas/ data models at run-time (instead of a static schema). the general-purpose hadoop stack is illustrated in figure 2. the initial layer is for data import, with two packages, flume designed to extract data from file-based systems (free-form text, log files, etc) and sqoop, designed to extract structured content from enterprise systems and relational databases. next, data moves to the storage layer, hdfs, which introduces redundancy and fault-tolerance in preparation for scalable computations. mapreduce introduces the computation in the form of map and reduce functions to distribute the computation across many nodes and then merge the results into a single set of outputs. on the same layer, hbase provides a database-style interface to hdfs to deploy programs that can read or write to specific subsets of data. s102 australasian journal of information systems de silva, burstein, jelinek & stranieri 2015, vol. 19, pp. s99-s115 complexities of big data figure 2. apache hadoop ecosystem the application layer presents end-user abstraction. converting any computation to mapreduce format is complicated, the application layer abstracts this with a set of purposebuilt packages; hive supports declarative queries for warehouse-type batch operations, pig provides a compiler for sequences of mapreduce sub-modules and mahout a package of machine learning techniques designed to use hadoop for scalable analysis of big data. oozie is a workflow/coordination scheduler system, which can be programmed to execute several different jobs (hive, pig, mahout) in sequence. other extensions such as r connectors to hadoop are available as early trials at the time of writing. most major software vendors provide extended implementations of hadoop with improvements for faster deployment and/or simplified management. as a discipline related to healthcare, bio-informatics has been quick to utilise the strength of hadoop with extended frameworks such as cloudburst and crossbow. these have been able to achieve record speeds for next generation sequencing via read mapping (schatz 2009). however in contrast, big data management and analytics has had limited impact on healthcare. adoption of electronic health records (ehr) in clinics and hospitals can be the much needed platform for bda projects. (jensen et al. 2012) present a complete review of the key issues in accumulating ehr data and integrating these with genetic data. (hanauer et al. 2011) performed a simple symptom–disease–treatment association rule mining on a large collection of ehr through which they were able to identify clinically relevant and accurate associations for seven distinct diseases. further impetus towards bda is through large-scale online data collection projects driven by crowd intelligence, such as daily strength (http://www.dailystrength.org/) and patientslikeme (http://www.patientslikeme.com/). bda outcomes from the accumulated data can be a useful resource for patient empowerment s103 australasian journal of information systems de silva, burstein, jelinek & stranieri 2015, vol. 19, pp. s99-s115 complexities of big data (miller 2012). other prominent examples do not link directly into bda but through a support technology such as mobile phones or wearable sensors. ginger.io is a bda application (ginger.io 2015) that utilises mobile sensors present in smartphones in order track, record and analyse physical movements, phone activity (such as calls, texts) in order to monitor behavioural health therapies for chronic illnesses. another bda application in healthcare, (propellerhealth 2015) utilises a gps-enabled tracker to monitor inhaler usage by asthmatics. the information collected is used to determine behaviours across populations and also integrates this information with known asthma catalysts to improve treatment and prevention of asthma. the lack of bda having a direct impact on healthcare can be attributed to the complexities delineated in the following section. 4 analytics in healthcare analytics in healthcare is driven by the gradual shift from disease-centred to patient-centred care (pcc). pcc models transcend traditional boundaries that isolate patients from their clinical context. patient-centred care was first featured in healthcare as one of the six aims for high-quality healthcare in a report ‘crossing the quality chasm’ published by the usa institute of medicine (bloom 2002). this report defines pcc as care that is “respectful of and responsive to individual patient preferences, needs, and values, and ensuring that patient values guide all clinical decisions”. in the traditional approach, the clinician addresses the medical condition and thereby cures/improves the health of the patient whereas with pcc this becomes a shared responsibility between healthcare professionals, the patient and family members. frequent communication and information sharing leads to a further accumulation of data besides the actual clinical data and can be used effectively to understand patients with similar circumstances. gartner’s business analytics framework (chandler et al. 2011) is a useful tool to identify the role of analytics in healthcare. the framework can be presented as a matrix to highlight its key elements (table 2). the primary activities ‘enable’, ‘produce’ and ‘consume’ run across the main entities of ‘people’, ‘processes’ and ‘platform’. people processes platform consume decision makers decision processes decision capabilities produce analysts/data scientists analytic processes analytics capabilities enable ict administrators information governance information capabilities table 2. business analytics framework (chandler et al. 2011) bda maintains its potential to fulfil the expectations of the ‘platform’ element. it can become the patient-centred healthcare platform for information management, analytics and decisionmaking capabilities. the following complexities need to be addressed for the eventuation of this platform. based on our investigations, the key complexities impacting bda in healthcare are granular data accumulation, temporal abstraction, multimodality, unstructured data, and integration of multi-source data. they are distributed across the ‘enable, produce, consume’ activities noted above. these will be explored bottom-up so that low-level explanations contribute to understanding at the high-level. 5 granular data accumulation from a general practitioner’s desktop computer to cardiac monitors in an emergency room, a multitude of clinical information systems capture patient information. this information exists at different levels of granularity, in diverse formats and recorded at varying frequency. for instance, sensor readings from a cardiac monitor are well-defined in terms of grain, format s104 australasian journal of information systems de silva, burstein, jelinek & stranieri 2015, vol. 19, pp. s99-s115 complexities of big data and frequency, however blood glucose measurements, although in the same format, can vary in terms of grain and frequency. a patient can record blood glucose levels at different times during the day when at home whereas a clinic may capture a single measurement but derive a different measure (glycated haemoglobin) to determine the three month average. this difference in granularity can be an extra dimension of information for bda when paired with medication, demographic or behavioural information. another example is recording medication information along with medical imaging. bda can be used to identify changes in medical images and relate these to changes in medications or dosage. bda is not limited by volume, velocity or variety so it is pertinent to capture and accumulate information at all levels of granularity. granular data accumulation also extends to capturing outcomes and feedback. outcomes from a specific exercise routine, dietary modifications or change of medication need to be captured and recorded. completeness of medical data from start to end of the patient lifecycle is crucial for successful bda in translational research. it is equally important to capture patient feedback as this reflects their experience of the medical condition. bda can be used to identify associations between outcomes, feedback, symptoms, medication and behavioural changes but the quality of the findings is heavily dependent on the completeness of the data accumulated. 6 temporal abstraction time is the supporting dimension for granularity as patient information is collected over time. temporal abstraction (ta) is defined as a process which takes in a set of time-stamped parameters, external events and abstraction goals to generate abstractions of the data to be used for interpretation of past/present data (shahar 1994). the intention is to transform temporal data from a simple, numerical form to informative and qualitative descriptions, which can be understood by clinicians. (stacey and mcgregor 2007) present a comprehensive review of techniques used in clinical data analysis systems. they highlight several inadequacies, such as confinement to temporal trends and level shifts, limited dimensionality of abstraction output and lack of integration with other analysis outcomes. complex ta is a further development to represent higher-level abstractions not directly from the data but from intermediate ta outcomes. (keravnou 1997) describes complex tas as compound time objects with a repeating element, a repeating pattern and progression pattern where the repeating element itself could be periodic. (bellazzi et al. 2000) developed a complex ta in the domain of diabetes monitoring to detect two overlapping interval abstractions and also the successive increase and decrease of blood glucose levels. another well-researched approach to complex tas is to represent the changing nature of a timer series using characters and words as symbols. (sharshar et al. 2005) separated a data stream into trend and stability for each data point and applied rules to convert the signal into transient, decrease, increase or constant categories. a further abstraction was applied when individual characters, defining the state of the signal, are merged to form words. these ‘words’ can be mined for clinically relevant associations and projections. ta and complex ta effectuate summarisation of highly granular data to abstract clinically relevant representations recorded over time for conventional data mining techniques. however with the advent of bda the clinical knowledge embedded into ta of the same becomes equally useful to navigate the large space of granular data accumulation. 7 multimodality mode is a resource for sense-making, it introduces context to an entity. many contexts/modes can be found in a clinical environment to represent and also identify a patient. modes such as demography, behaviour, symptoms, diagnosis, blood-gas measurements and medication are indicative of the patient’s condition, disease trajectory and future well-being. clinical decisionsupport greatly benefits from this multimodal representation of the patient’s state. capturing the multiple modes of an entire sample of patients is an equally rich resource for inference and prediction in translational research. multimodality in business analytics is frequently addressed by data warehousing technologies. despite its prevalence in many industries, its adoption by medical organizations has been limited. early implementations of clinical data s105 australasian journal of information systems de silva, burstein, jelinek & stranieri 2015, vol. 19, pp. s99-s115 complexities of big data warehousing (cdw) were aimed at solving specific clinical problems. there are still a few recent case studies that demonstrate the applicability of data warehouse concept in medical domain. for example, (wisniewski et al. 2003) describe the use of a data warehouse for hospital infection control. it was populated with data from three hospitals and demonstrated to be useful for measurement of antimicrobial resistance, antimicrobial use, the cost of infections, and detection of antimicrobial prescribing errors. (chute et al. 2010) present a review of the enterprise data trust at the mayo clinic, which is a collection of all electronic data organized to support information management, analytics and high-level decision-making. in recent research endeavours (hu et al. 2011; lowe et al. 2009) have proposed and implemented data warehousing solutions to address the information needs of translational research, which can among other functions integrate pathology and molecular data with a clinical data model to support a breast cancer translational research program (hu et al. 2011). stride (stanford translational research integrated database environment) is an informatics platform for clinical and translational research. it consists of a data management system, a cdw and a development framework for new applications. the complexity with healthcare data arises when designing a suitable dimensional model to encompass the variety of information (demographic to clinical) and type of information (structured and unstructured) accumulated. 8 unstructured data healthcare data is inundated with unstructured content consisting mainly of textual records ranging from clinician comments to patient feedback. textual records of this nature can exist at all levels of granularity noted earlier. patient feedback can be collected before and after a surgery, during a period of new medication or a behavioural change. in addition to text, other unstructured formats include images, audio/video recordings and associative datasets. given the inherent structure of the discipline, a majority of clinical text can be associated with wellformed ontologies. however, textual records received from patients need to be interpreted using a trusted knowledge base. the design and development of such user-warrant ontology becomes a complex task given the variety of terms that can be used to refer to medical conditions and symptoms. (nguyen et al. 2014; prakasa and de silva 2013) have studied the extraction of an ontology from end-user feedback in healthcare. many of the big data technologies have been developed to address the complexities of unstructured data. the central data models are key-value pair (kvp), column family stores, document databases and graph databases (chen et al. 2012). the lack of a fixed schema makes these data models flexible and scalable, however they do not follow standard properties found in relational databases. each type of unstructured data can be analysed independently for patterns of interest, however complexities arise when outcomes from such disparate sources needs to be integrated, in part or whole. 9 information fusion information fusion is a widely researched field, which is an efficient method for automatically or semi-automatically transforming information from different sources and different points in time into a representation that provides effective support for human or automated decision making (boström et al. 2007). focus is largely on the transformation of information, which includes means for combining and aggregating to infer as well as reduce information. much research has been conducted on multi-sensor fusion (khaleghi et al. 2013), image fusion (goshtasby and nikolov 2007) and web information fusion (yao et al. 2008). independently, intelligent fusion techniques have been developed based on post-perceptual integration phenomena (torra 2003) and cross-modal influence strategy (coen 2005). in healthcare, levels of granularity and the temporal factor need to be well aligned with the purpose of information fusion. structured and unstructured healthcare data accumulated at different levels of granularity will be processed by bda within the mutually exclusive modals to generate analytics outcomes. although these outcomes are beneficial on their own, they can be further fused to create a comprehensive account of a patient (for clinical decision support) or a medical s106 australasian journal of information systems de silva, burstein, jelinek & stranieri 2015, vol. 19, pp. s99-s115 complexities of big data condition (for translational research). the computational requirements of such large-scale information fusion can be efficiently handled by big data technologies, however the complexities of the data model for effective information fusion is largely unaddressed. 10 the analytics team it is pertinent to briefly discuss the analytics team most suited for bda applications in healthcare. the team broadly consists of domain experts and analytics experts. a clinical scientist will be the primary contributor of domain expertise. having a thorough knowledge of both the nature of medical data and expected outcomes, a clinical scientist can guide the knowledge discovery process. a physician is a preferred secondary role to further improve the domain expertise. the analytics experts will comprise a data scientist, a data analyst, a software developer and a project manager. depending on the size of the organisation and the analytics effort, the team can expand or take multiple roles. the data analyst is generally in charge of data extraction and processing following which the data scientist and analyst will team together for the analytics phase. a software developer is required for new interface development or integration with existing systems. another useful role for the analytics team is that fulfilled by a simplified version of the business intelligence competency centre (bicc) commonly found in large-scale analytics projects (hostmann et al. 2006). the bicc is responsible for the strategic plan, prioritisation, data quality, governance and uptake of analytics outcomes in business activities. these responsibilities can be also assigned to the roles identified earlier. the next section presents a case study of bda platform design and implementation as part of the diabetes screening research initiative (discri) project with a review of outcomes and their benefits (burstein et al. 2013). 11 addressing the complexities: the case study a prototypical trial was conducted on data accumulated by the diabetes screening complications research initiative (discri) run at a regional australian university (jelinek et al. 2006). it is a diabetes complications screening program in australia where members of the general public participate in a comprehensive one-stop health review. the screening clinic has been collecting data for over ten years and includes close to a hundred features including demographics, socio-economic variables, education background, clinical variables such as blood pressure, body-mass-index (bmi), kidney function, sensori-motor function as well as blood glucose levels, cholesterol profile, inflammatory markers, oxidative stress markers and use of medication. the dataset is reflective of typical big data accumulation in a clinical environment to be used for both clinical decision support and translational research. figure 3 illustrates the solution used to address the stated complexities to bda. s107 australasian journal of information systems de silva, burstein, jelinek & stranieri 2015, vol. 19, pp. s99-s115 complexities of big data figure 3. bda solution discri experiment at the lowest layer, data is accumulated at varied grains. high level data such as patient demographics and diagnosis are recorded by medical practitioners while low level blood glucose measurements, blood pressure, body mass index and others are recorded by lab technicians at the screening clinic and by patients and carers in their own homes. structured data is maintained in a relational database while unstructured data, mainly free-flow text such as patient feedback and clinician notes, is saved in flat files. clinical scientists derive standard statistics (patient numbers, average age, average clinician visits etc.) from the granular data for routine reporting. in the secondary layer, time-series data is fed into temporal abstraction techniques in order to embed clinical knowledge into the sequence of data points. blood pressure recordings, blood glucose levels and sensori-motor function data are transformed in this manner. free flow text is pre-processed to a format suitable for analysis; stop word removal, lemmatisation and named entity recognition are conducted in this phase. optionally, a verified ontology as specified by (nguyen et al. 2014; prakasa and de silva 2013) can be used to add context to patient comments and feedback. the third layer consists of two functions, dimensional modelling and the actual bda. temporal abstraction generates structured information with embedded clinical knowledge. for instance the number of visits per patient and the outcomes of each such visit are useful to determine the patient’s current condition and disease trajectory. temporal abstraction is necessary to extract this information that occurs over time at different stages of the medical condition. integrating time-dependent information with static data (such as demographics) necessitates a multi-dimensional structure. this inherent structure can be captured in a dimensional model and implemented as a data warehouse. a novelty in this dimensional model is the cardinality dimension. individual patients attend the screening clinic multiple times, the cardinality dimension captures each visit. except for personal information (which contained attributes such as gender, family history of medical s108 australasian journal of information systems de silva, burstein, jelinek & stranieri 2015, vol. 19, pp. s99-s115 complexities of big data conditions), all other dimensions were composed of attributes recorded for each visit/test. while the fact table would distinguish between records, the cardinality dimension was necessary to distinguish between patients. figure 4 presents the discri dimensional model. bda is the second function. the primary input is semi-structured text data containing patient feedback and clinician notes. clinical abstractions and granular data form the secondary input. figure 4. discri dimensional model the analytics outcomes from bda range from clustering, classification, association rules, predictions, summarisation and visualisation. this primary output is fused with the outputs from clinical abstractions and models of structured information. for instance, the association of patient age with feedback provided can be understood by fusing the bda outcome with the demographic dimension in the warehouse. similarly, fusion of other structured with the unstructured information leads to expedient insights on patients and their conditions. these insights can lead to the five pathways of value generation in healthcare noted earlier, right living, right care, right provider, right value and right innovation. the following figures present an interesting instance of fused outcomes. figure 5 illustrates the use of a temporal abstraction scheme and drill-down queries on the data warehouse to extract s109 australasian journal of information systems de silva, burstein, jelinek & stranieri 2015, vol. 19, pp. s99-s115 complexities of big data information on hypertension from granular data. patients with hypertension were identified by their age groups and by the number of years since diagnosis of hypertension. the drill-down feature in age groups detects a significant drop in the number of 5-10 year hypertension cases in the age sub-groups of 70-75 and 75-80. separately, clinician notes were clustered using apache mahout running on a hadoop instance. the 70-80 age group was found to be closely grouped and further intra-cluster exploration uncovered a distinction in multi-word term association. the same group with a drop in 5-10 year hypertension linked into multi-word term associations related to renal conditions. fusion of information from the structured and unstructured analysis led to this outcome. figure 5. distribution of number of years since diagnosis of hypertension by age groups. although one would normally suggest that the drop in the number of cases in the 80+ age group with greater than 20 years hypertension is largely due to mortality, the current bda has found that this is possibly related to renal disease. results from this large screening study provide further evidence for the role of bda in translational research. figure 6 indicates that urinary and kidney disease play a role in the reduction of people over 80 years attending screening clinics due to medical care moving from observation as is the case in a screening clinic to intervention in primary health care. figure 6. multi-word term association for 70-80 age group cluster, with the sub-group affected by urinary tract infections coloured in brown. 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 prostrate cancer rheumatic fever hiatus hernia rheumatoid… kidney stone urinary tract bladder infection gall stone family history chest pain underactive… terms association s110 australasian journal of information systems de silva, burstein, jelinek & stranieri 2015, vol. 19, pp. s99-s115 complexities of big data this highlights the necessity to include more urinary and renal testing in screening clinics to identify these conditions early in the older age group and provide timely personalised intervention. the aforementioned outcomes from temporal abstraction, information fusion and text analytics can be further scrutinised using conventional data mining techniques such as classification, association and bayesian techniques. an ideal scenario that can be realised is the capacity to predict the likelihood of clinical outcomes based on current knowledge of the patient and similar patients’ information. 12 discussion revisiting the three entities of the business analytics framework (chandler et al. 2011), people, processes and platform, it is evident that bda maintains capacity to fulfil the expectations of the platform entity across the three primary activities of consume, produce and enable. big data in healthcare demonstrates distinct characteristics in comparison to other domains. each patient’s healthcare record represents a temporal sequence of events, and each event maintains a number of attributes (of varying modalities) captured over the duration or a specific point in time. the complexities delineated in this paper arise due to this distinctive nature of healthcare big data. as reflected in the results reported from our experiments, expedient insights can only be derived in collaboration with the domain expert – a clinical scientist in our case study. access to the adequately designed bda platform addresses these complexities, and allows domain experts to interrogate the data and dynamically manipulate hypotheses formulation, thus fulfilling the requirements for decision support. . the evolving role of cds into clinical knowledge management is well positioned to leverage on the capabilities of such a bda platform (burstein et al, 2013). the fusion of highly structured domain knowledge with insights derived from unstructured patient data, for which no fixed schema can be developed in advanced, is a significant contribution towards updated clinical knowledge. the bda platform addresses complexities arising due to the nature of big data described in table 1. the parameters of volume, variety, veracity, sparseness and complexity are exemplified in the reported experiment. resolving the big data complexities in this manner lays a robust foundation for a complete business analytics framework as shown in table 2 with improved processes and empowered people. from the is participation theory perspective, online accessibility to real-time patient information, analytics outcomes and updated clinical knowledge leads to increased stakeholder participation in healthcare delivery and management. participation activities of type, richness, methods and conditions of all stakeholders from healthcare professionals, caregivers to patients themselves are empowered by access to relevant and reliable information. therefore it is very convincing that the five pathways to value generation are sustained by the strong foundation of a bda platform. 13 conclusion significant clinical knowledge and a unique understanding of disease patterns can be acquired by utilising big data technologies for data management and analytics in the healthcare discipline (chen et al, 2012). the nature of data and expectations of the healthcare professional has led to complexities when developing this inter-disciplinary focus area. the primary aim of this paper was to examine these issues in-detail from the perspectives of the participation theory (markus and mao, 2004) and illustrate how a potential solution was applied to the diabetes screening case. the paper presented the big data paradigm, supporting technologies and their relevance to bda in healthcare. thereafter, complexities to bda in healthcare were discussed in detail followed by a proposed solution that addresses these complexities and unifies structured and unstructured data collected within a healthcare environment. the method and outcomes from a prototypical trial conducted on the discri project were presented to implicate the s111 australasian journal of information systems de silva, burstein, jelinek & stranieri 2015, vol. 19, pp. s99-s115 complexities of big data significance and contribution of the proposed solution. the inherent limitations of most data collections, such as missing data, null values, incorrect values and unmatched records were observed and accounted for in the bda process. the fusion of structured and unstructured data is aptly demonstrated in the outcomes and is of significant value in a clinical context. to enhance translational research, data obtained from the annual screening clinic needs to be interpreted in terms of outcome measures following diverse treatment options present in the screening cohort. new associations between personal health status, intervention and individual outcome that also reflects a wider population use is an essential part of current healthcare research. following the updated participatory theory led to engaging domain experts in the process of bda platform design and implementation. we also confirmed that is specialists played a role of change agents in facilitating the adoption of bda platform and leading to a better appreciation of the possibilities of data analytics for clinical decision support through this emergent process of engagement. bda architecture for healthcare applications should overcome the complexities of granular data accumulation, temporal abstraction, multimodality, unstructured data and integration of multi-source data in order to provide a robust platform for effective workflows and improved engagement. the stakeholders of the bda solution have to be clearly identified and fully involved in the process of bda platform design. their role will be in defining the boundaries and expected outcomes of the platform, identifying the right data sources, transformation of unstructured data, integration with structure content and developing analytics pathways for cds. in our case study their participation led to the emergent success of the bda platform as it overcame the limitations to unified data exploration and led to analytics outcomes previously unknown to stakeholders. we also confirmed that strategic benefits of bda are not visible upfront. instead, an overall success of data analytics is generated by long-term use and appears of significant value in terms of knowledge management in a clinical setting. the authors are currently involved in expanding this work to include real-time sensor data into the bda process to further empower clinical decision support. future work can include expansion of this approach to other complex medical conditions which record a multitude of data points, such as cognitive decline (alzheimer's disease) and psychosis (schizophrenia and depression). references bellazzi, r., larizza, c., magni, p., montani, s., and stefanelli, m. 2000. "intelligent analysis of clinical time series: an application in the diabetes mellitus domain," artificial intelligence in medicine (20:1), pp. 37-57. bergvall-kåreborn, b., howcroft, d., ståhlbröst, a., and wikman, a. m. 2010. "participation in living lab: designing systems with users," in human benefit through the diffusion of information systems design science research. springer, pp. 317-326. beyer, m. a., and laney, d. 2012. the importance of ‘big data’: a definition, stamford, ct: gartner). bloom, b. s. 2002. "crossing the quality chasm: a new health system for the 21st century," jama: the journal of the american medical association (287:5), pp. 646-647. boström, h., andler, s. f., brohede, m., johansson, r., karlsson, a., van laere, j., niklasson, l., nilsson, m., persson, a., and ziemke, t. 2007. "on the definition of information fusion as a field of research," iki technical reports, hsiki -tr-07-006, pp.8-14. burstein, f., de silva, d., jelinek, h. f., and stranieri, a. 2013. "multivariate data-driven decision guidance for clinical scientists," ieee 29th international conference on data engineering workshops (icdew), 2013: ieee, pp. 193-199. burton, p. r., clayton, d. g., cardon, l. r., craddock, n., deloukas, p., duncanson, a., kwiatkowski, d. p., mccarthy, m. i., ouwehand, w. h., and samani, n. j. 2007. "genome-wide association study of 14,000 cases of seven common diseases and 3,000 shared controls," nature (447:7145), pp. 661-678. s112 australasian journal of information systems de silva, burstein, jelinek & stranieri 2015, vol. 19, pp. s99-s115 complexities of big data butler, d. 2008. "translational research: crossing the valley of death," nature news (453:7197), pp. 840-842. chandler, n., hostmann, b., rayner, n., and herschel, g. 2011. gartner's business analytics framework, gartner research:g00219420). chen, h., chiang, r. h., and storey, v. c. 2012. "business intelligence and analytics: from big data to big impact," mis quarterly (36:4), pp. 1165-1188. chute, c. g., beck, s. a., fisk, t. b., and mohr, d. n. 2010. "the enterprise data trust at mayo clinic: a semantically integrated warehouse of biomedical data," journal of the american medical informatics association (17:2), pp. 131-135. coen, m. h. 2005. "cross-modal clustering," proceedings of the national conference on artificial intelligence: menlo park, ca; cambridge, ma; london; aaai press; mit press; 1999, p. 932. davenport, t. h. 2006. "competing on analytics," harvard business review: 84, pp. 98-107, 134. davis, d. a., chawla, n. v., blumm, n., christakis, n., and barabási, a.-l. 2008. "predicting individual disease risk based on medical history," proceedings of the 17th acm conference on information and knowledge management: acm, pp. 769-778. de silva, d., burstein, f., stranieri, a., williams, k., and rinehart, n. 2013. "a participatory information management framework for patient centred care of autism spectrum disorder," 24th australasian conference on information systems (acis): rmit university, pp. 1-11. gartner. gartner it glossary big data. retrieved 1 mar, 2015, from http://www.gartner.com/it-glossary/big-data ginger.io. 2015. ginger.io for individuals. retrieved 1 mar, 2015, from https://ginger.io/ goshtasby, a. a., and nikolov, s. 2007. "image fusion: advances in the state of the art," information fusion (8:2), pp. 114-118. groves, p., kayyali, b., knott, d., and van kuiken, s. 2013. "the ‘big data’ revolution in healthcare," new york (ny): mckinsey global institute. hanauer, d., zheng, k., ramakrishnan, n., and keller, b. 2011. "opportunities and challenges in association and episode discovery from electronic health records," ieee intelligent systems (26:5), pp. 83-87. hostmann, b., rayner, n., and friedman, t. 2006. gartner's business intelligence and performance management framework, gartner inc. hu, h., correll, m., kvecher, l., osmond, m., clark, j., bekhash, a., schwab, g., gao, d., gao, j., and kubatin, v. 2011. "dw4tr: a data warehouse for translational research," journal of biomedical informatics (44:6), pp. 1004-1019. jelinek, h. f., wilding, c., and tinely, p. 2006. "an innovative multi-disciplinary diabetes complications screening program in a rural community: a description and preliminary results of the screening," australian journal of primary health (12:1), pp. 14-20. jensen, p. b., jensen, l. j., and brunak, s. 2012. "mining electronic health records: towards better research applications and clinical care," nature reviews genetics (13:6), pp. 395-405. keravnou, e. t. 1997. "temporal abstraction of medical data: deriving periodicity," in intelligent data analysis in medicine and pharmacology. springer, pp. 61-79. khaleghi, b., khamis, a., karray, f. o., and razavi, s. n. 2013. "multisensor data fusion: a review of the state-of-the-art," information fusion (14:1), pp. 28-44. s113 australasian journal of information systems de silva, burstein, jelinek & stranieri 2015, vol. 19, pp. s99-s115 complexities of big data lowe, h. j., ferris, t. a., hernandez, p. m., and weber, s. c. 2009. "stride–an integrated standards-based translational research informatics platform," amia annual symposium proceedings: american medical informatics association, p. 391. markus, m. l., and mao, j.-y. 2004. "participation in development and implementationupdating an old, tired concept for today's is contexts," journal of the association for information systems (5:11), p. 14. miller, k. 2012. "big data analytics in biomedical research," biomedical computation review, pp. 14-21. nguyen, b. v., burstein, f., and fisher, j. 2014. "improving service of online health information provision: a case of usage-driven design for health information portals," information systems frontiers, (17:3), pp. 493-511. prakasa, a., and de silva, d. 2013. "development of user warrant ontology for improving online health information provision," 24th australasian conference on information systems (acis): rmit university, pp. 1-12. propellerhealth. 2015. propeller health the leading mobile platform for respiratory health management. retrieved 1 mar, 2015, from http://propellerhealth.com/ roberts, s. f., fischhoff, m. a., sakowski, s. a., and feldman, e. l. 2012. "perspective: transforming science into medicine: how clinician–scientists can build bridges across research's “valley of death”," academic medicine (87:3), pp. 266-270. schatz, m. c. 2009. "cloudburst: highly sensitive read mapping with mapreduce," bioinformatics (25:11), pp. 1363-1369. sen, a., banerjee, a., sinha, a. p., and bansal, m. 2012. "clinical decision support: converging toward an integrated architecture," journal of biomedical informatics (45:5), pp. 10091017. shahar, y. 1994. "a knowledge-based method for temporal abstraction of clinical data," technical report stanford university. sharshar, s., allart, l., and chambrin, m.-c. 2005. "a new approach to the abstraction of monitoring data in intensive care," in artificial intelligence in medicine. springer, pp. 13-22. stacey, m., and mcgregor, c. 2007. "temporal abstraction in intelligent clinical data analysis: a survey," artificial intelligence in medicine (39:1), pp. 1-24. torra, v. 2003. "on some aggregation operators for numerical information," in information fusion in data mining. springer, pp. 9-26. wisniewski, m. f., kieszkowski, p., zagorski, b. m., trick, w. e., sommers, m., weinstein, r. a., and project, c. a. r. 2003. "development of a clinical data warehouse for hospital infection control," journal of the american medical informatics association (10:5), pp. 454-462. wright, a., and sittig, d. f. 2008. "a four-phase model of the evolution of clinical decision support architectures," international journal of medical informatics (77:10), pp. 641649. yao, j., raghavan, v. v., and wu, z. 2008. "web information fusion: a review of the state of the art," information fusion (9:4), pp. 446-449. copyright: © 2015 de silva, burstein, jelinek, stranieri. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. s114 australasian journal of information systems de silva, burstein, jelinek & stranieri 2015, vol. 19, pp. s99-s115 complexities of big data s115 paper title (paper title style) australasian journal of information systems duy, tran & ma 2017, vol 21, selected papers from ausdm ownership protection of outsourced data using watermarking 1 ownership protection of outsourced biomedical time series data based on optimal watermarking scheme in data mining trung pham duy university of canberra, australia duy.pham@canberra.edu.au dat tran university of canberra, australia wanli ma university of canberra, australia abstract in the biomedical and healthcare fields, the ownership protection of the outsourced data is becoming a challenging issue in sharing the data between data owners and data mining experts to extract hidden knowledge and patterns. watermarking has been proved as a rightprotection mechanism that provides detectable evidence for the legal ownership of a shared dataset, without compromising its usability under a wide range of data mining for digital data in different formats such as audio, video, image, relational database, text and software. time series biomedical data such as electroencephalography (eeg) or electrocardiography (ecg) is valuable and costly in healthcare, which need to have owner protection when sharing or transmission in data mining application. however, this issue related to kind of data has only been investigated in little previous research as its characteristics and requirements. this paper proposes an optimized watermarking scheme to protect ownership for biomedical and healthcare systems in data mining. to achieve the highest possible robustness without losing watermark transparency, particle swarm optimization (pso) technique is used to optimize quantization steps to find a suitable one. experimental results on eeg data show that the proposed scheme provides good imperceptibility and more robust against various signal processing techniques and common attacks such as noise addition, low-pass filtering, and resampling. keywords: ownership protection; biomedical data; watermarking; data mining; particle swarm optimization (pso). 1 introduction in recent years, with continuous development of information technology, an increasing number of hospitals and clinics has digitized patient's medical records and stored these records on independent data servers using e-health technology (li et al. 2011). many countries such as australia, germany, taiwan, and the usa have already deployed e-health technology in their national healthcare network. data sharing or information sharing is necessary for distributed systems in data mining. in healthcare, each collaborator (hospital) needs to share their private local databases with other collaborators in telemedicine system, telediagnosis, telesurgery, and hospital information system (bhatnagar and wu 2013). however, there are multiple danger zones like copyright and integrity violations of digital objects (gupta and raval 2012), and the sharing of medical data also exposes data holders to threat of data theft (bertino et al. 2005). in reality, lots of incidents can be found where e-health systems become found wanting due to the miss-management and privacy violations of sensitive health data. (clearinghouse 2017) provides significant evidence for the above fact and reports more than 44 million healthcare related privacy violations even with the existence of regulations for security and privacy of health information. the verizon data breach investigation review (solutions 2015) states that in 2014 there was only 26 reported incidents of breaches in the healthcare industry. the number of reported data breaches increased 900% by july 2015 to 234 data breaches. usa australasian journal of information systems duy, tran & ma 2017, vol 21, selected papers from ausdm ownership protection of outsourced data using watermarking 2 health insurer anthem had a massive breach which resulted in the leak of 80 million client records (westin 2015). the average annual cost of data breaches to the usa healthcare industry is as high as $7 billion (smith and gotel 2007). moreover, in (chen and xu 2013) further statistics reveal the privacy violations by providing examples where personal health information is stolen or acquired without the authorization of legally obliged parties. data owners, nonetheless, also need to maintain the principal rights over the datasets that they share, which in many cases have been obtained after expensive and laborious procedures. for example, in teleradiology, one of the most successful e-health services at present, security and privacy protection has become a critical issue (prior et al. 2009; ruotsalainen 2010). medical information security requirements are generally defined by the strict ethics and legislative rules of the security policy/profile, and concerned entitles must adhere to them. some countries have their own security and privacy policy; for example, health insurance portability and accountability act (hipaa) in usa (centers for and medicaid 1996), code of federal regulations number 45 (cfr 45) (health and human 2009), and europe's directive 95/46/ec (hänsch and serna) are expression of such a constraint. in australia, the australian law reform commission (australia. law reform 2002) produced the australian privacy law and practice report that is a comprehensive review of the privacy act of 1988. that reviews incorporates privacy regulations on electronic health information systems. cloud computing also significantly contributes to the development of e-health solutions, however considering the fact that cloud infrastructures are managed by third parties who may be curious about the data being stored, confidentiality, integrity and privacy concerns have been raised on the transmitted and stored data. it is necessary to have a right-protection mechanism that can provide detectable evidence for the legal ownership of a shared dataset, without compromising its usability under a wide range of data mining and cloud computing. in such scenarios, the shared data might be illegally sold to third parties by an unauthorized party. in order to cater for such a situation, the data should be right protected so that an unauthorized party might be sued in a court of law. this is only possible, if the data owner is able to prove that the illegally sold data are his property. therefore, it is important to not only protect the privacy of patient (alhaqbani and fidge 2008), but also the ownership (copyright) of the medical data shared with collaborative partners or third party vendors. therefore, it is important that medical data should be right protected in a manner where ownership could unambiguously be determined (kamran and farooq 2012). the important requirement regarding shared medical data is data ownership (copyright) protection. we need effective mechanisms to establish and protect the holders’ rightful possession of the data. two standard methods, namely encryption and watermarking, are currently used to achieve this aim (latifpour et al. 2015). in encryption methods, an encryption algorithm is used to encode information, and a receiver at the opposite site recovers the information with a known decoding key. although the information is encoded on transmission, there will be no guarantee for security of the information after it is decoded at the receiver. for solving such a problem, watermarking technique is introduced to provide the information security after decoding. as an alternative or complement to cryptography, watermarking is mainly used for the copyright protection. in other words, the watermarking is a technique that is able to embed a series of hidden information into an original signal, and to realize the security of information against unauthorized copying, and false claim of owning and exclusiveness. the role of watermarking becomes increasingly important because of the ease of data sharing, particularly through data clouds. watermarking has many applications such as ownership identification, proof of ownership, tamper detection and leak identification. 1.1 current issues in sharing and storing biomedical data one of the major technological and ethical issues governing electronic records is the issue of data privacy. tampered data can lead to false alarms or incorrect diagnoses of patient. as a result, copyright protection, data authentication and security have become challenging issues due to the illegal modification and distribution. as a data hiding and extraction method, digital watermarking is one of the solutions to address this problem and is used for digital rights australasian journal of information systems duy, tran & ma 2017, vol 21, selected papers from ausdm ownership protection of outsourced data using watermarking 3 protection, ownership verification and security purposes (bender et al. 1996). for instance, hospital information system addresses security problems and provides confidentiality, integrity and authentication via watermarking. watermarking applications for medical purposes have been extensively investigated with reference to their security (arsalan et al. 2012; fakhari et al. 2011; ko et al. 2011). although digital watermarking techniques represent a viable solution for the problem of enforcing ownership of medical data (bertino et al. 2005), preliminary studies in watermarking or information hiding techniques have been developed only for embedding text, images, audio or video in to a host signal. moreover the techniques developed for these databases do not transpose well to other biomedical time series data modalities such as eeg and ecg for the following reasons: 1) the redundancy in a time series of biomedical signals such as eeg or ecg is less compared with an image or audio, and hence embedding data in to these time series data is much more difficult since the reduced redundancy limits possibilities of hiding data and has not been investigated; 2) audio signal has slow time-varying feature while biomedical signal is the fast changing-time series; 3) the watermark scheme needs to address a significant challenge related to biomedical data that is insertion of a watermark must not result in changing health and medical data of a patient to a level where a decision maker (or system) can misdiagnose the patient; and 4) in most applications, biomedical time series data such as eeg or ecg is recorded from multiple channels and at relatively high sampling frequencies. some applications require storage and/or transmission of biomedical data recordings over an extended period of time. as a result, biomedical time series data recordings may lead to a large amount of data. 1.2 watermarking schemes addressing the current issues according to these above-mentioned biomedical data characteristics, there are three important issues that watermarking schemes need to address. first, watermarking scheme is required to provide trustworthy evidence for protecting the rightful ownership, while the perceptual difference between the watermarked and the original documents should be unnoticeable to the human observer. second, good watermarking scheme should satisfy the requirement of robustness and resist distortions due to common attacks. the watermark should be detectable and extractable after data manipulations were applied to the watermarked data. moreover, the inserted watermark should be imperceptible to intruders and they should not be able to corrupt it by launching malicious attack. last, watermark extraction is not able to have the original biomedical signal as a large size. these issues motivate us to design an appropriate watermarking scheme for outsourced biomedical data without affecting the perceptual quality of the underlying host signal, having high robust and the original biomedical signal is not required at watermark extraction. on the other hand, transform domains are proven to be more robust toward different attacks (mousavi et al. 2014), and singular value decomposition (svd)-based watermarking is one of the most powerful watermarking schemes in this domain. the robust performance of existing svd-based watermarking methods is not always better than that of frequency-based methods such as gaussian filtering and noising (tsai et al. 2012) for most of attacks (chang et al. 2005) developed in the spatial domain. a better approach to enhance the robustness of svd-based methods is to employ this transform along with the frequency transform for biomedical data. it has been reported that among the transform domain methods, discrete wavelet transform (dwt) is more suitable for achieving robust watermarking and imperceptible (mishra et al. 2014). the present work uses dwt-svd hybrid transform to carry out watermarking embedding for outsourced biomedical data. in general, regarding the extraction process, the watermarking methods can be classified in two schemes which are non-blind and blind. in the non-blind watermarking scheme, the extractor can extract watermark data using the original signal. as opposed to that, the blind watermarking methods do not need the original signal for extracting the watermark data. although the non-blind scheme's complexity is low, they suffer from the following two distinct disadvantages (gupta and raval 2012): australasian journal of information systems duy, tran & ma 2017, vol 21, selected papers from ausdm ownership protection of outsourced data using watermarking 4 i. security compromise: attacker may claim the ownership by inserting another watermark in the cover object as non-blind detection does not guarantee unequivocal claims of ownership by the content creator. ii. practical application constraints: the presence of original content is required during detection for every watermarking application in non-blind scheme. with a large amount of biomedical data, it is very difficult to ensure the present of original biomedical data in non-blind watermarking scheme. in order to overcome these disadvantages, blind scheme is suitable for biomedical time series data since the original signal is not required. 1.3 imperceptibility and robustness issues in watermarking techniques an acceptable watermarking technique including biomedical data watermarking needs to satisfy two main requirements which are imperceptibility and robustness (voyatzis and pitas 1999). imperceptibility refers to perceptual quality of the data being protected or the quality of biomedical signal should be retained after adding the watermark. imperceptibility can be evaluated using both objective and subjective measures. the perceptual difference between original biomedical data and watermarked one can be evaluated by peak signal noise ratio (psnr). a larger psnr value indicates that the watermarked biomedical signal more closely resembles its original signal, meaning that watermarked biomedical signal has better imperceptibility. the watermark scheme needs to address a significant challenge related to biomedical data that is insertion of a watermark must not result in changing health and medical data of a patient to a level where a decision maker (or system) can misdiagnose the patient. if a patient is misdiagnosed, it might not only put his life on risk but also result in significantly enhancing the cost of healthcare. this can be guaranteed by a good imperceptibility. according to (chen et al. 1998), psnr above 40 db indicates a good perceptual imperceptibility. robustness is ability to extract a watermark from a watermarked biomedical signal after various signal processing attacks. biomedical signals are not likely to be subject to the same type of malicious attack as downloaded image, audio or video files. however, attacks such as pre-processing signals or downsampling of large data files to allow more efficient data transmission could be an issue. the robustness of the watermark is verified against different attacks such as low pass filtering, addition of gaussian noise, different sampling rate, and cropping. it is sufficient if the embedded data is robust to simple signal processing techniques necessary for efficient transmission. normalized correlation (nc) and bit error rate (ber) are metrics to determine the robustness of watermarking scheme. 1.4 motivations an effective biomedical data watermarking scheme must satisfy both imperceptibility and robustness goals. if these two requirements are not well achieved, a poor perceptual imperceptibility leads to the fact that a decision maker (or system) can misdiagnose the patient. meanwhile intruders should be able to corrupt inserted watermark by launching malicious attack with less robust watermarking scheme, resulting in not providing trustworthy evidence for protecting the rightful ownership. however, there is a trade-off between these goals. for example, increasing the quality of watermarked signal results in a decrease of robustness against attacks. on the design of watermarking scheme, two significant but conflicting requirements, imperceptibility and robustness, should be taken into consideration because the targeted balance between the two requirements. hence, dealing with the trade-off as an optimization problem may adaptively achieve the specific balance for the intended application. in order to minimize such conflict, we need to find out an optimal balance of the contradictory watermarking requirements, thus improving watermarking performance. one recently developed and widely used way is to utilize evolutionary and artificial intelligence methods to view the watermarking problems as an optimization problem (arsalan et al. 2012). australasian journal of information systems duy, tran & ma 2017, vol 21, selected papers from ausdm ownership protection of outsourced data using watermarking 5 in this manner, the performance of the watermarking scheme can be improved. artificial intelligence (ai) techniques based on objective function such as tabu-search (sriyingyong and attakitmongcol 2006), differential evolution (de) (storn and price 1997), and genetic algorithm (ga) (kumsawat 2010) have been demonstrated to be very effective in solving conflicting requirements of watermarking. all these techniques intend to work on minimizing the trade-off between the two mandatory objectives of watermarking which are imperceptibility of the embedded watermark and robustness of the embedding scheme. a proper balance between imperceptibility and robustness also depends upon embedding strength or the strength-of-the-watermark (scaling factor) (mishra et al. 2014; run et al. 2012). a large value of scaling factor favors robustness while a small value favors imperceptibility. hence, scaling factor governs the trade-off between imperceptibility and robustness. conventional ai techniques have advantages in computing speed for watermarking as they do not bother about scaling factor provided by the user in advance. yet conventional ai methods cannot handle the automatic balance between imperceptibility and robustness in watermarking. there is no exact algorithm to choose the value of scaling factor. most of them are based on trial-and-error method, others use the fixed parameters without any optimization in the literature (wang et al. 2011). instead of using the fixed parameters, another trend is witnessed, intelligent systems and evolutionary algorithms are integrated into watermarking approach to optimize the parameters. in these artificial intelligence techniques, the metaheuristic algorithm is chosen to find the suitable scaling factor to overcome the optimization problem. there are several kinds of meta-heuristic algorithms, for example genetic algorithm (ga) (holland 1992), ant colony (dorigo et al. 1999), and particle swarm optimization (kennedy 2011). particle swarm optimization (pso) is another important meta-heuristic algorithm since it has fast convergence and few parameters to adjust (kennedy et al. 2001). pso is simple to implement and computationally efficient as well. basic principle of the pso algorithm is the clever exchange of information between the global and local optimal values. a global optimum is achieved by updating generations based on movement and intelligence in an evolutionary system. in a similar pattern of other evolutionary and stochastic computation techniques, such as ga and tabu-search, pso shares similar characteristics, but it was proven that pso could obtain better results in a faster and cheaper way compared with ga methods. besides, another fascinating feature of pso is that there are fewer parameters to adjust compared with ga methods (lei et al. 2013). due to its advantage over other global optimization techniques, a myriad of watermarking scheme is developed and implemented based on pso in this research. 1.5 research focus and contribution our current research work focuses on optimizing the trade-off between the two problems in biomedical data watermarking: imperceptibility/visual quality of the signed or attacked biomedical signal and the issue of robustness. we propose a blind biomedical data watermarking scheme based on the hybrid dwt-svd transform and identify the optimal quantization using pso. recent studies prove that computational intelligence techniques, especially pso (hassan et al. 2005) are ideally suited for solving constrained optimization problem in real-time. pso has the following advantages: 1) pso is easier to implement and there are fewer parameters to adjust; 2) pso has a more effective memory capability since every particle remembers its own previous best value as well as the neighborhood best; and 3) since all the particles use the information related to the most successful particle, pso is more efficient in maintaining the diversity of the swarm. in order to achieve a trade-off between imperceptibility and robustness, pso is utilized to search for the optimal embedding parameter in our study. an objective function used in this algorithm is a linear combination of psnr and nc. the nc is a metric to determine the robustness and therefore the research selects three different signal processing operations (additive noise, re-sampling, and low-pass filtering) as attacks to evaluate it. experimental results show that our proposed watermarking scheme yields a good imperceptibility and more robust against various signal processing and common attacks. australasian journal of information systems duy, tran & ma 2017, vol 21, selected papers from ausdm ownership protection of outsourced data using watermarking 6 the main contributions of this research are as follows 1. developing an appropriate efficient blind watermarking algorithm that is suitable for outsourced time series biomedical data such as eeg or ecg in data mining; and 2. determining the optimal balance of the contradictory watermarking requirements using pso to find out the optimal quantization steps for different host time series biomedical signals and watermarks. in addition, chaotic map with the chaotic characteristic is also used to enhance the confidentiality of the proposed watermarking scheme. with the proposed blind watermark detection algorithm, the biomedical watermarking scheme can extract the watermark thus saving a lot of space for storing the original biomedical data and watermark. the rest of this paper is organized as follows. in section 2, we review the state-of-the-art biomedical data watermarking schemes. section 3 introduces the related background of our proposed scheme including chaotic encryption, singular value decomposition (svd), discrete wavelet transform (dwt), and particle swarm optimization algorithm (pso). section 4 presents details of the proposed biomedical data watermarking scheme. our experimental results are presented and analyzed in section 5. finally, section 6 draws conclusions and presents discussions on the future work. 2 related work biomedical data sharing via distributed systems and data clouds in healthcare introduces new security and privacy threats as well as data integration issues. hipaa mandates that while transmitting information, a patient's privacy and confidentiality must be protected (lee and lee 2008). in addition, it is of crucial importance to implement a security protocol which will have powerful communication and storage security (malasri and wang 2007). encryption and cryptographic algorithms have been used to secure data during the communication and storage. as a result, the final data will be stored in encrypted format (hu et al. 2007; maglogiannis et al. 2009; wang et al. 2010). cryptography is the most common method of protecting digital content. however, once an encrypted data segment is decrypted, its content has no protection. in addition, large computational overhead is needed in encryption-based techniques. as an alternative or complement to cryptography, data can be hidden in other innocuous-looking objects so that their presence is not revealed through steganography or watermarking. watermarking techniques have been thoroughly studied as a means to achieve proof of ownership and transaction tracking (cox et al. 2007). encryption can offer confidentiality and integrity in content protection, and the decrypted content can be further protected using digital watermarking. watermarking techniques for embedding text, images, audio or video in a host signal have been developed. however, there has been little research on watermarking for time series biomedical data. based on the embedding information concept, watermarking algorithms can be broadly classified as either spatial or transform domain (zain and clarke 2011). spatial domain algorithms directly insert the watermark into the biomedical signal, whereas transform domain algorithms embed the watermark based on a modified version of the biomedical signal. in spatial domain, (kong and feng 2001) proposed an elementary watermarking technique for ecg signals. they describe three popular watermarking techniques applied to eeg signals: benders patchwork (bender et al. 1996), least significant bit (lsb) (van schyndel et al. 1994), and quantization index modulation (qim) (chen and wornell 2001) watermarking with regard to their ability to verify eeg signal integrity after noise contamination resulting from communication. lsb watermarking scheme in support of proof-of-ownership for ecg signals is proposed in (ibaida et al. 2011). however, lsb watermarks provide poor robustness to malicious alterations. the authors of (kaur et al. 2010) describe a spread spectrum watermarking scheme that embeds robust and imperceptible watermarks into ecg signals. however, such a scheme addresses security considerations only during communication of the data rather than over the course of sharing it. these spatial domain approaches are fast, simple and provide high capacity for embedding watermarks, however, they cannot survive or less robust against noise or signal processing australasian journal of information systems duy, tran & ma 2017, vol 21, selected papers from ausdm ownership protection of outsourced data using watermarking 7 attacks (zain and clarke 2011). furthermore, once the method is uncovered, embedded watermark can be easily modified by a third party. these techniques are simple and straightforward but do not strongly ensure protection against removal and robustness of the watermarked biomedical data. due to the shortcomings of watermarking in the spatial domain, most watermarking techniques operate on a transformed domain to provide high robustness. the robustness can be increased by making the human visual system (hvs) less sensitive (heylen and dams 2008). to achieve this, prior to embedding the watermark, transformation such as discrete fourier transform (dft), discrete cosine transform (dct), dwt, or svd is applied onto host signal to obtain the watermarked signal. in addition, the watermark embedded in the transform domain is more imperceptible than that in time domain according to theoretical analysis and simulation results (chen et al. 2014). (engin et al. 2005) proposed a wavelet transformation based watermarking technique for ecg. they used techniques like scrambling matrix to shuffle the secret data information embedded in the cover signal. (he et al. 2012) proposed a self-synchronized watermark technology to protect the ecg. their study confirmed that the use of wavelet-based quantization watermarking on ecg signal is adequate for patient protection. (ramu et al. 2016) proposes ecg stenography using continuous ant colony optimization and dwt-svd watermark embedding techniques. the authors of (pham et al. 2015) proposed an approach that uses discrete wavelet transform (dwt) to decompose eeg signals and svd to embed watermark into the decomposed eeg signal. (jero et al. 2014) proposed a steganography approach based on dwt and svd to hide patient confidential information along with the ecg data. generally, these above-mentioned watermarking methods usually employ pre-defined embedding rules to determine their embedding parameters, such as embedding strengths, and thresholds, either empirically or experimentally. since the embedded results relying on the predefined rules always show the same performance, these watermarking schemes cannot approach the inherent performance upper limit. if the performance of embedded outcome is not satisfactory in some respects, the only solution is to adjust the embedding parameters empirically so that better watermarked results are obtained. however, empirically settings the parameters is an awkward process that lacks systematic techniques (huang and wu 2009). in addition, watermarking algorithms need larger parameter space, therefore, it is often difficult to determine optimal watermarking parameters empirically or experimentally. as a result, these watermarking methods do not exhibit desirable performance (lai 2011). it is necessary to build a watermarking scheme which aims to provide a robust and adaptive system for biomedical data protection against copyright infringement issues. to attain or approach the upper performance limit of previously watermarking algorithms, we must determine their optimal watermarking parameters. however, as stated above, it is usually difficult to empirically determine optimal watermarking parameters. a popular way of solving the optimal watermarking problem is to consider it as an optimization problem (wang et al. 2011). 3 background 3.1 chaotic encryption chaotic maps have been frequently used in digital watermarking. chaotic encryption of a watermark image is performed using arnold transform (zheng et al. 2007) which is also called cat face transfer and is given by ( 𝑥′ 𝑦′ ) = ( 1 1 1 2 ) ( 𝑥 𝑦 ) (𝑚𝑜𝑑 𝑁) (1) where (x, y) is the pixel of the watermark image, (x', y') is the pixel of the watermark image after scrambling, and n is order of watermark image matrix. since the arnold transform is periodic, the scrambling number can be considered as the key to enhance the security. australasian journal of information systems duy, tran & ma 2017, vol 21, selected papers from ausdm ownership protection of outsourced data using watermarking 8 3.2 singular value decomposition (svd) let 𝐴 = (𝑎𝑖𝑗)𝑁×𝑁 be an n×n matrix. the svd of matrix a is represented in the form 𝐴 = 𝑈𝑆𝑉𝑇, where u and v are orthogonal matrices, s is a diagonal matrix with nonnegative elements, and superscript t denotes matrix transposition. the diagonal elements of s, denoted by 𝜎𝑖, 0 ≤ 𝑖 < 𝑁, are called the singular values (svs) of a and are assumed to be arranged in decreasing order, i.e., 𝜎𝑖 > 𝜎𝑖+1. the columns of u, denoted by ui , 0 ≤ 𝑖 < 𝑁 , are called the left singular vectors, while the columns of v, denoted by vi, are called the right singular vectors of a. the svd has some interesting properties as follows: (i) the sizes of the matrices for svd transformation are not fixed, and the matrices need not to be square, (ii) changing svs slightly does not affect the quality of the signal much, (iii) the svs are invariant under common signal processing operations, and (iv) the svs satisfy intrinsic algebraic properties. 3.3 discrete wavelet transform (dwt) dwt employs extensive time window for low frequencies and short time window for higher frequencies. dwt is widely used for the time-frequency analysis of biomedical signals (jahankhani et al. 2006; orhan et al. 2011), especially in a time series biomedical data such as eeg signal analysis due to its non-stationary characteristics. as eeg signal is the fast changing-time series with continuously random changes, we use the haar wavelet which is more suitable for such fast changing time-series compared to daubechies wavelets, mexican hat wavelets and morlet wavelets which are better suited for smoothly changing time series (percival and walden 2006). in addition, the haar wavelet is also simple, fast and exactly reversible which is necessary to reconstruct cover signal in digital watermarking. each wavelet decomposition of the original signal halves the frequency and length of the signal. the haar function ψ used as the mother wavelet generates a set of wavelets as follows: ∁𝑎,𝑏= ∑ 𝑐(𝑡)𝛹𝑎,𝑏(𝑡) 𝑁𝑠𝑎𝑚𝑝 (2) where 𝛹𝑎,𝑏(𝑡) = 1 √𝑠 𝛹𝑎,𝑏 ( 𝑡−𝜏 𝑠 ), a denotes the dilation index, b the translation index, s the scale factor and τ the displacement. 3.4 particle swarm optimization algorithm (pso) pso (kennedy 2011) is a population-based stochastic algorithm developed for continuous optimization. in pso, each particle which represents a potential solution will search for optimal coordinates in the problem space. each particle has its own set of attributes including position, velocity, and a fitness value which is obtained by evaluating a fitness function at its current position. the algorithm starts with the initialization of particles with random position and velocities so that they can move in the solution space. then, these particles search the solution space for finding better solutions. each particle keeps track of its personal best position found so far by storing the coordinates in the solution space. the best position found so far by any particle during any stage of the algorithm is also stored and is termed as the global best position. the velocity of every particle is influenced by its personal best position (autobiographical memory) and the global best position (publicized knowledge). the new position for every particle is calculated by adding its new velocity value to every component of its position vector. let d denote the swarm size. each individual particle 𝑖, 1 ≤ 𝑖 ≤ 𝐷 has its own position pi and velocity vi. these particles search for the optimal value of a given objective function iteratively, then locate their individual best positions 𝑝𝑖 𝑏𝑒𝑠𝑡 (pbest) and keep track of the global position 𝑝𝑔𝑖 𝑏𝑒𝑠𝑡 (gbest) from all best positions through a search space. with respect to the two best values, the velocity and position of particle 𝑖(1 ≤ 𝑖 ≤ 𝐷) in iteration t+1 are updated by: 𝑝𝑖(𝑡 + 1) = 𝑝𝑖(𝑡) + 𝑣𝑖(𝑡 + 1) (3) australasian journal of information systems duy, tran & ma 2017, vol 21, selected papers from ausdm ownership protection of outsourced data using watermarking 9 𝑣𝑖(𝑡 + 1) = 𝑐0𝑣𝑖(𝑡) + 𝑐1𝑟1 (𝑝𝑖 𝑏𝑒𝑠𝑡(𝑡) − 𝑥𝑖(𝑡)) + 𝑐2𝑟2 (𝑝𝑔𝑖 𝑏𝑒𝑠𝑡(𝑡) − 𝑥𝑖(𝑡)) (4) 𝑥𝑖(𝑡 + 1) = 𝑥𝑖(𝑡) + 𝑣𝑖(𝑡 + 1) (5) where c1 and c2 are referred to as cognitive and social parameters, respectively, which are positive constants, r1 and r2 are random numbers uniformly distributed in the range of [0, 1]. c0 is the inertia weight in the range of [0, 1], which controls the momentum of the particle and tunes changes in these values. these coefficients control how far a particle moves in a single iteration. vi is the moving distance in one-step for a particle i and is limited to the range of [vmin, vmax], where vmin and vmax are the minimum and maximum moving distance in one step, respectively. the qualitative measure of the selection of pso algorithm parameter can be found in (trelea 2003). from the pso equations, it is known that the trade-off is related to the pso parameters such as c0, itermax, c1 and c2. based on the analysis in (trelea 2003), the parameter couples that are close to the center of the stability triangle lead to quick convergence, while parameter couples that are close to its borders need many iterations to converge. after going through the whole process iteratively, the evaluated objective function reaches the desired termination criterion. 4 performance evaluation framework the performance of the proposed watermarking method is mainly investigated by measuring its imperceptibility and robustness. the imperceptibility requirements guarantee that the watermarked biomedical data are useable for diagnosis and other clinical uses. robustness, basically, is defined as the degree of resistance of a watermark scheme to modifications of host signal due to either common signal processing, or operation devised specifically in order to render the watermark undetectable (tefas et al. 2009). 4.1 perceptual transparency/imperceptibility the watermark must be embedded without affecting the perceptual quality of the underlying host signal. the procedure is imperceptible if the human audio/visual system (ha/vs) cannot differentiate between the original host signal and a host signal with inserted data. these perceptibility measurement terms are explained as applied to images, but they can also be used with time series data such as eeg and ecg, as the clinician views the ecg and eeg to make a diagnosis. similar to digital image watermarking, biomedical data such as eeg or ecg is based on human visual system (hvs) and is typically analyzed in two ways: 1) visual inspection by human experts and 2) automatic analysis using processing algorithms. watermarking techniques need to reconstruct biomedical data without introducing any errors in such analyses. according to (planitz and maeder 2005), biomedical data is mainly used for diagnosis, thus the imperceptibility of the watermark should be as high as possible. distortions to the original due to the watermark may result in wrong interpretation of the data. common trend for analyzing imperceptibility is through objective and subjective methods, and is utilized in the present study as well. as reported in the literature, imperceptibility is through an objective process carried out by considering psnr in eq. (6) which evaluates the perceptual difference between original biomedical data and watermarked one. psnr(x,x') is used to represent the imperceptibility, which denotes the peak signal to noise ratio between the original signal x and watermarked signal x'. psnr(x,x') measure is defined as follows: 𝑃𝑆𝑁𝑅 = 20 log10𝑚𝑎𝑥(𝑥) √1 𝑁 ∑ (𝑥 − 𝑥′)2𝑁 𝑛=1 (6) a larger psnr value indicates that the watermarked biomedical signal more closely resembles its original signal, meaning that watermarked biomedical signal has better imperceptibility. in biomedical watermarking application, perceptual similarity must be very high to avoid any risk of misdiagnosis. therefore, minimum psnr of 40–50 db is advised (chen and ramabadran australasian journal of information systems duy, tran & ma 2017, vol 21, selected papers from ausdm ownership protection of outsourced data using watermarking 10 1994). if the value of psnr is large, it indicates that the noise or distortion due to the embedded watermark is very small. 4.2 robustness robustness is the resistance of watermark signal against common signal processing and malicious attacks. medical signals are not likely to be subject to the same type of malicious attack as downloaded image, audio or video files. however, attacks such as pre-processing signals, or downsampling of large data files to allow more efficient data transmission could be an issue. the robustness of the watermark is verified against different attacks such as low pass filtering, addition of gaussian noise, different sampling rate, and cropping. it is sufficient if the embedded data is robust to simple signal processing techniques necessary for efficient transmission. normalized correlation (nc) and bit error rate (ber) are adopted to evaluate robustness in the proposed watermarking scheme. nc between original watermark x and extracted watermark after attack x' is a metric to determine the robustness, and is calculated as follows: 𝑁𝐶(𝑋, 𝑋′) = ∑ ∑ 𝑥(𝑛,𝑚) × 𝑥′(𝑛,𝑚)𝑀 𝑚=1 𝑁 𝑛=1 √∑ ∑ 𝑤(𝑛,𝑚)2 𝑀 𝑚=1 𝑁 𝑛=1 × √∑ ∑ 𝑤′(𝑛,𝑚)2 𝑀 𝑚=1 𝑁 𝑛=1 (7) in eq. (7), nc is between 0 and 1. if nc is close to 1, the similarity between original watermark x and extracted watermark x' is very high. otherwise, nc is close to zero, the similarity between x and x' is very low. the error in extracted watermark bits due to watermarking process can be measured using ber (loukhaoukha et al. 2011) as given in eq. (8). here, wret is the number of retrieved watermark bits without error; worg is the total number of original watermark bits. 𝐵𝐸𝑅 (𝑊,𝑊′) = ∑ 𝑊𝑟𝑒𝑡 𝑊𝑜𝑟𝑔 × 100 = ∑ ∑ 𝑤(𝑛,𝑚)⊕𝑤′(𝑛,𝑚)𝑀 𝑚=1 𝑁 𝑛=1 𝑁 ×𝑀 (8) where the symbol ⨁ is exclusive-or (xor) operator. the ber measures the difference between an original watermark and the corresponding extracted watermark. a lower ber suggests that the extracted watermark resembles the original watermark more closely. ber is used to evaluate our watermarking methods against signal distortions. 5 the proposed biomedical data watermarking scheme our proposed scheme adopts a blind digital watermarking scheme based on dwt-svd with pso optimization for time series biomedical data. the proposed method can extract the embedded watermark without any information from the original watermark. the proposed approach includes three primary phases which are watermarking embedding, watermarking extraction and quantization step optimization with pso. embedding and extraction phase are presented in fig. 1 and fig.2, respectively, and the quantization step optimization with pso is shown in fig. 3. 5.1 watermarking embedding watermarking scrambling algorithm is first used in order to dispel the pixel space relationship of the binary watermarking image and to improve the whole digital watermarking system. we use arnold transform to encrypt the watermark image. in this research, the key for arnold transform is denoted as k which is the number of scrambling. australasian journal of information systems duy, tran & ma 2017, vol 21, selected papers from ausdm ownership protection of outsourced data using watermarking 11 figure 1. diagram of embedding the biomedical signal watermark the steps of the biomedical watermarking embedding are summarized as follows: step 1: the watermark w after arnold transformation is converted into a onedimensional watermark sequence of length l. step 2: the original biomedical signal x is decomposed by two-level dwt using a haar wavelet filter in order to get three sets of coefficients d1, d2 and a2 (detailed and approximate coefficients, respectively). in our observations, larger decomposition level will not increase the watermarking robustness while it causes intensive computation. thus we choose two as decomposition level of the haar wavelet to trade-off between robustness and imperceptibility. step 3: the set of coefficients a2, corresponding to low frequency part, is divided into k non-overlapping segments, so that each watermark bit is inserted into one segment. this ensures a better distribution of watermark bits over the entire biomedical signal, improving the robustness of the watermark against different attacks. step 4: each segment is rearranged into a (r × r) matrix block named ml. step 5: svd is performed to decompose each matrix ml into three matrices: ul, sl, and vl. the svd operation is represented as follows: 𝑀𝑙 = 𝑈𝑙 × 𝑆𝑙 × 𝑉𝑙 𝑇 (9) step 6: insert the watermark. due to the stability of the matrix sl under different attacks, the insertion of the watermark is performed by manipulating the coefficient in the highest singular value si(1,1) of each matrix si by adaptive dither modulation (dm) quantization methods. as the first singular values have the highest energy values, they are used to embed the watermark in order to guarantee the robustness and transparency. in addition, the popular dm quantization method has good robustness and blind nature, thus it is used in the embedding process. this embedding strategy can be formulated in the following quantization function: { 𝑆′𝑖(1,1) = 𝑟𝑜𝑢𝑛𝑑 [ 𝑆𝑖(1,1) ∆ ] + 3 4 ∆ 𝑖𝑓 𝑤𝑙 = 1 𝑆′𝑖(1,1) = 𝑟𝑜𝑢𝑛𝑑 [ 𝑆𝑖(1,1) ∆ ] + 1 4 ∆ 𝑖𝑓 𝑤𝑙 = 0 (10) where δ is quantization step, round[.] is rounding to the nearest integer value. it is obvious that the quantization step is significant in terms of both robustness and imperceptibility. the larger the quantization step is, the more robust, but less transparent, the watermarking scheme is. therefore, the quantization step should be specially developed to achieve optimal performance. step 7: each modified singular value is reinserted into matrix si and inverse svd transformation is conducted to obtain the watermarked block m'l which is given by: australasian journal of information systems duy, tran & ma 2017, vol 21, selected papers from ausdm ownership protection of outsourced data using watermarking 12 𝑀′𝑙 = 𝑈𝑙 × 𝑆′𝑙 × 𝑉𝑙 𝑇 (11) step 8: the modified blocks are rearranged to one-dimensional vector and concatenated in order to obtain the modified approximation vector a'2 step 9: the watermarked signal x' is obtained by calculating the two-level inverse dwt using modified approximation vectors a'2 and the original detailed vectors d1 and d2. 5.2 watermark extraction figure 2. diagram of the biomedical signal watermark extraction the detection is rather simple when only the watermarked signal and the watermark embedded positions used in watermark embedding are needed to extract the watermark. the watermarking extraction includes the following steps: step 1: performing two-level dwt to the watermarked and possibly distorted host biomedical signal x' using a haar wavelet filter to obtain three sets of coefficients d'1, d'2 and a'2. step 2: dwt coefficient, a'2 is divided into different non-overlapping blocks m'l with the same block length as that in the watermark embedding process. step 3: svd transformation is applied on each block to produce singular values 𝑀′𝑙 = 𝑈′𝑙 × 𝑆′𝑙 × 𝑉′𝑙 𝑇 (12) step 4: the largest singular value of each diagonal matrix s'i located at the same position in the pre-embedding process is calculated. step 5: let ∅ = (𝑆′𝑙 − 𝑓𝑙𝑜𝑜𝑟 ⌈ 𝑆′𝑙 ∆ ⌉) × ∆, where floor[.] is the rounding function which rounds the elements to their nearest integer. the embedded meaningful watermark sequence is extracted as follows: { 𝑤′𝑙 = 1 𝑖𝑓 ∅ ≥ ∆ 2 𝑤′𝑙 = 0 𝑖𝑓 ∅ < ∆ 2 (13) step 6: organizing a 2-d matrix from the watermark sequence, then the binary watermark image w' is obtained by arnold transformation with key k. the biomedical watermarking extraction phrase does not require original cover signal at the receiver, thus the proposed approach constitutes a blind watermarking scheme. 5.3 pso optimization performance in the design of biomedical watermarking system, there are two goals that are always conflicted. these goals are imperceptibility and robustness. in order to minimize such conflict, this work employs the pso algorithm to search for optimal quantization step, thus allowing the system to achieve optimal performance. australasian journal of information systems duy, tran & ma 2017, vol 21, selected papers from ausdm ownership protection of outsourced data using watermarking 13 in biomedical watermarking, the quantization step is signal-dependent where different biomedical signals require different quantization steps, rather than a fixed one. as seen in insertion process, the watermark is carried into the biomedical signal by quantization of the selected coefficient of each matrix sl. large quantization step (high δ) of coefficients results in better robustness and more distortion of biomedical signal, while a small quantization step (low δ) leads to low robustness and low distortion. hence, the parameter δ must be carefully selected for each biomedical signal, in order to ensure the best performance in terms of imperceptibility and robustness. however, the empirical selection of δ is not an optimal solution. the selection process must be automatically performed in order to improve the performance of the proposed watermarking method. it is clear from this discussion that any objective function used to optimize watermark embedding should take both psnr and nc into account. to solve this problem, the pso optimization is used to find the adequate quantization step, for each signal, which guarantees the best imperceptibility-robustness compromise. the optimization process for finding the suitable quantization step, δ in our watermarking scheme is shown in fig. 3. the most critical step in the optimization process is the definition of a reliable objective function. an objective function is a fitness measure on solution represented by each chromosome. its value tells how well the chromosome satisfies the final goal. the objective value function should be designed as a function of both imperceptibility and robustness to obtain the optimal performance by pso, the objective function can be designed as 𝑂𝑏𝑗𝑒𝑐𝑡𝑖𝑣𝑒 = 𝑓(𝑖𝑚𝑝𝑒𝑟𝑐𝑒𝑝𝑡𝑖𝑏𝑖𝑙𝑖𝑡𝑦, 𝑟𝑜𝑏𝑢𝑠𝑡𝑛𝑒𝑠𝑠) (14) psnr and nc are employed to represent the imperceptibility and robustness of our scheme. therefore, the objective function is of the following form: 𝑂𝑏𝑗𝑒𝑐𝑡𝑖𝑣𝑒 = max(𝑃𝑆𝑁𝑅 + 𝛾 × 1 𝑅 ∑𝑁𝐶𝑖) 𝑘 𝑖=1 (15) where r is the number of attacks. weighting factors are introduced as significant difference that might take place between the metrics of the watermarked biomedical host and the extracted watermark. as the psnr is much larger as compared to the associated nc values therefore a weighting factor γ is used to balance out the influences caused by the two parameters. generally speaking, the psnr value should be greater than 40db (chen et al. 1998) which guarantees the good perceptual transparency, while the nc in eq. (7) value lies between 0 and 1. therefore, there is a strong need to include weighting factor. the values of quantization step are obtained by implementing pso algorithm. for each iteration in pso, the value of δ is examined for several attacks, such as noise attack, resampling attack, and cropping attack. due to the flexibility of the developed system, the other attacking schemes can easily be added to the system or replaced with those used in the pso optimization process. at the end of pso iteration, we will obtain the near optimum quantization step. to find an optimal solution in the objective function, pso is first initialized with a group of random particles, each of which represents a candidate solution to the problem, then searches for optima by updating generations. in order to gain the optimal performance, fobj should be optimized at pso processes. using the above objective function fobj, the quantization step δ can be optimally searched to achieve the best of both biomedical data signal quality and watermark robustness. australasian journal of information systems duy, tran & ma 2017, vol 21, selected papers from ausdm ownership protection of outsourced data using watermarking 14 figure 3. pso optimization performance process fig.3 illustrates the pso optimization performance process. the main steps can be summarized as follows: step 1 (initialization): generate an initial population of individuals (quantization steps) and evaluate the fitness values. repeat until maximum number of generations reached. step 2 (watermarking embedding): embed the watermark in cover biomedical signal using the solution (trial individual) (section 5.1). step 3 (attacks): applying the attacks (additive noise, re-sampling, and low-pass filtering) on watermarked biomedical signal. australasian journal of information systems duy, tran & ma 2017, vol 21, selected papers from ausdm ownership protection of outsourced data using watermarking 15 step 4 (extraction): extract the watermarks from the corrupted watermarked biomedical signals (section 5.2). step 5 (objective evaluation): evaluate the robustness between the watermark and extracted ones, and imperceptibility between watermarked signal and original one. using these compute the fitness value/objective , eq. (15). step 6 (pso based training): select the solution for next generation depending on the quality of the solution in pso training. termination: check the stopping criteria. if yes then stop, the best quantization step (δ) is found; otherwise go to step 2. 6 experiments and results in our experiments, the deap dataset (dataset for emotion analysis using electroencephalogram, physiological and video signals) which is an open database proposed by koelstra et al. (koelstra et al. 2012) was used as the original eeg signals. the 5 random channels (fp1, fc1, f3, p7 and t7) of 32 subjects were chosen to test. a binary logo image with size 32x32 will be used as the watermark image in fig. 7(a). we should examine the effect of quantization steps on imperceptibility and robustness to find out the searching range of quantization steps. figure 4. effect of quantization steps on psnr in different eeg channels. to examine the effect of quantization on visual quality of watermarked eeg signals, we plot its corresponding computed values as a function of δ. figure 4 depicts the effect of quantization steps on psnr for 5 random eeg channels. it is clear that psnr and δ are inversely proportional to each other for all channels. (chen and ramabadran 1994) suggest the minimum psnr of 40–50 db in their research, hence, quantization steps in figure 4 should be less than 30. australasian journal of information systems duy, tran & ma 2017, vol 21, selected papers from ausdm ownership protection of outsourced data using watermarking 16 figure 5. effect of quantization steps on nc values with different attacks we investigated the variation of nc(x, x') with respect to quantization steps (δ). figure 5 plots the effect of quantization steps on nc(x, x') value of fp1 channel of subject 1 with different attacks. it is clear from this figure that nc(x, x') values vary only between the range of δ which is [1, 20]. with δ > 20, the nc(w, w) values get stabilized. this is specifically true for all attacks including noise addition, re-sampling and low-pass filtering. for these attacks, the nc(x, x') values with respect to δ are throughout constant. thus, it is concluded that for the attacks used by us in this simulation, the range [1, 20] of δ is an appropriate range to determine the suitable quantization step. we therefore use this range for δ for our future computations for fp1 channel of subject 1. based on empirical experience and the trial and error, the pso optimization parameters were chosen to achieve the optimal robustness and transparency as follows c0 = 0.4, c1 = c2 = 1.8, number of particles = 30, number of generation = 50, and weighting factor γ = 50. 6.1 imperceptibility in our scheme, psnr was employed to evaluate the differences between original eeg signals and watermarked eeg signals. it should be noted that the larger psnr, the better imperceptibility. a larger psnr value indicates that the watermarked eeg signal more closely resembles its original signal, meaning that watermarked eeg signal has better imperceptibility. eeg channel quantization step (δ) psnr (in db) nc ber fp1 19.867 60.558 1 0 fc1 19.476 64.829 1 0 f3 19.573 59.118 1 0 p7 20.925 66.395 1 0 t7 19.978 70.275 1 0 average 19.964 64.235 1 0 table 1. performance metrics on average for different eeg signal channels of 32 subjects australasian journal of information systems duy, tran & ma 2017, vol 21, selected papers from ausdm ownership protection of outsourced data using watermarking 17 according to (chen et al. 1998), psnr above 40 db indicates a good perceptual fidelity. the psnr values (in db) of the watermarked eeg are shown in table 1, all of them are higher than 40 db, thus this indicates that diagnosability is not lost and degradation to the overall signal is acceptable. the comparable results in terms of psnr and bit error rate (ber) are listed in table 2. the results clearly show that the proposed method outperforms the other three methods. characteristics psnr (in db) ber proposed scheme scheme: dwt-svd with pso data type: eeg signal 64.235 0 pham's method (pham et al. 2015) scheme: dwt-svd data type: eeg signal 57.53 n/a ramu’s method (ramu et al. 2016) scheme: dwt-svd with continuous ant colony optimization (caco) data type: ecg signal 62.87 0 jero’s method (jero et al. 2014) scheme: dwt-svd data type: ecg signal 50.44 0 table 2. the performance comparison results for pnsr and ber figure 6 shows the difference between the original eeg signal and watermarked one. it can be seen that this difference is close to zero, it means the distortion to the original eeg due to the watermark is minimal. therefore, it shows that our watermarked eeg signal is near identical to the original eeg signal. figure 6: original eeg signal vs watermarked eeg signal (above), difference between original eeg signal and watermarked eeg signal (below) in channel fp1 of subject 01 6.2 robustness in order to evaluate the robustness of the proposed method against the common signal processing attacks, we used ber and nc measures. the following signal attacks were performed in matlab: australasian journal of information systems duy, tran & ma 2017, vol 21, selected papers from ausdm ownership protection of outsourced data using watermarking 18 1. noise addition: additive white gaussian noise (awgn) was added to the watermarked eeg signal with 20db. 2. low-pass filtering: the low-pass filter with cut-off frequency of 40hz was applied to all watermarked eeg signals. 3. re-sampling: the original eeg signals were sampled with a sampling rate of 128 hz. watermarked eeg signals were re-sampled at 64 hz and then restored by sampling again at 128 hz. in attack-free case, we extracted watermark from watermarked eeg signals using the proposed watermark extraction algorithm. table 1 shows ber = 0, nc = 1 in case of no attack, meaning that watermark can be accurately extracted from the watermarked eeg signal. the signal attacks were performed in matlab including noise addition, random cropping, low-pass filtering and re-sampling. eeg channel noise addition low-pass filtering re-sampling ber(%) nc ber(%) nc ber(%) nc fp1 0.32 0.9971 2.93 0.9762 3.71 0.9703 fc1 0.67 0.9947 1.41 0.9892 3.52 0.9715 f3 1.21 0.9861 3.51 0.9718 2.48 0.9802 p7 0.49 0.9961 3.42 0.9726 0.65 0.9947 t7 0.88 0.9929 2.72 0.9812 1.39 0.9869 average 0.71 0.9933 2.80 0.9782 2. 35 0.9807 table 3. performance metrics for different eeg signal channels under different attacks as seen in table 3, after applying attacks on watermarked eeg signals, it is observed that the values of ber are very low (less than 3%) while the values of nc is very high (close to one), which implies extracted watermark is very similar to the original watermark. figure 7(b-e) shows the extracted watermark after attacks. therefore, this indicates that the robustness of the proposed scheme is very good. in addition, ber < 3% could be corrected with the use of error correcting codes (wicker 1995). figure 7. result of watermark extraction at channel fp1 of subject 01. (a) original watermark. (b) free attack. (c) noise addition. (d) low-pass filtering. (e) re-sampling. 6.3 security for a secure watermarking scheme for biomedical data, security is a very important issue. to improve confidentiality, the key space should be large enough to boost confidentiality and render attacks, especially brute force attacks, are impossible. therefore, secret keys are adopted for security purposes, the proposed method utilizes chaotic encryption. the embedding and extraction processes in the proposed watermarking scheme depends on the secret key k, it is impossible to malicious attack to detect the watermark without this key. supposed attacker can have the information of the watermarked signal, watermarking algorithm, and the encryption function. however, attacker cannot generate the watermark australasian journal of information systems duy, tran & ma 2017, vol 21, selected papers from ausdm ownership protection of outsourced data using watermarking 19 without secret key. in addition, the proposed method possesses the high robustness which against attack is very important for a secured watermarking scheme. 6.4 error analysis to decide whether there is a watermark, the original watermark w is compared with the extracted watermark w'. if the ber between w and w' is less than a user-defined threshold t, there is a watermark, otherwise, there is no watermark. actually, the threshold t is determined by the probability of the detection error due to a false alarm or rejection detection (bhat et al., 2010). to determine the watermark threshold t, the false alarm and rejection are usually taken into consideration. the performance of a watermarking system is generally characterized by two types of errors (fan and wang 2009), the false-positive error and false-negative error. the false-positive error is the probability that an un-watermarked biomedical signal declared as watermarked by the decoder, while false-negative error is the probability that a watermarked biomedical signal declared as un-watermarked by the decoder. the probability of false-positive error pfp and probability of false-negative error pne can be computed as: 𝑃𝐹𝑃 = 2 −𝑚 ∑ ( 𝑚 ℎ ) 𝑚 ℎ=⌈𝜌𝑚⌉ (16) 𝑃𝐹𝑁 = ∑ [( ℎ 𝑚 )𝑃ℎ(1 − 𝑃)𝑚−ℎ] ⌈𝜌𝑚⌉−1 ℎ=0 (17) where (m h ) is the binomial coefficient, m is the total number of watermark bits, h is the total number of matching bits, and p is probability of the difference between extracted watermark and original watermark (w ≠ w'). according to (bhat et al. 2010), the desired false alarm error must be smaller than 10-6 order of magnitude. figure 8 and figure 9 show the probability of false-positive error pfp and the probability of false-negative error pfn, corresponding for watermark length m ∈ [0,100], which indicates that pfp and pfn approach 0 when watermark length m is larger than 20. in our method, m = 1024, hence the false positive probability and false negative probability are close to 0. indeed, we have h = ⌈(1-ber)×m⌉, therefore ber less than 20% meets this demand. if we set ber = 20%, then ρ = 0.8. in our method, m = 1024, eq. (16) gives pfp = 2.6209 x 10-88, hence the false positive is close to zero. figure 8. false positive error under different watermark length australasian journal of information systems duy, tran & ma 2017, vol 21, selected papers from ausdm ownership protection of outsourced data using watermarking 20 in eq. (17), the approximate value of p can be obtained from the ber under different attacks. as seen from table 1 and table 3, the average of ber is less than 3%, so p can be taken as 0.97. by substituting the values of m, p and p, eq. (18) gives pfn = 1.5286 x 10-102. figure 9. false negative error rate under different watermark length in summary, our experimental results show that the proposed blind watermarking scheme for biomedical data has good imperceptibility and strong robustness against several different attacks such as noise addition, low-pass filtering, and re-sampling. 7 conclusion and future work an ownership protection based on the proposed optimized watermarking scheme for biomedical data in data mining has been developed. the proposed watermarking scheme is based on dwt with the exploration of svd properties and dm quantization which make the scheme very robust to various common signal processing attacks. in our optimization process, pso was used to make an optimal trade-off between imperceptibility and robustness through effective selection of quantization steps to locate the best parameters to insert the watermarks. the quantization steps were optimally adapted to achieve the most suitable performance for various biomedical data with different characteristics for medical application. in addition, our watermark scheme possesses the characteristic of blind extraction which does not require the original biomedical signal in extraction, thus reducing a lot of space for storing the original biomedical data and watermark. the experimental results have revealed that the proposed watermarking scheme achieves good imperceptibility and strong robustness against common signal processing. our research can be used in e-healthcare application which need to share and transmit the biomedical data via network with ownership protection purpose. since information can be extracted exactly, health information such as patient's data can be embedded in biomedical signal, reducing the consequences of health information thefts, increasing the data security, and saving storage space and bandwidth requirement for transmission of biomedical data. it is obvious that our study is also preferable to facilitate data management in health information management systems. in the future work, we will consider the following problems: 1. enhancing privacy of biomedical data in data mining by tracing the source of an unauthorized release of biomedical data in networks. this is useful to monitor or trace back illegally produced copies of the data that may circulate. tracing the source will australasian journal of information systems duy, tran & ma 2017, vol 21, selected papers from ausdm ownership protection of outsourced data using watermarking 21 provide the information on who/where/how the biomedical data were accessed illegally. 2. implementing the error coding code in watermark extraction, reducing the error and enhancing the performance of watermarking scheme. 3. finding the most effective and suitable algorithm for optimal watermarking scheme. this work will save time and resources. it is necessary to investigate other evolutionary algorithms to enhance the performance with respect to the existing algorithms. 4. the proposed watermarking scheme for biomedical data is offline, it should be improved to carry out online scenarios. references alhaqbani, b., and fidge, c. 2008. "privacy-preserving electronic health record linkage using pseudonym identifiers," e-health networking, applications and services, 2008. healthcom 2008. 10th international conference on, pp. 108-117. arsalan, m., malik, s. a., and khan, a. 2012. "intelligent reversible watermarking in integer wavelet domain for medical images," journal of systems and software (85:4), pp. 883894. australia. law reform, c. 2002. principled regulation: report: federal civil & administrative penalties in australia. australian law reform commission. bender, w., gruhl, d., morimoto, n., and lu, a. 1996. "techniques for data hiding," ibm systems journal (35:3.4), pp. 313-336. bertino, e., ooi, b. c., yang, y., and deng, r. h. 2005. "privacy and ownership preserving of outsourced medical data," 21st international conference on data engineering (icde'05), pp. 521-532. bhat, v., sengupta, i., and das, a. 2010. "an adaptive audio watermarking based on the singular value decomposition in the wavelet domain," digital signal processing (20:6), pp. 1547-1558. bhatnagar, g., and wu, q. m. j. 2013. "biometrics inspired watermarking based on a fractional dual tree complex wavelet transform," future generation computer systems (29:1), pp. 182-195. centers for, m., and medicaid, s. 1996. "the health insurance portability and accountability act of 1996 (hipaa)," online at http://www.cms.hhs.gov/hipaa. chang, c.-c., tsai, p., and lin, c.-c. 2005. "svd-based digital image watermarking scheme," pattern recognition letters (26:10), pp. 1577-1586. chen, b., and wornell, g. w. 2001. "quantization index modulation methods for digital watermarking and information embedding of multimedia," journal of vlsi signal processing systems (27:1-2), pp. 7-33. chen, k., and ramabadran, t. v. 1994. "near-lossless compression of medical images through entropy-coded dpcm," ieee transactions on medical imaging (13:3), pp. 538548. chen, s.-t., guo, y.-j., huang, h.-n., kung, w.-m., tseng, k.-k., and tu, s.-y. 2014. "hiding patients confidential datainthe ecg signal viaa transform-domain quantization scheme," journal of medical systems (38:6), pp. 54-74. chen, t.-s., chang, c.-c., and hwang, m.-s. 1998. "a virtual image cryptosystem based upon vector quantization," image processing, ieee transactions on (7:10), pp. 1485-1488. chen, y., and xu, h. 2013. "privacy management in dynamic groups: understanding information privacy in medical practices," acm, pp. 541-552. australasian journal of information systems duy, tran & ma 2017, vol 21, selected papers from ausdm ownership protection of outsourced data using watermarking 22 clearinghouse, p. r. 2017. "chronology of data breaches: security breaches 2005–present," 2005, https://www.privacyrights.org/data-breach. cox, i., miller, m., bloom, j., fridrich, j., and kalker, t. 2007. digital watermarking and steganography. morgan kaufmann. dorigo, m., di caro, g., and gambardella, l. m. 1999. "ant algorithms for discrete optimization," artificial life (5:2), pp. 137-172. engin, m., cidam, o., and engin, e. z. 2005. "wavelet transformation based watermarking technique for human electrocardiogram (ecg)," journal of medical systems (29:6), pp. 589-594. fakhari, p., vahedi, e., and lucas, c. 2011. "protecting patient privacy from unauthorized release of medical images using a bio-inspired wavelet-based watermarking approach," digital signal processing (21:3), pp. 433-446. fan, m., and wang, h. 2009. "chaos-based discrete fractional sine transform domain audio watermarking scheme," computers and electrical engineering (35:3), pp. 506-516. gupta, a. k., and raval, m. s. 2012. "a robust and secure watermarking scheme based on singular values replacement," sadhana (37:4), pp. 425-440. hänsch, k., and serna, l. a. "directive 95/46/ec of the european parliament and of the council of 24 october 1995 on the protection of individuals with regard to the processing of personal data and on the free movement of such data, nov. 1995." hassan, r., cohanim, b., de weck, o., and venter, g. 2005. "a comparison of particle swarm optimization and the genetic algorithm," proceedings of the 1st aiaa multidisciplinary design optimization specialist conference, pp. 18-21. he, x., tseng, k.-k., huang, h.-n., chen, s.-t., tu, s.-y., zeng, f., and pan, j.-s. 2012. "wavelet-based quantization watermarking for ecg signals," computing, measurement, control and sensor network (cmcsn), 2012 international conference on, pp. 233-236. health, u. s. d. o., and human, s. 2009. "code of federal regulations. title 45," public welfare cfr (46). heylen, k., and dams, t. 2008. "an image watermarking tutorial tool using matlab," pp. 70750d-70751. holland, j. h. 1992. adaptation in natural and artificial systems: an introductory analysis with applications to biology, control, and artificial intelligence. mit press. hu, f., jiang, m., wagner, m., and dong, d.-c. 2007. "privacy-preserving telecardiology sensor networks: toward a low-cost portable wireless hardware/software codesign," ieee transactions on information technology in biomedicine (11:6), pp. 619-627. huang, c.-h., and wu, j.-l. 2009. "fidelity-guaranteed robustness enhancement of blinddetection watermarking schemes," information sciences (179:6), pp. 791-808. ibaida, a., khalil, i., and van schyndel, r. 2011. "a low complexity high capacity ecg signal watermark for wearable sensor-net health monitoring system," 2011 computing in cardiology, pp. 393-396. jahankhani, p., kodogiannis, v., and revett, k. 2006. "eeg signal classification using wavelet feature extraction and neural networks," modern computing, 2006. jva'06. ieee john vincent atanasoff 2006 international symposium on, pp. 120-124. jero, s. e., ramu, p., and ramakrishnan, s. 2014. "discrete wavelet transform and singular value decomposition based ecg steganography for secured patient information transmission," journal of medical systems (38:10), pp. 1-11. australasian journal of information systems duy, tran & ma 2017, vol 21, selected papers from ausdm ownership protection of outsourced data using watermarking 23 kamran, m., and farooq, m. 2012. "an information-preserving watermarking scheme for right protection of emr systems," ieee transactions on knowledge and data engineering (24:11), pp. 1950-1962. kaur, s., singhal, r., farooq, o., and ahuja, b. s. 2010. "digital watermarking of ecg data for secure wireless commuication," recent trends in information, telecommunication and computing (itc), 2010 international conference on, pp. 140-144. kennedy, j. 2011. "particle swarm optimization," in encyclopedia of machine learning. springer, pp. 760-766. kennedy, j. f., kennedy, j., eberhart, r. c., and shi, y. 2001. swarm intelligence. morgan kaufmann. ko, l.-t., chen, j.-e., shieh, y.-s., hsin, h.-c., and sung, t.-y. 2011. "nested quantization index modulation for reversible watermarking and its application to healthcare information management systems," computational and mathematical methods in medicine (2012). koelstra, s., mu, c., soleymani, m., lee, j.-s., yazdani, a., ebrahimi, t., pun, t., nijholt, a., and patras, i. 2012. "deap: a database for emotion analysis; using physiological signals," affective computing, ieee transactions on (3:1), pp. 18-31. kong, x., and feng, r. 2001. "watermarking medical signals for telemedicine," ieee transactions on information technology in biomedicine (5:3), pp. 195-201. kumsawat, p. 2010. "a genetic algorithm optimization technique for multiwavelet-based digital audio watermarking," eurasip journal on advances in signal processing (2010:1), p. 1. lai, c.-c. 2011. "a digital watermarking scheme based on singular value decomposition and tiny genetic algorithm," digital signal processing (21:4), pp. 522-527. latifpour, h., mosleh, m., and kheyrandish, m. 2015. "an intelligent audio watermarking based on knn learning algorithm," international journal of speech technology (18:4), pp. 697-706. lee, w.-b., and lee, c.-d. 2008. "a cryptographic key management solution for hipaa privacy/security regulations," ieee transactions on information technology in biomedicine (12:1), pp. 34-41. lei, b., song, i., and rahman, s. a. 2013. "robust and secure watermarking scheme for breath sound," journal of systems and software (86:6), pp. 1638-1649. li, z.-r., chang, e.-c., huang, k.-h., and lai, f. 2011. "a secure electronic medical record sharing mechanism in the cloud computing platform," pp. 98-103. loukhaoukha, k., chouinard, j.-y., and taieb, m. h. 2011. "optimal image watermarking algorithm based on lwt-svd via multi-objective ant colony optimization," journal of information hiding and multimedia signal processing (2-6:4), pp. 303-319. maglogiannis, i., kazatzopoulos, l., delakouridis, k., and hadjiefthymiades, s. 2009. "enabling location privacy and medical data encryption in patient telemonitoring systems," ieee transactions on information technology in biomedicine (13:6), pp. 946-954. malasri, k., and wang, l. 2007. "addressing security in medical sensor networks," acm, pp. 7-12. mishra, a., agarwal, c., sharma, a., and bedi, p. 2014. "optimized gray-scale image watermarking using dwt–svd and firefly algorithm," expert systems with applications (41:17), pp. 7858-7867. australasian journal of information systems duy, tran & ma 2017, vol 21, selected papers from ausdm ownership protection of outsourced data using watermarking 24 mousavi, s. m., naghsh, a., and abu-bakar, s. a. r. 2014. "watermarking techniques used in medical images: a survey," journal of digital imaging (27:6), pp. 714-729. orhan, u., hekim, m., and ozer, m. 2011. "eeg signals classification using the k-means clustering and a multilayer perceptron neural network model," expert systems with applications (38:10), pp. 13475-13481. percival, d. b., and walden, a. t. 2006. wavelet methods for time series analysis. cambridge university press. pham, t. d., tran, d., and ma, w. 2015. "a proposed blind dwt-svd watermarking scheme for eeg data," international conference on neural information processing, pp. 69-76. planitz, b., and maeder, a. 2005. "medical image watermarking: a study on image degradation," proc. australian pattern recognition society workshop on digital image computing, wdic. prior, f., ingeholm, m. l., levine, b. a., and tarbox, l. 2009. "potential impact of hitech security regulations on medical imaging," ieee, pp. 2157-2160 ramu, p., swaminathan, r., and others. 2016. "imperceptibility—robustness tradeoff studies for ecg steganography using continuous ant colony optimization," expert systems with applications (49), pp. 123-135. run, r.-s., horng, s.-j., lai, j.-l., kao, t.-w., and chen, r.-j. 2012. "an improved svd-based watermarking technique for copyright protection," expert systems with applications (39:1), pp. 673-689. ruotsalainen, p. 2010. "privacy and security in teleradiology," european journal of radiology (73:1), pp. 31-35. smith, r., and gotel, o. 2007. "using a game to introduce lightweight requirements engineering," ieee, pp. 379-380. solutions, v. e. 2015. "data breach investigations report," verizon, report. sriyingyong, n., and attakitmongcol, k. 2006. "wavelet-based audio watermarking using adaptive tabu search," 2006 1st international symposium on wireless pervasive computing, pp. 1-5. storn, r., and price, k. 1997. "differential evolution–a simple and efficient heuristic for global optimization over continuous spaces," journal of global optimization (11:4), pp. 341-359. tefas, a., nikolaidis, n., and pitas, i. 2009. "image watermarking-chapter 22: techniques and applications". trelea, i. c. 2003. "the particle swarm optimization algorithm: convergence analysis and parameter selection," information processing letters (85:6), pp. 317-325. tsai, h.-h., jhuang, y.-j., and lai, y.-s. 2012. "an svd-based image watermarking in wavelet domain using svr and pso," applied soft computing (12:8), pp. 2442-2453. van schyndel, r. g., tirkel, a. z., and osborne, c. f. 1994. "a digital watermark," ieee, pp. 86-90. voyatzis, g., and pitas, i. 1999. "the use of watermarks in the protection of digital multimedia products," proceedings of the ieee (87:7), pp. 1197-1207. wang, h., peng, d., wang, w., sharif, h., chen, h.-h., and khoynezhad, a. 2010. "resourceaware secure ecg healthcare monitoring through body sensor networks," ieee wireless communications (17:1 ), pp. 1536-1284. wang, j., peng, h., and shi, p. 2011. "an optimal image watermarking approach based on a multi-objective genetic algorithm," information sciences (181:24), pp. 5501-5514. australasian journal of information systems duy, tran & ma 2017, vol 21, selected papers from ausdm ownership protection of outsourced data using watermarking 25 westin, k. 2015. "encryption would’t have stopped anthem’s data breach." mit technology review, febraury. wicker, s. b. 1995. error control systems for digital communication and storage. prentice hall englewood cliffs. zain, j. m., and clarke, m. 2011. "reversible region of non-interest (roni) watermarking for authentication of dicom images," arxiv preprint arxiv:1101.1603. zheng, d., liu, y., zhao, j., and saddik, a. e. 2007. "a survey of rst invariant image watermarking algorithms," acm computing surveys (csur) (39:2), p. 5. copyright: © 2017 duy, tran & ma. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ ajis vol. 6 no. 1 september 1998 reengineering in australia: factors affecting success felicity murphy simsion bowles and associates melbourne, victoria e-mail: fmurphy@sba.com.au sandy staples department of information systems the university of melbourne parkville, victoria e-mail: s.staples@dis.unimelb.edu.au abstract business process reengineering (bpr) is being used in many organisations worldwide to realign operations. most of the research undertaken has been focused on north american or european practices. the study reported here replicates a us reengineering study in an australian context by surveying large public and private sector australian organisations. the study makes three main contributions by: (1) presenting a picture of bpr practices in australia, (2) clarifying factors critical to the success of reengineering projects in australia, and (3) providing a comparison of factors leading to success in australian bpr projects with those found in the us. keywords business process reengineering (dd0402) introduction it is clear when reading both academic and practitioner literature that business process reengineering (bpr) is an important phenomenon which has grown rapidly since the early 1990's. a survey of 500 cio's conducted by the accounting firm, deloitte and touche, found that the average cio was involved in 4.4 reengineering projects in 1993 up from 1.6 projects in 1992 (moad 1993). brancheau, janz and wetherbe (1996) found that top managers rated "facilitating and managing business process redesign" as the second most important issue they faced in 1994/5. although bpr has been recognised as an important phenomenon it has not always been an unqualified success. hammer and champy (1994) stated that between 50 and 70 percent of reengineering projects do not achieve the goals set out for them. butler (1997), however, suggested that failure statistics quoted in the literature have decreased with time, as experience and education about process redesign has grown. in the eight years since reengineering emerged, there has been some research done on the critical success factors but most of this work focused on north american or european practices (e.g., grover, jeong, kettinger & teng 1995, clemens, thatcher & row 1995, hall, rosenthal & wade 1993, miles, coleman & creed 1995). in the australian context, aside from the work done by weill, broadbent and butler, (broadbent & weill 1995, broadbent et al. 1996, butler 1993 & 1997), there have been very few studies examining the impact of bpr. this suggests that the understanding of australian bpr practices is limited and research is needed. with this in mind we undertook a study in australia that examined reengineering practices and critical success factors. our study had two main objectives. first, we wanted to examine the impact of factors that the literature suggested could influence the success of reengineering practice in australia. second, we wanted to compare our findings with reengineering knowledge in the united states (us). therefore, our two research questions were: 1. what are the critical success factors for bpr implementation in an australian context? 2. how do the critical success factors for bpr implementation in australia compare to us factors? the severity of common problems associated with reengineering and the criticality of their impact on project success was assessed to answer the first question. to address the second question, we replicated grover, jeong, kettinger, and teng's (1995) us-based research. grover et al. (1995) empirically examined the importance of 64 possible critical success factors as viewed by practitioners in 105 american organisations. the impact of the factors on perceived success was then tested. we did the same thing using the views of australian reengineering practitioners. 59 ajis vol. 6 no. 1 september 1998 after reviewing various definitions of business process reengineering, we found that grover et al.'s (1995) definition succinctly captured the key points of other authors and was therefore adopted for this study. thus, the definition of bpr used in this study was: a deliberate (planned) change, typically enabled by information technologies in an attempt to redesign a business process to achieve performance breakthroughs in measures such as quality, speed, customer service, and cost. the remainder of this paper is organised as follows. the next section summarises the literature about potential reengineering success factors. the third section describes the methodology used in our study. the fourth section presents the results. in the final section, the results are discussed and implications for practice and research are offered. a review of factors that potentially impact reengineering success a review of the literature was conducted to identify factors that writers and researchers suggested could potentially affect reengineering success. the results of this review, summarized below, are organised into six areas: management support, technological competence, process delineation, project planning, change management, and project management. management support: management support and leadership in reengineering efforts has been identified in much of the literature as perhaps the most significant contributing factor to the success of an initiative. top management support is necessary to develop faith throughout the organisation in the project and a respect for the project team (hall et al. 1993, hammer & stanton 1995, hoopes 1995). grover et al. (1995) identified a group of implementation problems regarding management support. the results of their survey found that while management support problems were "severe" (grover et al. 1995) (interpreted as difficult to address), they did not correlate highly with the overall measure of success in comparison to other implementation problems. they concluded that the resolution of these problems was a necessary but not sufficient condition for project success. technological competence: information technology is referenced in much of the literature as the enabler of the change to a process orientation. grint, case and willcocks (1995) conducted a survey and found that 58 percent of respondents rated it's role in enabling radical process redesign as critical, 68 percent rated it's support role in redesign as critical, and it management was rated as one of the top ten critical success factors for bpr projects. it can be the enabler of change but it can also be a potential constraint as most firms have existing it infrastructures built up over many years (hammer 1990, taylor 1996). the state of the it infrastructure can influence the choice of process to reengineer and the choice of either a radical or an incremental implementation plan (broadbent & weill 1995). the human component of technological competence, in terms of the level of it knowledge, experience and expertise within the organisation, also needs to be addressed when implementing change (martinez 1995). the results of the grover et al. (1995) study suggests that taking good care of the required technology may be necessary but not sufficient for the eventual success of reengineering projects. process delineation: process delineation entails defining the scope of the process, identifying process owners, identifying the implementation strategy to be used and establishing goals for the redesigned process (grover et al. 1995). redesign projects often aim at processes that are "too narrowly defined and therefore have little discernible impact on overall performance" (hall et al. 1993:121). the scope of the process chosen for redesign must be broadly defined in terms of cost improvements or increases in customer value for it to improve performance across the organisation. redesign projects can also be defined too radically with changes being too wide for organisations to cope with (davenport 1993, martinez 1995). process delineation should be undertaken carefully after having assessed the organisations resources and it should involve the identification of performance improvement goals for the redesigned process (demons 1995, miles et al. 1995). grover et al. (1995) argued that while process delineation problems may not be difficult to address they have a significant impact on project outcomes. project planning: project planning as defined by grover et al. (1995) consists of strategic and tactical planning. strategic planning concerns the organisational vision and strategy for the future whereas tactical planning concerns the plan for the implementation of the single change project being undertaken. the project initiative should "be driven by corporate strategy and supported at the highest levels of the organisation. ultimately it must make daily operations themselves the execution of strategy." (manganelli & raspa 1995). tactical project planning focuses largely on the reengineering team and the resources and authority that are placed at its disposal. the quality of resources and the amount of authority given to the project team indicate to the rest of the organisation the level of importance and management support the project is receiving. the resources 60 ajis vol. 6 no. 1 september 1998 that reengineering teams require include human (i.e., having the most highly skilled and knowledgeable people); financial (e.g., money for new it or other expenses); training for team members; and, in some cases, a consultant to provide expert advice (hammer & champy 1994, hammer & stanton 1995, keen 1995, miles et al. 1995). change management: manganelli and raspa (1995:42) note that "because change can be intimidating a fair amount of anxiety usually surfaces when a reengineering project is announced." in the grover et al. (1995) study, change management issues emerged as the set of problems with the most significant negative relationship with project outcome. one of the methods suggested for mitigating the resistance to change is the communication of the vision behind the impending change. it is suggested in much of the literature (davenport 1993, grint et al. 1996, hall et al. 1993) that the reasons and expectations for the change should be communicated organisation-wide as a way of motivating people and enticing them to commit themselves to the project. stoddard and jarvenpaa (1995:104) comment, "you always underestimate the amount of communication you have to do. always triple what you think it takes." involving the entire organisation in the change process is a good way to overcome lethargy and resistance within the organisation (romney 1996). human resource issues also need consideration when an organisation undergoes change (davenport 1993, hall et al. 1993, hammer & champy 1994, romney 1994). incentive and reward structures, new skills training, and other human resource policies may require alteration in response to the organisational restructuring. it is important that this is addressed if management wishes to cultivate the required values towards the reengineering project. project management: project management refers to the level of skills and experience senior management and reengineering teams have in managing large-scale projects. managing projects entails assessing performance throughout the project life span, assigning team members responsibility to the projects and managing the communication both within the project team and between the team and the rest of the organisation. poor project management can have a significantly negative impact on the success of reengineering projects (hammer & stanton 1995, martinez 1995). the grover et al. (1995) study found that although project management problems were not difficult to rectify they were highly influential on the outcome of the project. the lack of a recognised or appropriate bpr methodology to guide managers in implementation is also a problem that has been identified widely in the bpr literature (davenport 1993, grint et al. 1996, grover et al. 1995). the results of this review suggest that there are many important factors to consider when implementing a bpr program. these projects are difficult to manage and success is difficult to attain. the next section discusses the methodology we used to examine potential success factors in australian reengineering practice. methodology a quantitative research design was chosen to examine the importance of the success factors and their relationship to perceived success. a questionnaire was used in order to facilitate collection of information from a large and geographically disperse sample. this section describes the measures used, sampling method, and analysis methods employed. measurement after reviewing the bpr literature, it was found that the list of 64 potential bpr implementation problems presented by grover et al. (1995) covered the area concisely; thus, this list of factors was adopted for our study. grover et al. (1995) reviewed literature in the areas of organisational change, innovation, organisational development and socio-technical design, as well as the bpr literature to identify these factors. the 64 items are listed in table 1. grover et al. (1995) suggested that the 64 questions could be collapsed into six constructs: management support, technological competence, process delineation, project planning, change management, and project management. the management support construct measured the level of support, understanding, and leadership which senior management exhibited towards the reengineering project and was measured using 6 items. the technological competence construct assessed the existence and use of the organisations' it infrastructure, in terms of both hardware, software, and human components, which supported or enabled the reengineering effort. this construct was measured using 8 items. the process delineation construct measured the depth and breadth of the process chosen and the goals established for reengineering, and was measured using 8 items. the construct of project planning was addressed using 13 items which tapped the conduct of the project and the integration of the project with plans or strategies for the organisation as a whole. the change management construct measured the prevalence of resistance to change, the implementation of management systems to garner support for the reengineering effort, and the attitudes of groups within the organisation to the reengineering project (16 items). the construct of project management was measured using 13 items assessing the management of the project in terms of the human 61 ajis vol. 6 no. 1 september 1998 resources, the skill set required to implement reengineering, and the implementation of project performance measurement structures. the dependent variable adopted for our study and used in the analysis of the grover et al. (1995) items was perceived project success. perceived success, measured by one five-point scale question, is according to delone and mclean (1992) the most widely used measure of mis success. sample a nation-wide mail survey of australian organisations was conducted. the questionnaire was distributed to senior managers of the 1000 top australian companies (chosen on the basis of revenue) in the private or public sectors via a mailing list purchased from drake list management services. the questionnaire package contained a cover letter addressed to the senior manager asking him/her to pass the enclosed questionnaire on to a project leader of a recent reengineering project. both the senior manager and the project leader were offered a report of the findings of the study in an attempt to motivate them to respond. to increase the response rate, a reminder letter was sent to senior managers of non-responding organisations about three weeks after mailing the questionnaire package. the grover et al. (1995) study consisted of project specific questions and as such required the respondent to focus on and answer in the context of a single chosen project. to be consistent with the grover et al. (1995) study, our questionnaire was directed to a project leader. the unit of analysis in this survey was therefore, a single process redesign project. this was consistent with our objectives of examining factors that lead to project success and was consistent with the unit of analysis used by grover et al. (1995). of the 1000 surveys sent, 196 people indicated that they had not undertaken reengineering (an option we provided in the covering letter) and 104 returned a completed questionnaire. therefore, our response rate was 30.0 percent. the potential for non-response bias was addressed by profiling both early and late responders and comparing the two groups according to techniques described by fowler (1993). this analysis yielded no significant differences between the groups and supports the position that non-response bias did not appear to be a problem. on average, respondents had been employed with the current organisation for 10.15 years and reported having an average of 5.19 years and 4.5 projects worth of experience with reengineering. reengineering training had been received by only 31 percent of the project leaders. the average project length was reported as approximately 1 year and 9 months. the business areas that were the focus of reengineering consisted of a single business division in 29.7 percent of cases, two or more divisions in 32.7 percent of cases, and organisation-wide in 37.6 percent of cases. consultants were used in 63 percent of the reported projects and the consultant input into the reengineering exercise was generally regarded as valuable. the reengineering projects were considered reasonably successful. analysis the 64 potential problem items adopted from the grover et al. (1995) study were analysed using the same techniques as the grover study, to facilitate comparison. respondents rated the 64 problem items on a 5-point likert scale in terms of the perceived relative severity of the problem in their last reengineering project. the severity score for each problem was calculated as the percentage of respondents who rated it as either a major problem or an extreme problem (i.e. 4 or 5 on the 5point scale). the responses were re-coded to reflect a 0 for answers of 1,2, or 3 (i.e. not a problem) and a 1 for answers of 4 or 5 (i.e. a problem); this was the items severity score. the problems were then ranked in terms of their average severity scores. for example, the severity score for the first item in table 1 indicates that approximately 44% of our respondents had difficulty in finding skilled team members during their last bpr project. reliability of the items that addressed each construct was assessed via cronbach's alpha. the items for each construct were then summed to create a mean severity score. not all items were used to create the construct severity scores. to be comparable to grover et al.'s (1995) work, we chose to use the same items to measure the constructs (see table 2). these items were identified in grover et al.'s study via factor analysis. principal component analysis with varimax rotation was used in our study to test the validity of the constructs and confirm that the choice of using the grover et al. items was reasonable. the severity scores were then correlated with perceived project success in order to assess the impact of the factors on project success, thus answering our first research question. to address our second research question, the mean severity scores that we obtained were compared to the results found by grover et al. (1995) in their study of us reengineering practice. 62 ajis vol. 6 no. 1 september 1998 results table 1 summarises the rankings obtained from the respondents in our sample and compares the severity scores and rankings of each of the problem items with the scores and rankings found in the us study (grover et al. 1995). the severity scores for the australian replication were much higher than those found in the grover et al. (1995) study (i.e. the highest severity score in the us being 31.8 while the highest australian score was 44). while the items did not appear to be ranked similarly at first glance they were in fact significantly correlated with the grover et al. (1995) scores (pearson correlation coefficient = 0.604, p<.001). factor analysis was done to assess the validity of the construct measures and compare the results with the grover et al. (1995) study. we found some differences but these were largely for constructs that had numerous items. for the factors with fewer items (i.e. technological competence, management support) the results were similar to the grover et al. (1995) study. the relatively small sample size in our study made the stability of our factor analysis results questionable. due to the general similarities in factors between the grover et al. (1995) study and our results and in the interests of making the replication as accurate and comparable as possible we chose to use the items selected by grover et al. (1995) to measure the constructs for the balance of the analysis. the grover et al. (1995) factor analysis stabilised at nine factors which we used in our analysis (see table 2). three constructs were found to be multi-dimensional. the project planning items broke into two factors: strategic planning and tactical planning. the change management items also broke into two factors: human resources and change management. a second factor, dealing with time frame problems, also broke out of the list of project management items. the cronbach's alpha values indicate that the constructs have a reasonable degree of internal consistency supporting the validity of the nine factors. the average severity scores for the problem categories/constructs were calculated by averaging the percentage severity scores (see table 1) for the items chosen to represent that construct. the resulting mean severity score for the problem category (i.e., construct) was compared to the grover et al. (1995) score (table 2). the rankings of the australian problem category severity scores were not significantly correlated with the us severity scores (spearman's rho = 0.300, p = 0.433). the average severity scores from our study for each problem category were correlated with the perceived success measure (see the last column of table 2). all of the factors except technological competence were significantly correlated with perceived project success. the next section discusses the findings. 63 ajis vol. 6 no. 1 september 1998 table 1: severity rankings for the 64 problem items 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 question difficulty in finding business reengineering team members who have required skills and knowledge difficulty in gaining cross-functional co-operation top management's short-term view and quick-fix mentality the business reengineering effort takes too much time unreasonable expectations attributed to business process reengineering as a solution for all organisational problems line managers in the organisation unreceptive to innovation necessary changes in the human resource policies for business reengineering implementation were not made top management's insufficient understanding about business reengineering absence of management systems (e.g. incentive, training system) to cultivate required values lack of appropriate employee compensation incentives in the new process limited is application infrastructure failure to commit the required resources (financial, human resources, etc.) to business reengineering efforts lack of senior management leadership for reengineering efforts difficulty in measuring reengineering project performance uncertainty about the reengineering project's time-frame failure to anticipate and plan for organisational resistance to change failure to build support from line managers inadequate training for personnel affected by the redesigned process lack of alignment between corporate planning and ft planning limited database infrastructure not enough time to develop new skills for the redesigned process difficult to forecast human resources, financial, and other resource requirements ambiguity in job expectations for reengineering team members rigid hierarchical structures senior management's failure to commit to new values need for managing change is not recognised difficulty in financially justifying benefits of business reengineering failure to assess project performance in the early stages of business reengineering efforts to provide feedback manager's failure to support the new values and beliefs demanded by the redesigned processes insufficient understanding about the goals of top management in relation to business reengineering insufficient understanding about existing data, applications, and category1 pp cm pp pm cm cm cm ms cm cm tc pp ms pm pm cm cm cm pp tc cm pd pm cm cm cm pp pm ms ms tc australia severity 44 41 41 38 38 37 37 35 34 34 34 33 32 32 30 29 29 28 28 27 27 26 26 26 26 26 25 25 24 24 23 us severity 12.5 20.1 31.7 23.1 22.1 28.8 15.9 15.4 20.6 16.8 18.6 17.6 16.3 21.1 22.1 27.7 23 16.9 23.3 22.2 7 13.6 12.5 30.1 18.5 31.8 13.6 8.9 21.7 18.2 25.3 aust. rank 1 2 2 4 4 6 6 8 9 9 9 12 13 13 15 16 16 18 18 20 20 22 22 22 22 22 27 27 29 29 31 us rank 45 20 2 9 12 4 34 36 18 26 22 25 29 17 12 5 10 33 7 11 60 39 45 3 23 1 39 54 16 24 6 64 ajis vol. 6 no. 1 september 1998 it across the organisation pd process delineation; pm project management; pp project planning; cm change management; tc technological competence; ms management support table 1 continued: severity rankings for the 64 problem items 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 question difficulty in modelling and simulating the proposed change to the business process lack of experience in business reengineering failure to aggressively use it enablers lack of top management support in business reengineering efforts failure to consider the politics of the business reengineering efforts lack of business reengineering project champion focusing only on evaluation criteria that are easily measurable and quantifiable failure to consider existing organisational culture lack of is participation and assistance in the reengineering project failure to continually assess emerging it capabilities reengineering team members' conflict between the team responsibilities and functional responsibilities lack of authority given to reengineering team lack of expertise in it in your organisation failure to communicate reasons for change to members of the organisation lack of strategic vision difficulty in gaining control of reengineering efforts difficulty in establishing performance-improvement goals for the redesigned process failure to effectively monitor progress of project according to the schedule lack of appropriate bpr methodology failure to identify process owners who are responsible for entire business processes absence of appropriate training for bpr team members proposed changes to the process were too incremental, not radical enough poor communication between reengineering team members and other organisational members too much emphasis on analysing the existing process limited telecommunication infrastructure failure to understand customers' viewpoints in the business reengineering efforts lack of appropriate planning failure to include process owners throughout the business reengineering effort lack of external-consultant support for business reengineering efforts category pm pp tc ms cm ms pd cm tc tc pm pp tc cm pp pm pd pm pm pd pp pd pm pm tc pp pp pd pp australia severity 23 23 23 23 22 22 21 20 20 20 20 20 20 19 19 19 18 18 17 16 15 15 15 15 13 12 12 11 10 us severity 8.9 20.2 16.6 13.5 23.3 8.7 12.9 15.4 13.5 14.5 16.4 16.3 19.5 16.3 22.1 9.7 22.1 11.6 5.8 16.3 8.7 10.6 15.6 8.6 11.6 7.8 10.6 10.7 3.9 aust. rank 31 31 31 31 36 36 38 39 39 39 39 39 39 45 45 45 48 48 50 51 52 52 52 52 56 57 57 59 60 us rank 54 19 27 41 7 56 43 36 41 38 28 29 21 29 12 53 12 47 61 29 56 50 35 58 47 59 50 49 66 65 ajis vol. 6 no. 1 september 1998 61 62 63 64 scope of reengineered process was defined inappropriately identification of candidate process for reengineering not based on strategic planning poor communication among reengineering team members approach to reengineering was too radical pd pp pm pd 10 10 8 7 10.6 12.6 4.8 2.9 60 60 63 64 50 44 62 64 1 pd process delineation; pm project management; pp project planning; cm change management; tc technological competence; ms management support table 2: severity and ranking of reengineering problems in australia versus us construct label human resourc management su change manage time frame technological competence project manage] tactical plannin strategic plannii process delinea items from ' used to m( constn :s 9,18,: >lspii3( 29, 30 n2nf7, 16, 25, 45 4, 15, 32 11,40,31, 44, 5< ie44, 15, 28, 54 ; 12,27,52, ig 19,46,5! iofl2,48,51, 61 'auerdnbach's asure ct 1 .71 35, 37 .84 26, 36, .83 49 .67 >0, 34, .85 h, 50, .77 57, 60 .68 , 62 .69 >3, 59, .76 alptustraliar 1 2 3 4 5 6 7 8 9 r&nktralian s score f constru 31.0 27.3 27.1 24.8 22.9 21.2 18.4 17.8 17.0 evhfijrank (' ar et al. 19 :ts 6 5 1 4 2 8 9 3 7 jrk&fieverit 95) 14.8 15.6 21.9 16.4 18.2 11.7 10.3 17.2 14.0 ' scbcbtelatioi perceived si .264" .316* .366* .418* .202 .392* .375* .307* .429* with iccess pearson correlation coefficient. 2-tailed test. * significant at 0.05 level. ** significant at 0.01 level. perceived project success was measured one a 5 point scale ranging from 1 = success to 5 = failure. discussion the first research question we sought to address was what factors appear to significantly impact the success of australian reengineering projects. eight of the nine constructs were significantly associated with success. the only construct not significantly related to success was technological competence (although about 23 percent of projects had problems in this category). consistent with this, only two technology-related problems were in the top twenty list (table 1). these findings reinforce the notion of a shift in focus away from an it-centric approach to a more holistic approach to restructuring. it appears to be an important enabler, not a driver of organisational change and restructuring. the two change management constructs ranked number one and three in terms of being a problem for australian bpr project leaders and both were significantly associated with success. this implies that change management issues are not only common problems in bpr projects (i.e., present in roughly 30 percent of projects), but they are important problems to resolve since if they are not resolved, the likelihood of success declines. consistent with this, six of the top ten problem items in table 1 were change management issues. establishing cross-function cooperation, setting reasonable expectations for the project, and aligning incentive and reward systems with new processes and directions were all found to be common problems. these are clearly important problems for management to focus on and solve. the management support construct ranked second in terms of problem severity (i.e., occurred in about 27 percent of projects) and was significantly correlated with perceived success. this implies that a lack of management support remains a severe problem and it has a significant negative impact on success. while it has been suggested that the australian bpr push has come almost entirely from general management (simsion, 1997), the results of our study indicate that there may still be a lack of high level management support for reengineering. perhaps if the initial push comes from management, their support and leadership does not last long enough. respondents reported that the average time taken for a project was 1.75 years. management must have a realistic time frame in mind when initiating these types of projects and be willing to provide support and energy for the life of the project. the fact that the fourth most severe problem construct dealt with issues of timing, such as the project taking too long and uncertainty about the project's time frame, also suggests that managing the timing of the project and setting realistic expectations are common problems that are critical for project success. 66 ajis vol. 6 no. 1 september 1998 planning issues, both tactical and strategic, were experienced by about one-fifth of the project managers in our study, as were problems dealing with defining the process. while not as common as the problems described above, these groups of problems were still found to be significantly correlated with success so managers should not ignore these issues either. comparing the findings of the us-based grover et al. (1995) study with our australian results addresses the second research question. marked differences were found between the rankings of severity found by grover et al. (1995) in the us and the rankings that emerged in the australian study. some of the factors that were considered the most severe in the us study (for example technological competence and strategic planning, ranking 2 and 3 respectively) were not found to be as severe problems in the australian context (ranking 5 and 8 respectively). however, just looking at the ranking can be somewhat misleading. the severity scores indicate the percentage of respondents who found problems in those areas. for example, strategic planning was a problem in about 18 percent of australian projects (ranked 8th) and it was a problem for about 17 percent of us projects (ranked 3rd). in terms of frequency, they are quite similar. the most striking difference in our view was that the results suggest that australian bpr projects have about twice as many human resource and management support problems than do us projects. possible explanations for this such as the us projects having more resources and support come to mind. overall, the differences between the rankings from the us and the australian studies, reported here, were greater than we expected. it is generally recognised that cultural and economic differences between australia and the us are not great; hence, the differences in the results surprised us. future research to investigate why these differences exist would be useful, and would perhaps find lessons from us practice that australian practice could benefit from. although we tried to replicate the us study closely, the differences could also be potentially caused by two methodological factors. the first factor which could contribute to the differences in results is the time difference between the two studies. the grover et al. (1995) paper was published in the journal of management information systems in 1995, which implies that the data set was probably collected in 1993. much has been written and learnt about bpr in the four years since the grover et al. (1995) data was collected. consultant input in reengineering projects has also probably increased and the knowledge and experience base about best practices for reengineering in industry has grown. respondents may now have a better grasp and awareness of problems that were previously regarded as severe. differences in the samples may be a second factor contributing to the variation. the respondents for the grover et al. (1995) study were members of "the planning forum", which is the international business organisation focusing on strategic management and planning. planning forum members were selected because of their high interest and involvement in bpr projects (grover et al. 1995). this may have contributed to differences in the results as these respondents are trained to think in terms of strategy, and possibly have more experience and knowledge about programs of corporate restructuring and change management issues. the respondents in our study were project leaders who may have had a different perspective. as with all research, our work has a number of limitations. in our questionnaire, respondents were required to cast their minds back in order to answer many of the questions. the ability of the respondents to recall past events could potentially lead to errors. our study also relied on respondent perceptions. using a survey and perceptual measures is problematic because it introduces the potential for bias; however, for this research a mail survey was the most effective way to reach a large number of geographically dispersed respondents. we see several potentially important contributions of this study. the research extends the knowledge base on critical success factors for business process reengineering in australia. one purpose of this study was to replicate the grover et al. (1995) instrument and compare the conclusions drawn in the two studies. this comparison has highlighted several differences between the nature of reengineering in australia and the relationship between reengineering implementation problem severity and the impact of problems on success. the grover et al. (1995) instrument was also replicated with a view to validating it for use in future investigations of critical success factors for reengineering implementation. the factor analysis undertaken in the replication lends support to the validity of the instrument, although we would like to see the grover et al. (1995) instrument tested with larger samples in future research. ideally, all 64 items should be factor analysed together to test discriminant and convergent validity; we were unable to do this in our study due to sample size limitations. future research such as this would help verify how to collapse the list of 64 items into valid constructs. in conclusion, we found that reengineering was a practice being undertaken by a significant portion of australian organisations. reaping maximum gains from bpr projects is challenging, as shown by the prevalence and variety of problems identified in this study. we hope this study provides advice to practitioners on the factors to pay attention to in order to maximise the chance of a successful bpr implementation and hope it provides a base for future research on australian reengineering practice. 67 ajis vol. 6 no. 1 september 1998 acknowledgements the authors gratefully acknowledge the insightful advice and suggestions from peter seddon of the university of melbourne and from the anonymous reviewer. we would also like to thank ravi patnayakuni of the university of melbourne, for his financial support and encouragement of this work. references brancheau, j. c, janz, b. d. and wetherbe, j. c. (1996) key issues in information systems management 1994-95 sim delphi results. mis quarterly, 20:2, 225-242. broadbent, m. and weill, p. (1995). busting up the business: different approaches to business process redesign and it infrastructure investments. mis (australia), 42-48. broadbent, m., weill, p. and st. clair, d. (1996). the implications of it infrastructure for business process redesign. melbourne business school, the university of melbourne: melbourne. butler, c. (1993). the role of information technology in business process redesign: observations from the literature. melbourne business school, university of melbourne: melbourne. butler, c. (1997). business process redesign: the australian experience. melbourne business school, university of melbourne: melbourne, demons, e. k. (1995) using scenario analysis to manage the strategic risks of reengineering. sloan management review, 36:4, 61-71. demons, e. k., thatcher, m. e. and row, m. c. (1995) identifying sources of reengineering failures a study of the behavioral factors contributing to reengineering risks. journal of management information systems, 12:2, 9-36. de lone, w. and mclean, e. r. (1992) information systems success: the quest for the dependent variable. information systems research, 3:1, 60-95. fowler, f. j. j. (1993) survey research methods. sage publications: newbury park. grint, k., case, p. and willcocks, l. (1996) business process reengineering reappraised: the politics and technology of forgetting. in orlikowski, w. j., (ed.) information technology and changes in organizational work: proceedings of the ifip wg8.2 working conference on information technology and changes in organizational work, december 1995,. chapman & hall on behalf of the international federation for information processing (ifip); london. grover, v., jeong, s. r., kettinger, w. j. and teng, j. t. c. (1995) the implementation of business process reengineering. journal of management information systems, 12:1,109-144. hall, g., rosenthal, j. and wade, j. (1993) how to make reengineering really work. harvard business review, 71:6, 119131. hammer, m. (1990) reengineering work: don't automate obliterate. harvard business review, 68:4,104-112. hammer, m. and stanton, s. a. (1994) the reengineering revolution: a handbook. harper business: new york. hammer, m. and champy, j. (1993) reengineering the corporation : a manifesto for business revolution. nicholas brealey publishing, allen & unwin: london: st. leonards, n.s.w. hoopes, j. (1995) western civilisation versus the flat organisation. in burke, g. and peppard, j., (eds.); examining business process re-engineering: current perspectives and research directions, kogan page, london. keen, m. (1995) bpr: managing the change process or the process of change managing bpr? in burke, g. and peppard, j., (eds.); examining business process re-engineering: current perspectives and research directions, kogan page, london. manganelli, r. l. and raspa, s. p. (1995) why reengineering has failed. management review, 84:7, 39-43. martinez, e. v. (1995) successful reengineering demands is/business partnerships. sloan management review, 36:4, 5160. miles, r. e., coleman, h. j., jr. and creed, w. e. d. (1995) keys to success in corporate redesign. california management review, 37:3, 128-145. moad, j. (1993). does reengineering really work? datamation. pedhazur, e. j. and schmelkin, l. p. (1991) measurement, design, and analysis : an integrated approach. lawrence erlbaum associates: hillsdale, n.j. romney, m. (1994) business process re-engineering. cpa journal, 64:10, 30-32. 68 ajis vol. 6 no. 1 september 1998 romney, m. (1996) has reengineering left you financially stronger? cpa journal, 66:1. simsion, g. (1997) new life for bpr. management information systems, july. stoddard, d. b. and jarvenpaa, s. l. (1995) business process redesign tactics for managing radical change. journal of management information systems, 12:1, 81-107. taylor, j. a. (1995) don't obliterate informate! bpr for the information age. new technology work and employment, 10:2, 82-88. 69 paper title (paper title style) australasian journal of information systems wang, weeger & gewald 2017, vol 21, research article employee participation in corporate byod programs 1 factors driving employee participation in corporate byod programs: a cross-national comparison from the perspective of future employees xuequn wang murdoch university xuequnwang1600@gmail.com andy weeger neu-ulm university of applied sciences heiko gewald neu-ulm university of applied sciences abstract as individuals all around the world increasingly use mobile devices in their daily life, their desire to use the same devices in the workplace continuously grows. in response, organizations are more and more allowing their employees to use their own devices for both business and private purposes and offer so called ‘bring-your-own-device’ (byod) programs. for organizations with global operations there is a need to examine the drivers of byod demand across different national cultures to assess how to develop a successful byod program. based on recent literature on byod, we examine how different factors contribute to employees’ behavioural intention to participate in a byod program across different national cultures. the model was examined by surveying students from china, germany and u.s. in their final term. the results show significant cross-cultural differences, particularly regarding the 'perceived threats'. overall this study offers novel insights for cross cultural byod implementations. keywords: it consumerization, cultural differences, bring-your-own-device (byod), behavioural intention, utaut 1 introduction the widespread availability of mobile devices like smart phones and tablet computers in combination with increased wireless coverage and the accompanying decrease in costs for data communication has resulted in accelerated diffusion of mobile services, specifically in developed economies (min 2010). in 2013, worldwide sales of smart phones surpassed sales of basic feature mobiles (lomas 2014) and nowadays the latter are hardly sold anymore. individuals all over the world accustomed to the convenience of using their advanced mobile devices whenever and wherever they want (johnson and joshi 2012). this trend induced a major shift in the deployment and management of information technology (it). the driving force for the diffusion of innovative end user technologies shifted from the it-department to the consumer (beimborn and palizta 2013). contemporary employees are more comfortable with the functionalities of their consumer devices than with the devices provided by corporate it (gajar et al. 2013, steelman et al. 2016). there are also more knowledgeable about available options than generations before. in line with these developments, employers nowadays regard flexibility and mobility as essential attitudes (harris et al. 2012), and employees are increasingly expected to complete their business tasks quickly, efficiently and from anywhere (johnson and joshi 2012). in order to meet these requirements, employees increasingly expect to be able to fulfil their tasks in the manner they chose and to use the devices they are familiar with, i.e. their own devices. the trend of using consumer it resources (devices, systems and services) in both private and business contexts is referred to as the ‘consumerization of it’ (harris and junglas 2011; holtsnider and jaffe 2012; ingalsbe et al. 2011). ‘bring-your-own-device’ (byod) refers to australasian journal of information systems wang, weeger & gewald 2017, vol 21, research article employee participation in corporate byod programs 2 employers allowing and sometimes even encouraging their employees to use privately owned mobile devices to complete business tasks (lebek et al. 2013). byod initiatives are increasingly prevalent among enterprises worldwide (johnson and joshi 2012, steelman et al. 2016). although byod is getting more popular in practice, information systems (is) research has not yet reached full understanding of this phenomenon. in particular the factors influencing it consumerization were identified as area in need of further research (dernbecher et al. 2013). byod is a global issue following the trend of globalization. companies of all sizes operate in several countries, having to cater for a cross-cultural workforce. these corporations need to take the differences and characteristics of national cultures and values of their workforce into account. previous is literature has shown that culture is critical to understand is phenomena (leidner and kayworth 2006) such as technology adoption (li et al. 2010; park et al. 2004). however, previous is research has under-investigated country-based differences of the relationship between constructs (li et al. 2010). thus, there is a need to examine the factors driving employees across different countries to participate in byod programs. byod is a phenomenon that swamped from private life into the business context (steelman et al., 2016). the generation currently entering the workplace (so called "digital natives" (prensky 2001)), grew up being "always on". as such they cannot imagine a true offline world anymore (colbert et al., 2016). these young people often see byod as a necessity of daily life (fenn et al., 2011) and demand it from their future employers. as attracting highly skilled talent, especially in it, proves to be ever more difficult for companies these days, the pressure arising to provide byod programs increases. connecting the aforementioned arguments, the research question of this study is: what are the factors driving future employees across different cultural backgrounds to participate in corporate byod programs? our study examines how different factors influence individuals’ intention to participate in byod programs based on data from three economically strong countries (china, germany and u.s.) which host several multinational corporations. cross-cultural it use is still underinvestigated, and previous literature particularly has not fully examined if research models developed in one country can be generalized to other countries (zhao et al. 2007). recently, weeger et al. (2015) proposed and tested a model examining how various factors influence individuals' intention to participate in byod programs. the research presented here extends the aforementioned study to examine their model across different countries. thus, we contribute to global it research and provide insights for practitioners how to deploy byod programs across different countries. especially, companies may face challenges to operate in different countries of pacific asia due to varying cultures and workplace habits. therefore, understanding the effect of cultures can help those companies to more successful deploy their byod programs in different countries (french et al. 2014). the paper is structured as follows: in the following section, previous literature on it consumerization and byod, technology related service adoption and cross-cultural is research is discussed. based on this literature review, the research hypotheses and research model are developed. subsequently, research method and data analysis follow. finally, the implications for theory and practice are discussed. 2 literature review 2.1 it consumerization and bring-your-own-device (byod) it consumerization is frequently conceptualized as the dual use of consumer devices, applications and services for private and business purpose (harris and junglas 2011; holtsnider and jaffe 2012; ingalsbe et al. 2011), hence, defined as a scenario in which employees “invest their own resources to buy, learn, and use consumer technology at their workplace” (niehaves et al. 2012, p. 2). focusing on mobile hardware devices, byod is defined as the act of connecting a personally owned device to the corporate network and using it for business purposes (see e.g. harris et al. 2013). australasian journal of information systems wang, weeger & gewald 2017, vol 21, research article employee participation in corporate byod programs 3 it consumerization and byod are consequences of a reversed technology adoption lifecycle (baskerville 2011; jaramillo et al. 2013; weiß and leimeister 2012). in the past, innovative it first emerged in the professional context (e.g. the personal computer) and then spread into consumer markets. today, the diffusion of new it can also be triggered by employees and then find its way into business (e.g. the iphone). innovative it first enters the consumer market, employees adopt it for private purposes and get familiar with the technology and its characteristics (e.g. ease of use, flexibility). consequently they are expecting similar qualities from corporate it and/or are starting to use their private devices for business purposes without consent from corporate it. subsequently, boundaries between consumer and business technologies begin to blur (colbert et al., 2016). this trend can be observed with the diffusion of smartphones. the advantages of mobile devices targeted at the consumer sector have changed employee’s expectations towards corporate it (weiß and leimeister 2012). the devices provided by the it department are specifically developed for corporate use (e.g. the blackberry which is still very popular in corporate settings around the world) but frequently not match up to consumer devices in terms of usability and user experience (murdoch et al. 2010; harris et al. 2011). furthermore, it was demonstrated that conducting work tasks by using privately-owned devices provides benefits such as greater autonomy and flexibility (harris and junglas 2011; murdoch et al. 2010). some scholars even expect it consumerization to contribute positively to work performance by increasing satisfaction, flexibility and mobility (niehaves et al. 2013; romer 2014). on organizational level, significant cost reductions for organizations are expected (ingalsbe et al. 2011; holtsnider et al. 2012). on the other hand, the phenomenon is also seen critically and several disadvantages for employees and organizations are pointed out. on the employee level, it is supposed that productivity gains go with increased individual workload and blurred boundaries between business and private lives (ingalsbe et al. 2011; niehaves et al. 2012). for organisations, security issues, increasing complexity, loss of control and performance issues are pointed out as negative effects (colbert et al., 2016, niehaves et al., 2012; tokuyoshi 2013) and it is expected that it management in organizations is challenged for the foreseeable future (weiß and leimeister 2012). in an effort to avoid end users to connect their devices uncontrolled to the corporate network and to safeguard data security, technology standardization and compatibility (györy et al. 2012), many organizations establish official byod programs (harris et al., 2012). these initiatives enable employees to choose and use devices selected on their own. usually these programs comprise definitions of the technical requirements privately owned devices have to meet and policies with respect to byod usage, security, and liability which employees have to accept (crossler et al. 2014; harris 2012; vogel et al. 2010). for example, employees typically must permit their employer to remove all data from byod devices (“wipe out”) if the item is lost or the employee resigns (harris et al. 2012). 2.2 byod adoption and utaut byod initiatives are service offerings by organizations to their employees. recently, literature has begun to examine various aspects of byod usage. some studies focus on employees’ compliance to information security policies when using byod (alaskar and shen, 2016; crossler et al., 2014; hovav and putri, 2016). crossler et al. (2014) found that self-efficacy, response efficacy, threat severity and cost to comply can influence individuals’ motivation to comply with a byod policy. other studies examines byod intention and usage. wang and nemati (2016) draw complex adaptive systems as a theoretical lens to explore a holistic understanding of byod usage. on the other hand, few studies have examined byod across different cultures. prior literature shows that unified theory of acceptance and use of technology (utaut) is well suited to explain the adoption of technology related services (amberg et al. 2004; chen et al. 2007; hung and chang 2005; mallat et al. 2008). the main argument of utaut is that there are three variables (i.e., performance expectancy, effort expectancy and social influence) australasian journal of information systems wang, weeger & gewald 2017, vol 21, research article employee participation in corporate byod programs 4 impacting people’s behavioural intention, and two variables (i.e., facilitating conditions and behavioural intention) directly influencing technology use (venkatesh et al. 2003). besides, gender, age, experience and voluntariness moderate the relationships between these variables and intention and / or behaviour. recently, weeger et al. (2015) propose a modified utaut to examine people’ intention to adopt byod services. the authors conducted two modifications: first, facilitating conditions are excluded since the model does not examine byod usage behaviour. second, perceived threats are added to cover the negative aspect of byod adoption. in this study, we extend the previous work of weeger et al. (2015) by examining how the effects of various factors on byod adoption intention differ across various cultures, thus contributing previous literature by providing useful insights regarding how to deploy byod programs globally. 2.3 cross-cultural is research byod is a challenging phenomenon for it executives worldwide. although byod is present in virtually all developed economies, anecdotal evidence from multi-national companies shows that the motivational factors are quite different. this is supported by prior research which shows that characteristics of national culture strongly impact an individual’s adoption behaviour (srite and karahanna, 2006). national culture refers to “the collective programming of the mind that distinguishes the members of one group or category of people from another” (hofstede 2001, p. 9). hofstede (2001) proposes five dimensions of national cultures: power distance, uncertainty avoidance, individualism / collectivism, masculinity / femininity, and long / short term orientation. hofstede’s culture dimensions have been widely applied in various contexts. previous crosscountry is studies have examined the role of national cultures in influencing customer satisfaction with online stores (lee et al. 2009), behavioural intention towards usage of instant messaging services (li et al., 2010), purchasing behaviour in e-commerce (park et al. 2004) and security awareness (schmidt et al. 2008) to name just a few. 3 research model in this study, we examine an individual’s decision to participate in a corporate byod program across different cultures drawn on the model from weeger et al. (2015). the research model and its constituting constructs, proposed relationships and hypotheses are depicted in figure 1. behavioral intention (bi) performance expectancy (pe) effort expectancy (ee) social influence (si) perceived threats (pt) perceived business threats (pt-b) perceived private threats (pt-p) h5b (+) h2 (-) h3 (+) h4 (-) h5a (+) h1 (+) figure 1: research model australasian journal of information systems wang, weeger & gewald 2017, vol 21, research article employee participation in corporate byod programs 5 3.1 cultural values in response to mccoy et al.’s (2005, p. 211) call that as “globalization of businesses and systems continues to increase, our understanding about the adoption and use of it needs to apply to other cultures”, we aim to understand to what extend culture impacts byod decision making. particularly, it is of interest to understand if and how the factors driving individuals to participate in a byod program vary across different cultural backgrounds. among the five dimensions of national cultures proposed by hofstede, individualism /collectivism and uncertainty avoidance are particular relevant to understand the adoption of byod programs. uncertainty avoidance refers to the extent to which individuals feel threatened by ambiguous situations (hofstede 2001). people with high uncertainty avoidance try to avoid all forms of uncertainty by establishing formal rules and rejecting deviant ideas and behaviours. since outcomes of adopting byod services can be negative in certain contexts and are indeed uncertain, people with a high level of uncertainty avoidance may perceive the threats associated with byod services (i.e., perceived threats) more severely and choose not to adopt these services. individualism / collectivism describes the relationship between individuals and a group, and refers to if individuals’ interests are prioritized over those of a group (hofstede 2001). one important distinction between individualism and collectivism is individuals’ personality orientation (idiocentric vs. allocentric) (triandis 1989). people from individualistic cultures are more idiocentric, feel comfortable to “do their own thing” and deemphasize the interest of the group to which they belong (triandis et al. 1988). those people are often emotionally separated from groups and emphasize personal achievement and productivity. on the other hand, people from collectivist cultures are more allocentric, share the interests of the group, and function as interdependent members. therefore, individualism / collectivism probably influence the effects of performance expectancy, effort expectancy and social influence. 3.2 performance expectancy in line with utaut’s definition, performance expectancy is defined as the degree to which an individual believes that participating in a byod program will help him or her to attain gains in job performance (venkatesh et al. 2003). previous literature has identified several advantages of participating in a byod program such as higher flexibility and mobility (e.g. niehaves et al. 2012). in addition, niehaves et al. (2013) conclude that byod is also able to contribute to work performance as employees realize productivity gains when they are able to select it tools on their own. from a cultural perspective, the effect of performance expectancy is probably more important for people from individualistic cultures (e.g., u.s.) than for those from collectivistic cultures (e.g., china). people from individualistic cultures emphasize personal achievement and productivity. therefore, when considering participation in a corporate byod program, their behavioural intentions are more likely to be driven the perceptions of whether the byod program can provide pragmatic gains, such as improved job performance. subsequently we propose that: h1: there is a positive association between performance expectancy and intention to participate in a byod program, and this association is strongest in the u.s. and weakest in china. 3.3 effort expectancy individuals using their personal devices on the job are most likely familiar with the functionalities of their devices. nevertheless, some effort arises when participating in a corporate byod program (set up the device accordingly, reading the applicable corporate policies, complying with these policies, etc.). expectations about this effort probably cushions behavioural intention. in analogy to thompson et al. (1991) effort expectancy is defined as the degree to which participating in a byod program is perceived as relatively difficult. australasian journal of information systems wang, weeger & gewald 2017, vol 21, research article employee participation in corporate byod programs 6 individualism / collectivism is also expected to affect the relationship between effort expectancy and behavioural intention. people from individualistic culture (e.g., u.s.) put more emphasize on task performance and productivity. therefore, they tend to focus more on important goals (srite and karahanna 2006) and try to achieve those goals more efficiently than those from collectivistic cultures (dagwell et al. 1983). effort expectancy is directly related to efficiency: when the expected effort of the byod program is high, people need to spend more effort to finish their tasks. in such a context, people from individualistic natures are more likely to feel less efficient, decreasing their intention to participate in a byod program. therefore, we hypothesize that: h2: there is a negative association between effort expectancy and intention to participate in a byod program, and this association is strongest in the u.s. and weakest in china. 3.4 social influence social influence is defined as the degree to which an individual perceives that important others think he or she should perform certain behaviour, in this case enrol in a corporate byod program (moore and benbasat, 1991). although prior findings on the effect of social influence on behavioural intention are somehow mixed (e.g. venkatesh et al., 2003; arruda-filho et al., 2010; lópez-nicolás et al., 2008), having the opportunity to purchase fashionable devices such as an iphone or ipad that meet both private and business needs has the potential to enhance an individual’s image perceptions. thus, it is hypothesized that expected social gains will impact an individual’s behavioural intention. from a collectivistic culture (e.g., china) people probably function as interdependent members and share the interests of the group. therefore, people from collective cultures value the image and prestige within the group more as they emotionally attached to the group. in other words, in a strong collective culture, people tend to work collectively and respect others ideas and opinions; thus, they are more likely to comply with the opinions of important others (srite and karahanna 2006). as a consequence, if participating in a byod program is valued by colleagues and can positively increase individuals' image within the organization, people from collectivistic cultures are more likely to participate in a byod program. therefore, we propose: h3: there is a positive association between social influence and intention to participate in a byod program, and this association is strongest in china and weakest in the u.s. 3.5 perceived threats perceived threats are defined as the extent to which individuals apprehend that byod usage may worsen the status quo (weeger et al. 2015). perceived threats include two dimensions: business related threats and threats related to private life (niehaves et al. 2012). typical business related threats include being responsible for a disclosure or loss of business data if the device gets stolen, malwares disrupting the corporate it infrastructure, or violating company policies (niehaves et al. 2012). commonly cited threats to private life are that companies typically reserve the right to remove both private and business-related data from a lost device (harris et al. 2012), that the employers obtain private data from the device, and that the boundaries between private and business life are blurred. uncertainty avoidance can be useful to understand how the effects of perceived threats differ across different cultures. according to hofstede (2001), uncertainty avoidance represents how people feel threatened by unknown situations. therefore, the level of uncertainty avoidance shows how people are tolerant to various threats and risks. previous literature has found that people from high uncertainty avoidance cultures are more likely to show a higher level of risk aversion and a lower level of satisfaction in the context of e-commerce (lee et al. 2009). in the context of byod, people from a low level of uncertainty avoidance culture (e.g., china) are relatively comfortable with ambiguity and threats, so threats may not have a strong effect on intent to participate in a byod program. on the other hand, those from a high level of australasian journal of information systems wang, weeger & gewald 2017, vol 21, research article employee participation in corporate byod programs 7 uncertainty avoidance culture (e.g., germany) may try to avoid threats, and the effect of potential threats is assumed to have a stronger effect. therefore, it is hypothesized that: h4: there is a negative association between perceived threats and intention to participate in a byod program, and this association is strongest in germany and weakest in china. weeger et al. (2015) model perceived threats as a second-order multi-faceted construct encompassing perceived business threats and perceived private threats. perceived business threats are defined as “the extent to which an individual perceives using personal devices for business purposes as threatening his or her job performance” (p. 4). perceived private threats are defined as “the extent to which an individual perceives participating in a corporate boyd program as compromising his or her private life” (p. 3). both dimensions are expected to contribute to the overall threats perceived by an individual. subsequently, it is hypothesized that: h5a: there is a positive association between perceived business threats and perceived threats. h5b: there is a positive association between perceived private threats and perceived threats. 4 research methodology the model was tested using a questionnaire through an online survey among students from germany, china, and the u.s. below, the rational for choosing a student sample, instrument development and data collection are discussed below. 4.1 sample current students were born into the ‘digital age’ where technology is ubiquitous and are therefore referred to as ‘digital natives’ (prensky, 2001). it is expected that it consumerization will gain even more momentum in the future driven by these digital natives (fenn et al., 2011). as today's students are the work-force of tomorrow their relationship to technology will influence the future way of working. therefore we decided to examine student’s intention to participate in a byod program. in order to decide whether to participate in an official byod program relevant work experience is necessary. therefore, the research participants of this study are students in their final year of study who already collected significant work experiences during their studies (such as an internship or a practical semester). moreover, we focused on students with business, information systems or engineering majors only based on the assumption that these students are tech savvy and have an opinion on byod. germany, u.s. and china were selected to examine the effect of cultural values on byod adoption for two reasons. first, those three countries are among the top 14 mobile market with over 100 million mobile subscriptions (global mobile statistics 2013). thus, people from those countries probably often use mobile devices in their daily life and make sense of the research context. second, those three countries are from different regions and have different scores of hofstede (2013)’s national cultures, making cross-national comparison appropriate. specifically, previous literature has often selected china and u.s. as representative of the collectivist and individualistic culture respectively (hofstede 1991; li et al. 2010). by selecting germany as the additional research context, our study can complement previous cross-culture is literature and provide additional insights of individualism /collectivism culture (table 1). besides, germany is regarded as a high uncertainty avoidance culture, and it is appropriate to compare germany to china and u.s. on uncertainty avoidance. australasian journal of information systems wang, weeger & gewald 2017, vol 21, research article employee participation in corporate byod programs 8 power distance individualism masculinity uncertainty avoidance long-term orientation germany 35 67 66 65 31 china 80 20 66 30 118 u.s. 40 91 62 46 29 table 1: national cultural dimensions (hofstede 2013) 4.2 instrument in order to safeguard a similar understanding of byod programs among the participants, the applicable byod concept was given in the beginning of the questionnaire: corporate byod programs enable employees to use their private it devices for business purposes. this study builds upon the byod concept ‘we sponsor your device’ where the employee purchases a/several mobile device(s) (e.g., a smartphone, notebook, tablet), agrees to use the device(s) for business purpose and receives financial compensation from the company. this compensation is usually fixed and valid for a given time frame (e.g., $4,000 for four years). the employee can freely decide which devices he or she uses within this time frame. as the company usually does not provide the employee with additional devices for business purpose, he or she is fully responsible for having sufficient it equipment to fulfil his or her business needs. items from weeger et al. (2015) were used as basis. all items were discussed intensively within the research team to assess whether these items can capture corresponding constructs. furthermore, a pre-test within the target group was conducted to ensure that the questionnaire is comprehensible and unambiguous. additional tests were conducted to ensure that indicators of perceived threats are clearly distinguished and that every aspect of the construct has been captured. table 2 summarizes the latent variables, their types (reflective or formative), and the number of items. please note that perceived threats are measured as a second-order formative construct by perceived private threats and perceived business threats. the measurement items used in the instrument are presented in table a.1. construct type items behavioural intention (bi) reflective 3 performance expectancy (pe) reflective 4 effort expectancy (ee) reflective 3 social influence (si) reflective 4 perceived threats (pt) formative 8 perceived private threats (pbt) formative 4 perceived business threats (ppt) formative 4 table 2: construct operationalization 4.3 data collection in order to recruit participants, professors of higher education institutions in germany, the u.s. and china were invited to participate in the study1. data was collected during a period of four weeks in december 2012. professors in the research group asked undergraduate students in their last terms to participate in the online survey. to ensure that students can assess the effects of byod realistically, only students with relevant work experiences (such as an internship or a practical semester) were invited to participate. to reduce nonresponse bias during data collection, the professors regularly reminded their students to participate. in total, 485 responses were collected and reviewed to identify whether respondents completely finished the survey. our analysis revealed that 183 lacked at least one value 1 we gratefully acknowledge support for data collection provided by the following scholar: mahesh s. raisinghani. australasian journal of information systems wang, weeger & gewald 2017, vol 21, research article employee participation in corporate byod programs 9 (including demographics), resulting in 302 usable responses for the complete dataset. the demographics of the sample are depicted in table 32. gender male 54.0% country u.s. 38.7% female 46.0% china 33.4% age < 18 years 0.3% germany 27.8% 18 21 years 42.4% 22 25 years 28.8% 26 29 years 13.9% > 29 years 14.6% study focus business 72.5% engineering 16.2% interdisciplinary 11.3% table 3: demographics of the dataset 5 data overview the model was tested using pls3, a structural equation modelling method for complex predictive models and theory building (barclay et al. 1995; chin 1998; lohmöller 1989). pls is the preferred analytical technique of this study because of its compatibility with formative measurement (gefen et al. 2011). besides, shapiro-wilk tests were significant, showing that the measurements were not normally distributed. according to hair et al. (2014), pls is more appropriate with non-normally distributed data4. before evaluating the models, we conducted two common method variance (cmv) tests to examine whether common method bias was a concern (lindell and whitney 2001; podsakoff et al. 2003). first, an explanatory factor analysis of all items extracted four factors explaining 63.42% of the variance, with no single factor accounting for significant loading (at the p < 0.05 level) for all items. second, we used the second-smallest positive correlation among the manifest variables as a conservative estimate for cmv. after adjustment, all significant correlations remained significant. therefore, cmv is probably not a concern in this data set. 5.1 measurement model assessment the measures of reflective constructs from the german dataset demonstrate good psychometric properties. convergent validity was confirmed by meeting the following criteria (e.g. gefen and straub 2005; hulland 1999): (a) the loadings of each item were all significant and above the cut-off value of 0.60 (table 3); (b) the composite reliabilities (crs) of all constructs were above 0.70 (table 3); (c) the average variance extracted (ave) of all constructs was above the threshold value of 0.50 (table 4). discriminant validity was established by ensuring that the square root of ave for each construct exceeded the correlations between that construct and any other construct (gefen and straub, 2005) (table 5). 2 here we use “country of study” as indicator of culture. we also conducted separate analyses based on “country of origin” and found no significant differences. 3 the software smartpls 2.0 (ringle et al. 2005) was used to estimate the model and the bootstrap resampling method (using 5,000 samples) was used to determine the significance of the paths in the structural model. 4 we also checked skewness and kurtosis and their values are below 1.96. therefore, the values of skewness and kurtosis are not significant (p > .05) and our data is not highly non-normal. australasian journal of information systems wang, weeger & gewald 2017, vol 21, research article employee participation in corporate byod programs 10 scale item item mean item s. d. item loading ave cr bi1 2.74 1.09 0.93 0.86 0.95 bi2 2.68 1.04 0.92 bi3 2.73 1.10 0.92 ee1 2.51 0.86 0.78 0.66 085 ee2 2.05 0.82 0.86 ee3 2.38 1.02 0.78 pe1 2.94 0.97 0.88 0.67 0.89 pe2 2.25 0.92 0.73 pe3 2.98 1.11 0.85 pe4 2.96 0.99 0.82 si1 3.45 1.07 0.91 0.79 0.94 si2 3.38 1.10 0.92 si3 3.43 1.25 0.88 si4 3.57 1.03 0.84 table 4: descriptive statistics, item loadings and constructs’ ave and cr (germany) construct bi ee pe si bi 0.92 ee 0.32 0.81 pe 0.70 0.31 0.82 si 0.21 -0.22 0.16 0.89 table 5: correlation between constructs and square-root of aves (germany) for indicators of threat-constructs (pbt and ppt), collinearity needs to be assessed by examining variance inflation factor (vif) and tolerance value for each indicator. the results show that no indicators from perceived private threats and perceived business threats have a tolerance value below 0.20 or a vif value above 5, indicating that collinearity is probably not an issue. then the weights of each indicator were examined (table 6). only the weights of the indicator pbt4 from pbt and the indicator ppt1 from ppt are significant. construct pbt ppt pbt1 0.17 pbt2 0.22 pbt3 0.33 pbt4 0.61** ppt1 0.61** ppt2 0.47 ppt3 -0.03 ppt4 0.21 * p< 0.05, ** p< 0.01 table 6: indicators’ weights in formative constructs (germany) following the same process, we also examined the data from china (table 7, table 8 and table 9) and u.s. (table 10, table 11 and table 12), and the results show that reflective measures have good psychometric properties. vif and tolerance value were also examined for each data set and no indicators have a tolerance value below 0.20 or a vif value above 5. on the other hand, most of indicators’ weights are not significant, indicating that the results of threat constructs should be interpreted cautiously. australasian journal of information systems wang, weeger & gewald 2017, vol 21, research article employee participation in corporate byod programs 11 scale item item mean item s. d. item loading ave cr bi1 1.97 0.64 0.78 0.60 0.82 bi2 1.93 0.57 0.75 bi3 1.97 0.70 0.80 ee1 2.86 0.95 0.67 0.59 0.81 ee2 2.71 1.04 0.95 ee3 3.13 1.00 0.64 pe1 1.98 0.75 0.73 0.57 0.84 pe2 2.20 0.79 0.74 pe3 2.36 0.92 0.75 pe4 2.14 0.79 0.80 si1 2.44 0.81 0.71 0.57 0.84 si2 2.46 0.78 0.79 si3 2.61 0.81 0.72 si4 2.41 0.83 0.78 table 7: descriptive statistics, item loadings and constructs’ ave and cr (china) construct bi ee pe si bi 0.78 ee -0.12 0.77 pe 0.49 0.11 0.75 si 0.36 -0.22 0.28 0.75 table 8: correlation between constructs and square-root of aves (china) construct pbt ppt pbt1 1.23 pbt2 -0.88 pbt3 -0.04 pbt4 0.10 ppt1 1.22 ppt2 -0.54 ppt3 -0.09 ppt4 0.12 * p< 0.05 table 9: indicators’ weights in formative constructs (china) scale item item mean item s. d. item loading ave cr bi1 2.31 1.05 0.95 0.90 0.97 bi2 2.17 1.00 0.94 bi3 2.34 1.01 0.96 ee1 2.31 0.85 0.81 0.72 0.89 ee2 1.93 0.89 0.89 ee3 2.15 0.93 0.85 pe1 2.41 1.09 0.93 0.82 0.95 pe2 1.97 1.00 0.86 pe3 2.64 1.16 0.89 pe4 2.50 1.21 0.93 si1 2.98 1.08 0.90 0.73 0.91 si2 2.88 1.03 0.93 si3 2.94 1.05 0.81 si4 2.98 1.00 0.77 table 10: descriptive statistics, item loadings and constructs’ ave and cr (u.s.) australasian journal of information systems wang, weeger & gewald 2017, vol 21, research article employee participation in corporate byod programs 12 construct bi ee pe si bi 0.95 ee 0.40 0.85 pe 0.62 0.32 0.90 si 0.42 0.05 0.28 0.85 table 11: correlation between constructs and square-root of aves (u.s.) construct pbt ppt pbt1 0.550* pbt2 0.128 pbt3 0.346 pbt4 0.152 ppt1 0.607 ppt2 0.191 ppt3 0.340 ppt4 0.145 * p< 0.05 table 12: indicators’ weights in formative constructs (u.s.) 5.2 structural model assessment pls examines the structural model with path coefficients and r2 measures (gefen et al. 2000). furthermore, the effect size of each path was evaluated by means of cohen’s f² (cohen 1988). the effect size measures whether an independent variable has a substantial impact on a dependent variable. figure 2 and table 13 summarize the results of the structural model assessment for each dataset. behavioral intention (bi) performance expectancy (pe) effort expectancy (ee) social influence (si) perceived threats (pt) perceived business threats (pt-b) perceived private threats (pt-p) 1 | 0.36 2 | -0.01 3 | 0.86* 1 | -0.12 2 | 0.11 3 | -0.23** 1 | 0.14 2 | 0.21** 3 | 0.30*** 1 | -0.19* 2 | 0.10 3 | -0.19** 1 | 0.75** 2 | 1.01* 3 | 0.17 1 | germany 2 | china 3 | u.s. 1 | 0.57*** 2 | 0.43*** 3 | 0.43*** figure 2: model of results for germany (n=84), china (n=101) and the u.s. (n=117) australasian journal of information systems wang, weeger & gewald 2017, vol 21, research article employee participation in corporate byod programs 13 hypothesis germany (n=84) china (n=101) u.s. (n=117) β ƒ² β ƒ² β ƒ² h1 (pe→bi) 0.57*** 0.56 (large) 0.43*** 0.24 (med.) 0.43*** 0.30 (med.) h2 (ee→bi) -0.12 0.11 -0.23** 0.10 (weak) h3 (si→bi) 0.14 0.21** 0.06 (weak) 0.30*** 0.17 (med.) h4 (pt→bi) -0.19* 0.07 (weak) 0.10 -0.19** 0.08 (weak) * p < 0.05, ** p < 0.01, *** p < 0.001 table 13: path coefficients and effect sizes to rule out rival explanations, we added age and gender as controls since those two variables have been identified as important variables in the context of technology adoption (venkatesh et al. 2003). both variables were only found to reveal a significant impact on behavioural intention in the american dataset. the effect of both controls was minor, although significant, and there are no qualitative differences after adding age and gender. finally, cross-culture comparisons were conducted to test each hypothesis. specifically, the formula of keil et al. (2000) is used to assess the statistical differences of the path coefficients between countries: t = path coefficientgroup1 − path coefficentgroup2 [√ (m − 1)2 (m + n − 2) × segroup1 2 + (n − 1)2 (m + n − 2) × segroup2 2 ] × [√ 1 m + 1 n] where m is the sample size for group 1, and n is the same size for group 2. these findings are summarized in table 14 and discussed in the respective section below. hypothesis difference summary hypothesis supported? u.s-germany germanychina u.s.-china h1 (pe→bi) -0.15 (-12.90)* 0.14 (11.77)* -0.01 (-0.69) strongest for germans partially h2 (ee→bi) -0.11 (-9.16)* -0.24 (-15.52)* -0.34 (-23.92)* strongest for americans and weakest for chinese yes h3 (si→bi) 0.16 (-14.70)* -0.07 (5.77)* 0.086 (-8.49)* strongest for americans and weakest for germans no h4 (pt→bi) -0.01 (0.42) -0.28 (-12.57)* -0.29 (-15.26)* strongest for germans and americans partially each cell shows the difference of path coefficients and corresponding t value. * p < 0.05 table 14: overview of results 6 discussion 6.1 factors driving byod adoption and the impact of cultural background the results of this empirical study show that future employees' behavioural intention to participate in a corporate byod program is indeed a function of their perception of whether: 1) using personal devices at work will improve his or her job performance (performance expectancy); 2) usage is free of effort (effort expectancy); 3) it enhances one's social standing (social influence); and 4) usage is associated with business and private threats (perceived threats). australasian journal of information systems wang, weeger & gewald 2017, vol 21, research article employee participation in corporate byod programs 14 it is also shown, that cultural background has a strong impact on the factors motivating certain behavioural intentions. depending on the cultural setting the assessment of the benefits and risks associated with byod differs significantly between participants from different cultural backgrounds. performance expectancy is the strongest predictor of behavioural intention independently of the country of origin. furthermore, it is the only variable with a highly significant effect on behavioural intention across all datasets. the relevance of performance expectations is consistent with prior utaut studies (venkatesh et al. 2003). analysing the means of the measurement furthermore reveals that future employees in particular expect substantial improvements regarding their mobility on the job. however, the relationship between performance expectancy and behavioural intention is stronger for germans but weaker for americans and chinese (no significant difference between china and the u.s), which is not fully consistent with h1. this may be explained by the participants chosen for this study. as students -and specifically digital nativesare assumed to be more focussed on their own results it may well be that they transport these expectations towards their own performance into the expected job. as such, the generic view of hofstede may not hold true for the generation of digital natives. this finding is consistent with venkatesh and zhang (2010)’s argument that young people in u.s. and china tend to both emphasize on performance expectancy. the effect of effort expectancy on behavioural intention is only significant for the u.s. dataset. results here is consistent with h2: americans tend to achieve their goals more efficiently than chinese. our findings regarding the effect of social influence are rather mixed. venkatesh et al. (2003) argue that social influence may not significantly influence intention in voluntary contexts. however, our analysis reveals that social influence imposes the second strongest impact on behavioural intention, and indeed impacts behavioural intention to enrol in a byod program in certain cultures. specifically, the effect of social influence is strongest for americans and weakest for germans (non-significant), which does not support h3. one possible explanation is that the perceptions of who is important vary across different cultural contexts. for example, it is possible that for a culture with a high level of power distance (e.g., china), influence from managers is much stronger than that from colleagues. further research can build upon the findings of eckhardt et al. (2009) in order to assess whether the social influence of the various referent groups in private or workplace contexts differ within and across different cultures (e.g. the influence of a supervisor in contrast to the influence of colleagues and friends). the effect of perceived threats is significant for american and germany students (nonsignificant difference) but not for chinese students, which partially support h4. here chinese students do not perceive threats associated with byod to be an issue. those results are consistent with weber and hsee (1998), who show that comparing to germany and the u.s., chinese are closest to risk neutrality and that they exhibit the smallest degree of perceived risk aversion. hence, it is not surprising that their perceptions on potential threats associated with byod do not significantly impact their behavioural intention. on the other hand, the risk perceptions of germans and americans are comparable. regarding the components of perceived threats, perceived business threats appear to have a stronger effect than threats related to private life. in particular, the risk of being responsible for data theft and the risk of violating corporate policies are the strongest indicators, whereas the risk of corrupting the corporate network and service failures are weaker. as indicators of formative constructs are defining characteristics of the construct, the weights indicate an indicator’s contribution to the formative construct. as such, the results indicate that future employees are particularly concerned about losing business data and violating policies. perceived private threats are not significant (except for american students), and analysing indicator weights shows that increasing workload is not perceived as a potential threat. this finding provides evidence that students are already used to blurred boundaries between work australasian journal of information systems wang, weeger & gewald 2017, vol 21, research article employee participation in corporate byod programs 15 and leisure-times. future studies can further examine how people from various cultures perceive different aspects of threats (business vs. private threats). 6.2 implications for practice this study revealed several implications for practice, especially to firms which operate in multiple countries. first, the intention to enrol in a byod program is significantly related to performance expectancy in every cultural context analysed in this research. therefore, when offering byod programs, companies need to make sure that byod can support employees’ work and increase their performance. otherwise, employees may find that using personal devices cannot provide them enough benefits and may decide not to enrol in byod programs, even if companies provide them such an option. even more, such an experience can lead to frustration and eventually to a negative image of the it-department. this means that it departments not only need to simply offer a byod program, it must be meaningful (i.e. the employees must be able to get real work done using their private devices). this should not end with accessing emails on the private devices but needs to include a broader spectrum of daily work. second, although this study shows that effort expectancy, social influence, and perceived threats significantly influence people's intention participate in corporate byod programs, the results are mixed and context-dependent. therefore, companies need to be sensitive to the specific context in which they operate, focus on the corresponding factors and consider differentiating global byod initiatives by country. separate analysis shows that students from the u.s. are particularly concerned with effort expectancy. therefore, when companies with a significant american workforce design corporate byod programs, they need to take into account that participating in byod programs have to require little effort (e.g. to set up the devices for business-related task or to ensure that policies are not violated). third, social influence significantly impacts people’s intention to participate in byod programs in china and u.s., but not germany. therefore, when companies with workforce from china and u.s. offer corporate byod programs, they may let managers promote the services and provide enough support, so that employees who enrol in byod programs can improve their better image and reputation within the company. finally, participants from germany and u.s. feel concerned with perceived threats. therefore, companies with employees from these countries specifically need to reduce people’s concerns about potential threats from byod. further, german companies need to pay special attention to potential business-related threats, while american companies need to focus more on factors putting private-life on risk. 6.3 limitations and opportunities for further research this study focuses on students in their designation as future employees. although only students with at least three months of relevant work experience were selected, they still have limited work experience and experiences dealing with corporate it devices. for instance, a comparative study among entrants and more experienced employees could enlarge the knowledge in this field. also, by only focusing on students with work experiences, our study may have a bias and the results may not hold for all “future employees”. also, only students from certain fields of study have been selected. results may differ if students from a more diverse setting were included. second, only three universities from three countries respectively were selected to recruit participants, so the sample may not represent the whole population of future employees from these three countries. also, the results may not be able to generalize to other countries/cultural contexts. besides, measurement reflecting cultural values beyond the countries of origin was not collected. therefore, the dataset only allowed country-specific analyses of germany, china and the u.s. our theoretical model and proposed hypotheses mainly focus on two cultural dimensions proposed by hofstede (2001) (i.e., individualism / collectivism and uncertainty avoidance) australasian journal of information systems wang, weeger & gewald 2017, vol 21, research article employee participation in corporate byod programs 16 because these two dimensions are quite relevant to understand byod adoption. we do not mean that other dimensions cannot influence byod adoption. for example, srite and karahanna (2006) find that masculinity/femininity has a moderation effect on the relationship between effect expectancy and behavioural intention. therefore, masculinity/femininity could be another useful cultural dimension to understand byod adoption. further research should deepen the examination of cross-cultural differences and integrate espoused cultural variables as moderators of individuals’ beliefs on participating in byod programs. following the suggestions of srite and karahanna (2006), incorporating espoused national cultural values of masculinity / femininity, individualism / collectivism, power distance, and uncertainty avoidance as moderators seems to be a promising approach for further research. notwithstanding these limitations, the findings of this study open up the field for further research. further research should concentrate on corresponding theoretical approaches and incorporate additional constructs which can be important for an individual’s decision whether to participate in byod programs. besides, there can be other negative aspects of it consumerization behaviour, and future studies can borrow theories from other disciplines to further examine the effect of negative factors on byod adoption. 7 conclusion this study examined cross-national differences of the perceived drivers of it consumerization on an individual level exemplified by corporate byod programs. the study tested the model developed by weeger et al. (2015) with participants from germany, china, and u.s. the results indicate that there are significant cross-cultural differences that impact relevance and strength of the impact of the constructs proposed to influence an individual’s decision to enrol in a corporate byod program. only individual’s expectations regarding gains in job performance, such as increased effectiveness, mobility and productivity was found to impact behavioural intention regardless of an individual’s national culture. in contrast, the impact of effort expectancy, social influence, and perceived threats is highly context-dependent. furthermore, perceptions regarding work-related threats were only found to be significant in germany and china, whereby perceived threats related to private life seem to be relevant for future employees in the u.s. only. knowledge acquired from this study is particularly beneficial to multinational firms planning to set up or adjust global byod initiatives. the results documented in this paper provide a basis for future research on the antecedents and consequences of it consumerization in a global context. references alaskar, m., and shen, k. n. (2016). understanding bring your own device (byod) and employee information security behaviors from a work-life domain perspective. proceedings of 22nd americas conference on information systems. amberg, m., m. hirschmeier, and j.wehrmann. 2004. “the compass acceptance model for the analysis and evaluation of mobile services.”international journal of mobile communications2 (3): 248-259. arruda-filho, e. j. m., j. a. cabusas, and n. dholakia. 2010.“social behavior and brand devotion among iphone innovators.”international journal of information management 30 (6): 475-480. barclay, d., c. higgins, and r. thompson. 1995.“the partial least squares (pls) approach to causal modeling: personal computer adoption and use as an illustration.”technology studies 2 (2): 285-309. baskerville, r. 2011. “individual information systems as a research arena.”european journal of information systems 20 (3): 251-254. australasian journal of information systems wang, weeger & gewald 2017, vol 21, research article employee participation in corporate byod programs 17 bauer, r. a. 1967. “consumer behavior as risk taking.” in risk taking and information handling in consumer behavior, edited by d.f. cox, 23-33, cambridge, usa: harvard university press. beimborn, d., and m. palitza 2013.“enterprise app stores for mobile applications development of a benefits framework.” paper prested in nineteenth americas conference on information systems, chicago. cenfetelli, r. 2004.“inhibitors and enablers as dual factor concepts in technology usage.” journal of the association for information systems5 (11): 472-492. chen, c.-d., y.-w. fan, and c.-k. farn 2007. “predicting electronic toll collection service adoption: an integration of the technology acceptance model and the theory of planned behavior.” transportation research part c: emerging technologies15 (5): 300-311. chin, w. w. 1998. “commentary: issues and opinion on structural equation modeling.”mis quarterly 22 (1): vii-xvi. cohen, j. 1988. statistical power analysis for the behavioral sciences.hillsdale, nj: routledge academic. colbert, a., yee, n. and george, g. 2016. "the digital workforce and the workplace of the future." academy of management journal 59 (3): 731-739 crossler, r. e., j. h. long, t. m. loraas, and b. s. trinkle. 2014. “understanding compliance with bring your own device policies utilizing protection motivation theory: bridging the intention-behavior gap.” journal of information systems 28(1): 209-226. dagwell, r., r. weber, and r. kling. 1983. “system designer's user models: a comparative study and methodological critique.”communications of the acm 26 (11): 987-997. davis, f. d. 1989.“perceived usefulness, perceived ease of use, and user acceptance of information technology.” mis quarterly 13 (3): 319-340. dernbecher, s., r. beck, and s. weber. 2013.“switch to your own to work with the known: an empirical study on consumerization of it.” paper presented in nineteenth americas conference on information systems, chicago. eckhardt, a., s. laumer, and t. weitzel. 2009. “who influences whom? analyzing workplace referents' social influence on it adoption and non-adoption.” journal of information technology 24 (1): 11-24. fenn, j., and h. lehong. 2011. hype cycle for emerging technologies. gartner. french, a. m., and guo, c., and shim, j. p. 2014. current status, issues, and future of bring your own device (byod) communications of the association for information systems 35: 191-197. gajar, p. k., a. ghosh, and s. rai. 2013. “bring your own device (byod): security risks and mitigating strategies.” journal of global research in computer science 4(4): 62-70. gefen, d., e. e. rigdon, and d. straub. 2011. an update and extension to sem guidelines for administrative and social science research. mis quarterly, 35 (2): iii-xiv. gefen, d., and d. straub. 2005.“a practical guide to factorial validity using pls-graph: tutorial and annotated example.”communications of the association for information systems 16:article 5. gefen, d., d. straub, and m.-c. boudreau. 2000.“structural equation modeling and regression: guidelines for research practice.”communications of the association for information systems 16 (5): 91-109. global mobile statistics. 2013.“global mobile statistics 2013 part a: mobile subscribers; handset market share; mobile operators.”http://mobithinking.com/mobile-marketingtools/latest-mobile-stats/a#topsmartphonecountries. accessed on march 6th, 2014. australasian journal of information systems wang, weeger & gewald 2017, vol 21, research article employee participation in corporate byod programs 18 györy, a., a. cleven, u. falk, and w. brenner. 2012. “exploring the shadows: it governance approaches to user-driven innovation” paper presented in ecis proceedings 2012. hair, j. f., g. t. m. hult, c. m. ringle, and m. sarstedt 2014. a primer on partial least squares structural equation modeling (pls-sem). thousand oaks, ca: sage. harris, c. 2012.“mobile consumerization trends & perceptions it executive and ceo survey,” decisive analytics, llc. harris, j., b. ives, and i. junglas. 2012.“it consumerization: when gadgets turn into enterprise it tools.”mis quarterly executive 11 (3): 99-112. harris, j. g., and i. junglas. 2011.“the genie is out of the bottle: managing the infiltration of consumer it into the workforce,” accenture institute for high performance. harris, m. a., k. patten, and e. regan. 2013.“the need for byod mobile device security awareness and training” paper presented in nineteenth americas conference on information systems, chicago. harris, m., k. patten, e. regan, and j. fjermestad. 2012. “mobile and connected device security considerations: a dilemma for small and medium enterprise business mobility.” paper presented in eighteenth americas conference on information systems, seatle, washington. hofstede, g. 2001. culture’s consequences: comparing values, behaviors, instituions, and organization across nations. beverly hills, ca: sage. hofstede, g. 2013. “what about germany.” retrieved 04.07.2013, from http://geerthofstede.com. holtsnider, b., and b. d. jaffe. 2012. it manager's handbook: getting your new job done, waltham, ma: morgan kaufmann. hovav, a., and putri, f. f. (2016). this is my device! why should i follow your rules? employees’ compliance with byod security policy. pervasive and mobile computing. in press hulland, j. 1999. “use of partial least squares (pls) in strategic management research: a review of four recent studies.” strategic management journal 20 (2): 195-204. hung, s.-y., and c.-m.chang. 2005.“user acceptance of wap services: test of competing theories.” computer standards & interfaces 27 (4): 359-370. ingalsbe, j. a., d. shoemaker, and n. r. mead. 2011.“threat modeling the cloud computing, mobile device toting, consumerized enterprise – an overview of considerations” paper presented in seventeenth americas conference on information systems, detroit. jaramillo, d., n. katz, b. bodin, w. tworek, r. smart, and t. cook 2013. cooperative solutions for bring your own device (byod). ibm journal of research and development 57(6):5:1-5:11. johnson, n., and k. joshi. 2012.“the pathway to enterprise mobile readiness: analysis of perceptions, pressures, preparedness, and progression.” proceedings of eighteenth americas conference on information systems, seattle, washington. keil, m., b. c. y. tan, k. wei, t. saarinen, v. tuunainen, and a. wassenaar. 2000. "a crosscultural study on escalation of commitment behavior in software projects," mis quarterly, 24 (2), 299-324. lebek, b., k. degirmenci, and m. h. breitner. 2013.“investigating the influence of security, privacy, and legal concerns on employees' intention to use byod mobile devices.” paper presented in nineteenth americas conference on information systems, chicago. australasian journal of information systems wang, weeger & gewald 2017, vol 21, research article employee participation in corporate byod programs 19 lee, k., k. joshi, and m. bae. 2009. “a cross-national comparison of the determinants of customer satisfaction with online stores.”journal of global information technology management 12 (4): 25-51. li, d., chau, p. y. k., and slyke, c. v. (2010). a comparative study of individual acceptance of instant messaging in the us and china: a structural equation modeling approach. communications of the association for information systems (26): 85-106. leidner, d. e., and t. kayworth. 2006.“review: a review of culture in information systems research: toward a theory of information technology culture conflict.”mis quarterly 30 (2): 357-399. lindell, m. k., and d. j. whitney. 2001.“accounting for common method variance in crosssectional research designs.”journal of applied psychology 86 (1): 114-121. lohmöller, j.-b. 1989.latent variable path modeling with partial least squares, physicaverlag heidelberg. lomas, n. 2014. “gartner: smartphone sales finally beat out dumb phone sales globally in 2013, with 968m units sold.” http://techcrunch.com/2014/02/13/smartphonesoutsell-dumb-phones-globally/. feb. 13th, 2014. accessed on march 6th, 2014. lópez-nicolás, c., f. j. molina-castillo, and h. bouwman. 2008.“an assessment of advanced mobile services acceptance: contributions from tam and diffusion theory models.”information & management 45 (6): 359-364. mallat, n., m. rossi, v. k. tuunainen, and a. öörni. 2008.“an empirical investigation of mobile ticketing service adoption in public transportation.”personal and ubiquitous computing 12 (1): 57-65. mccoy, s., d. f. galletta, and w. r. king. 2005. “integrating national cultre into is research: the need for current individual-level measures.”communications of the association for information systems 15: 211-224. min, s.-j. 2010. "from the digital divide to the democratic divide: internet skills, political interest, and the second-level digital divide in political internet use." journal of information technology & politics (7:1): 22-35 moore, g. c., and i. benbasat. 1991.“development of an instrument to measure the perceptions of adopting an information technology innovation.”information systems research 2 (3): 192-222. murdoch, r., j. g. harris, and g. dvore. 2010. “can enterprise it survive the meteor of consumer technology?” accenture institute for high performance. niehaves, b., s. köffer, and k. ortbach. 2012.“it consumerization – a theory and practice review.” paper presented in eighteenth americas conference on information systems, seatle, washington. niehaves, b., s. köffer, and k. ortbach. 2013.“the effect of private it use on work performance towards an it consumerization theory.” paper presented in international conference on wirtschaftsinformatik, leipzig. pagani, m. 2004. “determinants of adoption of third generation mobile multimedia services.” journal of interactive marketing 18 (3): 46-59. park, j., d. lee, and j. ahn.2004.“risk-focused e-commerce adoption model: a cross country study.”journal of global information technology management 7 (2): 6-30. podsakoff, p. m., s. b. mackenzie, j.-y. lee, and n. p. podsakoff. 2003.“common method biases in behavioral research: a critical review of the literature and recommended remedies.”journal of applied psychology 88 (5): 879-903. prensky, m. 2001.“digital natives, digital immigrants.”on the horizon 9 (5): 1-6. australasian journal of information systems wang, weeger & gewald 2017, vol 21, research article employee participation in corporate byod programs 20 ragu-nathan, t. s., m. tarafdar, b. s. ragu-nathan, and q. tu. 2008. "the consequences of technostress for end users in organizations: conceptual development and empirical validation," information systems research 19 (4): 417-433. ringle, c.m., s. wende, and s. will. 2005. smartpls 2.0 (m3) beta. hamburg, http://www.smartpls.de. romer, h. 2014. best practices for byod security. computer fraud & security, 2014(1): 1315. schmidt, m. b., a. c. johnston, k. p. arnett, j. q. chen, and s. li. 2008. “a cross-cultural comparison of u.s. and chinese computer security awareness.” journal of global information management 16 (2): 1-103. srite, m., and e. karahanna. 2006. “the role of espoused national cultural values in technology acceptance.”mis quarterly 30 (3): 679-704. tarafdar, m., q. tu, b. s. ragu-nathan, and t. s. ragu-nathan. 2007. "the impact of technostress on role stress and productivity," journal of management information systems 24 (1): 301-328. tarique, i., and r. s. schuler. 2010.“global talent management: literature review, integrative framework, and suggestions for further research.”journal of world business 45 (2): 122133. thompson, r. l., c. a. higgins, and j. m. howell. 1991.“personal computing: toward a conceptual model of utilization.”mis quarterly 15 (1): 125-143. tokuyoshi, b. 2013. the security implications of byod. network security 2013 (4): 12-13. triandis, h. c. 1989.“the self and social behavior in differing cultural contexts.” psychological review 96 (3): 506-520. triandis, h. c., r. bontempo, m. j. villareal, m. asai, and n. lucca.1988.“individualism and colectivism: cross-cultural perspectives and self in group relationshps.”journal of personality & social psychology 54 (2): 323-338. steelman, z.; lacity, m. and sabherwal, r. 2016. "charting your organization’s bring-yourown-device voyage." misq executive 15 (2): 85-104. venkatesh, v., m. morris, g. davis, and f. davis. 2003.“user acceptance of information technology: toward a unified view.”mis quarterly 27 (3): 425–478. venkatesh, v. and zhang, x. 2010. “unified theory of acceptance and use of technology: u.s. vs. china,” journal of global information technology management 13(1): 5-27. vogel, r., t. kocoglu, and t. berger. 2010. desktopvirtualisierung: definitionen architekturen business-nutzen, wiesbaden: vieweg verlag, friedr, & sohn verlagsgesellschaft mbh. wang, w., and nemati, h. (2016). understanding usage of bring your own device (byod): a complex adaptive systems perspective. proceeding of 22nd americas conference on information systems. weber, e. u., and c. hsee. 1998. “cross-cultural differences in risk perception, but crosscultural similarities in attitudes towards perceived risk.” management science 44 (9): 1205-1217. weeger, a., x. wang, and h. gewald. 2015. “it consumerization: byod-program acceptance and its impact on employer attractiveness.” journal of computer information systems 56(1): 1-10. weiß, f., and j. leimeister. 2012. “consumerization it innovations from the consumer market as a challenge for corporate it.”business & information systems engineering 54 (6): 363-366. australasian journal of information systems wang, weeger & gewald 2017, vol 21, research article employee participation in corporate byod programs 21 zhao, x., b. b. flynn, and a. v. roth. 2007. “decision sciences research in china: current status, opportunities, and propositions for research in supply chain management, logistics, and quality management,” decision science 38 (1): 39-80. copyright: © 2017 wang, weeger & gewald. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems wang, weeger & gewald 2017, vol 21, research article employee participation in corporate byod programs 22 appendix a: instruments abbr. definition source bi1 if a byod program is offered, i intend to use it. adapted from venkatesh et al. (2003) bi2 if a byod program is offered, i predict i would use it. bi3 if a byod program is offered, i plan to use it. pe1 participating in the program would increase my effectiveness on the job. davis (1989) pe2 participating in the program would increase my mobility on the job. self-designed pe3 participating in the program would increase my job motivation. adapted from venkatesh et al. (2003) pe4 participating in the program would increase my productivity on the job ee1 participating in the program would be too complicated; it is difficult to understand which handling leads to the desired work support. adapted from thompson et al. (1991) ee2 it would take too long to learn how to participate in the program to make it worth the effort. ee3 participating in the program would lead to the situation that i would often require additional help. self-designed si1 i predict that people in a company who participate in the program have more prestige than those who do not. adapted from moore and benbasat (1991) si2 i predict that people in a company who participate in the program have a high profile. si3 i predict that participating in the program is a status symbol for people in a company. si4 i predict that people in my peer group who participate in the program can thereby improve their image. self-designed bpt1 participating in a corporate “bring your own device” program increase the risk that i lose business data self-designed based on niehaves et al. (2012) bpt2 participating in a corporate “bring your own device” program increase the risk that i corrupt the corporate network with malware bpt3 participating in a corporate “bring your own device” program increase the risk that i violate company policies bpt4 participating in a corporate “bring your own device” program increase the risk that i am not able to work because of a service failure ppt1 participating in a corporate “bring your own device” program increase the risk that i lose private data ppt2 participating in a corporate “bring your own device” program increase the risk that too restrictive corporate policies limit the usage of my private device ppt3 participating in a corporate “bring your own device” program increase the risk that private data can be viewed by my company ppt4 participating in a corporate “bring your own device” program increase the risk that increasing workload force me to do business after hours table a.1: measurement instrument (all items were measured on 5-point likert scales.) paper title (paper title style) australasian journal of information systems filippou, cheong & cheong 2018, vol 22, research on educational technologies acceptance of gamification in education 1 a model to investigate preference for use of gamification in a learning activity justin filippou school of business it & logistics rmit university melbourne, australia justin.filippou@rmit.edu.au christopher cheong school of business it & logistics rmit university melbourne, australia france cheong school of business it & logistics rmit university melbourne, australia abstract applying the engaging and motivating aspects of video games in non-game contexts is known as gamification. education can benefit from gamification by improving the learning environment to make it more enjoyable and engaging for students. factors that influence students’ preference for use of gamification are identified. students are surveyed on their experiences of playing a gamified quiz, named quick quiz, during class. quick quiz features several gamification elements such as points, progress bars, leader boards, timers, and charts. data collected from the survey is analysed using partial least squares. factors including ‘usefulness’, ‘preference for use’, ‘knowledge improvement’, ‘engagement’, ‘immersion’ and ‘enjoyment’ were found to be significant determinants. students were found to have a preference for use for gamification in their learning environment. keywords gamification, student engagement, learning and teaching, higher education 1 introduction education is a cornerstone of a society that produces informed and intelligent people. as the demography becomes more diverse, the way in which learning and teaching is delivered needs to evolve with the changing demographic of students. an important aspect to consider in this change is ensuring that the learning environment is engaging. engagement as a construct consists of behaviour, emotion and cognition (fredricks, blumenfeld, & paris, 2004; lamborn, newmann, & wehlage, 1992). if students pay attention to class activities, are interested in the material and are willing to invest in their learning, it can be said they are engaged. there are various ways in which learning can be delivered in accordance with these constructs, and thus made more engaging, with one such approach being the use of games for educational purposes. when games are used as the vehicle for a learning activity, this is referred to as game-based learning (gbl) (prensky, 2001). the appealing aspect of games is that they are fun and enjoyable to use, thus, replicating these experiences for learning is expected to lead to improvements in student engagement. students also find games and simulations to be more interesting as compared to traditional class activities (randel, morris, australasian journal of information systems filippou, cheong & cheong 2018, vol 22, research on educational technologies acceptance of gamification in education 2 wetzel, & whitehill, 1992). currently, digital learning environments, which are the systems used to carry out learning activities, are struggling to maintain engagement in part due to low levels of interactivity (qut, 2014), which is cause for concern as lower levels of engagement can lead to reduced academic performance (kuh, cruce, shoup, kinzie, & gonyea, 2008). the types of games used in gbl can be divided into two categories: commercial and serious games (connolly, boyle, macarthur, hainey, & boyle, 2012). commercial games are those that are designed primarily for entertainment purposes. however, they are able to be used for learning when used in a controlled way. for example, minecraft is a game where players mine blocks of resources and use them to build structures. the game was built for fun, yet it can be used to teach students problem solving skills by giving them building challenges to solve (bos, wilder, cook, & o'donnell, 2014; risberg, 2015). alternatively, serious games are designed and built with the primary intention to educate albeit in an entertaining way (djaouti, alvarez, jessel, & rampnoux, 2011). for example, a game called “kernel panic” visually represents computer science concepts such as pointers and data manipulation as 3d objects which the player can interact with (muratet, torguet, jessel, & viallet, 2009). while using both categories of games for learning can be effective, there are some limitations. firstly, a commercial game may not always be easy to align with learning objectives. secondly, developing a serious game from scratch can be costly for most educators or even institutions. a third option can be used to overcome these limitations, which is called gamification. gamification is the process of using game elements and thinking and applying these aspects to traditionally non-game contexts (deterding, dixon, khaled, & nacke, 2011; kapp, 2012; zichermann & cunningham, 2011). one benefit to using gamification is that it does not require an implementation of a fully 3d environment to be effective. rather, only the essential aspects of what makes playing games enjoyable is required. gamified learning activities can also be constructed with learning in mind from the beginning and so it is possible to balance learning and fun more effectively than entertainment or serious games alone. while studies have investigated the use of gamification in the learning environment in general (hamari, koivisto, & sarsa, 2014), little empirical evaluation has been conducted that focuses on what factors contribute to students accepting a gamified learning activity. identifying these factors will help instructors make evidence-based choices in the design of both the physical and digital learning environments, and hence improve the likelihood of gamification being used successfully. the research presented henceforth answers the question: what factors impact students’ preference for gamification in learning activities? to answer this question, a model is developed that is tested against students’ use of an in-class gamified learning activity. while it anticipated that all students may benefit from gamification being applied to learning, this research will target undergraduate students as a first step towards understanding the factors that impact preference for use. the following section provides context for this research by examining key literature on education and gamification as well as technology acceptance models used as inspiration for the research. 2 background the purpose of this research is to understand the factors that contribute to students engaging with a learning activity using gamification. the following sections explore how gamification and education can be combined as well as models that currently exist to measure technology australasian journal of information systems filippou, cheong & cheong 2018, vol 22, research on educational technologies acceptance of gamification in education 3 acceptance. combined, these sections will provide guidance for the development of a factor model of educational gamification in this research. 2.1 gamification and education gamification refers to the use of game mechanics, aesthetics and thinking in non-game contexts to engage users and solve problems (deterding et al., 2011; kapp, 2012; zichermann & cunningham, 2011). although gamification uses game elements, the resulting application is not akin to a commercial game. the term is also confused as the application of entertainment or serious games, which is incorrect, as gamification is the process to create something that is game-like, but not an actual video game (werbach, 2014). it could be argued that education is already game-like in that students are given tasks to complete (assignments), receive points for completing them (assignment being marked), which are converted to badges (grades) that they can use to “level up” (graduate to the next year level) (lee & hammer, 2011). this implies that gamification may be well suited for the learning environment. however, for gamification to be used in a meaningful way in the classroom it must also make a connection to the underlying non-game elements of the class activities (nicholson, 2012), which are the learning objectives. one cannot simply add game elements such as badges, points and rewards to any aspect of a class activity and expect engagement and learning to be improved (kapp, 2012). one advantage of using gamification is being able to tailor implementations to specific needs, due to the modular nature of game elements. when implemented correctly and appropriately within the learning environment, gamification can enable a number of benefits. for instance, the stress of failure within the learning process can be eased as students can deal with the feedback of failure in a positive manner by not worrying about the consequences (such as failing an assessment piece) (simões, redondo, & vilas, 2012). that is, as the consequence of a student failing a task is low, there is a likelihood that the student may attempt to solve the problem again (gee, 2008), encouraging curiosity to learn. 2.2 measuring technology acceptance numerous models such as theory of reasoned action (tra), theory of planned behaviour (tpb), innovation diffusion theory (idt) and social cognitive theory (sct) have been developed to measure technology acceptance (ajzen, 1991; ajzen & fishbein, 1980; bandura, 1986; rodger, 1995). unifying previous models of technology acceptance, the unified theory of acceptance and use of technology (utaut) incorporates relevant aspects of each model and can be used to reliably measure user acceptance of an instance of technology. its intention is to aid managers in understanding whether new technologies being introduced in the workplace would be successfully adopted (venkatesh, morris, davis, & davis, 2003). the utaut model uses six key determinants for intention and usage of technology. these being: (1) performance expectancy, (2) effort expectancy, (3) social influence, (4) facilitating conditions, (5) behavioural intention and (6) use behaviour. the relationships between these are mediated by 4 other determinants: (1) gender, (2) age, (3) experience and (4) voluntariness of use (venkatesh et al., 2003). utaut is a capable framework however it is not suitable for this research. utaut does not feature any measurement items for fun and enjoyment, which is a crucial aspect to measuring the success of an implementation using gamification. some of the determinants and mediators will be adapted for use in the model created for this research, however the model as originally defined will not be used. australasian journal of information systems filippou, cheong & cheong 2018, vol 22, research on educational technologies acceptance of gamification in education 4 2.3 related work the purpose of gamification is to replicate the type of engagement and motivation people traditionally experience with video games in learning contexts. one study, conducted by bourgonjon, valcke, soetaert, and schellens (2010), attempted to model student acceptance of video games in a classroom setting. this was achieved by measuring the students’ preference for video games, from which acceptance could be derived. the study identified various factors that influence preference for video games. these included: usefulness, learning opportunities, ease of use, gender and experience. a limitation of this study was that respondents were not actually asked to use a particular video game in class. this research addresses this limitation by providing students with a game-like learning activity and then surveying them on the experience using the gamified learning tool. the tool selected does not need to be technically advanced as low complexity games are capable of still being interesting to students (fu, su, & yu, 2009). 3 conceptual model in this research, a model is developed to measure students’ preference for use of a gamified learning activity, identifying factors that influence students’ views on this approach. at a highlevel, the conceptual model combines two distinct concepts of fun and learning. the constructs included in fun are: enjoyment, ease of use, immersion, engagement, game experience and social interaction. as part of learning, constructs include: knowledge improvement and usefulness. the following sub-sections explore these constructs further and the relationships they have between one another, culminating in the construction of the conceptual model. 3.1 usefulness and preference for use (use and prf) the foundation of the model being tested in this research is the relationship between usefulness and preference for use. students desire their learning experiences to occur in a way that is relevant and useful to them (prensky, 2010) and so if they find a particular learning tool useful, they are more likely to have a preference for using it. the learning tool in this research is the gamified activity, which supports students to improve in tangible ways such as in learning performance, productivity, and grades. any implementation of gamification needs to ensure that it supports these aspects of student learning. as such, the first hypothesis is: h1. usefulness of a gamified activity positively affects preference for use. 3.2 knowledge improvement (kno) if a class activity that has been gamified is perceived by students to improve their knowledge, then this will contribute to whether they find the activity useful overall. one may argue that knowledge improvement and usefulness are one and the same as the primary purpose of education is to gain knowledge, hence, a learning tool can only be useful if it enables knowledge improvement. however, knowledge improvement and usefulness in this research measure different aspects. usefulness measures the learning tool itself while knowledge improvement measures student perception of whether their knowledge has improved or not when using the gamified tool. for example, students may believe that a gamified learning tool assists with the process of studying by considering it a vehicle for learning. however, they may not attribute any knowledge improvement from the use of the tool even if that is what occurred. lack of attribution in this scenario may be due to self-serving bias, where people are more likely to attribute successes to themselves (miller & ross, 1975). if students do believe australasian journal of information systems filippou, cheong & cheong 2018, vol 22, research on educational technologies acceptance of gamification in education 5 they have improved their knowledge from using the tool, they are likely to also think that the tool is useful as it helped them to achieve their learning goal. hence, the impact of knowledge improvement can be viewed as a determinant of usefulness. this concept can be hypothesised as: h2. knowledge improvement positively affects usefulness of a gamified activity. 3.3 engagement (eng) and immersion (imm) the terms engagement and immersion are often used synonymously, however, there is a difference between the two. when considered from the perspective of video game design, one can define three types of interactivity: immersion, engagement and presence (mcmahan, 2003). immersion occurs when players are caught up in a game’s story or are deeply involved in the strategy of playing the game. engagement occurs when players derive enjoyment from interacting with the game mechanics. presence involves feeling part of a game’s world and is typically used to describe virtual reality experiences. as such, presence will be excluded from this research as it is out of scope. linking the definitions of interactivity to a gamified learning context, immersion can involve students thinking deeply about how best to use the learning tool at a strategic level. engagement can involve students making a connection to the game mechanics and enjoying the experience of playing. this aligns well with the definition of student engagement provided earlier: engaged students pay attention, are interested in the material and are curious to learn more. arguably, the areas of attention, interest, and curiosity are the mechanics of learning, much like points, badges and leader boards are mechanics of games. this is important because students who are engaged in their studies devote more time to developing their skills which will be useful to them later in life (kuh, 2003), and this one of the goals of formal education. to promote immersion, the system should attempt to induce students into a state where they devote their entire attention to it in a way that brings them immense satisfaction. this experience is called “flow” (csikszentmihalyi, 1990). when in a state of flow there is a greater chance that learning can occur (muntean, 2011). in other words, when students are in a state of flow with a class activity, they are completely immersed in it. being in a state of flow is characterised by the following conditions: (1) the likelihood of completing the task must be relatively good, (2) the goals are clear and immediate feedback is provided, and (3) complete concentration can be given to the task. if a gamified learning task creates these conditions, it is reasonable to expect that the student finds the tool to be useful, as it enabled them to become completely absorbed in something enjoyable. in summary, being engaged in the mechanics of a gamified tool and immersing students in the strategy of playing means that engagement and immersion as separate constructs are determinants of whether students will find the tool to be useful. these can be hypothesised as: h3. engagement in a gamified activity positively affects its usefulness. h4. immersion in a gamified activity positively affects its usefulness. 3.4 enjoyment (enj) enjoyment of a class activity is important for learning to take place (frenzel, goetz, lüdtke, pekrun, & sutton, 2009). if students are having fun completing an activity, then they are more likely to be relaxed, leading to increased ability to learn (prensky, 2001). the concept of australasian journal of information systems filippou, cheong & cheong 2018, vol 22, research on educational technologies acceptance of gamification in education 6 enjoyment for a gamified learning activity stems from why people play games, and that is for the fun of play. it is sometimes difficult to replicate fun and play in learning environments as formal assessment is required to measure academic progress, and students can develop feelings of stress and anxiety when they are being tested (cassady & johnson, 2002). therefore, fun and enjoyment should be prioritised when formal assessment is not being conducted. for example, instructors may use pop culture references to make educational material more contextual so students find it more enjoyable to discuss (cheung, 2001). it is likely that enjoyment would only impact preference for use and not usefulness, as students may not enjoy using a tool but they may see its value in helping them to learn. however, if the tool is not enjoyable, even if it is useful, student preference for use would likely be impacted. this leads to the hypothesis: h5. enjoyment of a gamified activity positively affects preference for its use. 3.5 experience (exp) in a game-like environment, the relationship between enjoyment and preference for use is expected to be moderated by video game experience. this research will be using digital gamification that makes an experience more ‘game-like’, so it is reasonable to presume that students’ prior history of playing video games would impact their proclivity for such an experience. the impact of video game experience can be two-fold. firstly, students who play video games often may be more likely to enjoy using an educational class activity that is gamified as it more closely resembles what they enjoy doing in their leisure time. conversely, students who play video games regularly may find the game elements featured in class to be lacklustre and as such may negatively impact their feelings of enjoyment related to preference for use. students who do not play video games at all may not find the activity enjoyable due to unfamiliarity. for these reasons, video game experience is considered a moderating factor and can be hypothesised as: h6. video game experience moderates the impact enjoyment has on preference for use. 3.6 social interaction (si) one way to help improve student academic engagement is to also support student social engagement (coates, 2007). students are likely to expect some type of social element built into a system that is game-like, particularly given the popularity of text and voice chat in online games. however, social interaction can also be achieved by interacting in the physical world, similar to traditional split-screen multiplayer games where players interacted by sitting together on the same couch. being physically next to others would likely spur interaction and would create a moderating effect on the impact engagement, immersion and enjoyment have on usefulness and preference for use. therefore, the moderating effect of social interaction can be hypothesised in the following ways: h7. social interaction moderates the impact enjoyment has on preference for use. h8. social interaction moderates the impact immersion has on usefulness. h9. social interaction moderates the impact engagement has on usefulness. australasian journal of information systems filippou, cheong & cheong 2018, vol 22, research on educational technologies acceptance of gamification in education 7 3.7 ease of use (eou) ease of use plays a large role in moderating the relationships between the determinant factors in the model. if something is easy to use then this allows one to concentrate on using the system for its intended purpose, rather than focus their attention on how to operate it (krug, 2014). a learning tool that is difficult to use would distract students from their main task of learning course material, leading to a possible reduction in a student’s perceptions of knowledge improvement, engagement, immersion and enjoyment. for this reason, the impact of ease of use can be hypothesised in the following ways: h10. ease of use moderates the impact enjoyment has on preference for use. h11. ease of use moderates the impact immersion has on usefulness. h12. ease of use moderates the impact engagement has on usefulness. h13. ease of use moderates the impact knowledge improvement has on usefulness. 3.8 overall model in summary, having outlined the hypotheses, a model can be illustrated to represent all of the variables and their moderators, as can be seen in figure 1. figure 1: conceptual acceptance model 4 methodology as part of regular class activities, several courses from a business information systems degree used a gamified multiple-choice quiz called quick quiz (cheong, cheong, & filippou, 2013). in total, three courses were selected. each of these courses is primarily it-based in terms of the content taught. all three courses feature a weekly one-hour lecture as well as a two-hour tutorial. quick quiz was used by students on laptops during tutorials for a period of three to four weeks. the instructors for each course developed their own course-related questions and were not given any strict instructions on how to integrate the tool into the tutorial. the instruction was simply to do so in a manner that suited the class. 4.1 quick quiz quick quiz is designed to be a fun and flexible way for instructors to test students on their knowledge of a given class topic. the general gameplay involves short, timed quizzes that contain a number of questions in multiple-choice format. each quiz features 5 questions with each given 60 seconds as a time limit for students to answer. students join a quiz when the australasian journal of information systems filippou, cheong & cheong 2018, vol 22, research on educational technologies acceptance of gamification in education 8 instructor has activated it and immediately begin answering questions. quick quiz utilises the multiple-choice question format as it is commonly used and therefore the tool can be tailored for use in many different subject areas. this format also allows students to quickly submit responses as compared to requiring open response questions. a number of game elements are evident in quick quiz. for instance, quick quiz features a point system, where the quicker students answer a question, the more points they receive (within the range of 50 to 100 points). if students answer questions incorrectly, they are awarded a participation score of 20 points. no points are awarded to the student if no response is given to a question. immediate visual feedback is also conveyed to the student using progress bars indicating time remaining, as well as ticks and crosses immediately informing students on the result of their selected answer (see figure 2). at the conclusion of a game, a leader board is displayed that ranks all of the students based on the points they were awarded (see figure 3 – student names have been anonymised). figure 2: student has selected an incorrect answer figure 3: personalised leader board shown at the completion of a quiz a number of features are available to instructors to address the learning component of playing the quiz. once students have completed the quiz, instructors have a number of charts at their disposal containing class performance data. a histogram is available that graphs all answers submitted by students for each question. this is a particularly useful tool as it becomes clear which questions students understood and those there was difficulty in answering. this allows feedback to be provided to students to aid them in their learning, and also allows instructors australasian journal of information systems filippou, cheong & cheong 2018, vol 22, research on educational technologies acceptance of gamification in education 9 to identify common misconceptions among students, by highlighting questions where student responses were diverse. quick quiz was purpose-built to be a fun exercise for students while being useful for learning. with several game elements including points scoring, timers, progress bars and leader boards, combined with the game thinking of having instructors provide immediate feedback on the class’ performance in the quiz, quick quiz is suitably gamified to use as a base to measure preference for use of gamification in education. 4.2 instrument development a paper-based survey was used in this research paper based surveys are more likely to have higher response rates (hayslett & wildemuth, 2004). given that there are only three courses available as targets for this study, maximising the possibility of survey response is imperative. the questionnaire features two parts. part a enquires about the general demographic details of the participant. part b of the questionnaire contains items that directly measure the conceptual model constructs, which are adapted from previous research studies and are known to be reliable (see appendix a for the list of questions). although the original questionnaire items being sourced were not used for gamification, they are general enough that adaptation can be reasonably executed. respondents were asked to rate how much they either agreed or disagreed with each statement using a 5-point likert scale. the scale was structured as: (1) strongly disagree, (2) disagree, (3) neither agree nor disagree, (4) agree, (5) strongly agree. students were also provided with a participant information and consent form stating that completion of the survey is voluntary and that no student is identifiable. 4.2.1 pilot and instrument refinement to verify the survey instrument, a pilot study was performed at the end of the semester with students in the target courses. the purpose of this was to ensure that the questions being asked were being interpreted by respondents correctly and as such, would provide appropriate data. this was achieved by having students play quick quiz and answer questions on the questionnaire. responses were analysed by observing the responses given in the survey and identifying any patterns that were unexpected. in total, there were 11 participants, including: • 10 males and 1 female • 7 aged 18-21 and 4 were aged 22-28 • 6 local and 5 international students • 9 full-time and 2 part-time students upon analysing the responses to the questions only a small number of issues were identified. respondents were asked to provide the course they were undertaking. many respondents confused this with the program title and not the course title. to rectify this, check boxes were used and only provided options for the target courses. part b of the questionnaire contained question numbering issues however this did not impact the respondents’ ability to answer the questions. the data collected as part of the pilot was not used in the main data collection. 4.3 statistical design the conceptual model includes a number of relationships between various factors based on hypotheses presented earlier. as such, structural equation modelling (sem) was used to perform a confirmatory factor analysis (cfa) as it is suited to this type of the empirical data australasian journal of information systems filippou, cheong & cheong 2018, vol 22, research on educational technologies acceptance of gamification in education 10 (haenlein & kaplan, 2004). more specifically, the partial least squares (pls) regression technique was used. the factors in the model are latent variables, with an inner and outer structural model, which is a type suited to pls (hair, black, & babin, 2010) linear regression and multiple-linear regression cannot be used as the relationships cannot be isolated to measure the individual effects. smartpls software was used to derive the models (ringle, wende, & will, 2005). the reliability of the model was determined by the cronbach’s alpha values for each of the latent variables, which indicates internal consistency of the measurement items in a range between 0 and 1 (santos, 1999). a value closer to 1 suggests greater reliability of the instrument. values of 0.7 or greater are considered optimal; however values between 0.5 and 0.7 are acceptable. any value below 0.35 is rejected (hair et al., 2010). finally, the data was cleaned using a four-step process: (1) identification if missing data is non-random and therefore ignorable, (2) determining impact of missing data to final results, (3) determining how random the missing data is and (4) applying a remedy (hair et al., 2010). missing data was handled by using the ‘complete case approach’ imputation method that involves only using complete and valid observations. 5 results the survey was distributed to a total of 119 enrolled students with 84 students completing and returning their survey. this represented an initial response rate of 70.59% from the available students. however, not all surveys returned were usable with 8 surveys being excluded from the usable data set as they were missing a significant number of answers and a complete case approach was being used. following the data cleaning there were a total of 76 usable observations, representing 63.86% of students from the population of classes sampled. 5.1 descriptive statistics the data collected is representative of the overall population of students in the target classes as most characteristics from the sample fall within one per cent of the population percentage. gaming demographic details are unavailable for the population and so a comparison is not possible. table 1 provides an overview of the demographic details. characteristics population sample count % count % gender male 97 81.51% 62 81.58% female 22 18.49% 14 18.42% age group 18 – 21 81 68.07% 55 72.37% 22 – 28 33 27.73% 20 26.31% 29 – 48 5 4.20% 1 1.32% 49 – 65 0 0% 0 0% > 65 0 0% 0 0% mode of study full time 113 94.96% 71 93.42% part time 6 5.04% 5 6.58% student type local 96 80.67% 61 80.26% international 23 19.33% 15 19.74% table 1: descriptive statistics australasian journal of information systems filippou, cheong & cheong 2018, vol 22, research on educational technologies acceptance of gamification in education 11 results from part a of the survey (demographic) revealed that the students typically play computer games on a very frequent basis with a total of 65.8% playing computer games either daily or weekly. the three most common reasons they play games were: ‘to play with others’, ‘boredom’ and ‘mental challenge’. mental challenge being a typical reason to play games is encouraging as it suggests that gamification may likely have higher preference for use as it can be used to promote the mental challenges presented to students through their class activities in a game-like manner. 59.3% of respondents had previously played educational games in class. although the survey did not enquire about whether students found those experiences to be positive or negative, it did ask about students’ interest in potentially learning with a game for future use. 63.2% of respondents were interested in learning with a game, so regardless of previous gaming experience, students generally are quite positive about the idea. this could mean that these students would be more likely to have a preference for use of a gamified system, given that they are generally interested in the idea of gaming. overall, it appears that the sample collected was representative of the population, and that this particular sample of students reports regularly playing video games which is important to provide context for analysing the results of the sem-pls analysis. 5.2 structural equation modelling results in the initial model output it was evident that a number of issues were present. firstly, cfa analysis identified a number of factor loadings for each of the measurement items as being quite low. a value greater than 0.7 is considered a reliable measure of that item, and any value less than 0.5 should be considered for removal (hair et al., 2010). some measurement items had values less than 0.5 and many of the moderators also exhibited low factor loadings. the results also indicated several instances of multicollinearity. analysing the t-statistics of the model uncovered a number of paths that did not have statistical significance, which needed to be addressed. it was evident that the conceptual model needed revision. several factors were however found to be reliable and so it was decided to restructure the model based on these, as well as re-consulting the literature. it is generally recommended that the number of samples should be proportional to the number of measurement items by a factor of 10 (flynn & pearcy, 2001). 76 samples were collected for this study, and based on the ratio just mentioned, the study would require at least 480 observations (48 items × 10). therefore, the revised model takes the smaller sample size into consideration by including fewer factors and measurement items. 5.3 revised conceptual model the model was revised over several iterations to ensure it accurately measured the necessary factors while improving the statistical significance of the overall model. this was achieved by reconsidering each factor and measurement item against how it applied to the use of quick quiz. the original model was designed to cover gamification implementations generally and so some factors may not be relevant to the particular implementation used as in this study. measurement items were also reviewed and items that did not match with the use of quick quiz were removed. some items were deemed to be unnecessary as quick quiz did not directly implement certain game features and hence these aspects could not be measured. by carrying out this process, it was expected that a more relevant and simpler model would lead to improved results. australasian journal of information systems filippou, cheong & cheong 2018, vol 22, research on educational technologies acceptance of gamification in education 12 factors that were removed from the model included: ‘gaming experience’, ‘social interaction’, ‘ease of use’, ‘gender’, ‘age’ as well as all of the moderators. this was firstly due to the low path coefficients these factors produced in the initial tests, and also as a result of the sample demographic data being homogenous. the sample contained respondents who played games regularly, were predominately male, and typically were within the range of 18-21 years old. therefore, video game experience and ease of use were presumed to not have a large impact on respondents and added unnecessary complexity to the model. this was also applicable to gender and age for the same reasons. social interaction was removed as quick quiz does not feature any component that enables this within the tool. although the original hypothesis discussion earlier listed physical proximity as a means for conducing social interaction, observations of the instructors indicated that this did not occur to significant levels during the game. several individual measurement items of the remaining factors were also removed to limit the total number of items further. the final conceptual model that was derived from this process can be found in figure 4. figure 4: revised conceptual model using the revised conceptual model, both the pls algorithm and the bootstrapping algorithm were run once more. the output of the pls algorithm can be found in figure 5 while the bootstrapping algorithm output can be found in figure 6. it can be observed from these outputs that a much-improved model was the result. all but 2 measurement items have a factor loading of 0.7 or greater, indicating very reliable measures of the factors. the remaining 2 items are both greater than 0.5 and while not as strong, can still be considered adequate. excluding enjoyment, each of the path coefficients had values greater than 0.2 indicating each had a direct positive impact on either usefulness or preference for use. enjoyment failed to produce a significant value, however its role in determining preference for use was considered vital and was retained. the amount of variance explained for usefulness dropped in this model, although remained at a significant value of 64.3%. variance in preference for use improved marginally to 48.9% variance explained. australasian journal of information systems filippou, cheong & cheong 2018, vol 22, research on educational technologies acceptance of gamification in education 13 figure 5: path coefficients figure 6: t-statistics 5.4 revised conceptual model reliability and validity an issue with the original model tested was the amount of multicollinearity among the measurement items. as can be deduced from table 2, almost all of the measurement items only show strong factor loadings for the factors they represented. the only exception to this is use1 and use2 which also had significant loadings for kno. however, the loadings these items represented for usefulness were very strong with loadings of 0.95 and 0.96 respectively, whereas loadings for kno were 0.70 for each, marginally over the cut-off. therefore, these values were considered acceptable for this study. australasian journal of information systems filippou, cheong & cheong 2018, vol 22, research on educational technologies acceptance of gamification in education 14 eng enj imm kno prf use eng1 0.646 0.4559 0.2647 0.4722 0.5334 0.3622 eng4 0.8455 0.3784 0.3917 0.5138 0.6117 0.5977 eng6 0.8081 0.3925 0.675 0.578 0.5091 0.6004 eng7 0.7746 0.4602 0.4813 0.3727 0.4254 0.508 enj3 0.5385 0.9289 0.4473 0.3922 0.4405 0.3735 enj6 0.2234 0.6 0.249 0.211 0.2039 0.3229 imm1 0.4434 0.3052 0.7557 0.3679 0.3351 0.4714 imm2 0.3946 0.308 0.8122 0.2495 0.2994 0.4199 imm3 0.4712 0.4721 0.8611 0.3748 0.373 0.5003 imm4 0.4994 0.3839 0.853 0.4144 0.3401 0.5546 imm5 0.5986 0.3452 0.7728 0.5622 0.5174 0.5977 imm6 0.3885 0.3783 0.6242 0.2962 0.346 0.3803 kno1 0.5189 0.4881 0.3571 0.854 0.6787 0.5653 kno2 0.507 0.3859 0.3154 0.7467 0.4292 0.4479 kno3 0.4967 0.2729 0.4416 0.8719 0.5905 0.6979 kno5 0.577 0.2464 0.5174 0.85 0.5785 0.6109 prf1 0.5463 0.469 0.4176 0.6213 0.8855 0.5867 prf2 0.6541 0.4181 0.4031 0.6938 0.9226 0.6447 prf3 0.5648 0.274 0.4519 0.4969 0.8374 0.564 use1 0.6897 0.4089 0.6106 0.7021 0.6434 0.9592 use2 0.6685 0.4165 0.6032 0.7029 0.6614 0.9666 use3 0.6439 0.4361 0.6372 0.6823 0.6134 0.9654 use4 0.6079 0.3932 0.569 0.6203 0.6754 0.9262 table 2: final model cross loading (full data) the values of each factor’s cronbach’s alpha can be found in table 3 and are all very strong for each factor, excluding enj which has a weaker value. although the cronbach’s alpha value for enj was weaker, it was still over the cut-off of 0.35 and so can remain in the final model. in addition to analysing the cronbach’s alpha value, the convergent validity and discriminant validity were also examined. convergent validity refers to the amount of variance the items of a construct share. an average variance extracted (ave) of at least 0.5 and greater is adequate for convergent validity to exist (fornell & larcker, 1981). table 3 outlines the ave for each factor in the model. each factor has an ave greater than 0.5 which confirmed discriminant validity. ave composite reliability r² cronbachs alpha communality redunda ncy eng 0.5963 0.8541 0 0.7733 0.5963 0 enj 0.6114 0.7505 0 0.4141 0.6114 0 imm 0.6144 0.9044 0 0.8723 0.6144 0 kno 0.6924 0.8997 0 0.8524 0.6924 0 prf 0.7788 0.9134 0.4887 0.8575 0.7788 0.0998 use 0.9111 0.9762 0.6426 0.9674 0.9111 0.2648 table 3: reliability measurements australasian journal of information systems filippou, cheong & cheong 2018, vol 22, research on educational technologies acceptance of gamification in education 15 the amount in which a construct is truly distinct from other constructs is known as discriminant validity. the square root of the ave for each construct should be greater than the other factor correlations for discriminant validity to be present (chin, 1998). table 4 outlines the correlations of the latent variables; from which, it can be deduced that the square root of the ave is greater than correlations between the other factors. eng enj imm kno prf use eng 1 enj 0.532 1 imm 0.6055 0.4662 1 kno 0.627 0.406 0.4974 1 prf 0.6679 0.4433 0.4781 0.6896 1 use 0.6841 0.4334 0.6339 0.7097 0.6793 1 table 4: latent variable correlation as a final step in presenting the results, the conceptual model hypotheses are evaluated. the null hypothesis is rejected if there was a path coefficient greater than 0.2, t-statistic greater than 2.0, and the cronbach’s alpha was 0.35 or greater. hypotheses with values in unacceptable ranges result in the null hypothesis failing to be rejected. table 5 outlines the results of each hypothesis tested. hypothesis finding h1 usefulness of a gamified activity positively affects preference for use. reject null hypothesis. h2 knowledge improvement positively affects usefulness of a gamified activity. reject null hypothesis. h3 engagement in a gamified activity positively affects its usefulness. reject null hypothesis. h4 immersion in a gamified activity positively affects its usefulness. reject null hypothesis. h5 enjoyment of a gamified activity positively affects preference for its use. reject null hypothesis. table 5: summary of hypotheses 6 findings and discussion this section provides a discussion of the results from two perspectives: (1) the reasons why some of the factors were retained and others were not, and (2) how the use of quick quiz may have influenced the results for this study. 6.1 factors the strong impact knowledge improvement had on usefulness confirms that students want their learning should be relevant (prensky, 2001). the results showed that with every increase in the perception of knowledge improvement, usefulness rose by 0.41. the indirect impact of knowledge improvement on preference for use can also be measured by multiplying the path coefficients. this leads to a coefficient of 0.25 (0.41 × 0.60), which is also notable. this significant relationship demonstrates that an important aspect to gamification design for learning tools is to prioritise student support for enhancing knowledge. australasian journal of information systems filippou, cheong & cheong 2018, vol 22, research on educational technologies acceptance of gamification in education 16 engagement and immersion were found to be significant factors, however the impact of each individually was not as strong as knowledge improvement. for every increase in engagement and immersion, there was an increase in usefulness of 0.26 and 0.27 respectively. however, as both factors are derived from interactivity in general, they can be added to better understand the effects on usefulness, resulting in a coefficient of 0.53 (0.263 + 0.271). the combined effect immersion and engagement have on usefulness is greater than that of knowledge improvement. this suggests that students may have stronger preference for use if they feel engaged and immersed in a gamified system rather than if the learning tool simply provides them with knowledge. although the factors can be combined, their implementation should be carried out as separate aspects. engagement is created by giving students a reason to invest their time and effort in using a system they believe will be for their benefit. alternatively, immersion is about absorbing students in the strategy of playing and ideally inducing a state of flow (csikszentmihalyi, 1990) which alters students’ perception of the environment around them while using the system. this allows deep concentration to occur, leading to a greater likelihood for learning to take place. hence, it is unlikely a single feature will achieve both goals and is likely to require a combination of game features and the adjustment of the physical environment where the game is taking place. the final factor of enjoyment had the weakest impact in the model. for every increase in enjoyment, preference for use only rose 0.183. however, with every combined increase in usefulness and enjoyment, preference for use rose 0.783 (0.600 + 0.183) which indicates a very strong relationship. this suggests that overall, two factors that are involved in students having a preference to use a gamified system is their perceived levels of enjoyment and usefulness, which reflects the aim of using gamification in education. that is, to make education fun and to better support the learning process. it is possible that students found playing a gamified quiz to be enjoyable due to the lack of consequences as a result of its use not being part of assessed coursework. this may have improved enjoyment, however it may have also weakened the value proposition of using quick quiz, in that there was no consequence to using it. therefore, as happy as students were to play, they would not be unhappy if it was not used. 6.2 use of quick quiz it is also important to note that quick quiz did not contain a vast array of game-like features. features such as points scoring, leader boards, and charts are some of the more prominent features used and yet several factors were identified as strong indicators of preference for use of a gamified learning tool. this may be a result of how quick quiz was used in class, rather than how its game-like features were implemented. at a point in time during the class (typically midway) instructors signalled to students that a round of quick quiz was to begin and so each student joined in the session online. as a first step, this has the effect of ‘setting the scene’ so that students switch their mindset from ‘study’ to ‘gameplay’ mode. when games commenced, instructors often provided running commentary as the leader board filled with students completing the quiz. at the conclusion of the round, instructors discussed the results with students in a conversational manner, allowing opportunity for students to explain why they selected their answers. when combined, the in-class experience presumably felt “gamelike” to students, hence why they were able to report on their preference for use. a finding that can be extrapolated from this scenario is that the importance of game mechanics in gamified learning tools are secondary to how the gamified experience is presented to students. australasian journal of information systems filippou, cheong & cheong 2018, vol 22, research on educational technologies acceptance of gamification in education 17 7 implications for research and pedagogy a number of factors were validated in this research regarding student preference for use of a gamified learning activity. the factors highlight some important aspects for educators to consider when using gamified learning tools. it is evident that knowledge improvement is a significant determinant of preference for using a gamified system, so educators should place emphasis on this aspect as a priority when designing or selecting a gamified system. in quick quiz, knowledge improvement was achieved through instructors conducting class discussions and feedback at the conclusion of a round. this is an example of how educators play an important role in creating engagement in a gamified system by being a trigger. instructors in this instance improved the experience of playing quick quiz by displaying a leader board to students and making encouraging comments throughout the game, thereby improving enjoyment of its use. the implication is that instructors and the physical learning environment play a significant role in ensuring a gamified learning tool is successful, and that software alone may not be as effective at creating an experience that is both fun and helps students learn simultaneously. 8 limitations there were a number of limitations in the research. firstly, the research was exploratory in nature and so the results are not directly generalisable. sample size availability also restricted the possible complexity of the conceptual model being tested. to validate a model of such complexity, the sample size is recommended to be proportional to the number of measurement items by a factor of 10 (flynn & pearcy, 2001). due to the limited number of courses quick quiz was used in, the population of potential respondents was less than the minimum suggested for statistical significance of the analysis. hence, the entire original model was unable to be tested with an appropriate amount of data observations. quick quiz was used solely in it-based undergraduate courses. hence, the demographic of the collected sample was homogenous limiting the possibility of deeper analysis from taking place. furthermore, quick quiz was trialled for only 3 to 4 weeks during a semester. this means that students were only exposed to the gamified quiz for a very limited timeframe, and so long-term perceptions of students using a gamified system could not be determined. 8.1 suggestions for future research given the limitations of this study, there is room for future research to be conducted to further validate student preference for use of gamified learning tools. firstly, the target population and sample size collected was relatively small, so re-conducting the study in a larger population of students may enable the more complex model to be further explored and examined. students from different fields of study should also be surveyed. this will allow researchers to identify whether the factors identified in this study apply to a broad range of students, with differing skillsets and academic interests. the possibility of a unified set of factors or alternatively, different factors for specific demographics of students could potentially be developed as a result of this. research can be conducted on comparing gamification preference for use between undergraduate and post-graduate students, evaluating whether enrolment in higher degrees moderate students willingness to engage in a learning activity that may be thought of as less serious. incorporating quick quiz into a teaching approach may be another avenue for further research. more broadly, investigating ways that gamified learning tools such as quick quiz australasian journal of information systems filippou, cheong & cheong 2018, vol 22, research on educational technologies acceptance of gamification in education 18 could be formally incorporated into teaching approaches may be of interest to researchers. although quick quiz was used as part of class activities, it was considered as a supplement to the established curriculum, rather than being an integral part of it. students also selfreported that knowledge improvement occurred as part of using the gamified quiz, however a study investigating whether this actually occurs would likely have an impact on pedagogical research. 9 conclusion the results of this research indicate that students have a preference for use of a gamified educational tool. the model constructed identified factors that were found to be reliable contributors to student preference for use, including: ‘usefulness’, ‘enjoyment’, ‘knowledge improvement’, ‘engagement’ and ‘immersion’. the factors of ‘gaming experience’, ‘social interaction’, ‘ease of use’, ‘gender’, ‘age’, were hypothesised but could not be reliably tested due to a restricted sample of data. it was found that a large set of game-like features was not required to determine preference for use. rather, the experience of using a gamified learning tool in-class based on how instructors facilitate use is more important. further work needs to be conducted to measure whether the results of this research are consistent across students of varying disciplines. the implications of measuring preference for use of gamification in education help to set a foundation for broader implementations of gamification in learning and teaching practice. acknowledgements this research project was approved by the rmit university business college human ethics advisory network (bchean project number 1000389). the authors would like to thank the participants for their involvement in this study and the reviewers for providing insightful feedback. references ajzen, i. (1991). the theory of planned behavior. organizational behavior and human decision processes, 50(2), 179-211. ajzen, i., & fishbein, m. (1980). understanding attitudes and predicting social behaviour. bandura, a. (1986). social foundations of thought and action. englewood cliffs, nj. bos, b., wilder, l., cook, m., & o'donnell, r. (2014). learning mathematics through minecraft. teaching children mathematics, 21(1), 56-59. bourgonjon, j., valcke, m., soetaert, r., & schellens, t. (2010). students’ perceptions about the use of video games in the classroom. computers & education, 54(4), 1145-1156. cassady, j. c., & johnson, r. e. (2002). cognitive test anxiety and academic performance. contemporary educational psychology, 27(2), 270-295. cheong, c., cheong, f., & filippou, j. (2013). quick quiz: a gamified approach for enhancing learning. paper presented at the pacific asia conference on information systems, jeju island, korea. cheung, c.-k. (2001). the use of popular culture as a stimulus to motivate secondary students' english learning in hong kong. elt journal, 55(1), 55-61. australasian journal of information systems filippou, cheong & cheong 2018, vol 22, research on educational technologies acceptance of gamification in education 19 chin, w. w. (1998). commentary: issues and opinion on structural equation modeling. mis quarterly, vii-xvi. coates, h. (2007). a model of online and general campus-based student engagement. assessment & evaluation in higher education, 32(2), 121-141. connolly, t. m., boyle, e. a., macarthur, e., hainey, t., & boyle, j. m. (2012). a systematic literature review of empirical evidence on computer games and serious games. computers & education, 59(2), 661-686. csikszentmihalyi, m. (1990). the psychology of optimal experience. new york: harper & row. deterding, s., dixon, d., khaled, r., & nacke, l. (2011). from game design elements to gamefulness: defining gamification. paper presented at the proceedings of the 15th international academic mindtrek conference: envisioning future media environments. djaouti, d., alvarez, j., jessel, j.-p., & rampnoux, o. (2011). origins of serious games serious games and edutainment applications (pp. 25-43): springer. flynn, l. r., & pearcy, d. (2001). four subtle sins in scale development: some suggestions for strengthening the current paradigm. international journal of market research, 43(4), 409424. fornell, c., & larcker, d. f. (1981). evaluating structural equation models with unobservable variables and measurement error. journal of marketing research, 39-50. fredricks, j. a., blumenfeld, p. c., & paris, a. h. (2004). school engagement: potential of the concept, state of the evidence. review of educational research, 74(1), 59-109. frenzel, a. c., goetz, t., lüdtke, o., pekrun, r., & sutton, r. e. (2009). emotional transmission in the classroom: exploring the relationship between teacher and student enjoyment. journal of educational psychology, 101(3), 705. fu, f.-l., su, r.-c., & yu, s.-c. (2009). egameflow: a scale to measure learners’ enjoyment of e-learning games. computers & education, 52(1), 101-112. gee, j. p. (2008). learning and games. the ecology of games: connecting youth, games, and learning, 3, 21-40. haenlein, m., & kaplan, a. m. (2004). a beginner's guide to partial least squares analysis. understanding statistics, 3(4), 283-297. hair, j. f., black, w. c., & babin, b. j. (2010). multivariate data analysis: a global perspective: pearson education, limited. hamari, j., koivisto, j., & sarsa, h. (2014). does gamification work?--a literature review of empirical studies on gamification. paper presented at the system sciences (hicss), 2014 47th hawaii international conference on. hayslett, m. m., & wildemuth, b. m. (2004). pixels or pencils? the relative effectiveness of web-based versus paper surveys. library & information science research, 26(1), 73-93. kapp, k. m. (2012). the gamification of learning and instruction: game-based methods and strategies for training and education. san francisco, ca: wiley. australasian journal of information systems filippou, cheong & cheong 2018, vol 22, research on educational technologies acceptance of gamification in education 20 krug, s. (2014). don't make me thing revisited: a common sense approach to web and mobile usability. usa: new riders. kuh, g. d. (2003). what we're learning about student engagement from nsse: benchmarks for effective educational practices. change: the magazine of higher learning, 35(2), 24-32. kuh, g. d., cruce, t. m., shoup, r., kinzie, j., & gonyea, r. m. (2008). unmasking the effects of student engagement on first-year college grades and persistence. the journal of higher education, 79(5), 540-563. lamborn, s., newmann, f., & wehlage, g. (1992). the significance and sources of student engagement. student engagement and achievement in american secondary schools, 11-39. lee, j. j., & hammer, j. (2011). gamification in education: what, how, why bother? academic exchange quarterly, 15(2), 146. mcmahan, a. (2003). immersion, engagement and presence. the video game theory reader, 67, 86. miller, d. t., & ross, m. (1975). self-serving biases in the attribution of causality: fact or fiction. psychological bulletin, 82(2), 213-225. muntean, c. i. (2011). raising engagement in e-learning through gamification. paper presented at the proc. 6th international conference on virtual learning icvl. muratet, m., torguet, p., jessel, j.-p., & viallet, f. (2009). towards a serious game to help students learn computer programming. international journal of computer games technology, 2009, 3. nicholson, s. (2012). a user-centered theoretical framework for meaningful gamification. proceedings gls, 8. prensky, m. (2001). digital game-based learning: mcgraw-hill. prensky, m. (2010). teaching digital natives: partnering for real learning: sage publications. qut. (2014). review of qut’s virtual learning environment (vle). retrieved from http://www.els.qut.edu.au/innovation/vler/docs/vlereview_report_fin_20140623.pd f randel, j. m., morris, b. a., wetzel, c. d., & whitehill, b. v. (1992). the effectiveness of games for educational purposes: a review of recent research. simulation & gaming, 23(3), 261276. ringle, c., wende, s., & will, a. (2005). smartpls. hamburg, germany: university of hamburg. risberg, c. (2015). more than just a video game: tips for using minecraft to personalize the curriculum and promote creativity, collaboration, and problem solving. illinois association for gifted children journal, 44-48. rodger, e. (1995). diffusion of innovations: new york: free press. santos, j. r. a. (1999). cronbach’s alpha: a tool for assessing the reliability of scales. journal of extension, 37(2), 1-5. australasian journal of information systems filippou, cheong & cheong 2018, vol 22, research on educational technologies acceptance of gamification in education 21 simões, j., redondo, r. d., & vilas, a. f. (2012). a social gamification framework for a k-6 learning platform. computers in human behavior. venkatesh, v., morris, m. g., davis, g. b., & davis, f. d. (2003). user acceptance of information technology: toward a unified view. mis quarterly, 425-478. werbach, k. (2014). (re) defining gamification: a process approach. paper presented at the international conference on persuasive technology. zichermann, g., & cunningham, c. (2011). gamification by design: implementing game mechanics in web and mobile apps. sebastopol, ca: o'reilly media. australasian journal of information systems filippou, cheong & cheong 2018, vol 22, research on educational technologies acceptance of gamification in education 22 appendix a: items used to measure acceptance construct item preference for use (adapted from bourgonjon et al. 2010) if i had the choice, i would choose to follow courses in which quick quiz was used. if i had to vote, i would vote in favour of using quick quiz in the classroom. i am enthusiastic about using quick quiz in the classroom. usefulness (adapted from bourgonjon et al. 2010) playing quick quiz improves my learning performance playing quick quiz increases my learning productivity playing quick quiz enhances my learning effectiveness playing quick quiz helps me to achieve better grades knowledge improvement (adapted from fu et al. 2009) the game increases my knowledge i catch the basic ideas of the knowledge taught i try to apply the knowledge in the game the game motivates the player to integrate the knowledge taught i want to know more about the knowledge taught engagement (adapted from whitton 2007) i wanted to complete the quiz. i wanted to explore all of the options available to me. i did not care how the quiz ended. i found the quiz satisfying. i felt absorbed in the quiz. i felt that time passed quickly. i felt excited during the quiz. immersion (adapted from fu et al. 2009) i forget about time passing while playing the game i become unaware of my surroundings while playing the game i temporarily forget worries about everyday life while playing the game i experience an altered sense of time i can become involved in the game i feel emotionally involved in the game enjoyment (adapted from feng et al. 2008) i feel unhappy when playing quick quiz. i feel worried when playing quick quiz. i feel happy when playing quick quiz. i feel exhausted when playing quick quiz. i feel miserable when playing quick quiz. i talk to myself when playing quick quiz. i make loud comments even if nobody is around when playing quick quiz. social interaction (adapted from fu et al. 2009) i felt cooperative toward other classmates i strongly collaborated with other classmates the cooperation in quick quiz is helpful to my learning quick quiz supports a social interaction between the players quick quiz supports communities within the game quick quiz supports communities outside the game experience (adapted from bourgonjon et al. 2010) i like playing video games. i often play video games. compared to people of my age, i play a lot of video games. i would describe myself as a gamer. i play different types of video games. ease of use i knew how to handle quick quiz in the classroom it was easy for me to use quick quiz the classroom australasian journal of information systems filippou, cheong & cheong 2018, vol 22, research on educational technologies acceptance of gamification in education 23 (adapted from bourgonjon et al. 2010) my interaction with quick quiz in the classroom was clear and understandable copyright: © 2018 filippou, cheong & cheong. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 background 2.1 gamification and education 2.2 measuring technology acceptance 2.3 related work 3 conceptual model 3.1 usefulness and preference for use (use and prf) 3.2 knowledge improvement (kno) 3.3 engagement (eng) and immersion (imm) 3.4 enjoyment (enj) 3.5 experience (exp) 3.6 social interaction (si) 3.7 ease of use (eou) 3.8 overall model 4 methodology 4.1 quick quiz 4.2 instrument development 4.2.1 pilot and instrument refinement 4.3 statistical design 5 results 5.1 descriptive statistics 5.2 structural equation modelling results 5.3 revised conceptual model 5.4 revised conceptual model reliability and validity 6 findings and discussion 6.1 factors 6.2 use of quick quiz 7 implications for research and pedagogy 8 limitations 8.1 suggestions for future research 9 conclusion paper title (paper title style) australasian journal of information systems blake, kerr & gammack 2016, vol 20, design research for dss development development of sleep disorder dss using design science 1 development of a dss and online tools to support sleep disorder consultations using design science. jacqueline blake school of business university of the sunshine coast jblake@usc.edu.au don kerr school of business university of the sunshine coast john gammack college of technological innovation zayed university abstract medical decision making takes place in a complex and dynamic environment, involving various individual and organisational stakeholders. in such contexts supportive information systems are required to be designed for practical efficacy, emphasising transparency, usability, and usefulness to their primary users. at the same time, they need to demonstrate improved efficiency and diagnostic accuracy. in addition, they also need to provide effective communication of relevant information. this paper describes the use of design science to develop and evaluate a dss incorporated into online tools to support sleep physician’s diagnosis of sleep disorders. the design science approach offers guidance for developing this solution, that ensure the artefacts fit into their usage context, and that development and evaluation principles that generalise to similar problem domains are identified. how this project fits in the design science guidelines is specified, thus demonstrating how this philosophy and methodology advances theory particularly relevant to other similar medical diagnostic domains. the usability and technical evaluation of the set of tools indicated provide clear practical benefits at patient, physician and organisational levels. keywords: decision support system; sleep disorder; medical consultation; diagnosis; usability 1 introduction this paper describes designing an information system artefact in the environment of medical consultation. medical consultation environments combine strict guidelines from biomedical theory and professional practice with discretionary elements sensitive to the individual’s situation. there are various models for medical consultation which have chronologically evolved from “doctor as engineer” towards more patient-centric approaches (pawlikowska et al. 2007), but generally embody essentially complex, two-way interactions involving history taking, problem solving and treatment or management decisions, with effective communication, information sharing and mutual understanding as central aspects. consultation for sleep disorders is representative of other specialised medical consultations in regard to the professional processes involved, and the requirement to communicate and understand relevant information. this paper uses the case of sleep consultation to describe the development of relevant support for this process. sleep disorders are still under-observed by primary physicians (bailes et al. 2008) and despite increased awareness of sleep disorders among physicians and the wider community there is a shortage of sleep disorder specialists in australia (australian medical workforce advisory committee 2000, hillman & lack 2013), but also in the usa (rogers & valerio 2011) and elsewhere, producing increasing burden on national health care systems (cooper, 2013), sleep disorders cost australia 5.1bn per year, and considerably more if productivity losses due to poor sleep habits are included (hillman & lack australasian journal of information systems blake, kerr & gammack 2016, vol 20, design research for dss development development of sleep disorder dss using design science 2 2013). the sleep community is therefore interested in changing the traditional model of care for sleep disorders to one that leverages information systems to provide efficiencies and timely communication flows (colten & alterogt 2006). health informatics is defined as “a combination of computer science, information science and health science designed to assist in the management and processing of data, information and knowledge to support healthcare and healthcare delivery (conrick 2006, p. 4). health informatics now forms part of the infrastructure for the delivery of healthcare with one of the major fields of research being decision support (haux 2010). health informatics allows the timely retrieval and filtering of patient and disorder information so that caregivers have appropriate information in a quickly assimilated format within their workflow. hirakis & karakounos (2006) and haux (2010) state that knowledge management in healthcare allows the development of best practice models, making guidelines explicit and encourages innovation through development of the resultant knowledge base. this allows clinicians access to updated knowledge, but also helps skilled caregivers, or the patients themselves, to understand the rationale to manage treatments. for effective knowledge management, information shared should, ideally, be captured in the flow of work practice, so that it does not impose an overhead on rekeying, communication and storage. in addition, for effective knowledge sharing, an agreed vocabulary is essential to avoid misunderstanding, and collaborative ontologies with controlled vocabularies are becoming established in the biomedical sector (simperl & luczak-rosch 2014). our research develops information tools to support both the patient and physician in the diagnosis of sleep disorders. following the design science research (dsr) approach, the artefact described in this paper aimed to improve communication and workflow, by redesigning a complex manual process that previously relied upon input from sleep physicians and sleep clinic staff to process data (hevner et. al 2004). the rest of the paper proceeds as follows, we first briefly review background literature to demonstrate the need for such tools, particularly in the area of sleep consultation and outline the case context and its requirements. we then review the dsr methodology and in detail demonstrate its application to develop and evaluate an it artefact for the illustrative case. in section 4 we discuss the findings and make clear the knowledge contribution. finally, we draw conclusions, discuss limitations and generalisation, and avenues for further research. approximately six per cent of the australian population have a sleep disorder but only 20 per cent of those affected by sleep disorders are diagnosed (access economics pty limited 2004), other countries report similar statistics (young, peppard & gottlieb 2002) with the people who are referred for treatment being strongly symptomatic (kramer et al. 1999). in recent times within australia there is an average wait for an initial consultation with a sleep physician of nine weeks with a further average wait of 21 weeks for a sleep study to diagnose a sleep disorder (flemons et al. 2004). while the average wait time for an initial consultation is six months in the united kingdom, belgium one to three weeks and united states of america two to twelve weeks. these wait times demonstrate a gap between demand for sleep disorder assistance and the ability of the most health services to provide for the current level of people referred to sleep clinics. this research project looked at the development of a system to support the diagnosis of sleep disorders. the requirements were unclear and the subject matter experts who were sleep specialists had limited time to provide guidance and feedback, so there was a need for a rigorous framework to ensure the developed artefact was of an acceptable quality as well as producing an application which would work in the environment and be useful and usable. the online sleep tools which incorporated a decision support system were developed to improve communication flows between the physician and patient. a qualitative approach was used, acknowledging the complex and dynamic medical socio-technical environment. the requirements for the online applications were captured using convergent interviews with physicians and allied health professionals who were working in the sleep disorder environment. observations were also made of sleep physicians during patient–physician consultations (with the patient’s permission). through a process of iterative development, australasian journal of information systems blake, kerr & gammack 2016, vol 20, design research for dss development development of sleep disorder dss using design science 3 prototypes of each application were built, using the feedback from health professionals. the extant literature was used to focus on the current sleep disorder context, and the theory drawn from this literature applied in a cyclical way to the sleep disorder clinic under study. design science research provided a framework which met these requirements. as stated by arnott and pervan (2014) design science research has become a popular method for developing decision support systems, this paper provides a case study of a design science project which resulted in the output of an it artefact which was evaluated as usable and useful by users. 2 design science design science is a problem solving approach aimed at changing an existing environment to one which better reflects current aims (boland 2002). walls, widmeyer and el sawy (2004, p. 45) describe the purpose of design theory as being “to guide artefact creation”. specifically, in information systems, design science creates and evaluates information technology artefacts designed as a solution to a problem (hevner et al. 2004). walls, widmeyer and el sawy (1992) differentiate design theory from natural and social science research by stating that in design theory research “the achievement of goals” is fundamental (walls, widmeyer & sawy 1992, p. 40). this contrasts with natural and social science theory which seeks to explain or predict phenomena. they further state that design theory puts explanatory, predictive and normative theory into practical use by designing an artefact to meet a goal. this may seem to contradict the critical realist stance adopted in this project; however, carlsson (2006) argues that critical realism is an appropriate stance for design science research because, although the goal is to produce an it artefact which will work for a class of problems, that it artefact must be evaluated within the socio-technical environment that forms its context. this context and social science theory guide the process of forming the requirements for an it artefact. the goal of a design science research project is the successful application of the designed object to make required changes in an environment. walls, widmeyer and el sawy (1992) based their paper upon simon’s (1969) book which argued the need for a formal methodology to govern design, and applied this approach to the information systems field. walls, widmeyer and el sawy (1992) start a formal definition of design theory by stating that “design is both a product and a process” (walls, widmeyer & sawy 1992, p. 42) and consequently, that the use of design theory methodologies is as important as the evaluation of the designed artefact itself. they state that the process of producing the artefact requires three elements: • a design method that describes how the artefact was constructed • a kernel theory from the natural or social sciences so that the design process is driven by extant process theory • a testable design process hypothesis, to evaluate whether the designed artefact meets the standards of its design class. further, the designed product must meet four requirements: • meta-requirements, which provide an understanding of the system requirements that are necessary in a solution to the identified problem • meta-design, which is knowledge of the types of artefacts shown to provide a solution in this problem domain • a kernel theory, as the artefact must be informed by the extant theory • a testable design product hypothesis, which is a plan for evaluating whether the designed artefact meets the requirements. australasian journal of information systems blake, kerr & gammack 2016, vol 20, design research for dss development development of sleep disorder dss using design science 4 the walls, widmeyer and el sawy (1992) diagram of the relationship between these requirements is shown in figure 1. this diagram was unchanged in a later paper critiquing the 1992 paper (walls, widmeyer & sawy 2004). figure 1 components of an information system design theory. (source: walls, widmeyer & el sawy, 1992). markus, majchrzak and gasser (2002, p. 180) state that the contribution of walls et al. (1992) serves to “articulate the boundaries within which particular design assumptions apply”. this means that the contribution was for a rigorous research method based upon theory for a class of problem when the projects resulted in an artefact designed to introduce change in an environment. march and smith (1995) also draw upon simon’s (1969) work in discussing design science but do not reference walls, widmeyer and el sawy (1992). they state that “rather than producing general theoretical knowledge, design scientists produce and apply knowledge of tasks or situations in order to create effective artifacts.” (march & smith 1995, p. 253). this implies that the focus of the process is on designing relevant practical artifacts “that serve human purposes” (march & smith 1995, p. 253) at the expense of building design theory. later authors have agreed with walls, widmeyer and el sawy (2004). these include gregor and jones (2007), kuechler and vaishnavi (2008), hevner et al. (2004) and venable (2006). however, baskerville (2008, p. 442) refers to the theory produced by design science as “theory discovery” where the theory is a by-product of the process of constructing an environmentchanging artefact. further, winter (2008, p. 472) refers to theory as an “intermediate artefact” that needs to be included in the list of artefacts which result from design research. he also states that to be classed as design research, the process and product of the research must retain some generalisibility to that class of problem domains. this is related to the walls, widmeyer and el sawy (1992) meta-requirements and meta-design in their formal definition of design theory. 3 methodology the design science approach used in this project was that proposed by hevner et al. (2004), using the klein and myers (1999) method. this approach has been acknowledged by venable kernel theories meta-requirements meta-design testable design product hypothesis testable design process hypothesis design method kernel theories australasian journal of information systems blake, kerr & gammack 2016, vol 20, design research for dss development development of sleep disorder dss using design science 5 (2006) as being accepted by the information systems community, although it lacks a clear statement about the role of theory. the hevner et al. (2004) guidelines in table 1 outline a framework for performing design science research. these requirements differentiate a research approach to developing an information technology artefact from an approach taken in routine system development. guidelines description 1: design as an artefact science research must produce a viable artefact in the form of a construct, a model, a method or an instantiation. 2: problem relevance the objective of design-science research is to develop technology-based solutions to important and relevant business problems. 3: design evaluation the utility, quality and efficacy of a design artefact must be rigorously demonstrated via well-executed evaluation methods. 4: research contributions effective design-science research must provide clear and verifiable contributions in the areas of the design artefact, design foundations, and/or design methodologies. 5: research rigor design-science research relies upon the application of rigorous methods in both the construction and evaluation of the design artefact. 6: design as a search process the search for an effective artefact requires utilizing available means to reach desired ends while satisfying laws in the problem environment. 7: communication of research design-science research must be presented effectively to both technology-oriented and management-oriented audiences. table 1: design science research guidelines, drawn from hevner, march, park and ram (2004). the guidelines shown in table 1 relate to this project as follows. 3.1 design as an artefact an information technology artefact consisting of a decision support system and input applications to support the diagnosis of sleep disorders was planned in this project. a definition of an information technology artefact is “the configuration of hardware and software, including user interfaces and algorithms intended to accomplish specific information processing of functions” (chiasson et al. 2007, p. s90). according to orlikowski and lacono (2001), information technology artefacts had historically not been a focus as part of information technology research, but had been treated as static objects. orlikowski and lacono (2001) surveyed the extant literature and found that there were five main categories of how the information technology artefacts had been perceived. they identified a “tool view”, which comprised four elements: a labour substitution tool, a productivity tool, an information processing tool and a social relations tool. the tool view sees an artefact as a static “thing” which carries out a required activity independently of its context. the artefact is considered to be a constant in the research with that which is “affected, altered or transformed” by the tool being the variable of interest (orlikowski & lacono 2001, p. 123). the four tools are described below. • labour substitution tool were the artefact is viewed as something that will replace human labour, allowing downsizing of the labour force to take place. • productivity tool were the artefact replaces older methods for carrying out work, thus increasing worker productivity by enabling flexible, collaborative work methods. australasian journal of information systems blake, kerr & gammack 2016, vol 20, design research for dss development development of sleep disorder dss using design science 6 • information processing tool: this is the view that technology is capable of altering and improving the way that workers process information, thus enabling it to better flow through an organisation. • social relations tool where technology acts to alter the way and mode of communicating within both an organisation and a social network. in the sleep study project, the development of an information technology artefact within the context of a public health clinic was viewed as both an information processing tool and a social relations tool. the evaluation of the online tools demonstrated that the tools provided a viable solution to the sleep disorder consultation domain. 3.2 problem relevance the use of decision support systems for providing evidence-based medicine has been described as one of the most important current and future research areas in health informatics (haux 2010). decision support systems have already been used successfully for diagnostic support (lin, hu & sheng 2006) with physicians shown to value decision support (dreiseitl & binder 2005). the relevance of this problem is that sleep disorders, as discussed in previous section are becoming an increasing burden to the health system. therefore, decreasing the health burden by increasing the quality of communication between a physician and patient, increasing a patient’s self-knowledge and increasing the rigour of the diagnostic process provides a key, in part, to this important problem. therefore, the problem addressed by this research provides a technology based solution to an important organisational problem within the sleep clinic and provides benefits to the wider community. 3.3 design evaluation in line with design science requirements, standard, well documented methods based on the extant literature have been used in the evaluation of the information system artefacts in this project. the sleep tool web application is intended to be used first, by patients with potentially limited computer literacy, and then by physicians who will use the application for diagnostic purposes. it was therefore essential to ensure that the application was usable. the international standards organisation (iso) defines usability in standard 9241–11 as "extent to which a product can be used by specified users to achieve specified goals effectively, efficiently and with satisfaction in a specified context of use" (zikmund 2003). this evaluation investigates whether both patients and physicians could use and understand the application to an acceptable standard. three types of users – end users, two sleep physicians and a software engineer – helped to assess the various tools produced in this research. all three types of users had the software tools demonstrated and used the tools to assess usability. the software engineer was given access to the sleep tool web application and was asked to provide his comments on the usability of the application as he completed all sections. the research objectives of this evaluation were for a professional software engineer to comment on the technical usability and design aspects of the sleep tools web application. the objective of the physicians’ investigation was to ensure that the sleep tool web application had value in diagnosing sleep disorders, and met the sleep physicians' needs that they had detailed during the phase of requirements discovery. the objectives of the end user evaluation were to ensure that the sleep tools web application was robust and that it was reliable and usable (blake, kerr & gammack 2016). hence the evaluation of the it artefact for utility, quality and efficacy was undertaken and satisfactory results produced using the application of usual usability testing methods. 3.4 research contribution the information technology artefact developed in this project provides clear and verifiable contributions to the support of consistent clinical diagnosis of sleep disorders. another australasian journal of information systems blake, kerr & gammack 2016, vol 20, design research for dss development development of sleep disorder dss using design science 7 contribution is the demonstration of improvements in the operational performance of the sleep clinic by the application of a set of online tools to the consultation interaction. the output from the sleep tools web application allows an efficient, consistent triage process before a physician sees a patient for the first time. the operational implication of this is that a patient with a high risk of sleep apnoea can be sent to have a polysomnography before a consultation with a sleep physician. this acts to shorten the diagnostic interaction by at least one consultation, thus freeing the physician to see more new patients and reducing the cost of treating each of these patients. another operational benefit of removing patient history taking from the consultation is the possibility of the physician being able to shorten the duration of the consultation. this means that the physician is then free to carry out other tasks or to see more patients in the same length of time. as a shortage of physicians trained in the sleep speciality already exists and, as dement (2008) suggests an aging population will exacerbate that shortage, then finding time gains and efficiencies during the consultation is important for patient wait times and the cost of treating the disorder. the sleep tools web application automatically saves the patient history and sleep diary data in a database. this database of aggregated patient history data may then form a valuable resource for researchers, as it will have the facility to export unidentified data in an extensible markup language (xml) format. this aggregated information is not currently available within this health service as hard-copy patient records are still used. 3.5 research rigor the framework for research quality proposed by healy and perry (2000) was used to ensure research rigor and quality in line with the design science approach. this framework was chosen because the critical realism qualitative approach used in the sleep study fitted the requirements of this framework. there are six quality criteria in this framework are detailed in the table below: criterion project analytic the project looked at a socio-technical environment, with a complex interaction of people and technology. therefore, this project was appropriately world three with abstract things that exist separate from people (healy & perry 2000, p. 120) contingent validity the resultant applications from the research were tested within the context of the research. multiple perceptions involving triangulation more than one physician and siu were included in the research. more than one method of data collection was used with an interviewee given the opportunity to correct notes after interpretation by the researcher. methodological trustworthiness a rigorous method of data collection was used with a documented chain of evidence, so that quotations could be used. analytic generalisation this project will add to work on the use of online tools in an ehealth environment. construct validity the construct validity of the project was tested by conducting pilot projects to test the validity of the tools. the maintenance of a chain of evidence and triangulation also aided compliance with this criterion. table 2: research quality criteria (adapted from healy & perry 2000). 3.6 design as a search process: a mixed method approach was used in this project, acknowledging the complex and dynamic medical socio-technical environment. the foundations of the sleep disorder process and australasian journal of information systems blake, kerr & gammack 2016, vol 20, design research for dss development development of sleep disorder dss using design science 8 requirements for the web applications were uncovered using a search process based upon qualitative methods and the existing literature. methods used were convergent interviews with physicians and allied health professionals who were working in the sleep disorder environment. observations were also made of sleep physicians during patient–physician consultations (with the patient’s permission). through a process of iterative development, prototypes of each application were built, using the feedback from health professionals. the extant literature was used to focus on the current sleep disorder context, and the theory drawn from this literature applied in a cyclical way to the sleep disorder clinic under study. this frame of reference made the project suitable for a design science project using a hermeneutic approach, with the outputs from the interviews and observations forming the text under study. the klein and myers’ (1999) hermeneutic framework was used to inform the hevner et al. (2004) design science research guidelines for the development of an it artefact which changes an environment to better reflect current aims (boland 2002). these guidelines were followed in the production of the it artefact that resulted from this project. klein and myers (1999) see the hermeneutic circle as the fundamental or meta-principle of hermeneutics. the hermeneutic circle is the action of studying components of the system for understanding, and looking at their interactions with the system as a whole (klein & myers 1999, trauth & jessup 2000). for example, in this study the context of sleep medicine was researched, followed by a search for the requirements for an online sleep diagnosis tool. following the development of the sleep tool web application, the context of the prince charles sleep investigation unit was reviewed to see how the introduction of this tool affected the relationships between users and technology. it is during this period of critical reflection on the problem domain that the research processes are formalised; at this point a deliberate quest for understanding takes place: the results from the cycle are written up and critiqued, and the tacit knowledge gained from this part of the study is made explicit (dick 1998). klein and myers (1999) describe this undertaking as “extending in concentric circles the unity of the understood meaning”. the hermeneutic cycle was also employed as part of this study by proceeding from the segment currently under study to the extant literature in the current problem domain. 3.7 communication of research the results of this research have been promulgated to the information systems community by publishing in journals (blake & kerr 2010, 2014; blake, kerr & gammack 2016). communication of the research within the sleep community has been undertaken at conferences (blake & kerr 2007) and in person at the sleep clinics under study. 4 discussion design science provided the foundation for the successful completion of this project by ensuring that the research activities and processes were based on a design method, kernel theory and testable design process hypothesis. one of the physician concerns was that the diagnosis and treatment of sleep disorder patients needed to be consistent across the sleep investigation unit. the reports from the sleep tools web application provide a logical protocol for providing a diagnosis that is based on industry standard criteria, the international classification of sleep disorders (american academy of sleep medicine 2005). the objective at the start of the research project was to provide the sleep physicians with a time-efficient, consistent diagnostic protocol. this project has endeavoured to do this by removing most of the responsibility for completing standard information gathering questionnaires from the sleep physicians to the patient. the sleep physician has access to reports which summarise and present the information from the online patient applications. the sleep diary report does this by applying programming logic to the patient’s data, by presenting the sleep physician with a graphical representation of the sleep patterns, and by calculating the required sleep statistics from the patient’s data. this enables the australasian journal of information systems blake, kerr & gammack 2016, vol 20, design research for dss development development of sleep disorder dss using design science 9 physician to access the sleep diary information without spending time making manual calculations. the output from the patient history questionnaire is funnelled through a decision support system. the patient history data is separated into sections with the patient demographic and familial information presented in a clear, easy-to-understand format for the physician. data that is specific to the sleep disorder is separated into disorder-specific then disorder-criteriaexplicit sectors, then processed through a series of small argument trees. this follows the literature specifically that of stranieri et al. (1999), who used this method for discretionary decision making in the judicial system based on the theory contained in toulmin (1958) book on the uses of argument. this method has worked in the sleep tools web application, with the argument trees presenting a clear explanation on the physician’s report about why a patient has met a criterion for a sleep disorder. 5 conclusions by having patients record their details before their initial consultation, the online tools assist the sleep physician to keep the consultation patient-driven rather than form-driven. this means that the patient arrives at the consultation more prepared, the online tools having helping to demonstrate some of the factors that impact on sleep. additionally, the sleep physician has more time during the consultation to move beyond making a diagnosis and into a personalised discussion about the treatment plan. therefore, the online tools developed using the design science framework help to support the physicians’ diagnosis by providing reports that present patient information in a quickly assimilated format, give an objective second opinion of a diagnosis, provide a standard protocol for making the diagnosis based on international sleep disorder classifications and improve patient and physician communication during a consultation. while this project was situated in and produced a product for the particular situation of a sleep clinic, the success of this artefact was predicated on an understanding of the requirements of a health environment, a knowledge of the types of artefacts shown to provide a solution and an extent theory proven to work in other domains. these features fit the requirements for design science. therefore, the design science framework shaped an it artefact that met the physicians concerns and needs. references access economics pty limited 2004, “wake up australia: the value of healthy sleep”, available at: https://www.sleep.org.au/documents/item/69 [accessed 17 january 2007] american academy of sleep medicine 2005, “international classification of sleep disorders 2nd ed”, westchester, illinois: diagnostic and coding manual. arnott, d & pervan, gp 2014, “a critical analysis of decision support systems research revisited: the rise of design science”, journal of information technology, vol. 29, no. 4, pp. 269-93. australian medical workforce advisory, c 2000, “the specialist thoracic medicine workforce in australia; supply and requirements 1999-2010”, australian medical workforce advisory committee, amwac, north sydney. bailes, s, baltzan, m, rizzo, d, fichten, cs, amsel, r & libman, e 2008, “a diagnostic symptom profile for sleep disorder in primary care patients”, journal of psychosomatic research, vol. 64, no. 4, pp. 427-33, sciencedirect. baskerville, r 2008, “what design science is not”, european journal of information systems: special issue on design science research, vol. 17, no. 5, pp. 441-3, proquest. australasian journal of information systems blake, kerr & gammack 2016, vol 20, design research for dss development development of sleep disorder dss using design science 10 blake, j & kerr, d 2007, “implementing an online sleep diary to better manage sleep disorder diagnosis”, in world sleep 07: proceedings of the world sleep 07 cairns, australia september 2-6. blake, j & kerr, d 2010, “development of an online sleep diary for physician and patient use”, knowledge management & e-learning: an international journal, special issue on ehealth: accessing knowledge for global health, vol. 2, no. 2. blake, j & kerr, d 2014, “sleep disorder diagnosis: an analytical approach”, decision analytics, vol. 1, no. 7. blake, jn, kerr, dv & gammack, jg 2016, “streamlining patient consultations for sleep disorders with a knowledge-based cdss”, information systems, vol. 56, pp. 109-19. boland, rj 2002, “design in the punctuation of management action, managing as design: creating a vocabulary for management education and research”, available at: http://design.case.edu/2002workshop/positions.html. [accessed 07 december 2009]. carlsson, sa 2006, 'design science research in information systems: a critical realist perspective', in 17th australasian conference on information systems design science research in is: proceedings of the17th australasian conference on information systems design science research in is adelaide. chiasson, m, reddyb, m, kaplanc, b & davidson, e 2007, 'expanding multi-disciplinary approaches to healthcare information technologies: what does information systems offer medical informatics?', international journal of medical informatics, vol. 76, no. supplement 1, pp. s89-s97. colten, hr & alterogt, mm 2006, sleep disorders and sleep deprivation: an unmet public health problem, national academies press, available at: http://books.nap.edu/catalog.php?record_id=11617#toc. [accessed 29 december 2008]. conrick, m 2006, health informatics: transforming healthcare with technology, thomson social science press, melbourne. dement, wc 2008, 'history of sleep medicine', sleep medicine clinics, vol. 3, no. 2, pp. 147256. dick, b 1998, “cyclic critical reflection”, available at: http://www.scu.edu.au/schools/gcm/ar/arm/op009.html [accessed 30 october 2015] deiseitl, s & binder, m 2005, “do physicians value decision support? a look at the effect of decision support systems on physician opinion”, artificial intelligence in medicine, vol. 33, no. 1, pp. 25-30. flemons, ww, douglas, nj, kuna, st, rodenstein, do & wheatley, j 2004, “access to diagnosis and treatment of patients with suspected sleep apnea”, american journal of respiratory and critical care medicine, vol. 169, pp. 668-72. gregor, s & jones, d 2007, “the anatomy of a design theory”, journal of the association for information systems, vol. 8, no. 5, pp. 313-35, viewed 09 june 2010. haux, r 2010, “medical informatics: past, present, future”, international journal of medical informatics, vol. 79, no. 9. healy, m & perry, c 2000, “comprehensive criteria to judge validity and reliability of qualitative research within the realism paradigm”, qualitative market research: an international journal, vol. 3, no. 3, pp. 118-26. hevner, ar, march, st, park, j & ram, s 2004, “design science in information systems research”, mis quarterly, vol. 28, no. 1, pp. 75-105. australasian journal of information systems blake, kerr & gammack 2016, vol 20, design research for dss development development of sleep disorder dss using design science 11 hillman, dr & lack, lc 2013, “public health implications of sleep loss: the community burden”, the medical journal of australia, vol. 199, no. 8, pp. 57-60. hillman, dr, murphy, as & pezzullo, l 2006, “the economic cost of sleep disorders”, sleep, vol. 29, no. 3, pp. 299-305. hirakis, o & karakounos, s 2006, “goals and benefits of knowledge management in healthcare”, in aa lazakidou (ed.), handbook of research on informatics in healthcare and biomedicine, idea group reference, pp. 193-200. klein, hk & myers, md 1999, “a set of principles for conducting and evaluating interpretive field studies in information systems”, mis quarterly, vol. 23, no. 1, pp. 67-93. kramer, nr, cook, te, carlisle, cc, corwin, rw & millman, rp 1999, “the role of the primary care physician in recognizing obstructive sleep apnea”, archives of internal medicine, vol. 159, no. 9, pp. 965-8. kuechler, b & vaishnavi, v 2008, “on theory development in design science research: anatomy of a research project”, european journal of information systems: special issue on design science research, vol. 17, no. 5, pp. 489-504, viewed 01 june 2010. lin, l, hu, pj-h & sheng, orl 2006, “a decision support system for lower back pain diagnosis: uncertainty management and clinical evaluations”, decision support systems, vol. 42, no. 2, pp. 1152-69. march, st & smith, gf 1995, “design and natural science research on information technology”, decision support systems, vol. 15, no. 4, pp. 251-66. markus, ml, majchrzak, a & gasser, l 2002, “a design theory for systems that support emergent knowledge processes”, mis quarterly, vol. 26, no. 3, pp. 179-212. orlikowski, w & lacono, sc 2001, “research commentary: desperately seeking the 'it' in it research--a call to theorizing the it artefact”, information systems research, vol. 12, no. june, pp. 121-34. patlak, m 2005, “tips to help you detect common sleep disorders”, american college of physicians, available at, http://www.acponline.org/clinical_information/journals_publications/acp_internist/ mar05/sleep.htm#pool [accessed 20 june 2016] pawlikowska, t, leach, j, lavallee, p, charlton, r & piercy, j 2007, “consultation models”, in r charlton (ed.), learning to consult, oxford, radcliffe rogers, ae & valerio, td 2011, “the role of advanced practice nurses (aprns) in specialized sleep practice”, in ns redeker & gp mcenany (eds), sleep disorders and sleep promotion in nursing practice, pringer publishing company llc, new york, pp. 371-8. simon, ha 1969, “the sciences of the artificial”, m.i.t. press, cambridge, massachusetts. simperl, e & luczak-rosch, m 2014, “collaborative ontology engineering: a survey”, the knowledge engineering review, vol. 29, no. 1, pp. 101-31. stranieri, a, zeleznikow, j, gawler, m & lewis, b 1999, “a hybrid rule–neural approach for the automation of legal reasoning in the discretionary domain of family law in australia”, artificial intelligence and law, vol. 7, no. 2-3, pp. 153-83. toulmin, se 1958, “the uses of argument”, cambridge university press. trauth, em & jessup, lm 2000, “understanding computer-mediated discussions: positivist and interpretive analyses of group support system use”, mis quarterly, vol. 24, no. 1, pp. 43-79. venable, jr 2006, “the role of theory and theorising in design science research”, in first international conference on design science research in information systems and technology: proceedings of the first international conference on design science australasian journal of information systems blake, kerr & gammack 2016, vol 20, design research for dss development development of sleep disorder dss using design science 12 research in information systems and technology, a hevner & s chatterjee (eds.), claremont, california, pp. 1-18. walls, j, widmeyer, g & sawy, e 1992, “building an information system design theory for vigilant eis”, information systems research, vol. 3, no. 1, pp. 36-59. walls, j, widmeyer, g & sawy, e 2004, “assessing information system design theory in perspective: how useful was our 1992 initial rendition”, journal of information technology theory and application, vol. 6, no. 2, pp. 43-58. winter, r 2008, “design science research in europe”, european journal of information systems vol. 17, no. 5, pp. 470-5. young, t, peppard, pe & gottlieb, dj 2002, “epidemiology of obstructive sleep apnea: a population health perspective”, american journal respiratory critical care medicine, vol. 165, no. 9, pp. 1217-39. zikmund, wg 2003, 7th edn, “business research methods”, thomson south-western, mason united states of america. copyright: © 2016 blake, kerr & gammack. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 design science 3 methodology 3.1 design as an artefact 3.2 problem relevance 3.3 design evaluation 3.4 research contribution 3.5 research rigor 3.6 design as a search process: 3.7 communication of research 4 discussion 5 conclusions paper title (paper title style) australasian journal of information systems stevens, guo & li 2018, vol 22, research on educational technologies students’ motivations to use technology in learning 1 typology and hierarchy of students’ motivations to use technology in learning kenneth j stevens university of new south wales k.stevens@unsw.edu.au zixiu guo university of new south wales yuan li hebei university of technology china abstract considerable discussion has taken place in practice and academe regarding the need for changes to the educational system to better suit current student’s approaches and preferences for technology use in learning. much of this discussion involves assumptions about the current students (referred to by some as ‘digital natives’) preference for independent learning and that students are motivated in similar ways to use technology to achieve and support their preferred learning style. this study sought to better understand student’s motivations for technology use in learning and whether assumptions about the homogeneity of motivations are warranted. we sought to identify students’ motivation typology and any groupings within these typologies, and understand the inter-relationship between motivations. using data collected from 16 information systems (is) students via the repertory grid interview technique (rgt), a cluster analysis segmented respondents into two distinct groups: ‘independent learners’ and ‘traditional learners’. a hierarchical framework of technology use motivations was developed for each group using interpretive structural modelling (ism) and cross-impact matrix multiplication applied to classification (micmac) was used to categorise each group’s motivation factors. results show that the two groups were driven to achieve the same learning goals by different paths and hence questioning the assumption of homogeneity in technology use motivations among the current student cohort. keywords: digital technology enabled learning; motivations; digital natives; typology; hierarchical framework. 1 introduction worldwide, educational systems have undergone revolutionary changes at a meteoric rate, with widespread implementation and adoption of advanced new technologies, changing curriculum, but also so-called transformation of today’s student study habits (brown, 2005). growing up and surrounded by digital technologies, today’s students are seen by some commentators as having fundamental differences to previous generations of students and have been given new labels, such as digital natives (prensky, 2001) or net generations (tapscott, 1998). these students are seen to have different thinking and learning styles, preferring experimental and discovery learning (prensky, 2001). the difference between these students and their predecessors is perceived as sufficiently significant and has given rise to calls for changes to the education system to accommodate the needs of this new cohort of learners (helsper & eynon, 2010; prensky, 2001; selwyn, 2009; tapscott, 1999). the widespread australasian journal of information systems stevens, guo & li 2018, vol 22, research on educational technologies students’ motivations to use technology in learning 2 adoption of web 2.0 technologies in learning environments has added strength to these calls as the learning and information processing capabilities of today’s students are considered to have been transformed by web 2.0 technologies (selwyn, 2009) and it is argued by some that “the old approach [of didactic teaching] is ill-suited to the intellectual, social, motivational, and emotional needs of the new generation” (tapscott, 1998, p.131). although superficially compelling and persuasive, the claimed need for fundamental change to educational systems has come under quite some critical scrutiny (bennett & maton, 2010; bennett, maton, & kervin, 2008; lai & hong, 2015; parkes, stein, & reading, 2015; smith et al., 2012). the notions of a homogenous generation with technical expertise and a distinctive learning style have been challenged in regard to both the lack of empirical support and theoretical justification (bennett & maton, 2010; lai & hong). over the last decade, a growing body of evidence provides a very mixed picture with heterogeneous views in terms of students access to various technologies, the technology-based activities they engaged in, their technological skills, their learning characteristics, and even the needs that were fulfilled by use of the technologies in learning (bennett & maton, 2010; bennett et al., 2008; kennedy, judd, churchward, gray, & krause, 2008). as literature has indicated that human behaviour is affected by various factors, such as gender, culture, motivations, it is problematic to claim that today’s generation has homogeneous characteristics in technology use (lai & hong, 2015). parkes et al. (2015) found that students were not well prepared for the university e-learning environment, in particular in skills required for discovery-based learning, such as analysing and evaluating, which were claimed by digital native proponents to be the key learning styles of today’s digital native students (lai & hong, 2015). at present a number of pressing questions remain unanswered about digital natives and the use of technology in learning. in particular, two research gaps have been identified. first, there has been limited academic attention on investigating the existence of a discernible typology for digital natives in terms of their motivations for technology use in learning. vodanovich et al. (2010) called for empirical research to identify and differentiate the various segments of technology users with a view to better understand the profiles of different types of digital natives. such research would address the concern that, no matter how technologically competent today’s generation is, there is no guarantee that they will have the same motivation for using technologies in the learning context as they do in other contexts, since effective learning is not only impacted by their technological skills, but also by their learning approaches and preferences (bennett et al., 2008; jonassen, hernandez-serrano, & choi, 2000). understanding the profiles of different groups of digital natives would assist educators in incorporating technologies in teaching in a more responsive manner. it may help minimise the resistance of some learners towards using technologies in learning and encourage collaborative and innovative learning. in addition, these profiles would also help organisations understand the technology use and learning motivations of their future workforce, since these digital natives may bring their technology use preferences into the workplace (vodanovich et al., 2010). this understanding is crucial to better comprehend their attitudes, perceptions, and patterns of technology use, and it is only through developing an understanding of these factors that educators and managers will be able to develop strategies and tactics to engage digital natives’ in learning via technologies. second, if we expect that there might be heterogeneity among students in terms of their motivations for using technologies in learning, it is important to understand the hierarchical australasian journal of information systems stevens, guo & li 2018, vol 22, research on educational technologies students’ motivations to use technology in learning 3 structure of the factors that influence their use of those technologies for each group of the students, considering people’s needs are hierarchical in nature (maslow, 1943). although research in psychology, organisational behaviour, and consumer behaviour have long recognised and investigated the hierarchical nature of human motivation (e.g.,bagozzi, bergami, & leone, 2003; pieters, baumgartner, & allen, 1995; wagner, 2007), this phenomenon has been widely overlooked in the literature regarding human communication and technology use behaviour (guo, lu, li, & li, 2011). the construction of these factors into an inter-related framework for each group is useful for illustrating the different drivers which influence the way students use technologies in learning (ledbetter, 2009; roberts, hann, & slaughter, 2006). such information would allow us to assess the relative importance of the factors and their direct and indirect hierarchical relationship so that we might predict the influence of each factor (hasan, shankar, & sarkis, 2007). such an inter-relationship framework may serve as a guide for taking appropriate actions to motivate each group of students to use technologies more responsively and effectively in their learning. the framework will also allow us to categorise factors based on their driver power and dependence power, enabling us to focus on the most significant factors influencing learning. as reynolds and gutman (1984, p.30) stated, “the lack of a model reflecting the relational linkages tends to make the interpretation highly subjective.” overall, this study is designed to fill these two research gaps and to advance our understanding of today’s student technology use motivations by investigating the following three specific questions: • are there different groups of students in regard to their motivations for using technologies in learning? • if such ‘student technology use typology’ exists, what are inter-relationships among these motivations for each group to use technology in learning? • what is the relative importance of each motivation in achieving each group students’ learning goals? an empirical study was undertaken to (1) identify a typology of students based on their reasons for using technologies in learning, (2) develop a hierarchical structural framework of those factors for each group of students identified, and (3) classify those factors based on their relative importance. to achieve these goals, the reasons why students use technologies in learning were identified through interviews of 16 technically competent students. using these technology use perceptions in learning, a cluster analysis was conducted to segment the cohort into two distinct groups. interpretive structural modelling (ism) technique (sage, 1977; warfield, 1974) was used to structure each of these group’s technology use hierarchy, and the micmac (a french acronym for “matrice d’impacts goises – multiplication appliqués a un classment”, meaning “cross-impact matrix – multiplication) (duperrin & godet, 1973) technique was used to classify the factors found into different categories. the remainder of this paper is organised as follows. first, the extant literature regarding the reasons for technology use by young people and use typologies and hierarchy frameworks are discussed. then the research method is introduced, the participants described and the data collection method and data analysis techniques explained. the findings are then presented, focusing on describing the characteristics of each group of students and its hierarchical australasian journal of information systems stevens, guo & li 2018, vol 22, research on educational technologies students’ motivations to use technology in learning 4 framework of technology use motivations. finally, the theoretical and practical implications of this study are discussed. 2 literature review 2.1 students’ motivations for using technologies in learning numerous studies across a broad range of contexts have investigated the factors influencing people’s use of technologies (e.g., ledbetter et al., 2011; lim, 2009; wakefield, wakefield, baker, & wang, 2011; yates, wagner, & majchrzak, 2010). a number of these studies have focused identifying the various factors that explain the use of digital technologies by young people. for example, employing a uses & gratifications approach to investigate the media habits of college students in the context of the new media, parker and plank (2000) found that students did not abandon traditional forms of communication media for the internet, with relaxation and escape being the key drivers of use. similarly, stafford (2005) found that distance education students used the internet to satisfy their content, social and information needs. in investigating facebook’s popularity with young adults, sheldon (2008) found that they use facebook for relationship maintenance, passing time, virtual community, entertainment, ‘coolness’, and companionship. in a survey conducted to determine why college students use wikipedia, lim (2009) found that it is used for quickly checking facts and finding background information. overall, these identified motivations were very similar to the motivations found in mass media and interpersonal communication studies (stafford, stafford, & schkade, 2004). however, when examining the motivations for using technologies within learning contexts, some new and distinct learning related factors have been identified. for instance, pena-shafe et al. (2005) found the key motivations for students’ participating in online discussions were meeting course requirements and gaining feedback from other students. lonn and teasley (2009) found that most students said saving time was the most important benefit of technologymediated learning systems. chou et al. (2010) found that the four most important motivations for students to use course management systems were (1) registering for a course, (2) monitoring their current status, (3) receiving and giving course-related messages/materials, and (4) communicating with instructors and students. in examining students’ motivations for using internet-based communication media in their learning context, cavus and kanbul (2010) found that students’ most important expectations from learning technologies were (1) accessing materials without time and place constraints, (2) having a secured system, (3) showing their assessment results, (4) getting prompt assessment feedback, and (5) interacting more with instructors. guo et al. (2011) found that students used computer mediated media for reasons such as accessibility, communication mode, content management, communication goals, interaction, information seeking, problem solving, and self-disclosure. collectively, these studies demonstrate that there is a very broad range of reasons why students use technologies in learning contexts and suggests that there is no consistent and cohesive understanding of those motivations in the literature, indicating that further work is required to improve this understanding. 2.2 motivation-based student technology use typology another way to study the use of technologies in learning by students is by examining their typology. typology is a means of categorising them into a limited number of groups or types based on various orientations (westbrook & black, 1985). this approach is common in the australasian journal of information systems stevens, guo & li 2018, vol 22, research on educational technologies students’ motivations to use technology in learning 5 marketing discipline, and while many marketing studies seem to be preoccupied with the typology of the online shopper (kau, tang, & ghose, 2003), some research has examined the typologies of students (who are treated as customers of education products offered by educational institutions) in a technology mediated learning environment (tao, 2008). in a study of british children (the go online project), livingstone et al. (2005) identified three groups of teenagers: ‘interactors’, the ‘civic-minded’, and the ‘disengaged’, each of which was distinctive in its social context and approach to the internet. a survey of dutch 10–23 year olds (van den beemt, akkerman, & simons, 2010) found four different clusters of interactive media users: ‘traditionalists’, ‘gamers’, ‘networkers’, and ‘producers’, each of which had specific uses and opinions about interactive media. similar findings arose in a survey of 1000 young brits, in which four internet use groups were identified: ‘peripherals’, ‘normative’, ‘all-rounders’, and ‘active participants’, which were differentiated by individual characteristics and contextual features (eynon & malmberg, 2011). tao (2008) examined the typologies of students based on their perceptions toward e-learning and identified two very distinct groups of students: ‘sceptics’ and ‘optimists’, who were seen to require different online approaches. viewed together, these studies paint a picture of considerable diversity among today’s generation in terms of computer skills, the kind of technologies they use in both their everyday life and in their learning. this diversity also extends to their attitudes toward technology use in learning. since the needs for using technologies should presumably account for students’ technology use behaviour, directly focusing on these needs represents a potentially illuminative approach to identifying the distinctive characteristics of students (westbrook & black, 1985). few studies, have however, examined the typology of students based on their motivations for using technologies in learning (vodanovich et al., 2010). this study aims to close this gap, and attempts to provide a holistic view of students, in terms of their social and psychological needs to be fulfilled when using technologies in learning. 2.3 hierarchical frameworks of technology use motivations previous studies of the motivations for technology use have found that the motivations are not isolated, static traits, but interrelated structures (ledbetter, 2009; markus, manville, & agres, 2000; rubin, 1983; vodanovich et al., 2010), suggesting that people select a technology for interrelated reasons. for instance, ledbetter (2009) speculated about a possible structural model among five online communication attitude variables that indicated their direct and indirect relationships, after identifying strong correlations among the variables. while the premise of considering these factors as a set of interrelated needs and expectations is a more meaningful and accurate explanation of technology use, the possible underlying hierarchical relationships among factors has not been addressed. as phang et al. (2010, p.345) stated: “studying the effect of the variables in isolation may not allow for the inter-relations to be uncovered and may result in ambiguous findings”. understanding the influences of the factors on each other and the hierarchy in which the factors sit is important, as it helps classify and categorise the factors, and thereby formulate their respective strategies, while also providing clarity of thought (hasan et al., 2007). more recently guo et al. (2011) developed a hierarchical framework of the reasons for media use by students. the framework identified the relative importance of each ‘reason’, thus assisting course instructors to identify those aspects related to the most important motivations, hence improving media use within the course. drawing from input-process-output framework and constructivist paradigm, eom (2016) proposed a system view of e-learning systems, demonstrating the interrelationships among input australasian journal of information systems stevens, guo & li 2018, vol 22, research on educational technologies students’ motivations to use technology in learning 6 variables (students, instructors, and e-learning system characteristics), process (learning cognitive process, student self-regulation, and dialogues between students and instructors), and output variables (learning outcomes and student satisfaction). eom (2016) found a significant impact of the dialogue (interaction) on improved student learning outcomes, suggesting that interaction can be a significant factor leading to students’ improved learning. this study seeks to explore the inter-relationship of technology use motivations for each identified segment of students. such a hierarchical framework of motivations would inform our understanding of not only what students want to achieve via technology (various motivations) but also the direct and indirect relationships among all motivations, hence provide insight as to the relative importance of each motivation in the students’ technology mediated learning, and how students higher level motivations can be fulfilled. 3 research method this study adopted a structural approach in order to (1) develop a student typology based on their motivations for using technologies in learning, (2) develop an inter-related framework of motivations for each group of students, and (3) classify the motivations into different categories based on their relative importance for each group of students. data for the study was collected using rgt from 16 university students. from these interviews, 646 raw constructs where elicited and consolidated into 11 factors using qualitative data analysis, from which a repgrid was constructed. cluster analysis was performed on the repgrid and the data generated from the analysis being analysed via ism and micmac, from which a hierarchical framework was produced and classification of motivations was achieved. 3.1 data collection method—rgt rgt is a structured interview process, involving the generation/selection of a list of concepts (‘elements’) about things and/or events to be studied and the forming of attributes (‘constructs’) based on the list of concepts and the linkages which exist among those concepts (latta & swigger, 1992 ). as the rgt process allows for uncovering the cognitive constructs of individuals (tan & hunter, 2002), without the use of a prior adoption of a theoretical framework, thus producing less biased results. in addition, rgt allows participants to express their own views in their own words and yet, due to its systematic nature, allows researchers to probe deeper into the responses to derive rich information. it has been widely used in organisational and information systems (is) research. in is research, the rgt technique has been used to elicit the qualities of excellent system analysts (hunter, 1997), explore the cognitive thinking processes of business and is executives (tan & gallupe, 2006), examine the skills of successful it project managers (napier, keil, & tan, 2009), understand website usability (tung, xu, & tan, 2009), and the most recent study of exploring analytical capabilities for combating e-commerce fraud identify (tan, guo, cahalane, & cheng, 2016). within the study of technology in learning, rgt has been used to identify students’ motivations for using information and communication technologies in learning (guo et al., 2011; guo, tan, & cheung, 2010). key steps in this interview process include (1) element selection, and (2) construct elicitation. australasian journal of information systems stevens, guo & li 2018, vol 22, research on educational technologies students’ motivations to use technology in learning 7 3.1.1 element selection: element selection aims to identify subjects within the domain of the investigation. the relevant elements for this study are information technologies used by the students in learning. a minimum of six elements are required to permit construct elicitation (tan & hunter, 2002). in this study the six elements used by the interviewees were supplied. the six elements used were the most commonly used technologies identified from the literature for learning (conventional websites, learning management systems, discussion forums, wikis and blogs) (ioannou, brown, & artino, 2015) and traditional face-to-face teaching (for comparison purposes). these predefined elements were used so that every participant elicited constructs based on the same set of elements (siau, tan, & sheng, 2010), with the constructs being the motivations for using technologies in learning, to be identified in the interviews. the names of each of the six technologies were then written on individual index cards. 3.1.2 construct elicitation: select triad, elicit raw constructs, and ladder constructs are the qualities that people attribute to the elements. constructs are bipolar in nature. they describe how some elements are alike and yet different from others (tan & hunter, 2002). two interviewing methods, “triading” and “laddering”, are employed to elicit constructs. first, each participant was asked to randomly select three index cards (select triad) from the stack. based on the three elements on the cards, the participant was asked: how are two technologies similar and yet different from the third in terms of your motivations for using these in your learning? to complete the processing of each triad, the participant was encouraged to provide a brief label that best described the motivation and its contrast. the labels for similarity and difference that were identified formed a bipolar construct, e.g., easy to use difficult to use. based on the construct identified, the researcher probed the participants with a series of “how” and “why” questions to clarify the meaning and uncover the underlying meanings (laddering process). the participant then placed the three cards back in the stack, shuffled the deck of index cards, selected another three cards, and the exercise was repeated. the construct elicitation process was then repeated to identify more constructs until either no new constructs were elicited from a triad or the participant became noticeably tired. 3.2 sample and data collection given the intensive and comprehensive nature of the rgt, sample sizes of fifteen to twenty five participants are considered to be more than adequate (tan & hunter, 2002). in this study, sixteen university students (13 males, 3 females), enrolled in two advanced is courses, were interviewed, with each interview ranging from 50 to 110 minutes in length. the participants ranged in age (yrs.) from 20 to 26, and all had been at university for at least 2.5 years (average of 3 years). the majority of students (94%) were studying information systems or software engineering at either undergraduate (88%) or masters (12%) level. all participants reported having used the internet for at least 7 years. all participants had used discussion forums and wikis extensively for online discussions, group assignments, personal journal reflections and other class communications. twelve participants had used wikis for at least 3 years in their university courses. all participants had made use of blogs while at university, with 11 participants using blogs as part of a formal, compulsory activity in one or more of their university classes. all students indicated that they had used blogs in their private life and had used facebook for more than 4 years. australasian journal of information systems stevens, guo & li 2018, vol 22, research on educational technologies students’ motivations to use technology in learning 8 3.3 data consolidation and categorisation the data collected comprised statements of the motivating factors (constructs) for using technology and statements regarding the relationships between those factors. by design, the rgt process allows participants to freely voice their opinions so to permit the best construct elicitation. as a result, a total of 646 raw constructs were provided by the 16 participants. the initial coding was undertaken by one researcher. a second researcher coded two interview transcripts for which a cross-coder reliability of 81.1% was reached, above the 80% acceptable level of cross-coder agreement for exploratory studies (krippendorff, 1980), suggesting that the coding schema was valid. a data reduction process consolidated similar constructs and removed insignificant constructs (those with less than 3 occurrences) (guo et al., 2010; siau et al., 2010). the consolidation resulted in 77 unique constructs. these 77 constructs were then categorised via an adjusted core-categorisation procedure (jankowicz, 2004) with the aim of maximising the similarity of meaning within the category and dissimilarity between categories. the 77 unique constructs were consolidated into 11 dimensions, as shown in table 1, in which each dimension is denoted as si, in sequence. the categorisation process was examined independently by two researchers with an initial agreement level of just over 80% agreement being achieved, with all remaining discrepancies being resolved via discussion and consensus between the researchers. motivation dimension code description access and content control s1 the security aspects of accessing the technology and the content maintained by the technology. accessibility s2 both the physical access to the technology and subsequent use of the technology (culnan, 1984). communication efficiency s3 the extent to which communication can be done conveniently, easily, frequently, and quickly. communication mode s4 the way in which the technology assists the learners to communicate, such as audio, video, or multimedia. communication quality s5 the extent to which communication is clear, in depth, effective, specific, and focused. course management s6 involves the ability of learning technologies to take an administrative role in learner’s learning. information seeking s7 the “purposive seeking for information as a consequence of a need to satisfy some goals.” (wilson, 2000 p.49) interaction s8 the exchangeability of sources and receivers, including interactions between students and learning contents, students and instructors, students and students, and students and technology itself (bouhnik & marcus, 2006; rice, 1987). learning capability s9 the ability to create a learning environment to develop learners’ critical thinking skills, to be independent, active and reflective, to collaborate and cooperate, and to be constructive (miers, 2004). managing contents s10 the ways people want to manage their data with technologies. self-disclosure s11 the extent to which any message about the self a person communicates to another (wheeless & grotz, 1976). table 1: summary of digital native’s technology use motivations australasian journal of information systems stevens, guo & li 2018, vol 22, research on educational technologies students’ motivations to use technology in learning 9 the richness of data allowed the researchers to also distinguish a total of 504 unique relationship nodes among the 646 raw constructs, where each node was in the form of one motivation construct being influenced by another, with the relationship type being defined as ‘influences’, where attaining factor ‘a’ influences achieving factor ‘b’ (warfield, 1994). as with the constructs, a data reduction process was undertaken on the relationship nodes, which resulted in 328 unique relationships. these relationships were then categorised using the motivation categories (table 1), giving rise the matrix of influence between the 11 categories, as shown in table 2. the relationships identified in this matrix represented the relation between any two unique constructs from any two factors for any participant. a total of 65 unique relationships between the various constructs were identified. 3.4 data analysis techniques the study sought to understand whether differences existed in the motivations for using technology in learning between different typologies of students. thus, a two-stage approach to clustering (hair, anderson, tatham, & black, 1998; punj & stewart, 1983) was used. initial solutions, using the average-linkage hierarchical method, with squared euclidean distance as a measure of similarity, provided a preliminary indication of the total number of clusters. following phang et al. (2010), the final cluster solution was then identified using the quick cluster k-means procedure. details of the analysis undertaken and clusters found are set in the results section. fr om to s1 s2 s3 s4 s5 s6 s7 s8 s9 s10 s11 s1: access and content control 4 5 5 1 9 11 13 8 5 s2: accessibility 2 8 3 1 3 4 1 s3: communication efficiency 1 1 2 s4: communication mode 1 3 11 4 2 2 7 s5: communication quality 1 1 1 s6: course management 4 2 1 3 3 s7: information seeking 2 5 1 2 s8: interaction 1 7 7 1 8 15 1 2 s9: learning capability 2 1 1 1 1 s10: managing contents 1 3 2 1 7 7 10 1 s11: self-disclosure 1 3 3 4 2 2 table 2: contextual relationships between factors interpretive structural modelling (ism) was used to generate hierarchical frameworks of motivations. ism is an interactive learning process, whereby a set of different interrelated variables affecting the system under consideration is structured into a comprehensive systemic model (sage, 1977; warfield, 1974). ism is considered as one of the best approaches to use to develop the models of the factors and their direct relationships (sage, 1977). its objective is “to expedite the process of creating a digraph, which can be converted to a structural model, and then inspected and revised to capture the user’s best perceptions of the situation” (malone, 1975, p. 399). by using the practical experience and knowledge of individuals and groups, ism provides a means by which order and direction can be imposed on the complex relationships among the australasian journal of information systems stevens, guo & li 2018, vol 22, research on educational technologies students’ motivations to use technology in learning 10 elements of a system (sage, 1977) and the limitations that individuals have in dealing with complex issues involving a significant number of variables at one time can be overcome (waller, 1975; warfield, 1976). ism provides both a comprehensible model of an inherently complex and usually impenetrable system (kanungo & anantatmula, 2008) and a means of integrating diverse viewpoints (vivek, banwet, & shankar, 2008). ism has been applied across a range of relevant areas including developing a better understanding of higher education program planning (hawthorne & sage, 1975), evaluating is effectiveness (kanungo, duda, & srinivas, 1999), determining information technology (it) enablers and barriers for knowledge management (bhattacharyya & momaya, 2009; s. kanungo & anantatmula, 2008), and understanding students’ motivations for using computer mediated communication (cmc) technologies in their learning contexts (guo et al., 2011). micmac (a french acronym for “matrice d’impacts goises – multiplication appliqués a un classment,” meaning “cross-impact matrix – multiplication applied to classification”) was used to classify the motivations into different categories for each group identified in cluster analysis. micmac is a data analysis technique related to ism and is a systematic analysis tool for categorising variables based on hidden and indirect relationships, as well as for assessing the extent to which they influence each other. consideration of the indirect relations in a hierarchical framework is important in a complex system, since indirect relations may have a great impact on the system behaviour, through influencing chains or reaction loops. many researchers have integrated ism and micmac techniques in a wide range of areas (e.g., guo, li, & stevens, 2012; guo et al., 2011; s. kanungo & anantatmula, 2008; lee, chao, & lin, 2010). mandal and deshmukh (1994) claim that the primary goal of micmac analysis is to evaluate the ‘driver power’ and dependence of the variables. driver power refers to the degree of influence that one variable has over another, while ‘dependence’ is defined as the extent to which one variable is influenced by others (hu, chiu, & yen, 2009). based on driver power and dependence, variables can be classified into four major types: independent, dependent, linkage, and autonomous. ‘independent variables’ have high driver power and relatively low dependence and have an important influence on the elements above them in the hierarchy. ‘dependent variables’ have high degree of dependence, but relative low driver power, forming the upper-most level in the ism hierarchy, often the final outcomes of the influence of all the other elements. ‘linkage variables’ possess relatively high dependence and driver power and are therefore strong linkage variables in the hierarchy. finally, ‘autonomous variables’ have lower driver power as well as low dependence, and turn out to be a set of relatively low-influence, stand-alone variables. in this study, the ism technique was used to develop the interpretive structural model of the variables for each identified group, as a way to shed light on the direct relationships between motivations. micmac was then used to map and understand the indirect and hidden relationships between the variables. table 3 presents the logical flow of both ism and micmac techniques adopted in this study (e.g., guo et al., 2012; guo et al., 2011; kanungo & anantatmula, 2008; lee et al., 2010). guo et al. (2011) provide a concise overview of the analysis procedures involved. australasian journal of information systems stevens, guo & li 2018, vol 22, research on educational technologies students’ motivations to use technology in learning 11 step description 1) identifying a set of factors affecting students’ use of technologies through interviews (11 factors were identified, see table 1). 2) establishing a pair-wise contextual relationship between factors through analysing interview data (see table 2). 3) developing an adjacency matrix (see appendix b tables a1 & a2). 4) developing a reachability matrix, and checking the matrix for transitivity (see tables 4 & 5). 5) partitioning the reachability matrix into different levels (see appendix b tables b1 & b2). 6) forming a canonical form of matrix (see appendix c tables c1 & c2). 7) drawing a directed graph (digraph) and removing the transitive links. 8) converting the resultant digraph into an ism by replacing variable nodes with statements (see figures 2 & 3). 9) conducting micmac analysis to classify factors into various categories based on their driver power and dependence (see figures 4 & 5). table 3: modelling approach 4 results 4.1 a typology of students based on the motivations for using technologies in learning to develop the typology of students based on their motivations, a matrix of the 11 motivations (rows) and the 16 participants (columns) was created, in which the cells were populated by the total number of times each motivation was mentioned by each participant. the matrix was duplicated, substituting the counts with the relative percentage that a participant mentioned each motivation. application of the k-means clustering method to the 11 motivation percentage scores for each participant indicated that a solution with two distinct clusters produced both the most efficient and interpretable result, with the exception of one outlier. based on the data indicating cluster centroids for the two-cluster solution, a radar diagram (figure 1) was generated to depict the factors that influence the use of technologies in these two clusters. the labelling for each cluster, namely cluster 1 as independent learners and cluster 2 as traditional learners, was determined by examining the centroid means of the factor score obtained from cluster analysis. figure 1: radar diagram of clusters .00 .05 .10 .15 .20 .25 s1: access and… s2: accessibility s3: communication… s4: communication… s5: communication… s6: course managements7: information seeking s8: interaction s9: learning capability s10: managing contents s11: self-disclosure cluster1 cluster2 australasian journal of information systems stevens, guo & li 2018, vol 22, research on educational technologies students’ motivations to use technology in learning 12 cluster 1 consisted of 6 males and 2 females, being 53.3% of all participants. this group scored significantly higher on learning capability, while scoring lower on accessibility, communication efficiency, and self-disclosure. they were all undergraduates, majoring in is or software engineering. cluster 2 consisted of 6 males and 1 female, being 46.7% of all participants. this group had a weaker belief that using technologies for learning improved their learning capabilities. however, they scored significantly higher in regard to accessibility, communication efficiency, and self-disclosure. this group consisted of 2 postgraduates (is majors) and 5 undergraduates (1 business major and 4 is majors). 4.2 hierarchical frameworks developed using ism the ism analysis used in the study required two key elements: (1) a set of variables considered for model development and (2) pair-wise contextual relationships among this set of variables. both table 1 and table 2 show the set of variables identified in this study and the contextual relationships among them. based on overall contextual relationships, a pair-wise contextual relationship for each of the clusters was identified. after removing the weak relations (those mentioned by less than two participants), the final contextual relationships between each pair of variable categories for clusters 1 & 2 were created (appendix a tables a1 & a2), in which cells were populated by 1s and 0s, whereby ‘1’ indicates the relationship and ‘0’ indicates otherwise. these binary matrices, which describe whether there is a direct relationship between the row and column variables, are termed adjacency matrices. based on each cluster’s adjacency matrix, reachability matrix (tables 3 & 4), level partitions (appendix b tables b1&b2), and canonical matrix (appendix c tables c1&c2) for both two clusters were calculated. then an ism was formed for clusters 1 & 2 respectively based on its own canonical matrix, as shown in figures 2 & 3. these two diagrams represent the structural linkages among factors that motivate students to use technologies in their learning. tables 4 & 5 also include each motivation’s driver power and dependence, which can be obtained from the reachability matrix by the summation of 1s in the corresponding rows and columns respectively (hu et al., 2009). m s1 s2 s3 s4 s5 s6 s7 s8 s9 s10 s11 driver power s1 1 0 1 0 1 0 1 1 1 1 0 7 s2 0 1 1 0 0 0 0 0 1 0 0 3 s3 0 0 1 0 0 0 0 0 0 0 0 1 s4 0 0 1 1 1 0 1 1 1 0 1 7 s5 0 0 0 0 1 0 0 0 0 0 0 1 s6 0 1 1 0 0 1 0 0 1 0 0 4 s7 0 0 1 0 1 0 1 0 0 0 0 3 s8 0 0 1 0 1 0 1 1 1 0 0 5 s9 0 0 0 0 0 0 0 0 1 0 0 1 s10 0 0 1 0 1 0 1 1 1 1 0 6 s11 0 0 1 0 1 0 1 1 1 0 1 6 dependence 1 2 9 1 7 1 6 5 8 2 2 44/44 table 4: cluster 1 reachability matrix australasian journal of information systems stevens, guo & li 2018, vol 22, research on educational technologies students’ motivations to use technology in learning 13 m s1 s2 s3 s4 s5 s6 s7 s8 s9 s10 s11 driver power s1 1 1 1 0 1 0 1 1 1 1 1 9 s2 0 1 1 0 1 0 1 1 1 0 1 7 s3 0 0 1 0 0 0 0 0 0 0 0 1 s4 0 0 1 1 1 0 1 1 1 0 1 7 s5 0 0 0 0 1 0 0 0 0 0 0 1 s6 0 0 1 0 1 1 1 1 1 0 1 7 s7 0 0 0 0 1 0 1 0 0 0 0 2 s8 0 0 1 0 1 0 1 1 1 0 1 6 s9 0 0 0 0 0 0 0 0 1 0 0 1 s10 0 0 1 0 1 0 1 1 1 1 1 7 s11 0 0 1 0 1 0 1 1 1 0 1 6 dependence 1 2 8 1 9 1 8 7 8 2 7 54/54 table 5: cluster 2 reachability matrix figure 2: independent learners’ ism from each cluster’s hierarchical framework, the direct relationships among all motivations can be identified. for independent learners, the driver variable of access and content control enabled managing contents and various communication modes of technologies to give students freedom in terms of the ways they express themselves in technology mediated learning environment. then, both managing contents and self-disclosure variables resulted in interaction, which influenced both information seeking and learning capability. course management determined accessibility. communication efficiency was dependent on both accessibility and information seeking. information seeking also influenced communication quality. in contrast, for traditional learners, access and content control was identified as the main driver of accessibility and managing contents, which also co-determined, along with communication mode and course management, interactions and self-disclosure. interactions and self-disclosure influenced each other, as well as information seeking, which was the only route for developing students’ communication and learning capabilities. s5 communication quality s4 communication mode s9 learning capability s3 communication efficiency s6 course management s1 access and content control s2 accessibility s11 self-disclosure s10 managing contents s7 information seeking s8 interaction australasian journal of information systems stevens, guo & li 2018, vol 22, research on educational technologies students’ motivations to use technology in learning 14 figure 3: traditional learners’ ism 4.3 classifications of motivations generated using micmac to understand the nature of these motivations in depth, the development of structural models with a micmac analysis is followed to identify both the direct and indirect relationships among all variables, and demonstrate the complete role of each motivation by classifying motivations into four different categories based on their driver power and dependence provided, as shown in tables 4 & 5 (guo et al., 2011). figures 4 & 5 show the categorisation results, in which quadrants i, ii, iii and iv represent independent, linkage, dependent, and autonomous variables respectively. the means of the driver power and dependence were used to divide the driver power and dependence diagram into the four quadrants. figure 4: independent learners’ driver power and dependence plot s5 communication quality s2 accessibility s9 learning capability s3 communication efficiency s8 interaction s1 access and content control s11 self-disclosure s7 information seeking s4 communication mode s10 managing contents s6 course management australasian journal of information systems stevens, guo & li 2018, vol 22, research on educational technologies students’ motivations to use technology in learning 15 figure 5: traditional learners’ driver power and dependence plot the driver power and dependence diagram shows the nature of how the identified motivations interact and play, and how the four types of variables differ from each other depending on the specific role they play in today’s students’ dynamic technology use process in learning. for comparison purposes, table 6 shows both groups’ technology use motivation roles in their corresponding hierarchical framework. it is clear that both groups had the same dependent variables, but differed in the other three categories, indicating the different underlying mechanism taken by each of the groups to achieve their learning goals. variable role independent learners^ traditional learners^ independent variables access and content control communication mode managing contents self-disclosure access and content control accessibility communication mode course management managing contents linkage variables interaction interaction self-disclosure dependent variables communication efficiency communication quality information seeking learning capability communication efficiency communication quality information seeking learning capability autonomics variables accessibility course management ^ variables that are common to both groups are highlighted in bold table 6: different roles of motivations in each group’s technology use 5 discussion this study yields four important findings. first, two groups of students, namely independent learners and traditional learners, were identified based on students’ motivations for using technology in learning. second, this study captured the hierarchical structure of these factors and their functional dependencies among each other for each group respectively. third, despite the similarity of learning goals between the groups, their individual paths toward their learning goals were different when it comes to learning with technologies. finally, it was australasian journal of information systems stevens, guo & li 2018, vol 22, research on educational technologies students’ motivations to use technology in learning 16 found that interaction is the key activity that leads to student learning goals for both groups. each of these key findings is discussed in detail below. 5.1 independent vs. traditional learners two distinct groups of students, labelled as independent learners and traditional learners, were found. the groupings were based on the different motivation preferences for using technologies in learning, in which the ‘learning capability’ factor was the main distinguishing factor between the two groups. the independent learners had significantly stronger preferences than traditional learners in terms of using technologies for improving learning capability. this suggests that it is only the independent learner group that carries the distinctive learning attributes claimed by some studies to be universal across today’s students. these distinctive attributes are summed up by selwyn, who describes students as “as being no longer a passive recipient of educational instruction, but instead cast into an active role of (re)constructing the nature, place, pace and timing of learning events as they wish” (selwyn, 2009, p. 367). the other group of learners retained a more traditional view of learning with technologies. this finding is consistent with those of spires et al. (2008), in which students’ technology use in schoolwork was found to be less creative and meaningful than their use of technology outside of school. the differences between the two groups in regard to learning capability are not reflected in differences in their technology experience (for which there was none), nor are the differences reflected in the demographics of the participants. the lack of differentiation between the groups in both of these aspects suggests that the explanation of differences between the groups lies elsewhere. the different approaches toward using learning technology may well be the underlying source of this difference as the learning capability pursued by the independent learner group was observed to be in line with constructive learning paradigm (biggs, 2003). this finding has considerable implications for the proponents of the ‘digital natives’ / ‘net generation’ / ‘millennial’ (conklin, 2013; jones, ramanau, cross, & healing, 2010) view of students, as our findings demonstrate the attributes that are often ascribed to all students are not in fact universal, as is discussed later. 5.2 hierarchical frameworks of independent and traditional learners analysis of the two structural models (figures 2 & 3) and the driver power and dependence diagrams (figures 4 & 5) for each group reveals a number of similarities as well as differences between the two groups in terms of the hierarchy and relative importance of the motivations. both groups had some common independent variables for their technology use in learning, although the driver powers varied across the variables, hence indicating differences in their relative importance. the traditional learner group had two additional independent variables, accessibility, and course management, whereas the independent learners had only selfdisclosure. the access and content control variable, having maximum driver power in the traditional learner group, was the key enabler influencing traditional learners’ technology use. hence, the security of the system and the extent of control the student has over content appear to have a direct impact on how they use the technology to manage their learning content, which may ultimately impact their learning outcomes. these independent variables, appearing at the bottom (left hand side, in this case) of the ism, are all considered technology related product attributes. without these technological attributes as given conditions, the interaction of students with the other aspects of the technology cannot be assured and, hence, australasian journal of information systems stevens, guo & li 2018, vol 22, research on educational technologies students’ motivations to use technology in learning 17 the goals of technology-enabled learning would be difficult to achieve. these ‘givens’, from a technology mediated learning process perspective, can be considered as aspects that are necessary, though not sufficient, to achieve the desired ends (guo et al., 2011). in practical terms this suggests that these technological features should be properly maintained, and continuously and consciously improved. the linkage variable ‘interaction’, sits in the middle of the hierarchy and is imperative for both groups in translating technology features into effective use to enhance communication performance and learning capabilities, although the relationship between this linkage variable and others differs between the two different groups. this finding is not only consistent with eom’s (2016) empirical finding of the significant impacts of the interaction on student learning outcomes and satisfaction, but also support eom’s argument of considering “interaction” as a process (linkage variable) through which students can enhance their learning in e-learning systems. the key difference between these two group’s hierarchical frameworks, was the role of the self-disclosure variable. for traditional learners, this variable was a linkage variable which interacted with “interaction”, while for independent learners, self-disclosure was a ‘given’ (independent) variable. this suggests that independent learners believe that it is important for them to have more control over the technologies they use in learning in terms of their extent to self-disclose. this is consistent with previous findings, such as, ledbetter (2009) identified self-disclosure as an important motivation for people who communicated online. interestingly, denker et al. (2017) found a negative impact of self-disclosure on students’ willingness to participate in social media (twitter) discussions as part of their coursework because of fears that the university instructors would learn about their personal life. this study was, however, conducted in learning systems which were only used for class learning (so-called walled garden), which may explain why our participants showed a positive relationship between self-disclosure and interaction. in the ism hierarchy, linkage variables are considered as ‘means’ to achieve individual eventual goals since they are related to both independent (‘given’) variables and dependent (‘end’) variables (guo et al., 2011). by their nature, these ‘means’ variables are ‘factors of instability’, since any action towards them would result in significant changes on other variables (hu et al., 2009). thus, finding a way to control, manipulate, or develop students’ interaction skills is critical to fulfil their learning goals. as the structural relationships of the ism move toward the top of the models, the same four dependent variables for both groups were found. information seeking emerged as the only route that drove the traditional learner group to use technologies to enhance their communication and learning capabilities. in contrast, independent learners’ learning capabilities can be directly influenced through interactions, whereby information seeking can only result in improved communication efficiency and communication quality. this suggests that traditional learners see that learning is about instructors creating content and students being passive recipients of that content. this group sees the role of technology as enabling them to retrieve course materials and learning content. conversely, independent learners see that learning is about interaction, engagement, peering learning, collaboration, and improving their learning capabilities (shih, feng, & tsai, 2008). despite being the ‘end’ variables, and not driving any other variables, the high dependence of these variables means australasian journal of information systems stevens, guo & li 2018, vol 22, research on educational technologies students’ motivations to use technology in learning 18 that they are important as action on any other variable will have an impact on them (hasan et al., 2007). finally, accessibility and course management, which were the ‘givens’ for the traditional learner group, were considered as autonomous variables for the independent learner group, which suggests for this group, that these two factors can be seen as disconnected and needing little attention. the traditional learner group however, these factors are important variables that do require attention. the differences in the importance of these two variables would seem to further support the notion that traditional learners see the technology as providing a means to access content, whereas independent learners are less concerned about these more administrative aspects of technology. 5.3 importance of interaction and the use of technology to manage learning content all students had very strong views regarding the use of technology for ‘interaction’ as this was the highest ranked factor by all interviewees. this result supports those of other studies and confirms that today’s students are collaborative and interactive (chen, 2014). this finding is also consistent with previous technology mediated learning literature, in which interaction is one of the most important elements in improving the value of the e-learning experience (laer & elen, 2017). however, parkes et al. (2015) reported that students perceived themselves to be less competent in interaction with the learning content and less prepared to seek interactions with other members of the learning community (especially instructors), hence challenging the notion of social constructivist principles. our findings provides a means of reconciling these different view by indicating that although students appreciate the benefits of interaction in e-learning, they may have difficulties fully realising interaction because they lack the skills needed for effective interactions with others. the use of technology to manage learning content appears to have become a standard practice by all students. this finding is consistent with previous studies in which students were found to use computer mediated communication media for file management, storage, and database repository (e.g., guo et al., 2010; pena-shafe et al., 2005). however, a recent study examining student’s preparedness for e-learning environment (parkes et al., 2015) shows that students perceived themselves less competent in terms of knowledge and use of e-learning systems. as both groups of students demonstrated the direct impact of the skills required to manage learning content in e-learning systems on interaction, it is crucial to find a way of enhancing student’s ability of managing their e-learning systems in order to promote high quality interaction and enhance their learning goals. 6 implications 6.1 implications for research this study makes a number of significant contributions to research about today’s students technology use in learning. first, by using students motivations towards the use of learning technologies, two distinct grouping of students have been identified, hence indicating today’s students are heterogeneous in terms of these motivations and not all students prefer an independent learning style, as has been claimed by some recent studies (buchanan & james, 2014; cheng, 2014; marriott, 2010). future research examining students’ technology use should take this heterogeneity into account when considering how technology should be implemented, and avoid assumptions as to how technology can be used in learning as our australasian journal of information systems stevens, guo & li 2018, vol 22, research on educational technologies students’ motivations to use technology in learning 19 findings suggest that the current cohort of students may not learn differently from previous generations; instead, they may only use different tools in that learning and have different learning preferences. the second contribution of the study is the identification of the hierarchical framework and classification of the motivations for using technologies in learning for the two groups identified in the study. this finding further demonstrates the heterogeneity of students in terms of not only their motivations, but also the inter-relationships among those motivations. since the inter-relationships identified in this study will aid in our understanding of the relative position and influence of the motivations to each other, the results of this study reveal that different groups of students may have different underlying mechanisms to achieve their learning goals in regard to technology mediated learning, as driven by the different factors, and, as such, a universal teaching approach would not seem appropriate. although our data was obtained from a small group of technological competent students, our results were based on rigorous data collection and analysis, and show that there was a set of inter-related factors that influenced students’ technology use in learning. in addition to the theoretical desirability of such a linkage, this study reaffirmed the value of the integration of ism and micmac methodologies in exploring structure and inter-relationships among variables. 6.2 implications for practice from a practical perspective, this study offers insights for university policy-makers and instructors who shape the policies and strategies on the use of information technologies for educational purposes, and for practitioners who wish to understand the new technologies today’s generations are using, and will use, in learning and future work. first, learning capability is the most distinct factor differentiating the two groups of students, indicating that there is heterogeneity among today’s students, in terms of their motivations toward learning with technologies. this suggests that calls for fundamental change to the existing education systems to cater for the needs of this new cohort of learners may in fact be somewhat premature as our data shows that not every student prefers discovery-based learning (bennett et al., 2008). this is not to say that radical change of current educational systems is unnecessary, but does suggest that whatever changes are required, they are unlikely to be captured in a single approach. university policy makers and instructors should differentiate and tailor their technology related business strategies, policies, and/or actions to accommodate the different learning approaches of the different types of students or risk student dissatisfaction and lessthan-effective learning performance. in addition, even though the current generation of students seem to enjoy an ‘engage and collaborate’ rather than a ‘command and control’ model within organisations (vodanovich et al., 2010), our findings suggests that this may not be the case for all students. second, the development of a hierarchical framework of motivations for each group helps in understanding how different motivations drive students’ technology use in learning. the classification of the motivations also allows us to understand their relative importance in the technology use process, i.e. which technological attributes are required for achieving their eventual learning goals. this integrated model is important, since it can assist identify why students want to use technologies (dependent variables) and what are the fundamental drivers to achieve those goals (independent variables). although both groups showed the importance of the linkage variable “interaction” in enhancing learning goals, it appears that some students, (notably the traditional learners) would benefit from training that helped them australasian journal of information systems stevens, guo & li 2018, vol 22, research on educational technologies students’ motivations to use technology in learning 20 develop a learning culture so that they can make full use of e-learning systems for interaction and collaboration (parkes et al., 2015). more importantly, it was found that there is a different flow of causal influence for each group, i.e., different mechanisms to achieve the learning goals, indicating that technological features favourable to one group may not be popular with the other group. instructors should be aware that intensive and high-quality interactions among the independent learners are key enablers leading to improved learning capabilities, whereas for traditional learners, information seeking has a more direct impact than interaction on improving learners’ learning outcomes. in addition, since the identified independent variables could have an overarching effect on the overall performance, if instructors fail to introduce the right technologies with appropriate attributes into their teaching, they may in fact demotivate students and inhibit technology use. this may be especially the case for traditional learners, given their attitudes toward e-learning systems. some suitable orientation that focuses on knowledge and use of learning management systems may be needed in order to make the students realize the full potential of the systems. furthermore, differences in the relationships between the variables means that the impact of the de-motivators may be different. for example, based on figures 2 & 3, if security (access and content control) is less than ideal, which discourages systems use, then the independent learners are likely to make less use of the content management aspects of the system, but continue other use, whereas traditional learners are likely to access the system less, hence making less use of all aspects of the system. finally, technology application developers may find that by accommodating the technological attributes that are more important for each of these different groups of students into their designs will result in better acceptance of the their software across the student popular. similarly marketing functions may also consider incorporating these ideas into their customized campaigns so that they take all current students along a series of steps leading from technology specific attributes to their desired learning goals, by the paths that they understand and appreciate (thompson & chen, 1998). without a better understanding of the technology use mechanisms of digital users, all organisations investing in training efforts for newly hired graduates might find that their strategies are misdirected or less than effective. 7 limitations and future directions a limitation of this study is the assumption that all participants were digitally savvy and capable. while there is no established measure of digital capability, a group of students with technical skills described by studies of recent cohorts of students was chosen deliberately, including the attributes ascribed to these students by those proposing the notion of digital natives (helsper & eynon, 2010; prensky, 2001; vodanovich et al., 2010). the small sample size, which may have lessened the rigour of the cluster analysis, is also another potential limitation. to increase the validity of these results, future studies need to expand on this work with a larger sample size, exploring the ideas and framework presented in this study. in addition, 11 technology use motivations and the ism models were developed based on the opinions of 16 students from one university. the use of a larger sample size, or other groups of students might reveal additional or different factors that influence their use. the framework developed in this study might also vary across different student cohorts as there are differences of opinion about the contextual relationships among the variables (faisal, banwet, & shankar, 2006). further research is required to provide a more robust shared mental model for current students. the ism developed in this study was generated based on the australasian journal of information systems stevens, guo & li 2018, vol 22, research on educational technologies students’ motivations to use technology in learning 21 interpretations of respondents, therefore empirical examination is needed to test the validity of this hypothesised model, with structural equation modelling (sem) technique as the recommended approach (qureshi, kumar, & kumar, 2008). in addition, while these two groups were similar in terms of their demographic factors and technological skills, one interesting avenue for future empirical research would be to explore whether there is a significant difference between these two groups in terms of their approaches to learning, challenging claims about the current generation of students distinctive and preferred independent learning style. in conclusion, to move educational practice forward, it may be necessary to change the educational systems to meet students’ learning needs as advances in technologies require societal changes in their approaches and practices to meet individuals’ changing needs (bennett & maton, 2010). however, such changes should be designed based on theory and supported by clear research evidence (bennett et al., 2008). drawing on current students’ technology use motivation perspective, this study used both ism and micmac techniques to re-assess our perceptions about students use of technologies in learning, by not only critically identifying and classifying the key factors that influence that use, but also by revealing the effects and inter-relationship of each of those factors on their technology use behaviours in learning. the differences in the technology use hierarchical structures between these two groups indicate that not all students use technologies in the same way when it comes to learning. references bagozzi, r. p., bergami, m., & leone, l. (2003). hierarchical representation of motives in goal setting. journal of applied psychology, 88(5), 915-943. bennett, s., & maton, k. (2010). beyond the 'digital natives' debate: towards a more nuances understanding of students' technology experience. journal of computer assisted learning, 26, 321-331. bennett, s., maton, k., & kervin, l. (2008). the ‘digital natives’ debate: a critical review of the evidence. british journal of educational technology, 39(5), 775-786. doi:10.1111/j.14678535.2007.00793.x bhattacharyya, s., & momaya, k. (2009). interpretive structural modelling of growth enablers in construction companies. singapore management review, 31(1), 73-97. biggs, j. (2003). teaching for quality learning at university : what the student does (2nd ed.). buckingham ; philadelphia: society for research into higher education : open university press. bouhnik, d., & marcus, t. (2006). interaction in distance-learning courses. journal of the american society for information science and technology, 57(3), 299-305. doi:10.1002/asi.20277 brown, m. (2005). learning spaces. in d. g. oblinger & j. l. oblinger (eds.), educating the net generation: educause. buchanan, g. b., & james. (2014). learning style and study skills differences across business and other academic majors. journal of education for business, 61(7), 303-307. australasian journal of information systems stevens, guo & li 2018, vol 22, research on educational technologies students’ motivations to use technology in learning 22 cavus, n., & kanbul, s. (2010). designation of web 2.0 tools expected by the students on technology-based learning environment. procedia social and behavioral sciences, 2(2), 5824-5829. chen, y. c. (2014). an empirical examination of factors affecting college students’ proactive stickiness with a web-based english learning environment. computers in human behavior, 31(31), 159-171. cheng, g. (2014). exploring students' learning styles in relation to their acceptance and attitudes towards using second life in education: a case study in hong kong. computers & education, 70, 105-115. chou, c., peng, h., & chang, c.-y. (2010). the technical framework of interactive functions for course-management systems: students' perceptions, uses, and evaluations. computers & education, 55(3), 1004-1017. doi:10.1016/j.compedu.2010.04.011 conklin, t. a. (2013). making it personal: the importance of student experience in creating autonomy-supportive classrooms for millennial learners. journal of management education, 37(4), 499-538. culnan, m. j. (1984). the dimensions of accessibility to online information: implications for implementing office information systems. acm transactions on information systems, 2(2), 141-150. doi:10.1145/521.523 denker, k. j., manning, j., heuett, k. b., summers, m. e., denker, k. j., manning, j., . . . manning, j. (2017). twitter in the classroom: modeling online communication attitudes and student motivations to connect. computers in human behavior. duperrin, j. c., & godet, m. (1973). methode de hierarchisation des elements d'un systeme. rapport economique du cea, r-45-41. eom, s. b. (2016). the determinants of students' perceived learning outcomes and satisfaction in university online education: an update. decision sciences journal of innovative education, 14(2), 185-215. eynon, r., & malmberg, l.-e. (2011). a typology of young people's internet use: implications for education. computers & education, 56(3), 585-595. faisal, m. n., banwet, d. k., & shankar, r. (2006). supply chain risk mitigatin: modeling the enablers. business process management journal, 12(4), 535-552. guo, z., li, y., & stevens, k. j. (2012). analyzing students’ technology use motivations: an interpretive structural modeling approach. communications of the association for information systems, 30(article 14). guo, z., lu, x., li, y., & li, y. (2011). a framework of students' reasons for using cmc media in learning contexts: a structural approach. journal of the american society for information science and technology, 62(11), 2182-2200. doi:10.1002/asi.21631 guo, z., tan, b., & cheung, k. (2010). students’ uses and gratifications for using computermediated communication media in learning contexts. communications of the association for information systems, 27(1), article 20. hair, j. f., jr, anderson, r. e., tatham, r. l., & black, w. c. (1998). multivariate data analysis (fifth eds. ed.). englewood cliffs, n.: prentice hall. australasian journal of information systems stevens, guo & li 2018, vol 22, research on educational technologies students’ motivations to use technology in learning 23 hasan, m. a., shankar, r., & sarkis, j. (2007). a study of barriers to agile manufacturing. international journal of agile systems and management, 2(1), 1-22. hawthorne, r. w., & sage, a. p. (1975). on applications of interpretive structural modeling to higher education program planning. social-economic planning sciences, 9(1), 31-43. helsper, e. j., & eynon, r. (2010). digital natives: where is the evidence? british educational research journal, 36(3), 503 520. doi:10.1080/01411920902989227 hu, h. y., chiu, s. i., & yen, t. m. (2009). modified ipa for order-winner criteria improvement: a micmac approach journal of applied sciences, 9(21), 3792-3803. hunter, m. g. (1997). the use of repgrids to gather interview data about information systems analysts. information systems journal, 7(1), 67-81. doi:10.1046/j.1365-2575.1997.00005.x ioannou, a., brown, s. w., & artino, a. r. (2015). wikis and forums for collaborative problembased activity: a systematic comparison of learners' interactions. internet & higher education, 24, 35-45. jankowicz, d. (2004). the easy guide to repertory grids. hoboken, nj: wiley. jonassen, d. h., hernandez-serrano, j., & choi, i. (2000). integrating constructivism and learning technologies. in j.m.spector & t.m.anderson (eds.), integrating and holistic perspectives on learning, instruction and technology: understanding complexity (pp. 103128). netherlands: kluwer academic publishers. jones, c., ramanau, r., cross, s., & healing, g. (2010). net generation or digital natives: is there a distinct new generation entering university? computers & education, 54(3), 722732. kanungo, r. n., duda, s., & srinivas, y. (1999). a structural model for evaluating information systems effectiveness. systems research and behavioral science, 16(6), 495-518. kanungo, s., & anantatmula, v. (2008). role of it and km in improving project management performance. vine, 38(3), 357-369. kau, a. k., tang, y. e., & ghose, s. (2003). typology of online shoppers. journal of consumer marketing, 20(2), 139-156. kennedy, g. e., judd, t. s., churchward, a., gray, k., & krause, k. l. (2008). first year students' experiences with technology: are they really digital natives? australasian journal of educational technology, 24(1), 108-122. krippendorff, k. (1980). content analysis: an introduction to its methodology. beverly hills sage. laer, s. v., & elen, j. (2017). in search of attributes that support self-regulation in blended learning environments. education & information technologies, 22(4), 1-60. lai, k. w., & hong, k. s. (2015). technology use and learning characteristics of students in higher education: do generational differences exist? british journal of educational technology, 46(4), 725–738. latta, g. f., & swigger, k. (1992). validation of the repertory grid for use in modeling knowledge. journal of the american society for information science and technology, 43(2), 115-129. doi:10.1002/(sici)1097-4571(199203)43:2<115::aid-asi2>3.0.co;2-i australasian journal of information systems stevens, guo & li 2018, vol 22, research on educational technologies students’ motivations to use technology in learning 24 ledbetter, a. m. (2009). measuring online communication attitude: instrument development and validation. communication monographs, 76(4), 463 486. doi:10.1080/03637750903300262 ledbetter, a. m., mazer, j. p., degroot, j. m., meyer, k. r., yuping mao, & swafford, b. (2011). attitudes toward online social connection and self-disclosure as predictors of facebook communication and relational closeness. communication research, 38(1), 2753. doi:10.1177/0093650210365537 lee, y.-c., chao, y. h., & lin, s.-b. (2010). structural approach to design user interface. computers in industry, 61(7), 613-623. lim, s. (2009). how and why do college studetns use wikipedia? journal of the american society for information science and technology, 60(11), 2189-2202. doi:10.1002/asi.21142 livingstone, s., bober, m., & helsper, e. j. (2005). active participation or just more information? young people's take-up of opportunities to act and interact on the internet. information, communication & society, 8(3), 287-314. lonn, s., & teasley, s. d. (2009). saving time or innovating practice: investigating perceptions and uses of learning management systems. computers & education, 53(3), 686-694. doi:10.1016/j.compedu.2009.04.008 malone, d. w. (1975). an introduction to the application of interpretive structural modeling. proceedings of the ieee, 63(3), 397-404. mandal, a., & deshmukh, s. g. (1994). vendor selection using interpretive structural modelling (ism). international journal of operations & production management, 14(6), 5259. markus, m. l., manville, b., & agres, c. e. (2000). what makes a virtual organization work? sloan management review, 42(1), 13-26. marriott, p. (2010). a longitudinal study of undergraduate accounting students' learning style preferences at two uk universities. accounting education, 11(1), 43-62. maslow, a. h. (1943). a theory of human motivation. psychological review, 50(4), 370-396. miers, j. (2004). belts or braces? is belts, (basic e-learning tool set) retrieved from http://www.teachers.ash.org.au/jmresources/research/04researchreport.pdf napier, n. p., keil, m., & tan, f. b. (2009). it project managers' construction of successful project management practice: a repertory grid investigation. information systems journal, 19(3), 255-282. doi:10.1111/j.1365-2575.2007.00264.x parker, b. j., & plank, r. e. (2000). a uses and gratifications perspective on the internet as a new information source. american business review, 18(2), 43-49. parkes, m., stein, s., & reading, c. (2015). student preparedness for university e-learning environments. internet & higher education, 25, 1-10. pena-shafe, j., altman, w., & stephenson, h. (2005). asynchronous online discussion as a tool for learning: students' attitudes, expectations, and perceptions. journal of interactive learning research, 16(4), 409-430. australasian journal of information systems stevens, guo & li 2018, vol 22, research on educational technologies students’ motivations to use technology in learning 25 phang, c. w., kankanhalli, a., ramakrishnan, k., & raman, k. s. (2010). customers' preference of online store visit strategies: an investigatin of demographic variables. european journal of information systems, 19, 344-358. pieters, r., baumgartner, h., & allen, d. (1995). a means-end chain approach to consumer goal structures. international journal of research in marketing, 12, 227-244. prensky, m. (2001). digital natives, digital immigrants. on the horizon, 9(5), 1-6. punj, g., & stewart, d. w. (1983). cluster analysis in marketing research:review and suggestions for application. journal of marketing research, 20(2), 134-148. qureshi, m. n., kumar, d., & kumar, p. (2008). an integrated model to identify and classify the key criteria and their role in the assessment of 3pl services providers. asia pacific journal of marketing and logistices, 20(2), 227-249. reynolds, t. j., & gutman, j. (1984). advertising is image management. journal of advertising research, 24(1), 27. rice, r. e. (1987). computer-mediated communication and organizational innovation. journal of communication, 37(4), 65-94. doi:10.1111/j.1460-2466.1987.tb01009.x roberts, j., hann, i.-h., & slaughter, s. (2006). understanding the motivations, participation, and performance of open source software developers: a longitudinal study of the apache projects. management science, 52(7), 984-999. rubin, a. m. (1983). television uses and gratifications: the interactions of viewing patterns and motivations. journal of broadcasting, 27(1), 37-51. sage, a. p. (1977). methodology for large-scale systems new york, ny: mcgraw-hill. selwyn, n. (2009). the digital native--myth and reality. aslib proceedings: new information perspectives, 61(4), 364-379. doi:10.1108/00012530910973776 sheldon, p. (2008). student favorite: facebook and motives for its use. southwestern mass communication journal, 23(2), 39-53. shih, m., feng, j., & tsai, c.-c. (2008). research and trends in the field of e-learning from 20012005: a content analysis of cognitive studies in selected journals. computers & education, 51(2), 955-967. siau, k., tan, x., & sheng, h. (2010). important characteristics of software development team members: an empirical investigation using repertory grid information systems journal, 20(6), 563-580. doi:10.1111/j.1365-2575.2007.00254.x smith, c., karayiannidis, y., nalpantidis, l., gratal, x., peng, q., dimarogonas, d. v., & kragic, d. (2012). dual arm manipulation—a survey. robotics & autonomous systems, 60(10), 1340-1353. spires, h., lee, j., turner, k., & johnson, j. (2008). having our say: middle grade students' perspectives on school, technologies, and academic engagement. journal of research in technology & education, 40(4), 497-515. stafford, t. f. (2005). understanding motivations for internet use in distance education. ieee transactions on education, 48(2), 301-306. doi:10.1109/te.2004.842904 australasian journal of information systems stevens, guo & li 2018, vol 22, research on educational technologies students’ motivations to use technology in learning 26 stafford, t. f., stafford, m. r., & schkade, l. l. (2004). determining uses and gratifications for the internet. decision sciences, 35(2), 259-288. tan, f. b., & gallupe, r. b. (2006). aligning business and information systems thinking: a cognitive approach. ieee transactions on engineering management,, 53(2), 223-237. doi:10.1109/tem.2006.872243 tan, f. b., & hunter, m. g. (2002). the repertory grid technique: a method for the study of cognition in information systems. mis quarterly, 26(1), 39-57. doi:10.2307/4132340 tan, f. t. c., guo, z., cahalane, m., & cheng, d. (2016). developing business analytic capabilities for combating e-commerce identity fraud: a study of trustev’s digital verification solution. information & management, 53(7), 878-891. tao, y.-h. (2008). typology of college student perception on institutional e-learning issues-an extension study of a teacher's typology in taiwan. computers & education, 50(4), 14951508. tapscott, d. (1998). growing up digital: the rise of the net generation. new york: mcgraw-hill. tapscott, d. (1999). educating the net generation. educational leadership, 56(5), 6-11. thompson, k. e., & chen, y. l. (1998). retail store image: a means-end approach. journal of marketing practice: applied marketing science, 4(6), 161-173. tung, l. l., xu, y., & tan, f. b. (2009). attributes of web site usability: a study of web users with the repertory grid technique international journal of electronic commerce, 13(4), 97-126. doi:10.2753/jec1086-4415130405 van den beemt, a., akkerman, s., & simons, r.-j. (2010). the use of interactive media among today's youth: results of a survey. computers in human behavior, 26(5), 1158-1165. vivek, s. d., banwet, d. k., & shankar, r. (2008). analysis of interactions among core, transaction and relationship-specific investments: the case of offshoring journal of operations management, 26, 180-197. vodanovich, s., sundara, d., & myers, m. (2010). digital natives and ubiquitous information systems. information systems research 21(4), 711-723. wagner, t. (2007). shopping motivation revised: a means-end chain analytical perspective. international journal of retailing & distribution management, 35(7), 569-582. wakefield, r. l., wakefield, k. l., baker, j., & wang, l. c. (2011). how websites socialness leads to website use. european journal of information systems, 20, 118-132. waller, r. j. (1975). application of interpretive structural modeling to priority-setting in urban systems management. in m.baldwin (ed.), portraits of complexity (vol. 9, pp. 104-108). columbus, oh: battelle memorial institute. warfield, j. n. (1974). toward interpretation of complex structural models. ieee transactions on systems, man and cybernetics, 4(5), 405-417. warfield, j. n. (1976). societal systems: planning, policy and complexity. new york: john wiley and sons. warfield, j. n. (1994). a science of generic design: managing complexity through system design. ames, ia: iowa state university press. australasian journal of information systems stevens, guo & li 2018, vol 22, research on educational technologies students’ motivations to use technology in learning 27 westbrook, r. a., & black, w. c. (1985). a motivation-based shopper typology. journal of retailing, 61(1), 78-103. wheeless, l. r., & grotz, j. (1976). conceptulization and measurement of reported selfdisclosure. human communication research, 2(4), 338-346. doi:10.1111/j.14682958.1976.tb00494.x wilson, t. d. (2000). human information behavior. informing science, 3(2), 49-56. yates, d., wagner, c., & majchrzak, a. (2010). factors affecting shapers of organizational wikis. journal of the american society for information science and technology, 61(3), 543-554. doi:10.1002/asi.21266 australasian journal of information systems stevens, guo & li 2018, vol 22, research on educational technologies students’ motivations to use technology in learning 28 appendix a. a s1 s2 s3 s4 s5 s6 s7 s8 s9 s10 s11 s1: access and content control 0 0 1 0 1 0 1 1 1 1 0 s2: accessibility 0 0 1 0 0 0 0 0 1 0 0 s3: communication efficiency 0 0 0 0 0 0 0 0 0 0 0 s4: communication mode 0 0 0 0 1 0 0 1 1 0 1 s5: communication quality 0 0 0 0 0 0 0 0 0 0 0 s6: course management 0 1 1 0 0 0 0 0 1 0 0 s7: information seeking 0 0 1 0 1 0 0 0 0 0 0 s8: interaction 0 0 1 0 1 0 1 0 1 0 0 s9: learning capability 0 0 0 0 0 0 0 0 0 0 0 s10: managing contents 0 0 0 0 0 0 1 1 1 0 0 s11: self-disclosure 0 0 0 0 0 0 0 1 1 0 0 table a1: cluster 1 adjacency matrix a s1 s2 s3 s4 s5 s6 s7 s8 s9 s10 s11 s1: access and content control 0 1 1 0 1 0 1 1 1 1 1 s2: accessibility 0 0 1 0 1 0 0 1 0 0 0 s3: communication efficiency 0 0 0 0 0 0 0 0 0 0 0 s4: communication mode 0 0 1 0 1 0 0 1 0 0 1 s5: communication quality 0 0 0 0 0 0 0 0 0 0 0 s6: course management 0 0 0 0 0 0 0 1 0 0 0 s7: information seeking 0 0 0 0 1 0 0 0 0 0 0 s8: interaction 0 0 1 0 1 0 1 0 1 0 1 s9: learning capability 0 0 0 0 0 0 0 0 0 0 0 s10: managing contents 0 0 1 0 0 0 1 1 1 0 0 s11: self-disclosure 0 0 0 0 1 0 1 1 0 0 0 table a2: cluster 2 adjacency matrix australasian journal of information systems stevens, guo & li 2018, vol 22, research on educational technologies students’ motivations to use technology in learning 29 appendix b. level si r（si） a（si） r∩a ⅴ 1 1, 3, 5, 7, 8, 9, 10 1 1 ⅱ 2 2, 3, 9 2, 6 2 ⅰ 3 3 1, 2, 3, 4, 6, 7, 8, 10, 11 3 ⅴ 4 3, 4, 5, 7, 8, 9, 11 4 4 ⅰ 5 5 1, 4, 5, 7, 8, 10, 11 5 ⅲ 6 2, 3, 6, 9 6 6 ⅱ 7 3, 5, 7 1, 4, 7, 8, 10, 11 7 ⅲ 8 3, 5, 7, 8, 9 1, 4, 8, 10, 11 8 ⅰ 9 9 1, 2, 4, 6, 8, 9, 10, 11 9 ⅳ 10 3, 5, 7, 8, 9, 10 1, 10 10 ⅳ 11 3, 5, 7, 8, 9, 11 4, 11 11 table b1: cluster 1 level partition level si r（si） a（si） r∩a ⅴ 1 1, 2, 3, 5, 7, 8, 9, 10, 11 1 1 ⅳ 2 2, 3, 5, 7, 8, 9, 11 1, 2 2 ⅰ 3 3 1, 2, 3, 4, 6, 8, 10, 11 3 ⅳ 4 3, 4, 5, 7, 8, 9, 11 4 4 ⅰ 5 5 1, 2, 4, 5, 6, 7, 8, 10, 11 5 ⅳ 6 3, 5, 6, 7, 8, 9, 11 6 6 ⅱ 7 5, 7 1, 2, 4, 6, 7, 8, 10, 11 7 ⅲ 8 3, 5, 7, 8, 9, 11 1, 2, 4, 6, 8, 10, 11 8, 11 ⅰ 9 9 1, 2, 4, 6, 8, 9, 10, 11 9 ⅳ 10 3, 5, 7, 8, 9, 10, 11 1, 10 10 ⅲ 11 3, 5, 7, 8, 9, 11 1, 2, 4, 6, 8, 10, 11 8, 11 table b2: cluster 2 level partition australasian journal of information systems stevens, guo & li 2018, vol 22, research on educational technologies students’ motivations to use technology in learning 30 appendix c m s3 s5 s9 s2 s7 s6 s8 s10 s11 s1 s4 s3 1 0 0 0 0 0 0 0 0 0 0 s5 0 1 0 0 0 0 0 0 0 0 0 s9 0 0 1 0 0 0 0 0 0 0 0 s2 1 0 1 1 0 0 0 0 0 0 0 s7 1 1 0 0 1 0 0 0 0 0 0 s6 1 0 1 1 0 1 0 0 0 0 0 s8 1 1 1 0 1 0 1 0 0 0 0 s10 1 1 1 0 1 0 1 1 0 0 0 s11 1 1 1 0 1 0 1 0 1 0 0 s1 1 1 1 0 1 0 1 1 0 1 0 s4 1 1 1 0 1 0 1 0 1 0 1 table c1: cluster 1 canonical matrix m s3 s5 s9 s7 s8 s11 s2 s4 s6 s10 s1 s3 1 0 0 0 0 0 0 0 0 0 0 s5 0 1 0 0 0 0 0 0 0 0 0 s9 0 0 1 0 0 0 0 0 0 0 0 s7 0 1 0 1 0 0 0 0 0 0 0 s8 1 1 1 1 1 1 0 0 0 0 0 s11 1 1 1 1 1 1 0 0 0 0 0 s2 1 1 1 1 1 1 1 0 0 0 0 s4 1 1 1 1 1 1 0 1 0 0 0 s6 1 1 1 1 1 1 0 0 1 0 0 s10 1 1 1 1 1 1 0 0 0 1 0 s1 1 1 1 1 1 1 1 0 0 1 1 table c2: cluster 2 canonical matrix copyright: © 2018 stevens, guo & li. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 literature review 2.1 students’ motivations for using technologies in learning 2.2 motivation-based student technology use typology 2.3 hierarchical frameworks of technology use motivations 3 research method 3.1 data collection method—rgt 3.1.1 element selection: 3.1.2 construct elicitation: select triad, elicit raw constructs, and ladder 3.2 sample and data collection 3.3 data consolidation and categorisation 3.4 data analysis techniques 4 results 4.1 a typology of students based on the motivations for using technologies in learning 4.2 hierarchical frameworks developed using ism 5 discussion 5.1 independent vs. traditional learners 5.2 hierarchical frameworks of independent and traditional learners 5.3 importance of interaction and the use of technology to manage learning content 6 implications 6.1 implications for research 6.2 implications for practice 7 limitations and future directions paper title (paper title style) australasian journal of information systems parsons 2016, vol 20, research on rural & remote ict broadband initiatives in education the social surplus of broadband initiatives in compulsory education david parsons the mind lab by unitec auckland, new zealand d.p.parsons@massey.ac.nz abstract in 2010, the new zealand government embarked upon an ambitious programme of broadband infrastructure investment, a process that will continue until at least 2019. part of this investment is specifically targeted at compulsory education, with initiatives that include bringing fibre connections to the school gate, supporting on-site network upgrades (including wireless) and providing teaching, learning and support services delivered through these networks. such investments are not made without some projections of the likely rate of return, but calculating return on investment (roi) in educational broadband is complex, as it encompasses a range of factors. this article reports on an interview-based study engaging a range of stakeholders in educational broadband provision. the study utilises a research model that considers the various elements of social surplus, namely; producer surplus (savings), producer surplus (profit) and consumer surplus (perceived value over and above cost), to explore the elements of social surplus that have been used to define educational broadband roi calculations and justify the scale of investment. the results indicate that all three components of social surplus are relevant, though the concept of profit can only be seen in the broader context of long term contributions to the economy. a note of caution is that projections of roi based only on positive returns fail to acknowledge the potential for some innovations to actually increase costs. further, purely quantitative models do not properly take into account qualitative components of consumer surplus. keywords: broadband, education, return on investment (roi), social surplus 1 introduction initiatives to extend broadband internet access are common worldwide, and governments as well as industry are supporting the deployment of broadband networks. these investments are not just seen as commercial initiatives. the digital agenda for europe, for example, aims to expand broadband access to enhance digital literacy, skills and inclusion (european commission, 2015.) there seems to be little resistance to the idea that such initiatives have social as well as economic benefits, including the reduction of digital divides. the main focus of debate is therefore the best means to realise these potential benefits of broadband (bauer, kim & wildman, 2005.) there is also a view that the most effective intervention strategy for enacting broadband policy, in the light of the empirical analysis, is a medium-intervention strategy, whereby governments guide and support, but do not entirely dictate, how broadband access should be expanded (cava-ferreruela & alabau-muñoz, 2006.) cambini & jiang (2009) broadly classify three regulatory approaches as having regional characteristics, namely; deregulation in the u.s., intervention in east asia and a middle way between these approaches in the e.u., thus medium intervention may be regarded as a european model. however, it should be noted that bouras, giannaka &tsiatsos (2009) saw a more diffuse set of exemplars for broadband growth, where european nations were prominent but not exclusive. nevertheless, supporting synergies between private and public sectors was identified as a best practice. this type of strategy requires cooperation between governments and commercial institutions, particularly in cases where the installation of broadband infrastructure cannot be driven purely by short term profit, such as in education (the focus of this particular study) and regional development. whenever long term national strategy is impacted by broadband initiatives, it becomes necessary for governments to become co-investors in deployment. in large jurisdictions, some of this type of development may be undertaken by municipal entities 1 australasian journal of information systems parsons 2016, vol 20, research on rural & remote ict broadband initiatives in education (feld et al 2005.) this support from government to industry to achieve desired policy outcomes may also involve ownership and governance changes such as the unbundling of previously centralised infrastructure to encourage competition. this ‘private public interplay’ has been particularly important in australia and new zealand (given, 2010.) 1.1 return on investment and social surplus the provision of broadband infrastructure is an expensive undertaking, and neither governments nor corporations are likely to embark on such initiatives without positive return on investment (roi) calculations. roi is measured primarily in financial terms, but the source of these finances can be derived from either profit or savings. in addition, these profit and savings calculations (known as provider surplus) are complemented by other types of benefit. consumer surplus (what something is ‘worth’ over and above what it costs to consume) encompasses many different definitions of value, which are not always easy to quantify, since these consumer gains are often not directly reflected in higher incomes or increases in gdp. however the value of consumer surplus may include actual financial value to the consumer, for example in savings over alternative options. consumer surplus is combined with producer surplus to create social surplus, the total cost or benefit of a given investment. hitt and brynjolfsson (1996) found evidence for the hypothesis that the consumer surplus created by information technology in general is positive and growing over time. therefore we might expect it to provide a substantial contribution to the overall social surplus projected to be gained from a broadband initiative. measuring such returns can, however, be complex. for example, rates of return may be ‘u-shaped’ over a period, showing significant returns only at the early and later parts of an adoption cycle (holt & jamison, 2009.) thus the point at which rates of return are measured may have a large impact on the results of analysis. some returns are relatively immediate, while others may take years to emerge. a further concern is that producer and consumer surplus must be in balance. focusing only on consumer surplus leads to inadequate investment, while an emphasis only on producer surplus has negative effects on social welfare (competition economists group, 2015.) thus the research reported in this article takes account of a range of different factors in identifying the main components of roi, both short term and long term, in broadband policy and deployment. 1.2 the new zealand context new zealand is recognised as a valuable case study in progressive digital legislation (eaves, 2013.) new zealand’s economy has some unique characteristics but shares others with many developed nations. it has been transformed from an agrarian to a more industrialized, free market economy, but in the process has deepened socio-economic divisions, suffered from mediocre economic performance (auditor general, 2013) and is heavily indebted internationally (index mundi, 2014.) despite a good education system, this has not led to gains in productivity or innovation, and youth unemployment is high (oecd, 2014.) major business sectors have historically been in primary products and associated with rural regions, but employment in agriculture has been in decline. manufacturing remains the largest employer, but real estate provides the largest number of enterprises (trading economics, 2010; statistics new zealand, 2014.) these problems of changing economic patterns, balance of payments deficits and youth unemployment are challenges faced by large numbers of countries across the world, and radical changes to education are frequently seen as essential to address these challenges. recent major policy initiatives in new zealand in the area of broadband-enabled digital teaching and learning, intended to address some of these issues, may thus provide us with globally relevant insights into the value of broadband investment in the education sector. the new zealand government’s tertiary education strategy 2015-2019 classifies the desired outcomes from tertiary education as being economic, social and environmental. specifically, it states that: ‘this strategy focuses in particular on the economic benefits that result from tertiary education and therefore on employment, higher incomes and better access to skilled employees for business as critical outcomes.’ (moe 2014a.) 2 australasian journal of information systems parsons 2016, vol 20, research on rural & remote ict broadband initiatives in education although this strategy focuses only on tertiary education, these themes resonate across the whole value chain of education, since the tertiary sector does not operate independently of its incoming cohorts from secondary education. the central concern is economic benefit as measured by income, a commonly used measure of the value of education (psacharopoulos, 1994.) the reference to skilled employees also implicitly references the producer profit generated by higher productivity and possibly also an increased supply of skilled labour, thus reducing its cost due to the effect of supply and demand. of course there is a tension between higher incomes and a greater supply of skilled labour, which requires increased productivity to overcome (chang, 2011.) 1.3 educational broadband policy implementation in new zealand the study reported in this article focuses not on tertiary education, which has been well connected to broadband infrastructure for many years, but on schools. the reason for this is that the new zealand government is focusing its current educational technology efforts on connecting all of the country’s schools to ultra-fast broadband (ufb.) as part of a larger investment in ufb infrastructure, e-education is one of five points in the government action plan for faster broadband, along with e-health, e-government, e-business and e-development (commerce commission, 2012.) government departments are taking major initiatives in providing national broadband for schools and internal school networks (including wireless.) the overall investment in ufb is $1.5 billion, the government contributing $1.35 billion with private co-investment (mbie, 2012.) $28.2 million of this sum is for fibre connections from school boundaries into the schools. as a result of this infrastructure build, 97.7% of schools and 99.9% of students will receive ultra-fast broadband capability, with the remaining 2.3% of schools in remote areas given wireless or satellite services (moe, 2014b.) the ufb project brings fast internet connections into the school grounds, but does not directly address how that connection may be used inside a school. to address the next stage of broadband provision the school network upgrade project (snup) has the objective of upgrading internal school networks, and includes a wireless option (moe, 2015a.) the estimated cost of these network upgrades, which are mostly funded by the government, with some contribution from schools, is estimated at $600 per student. new zealand has around 750,000 school students, so if all schools were upgraded the approximate cost would be around $450 million. a further initiative is the network for learning (n4l), a $211 million government-owned agency tasked with providing teaching, learning and support services on these new broadband networks (moe, 2015b.) as of march 2015, more than half a million students and teachers were using n4l (williams, 2015.) thus the total technology investment in digital teaching, learning and administration across these three initiatives is somewhere in the region of $700 million over a period of 5 years, approximately .5% of annual gdp (nzte, 2015.) schools that have already been connected to fibre networks have reported positive effects, including better student engagement, improved learning outcomes (especially in literacy), more online collaboration, new distance learning opportunities and more effective assessment and administration (crown fibre holdings, 2013.) the challenge, however is to identify what types of roi might be embedded in these broad claims. given the scale of investment in broadband connectivity and related services specifically targeted at compulsory education (i.e. the school sector) both by the new zealand government and individual schools (who need to further invest in their own staff and services to take advantage of broadband connectivity) it is important to investigate the assumptions made around the expected returns on this investment. since education is a complex area where there are many different stakeholders with multiple perspectives, qualitative studies such as the one reported in this paper can capture these varying viewpoints and gain insights into return on investment calculations that are not evident in purely quantitative analyses. 3 australasian journal of information systems parsons 2016, vol 20, research on rural & remote ict broadband initiatives in education while the results of the study reported in this article largely support the positive assumptions made about the benefits of broadband in education, they provide a much richer and more nuanced set of insights than the sources cited above. importantly, they also call into question the extent to which ufb will make a positive contribution to social surplus. the study identifies some areas in which the proposed surplus may be negatively impacted by additional costs and lower than expected returns. the next section provides a review of the literature in educational broadband investment, various aspects of return on investment in this context and the research questions for the study. this is followed by sections on the research methodology and the results of the study. section five provides a discussion of these results and presents an extended model of roi for broadband initiatives in education. the paper concludes with a summary of the main findings and some proposals for future work. 2 literature review this review of the literature begins with a discussion around the definition of broadband, particularly within the context of educational provision. it also addresses the components of cost benefit analyses and considers the less easily evaluated aspects of return on investment in educational broadband. finally it addresses the impact of geography on educational broadband provision. 2.1 defining broadband the definition of broadband connectivity is not fixed across times and contexts, since both technology and requirements vary over time. the unesco report on technology, broadband and education (unesco, 2013a) takes as its definition a minimum download speed for public internet access of at least 256 kbps (kilobits per second.) however, it also notes that speed is the key determinant of the range of educational services that can be offered on line, and that services such as video on demand require much faster speeds of at least 2 mbps (megabits per second.) hayes (2011) classifies anything less than 2 mbps as too slow to be classified as broadband. he defines basic broadband as being between 2 and 12 mbps and fast broadband as greater than 12 mbps. a further issue is that broadband speeds are normally stated in terms of download traffic, since broadband is generally asymmetric (i.e. upload is slower than download.) this can be an important restriction in the educational context since learning is about the creation as well as the consumption of material, so upload speed can also be a key factor in the value of broadband provision. hayes’ (2011) definition of very fast broadband is that it is not only faster than 20 mbps but is also symmetric. recognising the importance of symmetric connections, the new zealand commerce commission (commerce commission, 2012) specifies a series of application requirements related to symmetric broadband speeds. at the lower end (500 kbps to 1mbps) are systems such as web browsing and voice over ip. however the minimum requirement for remote education is stated as 5 mbps to 10 mbps while full online educational services require 10 mbps to 50 mbps. in particular, high speed symmetric broadband is needed for the two-way traffic of cloud based applications, which are increasingly used by schools, for example google classroom (pappas, 2015.) in new zealand, the ultra-fast broadband and rural broadband initiatives aim to provide typical download speeds of up to 100 mbps, with up to 1 gbps (gigabits per second) in some areas. however, the technologies used to deliver broadband in different areas of the country vary between fibre, cable, vdsl (very-high-bit-rate digital subscriber line) adsl (asymmetric digital subscriber line) and wireless, meaning different broadband speeds depending on the type of connection (mbie, 2015.) it should be noted that there are many factors affecting connection speed but, for example, dsl connections are unlikely to deliver more than around 70 mbps, and may be much slower. wireless broadband speeds also vary considerably depending on technology. 4 australasian journal of information systems parsons 2016, vol 20, research on rural & remote ict broadband initiatives in education 2.2 return on investment from broadband infrastructure return on investment calculations are based on cost benefit analyses. these analyses are quantitative in nature and aim to calculate the financial gains and losses that might result from a particular investment. such calculations do not necessarily concern themselves with equality of result and distribution of gains (robson, 2014.) however, government investment in broadband needs to consider as widely as possible the economic, social and environmental impacts of policy. in new zealand the ufb initiative aims to benefit not just business but also education, health, government and development (commerce commission, 2012.) one important component of a cost benefit analysis is to consider whether a given investment will reflect an intermediate reference scenario, whereby the most efficient rollout of services is achieved. this compares with pessimistic scenarios, where development is too slow or too limited, and optimistic scenarios where there is inefficient over-investment (robson, 2014.) whilst it is probably too early to make this assessment in the new zealand context, newman (2012) reports that current bandwidth provision may be inadequate, suggesting that government and schools may face a pessimistic reference scenario. as described in the introduction, overall return on investment calculations are based on a combination of producer surplus and consumer surplus, which together provide an overall assessment of social surplus. cost benefit analyses apply to both consumer and producer surplus, and faster broadband may impact on both. relevant factors may include better or new use of products, savings in time, travel and transaction costs and improvements in productivity. in addition there may be welfare effects such as reducing the costs of supplying publicly provided private goods such as health and education (robson 2014.) education, in particular, may be regarded as a merit good. merit goods are those that may not have immediate calculable benefits but whose consumption is seen as positive by the state (gomezbarroso & perez-martinez, 2005.) when considering welfare effects, cost benefit analyses are complicated by potential conflicts between economic benefits or waste on the one hand and welfare gains or losses on the other (robson 2014.) 2.3 educational broadband investment investing in broadband provision for education takes account not only of simple cost-benefit analyses but of welfare gains and merit goods. thus long term policy planning is required, and assessing the likely components of social surplus is particularly important. although there are direct profit and loss calculations to be made in the school sector, such as the costs of provisioning and technical support staff, education is also a long term investment in the future economy and sustainable social structures. again, this is internationally recognised. unesco’s policy guidelines on mobile learning emphasise the importance of providing robust and affordable broadband and mobile networks for education that consider equity, safety, advocacy and resources (unesco, 2013b.) efforts to calculate the return on investment in education as a result of the rollout of ultra-fast broadband networks in new zealand estimated gains of $3.6 billion over 20 years, a figure that includes consumer surplus. the overall benefit of ufb over that period was estimated at $33 billion (pullar-strecker, 2012), so the projected contribution of education is around ten per cent. in the report from bell labs and alcatellucent new zealand, which proposed these figures, the main benefits to education of broadband investment were seen as lower costs for skill enhancement (primarily through remote learning,) reduced costs for course materials, savings on field trips and better school attendance (by 25 million pupil-days per year.) the latter estimate was based on assumptions that using remote learning applications and better collaboration between parents and teachers would lead to fewer days of school missed due to sickness and truancy (alcatel-lucent, 2012.) another perspective to the debate is the potential negative effects of not investing in educational broadband, where the social experiences of students become increasingly disconnected from their educational environment (rosen, 2014.) further, where the broadband industry is primarily gaining producer surplus through profit, an approach that mitigates against investment in non-commercial education, there is a consequent negative affect on the overall competitiveness of the economy in international terms (crawford, 2013.) 5 australasian journal of information systems parsons 2016, vol 20, research on rural & remote ict broadband initiatives in education a further consideration from the perspective of government policy is that a hands-off approach from government may lead to schools making their own arrangements at a higher cost, since governments are able to leverage the power of mass purchasing to reduce the overall costs of equipment and connectivity (unesco, 2013a.) 2.4 return on investment for broadband in education unesco (2013a) suggest a strong connection between broadband and ict policies and education worldwide, and that broadband can be an accelerator for equitable progress in many aspects of global society. underlying this belief is an assumption of consumer surplus that is not part of conventional government statistics, but includes educational or civic benefits that go beyond direct economic measurement (greenstein & mcdevitt, 2012.) if these benefits cannot be directly measured, how can they be assessed? crandall & jackson (2001), in considering the benefits of broadband in retailing, transportation, home entertainment and health care, asserted that they lacked the tools or vision to properly analyse the benefits in the education sector. nevertheless there are some areas susceptible to direct financial measures, for example in delivering merit goods such as education, overall economic benefits depend on whether those services are provided efficiently (robson, 2014.) hayes (2011) notes some longer term measurable benefits, including increased educational attainment for school leavers, based on exposure to ict potentially improving examination performance, and the efficiency of online education. however even here there are many complexities, for example the impact of network effects in the use of cloud computing in education. 2.5 geographical factors the variation in broadband speed received by consumers as a result of the communication technologies used is partly a consequence of geography. the size and topography of a country can make a significant difference to how easy it is to reach the population with fixed broadband, and how many areas can economically be connected using fibre. a number of small island nations have already been able to provide fixed broadband connections to all of their primary and secondary schools, while uruguay has been able to provide fixed broadband to 95% of primary schools and all secondary schools. in contrast many other countries, such as colombia, require extensive wireless infrastructure to be able to provide broadband access to their schools (unesco, 2013a.) in new zealand, although the population is largely concentrated in a few cities, particularly auckland, there are many remote rural communities. the rural broadband initiative (rbi) aims to provide fixed broadband connectivity to 937 rural schools. in addition, the remote schools broadband initiative (rsbi) will provide wireless broadband services to new zealand’s 57 most remote schools (commerce commission, 2012). new zealand’s mixed pattern of provision, with fibre in the main centres and dsl and wireless covering other areas, provides a geographical model similar to many other nations, thus providing a more relevant international case study than, for example, a small island state (nzrs, 2015.) 2.6 research context and questions a review of the literature makes it clear that broadband provision is internationally recognised as being potentially of benefit to education. however it is also evident that the estimation of return on investment in the sector is complex, relying as it does on a number of longer term and difficult to calculate components of social surplus. nevertheless the significant costs involved mean that such questions need to be addressed as accurately as possible. situating this question in the new zealand context may prove informative to other nations. the literature shows that the way broadband connectivity is provided to schools varies with geography and national policy, but that new zealand’s complex mix of physical environments and public-private partnership on the european model provides an informative case study for other developed nations. from a review of the relevant literature, therefore, the following research questions were derived. 6 australasian journal of information systems parsons 2016, vol 20, research on rural & remote ict broadband initiatives in education 1. what assumptions underlie return on investment projections in the provision of broadband infrastructure to schools? 2. how can the components of social surplus be defined within the provision of educational broadband? 3 research methodology given the discussion around the components of social surplus, which are complex, long term and focused on merit goods, the research questions outlined at the end of the literature review were interpreted as largely qualitative questions. a number of quantitative estimates have been cited in this paper regarding broadband investment in education and expected rates of return. no doubt in time these figures will be confirmed or revised in government reports. what is less evident in the literature is an evaluation of the qualitative aspects of social surplus, despite the fact that much of the literature highlights their importance when dealing with merit goods such as education. to address this apparent lack of prior work, a qualitative interview study was designed to address how broadband policy in relation to the school system in new zealand might see a return on investment. figure 1 shows the research model used for the study, derived from the core components of roi that appear in the literature. in this model, educational broadband policy seeks justification in a return on investment calculation that has multiple components. that calculation needs to be defined by a given set of assumptions that are deemed relevant by policy makers. as previously discussed, the sum of components in the roi calculation can be termed social surplus, a sum that may be derived from three specific sources; producer profit, producer savings and consumer surplus. the research model was used both to help formulate the semi structured interview questions and also to guide the thematic coding process that was used to analyse the interview data, identifying how different aspects of social surplus might be identified in the context of education-related broadband policy in new zealand. it is important to note that this research concerns itself with social rates of return, which are used to justify investment in educational services, as opposed to private rates of return, which are concerned with how people invest in their own education. the two are, however, linked, since the higher the level of public subsidy for education, the higher the private rate of return (psacharopoulos, 1994.) further, as discussed later, we identified some private rates of return for lower socio-economic groups that were not directly related to formal education but were enabled by investments in educational broadband infrastructure. savings profit producer surplus consumer surplus social surplus roi calculation educational broadband policy contributes to contributes to contributes to contributes to defines justifies fig. 1. research model: roi for broadband initiatives in education 3.1 data description the study gathered data from a range of stakeholders from national government, local government, educational institutions and commercial organisations. since the study was focused on the new zealand context, all the interviewees were based in new zealand (auckland, wellington and hamilton.) the purpose of these interviews was to gather the 7 australasian journal of information systems parsons 2016, vol 20, research on rural & remote ict broadband initiatives in education perspectives of multiple informed stakeholders in the rollout of broadband to new zealand schools and to compare these perspectives with the published literature. a purposeful sample of policy makers and other relevant stakeholders (partly selected via snowball sampling) was chosen from a range of stakeholder institutions, and each of these representatives was interviewed using a semi-structured interview. in total, ten stakeholders were interviewed. each of these interviews took approximately one to one and a half hours, and was conducted face to face at the interviewee’s place of work, where possible (7 interviews.) where the interviewees could not be met face to face due to location, they were interviewed either by video call (2) or by telephone (1). the interviewees who contributed to the study represented the following organisations; national government (ministry of education), local government (auckland), secondary school, commercial internet service provider, educational trust, educational research institution, crown agency, industry organization and multinational mobile communications provider. the interviews took place in late 2013 and the first half of 2014, with follow up communication via email where necessary. in the interviews, the stakeholders were asked to respond to a series of semi-structured interview questions around aspects of roi in educational broadband. in addition to asking general questions around national ict policy, return on investment, levels of government investment in the broadband market and components of consumer surplus, respondents were also asked to discuss investment in 21st century skills (including digital skills) and matching the skills of school leavers to the employment market. the final question asked them to comment on the future direction of education policy relating to ict. 3.2 data analysis although the research model (figure 1) has positivist characteristics due to the economic axioms of social surplus, the investigation process took a naturalistic approach, gathering qualitative data from multiple perspectives. the emphasis of the questions was on the social realities of broadband initiatives in education, primarily addressing ̔ what’-type questions with a fixed set of enquiries. a standard iterative analysis was applied to the data, beginning with data reduction (the interviews were recorded, transcribed and coded in nvivo) then seeking meaning through data display (supported by nvivo tools), and finally drawing some conclusions from the data (folkestad, 2010.) the main categorical variable in the data was the type of institution represented by each stakeholder. 4 results the results of the interviews are presented here within the main components of the research model. first, producer surplus derived from savings is addressed, followed by producer surplus derived from profit. then, consumer surplus is addressed. finally, their relative contribution to social surplus is considered, along with private rates of return that were not included in the original model but emerged from the data. 4.1 producer surplus (savings) given that state supported education is not directly intended as a profit generating enterprise, most of the producer surplus in providing broadband access to schools is in potential savings. in this context, the primary producer is the government who are investing in infrastructure, measuring savings against what would be projected expenditure in the absence of broadband investment. the aspect of savings explicitly stated in the five point government action plan for faster broadband is shared procurement of internet access education services at a better price for schools (mbie, 2011.) reinforcing this message, the central government representative in the interview study emphasised the ‘benefits of centralised procurement to drive down costs,’ which they estimated at twenty per cent of the overall return on investment. it may be, however, that such centralised procurement may not always lead to the projected savings, because of the way that schools manage their own procurement policies: 8 australasian journal of information systems parsons 2016, vol 20, research on rural & remote ict broadband initiatives in education ‘schools don't typically like central control. there’s an all of government scheme for purchasing computers. we’ve got two and a half thousand odd schools, less than thirty have signed up, and yet it’s cheaper. they like to do their own thing.’ (internet provider) another potential issue with the projection of savings from centralised procurement is the degree of specification. this can be either too little or too much depending on your point of view. for example one comment was made regarding the implementation of the broadband network itself, suggesting that higher specifications could have been considered; ‘it would've been useful to have had a little more input from the market … seems a little like we’ve undercooked it’ (crown agency) this implication of a pessimistic reference scenario echoes comments referenced earlier from newman (2012.) nevertheless, ufb investment is expected to raise new zealand from its 2013 ranking of 27th in the world in terms of internet speeds and 24th in overall broadband quality, with fibre internet speed 10 to 20 times greater than many current copper connections (crown fibre holdings, 2013.) there is also a trade off in terms of investment between reaching extremely high speeds in some places (e.g. one gigabit per second) or reaching a guaranteed minimum speed across all connections, such as the european commission’s goal of 100% coverage of at least 30 megabits per second (shankland, 2014.) as discussed in the literature review section, broadband provision in new zealand will rely on a range of different technologies, not just fibre, and the most remote areas will have wireless connections. the investment tradeoffs mean deciding where the boundaries will lie between fibre and copper connections, and between fixed and wireless connections. the government plans for education thus follow the european model of guaranteeing a minimum speed across all connections. from the other perspective, that centralised procurements may be over-specified, a comment regarding the procurement of on-site network systems for schools stated: ‘some of the ministry schemes are totally over specified…the school is paying 20% of something they shouldn't be paying for’ (internet provider) this seems to indicate an optimistic reference scenario, albeit in a different context (robson, 2014.) it is worth noting, therefore, that centralised procurement may impact not only on the performance of the systems that are being used, but also on their costs, impacting negatively on the savings component of the producer surplus. another area where savings were identified was ‘streamlining school administration, moving a lot of school systems onto the cloud’ (local govt.) in fact the potential saving ‘around streamlining and making administration easier for schools’ was estimated at around thirty per cent of the overall roi by the national government stakeholder. this reduction assumes savings in the overhead of supporting in-house systems, for example hardware, software and support staff costs, by using cloud-based infrastructure. even more basic savings, such as a reduction in printing costs, also make a contribution. ‘our policy is taking away the paper … we won’t print our reports any more and we won’t even e-mail them. you've got to come and get them off our network, and you can get them any time you like’ (school) it should be noted at this point that one might regard cost savings within schools as part of the consumer surplus rather than producer surplus. however, given that schools are government funded, and the ultimate consumers are the students within the schools and their families, school savings are included here as part of the producer component of surplus. either way they contribute to social surplus. part of the hardware and software savings are based on the ability of a broadband wireless infrastructure to support a bring your own device (byod) policy in schools. using student owned devices reduces the need for schools to provide ict hardware and software for their pupils. 9 australasian journal of information systems parsons 2016, vol 20, research on rural & remote ict broadband initiatives in education reduction of overheads was also identified in terms of the way that teaching and learning could be provided in rural schools: ‘one of the immediate benefits [of a fibre network] was to reduce the number of hours that kids spent on buses being shipped from one school to another so they could receive specialist teaching’ (crown agency) another aspect of savings is the availability of free resources on line. ‘it's all here, it’s on youtube, it’s on other people’s blogs and twitter, join the google+ communities. it’s all here, you've just got to access it.’ (trust) it should be noted, however, that the interviewee here was not addressing all educational resources, but speaking in particular about a set of specific resources for teachers. there is another aspect to the cost of resource access, which is that in some cases the move from older media to online resources may in fact be more costly over time. ‘the school has a $10,000 spend a year, say, on textbooks, where we get five years of use. the model the publishers are talking about the moment is; we will give you access to our textbook for one year and then it disappears, which adds up to a lot more than $10,000. much more expensive’ (school) thus, again, we cannot count on a move towards a ufb infrastructure as representing opportunities to generate savings regardless of content. there may, in fact, be additional costs. if savings are being made, they are more likely to be made by providers of educational material: ‘from a cost perspective once you create a piece of mobile learning you can put it out to people with a one off cost’ (mobile provider) such providers need not, of course, be third party enterprises. however, the likelihood of material developed by teachers independent of professional publishing houses being generally useful is questionable: ‘from a system of publishing and editing that has filters, interventions, changes and modifications to a more individual thing, probably the finished product will be much more varied, from absolutely brilliant to; this works well in my class but it doesn't work for anyone else’ (school) from the responses of the interviewees, it can be seen that the role of savings in the producer surplus is expected to make a major contribution to the overall return on investment. in some areas, such as the reduction in printing costs, byod programmes and elimination of some travel, savings can be easily identified. however, it is clear that savings from activities such as centralised procurement and use of digital resources are by no means automatic, and in some cases there may in fact be increased costs rather than savings. it should also be recognised that savings might impact on the consumer surplus. for example parents who have to buy a digital device to support a byod programme may have a reduced assessment of overall value, and students whose face to face experiences are replaced by virtual experiences may again feel a loss of value. related issues are highlighted by newman (2012) who notes that a byod policy does not mean existing hardware in schools is no longer required, since byod provides only a complementary function and whether schools choose to become “hardware schools” or “web schools”, they will still have to maintain multiple infrastructures in the short term. 4.2 producer surplus (profit) estimating profit from state-sponsored education may not be directly relevant, but in the broader national perspective there is a potential contribution to future productivity and downstream profits though support of the knowledge economy. ‘the return on investment is a better educated, better prepared, graduating cohort’ (school) 10 australasian journal of information systems parsons 2016, vol 20, research on rural & remote ict broadband initiatives in education including this aspect of return from a profit perspective assumes that education can make a direct contribution to the economic well-being of the nation. producer profit from a government perspective is therefore predicated on higher tax incomes and a more economically productive economy. certainly the most relevant aspect of this is the readiness of those graduating from education to make a contribution to the economy. one area where it is possible to directly identify a contribution is in ensuring that graduating students have relevant digital skills. it is recognised that there is a shortage of skilled ict workers in new zealand. ‘right now we have a deficit of 15,000 [it people] how do we actually urgently and directly fix this situation? or do we forget about being a technology enabled country?’ (industry) certainly the provision of broadband into educational institutions is one of many key enablers for ensuring that the new zealand it industry is able to find the staff that it needs in the future. if course it is not the only one. pilkington (2008) notes that it may take time for teachers to begin to fully leverage ict in teaching and learning. there is also the possibility that, as ict use becomes the norm in schools, lessons to specifically teach ict skills will actually feature less on school timetables (underwood et al., 2005.) broadband provision is just the first step in the skills value chain, but actual returns are dependent on further developments in curricula, teaching and learning (unesco, 2013a.) in seeking to measure such outcomes, there is a general acceptance that social investment in education yields a social rate of return and that higher levels of education in general are reflected in higher earnings (psacharopoulos, 1994.) it is also clear that the it profession is relatively well paid in new zealand, where it was the second highest paid profession in 2014 (newson, 2014.) investment in educational broadband makes a direct (if partial) contribution to preparing the foundation for such careers. there are, however, some further concerns here. it is not clear to what extent higher earnings directly impact on national productivity (psacharopoulos, 1994.) there is also a concern that more technologically developed economies may actually need fewer educated people as a result of mechanisation (chang, 2011.) ultimately; ‘what really matters in the determination of national prosperity is not the education levels of individuals but the nation’s ability to organise individuals into enterprises with high productivity’ (chang, 2010, p.179.) thus we might conclude that, whilst educational investment overall might not always contribute to productivity, specific forms of investment that prepare future workers for productive enterprises, such as broadband infrastructure, can indeed support national prosperity. it is essential that school students are made familiar with the world of technology that they will meet in the workplace; ‘for our learners to be able to move confidently into the workplace they need to be at home in the digital world’ (trust) ‘no one is going to wake up on monday morning and turn the internet off. the world is moving that way, so it's ensuring all young people are moving that way as well…the payback is how successful we are in terms of the economy in the world.’ (school) this doesn’t just mean delivering specific technical skills that may or may not be demanded by a particular employer, but a broad, contemporary skill set that is aware of the relevance and use of technology. ‘most schools are in urban environments. the kids are going to go all over the place, so we can't predict a particular industry. it's got to be confidence, responsibility, “technology doesn't faze me”’ (school) an important aspect of this preparedness of entrants to the workforce for using technology is that the level of education of the population is a critical enabler for all internet and high technology applications (koutroumpis, 2009.) however we should also note that according to 11 australasian journal of information systems parsons 2016, vol 20, research on rural & remote ict broadband initiatives in education psacharopoulos (1994) the social rate of return of education in technical and scientific areas is actually lower than that for academic and general areas. in analysing the results of the interview data related to consumer surplus (profit) it is clear that this is difficult to measure in the short term, but there is a general consensus that education is critical to a technologically developed economy. however, education alone is not the only factor, indeed specifically technical education may not be as important in deriving long term rates of return as the provision of a broad contemporary skill set. ultimately, profit will be dependent on education being able to support the creation and sustenance of productive enterprises. one of the aims of the evolving digital school curriculum is to foster the kinds of creativity that can meet the needs of future enterprises. ‘enabling those kids to express themselves and really create away and be able to get credit for it … and then find a pathway into further study or out into the workforce’ (govt.) of course such reductionist views might be anathema to those to see education as much more than just preparation for the workforce. it should therefore be noted that these reflections are confined to the producer surplus (profit) element of the research model, and are not intended to address all aspects of educational broadband investment. 4.3 consumer surplus (social rates of return) the research model in figure 1 assumed that consumer surplus, in the context of this study, would be founded on social rates of return. however, the interview data indicated that private rates of return were also an important factor. in this section, social rates of return are discussed, with private rates of return covered in the following section. it might well be expected that consumer surplus has a large role to play in calculating the social surplus of investment in education. as indicated previously, in this study the primary ‘consumer’ is the school student, though as will be seen from some of the results, the concept of consumer also includes wider society. according to the central government interviewee, ‘half [of the roi is] within improved teaching and learning … leading therefore to better student outcomes.’ (govt.) if half the projected roi from educational broadband is consumer surplus in the context of teaching and learning, this is a significant area for investigation if we wish to identify the specific details of these roi assumptions. there is certainly a belief that ufb brings new opportunities that were not possible in a more traditional learning environment. ‘opportunities for learning as a young person are exponentially improved by ufb’ (trust) however, it is important that we seek to identify specific areas where broadband investment is bringing additional value to the consumer. one aspect that certainly gets attention is improvements the personal efficiency of educational delivery. for example, ‘[after the christchurch earthquake] the school itself wasn't functional but people would get on the internet and return to study’ (local govt.) another perspective on efficiency was that students who needed to be away from school for family or sporting events would be able to maintain their studies. ‘taking time out of school to go somewhere special, to be part of a sports competition, a special holiday with parents…now students can do that knowing that they can just access the work easily, so in terms of mobility of students and parental decisions…that will become a lot easier for parents and kids to do.’ (school) the question that may be raised by such concepts of efficiency is whether more is necessarily better or, perhaps more accurately, whether learning anywhere, anytime, is equal to learning in a classroom with peers and a face to face teacher. it is notable that education is one area where ‘output’ increases markedly as a result of broadband. koutroumpis (2009) relates this 12 australasian journal of information systems parsons 2016, vol 20, research on rural & remote ict broadband initiatives in education specifically to broadband enabling e-learning. one of the underlying assumptions is that there are direct benefits in terms of personal learning efficiency in situations where peers and teachers are not so significant, that is, in the context of immediate problem solving. ‘if you can get better access to information then you’re going to be able to solve problems or educate yourself with it…10-15 years ago you had to go to library. now i have that in my pocket.’ (mobile provider) of course the transition to digital provision of material does not just mean using a different channel to deliver the same content. there is also an expectation that a broadband infrastructure enables the provision of better, as well as more accessible, learning resources and tools. ‘moving a lot of school systems onto the cloud… gives you richer systems’ (local govt.) another important perspective is the potential benefits to wider society of improvements in the behaviour of school age children. ‘kids come to school early. there is a decline in truanting, in in-class off-task behaviour and other anti-social things’ (govt.) in previous sections addressing components of producer surplus we have seen that not all changes enabled by broadband infrastructure necessarily result in surplus. in fact, there may be the potential to actually increase costs. in the context of consumer surplus, that may also be the case. changes to educational processes and tools may impact negatively on existing value chains. for example, old modes of assessment will soon be outdated. ‘in about five years students will be able to do their assessments online any time they feel like it so that has huge implications for how the learning in schools is structured’ (researcher) such changes in structure may be beneficial to students, but equally they may put additional pressures on schools. radical change also impacts on the systems that educate staff, as well as students; ‘i think it changes the whole nature of how we train teachers’ (school) in fact, such changes might lead to consumer surplus, but at the cost of producer surplus, in that major change to systems of assessment and teacher training will be expensive to implement. again, one important message is that surplus is not automatic, or at least may vary by context. here, we might refer again to holt & jamison (2009) and the ‘u-shaped’ rates of return, where transition costs may be high. 4.4 social surplus (private rates of return) in analysing the interview data, it became clear that for a certain section of the community, those who were otherwise disadvantaged in society by their socio-economic status, private rates of return were an important benefit derived from some intended side effects of educational broadband provision. these side effects were generated by certain initiatives that have seen broadband infrastructures installed on school sites being extended to serve their surrounding communities, linked with access to digital devices that were primarily intended for educational use but also find their way into the hands of family members of school students. ‘the flow on effect for our lowest socio-economic community is that suddenly they are able to access a whole world that they were never able to before…it’s life changing’ (trust) one effect of this on private rates of return is the ability of low income families, given access to school based community broadband and educational digital tools, to shop on line and thus save themselves money. by such means, ufb can directly reduce digital divides. ‘making the wi-fi the schools have available in some way for the community to use so that poverty itself doesn’t become an impediment’ (researcher) 13 australasian journal of information systems parsons 2016, vol 20, research on rural & remote ict broadband initiatives in education in expanding on this comment, the interviewee noted that community access to broadband provided equity in access to knowledge. such positive effects, however, comprise only small steps towards long term equity, which cannot be provided only by current ufb investments. nevertheless, an awareness of the relationship between strategic investment and private rates of return for disadvantaged communities will help us to effectively plan future investment. ‘it'll take generations to truly resolve the inequity of experience or the inequity of outcome across our society and across demographics so the further forward we can look the better’ (crown agency) 5 discussion this article has addressed issues surrounding roi for broadband infrastructure in the new zealand school system, as revealed by a series of semi structured interviews with relevant stakeholders. a study of roi in a field like education is intrinsically difficult, because ultimately the outcomes of education are personal. ‘roi is probably one of the most unquantifiable of all what you're doing is providing an opportunity’ (researcher) nevertheless, there are some useful concepts in the literature that can be used to make such calculations. the first research question addressed in the research was ‘what assumptions underlie return on investment projections in the provision of broadband infrastructure to schools?’ this question was implicit in the literature review, which provided the basis for the research model and the interview questions. as a result, the study was based on a research model which focused on social surplus as the prime driver for roi in educational broadband policy. however it is clear from the analysis that such a simplistic model fails to address all of the relevant factors. from the results of the interview data, it has become clear that roi calculations of both producer and consumer surplus involve complex relationships of cost and benefit. cost and benefit calculations may vary in their timescales and impacts, and vary from the easily calculated to the somewhat speculative. in particular, it is hard to quantify consumer surplus, though this is regularly used by industry in calculating overall social surplus (e.g. alcatel-lucent, 2012; competition economists group, 2015.) the second research question was ‘how can the components of social surplus be defined within the provision of educational broadband?’ an analysis of the interview data led to a redefinition of the original research model. figure 2 shows this modified model, developed from the earlier model but incorporating the outcomes of the study. a new component in the model is a separate assessment of social and private rates of return when considering consumer surplus. for each component in the model, the positive and negative forces on surplus that emerged from the interview data have been indicated. in the savings component of producer surplus, procurement, resources and overheads were identified as key areas of importance, with costs of procurement and resources in some cases potentially leading to negative returns. profit calculations are long term, and can only be measured by the employability and earnings of students when they enter the workforce. one of the difficulties here is predicting skills demand in an unknown future, so broad rather than narrow skills sets are important. in assessing consumer surplus, components of social rates of return are largely positive, based around teaching and learning, behaviour and equity, though there are concerns about the long time scales required to truly address inequity. nevertheless, the impact on private rates of return was seen as very positive for the most disadvantaged members of the community, which was an encouraging finding from the study. 14 australasian journal of information systems parsons 2016, vol 20, research on rural & remote ict broadband initiatives in education savings profit producer surplus consumer surplus social surplus roi calculation educational broadband policy contributes to contribute to contribute to defines justifies procurement • + centralised • overspecified resources • + free • short term overheads • + reduced travel • + less printing • + byod skills • + work relevant digital skills • limited demand income • + higher earnings teaching and learning • + efficiency • + flexibility • + access behaviour • + positive attitudes equity • + increased access • long term process private rates of return greater equity • + community internet access • + device access • + consumer market access social rates of return contributes to fig. 2. extended model of roi for broadband initiatives in education 6 conclusions and future work estimates of roi for educational broadband focus on areas such as increasing access to digital learning materials, reducing the costs of producing and distributing such material, reducing the need for travel and improving student engagement and attendance. the results of this study generally support the main ideas underlying these assumptions. there are certainly savings to be made in the provision of learning materials and richer interactions between teachers, students and parents supported by broadband infrastructure. the study has also identified further important benefits, such as resilience to natural disasters and more flexibility for family experiences. however, it has also revealed some areas where there may be negative impacts on producer surplus, such as increased costs for professionally published learning materials, or unsuitable central provisioning. there is also a potential negative impact on consumer surplus where new channels of delivery may not be considered to have increased value by the consumer. for example the idea that savings can be made by reducing the need for field trips stresses quantitative gains while ignoring potential qualitative losses. most of the analysis was based on the social rate of return within the consumer surplus. however, one important finding from the study was that associated impacts on the private rate of return for the lowest socio-economic groups shows that educational broadband investments can have a positive effect on digital divides. 15 australasian journal of information systems parsons 2016, vol 20, research on rural & remote ict broadband initiatives in education in summary, this article has provided deeper and richer insights into the various components of social surplus associated with educational broadband investment than previously available material. it has identified both positive and negative effects within the overall calculations of roi. in so doing it raises some significant questions about the nature of roi projections for broadband in education that tend to emphasise gains but ignore potential losses. it is hoped that these findings will help to inform relevant stakeholders about the key issues of concern in gaining the most benefit from the national investment in broadband infrastructure. this work was based on a qualitative analysis of social surplus. future work might usefully leverage the model developed from this research to provide more insights into quantitative rates of return for broadband investment in the school sector. acknowledgements the author wishes to thank the interviewees who gave their time and expertise to this study. references alcatel-lucent. (2012) building the benefits of broadband: how new zealand can increase the social & economic impacts of high-speed broadband. available at: http://www.tmcnet.com/tmc/whitepapers/documents/whitepapers/2013/6687building-benefits-broadband-how-new-zealand-increase-social.pdf auditor general. (2013) public sector financial sustainability. available at: http://www.oag.govt.nz/2013/financial-sustainability/docs/public-sector-financialsustainability.pdf bauer, j., kim, j. & wildman, s. (2005) “an integrated framework for assessing broadband policy options”, michigan state law review, 21. bouras, c., giannaka, e. & tsiatsos, t. (2009) “identifying best practices for supporting broadband growth: methodology and analysis”, journal of network and computer applications, 32:795–807. cambini, c. & jiang, y. (2009) “broadband investment and regulation: a literature review”, telecommunications policy, 33:559–574. cava-ferreruela & alabau-muñoz. (2006) “broadband policy assessment: a cross-national empirical analysis” telecommunications policy, 30(8). chang, h-j. (2011) 23 things they don’t tell you about capitalism, penguin, london. commerce commission. (2012) high speed broadband services demand side study: final report. wellington: commerce commission new zealand. available at: http://www.comcom.govt.nz/dmsdocument/8917 competition economists group. (2015) welfare effects of ucll and uba uplift. available at: http://.comcom.govt.nz/dmsdocument/13038&ei=jd07vdz5c5d18qws4ogqdg&usg =afqjcne2beg8ma5v0wke128tc5epvh7-ca&sig2=jftg3kr_rdp_wu3tji4f6g crandall, r. w., & jackson, c. l. (2001) “the $500 billion opportunity: the potential economic benefit of widespread diffusion of broadband internet access”. washington: brookings institution. available at: http://www.att.com/public_affairs/broadband_policy/brookingsstudy.pdf crawford, s. (2013) captive audience: the telecom industry and monopoly power in the new gilded age, yale university press. crown fibre holdings. (2013) fact sheet: agreement with enable networks. available at: http://www.crownfibre.govt.nz/media/13092/fact%20sheet%20%20agreement%20with%20enable%20networks.pdf 16 australasian journal of information systems parsons 2016, vol 20, research on rural & remote ict broadband initiatives in education eaves, d. (2013) new zealand the world's laboratory for progressive digital legislation. tech president. available at: http://techpresident.com/news/wegov/24353/newzealand-worlds-laboratory-progressive-digital-legislation european commission. (2015) digital agenda for europe. available at: http://ec.europa.eu/digital-agenda/ feld, h., rose, g., cooper, m. & scott, b. (2005) connecting the public: the truth about municipal broadband. available at: http://www.freepress.net/sites/default/files/fplegacy/mb_white_paper.pdf folkestad, b. (2010) analysing interview data: possibilities and challenges. eurosphere working paper series 13. available at: http://eurospheres.org/files/2010/08/eurosphere_working_paper_13_folkestad.pdf . given, j. (2010) “take your partners: public private interplay in australian and new zealand plans for next generation broadband”, telecommunications policy, 34(9):540–549. gomez-barroso, j. & perez-martinez, j. (2005) “public intervention in the access to advanced telecommunication services: assessing its theoretical economic basis”, government information quarterly, 22:489 – 504 greenstein, s., & mcdevitt, r. (2012) “measuring the broadband bonus in thirty oecd countries”, oecd digital economy papers, no. 197. hayes, r. (2011) “valuing broadband benefits: a selective report on issues and options”, melbourne: melbourne business school. retrieved from https://web.archive.org/web/20121210000640/http://broadband.unimelb.edu.au/eng age/pubs/wp/valuingbroadbandbenefits.pdf hitt, l. & brynjolfsson, e. (1996) “productivity, business profitability, and consumer surplus: three different measures of information technology value”, mis quarterly, 20(2):121142. holt, l. & jamison, m. (2009) “broadband and contributions to economic growth: lessons from the us experience”, telecommunications policy, 33:575–581. index mundi (2014) new zealand economy profile 2014. available at: http://www.indexmundi.com/new_zealand/economy_profile.html koutroumpis, p. (2009) “the economic impact of broadband on growth: a simultaneous approach” telecommunications policy, 33(9):471–485. mbie. (2011) five point government action plan for faster broadband: e-education. ministry of business, innovation and employment. available at: http://www.med.govt.nz/sectors-industries/technology-communication/fastbroadband/government-action-plan-for-broadband/e-education mbie. (2012) background and implementation of ultra-fast broadband initiative. ministry of business, innovation and employment. available at: http://www.med.govt.nz/sectors-industries/technology-communication/fastbroadband mbie. (2015) fast broadband: new zealand's internet upgrade. available at: http://www.mbie.govt.nz/info-services/sectors-industries/technologycommunications/fast-broadband moe. (2014a) tertiary education strategy 2014 – 2019. available at: http://www.education.govt.nz/ministry-of-education/overall-strategies-andpolicies/tertiary-education-strategy moe. (2014b) learning with digital technologies. ministry of education. available at: http://www.minedu.govt.nz/theministry/educationinitiatives/ufbinschools.aspx 17 australasian journal of information systems parsons 2016, vol 20, research on rural & remote ict broadband initiatives in education moe. (2015a) n4l managed network. ministry of education. available at: http://www.minedu.govt.nz/theministry/educationinitiatives/ufbinschools/anetwo rkforlearning.aspx moe. (2015b) school network upgrade project (snup). ministry of education. available at: http://www.minedu.govt.nz/nzeducation/educationpolicies/schools/initiatives/icti nschools/ictinitiativesandprogrammes/ictinfrastructure/snup.aspx newman, e. (2012) high speed broadband services demand side study: issues paper 2 ehealth and e-education. wellington: commerce commission new zealand. available at: http://www.comcom.govt.nz/assets/telecommunications/studies/ufb-demandside/issues-paper-2-e-health-and-e-education-january-2012.pdf newson, r. (2014) where to work if it's money you're after. the new zealand herald, aug 6, 2014. available at: http://www.nzherald.co.nz/personalfinance/news/article.cfm?c_id=12&objectid=11304473 nzrs. (2015) national broadband map, available at: https://broadbandmap.nz/ nzte. (2015) statistics. new zealand trade and enterprise. available at: http://www.nzte.govt.nz/en/invest/statistics/ oecd. (2014) country statistical profile: new zealand. country statistical profiles: key tables from oecd. available at: http://dx.doi.org/10.1787/csp-nzl-table-2014-2-en pappas, c. (2015) what can google classroom offer to the world of elearning? elearning industry. available at: http://elearningindustry.com/google-classroom-a-free-learningmanagement-system-for-elearning pilkington, r. (2008) “measuring the impact of information technology on students’ learning”, in voogt & g. knezek (eds.), international handbook of information technology in primary and secondary education (part one). new york: springer, pp.1003-1018 psacharopoulos, g. (1994) “returns to investment in education: a global update”, world development, 22(9):1325-l343. pullar-strecker, t. (2012) huge payoff from ultrafast broadband predicted. business day. available at: http://www.stuff.co.nz/business/industries/6450713/huge-payoff-fromultrafast-broadband-predicted robson, a. (2014) cost-benefit analysis of public investment in high speed broadband network infrastructure: an analytical framework. available at: https://www.communications.gov.au/sites/g/files/net301/f/5._nbn_cba_analytical _framework_final_16_may_2014.pdf rosen, d. (2014) the school broadband scam. counterpunch. available at: http://www.counterpunch.org/2014/02/07/the-school-broadband-scam/ shankland, s. (2014) fast fiber-optic broadband spreads across developed world. cnet. available at: http://www.cnet.com/news/fast-fiber-optic-broadband-spreads-acrossdeveloped-world/ statistics new zealand. (2014) new zealand business demography statistics: at february 2014. available at: http://www.stats.govt.nz/browse_for_stats/businesses/business_characteristics/busi nessdemographystatistics_hotpfeb14.aspx trading economics. (2010) employment in agriculture (% of total employment) in new zealand. available at: http://www.tradingeconomics.com/new-zealand/employmentin-agriculture-percent-of-total-employment-wb-data.html underwood, j., ault, a., banyard, p., bird, k., dillon, g., hayes, m., selwood, i., somekh, b. & twining, p. (2005) “the impact of broadband in schools”, becta ict research. available 18 australasian journal of information systems parsons 2016, vol 20, research on rural & remote ict broadband initiatives in education at: http://dera.ioe.ac.uk/1616/12/becta_2005_impactofbroadband_report_underwood_ redacted.pdf unesco. (2013a) “technology, broadband and education: advancing the education for all agenda”, broadband commission working group on education. paris: unesco. available at: http://unesdoc.unesco.org/images/0021/002196/219687e.pdf unesco. (2013b) unesco policy guidelines for mobile learning. available at: http://unesdoc.unesco.org/images/0021/002196/219641e.pdf williams, s. (2015) 500,000+ students & teachers using n4l's managed network. techday. available at: http://educators.co.nz/story/500000-students-teachers-using-n4lsmanaged-network/7a63c8c4-7c19-11e4-bc2b0abdbf634b7d/?utm_campaign=20150326&utm_medium=email&utm_source=educa tors-news-alerts copyright: © 2016 parsons. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. 19 http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 1.1 return on investment and social surplus 1.2 the new zealand context 1.3 educational broadband policy implementation in new zealand 2 literature review 2.1 defining broadband 2.2 return on investment from broadband infrastructure 2.3 educational broadband investment 2.4 return on investment for broadband in education 2.5 geographical factors 2.6 research context and questions 3 research methodology 3.1 data description 3.2 data analysis 4 results 4.1 producer surplus (savings) 4.2 producer surplus (profit) 4.3 consumer surplus (social rates of return) 4.4 social surplus (private rates of return) 5 discussion 6 conclusions and future work microsoft word 1679-copy.docx australasian journal of information systems bhoola & giangreco 2018, vol 22, research article hr activities and practices for project success 1 hr activities and practices for project success: a multimethod approach from indian it firms vanita bhoola s. p. jain institute of management and research india vanita@spjimr.org antonio giangreco ieseg school of management, lille france abstract the purpose of this study is to examine the impact of human resource activities and practices (hraps) on value-driven project management in the indian it industry. it is a sector that offers various jobs for young it skilled labour force, but faces major challenges in recruiting and retaining talent. a multi-method research has been carried out, using delphi technique and a simultaneous equations model. using the delphi method, we developed an initial set of 136 globally accepted hraps. this was reduced to 20 most significant attributes applicable to indian it projects with the help of kendall’s coefficient of concordance (w) used over three rounds of iterations. later, from 431 observations, we selected the top 10 hraps based on mean ranks in round 3 to study the impact on value-driven project success, in terms of meeting project, customer and organizational goals. the study also uses triangulation through a multimethod research approach to achieve convergence. the significant hraps impacting valuedriven project management include efficient project leadership, effective communication, training of project leaders, ensuring job-matching and listening to team members’ issues. frequent team conflicts and personnel turnover, however, adversely impact it project success. our results confirm that the impact of hraps on project goals is more prominent and immediate as compared to meeting customer or organizational goals, which is often delayed. the results emphasize the necessity of integrating all stakeholders of it projects in order to accomplish value-driven success. keywords: value-driven project success, multi-method research, delphi method, full information maximum likelihood estimation, human resources management activities and practices. 1 introduction sourcing low cost resources to cut operating expenses has enabled information technology (it) activities and projects to be executed from destinations like india, brazil, the philippines, etc. where the workforce is young and intellectual capital is abundant (viji, 2016). the indian it industry is poised to become a global powerhouse as its services sector is projected to grow to usd 154 billion, with presence in over 52 countries, 500 global delivery centres, and 600,000 foreign nationals (nasscom, 2017). indian it multinationals recruit a workforce of 10 million and account for 67% of the us market, with an overwhelming global reputation (majumdar, 2013; thite et al., 2014). information technology is a typical industry where activities are developed according to the logic of projects and client satisfaction is considered as the uppermost indicator of project australasian journal of information systems bhoola & giangreco 2018, vol 22, research article hr activities and practices for project success 2 success (diegmann et al., 2017). despite the needs for different success criteria of customer satisfaction and organization goals, project success is mostly measured in terms of time and scope completion, and meeting costs (basten et al., 2014). consider the case of e-governance projects in india – apart from barriers like awareness, literacy rates, or security issues, to implementation at the grassroots level, such projects face hr challenges in aligning resources to meet business goals (prakash, 2016). hence, achieving success of it projects is complex. originating with rockart (1979), several studies have discussed the critical success factors relevant to it projects (blaskovics, 2016; esteves, 2014; keil et al., 2013; remus and wiener, 2010). based on a database of over 50,000 it firms built up between 2007 and 2016, standish group’s chaos report outline (2016) observes that less than 29% of these projects are successful. similarly, a recent investigation from the cloud provider innotas shows that 55% of the 126 it professionals surveyed between january and march 2015 reported project failures due to mismatched resource allocation and alignment with business goals (florentine, 2016). while reasons for project failure could be many (hughes et al., 2017), human resource management (hrm) emerges as a significant reason (jean-sébastien, 2016). zidane and bjorn (2018) document poor hrm has the fourth most important “universal delay factors” in project execution. the role of hrm to determine project success is well-researched, although mostly originated from the united states or europe (farndale et al., 2010; korsakienė et al., 2015). while this provides a rationale to consider india as a case in point, changing business environment, potential threats of labour turnover and practice of western-centric hr principles are increasing the risks of it project failure even further (thite et al., 2014). a specific focus to such studies that discuss the non-western world is rare (bhoola, 2015) and more research is welcome in this area (budhwar et al., 2016). a gap therefore exists in the literature. this research aims at filling this gap by identifying the hr activities and practices (hraps) that might affect the success of projects in the it industry, with a view later to formulating requirements for policies that might be particularly relevant for practitioners as hraps designed for it projects should be standardized as they need great attention in a project-based organization (keegan et al., 2012; huemann, 2010). this is done by detecting and testing a panel of hrap related measurement items that might affect the success of a project in the indian it industry, based on a dataset coming from a survey of 431 observations, collected during 2014-15, from 18 organizations that have provided project management training to their employees. in doing that, instead of limiting the definition of project success to the quadruple constraints of meeting cost, time, scope and quality, we consider value-driven project management that provides a holistic view to project success by aligning project goals with organization and customer goals (shenhar et al., 2001). value-driven project management is effective only if it is co-created along with people; else it will remain a vision, seldom achieved (bhoola, 2016). using the delphi method, we narrowed down on a set of hraps and used this to evaluate their impact on value-driven project management. the rest of the article is structured as follows. the following section provides a background of literature on it projects particularly in the indian context. we continue by presenting the research methodology and its rationale. in the following section, we elaborate on the findings from the delphi method, and then of the full information maximum likelihood estimation (fimle) method. we conclude by discussing the triangulation of the qualitative and quantitative methods, the theoretical contribution and managerial implications of the study. australasian journal of information systems bhoola & giangreco 2018, vol 22, research article hr activities and practices for project success 3 2 hr practices, it industry and india india has 179 languages and 154 dialects, with an uneven distribution of literacy rate (rao, 2015). though rich in culture and natural resources, india continues to face multiplicity of problems including, political and religious instability; increasing population, unemployment and poverty; relatively low per-capita income; unmanageable public sector companies and bureaucracy; and inequality in income distribution (budhwar, and debrah, 2013). regional differences, caste systems, socioeconomic status, and urban-rural divide add to the misery (rao, 2015). globalization and liberalisation policies have enforced competition, fostering more creativity and innovation among indian organizations (som, 2006). in order to manage its workforce, india mostly follows global best practices to implement hrm systems (bjorkman and lu, 2001). there is a growing theoretical appreciation that hr practices are contextual and culture specific (bossard and peterson, 2005; mcmahan, bell and virick, 1998; bowen, galang, & pillai, 2002; budhwar and bhatnagar, 2008; budhwar and khatri, 2001). given the socioeconomic and cultural diversity of india, imitating global practices could be mired with challenges. researchers have typically discussed western-focused cross-country and cross-cultural hr applications and practices. however, there is serious dearth of such studies based on indian hr perspectives (triandis, 1989; von glinow et al., 2002). with a growth reaching usd 1 trillion, the it industry in india is among the top job creators worldwide, with two-third of its operations and exports from north america, western europe, and asia pacific (patel and budhwar, 2010). with a whopping 55 percent share of global market, the indian it services sector provides direct employment to 2.8 million people in india; moreover, with its presence in 52 countries with over 500 global delivery centres it employs over 600,000 foreign nationals (rao and balasubrahmanya, 2017). recruiting, retaining, and developing human intellectual talent with a global mindset is one of the key success factors for indian offshore service providers (chadee, raman, and michailova, 2011), as global hr management capabilities play a key role in effectively leveraging the intangible resources for successful performance of these firms (lahiri, kedia, and mukherjee, 2012). multinational companies usually try to keep the balance between parent and subsidiary globally by also using standardized hr practices, (mathew, jain and bedi, 2012). in coping with the challenges coming from operating at the worldwide level, multinational indian organizations instead have been more reactive than proactive in its approaches to hrm. ignoring the hr function over decades has therefore built up pressure on the industry (thite, wilkinson budhwar and mathews, 2016). given the nature of the industry, the type of intellectual workforce and the high economic growth rate are leading to scarcity of manpower (yadav and yadav, 2017), so that the right approach to hr applications and practices becomes a key challenge for the resent and the future. it is indeed a tightrope walk to ensure the right balance of leadership, communication, motivation and also managing conflicts and turnover in order to lead to project success on one hand, and gaining the confidence of team members, customers and organization, on the other (todorović et al., 2015). hence, considering the present dynamic business environment, it is crucial for indian organizations the need to pursue more proactive hrm practices, by customizing it with indian traditional values, religious and ethnic backgrounds and introducing innovative culture-building practices (som, 2006); rao and balasubrahmanya, 2017). clearly, this calls for a transition of hr from traditional roles to roles that integrate all australasian journal of information systems bhoola & giangreco 2018, vol 22, research article hr activities and practices for project success 4 stakeholders who are directly or indirectly associated with the project, in order to achieve value-driven success (stulgienė and čiutienė, 2012). 3 previous studies while there are different factors that lead to successful project implementation, hr management and facilitating a collaborative work environment is of high relevance for it companies (ravid et al., 2012). it projects are characterized by speed, relevance, changing requirements, interdependency, complexity, uniqueness and ambiguity (bathallath, smedberg and kjellin, 2017). hence, lack of clarity in human responsibilities across various phases of project life cycle adversely impacts it project success (keegan, et al., 2012; hoda and murugesan, 2016). while hr practices are strategic in nature and provide a guiding force to the project organization, hr activities provide the implementation of the directives set through hr practices (koulikoff-souviron and harrison, 2010). hr activities and practices (hraps) include the strategic hr directions of the project organization that ensure recruitment and selection, personnel planning, training and development, performance management, teamwork, performance related pay, employment security, internal career possibilities, participation, communication, job design, job description, work-life balance and various other activities (van de voorde et al., 2012). in project-oriented companies, effective management of hraps like employee performance and development, turnover, team communication, experience learning, job-matching, etc. not only influence the project manager, but also the project and the organization (keegan, huemann and turner, 2012). unless hraps lead to cohesive team-building, healthy communication and transformational leadership, project success would remain a myth (aga, noorderhaven and vallejo, 2016). indeed, replacing the top-down approach with a more flexible mechanism, especially in the case of project resource allocation, by drawing experiences from the grass-root level of the organization is of critical importance (momeni and martinsuo, 2018). researchers have extensively studied the role of hr in affecting organisational performances (foss et al., 2015; van dierendonck et al., 2016). belout and gauvreau (2004) show that the role of hr in project management is changing beyond the “personnel factor” (pinto and prescott, 1988) and the traditional hrm (van marrewijk and timmers, 2003). project success depends much on management support, project manager’s role and allied support factors that make the role, responsibility and operation of the hr, effective (turner and müller, 2005). batistič and kenda (2018) talk about organizational socialization to enhance productivity. they provide a framework to describe how new members of a project should socialized effectively to ensure smooth inclusion and better contribution to the projects. in the indian context, the it industry is playing a leading role in designing innovative solutions to face unique and enterprising challenges by incorporating social, mobile, analytics and cloud technologies (sahoo and nauriyal, 2014). unlike infrastructure construction projects, it projects generally span multiple geographical locations, involving people from diversified cultures and backgrounds, using multiple work standards and technologies (lee and baby, 2013). hence, the presence of project team cohesion, employee-friendly working environment, supportive leadership, collaborative management and favourable hraps can influence the quality of deliverables and value-creation (khan and rasheed, 2015). hraps in it projects are mainly the domain of project managers, rather than the hr department or other australasian journal of information systems bhoola & giangreco 2018, vol 22, research article hr activities and practices for project success 5 line managers. this requires healthy interchange of communication between the hr department and project managers (keegan et al., 2012). hence, our focus on hraps is from the perspectives of project managers. we argue that though there are various factors that lead to successful it project implementation, effectively managing hraps is the key element to avoid it project failures (hughes et al., 2017). in project-oriented companies, hraps not only influence the line manager or the project manager, but also the organization at a larger level in the long run (keegan et al., 2012). while the role of hraps in it project success is widely researched, there is a dearth of such studies from the indian perspective (thomas and bhasi, 2012) that capture their ranking (cuellar, 2010). earlier studies dedicated to identifying hraps that influence it project performance and success, are mainly from the perspective of the western world (bhoola, 2015). there is dearth of literature that addresses the role of hrap in it projects considering india or southeast asia. also, previous research either focuses on a qualitative methodology based on delphi study (keil et al., 2013), in-depth interviews (thite et al., 2014) or compilation of case studies (keegan et al., 2012) or depends on a quantitative model (khan and rasheed, 2015) to determine the hrap factors. studies that use a multi-method research approach (remus and wiener, 2008) are rare. this study bridges the gap by initially starting with 136 globally accepted hraps that influence project success, then gradually narrowing down to just ten customized factors using delphi method, that are applicable to the indian context. the study uses triangulation (morse, 2003) to ensure convergence of findings. specifically, we posit to address the following research questions: rq1: what are the hraps, in order of importance, that influence it project success, in the indian context? rq2: are there hraps that are specific to the indian context, as compared to the global experience? rq3: how do the most important hraps influence value-driven project success? rq4: is it possible to develop a comprehensive picture that can triangulate the findings from multimethod strategies? 4 research methodology we wish to address rq1 by using the delphi method. the inputs from rq1 can be used to draw a comparative analysis of the findings and literature to answer rq2. drawing on the major findings from rq1, we will derive a smaller set of hraps. in order to address rq3 we will use a simultaneous equations model. the purpose of rq4 is to logically integrate the findings of rq1 and rq3 and check for triangulation. australasian journal of information systems bhoola & giangreco 2018, vol 22, research article hr activities and practices for project success 6 respondent age education gender designation number of projects managed 1 36 graduate male development team member 10 2 44 post graduate female project leader 29 3 40 graduate male team leader 20 4 44 pmp male quality manager 30 5 38 pmp male development team member 14 6 42 prince male project leader 26 7 36 certification in pm male systems analyst 10 8 42 graduate male team leader 27 9 36 certification in pm male project leader 10 10 37 graduate male development team member 12 11 39 post graduate male project manager 18 12 42 prince male project leader 27 13 41 graduate male project manager 21 14 38 pmp female quality manager 15 15 41 graduate male project leader 26 16 43 graduate male project manager 28 17 44 post graduate male project leader 32 18 37 graduate male project manager 13 19 38 pmp male project manager 16 20 38 graduate female development team member 17 21 41 graduate male project leader 23 table 1: profile of respondents of the delphi method keil et al. define the delphi method as “a group decision-making process where, through iterative feedback, the opinions of a panel of experts are elicited and consensus is reached” (2013, p.400). as the purpose of rq1 is to unearth the relevant aspects of hraps used in indian it project management and determine the relative importance of issues, we used the delphi method with three distinct phases namely, discovering, determining and ranking (keil et al., 2013; schmidt, 1997; schmidt et al., 2001). a convenient sample of 21 respondents was selected from the mumbai and bangalore chapters of project management institute (pmi), from the professional network of one of the authors. table 1 provides a glimpse of the profiles of the 21 respondents. the work experience of the respondents ranged between 14 and 22 years. the respondents handled anywhere between 10 and 32 it projects individually, 70% being cross border projects. we discussed the process with the panel and urged them to cooperate with us as it might require multiple rounds of iterations to achieve convergence in ranking. following schmidt (1997), we asked each respondent to list the hraps that they considered important to successfully implement it projects, subject to a minimum of six items. a total of 136 hraps were collated, as similar issues were addressed by the panellists differently. with the help of an english language editor, a unified description for each issue, considering the most commonly used phrases was generated. wherever there was ambiguity or lack of clarity, the respondent was contacted and consensus achieved. for duplicate issues, the most common description was retained (keil et al., 2013), so that we identified 44 unique hraps along with a brief description of each, from 21 respondents (see appendix 1). australasian journal of information systems bhoola & giangreco 2018, vol 22, research article hr activities and practices for project success 7 in the second phase, each respondent was then provided with the randomly ordered list of 44 hraps along with a brief description, and then asked to select 20 hraps, in the order of preference that strongly influence it project success (schmidt, 1997). at this stage, three respondents did not carry on with the study, despite several reminders. only those issues that were shortlisted by at least half of the respondents were retained and considered for the third phase (schmidt, 1997; watson, 2008). table 2 depicts the sequence of the delphi method. discovering • identification of 21 project managers and studying their profile • collection of 136 hraps along with brief description • consolidation and verification of responses to capture fair representation • generating 44 unique issues after paring, removing duplicates and mapping of terms determining • pared list of all 44 issues was randomly ordered • total participation at this phase was 18 respondents • each respondent was provided with the list to select top 20 issues • twenty issues shortlisted by at least half of the respondents were retained ranking • list pared to 20 hraps for ranking by 18 panellists • round i ranking was based on the frequency of selection of each issue by panellists • ordering of issues in rounds ii and iii on feedback was based on mean ranks. • kendall's w was calculated for each round, until an acceptable value was achieved. table 2: the three phases of delphi method: discovering, determining and ranking the third phase mostly involved ranking these 20 hraps by the same set of panellists, using an iterative process, until an acceptable level of consensus was achieved. following keil et al. (2013), the first round of ranking was based on the frequency of selection of each issue by the panellists. however, from the second round the ordering process was based on mean ranks, wherein the panellists were asked to rank each of the 20 hraps, in order of preference. they were also requested to write a brief note to justify their ranking procedure. specific instruction was given to avoid tie ranking. kendall’s coefficient of concordance (w) was measured for each round to monitor improvement in agreement among panellists. after achieving satisfactory consensus, we selected six panellists from pmi mumbai chapter to further brainstorm the rationale behind selection of the hraps. with the help of semistructured interviews, we attempted to further investigate the following: (a) why did some of these hr activities and practices emerge significant? (b) are these activities and practices specific to india? (c) how do organizations effectively manage these issues? these responses were then compiled and also related to the literature. this enabled us to address rq2. to answer rq3, we considered the top ten hraps as determined by the delphi method as independent variables to design the questionnaire. the questionnaire was designed to collect data from respondents to develop a quantitative model. as all the questions were perception driven, a 5-point likert scale was used for rating from excellent to poor. as discussed earlier, value-driven project management was captured by three outcome variables namely, meeting project goals, meeting customer goals, and meeting organizational goals, as documented by shenhar et al., 2001). as the objective was to determine the impact of the hraps on the three dimensions of project success, jointly, a simultaneous equations model (sem) was applied. this helped to provide a comparative analysis of the independent variables and their concurrent impact on the outcome variables. this helped to develop a comprehensive picture by integrating the findings of the delphi method and sem, to address rq4. triangulation allows multiple viewpoints to study australasian journal of information systems bhoola & giangreco 2018, vol 22, research article hr activities and practices for project success 8 a single phenomenon by using a combination of methodologies that can include qualitative and quantitative research that can be used to achieve greater validation thereby enabling convergence of results (jick, 1979; joslin and müller, 2016). 4.1 sampling, data collection and respondents’ profile the members of the selected sample had a minimum of five years of fulltime work experience and also had attended a training program in it project management. for data collection, one of the authors contacted 15 training centres that serve as pmi® registered education provider (r.e.p.) and provide pmp® certification preparatory courses. questionpro was used as a platform for collecting online responses. in each of the centres there was a trainer responsible for coordinating and collecting the data. the questionnaire was circulated among 3750 potential respondents over nine months, from april 2015, of the pmi india chapters of mumbai, delhi, hyderabad and bangalore. this process generated data of 431 observations, giving a response rate of 11.5%. variables categories % variables categories % age less than 25 11 gender male 86 25 to 28 38 female 14 28 to 35 33 project team size less than 5 19 36 and above 18 less than 10 15 education graduate 14 less than 25 29 post graduate 35 less than 50 15 pmp/prince certified 22 less than 100 10 other professional training 29 less than 1000 12 project category business application 42 designations project manager 16 engineering application 5 project leader 14 crm 5 team leader 28 erp 17 quality manager 6 web application 17 system analyst 12 system software 14 member, development team 24 table 3: sample characteristics (sample size: 431) as observed in table 3, the age distribution of the respondents is skewed between 25 and 35, and the gender distributed towards males. this type of job is sometimes an impediment for female employees, given their family expectations in the framework of the indian culture. 5 delphi findings the third phase of the delphi study presents the ranking rounds, where the order of importance of the top ranked hraps emerged (table 4). a total of three rounds of rankings were performed to achieve convergence among the respondents. all the 18 respondents participated for the first two rounds. the first round resulted in a kendall’s w of 0.227. based on mean rank of each hrap of the previous round, the second one generated a still lower kendall’s w of 0.443. though the null hypothesis of no significant agreement was rejected at 95 percent level of confidence, we were interested in improving the value, further. australasian journal of information systems bhoola & giangreco 2018, vol 22, research article hr activities and practices for project success 9 hence, we went for the third round. we received responses from 16 panellists. two respondents dropped out due to outstation travelling. the kendall’s w improved significantly to 0.732. the correlation coefficient (r = 0.716) indicated a very high agreement in ranking the 20 hraps. we informed the outcome to all 21 panellists. though some authors (brancheau and wetherbe, 1987) argue that standard deviations also indicate consensus, following schmidt (1997) we decided to ignore it as it does not apply to ordinal level data. 5.1 people management skills of project leadership people management skills of project leadership emerged as the most significant issue in ensuring project success. due to higher availability of jobs in emerging economies as compared to developed nations, employee attrition is higher (agarwal and mehta, 2014). hence, people management is a crucial component of effective leadership. a leadership structure that is empowering and relies on participative decision-making processes, as well as engages project team members by creating a contributing environment (keil et al., 2013; zhu and kindarto, 2016), is more preferred than autocratic processes with less delegation of authority. round – i round ii round iii hraps mean rank mean rank mean rank 1 people management skills of project leadership 3.5 3.222 1.833 2 effectiveness of project team communication 8.889 7.556 3.944 3 consultation with employees before assigning a project 8.111 9.167 5.056 4 job-matching to enhance performance 10.111 8.278 5.778 5 training of project leaders 9.111 6.944 6.167 6 extent of team member turnover 9.167 9.167 7.111 7 frequent conflicts among project team members 9.222 8.444 7.444 8 personnel turnover at organizational level 9.444 8 8.444 9 performance evaluation of individual members 9.667 7.444 7.444 10 listening 9 9.5 9.389 11 staff motivation 10.5 9.667 11.112 12 respecting indian culture and ethnic diversity 11 9.222 11.278 13 creating congenial environment for sharing 11 9.667 12.444 14 making employees feel important 11.056 10.167 13.001 15 promoting a women-friendly work environment 11.111 12.556 13. 722 16 embedding organizational core values in projects 11.722 13.111 14.389 17 promoting innovation 11.389 16.444 15.944 18 promoting formal risk management practices 12.833 16.389 17.556 19 building a global work culture 15.833 17.5 17.722 20 balancing standardization and customization 17.333 18.111 18.778 w 0.227 0.443 0.732 r 0.181 0.41 0.716 table 4: delphi method ranking results from three rounds respondent 12 admits that the role of a project manager as a leader is often a tightrope walk. s/he has to maintain the right balance between project team members and management. s/he has to ensure that the project goals are aligned with organizational goals. in sarcastic terms, s/he said: australasian journal of information systems bhoola & giangreco 2018, vol 22, research article hr activities and practices for project success 10 “without the ‘blessings’ of the top management and the hr department, often the project manager is like a commander of the army without a sword. having said that, the higher the distributed leadership, higher the delegation of responsibilities more is perceived ownership and urge to perform.” all panellists unanimously agreed that a significant component of project leadership is communication and people management. in fact, leadership quality is often judged by the ability to manage people and communicate. humility in all forms of communication, compassion to team members and ethical behaviour would ensure the right leadership traits. 5.2 effectiveness of project team communication similar to the outcome of keil et al., (2013), effective project team communication emerged as the second most significant hrap. team communication unfolds the barriers to project complexity and the interdependencies across various stakeholders to ensure value-driven project management (kennedy et al., 2017). respondent 2, who ranked it highest, said: “frequent conflicts originate from poor communication among team members. the project leader plays an important role over here. s/he has to live by example. often, informal settings like a dinner party, family gatherings, playing indoor games that involve teamwork enhance communication within team members. indians come from varied backgrounds and ethnic cultures – integrating the entire team is a task in itself.” respondent 4, who also ranked this issue highest said: “...often when everything is set and project scope is well-defined a wild request comes from the customer to incorporate changes based on a new technology that has been just launched. to ensure customer delight, yet not overrunning the project constraints, communication is the key to success.” in this way, it emerged that quality of the communication is of paramount importance not only in the critical phase of planning of the project, but also in dealing with emergencies and variations required by the customers. 5.3 consultation with employees before assigning a project this emerged as the third most important variable. respondent 12 puts it thus: “the decision about an employee joining a project team is a two-way traffic. it is like a marriage between two parties. both need to like each other and there has to be a certain amount of synergy within fellow team members to work together to achieve a certain goal. after all, we all represent the organization.” respondent 2 who faced a situation where her competencies were misrepresented while assigning a project, shared: “we do not stick to the same competencies lifelong. we acquire new skills and sometimes abandon the earlier ones, sometimes due to obsolescence. this is truer in the case of it projects. hence, a prior consultation and a candid discussion can be helpful. also, once i am assigned with a project, it does not go well, if i reject it. hence, a mutual consent works well.” consultation with employees before assigning a project goes therefore beyond the symbolic meaning of establishing a connection and building consensus around common objectives. hence, in the it sector, consultation becomes critical in building different team competences. australasian journal of information systems bhoola & giangreco 2018, vol 22, research article hr activities and practices for project success 11 5.4 job-matching to enhance performance most of the respondents who ranked consultation with employees before assigning a project, very highly also ranked this issue favourably. job-matching can be considered as a typical hrap under the responsibility of resource and delivery managers to ensure enhanced performance (santos et al., 2013). as respondent 17 puts it aptly: “job matching is not the same as competency mapping. job matching is ex-post and competency mapping is ex-ante. often, there is a time lag, due to which there could be discrepancies between the two. both are important while assigning an employee with a project, improper job matching originating from ignoring competency assessment can adversely affect project performance.” thus emerged the necessity to have a coherent alignment between the competencies detailed by each project team member and the job that is assigned to the person. it, being an industry where expectations of customers can be very demanding and sometimes disrupting, to have the right person in the right role is a key asset for the success of a project. 5.5 training of project leaders coetzer and gibbison (2016) discuss the importance of the hr department’s encouragement to spend intensive time on project management training for project managers and team members by using organizational coaches and peer coaching. a few respondents ranked this issue as having highest importance. respondent 7 argued: “if project leaders are not well trained and abreast with the latest innovative technologies in the it domain, there is nobody to look up to.” respondent 17 stated: “even if the management does not train its employees regularly, it is imperative that the project managers are well-trained and they are aware of the state-of-the-art technological advancements. as per my experience, project managers are always keen to train their teams as it helps them to bond and also enrich their own learning and experience…” technical training of project leaders is indeed a key issue in it project success, before assigning a project. the project leader is often expected to train his team members, especially when time is limited and the possibility of sending a member for external training is ruled out. 5.6 extent of team member turnover considering the importance of factors that influence team member turnover in emerging economies, various researchers have discussed this aspect in greater detail (zhou, et al., 2010). a host of factors including culture shock, remuneration disparity, perquisites, benefits as compared to industry standards, work environment, sense of ethics and distributive justice, other lucrative opportunities, and so on, play important role in team member turnover (carr et al., 2010). respondent 2 while expressing her/his concern said: “we all work in our respective project with a lot of passion. very often we work extended hours to coordinate across diverse time zones and meet client specifications. suddenly, when we come of know about a dear colleague and friend leaving, it hurts. seldom have i seen organizational efforts to retain the best talents.” on similar lines respondent 8 said: australasian journal of information systems bhoola & giangreco 2018, vol 22, research article hr activities and practices for project success 12 “good people learn fast and move on. they believe working in the same place for long hinders career growth. eventually, we are left with a set of incompetent people who rise to the top, not because of their performance, but for their so-called loyalty to the organization.” 5.7 frequent conflicts among project team members jiang et al. (2014) discuss how good understanding among team members that is aligned with overall organizational and it project goals can lead to agreement in decisions through conflicts. butler (1973) in his iconic research urges organizations to perform systematic analysis to identify the root causes and extent of implications of dysfunctional conflicts in projects initiating from unhealthy competition and personal vested interests. most of the respondents agreed that a certain degree of conflict is often inevitable while working on projects with stringent deadlines and constraints. as respondent 17 put it: “there are two types of conflicts, as i see it – constructive and destructive. a constructive conflict is always healthy as that enhances performance in the long run, if not the short run. destructive conflicts are mostly led by personal agenda. i have been a victim of such animosity.” respondent 2, quite contradictory to the earlier argued: “conflicts can arise from very small things if team members do not resonate the same vibrations. then conflicts can originate from ways in completing tasks, approach in handling customers’ queries, or fighting with limited resources.” as in other contexts, the existence of a conflict here does not represent a problem in itself. on the contrary, if the conflict is focused around an object related to the project (managing tasks, handling customers, etc.), the exchange of different views generates better solutions. when instead, the conflict is centred on relational issues between people, then it might become counterproductive of the achievement of the results. 5.8 personnel turnover at organizational level abdel-hamid (1992) discussed how turnover at the top management level can result in serious instability in it projects and disrupt succession planning, leading to schedule and budget overruns. parker and skitmore (2005) argue that turnover of management personnel happens mostly because of misalignment of organizational culture, alternative career options and personal growth opportunities. respondent 12, while speaking from her/his experience said: “when there is a turnover at team member level, it is not difficult for a project to recover and return to its equilibrium path. however, turnover at organizational level creates a black hole. when my project sponsor resigned, it left the project rudderless. as a young project manager, i used to look up to him for advice and guidance. it not only reduced the overall morale of the project team, but the confidence of the client was also shaken.” respondent 4, who is a quality manager said: “resignation of a top management executive can often have long term and irreparable damage to the organization. within my limited experience, i have observed that when a senior leader motivates, nurtures and personally takes interest in a particular vertical of business, his departure impacts the company’s bottom line.” australasian journal of information systems bhoola & giangreco 2018, vol 22, research article hr activities and practices for project success 13 while this issue is not of topmost importance, several panellists raised concerns on organisational turnover at the top management level, given that such incidents not only impact organizations but also projects. 5.9 performance evaluation of individual members frederiksen et al., (2017) argue that employees who work in complex project environments are generally evaluated by the project managers or supervisors. however, this is often subjective in nature leaving scope for dissatisfaction among team members. respondent 8 said: “a just structure for performance evaluation is a myth. it is rarely just. however, what makes it just and ethical is the transparency. also, having guidelines for performance evaluation is not enough – its implementation is equally important. it should be free from subjectivity, to the extent possible.” on a very practical note, respondent 2 said: “the organization should have the courage to demonstrate transparency in performance evaluation – it should be process driven. this could lead to heart-burn and attrition of valued employees. however, any compromise in this aspect just to retain an employee would jeopardise the whole effort.” this highlights the need of having an assessment system not only accurately designed, but also supported by a process properly carried out (katou et al., 2014). this is of great criticality, as the implementation of performance assessment has a cascading effect on other hraps such as remuneration, training, promotion, etc. 5.10 listening according to keil et al., (2013), extending a patient hearing to project teams’ concerns helps understanding project dynamics, associated risks, management of client expectations, and so on. respondent 12, a project leader rightly put it: “listening to my team members helps me solve half the battle. however, it is important that i am compassionate and feel for their emotions. if i don’t care a damn about their feelings and emotions, it impacts the project outcome. everything is interrelated.” respondent 5, who is a development team member opines: “i guess i love my work and i’m a silent worker. nevertheless, i often want my teammates to hear me. i want my team lead to ask me what i do and how i do, or whether i am facing any problem. sometimes, when work gets monotonous, i wish we can chat for a while and then again get back to work. i guess, the art of listening is an attribute of a giver.” these evidences show the importance of developing relational competencies including the capability of establishing empathy with the members of the project team. while this is not so unexpected, it is interesting to note that such an emotional skill is perceived as a key element of success for managers of it projects – a context in which usually high expectations exist in relation to the technical and managerial dimensions of the role. 5.11 other factors the top ten hraps ranked in the delphi method study have specific relevance to the indian context. so, in order to address rq2, we can infer that the most significant hraps that it australasian journal of information systems bhoola & giangreco 2018, vol 22, research article hr activities and practices for project success 14 companies should consider are local in nature. while is it agreed that some of these hraps have reference in literature in some form, there specific indian dimensions to each of these. there are a few other factors which did not figure in the top 10, but are given importance by several panellists. these include, respecting indian cultures and ethnic diversity, creating a congenial environment for sharing, making employees feel important, and promoting a women-friendly work environment. respondents expressed the need for creating a camaraderie feeling among team members that can improve productivity. purvanova (2013) discusses the need for creating a congenial environment that promotes fellow-feelings among team members. a respondent said: “after all, we spend most of our precious time with our colleagues. so why not consider it as our second home – our second family? if that is possible, interpersonal trust, reliance, interpersonal learning, sharing without inhibitions can happen spontaneously.” respondent 8 emphasized: “you don’t need to be too intelligent to fathom if your organization is employeefriendly. i think making you feel important is the primary factor. how can i give my 100% if i perceive that i am not important over here? the organizational culture percolates to the project manager. when i was a team member, i felt proud that my contribution was appreciated. i felt important. today, i treat my team members similarly. however, that does not imply that i will promote mediocrity.” respondent 2, a woman project leader said: “i wouldn’t have been able to work at … (company name withheld), unless the organization was empathetic to its women employees. ours is a patriarchal society. many a times, the responsibilities at home do not reduce, even if we earn a handsome salary and support the family. i am allowed flexible working hours. there is a crèche at my workplace. i leave my little daughter over there and can concentrate on my work. whenever required i can visit her during office hours. such facilities create a difference.” though respondents spoke about organizational values and work culture, they did not rank any of these issues with topmost priority. some spoke about the organization’s role in promoting innovation, but did not rank it as a significant trait under hrm. they argued that risk management is a technical or operational requirement for implementing a project rather than including it as an important issue under hr activities and practices. 6 quantitative model 6.1 measures to address rq3, we used a quantitative model. shenhar et al. (2001) identify 13 constructs to derive four dimensions to measure project success. the four dimensions are project efficiency, impact on the customer, business success, and preparing for the future. based on the geographical context, research scope and informal discussions with panellist of the delphi method, we adapted ten constructs from the 13, with minor modifications. these ten constructs formed the basis of outcome variables. ten mono-item constructs, identified from delphi method were considered as independent variables. a questionnaire was constructed australasian journal of information systems bhoola & giangreco 2018, vol 22, research article hr activities and practices for project success 15 to capture data from 431 respondents. they were asked to indicate on a 5-point likert scale (1 implying poor and 5 implying excellent) how they rated each of these constructs with reference to their workplace experience. each of these 20 constructs were presented in the form of statements, e.g. “the project that you are currently associated, tends to meet operational specifications.” the respondent was required to tick the most appropriate option in the 5-point likert scale. in order to reduce the outcome variables, an exploratory factor analysis was performed. this helped converting the ten constructs to three dimensions (see table 5). these three factors were treated as multi-item constructs to capture it project success. table 5 reports all the constructs that form the outcome and independent variables for the quantitative model. the results of exploratory factor analysis after varimax rotation for the three multi-item constructs are above 0.70 (three items fall short, though). reliability is achieved with cronbach’s alpha values higher than 0.70, as well. confirmatory factor analysis (cfa) was also performed to check convergent and discriminant validity. the goodness of fit indicators indicate that the model fits well with observed data (χ2 = 196.582, p < 0.001, goodness of fit index (gfi) = 0.971, comparative fit index (cfi) = 0.985, root mean square error of approximation (rmsea) = 0.048 and pclose = 0.000). the results with two factors was worse, hence, the above was considered. though statistically significant χ2 does not ensure a good fit of model, in larger samples with over 10 observed variables, significant p-values can often be expected (hair, et al., 2010). to ensure convergent validity, average variance extracted (ave) of each factor was calculated. the values were above recommended cut-offs (hair, et al., 2010). factor loadings from cfa and ave exceeded 0.50 cut-off, which ensured convergent validity (hair, et al., 2010). we found that the minimum correlation (min r) for each factor was statistically significant and higher than any other maximum correlation (max r) associated with the variables of other factors. it was also observed that the square root of ave was larger than the correlations between a particular construct and other constructs. this ensures discriminant validity. project goals were measured in terms of operational performance and technical performance and meeting cost and schedule targets. customer goals as visualized by the end user or customer were captured by addressing their changing needs, amicability of usage of the end product and overall satisfaction. lastly, organizational goals were measured in terms of achieving commercial success, gaining market share and meeting overall organizational mission. australasian journal of information systems bhoola & giangreco 2018, vol 22, research article hr activities and practices for project success 16 success dimension (multi-item constructs) success measures standardized loading average variance extracted construct reliability meeting project goals (mpg) meeting operational specifications 0.668 0.615 0.841 meeting technical specifications 0.777 meeting schedule targets 0.715 meeting cost targets 0.893 meeting customer goals (mcg) fulfilling changing customer needs 0.681 0.781 0.903 ensuring usability of product 0.876 enhancing customer satisfaction 0.787 meeting organizational goals (mog) achieving commercial success 0.697 0.511 0.774 gaining increased market share 0.792 aligning with overall organizational priorities 0.766 table 5: basic statistics of constructs forming outcome variables 6.2 quantitative model to address rq3, we used a simultaneous model – the full information maximum likelihood estimation (fimle) method to evaluate the impact of the ten independent variables (monoitems). the rationale behind selecting fimle is manifold. firstly, it is a simultaneous equations model that allowed us to consider three output variables and ten independent variables. secondly, after collection of data it was discovered that there were missing values. multiple imputation and fimle are generally used in such cases. while imputation methods replace or imputes missing values, fimle estimates population parameters that would produce the most likely estimates from analysed sample data (collins, schafer and kam, 2001). fimle allowed sorting missing data in a specific pattern, so that the rows that contain missing values are estimated together. it is a systems method of estimation and uses information of the endogenous variables effectively. thirdly, as fimle takes into account the error covariances across equations, it is also asymptotically efficient, in the absence of specification error. hence, we used fimle model to analyse how hraps influence value-driven project management, i.e., meeting project goals (mpg), meeting customer goals (mcg) and meeting organizational goals (mog), as discussed in table 2. the jth equation of the model can be represented as follows: �����, … , �� , �, … , , �� � �� [ j = 1, …, j ] … equation (i) where, yj are the endogenous variables, xj are the predetermined variables βg is a vector of unknown parameters and uj is the disturbance term. the fimle of the unknown parameters in equation (i) can be obtained by maximizing l following chapman and fair (1972), using eviews 9.0 software: � � � � � ln|2�| � � � �ln|�| � � �������� � ��′�� � ��!, with respect to the variance-covariance matrix of the structural form disturbances, where, π represents the reduced form parameters, j denotes the total number of observations, θ denotes the variance-covariance matrix of the reduced form disturbances, r denotes the restrictions. at this backdrop, the equations in the fimle model can be represented as follows: australasian journal of information systems bhoola & giangreco 2018, vol 22, research article hr activities and practices for project success 17 mpg = α0 + α1 x1 + α2 x2 + α3 x3 +… + αn xn …. equation (ii) mcg = β0 + β1 x1 + β2 x2 + β3 x3 + ... + βn xn …. equation (iii) mog = γ0 + γ1 x1 + γ2 x2 + γ3 x3 + ... + γn xn …. equation (iv) fimle could have been conducted with different coefficients of the predetermined variables across all the equations. however, as there was no much difference in results, it was avoided. as the number of predetermined variables in each equation is exactly the same as the number of predetermined variables in each equation, the system of equations can be considered as exactly identified. fimle is often when data contains missing values, as it uses raw data as input, as opposed to the observed covariance matrix, which contains less data (graham, 2009). the system of equations evaluates the influence of the selected hraps on the three dimensions of project success. hr activities and practices (independent variables) mpg mcg mog tolerance people management skills of project leadership 0.6757** 0.6542** 0.6521** 0.7463 (0.0109) (0.0103) (0.0106) effectiveness of project team communication 0.5232** 0.4617** 0.4740** 0.6592 (0.0413) (0.0339) (0.0488) training of project leaders 0.4574** 0.3596** 0.4198** 0.8329 (0.0465) (0.0483) (0.0476) job-matching to enhance performance 0.4324** 0.4200** 0.4144** 0.5658 (0.0472) (0.0481) (0.0489) consultation with employees before assigning a project 0.3789** 0.2423* 0.3664** 0.5381 (0.0599) (0.0615) (0.0603) listening 0.2188* 0.2187* 0.2003* 0.4546 (0.0796) (0.0716) (0.0797) frequent conflicts between project team members -0.2122* -0.1948* -0.1817* 0.4621 (0.0876) (0.0882) (0.0894) extent of team member turnover -0.1604* -0.1503 -0.143 0.5061 (0.1037) (0.1043) (0.1056) performance measurement of individual members 0.1672* 0.1218 0.1444 0.4389 (0.0845) (0.1112) (0.1086) personnel turnover at organizational level -0.1231 -0.1217 -0.1694* 0.4714 (0.1275) (0.1283) (0.0871) adjusted r-squared 0.3903 0.3649 0.3474 [note: “**” implies 1% level of significance and “* implies 5% level of significance.] table 6: fimle output as observed from the results in table 6, the three outcome variables are impacted significantly by the independent variables, though differently. typically, a tolerance below 0.30 would imply a multicollinearity problem (hair, et al., 2010). in this case, all the independent variables have a tolerance value higher than 0.30 (table 6). in general, the hraps have the highest influence on project goals (adjusted r-squared 0.3903), followed with meeting customer goals australasian journal of information systems bhoola & giangreco 2018, vol 22, research article hr activities and practices for project success 18 (adjusted r-squared 0.3649) and a trickling down effect on meeting organizational goals (adjusted r-squared 0.3474). this also implies that project success is not about meeting project goals, alone. value-driven project management has a significant impact on meeting customers’ goals and the goals of the organization. this is the one of the principal contribution of this study. people management skills of project leadership have the highest impact on the outcome variables. people management skills of project leader have a direct positive impact on project goals, more than the customer and organizational goals. on similar lines, effectiveness of project team communication has a higher impact on the immediate team that is dealing with the project. the percolation of effective communication is equally relevant for customer interactions and meeting their expectations, and last but not the least, it has a positive impact on organizational goals, as well. training of project leaders emerged as the third most impacting variable. it has the highest impact on project goals. the organization reaps the benefits of training project leaders across several projects, whereas the benefits to the customer are limited to a single project. the results show that training project leaders impacts organizational goals more than customer goals. we find that it is important to ensure that project requirements match with the competencies of the project team member. if job-matching is ensured, it has a significant contribution to project goals. in addition, the impacts on customer and organizational goals are no less. the study also finds that merely matching competencies with project requirements is not enough. it is important to consult with employees, before assigning a project. hence, if there has been a healthy consultation with employees before assigning a project, it has a high influence on the project goals. however, mostly such misalignment does not influence the customers. listening to project team members on issues related to the project and beyond the scope of the project influences all the three outcome variables more-or-less equally. in line with roebuck et al., (2015), we find that indian male project managers are less attentive in terms of listening skills than female managers. project managers and higher authorities need to be careful about frequent conflicts among project team members. this adversely impacts value-driven project management at all levels. team member turnover also adversely impacts the immediate project goals. however, it does not impact the customers or the organization in the long run. tripathi (2014) argues that reducing member turnover in project teams is one of the most important issues that the indian it sector encounters. high attrition rates manifest in the form of budget and schedule overruns. by contrast, personnel turnover at the organizational level affects organizational goals, leaving project and customer goals unaffected. performance measurement of individual members of the project team did not emerge as a significant variable, other than impacting project goals at 5% level of significance. this implies that organizations have timetested and robust hr processes for performance evaluation. 7 discussion the current study is focused on the features associated with hraps in indian it projects and their impact on project success. with the changing environment, firms are embracing a transition to project based organizations. this metamorphosis is redefining the role of hr to create a new work environment that can efficiently connect projects with team members, and align with dynamic customer needs and organizational goals and culture (gareis, 2013; keegan et al., 2012). with this backdrop, the major contribution of this study is that hr factors australasian journal of information systems bhoola & giangreco 2018, vol 22, research article hr activities and practices for project success 19 impact project goals more than customer goals and organizational goals. this reinforces the fact that from an organizational perspective, the diversified aspects of hr that impact project outcome cannot be considered in silos. 7.1 triangulation as proposed, the study has used a multi-method strategy to evaluate the impact of hraps on value-driven project success. the 16 panellists’ responses not only help understand the significance of the finally shortlisted hraps, but, also provide the india-specific approach as compared to global experience. though the panellists provide india perspective responses to all the ten factors, there are certain elements that emerge unique to the indian cultures and values. for example, giving the employee equal right to select a project or join a project team for a new project is practically non-existent in india, while well-practiced in the western world. a well-structured mechanism to ensure free-flow of communication at all levels is comparatively weak in india, as compared to the western world. also, in india, as opposed to the western world, people management processes practiced by project leadership is considered as an art, without having a scientific process that could govern it. also, it is seen that assessment systems are more ad hoc in nature, as compared to well-designed systems and processes practiced in most of the western world. this answers rq2. the measurement of triangulation can be considered to be successful if the results of the ranking of delphi from 21 panellists match with the findings of the fimle model. while the purpose of the delphi method was to rank the most significant hrap that impacts project success, the purpose of fimle was to critically evaluate the influence of the top ten hraps derived from delphi on value-driven project success. the first two variables namely people management skills of project leadership and effectiveness of project team communication have the same order of influence on project success. in the delphi method, consultation with employees before assigning project and training of project leaders were ranked third and fifth. in the fimle model they interchanged their ranking. job-matching to enhance performance ranks fourth in both the methods. listening skills, which was the last ranked variable in delphi, emerged as the sixth most influential variable in the fimle model. this is a significant deviation among the two methods. extent of team member turnover, which ranks sixth in delphi ranks eighth in the fimle model. frequent conflicts among project team members, ranks seventh in both the methods. turnover at organizational level is eighth in delphi, but emerges significant only in meeting organizational goals in the fimle model. performance measurement of individual members ranks ninth in both the methods. however, for fimle, it impacts only at the project level and not at the customer or organizational levels. hence, broadly we can state that except for a couple of cases, the results of the delphi and the fimle techniques are mostly synchronized. the triangulation of the findings from multi-method research thus provides a comprehensive response to rq4. based on findings, we can infer that it is possible to develop a comprehensive picture that can triangulate the findings from multi-method strategies with converging outcomes. 7.2 theoretical contribution the present study contributes to the literature of project management and hrm in different ways. first, from a methodological point of view, this study is one of the few to consider a simultaneous equations model with multiple outcome variables to investigate the impact of hraps developed from a delphi technique on value-driven it project management. second, while shortlisting twenty top ranked hraps from a total of 136 measures that are globally australasian journal of information systems bhoola & giangreco 2018, vol 22, research article hr activities and practices for project success 20 acceptable (rq1), we were able to distinguish the top ten hraps that are more relevant to the indian it projects (rq2). third, this is one of the few studies that consider project success from the viewpoint of three major stakeholders that helped to capture the differentiated impact of hraps on project, customer and organizational goals, using the fimle simultaneous system of equations. though no major difference in the adjusted r-squared of the three equations of the fimle model emerged, this study shows that hraps have the highest influence on project goals, followed with customer and then, organizational goals (rq3). this means that valuedriven project management is not about the project alone, but also about meeting customers’ expectations and organizational goals. last, in the methodological triangulation of the results generated from the delphi method and simultaneous equations model, there was sufficient evidence to believe that there was congruence in the findings from the qualitative and quantitative models, thereby claiming successful triangulation (rq4). more specifically, in relation to “how do the most important hr practices influence valuedriven project success” (rq3), we find that good people management skills of project leadership, project team communication, training of project leaders, maintaining currency and competence, job-matching to improve performance and effective listening skills are certain key hraps that positively impact project goals. these factors also play crucial roles in meeting customer goals and organizational goals, albeit weaker than meeting project goals. these results point to the fact that a mix of harder hraps (e.g., job-matching to improve performance), mixed with soft ones (e.g., effective listening skills) need to be combined in order to ensure the success of it projects. partially not expected, team member turnover adversely impacts only the project and not the customers or the organization, as a whole. this output is only somehow consistent with weimann et al. (2013) who discuss how personnel turnover at organizational level can adversely impact project performance, and in turn, lead to customer dissatisfaction. similarly, attrition at the top management level has little impact at the project level or clients. the results of the study also underlined the importance of managing conflicts as they might adversely impact at all levels, though not strongly. 7.3 managerial implications the findings of our study point alto to several managerial implications. whereas from the researchers’ point of view, the results tend to validate the influence of a set of hr applications and practices (hrap) on it project management, for hr professionals, this study provides crucial information on context-specific nature of hrap which can be used to develop new hr schemes and policies for it project-oriented organizations. while the study is typical to the indian business environment, many of the findings are applicable to other emerging economies like, philippines, vietnam, malaysia, mexico, thailand, etc., where hraps require further streamlining. similarly interesting are the insights for both project leaders and hr managers in indian it organizations. though developments in hrm are well documented in literature, its link with it project-oriented organizations and value-driven project success is rare. our results point at four most significant elements for efficient hrap in it projects that not only impact the project, but also customer and organizational goals, to ensure value-driven project management. these are people management skills of project leadership, team communication, training of project leaders, ensuring job-matching and listening to team members’ issues. often, in indian it firms, the project leadership is technically sound and highly result-oriented, though low on managing people. we suggest that potential leaders should be identified early australasian journal of information systems bhoola & giangreco 2018, vol 22, research article hr activities and practices for project success 21 and their training should actively include people management and the criticality of effective communication to enable project members to work in teams, amicably. the softer skills are often ignored in it projects, as technology takes precedence (carson and kelly, 2015). this has adverse long-term impact on project, customers and organisation. it is also not uncommon in indian it firms to allocate employees projects, not because they specialize in the areas, but, because there are vacant slots to be filled. while this help in short-term gains in ensuring labour utilization and saturation, however, in the long term it adversely impacts organizational commitment of employees, thereby leading to reduced productivity. such acts can potentially lead to employee turnover, which is another significant challenge that the hr deals with, in india. also, project leadership has to take a crucial role to reduce frequent team conflicts, by opening the doors of smooth communication. both, frequent conflicts among team members and employee turnover negatively impact it project success. although there is no simple recipe to ensure project success, undoubtedly effective hrap plays a vital role in motivating project team members to promote value-driven project management. efficient hraps not only positively impact projects, but also other stakeholders like customers and organizations. this is particularly true for it projects, which are often highly technology driven and are frequently subject to change in project scope (sutterfield et al., 2006). clearly, this calls for a transition of hr from traditional roles to roles that integrate all stakeholders who are directly or indirectly associated with the project, in order to achieve value-driven success (stulgienė and čiutienė, 2012). it is a tightrope walk to ensure the right balance of leadership, communication, motivation and also managing conflicts and turnover; leading to project success on one hand, and gaining the confidence of team members, customers and organization, on the other (todorović et al., 2013). in india, unlike the developed countries where responsibilities are streamlined and agendadriven, hrap is often inclusive of technical managers and it project leaders. in most cases, these managers and leaders work hand-in-hand with hr and take personal interest in selection, recruitment, assessment of employees, training, diffusion of the culture of the company, and team building. this is in addition to their regular managerial activities. in such situations, it is therefore favourable to replicate what many medium-sized or even large organizations do by using hr generalists to help execute projects and align with organizational goals, as well. hr department can focus on employee training and motivation schemes that can include enhancing knowledge and skills for better performance. a major deficiency noted in indian it project management is lack of listening to employees’ grievances, especially when there are conflict between team members or with project seniors. often lending a patient hearing to employees’ problems and being empathetic, can provide solace and inspire them. motivating team members can potentially solve team conflicts and enhance team communication and project performance (claudia, 2010). effective communication also includes consultation with employees before assigning a project. the hierarchy-based indian work culture often restricts leaders to consult with employees. such consultations can reduce significant burden of team members and project manager. the role of the hr department in major decisions of promotions or transfers should be in conjunction with project managers to ensure project team cohesiveness and effectiveness, thereby effectively managing attrition. in other words, the hr department can not only be an effective watchdog but also the ringmaster in empowering project leadership, thereby keeping a vigilant eye on the alignment of project goals with client requirements and organizational vision to ensure inclusive growth, in that becoming a true business partner. australasian journal of information systems bhoola & giangreco 2018, vol 22, research article hr activities and practices for project success 22 8 conclusions in this study we adopt a multi-method research methodology. the delphi method is used to identify the top 20 hraps that influence the success of value-driven project management in india in terms of meeting project, customer and organizational goals. using a multi-method research approach, we also attempt to check for triangulation and achieve reasonable convergence. the results show that efficient project leadership, effective communication, relevant training, listening to team members, and avoiding conflicts and turnover are some of the crucial hr practices that can help in attaining value-driven project management. also, the immediate impact of effective hraps is on the it project; the impact on meeting customers and organizational goals comes next. the study clearly depicts that the global best practices in hrap cannot be blindly followed in india; special consideration to customs, cultures, geographical and demographic diversity requires customized approach to hr for effective results in it projects. the use of self-assessed scales in developing the questionnaire for the quantitative model might raise common problems of method variance, though they all show proper psychometric capabilities. however, we argue that the triangulation approach largely takes care of the validity issues. moreover, our study focuses on hraps and their influence on value-driven project management. impact of other factors that are project-specific (e.g., technical knowhow, tools and technology, project estimation and planning, gold plating, etc.), organizational (e.g., lack of managerial commitment, organizational structure, reward mechanism, leadership strategies, linking project to business objectives, cross-department knowledge sharing, etc.) or environmental (e.g., inadequate third party performance, working with contractors and sub-contractors, change of project scope, cross cultural issues, business uncertainty, etc.) are ignored in which can have significant influence of project performance (uzzafer, 2013). finally, the study concentrates on a specific geographical context indian it projects. a comparative analysis across different countries with a cross-border perspective can be very enriching in validating the results and revealing newer dimensions of research. our research is certainly not free from limitations. this provides scope for future studies. this study is cross-sectional in nature and this limits the possibility for rigorous testing of the causeand-effect relationship between the independent and outcome variables, across time periods. a longitudinal study that can incorporate the managerial implications of the current findings on the functioning of hr departments of it project-based organizations can potentially derive interesting outcomes. a study using randomized control trials with control groups where the hr department get no inputs for improvement and the other that receives the experimental treatment based on the findings of this study, can effectively test the influence of hraps on it project success. references abdel-hamid, t.k. (1992), “investigating the impacts of managerial turnover/succession on software project performance”. journal of management information systems, vol. 9 no. 2, pp. 127-144. agarwal, r. n., & mehta, a. (2014). impact of performance appraisal and working environment on the job satisfaction and attrition problem in the indian it industry. paradigm, 18(1), 73-85. australasian journal of information systems bhoola & giangreco 2018, vol 22, research article hr activities and practices for project success 23 basten, d., joosten, d., mellis w. and wallmueller, c. (2014), “keep it simple–the challenge of interlaced it architecture at gothaer systems”. journal of information technology teaching cases, vol. 4 no. 1, pp. 34-40. bathallath, s., smedberg, å, & kjellin, h. (2017). impediments to effective management of project interdependencies: a study of it/is project portfolios. journal of electronic commerce in organizations (jeco), 15(2), 16-30. batistič, s., & kenda, r. (2018). toward a model of socializing project team members: an integrative approach. international journal of project management, 36(5), 687-700. belout, a., & gauvreau, c. (2004). factors influencing project success: the impact of human resource management. international journal of project management, 22(1), 1–11. bhoola, v. (2015), “impact of project success factors in managing software projects in india: an empirical analysis”. business perspectives and research, vol. 3 no. 2, pp. 109-125. bhoola, v. (2016), businesses need to pursue value-driven project management. training & development, available at: https://www.td.org/publications/magazines/td/tdarchive/2016/12/webex-businesses-need-to-pursue-value-driven-projectmanagement. (accessed 19 may 2017) blaskovics, b. (2016), “the impact of project manager on project success -the case of ict sector”. society and economy in central and eastern europe, vol. 38 no. 2, pp. 261-281. brancheau, j.c. and wetherbe, j.c. (1987), “key issues in information systems management”. mis quarterly, vol. 11 no. 1, pp. 23-45 budhwar, p.s., varma, a. and patel, c. (2016), “convergence-divergence of hrm in the asiapacific: context-specific analysis and future research agenda”. human resource management review, vol. 26 no. 4, pp. 311-326. butler, a.g. (1973), “project management: a study in organizational conflict”. academy of management journal, vol. 16 no. 1, pp. 84-101. carr, s.c., mcwha, i., maclachlan, m. and furnham, a. (2010), “international–local remuneration differences across six countries: do they undermine poverty reduction work?” international journal of psychology, vol. 45 no. 5, pp. 321-340. carson, c. w. and kelly, p. (2015). soft skills are vital for effective project controls. cost engineering. 57(2), pp 4 – 11. chapman, d. r., fair, r. c. 1972. full-information maximum likelihood program: user's guide. princeton university, econometric research program. claudia, l.c. (2010), “issues of human resources, members of project teams in the ngos: a case study”. annals of the university of oradea, economic science series, vol. 19 no. 2. coetzer, and gibbison, g. (2016), “mediating influence of time management on the relationship between adult attention deficit and the operational effectiveness of project managers”. journal of management development, vol. 35 no. 8, pp. 970-984. collins, l. m., schafer, j. l., & kam, c. m. (2001). a comparison of inclusive and restrictive strategies in modern missing data procedures. psychological methods, 6(4), 330. cronbach, l.j. (1990), essentials of psychological testing. harper & row publishing, new york. australasian journal of information systems bhoola & giangreco 2018, vol 22, research article hr activities and practices for project success 24 cuellar, m. (2010), “assessing project success: moving beyond the triple constraint”. international research workshop on it project management, paper. 13. diegmann, p., basten, d. and pankratz, o. (2017), “influence of communication on client satisfaction in information system projects: a quantitative field study. project management journal, vol. 48 no. 1, pp. 81-99. esteves, j.m. (2014), “an empirical identification and categorisation of training best practices for erp implementation projects”. enterprise information systems, vol. 8 no. 6, pp. 665683. farndale, e., scullion, h. and sparrow, p. (2010), “the role of the corporate hr function in global talent management”. journal of world business, vol. 45 no. 2, pp. 161-168. florentine, s. (2016), “more than half of it projects still failing”, available at: http://www.cio.com/article/3068502/project-management/more-than-half-of-it-projectsstill-failing.html. (accessed 5 may 2017) foss, n.j., pedersen, t., fosgaard, m.r., and stea, d. (2015), “why complementary hrm practices impact performance: the case of rewards, job design, and work climate in a knowledge-sharing context. human resource management, vol. 54 no. 6, pp. 955-976. frederiksen, a., lange, f. and kriechel, b. (2017), “subjective performance evaluations and employee careers”. journal of economic behavior & organization, vol. 134, pp. 408-429. gareis, r. (2013), “changes of organizations by projects”. international journal of project management, vol. 28 no. 4, pp. 314-327. graham, j. w., 2009. missing data analysis: making it work in the real world. annual review of psychology, 60, 549-576. hair, j. f. j., black, w. c., babin, b. j., & anderson, r. e. (2010). multivariate data analysis seventh edition prentice hall. hoda, r., & murugesan, l. k. (2016). multi-level agile project management challenges: a selforganizing team perspective. journal of systems and software, 117, 245-257. huemann, m. (2010), “considering human resource management when developing a project-oriented company: case study of a telecommunication company”. international journal of project management, vol. 28 no. 4, pp. 361-369 hughes, d. l., rana, n. p., & simintiras, a. c. (2017), “the changing landscape of is project failure: an examination of the key factors”. journal of enterprise information management, vol. 30 no. 1, pp. 142-165. innotas – “the project and portfolio management landscape, 2015”, available at: http://go.innotas.com/surveyresults-theproject-and-portfoliomanagement2015edition_.html. (accessed 13 may 2017) jean-sébastien, v. (2016), “why projects fail.” [white paper], available at: https://www.linkedin.com/pulse/why-projects-fail-jean-sébastien-vachon (accessed 21 october 2016). jick, t.d. (1979), “mixing qualitative and quantitative methods: triangulation in action”. administrative science quarterly, vol. 24 no. 4, pp. 602-611. australasian journal of information systems bhoola & giangreco 2018, vol 22, research article hr activities and practices for project success 25 jiang, j.j., chang, j.y., chen, h.g., wang, e.t. and klein, g. (2014), “achieving it program goals with integrative conflict management”. journal of management information systems, vol. 31 no. 1, pp. 79-106. joslin, r. and müller, r. (2016), “the impact of project methodologies on project success in different project environments”. international journal of managing projects in business, vol. 9 no. 2, pp. 364-388. katou, a.a., budhwar, p.s. and patel, c. (2014), “content vs. process in the hrm-performance relationship: an empirical examination”, human resources management, vol. 53, pp. 527– 544. keegan, a., huemann, m. and turner, j.r. (2012), “beyond the line: exploring the hr responsibilities of line managers, project managers and the hr department in four project-oriented companies in the netherlands, austria, the uk and the usa”, international journal of human resource management, vol. 23 no. 15, pp. 3085-3104. keil, m., lee, h.k. and deng, t. (2013), “understanding the most critical skills for managing it projects: a delphi study of it project managers”. information & management, vol. 50 no. 7, pp. 398-414. kennedy, d.m., sommer, s.a. and nguyen, p.a. (2017), “optimizing multi-team system behaviors: insights from modeling team communication”. european journal of operational research, vol. 258 no. 1, pp. 264-278. khan, a.s. and rasheed, f. (2015), “human resource management practices and project success, a moderating role of islamic work ethics in pakistani project-based organizations”, international journal of project management, vol. 33 no. 2, pp. 435-445. korsakienė, r., stankevičienė, a., šimelytė, a. and talačkienė, m. (2015), “factors driving turnover and retention of information technology professionals”, journal of business economics & management, vol. 16 no. 1, pp. 1-17. koulikoff-souviron, m., & harrison, a. (2010). evolving hr practices in a strategic intra-firm supply chain. human resource management, 49(5), 913-938. lee, o.k. and baby, d.v. (2013), “managing dynamic risks in global it projects: agile riskmanagement using the principles of service-oriented architecture”. international journal of information technology & decision making, vol. 12 no. 6, pp. 1121-1150. majumdar, s.k. (2013), “r&d spending and the rewards to human capital in india’s it sector”, vikalpa, vol. 38 no. 4, pp. 37. momeni, k., & martinsuo, m. m. (2018). allocating human resources to projects and services in dynamic project environments. international journal of managing projects in business, 11(2), 486-506. morse, j.m. (2003), “principles of mixed methods and multimethod research design”, in tashakkori, a. and teddlie, c. (eds.), handbook of mixed methods in social and behavioral research, sage publications, thousand oaks (ca., usa), pp. 189-208. nasscom. (2017), “the it-bpm sector in india: strategic review 2017”, available at: http://www.nasscom.in/research-reports. (accessed 12 may 2017) australasian journal of information systems bhoola & giangreco 2018, vol 22, research article hr activities and practices for project success 26 parker, s.k. and skitmore, m. (2005), “project management turnover: causes and effects on project performance”, international journal of project management, vol. 23 no. 3, pp. 205214. prakash, a. (2016), “e-governance and public service delivery at the grassroots: a study of ict use in health and nutrition programs in india”. information technology for development, vol. 22 no. 2, pp. 306-319. purvanova, r.k. (2013), “the role of feeling known for team member outcomes in project teams”. small group research, vol. 44 no. 3, pp. 298-331. remus, u. and wiener, m. (2010), “a multi‐method, holistic strategy for researching critical success factors in it projects”, information systems journal, vol. 20 no. 1, pp. 25-52. rockart, j.f. (1979), “chief executives define their own data needs”. harvard business review, vol. 57 no. 2, pp. 81-93. roebuck, d., bell, r. l., raina, r., & lee, c. e. c. (2015). the effects of home country, gender, and position on listening behaviors. sahoo, b. and nauriyal, d. (2014), “determinants of software exports from india”. international economics and economic policy, vol. 11 no. 4, pp. 455-479. santos, c., gonzalez, t., li, h., chen, k.y., beyer, d., biligi, s. and zhang, a. (2013), “hp enterprise services uses optimization for resource planning”. interfaces, vol. 43 no. 2, pp. 152-169. schmidt, r.c. (1997), “managing delphi surveys using nonparametric statistical techniques”. decision sciences, vol. 28 no. 3, pp. 763-774. schmidt, r., lyytinen, k. keil, mark and cule, p. (2001), “identifying software project risks: an international delphi study”. journal of management information systems, vol. 17 no. 4, pp. 5-36. shenhar, a. j., dvir, d., levy, o., & maltz, a. c. (2001). project success: a multidimensional strategic concept. long range planning, 34(6), 699-725. standish group’s chaos report outline (2016), available at: http://www.standishgroup.com/outline (accessed 15 april 2017) stulgienė, a. and čiutienė, r. (2012), “hr challenges in transition to project management (project-based organization)”. economics and management, vol. 17 no. 3, pp. 1214-1218. sutterfield, j.s., friday-stroud, s.s. and shivers-blackwell, s.l. (2006), “a case study of project and stakeholder management failures: lessons learned”. project management quarterly, vol. 37 no. 5, pp. 26. thite, m., budhwar, p.s. and wilkinson, a. (2014), “global hr roles and factors influencing their development: evidence from emerging indian it services multinationals”. human resource management, vol. 53 no. 6, pp. 921-946. todorović, m.l., mitrović, z. and bjelica, d. (2013), “measuring project success in projectoriented organizations”. management, vol. 68, pp. 41-48. tripathi, s.s. (2014), “the role of managers as agents in successful service innovations: evidence from india”, technology innovation management review, august, pp. 18-26. australasian journal of information systems bhoola & giangreco 2018, vol 22, research article hr activities and practices for project success 27 turner, j. r., & müller, r. (2005). the project manager’s leadership style as a success factor on projects: a literature review. project management journal, 36, 49–61. uzzafer, m. (2013), “a contingency estimation model for software projects”. international journal of project management, vol. 31 no. 7, pp. 981-993. van marrewijk, m., & timmers, j. (2003). human capital management: new possibilities in people management. journal of business ethics, 44(2), 171-184. van dierendonck, d., lankester, a., zmyslona, m. and rothweiler, h. (2016), “linking hrm practices and institutional setting to collective turnover: an empirical exploration”. administrative sciences, vol. 6 no. 4, pp. 18. viji, b. (2016), “enhancement of employee performance through competency mapping in it sector – an empirical analysis”, clear international journal of research in commerce & management, vol. 7 no. 9. watson, t. (2008), “public relations research priorities: a delphi study”. journal of communication, vol. 12 no. 2, pp. 104-123. weimann, p., pollock, m., scott, e. and brown, i. (2013), “enhancing team performance through tool use: how critical technology-related issues influence the performance of virtual project teams”. ieee transactions on professional communication, vol. 56 no. 4, pp. 332-353. zhu, y.q. and kindarto, a. (2016), “a garbage can model of government it project failures in developing countries: the effects of leadership, decision structure and team competence”. government information quarterly, vol. 33 no. 4, pp. 629-637. zhou, e.i., lu, z., li, x., li, t., papola, t.s., pais, j. and sahu, p.p. (2010). “remuneration differences in the emerging economies of china and india”. international journal of psychology, vol. 45, pp. 360-370. zidane, y. j. t., & andersen, b. (2018). the top 10 universal delay factors in construction projects. international journal of managing projects in business, 11(3), 650-672. australasian journal of information systems bhoola & giangreco 2018, vol 22, research article hr activities and practices for project success 28 appendix 1 list of hr activities and practices (hrap) items used in the study no hrap no hr activities and practices 1 access to internal personnel 23 just and fair treatment of employees in workplace 2 adequate training of project team members 24 lack of english communications skills of team members 3 adequate training to ensure competence 25 listening 4 appointment of project manager / leader for the project 26 making employees feel important 5 balancing standardization and customization 27 performance measurement of individual members 6 building a global work culture 28 periodic formal review of status reports versus plan 7 conflicting and continuous requirement changes 29 personnel turnover at organizational level 8 consultation with employees before assigning a project 30 proactiveness and helpfulness of hr department 9 creating congenial environment for sharing 31 project leadership and people management skills 10 diversification within the project team 32 promoting a women-friendly work environment 11 effectiveness of project team communication 33 promoting formal risk management practices 12 embedding organizational core values in projects 34 promoting innovation 13 extent of team member turnover (members resigning) 35 promotions & salaries and individual performances 14 freedom of project manager to select the project team 36 recognition through rewarding for working beyond office hours 15 freedom of selection of development platforms & tools 37 relevant experience of team members 16 frequent conflicts between project team members 38 respecting indian cultures and ethnic diversity 17 frequent shuffling of project team members 39 restrictions on women members on travel & stay outside 18 holding individuals accountable for tasks assigned 40 restrictions on working hours for women members 19 individual needs and values 41 staff motivation 20 ineffectiveness of project manager 42 strict enforcement of attendance 21 inexperience of project leader 43 taking of unnecessary safety risks to perform job 22 job-matching to enhance performance 44 training of project leaders copyright: © 2018 bhoola & giangreco. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which australasian journal of information systems bhoola & giangreco 2018, vol 22, research article hr activities and practices for project success 29 permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. ajis vol. 6 no. 1 september 1998 an investigation into the feasibility of tasmanian stonefruit growers employing electronic commerce stewart pedersen school of information systems faculty of commerce and law university of tasmania hobart, australia email: stewart.pedersen@infosys.utas.edu.au abstract this research presents an assessment of the feasibility of tasmanian stonefruit growers adopting electronic commerce. in order to achieve this aim. a number of qualitative techniques were employed under an action research methodology. although not all applications of electronic commerce are suitable within the environment under study, it is suggested that there are ways in which electronic commerce could provide value. however, inhibitors, some common to small to medium sized enterprises (smes), do exist. there is a need for these issues to be understood prior to adopting electronic commerce. keywords hb03 agribusiness is, aiolol action research, da0201 small business introduction it is well documented that many organisations are realising benefits from the utilisation of electronic commerce. for example, some are employing edi and jit to manage their supply chains (mackay 1996; schware & kimberley 1995b; mccarthy et al. 1997). others are harnessing the internet to distribute product information, reach niche" markets and so forth (plunkett 1997; mccarthy etal. 1997; kalakota & whinston 1996). however, not all electronic commerce applications are suitable in all circumstances. selling products via the internet is one such case. internet sales gregor et al. (1997:599) argue that "other things being equal, the suitability of products for the direct market of the world wide web increases with (i) lower levels of asset specificity, (ii) higher levels of information intensity and, (iii) higher levels of adequacy of description". further, gregor et al. (1997:592) state that "many agricultural products have asset specificity in that they are time specificthey need to be marketed, or reach a consumer, within a specified, relatively limited time period". the authors note that the agricultural products being sold directly to customers via the internet, such as bull semen and farm supplies, have low asset specificity. they can be stored suitably for an extended period of time. information intensity refers to the amount of information required to describe a product. the more information required, the higher the information intensity. adequacy of description refers to the degree to which the product can be satisfactorily described. products such as bull semen are more suitable for internet sales because actually viewing the product would not reveal more information (gregor et al. 1997). however, this may not be the case for other agricultural commodities, particularly those bought fresh for consumption. disintermediation the sale of products directly over the internet can involve the bypassing of entities commonly found in a supply chain, such as wholesalers and retailers. edi can also support this. indeed, one of strengths of electronic commerce is that it allows for the removal of those intermediaries who are not adding valued industries where this has occurred include airlines (noe etal. 1997; row 1996) and publishers (golden 1996). 6 blanchard, p., (4/8/97) personal conversation, tradegate ega. 79 ajis vol. 6 no. 1 september 1998 intermediaries in agriculture however, it is possible to argue for the necessity of intermediaries for some agricultural products. intermediaries perform a variety of functions for those above and below them in the supply chain. for example, amongst other things, wholesalers assist suppliers (generally producers or importers) by providing market information. they also assist customers (in this case, retailers) by providing prompt, low cost delivery (mccarthy et al. 1997). matsuda (1994) notes the importance of the role of wholesalers in the agricultural industry. this role derives from the nature of the industry. from the point of view of supply, there are large numbers of small, geographically dispersed growers, and from the point of view of demand, there are also great numbers of dispersed consumers. the wholesale markets act to bridge the gap between growers and consumers. kinsey (1994) suggests changes in consumer attitudes regarding food consumption have resulted in a move towards small, diverse and customised niche" markets, forcing retailers to handle broader lines of commodities more efficiently. as a corollary, retailers prefer to purchase the commodities they intend to sell at a market where all the commodities they require are assembled. further, matsuda et al. (1997) state that wholesale markets are more efficient and effective at finding an accurate market price. neither the farmers, who do not individually produce sufficient volumes, nor the consumers, who purchase only small amounts of produce, can influence the market price. both are considered price takers (matsuda 1996). intermediaries, in the form of wholesalers, perform an important pricing function, and add value. these factors have contributed to the need for intermediary institutions to coordinate transactions in relation to supply and demand. mccarthy et al. (1997:412) hold that "eliminating wholesalers would not eliminate the need for the functions they provide, and it would be wrong to assume that direct channels would necessarily be more efficient". value in agriculture electronic commerce can add value in agricultural endeavours. gregor et al. (1997) use porter's five-forces model to illustrate. porter (1990) offers the following as factors that shape an industry: • bargaining power of suppliers; • bargaining power of buyers; • threat of substitute products or services; • rivalry among existing competitors; and, • threat of new entrants. the following is a summary provided by gregor et al. (1997:592) of three generic strategies to counter these forces, and the manner in which they can be supported by electronic commerce: • cost leadership, which can be achieved through cheaper product promotion, i.e. via the internet; • product differentiation, which can be achieved by cost leadership and improved customer service and delivery; and, • innovation in business methods, which can be achieved by improved methods of production, marketing and distribution. for example, increasing potential marketing exposure via the internet. infrastructure given that electronic commerce can provide value in agricultural endeavours, it is important to note that any attempt to employ electronic commerce within a given industry requires an assessment of the technical infrastructure (bloch et al. 1996). authors such as kalakota and whinston (1996) and schware and kimberley (1995a, 1995b) indicate that the benefits of electronic commerce are reliant upon other trading partners employing electronic commerce. even if a specific industry has the hardware and software requirements, having the ability to use electronic commerce is of little use if other necessary entities in the value or supply chain are unable to participate. change as the uptake of electronic commerce essentially involves change, attention must be paid to the organisational as well as technical infrastructure. bloch et al. (1996:110) state that "...building any kind of electronic commerce system requires a strategy, i.e. a vision of the goal to achieve...". change management literature illustrates that past 80 ajis vol. 6 no. 1 september 1998 attempts to change are important to consider when planning new changes. historical failures and successes can have positive and negative effects, and intrinsically have an impact on the culture within an organisation or industry. as noted by burnes (1996) and dawson (1996), if the proposed changes contradict the existing culture, implementation is considerably more difficult. both the technical and organisational infrastructure must be taken into account when considering incorporating new methodologies within a given industry. research objectives and question the utilisation of electronic commerce is permeating all industry sectors, including primary industry. primary industry has traditionally been important to the tasmanian economy. the growing of stonefruit has an established history within the state, and forecasts predict a rise in production. given the increased use of electronic commerce globally, and an expanding stonefruit industry, it is necessary for tasmanian stonefruit growers to have an understanding of how electronic commerce technologies may be applied within their business processes. thus, the objectives of this research were as follows: • to discover how electronic commerce could be employed by tasmanian stonefruit growers; and, • to empower the tasmanian stonefruit growers with this knowledge and engender an understanding of the possible applications and limitations. to address these objectives, the following research question was been formulated: what potential is therefor the utilisation of electronic commerce by tasmanian stonefruit growers! this investigation is important, not only in view of the increasing use of electronic commerce, but also because funding for agricultural research in australia and most overseas countries has been scaled down over the last 10-15 years. research into the areas of production, marketing and infrastructure must continue if our agricultural industries are to remain viable . research design and methodology given that pursuit of the research question required the researcher's active participation in the environment under study, the subjective ontology associated with interpretivism was appropriate. further, achievement of the research objectives requires the participation of growers themselves. hence an action research methodology was adopted. the concept of action research stems from the work of social psychologist kurt lewin (burnes 1996). lewin formulated a model consisting of a series of spiraling actions, with a distinct emphasis on group activity (schermerhorn et al. 1988). the methodology associated with lewin's model falls under the interpretivist approach (acterberg et al. 1991). bailey et al. (1991:504) offer the following definition: action research is a process of systematically collecting data on an organisation, feeding it back to the members for action planning and evaluating results by collecting and reflecting on more data after the planned actions have been taken. data collection and analysis in order to gain an understanding of the pertinent issues, data collection is a major part in the action research process (schermerhorn et al. 1988). the methods utilised in the collection of data are situationally dependent; the researcher, or change agent, asks questions, conducts interviews, distributes questionnaires, makes field notes and listens to the concerns of those involved (schermerhorn et al. 1988; robbins 1994). however, in all cases these activities are carried out participatively (burnes 1996). in this research a variety of methods were used to collect data, including those mentioned above. the available literature was approached for instruction on the employment of the data collection techniques. pilot studies were performed prior to the administration of a quantitative questionnaire, as the literature suggests (minichiello et al. 1995). the questionnaire was administered to 60 of the 80 growers. the results formed a basis for further investigation. data collected through field notes, personal conversations and interview transcripts were analysed using grounded theory to identify common themes (strauss 1987). brown, g., (26/09/97) personal conversation, department of primary industry and fisheries, hobart, tasmania 81 ajis vol. 6 no. 1 september 1998 validity validity is an important factor in qualitative research, as the researcher intrinsically applies their own norms and values when analysing the data. for this reason the researcher must verify the information with the participants to ensure the interpretations are accurate. neuman (1994:357) states that "member validity occurs when a researcher takes field results back to members, who judge their adequacy". participants confirmed key research findings, and thus validity was achieved. external validity is the ability to generalise experimental findings to events and settings outside the experiment itself (neuman 1994:185). high external validity means the results can be generalised to many situations and many groups of people. low external validity means that the results only apply to a very specific setting (neuman 1994:135). the researcher has not attempted to generalise beyond the scope of the study. however, it is conceivable that the findings may be applicable to other, similar, situations. relevant utilisation it is appropriate to support the selection of a methodology by offering evidence of its successful utilisation in previous investigations within similar industries. whyte (1991) reports that interest in participatory strategies for agricultural research became more widespread long after their application within other industries. since its adoption however, action research has been used as a successful means for evoking and managing change in numerous agricultural research undertakings. cyclical approach conducting the research as a participatory endeavour required that growers were aware of the research objectives and that a sufficient number of growers were willing to work towards those objectives. the identification of potential key agents of change was also necessary. throughout the investigation the researcher remained in contact with the growers by telephone, conducted visits to their orchards and used email where available. grower's queries on the subject of electronic commerce were either answered immediately, or researched and the information provided at a later date. each time information was fed back, new issues arose, and the process continued. for example, many growers originally thought that electronic commerce was limited to the ability to sell their fruit directly over the internet. growers were informed that this was only one application of electronic commerce, and that there were many more. some applications they were familiar with, such as eftpos, and some such as edi were new to most growers. once this had been established, growers asked how they could use these technologies, and in particular, how they could sell their fruit over the internet. this led to an investigation of the appropriateness of selling certain products via the internet, and to the conclusion that stonefruit was not a product suited to such sales. growers asked what they could use it for then, and the investigation revealed that the internet could be used for such things as niche" marketing of value added products. the process was one of learning for both the researcher and the growers. at all times growers were encouraged to discuss issues among themselves. it was emphasised that any decision to use electronic commerce technologies in a given fashion must be made by them. the researcher's role was that of consultant and facilitator. overview of the tasmanian stonefruit industry a precise figure regarding the total number of stonefruit growers in tasmania is difficult to acquire. the department of primary industry and fisheries considers plantings over one acre in size to be an orchard, while the australian bureau of statistics only recognises holdings returning in excess of $22,500 annually. however, in view of the nature of the industry in tasmania, with all growers falling into the sme category and about half not using growing as a primary form of income, it was deemed more appropriate to consider the dpif figures. the dpif recognises approximately 80 stonefruit orchards statewide. of these, about 55 are cherry orchards, with apricots constituting most of the remainder. cherries and apricots are popular with growers as tasmania has a longer 82 ajis vol. 6 no. 1 september 1998 growing season than the mainland. the vast majority of orchards are family owned; there are only two corporate orchards. most growers have just one stonefruit crop, but only a few grow stonefruit exclusively. of the 80 orchardists, half belong to the tasmanian stonefruit growers' association. these 40 produce around 8090% of the total stonefruit grown, and use their holdings as a primary source of income. the remaining 40 growers have other forms of employment and use stonefruit production as an income supplement. the average smaller grower produces from a holding of one or two hectares, which provides insufficient returns for sole reliance. industry trends in 1984 tasmania produced 40 tonnes of cherries. since 1987-88 there has been an increase in new orchards of about 10% annually according to the dpi, and this trend is expected to continue. cherries have been the main focus of new orchards, and it is predicted that by the year 2000 tasmania will have a production potential of about 1000 tonnes of cherries annually, not counting new plantings. currently the production potential is around 600 tonnes, although in 1996 adverse environmental conditions resulted in a yield estimated at 200 tonnes. flow of goods and information growers are responsible for a myriad of activities ranging from soil preparation and crop production to grading and marketing their produce. these activities require the exchange of goods in the form of supplies and produce, and information, such as market prices. a graphical depiction of the flow of goods and information is provided in figure 1: figure 1. flow of goods and hforrration in the tasmanian stone-fruit industry findings it is not possible to select a definitive end point to the research, as the process, under an action research methodology, is ongoing. consequently, the findings essentially take the form of a summary of what has been thus far achieved. growers' response the response rate for the questionnaire was 57.5%. the findings indicate that growers had different reactions to the idea of employing electronic commerce. of those who responded, 95% indicated they would like further information on the subject of electronic commerce. growers across all age groups expressed interest. growers' and information technology all of the growers participating in this investigation had access to a fax machine. of the 22 growers who responded to the questionnaire, 72% owned or had access to a computer, and of those, 48% had modems. of those growers so 83 ajis vol. 6 no. 1 september 1998 enabled, 33% had a connection to the internet. the findings show that growers are using a mixture of platforms. they used their computers for a variety of purposes, ranging from assisting with children's schoolwork to one respondent who wrote programs for a hobby. only three respondents indicated that they used a computer for farm management tasks. none of the participants indicated that electronic commerce activities are currently undertaken, with the exclusion of the exchange of business information via a fax machine or telephone. growers' perception of electronic commerce at the commencement of this investigation a misconception existed amongst growers that electronic commerce, in the context of their undertakings, meant the ability to sell their stonefruit over the internet. only a few growers were aware of the broader nature and implications of electronic commerce, such as edi and bar-coding. status of infrastructure the findings indicated a lack of supporting infrastructure. wholesalers on the mainland do not use edi or email, nor do local transport companies. however, growers wishing to export may communicate with customs using standardised electronic forms, or with airlines via the internet. past attempts at cooperative ventures the findings indicated that there have been a mixture of successes and failures regarding historic attempts to act cooperatively. for example, about 12 years ago, growers joined together in a cooperative venture to market their fruit. unfortunately this was unsuccessful as many growers felt the distribution of profits was unfair. this has left some older growers wary of such proposals. the findings indicate that younger growers seem more inclined to enter joint operations. growers have had cooperative successes. one such incident involved the purchase of a large number of pine posts. although the total amount purchased far exceeded the requirements of any individual grower, the bulk procurement enabled savings to be made by those involved. growers currently market their produce under a variety of brands. there is no single brand used. the tasmanian stonefruit growers association does offer generic packaging, but this is used by only a small number of growers. the findings also show that growers in general act as individuals, not as a unified group. growers' concerns growers revealed concerns when informed of the manner in which information technology and electronic commerce could be employed. they include: • their limited technical knowledge; • their limited resources; • their need for immediate return on investment; • their preference for manual methods; • their cost of establishment; and, • the difficulty in getting sufficient cooperation and commitment. the concerns essentially fit into three categories: finance, knowledge and cooperation. growers also expressed concern regarding the lack of a quality control. there is currently no universal methodology for addressing this issue in place. growers are not concerned about the amount of inventory required to be kept, nor administrative costs. however, in the past goods have been lost in transit. growers reported a need for relevant and timely information. on the basis of the survey responses and personal conversations, it was ascertained that growers have the following three main information requirements: • weather; • rural news; and, • new products. 84 ajis vol. 6 no. 1 september 1998 discussion beginning with an understanding of the technologies electronic commerce incorporates, it was possible to establish how these technologies could be applied and their suitability in particular circumstances. not all of the benefits of electronic commerce being realised in the business world are available to the tasmanian stonefruit growers. not all of the current practices are applicable. further, some factors would inhibit attempts to adopt electronic commerce practices. the literature suggests that products such as stonefruit are not suited to direct internet retailing. they are asset specific in that they need to be marketed within a short time of harvesting (gregor et al. 1997). they fail to have adequacy of description, as growers consider that consumers like to see what they are buying. a further issue, raised by the growers, is the logistics involved in getting small quantities of perishable produce to geographically dispersed consumers. on this basis it would appear that direct sales to consumers are not viable. this does not preclude electronic commerce from being be used to support relationships between other entities in the industry, such as wholesalers or retailers (mccarthy et al. 1997). this may provide benefits by lowering inventory and administration costs (mackay 1996). it is noted that inventory and administration are not of primary concern to growers at this point. however, the literature suggests that the current lack of supporting technical infrastructure would pose an important hurdle, and that there would be initial costs involved in establishing a suitable infrastructure if universal participation was desired. a further issue is that the other entities in a given industry must have the technology available to support any attempt at using electronic commerce technologies (bloch et al. 1996). it is reasonable to extrapolate that some sort of critical mass is required. this lack of infrastructure limits the short-term opportunities for stonefruit growers to employ electronic commerce (bloch et al. 1997). utilising the existing electronic communication's infrastructure, via telephones and fax machines, arguably constitutes a degree of participation in the world of electronic commerce (schware & kimberley 1995b). as noted in a recent report from monash university, some authors insist that communication involving computer applications is required (anon 1996). however, there is currently no support for edi or bar-coding. the very nature of the stonefruit industry also has an impact on the applicability of electronic commerce. as suggested earlier, the ability to facilitate disintermediation is one of the strengths of electronic commerce. however, the literature reveals strong arguments supporting the role of intermediaries in the marketing and distribution of agricultural products. these intermediaries provide value for the stonefruit growers in a variety of ways, such as bringing all of the produce and buyers together in one place, and taking the risk of payment from the customer away from the grower (mccarthy 1997). they also perform a pricing function (matsuda 1996; 1997). a further inhibitor to the adoption of electronic commerce arises from the stonefruit grower's historical attempts to change practices or act cooperatively. as noted earlier, past attempts at change have an intrinsic effect on future attempts. it was found that younger growers were more eager to adopt electronic commerce technologies. older growers, many of whom have had experience with past attempts to change, expressed more concern. potential applications of electronic commerce although the literature suggests that stonefruit is unsuited for sales directly to consumers via the internet, the internet can provide a medium for relatively inexpensive mass and niche" marketing, and for providing product information. it can also be a useful means for receiving customer feedback to facilitate improvement, an important part of marketing (mccarthy etal. 1997). the literature indicates that growers who do sell to export markets are often contacted by overseas agents to see if they can supply. luostarinen and welch (1993) report that many initial outward operations are the result of foreign inquiry or unsolicited orders; hence an internet presence could provide benefits by increasing the awareness of tasmania's stonefruit grower's capabilities. the internet is also a useful means for communicating with those associates and trading partners who have an online connection (kalakota & whinston 1996). these entities could include other growers, customs and airlines for example. another possible use is for gathering information (kalakota & whinston 1996). the internet provides a means to satisfy the grower's stated information requirements. an electronic commerce framework can also incorporate quality control. there are currently no universal quality control processes in place within the stonefruit industry. the findings indicate that growers are aware of the 85 ajis vol. 6 no. 1 september 1998 importance of quality control, and are keen to remedy the situation. information technology can assist primary producers in this matter as shown by schware and kimberley (1995b). the use of edi and/or bar-coding, if judged viable, could assist in overcoming the problem of goods going missing in transit. hill (1997) notes that this is a concern for many organisations. the findings suggest this is not a major issue for growers, but the transfer of the title of ownership of goods in transit is a grey area that could be addressed, and the solution supported by electronic commerce technologies (hill 1997). some technologies such as edi are not currently supported, and are regarded as price prohibitive in consideration of the low volume of fruit and administration. however other technologies, such as the internet are available and accessible. should growers elect to employ electronic commerce technologies, it is suggested that an incremental approach be adopted. growers could begin their electronic commerce efforts by establishing web sites that provide product information. a decision would be required as to whether one web site for all tasmanian stonefiruit growers is desired, or if growers should pursue their own line of marketing. the literature suggests a cooperative undertaking would provide the most benefit, and be least confusing to trading partners. as part of the incremental process, growers could also begin to use email where possible. they could communicate with each other, and with other entities (mccarthy el al. 1997). the next step could be the adoption of edi, if it is deemed viable and necessary. by approaching electronic commerce in this incremental manner, growers can overcome some of their concerns as their knowledge and experience increases (blanchard 1997). due to the variety of hardware and software platforms being used, there may be a role for an information or electronic broker. this entity could, amongst other things, provide technical assistance, establish and maintain a web site, and act as a central body for the distribution of information (rugge & glossbrenner 1995). thus growers could participate regardless of their individual technical and financial resource status. it is evident that there are significant possibilities for electronic commerce applications within the environment under study, and there are also given inhibiting elements. the current practices within the industry function to the general satisfaction of the growers. however it is accepted that change is both inevitable and necessary, and the specific nature of the change may be influenced or even dictated by parties other than the growers themselves. it appears that through interaction and participation growers are developing an understanding of the issues the research has raised. growers realise that there is a need for awareness of the potential of electronic commerce, and that this awareness must precede any implementation. concurrence with other studies the researcher could find no other investigations that considered the potential for the use of electronic commerce in primary industry endeavours involving smes. however, by characterising stonefruit growers as smes, it is possible to find some, more general, concurrence. lawrence (1996) found that smes had similar concerns regarding the adoption of electronic commerce. these findings were in turn supported by cameron and clarke (1996) and lee and clarke (1996). the general misconception within the tasmanian stonefruit industry regarding the possible application of electronic commerce is supported by findings from monash university. the report states that "edi, one of the subset technologies of electronic commerce, is often confused by smes as being ec" (anon 1996:13). the research findings regarding a lack of infrastructure to support electronic commerce implementation within the stonefruit industry are also reflected by the monash university report. the report states that one of the six key issues deterring smes from adopting electronic commerce is that there is "...no 'critical mass* of electronic commerce and communications partners" anon (1996:17). further research the findings provide a basis for further investigation. for example, the fact that stonefruit is not a good product to sell directly over the internet has implications for managing risk. forward selling is one way of managing the risk of price uncertainty^. further research could investigate the role electronic commerce can play in risk management. the ° kilpatrick, s., (23/09/97) personal conversation, associate director, centre for research and learning in regional australia, newham campus, university of tasmania. 86 ajis vol. 6 no. 1 september 1998 research could also provide a platform for further investigations into change management in smes. it is noted that there is a lack of information available on this matter. research into the manner in which smes information requirements evolve with the adoption of electronic commerce could also provide interesting insights. the important role that agricultural industries play in the australian economy proclaims the need to support these industries in their efforts to adopt new technologies and remain globally competitive. in this context, research can play an important role in both identifying and raising awareness of the issues, opportunities and benefits of electronic commerce to primary producers. this research may provide a useful starting point and framework for the substantial work that remains to be done. further, it indicates potential areas of interest or concern to those involved in similar industries. references achterberg, j., (1991) information systems research in the postmodern period, pp281-294, in nissen h, klein h, hirscheim r, information systems research: contemporary approaches and emergent traditions, elsevier science publishers, north holland. anon., (1996) advice on electronic commerce programs for small to medium sized enterprises (smes), prepared for the department of industry science and tourism, centre for electronic commerce, monash university. bloch, m., pigneur, y., segev, a., (1996) leveraging electronic commerce for competitive advantage: a business value framework, ninth internationa] conference on edi-ios, bled., slovenia. bumes, b., (1996) managing change: a strategic approach to organisation dynamics. pitman publishing, london. cameron, j., clarke, r., (1996) towards a theoretical framework for collaborative electronic commerce projects involving small and medium sized enterprises, ninth international conference on edi-ios, bled,. slovenia. dawson, p., (1994) organisational change, a processual approach, paul chapman publishing, london. gregor, s., newman, s., lamer, t., (1997) the australian meat industry, the electronic market hypothesis and internet marketing, tenth international conference on edi-ios, bled.,. slovenia golden, w., (1996) electronic commerce at work: kennys bookshop & art galleries, galaway, ireland, ninth international conference on edi-ios, bled., slovenia. hill, c, (1997) international business: competing in the global marketplace, 2nd ed., irwin books., u.s.a. kalakota, r., whinston, a., (1996) frontiers of electronic commerce, addison wesley, new york. kinsey, j., (1994) changes in food consumption from mass marketing to niche' market, in matsuda, t., (1997) agricultural wholesale markets in an electronic commerce environment, reprinted from the technical bulletin of the faculty of horticulture, chiba university, no. 51. lawrence, k., (1996) electronic commerce & smes, university of tasmania, honours thesis lee, h., clark, r., (1996) economic benefits and adoption barriers of electronic market systems, ninth international conference on edi-ios, bled., slovenia. luostarinen, r. and welch, l., (1993) international business operations, kyiiri oy, helsinki. mackay d., (1996) edi and the australian automotive industry, in fischer, in fischer, l., electronic commerce, profiting from business on-line, future strategies inc. book division, florida. matsuda, t. 1994, the use of information technology to achieve accurate pricing in agricultural commodity markets in japan, information technology and people, mcb university press, vol. 7 no. 3, pp 37-49. matsuda, t., (1996) electronic commerce and electronic markets for agriculture, ninth international conference on edi-ios, bled., slovenia. matsuda, t., (1997) agricultural wholesale markets in an electronic commerce environment, reprinted from the technical bulletin of the faculty of horticulture, chiba university, no. 51 matsuda, t., clark, t. and lee, h., (1997) electronic commerce for agricultural transactions: role of intermediaries and accurate pricing, in j.f. nunamaker and r. h. sprauge (eds.). proceedings of the thirteenth annual hawaii international conference on systems science, vol fv, 13-30. mccarthy, e., perreault, w. and quester, p., (1997) basic marketing, a managerial approach, irwin publishing, london. minichiello, v., aroni, r., timewell, e., alexander, l., (1995) indepth interviewing, 2nd ed., longman australia. 87 ajis vol. 6 no. 1 september 1998 neuman w l., (1994) social research methods: qualitative and quantitative approaches, 2nd edition, allyn and bacon, sydney. noe, r., hollenbeck, j., gerhart, b., wright, p. (1997) human resource management: gaining a competitive advantage, 2nd ed., irwin books, burr ridge. plunkett, s., (1997) "£' is for extracting the internet-commerce digit", business review weekly, vol. 19, no. 35. porter, m., (1990) the competitive advantage of nations, the free press, new york. robbins, s., and barnwell, n., (1994) organisational theory in australia. 2nd ed., prentice hall, sydney. row, h., (1996) the electric handshake, datamation, october. rugge, s., glossbrenner, a., (1995) the information brokers handbook, 2nd ed. mcgraw hill, new york. schermerhorn, j., hunt, j., osborn, r., (1988) managing organisational behaviour, 3rd ed., john wiley and sons, new york. schware, r. and kimberley, p., (1995a) information technology and national trade facilitation: making the most of global trade. the world bank, technical paper no. 316. schware, r. and kimberley, p., (1995b) information technology and national trade facilitation: guide to best practice. the world bank, technical paper no. 317. strauss, a., (1987) qualitative analysis for social scientists, cambridge university press. england. wyte, w., (1991) participatory action research, sage publications, california. acknowledgements the author would like to extend unreserved gratitude to the tasmanian stonefruit growers for their participation in this investigation. thanks also the academic staff and post-graduates from the school of information systems, university of tasmania, for their invaluable input. 88 paper title (paper title style) australasian journal of information systems hamzehei, jiang, koutra, wong & chen 2017, vol 21, selected papers from ausdm topic-based social influence measurement 1 topic-based social influence measurement for social networks1 asso hamzehei university of new south wales assoh@cse.unsw.edu.au shanqing jiang university of new south wales danai koutra university of michigan raymond wong university of new south wales fang chen university of new south wales abstract social science studies have acknowledged that the social influence of individuals is not identical. social networks structure and shared text can reveal immense information about users, their interests, and topic-based influence. although some studies have considered measuring user influence, less has been on measuring and estimating topic-based user influence. in this paper, we propose an approach that incorporates network structure, usergenerated content for topic-based influence measurement, and user’s interactions in the network. we perform experimental analysis on twitter data and show that our proposed approach can effectively measure topic-based user influence. keywords: topic-based social influence, social networks analysis, influence measurement 1 introduction although social influence has been an area of interest for researchers in sociology and more recently in computer science, still there is no agreement on its definition. a very early definition for influential people is “individuals who were likely to influence other persons in their immediate environment” (katz 1957). social influence has either been studied to identify influential users (opinion leaders or authorities), topical or topic-based influential users (riquelme 2015). social science studies, e.g. (katz and lazarsfeld 1955), have acknowledged the fact that the social influence of individuals is not identical. katz (1957) introduced three main factors that are related to an individual’s social influence such as: who one is, what one knows, and whom one knows. the individual’s social influence can be much more easily observed on social media while it is confirmed that the social influence factors are similar in social networks to those in the real society (eccleston and griseri 2008, libai, bolton et al. 2010). for example, eirinaki, monga et al. (2012) introduced two factors (popularity and activity) as factors related to social influence on online social networks (osn). one of the main measures studied for influence is information diffusion, which measures how important a user is in spreading information in the network. this is equivalent to identify central and hub nodes in the network (hajian and white 2011, jin and wang 2013). opinion leaders and discussion starters also have been studied as a measure of social influence (jabeur, tamine et al. 2012). a user’s position in the network (jin and wang 2013), content (hu, fang et al. 2013), and activities (pal and counts 2011) have been also studied as influence measures. 1 an earlier version, (hamzehei et al. 2016), of this work is presented at ausdm16. australasian journal of information systems hamzehei, jiang, koutra, wong & chen 2017, vol 21, selected papers from ausdm topic-based social influence measurement 2 another aspect of studied influence has been the scale of affected users by a post on social network or intensity of emotional and cognitive impact (mcneill and briggs 2014). according to probst, grosswiele et al. (2013), influential users have different influences on different topics and a very influential user is not necessarily influential on all topics. it is indicated in kardara, papadakis et al. (2015) that topic-based influence measures are more effective and functional than the global ones. one of the differences of topic-related influence studies to network structure analysis is that it takes the posts’ (e.g., tweets) content into account. when we consider user influence on topics, no longer the whole network needs to be analysed, which improves the performance of measures. however, there are drawbacks and shortcomings in the topic-based influence studies. in most of the existing works, they have aimed at making influential user detection more effective in retrieving the top n users only. less effort is dedicated in discriminating influential from noninfluential users. also, approaches that use supervised learning (e.g., svm) suffer from their dependency on labelled data, which is extremely expensive to prepare for the immense data of social networks. another considerable issue in these studies is their approach evaluation. this is a difficult task as influence is subjective. more importantly, prediction of user influence is remained as a problem to address in the state-of the-art. topic-based user influence measurement and identification are important challenges and the focuses of this paper. this task is significantly important for different applications such as marketing, election campaigns, or recruiting employees for a company. in this work, we measure topic-based user influence on observed topics in which they have shown their interests by posting in social networks. our approach, called tsim (topic-based social influence measure), incorporates network structure, user generated contents, users history of activities, and network users engagement in user’s activity. our approach represents users with their topic interests and their social influence on each observed topic. in more detail, our contributions are:  we propose a novel topic-based influence measurement approach to integrate the usertopic relationships, topic content information, and social connections between users into the same principled model.  instead of considering user-to-user influence and global user influence, the proposed model considers individuals’ influence and interests in a topic, which gives the capability of predicting one’s influence on a new topic.  finally we have prepared a unique dataset from real-world social networks for testing and evaluating the proposed approach that contains all the social media related metadata. the remainder of this paper is organized as follows. we first discuss existing approaches for topic-based influence analysis in section 2. we then present the background in section 3. next, we define the research problem, and then propose our approach and algorithms in section 4. we describe our dataset and discuss the results in section 5. finally, we conclude the paper in section 6. 2 related work one of the main approaches to study user influence in social networks has been through network structure as well as user’s position and connectivity in the network. the traditional centrality measures such as closeness and betweenness are measured for users, to discover how well connected a user is to the rest of users in the network and whether a user is acting as a hub (romero, galuba et al. 2011). the major adopted algorithms for network structure based influence measurement include pagerank (haveliwala 2002) and hits (kleinberg 1999). numerous works have applied pagerank algorithm variations on social network graph to rank user influence according to the network structure. an example of pagerank algorithm variations is the work by kwak, lee et al. (2010), in which they ranked users by applying australasian journal of information systems hamzehei, jiang, koutra, wong & chen 2017, vol 21, selected papers from ausdm topic-based social influence measurement 3 pagerank on follower/following graph in twitter (along with number of followers and number of retweets). the network structure is relatively static compared to the activities of users in social networks. some studies have included the social network related meta data (in case of twitter, the meta data are retweets, mentions, and likes) (hajian and white 2011). topic-based influence. following the influence studies (overall user influence) on social networks, less studies have shed light on topic-based influence. more recently, topic-based influence studies have combined content of user posts with link-based metrics. haveliwala (haveliwala) proposed a topic-sensitive extension of pagerank to rank query results in regards to the query topics. the idea of topic-sensitive pagerank was later used and adjusted for social networks such as twitter for ranking topic-based user influence. also, topical authorities studied by pal and counts (2011). they proposed a gaussian-based ranking to rank users efficiently. they used probabilistic clustering to filter feature space outliers and showed that mentions and topical signals are more important features in ranking authorities. kong and feng (2011) intended to identify and rank users that are posting quality tweets. they defined a topic-based high quality tweet with the author’s topic-specific influence, topic related author’s behaviour. they applied their proposed metric on graph of following and retweets. xiao, noro et al. (2014) aimed at detecting topic related influential users by looking at hashtag user communities where hashtags are pre-identified from news keywords. they proposed retweetrank and mentionrank as content-based and authority-based influential users. similarly, hu, fang et al. (2013) worked on detecting topical authorities with the assumption that retweeting propagates topical authority. montangero and furini (2015) also measured twitter topic-based user influence where they identify topics by hashtags. although hashtags can reveal the tweet’s topic correctly, over 80% of tweets do not have hashtags. these results are neglecting the majority of tweets and can mislead a topic-based user influence, as 4 out of 5 of her tweets are not considered for measuring her influence. cataldi and aufaure (2014) estimated twitter user influence for topics of conversations based on pagerank. for that purpose they build a topic information exchange graph to take the information diffusion and degree of information shared into account for user influence estimation. they manually considered seven topic categories and later assign each tweet to those categories through an ngram model. however, their approach is unable to identify topics in the lower level of the main categories. for example, if someone is detected as influential in the sports category we do not know which sport the influence belongs to. weng, lim et al. (2010) proposed twitterrank, a pagerank extension, that measures user influence by calculating topical similarities of users and their network connections. for topic identification, they used the unsupervised text categorization technique, lda, by aggregating all tweets of a user into a document. although this approach is presented as topic-sensitive, this approach cannot discriminate the user influence for the topics. sung, moon et al. (2013) proposed another extension of pagerank, and unlike (weng, lim et al. 2010), it does not need predefined topics for topic-based user influence. in (cano, mazumdar et al. 2014), a pagerank-based user influence rank algorithm introduced that the user links have weights based on their topics of interest similarities. in (liu, shen et al. 2014), their topic-based influence framework considers retweet frequency and link strength. the link strength is estimated by poisson regression-based latent variable model on user’s frequency of retweeting each other. welch, schonfeld et al. (2011) found out that topical relevance is better detectable through the retweet link rather than following links. they used two variations of pagerank algorithm to on retweet and following graphs for that purpose. in a recent work by katsimpras, vogiatzis et al. (2015), they proposed a supervised random walk algorithm for topic sensitive user ranking. as it is obvious from the algorithm name, it needs labelled data, which is not very practical in many cases especially with the volume of social networks. it is worth mentioning that similar works exist that are only after the identification of global influencers instead of influencers for specific topics. an example of such works is barbieri, bonchi et al. (2013) where they extended the linear threshold model and independent cascade model to be topic-aware, the topics are still obtained based on the network structure, while totally ignoring the valuable content information. australasian journal of information systems hamzehei, jiang, koutra, wong & chen 2017, vol 21, selected papers from ausdm topic-based social influence measurement 4 3 background next, we give preliminaries for probabilistic topic modelling and pagerank. 3.1 probabilistic topic modelling given a set of documents denoted by d = [d1, …, dq], topic modelling generates a set of t topics denoted by 𝒯 = [t1, ⋯, tj]. each topic is related to a weighted representation over m words denoted by tj = [w1⋯wm], where wj is the weight representing the contribution of word wm to topic tj. probabilistic topic modelling, such as latent dirichlet allocation (lda), represents a low dimensional space of corpus by detecting a set of latent topics. the basic idea of probabilistic topic modelling is having a z hidden variable for each word’s co-occurrence in the collection of documents. z can range among j topics where each topic is a distribution over a fixed vocabulary. given a corpus, a document may contain multiple topics and the words are assumed to be generated by those topics. a probabilistic topic model can be generated over a process as follows (blei, ng et al. 2003): 1. obtain a distribution over topics to generate a document (in lda this distribution is drawn from a dirichlet distribution with a corpus-specific hyperparameter α) 2. then for each word to be generated; a. assign topics by drawing upon the document-specific distribution over topics b. finally, generate a word from distribution of topics over words in dictionary, which means words of each document come from a mixture of topics. we aim to use probabilistic topic modelling to represent items as a set of topics and also detect social network users interest by applying topic modelling on their timelines. 3.2 pagerank pagerank is a webpages ranking algorithm that calculate rank xi for vertex vi based on the rank of other vertices in the graph that point to vertex vi. assume g(v,e) denotes a directed graph, where the set v of vertices consists of i users and users relationships are the edges set e. considering ui as a user equal to vertex vi in the graph g, the directed edge (i, i') exists if user ui is connected to user ui’. the directed vertices of the graph g contained in the asymmetric adjacency matrix l = (li,i'), where l(i,i') = 1 if ui → ui’ and li, i' = 0 otherwise. out-degree dout(i) is the number of users that points to user i. 𝑋𝑖 = ∑ 𝐷𝑜𝑢𝑡(𝑖′)−1𝑋𝑖 (𝑖′,𝑖)∈𝐸 the above equation is a recursive function that gives any vertex points to vertex vi, a fraction of the rank inversely. in pagerank, each out-going link from vi is weighted by 1/oi, thus every node has the same total out-going weights. each node has a total of one vote. pagerank uses an idea that a “good” node should connect to or be pointed to by other “good” nodes. however, instead of mutual reinforcement, it adopts a web surfing model based on a markov process in determining the scores: 𝑥 = 𝐼𝑜𝑝(𝑥) where the iop is an authority that is pointed to by many hubs and the iop operation is defined to be 𝐼𝑜𝑝(. ) = 𝐿𝑇𝐷𝑜𝑢𝑡 −1 ≡ 𝑃𝑇 . this amounts to rescale the adjacency matrix l such that each row is sum-to-one. thus, p = (pi,i') is a stochastic matrix, since ∑vpi, i' = 1, pi, i' ≥ 0. pi, i' represent the probability of a web surfer making a transition from webpage vi to vi’. starting from any webpage vi, a surfer goes to any one of the hyperlinked webpages with equal probability 1/oi. australasian journal of information systems hamzehei, jiang, koutra, wong & chen 2017, vol 21, selected papers from ausdm topic-based social influence measurement 5 at any moment, millions of people are using the social networks. pagerank assumes the users follow the random surfing model in viewing and engaging with the rest of network. they will reach the equilibrium (stationary) distribution under general conditions. if a node has a high probability in the equilibrium distribution that means more nodes will point to that node. therefore, the equilibrium distribution of users in social network is a measure of a node’s importance, which is the authority score in pagerank. the equilibrium distribution x is determined by 𝑃𝑇𝑥 = 𝜆𝑥 and x satisfies ∑kx(k)=1. one can obtain the solution iteratively. note that λ = 1 if the markov process has an equilibrium distribution x. pagerank models two types of random jumps on the internet. (i) link-tracking jump: a user often follows other users in the network by simply clicking on them; this is modelled by ltdout −1. (ii) link-interrupt jump: a user sometimes observes to engage with a user that they are not already connected to each other. pagerank models such link-interrupt jump with a simple uniform distribution (1 − α)/n. the full stochastic matrix of transition probability is pt = iop(.)=αltdout −1 + (1 − α)(1/n)eet where α = 0.8 ∼ 0.9. here e = (1, 1, … ,1)t; thus eet is a matrix of all 1’s (arasu, novak et al. 2002). 4 topic-based social influence measurement 4.1 problem definition assume g(v,e) denotes a social network graph, where users are the vertex set v and users relationships are the edges set of e. assume that users publish a set of texts d = [d1, d2, …, dq], and talk about different topics 𝒯 = [t1, t2, …, tj]. each user text (post) dq holds one or more topics and receives engagement from other users by replying, liking, or re-publishing it. the engagement of other users in a post can reveal the influence of that particular post among its audience. symbol description t a topic fi, j influence of user ui in tj ff(i, j) follower strength influence measure for user i in tj fa(i, j) activity influence measure for user ui in tj fe(i, j) engagement influence measure for user ui in tj fc(i, j) centrality influence of measure for user ui in tj xi rank of user ui calculated through pagerank iop operation of identifying authorities l asymmetric adjacency matrix representing directed edges pi, i' probability of user ui engage with user ui’ dout(i) number of user that points to user ui in graph g nd number of words in document d table 1: key notations we denote a = {aii′} as the n × n matrix which shows the social ties among users in the social network g. for the pair of users i and i′, aii′ ∈ [0, 1] shows the weight of the relationship between users ui and ui′, which we treat as the influence of user i on user i′ (the higher the value of aii′, the higher the corresponding influence). the matrix a is not symmetric, as the influence australasian journal of information systems hamzehei, jiang, koutra, wong & chen 2017, vol 21, selected papers from ausdm topic-based social influence measurement 6 of user i on user i′ is not necessarily equal to influence of user i′ on user i. we also assume that user post is visible to all users in g. figure 1: tsim work flow. quantifying the topic-based influence of each user based on social ties and other users’ engagement in social networks, we can identify the influence of user i on topic j, represented as fij. then we have matrix f = [fij]i × j that represents influence of all the users in all identified topics. 4.2 our approach to measure social influence on an observed topic in a social network, we propose tsim, topicbased social influence measure, which measures topic-based individuals influence in social networks. in a nutshell, our model contains two main phases:  identifying topics on social networks according to users generated contents, and  measuring individuals influence for the detected topics. figure 1 shows our approach’s work flow. first, we collect user-generated content from the social media. for each user-generated content, we collect related information such as list of users that have re-published the content (ri) and list of users that have engaged in that content (mi) as well as metadata connected to the content. we identify the topics by applying probabilistic topic modelling, lda, on all the user-generated text. each topic contains a set of posts with all their related information and metadata, such as; content, replies, and republishing. for each tuple of (useri, topicj), we measure the influence of user ui on topic tj as fij * which comprises of four measures ff, fa, fe, fc. the details of influence measurement algorithm are shown in figure 2 and section 4.3. the measure fij * identifies user influence for the identified topics and users can be ranked according to their fij * score for each topic. 4.3 influence measurement we define social influence in a social network as importance of a user in the social network graph, user’s activities, and involvement of others in the user’s posts. social influence can be analysed through different modalities network structure and user’s position in the network, scale of a user’s post diffusion in the network, a user’s activities and engagement in the social network, and message content that a user broadcast in the network (embar, bhattacharya et al. 2015). from the network structure, we identify influence related attributes, such as user friends and centrality of user in the social network. from the content of broadcasted text, we can identify australasian journal of information systems hamzehei, jiang, koutra, wong & chen 2017, vol 21, selected papers from ausdm topic-based social influence measurement 7 one or more topics, thus, the influence of that user on different aspects. for instance, in twitter, a post can contain user mentions, receive replies, and get retweeted by other users. all this information can reveal social influence of a user. let denote dt as the set of collected texts related to topic tj from the set of topics 𝒯. each text di contains a set of attributes as (ui, ci, ri, mi, fi) where ui is the author of the text, ci is the text, ri is the list of users republished the text, mi is the list of mentions for that text, and fi is the number of followers of the text author. we define the following dimensions for measuring social influence of a user on a topic as following: algorithm: influence measurement input: list of topics, collection of user posts for each topic, interaction graphs, number of friends of each user. output: matrix of user influence on each topic. 1. for topic in topics do 2. for user in users do 3. 𝐹𝑓(𝑖) ← # 𝑓𝑟𝑖𝑒𝑛𝑑𝑠 4. 𝐹𝑎(𝑖, 𝑗) ← ∑ 𝛿(𝑑𝑖)𝑑𝑖∈𝐷𝑗 5. 𝐹𝑒(𝑖, 𝑗) ← ∑ (𝛿(𝑅𝑖) + 𝛿(𝑀𝑖𝑑𝑖∈𝐷𝑗 )) 6. 𝐹𝑐(𝑖, 𝑗) ← 𝑃𝑅(𝑢𝑖, 𝐺(𝐷𝑗)) 7. 𝐹𝑖𝑗 ∗ ← 𝑎𝑔𝑔𝑟𝑒𝑔𝑎𝑡𝑖𝑜𝑛 𝑜𝑓 𝐹𝑓(𝑖), 𝐹𝑎(𝑖, 𝑗) , 𝐹𝑒(𝑖, 𝑗), 𝐹𝑐(𝑖, 𝑗) 8. return matrix of user influence on topic figure 2:topic-based influence measurement algorithm follower scale: this measure depicts the number of friends a user has in the network. this value is constant across all topics for a user and is independent of topics. it shows the strength of social ties of a user. although the number of social connections can be an indicative of influence, it does not carry information on any specific topic. the following influence measures are more topic-specific. topic activity: this measure captures topic-related activities of a user. fa(i, j) denotes influence of user ui in terms of activities related to topic tj and we define it as: 𝐹𝑎(𝑖, 𝑗) = ∑ 𝛿(𝑑𝑖) 𝑑𝑖∈𝐷𝑡 where δ(di) is 1 if di belongs to texts set for topic t and is 0 otherwise. it intuitively measures the volume of topic tj-related activities of user ui. australasian journal of information systems hamzehei, jiang, koutra, wong & chen 2017, vol 21, selected papers from ausdm topic-based social influence measurement 8 figure 3: retweet and mention graph topic-based attractiveness: this measure indicates how other users are attracted to useri’s post. it takes other users’ feedback on user ui’s activities into account. we define it as 𝐹𝑒(𝑖, 𝑗) ← ∑ (𝛿(𝑅𝑖) + 𝛿(𝑀𝑖) 𝑑𝑖∈𝐷𝑡 where δ(ri) is the number of times di is republished by other users and δ(mi) is the number of mentions or replies of di. network centrality: centrality of a user is another indicator of her influence in a social network. pagerank was introduced first for ranking webpages for search engines, and can be used here to calculate topic-specific centrality of users in the social graph. figure 3 shows an interaction graph of users on a post generated by user 1. to that end, we perform pagerank on the induced graph of interactions on a specific topic tj. the interaction graph is a better representative of the topical relevance of two users rather than friendship graph (welch, schonfeld et al. 2011). we denote it as: 𝐹𝑐(𝑖, 𝑗) ← 𝑃𝑅(𝑢𝑖, 𝐺(𝐷𝑡)) where g(dt) is a graph corresponding to users over documents set dt for topic t. pr(ui, g(dt)) indicates the pagerank score of user ui in the graph g(dt). in this work, we reconstruct the interaction graph, (e.g., retweet and mention graphs from twitter), to measure topic specific centrality of users by pagerank. aggregating influence scores: the four influence measures described above ff, fa, fe, fc will be aggregated to form a single influence score f* for user ui in topic tj. for the first attempt, we averaged the measures, which gives every measure the same share in the overall influence score. for the future works, we investigate other methods for aggregating the measures. 5 results and experiments in this section, we discuss the details of conducted experiments. it includes the data and the influence measurement performed by our proposed method. user topic1 topic2 topic3 topic4 vnfrombucharest [0, 0, 0, 0] [0, 0, 0, 0] [0.1, 0.02, 0.51, 0.01] [0, 0, 0, 0] charliedatamine [0.12, 0.01, 0.34, 0.2] [0, 0, 0, 0] [0.1, 0.02, 0.51, 0.16] [0.28, 0.2, 0.38, 0.16] sepehr125 [0, 0, 0, 0] [0, 0, 0, 0] [0, 0, 0, 0] [0, 0, 0, 0] sdatamanagement [0.12, 0.02, 0.35, 0.7] [0.05, 0.04, 0.15, 0.7] [0.6, 0.04, 0.51, 0.67] [0, 0, 0, 0] yisongyue [0, 0, 0, 0] [0, 0, 0, 0] [0, 0, 0, 0] [0.07, 0.05, 0.27, 0.08] table 2: a sample from the influence matrix before aggregating the 4 influence measures of user i in topic t. australasian journal of information systems hamzehei, jiang, koutra, wong & chen 2017, vol 21, selected papers from ausdm topic-based social influence measurement 9 user topic1 topic2 topic3 topic4 vnfrombucharest 0 0 0.159 0 charliedatamine 0.157 0 0.197 0.254 sepehr125 0 0 0 0 sdatamanagement 0.29 0.227 0.457 0 yisongyue 0 0 0 0.117 table 3: a sample from the influence matrix after aggregating the 4 influence measures of user i in topic t. 5.1 dataset to validate our proposed method, we collected a unique dataset from twitter using the twitter search api. we targeted the machine learning domain and identified core 500 users that have mentioned machine learning as a keyword in their profile description. to choose the users, we selected a set of machine learning users as seeds and crawled among their friends and friends of friends for other machine leaning-related users. for the prepared list of users, we gathered their timeline tweets which for most of the users covers their tweets for the last 5 years. for each tweet, we also, collected the related meta-data such as the list of users who have replied to each tweet (mention list) and the list of users who have retweeted each tweet (retweet list). the final dataset contains 101,363 tweets with their related metadata, mention lists, and retweet lists. the network that is built on retweet list contains 301,870 nodes. 5.2 evaluation our experiments contain a main task of user influence measurement on the identified topics from the tweet corpus. we evaluate the measured user influence through expert opinion and user citations on the topics that the user has published in scientific conferences and journals. we collected publications through google scholar for validation. the community of study is intentionally chosen as researchers then we are able to cross-validate our results through the users influence in research community measured by topics of their publications and citations. 5.2.1 topic-based influence measurement next, we proceed with identifying topics from the collection of all tweets and then measuring influence. the number of topics generated by lda can affect the quality of features that will be used in tsim. we determined a number of topics through cross validation that we could receive higher recall in user influence prediction. in our proposed approach, we perform probabilistic topic modelling for identifying the topics in the tweets dataset. figure 4: number of users belonging to topics australasian journal of information systems hamzehei, jiang, koutra, wong & chen 2017, vol 21, selected papers from ausdm topic-based social influence measurement 10 figure 5: user's influence measure for three sample topics the user tweets gathered from their timelines, belong to the identified topics with a probability. we set the probability threshold to 0.1 to consider whether a tweet belongs to a topic. each tweet is mapped to at least one topic. now that for each topic we have a collection of related tweets with their mention and retweet lists, we can measure user influence for them. in section 4.1, we defined influence based on 4 measures; follower strength, activity, engagement, and network centrality. follower strength will be taken from the number of users follow the user ui on twitter. activity represents the number of tweets user ui has in topic tj. engagement is the sum of number of mentions and retweets for all of user ui’s tweets in topic tj. for measuring network centrality, we build the retweet graph for each topic separately from the corresponding retweet list and measure centrality of that user node through pagerank algorithm. table 2 shows a small sample of the 4 calculated measures of topic-user influence. the zero scores mean that user ui did not have any tweet for that corresponding topic. the non-zero scores are normalized to lie in the range of [0,1] and higher score means higher influence for that topic. the measured influence scores are aggregated and a sample of aggregated scores is shown in table 3. average percentage of users active in topics 13.8% average of influence measures of active users in topics 0.125 average of influence measures of all users in each topic 0.015 percentage of users have influence of greater than 0.1 in topics 6% percentage of users have influence of greater than 0.2 in topics 1.4% table 4: statistical information on users, topics, and influence measures users are not active on all topics. some topics can be more popular with more active users and some topics can be less popular with less number of active users in social networks. we observed that the top 5% of most popular topic have 23% of active users in average. the percentage of active users for least 5% popular topics was just 4%. figure 4 shows the distribution of users in the topics sorted based on the number of active users in the topics. a user is active in a topic if has engaged in a topic by posting about it. moreover, the influence of users in each topic is not equally distributed. on average, topics have 13.8% of active users. it means 86.2% of users have zero influence on topics. in average 6% of users have influence of greater than 0.1 and just 1.4% of users have influence of greater than 0.2. this indicates that our proposed influence measure is capable of distinguishing very influential users from other for topics. figure 5 shows influence measure for three selected topics. in this figure, topic a is selected from the top 5% popular topics. topic b selected from topics with number of active users close to the average and topic c is from the least 5% popular topics. the figure shows how influence measure exponentially decreases from high influential users to non-influential users. the statistics provided are presented in table 4. australasian journal of information systems hamzehei, jiang, koutra, wong & chen 2017, vol 21, selected papers from ausdm topic-based social influence measurement 11 screen name f screen name f screen name f screen name f kdnuggets 0.63 randal_olson 0.62 analyticbridge 0.70 analyticbridge 0.57 analyticbridge 0.49 analyticbridge 0.55 randal_olson 0.49 ml_toparticles 0.55 deeplearning4j 0.33 jmgomez 0.53 datasciencectrl 0.41 datasciencectrl 0.37 kirkdborne 0.31 ibmbigdata 0.51 bernardmarr 0.35 ibmbigdata 0.24 datasciencectrl 0.31 kdnuggets 0.49 eddelbuettel 0.34 kdnuggets 0.21 table 5: a sample of topics and their 5 top influencers measured by our proposed topicbased influence measurement system. screen name f screen name f screen name f screen name f jmgomez 0.51 kdnuggets 0.93 analyticbridge 0.58 kdnuggets 0.62 randal_olson 0.48 kirkdborne 0.43 kdnuggets 0.56 analyticbridge 0.54 analyticbridge 0.45 smolix 0.34 mjcavaretta 0.47 randal_olson 0.47 stanfordnlp 0.43 mapr 0.32 charliedatamine 0.44 datasciencectrl 0.36 bigdata 0.36 mjcavaretta 0.30 jure 0.35 paulblaser 0.36 table 6: table 4 continueda sample of topics and their 5 top influencers measured by our proposed topic-based influence measurement system. screen name f screen name f screen name f screen name f xamat 0.70 analyticbridge 0.69 analyticbridge 0.58 analyticbridge 0.90 analyticbridge 0.54 mapr 0.60 randal_olson 0.55 datasciencectrl 0.45 kdnuggets 0.40 bernardmarr 0.58 ibmbigdata 0.36 hmason 0.42 jmgomez 0.36 odbmsorg 0.56 oracleanalytics 0.32 kirkdborne 0.35 kirkdborne 0.32 infochimps 0.53 marklogic 0.30 paulblaser 0.31 table 7: table 5 continueda sample of topics and their 5 top influencers measured by our proposed topic-based influence measurement system. tables 4, 5, and 6 show the top 5 influencers for selected topics. the sample of topics presented in the tables contain machine learning topics such as neural networks, deep learning, big data, social networks, text mining, nlp, database, visualization, and more specific topics such as hadoop. for the task of validation of the influence results, there is no standard method in the literature to validate the algorithm output. one of the reasons we have chosen the machine learning and data science community on twitter as our community of study was the wide availability of experts in the domain that allows us to verify the identified influential users through our algorithm. we manually verify the top topic-based influential users through expert opinions, their twitter, and google scholar accounts. for example, for the topic nlp, stanford nlp group appeared in the top 5 influential accounts on twitter. for “recommender systems” topic, xavier amatriain, who is known for his works on recommender systems, received a high influence score. also, for the topic “neural networks”, alex smola was in the top 5 influencers who have extensively published on neural network topic. in the topic “social networks”, jure leskovec, who is well-known in the social networks community, was among the top influencers. 5.3 implications and applications this section describes the real-world implication and applications of our model. identifying topic-based influential users is similar to the problem of finding experts and authorities. spotting the elite group of users for topics can improve available systems such as search engines. the query result for both contents and users can be returned and ranked using the score provided by our system. one of the main applications of this work is in marketing. marketing campaigns can be implemented through the influential users in the related topic to have more productive and cost effective campaign. influential users act as hubs in the network and have a central position in the network in terms of information diffusion, also they attract and engage more users into their conversations. our model is able to detect the new and surprising topics. this capability gives the strength to our model that works in real world and detects new topics and related influential users. as a australasian journal of information systems hamzehei, jiang, koutra, wong & chen 2017, vol 21, selected papers from ausdm topic-based social influence measurement 12 result, there wouldn’t be a need for manually defining the topics and consequently the recent and new topic would not be missed. tsim, also can be applied to detect topics at what period get viral and who are influential in those topic in different period of time. 6 conclusions in this study, we have presented an approach, tsim, to measure topic-based user influence in social networks. we have identified topics from user posts on social networks, and measured each user’s influence on each topic. tsim is then used to calculate user influence for the observed topics. our main contributions include:  the proposal of a effective method to measure topic-based influence for social network users  opening a new discussion for user influence prediction in social networks that has not been explored in the literature. finally, we have tested tsim using a unique dataset that we collected from twitter, which we are making it available online. in future work, we are interested to measure topic-based user influence over time, and study how influence changes over time. prediction of user influence on unobserved topics is also currently under our investigation. we will also investigate other methods to combine influence measures. references arasu, a., j. novak, a. tomkins and j. tomlin (2002). pagerank computation and the structure of the web: experiments and algorithms. proceedings of the eleventh international world wide web conference, poster track. barbieri, n., f. bonchi and g. manco (2013). "topic-aware social influence propagation models." knowledge and information systems 37(3): 555-584. blei, d. m., a. y. ng and m. i. jordan (2003). "latent dirichlet allocation." the journal of machine learning research 3: 993-1022. cano, a. e., s. mazumdar and f. ciravegna (2014). "social influence analysis in microblogging platforms–a topic-sensitive based approach." semantic web 5(5): 357-372. cataldi, m. and m.-a. aufaure (2014). "the 10 million follower fallacy: audience size does not prove domain-influence on twitter." knowledge and information systems 44(3): 559580. eccleston, d. and l. griseri (2008). "how does web 2.0 stretch traditional influencing patterns." international journal of market research 50(5): 591-161. eirinaki, m., s. p. s. monga and s. sundaram (2012). "identification of influential social networkers." international journal of web based communities 8(2): 136-158. embar, v. r., i. bhattacharya, v. pandit and r. vaculin (2015). online topic-based social influence analysis for the wimbledon championships. proceedings of the 21th acm sigkdd international conference on knowledge discovery and data mining. sydney, nsw, australia, acm: 1759-1768. hajian, b. and t. white (2011). modelling influence in a social network: metrics and evaluation. privacy, security, risk and trust (passat) 2011 ieee third inernational conference on social computing (socialcom), 2011 ieee third international conference on, ieee. hamzehei, a., s. jiang, d. koutra, r. wong and f. chen (2016). tsim: topic-based social influence measurement for social networks. proceedings of the 14th australasian data mining conference. canberra, australia. australasian journal of information systems hamzehei, jiang, koutra, wong & chen 2017, vol 21, selected papers from ausdm topic-based social influence measurement 13 haveliwala, t. h. (2002). topic-sensitive pagerank. proceedings of the 11th international conference on world wide web. honolulu, hawaii, usa, acm: 517-526. hu, j., y. fang and a. godavarthy (2013). topical authority propagation on microblogs. proceedings of the 22nd acm international conference on conference on information & knowledge management, acm. jabeur, l. b., l. tamine and m. boughanem (2012). active microbloggers: identifying influencers, leaders and discussers in microblogging networks. string processing and information retrieval, springer. jin, x. and y. wang (2013). "research on social network structure and public opinions dissemination of micro-blog based on complex network analysis." journal of networks 8(7): 1543-1550. kardara, m., g. papadakis, a. papaoikonomou, k. tserpes and t. varvarigou (2015). "largescale evaluation framework for local influence theories in twitter." information processing & management 51(1): 226-252. katsimpras, g., d. vogiatzis and g. paliouras (2015). determining influential users with supervised random walks. proceedings of the 24th international conference on world wide web. florence, italy, acm: 787-792. katz, e. (1957). "the two-step flow of communication: an up-to-date report on an hypothesis." public opinion quarterly 21(1): 61-78. katz, e. and p. f. lazarsfeld (1955). personal influence, the part played by people in the flow of mass communications, transaction publishers. kleinberg, j. m. (1999). "authoritative sources in a hyperlinked environment." j. acm 46(5): 604-632. kong, s. and l. feng (2011). a tweet-centric approach for topic-specific author ranking in micro-blog. advanced data mining and applications, springer: 138-151. kwak, h., c. lee, h. park and s. moon (2010). what is twitter, a social network or a news media? proceedings of the 19th international conference on world wide web, acm. libai, b., r. bolton, m. s. bügel, k. de ruyter, o. götz, h. risselada and a. t. stephen (2010). "customer-to-customer interactions: broadening the scope of word of mouth research." journal of service research 13(3): 267-282. liu, x., h. shen, f. ma and w. liang (2014). topical influential user analysis with relationship strength estimation in twitter. ieee international conference on data mining workshop (icdmw), 2014. mcneill, a. r. and p. briggs (2014). understanding twitter influence in the health domain: a social-psychological contribution. proceedings of the 23rd international conference on world wide web. seoul, korea, acm: 673-678. montangero, m. and m. furini (2015). trank: ranking twitter users according to specific topics. 12th annual ieee consumer communications and networking conference (ccnc), 2015, ieee. pal, a. and s. counts (2011). identifying topical authorities in microblogs. proceedings of the fourth acm international conference on web search and data mining. hong kong, china, acm: 45-54. probst, f., l. grosswiele and r. pfleger (2013). "who will lead and who will follow: identifying influential users in online social networks." business & information systems engineering 5(3): 179-193. riquelme, f. (2015). "measuring user influence on twitter: a survey." arxiv preprint arxiv:1508.07951. australasian journal of information systems hamzehei, jiang, koutra, wong & chen 2017, vol 21, selected papers from ausdm topic-based social influence measurement 14 romero, d. m., w. galuba, s. asur and b. a. huberman (2011). influence and passivity in social media. machine learning and knowledge discovery in databases: european conference, ecml pkdd 2011, athens, greece, september 5-9, 2011, proceedings, part iii. d. gunopulos, t. hofmann, d. malerba and m. vazirgiannis. berlin, heidelberg, springer berlin heidelberg: 18-33. sung, j., s. moon and j.-g. lee (2013). the influence in twitter: are they really influenced? behavior and social computing, springer: 95-105. welch, m. j., u. schonfeld, d. he and j. cho (2011). topical semantics of twitter links. proceedings of the fourth acm international conference on web search and data mining. hong kong, china, acm: 327-336. weng, j., e.-p. lim, j. jiang and q. he (2010). twitterrank: finding topic-sensitive influential twitterers. proceedings of the third acm international conference on web search and data mining. new york, new york, usa, acm: 261-270. xiao, f., t. noro and t. tokuda (2014). "finding news-topic oriented influential twitter users based on topic related hashtag community detection." j. web eng. 13(5-6): 405-429. copyright: © 2017 hamzehei, jiang, koutra, wong & chen. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems ashraf, grunfeld & quazi 2015, vol. 19, research on indigenous ict impact of ict on quality of life impact of ict usage on indigenous peoples’ quality of life: evidence from an asian developing country md mahfuz ashraf university of dhaka mashraf@univdhaka.edu helena grunfeld victoria institute of strategic economic studies victoria university helena.grunfeld@live.vu.edu.au ali quazi university of canberra ali.quazi@canberra.edu.au abstract indigenous communities across the world have been suffering disadvantages in several domains, e.g. erosion of land rights, language and other cultural aspects, while at the same time being discriminated against when prepared to integrate into the dominant cultures. it has been argued in the literature that information communication technologies (icts) have the potential of contributing to addressing some of these disadvantages – both in terms of rebuilding what has been eroded and facilitating integration into non-indigenous societies. in trying to understand how icts can be useful for these processes, it is important to do so from a conceptual framework that encompasses the multi-dimensionality of the issues faced by indigenous communities. the conceptual frameworks frequently used in the ict literature tend to focus on adoption, use and diffusion of technologies rather than how the use of icts affects the livelihoods of the users, which is the focus of this paper. the conceptual framework is informed by the capability approach (ca), in particular by the five freedoms identified in the seminal work of amartya sen (2001), “development as freedom” (daf). data were collected from a purposive sample in an indigenous community in bangladesh, using a qualitative method to map how icts had affected the lives of these community members the findings suggest that the participants perceived that icts had made positive contributions, particularly the benefits they gained from learning how to use computers in the domains that are relevant from the perspective of the five freedoms espoused in daf. the findings reported in this paper are useful for policy formulation in bangladesh. as the study is contextualised in a transitional economy setting and can therefore not be generalised, but we believe that the conceptual framework has much to offer future research designed to understand how icts can improve the livelihoods of indigenous individuals and communities. keywords: information communication technologies (icts); indigenous people; amarty sen’s five freedoms; qualitative approach; bangladesh. 1 introduction indigenous people have not only faced serious discrimination in terms of their basic rights and erosion of their ancestral lands, languages, cultures, and forms of governance, but also in terms of access to basic social services (gigler 2009, hunter 2005). today, information communication technologies (icts) offer unprecedented opportunities to enhance educational systems, improve policy formulation and execution, and expand the range of opportunities for income generation and social change among socially disadvantaged groups e.g. the poor and people living in remote locations are often represented in both of these groups (randoll 2011, mayerhofer & taylor 2010, randoll, fleissner, stevenson & gardner 2013, thapa and saebo 2014). their experiences of using icts are often different than in other communities (radoll 2010, 2011). one of the reasons for this is that knowledge of traditional culture is an important 1 australasian journal of information systems ashraf, grunfeld & quazi 2015, vol. 19, research on indigenous ict impact of ict on quality of life asset to their social capital and constitutes a key livelihood resource (lwoga, ngulube, & stilwell, 2010). so ict applications specific to indigenous communities are associated with storage, management, retrieval, dissemination and preservation of indigenous knowledge which helps to maintain a sense of community (daly & haahr 2007, taylor 2012). another common ict application for indigenous communities is interaction between indigenous communities on a global level (lima & brito 2012, raitt & roy 2003, stavenhagen 2002). a major obstacle to the deployment of icts in remote indigenous communities has been the costs of establishing and maintaining reliable infrastructure (acma 2008). low literacy and numeracy rates often limit the ability to use other than simple forms of icts (salazar 2007). while there is much promise that ict projects will contribute to improved livelihoods of indigenous people (table 1), without rigorous research on outcomes and impacts of initiatives aimed at achieving this, not much is known of the extent to which different community members have actually benefitted from icts and/or if there has been any adverse outcomes (corbett, singleton & muir, 2009). such information is important for policy formulation and for the design of future initiatives. some previous frameworks have adopted indicators for measuring the impact of ict on socially disadvantaged communities (thapa & sæbø 2014) from other contexts. several of those could have been relevant for this study, e.g. the sustainable livelihoods framework and using the millennium development goals as a framework. the suitability of the capability approach (ca) for exploring impacts of icts on indigenous communities lies in the multidimensional nature of this framework and it focuses on human development and freedoms. in the ca development and freedom are closely associated in that development is defined as capability expansion that enables individuals to choose the lives they value and have reason to value (sen 2001, 1989). through the application of conversion factors, these capabilities are converted into functioning, which reflect what people are actually doing with their capabilities when applying these concepts to icts, physical access in combination with requisite skills provides individuals with the capability of using icts. the use of icts is thus a functioning, which not all who have access to icts would necessary avail themselves of. access to icts can potentially expand the set of functionings through relevant information being easily accessible, together with the greater ease with which it becomes possible to communicate. there has been a growing interest in using the ca in ict research (e.g. andersson, gro¨nlund & wicander 2012, hatakka & lagsten 2012, johri & pal 2012, thapa & sæbø 2014). there are two basic streams in the literature operationalising sens’s theory: understanding market power/structure and understanding or measuring the five freedoms. despite the ca’s methodological difficulties in operationalising the ca, there have been many attempts to do so, using both qualitative and quantitative methodologies. for instance, alampay (2006) used a quantitative approach in evaluating ict access and use in the philippines, wresch and fraser (2012) quantified only one aspect of freedom (economic freedom) in their research of ecommerce in 23 caribbean companies and lunat (2009) interpreted five freedoms through the lens of structuration theory (st) in the university sector in palestine. according to zheng and walsham 2008 (cited in hatakka & lagsten 2012, page 26) ‘ even though sen’s theories can be abstract and sometimes hard to apply on empirical data, the key concepts can be used as sensitising devices in analysing empirical cases’. this is what is done in the research reported in this paper. the purpose of the research was to explore whether and the extent to which an ict project contributed to any change in the functionings in an indigenous community in a rural area of bangladesh. the functionings are assessed within the framework of sen’s (2001) five freedoms, which were operationalised in this research. to this end, our research question is: can sen’s five indicators of freedom be used in evaluating impact of ict interventions on indigenous community/people development in rural areas of bangladesh? 2 australasian journal of information systems ashraf, grunfeld & quazi 2015, vol. 19, research on indigenous ict impact of ict on quality of life author (year) objective research design study area/level key findings radoll, p. j. (2011) to examine the factors and develop the theoretical framework that affect adoption of icts. grounded theory ict adoption by indigenous community in australia the practice of using icts in everyday activities outside the indigenous field creates new practices where individuals can take those new practices into the indigenous field and adopt icts in the households. taylor, a. (2012) to investing how development of indigenous people is facilitated through using mobile phone. in-depth interview mobile internet & indigenous groups in australia. • access to new social, business and transnational networks via internet based technologies co-facilitates opportunities for other forms of mobilisation to be harnessed by indigenous people • identified the baseline indicators about the extent of ict ownership and its use in remote indigenous communities. amato, s. (2013) to provide a structural overview about learning approaches of indigenous in south east asian region interview methods (discussion) ict & indigenous population in south-east asian. • the development interventions contribute to the uses of mass communication tools for indigenous residents. • the engagement of local governments and indigenous groups to ict initiatives has worked to the extent that their involvement has gradually been chosen for poverty reduction. watson, s. (2013) to explore how higher education for australian indigenous is achieved by using ict. literature review ict & indigenous people in australia. • found the possibilities for increased educational outcomes with the uptake of digital technology in indigenous learning contexts. • found a good fit between the cultural characteristics of indigenous people, and the connectivity enabled by digital learning environments. johnston, a. l. k., (2010) to evaluate the effectiveness of aboriginal programs through the use of digital technology. theoretical framework technology and aboriginal people at the global level. • technological approach enhances the use of traditional knowledge to convey the importance of evaluation. • a visual tool helps convey teaching about how to self-evaluate the program’s as well as the individual’s concerns. suresh, s., &nath, l. (2014) to build the concept of telemedicine system a crosssectional interviewing methods ict & health service for indigenous population in india. • there is tremendous scope for public private partnership where government centres are already available and private organization can use its expertise. • telemedicine can enable experts to make more timely adjustments to healthcare service, which reduces the number of hospitalizations and hospital visits, thereby creating new horizons. lima, a. p. d., & brito, c. (2012) to analyse the tribal dimension of groups of individuals who organize themselves in communities based on ict use of focus groups in interacting and getting acquainted with community people. ict & tribal community in portugal. • online consumers are more active, participatory, resistant, activist, socially engaged than members of mobile communications communities. • the value of the connection involving members can overcome geographic and demographic barriers, based on a shared passion and on the significance and influence of the opinions exchanged. • the main practical contributions are exploration of emotions, passions, feelings, creation of and encouragement for the use of media interaction and creation of an identity. table 1: summary of icts adoption and implementation in indigenous community 3 australasian journal of information systems ashraf, grunfeld & quazi 2015, vol. 19, research on indigenous ict impact of ict on quality of life 2 literature & operationalisation of five freedoms conceptually, the five freedoms are informed by the ca, which has evolved since the1970s as a collective effort by scholars from different disciplines (e.g. alkire, 2005; comin, 2001; corbridge, 2002; gasper, 1997; nussbaum, 2000, 2006; robeyns, 2001, 2005; stewart, 2005; stewart & deneulin, 2002). the ‘expansion of freedom … both as the primary end and as the principal means of development’ (sen, 2001,:xii) is central to the ca, according to which development is an extension of freedom. freedoms are the basic building blocks for development, together with ‘the expansion of “capabilities” of persons to lead the kinds of lives they value and have reason to value’ (sen, 2001, p.18). there is a mutual relationship between freedoms and capabilities in that certain capabilities are necessary for the exercise of freedom, which in turn facilitates the achievement of capabilities. another way of looking at capabilities is as freedoms to achieve functionings, which describe what a person is actually doing or achieving with his or her capabilities. the ability to convert capabilities into functionings is influenced by personal and external, e.g. institutional factors. in terms of icts, this means that having access and skills to use these technologies represent capabilities, whereas the actual use of them, e.g. to tweet, is a functioning. there can be several impediments preventing capabilities from being converted to functionings. in the case of the tweets, lack of political freedom to express certain views could be one barrier. when moving from the conceptual to the practical domain, the ca becomes important for policy formulation, as sen (1982) suggested that the right to capabilities should form part of a society’s goal. the process of deciding on important capabilities and their translation into functionings should be of a participatory and deliberative nature, with input from a vast array of stakeholder, including individuals and communities. the versatility of the ca is illustrated by its application in diverse fields, ranging from definition by children of their capabilities in an endeavour to understand appropriate dimensions of children’s well-being (biggeri, et al. 2006) to gain insights into how to resolve a river dispute between different indian states (anand, 2007) and an analysis of poverty alleviation programmes in new zealand and samoa (schischka, dalziel, & saunders, 2008). there is a vast philosophical discourse on social justice and human rights from a ca perspective (e.g. nussbaum, 2003). in applying the ca, the focus is often on the expansion of freedoms, which often requires the removal of “unfreedoms” in the form of various types of deprivations that can occur in different domains. access to ict can directly and indirectly influence the various dimensions of freedoms, e.g. by increasing the awareness of people about their rights and entitlements to these freedoms and give guidance about ways in which they can be achieved. one measure of the success of interventions based on icts is therefore the extent to which they have effectively enhanced freedoms. the five dimensions of freedoms necessary for individuals and communities to flourish identified by sen (2001) are: political freedoms, economic freedoms, social opportunities, transparency guarantee and protective security. a) political freedoms political freedoms represent the opportunities enabling people to participate in deciding principles for governance and associated structures, including the freedom of expression, freedom to scrutinise and criticise authorities, the right to elect and be elected. these freedoms and rights require uncensored media. b) economic freedoms economic freedoms include rights to utilise economic resources for the purpose of consumption, production, and exchange. these extend to rights to employment, to participation in markets,, access to credit and other financial instruments as well as a proper share in national income and growth. 4 australasian journal of information systems ashraf, grunfeld & quazi 2015, vol. 19, research on indigenous ict impact of ict on quality of life c) social opportunities social opportunities cover the provisions that societies offer for education, health care and other basic services. these opportunities foster the substantial freedoms of individuals to lead better lives and are also important for the achievement of political and economic freedoms. d) transparency guarantees transparency guarantees comprise practices required to prevent corruption, financial mismanagement and other underhand dealings and include disclosure of information to all stakeholders and trust between parties. e) protective security protective security are arrangements aimed at reducing abject misery among those excluded from market opportunities and refers to social safety nets in the form of various transfer payments such as unemployment benefits, old age benefits, facilities for people with disabilities and statutory income supplements. 3 research methodology this research follows an interpretive approach through using qualitative technique that provides researchers an opportunity to uncover the information from the deep of minds of the respondents using in-depth interviews, observation and focus group discussion. an interpretive research lends itself to investigation of complex human processes within their social context (evert, 2003; myers & avison, 2002; walsham, 2002). the aim of the data collection was to gather information to answer the research question. a flexible research instrument was designed to enable us to ask questions that could capture the perspectives of participants and respondents within their settings (evert, 2003; taylor & bogdan, 1984). we applied a purposive sampling procedure to select the ict project in an indigenous dominated village. criteria included the length of time an initiative had been in operation (minimum 5 years) and convenience. respondents included the operator/trainer (in-charge of operations), users, government officials, and other villagers who were aware of the ict project. the sample size was not pre-determined and data were gathered from the above categories and from local secondary sources (official records, reports and statistics from the local public offices) during the one month stay in the village. together these sources formed a rich data set. this research site for this study is from one of the grameen phone community information centres (gpcic), a shared ict access facility where participants can access a wide range of ict services, e.g. internet, voice communication, videoconferencing, and locally relevant and customised information services on topics such as agriculture, education, health, legal, environment and politics. the gpcic that is the subject of this study is located in the chittagong hill tracts (chts), which is mainly inhabited by the indigenous peoples of bangladesh. about one third of the tribal population live in this region. the different ethnicities of chts have their own traditional social systems, practices, customs, languages, literature, heritage, religious practices, costumes, food habits and festivals (beod 2014). data were collected from a small village, ruma, which has a population of approximately 80 households. it is located in the bandarban district, which has a population of 19,000 and an area of 492.1 square km (bbs 2013). the literacy rate of the region is 27.8% (7+years) and the main livelihood of the area is handicraft production (bbs 2013). a flexible research instrument was designed to enable us to ask questions that could capture the perspectives of participants and respondents within their settings. in total 35 participants have been interviewed (20 male and 15 female). the respondents were asked unstructured questions about the use, challenges, outcome and opportunities of ict on development in ways that encouraged them to tell their stories. questions included but not limited to: 1. how do icts benefit you? 2. how do icts help create more employment opportunity? 5 australasian journal of information systems ashraf, grunfeld & quazi 2015, vol. 19, research on indigenous ict impact of ict on quality of life 3. how do icts help you in getting socio-cultural-political support? 4. how do icts help in demonstrating creativity? during the interview sessions, the participants shared their experiences and attitudes relating to icts. the interviews were audio recorded and then transcribed. some important notes were maintained in writing by the researchers during the interviews. the stories were later extracted from the transcripts. the respondents were given a brief on the research purpose and asked whether they want to participate in the study and their rights to privacy. data analysis is presented into, “stories” focusing on how the use of icts has brought changes in the lives of indigenous people across their community. following stake (1995), we avoid ‘generalisations’ and give a clear description of the context along with the interpretation in our case study before introducing the theme statements for each study site, thereby facilitating ‘particularisation’, as advocated by stakes (1995). in this paper we describe particular case studies and interpret the participants’ experiences in context; so the focus is on the uniqueness of each case. taking this approach to our research produced unique knowledge within a particular context. the quality of research was enhanced through the following measures: • triangulation through the various data-collection techniques and data sources for our research, which increased the data validity and reliability and reduced the risk of bias; • a wide spectrum of perspectives was obtained by in-depth interviews, focus group discussions, observations and document reviews involving different participating groups. this enabled us to obtain interpretations from different participants, as proposed by benbasat, goldstein and mead (1987), cited in myers & avison (2002); • we followed klein and myers (1999) as a guide to principles for conducting and evaluating interpretive field study research. 4 setting the context: the population of bangladesh was 149,772,364 in 2011, of which 1,586,141 (1.1% of the total population) belonged to indigenous nationalities (bbs 2014). most of the indigenous peoples live in clusters in different parts of the country and are commonly called “adivasi” (ancient people) and most of them also identify themselves as “adivasi”. there are about 45 distinctly recognisable indigenous nationalities in bangladesh (rahman, 2013). the chittagong hill tracts (chts) is a remote area located in the southeastern part of bangladesh bordering myanmar to the southeast, the indian state of tripura to the north, mizoram to the east and the chittagong district to the west. chts consist of three hilly districts: rrangamati, khagrachhari and bandarban. the area of the chts is approximately one-tenth of the total area of bangladesh (cht n.d) 4.1 socio-economic condition of indigenous peoples in bangladesh: the population in the chts is considered as one of the most vulnerable because of low incomes, high poverty and inadequate and poor housing, health, water, sanitation and education facilities and inter-community relations (hossain, 2013). sharmin (2011) identified different types of discrimination, including unfair prices for goods produced, unfair wages, violence against children and adolescents, discrimination in the classrooms, access to credit, high rates of land tax/ mutation and low levels of government services and other facilities. the opportu¬nities for diversified livelihoods are very limited in the hilly areas, which also limit the scope for agriculture as the main livelihood. those who do not own any or insufficient land work as farm labourers or tenant farmers. there is also some fishing opportunities in the area (hossain, 2013). approximately 62% households in the region across ethnic groups lived below the absolute poverty line (2,122 kcal/day), while 36% were destitute (living on less than 1,805 k.cal/day) (barkat et al. 2008). in terms of per capita savings, the average amount per household member was tk.702 in the region, while the average among the indigenous 6 australasian journal of information systems ashraf, grunfeld & quazi 2015, vol. 19, research on indigenous ict impact of ict on quality of life population was tk.467 and tk.890 across bangladesh (barkat et al., 2008).the average annual net income of a rural household in the region was about tk. 65,852, while the same in rural bangladesh was 1.28 times higher (barkat et al. 2008). 4.2 present ict scenario in bangladesh: as early as 1991, the export promotion bureau (epb) of bangladesh with support from united nations development programme (undp) and international trade centre (itc) had taken an initiative to explore market potential of icts (digital bangladesh 2009). after that considering the necessity of ict the ruling political party of bangladesh named “bangladesh awami league” announced the “digital bangladesh” in pre-election commitment. “digital bangladesh” is now a government policy aimed at establishing e-government, e-business, elearning, e-health, e-employment, e-environment, e-agriculture, e-science to ensure job placement and poverty reduction (bhuiyan, 2011). the total number of internet subscribers has reached 39353.142 thousand at the end of july, 2014 where 37846.096 thousand subscribers use mobile internet, 276.046 thousand use wimax and 1231.00 thousand use their internet by isp+pstn indicating that 26.3% of the total population use internet services (btrc, 2014). despite these developments internet growth is constrained by poor telephone infrastructure, low international bandwidth and high-dial-up tariffs levied on internet users. in october, 2012 state owned mobile operator company “teletalk” introduced 3g meaning third generation, mobile services. 5 results and discussion this section presents the results from the field research, in the form of story-telling by the participants and analyses the discussion from the perspective of the five freedoms identified by sen (2001). we treat participants’ statements as “proof of concept”. we believe the reliability of our interpretations is reinforced through the provision of statements from many participants, whether these are unique or common within and between different groups of respondents. 5.1 political freedom: the politics of indigenous identity has become a global phenomenon with numerous groups active at the international level, raising issues related to prior occupation, way of life associated with land and lack of safeguards for future generations (lawson, 2014). the 1989 international labour office convention 169 (ilo 2014) on indigenous and tribal people recognised the aspirations of indigenous peoples to ‘exercise control over their own institutions, ways of life and economic development and to maintain and develop their identities, languages and religions, within the framework of the states in which they live’. after many years of drafts and negotiations, the united nations declaration on the rights of indigenous peoples (hereafter ‘undrip’) was adopted on the 13th of september 2007 (wright, tomaselli & ganoza, 2014). it enshrines both the individual and collective rights of indigenous peoples: including the rights to self-determination, education, development, land and natural resources, intellectual property, culture, and the right to treaty recognition (xanthaki, 2009). to address the challenges faced by the local indigenous peoples, gpcic has continuously worked on raising their awareness of existing laws, including their legal and human rights, and encouraging them to seek assistance from a formal court rather than a ‘salish’ (informal village court); and motivating people to raise their voices against the injustices of the ‘salish’. our field data demonstrate that participants have become conscious of many legal issues (divorce law, human rights, etc.) and now approach the courts for legal assistance. they realise that the ‘salish’ system takes advantage of their ignorance regarding different legal aspects; and that it has imposed injustice upon them on many occasions. however, the situation is changing since the indigenous people have been involved with gpcic. one example is cited below: a poor housewife came to a gpcic complaining that her husband beat her regularly; though she intended to make the complaint in ‘salish’, she knew that it wouldn’t succeed without good connections with influential people in her village. as soon as she 7 australasian journal of information systems ashraf, grunfeld & quazi 2015, vol. 19, research on indigenous ict impact of ict on quality of life understood the legal aspects of women’s and children’s rights from her gpcic training sessions, she decided to seek assistance from the legal court rather than from ‘salish’. later, the court called in the plaintiff’s husband and proper justice was done. there are two sources of legal support for disadvantaged people living in rural and remote areas of bangladesh. one of these is the central government, which has a constitutional obligation to provide legal services to all its citizens. the bangladesh ministry of social welfare has a unit in every district judicial magistrates court to assist poor people with legal protection, free of charge. local government agencies, known as union parishad (the micro rural administrative units) are also supposed to mediate disputes among the rural/indigenous peoples which is some form of legal assistance and is practices as, it saves time, money and effort of both parties to the dispute. along with this, there are also traditional legislative councils in villages, such as ‘panchayeta’, ‘darbar’, ‘salish’ etc. but functional inefficiencies, lack of transparency and accountability hinder the proper availability of these services to those who are entitled to them. in the case of traditional legislative councils, there is class and religious bias. religious and cultural fundamentalisms have also contributed to the malfunctioning of this system. since the indigenous people live in remote districts which are inadequately serviced by telecommunications infrastructure they have very limited access to icts. furthermore, because of their poverty, they cannot afford expensive icts. in bangladesh, the government has proposed to introduce an e-voting system which will further disadvantage indigenous people due to lack of adequate ict facilities. they are already disenfranchised due to their low awareness, partially stemming from lack of communication facilities, depriving them of information on policies of various candidates standing for local and national elections. as one young (female) user of the gpcic commented: “being indigenous, i also have the equal right to vote but i can hardly exercise it at the moment. i hope one day i will be able to exercise my political right to participate in the governance of the country by selecting my representative at the local and national levels.” through the help of gpcic, users of the centre played an active role in the political and legal systems and also became involved in implementing development programmes. those who did not participate in this process were aware of its importance, as illustrated by a young (male) participant: “when any important decision is taken in the village arbitration, we become so curious to see whether the justice is in place or not because the outcomes of the arbitration would affect our interests, anyway.” as the gpcic created awareness of legal issues, encouraged users to play an active role in the planning and maintenance of village facilities, express opinions, participate in setting the agenda for development and to interact with local government officials, it contributed to the political freedoms of members in this indigenous community. 5.2 economic freedom: irrespective of ethnicity, 95% households in the cht areas own some land (bbs 2013) and this ownership provides them with some degree of economic freedom. however, more than 80% of the households do not have electricity and this limits their level of economic freedom. 52% of the total household members is either employed or employable. plough and jhum cultivation has been found in more than 50% of all indigenous households, jhum cultivation is the second most pronounced source of income across the districts (barkat et al., 2008). there have been many initiatives e.g. technical education and training for youth, women and other disadvantaged groups by different government and non-government organisations to improve the economic status of indigenous people in the cht areas. the key objective of gpcic in the chts is to make the benefits of the dynamic ict sector in bangladesh available for the economic development challenge of indigenous groups through the making use of icts. the gpcic participants get opportunities to improve their livelihoods after obtaining training 8 australasian journal of information systems ashraf, grunfeld & quazi 2015, vol. 19, research on indigenous ict impact of ict on quality of life from gpcic. the benefits of computer training far exceeded the low course fees charged to make the gpcic sustainable, as articulated by a male teacher in an interview: “i must say that this computer training with a minimal cost is very useful especially for youth. in the present time, the knowledge of basic operations of it is required for all kinds of people. there’s no such training institute available in our village.” livelihoods were improved as participants learned from gpcic about opportunities to generate income from online commerce. one male participant mentioned: ‘my mother runs the family since my father passed away many years ago. she is the only earning source of our family who makes and sell handicraft goods. since she is getting older, i thought to help her by any means. one of my friends from gpcic told me that internet/web site could be an alternative way to sell handicraft goods. i joined gpcic, learned web site designing and used this skill in selling the products. this helped me in boosting sales and thereby enabled me to better help the family.’ the gpcic program included development of entrepreneurial skills, especially for the youth. one young male participant expressed his gratitude to the ict program: “i have neither higher education nor sufficient capital to invest. with the motivation of past trainee at gpcic, i got general training which helps me to understand the basics of computing and doing business. after successfully completing the courses i started setting up a computer compose business with the help of my friends. my business is now going very well.” through training in ict and entrepreneurial skills, gpcic contributed to improving the economic freedom of participants. the removal of some barriers to gaining economic benefits, the centre contributed to the improvement of livelihoods of indigenous community. the word of mouth communication of participants about gpcic to prospective members of indigenous community playing pivotal roles in attracting increasing number of participants to the program. 5.3 social freedom: the chts areas are in a post-conflict situation, following years of civil unrest, which officially ended in 1997 with the signing of the peace accord between the indigenous leaders and the government of bangladesh (cht n.d.). one of the elements of the peace accord was to recognise the rights of indigenous communities to land and other sovereign issues. although the situation has improved over the last few years in the chts areas regarding access and use of basic social services such as education, health, nutrition, water and sanitation services, there are nevertheless significant health issues prevailing in the region. gpcic contributes to improvement of health in the community, e.g. by boosting their awareness about health through showing documentaries about hiv/aids. one young participant conveyed what she had learnt about aids and its prevention through gpcic: “in gpcic, i watched a cd on aids showing that aids is not a contagious disease. i also learnt that aids is caused by sexual interactions, blood transfusion or sharing syringes or needles with people infected by aids. with the help of gpcic i am now fully aware of the causes of aids” the manager of the gpcic operates the programs with the guidance and support of a selected management committee. the participants of gpcic are encouraged to develop their management and leadership skills, e.g. by arranging cultural and sports events in the community. for example, having learned about drugs at the gpcic, an adult participant (male) started an anti-drug campaign that encourages children and youth to visit the gpcic. according to one male participant, “the anti-drug campaign organised by the gpcic contributed to enhancement of our consciousness about what is good and what is bad for the society. more participants should join the program to keep themselves away from drugs’. 9 australasian journal of information systems ashraf, grunfeld & quazi 2015, vol. 19, research on indigenous ict impact of ict on quality of life the gpcic also introduced mobile banking system for securing financial transaction in the community. according to one young (female) participant: “we used to send or receive money to other parts of bangladesh for personal or business purpose through informal channels which is obviously very risky. after gpcic introduced mobile banking system, we felt very secure with our transactions.” the gpcic programs are well accepted by family members of the participants and one of the main reasons behind this is the equal learning opportunities provided by the nondiscriminatory course composition and training curriculum. a female trainee supported this view: “my participation helps the girls convince their parents to participate in the ict course and many parents visited the gpcic and express their willingness to continue their support.” the gpcic has thus contributed to social freedoms by enabling community members to secure better control over their health through training on various healths related issues. through its non-discriminatory policies gpcic encouraged gender equality. 5.4 transparency guarantees: while access to information is necessary for transparency, which is in turn required to discover corrupt conduct, it is not sufficient to achieve transparency freedoms. other measures required include accountability, education, free media and free and fair elections (lindstedt & naurin, 2010). the proper balance between confidentiality required by some government decisions and the rights of civil society is subject to public debate. citizens have different degrees of interest in and demand for governmental transparency (piotrowski & van ryzin 2007). gpcic provided different kinds of information to the community members and assisted them with gaining knowledge in diverse areas and establishing good governance across public/private institutions. ict programs also help the schools in concerned village to initiate lab-based education system, as suggested by a committee member: “the success of this program helps us to introduce ‘lab based education’ in our village. since students are familiar with computer through this ict training, it was much easier for us to foster lab based education amongst indigenous people”. also a member involved in managing organisation of gpcic felt that: “the participants can learn computer, read books & newspapers in addition to their normal curriculum from the internet. now a day they play creative games instead of the traditional game. as a result the mental development may rise higher than physical growth. in this centre a new method of learning & teaching has been introduced where both trainer and trainee participate & interact.” learning about current affairs at gpcic also helped participants understand the importance of transparency with respect to the workings of the legal system and environmental issues. with the help of gpcic, community members were able to witness organisational accountability and transparency and the importance of these for their basic rights in society. it is widely acknowledged that corruption has negative effects on economic growth, investment, and social welfare and illicit flows associated with it has become the subject of much attention (oecd 2014). relly & sabharwal (2009) found that many countries ranked as ‘most transparent’ had significantly higher levels of access-to-information laws, telecommunication infrastructure, e-government, free press, and higher income levels, than countries with low transparency levels. in bangladesh, the information right act 2009 is designed to provide access to information in an effort to reduce corruption. gpcic provides free access to and interpretation of the information right act to indigenous people. according to one adult (male) participant: “we have been informed of the activities of local government land office through gpcic. we can handle land registration on the internet and don’t need to go to office 10 australasian journal of information systems ashraf, grunfeld & quazi 2015, vol. 19, research on indigenous ict impact of ict on quality of life physically. this reduces the time of processing our registration and also avoids paying bribes to corrupt officials.” gpcic has facilitated access to transparency guarantees for the indigenous population, through training showing as to how to access information that will make government action more transparent. also gpcic provided tools, such as a land registration system through which they have been able to experience greater transparency. 5.5 protective security: in addition to social safety net that are important to people everywhere, protective security in the context of climate change and other natural disasters (e.g. cyclones, floods, and droughts) is particularly important in bangladesh. these natural calamities devastate people’s lives and are also very damaging for the agricultural economy and people’s livelihoods. in the cht areas, earthquakes and heavy rainfall are common phenomena that adversely affect local communities. ict-based information provided by gpcic had provided protective security for the local community. one young (female) participant member said: “we get information related to natural disasters (e.g. flood, earthquake etc.) from the gpcic centre. this helps us with our advance preparations for disasters management.” the participants also shared the information related to natural disasters with others, as indicated by another adult (male) participant: “when i get disasters information on heavy rain or earthquake which would affect my family and others, i warn my bothers/sisters/family members and others to take pre-caution.” according to the un general assembly (2007), indigenous peoples have the right to the conservation and protection of the environment, health and the productive capacity of their lands or territories and resources. trees are important to protect soil and human lives during heavy rainfall in hilly areas. unfortunately, illegal logging has become common in cht areas. to protect the environment there, the participants are committed to uncover this type of crime with the help of management. one of the adult (male) participants explained: “we are divided into different groups with 5 to7 persons including government officials. we visit our forests occasionally and check out any unexpected events. we also arrange road shows to increase awareness of the diverse effects of environmental damages.” the main mechanisms through which the gpcic has contributed to the improved protective security of the indigenous population is through providing access to the early warning system for disasters and by facilitating the detection and prosecution of illegal loggers. the field research results provided us with the perceptions of indigenous people in the cht area of bangladesh, rather than any ‘objective’ measures of causality between gpcic and sen’s (2001) five freedoms. it is to be noted that since other activities contribute to socio-economic development of the indigenous community, it is not possible to attribute development/changes only to this project. 11 australasian journal of information systems ashraf, grunfeld & quazi 2015, vol. 19, research on indigenous ict impact of ict on quality of life freedom type major findings of the study political freedoms in general the gpcic was found to be effective in term of developing awareness in indigenous people about exercising their political freedoms. in particular gpcic helped create awareness of legal issues, encourage users to play an active role in the planning and maintenance of village facilities, express opinions, participate in setting the agenda for development and to interact with local government officials, it contributed to the political freedoms of members in this indigenous community. economic freedoms gpcic made positive contributions towards improvement of indigenous peoples’ economic freedom through specific training in ict usage and entrepreneurial skills. in particular the training was instrumental in the removal of some barriers to gaining economic benefits resulting in improvement of livelihoods of the indigenous community. social opportunities the gpcic contributed to social freedoms by enabling community members to have better control over their health through training on critical health issues affecting indigenous people. most importantly, through its non-discriminatory policies, gpcic encouraged gender equality amongst indigenous community. transparency guarantees gpcic has facilitated access to transparency guarantees for the indigenous population, through training in how to access information in making government action more transparent and by providing tools, such as land registration system enabling indigenous people to experience greater transparency. protective securities the main mechanisms by which the gpcic has contributed to the improved protective security of the indigenous population is through providing access to the early warning system for disasters as well as by facilitating the detection and prosecution of illegal loggers. table 2: summary of results at a glance 6 concluding remarks this pioneer study has addressed a lacuna in the extant literature as to the possible influence of application of information technology on indigenous peoples’ lives in general and in the context of a developing country. more specifically this research examined if icts have any role to play in achieving the five freedoms developed by noble laureate amarty sen (2011). the qualitative results emerging from the study of the indigenous people in the remote regions of bangladesh show that ict training was instrumental in creating consciousness amongst indigenous people about achievement of many elements of the five freedoms. participants mentioned about a wide range of issues pertaining to their quality of lives that were enhanced through the programs set by the grameen phone community information centres (gpcic). in particular this research found that gpcic’s programmes has potential to reduce various forms of unfreedoms, thereby opening up new opportunities in the five freedom types in domains ranging from income generating activities, education, health, improved transparency and disaster warnings. ict based training has, thus, emerged an informational tool to help indigenous people overcome the disadvantages and discriminations they have been exposed to in bangladesh. it is noteworthy that gpcic is not only devoted to imparting mere ict training, rather its programmes are more broad based in terms of addressing the various issues impacting the lives and livelihood of indigenous people in bangladesh. in this sense, this study sets the stage for further research encompassing the emerging issues affecting socioeconomic and political future of the indigenous population in other developing countries where the situation would probably be the same. it is not denying the fact that in addition to the impact of the gpcic, other factors may have contributed to the realisation of five freedoms which may have been integrated in the comments made by the participants of this research. nevertheless, icts have emerged as a useful communication tool which was made easily accessible (including training) to indigenous population of bangladesh many of whom cannot 12 australasian journal of information systems ashraf, grunfeld & quazi 2015, vol. 19, research on indigenous ict impact of ict on quality of life afford it simply because of their low income status in the community. although icts cannot have direct impacts on behavioural changes of indigenous population in bangladesh, the tool can be useful in creating awareness about the importance of the issues that are of interests to indigenous people which may have indirect effects on their behavioural changes in relation to the five dimensions of freedoms. interestingly, this reality was paramount in the comments of the participants of this study. 6.1 limitations and future research this paper has a number of limitations that are to be considered in generalising the results across other developing and well as developed countries. first, this paper uses qualitative technique in collecting and analysing data based on a limited sample size in only one developing country. future research should use mixed methodology approach using more broad based samples drawn from a number of developing countries to ensure better generalisation of the findings. second, when focussing on the gpcic, there is a question of attribution in terms of whether icts alone facilitated the changes in participants’ behaviours or icts were provided in a mediated form in conjunction with other activities? this issue need to be addressed in future research by isolating icts’ impact from those of others. third, indigenous peoples’ rights and freedoms are affected by inter-communal tensions with the main stream population living side by side with them. this issue need to be investigated further in future studies. third, it would not be wise to draw conclusion about the influence of icts on indigenous peoples’ lives based on a limited study conducted in a single point in time. longitudinal studies focusing on behaviour changes in indigenous people would capture the changes initiated by ict over a period of time. finally, future research can use other appropriate frameworks along with sen’s (2011) four dimensional framework of freedoms to broaden and deepen our understanding of the complexities associated with icts and achievement of freedoms of indigenous people. references acma. (2008). telecommunications in remote indigenous communities. canberra: commonwealth of australia. alampay, e. (2006). analysing socio-demographic differences in the access & use of icts in the philippines using the capability approach. the electronic journal for information systems in developing countries, 27(5), 1–39. alkire, s. (2005). why the capability approach? journal of human development, 6 (1), 115133. amato, s. (2013),"addressing indigenous (ict) approaches in south-east asian learning systems", multicultural education & technology journal, 7(1), pp. 46 -63. anand, p. b. (2007). capability, sustainability, and collective action: an examination of a river water dispute, journal of human development, 8(1), 109–132. andersson, a., grönlund, a. &wicander, g., (2012): development as freedom – how the capability approach can be used in ict4d research and practice, information technology for development, 18:1, 1-4 barkat, a. et al. (2008). socio-economic baseline survey of chittagong hill tracts, conducted for undp-chtdf, human development research centre (hdrc), dhaka. bbs (2013). bangladesh bureau of statistics. statistics and informatics division (sid), government of bangladesh. beod (2014), bangladesh ethnobotany online database, dhaka, bangladesh. retrieved july 2014 from http://www.ebbd.info/indigenous-communities.html bhuiyan, s. h., (2011), modernizing bangladesh public administration through e-governance: benefits and challenges, government information quarterly 28, pp. 54–65. 13 australasian journal of information systems ashraf, grunfeld & quazi 2015, vol. 19, research on indigenous ict impact of ict on quality of life btrc (2014), internet subscribers in bangladesh: bangladesh telecommunication regulatory commission, retrieved from http://www.btrc.gov.bd/content/internetsubscribers-bangladesh-july-2014 biggeri, m., libanora, r., mariani, s. &menchini, l (2006). children conceptualizing their capabilities: results of a survey conducted during the first children's world congress on child labour. journal of human development, 7(1), 59 – 83. comin, f. (2001 june). operationalisingsen’s capability approach. paper prepared for the conference justice and poverty, examining sen’s capability approach. cambridge. retrieved august 20, 2006 from http://stedmunds.cam.ac.uk/vhi/sen/papers/comim.pdf. corbridge, s. (2002). development as freedom: the spaces of amartya sen. progress in development studies, 2(3), 183-217. cht (n.d). chittagong hill tracts issues: geographic and physical feature, ministry of chittagong hill tracts affairsweb site. retrieved august 22, 2014. from http://www.mochta.gov.bd/index.php/index/othercontent/other-details__14/7/0/12 corbett, j., singleton, g., & muir, k. (2009). web 2.0 for aboriginal cultural survival: a new australian outback movement. participatory learning and action, 59(1), 71-78. daly, e. m., &haahr, m. (2007). social network analysis for routing in disconnected delaytolerant manets. in proceedings of the 8th acm international symposium on mobile ad hoc networking and computing (pp. 32-40). acm. digital bangladesh (2009), access to information programme, prime minister's office, dhaka, bangladesh. evert, g. (2003). all research is interpretive! journal of business & industrial marketing, 18(6/7), 482 492. gasper, d. (1997). sen’s capability approach and nussbaum’s capabilities ethics. journal of international development, 9(2). gigler, b. s. (2009). poverty, inequality and human development of indigenous peoples in bolivia. center for latin american studies. working paper series 17, georgetown university. hatakka, m., &lagsten, j. (2012). the capability approach as a tool for development evaluation–analyzing students’ use of internet resources. information technology for development, 18(1), 23-41. hossain, d.m. (2013), socio-economic condition of the indigenous people in the chittagong hill tracts (cht) of bangladesh, middle east journal of business, 8(2), 22-30. hunter, j. (2005), the role of information technologies in indigenous knowledge management, australian academic & research libraries, 36(2),109-124, doi:10.1080/00048623.2005.10721252 ilo (2014), convention no 169, international labor organisation, retrieved august 20, 2014. from http://www.ilo.org/indigenous/conventions/no169/lang--en/index.htm johri, a. & pal, j. (2012): capable and convivial design (ccd): a framework for designing information and communication technologies for human development, information technology for development, 18:1, 61-75 johnston, a. l. k., (2010), using technology to enhance aboriginal evaluations, the canadian journal of program evaluation, 23 (2), 51–72 lima, a. p.d., &brito, c. (2012). an examination of the tribal community dimensions of ict users. journal of internet commerce, 11, 291–308, doi: 10.1080/15332861.2012.729468 14 australasian journal of information systems ashraf, grunfeld & quazi 2015, vol. 19, research on indigenous ict impact of ict on quality of life lunat, z. (2009). the palestinian hidden transcript: domination, resistance and the role of ict in achieving freedoms. the electronic journal for information systems in developing countries, 37(1), 1–22. lwoga, e. t., ngulube, p., & stilwell, c. (2010). the management of indigenous knowledge with other knowledge systems for agricultural development: challenges and opportunities for developing countries. in scientific and technical information and rural development–iaald xiiith world congress, montpellier (pp. 26-29). mayerhofer, p., & taylor, a. (2010). the influence of ict on indigenous mobility, population studies research brief 2010059 nussbaum, m. (2000). women and human development: the capabilities approach. cambridge: cambridge university press. nussbaum, m. (2003). capabilities as fundamental entitlements: sen and social justice. feminist economics, 9(2/3), 33-59. nussbaum, m. (2006). capabilities as fundamental entitlement. in a. kaufman (ed.), capabilities equality: basic issues and problem, 44-70. new york: routledge. oecd (2014), better policies for development 2014: policy coherence and illicit financial flows, oecd publishing. doi: http://dx.doi.org/10.1787/9789264210325-en. radoll, p., fleissner, s., stevenson, d., & gardner, h. (2013). improving ict support for aboriginal land councils in new south wales. paper presented at the proceedings of the sixth international conference on information and communications technologies and development: notes-volume 2. radoll, p. j. (2011). the primary determinants of indigenous household information and communication technology adoption: remote, rural and urban. australasian psychiatry, 19(1 suppl), s49-s52. radoll, p. j. (2010). stone chips to silicon chips: a grounded theory of information and communication technology adoption in australian indigenous households rural, urban and remote. phd thesis. the australian national university. page 345 rahman, m. (2013). problems and prospects of development: intervention for the indigenous people bangladesh. raitt, d., & roy, l. (2003),"the impact of it on indigenous peoples", the electronic library, 21 (5), pp. 411 – 413. doi: http://dx.doi.org/10.1108/02640470310501412 robeyns, i. (2001). understanding sen’s capability approach. wolfson college, cambridge. uk. retrieved on august 20, 2006, from http://www.ingridrobeyns.nl/downloads/under_sen.pdf. robeyns, i. (2005). the capability approach: a theoretical survey. journal of human development, 6(1), 93-114. salazar, j. f. (2007). indigenous peoples and the cultural construction of information and communication technology (ict) in latin america. information technology and indigenous people, 14-26. schischka, j., dalziel, p., saunders, c. (2008). applying sen's capability approach to poverty alleviation programs: two case studies. journal of human development, 9 (2), 229 – 246. sen, a. (1982). rights and agency. philosophy and public affairs, 11(1), 3-39. sen, a. (1989) development as capability expansion, journal of development planning, no. 19, pp. 41-58. sen, a. (2001) development as freedom, london: oxford university press, first published by alfred a. knopf in 1999. 15 australasian journal of information systems ashraf, grunfeld & quazi 2015, vol. 19, research on indigenous ict impact of ict on quality of life sharmin, s. (2011). socio-economic situation and land rights of the indigenous people in bangladesh. oida international journal of sustainable development, 2(11), 85-96. suresh, s., &nath, l. (2014). challenges in managing telemedicine centers in remote tribal hilly areas of uttarakhand, indian journal of community health, 25(4), 372-380. stavenhagen, r. (2002). indigenous peoples and the state in latin america: an ongoing debate. multiculturalism in latin america: indigenous rights, diversity, and democracy, 24-44. stewart, f. &deneulin, s. (2002). amartyasen's contribution to development thinking. studies in comparative international development, 37(2), 61-70.3. stewart, f. (2005). groups and capabilities. journal of human development, 6(2), 185-204. taylor, a. (2012). information communication technologies and new indigenous mobilities? insights from remote northern territory communities. journal of rural and community development, 7(1), 59-73. taylor, s. j., &bogdan, r. (1984). introduction to qualitative research methods: the search for meanings (2nd ed.). new york: wiley. thapa, d., &saebo, o. (2014). exploring the link between ict and development in the context of developing countries: a literature review. the electronic journal of information systems in developing countries, 64. watson, s. (2013). new digital technologies: educational opportunities for australian indigenous learners, the australian journal of indigenous education, 42(1), pp. 58-67, doi 10.1017/jie.2013.8 wresch, w. & fraser, s. (2012): ict – enabled market freedoms and their impacts in developing countries: opportunities, frustrations, and surprises, information technology for development, 18:1, 76-86 zheng, y. &walsham, g. (2008). inequality of what? social exclusion in the e-society as capability deprivation. information technology & people, 21(3), 222-243. copyright: © 2015 ashraf, grunfeld & quazi. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. 16 http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 literature & operationalisation of five freedoms 3 research methodology 4 setting the context: 4.1 socio-economic condition of indigenous peoples in bangladesh: 4.2 present ict scenario in bangladesh: 5 results and discussion 5.1 political freedom: 5.2 economic freedom: 5.3 social freedom: 5.4 transparency guarantees: 5.5 protective security: 6 concluding remarks 6.1 limitations and future research paper title (paper title style) australasian journal of information systems mackrell & mcdonald 2016, vol 20, design research for dss development evaluation using action design science 1 an evaluation view of an ensemble artefact for decision support using action design research dale mackrell university of canberra dale.mackrell@canberra.edu.au craig mcdonald university of canberra abstract this paper investigates the integration of content, context and process (ccp) into the action design research (adr) framework to account for the interplay of organisational issues in artefact design and development. the investigation is conducted through a case study in which successive ict student teams incrementally build, over several semesters, a tailored, low cost business intelligence (bi) system as an ensemble artefact for an organisation in the not-forprofit (nfp) sector. during project development, ccp’s human-centred approach to evaluation complements adr’s more prescribed technology-driven software testing. the integration of ccp into adr as an evaluation view offers an holistic approach to assessing an ensemble artefact. the resultant conceptual framework is presented as a model with an explication of unexpected design and research outcomes. keywords: action design research; action research; design science; evaluation; qualitative; not-for-profit 1 introduction scholarly articles of design science research in decision support systems (dss) development have highlighted unresolved key issues that need attention: relevance, evaluation, theorising, and strategic focus, to name a few (arnott & pervan 2012; miah & gammack 2014). arnott and pervan, in 2005 and 2008, identified that the relevance and quality of dss research would improve by increasing the number of case studies, especially interpretive field studies. more currently, arnott and pervan (2012) call for stronger theoretical contributions to design foundations and methodologies in dss design science research while lamenting the dearth of articles in the contemporary and related field of business intelligence (bi) which has raised the expectations of managers for strategic-focussed enterprise-wide computer-based decision support1. arnott and pervan (2012) claim evaluation to be an endemic weakness in dss design science research and recommend that multi-method evaluation including field-based review and action research be encouraged. a contemporary approach that draws on action research and design science is action design research (adr) which values both organisational relevance and technological rigour with the intention of extending design knowledge (sein, henfridsson, purao, rossi & lindgren 2011). adr aims at building innovative it artefacts interactively in an organisational setting. this socio-technical approach takes place through a predefined reflexive process to address a problem-solving situation while stakeholders learn from the intervention. underlying adr is a strong reliance on review through a cycle of building, intervention and evaluation (bie) (sein et al. 2011). the notion of ‘ensemble artefact’ is proposed by sein et al (2011, p.38) to reflect the view that organisational structures are embedded into the artefact during its development and use. the concept originates as an ‘ensemble view’ in the influential paper by orlikowski and iacona (2001, p.126) which restores a focus to the it artefact in information systems (is) research 1 sharda, delen and turban (2014) claim that dss of the 1970s evolved into bi systems in the mid-1990s, with both dss and bi systems having a focus on supporting decision makers. australasian journal of information systems mackrell & mcdonald 2016, vol 20, design research for dss development evaluation using action design science 2 while recognising “the dynamic interactions between people and technology”. goldkuhl2 (2012; 2013) contests the transformation from ‘ensemble view’ to ‘ensemble artefact’. in their rejoinder to goldkuhl’s critique, purao, henfridsson, rossi and sein (2013, p.76) retort that the term, ‘ensemble artefact’, is ‘succinct shorthand’ for a more holistic view of the organisational and social context of an it artefact rather than merely as a software-hardware tool as emphasised in traditional design science. debate with respect to the troubled conceptual shift in nomenclature is not to the forefront of this paper. in a similar vein, this paper does not dwell on the distinction in terms, design science, design research and design science research, leaving this to other authors (winter 2008, baskerville 2008). in the main, this paper refers to design science research using the acronym dsr for simplicity. purao et al. (2013, p.79) stress that adr is still an ‘open endeavour’. the emphasis of this paper is on evaluation, thus heeding concerns regarding ‘user-centredness’ as a significant but neglected aspect of dss design (miah & gammack 2014) as well as issues of relevance, theorising, and strategic focus (arnott & pervan 2012). from a growing body of evaluation literature, stockdale and standing (2006) recognise the absence of an evaluation framework to reflect social, political and cultural factors that influence project development. they argue that many is evaluations overlook the social context in organisations by focussing on the it artefact. they propose an interpretive evaluation framework by adapting the content, context and process (ccp) model from symons (1991). ccp, combined with technical testing, offers a comprehensive and more holistic appraisal of is projects. the value of holistic evaluations in adr is illustrated in this paper in an ongoing research project in the not-for-profit sector. this paper has significance from three main perspectives: practical, theoretical and methodological. the practical contribution of the project is to design a data mart and reporting tools in the cloud to capture, store and retrieve quality integrated data as first steps to a bi solution. the construction of operational it artefacts takes place progressively in connections act, a local canberra not-for-profit organisation (nfp). the aim is to keep costs low and to utilise available resources student teams, funding from grants, university technical expertise, community support, and open source software options. this project is ongoing and empirical, with two to four student teams each semester allocated to the project until the planned implementation in late 2016. the study has interventionist goals and serves as a means of deriving a design guide useful to other nfps wishing to implement bi using adr. this is the methodological contribution of the study. this paper continues with a discussion of adr positioning the paper within existing research and follows with an illustration of evaluation in the connections act setting. from this project, a ccp evaluation model, integrated with adr as an evaluation view, extends the conceptual work of sein et al. (2011). this could be described as ‘partial theory’ according to the classification by gregor, imran and turner (2010) with attention on the socio-technical artefact in its setting, design principles of form and function, and associated theories. the integrated and extended framework is modelled and explained in the discussion section while future research is proposed in the concluding section. 2 explaining action design research action design research (adr) is a research approach advanced by sein et al. (2011) to address a problem-based situation by building innovative technical artefacts in an organisational setting using a structured and predefined process and to learn from the activity. adr is a methodological solution which combines both canonical action research (ar) with its enduring research tradition (davison & martinsons 2004) and design science research (dsr) being an established research approach in the is discipline (hevner, march, park & ram 2004). ar, in an iterative process, investigates a contextual phenomenon through researcher intervention and generates theory through change, learning and reflection with scant regard for the it 2 goldkuhl (1981, cited in goldkuhl 2013) formulated the slogan of information systems as social systems only technically implemented. australasian journal of information systems mackrell & mcdonald 2016, vol 20, design research for dss development evaluation using action design science 3 artefact. on the other hand, dsr positions it artefacts at the core of the is discipline yet may fail to adequately recognise the role of the organisational and social context shaping both the design and the deployed artefact (sein et al. 2011). the adr approach proposed by sein et al. (2011) builds on cole, purao, rossi and sein (2005) to prescribe four stages within which are seven principles, set out in table 1. this paper concentrates on stage 2 in table 1: building, intervention and evaluation (bie). sein et al (2011, p.40) contend that adr is an innovative research method which recognises that the artefact, framed by march and smith (1995, p. 256) as ‘constructs, models, methods, and instantiations’, emerges from interaction with the organisational context. figure 1 shows a typical bie which suits adr efforts that emphasise creating an innovative technological design at the outset, termed the it-dominant bie (sein et al. 2011). stage 1: problem formulation principle 1: practice-inspired research principle 2: theory-ingrained artefact stage 2: building, intervention and evaluation principle 3: reciprocal shaping principle 4: mutually influential roles principle 5: authentic and concurrent evaluation stage 3: reflection and learning principle 6: guided emergence stage 4: formalisation of learning principle 7: generalised outcomes table 1. action design research stages and principles (sein et al. 2011) sein et al. (2011) use an alternative, slightly modified diagram with deeper end-user participation to describe adr efforts to generate design knowledge when the primary source of innovation is organisational intervention, that is, organisation-dominant bie rather than it-dominant bie. bie of the adr approach draws on three principles: reciprocal shaping, mutually influential roles, and authentic and concurrent evaluation. together, these principles emphasise the inseparability of the domains, the it artefact and the organisational context, that influence the adr project (sein et al. 2011, p.43). figure 1: the adr generic schema for building, intervention and evaluation bie (sein et al. 2011) australasian journal of information systems mackrell & mcdonald 2016, vol 20, design research for dss development evaluation using action design science 4 the relationship between ar and dsr has been widely analysed for some time (iivari & venable, 2009; järvinen 2007; maccani, donnellan, & helfert 2015; papas, o’keefe & seltsikas 2012). iivari and venable (2009) critique the paradigmatic assumptions of ar and dsr by examining the ontology, epistemology, and methodology of each to conclude that adr can reconcile divergent view but care needs to be taken due to fundamental philosophical differences. papas et al. (2012) express less enthusiasm for adr, suggesting that canonical ar may suffice assuming the criteria of interventionist research are satisfied, noting that the it artefact is normally a by-product of ar not the goal, while järvinen (2007) claims ar and dsr may be used together in a study with relative ease. the philosophical foundations of ar and dsr have been studied by maccani et al. (2015, p.7), drawing on iivari and venable (2009). from the preliminary findings of their research-inprogress, maccani et al (2015) determine that dsr assumes a positivist epistemology (especially in the engineering field) and is more likely to be design-derived, implying a technical approach to evaluation of the it artefact with software testing expected to be performed. adr, on the other hand, because of its participative nature, is more likely to have an anti-positivist or interpretivist position as an underlying assumption with the artefact being shaped by the characteristics of the evaluating individual such as the user. 2.1 views on evaluation evaluation has long been a topic of discussion in the ar and dsr literature with winter (2008) and goldkuhl and sjöströmb (2015) pointing out the lack of evaluative guidelines. it is noted by sein et al. (2011, p.39) that in most dsr efforts, “awareness of the problem precedes the development of the artefact”, which is then followed by evaluation. this is essentially the ‘build and then evaluate’ cycle (hevner et al. 2004; march & smith 1995) where evaluation is recognised as a separate and distinct activity. goldkuhl and sjöströmb (2015) insist that evaluation should occur at all stages of the enquiry process, citing adr as an approach which integrates evaluation into building and intervention activities. there is a strong motivation in adr to capture organisational aspects of the intervention in a ‘meta-approach’ to evaluation combining elements of both ar and dsr. both bie schemas by sein et al. (2011), namely it-dominant bie and organisation-dominant bie, demonstrate an evaluation cycle for the alpha version of the artefact which is formative and another for the beta version which is summative. sein et al. (2011) claim that controlled evaluations may be difficult to achieve in an adr project due to the emergent nature of the artefact. in prescribing alpha and beta evaluations, sein et al. (2011) specify evaluation timings but give a lower profile to the details of what, why, who and how of adr evaluation. in prototyping situations or when project goals are indistinct, it is possible that more iterations are required, in which case, evaluation cycles are dependent on the version of the artefact. in the section following, we examine the content, context and process (ccp) evaluation framework from stockdale and standing (2006) to assess its suitability as an interpretive methodology to supplement positivist technical assessments. 2.2 where does the ccp model of evaluation fit? stockdale and standing (2006) note the preponderance of technical assessments in is evaluations. they propose an interpretive approach for evaluating the complex interplay of organisational issues to complement software testing. stockdale and standing (2006) refer to the what, why, who, when and how of is evaluation in their paper extending symon’s (1991) content, context and process (ccp) framework. the ‘what’ is represented by the content construct to discern which measures and metrics comprise the evaluation, confounded in is with fewer measurable societal and organisational impacts. the context concept requires the ‘why’ and ‘who’ to be considered, this being internal and external organisational influences and various stakeholders. the process concept takes into account the ‘when’ and ‘how’. importantly, symons (1991, p. 211) argues for evaluation to continue throughout the project lifecycle, thus reducing the risk of failure. australasian journal of information systems mackrell & mcdonald 2016, vol 20, design research for dss development evaluation using action design science 5 likewise, pries-heje, baskerville and venable (2008) propose a strategic framework for dsr evaluation. they refer to the what, when and how of dsr evaluation, namely, product and process, ex ante and ex post, and artificial or naturalist settings, with lesser focus on the why and who. in the ex ante perspective, a technology artefact is evaluated before selection while in the ex post perspective, it is evaluated after appropriation. pries-heje et al. (2008) explain that dsr evaluation is often oriented towards ex post timings even though economic costs are paramount in ex ante analysis. this is certainly true for the private sector. in both public and not-for-profit sectors, the profit motive is less, with the recognition that other factors are at play such as concern for social well-being. as a socio-technical methodology, it would be expected that adr evaluation covers both ex ante and post ante perspectives. mackrell and mcdonald (2014) contend that humanand technology-centred evaluations together produce more complete knowledge than evaluations which focus separately on either social or technical dimensions. this paper takes another look at adr evaluation, to bring together the ccp framework and relevant concepts from existing dsr evaluation to propose an adr evaluation view. cleven, gubler and hűner (2009) claim that evaluation should not be an isolated activity. in their study, they surveyed existing methods of evaluating dsr artefacts, identified characteristics then drew up an evaluation framework. despite their comprehensive efforts, there is little emphasis on the organisational setting. for the purposes of this paper, we will initially adopt the definition of evaluation offered by scriven (1998, p.85) and cited in cleven et al. (2009): evaluation is the process of determining the worth, merit, or significance of entities; and evaluations are the outcome of that process. evaluation may be external or internal, or a mix of these; and it may be quantitative or qualitative, or a mix of these. it is strongly although not always sharply distinct from explanation. this evaluation definition is re-assessed at the conclusion of this paper in light of our findings. 3 illustrating adr with a case study a research approach for the study was sought which was prescriptive in the it artefact development lifecycle (it-dominant bie) necessitating technology-driven testing. the research approach also needed to be socially participative, reflective and reflexive (organisation-dominant bie), requiring a qualitative evaluative method, such as that offered by ccp. the features of adr fulfilled these requirements and guided the project’s development in a naturalistic organisational setting. 3.1 research site the project is a case study with connections act incorporated, a local canberra community organisation committed to helping disadvantaged communities and families who are homeless (or at risk of becoming homeless). connections act consists of 18 paid staff who assist over 3,000 people in the act annually through two services: first point, the central intake service for the homeless, and canfacs, a service for homeless fathers and their children. information systems in connections act consist of: • excel spreadsheets for managing the occupancy and maintenance of properties for canfacs. data related to nine properties and rental bookings are located in various non-structured repositories such as spreadsheets and whiteboards matrices. • sql database called ship (specialist homelessness information platform) with a web-based interface built by a melbourne-based external consultant for the collection of demographic homeless client data for first point. the system has limited standard and customised reporting features. • sql database called daily capacity report (dcr) developed with alpha 5 technology built for first point by a melbourne-based external consultant for australasian journal of information systems mackrell & mcdonald 2016, vol 20, design research for dss development evaluation using action design science 6 tracking accommodation and outreach availability. the system has limited reporting features. • bitcloud is a commercial enterprise for hosting applications in the cloud at a commercial financial cost to connections act. the managers at connections act struggle with onerous compliance and ad hoc reporting since often client data capture is cumbersome, data quality is poor, information retrieval is haphazard, and software is inadequate. nfps such as connections act are increasingly reliant on accurate and quickly-retrievable organisational data for regulatory reporting and on-going funding. this is difficult to realise for most nfps since the software available is predominantly suited to organisations in the for-profit sector which have different needs and resource-bases, and are not liable to compliance reporting as are nfps (mackrell 2012). by addressing connections act's data and software challenges, this project will enable the organisation to concentrate its limited resources on core business (helping the homeless). 3.1.1 research approach a pragmatic approach to the project was adopted. this is described below within the four stages and seven principles of adr proposed by sein et al. (2011) and set out in table 1. while the stages of adr are portrayed neatly in sequence, in reality, with reliance on evaluation, reflexivity and intervention, the stages overlap and parallel during iterations. stage 1: problem formulation in early 2012, the executive officer (eo) of connections act contacted the lead researcher for assistance with their deteriorating dcr database. the initial fact-finding stage of the project took place in late 2012, utilising a human-centred ‘sense-making’ approach to identify the technical, organisational, informational and decisional needs of connections act. the lead researcher conducted nine single face-to-face, in-depth interviews with senior connections act managers and one board member, and three group meetings with university of canberra bi and data warehouse experts. this provided background to the problem enabling basic requirements elicitation. analysis of interview transcripts and organisational documents suggested that silos of data existed in the organisation due to non-linking databases and spreadsheets, in turn thwarting the extraction and use of the valuable information they contained. as an upshot of this phase, it was decided that a practical outcome of the project would be the design, development and deployment of stable and reliable data repositories and is management systems as bi artefacts for both first point and canfacs as the preliminary stage to bi. as stated earlier and presented in table 1, sein et al. (2012) propose seven principles within the four stages. the two principles which apply in the problem formulation stage are principle 1: practice-inspired research and principle 2: theory-ingrained artefact. the research activity is problem-inspired since the trigger for the project was a phone call from the connections act eo who was receptive to practice-inspired research and welcomed a joint venture with the university of canberra. the projected artefact is theory-embedded, informed in the initial stages through sense-making, and structured via tailored industry frames of reference including business process models and mapping. stage 2: building intervention and evaluation (bie) gregor and hevner (2013, p.339), in their recent paper, use the term ‘program’ to define a significant research program which “typically encompasses many researchers over several years”, while a ‘project’ refers to a “portion of a larger program” which may be interim attempts at theorising. the researchers, in this paper, remain reticent about terminology but stay with the terms ‘project’ and ‘mini-project’ because of the initiative’s strong practical purpose in a single organisation. the entire project is a complex long-term undertaking, broken into mini-projects, to build a bi solution for a nfp, connections act. the bi deliverables consist of numerous data, software and process artefacts, which will be described in detail in a future paper. the project is part of australasian journal of information systems mackrell & mcdonald 2016, vol 20, design research for dss development evaluation using action design science 7 a work-integrated learning (wil) subject offered as a capstone to students in their final semester of studies. each student team is provided with a proposal of essential and desirable deliverables as a distinct mini-project expected by the end of semester. student teams, usually comprising a project leader, analyst, developer and tester, build the artefacts. the teams are well supported with scaffolding and mentoring each semester by the subject convenor, tutor, project coordinator, university and community technical experts, and importantly, the client. teams are expected to fulfil documentation requirements, complete a comprehensive testing schedule, and deliver to the client a functioning artefact, all of which is assessed and graded. these assessments are formal, with a concentration on technical and tangible aspects of system design and development. the social dimension is called into play through less formal qualitative evaluations of student teams, client users and managers. these interviews/meetings are face-to-face, semistructured, single or team conducted at several points in the semester: at the start and end of each semester, and mid-semester and/or any crises point. while these interviews/meetings are comparatively informal, they are guided by a prepared interview script or meeting agenda. the principles of sein et al. (2011) which apply at the building, intervention and evaluation stage are principle 3: reciprocal shaping, principle 4: mutually influential roles, and principle 5: authentic and concurrent evaluation. reciprocal shaping is described as the iterative process to construct and reconstruct both the it artefact and organisational context when solving ‘wicked problems’. one of the so-called ‘wicked problems’ is excel spreadsheets, created by the client as a stop-gap measure in lieu of a structured data repository (houghton & mackrell 2014). this exemplifies mutually influential roles which transpire when project participants learn from each other. as student teams progressively develop a database to replace excel spreadsheets, the client managers steadily become aware of technical and business potentials from storage, analysis and retrieval of data. this realisation leads to greater commitment to the project by the clients and the prospect of solution intervention by the researchers. a senior manager stated in an interview, “yeah. look, i suppose the key takeaway for you today is that certainly i'm still quite strongly committed to this project. we've both invested an awful lot in it, as have the students, and we're quite conscious of the need to adapt and be creative and look for better, more productive ways of doing things”. regarding authentic and concurrent evaluation, it has already been identified that evaluation is a key characteristic of this project with evaluation cycles dependent on more than routine ex ante and ex post alpha and beta versions, with artefact development complicated by discrete building phases associated with varying teams. stage 3: reflection and learning the principle which applies at the reflection and learning stage is principle 6: guided emergence, referring to the interplay between guidance through design, and emergence through evaluation. the learning stage relies on evaluation, both humanand technologycentred to encourage reflection and ensure more useful project outcomes. mckay, marshall and heath (2010) argue that technology-centred concepts of dsr have been too limiting, and that human-centred design science should be built-in. the project involves many stakeholders: students, teaching staff, technical experts (university and community), clients, project coordinators, and researchers. all of the actors have a story to tell, not least of all the students as part of their capstone learning experience. mutual learning takes place which is multi-faceted: being technical, social, organizational and educational. while debate continues regarding the role of theory and theorising in dsr (cole et al. 2005; venable 2006), adr affords the capacity to critically reflect on activities, then formulate and utilise theories to embody statements of knowledge for its creative practical activities. this argument is consistent with gregor’s theory for design and action which says “how to do something” giving prescriptions of “methods, techniques, principles of form and function” (gregor 2006, p. 620). australasian journal of information systems mackrell & mcdonald 2016, vol 20, design research for dss development evaluation using action design science 8 gregor et al. (2010) note that dsr draws on theory we already know, otherwise known as kernel theory or justificatory theory. in this case, it is sense-making theory and ‘small wins’ theory. weick (1984) in “small wins: redefining the scale of social problems”, explains that small wins help to create an environment where change is not overwhelming and therefore more likely to happen. in order to solve massive social problems, weick (1984, p.43) defines small wins as a “series of concrete, complete outcomes of moderate importance". the strategy of focusing on “a series of controllable opportunities of modest size that produce visible results”, works directly on the construction of a problem and indirectly on its resolution. once a small win has been accomplished, forces are set in motion favouring another small win. there are important emergent theoretic topics such as user participation (marcus & mao 2004) and is design (markus, majchrzak & gasser 2002), still to be pursued. the study touches on technical design topics such as the open source software options to build it artefacts allowing nfps to enter the bi and data warehouse arena at reasonable cost, applicable to their needs (klawans 2006). stage 4: formalisation of learning the study is a combined venture of connections act and the university of canberra. the situated practice-based learning of both design principles and workplace change will be generalised as a conceptual framework extendable to the not-for-profit sector. the intent at this juncture is to disseminate new knowledge to share through publishing papers, conducting workshops, and instigating organisational change. this activity formalisation of learning complies with principle 7: generalised outcomes, discussed further in the concluding section of this paper. 4 formulating a socio-technical evaluation framework for adr in this section, both social appraisals as the ccp approach, and technical audits as traditional project milestones are discussed in terms of the project, especially in the bie stage. because of the nature of the project, the timing of evaluations is recurrent and iterative, not limited to ex ante and post ante for alpha and beta versions (pries-heje et al. 2008; sein et al. 2011). priesheje et al. (2008) drawing on venable (2006) refer to artificial and natural settings. this is a work-integrated learning (wil) project: wil being an umbrella term for learning activities such as activity-based learning, work-based learning and co-operative education, with attempts to link learning in academic and practice settings. wil is not easily defined because it covers a diverse range of activities from internships and placements to real client projects and other practical, contextualised tasks (fitch 2011). while much software testing takes place in an artificial setting, such as the university, the client is a real organisation which is a natural setting. as well, the opinions of stakeholders emerge as highly significant for shaping both product (bi artefact) and development processes, thus increasing the chances of project realisation. for this project, technical success was broadly defined in terms of releasing to the client functioning and useful software without too many disruptions. social outcomes for the project were never clearly articulated but emerge as the study took form. 4.1 social evaluations: ccp integrated into adr in their paper, stockdale and standing (2006) draw on symon’s (1991) content, context and process (ccp) framework to propose an holistic approach to evaluation in information systems (is). the organisational, political and cultural factors that influence artefact development incorporate the social aspects of appraisals. in other words, the what, why, who, when and how of is evaluation. as explained by stockdale and standing (2006), the ‘what’ in the project lifecycle is represented by the content construct to know what is being measured and which metrics, if any, comprise the evaluation. the element of context requires the ‘why’ and ‘who’ to be considered, these being internal and external organisational influences and various stakeholders. the process construct takes into account the ‘when’ and ‘how’. importantly, symons (1991, p. 211) argues for evaluation to continue throughout the project lifecycle, thus reducing the risk of failure. ccp applied to the case study is discussed below. although ccp australasian journal of information systems mackrell & mcdonald 2016, vol 20, design research for dss development evaluation using action design science 9 constructs are categorised separately, in reality they are intertwined and difficult to pull apart for analysis. 4.1.1 content (what is evaluated) the ‘what’ is both the bi artefact (technical details are explained briefly in the section below on technical testing in adr) and the processes employed to construct the artefact within the organisational settings of university and client, and the means by which assessments are performed. measurements are not generally used for qualitative appraisals. rather, assessments are in the form of interviews conducted by the researchers at regular intervals during the project lifecycle. these interviews are recorded, analysed and interpreted according to the researchers’ frame of reference, and triangulated though student interviews and the reflective log written by the lead researcher. before the start of every semester, the researchers meet with the client, a meeting usually attended by four senior managers one manager from first point, one manager from canfacs, and both the executive officer and business manager from the parent organisation connections act to deliberate over the following: • status of the project: what progress was made over the last semester? what progress is expected over the coming semester? • team-level deliverables: is the scope of the mini-project achievable for each team? have the teams produced the deliverables? once the project had been ongoing for two semesters, the client managers gain confidence in their abilities to comment on the progress of the project. this was welcomed by the researchers. as the business manager stated on behalf of the other managers: even if we don't quite achieve the big vision this semester, it's still a good step on the way i think. i guess there are a couple of things that i'm really looking for an outcome on. that is a better sense of coordination around what the teams are doing, and the second thing is around being clear about what they're actually working on and what they're capable of delivering by the end of this semester. regular meetings with the client are invaluable with decisions made in a participative manner, this being the modus operandi of many nfps (mackrell and van den boogaard 2012). for example, consensus was reached amicably at the meeting held prior to the incoming semester, when the business manager confided: well, at the start of the meeting, i probably would have been hesitant to say yes we'll go with three [mini-projects] but the data warehousing one seems to it will probably require minimal amount of input and engagement from us. so happy to see what the view of others [managers] is on that score. but it seems like it's viable to go with three [mini-projects]. the information gathered at these meetings is triangulated through consultations with student teams and the research log. every semester, new student teams are set up by the subject convenor. the researchers have little control over the composition of the student teams, therefore, previous experience, knowledge and skills of team members are rarely known. meetings with student teams take place a week or two after they are allocated to teams and are assigned their project with the prepared proposals. during the semester, the number of meetings with student teams is directly proportional to the amount of difficulty they are experiencing. that is, the more difficulty, the more meetings with both the researchers and the university technical experts who are mentoring the students. all teams are interviewed for a closure meeting after their final presentation and exam. with their final presentations and exams behind them, students tend to be more honest, contemplative and critical in their assessments. for instance, one student commented: “i think it was a good experience. definitely in terms of stakeholder management and stuff like that. if it was smooth sailing, we probably wouldn't have learned anything …”. australasian journal of information systems mackrell & mcdonald 2016, vol 20, design research for dss development evaluation using action design science 10 another form of qualitative socially-derived evaluation is the research log documented regularly and reliably by the lead researcher. this log is invaluable for verifying names of stakeholders, dates of activities, decisions made at meetings, and so on. together with academic assessments related to software and project progress, as discussed in the section on technical audits below, they form a holistic appraisal of project content. 4.1.2 context (who is evaluated, and why) stockdale and standing (2006) refer to internal and external influences on a project. dealing first with internal influences of the project, the sway of stakeholders, as the ‘who’, was often compelling and conflicting. a dilemma for students was the prioritising of client and academic demands. whom do the students please? one student agonised when waiting for the client to respond to a query expressed this, “so we have two sides to consider, client and university. if we wait for the client and try and reach a solution, we might fail university … we are learning to use pressure to advantage”. the situation emulates complex and competing workplace realities. there is a tendency to forget that incoming teams are also stakeholders. handing over to a new team is facilitated by comprehensive handover documentation and the latest source code from the previous team. additionally, the eo of connections act has recognised the hurdles of transferal and briefs the teams at an induction session early in the semester, soon after new teams have formed. this has proven to be very effective, especially in reducing lag time as new teams settle in and familiarise themselves with the organisation and the technical complexities such as bitcloud (koparkar and mackrell 2015). regarding external influences, an important stakeholder is the act government. strategic proposals by the directorate (act government) to extend first point central intake service for the homeless to other vulnerable citizens such as those with disabilities is ‘in the pipeline’, implying potential for growth. the business manager elucidated: … we are still in a wait and see kind of environment for policy direction around housing and homelessness from the new government. but also because we're currently participating in an initiative by the act government to look at building one human services gateway here in the act … so we manage the system that the central intake system for homelessness but there are also gateways for disabilities, for child youth and family support. this was confirmed by the executive officer of connections act, as such: okay. just in a nutshell my understanding is that the act government is looking at better integrating services but looking at a central intake model for human services more broadly... the dcr is virtually the only successful [software] tool in town that gets notifications of vacancies and places people into them. we do that in homelessness but clearly that could be in … other sorts of human service deliverables. so its applicability to the very core business of what human services is … is becoming more and more valuable. the context in which the project is being developed is the ‘why’ influence. the client is operating in a turbulent environment, dependent on funders for survival; hence, the reference to the dcr as a successful tool is pertinent as connections act managers consider commercialisation of their software in a strategy towards social enterprise, an effort to decrease dependency on tenuous grant funding. 4.1.3 process (when and how evaluation occurs) pries-heje et al (2008) classify evaluation settings as either artificial or naturalistic. artificial evaluations occur in a contrived, non-realistic manner, such as simulations and software testing. natural evaluations explore performance to solve real problems in a real environment such as an organisation. the students tested their software artefacts in both settings, at the university and at the client’s premises to demonstrate functionality and achieve client acceptance sign-off. australasian journal of information systems mackrell & mcdonald 2016, vol 20, design research for dss development evaluation using action design science 11 during the lifetime of the project, there are as many iterations of development as there are semesters. the researchers determined it was essential to conduct evaluative interviews at important intervals. these evaluations are pre-semester and normative (ex ante), and post semester and summative (ex post). processes to be adopted during the semester are discussed at the pre-semester meeting with the client managers. further meetings with the client take place at the end of semester, and at any critical points during the semester. occasionally, this has resulted in some very frank discussions, as evident in the dialogue below, from the business manager. that with the benefit of hindsight it was probably a bit too ambitious, that it probably could have done with a greater sense of coordination, certainly early in the semester. probably the handover from the semester one student teams wasn't done as effectively as it could have been done. there were some issues around the process and our availability and connections when the students needed to talk to us. process evaluations are fundamental for shaping the products and conduct of the project. it is inevitable that all stakeholders, researchers, developers and clients, err occasionally. adjustments need to be made reflexively. this has the effect of building a more robust community of practice for the discussion of issues. the business manager, speaking on behalf of the other managers, said: all of this work fits within the broader scheme of things for us. we need to adapt and change or there won't be a connections act in the future. so you know all the work that we're doing, not just this piece of work, but other things that we're doing around social enterprise initiatives, common ground, are looking for other sources of more diversified funding. the small wins approach was encouraged during the project to give the client confidence in the ability of teams to produce a working solution. as disclosed by a team project manager, “if we gave her [client manager] a database right at this time, she would have nothing that she could actually use. that’s why i put in the reporting part …”. at the start of semester, each student team was provided with specifications articulating essential and desirable deliverables. nevertheless, some leeway was encouraged so that each team could account for its strengths and weaknesses. the concept of small wins aligns well with adr and its attention to evaluation and intervention, with the incremental building of it artefacts through iterations (weick 1984). 4.2 technical testing in adr maccani et al. (2015, p.5) subscribe to both positivist and anti-positivist assumptions underpinning adr philosophy. design processes of bie lean towards anti-positivism, being highly participatory and socially constructed. methodologies for building and testing of the artefact are more prescriptive, although not discussed in any great depth in this paper. software testing is standard for any project to demonstrate the worth of the artefact through pre-determined criteria such as utility, efficacy, validity and quality (gregor and hevner 2013). evidence of software quality is contained in written specifications and documentation such as project charters, models of entity-relationship diagrams (erds), use case diagrams, testing schedules etc as well as in student team reports and presentations, as mandated by academic requirements. much of this documentary evidence is built and evaluated in an artificial environment prior to client acceptance in a naturalistic organisational setting where every effort is made to involve the end-user client and researcher throughout the process. 5 discussion of findings the adr approach as presented in table 1 was adopted for the project. adr offers a means of bridging the conceptual gap between dichotomies of social versus technical, informal versus formal evaluations, and artificial and naturalistic, and thereby overcoming concerns regarding philosophical disparities (iivari & venable 2007). ccp-driven evaluations have contributed to producing the holistic product-process insights of an ensemble artefact. by integrating ccp australasian journal of information systems mackrell & mcdonald 2016, vol 20, design research for dss development evaluation using action design science 12 social evaluations and technical audits into the adr approach, the bie generic schema described by sein et al. (2011) in figure 1 is no longer representative of the project developed as a non-commercial undertaking for a nfp. an extended version of sein’s bie generic schema in figure 1 is modelled in figure 2 with insights from our study. in the following text, the number in parenthesis followed by dot points loosely corresponds to annotations in figure 2. (1) context (why and who is evaluated) • changing context influences the motivation of all stakeholders, for example, the client is keen to develop a social enterprise culture for the nfp • act government as a stakeholder sets regulatory pressures • executive officer is the project champion and vital to the project • client managers and users are influential stakeholders • incoming student wil team is considered a stakeholder • university and community mentors and experts are stakeholders (2) content (what is evaluated) • technical and social evaluations required to improve project’s chances of success by shaping the project closer to the needs of stakeholders. evaluations are instrumental to inform learning and knowledge through critical reflection. • strategic goal of project is clear however precise aims are not distinct since working from an aspirational top-down rather than a grass roots bottom-up approach • progress of mini-projects towards project aims is incremental, dependent on many variables such as learning, skills and abilities of student teams, much of which is unknown before the start of each semester (3) process (when and how does evaluation occur) • iterative development periods for it artefacts has discrete, short time periods of one semester • technical design, building and evaluation of it artefacts is embedded into academic assessments (4) research outputs (research contributions, design principles, ensemble artefact, interventions) • client takes a cautious approach, aware of student team limitations. concept of ‘small wins’ by teams is important to build client confidence • procedures help to mitigate handover delays at start of each semester with documentation on status of project from previous student team and instructions on bitcloud access • induction session are arranged by client executive officer for each new team • regular and recurring contact with client by teams is planned • use of open-source software reduces costs • interventions occur in both educational and organisational practices in summary, socio-technical evaluations conducted within the adr framework have revealed unexpected themes, such as additional iterations, careful stakeholder management, and design initiatives to be included in a bie generic schema as design guidelines for community projects. these have contributed to shaping the project to suit the needs of all stakeholders: the client connections act, wil students, and the researchers. the executive officer communicated his gratification with the words “… a shared commitment, you're sticking with us, we're sticking with you. we're learning as we're going along.” australasian journal of information systems mackrell & mcdonald 2016, vol 20, design research for dss development evaluation using action design science 13 figure 2: the adr generic schema for socio-technical bie (adapted from sein et al. 2011) incorporating ccp as it applies to the case study in the areas of 1. context eg stakeholder evaluations 2. content eg socio-technical evaluations 3. process eg supplementary iterations 4. research outputs eg research contributions, product-process design insights as indicated at the start of this paper, key issues of relevance, evaluation, theorising, and strategic focus have been identified as neglected and unresolved in dss research (arnott & pervan 2012; miah & gammack 2014). human activity is complex. case study research necessarily takes a particular stance, as in this case study by the provision of a relevant naturalistic setting. adr turned out to be suitably pragmatic, capturing organisational and technical aspects of the intervention in a ‘meta-approach’ to evaluation. however, no stance is isolated. it resonates with other approaches and these are worth indicating. firstly, over the last decade there has been an evolution in thinking about design research and it now presents a complex picture. the adr framework from sein et al. (2011) was selected as the basis of this research, but is one of many that could have been chosen. one of the reasons for its selection was its recognition of the process of artefact development, not just products, through cycles of evaluation to realise an ensemble artefact. in an era of agile development, the sein framework may seem rather traditional yet the research reported here suggests that the agile development of artefact components, independent of the involvement of all stakeholders, misses out on some significant benefits that can accrue along with systems development. in particular, this research shows that stakeholder-engaged project development can have much wider impacts in terms of improved systems literacy within an organisation, and increased technological engagement, that attitudes change, from systems being seen as necessary evils to them being positive contributors to the organisation. this was unexpected as the systems being developed were 'back office systems' which are usually recognised prosaically in terms of efficiency and effectiveness. secondly, there are interwoven design trends and conceptual themes that complement the findings of this paper, for example, value sensitive design (yoo, huldtgren, palzkill woelfer, hendry and friedman 2013) and user experience design (adikari, mcdonald and campbell 2013). in parallel with sein et al. (2011), these themes, in their own ways, shift the focus of ict evaluation from the product's behaviour to its relationship with stakeholders. in future, a coalition of research approaches sharing a similar approach may yield greater benefits than single approaches alone. australasian journal of information systems mackrell & mcdonald 2016, vol 20, design research for dss development evaluation using action design science 14 thirdly, research is often conceived as being for the researcher in their search for knowledge accumulation. however 'transformative research' (for example, that advocated by malhotra, melville, and watson 2013) promotes the benefits possible for all stakeholders. in this case, while the researchers are producing results for the research community, the organisation is benefitting greatly (through facilities provision and shifts in attitude towards greater engagement with their own systems) as are the students (through authentic development experience). this project is having transformative affects and is showing emergent gains beyond the initial goals of stakeholders, especially attitudinal and educational for all participants. lastly, the nfp sector’s use of bi as a strategic focus is of importance from many different perspectives. the number of people and devices interconnected is set to grow massively in the next few years. all this data is creating an opportunity for bi to become essential in the decision-making process with the aim of improving the outcome of these decisions. crump and raja (2013) report on findings taking a different technology approach, that of shared services for nfps. they are interested in evaluating organisational readiness to assess and adopt different kinds of ict systems an approach that would complement the research reported here. 6 concluding remarks this paper proposes iterative and holistic stakeholder evaluations through integration of content, context and process (ccp) appraisals into action design research (adr) to complement more prescribed technology-driven project and software testing. the ccp framework focusses on qualitative human-focussed evaluation and is useful in providing both descriptive knowledge about natural phenomena as well as prescriptive knowledge (gregor and hevner 2013) through design principles. this socio-technical method of evaluation meshes well with ensemble artefact design and development such as bi software. sein et al. (2011) recognise that a case study using adr is highly situated, offering the potential of theorising from the specific and unique to the generic and abstract. from the outset, the project being analysed in this paper had dual purposes to make a difference in the case study site, connections act, and to use the experience to create a set of analytic, design, intervention and evaluation guidelines for general use in the not-for-profit sector. this paper is a step towards those dual purposes. sein et al. (2011, p.45) argue that “design principles capture the knowledge gained about the process of building solutions for a given domain, and encompass knowledge about creating other instances that belong to this class”. a number of design principles are emerging from the case study that apply generally. some examples include the use of open source software and cloud warehousing, project development with wil students supported by community experts and facilitated by academic researchers, ccp evaluations for qualitative stakeholder appraisals, and the concept of small wins, akin to gregor et al.’s ‘sweet spot’ (gregor et al. 2010). the third purpose is to extend adr knowledge through empirical research. the quadrant which is most apt is exaptation: known solutions extended to new problems. gregor and hevner (2013, p.347) describe this as “… contributions where design knowledge that already exists in one field is extended or refined so that it can be used in some new application area”. the adr generic schema for bie (sein et al. 2011) illustrated in figure 1 is intended for projects in commercial settings. this paper extends the model into the nfp sector with multiple iterations for wil student-built it artefacts. the paper does not articulate minutiae of artefact design, stakeholder participation and artefact appropriation. these are the domains of other papers. in light of our adr study set in a nfp, we now put forward an alternative definition of evaluation than the one from cleven et al. (2009) cited earlier. this definition incorporates the evaluation view of an ensemble artefact: australasian journal of information systems mackrell & mcdonald 2016, vol 20, design research for dss development evaluation using action design science 15 evaluation is the process of determining the quality of the design and development of an ensemble artefact, being context, content and the process itself. evaluations may be social or technical, formal or informal, qualitative or quantitative or a mix of these. along with interventionist practices, critical reflections and learning are outcomes of the evaluative process. references adikari, s., mcdonald, c. and campbell, j. (2013) "agile user experience design: a design science enquiry" proceedings of the 24th australasian conference on information systems (acis), rmit melbourne vic. arnott, d. and pervan, g. (2005) “a critical analysis of decision support systems research” journal of information technology, 20(2), 67–87. arnott, d. and pervan, g. (2008) “eight key issues for the decision support systems discipline” decision support systems, 44(3), 657–672. arnott, d. and pervan, g. (2012) “design science in decision support systems research: an assessment using the hevner, march, park, and ram guidelines” journal of the association for information systems, 13(11), 923–949. baskerville, r. (2008) “what design science is not” european journal of information systems, editorial, 17, 441–443. cleven, a., gubler, p. and hűner, k. (2009) “design alternatives for the evaluation of design science research artifacts” proceedings of the international conference desrist 2009, malvern pa usa. cole, r., purao, s., rossi, m. and sein, m. (2005) “being proactive: where action research meets design research” proceedings of the 26th international conference on information systems (icis), las vegas usa. crump, b. and raja, p. (2013) "a case for non-profit organisations to engage in the use of shared computing services" electronic journal of information systems evaluation, 16(4), 270-277. davison, r. m. and martinsons, m. g. (2004) “principles of canonical action research” information systems journal, 14(1), 65-86. fidock, j., carroll, j. and rynne, a. (2010) “evaluating information systems: an appropriation perspective” in d. hart and s. gregor (eds) information systems foundation: the role of design science, anu epress, canberra act. fitch, k. (2011) “developing professionals: student experiences of a real-client project” higher education research & development, 30(4), 491-503. goldkuhl, g. (1981) “systemering som formalisering av människors språk – en humaninfologisk teori” [in swedish] norddata81, copenhagen. goldkuhl, g. (2012) “design research in search for a paradigm: pragmatism is the answer” in m. helfert, and b. donnellan (eds) practical aspects of design science – proceedings edss2011, springer, berlin. goldkuhl, g. (2013) “from ensemble view to ensemble artefact an inquiry on conceptualizations of the it artefact” systems, signs & actions, 7(1), 49-72. goldkuhl, g. and sjöströmb. j. (2015) “closing the practice loop: practice design research” proceedings of the ais sigprag pre-icis workshop 2015 practice-based design and innovation of digital artifacts. gregor, s. (2006) “the nature of theory in information systems” mis quarterly, 30(3), 611642. australasian journal of information systems mackrell & mcdonald 2016, vol 20, design research for dss development evaluation using action design science 16 gregor, s. and hevner, a. (2013) “positioning and presenting design science research for maximum impact”, mis quarterly. 37(2), 337-355. gregor, s., imran, a. and turner, t. (2010) “designing for a ‘sweet spot’ in an intervention in a least developed country: the case of e-government in bangladesh” proceedings of sig globdev 3rd annual workshop, st louis usa. hevner, a., march, s., park, j. and ram, s. (2004) “design science in information systems research” mis quarterly, 28(1), 75-195. houghton, l. and mackrell, d. (2014) “sensemaking as feral information systems: conceptual and framework development” in l. houghton and d. kerr (eds) feral information systems, idea group inc. hershey pa usa. iivari, j. and venable, j. (2009) “action research and design science research – seemingly similar but decisively dissimilar” proceedings of the 17th european conference in information systems (ecis). verona italy. järvinen, p. (2007) “action research is similar to design science” quality & quantity. 41, 3754. klawans, b. (2006) “saving time and money why open-source bi makes sense” business intelligence journal. 11(4), 18-24. koparkar, p. and mackrell, d. (2015) “how fluffy is the cloud ?: cloud intelligence for a notfor-profit” proceedings of the 26th australasian conference on information systems (acis), university of south australia, adelaide sa. maccani, g., donnellan, b. and helfert, m. (2015) “action design research: a comparison with canonical action research and design science” in b. donnellan, r. gleasure, m. helfert, j. kenneally, m. rothenberger, m. chiarini tremblay, d. vandermeer, and winter, r. (eds.) at the vanguard of design science: first impressions and early findings from ongoing research, research-in-progress papers and poster presentations from the 10th international conference desrist 2015. dublin ireland, 20-22 may. 69-76. mckay, j. and marshall, p. (2001) “the dual imperatives of action research” information technology and people, 14(1), 46-59. mckay, j., marshall, p. and heath g. (2010) “an exploration of the concept of design in information systems” in hart & gregor (eds). information systems foundation: the role of design science, anu epress, canberra act. mackrell, d. (2012) “delivering optimal health outcomes: making sense of a business intelligence decision in a not-for-profit organisation” proceedings of the pacific asia conference on information systems (pacis). ho chi minh city vietnam. mackrell, d. and mcdonald, c. (2014) “action design research: a case study of business intelligence in non-profit organizations” proceedings of the sigdss ifip wg 8.3 international conference, university pierre and marie curie (upmc) paris. mackrell, d. and van den boogaard, m. (2012) “socio-technical framework for improved decision-making in not-for-profit organisations” proceedings of the 23rd australasian conference on information systems (acis). deakin university, geelong vic. malhotra, a., melville, n. and watson, r. (2013) "spurring impactful research on information systems for environmental sustainability" mis quarterly, 37(4), 1265-1274, december. march, s. and smith, g. (1995) “design and natural science research on information technology” decision support systems, 15(4), 251-266. markus, m., majchrzak, a. and gasser, l. (2002) “a design theory for systems that support emergent knowledge processes” mis quarterly, 26(3), 179-212. australasian journal of information systems mackrell & mcdonald 2016, vol 20, design research for dss development evaluation using action design science 17 markus, m. and mao, j. (2004) “participation in development and implementation – updating an old, tired concept for today’s is contexts” journal of the association for information systems, 5(11-12), 514-544. miah, s. and gammack, j. (2014) “ensemble artifact design for context sensitive decision support” australasian journal of information systems, 18(2), 5-20. orlikowski, w. j. and iacono, c. s. (2001) “desperately seeking the “it” in it research – a call to theorizing the it artefact” information systems research, 12(2), 121-134. papas, n., o’keefe, r. and seltsikas, p. (2012) “the action research vs design science debate: reflections from an intervention in egovernment” european journal of information systems, 21(2), 147–159. pries-heje, j., baskerville, r. and venable, j. (2008) “strategies for design science research evaluation” proceedings of the european conference on information systems (ecis), galway ireland. purao, s., henfridssonb, o., rossi, m. and sein, m. (2013) “systems ensemble artifacts: from viewing to designing in action design research” signs & actions, 7(1), 73–81. scriven, m. (1998) “the new science of evaluation” international journal of social welfare, 7(2), 79-86. sein, m., henfridsson, o., purao, s., rossi, m. and lindgren, r. (2011) “action design research” mis quarterly, 35(1), 35-56. sharda, r., delen, d. and turban, e. (2014) business intelligence and analytics systems for decision support, 10th edition, pearson. stockdale, r. and standing, c. (2006) “an interpretive approach to evaluating information systems: a content, context, process framework” european journal of operational research, (173), 1090-1102. symons, h. (1991) “a review of is systems evaluation: content, context and process”. european journal of information systems (ejis), 1(3), 205-212. venable, j. (2006) “the role of theory and theorising in design science research” proceedings of the 1st international conference on design science (desrist). claremont ca usa. weick, k. (1984) “small wins: redefining the scale of social problems” american psychologist, 39(1), 40-49. weick, k. (1995) sense-making in organizations, sage publications, thousand oaks ca. winter, r. (2008) “design science research in europe” european journal of information systems, 17, 470-475. yoo, d., huldtgren, a., palzkill woelfer, j., hendry, d. and friedman, b. (2013) "a value sensitive action-reflection model: evolving a co-design space with stakeholder and designer prompts" proceedings of the sigchi conference on human factors in computing systems (chi '13), 419-428. copyright: © 2016 mackrell & mcdonald. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems mackrell & mcdonald 2016, vol 20, design research for dss development evaluation using action design science 18 1 introduction 2 explaining action design research 2.1 views on evaluation 2.2 where does the ccp model of evaluation fit? 3 illustrating adr with a case study 3.1 research site 3.1.1 research approach 4 formulating a socio-technical evaluation framework for adr 4.1 social evaluations: ccp integrated into adr 4.1.1 content (what is evaluated) 4.1.2 context (who is evaluated, and why) 4.1.3 process (when and how evaluation occurs) 4.2 technical testing in adr 5 discussion of findings 6 concluding remarks paper title (paper title style) australasian journal of information systems pye, holt & salzman 2018, vol 22, research on educational technologies student engagement with blended learning environments 1 investigating different patterns of student engagement with blended learning environments in australian business education: implications for design and practice graeme pye deakin university graeme.pye@deakin.edu.au dale holt deakin university scott salzman deakin university abstract this research reports on a diverse student audience engaging in an australian university’s undergraduate commerce program core unit which was offered across three separate geographic campus locations and online. the research extends upon work undertaken on student engagement in online settings and lies in the domain of blended learning design and practice in the australian higher education business context. the research adopted a dualperiod surveying approach aimed at comparing patterns of student engagement within a major business subject offered in two different trimesters in one academic year. the survey analysis compared student perceptions of engagement with the subject and identified significant variations in patterns of engagement based on key background and demographic factors. findings, inter alia, are presented across seven major student engagement dimensions as applied to the interplay between online and located/campus learning (i.e. online active learning, online social interaction, online collaboration, online teaching, online assessment, online relevance, and online contact with staff). implications for blended learning design, elearning and practice in such complex environments involving diverse student audiences are examined. keywords: student engagement; blended learning; australia; business education 1 introduction this research represents the initial findings of a multi-year project investigating student experiences of learning in contemporary blended learning environments in the field of business education. the project is based on developments in a large faculty of business and law in an australian university that operates across three campuses in three separate cities in metropolitan, regional and rural settings in the australian state of victoria. australian and international universities continue to strategically seek competitive advantage by exploiting information and communication technologies (ict) to service the needs of a diverse population of students, irrespective of their study location, through extending the options for flexible education, distance education, elearning and blended learning. the site of this research, deakin university, has committed to the aspiration of pushing the digital frontier, with its accompanying ‘cloud’ (online) and ‘located’ (campus-based) learning vision. this vision is based on the university’s historical commitment to distance and online education (for major educational technology developments within the institution see challis, australasian journal of information systems pye, holt & salzman 2018, vol 22, research on educational technologies student engagement with blended learning environments 2 holt, & rice, 2005; holt & challis, 2007; palmer & holt, 2014). the research project aims to enhance the quality of business education in australian higher education by enlightening the powerful and dynamic interplay of student learning online and on location by: developing, piloting and refining a student engagement survey instrument related to cloud and located learning; ascertaining the nature and extent of student engagement in such blended learning environments; and developing a set of design principles and practices to enhance student engagement. the student engagement survey data collection is located within an educational design research orientation to advancing contemporary blended learning designs and practices through an iterative process of learning design principles interacting with practical interventions through evidence-driven approaches. the research focuses on ubiquitous learning management system (lms) environments only and not the specific educational technology field of gaming and simulation spaces, including, for example, student engagement in virtual 3-d platforms like second life (see holt, segrave & cybulski, 2011). this paper focuses on findings and implications from the summary results of a student engagement survey to students enrolled in two major offerings of a large, first year foundational disciplinary subject on business information systems offered in the faculty’s commerce program. the unit was offered on location across three campuses and online via the cloud. the survey is based on significant international work reported in the higher education literature relating to the key dimensions of student engagement in online learning environments. the subject had undergone significant change in response to the university’s strategically directed cloud and located learning course enhancement processes. the research highlights both commonalties and differences in patterns of student engagement based on key demographic and background data. gender, age and cultural differences are all highlighted. the findings contribute to the better balance of research undertaken on students as opposed to staff experiences of blended learning and teaching, along with foregrounding major design and practice implications. 2 background various lines of conceptual framing have been developed to promote students learning online (garrison & anderson, 2003; bonk & zhang, 2008; salmon, 2011). the current research aims to examine student perceptions of their experiences of evolving online and blended learning environments through the conceptual framing of student engagement. this research contributes to the large body of work on promoting student learning in online and blended learning settings in two important respects. first, it adds to the research undertaken on student experiences of blended learning and teaching. second, the aim is to investigate students’ views on how they are experiencing engagement, based on a strong line of higher education research in this field and to extend these investigations into the particular field of students engaging with contemporary blended learning environments. student engagement focuses on the time and effort students put into purposively designed educational activities directed at achieving the desired learning outcomes (coates & mccormick, 2014). the student engagement enterprise synthesises significant bodies of knowledge on the student experience in higher education in key fields covering quality of effort (pace, 1979; 1995), principles for good practice in undergraduate education (chickering & gamson, 1987), student learning involvement principle (astin, 1985) and student integration into university life and persistence in study (tinto 1993; 1997). australasian journal of information systems pye, holt & salzman 2018, vol 22, research on educational technologies student engagement with blended learning environments 3 the phenomenon of student engagement has been well investigated and developed in the united states (kuh, 2003; kuh, kinzie, schuh, whitt & associates, 2005). the student engagement enterprise and its empirically-driven approach as represented in the national survey of student engagement (nsse) originated in the us in the late 1990s and spread with major adaptations and applications in canada, australia and new zealand, south africa, china and ireland (coates & mccormick, eds., 2014) and the united kingdom (howson & buckley, 2017). while a powerful vehicle for illuminating the realities of the students’ learning experience for quality assurance and improvement purposes, it has not been beyond critique in the australian context (hagel, carr & devlin, 2012). drawing on the various bodies of knowledge and survey methodology relating to student engagement more generally, coates (2006) investigated how learning technologies, as particularly prominent through the large-scale implementation of lmss throughout higher education, were impacting campus-based student engagement. citing coates (2006, p.69), student surveys are structured conversations between learners and universities. taking account of the more general set of dimensions constituting student engagement, coates (2006) developed a set of student engagement constructs for online learning environments (as related to online learning engagement in campus-based located settings). those used in this study cover: • online active learning: key ways in which students use online systems to enhance learning; • online social interaction: use of online learning systems to support general forms of social interaction; • online collaboration: students’ use of online systems to work collaboratively with peers; • online teaching: whether teachers use online learning systems to promote effective learning; • online academic relevance: use of online systems to enhance the relevance and context of study. • online contact with staff: extent and ease of use of online systems by students in contacting staff. this set of dimensions are directed at discovering what students self-report that they actually do online, as opposed to what teachers might think they should be doing. coates’ approach elicits student views on what and how much they have done of relevance to their academic pursuits. not foregrounded explicitly in a separate construct by coates (2006) was online assessment encompassing the various lms features, functions and approaches which might enable stronger student engagement with assessment tasks and feedback in online and blended learning settings. the assessment features of lmss are seen as potentially key benefits in supporting teaching and student learning online (bennett, dawson, bearman, molloy & boud, 2017), particularly relating to online assessment feedback (see parkin, hepplestone, holden, irwin & thorpe, 2012; jonsson, 2014; dawson, 2017). more generally, assessment review and redesign involving a major focus on more effective assessment feedback is seen as a compelling means of enhancing the student learning experience in higher education (boud, 2010; nicol, 2010; boud & molloy, 2013; henderson, boud, molloy, et al., 2016; small & attree, australasian journal of information systems pye, holt & salzman 2018, vol 22, research on educational technologies student engagement with blended learning environments 4 2016; winstone nash, rowntree & parker, 2017). redesigning assessment has been a key activity in reviewing and renewing curriculum design and delivery in australian higher education, as advanced by senior university educational leadership (see oliver, 2013; lawson, 2015). a discrete construct relating to online assessment was seen as an important addition to those drawn from coates’ (2006) work. these constructs of online student learning inform the development of a student engagement survey for cloud and located learning in this research relating to a business education context. as reported by pye, holt, salzman, bellucci and lombardi (2015), certain statistically significant differences were detected within and across the online student engagement constructs in the first offering of the business information systems subject as related to key background and demographic factors, i.e. gender, age, mode of enrolment, culture. this research further investigates the existence of such differences. coates (2006, p.11) set aside the investigation of differences in patterns of student engagement, and a focus on whether or not such differences exist, and the reasons why this might be the case. this research is an important extension of his work, particularly relating to potential differences in student engagement based on key background and demographic factors like culture and gender. culture and gender are major areas of study in higher education. culture has figured prominently given the substantial number of international students who study in australia, particularly those drawn from asia, and china specifically. cultural differences have been illuminated relating to chinese confucian and pedagogical traditions and practices, and their intersections with western higher education approaches (choi & nieminen, 2013). these concerns have been extended into cultural factors pertaining to online learning (chen, bennett & maton, 2008; chen, hsu & caropreso, 2006; liu, liu, lee & magjuka, 2010). liu et al. (2010, p.179), citing zhang (2007), observe that: “numerous studies that have examined the cultural differences between western and eastern education have provided a consistent picture that describes eastern education as a group-based, teacher-dominated, centrally organized pedagogical culture with examinations as the essential way to define performance and compete for higher social status.” often cited is the seminal work of hofstede (2001) and his identification of different dimensions of culture, namely; power distance (the degree of acceptance of unequal power relationships); individual-collectivism; uncertainty avoidance; and masculinity-femininity. while not without its critics, it is a helpful way of framing possible explanations of differences in patterns of international/domestic student engagement online reported in this study. gender factors have also received attention in relation to engagement with the internet in general, particularly ict educational research. gender differences relating to levels of online anxiety, self-efficacy in teaching using computers (wong, teo & russo, 2012; teo, fan & du, 2015), learning styles in the computer sciences (lau & yuen, 2010), ict literacy (markauskaite, 2006) and power inequalities, conflict and harassment (li, 2005) have, for example, been reported. gender role orientations to interacting with the internet have been examined from a broader social psychological perspective (wallace, 1999) and highlights how unhelpful gender stereotypes can play out online. closer to this study is work reported on gender issues in online learning (mcsporran & young, 2001; gunn, french, mcleod, mcsporran & conole, 2002). these studies present often conflicting evidence around gender (in)equality online, and whether, in fact, in-person, face-to-face gender power inequalities are just socially australasian journal of information systems pye, holt & salzman 2018, vol 22, research on educational technologies student engagement with blended learning environments 5 reconstructed online. however, their main finding is that inclusive, universal design of online learning environments is the best way forward to ensuring fair and reasonable treatment and positive learning for all categories of students. gender in this study is investigated in the context of student engagement with blended learning, and represents a somewhat different perspective. a study of closer relevance to this investigation is that by tsai, liang, hou & tsai (2015). this work is returned to later in this paper when findings are discussed and implications examined. 3 research methodology this investigation is situated within the realm of educational technology research relating to design-based research. this domain of research activity develops and receives considerable attention in international educational technology communities (kopcha, schmidt & mckenney, 2015; reeves, 2015). mckenney and reeves (2012, p.7) define educational designbased research: “…as a genre of research in which the iterative development of solutions to practical and complex educational problems also provides the context for empirical investigation, which yields theoretical understanding that can inform the work of others.” the business subjects researched as part of this overall study, including the one which is the focus of this paper, have all undergone major course enhancement as part of the university’s commitment to review and renewal of its courses. course enhancement has covered all aspects of curriculum design, delivery and assessment. units surveyed continue to review and refine their blended learning design approaches as stimulated by the university’s course transformation agenda. informed by contemporary developments in graduate attribute development, authentic assessment and cloud/located learning, student engagement research is seen as a major means of evaluating student learning experiences to date, and to help inform further development of the blended learning designs in action. the twin pillars of student engagement and design-based research form a robust methodology for advancing theory and practice. 3.1 key research questions for informing design and practice analysis of the literature on student engagement, online student engagement and the demographic and background factors identified as potentially shaping patterns and variations in online student engagement, led to the formulation of two major research questions and four associated hypotheses which form the basis of survey analysis and interpretation. the survey instrument was designed to include the key dimensions of student engagement, and to allow commonalities and differences in patterns of student engagement to be analysed by the key background and demographic factors. the importance of the differences was considerably highlighted in the actual undertaking of the analytical work with the data set, and so the investigation has been partly grounded in nature with lines of research inquiry being responsive to the actual results of the data analysis. the research questions and hypotheses are: • rq1: were there any significant differences in the ways responding students engaged with the subject over different trimester period offerings? at face value, it was not expected to find significant differences given the subject design and australasian journal of information systems pye, holt & salzman 2018, vol 22, research on educational technologies student engagement with blended learning environments 6 delivery was essentially unchanged for both offerings, unless significant differences were identified in the actual student cohorts undertaking the subject in the different teaching periods. • rq2: in analysing student engagement data over the two subject offerings, did any significant differences emerge in student engagement based on key demographic and background factors? • hypothesis 1: responding males and females should rank similarly with respect to their overall level of agreement with each of the different construct themes and each of the items associated with each theme. • hypothesis 2: both located (on-campus) and cloud (off-campus) participating students should rank similarly with respect to their overall level of agreement with each of the different construct themes and each of the items associated with each theme. • hypothesis 3: both domestic and international responding students should rank their overall level of agreement with each of the different construct themes similarly and each of the items associated with each theme. • hypothesis 4: students of different age groups who responded should rank their overall agreement with each of the different construct themes similarly and each of the items associated with each theme. finally, if significant differences can be identified, what implications would such findings have for designing and practicing within such blended learning environments? 3.2 online student engagement survey design this research uses a survey design to ascertain student views on how, and how much, they are engaging with their collective cloud and located learning environments. the online survey method utilised gave the most number of students the best opportunity to share their views on what they have each personally experienced. drawing on an already developed, trialled and validated student engagement survey related to online learning by coates (2006) outlined above, the survey developed for this research is relevant to the experiences to be elicited, and efficient to administer with large student cohorts. the student engagement survey used the six key constructs identified by coates (2006) and associated activities. a seventh student engagement dimension was added relating to various activities associated with online assessment. in addition to the seven constructs of student engagement (see appendix), several other pertinent factors were surveyed as summarised in table 1. australasian journal of information systems pye, holt & salzman 2018, vol 22, research on educational technologies student engagement with blended learning environments 7 other survey factors: • background information: gender, primary campus of study, age, student status (domestic or international); • accessing and downloading learning resources to mobile devices; • use of social media for social/collaborative purposes; • effectiveness of integration of cloud (online) and located (campus) environments (on-campus students only); • amount of time studying unit on average per week; • achievement of unit learning outcomes; • amount of time spent studying unit online; • amount of study time spent studying unit away from campus (on-campus students only); • learning benefits of online environment; • most and least effective aspects of unit blended learning environment. table 1: other survey factors questions relating to use of social media for social/collaborative purposes, and most and least effective aspects of the subject’s blended learning environment were open-ended. given space constraints, the qualitative analysis of the large number of responses to these items is not within the scope of this paper. 3.3 data collection and response rate the commerce program core unit, business information systems was surveyed in trimester 1 and 2, 2014, using the university’s online surveying tool, opinio. in trimester 1, the number of opinio invitee emails was 1348. minus the three survey administrations parties (two team members & sis group), the total number of student invitees was 1345. the self-selected sample was collected with a response rate of 10.26% for trimester 1, 2014 (n = 138): population sample response rates (1345) response rates (138) invitees (totals): 1345 138 10.26% invitees (survey completions): 1345 104 7.73% 75.36% invitees (survey incompletes): 1345 34 2.53% 24.63 total: 138 10.26% 99.99% table 2: trimester one, 2014 response rates comparisons between sample and population demographic and background factors for the trimester 1 offering of the subject were: australasian journal of information systems pye, holt & salzman 2018, vol 22, research on educational technologies student engagement with blended learning environments 8 students sample population no. of respondents 104 1345 response rate (completions) 7.73% — gender female 44 (42%) 503 (38%) male 60 (58%) 842 (62%) mode of enrolment located/on-campus 83 (80%) 1194 (89%) cloud/online/off-campus 21 (20%) 151 (11%) campus melbourne burwood 60 (58%) 898 (67%) geelong waterfront 15 (14%) 254 (19%) warrnambool 8 (8%) 42 (3%) cloud/online/off-campus 21 (20%) 151 (11%) domestic/international domestic 95 (91%) 1218 (91%) international 9 (9%) 127 (9%) age 18-21 71 (68%) 1001 (74%) 22-25 8 (8%) 223 (17%) 26-30 6 (6%) 54 (4%) 31-36 12 (11%) 41 (3%) 37-42 1 (1%) 13 (1%) above 42 6 (6%) 13 (1%) table 3: trimester one, 2014 comparison between sample and population demographics in trimester 2, for the subject, the number of opinio invitee emails was 516. minus the two survey administrations parties (two team members), the total number of student invitees was 514. the self-selected sample was collected with a response rate of 8.17% for trimester 2, 2014 (n = 42): population sample response rates (514) response rates (42) invitees (totals): 514 42 8.17% invitees (survey completions): 514 35 6.81% 83.33% invitees (survey incompletes): 514 7 1.36% 16.67% total: 42 8.17% 100% table 4: trimester two, 2014 response rate comparisons between sample and population demographic and background factors: australasian journal of information systems pye, holt & salzman 2018, vol 22, research on educational technologies student engagement with blended learning environments 9 students sample population no. of respondents 42 514 response rate (completions) 8.17% — gender female 17 (40%) 265 (52%) male 25 (60%) 249 (48%) mode of enrolment located/on-campus 27 (64%) 396 (77%) cloud/online/off-campus 15 (36%) 118 (23%) campus melbourne burwood 26 (62%) 396 (77%) geelong waterfront 1 (2%) warrnambool cloud/online/off-campus 15 (36%) 118 (23%) domestic/international domestic 30 (71%) 338 (66%) international 12 (29%) 176 (34%) age 18-21 21 (50%) 233 (45%) 22-25 9 (21%) 175 (34%) 26-30 6 (14%) 64 (12%) 31-36 3 (7%) 19 (4%) 37-42 2 (5%) 16 (3%) above 42 1 (3%) 7 (2%) table 5: trimester two, 2014 comparison between sample and population demographics the survey system saves responses after each page is completed, so some partially completed responses exist. in the following sections, the maximum applicable number of useful responses is used, and that number indicated. 4 data analysis and findings all data was managed using microsoft xl 2013 and analysed using xl statistics version: 09.09.24. it is noted the inherent bias often present in self-selecting, online surveys implies that it is possible that some groups might be disproportionately represented. conclusions therefore relate to self-reported levels of agreement. all error considered is representative of a 95% confidence interval and all hypotheses tests use critical alpha set at 5%. the pearson’s chisquare test was used to test if two sets of categorical data were independent. to assess for differences between two means, the t-test was used. testing for a difference between more than two means required the use of the one way analysis of variance test and where necessary, post hoc tests were assessed using a t-test. no corrections were made for possible family-wise error. where assumptions required for any statistical analysis could not be met, summary statistics only are presented. to allow for simpler explanations, some data was recoded. deakin university’s lms is branded ‘clouddeakin’. all references to clouddeakin below are to the university’s lms and its various features, functions and uses. australasian journal of information systems pye, holt & salzman 2018, vol 22, research on educational technologies student engagement with blended learning environments 10 research question 1: were there any significant differences in the ways students engaged with the subject over different period offerings? differences in composition of cohorts only two major differences in the comparison of the cohorts were identified in the two trimester subject offerings. trimester 2 responses had a significantly higher proportion of international students and significantly less domestic students. with respect to internationals students, they represented 29% of the responding student cohort in trimester 2 compared to 9% in trimester 1. this was an increase of 20%. with respect to domestic students, they represented 91% of the cohort in trimester 1, yet only 71% in trimester 2. a proportional decrease of 20% (chi-square 9.638, df 1, p-value = 0.002). regarding the percentage of first, second and third year students studying the subject in each trimester (1 & 2) of 2014, a statistically significant difference was evident (chi-square 6.593, df 2, p-value = 0.037). the percentage of first year students participating in the second trimester of 2014 reduced from 84% to 64%, and alternatively, the percentage of second year students participating in trimester 2 was greater than it was in trimester 1, rising from 13% to 25%. that is, it appears that trimester 2, 2014 had a larger proportion of comparatively academically mature students. differences in student engagement no differences were detected between responding students from both trimester 1 and trimester 2, 2014 regarding the amount of time that they indicate they are studying online through the lms. with respect to how well students think they will achieve as a function of how much study they indicate that they did, students from both trimester 1 and trimester 2, 2014 do not have significantly different opinions. that is, regardless of the amount of study that students indicate that they do each week, only 4.6% of responding students think that they have achieved all of the required learning outcomes. results consistent with trimester 1 subject offering were: • 75.9% of all responding students indicated that cloud learning and located classroom learning were modes of education that were both relevant, and enhanced each other. • 56.6% of all respondents at least agreed that cloud learning maximised their learning from on campus classes. • 75.7% of all respondents at least agreed that cloud learning allowed them to organise their study time more effectively around their work and their personal commitments and, • 81.6% of all respondents at least agreed that cloud learning allowed them to learn the subject materials more conveniently. overall, with the exception of assessment (t 2.078, df 578, p-value -0.038) with responding students indicating slightly, yet a significantly greater level of agreement in trimester 2, 2014 compared to trimester 1 (a mean in t2 of 4.1 compared to a mean of 3.9 in t1), there was no change witnessed in the way that students responded to the different items of the themed constructs between trimester 1 and trimester 2 of 2014. australasian journal of information systems pye, holt & salzman 2018, vol 22, research on educational technologies student engagement with blended learning environments 11 research question 2: in analysing student engagement data over the two subject offerings, did any significant differences emerge in student engagement based on key demographic and background factors? hypothesis 1: responding males and females should rank similarly with respect to their overall level of agreement with each of the different construct themes and each of the items associated with each theme. figure 1: mean score for each of the hypothesised latent constructs as a function of gender regarding the statements themed around “assessment”: 1. assignments were easy to locate online; 2. tests were easy to locate online; 3. assignments were easy to submit online; 4. online assignment submission gave me more time to complete assessable work; 5. online assignment feedback helped me improve my understanding of the topic, and; 6. online assignment feedback helped me prepare for subsequent assessable work. responding females are more inclined to be in agreement with these statements when compared to responding males (t 2.950, df 669, p-value = 0.003). both genders however, are at least, in agreement with the statements regarding “assessment”. regarding the statements themed around “relevance”: 1. using clouddeakin made my study seem more relevant; 2. using clouddeakin made me feel part of the university, and; 3. using online materials helped me put my study in real-world contexts. responding females are more inclined to be in agreement with these statements when compared to responding males (t 2.859, df 360, p-value = 0.004). it is noted that both genders are at least, in agreement with the statements regarding “relevance”. regarding the statements themed around “contact with staff”: 1. i used clouddeakin to contact academic staff; 2. i found it easy to communicate with staff online, and; 3. i had individual contact with academic staff online. responding females are more inclined to be in agreement with these statements when compared to responding males (t 2.022, df 321, paustralasian journal of information systems pye, holt & salzman 2018, vol 22, research on educational technologies student engagement with blended learning environments 12 value = 0.044). both genders are however, at least in agreement with the statements regarding “contact with staff”. overall, these detected differences were unexpected. hypothesis 2: both located (on-campus) and cloud (off-campus) participating students should rank similarly with respect to their overall level of agreement with each of the different construct themes and each of the items associated with each theme. figure 2: mean score for each of the hypothesised latent constructs as a function of studying in an on or off campus mode a significant difference is evident between responding located (on-campus) and cloud (offcampus) students with respect to the statements made around the theme “active learning” (t -2.786, df 108, p-value = 0.006). although both located and cloud students are generally in “agreement” with the statements made regarding “active learning” (1. i used online materials to improve my learning; 2. i used online materials to make classroom learning more meaningful; 3. i identified expected academic standards using the clouddeakin unit site, and; 4. i found that online materials challenged me to learn), cloud students are more inclined to be in agreement with these statements compared to located students. compared to located students, responding cloud students appear to have less difficulty finding and using resources. a significant difference is evident between responding located and cloud students with respect to their level of agreement with the statements made around the theme “teaching” (t -3.455, df 158, p-value = 0.001). although both located and cloud students are generally in “agreement” with the statements made regarding “teaching” (1. teaching staff participated in online discussion; 2. i found it easy to explain my ideas in online discussions; 3. i had helpful online discussions with other students, and; 4. i met new people when using the clouddeakin australasian journal of information systems pye, holt & salzman 2018, vol 22, research on educational technologies student engagement with blended learning environments 13 system), responding cloud students are more inclined to be in agreement with these statements compared to located students and cloud students appear to have also found the lms discussion forums related to teaching more helpful. a significant difference is evident between responding located and cloud students with respect to their level of agreement with statements made around the theme “contact with staff” (t -2.670, df 114, p-value = 0.009). regarding the statements made about “contact with staff” (1. i used clouddeakin to contact academic staff; 2. i found it easy to communicate with staff online, and; 3. i had individual contact with academic staff online), responding cloud students are more inclined to be in agreement with these statements compared to located students, cloud students knew where and how to contact fellow students and staff. hypothesis 3: both domestic and international responding students should rank their overall level of agreement with each of the different construct themes similarly and each of the items associated with each theme. figure 3: mean score for each of the hypothesised latent constructs as a function of being an international or a domestic student with the exception of “active learning”, when comparing responding international and domestic students, all themed constructs revealed a significant difference: social interaction (t -3.463, df 124, p-value = 0.001); collaboration (t -3.481, df 119, p-value = 0.001); teaching (t 8.045, df 244, p-value < 0.00000) very highly significant; assessment (t -3.167, df 198, p-value = 0.002); relevance (t -6.396, df 108, p-value < 0.00000) very highly significant; and contact with staff (t -2.799, df 85, p-value = 0.006). with the exception of “active learning”, when compared to responding domestic students, responding international students were significantly more likely to be in agreement with all of the statements considered. australasian journal of information systems pye, holt & salzman 2018, vol 22, research on educational technologies student engagement with blended learning environments 14 hypothesis 4: students of different age groups who responded should rank their overall agreement with each of the different construct themes similarly and each of the items associated with each theme. age data was recoded into two groups: 25 years old or less, and those over 35. figure 4: mean score for each of the hypothesised latent constructs as a function of being a younger (25 years or less) or an older student (older than 25 years) differences in the responding students age groups across the constructs can be identified for: active learning (t -3.211, df 293, p-value = 0.001); teaching (t -3.687, df 410, p-value < 0.000) were highly significant; and contact with staff (t -4.642, df 187, p-value < 0.00000) very highly significant. for each of these three construct themes (active learning, teaching and contact with staff), when compared to respondents who are less than 26 years of age, responding students who are 26 years or older are more likely to agree with statements made regarding these overarching themes. • active learning: i used online materials to improve my learning; i used online materials to make classroom learning more meaningful; i identified expected academic standards using the clouddeakin unit site; • teaching: i found that online materials challenged me to learn staff used clouddeakin to clarify what was required to do well; staff used clouddeakin in ways that improved the overall teaching; academic staff made an effort to communicate with students online; academic staff used clouddeakin to provide students with extra assistance; teaching staff used clouddeakin to tailor activities for students; and teaching staff used clouddeakin to discuss interesting issues, and; • contact with staff: i used clouddeakin to contact academic staff; i found it easy to communicate with staff online; i had individual contact with academic staff online. australasian journal of information systems pye, holt & salzman 2018, vol 22, research on educational technologies student engagement with blended learning environments 15 that is, “older students” (> 25 years) appear to appreciate the resource, or perhaps get more out of using the resource, when compared to younger students (25 years old or less). 5 discussion of findings for design and practice the comparative study of the same subject offered in two different trimesters did not yield any major significant differences in student engagement, with the exception of assessment. this may be explained by the fact that a fault in the online assessment rubric existed in the first offering, but was rectified in the second offering. therefore, assessment was seen to be easier in the second compared to the first offering of the subject. no significant difference between offerings was largely expected as the subject’s curriculum, assessment and online environment was unchanged. the only significant change was the trimester 2, 2014 offering was located on one campus and the cloud rather than across the three campuses including the cloud as in the trimester 1, 2014 offering. the smaller and possibly less diverse student group was a factor, but limited in impact overall. however, the research has identified some significant differences in self-reported agreement with different online student engagement constructs with the completion of the two-offering comparative analysis. first, the differences between international and domestic student views is significant across all constructs with the exception of active learning. a caveat is in order. clearly, the domestic student cohort might represent significant cultural diversity, which has not been delineated in this work. similarly, international students might come from a range of cultural backgrounds, although it can be assumed that they are primarily from asian countries. these important cultural nuances potentially within cohorts must be acknowledged. however, the detected differences are worthy of recognition and exploration. cultural differences are usually assumed in cross-cultural communication behaviours in online learning environments (chen, et al. 2006), with the challenge usually being to include students from other cultures centrally into the planned learning experience. this study highlights the special perceived value that international students attribute to the functions and features of a mainstream online learning system, centring on an lms. it can be seen that comparatively, international students particularly like the highly structured and regulated nature of the environment for facilitating their learning. the teacher’s role as the clearly defined authority for orchestrating resources, monitoring and steering discussions, answering questions, and setting up devices like online assessment rubrics for detailed and structured assessment feedback, is strongly evident. one could suggest this student cohort feels particularly comfortable, safe and productive in such instructivist-oriented online and blended learning environments. it is a cautionary finding, yet fortified by hofstede’s (2001) cultural dimensions relating certain group’s behaviours in accepting power differentials in the student-teacher relationship online, and in demonstrating behaviours embracing of certainty in the formulation of design expectations and associated practices. more authentic, constructivist environments may generate unique and exciting learning benefits, but they may be particularly confronting to student cohorts from different cultural and by implication, educational backgrounds. second, gender differences were identified, with females indicating overall higher perceived levels of online engagement across all constructs, with significant differences in the dimensions of assessment, relevance and contact with staff. one can only speculate as to whether and why females might be more attentive to assessment online, perhaps more so seeing the relevance of unit resources and also tend to interact more strongly with staff. there is evidence that females might be more proficient with online communication and discussions australasian journal of information systems pye, holt & salzman 2018, vol 22, research on educational technologies student engagement with blended learning environments 16 than males. this is certainly the finding of tsai et al. (2015) where males were not as active as females in online asynchronous discussions. they concluded that female students were more strategic learners by adapting themselves better than the males in these contexts. whether females’ historically socially constructed roles in giving, sharing and responding to others more empathetically is becoming a more discernible point of difference in engagement online and in blended settings, requires more substantial research. while females were seen as more disadvantaged than males at the beginning of modern ict developments in relation to access and technical skills, this no longer seems the case, and at least some evidence sees the advantage swinging towards females in current technology-enabled environments. third, mode of enrolment differences between located (on-campus) and cloud (off-campus) students are perhaps unsurprising for those with long histories and experiences of both oncampus and distance education. traditionally, off-campus students have been mature-aged. in this study, 67% of responding off-campus students were older than 31, whereas only 13% of on-campus students were in this same age group. that is, off-campus students are more settled in personal and professional life and hence, more focused and motivated when returning to study to achieve particular career-related goals. in addition, to strength of focus and motivation, they can demonstrate more developed self-management capabilities. in order to achieve specific objectives, one can see how this cohort might engage more strongly with online teaching opportunities, contacts with staff and be more active in their learning strategies. well-ordered lms environments might also facilitate the achievement of their objectives in their relatively time-poor worlds. while not wishing to draw extreme and potentially distorting distinctions, on-campus cohorts can tend to be predominantly schoolleaver, with less well articulated motivations and goal setting around their chosen studies. they may be less focused and, as a consequence, less engaged across-the-board with their online and blended learning experiences. it could also be speculated that on-campus students, as predominantly schooler leaver, or at least younger students, might be more attracted to engage with and through social media away from the institutionally-supported lms. hence they might be shifting their engagement from one type of online learning system to another type of social media networking constellation of applications and spaces. these distinctions are now considered in terms of related age group differences in student engagement. fourth, the age difference was evident for active learning, teaching and contact with staff student engagement constructs, with older students more likely to agree with statements made regarding these overarching themes compared to younger students. these findings map closely with differences detected between located (on-campus) and cloud (off-campus) cohorts in student engagement. this is understandable given older students tend to be more heavily represented as off-campus enrollees compared to younger students as on-campus enrollees. a point to be made though is that a reasonable proportion of older students do study part-time on-campus. they also might engage more strongly with provided online learning resources and communicative opportunities than younger, on-campus students. again, irrespective of mode of enrolment, older students might be more highly focused in their motivation to engage with what is provided in online learning environments. cohort differences clearly exist. how can they be considered and accommodated? design and practice still pragmatically revolves around providing online learning space options. a key lynchpin though remains, the general purpose, well-structured and well-regulated lms environment. what can be highlighted is that well-structured lms spaces should use australasian journal of information systems pye, holt & salzman 2018, vol 22, research on educational technologies student engagement with blended learning environments 17 consistently applied and maintained design interfaces which allow students to easily and effectively access and use key lms learning resources, assessment and discussion functions. creating and embedding or simply using the standard interface features of the lms is important to maximize quality student engagement, irrespective of demographic or other background differences. however, such design interfaces, simple or more sophisticated, must be maintained to a high level of integrity over time and this requires teaching staff to be properly trained and supported in their consistent use. student engagement can be maximized when consistent interface design is implemented across full courses of study. breakdowns on the effective set-up and use of design interfaces in lmss can have a deleterious effect on patterns of student engagement. the application of universal design principles is important in the lms domain to promote quality online student engagement across the board. other environments may to a greater or lesser extent, be integrated with the lms. at this point, the evidence suggests it would be unwise to discount such environments, as for example, this might be jarring and confronting to the international student experience. teachers need to appreciate alternative social media platforms for student engagement, and the ways different cohorts might engage differently with commonly set up online sites in an lms. catering for all and drawing in students constructively to what is provided for most, remains the ongoing design and practice challenge for all online educators. moreover, it needs to be recognized that there are limits to engaging students in online spaces, and that blended learning approaches need to see face-to-face, in-person spaces specifically designed for complementary interactions between teachers and students which foreground student differences. 6 future developments the research will progressively investigate the effects of design interventions on student engagement on particular subjects offered at different points of time over the research project period. the research will also report on more generalised blended learning design and practice activities which contribute to higher student engagement irrespective of the discipline, year of offer, and subject-based designs and practices in the business education domains. while generic designs and practices will be developed, particular attention will also be given to further investigating significant identified differences in student engagement based on demographic and background factors. clearly, a one-size-fits-all approach cannot necessarily meet the needs of such large and diverse student audiences (see armatas, holt, & rice, 2003). the issues identified here also lend themselves to further exploration through interpretive qualitative research. such research methodologies would lead to further explanations as to why the differences identified exist and what the implications might be for enhanced online design and practice. further meta-analysis is currently being conducted to validate the online student constructs used in the survey. this should help confirm and refine the survey’s relevance in investigating key dimensions of student engagement in online and blended learning settings. the study of online blended learning can be further extended into wholly online learning environments, which are now prevalent in global distance education. the survey study has now been adopted to investigate patterns of student engagement in purely online learning and further applications should prove illuminating. 7 conclusion multi-modal, multi-campus, diverse audience blended learning designs and practices is a hallmark of australian higher education in the realm of business education. these types of australasian journal of information systems pye, holt & salzman 2018, vol 22, research on educational technologies student engagement with blended learning environments 18 blended learning environments raise pressing challenges in providing all students, irrespective of location, mode of delivery, background and motivation, with the best possible opportunity of mastering unit and course-wide learning outcomes. assuring the mastery of desired learning outcomes is now central to university strategic curriculum design and course enhancement efforts. it is also a growing imperative in meeting external regulatory and professional accreditation requirements. designing for student engagement provides a theoretically and conceptually grounded approach to enhancing and assuring student learning. the study has trialed the use of an online student engagement survey over two offerings of the same subject, with a dual focus on online and located/campus-based learning experiences, and the relationship between the two. while few significant differences were identified in overall student engagement over the two periods of offering of the subject, on further analysis interesting differences in the nature of student engagement were found relating to gender, international/domestic students, located (on-campus)/cloud (off-campus) students and older (matured aged) and younger (school leaver) students. reasons for the identified differences have been explored. more and different styles of research should be used to further examine causes of difference in student engagement based on these key demographic and background factors. the balancing act still remains the development of online environments that cater for the needs of students who rely almost entirely on them to achieve the required learning outcomes, and those who best benefit from dual engagement with both online and located/campus learning opportunities for learning achievement. paramount is the need to understand the diversity in the student population and that different student cohorts may engage in different ways with blended learning designs. one of the most significant findings for institutions strategically committed to blended and online learning is the continued importance of substantial investment in online learning systems, which can best enable the variety of designs and practices required to accommodate different student needs, backgrounds and circumstances. major investments in one centrally controlled institutional system, as the core online learning interface and platform, for an institution’s student population seems vindicated through the results of this study. the ubiquitous lms platform now well entrenched in most universities’ technological infrastructure seems like it has and will continue to meet the test of time. references armatas, holt, d. & rice, m. (2003). impacts of an online-supported, resource-based learning environment: does one size fit all? distance education, 24(2), 141-158. doi: 10.1080/0158791032000127446 astin, a.w. (1985). achieving educational excellence: a critical analysis of priorities and practices in higher education. san francisco, ca: jossey-bass. bennett, s., dawson, p., bearman, m., molloy, e., & boud, d. (2017). how technology shapes assessment design: findings from a study of university teachers. british journal of educational technology, 48(2), 672-682. doi: 10.1111/bjet.12439 bonk, c.j. & zhang, k. (2008). empowering online learning: 100 + activities for reading, reflecting, displaying, & doing. san francisco: john wiley & sons. boud, d. (2010). student assessment for learning in and after courses. final report for senior fellowship. australian learning and teaching council (altc): australasian journal of information systems pye, holt & salzman 2018, vol 22, research on educational technologies student engagement with blended learning environments 19 http://www.olt.gov.au/resource-student-assessment-learning-and-after-courses-uts2010 boud, d. & molloy, e. (2013). (eds.). feedback in higher and professional education understanding it and doing it well. london: routledge. challis, d., holt, d.m. & rice, m. (2005), 'staff perceptions of the role of technology in experiential learning: a case study from an australian university', australasian journal of educational technology, 21(1), 19-39. chen, r. t-h., bennett, s. & maton, k. (2008). the adaptation of chinese international students to online flexible learning: two case studies. distance education, 29(3), 307-323. doi: 10.1080/01587910802395821 chen, s.j., hsu, c.l. & caropreso, e.j. (2006). cross-cultural collaborative online learning: when the west meets the east. international journal of technology in teaching and learning, 2(1), 17-35. chickering, a.w. & gamson, z.f. (1987). seven principles for good practice in undergraduate education. aahe bulletin, 39(7), pp. 3-7. choi, s. h.-j. & nieminen, t.a. (2013). factors influencing the higher education of international students from confucian east asia. higher education research & development, 32(2), 161-173. doi: 10.1080/07294360.2012.673165 coates, h. (2006). student engagement in campus-based and online education: university connections. routledge: london. coates, h. & mccormick, a.c. (eds.) (2014). engaging university students international insights from system-wide studies. springer: singapore. coates, h. & mccormick, a.c. (2014). introduction: student engagement – a window into undergraduate education. in h. coates & a.c. mccormick (eds.) (2014). engaging university students international insights from system-wide studies. springer: singapore, pp.1-12. dawson, p. (2017). assessment rubrics: towards clearer and more replicable design, research and practice. assessment & evaluation in higher education, 42(3), 347-360. doi: 10.1080/02602938.2015.1111294 garrison, d.r. & anderson, t. (2003). e-learning in the 21st century. a framework for research and practice. london: routledge falmer. gunn, c., french, s., mcleod, h., mcsporran, m. & conole, g. (2002). gender issues in computer-supported learning. research in learning technology, 10(1), 32-44. hagel, p., carr, r. & devlin, m. (2012). conceptualising and measuring student engagement through the australasian survey of student engagement (ausse): a critique. assessment & evaluation in higher education, 37 (4), 475-486. doi: 10.1080/02602938.2010.545870 henderson, m., boud, d., molloy, e., dawson, p., phillips, m., ryan, t., walker, h. & mahoney, p. (2016). feedback for learning closing the assessment loop. federal australian government office for learning and teaching (in progress). project website: http://newmediaresearch.educ.monash.edu.au/feedback/ australasian journal of information systems pye, holt & salzman 2018, vol 22, research on educational technologies student engagement with blended learning environments 20 hofstede, g. (2001). culture's consequences: comparing values, behaviors, institutions, and organizations across nations (2nd ed.). thousand oaks, ca: sage publications. holt, d.m. & challis, d. (2007). from policy to practice: one university’s experience of implementing strategic change through wholly online teaching and learning, australasian journal of educational technology, 23(1), 110-131. holt, d., segrave, s. & cybulski, j. (2011) (eds.) professional education using e-simulations: benefits of blended learning design. hershey, us: igi global. howson, c.k. & buckley, a. (2017). development of the uk engagement survey. assessment & evaluation in higher education, 42(7), 1132-1144. doi: 10.1080/02602938.2016.1235134 jonsson, a. (2014). rubrics as a way of providing transparency in assessment. assessment & evaluation in higher education, 39(7), 840-852. doi: 10.1080/02602938.2013.875117 kopcha, t.j., schmidt, m.m. & mckenney, s. (2015). editorial: special issue on educational design research (edr) in post-secondary learning environments. australasian journal of educational technology, 31(5), i-ix. kuh, g.d. (2003). what we’re learning about student engagement from nsse. change, 35(2), 24-31. kuh, g.d., kinzie, j., schuh, j.h., whitt, e.j., & associates (2005). student success in college: creating conditions that matter. san francisco: jossey-bass. lau, w.w.f & yuen, k.h.a. (2010). gender differences in learning styles: nurturing a gender and style sensitive computer science classroom. australasian journal of educational technology, 26(7), 1090-1103. lawson, r. (2015). curriculum design for assuring learning – leading the way. final report for senior fellowship. office for learning and teaching (olt): http://www.olt.gov.au/resource-curriculum-design-assuring-learning-leading-way2015 li, q. (2005). gender and cmc: a review on conflict and harassment. australasian journal of educational technology, 21(3), 382-406. liu, x., liu, s., lee, s.-h & magjuka, r.j. (2010). cultural differences in online learning: international student perceptions. educational technology & society, 13(3), 177-188. markauskaite, l. (2006). gender issues in preservice teachers’ training: ict literacy and online learning. australasian journal of educational technology, 22(1), 1-20. mckenney, s.e. & reeves, t.c. (2012). conducting educational design research. new york: routledge. mcsporran, m. & young, s. (2001). does gender matter in online learning? research in learning technology, 9(2), 3-15. nicol (2010). re-engineering assessment practices in higher education: http://www.reap.ac.uk/home.aspx oliver, b. (2013). graduate attributes as a focus for institution-wide curriculum renewal: innovations and challenges. higher education research & development, 32(3):450-63. australasian journal of information systems pye, holt & salzman 2018, vol 22, research on educational technologies student engagement with blended learning environments 21 pace, c.r. (1979). measuring outcomes of college: fifty years of findings and recommendations for the future. san francisco, ca: jossey bass. pace, c.r. (1995). from good practices to good products: relating good practices in undergraduate education to student achievement. paper presented at the association for institutional research, boston. palmer, s. & holt, d. (2014). development of student and academic staff perceptions of the elements of an online learning environment over time. australasian journal of educational technology, 30(4), 375-89. parkin, h.j., hepplestone, s., holden, g., irwin, b. & thorpe, l. (2012). a role for technology in enhancing students’ engagement with feedback. assessment & evaluation in higher education, 37(8), 963-973. doi: 10.1080/02602938.2011.592934 pye, g., holt, d., salzman, s., bellucci, e. & lombardi, l. (2015). engaging diverse student audiences in contemporary blended learning environments in australian higher business education: implications for design and practice. australasian journal of information systems, 19, 1-20. reeves, t.c. (2015). educational design research: signs of progress. australasian journal of educational technology, 31(5), 613-20. salmon, g. (2011). e-moderating. the key to teaching and learning online. new york: routledge. small, f. & attree, k. (2016). undergraduate student responses to feedback: expectations and experiences, studies in higher education, 41(11), 2078-2094. doi: 10.1080/03075079.2015.1007944 teo, t., fan, x. & du, j. (2015). technology acceptance among pre-service teachers: does gender matter? australasian journal of educational technology, 31(3), 235-251. tinto, v. (1993). leaving college: rethinking the causes and cures of student attrition. chicago, il: university of chicago press. tinto, v. (1997). classrooms as communities: exploring the educational character of student persistence. journal of higher education, 68(6), pp.599-623. tsai, m.j., liang, j-c., hou, h-t. & tsai, c-c. (2015). males are not as active as females in online discussion: gender differences in face-to-face and online discussion strategies. australasian journal of educational technology, 31(3), 263-77. wallace, p. (1999). the psychology of the internet. new york: cambridge university press. winstone, n.e., nash, r.a., rowntree, j. & parker, m. (2017). ‘it’d be useful, but i wouldn’t use it’: barriers to university students’ feedback seeking and recipience. studies in higher education, 42(11), 2026-2041. doi: 10.1080?03075079.2015.1130032 wong, k-t., teo, t. & russo, s. (2012). influence of gender and computer teaching efficacy on computer acceptance among malaysian student teachers: an extended technology acceptance model. australasian journal of educational technology, 28(7), 1190-1207. zhang, j. (2007). a cultural look at information and communication technologies in eastern education. education technology, research and development, 55(3), 301-314. australasian journal of information systems pye, holt & salzman 2018, vol 22, research on educational technologies student engagement with blended learning environments 22 appendix: online student engagement constructs for contemporary blended learning environments online active learning • i used online materials to improve my learning • i used online materials to make classroom learning more meaningful • i identified expected work standards using the clouddeakin (lms) unit site • i found that online materials challenged me to learn online social interaction • teaching staff participated in online discussions • i found it easy to explain my ideas in online discussions • i had helpful online discussions with other students • i met new people when using clouddeakin (lms) online collaboration • i used clouddeakin (lms) with other students around campus (on-campus students only) • i used clouddeakin (lms) to do academic work with other students • i used clouddeakin (lms) to work with other students outside of class (on-campus students only) • i used clouddeakin (lms) to communicate with other students online teaching • staff used clouddeakin (lms) to clarify what was required to do well • staff used clouddeakin (lms) in ways that improved the overall teaching • academic staff made an effort to communicate with students online • staff used clouddeakin (lms) to provide students with extra assistance • teaching staff used clouddeakin (lms) to tailor activities for students • teaching staff used clouddeakin (lms) to discuss interesting issues online assessment • assignments were easy to locate online • tests were easy to undertake online • assignments were easy to submit online • online assignment submission gave me more time to complete assessable work • online assignment feedback helped me improve my understanding of the topic • online assignment feedback helped me prepare for subsequent assessable work australasian journal of information systems pye, holt & salzman 2018, vol 22, research on educational technologies student engagement with blended learning environments 23 online academic relevance • using clouddeakin (lms) made my studying seem more relevant • using clouddeakin (lms) made me feel part of the university • using online materials helped me put my study in real-world contexts online contact with staff • i used clouddeakin (lms) to contact academic staff • i found it easy to communicate with staff online • i had individual contact with academic staff online copyright: © 2018 pye, holt & salzman. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 background 3 research methodology 3.1 key research questions for informing design and practice 3.2 online student engagement survey design 3.3 data collection and response rate 4 data analysis and findings 5 discussion of findings for design and practice 6 future developments 7 conclusion paper title (paper title style) australasian journal of information systems naparat, calhane & finnegan 2015, vol. 19, selected papers from acis healthy community & healthy commons healthy community and healthy commons: ‘opensourcing’ as a sustainable model of software production damrongsak naparat unsw australia business school sydney, australia d.naparat@unsw.edu.au michael cahalane unsw australia business school sydney, australia m.cahalane@unsw.edu.au patrick finnegan unsw australia business school sydney, australia p.finnegan@unsw.edu.au abstract many commercial software firms rely on opensourcing as a viable model of software production. opensourcing is a specific form of interaction between firms and open source software (oss) communities for collaboratively producing software. the existing literature has identified opensourcing as a viable form of software production, which could be a substitute for “in-house” or “outsourced” software development. however, little is known about how opensourcing works or is sustained in the long term. the objective of this research is to explain the factors affecting the sustainability of opensourcing as a model of software production. the study employs a single case study of hospital software in thailand to understand how firms and the communities can live symbiotically and sustain their collaboration to peer-produce vertical domain software. the analysis reveals six mechanisms (positive experience, trust in the leadership of the project leader, the demonstration of reciprocity, marketing the community, enriching knowledge, and face-to-face meetings) and demonstrates how they operate in conjunction with each other to sustain opensourcing. keywords: opensourcing, sustainable, mechanism-based theorising, software production, vertical domain software 1 introduction opensourcing is a particular form of crowdsourcing. crowdsourcing involves “a company or institution taking a function once performed by employees and outsourcing it to an undefined (and generally large) network of people in the form of an open call” (howe 2006). opensourcing is a crowdsourcing method for software production where commercial firms and oss communities collaboratively develop software of interest to firms (ågerfalk and fitzgerald 2008). opensourcing has implications for both profit and not-for-profit firms since it allows them to leverage the intelligence and innovation of ‘the crowd’ by incorporating such innovation into their software products. large firms that have practiced opensourcing include apple and ibm (west 2003). ågerfalk and fitzgerald (2008) have identified opensourcing as an interesting phenomenon. this is because opensourcing could be a viable alternative form of software production, in particular for the development of vertical domain software. however, the major challenge of opensourcing is its sustainability as it relies on the open source software (oss) development model. specifically, when software is peer-produced (e.g., oss development), it is subject to several threats, such as under provisioning of contributions, unilateral appropriation of 1 australasian journal of information systems naparat, calhane & finnegan 2015, vol. 19, selected papers from acis healthy community & healthy commons software, and misapplication of software, that could prevent its continuance (benker 2002; o’mahony 2003; shah 2006). a definition for “sustainable opensourcing” is lacking; hence we define it as the ability of an opensourcing project to (1) prevent the community from being damaged so that it exhibits the continuity of its software development and maintain software of the firms’ interest (healthy community), and (2) nurture and improve the information commons of the project over a predetermined period of time (healthy commons). the first dimension of sustainable opensourcing concerns a healthy community, meaning an open source project has a significant number of participants (critical mass) to perform software development and maintenance activities. the healthy community is related to (i) how the community attracts participants and (ii) how the community sustains participation. the second dimension, a healthy commons, is concerned with the quality of contributions and the quality of software produced by the community. information goods such as software are not subject to the overgrazing problem; however, the lack of improvement or nurturing of the commons can result in the tragedy of the commons in the context of software production (cf. andreev et al. 2010). we argue that highquality contributions are the crucial ingredient to produce high-quality software. therefore, for opensourcing to be sustainable, its community and its commons must be healthy. the objective of this paper is to explain the factors affecting the sustainability of opensourcing as a model for software production, especially for the development of vertical domain software. the paper begins by outlining the challenges that could prevent opensourcing from being sustainable, with the perspective that those challenges could severely affect either the health of the community or the health of the commons. we argue that sustainable opensourcing can be better understood through process theory by revealing the mechanisms that sustain opensourcing. this is followed by a description of the case study and research methodology. we use a single case study of the development of a hospital information system (his) in thailand to discover mechanisms that help sustain opensourcing. this is because his development is a good demonstration of vertical domain software development, where business processes and user requirements are complex and are not known amongst developers. hence there was a need for the software company to collaboratively produce software with a community of health care practitioners. the findings are then presented in the form of six theoretical propositions that, collectively, explain how each mechanism works in conjunction with others to sustain opensourcing. finally, we provide recommendations for practitioners interested in pursuing opensourcing as a model for software production. 2 challenges to sustainable opensourcing software production models such as “in-house” development or outsourcing rely on formal governance mechanisms to enable and sustain the process of software production. when software is produced within a firm, “hierarchical” governance mechanisms (e.g., management controls and incentive structures) (cf. williamson 1996) ensure that the software development process can deliver a desirable outcome. in outsourcing, software is sourced through markets where the price mechanism (cf. williamson 1996) plays a role in choosing the supplier. when the supplier is identified, it is the outsourcing contract that governs the production of software (bush et al. 2008). software outsourcing can be repeated as many times as necessary through market mechanisms, thus, while outsourcing providers may change, outsourcing is considered a sustainable form of software production. in contrast, opensourcing relies on a hybrid form of governance, consisting of hierarchical and community-based social mechanisms to produce software (naparat and finnegan 2013). we argue that the core problem with this hybrid governance is that while hierarchy seems to work perfectly for controlling and coordinating paid developers, who are sponsored or employed, firms do not have formal control over the voluntary participants in opensourcing. lacking complete control over the opensourcing community presents several challenges related to the sustainability of opensourcing as a model for software production. those challenges are (i) under provisioning (low level of contribution), (ii) unilateral appropriation of software, and (iii) misappropriation of software (benker 2002; o’mahony 2003; shah 2006). 2 australasian journal of information systems naparat, calhane & finnegan 2015, vol. 19, selected papers from acis healthy community & healthy commons first, the under provisioning challenge is related to, in benkler’s (2002) terms, the motivation problem. the oss literature reports different types of motives that drive participants to participate in oss development. broadly, these motives could be categorised into intrinsic (e.g., learning, fun) and extrinsic rewards (e.g., money, reputation, use of software) (von krogh et al. 2012). some of these motives last longer than the others, thus sustaining participants’ contributions over a longer period (cf. fang and neufeld 2009). moreover, participants’ motivation can be undermined, and when their motivation is low and continuing to participate does not seem to be beneficial, participants leave the community (benkler 2002; shah 2006). this is important because the loss of participants adversely affects the health of the community. second, unilateral appropriation is evident when a particular individual attempts to direct the software development to benefit themself in a way that prevents other participants from benefiting (benkler 2002). dahlander and wallin (2006) suggest that for firms to push their agenda, they need to assign their employees to work with oss communities. these employees need to gain legitimacy and take important positions in those projects in order to influence the work of the community. however, focusing too much on a firm’s agenda could lead to unilateral appropriation, which in turn may introduce conflicts between firm-based developers and voluntary developers. shaikh and cornford (2010) found that such conflict arises when firmbased (paid) developers attempt to coerce voluntary participants to meet a firm’s requirement. ågerfalk and fitzgerald (2008) suggest that it is important that firms, before pushing their own agenda, require consent from others in the community. unilateral appropriation could lead to a greater degree of monitoring and scrutinising of code, and stronger peer review, resulting in the community incurring the cost for monitoring (e.g., more time for peer review) (shaikh and cornford 2010). in sum, the lack of trust and higher cost of software production can lead to the departure of participants (adler 2001; benkler 2002), thus damaging the health of the community. finally, the misappropriation of software is another challenge that could prevent opensourcing from being sustainable. according to shah (2006), while firms can establish software development projects and allow outsiders to contribute to the projects, the use of non-open source licences makes participants feel reluctant to participate because they think that the license would make it difficult for them to get the permission to use the software. even in several exemplary open source projects, such as the linux kernel or debian linux, the communities have to establish foundations and employ an open source licence as a legal means of protecting their projects from being misappropriated by individuals (o’mahony 2003). we argue that misappropriation of software leads to the departure of participants because it prevents other participants from benefiting from the software produced. again, this dramatically damages the health of the community. 3 research design the research objective is to explain the factors affecting the sustainability of opensourcing as a model of software production. we use process theory as it allows us to unearth mechanisms and deep structures that allow us to achieve the explanation necessary to meet our research objective. the identification of mechanisms is necessary because mechanisms are “the nuts and bolts processes by which cause and affect relationships in the social world come about” (gross 2009, p. 386). therefore, we employ mechanism-based theorising to explore the “nuts” and “bolts” that sustain opensourcing. mechanism-based theorising seeks to explain how and why a process (p) with certain inputs (i) can produce outcomes (o) (gross 2009; hedström and sweberg 1998; hedström and ylikoski 2010). in the context of the use of mechanism-based theorising in a social situation, gross (2009) explains that; [a] social mechanism is a more or less general sequence or set of social events or processes analysed at a lower order of complexity or aggregation by which—in certain circumstances—some cause x tends to bring about some effect y in the 3 australasian journal of information systems naparat, calhane & finnegan 2015, vol. 19, selected papers from acis healthy community & healthy commons realm of human social relations. this sequence or set may or may be analytically reducible to the actions of individuals who enact it, may underwrite formal or substantive causal processes, and may be observed, unobserved, or in principle unobservable. (p. 364) gross (2009) argues that social mechanisms are composed of a chain of four elements: actors, problem situations, habitual responses, and resources—or the a-p-h-r chain. the identification of the a-p-h-r and their relationships allow a mechanism to be revealed. the search for mechanisms can be carried out at the level of an individual. therefore, we adopt dbo theory to explain an individual’s behaviour. dbo theory is composed of three important theoretical terms: desire (d), belief (b), and opportunity (o), which are required to analyse actions and interactions of individuals. dbo theory places emphasis on individuals’ actions and argues that human actions are carried out through a combination of desires, beliefs, and opportunities (hedström 2005). in opensourcing, participants encounter a series of problems related to software production. participants employ their resources (e.g., skills, knowledge, time) to respond to these problems by performing individual and collective actions to reach solutions. 3.1 data collection we followed an in-depth single case study for this research. a single case study is suitable for explanatory research because it allows the researcher to trace the case closely over time (yin 2003). a single case study was conducted with the hospital operating system (hos) community (details in the next section). the hos community was chosen because it produced software in a continuing manner for more than 10 years, which we considered to be long enough to demonstrate sustainability. the data collection for the study was conducted over the course of 18 months, from july 2011 to december 2012. multiple data collection techniques were used to collect data: (i) semistructured interviews, (ii) document analysis, and (iii) online/offline field observations, as they allow for data triangulation, to enhance validity and reliability of the mechanisms underpinning sustainable opensourcing. we conducted 33 semi-structured interviews with four groups of participants. these included (i) the project founder and the company’s ceo, who represented the sponsoring firm; (ii) eight open source technology ltd (ost) employees who participated in the project; (iii) 20 practitioners from rural hospitals who were voluntary participants; (iv) former participants; and (v) officers from central public health care authorities (see table 1). the interviews with different types of participants allowed us to minimise the “elite bias” problem (cf. miles and huberman 1994). the interviews were aided by the use of interview guides. each interview lasted 60–90 minutes. documents related to the hos project were gathered through the internet and via personal contacts. documents included news from online newspapers, user requirement logs, bug fix logs, and charts and diagrams depicting business processes, as well as the project and the sponsoring firm’s websites. the online field observation was conducted through the community’s online web forum and the community’s facebook group. moreover, we conducted offline field observations by participating at several informal meetings amongst the community, formal hos user trainings, and leisure activities. the field observations allowed the researcher to gain the trust of informants, which resulted in informants being more willing to reveal their opinions (jorgensen 1989). transcribed interviews, gathered documents, and field notes yielded more than 500 pages of transcripts. data analysis was conducted using open/axial coding and selective coding techniques (cf. charmaz 2008; corbin and strauss 2008). concepts and their relationships were identified via an open/axial coding process, thereby revealing construct categories and subcategories. next, mechanisms were identified from the constructs through the use of the ap-h-r chain and the dbo theory (cf. gross 2009; hedström 2005). the researcher followed the approach of corbin and strauss (2008) and identified relationships between concepts, contexts, conditions, actions, interactions, and actors who perform the actions. the 4 australasian journal of information systems naparat, calhane & finnegan 2015, vol. 19, selected papers from acis healthy community & healthy commons identification of relationships amongst these elements allowed the researcher to discover mechanisms, how they operate, what the outcomes of the mechanisms were, and how the mechanisms were linked or how they operated in conjunction with one another. finally, “core” mechanisms that sustain opensourcing were identified through selective coding. type of participant no. of interviews interview duration (hours : minutes) description project founder 2 1:40 rationale for opensourcing ceo 1 1:32 roles of firm paid developers 11 11:37 roles, short-/long-term motivations voluntary participants 14 approx. 20 hours roles, short-/long-term motivations former participants 3 2:32 reasons for leaving/rejoining the community provincial public health officer 1 1:20 perspective on oss medical software, provincial health policies national health security officer 1 0:30 perspective on oss medical software, national health policies total 33 approx. 40 hours table 1: interviews conducted in the study 4 the case study and analysis 4.1 the case study background the hospital operating system (hos) project is an opensourcing project that employs a community-driven model for information and communications technology (ict) in rural hospitals. the primary goal of the project is to develop hospital information systems software for small rural hospitals. the hos project began in 1999 as a 17-month, government-funded research project, with its initial plan to implement the system at 10 hospitals. hos was implemented at 17 hospitals nationwide when the research projected was completed. in 2000, the project founder, kongkiat kespetchara, turned the project into a commercial venture and founded open source technology co. ltd. (ost). the project continued to use the communitydriven model to crowdsource production. participants in the hos community included practitioners from rural hospitals (e.g., doctors, nurses, pharmacists) and paid software engineers (ost’s employees). hos proved itself as an exemplary sustainable opensourcing project because its collaborative software development model has worked for more than 10 years and is still in operation. rural hospitals using hos gained the benefit of process improvement by reducing patient risk and lost data, as well as increasing data accuracy and financial returns. hos was released under a general public license (gpl), but ost sells services around hos, including software installation and configuration, custom module development, and providing related computer and networking hardware. 5 australasian journal of information systems naparat, calhane & finnegan 2015, vol. 19, selected papers from acis healthy community & healthy commons 4.2 analysis the analysis revealed a chain of six mechanisms working in conjunction with each other to sustain opensourcing (see table 2). following the mechanism-based theorising approach, the presentation of each mechanism includes the description of a situational problem, the definition of the mechanism, a discussion of the operation of the mechanism to solve a situational problem, and the presentation of the supporting evidence from the case study. mechanism description sustainability dimension positive experience positive knowledge, perception, and attitude that participants have accumulated from their previous participation healthy community trust in the leadership of the project leader the participants’ firm belief that the project leader is visionary, knowledgeable, and has enough resources and control to steer the opensourcing project in a way that benefits the community at large healthy community healthy commons demonstration of reciprocity examples of reciprocal acts performed by participants healthy community marketing the community how the opensourcing community employs methods to create awareness and earn a good reputation for the project. healthy community enriching knowledge how the opensourcing community improves the domain-specific and technical knowledge of its participants healthy commons face-to-face meetings regular/ongoing offline face-to-face meetings organised by the opensourcing community to allow participants to meet each other in person healthy community table 2: mechanisms that sustain opensourcing 4.3 the positive-experience mechanism it is found that the erosion of participants’ motivation is a challenging issue that opensourcing communities need to overcome. it was unavoidable that some participants left the community during the last 10 years. however, the community managed to retain the majority of participants, especially active participants. in addition, the community has attracted new participants. the analysis revealed that the positive experience of participants plays a critical role in maintaining participants’ motivation. we argue that a positive experience comes about through positive knowledge, perceptions, and attitudes that participants have from their previous participation in the project. positive experience creates the desire for participants to continue to participate. participants continue their participation with the expectation that they will receive the same or better experience from their future participation. there are two sources that bring a positive experience to participants: (i) the timely delivery of software solutions to participants, and (ii) good community culture. first, the hos community consistently delivered software and software updates to satisfy its participants and end users. since participants were primarily driven by use of software, to be able to obtain a high-quality software solution satisfied their needs. for instance, one participant mentioned “although hos is a free piece of software, its quality is as high as (if not 6 australasian journal of information systems naparat, calhane & finnegan 2015, vol. 19, selected papers from acis healthy community & healthy commons better than) expensive commercial packages.” moreover, the community responded to bugs and error reports by providing bug fixes and solutions to errors in a timely manner. as for software maintenance, the community provided prompt and accurate solutions for any technical problems raised by participants. this led to a good impression for participants, specifically the feeling that they were part of the community, that their contributions were valued, and that their problems were not ignored. thus, they wanted to continue to participate. for instance, in discussing his positive experience, one long-term participant mentioned, “i remember having some great difficulties setting up hos. there was one participant who was willing to help me fix the problem, which took quite a long time to finish and that was during the weekend. i appreciated that greatly.” next, it was part of the community culture that the community valued participants’ contributions, especially their requirements and domain-specific knowledge. participants had their say on which user requirements they wanted the community to implement and how they should do it. all essential business processes and user requirements specified by participants were implemented. this impressed the participants, and they realised that their contributions and efforts were not wasted. participants were engaged, and they felt that they were part of the community. this led to a positive experience and instilled in the participants the desire to continue their work. another positive aspect of the community culture was that the hos participants regarded each other as friends. the presence of friendship created a pleasurable community climate. most of the participants mentioned this; for instance, one participant noted, “the hos community culture is very good. we live as friends or kin. i wonder if i could experience the same thing in other communities.” accumulated positive experience creates positive emotional feelings for participants, such as the feeling of passion in working with the community or loyalty to the community. such emotions play a crucial role in sustaining participation and encouraging participants to spread positive images about the software and the community to the public, allowing the community to attract new participants. one long-term participant mentioned, “i love the hos community . . . i will never leave the community or switch to other software.” he even had his signature attached to every post on the web forum, stating, “i will love hos until the day i die.” another participant, who was very passionate about using and contributing to the community, said, “i want every rural hospital to use hos. whenever i meet staff from other hospitals, i always persuade them to use hos and join the hos community.” positive experience keeps the community healthy by maintaining current participants and attracting new participants. therefore, our first proposition is presented as follows: proposition 1: positive experience sustains opensourcing by instilling in the participants a desire to (i) continue to participate, and (ii) spread a positive image of the community, thus attracting new participants. 4.4 trust in the project leader we found that the ability of the community to eliminate and solve conflicts amongst participants creates a positive experience for them. disagreements amongst participants normally led to unsatisfactory emotions towards each other, thus undermining their motivation to continue participating in the project. our analysis revealed that trust in the leadership ability of the project leader allowed the project leader to take a role in resolving conflicts very efficiently and effectively. the ability of the project leader to resolve conflicts quickly and satisfy participants influenced the participants’ positive experience and their desire to continue participating. the fact that the project leader was the same person as the company owner meant that the leader had hierarchical power to mobilise the company’s resources (e.g., paid developers, computer infrastructures) to serve the community. the project leader also gained trust from participants by being a visionary, knowledgeable, and a trustworthy person. as a trustworthy and respectful leader, voluntary participants were inclined to follow his decisions, and this minimised conflicts and resistance that could hamper the software development process. 7 australasian journal of information systems naparat, calhane & finnegan 2015, vol. 19, selected papers from acis healthy community & healthy commons the following evidence from our case study demonstrates how trust in the leadership allows the project leader to resolve conflicts and maintain a positive experience for the participants. during the informal conversations amongst long-term volunteer participants, one participant mentioned, “trust in the community was shaken because hos development seemed to stall without any reason.” another participant added, “there were no fixes or updates released. it was miserable when this happened.” later, voluntary participants realised that ost put all of their efforts to produce another commercial hos software and parked the hos project. the response to this was “[w]e [volunteer participants] got together and wrote e-mails to the ceo and the founder, stating that ost should pay attention to the community’s needs and continue the hos development. we understand that ost has limited human-resource capacity and has to survive financially, but ost cannot totally ignore the hos community. otherwise, we have to switch to other software and discontinue our participation.” later the project leader called for a face-to-face meeting between the company and key long-term participants. at the meeting, the project leader assured the community that the hos development would continue. after the meeting, the project leader mobilised the company’s resources to get the project active once again. moreover, the project leader made another promise to the community that “ost will continue to support the hos project even if there was only one hospital using it, and it will always be open source” this statement dismissed doubts and questions that voluntary participants had in mind, such as “how long will ost keep supporting us?” and “should i stay in the community?” while conflicts undermine participants’ motivation and deflect participants from software development tasks, trust in the leadership allows conflicts to be resolved quickly. we argue that the sustenance of participants’ motivation promotes a healthy community. moreover, because conflicts are quickly resolved, participants can focus on software development, thereby improving the health of the commons. therefore, our second proposition is presented as follows: proposition 2: trust in the leadership of the project leader sustains opensourcing by allowing the project leader to quickly solve conflicts, thereby (i) allowing participants to focus on software development and improving software quality and (ii) creating a positive experience for the participants. 4.5 the demonstration of reciprocity we argue that the crucial problem of opensourcing is to sustain participation. as opensourcing lacks a managerial hierarchy, and an authoritative chain of command, and financial reward is only effective for paid developers, the ongoing participation of voluntary participants becomes a challenge. our analysis revealed that it is the trust in reciprocity that triggers the exchange of resources to produce software at the beginning. however, it is the demonstration of reciprocity that sustains the ongoing participation. to clarify, when participants see the demonstration of reciprocity, it affirms their trust in other participants. in addition, it increases their trust in the act of reciprocity. the demonstration of reciprocity thus instils in the participants a desire to continue to contribute because they believe that others will reciprocate. the company and the community have demonstrated that they have lived up to each other’s expectations and have not failed to reciprocate. voluntary participants demonstrated that they regularly performed their tasks, including specifying user requirements, reporting bugs, conducting software testing, and providing user-to-user help. similarly, the company demonstrated that it could transform the business processes and user requirements into a software solution. bugs and errors were fixed through numerous software updates. from 2000 to 2012, the community released three major hos versions, with numerous bug fixes and minor releases in between. prompt responses and reciprocal acts from other participants (in the context of user-to-user help) also increased the participants’ trust in the principle of reciprocity, which in turn instilled in them a desire to continue to participate. therefore, we present our third proposition as follows: 8 australasian journal of information systems naparat, calhane & finnegan 2015, vol. 19, selected papers from acis healthy community & healthy commons proposition 3: the demonstration of reciprocity sustains opensourcing by increasing trust in other participants, thereby increasing their expectation of reciprocity. 4.6 marketing the community our analysis shows that the ability of the community to attract new participants and retain current participants is critical for the community to be healthy and viable. a proactive marketing of the community mechanism was employed to create public awareness and attract new participants. at the community building stage (i.e., early 2000), the company took an active role in recruiting practitioners to build up the community. direct mails to rural hospitals and attending conferences allowed the company to receive attention from a significant number of practitioners, which was enough to create a critical mass of voluntary participants. moreover, the company attended several medical conferences to publicise the hos project in order to attract more participants. the company also attempted to internationalise the project and attract more international participants by producing an english version of hos. in addition, hos participants used ‘word of mouth’ to spread their positive experience about participating in the community, as well as the high quality of the software. word of mouth was a very effective viral marketing mechanism for the hos community as it helped attract new participants and expand their network of users. the company received several prestigious national awards for the development of the hos. these awards illustrated that community-based software development was efficient and effective and the community was capable of producing high-quality software. good reputation of the project nourished participants’ motivation and ensured that they did not waste their efforts. it can be seen that the marketing the community mechanism allows the community to maintain current participants and attract new participants, thus improving the health of the community. therefore, we present our fourth proposition as follows: proposition 4: “marketing the community” mechanism sustains opensourcing by (i) building public awareness about the high-quality software product and the production process, thus (ii) attracting new participants and (iii) helping retain current participants through increasing reputation reward. 4.7 enriching knowledge the analysis revealed that the ongoing knowledge transfer between the company and the community is critical as it allows participants to improve the quality of their contributions, as well as to perform more difficult tasks, which leads to a better pool of software (healthy commons). there were two types of knowledge that were transferred amongst participants: domainspecific knowledge and technical knowledge. domain-specific knowledge included information, know-how, business processes, and insight into the public healthcare domain. this type of knowledge was embedded locally in practitioners (i.e., voluntary participants). domain-specific knowledge changed and evolved constantly. therefore, participants needed to constantly transfer domain-specific knowledge to the community and to the company. in addition, it was revealed that the company organised various training sessions with varying degrees of difficulty, ranging from fundamental to very advanced, to transfer technical knowledge to hos users and voluntary participants. for example, the basic training included using hos functions and basic hos system administration and maintenance. more intermediate training sessions were infrastructure management (operating system installation and configuration and database management), sql programming, using data-query tools, and using advanced report-building tools. the advanced training sessions included java programming and hos module programming. moreover, we found that community-led mutual learning amongst peers allowed participants to gain domain-specific knowledge and technical knowledge. this learning was achieved through online and offline discussions on various topics. through these discussions, knowledge was transferred and circulated amongst 9 australasian journal of information systems naparat, calhane & finnegan 2015, vol. 19, selected papers from acis healthy community & healthy commons participants. the hos community also had a peer-mentoring programme—meaning more experienced and knowledgeable participants assumed a mentoring role to help lessexperienced participants (typically those who were new to hos). we found that the effective way to do peer mentoring was by demonstrating how to perform tasks. the demonstrations were conducted both online and offline. the use of real-time interactive technologies (e.g., video conferencing, real-time messaging, and remote login) facilitated online mentoring. offline mentoring was conducted through face-to-face meetings. however, offline mentoring normally occurred between participants who lived in nearby geographic locations. otherwise, mentoring took place at special events, such as formal training sessions or the hos annual conference. we found that technology transfer allowed participants to perform more tasks. many community members began participating by providing user requirements. however, after several formal sql programming training sessions, they also began to contribute sql code to the community, which significantly improved the data-reporting functionality of the hos. moreover, formal java programming training sessions transferred sufficient programming skills to participants for them to be able to contribute the hos core code. this demonstrated that voluntary participants could carry out more difficult tasks if their skills and knowledge were enriched. many participants gained hos system administration and maintenance skills from peer mentoring and formal user training, so now they provide user-to-user help to novice participants. this emphasises the importance of “enriching knowledge” in improving participants’ knowledge and skills. by improving participants’ knowledge and skills, participants can do more tasks and improve the quality of their contribution, thus promoting a healthy commons. therefore, we present our fifth proposition as follows: proposition 5: enriching knowledge sustains opensourcing by (i) transferring and improving participants’ domain-specific and technical knowledge, thus (ii) allowing them to improve their contributions to the development of software. 4.8 face-to-face meetings an important finding was that face-to-face meetings are crucial for sustaining opensourcing as they allow participants to effectively communicate with each other, as well as develop and strengthen their casual relationships (healthy community). the community organised face-toface meetings on a regular basis, including an annual meeting, training sessions, leisure activities, and site visits to hospitals. although participants established good relationships through online interactions, offline interactions significantly enhanced their relationships. participants wanted to see each other in person if there was an opportunity. spending time together to perform leisure activities introduced another aspect of being a community participant, socialising. the hos community regularly organised leisure activities after formal meetings and training sessions. activities such as playing soccer encouraged participants to mix and to work as a team while also having fun. staying at the same hotel allowed participants to have opportunities to have informal conversations and to get to know each other better. informal dinners, drinks, or excursions to various tourist attractions were useful as icebreakers. participants, once they were more acquainted with each other, were more open to establishing close relationships. as evidenced from the field observations, several participants became good friends and were more eager to help each other. in summary, personal relationships go hand in hand with work relationships. it creates a good community culture where participants actually work with their friends, rather than just colleagues. the duality of work and fun, or work relationship and casual relationship, leads to a positive experience for the participants. personal relationships are key to enhancing the community culture and to sustaining participants’ motivation, thus enhancing the health of the community. therefore, we present our sixth proposition as follows: 10 australasian journal of information systems naparat, calhane & finnegan 2015, vol. 19, selected papers from acis healthy community & healthy commons proposition 6: face-to-face meetings sustains opensourcing by (i) building casual relationships in addition to work relationships, thereby (ii) enhancing the community culture and participant experience and (iii) reinforcing participants’ desires to continue participating. 5 conclusion this study aimed to explain the factors affecting the sustainability of opensourcing as a model of software production. to achieve this, we employed mechanism-based theorising and a case study of the production of hospital software to reveal mechanisms underpinning sustainable opensourcing. we have contributed to the opensourcing literature and practice in several ways. first, acknowledging the lack of a definition of “sustainable opensourcing,” we have defined the term and argued that sustainable opensourcing comprises two dimensions: “the healthy community” and “the healthy commons.” second, we have presented a chain of six mechanisms underpinning sustainable opensourcing. we have revealed that positive experience, the demonstration of reciprocity, marketing the community, and face-to-face meetings significantly enhance the health of the community as they retain current participants and attract new participants. enriching participants’ knowledge plays a significant role in sustaining opensourcing by allowing the community to improve the quality of contributions, thus promoting a healthy commons. furthermore, “trust in the leadership” facilitates a healthy community and a healthy commons. third, findings from the study emphasise the role of faceto-face meetings and casual relationships in sustaining opensourcing. although the community uses a number of online collaboration and communication platforms to communicate effectively, face-to-face meetings are still necessary to allow participants to strengthen their relationships and develop trust. although participants are motivated by utilitarian rewards, such as money and software use, personal relationships are also crucial as they enhance the community culture and hold the community together over the long term. this study is limited by the use of a single case study. moreover, thai culture and the domain of business (medical) are expected to influence the result of the study. we then call for more case studies conducted in different settings. future qualitative studies should be conducted in other vertical domains and/or different cultural settings. findings from future studies will allow the theory proposed in this study to be refined and improved. having presented theoretical propositions, this study provides a good foundation for future quantitative research. future quantitative research can delineate the hypotheses from the propositions to perform theory testing. references ågerfalk, p. j., and fitzgerald, b. 2008. “outsourcing to an unknown workforce: exploring opensourcing as a global sourcing strategy,” mis quarterly (32:2), june, pp. 385–409. adler, p. s. 2001. “market, hierarchy, and trust: the knowledge economy and the future of capitalism,” organization science (12:2), april, pp. 215–234. andreev, p., feller, j., finnegan, p., and moretz, j. 2010. “conceptualizing the commonsbased peer production of software: an activity theoretic analysis,” icis 2010 proceedings. benkler, y. 2002. “coase’s penguin, or, linux and ‘the nature of the firm,’” the yale law journal (112:3), december, pp. 369–446. bush, a. a., tiwana, a., and tsuji, h. 2008. “an empirical investigation of the drivers of software outsourcing decisions in japanese organizations,” information and software technology (50:6), may, pp. 499–510. charmaz, k. 2008. “constructionism and grounded theory,” in handbook of constructionist research. new york, ny: the guilford press. corbin, j., and strauss, a. 2008. basics of qualitative research 3e. thousand oaks, ca: sage publications. 11 australasian journal of information systems naparat, calhane & finnegan 2015, vol. 19, selected papers from acis healthy community & healthy commons dahlander, l., and wallin, m. w. 2006. “a man on the inside: unlocking communities as complementary assets,” research policy (35:8), october, pp. 1243–1259. fang, y., and neufeld, d. 2009. “understanding sustained participation in open source software projects,” journal of management information systems (25:4), spring, pp. 9– 50. gross, n. 2009. “a pragmatist theory of social mechanisms,” american sociological review (74:3), june, pp. 358–379. hedström, p. 2005. dissecting the social: on the principles of analytical sociology. new york, ny: cambridge university press. hedström, p., and swedberg, r. 1998. social mechanisms. an analytical approach to social theory. cambridge, uk: cambridge university press.
 hedström, p., and ylikoski, p. 2010. “causal mechanisms in the social sciences,” annual review of sociology (36:1), april, pp. 49–67. howe, j. 2006. “crowdsourcing: a definition,” retrieved 8 march, 2011 from http://crowdsourcing.typepad.com/cs/2006/06/crowdsourcing_a.html jorgensen, d. l. 1989. participant observation : a methodology for human studies. thousand oaks, ca: sage publications. miles, m. b., and huberman, a. m. 1994. qualitative data analysis: an expanded sourcebook 2e. thousand oaks, ca: sage publications. naparat, d., and finnegan, p. 2013. “crowdsourcing software requirements and development: a mechanism-based exploration of ‘opensourcing,’” amcis 2013 proceedings. o’mahony, s. 2003. “guarding the commons: how community managed software projects protect their work,” research policy (32:7), july, pp. 1179–1198. shah, s. k. 2006. “motivation, governance, and the viability of hybrid forms in open source software development,” management science (52:7), july, pp. 1000–1014. shaikh, m., and cornford, t. 2010. “‘letting go of control’ to embrace open source: implications for company and community,” hicss 2010 proceedings. von krogh, g., haefliger, s., spaeth, s., and wallin, m. w. 2012. “carrots and rainbows: motivation and social practice in open source software development,” mis quarterly (36:2), june, pp. 649–676. west, j. 2003. “how open is open enough?: melding proprietary and open source platform strategies,” research policy (32:7), july, pp. 1259–1285. williamson, o. e. 1996. the mechanisms of governance. new york, ny: oxford university press. yin, r. 2003. case study research: design and methods 3e (applied social research methods, vol. 5). thousand oaks, ca: sage publications. an earlier version of this paper was presented at the australasian conference on information systems (acis) 2014 in auckland, new zealand. copyright: © 2015 naparat, cahalane & finnegan. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. 12 http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems naparat, calhane & finnegan 2015, vol. 19, selected papers from acis healthy community & healthy commons 13 1 introduction 2 challenges to sustainable opensourcing 3 research design 3.1 data collection 4 the case study and analysis 4.1 the case study background 4.2 analysis 4.3 the positive-experience mechanism 4.4 trust in the project leader 4.5 the demonstration of reciprocity 4.6 marketing the community 4.7 enriching knowledge 4.8 face-to-face meetings 5 conclusion paper title (paper title style) australasian journal of information systems miah & genemo 2016, vol 20, design research for dss development design science methodology for expert systems devt 1 a design science research methodology for expert systems development shah jahan miah college of business, victoria university footscray park campus, melbourne, australia shah.miah@vu.edu.au hussein genemo college of engineering and science, victoria university footscray park campus, melbourne, australia abstract the knowledge of design science research (dsr) can have applications for improving expert systems (es) development research. although significant progress of utilising dsr has been observed in particular information systems design – such as decision support systems (dss) studies – only rare attempts can be found in the es design literature. therefore, the aim of this study is to investigate the use of dsr for es design. first, we explore the es development literature to reveal the presence of dsr as a research methodology. for this, we select relevant literature criteria and apply a qualitative content analysis in order to generate themes inductively to match the dsr components. second, utilising the findings of the comparison, we determine a new dsr approach for designing a specific es that is guided by another result – the findings of a content analysis of examination scripts in mathematics. the specific es artefact for a case demonstration is designed for addressing the requirement of a ‘wicked’ problem in that the key purpose is to assist human assessors when evaluating multi-step question (msq) solutions. it is anticipated that the proposed design knowledge, in terms of both problem class and functions of es artefacts, will help es designers and researchers to address similar issues for designing information system solutions. keywords: design science research; expert systems; multi-step question; method marking concept; is design 1 introduction improvement of artefact design knowledge is one of the essential components of information systems (is) design research (hevner et al., 2004; hevner, 2007). is design that applies contemporary methodologies can provide new knowledge that could improve particular is artefact designs. the knowledge of design science research (dsr) has been explored and utilised for particular is design aspects; for example in decision support systems (dss) design (arnott & pervan, 2014). this implies that dsr methodologies can have an application in improving expert systems (es) development research. one of the methods used to extend the application of dsr methodologies in es development research requires the analysis of es literature and its successful application, demonstrated in a relevant problem domain. the aim of the first part of the study was to conduct a qualitative content analysis to discover the application of dsr in existing es design studies over the period 2005 to 2015. these findings were subsequently used to form new design knowledge for developing and evaluating a specific es by implementing a problem domain-specific method (called the method marking concept – mmc) to assess algebraic questions with multi-parts in mathematics. this provided a case demonstration, now reported in this paper. es is one of the rapidly-growing is applications that require contemporary design methodologies. a good es artefact design must make a clear contribution, not only to target problem solving and its practices, but also to produce interesting methodological contributions to particular is design theory (miah, kerr & gammack, 2009). classical es design approaches, such as traditional system development and prototyping, provide little support to es australasian journal of information systems miah & genemo 2016, vol 20, design research for dss development design science methodology for expert systems devt 2 developers for conveying the methodological contributions for es design. hull and kay (1991) clearly articulated a requirement of comprehensive methodology for es design. the requirement has also been reinforced in a relatively recent study by mansiya et al. (2014) who recommended that methodology for the es design must be able to build more than scientific innovation in order to solve problems. mansiya et al. (2014) argued that the es development methodology should be able to capture theoretical, technological and organisational development requirements as the es is one of the few types of is that are often developed on artificial intelligence, and that have a defined use for human beings and have a practicalspecific application. we therefore contend in this paper that an es developer needs to employ approaches that are different from traditional methods because es development includes knowledge-based activities that are significantly different from the traditional is approach (angeli, 2010) and, as a contemporary methodology, dsr can offer benefits when particular is artefact innovations such as decision support systems are being designed (arnott & pervan, 2014). the purpose of the study was to reinforce the use of dsr for es design by exploring its current status in the literature, and showcasing an es design using dsr in a practical problem context. first, we explored the es development literature to reveal the presence of dsr as a research methodology. for this we selected relevant literature criteria and applied a qualitative content analysis in order to generate themes inductively to match the dsr components. the findings are presented through the seven guidelines of dsr proposed by hevner et al. (2004). second, by utilising the findings of the comparison, we determined a new dsr approach for designing a specific es that was guided by another result – that of a content analysis of exam scripts in mathematics. the specific es artefact was designed for addressing the requirement of a ‘wicked’1 problem in that the key purpose was to assist human assessors when evaluating multi-step question (msq) solutions in mathematics. the paper is structured as follows. following this introduction, section 2 study background introduces areas of the study including the practical problem domain. subsequently, section 3 research approaches describes how the study is conducted, section 4 es artefact design provides the details of the es artefact, section 5 es artefact evaluation includes the artefact evaluation details, and finally section 6 discussion and conclusion summarises the key contributions and further directions in the study. 2 study background 2.1 expert systems an es is a computer based is that uses artificial intelligence (ai) technologies. research in ess is centred on finding the best ways to represent human skills, expertise and reasoning in the form of computer software that mimics human activities in solving problems (mansiya et al., 2014). ess play important roles in many fields: (1) as decision support and knowledge processing systems to aid medical practitioners in taking proper decisions (boulos, 2012; kaidar et al., 2013); (2) as advisors to educators (hwang et al., 2011) and farmers (khan et al., 2008); (3) in mathematics knowledge assessment, especially in tutoring (jaques et al., 2013); and (4) in diagnosing plant diseases (khan et al., 2008). all types of ess require knowledge to operate with their functionalities. there are three main categories of knowledge: declarative, procedural and meta-knowledge (turban et al., 2005, p. 582). declarative knowledge is identified as the type of knowledge that is based on facts and truth, and which is represented in the form of factual statements that someone can understand and rationalise (anderson, 1988; giarratano & riley, 2005; stalnaker, 2012). stalnaker (2012) argues that the differences 1 wicked problems are defined through five relevant characteristics: unstable requirements due to unclear environmental context; complex interactions between components of the problem and its solutions; inherent flexibility to change design processes; critical dependence upon human users’ cognitive abilities; and critical dependence upon human users’ social abilities (hevner et al., 2004, p. 81) australasian journal of information systems miah & genemo 2016, vol 20, design research for dss development design science methodology for expert systems devt 3 between procedural and declarative knowledge arise from the way they are used, and this difference has its roots in computer science. declarative knowledge is required in all stages of the es cycle; for example domain experts share facts using declarative knowledge during knowledge acquisition. on the other hand, procedural knowledge is knowledge that requires information about doing things (jaques et al., 2013). figure 1 shows three basic building blocks that constitute a functioning es (sasikumar et al., 2007, p. 14; turban et al., 2005, p. 554-555). the larger box contains the knowledge base and inference engine, which are the kernel of a basic es, while the smaller box holds a user interface through which human users interact. the knowledge base stores the domain knowledge that the inference engine utilises to accomplish requested tasks (durkin, 1994, p. 28; giarratano & riley, 2005, p. 6). domain knowledge is extracted from experts who have expertise in a specific domain to solve the problem (giarratano & riley, 2005, p. 7). the components of the knowledge base are (1) facts and theories of problems, and (2) special heuristics or rules that direct the use of knowledge to solve specific problems in a particular domain (turban et al., 2005, p. 556). the task of the inference engine is to use the knowledge (of the knowledge base) to reach a conclusion – that is, perform tasks – through reasoning similar to that employed by humans (turban et al., 2005, p. 556). there are also other components, such as a user interface, that are required to complete the functionality of an es. figure 1: basic expert system components 2.2 es design methodologies and design science research traditional system development methodologies have been used as dominant methods for constructing es. the methodologies are defined widely in order to capture broader aspects for guiding activities; this is due to the diversity of tasks and participants involved in developing is artefacts. in this article we accept the definition of is as “a system in which human participants and/or machines perform work (processes and activities) using information, technology, and other resources to produce informational products and/or services for internal or external customers” (alter, 2008, p. 6). specialised is includes various decision support systems (e.g. group decision support systems, negotiation support systems), knowledge based systems, executive information systems, management information systems, intelligent systems and ess (arnott & pervan, 2008; power, 2010). these systems help to continuously improve ways of decision making as they provide appropriate information or knowledge support in order to perform tasks. an es is a specialised is that is aimed at solving complex problems in a particular domain, or to assist in making decisions in a particular domain by imitating human experts (sasikumar et al., 2007; giarratano & riley, 2005; turban et al., 2005). the traditional development methodologies encompass steps for identifying and analysing problems and system requirements, designing solution strategies, implementing system solutions and testing the system in the problem domain (coussement et al., 2015; shao et al., 2009; haji et al., 2014). for instance, haji et al. (2014) utilised a basic life-cycle methodology for designing an es for educational purposes. their methodology consisted of phases such as problem definitions, es architecture design and implementation. sanders et al. (2009) also used phases such as design, development and evaluation. these types of traditional australasian journal of information systems miah & genemo 2016, vol 20, design research for dss development design science methodology for expert systems devt 4 methodologies are consisted with a starting phase of knowledge acquisition or problem identification (for example, see andersen et al., 2013), then designing the es artefact by, for example, designing different components (e.g. knowledge base, inference engine, and user interface) and finally with a phase of evaluation within the practical problem context (wu et al., 2012). for the es design, in many cases, these methodologies are supportive of the knowledge that needs to be acquired in meeting the particular demands of the es design. beyond the capacity of the traditional is methodologies, design science has gained momentum in is research for designing contemporary solutions since nunamaker, chen and purdin (1990) first introduced this paradigm as an effective design methodology. hevner et al. (2004) described how the dsr is particularly relevant for modern-day is research, because it helps is researchers confront two of the major long-term issues within is design: (1) the absence of rigour in designing innovative artefacts and (2) the nature of is research outputs, many of which produce irrelevant knowledge that is not practically applicable to real-world problem solutions (orlikowski & iacono, 2001; benbasat & zmud, 1999). dsr is now generally accepted as a valid design methodology for many specialised is designs (e.g. dss development) as it contributes to knowledge and design theories beyond practical problem solving (arnott & pervan, 2014; gregor & hevner, 2013; hevner & chatterjee, 2010). it implies that progression of dsr in the is development field, in general, is significant; consequently, since es is a subset of is, a dsr methodology may also bring benefits for novel es design. dsr provides methodologies emanating from engineering and the artificial sciences (simon, 1996). it “seeks to create innovations that define the ideas, practices, technical capabilities, and products through which the analysis, design, implementation, management, and use of information systems can be effectively and efficiently accomplished” (hevner et al. 2004, p. 76). this implies that dsr is particularly relevant for innovative solution design such as an es, because it may better support designers/researchers in exploring its grounding knowledge (e.g. knowledge contribution) and embedding behavioural or human aspects into the artefact design. a major publication having an impact in is research is the study by hevner et al. (2004), in which seven dsr guidelines are offered to develop and evaluate is artefacts. since 2004, the seven guidelines have been used for innovative dss development (miah, kerr and von-helens, 2014), business modelling implementation (ahmed & sundaram, 2012) and other types of is design. the design guidelines derived from knowledge and understanding of design problems by hevner et al. (2004) are predominantly representative of dsr as it is inherently a problem solving process. according to hevner et al. (2004) the first guideline provides support for the creation of a solution artefact, the second guideline is for specifying the problem domain while the third guideline is for evaluating the solution artefact. the fourth guideline is supportive of developing an innovative artefact to address new wicked problems or improve the solution of existing problems. the fifth guideline is to ensure that the artefact must be rigorously defined, formally represented and internally consistent with problems being addressed. the sixth guideline supports the need for a search process that will find an effective solution. finally, the seventh guideline provides support for communicating the designed artefact to both technical and organisational users. these guidelines direct is research to create purposeful artefacts which are considered dsr output having different characteristics: constructs, models, methods, and instantiations (march & smith, 1995; hevner et al., 2004). improvements in artefact design knowledge are an essential component of design research (hevner et al., 2004; hevner, 2007). this understanding can be helpful in guiding is design. following this, it is important to extend design knowledge to es, another field of specialised is design. one of the methods for finding the application of a dsr methodology in es development research requires the analysis of es literature. therefore, we attempted to conduct a qualitative content analysis to discover the application of dsr in existing es design studies from the period 2005 to 2015. the findings can be used to gather new design knowledge for developing and evaluating a specific es by implementing mmc in assessing algebraic questions with multi-parts. these findings also constitute a case representation, as reported in this paper. australasian journal of information systems miah & genemo 2016, vol 20, design research for dss development design science methodology for expert systems devt 5 2.3 issues of assessing multi-step questions authentic assessment that focuses on measuring important skills – such as “intellectual quality” and making improvement in students’ learning (palm, 2008) – plays important roles in influencing students’ approaches to learning (price, carroll, o’donovan, & rust, 2011; sangwin, 2012). necessary steps should be taken to ensure that students’ understanding of a domain is fulfilled by exploring their knowledge through assessment, which acts like a “bridge between teaching and learning” (wiliam, 2005). wiliam (2005) also explained the role of questions in a japanese school, in which teachers spent a significant amount of time developing questions to measure their teaching success. assessment is used to examine and report individuals’ knowledge in a particular domain so that actions can be taken, based on the outcome of the assessment. as the authenticity of assessment content is crucial, so too is the selection of an appropriate tool that determines how and what to assess. however, in mathematics, not all the assessments that are achieved using pencil and paper can be delivered using information technologies. computer aided assessment (caa) for mathematics is limited only to checking a final answer for multistage questions (livne et al., 2007; sangwin et al., 2009; sangwin, 2012; beevers et al., 1999). the only solution to this problem has been to apply a partial credit method which could award marks for a partially correct portion of the solution, even if the final answer was wrong (beevers et al., 1999; lawson, 2012). many software packages for mathematical assessment have been developed, with the help of caa software, to implement the partial credit concept. these software programs designed questions, collected answers and assessed them in the same way (sangwin, 2012; livne et al., 2007; beevers et al., 1999). careful examination of the way in which questions have been redesigned to enable partial credit shows that this concept was first used in 1985. at this time, interactive past papers (ipp) – a computer aided learning in mathematics (calm) project – designed questions using layers of steps that were made up of a single key-step and sub-sets that belonged to this key-step (beevers et al., 1999). students were given options either to provide the final answer in the keystep option or to select the sub-sets part if their intention was to be awarded a partial credit (beevers et al., 1999). multi-step questions (msqs) have been redesigned by breaking them into sub-questions that require only one stage to solve, thereby enabling the implementation of the partial credit concept. the idea of splitting complex questions raises problems with the assessment authenticity of these types of questions, due to the facts that (1) the choice and use of algorithms to solve the question are not in themselves tested, because methods are shown to the students and (2) a particular step is forced upon the students to follow, which might be new or hard to use (lawson, 2012). the opportunity for giving valuable feedback is also reduced or lost, because the feedback information is based on the outcome of the assessment, the authenticity of which is doubtful. therefore, the problem of assessing the students’ full workings, while evaluating msqs using caa for mathematics, can be classified as a wicked problem. in mmc, a student’s ability to produce a final result can be determined by assessing the sequence of steps used to produce the steps and final answers (genemo et al., 2015). 3 research approaches 3.1 research structure the objective of the study was initially to explore es development literature to reveal the use of dsr as research methodology. to select relevant literature, specific criteria were set and qualitative content analysis was applied to inductively generate themes. the themes were then used as a basis to form categories that were matched for the presence of dsr as the es development methodologies. the findings of the comparison determined the dsr approach for designing our specific es, guided by the results of the content analysis of examination scripts in mathematics. figure 2 shows the steps taken in conducting the study. australasian journal of information systems miah & genemo 2016, vol 20, design research for dss development design science methodology for expert systems devt 6 figure 2: steps of activities undertaken in the study 3.2 paradigm and methodologies the research paradigm influences “the practice of research and the practical research needs combination of philosophical ideas, general procedures and detailed methods” (creswell, 2003, p. 8). johnson and onwuegbuzie (2004) suggested that a research project should start with a theory followed by data collection that either supports or disproves the theory. in our study, when examining documents, we investigated their text with the aim of extracting information that would be utilised in forming criteria, knowledge and methodologies that would drive designing, building and evaluating the artefact, without requiring a theory to begin with. there were two purposes for investigating these text documents in this study. the first was to study the phenomenon related to discovering both development methodologies of es artefacts by reviewing existing es research articles. the second was to analyse the examined scripts to uncover errors and solution strategies that would be used as a basis for designing an es. therefore, we found that pragmatism was the most appropriate paradigm to conduct this research as the approach was not restricted to any particular philosophy, thus giving researchers the freedom to select “the methods, techniques, and procedures of research that best meet their needs and purposes” (creswell, 2003). methodologies provide general procedures that determine the selection and use of research methods (creswell, 2003; crotty, 1998). the required research outputs help in selecting and using particular research methods (crotty, 1998) – given that sufficient information about methodologies of research is vital for conducting is research. our research used a mixed methods approach since this approach is generally associated with the pragmatism paradigm. in the mixed methods approach, both qualitative and quantitative strategies and methods are used to collect and analyse data in a single study (johnson & onwuegbuzie, 2004; creswell, 2003). 3.3 qualitative research method is researchers need “to develop a rigorous, systematic inquiry” (creswell, hanson, clark plano, & morales, 2007, p. 236) to conduct research. this can be done by using qualitative, or quantitative, or mixed methods in a single research project. more details about qualitative research are given in the subsequent paragraphs. guest et al. (2013) suggest that it is important to define the purpose and focus of any qualitative study. creswell (2003) argues that the purpose of qualitative research is “to understand a particular social situation, event, role, group, or interaction. such research is largely an investigative process by which the researcher gradually makes sense of a social phenomenon by contrasting, comparing, replicating cataloguing and classifying the object of study” (creswell, 2003, p. 226). creswell’s explanation of the function of qualitative research shows the significant roles that qualitative research plays in revealing rich information about phenomena. in the documents that we analysed, the phenomena that were investigated were australasian journal of information systems miah & genemo 2016, vol 20, design research for dss development design science methodology for expert systems devt 7 the errors and solution strategies in the students’ works, and insights that could be gleaned about es development as the existing es research articles were being reviewed. the research question is considered to be the driver of the whole research. according to ellis and levy (2008), “the nature of what is going wrong – the problem – very much sets the parameters for what can be done” (p. 22). creswell et al. (2007) also state that the research question “informs the approach or design used in qualitative research to collect and analyse the data” (p. 238). the most suitable research design is required if the identified qualitative research question is to be studied effectively. according to creswell et al. (2007), there are five prevalent qualitative research designs which highlight “the procedures involved in actually conducting qualitative studies” (p. 237). creswell et al. (2007) refer to research design as “approaches to qualitative research that encompass formulating research questions and procedures for collecting, analysing, and reporting findings” (p. 237). from the five qualitative research designs, the phenomenology design seems most appropriate for collecting and analysing documents (in our case, both forms of documents). however, the inductive content analysis suggested by elo and kyngäs (2008) seems more appropriate for the reason-building concepts and/or categories from the content analysis of the documents. qualitative research has “an inductive and flexible nature” (guest et al. 2013, p. 4), and the flexibility aspects offer the prospect of researchers being able to alter “the sampling proce¬dures during the data collection process based on incoming data” (guest et al. 2013, p. 4). in qualitative research, generally, the research questions are open-ended, to enable researchers to probe the problem for further information; this process of probing for more information leads to producing more-valid research data (guest et al. 2013, p. 4). in our study, the research materials were approached with this type of open-ended relevant question in mind to allow us to thoroughly investigate the research objects iteratively. these qualitative research objectivities of identifying and exploring concepts and categories, describing, and explaining them, then assisted in accumulating the appropriate knowledge. the accumulated knowledge was then useful in constructing an es artefact. these objectives can be contrasted with the purposes of qualitative research identified by creswell (2003) – as discussed above. 3.3.1 qualitative document content analysis content analysis is one of the qualitative and quantitative research methods that is applied to data obtained from human interaction processes – verbal or visual – and written documents for the purpose of analysing the data (forman & damschroder, 2008; mayring, 2000; elo & kyngäs, 2008). in general, written data sources “can include published and unpublished documents, company reports, memos, letters, reports, email messages, faxes, newspaper articles and so forth” (creswell, 2012, p. 223); this underlines the ability of researchers, in qualitative studies, to exploit documents as one of the important sources of information to explore phenomena. these documents are not restricted to one particular problem or domain. the expansion of is research into managerial and organisational issues has created greater interest in using qualitative research methods (myers, 1997). there are many is researchers who have been applying both qualitative and quantitative analyses in different research domains. for example, al-debei and avison (2010) used content analysis in their examination of a business model in the field of is using relevant literature, while arnott and pervan (2012) explored the dsr for dss research articles employing a content analysis method. indulska and recker (2010) also conducted qualitative content analysis on dsr articles to establish the claims of these articles on usage of a dsr paradigm in their research. in our study, we followed this same path for two different purposes: 1) to analyse development methodologies in previous es research articles and 2) to identify common errors and solution strategies in the examined scripts. deductive and inductive analyses are widely employed in qualitative research. inductive analysis is an appropriate method when “there are no previous studies dealing with the phenomenon or when it is fragmented” (elo & kyngäs, 2008). in our research, the phenomena in both types of documents needed to be evaluated using the inductive method, in particular for the classification and categorisation of the attributes in the phenomena. the categories australasian journal of information systems miah & genemo 2016, vol 20, design research for dss development design science methodology for expert systems devt 8 were formed from the analysis of the es articles and examination scripts. in general, the result of a qualitative content analysis is the summary of the original information and is presented in the form of concepts and/or categories that express the investigated phenomenon (elo & kyngäs, 2008). 3.4 document analysis process the es documents were collected to uncover information about the application of dsr in the es development practices, while students’ examination scripts were assembled for the exploration of common errors and question-solving strategies. in the absence of the usage of a dsr paradigm, the analysis focused on revealing methodologies utilised in developing es artefacts. each type of document was analysed for a different purpose, and the purposes that were selected were based on the criteria that they should contain qualitative data to answer the research questions. for the analysis process applied to es articles, see section 3.4.1; for examination scripts, see section 3.4.2. before and during the es articles’ content analysis processes, researchers frequently revisited and studied dsr articles – especially hevner et al.’s (2004) design guidelines and walls’ (1992) information systems design theory (isdt) – so that relevant data could be explored/observed and extracted. also, in content analysis, “there are no systematic rules for analysing data” (elo & kyngäs, 2008). however, as guidance to perform the analysis in our research, we followed the three phases stated by elo and kyngäs (2008): preparing, organising and reporting. 3.4.1 es article analysis the first concern in the preparation phase is to determine what to analyse that is pertinent to the research problems. the authors searched research library databases using the terms ‘design science’, ‘expert systems’, ‘rule base’ and ‘rule based’. extensive search efforts were made to find the specific type of es research articles that employed a dsr paradigm in creating es artefacts, but no research article had, at that time, been found. therefore, we decided to look for the presence of information about basic components of es; these are knowledge bases, knowledge elicitation techniques, knowledge representation techniques and inference engines, and design processes. the other conditions were that the es articles should contain descriptions of es design processes, evaluation methodologies and rigorous processes. once the purpose of analysis is determined, it is important to decide on the most suitable units of analysis. the selected es articles having the specifications detailed above would be regarded as the relevant unit of analysis. for example, kim and kuljis (2010) employed content analysis of web-based content, in that they selected the profile page as the unit of analysis, using the rationale that the page contained enough data for their analysis to answer the research questions. elo and kyngäs (2008), referring to this issue, stated that “deciding on what to analyse in what detail and sampling considerations are important factors before selecting the unit of analysis”. for our research, we faced challenges in resolving the sample and size of the es articles to be gathered. many of the articles that claimed to concern es artefact development and implementation did not include the data that was required to fulfil our objectives. we therefore decided to choose the article selection measures based on the articles’ content. according to kim and kuljis (2010), “what units need to be identified for sampling will be determined by the research question or hypothesis” (p. 373). as for the size of the sample used, for exploratory purposes, we accepted that the number of es articles analysed was reasonable in our context of study; specifically, to discover es creation and/or developing methodological patterns. more criteria were added to the es article selection process. these criteria were (1) to include articles with publication dates from 2005 to 2015, (2) to exclude, as much as possible, articles published in unranked journals, (3) to exclude conference papers and (4) to include only rulebased es types. the rationale for applying these criteria was to gather enough relevant and reliable sample articles. the reason for selecting only rule-based ess was for the later implementation of mmc, when assessing msq solutions, using this type of es; the lessons from the findings of the analysis were used in introducing the dsr paradigm into the australasian journal of information systems miah & genemo 2016, vol 20, design research for dss development design science methodology for expert systems devt 9 development of the mmc es artefact. resolving the problems of choosing apposite articles helped us to move to the articles’ content analysis process. the actions that were taken in the preparation phase led to the gathering of 53 relevant rulebased es articles. however, only 2o articles were analysed as they were the case representatives of the rule-based es literature originated in different problem domains. the problem space defined in the selected 20 articles was somehow matched through to the five relevant characteristics of wicked problems. also, these case representatives were somehow adopted different development methodologies that were analogous to dsr components. the researchers needed to familiarise themselves with the content of the articles. elo and kyngäs (2008) propose that researchers should fully explore the units to be analysed so as to comprehend the contents, and so that meaningful categories and/or concepts can be developed from the analysis outcomes. “document analysis involves skimming (superficial examination), reading (thorough examination), and interpretation” (bowen, 2009). in our research, the skimming actions were performed during the gathering of the articles. the details of exploring the articles were accomplished during the organising phase. table 1 shows the sources of the sample articles analysed in this study. name of journals publishers/ countries impact factors/ scimago ranking no of es published articles percentage of published rulebased es design articles expert systems with applications elsevier, uk 2.53/q1 5 25% decision support systems elsevier, uk 2.4/q1 2 10% int. j. of computer and communication tec. inder-science, india 2.40/q4 1 5% the arts in psychotherapy elsevier, uk 0.79/q3 1 5% int. j. of science and modern engineering ijisme, india 1.07/q4 1 5% int. j. of computer science ijcsi, 0.44/na 1 5% assembly automation emerald, us 0.91/q2 1 5% automation in construction elsevier, uk 2.51/q1 1 5% j of software engineering and applications scientific research 0.94/na 1 5% int. j. intelligent systems and applications modern education and computer science 0.38/na 1 5% electrical power and energy systems elsevier, uk 2.19/q1 1 5% computer-aided civil and infrastructure engineering willey, us 4.90/q1 1 5% educational technology & society ieee computer society, us 0.82/q1 1 5% knowledge based systems elsevier, uk 2.70/q1 2 10% table 1: sources of sample articles in the organising and reporting phases, the inductive content analysis comprises “open coding, creating categories and abstraction” (elo & kyngäs 2008, p. 109). according to elo and kyngäs (2008) during the open coding, researchers are meant to take notes and headings in the text while reading the sample. in our research, the gathered copies of the es articles were saved in australasian journal of information systems miah & genemo 2016, vol 20, design research for dss development design science methodology for expert systems devt 10 hard and electronic formats. in the electronic format, headings and notes were added as comments to the articles while they were being read. similarly, text highlighting, copying and pasting each article’s selected content were used to gather and code relevant data. further reading of the commented articles resulted in correcting and/or adjusting the coded data to ensure the collection of apt and sufficient data from the articles. here, the act of selecting headings was to form a new categorisation (elo & kyngäs, 2008). elo and kyngäs (2008) also refer to the categorisation process as including the interpretation process, which helps with grouping into categories; these categories are then used to describe the phenomenon that has been analysed. in our study, the headings that emanated from the articles’ coding process were grouped to create categories, and these categories were regrouped under higher order headings or categories. figure 3 shows the categories that have been inductively generated in our study. figure 3: categories that have been inductively generated through the themes in this analysis, 15 themes were inductively generated from the samples. the themes that were based on the key elements of es design methodology can be informed through the seven category names. although we found a few inconsistent remarks in generalising the categories, in general (1) the description of an es solution helped to determine artefact types and its purposes, (2) both descriptions related to problem definitions and knowledge acquisition details informed us on problem relevance, (3) the description of es solution evaluation denoted the artefact’s evaluation, (4) descriptions of the design process, components and knowledge representation indicated the contribution of the study, (5) the details related to used methodologies and approaches gave rigour of the study, (6) appropriate approaches used in designing the inference engine denoted the research as a search process and (7) the details related to the es evaluation method defined how the es design is communicated to its target audience. we matched these categories with hevner et al.’s (2004) definition of seven principles, and through these we anticipated that the activities of es design research – such as those used for identifying problem relevance and overall design and evaluation of the es artefact – can be well-recognised and accomplished to achieve design research outcomes. the following reporting phase gives details on how the categories are reported as part of our study. the process of abstraction takes the following steps: (1) generating categories that reflect the general description of the themes using category names that describe the characteristics that the categories refer to, (2) iterating the abstraction process until the categories’ representation of the general description of the phenomena is satisfactory and (3) grouping the subcategories that share similar specifications under higher order categories. the created categories in figure 3 were used to explain the development methodologies of es artefacts and processes through the creation of general categories that reflect “content-characteristic words” (elo & kyngäs, 2008); that is, 15 themes. these category names replicate the keywords and terminologies that are used in the development of es artefacts. finally, in the reporting phase, australasian journal of information systems miah & genemo 2016, vol 20, design research for dss development design science methodology for expert systems devt 11 we reported the findings through the skeleton of hevner et al. (2004) as discussed above. in appendix a the details can be viewed in table a1. 3.4.2 examination script analysis in any es development, the knowledge base is one of the main components of an es artefact. in our target problem domain, one of the knowledge sources was examination scripts that were required for analysis in designing the es. the insights obtained from the selected scripts were able to provide relevant data to answer pertinent research questions related to the phenomenon of a msqs solution assessment by applying the mmc method. the criteria – to include questions from the scripts in the qualitative analysis study – were carefully examined and documented. in this, we used mathematical knowledge to decide appropriate criteria for question selection. in the preparation phase, there are several factors that determine the selection of the unit of analysis. this unit is determined by the nature of the question that has been selected from the scripts; that is, the scripts must contain the appropriate knowledge to be assessed through their reasonable question solution. the knowledge domains that we assessed in these examinations were (1) solving linear equations using the elimination method and back substitution and (2) integration using a substitution technique and by parts. both types of question’s solutions require steps that use an algorithm to produce the final answers. the comprehension of detailed domain knowledge assists the researcher in deciding on the type of documents to be analysed to solve the research problems. the appropriate domain knowledge is checked to confirm the data’s integrity and the validity of the content analysis outcome. the data in the question’s solution can be used to identify solution strategies and possible errors in the student’s solution. using the scripts’ domain knowledge criteria, we collected 732 scripts. the scripts were scanned to define the sample size and specification. to achieve these, the scripts were divided into subjects; then, related questions were regrouped within the defined subjects. the rearranged questions were also segmented into two groups based on the marks awarded by the examination assessors to the questions’ solutions. questions whose solutions could attain a possible score of 6 marks were split into two groups. those questions whose solutions could attain part marks of 0-3 were allocated to one group. the second group was composed of questions whose solutions could attain part marks of 4-6. questions whose solutions could attain part marks of only 3 or 4 were assigned to either group. the same method was applied to questions that had 5 marks assigned to a correct solution. in this subdivision, the questions solutions with lower awarded marks that have 0-2 marks grouped together, while the others with the marks 3-5 was grouped in different set. again in this group, with 5 with full mark, questions solutions with marks 2-3 were assigned to the lower or higher mark group. the above mentioned grouping of scripts formed our sampling process. equal numbers of scripts, from each subject, containing equal number of low and high marks were chosen for the document analysis process. out of 732, only 532 scripts were selected through the sampling process. in the current paper, as the analysis process is on-going, laborious as well as time consuming, 45 examination scripts as case representatives were selected for the analysis process; these scripts were combined from 3 different examinations of the mathematical foundational subject. the knowledge examined in the questions was the same, although variations in the variables’ labels and coefficient values were different. half of the scripts contained solutions whose questions lay within a low range of marks, while the other half lay within the higher range of marks. as mentioned above, the scripts were divided based on the subject, related questions and the number of marks given to the students’ work. considering rules of research ethics (defined in our approved ethical application), we removed the identities of students from the scripts, numbered each script, then copied the pages that included the solutions to be analysed. each copied page was given the script number. we kept the details of the subject and the examination-year of the scripts. the list of each question’s information, showing where it originated, assisted us to track questions; this tracking process was very useful during the code formation, collection and grouping processes. australasian journal of information systems miah & genemo 2016, vol 20, design research for dss development design science methodology for expert systems devt 12 the organising phase involved induction analysis. the copied questions’ solution scripts were examined and the open coding actions – which involved evaluating the algorithm applied, and the results of applying it – were performed. the explorations revealed whether the correct solution strategies had been applied to solve the questions, and whether or not any errors had been made during the method’s application. the relevant codes were recorded in a separate note so as to keep the original data intact, because further re-reading was required to readjust the initial observation and/or to add more code to elicit more relevant data from the scripts; therefore, it was important to use separate media to record the analysis activities and findings. we discovered the importance of this issue during script analysis, when adding heading and titles to the script itself accidently erased the original script information; this led to discarding that script from the scripts’ analysis list. at the completion of the coding process another 7 scripts were excluded because they did not encompass examinable information to resolve or assist in answering the research questions. the next process was to gather all these codes from our note-book record to form categories. as for the solution strategies that were used by learners, general categories that are shown in figure 4 (comprising figures a, b and c) have been constructed. the applied strategies can be compared to the expected, taught methods to find out how much these strategies deviate from the expected solution methods. this is possible by conducting statistical analysis to realise the frequency of these solution methods by students. a description of the formation of these categories is presented in the next section. the selected questions in the subject (called engineering mathematics 1) asked the problem solvers to find the values of variables using matrix methods. although the steps followed in the examination solution’s scripts of 2011-2015 were the same, the coefficients of the equations’ variables, and the values assigned to equations, were different in each examination. therefore, each year’s examination solutions were analysed separately to collect solution methods that applied in the elimination of variables, and the computation of variables’ values. the errors made in the examination scripts were similar; therefore, these errors could be categorised together and checked for the frequency of errors. figure 4 (a) shows the categories that display equations and variables belonging to the equations; figure 4 (b) shows the variables category and its sub-categories related to the variables’ elimination process along with the result of the elimination process; figure 4 (c) shows the variables’ values finding process categories and its sub-categories, and the possible outcomes of the variables’ value-finding process. the categories that were produced from the qualitative content analysis result were then used as a source to create a questionnaire to be employed in the quantitative content analysis phase. the questionnaire was used to collect quantitative data from examination scripts that had been reserved for the quantitative phase study. the scripts having the same types of content were divided – using the same criteria mentioned above – into two groups. one group was analysed using a qualitative content analysis method while the other group was set aside for the quantitative content analysis method. in this paper, we have excluded the details of the quantitative content analysis. australasian journal of information systems miah & genemo 2016, vol 20, design research for dss development design science methodology for expert systems devt 13 figure 4: categories formed from the 82 examination scripts’ analysis 3.5 es artefact development method motivated by the demonstrated use of dsr in es design, we employed hevner et al.’s (2004) seven guidelines for conducting our design study to develop the mmc es artefact. as a specific approach to es development, we justified the seven guidelines through the traditional phases of waterman’s (1986): identification, conceptualisation, formalisation, implementation and testing. waterman’s phases have been demonstrated as being in general use over recent years for es development. on the other hand, hevner’s guidelines are generally applicable to any is research. we have established an association between hevner’s guidelines and waterman’s es development phases to show how the dsr approach can offer benefits in the development of the mmc es artefact (see figure 5 below). australasian journal of information systems miah & genemo 2016, vol 20, design research for dss development design science methodology for expert systems devt 14 figure 5: development phases of the mmc es artefact in the identification phase – conforming to hevner et al.’s (2004) “design as artefact and problem relevance” guideline – the research requirements were identified through the research questions’ phenomena studies and the literature review, which identified related knowledge and issues that required further research investigation. the conceptualisation phase conformed to hevner et al.’s (2004) “research rigour and design as a search process” guidelines. in this phase, elements in the document analysis findings were searched and relationships between them established; interactions between elements and the interactions’ controls were also determined in this phase. the formalisation phase conformed to hevner et al.’s (2004) “design as a search process” guideline. at this stage, the knowledge that was obtained in the conceptualisation phase was transformed into a useable form for es design. at the implementation phase – conforming to hevner et al.’s (2004) “design as a search process” guideline – the formatted knowledge was stored in the es for the effective assessment operation. the testing phase conformed to hevner et al.’s (2004) “design evaluation” guideline, which verified the element’s existence, the validity of relationships and interactions, and the artefact’s performance and utility. table 2 shows the details of the processes illustrated in figure 5. the flowchart sequences and activities (in figure 5) are based on waterman’s es development phases and hevner et al.’s (2004) guidelines. australasian journal of information systems miah & genemo 2016, vol 20, design research for dss development design science methodology for expert systems devt 15 identify es feasibility identifying requirements for possible devlopment of es. acquire knowledge 1 continue extracting knowledge from document analysis findings (errors and solution algorithms and classifying them into groups) until satisfied that enough knowledge is acquired. transform the knowledge 2 transform the acquired knowledge and do this rigorously until sufficient knowledge is transformed into the format that is recognised by the es. add the knowledge to the knowledge base 3 put the knowledge into the knowledge base and run the es prototype. this is the implementation phase of the es. test/evaluate the es test the system for the processes in the boxes numbered 1-3. also validate the performance and utility of the es. based on the evaluation results, go to one or more of the numbered processes. 1, 2, 3 if the evaluation feedback requires modification, go to the processes 1, or 2 or 3 or combination of them to satisfy the requirements in the feedback returned by the evaluation/test phase. conclude and report the research contribution this shows the end of successful artefact development and, as the result, communicate the research contribution to the knowledge base. table 2: details of processes in the mmc es artefact development phases flowchart 4 es artefact design walls et al. (1992) provided a framework to describe the design artefact. the framework is wellknown as information systems design theories (isdt) for effectively guiding the artefact design within a problem context. we adopted this framework to define the artefact as a solution model, through the components of design such as meta-requirements, meta-design, kernel theories and testable hypothesis. table 3 shows the details of the es artefact. the architecture of the mmc es artefact is shown in figure 6; it includes the knowledge base, knowledge source, blackboard, inference engine, and user interface. the information – facts and rules – in the knowledge base that is required for the operation of the mmc es artefact is constructed from the knowledge source. the blackboard module – also called the working memory in rule-based es – is a medium acting as a database; its purpose is to store data about a specific case of the current problem that the es is focusing on (sasikumar et al., 2007). the inference engine takes input from users and attempts to resolve problems using the control through the rule-based technique. the purpose of the user interface is to allow users to interact during the operation of the artefact. australasian journal of information systems miah & genemo 2016, vol 20, design research for dss development design science methodology for expert systems devt 16 theory component description as evident in this research design product meta -requirements class of problems to which theory applies wicked problem, complex problems that are made up of many interrelated elements, assessment using mmc meta-design class of artefacts that meet meta-requirements rules based approach, procedural and declarative knowledge formation, solution steps tracking kernel theories theoretical grounding of the design product knowledge of assessment (theories for what to be assessed) concept by bloxham and boyd (2007); theories of anderson (1988) for declarative knowledge evaluation of design product outcomes reflection and evaluation of artefact in use setting descriptive method also seems appropriate for evaluating mmc artefact model while for method and instantiation artefacts, testing and analytical approaches will be utilised. design process design method procedures adopted for design and co-constructing artefact content analysis in the process of dsr for artefact design kernel theories theoretical grounding of the design process assessment theory in designing to the requirements, and math knowledge (based on mmc) and by applying the design guideline 6 of hevner et al. (2004) evaluation of design process outcomes reflection and evaluation of design process in relation to achievement in terms of achievement of metarequirements, and metadesign we evaluated the artefact in all design and development phases. table 3: the design details of the mmc es artefact figure 6: overall architecture of the proposed solution artefact 4.1 assessing msq questions’ solutions figure 7 illustrates the operational process of the proposed es artefact to assess an msq solution. the inference engine uses available knowledge to determine if there is knowledge that australasian journal of information systems miah & genemo 2016, vol 20, design research for dss development design science methodology for expert systems devt 17 can be used for the step solution submitted by a user. if there is no solution, the system records the reason for the absence of the solution and checks if all the steps that are required to answer the question have been fully used. if the knowledge to assess the current step is available, the system assesses the solution at that step, assigns a mark to it, and gives the user the choice to continue to the next step or to stop answering the question. if all the steps are completed, or the user has requested to end the assessment process before answering the question fully, the system adds the steps’ marks, and displays the total mark. that ends the assessment process. figure 7: working principle of the proposed solution artefact the mmc es artefact function is based on providing questions, an answer, and feedback as well as working areas for users. users select the number of steps that the current question requires to produce the final answer, and go through each step to work on the step’s answer. the final step would be the answer to the msq. at any time, users can stop working on the current step’s solution and submit the whole answer to end the current question’s answering process or session. the system displays each step’s answers based on the number of steps answered. our es artefact assesses and awards answers to steps in questions based on the criteria set by the assessors, who have the privilege to prepare question and assessment criteria for storing in the system. at the end of the session, or upon the request of the users, the current question’s answer, with feedback, is displayed. the system allows developers to add features to the functionality of the mmc es artefact, such as modifying or adding knowledge to the knowledge base or setting new rules. if the artefact is not able to assess the user’s submitted solution, the artefact saves the submitted solution, giving developers or/and assessors the opportunity to review and take actions that enhance the artefact’s functionality. 5 es artefact evaluation the type of evaluation criteria are determined by the research problems, the objectives of designing problems solutions, and the research questions that are originated in the particular environment (hevner et al., 2004; vaishnavi and kuechler, 2015). the two phenomena that have been explored in this article are es development research articles and the content of examination scripts. the process of evaluating the two sources began at the point where the research problems were established; the criteria and methodology utilised to analyse the two types of documents have been explained in detail in the content analysis sections, and these criteria, and the methodology, included the evaluation process. australasian journal of information systems miah & genemo 2016, vol 20, design research for dss development design science methodology for expert systems devt 18 for the evaluation of the es artefact, we used an experimental approach as it is one of the defined evaluation methods in gregor and hevner (2013). the criteria that govern or control the evaluation execution or process is based on the findings and knowledge of the examination scripts and es articles’ content analyses. the findings from the analysis of examination scripts revealed the complexity of the nature of implementing mmc using an it artefact. gregor and hevner, (2013) suggested that the evaluation process should go through a proof-ofdemonstration to confirm the significance and value of the artefact. therefore, the proposed es is evaluated in terms of its validity, utility, quality and efficacy. for the research contribution to be credible, the design evaluation should clearly reveal the practical contribution that the research has made to the application’s context/environment (gregor and hevner, 2013). the practical contribution of the proposed artefact is then demonstrated through the system functions using the mmc approach. within the capacity of our artefact as a model demonstration, the design approach was guided by the target problem-specific details. further improvements are required to make the artefact completely instantiated to realise mmc for its quality and efficacy. in relation to the validity of the proposed artefact, figure 7 showed the key elements of the findings that have been executed in the solution strategies of the es. 6 discussion and conclusion this study has reinforced the application of dsr knowledge for es development studies. we investigated its current status and suitability for showcasing an es design using dsr that was guided by context-specific findings in a practical problem context. the proposed artefact has been called the mmc es artefact, because our solution strategy was based on the method marking concept. to investigate the relevant es design literature we adopted a qualitative content analysis to inductively generate themes. the themes of es design were used as a basis to form categories that were matched for the presence of dsr, and represented its applications as the es development methodology. the findings of the comparison determine importance of dsr approach for designing our specific es guided by the result of the content analysis of examination scripts in mathematics. the primary contribution of the study is that by conducting a rigorous dsr study using content analysis, we developed basic knowledge about the target artefact design. further, we introduced the adaptation of content analysis to conduct dsr projects. we have demonstrated how an es artefact should be designed through investigating phenomenon relevant to a literature base and practical application area. the two phenomena that have been explored in this article are es development research articles and examination script content. the process we generated to evaluate the two sources started from the point where the research problems were outlined; the criteria and methodology utilised to analyse the two types of documents have been explained in detail. the secondary contribution of the study is that it has described an innovative es artefact design by showcasing it in addressing a practical problem in the education domain. we used walls et al.’s (1992) isdt framework to define the proposed es artefact through the components of design such as meta-requirements, meta-design, kernel theories and a testable hypothesis of the artefact as a product. according to walls et al.’s (2004) different levels of usage of isdt framework, we fulfilled the conditions of level 1 knowledge contribution that represents studies that define design features and requirements of a new class of is design, when they use the walls’ framework. we also approached level 2 – knowledge contribution, as we also outlined the meta-requirements for the new class of is (e.g. mmc es artefact) and its instances. mainstream is researchers over the past decades have shown limited interest in es development research of the type reflected in our results of the content analysis. none of the top eight is journals (information systems journal, information systems research, european journal of information systems, journal of information technology, journal of management information systems, management information systems quarterly, information and australasian journal of information systems miah & genemo 2016, vol 20, design research for dss development design science methodology for expert systems devt 19 management, and journal of the association for information systems) published fully developed rules-based es development studies. however, several engineering and other domain-specific journals, indicated in table 2, published fully-researched es design articles to meet the demand for knowledge in the field. although our contribution is an initial solution model, the work described in this paper has defined the relevance and importance of the artefact, which shows its competence to implement mmc and make valuable contributions to the education problem domain. being able to assess the complete working of learners’ solutions to questions has major advantages for all stakeholders – learners, educators, policy makers and any allied bodies in the education and associated domains. an assessment outcome can influence teaching and learning methods and contents. the same concept can be applied in similar domains, such as physics, chemistry and any other discipline where solutions to wicked problems are required. there are many factors that might affect the functional characteristics of mmc artefacts being built; these are the data source selection criteria, methods and types of knowledge extracted, and the extent of design and evaluation involved. future research will be needed to minimise the limitations associated with the functionality of these artefacts. references al-debei, m. m. and avison, d. (2010). developing a unified framework of the business model concept. european journal of information systems, 19, 359-376. alter, s. (2008). defining information systems as work systems: implications for the is field, european journal of information systems, 17(5), 448-469 anderson, j. r. (1988). foundations of intelligent tutoring systems, lawrence erlbaum associates, inc., publishers, new jersey. andersen, m. gagne, j m l and kleindienst, s (2013). interactive expert support for early stage full-year daylighting design: a user’s perspective on lightsolve, automation in construction, 35, 338–352 angeli, c. (2010). diagnostic expert systems: from expert’s knowledge to real-time systems, in advanced knowledge based systems: model, applications & research (eds. sajja & akerkar), 1, 50 – 73 arnott, d. and pervan, g. (2008). eight key issues for the decision support systems discipline, decision support systems, 44(3), 657-672 arnott, d. and pervan, g. (2012). design science in decision support systems research: an assessment using the hevner, march, park, and ram guidelines. journal of the association for information systems, 13, 923-949. arnott, d. and pervan, g. (2014). a critical analysis of decision support systems research revisited: the rise of design science. journal of information technology, 29(14), 269– 293. beevers, c., wild, d., mcguire, g., fiddes, d., & youngson, m. (1999). issues of partial credit in mathematical assessment by computer, research in learning technology, 7(1), 2632. benbasat, i., and zmud, r. w. (1999). empirical research in information systems: the practice of relevance, management information systems quarterly, 23 (1) 3-16. bloxham, s., and boyd, p. (2007). developing effective assessment in higher education: a practical guide: a practical guide: mcgraw-hill education,uk. boulos, m. n. k. (2012). expert system shells for rapid clinical decision support module development: an esta demonstration of a simple rule-based system for the diagnosis of vaginal discharge, healthcare information research, 18, 252–258. australasian journal of information systems miah & genemo 2016, vol 20, design research for dss development design science methodology for expert systems devt 20 bowen, g. a. (2009). document analysis as a qualitative research method. qualitative research journal, 9, 27-40. creswell, j. (2003). research design: qualitative, quantitative, and mixed methods, sage publications, california, usa. creswell, j. w. (2012). educational research: planning, conducting, and evaluating quantitative, pearson. creswell, j. w., hanson, w. e., clark plano, v. l., and morales, a. (2007). qualitative research designs: selection and implementation, the counselling psychologist, 35(2), 236-264 crotty, m. (1998). the foundations of social research: meaning and perspective in the research process. coussement, k., benoit, d.f., and antioco, m. (2015). a bayesian approach for incorporating expert opinions into decision support systems: a case study of online consumersatisfaction detection, decision support systems, 79, 24–32. durkin, j. (1994). expert systems: a view of the field, ieee expert: intelligent systems and their applications, 11 (2), 56-63. ellis, t. j. and levy, y. (2008). framework of problem-based research: a guide for novice researchers on the development of a research-worthy problem. informing science: international journal of an emerging transdiscipline, 11, 17-33. elo, s. and kyngäs, h. (2008). the qualitative content analysis process. journal of advanced nursing, 62, 107-115. forman, j. and damschroder, l. (2008). qualitative content analysis. empirical research for bioethics: a primer. oxford, uk: elsevier publishing, 39-62. genemo, h., miah, s.j and mcandrew, a. (2015). a design science research methodology for developing a computer-aided assessment approach using method marking concept, a design science research methodology for developing a computer-aided assessment approach using method marking concept, education and information technologies, 10.1007/s10639-015-9417-1, 1-16 giarratano, j. c. and riley, g. (2005). expert systems: principles and programming, cambridge, mass.: 4th ed.,thomson course technology gregor, s. and hevner, a. r. (2013). positioning and presenting design science research for maximum impact. management information systems quarterly, 37, 337-356. guest, g., namey, e. e. and mitchell, m. l. (2013). collecting qualitative data: a field manual for applied research, thousand oaks: sage publications. haji, e. el., azmani, a., and harzli, m.el. (2014). multi-expert system design for educational and career guidance: an approach based on a multi-agent system and ontology, international journal of computer science issues, 11(5), 46-52 hevner, a. and chatterjee, s. (2010). design research in information systems: theory and practice, springer science & business media. hevner, a. r., march, s. t., park, j. and ram, s. (2004). design science in information systems research. management information systems quarterly, 28(1), 75-105. hevner, a.r. (2007). a three cycle view of design science research, scandinavian journal of information systems (19)2, 87–92. hull, l.g. and kay, p. (1991). expert system development methodology and management, proceedings of the ieee/acm international conference on developing and managing expert system programs, washington, dc australasian journal of information systems miah & genemo 2016, vol 20, design research for dss development design science methodology for expert systems devt 21 hwang, g-j., chen, c-y., tsai, p-s.,and tsai, c-c. (2011). an expert system for improving webbased problem-solving ability of students, expert systems with applications, 38 (7), 8664–8672. indulska, m. and recker, j. (2010). design science in is research: a literature analysis. information systems foundations, 285. jaques, p. a., seffrin, h., rubi, g., morais, f. d., guilardi, c., bittencourt, i. i. et al. (2013).rule-based expert systems to support step-by-step guidance in algebraic problem solving: the case of the tutor pat2math. expert systems with applications, 40 (14), 5456-5465 johnson, r. b., and onwuegbuzie, a. j. (2004). mixed methods research: a research paradigm whose time has come, educational researcher, 33(7), 14-26. kaidar, s. m., hussain, r. i., bohani, f.a., sahran, s., zainuddin, n.b., ismail, f., thanabalan, j., kalimuthu, g. and abdullah, s.n. h. s. (2012). brain tumor treatment advisory system. soft computing applications and intelligent systems. springer, uk khan, f. s., razzaq, s., irfan, k., maqbool, f., farid, a., illahi, i. and amin, t. u (2008). wheat: a web-based expert system for diagnosis of diseases and pests in pakistani wheat, proceedings of the world congress on engineering, 2–4. kim, i. and kuljis, j. (2010). applying content analysis to web based content, proceedings of the 32nd ieee international conference on information technology interfaces (iti), cavtat/dubrovnik, 283 – 288. lawson, d. (2012). computer-aided assessment in mathematics: panacea or propaganda?, international journal of innovation in science and mathematics education (formerly cal-laborate international), 9(1). livne, n. l., livne, o. e., and wight, c. a. (2015). can automated scoring surpass hand grading of students’ constructed responses and error patterns in mathematics, in conole et al. (ed) design for teaching and learning in a networked world, springer. mansiya, k., alma, k.z., marzhan, m.t.m. and kanat, n. (2014) the methodology of expert systems. international journal of computer science and network security, 14 (2 ), 1462. march, s.t. and smith, g. (1995). design and natural science research on information technology, decision support systems 15(4), 251-266 miah, s.j. kerr, d. and von-helens, l (2014). a collective artefact design of decision support systems: design science research perspective, information technology & people, 27 (3), 259-279. miah, s.j., kerr, d. and gammack, j. (2009). a methodology to allow rural extension professionals to build target specific expert systems for australian rural business operators, expert systems with applications, 36 (1), 735-744 myers, m. d. (1997). qualitative research in information systems. management information systems quarterly, 21, 241-242. nunamaker, j.f., chen, m., and purdin, t.d.m. (1991). systems development in information systems research, journal of management information systems, 7(3), 89-106. orlikowski w. j. and iacono c. s. (2001). desperately seeking the “it” in it research – a call to theorizing the it artifact, information systems research, 12 (2), 121-134 palm, t. (2008). performance assessment and authentic assessment: a conceptual analysis of the literature?, practical assessment, research & evaluation, 13(4), 1-11. australasian journal of information systems miah & genemo 2016, vol 20, design research for dss development design science methodology for expert systems devt 22 price, m., carroll, j., o’donovan, b., and rust, c. (2011). if i was going there i wouldn’t start from here: a critical commentary on current assessment practice, assessment & evaluation in higher education, 36(4), 479-492. power d (2010). what are concerns about end-user dss development? available: http://dssresources.com/faq/index.php?action=artikel&id=208 sanders, d., tan, y.c., rogers, i., and tewkesbury, g.e. (2009). an expert system for automatic design for assembly, assembly automation, 29 (4), 378 388 sangwin, c. (2012). computer aided assessment of mathematics, oxford university press, uk. sangwin, c., cazes, c., lee, a., and wong, k. l. (2010). micro-level automatic assessment supported by digital technologies mathematics education and technology-rethinking the terrain (pp. 227-250): springer. sasikumar, m., ramani, s., raman, s. m., anjaneyulu, k., and chandrasekar, r. (2007). a practical introduction to rule based expert systems, narosa publishers, india. shao, x y, chu, x.z, qiu, h.b., gao, l. and yan, j. (2009). an expert system using rough sets theory for aided conceptual design of ship’s engine room automation, expert systems with applications, 36 (2), 3223–3233 shiau, w.l. (2011). a profile of information systems research published in expert systems with applications from 1995 to 2008, expert systems with applications 38, 3999–4005 simon, h. a. (1996). the sciences of the artificial (3rd ed.), mit press, cambridge, ma. stalnaker, r. (2012). intellectualism and the objects of knowledge. philosophy and phenomenological research, 85(3), 754-761 turban, e., aronson, j. and liang, t.-p. (2005). decision support systems and intelligent systems 7th edition, pearson prentice hall. vaishnavi, v. k. and kuechler, w. (2015). design science research methods and patterns: innovating information and communication technology, crc press. walls j.g., widmeyer, g.r and el sawy, o.a. (1992) building an information system design theory for vigilant eis. information systems research 3(1), 36-59. walls, j.g, widmeyer,g.r. and el sawy, o.a. (2004). assessing information system design theory in perspective: how useful was our 1992 initial rendition?. journal of information technology theory and application 6 (2), 43-58. waterman, d. a. (1986). a guide to expert systems: reading, mass. addison-wesley, uk. wiliam, d. (2005). keeping learning on track: formative assessment and the regulation of learning. making mathematics vital, 20-34. wu,p-h., hwang, g-j., su, l-h., and huang, y-m. (2012). a context-aware mobile learning system for supporting cognitive apprenticeships in nursing skills training, journal of educational technology & society, 15 (1), 223-236. australasian journal of information systems miah & genemo 2016, vol 20, design research for dss development design science methodology for expert systems devt 23 appendix a: table a1: analysis of the 20 sample es articles s am p les artefacts (model, methods, constructs and instantiations) problem relevance evaluation of design contribution rigour design as search process communication of research 1 instantiations engine room design case study rapid intelligent design solution rigour applied at all of the design phases designer achieved desired problem specific goals, optimal design to both industry and academic audience 2 instantiations clinical health performance evaluations with measures of specificity, sensitivity and accuracy. used descriptive approach solution development rigour is applied at multiple phases of interviews to ensure quality and used questioners enough relevant patient data were collected as kb components and literature in the relevant areas was reviewed and verified. for knowledge representation fuzzy production rules and inference engine implemented diagnostic rules and organising rules using concepts of modules and used meta-rules and rule checking techniques the system is communicated to industry stakeholders and academics by publication for academic audience 3 instantiations fault diagnosis of dredgers performance testing; used experimental approach solution design for fault diagnosing and prevention interviews and field experiences and actual operation of dredgers. publications and technical documents were used the es was made up with neural networks and hybrid knowledge to ensure appropriate diagnosis academic audience australasian journal of information systems miah & genemo 2016, vol 20, design research for dss development design science methodology for expert systems devt 24 4 instantiations support learning activity assessed how the content was developed through the use of experimental approach interactive es development to achieve the learning goals through education objectives the repertory gird by teaching and students to ensure the accuracy of the knowledge. the rigour for evaluation is the knowledge is represented by repertory grid for a problem domain experimented results are presented to participants and academics community by publication 5 instantiations quantitative research user satisfaction through the use of experimental and case study approaches es provides help them to analyse research questionnaires. rigour was ensured during the evaluation and artefact design process through es developed a knowledge base from the experience of senior researchers design process incorporate object oriented approach into es, the knowledge comes from interview of expert of the data analysis to construct ontology experimented results are presented to senior and junior researchers and academic community by publication 6 instantiations chinese medicine evaluation was done in terms of verifications & validations using redundancy, circularity and incompletene ss rules provided tool for physician, family doctor and medical student for knowledge management rigour was ensured through phenomenon study using behavioural sciences reviewers commented on the findings of the issues and application of patients evaluation used ontology for knowledge base of the es, followed knowledge extracted from expert (interviews) in the field of chinese medicine and also applied object oriented technique and security technique for data research findings were presented to students for educational purpose and academics by publication 7 instantiations constructions (engineering) issues multiple evaluation methods (field testing, unit testing, satisfaction testing, developing es for providing construction knowledge in may conditions for engineers and transferring rigour is applied in collecting data the behavioural science knowledge are used. reviewing conducted literature review and observation to collect knowledge, analysing collected knowledge using iterative search process academic audience australasian journal of information systems miah & genemo 2016, vol 20, design research for dss development design science methodology for expert systems devt 25 integration testing) were used for verifications & validations. using optimisation model to evaluate the efficiency of es knowledge from engineers to engineers collected knowledge and content analysis 8 instantiations electrical control experiment method, the evaluation was conducted using representativ e seven problem cases improved coordination settings of protection devices (generators) by solving complexity of the coordination problem systematically designed and evaluated the es and knowledge gathered from relevant sources sub-divisions of the es solution in four portions academic audience 9 instantiations healthdiagnosing asthma practical scenario based testing for the verifications & validations diagnosing asthma at initial stages for ka, they used interview, task performance, protocol and text books, domain experts and users. they used fuzzy technology and semantic network to representation of knowledge used life-cycle based methodology, taking participatory design process. the performance of the system published in journal and communicated to the target stakeholders australasian journal of information systems miah & genemo 2016, vol 20, design research for dss development design science methodology for expert systems devt 26 10 instantiations selecting information systems development methodologies the es was evaluated through face validation, predictive validation and blind validation automated the process of section of appropriate systems development methodologies used standard es development phases there are phases such as phase 1: identification phase, phase 2: determination phase, phase 3: selection phase, and the phase 4: expert system development phase. published in journal and communicated to the target stakeholders and academic audience 11 model student consulting system none web-based student consulting es for the educational advisers to solve student’s rules and regulations related issues rigour was applied through the methodology comprised with specific activities such as fizzification of the crisp variable, rule extraction, data centralisation, putting info on web, considering the exclusion issue of a student, look inside the system, implementation environment used fuzzy logic technique to design the es academic audience 12 instantiations housing construction and design user satisfaction testing through iterative methodology provided comprehensive understanding of daylighting and offers user interactivity regarding design choices rigour was applied through iterative method in the design and evaluation phases for ka the es used a daylighting database which contains information regarding the effects of a variety of design conditions on resultant daylighting performance and controlled by a fuzzy rule based decision making logic that is used to determine those design communicated to participants and academic audience australasian journal of information systems miah & genemo 2016, vol 20, design research for dss development design science methodology for expert systems devt 27 changes most likely to improve performance for a given design 13 instantiations design in product manufacturing the es is evaluated through test cases using real product data provided designers suggestions for changing the design (design for assembly automation) at early stage rigour was applied in making sure that effective knowledge acquired for storing rules and facts about the particular knowledge domain the es was designed through steps such as the steps involved creating: cad expert to interpret cda drawing files and extract assembly data, user interface, automated assembly expert, human assembly expert, and design analysis expert communicated to target users and academic audience 14 model education and career development none es designed for providing educational and career guidance by envisaging four areas of expertise and thus four experts: pedagogical expert, psychological expert and sociological expert and economic expert rigour was applied in demonstrating the design process concepts. used a multi-agent approach and the semantic web in developing the solution to enable communication and collaboration between the agents were the key requirements that are identified in the paper. simply applied a project management approach/software engineering (indirectly). the paper is focused on describing the technical architecture of their es academic audience 15 model management of academics by the head of department in university none es designed for providing assistance to the head of department in universities for assisting in employing rigour was applied by following design steps such as for rules in developing decision trees on the facts conducted a systematic study by interviewing heads of three different departments in order to identify 30 different instances of decision making academic audience australasian journal of information systems miah & genemo 2016, vol 20, design research for dss development design science methodology for expert systems devt 28 lecturers based on some default factors which extracted of human expert knowledge 16 instantiations art and design a case study based evaluation was conducted analysis and interpretation of drawing art therapy tools rigour was applied through design and evaluation process conduced systematic design process using existing theories and practice based knowledge and expert knowledge in art therapy communicating about the es to domain experts and published in journal 17 instantiations operation of competency management evaluation was conducted through the verification and validations using case study (evaluating performance of the system, effectiveness and accuracy) provided support to enhance the management of competence rigour was applied through adopting design and development phases of es (task analysis, knowledge acquisition, prototype development, expansion and refinement and verification and validations) knowledge acquired through interviewing of the field experts (competency managers) communicating about the stakeholders and published in journal for academic audience 18 method online consumer satisfaction experiment method, for validity testing es designed for detecting online consumer satisfaction by providing predictive results incorporated data mining system with es using a bayesian fusion framework rigour was applied through a case study conducted a systematic design process that consists of data collection, data preparation and data analysis. academic audience australasian journal of information systems miah & genemo 2016, vol 20, design research for dss development design science methodology for expert systems devt 29 copyright: © 2016 miah & genemo. this is an open-access article distributed under the terms of the creative commons attributionnoncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. 19 instantiations feedback reviews and bidding in online auctions experiment method, for testing accuracy of prediction es designed for examining feedback and bidding of online auctions rigour was applied through a systematic data collection and model design phases the es used integrated approach of fuzzy logic expert system model and a statistical hedonic regression model academic audience 20 method healthcare experiment method es designed for detecting consumer satisfaction rigour was applied through systematic analysis conducted a systematic design process that consists of data collection and design solution development academic audience http://creativecommons.org/licenses/by-nc/3.0/au/ http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 study background 2.1 expert systems 2.2 es design methodologies and design science research 2.3 issues of assessing multi-step questions 3 research approaches 3.1 research structure 3.2 paradigm and methodologies 3.3 qualitative research method 3.3.1 qualitative document content analysis 3.4 document analysis process 3.4.1 es article analysis 3.4.2 examination script analysis 3.5 es artefact development method 4 es artefact design 4.1 assessing msq questions’ solutions 5 es artefact evaluation 6 discussion and conclusion microsoft word 1828-5983-2-le.docx australasian journal of information systems irawan, foster & tanner 2018, vol 22, research article mandated adoption and implementation of an academic information system 1 the mandated adoption and implementation of an academic information system: empirical evidence from an indonesian university sandra irawan universitas negeri malang irawan.sandra@um.ac.id susan foster monash university kerry tanner monash university abstract under the scenario of contingent authority innovation-decision, organisation managers make the initial decision to adopt an innovation and mandate its use to the employees. although accelerating adoption by the employees, the ensuing stages of implementation are often problematic partly due to its non-voluntary nature. utilising an interpretive case study, this research aimed to explore the nature of the mandated adoption and implementation of an academic information system (ais) for academics in an indonesian university. gallivan’s (2001b) framework for innovation adoption and implementation was modified and then applied as a lens to investigate the case. the results indicated that the mediating factors (i.e., managerial interventions, subjective norms, and facilitating conditions) played a vital role in reducing the resistance resulting from the authoritarian approach to mandating usage. based on the findings, contributions were made by extending the existing framework and providing insights for the university executives regarding the preand postimplementation managerial interventions. keywords: mandated adoption; contingent authority innovation-decision; ict innovation; resistance; indonesian university 1 introduction higher education institutions worldwide have rapidly embraced ict (information and communications technology) as a solution in providing reliable and effective services to address the global increase in student enrolment (united nations educational, scientific and cultural organization, 2009, 2014). however, ict implementation requires higher education institutions to devise effective strategies to minimise any obstacles they may encounter (asian development bank, 2011). rogers (2003) argued that as a social system, higher education institutions, especially those owned and operated by the government, are associated with a bureaucratic and paternalistic culture. it is imperative for information systems academics and professionals to understand the particular challenges for the implementation and adoption of an innovation that such a culture poses. typically, an initial decision to adopt an innovation is made by the executives, who mandate the adoption to organisational members—that is, university staff and students, who have no choice but to adopt the innovation (gallivan, 2001b; rogers, 2003). this scenario is described as a contingent authority innovation-decision. with the contingent nature of the australasian journal of information systems irawan, foster & tanner 2018, vol 22, research article mandated adoption and implementation of an academic information system 2 adoption, managerial interventions such as training and infrastructure support are commonly made available to ensure a successful adoption (gallivan, 2001b; venkatesh & bala, 2008). simultaneously, subjective norms and facilitating conditions emerge as factors that mediate between the initial adoption decision by the executives and the subsequent adoption by the staff and students (gallivan, 2001b). in the context of higher education institutions, potential gaps exist in exploring the mandated adoption of innovations as the issue of mandatoriness was often taken for granted (brown, massey, montoya-weiss, & burkman, 2002; gallivan, 2001b). in the context of higher education, research on the mandated adoption of innovations is scarce as more focus was given towards the identification of challenges and barriers for ict usage (al-shboul, alsaideh, & al-labadi, 2017; huda & hussin, 2010; meerza & beauchamp, 2017; setiawan, 2012) and the voluntary adoption of simple innovations (i.e. pc, laptop, internet, and office applications) (chaputula, 2012; othman et al., 2013; shaikh, 2009; wilson et al., 2014). the significance of this research is to explore in depth the mandated adoption and implementation of an academic information system (ais) for the academics in an indonesian university. this particular university was chosen as the adoption process of the ais by the academics was contingent upon its prior adoption by the university. the ais was first introduced to the academics in 2012, allowing sufficient time for the assimilation to take place and for the research to assess its uptake. this research sought to address the following questions: 1. what is the nature of the adoption and implementation of the ais in the university? 2. how do the mediating factors influence the mandated adoption of the ais by the academics? by modifying gallivan’s (2001b) framework, an investigation was conducted to explore the multilevel perspectives of the actors involved in the case, including the academics, the university executives, and the administrative staff. this research contributes by refining gallivan’s (2001b) framework to suit the context of mandated innovation adoption in higher education institutions and providing insights for the university executives regarding the managerial interventions. the paper is structured as follows. the next section presents the background of the literature related to the mandated adoption of innovations in organisations. the methodology and the results of this research are then elaborated. the paper concludes with a discussion of the findings and a conclusion. 2 background this section reviews the available literature on the adoption of innovations in organisations and higher education institutions. the preliminary conceptual framework that was used as a lens to analyse the case is presented at the end of this section. 2.1 organisational adoption of innovations the term ‘innovation’ is defined as “an idea, practice, or object that is perceived as new by an individual or other unit of adoption” (rogers, 2003: p. 12). this definition of innovation australasian journal of information systems irawan, foster & tanner 2018, vol 22, research article mandated adoption and implementation of an academic information system 3 implies that the degree of newness of the innovation is subjectively dependent on the perception of the individual or the organisation adopting it (rogers, 2003; tilton, 1971). rogers (2003:404) defines an organisation as “a stable system of individuals who work together to achieve common goals through a hierarchy of ranks and a division of labour”. within the context of organisations, van de ven and rogers (1988) argue that innovation implementation has been recognised as a problematic issue due to failures that frequently accompany its introduction to employees. past studies on the broad area of innovation adoption and implementation within organisations contribute to the greater body of knowledge. among these studies are from the area of sociology, the diffusion of innovation (doi) theory (rogers, 1962, 2003), which arose early in 1962 and evolved out of early investigations into the adoption of innovations by autonomous individuals. after more than five decades, doi has evolved to accommodate complex organisational analyses such as when the adoption of innovations is decided at the organisational level rather than at the individual level. in the area of information systems, among the prominent theories in innovation studies are the technology acceptance model (tam) (davis, 1986, 1989; davis, bagozzi, & warshaw, 1989) and information systems (is) success model (delone & mclean, 1992). in particular, delone & mclean’s (1992) is success model was then further extended by seddon (1997). tam was also extended to tam2 (venkatesh & davis, 2000) by taking account of social influences and facilitating conditions constructs. tam3 (venkatesh & bala, 2008) further extends the model by incorporating the determinants of perceived ease of use (venkatesh, 2000). utaut (venkatesh, morris, g. b. davis, & f. d. davis, 2003) integrates eight prominent information technology acceptance models, including doi and tam. in 2012, utaut2 (venkatesh, thong, & xu, 2012) emerged by accommodating the consumer use context. although considered as dominant theories for decades, the legacy of tam and utaut has not been free of criticism (e.g., bagozzi, 2007; benbasat & barki, 2007; chuttur, 2009; lee, kozar, & larsen, 2003). among the criticisms is that tam focused more on the decision to accept or reject an innovation made by an individual and disregarded any decision-making aspects that involved group, social, and cultural aspects (bagozzi, 2007). other criticisms highlight the difficulties for quantitative-based theories such as tam and utaut in providing sound and actionable advice to organisational managers (benbasat & barki, 2007; brown et al., 2002; lee et al., 2003). moreover benbasat and barki (2007) believed that tam has returned ‘full circle’ as the addition of new constructs in the subsequent extensions of tam overlap with the constructs used in past theories (i.e., the theory of reasoned action (fishbein & ajzen, 1975) and the theory of planned behaviour (ajzen, 1985, 1991)). rogers (2003) criticised past studies that claimed to have investigated the determinants of innovation implementation in organisations, but only explored its initial procurement. recent studies suggested that future research should explore the actual implementation and focus on the importance of managerial interventions (venkatesh & bala, 2008). a deeper understanding of this topic can also be achieved by examining the multilevel perspectives (individual, group, and organisational levels) of the actors involved (burton-jones & gallivan, 2007; gallivan, 2001a). past studies investigating the multilevel perspectives of the actors and the key role of managerial interventions were limited, although they have the potential to provide managers with actionable advice regarding the various interventions that can assist implementation (burton-jones & gallivan, 2007; venkatesh & bala, 2008). australasian journal of information systems irawan, foster & tanner 2018, vol 22, research article mandated adoption and implementation of an academic information system 4 gallivan (2001b) has provided a framework which includes an investigation of managerial interventions in the mandated adoption and implementation of an innovation. the framework is described as a hybrid as it “combines some constructs from traditional individual adoption models with features of process and stage research models of organisational-level implementation” (gallivan, 2001b, p. 78). the hybrid nature of the framework allows for an exploration of a case from the multilevel perspectives of the actors involved. the framework is presented in the next section. 2.2 gallivan’s framework for mandated adoption of innovations as shown in figure 1, gallivan’s (2001b) framework follows the scenario of contingent authority innovation-decisions, in which innovation adoption is regarded as a two-stage process (leonard-barton & deschamps, 1988; zaltman, duncan, & holbek, 1973), consisting of a primary adoption by organisation managers, followed by a secondary adoption by the employees (frambach & schillewaert, 2002; gallivan, 2001b). upon developing the framework, gallivan utilised it to investigate the implementation of a client/server development in the is (information systems) division of four large insurance firms. he further advocated the use of the framework as a theoretical lens for future research to analyse the organisational adoption and implementation of innovations in different contexts. corporate it management division it management managerial intervention subjective norms facilitating conditions secondary (individual) adoption process assimilation stage consequences primary authority adoption decission (not directly studied here) secondary adoption and organizational assimilation processes organizational consequences (not directly studied here) group or department it management • authority decision to adopt or captive use • training • support • coworkers/peers • professional network • senior management • clients • subordinates • innovation attributes • organizational attributes • individual attributes 1. initiation 2. adoption 3. adaptation 4. acceptance 5. routinization 6. infusion • extended use • integrative use • emergent use example: (possible outcomes for client/ server development) for it department faster development easier maintenance reduced hardware costs reduce maintenance usage leading edge image/status for user departments greater ease of use greater flexibility faster learning curve faster response time perceived greater control figure 1. framework for innovation adoption and implementation (gallivan, 2001b, p. 60) the vertical dotted lines separate the framework into three segments, namely primary adoption decision, secondary adoption and organisational assimilation processes, and organisational consequences (gallivan, 2001b). the first segment describes the primary adoption decision by the managers, either at the corporate, division or department level of the organisation. the second segment explores the secondary adoption and the assimilation processes after the primary innovation adoption decision is made. the last segment focuses on the implementation consequences. the reverse arrows from “organisational consequences” australasian journal of information systems irawan, foster & tanner 2018, vol 22, research article mandated adoption and implementation of an academic information system 5 back to “assimilation stage” and to “secondary adoption process” indicate the complexity and the dynamic process of implementation (gallivan, 2001b). following the primary adoption by the managers, several mediating factors (i.e. managerial interventions, subjective norms, and facilitating conditions) will emerge to influence the secondary adoption process by the employees. the managerial interventions describe “the actions taken and resources made available by managers to expedite secondary adoption, including mandating usage” (gallivan, 2001b, p. 61). these interventions may comprise of training, support, and resources made available by the managers to accelerate the secondary adoption by the employees. the subjective norms construct describes “individuals’ belief about the expectation of relevant others regarding their own secondary adoption behaviour” (gallivan, 2001b, p. 61). the facilitating conditions construct is “a broad category that captures other factors that can make implementation moreor less-likely to occur” (gallivan, 2001b, p. 61) and comprises of innovation, organisational, and individual attributes. in a scenario of contingent authority innovation-decision, the issue is “not whether employees adopt the innovation (since this is assumed), but rather when and how they adopt it -through what experiences, with what obstacles encountered, and how these events influence organizational assimilation and outcomes” (gallivan, 2001b, p. 62). to provide a state-of-the-art landscape of research in higher education, a review was also conducted on previous ict studies in several countries with regard to this context. this is presented in the next section. 2.3 ict studies in higher education institutions table 1 presents a review of state-of-the-art ict studies conducted within the context of higher education institutions in several countries. no author (year), country research topic, ict artefact, voluntariness source of data key findings 1 al-shboul, al-saideh, & allabadi (2017), jordan examining the perceptions and barriers concerning the use of ict tools by students in jordanian universities survey of 725 students from 4 jordanian universities positive perceptions of students towards ict in higher education 2 chaputula (2012), malawi examining the voluntary adoption of simple ict innovations (word processor, internet, laptop and mobile phone) for teaching, research and internet browsing survey of 317 students and 113 academic staff and one librarian poor infrastructure and lack of ict skills, high cost of internet access and persistent power outages 3 huda & hussin (2010), indonesia examining the implementation barriers of ict in general (administration, teaching, learning, and research). interviews of 2 toplevel university executives the main inhibitors are work culture and individual perception towards ict australasian journal of information systems irawan, foster & tanner 2018, vol 22, research article mandated adoption and implementation of an academic information system 6 no author (year), country research topic, ict artefact, voluntariness source of data key findings 4 meerza & beauchamp (2017), kuwait examining the critical factors for the use of general ict to support learning at universities structured questionnaire and interviews of undergraduate students usefulness, ease of use of ict, and peer learning as key factors for ict utilisation 5 othman, pislaru, kenan, & impes (2013), libya examining the usage of simple ict innovations (tools used for online learning, virtual classroom and emodule) for student learning activities open and closedended questionnaire support staff needed to reduce academic staff's burden, a fixed budget to maintain and develop ict, training and incentives for staff 6 setiawan (2012), indonesia examining the implementation barriers and challenges for ict in general interviews of 20 staff, informal discussion and document collection in a university limited human resources, lack of management commitment, funding and rewards and ineffective communication and coordination 7 shaikh (2009), pakistan examining the role and voluntary usage of simple ict innovations (ms office, google) in teaching and learning survey to 30 faculty members, students, parents, admin staff, and ict policy makers lack of training for teaching / support staff, poor infrastructure, and lack of effective ict policy 8 wilson, tetemensah, & boateng (2014), ghana examining the voluntary usage of simple ict innovations (mobile phones, tv, radio, pc, laptop, digital camera, voice recorder) for student’s personal learning questionnaire to 500 students and focus-group discussion in a ghana university students and teachers have low technology literacy skills and training is needed table 1. previous ict studies in the context of higher education the review indicated that research focusing on the mandated adoption of innovations in the context of higher education institutions is scarce. the focus of the research was more towards the identification of challenges and barriers for ict usage in higher education institutions (alshboul, al-saideh, & al-labadi, 2017; huda & hussin, 2010; meerza & beauchamp, 2017; setiawan, 2012) and the voluntary and non-contingent adoption of simple innovations (i.e. pc, laptop, internet, and office applications) (chaputula, 2012; othman et al., 2013; shaikh, 2009; wilson et al., 2014). the issue of mandatoriness was often taken for granted in previous studies and yet it may prove to be significant in influencing the subsequent adoption of the innovations by the users (brown et al., 2002; gallivan, 2001b). therefore the researcher chose to explore the ais as it was considered to be a complex ict innovation that posed challenges due to its contingent nature and its mandated adoption in the university. as the ais was introduced to the academics in 2012, this research was able to assess its uptake, as sufficient time had passed for it to be assimilated into the university. australasian journal of information systems irawan, foster & tanner 2018, vol 22, research article mandated adoption and implementation of an academic information system 7 the following section presents the preliminary conceptual framework that was chosen and used in this research. 2.4 preliminary conceptual framework based on the analysis of previous studies, gallivan's (2001b) framework for innovation adoption and implementation was chosen as a theoretical lens to explore the case in this research. the qualitative nature of the framework has a degree of flexibility to anticipate novel findings with regard to the factors that may influence innovation adoption and implementation. however applying the framework was challenging as the specific arrangement and timing for when the factors begin to give influence were not specifically addressed by gallivan (2001b). moreover, the context of the case and the nature of the innovation in this research was different from those of gallivan. for example, gallivan focused his investigation on insurance firms in the united states, while this research focuses on a university in indonesia. in terms of the innovation, gallivan’s research investigated the adoption of a client/server development software that was purchased commercially, while this research explored the adoption of an academic information system that was developed in-house. framework modifications were needed to address the challenge and the differences. venkatesh and bala’s (2008) ‘research agenda for interventions’, which was also inspired by the it implementation process (cooper and zmud, 1990), were considered useful for the modification of the framework. the modified framework is illustrated in figure 2. primary adoption process secondary adoption process pre-implementation phase post-implementation phase assimilation process innovation deployment mediating factor pre-implementation interventions mediating factors post-implementation interventions subjective norms facilitating conditions figure 2. modified framework for innovation adoption and implementation (based on gallivan, 2001b; venkatesh & bala, 2008) the modified framework consists of two phases: pre-implementation (prior to innovation deployment) and post-implementation (after innovation deployment). three main processes occurred during these phases: primary adoption process, secondary adoption process, and assimilation process. australasian journal of information systems irawan, foster & tanner 2018, vol 22, research article mandated adoption and implementation of an academic information system 8 the pre-implementation phase describes the decisions and actions made by the managers during the initial implementation of the innovation, which involves the primary adoption process and the mediating factors (i.e. managerial interventions). the post-implementation phase highlights the perspectives and experience of the employees as the secondary adopters of the innovation, which includes the secondary adoption process, the assimilation process, and several mediating factors (post-implementation interventions, subjective norms, and facilitating conditions). the primary adoption process describes the initial decision made by the managers as the primary adopters of the innovation. to support secondary adoption by the employees, the pre-implementation interventions are deliberately provided by the managers. in the secondary adoption process, employees start to play their role as secondary adopters of the innovation. the assimilation process highlights the use of the innovation by the employees as it assimilates into the organisation. the reverse arrows from “assimilation process” back to “secondary adoption process” indicate the dynamic process of implementation (gallivan, 2001b). by incorporating individual and organisational levels of analysis, the framework examines the innovation adoption and implementation using the multilevel perspectives of the actors involved (burton-jones & gallivan, 2007; gallivan, 2001a). by so doing, the research can provide organisation managers with a deeper understanding of the case and assist them in making informed decisions with regard to innovation implementation. the methodology of the research is outlined in the next section. 3 methodology this research took an interpretivist approach by utilising constructivism as its philosophical grounding (williamson & johanson, 2013). utilising this approach, collective meanings were constructed from the research participants by taking account of the norms and values embedded in their beliefs, decisions, and actions with regard to the case being studied (myers, 2013; orlikowski & baroudi, 1991; walsham, 2006; williamson & johanson, 2013). figure 3 illustrates the research design that was used as a guide during the course of this study. the aim and questions of the research drove the research theme to focus on the literature concerning organisational innovation studies of information systems and ict studies in higher education institutions. a mixed approach literature review was used in this research as the researcher had already decided the research theme prior to commencing the review with the purpose of gaining a thorough understanding of the literature (bandara, furtmueller, gorbacheva, miskon, & beekhuyzen, 2015; boell & cecez-kecmanovic, 2015). the case study research method was utilised, as it was considered appropriate to accommodate the focus on human interpretations and meanings, with regard to the phenomenon being studied (eisenhardt, 1989; walsham, 1995). the single case design was specifically chosen because it accommodates the focus on the breadth and depth of the case (yin, 2014) and provides a richer understanding and lessons-learned information from the experience of the actors and the organisation (creswell, 2013; stake, 1995). australasian journal of information systems irawan, foster & tanner 2018, vol 22, research article mandated adoption and implementation of an academic information system 9 figure 3. research design the main technique used in this research was semi-structured interviews, supported by documentation and physical artefacts (yin, 2014). interviews were undertaken at the indonesian university in november-december 2015. these techniques are elaborated on below. 1. semi-structured interviews the semi-structured interview was used to accommodate research questions based on the preliminary conceptual framework. the use of this technique brought benefits to the researcher as it encouraged two-way communication and allowed new perspectives and ideas to emerge during interviews. a general overview of the participants is presented in table 2. participant total academics executives administrative staff head of administration staff member level university 5 3 1 9 faculty a 3 1 4 faculty b 6 1 1 8 faculty c 4 2 6 faculty d 6 1 7 total 19 5 5 5 34 table 2. interviewed participants at the university ict studies in higher education institutions organisational innovation studies in information systems data collection interviews documentation physical artefacts literature review research aim and research questions preliminary conceptual framework case study data analysis refined framework australasian journal of information systems irawan, foster & tanner 2018, vol 22, research article mandated adoption and implementation of an academic information system 10 the interviews were carried out with 34 stakeholders from three categories: the university academics (19 participants), the top-level university executives (five participants), and the administrative staff (ten participants). each of the categories had different roles and therefore was given a different set of questions during their interviews. a relatively balanced number of academics were sought from four faculties, based on gender and age, to ensure a fair representation across the spectrum. the five top-level executives who were interviewed included vice-rector a, vice rector b, the director of ict, the head of the ict centre, and the head of quality assurance. the interviews lasted 30-60 minutes each, were conducted using the indonesian language, and were all audio-recorded. 2. documentation the documentary evidence collected during this research was in the form of online and offline documents. the online documents were retrieved from the official website and two blogs maintained by the university. the offline documents were obtained from three books, two decrees, two written policies, and one statistical report of the university. the results from the documentation were then used to provide a contextual account of the case being studied and to corroborate the information from the interviews. 3. physical artefacts an exploration of the physical artefacts was needed as the case involved a transformation from the previous system to a new system. physical artefacts from the previous system were the optical scanners, which were used to scan forms in the previous paper-based system. the artefact which was considered as the innovation was the academic information system. the researcher paid particular heed to klein and myers’ (1999) principle of multiple interpretations and principle of dialogical reasoning in designing the study. there were very different and at times contradictory perspectives from the participants (i.e., the executives and the academics); and the preliminary framework was constantly refined based on the findings. figure 4 illustrates the method for data analysis for this research. to acquire valid meaning from the qualitative data, the method of analysis as advocated by miles, huberman, and saldaña (2014) and saldaña (2015) was employed. figure 4. method for data analysis the coding stage was divided into the first cycle and second cycle coding as the data sources were analysed using thematic analysis. ‘in vivo coding’ and ‘process coding’ methods were employed during the first cycle coding (saldaña, 2015). “in vivo coding” (i.e. not to be australasian journal of information systems irawan, foster & tanner 2018, vol 22, research article mandated adoption and implementation of an academic information system 11 confused with the nvivo software), also known as “verbatim coding”, is a method of analysis that is used to extract the participants’ own terms as the source for codes (miles et al., 2014; saldaña, 2015). the use of the “process coding” method assisted the researcher in compiling codes based on the activities shown and described by the interview participants (saldaña, 2015). as the researcher was guided by a preliminary framework, ‘elaborative coding’ was selected as the second cycle coding method, as this method not only preserves the themes and categories from the preliminary framework, but also allows new themes and categories to emerge during the data analysis (miles et al., 2014; saldaña, 2015). 4 results this section presents findings with regard to the mandated adoption and implementation of the ais, which is structured based on the preliminary conceptual framework. based on the results of the first and second cycle coding of events occurring in the preand postimplementation phases, details regarding the primary adoption, secondary adoption, assimilation processes, and the mediating factors are provided. 4.1 pre-implementation phase this phase explored the decisions and actions, as envisaged by five university executives, who were involved as the decision makers in the pre-implementation phase of the ais at the university. information was also retrieved from the administrative staff to substantiate details at the operational level, which could not be provided by the executives. 4.1.1 primary adoption process the thematic analysis for the primary adoption process focused on the interviews conducted with five executives and ten administrative staff at the university (figure 5). figure 5. process of the primary adoption the major themes surrounding the primary adoption of the ais were highlighted with problems in the previous paper-based system, such as grade manipulation and lengthy procedures. the problems were exacerbated with the increase in the number of university stakeholders. the two pillars of university strategic planning problems in the previous system primary adoption decision of ais development of ais ict revitalisation team ict revitalisation policy five top-level executives (university board) active scanning of organisational need (pull strategy) australasian journal of information systems irawan, foster & tanner 2018, vol 22, research article mandated adoption and implementation of an academic information system 12 there used to be a long delay for the students’ grade submissions in the faculties. grade manipulation was also possible. people other than the relevant academics were handling the students’ grade forms and the final output sometimes did not match with the grades given by the academics. it happened everywhere. (vice-rector b) with regard to the initial decision to adopt the ais, the result of the analysis showed that it was made solely by the executives as a part of an ict revitalisation policy. we viewed it as a demand for technology. in such cases, we don’t really need to consult with the academics. the meetings held by the university board were enough to decide its adoption. (head of quality assurance) with regard to the adoption of ais for the academics, mandatoriness was a clear theme as the phrase “whether you want it or not” was used repeatedly during the interviews. for the ais, it is clearly mandatory. it is obligatory. it is a must whether you want it or not because it is a portal that you have to pass through. (director of ict) as the university is owned and operated by the government, its bureaucratic and paternalistic culture inherited a hierarchical structure, in which the executives had the right to impose decisions that impacted the routine tasks of the academics, who were seen to have lower rank (asian development bank, 2012; boyne, 2002; rogers, 2003). the next section describes the interventions given by the executives to support the adoption of the ais by the academics. 4.1.2 pre-implementation interventions once the decision was made by the top-level executives to implement the ais, the preimplementation process was set in motion. a considerable amount of the budget was invested in the ict infrastructure and human resources as forms of managerial interventions. a large portion of our budget is directed towards ict: to develop ict, to increase the bandwidth, and so on. we also encourage the addition of ict staff. (vice-rector b) based on the analysis of the interviews, several codes were identified as managerial interventions during the pre-implementation of the ais, as listed in table 3. no interventions execution level 1 ais development, user manual, and online helpdesk university level 2 dissemination seminars university level 3 ais championing university level 4 infrastructure support university level 5 usage mandate university level table 3. pre-implementation interventions the ais was developed in-house by the ict centre of the university to ensure there was flexibility to modify the ais when needed at a future time. the user manual and online helpdesk were provided and embedded in the functions of the ais. following the development of the ais, a dissemination seminar was conducted as the first attempt to properly introduce the ais to the academics. australasian journal of information systems irawan, foster & tanner 2018, vol 22, research article mandated adoption and implementation of an academic information system 13 we presented it (ais) to the academics in the form of dissemination seminars and by listing the shortcomings that have not yet been addressed based on their demands. (director of ict) the director of ict in particular, with his legitimacy and ict competency, exhibited transformative leadership and played an influential role among the executives, the academics, and the staff. prior research has acknowledged the importance of using champions during the adoption and implementation of complex innovations (beath, 1991; chatterjee, grewal, & sambamurthy, 2002; jasperson, carter, & zmud, 2005; leonard-barton, 1988; norris, 1999; purvis, sambamurthy, & zmud, 2001; taylor & todd, 1995). 4.2 post-implementation phase this section focuses on the analysis of the perspectives and experiences of the academics as they began to adopt and use the ais. information from the executives and the administrative staff of the university are also presented to substantiate the findings. 4.2.1 secondary adoption and assimilation processes the high level of resistance from the academics became a major theme following the deployment of the ais. the head of the ict centre confirmed that resistance was expressed in the form of protests and complaints to the executives and the staff at the ict centre. for the ais, there was a large amount of resistance when it was first introduced due to the transition from manual or semi-digital to fully digital. the resistance was in the form of protests and complaints. (head of the ict centre) figure 6 illustrates the result of coding with regard to the cause of the resistance by the academics during the secondary adoption process. figure 6. resistance in the secondary adoption process based on further analysis of the interviews with the academics, ineffectiveness was found to be a major theme for the pre-implementation interventions as the academics were not given sufficient information and skills to operate the ais prior to its deployment. together with the usage mandate enforced by the executives, the ineffective pre-implementation interventions placed a larger burden on the post-implementation interventions. during data coding, the drastic change of role of the academics unstable early version of ais ais abrupt implementation lack of ict competence of the academics lack of information regarding ais warning mechanism contribute counter usage mandate resistance australasian journal of information systems irawan, foster & tanner 2018, vol 22, research article mandated adoption and implementation of an academic information system 14 phrase “whether we like it or not” was also repeatedly uttered by the academics during interviews, implying that they were not given any choice but to adopt the ais. the ais was made mandatory from the institution, so whether we like it or not, all academics must use it. (academic 10, male, faculty c) other causes contributing to the resistance included the lack of information regarding the ais, the abrupt implementation of the ais, the unstable early version of the ais, the academics’ lack of ict competence, and their drastic change of role (from passive to active users). there was no information. we only knew about it from the dissemination seminar. we did not know about it beforehand. (academic 10, male, faculty c) it was abruptly implemented and we had to use it. (academic 3, male, faculty a) surprisingly, the lack of participation and involvement of the academics in the primary adoption decision tended to be tolerated as they were already used to such a decision-making process. i choose to see it this way. the decision has been made and it is obligatory, it has to be done. (academic 16, male, faculty d) due to the high resistance, the usage mandate was then substantiated via a warning mechanism for the academics who delayed or resisted the use of the ais. this involved the faculties sending warning letters to academics who failed to finish their task using the ais by a given deadline, with an escalation procedure starting from the heads of department up to the faculty deans. this mechanism was proven to be effective in countering the resistance, as sanctions that impacted their wellbeing as civil servants would be given if they continued to oppose. those who have not submitted will receive a letter from the head of the department to immediately proceed with the submission. the department will contact the academics regarding that matter. if the head of the department cannot solve it, we would escalate the problem to the vice-dean and let him take care of the matter. usually, the problem is resolved once it reaches the vice-dean. (staff 5, male, faculty a) in the end, the warning mechanism was able to counter the high resistance demonstrated by the academics. all of the academics claimed to have adopted the ais, although two main methods of usage were captured: direct and indirect. out of the 19 academics, 17 were found to directly use the ais. although mostly dominated by younger academics, three senior academics who used the ais directly were found to have had an informal ict background as they were either educated abroad or had ict-related experience. two academics used the ais indirectly by delegating its use to four categories of surrogate: fellow academics, administrative staff, teaching assistants, and family relatives. the main reason for use delegation was low ict competence due to the age factor and limited ict background. for the grade submission, i rely on my teaching assistant. my assistant was the one doing it for me. that’s a real burden for me. (academic 2, female, faculty a) other reasons for delegating use included time constraints and laziness. the issue of power relation was also evident as some of the surrogates had no choice but to fulfil the request from the senior academics due to respect. australasian journal of information systems irawan, foster & tanner 2018, vol 22, research article mandated adoption and implementation of an academic information system 15 the next section describes the interventions that emerged following the deployment of the ais. 4.2.2 post-implementation interventions during the data analysis, several codes related to post-implementation interventions emerged and are shown in table 4. no interventions execution level 1 ais reinvention university level 2 dissemination seminars and training faculty a, c, and d 3 ais championing faculty a and b 4 warning mechanism university level 5 mentoring faculty a and b 6 peer support all faculties table 4. post-implementation interventions in the post-implementation phase, new interventions emerged such as training, mentoring, peer support, and the championing of the ais by the faculty heads of administration. these interventions were faculty-specific and were held without much coordination with the university. the practice of mentoring was done by assigning academic staff as mentors, who provided personalised guidance concerning the ais to the academics. this faculty provides mentoring for the academics based on their departments. the purpose is so that we don’t have to go too far to the ict centre for assistance. (academic 5, male, faculty b) the mentoring in this research was associated with the technique of ‘accompanying’, in which the mentors made a commitment to guide the mentee side-by-side during the learning process (aubrey & cohen, 1995). such a practice was found to be effective especially in assisting senior academics in the university. prior research suggested that mentoring facilitates the transfer of tacit knowledge that cannot be done effectively through formal training (hsieh & hsu, 2013) and provides a flexible one-to-one approach that allows a longer time to adapt to new innovations (gallivan, 2004). it was also found that the academics felt more comfortable in seeking support from younger or tech-savvy fellow academics rather than formally forwarding their queries to the ict centre. i studied it at first. i then asked my colleagues in case i bump into difficulties. we have 58 academics here. if any of us has figured out a solution, we will ask him. so it’s collegial learning. (academic 16, male, faculty d) the peer support in this research refers to a specific approach of conducting informal tutorials to support peers in the same organisation (jasperson et al., 2005). informal support from peers, as a form of ‘help-seeking behaviour’, is more preferred if compared to the formal support (ram & jung, 1991). other studies also found that such influence can enhance innovation implementation by forming favourable perceptions of an innovation and reduce anxiety towards it (fiato, 2012; venkatesh & davis, 2000). australasian journal of information systems irawan, foster & tanner 2018, vol 22, research article mandated adoption and implementation of an academic information system 16 the findings also showed that the training for the ais was not centralised in the university but instead was conducted in faculties based on their own initiatives. although previous studies suggested that training is an important intervention (gallivan, spitler, & koufaris, 2005; venkatesh & bala, 2008), interestingly it was not considered to be the main intervention as it was held with little or no coordination on the part of the university. the role of heads of administration as faculty champions was found to be pivotal in the context of a mandated innovation adoption. their legitimacy, access, and ict competency, were found to reduce the negative impacts from the lack of coordination between the faculty and the university. in general, the exploration of the multilevel perspectives of the participants revealed the different perspectives between the executives and the academics concerning the implementation of the ais. the executives perceived it as part of a larger strategic plan of revitalising ict in the university. however the usage mandate and other causes of resistance led the academics to perceive it as merely a radical switch from the previous system and thus influenced their willingness to adopt and use the ais. 5 discussion this section discusses and explains the findings related to the mandated adoption and implementation of the ais in the indonesian university. corresponding to the research questions, the nature of the adoption and implementation of the ais is discussed in the pre and post-implementation phases (sections 5.1 and 5.2). the mediating factors and their influences towards the adoption of the ais by the academics were also identified in these phases. based on the findings in each of the phases, changes were made to the preliminary framework and resulted in the final refined framework presented in section 5.3. 5.1 pre-implementation phase the pre-implementation phase includes the primary adoption process of the ais, which led to its deployment (see figure 7). implementation ais deployment primary adoption process pre-implementation phase figure 7. pre-implementation phase due to the bureaucratic and paternalistic culture of the university, the primary adoption process followed an authoritarian approach, as the initial adoption of the ais was solely decided by the executives without involving the academics (caudle, gorr, & newcomer, 1991; heintze & bretschneider, 2000; zaltman et al., 1973). two principal decisions were made by the executives during this process: the decision to adopt the ais and the decision to mandate its usage. the executives’ authoritarian decision to adopt the ais tended to be tolerated by the academics, as they were already accustomed to the paternalistic culture. however, the decision to mandate the use of the ais for the academics was found to have more complex australasian journal of information systems irawan, foster & tanner 2018, vol 22, research article mandated adoption and implementation of an academic information system 17 implications that significantly influenced the subsequent stages of implementation. to add to the pressure, the previous system was completely abandoned when the ais was deployed, leaving no alternative for the academics but to adopt it. during the pre-implementation phase, several mediating factors were identified and brought changes to the preliminary framework (see figure 8). mediating factor pre-implementation interventions mediating factors pre-implementation interventions • ais development, user manual, & online helpdesk • university-level dissemination seminar • championing by the director of ict • infrastructure support • usage mandate subjective norms • executives facilitating conditions • organisation attributes (bureacratic & paternalistic culture) • innovation attributes (in-house innovation) figure 8. changes in the mediating factors in the pre-implementation phase initially, the preliminary framework proposed pre-implementation interventions as the sole mediating factor in the pre-implementation phase. however the findings have captured other factors from the subjective norms and facilitating conditions constructs. the pre-implementation interventions began with the development of the ais with its user manual and online helpdesk as well as the provisioning of ict infrastructure support. a university-level dissemination seminar was then provided to demonstrate the early version of the ais to the academics and to highlight the usage mandate. a championing strategy was demonstrated by the director of ict to convince the academics with regard to the benefits of the ais compared to the previous system. the subjective norms construct was strongly demonstrated through the enforcement of the usage mandate and the warning mechanism for the academics to use the ais. organisational and innovation attributes emerged as facilitating conditions in this early phase of implementation. the bureaucratic and paternalistic culture defined the organisation attributes and affected the decision-making process in the university. the innovation attributes were made clear as the requirement for the ais was to be developed in-house by the university. 5.2 post-implementation phase the post-implementation phase outlines the processes that took place after the deployment of the ais, namely the secondary adoption process and the assimilation process (see figure 9). the secondary adoption process was accompanied by resistance from the academics, mainly due to the usage mandate. however, the enforcement of the warning mechanism managed to counter the resistance and forced the academics to adopt the ais. thus the academics’ decision to adopt was associated more with the effort to avoid the warning mechanism rather than their personal motivation to use the ais (brown et al, 2002). australasian journal of information systems irawan, foster & tanner 2018, vol 22, research article mandated adoption and implementation of an academic information system 18 secondary adoption process implementation post-implementation phase assimilation process ais deployment figure 9. post-implementation phase following the adoption of the ais by the academics, it began to be incorporated into the dayto-day tasks and signalled the beginning of the assimilation process (gallivan, 2001b). the reverse arrow reflects the complex and dynamic process of the implementation, such as when the academics constantly adapt to the continuous enhancement of the ais. another example includes the difficulties for some of the senior academics in completely adopting the ais. the negative implication of the mandate has resulted in the identification of two usage methods for the ais: direct and indirect use (burton-jones & straub jr, 2006; tong, teo, & tan, 2008). according to tong et al. (2008, p.2), direct use occurs when “a user independently uses an information system to accomplish an organisational task”, while indirect use is defined as “the state in which a user employs an is indirectly through one or more intermediaries (i.e., other colleagues) to accomplish an organisational task” (tong et al., 2008, p.2). in the case of the ais, most of the young and tech-savvy academics in the university used it directly while several senior academics decided to delegate its use to surrogates. several mediating factors were found to provide support during the assimilation of the ais and brought changes to the preliminary framework as shown in figure 10. mediating factorsmediating factors post-implementation interventions post-implementation interventions • ais reinvention • faculty-level dissemination seminar & training • championing by faculty heads of administration • warning mechanism • mentoring • peer support subjective norms facilitating conditions subjective norms • administrative staff • fellow academics • teaching assistants • executives • relatives facilitating conditions • individual attributes (ict competence, time constraints, power relations, laziness) figure 10. changes in the mediating factors in the post-implementation phase the post-implementation interventions consisted of ais reinvention, faculty-level dissemination seminars and training, ais championing, warning mechanism, mentoring, and peer support. although the dissemination seminar had already been held at the university level, additional seminars and training were held at the faculty level to cope with its ineffectiveness. the ais was refined gradually to accommodate requests from the executives and the academics. the cost of non-compliance was high, due to the enforcement of the warning mechanism for delaying or rejecting the use of the ais. the role of heads of australasian journal of information systems irawan, foster & tanner 2018, vol 22, research article mandated adoption and implementation of an academic information system 19 administration as local champions was regarded as vital in all of the faculties. in these faculties, informal peer support and mentoring were found to be effective in assisting academics with low ict competence, as they required a longer time to adapt to the ais. the subjective norms construct demonstrated the influences coming from administrative staff, teaching assistants, executives, fellow academics, and family relatives in the form of warning mechanism, peer support, mentoring, and assistance in the use of the ais. the facilitating condition construct is highlighted by the individual attributes of the university academics, including ict competence, time constraints, power relations, and laziness. the empirical evidence has brought changes to the preliminary conceptual framework. these changes are summed up and the final refined framework is presented in the following section. 5.3 final refined framework a contribution of this research to theory is by refining gallivan’s (2001b) framework based on the analysis of the findings. the final refined framework is shown in figure 11. mediating factors primary adoption process mediating factors pre-implementation interventions • ais development, user manual, & online helpdesk • university-level dissemination seminar • championing by the director of ict • infrastructure support • usage mandate secondary adoption process pre-implementation phase post-implementation phase post-implementation interventions • ais reinvention • faculty-level dissemination seminar & training • championing by faculty heads of administration • warning mechanism • mentoring • peer support subjective norms • executives facilitating conditions • organisation attributes (bureacratic & paternalistic culture) • innovation attributes (in-house innovation) assimilation process ais deployment subjective norms • administrative staff • fellow academics • teaching assistants • executives • relatives facilitating conditions • individual attributes (ict competence, time constraints, power relations, laziness) figure 11. final refined framework building upon the initial framework described in section 2.4, an exploration of the mandated adoption and implementation of the ais in the indonesian university was conducted. the findings confirmed that the adoption of the ais followed the scenario of contingent authority innovation-decisions. as the primary adoption of the ais was decided by the executives, a usage mandate and then later a warning mechanism were enforced on the academics. the three constructs of the mediating factors (e.g. interventions, subjective norms, and facilitating conditions) were identified in the pre-implementation and post-implementation phases. several interventions from the pre-implementation phase (e.g. ais championing by the director of ict and infrastructure support) were continued in the post-implementation phase. other interventions (e.g., the development of the ais and the university-level australasian journal of information systems irawan, foster & tanner 2018, vol 22, research article mandated adoption and implementation of an academic information system 20 dissemination seminar) were continued by similar interventions in the post-implementation phase (e.g., ais reinvention and faculty-level dissemination seminars). in the postimplementation phase, several new faculty-specific interventions emerged without much coordination with the university, including mentoring, peer support, training, and ais championing by heads of administration. the subjective norms construct incorporates the sources of influence during the implementation of the ais, which includes the executives, fellow academics, administrative staff, teaching assistants, and family relatives. several facilitating conditions (e.g., organisation and innovation attributes) started to provide influences starting from the pre-implementation phase, while the influences from the individual attributes occurred later on, as the ais began to be used by the academics. 6 conclusion previous innovation studies were mostly dominated by quantitative studies that measure innovation acceptance by individuals in a social system. an investigation on mandated innovation adoption from the multilevel perspectives of the actors involved was needed as it may reveal the role of the usage mandate, managerial interventions and other influential factors. a contribution of this research to theory is by modifying and then extending gallivan’s (2001b) framework based on the empirical evidence. as suggested in the final framework, the adoption of the ais was associated with the scenario of contingent authority innovationdecision. the authoritarian approach during the pre-implementation phase was met with resistance and thus burdened the following phase of implementation. however the mediating factors identified in this research were found to play a pivotal role in reducing the resistance. this research contributes to practice by providing insights to the university executives regarding the characteristics of the preand post-implementation managerial interventions that influenced the adoption of the ais by the academics. the insights may assist the executives in evaluating their strategies for the current implementation of the ais as well as in introducing future innovations. nonetheless as with any research, there are limitations to the study as it was based on data from just one higher education institution and focuses only on a single innovation. future research can build upon the framework by conducting a study that provides evidence from multiple organisations, from multiple innovations, or by comparing distinct adoption settings (i.e., mandatory and voluntary). other research opportunities exist in exploring the extent of the mandatoriness in several different research contexts or innovation adoption scenarios. australasian journal of information systems irawan, foster & tanner 2018, vol 22, research article mandated adoption and implementation of an academic information system 21 references ajzen, i. (1985). from intentions to actions: a theory of planned behavior. in j. kuhl & j. beckmann (eds.), action contol (pp. 11–39). heidelberg, germany: springer. ajzen, i. (1991). the theory of planned behavior. organizational behavior and human decision processes, 50(2), 179–211. doi: 10.1016/0749-5978(91)90020-t al-shboul, m., al-saideh, m., & al-labadi, n. (2017). learners' perspectives of using ict in higher education institutions in jordan. instructional technology, 14(3), 27-86. asian development bank. (2011). higher education across asia: an overview of issues and strategies. manila, philippines: asian development bank. asian development bank. (2012). private higher education across asia: expanding access, searching for quality. manila, philippines: asian development bank. aubrey, r., & cohen, p. m. (1995). working wisdom: timeless skills and vanguard strategies for learning organizations. san fransisco, ca: jossey-bass publishers. bagozzi, r. p. (2007). the legacy of the technology acceptance model and a proposal for a paradigm shift. journal of the association for information systems, 8(4), 3. doi: 10.17705/1jais.00122 bandara, w., furtmueller, e., gorbacheva, e., miskon, s., & beekhuyzen, j. (2015). achieving rigor in literature reviews: insights from qualitative data analysis and tool-support. communications of the association for information systems, 37, 154–204. beath, c. m. (1991). supporting the information technology champion. mis quarterly, 355–372. doi: 10.2307/249647 benbasat, i., & barki, h. (2007). quo vadis tam? journal of the association for information systems, 8(4), 7. doi: 10.17705/1jais.00126 boell, s. k., & cecez-kecmanovic, d. (2015). on being ‘systematic’ in literature reviews in is. journal of information technology, 30(2), 161–173. doi: 10.1057/jit.2014.26 boyne, g. a. (2002). public and private management: what’s the difference? journal of management studies, 39(1), 97–122. doi: 10.1111/1467-6486.00284 brown, s. a., massey, a. p., montoya-weiss, m. m., & burkman, j. r. (2002). do i really have to? user acceptance of mandated technology. european journal of information systems, 11(4), 283–295. doi: 10.1057/palgrave.ejis.3000438 burton-jones, a., & gallivan, m. j. (2007). toward a deeper understanding of system usage in organizations: a multilevel perspective. mis quarterly, 31(4), 657–679. doi: 10.2307/25148815 burton-jones, a., & straub jr, d. w. (2006). reconceptualizing system usage: an approach and empirical test. information systems research, 17(3), 228–246. doi: 10.1287/isre.1060.0096 caudle, s. l., gorr, w. l., & newcomer, k. e. (1991). key information systems management issues for the public sector. mis quarterly, 15(2), 171–188. doi: 10.2307/249378 chaputula, a. h. (2012). state, adoption and use of icts by students and academic staff at mzuzu university, malawi. program electronic library and information systems, 46(4), 364– 382. doi: 10.1108/00330331211276477 australasian journal of information systems irawan, foster & tanner 2018, vol 22, research article mandated adoption and implementation of an academic information system 22 chatterjee, d., grewal, r., & sambamurthy, v. (2002). shaping up for e-commerce: institutional enablers of the organizational assimilation of web technologies. mis quarterly, 26(2), 65–89. doi: 10.2307/4132321 chuttur, m. y. (2009). overview of the technology acceptance model: origins, developments and future directions. working papers on information systems, 9(37), 9–37. cooper, r. b., & zmud, r. w. (1990). information technology implementation research: a technological diffusion approach. management science, 36(2), 123–139. doi: 10.1287/mnsc.36.2.123 creswell, j. w. (2013). qualitative inquiry and research design: choosing among five approaches (3rd ed.). thousand oaks, ca: sage. davis, f. d. (1986). a technology acceptance model for empirically testing new end-user information systems: theory and results (doctoral dissertation, massachusetts institute of technology). retrieved from https://dspace.mit.edu. davis, f. d. (1989). perceived usefulness, perceived ease of use and user acceptance of information technology. mis quarterly, 13(3), 319–340. doi: 10.2307/249008 davis, f. d., bagozzi, r. p., & warshaw, p. r. (1989). user acceptance of computer technology: a comparison of two theoretical models. management science, 35(8), 982–1003. doi: 10.1287/mnsc.35.8.982 delone, w., and mclean, e.r. (1992). information systems success: the quest for the dependent variable. information systems research, 3(1), 60–95. doi:10.1287/isre.3.1.60 eisenhardt, k. m. (1989). building theories from case study research. academy of management review, 14(4), 532–550. doi: 10.5465/amr.1989.4308385 fiato, k. a. (2012). enhancing electronic medical records (emr) implementation: peer support interventions influencing nurses' acceptance and use (doctoral dissertation, capella university). available from proquest dissertations and theses database. (umi no. 3499912). fishbein, m., & ajzen, i. (1975). belief, attitude, intention, and behavior: an introduction to theory and research. reading, ma: addison-wesley publishing company. frambach, r. t., & schillewaert, n. (2002). organizational innovation adoption: a multi-level framework of determinants and opportunities for future research. journal of business research, 55(2), 163–176. doi: 10.1016/s0148-2963(00)00152-1 gallivan, m. j. (2001a). meaning to change: how diverse stakeholders interpret organizational communication about change initiatives. ieee transactions on professional communication, 44(4), 243-266. doi: 10.1109/47.968107 gallivan, m. j. (2001b). organizational adoption and assimilation of complex technological innovations: development and application of a new framework. acm sigmis database, 32(3), 51–85. doi: 10.1145/506724.506729 gallivan, m. j. (2004). examining it professionals' adaptation to technological change: the influence of gender and personal attributes. acm sigmis database, 35(3), 28–49. doi: 10.1145/1017114.1017119 australasian journal of information systems irawan, foster & tanner 2018, vol 22, research article mandated adoption and implementation of an academic information system 23 gallivan, m. j., spitler, v. k., & koufaris, m. (2005). does information technology training really matter? a social information processing analysis of coworkers' influence on it usage in the workplace. journal of management information systems, 22(1), 153–192. doi: 10.1080/07421222.2003.11045830 heintze, t., & bretschneider, s. (2000). information technology and restructuring in public organizations: does adoption of information technology affect organizational structures, communications, and decision making? journal of public administration research and theory, 10(4), 801–830. doi: 10.1093/oxfordjournals.jpart.a024292 hsieh, s.t., & hsu, p.y. (2013). mentoring effects in the successful adaptation of information systems. information development, 31(2), 164–175. doi: 10.1177/0266666913511263 huda, m. q., & hussin, h. (2010). ict implementation barriers and organizational issues in islamic-based higher education institution: the case of syarif hidayatullah state islamic university (uin) jakarta. proceedings of the 3rd international conference on information and communication technology for the moslem world (ict4m) 2010 (pp. a18–a25). jakarta, indonesia: ieee. jasperson, j. s., carter, p. e., & zmud, r. w. (2005). a comprehensive conceptualization of post-adoptive behaviors associated with information technology enabled work systems. mis quarterly, 29(3), 525–557. doi: 10.2307/25148694 klein, h. k., & myers, m. d. (1999). a set of principles for conducting and evaluating interpretive field studies in information systems. mis quarterly, 23(1), 67–93. doi: 10.2307/249410 lee, y., kozar, k. a., & larsen, k. r. (2003). the technology acceptance model: past, present, and future. communications of the association for information systems, 12(1), 50. leonard-barton, d. (1988). implementation characteristics of organizational innovations: limits and opportunities for management strategies. communication research, 15(5), 603– 631. doi: 10.1177/009365088015005006 leonard-barton, d., & deschamps, i. (1988). managerial influence in the implementation of new technology. management science, 34(10), 1252–1265. doi: 10.1287/mnsc.34.10.1252 meerza, a. h., & beauchamp, g. (2017). factors influencing attitudes towards information and communication technology (ict) amongst undergraduates: an empirical study conducted in kuwait higher education institutions (kheis). the turkish online journal of educational technology, 16(2), 35-42. miles, m. b., huberman, a. m., & saldaña, j. (2014). qualitative data analysis: a methods sourcebook. thousand oaks, ca: sage. myers, m. d. (2013). qualitative research in business and management. thousand oaks, ca: sage. norris, d. f. (1999). leading edge information technologies and their adoption: lessons from us cities. in information technology and computer applications in public administration: issues and trends (pp. 137–156). igi global. doi: 10.4018/978-1-87828-952-0.ch008 orlikowski, w. j., & baroudi, j. j. (1991). studying information technology in organizations: research approaches and assumptions. information systems research, 2(1), 1–28. doi: 10.1287/isre.2.1.1 australasian journal of information systems irawan, foster & tanner 2018, vol 22, research article mandated adoption and implementation of an academic information system 24 othman, a., pislaru, c., kenan, t., & impes, a. (2013). analyzing the effectiveness of it strategy in libyan higher education institutes. international journal of digital information and wireless communications, 3(3), 114–129. purvis, r. l., sambamurthy, v., & zmud, r. w. (2001). the assimilation of knowledge platforms in organizations: an empirical investigation. organization science, 12(2), 117– 135. doi: 10.1287/orsc.12.2.117.10115 ram, s., & jung, h.-s. (1991). “forced” adoption of innovations in organizations: consequences and implications. journal of product innovation management, 8(2), 117–126. doi: 10.1016/0737-6782(91)90005-j rogers, e. m. (1962). diffusion of innovations (1st ed.). new york, ny: free press. rogers, e. m. (2003). diffusion of innovations (5th ed.). new york, ny: free press. saldaña, j. (2015). the coding manual for qualitative researchers. thousand oaks, ca: sage. seddon, p. (1997). a respecification and extension of the delone and mclean model of is success. information systems research, 8(3), 240–253. doi:10.1287/isre.8.3.240 setiawan, i. (2012). a case study of it implementation in public university: its barriers and challenges. jurnal ilmu komputer, 5(2), 43–52. shaikh, z. a. (2009). usage, acceptance, adoption, and diffusion of information & communication technologies in higher education: a measurement of critical factors. journal of information technology impact, 9(2), 63–80. stake, r. e. (1995). the art of case study research. thousand oaks, ca: sage. taylor, s., & todd, p. a. (1995). understanding information technology usage: a test of competing models. information systems research, 6(2), 144–176. doi: 10.1287/isre.6.2.144 tilton, j. e. (1971). international diffusion of technology: the case of semiconductors (vol. 4). brookings institution press. tong, y., teo, h.-h., & tan, c.-h. (2008). direct and indirect use of information systems in organizations: an empirical investigation of system usage in a public hospital. proceedings of the 29th international conference of information systems (icis) 2008 (pp. 138). paris, france: ais electronic library. united nations educational, scientific and cultural organization. (2009). icts for higher education. paper presented at the world conference on higher education, paris, france. united nations educational, scientific and cultural organization. (2014). higher education in asia: expanding out, expanding up. montreal, qc: unesco institute for statistics. van de ven, a. h., & rogers, e. m. (1988). innovations and organizations: critical perspectives. communication research, 15(5), 632–651. doi: 10.1177/009365088015005007 venkatesh, v. (2000). determinants of perceived ease of use: integrating control, intrinsic motivation, and emotion into the technology acceptance model. information systems research, 11(4), 342. doi: 10.1287/isre.11.4.342.11872 venkatesh, v., & bala, h. (2008). technology acceptance model 3 and a research agenda on interventions. decision sciences, 39(2), 273–315. doi: 10.1111/j.1540-5915.2008.00192.x australasian journal of information systems irawan, foster & tanner 2018, vol 22, research article mandated adoption and implementation of an academic information system 25 venkatesh, v., & davis, f. d. (2000). a theoretical extension of the technology acceptance model: four longitudinal field studies. management science, 46(2), 186–204. doi: 10.1287/mnsc.46.2.186.11926 venkatesh, v., morris, m. g., davis, g. b., & davis, f. d. (2003). user acceptance of information technology: toward a unified view. mis quarterly, 27(3), 425–478. doi: 10.2307/30036540 venkatesh, v., thong, j. y., & xu, x. (2012). consumer acceptance and use of information technology: extending the unified theory of acceptance and use of technology. mis quarterly, 36(1), 157–178. walsham, g. (1995). interpretive case studies in is research: nature and method. european journal of information systems, 4(2), 74–81. doi: 10.1057/ejis.1995.9 walsham, g. (2006). doing interpretive research. european journal of information systems, 15(3), 320–330. doi: 10.1057/palgrave.ejis.3000589 williamson, k., & johanson, g. (2013). research methods: information, systems and contexts. prahran, australia: tilde university press. wilson, k. b., tete-mensah, i., & boateng, k. a. (2014). information and communication technology use in higher education: perspectives from students. european scientific journal, 10(19), 161–171. yin, r. k. (2014). case study research: design and methods (5th ed.). los angeles, ca: sage. zaltman, g., duncan, r., & holbek, j. (1973). innovations and organizations. new york, ny: wiley. copyright: © 2018 irawan, foster & tanner. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. paper title (paper title style) australasian journal of information systems afrashteh, someh & davern 2020, vol 24, selected papers from the is foundations conference explanations as discourse 1 explanations as discourse: towards ethical big data analytics services sadaf afrashteh the university of melbourne safrashteh@student.unimelb.edu.au ida someh school of business, the university of queensland michael davern faculty of business and economics the university of melbourne abstract big data analytics uses algorithms for decision-making and targeting of customers. these algorithms process large-scale data sets and create efficiencies in the decision-making process for organizations but are often incomprehensible to customers and inherently opaque in nature. recent european union regulations require that organizations communicate meaningful information to customers on the use of algorithms and the reasons behind decisions made about them. in this paper, we explore the use of explanations in big data analytics services. we rely on discourse ethics to argue that explanations can facilitate a balanced communication between organizations and customers, leading to transparency and trust for customers as well as customer engagement and reduced reputation risks for organizations. we conclude the paper by proposing future empirical research directions. keywords: big data; analytics services; ethics; explanation facilities; discourse ethics 1 introduction big data analytics has become a prevalent phenomenon in modern society and is a topic of much discussion in the information systems (is) field. big data is typically characterized by its large volume, the variety of attributes captured, and the velocity at which the data accumulates, primarily through online interfaces and digital media devices (constantiou & kallinikos, 2015). big data is too diverse and fast-paced to be adequately captured and analysed using traditional data processing tools (markus & topi, 2015). in the big data analytics ecosystem, data is harvested through digital devices from even the most mundane activities of an individual’s everyday life. the captured data is analysed by algorithms to both tailor products and services to support individuals, and to enable organizational decision making. big data analytics is playing an increasing role in business success, but its effects on both individuals and society are of concern (newell & marabelli, 2015; someh, davern, breidbach, & shanks, 2019). algorithmic decision-making that leverages big data analytics to predict customer behaviour is becoming a widespread organizational practice. this decision-making approach is a doubleedged sword: with benefits accruing to businesses often at the cost of negative consequences for the individual customers that are the source of the data. as a result, a growing stream of research has begun investigating the ethical and social consequences of big data analytics such australasian journal of information systems afrashteh, someh & davern 2020, vol 24, selected papers from the is foundations conference explanations as discourse 2 as discrimination, de-individualization and lack of privacy (bélanger & crossler, 2011). the lack of accountability in the algorithmic decision-making employed in big data analytics means that these negative consequences often continue unchecked (citron & pasquale, 2014). in response to concerns about the consequences of big data analytics, regulators have called for greater transparency in algorithmic decision-making. for example, the recently declared general data protection regulation (gdpr) of the european union, has introduced a “right to explanation” for the data subject (i.e., the individual whose data is being processed). specifically, the data subject has the right to get “meaningful information about the logic involved in the process of data analysis” (goodman & flaxman, 2016b). however, it is unclear what constitutes “meaningful information” for providing adequate transparency to mitigate the negative ethical and social consequences of big data analytics. in this study, we investigate the role of explanation facilities for algorithmic decision-making as a means for communicating “meaningful information” to data subjects – the users of services enabled by big data analytics. our research question is thus: how can explanations communicate “meaningful information” to empower and engage users of big data analytics services? to address this question, we blend work on discourse ethics and big data analytics with the literature on explanations in intelligent systems. our goal is to produce a framework to enable us to empirically examine how explanations of big data analytics can effectively communicate “meaningful information” and ultimately mitigate some of the negative ethical consequences of big data analytics. 2 theoretical perspectives 2.1 algorithmic decision-making and ethics algorithms can assist decision-making processes through uncovering hidden patterns in data. they can empower decision makers by providing solutions for complex problems (lepri, oliver, letouzé, pentland, & vinck, 2018). while algorithmic decision-making provides numerous benefits for individuals, organizations and society, they can also lead to negative and potentially unethical consequences. benefits of algorithmic decision making include personalised services and products for individuals, innovations and improved decisionmaking for organizations and improved public health, safety and security at the society level. however, the emerging literature on big data analytics has recognized a variety of unethical consequences of algorithmic decision-making such as discrimination, lack of transparency and lack of accountability. we identify three causes that underlie these ethical issues: the analytic approach, algorithmic model construction, and opaque models. algorithmic decision-making employs big data collected from a variety of different sources such as open data sets and customer-generated data. to get the value from the gathered data organizations can employ either an inductive or deductive approach to analysis, although, in big data analytics, the inductive approach is more common. in a deductive approach there is “theory” that drives purposeful data collection and analysis. ethical issues arise as this “theory” can lead to a confirmation bias in the collection, analysis and interpretation of data (günther, rezazade mehrizi, huysman, & feldberg, 2017). in an inductive approach, data is often collected without a specific intended purpose and analysed to generate “insights” – patterns in the data that can guide action. the patterns australasian journal of information systems afrashteh, someh & davern 2020, vol 24, selected papers from the is foundations conference explanations as discourse 3 however may simply be statistical artefacts like spurious correlations from which causal predictions are often erroneously imputed. clustering algorithms are often applied in this inductive manner to classify, profile and categorize individuals, either directly or indirectly, based on sensitive attributes like gender, religion and age, resulting in discrimination. given the inductive approach, individuals may be classified as belonging to a group they themselves do not identify with – in what is known as de-individualization. the potential ethical consequences that can arise from both inductive and deductive analytic approaches, is exacerbated by the lack of power and influence the data subject has over of the analysis, relative to the data controller responsible for directing the analysis. algorithmic decision-making works by building a model from historical training data that allows it to then be applied to future decisions, whether through predictions, classifications or other action. this machine learning approach to model construction can be ethically problematic in three ways: (1) incorrect labels (2) sampling bias and (3) incomplete data. first, through differences between labels applied to attributes in the training data and the data set to which the model is to be applied. to the extent that the attribute labels and definitions from the training data are incorrect or different to the context of application, the validity of the model is brought into question. as a result, individuals may be misclassified or otherwise misdealt with. in a similar vein, a biased sample data set can lead to a model that is discriminatory. for example, consider the modelling of suburban crime rates. if the historical training data comprises a disproportionate number of suburbs with a high crime rate, the model may be biased and suggest inappropriate actions or interpretation in settings not adequately represented in the training data (i.e., lower crime rate suburbs). finally, incomplete training, where important characteristics of the actual population are at best only partially present can lead to a model of questionable validity. somewhat ironically, this can occur out of an ethical concern to preserve individual data privacy, but it may cause ethically questionable algorithmic decision-making. despite the benefits of algorithmic decision-making, most algorithms function as black-boxes. in the context of big data analytics, this is often because the way the machine learning algorithm learns is not understandable by humans and so it is inherently an opaque decision process (sinha & swearingen, 2002). even when there is a human interpretable model and algorithm, organizations can as a matter of policy not disclose the workings of the algorithm to internal users, let alone the subjects of the data (e.g., for competitive reasons). whether inherent or by policy, the opacity of algorithmic decision-making can lead users and data subjects to distrust the algorithmic model and its decision (ribeiro, singh, & guestrin, 2016). more importantly, to the extent that algorithmic decision-making is opaque, it is hard to ensure full and proper accountability for the decisions made. this has led to calls for improving transparency of algorithmic decision-making. transparency enables greater accountability for decisions made, thereby enhancing trust (lepri et al., 2018), but is particularly problematic when the machine learning methods employed are opaque by nature. 2.2 discourse ethics ethical issues have long been a concern in is (markus & topi, 2015; martin, 2015) and a variety of theoretical approaches have been used to explore ethics in the is context. we follow mingers and walsham (2010) and use habermas’ discourse ethics as our theoretical approach. we adopt a discourse ethics approach because, unlike other ethics theories, discourse ethics focuses on the communication between the stakeholders as the process by which ethical australasian journal of information systems afrashteh, someh & davern 2020, vol 24, selected papers from the is foundations conference explanations as discourse 4 outcomes are achieved (as opposed to focusing on defining ethical outcomes per se). this focus on the process of communicative action lends itself to investigating our research question on the communication of “meaningful information” to users. central to discourse ethics is the notion of an ideal speech situation, in which stakeholders engage as equals in the discourse, there is no coercion, and stakeholders have opportunity to question the claims of others, and to present their own claims and needs (mingers & walsham, 2010). within the algorithmic decision-making context, ethical discourse will emerge when users and decision-makers experience the ideal speech situation. to achieve this aim, users’ need to understand the underlying logic of the decision-making process. introducing transparency opens a lens to reach a fairer communication among stakeholders by providing users with greater clarity about the application of algorithmic decision-making models in the services they use (lepri et al., 2018). system generated explanations can be used as a means to enhance transparency. there is a well-established literature in is on how such explanations can shed light on the logic and conclusions made by decision-making technologies and enhance ease of use and trust (gregor & benbasat, 1999). when users of big data analytics services are better informed about the processes of algorithmic decision making with big data, they are more likely to engage with decision-makers (günther et al., 2017) and the resulting discourse may move closer to the ideal speech situation. 2.3 explanations in intelligent systems the ability of systems to provide explanations of system decisions or recommendations has long been seen as critical to the acceptance of the decisions and recommendations by users (dhaliwal & benbasat, 1996; hayes-roth & jacobstein, 1994; ye & johnson, 1995). from the outset explanations have been defined as providing useful descriptions of why requested data was needed, and the reasoning employed in processing that data, and the basis for any recommendation or decision (clancey, 1983). explanations have been found to help users in three different ways: (1) to cater for user requirements when their expectations are not fulfilled, (2) to facilitate learning and (3) to deliver information required for problem-solving and decision-making (gregor & benbasat, 1999). we conducted a systematic review of the is literature focused on explanations using the following search criteria. first, we chose as a foundation, gregor and benbasat’s (1999), synthesis of the explanation literature, and searched forward for papers that cite this foundation work. we chose this paper as our foundation, because it provided a highly cited, comprehensive synthesis of the explanation literature, and it historically coincided with a time when data driven approaches (data mining, knowledge discovery, and what would now be called data analytics) were just beginning to garner traction in is practice (in large part due to the increased availability of machine readable data brought about by the boom in ecommerce). second, we limited our search to papers published in ais basket of eight journals. third, we selected papers that used “explanation” or “explainability” in their abstract. fourth, we chose papers with an empirical component that were published from 1999 to 2018. five, papers before 2007 with less than 25 citation counts were excluded. our search resulted in 10 papers relevant to our study, as shown in table 1. explanations are used across a range of intelligent systems such as knowledge-based systems (kbs) (arnold, clark, collier, leech, & sutton, 2006; dhaliwal & benbasat, 1996; gregor, 2001; gregor & benbasat, 1999; mao & benbasat, 2000; smedley & sutton, 2007), decision support systems (dss) (berendt & preibusch, 2014; gönül, önkal, & lawrence, 2006; tan, tan, & teo, australasian journal of information systems afrashteh, someh & davern 2020, vol 24, selected papers from the is foundations conference explanations as discourse 5 2012), expert systems (arnold et al., 2006; bohanec, kljajić borštnar, & robnik-šikonja, 2017; ye & johnson, 1995), recommendation agents (ra) (mcsherry, 2005; sinha & swearingen, 2002; wang & benbasat, 2007) and case-based reasoning systems (sørmo, cassens, & aamodt, 2005). in all these cases, both machines and individuals play a part in the decision-making and problem-solving process. explanations assist human decision makers by reducing their cognitive load in the decision-making process as well as enhancing the quality of decisions and recommendations made (gregor, 2001). from the users’ perspective, explanations positively influence perceptions of the system and can also help users learn in the problem domain (gregor & benbasat, 1999), primarily by enhancing transparency. in the consumer facing context of recommendation agents, this transparency helps build customer trust in system recommendations by exposing the logic of the underlying the processes to the customer (sinha & swearingen, 2002). a key issue in the prior literature is the role of expertise of the user in the application domain. the effects of explanations vary significantly depending on the expertise of the user. feedforward explanations conveying declarative knowledge are more useful for novices as they provide information on the system’s inputs and their relationships. however, experts prefer feedback explanations that facilitate transfer of procedural knowledge. feedback explanations provide information on how a decision has been made (arnold et al., 2006). in the context of big data analytics services, the expertise of the user – the data subject – may be unknown but is likely skewed toward the novice end of the spectrum. the content of explanations has been categorized into a taxonomy of four different types of information provided to users: terminology, trace, justification and strategy (gregor & benbasat, 1999). terminological explanations are the "knowledge of the concepts and relationships of a domain that experts use to communicate with one another" (swartout & smoliar, 1987). trace explanations describe, ex post, the actual reasoning process employed by the system to reach a specific decision or conclusion. justification explanations provide the basis, often in terms of domain specific knowledge, for the conclusion or recommend made by a system. strategy explanations explain the goals of the system that underlie its reasoning process (sørmo et al., 2005). a parallel taxonomy to that of gregor and benbasat’s (1999) comes from the literature on decisional guidance, which is defined as how a system enlightens or persuades users in their decision making (silver 1991). conceptually silver distinguishes between informative guidance which seeks to enlighten users, and suggestive guidance which seeks to sway users (silver, 2006). informative guidance is somewhat akin to terminological explanations but there is not a clear mapping between the two taxonomies and decisional guidance has been operationalised in a variety of ways (davern & parkes 2010). pragmatically, we chose to remain with the gregor and benbasat (1999) taxonomy as it is arguably more directly translatable into features in an artefact, consistent with the design science approach. an important aspect of providing explanations is how to communicate them to users. prior literature reveals a variety of different mechanisms have been employed to provide explanations, including automated provision, user-invoked and intelligent. automatic explanations are always available without the users’ request. conversely, user-invoked explanations are presented only at the request of the user. alternatively, the provision of explanations itself can employ intelligent approaches to tailor the response to a particular user or type of user based on some underlying model of the user. at a more pragmatic level there australasian journal of information systems afrashteh, someh & davern 2020, vol 24, selected papers from the is foundations conference explanations as discourse 6 are also choices in the presentation format of the explanations, for example textual, audiovisual, animation and other multimedia techniques. at issue here in both the presentation format and provision mechanisms is interactivity, one the key cognitive qualities of information systems (davern et al 2012). the guidance provided by explanations serves to inform and persuade users (silver, 2006). toulmin’s model of argumentation has been the primary approach in the literature to exploring the persuasiveness of explanations (ye & johnson, 1995). this model identifies six elements of argumentation: claims, data, warrants, backing, qualifiers and possible rebuttals. a claim is a state which is proposed to be accepted. data is the basis for the argument. warrants provide the connection from data to the claim, thereby justifying the claim. backing is concerned with supporting the trustworthiness of warrants in case their validity is doubted. to incorporate issues of the degree of certainty of a claim, qualifiers are used. possible rebuttals indicate conditions in which the warrant is not applicable and as a result the conclusion can be overturned. toulmin’s model provides a useful structure of the different components to an explanation (i.e., is the explanation data, backing, qualifier etc.) (gregor & benbasat, 1999). a broader literature on explanations has also emerged in the computer science discipline (letham, rudin, mccormick, & madigan, 2015; ustun & rudin, 2016), frequently referred to as explainable artificial intelligence (xai). ai systems autonomously learn from data and mimic human behaviour. the xai literature focuses on the design of ai models that are inherently explainable while maintaining high learning performance of the underlying algorithms. one example is the defence advanced research project agency (darpa)’s xai initiative that aims to develop new ai systems that can explain their rationale, determine their strengths and weaknesses, and provide an understanding of how they will act in the future. researchers have also developed tools such as lime (local interpretable model-agnostic explanations) that produce instance-specific explanations for the outputs of any classifier algorithms (ribeiro, singh, & guestrin, 2016; valenzuela-escárcega, nagesh, & surdeanu, 2018). tools like lime focus on trying to ex post explain, an otherwise opaque, analytics result, rather than the actual reasoning processes employed in getting the result. while this can be useful, it is not aimed at providing meaningful information that could be communicated with the customers of big data analytics services. australasian journal of information systems afrashteh, someh & davern 2020, vol 24, selected papers from the is foundations conference explanations as discourse 7 study theoretical foundation method/context independent variables dependent variable findings (arnold et al., 2006) toulmin’s model of argument, adoptive control of thoughtrational theory experiment in insolvency existence of explanation, types of explanation adherence explanation accesses novices use feedforward explanations to acquire declarative knowledge while experts use feedback explanations to acquire procedural knowledge. feedback enhances adherence to the recommendation. (gregor, 2001) cognitive effort cognitive learning experiment in cooperative problem solving systems with and without requirement of cooperative problem solving use of explanation problem solving performance a requirement for cooperative problem solving was associated with greater use of explanations. positive relationship between explanations and improved performance was more noticeable when problems requiring cooperation were undertaken. the frequency of use of explanations in total was positively related to problem-solving performance. (limayem & desanctis, 2000) theory of breakpoints in group interaction experiment in group decision support system context objective variables, perceptions of the group decision process and outcomes, perceptions of the multicriteria decision making models and group decision system support model understanding system explanations can improve decisional outcomes due to improvement in user understanding of decision models (wang & benbasat, 2007) theory of interpersonal communication experiment in e-commerce types of explanation as how, why and trade-off explanations three beliefs in trust as benevolence, competence and integrity the use of how explanations increases users’ competence and benevolence beliefs. the use of why explanations increases their benevolence beliefs. the use of trade-off explanations increases their integrity beliefs. (mao & benbasat, 2000) cognitive effort theory, novice-expert differences theories, question asking theories experiment in financial analysis level of users’ expertise nature of explanation use explanation types experienced professionals requested more trace and fewer justification and strategic explanations than novices. experts’ knowledge makes it easier for them to detect anomalies in kbs output. australasian journal of information systems afrashteh, someh & davern 2020, vol 24, selected papers from the is foundations conference explanations as discourse 8 study theoretical foundation method/context independent variables dependent variable findings (tan et al., 2012) cognitive effort theory experiment in customer decision aid context explanation-featured decision aid (trace, justification, strategy) perceived decision confidence decision time decision quality more elaborated explanation aid could increase a consumer's decision confidence leading to less cognitive effort and lower product choice made. (li & gregor, 2011) theory of explanations experiment in online advisory services service type (with or without explanation) level of users’ decision process satisfaction level of decisionadvisory transparency different types of self-assessment tools lead to different levels of decision support effectiveness, measured in terms of decision process satisfaction and decision-advice transparency. decision-advice transparency is shown to have a stronger influence over empowerment outcomes. (giboney, brown, lowry, & nunamaker jr, 2015) cognitive fit theory person-environment fit paradigm experiment in deception detection explanation quality perceived usefulness of kbs -perceived ease of use explanation cognitive fit perceived usefulness of kbs explanation quality explanation evaluation time explanation influence explanation quality will increase explanation influence and perceived usefulness of the kbs. perceived usefulness of the kbs will increase explanation influence. perceived ease of use will increase perceived usefulness. explanation cognitive fit will increase explanation quality and explanation evaluation time. (martens & provost, 2014) three-gap framework for explanations case study in web page classification global explanations instance-level explanations explanation performance global explanations in the form of a decision tree or a list of the most indicative words do not provide a satisfactory solution. (rader, cotter, & cho, 2018) experiment in social media types of explanations awareness correctness interpretability accountability the what, how, and why explanations all supported both awareness and accountability. only the what explanations supported correctness. only the how explanations supported interpretability. table 1. explanations in information systems literature australasian journal of information systems afrashteh, someh & davern 2020, vol 24, selected papers from the is foundations conference explanations as discourse 9 3 a framework for explanations in big data analytics we develop a research framework, presented in figure 1, that illustrates how explanation facilities can be used to communicate meaningful information about algorithmic decision making to external customers. the communication occurs through a discourse process (habermas, 1984) and can result in both customer-related and organizational outcomes. our framework consists of four components: explanation facility, ethical discourse, customer outcomes, and organizational outcomes. the framework and the definition of concepts follows. figure 1: a framework for explanations in big data analytics 3.1 the explanation facility in big data analytics the explanation facility seeks to provide meaningful information to external customers about analytics and decisions made within a big data service. extrapolating from the prior work on explanations, we define three components of explanation facilities in big data analytics: content type, interactivity and timing. content type follows gregor and benbasat’s (1999) explanations taxonomy of terminology, trace, justification and strategic. interactivity captures aspects of presentation format and provision mechanism from the prior literature but recognizes a broader range of means of engaging with users to provide explanations (e.g., interactive and immersive visualizations) (davern et al 2012). timing of explanations in big data analytics draws on the feedback and feedforward distinction (arnold et al., 2006; ye & johnson, 1995) and recognizes explanations can vary in what and when they are provided. for example ex ante what data is required and why, or ex post what data was used and how it was analysed. by providing meaningful information to customers of big data analytics services, the explanation facility can raise awareness and help empower customers in their interaction with service providers. 3.2 ethical discourse discourse process refers to the communication and engagement between service providers and customers that is enabled and influenced by the provision of explanations. since the goal is to nudge the discourse towards the ideal speech situation (habermas, 1984; mingers & explanation facility in big data analytics content type presentation format provision mechanisms timing and interactivity ethical discourse equal participation balanced communication visible reasoning customer outcomes trust transparency organizational outcomes customer engagement reduced reputation risk australasian journal of information systems afrashteh, someh & davern 2020, vol 24, selected papers from the is foundations conference explanations as discourse 10 walsham, 2010) we conceptualize the discourse as having three components: equal participation, balanced communication, and visible reasoning. equal participation requires a level playing field in the interaction where the service provider does not exploit a power differential to coerce the customer into providing data or making particular choices. balanced communication means that customers (and indeed organizations) can freely question and negotiate to achieve an agreement on decision-making processes and data usage. a balanced communication and negotiation between customers and organizations can create a win-win situation for both stakeholders (günther et al., 2017). as part of ensuring balanced communication and levelling the playing field, the reasoning processes of the algorithmic decision-making needs to be made visible. conceptually, this could be done by explicating the different elements of toulmin’s argumentation model as part of the discourse. we propose that an appropriately designed explanation facility (i.e., comprising the three components we have identified) will enable an ethical discourse between service providers and customers. proposition 1: an appropriately designed explanation facility in big data analytics services will enable an ethical discourse between service providers and customers. 3.3 customer outcomes customer outcomes are the benefits to customers of providing explanations and enabling an appropriate discourse. transparency and trust are two key outcomes for customers. transparency emerges when decision-making processes and outcomes are explained to users in a way that they can understand (de laat, 2018; mittelstadt, 2016; sinha & swearingen, 2002). explanation facilities and ethical discourse can also help individuals build more trust in a service when they know there is transparency and they can understand why certain decisions are made (lepri et al., 2018; ribeiro et al., 2016; sinha & swearingen, 2002; wang & benbasat, 2007). as such, we propose that: proposition 2: an ethical discourse between service providers and customers will lead to outcomes beneficial to customers such as trust and transparency. 3.4 organizational outcomes for organizations, explanations to customers will positively influence customers’ perceptions about the organization. the shared understanding between organizations and customers about the logic underlying the decision-making process will lead to better customer engagement (günther et al., 2017). more broadly, by enabling an appropriate discourse process, explanations can help ensure customers and organizations have consistent expectations, and the actions of organizations are consistent with those expectations. this substantially reduces the organization’s reputation risk in the use of big data analytics (silver, 2006). thus, we propose that: proposition 3: an ethical discourse between service providers and customers will lead to beneficial outcomes to organizations such as customer engagement and reduced reputation risk. 4 conclusion and future research in this paper, we reviewed literature on explanations and utilized discourse ethics to propose a framework with a set of propositions for the use of explanations in big data analytics services. our framework provides insight into the use of explanations to address ethical issues with big data analytics, and more pragmatically contributes to understanding what could constitute “meaningful information” for users as required by the gdpr (goodman & flaxman, 2016a). australasian journal of information systems afrashteh, someh & davern 2020, vol 24, selected papers from the is foundations conference explanations as discourse 11 the framework shows that appropriately designed explanation facilities gives rise to an ethical discourse in which organizations and customers equally participate and negotiate decisionmaking processes and outcomes that affect individuals. by providing clarity and transparency to users about the process and conclusions of algorithmic decision-making such a discourse can enhance customer trust, and engagement (bussone, stumpf, & o'sullivan, 2015; gregor & benbasat, 1999; lepri et al., 2018). while prior research in intelligent systems and explainable ai have explored the design of explanation facilities the focus has been primarily on improving decision-making or reliance on systems by organizational decision makers (gregor, 2001; tan et al., 2012; wang & benbasat, 2007). the context of big data analytics enabled services and the regulatory requirement for “meaningful information” to be provided to data subjects brings to the fore the ethical dimension. we built on the prior literature to address this concern, and in integrating in the theoretical lens of discourse ethics consider explanations not just as a means for improving decision quality, but also for enabling ethical discourse – and shifting that discourse towards the notion of ideal speech. while we consider the various design, elements identified in prior research, in expanding the goal beyond decision quality to include enabling an ethical discourse we expect will lead to quite different developments of even established design elements. more broadly our approach emphasizes the important role of explanations as communication tools, which brings into the mix new theories in ethical discourse and opens the possibility for drawing theoretical insight from the established literature on organizational communication enabled by it artefacts (te’eni, 2001). we look to future research to elaborate on and empirically test our propositions and explore the role of explanations in enabling ethical discourse to the consequent benefit of a broader range of stakeholders (i.e.,. both organizations and customers). from the practical perspective, our proposed framework provides a guide for organizations on how they can design their explanation facilities to ethically benefit from deploying big data analytics services. moreover, given the discourse ethics lens, they can be guided how to develop their communication strategies in their interactions with their customers. we look forward with anticipation to seeing how the design of explanation facilities evolves in practice as organization increasingly adopt, either voluntarily or through regulation, an ethical lens on design. further research is required to understand how organizations can formulate discourse strategies and tools. an example of these strategies can be designing social bots as explanations interface. these bots can provide transparency with presenting explanations about systems’ outputs to customers through which customers can be informed about why the results are reached and interactively share how they feel afterwards. also, customers can provide organizations with their feedback about systems’ performance and this will result in improving the systems as well. in this regard, further studies can be done exploring which types of explanations are more effective leading to greater transparency and customers’ engagement in discourse to reach the level playing field situation. australasian journal of information systems afrashteh, someh & davern 2020, vol 24, selected papers from the is foundations conference explanations as discourse 12 references arnold, v., clark, n., collier, p. a., leech, s. a., & sutton, s. g. (2006). the differential use and effect of knowledge-based system explanations in novice and expert judgment decisions. mis quarterly, 30(1), 79-97. doi:10.2307/25148718. bélanger, f., & crossler, r. e. (2011). privacy in the digital age: a review of information privacy research in information systems. mis quarterly, 35(4), 1017-1042. doi:10.2307/41409971. berendt, b., & preibusch, s. (2014). better decision support through exploratory discrimination-aware data mining: foundations and empirical evidence. artificial intelligence and law, 22(2), 175-209. doi:10.1007/s10506-013-9152-0. bohanec, m., kljajić borštnar, m., & robnik-šikonja, m. (2017). explaining machine learning models in sales predictions. expert systems with applications, 71, 416-428. doi:10.1016/j.eswa.2016.11.010. bussone, a., stumpf, s., & o'sullivan, d. (2015). the role of explanations on trust and reliance in clinical decision support systems. paper presented at the healthcare informatics (ichi), 2015 international conference. citron, d. k., & pasquale, f. (2014). the scored society: due process for automated predictions. washington law review, 89, 1. clancey, w.j. (1983). the epistemology of a rule-based expert system – a framework for explanation. artificial intelligence, 20(3), 215-251. constantiou, i. d., & kallinikos, j. (2015). new games, new rules: big data and the changing context of strategy. journal of information technology, 30(1), 44-57. doi:10.1057/jit.2014.17 davern, m.j. & parkes, a. (2010). incommensurability in design science: which comes first – theory or artefact? in d. hart & s. gregor (eds.), information systems foundations: the role of design science, 75-90: anu e-press. davern, m.j., shaft, t. & te’eni, d. (2012) cognition matters: enduring questions in cognitive is research. journal of the association for information systems, 13(4), article 1. de laat, p. b. (2018). algorithmic decision-making based on machine learning from big data: can transparency restore accountability? philosophy & technology 31(4), 525-541. doi:10.1007/s13347-017-0293-z dhaliwal, j. s., & benbasat, i. (1996). the use and effects of knowledge-based system explanations: theoretical foundations and a framework for empirical evaluation. information systems research, 7(3), 342-362. doi:10.1287/isre.7.3.342 giboney, j. s., brown, s. a., lowry, p. b., & nunamaker jr, j. f. (2015). user acceptance of knowledge-based system recommendations: explanations, arguments, and fit. decision support systems, 72, 1-10. gönül, m. s., önkal, d., & lawrence, m. (2006). the effects of structural characteristics of explanations on use of a dss. decision support systems, 42(3), 1481-1493. doi:10.1016/j.dss.2005.12.003 goodman, b., & flaxman, s. (2016a). eu regulations on algorithmic decision-making and a “right to explanation”. paper presented at the icml workshop on human interpretability in machine learning (whi 2016), new york, ny. http://arxiv. org/abs/1606.08813 v1. australasian journal of information systems afrashteh, someh & davern 2020, vol 24, selected papers from the is foundations conference explanations as discourse 13 goodman, b., & flaxman, s. (2016b). european union regulations on algorithmic decisionmaking and a" right to explanation". arxiv preprint arxiv:1606.08813. doi:10.1609/aimag.v38i3.2741 gregor, s. (2001). explanations from knowledge-based systems and cooperative problem solving: an empirical study. international journal of human-computer studies, 54(1), 81105. doi:10.1006/ijhc.2000.0432 gregor, s., & benbasat, i. (1999). explanations from intelligent systems: theoretical foundations and implications for practice. mis quarterly, 497-530. doi:10.2307/249487 günther, w. a., rezazade mehrizi, m. h., huysman, m., & feldberg, f. (2017). debating big data: a literature review on realizing value from big data. the journal of strategic information systems, 26(3), 191-209. doi:10.1016/j.jsis.2017.07.003 habermas, j. (1984). the theory of communicative action (vol. 2): beacon press. hayes-roth, f., & jacobstein, n. (1994). the state of knowledge-based systems. communications of the acm, 37(3), 26-39. lepri, b., oliver, n., letouzé, e., pentland, a., & vinck, p. (2018). fair, transparent, and accountable algorithmic decision-making processes. philosophy & technology. 31(4), 611-627. doi:10.1007/s13347-017-0279-x li, m., & gregor, s. (2011). outcomes of effective explanations: empowering citizens through online advice. decision support systems, 52(1), 119-132. doi:10.1016/j.dss.2011.06.001 limayem, m., & desanctis, g. (2000). providing decisional guidance for multicriteria decision making in groups. information systems research, 11(4), 386-401. doi:10.1287/isre.11.4.386.11874 mao, j., & benbasat, i. (2000). the use of explanations in knowledge-based systems: cognitive perspectives and a process-tracing analysis. journal of management information systems, 17(2), 153-179. doi:10.1080/07421222.2000.11045646 markus, m. l., & topi, h. (2015). big data, big decisions for science, society, and business. paper presented at the report on a research agenda setting workshop. bentley university. martens, d., & provost, f. (2014). explaining data-driven document classifications. 38(1): 7399. doi:10.25300/misq/2014/38.1.04 martin, k. e. (2015). ethical issues in the big data industry. mis quarterly executive, 14(2),, 6785. mcsherry, d. (2005). explanation in recommender systems. artificial intelligence review, 24(2), 179-197. doi:10.1007/s10462-005-4612-x mingers, j., & walsham, g. (2010). toward ethical information systems: the contribution of discourse ethics. mis quarterly, 34(4), 833-854. doi:10.2307/25750707 mittelstadt, b. (2016). automation, algorithms, and politics| auditing for transparency in content personalization systems. international journal of communication, 10, 12. newell, s., & marabelli, m. (2015). strategic opportunities (and challenges) of algorithmic decision-making: a call for action on the long-term societal effects of ‘datification’. the journal of strategic information systems, 24(1), 3-14. doi:10.2139/ssrn.2644093 australasian journal of information systems afrashteh, someh & davern 2020, vol 24, selected papers from the is foundations conference explanations as discourse 14 rader, e., cotter, k., & cho, j. (2018). explanations as mechanisms for supporting algorithmic transparency. paper presented at the proceedings of the 2018 chi conference on human factors in computing systems. ribeiro, m. t., singh, s., & guestrin, c. (2016). "why should i trust you?". proceedings of kdd’16: 1135-1144. doi:10.1145/2939672.2939778 silver, m. (1991). decisional guidance for computer-based decision support. mis quarterly, 15(1), 105-122. silver, m. (2006). decisional guidance: broadening the scope human computer interaction and management information systems: foundations: m.e. sharpe. sinha, r., & swearingen, k. (2002). the role of transparency in recommender systems. paper presented at the chi'02 extended abstracts on human factors in computing systems. smedley, g., & sutton, s. g. (2007). the effect of alternative procedural explanation types on procedural knowledge acquisition during knowledge-based systems use. journal of information systems, 21(1), 27-51. doi:10.2308/jis.2007.21.1.27 someh, i., davern, m., breidbach, c. f., & shanks, g. (2019). ethical issues in big data analytics: a stakeholder perspective. communications of the association for information systems, 44(1), 34. sørmo, f., cassens, j., & aamodt, a. (2005). explanation in case-based reasoning– perspectives and goals. artificial intelligence review, 24(2), 109-143. doi:10.1007/s10462005-4607-7 swartout, w. r., & smoliar, s. w. (1987). on making expert systems more like experts. expert systems, 4(3), 196-208. doi:10.1111/j.1468-0394.1987.tb00143.x tan, w.-k., tan, c.-h., & teo, h.-h. (2012). consumer-based decision aid that explains which to buy: decision confirmation or overconfidence bias? decision support systems, 53(1), 127-141. doi:10.1016/j.dss.2011.12.010 te’eni, d. (2001). a cognitive affective model of organizational communication for designing it. mis quarterly, 25(2), 251-312. ustun, b. & rudin, c. (2016). supersparse linear integer models for optimized medical scoring systems. machine learning, 102(3), 349-391. wang, w., & benbasat, i. (2007). recommendation agents for electronic commerce: effects of explanation facilities on trusting beliefs. journal of management information systems, 23(4), 217-246. doi:10.2753/mis0742-1222230410 ye, l. r., & johnson, p. e. (1995). the impact of explanation facilities on user acceptance of expert systems advice. mis quarterly, 157-172. doi:10.2307/249686 zupon, a., alexeeva, m., valenzuela-escárcega, m., nagesh, a., & surdeanu, m. (2019, june). lightly-supervised representation learning with global interpretability. in proceedings of the third workshop on structured prediction for nlp, 18-28. australasian journal of information systems afrashteh, someh & davern 2020, vol 24, selected papers from the is foundations conference explanations as discourse 15 copyright: © 2020 afrashteh, someh & davern. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v24i0.2519 http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems rolleston 2017, vol 21, research on indigenous ict social media exploration 1 social media exploration 1.1 te kahu rolleston tekahu@wclc.org.nz he uri tēnei mai i ngā tai e papaki tu ana ki mauao, no te ia moana o te awanui, te ia-auraki o tauranga ka rere i te ao ka rere i te po. ko ngaiterangi te iwi, ko ngā moutere whakaruruhau o matakana ngā turangawaewae ōku. koira te pūtake o tōku tāngaengae. tauranga moana tauranga tangata. i descend from the tides that crash and break against my ancestral mountain named mauao. i am of the great running currents of my sea fearing people, the vein of our lives, te awanui. the islands of matakana are where i plant my feet and call home. that is my very foundation from where i will always draw life. tauranga’s sea, and tauranga’s people are one. we have come from surfing waves, to surfing the web, we used to catch fish in our nets, now people catfish the internet. undiscovered and unknown are the depths of this new sea. a new space, for exploration, but what does that really mean? for my iwi, for my tribe, for natives and first nations worldwide? our hands are bound, shackled to keyboards tied. as internet speeds fall and rise with the importance of the tides. will we drown in these currents, in our attempt to keep up with what’s current? who knows! it’s for us, the first generation, of indigenous digital natives, to explore the space, to ensure it’s navigated in a way, which is culturally safe. i need to know, before the end of my time, how will the net and social media impact my sacred traditional marae? because like it or not, don’t ask why for, wi-fi is here on lock like hotspots. people often ask me, how do you get people online to engage with your words? i call those people lone bulls because they just want to be herd. but i tell them the first thing you need to understand, is your social media reach and span. that can be your friend list, of if you have a group or band, it means finding how hot or cold are the cycles of your fans. then you can start viewing it from people’s perspectives, making it easier to get at em. over time, you will be able to roughly predict peoples posting patterns and online habits. australasian journal of information systems rolleston 2017, vol 21, research on indigenous ict social media exploration 2 something cool about groups pages that’s pretty wicked, is they actually give you a breakdown of a pages statistics. so you know which posts get the most activity, on what day, at what time do you get the most traffic visiting. from here you can determine what’s optimum posting time! if you want people to see what you post, then post, what you post when more people are online. basically i’ve found the key that works best for me, for the amount of people your posts reach is timing, and aligning it with when people are in front of the screen. in my opinion there’s 2 more secrets, one, is a network to share your stuff so it keeps reaching. secondly the content of what you’re speaking. speak your truth, that’s how you set movements in movement. that’s why i speak 100 to the power of word, i frack into souls like digging on a mineral search, so i may speak to people’s hearts. art, is not the reason for these words i chisel and carve, these words that i carve, offer a voice to those who have something to say but cant. the wood is people’s thoughts, the chisel both my finger tips typing and tongue. the mallet swings with a tap on the enter tab, or the shift of my lung. social media allowed this to happen, now people read my words online. i’m suddenly a powerful poet with passion, they used to write me off as a darky. now i’m that native that inspires and educates the masses, using his talent and gift for good, while weaving poetry and rapping. biggie told us it was all a dream, but chris brown told us it was all a tweet. copyright: © 2017 rolleston. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems rolleston 2017, vol 21, research on indigenous ict social media exploration 3 paper title (paper title style) australasian journal of information systems lamp 2018, vol 22, editorial annual report 1 annual report john lamp ajis editor-in-chief deakin business school deakin university ajis has continued to grow and develop in 2017. fourteen general research articles appeared, and a further 46 in the following special sections (table 1) on research areas and enhanced conference papers: special section papers research on applied ethics involving emerging ict technologies 4 research on health information systems 8 research on indigenous use of information and communication technologies 10 selected papers from the australasian conference on data mining (ausdm) 9 selected papers from the australasian conference on information systems (acis) 5 table 1: special sections published in 2017 one of the highlights of the year was recognition of the significance of the special section on indigenous use of information and communication technologies by the university of southern california’s centre on public diplomacy (fig 1) figure 1: http://uscpublicdiplomacy.org/story/indigenous-peoples-digital-diplomacy australasian journal of information systems lamp 2018, vol 22, editorial annual report 2 the traditional measures of journal reputation have also improved this year with the ajis hindex jumping three points to 35. ajis readership continues to improve (fig 2), and special note must be made of the increasing readership in southern asia, up 72.2%, and northern america, up 41.3%. figure 2: ajis regional readership in 2017 182 papers were submitted, up 18% on 2016. this continues to place a significant demand on section editors and reviewers, without whom ajis could not function. their input, as always is vital, and very much appreciated. to assist with recruitment of reviewers i presented a workshop at the 2017 australasian conference on information systems (acis) doctoral consortium on journal publication, the reviewing process, and the benefits to phd students, in particular, of participating in this activity. part of the session involved the students assessing several anonymised submissions as if they were reviewers. as well as getting a better understanding of the process, students got to see papers as they were submitted, rather than as they would appear. the session was very well received, and i have been asked to run it again at the acis 2018 dc. the five most popular ajis articles in 2017 were: baskarada, s., & koronios, a. (2013). data, information, knowledge, wisdom (dikw): a semiotic theoretical and empirical exploration of the hierarchy and its quality dimension. australasian journal of information systems, 18(1). doi: http://dx.doi.org/10.3127/ajis.v18i1.748 parkes, a. (2000). is a picture worth a thousand words? the use of non-text items in information systems research. australasian journal of information systems, 8(1). doi: http://dx.doi.org/10.3127/ajis.v8i1.258 pham, h., pham, d., brennan, l., & richardson, j. (2017). information security and people: a conundrum for compliance. australasian journal of information systems, 21. doi: http://dx.doi.org/10.3127/ajis.v21i0.1321 0 2000 4000 6000 8000 10000 12000 14000 australasia northern america southern asia southeast asia northern europe western europe western asia eastern asia eastern africa south america 2016 2017 australasian journal of information systems lamp 2018, vol 22, editorial annual report 3 seddon, p. , & kiew, m. (1996). a partial test and development of delone and mclean's model of is success. australasian journal of information systems, 4 (1). doi : http://dx.doi.org/10.3127/ajis.v4i1.379 swatman, p. (1994). business process redesign using edi: the bhp steel experience. australas ian journal of information systems, 1 (2). doi: http://dx.doi .org/10.3127/ajis.v1i2.422 it is very good to see pham et al (2017) in the top five in its year of publication. the ajis advisory board has been most helpful as always and i must acknowledge them. australasian association for information systems deborah bunker, university of sydney don kerr, university of the sunshine coast australian council of professors and heads of information systems caroline chan, rmit university glenn stewart, queensland university of technology professors and heads of information systems, new zealand annette mills, university of canterbury felix tan, excelsia college to all those who have given their time to ajis in 2017, many thanks, you know we would not exist without your valuable input. copyright: © 2018 lamp. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems burmeister 2018, vol 22, post publication review bullying in the australian ict workplace 1 post publication review al-saggaf, y., & ceric, a. (2017). bullying in the australian ict workplace: the views of australian ict professionals, australasian journal of information systems, 21. doi: 10.3127/ajis.v21i0.1322 review this is a timely and important contribution to information systems (is) literature that gives insights into bullying in the information and communication technology (ict) workplace. it reveals the extent of the problem in the australian context, through the largest survey of australian ict professionals yet conducted. with 27% of respondents identifying bullying as an issue that is not being effectively addressed at present. interestingly their findings revealed that bullying concerns were the most significant amongst mature respondents, such as those in managerial roles. i find that the single biggest drawback of this article is that nowhere does it define what the authors mean by ‘professionalism’. there are numerous definitions of the term in the literature. particularly in relation to ict the definitions have evolved over time (berleur et al., 2008; bowern, burmeister, gotterbarn, & weckert, 2006; burmeister, 2015, 2017). even now, the international federation of information processing are exploring changing it again, due to their efforts to create an international certification standard that will be portable across borders, for ict professionals, either developing systems that cross borders, or for those whose work requires them to work in multiple countries. however, the authors appear to assume that the definition of professionalism is fixed and that all readers will know what that is. thus they refer to ‘professional status’ (section 2 and 4.2.2), ‘unprofessional behaviour’ (section 2), and ‘permanent professionals’ (section 7) without defining what to them professionalism means, whilst spending considerable space in the paper to define other terms, particularly those related to bullying behaviours. another area which the authors could have explored in more depth is that of moral repair. in section 4.1 concerning their findings and in section 3.2 which draws on uk literature, the authors show that women who have been subjected to bullying behaviour frequently avoid the situation ‘by leaving the organisation’. it would have been interesting to know if there are moral repair strategies which management can employ, to avoid losing valuable employees. this is particularly important because some of the literature they pointed to shows that not only do women leave the organisation, but some leave the ict sector altogether. our sector is one in which we have low female participation and many efforts are engaged in to attract more women into our profession. it is all the more important therefore to ensure we do not lose women from the sector and explore what measures can be taken to effect moral repair. finally, i declare a conflict of interest. namely, i was acknowledged in the paper as having contributed to some of the data gathering for the research, and the authors of (al-saggaf & ceric, 2017) are colleagues of mine. i have published with both previously (al-saggaf & burmeister, 2012; al-saggaf, burmeister, & schwartz, 2017; al-saggaf, burmeister, & weckert, 2015; burmeister, phahlamohlaka, & al-saggaf, 2015; vella, burmeister, ceric, & barnden, 2016). australasian journal of information systems burmeister 2018, vol 22, post publication review bullying in the australian ict workplace 2 oliver burmeister, charles sturt university, oburmeister@csu.edu.au author response the authors were contacted but failed to provide a response. references al-saggaf, y., & burmeister, o. k. (2012). improving skill development: an exploratory study comparing a philosophical and an applied ethical analysis technique. computer science education, 22(3), 1-19. al-saggaf, y., burmeister, o. k., & schwartz, m. (2017). qualifications and ethics education: the views of ict professionals. australasian journal of information systems, 21. al-saggaf, y., burmeister, o. k., & weckert, j. (2015). reasons behind unethical behaviour in the australian ict workplace: an empirical investigation. journal of information, communication & ethics in society, 13(3/4), 235-255. al-saggaf, y., & ceric, a. (2017). bullying in the australian ict workplace: the views of australian ict professionals. australasian journal of information systems, 21. doi:10.3127/ajis.v21i0.1322 berleur, j., burmeister, o., duquenoy, p., gotterbarn, d., goujon, p., kaipainen, k., . . . whitehouse, d. (eds.). (2008). ethics of computing committees: suggestions for functions, form, and structure. laxenburg, austria: ifip press. bowern, m., burmeister, o. k., gotterbarn, d., & weckert, j. (2006). ict integrity: bringing the acs code of ethics up to date. australasian journal of information systems, 13(2), 168-181. doi:http://dx.doi.org/10.3127/ajis.v13i2.50 burmeister, o. k. (2015). improving professional it doctorate completion rates. australasian journal of information systems, 19, 55-70. doi:10.3127/ajis.v19i0.1073 burmeister, o. k. (2017). professional ethics in the information age. journal of information, communication & ethics in society, 15(2). burmeister, o. k., phahlamohlaka, j., & al-saggaf, y. (2015). good governance and virtue in south africa's cyber security policy implementation. international journal of cyber warfare and terrorism, 5(1). vella, s. f., burmeister, o. k., ceric, a., & barnden, a. (2016). a systematic narrative review of literature on catholic schools in australia to better understand the role of school leadership deploying and integrating information and communication technology (ict) in this environment. ejournal of catholic education in australasia, 13(1). retrieved from http://researchonline.nd.edu.au/ecea/vol3/iss1/4 copyright: © 2018 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits nonhttp://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems burmeister 2018, vol 22, post publication review bullying in the australian ict workplace 3 commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. paper title (paper title style) australasian journal of information systems aljaroodi, adam, chiong & teubner 2019, vol 23, research article avatars and embodied agents in experimental is research 1 avatars and embodied agents in experimental information systems research: a systematic review and conceptual framework hussain m. aljaroodi school of electrical engineering and computing university of newcastle australia marc t. p. adam school of electrical engineering and computing university of newcastle australia raymond chiong school of electrical engineering and computing university of newcastle australia raymond.chiong@newcastle.edu.au timm teubner einstein center digital future technical university of berlin germany abstract computerised graphical representations of human users and computer agents, known as avatars and embodied agents, have been extensively explored and investigated in information systems (is) research and practice. such digital representations can be employed in either 2d or 3d. in order to facilitate research on user and agent representations and their applications in is, we conduct a systematic literature review and establish the current state of research on humans’ perceptions and behaviours when interacting with avatars and embodied agents. our findings are based on an analysis of 90 articles published in top outlets in the is field. this review identifies 1) different types of avatar and embodied agent-mediated interactions with users, 2) current application domains of such representations, 3) their dimensionality, 4) affected psychological constructs, and 5) practical considerations for the design of such digital representations. finally, we discuss limitations of current research and, based on these, directions for future work. keywords: avatars, embodied agents, experiments, literature review 1 introduction to facilitate computer-mediated transactions at the nexus of the digital and physical world, user interface designers frequently employ design elements that resemble human features as digital representations of humans and computer agents. digital representations of human users are commonly referred to as avatars, while representations of computer agents are referred to as embodied agents (fox et al., 2015). in information systems (is) research and practice, digital representations have successfully been employed in a wide range of australasian journal of information systems aljaroodi, adam, chiong & teubner 2019, vol 23, research article avatars and embodied agents in experimental is research 2 applications, such as recommendation agents in e-commerce (s. lee & choi, 2017; qiu & benbasat, 2009), chat bots (burgoon et al., 2016), tools for behavioural change (sebastian & richards, 2017; song, kim, kwon, & jung, 2013), and tutors in e-learning (beege et al., 2017; heidig & clarebout, 2011). over the past two decades, researchers have conducted experiments to better understand users’ perceptions and behaviours in their interactions with avatars and embodied agents. through experiments, researchers are able to observe actual interactions between users and avatars or embodied agents, and their responses to these interactions, under controlled conditions. importantly, experiments allow researchers to identify causal effects through controlled variations, in order to see how different avatar and embodied agent design features affect outcome variables (shadish et al., 2002). in this paper, we provide a state-of-the-art account of experimental research on human interaction with avatars and embodied agents in 2d as well as 3d environments by means of a systematic literature review (slr), aiming to address the following overarching research question: rq: what is the current body of knowledge in experimental is research on design considerations, application contexts, behavioural outcomes, and influences on users’ psychological constructs of avatars and embodied agents? the review covers 90 articles published in top-tier is journals and conferences. it provides an overview of users’ perceptions of avatars and embodied agents, their effects on human behaviour, and how such graphical representations can be applied in a variety of settings to facilitate computer-mediated interactions between humans, avatars, and computer agents. the main contributions of our slr can be summarised as follows. first, we develop a framework that conceptualises design parameters of avatars and embodied agents, and various interaction types. these interaction types include a user interacting with their own avatar and a user interacting with another user’s avatar or embodied agents. second, we identify the application domains in which avatars and embodied agents are being employed. third, we pinpoint key theories applicable to designing avatars and embodied agents. fourth, we discern common design concepts and, fifth, identify methods used for investigating avatars, embodied agents, and their influence on users’ perceptions and behaviours. this paper therefore may serve as a reference point for researchers working on experiments related to avatars and embodied agents, providing them with a guide on common design parameters and concepts as well as an overview of theories for understanding the interactions between humans, avatars, and embodied agents. for practitioners, this review may serve as a starting point for design considerations when employing avatars and embodied agents in practice. the remainder of this paper is organised as follows. in section 2, we describe the literature review methodology. we then outline the fundamentals of avatars and embodied agents in the context of is research and establish a conceptual framework of digital representations of users or other entities (i.e., another person or a computer agent) in section 3.1 subsequently, in 1 following the terminologies used by ang et al. (2012), we use the term “avatar” to refer to an avatar representing another human being in the digital interaction. note that other people’s avatars are sometimes also referred to as personas in some studies (vasalou & joinson, 2009). clearly, what is a selfavatar and what is an avatar depends on the perspective that one takes, e.g., the user perceiving their own avatar (self-avatar) or that same avatar being perceived by another user (avatar). in other words, the distinction between self-avatars and avatars is mainly driven by the perspective one takes. australasian journal of information systems aljaroodi, adam, chiong & teubner 2019, vol 23, research article avatars and embodied agents in experimental is research 3 sections 4, 5, and 6, we summarise findings on self-avatars, avatars, and embodied agents, respectively. finally, section 7 concludes with a general discussion, implications for the use of avatars and embodied agents in practice, as well as directions for future research. 2 research methodology of literature review this paper follows the methodological guidelines provided by kitchenham & charters (2007) for performing an slr. we focus on studies on avatars and embodied agents conducted under controlled laboratory and field settings with human participants. the process is divided into three distinct stages: plan, conduct, and report, as can be seen in figure 1. in the plan stage, we identify the need for an slr, develop a review strategy, and evaluate it. during the conduct stage, we then execute the search in databases, select appropriate studies, and analyse them. finally, we document and report our findings in the report stage. an initial exploratory search was conducted on google scholar using the search query “avatar” or “virtual self” and “experiment” as a starting point. while reviewing the obtained results, we extracted key terms used in these studies to further develop our search string. next, we selected a well-established set of databases for conducting slrs in the is context (i.e., ais electronic library, sciencedirect, taylor & francis online, wiley online library). we then performed a full-text search on these databases using (“avatar” or “embodied agent” or “virtual self” or “virtual presence”) and “experiment”. the initial search string included the term virtual-self, as this concept is frequently considered in research on avatars and embodied agents, and is hence in line with the research question we would like to address in this literature review. after carefully screening through results of the initial search, we extended the search string to include characteristic terms that have appeared within the reviewed papers. specifically, we included the concept of presence in the search string, as it represents one of the main constructs that is being investigated in digital representations and human interactions, and the term embodied agent was included too, as it refers to graphical representations of computer agents in the interaction and sometimes is used interchangeably with the term avatar. finally, we constrained the search to include only experimental studies because of their potential to reveal actual interactions between humans, avatars and embodied agents under controlled conditions (see also the introduction section). we included all academic journals under the category of information management (im) in the abs academic journal guide 2015 (abs, 2015). in addition, we included full research papers published in two top is conferences as recognised by acphis (2013).2 in terms of selection criteria, we included only peer-reviewed papers reporting on experimental is research with actual interactions between human participants and avatars or embodied agents. we excluded studies that 1) focused only on surveys without actual interactions between users and digital representations, 2) investigated robots rather than virtual representations, 3) included disembodied, that is, non-graphical representations, or 4) were research-in-progress. 2 conferences included are the international conference on information system (icis) and european conference on information system (ecis). these two conferences were selected as they were ranked a by ifip (2015). australasian journal of information systems aljaroodi, adam, chiong & teubner 2019, vol 23, research article avatars and embodied agents in experimental is research 4 figure 1. stages of the slr and distribution of studies we used the defined search string and searched the title, abstract, and keywords of studies in the selected databases. this search resulted in 715 hits (see the conduct stage in figure 1), of which 625 were journal publications and 90 were conference publications. then, the first and second authors independently screened the title, abstract, and keywords against the research question and selection criteria to identify relevant papers for the review. the agreement rate between the two authors was 94.97% (679 out of the 715 papers). discrepancies were resolved by discussion with the fourth author (36 papers, 15 included, 21 excluded). finally, the first author performed a full text review on the studies, resulting in 90 research papers (86 from journals, 4 from conferences). table a1 in the appendix provides an overview of all included studies, providing concise information on the specific type of interaction, dimensionality, context, psychological constructs, environments, and sample size for each study. 3 conceptual framework according to representation theory (rt), the fundamental purpose of an is is to provide users with a faithful representation of some real-world phenomena (recker et al., 2019). the main tenets of rt posit that is will 1) be employed if they provide users with a more cost-effective way to obtain knowledge about a focal real-world phenomenon than observing it directly, and 2) yield higher usefulness the more faithful the phenomena are represented. common examples for the application of rt include data modelling, object-oriented modelling, and process modelling (burton-jones et al., 2017; recker et al., 2019). similar notions can be applied when it comes to avatar design: an avatar provides a digital representation of a person in the real-world, while an embodied agent represents a computer agent. employing these representations can increase cost-effectiveness, e.g., in the provision of health and education services over a distance and/or to a large audience (beege et al., 2017; heidig & clarebout, 2011; noorbergen, adam, attia, cornforth, & minichiello, 2019); it may also increase faithfulness of the representation, as avatars usually have human features that capture the naturalness of interpersonal encounters in the real-world (nowak & rauh, 2006). in this paper, we distinguish two perspectives in terms of human interactions with avatars, namely self and other. the “self” perspective refers to a user’s interaction with their own avatar, often involving facial and body features, gestures, sound, or other forms of behaviour and expression. following this perspective, the avatar is referred to as a self-avatar (vasalou & joinson, 2009). the “other” perspective refers to a user’s view on other humans (computer agents) represented by avatars (embodied agents). distribution of studies # basket of eight (i.e., ejis, isj, isr, jais, jit, jmis, jsis, misq) 6 top is conferences (i.e., ecis, icis) 4 other is outlets in abs ranking (e.g., chb, dss, eswa, ijec, ijhcs, i&m, i&o, it&p, jcmc) 80 σ 90 1) plan stage 2) conduct stage 3) report stage step 3: report findings results (sections 3 – 6) step 1.1: need for slr step 1.2: review strategy step 1.3: evaluate strategy introduction (section 1) search strategy (section 2) step 2.1: search (715) step 2.2: select (90) step 2.3: extract/analyze abstract/keyword/title (153) full-text review of studies wiley online library (46) science direct (478) ais electronic library (169) taylor & francis online (22) australasian journal of information systems aljaroodi, adam, chiong & teubner 2019, vol 23, research article avatars and embodied agents in experimental is research 5 through their self-avatars, users may directly or indirectly engage with others. based on the different spheres and levels of interactions, we develop a conceptual framework that differentiates between five different interaction types, as shown in figure 2. from the figure, we see that interactions with avatars can be distinguished as human-to-self-avatar (h2sa), human-to-avatar (h2a), and self-avatar-to-avatar (sa2a). similarly, humans may interact with embodied agents directly (human-to-embodied-agent, h2ea) or through their avatars (selfavatar-to-embodied-agent, sa2ea). among the 90 reviewed studies, a majority considered sa2a (34), followed by h2a (21), h2ea (20), h2sa (17), and sa2ea (8) interactions. note that here the total adds up to more than 90 because some studies have investigated more than one interaction type. the framework captures commonly examined psychological constructs for specific interaction types. table 1 provides definitions for the psychological constructs that appear in the framework and their usage in different types of interactions. figure 2. conceptual framework of digital representations and their interactions note: all icons used have been sourced from svg silh and pixabay (https://svgsilh.com & https://pixabay.com; creative commons cc0 1.0). the greyed-out icons are not considered in the respective interaction type. for readability, only constructs that are used by at least two studies are shown. realism includes behavioural, emotional, or visual realism. the complete list of constructs can be found in table a1 in appendix a. australasian journal of information systems aljaroodi, adam, chiong & teubner 2019, vol 23, research article avatars and embodied agents in experimental is research 6 constructs description usage anthropomorphism the degree of humanness of the digital representation as perceived by the user h2a: nowak et al. (2015); nowak & rauh (2006, 2008) h2ea: y. kim & sundar (2012a); krämer et al. (2013) attractiveness the degree of attractiveness of the digital representation as perceived by the user h2a: keng & liu (2013); leding et al. (2015); nowak et al. (2015); nowak & rauh (2006); westerman et al. (2015) h2ea: burgoon et al. (2016) credibility the degree to which the digital representation is perceived as believable h2a: nowak et al. (2015); nowak & rauh (2006, 2008) h2ea: y. kim & sundar (2012a) enjoyment the degree to which the user enjoys the interaction with the digital representation h2a: li & lwin (2016); rodrigues et al. (2016) h2ea: s. lee & choi (2017); qiu & benbasat (2009, 2010) sa2a: guadagno et al. (2011); song kim, kwon, et al. (2013) sa2ea: guadagno et al. (2011); jin (2010) identification the degree to which a user identifies themselves with the digital representation h2sa: seo et al. (2017); song, kim, kwon, et al. (2013) immersion the degree to which a user feels immersed in the virtual environment during the interaction with the digital representation sa2a: gerhard et al., (2004); goel et al. (2013); grinberg et al. (2014) sa2ea: gerhard et al. (2004) intention to use the user’s intention to use the digital representation h2a: y. lee et al. (2009); nowak & rauh (2006); rodrigues et al. (2016); westerman et al. (2015) h2ea: gong (2007); s. lee & choi (2017); qiu & benbasat (2009, 2010) h2sa: fox et al. (2013); suh et al. (2011) sa2a: fox et al. (2013); galanxhi & nah (2007); lester & king (2009) likeability the degree to which the user likes the digital representation h2a: bacev-giles & haji (2017) h2ea: brave et al. (2005); gong (2007); groom et al. (2009); kang & gratch (2014); nunamaker et al. (2011) sa2a: ang et al. (2013); guadagno et al. (2011); hanus & fox (2015) sa2ea: guadagno et al. (2011) presence the degree to which a user feels present in the virtual environment with the digital representation h2a: kang & watt (2013); y. lee et al. (2009); li & lwin (2016); rodrigues et al. (2016) h2ea: kang & gratch (2014); y. kim & sundar (2012a); qiu & benbasat (2009, 2010) h2sa: fox et al. (2013); seo et al. (2017); won et al. (2015) sa2a: felnhofer et al. (2018, 2014); fox et al. (2013); franceschi et al. (2009); gerhard et al. (2004); grinberg et al. (2014); guadagno et al. (2011); guegan et al. (2017); hammick & lee (2014); jin (2012); kohonen-aho & tiilikainen (2017); ratan & sah (2015); sutcliffe & alrayes (2012); wu et al. (2015) sa2ea: felnhofer et al. (2018); gerhard et al. (2004); guadagno et al. (2011); kohonen-aho & tiilikainen (2017) australasian journal of information systems aljaroodi, adam, chiong & teubner 2019, vol 23, research article avatars and embodied agents in experimental is research 7 constructs description usage similarity the degree to which the user perceives that the digital representation is similar to them h2a: bacev-giles & haji (2017); j. kim (2011); nowak & rauh (2006); wrzesien et al. (2015) h2ea: beege et al. (2017; tamborini et al. (2018) h2sa: seo et al. (2017); suh et al. (2011); vasalou & joinson (2009); wrzesien et al. (2015) sa2a: dolgov et al. (2014); fehrenbacher & weisner (2017); guegan et al. (2016); h.-k. kim & kim (2016); j. kim (2009); j. kim & park (2011); martens et al. (2018); spence et al. (2013); westerman et al. (2015) trust the degree to which the digital representation is considered trustworthy by the user h2a: balas & pacella (2017); nowak et al. (2015) h2ea: brave et al. (2005); gong (2007); nunamaker et al. (2011); qiu & benbasat (2009); riedl et al. (2011, 2014); tamborini et al. (2018) sa2a: ang et al. (2013); fehrenbacher & weisner (2017); galanxhi & nah (2007); guadagno et al. (2011); jin (2012) sa2ea: guadagno et al. (2011) table 1. definitions of constructs and their usage in different interaction types 3.1 avatars previous research has studied generic avatars (e.g., franceschi et al., 2009), customised avatars (e.g., nowak et al., 2015), or a combination of both (e.g., suh et al., 2011). generic avatars are pre-created with some characteristics such as gender, but cannot be customised by users (dolgov et al., 2014; ratan & sah, 2015). customised avatars, in contrast, allow users to make deliberate choices on appearance (guegan et al., 2016). in h2sa interaction, a user controls the movement and/or appearance of their self-avatar by some controlling device (pessoa et al., 2014). note that h2sa does not necessarily involve other users’ avatars. here, the interaction between the user and their self-avatar often only expresses the user’s inputs through the avatar (won et al., 2015). in h2a interactions, users interact with others’ avatars. they may, for instance, observe the avatar’s appearance and behaviour and implicitly evaluate the entity behind it (nowak & rauh, 2008). in sa2a interaction, human users control their respective avatars to interact with others through them (e.g., sutcliffe & alrayes, 2012). typical purposes are collaborations and meetings in business contexts (guegan et al., 2017; huang et al., 2010), collaborative problem solving (guegan et al., 2016), or multi-person entertainment (dolgov et al., 2014; martens et al., 2018). 3.2 embodied agents embodiment refers to the visual representation of an abstract idea, a thing, or a person (taylor, 2002). an embodied agent, in this sense, describes “a perceptible digital representation whose behaviours reflect a computational algorithm designed to accomplish a specific goal or set of goals” (bailenson & blascovich, 2004, p. 64). the majority of the reviewed studies investigated embodied agents that exhibit at least some human-like features. in this context, anthropomorphism refers to the degree of humanness of embodied agents, that is, “the extent to which an image is perceived to resemble human characteristics and has human morphology” (nowak & rauh, 2006, p. 154). exemplary applications include the provision of information (carlotto & jaques, 2016), product recommendations in e-commerce (qiu & benbasat, 2009), or screening interviews (e.g., nunamaker et al., 2011). in sa2ea interaction, a user controls an avatar and interacts with an embodied agent through their avatar. common australasian journal of information systems aljaroodi, adam, chiong & teubner 2019, vol 23, research article avatars and embodied agents in experimental is research 8 examples can be found in automated business processes such as job interviews or information delivery (y. kim & sundar, 2012b). 3.3 artefact design for digital representations of humans, davis et al. (2009) identified three main factors that affect people’s perception of self-avatars and (others’) avatars. these are 1) avatar appearance, 2) perceived presence (i.e., the sense of being in an environment with another person) (steuer, 1992; won et al., 2015), and 3) immersion (i.e., users’ perception to be interacting with the virtual environment rather than their physical environment) (guadagno et al., 2007). these factors are inherently different to those in the human perception of embodied agents. our conceptual framework provides an overview of the different forms of appearances employed in the reviewed studies. it considers the dimensionality in which the interaction takes place (i.e., 2d and/or 3d), the dynamics of the visual interaction (i.e., static, motion), and the overall level of embodiment (i.e., face only, half body, full body). furthermore, the framework lists a number of design considerations for the different types of digital representations that will be further explored in the subsequent sections. table 2 summarises the artefact design that has been considered for different interaction types. artefact design interaction type total h2sa sa2a sa2ea h2a h2ea body full 15 30 8 6 7 66 half 0 0 0 5 7 12 face only 2 4 0 6 6 18 dynamics static 3 4 0 12 5 24 motion 14 30 8 6 15 73 dimensionality 2d 2 6 0 10 9 27 3d 16 31 8 8 12 75 total 52 105 24 53 61 table 2. artefact design for different interaction types note: numbers here refer to the number of studies for the respective artefact design and interaction type. the total number adds up to more than 90, due to the fact that some studies have investigated more than one interaction type or artefact design. one important aspect in this regard is customisation (e.g., guadagno et al. 2011; nowak & rauh, 2008). customisability refers to the possibility to design and modify an avatar’s appearance (behm-morawitz, 2013), that is, the degree “to which [it] can be created, selected, or changed to comply with user preferences” (teng, 2010, p. 1549). in 36 out of the 90 reviewed studies, researchers allowed participants to customise their self-avatars with personalised features (e.g., see vasalou & joinson, 2009). in other experimental settings, the degree of visual realism of an avatar is controlled to achieve specific interactional outcomes, for instance, to increase avatars’ credibility (e.g., nowak & rauh, 2006). here, self-avatars may depict photorealistic, fictional, or supernatural characters (vasalou & joinson, 2009). 3.4 presence and immersion one central concept for understanding avatars and embodied agents is presence. in our framework, it represents one of the most frequently-investigated constructs. digital interaction is generally considered an unnatural way of human communication, as the interaction partners are separated physically (carter et al., 2015; grudin, 2001). however, avatars can be used to establish presence anyway (kang & watt, 2013), that is, user a’s sense of being in an australasian journal of information systems aljaroodi, adam, chiong & teubner 2019, vol 23, research article avatars and embodied agents in experimental is research 9 environment with user b, where users are digitally represented across space or time, and react to each other or to stimuli (heeter, 1992; steuer, 1992; won et al., 2015). there are different concepts of presence, which include individual-, physical-, social-, co-, tele-, and virtual presence (see franceschi et al. (2009) and nowak & biocca (2003) for details). closely related to presence, immersion refers to the degree to which users experience being within a virtual rather than in their actual physical environment (guadagno et al., 2007). realistic environments yield higher presence, which in turn lead to higher immersion (guadagno et al., 2007; jin, 2012). in gaming, for example, teng (2010) found that higher immersion enhances user loyalty to the game. immersed users can also be more socially engaged (grinberg et al., 2014).3 3.5 experimental research on avatars and embodied agents laboratory and field experiments are typically employed to investigate human behaviour when interacting with avatars and embodied agents. in fact, the majority of experimental studies in the field of is involving avatars or embodied agents have been conducted in laboratory settings, where researchers are able to tightly control environmental parameters, mitigate confounding effects, and thus can establish causality of effects between system design and human perceptions, physiology, and behaviour. in our slr, for example, as many as 84 studies were conducted in laboratory environments, while only five studies were conducted in the field and one based on both laboratory and field experiments. the six studies that conducted field experiments controlled different aspects of real-world settings. these include customers of an electronic banking application (rodrigues et al., 2016), self-tracking of physical activities in users’ everyday environments (sanchez-valdes & trivino, 2015), airport security screenings (nunamaker et al., 2011), and students’ performance, engagement, and presence in an online course (barata et al., 2017; franceschi et al., 2009; lester & king, 2009). 3.6 application contexts avatars and embodied agents have become widely used as design elements in is research. by providing a graphical representation of humans and computer agents, they have been shown to create positive user experience (e.g., usability, collaboration), support learning processes (e.g., performance, engagement), and facilitate behavioural intention (e.g., physical activity, usage intention, purchase intention) in a wide range of applications. based on the studies we reviewed, the application contexts include business, education, entertainment, health, and social interaction. table 3 summarises the number of studies for each application context and interaction type. 3 importantly, the use of avatars can also increase involvement and trust between users behind the avatars. the outlined concepts of presence and immersion are central to research on avatars, and we will repeatedly use and refer back to these notions in the subsequent sections. australasian journal of information systems aljaroodi, adam, chiong & teubner 2019, vol 23, research article avatars and embodied agents in experimental is research 10 context interaction type total h2sa sa2a sa2ea h2a h2ea business 1 6 1 6 8 22 education 3 5 0 1 1 10 entertainment 6 8 1 1 5 21 health 2 0 2 1 1 6 social 5 15 4 9 3 36 total 17 34 8 18 18 table 3. application contexts for different interaction types note: numbers here refer to the number of studies for the respective context and interaction type. the total number adds up to more than 90, due to the fact that some studies have investigated more than one interaction type for the respective context. 4 interaction with self-avatars 4.1 self-avatar concepts digital environments enable users to interact and communicate in ways that go beyond those of traditional channels (goel et al., 2013; hämäläinen et al., 2008; mccreery et al., 2012). to do so, they need to represent themselves therein (mallan, 2009). self-avatars function as tools for representation, immersion, and interaction with others in virtual worlds (ducheneaut et al., 2009; koles & nagy, 2012). this section synthesises the results of studies on a user’s interaction with their own avatar, covering the design dimensions customisability, realism, and time, as well as the avatar’s effects on user perceptions and behaviour. overall, 11 studies (13.1% based on our slr) considered h2sa interactions in different contexts such as health (h. e. kim et al., 2017; sanchez-valdes & trivino, 2015), business and entertainment (kaye et al., 2018; peña & chen, 2017; won et al., 2015), and education and social interaction (seo et al., 2017; song, kim, kwon, et al., 2013; vasalou et al., 2008; vasalou & joinson, 2009). for the dimension of time, some avatar concepts refer to a future state of the represented person (song, kim, kwon, et al., 2013), while avatars in a now state refer to the current or currently desired state. a special case of future self-avatars is the idealistic representation. based on the self-idealism paradigm, an ideal self refers to what the user aims to become in the future and may include aspects of education, socialisation, role models, societal norms, and preferences (dunn & guadagno, 2012). for the dimension of realism, there are mirrored and fictional representations (e.g., peña et al., 2016; suh et al., 2011; vasalou & joinson, 2009). the concept of a mirrored self refers to selfavatars that resemble the actual person as much as possible (vasalou et al., 2008), which may include mirrored behaviours and gestures, representing aspects of the user in the real world (burton-jones et al., 2017; recker et al., 2019). it can be seen as a simulated, mirror version of the self, and reflective of the offline self, as well as has the potential to influence the individual in meaningful ways. for example, previous research has demonstrated that avatars that show the consequences of smoking by means of altered physical appearance are capable of creating a sense of risks associated with smoking (song, kim, kwon, et al., 2013). the concept of fictional self, in contrast, represents the user as a fictional, possibly fantastic character. users may represent themselves as fictional characters to escape from their actual selves (greenwood, 2008), or to gain new perspectives, experiences, and knowledge (shedlosky-shoemaker et al., 2014). for example, a user may hide their identity and use an authoritative or persuasive avatar for a new role. australasian journal of information systems aljaroodi, adam, chiong & teubner 2019, vol 23, research article avatars and embodied agents in experimental is research 11 4.2 influences on user perceptions and behaviours while much research considers how one user’s avatar affects other users, a user’s (self-) avatar can also affect their own perceptions and behaviours in the virtual environment and in the real world (yee et al., 2009). this phenomenon is known as the proteus effect (yee & bailenson, 2007). the effect is rooted in self-perception theory (yee et al., 2009), positing that there will be modifications in self-perception based on the behaviour and/or characteristics of a user’s avatar (fox et al., 2013). for instance, users may adopt new behaviours (desirable or undesirable) from their self-avatars (fox & ahn, 2012; seo et al., 2017), and exhibit increased confidence and willingness to disclose information when being represented by attractive selfavatars (yee and bailenson, 2007). also, users exhibit increased physical activity when represented by athletic instead of obese self-avatars (peña & kim, 2014). to understand the proteus effect, it is important to consider how the characteristics of a user’s avatar affect behaviour specifically. relevant mediating constructs in this regard include 1) self-relevance, 2) self-identification, 3) emotional attachment, 4) presence, and 5) anthropomorphism (behmmorawitz, 2013; jin, 2012; nowak & rauh, 2006; pessoa et al., 2014; ratan & dawson, 2016; seo et al., 2017; song, kim, kwon, et al., 2013; vasalou et al., 2008). self-relevance describes “the extent to which the user of a mediated representation that appears and/or behaves like the user is closely connected to the self” (ratan & dawson, 2016, p. 3). selfidentification refers to “the cognitive connection between an individual and an avatar, with the result being that the individual regards the avatar as a substitute self or has such an illusion” (suh et al., 2011, p. 715). both concepts capture how users identify and relate to their digital representation. importantly, customisability increases self-identification and self-relevance (bailey et al., 2009; lim & reeves, 2010). in turn, self-identification and self-relevance are found to increase emotional attachment with the avatar (suh et al., 2011). also, users pay more attention and feel physically more similar to self-avatars when these avatars reflect them visually as compared to avatars that reflect others (including celebrities, ideal faces, and strangers) (seo et al., 2017). next, emotional attachment describes “an emotion-laden, target-specific bond between a person and a specific object” (thomson et al., 2005, p. 77). higher degrees of self-relevance and selfidentification typically yield stronger emotions when interacting with the environment through one’s avatar (ratan & dawson, 2016; ravaja et al., 2006). in addition, users who customised their self-avatar also experience greater emotional attachment to it – which, in turn, increases the avatar’s power to affect their behaviour (h.-k. kim & kim, 2016). emotional attachment with one’s avatar also increases perceived presence in the virtual environment, which can be defined as the state in which “users experience their virtual self as if it were their actual self, perhaps even leading to an awareness of themselves inside the virtual environment” (tamborini & skalski, 2006, p. 227). also, it has been shown that self-avatars can stimulate presence for blind users, helping them to explore environments virtually and identify objects in virtual spaces before visiting the actual site (cobo et al., 2017). anthropomorphism captures how a human-like avatar appears, including naturalness and richness with regards to typical human cues (nowak & rauh, 2006). in the context of selfavatars, it has been shown that this factor increases users’ emotional attachment and their perceptions of the avatar’s credibility (nowak & rauh, 2006). evil-looking and corpse-like avatars, on the other hand, affect users’ perception of engagement and behaviours negatively when contrasted against heroic and attractive avatars (peña & chen, 2017). australasian journal of information systems aljaroodi, adam, chiong & teubner 2019, vol 23, research article avatars and embodied agents in experimental is research 12 5 interaction with another person’s avatar 5.1 avatar concepts in this section, we explore how users interact with other users represented by avatars. overall, 22 of the reviewed studies (26.2%) considered h2a interactions, that is, the direct interaction of a user with another user’s avatar. moreover, 28 studies (33.3%) considered sa2a interactions, that is, interactions between avatars. the studies are of various contexts such as entertainment (e.g., dolgov et al., 2014; dunn & guadagno, 2012; goel et al., 2013; martens et al., 2018), education (e.g., ang et al., 2013; franceschi et al., 2009; h. e. kim et al., 2017; lester & king, 2009; sutcliffe & alrayes, 2012), business (e.g., hanus & fox, 2015; nowak et al., 2015), health (e.g., spence et al., 2013), and social interaction (e.g., jin, 2012; kohonen-aho & tiilikainen, 2017; wu et al., 2015). previous research suggested two marked goal-directed dimensions of avatar design, namely persuasiveness and empathy/emotionality. persuasive avatars, also referred to as authoritative avatars (beale & creed, 2009), are avatars designed to affect users’ choices (bengtsson et al., 1999; hanus & fox, 2015). one way to achieve persuasiveness is to give users the ability to customise the avatar before the interaction (hanus & fox, 2015). for instance, an avatar may be customised to represent a doctor with high authority in the medical field (peña et al., 2009). in contrast, the concept of empathic/emotional avatars refers to avatars designed with nonverbal cues and behaviours such as smiling, which can create an emotional connection with users (h.-k. kim & kim, 2016). felnhofer et al. (2018) found that users interacting with avatars that are controlled by other users reported higher levels of empathy and involvement than users interacting with embodied agents. 5.2 influences on user perceptions and behaviours how an avatar is perceived by its interaction partner(s) is crucial for how it affects their behaviours. first, avatars of high anthropomorphism are perceived as more credible by their interaction partners (nowak & rauh, 2008) and can improve communication (kang & watt, 2013; khashe et al., 2017). visually realistic avatars are found to have higher capability of mitigating undesired and facilitating desired behaviours than unrealistic ones (nowak et al., 2015), e.g., promoting compliance (khashe et al., 2017). in this vein, kang & watt (2013) found that more visually realistic avatars elicit higher levels of immersion, presence, and trust, which enriches interaction between users and creates more truthful interactions (hooi & cho, 2013; sutcliffe & alrayes, 2012), and increases users’ intention to work together to achieve a common goal (lim & reeves, 2010). ang et al. (2013) found that perceived trust and likeability increase if an avatar exhibits human-like behaviours. thus, conveyed by perceptions of presence and trust, visually realistic avatars will typically have stronger impacts on user behaviour (khashe et al., 2017). h. e. kim et al. (2017) found that avatar-based training can lower anxiety and facilitate interpersonal encounters for patients with social anxiety. second, similarity refers to the perceived degree of psychological and/or physiological resemblance between a user and their interaction partner’s avatar, including visual appearance, demographic factors, values, beliefs, behaviours, education, training, or knowledge (mccroskey et al., 1975; nowak & rauh, 2006). higher similarity yields stronger perceptions of group belonging and virtual team identification (guegan et al., 2017; j. kim & park, 2011; midha & nandedkar, 2012), playing an important role for interaction effectiveness (mccroskey et al., 1975; webster & wong, 2008). guegan et al. (2017) and wrzesien et al. (2015) australasian journal of information systems aljaroodi, adam, chiong & teubner 2019, vol 23, research article avatars and embodied agents in experimental is research 13 also noted that similarity between avatars and users impacts social behaviours and group structuring. bailenson et al. (2006) pointed out that users tend to find others who are physically and psychologically similar to themselves not only more attractive, but also more persuasive. however, this perception of similarity and presence depends on users’ senses of shared context with other users (kohonen-aho & tiilikainen, 2017). users pay more attention within virtual environments when avatars reflect their own gender (martens et al., 2018). users represented by foreign avatars are perceived not similar to them, considered as out-group members, and hence excluded from the group (schneider et al., 2017). moreover, the perception of similarity is not limited to gender and physical appearance, but also related to ethnicity and culture. for example, spence et al. (2013) found that avatars that reflect users’ ethnicity are associated with an increased usage intention compared to avatars that do not. third, beyond realism and similarity, leding et al. (2015) showed that avatars with high perceived attractiveness exert strong influences on human behaviour. in this regard, westerman et al. (2015) found that users are more inclined to interact with physically attractive rather than unattractive avatars. for online social interaction, bacev-giles & haji (2017) found that users rated avatars as more attractive and likeable if the avatars are perceived to be similar to them. for online advertisements, keng & liu (2013) found that using dynamic, attractive, and 3d (as compared to static, 2d, or unattractive) avatars increase advertisement effectiveness. moreover, guadagno et al. (2011) found that a smile can create a sense of connection and increase trust between the interaction partners, enjoyment, and satisfaction. however, when avatars are compared to real human images directly, balas & pacella (2017) found that users rated avatars less trustworthy. also, people would refuse to share knowledge with a colleague represented by an avatar (fehrenbacher & weisner, 2017). fabri et al. (2005) found that the use of emotional (as compared to non-emotional) avatars increase user involvement in social communication tasks. fourth, users are more likely to be persuaded if an avatar exhibits angry (rather than soft) and/or male (rather than female) voice (lehto & oinas-kukkonen, 2011). in a recent study (khashe et al., 2017), it is found that users may be more easily persuaded when a message is communicated by avatars rather than voice or text. in this regard, androgyny refers to “a rating of the avatar’s (lack of) masculinity or femininity” (nowak & rauh, 2006, p. 154), affecting several perceptual and behavioural measures. for example, high avatar androgyny is found to reduce the associated user’s credibility (nowak & rauh, 2008). similarly, inappropriately dressed avatars are rated as less anthropomorphic and less credible (nowak et al., 2015). finally, uniqueness indicates that an avatar is different from other digital representations. uniqueness of avatars, such as unusual physical features, or social or psychological differences, can impair interaction effectiveness (kang & watt, 2013; j. kim, 2009). j. kim (2011) stated that users represented by avatars that have visual cues similar to those of their group members perceive a higher level of threat to their uniqueness, compared to those represented by avatars with unique visual cues, leading to lower agreement rates with others. 6 interaction with embodied agents 6.1 embodied agent concepts in this section, we focus on embodied agents and their effects on interaction partners. overall, 19 of the reviewed studies (22.6%) considered h2ea interactions and 4 (4.8%) considered sa2ea interactions. the studies are of multiple contexts such as business (e.g., s. lee & choi, australasian journal of information systems aljaroodi, adam, chiong & teubner 2019, vol 23, research article avatars and embodied agents in experimental is research 14 2017; nunamaker et al., 2011; qiu & benbasat, 2009), entertainment (e.g., brave et al. 2005; mousas et al. 2018; peña et al. 2016), education (e.g., ang et al., 2013; carlotto & jaques, 2016), health (e.g., beege et al., 2017; jin, 2010; y. kim & sundar, 2012b), and social interaction (e.g., dechant et al., 2017; kang & gratch, 2014; krämer et al., 2013). the computer as social actors (casa) paradigm posits that humans apply social rules and norms when interacting with computers (bailenson & blascovich, 2004; nass et al., 1994). specifically, visually and behaviourally realistic embodied agents yield higher engagement and involvement between interaction partners (burgoon et al., 2016; gerhard et al., 2004). hence, much of the existing literature on embodied agents is centred on different forms of agent realism, where the differences between human, behavioural, visual, and emotional realism are distinguished. different design considerations for embodied agents in terms of 1) realism and 2) persuasive and empathic design concepts have also been suggested by previous research. 6.2 realism first, human realism of embodied agents refers to behavioural, visual, and emotional characteristics (e.g., face and voice) of embodied agents that should be consistent with human characteristics (mitchell et al., 2011). it represents an important factor for humans as they are more likely to interact with human-like than non-human-like embodied agents (groom et al., 2009; parise et al., 1999). in this sense, higher human realism may contribute to a higher degree of faithfulness of the digital representation, as it more closely resembles interpersonal interactions in the real world. second, behavioural realism refers to the presence of natural movements, gestures, and reactions of embodied agents (von der pütten et al., 2010). manipulation of behavioural realism was found to influence human and avatar responses (bailenson et al., 2005; slater et al., 2009), where higher levels of behavioural realism would lead to more likeable and trustworthy embodied agents (groom et al., 2009). high levels of behavioural realism are usually considered more appropriate in terms of social behaviour and responses (guadagno et al., 2007; nass & moon, 2000). embodied agents equipped with realistic motion and physical appearance have greater influence than static or remoted embodied agents, as they stimulate genuine social interactions (mousas et al., 2018). third, visual realism refers to “the degree to which the images of the simulated world are perceived to be real” (c. lee et al., 2013, p. 548). according to slater et al. (2009), visual realism has two components. first, geometric realism captures whether the digital representation looks like the real object that it represents. second, illumination realism refers to the fidelity of lighting (slater et al., 2009; strauss, 1990). notably, visually detailed embodied agents are found to create a sense of social presence (kwon et al., 2013). fourth, emotional realism refers to the expression of emotions similar to that of humans (beale & creed, 2009). emotion expression entails the ability to establish emphatic relationships between the embodied agents and users (ahn et al., 2007). in e-learning, for instance, embodied agents with realistic emotional expression can enhance learning experiences and outcomes (dickey, 2005). as beale & creed (2009) discussed, the importance of emotional consistency in embodied agents and appropriateness to the context could enhance the overall interaction. australasian journal of information systems aljaroodi, adam, chiong & teubner 2019, vol 23, research article avatars and embodied agents in experimental is research 15 6.3 persuasive and empathic embodied agents human-like embodied agents (e.g., a salesperson; hanus & fox, 2015) can be more persuasive when compared to using text or voice in the context of e-commerce (qiu & benbasat, 2010). importantly, embodied agents that represent human features are likely to be viewed as more authoritative and persuasive by their interaction partners (qiu & benbasat, 2009). for example, bailenson & yee (2005) found that embodied agents with high behavioural realism that mimic user gestures are more persuasive and likeable than agents without this feature. in the context of health, however, sebastian & richards (2017) found that embodied agents representing doctors are not more persuasive than actual medical doctors for mental health patients. empathic embodied agents display emotions, which can influence users by increasing likeability and trustworthiness (brave et al., 2005). for education, maldonado et al. (2005) found that emotional embodied agents positively enhanced students’ performance, experience, and perceptions of the embodied agents’ intelligence and credibility. beege et al. (2017) noted that embodied agents’ age is irrelevant when activated alongside with text. additionally, bickmore & picard (2005) found that embodied agents equipped with a range of non-verbal behaviours (e.g., hand gestures, nod, eye gaze) increase the connection between users and embodied agents, which lead to greater likeability and trust. importantly, users perceived empathic embodied agents as more caring, sociable, likeable, and trustworthy, and they reported to perceive significantly higher levels of support when an embodied agent created the feeling of an actual partner during the interaction (brave et al., 2005; s. lee & choi, 2017). 6.4 the uncanny valley while human realism is important, all too realistic embodied agents bear the risk of making people feel uncomfortable and often result in negative reactions (burgoon et al., 2016; mara & appel, 2015). this effect is known as the “uncanny valley”, which, according to macdorman & ishiguro (2006) and mori (1970), refers to human-like digital representations that appear almost, but not exactly like real human beings and consequently stimulate feelings of eeriness and revulsion. since people usually know that the displayed entity is artificial (regardless of whether representing a human or not), and not a video stream of an actual human being, interaction design should keep a safe distance from realism in order to avoid the uncanny valley effect. in this sense, designers may employ a stylised or “comicified” representation with empathic features (teubner, adam, camacho, & hassanein, 2014). it is also essential to ensure a balance between appearance and behaviour of embodied agents to avoid the uncanny valley effect (allbeck & badler, 2004; groom et al., 2009; mitchell et al., 2011). on one hand, users dislike embodied agents that are too realistic (e.g., human replica; groom et al., 2009) and exhibit negative responses to unrealistic (non-human-like) embodied agents such as zombies (mousas et al., 2018). on the other hand, people experience lower levels of satisfaction when communicating with a less realistic (e.g., animalistic) digital representation. hence, designers should include characteristics that increase relatedness, feelings of being understood, attractiveness, empathy, and so forth that appear natural to users to avoid the feeling of eeriness. interacting with embodied agents that do not reflect users’ ethnicity is rated as less trustworthy compared to interacting with embodied agents that do (tamborini et al., 2018). australasian journal of information systems aljaroodi, adam, chiong & teubner 2019, vol 23, research article avatars and embodied agents in experimental is research 16 it is worth noting that this eeriness is not only about human-like physical appearances of embodied agents, but also about the display of emotion states. the design of embodied agents should focus not only on human-like appearance and behaviour, but also on emotions displayed by embodied agents (beale & creed, 2009; saygin, 2011). an embodied agent that smiles with a concerned voice is suggested to be problematic, as it exhibits inconsistency in audio-visual realism (beale & creed, 2009), which may in turn reduce users’ perceived faithfulness of the digital representation. designers should, therefore, consider contextual appropriateness to embodied agent emotional displays to avoid the uncanny valley effect. 7 discussion and conclusion 7.1 summary of findings over the past twenty years, researchers from the is domain have conducted controlled experiments in laboratory and field environments to better understand the interactions of human users with digital visual representations of humans (avatars) and computer agents (embodied agents). to stimulate research that extends the knowledge of avatars and embodied agents, and potentially enables innovative applications of such digital representations, we conducted the present slr. based on this review, we provided a comprehensive overview of the extant literature on experimental research on avatars and embodied agents and identified key aspects that affect human perceptions and behaviours when interacting with avatars or embodied agents. table 4 provides a summary of our findings. interaction types design considerations psychological constructs examples of implications of design choices examples of interaction outcomes h2sa customisation, proteus effect similarity users feel more self-relevance and similar to their avatars if they can customise them positive effects on helping behaviour (e.g., dolgov et al., 2014) presence users feel more socially present using their self-avatars increased emotional attachment to avatars (e.g., h.-k. kim & kim, 2016) h2a / sa2a customisation, realism, proteus effect similarity users feel similar to customised avatars that resemble them in terms of gender, physical appearance, and ethnicity increased sense of group belonging (e.g., guegan et al., 2017) attractiveness users prefer to interact with or use attractive avatars reduced risk-taking behaviour (e.g., h.-k. kim & kim, 2016) anthropomorphism users prefer to interact with more human-like avatars in terms of visual, behavioural, and emotional realism perceptions of higher naturalness in interaction (e.g., garau et al., 2003) presence users feel more socially present in the interaction with human-like avatars higher degrees of immersion (e.g., franceschi et al., 2009) australasian journal of information systems aljaroodi, adam, chiong & teubner 2019, vol 23, research article avatars and embodied agents in experimental is research 17 interaction types design considerations psychological constructs examples of implications of design choices examples of interaction outcomes h2ea / sa2ea realism, uncanny valley anthropomorphism users prefer adequately realistic and human-like embodied agents that exhibit consistency in terms of visual, behavioural, and emotional realisms (human-like realisms) irritation and disengagement for too low (e.g., mousas et al., 2018) or too high (e.g., macdorman et al., 2009) levels of anthropomorphism enjoyment users perceive interactions with embodied agents as less enjoyable if these exhibit inconsistency in terms of visual, behavioural, and emotional realisms disengagement from the interaction if not enjoyable (e.g., groom et al., 2009) similarity users prefer to interact with embodied agents that are similar to them (e.g., in terms of gender, appearance, or ethnicity) higher involvement and trust in the embodied agent (e.g., tamborini et al., 2018) presence users prefer to interact with humanlike embodied agents higher levels of liking and credibility as a source of information (e.g., y. kim & sundar, 2012a) table 4. summary of findings from the slr in the context of human interaction with self-avatars (h2sa), experiments have shown that when people are given the freedom to customise their avatar, it would not necessarily reflect their actual selves (vasalou & joinson, 2009). customisation enables users to create a sense of relevance with their avatars, which, in turn, can increase identification with the self-avatar. notably, if there is a strong sense of self-relevance, this may affect the user’s behaviour positively or negatively (proteus effect) (fox et al., 2013). emotional attachment with the selfavatar is an important aspect of the digital representation if the purpose of interaction is to change user behaviour. perceived similarity and customisation of digital representations lead to emotional attachment with avatars (h.-k. kim & kim, 2016) and higher levels of selfpresence (seo et al., 2017). basically, customisation should be available to facilitate behavioural change, as users would customise their avatars to their liking. in the context of human and self-avatar interaction with another person’s avatar (h2a and sa2a), the user or self-avatar interacts with another person’s avatar to achieve a common goal. research has shown that avatar representations affect the interaction partners’ perceptions via the construct of anthropomorphism. garau et al. (2003) found that interactions with realistic avatars, in terms of visual (human-like) and behavioural realisms (consistent with human behaviours), are considered as more natural (i.e., to feel more like a face-to-face interaction). attractiveness of digital representations influences the interaction partners’ behaviours in terms of risk taking (h.-k. kim & kim, 2016). psychological similarity emphasises the importance for the avatar to be part of a team (guegan et al., 2017), which means that the avatar’s customisation should be consistent, so it can be identified as part of the team or group. uniqueness and similarity are two important constructs for ensuring that an avatar can be identified by the interaction partners. immersion and presence, which work interchangeably, are important for h2a and sa2a interactions too. that is, presence increases immersion and vice versa. presence is magnified by the fact the interaction partners can see and hear each other (franceschi et al., 2009), which attracts user attention. interacting with avatars can result australasian journal of information systems aljaroodi, adam, chiong & teubner 2019, vol 23, research article avatars and embodied agents in experimental is research 18 in behavioural change due to the proteus effect, which affects users’ behaviours either by acting similarly or differently to other avatars in the interaction. in the context of human and self-avatar interaction with an embodied agent (h2ea and sa2ea), research showed that human realism in an embodied agent should reflect natural human behaviour, appearance, and emotion. it is important to ensure that all human realism dimensions are matched with actual human behaviour, emotion, and visual cues in order to avoid the uncanny valley effect (groom et al., 2009), which means that unmatched human realism can irritate people and lead to disengagement (mitchell et al., 2011). human realism is an important aspect of embodied agents’ representation, to create a digital representation that has human-like features and also to create a sense of social cues in the interaction. this is because humans usually prefer to digitally interact with a representation that is similar to them. the perception of similarity here is not limited to physical appearance or gender between users and digital representations, but also includes ethnicity and culture (tamborini et al., 2018). human realism and psychological similarity can increase the sense of presence, immersion, and involvement with the embodied agent. these, in turn, will ensure enjoyment, information credibility, likeability, and trust between the interaction partners. similarly, information provided by an embodied agent is more credible and believable if it exhibits human-like features (kim & sundar, 2012a), and if these features are consistent with actual human features. expectedly, likeability of embodied agents hinges on matching human features realistically. in summary, human realism can induce trust (riedl et al., 2014), which enables the interaction partners to share information or work collectively with the embodied agent. to summarise, appearance and behaviour of avatars and embodied agents are important antecedents of interaction outcomes and may lead to negative (e.g., disengagement and untrustworthiness) or positive (e.g., enjoyment) user reactions. according to guadagno et al. (2007), the more anthropomorphic an embodied agent or avatar appears, the more realistic its interactional behaviour such as facial expressions or smiles needs to be for the digital representation to stimulate appropriate social/behavioural responses. hence, in line with rt (burton-jones et al., 2017; recker et al., 2019), avatars and embodied agents may yield higher faithfulness (e.g., in terms of credibility, similarity, presence) if these representations closely resemble the real-world phenomena of human cues in interpersonal interactions (e.g., groom et al., 2009; nowak & rauh, 2006). in contrast, less anthropomorphic representations, that is, with inconsistencies in behavioural, emotional, and visual realism, may lead to adverse outcomes in the interaction between humans and digital representations. 7.2 practical implications most digital interactions of avatars and embodied agents focus on stimulating changes in user experience (e.g., emotion) and facilitating targeted behaviours. based on the present literature review, we have identified six factors designers should particularly take into account when considering the design and implementation of avatars and embodied agents in practice. first, designers should be aware of the proteus effect, which relates an avatar’s characteristics to its “owner’s” own behaviour (fox et al., 2013; peña et al., 2009, 2016; yee and bailenson, 2007). the proteus effect occurs when users are represented by avatars in a virtual environment (yee et al., 2009). it can have positive implications when there is a need to change certain behaviours or perceptions. in this sense, a positive appearance of avatars can increase user confidence – especially in conversations or negotiation tasks (won et al., 2015; yee & australasian journal of information systems aljaroodi, adam, chiong & teubner 2019, vol 23, research article avatars and embodied agents in experimental is research 19 bailenson, 2007). expectedly, a negative connotation (e.g., overweight avatars) can dampen user motivation and mood by reinforcing unhelpful judgment, while an athletic avatar may promote liveliness and boost physical activity (peña & kim, 2014). it is hence important for designers to carefully take into account the proteus effect, as their virtual designs may have unimagined or undesired consequences in the physical world (h.-k. kim & kim, 2016). second, designers need to consider the uncanny valley effect, which occurs when the appearance of a representation looks almost like a real human, but just not fully so (beale & creed, 2009; tinwell et al., 2011). since virtually all avatar designs will not achieve full human realism, designers should – despite their efforts to embed human and social cues such as facial features and gestures – keep a safe distance from all too realistic avatars. in this sense, a deliberate abstraction and stylisation, for instance, using comic faces, may be a viable option. moreover, they should employ attractive and empathetic avatars to reduce uncanny impulses. third, designers need to consider the concept of presence for the design of digital interaction, as this does not represent the natural form of communication among humans. designers should focus on features that stimulate presence to make the interaction appear social. previous research has shown that digital representations that exhibit consistent behavioural realism yield greater social influence (kang & gratch, 2014). importantly, digital representations that are similar to users are perceived as more sociable, enjoyable, and useful to interact with than dissimilar ones (bacev-giles & haji, 2017; qiu & benbasat, 2010; seo et al., 2017). designers should be aware of this to stimulate presence. fourth, persuasive design of digital representations is used to influence users’ behaviours or decisions towards a specific idea. research has shown that users prefer to interact with powerful and authoritative avatars, which can achieve better outcomes (fox et al., 2013; schultze, 2011). also, digital representations with strong facial expressions increase persuasiveness. hence, for behaviour change, more engaged avatars are preferable over passive ones (hyde et al., 2015). moreover, persuasion can be supported by higher anthropomorphism (e.g., using detailed facial expressions). overall, more visually realistic avatars convey stronger messages, increase the chances to persuade users (khashe et al., 2017) and, in turn, guide behaviour (aljaroodi, adam, chiong, cornforth, & minichiello, 2017; noorbergen et al., 2019). fifth, designers should consider empathic elements in designing digital representations. empathy creates a sense of social interaction for the interaction partners. experiments with digital representations have shown that a simple display of emotion can change the result of an interaction. importantly, users dislike embodied agents that display inconsistent emotions (beale & creed, 2009) and exhibit negative responses to agents with pale and gaunt appearances (mousas et al., 2018). additionally, empathic elements should be appropriately matched with human emotions and their transitions (burgoon et al., 2016). empathic elements are important aspects of designing digital representations, because they make the interaction more supportive, enjoyable, trusting, and open for the users (guadagno et al., 2011). empathy leads to more positive interactions, including greater likeability and trustworthiness between the interaction partners (brave et al., 2005). designers should consider including characteristics of non-verbal and verbal behaviour to increase emotional attachment (beale & creed, 2009). lastly, designers should ensure that customisability is available to users, as it enhances the connection between users and their digital representations (hanus & fox, 2015; ratan & sah, 2015). enabling users to create their own avatars would make the users more carefully engaged australasian journal of information systems aljaroodi, adam, chiong & teubner 2019, vol 23, research article avatars and embodied agents in experimental is research 20 in preventative (e.g., quitting unhealthy habits) or proactive behaviour (e.g., picking up healthy routines) (y. kim & sundar, 2012b). customisation is used to change or enhance the user’s behaviour and try to elevate the user’s level of emotional attachment (ducheneaut et al., 2009; hanus & fox, 2015). therefore, designers should ensure that digital representations can be customised by the users to create a stronger emotional connection. 7.3 future research directions building on the findings of our literature review, we have identified three major directions for future is research on avatars and embodied agents. first, in terms of research methodology, it needs to be noted that the majority of reviewed studies were exclusively conducted experiments in laboratory settings (84 out of 90 studies) and only six studies considered a field setting (e.g., barata et al., 2017; rodrigues et al., 2016; sanchez-valdes & trivino, 2015). there is limited knowledge as to how critical the real-world context is for transferring results of laboratory studies to the field. this can be addressed in future research by putting a stronger emphasis on field experiments. further, so far only eight studies considered physiological measurements (e.g., see lim & reeves, 2010). however, it is known that human emotional responses involve unconscious processes (walla & panksepp, 2013). hence, there is a need for more physiological studies to uncover the unconscious effects of digital representations on users and their links to user perceptions. it is also necessary to note that our review has focused exclusively on experimental research. there is thus a need for future research to discern whether the results presented here are intrinsic to experimental studies. this can be achieved through a complementary review that focuses on other (non-experimental) research methods or through multi-method approaches that combine experimental and non-experimental approaches (e.g., focus groups, interviews). second, our review has identified several interesting knowledge gaps in terms of avatar appearance. so far, studies on self-avatars have exclusively focused on face-only or full-body avatars (see table 2). it would be interesting to investigate half-body avatars in the self-avatar context and contrast it to the results of face-only and full-body avatars. further, while research has emphasised the importance of high similarity between the user and a digital representation, the majority of studies on similarity considered just the avatar’s physical appearance (e.g., body size, clothing, hair) or gender. only three studies have explicitly considered the user’s ethnicity (fehrenbacher & weisner, 2017; spence et al., 2013; tamborini et al., 2018). digital representations in these three studies were based on, and hence represented, the ethnicity of (1) african-american and caucasian people (spence et al., 2013; tamborini et al., 2018), and (2) asian and caucasian people (fehrenbacher & weisner, 2017). it is known that visual appearances that are appropriate for one culture can be inappropriate in another (abdallah & douglas, 2010; aljaroodi, chiong, & adam, 2017; yusof & zakaria, 2007). therefore, further research is needed to explore the effects of similarity and other constructs (e.g., credibility, likeability) in the context of users’ culture and ethnicity. also, the scope should be broadened beyond asian, african-american, and caucasian cultures (e.g., arabian and indigenous cultures). third, the degree of overlap in the psychological constructs investigated by the different studies is quite low, even within the five interaction types (see figure 2). while presence and similarity emerge as the most widely-used constructs, other constructs such as attractiveness, enjoyment, and likeability have only been investigated by about 10% of the studies. also, there is no study that has explored the 11 constructs listed in table 1 simultaneously. there is thus australasian journal of information systems aljaroodi, adam, chiong & teubner 2019, vol 23, research article avatars and embodied agents in experimental is research 21 a lack of understanding of the interrelationships between them (e.g., credibility, likeability, enjoyment, and similarity), and how these relationships between constructs may be dependent on whether the user is interacting with an avatar or an embodied agent. 7.4 concluding remarks our study has provided a concise overview of the different digital representations investigated in experimental is research, including avatars and embodied agents. the degree of anthropomorphism represents a pivotal lever for designers of digital user representations. notably, digital representations can affect people in manifold ways, including the proteus and uncanny valley effects. we hope that researchers and practitioners will find this systematic review useful as a reference for experimental research on avatars and embodied agents to inform implementations of digital user representations in practice. references abdallah, s., & douglas, j. (2010). students’ first impression of second life: a case from the united arab emirates. turkish online journal of distance education, 11(3), 183–192. abs. (2015). academic journal guide 2015. association of business schools (abs). acphis. (2013). recommended is conferences. australian council of professors and heads of information systems (acphis). ahn, h., teeters, a., wang, a., breazeal, c., & picard, r. (2007). stoop to conquer: posture and affect interact to influence computer users’ persistence. in 2nd international conference on affective computing and intelligent interaction (pp. 582–593). heidelberg, germany. aljaroodi, h. m., adam, m. t. p., chiong, r., cornforth, d. j., & minichiello, m. (2017). empathic avatars in stroke rehabilitation: a co-designed mhealth artifact for stroke survivors. in desrist proceedings (pp. 73-89). karlsruhe, germany. aljaroodi, h. m., chiong, r., & adam, m. t. p. (2017). designing persuasive avatars in mhealth for arabic culture: a qualitative study. in acis proceedings (pp. 1–10). hobart, australia. allbeck, j. m., & badler, n. i. (2004). representing and parameterizing agent behaviors. in life-like characters (pp. 19–38). heidelberg, germany. ang, c. s., bobrowicz, a., siriaraya, p., trickey, j., & winspear, k. (2013). effects of gesturebased avatar-mediated communication on brainstorming and negotiation tasks among younger users. computers in human behavior, 29(3), 1204–1211. bacev-giles, c., & haji, r. (2017). online first impressions: person perception in social media profiles. computers in human behavior, 75, 50–57. bailenson, j. n., & blascovich, j. (2004). avatars. in encyclopedia of human–computer interaction (pp. 64–68.). massachusetts, united states: berkshire publishing group. bailenson, j. n., garland, p., iyengar, s., & yee, n. (2006). transformed facial similarity as a political cue: a preliminary investigation. political psychology, 27(3), 373–385. bailenson, j. n., swinth, k., hoyt, c., persky, s., dimov, a., & blascovich, j. (2005). the independent and interactive effects of embodied-agent appearance and behavior on selfreport, cognitive, and behavioral markers of copresence in immersive virtual environments. presence, 14(4), 379–393. australasian journal of information systems aljaroodi, adam, chiong & teubner 2019, vol 23, research article avatars and embodied agents in experimental is research 22 bailenson, j. n., & yee, n. (2005). digital chameleons: automatic assimilation of nonverbal gestures in immersive virtual environments. psychological science, 16(10), 814–819. bailey, r., wise, k., & bolls, p. (2009). how avatar customizability affects children’s arousal and subjective presence during junk food–sponsored online video games. cyberpsychology & behavior, 12(3), 277–283. balas, b., & pacella, j. (2017). trustworthiness perception is disrupted in artificial faces. computers in human behavior, 77, 240–248. barata, g., gama, s., jorge, j., & gonçalves, d. (2017). studying student differentiation in gamified education: a long-term study. computers in human behavior, 71, 550–585. beale, r., & creed, c. (2009). affective interaction: how emotional agents affect users. international journal of human-computer studies, 67(9), 755–776. beege, m., schneider, s., nebel, s., mittangk, j., & rey, g. d. (2017). ageism age coherence within learning material fosters learning. computers in human behavior, 75, 510–519. behm-morawitz, e. (2013). mirrored selves: the influence of self-presence in a virtual world on health, appearance, and well-being. computers in human behavior, 29(1), 119–128. bengtsson, b., burgoon, j. k., cederberg, c., bonito, j., & lundeberg, m. (1999). the impact of anthropomorphic interfaces on influence understanding, and credibility. in hicss proceedings (pp. 1–15). maui: united states. bickmore, t. w., & picard, r. w. (2005). establishing and maintaining long-term humancomputer relationships. acm transactions on computer-human interaction, 12(2), 293– 327. brave, s., nass, c., & hutchinson, k. (2005). computers that care: investigating the effects of orientation of emotion exhibited by an embodied computer agent. international journal of human-computer studies, 62(2), 161–178. burgoon, j. k., bonito, j. a., lowry, p. b., humpherys, s. l., moody, g. d., gaskin, j. e., & giboney, j. s. (2016). application of expectancy violations theory to communication with and judgments about embodied agents during a decision-making task. international journal of human-computer studies, 91, 24–36. burton-jones, a., recker, j., indulska, m., green, p., & weber, r. (2017). assessing representation theory with a framework for pursuing success and failure. mis quarterly, 41(4), 1307–1333. carlotto, t., & jaques, p. a. (2016). the effects of animated pedagogical agents in an englishas-a-foreign-language learning environment. international journal of human-computer studies, 95, 15–26. carter, m., allison, f., downs, j., & gibbs, m. (2015). player identity dissonance and voice interaction in games. in proceedings of the 2015 annual symposium on computer-human interaction in play (pp. 265–269). london, united kingdom. cobo, a., guerron, n. e., martín, c., del pozo, f., & serrano, j. j. (2017). differences between blind people’s cognitive maps after proximity and distant exploration of virtual environments. computers in human behavior, 77, 294–308. australasian journal of information systems aljaroodi, adam, chiong & teubner 2019, vol 23, research article avatars and embodied agents in experimental is research 23 davis, a., khazanchi, d., murphy, j., zigurs, i., & owens, d. (2009). avatars, people, and virtual worlds: foundations for research in metaverses. journal of the association for information systems, 10(2), 90–117. dechant, m., trimpl, s., wolff, c., mühlberger, a., & shiban, y. (2017). potential of virtual reality as a diagnostic tool for social anxiety: a pilot study. computers in human behavior, 76, 128–134. dickey, m. d. (2005). three-dimensional virtual worlds and distance learning: two case studies of active worlds as a medium for distance education. british journal of educational technology, 36(3), 439–451. dolgov, i., graves, w. j., nearents, m. r., schwark, j. d., & brooks volkman, c. (2014). effects of cooperative gaming and avatar customization on subsequent spontaneous helping behavior. computers in human behavior, 33, 49–55. ducheneaut, n., wen, m. h., yee, n., & wadley, g. (2009). body and mind: a study of avatar personalization in three virtual worlds. in the sigchi conference on human factors in computing systems (pp. 1151–1160). boston, united states. dunn, r. a., & guadagno, r. e. (2012). my avatar and me gender and personality predictors of avatar-self discrepancy. computers in human behavior, 28(1), 97–106. fabri, m., moore, d. j., & hobbs, d. j. (2005). empathy and enjoyment in instant messaging. in mckinnon, l., bertlesen, o., bryan-kinns, n. (eds.), proceedings of 19th british hci group annual conference (hci2005) (pp. 4–9). edinburgh, united kingdom fehrenbacher, d. d., & weisner, m. m. (2017). avatars and knowledge sharing. in icis proceedings (pp. 1–18). seoul, south korea. felnhofer, a., kafka, j. x., hlavacs, h., beutl, l., kryspin-exner, i., & kothgassner, o. d. (2018). meeting others virtually in a day-to-day setting: investigating social avoidance and prosocial behavior towards avatars and agents. computers in human behavior, 80, 399– 406. felnhofer, a., kothgassner, o. d., hauk, n., beutl, l., hlavacs, h., & kryspin-exner, i. (2014). physical and social presence in collaborative virtual environments: exploring age and gender differences with respect to empathy. computers in human behavior, 31(1), 272– 279. fox, j., & ahn, s. j. (2012). avatars: portraying, exploring, and changing online and offline identities. in r., luppicini (ed.), handbook of research on technoself: identity in a technological society (pp. 225–271). pennsylvania, united states: igi global. fox, j., ahn, s. j. (grace), janssen, j. h., yeykelis, l., segovia, k. y., & bailenson, j. n. (2015). avatars versus agents: a meta-analysis quantifying the effect of agency on social influence. human-computer interaction, 30(5), 401–432. fox, j., bailenson, j. n., & tricase, l. (2013). the embodiment of sexualized virtual selves: the proteus effect and experiences of self-objectification via avatars. computers in human behavior, 29(3), 930–938. franceschi, k., lee, r. m., zanakis, s. h., & hinds, d. (2009). engaging group e-learning in virtual worlds. journal of management information systems, 26(1), 73–100. australasian journal of information systems aljaroodi, adam, chiong & teubner 2019, vol 23, research article avatars and embodied agents in experimental is research 24 galanxhi, h., & nah, f. f. h. (2007). deception in cyberspace: a comparison of text-only vs. avatar-supported medium. international journal of human-computer studies, 65(9), 770– 783. garau, m., slater, m., vinayagamoorthy, v., brogni, a., steed, a., & sasse, m. a. (2003). the impact of avatar realism and eye gaze control on perceived quality of communication in a shared immersive virtual environment. in the sigchi conference on human factors in computing systems (pp. 529–536). fort lauderdale, united states. gerhard, m., moore, d., & hobbs, d. (2004). embodiment and copresence in collaborative interfaces. international journal of human-computer studies, 61(4), 453–480. goel, l., johnson, n., junglas, i., & ives, b. (2013). predicting users’ return to virtual worlds: a social perspective. information systems journal, 23(1), 35–63. gong, l. (2007). is happy better than sad even if they are both non-adaptive? effects of emotional expressions of talking-head interface agents. international journal of humancomputer studies, 65(3), 183–191. greenwood, d. n. (2008). television as escape from self: psychological predictors of media involvement. personality and individual differences, 44(2), 414–424. grinberg, a. m., careaga, j. s., mehl, m. r., & o’connor, m. f. (2014). social engagement and user immersion in a socially based virtual world. computers in human behavior, 36, 479– 486. groom, v., nass, c., chen, t., nielsen, a., scarborough, j. k., & robles, e. (2009). evaluating the effects of behavioral realism in embodied agents. international journal of humancomputer studies, 67(10), 842–849. grudin, j. (2001). desituating action: digital representation of context. human-computer interaction, 16(2), 269–286. guadagno, r. e., blascovich, j., bailenson, j. n., & mccall, c. (2007). virtual humans and persuasion: the effects of agency and behavioral realism. media psychology, 10(1), 1–22. guadagno, r. e., swinth, k. r., & blascovich, j. (2011). social evaluations of embodied agents and avatars. computers in human behavior, 27(6), 2380–2385. guegan, j., buisine, s., mantelet, f., maranzana, n., & segonds, f. (2016). avatar-mediated creativity: when embodying inventors makes engineers more creative. computers in human behavior, 61, 165–175. guegan, j., segonds, f., barre, j., maranzana, n., mantelet, f., & buisine, s. (2017). social identity cues to improve creativity and identification in face-to-face and virtual groups. computers in human behavior, 77, 140–147. hämäläinen, r., oksanen, k., & häkkinen, p. (2008). designing and analyzing collaboration in a scripted game for vocational education. computers in human behavior, 24(6), 2496– 2506. hammick, j. k., & lee, m. j. (2014). do shy people feel less communication apprehension online? the effects of virtual reality on the relationship between personality characteristics and communication outcomes. computers in human behavior, 33, 302–310. australasian journal of information systems aljaroodi, adam, chiong & teubner 2019, vol 23, research article avatars and embodied agents in experimental is research 25 hanus, m. d., & fox, j. (2015). persuasive avatars: the effects of customizing a virtual salespersons appearance on brand liking and purchase intentions. international journal of human-computer studies, 84, 33–40. heeter, c. (1992). being there: the subjective experience of presence. presence: teleoperators & virtual environments, 1(2), 262–271. heidig, s., & clarebout, g. (2011). do pedagogical agents make a difference to student motivation and learning? educational research review, 6(1), 27–54. hofer, m., hüsser, a., & prabhu, s. (2017). the effect of an avatar’s emotional expressions on players’ fear reactions: the mediating role of embodiment. computers in human behavior, 75, 883–890. hooi, r., & cho, h. (2013). deception in avatar-mediated virtual environment. computers in human behavior, 29(1), 276–284. huang, r., kahai, s., & jestice, r. (2010). the contingent effects of leadership on team collaboration in virtual teams. computers in human behavior, 26(5), 1098–1110. hyde, j., carter, e. j., kiesler, s., & hodgins, j. k. (2015). using an interactive avatar’s facial expressiveness to increase persuasiveness and socialness. in the 33rd annual acm conference on human factors in computing systems (pp. 1719–1728). seoul, south korea. ifip. (2015). ifip tc8 ranking of is conferences. international federation for information processing: technical committee 8 (ifip: tc8). jin, s.-a. a. (2010). the effects of incorporating a virtual agent in a computer-aided test designed for stress management education: the mediating role of enjoyment. computers in human behavior, 26(3), 443–451. jin, s.-a. a. (2012). the virtual malleable self and the virtual identity discrepancy model: investigative frameworks for virtual possible selves and others in avatar-based identity construction and social interaction. computers in human behavior, 28(6), 2160–2168. kang, s. h., & gratch, j. (2014). exploring users’ social responses to computer counseling interviewers’ behavior. computers in human behavior, 34, 120–130. kang, s. h., & watt, j. h. (2013). the impact of avatar realism and anonymity on effective communication via mobile devices. computers in human behavior, 29(3), 1169–1181. kaye, l. k., pennington, c. r., & mccann, j. j. (2018). do casual gaming environments evoke stereotype threat? examining the effects of explicit priming and avatar gender. computers in human behavior, 78, 142–150. keng, c. j., & liu, c. c. (2013). can avatar and self-referencing really increase the effects of online 2-d and 3-d advertising? computers in human behavior, 29(3), 791–802. khashe, s., lucas, g., becerik-gerber, b., & gratch, j. (2017). buildings with persona: towards effective building-occupant communication. computers in human behavior, 75, 607–618. kim, h.-k., & kim, s.-h. (2016). understanding emotional bond between the creator and the avatar: change in behavioral intentions to engage in alcohol-related traffic risk behaviors. computers in human behavior, 62, 186–200. australasian journal of information systems aljaroodi, adam, chiong & teubner 2019, vol 23, research article avatars and embodied agents in experimental is research 26 kim, h. e., hong, y., kim, m., jung, y. h., kyeong, s., & kim, j.-j. (2017). effectiveness of selftraining using the mobile-based virtual reality program in patients with social anxiety disorder. computers in human behavior, 73, 614–619. kim, j. (2009). “i want to be different from others in cyberspace” the role of visual similarity in virtual group identity. computers in human behavior, 25(1), 88–95. kim, j. (2011). two routes leading to conformity intention in computer-mediated groups: matching versus mismatching virtual representations. journal of computer-mediated communication, 16(2), 271–287. kim, j., & park, h. s. (2011). the effect of uniform virtual appearance on conformity intention: social identity model of deindividuation effects and optimal distinctiveness theory. computers in human behavior, 27(3), 1223–1230. kim, y., & sundar, s. s. (2012a). anthropomorphism of computers: is it mindful or mindless? computers in human behavior, 28(1), 241–250. kim, y., & sundar, s. s. (2012b). visualizing ideal self vs. actual self through avatars: impact on preventive health outcomes. computers in human behavior, 28(4), 1356–1364. kitchenham, b., & charters, s. (2007). guidelines for performing systematic literature reviews in software engineering. technical report ebse. kohonen-aho, l., & tiilikainen, s. (2017). constructing shared context for temporary teams in virtual worlds with informal interaction. in icis proceedings (pp. 1–19). seoul, south korea. koles, b., & nagy, p. (2012). virtual customers behind avatars: the relationship between virtual identity and virtual consumption in second life. journal of theoretical and applied electronic commerce research, 7(2), 87–105. kothgassner, o. d., griesinger, m., kettner, k., wayan, k., volkl-kernstock, s., hlavacs, h., beutl, l., felnhofer, a. (2017). real-life prosocial behavior decreases after being socially excluded by avatars, not agents. computers in human behavior, 70, 261–269. krämer, n., kopp, s., becker-asano, c., & sommer, n. (2013). smile and the world will smile with you the effects of a virtual agent’s smile on users’ evaluation and behavior. international journal of human-computer studies, 71(3), 335–349. kwon, j. h., powell, j., & chalmers, a. (2013). how level of realism influences anxiety in virtual reality environments for a job interview. international journal of human-computer studies, 71(10), 978–987. leding, j. k., horton, j. c., & wootan, s. s. (2015). the contrast effect with avatars. computers in human behavior, 44, 118–123. lee, c., rincon, g. a., meyer, g., hollerer, t., & bowman, d. (2013). the effects of visual realism on search tasks in mixed reality simulation. ieee transactions on visualization and computer graphics, 19(4), 547–556. lee, j.-e. r. (2014). does virtual diversity matter?: effects of avatar-based diversity representation on willingness to express offline racial identity and avatar customization. computers in human behavior, 36, 190–197. australasian journal of information systems aljaroodi, adam, chiong & teubner 2019, vol 23, research article avatars and embodied agents in experimental is research 27 lee, s., & choi, j. (2017). enhancing user experience with conversational agent for movie recommendation: effects of self-disclosure and reciprocity. international journal of human-computer studies, 103, 95–105. lee, y., kozar, k. a., & larsen, k. r. (2009). avatar e-mail versus traditional e-mail: perceptual difference and media selection difference. decision support systems, 46(2), 451–467. lehto, t., & oinas-kukkonen, h. (2011). persuasive features in web-based alcohol and smoking interventions: a systematic review of the literature. journal of medical internet research, 13(3), e46. lester, p. m., & king, c. m. (2009). analog vs. digital instruction and learning: teaching within first and second life environments. journal of computer-mediated communication, 14(3), 457–483. li, b. j., & lwin, m. o. (2016). player see, player do: testing an exergame motivation model based on the influence of the self avatar. computers in human behavior, 59, 350–357. lim, s., & reeves, b. (2010). computer agents versus avatars: responses to interactive game characters controlled by a computer or other player. international journal of humancomputer studies, 68(1), 57–68. macdorman, k. f., green, r. d., ho, c.-c., & koch, c. t. (2009). too real for comfort? uncanny responses to computer generated faces. computers in human behavior, 25(3), 695–710. macdorman, k. f., & ishiguro, h. (2006). the uncanny advantage of using androids in social and cognitive science research. interaction studies, 7(3), 297–337. maldonado, h., lee, j. r., brave, s., nass, c., nakajima, h., yamada, r., … morishima, y. (2005). we learn better together: enhancing elearning with emotional characters. in koschmann, t., suthers, d., chan, t.w. (eds.), proceedings of th 2005 conference on computer support for collaborative learning: learning 2005: the next 10 years! (pp. 408–417). mallan, k. m. (2009). look at me! look at me! self-representation and self-exposure through online networks. digital culture and education, 1(1), 51–66. mara, m., & appel, m. (2015). science fiction reduces the eeriness of android robots: a field experiment. computers in human behavior, 48, 156–162. martens, a. l., grover, c. a., saucier, d. a., & morrison, b. a. (2018). an examination of gender differences versus similarities in a virtual world. computers in human behavior, 84, 404–409. mccreery, m. p., krach, s. k., schrader, p. g., & boone, r. (2012). defining the virtual self: personality, behavior, and the psychology of embodiment. computers in human behavior, 28(3), 976–983. mccroskey, j., richmond, v., & daly, j. (1975). the development of a measure of perceived homophily in international communication. human communication research, 1(4), 323– 332. midha, v., & nandedkar, a. (2012). impact of similarity between avatar and their users on their perceived identifiability: evidence from virtual teams in second life platform. computers in human behavior, 28(3), 929–932. australasian journal of information systems aljaroodi, adam, chiong & teubner 2019, vol 23, research article avatars and embodied agents in experimental is research 28 mitchell, w. j., szerszen, k. a., lu, a. s., schermerhorn, p. w., scheutz, m., & macdorman, k. f. (2011). a mismatch in the human realism of face and voice produces an uncanny valley. i-perception, 2(1), 10–12. mori, m. (1970). bukimi no tani (the uncanny valley). energy, 7(4), 33–35. mousas, c., anastasiou, d., & spantidi, o. (2018). the effects of appearance and motion of virtual characters on emotional reactivity. computers in human behavior, 86, 99–108. nass, c., & moon, y. (2000). machines and mindlessness: social responses to computers. journal of social issues, 56(1), 81–103. nass, c., steuer, j., & tauber, e. r. (1994). computers are social actors. in proceedings of the conference on human factors in computing systems (pp. 72–78). massachusetts, united states. noorbergen, t. j., adam, m. t., attia, j. r., cornforth, d. j., & minichiello, m. (2019). exploring the design of mhealth systems for health behavior change using mobile biosensors. communications of the association for information systems, 44, 944-981. nowak, k. l., & biocca, f. (2003). the effect of the agency and anthropomorphism on users’ sense of telepresence, copresence, and social presence in virtual environments. presence: teleoperators and virtual environments, 12(5), 481–494. nowak, k. l., fox, j., & ranjit, y. s. (2015). inferences about avatars: sexism, appropriateness, anthropomorphism, and the objectification of female virtual representations. journal of computer-mediated communication, 20(5), 554–569. nowak, k. l., & rauh, c. (2006). the influence of the avatar on online perceptions of anthropomorphism, androgyny, credibility, homophily, and attraction. journal of computer-mediated communication, 11(2), 153–178. nowak, k. l., & rauh, c. (2008). choose your “buddy icon” carefully: the influence of avatar androgyny, anthropomorphism and credibility in online interactions. computers in human behavior, 24(4), 1473–1493. nunamaker, j. f., derrick, d. c., elkins, a. c., burgoon, j. k., & patton, m. w. (2011). embodied conversational agent-based kiosk for automated interviewing. journal of management information systems, 28(1), 17–48. parise, s., kiesler, s., sproull, l., & waters, k. (1999). cooperating with life-like interface agents. computers in human behavior, 15(2), 123–142. peña, j., & chen, m. (2017). with great power comes great responsibility: superhero primes and expansive poses influence prosocial behavior after a motion-controlled game task. computers in human behavior, 76, 378–385. peña, j., hancock, j. t., & merola, n. a. (2009). thee priming effects of avatars in virtual settings. communication research, 36, 838–856. peña, j., khan, s., & alexopoulos, c. (2016). i am what i see: how avatar and opponent agent body size affects physical activity among men playing exergames. journal of computermediated communication, 21, 195–209. peña, j., & kim, e. (2014). increasing exergame physical activity through self and opponent avatar appearance. computers in human behavior, 41, 262–267. australasian journal of information systems aljaroodi, adam, chiong & teubner 2019, vol 23, research article avatars and embodied agents in experimental is research 29 pessoa, t. m., coutinho, d. s., pereira, v. m., de oliveira ribeiro, n. p., nardi, a. e., & de oliveira e silva, a. c. (2014). the nintendo wii as a tool for neurocognitive rehabilitation, training and health promotion. computers in human behavior, 31(1), 384– 392. qiu, l., & benbasat, i. (2009). evaluating anthropomorphic product recommendation agents: a social relationship perspective to designing information systems. journal of management information systems, 25(4), 145–182. qiu, l., & benbasat, i. (2010). a study of demographic embodiments of product recommendation agents in electronic commerce. international journal of human-computer studies, 68(10), 669–688. ratan, r. a., & dawson, m. (2016). when mii is me: a psychophysiological examination of avatar self-relevance. communication research, 43(8), 1065–1093. ratan, r. a., & sah, y. j. (2015). leveling up on stereotype threat: the role of avatar customization and avatar embodiment. computers in human behavior, 50, 367–374. ravaja, n., saari, t., salminen, m., laarni, j., & kallinen, k. (2006). phasic emotional reactions to video game events: a psychophysiological investigation. media psychology, 8(4), 343– 367. recker, j., indulska, m., green, p. f., burton-jones, a., & weber, r. (2019). information aystems as representations : a review of the theory and rvidence. journal of the association for information systems, 20(6), 735–786. riedl, r., mohr, p. n. c., kenning, p. h., davis, f. d., & heekeren, h. r. (2011). trusting humans and avatars: behavioral and neural evidence. in icis proceedings. shanghai, china. riedl, r., mohr, p. n. c., kenning, p. h., davis, f. d., & heekeren, h. r. (2014). trusting humans and avatars: a brain imaging study based on evolution theory. journal of management information systems, 30(4), 83–114. rodrigues, l. f., oliveira, a., & costa, c. j. (2016). playing seriously how gamification and social cues influence bank customers to use gamified e-business applications. computers in human behavior, 63, 392–407. sanchez-valdes, d., & trivino, g. (2015). linguistic and emotional feedback for self-tracking physical activity. expert systems with applications, 42(24), 9574–9586. saygin, a. p. (2011). the thing that should not be: predictive coding and the uncanny valley in perceiving human and humanoid robot actions. social cognitive affective neuroscience, 7(4), 413–422. schneider, f. m., zwillich, b., bindl, m. j., hopp, f. r., reich, s., & vorderer, p. (2017). social media ostracism: the effects of being excluded online. computers in human behavior, 73, 385–393. schultze, u. (2011). the avatar as sociomaterial entanglement: a performative perspective on identity, agency and world-making in virtual worlds. in icis proceedings. shanghai, china. australasian journal of information systems aljaroodi, adam, chiong & teubner 2019, vol 23, research article avatars and embodied agents in experimental is research 30 sebastian, j., & richards, d. (2017). changing stigmatizing attitudes to mental health via education and contact with embodied conversational agents. computers in human behavior, 73, 479–488. seo, y., kim, m., jung, y., & lee, d. (2017). avatar face recognition and self-presence. computers in human behavior, 69, 120–127. shadish, w. r., cook, t. d., & campbell, d. t. (2002). experimental and quasi-experimental designs for generalized causal inference. boston, ma: houghton mifflin. shedlosky-shoemaker, r., costabile, k. a., & arkin, r. m. (2014). self-expansion through fictional characters. self and identity, 13(5), 556–578. slater, m., khanna, p., mortensen, j., & yu, i. (2009). visual realism enhances realistic response in an immersive virtual. ieee computer graphics and applications, 29(3), 76–84. song, h., kim, j., kwon, r. j., & jung, y. (2013). anti-smoking educational game using avatars as visualized possible selves. computers in human behavior, 29(5), 2029–2036. song, h., kim, j., tenzek, k. e., & lee, k. m. (2013). the effects of competition and competitiveness upon intrinsic motivation in exergames. computers in human behavior, 29(4), 1702–1708. spence, p. r., lachlan, k. a., spates, s. a., & lin, x. (2013). intercultural differences in responses to health messages on social media from spokespeople with varying levels of ethnic identity. computers in human behavior, 29(3), 1255–1259. steuer, j. (1992). defining virtual reality: dimensions determining telepresence. journal of communication, 42(4), 73–93. strauss, p. s. (1990). a realistic lighting model for computer animators. ieee computer graphics and applications, 10(6), 56–64. suh, k., kim, h., & suh, e. k. (2011). what if your avatar looks like you? dual-congruity perspectives for avatar use. mis quarterly, 35(3), 711–729. sutcliffe, a., & alrayes, a. (2012). investigating user experience in second life for collaborative learning. international journal of human-computer studies, 70(7), 508–525. sylvia, z., king, t. k., & morse, b. j. (2014). virtual ideals: the effect of video game play on male body image. computers in human behavior, 37, 183–188. tamborini, r., novotny, e., prabhu, s., hofer, m., grall, c., klebig, b., hahn, l. s., slaker, j., ratan, r. a., bente, g. (2018). the effect of behavioral synchrony with black or white virtual agents on outgroup trust. computers in human behavior, 83, 176–183. tamborini, r., & skalski, p. (2006). the role of presence in the experience of electronic games. in p. vorderer & j. bryant (eds.), playing video games: motivations, responses, and consequences (pp. 225–240). mahwah, united states: lawrence earlbaum associates. taylor, t. l. (2002). living digitally: embodiment in virtual worlds. in from r. schroeder (ed.)the social life of avatars: presence and interaction in shared virtual environments (pp. 40–62). london: springer-verlag. teng, c.-i. (2010). customization, immersion satisfaction, and online gamer loyalty. computers in human behavior, 26(6), 1547–1554. australasian journal of information systems aljaroodi, adam, chiong & teubner 2019, vol 23, research article avatars and embodied agents in experimental is research 31 teubner, t., adam, m. t. p., camacho, s., hassanein, k. (2014). understanding resource sharing in c2c platforms: the role of picture humanization. in acis proceedings (pp. 110). auckland, new zealand. thomson, m., macinnis, d. j., & park, w. (2005). the ties that bind: measuring the strength of consumers’ emotional attachments to brands. journal of consumer psychology, 15(1), 77– 91. tinwell, a., grimshaw, m., abdel nabi, d., & williams, a. (2011). facial expression of emotion and perception of the uncanny valley in virtual characters. computers in human behavior, 27(2), 741–749. vasalou, a., joinson, a., bänziger, t., goldie, p., & pitt, j. (2008). avatars in social media: balancing accuracy, playfulness and embodied messages. international journal of humancomputer studies, 66(11), 801–811. vasalou, a., & joinson, a. n. (2009). me, myself and i: the role of interactional context on selfpresentation through avatars. computers in human behavior, 25(2), 510–520. von der pütten, a. m., krämer, n. c., gratch, j., & kang, s. h. (2010). “it doesn’t matter what you are!” explaining social effects of agents and avatars. computers in human behavior, 26(6), 1641–1650. walla, p., & panksepp, j. (2013). neuroimaging helps to clarify brain affective processing without necessarily clarifying emotions. in fountas k. n., ed., novel frontiers of advanced neuroimaging (pp. 93–118). intechopen. webster, j., & wong, w. k. p. (2008). comparing traditional and virtual group forms: identity, communication and trust in naturally occurring project teams. the international journal of human resource management, 19(1), 41–62. westerman, d., tamborini, r., & bowman, n. d. (2015). the effects of static avatars on impression formation across different contexts on social networking sites. computers in human behavior, 53, 111–117. won, a. s., bailenson, j., lee, j., & lanier, j. (2015). homuncular flexibility in virtual reality. journal of computer-mediated communication, 20(3), 241–259. wrzesien, m., rodríguez, a., rey, b., alcañiz, m., baños, r. m., & vara, m. d. (2015). how the physical similarity of avatars can influence the learning of emotion regulation strategies in teenagers. computers in human behavior, 43, 101–111. wu, j., mattingly, e., & kraemer, p. (2015). communication in virtual environments: the influence of spatial cues and gender on verbal behavior. computers in human behavior, 52, 59–64. yee, n., & bailenson, j. n. (2007). the proteus effect: the effect of transformed selfrepresentation on behavior. human communication research, 33(3), 271–290. yee, n., bailenson, j. n., & ducheneaut, n. (2009). the proteus effect: implications of transformed digital self-representation on online and offline behavior. communication research, 36(2), 285–312. yusof, s. a. m., & zakaria, n. (2007). islamic perspective: virtual worlds as a western-centric technology. sigmis database, 38(4), 100–103. australasian journal of information systems aljaroodi, adam, chiong & teubner 2019, vol 23, research article avatars and embodied agents in experimental is research 32 appendix a: literature summary table table a1. experimental studies on avatars and embodied agents in is literature. authors (year) [outlet] brief description avatar / embodied agent design [interaction] application [context] psychological construct(s) #p (env) gerhard et al. (2004) [ijhcs] explores hybrid avatar/agent model to achieve permanent users’ presence and increase user experience full body, motion, 3d [sa2a/sa2ea] collaboration [business] awareness, immersion, involvement, presence 20 (lab) brave et al. (2005) [ijhcs] effects of an embodied agent’s emotions upon users face only, static, 2d [h2ea] blackjack game [entertainment] emotionality, likeability, trust 96 (lab) nowak & rauh (2006) [jcmc] effects of avatars physical appearance on users’ perceptions face only, static, 2d [h2a] interaction [social] androgyny, anthropomorphism, attractiveness, credibility, intention to use, similarity 255 (lab) galanxhi & nah (2007) [ijhcs] effects of avatars on deceptions vs. truth on users in online chats face only, static, 2d [sa2a] interaction [social] deceptiveness, intention to use, trust, user anxiety 94 (lab) gong (2007) [ijhcs] effects of a happy talking-head ea on users’ trustworthiness, and competence of the ea face only, motion, 2d [h2ea] book review [entertainment] competence, emotionality, intention to use, likeability, trust 24 (lab) hämäläinen et al. (2008) [chb] effects of avatars on group learning activity performance in collaborative environments full body, motion, 3d [sa2a] collaboration [education] knowledgeability 64 (lab) nowak & rauh (2008) [chb] effects of a user’s avatar on other’s perceptions of the avatar face only, static, 2d [h2a] interaction [social] androgyny, anthropomorphism, credibility 255 (srv) 230 (lab) vasalou et al. (2008) [ijhcs] how users present themselves via avatar in social media (actual self and ideal self) full body, motion, 3d [h2sa] social media [social] emotional attachment, self-representativeness 20 (lab) franceschi et al. (2009) [jmis] compares presence in text-based virtual environment (blackboard), virtual world (second life), classroom full body, motion, 2d/3d [sa2a] elearning [education] group presence, presence, user engagement 24 n. a. 24 (fld) 24 n. a. groom et al. (2009) [ijhcs] effects of ea’s behavioural realism on people’s responses full body, motion, 3d [h2ea] survival game [entertainment] likeability, persuasiveness, behavioural realism, user comfort 80 (lab) j. kim (2009) [chb] effects of avatars’ visual similarity on group identity and conformity face only, static, 2d [sa2a] group identity [social] similarity 164 (lab) y. lee et al. (2009) [dss] explores how users perceive avatar email differently from traditional email full and half body, static, 2d [h2a] communication [business] intention to use, presence 742 (srv) 505 (srv) 452 (lab) lester & king (2009) [jcmc] the study compares student performance between traditional classroom and virtual classroom (blackboard and second life) full body, motion, 2d/3d [sa2a] classroom [education] knowledgeability 161(fld) 173 (fld) qiu & benbasat (2009) [jmis] effects of an embodied agent on users’ social presence, trust, and enjoyment half body, static, 2d [h2ea] ecommerce [business] enjoyment, intention to use, presence, trust, usefulness 168 (lab) vasalou & joinson (2009) [chb] explores the different self-presentation designs via avatar for blogging, dating, & gaming full body, static, 3d [h2sa] blogging, dating, gaming [social] attractiveness, selfawareness, selfrepresentativeness, similarity 71 (lab) australasian journal of information systems aljaroodi, adam, chiong & teubner 2019, vol 23, research article avatars and embodied agents in experimental is research 33 authors (year) [outlet] brief description avatar / embodied agent design [interaction] application [context] psychological construct(s) #p (env) huang et al. (2010) [chb] effects of avatars on team’s performance interaction in a virtual meeting full body, motion, 3d [sa2a] virtual meeting [business] cooperativeness, satisfaction 485 (lab) jin (2010) [chb] effects of a virtual agent on enjoyment and evaluation of computer-aided education full body, motion, 3d [sa2ea] stress management [health] educational value, enjoyment, selfefficacy 60 (lab) qiu & benbasat (2010) [ijhcs] effects of an embodied agent on users’ social experience half body, static, 2d [h2ea] ecommerce [business] enjoyment, intention to use, presence, usefulness 188 (lab) von der pütten et al. (2010) [chb] evaluates whether interacting with either an avatar or an agent lead to higher level of attention full body, motion, 3d [h2ea/h2a] social behaviour [social] behavioural realism, presence, rapport 83 (lab) guadagno et al. (2011) [chb] effects of empathy on people when interacting with virtual human representation full body, motion, 3d [sa2a/sa2ea] interaction [social] emotionality, enjoyment, likeability, presence, satisfaction, trust, user comfort, supportiveness 38 (lab) j. kim (2011) [jcmc] explores how group visual similarity strengthen the group identification face only, static, 2d [h2a] discussion [business] similarity 258 (lab) j. kim & park (2011) [chb] effects of an avatar’s uniform appearance on users’ willingness to agree to a majority opinion in virtual groups face only, static, 2d [sa2a] discussion [social] similarity 345 (lab) nunamaker et al. (2011) [jmis] effects of an automated interviewing agent on arousal, behaviour, and cognitive effort face only, motion, 3d [h2ea] interview [business] knowledgeability, likeability, trust 88 (lab) 81 (lab) 29 (fld) riedl et al. (2011) [icis] investigates trust in terms of participants playing against humans or against embodied agent face only, static, 2d [h2h/h2ea] trust game [business] trust 18 (lab) schultze (2011) [icis] explores how virtual worlds users construct agency (possible and ideal self), identity, and worlds within inworlds experience full body, motion, 3d [h2sa] second life [entertainment] agency 35 (lab) suh et al. (2011) [misq] explores effects of avatar similarity to actual self on users’ attitudes toward the use of avatar and identification full body, motion, 2d/3d [h2sa] virtual store [business] intention to use, similarity 92 (lab) dunn & guadagno (2012) [chb] effects of users’ gender and personality type on avatar representation full body, motion, 3d [sa2a] video game [entertainment] gender, personality type, self-esteem, selfrepresentativeness 174 (lab) jin (2012) [chb] effects on avatars on user’s selfdisclosure interpersonal communication full body, motion, 3d [sa2a] interaction [social] presence, trust 117 (lab) 148 (lab) y. kim & sundar (2012a) [chb] effects of computer anthropomorphism on presence and credibility full body, motion, 2d [h2ea] health information [health] anthropomorphism, credibility, presence 93 (lab) y. kim & sundar (2012b) [chb] effects of realistic self-avatar on risktaking behaviour full body, motion, 3d [sa2ea] health information [health] attractiveness, visual realism 95 (lab) mccreery et al. (2012) [chb] effects of the big five personality dimensions on users’ helping behaviour in virtual environment full body, motion, 3d [sa2a] video game [entertainment] personality type, selfinterest, selfrepresentativeness 39 (lab) sutcliffe & alrayes (2012) [ijhcs] the study compares students’ performance in blackboard and second life in terms of usability and user experience full body, motion, 2d/3d [sa2a] collaboration [education] emotionality, presence, user collaboration 38 (lab) 63 (lab) ang et al. (2013) [chb] effects of avatar body gestures on performance and satisfaction in school tasks full body, motion, 3d [sa2a] school task [education] intelligence, likeability, satisfaction, trust 64 (lab) australasian journal of information systems aljaroodi, adam, chiong & teubner 2019, vol 23, research article avatars and embodied agents in experimental is research 34 authors (year) [outlet] brief description avatar / embodied agent design [interaction] application [context] psychological construct(s) #p (env) fox et al. (2013) [chb] influences of choosing avatar that is sexualised or non-sexualised on user’s self-objectification full body, motion, 3d [h2sa/sa2a] interaction [social] intention to use, presence 86 (lab) goel et al. (2013) [isj] investigates what influences users to return to a virtual world full body, motion, 3d [sa2a] interaction [social] immersion, intention to return, presence, social awareness 175 (lab) kang & watt (2013) [chb] effects of avatar realism on communication socially face only, motion, 2d [h2a] communication [social] anthropomorphism, presence, realism (visual, behavioural), satisfaction 198 (lab) keng & liu (2013) [chb] effects of an avatar’s attractiveness on online advertisement face only, static, 2d [h2a] advertising [business] attractiveness, purchase intention 1514 (lab) 687 (lab) krämer et al. (2013) [ijhcs] effects of an agent’s smile on participants’ evolution and smiling behaviour half body, motion, 3d [h2ea] conversation (smiling) [social] anthropomorphism, emotionality (smile) 104 (lab) kwon et al. (2013) [ijhcs] effects of various virtual environment conditions upon users’ level of anxiety half body, motion, 2d/3d [h2ea] job interview [business] visual realism, user anxiety 60 (lab) 20 (lab) song, kim, kwon, et al. (2013) [chb] effects of smoking avatar representations on users’ perceived risk of smoking full body, motion, 3d [h2sa] game [education] identification, intention to quit 62 (lab) song, kim, tenzek, et al. (2013) [chb] effects of an avatar on competing in exercise video games on users’ playing time full body, motion, 3d [sa2a] exergame [entertainment] competitiveness, enjoyment, mood, selfefficacy 72 (lab) spence et al. (2013) [chb] effects of the avatar ethnicity and similarity on health-related information face/ shoulder only, static, 2d [h2a] communication [health] behavioural intentions, similarity (ethnicity) 207 (lab) dolgov et al. (2014) [chb] effects of avatar customisation on helping behaviour full body, motion, 3d [sa2a] video game [entertainment] identification, similarity 54 (lab) 25 (lab) felnhofer et al. (2014) [chb] effects of users’ gender and age on presence, empathy, and collaboration full body, motion, 3d [sa2a] café [social] empathy, gender, presence 124 (lab) grinberg et al. (2014) [chb] effects of avatar on social engagement and immersion in the virtual environment full body, motion, 3d [sa2a] interaction [social] immersion, presence, user engagement 35 (lab) hammick & lee (2014) [chb] effects of an avatar on shy users to express themselves freely full body, motion, 3d [sa2a] communication [social] interactivity, presence 58 (lab) kang & gratch (2014) [chb] effects of embodied agents’ behavioural realism on users’ selfdisclosure half body, motion, 3d [h2ea] counselling interviews [social] behavioural realism, emotional state, presence, likeability 171 (lab) j.-e. r. lee (2014) [chb] effects of avatar’ diversity on offline racial identity half body, motion, 3d [h2a] social identity [social] user willing to reveal racial identity 56 (lab) peña & kim (2014) [chb] effects of avatars’ size on physical activity in exergame using accelerometers full body, motion, 3d [sa2a] video game [entertainment] na 96 (lab) riedl et al. (2014) [jmis] investigates trust in terms of participants playing against humans or against embodied agent to make decision and learn face only, static, 2d [h2h/h2ea] trust game [business] trust 18 (lab) sylvia et al. (2014) [chb] effects of playing video games with an unrealistic male body on actual body satisfaction full body, motion, 3d [h2sa] video game [entertainment] satisfaction, body and self-esteem 50 (lab) hanus & fox (2015) [ijhcs] effect of customising an avatar on intrinsic motivation, brand liking and purchase intention full body, motion, 3d [sa2a] sale [business] agency, likeability, motivation, purchase intention 80 (lab) leding et al. (2015) [chb] effects of an attractive avatar on the perception of viewing actual human full body, static, 3d [h2a] attractiveness [social] attractiveness 120 (lab) 234 (lab) australasian journal of information systems aljaroodi, adam, chiong & teubner 2019, vol 23, research article avatars and embodied agents in experimental is research 35 authors (year) [outlet] brief description avatar / embodied agent design [interaction] application [context] psychological construct(s) #p (env) nowak et al. (2015) [jcmc] effects of an avatar’s clothing and anthropomorphism on trust and message clarity full body, static, 2d [h2a] dating or job interview [business] anthropomorphism, appropriateness, attractiveness, credibility, trust 397 (lab) ratan & sah (2015) [chb] effects of an avatar’s identity characteristics on avatar users’ behaviours full body, motion, 3d [sa2a] video game [entertainment] engagement, motivation, selfrepresentation, presence, selfrelevance 64 (lab) sanchez-valdes & trivino (2015) [eswa] effects of an avatar emotion on users’ physical activities face only, static, 2d [h2sa] physical activity [health] emotionality, motivation 3 (fld) westerman et al. (2015) [chb] effects of different types of avatars created for social or dating/ romantic situations on users half body, static, 2d [h2a] networking [social] attractiveness, intention to use, similarity 206 (lab) won et al. (2015) [jcmc] effects of an avatar’s movement on user’s movements on the real world full body, motion, 3d [h2sa] communication [entertainment] motivation, presence 53 (lab) 20 (lab) wrzesien et al. (2015) [chb] influence of the physical similarity of avatars on user’s emotion regulation training full body, motion, 3d [h2a/h2sa] emotion regulation [education] similarity 22 (lab) wu et al. (2015) [chb] the study explores the effects on verbal behaviour in virtual interaction full body, motion, 3d [sa2a] communication [social] behavioural intention, presence 32 (lab) burgoon et al. (2016) [ijhcs] effects of an embodied agent on the outcomes of decision-making tasks face only, motion, 3d [h2h/h2ea] communication [business] attractiveness, expectedness, dependability, receptiveness 70 (lab) carlotto & jaques (2016) [ijhcs] effects of an embodied agent on users’ in a learning environment full body, motion, 2d [h2ea] learning [education] expressiveness, motivation, relevance, satisfaction, willingness to repeat 72 (lab) guegan et al. (2016) [chb] effects of avatars on users’ creativity performance in collaborative meeting full body, motion, 3d [sa2a] meeting [business] similarity 114 (lab) 54 (lab) h.-k. kim & kim (2016) [chb] explores the emotional bonds between the individual and their avatar full body, motion, 3d [sa2a] driving [social] attractiveness, emotional closeness, similarity 111 (lab) li & lwin (2016) [chb] effects of self-avatars on user’s presence and enjoyment full body, motion, 3d [h2a] video game [entertainment] enjoyment, identification, presence 322 (lab) peña et al. (2016) [jcmc] influences of avatar’s body size on physical activity using accelerometers full body, motion, 3d [sa2ea] exergame [entertainment] na 96 (lab) rodrigues et al. (2016) [chb] effects of avatars use in business application on customers half body, static, 3d [h2a] gamification ebanking app [business] ease of use, enjoyment, usefulness, intention to use, presence 183 (fld) bacev-giles & haji (2017) [chb] examines impressions of avatar online profiles for gender stereotype impressions full body, static, 3d [h2a] interaction [social] gender, likeability, similarity 239 (lab) beege et al. (2017) [chb] investigates the influence of stereotypes on learning, cognitive load and motivation as well as time spent with the embodied agent full body, motion, 3d [h2ea] health material [health] gender, similarity 84 (lab) balas & pacella (2017) [chb] examines the perceived trustworthiness of avatars' and humans' faces face only, static, 2d [h2a] interaction [social] sensitivity, trust 51 (lab) 20 (lab) barata et al. (2017) [chb] examines students' performance in gamified collage course full body, motion, 3d [h2sa] learning [education] ease of use, continuity, engagement 35 n.a. 52 n.a. 54 (fld) cobo et al. (2017) [chb] examines distant exploration of places and time spent with avatar for blind people using smart phone full body, motion, 3d [h2sa] interaction [social] na 19 (lab) australasian journal of information systems aljaroodi, adam, chiong & teubner 2019, vol 23, research article avatars and embodied agents in experimental is research 36 authors (year) [outlet] brief description avatar / embodied agent design [interaction] application [context] psychological construct(s) #p (env) dechant et al. (2017) [chb] examines virtual reality to diagnose social anxiety full body, motion, 3d [sa2ea] interaction [social] user anxiety 119 (lab) fehrenbacher & weisner (2017) [icis] examines knowledge sharing while interacting with real human images vs avatars face only, static, 3d [sa2a] communication [social] similarity (ethnicity), trust 97 (lab) guegan et al. (2017) [chb] examines the effects of visual cues of social identity on group performance full body, motion, 3d [sa2a] brainstorming [business] identification, presence (social cues) 72 (lab) hofer et al. (2017) [chb] examines the effects on avatars' expression on users fear reactions. full body, motion, 3d [h2sa/sa2a] video game [entertainment] agency, physical appearance (avatar embodiment) 103 (lab) 104 (lab) khashe et al. (2017) [chb] investigates influence of avatars' communication style and gender on user compliance half body, motion, 3d [h2a] communication [business] communication style, gender 214 (lab) h. e. kim et al. (2017) [chb] examines the effects of mobile-based self-training performance for social anxiety full body, motion, 3d [h2sa] self-training [health] avoidance 22 (lab) kohonen-aho & tiilikainen (2017) [icis] examines the effects of virtual world in virtual team performance of informal interaction full body, motion, 3d [sa2a] virtual team [business] presence 36 (lab) kothgassner et al. (2017) [chb] examines a virtual social exclusion scenario and tests for behavioural consequences full body, motion, 3d [sa2a/sa2ea] interaction [social] emotional state, presence, self-esteem 45 (lab) s. lee & choi (2017) [ijhcs] investigates embodied agents’ recommendations for movies on users’ experience and self-disclosure full body, motion, 3d [h2ea] online services [business] enjoyment, intention to use, satisfaction 225 (lab) peña & chen (2017) [chb] examines how activating prosocial and antisocial concepts affect successive prosocial behaviours and intentions full body, motion, 3d [h2sa] video game [entertainment] prosocial behaviour (helping), prosocial intention 309 (lab) schneider et al. (2017) [chb] examines the effects of online exclusion face only, static, 2d [h2a] interaction [social] belongingness, emotional state, selfesteem 105 (lab) 85 (lab) sebastian & richards (2017) [chb] examines the effects of educational embodied agents on performance of mental health patients using videos for intervention half body, motion, 3d [h2ea] training [health] na 245 (lab) seo et al. (2017) [chb] investigates self-presence when users are exposed to self-avatars face only, static, 3d [h2sa] interaction [social] identification, presence, similarity 25 (lab) tamborini et al. (2018) [chb] examines the effect of embodied agents’ ethnicity on trust motion, full body, 3d [h2ea] video game [entertainment] similarity (ethnicity), trust 205 (lab) felnhofer et al. (2018) [chb] investigating social avoidance and prosocial behaviour towards avatars and agents full body, motion, 3d [sa2a/sa2ea] interaction [social] agency, empathy presence, user anxiety 95 (lab) kaye et al. (2018) [chb] examines the effects of explicit priming and avatar gender full body, motion, 3d [h2sa] video game [entertainment] competence, identity, self-esteem 120 (lab) martens et al. (2018) [chb] examines avatars' and users' difference and similarities on time spent with avatar full body, motion, 3d [sa2a] video game [entertainment] similarity 124 (lab) mousas et al. (2018) [chb] examines the effects of appearance and motion of embodied agents on users’ emotional reactivity half body, motion, 3d [h2ea] virtual character [entertainment] emotional reactivity, emotional valence 72 (lab) note: misq: management information system quarterly; jmis: journal of management information system; isj: information system journal; eswa: expert systems with application; dss: decision support system; jcmc: journal of computer-mediated communication; icis: international conference on information system; ijhcs: international journal of human computer studies; chb: computer in human behaviors; h2h: human to human interaction; na: not applicable; lab: laboratory experiment, srv: survey; fld: field; n.a.: no avatars used; #p: number of participants; 2d: two dimensional; 3d: three dimensional; env.: interaction environment australasian journal of information systems aljaroodi, adam, chiong & teubner 2019, vol 23, research article avatars and embodied agents in experimental is research 37 acknowledgement the first author would like to acknowledge the full scholarship from the institute of public administration (ipa) in the kingdom of saudi arabia to study a phd degree in is at the university of newcastle, australia. we would also like to thank the editor and two anonymous reviewers for their insightful feedback and suggestions, which assisted in improving the overall quality of the paper. copyright: © 2019 aljaroodi, adam, chiong & teubner. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 research methodology of literature review 3 conceptual framework 3.1 avatars 3.2 embodied agents 3.3 artefact design 3.4 presence and immersion 3.5 experimental research on avatars and embodied agents 3.6 application contexts 4 interaction with self-avatars 4.1 self-avatar concepts 4.2 influences on user perceptions and behaviours 5 interaction with another person’s avatar 5.1 avatar concepts 5.2 influences on user perceptions and behaviours 6 interaction with embodied agents 6.1 embodied agent concepts 6.2 realism 6.3 persuasive and empathic embodied agents 6.4 the uncanny valley 7 discussion and conclusion 7.1 summary of findings 7.2 practical implications 7.3 future research directions 7.4 concluding remarks paper title (paper title style) australasian journal of information systems bhimavarapu et al. 2021, vol 25, research on consumer evaluation interactive app-based services: recovery evaluations of mobile applications 1 interactive app-based services: recovery evaluations and the mediating role of satisfaction in the relationship between customer-brand engagement and electronic word of mouth radhakrishna bhimavarapu ibs hyderabad, a constituent of ifhe, deemed to be university, india pankaj kumar mohanty ibs hyderabad, a constituent of ifhe, deemed to be university, india anitha acharya ibs hyderabad, a constituent of ifhe, deemed to be university, india manish gupta school of management, mahindra university, india manish.gupta.research@gmail.com abstract based on the theory of co-creation and resource-based theory, this study examines the differential impact of varying levels of customer participation in service recovery (firm, customer, and joint) on recovery satisfaction, customer-brand engagement (cbe), and electronic-word of mouth (e-wom). besides, using social exchange theory, this study examines the mediating role of consumers’ recovery satisfaction in the relationship between the types of service recovery and electronic word of mouth (e-wom). the data to accomplish the research objectives was collected from 495 customers of interactive food delivery apps in india. the experimental design was used to collect the data. the multivariate analysis of variance (manova) and process macro techniques were used to test the hypotheses. the findings suggest that joint recovery results in better recovery evaluations such as cbe, recovery satisfaction, and e-wom. also, recovery satisfaction mediated the relationship between cbe and e-wom. these findings provide empirical evidence for the linkages among the theory of co-creation, resource-based theory, and social exchange theory. the marketers of interactive food delivery apps may use these findings to ascertain which recovery type they should emphasize to improve recovery satisfaction. moreover, they are encouraged to take steps for improving recovery satisfaction as cbe affects e-wom through recovery satisfaction. keywords: service recovery, satisfaction, customer-brand engagement, e-word of mouth, interactive food delivery apps. 1 introduction technology-based interactive apps, especially mobile apps, are widely used for service delivery by individuals, businesses, and communities (verkijika & wet, 2019). according to a market research study, mobile app downloads across the globe are expected to cross 258 billion by 2022 (statista, 2019). however, the competition is intense (lim et al., 2015) in the app market and service failure is one of the major reasons that may lead to customer defection thereby hampering an app’s market share (hsu et al., 2019). the customer retention rate, according to statista 2019 study, decreases significantly over time in the case of mobile apps. thus, it is australasian journal of information systems bhimavarapu et al. 2021, vol 25, research on consumer evaluation interactive app-based services: recovery evaluations of mobile applications 2 important for the app-based service firms to know the ways and means through which they can improve the customer retention rate. in the academic literature, the topic ‘service recovery in the app-based services’ has received little attention despite its growing importance (hsu et al., 2019). a few of the past studies suggest that customers’ involvement in the service recovery process yields better results and improves customer satisfaction (cheung & to, 2016). although limited in number, prior studies on app-related service failures have examined the antecedents of the technology failure (dabholkar & spaid, 2012); interactivity (zhu et al., 2013); technology anxiety (hsu et al., 2019). there are few studies examining customer participation in the recovery process and its outcomes in the mobile app services. co-creation of recovery helps a firm to improve customer recovery perceptions if the customers are a part of cocreation during the service delivery (heidenreich et al., 2015). scholars have argued that customer participation in the service recovery process facilitates interaction between the customer resources such as time and effort with the firm resources such as employees and technology during the co-created recovery with the help of resource integration (xu et al., 2014). further, prior studies have examined the impact of co-created recovery on recovery outcomes such as recovery satisfaction and future intention to co-create (dong et al., 2016; suganathan et al., 2017; bagherzadeh et al., 2020). it may be that there is greater scope for interaction through resource integration in the recovery process (xu et al., 2014). customer involvement in service recovery may enhance customer’s emotional bonding with the service provider resulting in engagement with the brand (van vaerenbergh et al., 2018). the term customer-brand engagement is defined as “a consumer's positively valanced brandrelated cognitive, emotional and behavioral activity during or related to focal consumer/brand interactions” (hollebeek et al., 2014). researchers advocate that role of customer engagement with the brand in online service recovery needs to be explored which will help in understanding the customer-brand relationship (islam et al., 2019). specifically, more research is needed in the direction of the consequences of the recovery when firm assumes the responsibility of recovery solution (firm recovery), when customers co-create the recovery solution (joint recovery), when customers assume the sole responsibility of recovery solution (customer recovery), specifically in terms of customer’s engagement with the service provider (so et al., 2020). it is important to uncover the relationship between customer participation in service recovery and customer-brand engagement (cbe hereafter) as it facilitates designing better recovery mechanisms (islam et al., 2019). additionally, it is also required to validate the impact of customer participation in recovery when customers recover from the service failure since customer-firm interaction fosters customer-brand relationships. thus, the study wishes to contribute to the existing literature by understanding the relationship between customer participation in service recovery and cbe and e-wom. mobile apps are the interfaces with the least employee interference and the interaction facilitated during customer participation in service recovery helps in bridging the service gap (sugathan et al., 2017). thus, it is important to understand the impact of customer participation in service recovery in the context of mobile apps to confirm whether customer collaboration yields favorable outcomes. 2 theoretical framework development service recovery is an effective tool to alleviate customers’ negative behaviors followed by a service failure. theory of co-creation and resource-based theory suggests that two parties involved in the value creation process have mutual benefits associated with the outcome australasian journal of information systems bhimavarapu et al. 2021, vol 25, research on consumer evaluation interactive app-based services: recovery evaluations of mobile applications 3 (heidenreich et al., 2014; ranjan & read., 2016). firms and customers can co-create a service recovery to improve the service experience and recovery evaluations (roggeveen et al., 2011; dong et al., 2016; suganathan et al., 2017). recovery evaluations are better with the co-creation of recovery (joint recovery) as it facilitates better integration of firm resources and customer resources in the form of effort, information, and time (xu et al., 2014). customer participation in service recovery has emerged as a research domain due to the emerging role of customers in different types of services (suganathan et al., 2017; bagherzadeh et al., 2020). prior studies provide evidence that customer participation in service recovery (dong et al., 2008, roggeveen et al., 2011) facilitates better service recovery while using self-service technology platforms such as mobile apps (zhu et al., 2013). drawing inputs from the theory of co-creation and resource-based theory, three variants of service recovery, firm recovery, joint recovery, and customer recovery were proposed by the past researchers (fang et al., 2008; dong et al., 2008; park & ha, 2016; xu et al., 2014; dong et al., 2016). past studies have examined the impact of customer participation in the service recovery process and found that customer contribution to the recovery has a significant impact on the recovery outcomes. it was found that co-created service recovery (joint recovery) generally leads to better customer service recovery evaluations in terms of recovery satisfaction and repurchase satisfaction (dong et al., 2008; roggeveen et al., 2011; xu et al., 2014). ever since the emergence of co-created service recovery, most of the studies have tried to investigate antecedents and consequences of customer participation in the recovery process (dong et al., 2008; xu et al., 2012; zhu et al., 2013; dong et al., 2016). further, few studies have examined the customer journey from customer participation in service delivery to service recovery (heidenreich et al., 2015; suganathan et al., 2017; bagherzadeh et al., 2020). these studies provide pragmatic evidence that customer collaboration in the service recovery process improves the recovery evaluations. it was found that joint recovery significantly mitigates the effect of service failures and improves customer satisfaction than the other two (firm, customer) recovery types. however, there is scant research connecting the domain of service recovery with cbe especially when customers collaborate with the service firm to co-create the recovery solution (islam et al., 2019). customer participation service recovery may influence customer engagement as it facilitates resource integration in the form of information sharing, giving efforts in the recovery process. customer’s involvement in the recovery might influence the customer-brand relationship. pragmatic evidence suggests that customer involvement is a potential antecedent of cbe (france et al., 2016). thus, the study studies the relationship between service recovery and cbe to fill the void of research in studying the nexus between service recovery and customer-brand relationship in the interactive service platforms. the above discussion highlights the importance of investigating the impact of three recovery types will give more meaningful insights for the service recovery in the interactive app service platforms. further, the study also investigates the impact of customer-brand engagement on e-wom as it is one of the effective ways to exhibit customer-brand engagement (tu et al., 2018). three recovery types were used in this study as recommended by dong et al. (2016). recovery types are classified into three (firm recovery, joint recovery, and customer recovery) based on the customers’ level of participation in the recovery. firm recovery refers to the situation where a firm itself does the recovery with no/less amount of effort from the customer. joint recovery refers to the situation where the customer and service provider both contribute to the australasian journal of information systems bhimavarapu et al. 2021, vol 25, research on consumer evaluation interactive app-based services: recovery evaluations of mobile applications 4 recovery. customer recovery refers to the situation where a customer himself finds out a way to recover from the failure without any help from the service provider. 2.1 service recovery and recovery satisfaction the theory of co-creation proposes that customers create value by sharing information, investing their resources, and customizing the service outcomes (fang et al., 2008). this theory can be extended to the domains of service recovery in the form of mandatory or nonmandatory customer participation depending on the service context (dong et al., 2016). when a service failure occurs, customers contribute to the recovery by providing the required information in a remote service setting such as a mobile application (roggeveen et al., 2011). customers participating in service recovery attach credit to their contribution and report better satisfaction with the outcome (dong et al., 2008). the theory of co-creation coupled with resource-based theory suggests that resource collaboration enriches the co-creation experience as there is an adequate fit of mutual interests of both the parties involved. das and teng (2000), in their ‘resource-based theory’ also state that the two parties involved in a collaborative work realize greater value with combined synergies. there is less scope for customer-firm interaction to integrate their resources in firm recovery and customer recovery as firms may not have access to the failure-related information. thus, it is difficult for a firm to identify an ideal solution for failure as there is less scope for paying heed to customers’ expectation of the recovery solution (heidenreich et al., 2014; dong et al., 2016). joint recovery offers more scope to accommodate the customer recovery expectations (dong et al., 2016), thus leading to greater recovery satisfaction. drawing inputs from the above theories it can be inferred that joint recovery has a positive influence on recovery satisfaction when customers collaborate with the service provider using their resources, as they can exercise more control and exert influence on the solution to the service failure. thus, the below hypothesis can be stated: h1a: in case of service recovery, customer recovery results in greater recovery satisfaction than firm recovery. h1b: in case of service recovery, joint recovery results in greater satisfaction than customer recovery. h1c: (combining h1a and h1b) in case of service recovery, joint recovery results in greater satisfaction than firm recovery and customer recovery. 2.2 service recovery and cbe previous studies on customer-brand relationship have considered cbe as a psychological state that occurs by customer interactions with the brand (france et al., 2016; mohanty & dey, 2020). extant research suggests that customer participation and consumer interactions with the service provider offer mutual benefits to both the customer and service provider (bagozzi & dholakia, 2006). it is also suggested that such interactions will make the consumer more enthusiastic about the brand, thereby resulting in positive engagement with the service provider/brand (vivek et al., 2012). customer participation in service recovery offers scope for the interactions that might lead to increased engagement with the brand (hazee et al., 2017). service recovery can be a relational driver of cbe or antecedent an antecedent to behaviors reflecting cbe (liu et al., 2021; so et al., 2020). social exchange theory posits that customers are better engaged with a brand when brand resources are integrated with customers’ resources during the customer journey with the australasian journal of information systems bhimavarapu et al. 2021, vol 25, research on consumer evaluation interactive app-based services: recovery evaluations of mobile applications 5 service firm (guo et al., 2017). a customer’s collaboration to the recovery process involves integration of customers’ resources (time, effort, knowledge) with firm resources (technology, employees, guidelines) which leads to greater cbe. thus, drawing inputs from social exchange theory and studies in the domain of customer engagement and service recovery, the following hypotheses are framed. h2a: in case of service recovery, customer recovery results in greater cbe than firm recovery. h2b: in case of service recovery, joint recovery results in greater cbe than customer recovery. h2c: (combining h2a and h2b) in case of service recovery, joint recovery results in greater cbe than firm recovery and customer recovery. 2.3 service recovery and e-word of mouth (e-wom) in the era of digitization, the electronic form of word of mouth (e-wom) has overtaken the traditional form of word of mouth (srivastava & sivaramakrishnan, 2020), and customers are no longer believe the product/service-related information provided by the brand. instead, they trust the consumer-generated review information posted on various websites/apps (al-htibat & garanti, 2019). the term e-wom is defined as the extent to which an individual says positive things about the brand to others on the internet (srinivasan et al., 2002). extant literature has widely studied the concept of e-wom in the services marketing domain as customers use it as a key factor to arrive at purchase decisions (haywood, 1989). there is a greater possibility that customers engage in negative word of mouth when they face service failure. the theory of cocreation posits that customers evaluate their efforts more positively which results in positive post-consumption behaviors such as word of mouth and switching intention (zhu et al., 2013). furthermore, e-wom has been used increasingly in service recovery-related studies due to its importance associated with customers’ post-purchase behavior (xie et al., 2019). service recovery studies have examined the impact of different types of service recovery satisfaction (choi & choi 2014; jung & seock, 2017) and found that service recovery strategies improve the positive word of mouth and mitigate negative word of mouth. additionally, research studies found that co-creation of service recovery enhances service performance and improves behavioral intentions of the customers as they actively contribute to the service (xie et al., 2019). co-creation of recovery implies that firm resources and customer resources are used to arrive at the desired resolution for the customers which improves the post-purchase behaviors of the customers after a service failure. although customer recovery facilitates greater control, the onus lies on the customer to make efforts to overcome the service problem. thus, customer recovery may not result in better post-purchase behavior than joint recovery. thus, the following hypotheses are framed: h3a: in case of service recovery, customer recovery results in greater e-wom than firm recovery. h3b: in case of service recovery, joint recovery results in greater e-wom than customer recovery. h3c: (combining h3a and h3b) in case of service recovery, joint recovery results in greater ewom than firm recovery and customer recovery. australasian journal of information systems bhimavarapu et al. 2021, vol 25, research on consumer evaluation interactive app-based services: recovery evaluations of mobile applications 6 2.4 cbe and e-word of mouth e-wom forms the consumer attitude toward a brand and influences the expectations of customers from a brand in the interactive platforms (loureiro et al., 2017). this is also an act of meeting and sharing opinions on the internet, which provides usability, accessibility of information that was unavailable in traditional offline wom (seo et al., 2020, p.3). customer participation in service creation and information sharing will lead to increased positive wordof-mouth intentions among the customers (dessart et al., 2015), with the improved engagement with the brand during meaningful interactions in online platforms (srivastava & sharma, 2017). studies on customer engagement have emphasized the creation of cbe, which occurs by interactive customer experiences with the brand and in turn influences the consumer attitude and behaviors (mollen & wilson, 2010; dessart et al., 2015). recent studies on cbe have explored the role of cbe in enhancing word-of-mouth intention (mohanty & dey, 2020; acharya, 2020). it has been argued that customer’s affective engagement drives emotional attachment with the brand, and it has a higher probability of resulting in positive word-of-mouth in online platforms (jung & seock, 2017). it was found that customers with high emotional attachment with the brand display high level of wom than those of less engaged customers (hudson et al., 2015). thus, the following hypothesis can be framed to examine the impact of cbe on e-wom: h4: cbe has a positive impact on e-wom. 2.5 cbe, recovery satisfaction and e-wom extant literature has studied customer satisfaction as a consequence of cbe. past research studies generally suggest that customer satisfaction consists of an overall evaluation of the consumer-brand interactions following purchase or consumption of a product or a service (hollebeek et al., 2014). before the development of cbe literature in marketing, the term employee engagement received considerable attention from the organization behavior (ob) researchers (robinson et al., 2004; saks, 2006). empirical investigation indicated a positive relationship between employee engagement and employee job satisfaction (saks, 2006). analogously, the nature of such a relationship may also extend to the marketing domain for understanding the relationship between cbe and customer satisfaction (hollebeek, 2011). it is argued that customer emotional attachment with the brand has positive impacts on e-wom and the brand relationship quality constructs such as trust, commitment & satisfaction (hollebeek, 2011; hudson et al., 2015). empirical investigations have also found a significant positive impact of customer engagement on the relationship quality (consists of; trust and customer satisfaction) (so et al., 2014). in addition, researchers have also suggested that customer engagement with the brand enhances customer satisfaction and therefore, customer satisfaction could be a potential consequence of cbe (hollebeek, 2011). empirical findings suggest a positive impact of cbe on customer satisfaction (srivastava & sivaramakrishnan, 2020). on the other hand, study on customer satisfaction have acknowledged the importance of customer satisfaction for enhancing the word-of-mouth intention of customers (kim et al., 2009). an array of empirical research in the domain of marketing has found a positive relationship between customer satisfaction and wom (sivadas & jindal, 2017). the same relationship may be expected between cbe and recovery satisfaction since cognitive and affective responses are increasingly viewed as essential in researching how customers evaluate service recovery (schoefer, 2008, ozkan-tektas & basgoze, 2017). further, research australasian journal of information systems bhimavarapu et al. 2021, vol 25, research on consumer evaluation interactive app-based services: recovery evaluations of mobile applications 7 on service recovery found that cognitive and affective response of customer plays an important role in the formation of recovery satisfaction (schoefer, 2008). further, the empirical investigations indicated a significant effect of customer positive emotion on recovery satisfaction (schoefer, 2008; xu et al., 2019). research has also found the indirect effect of customer's cognitive and affective responses on recovery satisfaction in the service failure context (varela-neira et al., 2008). it can be argued that the recovery process facilitates cognitive and affective engagement in the form of positive emotion which acts as a judgment factor for recovery satisfaction and post-recovery behavior of the customer. based on the above arguments, the following hypothesis can be stated: h5: recovery satisfaction mediates the relationship between cbe and e-wom. figure 1. conceptual framework 3 methodology 3.1 sampling and data collection procedure data were gathered online from the consumers of food delivery applications (apps) in india (e.g., zomato & swiggy). the context was chosen based on three major reasons. these are a) fastest growing industry with a compounded average growth rate (cagr) of 11 percent (research & markets, 2020); b) lower customer retention rate compared with other app-based service categories (statista, 2019); and c) provide an interactive platform for better service creation (lim et al., 2015). the study used a between-subjects experimental design for the primary data collection. three different scenarios were designed featuring the service recovery (customer, firm, and joint) conditions in the app-based food delivery services (refer to appendix for scenarios). all the three scenarios were manipulated by varying the level of customer participation in the recovery process consistent with dong et al. (2008); and dong et al. (2016). firm recovery scenario had limited participation from the customer to the recovery process. joint recovery condition had contribution from both the customer and firm to the recovery process. customer recovery condition had the major contribution and efforts from the customer to the recovery process. a sample size of 495 food delivery app users in the millennial category participated during the online experiments. the sample size of 495 was based on the three major criteria. service recovery • firm recovery • customer recovery • joint recovery recovery satisfaction customer-brand engagement e-wom h3a-h3c australasian journal of information systems bhimavarapu et al. 2021, vol 25, research on consumer evaluation interactive app-based services: recovery evaluations of mobile applications 8 these are a) for maintaining equal no-of sample size across three groups of experimental conditions (firm recovery, customer recovery, and joint recovery) in manova; b) in order to meet the criteria of multivariate normality assumptions, the study followed the guideline provided by koenker & bassett (1978). according to koenker & bassett (1978) regression estimates follow an asymptotic multivariate normal distribution in case of large sample sizes; c) study has also suggested that a researcher would have better power to detect small effect size in regression analysis with approximately 30 respondents per variable (voorhis & morgan, 2007). additionally, it was also found that the empirical power of all the test statistics reaches 100% when the sample size is as large as 500 (tong & bentler, 2013). millennials were recruited for the study as past studies suggest that millennial customers are more empowered with the usage of electronic devices and involved in the interaction with the apps and service provider (vivek et al., 2012). additionally, the study has also found that millennials are three times more likely to order food online than their parents (cheng, 2018). a screening question asked to check whether the respondent has ordered food online and has experienced at least one service failure in the last six months. qualified respondents were then randomly assigned to any one of the three scenarios of the study. three independent variables were manipulated based on customer collaboration with the service recovery. the study also included one question on scenario realism to understand if the scenarios replicate real time true scenarios faced by customers. respondents were asked to evaluate the recovery after reading the scenario and provide ratings for recovery satisfaction, cbe, and e-wom. 3.1.1 sample characteristics the demographic profile of the 495 sample respondents reflected their representation of the target population. the sample had 57% of male and 43% of female respondents. a majority (55.4%) of the respondents held a post-graduate degree. nearly 36% of the respondents had a family income of indian national rupee 500,000-1,000,000 followed by more than 1,000,0001,500,000 (24%). 3.2 measures all the constructs were measured using a five-point likert scale of 1 to 5, where 1 indicates ‘strongly disagree’ and 5 indicates ‘strongly agree’ except the manipulation check for recovery scenarios. these scenarios were measured using a summative score of both customer and firm contribution to the recovery. the summative scale was required to measure the relative contribution of the customers with respect to the firm in the service recovery process. manipulation of three service recovery types was checked using a single item scale by dong et al. (2016). the scale item: ‘using a 10-point scale, please rate both your and firm contribution to the described recovery solution (both should add up to 10)’. recovery satisfaction was measured using a three items scale by dong et al. (2016). a sample item includes: ‘how satisfied were you about the solution to this problem?’. cbe was measured using a 10 items scale of hollebeek et al. (2014). a sample item includes: ‘using my food app gets me to think about that brand’. similarly, e-wom was measured using a four items scale adopted from srinivasan et al. (2002). a sample item of the e-wom scale includes: ‘i would like to introduce the food app brand to others. 3.3 data analysis the data gathered was analyzed mainly in two phases. the first phase consisted of preliminary analysis comprising the test of internal consistency, bivariate pearson correlation coefficient, australasian journal of information systems bhimavarapu et al. 2021, vol 25, research on consumer evaluation interactive app-based services: recovery evaluations of mobile applications 9 mean, standard deviation (sd), skewness, and kurtosis. the possible impact of common method bias (cmb) was also checked using harman’s single factor test for all the latent constructs. the second phase of the data analysis consisted of hypotheses testing. multivariate analysis of variance (manova) was used to examine the differential impact of three service recovery types on recovery satisfaction h1(a-c), cbe (h2(a-c), and e-wom (h3(a-c). following hayes (2017) procedure of examining the mediating effect, regression analysis was performed using process macro, model-4 to examine the mediating role of recovery satisfaction between cbe and e-wom relationship (h4-h5). 3.3.1 common method bias to avoid and detect the possible impact of common method variance bias (cmb) in survey data, the study used both ex-ante measures and ex-post techniques suggested by podsakoff et al. (2003). several ex-ante strategies were adopted by the study to prevent the cmb effect. these are (1) items of the latent constructs were jumbled up and were not labelled, (2) respondents were informed about the anonymity and confidentiality of their responses during the data collection. however, to test the possible existence of cmb in the data, harman’s single factor test was performed by constraining all the variables into a single factor with rotation constraint. the results revealed that the total variance explained by a common factor solution accounted for 39.37%, which is less than the cut-off limit (50%) (podsakoff et al., 2003; malhotra et al., 2006). the study also checked the full collinearity test as a comprehensive procedure for the identification of cmb (koch & lynn, 2012). the results indicated that all vif values of the construct were below the 3.3 threshold mark (kock, 2015), indicating no cmb problem. finally, the study also followed the correlation matrix approach (presented in table 1) suggested by pavlou et al. (2007) to detect cmb. the result does not indicate any highly correlated factors (r > .90) in the analysis. therefore, the result suggested the absence of cmb in the data. 3.3.2 preliminary analysis the findings of this study provide insights into the relationships among the constructs under investigation. in the first phase of data analysis, the internal consistency of the constructs was tested. cronbach’s alpha values for all the latent constructs; recovery satisfaction, cbe, and electronic-word-of-mouth were found to be 0.91, 0.88, and 0.78 respectively (refer table 1). since all the constructs had a cronbach’s alpha value of more than 0.70, internal consistency was achieved (nunnally, 1978; hair et al., 2010). the data collected from the sample also meets the normality assumptions criteria as the skewness and kurtosis value lie between the range of ± 2 and ±7 respectively, suggested by trochim & donnelly (2008). min max mean sd skewness kurtosis 1 2 3 1 rs 1.0 5.0 3.35 0.9 –0.26 –0.23 0.91 2 cbe 1.7 5.0 3.67 0.6 –0.92 1.43 .51** 0.88 3 e-wom 1.0 5.0 3.82 0.7 –0.95 1.54 .44** .68** 0.78 table 1. descriptive statistics, correlation coefficients, and cronbach’s alpha (n = 495) ** ** p < .01. note: values along the diagonal in boldface represent cronbach’s alpha coefficients; rs = recovery satisfaction; cbe = consumer-brand engagement; e-wom = electronic-word of mouth. australasian journal of information systems bhimavarapu et al. 2021, vol 25, research on consumer evaluation interactive app-based services: recovery evaluations of mobile applications 10 3.3.3 scenario realism and manipulation check realism check was performed to know whether the scenarios replicate true recovery scenarios faced by customers in real-time. the results indicated that the mean score of all the recovery scenarios (mean joint = 4.13, and mean firm = 4.35, and mean customer =4.11) represent realistic scenarios. manipulation check of three levels of customer contribution in recovery scenarios was done using anova. the results revealed a significant difference among the levels of customer contribution to the recovery scenarios across three recovery scenarios (f 822.93, p < .001). the mean score of customer contribution in customer recovery scenario (mean customer = 8.08) is higher than the mean scores of joint recovery and firm recovery scenarios (mean joint = 4.89; and mean firm = 2.55). 4 results once psychometric properties of the constructs were found to be valid, and manipulations were found to be successful the study proceeded for hypothesis testing. manova and regression analysis using process macro were performed to test the hypothesized relationships. manova results indicated that the main effect of service recovery on recovery satisfaction, cbe, and e-wom were significant. the mean scores of all the dependent variables are presented in table 2. the homogeneity of variance test for all the mean scores of the dependent variables was found to be insignificant indicating the robustness of the model. recovery type n recovery satisfaction (rs) mean cbe mean e-wom mean robustness test firm recovery 165 3.02 3.42 3.59 wilk’s lambda = 0.89 f = 9.98*** joint recovery 165 3.67 3.91 4.05 customer recovery 165 3.34 3.67 3.80 table 2. main effects of service recovery on recovery satisfaction, cbe & e-wom *** p < .01. dependent variable mean difference std. error recovery satisfaction joint recovery firm recovery .6419*** 0.09874 customer recovery .3280** 0.09874 cbe joint recovery firm recovery .4927*** 0.07247 customer recovery .2473** 0.07247 e-wom joint recovery firm recovery .4639** 0.08006 customer recovery .2415** 0.08006 table 3. multiple comparison of means ** p < .01. *** p < .001. the main effect of service recovery on recovery satisfaction was found to be significant (f = 21.13, p < .05). the study also tested for the homogeneity of variance using levene statistic. the result indicated that the homogeneity of variance test was found to be insignificant australasian journal of information systems bhimavarapu et al. 2021, vol 25, research on consumer evaluation interactive app-based services: recovery evaluations of mobile applications 11 (levene statistic: 0.311). the mean score presented in table 2 indicated that the mean score of joint recovery in recovery satisfaction (rsjr = 3.67) is greater than customer recovery (rscr = 3.34) and firm recovery (rsfr = 3.02). the multiple comparisons of means result presented in table 3 found significant differences among the group means values, supporting h1a, h1b, and h1c. the main effect of service recovery on cbe was found to be significant (f= 23.11, p < .05), and the test of homogeneity for variances was found to insignificant suggesting variance within the mean scores (levene statistic = .293). further, the results of multiple comparisons of means analysis provided support for the hypotheses pertaining to the relationship between three recovery types and cbe. joint recovery resulted in more customer engagement with the brand (cbejr = 3.91) than firm recovery (cbefr = 3.42) or customer recovery (cbecr = 3.67), supporting h2a, h2b, and h2c. the main effect of service recovery on e-wom was found to be significant (f = 16.794, p < .05) suggesting the differential impact of the three recovery types on e-wom. levene statistic (.476) indicated a non-significant homogeneity in the mean scores of e-wom. the results indicated that joint recovery results in greater e-wom intention (e-womjr = 4.05) than firm recovery (ewomfr = 3.59) and customer recovery (e-womcr = 3.80), in support of h3a, h3b, and h3c. the present study tested the mediation-related hypothesis, h5 (cbe recovery satisfaction  e-wom) by using the procedure suggested by hayes (2017). the statistical significance of the mediation test was checked by executing the bootstrapping procedure of the spss process macro. the results indicated that both the direct and the indirect effect of cbe on e-wom (βindirect = 0.06 and βdirect = 0.69) were statistically significant at a 95% confidence interval, given that the interval contains non-zero elements (llci=0.02, ulci =0.11). the above results are in support of the validity of study hypothesis h4 and h5. the mediating role of recovery satisfaction was found in the relationship between cbe and e-wom. moreover, the standardized value of cbe (β = 0.62) is higher than the value of recovery satisfaction (β = 0.11) indicating a predominant role of cbe in explaining e-wom. 5 discussion this study examined the relationship among the service recovery, recovery satisfaction, cbe, and e-wom drawing inputs from theory of co-creation, resource-based theory, and social exchange theory. our study investigated the differential impact of service recovery with varying levels of customer-firm interaction in app-based service recovery. the research questions were: a) does the type of service recovery impact the recovery satisfaction and cbe differently? b) does the type of service recovery influence the e-wom differently? c) does recovery satisfaction mediate the relationship between cbe and e-wom? in line with the existing literature (roggeveen et al., 2011; dong et al., 2016), the present study found joint recovery to have better impact on satisfaction associated with the recovery. roggeveen et al. (2011) provided empirical evidence that customer collaborating with firm during the recovery process report better recovery outcomes than those who do not collaborate with firm when they face service failure with the airline service providers. the present study results augment this effect suggesting that joint recovery produces better recovery outcomes. examining service recovery in a technology-based service, heindenreich et al. (2014) found that joint recovery provides better recovery outcomes. the results from the australasian journal of information systems bhimavarapu et al. 2021, vol 25, research on consumer evaluation interactive app-based services: recovery evaluations of mobile applications 12 present study strengthen the arguments in favor of joint recovery suggesting that co-creation of recovery helps when customers face a service failure with a food delivery app which is a technology-based service. additionally, the findings of the study extend the work of dong et al. (2016) by examining the relationship between joint recovery to customer-brand relationship. the results also indicate that consumers collaborating with the service provider for the recovery solution exhibit greater cbe than consumers who receive recovery solutions from the firm or who fix the service failure with their own efforts or minimum support from the firm. the hypotheses results are consistent with the theoretical inputs drawn from resourcebased theory and social exchange theory which suggests that interaction in the joint recovery facilitates information sharing and extending efforts during the service recovery process. thus, it was empirically tested that joint recovery results in greater cbe than firm recovery and customer recovery. similarly, the results also indicate that co-creation of recovery improved the e-wom among the customers after a service failure. it is consistent with extant literature (jung & seock, 2017) which suggests that co-creation of service recovery emanates positive word of mouth. it can be inferred that the customer-firm interactions facilitate a greater control over the service performance leading to positive word of mouth. furthermore, the results support the mediation role of recovery satisfaction on the relationship between cbe and ewom. this offers a new perspective to the service recovery literature by examining the role of recovery satisfaction in determining the customer brand relationships. 5.1 theoretical contributions the study offers four major theoretical contributions. first, drawing inputs from the theory of co-creation and resource-based theory, the study investigated the relationship between customer participation in service recovery and cbe. this offers a new perspective to both service recovery and customer-brand relationship domains, suggesting that collaborating with customer during service recovery might provide an opportunity to improve cbe. customers invest their time and share the information and put efforts to recover from the service failure which requires integration of firm's resources at the same time. this proves that resource integration process involved in a joint recovery can be source of motivation for the customers to engage with the service firm. second, drawing inputs from social exchange theory, the study investigated the relationship between cbe and e-wom. customers realize the benefits associated with joint recovery and exhibit better engagement with service provider which leads to e-wom responses from the customers. customers’ engagement with the service provider can become a relational driver to emote positive wom intentions after a successful service recovery. this is consistent with social exchange theory as customers are encouraged by the interactions and are likely to exhibit social behaviors in the form of e-wom. customers’ engagement with the service provider can become a relational driver to emote positive wom intentions after a successful service recovery. third, the mediating role of recovery satisfaction in the relationship between cbe and e-wom was found to be significant, suggesting that recovery satisfaction will influence the relationship between, cbe and e-wom. further, the study also contributes the extant literature as it addresses the call to explore the role of cbe in service recovery (hollebeek, 2018), especially when customers co-create the recovery (van vaerenbergh et al., 2018). australasian journal of information systems bhimavarapu et al. 2021, vol 25, research on consumer evaluation interactive app-based services: recovery evaluations of mobile applications 13 finally, the study provides extended support for considering service recovery as a journey which can be used as a touch point to build long term relationships with customers (van vaerenbergh et al., 2019). drawing on the above theories, the present study offers meaningful insights into the customer-firm collaboration during service recovery and its differential impact on recovery satisfaction, cbe, and e-wom. 5.2 managerial implications it is difficult for any firm to offer service with zero defects; therefore, firms can use the cocreation of recovery to improve customer-firm interaction during the recovery process. the study offers four managerial implications for the industry practitioners. first, it is advisable for the firms to design service recovery in the mobile apps with better interactivity as customer collaboration with the firm has a wider scope for engagement with the service provider. for instance, firms like swiggy, uber, ola etc. have introduced interactive chatbots to help with any deviations in the service. second, it is suggested that the service firms invest more in creating awareness about the joint recovery process as it might help the customers to be future ready to handle service failure as major service failures in a mobile app service take place independent of employee interference. firms can leverage social media to create awareness of the joint recovery process since the virtual presence facilitates awareness among the observing customers in the social media (schaefers & schamari, 2016) third, service firms can also promote brand communities that help the customers to share their stories of successful recovery instances to enhance e-wom. fourth, service firms should offer rewards for integrating their resources during the recovery process which fosters a better customer brand relationship (shulga & busser., 2020). for example, customers who participate in the joint recovery can be rewarded with a coupon or additional discount which might motivate a customer to maintain long-term relationship with the brand (hwang et al. 2020). the findings indicate that customer’s participation in service recovery plays a key role in determining the customer recovery evaluations. thus, service firms should emphasize on defining customer’s role as a partial employee during the recovery process. the findings of the study provide managers with the opportunity to identify the resources that can be integrated during joint recovery process. the service firm should collect the all the information about the resources that customers are willing to integrate during the recovery process so that the firm identifies appropriate resources to be deployed for improved customer-brand relationship after a service failure. 6 limitations and directions for future research like other behavioral studies, this study also has a few limitations. first, the results of the study cannot be generalized across all the service context as the nature of service failures might vary depending on the context (forbes, 2008). second, the study has used single platform interaction happens in a service recovery; however, the service interactions might be complex as the service providers are integrating the services across multi-channels. thus, studying the multi-channel interactions in the service recovery may offer insightful results. third, the impact of service recovery initiation (xu et al., 2014) on the relationship among service recovery, cbe and e-wom can be an interesting future research direction as the customer initiation of recovery may reduce or increase the cbe based on the outcome. australasian journal of information systems bhimavarapu et al. 2021, vol 25, research on consumer evaluation interactive app-based services: recovery evaluations of mobile applications 14 fourth, future research may also investigate the role of recovery speed on the relationship between service recovery and customer engagement with the brand as the prior studies in service recovery suggest that recovery speed influences the impact of recovery on customer evaluations and customer psychological association with the brand (roggeveen et al. 2011). finally, it is also interesting to examine the comparison of the resource exchange-based joint recovery with a goal-based response to the service failure which addresses the goal heterogeneity (van vaerenbergh et al., 2019). for example, a customer who faces service failure while looking for utilitarian need fulfillment may respond differently from a customer who faces hedonic need fulfillment (park & ha, 2016). 7 conclusion this research work is one of the first to investigate the relationships among service recovery, cbe, and e-wom from the perspective of customer participation in service recovery. this study provides the evidence for the impact of co-creation of recovery on cbe and recovery satisfaction, which in turn will influence the e-wom intentions of the customers followed by a successful service recovery. findings of the study also provide support for the mediating effect of recovery satisfaction in the relationship between cbe and e-wom after a successful service recovery. references acharya, a. (2020). the impact of brand familiarity, customer brand engagement and selfidentification on word-of-mouth. south asian journal of business studies, 10(1), 29-48. al-htibat, a., & garanti, z. (2019). impact of interactive ereferral on tourists’ behavioral intentions. marketing intelligence & planning, 37(5), 527-541. bagherzadeh, r., rawal, m., wei, s., & saavedra torres, j. (2020). the journey from customer participation in service failure to co-creation in service recovery. journal of retailing and consumer services, 54, 102058. bagozzi, r., & dholakia, u. (2006). antecedents and purchase consequences of customer participation in small group brand communities. international journal of research in marketing, 23(1), 45-61. cheng, a. (2018). millennials are ordering more food delivery, but are they killing the kitchen, too?. forbes, retrieved october 17, 2019, from https://www.forbes.com/sites/andriacheng/2018/06/26/millennials-are-orderingfood-for-delivery-more-but-are-they-killing-the-kitchen-too/?sh=31974523393e cheung, f., & to, w. (2016). a customer-dominant logic on service recovery and customer satisfaction. management decision, 54(10), 2524-2543. choi, b., & choi, b. (2014). the effects of perceived service recovery justice on customer affection, loyalty, and word-of-mouth. european journal of marketing, 48(1/2), 108-131. dabholkar, p., & spaid, b. (2012). service failure and recovery in using technology-based selfservice: effects on user attributions and satisfaction. the service industries journal, 32(9), 1415-1432. australasian journal of information systems bhimavarapu et al. 2021, vol 25, research on consumer evaluation interactive app-based services: recovery evaluations of mobile applications 15 das, t., & teng, b. (2000). a resource-based theory of strategic alliances. journal of management, 26(1), 31-61. dessart, l., veloutsou, c., & morgan-thomas, a. (2015). consumer engagement in online brand communities: a social media perspective. journal of product & brand management, 24(1), 28-42. dong, b., evans, k., & zou, s. (2008). the effects of customer participation in co-created service recovery. journal of the academy of marketing science, 36(1), 123-137. dong, b., sivakumar, k., evans, k., & zou, s. (2016). recovering coproduced service failures. journal of service research, 19(3), 291-306. fang, e., palmatier, r., & evans, k. (2008). influence of customer participation on creating and sharing of new product value. journal of the academy of marketing science, 36(3), 322-336. forbes, l. (2008). when something goes wrong and no one is around: non-internet self-service technology failure and recovery. journal of services marketing, 22(4), 316-327. france, c., merrilees, b., & miller, d. (2016). an integrated model of customer-brand engagement: drivers and consequences. journal of brand management, 23(2), 119-136. guo, l., gruen, t., & tang, c. (2017). seeing relationships through the lens of psychological contracts: the structure of consumer service relationships. journal of the academy of marketing science, 45(3), 357-376. hair, j. f., black, w. c., babin, b. j., & anderson, r. e. (2010). multivariate data analysis (7th ed.). upper saddle river, new jersey: prentice hall. hayes, a. f. (2017). introduction to mediation, moderation, and conditional process analysis: a regression-based approach. guilford publications. haywood, k. m. (1989). managing word of mouth communications. journal of services marketing, 3(2), 55-67. hazée, s., van vaerenbergh, y., & armirotto, v. (2017). co-creating service recovery after service failure: the role of brand equity. journal of business research, 74, 101-109. heidenreich, s., wittkowski, k., handrich, m., & falk, t. (2014). the dark side of customer cocreation: exploring the consequences of failed co-created services. journal of the academy of marketing science, 43(3), 279-296. hollebeek, l. (2011). demystifying customer brand engagement: exploring the loyalty nexus. journal of marketing management, 27(7-8), 785-807. hollebeek, l. (2018). individual-level cultural consumer engagement styles: conceptualization, propositions and implications. international marketing review, 35(1), 42-71. hollebeek, l., glynn, m., & brodie, r. (2014). consumer brand engagement in social media: conceptualization, scale development and validation. journal of interactive marketing, 28(2), 149-165. australasian journal of information systems bhimavarapu et al. 2021, vol 25, research on consumer evaluation interactive app-based services: recovery evaluations of mobile applications 16 hsu, l., wang, k., chih, w., & lin, w. (2019). modeling revenge and avoidance in the mobile service industry: moderation role of technology anxiety. the service industries journal, 124. hudson, s., roth, m., madden, t., & hudson, r. (2015). the effects of social media on emotions, brand relationship quality, and word of mouth: an empirical study of music festival attendees. tourism management, 47, 68-76. hwang, y., gao, l., & mattila, a. s. (2020). what recovery options to offer for loyalty reward program members: dollars vs. miles?. international journal of hospitality management, 87, 102496. islam, j. u., rahman, z., & hollebeek, l. d. (2019). the role of consumer engagement in recovering online service failures: an application of service-dominant logic. in handbook of research on customer engagement. edward elgar publishing. jung, n., & seock, y. (2017). effect of service recovery on customers’ perceived justice, satisfaction, and word-of-mouth intentions on online shopping websites. journal of retailing and consumer services, 37, 23-30. kim, w., ng, c., & kim, y. (2009). influence of institutional dineserv on customer satisfaction, return intention, and word-of-mouth. international journal of hospitality management, 28(1), 10-17. kock, n. (2015). common method bias in pls-sem: a full collinearity assessment approach. international journal of e-collaboration (ijec), 11(4), 1-10. kock, n., & lynn, g. (2012). lateral collinearity and misleading results in variance-based sem: an illustration and recommendations. journal of the association for information systems, 13(7). koenker, r., & bassett, g. (1978). regression quantiles. econometrica, 46(1), 33-50. doi: 10.2307/1913643 kozinets, r., de valck, k., wojnicki, a., & wilner, s. (2010). networked narratives: understanding word-of-mouth marketing in online communities. journal of marketing, 74(2), 71-89. lim, s., bentley, p., kanakam, n., ishikawa, f., & honiden, s. (2015). investigating country differences in mobile app user behavior and challenges for software engineering. ieee transactions on software engineering, 41(1), 40-64. liu, x., shin, h., & burns, a. (2021). examining the impact of luxury brand's social media marketing on customer engagement: using big data analytics and natural language processing. journal of business research, 125, 815-826. loureiro, s. m. c., gorgus, t., & kaufmann, h. r. (2017). antecedents and outcomes of online brand engagement: the role of brand love on enhancing electronic-word-ofmouth. online information review, 41(7), 985-1005. australasian journal of information systems bhimavarapu et al. 2021, vol 25, research on consumer evaluation interactive app-based services: recovery evaluations of mobile applications 17 malhotra, n., kim, s., & patil, a. (2006). common method variance in is research: a comparison of alternative approaches and a reanalysis of past research. management science, 52(12), 1865-1883. mohanty, p., & dey, d. (2020). consumer-brand engagement with e-commerce market place brands. journal of electronic commerce in organizations, 18(3), 21-37. mollen, a., & wilson, h. (2010). engagement, telepresence and interactivity in online consumer experience: reconciling scholastic and managerial perspectives. journal of business research, 63(9-10), 919-925. nunally, j. c. (1978). psychometric theory (2nd ed.). new york: mcgraw-hill. ozkan-tektas, o., & basgoze, p. (2017). pre-recovery emotions and satisfaction: a moderated mediation model of service recovery and reputation in the banking sector. european management journal, 35(3), 388-395. park, j., & ha, s. (2016). co-creation of service recovery: utilitarian and hedonic value and post-recovery responses. journal of retailing and consumer services, 28, 310-316. pavlou, p. a., liang, h., & xue, y. (2007). understanding and mitigating uncertainty in online exchange relationships: a principal-agent perspective. mis quarterly, 31(1), 105-136. podsakoff, p. m., mackenzie, s. b., lee, j., & podsakoff, n. p. (2003). common method biases in behavioral research: a critical review of the literature and recommended remedies. journal of applied psychology, 88(5), 879-903. ranjan, k. r., & read, s. (2016). value co-creation: concept and measurement. journal of the academy of marketing science, 44(3), 290-315. research & markets (2020). online food delivery services global market report 2020-30: covid19 growth and change. retrieved may 17, 2020, from https://www. researchandmarkets.com/reports/5024095/online-food-delivery-services-globalmarket?utm_source=dynamic&utm_medium=bw&utm_code=jzdnfb&utm_ campaign=1388109+-+global+online+food+delivery+services+market+(2020+to+ 2030)+-+covid-19+growth+and+change&utm_exec=jamu273bwd. robinson, d., perryman, s., & hayday, s. (2004). the drivers of employee engagement. brighton: institute for employment studies. roggeveen, a. l., tsiros, m., & grewal, d. (2011). understanding the co-creation effect: when does collaborating with customers provide a lift to service recovery? journal of the academy of marketing science, 40(6), 771-790. saks, a. m. (2006). antecedents and consequences of employee engagement. journal of managerial psychology, 21 (7), 600-619. schaefers, t., & schamari, j. (2016). service recovery via social media: the social influence effects of virtual presence. journal of service research, 19(2), 192-208. australasian journal of information systems bhimavarapu et al. 2021, vol 25, research on consumer evaluation interactive app-based services: recovery evaluations of mobile applications 18 schoefer, k. (2008). the role of cognition and affect in the formation of customer satisfaction judgements concerning service recovery encounters. journal of consumer behaviour, 7(3), 210-221 seo, e. j., park, j. w., & choi, y. j. (2020). the effect of social media usage characteristics on ewom, trust, and brand equity: focusing on users of airline social media.  sustainability, 12(4), 1-18. shulga, l. v., & busser, j. a. (2020). hospitality employee and customer role in value cocreation: personal, organizational and collaborative outcomes. international journal of hospitality management, 91, 102648. sivadas, e., & jindal, r. p. (2017). alternative measures of satisfaction and word of mouth. journal of services marketing, 31(2),119-130. so, k. k., king, c., sparks, b. a., & wang, y. (2014). the role of customer engagement in building consumer loyalty to tourism brands. journal of travel research, 55(1), 64-78. so, k. k. f., li, x., & kim, h. (2020). a decade of customer engagement research in hospitality and tourism: a systematic review and research agenda. journal of hospitality & tourism research, 44(2), 178-200. srinivasan, s. s., anderson, r., & ponnavolu, k. (2002). customer loyalty in e-commerce: an exploration of its antecedents and consequences. journal of retailing, 78(1), 41-50. srivastava, d., & sharma, r. w. (2017). developing a model for studying the antecedents and effects of word of mouth (wom) and e-wom marketing based on literature review. jindal journal of business research, 6(1), 25-43. srivastava, m., & sivaramakrishnan, s. (2020). the impact of ewom on consumer brand engagement. marketing intelligence & planning, 39(3), 469-484. statista (2019). mobile app downloads worldwide from 2018 to 2024. retrieved may 15, 2020 from https://www.statista.com/statistics/1010716/apple-app-store-google-play-appdownloads-forecast/ sugathan, p., ranjan, k. r., & mulky, a. g. (2017). a typical shifts post-failure: influence of co-creation on attribution and future motivation to co-create. journal of interactive marketing, 38, 64-81. tong, x., & bentler, p. m. (2013). evaluation of a new mean scaled and moment adjusted test statistic for sem. structural equation modeling: a multidisciplinary journal, 20(1), 148-156. trochim, w. m., & donnelly, j. p. (2008). research methods knowledge base. mason, oh: atomic dog/cengage learning. tu, y., neuhofer, b., & viglia, g. (2018). when co-creation pays: stimulating engagement to increase revenues. international journal of contemporary hospitality management, 30(4), 2093-2111. van vaerenbergh, y., hazée, s., & costers, a. (2018). customer participation in service recovery: a meta-analysis. marketing letters, 29(4), 465-483. https://www.statista.com/statistics/1010716/apple-app-store-google-play-app-downloads-forecast/ https://www.statista.com/statistics/1010716/apple-app-store-google-play-app-downloads-forecast/ australasian journal of information systems bhimavarapu et al. 2021, vol 25, research on consumer evaluation interactive app-based services: recovery evaluations of mobile applications 19 van vaerenbergh, y., varga, d., de keyser, a., & orsingher, c. (2019). the service recovery journey: conceptualization, integration, and directions for future research. journal of service research, 22(2), 103–119. voorhis, c. r., & morgan, b. l. (2007). understanding power and rules of thumb for determining sample sizes. tutorials in quantitative methods for psychology, 3(2), 43-50. verkijika, s., & de wet, l. (2019). understanding word-of-mouth (wom) intentions of mobile app users: the role of simplicity and emotions during the first interaction. telematics and informatics, 41, 218-228. vivek, s. d., beatty, s. e., & morgan, r. m. (2012). customer engagement: exploring customer relationships beyond purchase. journal of marketing theory and practice, 20(2), 122-146. xie, x. z., tsai, n. c., xu, s. q., & zhang, b. y. (2019). does customer co-creation value lead to electronic word-of-mouth? an empirical study on the short-video platform industry. the social science journal, 56(3), 401-416. xu, y., tronvoll, b., & edvardsson, b. (2014). recovering service failure through resource integration. the service industries journal, 34(16), 1253-1271. xu, x., liu, w., & gursoy, d. (2019). the impacts of service failure and recovery efforts on airline customers’ emotions and satisfaction. journal of travel research, 58(6), 1034-1051. zhu, z., nakata, c., sivakumar, k., & grewal, d. (2013). fix it or leave it? customer recovery from self-service technology failures. journal of retailing, 89(1), 15-29. appendix: service recovery scenarios firm recovery you have ordered food from a restaurant nearby through your favourite ‘food’ delivery app. upon receiving the food, you have realized that the delivery boy has delivered incomplete order (few items were missing from the order). when you have checked with the delivery boy you were informed that the refund will be provided for the missing items. you have shared the order number and list of items missing through the chat board provided on the mobile app. you have received the refund for the missing items. joint recovery you have ordered food from a restaurant nearby through your favourite ‘food’ delivery app. upon receiving the food, you have realized that the delivery boy has delivered incomplete order (few items were missing from the order). when you have checked with the delivery boy you were informed that the refund will be provided for the missing items. you have shared the order number and list of items missing with the help of support system available on the app. you were provided with a link which had guidelines about how to get refund. the process required you to share the order details and pictures of the items received along with details of missing details. you have followed the instructions provided to initiate the refund process. you have received the refund for the missing items. australasian journal of information systems bhimavarapu et al. 2021, vol 25, research on consumer evaluation interactive app-based services: recovery evaluations of mobile applications 20 firm recovery you have ordered food from a restaurant nearby through your favourite ‘food’ delivery app. upon receiving the food, you have realized that the delivery boy has delivered incomplete order (few items were missing from the order). when you have checked with the delivery boy you were informed that the refund will be provided for the missing items. you have found the helpline number after gathering information form internet and checked for the refund process of food delivery service provider with your efforts. you have received the refund after initiating the refund process. copyright: © 2021 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v25i0.3199 http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 theoretical framework development 2.1 service recovery and recovery satisfaction 2.2 service recovery and cbe 2.3 service recovery and e-word of mouth (e-wom) 2.4 cbe and e-word of mouth 2.5 cbe, recovery satisfaction and e-wom 3 methodology 3.1 sampling and data collection procedure 3.1.1 sample characteristics 3.2 measures 3.3 data analysis 3.3.1 common method bias 3.3.2 preliminary analysis 3.3.3 scenario realism and manipulation check 4 results 5 discussion 5.1 theoretical contributions 5.2 managerial implications 6 limitations and directions for future research 7 conclusion paper title (paper title style) australasian journal of information systems w lim, a lim & phang 2019, vol 23, research article conceptual framework for social media adoption 1 toward a conceptual framework for social media adoption by non-urban communities for non-profit activities: insights from an integration of grand theories of technology acceptance weng marc lim swinburne university of technology lim@wengmarc.com ai ling lim swinburne university of technology cynthia su chen phang swinburne university of technology abstract this paper describes perceptions of social media as a communication tool and source of information for non-profit activities in non-urban communities. it proposes a conceptual framework predicated on the unified theory of acceptance and use of technology and the information systems success model as theoretical lenses and suggests that four key factors influence the intentions of non-urban communities to adopt and use social media for communication and information of non-profit activities: performance expectancy (i.e. perception of the benefits of using non-profits’ social media to enhance performance), effort expectancy (i.e. perception of how easy it will be to use non-profits’ social media), social conditions (i.e. expectation of others in non-urban communities to use non-profits’ social media), and facilitating conditions (i.e. access to resources and technology as well as selfefficacy with respect to use of non-profits’ social media). it also suggests that three key enablers allow non-profits to leverage those key factors: information quality (i.e. quality of information about non-profit activities), system quality (i.e. quality of social media interface and technology used by non-profits), and service quality (i.e. quality of interactions between representatives of non-profits and non-urban communities on social media). further empirical testing to validate the conceptual framework, using a non-urban community sample, and strategic investments in key enablers and determinants of high impact and significance should help non-profits to develop effective social media strategies to widen their reach to and in nonurban communities. keywords: unified theory of acceptance and use of technology; information systems success model; social media; non-profits; non-urban communities 1 introduction in the past, non-profits—i.e. organizations that pursue non-commercial goals (e.g. social advocacy, cause, or welfare, such as human rights and provision of educational and medical services) and use their revenue for these purposes rather than aiming to make a profit— attempted to reach and communicate with their target audience by organizing and publishing via traditional bricks-and-mortar avenues (e.g. carnivals, open days, magazines, newspapers) (drucker, 2011). however, the rise of internet and rapid technological proliferation, including in non-urban areas (i.e. semi-urban or semi-rural and rural areas) (parker, 2015), has led many australasian journal of information systems w lim, a lim & phang 2019, vol 23, research article conceptual framework for social media adoption 2 non-profits to consider using social media to connect with people and improve their reach (syrdal & briggs, 2016). given that many people in non-urban communities are digital immigrants (i.e. born or brought up before the widespread use of the internet and digital technologies) rather than digital natives (as is the case in urban areas; boonaert & vettenburg, 2011), it is essential for non-profits targeting non-urban communities to have a good understanding of how people in those communities expect to use social media for communication and information relating to non-profit activities. simply replicating strategies developed for use in urban areas may lead to potential resistance to using social media for non-profit activities amongst non-urban communities. in other words, it may not be possible to achieve critical mass of social media for non-profit activities in these areas if incorrect assumptions are made about social expectations in relation to social media for non-profit activities. this paper aims to shed new light on this topic through a conceptual framework predicated on the unified theory of acceptance and use of technology that highlights the factors that influence adoption of social media for communication and information relating to non-profit activities by non-urban communities (venkatesh et al., 2003). it also uses the information systems success model as a theoretical framework for explaining the enablers or facilitators of those factors (delone & mclean, 1992, 2003). it is hoped that this conceptual framework, which to the best of knowledge of the authors is the first to conceptually and contextually integrate the two grand theories of technology acceptance to better explain how exactly a particular technology (or technologically-mediated systems, and in this case, non-profits’ social media) can be designed to create and satisfy expectancies and conditions required for adoption, will help non-profits to develop effective social media strategies that widen their reach to and in non-urban communities and thus dramatically increase their ability to initiate, innovate, and deliver non-profit activities. in doing so, this paper also answers the call by lim (2018a, 2018b) in the australasian journal of information systems to revisit and reposition existing theories in information systems and technology as impetuses for greater disciplinary and managerial progress. 2 social media social media is a term used to refer to technologically-mediated platforms (e.g. facebook, instagram, twitter) that enable and facilitate the creation and sharing of information amongst people who are using technological devices connected to digital networks (lim, 2016; lim et al., 2012; soroka et al., in press). more specifically, people on social media can create and share—both individually and collectively—information such as expressions (e.g. emoticons, likes) and comments (e.g. facts, ideas, narratives, opinions) (de la peña & quintanilla, 2015; kietzmann et al., 2011; sargeant & shang, 2017). the technological devices people use to access social media, such as desktops, laptops, smartphones, and tablets, are usually connected to the internet (müller et al., 2012; santosa & wigdor, 2013). more important, social media has increased the extent to which people are connected to each other and able to receive the latest news and events (lim et al., 2012; liu et al., 2016; sargeant & shang, 2017). given the critical mass and power in the form of connectivity and reach characterizing social media, many organizations are now pursuing strategies designed to give them a strong presence on social media to allow them to reach their intended target audience more effectively and communicate with them about products and initiatives. however, most research on social media are concentrated on commercial organizations (agnihotri et al., 2016; australasian journal of information systems w lim, a lim & phang 2019, vol 23, research article conceptual framework for social media adoption 3 kaplan & haenlein, 2010), especially those targeted in urban areas and communities (lamberton & stephen, 2016; xiang & gretzel, 2010), and thus, there is limited understanding of how other types of organizations and communities, such as non-profits and non-urban communities, might benefit from using social media, if the right approach was adopted. the social media strategies that prove effective for non-profits and non-urban communities are likely to be different from those used by commercial organizations and urban communities due to the inherent differences in focus (e.g. profit versus non-profit goals) and user characteristics (e.g. digital natives versus digital immigrants). thus, this study seeks to advance understanding in this area, specifically by shedding new light on how the use of social media for communication and information relating to non-profit activities is viewed by people in non-urban communities. 3 technology acceptance technology acceptance is a concept that encapsulates the perceptions and intentions people have with regard to adopting a particular technology or technologically-mediated product (e.g. goods, services, systems) (davis, 1989; delone & mclean, 1992; lim & ting, 2012; venkatesh et al., 2003). there are many theories of technology acceptance, but this paper focuses on two of the most recent grand theories, namely the unified theory of acceptance and use of technology and the information systems success model, taking them as the starting point for a conceptual framework describing the perceptions and intentions of people in nonurban communities in relation to the use of social media for communication and information relating to non-profit activities. the unified theory of acceptance and use of technology is a popular grand theory of technology acceptance that was developed by mapping and integrating eight prominent information systems and technology theories (venkatesh et al., 2003), namely the theory of reasoned action (fishbein & ajzen, 1975), the technology acceptance model (davis, 1989), the motivational model (davis et al., 1992), the theory of planned behaviour (ajzen, 1991), a combined theory of planned behaviour and technology acceptance model (taylor & todd, 1995), a model of personal computer use (thompson et al., 1991), the diffusion of innovations theory (moor & benbasat, 1991), and the social cognitive theory (compeau & higins, 1995). it contends that user acceptance and use of technology can be explained in terms of four factors, namely performance expectancy, effort expectancy, social conditions, and facilitating conditions (venkatesh et al., 2003). the information systems success model is another popular grand theory of technology acceptance; it was developed from existing research on communications, information influence, and management information systems and designed to provide a multidimensional view of information systems success (delone & mclean, 1992, 2003; mason, 1978; shannon & weaver, 1949). it proposes, through a temporal, process model, that information systems have several features that vary in quality, namely information quality, system quality, and service quality (delone & mclean, 1992, 2003). in turn, these features influence the extent to which an information system elicits the desired responses from target users (e.g. adoption and usage intentions). more important, this paper contends that to achieve a good understanding of how people in non-urban communities view the use of social media as a source for communication and information relating to non-profit activities, it is essential to integrate the unified theory of australasian journal of information systems w lim, a lim & phang 2019, vol 23, research article conceptual framework for social media adoption 4 acceptance and use of technology and the information systems success model. more specifically, the unified theory of acceptance and use of technology provides insights into the expectations (i.e. performance and effort) and conditions (i.e. social and facilitating) that are required to encourage non-urban communities to adopt and use social media for non-profit activities, whilst the information systems success model provides insights into how nonprofits’ should develop a presence on social media and what form their social media content should take if they are to fulfil the conditions needed for non-urban communities to accept the use of social media for communication and information relating to non-profit activities. the integration, in turn, offers a unified view of the key enablers (i.e. information systems success model) and determinants (i.e. unified theory of acceptance and use of technology) of intention to adopt and use non-profits’ social media by non-urban communities, which otherwise would not have been possible; that is to say, without the integration, we would be facing a situation comparable to that of a missing puzzle piece (e.g. knowing the enablers, but not the determinants, and vice versa). 4 digital divide digital divide—i.e. the technology adoption gap between entities of varying socioeconomic backgrounds and interests with respect to the access and usage of information and communication technologies and the internet for a wide variety of activities (oecd, 2001)— is a continuing issue for many developing (e.g. china [fong, 2009] and malaysia [halili & sulaiman, 2018]) and developed countries (e.g. the us [burrell, 2018] and the uk [philip et al., 2017]). most often, digital divide exists among (i) urban and non-urban communities, which occurs as a result of inequities in internet access, the level of technical connection and social support, and the ability to search, evaluate, and act on information facilitated by information and communication technologies, among others (dimaggio et al., 2010; salemink et al., 2017), and (ii) for-profit and non-profit organizations, which occurs as a result of different socioeconomic goals (e.g. corporate usage [e.g. across business units and departments] by and commercial goals [e.g. to make a sale] of for-profits versus social impact goals [e.g. community empowerment, emergency preparedness] of non-profits) (aichner & jacob, 2015; durkin et al., 2013; campbell et al., 2014). in order for academics to contribute to the agenda of overcoming the digital divide between urban and non-urban communities and between for-profit and non-profit organizations, this paper contends that the integration and contextualization of grand theories of technology acceptance is especially important and must be considered. specifically, the integration of grand theories will overcome the criticism of theoretical simplicity, which occurs when each theory is taken individually, whereas the contextualization of grand theories will allow for greater practical value to emerge from theorization in academic research (lim, 2018a, 2018b). in doing so, scholars should be able to precisely offer nuance recommendations for designing technologically-mediated systems (i.e. non-profits’ social media) that resonate with intended target communities (i.e. non-urban communities), and thus, satisfy expectancies and conditions required for technology adoption. 5 conceptual framework the conceptual framework describes how people in non-urban communities view the use of social media for communication and information relating to non-profit activities (see fig. 1). it has three main parts: key enablers of social media quality for non-profits, key determinants australasian journal of information systems w lim, a lim & phang 2019, vol 23, research article conceptual framework for social media adoption 5 of adoption and use of non-profits’ social media, and key predictors of non-urban communities’ adoption and usage of non-profits’ social media. it is important to note that whilst the concepts herein are taken from grand theories of technology acceptance (i.e. unified theory of acceptance and use of technology and information systems success model), they have been adapted to make them applicable to non-profits that wish to use social media to extend their reach to and in non-urban communities. 5.1 key enablers the first part of the conceptual framework focuses on the key enablers of social media quality for non-profits. the information systems success model is used to identify and describe three main areas that non-profits should consider when developing their social media presence if they wish to communicate effectively with non-urban communities, namely information quality, system quality, and service quality (delone & mclean, 1992, 2003). in the context of non-profits’ social media, ‘information quality’ refers to the quality of information about nonprofit activities (e.g. its clarity, trustworthiness, and relevance to target non-urban communities), ‘system quality’ refers to the quality of their social media interface and technology (e.g. the social media platforms chosen by non-profits should be compatible with those that are used by target non-urban communities; social media plug-ins should run smoothly and robustly on the kinds of devices used by target non-urban communities), and ‘service quality’ refers to the quality of social media interactions between representatives of non-profits and people in non-urban communities (e.g. communications and responses should be in local dialects or languages and should display empathy and sensitivity to local culture and practices). 5.2 key determinants the second part of the conceptual framework explains the key determinants of non-urban communities’ adoption and use of social media for communication and information relating to non-profit activities. it uses the unified theory of acceptance and use of technology to identify and describe four key factors that people in non-urban communities are likely to consider when deciding whether or not to adopt and use non-profits’ social media: performance expectancy, effort expectancy, social conditions, and facilitating conditions (venkatesh et al., 2003). in the context of non-profits’ social media, ‘performance expectancy’ refers to the perception of how beneficial it will be to use non-profits’ social media to enhance performance (e.g. solving important problems for target non-urban communities), ‘effort expectancy’ refers to how easy it will be to use non-profits’ social media (i.e. how easy it is for people in non-urban communities to become skilful at navigating through and interacting with the non-profits’ social media content), ‘social conditions’ refers to expectation of others in non-urban communities to use non-profits’ social media (e.g. buy-in from community and opinion leaders in non-urban communities), and ‘facilitating conditions’ refers to the accessibility of resources and technology as well as self-efficacy with respect to use of nonprofits’ social media (i.e. availability of computers, smartphones, and tablets in non-urban communities and the extent of their technological know-how and confidence). 5.3 behavioural consequences the third part of the conceptual framework encapsulates the behavioural consequences, for non-profits, of strategic investment in developing effective social media content and a strong social media presence in order to extent their reach to and in non-urban communities. australasian journal of information systems w lim, a lim & phang 2019, vol 23, research article conceptual framework for social media adoption 6 however, it should be noted that though behavioural outcomes can only be accurately measured when target communities actually adopt and use non-profits’ social media, it can be predicted from measures of intention to use or recommend non-profits’ social media. this approach to predicting the technology acceptance behaviour of target users is consistent with the grand theories of technology acceptance on which the conceptual framework is predicated (delone & mclean, 1992, 2003; venkatesh et al., 2003). 5.4 propositions more important, the conceptual framework is built on two important propositions that establish the linkages between its three main parts: p1. key enablers of non-profits’ social media quality (i.e. information quality, system quality, and service quality) are positively related to the key determinants of adoption and usage of non-profits’ social media (i.e. performance expectancy, effort expectancy, social conditions, and facilitating conditions) by non-urban communities. p2. key determinants of adoption and usage of non-profits’ social media (i.e. performance expectancy, effort expectancy, social conditions, and facilitating conditions) are positively related to intention to adopt and use non-profits’ social media by non-urban communities. in a nutshell, the conceptual framework posits that non-profits whose social media content and presence are perceived to be of high quality with respect to information, system, and service are likely to create and satisfy, and thus, positively shape more favourable perceptions about the performance, effort required, communal buy-in, and fit in the technological and efficacy conditions necessary to participate and use non-profits’ social media, and thus encourage greater intentions to adopt and use non-profits’ social media for communication and information about non-profit activities. when these key enablers and factors are present, this overarching encapsulation of conceptual reasoning is in line with the positivistic linearity in spill-over impacts between antecedents and consequences of technology acceptance as conceptualized in the unified theory of acceptance and use of technology and the information systems success model (delone & mclean, 1992, 2003; venkatesh et al., 2003), as well as empirical investigations of grand theories in technology acceptance (e.g. bhattacherjee et al., 2008; delone & mclean, 2004; guo & barnes, 2012; peter & mclean, 2009; seddon & kiew, 1996). australasian journal of information systems w lim, a lim & phang 2019, vol 23, research article conceptual framework for social media adoption 7 note: p = proposition; issm = information systems success model; utaut = unified theory of acceptance and use of technology. fig. 1 key enablers and determinants of intention to adopt and use non-profits’ social media by non-urban communities 6 conclusion in short, the conceptual framework described in this paper posits that in non-urban communities, intention to adopt and use social media as a source of communication and information relating to non-profit activities is influenced by four key factors: performance expectancy, effort expectancy, social conditions, and facilitating conditions. furthermore, it posits that three key enablers allow non-profits to leverage those key factors: information quality, system quality, and service quality (delone & mclean, 1992, 2003; venkatesh et al., 2003). notwithstanding the conceptual contributions herein, further empirical testing, using a non-urban community sample (e.g. dayaks in sarawak, malaysia and torres strait islanders in queensland, australia), of the propositions and relationships underpinning the conceptual australasian journal of information systems w lim, a lim & phang 2019, vol 23, research article conceptual framework for social media adoption 8 framework is required. following that, non-profits should be able to use the information about the impact and significance of key enablers and factors to inform development of effective social media strategies that widens their reach to and in non-urban communities. references agnihotri, r., dingus, r., hu, m. y., & krush, m. t. (2016). social media: influencing customer satisfaction in b2b sales. industrial marketing management, 53, 172-180. doi:10.1016/j.indmarman.2015.09.003 ahmed, k. m., & campbell, j. (2015). citizen perceptions of e-government in the kurdistan region of iraq. australasian journal of information systems, 19. doi:10.3127/ajis.v19i0.1201 aichner, t., & jacob, f. (2015). measuring the degree of corporate social media use. international journal of market research, 57 (2), 257–276. doi:10.2501/fijmr-2015-018 ajzen, i. (1991). the theory of planned behavior. organizational behavior and human decision processes, 50, 179-211. doi:10.1016/0749-5978(91)90020-t bhattacherjee, a., perols, j., & sanford, c. (2008). information technology continuance: a theoretic extension and empirical test. journal of computer information systems, 49 (1), 1726. doi:10.1080/08874417.2008.11645302 boonaert, t., & vettenburg, n. (2011). young people's internet use: divided or diversified? childhood, 18, 54-66. doi:10.1177/0907568210367524 burrell, j. (2018). thinking relationally about digital inequality in rural regions of the u.s. first monday, 23 (6). doi:10.5210/fm.v23i6.8376 campbell, d. a., lambright, k. t., & wells, c. j. (2014). looking for friends, fans, and followers? social media use in public and nonprofit human services. public administration review, 74 (5), 655-663. doi:10.1111/puar.12261 compeau, d. r., & higgins, c. a. (1995). application of social cognitive theory to training for computer skills. information systems research, 6, 118-143. doi:10.1287/isre.6.2.118 davis, f. d. (1989). perceived usefulness, perceived ease of use and user acceptance of information technology. mis quarterly, 13, 319-339. doi:10.2307/249008 davis, f., bagozzi, r., & warshaw, p. (1992). extrinsic and intrinsic motivation to use computers in the workplace. journal of applied social psychology, 22, 1111-1132. doi:10.1111/j.1559-1816.1992.tb00945.x de la pe-a, a., & quintanilla, c. (2015). share, like and achieve: the power of facebook to reach health‐related goals. international journal of consumer studies, 39, 495-505. doi:10.1111/ijcs.12224 delone, w. h., & mclean, e. r. (1992). information systems success: the quest for the dependent variable. information systems research, 3, 60-95. doi:10.1287/isre.3.1.60 delone, w. h., & mclean, e. r. (2003). the delone and mclean model of information systems success: a ten year update. journal of management information systems, 19 (4), 930. doi:10.1080/07421222.2003.11045748 australasian journal of information systems w lim, a lim & phang 2019, vol 23, research article conceptual framework for social media adoption 9 delone, w. h., & mclean, e. r. (2004). measuring e-commerce success: applying the delone & mclean information systems success model. international journal of electronic commerce, 9, 31-47. dimaggio, p., hargittai, e., neuman, w.r., robinson, j.p. (2010). social implications of the internet. annual review of sociology, 27, 307-336. doi: 10.1146/annurev.soc.27.1.307 drucker, p. f. (2011). managing the non-profit organization: practices and principles. new york, ny: routledge. durkin, m., mcgowan, p., & mckeown, n. (2013). exploring social media adoption in small to medium-sized enterprises in ireland. journal of small business and enterprise development, 20 (4), 716-734. doi:10.1108/jsbed-08-2012-0094 fishbein m., & ajzen, i. (1975). belief, attitude, intention, and behaviour: an introduction to theory and research. reading, ma: addison-wesley. fong, m. w. (2009), digital divide between urban and rural regions in china. electronic journal of information systems in developing countries, 36 (6), 1-12. doi:10.1002/j.16814835.2009.tb00253.x guo, y., & barnes, s. j. (2012). explaining purchasing behavior within world of warcraft. journal of computer information systems, 52 (3), 18-30. halili, s. h., & sulaiman, h. (2018). factors influencing the rural students’ acceptance of using ict for educational purposes. kasetsart journal of social sciences. doi:10.1016/j.kjss.2017.12.022 kietzmann, j. h., hermkens, k., mccarthy, i. p., & silvestre, b. s. (2011). social media? get serious! understanding the functional building blocks of social media. business horizons, 54, 241-251. doi:10.1016/j.bushor.2011.01.005 lamberton, c., & stephen, a. t. (2016). a thematic exploration of digital, social media, and mobile marketing: research evolution from 2000 to 2015 and an agenda for future inquiry. journal of marketing, 80 (6), 146-172. doi:10.1509/jm.15.0415 lim, w. m. (2018a). dialectic antidotes to critics of the technology acceptance model: conceptual, methodological, and replication treatments for behavioural modelling in technology-mediated environments. australasian journal of information systems, 22. doi:10.3127/ajis.v22i0.1651 lim, w. m. (2018b). revisiting concepts and theories in information systems and technology. australasian journal of information systems, 22. doi: 10.3127/ajis.v22i0.1733 lim, w. m. (2016). social media in medical and health care: opportunities and challenges. marketing intelligence & planning, 34, 964-976. doi:10.1108/mip-06-2015-0120 lim, w. m., & ting, d. h. (2012). e-shopping: an analysis of the technology acceptance model. modern applied science, 6 (4), 49-62. doi:10.5539/mas.v6n4p49 lim, w. m., ting, d. h., puspitasari, m., prasetya, i., & gunadi, s. (2012). overexposure in social networking sites. journal of research for consumers, 22, 25-40. liu, b. f., fraustino, j. d., & jin, y. (2016). social media use during disasters: how information form and source influence intended behavioral responses. communication research, 43, 626-646. doi:10.1177/0093650214565917 australasian journal of information systems w lim, a lim & phang 2019, vol 23, research article conceptual framework for social media adoption 10 mason, r. o. (1978). measuring information output: a communication systems approach. information & management, 1, 219-234. doi:10.1016/0378-7206(78)90028-9 müller, h., gove, j., & webb, j. (2012). understanding tablet use: a multi-method exploration. in proceedings of the 14th international conference on human-computer interaction with mobile devices and services, 1-10. new york, ny: acm. doi:10.1145/2371574.2371576 moore, g. c., & benbasat, i. (1991). development of an instrument to measure the perception of adopting an information technology innovation. information systems research, 2, 192223. doi:10.1287/isre.2.3.192 oecd (organisation for economic co-operation and development). (2001). understanding the digital divide. retrieved october 30, 2018, from http://www.oecd.org/dataoecd/38/57/1888451.pdf parker, s. (2015). urban theory and the urban experience: encountering the city. new york, ny: routledge. petter, s., & mclean, e. r. (2009). a meta-analytic assessment of the delone and mclean is success model: an examination of is success at the individual level. information & management, 46, 159-166.doi:10.1016/j.im.2008.12.006 philip, l., cottrill, c., farrington, j., williams, f., & ashmore, f. (2017). the digital divide: patterns, policy and scenarios for connecting the ‘final few’ in rural communities across great britain. journal of rural studies, 54, 386-398. doi:10.1016/j.jrurstud.2016.12.002 salemink, k., strijker, d., & bosworth, g. (2017). rural development in the digital age: a systematic literature review on unequal ict availability, adoption, and use in rural areas. journal of rural studies, 54, 360-371. doi:10.1016/j.jrurstud.2015.09.001 santosa, s., & wigdor, d. (2013). a field study of multi-device workflows in distributed workspaces. in proceedings of the 2013 acm international joint conference on pervasive and ubiquitous computing, 63-72. new york, ny. doi:10.1145/2493432.2493476 sargeant, a., & shang, j. (2017). social media – fundraising principles and practice (2nd ed.). new york, ny: john wiley & sons. doi:10.1002/9781119228974 seddon, p., & kiew, m. y. (1996). a partial test and development of delone and mclean's model of is success. australasian journal of information systems, 4 (1), 90-109. doi:10.3127/ajis.v4i1.379 shannon, c. e., & weaver, w. (1949). the mathematical theory of communication. urbana, il: university of illinois press. soroka, s., daku, m., hiaeshutter-rice, d., guggenheim, l., & pasek, j. (in press). negativity and positivity biases in economic news coverage: traditional versus social media. communication research. doi:10.1177/0093650217725870 syrdal, h. a., & briggs, e. (2016). developing a positive organizational image: the role of engagement with social media content. in fowler, j. g., & weiser, j. (eds.), 2016 society for marketing advances proceedings, 74-83. taylor, s., & todd, p. (1995). understanding information technology usage: a test of competing model. information systems research, 6 (2), 144-176. doi:10.1287/isre.6.2.144 australasian journal of information systems w lim, a lim & phang 2019, vol 23, research article conceptual framework for social media adoption 11 thompson, r. l., higgins, c. a., & howell, j. m. (1991). personal computing: toward a conceptual model of utilization, mis quarterly, 15 (1), 124-143. doi:10.2307/249443 venkatesh, v., morris, m. g., davis, g. b., & davis, f. d. (2003). user acceptance of information technology: toward a unified view. mis quarterly, 27 (3), 425-478. doi:10.2307/30036540 xiang, z., & gretzel, u. (2010). role of social media in online travel information search. tourism management, 31 (2), 179-188. doi:10.1016/j.tourman.2009.02.016 copyright: © 2019 w lim, a lim & phang. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems warren 2020, vol 24, selected papers from the 2019 aice conference fake news in australian 2019 election 1 fake news case study during the australian 2019 general election matthew warren rmit university centre for cyber security research and innovation rmit university, melbourne, australia. matthew.warren2@rmit.edu.au abstract social media is used by all aspects of society from citizens to businesses, but it also now used by political parties. political parties use social media to engage with voters as a method of attract new voters or reinforcing the views of political parties’ current supporters. an important consideration is the ethical conduct of political parties and politicians in how they use social media. it is now recognized that social media can also have negative aspects seen by the introduction of fake news. these negative aspects of social media are often overlooked and have not been explored from a research perspective. this paper looks at the australian 2019 general election and discusses a major fake news example that occurred during that election. the paper will also describe the different types of social media data was collected during the study and also present the analysis of the data collected as well discussing the research findings including the ethical issues. keywords: australia; election; 2019; fake news; social media. 1 introduction social media has been defined as "a group of internet based applications that build on the ideological and technological foundations of web 2.0, and that allow the creation and exchange of user-generated content” (kaplan & haenlein, 2010). social media impacts all aspects of society from citizens to businesses but also political parties. social media offers real challenges for political parties as there is an increased acceptance of social media by voters. it also means that political discussions are conducted in a public forum and voters have the ability to contribute to the discussion. this means that political parties may have little control over the discussion or even lose control of the discussion that occur online. the means that social media has real challenges for political parties. so why is social media so important for political parties. it is important because of the large and rapidly increasing number of users (voters) using social media and their increased online expectations. it is also important because users (voters) have expectation around the use of technology to engage with a variety of political organisations and individuals, social media has become the accepted standard due to its of widespread use and ease of use, there is the expectation that users (voters) can engage with political parties via this medium. from a political party perspective, social media provides a cost-effective medium to reach-out to large number of users (voters), it provides a rich two-way engagement with users (voters) and by its nature creates interaction. social media also offers business benefits for political parties, by using social media they could engage with many more users (voters) rather than using traditional media, so it means their investment in social media could give greater australasian journal of information systems warren 2020, vol 24, selected papers from the 2019 aice conference fake news in australian 2019 election 2 returns. another key aspect of the use of social media by political parties is that it allows them to influence voters and the way they could vote; this is also known as information operations. information operations also known as influence operations, includes the collection of tactical information about an adversary as well as the dissemination of propaganda in pursuit of a competitive advantage over an opponent (waltzman, 2017). as thompson (2019) states political ethics is the practice of making moral judgments about political action it is divided into two branches: the ethics of process, which focuses on public officials, the methods they use, and the institutions in which they act; and the ethics of policy, which concentrates on judgments about the policies and laws governments make. a key issue for individual politicians and political parties is whether they also focus on making judgements in an ethical manner and how they interact and use of social media as part of their engagement strategy. on the 11th april 2019 the australian government called for a national election to elect the 46th parliament of australia with an election date set for the 18th may 2019 (abc, 2019). but during the 2019 australian election a big unknown was whether there would be examples of fake news and what impact these would cause have upon the australian election and its outcome. 2 literature review the australian government was concerned about the impact of political interference in the 2019 australian general election. prior to the july 2018 federal by-elections held across four states in australia, the australian government established a multi-agency body, the electoral integrity assurance taskforce, to address risks to the integrity of the electoral system particularly in relation to cyber interference and to protect australia (buckmaster and wils, 2019). propaganda is information or ideas that are spread by an organised group or government to influence people’s opinions, especially by not giving all the facts or by secretly emphasizing only one way of looking at the facts (sample et al, 2018). historically propaganda and deception has had a long history associated with warfare (taylor 2013); however, in the past, the distribution or reach of propaganda was limited (crilley 2001) by the media (e.g., personto-person, newspapers and other printed materials, radio, and television), therefore it was less effective due to speed and cost. but with the development of social media we have seen the ability to share to information at limited costs to a global audience in real time, this has resulted in the development of the concept of fake news. there is no clear definition of what fake news is, but a recent uk parliamentary interim report on fake news (uk parliament, 2018) defined some of the key criteria that could be used to describe fake news, the criteria defined was: • fabricated content: completely false content; • manipulated content: distortion of genuine information or imagery, for example a headline that is made more sensationalist, often popularised by ‘clickbait’; • imposter content: impersonation of genuine sources, for example by using the branding of an established news agency; • misleading content: misleading use of information, for example by presenting comment as fact; australasian journal of information systems warren 2020, vol 24, selected papers from the 2019 aice conference fake news in australian 2019 election 3 • false context of connection: factually accurate content that is shared with false contextual information, for example when a headline of an article does not reflect the content; • satire and parody: presenting humorous but false stores as if they are true. although not usually categorised as fake news, this may unintentionally fool readers. the final uk parliamentary report on fake news (uk parliament, 2019) came up with some different findings. the final report proposed that the uk government accepted the view that the term ‘fake news’ is misleading, and the following terms should be used instead ‘disinformation’ and ‘misinformation’. the final uk parliamentary report (uk parliament, 2019) defined the key terms as being, disinformation as the deliberate creation and sharing of false and/or manipulated information that is intended to deceive and mislead audiences, either for the purposes of causing harm, or for political, personal or financial gain and ‘misinformation’ refers to the inadvertent sharing of false information. some of the other key terms in relation to the negative use of social media include: • information operations (now often called cyber operations) cyber operations are the means possessing the resources, skills, knowledge, operational concepts and procedures to be able to have an effect in cyberspace. (aspi, 2018); • influence operations the collection of tactical information about an adversary as well as the dissemination of propaganda in pursuit of a competitive advantage over an opponent. (waltzman, 2017); • fake news the malicious publication, dissemination and reproduction, by whatever means, of false news and documents which have been fabricated or falsified or mendaciously attributed to third parties. (french government, 2018). 3 research question the author had identified one key research question for the study. the research question was “was there any occurrence of fake news during the australian general election in 2019?”. 4 research the aim of the research was to monitor social media during the australian election to determine instances of fake news and the sources of those fake news stories. during the election there was one major incident identified by the author regarding fake news. the incident related to death tax and the story originated with a media release prior to the 2019 australian general election. on the 24th january, 2019 the australian treasurer, josh frydenberg released a media statement online (frydeberg, 2019) (see figure 1), stating “facing growing pressure over labor’s disastrous housing and retirees taxes, bill shorten today sought to deflect attention by flippantly remarking that the next thing they say will be “that labor wants to introduce death taxes”. the accusation was that labor, and the greens have signed a secret agreement to introduce a 40 per cent "death tax" (sydney morning herald, 2019a). australasian journal of information systems warren 2020, vol 24, selected papers from the 2019 aice conference fake news in australian 2019 election 4 figure 1. josh frydenberg online media statement (24th january 2019). the data collection for the research took place between 22nd april and 16th may 2019 and took the form of monitoring stories regarding death tax on social media platforms either in the form of collecting stories or searching hashtags such as #deathtax. the posts were categories as being videos posted, articles or links shared via social media or the sharing of graphical pictures known as memes. the additional information that was collected was the views of video files, the number of likes per post, the comments that were made about the posts and the number of times the posts were shared. the data collected is broken down into the following samples. posts views likes comments shares james mcgarth (liberal national party (lnp)) videos 1 10934 67 18 39 george christensen (lnp) article 17 not applicable (na) 3600 1444 3848 videos 1 90730 44 28 17 memes 5 na 958 362 782 alex hawke (liberal party (lp)) videos 1 1400 79 40 24 articles 1 na 9 5 9 jane hume (lp) videos 1 25000 268 78 189 matthew fraser (the nationals party) articles 1 na 55 23 26 videos 1 1500 54 18 22 meme 1 na 46 11 37 posts views likes comments shares total 30 129564 5180 2027 4993 table 1. death tax posts by individual politicians within the coalition parties. australasian journal of information systems warren 2020, vol 24, selected papers from the 2019 aice conference fake news in australian 2019 election 5 posts views likes comments shares liberal party videos 3 221000 2371 1062 960 articles 2 na 3400 2031.00 2700.00 lnp (qld) memes 2 na 1200 749 620 videos 2 206000 376 404 275 country liberal party (nt) videos 4 265000 48 2 39 posts views likes comments shares total 13 692000 7395 4248 4594 table 2. death tax posts by coalition parties. posts views likes comments shares pauline hanson one nation videos 1 187000 4100 2500 4400 one nation party articles 1 na 262 103 262 great australian party articles 1 na 219 141 496 posts views likes comments shares total 3 187000 4581 2744 5158 table 3. death tax posts by minor political parties. posts views likes comments shares 46 1008564 17156 9019 14745 ratio per post 67238 477 251 410 table 4. total of death tax posts and impact. 5 discussion in terms of the death tax posts, the individual posts were by australian coalition politicians (members of the liberal party, national party or liberal / national party (queensland), country liberal party (northern territory)) (see table 1), the sample took the form of individual posts by coalition politicians, apart from george christensen (liberal national party, queensland) who posted 17 items during the data collection period and had the greatest impact of the coalition politicians. the coalition parties posted 13 posts (see table 2) about the death tax and these took the form of videos and memes, the videos were very popular with around 692000 views of these videos. the death tax stories was also posted by minor parties (one nation and the great australian party) and the leader of the one nation party (pauline hanson) (see table 3). the video by pauline hanson took the form of interviewing voters at a tasmania country fair about the death tax item and was viewed by 187000 people (see figure 2) and had the most views of all the videos posted on youtube by the minor political parties. australasian journal of information systems warren 2020, vol 24, selected papers from the 2019 aice conference fake news in australian 2019 election 6 figure 2. pauline hanson interviews voters about labor’s death tax plans (note: the youtube clip was removed after the general election finished). in terms of the summaries of the different posts (see table 4), the videos related to the death tax was viewed by over a million people and average view of each video (of the 15 videos) was 67238 views. in terms of the death tax posts, on average each post was liked 477 per post, an average of 251 comments per post were made and on average each post was shared 410 times. from the data collected and the assessment made, the death tax stories had a big impact upon social media users, in terms of their views of videos, liking of posts, comments made and sharing of posts especially with over one million views of the videos posted on facebook or youtube. how did the labor party respond to the situation, “the opposition leader bill shorten has hit out at the coalition for a case of "fake news" being spread on social media, saying his opponents should be ashamed for mounting a scare campaign about a tax on inheritances” (sydney morning herald, 2019a). the labor party asked facebook, to remove the death tax content from the facebook platform and the posts should be treated as fake news (sydney morning herald, 2019b), facebook never removed the posts. in addition, the labor party ran online videos accusing the coalition of introducing their own death taxes and also sharing information about the death tax situation via their website (labor, 2019), this is shown in figure 3. australasian journal of information systems warren 2020, vol 24, selected papers from the 2019 aice conference fake news in australian 2019 election 7 figure 3. labor advertising against the liberals death tax. using the french government (french government, 2018) definition of fake news, the death tax story and information was maliciously published and disseminated by various methods on social media and the information had been fabricated and falsely attributed to third parties, in this case being the labor party. but by using the more detailed united kingdom parliament (uk government, 2018) definition of fake news, as shown in table 5 we can gain a greater understanding of how the fake news was created and evaluated against the six fake news criteria items. fake news criteria example discussion fabricated content. the presentation of an untrue fact as being a true fact, in this case the fake news example was that the labor party was proposing a death tax. australasian journal of information systems warren 2020, vol 24, selected papers from the 2019 aice conference fake news in australian 2019 election 8 fake news criteria example discussion manipulated content. several items were posted distorting the original genuine news item, in this example a headline from the the daily telegraph newspaper. imposter content. items were posted implying that they were official news items from the australian newspaper, and they were not. the posts had been created to give the impression that they were genuine news item from respected media outlets. misleading content. taking an historical quote / comment of a labor senate candidate in 2017 and implying that it was official labor party policy in 2019. australasian journal of information systems warren 2020, vol 24, selected papers from the 2019 aice conference fake news in australian 2019 election 9 fake news criteria example discussion false context of connection. using a historical interview of the former labor party leader (2013) and implying that it related to the labor party leader of 2019. satire and parody. a number of graphical memes were generated and distributed about the death tax. they also included memes linked to the games of thrones and marvel film series. table 5. evaluation of australian fake news case using the uk government criteria. australasian journal of information systems warren 2020, vol 24, selected papers from the 2019 aice conference fake news in australian 2019 election 10 using the united kingdom parliament definition of fake news, all six of the fake news criteria were shown to have been meet. in terms of the original research question presented at the start of the paper “was there any occurrence of fake news during the australian general election in 2019”, the paper has demonstrated that fake news did occur during the australian 2019 general election. 6 conclusion as we have seen social media impacts all aspects of society from citizens to businesses but also political parties. we now see that all poltical parties use social media to engage with voters and this use of social media becomes key at election times to engage with voters and potential voters. the paper has limitations, in terms of the social media data sample that was collected during the given time period and the analysis of the data sample that was collected . but the paper has made a major contrunition is terms of identifiying that there was a major example of fake news during the australian general election in 2019. the paper shows that a fake news incident relating to the death tax had occurred during the 2019 election, that associated video clips had been watched over a million times, that associated posts had over seventeen thousand likes, nine thousand comments had been made about the related posts and the fake news clips had been reshared over fourteen thousand times with other social media users. as discussed at the start of the paper, a key issue for individual politicians and political parties is whether they focus on making judgements and whether they act in an ethical manner. as identified in the paper during the 2019 australian general election there were a number of political parties and individual politicians that did not act in an ethical manner. when the research project started, the author was under the assumption that any fake news generated during the australian election would be from overseas entities trying to influence the outcome of the australian election results and not australian political parties and politicians generating fake news. references abc (australian broadcasting corporation). (2019). federal election 2019: prime minister scott morrison sets may 18 election date, https://www.abc.net.au/news/2019-04-11/primeminister-scott-morrison-calls-federal-electionmay-18/10991614, accessed 10/6/19. aspi (australian strategic policy institute). (2018). defining offensive cyber capabilities, url: https://www.aspi.org.au/report/definingoffensive-cyber-capabilities, accessed 10/6/19. buckmaster, l. and wils, t. (2019), responding to fake news, australian parliament briefing paper, canberra, australia. crilley, k. (2001). “information warfare: new battle fields terrorists, propaganda and the internet”, in aslib proceedings vol. 53, no. 7, pp. 250–264. mcb up ltd. french government. (2018). fake news: a bill to combat the manipulation of information https://www.gouvernement.fr/en/fake-news-a-bill-to-combat-the-manipulation-ofinformation, accessed 10/6/19. australasian journal of information systems warren 2020, vol 24, selected papers from the 2019 aice conference fake news in australian 2019 election 11 frydeberg, j. (2019). media release death taxes – you don’t say, bill!, url: https://joshfrydenberg.com.au/wp-content/uploads/2019/01/treasurer-media-releasedeathtaxes-you-dont-say-bill.pdf, accessed 10/6/19. kaplan, a. m., & haenlein, m. (2010). users of the world, unite! the challenges and opportunities of social media. business horizons, 53(1), 59–68. doi:10.1016/j.bushor.2009.09.003. labor party. (2019). the facts about an inheritance tax, url: https://www.alp.org.au/the_facts_about_an_inheritance_tax, accessed 10/6/19. sample, c, mcalaney, j and thackray, h. (2018). a cultural exploration of the social media manipulators, 17th european conference of cyber warfare, oslo, norway. sydney morning herald. (2019a). 'it is a lie': bill shorten targets liberals for death tax 'fake news' on facebook, url: https://www.smh.com.au/federal-election-2019/it-is-a-lie-bill-shortentargets-liberals-for-death-tax-fake-news-on-facebook-20190420-p51fu6.html, accessed 10/6/19. sydney morning herald. (2019b). 'labor demands facebook remove 'fake news' posts about false death tax plans, url: https://www.smh.com.au/federal-election-2019/labor-demandsfacebook-remove-fake-news-posts-about-false-death-tax-plans-20190419-p51fpk.html, accessed 10/6/19. taylor, p. m. (2013). munitions of the mind: a history of propaganda. manchester university press, uk. thompson, d.f. (2019). political ethics. in international encyclopedia of ethics. wiley publishers, uk. uk parliament. (2018). disinformation and ‘fake news’: interim report, fifth report of session 2017–19, london, uk. uk parliament. (2019). disinformation and ‘fake news’: final report, fifth report of session 2017–19, london, uk waltzman, r. (2017). the weaponization of information, rand, url: https://www.rand.org/content/dam/rand/pubs/testimonies/ct400/ct473/rand_ct47 3.pdf, accessed 10/11/19. copyright: © 2020 warren. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v24i0.2803 http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems warren 2020, vol 24, selected papers from the 2019 aice conference fake news in australian 2019 election 12 paper title (paper title style) australasian journal of information systems bradshaw, pulakanam & cragg 2015, vol. 19, pp. s197-s217 knowledge sharing between consultants and smes knowledge sharing in it consultant and sme interactions adrian bradshaw james cook university, singapore adrian.bradshaw@jcu.edu.au venkateswarlu pulakanam university of canterbury venkat.pulakanam@canterbuty.ac.nz paul cragg university of canterbury paul.cragg@canterbury.ac.nz abstract many small and medium enterprises (smes) depend on consultants to overcome knowledge barriers, especially for it projects. this paper aims to determine how it consultants affect the it knowledge of smes when it consultants and smes interact. data were collected using faceto-face interviews with both it consultants and sme managers. the study is the first to identify what and how smes learn from consultants during an it implementation project. consultants help smes gain different types of knowledge, employing a broad range of knowledge sharing mechanisms. as consultants are an important part of the knowledge creation processes of smes, smes should strive to form long-term relationships with consultants and use these interactions to develop it knowledge within the sme. keywords: knowledge sharing mechanisms; knowledge assets; it projects; it consultants; sme; accounting information systems. 1 introduction external experts are important to many small and medium sized enterprises (smes) for successful it implementation (see seminal work by soh et al. 1992, thong et al. 1994 & 1997, otiz de guinea et al. 2005). the main reason why external experts like consultants are necessary is because smes face knowledge barriers and depend on consultants to overcome these barriers (thong 2001; nevo et al. 2007; carey 2008 and chen et al. 2008). in helping smes to overcome knowledge barriers, consultants act as bridging intermediaries, in which they disseminate specialised knowledge and share ideas and experiences (carey 2008). while many smes “hire know-how” from advisors and consultants, it has been reported in the literature that smes were not effective at leveraging knowledge from external parties (chen et al. 2006, p9). this finding highlights the need and importance of understanding how the use of consultants influences the growth of it knowledge in smes. as a first step, it is necessary to understand the processes by which knowledge is acquired and shared, as well as the barriers hirlak 2013). in addition, it is further necessary to understand the types of knowledge that consultants help smes to create. this is important as it may assist smes to develop strategies to overcome their lack of it knowledge. in today’s fast changing business landscape, knowledge-based resources and processes may be the key to business performance improvement (schiuma 2012). accordingly, smes have to find ways to adequately manage this aspect, which involves knowledge identification, knowledge storage/retention, knowledge utilisation, knowledge creation and knowledge sharing. managing it knowledge in smes may be particularly challenging because of the reliance of smes on it consultants to help them overcome their lack of it knowledge. this paper therefore seeks to determine how it consultants affect the it knowledge of smes, from studying the interaction between consultants and smes. knowledge creation/transfer (nonaka & toyama 2005) and knowledge sharing mechanisms (boh 2007; boh & wong 2013) s197 australasian journal of information systems bradshaw, pulakanam & cragg 2015, vol. 19, pp. s197-s217 knowledge sharing between consultants and smes are used to understand how smes create it knowledge with the assistance or influence of consultants. this research is important as it focuses on two sectors of national importance; that is, the sme sector and the it services sector. first, this research is of interest to the sme sector, which plays a significant role in many national economies (taylor & murphy 2004). for example, in new zealand in 2013, 99 percent of enterprises were smes with fewer than 50 employees (ministry of business, innovation and employment [mbie] 2014). similarly, in many countries, smes account for a very high percentage of firms. second, the research extends it services research, which is currently weak. this weakness may be a result of the relatively recent and rapid growth of the it services sector, which has also become important to many national economies. it services are considered essential for improving business processes and infrastructure, especially in the context of economies dominated by smes (nevo et al. 2007; hu et al. 2014). it services is an important industry subsector within the it sector. for example, according to ‘the new zealand sectors report 2013: information and communication technology’ (mbie 2013), in new zealand in 2012, 83 percent of all it firms were in it services (which includes it product firms) and, 42% of all angel and venture investments were in it systems and it service sectors. also, according to this report, the it sector was “a story of success” and employment growth was driven by telecommunications and it services. further, it is a fast growing industry sector in many countries (oecd 2012). many smes are clients of the growing it services sector, so this research is important as it focuses on these two sectors. the many it consultants who are providing it services to smes need research relating to them. lastly, the research adds to the understanding of knowledge sharing and management within the context of it consultant and sme interactions. in today’s rapidly changing business environment, knowledge and knowledge management play a critical role in improving productivity and maintaining competitiveness. in this paper, it implementation refers to the process where a software system or solution is sourced, installed and configured within an organisation. it implementation in this study focuses on short-term projects where external it specialists (consultants) provide services for the purpose of implementing it solutions at the contracting sme. large scale outsourcing projects, long-term (multi-year) service agreements and on-demand technical services are excluded (nevo et al. 2007). for this study, an it consultant is viewed as any individual or organisation contracted to assist smes with the implementation of it. these include mega-consultants, vendor-consultants, independent consultants and independent reseller-consultants (these categories are adapted from basil et al. (1997) and are still relevant today. see table 1 for a brief description of each category). the scope of the study included accounting practices, as many accounting practices act as it consultants, and many accounting practices focus on smes. also, some accounting practices encourage their clients to use specific software and offer software support. thus, many accounting practices can be classified as either independent consultants or independentreseller consultants. s198 australasian journal of information systems bradshaw, pulakanam & cragg 2015, vol. 19, pp. s197-s217 knowledge sharing between consultants and smes consultant type description mega consultants large companies that provide a variety of consulting services e.g. deloitte independent consultants small or large companies that provide consultancy services but are not affiliated with any particular software vendor vendor-consultants companies that provide consultancy services to support their software solutions independent-reseller consultants small or large companies that provide consultancy services and sell software solutions but are not affiliated with any particular software vendor accounting practices accountants who encourage their clients to use specific software and offer software support table 1. types of it consultants (basil et al. 1997) there is no established or widely accepted definition of small business. curran & blackburn (2001) discuss that the definition of a small business may be based on the number of employees, on the organisation’s financial turnover, or may be qualitative. in this paper, firms with fewer than 50 full-time employees, which account for 99% of all new zealand enterprises, are considered as smes (mbie 2014). this reflects the definition used by the government of new zealand, where our sample is based. other countries have adopted different definitions for smes. the rest of the paper is organised as follows: in the next section, we review the literature on consultant-client interactions in implementing it projects, and knowledge creation and sharing. then we describe the research objectives and methodology, before reporting case study findings. the latter sections of the paper discuss the study’s contributions, limitations of the research, and suggestions for future research. 2 literature review 2.1 it consultant-client research past research has recognised the importance and value of external experts to the implementation of it in small or large organisations. yet the pivotal role consultant engagement plays on the use and management of it has not been fully researched. the extant research on consultant-client interactions is diverse, fragmented, sparse and sporadic (ko et al. 2005; swanson 2010; lech 2011). the initial focus of consultant-client research was on consultant performance, based on client satisfaction (haines & goodhue 2003; yoon et al. 2004). more recent studies identify various intermediary roles of consultants (howcroft & light 2008; carey 2008) and examine the consultant engagement process (chen et al 2008), consultant-client relationships (nevo et al 2007) and the creation or transfer of knowledge (lech 2011). six broad themes of extant research about it consultants are presented in table 2: strategies for vendor, software and consultant selection; functional activities, roles and responsibilities of it consultants; effectiveness of consultants and factors influencing it success; consultant engagement management including consultant-client relationships; knowledge creation and transfer; and it consultancy trends and other. table 2 cites about twenty studies, which support swanson’s (2010) view of limited research specifically related to it consultants. it is also evident from table 2 that much of the recent literature focusses on the impact of consultants on it success in large organisations, and consultant engagement and consultant-client relationships in smes. only two research papers s199 australasian journal of information systems bradshaw, pulakanam & cragg 2015, vol. 19, pp. s197-s217 knowledge sharing between consultants and smes (ko et al. 2005; lech 2011) focus on knowledge creation and transfer, with both addressing it implementation in large organisations. overall, most of the recent research is biased towards it implementation in large organisations (see for example, ifinedo (2011) and lech (2013)). smes large organisations (may also include smes) other 1. strategies for vendor, software and consultant selection gable (1991, 1996) – mmca 2. functional activities, roles and responsibilities of it consultants lech (2013) – es 3. effectiveness of consultants and factors influencing it success soh et al. (1992) – mmca thong et al.(1994, 1997) – mmca thong (2001) mmca haines & goodhue (2003) – erp wang & chen (2006) erp ifinedo (2011) – erp yoon, et al. (2004) 4. consultant engagement process including consultantclient relationships carey (2008) – wd howcroft & light (2008) – cr chen et al. (2008) erp chang et al. (2013) – erp nevo et al.(2007) – ibs 5. knowledge creation and transfer ko et al. (2005) – erp lech (2011) – erp 6. it consultancy trends and other basil, yen, & tang (1997) chen & wang (2006) swanson (2010) table 2. consultant-client research in the context of implementing it projects (where appropriate the specific it projects studied have been identified: erp = enterprise resource planning; es = enterprise systems; mmca = miniand micro-computer application software; ibs = internet business solutions; wd = web design; cr = customer relationship software) based on the summary provided in table 2, there is a clear gap in research on knowledge transfer/creation, and the role of consultants in implementing it in smes. it is this gap that this paper attempts to fill. 2.2 knowledge management and consultants knowledge is an organisation’s most valuable resource of lasting competitive advantage. it has become an area of interest for researchers in many disciplines over the years. knowledge is defined differently depending on the context of the research. for example, alavi & leidner (2001) define knowledge as “a justified belief that increases an individual's capacity to take effective action” (p109). on the other hand, beesley & chalip (2011) define knowledge as “information with meaning that exists within the individual” (p328). nonaka, krogh & voelpel (2006) recognise that knowledge can exist in two forms, explicit or tacit. nonaka et al. (2006) point out that knowledge that can be uttered, formulated in sentences or captured in drawings and writing is explicit; knowledge tied to the senses, movement skills, physical experiences, intuition or implicit rules of thumb is tacit. smes do not manage knowledge in the same way as larger organisations (desouza & awazu 2006). importantly, socialisation dominates the knowledge creation process in smes, and they tend to lack knowledge repositories. “technology is never made part of the knowledge s200 australasian journal of information systems bradshaw, pulakanam & cragg 2015, vol. 19, pp. s197-s217 knowledge sharing between consultants and smes management equation” (desouza & awazu 2006, p40). the socialisation mode involves sharing tacit knowledge among individuals (nonaka et al. 2006). alavi & leidner (2001) note that knowledge transfer occurs: between individuals; from individuals to explicit sources; from individuals to groups; between groups; across groups and from groups to the organisation. consequently, consultants, as intermediaries, have a key role to play in the knowledge transfer and knowledge creation process in smes. howcroft & light (2008) argue that consultants act as ‘conduits’ by standing between it suppliers and smes, where they provide services such as advice to assist with finding appropriate products, the implementation and customisation of the products, training and support services and the integration of software with existing systems. it is conceivable that during the sme-consultant process the transfer of knowledge occurs, a point that carey (2008) underscores by describing consultants as bridging intermediaries who disseminate knowledge. therefore, it is argued that the use of consultants for the it implementation process is a key part of the knowledge creation/transfer process in smes and consequently crucial to the organisation’s knowledge management practices. this view is supported by durst & edvardsson (2012) who call for studies that examine both sides of the knowledge sharing process in smes; that is, both sender (for implementation projects this would be the consultant) and receiver (sme). 2.3 knowledge creation and sharing in this paper we equate knowledge transfer with knowledge sharing, as do other researchers (see ko et al. 2005). knowledge transfer is defined in many ways but essentially it involves an exchange of knowledge from giver to receiver (jelavic 2011). it is purposeful, particularly in the implementation of it systems and creates value for the client organisation. the core concept of knowledge transfer in it system implementation is to ensure the effective application of the respective it systems in the client organisation (werner et al. 2014). it implementation is knowledge intensive. within this setting, knowledge transfer happens in two directions: business knowledge transfers from client to consultant; and it technical knowledge transfers from consultant to client (lech 2011). the transfer of knowledge from consultant to client is the focus of this research. but how is knowledge created and shared? nonaka & takeuchi (1995) proposed a unified model of organisational knowledge creation and transfer, and argued that knowledge was created through the interaction and intersection between tacit and explicit knowledge. the researchers noted that these interactions occurred along four modes: socialisation; externalisation; combination; and internalisation. this is referred to as the seci model. socialisation is the process of converting new tacit knowledge into the existing base of tacit knowledge. externalisation is the process of articulating tacit knowledge into explicit knowledge. combination is the process of converting explicit knowledge into more complex and systemic sets of explicit knowledge. finally, internalisation is the process of embodying explicit knowledge into tacit knowledge. drawing on the knowledge management literature, boh & wong (2013) propose a framework of knowledge sharing mechanisms, to reflect how knowledge is shared. boh & wong (2013) define knowledge sharing mechanisms as “organizational practices adopted to facilitate the sharing, integrating, interpreting, and applying of know-what, know-how, and know-why embedded in individuals and groups” (p123). boh & wong identify two dimensions to their framework: codification versus personalisation, and formal versus informal. the result is a two-by-two framework with four quadrants, which they refer to as quadrants 1 to 4 (table 3). s201 australasian journal of information systems bradshaw, pulakanam & cragg 2015, vol. 19, pp. s197-s217 knowledge sharing between consultants and smes informal formal personalisation quadrant one mechanisms quadrant two mechanisms codification quadrant four mechanisms quadrant three mechanisms table 3. knowledge sharing mechanisms framework (boh 2007; boh & wong 2013) their quadrant 1 (informal personalisation) focuses on informal person-to-person knowledge sharing, such as chatting in cafes, hallways and at social events. by comparison, quadrant 2 (formal personalisation) reflects arranged sessions for person-to-person interaction, such as training sessions, joint exercises and communities of practice. their quadrants 3 and 4 reflect codification rather than personalisation. codification is where knowledge is “carefully articulated, captured, and stored in documents and databases” (p126). quadrant 3 (formal codification) focuses on person-to-document knowledge sharing, and includes organisationwide repositories and intranets. quadrant 4 (informal codification) covers informal and adhoc exchanges of documents, reflecting a tendency for people to use personal contacts for documents rather than electronic databases (quadrant 3). this framework is particularly useful, and more suitable for our purposes, as the four quadrants cover the range of knowledge sharing contexts that occur during a typical consultant driven it implementation. 2.4 it implementation and knowledge assets (kas) if knowledge transfer/creation is an important aspect of the knowledge management (km) process (durst & edvardsson 2012), then the outcomes of knowledge transfer/creation are the necessary target of the knowledge identification, knowledge storage/retention and knowledge utilisation processes of km. the outcomes of knowledge transfer/creation are knowledge assets (nonaka et al. 2006). from a knowledge-based perspective, knowledge assets are either the inputs or the outcomes of knowledge creating processes within the firm. nonaka et al. (2000) identify four types of knowledge assets: experiential; conceptual; systemic; and routine. experiential knowledge assets consist of the tacit knowledge that is built through shared hands-on or working experience among individuals. conceptual knowledge assets consist of explicit knowledge articulated through images, symbols and languages. systemic knowledge assets consist of systematised and packaged explicit knowledge, such as explicitly stated technologies, product specifications, manuals, documentation and packaged information about customers and suppliers. routine knowledge assets consist of the tacit knowledge that is embedded and regulated in the actions and practices of a firm. building on the idea that consultants disseminate specialised knowledge and are a part of the knowledge creation process, what knowledge assets result during an it implementation project? given the lack of it knowledge in smes, and the knowledge dissemination ability of consultants, it stands to reason that various types of knowledge should be created in smes. the management of this newly created knowledge could have a major impact in helping smes overcome their knowledge barriers. the question raised here is supported by the finding of durst & edvardsson (2012) who found there was little research on the types of knowledge provided in a firm, and on what types of knowledge was needed to deal with present and future business challenges. in the it implementation context, this suggests that there is a need to know the types of knowledge that consultants disseminate to smes, as well as what types of knowledge are needed to help smes overcome their it knowledge barriers. in summary, it has been argued that knowledge transfer/creation during it implementation is an important aspect of the knowledge management process, and that consultants play a key part in the process. it has also been argued that there is a need to identify the types of knowledge assets resulting from the knowledge creation process, and a further need to identify which types of knowledge assets are beneficial to overcoming knowledge barriers in smes. s202 australasian journal of information systems bradshaw, pulakanam & cragg 2015, vol. 19, pp. s197-s217 knowledge sharing between consultants and smes 2.5 research gap and modified framework combining the concepts discussed above, and especially knowledge sharing mechanisms and knowledge assets, it is argued that the creation of knowledge assets can be related to knowledge sharing mechanisms. since knowledge sharing mechanisms detail how knowledge is shared, and knowledge assets are the result of knowledge sharing/creation, it seems likely that there should be an association between knowledge sharing mechanisms and knowledge assets. this implies that each knowledge sharing mechanism used by smes could result in the creation of knowledge assets. in addition, since knowledge sharing mechanisms are generally defined as organisational practices used to facilitate the sharing of knowledge, and consultants disseminate knowledge, there are knowledge sharing mechanisms that may be influenced or even facilitated by consultants. this represents a point of departure from previous research. prior research on knowledge sharing mechanisms has focussed on the advantages and disadvantages of various knowledge sharing mechanisms, the characteristics of knowledge sharing mechanisms, and more recently the classification of knowledge sharing mechanisms (boh 2007) as well as the perceived usefulness of various types of knowledge sharing mechanisms (boh & wong 2013). in this paper we do not seek to investigate individually either knowledge sharing mechanisms, nor consultants, nor knowledge assets. rather, we combine the creation of knowledge assets with knowledge sharing mechanisms to understand the knowledge sharing intermediary role of consultants and the impact of this role on smes. in particular we seek to understand how the knowledge sharing role of consultants leads to the creation of knowledge assets. table 4 is the general modified framework developed by combining knowledge sharing mechanisms from boh & wong (2013) with knowledge-based theory (i.e. knowledge assets are the result of knowledge creation/transfer). informal formal personalisation quadrant one mechanisms knowledge assets quadrant two mechanisms knowledge assets codification quadrant four mechanisms knowledge assets quadrant three mechanisms knowledge assets table 4. modified knowledge mechanisms framework the concepts combined to create table 4 have all been previously examined separately. combining these concepts in the context of it implementation and the use of consultants presents a unique opportunity to address the research gap previously identified pertaining to knowledge transfer/creation, and the role of consultants in implementing it in smes. 3 research objectives this study aimed to address the following two research questions: 1. how do consultants share knowledge with smes during it implementation projects? 2. what knowledge assets result from the knowledge sharing process between consultants and smes during it implementation projects? s203 australasian journal of information systems bradshaw, pulakanam & cragg 2015, vol. 19, pp. s197-s217 knowledge sharing between consultants and smes 3.1 research methodology and data collection the context for the study was the implementation of it by smes with the aid of consultants. the study was designed as a series of interviews with both sides of the it implementation project; that is, it consultants and smes. consultants were treated as typical examples of individual consultants and were interviewed about their experiences with it implementations with smes. in addition, each sme provided one example of an it implementation project. in this sense, the unit of analysis for this research is the sme-consultant relationship. the case design follows the advice of oates (2006) and easterby-smith et al. (2008), where case studies can take an in depth look at one, or even a smaller number of organisations, a department, an information system, or individuals. the case study method was selected as an appropriate approach because it allows it implementation projects to be studied in their real-life context (yin 2003). this approach allows a holistic view of the implementation process and the consultant-client interaction. accounting information systems (ais) were chosen as they are widely used by smes. while some smes might use enterprise resource planning systems (erps) and others may use customer relationship management systems (crms), most smes would make use of ais. examples of off-the-shelf ais packages often adopted by smes include myob, quickbooks and accredo. the main criterion for selecting consultants was to consider consultants who primarily work with, and had previously implemented, ais in smes. consultants were chosen by first obtaining a list of accounting packages designed specifically for smes. consultants were then randomly chosen, at least one for each of the packages used by smes. this was to ensure that the major accounting packages used by smes were accounted for in the study. data were collected primarily by face-to-face interviews and supplemented by supporting material such as software brochures and company websites. the main selection criteria for smes (service or manufacturing), was that the company had, with the assistance of a consultant, implemented, or was in the process of implementing, a new ais or an upgrade of an existing system. each sme had to meet the definition used in new zealand for smes, ie, an independent firm with up to 50 employees (mbie, 2014). smes were recruited using industry contacts. the series of interviews with consultants represents the sender side of the knowledge transfer process, while the sme interviews represent the receiver side of the process. to ensure consistency between the two sides it was necessary to ensure that the implementation steps followed by consultants were the same. consultants carry out the following steps or processes during implementation projects: (i) initiation, (ii) analysis and recommendations, (iii) installation, (iv) configuration and integration, (v) training and (vi) maintenance (on-going support). it was important to establish that the implementation of the ais followed the same steps. this meant that it was reasonable to assume that the smes would have experienced a similar implementation process. in essence, the implementation process can then be treated as a constant, making it easier to examine the consultant-sme interactions. it also meant that once consistent findings in the data were observed, it could be assumed that data saturation had been reached. therefore the sample size for this research was not predetermined. we stopped collecting data from smes and consultants when we felt that we reached ‘saturation’ (corbin & strauss 2008); that is, when we felt that we were not gaining any more new insights from the data collected. tables 5 and 6 provide a summary of the consultants and smes involved in the study. eight consultants were interviewed. these covered the range of types of consultants working with smes, and represented four of the major ais used by smes. s204 australasian journal of information systems bradshaw, pulakanam & cragg 2015, vol. 19, pp. s197-s217 knowledge sharing between consultants and smes consultant type of consultant expertise consultant1abs reseller quickbooks consultant2ap independent myob consultant3cad accountant myob consultant4jk independent-reseller myob consultant5ow independent-reseller myob & infusion consultant6aif independent myob, accredo consultant7at accountant myob consultant8ecr independent-reseller myob, accredo table 5. summary of the eight consulting firms in the study sme no. of employees sector type of consultant employed ais interviewee sme1eat 50 manufacturing independentreseller quickbooks accountant sme2id 6 manufacturing and trades independentreseller myob manager sme3amn 7 manufacturing and trades accountant myob manager sme4am 16 manufacturing independentreseller infusion financial controller sme5dm 11 manufacturing and retail accountant quickbooks owner sme6sm 30 manufacturing independentreseller accredo owner sme7ag 17 service accountant myob manager table 6. summary of the seven smes in the study seven smes were included for the receiver side of the knowledge transfer process. the largest sme had 50 employees and the smallest had 6 employees. the primary data were collected via semi-structured interviews. the interviews were tape recorded with consent from the interviewees, and were approximately one hour in length. two interview protocols were used; one for the consultants and one for the smes. for the sme interviews, the main persons who worked with the consultant on the project were interviewed. the interview questions were divided into three sections: (1) personal and company details, (2) implementation project details, and (3) knowledge assets. these sections applied to both the consultant and the sme interviews. a sample of the questions that guided each set of interviews is provided below. most questions in sections 2 and 3 were used in an open-ended way, with sub-questions as prompts to gain a rich picture for the topic. consultant interviews 1. how do you go about sharing hands-on accounting expertise with clients? 2. how do you ensure that the client gains accounting know-how? 3. could you describe any organisational routines surrounding the use of the accounting system established by yourself? how are these routines established? s205 australasian journal of information systems bradshaw, pulakanam & cragg 2015, vol. 19, pp. s197-s217 knowledge sharing between consultants and smes 4. how do you go about sharing your accounting skills and knowledge? 5. how do you ensure that the organisation makes use of information stored in the accounting database or data-store? sme interviews 1. who were the main sources of hands-on accounting software experience (employees, partners, suppliers, consultants, accountants)? 2. how is the hands-on experience shared throughout the organisation? 3. how did you acquire your knowledge of how-to do accounting (know-how)? 4. could you describe any organisational routines governing the use of the accounting software? how were these routines established? how are these routines shared among employees? 5. how do you make use of the information stored in the accounting database or datastore? how did the accountant assist you with this? 3.2 coding process data were collected primarily by interviews and supplemented by supporting material such as it brochures and company websites. all interviews were recorded, transcribed and then analysed using tams analyzer, a software for analysing qualitative data for macintosh computers. copies of the transcripts were given to the interviewees for their verification before the transcript was accepted for analysis. data were coded according to the predefined categories determined by the study framework (see table 4). data were mapped according to the major themes of the framework and this provided the initial coded lists. each initial coded list was then recoded at a lower level to create subcategories or codes of the initial ones. memos and annotations were used to assist in the recoding process. within each group (that is, consultant interview data and sme interview data), searching and pattern matching were used to categorise the sender and receiver sides of the knowledge sharing process. 4 analysis and findings the aim of this research was to investigate how knowledge was shared and what knowledge was shared during it implementation projects. the case evidence was analysed using the modified framework of knowledge sharing mechanisms (boh & wong 2013) and the types of knowledge assets identified by nonaka et al. (2000). by following this approach the results reveal how knowledge was shared between consultants and smes, as well as the type of knowledge (knowledge assets) that resulted. each of the four parts of the boh & wong (2013) framework is discussed below, with the associated knowledge assets. where a knowledge sharing mechanism is discussed, its corresponding quadrant (q1, q2, q3, q4) is displayed in brackets. the results are summarised in table 7 (knowledge sharing mechanisms and associated types of knowledge assets) and table 8 (example knowledge assets in consultant and sme interactions). s206 australasian journal of information systems bradshaw, pulakanam & cragg 2015, vol. 19, pp. s197-s217 knowledge sharing between consultants and smes quadrant 1. (informal personalisation) mechanisms: word of mouth sharing through informal, one-on-one training sessions and phone calls over a period of a few months; that is, not just at the time of implementing the system. type of knowledge asset: experiential (knowhow to use the new system); systemic (reports from the new system) quadrant 2. (formal personalisation) mechanisms: formal training sessions by consultants for smes, involving hands-on use of the system, as well as discussion. formal project meetings, including project initiation, updates and reviews. type of knowledge asset: experiential (know-how to use the new system), and accounting knowledge (how to undertake some accounting activities). quadrant 4. (informal codification) mechanisms: written monthly checklist of tasks. written tips and tricks. manuals written voluntarily. procedures written voluntarily. sharing documents informally, e.g., emails sent from consultants to smes, e.g., on how to do something. type of knowledge asset: systemic (‘how to’ documents); routine (accounting routines). quadrant 3. (formal codification) mechanisms: written, step-by-step documents/guides, including monthly checklist of tasks, and tips and tricks. formal resources like user manuals, software documentation, cds, dvds, and on-line help newsletters. type of knowledge asset: systemic (‘how to’ documents and manuals). table 7. knowledge sharing mechanisms and associated types of knowledge assets 4.1 quadrant 1: informal personalisation informal training was the main knowledge sharing mechanism; that is, knowledge was shared from consultant to sme mainly through informal training sessions. at the early part of the implementation project, training was formal (q2), followed by numerous informal one-on-one sessions (q1) over a period of months. consultant4jk revealed, “... generally 90% of the training that i do is out at the clients’ workplace, one-onone training... showing them what to do.” consultants provided verbal instructions and suggestions (q1) to sme staff, who may in turn create written procedures of their own (q4). much of the on-going contact was informal, with sme staff contacting consultants on a casual/ad-hoc (q1) basis to seek advice, guidance and assistance with problems. thus there was typically an on-going relationship between consultants and smes. consultant4jk believed that on-going contact with the client was essential in the implementation process and noted that on-site visits, involving one-on-one training with smes, occurred over several months (q1). thus informal networks were created between consultants and some staff within each sme. the consultants did not help sme staff develop informal networks with users in other firms. the consultants built up their own knowledge over an extensive period of time, especially on how to integrate and use the software in ways that achieved various goals in different smes. for example, consultants tried to ensure that sme staff had easy access to the knowledge stored in the accounting database. consultant1abs stated, “i would, as part of the training, show them how to create the reports (an experiential ka), and explain the usefulness of those reports and why they exist and who usually would want information from those reports.” similarly, consultant3cad pointed out, s207 australasian journal of information systems bradshaw, pulakanam & cragg 2015, vol. 19, pp. s197-s217 knowledge sharing between consultants and smes “so i’ll sit down with them and find out what information they really want .... i go through with them the sort of standard reports (systemic ka) that they ought to be using so that they can get the information that they’re looking for. i’d also make recommendations on the sort of reports that i think they ought to look at.” this on-going informal training by it consultants resulted in sme staff becoming advanced users of the accounting system. typically, at the start of a project, no sme staff possessed knowledge of how to use the new accounting software. during the project, sme staff not only acquired hands-on knowledge of how to use the new software (an experiential ka), but they also acquired broader know-how knowledge (an experiential ka). for example, at sme3amn, staff learned how to use the system, then in later months gained data from the new system of their increased use of some products. they then used this product-use data to negotiate a better price with their supplier as they were using more of the particular product. the consultants facilitated the process of empowering users to use the software, and continually grow their hands-on knowledge. consultants facilitated this process when they showed users how to create various reports, as well as which reports would be of value to their company. this knowhow/how-to knowledge are examples of experiential kas, while the report itself is an example of a systemic ka. sme staff learned how to make good use of the accounting database (e.g., to create a profit and loss report), which is new knowledge constructed out of the knowledge stored in the database. experiential knowledge assets dominate quadrant one activity, with some evidence of the use of systemic knowledge assets. the main example of experiential kas was the know-how acquired by sme staff to use the new ais. sme staff learned to use the software for various tasks with help from the consultant. such shared experiences and activities facilitated the sharing and development of tacit hands-on knowledge. there was also evidence of systemic kas resulting from quadrant one type mechanisms, especially where knowledge was packaged in the form of a report. therefore, knowledge was shared between consultants and smes through the informal personalisation mechanism where experiential and systemic knowledge assets emerged (or were created as a result). refer to table 8 for a list of relevant experiential and systemic knowledge assets. 4.2 quadrant 2: formal personalisation the initial training sessions (q2) tend to be structured, aimed at equipping sme staff with sufficient hands-on knowledge to use the software. sme staff gained this new knowledge from face-to-face contact, rather than from reading books or manuals. in addition, some sme staff attended training sessions held by the software suppliers. formal meetings were another quadrant two mechanism used by consultants. several formal meetings took place involving consultants and sme staff. these meetings occurred throughout the course of a project; for example, initiation, update and review meetings, and follow-up meetings after the completion of the project. consultant4jk noted that they had follow-up meetings designed to ensure that smes were progressing well with the software. sme3amn noted that they had regular quarterly meetings with the consultant, who was also their accountant. sme2id stated that since the completion of the project, the consultant made a scheduled yearly visit to review how staff were progressing and to address any issues. sme staff watched, practiced, listened and interacted with the consultants (q2). it was through these interactions, both prior to implementation and during the implementation, that smes gained know-how knowledge of accounting, the use of accounting software, and the potential benefits and implications of using the software. these types of know-how knowledge are examples of experiential kas. therefore, knowledge was also shared between consultants and smes through the formal personalisation mechanism, and experiential knowledge assets were the main outcome of this knowledge transfer or sharing. s208 australasian journal of information systems bradshaw, pulakanam & cragg 2015, vol. 19, pp. s197-s217 knowledge sharing between consultants and smes 4.3 quadrant 3: formal codification the consultants often provided written step-by-step documents or guides (q3) to help smes build their hands-on expertise. this is an example of a formal codification mechanism. boh & wong (2013) define knowledge that is shared through a codification mechanism as knowledge that is carefully codified and stored in databases and documents. the codified knowledge can then be accessed and used easily by employees. the consultants also provided written tips and tricks (written documents are an example of systemic kas). consultant4jk generally left new clients with “six or seven pages” of guidelines as “basic knowledge, how to do this, how to do that”. consultant5ow provided “a step-bystep guide of basic knowledge to help get them started” (all examples of systemic kas). consultant1abs aimed to “try and break it down into as many small steps as possible and try and layout the process for them to get their work done.” consultant7at referred to the written document as "business best practice”. “for example, as part of the follow up (q2), we have the monthly checklist and the instructions (q3).” the checklist provided by the consultant would be a list of tasks that smes were expected to undertake each month. consultants also made other quadrant three type resources available to smes, including software documentation, manuals, cds, dvds, newsletters, and on-line help (all examples of systemic kas). for example, the manager of sme2id used the software manuals to teach himself how to use the software. newsletters were also used to pass information onto smes. the newsletters were published by the various software vendors and consultants to make these available to the smes. the trend appears to be that online help, either as part of the software or over the internet at the vendor's site, has replaced manuals and other materials. printed manuals and cds were still provided by some vendors. where this was the case, consultants ensured that this material was given to smes. it should be noted that formal and informal training, and the use of routine documents and procedures were more commonly used than software manuals and cds. although consultants informed smes about the online documentation and manuals, only sme2id reported using the software manuals. furthermore, very few sme staff used on-line help or any documentation that may have been included with the software. consultant1abs stated that “... the books and cds, they get put on a shelf and never looked at again.” consultant2ap noted that very few people used the actual reference material. he pointed out that, “they don’t want to fool around and look in a book, go on a website; it might or might not work, and all the rest of the drama that goes with it.” consultant3cad said that, although he supported the notion that reference material was not used, he also noted, “what i find is that it varies from person to person. some people really like to use the help; other people will just pick up the phone and phone me. yeah, it really does vary.” systemic knowledge assets dominated quadrant three. systemic knowledge assets consist of systematised and packaged explicit knowledge. this kind of knowledge is relatively easy to communicate, store, and distribute. examples of systemic knowledge assets included the software database, reports, online help documents and other documents provided by consultants. these would be documents such as routines and procedures for the accounting function and use of the software such as tips and tricks, “how-tos”, step-by-step guides and best practice guides codified by consultants. thus consultants used the knowledge and experience they had to assist smes by providing tailored solutions to meet the needs of each sme. therefore, the formal codification mechanism resulted in systemic knowledge assets. 4.4 quadrant 4: informal codification as the sme staff became more confident in using the software, and as the use of the software became embedded in their daily routines, sme staff created their own written documents detailing routines and procedures. thus it was common for sme staff to create their own set of notes to guide them (q4), especially in a range of periodic (daily, weekly, etc.) routine s209 australasian journal of information systems bradshaw, pulakanam & cragg 2015, vol. 19, pp. s197-s217 knowledge sharing between consultants and smes activities related to using the system. this point was best captured in the following quote from consultant5ow, “so they will either write it out as you work through it with them: so they’ll essentially write their own procedure for it, but you work with them while they’re doing that. i find that better because the way i word something may not be the way they would word something; so if i do it for them they may not understand that when i’m not there. so it’s better to be written in their words so that they can then use it in hindsight.” consultant1abs indicated, “i usually recommend a process. i haven’t formalised it with documentation.... but i would usually recommend it verbally and get them to write it down.” consultant3cad stated that he had no formalised routines but would “... go over with the client what is needed to be done daily, weekly and monthly,” and recommended to the client the necessary procedures to follow. consultant4jk stated that, with new clients, who were usually unaware of how the entire accounting process worked, she recommended a basic bookkeeping routine to help with their needs. routine knowledge assets are embedded and regulated in the actions and practices of a firm and consist of know-how, working practice and organisational routines for carrying out dayto-day activities (chou & he 2004). this is knowledge that is embedded in day-to-day activities, including the accounting routines sme staff carry out on a daily or periodic basis. therefore, while the procedures and routines written by staff are considered systemic knowledge assets, the know-how gained from the practice or execution of the routines would be considered routine knowledge assets. therefore, the informal codification mechanism gives rise to systemic and routine knowledge assets. knowledge asset outcomes experiential kas1. gain know-how and hands-on-knowledge of the use of ais.2. using the system to connect with customers3. use, potential benefits and implication of using the softwarea. how much one client is purchasing more than the otherb. visibility of raw material used, which helps to re-negotiate price4. accounting knowledge5. trust6. improvisation systemic kas1. types of customised reports of value to sme2. routine documents or procedure documents3. basic knowledge guides of how to do (breakdown into small steps)4. generate new customised reports, including profit and loss reports5. sme writes routine documents or procedure documents routine kas1. daily, weekly and monthly routines of the accounting function2. invoicing, accounts payable, payroll routines table 8. example knowledge asset outcomes in it consultant and sme interactions it is worth noting that in practice, boh & wong (2013) found it hard to differentiate between quadrants 1 and 4 as “knowledge exchange via informal codification mechanisms tends to s210 australasian journal of information systems bradshaw, pulakanam & cragg 2015, vol. 19, pp. s197-s217 knowledge sharing between consultants and smes take place concurrently with informal personalisation mechanisms” (p131). when individuals share knowledge, they often discuss (quadrant 1) and exchange documents (quadrant 4). for example, during informal discussion and one-on-one training (quadrant 1) the consultant may produce written tips and tricks (quadrant 4). however, a number of documents were created by the employees of the smes, for example manuals written voluntarily. this made it easier to differentiate between quadrants 1 and 4. 5 discussion this study makes two significant contributions. first it improves our understanding of ‘how’ smes acquire knowledge from it consultants. secondly, the study identifies ‘what’ types of knowledge smes gain from it consultants. the study thus significantly improves our understanding of the intermediary role of consultants, as discussed by howcroft & light (2008) and carey (2008). the study identified common knowledge sharing mechanisms used by consultants with smes. all four of the boh & wong (2013) types of knowledge sharing mechanism were identified. informal training (q1) was the most commonly used mechanism, which was usually preceded by formal training (q2). other common mechanisms were informal (q4) and formal documents (q3) that detailed what tasks to undertake and the process for carrying them out. desouza & awazu (2006) found that nonaka’s concept of socialisation (nonaka et al. 2006) dominated the sharing of knowledge in smes, and that both formal and informal socialisation methods were used. the current study supports this observation, as the mechanisms used to facilitate knowledge sharing were formal and informal and involved socialisation with the consultants. this study found mixed support for desouza & awazu (2006), who claimed that smes were good at exploiting external knowledge. while the current study found that smes used consultants to gain technical knowledge of ais and to build accounting knowledge and skills, many other opportunities were spurned. for example, the smes did not seek to broaden their it and project management skills by improving, for instance, their ability to identify the firm’s it needs, and to analyse potential it solutions. in effect, the smes used consultants as a one-stop-shop for matters of accounting and ais. additionally, the people-based mechanisms observed by desouza & awazu (2006) were also apparent in this study, for example face-to-face meetings, observations, and training. they argued that the predominant use of people-based mechanisms meant that knowledge generated is immediately put into practice. while the nature of it may be such that the immediate application of knowledge occurs, the idea that knowledge management is based on people and not technology is noted in the current study. the only knowledge repository was the ais database, and all other knowledge remained with individuals. however, informal codification mechanisms can quickly become formal codification mechanisms. for example, some of the manuals and procedures written voluntarily by users became the organisation’s routine for ais and accounting operations. the current study also contributes by identifying what types of knowledge are shared during consultant/sme interactions. importantly, the study identified three types of ‘specialised knowledge’ that consultants help to create: (i) experiential, such as know-how, hands-on, and trust; (ii) routine, such as established routines that facilitate learning; and (iii) systemic, such as reports generated from the software database (chou & he 2004). thus, this study also improves our understanding of the kinds of knowledge assets that are created when consultants assist smes with the implementation of ais. this is important because knowing what knowledge assets are created in smes may assist with the identification of tools and processes to aid smes with knowledge management. in addition to it knowledge assets, the study found that consultants also helped with accounting knowledge. while the majority of knowledge created initially is focused on use of the software, experiential knowledge assets such as trust, best business practice (accounting) and improvisation were evident. such tacit knowledge equips the firm with the potential to turn routine use to benefit the organisation. experiential knowledge assets dominate boh & wong’s quadrants q1 and q2; s211 australasian journal of information systems bradshaw, pulakanam & cragg 2015, vol. 19, pp. s197-s217 knowledge sharing between consultants and smes that is, informal personalisation and formal personalisation. experiential knowledge assets consist of tacit knowledge that is built through hands-on or shared working experience (nonaka et al. 2000). tacit knowledge is personal knowledge held by the individual, based on the individual's subjective insight, intuition and hunches (nonaka et al. 2000). this kind of knowledge is acquired mainly through association with other people and requires shared activities or experiences to be imparted from one person to another. nonaka argues that shared experiences are vital for tacit knowledge to be shared. the shared experience involves using the software for various tasks with the consultant 'shadowing' the users. such shared experiences and activities facilitate the sharing and development of tacit hands-on knowledge. the study also clarifies the link between the use of external experts, like consultants, and the creation of knowledge assets in smes. while studies have indicated that knowledge from outside an organisation can come from consultants (desouza & awazu 2006; swanson 2010), this study clarifies these links by showing how consultants become a part of the knowledge creation processes in organisations like smes. consultants, as part of the knowledge creation process, share various types of knowledge assets through numerous knowledge sharing mechanisms (nonaka et al. 2000; boh & wong 2013). as a consequence, consultants affect the know-how of firms as reported by swanson (2010). the consultants interviewed in this study all indicated that it was in the interest of consultants to ensure that their clients gained sufficient knowledge of ais (know-how), as this positively influenced the satisfaction of smes and increased the likelihood that the sme would engage the consultant on a regular basis. this also had the knock-on effect that the sme would recommend the consultant to other smes. 6 conclusions a series of interviews with managers of smes and with it consultants helped identify three types of knowledge assets that were shared during it implementation projects. the study, therefore, demonstrated that consultants have an impact on knowledge assets in smes. as a result, it is recommended that owners/managers, administrators and other users of ais in smes take the opportunity to build knowledge assets while engaging consultants. in light of the findings of this study, managers and administrators should seek to create ‘strategic partnerships’ with consultants. the partnership suggested here should be aimed at developing the organisation's it knowledge assets, thus ensuring that smes do not become totally dependent on consultants for ais knowledge. this suggestion is in some ways similar to that given by ko et al. (2005) who suggest that the relationship between consultant and client should be such that they interact frequently, which would facilitate the flow and interpretation of knowledge. consultants should view the process of implementing ais in smes holistically, involving more than the selection and installation of software for the sme, but also the development of it knowledge assets. some consultants have already begun to embrace this view, and are encouraged to continue in order to assist smes to benefit by investing in it as well as benefitting from deeper engagement with their consultant. an ideal way of influencing it knowledge assets is to develop on-going relationships/partnerships with smes. it may be tempting to think that encouraging consultants to affect the knowledge assets of smes may not be in the interest of consultants. such thinking is likely based on the idea that, as smes develop knowledge assets, they may no longer require the assistance of consultants. this view is misleading, since smes will continue to need the assistance of consultants as they develop their knowledge assets. an increase in the it knowledge assets of smes is likely to result in a demand for greater use of it, which will require assistance from consultants. some of the consultants in this study noted that even when smes were knowledgeable, consultants still assisted with more advanced use of the system. therefore, when consultants positively affect knowledge assets, it will be beneficial to both consultants and smes. this study has a number of limitations. the study highlighted various knowledge assets that are shared by consultants. this may not be an exhaustive list. it is possible that there are other examples of knowledge assets created in smes but not identified by this study. however, this s212 australasian journal of information systems bradshaw, pulakanam & cragg 2015, vol. 19, pp. s197-s217 knowledge sharing between consultants and smes does not subtract from the overall finding that consultants assist in the sharing and creation of it knowledge assets in smes. another limitation is that the findings of this study may not be generalisable due to the small sample size and the use of a convenience sample. however, the findings are likely to be relevant to smes in comparable situations. also, our findings may or may not be applicable beyond the ais context. one other limitation of this research is that certain factors that may affect or mediate the findings of this study were not investigated. these factors include the attributes of consultants, the learning capability of employees of smes, and the reasons why smes implement it. the findings of this research and the above limitations reveal new opportunities for further research into it consultants and smes. in particular, there is a need to examine further the factors that may affect or mediate the impact consultants have on it knowledge assets in smes. for example, research could investigate the attributes of consultants; how do the abilities or skills of consultants affect the impact that consultants have on it knowledge assets? also, what effect does the reason behind the project have on the impact that consultants have on it knowledge assets? further, to what extent does the absorptive capacity of smes (that is, the “capacity of the recipient to assimilate, value and use knowledge and information transferred”) affect the impact that consultants have on it knowledge assets (carayannis et al. 2012, p141)? and finally, what it knowledge assets help predict it success in smes? acknowledgments the authors would like to thank the referees and, particularly, the editors, craig parker and stephen burgess, for their constructive comments in improving this paper. 7 references alavi, m. & leidner, d.e. (2001) "review: knowledge management and knowledge management systems: conceptual foundations and research issues", mis quarterly, 25(1): 107-136. basil, p., yen, d.c. & tang, h.l. (1997) "information consulting: developments, trends and suggestions for growth", international journal of information management, 17(5): 303-323. doi: 10.1016/s0268-4012(97)00013-3 beesley, l. & chalip, l. (2011) "seeking (and not seeking) to leverage mega-sport events in non-host destinations: the case of shanghai and the beijing olympics", journal of sport & tourism, 16(4): 323-322. doi: 10.1080/14775085.2011.635016 bessant, j. & rush, h. (1995) "building bridges for innovation: the role of consultants in technology transfer", research policy, 24(1): 97-114. boh, w.f. (2007) "mechanisms for sharing knowledge in project-based organizations", information and organization, 17(1): 27-58. doi: 10.1016/j.infoandorg.2006.10.001 boh, w.f. & wong, s.s. (2013) "organizational climate and perceived manager effectiveness: influencing perceived usefulness of knowledge sharing mechanisms", journal of the association for information systems, 14(3): 122-152. carayannis, e.g., pirzadeh, a. & popescu, d. (2012) "epistemic communities, knowledge transfer, and institutional learning", in: institutional learning and knowledge transfer across epistemic communities, springer,. 123-150. carey, j. (2008) "role misconceptions and negotiations in small business owner/web developer relationships", journal of management & organization, 14(1): 85-99. champion, d.p., kiel, d.h. & mclendon, j.a. (1990) "choosing a consulting role", training & s213 australasian journal of information systems bradshaw, pulakanam & cragg 2015, vol. 19, pp. s197-s217 knowledge sharing between consultants and smes development journal, 44(2): 66-69. chang, j.y.t., wang, e.t.g., jiang, j.j. & klein, g. (2013) "controlling erp consultants: client and provider practices", the journal of systems and software, 86(5): 1453-1461. doi: 10.1016/j.jss.2013.01.030 chen, r.-s., jih, w.-j., sun, c.-m. & helms, m. (2008) "role negotiation and interaction: an exploratory case study of the impact of management consultants on erp system implementation in smes in taiwan", information systems management, 25(2): 159173. doi: 10.1080/10580530801941371 chen, s., duan, y., edwards, j.s. & lehaney, b. (2006) "toward understanding interorganizational knowledge transfer needs in smes: insight from a uk investigation", journal of knowledge management, 10(3): 6-23. doi: 10.1108/13673270610670821 chou, s.-w. & he, m.-y. (2004) "knowledge management: the distinctive roles of knowledge assets in facilitating knowledge creation", journal of information science, 30(2): 146-164. doi: 10.1177/0165551504042804 corbin, j.m. & strauss, a.l. (2008) basics of qualitative research: techniques and procedures for developing grounded theory, sage, thousand oaks, calif. curran, j. & blackburn, r.a. (2001) researching the small enterprise, sage publications ltd, gb. desouza, k.c. & awazu, y. (2006) "knowledge management at smes: five peculiarities", journal of knowledge management, 10(1): 32-43. doi: 10.1108/13673270610650085 durst, s. & edvardsson, i.r. (2012) "knowledge management in smes: a literature review", journal of knowledge management, 16(6): 879-903. doi: 10.1108/13673271211276173 easterby-smith, m., thorpe, r. & jackson, p. (2008) management research, sage, london. gable, g.g. (1991) "consultant engagement for computer system selection: a pro-active client role in small businesses", information & management, 20(2): 83-93. doi: 10.1016/0378-7206(91)90046-5 gable, g.g. (1996) "a multidimensional model of client success when engaging external consultants", management science, 42(8): 1175-1198. doi: 10.1287/mnsc.42.8.1175 haines, m.n. & goodhue, d.l. (2003) "implementation partner involvement and knowledge transfer in the context of erp implementations", international journal of humancomputer interaction, 16(1): 23-38. doi: 10.1207/s15327590ijhc1601_3 howcroft, d. & light, b. (2008) "it consultants, salesmanship and the challenges of packaged software selection in smes", journal of enterprise information management, 21(6): 597-615. doi: 10.1108/17410390810911203 hunter, m.g. (2005) "international information systems and small business project: a western canadian perspective", in: m.g. hunter, s. burgess & a. wenn (eds) small business and information technology: research techniques and international case studies, heidelberg press, heidelberg. hunter, m.g., diochon, m., pugsley, d. & wright, b. (2001) "unique challenges for small business adoption of information technology: the case of the nova scotia ten", in: s. s214 australasian journal of information systems bradshaw, pulakanam & cragg 2015, vol. 19, pp. s197-s217 knowledge sharing between consultants and smes burgess (ed.) managing information technology in small business: challenges and solutions: challenges and solutions, idea group pub, hershey, pa, p. 98. ifinedo, p. (2011) "examining the influences of external expertise and in-house computer/it knowledge on erp system success", the journal of systems & software, 84(12): 20652078. doi: 10.1016/j.jss.2011.05.017 jelavic, m. (2011) "socio-technical knowledge management and epistemological paradigms: theoretical connections at the individual and organisational level", interdisciplinary journal of information, knowledge and management, 6: 1-16. ko, d.g., kirsch, l.j. & king, w.r. (2005) "antecedents of knowledge transfer from consultants to clients in enterprise system implementations", mis quarterly, 29(1): 59-85. lech, p. (2011) "knowledge transfer procedures from consultants to users in erp implementations", the electronic journal of knowledge management, 9(4): 318-327. lech, p. (2013) "functional consultants' role in enterprise systems implementations", 7th european conference on information management and evaluation, ecime 2013, reading, pp. 84-92. ministry of business, innovation and employment (mbie). (2013) the new zealand sectors report 2013: information and communications technology, http://www.mbie.govt.nz/pdf-library/what-we-do/business-growth-agenda/sectorsreports-series/sector-report-information-communication-technology.pdf [accessed 1 february 2015]. ministry of business, innovation and employment (mbie). (2014) the small business sector report 2014, http://www.mbie.govt.nz/what-we-do/business-growthagenda/sectors-reports-series/pdf-document-library/the-small-business-sectorreport-2014.pdf [accessed 1 february 2015]. nevo, s., wade, m.r. & cook, w.d. (2007) "an examination of the trade-off between internal and external it capabilities", journal of strategic information systems, 16(1): 5-23. doi: 10.1016/j.jsis.2006.10.002 nonaka, i. (1991) "the knowledge-creating company", harvard business review, 69(6): 96104. nonaka, i. & takeuchi, h. (1996) "the knowledge-creating company: how japanese companies create the dynamics of innovation", long range planning, 29(4): 592. doi: 10.1016/0024-6301(96)81509-3 nonaka, i. & toyama, r. (2005) "the theory of the knowledge-creating firm: subjectivity, objectivity and synthesis", industrial and corporate change, 14(3): 419-436. doi: 10.1093/icc/dth058 nonaka, i., toyama, r. & konno, n. (2000) "seci, ba and leadership: a unified model of dynamic knowledge creation", long range planning, 33(1): 5-34. doi: 10.1016/s0024-6301(99)00115-6 nonaka, i., von krogh, g. & voelpel, s. (2006) "organizational knowledge creation theory: evolutionary paths and future advances", organization studies, 27(8): 1179-1208. doi: 10.1177/0170840606066312 s215 australasian journal of information systems bradshaw, pulakanam & cragg 2015, vol. 19, pp. s197-s217 knowledge sharing between consultants and smes oates, b.j. (2006) researching information systems and computing, sage, london. oecd (2012) oecd internet economy outlook 2012, http://www.keepeek.com/digitalasset-management/oecd/science-and-technology/oecd-internet-economy-outlook2012_9789264086463-en page1 [accessed 1 february 2015]. ortiz de guinea, a., helen, k. & gordon, h.m. (2005) "information systems effectiveness in small businesses: extending a singaporean model in canada", journal of global information management (jgim), 13(3): 55-79. doi: 10.4018/jgim.2005070104 schiuma, g. (2012) "managing knowledge for business performance improvement", journal of knowledge management, 16(4): 515-522. doi: 10.1108/13673271211246103 soh, c.p.p., yap, c.s. & raman, k.s. (1992) "impact of consultants on computerization success in small businesses", information & management, 22(5): 309-319. doi: 10.1016/0378-7206(92)90077-s swanson, e.b. (2010) "consultancies and capabilities in innovating with it", the journal of strategic information systems, 19(1): 17-27. doi: 10.1016/j.jsis.2009.12.001 taylor, m. & murphy, a. (2004) "smes and e-business", journal of small business and enterprise development, 11(3): 280-289. doi: 10.1108/14626000410551546 thong, j.y.l. (2001) "resource constraints and information systems implementation in singaporean small businesses", omega, 29(2): 143-156. doi: 10.1016/s03050483(00)00035-9 thong, j.y.l., yap, c.-s. & raman, k.s. (1994) "engagement of external expertise in information systems implementation", journal of management information systems, 11(2): 209-231. thong, j.y.l., yap, c.-s. & raman, k.s. (1997) "environments for information systems implementation in small businesses", journal of organizational computing and electronic commerce, 7(4): 253-278. doi: 10.1207/s15327744joce0704_1 wang, e.t.g. & chen, j.h.f. (2006) "effects of internal support and consultant quality on the consulting process and erp system quality", decision support systems, 42(2): 10291041. doi: 10.1016/j.dss.2005.08.005 werner, k., dickson, g. & hyde, k.f. (2015) "learning and knowledge transfer processes in a mega-events context: the case of the 2011 rugby world cup", tourism management, 48: 174-187. doi: 10.1016/j.tourman.2014.11.003 barriers on knowledge sharing and individual innovation behaviour", international journal of knowledge management (ijkm), 9(2): 38-61. yin, r.k. (2003) case study research: design and methods, sage publications, thousand oaks, calif. yoon, s. & suh, h. (2004) "ensuring it consulting servqual and user satisfaction: a modified measurement tool", information systems frontiers, 6(4): 341-351. doi: 10.1023/b:isfi.0000046376.10364.16 s216 australasian journal of information systems bradshaw, pulakanam & cragg 2015, vol. 19, pp. s197-s217 knowledge sharing between consultants and smes copyright: © 2015 bradshaw, pulakanam, cragg. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. s217 paper title (paper title style) australasian journal of information systems zwanenburg & qureshi 2019, vol 23, selected papers from acis 2017 anticipating, avoiding, and alleviating measurement error 1 anticipating, avoiding, and alleviating measurement error: a synthesis of the literature with practical recommendations sander zwanenburg university of otago sander.zwanenburg@otago.ac.nz israr qureshi anu college of business and economics abstract researchers’ ability to draw inferences from their empirical work hinges on the degree of measurement error. the literature in information systems and other behavioural disciplines describes a plethora of sources of error. while it helps researchers deal with them when taking specific steps in the measurement process, like modelling constructs, developing instruments, collecting data, and analysing data, it does not provide an overall guide to help them prevent and deal with measurement error. this paper presents a synthesis of the insights in the literature through a decomposition of the logic of measurement. it shows how researchers can classify sources of error, evaluate their impact, and refine their measurement plans, in terms of specific steps or overall measurement approaches. we hope this will aid researchers in anticipating, avoiding, and alleviating error in measurement, and in drawing valid research conclusions. keywords measurement, construct, indicator, model, operationalization. 1 introduction the ability to draw valid inferences from empirical research in information systems and other behavioural disciplines hinges on the degree of measurement error (cote & buckley, 1988; mackenzie, podsakoff, & podsakoff, 2011). decades of methodological studies on measurement error have helped researchers and reviewers understand and recognize a variety of sources of such error (podsakoff, mackenzie, & podsakoff, 2012). they have also helped in the development of measurement guidelines (dillman, 2000; lewis, templeton, & byrd, 2005; mackenzie et al., 2011). however, measurement problems persist. in one information systems journal, for example, five of eight main obstacles that are frequent causes of desk rejects are (1) failures to recognize a lack of construct clarity, (2) common method bias, (3) formative constructs, (4) issues with self-report data, and (5) issues with data-analytic techniques (gregor & klein, 2014). existing measurement practices “fail to address the full landscape of measurement issues and fail to prioritize the fundamental aspects” (p451, burton-jones & lee, 2017). more broadly, the need for more rigor in methodology has been underscored in the establishment of the ais transactions on replication research, and in the frequent failures to reproduce findings from behavioural research (open science collaboration, 2012, 2015; servick, 2018). to help researchers measure well and draw valid conclusions, we see an opportunity in enhancing the provision of guidelines on anticipating, avoiding, and alleviating error. while australasian journal of information systems zwanenburg & qureshi 2019, vol 23, selected papers from acis 2017 anticipating, avoiding, and alleviating measurement error 2 the huge number of studies on specific sources of measurement error, such as social desirability, common method, or violations of statistical assumptions, has helped us understand these sources, they may leave many researchers confused about what needs to be done about all of these given a measurement task (bagozzi, 2011; burton-jones & lee, 2017; mackenzie et al., 2011; spector, 1992). arguably, researchers face “an almost unworkable number of tests to comply with” (burton-jones & lee, 2017, p451) and “there are so many issues to consider … that we might be apt to throw up our hands in frustration” (bagozzi, 2011, p288). the literature that considers all measurement error, on the other hand, treat it either as a whole or unpacks it into abstract components, such as bias versus random error (cote & buckley, 1988), and item-level versus construct-level error (mackenzie et al., 2011). being abstract and evaluated after data is collected, they do not always lend themselves to draw concrete implications for improving measurement plans (zyphur & pierides, 2017). compounding this issue, many guidelines that help researchers develop measurement, carry often implicit methodological assumptions (bollen & lennox, 1991; devellis, 2003; lewis et al., 2005; mackenzie et al., 2011). common assumptions are that measurement relies on reflective models, self-report questionnaires, and on one-off assessments. inadvertently, these assumptions may encourage uncritical adoption of ready-made formulas (zyphur & pierides, 2017), and prevent researchers from considering alternative methods or models of measurement, which could be valuable substitutes or complements (zwanenburg, 2015). we therefore believe that a comprehensive classification of the sources of measurement error and their potential remedies will help researchers anticipate, avoid, and alleviate measurement error through revisions of measurement plans. in this paper, we take initial steps toward this objective. based on the literature on sources of measurement error, we introduce a framework to classify these, and present the classification. we then discuss how an evaluation of their potential impact can inform decisions to improve plans of measurement. this can apply to a wide range of measurement approaches, e.g. the use of reflective, formative, and other measurement models; questionnaire surveys and other data collection approaches; qualitative and quantitative measurement variables. thus, we hope this will help many researchers systematically evaluate potential measurement error, and identify appropriate improvements to avoid or alleviate their impact. 2 the meaning and premise of measurement error across the wide range of contexts of measurement, the objective of measurement is to obtain estimates of a construct that fit the meaning of that construct. this fit is the logical basis for drawing inferences, such as research conclusions (burton-jones, 2009) and is known as the validity of measurement (markus & borsboom, 2013; nunnally & bernstein, 1994; o'learykelly & vokurka, 1998; peter, 1981).1 error of measurement is the inverse: the gap between what is to be measured and what is actually measured (nunnally & bernstein, 1994). 1 this differs from what is termed ‘construct validity,’ a property of test score interpretations (see e.g. borsboom et al. 2009, cronbach 1989). it also differs from validity as ‘the lack of systematic error’ (e.g. carmines and zeller, 1979; adcock and collier 2001), as complementary to ‘reliability’ as the lack of random error. in our definition, reliability is a form of validity. other forms, like content validity, cross validity, face validity, refer to positive findings from specific tests that can indicate problems with australasian journal of information systems zwanenburg & qureshi 2019, vol 23, selected papers from acis 2017 anticipating, avoiding, and alleviating measurement error 3 this definition is consistent with various classical theories of measurement error, viewing error as the difference between the true value and the observed value (nunnally & bernstein, 1994). true value here refers to the value that corresponds to the meaning of a construct when fully defined and applied to an instantiation.2 like its estimates, it can be categorical, continuous, or of another type. note that estimates of error, sometimes confusingly called ‘error’ or ‘standard error’, can deviate from actual error because of violations to the assumptions underlying the methods of their estimation, such as linearity and independence of indicators. what is error in a measurement thus depends on the meaning of the construct of that measurement. a lack of clarity prevents the researcher from evaluating error and thus the validity of measurement and the broader research conclusions. while a clear meaning of the construct is a straightforward premise to the evaluation of error, ambiguity is a common problem (gregor & klein, 2014; mackenzie et al., 2011). indeed, as devellis (1991, p51) noted: many researchers think they have a clear idea of what they wish to measure, only to find out that their ideas are more vague than they thought. frequently, this realization occurs after considerable effort has been invested in generating items and collecting data—a time when changes are far more costly than if discovered at the outset of the process. the meaning of a construct is clear when (a) the instantiations of the construct are clear, (b) each instantiation can only have one true value, and (c) the construct is embedded in a framework of other constructs. 2.1 clear instantiations instantiations of a construct can refer to entities or relationships between them (burton-jones & lee, 2017).3 for example, an app’s usefulness can be thought of as an attribute of that app but perceptions of its usefulness are attributes of the relationship between this object and its observer (gregor & klein, 2014). the difference is critical, and choosing one over the other can carry many implications for the design of measurement. for example, if the target of measurement is the app’s usefulness and multiple perceptions are a meansof accessing that, how are different perceptions combined in a measurement model? another important consideration in clarifying the instantiations of a construct is their relation to time. when targets of measurement are not tied to a particular moment or event, they can be subject to change over time. this can be slow or rapid, continuous or incidental: an app’s user might ‘get the hang of it’, or might become frustrated with a bad update. one instantiation validity based on the domain of the construct, the sample, or the inspection of measurement respectively. 2 true values are sometimes called true scores, which operationalists see as outcomes of measurement processes themselves (nunnally & bernstein, 1994). this view is problematic because it detaches the concept of validity from the meaning of constructs (markus & borsboom, 2013). 3 some authors use the terms object and entities interchangeably. here, the word object is used in relation to the measurement target itself and entity in relation to possible referents of it, as implied by the meaning of a construct and its instantiations. the object of a measurement can also refer to a relationship between entities. australasian journal of information systems zwanenburg & qureshi 2019, vol 23, selected papers from acis 2017 anticipating, avoiding, and alleviating measurement error 4 of it can be a selection from multiple moments, or an aggregate.4 again, the difference can be critical. in sum, to evaluate measurement validity, it must be clear what the instantiations of a construct refer to structurally and temporally. 2.2 one instantiation, one value a common issue with constructs is having multiple possible true values per instantiation, often due to multiple interpretations. an example is the construct of smartphone use frequency. assuming a questionnaire is a method of choice, one might try measure it by asking, “how often do you use your smartphone?” with answer options ranging from never to very often. but what is often? a respondent may evaluate this through comparisons over time, comparisons with perceptions of their peers’ behaviour, or with perceived domestic, organisational, or societal norms (tourangeau, rips, & rasinski, 2000; zyphur & pierides, 2017). different evaluations may correspond to different true values, introducing ambiguity and impeding the evaluation of validity. they may be prevented through a clarification, ultimately based on previous work, standards in the literature, nomological networks, and/or the specific purposes of the measurement (zyphur & pierides, 2017). for example, if for the specific purposes of the measurement it is desirable to have a social-relative measure, one can refine the definition of the construct and the operations of its measurement items to prevent this ambiguity (e.g. compared to your peers, how often do you use your smartphone?). 2.3 interrelated constructs elucidating the relationships between related constructs will help identify ways to define a construct and measure it. such relations may be deterministic or probabilistic. when they are known, they constitute a construct’s ‘internal theory’ (goertz, 2006; mackenzie et al., 2011). some relations might be the object of a study. they may also be part of or implied by the definition of the construct, such that measurement of the related concepts can be part of the measurement of the focal construct. for example, someone’s usage of a device will be equivalent to the usage of the device itself, which may be captured in automatic logs, minus the usage of the device’s other users. potentially, capturing these related concepts to measure the construct may produce better validity than a direct approach. thus, an evaluation of related constructs not only aids the clarity of the construct, it can also identify ways to measure it. in sum, identifying the instantiations, the multiplicity of their true values, and the relations with other constructs will be instrumental in anchoring this meaning conceptually and satisfying the premise of anticipating, avoiding, and alleviating error. 3 a classification of sources of error sources of error undermine the logic of measurement in different ways and at different places. it may stem from random processes, affecting each data point separately, or systemic ones, affecting one or multiple arrays of data points, within or across data sets (cote & buckley, 1988). it may be attributed to a variety of aspects of a measurement, such as an instrument, instructions of a questionnaire, a question’s wording or content, a response scale, a sequence 4 in this paper, selecting and sampling instantiations to measure are important study design considerations but are not treated in this paper. while unmeasured instantiations are of concern to drawing research inferences, they are outside of the scope of measurement proper. australasian journal of information systems zwanenburg & qureshi 2019, vol 23, selected papers from acis 2017 anticipating, avoiding, and alleviating measurement error 5 of questions, a trait or state of a respondent, a measurement model, an overall method of collecting data, and assumptions of estimation techniques. error may be attributable to an interplay of factors associated with these aspects. to allow researchers to systematically identify and evaluate potential sources of error that threaten the validity of their measurement, we first consider the logic of measurement in general and decompose it into conceptual, operational, and inferential components. we then illustrate how this logic can be further unpacked for particular instances of measurement. in the last subsection, we will illustrate how this allows for a tailored classification of sources of error. 3.1 decomposing the logic of measurement let us consider a single measurement instantiation such as the usefulness of a new app, using a non-zero set of indicators, which can be based, for example, on one or multiple questions, assessments, informants, or other sources of information. logically, the measurement task is to use these indicators to connect the meaning of this construct’s instantiation, i.e. an unobserved true value, with an estimate value. unpacking this logical relationship can help classify error, as illustrated in figure 1. figure 1: left: a decomposition of the logical relationship of measurement into conceptual, operational, and inferential relationships, using three indicators. right: while some error may be controlled for, various sources of error can affect the conceptual, operational, and inferential relationships (marked ce, oe, and ie for conceptual, operational, and inferential error). this figure distinguishes between the targets of a measurement (i.e. what is to be captured) in white shapes and the actual data (i.e. what is actually captured) in black shapes. at a generic level, we can decompose the logical relationship into conceptual, operational, and inferential ones, as shown in the left panel. in the diagram, the meaning of the construct on top is translated into referents of three indicators, which are operated to yield three records, australasian journal of information systems zwanenburg & qureshi 2019, vol 23, selected papers from acis 2017 anticipating, avoiding, and alleviating measurement error 6 which are then combined to infer an estimate of the construct. as visualised in the right panel, sources of error can introduce interference to this logical flow at each of these steps, some of which can be removed though data analytical techniques. 3.1.1 conceptual relationships here, a conceptual relationship refers to the relation between the meaning of a construct and what an indicator is to record, i.e. its referent. this referent may be identical to the meaning of the construct, or it may be a part, cause, effect, or manifestation of it: anything that stands in some relation to the construct (law, wong, & mobley, 1998; mackenzie et al., 2011; polites, roberts, & thatcher, 2012). for example, the likert-type item “learning to operate the web site is easy” carries meaning that can be seen as identical to or a manifestation of the meaning of the construct, the ease of use of the same web site (gefen, karahanna, & straub, 2003). when an item would refer to ease of use in navigating the web site, this can be seen as referring to part of the meaning of the construct. conceptual relationships can be compounded (edwards & bagozzi, 2000). such compounded relationships are sometimes explicitly modelled using sub-constructs (law et al., 1998; polites et al., 2012). when accurate knowledge of the referents perfectly inform the true value of the construct, there is no error at this conceptual level. for example, if we know how much someone uses the web per weekday and in the weekend we can infer how much this person uses the web during the week, without error. conceptual relationships are erroneous, however, when referents of indicators relate to the meaning of the construct in a non-deterministic way. for instance, measuring extraversion with a question about the frequency of going to parties relies on the idea that more extraverted people tend to go to parties more. error may also relate to the referents of a set of indicators. this set may lack content validity, such as when they comprise a list that lacks mutual exclusivity or completeness, or when they include extraneous items (haynes, richard, & kubany, 1995; lewis et al., 2005; messick, 1989). 3.1.2 operational relationships an operational relationship refers to the relation between the referent of an indicator and its record, what is actually recorded. error at this level stems from the actual physical and psychological processes that influence the generation of a datum. for question and answerbased measurement, for example, these include capturing the referent of an indicator in a question, expressing that question, hearing or reading it, interpreting it, evaluating it, responding to it, and capturing that response in a record (dillman, 2000). much error in question-based measurement stem from these processes (podsakoff et al., 2012; tourangeau et al., 2000). as indicated in the right panel of figure 1, a source of error can introduce interference in multiple operations. for example, social desirability error or error stemming from an unclear questionnaire introduction can affect the validity of some or all of a construct’s indicators. 3.1.3 inferential relationships an inferential relationship refers to the relation between the records of the indicators and the estimate of a construct, as captured in mathematical and logical equations and operations. inferential relationships can be modelled additively, multiplicatively, or otherwise, based on reflective, formative, and other models (mackenzie et al., 2011; mellenbergh, 1994; polites et al., 2012). this is based on the conceptual and operational relationships including an understanding of how sources of error could affect them. for example, factor analytical australasian journal of information systems zwanenburg & qureshi 2019, vol 23, selected papers from acis 2017 anticipating, avoiding, and alleviating measurement error 7 approaches typically assume that random error has affected each record, following a normal distribution with mean zero and a standard deviation that is estimated (bollen & lennox, 1991; nunnally & bernstein, 1994). hence, these procedures can control for error to the degree it agrees with these assumptions. when these inferences are based on invalid assumptions or are erroneously implemented, they may fail to properly control for error and can introduce new error in the measurement process (rigdon, 2012). for example, a factor model may be mis-specified or unidentifiable leading to false conclusions (aguirre-urreta & marakas, 2012; anderson & gerbing, 1988; jarvis, mackenzie, & podsakoff, 2003). together, the conceptual, operational, and inferential components of the logic of measurement constitute a generic error classification framework. it is complete and non-exclusive in that any source of measurement error can be associated with a shortcoming in these components. it can be used to structure and systematize the evaluation of sources of error regardless of the form or method of measurement, as no methodological assumptions have been made. however, because of this, these relationships can still be both abstract and complex. unpacking them further for specific instances of measurement can shed further light on the sources of error that can threaten their validity. 3.2 identifying specific logical links the conceptual, operational, and inferential relationships can be abstract and complex as they may constitute a chain of logical links (law et al., 1998; polites et al., 2012; tourangeau et al., 2000). these more elementary links may connect one idea to one or multiple others, such as a concept to a proxy, an abstraction to its manifestations, a whole to its parts, a cause to one or multiple effects. elucidating these links will ease the identification of sources of errors. as an extreme example, consider capturing the ease of use of an app with a single estimate by using four-dimensional indicator data involving (1) multiple parts (e.g. functions) of that app, (2) multiple users (who act as informants), (3) multiple questions, and (4) multiple times of assessment. here, four consecutive one-to-multiple links can bridge this zero-dimensional construct (i.e. its estimate is a single point or a zero-dimensional array5) with the 4d indicators at the conceptual level, as illustrated in figure 2. further consecutive links unpack the causal chain of the measurement operations, from presenting a specific question to recording a response.6 at each consecutive link, we can identify potential sources of error by asking specific questions, like those listed on the right hand side of the figure. thus, decomposing the logic of a measurement into elementary logical can help researchers obtain a tailored error classification framework. 5 the word ‘dimension’ is used here in the context of a dataset, where aspects or facets of a construct may be positions on one dimension (e.g. its composition); elsewhere these aspects or facets may be called dimensions themselves (law et al., 1998; polites et al., 2012). 6 in this figure, the inferential relationship is not further decomposed. while some of its steps can be modelled as sequential logical links, such as data cleaning and dealing with missing data, its estimation step is often iterative in nature. australasian journal of information systems zwanenburg & qureshi 2019, vol 23, selected papers from acis 2017 anticipating, avoiding, and alleviating measurement error 8 figure 2: this example illustrates how decomposing the logic of a measurement into consecutive logical links can help researchers identify specific sources of error. 3.3 tying logical links to sources of error extant knowledge of patterns observed before can inform which sources of error are associated with these links. table 1 illustrates this by classing sources of error of a measurement according to its consecutive logical links involving one reflective model and one-off selfreported questionnaire data. table 2 classes common sources of error associated with a variety of logical links that may be applied in a modular fashion in a variety of measurements. australasian journal of information systems zwanenburg & qureshi 2019, vol 23, selected papers from acis 2017 anticipating, avoiding, and alleviating measurement error 9 logical link associated sources of error construct to manifestation a well-defined construct may be inappropriately modelled in terms of its manifestations (aguirre-urreta & marakas, 2012; jarvis, mackenzie, & podsakoff, 2012; petter, rai, & straub, 2012). a proclaimed manifestation may in fact not be consistent with the definition of the construct. it may be caused by other constructs, or the construct may only give rise to it under conditions that are not met, or it may not stand in a relationships to it as modelled (cook & campbell, 1979; rigdon, 2014a). manifestation to question a question may inappropriately capture a manifestation of a construct. for example, a question may refer to something else or may be unclear (e.g. dillman, 2000; netemeyer, bearden, & sharma, 2003; tourangeau et al., 2000). its response scale may be confusing, inconsistent with the question, or unable to capture accurate answers (e.g. verhagen, van den hooff, & meents, 2015). question to evaluation participants may inappropriately evaluate a question. for example, they may lack the motivation, energy, vocabulary, and other cognitive abilities to do so (churchill, 1979; nunnally & bernstein, 1994; tourangeau et al., 2000; viswanathan, 2005). a question may be too difficult, inaudible, or illegible. the instructions may be unclear and the time pressure and incentives may be inappropriate. the time of the day, the location, and the order of the questions may have an unintentional influence on their evaluations (dillman, 2000; drury & farhoomand, 1997; harrison, mclaughlin, & coalter, 1996; podsakoff et al., 2012; schwarz & sudman, 1992). a participant may have distracting thoughts and feelings while evaluating a question, due to idiosyncratic associations with certain words, or perceptions of fatigue, hunger, pain, noise, a phone ringing, or even due to simultaneous actions (e.g. edwards, 2008). evaluation to response an evaluation may not be reported. questions may be too sensitive to answer honestly (e.g. dillman, 2000; netemeyer et al., 2003; tourangeau et al., 2000). the participant’s anonymity or the lack thereof may affect the honesty of the response. a participant may have certain response tendencies (e.g. podsakoff, mackenzie, lee, & podsakoff, 2003), or lack the motivation or incentives to provide accurate answers (e.g. aronson, wilson, & brewer, 1998; podsakoff et al., 2003; podsakoff et al., 2012; richman, kiesler, weisband, & drasgow, 1999). response to record an accurate response may be inappropriately recorded due to its illegibility, a data entry mistake, a technical failure, and so on. record to factor score a well-recorded response may not be used appropriately for calculating factor scores as its assumptions may be violated. the specification of the factor model may deviate from the conceptual model, i.e. it may deviate from the specified relations between a construct and the referents of its indicators (e.g. petter, straub, & rai, 2007; rigdon, 2013). factor score to estimate a factor score may be an inappropriate estimate when the assumptions underlying the factor analysis are violated. these typically include local independence, linearity of relationships, and homogeneity of relationships across entities (e.g. becker, rai, ringle, & völckner, 2013; havlicek & peterson, 1977; jarque & bera, 1987; meredith, 1993; petter et al., 2007). further, factors scores are indeterminate: their validity depends on the arbitrary method chosen to calculate them (mulaik, 2010; rigdon, 2012). table 1: sources of error associated with common logical links in a typical case of measurement australasian journal of information systems zwanenburg & qureshi 2019, vol 23, selected papers from acis 2017 anticipating, avoiding, and alleviating measurement error 10 link associated sources of error construct to referent a construct may be inappropriately modelled in terms of its manifestations, effects, constituent parts, dimensions, or other referents of indicators that stand in some relation to the construct (goertz, 2006; petter et al., 2012). for example, a specification of its parts may be incomplete, superfluous, or it may contain overlapping parts (haynes et al., 1995; mackenzie et al., 2011). a dimensional model may inappropriately specify how dimensions combine to make up the construct (law et al., 1998). referent to referent the referent of an indicator may be modelled in terms of its own referents, yielding a hierarchical model (e.g. edwards, 2001; law et al., 1998; polites et al., 2012). these links can suffer from the same sources of errors as the links between the construct to be measured and the referents of its immediate indicators. referent to detection a referent of an indicator may be implemented through autonomous systems and physical detectors, such as those that automatically log events or aim to detect heart rate, skin conductance, eye movements, gamma waves, and so forth. errors may stem from the processes of designing, installing, calibrating, and operating the instrument, depending on the specific apparatus. logs of device use, for example, may be inappropriately ascribed to a principal user and ignore the use of replacements and other alternatives. further, using detectors to infer referents of more abstract psychological constructs can be problematic (dimoka, 2012; fazio & olson, 2003). for example, we still know little about how to best infer people’s stress, affect, and reward from detections of skin conductance, heart rate variability, and activation of the nucleus accumbens (basal forebrain) respectively (e.g. carlson, 2013). referent to record a referent of an indicator may be linked directly to a record when relying on past measurements, or ‘secondary data’, stored in databases, documents, and logs. the sources of error corresponding to this link include all inconsistencies between the original measurement operations and what the result of these operations – i.e. the record – is taken to mean (burton-jones & lee, 2017; ketchen, ireland, & baker, 2013; wennberg, 2005). referent to stimulus a referent of an indicator may be inappropriately implemented into a question, picture, sound, or any other linguistic or non-linguistic stimulus. for example, a sound may be inaudible, a question illegible, or the membership of a stimulus to an intended category may be ambiguous (e.g. dillman, 2000; greenwald, mcghee, & schwartz, 1998). stimulus to response a good stimulus may not produce the appropriate response, when, for example, the instructions are unclear or evoke an inappropriate degree of time pressure, social pressure, or other forms of stress. the lag with which stimuli are presented may obscure the interpretation of response times (e.g. greenwald et al., 1998). record to estimate records may be inappropriately combined to produce estimates. data cleaning steps may involve clerical errors. missing data may be inputted inaccurately due to the shortcomings of the procedures (little & rubin, 2014; schafer & graham, 2002). the mathematical procedures of estimation may involve assumptions that do not hold (jarvis et al., 2012; rigdon, 2012; viswanathan, 2005). record to model estimate records can be used to test the fit of models or the support for inter-construct hypotheses without separately obtaining estimates for constructs. sources of error are violations to the assumptions underlying these estimation techniques, related to linearity, normality, measurement invariance, or independence of distributions (e.g. becker et al., 2013; havlicek & peterson, 1977; jarque & bera, 1987; meredith, 1993; mulaik, 2010; petter et al., 2007; rigdon, 2012). table 2: sources of error associated with logical links of measurement 4 the management of sources of error evaluating a plan of measurement and informing its revisions require not only an understanding of where error comes from, but also what the impact of its sources are. this impact depends on how much interference each source introduces at each logical link, and how much of this interference is transformed, e.g. by way of dilution, compensation, or australasian journal of information systems zwanenburg & qureshi 2019, vol 23, selected papers from acis 2017 anticipating, avoiding, and alleviating measurement error 11 statistical control. in other words, sources of error are best managed with an understanding of the patterns in where and how much they impact a measurement system. 4.1 patterns in error sources of error require most attention when they are systemic rather than incidental, i.e. when they affect the entire measurement system, or a subsystem.7 figure 1 illustrates examples of sources of error that are incidental (labelled oe2), systematic (labelled ce1, ce2, oe3, ie1), and in between incidental and systematic (labelled oe1). sources of error are more systemic when they apply to more dimensions of the indicator data, whether they do so at the conceptual, operational, or inferential level. for example, if the indicators refer to effects of a construct, two systemic and conceptual sources of error are (1) the existence of alternative explanations that can account for the effects and (2) the construct failing to cause the effects. examples of systemic operational sources of error are violated assumptions underlying the indicators, e.g. when respondents do not have the required knowledge, ability, or motive to answer questions accurately. an incidental operational error may be due to one question being ambiguous. an example of a systemic source of error at the inferential level is a violation to the assumptions underlying estimation, causing new interference across estimates. patterns of systemic error can be related to not only the aspects of a chosen measurement method (e.g. particular items, instruments, assessments or estimation procedures), also with the meaning of the construct itself. figure 3 illustrates this with a joint distribution of four separate continuous indicators and the continuous construct they refer to. here, error in indicator a is independent of the construct, while error in indicators b, c, and d co-varies with the construct. figure 3: illustrations of joint distributions of indicators and the construct they refer to (both continuous). error in indicator a is uniform, while error in indicators b, c, and d co-vary with the construct. consider one specific source of error in a measurement where an indicator refers to the frequency of an event that is an effect of the construct, and the effect could also be triggered by other causes. here, the joint distribution might be similar to that of indicator b in figure 3, where high levels of the indicator tell us little about the construct. if the indicator would feature in a standard factor analysis, this error would violate the assumption that 7 we can speak of subsystems when constructs are modelled as consisting of multiple sub-constructs, when data is collected repeatedly, or from multiple informants, or when data is multidimensional for another reasons. australasian journal of information systems zwanenburg & qureshi 2019, vol 23, selected papers from acis 2017 anticipating, avoiding, and alleviating measurement error 12 measurement error is normally and independently distributed as the level of error depends on the level of the construct. another example is the bias introduced through socially desirability. when the response scale of an indicator refers to behaviours that vary in their social desirability, the values corresponding to the least socially desirable should be more indicative of the actual behaviours than those corresponding to the others. similarly, when, for example, a computing skill is being tested with a quiz, and a question is very easy, only wrong answers are indicative of the level of skill. this may follow a joint distribution similar to indicator d in figure 3. in the case of categorical variables, such as a condition being either present or absent, there too can patterns in error co-vary with true values. for example, an indicator will have high sensitivity (i.e. high true positive rate) and low specificity (i.e. low true negative rate), when it is based on manifestations that are necessary but not sufficient. vice versa, an indicator will have low sensitivity and high specificity when it relies on a manifestation that is sufficient but not necessary. anticipating these patterns in potential measurement error can help inform the remedies that can best improve the validity of a plan of measurement. 4.2 remedies whether a source of error is associated with one indicator or an entire system of measurement, four categories of remedies may be considered: modify, control, add, and drop. 4.2.1 modify the most preferred remedy is to modify a plan of measurement such that it no longer is sensitive to the source of error, without exposing it to another. for example, indicator b, c, or d from figure 3 would be improved if it could be modified such that it would behave according to indicator a. for example, when the referent of a categorical indicator is necessary but not sufficient to indicate a particular construct, modifying it to one that is both necessary and sufficient would improve its usefulness. to illustrate, consider the item “based on my experience with the online vendor in the past, i know it is trustworthy” used to indicate trust in the online vendor (gefen et al., 2003). arguably, one might trust online vendors based on reputation, reviews, or certificates, rather than one’s experience with it, and thus the indicator may underestimate trust. modifying a set of items by including a complete range of trust sources, or removing the references to these sources altogether, will remove this particular source of error. while various guidelines in the literature help researchers deal with specific item-specific issues (clark & watson, 1995; dillman, 2000; haynes et al., 1995; mackenzie et al., 2011; tourangeau et al., 2000), protecting them against interference is easier said than done (spector, 2006). interference tends to stem from a complex and hidden interplay of contextual, idiosyncratic, and circumstantial factors. for example, a researcher may not know fully the conditions under which a construct gives rise to its manifestations. it may be impossible to ensure that participants interpret all questions as intended and report answers accurately. sometimes, shielding the measurement process against one threat exposes it to another. for example, indirect questioning may help prevent socially desirable responses but it may capture content outside of the construct’s domain (fisher, 1993). one may wish to rely on methods different from question-and-answer by logging events or recording response times, eye movements, skin conductance, or other physical phenomena (bradley, greenwald, petry, australasian journal of information systems zwanenburg & qureshi 2019, vol 23, selected papers from acis 2017 anticipating, avoiding, and alleviating measurement error 13 & lang, 1992; carlson, 2013; segerstrom & nes, 2007). as shown in table 2, such alternative means may suffer from their own sources of error that are hard to prevent. inevitably, potential threats to validity can be found along the entire logical relationship between a construct and its estimate. in spector’s (2006, p230) words, “each operationalization of a variable or methodtrait combination carries with it a unique set of potential biases.” 4.2.2 control a second category of remedies is to deal with the interference from a given source of error and control for it, through measured or unmeasured approaches. for example, instruments that measure a respondent’s sensitivity to give social desirable rather than honest answers can be used as a measured control for social desirability bias (paulhus, 1988). similarly, other response tendencies such as acquiescence can be measured to allow for statistical control (paulhus, 1991; winkler, kanouse, & ware, 1982). while such measured approaches are more precise than unmeasured ones, they do impose an operational burden; it is impractical to measure many sources of error. unmeasured approaches to control for error are common in factor analytic approaches, where error is modelled as affecting the indicator data. typically, in factor analysis each unique indicator, such as a questionnaire item, is modelled as affected by random error that is completely independent and normally distributed around zero (harman, 1976; nunnally & bernstein, 1994). while this does account for sources of error that behave accordingly, in many measurements indicators resemble one another – they may rely on the same questionnaire, assessment, response scale, and relation to the construct – often meaning that they share sources of error. sometimes, such common method error is modelled as affecting a set of indicators (harman, 1976; lindell & whitney, 2001; tehseen, ramayah, & sajilan, 2017; williams, hartman, & cavazotte, 2010). while popular, this technique can be problematic, as it relies on various assumptions (chin, thatcher, & wright, 2012). a common assumption is that the common method error can be modelled as a unitary construct, which limits the ability to control for separate common sources of error, such as both acquiescence and social desirability. while unmeasured approaches to control for sources of error do not directly impact the operations of measurement, the more realistic their models, the more data is needed to identify these models statistically. hence, measured and unmeasured approaches come with operational costs and constraints respectively. knowledge of the specific sources of error and their behaviour will help inform how to implement these approaches with reasonable assumptions, to the best effect. 4.2.3 add another way to mitigate the impact of sources of error is by introducing indicators, methods, or other measurement subsystems, that are less sensitive to these sources of error (campbell, 1957; podsakoff et al., 2012). as a result, these sources of error become less systemic and incidental to a smaller part of the measurement plan; error would be diluted (peter, 1981). ways to diversify a set of indicators include appending self-report with peer-report; a single survey with the momentary assessment method; and a measurement of a construct as a sum of its parts with an indicator that refers to the construct as a cause of its effects. diversifying indicators works well when new indicators are not clearly inferior to extant ones (burtonjones, 2009), but complement them in terms of their ‘error profile’. the fewer indicators a source of error affects, the more limited its impact on the validity of measurement (burtonaustralasian journal of information systems zwanenburg & qureshi 2019, vol 23, selected papers from acis 2017 anticipating, avoiding, and alleviating measurement error 14 jones, 2009; houts, cook, & shadish, 1986; nunnally & bernstein, 1994). as a peculiarity of just one indicator a source of error often has the least impact on the validity of measurement, depending on the reliance on that indicator in the method of estimation. apart from diversifying a set of indicators, one can also specifically target a new indicator that is less sensitive to a given source of error. for example, indicators b and c in figure 3 complement one another by being sensitive to opposite ends of the scale of the construct. adding c to b or conversely may be especially worthwhile if either cannot be changed in an indicator that behaves more like indicator a, noting that this non-traditional use of indicators does require compatible estimation techniques. consider, for example, two items in a computer aptitude test, one difficult – and thus indicative of aptitude only at the higher end – and one easy, and thus indicative only at the lower end. such items can be combined with models based on item response theory such that the combined test can be indicative of aptitude along its entire scale (embretson & reise, 2013). while introducing dissimilar indicators may be operationally expensive, it can effectively reduce the impact of multiple sources of error that are specific to the extant set of indicators. if the new indicators themselves do not introduce a problematic degree of error, this approach is especially worthwhile. 4.2.4 drop in some cases, simply dropping an indicator is an effective way of removing error (mackenzie et al., 2011). it can only be done, however, when the indicator is redundant in terms of its referent and method. that is, it cannot be culled if either its meaning or the means through which it accesses this meaning plays a critical role in the logic of measurement. in sum, the degree with which sources of error undermine the validity of measurement depends on where they interfere, how severely they interfere, and how much of their impact can be reduced. often, their presence is inevitable and their impact elusive. yet even while deficient, a thorough evaluation of these sources of error should inform the design of measurement as it can lead to better validity. 5 discussion this paper has provided a framework for unpacking the logical relationship of measurement, used that framework to evaluate and classify sources of error, and provided a strategy for the identification of remedies based on this classification. we believe the framework, classification, and the strategy can guide researchers as it builds on and synthesizes existing literature. by decomposing the logic of measurement into conceptual, operational, and inferential relationships, the framework ties in literature that is often unjustifiably disjointed. work on constructs (e.g. barki, 2008; goertz, 2006) and conceptual modelling (e.g. polites et al., 2012), including the discussion on reflective and formative models (e.g. jarvis et al., 2012), is often disjointed from literature on the practical operations of measurement, even though it carries direct implications for the operations of measurement (e.g. “what questions can we ask and are these any good?”). similarly, while the literature on estimation and the use of statistical techniques and tests is reliant on broad assumptions regarding the conceptual and operational relationships, it rarely examines these (rigdon, 2014b; zyphur & pierides, 2017). whether they are conceptual, operational, or inferential, sources of error are manifold and entangled; dealing australasian journal of information systems zwanenburg & qureshi 2019, vol 23, selected papers from acis 2017 anticipating, avoiding, and alleviating measurement error 15 with one often raises other concerns. appropriate management of these thus requires a holistic, systemic view. our framework, classification, and strategy help with this in various ways. first, decomposing the logical relationship of measurement into conceptual, operational, and inferential relationships, the framework eases a complete and systematic evaluation of sources of error, since the relationships are mutually exclusive and completely exhaustive. as demonstrated with the classification of error, this evaluation can be further eased by breaking down these relationships into elementary logical links, allowing for a measurement-specific classification of potential error. tying the identification of sources of error to this logical chain of measurement is arguably more intuitive than the more traditional approach of tying it to entities within measurement such as constructs or indicators – see e.g. the discussion on construct-level and item-level error in mackenzie et al. (2011). ultimately, sources of error affect the processes of measurement or the relationships between such entities, not entities themselves. therefore, our recommendations should further help researchers in evaluating sources of error beyond the help provided by existing error classification frameworks. second, the recommendations we offer stimulate a complete evaluation of sources of error rather than a more stepwise or ad-hoc evaluation. for example, past guidance in the development of one-off questionnaire instruments has recommended the generation of items first, then the specification of the measurement model, and later still the estimation procedures, with the consideration of potential sources of error at each step (mackenzie et al., 2011). while these procedures do allow for iteration and subsequent refinement, at each step only narrow, local remedies are sought, under the implicit assumption the measurement approach has been set in stone. common assumptions are that a single assessment is to take place, to have a single informant per instantiation, to use a questionnaire, to use reflective models, and to model the indicators as linear combinations of the construct and normally distributed error (churchill, 1979; devellis, 2003; mackenzie et al., 2011). these choices are limiting themselves (burton-jones, 2009; podsakoff et al., 2003; rigdon, 2013), and assuming them implicitly does not inspire a holistic view on the entire measurement plan. our recommendations encourage taking such a view, and revising measurement plans through its models, data collection techniques, informants, assessments, estimation techniques etc. in particular, they encourage adding methods that are maximally different, as these are least likely to share sources of error (campbell & fiske, 1959; peter, 1981), and best help to triangulate measurement. at the very least, being method-agnostic, the recommendations may alleviate the issue of dogmatic application of ready-made formulas (zyphur & pierides, 2017). relatedly, our recommendations help advance the consideration of measurement error in the development process. most of the discussion of measurement error in the literature assumes data has been collected, when considerable effort has been spent and opportunities for avoiding and alleviating sources of error have narrowed. it focuses on statistical measures of various forms of validity, or lack thereof, and not on taking measures to improve the actual validity of measurement (zyphur & pierides, 2017). while less quantitative, a-priori evaluation of potential error can thus be more instructive. this paper has taken a first step in integrating extant insights into sources of error and offering recommendations to researchers with measurement plans for evaluating a-priori and systematically potential sources of error, whether these are conceptual, operational, or inferential. such a comprehensive evaluation provides a basis for improving a plan of australasian journal of information systems zwanenburg & qureshi 2019, vol 23, selected papers from acis 2017 anticipating, avoiding, and alleviating measurement error 16 measurement in an integrated, rather than ad-hoc manner, and allow for the identification of remedies that address multiple sources of error at once. as such, this strategy can complement and enhance existing approaches toward the development of measurement (mackenzie et al., 2011). future steps to better measurement guidance could help researchers exploit practical opportunity for creative measurement solutions and simultaneously follow well-established best practices (burton-jones, 2009). for example, much measurement in the academic behavioural disciplines is part of a broader attempt to confirm structural models, where structural equation modelling is a well-established approach to do so. while it is often implemented with reflective measurement models where each indicator corresponds to one question on a manifestation of its construct, asked in a one-off self-report questionnaire, there are many possible variations. within that approach, some literature has dealt with the use of alternative measurement models (diamantopoulos, 2011; diamantopoulos & temme, 2013; polites et al., 2012), and other literature has dealt with the use of indicator data from repeated assessment (crowder, 2017; muthén, 2002). a synthesis of streams of work like these positioned in the context of measurement development could help researchers identify and evaluate a broad range of possibilities, such that measurement can suit its circumstances (zyphur & pierides, 2017). this would more clearly show possible responses to the evaluation of sources of error that undermine a more traditional plan of measurement. hence, our hope is that this paper will inspide further steps in the development of guidelines that help researchers anticipate, avoid, and alleviate source of measurement error. it should pave the way for well-designed, valid measurement within our disciplines. references aguirre-urreta, m. i., & marakas, g. m. (2012). revisiting bias due to construct misspecification: different results from considering coefficients in standardized form. mis quarterly, 36(1), 123-138. anderson, j. c., & gerbing, d. w. (1988). structural equation modeling in practice: a review and recommended two-step approach. psychological bulletin, 103(3), 411-423. aronson, e., wilson, t. d., & brewer, m. b. (1998). experimentation in social psychology. bagozzi, r. p. (2011). measurement and meaning in information systems and organizational research: methodological and philosophical foundations. mis quarterly, 35(2), 261-292. barki, h. (2008). thar's gold in them thar constructs. acm sigmis database, 39(3), 9-20. becker, j.-m., rai, a., ringle, c. m., & völckner, f. (2013). discovering unobserved heterogeneity in structural equation models to avert validity threats. mis quarterly, 37(3), 665-694. bollen, k., & lennox, r. (1991). conventional wisdom on measurement: a structural equation perspective. psychological bulletin, 110(2), 305. bradley, m. m., greenwald, m. k., petry, m. c., & lang, p. j. (1992). remembering pictures: pleasure and arousal in memory. journal of experimental psychology: learning, memory, and cognition, 18(2), 379. burton-jones, a. (2009). minimizing method bias through programmatic research. mis quarterly, 33(3), 445-471. australasian journal of information systems zwanenburg & qureshi 2019, vol 23, selected papers from acis 2017 anticipating, avoiding, and alleviating measurement error 17 burton-jones, a., & lee, a. s. (2017). thinking about measures and measurement in positivist research: a proposal for refocusing on fundamentals. information systems research, 28(3), 451-467. campbell, d. t. (1957). factors relevant to the validity of experiments in social settings. psychological bulletin, 54(4), 297. campbell, d. t., & fiske, d. w. (1959). convergent and discriminant validation by the multitrait-multimethod matrix. psychological bulletin, 56(2), 81. carlson, n. (2013). physiology of behavior (11 ed.). new jersey: pearson education, inc. chin, w. w., thatcher, j. b., & wright, r. t. (2012). assessing common method bias: problems with the ulmc technique. mis quarterly, 36(3), 1003-1019. churchill, g. a. (1979). a paradigm for developing better measures of marketing constructs. journal of marketing research, 16(1), 64-73. clark, l. a., & watson, d. (1995). constructing validity: basic issues in objective scale development. psychological assessment, 7(3), 309. cook, t. d., & campbell, d. t. (1979). quasi-experimentation: design and analysis for field settings: rand mcnally. cote, j. a., & buckley, m. r. (1988). measurement error and theory testing in consumer research: an illustration of the importance of construct validation. journal of consumer research, 14(4), 579-582. cronbach, l. j. (1989). construct validation after thirty years. intelligence: measurement, theory, and public policy, 3, 147-171. crowder, m. (2017). analysis of repeated measures: routledge. devellis, r. f. (1991). scale development: theory and applications. newbury park, ca: sage publications. devellis, r. f. (2003). scale development: theory and applications (vol. 2nd). thousand oaks, california: sage publications. diamantopoulos, a. (2011). incorporating formative measures into covariance-based structural equation models. mis quarterly, 35(2), 335-358. diamantopoulos, a., & temme, d. (2013). mimic models, formative indicators and the joys of research. ams review, 3(3), 160-170. dillman, d. a. (2000). mail and internet surveys: the tailored design method (vol. 2): wiley new york. dimoka, a. (2012). how to conduct a functional magnetic resonance (fmri) study in social science research. mis quarterly, 36(3), 811-840. drury, d. h., & farhoomand, a. (1997). improving management information systems research: question order effects in surveys. information systems journal, 7(3), 241-251. edwards, j. r. (2001). multidimensional constructs in organizational behavior research: an integrative analytical framework. organizational research methods, 4(2), 144-192. australasian journal of information systems zwanenburg & qureshi 2019, vol 23, selected papers from acis 2017 anticipating, avoiding, and alleviating measurement error 18 edwards, j. r. (2008). to prosper, organizational psychology should… overcome methodological barriers to progress. journal of organizational behavior, 29(4), 469-491. edwards, j. r., & bagozzi, r. p. (2000). on the nature and direction of relationships between constructs and measures. psychological methods, 5(2), 155. embretson, s. e., & reise, s. p. (2013). item response theory: psychology press. fazio, r. h., & olson, m. a. (2003). implicit measures in social cognition research: their meaning and use. annual review of psychology, 54(1), 297-327. fisher, r. j. (1993). social desirability bias and the validity of indirect questioning. journal of consumer research, 303-315. gefen, d., karahanna, e., & straub, d. w. (2003). trust and tam in online shopping: an integrated model. mis quarterly, 27(1), 51-90. goertz, g. (2006). social science concepts: a user's guide: princeton university press. greenwald, a. g., mcghee, d. e., & schwartz, j. l. (1998). measuring individual differences in implicit cognition: the implicit association test. journal of personality and social psychology, 74(6), 1464. gregor, s., & klein, g. (2014). eight obstacles to overcome in the theory testing genre. journal of the association for information systems, 15(11), i. harman, h. h. (1976). modern factor analysis (vol. 3rd). chicago: university of chicago press. harrison, d. a., mclaughlin, m. e., & coalter, t. m. (1996). context, cognition, and common method variance: psychometric and verbal protocol evidence. organizational behavior and human decision processes, 68(3), 246-261. havlicek, l. l., & peterson, n. l. (1977). effect of the violation of assumptions upon significance levels of the pearson r. psychological bulletin, 84(2), 373. haynes, s. n., richard, d., & kubany, e. s. (1995). content validity in psychological assessment: a functional approach to concepts and methods. psychological assessment, 7(3), 238. houts, a. c., cook, t. d., & shadish, w. r. (1986). the person-situation debate: a critical multiplist perspective. journal of personality, 54(1), 52-105. jarque, c. m., & bera, a. k. (1987). a test for normality of observations and regression residuals. international statistical review/revue internationale de statistique, 55(2), 163-172. jarvis, c. b., mackenzie, s. b., & podsakoff, p. m. (2003). a critical review of construct indicators and measurement model misspecification in marketing and consumer research. journal of consumer research, 30(2), 199-218. jarvis, c. b., mackenzie, s. b., & podsakoff, p. m. (2012). the negative consequences of measurement model misspecification: a response to aguirre-urreta and marakas. mis quarterly, 36(1), 139-146. ketchen, d. j., ireland, r. d., & baker, l. t. (2013). the use of archival proxies in strategic management studies castles made of sand? organizational research methods, 16(1), 32-42. australasian journal of information systems zwanenburg & qureshi 2019, vol 23, selected papers from acis 2017 anticipating, avoiding, and alleviating measurement error 19 law, k. s., wong, c.-s., & mobley, w. m. (1998). toward a taxonomy of multidimensional constructs. academy of management review, 23(4), 741-755. lewis, b. r., templeton, g. f., & byrd, t. a. (2005). a methodology for construct development in mis research. european journal of information systems, 14(4), 388-400. lindell, m. k., & whitney, d. j. (2001). accounting for common method variance in crosssectional research designs. journal of applied psychology, 86(1), 114-121. little, r. j., & rubin, d. b. (2014). statistical analysis with missing data (vol. 333): john wiley & sons. mackenzie, s. b., podsakoff, p. m., & podsakoff, n. p. (2011). construct measurement and validation procedures in mis and behavioral research: integrating new and existing techniques. mis quarterly, 35(2), 293-334. markus, k. a., & borsboom, d. (2013). frontiers of test validity theory: measurement, causation, and meaning: routledge. mellenbergh, g. j. (1994). generalized linear item response theory. psychological bulletin, 115(2), 300. meredith, w. (1993). measurement invariance, factor analysis and factorial invariance. psychometrika, 58(4), 525-543. messick, s. (1989). meaning and values in test validation: the science and ethics of assessment. educational researcher, 18(2), 5-11. mulaik, s. a. (2010). foundations of factor analysis (2nd edition ed.). boca raton, fl: taylor and francis group. muthén, b. o. (2002). beyond sem: general latent variable modeling. behaviormetrika, 29(1), 81-117. netemeyer, r. g., bearden, w. o., & sharma, s. (2003). scaling procedures: issues and applications: sage. nunnally, j. c., & bernstein, i. h. (1994). psychometric theory (3 ed.). new york, ny: mcgrawhill. o'leary-kelly, s. w., & vokurka, r. j. (1998). the empirical assessment of construct validity. journal of operations management, 16(4), 387-405. open science collaboration. (2012). an open, large-scale, collaborative effort to estimate the reproducibility of psychological science. perspectives on psychological science, 7(6), 657660. open science collaboration. (2015). estimating the reproducibility of psychological science. science, 349(6251), aac4716-4711-4718. paulhus, d. (1988). balanced inventory of desirable responding (bidr). acceptance and commitment therapy. measures package, 41. paulhus, d. l. (1991). measurement and control of response bias. in j. p. robinson, p. r. shaver, & l. s. wrightsman (eds.), measures of social psychological attitudes, vol. 1. measures of personality and social psychological attitudes (pp. 17-59). san diego, ca, us: academic press. australasian journal of information systems zwanenburg & qureshi 2019, vol 23, selected papers from acis 2017 anticipating, avoiding, and alleviating measurement error 20 peter, j. p. (1981). construct validity: a review of basic issues and marketing practices. journal of marketing research, 133-145. petter, s., rai, a., & straub, d. (2012). the critical importance of construct measurement specification: a response to aguirre-urreta and marakas. mis quarterly, 36(1), 147-155. petter, s., straub, d., & rai, a. (2007). specifying formative constructs in information systems research. mis quarterly, 31(4), 623-656. podsakoff, p. m., mackenzie, s. b., lee, j.-y., & podsakoff, n. p. (2003). common method biases in behavioral research: a critical review of the literature and recommended remedies. journal of applied psychology, 88(5), 879-903. podsakoff, p. m., mackenzie, s. b., & podsakoff, n. p. (2012). sources of method bias in social science research and recommendations on how to control it. annual review of psychology, 63, 539-569. polites, g. l., roberts, n., & thatcher, j. (2012). conceptualizing models using multidimensional constructs: a review and guidelines for their use. european journal of information systems, 21(1), 22-48. richman, w. l., kiesler, s., weisband, s., & drasgow, f. (1999). a meta-analytic study of social desirability distortion in computer-administered questionnaires, traditional questionnaires, and interviews. journal of applied psychology, 84(5), 754. rigdon, e. e. (2012). rethinking partial least squares path modeling: in praise of simple methods. long range planning, 45(5), 341-358. rigdon, e. e. (2013). lee, cadogan, and chamberlain: an excellent point... but what about that iceberg? ams review, 3(1), 24-29. rigdon, e. e. (2014a). comment on “improper use of endogenous formative variables”. journal of business research, 67(1), 2800-2802. rigdon, e. e. (2014b). rethinking partial least squares path modeling: breaking chains and forging ahead. long range planning, 47(3), 161-167. schafer, j. l., & graham, j. w. (2002). missing data: our view of the state of the art. psychological methods, 7(2), 147. schwarz, n., & sudman, s. (1992). context effects in social and psychological research. new york: springer-verlag. segerstrom, s. c., & nes, l. s. (2007). heart rate variability reflects self-regulatory strength, effort, and fatigue. psychological science, 18(3), 275-281. servick, k. (2018, 27 august 2018). 'generous’ approach to replication confirms many highprofile social science findings. science. spector, p. e. (1992). summated rating scale construction: an introduction: sage. spector, p. e. (2006). method variance in organizational research truth or urban legend? organizational research methods, 9(2), 221-232. tehseen, s., ramayah, t., & sajilan, s. (2017). testing and controlling for common method variance: a review of available methods. journal of management sciences, 4(2), 142-168. australasian journal of information systems zwanenburg & qureshi 2019, vol 23, selected papers from acis 2017 anticipating, avoiding, and alleviating measurement error 21 tourangeau, r., rips, l. j., & rasinski, k. (2000). the psychology of survey response: cambridge university press. verhagen, t., van den hooff, b., & meents, s. (2015). toward a better use of the semantic differential in is research: an integrative framework of suggested action. journal of the association for information systems, 16(2), 108-143. viswanathan, m. (2005). measurement error and research design: sage. wennberg, k. (2005). entrepreneurship research through databases: measurement and design issues. new england journal of entrepreneurship, 8(2), 9-19. williams, l. j., hartman, n., & cavazotte, f. (2010). method variance and marker variables: a review and comprehensive cfa marker technique. organizational research methods, 13(3), 477-514. winkler, j. d., kanouse, d. e., & ware, j. e. (1982). controlling for acquiescence response set in scale development. journal of applied psychology, 67(5), 555. zwanenburg, s. p. (2015). how to tie a construct to indicators: guidelines for valid measurement. paper presented at the international conference on information systems, fort worth. zyphur, m. j., & pierides, d. c. (2017). is quantitative research ethical? tools for ethically practicing, evaluating, and using quantitative research. journal of business ethics, 143(1), 1-16. copyright: © 2019 zwanenburg & qureshi. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 the meaning and premise of measurement error 2.1 clear instantiations 2.2 one instantiation, one value 2.3 interrelated constructs 3 a classification of sources of error 3.1 decomposing the logic of measurement 3.1.1 conceptual relationships 3.1.2 operational relationships 3.1.3 inferential relationships 3.2 identifying specific logical links 3.3 tying logical links to sources of error 4 the management of sources of error 4.1 patterns in error 4.2 remedies 4.2.1 modify 4.2.2 control 4.2.3 add 4.2.4 drop 5 discussion paper title (paper title style) australasian journal of information systems hasan & linger 2020, vol 24, research article letting the public in 1 letting the public in: dialectic tensions when local governments move beyond e-government to e-democracy helen hasan university of woolongong hasan@uow.edu.au henry linger monash university abstract among the raft of information systems (is) applications developed for use by local governments are those that attempt to introduce more open community engagement (ce) and facilitate e-democracy. in this paper, we report on a longitudinal study that reveals how the open nature of e-democracy challenges the practices of government bureaucracies. in 2012, we partnered with the community engagement team of a local government council in australia, to study their planning for, and use of, is for ce. our study involved an action research intervention to gain a rich understanding of the contradictory demands of the bureaucratic imperative of the council and the informal activities of the community. this was the first step of a longitudinal qualitative study of the council’s edemocracy efforts over the ensuing seven years. our analysis has been conducted through a dialectic lens, informed by the cynefin sense-making framework. our theoretical contribution is an e-democracy framework that incorporates the dialectic between the ordered environment of government and the community view that is ill-defined and unordered. as a practical contribution, government organisations can use the framework to assess the current status of their ce and design a ce strategy to make interactions with civil society more meaningful. keywords: open government, community engagement, e-democracy, action research 1 introduction this paper addresses the tensions that are created when bureaucratic local government organisations attempt to introduce more open community engagement (ce) using contemporary information systems (is). governments have been enthusiastic early adopters of the internet, setting up websites to inform citizens and implementing systems to enable citizens to conduct transactions with government online. this heralded the advent of egovernment (medaglia 2012, carter & bélanger 2005). in our study, such e-government systems have always been controlled by the local council and usually for one-way information dissemination. any infrequent two-way interaction with citizens was undertaken at face-toface community forums which were usually only attended by the same few people. the motivation for the council ce project, in which we were invited to participate, was to ascertain if online ce systems could enable more open interactions between the local council and a wider range of citizens. the intent of such open interaction was to give citizens a stronger voice in council decision making. when our research was being planned in 2011, a search of australian local government websites found many which provided a feedback form for citizens to provide comments on issues that concerned them. however, we found no instances of genuine interactive online engagement, in the form of a 2-way dialogue, between local governments and the community. australasian journal of information systems hasan & linger 2020, vol 24, research article letting the public in 2 at the time, the australian government 2.0 taskforce (gruen 2010, pp iii) suggested that “...the existing public service culture of hierarchical control and direction must change sufficiently to encourage and reward engagement [and that] to encourage citizen feedback, consultation and engagement processes should be adaptable and diverse”. with the increasing use of social media among citizens in their work, as well as public and private lives, government interest in appropriating this technology for more open community engagement has grown. in an extensive review of the literature, published at the time of the first stage of our study, magro (2012) reported an explosion of research on social media and edemocracy. this had implications not only for participation in the democratic process by citizens but also for a reassessment of the basic culture and organising principles of government, in particular, local or municipal government. however, such e-democracy initiatives have generally had limited acceptance by citizens (tambouris et al., 2012) and, for a variety of reasons, have failed to deliver on their promise of transforming the political system and empowering civil society (oecd 2003, toots 2019, mahrer & krimmer 2005). we thus posed the following research question: how does the open nature of online exchanges within communities challenge the practices and culture of government bureaucracies? to answer this question, we conducted an investigation into the planning for, and use of, is for ce by one local government council (referred to as ‘council’) in australia. of the three tiers of democratic government in australia, federal, state and local, local government is closest to the citizen and has traditionally been most active in organising and managing ce activities. for this reason, our study focusses on local government, but we would argue that our findings provide insights that are relevant to other levels of government and to public sector organisations in general. the researchers were approached by the local council and asked to assist their ce team in developing the council’s online ce program. in the discussions that followed it became apparent that our is research expertise could make a practical contribution to the design of the is to support the ce program. it also became evident that the problem confronting the council presented an interesting opportunity to develop a theoretical understanding of e-democracy. on the basis of our mutual interests, the council agreed that our partnership should begin with an action research (ar) approach (baskerville & wood-harper 1996) that would create a “solution” to the council’s practical problem while at the same time supporting our research agenda to develop theoretical knowledge of value to a research community (mathiassen, et al 2012; mckay & marshal 2001). our initial intervention produced both insightful research findings and useful practical outcomes. however, tensions remained between the open nature of digital media used to engage citizens and the council’s imperative for closed, secure is systems. after the initial ar intervention stage of our study, we continued the research as a longitudinal qualitative study of the council’s ongoing efforts to use is to engage citizens using observations and interviews. this temporal perspective provides a “…sufficiently long history that would permit following the system's evolution over time” (toots 2019, p550) in order to understand the ongoing processes and tensions in the council’s ce efforts. we analysed the data collected from all stages of the study through a dialectical lens informed by the cynefin sense-making framework (snowden 2002). cynefin distinguishes ordered australasian journal of information systems hasan & linger 2020, vol 24, research article letting the public in 3 problems which may be simple or complicated but have a “solution” that is knowable, from unordered problems which are inherently complex, or even chaotic, and hence potential “solutions” emerge from action taken to cope with the problem. most substantial real-world challenges, such as local government community engagement, have elements of order and unorder where the order-unorder relationship displays tensions (nalbandian et al. 2013) that can be characterised as dialectic. we base our understanding and application of dialectics on the hegelian triad (mueller 1958). the basis of this triad is that truth, understanding and resolution of complex issues are found neither in the thesis nor the antithesis, but in an emergent synthesis which reconciles the two. in particular, we suggest that issues confronting government community engagement cannot be resolved exclusively through an ordered or unordered paradigm but need to be approach as a dialectic synthesis of both. nalbandian et al. (2013) frame this dialectical tension as the bureaucratic drive for standardisation and efficiency, an ordered approach to administrative modernisation that is challenged by the need to accommodate unordered variations that reflects the diversity of the community. significantly, to resolve this tension, the authors set the ability to engage the community as one of the major challenges facing local government. we document our theoretical approach, supported by the study findings, to develop a framework to guide those who conduct research and practice in this area or who address similar complex problems. 2 background in this paper, we frame our discussion around the construct of e-democracy (oecd 2003) to capture the transformation of the political system (grönlund 2002) with a focus on more active citizen engagement in government processes and decision making. the term e-participation is another term used in the literature to cover similar ideas (porwol et al 2013) and is referred to as “…sociotechnical information systems [acting] as instruments for democratic dialogue between citizens and governments.” (toots, 2019 p547). our paper seeks to understand, from the perspective of the government organisation, how socio-technical systems support democratic dialogue and adapt to meet the external demands for direct political representation thereby transforming the political system (freeman and quirke 2013). our focus therefore is on “government technology”, rather than “civic technology”, and how internal bureaucratic norms and processes affect the adoption of new technological forms. the starting point for our study is acknowledging and recognising the tensions between open forms of organising afforded by open digital technologies such as social media and the traditional closed nature of bureaucratic public organisations. we note that, while there is abundant extant literature on e-government, there are few studies that investigate the successful use of online is for e-democracy where citizens can have input to government decision making on issues that are of concern to them (see for example toots 2019 for a discussion about the failure of such is system). the tensions in public sector institutions between open and closed forms of organising, free and controlled flows of information, as well as exploration and exploitation of knowledge, are not new (see for example deem 2004). it is well accepted that bureaucracy “…affords a level of efficiency that modern society cannot do without, but it achieves this efficiency only at the terrible price of alienation” (adler 2012 p 246, quoting weber 1958). however, the contemporary impact of is in bureaucratic organisations brings new dimensions to this problem. using government organisations as an example, allen et al. (2001) undertook a case australasian journal of information systems hasan & linger 2020, vol 24, research article letting the public in 4 study of effectively harnessing is as an enabling force to meet the present and emerging challenges of the digital age. they found that the necessary transformation in public sector governance and accountability is likely to be blocked by an antiquated administrative culture that may be ill suited for a digital world. the advent of web 2.0 as our study began, prompted much debate about the role of the citizen and the affordance of social and digital technologies to enable new, more influential roles for citizens (macintosh 2014, linders 2012, magro 2012). most of this literature is focused on the idea of the citizen as a potential partner in the co-production of services and a participant in inclusive dialogues (chadwick 2011, 2008). appropriating is to include citizens in decisionmaking is viewed as strengthening the democratic system by empowering the citizen (coleman 2001, ciborra 2002) and is firmly grounded in oppositional social movements like those witnessed during the “arab spring” (oh et al. 2015). but these optimistic (or at times pessimistic) positions are focused on the ability of citizens to use digital technologies to express themselves and modify their relationship with government, with the idealistic intent to transform government (habermas 1989). this position is also grounded in a particular understanding of the affordances of digital technologies (froomkin 2004). such normative ideals are generally expressed from the “user perspective” and lose sight of government as an organisation concerned with, and accountable for, the delivery of legislated policy and services. moreover, those normative ideals ignore the reluctance of public sector officials to introduce citizen participation in their routine operations as is evidenced in our study. one explanation for this reluctance stems from an entrenched internal culture of command and control that is diametrically opposed to open co-operative relationships with external entities. what is more relevant to this paper is that the dominant control culture of public sector organisations is reflected in the design of the is artefact (lee et al. 2015) used by the bureaucracy to achieve its mandated “efficiencies”. with few exceptions, the literature on online ce in public sector organisations covers government direct service organisations such as health, transport, police, or disaster management (see for example bunker & smith 2009). by contrast, there is little literature on online ce, initiated or supported by governments themselves, which addresses government decision making. local or regional governments are closest to the community and our research was motivated by two reports (chadwick 2011, hull et al 2011), published at the time we began our study, highlighting the limited achievements in government attempts at e-democracy. both these studies found that internal ways of organising were the specific institutional impediments of e-democracy in government settings. summaries of work in the area, published during the conduct of our study, have been used to produce (1) an open government maturity model (lee & kwak 2012) and (2) a model of factors involved in social media adoption by governments (sharif et al. 2015). we draw on these models as a starting point for the development of theory in the form of an integrated edemocracy framework. a recent overview of the topic (oni & okunoye 2018) concluded that success in e-democracy implementation is a complex mix of technological, economic, political, legal and cultural issues and that the barriers to greater online citizen engagement are not technological but cultural, institutional and constitutional. australasian journal of information systems hasan & linger 2020, vol 24, research article letting the public in 5 3 methodology the methodology for this study has a firm theoretical basis in dialectic relationships informed by the cynefin sense-making framework. we conducted a longitudinal study in 2 stages: (1) an initial action research (ar) study that established the council’s ce program involving their use of is to fulfil their legislated ce obligations and (2) subsequent tracking and evaluations of the council’s efforts in adopting online tools for ce. in this second stage our data collection was limited to observations and interviews. data collected during both stages of the study were qualitative and analysed and interpreted by the research team. this analysis was supplemented by content analysis of the ar data using the leximancer tool. in the ar stage, the data was also analysed and interpreted by the council’s ce team, in collaboration with the researchers, to address their practical problem. the project was conducted with university ethics approval that covered all interactions and data collection processes. council and their ce team were all aware of the research aspects of the interventions and evaluations. 3.1 theoretical basis of the research underpinning this research is the dialectic and holistic cynefin sense-making framework of knowledge management (snowden 2002; snowden and boon 2010). cynefin contrasts two domains of order: the known or simple domain and the knowable or complicated domain, with two domains of unorder; the complex and chaotic. cynefin posits the order-unorder dialectic as a way of understanding the contradictory demands of exploiting knowledge through ordered “scientific” processes and the complexity of socio-technical systems that constitute the exploratory and unordered aspects of the problem space. in particular, cynefin depicts how “order” is fundamentally different from “unorder” (snowden 2002; kim and kaplan 2006) in terms of actors’ behaviour, leadership style and the different tools, practices and conceptual understanding they bring to the problem space. following the canonical action research approach of davison et al. (2012), we use the cynefin framework as an instrumental theory (angeles 1992) to design our interventions and evaluations and as a theoretical lens to explain the tensions in the phenomenon we are investigating. specifically, we challenged the council to consider their online community engagement in terms of a dialectic synthesis of the informal unordered perspective of the community with the council’s usual ordered bureaucratic approach to new initiatives. 3.2 the setting the setting of the study is a town in regional australia where the city council and the university are leading civic organizations with long standing links and a history of joint endeavours. in 2011, the researchers were approached by members of the council for assistance in planning their strategy for online community engagement. we later ascertained that the council’s approach was in response to new statutory requirements to broaden ce, following changes by the new south wales (nsw) state government to the local government act (nsw dlg, 2009). this change in the act required all local government councils to develop and implement their own 10-year community strategic policy (csp). one aim of the act was to ensure that all citizens had appropriate opportunities to participate in local government planning and decision-making. the need for community engagement was brought home to the council during an episode when they introduced parking meters to the cbd. this council initiative faced great resistance from the community who felt that they had australasian journal of information systems hasan & linger 2020, vol 24, research article letting the public in 6 not been consulted whereas the council believed that consultation had been undertaken at a previous town meeting. the practical problem enunciated by the ce team was the challenge of using online systems to develop creative, cost effective, reliable methods to engage greater numbers and diversity of members of the community in council planning and decision-making. their stated objective was to capture community needs and aspirations for the future, although the underlying motivation was compliance with the new state legislation. our research involvement began with an action research intervention in 2012, followed by evaluations which were undertaken over the subsequent seven years to ascertain how successful the council had been in their use of is for ce over this time and what impact, if any, this had on the council. 3.3 action research in the context of ce collaboration with the council was based on the separate but related agendas of the ce team and our imperatives as university researchers. this made it an ideal project for action research (ar), a recognised research method in a number of fields (e.g. van eynde & bledsoe 1990), as it deliberately sets out to solve a practical problem and at the same time to advance the body of disciplinary knowledge on a topic represented by the problem. ar allows action to be framed and informed by a conceptual framework derived from theory (baskerville & wood harper 1996) and for that action to be relevant by providing practical solutions to a recognised problem. ar involves several phases, namely diagnosis, planning, action, evaluation and learning (mathiassen et al 2012). this cycle is considered an idealised template to be adjusted to the problem situation (baskerville and wood-harper 1996). however, recurrent criticism of ar is that it is essentially a consultancy with little research rigor. for this reason, mckay and marshall (2001) argue that ar requires both a problem-solving cycle and an explicit research cycle that establishes and articulates the research agenda of the intervention. the explicit adoption of this dual agenda requires the design of the intervention to both solve the practical problem but also to provide opportunities for collecting data as an evidence base for the research contribution. in our case, the problem-solving agenda of the ce team is better community engagement and more participation of citizens in council’s decision-making. this paper focuses on our research agenda to better understand e-democracy by exploring the dialectic tensions between the council’s ordered, bureaucratic adoption of new is for ce, and the unordered, informal community members’ response to council initiatives and decision making. 3.4 stage 1: the action research intervention following the phases of ar, in the diagnosis phase we identified the council’s problems and motivations. at the time all ce activities involved face-to-face community forums which were attended by those few citizens who had the time and interest to attend. the new act required broader ce involvement by citizens particularly those too busy or otherwise unable to attend such forums. the immediate problem for the ce team was to provide a draft csp to council that would include a credible description of the means, preferably involving new technologies, by which they could engage the wider community in the ar planning phase, the ce team and researchers discussed the suitability of groupware systems for community engagement. the action research approach lent itself to the use of is groupware because our intervention would demonstrate how such a system could be used for australasian journal of information systems hasan & linger 2020, vol 24, research article letting the public in 7 ce. our intervention involved the ce team using groupware in face-to-face mode to understand the potential of the system to support ce when used by citizens in online mode. the ce team had heard about the zing groupware system1, developed by a research student at the university, and we suggested that zing be trialled in face-to-face mode in a workshop. this class of technology has several advantages for practice and research. typically, in open “town hall” style meetings it is often only the loudest voices that are heard. with groupware everyone has equal status and can write and submits their ideas in their own words. moreover, because a groupware system captures participants’ contributions as text, it automates research data collection replacing the need for recording, transcribing and notetaking by researchers. the workshop was held outside council premises at a neutral site. the motivation behind this decision was to remove both the ce team and researchers from their usual work environment and to encourage council staff to be open and forthright in their participation. as most of the ce team and the researchers resided in the local area, their participation in the workshop would have a dual agenda. on one hand all participants could act as citizens, expressing their personal views on issues that the council raised. on the other hand, participants would act as professionals, reflecting on their engagement as citizens. by playing dual roles, participants were able to examine issues from both points of view and recognise and appreciate the inherent tension between a normative organisational imperative and the diversity of individual views of the issues. the inherent duality of roles was discussed and explored extensively during the planning and design of the workshop. preparatory discussions were conducted between the two leaders of the ce team and the two lead researchers to create a set of questions to guide the workshop discussion. this meeting produced the set of questions listed in appendix 1 that would be pre-loaded into the zing system for the workshop. this was done so that, in the evaluation phase of ar, the analysis of the data collected in zing would help in the development of a practical solution; in this case to articulate the council’s ce strategy in the mandatory csp. the action phase of the ar cycle was the conduct of the workshop. this was considered an intervention as it provided a hands-on experience of the interface and functionality of a class of technologies suitable for community engagement, exemplified by zing. moreover, the intervention provided the ce team with an opportunity to reflect on what is possible to do with the technology based on their practical experience. the workshop was also an opportunity to collect a variety of views from council staff on their past experiences, current attitudes and possible future adaptation of the council’s processes for community engagement. participation in the workshop gave council staff, including the ce team, an insight into the affordances and capabilities of the technology to support effective citizen engagement with local government. this workshop also addressed the research agenda of ar. the zing sessions were designed to demonstrate how is impacts online engagement and provide insight into how it might influence ce. data collected during the action phase helped to identify the implications of undertaking such a technology strategy for community engagement and provided the empirical basis for theorising e-democracy. 1 http://zingthing.com/ australasian journal of information systems hasan & linger 2020, vol 24, research article letting the public in 8 the half-day workshop involved 11 participants including the ce team, other council staff and four researchers. after introductions and a briefing, a zing session was held to brainstorm answers to the prepared questions to explore the perceptions and ideas of community engagement of all participants. this was followed by a debriefing and reflection session, recorded in zing (q14 in appendix 1), on the participants’ experience of using the zing system and their views on the use of such technologies in ce. the evaluation phase was conducted shortly after the workshop and involved review of the raw data from the zing sessions by both the ce team and the researchers. the data was presented as lists of responses entered by all participants in answer to each of the questions posed during the zing session the ce team were asked to summarise their impression of the responses and its contribution to the development of csp. the researchers categorised the responses through the lens of the cynefin order-unorder dialectic, taking into account the ce team’s perceptions of the data, to draw theoretical insights into the use of is to engage citizens. the evaluation phase was then continued into a second stage of the research. 3.5 stage 2: tracking and evaluating ce our longitudinal study evaluated the long-term impact of the practical outcomes of ar on the council’s ce activities. this was conducted over a period of seven years from 2012 to 2018. the researchers collected data from ce documents produced by council during that time and observed their attempts at implementing is for ce. in addition, we also conducted two semistructured interviews with council officers in 2015 and 2018. the interviews during the evaluation stage were limited as our “formal” engagement with council ended with the release of the csp document. moreover, the interviews were conducted at critical points in the development of the council’s ongoing ce program as the technology environment evolved over time. our evaluations, discussed in detail in section 4, included: • content analysis of the 10-year csp document released in 2012 • an assessment of the online ce website set up in 2014 as a response to the 2012 strategy • an assessment of the use of social media by the council since 2015 • content analysis of a draft ce council policy document released in 2018 • opportunistic semi-structured interviews conducted in 2015 and 2018 with the manager of the ce team analysis of this data led to a better understanding of what the council’s ce team had learnt from our intervention and provided us with a clearer view of the dialectic tensions that hindered the full implementation of the council’s ce strategy. 4 research findings analysis of the detailed data collected from stage 1 ar provided immediate practical application to the council’s policy development. moreover, we discovered that our intervention would have longterm consequences, so we continued the evaluation phase in stage 2 over a period of seven years. australasian journal of information systems hasan & linger 2020, vol 24, research article letting the public in 9 the cynefin lens enabled us to tease out the dialectic tensions between the perceived ordered and unordered aspects of the ce problem. the results of our evaluation of both stages, reported here, contributed to the development of our e-democracy framework. 4.1 stage 1: collection of data from the workshop a substantial amount of data was collected at the workshop reflecting the strong engagement of the participants with the topic. at the workshop, the user interface and functionality of the zing system rapidly became ubiquitous and participants concentrated on what they wanted to say and what others were saying in response to each question. this prompted lively discussions and many ideas were entered into the zing system, giving the ce team a real world understanding of the dynamics of computer mediated interactive conversation and diversity of citizen views. the richness of the data highlights the range of dialectic tensions that exists. an example is participants’ views of their city, shown in their responses to questions 1 and 2 (table 1). all participants, who lived locally, gave very positive views about the city, showing their devotion to the city and pride that they all have both as citizens and council workers. however, participants also recognised the problems that the city faces as shown by their answers to the second question. a synthesis of both sets of views represents the reality of the situation, highlighting the dialectic tension between positive and negative aspects of the community and its environment. this complexity frames the community as essentially unordered in contrast to the essentially ordered nature of a government organisation revealed in the following section of the paper. q1 what are the really good things about living in our city q2 what are some of the problems facing our community? beaches, bush and restaurant lifestyle it's a city without being too big and busy location, climate surf sun people the innovation campus, proximity to sydney, cheaper real estate influences of different cultures – food relaxed environment rising crime rates youth unemployment lack of infrastructure dying inner city centre ageing communitychanging needs for housing and infrastructure working poorlevel of increasing disadvantage in pockets around the city city shopping precinct is uninteresting poor council and government image table 1: responses to questions 1 and 2 4.2 stage 1: manual inspection of zing data by the ce team following the workshop, all the raw data collected from the zing system was sent to the ce team and a week later they had identified three core themes in the data expressed in the following statements: (a) building learning and knowledge exchange networks for ce (b) educating the community for ce (c) developing council staff capability for ce it should be noted that regarding statement (a) the term “learning and knowledge exchange networks” was frequently used by the researchers in meetings with the ce team prior to the australasian journal of information systems hasan & linger 2020, vol 24, research article letting the public in 10 workshop and seemed to have become part of their terminology as it resonated with what they wanted to express. statements (b) and (c) appear to reflect the traditional view of the relationship between council and citizens. statement (b) implies that council knows best, and it therefore has the knowledge, power and responsibility to educate the citizens on how to use appropriate channels to provide feedback to council on topics identified by council. statement (c) gives the impression of a bureaucratic approach to determining the council’s way of doing ce and developing the capability for that. these three broad themes essentially incorporate ideas that align with the council’s internal ordered way of thinking and organising. this contrasted to the multiple perspectives evident in the discussion at the workshop. many of the more innovative ideas that emerged in the unordered enthusiasm of the workshop seemed to get lost once the participants were back at work. we postulate that this was due to the fact that a week had passed, and the culture of the work environment dominated the attention of the council ce team. we formed the view that future approaches to ce would in essence stay the same even if new technologies are introduced. 4.3 stage 1: manual inspection of zing data by the researchers a manual inspection was made by the researchers of the zing data (the text as typed anonymously by workshop participants) using the lens of cynefin to categorise responses as ordered or unordered (ali 2014). consensus on the categorisation of each statement in the zing data was based on discussion between the researchers. the most meaningful and insightful findings came from questions 5, 6, 10 and 11 (appendix 1). question 5 is “from council’s point of view what should ce involve?” the ce team could speak with authority on this and the results in table 2 inform us of participants’ perspectives on organisational practices. ordered unordered [council must] be quite clear on the level of influence the community will have on the decision is it inform or collaborate or empower [council must] inform community how the final decision will be made [we need] a clear understanding of what the input from the community will be [we need] compliance with government regulations [we need] clear process, clear guidelines, purpose [we need] a whole of organisation approach sourcing knowledge from community members open and transparent opportunity to give feedback all stakeholders not just the squeaky wheels with a barrow to push process is as important as the outcomerelationships and collaboration keeping the momentum going listening to the community and not going in with fixed ideas connecting with hard to reach groups developing community representative groups abilities to engage ns understand ce getting community ownership of issues from the beginning internal council engagement using different approaches creative methodsexciting and interesting opportunities to have your sayboth at a pre-planning phase (i.e. at the very beginning before an idea has been informed) and on things that are more progressed technology, blogs, facebook, interactive tools fast feedback table 2 responses to question 5 question 6 is “what should ce involve when viewed from the community’s standpoint?” this question required workshop participants to consider ce issues as citizens, from the local australasian journal of information systems hasan & linger 2020, vol 24, research article letting the public in 11 community perspective, was not difficult and all participants agreed that this was quite authentic as most of them lived in the local community. responses were predominantly unordered as shown in table 3. unordered (predominately) transparency and honesty feedback on ideas and what happened to the information provided convenience needs to cross boundaries within the community being listened to accommodate different perspectives and interests council needs to work together and not be separate parts take into consideration the different cultures, languages, etc. changing from community viewpoint that council will do what suits them anyway too much political interference from bully-boys groups genuine attitudeopen to my opiniondon't play the expert card all the time technology, email, internet table 3 responses to question 6 questions 10 and 11 provide an opportunity to reflect on issues associated with the communication between council and the community. question 10 is “what barriers to ce exist now (within the council and within the community)?” question 11 is “how would you propose these barriers be overcome?” these questions provided opportunities for council workers to be creative and innovative in the way they saw council engaging with the community. ordered unordered time, money, resources, bureaucracy council culture, attitudeinternal take up and commitment internal beliefs perceived agendas, different factions, historical events, fearof change lack of control table 4 responses to question 10 ordered unordered more money, more staff, budget and resources when community sees results from the implementation of integrated planning reporting educate the community specific targeted groups so as to bring them into the ce fold engagement team values as an expert in the process early onthe ideas and strategies are listened and responded to early ce reference group development of greater internal capacity to conduct comm. engagement trial and assess new structures for ce and research alternate methods from elsewhere other councils etc it is better if initiatives come from the people find ways to engage with the whole community, i.e.; technology, social media openness to social media as a forum and method of ongoing engagement table 5 responses to question 11 this collection of responses informed the researchers of the significant elements that should be included in our focal theory, an e-democracy framework. the responses provide a rich picture of what ce could be in a more democratic, open and participatory environment in accord with the affordances of digital group technologies such as zing. australasian journal of information systems hasan & linger 2020, vol 24, research article letting the public in 12 4.4 stage 1: automated analysis of zing data the data from the workshop was also analysed using the content analysis tool leximancer and the results are shown in the maps of appendix 2. leximancer maps arrange concepts in patterns that represent significant themes and the influence between them. this analysis revealed a different and richer picture of the participants’ view of ce to that of the manual inspection. the large central themes are ce, community and groups. viewing the map as a whole through the cynefin lens, the smaller outlying themes on the left-hand side of the map are more ordered particularly those of public works, barriers, online, city and stakeholders while those on the righthand side, particularly that of individual, are more unordered. looking at these more closely, we present the following interpretation of the patterns of concepts. the ce theme links together many of the concepts related to traditional top-down ways of working such a leadership, technology, strategy, development activities linking through to bureaucracy in the barrier theme. together with the close-by themes of public works, city and stakeholders, it speaks to an ordered view of government. table 6 lists the main concept in these themes. we found it interesting that the concept online is located in this ordered pattern, pointing to a bureaucratic attitude to digital interaction. theme concept public works neighbourhood, forums, spaces, accessibility barriers bureaucracy online relationship, attitude stakeholders compliance city citizens, authority table 6: themes and concepts on the ordered side of the maps the group theme contains concept of connecting, people, change, initiatives, projects, decision, and momentum, linking through to the individual theme. this is a set of concepts reflecting a more progressive, unordered view that would be open to the increased involvement in government processes of individuals and a broader range of community groups. the community theme is central with a large set of concepts overlapping with adjacent themes. on the side closest to the groups theme are the concepts issues, diverse and different which are associated with an unordered view of doing things. on the other hand, concepts of results, accessed and external are linked to the ordered aspect of this theme. central to the community theme are the concepts of consultation, collaboration and engagement which imply a progression towards a more open orientation of ce. this is reflected in the open government maturity model (lee & kwak 2012) depicted in figure 2. these patterns exemplify dialectic relationships between local government and community ways of working. many of the workshop participants both worked for council and lived locally so were members of the community. at the workshop, held outside council premises, they expressed many views from the community perspective, such as “we want council to listen”, “it is better if initiatives come from the people”. however, their feedback a week later when they were back at the office reflected the council perspective that conformed to formal council structures, processes, and one-way communication from council. six months after the workshop a formal csp was submitted, as required by legislation, with little sign of any australasian journal of information systems hasan & linger 2020, vol 24, research article letting the public in 13 innovative use of technology for cooperative exchanges between council and the community. it appears that, despite an expressed willingness to explore new ways of working, there was little appreciation of the potential of technologies as they were still seen as risky and a challenge for the entrenched, ordered, organising principles of council. 4.5 stage 1 outcomes – towards the council’s ce policy of 2012 the immediate concern of the ce team after the zing workshop was to complete their 10-year csp. figure 1 shows the statements on the council’s intentions for online ce which were included by the ce team in the csp. these statements show that the overall objective of the council’s ce strategy is to enable residents to “take an active role in decisions”. of particular interest to our research question, is that council has in place a strategy (4.1.2) to use technology and social media in their effort to meet this objective. the two community indicators send conflicting messages on this. the first focuses on the importance on how people feel, which could be construed as an unordered aspect. the second focusses on “formal engagement activities” which certainly reveals an ordered orientation and a dialectic tension with the first indicator. figure 1 statements concerning the use of technology in the ce strategic policy of 2012 appendix 3a shows the ce page on the council’s website in early 2012. the emphasis is on the provision of information from council to citizens. the “have your say” page only mentions community forums and kiosks, with no online ce facility. 4.6 stage 2 findings from ongoing assessment of the council’s online ce following the completion of stage 1 of the research in 2012, we continued to observe and record online ce initiatives of the council from our perspective as citizens. in order to get the council’s perspective on these initiatives, interviews were conducted in 2015 and 2018 with the manager of the ce team. the results are summarised in table 7 australasian journal of information systems hasan & linger 2020, vol 24, research article letting the public in 14 years initiative citizen perspective council perspective 2014now councils ce website “our place, our voice, our future” set up in 2014 using the “bang the table” (btt) platform see appendix 3b also adopted by several other councils about 300 citizens are registered from a population of 200,000. there are strict usage policies, users are blocked for non-compliance. with occasional exceptions, council only allows discussion on topics nominated by them, which does not inspire many posts and discourages any vigorous debate. only council can post documents or videos council uses the site to publicise new programs, promptly respond to any posts by citizens. polls are used by council runs citizen surveys on proposed projects. these are widely publicised with good online responses (1000-2000). council believes that it has broadened its ce reach 2015 now council use of twitter @wollongong_city currently 6,411 followers some retweet council posts. council also retweets posts they find interesting and serves their purpose the twitter posts for the week are planned in advance while sometimes additional ones are sent out ad hoc if there are new local developments 2015 now council official facebook page @cityofwollongong· aims to inform citizens and promote the city page has 30,00 likes. in 2015 posting by citizens was moderated appendix 3c shows its facebook policy in 2015 with the rules for posting on the right-hand side. in 2018 these rules disappeared, and citizens can no longer post on the page. one of the ce team is responsible for posting on behalf of council and in 2015 for moderating posts by citizens. table 7 summary of stage 2 results in the 2015 phone interview, the council ce manager said that her team had applied what they had learnt from the ar stage of our research when writing the 10-year ce strategy in 2012. when it came to implementing the strategy, the ce team had spent most of 2013 changing the council’s branding which was reflected in the look and feel of the whole council website. the ce team also began to incorporate some web 2.0 functionality and found our research findings extremely valuable. this led to the adoption of the btt software shown in appendix 3b and described in table 7 providing evidence that it is not a tool that suits the unordered nature of the community but does suit the ordered imperatives of council. it was mentioned in the interview that individual citizens now contact the council by email and these emails are answered by a personal return email. sometimes a concern raised by a citizen in an email leads to a posting on the btt site if the ce team thinks that other citizens might benefit. the overall message we received from the 2015 interview was that there was progress on council’s use of is for ce and that the ce manager believed that changes would continue as their confidence and understanding grew. later in 2015, council had begun to use social media as shown in table 7 in 2018 we interviewed a new ce team manager who wanted to make more use of social media but expressed frustration with the way the two main online ce platforms, btt and facebook, were controlled by council policy in a structured and bureaucratic manner. neither platform promotes any real discussion, discouraging dissenting opinions. she commented that the potential that social media once had for citizen engagement in local government has been australasian journal of information systems hasan & linger 2020, vol 24, research article letting the public in 15 negated by its misuse world-wide. there are now many twitter, instagram and facebook accounts set up by groups and individuals that promote discussion and advocacy on many local issues involving council but with no official council posts. 4.7 stage 2 the 2018 council ce policy document in 2018 the council released a new draft community engagement policy calling for comment from its citizens. this new document is inspired by the approach taken by the international association for public participation (iap2) which seeks to promote and improve the practice of public participation or community and stakeholder engagement, incorporating individuals, governments, institutions and other entities that affect the public interest throughout the world”2. the iap2 australasia strategic plan was ratified as a quality assurance standard in november 2017 with the intention to implement it in local governments across australia. our intention in the following section of the paper is to develop a framework, based on our research findings, that can guide those councils currently adopting iap2 in a way that accommodates a synthesis of their need for bureaucratic order with the unorder of diverse community expectations. 5 theorising community engagement: an e-democracy framework in action research, a focal theory is used to provide the intellectual basis for problem solving (davison et al., 2012). in our case, this takes the form of a framework to theorise e-democracy as both a research contribution and as a device for solving practical ce problems. the framework is developed from the extant literature and augmented by our findings. 5.1 the foundations of the framework as foreshadowed in the background section of the paper, our theorising of e-democracy builds on the two published models depicted in figures 2 and 3. the first of these models, figure 2, is lee and kwak’s open government maturity model (lee & kwak 2012) published not long after we began the first stage of our research. viewed through the dialectic lens of cynefin we argue that the bottom right section of the model, citing technical/managerial complexity and challenges/risks, reflects the ordered concerns of council. it must be noted that the use of the term complexity in the model is used in a way that does not concur with snowden’s use of the term. in the context of the model, it reflects concerns about fitting such “complexity” into the bureaucratic structures and processes aligned with snowden’s use of the ordered phrase “complicated but knowable”. the top left section, open government maturity, public engagement and public value, reflects the unordered concerns of the community. we would interpret the model as a synthesis of the ordered-unorder dialectic and the five ascending phases in this maturity model would logically form one dimension of our proposed e-democracy framework. 2 https://www.iap2.org.au/about-us/about-iap2-australasia australasian journal of information systems hasan & linger 2020, vol 24, research article letting the public in 16 figure 2. open government maturity model (ogmm) (from lee & kwak 2012) in the light of our study, the maturity phases are interpreted as follows: • data transparency: an ordered push strategy with government distributing information but little or no opportunities for any engagement • open participation: government dictating where, how and when input from citizens is required. this is still ordered but some unorder is introduced as government does not control citizen input • open collaboration: invites citizen engagement but with the agenda strictly controlled by government. here there is recognition of unorder by government but with attempts to impose order by controlling the agenda • ubiquitous engagement is where there is a synthesis of the ordered bureaucratic needs of government systems and the unordered diversity of views held by local residents, where government and citizens share ownership and responsibility for cooperative decision-making. the elements in lee & kwak’s (2012) model can be interpreted, outside the confines of a maturity model, as the general nature of the community engagement and the outcomes of this engagement. in this broader interpretation, these elements are aligned with the concepts identified in the workshop and represent the social behaviour of (local) government. in this context, our understanding of online ce follows oni and okunoye (2018) who use the terms e-engagement for is that supports government consultation and participation with citizens and e-consultation as exchanges between government and citizens using the internet. they suggest that successful e-democracy implementation goes further and should be based on ubiquitous technology where social and cultural considerations are the focus. sharif et al (2015) published their model of social media adoption, as shown in figure 3. when viewed through the dialectic lens of cynefin, we see the ordered concerns of council management (risk, policy etc) on the right-hand side and on the left, the unordered (faddish) community demands and perceived benefit concerns of the community. in sharif et al.’s (2015) model the three over-riding factors are technology, organisation and environment. these factors represent areas that are impacted, or themselves have an impact, on the use of social media. however, based on the findings of our study we judged these factors as necessary, but not australasian journal of information systems hasan & linger 2020, vol 24, research article letting the public in 17 sufficient, to represent the dialectic tensions between the ordered bureaucracy of government and the unordered diversity of community. figure 3 interaction of factors impacting social media adoption (from sharif et al 2015) we interpret sharif et al.’s (2015) model as a synthesis of the ordered-unorder dialectic perspectives and adopt the three factors of technology, organisation and environment as a second dimension of our proposed e democracy framework. however, in light of our research findings, we propose to distinguish between the organisational environment and the community environment and also include the communication between these two environments giving us five factors: • technology: from the zing experience discussed under question 14 and the evaluation of post workshop online ce efforts over seven years • organisational practice: from the analysis of responses to question 5 in table 2 and feedback from the ce team • organisational environment: from the analysis of the leximancer concept map and post workshop interview • community environment: from the analysis of responses to questions 1, 2 and 5 in tables 1 and 3 • communication: from the analysis of responses to questions 10 and 11 in tables 4 and 5 5.2 development of our framework in developing our framework, we have adopted hirschheim et al.’s (1996) nomenclature, used in their framework for the intellectual structures of is development, drawing on the social australasian journal of information systems hasan & linger 2020, vol 24, research article letting the public in 18 action theories of habermas (1987) and etzioni (1968). on the one hand, we use the concept of orientation in terms of the intention and behaviours on the part of the government organisation towards ce. in this sense, we have adapted lee and kwak’s elements for the orientation dimension of the framework as a continuum of order-unorder as discussed in the previous section. on the other hand, hirschheim et al.’s domains of change identifies what aspects are transformed as a result of ce activity. the five factors adapted from the sharif et al (2015) model and listed above, form the second dimension of the framework. the resulting edemocracy framework, depicted in figure 4, correlates these two dimensions as a matrix to create different views of ce activity and possibilities for changes as one moves around the matrix. in figure 4 we have populated this matrix with typical ce activities as an illustration of how ce can be understood in terms of the order-unorder dialectic. in our study we have observed ce in the data transparency, open participation and open collaboration orientations but have not observed a genuine ubiquitous engagement orientation. in the continuum of each domain of change, ce moves from more order towards a synthesis of order and unorder as ce progresses across the orientations from left to right. figure 4: the e-democracy framework australasian journal of information systems hasan & linger 2020, vol 24, research article letting the public in 19 it is important to note that the e-democracy framework depicted in figure 4 results from our study’s research agenda articulated in the action research of stage 1, with examples from the study inserted in relevant cells of the table 5.3 implication of the e-democracy framework for ce practice our research began by adopting an action research design with a dual agenda: (1) to solve the practical problem of implementing online ce to enhance e-democracy, and (2) to provide opportunities for collecting data as an evidence base for an e-democracy framework as the research contribution. the research agenda has dominated the paper to this point with its outcomes depicted in figure 4 above. we now return to the practical ce agenda and consider how our research outcomes can be used to influence current practice. our study began almost 10 years ago, when the council had to respond to the nsw state government act requiring all local councils to develop and implement their own 10-year ce strategy. ten years later the council was required by the state to develop a new ce policy, this time based on the iap2 framework, depicted in appendix 4. based on our longitudinal evaluation of council’s implementation of ce, we belief that even after 10 years, dialectic tensions remain a challenge for ce. the dialectic tension concerns the council’s need for bureaucratic order and the unordered nature of diverse community expectations. we suggested that our e-democracy framework (figure 4) could be used as a guide for those local government councils currently adopting iap2, thereby allowing them to design a ce strategy which incorporates a synthesis of order and unorder. when we first saw iap2, we were surprised that it proposed that collaboration is the ultimate level of engagement and is only used for long term planning. our framework follows lee and kwak’s lead in recognising ubiquitous engagement as the most mature level of ce where the community takes open government as the norm and is supported by open technology. in contrast to iap2, our framework can be used as a maturity model where councils can identify where each domains of change in their ce strategy lies along the orientation continuum. guided by the examples described in figure 4, councils can populate the cells of the framework with their own initiatives and strategies to assess how these plans impact the level of maturity of each domain of change. 6 concluding remarks the findings of our investigation indicate that the traditional way of organising by governments continues to be predominantly ordered, bureaucratic, hierarchical and closed in the sense that there is a clear boundary between the council and the community. council’s public websites are designed to provide information to external constituents, announce what council is doing and more recently provide structured opportunities for feedback on issues determined by council. this is typical of what is commonly called e-government and which we characterise as ordered. the research we have reported indicates a movement towards the more complex concept of e-democracy where knowledge flows both ways across the boundary between the formal structures and ways of working of governments and the informal activities of the community. we posed the research question: “how does the open nature of online exchanges within communities challenge the practices and culture of government bureaucracies?” in answering this question, we introduced a theoretical approach, based on the cynefin sense-making australasian journal of information systems hasan & linger 2020, vol 24, research article letting the public in 20 framework melded with the hegalian dialectic triad, to encompass the complexities of edemocracy. this approach leads to an understanding and resolution of complex issues where opposing views (thesis and antithesis), are replaced by an emergent synthesis which reconciles the two. in particular, our findings suggest that issues confronting government-community engagement cannot be resolved exclusively through an ordered or unordered paradigm but needs to be approached as a dialectical synthesis of both. genuine e-democracy represents a radical unordered departure from the current ordered practices of e-government that are comfortable for the bureaucracy because they conform to the traditional way of doing business, albeit with different technology. e-democracy thus represents a departure from the “comfort zone” of the pre-digital era and creates contradictions. the dialectical challenge facing government is the need to deliver mandated efficiencies while also providing opportunities for diverse constituents in civil society to be involved in decision making and co-production of services (nalbandian et al. 2013). our research has demonstrated that digital technologies have the capability and affordance to enable open interaction between government and citizens. there is little doubt that is, in particular social media, has the potential to enable more open interaction between government and citizens that is at the core of e-democracy. such technology, when deployed appropriately within an is artefact, allows governments to build the capability for working with the unordered networked and informal cultures of civil society. however, the technologies that facilitate e-democracy challenge the government’s traditional ways of organising. tensions between the openness of is driven ce and the everyday working of council were evident in our findings. on the other hand, our long-term view of the council’s progress in implementing their ce strategy shows that they are aware of the expectation of civil society and the desire of citizens to be involved. and they have shown their willingness to implement technology to meet those expectations. what is also apparent is that the real issue confronting council is resolving the dialectic between their ordered operational imperatives and the demands for unordered citizen participation. in our study, a complete synthesis to resolve this contradiction has not yet been found. however, we speculate that the evolving appropriation of newer is platforms can underpin the emergence of authentic e-democracy in the form of citizen journalism and use of social media by politicians and government leaders. in exploring these issues, we have used our study to theorise e-democracy in a way that accounts for the contradictory demands on government. in this framework we have captured the behavioural and structural elements that define e-democracy. this opens the possibility of identifying new hybrid forms of organising that accommodate the dialectic between ordered and unordered actions that constitute ce. this suggests a research agenda to explore and theorise ce strategies in order to populate the cells within the matrix and establish a broader theoretical basis for ce. but at a practical level the framework can also be used to map the current status of ce of an organisation or to design a ce strategy to make interactions with civil society more meaningful. it must be noted that as an action research study, we have interacted and examined only one council. however, our e-democracy framework provides a theoretical and empirically driven construct to study the broad landscape of e-democracy to identify good community engagement practices and even set an agenda for meaningful e-democracy at all levels of government. australasian journal of information systems hasan & linger 2020, vol 24, research article letting the public in 21 references adler p. (2012). the sociological ambivalence of bureaucracy: from weber via gouldner to marx, organization science 23(1), 244–266 ali, i. (2014). methodological approaches for researching complex organizational phenomena. informing science: the international journal of an emerging transdiscipline, 17, 59-73. allen b., juillet l., paquet g., & roy j. (2001). e-governance & government on-line in canada: partnerships, people & prospects, government information quarterly 18 93–104 baskerville, r.l. & wood-harper, a.t. (1996). a critical perspective on action research as a method for information systems research, journal of info. technology (11), 235-246. bunker, d. & smith, s. (2009). disaster management and community warning systems: interorganisational collaboration and ict innovation. proceedings of pacis2009 #36. carter l. & bélanger f. (2005). the utilization of e-government services: citizen trust, innovation and acceptance factor, information systems journal 15, 5–25 ciborra, c. (2002). the labyrinths of information. oxford: oxford university press. chadwick a. (2011). explaining the failure of an online citizen engagement initiative: the role of internal institutional variables, journal of info.technology & politics, 8 21–40 chadwick, a. (2008). web 2.0: new challenges for the study of e-democracy in an era of informational exuberance. isjlp, 5, 9. coleman, s. (2001). the transformation of citizenship’, in b. axford and r. huggins (eds), new media and politics, london, sage, 109–126. deem r. (2004). the knowledge worker, the manager-academic and the contemporary uk university: new and old forms of public management? financial accountability and management, 20(2) 107-128. etzioni, a. (1968). the active society: theory of societal and political processes. new york: the free press freeman, j., & quirke, s. (2013). understanding e-democracy: governmentled initiatives for democratic reform. ejournal of edemocracy and open government, 5(2), 141–154. froomkin, m.a. (2004). "technologies for democracy" in shane, p. ed. democracy online: the prospects for political renewal through the internet, new york: routledge gray, d. (2009) . complicated vs. complex, http:/ /communicat ionnation.blogspot.com/2009/11/complicated vscomplex.html grönlund, a. (2002). electronic government – efficiency, service quality and democracy. in: electronic government:design, applications and management, grönlund, a. (ed.), pp. 23– 50. idea group, hershey and london, uk. gruen n (2009). engage: getting on with government 2.0 report of the government 2.0 taskforce, australian government information management office habermas, j. (1989). the structural transformation of the public sphere. cambridge, ma:mit press. australasian journal of information systems hasan & linger 2020, vol 24, research article letting the public in 22 habermas, j. (1987). the theory of communicative action--volume two; lifeworld and systems: a critique of functionalist reason. boston, ma: beacon press. hasan h. warne l. & linger h. (2007). the sensible organization: a new agenda for is research, proceedings of icis 2007 montreal hasan h. (2008). back to the future for km: the case for sensible organisation, invited paper for journal of km research and practice, 6(1) 26-30. hirschheim, r., klein, h. k., & lyytinen, k. (1996). exploring the intellectual structures of information systems development: a social action theoretic analysis. accounting, management and information technologies, 6 (1-2): 1-64. hull d. west g. & cecez-kecmanovic d. (2011). two models of e-democracy: a case study of government online engagement with the community, australian school of business research paper no. 2011 – irrc -01 kim r. m. & kaplan s. m., (2006). "interpreting socio-technical co-evolution: applying complex adaptive systems to is engagement", infor. technology & people, 19(1) 35 – 54 lee, a. s., thomas, m., & baskerville, r. l. (2015). going back to basics in design science: from the information technology artifact to the information systems artifact. information systems journal, 25(1), 5-21. lee g. & kwak y.h. (2012). an open government maturity model for social media-based public engagement, government information quarterly 29 492–503 linders, d. (2012). from e-government to we-government: defining a typology for citizen coproduction in the age of social media. government information quarterly, 29(4) 446-454. medaglia, r. (2012). eparticipation research: moving characterization forward (2006–2011). government information quarterly, 29, 346–360 macintosh, a. (2004). characterizing e-participation in policy-making. in proceedings of the 37th annual hawaii international conference on system sciences (pp. 10-pp). ieee. magro m (2012). a review of social media use in e-government adm. sci. 2(2), 148-161; doi:10.3390/admsci2020148 mahrer, h., & krimmer, r. (2005). towards the enhancement of e‐democracy: identifying the notion of the ‘middleman paradox’. information systems journal, 15(1), 27-42. mathiassen, l., chiasson, m. & germonprez, m. (2012). style composition in action research publication. mis quarterly, 36(2), 347-363. mckay, j. & marshall, p. (2001). the dual imperatives of action research. information technology & people, 14(1) 46-59. mueller g. (1958). the hegel legend of thesis-antithesis-synthesis" journal of the history of ideas 19(3) 411-414. nalbandian, j., o'neill jr, r., michael wilkes, j., & kaufman, a. (2013). contemporary challenges in local government: evolving roles and responsibilities, structures, and processes. public administration review, 73(4), 567-574. australasian journal of information systems hasan & linger 2020, vol 24, research article letting the public in 23 newell, s. swan j. & galliers r. (2000). a knowledge-focused perspective on the diffusion and adoption of complex information technologies: the bpr example, info systems j. 10 239259 oecd (2003). promise and problems of e-democracy: challenges of online citizen engagement, paris, oecd publishing. retrieved 27 april 2020 from www.oecd.org/governance/digitalgovernment/35176328.pdf oh o. eom c. & rao h. (2015). role of social media in social change: an analysis of collective sense making during the 2011 egypt revolution, information systems research 26/1, 210– 223 oni a. & okunoye a. (2018). critical success factors in e-democracy implementation in encyclopedia of information science and technology, fourth edition doi: 10.4018/978-15225-2255-3.ch309. porwol, l., ojo, a., & breslin, j. (2013). on the duality of e-participation – towards a foundation for citizen-led participation. in a. kö, (vol. ed.), egovis/edem 2013, lecture notes in computer science. 8061. egovis/edem 2013, lecture notes in computer science (pp. 211–225). sharif, h., troshnani, i. &davidson r. (2015). public sector adoption of social media, journal of computer information systems 55(4) 53-61 snowden, d. (2002). complex acts of knowing: paradox and descriptive self-awareness. journal of knowledge management, 6(2). snowden, d. & boone, m. (2007). "a leader's framework for decision making." harvard business review november 2007: 69-76. tambouris, e., macintosh, a., smith, s., panopoulou, e., tarabanis, k., & millard, j. (2012). understanding eparticipation state of play in europe. information systems management, 29(4), 321–330 toots, m. (2019). why e-participation systems fail: the case of estonia's osale. ee. government information quarterly, 36(3), 546-559. van eynde, d. & bledsoe, j., (1990). the changing practice of organization development, leadership & organization development journal, 11(2) 25-30. australasian journal of information systems hasan & linger 2020, vol 24, research article letting the public in 24 appendix 1: zing questions the questions determined by the pre-workshop meeting australasian journal of information systems hasan & linger 2020, vol 24, research article letting the public in 25 appendix 2: leximancer output for clarity, appendix 2a only shows theme labels and appendix 2b only shows the concepts within these themes. appendix 2a: the leximancer concept map of themes in the zing data. australasian journal of information systems hasan & linger 2020, vol 24, research article letting the public in 26 appendix 2b: the leximancer map of concepts in the zing data australasian journal of information systems hasan & linger 2020, vol 24, research article letting the public in 27 appendix 3: ce online presence appendix 3a: the ce page of the council website in 2012 australasian journal of information systems hasan & linger 2020, vol 24, research article letting the public in 28 appendix 3b: the ce website in 2015 using the btt application australasian journal of information systems hasan & linger 2020, vol 24, research article letting the public in 29 appendix 3c: instructions on the council facebook page in 2015 australasian journal of information systems hasan & linger 2020, vol 24, research article letting the public in 30 appendix 4 summary of the ce policy in the 2018 draft document using the iap2 approach copyright: © 2020 hasan & linger. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v24i0.1897 http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems hasan & linger 2020, vol 24, research article letting the public in 31 paper title (paper title style) australasian journal of information systems robinson 2020, vol 24, selected papers from the 2019 aice conference ontology and ethics of virtual influencers 1 towards an ontology and ethics of virtual influencers ben robinson sydney university brob9489@uni.sydney.edu.au abstract in 2018, time magazine named miquela souza one of the 25 most influential people on the internet, despite the fact she is not a person at all. miquela is the first digitally created virtual influencer. this paper provides an initial analysis of some of the ontological and ethical issues associated with the rise of virtual influencers on social media platforms like instagram. through a focus on miquela, it is argued that while these fabricated identities may cause uneasiness at first, there is nothing morally significant that distinguishes them from natural, ‘real life’ influencers. but, far from ‘business as usual’, the inability to separate ‘virtual’ and ‘real life’ influencers raises important questions about the ethical construction of identity, and how this may affect the ongoing preservation of social values like trust in online spaces. the paper draws on literature in personal identity and agency theory to establish the ontological claim that there is no meaningful difference between miquela and other ‘real life’ influencers, which leads to the discussion about ethical issues including moral responsibility and motivation, and transparency. as of may 2020, this appears to be the first peer-reviewed article theorising about virtual influencers. there are significant opportunities for further research, both in terms of how we should conceptualise these identities, as well as more empirically based social research into how to preserve social values like trust in online spaces. keywords: ethics; ontology; ai ethics; social media ethics 1 introduction the rise of virtual identities in online spaces raises a number of important philosophical questions. this discussion paper offers an initial analysis of some of the ontological and ethical issues associated with virtual influencers on instagram. through an analysis of miquela, the first virtual influencer on instagram, i argue that while these fabricated identities may cause uneasiness at first, there is nothing morally significant that distinguishes them from natural, ‘real life’ influencers. but, far from ‘business as usual’, the inability to separate ‘virtual’ and ‘real life’ influencers raises important questions about the ethical construction of identity, and how this may affect the ongoing preservation of social values like trust in online spaces. the paper has three parts. firstly, i will give an outline of miquela, the first and most popular virtual influencer, before talking about the rise of these identities more generally, including the technology (or lack of) that underpins them. secondly, i will discuss some ontological questions including whether their existence online is comparable to existence in real life, and whether this constitutes agency. thirdly, i will discuss some ethical implications, including the need to assign moral responsibility and understand transparency. 2 miquela and the rise of virtual influencers there is no strict definition of an influencer, however it can be understood broadly as someone who holds social power and shapes the behaviour of others through their words and actions. these can be online and offline, but the term ‘influencer’ is particularly used in the context of australasian journal of information systems robinson 2020, vol 24, selected papers from the 2019 aice conference ontology and ethics of virtual influencers 2 social media platforms such as instagram, facebook and snapchat. the kardashian family have pioneered the influencer economy, with kym kardashian and kylie jenner reportedly charging brands up to $1 million per online post. in 2019, the influencer market was valued at $8 billion and this is expected to grow to $15 billion by 2022. (schomer 2019) miquela sousa, better known as lil miquela, is the first computer generated social media influencer.1 since miquela’s conception in 2017, she has gained over 2 million instagram followers and makes a considerable amount of advertising profit for her creators by modelling the clothing of brands including prada, samsung and calvin klein. she has an identity and life history; she is a progressive 19-year-old musician and arts student who supports black lives matter and transgender rights. she poses with a dull gaze in prominent los angeles locations like any other of the thousands of social media influencers on instagram. her photo captions are conversational and hip, and she has released a number of pop songs. in 2018, time named her as one of the 25 most influential ‘people’ on the internet. (time 2018) miquela is created by brud, a little-known media agency that describes itself as a “group of los angeles based problem solvers specializing in robotics, artificial intelligence and their applications to media businesses.” (lewczyk 2020) it is unclear how the images of miquela are made. some think that she is completely computer generated; others think she is only partially computer generated, with her image based off of a real life human model whose limbs and head has been digitally distorted before being uploaded online.2 it is also unclear where miquela’s personality, life history and the ‘content’ of her posts come from. while it is possible that it is based off algorithms and machine learning, the likely hypothesis is that the content of her posts, and her narrative and identity are creations of a group of people at brud. brud is incredibly secretive about the whole project, but what is clear is that while miquela does not exist in the flesh-and-bone sense, she has a distinct identity and life history, and carries significant cultural clout evidenced by her millions of followers who are seemingly unperturbed by her ‘un-reality’. so, there are several unanswered questions, including both how the images of miquela are made, and who determines the content of her posts. the secretiveness of brud is no doubt a strategy that feeds into the ‘myth building’ surrounding miquela – and so far, this seems to have worked. earlier this year, brud closed on a deal for $125 million investment from spark capital. (schieber 2019) and since miquela’s creation, there have been proliferation of other virtual influencers on social media, especially instagram.3 virtual models are now also being widely used in fashion with marketers using cgi to create the ‘perfect’ body and face to align with the aesthetic of certain brands. part of the appeal of virtual influencers or virtual models is that pr risk can be completely controlled – given all the actions of these influencers are deliberated over from a group of people, the risk of the influencer saying something politically incorrect or misaligned with the brand is close to zero, and there can even be clauses written into contracts that the image of the influencer will remain a certain way and serve a brand’s best interest. miquela is different 1 link to miquela’s instagram page (@lilmiquela): https://www.instagram.com/lilmiquela/?hl=en 2 for a conjectural discussion about how the images of miquela are made, see eror (2018) 3 following the success of miquela, brud has created at least two other virtual influencers, with their own unique identity, history and visual aesthetic. see blawko https://www.instagram.com/blawko22/ and bermuda https://www.instagram.com/bermudaisbae/ australasian journal of information systems robinson 2020, vol 24, selected papers from the 2019 aice conference ontology and ethics of virtual influencers 3 from virtual models because she has an identity and narrative. but, despite the myth building from brud, there is no evidence that machine learning is involved with the content of her posts. pundits have claimed that virtual influencers are the future of ads, fashion and commerce, and juniper research estimates that the global fashion industry will invest $3.6 billion in artificial intelligence technology this year. (smith 2019) 3 ontological issues the ontological status of virtual influencers like miquela is unclear. physically, it is obvious she does not exist – she is neither a human person nor a robot. the photos of her are partially or fully computer generated and there is no ‘real life’ miquela that corresponds to the instagram fictional identity. but, this identity certainly exists on the online social network and her actions exert considerable influence on the preferences, buying habits, and trends of her followers. when her followers see a new image of miquela on their phone screen, their relationship with her as an identity is appears almost indistinguishable from other ‘real life’ social media influencers like kim kardashian. miquela is a living, active identity with a unique visual aesthetic, personality and history that has developed and changed over time. the fact that miquela’s creation and maintenance is a blend of human and computer inputs is seemingly unimportant from the user’s perspective she is very much perceived as real on instagram, with her fans liking, commenting on, and sharing her posts, like they would to any other ‘real life’ influencer4. so, on one hand, she clearly does not exist as a physical object in the world, but on the other hand, her identify and self-hood seems to exist on social media just as authentically as other online identities. this raises questions about personal identity, and what the requirements are for a self to ‘exist’ in online spaces. in the following section on ethics, i argue that the there are no meaningful differences miquela’s selfhood and the selfhood of other ‘real life’ instagram influencers and users. but, here, i want to support this argument by briefly explaining how miquela’s identity appears consistent with the dominant theory in personal identity literature; that of psychological-continuity. the psychological-continuity view of personal identity says that a person’s self exists if there is persistence over time between mental states (olsen 2019). of course, we cannot say that miquela exhibits consciousness (or mental states), but there is undoubtedly persistence and psychological continuity in the identity narrative that she displays to her followers on instagram. from the perspective of the instagram user, there is no meaningful difference in the psychological continuity displayed by miquela and that by other ‘real life’ influencers. both have seemingly unique values and beliefs expressed as personalities that slowly mould over time but which retain their underlying form. a second concern is whether virtual influencers can be considered agents. the philosophy of action provides a standard conception of agency, understood in terms of intentionality, where intentionality is explained as causation between mental states and events (schlosser 2019). so, for a virtual influencer to be considered an agent, it has to be able to act autonomously with a causal connection between its mental states and events. it is quite clear this is not the case for the basic model of virtual influencers like miquela. she has no mind separate from her creators. 4 by unimportant i don’t mean that there are no differences between something that is real and something that is perceived as real, i just mean that from the perspective of an instagram user, this isn’t salient. it appears that people know that she is not real, but act as if she is. looking through the lens of their phone screen, instagram users interact with virtual influencers in the same way as real influencers. australasian journal of information systems robinson 2020, vol 24, selected papers from the 2019 aice conference ontology and ethics of virtual influencers 4 maybe in the future, when influencers start using machine learning and creating content autonomously without direct human input, this might change. indeed, in april this year, company 1sec unveiled its first virtual influencer, a japanese-american boy named liam nikuro whose creation will apparently involve “innovative content in combination with ai technology.” (tiffany 2019) but, as it currently stands, no details have been released about what this ai technology will involve. as such, i think we should understand these virtual influencers simply as tools used by human agents. that is, despite their human features, they have the same ontological status as say a car or a house – something used by humans to exert their own agency. the question of ontology and agency is intimately linked with questions about ethics and moral responsibility. i will now talk about some of the ethical issues surrounding virtual influencers. 4 ethical issues in practice, there is currently no difference between the way that virtual and ‘real’ identities are treated on platforms like instagram. both upload content and interact with their followers. but, perhaps we should be treating these two cases differently, and perhaps instagram should start monitoring the accounts of its users to separate fact from fiction and to ensure important social values like trust are maintained. if, for instance, it could be demonstrated that the rise of virtual influencers is having a negative effect on trust and cooperation online, this may detract from the legitimacy of platforms and lead to less good moral outcomes.5 i will now discuss two reasons why it may be important to draw distinctions between real and virtual influencers: motivation and moral responsibility. i will then argue that perhaps transparency is not as important as first intuited. it is difficult to ascertain the motivation of virtual influencers like miquela. it appears that it is simply a way of making money for brud, her creators, with her identity as an artist and social progressive chosen because it makes her popular. in this way, it just seems like a new form of advertising. while this might seem objectionable, especially because of the social causes brud is co-opting in order to make her popular, in reality it is no different from what ‘real life’ influencers do. they promote the best version of themselves, often exaggerating their successes and using photoshop to make themselves look more attractive. their business model is surprisingly transparent; the more followers they have, the more they can charge businesses to promote their products. while it may be argued that there is a difference between exaggerating one’s identity and completely forging one’s identity, it is unclear how this is a meaningful difference. both are forms of misleading practice with an ulterior motive (making money), so if it is acceptable in the first case (real life influencers), it should be acceptable in 5 and if trust declines online or on certain platforms, there may be negative moral consequences. for instance, floridi (2010) has spoken about ‘ethical infrastructure’ that gives rise to situations of ‘distributed morality’ – aggregate good or bad outcomes are facilitated or hindered by the presences of social values like trust. an example is giving to charity – if trust declines on instagram, people may be less inclined to donate because they don’t know for sure that their money will be safe and delivered to the proper recipient, similarly for online shopping. australasian journal of information systems robinson 2020, vol 24, selected papers from the 2019 aice conference ontology and ethics of virtual influencers 5 the second (virtual influencers).6 motivation does not seem to be a salient factor distinguishing real and virtual influencers. a second concern relates to moral responsibility and transparency. while real life influencers are clearly responsible for the content of their posts, it is less clear how virtual influencers could be held responsible in the same way. as it currently stands, this is not a big issue. if we assume that miquela is created by a group of developers at brud, that group of developers are the one’s responsible for miquela’s content: whether it be selling new prada sunglasses or encouraging young people to go to black lives matter protests. while transparency is definitely an issue, especially with how secretive brud is, this is a practical rather than theoretical concern. it is simply a matter of finding out who these developers are and who was responsible for certain posts identities like miquela make. if, however, in the future virtual identities like miquela start acting autonomously and create content based off algorithms and, say, mining ‘trending’ topics on instagram, attributing moral responsibility for their actions would be more difficult. understanding the moral responsibility of virtual influencers in these cases would rely on the kind of analysis developed in the moral responsibility and artificial agent literature.7 a further concern is more normative: should we be advocating for more honesty and transparency with how we depict ourselves online, and the standards from others we should expect? i am not going to argue for this here, but i think it is important to note that people hold different intuitions about this. for instance, there is evidence to suggest that people like ‘authentic’ influencers who show a range of emotions, who show that their life isn’t great all the time, who speak about mental health, who posts photos without makeup on etc. this would suggest that values like authenticity are important (for some people). it would be interesting to see if these are the same people who follow virtual influencers. i would hazard a strong bet that they are not. it’s likely that different people, and possibly groups of people, generations even, have different intuitions about the authenticity and transparency 6 again, one might rebut by saying that there is a difference between a small lie and a large lie. the real life influencer from byron bay who only uploads gorgeous pictures of themselves in beautiful beach locations and who edits these photos to make their skin look perfect, their teeth whiter and the sunset a deeper colour of orange may not be telling the whole truth, especially when considered that the version of their life displayed is incredibly sanitised and carefully selected, but perhaps this is different from an identity that is totally fictionalised, i.e. virtual influencers. someone making this point might refer to the moral condemnation that ‘cat-fishing’ accrues (where people forge their identity on dating apps to get more hits). however, this is disanalogous because unlike a cat-fish on a dating app pretending to be someone else, i.e. lying, some of the virtual influencer online are completely open about the fact they aren’t real. for instance, miquela knows that she is a creation by silicon valley entrepreneurs and is open about this to her followers. in such instances, there is a case to be made that virtual influencers are actually the most authentic influencers on the market – they are being completely transparent and open. but, even for cases where virtual influencers are not as open about their creation with their followers, their dishonesty is not disproportionate to what real life influencers do when exaggerating the success and beauty of their lives – both cases operate with the same underlying principle of dishonesty in order to attract followers. 7 for instance, thomas powers (2013) argues that computer systems can be considered moral agents in virtue of their own “causally efficacious intentional states”. holding computers morally responsible (like miquela if her content was based on ai) does not depend on them having consciousness or intelligence per se. australasian journal of information systems robinson 2020, vol 24, selected papers from the 2019 aice conference ontology and ethics of virtual influencers 6 requirements for online identities, and, by extension, how much they are willing to trust fictional identities online. given the rise in discussions about trust online, this issue would be well some personal anecdotal evidence – when discussing this paper with friends, a common theme is that younger generations are more trusting of technology and online spaces than older generations. what piqued my interest in this topic was because i was incredulous that these virtual influencers had amassed such a massive online following and for reasons not simply out of the shock value or novelty factor – it seemed that younger generations just genuinely don’t care that virtual influencers aren’t real. everyone speaks about why transparency is important with emerging tech, but not everyone holds this intuition, particularly younger people8. there are deeper debates to be had about the value of transparency and the value of explanation itself.9 it is not an apriori truth that we have a right to know who controls the content of posts by virtual influencers like miquela, and the divergent moral intuitions between generations highlight the need for strong argumentation.10 this is an area for further social research – given the rise in discussions about trust online, there may be important policy implications if it were established that different age groups hold different intuitions about particularly recently in the context of discussions about the uptake of the covid-19 app in australia if it were established that different age groups held different intuitions indeed, social research into the potential differences among age-groups 5 conclusion this discussion is the first step in understanding the ontology and ethics of virtual influencers. it seems likely that the popularity of these identities will only increase in years to come, and so several important questions remain on both a theoretical and empirical level. theoretical questions include further refinement about the ontological and ethical status of these identities, whether social media platforms have a responsibility to mediate its user’s accounts and online personas, and whether individuals have a responsibility to construct their identity online in honest and truthful ways. empirical questions include further research into the preferences and moral intuitions of people towards fictional identities online and how this affects the trustworthiness of these platforms, or the brands that these identities speak for. the lines between real life and online existence are becoming increasingly blurred as our engagement with platforms and online spaces deepens and becomes more ubiquitous. more theorising needs to be done to ensure social values like trust and transparency are maintained, and to justify the importance of concepts like transparency and explanation. 8 here, i am treating ‘realness’ and ‘transparency’ as broadly analogous concepts. 9 for a positive account, see afrashteh, someh and davern (2020) who argue that effective explanation facilitates trust and transparency online. 10 while outside the scope of this paper, arguments can be made that the transparency requirements of emerging technologies are overstated. when we seek professional advice from a doctor or a lawyer, we take this advice without having complete or even partial understanding about the reasoning or knowledge base that has led them to this conclusion. we think that transparency is important, but many of the decisions we make are based off trust, habit and guess work. in any case, if transparency and explanation are continued to be championed as pillars of ‘ethical design’ in technology, there ought to be strong philosophical justifications for why these values are important. australasian journal of information systems robinson 2020, vol 24, selected papers from the 2019 aice conference ontology and ethics of virtual influencers 7 references afrashteh, someh & davern (2020) ‘explanations as discourse: towards ethical big data’, australasian journal of information systems, vol 24, selected papers from the is foundations conference eror (2018) “meet lil miquela, the ai influencer“ highsnobiety, accessed 10 may 2020 from: https://www.highsnobiety.com/p/lil-miquela-cover-story-issue-16/. floridi, l., (2013) ‘distributed morality in an information society’, science and engineering ethics,.19(3), pp.727-743 lewczyk m. (2020) “what are virtual influencers, and what do they mean for influencer marketing?” sideqik accessed 10 may 2020 from: https://www.sideqik.com/influencermarketing/virtual-influencers olson, eric t., (2019) "personal identity", the stanford encyclopedia of philosophy (fall 2019 edition), edward n. zalta (ed.), accessed 10 april 2020 from https://plato.stanford.edu/archives/fall2019/entries/identity-personal/ powers, t. m., (2013) ‘the moral agency of computers’, topoi, vol.32(2), pp.227-236. schlosser, markus, (2019) "agency", the stanford encyclopedia of philosophy (winter 2019 edition), edward n. zalta (ed.), accessed 10 april 2020 from https://plato.stanford.edu/archives/win2019/entries/agency/ schomer, a. (2019) ‘influencer marketing report’, business insider, 16 july 2019. accessed 15 august 2019 from: https://www.businessinsider.com/the-2019-influencer-marketingreport-2019-7/?r=au&ir=t shieber, j. (2019) ‘more investors are betting on influencers like lil miquela’ tech crunch, 15 january 2019. accessed 15 august 2019 from: https://techcrunch.com/2019/01/14/moreinvestors-are-betting-on-virtual-influencers-like-lil-miquela/ smith, s. (2019) ‘ai spending by retailers to reach $12 billion by 2023’, juniper research. accessed 15 august 2019 from: https://www.juniperresearch.com/press/pressreleases/ai-spending-by-retailers-reach-12-billion-2023 tiffany, k. (2019) ‘lil miquela and the virtual influencer hype, explained’, vox, 3 june 2019. accessed 15 august 2019 from: https://www.vox.com/thegoods/2019/6/3/18647626/instagram-virtual-influencers-lil-miquela-ai-startups time (2018) “the 25 most influential people on the internet” time. accessed 10 may 2020 from: https://time.com/5324130/most-influential-internet/ turilli, vaccarro & taddeo (2010) ‘the case of online trust’, knowledge, technology & policy, 2010, vol.23(3), pp.333-345. copyright: © 2020 robinson. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems robinson 2020, vol 24, selected papers from the 2019 aice conference ontology and ethics of virtual influencers 8 doi: https://doi.org/10.3127/ajis.v24i0.2807 paper title (paper title style) australasian journal of information systems singh & rumantir 2015, vol. 19, pp. s117-s132 two-tiered clustering experiments two-tiered clustering classification experiments for market segmentation of eftpos retailers ashishkumar singh faculty of information technology caulfield monash university ashish.singh@monash.edu grace rumantir faculty of information technology caulfield monash university grace.rumantir@monash.edu abstract almost all the papers reported in the literatures on market segmentation modeling using retail transaction data deal with finding groupings of customers. this paper proposes the application of clustering and classification techniques for finding groupings of retailers who use the electronic funds transfer at point of sale (eftpos) facilities of a major bank in australia in their businesses. the rfm (recency, frequency, monetary value) analysis on each retailer is used to reduce the large data set of customer purchases through the eftpos network into attributes that may explain the business activities of the retailers. preliminary results published in singh, rumantir and south (2014) show that groupings of retailers with distinct combinations of rfm values can be established. encouraged by the promising business insights gained from the clustering experiments using the rfm values, in this paper, we report our findings in extracting business rules pertinent to each cluster. for this purpose, we incorporate attributes of the eftpos transaction data in addition to the derived rfm attributes to build a decision tree to facilitate the extraction of the business rules. keywords: market segmentation, clustering, decision tree, business rules extraction, rfm analysis, eftpos. 1 introduction electronic funds transfer at point of sale (eftpos) is one of the leading methods of payment in the retail industry. payments on eftpos terminals are done using debit and credit cards, the most common non-cash payment methods. in 2014, over 2.4 billion eftpos transactions were put through over 820 thousand eftpos terminals in australia. these transactions amount to over 139 billion dollars (eftpos annual report, 2015). the banking sector gains profits in providing retailer with eftpos machines through set up fee, periodic service fee and transaction fee on each purchase put through an eftpos machine. banks also profit through the availability of interest free fund en-route to the designated retailer's bank account after being debited from the payee's account and the availability of the deposited fund into the retailer's account itself. in order for the banking sector to develop this part of the business further, it can make use of market segmentation modelling to gain better understanding of the business behaviours of the retailers using the eftpos facilities. the concept of market segmentation was first introduced in smith (1956) and defined as the “process of subdividing a market into distinct subsets of customers that behave in the same way or have similar needs. each subset may conceivably be chosen as a market target to be reached with a distinctive marketing strategy” (doyle 2011). the heart of any good market segmentation tool is in its ability to analyse, understand and draw good market segments based on customer purchasing behaviours. whilst market segmentation has been extensively applied on transaction data to find insights into customer purchasing behaviours in the market, very limited work has been done to find insights into retailer business activities. s117 australasian journal of information systems singh & rumantir 2015, vol. 19, pp. s117-s132 two-tiered clustering experiments figure 1 show the workflow followed in this paper in conducting the two-tiered experiments to find retailer business characteristics from the transactions they have generated through the eftpos facilities of a major bank in australia figure 1: two tiered market segmentation framework first, clustering techniques are employed on eftpos transaction data to identify segments of retailers who have certain common characteristics. the selection of attributes plays an important role in good clustering analysis. we use the rfm (recency, frequency, monetary value) analysis, popular in marketing, to reduce the data set for the clustering experiments. second, to gain further insights into the characteristics of the retailers in each cluster in the form of common business rules, we build a decision tree using the r, f, m values and a few other eftpos attributes which may help explain a particular retailer, e.g. the business the retailer is in, whether or not the eftpos machine used belongs to the bank the retailer is a customer of, the type of card with which the transaction has been paid, etc. the paper is organized as follows: section 2 gives a summary of the review of the literatures on market segmentation in the retail industry in the past decade; section 3 explains the first tier of the experiments in building the retailer segments, i.e. the data reduction/transformation using the rfm analysis, the clustering techniques used and the results of the experiments. section 4 explains the second tier of the experiments in finding insights into the retailer segment using tree-based classification, i.e. the data pre-processing of the additional eftpos attributes used, the resulting classification model and the business rules extracted; section 5 concludes the paper and explains our plans for the future. 2 related work table 1 gives the summary of the review of the literatures on related work on market segmentation in the retail industry with respect to the attribute selection techniques and clustering techniques employed. all of the papers reviewed report the use of transaction data as input for segmentation experiments on retail customers. only one paper, i.e. bizhani & tarokh (2011), reports segmentation experiments on eftpos retailers. this is the only work on eftpos data we have found in the literatures. the work reported in bizhani & tarokh (2011) is on a much smaller data set (30,524 eftpos terminals with 1,030,120 total transactions) and considers individual eftpos machines as “retailers”. the data set we have been acquiring for our work consists of 77.5 million transaction records of over 1 million unique retailers (each retailer may use multiple eftpos terminals) and each transaction instance has around 60 variables. hence, based on the volume of the eftpos data set used, this project can be categorised as a big data project. our experience in acquiring, secured-storing and processing the commercial in confidence eftpos data has been reported in singh, rumantir, south & bethwaite (2014). the second and third columns of table 1 are for the two attribute selection techniques identified in the literatures as the most popular for market segmentation in the retail industry, i.e. socio-demographic analysis and rfm analysis. kim et al. (2005) and lee & park (2005) report good results on customer segmentation modelling using socio-demographic characteristics of customer households (e.g. average size of households, average age of s118 australasian journal of information systems singh & rumantir 2015, vol. 19, pp. s117-s132 two-tiered clustering experiments residents, proportions of residents with different marital status, etc). previous research suggests that customer purchase history is far better and more accurate predictors of future purchase behaviour when compared to their demographic characteristics (gupta et al., 2006). this finding is in line with our finding as summarised in table 1 that rfm analysis on customer transactions is more widely used as attribute selection technique in market segmentation in the retail industry. related work attribute selection techniques clustering techniques sociodemographic analysis rfm analysis k-means hierarchic al clustering other (chen et al., 2012) (lefait & kechadi, 2010) x x (hsieh 2004) x x neural network (bizhani & tarokh, 2011) x x x unsupervised learning vector quantization (chen et al., 2012) x x (olson et al., 2009) x x (namvar et al., 2010) x x x (kim et al., 2005) (lee & park, 2005) x x neural networks (dennis et al. 2003) x x (ho et al. 2012) x genetic algorithms (salvador & chan 2004) x x (d. gaur & s. gaur 2013) x x x (zakrzewska & murlewski 2005) x x x density based clustering (alam et al. 2010) (yoon et al. 2013) x (li et al. 2009) x chameleon (suib & deris 2008) x hierarchical pattern based clustering table 1. summary of the review of the literatures on market segmentation in the retail industry in the past decade clustering is used in market segmentation because the number of possible segments in the industry is usually unknown. chen, sain, & guo (2012) use k-means clustering algorithm to segment customers of an online retail business. they pre-process the data set using the rfm s119 australasian journal of information systems singh & rumantir 2015, vol. 19, pp. s117-s132 two-tiered clustering experiments analysis and then use sas enterprise miner to perform the clustering analysis. their study concludes that only 5% of the total number of customers has contributed to 25% of the total sales. li, wang, & xu (2009) develop a two-stage clustering algorithm for customer segmentation of the customers of chinese petroleum corp. their research identifies a cluster of the most valuable customers to the corporation as the customers not only purchase petroleum frequently but also spend a lot of money on each purchase. with respect to clustering algorithms used, k-means and hierarchical clustering are found to be the most popular techniques for market segmentation in the retail industry. most of the papers use either decision tree or association rules analysis to generate business rules pertinent in the clusters. 3 clustering experiments for this project, eftpos transaction data are being collected from one of the four major banks in australia. this paper reports on the preliminary stage of our project where we use data from a total period of 18 days, starting from 19 september 2013 to 7 october 2013. each transaction record has 55 attributes. the 18 daily data files contain approximately 77.5 million transaction records from over 1 million unique retailers. this high volume of data makes even the basis operations, such as calculating the total monetary amount of each retailer from the data set, very time consuming and resource intensive. strategies to overcome this big data problem is reported in singh, rumantir, south & bethwaite (2014). 3.1 rfm analysis the rfm (recency, frequency, monetary value) analysis was proposed in hughes (2006). this paper proposes the use of these three attributes to group retailers exhibiting similar business activities: recency the recency value of a retailer is the time interval between a global datum and his/her latest transaction. a retailer with a smaller recency value is seen to be more current in his/her business activities than a retailer with a bigger recency value. the midnight after the day of the last transaction in the data set (i.e. the midnight of 8 october 2013 (00:00:000000 in hh:mm:ssssss format) is chosen as the global datum. hence, the recency value of each retailer is calculated from the date of his/her latest transaction up to midnight on 8 october 2013. frequency the total number of transactions put through all of the eftpos terminals belonging to a retailer forms the frequency value of the retailer. monetary value the total amount of transactions put through all of the eftpos terminals belonging to a retailer forms the monetary value of the retailer. figure 2 shows the bar chart of the distribution of the recency, frequency and monetary values of all of the retailers in the data set. the bar chart shows that the frequency and monetary values are skewed to the lower end of the spectrum. the bar chart in figure 2 is open ended at the top end because there are small numbers of very large values of the three attributes in the data set. this is shown more clearly in the bar charts in figure 3 where the horizontal axis of each bar chart is intentionally “squeezed” in the middle to show these extreme values and also because there are very few data in this range (no data for frequency and monetary values). figure 3 also shows the correlations amongst the three attributes. there is positive correlation between frequency and monetary values and no correlation between recency with the other two attributes. this implies that retailers that generate a lot of transactions tend to accumulate large monetary values within the observation time period. s120 australasian journal of information systems singh & rumantir 2015, vol. 19, pp. s117-s132 two-tiered clustering experiments figure 2. the distribution of recency, frequency and monetary values of all retailers 3.2 clustering techniques being the most popular methods for market segmentation in the retail industry found in the literatures, we use k-means and agglomerative hierarchical clustering (ahc) in this preliminary work. clustering analysis on a large data set is time consuming and processor intensive. for this work, parallelisation in the form of multi-threading using a cluster of intel xeon and amd opteron cpus of various clock speeds with 16 cores, 32 gb ram and 500gb hard disk size is employed. our experience in acquiring, secured-storing and processing the commercial in confidence eftpos data is reported in singh, rumantir, south & bethwaite (2014). figure 3. the correlations amongst the three attributes. the middle of each of the x-axis are deliberately truncated to show the small number of extreme values at the upper end, made possible as there are no data frequency and monetary values and very few recency values in the middle range s121 australasian journal of information systems singh & rumantir 2015, vol. 19, pp. s117-s132 two-tiered clustering experiments for the k-means clustering, the calculation of the euclidean distance between each data point to the centroid of a cluster is done in parallel over subsets of the data set. for the ahc, the calculation of the ward minimum variance to evaluate the inter-cluster distance measure (starting with clusters with 1 member data point) and the creation of the clusters through agglomerative process are done on 60% of the data set with the remaining 40% are allocated to the clusters based on euclidean distance. in this project, we create clustering models with 2 to 25 clusters using both k-means and ahc. both sets of clustering models are then tested based on dunn’s index (dunn† 1974) and davisbouldin index (davies & bouldin 1979) to find the optimum number of clusters which result in high intra-cluster similarity and high inter-cluster dissimilarity. dunns index is given as: where: is the inter-cluster distance between clusters ci and cj; is the intra-cluster distance of cluster ck; c is the number of clusters. also davis-bouldin index is given as: where: is the inter-cluster distance between two clusters ci and cj; and are the intra-cluster distances of clusters ci and ck respectively; c is the number of clusters. two inter-cluster distance measures and two intra-cluster distance measures are used to evaluate these two indices as outlined in table 2. table 2. two pairs of inter-cluster and intra-cluster distance measures to be used in dunns and davies bouldin indices. 3.3 clustering results and analysis the dunns and davies–bouldin indices for each of the 24 clustering models (with 2 to 25 clusters respectively) created using each of the two clustering techniques are plotted in figure 4 and figure 5 respectively. comparing the results between the k-means and ahc models, we can see that the ahc models with 16 and 17 clusters have higher dunn’s indices than any of the k-means models. the k-means models with 13 clusters or more tend have higher davies– bouldin indices than the ahc models with the same number of clusters. it is then decided to analyse the ahc model with 19 clusters with a view that manual merging of clusters might be warranted post-analysis. table 3 shows the ahc clusters, each with the centroid represented by the average rfm values. each of the r, f, m values in the data set is divided into 3 quantiles, with labels 1 to 3. based s122 australasian journal of information systems singh & rumantir 2015, vol. 19, pp. s117-s132 two-tiered clustering experiments on the centroid values, each cluster is assigned a 3 digit label which values depend on which range each of its r, f, m levels falls in. low level in recency suggests the retailer has recent transactions. hence, low level in recency and very high levels in both frequency and monetary value suggest the feature of a retailer with active, successful business. in table 3, these retailers (labelled as active profitable retailers) can be found in clusters 1, 2, 18 (constitute 34.23% of the total retailers). the bank may wish to provide incentives for these retailers to maintain their excellent business performance. figure 4. a plot of the dunns index of each model created using k-means technique and the dunns index of each model created using agglomerative hierarchical clustering technique figure 5. a plot of the davis-bouldin index of each model created using k-means technique and the dunns index of each model created using agglomerative hierarchical clustering technique there are retailers with high levels of frequency and monetary values but with moderately high recency level (in clusters 15, 16, and 19, which constitute 30.26% of the total retailers). these retailers (labelled as recently less active retailers) are very valuable to the bank as they have generated a lot of transactions and had a high revenue level despite having been less active lately. however they are at risk of becoming even less active which will inadvertently lower s123 australasian journal of information systems singh & rumantir 2015, vol. 19, pp. s117-s132 two-tiered clustering experiments their frequency and monetary levels. the bank needs to take immediate remedial actions to ensure that this risk can be alleviated. retailers characterized by moderately high recency level with medium levels of frequency and monetary values (labelled as used to be active profitable retailers) are seen in clusters 3, 4, 5, 6, 11, 12, 13, 14 and 17 which constitute 27.63% of the total retailers. these retailers have businesses with just moderate level of activities obviously as a result of having been inactive for some time. the bank may want to engage these retailers into more active participation in generating eftpos transactions as this will consequently boost the levels of their frequency and monetary values. cluster # amount of data (%) average recency average frequency average monetary value r,f,m labels (1 to 3 quantiles) 1 10.30 2.368704 0.004604 0.003177 133 2 15.27 1.018574 0.038917 0.017611 123 3 2.41 67.028300 0.000117 0.000333 322 4 2.30 71.897224 0.000105 0.000313 322 5 1.72 47.020206 0.000343 0.000331 322 6 3.20 42.105650 0.000293 0.000681 322 7 1.31 82.475670 0.000085 0.000163 312 8 2.14 76.985930 0.000071 0.000287 312 9 2.11 97.306885 0.000025 0.000220 312 10 2.32 92.311350 0.000067 0.000303 312 11 3.25 57.261173 0.000191 0.000473 322 12 2.79 62.190346 0.000169 0.000406 322 13 5.12 27.091051 0.000393 0.000818 323 14 4.46 32.134422 0.000318 0.000703 322 15 7.36 21.853570 0.000626 0.001149 223 16 11.78 17.150494 0.001816 0.002548 233 17 2.39 52.247010 0.000147 0.000263 322 18 8.66 6.925114 0.011165 0.004900 133 19 11.12 12.214928 0.002484 0.002045 233 table 3. clustering results retailers characterized by very high recency level with low level of frequency value and medium level of monetary value are seen in clusters 7, 8, 9 and 10. these retailers have been inactive in the eftpos network for a long period of time and/or have low level business activities overall. they constitute 7.88% of the eftpos market for the bank. the bank can see this type of retailers as growth area in the eftpos business sector, hence can be labelled as growth area retailers. it however needs to determine if it is cost effective to launch aggressive marketing strategy to help improve the performance of these retailers. if it is decided that no aggressive campaigning is to be done for these retailers, due to the size of this market, some kind of marketing campaign will nevertheless still be required to maintain them. figure 6 shows the aggregate breakdown of the retailers. s124 australasian journal of information systems singh & rumantir 2015, vol. 19, pp. s117-s132 two-tiered clustering experiments figure 6. retailer categories: aggregate breakdown of the 19 clusters of retailers shown in table 3 4 decision tree experiments to find further insights into the 19 retailer clusters, a decision tree is built using the r, f and m values together with other transaction attributes that may have information to better understand the characteristics of the retailer who has generated the respective transactions. the process of building the decision tree will determine if an attribute used in the experiments is indeed a relevant characteristic of a retailer. 4.1 data preprocessing in addition to the r, f and m values, table 4 shows the attributes in the eftpos transaction data that are used in the decision tree experiments. data pre-processing in the form of reducing of the number of possible values an attribute can have is done for most of the attributes. the link between the retailer’s financial institution id (fiid) and the customer’s fiid can be explored further. since we are only interested in the transactions that have to do with the retailers of the bank under study (we call it thebank), we can just use binary values for these attributes. also, we observe that there are the following 4 possible combinations of values in the transactions. this leads us to replace both attributes with a new combined attribute having 4 possible pairs of binary values: thebank’s eftpos machine? = yes and thebank’s card? = yes 11 thebank’s eftpos machine? = yes and thebank’s card? = no 10 thebank’s eftpos machine? = no and thebank’s card? = yes 01 thebank’s eftpos machine? = no and thebank’s card? = no 00 s125 australasian journal of information systems singh & rumantir 2015, vol. 19, pp. s117-s132 two-tiered clustering experiments no name description original values preprocessed values end up in decision tree? 1 standard industrial classification (sic) code a four-digit code to identify the industry the retailer is in a four-digit code representing division, sub-division, group and class 1 digit code with a total of 11 divisions 2 retailer’s financial institution id (fiid) the fiid of the institution with which the retailer is associated – the owner of the eftpos machine at retailer the name of a bank thebank or other bank (yes/no) 3 customer’s financial institution id (fiid) the fiid of the institution that issued the card the name of a bank thebank or other bank (yes/no) 4 transaction code a code identifying the type of transaction 18 types 3 new types: purchase related, cash advanced, enquiries 5 card type a code identifying the card type associated with the transaction 3 types debit, credit, other 6 account type a code identifying the type of account associated with the transaction. 4 types dda, savings, credit, none table 4. eftpos attributes used for decision tree experiments 4.2 decision tree induction as illustrated in figure 7, a decision tree starts with a root node and ends with leaf nodes. in our project, the root node is the attribute that is the most useful in identifying which class a retailer belongs to. the leaf nodes represent the classes. in the middle are the child nodes, each child node is an attribute with branches representing the possible values of the attribute. business rules can be extracted by traversing down the decision tree from the root node to each of the leave nodes. in this project, the 19 clusters a retailer can belong to are the 19 classes of the decision tree. s126 australasian journal of information systems singh & rumantir 2015, vol. 19, pp. s117-s132 two-tiered clustering experiments figure 7. a generic decision tree to build the decision tree we need to determine which attribute to place at the root node, and the subsequent child nodes. the concept of information gain is used to make this choice at each node. information gain is the difference between the amount of information that is needed to make a correct classification before a decision/split at a node is made, and the amount of information required after the split. if the amount of information required after the split is lower, then the split is said to have decreased the disorder in the original data. the attribute that yields the most information gain will be selected to make the split at a node. we use entropy, the most common measure of the amount of information at a node: (equation 1) where: pi is the fraction of the instances classified as class i. in this paper, once the decision tree has been created, pruning is done by removing child nodes which (1) have been selected based on very small amount of information gain and (2) lead to leaf nodes with very small number of instances. this results in only two of the attributes in table 4, i.e. sic code and retailer’s fiid (i.e. if the eftpos machine belongs to thebank) ending up in the final decision tree. this means the type of card used in a transaction is not significant for the purpose of retailer classification. majority voting is the simplest most common way of labelling a cluster/class. majority voting is employed for classifying a leaf node where not all instances belong to the same class. the decision tree is built using 80% of the total number of retailers and is tested using the remaining 20%. the performance of the decision tree is evaluated by calculating the classification accuracy of the retailers into their respective clusters:= (equation 2) the final decision tree for the retailer classification shown in figure 8 has 72% accuracy on test data. s127 australasian journal of information systems singh & rumantir 2015, vol. 19, pp. s117-s132 two-tiered clustering experiments 4.3 business rules extraction and analysis the decision tree shown in figure 8 has recency as the root node. this means the results of the entropy calculation used to build the decision tree is consistent with the findings in marketing and advertising literatures that out of the three values in rfm analysis, recency is the most important factor in predicting the likelihood of a customer performing a repeat business based on past purchasing behaviour. figure 8. decision tree for the retailer classification into segments using the r, f, m values and the additional eftpos attributes in figure 8, r, f and m represent recency, frequency and monetary values respectively. q1, q2 and q3 are the three quantiles of the rfm values. c1 to c19 represents clusters 1 to 19. d5 (transportation, communication, electric, gas and sanitary services), d7 (retail trade) and d9 (services) are the industry divisions of the sic codes. in figure 8 we see that monetary value and frequency are high up in the tree as expected. this means the additional eftpos attributes, i.e. whether or not the eftpos machine belongs to thebank and sic code, serve as additional explanations of the retailer clusters which have been built with just using the three attributes, recency, frequency and monetary value. more specifically, by using the retailer categories given in the pie chart in figure 6, we can see that the left hand side of the decision tree in figure 8 lead to the classification retailers into the active profitable retailers category (clusters 1, 2, 18). in addition to the rfm characteristics that have been identified from the clustering experiments, the decision tree shows that retailers in clusters 1 and 2 use thebank’s eftpos machines while retailers in cluster 18 doesn’t, they use other bank’s eftpos machines. the bank may see this as a business opportunity to offer the retailers in cluster 18 to use thebank’s eftpos machines. with the recently less active retailers category (clusters 15, 16 and 19), the decision tree reveals that the retailers in cluster 19 operate in the services industry division and those in both clusters 15 and 16 operate in the retail trade industry division. also since it is revealed that retailers in cluster 15 use thebank’s eftpos machines. this means, if the bank would like to launch a remedial action in a cost effective manner, it can start with retailers in cluster 15. s128 australasian journal of information systems singh & rumantir 2015, vol. 19, pp. s117-s132 two-tiered clustering experiments with the used to be active profitable retailers, the decision tree reveals that the retailers in cluster 17 are in the retail trade division and do not use thebank’s eftpos machines. the decision tree does not provide additional insights in to the other retailer clusters in this category (i.e. clusters 3, 4, 5, 6, 11 and 12). further investigations, e.g. in the form of using more specific sic codes and finding other more relevant attributes, might need to be done. with the growth area retailers category, the decision tree reveals that the retailers in cluster 7 are in transportation, communication, electric, gas and sanitary services industry division. this may indicate that thebank’s eftpos market share in this industry division may be shrinking. table 5 gives the summary of the rules extracted from the decision tree. each rule is evaluated using the lift metric: (equation 3) it shows that all the rules have lift values much more than 1 which indicates that the rules are significant. retailer category rule lift active profitable retailers (r=q1 && m=q3 && bank's eftpos=yes && f=q3) => c1 6.94 (r=q1 && m=q3 && bank's eftpos=yes && f=q2) => c2 4.64 (r=q1 && m=q3 && bank's eftpos=no) => c18 8.44 recently less active retailers (r=q2 && m=q3 && sic=d7 && f=q2 && bank's eftpos=yes) => c15 9.60 (r=q2 && m=q3 && sic=d7 && f=q3) => c16 5.85 (r=q2 && m=q3 && sic=d9) => c19 5.89 used to be active profitable retailer (r=q3 && m=q2 && f=q1 && sic=d7) => (c3, c4, c6) needs further investigation(r=q3 && m=q2 && f=q2 && bank's eftpos=no) => c5 (r=q3 && m=q2 && f=q2 && bank's eftpos=yes && sic=d9) => (c11, c12) (r=q3 && m=q3) => c13 15.84 (r=q2 && m=q3 && sic=d7 && f=q2 && bank's eftpos=no) => c17 15.88 growth areas (r=q3 && m=q2 && f=q1 && sic=d5) => c7 16.64 (r=q3 && m=q2 && f=q1 && sic=d7) => (c8, c9, c10) needs further investigation table 5. business rules extracted from the decision tree in figure 8 5 conclusions and future work this paper proposes the use of clustering and classification techniques to analyse retailers on an eftpos network. the clustering experiments group the retailers based on the similarities in their business activities as characterized by how recent their business activities are, how frequent they conduct their business on the eftpos network and how much money their business activities have generated over a period of time. the preliminary results show that there are distinct combinations of rfm values of retailers in the clusters that may give the bank indications of the different marketing strategies that can be applied to each of the retailer types. to further analyse each cluster, we build a decision tree to find out more on the characteristics of the business and the background of the retailers. out of the six additional eftpos s129 australasian journal of information systems singh & rumantir 2015, vol. 19, pp. s117-s132 two-tiered clustering experiments attributes used to build the decision tree, two attributes namely the standard industrial classification (sic) code of the retailers and whether or not the retailer uses the eftpos machine belonging to the bank under study, have been calculated to have significant influences to the classification of a retailer into a cluster, hence are included in the final decision tree. these attributes provide further explanations into the characteristics of the retailers in their respective clusters. the next step of this project will be broken down into 3 categories. first, observing if there are latent variables in the data set that may influence the variations in the volume of transactions in different days and possibly different periods in a day. second, experimenting other clustering techniques to see if better quality clusters can be formed. third, building improved classification or causal models to find explanatory rules on the characteristics of each cluster by using more specific sic code and adding exogenous variables like socio-demographic, advertising, and social media data. the data set used in this preliminary work is just from the first 18 days of our eftpos data extraction. we have since collected a few months of eftpos transaction data that will allow us to conduct more extensive big data analysis with consequently more convincing results. this will also open up new research avenues into the kinds of suitable big data computing techniques for market segmentation projects involving mapreduce/hadoop system that we have put in place for this project. acknowledgments the authors wish to thank annie south, jon scheele and jo spencer for their help in data acquisition and blair bethwaite for his help in setting up the secure computing environment to enable the execution of this commercial in confidence big data project. references alam, s. et al., 2010. particle swarm optimization based hierarchical agglomerative clustering. in web intelligence and intelligent agent technology (wi-iat), 2010 ieee/wic/acm international conference on. pp. 64–68. bizhani, m. & tarokh, m.j., 2011. behavioral rules of bank’s point-of-sale for segments description and scoring prediction. int. j. industrial eng. comput, 2, pp.337–350. chen, d., sain, s.l. & guo, k., 2012. data mining for the online retail industry: a case study of rfm model-based customer segmentation using data mining. journal of database marketing \& customer strategy management, 19(3), pp.197–208. chen, y.-s. et al., 2012. identifying patients in target customer segments using a two-stage clustering-classification approach: a hospital-based assessment. computers in biology and medicine, 42(2), pp.213–221. davies, d.l. & bouldin, d.w., 1979. a cluster separation measure. pattern analysis and machine intelligence, ieee transactions on, (2), pp.224–227. dennis, c. et al., 2003. market segmentation and customer knowledge for shopping centers. in information technology interfaces, 2003. iti 2003. proceedings of the 25th international conference on. pp. 417–424. doyle, c., 2011. a dictionary of marketing, oxford university press. dunn†, j.c., 1974. well-separated clusters and optimal fuzzy partitions. journal of cybernetics, 4(1), pp.95–104. eftpos annual report, eftpos australia, http://www.eftposaustralia.com.au/about/annual-reports/ (accessed july 2015) gaur, d. & gaur, s., 2013. comprehensive analysis of data clustering algorithms. in future information communication technology and applications. springer, pp. 753–762. s130 australasian journal of information systems singh & rumantir 2015, vol. 19, pp. s117-s132 two-tiered clustering experiments gupta, s., hanssens, d., hardie, b., kahn, w., kumar, v., lin, n.,sriram, s. (2006). modeling customer lifetime value. journal of service research, 9(2), 139–155. ho, g.t. et al., 2012. customer grouping for better resources allocation using ga based clustering technique. expert systems with applications, 39(2), pp.1979–1987. hsieh, n.-c., 2004. an integrated data mining and behavioral scoring model for analyzing bank customers. expert systems with applications, 27(4), pp.623–633. hughes, a.m., 2006. strategic database marketing, mcgraw-hill. kim, y. et al., 2005. customer targeting: a neural network approach guided by genetic algorithms. management science, 51(2), pp.264–276. lee, j.h. & park, s.c., 2005. intelligent profitable customers segmentation system based on business intelligence tools. expert systems with applications, 29(1), pp.145–152. lefait, g. & kechadi, t., 2010. customer segmentation architecture based on clustering techniques. in digital society, 2010. icds’10. fourth international conference on. pp. 243–248. li, j., wang, k. & xu, l., 2009. chameleon based on clustering feature tree and its application in customer segmentation. annals of operations research, 168(1), pp.225–245. namvar, m., gholamian, m.r. & khakabi, s., 2010. a two phase clustering method for intelligent customer segmentation. in intelligent systems, modelling and simulation (isms), 2010 international conference on. pp. 215–219. olson, d.l. et al., 2009. comparison of customer response models. service business, 3(2), pp.117–130. salvador, s. & chan, p., 2004. determining the number of clusters/segments in hierarchical clustering/segmentation algorithms. in tools with artificial intelligence, 2004. ictai 2004. 16th ieee international conference on. pp. 576–584. singh, a., rumantir, g., south, a., bethwaite, b.: clustering experiments on big transactional data for market segmentation. proceedings of the third ase international conference on big data science and computing, beijing (2014), acm 978-1-4503-2891-3/14/08, http://dx.doi.org/10.1145/2640087.2644161 singh, a., rumantir, g., south, a.: market segmentation of eftpos retailers. proceedings of the twelfth australasian data mining conference, brisbane, conferences in research and practice in information technology, vol. 158. richi nayak, xue li, lin liu, kokleong ong, yanchang zhao, paul kennedy eds (2014) (in press) smith, w.r., 1956. product differentiation and market segmentation as alternative marketing strategies. the journal of marketing, 21(1), pp.3–8. suib, d.s. & deris, m.m., 2008. an efficient hierarchical clustering model for grouping web transactions. international journal of business intelligence and data mining, 3(2), pp.147–157. yoon, s.-h. et al., 2013. a data partitioning approach for hierarchical clustering. in proceedings of the 7th international conference on ubiquitous information management and communication. p. 72. zakrzewska, d. & murlewski, j., 2005. clustering algorithms for bank customer segmentation. in intelligent systems design and applications, 2005. isda’05. proceedings. 5th international conference on. pp. 197–202. copyright: © 2015 singh & rumantir. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which s131 australasian journal of information systems singh & rumantir 2015, vol. 19, pp. s117-s132 two-tiered clustering experiments permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. s132 paper title (paper title style) australasian journal of information systems ahmad, traiq, shabbir & khan 2017, vol 21, selected papers from ausdm spectral methods for immunization of large networks 1 spectral methods for immunization of large networks muhammad ahmad department of computer science school of science and engineering lahore university of management sciences pakistan juvaria tariq department of computer science school of science and engineering lahore university of management sciences pakistan mudassir shabbir department of computer science information technology university pakistan imdadullah khan department of computer science school of science and engineering lahore university of management sciences pakistan imdad.khan@lums.edu.pk abstract given a network of nodes, minimizing the spread of a contagion using a limited budget is a well-studied problem with applications in network security, viral marketing, social networks, and public health. in real graphs, virus may infect a node which in turn infects its neighbour nodes and this may trigger an epidemic in the whole graph. the goal thus is to select the best k nodes (budget constraint) that are immunized (vaccinated, screened, filtered) so as the remaining graph is less prone to the epidemic. it is known that the problem is, in all practical models, computationally intractable even for moderate sized graphs. in this paper we employ ideas from spectral graph theory to define relevance and importance of nodes. using novel graph theoretic techniques, we then design an efficient approximation algorithm to immunize the graph. theoretical guarantees on the running time of our algorithm show that it is more efficient than any other known solution in the literature. we test the performance of our algorithm on several real world graphs. experiments show that our algorithm scales well for large graphs and outperforms state of the art algorithms both in quality (containment of epidemic) and efficiency (runtime and space complexity). keywords: graph immunization; eigendrop; closed walks 1 introduction consider a large network of hospitals distributed, geographically, over a region. due to presence of an active pathogen, there is a danger of pandemic in the region. we assume that pathogen can easily travel between linked hospitals. we are concerned with keeping the maximum of the hospital network clean of pandemic. based on their geographic vicinity (or some other criteria), it is known for every pair of hospitals x,y whether contamination at x forebodes certain contamination at y. if this is true about some pair of hospitals x,y, we call x and y as linked (i.e. x and y is an edge in the network graph). now a team of scientists have developed a vaccine for the pathogen. unfortunately, due to cost or production constraints the vaccine is only available in limited quantity and only a small fraction of hospitals can receive it. it is assumed that once a hospital receives the vaccine it australasian journal of information systems ahmad, traiq, shabbir & khan 2017, vol 21, selected papers from ausdm spectral methods for immunization of large networks 2 cannot be contaminated nor can it contaminate any other hospital. given the scarcity of the vaccine resource it is a very serious question to ask whether a given hospital should or should not receive the vaccine so as to minimize the overall spread of the pathogen in the region. this question, in essence, is the topic of this work, which appears in various other scenarios. an efficient solution to this problem can be applied to diverse array of high-impact applications in public health. it will also be an important tool in cyber-security solutions. finding important players in social networks is a fundamental problem in viral marketing, online advertisement and social networks monitoring. in its essence, the problem of finding important nodes in a network can be reduced to the main problem of this paper. this is known, in the literature, as the network immunization problem. the problem is studied in terms of graphs where each node represents a hospital (or any other resourcehungry entity) and an edge between a pair of nodes represents the link between the pair of hospitals. we use the susceptible−infected−susceptible (sis) model of infection spread in the network that is detailed in section 3. as in (chen 2016), we formulate the problem as follows. problem 1: given a simple undirected graph g = (v,e) and an integer k find a set s of k nodes such that if we “immunize” s, renders g least “vulnerable” to an attack over all choices of s. clearly, the problem is ill-defined, unless a precise and quantifiable definition of the vulnerability of a network is provided. to this end, it turns out that the largest eigenvalue of the adjacency matrix of the graph is a good measure of vulnerability of the graph (chakrabarti 2008). 1.1 problem formulation and solution approach for a simple undirected graph g=(v,e) with |v|=n and |e|=m, the adjacency matrix ag (or just a whenever the graph g is clear from the context) is an n×n binary matrix with columns and rows representing the vertices of g and cell entries representing the edges i.e. ag(i,j)=1 if and only if there is an edge from vertex i to vertex j. eigen spectrum, {𝜆𝑖}𝑖=1 𝑛 , of adjacency and laplacian matrices has been studied a lot for certain graph properties (chung 1997). of particular interest to us is the largest eigenvalue (denoted by λ1(a) or λ1(g) or λ1 when g is obvious from the context) of the adjacency matrix also referred to as the first eigenvalue in literature. epidemic threshold is an intrinsic property of a network. it is well known in the epidemiology literature that if the “virus strength” is more than the network's epidemic threshold, then an outbreak will occur. the epidemic threshold of an undirected graph g is directly proportional to λ1(g) (chakrabarti 2008). for this reason in the epidemiology community, the largest eigenvalue of graphs is used to study mathematical models of diseases spreads and outbreaks (ganesh 2005). in general λ1 is related to the global connectivity of a graph. for example we know that δ ≥ 𝜆1 ≥ 𝑑𝑎𝑣𝑔 where δ and davg are the maximum and average degrees in the graph, respectively (west 2001). in the context of the discussion above it seems only logical to delete/immunize the vertices in g such that the remaining graph has minimum possible largest eigenvalue. let g=(v,e) be a graph and let a be its adjacency matrix. for a subset of vertices s ⊆ v, we denote by g-s the subgraph induced on vertices in v∖s. similarly, for s ⊆ v, we define a-s to be the submatrix of a obtained by deleting rows and columns in a at corresponding indices in s. by definition of induced subgraph, it is clear that a-s is the adjacency matrix of g-s. we will denote by gs the subgraph of g induced by vertices in s and as denotes the adjacency matrix of gs. the precise formulation of the above problem is given as follows: problem 2: given a simple undirected graph g = (v,e) find a set s of k vertices so that λ1(a-s) is the minimum possible over all choices of s ⊆ v. australasian journal of information systems ahmad, traiq, shabbir & khan 2017, vol 21, selected papers from ausdm spectral methods for immunization of large networks 3 since λ1(g) can be computed in o(m) time (chen 2016), the natural algorithm to find the optimal set s takes time 𝑂 (( 𝑛 𝑘 ) 𝑚), which is exponential in k. indeed lemma 1 asserts that the general the problem is intractable (see section 2 for a proof). lemma 1: problem 2 is np-hard. in this work we propose a novel algorithm to approximately solve problem 2. our algorithm is quite intuitive, promises a better approximation compared to known approaches, and is easy to implement. we propose a combinatorial formulation of the problem in which we construct a subset of vertices based on a defined shield value. our shield value measures the contribution of the vertices towards the largest eigenvalue of the graph. definition of our shield value uses basic theoretical tools from spectral graph theory and relies on simple combinatorial interpretation of largest eigenvalue. removing vertices with high shield values would yield optimal immunization results. we provide an efficient approximation algorithm to estimate the shield values of vertices. we provide detailed analysis of running time and space complexity of our algorithm. we also empirically evaluate our algorithm on several real world data sets. the results show that our algorithm achieves much better immunization performance compared to previously known solutions. moreover, ideas developed in this work are general and can have many other potential applications. indeed this method has been successfully used for estimating the spectral radius of large graphs (abbas 2017). rest of the paper is organized as follows. in section 2 we provide a detailed background to problem 2 and discuss its computational intractability and approaches to approximate it. an extensive literature review of the immunization problem is provided in section 3. we propose our algorithm in section 4 along with its approximation guarantees and complexity analysis. section 5 contains results of experimentation on real world graphs. this section also provides comparison of performance of our algorithm with other known algorithms. afterwards we conclude with a discussion on future directions. a preliminary version of this paper has already been appeared in (ahmad 2016). 2 background as argued earlier, the brute-force solution, that checks the eigendrop achieved by each possible subset of size k and selects the subset achieving the largest eigendrop, has runtime 𝑂 (( 𝑛 𝑘 ) 𝑚), where m is the number of edges in the graph. this running time is exponential in size of input (k). we prove that the problem under consideration is np-hard. proof: a simple reduction from minimum vertex cover problem goes as follows: if there exist k vertices that cover all the edges in the graph, then from the following fact, deleting those vertices will imply that λ1(a-s) = 0. fact 1: empty graph (graph with no edges) on n vertices has eigenvalue zero with multiplicity n. conversely if there exists a k-set s of vertices such that λ1(a-s) = 0, then s must be a vertex cover; since otherwise it contradicts the following implication of the perron-frobenius theorem. fact 2: (serre 2002) deleting any edge from a simple connected graph g strictly decreases the largest eigenvalue of the corresponding adjacency matrix. for a reduction from max independent set problem that doesn't use perron-frobenius theorem see appendix of (chen 2016). although problem 2 is np-hard, the following greedy algorithm guarantees a (1 − 1 𝑒 )-approximation to the optimal solution to problem 2. approximation guarantee of greedy-1 follows from theorem 1. australasian journal of information systems ahmad, traiq, shabbir & khan 2017, vol 21, selected papers from ausdm spectral methods for immunization of large networks 4 algorithm 1 : greedy-1 (𝑮, 𝒌) 𝑆 ← ∅ 𝑤ℎ𝑖𝑙𝑒 |𝑆| < 𝑘 𝑣 ← 𝑎𝑟𝑔𝑚𝑎𝑥 v∈v∖s (𝜆1(𝐴−𝑆∪{𝑣} )) 𝑆 ← 𝑆 ∪ {𝑣} 𝑟𝑒𝑡𝑢𝑟𝑛 𝑆 theorem 1: (nemhauser 1978) let f be a non-negative, monotone and sub-modular function, 𝑓: 2𝛺 → ℝ. suppose 𝔸 is an algorithm, that choose a k elements set s by adding an element u at each step such that 𝑢 = 𝑎𝑟𝑔 𝑚𝑎𝑥 𝑥∈𝛺∖ 𝑆 𝑓(𝑆 ∪ {𝑥}). then 𝔸 is a (1 − 1 𝑒 )-approximate algorithm. in (chen 2016) each subset was assigned a score, called shield-value that was meant to approximate eigendrop achieved by that set. for a set s of size k, shield value, sv(s) can be computed in o(k2) when the eigenvector corresponding to largest eigenvalue is known. hence, the straightforward method of finding the set with largest shield value takes 𝑂 (( 𝑛 𝑘 ) 𝑘2) time. using the fact that the objective function based on shield value is sub-modular, (chen 2016) gave a greedy algorithm with runtime o(nk2+m). by theorem 1 their algorithm guarantees a (1 − 1 𝑒 )-approximation to the optimal shield value. 3 related work while the focus of this paper is to target node immunization problem using spectral graph theoretic techniques, there is vast amount of literature on this problem with approaches from diverse areas of the subject. initially in 2003 brieseneister, lincoln and porras (briesemeister 2003) studied the propagation styles of viruses in communication networks. along with this, the effects of graph topology in the spread of an epidemic are described in (ganesh 2005) and the conditions under which an epidemic will eventually die out are discussed. similarly chakrabarti et. al in (chakrabarti 2008) devise a nonlinear dynamical system (nlds) to model virus propagation in communication networks. they use the idea of birth rate β, death rate δ, and epidemic threshold τ, for a virus attack where birth rate is the rate with which infection propagates, death rate is the node curing rate and epidemic threshold is a value such that if 𝛽 𝛿 < 𝜏, infection will die out quickly else if 𝛽 𝛿 > 𝜏 infection will survive and will result in an epidemic. virus propagation is studied in both directed and undirected graphs. for undirected graphs, they prove that epidemic threshold τ equals 1/λ, where λ is the largest eigenvalue of the graph. thus for a given undirected graph, if β/δ>1/λ, then the epidemic will die out eventually. but none of these specifically discuss the graph immunization problem. afterwards the problem is studied using the edge manipulation scheme. in (kuhlman 2013) dynamical systems are used to delete appropriate edges to minimize contagion spread. while tong et al. in (tong 2012) remove k edges from the graph in a manner that eigendrop (difference in largest eigenvalues of original and resultant graphs) is maximized. for this edges are selected on the basis of left and right eigenvectors of leading eigenvalue of the graph such that for each edge ex, score(ex) is the dot product of the left and right eigenvectors of leading eigenvalue of the adjacency matrix of a. graph vulnerability is defined as measure of how much a graph is likely to be affected by a virus attack. as in (tong 2012), the largest eigenvalue is selected as a measure of graph vulnerability, in (chen 2016) they also use largest eigenvalue for the purpose but instead of removing edges, nodes are deleted to maximize the eigendrop. undirected, unweighted graphs are considered and nodes are selected by an approximation scheme using the eigenvector corresponding to largest eigenvalue which cause the maximum drop. australasian journal of information systems ahmad, traiq, shabbir & khan 2017, vol 21, selected papers from ausdm spectral methods for immunization of large networks 5 zhang et al. and song et al. adapt the non-preemptive strategy i.e. selection of nodes for immunization is done after the virus starts propagating across the graph. for this they use discrete time model to obtain additional information of infected and healthy nodes at each time stamp. in (song 2015) directed and weighted graphs are used in which weights represent the probability of a healthy node being infected by its neighbours and node selection is done on the basis of these probabilities. then results are evaluated on the basis of save ratio (sr) which is the ratio between the number of infected nodes when k nodes are immunized over the number of infected nodes when no node is immunized. (zhang 2014a) and (zhang 2014b) consider undirected graphs and incorporates dominator trees for selecting nodes. results are evaluated in terms of expected number of remaining infected nodes in the graph after the process of immunization. in filter placement (erdös 2012), those nodes are identified which are cause of maximum information multiplicity. moreover some reverse engineering techniques are also used for similar problems. prakash et al. (prakash 2012) study the graphs in which virus has already spread for some time and they point out those nodes from where the spread started. from this they find out the likelihood of other nodes being affected. another direction to look at the problem is to consider graphs in which some nodes are already infected and these nodes can spread virus among other reachable nodes or graphs in which all nodes are contaminated and the goal is to decontaminate the graph by using some agent nodes which traverse along the edges of the graph and clean the nodes. the problem is usually referred to as decontamination of graph or graph searching problem. different models are studied to solve the problem and most of them assume the monotonicity in decontamination i.e. once a node is decontaminated then it cannot get contaminated again (bienstock 1991), (flocchini 2008), (flocchini 2007), (fraigniaud 2008). but non-monotonic strategies are also studied (daadaa 2016). other work that is related to node immunization is the selection of most influential nodes in a given network to maximize the diffusion of new information in a network. kempe et al. provided the provably efficient approximation algorithm for the problem (kempe 2003). seeman and singer (seeman 2013) use stochastic optimization models to maximize the information diffusion in social networks. influence maximization problem is slightly different from immunization problem as in influence maximization problem the goal is to select nodes for seeding which will maximize the spread on new idea while in node immunization problem goal is to select nodes which will help in minimal spread of virus. 4 our proposed algorithm in (chen 2016) they defined shield value that was an approximation to the eigendrop. we define our shield value to be the eigendrop and approximate the eigenvalue computation instead. we achieve better theoretical guarantees with our shield value definition. furthermore, it is easy and computationally efficient to approximate our shield value. 4.1 our shield value and its justification we use the following well known facts from linear algebra and graph theory. given an n×n matrix a, let trace of a be denoted by tr(a), fact 3: [c.f. (strang 1976)] 𝑡𝑟(𝐴) = ∑ 𝑎𝑖𝑖 𝑛 𝑖=1 = ∑ 𝜆𝑖(𝐴) 𝑛 𝑖=1 fact 4: [c.f. (west 2001) p. 455 ] 𝑡𝑟(𝐴𝑝) = ∑ 𝜆𝑖(𝐴𝑝) 𝑛 𝑖=1 = ∑ 𝜆𝑖(𝐴)𝑝 𝑛 𝑖=1 australasian journal of information systems ahmad, traiq, shabbir & khan 2017, vol 21, selected papers from ausdm spectral methods for immunization of large networks 6 clearly, for even powers p, tr(ap) is an upper bound on 𝜆1 𝑝 and as p grows tr(ap) approaches 𝜆𝑚𝑎𝑥 𝑝 (𝐴) where 𝜆𝑚𝑎𝑥 = max{|𝜆𝑖(𝐴)|}. we therefore, try to remove a set s of k vertices from the graph such that 𝑡𝑟((𝐴−𝑆)𝑝) is minimized. the goodness of a set s in this setting is defined as 𝑓𝑝(𝑆) = 𝑡𝑟((𝐴−𝑆)𝑝) (1) define 𝑔𝑝(𝑆) = 𝑡𝑟(𝐴𝑝) − 𝑡𝑟((𝐴−𝑆)𝑝) (2) note that minimizing fp(s) is same as maximizing gp(s). given a graph g=(v,e), a walk w of length l in g is a sequence of vertex 𝑣0, 𝑣1, … , 𝑣𝑙 s such that for 0 ≤ 𝑖 ≤ 𝑙 − 1 (𝑣𝑖 , 𝑣𝑖+1) ∈ 𝐸. we say that w is a walk from v0 to vl. if v0 = vl, then w is called a closed walk. for v∈ v, let ℂ𝕎𝑖(𝐺, 𝑣) be the set of closed walks of length i in g containing the vertex v. suppose 𝐶𝑊𝑖(𝐺, 𝑣) = |ℂ𝕎𝑖(𝐺, 𝑣)|. when x⊆v, ℂ𝕎𝑖(𝐺, 𝑋) = ⋃ ℂ𝕎𝑖(𝐺, 𝑥)𝑥∈𝑋 . we similarly denote by 𝐶𝑊𝑖(𝐺, 𝑋) to be the cardinality of the set ℂ𝕎𝑖(𝐺, 𝑋). when x=v, we refer to ℂ𝕎𝑖(𝐺, 𝑋) as ℂ𝕎𝑖(𝐺). so ℂ𝕎𝑖(𝐺) is the set of all closed walks of length i in g. we use the following well-known fact from graph theory fact 5:[c.f. (west 2001) p.455] given a graph g with adjacency matrix a, 𝐶𝑊𝑝(𝐺) = 𝑡𝑟(𝐴𝑝) = ∑ 𝜆𝑖 𝑝 𝑛 𝑖=1 . it is clear from the definitions and fact 5 that for ⊂ v, 𝐶𝑊𝑝(𝐺) = 𝐶𝑊𝑝(𝐺−𝑆, 𝑉 ∖ 𝑆) + 𝐶𝑊𝑝(𝐺, 𝑆). (3) this identity is same as the one already mentioned in (3). this similarly suggests a strategy namely one should delete a set s of k vertices such that 𝐶𝑊𝑝(𝐺−𝑆, 𝑉 ∖ 𝑆) is minimized for a large even integer p (equivalently 𝐶𝑊𝑝(𝐺, 𝑆) is maximized). the goodness of a set of vertices s⊆v, in terms of number of closed walks is given by 𝑔𝑝(𝑆) = 𝐶𝑊𝑝(𝐺, 𝑆) (4) which as noted above we would like to maximize. hence we reduced the problem of finding a set maximizing the eigendrop to the problem of finding a set of vertices that are part of many closed walks. we note that this latter problem is of a combinatorial nature and more amenable to techniques from graph theory. algorithm 2 : greedy-2 (𝑮, 𝒌, 𝒑) 𝑆 ← ∅ 𝒘𝒉𝒊𝒍𝒆 |𝑆| < 𝑘 𝒅𝒐 𝑣 ← 𝑎𝑟𝑔𝑚𝑎𝑥 𝐶𝑊𝑝 v∈v ∖s (𝐺𝑆, 𝑣) 𝑆 ← 𝑆 ∪ {𝑣} 𝒓𝒆𝒕𝒖𝒓𝒏 𝑆 an algorithm based on this intuition is given in algorithm greedy-2(g,k,p). quality guarantee of algorithm greedy-2(g,k,p) follows from sub-modularity of the optimization function (4) and theorem 1. next we show that the objective function given in (4) is nonnegative, monotone sub-modular, i.e. we show that gp(s) has the property of diminishing return. australasian journal of information systems ahmad, traiq, shabbir & khan 2017, vol 21, selected papers from ausdm spectral methods for immunization of large networks 7 lemma 2: the function gp(s) given in (4) is non-negative, monotonically non-decreasing and sub-modular. proof: since gp(s) counts the number of closed walks of length p, it is clearly non-negative. by definition of gp(s) for x⊆ y we have 𝑔𝑝(𝑌) − 𝑔𝑝(𝑋) = 𝐶𝑊𝑝(𝐺, 𝑌) − 𝐶𝑊𝑝(𝐺, 𝑌) = 𝐶𝑊𝑝(𝐺, (𝑌 ∖ 𝑋) ∪ 𝑋) − 𝐶𝑊𝑝(𝐺, 𝑋) = 𝐶𝑊𝑝(𝐺, 𝑋) + 𝐶𝑊𝑝(𝐺−𝑋, 𝑌 ∖ 𝑋) − 𝐶𝑊𝑝(𝐺, 𝑋) = 𝐶𝑊𝑝(𝐺−𝑋, 𝑌 ∖ 𝑋) ≥ 0 where the last inequality follows from non-negativity of gp(s). hence gp(s) is monotonically non-decreasing function. for sub-modularity of gp(s), let x,y,z⊂v such that x⊂y and z∩y=∅. let l=x∪z and r=y∪z. we have (𝑔𝑝(𝑅) − 𝑔𝑝(𝑌)) − (𝑔𝑝(𝐿) − 𝑔𝑝(𝑋)) = 𝐶𝑊𝑝(𝐺, 𝑅) − 𝐶𝑊𝑝(𝐺, 𝑌) − 𝐶𝑊𝑝(𝐺, 𝐿) + 𝐶𝑊𝑝(𝐺, 𝑋 = 𝐶𝑊𝑝(𝐺, 𝑌 ∪ 𝑍) − 𝐶𝑊𝑝(𝐺, 𝑌) − 𝐶𝑊𝑝(𝐺, 𝑋 ∪ 𝑍) + 𝐶𝑊𝑝(𝐺, 𝑋) = 𝐶𝑊𝑝(𝐺, 𝑌) + 𝐶𝑊𝑝(𝐺−𝑌, 𝑍) − 𝐶𝑊𝑝(𝐺, 𝑌) − 𝐶𝑊𝑝(𝐺, 𝑋) − 𝐶𝑊𝑝(𝐺−𝑋, 𝑍) + 𝐶𝑊𝑝(𝐺, 𝑋) = 𝐶𝑊𝑝(𝐺−𝑌, 𝑍) − 𝐶𝑊𝑝(𝐺𝑋, 𝑍) ≤ 0 where the last inequality follows from the fact that by definition ℂ𝕎𝑝(𝐺−𝑌, 𝑍) ⊆ ℂ𝕎𝑝(𝐺−𝑋, 𝑍), hence 𝐶𝑊𝑝(𝐺−𝑌, 𝑍) ≤ 𝐶𝑊𝑝(𝐺−𝑋, 𝑍) for a vertex v, 𝐶𝑊𝑝(𝐺, 𝑣) can be computed by the powers of a i.e 𝐴2, 𝐴3, ⋯ , 𝐴𝑝. clearly computing all these matrices for large p when g is significantly large as well is computationally expensive. so the algorithm greedy-2(g,k,p) is practically impossible to be executed. so we set p=4 to approximate the result and our practical experimentation shows that p=4 gives good enough quality guarantee. although we have also done approximation using p=6 in (tariq 2017), which certainly gives better results, we focus on p=4 only in this paper. first we give a closed form expression for 𝐶𝑊4(𝐺, 𝑣) in terms of degrees and codegrees. for a given graph g, 𝑁𝐺(𝑥) = {𝑦 ∈ 𝑉(𝐺): 𝐴𝐺(𝑥, 𝑦) = 1} and 𝑑𝐺(𝑥) = |𝑁𝐺(𝑥)|. define 𝑁𝐺(𝑥, 𝑦) = {𝑧 ∈ 𝑉(𝐺): 𝐴𝐺(𝑥, 𝑧) = 1 ∧ 𝐴𝐺(𝑦, 𝑧) = 1} to be the common neighborhood of x and y in g. let 𝑑𝐺(𝑥, 𝑦) = |𝑁𝐺(𝑥, 𝑦)|, note that 𝑁𝐺(𝑥, 𝑥) = 𝑁𝐺(𝑥) and 𝑑𝐺(𝑥, 𝑥) = 𝑑𝐺(𝑥 ). when g is clear in the context, we refer to 𝑁𝐺(𝑥, 𝑦) as n(x,y) and similarly to dg (x,y) as d(x,y). lemma 3: for any vertex v∈ v, 𝐶𝑊4(𝐺, 𝑣) = 2𝑑(𝑣)2 + 4 ∑ 𝑑(𝑢, 𝑣)2 {𝑢∈𝑉, 𝑢≠ 𝑣} . proof: a closed walk w:(v,x,u,y,v) of length 4 can be interpreted as the concatenation of two walks of length 2 with same end points u and v. the number of walks of length 2 with the end points u, v is 𝐴𝐺 2 (𝑢, 𝑣). we want to count the closed walks of length 4 that contain a fixed vertex v atleast once. note that v can occur at most twice in a closed walk of length 4. in a closed walk of length 4 there are 4 positions for vertices (since first and last vertex is same, we consider it one position). first we count the closed walks of length 4 that contain v exactly in one position. call the set of closed walks of length 4 with v at ith position as 𝐶4 (𝑣, 𝑖). this gives the total number of closed walks of length 4 with appearance of v exactly once as ∑ |𝐶4(𝑣, 𝑖)|4 {𝑖=1} . any closed walk in 𝐶4(𝑣, 1) is of the form (v,a,b,c,v), where a,b,c are vertices in v(g). number of these walks is ∑ (𝐴2(𝑣, 𝑏))2 {𝑏≠ 𝑣} . clearly (c,v,a,b,c) represents any closed walk in 𝐶4(𝑣, 2). note that for a fixed a,b and c, (c,v,a,b,c) is one position clockwise rotation of closed walk (v,a,b,c,v). this implies that corresponding to every closed walk in 𝐶4(𝑣, 1) there is a closed walk in 𝑖𝑛 𝐶4(𝑣, 2) and vice versa. hence the number of walks in 𝑖𝑛 𝐶4(𝑣, 2) is also ∑ (𝐴2(𝑣, 𝑏)) 2 {𝑏≠ 𝑣} . similarly we get australasian journal of information systems ahmad, traiq, shabbir & khan 2017, vol 21, selected papers from ausdm spectral methods for immunization of large networks 8 that the number of walks in 𝐶4(𝑣, 3) and 𝐶4(𝑣, 4) is also the same. so we have ∑ |𝐶4(𝑣, 𝑖) |4 {𝑖=1} = 4 ∑ (𝐴2(𝑢, 𝑣)) 2 { 𝑢≠ 𝑣} . second we consider closed walks in which v appears twice. now it is possible in two ways: 1) v appears in 1st and 3rd position as (v,a,v,c,v) or 2) v takes 2nd and 4th position as (a,v,c,v,a). clearly there are 2(𝐴2(𝑣, 𝑣))2 such walks. this gives total number of closed walks of length 4 containing a vertex v as 𝐶𝑊4(𝐺, 𝑣) = 2(𝐴2(𝑣, 𝑣))2 + 4 ∑ 𝐴2(𝑢, 𝑣)2 {𝑢≠ 𝑣} which is same as what we required. we incorporate the above formula in the following algorithm. for a given vertex v, 𝑠𝑐𝑜𝑟𝑒𝐺(𝑣) = 4 ∑ 𝑑(𝑢, 𝑣)^2 − 2𝑑(𝑣)^2{𝑢∈ 𝑉(𝐺)} then ∑ 𝑑(𝑢, 𝑣)2 {𝑢≠ 𝑣} can be computed by taking the characteristic vector 𝜒𝑣 of n(v) (a binary vector of length n where the 𝜒𝑣[𝑖] = 1 ↔ (𝑣, 𝑣𝑖) ∈ 𝐸). then for each 𝑣𝑒𝑟𝑡𝑒𝑥 𝑢 ∈ 𝑉 ∖ {𝑣} we go through each neighbour x of u in its adjacency list and check if 𝜒𝑣[𝑥] = 1 to increment d(v,u). it takes o(m) time to compute the 𝑠𝑐𝑜𝑟𝑒𝐺(𝑥) for a vertex x, to get the vertex with maximum score it takes o(nm) time. note that after removing k vertices (for constant k) the graph still has o(m) edges. now we give an efficient approximation to 𝑠𝑐𝑜𝑟𝑒𝐺(𝑥) that not only can be computed in linear time but also can be updated after removing a vertex y in time proportional to d(x). we have ( ∑ 𝑑(𝑢,𝑣){𝑢≠𝑣} 𝑛 ) 2 ≤ (∑ 𝑑(𝑢,𝑣)2 {𝑢≠𝑣} ) 𝑛 ≤ ( ∑ 𝑑(𝑢,𝑣)2 {𝑢≠𝑣} 𝑛 ) (5) the first inequality is the cauchy-schwarz inequality, [c.f (kreyszig 1989)], while the second follows from the fact that d(u,v) is non-negative for each u,v. in view of the above inequality, we approximate the 𝑠𝑐𝑜𝑟𝑒_𝐺(𝑣) by 𝑠𝑐𝑜𝑟𝑒_𝐺′(𝑣) given as 𝑠𝑐𝑜𝑟𝑒𝐺 ′ (𝑣) = 2𝑑𝐺 2 (𝑣) + 4(∑ 𝑑𝐺(𝑢,𝑣){𝑢≠ 𝑣} ) 2 . (6) our motivation to use 𝑠𝑐𝑜𝑟𝑒𝐺 ′ (𝑥) is that not only it is easy to compute but also after a vertex is deleted it is easy to update the scores of all vertices in the remaining subgraph. 4.2 algorithm now we give the algorithm to compute the results. first we show procedure to find 𝑠𝑐𝑜𝑟𝑒𝐺 ′ (𝑣) for graph g in algorithm compute-score(g), then in algorithm update-score(g,vi)we give algorithm to update the scores of vertices when a certain vertex vi is deleted from the graph and finally we discuss the greedy approach to approximate that which vertices should be deleted in order to immunize the graph in algorithm greedy-3(g,k) and along with these we discuss the time and space complexity in this section. now we give algorithm to compute 𝑠𝑐𝑜𝑟𝑒𝐺 ′ (𝑣); australasian journal of information systems ahmad, traiq, shabbir & khan 2017, vol 21, selected papers from ausdm spectral methods for immunization of large networks 9 algorithm 3 : compute-score (g) 1: deg ← 𝑍𝐸𝑅𝑂𝑆(𝑛) ⊳ 𝐼𝑛𝑖𝑡𝑖𝑎𝑙𝑖𝑧𝑒 𝑡ℎ𝑒 𝑑𝑒𝑔𝑟𝑒𝑒 𝑎𝑟𝑟𝑎𝑦 𝑡𝑜 𝑛 𝑧𝑒𝑟𝑜𝑠 2: 𝑐𝑜𝑑𝑒𝑔𝑆𝑢𝑚 ← 𝑍𝐸𝑅𝑂𝑆(𝑛) ⊳ 𝐼𝑛𝑖𝑡𝑖𝑎𝑙𝑖𝑧𝑒 𝑡ℎ𝑒 𝑐𝑜𝑑𝑒𝑔𝑟𝑒𝑒 𝑠𝑢𝑚 𝑎𝑟𝑟𝑎𝑦 𝑡𝑜 𝑛 𝑧𝑒𝑟𝑜𝑠 3: 𝑠𝑐𝑜𝑟𝑒𝐺 ′ ← 𝑍𝐸𝑅𝑂𝑆(𝑛) ⊳ 𝐼𝑛𝑖𝑡𝑖𝑎𝑙𝑖𝑧𝑒 𝑎𝑙𝑙 𝑠𝑐𝑜𝑟𝑒𝑠 𝑡𝑜 𝑧𝑒𝑟𝑜𝑠 4: 𝒇𝒐𝒓 𝑒𝑎𝑐ℎ 𝑣𝑒𝑟𝑡𝑒𝑥 𝑣𝑖 𝒅𝒐 5: 𝒇𝒐𝒓 𝑒𝑎𝑐ℎ 𝑛𝑒𝑖𝑔ℎ𝑏𝑜𝑟 𝑣𝑗 𝑜𝑓 𝑣𝑖 𝒅𝒐 6: degg[ 𝑣𝑖] ← degg[ 𝑣𝑖] + 1 7: 𝒇𝒐𝒓 𝑒𝑎𝑐ℎ 𝑣𝑒𝑟𝑡𝑒𝑥 𝑣𝑖 𝒅𝒐 8: 𝒇𝒐𝒓 𝑒𝑎𝑐ℎ 𝑛𝑒𝑖𝑔ℎ𝑏𝑜𝑟 𝑣𝑗 𝑜𝑓 𝑣𝑖 𝒅𝒐 9: 𝑐𝑜𝑑𝑒𝑔𝑆𝑢𝑚[𝑣𝑗] ← 𝑐𝑜𝑑𝑒𝑔𝑆𝑢𝑚[𝑣𝑗] + degg[𝑣𝑖] − 1 10: 𝒇𝒐𝒓 𝑒𝑎𝑐ℎ 𝑣𝑒𝑟𝑡𝑒𝑥 𝑣𝑖𝒅𝒐 11: 𝑠𝑐𝑜𝑟𝑒𝐺 ′ [𝑣𝑖] ← 2 ∗ degg[𝑣𝑖] 2 + 4 ∗ 𝑐𝑜𝑑𝑒𝑔𝑆𝑢𝑚[𝑣𝑖]2 lemma 4: runtime of algorithm compute-score(g) is o(m). proof: it is clear that line 6 of algorithm compute-score(g) takes o(1) time and it is executed o(m) times. since loop at line 5 is iterated over all neighbours of a fixed vertex, vi. hence for vi, line 6 is executed 𝑑𝐺(𝑣𝑖) times. thus for all 𝑣𝑖 ∈ 𝐺, line 6 runs for ∑ 𝑑_𝐺(𝑣_𝑖){𝑣𝑖∈𝑉(𝐺)} = 2𝑚 (bondy 1976). same is true for line 9. line 11 has constant time computation while it is computed for every vertex, thus it takes o(n) time. so total time taken to compute score of every vertex is o(m). for a given vertex v of g, we update the score of vertices after removing v in the following way; algorithm 4 : update-score (𝑮,𝒗𝒊) 1: 𝒇𝒐𝒓 𝑒𝑎𝑐ℎ 𝑛𝑒𝑖𝑔ℎ𝑏𝑜𝑟 𝑣𝑗 𝑜𝑓 𝑣𝑖 𝒅𝒐 2: degg[ 𝑣𝑗] ← degg[ 𝑣𝑗] − 1 3: 𝑐𝑜𝑑𝑒𝑔𝑆𝑢𝑚[𝑣𝑗] ← 𝑐𝑜𝑑𝑒𝑔𝑆𝑢𝑚[𝑣𝑗] − (degg[𝑣𝑖] − 1) 4: 𝒇𝒐𝒓 𝑒𝑎𝑐ℎ 𝑛𝑒𝑖𝑔ℎ𝑏𝑜𝑟 𝑣𝑘 𝑜𝑓 𝑣𝑗 𝒅𝒐 5: 𝑐𝑜𝑑𝑒𝑔𝑆𝑢𝑚[𝑣𝑘] ← 𝑐𝑜𝑑𝑒𝑔𝑆𝑢𝑚[𝑣𝑘] − 1 6: degg[ 𝑣𝑖] ← 0 7: 𝑐𝑜𝑑𝑒𝑔𝑆𝑢𝑚[ 𝑣𝑖] ← 0 8: 𝑠𝑐𝑜𝑟𝑒𝐺 ′ [𝑣𝑖] ← 0 9: 𝒇𝒐𝒓 𝑒𝑎𝑐ℎ 𝑛𝑒𝑖𝑔ℎ𝑏𝑜𝑟 𝑣𝑗 𝑜𝑓 𝑣𝑖 𝒅𝒐 10: 𝑠𝑐𝑜𝑟𝑒𝐺 ′ [𝑣𝑗] ← 2 ∗ degg[𝑣𝑗] 2 + 4 ∗ 𝑐𝑜𝑑𝑒𝑔𝑆𝑢𝑚[𝑣𝑗] 2 11: 𝒇𝒐𝒓 𝑒𝑎𝑐ℎ 𝑛𝑒𝑖𝑔ℎ𝑏𝑜𝑟 𝑣𝑘 𝑜𝑓 𝑣𝑗 𝒅𝒐 12: 𝑠𝑐𝑜𝑟𝑒𝐺 ′ [𝑣𝑘] ← 2 ∗ degg[𝑣𝑘]2 + 4 ∗ 𝑐𝑜𝑑𝑒𝑔𝑆𝑢𝑚[𝑣𝑘]2 lemma 5: algorithm update-score(g,vi) takes o(m) time to update score with respect to parameter vi. proof: in algorithm, line 2, 3 and 8 takes constant time steps and both are computed 𝑑𝐺(𝑣_𝑖) times. however line 5 and 12 are computed ∑ 𝑑𝐺(𝑣𝑗){𝑣𝑗∈ 𝑁𝐺(𝑣𝑖)} times which is upper bounded by m. hence for a fixed vertex vi runtime of algorithm is o(m). we give here the proof of correctness of the algorithms compute-score(g) and updatescore(g,vi). australasian journal of information systems ahmad, traiq, shabbir & khan 2017, vol 21, selected papers from ausdm spectral methods for immunization of large networks 10 lemma 6: for each vertex 𝑣 ∈ 𝑉(𝐺) algorithm compute-score(g) computes the score 𝐺, 𝑣𝑖 as defined in (5). proof: it is clear that the algorithm compute-score(g) computes the first term correctly, as each neighbour vi of v contributes 1 to the degree of v. to see why the second term is computed correctly, consider the following fact: fact 6: for ∈ 𝑉(𝐺) , ∑ 𝑑𝐺(𝑢, 𝑣){𝑢≠ 𝑣} = ∑ (𝑑𝐺(𝑤) − 1){𝑤∈ 𝑁𝐺(𝑣)} proof: the left hand side is counting all occurrences of all vertices w such that w is a common neighbour of u and some vertex v. essentially counting all paths of length 2 from u to v where w is the center vertex. we count these structures by counting the number of times each center vertex thus appear. since v is fixed, the number of times a vertex w appears in such a structure is exactly the number of neighbours of w, i.e. 𝑑𝐺(𝑤) times. now since v is fixed and it is also a neighbour of w, we subtract one from it. hence the expression on the right hand side follows. lemma 7: for all vertices 𝑢, 𝑣 ∈ 𝑉(𝐺) algorithm update-score(𝐺, 𝑣𝑖) computes the 𝑠𝑐𝑜𝑟𝑒𝐺−{𝑢} ′ (𝑣) as defined in (5). proof. to see this, consider a vertex v which is neighbour of u. we note that first contribution of u in score' of v is in the first term i.e. degree of v, so we decrease the degree of v by one. clearly from the figure below, u adds 𝑑𝐺(𝑢) − 1 in codegree sum of v, which is the second term of the 𝑠𝑐𝑜𝑟𝑒𝐺 ′ (𝑣), given as ∑ 𝑑𝐺(𝑢, 𝑣){𝑢≠ 𝑣} . now consider the vertices vi, which are neighbours of neighbours of u. score of these vertices is also effected by removing u since u contribute one to codegree sum of vi. here is the algorithm to select k vertices in the given graph g such that approximated eigendrop is maximized after deleting those vertices. algorithm 5 : greedy-3 (𝑮, 𝒌) 1: 𝑆 ← ∅ 2: compute-score(g) 3: 𝒘𝒉𝒊𝒍𝒆 |𝑆| < 𝑘 𝒅𝒐 4: 𝑣 ← 𝑎𝑟𝑔𝑚𝑎𝑥 𝑢∈𝑉∖𝑆 𝑠𝑐𝑜𝑟𝑒𝐺 ′ [𝑢] 5: 𝑆 ← 𝑆 ∪ {𝑣} 6: update-score(g,v) 7: 𝒓𝒆𝒕𝒖𝒓𝒏 𝑆 theorem 2: the computational complexity of the algorithm greedy-3(g,k) is 𝑂(𝑛 + 𝑘𝑚 + 𝑘 log (𝑛)), while the space complexity is o(m+n+k). australasian journal of information systems ahmad, traiq, shabbir & khan 2017, vol 21, selected papers from ausdm spectral methods for immunization of large networks 11 proof. by lemma 4, line 2 takes o(m) time. the scores for all vertices can be stored in a maxheap, which can be built in o(n) time while each update-key and extract-max takes 𝑂(log ( 𝑛)) time (cormen 2009). we extract max from the heap k times, hence its total runtime is 𝑂(log ( 𝑛)). as argued by lemma 5, time consumed in each call to update-score takes o(m) time, we get that total time taken by the algorithm is 𝑂(𝑛 + 𝑘𝑚 + 𝑘 𝑙𝑜𝑔 𝑛). for the space complexity, in addition to storing the graph that takes o(n+m) space, we need o(n) space to store the three additional arrays. 5 experimental section we present results of our approximation algorithm in this section. we evaluate the goodness of our algorithm on real world graphs to quantify the scalability and effectiveness on large graphs. throughout this paper we have used susceptible−infected−susceptible (sis) model for virus propagation in graphs. for benchmark comparison we also implemented maxdegree and updated-max-degree. max-degree picks the top k maximum degree vertices for immunization while updated-max-degree selects the vertex with the maximum degree and deletes that vertex and repeats k times. we use the net-shield implementation which is available online1. we implemented our proposed algorithm in matlab and our implementation along with source code and documentation is available online on the given link2. name node count edge count karate 34 78 oregon 10,670 22,002 aa 418,236 2,753,798 table 1 summary of datasets the data sets used in experimentation are described in table 1. all the real graphs used for experimentation obey power law distribution of degrees. the first data set is of a local karate club and is named as karate graph3. nodes of the graph represent members of the club and an edge between nodes show that corresponding members are friends with each other. graph consists of 34 nodes and 78 edges. the graph is undirected and unweighted. the second data set is from oregon as (autonomous system)4 router graphs, which are as level connectivity networks inferred from oregon route views. there are a number of oregon as graphs available and each node represents a router and an edge between two routers represents a direct peering relationship between two routers. we have selected one set from oregon router graphs having 10,670 nodes and 22,002 edges. the graph is undirected and unweighted. nodes selected by greedy algorithm are those routers whose immunization will maximally reduce the spread of virus. the third data set (aa) is from dblp5 dataset. in graph a node represents an author and presence of an edge between two nodes shows that two authors have a co-authorship. in dblp there is total node count of 418,236 and the number of edges among nodes is 2,753,798. we extracted smaller graphs by selecting co-authorship graph of only one journal (e.g displays, international journal of computational intelligence and applications, international journal of internet and enterprise management, etc.). we ran our experiments on 20 different smaller co-authorship networks based on co-authorship graphs of 20 different journals. for the smaller sub graphs that we have extracted from dblp dataset, node count goes up to few 1 https: //www.dropbox.com/s/aaq5ly4mcxhijmg/netshieldplus.tar 2 www.dropbox.com 3 http://konect.uni-koblenz.de/networks/ucidata-zachary 4 http://snap.stanford.edu/data/oregon1.html 5 http://dblp.uni-trier.de/xml/ australasian journal of information systems ahmad, traiq, shabbir & khan 2017, vol 21, selected papers from ausdm spectral methods for immunization of large networks 12 thousands and edge count goes up to few ten thousands. co-authorship graph of actainf contains 1,791 nodes and 1,659 edges, graph of ai communication journal has node count of 1,203 and edge count of 2,204 , sub graph of asia-pacific journal of operational research (apjor) has 1,132 nodes and 1,145 edges, computer in industry journal contains 2,844 nodes and 4,466 edges among nodes, journal of ieee wireless has total of 7,882 authors and 16,557 co-authorship links among authors. detail of sub graphs of dblp data set is also given in table 2. these sub graphs are also undirected and unweighted. name node count edge count actainf 1,791 1,659 ai communication 1,203 2,204 apjor 1,132 1,145 computer in industry 2,844 4,466 computing and informatics 1,598 2,324 ecological informatics 1,990 4,913 ieee wireless 7,882 16,577 ijcia 848 975 ijiem 373 357 table 2 summary of dblp subgraphs we did extensive experimentation with varying count of nodes to be immunized. in the results shown, x-axis shows the value of k which is count of immunized nodes and y-axis shows the percentage of eigendrop which is the achieved benefit after deleting k nodes from graph. results are evaluated on the basis of percentage of eigendrop. eigendrop is difference of largest eigenvalues of original graph and perturbed version of graph after immunization of k nodes. δ𝜆 = 𝜆 − 𝜆(𝑆) (7) where s is the set containing nodes to be immunized having cardinality k. it is clear from the results that our greedy algorithm outperforms netsheild approach and other approaches like top k degree and updated maximum degree approach. our greedy algorithm is scalable for larger values of k as well as for larger graphs as our algorithm has less running time complexity than netsheild, top k degree and updated maximum degree. figure 1 eigendrop of actainf graph australasian journal of information systems ahmad, traiq, shabbir & khan 2017, vol 21, selected papers from ausdm spectral methods for immunization of large networks 13 figure 2 eigendrop of aicommunication graph figure 3 eigendrop of apjor graph figure 4 eigendrop of computer in industry graph australasian journal of information systems ahmad, traiq, shabbir & khan 2017, vol 21, selected papers from ausdm spectral methods for immunization of large networks 14 figure 5 eigendrop of computing and informatics graph figure 6 eigendrop of ecological informatics graph figure 7 eigendrop of ieee wireless graph australasian journal of information systems ahmad, traiq, shabbir & khan 2017, vol 21, selected papers from ausdm spectral methods for immunization of large networks 15 figure 8 eigendrop of ijcia graph figure 9 eigendrop of ijiem graph figure 10 eigendrop of oregon graph 6 conclusion in this work, we explored some links between established graph vulnerability measure and other spectral properties of even powers of adjacency matrix of the graph. we define shield value in terms of trace of the adjacency matrix of the graph. based on these insights we present a greedy algorithm that iteratively selects k nodes such that the impact of each node is maximum in the graph, in the respective iteration, and thus we maximally reduce the spread of a potential infection in the graph by removing those vertices. our algorithm is scalable to large graphs since it has linear running time in the size of the graph. we have conducted australasian journal of information systems ahmad, traiq, shabbir & khan 2017, vol 21, selected papers from ausdm spectral methods for immunization of large networks 16 experiments on different real world communication graphs to confirm the accuracy and efficiency of our algorithm. our algorithm outperforms the state of the art algorithm in performance as well as in quality. the main limitation of this work is that we used only p=4, but it is theoretically clear that using large values of p would yield better immunization performance. for the future work, we will consider the large values of the parameter p, used in the definition of our shield value. references abbas, s., tariq, j., zaman, a., & khan, i. (2017). sampling based efficient algorithm to estimate the spectral radius of large graphs. in 37th ieee international conference on distributed computing systems workshops, icdcs workshops (pp. 175-180). ieee. doi: 10.1109/icdcsw.2017.71 ahmad, m., tariq, j., farhan, m., shabbir, m., & khan, i. (2016). who should receive the vaccine? in 14th australasian data mining conference, australian computer society. bienstock, d., & seymour, p. (1991). monotonicity in graph searching. journal of algorithms, 12(2), 239-245. doi : 10.1016/01966774(91)90003-h bondy, j. a., & murty, u. s. r. (1976). graph theory with applications (vol. 290). london: macmillan. briesemeister, l., lincoln, p., & porras, p. (2003). epidemic profiles and defense of scale-free networks. in proceedings of the 2003 acm workshop on rapid malcode, worm, (pp. 67-75). acm. doi: 10.1145/948187.948200 chakrabarti, d., wang, y., wang, c., leskovec, j., & faloutsos, c. (2008). epidemic thresholds in real networks. acm transactions on information and system security (tissec), 10(4),. doi: 10.1145/1284680.1284681 chen, c., tong, h., prakash, b. a., tsourakakis, c. e., eliassi-rad, t., faloutsos, c., & chau, d. h. (2016). node immunization on large graphs: theory and algorithms. ieee transactions on knowledge and data engineering, 28(1), 113-126. doi: 10.1109/tkde.2015.2465378 chung, f. r. (1997). spectral graph theory (no. 92). american mathematical soc. cormen, t. h., stein, c., rivest, r. l., & leiserson, c. e. (2001). introduction to algorithms (2nd ed.). mcgraw-hill higher education. daadaa, y., jamshed, a., & shabbir, m. (2016). network decontamination with a single agent. graphs and combinatorics, 32(2), 559-581. doi: 10.1007/s00373015-1579-5 erdös, d., ishakian, v., lapets, a., terzi, e., & bestavros, a. (2012). the filter-placement problem and its application to minimizing information multiplicity. proceedings of the vldb endowment, 5(5), 418-429. flocchini, p., huang, m. j., & luccio, f. l. (2007). decontaminating chordal rings and tori using mobile agents. international journal of foundations of computer science, 18(03), 547-563. doi: 10.1142/s0129054107004838 flocchini, p., huang, m. j., & luccio, f. l. (2008). decontamination of hypercubes by mobile agents. networks, 52(3), 167-178. doi: 10.1002/net.20240 fraigniaud, p., & nisse, n. (2008). monotony properties of connected visible graph searching. information and computation, 206(12), 1383-1393. doi: 10.1016/j.ic.2008.09.002 ganesh, a., massoulié, l., & towsley, d. (2005). the effect of network topology on the spread of epidemics. in infocom 2005. 24th annual joint conference of the ieee computer and communications societies. proceedings ieee (vol. 2, pp. 1455-1466). ieee. doi: 10.1109/infcom.2005.1498374 australasian journal of information systems ahmad, traiq, shabbir & khan 2017, vol 21, selected papers from ausdm spectral methods for immunization of large networks 17 kempe, d., kleinberg, j., & tardos, é. (2003). maximizing the spread of influence through a social network. in proceedings of the ninth acm sigkdd international conference on knowledge discovery and data mining (pp. 137-146). acm. doi: 10.1145/956750.956769 kreyszig, e. (1989). introductory functional analysis with applications (vol. 1). new york: wiley. kuhlman, c. j., tuli, g., swarup, s., marathe, m. v., & ravi, s. s. (2013). blocking simple and complex contagion by edge removal. in ieee 13th international conference data mining (icdm), (pp. 399-408). ieee. doi: 10.1109/icdm.2013.47 nemhauser, g. l., wolsey, l. a., & fisher, m. l. (1978). an analysis of approximations for maximizing submodular set functions—i. mathematical programming, 14(1), 265-294. doi: 10.1007/bf01588971 prakash, b. a., vreeken, j., & faloutsos, c. (2012). spotting culprits in epidemics: how many and which ones?. in ieee 12th international conference on data mining (icdm), (pp. 11-20). ieee. doi: 10.1109/icdm.2012.136 seeman, l., & singer, y. (2013). adaptive seeding in social networks. in ieee 54th annual symposium on foundations of computer science (focs), (pp. 459-468). ieee. doi: 10.1109/focs.2013.56 serre, d. (2002). matrices, volume 216 of graduate texts in mathematics. song, c., hsu, w., & lee, m. l. (2015). node immunization over infectious period. in proceedings of the 24th acm international conference on information and knowledge management (pp. 831-840). acm. doi: 10.1145/2806416.2806522 strang, g. (2006). linear algebra and its applications. belmont, ca: thomson, brooks/cole. tariq, j., ahmad, m., khan, i., & shabbir, m. (2017). scalable approximation algorithm for network immunization. in 21st pasific asia conference on information systems, pacis. tong, h., prakash, b. a., eliassi-rad, t., faloutsos, m., & faloutsos, c. (2012). gelling, and melting, large graphs by edge manipulation. in proceedings of the 21st acm international conference on information and knowledge management (pp. 245-254). acm. doi: 10.1145/2396761.2396795 west, d. b. (2001). introduction to graph theory (vol. 2). upper saddle river: prentice hall. zhang, y., & prakash, b. a. (2014a). dava: distributing vaccines over networks under prior information. in proceedings of the 2014 siam international conference on data mining (pp. 46-54). siam. doi: 10.1137/1.9781611973440.6 zhang, y., & prakash, b. a. (2014b). scalable vaccine distribution in large graphs given uncertain data. in proceedings of the 23rd acm international conference on information and knowledge management (pp. 1719-1728). acm. doi: 10.1145/2661829.2662088 copyright: © 2017 ahmad, traiq, shabbir & khan. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems ahmad, traiq, shabbir & khan 2017, vol 21, selected papers from ausdm spectral methods for immunization of large networks 18 the australasian journal of information systems (ajis) began in 1993 and was based on the simple realisation that there was a need for an academic ‘outlet’ for is research in australia australasian journal of information systems volume 18 number 2 2014 45 e-collaboration impacts in australia and hong kong deepinder bajwa western washington university email: deepinder.bajwa@wwu.edu floyd lewis western washington university email: floyd.lewis@wwu.edu graham pervan curtin university email: g.pervan@curtin.edu.au vincent lai chinese university of hong kong email: vslai@cuhk.edu.hk abstract this paper investigates e-collaboration impacts across organizations in australia and hong kong. the two regions were selected because of diversity in geographical dispersion and cultural differences. a myriad of e-collaboration activities were included in the study. data collected and analyzed from 73 organizations in australia and 94 organizations in hong kong suggests that there are no significant differences in the level of electronic support for collaboration activities in both the regions. however, significant differences were detected in the perceived impacts of e-collaboration between the two study regions. interestingly, there was high level of agreement on ranking of ecollaboration activities and perceived e-collaboration impacts in both the regions. implications of our findings for practice and research are discussed. keywords e-collaboration, collaboration technologies, collaboration tasks, collaboration impacts, technology use introduction broadly speaking e-collaboration can be defined as “collaboration among individuals engaged in a common task using electronic (and information) technologies” (kock 2009). practitioners have long recognized the importance of collaboration and recent and future trends suggested that e-collaboration is likely to continue to grow. according to idc, the team collaboration applications market grew almost 15 percent during the first half of 2012 (http://www.techworld.com.au/article/441245/idc_software_market_crm_virtualisation_collaboration _show_strongest_growth/). by some estimates, the global team collaboration and web conferencing market is expected to reach $19.97 billion by the year 2015, a compound annual growth rate of 10.4%. (http://www.researchandmarkets.com/reports/1721862/global_team_collaboration_software_and_aud io). australasian journal of information systems volume 18 number 2 2014 46 researchers have also been investigating e-collaboration (or collaborative information technologies)3 for decades. there have been hundreds of studies collectively, exploring traditional collaboration technologies such as gss (for a review, see arnott and pervan, 2005; fjermestad and hiltz, 20002001), videoconferencing (webster, 1998), and proprietary groupware (for a review, see karsten, 1999). research on virtual team collaboration (for a review, see karoui et al. 2010) point to the large amount of existing attention e-collaboration has received from academic researchers. despite the extant literature on e-collaboration, organizational level deployment of e-collaboration has received little attention and global comparisons are practically non-existent. in this paper, we systematically explore similarities and differences in e-collaboration and its impacts across organizations in australia and hong kong. the two regions were selected not only because of the enormous difference between them in terms of the degree of geographical dispersion of people and organizations, but also because of their differences in cultural values. of the five dimensions of cultural traits identified by hofstede (1980), australia and hong kong are comparable in the masculinity dimension, but differ significantly in individualism, power distance, uncertainty avoidance, and long term orientation traits. research background at the organizational level, studies investigating e-collaboration and its impacts have been scarce and often limited in scope. an early study of lotus notes (a popular proprietary groupware cit) in a large office within a service organization found that use of the technology did not match up to expectations (orlikowski, 1992) and only lead to enhancements in personal productivity. use of notes was limited to emails, file transfers, and accessing news services rather than collaborating by cooperating and sharing information. similar trends were observed by vandenbosch and ginzberg (1996-1997) who found that the introduction of notes to support the professional staff of a large insurance company did not lead to any increase in collaboration over a six-month period. users that had been collaborating prior to introduction of notes continued to maintain their level of collaboration and users that did not collaborate continued to sustain their status. in a more comprehensive review of another 16 case studies investigating notes, karsten (1999) found that use and impacts of cit may not be consistent across organizations. results from a study of another cit (desk-top videoconferencing) parallel those from studies of lotus notes. although 100% of the employees had access to desk-top video conferencing 7 months after it was implemented, there was no notable increase in use levels of the technology after 12 months. in fact, for some users, use levels dropped in the last 5 months (webster, 1998). lou et al. (2006) studied the use and acceptance of notes through three theoretical lens (media richness, social influence, and critical mass) and found only social influence had an impact on groupware use behavior, and that impact was only moderate. more recent efforts exploring organizational impacts of e-collaboration suggest an increase in the firm’s ability to innovate (merono-cerdan et al., 2008; chen et al., 2009; antikainen et al., 2010; troshani et al., 2011), greater productivity (ding et al., 2010), and improved financial performance (chen et al., 2009). further, the increased use of e-collaboration to replace travel has had a positive impact on 3 collaborative information technologies (cits) are essentially it applications that can support collaboration, which by definition implies e-collaboration. australasian journal of information systems volume 18 number 2 2014 47 environmental friendliness and sustainability (molla and abareshi, 2012). in summary, the majority of the past research on organization-level e-collaboration and its impacts has focused primarily on a single collaboration technology in the context of a specific organization. some recent studies have investigated adoption of multiple collaborative technologies across organizations (bayo-moriones and lera-lopez, 2007) and in several global regions (bajwa et. al., 2008). however, there are not large scale investigations exploring e-collaboration activities and its impacts across organizations. recent practitioner research points to the notion that that collaboration technologies (voip, text messaging, screen sharing, video conferencing, and presence awareness tools) are becoming increasing popular globally and organizations that are deploying these ip-enabled advanced technologies are performing better than those that are not investing in them (frost and sullivan, 2009). in light of these practitioner findings, we specifically address the similarities and differences in e-collaboration activities and its impact between organizations in australia and hong kong. theoretical foundations from an information system (is) or information technology (it) use standpoint, the original delone and mclean (1992) model of success has been the premise of many research investigations. according to the authors, the partial model relationships suggest that is/it use leads to impacts (or net benefits) at the individual level and at the organizational level. it use, in general, has been conceptualized at the system level in many research investigations. for example past studies have focused on use of email systems, audio conferencing systems, video conferencing systems, group support systems, enterprise resource planning systems, decision support systems, and expert systems. although such studies have increased our understanding of antecedents of system use, they do not fully capture the extent of deployment of the capabilities/functionalities of the it system under investigation. for example, an email system may have the capabilities to support communication, information dissemination, schedule meetings, organize tasks, amongst others. however, studies investigating email use may not necessarily focus on the deployment of the functionality of email systems to support various e-collaboration activities. to better understand it use, researchers have suggested a “function-centric” approach (jasperson et. al., 2005) that focuses on the capabilities of the it to support activities rather than the system as a whole. our research adopts this premise to investigate e-collaboration. methodology a hybrid approach (case studies and survey design) was selected for the research initiative. we conducted four case studies and then followed up with a survey. the first two case studies were conducted in the us. our objective was to understand more about the implementation of e-collaboration technologies, how they are used, and their possible impacts on organizations. the two case study organizations varied significantly in the size and scope of their geographical operations. telephone interviews were conducted with five managers in a large organization and one manager in a small organization. each participant was sent a detailed document of our research agenda and sample semistructured questions. the average time of each interview was about forty-five minutes and all the interviews were recorded. this was followed by two case studies in australia. a similar interview process was adopted except that the interviews were conducted face-to-face with the senior most it executive in each of the two organizations. average time for each interview was about sixty minutes. the conversations were recorded. in all the cases, we found that organizations typically use a myriad of technologies for a wide range of e-collaboration activities and the more impacts of ecollaboration are perceived as its scope increases from intrato inter-organization. australasian journal of information systems volume 18 number 2 2014 48 the outcomes of the case study provided inputs to our survey design. we adopted the approach suggested by sethi and king (1991) to formulate item measures for the survey. after a review of the literature, we found no validated measures for our study constructs. since most of our constructs had not been explored before in the context e-collaboration, we developed our own measures. six researchers participated in developing these measures. all of them have been involved in collaboration research for nearly two decades. the researchers formulated relevant items from the practitioner and academic literature. discussions were then initiated to determine the clarity of the formulated items and their measurement scales. the resulting survey instrument was then pilot tested in three australian organizations. two of these organizations were the same as our case study organizations. feedback from the pilot study participants was used to make slight amendments to the original instrument. we next focus on the construct measures for our study constructs. measures e-collaboration activities paralleling the “function-centric” approach (jasperson et al., 2005), we developed a measure of ecollaboration that reflected the extent to which key collaboration activities are supported by information technology. we reviewed the relevant literature (easley et al., 2003; hansen, and jarvelin, 2005; nunamaker, et al., 1996-1997; karsten, 1999) to identify collaboration activities in order to formulate item measures. in addition to this, we tapped into our own experience to identify collaboration activities. these experiences were based on past research efforts and e-collaboration activities in our classes. we also evaluated selected collaboration technologies currently available on the market to identify key capabilities and activities that would be supported in e-collaboration. based on these procedures, ten items that comprehensively captured collaboration activities were identified. these included: communications, information and knowledge sharing, decision-making, report writing/information pooling, planning, scheduling, progress monitoring, time management, issue resolution, and discussion and brainstorming. a five point scale (1 = never used, 2, 3 = occasionally used, 4, 5 = always used) was deployed to measure the extent to which it was used to support each of the ten collaboration activities in the organization. e-collaboration impacts a review of past studies and other relevant literature (applegate et al., 2006; ahuja et al., 2009; karsten, 1999; kock, 2009; thomas and bostrom, 2006; vandenbosch and ginzberg, 1996-1997; wasonga, 2007) identified eighteen different impacts. while fourteen of these impacts were desirable outcomes, four of them were negative outcomes of e-collaboration. accordingly, we formulated fourteen items to measure the desired impacts and four items to measure adverse impacts. the desired impacts included: improvements in existing products/services, improved relationships with customers, suppliers, and business partners, improvements in existing business processes, structural changes, time savings, quick reaction to changes, speed of decision-making, increased productivity, appropriate responses to changes, facilitation of innovations, improved quality of decisions, and marketing the right products/services. the adverse impacts included: increased fragmentation of work, increased information overload, decreased management control, and decreased independence. a five point likerttype scale (strongly disagree to strongly agree) was used for all the eighteen items to measure the level of agreement with each of the impacts resulting from e-collaboration in the organization. australasian journal of information systems volume 18 number 2 2014 49 data collection data was collected in the two regions over a one year period. in the first stage, data collection was initiated in australia. an email and a hard copy of the questionnaire was sent to cios of 500 of the top 1000 organizations in australia. the email provided a link to the electronic version of the survey in case the participants preferred the option of responding online. a cover letter was included in the mailed surveys. this letter explained the purpose of our study. clear instructions (including definition of it supported collaboration, voluntary participation, confidentiality of responses etc.) for the participants were incorporated in the cover letter and they were requested to forward the survey to the executive most knowledgeable about it-supported collaboration in their organization, if other than themselves. these instructions were also included on the web survey. a second hard copy mailing was sent out after about a month. a total of 83 responses (web and traditional mail) were received from australian organizations. sixty eight questionnaires were returned unopened possibly because the addressed executive could not be located. ten survey responses were discarded due to substantial missing data. as a result, 73 usable responses were received from 432 questionnaires that reached the addressed executives. this represents a response rate of about 16.9%. we conducted non-response bias tests and found no significant differences in majority of the item measures between the early and late respondents. in hong kong, the questionnaire along with the same cover letter was mailed to senior mis executives of the top 600 organizations. the first mailing resulted in responses from 48 organizations. one month later, the non-responding organizations were sent a reminding letter and another questionnaire, encouraging them to respond to our survey. the second round of mailing resulted in 29 completed responses. two months later, another round of mailing was undertaken following similar procedures as in previous rounds. this resulted in another 17 responses. a total of 94 useable responses were received from organizations in hong kong. thirty two questionnaires were undelivered and returned by the post office, resulting in a response rate of 16.5%. we conducted non-response bias tests between the early and late respondents and found no significant differences in the majority of the item measures. results response profile tables 1 and 2 compare the profile of our respondents (organization level and total number of employees in the organizations) across australia and hong kong. the respondent positions from both the regions were re-coded into 3 tiers. top tier included responses from presidents, c-level executives (cio, cfo, ceo, coo), directors, and vice presidents. middle tier included responses from general managers, senior managers, functional area managers, system and database administrators, while the lower tier included responses from system analysts, it specialists, senior systems engineers, senior software engineers, amongst others. although our survey had been mailed to the cio in all the australian organizations and the senior mis executive in organizations in hong kong, our respondents held a wide range of positions, indicating that the survey had actually been forwarded to the person that (in all probability) was most knowledgeable about it supported collaboration work in their respective organizations. we received responses from small-medium enterprises and from large corporations. a wide range of industries were represented in the responses from both the regions. these included: aviation, manufacturing, financial services, engineering, utilities, education, oil and gas, mining, healthcare, government, legal services, retail, electronics, it, defense, and leisure and entertainment amongst others. this reflects widespread deployment of e-collaboration across large and small organizations and in most (if not all) industries. australasian journal of information systems volume 18 number 2 2014 50 organization level australia [n = 68] hong kong [n = 88] top tier 25 [36.8%] 21 [23.9%] middle tier 39[57.4%] 48[54.5%] lower tier 4 [5.9%] 19 [21.6%] table 1: management level of respondents number of employees australia [n = 67] hong kong [n = 83] less than 100 11 [16.4%] 32 [38.6%] 100 to 999 7 [10.4%] 13 [15.7%] 1000 to 4999 36 [53.7%] 21 [25.3%] more than 5000 13 [19.4%] 17 [20.5%] table 2: organization size (number of employees) of responding firms data analysis e-collaboration activities our review of the literature identified ten collaboration activities. figure 2 shows the extent to which the ten collaboration activities are being supported by its in australia and hong kong. the results indicate that its are used more frequently to support communications, information and knowledge sharing, report writing/information pooling, scheduling, and monitoring progress (in rank order) in both the study regions. on the other hand, discussion and brainstorming is the least supported activity. we conducted t-tests to explore any differences in the means of level of use of its to support collaboration activities between the two regions. the significant differences are also shown in figure 2. the results indicate that its are used more frequently to support decision-making and issue resolution in australian organizations than their counterparts in hong kong. australasian journal of information systems volume 18 number 2 2014 51 **p<0.05 figure 2: e-collaboration activities we computed kendall’s coefficient of concordance (w) to statistically examine the average agreement in rankings of it use to support various collaboration activities between the two regions (see table 3). kendall's w can range from 0 1, where 0 indicates no agreement in rank order, and 1 indicates perfect agreement in rank order. kendall’s coefficient of 0.98 (f = 54, p<0.001) indicates that there is a very high level of agreement between respondents in hong kong and australia in terms of their rank order responses. mean response rank australia hong kong australia hong kong communications 3.35 3.50 1 1 information and knowledge sharing 2.97 2.98 2 2 report writing/information pooling 2.65 2.77 3 3 scheduling 2.57 2.64 4 4 monitoring progress 2.45 2.52 5 5 planning 2.43 2.38 6 6 time management 2.30 2.26 9 7 decision-making 2.43 2.06 7 8 issue resolution 2.41 2.06 8 9 discussion and brainstorming 1.91 1.89 10 10 table 3: rank order of e-collaboration activities 0 1 2 3 4 communications information and knowledge sharing report writinginformation pooling scheduling monitoring progress planning time management **decision-making **issue resolution discussion and brainstorming cit use to support collaboration activities hong kong australia australasian journal of information systems volume 18 number 2 2014 52 e-collaboration impacts figure 3a below shows the level of agreement with the desired impacts resulting from e-collaboration. in general, companies in hong kong had higher level of agreement with these impacts than their counterparts in australia. we conducted t-tests to explore any differences in means of level of agreement with these impacts between the two regions. the significant differences are also shown in figure 3a. the results indicate that the majority of the impacts have significantly higher level of agreement in organizations in hong kong than their australian counterparts. interestingly, respondents in australia had significantly higher level of agreement that use of e-collaboration facilitated innovations than respondents in hong kong. figure 3b shows the level of agreement with the adverse impacts resulting from e-collaboration. once again, companies in hong kong had higher level of agreement with these impacts than their counterparts in australia. we conducted t-tests to explore any differences in means of level of agreement with these impacts between the two regions. the significant differences are also shown in figure 3b. the results indicate significantly higher level of agreement that e-collaboration increased information overload and increased fragmentation of work in organizations in hong kong than in australia. *p<0.10 **p<0.05 figure 3a: desired e-collaboration impacts 1 2 3 4 5 **improved existing products/svcs. resulted in saving time **improved relationships with customers improved our existing business processes **led to quick reaction to changes allowed more work to get done **increased the speed of decisions improved relationships with suppliers **led to more appropriate responses to… **improved relationships with business… *improved quality of decisions **helped to market the right products/svcs. **made our org. structure flatter *facilitated innovations impact means hong kong australia australasian journal of information systems volume 18 number 2 2014 53 *p<0.10 **p<0.05 figure 3b: adverse e-collaboration impacts we computed kendall’s coefficient of concordance (w) to statistically examine the average agreement in rankings of all the impacts of e-collaboration between the two regions (see table 4). kendall’s coefficient of 0.895 (f = 8.5, p< 0.005) indicates a high level of agreement between respondents in hong kong and australia in terms of their rank order of impacts. however, it is interesting to note that there were some large differences in the ranking for a few items: “quick reaction to changes” was ranked much higher in hong kong than in australia while “facilitated innovations” was ranked much higher in australia than in hong kong. 1 2 3 4 5 *increased information overload **increased fragmentation of work decreased independence decreased management control negative impact means hong kong australia australasian journal of information systems volume 18 number 2 2014 54 mean responses rank australia hong kong australia hong kong improved existing products/services 3.73 4.09 3 1 time savings to accomplish tasks 3.93 4.05 2 2 improved relationships with customers 3.68 3.97 5 3 improved existing business processes 3.97 3.82 1 4 quick reaction to changes 3.34 3.81 13 5 more work accomplished 3.69 3.76 4 6 faster decision-making 3.38 3.74 11 7 increased information overload 3.48 3.72 8 8 improved relationships with suppliers 3.56 3.71 6 9 more appropriate changes to responses 3.39 3.66 10 10 improved relationships with business partners 3.35 3.62 12 11 improved the quality of decisions 3.39 3.60 9 12 helped market right products/services 3.24 3.54 14 13 flatter organization structure 3.11 3.42 15 14 increased fragmentation of work 2.90 3.41 16 15 facilitated innovations 3.55 3.32 7 16 decreased independence 2.67 2.86 17 17 decreased management control 2.63 2.73 18 18 table 4: rank order of e-collaboration impacts discussion the objectives of this paper were to explore e-collaboration and its impacts across organizations in australia and hong kong. we next turn to the discussion of our results from analyses of data to address each of the research objectives. e-collaboration activities e-collaboration has several dimensions (i.e. it includes multiple activities) and our results confirm these findings in both the study regions. we also found that there was high agreement on e-collaboration activities in australian and hong kong organizations. since e-collaboration activities vary in the extent to which technology is used to support them, it may be possible to identify “primary” and “secondary” components of collaboration. the top five most frequently it supported collaboration activities across both the study regions included: communications, information and knowledge sharing, report writing/information pooling, scheduling, and monitoring progress. these could well be the primary collaboration activities that are supported by its. other activities that are less supported could well be secondary collaboration activities. australasian journal of information systems volume 18 number 2 2014 55 significant differences were detected in decision-making and issue resolution in e-collaboration between australian organizations and their counterparts in hong kong. this could be attributed to organization size and structure. although most of the responding companies in hong kong were multinational corporations, a significant portion of them were family-owned businesses. these latter types of organizations normally have a more centralized decision structure (and therefore depend less on itsupported decision making) as family members or close relatives hold top management positions, thereby permitting rapid decision making and avoidance of ‘agency’ problems that confront western organizations (weidenbaum 1996). the decision making style in these organizations is more directive and authoritarian, thus requiring less distributed decision collaboration and issue resolution. for the australian organizations, many decisions and issues are at the operational level and need to be resolved by people who are far apart geographically. they are thus more reliant on its for collaboration rather than face-to-face contact. more frequent use of its to support decision-making and issue resolution in australian organizations may also be reflective of the wide geographical dispersion of employees across states (sydney and melbourne are both over 4,000 km from perth, for example). this dispersion is even more significant for australian mining companies (which are well represented in the australian sample) where, even when in the same state, headquarters and operations may be over 1,500 km apart, thus necessitating the use of its to make decisions and resolve issues. e-collaboration impacts e-collaboration can lead to organization-wide impacts and our findings detected many of these impacts in both the study regions. once again, we also found significantly high agreement on ranking of impacts in australian and hong kong organizations. the five highest ranking impacts where agreement levels were also high included: improved existing products/services, time savings to accomplish tasks, improved relationships with customers, improved existing business processes, and more work accomplished as a result of e-collaboration. thus respondents in both the study regions perceive that ecollaboration can result in strategic and operational level effectiveness and efficiency-oriented impacts. the five lowest ranking impacts where agreement levels were also low included: helped to market right products/services, flatter organization structure, increased fragmentation of work, decreased independence, and decreased management control. three of these five lowest ranking impacts were “adverse impacts”, thereby suggesting that the desired e-collaboration impacts far outweigh the undesired consequences with the exception of “information overload,” which was ranked eighth in both the study regions. this is an issue of concern as information overload due to mass interaction is likely to end active participation by users (jones et al., 2004). this may be especially critical when organizations adopt technologies to support online communities for their collaboration activities. we also found significant differences in ten of the eighteen impacts between organizations in australia and hong kong. in almost all the cases, organizations in hong kong had significantly higher level of agreement towards desired and adverse impacts than their australian counterparts. although there are very few significant differences in the e-collaboration activities in organizations between the two regions (significant differences were detected only for decision making and issue resolution), other forces may be in play to explain the differences in impacts. one possible explanation is that end-users in hong kong have a significantly greater level of influence in implementing e-collaboration than in australia. this offers more opportunities for users to experiment with the usefulness of it tools in enhancing their collaborative activities. some its are complex and multi-faceted, making it difficult for users to comprehend their impacts when they are deployed. therefore, “learning by using” allows users to develop an understanding of the strengths and weaknesses of the technology. as a consequence, e-collaboration in hong kong has developed a steeper learning curve, thus resulting in their creation of new knowledge (alder and clark 1991; faraj et al., 2011) to use its to collaborate more effectively and efficiency. in the end, these users may also develop a stronger feeling of how collaborative tools could impact their jobs and organizations, in both the positive and negative ways. australasian journal of information systems volume 18 number 2 2014 56 the causes for higher agreement in the majority of the impacts could also be attributed to other factors including, the reach of e-collaboration in hong kong organizations and the time lag of collecting data in in hong kong. we also found significantly higher level of agreement that e-collaboration in australian organizations “facilitated innovations” than in their counterparts in hong kong. this may be reflective of a greater mix of organizational sizes in hong kong. there were a greater proportion of small organizations represented in the hong kong sample (including family owned businesses). the centralized decision structure of these small family owned organizations may have limited innovative behavior and any related impacts. as for the australian sample, higher impact levels for facilitating innovations may be due to the influence of mining companies which are known to be highly innovative users of all its. nevertheless, general trends indicate that organizations in both regions have realized significant impacts from e-collaboration. this suggests that, for the most part, organizations have the necessary elements (culture of openness and information sharing, structural elements, governance systems etc.) aligned well to facilitate e-collaboration. this may be more prominent in hong kong (than in australia) as respondents indicated higher agreement levels on the majority of the impacts. conclusions in this paper, we have explored e-collaboration activities and its impacts across organizations in australia and hong kong. given the exploratory nature of the research, our objective was not to propose and test relationships between our study constructs. instead, it was our intent to extend and expand the existing literature on e-collaboration by comparing trends in under-investigated global regions. the two study regions selected vary significantly along cultural and geographical dimensions. despite this diversity, we found many common trends that have important implications for practice and research. from the practitioner standpoint, our findings provide benchmarks to assess e-collaboration activities and the organizational level impacts resulting from e-collaboration. organizations use its to support many collaboration activities. benchmarks can assist practitioners in their regions to identify ecollaboration opportunities in their respective organizations. an assessment of e-collaboration impacts may also encourage practitioners to justify investments in order to become more competitive. from a research standpoint, this study contributes to our understanding about e-collaboration activities and its impacts across organizations in two global regions. most e-collaboration studies have focused on a single technology in a specific organization. our study focused on several activities that could be supported by many different technology solutions. such a functional-centric approach provides more in-depth understanding of e-collaboration as it is independent of the technology platforms used to support the phenomenon. the high agreement in rank order of e-collaboration activities in both the study regions supports the notion that collaboration can be viewed as a set of activities that are consistent across national boundaries. future research could further explore “primary” and “secondary” e-collaboration activities and their impacts in other global regions. a more detailed assessment of the effects of e-collaboration on impacts can also be made by conducting additional longitudinal investigations. this would help explain the directional or differential effect between e-collaboration and its impacts over time. while our study has extended and expanded the existing literature on e-collaboration, we must also recognize the limitations of our research. first, a single respondent in each organization was used to collect data on organizational level constructs. although care was taken to reach the person most knowledgeable about the study objectives, single respondent perceptions about organizational level constructs can be problematic even though they have been used extensively in many it studies. australasian journal of information systems volume 18 number 2 2014 57 however, 85% of our respondents were top and middle tier executives. given that these informants are likely to have access to information about organizational level phenomenon in most modern organizations adds credibility to the responses. second, data collection in the two study regions took place over a year and there could be “lag” effects of e-collaboration deployment and its impacts. despite these limitations, our research makes significant contributions to the existing literature. references ahuja, v., yang, j., & shankar, r. (2009). benefits of collaborative ict adoption for building project management. construction innovation, 9(3), 323-340. alder, p.s., & clark, k.b. (1991). behind the learning curve: a sketch of the learning process. management science, 37(3), 267-281. antikainen, m., makipaa, m., & ahonen, m. (2010). motivating and supporting collaboration in open innovation. european journal of innovation management, 13(1), 100-119. applegate, l.m., austin, r., & mcfarlan, f.w. (2006). corporate information strategy and management: text and cases. boston, ma: irwin/mcgraw-hill. arnott, d., & pervan, g. (2005). a critical analysis of decision support systems research. journal of information technology, 25(2), 67–87. asproth, v., & nystrom, c.a. (2008). communication and information exchange among smes and their local environment. journal of organisational transformation and social change, 5(2), 175-189. bajwa, d.s., lewis, l.f., pervan, g., lai, v.s., munkvold, b.e., & schabe, g. (2008). factors in the global assimilation of collaborative information technologiies: an exploratory investigation in five regions. journal of management information systems, 25(1), 131-165. bayo-moriones, a., & lera-lopez, f. (2007). a firm-level analysis of determinants of ict adoption in spain. technovation, 27(6-7), 352-366. camarinha-matos, l.m., afsarmanesh, h., galeano, n., & molina, a. (2009). collaborative networked organizations concepts and practice in manufacturing enterprises. computers & industrial engineering, 57, 46-60. chen, j.s., tsou, h.t., & huang, a.y.h. (2009). service delivery innovation: antecedents and impact on firm performance. journal of service research, 12(1), 36-55. delone, w.h., & mclean, e.r. (1992). information systems success: the quest for the dependent variable. information systems research, 3(1), 60-95. ding, w.w., levin, s.g., stephan, p.e., & winkler, a.e. (2010). the impact of information technology on academic scientists’ productivity and collaboration patterns. management science, 56(9), 1439-1461. easley, r.f., devaraj, s., & crant, j.m. (2003). relating collaborative technology use to teamwork quality and performance: an empirical analysis. journal of management information systems, 19(4), 247-268. faraj, s., jarvenpaa, s.l., & majchrzak, a. (2011). knowledge collaboration in online communities. organization science, 22(5), 1224-1239. fjermestad, j., & hiltz, s.r. (2000-2001). group support systems: a descriptive evaluation of case and field studies. journal of management information systems, 17(3), 115-159. hansen, p., & jarvelin, k. (2005). collaborative information retrieval in an information-intensive domain. information processing & management, 41(5), 1101-1119. australasian journal of information systems volume 18 number 2 2014 58 hofstede, g. (1980). culture's consequences: international differences in work-related values. newbury park, ca: sage. jasperson, j., carter, p.e., & zmud, r.w. (2005). a comprehensive conceptualization of post-adoptive behaviors associated with information technology enabled work systems. mis quarterly, 29(3), 525-557. jones, q., ravid, g., & rafaeli, s. (2004). information overload and the message dynamics of online interaction spaces: a theoretical model and empirical exploration. information systems research, 15(2), 194-210. karoui, m., gurkan, a., & duderzet, a. (2010). virtual team collaboration: a review of literature and perspectives. proceedings of the sixteenth americas conference on information systems, lima, peru, august 12-15, 2010. karsten, h. (1999). collaboration and collaborative information technologies: a review of the evidence. database for advances in information systems, 30(2), 44-65. kock, n. (2009). the evolution of costly traits through selection and the importance of oral speech in e-collaboration. electronic markets, 19, 221-232. lou, h., scamell, r., & shah, j.r. (2006). use of a groupware product: a test of three theoretical perspectives. journal of computer information systems, 46(4), 35-45. meroño-cerdán, a.l., soto-acosta, p., & lópez-nicolás, c. (2008). how do collaborative technologies affect innovation in smes? international journal of e-collaboration, 4(4), 33-50. molla, a., & abarsehi, a. (2012). organizational green motivations for information technology: empirical study. journal of computer information systems, 52(3), 92-102. ngwenyama, o.k. (1998). groupware, social action and organizational emergence: on the process dynamics of computer mediated distributed work. accounting, management and information technologies, 8(2-3), 127-146. nunamaker jr, j.f., briggs, r.o., mittleman, d.d., vogel, d.r., & balthazard, p. (1996-1997). lessons from a dozen years of group support systems research: a discussion of lab and field findings. journal of management information systems, 13(3), 163-207. orlikowski, w.j. (1992). the duality of technology: rethinking the concept of technology in organizations. organization science, 3, 298-327. sethi, v., & king, w.r. (1991). construct measurement in information systems research: an illustration in strategic systems. decision sciences, 22(3), 455-463. thomas, d., & bostrom, r. (2006). building trust and cooperation through technology adaptation in virtual teams: empirical field evidence. information systems management, 25, 45-56. troshani, i., rampersad, g., & plewa, c. (2011). adopting innovation management software in university innovation commercialization. journal of computer information systems, 52(2), 8392. vandenbosch, b., & ginzberg, m.j. (1996-97). lotus notes and collaboration: plus ça change. journal of management information systems, 13(3), 65-81. wasonga, t.a. (2007). using technology to enhance collaborative learning. international journal of educational management, 21(7), 585-592. webster, j. (1998). desktop videoconferencing: experiences of complete users, wary users, and nonusers. mis quarterly, 22(3), 257-286. weidenbaum, m. (1996). the chinese family business enterprise. california management review, 38(4), 141-156. paper title (paper title style) australasian journal of information systems thomas, burmeister & low 2019, vol 23, research on applied ethics (cybersecurity) ethical conduct by penetration testers 1 the importance of ethical conduct by penetration testers in the age of breach disclosure laws. georg thomas charles sturt university gethomas@csu.edu.au oliver burmeister charles sturt university gregory low sql down under abstract across the globe, there has been a noticeable increase in the adoption of breach disclosure laws that are designed to protect the privacy of individuals. to validate the security controls implemented by an organisation to protect sensitive data, penetration testers are often engaged to test the security of information systems and to report any vulnerabilities. using an interpretivist, constructivist approach, this article reports on a pilot study that compares usa and australian approaches to ethical hacking. the need for regulation of ethical hacking to help protect organisations from unethical conduct was a recurring theme. with the changes in privacy regulations across the world, unauthorised disclosure of personal and privileged information could result in significant consequences. this paper explores the importance of ethical conduct by penetration testers based on empirical research and the potential for misuse of information. keywords: penetration testing, ethics, privacy legislation, data misuse, privacy. 1 the rise of the breach disclosure law although privacy laws can be dated back to the 15th century and the breach of confidentiality tort to the early twentieth century (solove, 2006, p17), the last decade has seen an emphasis on privacy relating information systems in security and other areas, as it relates to electronic data (solomon, 2014; weidenmier and ramamoorti, 2006; thomas, duessel and meier, 2017; thomas, burmeister and low, 2017; bowern, burmeister, gotterbarn and weckert, 2006; burmeister, islam, dayhew and crichton, 2015; livari and livari, 2011). privacy is difficult to define as what one person considers privacy, may not be reflected by another. at a high-level privacy could be defined as the right for an individual to keep their information private. li et al. (2013) saw privacy as a social concept and privacy breaches in terms of societal perceptions. personal information is defined as “information or an opinion about an identified individual, or an individual who is reasonably identifiable” (office of the australian information commissioner, 2018). this includes information such as credit information, health information, tax information, and other sensitive information. closely associated with the concept of privacy is that of confidentiality, and thus with breaches of confidentiality (bernoth, dietsch, burmeister and schwartz, 2014). bernoth et al. (2014, p 454) claimed that “in discussing breaches of confidentiality it is important to remember that there are justifiably instances when breaches ought to occur and, therefore, not all breaches australasian journal of information systems thomas, burmeister & low 2019, vol 23, research on applied ethics (cybersecurity) ethical conduct by penetration testers 2 are unethical.” similarly, as seen in the discussion section of this article, there are legitimate times when an ethical hacker breaches privately held, confidential information. the latest iterations of privacy laws across the globe generally impose specific notification requirements and often significant financial penalties for non-compliance in relation to the unauthorised disclosure of personal information. commonly referred to as breach notification, in the event that an organisation believes that personal information they hold is disclosed to an authorised party, there may be notifications requirements depending on the laws of the jurisdiction (muntermann & roßnagel, 2009). in 2017, verizon (2018) confirmed over 2,216 data breaches, and the identity theft resource center (2017) reported that over 174,402,528 records were exposed. the number of breaches reported by verizon is up 14.5% from 2016 and 2,462.2% from 2008 and these are only those that were reported or discovered. while it is likely that the sophistication of cyber incident detection systems and a general increase in awareness may account for much of the increase compared to 10 years ago, the disclosure of over 174 million records in 2017 itself is alarming and it is unsurprising that governments across the world have started to address the issue of data security through legislation. 2 ethical hacking penetration testing, also known as ethical hacking, is an offensive manoeuvre carried out by a security professional with the aim of gaining access to an organisations’ systems or data to validate the effectiveness of the security controls that have been implemented. traditionally, a defensive approach through the use of multilayered technical controls such as firewalls, network segmentation, and antivirus software was used to protect organisations (thomas, burmeister, low, 2017). this purely technical approach then evolved to address people-based aspects otherwise known as the “human factor”, largely driven by a significant increase in email-based attacks or ‘phishing attacks’ (eminağaoğlu, uçar & eren 2009, p223). the term ‘hacker’ originated at mit in the 1960’s to describe someone who had the ability to understand and manipulate technology. although this is still true of hackers, their skills have evolved outside of just technical capabilities to include the ability to manipulate people. additionally, hackers are now categorised into three distinct categories that identify their motives. 2.1 black hat hackers a black hat hacker also known as a ‘cracker’ (graves, 2010, p3) is a hacker with malicious intent. the intent of a black hat hacker is usually financially motivated, however, there are other motives also, such as causing disruption to systems or committing espionage. black hat hackers operate illegally and can be individuals, teams, part of an organised crime syndicate, or even terrorist groups. according to verizon (2018), in 2017, 50% of breaches were attributed to organised crime and 76% of breaches were financially motivated. 2.2 white hat hackers a white hat hacker also known as an ethical hacker is a hacker who uses their skills for the purpose of protecting organisations against malicious hackers by identifying vulnerabilities in an organisation’s security controls. through a formal engagement, a white hat hacker uses the same tools and techniques as a black hat hacker to test the security of the target organisation (graves, 2010, p3). white hat hackers are generally bound by contractual obligations such as australasian journal of information systems thomas, burmeister & low 2019, vol 23, research on applied ethics (cybersecurity) ethical conduct by penetration testers 3 non-disclosure agreements and specific rules of engagement that provide authorisation, which allows them to conduct tests that would otherwise be illegal (thomas, low and burmeister, 2018). 2.3 grey hat hackers between the black and white hat hackers is the grey hat hacker. grey hat hackers are still considered to operate illegally, but their motives aren’t generally considered malicious. unlike a black hat, a grey hat may provide more ‘white hat’ services, such as identifying weaknesses in systems that they then highlight to the organisation with the weakness. this is conducted without authorisation, however. nation state hackers are also considered grey hats as what they do can also be considered questionable. on the one hand, nation state hackers act in the best interests of the country they are representing as part of cyber warfare and helping to keep the nation they represent safe through intelligence gathering. on the other hand, the targeted nation would consider the hack to be unauthorised and illegal (burmeister, phahlamohlaka and al-saggaf, 2015). an interesting additional grey hat category is the hacktivist. where a hacker is motivated by the promotion or defence of a cause, rather than for personal or financial gain, this is known as hacktivism (hargrave, 2012). the term hacktivism is a portmanteau of hack and activism. mikhaylova (2014) describes hacktivism as an emerging form of political and social participation. common examples of hacktivism include the defacement of websites or the leaking of confidential (and often classified) information to the public or media. two wellknown hacktivist groups are anonymous (carter, 2012) and wikileaks (thomas, low, burmeister, 2018; beyer, 2014; ludlow, 2010). 3 the breach disclosure law landscape today although laws around privacy and the security of personal information have existed for some time, there has been a noticeable increase in laws and regulations that specifically require the disclosure of a data breach. previously, it was largely up to the discretion of an organisation whether they disclosed a breach and often times, unless a breach made headlines, it would likely go unreported. in many cases, breaches were not even identified by the organisations who were the victims of the breach. in 2016 the average time it took for a breach to be detected was 99 days, down from 146 days in 2015 (mandiant, 2017). this alarming data means that on average, an attacker has almost four months to exfiltrate data from an organisation before they are detected. although it varies from organisation to organisation, current internet speeds would allow very substantial volumes of data to be leaked. such data, if personally identifiable and depending on the type, could be sold on the dark web. the dark web is part of the deep web (not searchable by stand search engines such as google) that has been hidden and requires special browsers to access. it is commonly used to sell illegal items and personal information that is sold could be used to enable identify theft, and from that, credit card fraud (chertoff, simon, 2015). it is of no surprise that governments and lawmakers have now started to address the issue to ensure that those who are affected by a breach are notified in a timely manner. although it can be argued that once your personal information is disclosed it can be difficult, if not impossible, to contain it, having the awareness that one’s personal information was disclosed would allow individuals to take steps to help to protect themselves from fraudulent activities. many countries including australia, canada, and the philippines have already, or are in the process australasian journal of information systems thomas, burmeister & low 2019, vol 23, research on applied ethics (cybersecurity) ethical conduct by penetration testers 4 of, implementing data breach disclosure laws. not all countries and jurisdictions have implemented such laws, but the number is increasing. 3.1 australia the notifiable data breaches scheme came into effect on february 22, 2018. this amendment to the privacy act 1988 requires that eligible organisations are required to notify affected individuals who may suffer serious harm and the australian privacy commissioner in the event of a breach of personal information (office of the australian information commissioner, 2018). the office of the australian information commissioner (oaic) provides some guidance on securing personal information which includes recommending testing of the security controls in place. since the scheme came into effect, there has been a steady increase in reported breaches each quarter, with a total of 812 breaches reported as at 31 december, 2018. most of the reported incidents were due to malicious or criminal attack (office of the australian information commission, 2018). 3.2 canada canada’s data breach disclosure requirement came into effect on november 1, 2018 (ling, 2018). these requirements are an amendment to the digital privacy act (2015) and require that affected individuals and the canadian office of the privacy commissioner (opc) are notified in the event of breaches of personal information that may result in a “real risk of significant harm” (government of canada, 2018). 3.3 europe possibly the most stringent regulations to date are those of the european union (eu). the eu general data protection regulation (gdpr) came into effect on may 25, 2018 and superseded the existing directive 95/46/ec that has been in place since 1995 (thomas, 2018). gdpr has a number of noteworthy requirements that include investigation and reporting of a suspected breach within 72 hours, the right to be forgotten, the right to access data held by an entity, the right to data portability, and the right to have errors in the data rectified (european commission, n.d.). 3.4 philippines the data privacy act of 2012 in the philippines requires that the national privacy commission and affected data subjects be notified in the event that personal sensitive information or other information that may be used to commit identify fraud or real risk of serious harm is acquired by an unauthorised person (wall, 2017). 3.5 the united states at the time of writing the united states does not have a uniformed data breach disclosure law, however most states have now enacted data breach disclosure laws (romanosky, telang, acquisiti, 2011). states such as new york have even gone as far as making specific security requirements law, however these requirements, such as those in the nydfs cybersecurity regulation (23 nycrr 500) only apply to financial services and insurance (new york state department of financial services, 2017). australasian journal of information systems thomas, burmeister & low 2019, vol 23, research on applied ethics (cybersecurity) ethical conduct by penetration testers 5 3.6 specific regulations in addition to country or state specific legislation there are a number of other specific regulations that must be considered. these regulations, although not as broad, generally apply to industries that are more susceptible to breaches of privacy. the health insurance portability and accountability act of 1996 or hipaa in the united states has a specific breach notification rule, which requires covered entities to notify affected individuals and the us department of health and human services in the event of a breach of personal information (u.s. department of health & human services, 2013). in australia, the australian prudential regulation authority (apra) has proposed a new standard that requires entities which they regulate to notify apra of material information security incidents (australian prudential regulation authority, 2018). 3.7 other laws in addition to specific breach disclosure laws, there are relevant laws that should be considered, such as those that relate to computer crimes. section 10.7 of the australian criminal code act of 1995 (cth) for example, covers computer offences. the offences include the unauthorised access, modification, or impairment of data. likewise, the european union has a directive for attacks against information systems. the directive 2013/40/eu requires measures to help prevent attacks and identity theft as well as sufficient penalties for those that carry out attacks against information systems. 4 ethical hacking and breach disclosure laws many of these new laws and regulations either recommend or require that appropriate compliance and assurance is considered. ethical hacking is seen as one method for the prevention of unauthorised disclosure of electronic records by identifying vulnerabilities within systems (myers, frieden, bherwani, henning, 2008). this is highlighted in a few examples such as the office of the australian information commissioner that provides guidance on securing personal information. this guidance includes regular testing of systems including considering what type of information is used for testing and, if it is personal information, managing the associated risks (office of the australian information commissioner, 2015). apra’s proposed standard includes the requirement for systematic testing and assurance to test the effectiveness of security controls (australian prudential regulation authority, 2018) as does the nydfs cybersecurity regulation (new york state department of financial services, 2017). although not specifically required by hipaa, the national institute of standards and technology in the united states recommend penetration testing (ethical hacking) be conducted as part of their guidance for implementing the hipaa security rule (scholl et al., 2008). 5 method an interpretivist, constructivist research approach was undertaken. the use of qualitative research is aimed at exploring and understanding the human problem (creswell, 2009, p4). due to the nature of the research, this approach was the most suitable as the research aims to first identify and confirm the issue and then gain insight into it. through an interview process, the research aimed to gain an understanding of the ethical issues that surround ethical hacking. purposive sampling involving 28 participants from law firms in australia and the united states of america, and ethical hackers from consulting firms. australasian journal of information systems thomas, burmeister & low 2019, vol 23, research on applied ethics (cybersecurity) ethical conduct by penetration testers 6 after an informed consent process, each participant was asked a series of interview questions. the questions chosen were targeted and based on the type of participant. all participants were asked whether they could identify any ethical issues and concerns with penetration testing in general and in the context of their roles. law firm participants were asked whether they were aware if their respective firms had undertaken penetration testing previously and if so, had there had been a due diligence process associated with those engagements. ethical hacker participants were asked whether they had gained access to confidential information and what was the course of action by which they had obtained such data. interview audio was recorded and transcribed for analysis and a process of thematic coding the data was then undertaken. this process of coding involved naming and labelling segments of data to categorize, summarize and account for them (charmaz, 2006). the resultant data was analysed for common themes and interpreted along with the triangulation between identified themes and categories, and the literature. the focus of this article is data obtained which relates to data breach legislation. 6 discussion of results current literature is largely focused on academic research. jamil and khan (2011) discuss issues of teaching students to hack without knowing their true intentions. the article further discusses how the outcome of these teachings could result in both bad and good behaviours of students (jamil, khan, 2011). this literature also focuses on those that are being taught to hack through academic institutions and doesn’t consider existing professionals who have changed into the ethical hacking field. radziwill, romano, shorter, and benton (2015) discuss how teaching students to hack in school, although it may start out innocently, could result in students stumbling into black hat hacking and facing legal consequences. there is also the possibility that such unethical behaviour by students is encouraged, but this hasn’t been explored and could be examined in further research. the legal implications of ethical hacking need to be considered. because ethical hackers are studying criminal activity, some of the activities the ethical hacker undertakes cannot be distinguished from the crimes themselves (brodkin, 2009). 6.1 confidentiality the thematic analysis revealed confidentiality of information as a major theme related to data breaches. 67% of the research participants and 90% of legal professionals highlighted the importance of confidentiality. this concern is supported by the requirement for legal privilege by legal professionals. “[law firms have a] duty of confidentiality to our clients. this is managed in some ways through reciprocal confidentiality agreements that are imposed upon pen-testers, but i think there is always an inherent uneasiness about the potential that a pen-tester may access some client’s information.” – partner, australia “clients need to have absolute faith when they engage a lawyer that the flow of their information remains private and is protected.” – lawyer, australia “we need to ensure that client confidences are maintained because that’s one of the real primary reasons why clients come to a law firm. they have an expectation of absolute confidence in relation to their confidential information” – lawyer, australia australasian journal of information systems thomas, burmeister & low 2019, vol 23, research on applied ethics (cybersecurity) ethical conduct by penetration testers 7 “in the us, each state has their own ethics rules but as far as i am aware all states have rules regarding confidentiality of information to preserve the attorney client privilege and that’s fiercely guarded. the idea is it’s a very protected privilege because the idea is that you want your clients to be forthcoming with you; and we want to instil trust and confidence in our clients and our relationships so it’s an ethical obligation to protect that attorney client privilege.” lawyer, united states this requirement for legal privilege is the same in australia and the united states, with the same requirements from the australian solicitors conduct rules and the american bar association. taking evidence is something that was also a common occurrence by penetration testers. this practice is to provide proof that a vulnerability exists and highlight the type of information or system that was accessed. it is not uncommon for such evidence to be included in the final report. the question of how this information, especially if sensitive, was protected by the testing firm was raised. one respondent stated: “if access is gained, how is that information protected and what sort of systems do they [ethical hackers] have in place to make sure that information doesn’t go any further?” – law firm security, australia “if there was to be some sort of compromise [of the ethical hacking provider], potentially sensitive information could be exposed or at worst case modified which in terms of wills and stuff like that could be very disastrous for the parties involved” – lawyer, australia “the last thing you want is just to have a rogue cowboy pen-testing from home, and they lose their laptop on the bus on the way to work and all of your data is sitting on their laptop and it gets lost.” law firm security, australia when considering what the data breach disclosure laws are designed to do; help reduce the unauthorised disclosure of personal information; it is important to consider not only what personal information an organisation holds but who has access to it. data breach disclosure laws largely rely on organisations performing some level of duty of care. the consequences of a breach, in jurisdictions that have breach disclosure laws would likely result in public disclosure, and could result in significant impact to the breached organisation. law firms for example, have a duty to ensure the confidentiality of their clients’ data and unauthorised access of that information could result in serious reputational and financial harm. “rules around privilege or loss of privilege if disclosed would be disastrous” (lawyer, australia) “with the new laws around disclosing [a breach], everyone is most concerned not necessarily in the actual loss of the data or the breach itself, but in having to come out and admit that that breach has then happened.” – lawyer, australia “from a legal point of view, one thing that always comes up is the data that lawyers manage and control, depending on the law firm, and who your customers are, but [the data] could be subject to privacy legislation such as the gdpr” security consultant, united states. there may also be additional implications, such as compliance with ‘the right to be forgotten’ requirement of gdpr for information that may have been taken as evidence or inadvertent australasian journal of information systems thomas, burmeister & low 2019, vol 23, research on applied ethics (cybersecurity) ethical conduct by penetration testers 8 disclosure into the public domain. it is important to consider these regulatory issues and laws that various countries have regarding the safeguarding of privacy (thomas, duessel, meier, 2017). 6.2 regulation another theme to emerge from the data was that of regulation. ethical hackers should be regulated or licensed, as a form of professionalism and this too relates to the issue of data breaches. “by chartering and coming together as a regulated professional organisation, you can explore whether things like insurance, standards, and discipline are mandatory. effectively like doctors and lawyers, which is to say that you are not entitled to practice without this certification.” (partner, australia) “i think the credentialization of the profession if you want to call it that is, is a good and worthy objective. but i think there’s a differentiation between the professions that you mentioned as examples [lawyers, doctors, and accountants] and that specific advances that are made in technology absolutely outstrip any of the other areas.” – lawyer, united states. “like any sort of profession, i think it’s important to have some sort of regulation, it gives outsiders the view that it’s more credible pursuit and also some comfort. we talk about background checks or whatever it might be, [regulation ensures this] is being done and to make sure it’s competent testing and there’s no risks brought by the testers themselves.” lawyer, australia not only is ict a relatively new profession, but within ict, ethical hacking is rarely identified as one of the job classifications for an ict professional (weidenmeir and ramamoorti, 2006; burmeister, 2015; 2017). there is currently neither a mandatory or unified code of ethics that exists within ict (capurro and britz, 2010; burmeister, 2013; whitehouse et al., 2016) or that applies to ethical hackers. while there are codes of conduct or ethics available, these are considered voluntary and only mandatory for those who are members or certified by various certification bodies such as isc2, isaca, or ec-council to name a few (thomas, burmeister, low, 2018). many of the interview participants raised the issue of regulation or licensing, but also questioned the effectiveness of such regulation and licensing citing the rate that technology advances, compliance issues, and costs as key issues. risk management is at the core of a successful security program (stoneburner, goguen, & feringa, 2002) and the engagement of an ethical hacker assists in identifying risk areas that need to be addressed. “it was important from our point of view that the individuals we engaged to do the testing don’t add to that risk.” – law firm ciso, australia “when we engage in work there are a lot of requirements such as background checks, financial checks, reputation checks, etc.”partner, australia “they’re [the ethical hacking provider] a recognised and trusted body that you’re working with and that there is a set standard contractual agreement put down in terms of non-disclosure. you always look for those kinds of points when you’re actually dealing with the providers that you’re seeking to do the testing with.” lawyer, united states australasian journal of information systems thomas, burmeister & low 2019, vol 23, research on applied ethics (cybersecurity) ethical conduct by penetration testers 9 it is clear that there are several regulations and laws that are aimed at preventing or at the very least requiring disclosure of a breach. it is key that any ethical hacker is aware of the regulations and laws that apply to them when conducting an engagement. the united states provides a good example of how requirements can vary. each state can potentially have their own laws and requirements, and from a lawyer’s perspective, they would expect an ethical hacker who could be conducting an engagement across different jurisdictions to be compliant. “an ethical hacking company would want to be certain that they are in compliance with those ethics rules and conducting the hacking in the way that would be in compliance with every state’s ethical rules.” – lawyer, united states in the event that an ethical hacker acts unethically, the increase in risk to an organisation could be significant depending on the realised consequences. while it is fairly clear that criminal and malicious attacks would be subject to the computer offences section of the australian criminal code act, the applicability of criminal consequences should also be considered when the act of an ethical hacker meets the criteria of a serious offence, regardless of whether they were provided authorisation to conduct a test or if they contravene a law from another jurisdiction even unknowingly. the unprofessionalism of inadequate testing could result in a threat exploiting a vulnerability, which may result in unauthorised disclosure of information or disruption. inappropriate information handling or access by the ethical hacker themselves, could also result in unauthorised disclosure and even the misuse of information. “you’re giving access to a person who is an outside third party, so do you have an obligation to notify clients? (as part of legal professional ethics requirements)”partner, united states “if there was to be some sort of compromise, potentially sensitive information could be exposed or at worst case modified which in terms of wills and stuff like that could be very disastrous for the parties involved”lawyer, australia because in many instances an ethical hacker is provided with instructions to gain access to any data they can, or they may inadvertently come across it in the course of performing their engagement, it is possible that information that may have contractual obligations protecting it may be accessed. in these instances, there is a requirement to notify the client and possible litigation against the firm may result. 7 conclusion the increase in breach disclosure laws around the world has heightened the need for organisations to further protect their sensitive information. this need, and the increased hostility on the internet in general, is leading to a large demand for cyber security professionals. in the usa alone, nist (2018) suggest an increase of over 300,000 new professionals, above an existing base of over 700,000 professionals. the relative inexperience of the cyber security professional population is itself a cause for concern, both in terms of maintaining professional and ethical standards, and in terms of achieving effective outcomes, and being able to monitor the quality of those outcomes. a significant percentage of this large group of cyber security professionals will at some point be involved in ethical hacking activities, yet to date there has been little research on the importance of ethical conduct by these ethical hackers. australasian journal of information systems thomas, burmeister & low 2019, vol 23, research on applied ethics (cybersecurity) ethical conduct by penetration testers 10 organisations need to consider the risks and how they might address those risks. if an ethical hacker gains access to personal information at an organisation they are testing, controls to secure and handle such information appropriately must be considered, to avoid inadvertent disclosure that could affect not only the target firm, but the privacy of the individuals that the data relates to. this paper identified the need to consider a regulatory approach to ethical hacking that includes requirements such as a uniformed code of ethics and licensing requirements. there may be some difficulties with this approach however, especially when it comes to cross jurisdiction testing. like in law, accounting, or medical licensing, it may be necessary to look at bridging or multi-jurisdiction licensing options to ensure compliance with laws of specific countries. as most organisations that employ ethical hackers are not able to assess their effectiveness, there is also a need to be able to assess and monitor the performance of ethical hackers. it is likely that there is a need for organisations to perform higher-level forms of phishing attacks against the ethical hackers, to see if they are finding what they should, and to ensure they are responding appropriately to the knowledge they appear to have gained during the attack. with the demand for cyber security professionals who conduct ethical hacking services and the importance and sensitivity of such a role, more research and a more formal approaches to ensure appropriate ethical and professional conduct, and compliance is required. references australian prudential regulation authority (2018). apra to introduce first prudential standard aimed at tackling growing threat of cyber attacks. retrieved from: http://www.apra.gov.au/mediareleases/pages/18_10.aspx bernoth, m., dietsch, e., burmeister, o. k., & schwartz, m. (2014). information management in aged care: cases of confidentiality and elder abuse. journal of business ethics, 122. doi:10.1007/s10551-013-1770-7 beyer, j. l. (2014). the emergence of a freedom of information movement: anonymous, wikileaks, the pirate party, and iceland. journal of computer‐mediated communication, 19(2), 141-154. bowern, m., burmeister, o., gotterbarn, d., & weckert, j. (2006). ict integrity: bringing the acs code of ethics up to date. australasian journal of information systems, 13(2). http://dx.doi.org/10.3127/ajis.v13i2.50 brodkin, j. (2009). the legal risks of ethical hacking. network world (online). retrieved december 17, 2015 from http://www.networkworld.com/news/2009/042409-usenixhacking.html burmeister, o. k. (2017). professional ethics in the information age. journal of information, communication & ethics in society, 15(4), 348-356. burmeister, o. k., islam, m. z., dayhew, m., & crichton, m. (2015). enhancing client welfare through better communication of private mental health data between rural service providers. australasian journal of information systems, 19, doi:10.3127/ajis.v19i0.1206. australasian journal of information systems thomas, burmeister & low 2019, vol 23, research on applied ethics (cybersecurity) ethical conduct by penetration testers 11 burmeister, o. k., phahlamohlaka, j., & al-saggaf, y. (2015). good governance and virtue in south africa's cyber security policy implementation. international journal of cyber warfare and terrorism, 5(1). burmeister, o.k. (2015). "improving professional it doctorate completion rates," australasian journal of information systems (19). http://dx.doi.org/10.3127/ajis.v19i0.1073 burmeister, o. k. (2013). achieving the goal of a global computing code of ethics through an international-localisation hybrid. ethical space: the international journal of communication ethics, 10(4). carter, a. (2012). “from anonymous to shuttered websites, the evolution of online protest”, cbc. retrieved from: https://www.cbc.ca/news/canada/from-anonymous-to-shutteredwebsites-the-evolution-of-online-protest-1.1134948 charmaz, kathy (2006). constructing grounded theory: a practical guide through qualitative analysis. sage publications inc., thousand oaks, california. p43. chertoff, m., & simon, t. (2015). the impact of the dark web on internet governance and cyber security. https://www.cigionline.org/publications/impact-dark-web-internetgovernance-and-cyber-security creswell, j. (2009). research design: qualitative, quantitative, and mixed methods approaches, third edition. sage publications inc. thousand oaks, california criminal code act of 1995 (cth) . retrieved from: http://www5.austlii .edu.au/au/legis/cth/consol_act/cca1995115/sch1.html capurro, r., & britz, j. b. (2010). in search of a code of global information ethics: the road travelled and new horizons. ethical space: the international journal of communication ethics, 7(2/3), 28-36. davis wright tremaine llp (2016) summary of u.s. state data breach notification statuses. retrieved from: https://www.dwt.com/statedatabreachstatutes/ directive 2013/40/eu (2013). retrieved from: https://eurlex.europa.eu/lexuriserv/lexuriserv.do?uri=oj:l:2013:218:0008:0014:e n:pdf eminağaoğlu, m., uçar, e., & eren, ş. (2009). the positive outcomes of information security awareness training in companies–a case study. information security technical report, 14(4), 223-229. european commission (n.d.) protection of personal data. retrieved from: https://ec.europa.eu/info/law/law-topic/data-protection_en government of canada (2018). digital privacy act. retrieved from http://lawslois.justice.gc.ca/eng/annualstatutes/2015_32/fulltext.html graves, k. (2010). certified ethical hacker study guide. wiley publishing inc, indiana, usa identity theft resource center (2017). data breach reports. retrieved from: http://www.idtheftcenter.org/images/breach/2017breaches/databreachreport_2017.pdf jamil, d. a. n. i. s. h., & khan, m. n. a. (2011). is ethical hacking ethical? international journal of engineering science and technology, 3(5). australasian journal of information systems thomas, burmeister & low 2019, vol 23, research on applied ethics (cybersecurity) ethical conduct by penetration testers 12 li, n., qardaji, w., su, d., wu, y., & yang, w. (2013). membership privacy: a unifying framework for privacy definitions. in proceedings of the 2013 acm sigsac conference on computer & communications security (pp. 889-900). acm. ling, t. (2018). canada: security breach notification requirements commence on november 1, 2018. retrieved from: http://www.bakerinform.com/home/2018/4/6/canada-security-breachnotification-requirements-commence-on-november-1-2018 livari, j., & livari, n. (2011). varieties of user-centredness: an analysis of four systems development methods. information systems journal, 21(2), 125-153. doi:10.1111/j.13652575.2010.00351.x ludlow, p. (2010). wikileaks and hacktivist culture. the nation, 4, (25-26). mandiant (2017). m-trends 2017: a view from the front lines. retrieved from: https://www.fireeye.com/blog/threat-research/2017/03/m-trends-2017.html mikhaylova, g (2014), the “anonymous” movement: hacktivism as an emerging form of political participation . retrieved from: https://digital.library.txstate.edu/bitstream/handle/10877/5378/mikhay lova-thesis-2014.pdf?sequence=1 moher d, liberati a, tetzlaff j, altman dg, the prisma group (2009). preferred reporting items for systematic reviews and metaanalyses: the prisma statement. plos med 6(7): e1000097. doi:10.1371/journal.pmed1000097 muntermann, j., & roßnagel, h. (2009). on the effectiveness of privacy breach disclosure legislation in europe: empirical evidence from the us stock market. in nordic conference on secure it systems (pp. 1-14). springer, berlin, heidelberg. myers, j., frieden, t. r., bherwani, k. m., & henning, k. j. (2008). ethics in public health research: privacy and public health at risk: public health confidentiality in the digital age. american journal of public health, 98(5), 793-801. national privacy commission (n.d.) republic act 10173 – data privacy act of 2012. retrieved from https://privacy.gov.ph/data-privacy-act/ new york state department of financial services (2017). cybersecurity requirements for financial services companies. retrieved from: https://www.dfs.ny.gov/legal/regulations/adoptions/dfsrf500txt.pdf national institute of standards and technology (nist) (2018). new data show demand for cybersecurity professionals accelerating. retrieved from: https://www.nist.gov/newsevents/news/2018/11/new-data-show-demand-cybersecurity-professionals-accelerating office of the australian information commissioner (n.d.). notifiable data breaches scheme. retrieved from: https://www.oaic.gov.au/privacy-law/privacy-act/notifiable-databreaches-scheme office of the australian information commissioner (2017). guide to securing personal information. retrieved from: https://www.oaic.gov.au/agencies-andorganisations/guides/guide-to-securing-personal-information australasian journal of information systems thomas, burmeister & low 2019, vol 23, research on applied ethics (cybersecurity) ethical conduct by penetration testers 13 office of the australian information commissioner (2017). what is personal information? retrieved from: https://www.oaic.gov.au/agencies-and-organisations/guides/what-ispersonal-information#how-does-the-privacy-act-define-personal-information radziwill, n., romano, j., shorter, d., & benton, m. (2015). the ethics of hacking: should it be taught? software quality professional, 18(1), 11-15. retrieved from http://search.proquest.com.ezproxy.csu.edu.au/docview/1764191740?accountid=10344 romanosky, s., telang, r., & acquisti, a. (2011). do data breach disclosure laws reduce identity theft?. journal of policy analysis and management, 30(2), 256-286. scholl, m, stine, k., hash, j., bowen, p., johnson, l., smith, c., steinberg, d. (2008). sp 800-66 rev. 1 – an introductory resource guide for implementing the health insurance portability and accountability act (hipaa) security rule. national institute of standards & technology. p31 solomon, d. b. (2014). employee and organization security value alignment through value sensitive security policy design. (phd dissertation), nova southeastern university, retrieved from http://nsuworks.nova.edu/gscis_etd/4 solove, d. (2006) . a brief history of information privacy law. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=914271 stoneburner, g., goguen, a. y., & feringa, a. (2002). sp 800-30. risk management guide for information technology systems. http://www.icsdefender.ir/files/scadadefenderir/paygahdanesh/standards/nist%20-%20800-30r0%20%20risk%20management%20guide%20for%20it%20systems.pdf thomas, g. (2018). should my australian business be concerned about gdpr? retrieved from: https://www.praesecure.com/2018/02/28/should-my-australian-business-be-concernedabout-gdpr/ thomas g., burmeister, o. k., low, g. (2017). issues of implied trust in ethical hacking: proceedings for the australasian conference on information systems 2017 conference. hobart. thomas, g., duessel, p., & meier, m. (2017). ethical issues of user behavioral analysis through machine learning. journal of information system security, 13(1). thomas g., low g., burmeister o. (2018) “who was that masked man?”: system penetrations—friend or foe?. in: prunckun h. (eds) cyber weaponry. advanced sciences and technologies for security applications. springer, cham u.s. department of health & human services (2013). breach notification rule. retrieved from: https://www.hhs.gov/hipaa/for-professionals/breach-notification/index.html wall, a. (2017). summary: philippines data privacy act and implementing regulations. international association of privacy professionals. retrieved from: https://iapp.org/news/a/summaryphilippines-data-protection-act-and-implementing-regulations/ weidenmier, m. l., & ramamoorti, s. (2006). research opportunities in information technology and internal auditing. journal of information systems, 20(1), 205-219. whitehouse, d., duquenoy, p., kimppa, k. k., burmeister, o. k., gotterbarn, d., kreps, d., & patrignani, n. (2016). twenty-five years of ict and society: codes of ethics and cloud computing. acm sigcas computers and society, 45(3). doi:10.1145/2874239.2874242 australasian journal of information systems thomas, burmeister & low 2019, vol 23, research on applied ethics (cybersecurity) ethical conduct by penetration testers 14 verizon (2018). 2018 data breach investigations report, 11th edition. retrieved from: https://www.verizonenterprise.com/resources/reports/rp_dbir_2018_report_en_xg.pd f copyright: © 2019 thomas, burmeister & low. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems lim 2020, vol 24, post publication review research directions in information systems 1 mazaheri, e., lagzian, m., & hemmat, z. (2020). research directions in information systems field, current status and future trends: a literature analysis of ais basket of top journals. australasian journal of information systems, 24. https://doi.org/10.3127/ajis.v24i0.2045 review as we enter the second decade of the new millennium, the paper by mazaheri et al. (2020) provides a timely update on the state of information systems research over the last decade. using a systematic review approach, mazaheri and colleagues considered the assortments and trajectories of topics and methodologies of information systems research published in the top basket of information systems journals selected by the association for information systems— namely: • european journal of information systems • information systems journal • information systems research • journal of the association for information systems • journal of information technology • journal of management information systems • journal of strategic information systems • mis quarterly the results from the systematic review suggest that electronic business and commerce, general information systems research, and information systems adoption and usage were the top three topics and that survey, mathematical modelling, and case study were the top three methodologies for information systems research in the top basket of information systems journals. notwithstanding the rich insights shed on the state of information systems research in the top basket of information systems journals over the last decade, mazaheri and colleagues have rightly acknowledged the limitation of their systematic review in understating the interest and importance of emerging topical trends, such as blockchain, fintech, internet of things, and sharing economy. this is unsurprising given that scholarly interest on these topics have only proliferated recently and that scholarly publishing is often a lengthy, rigorous process. in this regard, this limitation is appropriate and acceptable. nevertheless, the promise of systematic reviews was not fully delivered in that paper. in particular, the goal of systematic reviews is to deliver a replicable and meticulous summary of specific sources of information that informs and shapes future research agenda (paul et al., 2019)—the former (or the summary) was richly conveyed, but the latter (or the research agenda) was underdeveloped by mazaheri et al. (2020). the duality in the goal of systematic reviews should therefore be considered by information systems scholars interested to conduct systematic reviews, whereby a significant portion of the paper should be dedicated for developing a research agenda that advances conceptual (lim, 2018), contextual (lim et al., 2019), and methodological (lim & mandrinos, 2020) understanding of information systems. australasian journal of information systems lim 2020, vol 24, post publication review research directions in information systems 2 to this end, it is important to note that the australasian journal of information systems, which is ranked “a” by the australian council of professors and heads of information systems (acphis) and the australian business deans council (abdc), is a leading information systems journal outside the top basket that offers the opportunity for information systems scholars to participate in constructive discussion leading to the strengthening of ideas and arguments of its past papers. this exemplifies the contribution of post-publication reviews, and more importantly, the additional value that information systems journals outside the top basket can contribute to enhancing theory and practice in the discipline of information systems. weng marc lim swinburne university of technology lim@wengmarc.com / marclim@swin.edu.au / wlim@swinburne.edu.my author response the authors would like to thank the reviewer for his comments on our article. references lim, w. m. (2018). dialectic antidotes to critics of the technology acceptance model: conceptual, methodological, and replication treatments for behavioural modelling in technology-mediated environments. australasian journal of information systems, 22. doi:10.3127/ajis.v22i0.1651 lim, w. m., & mandrinos, s. (2020). incidental findings on multimodel comparison, hypothesis testing error, and its solution: the case of information, knowledge, and career pursuit. australasian journal of information systems, 24. doi:10.3127/ajis.v24i0.2555 lim, w. m., lim, a. l., & phang, c. s. c. (2019). toward a conceptual framework for social media adoption by non-urban communities for non-profit activities: insights from an integration of grand theories of technology acceptance. australasian journal of information systems, 23. doi:10.3127/ajis.v23i0.1835 mazaheri, e., lagzian, m., & hemmat, z. (2020). research directions in information systems field, current status and future trends: a literature analysis of ais basket of top journals. australasian journal of information systems, 24. doi:10.3127/ajis.v24i0.2045 paul, j., dwivedi, y., pappu, r., & roy, s. k. (2019). research in strategic marketing: past and future: developing new models, reviews, measures, methods with future research agenda. journal of strategic marketing, 27(5), 369–372. doi:10.1080/0965254x.2019.1572272 copyright: © 2020 lim. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v24i0.2921 http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems lim 2020, vol 24, post publication review research directions in information systems 3 paper title (paper title style) australasian journal of information systems kopplin 2023, vol 27, research article a configurational view on technology acceptance 1 a configurational view on technology acceptance: the example of highly integrated collaboration platforms cristopher kopplin university of bayreuth, germany cristopher.kopplin@uni-bayreuth.de abstract highly integrated software environments for various routine and non-routine tasks promise productivity gains for organizations. to fulfill this promise, users need to be willing to employ the new technology. a combined perspective of sufficient and necessary conditions in the form of fuzzy-set qualitative comparative analysis (fsqca) and, necessary condition analysis (nca) is used to examine the technology acceptance of workstream collaboration tools, advancing examinations from a multivariate perspective to a more holistic view. one hundred thirty participants were trained in the software application slack for three months. following the training period, configurational analysis using fsqca and nca based on a unified theory of acceptance and use of technology (utaut) framework was conducted based on 116 qualified questionnaires. necessity assessment shows that all influence factors exhibit necessity properties, with facilitating conditions and effort expectancy most substantially constraining an individual’s intention to use. sufficiency evaluation confirms utaut’s variable choice and identifies social influence as a key condition that enables intention to use. segmentation according to gender further reveals that effort expectancy and facilitating conditions are necessary conditions for female users but not for males. keywords: workstream collaboration, technology acceptance, collaboration software, fsqca, nca. 1 introduction previous work has examined collaboration technology, employing the word as an umbrella term for individual technologies such as group decision support systems (desanctis & gallupe, 1987), e-mail and videoconferencing (bajwa et al., 2005), instant messaging (lou et al., 2005), and even sms (brown et al., 2010). research has further noted that in many cases, the adoption and use of multiple technologies aiming at fostering collaboration may be the most fruitful approach (bajwa et al., 2005; desanctis et al., 2001). a novel type of application named workstream collaboration tools (wct) advance this notion by providing access to a variety of functions that have been carried out by distinct tools, integrating them in a single place and making them accessible through one dashboard and syntax (gartner, 2018; kopplin & baier, 2020; reynolds). wct target improving communication and collaboration by the grouping of different communication channels, enabling automation such as embedding artificial intelligence and allowing deep integration of third-party tools to provide a substantial degree of tool customization (gartner, 2018; reynolds, 2018). in this regard, they allow easy access to business intelligence and analytics, which is particularly relevant to organizations that lack experience or budget, such as smes (wee et al., 2022), and provide the opportunity for continuous modification and improvement, catering to the notion of digital kaizen (dang-pham et al., 2022). wct provide an extensive set of functionalities and are inherently designed for the multi-purpose application, and many of their design features users australasian journal of information systems kopplin 2023, vol 27, research article a configurational view on technology acceptance 2 are already familiar with from the consumer context (koch et al., 2019). in sum, they promise support for productivity, innovation, and growth (verma et al., 2022). predominant applications are slack, which offers features that range from mere scheduling and file sharing to operating external software tools, and microsoft teams, which recently surpassed slack in numbers of users and is now in the first place regarding size (microsoft, 2020; novet, 2019; slack, 2020). hence, alongside the technological progress, in general, collaboration technology also advances, becoming more versatile, mobile (i.e., supporting inter-device compatibility encompassing laptop computers, smartphones, tablets, and even car interfaces such as android auto and apple carplay), and complex. the range of features inherent to wct may be considered highly congruent with the notion of the umbrella term collaboration technology as specified by brown et al. (2010, p. 12): “collaboration technology is a package of hardware and software that can provide one or more of the following: (1) support for communication among participants, such as electronic communication to augment or replace verbal communication; (2) information-processing support, such as mathematical modeling or voting tools; and (3) support to help participants adopt and use the technology, such as agenda tools or real-time training […].” wct seek to offer solutions for all three aspects within a single integrative digital environment, hence condensing the ‘package of hardware and software’ into a clear and simple operating dashboard. consequently, wct may be framed as aiming for an optimal endpoint of collaboration technology evolution: providing barely limited access to functions and third-party applications offering communication, information-processing support, and help and training options. besides, drastic events such as the covid-19 pandemic have stressed the criticality of being able to react quickly to substantial changes in the work environment such as switching to work-at-home (conger, 2020), which requires the availability of reliable software solutions for daily business. indeed, organizations were found to rapidly adopt individual applications such as video conferencing tools and social mediabased software (dey et al., 2020). wct can help integrate these various applications and provide an accessible infrastructure for users’ different tools and functionalities. to provide consistency with previous findings in the field, the unified theory of acceptance and use of technology (utaut, venkatesh et al., 2003) is employed as a guiding framework. utaut has been used in thousands of studies and has been compiled and is implemented by means of regression-based procedures. hence, there is a substantial lack of methodological plurality to assess the model’s power, and the net effect perspective of regression-based analyses omits potential interactions between influence factors (woodside, 2013). hedonic motivation is added to take the notion of dual-purpose information systems (is) into account (wu & lu, 2013). one-hundred and thirty participants were trained in using slack for three months and subsequently answered a utaut-based questionnaire. insights into actual usage behaviour are gained through observations of installed integrations, exchanged messages, and dedicated survey questions. taking into account the complexity of wct, the study at hand employs fuzzy-set qualitative comparative analysis (fsqca), which allows the identification of so-called causal recipes, i.e., combinations of conditions that are sufficient to elicit the outcome of interest (ragin, 2009). this approach allows the detection of distinct yet equifinal causal paths evoking the phenomenon under investigation, i.e., intention to use a wct, and thus incorporates potential australasian journal of information systems kopplin 2023, vol 27, research article a configurational view on technology acceptance 3 heterogeneity in user perceptions. further, in contrast to the majority of the extant empirical findings that substantially rely on regression-based methods, fsqca can depict interactions among predictors. in addition to sufficient conditions, set theory also enables the researcher to find must-haves, i.e., necessary conditions that need to be present for the outcome to occur. for this purpose, necessary condition analysis (nca) has been proposed as an advancement of fsqca’s necessity assessment (dul, 2016a, 2016b). thus, the study at hand seeks to (1) shed light on the causal combinations that lead to individuals’ intention to use wct, and (2) identify necessary conditions that must be present to allow the intention to use to occur. the remainder of the paper is structured as follows: section 2 provides an overview of the literature and offers a brief insight into wct’s history. the research model is constituted in section 3, followed by results in section 4, and a discussion in section 5. the last two sections conclude the study’s findings, address its limitations, and propose future research paths. 2 related work 2.1 workstream collaboration tools communication and collaboration are central and interconnected elements of organizations and yield multiple impacts on subjects such as work efficiency (tjosvold and tsao, 1989), problem-solving (gray, 1985), information and knowledge sharing (cabrera & cabrera, 2002; hendriks, 1999; inkpen, 1996), and innovation (cooper, 2019; cooper & kleinschmidt, 1987). in fact, communication may be the underlying factor creating and sustaining organizations (schoeneborn et al., 2019). since the introduction of information technology, organizations have employed several cycles of software applications that seek to facilitate exchange among employees (bloom et al., 2014; hinds & kiesler, 1995; nunamaker et al., 1991). the extant literature has emphasized the importance of building trust in these virtual teams collaborating via digital applications (hacker et al., 2019), and wct serve three main challenges of establishing trust (pearlson et al., 2016): communication, technology, and team diversity. wct provide different but standardized communication channels, allowing simple meeting scheduling across time zones and calendar applications, and videochat functionality as well as the use of emojis facilitate dynamic communication. the opportunity to use integrations, either stand-alone or as connections to pre-existing applications, such as e-mail services, offer a standard set of tools, facilitating learning and support among team members. finally, many wcts implement influences from consumer-context software, such as whatsapp, discord, and facebook messenger, and include emojis, stickers, gifs, and other functions to establish team identity. recent wct developments are characterized by the integration of instant messaging, primarily consisting of text-chat functionality and presence information, and voice-based communications, and increasingly comprise additional channels (riemer & frößler, 2007). the demand for such highly integrated tools is fuelled by a number of workplace developments, such as the rising importance of virtual, locally dispersed teams that are composed according to knowledge and skills rather than place (tuma, 1998). attempts to provide communication in real-time date back to the 1990s (riemer & frößler, 2007), and have become ubiquity due to the omnipresence of online-compatible devices such as laptops, tablets, and smartphones (ladd et al., 2010). the advent of mobile and inter-device computing with its specific requirements regarding input and output modes has also influenced organizational is through consumerization (harris et al., 2012; jarrahi et al., 2017). a significant precondition for australasian journal of information systems kopplin 2023, vol 27, research article a configurational view on technology acceptance 4 multiplexed platforms is the convergence of infrastructure and “interoperable applications and services on an integrated machine” (riemer & frößler, 2007, p. 286). these characteristics have sparked a multiplicity of denominations, such as real-time communication and real-time collaboration systems (riemer & frößler, 2007), enterprise communication and collaboration (kryvinska et al., 2009), unified communications and collaboration (alias et al., 2017; chung & shin, 2011), unified communications (riemer & taing, 2009), and unified messaging (lai et al., 2002), which share many commonalities and are often used interchangeably. wct are referred to as the next development stage of integrated communication and collaboration platforms (gartner, 2018). they gained traction when global player microsoft entered the market in 2017 and presented their own platform solution called teams (unify square, 2019). other well-established companies contributed their respective applications, such as ibm, cisco, or google (gartner, 2018). figure 1 gives an overview of a prototypical application. reynolds (2018) puts their functionality in a nutshell: they bring “messaging, notifications, files, bots, tools and people together to create a private, persistent and searchable digital workspace that teams can use to do their work in a transparent, effective and efficient manner”. figure 1. scope of wct, based on gartner (2018) in essence, wct combine popular features such as text messaging in group and direct channels with novel instruments such as chatbots and automation, allowing faster and more efficient execution of tasks. however, their real excellence is achieved by a strong focus on facilitating third-party integrations. in this vein, the workflow using bots and automation can be transferred to existing tools such as project management in trello and asana or social media curation on twitter and facebook. however, organizations may also write their own extensions. due to this flexibility and variety of development, wct not only provide a starting point to implement the continuous improvement philosophy of digital kaizen (dang-pham et al., 2022), but also facilitate taking on novel tasks (from the organization’s perspective) such as diving into business intelligence and analysis, which is frequently challenging particularly for smes (wee et al., 2022). taken together, what differentiates wct from predecessors is their substantial support of non-routine tasks, adding to daily business routines that could already be handled by existing tools (reynolds, 2018). regarding device compatibility, wct focus on a combination of laptop and smartphone/tablet support (microsoft, 2020; slack, 2020) but also cover recent australasian journal of information systems kopplin 2023, vol 27, research article a configurational view on technology acceptance 5 technological advancements such as smartwatches (circuit, 2020) and in-vehicle information systems via android auto and apple carplay (unify, 2020). table 1 provides an overview of relevant wct for an impression of the segment’s development. application vendor launch segment azendoo azendoo 2012 bu, new player chanty chanty 2017 bu, new player circuit unify software and solutions 2014 bu, new player flock flock 2014 bu, new player ca flowdock ca technologies 2010 bu, new player exo platform exo platform 2014 bu, new player fuze fuze 2016 bu, new player glip ringcentral 2015 bu, new player hive hive 2016 bu, new player jandi toss lab 2015 bu, new player jostle jostle corporation 2011 bu, new player keybase teams keybase 2017 bu, new player mattermost mattermost 2015 bu, new player microsoft teams microsoft 2017 td, incumbent moxtra moxtra 2013 bu, new player rocket.chat rocket.chat 2018 bu, new player ryver ryver 2015 bu, new player samepage samepage labs 2013 bu, new player symphony symphony communication services 2015 bu, new player slack slack 2013 bu, new player stackfield stackfield 2014 bu, new player twist doist 2017 bu, new player webex teams cisco webex 2018 (rebrand of spark, 2014) bu, new player wickr pro wickr 2016 bu, new player wimi wimi 2011 bu, new player workplace by facebook facebook 2016 td, incumbent quip salesforce (acquisition) 2013 td, incumbent zoho cliq zoho corporation 2017 bu, new player zulip dropbox (acquisition) 2015 td, incumbent table 1. wct market overview note. td = top-down, bu = bottom-up as the application and vendor columns indicate, the market is split into two segments: major companies from originally different backgrounds, such as microsoft and cisco, and specialized single-product companies. two main business models prevail: dissemination of proprietary software (e.g., chanty, circuit, slack, teams) and provision of open-source applications (e.g., exo platform, mattermost, rocket.chat, zulip). single-product companies’ backgrounds share many commonalities: in many cases, internal communication solutions were needed while working on a different project, so a solution was created in-house (mattermost, 2015; slack, 2019). in essence, two approaches to the market can be observed: a bottom-up approach driven by client needs, frequently realized through startup foundation, and a top-down approach that is propelled by market growth, which is a strategy preferably employed by incumbents. australasian journal of information systems kopplin 2023, vol 27, research article a configurational view on technology acceptance 6 2.2 technology acceptance of workstream collaboration tools technology acceptance research investigation collaborative technology has a long history and follows the trajectory of technological advancement. hence, the variety of examined technologies is rather large: early work studied group decision support systems (desanctis & gallupe, 1987), later followed by e-mail and videoconferencing systems (bajwa et al., 2005), sms (brown et al., 2010), and their multi-/omni-platform predecessor instant messaging (lou et al., 2005). although varying in their functional range, collaboration technologies seek to provide communication, which can either augment or replace verbal communication, employ tools for information processing, and offer easy-access functionality, e.g., in the form of realtime training (brown et al., 2010). while not all of these aspects need to be fulfilled by a technology to be considered a collaboration technology (brown et al., 2010), recent platform or dashboard-based applications offer a wide range of features that covers these three pillars, as figure 1 shows. hence, novel collaboration technology, such as wct, exceed their predecessors in functional diversity and platform compatibility, making them more complex than previous applications. even more, wct may implement other collaboration technology such as instant messaging, videoconferencing, and e-mail. while the former two are standard features of wcts, such as slack or microsoft teams, the latter is sometimes presented as outdated and to be replaced with instant messaging but is also supported through wct integrations if the user wishes to continue using e-mail. against the backdrop of the conceptualization by brown et al. (2010), it appears that what sets wct apart from earlier collaboration technology is that all three aspects are indeed included in their features, instead of being optional. due to this substantial difference in scope and accessibility, it is necessary to update our knowledge on user perception and evaluation of modern-day collaboration technology. extant research on wct is scarce and fragmented, focusing on divergent topics such as their potential to provide an e-learning platform (pal and vanijja, 2020), their impact on knowledge work (lansmann et al., 2019), chatbot-based emotion management for distributed teams (benke et al., 2020), and chatbot-mediated task management (toxtli et al., 2018). in general, wcts’ chatbot functionalities appear to be their most investigated aspect. this narrow focus appears to omit their overall, complex quality, which is considered to both constitute a distinct new form of application and to change social structures established through the usage of previous collaboration technology (stoeckli et al., 2020). 3 conceptual model wct in their function to support communication and collaboration likely represent dualpurpose information systems (wu & lu, 2013), meaning that users emphasize both utilitarian and hedonic outcomes of their use behaviour. further, taking one of the three main characteristics of collaboration technology into account (brown et al., 2010), a supportive environment should be required, providing information and help with navigation, software failure, customization, and the like. further, as effective communication and collaboration demand that the involved individuals share some sort of communication channels, it is expected that social influence plays a role in the usage intention of wct. as set-theoretic procedures, such as fsqca, allow for theoretical and configurational multiplicity (park et al., 2020), it appears fruitful to start with a well-established framework in order to explore potential paths for examining multiplicity. one of the most dominant frameworks is utaut australasian journal of information systems kopplin 2023, vol 27, research article a configurational view on technology acceptance 7 (venkatesh et al., 2003), suggesting the explanation of an individual’s intention to use a particular technology as the result of a number of perceived properties, which will be detailed below. utaut has been cited in almost 45,000 studies, rendering it a seminal model for the study of technology acceptance, and offering insights into the acceptance of hundreds of technologies. it is important to note that the model is usually employed using regression-based methods, which have been criticized to omit important insights into causality (woodside, 2013). particularly, the results of a typical utaut study will present an effect size or a related measure for each of the independent variables, showing their isolated contribution considering the value of the dependent variable. this net effect perspective neglects potential interactions between influence factors, and only one generalized model is estimated and proposed for the whole dataset. on the other hand, fsqca allows the identification of several different configurations of influence factors, catering to the idea of equifinality (e.g., pappas & woodside, 2021). hence, examining the well-established utaut framework using fsqca allows more detailed insights into the factors’ relationships, potential interactions between influence factors, and different yet equifinal configurations of influence factors. due to wct’s strong influences from consumerization – a notion that postulates current organizational technology is heavily influenced by mechanisms and devices that employees already know from the consumer context (gewald et al., 2017; harris et al., 2012; jarrahi et al., 2017) – it is deemed reasonable to include notions of hedonic motivation for contextualization (venkatesh et al., 2016). hence, utilitarian aspects are captured in the form of performance expectancy and effort expectancy, denoting the productivity gains and ease of use of the application. hedonic factors are implemented through the variable hedonic motivation from utaut2. the notion of a supportive environment is depicted as facilitating conditions, and social influence serves as a measure of a user’s social surroundings and its influence on the individual regarding the use of the respective application. table 2 provides an overview of the factors and their conceptualizations. besides, moderating effects of age, gender, experience with the technology, and voluntariness of use are postulated (venkatesh et al., 2003). factor conceptualization adapted from performance expectancy individuals’ beliefs that the technology will help to increase work-related performance perceived usefulness, extrinsic motivation, jobfit, relative advantage, outcome expectations effort expectancy individual’s beliefs of a technology’s operability effort perceived ease of use, complexity, ease of use social influence individuals’ beliefs that other persons would like them to use the technology subjective norm, social factors, image facilitating conditions individuals’ beliefs that technical and organizational support is provided perceived behavioral control, facilitating conditions, compatibility hedonic motivation individuals’ beliefs that using the technology is entertaining perceived enjoyment table 2.utaut and utaut2 factors used for the study items for performance expectancy, effort expectancy, social influence, and facilitating conditions were adopted from venkatesh et al. (2003). hedonic motivation was captured using australasian journal of information systems kopplin 2023, vol 27, research article a configurational view on technology acceptance 8 the wording suggested by venkatesh et al. (2012). all items were measured on a 5-point likerttype scale and are attached in appendix a. figure 2 displays the final framework. figure 2. adapted utaut framework based on venkatesh et al. (2003) note. representation layout adopted from pappas (2018) further, age, gender, experience, and voluntariness of use are included as moderator variables in utaut (venkatesh et al., 2003). as individuals are trained for a limited amount of time, experience cannot be measured reasonably. besides, participation was voluntary; hence voluntariness of use cannot be adequately depicted as it would be the case in a real-world organizational setting. thus, age and gender are kept for analysis. in the following, names of latent variables are abbreviated as follows: performance expectancy pe, effort expectancy ee, social influence si, facilitating conditions fc, hedonic motivation hm, and behavioural intention to use bi. in order to assess the adapted framework displayed in figure 2, an appropriate analysis requires the capability to map interactions between influence factors. for this end, fsqca is used (ragin, 2009). this procedure allows the inclusion of equifinality, i.e., examining whether different combinations of influence factors (so called conditions) are capable of producing the outcome. consequently, instead of providing isolated effect sizes for the conditions, condition combinations that are sufficient to elicit the outcome under investigation are being detected. in fsqca methodology, it is possible that several different combinations, or configurations, are able to evoke the outcome. these different causal paths that all lead to the same result are described by the term equifinality. another intriguing property of fsqca is the notion of an ecology of configurations (park et al., 2020). this term denotes the idea that multiple theoretical explanations may overlap in the examination of a particular dataset. after retrieving the different configurations eliciting the outcome of interest, some of these configurations may be explained be the same theoretical perspective, while others may reflect a different theory, or lack an explanation altogether, termed theoretical multiplicity. on the other hand, as park et al. (2020) put it: “[…] a given phenomenon may be accounted for by three, or k configurations […]” (p. 1500) called configurational multiplicity. thus, fsqca helps shed new light on technology acceptance of australasian journal of information systems kopplin 2023, vol 27, research article a configurational view on technology acceptance 9 wct and the utaut framework by giving insights into these two classes of plurality, potentially opening new research paths for a well-established field of study. in contrast to its predecessor, which was denoted crisp-set qca or csqca for differentiation, non-dichotomized membership scores ranging between 0 and 1 are employed (mendel & korjani, 2012; ragin, 2009; schneider & wagemann, 2010). due to its foundations in set theory and fuzzy logic, the method allows the identification of interactions between conditions (which roughly correspond to factors in multivariate analysis) and explicitly stresses the configurational nature of phenomena, i.e., the existence of multiple different condition combinations leading to equivalent outcomes is expected (ragin, 2009, 1987). the utaut model serves as a theoretical baseline for condition identification in the study’s context; hence, the next step requires the calibration of each condition’s measures into fuzzy sets to determine membership scores (schneider & wagemann, 2010). the calibration procedure is crucial for the results of any fsqca applications, and several methods for the calculation of membership scores have been proposed. empirical calibration methods exist, such as employing the median as the cross-over point and the 5 % and 95 % quantiles as thresholds for full non-membership and full membership, respectively (woodside, 2013). however, due to the (quasi-)symmetrical nature of likert-type scales, which are frequently used, a theoretical calibration using the scale points as anchors is deemed more representative of the measures (see, e.g., ordanini et al., 2014). besides, in contrast to multivariate methods, fsqca does not make assumptions of symmetrical effects (woodside, 2013). commonly, models are interpreted in the way that a postulated relationship between two measures can be inverted, i.e., when the extant literature suggests that the presence of an influence factor should lead to the presence of an outcome, researchers often expect that the absence of this influence factor should be linked to the absence of the outcome. however, implicit in this expectation is the assumption of symmetrical effects. instead, two separate analyses are conducted for the absence and the presence of the outcome variable in fsqca, respectively (ragin, 1987). consequently, differences in condition interactions eliciting the outcome and the negation of the outcome are identified. 4 methodology 4.1 sample strategy and training over three months, 130 participants were recruited from university students and trained in operating slack. more advanced techniques such as using chatbot automation were rehearsed after familiarizing with the tool’s essential features. at the end of the three-month period, a survey was scheduled for all 130 users. training in software usage was designed to cover all relevant aspects of daily operation in business. this approach is consistent with the literature, which seeks to explain and predict acceptance “after a brief period of interaction with the systems” (szajna, 1996, p. 85). two exercise sheets were composed, drawing on literature on software utilization in office environments, wct vendor sites, and blogs dealing with the topic, as well as informal interviews with experienced slack users for task definition. at the beginning of the training period, all participants received e-mails with invitations to a dedicated slack workspace. two channels were set up, one for posting answers to the exercise sheets and one for general usage. after all participants had joined the workspace, an australasian journal of information systems kopplin 2023, vol 27, research article a configurational view on technology acceptance 10 introductory face-to-face presentation was given, briefly explaining the application and the study's course. two weeks were scheduled for informal tool exploration to get familiar with its user interface and basic functionality. participants were asked to use the desktop version. at the end of this period, the first exercise sheet was presented in a second lecture. tasks comprised using ‘slash commands’ (commands beginning with a forward slash, slack’s standard syntax) and interacting with slackbot, a pre-installed chatbot in every workspace. three weeks later, the second sheet was introduced in the same manner. participants had been using the application for about a month, giving insights into everyday operations. exercises from the second sheet covered the personalization of channels and the workspace as a whole, focusing on customizing information flow and embedding third-party integration. after familiarizing with the application’s desktop version, the smartphone app was introduced to take flexibility in workplace selection into account. all tasks were completed using the app as well. the final survey was launched at the end of the three months. in the questionnaire, respondents were shown a list of prototypical business tasks and asked to indicate which of these slack is deemed an adequate solution for. tasks were selected drawing on literature in the field, practitioner blogs and websites, vendor information on wct, and subsequent focus group evaluation (n = 8), on the lines of material construction for substitution-in-use as carried out in ratneshwar and shocker (1991). 4.2 descriptive statistics the training period was set from june to august 2019. in total, 116 questionnaires were collected. 14 participants dropped out at the end of the training period. the sample size satisfies recommendations for the condition/case proportion, which should range below 0.20 for five conditions in a mediumto large-n setting (maggetti & levi-faur, 2013; marx, 2010). respondents were between 22 and 29 years old (mean 24.34, median 24, sd 1.50). male and female respondents were balanced, yielding 58 participants in each group. respondents were asked if they were willing to initiate slack’s introduction in their workplace, given that no alternative application is already in use. 66.4 % answered yes, while 33.6 % negated. they were also asked for preferences in implementation: 83.6 % favor automation, such as slackbot. 32.8 % like to see an augmented reality version, allowing integrating the digital and the physical realm through smart glasses. virtual reality, implemented through headsets that fade out an individual’s environment, is labelled desirable by 28.4 %. eventually, respondents were asked for experience with slack. unsurprisingly, most respondents stated the training period (mean 3.33, median 2, sd 4.94, values in months). one respondent was treated as an outlier and excluded from analysis as he indicated 36 months, which was over half a year more than the next highest value. assessment of bi resulted in a mean of 2.56 (sd = 1.01), exhibiting a relatively low disposition to use the technology. participants also indicated whether they prefer the desktop or the mobile version of slack, yielding a surprisingly clear majority for the desktop version (96 %). scanning and trying out integrations were also part of the training. table 3 gives an overview of the top integrations that were stated to be helpful. although one of wscs’ inherent features is providing a persistent space for files and conversation, four cloud storage solutions can be found among the top eight implementations. australasian journal of information systems kopplin 2023, vol 27, research article a configurational view on technology acceptance 11 implementation mentions (%) top integrations field mentions (%) desktop app 96 4 google drive cloud storage 58.5 dropbox cloud storage 40.6 skype communications 31.1 trello project management 29.2 google calendar calendar 15.1 github cloud storage 7.5 wetransfer file sharing 7.5 microsoft onedrive cloud storage 6.6 table 3. most popular slack integrations 4.3 measurement evaluation membership calibration is a critical step in fsqca. hence, an assessment of the employed measures is carried out beforehand to ensure that valid and reliable information is used for calibration. internal consistency is checked using factor analysis, cronbach’s α, and the average variance extracted (ave). sufficient values are above 0.50 for ave and above 0.70 for cr and cronbach’s α (hair et al., 2019). for one item (fc4), the loading was very low (0.577), and it was removed from further analysis. a summary of all latent variables is provided in appendix c. table 4 summarizes the measurement evaluation results. variable indicators mean (sd) cronbach’s alpha ave pe 4 2.67 (0.84) 0.855 0.701 ee 4 3.97 (0.74) 0.893 0.761 hm 3 0.887 0.873 0.807 si 4 2.07 (0.95) 0.881 0.745 fc 4 (3) 4.05 (0.65) 0.672 0.608 bi 3 2.56 (1.09) 0.935 table 4. measurement assessment note. sd = standard deviance, ave = average variance extracted for assessment of discriminant validity, a triad of the fornell-larcker criterion (fornell et al., 1981), an examination of cross-loadings (hair et al., 2019a), and the heterotrait-monotrait ratio (henseler et al. 2015) is used. appendices c and d exhibit tables for fornell-larcker and htmt, indicating discriminant validity. htmtinference is calculated, corroborating discriminant validity as the null value of 1 is excluded from the 95 percent and 99 percent confidence intervals, respectively (henseler et al., 2015). for cross-loadings, all indicators load highest on their respective variable, corroborating discriminant validity. after establishing the validity and reliability of the employed measures, calibration into fuzzy sets was conducted. as all conditions were measured on a five-point likert-type scale, the value 3 may naturally serve as the point of maximum ambiguity (i.e., cross-over), while 1 is employed as the threshold for full non-membership, and 5 is used for full membership (see also ordanini et al., 2014). australasian journal of information systems kopplin 2023, vol 27, research article a configurational view on technology acceptance 12 5 findings 5.1 necessary conditions analysis using fsqca and nca necessary conditions analysis using fsqca in the first step, xy plots are used to get an overview of the conditions, and a necessary condition analysis is performed. a condition is necessary when (1) the absence of the condition corresponds to the absence of the outcome, and (2) the presence of the outcome corresponds to the presence of the condition (braumoeller & goertz, 2000; dul, 2016a). contrary to a sufficient condition, the presence of a necessary condition may occur when the outcome is absent (vis & dul, 2018). to assess the pool of conditions for necessity, each condition is analysed for the degree to which it constitutes a superset of the outcome (i.e., consistency). the coverage values imply the empirical relevance of the relation and help identify conditions that are necessary but irrelevant (ragin, 2006). table 5 summarizes the findings. note that in all cases, bi is the outcome (i.e., y), and high values correspond to consistency, while the lower values can be interpreted as coverage (ragin, 2009). condition (x) consistency x ← y (coverage) consistency x → y (consistency) pe 0.676 0.905 ee 0.469 0.974 hm 0.646 0.922 si* 0.825 0.583 fc 0.455 0.978 table 5. necessary condition analysis in kind using fsqca note. *the consistency/coverage interpretation is reversed for si following the logic of fuzzy sets, a condition is necessary when membership scores in the outcome are lower or equal to membership scores in the condition (dul, 2016a; schneider & wagemann, 2010; vis & dul, 2018). this subset relation can be observed using xy plots, depicting membership in the condition on the horizontal axis and membership in the outcome on the vertical axis. in the complementary case, i.e., when membership scores for the outcome are higher than membership scores for the condition, indications for a sufficient condition have been found. the ideal xy plot then shows a triangular case distribution (braumoeller, 2017; ragin, 2006). the graphical displays provide a more detailed picture of the summary in table 5; however, scores from the table give an impression of the set relations: for pe, ee, hm, and fc, consistency scores pass the recommended threshold of 0.90 (dul, 2016a; schneider & wagemann, 2012). however, an even more conservative value of 0.95 has been proposed, particularly for settings that aim at hypothesis testing (see, e.g., maggetti & levi-faur, 2013). as utaut and its respective path relations is an established model, the strict threshold is employed for the study at hand, revealing ee and fc as necessary conditions in kind. consequently, empirical evidence for ee and fc being necessary conditions could be established (see also the xy plots in appendix e, where most cases are below the linear slope). assessment of their empirical relevance, i.e., their coverage scores, yields particularly high values, accounting for about two-thirds of each case. consequently, ee and fc are identified to be necessary and empirically relevant (ragin, 2006). all graphs are also plotted for the inverse cases of the conditions being absent and the outcome being absent. figure 3 illustrates both variants for fc, displaying substantial evidence for fc being a necessary condition for bi (left-hand plot) and ~fc being a sufficient condition for ~bi (right-hand plot). australasian journal of information systems kopplin 2023, vol 27, research article a configurational view on technology acceptance 13 figure 3. xy plots for fc and bi, and ~fc and ~bi note. values indicate case ids however, in the case of si, an inverse result is found: as can be observed from the xy plots, most cases are located above the linear slope. hence, strong evidence for a sufficient condition was found, i.e., si → bi. the consistency score is 0.825, and the corresponding coverage is 0.583, indicating substantial empirical relevance. in consequence, si appears to be a sufficient condition for bi. analyses of sufficiency will be conducted in more detail after the complementing nca. necessary condition analysis using nca although fsqca provides guidance on the identification of necessary conditions, its primary aim is to detect sufficient conditions (bol & luppi, 2013). consequently, a second approach, namely nca, is employed for analysis. a ceiling regression with free disposal hull (cr-fdh) is used (dul, 2016b). nca’s idea is that necessary conditions put constraints on the outcome so that a certain level of the necessary condition is required to elicit the outcome’s presence. drawing on xy plots (with the horizontal axis corresponding to the condition and the vertical axis depicting the outcome), the ceiling line ascertained by nca separates the ‘full’ space of observations from the ‘empty’ space in the upper left corner. the larger this empty space is, the stronger the constraint that is put on the outcome. the results are summarized in table 6. a bootstrapping procedure drawing 10,000 subsamples was used to assess statistical significance. all conditions were found to exhibit significant effects on a 0.05 level. following dul’s (2016b) recommendations, the effect size of si can be considered small, the effects of pe, ee, and hm are medium, and fc yields a large effect. nca provides substantial evidence for the necessity of all five conditions. thus, in the next step, a more fine-grained evaluation is conducted. australasian journal of information systems kopplin 2023, vol 27, research article a configurational view on technology acceptance 14 condition (x) ceiling zone observations accuracy d p pe 0.184 2 0.983 0.227 < 0.001 ee 0.164 1 0.991 0.236 0.018 hm 0.216 3 0.974 0.267 < 0.001 si 0.012 2 0.983 0.015 0.003 fc 0.232 3 0.974 0.314 0.045 table 6 .nca results note. observations refer to cases that are located above the ceiling line, d = effect size, p = p-value derived from a bootstrapping procedure with 10,000 draws. for this purpose, a bottleneck table was crafted (table 7). a can be seen, a very low level of the outcome (10 %) already requires small values of hm and fc. considering that a relatively high level of bi needs to be achieved to elicit a regular usage pattern for wct, four out of the five conditions need to be taken into account (pe, ee, hm, and fc; considering the range of the outcome up to 90 %). the observation that for low outcome levels, most (or even all) conditions are not necessary, but become critical for higher levels of y, is a typical finding for necessary conditions (dul, 2016b). for the data at hand, the most substantial constraint emanates from fc. altogether, findings from fsqca (i.e., necessity in kind) and nca (i.e., necessity in degree) are asymptotically equivalent, as table 7 shows. for the full range of y, ee and fc exhibit the highest requirements, and fsqca identified ee and fc as necessary. also, si was not detected as necessary in kind, and necessity in degree yields a minimal effect (d = 0.015). table 7 displays that si, for most target values of y, does not exhibit constraints. hence, we may conclude that in this case, fsqca and nca both agree on the minor role of si. however, it becomes apparent that a mere analysis of kind would have neglected nca’s granular insights. y pe ee hm si fc 0 nn nn nn nn nn 10 nn nn 3.2 nn 3.2 20 nn nn 9.0 nn 10.3 30 3.3 3.1 14.9 nn 17.3 40 11.6 11.9 20.8 nn 24.3 50 19.9 20.6 26.6 nn 31.3 60 28.2 29.3 32.5 nn 38.3 70 36.6 38.1 38.4 nn 45.4 80 44.9 46.8 44.2 nn 52.4 90 53.2 55.5 50.1 nn 59.4 100 61.5 64.3 55.9 38.2 66.4 table 7: bottleneck for nca note. y is stated in percent of the observed values, e.g., y = 100 corresponds to the highest empirical outcome 5.2 truth table construction and minimization a truth table is constructed (ragin, 2009, 1987; schneider & wagemann, 2010). before collapsing it to receive solution terms, thresholds for consistency and frequency (i.e., the minimum number of cases) need to be specified (krogslund et al., 2015). regarding the frequency threshold, a minimum case number of two was chosen to derive fine-grained results. necessary conditions have been maintained to provide a complete picture of the results. although a consistency threshold of 0.75 had been originally proposed (rihoux & ragin, 2009), a higher value was aimed for. following the recommendation by pappas and australasian journal of information systems kopplin 2023, vol 27, research article a configurational view on technology acceptance 15 woodside (2021), the truth table was inspected for a natural breaking point, i.e., a relatively large gap in the sorted list of consistency values. the first large gap occurred between 0.805163 and 0.764102, indicating a breaking point. consistent with this empirical result and the recent literature (e.g., bacon et al., 2019), a consistency threshold of 0.8 was chosen. the truth table is collapsed, drawing on the quine-mccluskey algorithm. besides, the introduction of simplifying assumptions allows to find a balance between complexity and parsimony and is commonly carried out in two steps: usage of all possible assumptions leads to the parsimonious solution, while the intermediate one includes only theoretically substantiated simplifying assumptions (schneider & wagemann, 2013; thomann & maggetti, 2020). drawing on the extant literature employing utaut, the presence of each condition is supposed to contribute to the presence of bi for the derivation of the intermediate solution. intermediate solution m1 m2 m3 pe ⬤ ⬤ ee ⊗ ⊗ hm ⬤ ⬤ si ⬤ fc ⬤ ⬤ ⬤ consistency 0.833 0.810 0.857 raw coverage 0.394 0.396 0.565 unique coverage 0.006 0.009 0.272 solution consistency 0.782 solution coverage 0.675 table 8. intermediate fsqca solution note. consistency threshold = 0.80, frequency threshold = 2, simplifying assumptions for the intermediate solutions: the presence of pe, ee, si, hm, and fc should contribute to bi, large circles = core condition, small circles = peripheral condition a fundamental value of any crafted model is its consistency, indicating the proportion of cases for a particular combination of conditions that agree in the outcome, which should be at least 0.75 to assume the existence of a subset relation (ragin, 2006). commonly deviating from the perfect value of 1.0, consistency is interpreted as the approximation of a subset relation (maggetti & levi-faur, 2013; veri, 2018). coverage indicates a consistent subset’s empirical relevance, i.e., only consistent subsets can be meaningfully interpreted. table 8 displays the findings, marking core and peripheral conditions, respectively. three solutions were identified (m1, m2, and m3), yielding a satisfying overall consistency of 0.782. two distinct coverage measures can be used to assess the empirical relevance of a given solution (schneider & wagemann, 2010): the raw coverage details the share of the outcome that the solution covers in total, while the unique coverage refers to the share uniquely covered by this particular solution. in a similar manner, the overall coverage of the term including m1, m2, and m3 explains “to what extent the outcomes of interest can be determined based on the extracted set of solutions” (mikalef & krogstie, 2020, p. 272). the first solution term, m1, exhibits the presence of pe and fc combined with the absence of ee, where fc is a peripheral conditions. m1 both yields a high level of consistency (0.833) as well as a substantial coverage value (0.394), i.e., it explains a relevant number of cases. although sharing the absence of ee, m2 shows the presence of hm and fc. again, fc is a australasian journal of information systems kopplin 2023, vol 27, research article a configurational view on technology acceptance 16 peripheral condition. thus, in contrast to m1, pe is replaced with hm. this solution also exhibits a high consistency (0.810) and explains a relevant number of cases (coverage = 0.396). m3, again a consistent solution (consistency = 0.857) and the solution term with the highest unique coverage (0.272), integrates the presence of pe, hm, si, and fc, albeit treating ee as indifferent. importantly, the presence of si is the only core condition in this solution. in total, m1 and m2 reflect similar notions of low effort being an important condition but present different nuances in the wct’s main benefit: m1 emphasizes the importance of utilitarian aspects (in the form of pe), and m2 focuses on hedonic facets (in the form of hm). facilitating conditions, such as dedicated team members, software support, and internal documentation, play a peripheral role. m3 is an interesting solution in the way that it largely differs from the very similar models m1 and m2. here, the influence of an individual’s social surroundings is sufficient to evoke the intention to use the wct, as marked by the core condition si. in stark contrast to the other solutions, the absence of ee is not required, and even more, ee is treated indifferently. rather, the presence of all remaining conditions – pe, hm, and fc – are identified as peripheral conditions. in terms of raw coverage and unique coverage, it can be concluded that m3 yields the most substantial explanatory power of the three solutions. thus, the presence of si appears to be a good predictor for bi of wct, while pe, hm, and fc play peripheral roles. 5.3 inclusion of gender in line with the utaut framework, the effect of gender is assessed. for this purpose, two different streams of analyses are conducted. for the first series of runs, the dataset is split into subgroups for men and women, i.e., no dummy coding was used but separate datasets. for an overview, fsqca is used to identify necessary conditions in kind. table 9 summarizes the results. on a qualitative level, both subgroups yield similar results. for men and women, si exhibits properties of a sufficient condition. employing the conservative threshold of 0.95 for consistency, pe, ee, hm, and fc are identified as necessary in kind for the female segment, while ee and fc are detected for the male segment. in the case of fc, the female subgroup yields a striking consistency of 1.000. consequently, a perfect subset relation was found. interestingly, a switch from the 0.95 consistency threshold to the initially suggested value of 0.90 would not lead to different results, although hm’s consistency is reasonably close for the male subgroup (0.894). the findings, thus, are treated as reliable. condition (x) consistency x ← y (coverage) consistency x → y (consistency) men women men women pe 0.694 0.657 0.863 0.954 ee 0.484 0.452 0.971 0.978 hm 0.683 0.609 0.894 0.955 si* 0.804 0.846 0.524 0.652 fc 0.468 0.441 0.960 1.000 table 9. necessary condition analysis in kind using fsqca note. *the consistency/coverage interpretation is reversed for si nca is employed to gain insights into a necessity in degree. for the female subgroup, significant constraints could be found for all conditions. however, consistent with the overall dataset, si’s effect size appears negligible (d = 0.070). for the male segment, pe, hm, and si exhibit significant effects. again, si’s influence is minimal (d = 0.025) and may be neglected. compared side to side, the most striking difference is found for fc: while for women, the australasian journal of information systems kopplin 2023, vol 27, research article a configurational view on technology acceptance 17 condition yields a large effect, men are not affected at all (p = 0.597). in the case of pe, a large effect is found for women, while for men, it is medium. hm’s results are equivalent. ee has a large effect for women but no convincing influence for men (p = 0.073). table 10 displays the findings. condition (x) ceiling zone observations accuracy d p subgroup: women pe 0.270 2 0.966 0.361 < 0.001 ee 0.258 3 0.948 0.372 0.009 hm 0.302 3 0.948 0.385 < 0.001 si 0.052 1 0.983 0.070 0.028 fc 0.373 4 0.931 0.506 0.002 subgroup: men pe 0.184 1 0.983 0.227 < 0.001 ee 0.230 3 0.948 0.332 0.073 hm 0.189 2 0.966 0.234 0.001 si 0.020 1 0.983 0.025 0.003 fc 0.139 0 1.000 0.201 0.597 table 10. nca results note. observations refer to cases that are located above the ceiling line, d = effect size, p = p-value derived from a bootstrapping procedure with 10,000 draws for a more detailed picture of the segmentation, the bottleneck technique is used, as displayed in table 11. starting with si, the condition does not impose constraints for the most part for both segments. while in the female subgroup, effects become apparent for moderate values of y (i.e., 70 %), the male subgroup does not show restrictions until the full range of the outcome (i.e., 100 %). however, this constraint is twice as large as for women. in the cases of pe and fc, restrictions become visible earlier for women than for men; however, in contrast to si, they are also higher than for men. y pe ee hm si fc women men women men women men women men women men 0 nn nn nn nn nn nn nn nn 2.3 nn 10 4.5 nn nn nn nn 2.5 nn nn 12.0 nn 20 12.4 nn 0.2 3.8 6.3 7.7 nn nn 21.6 nn 30 20.3 3.3 11.7 13.2 16.7 12.9 nn nn 31.3 3.9 40 28.2 11.6 23.3 22.6 27.1 18.1 nn nn 40.9 11.0 50 36.1 19.9 34.9 32.0 37.5 23.3 nn nn 50.6 18.0 60 44.0 28.2 46.5 41.4 47.9 28.5 nn nn 60.2 25.1 70 51.8 36.6 58.1 50.8 58.3 33.7 7.9 nn 69.9 32.1 80 59.7 44.9 69.7 60.3 68.7 38.9 17.5 nn 79.5 39.2 90 67.6 53.2 81.3 69.7 79.1 44.1 27.1 nn 89.2 46.2 100 75.5 61.5 92.9 79.1 89.5 49.3 36.7 71.4 98.8 53.2 table 11. bottleneck for nca note. y is stated in percent of the observed values; e.g., y = 100 corresponds to the highest empirical outcome for fc, the differences are striking: depending on the desired level of the outcome, the female subgroup requirements are about twice as high in total, considering favourable outcome levels above 50 %. below this point, the disparities are more extreme. for pe, the factor is about 1.3 for high outcome values and considerably larger for low levels. ee shows similar patterns for both segments, albeit the constraints are somewhat divergent for the highest outcome levels. demands put on hm are higher for females than for males: while for men, moderate values australasian journal of information systems kopplin 2023, vol 27, research article a configurational view on technology acceptance 18 are satisfactory to achieve the full outcome range (about 50 % of hm), fairly desirable outcome levels may only be achieved through above-average condition values for women (i.e., more than 50 % of hm). the subsets yield 58 observations each. table 12 summarizes the results. the solutions yield satisfying consistencies (m4 = 0.882, m5 = 0.844). consistency and coverage values are higher for the female segment than for males; still, all measures indicate empirical relevance. coverage values show that both solutions explain a substantial number of cases: m4 exhibits a value of 0.639, and m5 yields a coverage value of 0.501. as for both subsets, only one solution is identified each, raw coverage values are equivalent to unique coverage. in contrast to necessity in kind, the sufficiency examination could not find striking differences considering gender. nevertheless, the variation in coverage implies that the solution terms’ empirical relevance is slightly higher for women than men. both solutions, however, emphasize the role of si. this finding has been shown in m3, which was the solution with the largest coverage values for the full dataset. still, m3 differs from these subset solutions in the way that ee is not treated as an indifferent condition but as a peripheral one. m4 and m5 are striking as they show less parsimonious models than the solutions for the full dataset albeit presenting identical terms. subgroup: women subgroup: men intermediate solution m4 parsimonious solution m5 pe ⬤ pe ⬤ ee ⬤ ee ⬤ hm ⬤ hm ⬤ si ⬤ si ⬤ fc ⬤ fc ⬤ consistency 0.882 consistency 0.844 raw coverage 0.639 raw coverage 0.501 unique coverage 0.639 unique coverage 0.501 solution consistency 0.882 solution consistency 0.844 solution coverage 0.639 solution coverage 0.501 table 12. fsqca solutions for the gender-specific subgroups note. females n = 58, males n = 58 5.4 predictive validity and sensitivity analysis in order to complete data analysis, an assessment of the model’s predictive validity was conducted. the sample is randomly divided into a training and a holdout partition to erect the model and predict the remaining observations (pappas & woodside, 2021). a 50/50 split is used, calculating a consistency of 0.883 and a coverage of 0.771 for the training sample. next, the solutions m1, m2, and m3 are coded as conditions in the holdout sample, using fuzzy logic. in the final step, these conditions can be plotted against the outcome, bi, in the holdout sample. m1 shows a consistency of 0.860 and a coverage of 0.413, indicating that the argument that m1 is a subset of bi is consistent with the data, and the model accounts for 41.3 % of the sum of the memberships in bi. similarly, m2 exhibits a consistency of 0.781, and a coverage of 0.416, and m3 achieves a consistency of 0.819, and a coverage of 0.516, yielding the same interpretation. in sum, it can be concluded that the identified solutions are able to make meaningful predictions for new samples, and predictive validity could be established. australasian journal of information systems kopplin 2023, vol 27, research article a configurational view on technology acceptance 19 finally, sensitivity analyses were conducted to evaluate the adequacy of the calibration (see, e.g., mikalef and krogstie (2020). to do so, the thresholds for full set membership and full nonmembership were slightly changed. as symmetric likert-type scales were used, the cross-over point was maintained, because no meaningful threshold outside the middle of the scale should exist. qualitatively, the identified solutions did not vastly change, maintaining the core conditions and the overall pattern of the peripheral ones. hence, it can be concluded that the calibration into fuzzy sets was appropriate and did not elicit patterns on its own. 5.5 specific propositions the application of fsqca allows researchers to test specific propositions (pappas and woodside, 2021). to do so, a new model is calculated, including the relevant conditions, and the model is plotted against the proposed outcome. for the context of technology acceptance, two propositions are assessed to gain further insights into user perception: (p1) wct yield a dual-use property, i.e., a combination of pe and hm should be linked to bi. wu and lu (2013) show that many applications, particularly communication-related ones, exhibit a dualpurpose role, i.e., users seek both utilitarian and hedonic gratifications from them. this dual role can be tested using pe and hm from the utaut2 framework. (p2) wcts as collaboration and communication platforms may serve a social hedonic role, i.e., a combination of hm and si should be linked to bi. wcts provide various different communication channels such as text and video and are influenced by application from the consumer context, such as whatsapp, discord, and facebook messenger. these applications have witnessed the inclusion of increasingly playful content, such as gifs, stickers, and emoji reactions to previous messages. in addition to sending information, they allow for an entertaining mode of operation. in the case of p1, a consistency of 0.867 and a coverage of 0.752 support the dual role of wcts, indicating that pe and hm considerations alone may inform the decision for or against a particular application. p2 saw the combination of hm and si, yielding a consistency of 0.846 and a coverage of 0.567. again, support for the proposition could be established, suggesting that wct acceptance is informed by social considerations and the opportunity to maintain entertaining communications, which is enabled by a vast array of emojis, gifs, stickers, and other features. 5.6 use behavior participants were asked about their actual usage behaviour besides the mandatory exercise sheets. a six-item ordinal scale was used, ranging from ‘i do not use slack’ to ‘i use slack several times a day’. 69.0% do not use slack while 31.0% do. next, participants were shown 18 use scenarios. they were asked to indicate which of the following are part of their consideration set (not using the specific term) for each of the tasks: slack, a dedicated, standalone application, or an “offline” tool (such as a phone). table 13 summarizes the results. the first two columns of table 13 can also be thought of as a transposed partition-by-use matrix, with two data rows (one for the data headers and one for slack evaluated over all known partitions) and 18 columns indicating use scenarios, i.e., intended purposes. onesample χ²-testing could not find evidence to reject the null hypothesis (i.e., all responses are equally likely) for use scenarios 1, 10, and 11. overall, evaluations follow a pattern, indicating the presence of preferences. australasian journal of information systems kopplin 2023, vol 27, research article a configurational view on technology acceptance 20 use scenario slack stand-alone offline χ² p (df = 2) 1 meet for lunch / coffee breaks 27.6 33.6 38.8 2.190 0.335 2 create polls° 66.4 31.0 2.6 71.086 < 0.001 3 read email 8.6 88.8 2.6 161.190 < 0.001 4 send email 7.8 89.7 2.6 166.052 < 0.001 5 read social media 5.2 87.9 6.9 155.655 < 0.001 6 write social media 3.4 90.5 6.0 170.810 < 0.001 7 share files° 62.9 34.5 2.6 63.431 < 0.001 8 phone colleagues 12.9 32.8 54.3 29.810 < 0.001 9 phone externals 3.4 38.8 57.8 52.879 < 0.001 10 hold meetings° 44.8 25.9 29.3 7.103 0.029 11 take quick notes° 44.0 31.0 25.0 6.534 0.038 12 manage calendar 28.4 62.1 9.5 49.362 < 0.001 13 support customers 25.0 66.4 8.6 61.672 < 0.001 14 analyze website statistics 26.7 71.6 1.7 87.121 < 0.001 15 socialize (in-house)° 57.8 15.5 26.7 33.328 < 0.001 16 find support for current challenges° 56.0 24.1 19.8 27.224 < 0.001 17 track and manage projects° 49.1 45.7 5.2 41.603 < 0.001 18 manage documentation 26.7 67.2 6.0 67.466 < 0.001 table 13. use scenario evaluation note. values in %, slack is preferred for this use scenario the first two columns of table 13 can also be thought of as a transposed partition-by-use matrix, with two data rows (one for the data headers and one for slack evaluated over all known partitions) and 18 columns indicating use scenarios, i.e., intended purposes. onesample χ²-testing could not find evidence to reject the null hypothesis (i.e., all responses are equally likely) for use scenarios 1, 10, and 11. overall, evaluations follow a pattern, indicating the presence of preferences. 6 discussion 6.1 summary the findings shed new light on technology acceptance of collaboration technology. in total, three solutions (m1, m2, and m3) were identified that explain the presence of bi. two of these – m1 and m2 – share a similar pattern, emphasizing the absence of ee as a core condition and the presence of fc as a peripheral one. they differ, however, in their presence of either pe (m1) or hm (m2). thus, one model stresses wcts’ utilitarian aspects, while the other includes their hedonic dimension. m3, on the other hand, yields si as a core condition and treats the remaining utaut variables as peripheral ones, except for ee, which is detected as an indifferent condition. what is particularly intriguing about this solution is the fact that it yields the highest coverage, i.e., explains the most substantial number of cases. dataset segmentation according to gender corroborates the importance of m3: gender-specific solutions show the same pattern, however also include the presence of ee as a peripheral condition. surprisingly, pe was not a necessary condition for the male subset, contradicting existing work on necessary conditions in technology acceptance (kopplin et al., 2021) but also questioning the variable’s predominant position in technology acceptance frameworks. commonly, multivariate analyses identify pe as a major factor (im et al., 2011; venkatesh et al., 2003; zhou et al., 2010). its role may be limited to that of a sufficient condition, depending on the setting, which corroborates demands for contextualization (venkatesh et al., 2016). regarding hm, the results provide convincing evidence of the relevance of including hedonic concepts in workaustralasian journal of information systems kopplin 2023, vol 27, research article a configurational view on technology acceptance 21 related settings. this importance has been stressed in the literature by the notions of dualpurpose is and consumerization (harris et al., 2012; jarrahi et al., 2017; wu & lu, 2013). for wct, in particular, the main functionalities are common to the user from consumer settings: a persistent text-based communication channel is at the core of a wct application, which resembles the look and feel of sms, online chatrooms, and smartphone-based instant messaging. audio and video calls are part of most mobile devices’ standard equipment, such as the facetime app provided by apple (apple, 2020). a similar case can be made for thirdparty integrations, which are a significant characteristic of wct (gartner, 2018). from the consumer context, many is users are familiar with searching, installing, and utilizing thirdparty apps for various tasks. consequently, gaining the opportunity to upgrade a work-related platform may appear ergonomic; although, for software applications in the organizational context, this operation mode is rather uncommon. 6.2 theoretical implications findings for the total sample, i.e., m1 and m2, reflect the dyadic nature of dual-purpose information systems (wu & lu, 2013). both configuration explain about the same proportion of cases; however, m1 emphasizes the utilitarian nature of wct, while m2 focuses on their hedonic dimension. consequently, wct may be treated as dual-purpose information systems. the presence of fc as a peripheral condition indicates that individuals include their environmental resources regarding the technology in their decision-making: facilitating conditions may manifest in the form of a dedicated software support, a power user helping others, rich documentation, or professional training. thus, technology acceptance, at least in the context of wct, does not merely reflect user perception of the technology’s properties but considers the surroundings in which the target technology is to be implemented. the parsimonious nature of solutions m1 and m2, treating about half of the utaut variables as indifferent, suggests that the extensive framework may not be required in its full range to explain technology acceptance. m3 exhibits the most substantial empirical relevance considering its coverage values. here, si is the only core condition, extended by peripheral ones except for ee. thus, technology acceptance may be significantly shaped through social consent, which would imply a less important role of the wcts’ characteristics. as research on technology acceptance, adoption, and user experience advances, it appears likely that competing applications converge in their perceived qualities such as pe, ee, and hm. consequently, the final decision for or against technology acceptance in an industry offering similar solutions may be shifted to other considerations, such as the social environment in which the application is planned to be used. from a necessity perspective, fsqca necessity analysis results showed that low perceived effort (i.e., ee) is a critical condition. this finding corroborates existing necessity results for technology acceptance models (kopplin et al., 2021) and provides an important supplement to the existing insights into the role of effort. the majority of findings on technology acceptance uses structural equation modeling, which frequently identifies perceptions of low effort as exerting a small impact or being not significant as an independent variable (see, e.g., king & he, 2006). besides ee, fc was detected as a necessary condition both by fsqca and nca, yielding a large effect (dul, 2016b). fc emphasizes the perceived compatibility of the technology with the individual’s environment and denotes a user’s “belief related to one’s control over the use of is” (venkatesh et al., 2011, p. 534). thus, it is relatively unsurprising that environmental conditions have a crucial impact on usage intention regarding a highly australasian journal of information systems kopplin 2023, vol 27, research article a configurational view on technology acceptance 22 integrated, pervasive collaboration platform. these substantial constraints suggest that users require a supportive environment, which may be attributed to the wct’s feature range itself in the form of training, documentation, and helpful navigation tutorials, confirming that the collaboration technology qualities proposed by brown et al. (2010) also apply to wct. even more, fc is demanded to allow use behaviour. drawing on the bottleneck technique (dul, 2016b), fc was identified as a chokepoint for any level of bi and thus needs to be included in managerial decisions. summarizing the necessity assessment, two approaches were used to identify necessary conditions: an fsqca procedure and nca. at first glance, results appear contradictory: fsqca identifies ee and fc as necessary conditions, while nca finds statistically significant constraints for all five conditions. here, it is important to note that the consistency thresholds for fsqca were set to a very high value (0.95). a more liberal threshold of 0.90 may have been used (dul, 2016a; schneider & wagemann, 2012). in this case, fsqca and nca yield similar results: pe, ee, hm, and fc pass the fsqca threshold and are identified as necessary. both analyses differ in their assessment of si: fsqca offers evidence for a sufficient condition. nca, by its nature, can only provide insights into a potential necessity, and thus does not establish sufficiency for si. on the other hand, while the bootstrapping procedure finds a statistical significance for si’s ceiling zone, the effect size is very small (d = 0.015). thus, the constraints imposed by si seem negligible, and its role is best described as a sufficient condition in the context at hand and its potential role as a weak necessary condition may not play a major role in practice. 6.3 practical implications considering the sufficient solutions m1 through m3, it appears plausible that utilitarian and hedonic aspects are important for the field of wct. however, the empirically most relevant solution m3 suggests a focus on social influence, i.e., what tips the scale may be the agreement between prospective users on which application to implement. in an industry with highly similar tools, such as wct, this means that vendors need to leverage their first-mover advantage and prioritize user count over financial gain in order to gain market traction. commonly, wct applications offer free versions that can be upgraded (i.e., a freemium model). this strategy is consistent with the results at hand and should be strengthened by leveraging an organization’s social connections. for example, employees holding a lighthouse position, such as project managers and team leaders, may be incentivized with free premium versions of the software to expose their surroundings to a particular solution. similarly, smes may be provided scalable packages that grow in their functionality range as the business expands. here, wct vendors may focus on the accessibility of business intelligence for smes. low perceived effort was identified as a necessity from the users’ perspective. as this quality is inherent in the chosen software application, organizations seeking to implement a wct solution need to thoroughly assess their demands and available providers to find a suitable match. it appears unfeasible to select a well-known application merely for the sake of its popularity without considering the organization’s individual characteristics. further, facilitating conditions, which may be formed by the organization, have been found to be necessary for usage intentions. this demand may be met in the form of customized chatbot support (e.g., in the case of slack’s slackbot), dedicated onboarding and help depositories, and within-application documentation. however, it is possible for organizations to provide assistance outside of the wct’s digital sphere, e.g., by means of workshops and introductory australasian journal of information systems kopplin 2023, vol 27, research article a configurational view on technology acceptance 23 courses. still, these may be either conducted via the wct’s functionalities, such as videoconferencing, or recorded and made available for on-the-fly access during usage. examining use scenarios, it becomes evident that integrated platforms are not favoured for all kinds of tasks. stand-alone applications are preferred in the areas of e-mail and social media, both of which are scenarios with quite a long history of dedicated software. for tasks that require a rather high degree of specialization, as in the cases of website analytics, customer support, and documentation, participants also indicated a preference for dedicated, standalone software. interestingly, voice calls were stated to be made using phones, i.e., neither an integrated wct nor well-known applications such as skype or google hangouts. wct were found to be attractive for productivity-related tasks such as polls, file sharing, note-taking, and project management, but also for socializing. this is also consistent with the most used integrations: cloud storage and project management. while the others fit the utilitarian assessment, socializing likely corresponds to the instant messaging-based, somewhat playful environment that many wct exhibit. furthermore, various integrations (such as donut for slack) are solely dedicated to onboarding and socializing. indeed, participants’ evaluation of hedonic aspects suggests the relevance of a joyful component, which may be implemented in the form of a dedicated third-party integration or customized chatbot information that reflects the organization’s culture. 6.4 limitations and future research as for all scientific studies, some limitations need to be addressed. first and most important, while the study’s slack workspace was continuously updated with content both by participants and supervisors and may be considered realistic in amount and type of information, respondents might have been well aware that many contributions did not demand a personal response. this may contrast real-world digital workspaces, where any content could require a reaction. further, participation was voluntary, and all respondents were students with varying degrees of work experience. concerning use scenarios, these were compiled drawing on earlier research, practitioner data sources such as blogs, a focus group consisting of eight individuals, and followed the lines of ratneshwar and shocker (1991). however, these scenarios may differ across contexts and environmental settings and may only serve as a first indication. further, all participants were located in germany, and the investigation took place before the impact of the covid-19 pandemic. overall, bi was assessed rather indifferent. this poses the question of why users might prefer other solutions even when confronted with an all-around tool. findings in related areas, such as provided by amoroso and lim (2017), gefen (2003), venkatesh, thong, and xu (2012), and polites and karahanna (2012) propose that mere habit may be highly influential. future research might focus on the inter-device nature of wct that allows a ubiquity of work processes and work-related communication. this “’always available’ work culture” (jarrahi et al., 2017, p. 570) of consumerization blurs the boundaries between work and private settings (mazmanian et al., 2013). extant research suggests that this kind of interference elicits feelings of technology-induced stress (ayyagari et al., 2011; tarafdar et al., 2007; weil & rosen, 1997). consequently, it is essential to understand the nature of these stress effects, and particularly examine whether they are positive (i.e., eustress) or negative (i.e., distress) in their impact on employees (tarafdar et al., 2019). australasian journal of information systems kopplin 2023, vol 27, research article a configurational view on technology acceptance 24 the study was conducted shortly before the outbreak of the covid-19 pandemic. this p.emic strikingly displayed the importance of remote work and, thus, also emphasizes the role of wct. future research might investigate changes in user perception and organizational commitment to this type of tool in the post-covid period. for example, organizations increasing their involvement in digital collaboration are expected to invest in facilitating conditions, and the achieved normality of home-office and remote work during the pandemic – although rather forced – may have altered user perceptions of collaboration tools. thus, it appears important for the field to examine whether these impacts yield a favourable or an adverse effect on future wct usage. 7 conclusion using the established utaut model with a supplementary measure for hedonic motivation as a framework, both fsqca and nca corroborated the relevance of the model’s variable set from a novel perspective. on a large scale, i.e., when the full range of bi is considered, fc yields the most substantial impact, followed by ee. this result stresses the paramountcy of creating a supportive, technologically compatible software environment when introducing a wct. this environment may comprise standardized manuals and help guides that users can access at any time, as well as dedicated personnel administrating the workspace. particularly the vast amount of available third-party integrations requires organizations to establish rules and guidelines on mandatory and voluntary expansions, the degree of autonomy each enduser has when it comes to personalization, and the mode of operation these integrations require. for many integrations are connectors to full-size stand-alone applications, individuals should also be informed about whether the standard mode of operation is from within the wct or direct access to the application to reduce friction and prevent misunderstandings. references alias, e. s., mukhtar, m., & jenal, r, (2017). embedding key user values for the adoption of unified communication and collaboration service. in proceedings of the 2017 6th international conference on electrical engineering and informatics (iceei), 1-4. amoroso, d., & lim, r. (2017). the mediating effects of habit on continuance intention. international journal of information management, 37, 693-702. apple (2020) facetime. https://apps.apple.com/de/app/facetime/id1110145091 ayyagari, r., grover, v., & purvis, r. (2011). technostress: technological antecedents and implications. mis quarterly, 35(4), 831-858. bacon, e., williams, m. d., & davies, g. h. (2019). recipes for success: conditions for knowledge transfer across open innovation ecosystems. international journal of information management, 49, 377-387. bajwa, d. s., lewis, l. f., pervan, g., & lai, v. s. (2005). the adoption and use of collaboration information technologies: international comparisons. journal of information technology, 20, 130-140. benke, i., knierim, m. t., & maedche, a. (2020). chatbot-based emotion management for distributed teams: a participatory design study. in proceedings of the acm on humancomputer interaction, 4, 1-30. australasian journal of information systems kopplin 2023, vol 27, research article a configurational view on technology acceptance 25 bol, d., luppi, f. (2013). confronting theories based on necessary relations: making the best of qca possibilities. political research quarterly, 66, 205-210. bloom, n., garicano, l., sadun, r., & van reenen, j. (2014). the distinct effects of information technology and communication technology on firm organization. management science, 60, 2859-2885. braumoeller, b. f., & goertz, g. (2000). the methodology of necessary conditions. american journal of political science, 844-858. brown, s. a., dennis, a. r., & venkatesh, v. (2010). predicting collaboration technology use: integrating technology adoption and collaboration research. journal of management information systems, 27, 9-54. cabrera, a., & cabrera, e. f. (2002). knowledge-sharing dilemmas. organization studies, 23, 687-710. chung, k.-s., & shin, y.-m. (2011). service components for unified communication and collaboration of an soa-based converged service platform, in stephanidis c (ed) hci international 2011 – posters’ extended abstracts, pp. 491-495. circuit (2020). www.circuit.com. conger, s. (2020). the impact of the covid-19 pandemic on information systems management. information systems management, 37, 327-331. cooper, r. g. (2019). the drivers of success in new-product development. industrial marketing management, 76, 36-47. cooper, r. g., & kleinschmidt, e. j. (1987). new products: what separates winners from losers? journal of product innovation management, 4, 169-184. dang-pham, d., hoang, a.-p., vo, d.-t., & kautz, k. (2022). digital kaizen: an approach to digital transformation. australasian journal of information systems, 26, selected papers from acis 2020. desanctis, g., & gallupe, r. b. (1987). a foundation for the study of group decision support systems. management science, 33, 589-609. desanctis, g., wright, m., & jiang, l. (2001). building a global learning community. communications of the acm, 44, 80-82. dey, b. l., al-karaghouli, w., & muhammad, s. s. (2020). adoption, adaptation, use and impact of information systems during pandemic time and beyond: research and managerial implications. information systems management, 37, 298-302. dul, j. (2016a). identifying single necessary conditions with nca and fsqca. journal of business research, 69, 1516-1523. dul, j. (2016b). necessary condition analysis (nca) logic and methodology of “necessary but not sufficient” causality. organizational research methods, 19, 10-52. fornell, c., & larcker, d. f. (1981). evaluating structural equation models with unobservable variables and measurement error. journal of marketing research, 18, 39-50. gartner (2018). market guide for workstream collaboration. australasian journal of information systems kopplin 2023, vol 27, research article a configurational view on technology acceptance 26 gefen, d. (2003) tam or just plain habit: a look at experienced online shoppers. journal of organiza-tional and end user computing, 15(3), 1-13. gewald, h., wang, x., weeger, a., raisinghani, m. s., grant, g., sanchez, o., & pittayachawan, s. (2017). millennials’ attitudes toward it consumerization in the workplace. communications of the acm, 60, 62-69. hacker, j., johnson, m., saunders, c., & thayer, a. l. (2019). trust in virtual teams: a multidisciplinary review and integration. australasian journal of information systems, 23. hair, j. f., ringle, c. m., & sarstedt, m. (2011). pls-sem: indeed a silver bullet. journal of marketing theory and practice, 19, 139-152. hair, j. f., risher, j. j., sarstedt, m., & ringle, c. m. (2019). when to use and how to report the results of pls-sem. european business review, 31, 2-24. harris, j., ives, b., & junglas, i. (2012). it consumerization: when gadgets turn into enterprise it tools. mis quarterly executive, 11(3), 99-112. hendriks, p. (1999). why share knowledge? the influence of ict on the motivation for knowledge sharing. knowledge and process management, 6, 91-100. henseler, j., ringle, c. m., & sarstedt, m. (2015). a new criterion for assessing discriminant validity in variance-based structural equation modeling. journal of the academy of marketing science, 43, 115-135. hinds, p., & kiesler, s. (1995). communication across boundaries: work, structure, and use of communication technologies in a large organization. organization science, 6, 373-393. im, i., hong, s., & kang, m. s. (2011). an international comparison of technology adoption: testing the utaut model. information & management, 48, 1-8. inkpen, a. c. (1996). creating knowledge through collaboration. california management review, 39, 123-140. jarrahi, m. h., crowston, k., bondar, k., & katzy, b. (2017). a pragmatic approach to managing enterprise it infrastructures in the era of consumerization and individualization of it. international journal of information management, 37, 566-575. king, w. r., & he, j. (2006). a meta-analysis of the technology acceptance model. information & management, 43, 740-755. koch, h., yan, j., zhang, s., milic, n., & curry, p. (2019). how consumer technology is changing the it function: a multi-case study of three fortune 500 companies. information systems management, 36, 336-349. kopplin, c., & baier, d. (2020). how to construct an ideal collaboration tool for coworking spaces : a sp-cbc application. archives of data science series a, 6, 1-17. kopplin, c. s., gantert, t. m., & maier, j. v. (2021). acceptance of matchmaking tools in coworking spaces: an extended perspective. review of managerial science, 16(6), 19111943. krogslund, c., choi, d. d., & poertner, m. (2015). fuzzy sets on shaky ground: parameter sensitivity and confirmation bias in fsqca. political analysis, 23, 21-41. australasian journal of information systems kopplin 2023, vol 27, research article a configurational view on technology acceptance 27 kryvinska, n., auer, l., & strauss, c. (2009). the place and value of soa in building 2.0generation enterprise unified vs. ubiquitous communication and collaboration platform, in lloret mauri j (ed) third international conference on mobile ubiquitous computing, systems, services and technologies, pp. 305-310. ladd, d. a., datta, a., sarker, s., & yu, y. (2010). trends in mobile computing within the is discipline: a ten-year retrospective. communication of association of information systems, 27, 285-316. lai, j., mitchell, s., viveros, m., wood, d., & lee, k. m. (2002). ubiquitous access to unified messaging: a study of usability and the use of pervasive computing. international journal of human-computer interaction, 14, 385-404. lansmann, s., schallenmüller, s., & rigby, m. (2019). teams everywhere–investigating the impact of microsoft teams on knowledge worker. in proceedings of the pre-icis. lou, h., chau, p. y. k., & li, d. (2005). understanding individual adoption of instant messaging: an empirical investigation. journal of the association for information systems, 6, 102-129. maggetti, m., & levi-faur, d. (2013). dealing with errors in qca. political research quarterly, 66, 198-204. marx, a. (2010). crisp-set qualitative comparative analysis (csqca) and model specification: benchmarks for future csqca applications. international journal of multiple research approaches, 4, 138-158. mazmanian, m., orlikowski, w. j., & yates, j. (2013). the autonomy paradox: the implications of mobile email devices for knowledge professionals. organization science, 24, 1337-1357. mendel, j. m., & korjani, m. m. (2012). charles ragin’s fuzzy set qualitative comparative analysis (fsqca) used for linguistic summarizations. information sciences, 202, 1-23. microsoft (2020) microsoft teams. https://www.microsoft.com/de-de/microsoft365/microsoft-teams/group-chat-software novet, j. (2019). microsoft says its teams app is bigger than slack and growing faster, https://www.cnbc.com/2019/07/11/microsoft-teams-passes-slack-in-daily-users-rajeshjha-explains-why.html. nunamaker, j. f., dennis, a. r., valacich, j. s., vogel, d., & george, j. f. (1991). electronic meeting systems. communications of the acm, 34, 40-61. ordanini, a., parasuraman, a., & rubera, g. (2014). when the recipe is more important than the ingredients. journal of service research, 17, 134-149. pal, d., & vanijja, v. (2020). perceived usability evaluation of microsoft teams as an online learning platform during covid-19 using system usability scale and technology acceptance model in india. children and youth services review, 119, 105535. pappas, i. o. (2018). user experience in personalized online shopping: a fuzzy-set analysis. european journal of marketing, 52, 1679-1703. australasian journal of information systems kopplin 2023, vol 27, research article a configurational view on technology acceptance 28 pappas, i. o., & woodside, a. g. (2021). fuzzy-set qualitative comparative analysis (fsqca): guidelines for research practice in information systems and marketing. international journal of information management, 58, 102310. park, y. k., fiss, p. c., & el sawy, o. a. (2020). theorizing the multiplicity of digital phenomena: the ecology of configurations, causal recipes, and guidelines for applying fsqca. mis quarterly, 44, 1493-1520. pearlson, k. e., saunders, c., & galletta, d. f. (2016). managing and using information systems: a strategic approach. wiley. polites, g. l., & karahanna, e. (2012). shackled to the status quo: the inhibiting effects of incumbent system habit, switching costs, and inertia on new system acceptance. mis quarterly, 36, 21-42. ragin, c. c. (1987). the comparative method: moving beyond qualitative and quantitative strategies. university of california press. ragin, c. c. (2006). set relations in social research: evaluating their consistency and coverage. political analysis, 14(3), 291-310. ragin, c. c. (2009). redesigning social inquiry: fuzzy sets and beyond. university of chicago press. ratneshwar, s., & shocker, a. d. (1991.) substitution in use and the role of usage context in product category structures. journal of marketing research, 28(3), 281-295. reynolds, j. (2018). workstream collaboration: what it is & why you need it, https://mattermost.com/blog/why-workstream-collaboration/. riemer, k., & frößler, f. (2007). introducing real-time collaboration systems: development of a conceptual scheme and research directions. communications of the association for information systems, 20(1), 17. riemer, k., & taking, s. (2009). unified communications. business & information systems engineering, 1, 326-330. rihoux, b., & ragin, c. c. (2009). configurational comparative methods: qualitative comparative analysis (qca) and related techniques. sage publications. schneider, c. q., & wagemann, c. (2010). qualitative comparative analysis (qca) and fuzzysets: agenda for a research approach and a data analysis technique. comparative sociology, 9, 376-396. schneider, c. q., & wagemann, c. (2012). set-theoretic methods for the social sciences: a guide to qualitative comparative analysis. cambridge university press schneider, c. q., & wagemann, c. (2013). doing justice to logical remainders in qca: moving beyond the standard analysis. political research quarterly, 66, 211-220. schoeneborn, d., kuhn, t. r., & kärreman, d. (2019). the communicative constitution of organization, organizing, and organizationality. organization studies, 40, 475-496. slack (2020). https://slack.com/intl/de-de/ stoeckli, e., dremel, c., uebernickel, f., & brenner, w. (2020). how affordances of chatbots cross the chasm between social and traditional enterprise systems. electronic markets, 30, 369-403. australasian journal of information systems kopplin 2023, vol 27, research article a configurational view on technology acceptance 29 szajna, b. (1996). empirical evaluation of the revised technology acceptance model. management science, 42, 85-92. tarafdar, m., cooper, c. l., & stich, j.-f. (2019). the technostress trifecta techno eustress, techno distress and design: theoretical directions and an agenda for research. information systems journal, 29, 6-42. tarafdar, m., maier, c., laumer, s., & weitzel, t. (2020). explaining the link between technostress and technology addiction for social networking sites: a study of distraction as a coping behavior. information systems journal, 30, 96-124. tarafdar, m., tu, q., ragu-nathan, b. s., & ragu-nathan, t. s. (2007). the impact of technostress on role stress and productivity. journal of management information systems, 24, 301-328. thomann, e., & maggetti, m. (2020). designing research with qualitative comparative analysis (qca): approaches, challenges, and tools. sociological methods & research, 49, 356-386. tjosvold, d., & tsao, y. (1989). productive organizational collaboration: the role of values and cooperation. journal of organizational behavior, 10(2), 189-195. toxtli, c., monroy-hernández, a., & cranshaw, j. (2018). understanding chatbot-mediated task management. in proceedings of the 2018 chi conference on human factors in computing systems. tuma, a. (1998). configuration and coordination of virtual production networks. international journal of production economics, 56, 641-648. unify (2020) circuit is coming to apple carplay. https://unify.com/en/blog/circuit-sprint-118 unify square (2019). defining and understanding workstream collaboration, https://www.unifysquare.com/blog/what-is-workstream-collaboration/. venkatesh, v., morris, m. g., davis, g. b., & davis, f. d. (2003). user acceptance of information technology: toward a unified view. mis quarterly, 27(3), 425-478. venkatesh, v., thong, j. y. l., chan, f. k. y., hu, p. j.-h., & brown, s. a. (2011). extending the two-stage information systems continuance model: incorporating utaut predictors and the role of context. information systems journal, 21, 527-555. venkatesh, v., thong, j.y.l., & xu, x. (2012). consumer acceptance and use of information technology: extending the unified theory of acceptance and use of technology. mis quarterly, 36(1), 157-178. venkatesh, v., thong, j. y. l., & xu, x. (2016). unified theory of acceptance and use of technology: a synthesis and the road ahead. journal of the association for information systems, 17, 328-376. veri, f. (2018). coverage in fuzzy set qualitative comparative analysis (fsqca): a new fuzzy proposition for describing empirical relevance. comparative sociology,17, 133-158. verma, a., giri, a. k., & debata, b. (2022). leapfrogging into knowledge community: information and communication technology for human development. australasian journal of information systems, 26. australasian journal of information systems kopplin 2023, vol 27, research article a configurational view on technology acceptance 30 vis, b., & dul, j. (2018). analyzing relationships of necessity not just in kind but also in degree: complementing fsqca with nca. sociological methods & research, 47, 872-899. wee, m., scheepers, h., & tian, x. (2022). understanding the processes of how small and medium enterprises derive value from business intelligence and analytics. australasian journal of information systems, 26. weil, m. m., & rosen, l. d. (1997). technostress: coping with technology@ work@ home@ play. wiley. woodside, a. g. (2013). moving beyond multiple regression analysis to algorithms: calling for adoption of a paradigm shift from symmetric to asymmetric thinking in data analysis and crafting theory. journal of business research, 66, 463-472. wu, j., & lu, x. (2013). effects of extrinsic and intrinsic motivators on using utilitarian, hedonic, and dual-purposed information systems: a meta-analysis. journal of the association for information systems, 14, 1. zhou, t., lu, y., & wang, b. (2010). integrating ttf and utaut to explain mobile banking user adoption. computers in human behavior, 26, 760-767. appendix a items (retranslated from german) and outer loadings construct items loadings reference performance expectancy pe1. i find slack useful in my daily life. 0.808 venkatesh et al. (2003), venkatesh et al. (2012) pe2. using slack increases my chances of achieving things that are important to me. 0.843 pe3. using slack helps me accomplish things more quickly. 0.884 pe4. using slack increases my productivity. 0.811 effort expectancy ee1. learning how to use slack is easy for me. 0.876 venkatesh et al. (2003), venkatesh et al. (2012) ee2. my interaction with slack is clear and understandable. 0.866 ee3. i find slack easy to use. 0.883 ee4. it is easy for me to become skillful at using slack. 0.864 social influence si1. people who are important to me think that i should use slack. 0.857 venkatesh et al. (2003), venkatesh et al. (2012) si2. people who influence my behavior think that i should use slack. 0.915 si3. people whose opinions i value prefer that i use slack. 0.908 si4. colleagues think that i should use slack. 0.769 facilitating conditions fc1. i have the resources necessary to use slack. 0.706 venkatesh et al. (2003), venkatesh et al. (2012) fc2. i have the knowledge necessary to use slack. 0.799 australasian journal of information systems kopplin 2023, vol 27, research article a configurational view on technology acceptance 31 fc3. slack is compatible with other technologies i use. 0.744 fc4. i can get help from others when i have difficulties using slack 0.577 (dropped) hedonic motivation hm1. using slack is fun. 0.924 venkatesh et al. (2012) hm2. using slack is enjoyable. 0.912 hm3. using slack is very entertaining. 0.857 in3. i feel constantly connected to work. 0.447 behavioral intention bi1. i intend to continue using slack in the future. 0.947 venkatesh et al. (2012) bi2. i will always try to use slack in my daily life. 0.926 bi3. i plan to continue to use slack frequently. 0.952 australasian journal of information systems kopplin 2023, vol 27, research article a configurational view on technology acceptance 32 appendix b indicator covariance matrix australasian journal of information systems kopplin 2023, vol 27, research article a configurational view on technology acceptance 33 appendix c fornell-larcker assessment appendix d heterotrait-monotrait ratio bi ee fc hm pe si bi ee 0.346 fc 0.449 0.738 hm 0.691 0.452 0.515 pe 0.730 0.356 0.460 0.746 si 0.581 0.294 0.381 0.607 0.808 appendix e xy plots, values indicate case ids bi ~bi pe ~pe bi ee fc hm pe si bi 0.942 ee 0.320 0.872 fc 0.408 0.538 0.749 hm 0.630 0.406 0.419 0.898 pe 0.665 0.316 0.408 0.659 0.836 si 0.536 0.265 0.357 0.538 0.707 0.863 australasian journal of information systems kopplin 2023, vol 27, research article a configurational view on technology acceptance 34 ee ~ee hm ~hm copyright: © 2023 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v27i0.4043 http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 related work 2.1 workstream collaboration tools 2.2 technology acceptance of workstream collaboration tools 3 conceptual model 4 methodology 4.1 sample strategy and training 4.2 descriptive statistics 4.3 measurement evaluation 5 findings 5.1 necessary conditions analysis using fsqca and nca 5.2 truth table construction and minimization 5.3 inclusion of gender 5.4 predictive validity and sensitivity analysis 5.5 specific propositions 5.6 use behavior 6 discussion 6.1 summary 6.2 theoretical implications 6.3 practical implications 6.4 limitations and future research 7 conclusion paper title (paper title style) australasian journal of information systems kutay 2021, vol 25, research on indigenous use of knowledge elicitation information and communication technologies 1 knowledge elicitation with aboriginal australian communities cat kutay charles darwin university, darwin, australia cat.kutay@cdu.edu.au abstract this paper provides techniques for engagement and data collection in researching with aboriginal australian cultures, acknowledging significant differences in forms of communication and usage of information systems to the mainstream australian culture. an approach to trust in relationships is developed by interpreting cultural aspects arising from the diverse relationships to technology developed by aboriginal users. this work uses the existing honeycomb model for social media development as a base framework for collaborative web systems and online knowledge sharing in the indigenous domain. we present a series of product development research projects based in universities in nsw australia, in particular user experience studies, to explain the relationship between the researchers and users and the products that are created in terms of the model. some concepts and processes fundamental to engagement with aboriginal australian communities in the supply of appropriate information sharing technology are discussed in this context. for if aboriginal people are to engage in is development, we are sharing the knowledge or the culture that is embedded in the technology which can have detrimental effects. either we are asking them to enter the culture that created the technology, that is assimilate, or we use engagement in design to change that technology to suit the culture. keywords: aboriginal knowledge systems, protocols for engagement, appropriate information technology 1 introduction research into aboriginal requirements for information technologies is crucial for developing resources suitable for use by aboriginal australian communities as a means to share information. the requirements for such knowledge systems also considers the effect on community wellbeing of incorrect sharing, and by engaging for change we can reverse some of the detrimental effects of it as it has been developed to date (du, 2017). it is challenging to design appropriate technologies based on the requirements of a community culture that has been alienated from the culture where the original technology was developed. for aboriginal communities in urban and remote areas, we work with different languages and world meanings. the historical dispossession of land, forced removal and the institutionalisation of aboriginal people in australia has reduced aboriginal engagement, or willingness to engage in online systems (radoll, 2015). this paper considers aspects of the development of consultation processes with the community over many years. the significant features of research that arise in this context are around the relationship of users to each other and to any knowledge shared; and the methods used in knowledge sharing as a community narrative. it is this communal approach to knowledge that create particular differences. within a community focus on knowledge sharing, we consider the values and australasian journal of information systems kutay 2021, vol 25, research on indigenous use of knowledge elicitation information and communication technologies 2 relations around sharing information and how to promote engaging in a dialogical process with the community (iversen et al., 2012). while we consider how to negotiate knowledge sharing with aboriginal communities, it is helpful to understand why there are different protocols for this process, to understand how these protocols translate into suitable actions and sustainable relationships across cultures (ormond & slogget, 2008). for example, integrating is with another culture involves twoway learning with the sentiment of “ngapartji: we give, you give”. 2 background indigenist research is research by indigenous australians whose primary informants are indigenous australians and whose goals are to serve and inform the indigenous struggle for equality. (rigney 2001). for aboriginal communities there is an emphasis on knowledge negotiation prior to public sharing, in a form of oral peer review. a person will consider the significance of a new factor in the environment then stories about this will be discussed and shared for a while before they are considered part of the established knowledge of the region (kutay, 2019). supplying information systems to support cultural ways of knowing and knowledge sharing can conflict with such cultural and social values. to deliver technology for users to control and improve their own wellbeing we must deal with these conflicts, particularly to develop systems that support long term sustainability and economic benefits for communities, not just individual wellbeing. there are issues for indigenous cultures relating to the representation of traditional knowledge systems in new interfaces (verran & christie, 2007). this requires an analysis of shared meaning amongst users from the community and the dominant culture. for example, considering unbalanced power, problems of trust and the development of relations with the users (winschiers-theophilus et al., 2012), but also new aspects around relationships and how these can be incorporated into online sharing. also, there is a tendency in research with other cultures for researchers to compensate for differences they consider deficiencies that need to be countered rather than looking at opportunities to create new and novel networking systems, yet this latter aspect is an important aspiration in is (sidorova et al., 2008). the traditional techniques of socialisation of tacit knowledge can inform and enhance our present online systems for education and training (kutay & aurum, 2007). the difference in standpoint from an aboriginal and non-aboriginal perspective will vary according to the domain of investigation. we look at the variation of ideas in the knowledge sharing and collaboration processes and how to incorporate these ideas in ongoing research. these systems focused on knowledge sharing in the form of stories as audio or videos, as well as images, text and documents promoting events. we use web sites and mobile apps, but also require data management to support protocols and included immersive game environments. this work is the result of years of design and development work by aboriginal and nonaboriginal researchers in urban and rural nsw aboriginal communities. australasian journal of information systems kutay 2021, vol 25, research on indigenous use of knowledge elicitation information and communication technologies 3 3 literature review strong cultural identity and community and family connections, which can be enhanced through social media, are linked to improved educational and health outcomes (rice et al., 2016) social network growth online can mirror relationships that are strong in aboriginal families and communities. however, there are clearly cultural or institutional issues that affect the take up of any digital resources. the development of technology should always consider the target audience and their world views, particularly when aiming to support psychological wellbeing, since this itself is a construct that has different representations in the epistemology of different cultures (goldfinch et al., 2016). in terms of wellbeing there are the two-fold benefits for information systems development, of acknowledging and crediting aboriginal knowledge; and providing the platforms for sharing stories to create greater identity and value with the community. 3.1 indigenous relations with information systems it is through developing a relationship to knowledge and valuing the means by which we gain this knowledge, that indigenous people gain insight and claim a right to express this knowledge (wilson, 2008). from previous research into aboriginal use of the internet (radoll, 2010; dyson & underwood, 2006; kutay, 2010; akama et al., 2016) it has been found that it is important to consider aboriginal australians’ relationship to online media through various aspects from how indigenous people relate to knowledge. the areas extracted from previous research can be summarised as: • control and identity: the general lack of experience in it reduces the possibilities envisaged by the community in designing with this medium. this creates a lack of understanding of what is possible with technology to express community ideals and what are the risks. however, an online presence is considered part of aboriginal expression of identity. (rice et al., 2016) • communication: aboriginal english or aboriginal oral languages are usually the peoples’ first language, so audio or visual formats can be more suitable than text. also, a language carries a mode of thought, and existing cultural ontologies that are used online create boundaries in the interface, the choice of content matter, and how to link material. (eades, 2013) • collaboration and ownership: the shared governance used in aboriginal communities is not easily replicated in database systems. a suitable is system requires control to be divulged more to the users. many of the issues relate to ownership, where the community can feel they understand and control the system enough to feel trust and empowerment in using the system. (radoll, 2010) • access and transferability: build systems for the majority of users who are limited to mobile internet access and incorporate the needs of disabled users who are well integrated into the community. to improve access, we also look at how resources developed for one group may not be relevant for other groups in terms of interface design, preferred functionality or accessible data, however patterns in design can be extracted (dyson & underwood, 2006). australasian journal of information systems kutay 2021, vol 25, research on indigenous use of knowledge elicitation information and communication technologies 4 • immersive: increased value of immersive online environments such as vr games to provide a knowledge interface, given their relation to the holistic approach of aboriginal ceremony used in teaching. also, this helps counter the fact that online communication removes knowledge from its physical environment which is a fundamental part of knowledge transmission in aboriginal society (kutay, 2010). • relationship to knowledge: provide different access to knowledge based on these relationships; and ensure the knowledge derived through the development process is part of the final product; and being part of this process allows aboriginal users to feel ownership of the product. also acknowledging the difference between different communities’ relationships requires flexibility in the knowledge management between versions of a system (akama et al., 2016) • trust relationships: the issues of trust of both the online systems and of relations with non-aboriginal viewers when sharing knowledge is well studied. aboriginal protocols are based around contributors' relationships to each other and provide a way to trust information that is shared, and these need to be emulated in some form online. the need for trust includes: establishing relationships with those sharing knowledge through online tools; ensuring that knowledge will not be misused or misinterpreted out of context by those without relationship to the knowledge; maintaining protocols for knowledge sharing through the online interface; and supporting normal community methods of verifying oral knowledge through collaboration (povinelli, 1993). • novel approaches: users with novel ideas for what could be done creates innovative development ideas beyond the scope of these projects. locating the situation through different cultural framing is part of finding the solution and forms a continuous iterative development in co-creation (soro et al., 2016). in the gathering of community requirements for a new system, the is designers often act as both facilitators and change agents, which is contentious (winschiers-theophilus et al., 2012). however, the process of agile development does facilitate the user to understand various prototypes which helps them assess the options available and realise the flexibility of ict’s. digital literacy is often not an issue in community (prayaga et al., 2017) so we consider other factors that have come out of our projects. many of the factors presented in this study contribute to both community engagement, important for the cross-cultural technology development; and ensure aboriginal people see benefit from projects in community terms. what is required is an understanding of the significance of any information given (e.g., a painting to be used on the web or mobile app) as spiritual and linked to the wellbeing of the community and country, not just the artist or author. 3.2 honeycomb model the honeycomb model (kietzmann et al., 2011) has provided a tool to analyse key features in new social media systems for their effectiveness. the concepts in this model were found to map well to all the issues above which have been located in research into aboriginal relationships with internet sharing. social media is an environment that considers people in relationships, an aspect that is significant in aboriginal world views. australasian journal of information systems kutay 2021, vol 25, research on indigenous use of knowledge elicitation information and communication technologies 5 this process is best understood in terms of socialisation of knowledge (nonaka & takeuchi, 1995) often initially through teams of developers and community members, for example in student run projects (c.f. ayala et al., 2015). also, the aim of these projects is for more broader socialisation of this knowledge, either within community or externally, through internet media. considering the honeycomb framework is useful for analysing socialisation on the web in specific platforms, we looked at how this applies in past experiences and the aspects important for the future development of systems for aboriginal community sharing. the model has been used to understand significant factors in developing social media and gaining benefits from this type of online community for businesses and other users (silva et al., 2020). while this paper is about more conventional websites such as blogs and content management systems (cms) sites, it is about how we extend these to provide a social environment for collective knowledge to be collected and presented. figure 1. the honeycomb framework of social media (kietzmann et al., 2011 p. 243) 3.3 requirements gathering in the indigenous space ethnographic research tools and narrative methods have been used in software requirements gathering for many years and have aligned well with the storytelling approach to knowledge sharing used by aboriginal groups (lukosch et al., 2009; george et al., 2011). new aspects also arise to be considered, such as humour, dance and music to be incorporated in systems, but also new interpretations on imagery (excluding “the other”) and relationship between the users. novel requirements such as limiting access to knowledge according to a user’s status and their prior knowledge (radoll, 2015) are important in aboriginal society and should be maintained online to develop community trust. australasian journal of information systems kutay 2021, vol 25, research on indigenous use of knowledge elicitation information and communication technologies 6 existing models of social media systems used in information systems research provide a unique opportunity to examine aboriginal use of these systems as a form of sharing and developing public relationships with aboriginal knowledge. the honeycomb model (kietzmann et al., 2011) is used to evaluate group sharing sites developed within community groups (c.f. senadheera et al., 2017) as a form of limited social media. 4 projects studied the series of projects in this study as listed in table 1 and discussed below exemplifies the gradual process in creating an aboriginal knowledge sharing system. time is needed for aboriginal people to develop a relationship to the technology and system of knowledge access that has been created on the internet and develop ways to manage knowledge sharing in line with existing values and beliefs. the projects developed from collecting material for individuals to use; to providing networks between groups; to interfaces for sharing material to be used as a community narrative in teaching; and using the community material to develop games. it has been interesting that other forms of networking that have developed have been adaptations of existing technology like facebook through restricted groups, and creation of bespoke systems like keeping culture (n.d.), mukurtu (n.d.) and limba (n.d.) have developed into templates for other communities to use and adapt. type of resource content type aim online repositories stories sorted around themes such as stolen generations knowledge collection: public education through a series of oral histories around a theme search engine of reports for daa compare and combine reports on the wellbeing outcomes of projects across many different communities (kutay, 2010) knowledge collection: support workers to collect material around a theme from community reports sydney koori interagency network (skin) regions communications mashups in php, javascript and python as a template for a multi-site system (kutay, 2010) collaborative information service: for information sharing across ngo’s and depts with aboriginal workers indigenous remote media outlets to collect and distribute material community upload and video streaming through indigenous remote communications association (irca) collaborative information service: to share media to re-broadcast through satellite to all remote communities. sharing language resources to combine into worksheet for teaching combining online language resources for teaching. resources analysed from repository to show relevant material on worksheet (kutay et al, 2012), collaborative knowledge sharing: for teachers and students to learn language through oral and text repetition across different interfaces. academiccommunity sharing stories tools to allow community to link stories online into teaching resources (soro et al. 2016), altc (kutay et al., 2012) collaborative knowledge sharing: community stories linked under teaching topics for university engagement in cultural knowledge gamification of knowledge linking online material through application programming interface (api) to games and mobile apps tacit teaching; interfaces to share tacit components of knowledge and reinforce narratives. table 1. the studied projects this process of networking around information systems began through gatherings such as conferences and workshops run by indigenous groups, through projects shared at public australasian journal of information systems kutay 2021, vol 25, research on indigenous use of knowledge elicitation information and communication technologies 7 award ceremonies, and through those seeking for a solution to a problem by approaching other communities as to how they are dealing with such issues. aboriginal experience of distrust and misunderstanding merely highlight the need for this initial face-to-face component of knowledge sharing. meeting in person is the only way we can truly trust and understand the person we are talking to, as was found in the initial project with the department of aboriginal affairs in nsw (daa), when trying to share project successes, a detailed account of how the project was implemented in a specific context is required to verify if it will apply to a new context. the ongoing issues with online security merely exemplify this conundrum for online aboriginal businesses. 5 knowledge collection there are significant aspects to consider in order for information systems to support aboriginal sharing of knowledge. we need to understand the perspective of the users, and how these develop over time with growing experience and engagement. …[t]he fact that our vision of the world is a vision from somewhere, that it is inextricably based in an embodied and therefore partial perspective, …. makes us personally responsible for it. the only possible route to objectivity on this view is through collective knowledge of the specific locations of our respective visions. (suchman, 2002, p. 96) 5.1 storysharing from community the first projects were to stream stories on the stolen generation (outside youtube for security reasons) and the second to develop an interface for a web portal of community or government information on aboriginal projects at the nsw department of aboriginal affairs (daa). this site was designed as an annotation system. these were both designed for communities or community members to share experiences with other communities or individuals who may have similar issues (kutay, 2012) as well as educating the public or government workers on community concerns. the work used previous activities and experiences of aboriginal communities in story sharing and collating knowledge under themes for ceremonies (verran, 2007), as done prior to the online experience. the requirements focused on the users’ affective motive or intent, their perception or awareness, their desires and beliefs (fishbein & azjen, 1975), as well as the familiar forms of knowledge sharing, and considered how to transfer this experience to online use (verran et al., 2007; kutay, 2011). 5.2 collaborative information sharing for aboriginal workers the second two projects were to: develop an information sharing resource for government and community workers in a joint sydney koori interagency network (skin); and for remote radio and tv to share broadcast material (irca). for the joint skin groups, prior to the project this communication was done through face-toface meetings and manual distribution of emails, but this was causing duplication of information and was providing a bottleneck that was managed by a few aboriginal workers in government who were under pressure to stop spending their time on this work. however, they saw this as a vital part of their role in engaging and informing the community. australasian journal of information systems kutay 2021, vol 25, research on indigenous use of knowledge elicitation information and communication technologies 8 for remote communication groups there were many independent remote indigenous broadcasting services (ribs) stations set up in different languages, however they could often share media. while sharing the broadcast expense by combining into one network was advantageous, the sharing of stories around common themes or stories was also considered a major benefit (sawhney & ratnadeep, 2008). in both cases the final sites were well received but over time the growing experience of the network members necessitated upgrades to more modern looking sites, and skin developed sites as separate entities for each regional group. the media communication site now has extended to a live video streaming service for radio and tv shows but retained the collaboration. 5.3 collaborative knowledge sharing for the community sharing projects, we were working with community creating knowledge that was recorded specifically for the site. a long-term project run with community language nests and language centres around nsw, has been to supply online systems to provide an interface to language resources in any media. such resources are: recordings from the australian institute of aboriginal and torres strait islander studies (aiatsis, n.d.); transcriptions of these; images from aboriginal people; grammar and examples from linguistic research; and new recording from existing speakers usually as videos in situ on their own land (on country). in language sharing work we used web services across many communities and allowed users to select not just the language groups they relate to, but also the family or clan as each subgroup had different approaches to images to use and pronunciation of language. the site included systems to analyse the various material available in the repository and select for a worksheet topic, then display this on the interface. also, this has been linked to a mobile app to reinforce the oral wordlists with example sentences. a similar project sharing stories was run with a two-year federal government office of teaching and learning grant (altc). aboriginal academics were involved in a story gathering process to provide material to augment an online site for training students in the indigenous perspective on their discipline, including education, law and social policy (kutay et al., 2012). the project was to develop interactive learning scenarios based on the experiences of community members in an educational, legal or social development setting. stories were given by the community and provided personal experiences, and were combined to provide the introduction to ideas to discuss and develop in the scenarios. while the scenario scripts were written based on the ideas provided by community members, many were not generalisable in the form of community narratives. also, those developed for lecturers in one state did not represent the experience of aboriginal instructors in other states. the various histories of colonisation and the experiences in reserves or missions creates different approaches to education. 5.4 gamification of collated knowledge the recent projects have built on the material collected online and developed tools to extend these into interactive mobile apps and games as a learning support. this was mostly focused on the language teaching sites, but we are extending this to more procedural knowledge. australasian journal of information systems kutay 2021, vol 25, research on indigenous use of knowledge elicitation information and communication technologies 9 gaming strategies are also useful platforms to introduce various cultural priorities such as requiring the player to collaborate to finish (lightning runners, 2018) or to expand on earlier stories in the game in the context of the developing environmental experience of the player. the first game template has been developed using the api interface to the language learning web systems. this has been used for simple mobile app for word reinforcement through oral examples. now it is being embedded in a game where the language can be used in a context. aboriginal animators are developing the animations in the game to provide authenticity of gestures and interactions of characters. the game is based on the dharug language of sydney to embed in a 2.5d representation of that environment using the images of joseph lycett, an early convict artist who provides use with authentic representations of sydney during early invasion 6 community of practice we now look at the findings in each project relating to the aspects extracted from previous research as described in section 2, relating previous research to the honeycomb model and draw out the relevant experiences from the projects described. this relates back to the growing interest in aboriginal culture in australia and encourages the utilisation of the aboriginal community as innovative partners in it development, both for aboriginal wellbeing, but also for broader engagement in changing how we relate to technology. the aboriginal community has for a long time maintained independent networks to mainstream australia, so we look at how this practice has been developing online and how community has taken control of this social and voluntary media to create new networks of practice (wasko et al., 2004). using the honeycomb model of social media analysis, we consider how this can be expanded around aboriginal use of internet media, as a tools for requirements gathering. 6.1 identity and control the lack of community control revolves around the lack of ‘elders’ or knowledge holders in this space which makes it hard for people to move forward with certainty. this creates a lack of experimenting that developed understanding of what is possible with technology and what are the risks. this falls in the area of identity in the honeycomb model as the community does not identify with any expertise in the domain of providing requirements for a system. the way that community groups will adapt or adopt technology is important, and how to engage such groups early in control of the process to increase engagement. the concept of community or personal control of technical services is quite foreign and hence there has often been limited engagement in co-design (carroll & rosson, 2007). the starting point for each co-design was setting up the project from a community request for technology and engaging the community in this process. this involves considering communities as the project managers and taking a broader perspective of the values, norms, beliefs and assumptions which give legitimacy to engagement with information systems (mignerat & rivard, 2009; kutay, 2011). since in institutions we know that “coercive, mimetic and normative pressures are control mechanisms exerted” to constrain adaption of technology (mignerat & rivard, 2009) we look at the main areas of control to be divulged to community to enhance engagement. australasian journal of information systems kutay 2021, vol 25, research on indigenous use of knowledge elicitation information and communication technologies 10 • platform the complexity of available platform options, and the risk that some options will not be viable for long is a difficult area for community to engage in the decision process. in practice the platform has been chosen from what is available in the space (e.g., email systems, cms) and used in a mashup prototype to allow community to familiarise with the system. however, with the rapidly changing it environment, sometimes the system chosen does not survive long for re-use, particularly in the mobile app domain. • interface community can select and develop the interface design. artistic members appreciate the opportunity to produce relevant work if time and funds are available. also, this engages an understanding of the underlying ontology of the system as part of the requirements, as the values and perspective of the client are very different (tacchi & watkins, 2007). when extending into gaming, animation skills from the community are harnessed in a new and exciting ways. • data access as well as communities wishing to control access to data by specifying a user’s affiliation to communities and family or clans, we encountered designs for new forms of access such as the use of paywalls for a collation of community artefacts. 6.2 conversations the language and grammar or ontology of the site requires a redesign of our approach to knowledge networking. for a project to succeed with community we need two way sharing to develop understand of knowledge structure across cultures. we used online engagement familiar to users where aboriginal people can practise more familiar and informal forms of information sharing that are similar to prior activities (smith et al., 2000). integrating social networks processes with serious information sharing systems can introduce users with resources they both consider worthwhile, and trust to be productive by referral from the community. however, we need to understand the relationships being developed. there are two main factors that stand out in an analysis of the community relationships: existing divisions between families or clans; and involvement of all ages in knowledge projects at different levels (kutay, 2007). there is also the unique narrative form of knowledge sharing across groups, through the dreaming stories that are told from childhood. these grow with the listener to become the framework of a holistic approach to knowledge (sveiby & skuthorpe, 2006) for future understanding. 6.3 groups for the management of a social network, the existence of distinct groups within the aboriginal community needs to be acknowledged and all contributions to decision about any systems need to be considered. the skin website was always a conflictive process of requirements gathering as different centres had different needs and tended to fear that others needs would affect the way their own site was structured. while the agencies wanted to run with their own priorities, they also wanted a unified system as there was a lot of commonalities across the teams in sydney. australasian journal of information systems kutay 2021, vol 25, research on indigenous use of knowledge elicitation information and communication technologies 11 the other aspect was access between groups. the design of databases around a central administrator caused much concern for aboriginal users and the use of aboriginal knowledge protocols should allow systems to restrict who can access different artefacts (radoll, 2009). we are supplying front end systems such as mukurtu (n.d.), which allow the community to set up protocols of access. this however retains the original power of the single administrator, rather than a role-based system, which creates trust issues with users. as in the ownership over knowledge, aboriginal people expect a high level of ownership of and engagement with technology. this process requires two features in a technology: the technology fits the culture, i.e., it is appropriate and hence useable; and the community is engaged in the design or maintenance of the technology in some way. then we are embedding the new cultural knowledge in that it, the knowledge becomes part of the artefact, and so aboriginal people can consider they own the product. 6.4 sharing access is an issue about the ability to connect and share material, a two-way process. when developing for mobile use there is a desire to provide cross-platform systems due to the small community of users being catered for. however, to store material from users a native system or good internet access at all times is required, so browser-based technologies on mobile were not feasible at the start of this work. now with progressive web apps (pwa) and background syncing, the sharing of data from mobiles, in fact the authoring of online material from any device, is much improved. the value being given to the user is increasing. apps such as questagame which is a citizen science project, have been taken up by indigenous groups looking to promote sustainability through education in environment across all ages, and this desire to share online has increased during the coronavirus pandemic. due to the small market for the applications developed with aboriginal communities, template systems are being promoted to allow communities to adapt the interface for a specific use while retaining functionality and storage that is managed and costed in common (pradhan et al., 2018). these designs require an analysis of cases across many communities (and indigenous enterprises) to consider the common features of the model to design the template and ensure the functionality covers all system types. this has been difficult in the conflicting environment of government funding for the development of aboriginal applications, but some departments are starting to realise that many grants are going to similar projects and some cohesion could improve the production of resources (e.g., language mobile apps). 6.5 presence for aboriginal users in the system the issue of presence is not so much who is present, as people live within a strong community outside social media, it is more around the ability to immerse in the environment of the knowledge being shared, so that it is true to the time and place that is being recorded. existing technology for developing games and augmented reality systems provides opportunity to develop more authentic environments for knowledge sharing that will create trust amongst more traditional indigenous groups, such as the kakadu elders who share their australasian journal of information systems kutay 2021, vol 25, research on indigenous use of knowledge elicitation information and communication technologies 12 stories with the young in a new augmented reality project (indigital, n.d.) where the story is present with the background of an existing artwork. aboriginal knowledge sharing practices are based in the country where the knowledge is from and much of this relationship can be maintained through existing virtual and augmented reality technology (kutay, 2018). also, the use of storytelling techniques to link material around common themes relates to aboriginal knowledge sharing practises. in this way a more holistic understanding of knowledge can be taught through community collections of stories. 6.6 reputation the issue that arises over trust and security is over who you are sharing the information with. it is important to know who is present and what can be shared. the reputation of a person sharing knowledge is based on their relationship to the knowledge being shared, either through aboriginal law or through personal connection (povinelli, 1993) on whom they can share this knowledge relies on similar aspects of relationship (kutay, 2021). given the recognised conflict in security between information sharing and information protection we found it important to involve community in being able to set up the security and access systems required. the link between learning a system and uptake of is exist, but particularly security issues have an effect on user engagement (wasko et al., 2004). online systems like limba (n.d.) provide a system of communities and protocols that can be selfselected, or community imposed, providing some autonomy in this learning process. 6.7 relationships developing relationships online is not just between people but with the knowledge that is shared. this may be around land where the material is from, the status of the provider, etc. but there is also the relationship to the technology itself, in that community will not be engaged if there is not a cultural expression within the requirements to place the material in a suitable context. time is required to gather user stories and suitable imagery to provide a visual and intuitive interface, in particular there is a need for suitable icons that represent aboriginal priorities. similar work for the penan language project has shown that such visual cues will vary across generations, as well as from other factors (zaman & winschiers-theophilus, 2015). the interface will be the environment in which this knowledge is now presented. 6.8 beyond the aspect that the honeycomb model does not go into is the opportunities for the users to innovate and advance the social media network over time. since these projects were carried out there has been further developments, such as growing aboriginal use of social media (e.g., the shared twitter handle indigenousx) and the rapid uptake of mobile technology shows a skill in innovation and technology (such as indigital, n.d.). it is the role of those who study information systems to observe and credit the new approaches being developed. australasian journal of information systems kutay 2021, vol 25, research on indigenous use of knowledge elicitation information and communication technologies 13 figure 2. honeycomb framework for embedding aboriginal australian community in design 7 acceptance model for knowledge management when sharing aboriginal knowledge, we defer to the country or territory of the speakers. knowledge about country is embodied in that land. aboriginal people in reviving language see this as reclaiming it from country, that the knowledge still resides there and only being relearnt. this is partly as much of the knowledge is about country but also as the knowledge sharing system, the dreamtime stories, are very much based in country, using landscape features to ‘trigger’ an aspect of the story to embellish to deepen the knowledge of what that feature represents e.g. ‘this is where the flying fox ate the lilly pilly’ will link to further material about the plants the flying fox eats, where they grow and the seasons of prolific food, and often the stories will locate in place and season by the stars that are overhead at the end (kutay, 2019). these projects are a way of highlighting the areas ahead in this process. the areas of system requirements can expand to consider consensus for social good; technology acceptance models based on cultural beliefs, attitudes, intentions, and behaviour for cross-cultural development; the study of social media use amongst community members will help understand how social relationship networks form in general; and knowledge management strategies developed in community will provide ideas for novel interfaces for elearning and online knowledge sharing enterprises. we present here some further issues for consideration around a knowledge sharing model for aboriginal australian information systems development projects. these provide examples of the honeycomb model in practice. australasian journal of information systems kutay 2021, vol 25, research on indigenous use of knowledge elicitation information and communication technologies 14 7.1 sharing in different languages the process of sharing technology involves training in the use and creation of this technology, so the language and the knowledge travel with the artefact as it is traded (pascoe, 2014). aboriginal people talk about a story or piece of knowledge being ‘in’ an artefact, that a piece of the person creating a tool or story remains in that artefact. hence for aboriginal people to engage with the technology within their community life and language sharing is important but hindered by cost and suitable infrastructure in homes to access the internet (gaidon, 2007). in australia designers work in a language of a very foreign culture, that derived from england. while cultural protocols provide cohesion in protocols across different peoples through sharing technology and knowledge, aboriginal people are still often divided through language and cultural difference. it must be acknowledged that as researchers we are entering that conflicted domain (nakata, 2010) when we seek to develop mutual understanding. we need to understand our own systems before we can open these to change. 7.2 working across communities planning a project to be carried out around a common theme requires that time is spent so that all options are discussed openly, and the effect of each option be understood by all parties. this enables aboriginal people to take agency in the process and to ensure that the consequences are understood by all, through consensus decision making. while the aim is to share resource between many groups at once with similar needs, this is not always feasible due to the different aspirations of different communities. at other times the role of developer is to find the common features to provide an integrated interface for many groups combining their resources. time must also be given to gather the stories and imagery that are relevant to a project, to provide a visual and intuitive interface. in particular there is a need for suitable icons that represent aboriginal priorities, as shown in the recent development of aboriginal emojis. similar work for the penan language project has shown that such visual cues will vary across generations, as well as from other factors (zaman & winschiers-theophilus, 2015). 7.3 conversations for sharing knowledge through narrative the use of community narratives provides opportunities for different elders or experts to insert new information at relevant points in the story, a form of narrative wiki; and a way to allow community to contribute to growing the story over time as a collective narrative (kutay, 2019). this provides opportunity for different voices to enter the narrative from their expertise, to make the internet a more collaborative presentation of information. however, information that is specific to aboriginal people, the knowledge of the land in australia, is a knowledge that is very time and place bound. it cannot be shared in perpetuity without moderation of what is relevant at the time of viewing; what part of the cycle we are in (la niña, el niño); prior understanding of the viewer, etc. 7.4 identity is holistic knowledge approaching knowledge that is holistic, from a european framework that seeks to break down knowledge into component parts, means that we need to start with the whole picture in order to find common ground to talk. this conflict between knowledge systems creates issues for australasian journal of information systems kutay 2021, vol 25, research on indigenous use of knowledge elicitation information and communication technologies 15 aboriginal people working with projects that are not holistic, as was described through the work to convey knowledge of firestick burning to western scientists: the very features which provide its main significance in an aboriginal knowledge tradition (where this particular plant happens to be for these particular people at this particular point in their lives, who is telling the story, why they are telling it here and now, and how the story fits into the wider networks of kinship, art, music, ceremonials and philosophy) are lost in the process of abstraction which records how any plant of this particular species in any location might be of use to any person at any time (christie, 2005) 7.5 respect by following protocols there are existing protocols for research in aboriginal knowledge (aiatsis ethics, 2020) which also cover some similar points, such as the need to understand the stories of each group of people. these define the access rights of the project and the way that community will retain control over the work. in relating to the community, the researcher can learn how people are expected to behave in a culture. simple rules such as “do not look aboriginal people in the eye, this is rude” can be irrelevant or counter-productive for those community people who have grown up in the dominant culture, or who do not have that reservation. while there are common features across aboriginal culture, much of the knowledge of the protocols will result from engagement with the community and making many mistakes (kutay, 2021). 7.6 reputation involves understanding context the two-way partnership with a community comes from understanding the people and their needs sufficiently to explain to them the technology and engage them in the design. this increases the reputation of any external researcher as they show they understand the people. this process also enables the researchers to understand the limits of the designs and concepts to the new environment. it is important when returning to a project with a new prototype that the community be informed of what changes have been made and the reason. these must be shown as derived from the community engagement with the technology, since that may have been misinterpreted. to hand over control we need also to hand over understanding of what is being controlled. 7.7 creating novel uses less often projects deal with the benefits that aboriginal people bring to then, which is due to the reputation of aboriginal people as knowledge holders. when developing applications for aboriginal users are we providing something for the benefit and wellbeing of the user, or is the technology another extension of mainstream control and influence? are we providing people with an opportunity to both take advantage of new technology, and to use this to improve wellbeing? are we open to accept a new use for technology, such as the requirements of limiting access to knowledge according to a user’s status; prior knowledge; and deceased status (radoll, 2015)? information systems research provides a unique opportunity to use existing models such as the honeycomb model. such models can be expanded to analyse the adaption and australasian journal of information systems kutay 2021, vol 25, research on indigenous use of knowledge elicitation information and communication technologies 16 implementation of resources in aboriginal networks and enterprises (first nations media, n.d) where community identity exists across vast areas of australia. 7.8 relationship prior knowledge from an aboriginal holistic viewpoint, the scientific method often used of isolating a variable or two to explain a life phenomenon is not considered a valid approach to knowledge. in fact, the entire environment will both be part of the variables and be the location of the knowledge (bidwell et al., 2008; verran & christie, 2007). this relationship to knowledge is different to that held by non-aboriginal researchers who can only hope to understand a translation of such a view of the world, and how people act in this world, through aboriginal people who have the time and inclination to cross the ‘cultural frontier’ (nakata, 2010). an important aspect is the significance attached to relationships rather than individual roles (kutay, 2014). hence the collection of data on the community use of the technology (cf. soro et al., 2016) is an important guide to re-develop the prototype over time, and this data can be interpreted by the community itself. it does not need to be an assessment imposed externally. 7.9 understanding time in developing relationships to knowledge, the aboriginal viewpoint incorporates a continuum that blends into a spiritual view of the world. we cannot ignore or belittle this perspective simply if it does not fit western knowledge system. aboriginal spirituality is expressed as the way of knowing things from a pattern, matching over events, e.g., the sight of a willie wagtail is the sign of news. this means that the environment is acknowledging the individual and providing signs, forecasts, and warnings, that there is significance in the world. timelessness as used in the dreaming stories is part of the method for verifying knowledge on a large-scale level, which is not experimental or world changing, but world observing and world understanding. it is a knowledge system that links more closely to the specific environmental location or system on which we depend for living, than does western scientific process, and it holds a much greater value for human wellbeing as part of a community than an economic approach to land management. this could be supported by a system that analyses the user’s actions, e.g., facebook relationships, and provides feedback on how strongly they are relating to those around them and use more natural (e.g., animal) images to portray concepts to incorporate a broader understanding of what is significant in our world. 8 conclusion aboriginal australian people’s world view is strongly influenced by the environment in which the culture developed. we believe the technology and knowledge transfer needed to maintain this culture can be better supported in information systems through activities that allow communities to participate in the design and management of systems. we provide a model that has been developed from experience mainly in urban and rural nsw with aboriginal community projects. however, we believe these issues will arise in other areas, such as some work done in the northern territory with remote radio support online. australasian journal of information systems kutay 2021, vol 25, research on indigenous use of knowledge elicitation information and communication technologies 17 this may start with the interface design, the ontology of the knowledge system; and then extend to developing the components. so eventually communities can be in control of the systems and provide ongoing innovations in this area. frameworks that highlight the significant features of a system, the different aspects to be considered in design, are important in this work. acknowledgement this work was supported by various aboriginal organisations who sought to engage in developing and using information systems for sharing ideas with the broader community. thanks for their patience to robert leslie, coordinator joint sydney koori interagency network, gary williams, ceo muurrbay aboriginal culture and language cooperative, lynette riley, sydney university koori centre, anthony siever and cathy eatock, nsw dept, aboriginal affairs, dixie link-gordan, mudgin-gal aboriginal women’s cooperative, daniel featherstone and rita cattoni irca. i would also like to acknowledge olivera marjanovica, professor of information systems at the university of technology of sydney who spent some time advising me on my research focus and directing me how to present my work. references aiatsis (n.d.). https://iats.ent.sirsidynix.net.au/client/en_au/external. retrieved august 15, 2021 aiatsis ethics (2020). australian institute of aboriginal and torres strait islander studies (aiatais) code of ethics for aboriginal and torres strait islander research. https://aiatsis.gov.au/sites/default/files/2020-10/aiatsis-code-ethics.pdf. retrieved august 15, 2021. akama, y., keen, s., & west, p. (2016) speculative design and heterogeneity in indigenous nation building, dis 2016. ayala, m. r. o., jiménez, a. r. h., marisol, i. r. g. y., & hoyo, a. r. (2015). socialization of knowledge through collaborative projects. the renovarte café case. american journal of educational research, 3(3), pp. 259-262. bidwell, n.j., standley, p., george, t., & steffensen, v. (2008). the landscape’s apprentice: lessons for place-centred design from grounding documentary. in proceedings of dis 2008, 88–98. https://doi.org/10.1145/1394445.1394455 carroll, j., rosson. m. (2007). participatory design in community informatics. design studies 28. doi:10.1016/j.destud.2007.02.007 christie, m. (2005). aboriginal knowledge traditions in digital environments, australian journal of indigenous education, 34, p. 61-66. du, j.t. (2017). research on indigenous people and the role of information and communications technology in development: a review of the literature, journal of the australian library and information association, 66(4), 344-363, doi: 10.1080/24750158 2017.1397857. dyson, l., underwood, j. (2006) indigenous people on the web. journal of theoretical and applied electronic commerce research, april, 1(1), pp. 65–76. eades, d. (2013). aboriginal ways of using english. aboriginal studies press australasian journal of information systems kutay 2021, vol 25, research on indigenous use of knowledge elicitation information and communication technologies 18 first nations media (n.d.) https://firstnationsmedia.org.au/projects/broadband-bush. retrieved, august 15, 2021 fishbein, m., ajzen, i. (1975). belief, attitude, intention, and behavior: an introduction to theory and research. reading, ma, usa: addison-wesley publishing company. george, r., nesbitt, k., donovan, m., & maynard, j. (2011). focusing on cultural design features for an indigenous website. proceeding of the australasian conference on information systems 2011, article 17. https://aisel.aisnet.org/acis2011/17 goldfinch, t., jolley, l., prpic, j. k., leigh, e. (2016). australian engineering educators’ perceptions of indigenous cultures and challenges of minority inclusion. 44th sefi conference, 12-15 september 2016, tampere, finland ictv (n.d.) https://ictv.com.au/about-us. retrieved august 15, 2021. indigital (n.d.) https://www.indigital.net.au/. retrieved august 15, 2021. iversen, o. s., halskov, k., & leong, t. w. (2012). values-led participatory design. codesign, 8(2-3), 87-103. kietzmann, j., hermkens, k., mccarthy, i., & silvestre, b. (2011). social media? get serious! understanding the functional building blocks of social media. business horizons, 54 (3), 241–251. doi:10.1016/j.bushor.2011.01.005 keeping culture (n.d.) https://www.keepingculture.com/. retrieved august 15, 2021. kutay, c., aurum, a., (2007). knowledge transformation for education in software engineering. international journal of mobile learning and organisation, 1, pp. 58-80. kutay, c. (2010). issues for australian indigenous culture online. in e. blanchard, d. & allard (eds.), handbook of research on culturally-aware information technology: perspectives and model, pp. 337-361. hershey, pa, usa: igi global. kutay, c., (2011). hci study for culturally useful knowledge sharing. lecture notes in computer science (including subseries lecture notes in artificial intelligence, volume 7697 lncs, heidelberg, germany: springer verlag, pp. 69-78. kutay, c., mooney, j., riley, l. & howard-wagner, d. (2012). experiencing indigenous knowledge online as a community narrative. the australian journal of indigenous education, 41(1), pp. 47-59. kutay, c. (2014.) one person's culture is another one's entertainment. proceedings of interactive entertainment, newcastle, december, 2014 doi: https://dl.acm.org/ doi/proceedings/10.1145/2677758 kutay, c. (2018). advances in culturally-aware intelligent systems and in cross-cultural psychological studies. in c. faucher (ed.). springer science and business media deutschland gmbh, pp. 63-96. intelligent systems reference library, vol. 134. kutay, c. (2019). locating knowledge. archives and manuscripts, 47(1), pp. 72-93. kutay (2021) aboriginal knowledge online. ab-original: journal of indigenous studies and first nations, forthcoming. https://dl.acm.org/toc/ijmlo/2007/1/1 australasian journal of information systems kutay 2021, vol 25, research on indigenous use of knowledge elicitation information and communication technologies 19 lightning runners (2018). http://heartbeat.westernsydney.edu.au/play-lightning-runners/. retrieved august 15, 2021. limba (n.d.) https://limba.com.au/. retrieved august 15, 2021. lukosch, s., klebl, m., buttler, t., & hackel, m. (2009). eliciting requirements with audiobased collaborative storytelling. in proceedings of the international conference on group decision and negotiation (gdn), pp. 99-104. mignerat, m., rivard, s. (2009). positioning the institutional perspective in information systems research, journal of information technology, 24(4), pp. 369-391. https://doi.org/ 10.1057/jit.2009.13 mukurtu (n.d.) https://mukurtu.org/. retrieved august 15, 2021. nakata, m. (2010). the cultural interface of islander and scientific knowledge. the australian journal of indigenous education, 39(supplement), pp. 53-57. nonaka, i., takeuchi, h. (1995). the knowledge-creating company. how japanese companies create the dynamics of innovation, oxford university press, oxford. ormond-parker, l., sloggett, r. (2008). local archives and community collecting in the digital age. archival science: international journal on recorded information, 12(2), pp. 191-212. pascoe, b. (2014). the dark emu, magabala books, broome, wa, australia. povinelli, e. (1993). labour's lot: the power, history and culture of aboriginal action, university of chicago press, il, usa. pradhan, s., beetson, s., & kutay, c. (2018). building digital entrepreneurial platform through local community activity and digital skills in aboriginal australia, presented at australasian conference on information systems, sydney, http://www.acis2018.org/wpcontent/uploads/2018/11/acis2018_paper_187.pdf prayaga, p., rennie, e., pechenkina, e., & hunter, a. (2017). digital literacy and other factors influencing the success of online courses in remote indigenous communities. 10.1007/978-981-10-4062-7_12. retrieved august 15, 2021. radoll, p. (2009). under development: reconstructing australian aboriginal governance by systems design. interactions, 16(3), pp. 46-49. radoll, p. (2010). stone chips to silicon chips. phd thesis, australia national university, canberra. radoll, p. (2015). aboriginal peoples, education and information and communication technologies in australia. in n. bidwell & h. winschiers-theophilus (eds.), at the intersection of indigenous and traditional knowledge and technology design. ca, usa: informing science press. rigney, l.i. (2001). a first perspective of indigenous australian participation in science: framing indigenous research towards indigenous australian intellectual sovereignty, kaurna higher education journal, 7, pp. 1–13. rice, e., haynes, e., royce, p., &thompson, s. (2016). social media and digital technology use among indigenous young people in australia: a literature review. international journal for equity in health, 15, 81. australasian journal of information systems kutay 2021, vol 25, research on indigenous use of knowledge elicitation information and communication technologies 20 sawhney, h., ratnadeep, v. (2008). lateral connectivity at the margins: ritual communication and liminality on aboriginal networks. science, technology and society, 13(2), 345-368. https://journals.sagepub.com/doi/abs/10.1177/0971721808013002 09 senadheera, v., warren, m. & leitch, s. (2017). social media as an information system: improving the technological agility, enterprise information systems, 11(4), pp. 512-533. doi: 10.1080/17517575.2016. 1245872 sidorova, a., evangelopoulos, n., valacich, j. s., & ramakrishnan, t. (2008). uncovering the intellectual core of the information systems discipline, mis quarterly, 32(3), pp. 467–482. silva, s.c., feitosa, w., duarte, p., & vasconcelos, m. (2020). how to increase engagement on social media using the honeycomb model: a case study in a portuguese hr company, rege revista de gestão, 27(2), pp. 153-167. https://doi.org/10.1108/rege-02-2019-0030 smith, c., burke, h., & ward, g. (2000). globalisation and indigenous people: threat or empowerment? in c. smith & g. ward (eds.), indigenous cultures in an interconnected world, vancouver, bc, canada: university of british columbia press, pp. xviii-24 soro, a., brereton, m., taylor j.e., lee hong, a.g., & roe, p. (2016). cross-cultural dialogical probes. in k. awori & n. bidwell (eds.), proceedings of the first african conference on human computer interaction. association for computing machinery, united states of america, pp. 114-125. suchman, l. (2002). located accountabilities in technology production. scandinavian journal of information systems, 14, pp. 91-105 sveiby, k.-e., skuthorpe, t. (2006)., treading lightly, allen & unwin, sydney, australia. tacchi, j., watkins, j. (2007). participatory research and creative engagement with icts. in acm sensys workshop on sensing on everyday mobile phones in support of participatory research, november 6, 2007. verran, h., christie, m. (2007). using/designing digital technologies of representation in aboriginal australian knowledge practice, human technology, 3(2), pp. 214-27. verran, h, christie, m, anbins-king, b, van weeren t., & yunupingu, w. (2007). designing digital knowledge management tools with aboriginal australians, digital creativity, 18(3), pp. 129-142, doi: 10.1080/14626260701531944 wasko, m. m., faraj, s., & teigland, r. (2004). collective action and knowledge contribution in electronic networks of practice. journal of the association for information systems, 5(11), p. 2. wilson, s. (2008). research is ceremony, indigenous research methods. black point, nova scotia, canada: fernwood publishing. winschiers-theophilus, h., bidwell, n., & blake, e. (2012). community consensus: design beyond participation. design issues, 28(3), 89-100. doi: 10.1162/desi_a_00164 zaman, t., winschiers-theophilus, h. (2015). penan’s oroo’ short message signs (po-sms): co-design of a digital jungle sign language application. in j. abascal, s. barbosa, m. fetter, t. gross t., p. palanque, & m. winckler (eds.), human-computer interaction – australasian journal of information systems kutay 2021, vol 25, research on indigenous use of knowledge elicitation information and communication technologies 21 interact 2015. interact 2015. lecture notes in computer science, 9297. cham, switzerland: springer publishing company. https://doi.org/10.1007/978-3-319-226682_38 copyright: © 2021 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v25i0.2907 http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 background 3 literature review 3.1 indigenous relations with information systems 3.2 honeycomb model 3.3 requirements gathering in the indigenous space 4 projects studied 5 knowledge collection 5.1 storysharing from community 5.2 collaborative information sharing for aboriginal workers 5.3 collaborative knowledge sharing 5.4 gamification of collated knowledge 6 community of practice 6.1 identity and control 6.2 conversations 6.3 groups 6.4 sharing 6.5 presence 6.6 reputation 6.7 relationships 6.8 beyond 7 acceptance model for knowledge management 7.1 sharing in different languages 7.2 working across communities 7.3 conversations for sharing knowledge through narrative 7.4 identity is holistic knowledge 7.5 respect by following protocols 7.6 reputation involves understanding context 7.7 creating novel uses 7.8 relationship prior knowledge 7.9 understanding time 8 conclusion paper title (paper title style) australasian journal of information systems ye, de salas, ollington & mckay 2017, vol 21, research article applying turner’s theory of power in a troubled is implementation 1 applying turner’s three-process theory of power to the study of power relations in a troubled information systems implementation michelle yaqian ye university of tasmania yaqian.ye@utas.edu.au kristy de salas university of tasmania nadia ollington university of tasmania judy mckay swinburne university of technology abstract this paper explores the nature and exercise of power in an interpretive case study of a troubled information systems (is) implementation in a university in the asia pacific region using turner’s three-process theory of power based on social identity theory and selfcategorisation theory. the findings demonstrate the value of turner’s theoretical lens as well as its insufficiency for explaining all power related activities. this research has led to the development of an extended three-process theory of power by adding the alternative components that emerged from the data in the case study in relation to the nature and exercises of power. based on the findings, we further recommend specific guidelines for is theoreticians and practitioners including advice to project managers on a range of key issues. thus, this paper contributes theorising the sources of power and tactical applications of power in given situations, particularly in is implementation projects. keywords: power; power relations; it project; is implementation; resistance; social identity; psychological group 1 introduction information systems (is) project management has been seen as a challenging task, with critical reasons for failure including a lack of top management engagement and support, a weak project champion, inadequate information requirements determination, communication issues, organisational politics, lack of user involvement and participation, and change management problems generally (flowers 1997; kappelman, mckeeman and zhang 2006; oz and sosik 2000). given that political issues are among the critical causes of failure of is implementation projects (kappelman et al. 2006; oz and sosik 2000), power and how it is exercised are important factors in successful is projects (jasperson et al. 2002). is implementation projects redistribute information and power in organisations, and thus power relations are implicated in such projects and affect both project progress and ultimately project success (backhouse, hsu and silva 2006; silva and fulk 2012). the effective and ethical use of power may be necessary in order to achieve innovation and change through is implementation projects (ngwenyama and nielsen 2014). indeed, while ‘power over’ may imply domination and bullying (clegg 1989; dahl 1957), ‘power to’ may imply the need to exercise power in order to get things done (gohler 2009). past research on power in is implementation mainly focuses on the relations between project stakeholder groups, such as between project practitioners and system users (doolin 2004; kerr, houghton and burgess 2007; silva and backhouse 2003). existing research lacks a real analysis of power relations between project team members (hussain and cornelius 2009; silva and fulk 2012), which is argued to be an important facet of power relations. more recently, australasian journal of information systems ye, de salas, ollington & mckay 2017, vol 21, research article applying turner’s theory of power in a troubled is implementation 2 chang and yeh (2014) argued that the relationships between intra-project team disagreements and conflict communications are important factors for project performance and decisionmaking. therefore, we attempt to gain a deeper understanding of power relations within an is implementation by including the analysis of intra-project team power relations, which would fill a void existing within the current research literature. most is literature on power in is implementation is focused on the work of social and political theorists, in particular, clegg (silva 2007; silva and backhouse 2003; smith et al. 2010), giddens (brooks 1997; chu and smithson 2007; hussain and cornelius 2009), foucault (doolin 2004; knights and vurdudakis 1994), lukes (howcroft and light 2006; markus and bjorn-andersen 1987) and latour (bloomfield et al. 1997). the work of these theorists is argued to be abstract and fails to deliver a clear and useful picture regarding the source of power and tactical applications of power in given situations (ye et al. 2014), and as such, the analyses based on these works offer little practical guidance regarding the effective management of power in is implementation. although there are publications by academics in the is field that contribute a pragmatic and tactical analysis of power relations in is implementation (mathiassen and napier 2014; sabherwal and grover 2010), these contributions lack a theoretical grounding to underpin the behaviours they describe. the current paper aims to improve this situation by applying a social psychological theory which provides researchers with a basis for theoretically informed yet practical research in power relations in is implementation, namely, turner’s (2005) three-process theory (tpt) of power. further, the current study identifies the ways in which the theory can be contextualised and/or extended and thus leads to a possible reconceptualisation of the boundaries in which the theory helps to explain the investigated phenomena. there are two overarching research questions (rqs): rq1: what power relations can be identified within an is implementation project? rq2: does tpt provide a useful theoretical lens for understanding the interplay of power relations within an is implementation project? in the remainder of the paper, we provide: (1) a summary and explanation of the key propositions of turner’s tpt and its associated underpinning theories of social identity theory (sit) and self-categorisation theory (sct); (2) the methodology, explaining the research context, data collection and analysis; (3) the findings, comprising the final themes and codes, their relationships to the tpt propositions, and the proposed extension to the tpt with the alternative components that emerged from the data; (4) the discussion, concluding the findings and answers to the rqs; and finally (5) the conclusion, providing insights for project managers or system implementers and discussing future research directions. 2 propositions of turner’s three-process theory of power turner’s tpt has a firm theoretical and empirical basis in sit and sct (tajfel and turner 1979; turner et al. 1987). sit was developed to explain the psychological basis of intergroup behaviours, particularly the discrimination of in-group members against out-group members (i.e. ‘us’ versus ‘them’) (hogg and turner 1985). sct argues that individuals have a rough hierarchy of categories or identities that they refer to in their social life, and belonging to a particular category is equivalent to belonging to a psychological group in which a series of values and interpretations are shared (hornsey 2008; turner 1991). the acceptance of a category or an identity may occur over a long period of time, but can at times occur quickly in a particular situation. thus based on sit and sct, turner (2005) introduces the concept of psychological group formation for describing the nature or source of power (see figure 1). australasian journal of information systems ye, de salas, ollington & mckay 2017, vol 21, research article applying turner’s theory of power in a troubled is implementation 3 figure 1. the three-process theory of power (adapted from turner (2005)). based on the above concept, turner (2005) argues that psychological group formation produces influence which gives people power through the processes of persuasion, authority, and coercion. turner also suggests that in turn, power leads to the creation and control of resources. it is argued that power is not grounded in material resources but it is group identity that confers power. the five propositions of turner’s theory are discussed below. proposition 1 – people tend to self-categorise into psychological group(s) and these self-categorisations become relevant in determining behaviours in particular contexts or situations. an example of such categories may be those categories applying to two female academics, one an accounting academic and the other a management academic, both in university x. these two persons would likely belong to such groups as university x, the business school of university x, academics, and women; these groups being in a rough and perhaps overlapping hierarchy. in a faculty budget situation, these two women may be rivals and power players in the competition for resources, but in a different situation, may both support the university in building its reputation in competition with other universities. at an even more inclusive level, both academics may support women in general in equal pay and other social justice issues. thus, different categories have salience in different situations. proposition 2 – psychological group formation produces a situation of mutual influence through a shared identity, which forms the basis of power through persuasion. people in a psychological group perceive themselves as more similar to each other than they are with people in a different group, and therefore they expect to have similar views within their psychological group. people are more likely to be persuaded by intragroup members as they usually have shared attitudes, beliefs and experiences. this forms the basis for intragroup influence, and promotes the exercise of power through persuasion (turner 1987). for example, andrew and michael are two members of y team in the x project team, where andrew is the y team leader and michael is a y team worker. when michael’s identity as a member of y team becomes salient and the matter of discussion is about the improvement of their team performance, andrew tends to be perceived by michael as an in-group member based on the shared identity, and thus they tend to engage in mutual persuasion to reach agreement. the shared group identity unifies and empowers andrew and michael by encouraging in-group consensual support. intragroup similarities between andrew and michael are apparent in this context because they are categorically interchangeable as members of y team (oakes, haslam and turner 1994; turner 1984). proposition 3 – authority is the power based on in-group norms that group members ought to follow a specific person or position (leader) who has the right to control them in certain matters. australasian journal of information systems ye, de salas, ollington & mckay 2017, vol 21, research article applying turner’s theory of power in a troubled is implementation 4 authority in tpt is power that is legitimated by in-group norms that have a shared social identity as their basis. it is conferred by “formal agreement, custom or the norms inherent in group activity” (turner 2005, p. 11), thus facilitating collective action to achieve common goals and quick decisive action at times and in situations where it is necessary. thus the tacit or formal agreement involved in authority permits a designated group member to control others in the in-group. authority, then, short-circuits the debates, deliberations, and arguments that may take place without such a working agreement in place. a typical in-group in which authority commonly operates, is the psychological category or group of committed members of a business or government organisation. in order to retain the group membership and the benefits that flow from being an employee, members of business and government organisations submit voluntarily to the authority hierarchy of the organisation. to take the example further, consider the two members andrew and michael of y team in the x project team, where andrew is the y team leader. at the personal level michael and andrew may disagree, however if the disagreement is about x project matters then the authority hierarchy of the organisation becomes salient and michael is likely to go along with andrew because andrew has the authority of a team leader. michael then agrees, not necessarily because he is persuaded, but because he is submitting to the legitimate authority. that is, michael agrees that andrew has the authority right conferred by the group norms (team hierarchy) to mandate that to him. proposition 4 – coercion is the power to control a target against their will through the deployment of resources to constrain and manipulate their behaviour. turner (2005, p. 12) refers to coercion as “authority in a dark mirror”. it is the form of power employed when one is not able or is not willing to persuade, and when one does not possess legitimate authority. given this, it is likely that persons resorting to coercion in a given situation may lack a shared identity with the target(s) of the coercion. as such, they may have little basis for persuasion and are perhaps also less likely to possess any legitimate authority in the view of the target(s). as an example of coercion, consider the example of michael and andrew. y team worker, michael, is told by his team leader to work through meal breaks until midnight in order to finish work on a number of important deliverables, but michael believes andrew’s command to be outside of andrew’s authority. in this context, andrew may have become a coercive agent of his superior in the x project team, at least perceived by michael. proposition 5 – when coercion is perceived to a degree resistance can occur, which leads to a threat to the psychological group identity that people wish to retain. in the above example in proposition 4, the situation was considered overt resistance. however, in the face of threats to fire him, michael eventually complies. at the same time michael may take action to quietly resist andrew’s command such as secretly taking meal breaks, and further, now mistrusting andrew, decide to look for ways to resist future coercion attempts. thus, the use of coercive power is not without its problems. coercion tends to generate mistrust in targets, and weakens the possibilities of the future use of persuasion and authority (kramer 1999). thus it undermines possibilities for legitimate influence. further, coercion provokes and encourages resistance and leads to the need for surveillance in order to enforce its conditions (turner 2005). thus, a risk of coercion is that it tends to weaken the power of those applying it as it may bring into being an adversary opposed to the source of the coercion. tpt thus includes an explanation of phenomena as resistance (van dijk and van dick 2009) and persistence with failing projects (haslam et al. 2006). the five propositions articulated above form the basis of turner’s (2005) theoretical lens that guided the current research. the social identity approach (i.e. sit and sct) has been shown to have large value for the understanding of power related behaviours in information technology (it) based organisational change (schwarz and watson 2005; tansley, huang and foster australasian journal of information systems ye, de salas, ollington & mckay 2017, vol 21, research article applying turner’s theory of power in a troubled is implementation 5 2013), non-it-based organisational change (currie, finn and martin 2010; van dijk and van dick 2009) and in other social science disciplines (fritsche et al. 2013; hogg et al. 2005; hornsey, blackwood and o'brien 2005). based on the firm theoretical foundation of sit and sct, turner’s theory appears to be an insightful framework for understanding and interpreting different group members’ experiences and perceptions of persuasion, authority, and/or even coercion in project events surrounding the issues and problems inhibiting a project’s progress. it enables researchers to draw on particular power-related events in project groups, which are usually constituted by different professional groups/stakeholders, different gender groups, or different hierarchical levels. 3 methodology this research is an exploratory study. the ontological preference of the current research is for people’s perceptions and actions informed by the organisational and social contexts within which they work and live. this ontological view, as explained by orlikowski and baroudi (1991), emphasises the subjective meanings that participants assign to the world around them, rather than the objective belief that the social and physical worlds exist independently of the human actors. with reference to this study, a subjective ontology is acknowledged. holding to the belief that to understand the meaning of the social world one must interpret it (schwandt 1994), and given the focus of this research being the identification and interpretation of human relational factors affecting an is implementation project, an interpretivist approach to undertaking the study was adopted. based on the subjective ontology and interpretivist epistemology, a longitudinal case study was conducted in a university in the asia pacific region over a period of two years (from may 2012 to june 2014). the case study method was adopted as it is particularly well-suited for understanding the interrelationships between is implementation change and management practices in an organisational context (benbasat, goldstein and mead 1987). 3.1 research context the site for this research was a university in the asia pacific region, named as asiapac university (pseudonym). the focus of analysis in the case study is the power relations and resistance behaviours throughout the implementation of a large student system (ss) that integrates approximately 150 systems covering various business areas, a core business transformation project in asiapac university. the project was initiated in 2006 and eventually went live in 2014 (see figure 2). since its first business case was approved in 2007, the project was troubled with multiple time and budget overruns. the business case was revised twice as was the go-live date. until the end of our case study in 2014, the project had stalled in the pre-go-live stage of a project lifecycle. significant project structure changes also occurred, particularly to project leaders. australasian journal of information systems ye, de salas, ollington & mckay 2017, vol 21, research article applying turner’s theory of power in a troubled is implementation 6 figure 2. the timeline of the ss project. the ss project was initiated in 2006 with the objective to assess and review options for transforming the university’s approach to student and academic administration, and the systems required to support the service delivery. in late 2007, the ss project steering committee endorsed a business case and approved the recommended option – to buy a commercially available student information system. the project was allocated a budget of around $11 million and the go-live date for the ss system was june 2011. a project team of 11 members was assembled. early progress was slow and some problems emerged, but it was assumed that the project was progressing reasonably well. nonetheless with some anxieties voiced, an external review was announced in 2010. this review was highly critical and caused shock waves in the university council, which reported that given the progress to date, the budget and timeline were insufficient and the project team would not be able to meet the targeted 2011 implementation timeframe. discussions among council members and the vice chancellor led to the vice chancellor replacing the project director after 4 years heading up the project team. the new project director was recruited from outside the university, previously the ceo of an online service company, despite having no experience of directing and managing large and complex is implementation projects. the second project director reviewed the original business case and had a new business case prepared, which was approved in december 2010 with a new timeline, and an increased budget and resources. the new project budget was almost doubled to $22 million and the project team was increased to over 30 members. however, the project fared little better under the new project director. the second project director had a remote ‘ceo style’ and did not tend to set clear directions for the project team leaders. his lack of hands-on project management became obvious to a number of senior members of the project team. his upward communications to the senior management team in the university were reassuring and so it was another shock to the council and vice chancellor when a second external review of the project showed it to be in a parlous state, having no chance of meeting the planned go-live date. the second project director, seeing the writing on the wall, left the university. after the second project director left, the project director’s deputy took charge of the ss project with a view to her holding this position until the next project director was appointed. the temporary project director struggled to give the project team a sense of stability and progress. a number of people in the project team became disillusioned with the lack of direction and progress and left the university. painfully aware that the project was not making australasian journal of information systems ye, de salas, ollington & mckay 2017, vol 21, research article applying turner’s theory of power in a troubled is implementation 7 progress and bedevilled by a number of fractious disputes within the project team, the temporary project director resigned. in late october 2012, a new project director who had experienced complex projects was recruited from outside the university. the new project director, needing to accurately and publicly establish the current state of the project, instituted another comprehensive external review of the project and then based on the findings, argued that the april 2013 go-live target was no longer possible. however, the senior management of asiapac university was now getting impatient with the retreating go-live dates. thus the vice chancellor brought considerable pressure to bear on the new project director, insisting, among other things that she meet the deadline of september 2013 as a compromise of the previous april deadline. this time however, the new project director was a professional and highly experienced project manager who could clearly see that the deadline was completely unrealistic and so she refused to commit to it. this caused confrontation between her and the senior management. for a while it seemed that the new project director would be dismissed and she walked out of the position in protest to the unrealistic deadline. during this time of uncertainty for the ss project, the vice chancellor requested that the university cio get involved in the project to assess whether the deadline was in fact unattainable as the new project director was asserting. the cio spoke with the project team members and was told quite firmly by a number of them that the new project director was correct. the upshot was that the new project director was requested to come back to take charge of the project and the university council was informed that the new project director’s assessment of july 2014 as an approximate go-live date was now the official target date for the implementation of the new system. the new project director then started detailed work on the new project management plan and the third business case based on the revision of timeline, resource and budget at the same time of working with the external review team and re-structuring the project team. by that time, the new project budget was almost doubled again to $40 million and the size of the project team was significantly increased to over 60 members. the ss project then began to move in the right direction under the third project director’s leadership and finally went live within the designated timeline. 3.2 data collection and analysis the data collection and analysis process involved two phases as shown in figure 3, during which a series of interviews were conducted (18 and 30 respectively). in the first phase, the matters of power relations, politics, and group influence emerged from the initial data collection. in the second phase, noting the emergence of aspects of turner’s theory from the preliminary analysis, a more focused, theoretically informed approach was conducted in which turner’s theory was used a lens to guide subsequent data collection and analysis. australasian journal of information systems ye, de salas, ollington & mckay 2017, vol 21, research article applying turner’s theory of power in a troubled is implementation 8 figure 3. data collection and analysis process. in both phases, data was collected by: 1 forty-eight in-depth, open-ended interviews of between 50-90 minutes with 46 participants (see table 1), among whom two were heavily involved project leaders, and as such were interviewed twice for further in-depth conversation. the interviews were delivered face-to-face, except for one telephone interview. each of the 47 face-to-face interviews was recorded by using a digital audio voice recorder, and written-notes were taken during the telephone interview. at the beginning of each interview, key concepts were explained by the researcher and a few general questions were asked to elicit conversation. the researcher used improvisation strategies, and listening strategies as suggested by myers and newman (2007), in order to construct questions or provide prompts based on the participant’s response. 2 non-participant observations in ten project meetings, three user workshop sessions and informal discussions with participants. the observations were written within the field notes, which also included a reflective diary and a factual event listing in order to provide additional assistance in contextualising and interpreting the data. 3 study and analysis of the project related documents. primary documentation was provided by project managers including project plans, business cases, project team structure diagrams, meeting agendas and meeting minutes. secondary documentation was obtained from the publicly available resources on the organisational website including the monthly issues, presentations and demonstrations for users and project news. interviews were the primary source of data, with the other two sources used to contextualise and confirm the researchers’ understanding of data throughout the analysis phase. the australasian journal of information systems ye, de salas, ollington & mckay 2017, vol 21, research article applying turner’s theory of power in a troubled is implementation 9 recordings from the interviews were transcribed line by line together with written notes being read through in order to understand the participants’ interpretations of events. inside the project team outside the project team project leaders (pl1-5) (project managers/directors, assistant/deputy project directors) senior management (sm1-5) (deputy vice chancellors, chief operating officer, deans and associate deans of faculties) project middle management (pm1-8) (project team stream leaders, team leaders, senior consultants, communication manager) business administrative group (bs1-6) (heads of service in various business areas of the university) business analysts (ba1-8) (business analysts in the project team) it division management group (tm1-2) (chief information officer and the associate director in it division of the university) it workers (tw1-5) (system programmers, system developers, system testers in the project team) transition support group (to1-5) (transition support staff recruited for the ss implementation) training team (tt1-2) (training developer, training agent) table 1. roles of the participants. data was analysed using a grounded theory based coding paradigm developed by strauss and corbin (1990), which included three conceptual coding levels: open, axial and selective. open coding includes an inductive coding process from the raw data and axial coding uses the process of relating codes to each other until themes emerge. once themes had emerged from the data categories and sub-categories, selective coding was carried out. this final phase included taking note of the social behaviours and activities that were in the coded data set, how the coded data represented the emerged themes, and further, how the themes and the data set could answer the research questions (strauss and corbin 1990). in order to establish rigour and trustworthiness in the research process and therefore the findings, the current research drew upon the set of principles for evaluating interpretive field research proposed by klein and myers (1999) (see appendix). 4 findings with the application of the three-phase coding strategy, two themes emerged; namely human relational factors affecting the project and non-human-relational factors affecting the project (see figure 4). the first theme is the primary focus of this research project whereas the second theme helps contextualise and understand the human relations. under the first theme, there are two sub-themes: project team relationships and stakeholder relationships. australasian journal of information systems ye, de salas, ollington & mckay 2017, vol 21, research article applying turner’s theory of power in a troubled is implementation 10 figure 4. themes and sub-themes. it was found that the major issues and problems that affect the implementation and institutionalisation of the is implementation included both human relational issues and nonhuman-relational issues. the human relational issues included: • the intra-project-team human relational issues:  ineffective project leadership,  ineffective project team communications,  negative perception of the project team culture,  project team instability,  project team member conflicts; and • the project stakeholder group relational issues:  negative perception of organisational culture,  organisational instability,  ineffective stakeholder communications,  stakeholder conflicts. the non-human-relational issues included • inexperience of the project team and the whole university for an it project of this size; • lack of expertise in the project; • lack of project management discipline in the project; and • underestimation of project scope and complexity, and accordingly, the budget and timeline. the emergence of the human relational issues reflects the importance of exploring social and power relations within is implementation projects. the following diagram (figure 5) graphically summarises the relationship of each of turner’s propositions to the first theme human relational factors affecting the project, more specifically, to their relevant axial codes. the causal relationships between the axial codes and between the categories are also shown in the diagram. australasian journal of information systems ye, de salis, ollington & mckay 2017, vol 21, research article applying turner’s theory of power in an troubled is 11 figure 5. relationship of tpt propositions to the axial codes and categories of human relational factors. australasian journal of information systems ye, de salis, ollington & mckay 2017, vol 21, research article applying turner’s theory of power in an troubled is 12 4.1 project team human relational inhibitors five human relational issues were identified within the ss project team. the findings suggested that ineffective project team communications and ineffective project leadership were largely driven from potential discrimination between different psychological groups. in particular, people in the ss project team tended to see two psychological groups: the business oriented people consisting mainly of business analysts and technically oriented people consisting mainly of system developers and testers. these groups consisted of persons with similar worldviews concerning the work of the project including views on the right way to do things and views on the way problems should be approached and dealt with. although the relationships between the groups were not necessarily antagonistic, the intergroup communicative activities tended to be affected negatively by different worldviews between the groups. a project director described the two project staff psychological groups as follows: “within the rest of the team members, there is a number of, it’s not formal, it’s more informal, and it’s all about the skill sets they have, so the business analysts can kind of be drawn together, either for ‘i’ve got this problem i don’t know how to deal with it, do you know how to, (with) one of those have you got an example you could give me’ or just it’s a bit of a natural ‘you are at my level, so i talk to you’ that pecking order stuff as well, so there is business analysts. there is (a group of) developers. this is really, you would love this: the group in that room in there, 115, we now allocate that to these testing people in there, and there is a developer in there, and i mean these are your real propeller head, techo kind of things, and they almost look alike, you never hear from them, they are all males, you know, some of them are a bit of that, you know, 1980s long hair in a ponytail, they are centric in their clothing, one with a binni hat, he isn’t even bold head, so they are just a little bit different, but you know they are developers. i’ve never seen them talk with anyone else other than their own group. when we sometimes have social functions, you know, on the friday a sausage sizzle or something like that, they will be, it’s almost automatic, they will go into their huddle” (a project leader, pl5, second interview, lines 126-140). when the issues of ineffective communication and ineffective leadership deteriorated to a point, project team member conflicts occurred. project team member conflicts occurred that were driven from psychological group difference and discrimination. in particular, there seemed to be evident divides between the ex-business staff and externally recruited project specialists, as suggested by two project managers: “there were, in some ways, quite evident divides between university-sourced staff on the project and contractors. that’s normal for a project like this, but this one seemed to be a bit more hostile” (a project middle manager, pm8, lines 23-25). “within my team, i can see that i’ve got little cliques within my team, so people who have been here forever, you know, the old timers, and the new ones tend to stay together, and there is only a couple of people that have managed to bridge the gap. when we had the admissions and the [student system] project, that admissions team stuck together and made decisions without even regarding anyone else in this team even though it was team of five. they were just so cliquey that you couldn’t break into it, and [their team leader] encouraged that to be ‘we are special, we are this…’ that sort of thing” (a project middle manager, pm4, lines 468-474). these conflicts further led to negative perception of project team culture and project team instability. the project team culture was then negatively perceived as unpleasant, political, and even ‘bullying’. moreover, frequent and considerable changes happened as a result in project staff, leadership, reporting lines, and even project team restructure. “i think with the various examples of not so much formal complaints, but when people left with their exit interviews they either hinted at or were quite specific, and it comes in under this word that is used all the time now around ‘bullying’. the word ‘bullying’ australasian journal of information systems ye, de salis, ollington & mckay 2017, vol 21, research article applying turner’s theory of power in an troubled is 13 seems to, it does mean coercion i think, and it’s quite commonly used now” (a project leader, pl1, second interview, lines 422-425). “the whole, the way the structure change in the project, it was a form of bullying. (…) i was told to report to [a different project manager], and a month and half later, nothing was done and i just went ‘no, i’m done. i’m leaving.’ i’m not the only one who was leaving too, so two members were leaving at that time. the whole middle management level vanished as well. it was a very frustrating period and we just couldn’t keep up with it” (a project middle manager, pm6, lines 65-68). 4.2 stakeholder relational inhibitors outside the ss project team, ineffective stakeholder communications also appeared to be the result of potential discrimination between different psychological groups. the psychological group difference was likely to be more significant between stakeholder groups than between sub-groups within the project team. these ineffective communications were largely presented between the ss project team and the business stakeholder group, and reflected in the lack of senior engagement and support. a change consultant said, “it’s a game where parties didn’t know how to spoil the old way of operating so that you could be successful. so the programme didn’t know how to change the executive, and the executive didn’t know how to change all the deans, associate deans, and anyone else who’s involved in the game making decisions, so both parties were complied in the game that couldn’t succeed, so throwing a lot of money against the wall” (a project middle manager (change consultant), pm1, lines 208-212). it was also found that the alignment with the ss project seemed to be perceived negatively by the business staff group. for example, the early project leaders were the ‘accidental project managers’ (hunsberger 2011; turner, keegan and crawford 2003), who were seconded from the business into the ss project. when the secondment began, there was an influential close relationship between the project and the business as the seconded project leaders maintained their influence in the university. however, when the seconded project leaders assimilated into the project and became more ‘project-like’, the business staff group started to feel that these seconded project leaders’ influence separated the ss project and the business considerably as they made decisions in the project based on little consultation with the current business staff members. a project leader said, “our second issue that i would categorise has been the way, in 2010, when the project review happened and two people were seconded from the business as director and system director, at the time that was a very good move, it gave a lot more connection, but over time, the project and the business separated, so decisions were being made within the project that the business didn’t have this ability of, so the connection and engagement wasn’t so efficient.” (a project leader, pl4, lines 109-113). this finding appears to reflect that people form groups psychologically (turner 1978), and such psychological group formation can impact on the progress of an is implementation (schwarz and watson 2005). aside from the factor of psychological group discrimination, the stakeholder communications were also affected by negative perception of organisational culture and organisational instability. the organisational culture was perceived differently by different people, or from different points of view. some interpreted the university as a bureaucratic environment, which led to inevitable bureaucratic inefficiency (clegg, harris and hopfl 2011). the ss project staff felt hampered by too many meetings and noted a lack of efficient decision making. “i also think it seems to be very political here and there is quite a lot of high levels of politics. there is also a lot of i guess bureaucracy or politics that happen at a high level that seem to have an impact down here, so sometimes it’s [ss] meetings, and they are a lot better now, but there was a time where people were just talking about all this australasian journal of information systems ye, de salis, ollington & mckay 2017, vol 21, research article applying turner’s theory of power in an troubled is 14 high level decision sorts of politics stuff that didn’t really have any bearing on getting this done” (a business analyst, ba3, lines 44-49). others interpreted the culture of the university as too participative, and thus people could be overly flexible with changes they wanted in business processes. this made the implementation of an off-the-shelf package like ss extremely difficult as ready-made off-the-self products are not developed for specialised or individual needs. “it’s also an element of flexibility of what they can do and what they are going towards is something like locked down with rules and you have to follow what the system does, you make a change, the impact just, like, blows out. so they can’t continue doing that and that’s a very big cultural shift for this place particularly” (a project middle manager, pm4, lines 451-454). no matter which way the organisational culture was perceived, however, these negatively perceived cultural aspects constrained the progress of the ss project. further, other organisational changes in the university and the consequent high university staff turnover also exacerbated the communications between the stakeholder groups around the ss project. 4.3 project team human relational & stakeholder relational improvements the improvements of the ss project were presented differently within the project team and outside the project team. things within the project team that were improved and then facilitated the project were the leadership and team culture, whereas outside the project team, improvements were mainly presented in stakeholder communications. it can be inferred that stakeholder communications play a critical role in determining the success of a project. the reason that the organisational culture and the organisational leadership were not among the things that were improved may be due to the fact that these two organisational factors were relatively stable and not easily changed by a single it project. 4.4 project team human relational strategies interestingly, the strategies that made the improvements possible and used within the project team were found to be different from those outside the project team. acceptance of authority was the effective strategy inside the project team for managing the human relational issues. although persuasion also emerged in the codes with respect to the intra-project-team relations, it occurred within a psychological group and acted in the way that the project staff supported their group members against the others whom they did not identify with. thus, persuasion was not interpreted as a strategy for dealing with the intergroup issues within the project team. instead, it appeared that the project staff tended to accept a command or a decision made by their superior in the project team. it was indicated by a number of participants that this was because they were willing to submit to the legitimate authority of their superior that was conferred by the project team hierarchy. an example was that the first project director disagreed with the project steering committee decision of bring in consultants for an extensive project review but obeyed it because he viewed the steering committee as legitimate authority he was willing to follow. he said, “i thought it was not going to help the project in any sense. it was an enormous amount of work to bring consultants in cold. they hadn’t had any previous connection with the project. (…) my initial reaction was ‘this is rubbish, it’s just going to distract us and all the rest of it’. however, i don’t have an authority to actually stop this thing happening and therefore i’ve got to go along with it, and i can actually see that they’ve got a point in what they were asking for” (a project leader, pl1, second interview, lines 231-233; 315-318). when the researcher continued to ask what would have been his response if this decision had been made, not by persons senior to him in the organisational hierarchy, but by someone at his own level. he replied: australasian journal of information systems ye, de salis, ollington & mckay 2017, vol 21, research article applying turner’s theory of power in an troubled is 15 “i would have taken a much stronger argument against it certainly, yes” (a project leader, pl1, second interview, line 326). it was inferred from the conversation that the first project director saw himself as belonging to the ss project staff group, and at a higher level, to the psychological group of asiapac university employees, and as such, he accepted the hierarchy of the project team and the organisation as legitimate and hence saw it as right that the steering committee could mandate an external review. he did not agree with the decision but viewed it as within the legitimate authority of the steering committee so that he had to obey. this supports turner’s (2005) notion of authority in the way that authority short-circuits the debates when a voluntary submission to the group norms or structure is taking place. 4.5 stakeholder relational strategies outside the project team, authority power appeared to be no longer effective between the project team and the other stakeholder groups. this was due to the fact that the project leaders did not feel they had the legitimate right to order a business stakeholder to do something, especially when the project leaders came from outside the university. the business stakeholders did not see the ss project leaders as higher than themselves in the organisational hierarchy either. many business stakeholders did not see the ss project leaders as sitting in their reporting line. due to the lack of authority power over university business stakeholders, the ss project team eventually had to resort to persuasion strategies to manage the stakeholder relational issues. however, a direct persuasion seemed to be difficult between the ss project team and the business staff group as they did not tend to identify with each other. the project team had to rely on a number of business representatives to act as the ‘change champions’ for persuading their university colleagues. the persuasion between the change champions and their business people was enabled by the sharing of a ‘business-stakeholder’ identity and accordingly the shared belief and attitude within their psychological group. this phenomenon supports turner’s (2005) notion of persuasive power that springs from psychological group formation. the second strategy between project stakeholders that emerged to be effective was challenging organisational hierarchy. the data suggested that, among ineffective project team communications, a critical issue was that the project team staff seconded from the university business areas were submissive to the organisational hierarchy, and thus they tended to convey better-than-real news. a project consultant said, “i think if they come from this university, they’re immediately subordinate to the traditional corporate hierarchy here rather than the project hierarchy, so they don’t know how to counter or challenge, understandably, for all organisational power. if you come from other universities, as a contractor, you will be more comfortable about that. if you come from a vendor or service provider, a business like ours, then you assume you have to confront and in some ways challenge the negotiated order or the world will be as it was” (pm1, lines 454-459). when the externally recruited project staff came on board, they challenged the organisational hierarchy when facing unrealistic mandates. in particular, the third project director reported the whole story to the university senior group. her challenging the hierarchy was made possible because she and her externally recruited project specialists were less submissive to the university authority than the university-sourced project staff. the third project director did not see the unrealistic mandates as within the legitimate authority of the university senior group. more importantly, it was also her firm approach and her expertise and experience in managing big it projects that made her resistance to the organisational hierarchy possible. 4.6 extension to the tpt the failure to find strong support for a social-group-identity-and-power link reflects the importance of considering other determinants of power outside the social identity approach (tajfel and turner 1979; turner et al. 1987). based on the findings discussed above, australasian journal of information systems ye, de salis, ollington & mckay 2017, vol 21, research article applying turner’s theory of power in an troubled is 16 figure 6. augmented three-process theory of power. australasian journal of information systems ye, de salis, ollington & mckay 2017, vol 21, research article applying turner’s theory of power in an troubled is 17 opportunities of augmenting the tpt have been found for a more thorough understanding of the nature and the operation of power (see figure 6 above). as a by-product of the knowledge gained from this research, turner’s (2005) tpt is augmented by including the components marked in dotted boxes in figure 6. as illustrated in this diagram, the augmentations consist of two major parts: around the determinants of psychological group formation and around the power-resource causal relationship. from our research findings, we have uncovered a number of factors that affected psychological categorisations in the ss project, which in turn affected power relational behaviours. in addition to shared social group identity, it is useful to consider alternative factors that have been found important in affecting how people identify with each other psychologically. in line with the extant literature, it has been revealed that sharing technical language and terminology (foucault 1977), being skilful in communication (yukl, falbe and youn 1993), and/or having matched personality (anderson and spataro 2008) could lead to the tendency of people identifying with each other into the same psychological group. the findings also suggest that this categorisation does not need to rely on sharing work-based identity or sharing similar social background. therefore, adding these alternative determinants of psychological group formation to turner’s theory furthers the understanding of social categorisation (sachdev and bourhis 1985; tajfel et al. 1971) and the understanding of the nature of power (simon and oakes 2006; turner 2005). with regard to power and resources, turner’s tpt highlights the one-way causality of power to the creation and control of resources. the findings support turner’s argument by demonstrating that controlling resources could not necessarily lead to effective power exercise, and gaining power did lead to the creation and control of resources. however, the findings also indicate that the acquisition of expertise and knowledge were found to be the reasons that people could successfully challenge power. if one views expertise and knowledge as some form of resource, it can be argued that the control of resources could lead to power, in particular, through expert power (french and raven 1959) or informational power (deutsch and gerard 1955). thus, the acquisition of expertise and knowledge, regarded as part of resources, has been included in the construction of the augmented theory as the element that can assist the resistance to intergroup coercion. this finding adds to turner’s proposition by explaining how resistance may succeed in its expectation. that is, the acquisition of certain resources (i.e. expertise and knowledge) may lead to successful resistance to the coercion that springs from intergroup discrimination. 5 discussion following from the above findings, this section will conclude and discuss the findings in relation to the rqs. rq1: what power relations can be identified within an is implementation project? rq2: does tpt provide a useful theoretical lens for understanding the interplay of power relations within an is implementation project? turner’s perspective (1987; 2005) is mostly eloquent in demonstrating how social influence and power can be gained and exercised, and specifically, how social group identities affect people’s categorisation of psychological groups which determines different ways of power exercises that people resort to: persuasive, authoritarian or even coercive. while turner’s tpt provides a useful theoretical lens to investigate and explain most power relational activities in the ss project, some alternative factors as determinants of power-related behaviours were discovered to be important, and as such, they need to be conceptualised and integrated into the understanding of power. thus a more thorough understanding may be gained when turner’s theory is combined with some aspects of other relevant theories, and with some aspects related to personal characteristics and influence tactics. australasian journal of information systems ye, de salis, ollington & mckay 2017, vol 21, research article applying turner’s theory of power in an troubled is 18 tpt has been extended by including the discovered alternative determinants of social categorisation and the potential relationship of the acquisition of resources (expertise and knowledge) to power through resistance. the alternative determinants together with the basis of social group identity, and the extended power-resource explanation, have been found to contribute to the formation of psychological groups, and to the prediction of behaviours, thereby improving the explanatory power of tpt. 6 conclusion in this paper, we aimed to explore the value of applying a social psychological perspective, namely turner’s (2005) three-process theory of power, to explain and build a deeper understanding of power related behaviours in the context of is implementation. 6.1 theoretical implications at a theoretical level this research has highlighted that the application of turner’s theory to the is case study has discovered usefulness in explaining behaviours around gaining and exercising power. this research has additionally demonstrated that there were alternative sources of power and additional aspects of power-resource links affecting the is implementation process which turner’s theoretical lens does not cover. by applying and augmenting the social psychological framework, this research has built a deeper theoretical understanding of power relations in is implementation. this understanding was not only limited to the relations between project stakeholders; it also included the intra-project team relations. therefore, a major theoretical contribution of this research is the introduction of a more tangible and detailed analysis through the social identity approach (tajfel and turner 1979; turner et al. 1987) for understanding and explaining the nature of power. this has been proven useful in similar research (schwarz and watson, 2005; van dijk and van dick, 2009; tansley et al., 2013). turner’s (2005) theoretical lens formed a basis for understanding the interand intra-group phenomena and highlighting the thinking and action of individuals such as how the ‘change champions’ made possible collective products through psychological processes to facilitate the is implementation and institutionalisation. it is worth noting that another contribution is to turner’s tpt of power itself. in the augmented tpt, the alternative determinants together with the extended power-resource explanation contribute to the understanding of social categorisation and power relations, and to the prediction of behaviours, thereby improving the explanatory power of turner’s theory. 6.2 practical implications in practical terms, this research adds to knowledge concerning project management; in particular, concerning the effective management of power relations in it projects. based on the findings, we recommend guidelines for it project managers or system implementers (see table 2), on ways to persuade and to overcome political and non-political issues during is implementation. these recommendations will provide guidance to is theoreticians and practitioners what constitutes effective and ethical management of power relations, as well as non-human-relational aspects, in is implementation projects. australasian journal of information systems ye, de salis, ollington & mckay 2017, vol 21, research article applying turner’s theory of power in an troubled is 19 human relational guidelines non-human-relational guidelines  emphasising possible shared membership for persuasion  sharing terminology and language for persuasion  influencing with expertise  respecting and gaining trust  developing extrovert personality and good social skills  giving authoritative direction when necessary  avoiding over-submission to organisational hierarchy  having strong project management discipline  resourcing experienced project expertise  estimating and planning project accurately  considering potential uniqueness of organisational requirements and culture table 2. guidelines for it project managers 6.3 limitations of the study there are limitations within this study. first, the research was conducted in a single case study based on one higher education institution. thus the findings could be limited by focusing on one organisation and a certain era in time. while the longitudinal data collection provided the researcher with sufficient data to produce lessons for implementing the new system at asiapac university, we did not provide a comparison with other higher education institutions. insight may have been gained through such a comparison, which could be a basis for future research. second, our assumptions based around personality traits are somewhat weak; particularly considering personality traits were not ‘measured’. nevertheless, these findings highlight the importance of further work in this area. for example, future work should aim to incorporate a survey approach to measure personality traits in order to tease out possible relationships between personality and it project manager performance in managing project issues. a third limitation of the paper is that the case study was conducted mainly during the pre-go-live stage of the ss project and ended at the project go-live stage. this made it difficult to compare project issues and required project manager skills at different project stages. future work could look into this aspect as different project phases often have different requirements on management skill sets. 6.4 future research directions there remain many possibilities for future research. it is envisaged that future research could consider a different organisational context such as non-educational organisation or private business sector. future work could thus investigate the factors affecting is implementation within different organisational cultures, and as such, the value of applying turner’s (2005) tpt of power could be further explored. as the interpretivist case study approach appears to be a useful way of exploring power issues and the value of turner’s theoretical lens, further research concerning political issues in a single organisation may benefit from adopting a similar method. nevertheless, future research exploring power relations may find that a better understanding might be gained when a combined method is used. a combined approach for example, may enable both an in-depth understanding of the power interplays in particular political events, as well as the exploration of possible causal relationships between participants’ social or work-based identities and their power behaviours (turner 2005). references anderson, c., and spataro, s.e. 2008. "personality and organizational culture as determinants of influence," journal of applied psychology (93:3), may, pp 702-710. australasian journal of information systems ye, de salis, ollington & mckay 2017, vol 21, research article applying turner’s theory of power in an troubled is 20 backhouse, j., hsu, c.w., and silva, l. 2006. "circuits of power in creating de jure standards: shaping and international information systems security standard," mis quarterly (30:special issue), august, pp 413-438. benbasat, i., goldstein, d.k., and mead, m. 1987. "the case research strategy in studies of information systems," mis quarterly (11:3), september, pp 369-386. bloomfield, b.p., coombs, r., owen, j., and taylor, p. 1997. "doctors as managers: constructing systems and users in the national health service," in: information technology and organizations: strategies, networks and integration, b.p. bloomfield, r. coombs, d. knights and d. littler (eds.). oxford: oxford university press, pp. 112134. brooks, l. 1997. "structuration theory and new technology: analysing organizationally situated computer-aided design (cad)," information systems journal (7:2), april, pp 133-151. chang, t.j., and yeh, s.-p. 2014. "intra project team disagreement, conflict communications, and team performance in cross-functional new product teams: a decision-making quality perspective," south african journal of economic and management sciences (17:1), 2014, pp 91-107. chu, c., and smithson, s. 2007. "e-business and organizational change: a structurational approach," information systems journal (17:4), october, pp 369-389. clegg, s., harris, m., and hopfl, h. 2011. managing modernity beyond bureaucracy? oxford: oxford university press. clegg, s.r. 1989. "circuits of power: a framework for analysis," in: frameworks of power, s.r. clegg (ed.). london: sage publications, pp. 187-240. currie, g., finn, r., and martin, g. 2010. "role transition and the interaction of relational and social identity: new nursing roles in the english nhs," organization studies (31:7), july 1, 2010, pp 941-961. dahl, r. 1957. "the concept of power," behavioural science (2:3), july, pp 201-215. deutsch, m., and gerard, h.b. 1955. "a study of normative and informational social influences upon individual judgment," journal of abnormal & social psychology (51:3), pp 629-636. doolin, b. 2004. "power and resistance in the implementation of a medical management information system," information systems journal (14:4), pp 343-362. flowers, s. 1997. "information systems failure: identifying the critical failure factors," in: failure and lessons learned in information technology management: an international journal, j. liebowitz (ed.). elmsford, new york: cogizant communication corp., pp. 19-30. foucault, m. 1977. discipline and punish: the birth of the prison. london: penguin. french, j.r.p., and raven, b.h. 1959. "the bases of social power," in: studies in social power, d. cartwright (ed.). ann arbor, michigan: the university of michigan, pp. 150-167. fritsche, i., jonas, e., ablasser, c., beyer, m., kuban, j., manger, a.-m., and schultz, m. 2013. "the power of we: evidence for group-based control," journal of experimental social psychology (49:1), pp 19-32. gohler, g. 2009. "'power to' and 'power over'," in: the sage handbook of power, s. clegg and m. haugaard (eds.). london: sage publications. haslam, s.a., ryan, m.k., postmes, t., spears, r., jetten, j., and webley, p. 2006. "sticking to our guns: social identity as a basis for the maintenance of commitment to faltering australasian journal of information systems ye, de salis, ollington & mckay 2017, vol 21, research article applying turner’s theory of power in an troubled is 21 organizational projects," journal of organizational behavior (27:5), august, pp 607628. hogg, m.a., martin, r., epitropaki, o., mankad, a., svensson, a., and weeden, k. 2005. "effective leadership in salient groups: revisiting leader-member exchange theory from the perspective of the social identity theory of leadership," personality and social psychology bulletin (31:7), july 1, 2005, pp 991-1004. hogg, m.a., and turner, j.c. 1985. "interpersonal attraction, social identification and psychological group formation," european journal of social psychology (15:1), january, pp 51-66. hornsey, m.j. 2008. "social identity theory and self-categorization theory: a historical review," social and personality psychology compass (2:1), january, pp 204-222. hornsey, m.j., blackwood, l., and o'brien, a. 2005. "speaking for others: the pros and cons of group advocates using collective language," group processes & intergroup relations (8:3), jul 2005, pp 245-257. howcroft, d., and light, b. 2006. "reflections on issues of power in packaged software selection," information systems journal (16:3), july, pp 215-235. hunsberger, k. 2011. "the accidental project manager," in: pm network. drexel hill: project management institute, pp. 28-33. hussain, z.i., and cornelius, n. 2009. "the use of domination and legitimation in information systems implementation," information systems journal (19:2), march, pp 197-224. jasperson, j., carte, t.a., saunders, c.s., butler, b.s., croes, h.j.p., and zheng, w. 2002. "review: power and information technology research: a metatriangulation review," mis quarterly (26:4), december, pp 397-459. kappelman, l.a., mckeeman, r., and zhang, l. 2006. "early warning signs of it project failure: the dominant dozen," information systems management (23:4), fall, pp 3136. kerr, d.v., houghton, l., and burgess, k. 2007. "power relationships that lead to the development of feral systems," australasian journal of information systems (14:2), pp 141-152. klein, h.k., and myers, m.d. 1999. "a set of principles for conducting and evaluating interpretive field studies in information systems," mis quarterly (23:1), pp 67-94. knights, d., and vurdudakis, t. 1994. "foucault, power, resistance and all that," in: resistance and power in organizations, j.m. jermier, d. knights and w.r. nord (eds.). london and new york: routledge, pp. 167-198. kramer, r.m. 1999. "trust and distrust in organizations: emerging perspectives, enduring questions," annual review of psychology (50:1), february, pp 569-598. markus, m.l., and bjorn-andersen, n. 1987. "power over users: its exercise by system professionals," communications of the acm (30:6), june, pp 498-504. mathiassen, l., and napier, n. 2014. "exploring win-win contracts: an appreciative inquiry into it project management," journal of information technology theory and application (14:3), pp 5-29. myers, m.d., and newman, m. 2007. "the qualitative interview in is research: examining the craft," information and organization (17:1), pp 2-26. ngwenyama, o., and nielsen, p.a. 2014. "using organizational influence processes to overcome is implementation barriers: lessons from a longitudinal case study of spi australasian journal of information systems ye, de salis, ollington & mckay 2017, vol 21, research article applying turner’s theory of power in an troubled is 22 implementation," european journal of information systems (23:2), mar 2014, pp 205222. oakes, p.j., haslam, s.a., and turner, j.c. 1994. "outgroup homogeneity and illusory correlation revisited," in: stereotyping and social reality, p.j. oakes, s.a. haslam and j.c. turner (eds.). oxford, uk: blackwell, pp. 160-185. orlikowski, w.j., and baroudi, j.j. 1991. "studying information technology in organizations: research approaches and assumptions," information systems research (2:1), march, pp 1-28. oz, e., and sosik, j. 2000. "why information systems projects are abandoned: a leadership and communication theory and exploratory study," the journal of computer information systems (41:1), fall, pp 66-78. sabherwal, r., and grover, v. 2010. "a taxonomy of political processes in systems development," information systems journal (20:5), pp 419-447. sachdev, i., and bourhis, r.y. 1985. "social categorization and power differentials in group relations," european journal of social psychology (15:4), pp 415-434. schwandt, t.a. 1994. "constructivist, interpretivist approaches to human inquiry," in: handbook of qualitative research, n.k. dezin and y.s. lincoln (eds.). thousand oaks, ca: sage. schwarz, g.m., and watson, b.m. 2005. "the influence of perceptions of social identity on information technology-enabled change," group & organization management (30:3), june, pp 289-318. silva, l. 2007. "epistemological and theoretical challenges for studying power and politics in information systems," information systems journal (17:2), april, pp 165-183. silva, l., and backhouse, j. 2003. "the circuits-of-power framework for studying power in institutionalization of information systems," journal of the association for information systems (4:6), november, pp 294-336. silva, l., and fulk, h.k. 2012. "from disruptions to struggles: theorizing power in erp implementation projects," information and organization (22:4), october, pp 227-251. simon, b., and oakes, p. 2006. "beyond dependence: an identity approach to social power and domination," human relations (59:1), january, pp 105-139. smith, s., winchester, d., bunker, d., and jamieson, r. 2010. "circuits of power: a study of mandated compliance to an information systems security de jure standard in a government organization," mis quarterly (34:3), september, pp 463-486. strauss, a.l., and corbin, j. 1990. basics of qualitative research: grounded theory procedures and techniques. thousand oaks, ca: sage publications. tajfel, h., flament, c., billig, m.g., and bundy, r.f. 1971. "social categorization and intergroup behaviour," european journal of social psychology (1:2), april/june, pp 149-178. tajfel, h., and turner, j.c. 1979. "an integrative theory of intergroup conflict," in: the social psychology of intergroup relations, w.g. austin and s. worchel (eds.). monterey, ca: brooks/cole, pp. 33-47. tansley, c., huang, j., and foster, c. 2013. "identity ambiguity and the promises and practices of hybrid e-hrm project teams," the journal of strategic information systems (22:3), pp 208-224. turner, j.c. 1978. "social categorization and social discrimination in the minimal group paradigm," in: differentiation between social groups: studies in the social psychology of intergroup relations, h. tajfel (ed.). london: academic press, pp. 101-140. australasian journal of information systems ye, de salis, ollington & mckay 2017, vol 21, research article applying turner’s theory of power in an troubled is 23 turner, j.c. 1984. "social identification and psychological group formation," in: the social dimension, h. tajfel (ed.). cambridge: cambridge university press, pp. 518-538. turner, j.c. 1987. "a self-categorization theory," in: rediscovering the social group: a selfcategorization theory, j.c. turner, m.a. hogg, p.j. oakes, s.d. reicher and m. wetherell, s. (eds.). oxford, uk: blackwell, pp. 42-67. turner, j.c. 1991. "the dual-process model, self-categorization and social influence," in: social influence, j.c. turner (ed.). milton keynes: open university press, pp. 143-173. turner, j.c. 2005. "explaining the nature of power: a three-process theory," european journal of social psychology (35:1), january, pp 1-22. turner, j.c., hogg, m.a., oakes, p.j., reicher, s.d., and wetherell, m.s. 1987. rediscovering the social group: a self-categorization theory. oxford, uk: blackwell. turner, j.r., keegan, a.e., and crawford, l.h. 2003. "delivering improved project management maturity through experiential learning," in: people in project management, j.r. turner (ed.). aldershot, uk: gower. van dijk, r., and van dick, r. 2009. "navigating organizational change: change leaders, employee resistance and work-based identities," journal of change management (9:2), april, pp 143-163. ye, m., marshall, p., mckay, j., and de salas, k. 2014. "power relations in information systems implementation: the potential contribution of turner's three-process theory of power," in: business technologies in contemporary organizations: adoption, assimilation, and institutionalization, a. haider (ed.). hershey, pa, usa: igi global. yukl, g., falbe, c.m., and youn, j.y. 1993. "patterns of influence behaviour for managers," group & organization management (18:1), march, pp 5-28. copyright: © 2017 ye, de salas, ollington & mckay. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems ye, de salis, ollington & mckay 2017, vol 21, research article applying turner’s theory of power in an troubled is 24 appendix application of klein and myers’ evaluation criteria to the case study australasian journal of information systems ye, de salis, ollington & mckay 2017, vol 21, research article applying turner’s theory of power in an troubled is 25 paper title (paper title style) australasian journal of information systems lim & burmeister 2015, vol. 19, pp. p7-p10 post publication review post publication review burmeister, o. k. (2015). improving professional it doctorate completion rates. australasian journal of information systems, 19, 55-70. doi: http://dx.doi.org/10.3127/ajis.v19i0.1073 review completion and attrition rates for doctoral degrees are a topic of pressing importance among universities (e.g. top management, faculty members, graduate students) and funding agencies (e.g. public funding from government agencies, private funding from profitand non-profitoriented enterprises) worldwide (council of graduate schools, 2008a, 2008b, 2009, 2010). a recent qualitative study by burmeister (2015) made a commendable attempt to address this problem by examining the enabling conditions that facilitate and the obstacles that impede the successful completion of doctoral degrees, particularly among students who enroll in professional doctorates in the field of information technology. that study finds that the keys to improving the completion rate and reducing the attrition rate of doctoral programs are situated in seven broad areas: retention, student engagement with supervisors, supervision appointment, feedback on progress, student engagement in the course, contribution, and student involvement in institutional communities of practice. despite some notable insights and useful implications from burmeister’s study, this rejoinder makes several observations that could have improved the quality of the paper. in the initial parts of the paper, burmeister makes several distinctions between the types of doctoral programs that exist in educational institutions, including fulland part-time research and professional doctorates. this raises several important, interrelated questions on the relevance of making these distinctions—for example, is there a need to distinguish the different types of doctoral curricula? if yes, what are the motivations/rationales for making such distinctions and what are the implications that could have been overlooked if such distinctions were absent? thus, clarifying the importance of making distinctions within a broad area of study should help solidify the motivations/rationales for conducting context-specific investigations. next, burmeister reviews the extant literature on doctoral education, particularly in the areas of completion rates and attrition risks in doctoral programs and communities of practice, followed by some contextual background pertaining to the information technology professional doctorate under study. however, the review is generally descriptive, and thus a sense of critical evaluation of the current state of literature in the area is lacking. instead, the paper would have benefited from some critical reflection on the extant literature entwined with the context of the study itself, such as by elucidating the limitations of existing studies in explaining how to improve completion rates and reduce attrition rates in professional doctorates in the field of information technology and by indicating how the current study contributes to extending the line of inquiry in the area. for example, the paper could have highlighted the point at which the current study adopted a practical rationality route to investigation to explain the rationales behind attritions in doctoral degrees, thus closing the theory–practice gap that is prominent in existing studies that have followed a scientific rationality route to investigation (sandberg & tsoukas, 2011). following the literature review, burmeister explains the methodological aspects of the study— a thematic analysis was conducted using an interpretivist, constructivist approach on a sample of 44 semi-structured interviews consisting of supervisors, students from the information technology research and professional doctoral programs, and university support staff. however, the methodological discussion appears inadequate, as important information pertaining to the rationale for sample selection in terms of the number and type of people interviewed is absent in the paper, which in turn raises concern about data saturation and appropriateness of the sample to the research objectives and questions. moreover, the trustworthiness (i.e. credibility, dependability, transferability, and confirmability; similar to p7 australasian journal of information systems lim & burmeister 2015, vol. 19, pp. p7-p10 post publication review concepts of reliability and validity in quantitative studies) of the thematic analysis and subsequent interpretations are not addressed in the paper. the disclosure of such information is paramount to determine and establish rigor in the execution of and reporting of findings in a qualitative study (guba & lincoln, 1989). last, burmeister discusses the results of the study with regard to the improvements in and impediments to completion rates of information technology professional doctorates, but the implications of the findings to information technology and educational theory and practice are relatively weak. for example, it is unclear whether the findings from the current study are in line (or in opposition) with existing studies in the area (e.g. in comparison with studies on completion and attrition rates in other fields or types of doctoral programs; e.g. litalien & guay, 2015); it is also unclear which findings can be deemed as “new” from the current investigation. moreover, much ambiguity persists in several of the study’s themes, such as the barriers to progression and further research directions for “retention,” aids to progression for “feedback on progress,” and policy implications for “student engagement in the course” and “contribution.” addressing these issues would have contributed to a “more complete” paper. in short, the study excels as a form of “good research” (as it addresses an important and pressing issue in contemporary academic and professional community) but falls short of becoming a “good paper” (due to identified limitations in the paper). it is the hope of this rejoinder that the issues raised and suggestions offered will contribute to improve the quality of papers in top-tier journals, especially those accepted and published in australasian journal of information systems. weng marc lim school of business monash university (malaysia campus) lim.weng.marc@monash.edu acknowledgment many thanks to john lamp for introducing the post-publication review section in australasian journal of information systems—it is certainly a good avenue for promoting healthy academic discourse. also, deepest appreciation goes to oliver k. burmeister for embarking on a meaningful journey to improve the academic and professional ecosystem, particularly in the area of information technology professional doctorates. references burmeister, o. k. (2015). improving professional it doctorate completion rates. australasian journal of information systems, 19, 55-70. doi: http://dx.doi.org/10.3127/ajis.v19i0.1073 council of graduate schools. (2008a). ph.d. completion and attrition: analysis and baseline demographic data from ph.d. completion project. washington, dc: council of graduate schools. council of graduate schools. (2008b). ph.d. completion and attrition: analysis and baseline program data. washington, dc: council of graduate schools. council of graduate schools. (2009). ph.d. completion and attrition: findings from exit surveys of ph.d. completers. washington, dc: council of graduate schools. council of graduate schools. (2010). ph.d. completion and attrition: policies and practices to promote student success. washington, dc: council of graduate schools. guba, e. g., & lincoln, y. s. (1989). fourth generation evaluation. newbury park, ca: sage publications. litalien, d., & guay, f. (2015). dropout intentions in phd studies: a comprehensive model based on interpersonal relationships and motivational resources. contemporary p8 australasian journal of information systems lim & burmeister 2015, vol. 19, pp. p7-p10 post publication review educational psychology, 41, 218-231. doi: http://dx.doi.org/10.1016/j.cedpsych.2015.03.004 sandberg, j., & tsoukas, h. (2011). grasping the logic of practice: theorizing through practical rationality. academy of management review, 36 (2), 338-360. doi: http://dx.doi.org/10.5465/amr.2011.59330942 author response the introduction of peer rejoinders and responses is as dr lim points out, a good avenue for promoting healthy academic discourse. i also appreciate that he made an effort to critically review my recent article. in the main, i agree with dr lim’s comments. taking them in turn, firstly he points out that my distinctions between the types of doctoral programs (full-time or part-time, and research or professional doctorates) could be subject to debate. indeed, he did not say ‘phd’ and professional doctorates, but ‘research’ and professional doctorates. however, as i think i demonstrated in the paper, i believe that the professional doctorate is a research degree. still it simply confirms his point that these distinctions require further investigation. pedagogically i do think that a part-time professional doctorate undertaken by a mid-career professional, should be distinguished from a full-time phd, which is often undertaken by a young, single person about to embark on a professional career. another point dr lim makes is that there is a need to close the theory-practice gap. i totally agree and, like the point above, think this is another avenue that requires further research in regards to it doctorates. he then rightly points out the need for methodological rigour. here and elsewhere he would have liked more detail, and later he questions the review process for the journal. actually the reviewers similarly wanted more in the methodology and elsewhere, but at over 8,000 words, it was already over the size limits for the journal and therefore in the end a compromise had to be reached between what would be in the paper and which bits were more important to include than others. i hope to publish more from this study and dr lim’s comments help me to see what i might focus on for future articles. finally, dr lim points to phd literature to suggest that my article needs better alignment with phd studies in other disciplines. however, although i included phd studies, my focus was professional doctorates in it. much has been written about doctorates, especially phds and most of it is in area related to education disciplines. very little exists in it. academic discourse such as this will hopefully lead to more research on professional and other doctorate models in it. oliver k. burmeister school of computing and mathematics charles sturt university oburmeister@csu.edu.au copyright: © 2015 lim & burmeister. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. p9 australasian journal of information systems lim & burmeister 2015, vol. 19, pp. p7-p10 post publication review p10 paper title (paper title style) australasian journal of information systems dang-pham, kautz, pittayachawan & bruno 2019, vol 23, selected papers from acis 2017 inf security climate & support network 1 explaining the development of information security climate and an information security support network: a longitudinal social network analysis duy dang-pham school of science and technology rmit vietnam school of business it and logistics rmit university duy.dangphamthien@rmit.edu.vn karlheinz kautz school of business it and logistics rmit university siddhi pittayachawan school of business it and logistics rmit university vince bruno school of business it and logistics rmit university abstract: behavioural information security (infosec) research has studied infosec at workplaces through the employees’ perceptions of infosec climate, which is determined by observable infosec practices performed by their colleagues and direct supervisors. prior studies have identified the antecedents of a positive infosec climate, in particular socialisation through the employees’ discussions of infosec-related matters to explain the formation of infosec climate based on the employees’ individual cognition. we conceptualise six forms of socialisation as six networks, which comprise employees’ provisions of (1) work advice, (2) organisational updates, (3) personal advice, (4) trust for expertise, (5) infosec advice, and (6) infosec troubleshooting support. the adoption of a longitudinal social network analysis (sna), called stochastic actororiented modelling (saom), enabled us to analyse the changes in the socialising patterns and the infosec climate perceptions over time. consequently, this analysis explains the forming mechanisms of the employees’ infosec climate perceptions as well as their socialising process in greater detail. our findings in relation to the forming mechanisms of infosec-related socialisation and infosec climate, provide practical recommendations to improve organisational infosec. this includes identifying influential employees to diffuse infosec knowledge within a workplace. additionally, this research proposes a novel approach for infosec behavioural research through the adoption of sna methods to study infosec-related phenomena. keywords: behavioural information security; information security climate; information security management; social network analysis; stochastic actor-oriented modelling australasian journal of information systems dang-pham, kautz, pittayachawan & bruno 2019, vol 23, selected papers from acis 2017 inf security climate & support network 2 1 introduction as organisations are rapidly adopting innovative technology and strategic information systems to support data-intensive operations, maintaining organisational information security (infosec) has become a priority. technical infosec measures such as anti-virus software and firewalls are no longer sufficient for organisational infosec because of the increased targeting of organisational employees. therefore, management must find effective ways to equip their employees with adequate infosec knowledge and skills. these managerial efforts help develop a secure workplace where employees are aware of their infosec duties and voluntarily perform infosec behaviours enabling adequate organisational infosec to be achieved. there has been an emerging research theme in the behavioural infosec research field that focuses on the relationship between the work environment and the employees’ infosec perceptions and behaviours. willison and warkentin (2013) extended the security action cycle by adding ‘pre-kinetic events’ that were explained to result in the employees’ negative perceptions of the workplace subsequently leading to infosec abuses. baskerville, park and kim (2014) analysed the workplace’s infosec vulnerabilities exploitable by potential perpetrators to commit infosec violations. to prevent insider threats and raise infosec awareness, situational support and peer-learning are critical for educating the employees about the skills and knowledge to comply with infosec directives (warkentin, johnston, & shropshire, 2011). the active sharing of infosec advice among employees also reduces the time spent on re-inventing infosec solutions and enables efficient allocation of organisational resources to other infosec tasks of higher importance (safa, solms, & von solms, 2016). negative perceptions of the workplace, exploitable vulnerabilities, situational and peerlearning support, and the sharing of infosec advice among employees are some of the topics discussed in the research literature. we argue that there are two critical issues concerning intra-organisational provisions of infosec resources such as infosec advice and troubleshooting support. they focus on: (1) the ways to provide these infosec resources and (2) the effectiveness of such provisions measured by improvements in the employees’ infosec perceptions and behaviours. recent research has primarily addressed the first issue by identifying the factors which facilitate the employees’ sharing of infosec knowledge in the organisational context (rocha flores, antonsen, & ekstedt, 2014; safa et al., 2016). measuring the effectiveness of the provisions of infosec resources is challenging, since it ideally requires a longitudinal design to observe improvements in the employees’ and organisational infosec before and after such provisions. there have been a few action research projects which monitored infosec-related interventions and evaluated their impacts throughout the research process (see e.g., puhakainen and siponen 2010; tsohou et al. 2013). however, these studies focus on the changes in the employees’ individual perceptions and behaviours and overlook the changes in the infosec environment. acquiring a comprehensive understanding about the social dynamics that constantly take place within a work environment, which entail how the employees’ infosec perceptions shape their provisions of infosec resources and vice versa, can reveal opportunities to enhance the organisational infosec through manipulating features of the work environment. on this background, the objective of this research is to acquire such comprehensive understanding about the simultaneous formation of the employees’ infosec perceptions and of the infosec environment conceptualised as a network of infosec support provisions between the employees. australasian journal of information systems dang-pham, kautz, pittayachawan & bruno 2019, vol 23, selected papers from acis 2017 inf security climate & support network 3 we employed a longitudinal social network analysis (sna) method, called stochastic actororiented modelling (saom) (steglich, snijders, & pearson, 2010), to investigate the relationship between the employees’ infosec perceptions and their socialisation in a large organisation before and after an infosec awareness program was implemented. we selected the sna approach and saom method because it allowed a simultaneous analysis of both the individual factors (i.e., the employees’ infosec perceptions) and environmental factors (i.e., the employees’ socialisation and its structural features). we explored the concept of infosec climate (chan, woon, & kankanhalli, 2005; schneider & reichers, 1983) and its forming process which was facilitated by the employees’ socialisation (ashforth, 1985; weick, 1995). unlike current studies which conceptualised socialisation as part of the employees’ perceptions (see e.g., chan et al., 2005; goo, yim, & kim, 2014; jaafar & ajis, 2013), we analysed socialisation in the form of networks with network ties representing the actual provisions of work advice, organisational updates, trust for expertise, infosec advice and troubleshooting support among employees. longitudinal data between two separate points in time were collected and analysed to reveal the changes in network ties and in climate perceptions over time. we applied and present an original approach to study infosec-related phenomena, which employed sna method to examine the networks of infosec-related interactions in conjunction with the individual employee’s infosec perceptions. through the analysis, we explored the socialising mechanisms brought about by these networks, which facilitated the formation of infosec climate perceptions, as well as the impacts of the networks on each other. acquiring knowledge of these socialising mechanisms enables the identification of those employees influential in the infosec domain, who can assist management in raising organisational infosec awareness through their influence. 2 conceptual framework in the following sections, we discuss the concepts of infosec climate and workplace socialisation. then, we explain the relationship between these two concepts by consulting social influence theories. based on this we propose a conceptual framework that describes the key mechanisms contributing to the formation of infosec climate within a workplace. this conceptual framework and the described mechanisms will be evaluated by the saom method. 2.1 infosec climate perceptions infosec climate refers to the common and recognised practices in the workplace, defining how infosec is treated by the organisation (lowry & moody, 2013). the concept of infosec climate has been adopted by infosec researchers from the work safety climate literature. chan et al. (2005) defined it as the employees’ perceptions of the observed infosec environment, consisting of their colleagues and supervisors’ infosec behaviours. infosec climate fosters and maintains infosec compliance and infosec culture (chan et al., 2005), as well as promotes the provisions of infosec advice and troubleshooting support (dang-pham, pittayachawan, & bruno, 2017). the perceptions of the infosec climate, in this research, are defined as comprising of the employees’ perceptions of colleagues and supervisors’ infosec behaviours (chan et al., 2005; goo et al., 2014; jaafar & ajis, 2013). these perceptions indicate how infosec matters are treated and prioritised in the workplace (lowry & moody, 2013). dourish and anderson (2006) discuss that infosec-related perceptions and behaviours have both individualistic and collective characteristics. in daily work, the employees’ decisions to australasian journal of information systems dang-pham, kautz, pittayachawan & bruno 2019, vol 23, selected papers from acis 2017 inf security climate & support network 4 perform infosec behaviours are influenced by factors in the work environment (padayachee, 2012; sommestad, hallberg, lundholm, & bengtsson, 2014). subjective norms, which refer to the pressure exerted by the people who appear important to a person, are confirmed across studies as an important contributing factor of both desirable and malicious infosec behaviours (bulgurcu, cavusoglu, & benbasat, 2010; cheng, li, li, holm, & zhai, 2013; guo & yuan, 2012; herath & rao, 2009; ifinedo, 2014; lee & lee, 2002). subjective norms describe the shared patterns of thoughts and behaviours that can be constructed via communication (hogg & reid, 2006). recent behavioural infosec research has investigated the sharing of infosec advice as a means to raise employees’ infosec awareness (rocha flores et al., 2014; safa et al., 2016; warkentin et al., 2011), and such sharing was also found to create normative infosec behaviours such as risky infosec workarounds (kirlappos, parkin, & sasse, 2014). this has led the research to the following discussion of the socialisation’s impact on the formation of infosec climate perceptions. 2.2 socialisation and the formation of infosec climate perceptions socialisation refers to an employee’s inclusion in a workplace through communication and provisions of organisational resources, which facilitate organisational learning and development of organisational climates (morrison, 1993); such socialisation plays a major role in the formation of infosec climate by facilitating the sense-making activities among employees (chan et al., 2005; goo et al., 2014; jaafar & ajis, 2013) to reach a consensus on meanings (ashforth, 1985; schneider & reichers, 1983; weick, 1995). socialisation in the forms of provisions of work advice and trust helps to reduce uncertainty, i.e., lack of information, and clarifies ambiguity, i.e., too many overlapping information, respectively (saint-charles & mongeau, 2009). in the infosec context, the socialisation among employees, their colleagues and direct supervisors has been conceptualised as involving discussions about infosec-related matters (chan et al., 2005; goo et al., 2014; jaafar & ajis, 2013). the increased infosec-related socialisation of employees with their colleagues and supervisors raises awareness of the infosec practices employed by their organisation, which contributes to the development of an infosec climate (chan et al., 2005). the socialisation within a workplace can be conceptualised and studied in two different forms, either reflecting the individual’s perceived level of socialisation or characterising the actual social interactions between pairs of employees. the former form has been the focus of previous infosec climate research (chan et al., 2005; goo et al., 2014; jaafar & ajis, 2013), whereas studies in the management and organisational behaviour disciplines have investigated workplace socialisation in the form of organisational networks (see e.g. ibarra & andrews, 1993; saintcharles & mongeau, 2009). organisational networks can be categorised into instrumental and expressive interaction networks (borgatti, everett, & johnson, 2013; ibarra & andrews, 1993; saint-charles & mongeau, 2009). in line with our research context, we further add a third type of networks i.e., the infosec support network that comprises infosec-related interactions between the employees. we explain the three types of networks as follow. instrumental networks refer to the interactions and relationships that are related to work-role and work situations, and expressive networks are the links that concern emotional matters such as friendship and social support (ibarra & andrews, 1993; saint-charles & mongeau, 2009). in line with these prior studies, we considered the employees’ provision of work advice as part of the instrumental networks. since performing infosec behaviours requires australasian journal of information systems dang-pham, kautz, pittayachawan & bruno 2019, vol 23, selected papers from acis 2017 inf security climate & support network 5 understanding relevant policies and directives, individuals who have first access to the latest organisational updates can be perceived as influential by other employees. the instrumental network in our study is thus defined as comprising the provisions of work advice and organisational updates. consistent with organisational network studies, the expressive networks in our research comprised the employees’ provision of personal advice and trust. interpersonal trust is multifaceted, which involves trusting a person for their expertise and personal characteristics such as benevolence and integrity (mcknight, 2002; mcknight & chervany, 1996). while nominating a person capable of discussing personal matters indicates the nominator’s trust in the person’s perceived goodness, the ‘trust for expertise’ network consisting of nominated people who are trusted for expertise covers the other facet of trust. as a result, the expressive network in our study consists of the socialisation that facilitates the provision of personal advice and trust for expertise. the infosec support network refers to the provisions of infosec advice and troubleshooting support among the employees. in the infosec context, the provision of infosec advice and troubleshooting support represent the socialisation which involves discussions about infosec matters that shape the employees’ perceptions of infosec climate (chan et al., 2005). the nomination as a person that is capable of providing infosec advice and troubleshooting support indicates that the nominee possesses expert power to influence other colleagues’ infosec perceptions and behaviours (french & raven, 1959). in summary, our study is concerned with three types of organisational networks, namely the instrumental, expressive, and infosec support networks. these represent the different types of workplace socialisation that would form the employees’ perceptions of infosec climate. in the next section, we discuss the two different mechanisms, i.e., selection and influence processes, to explain how employees would shape their infosec climate by socialising with their colleagues within the three aforementioned networks. these processes, together with the organisational networks, provide the conceptual framework that will be examined with the longitudinal sna method. 2.3 selection and influence processes the formation of infosec climate is achieved when the employees reach a consensus on or institutionalise common perceptions of infosec climate through their socialisation. there are two possible explanations for that phenomenon. first, employees may deliberately select and socialise with those whom they see as possessing similar traits; this is also called the homophily effect (borgatti et al., 2013; mcpherson, smith-lovin, & cook, 2001). on this basis, employees who have similar climate perceptions may already form ties and socialise with each other without changing perceptions, i.e., the influence process does not take place. second, employees may institutionalise common infosec climate perceptions as a result of their socialisation, or in other words there is an influence process that makes the employees adjust their perceptions of infosec climate to match with those of their socialised partners. these two processes are referred to as selection and influence processes respectively (steglich et al., 2010). the relationship between selection and influence processes has been a recurring topic of debate and prominent explanation for social phenomena that involve collective behaviours and perceptions (steglich et al., 2010). understanding the impacts of selection and influence processes on a behaviour or perception offers some benefits. for instance, a confirmed effect australasian journal of information systems dang-pham, kautz, pittayachawan & bruno 2019, vol 23, selected papers from acis 2017 inf security climate & support network 6 of social influence on an infosec behaviour, e.g., infosec compliance or violation, would justify the implementation of interventions that aim to alter peers’ behaviours to create contagious changes. in contrast, a confirmed selection effect would indicate that similar behaviours in the workplace are formed and maintained by the deliberate socialisation between employees of matching profiles. therefore, to change those behaviours would require interventions that modify or remove the existing socialisation between people, rather than focusing on changing the behaviours per se (dijkstra et al., 2010). prior studies on infosec climate (chan et al., 2005; goo et al., 2014; jaafar & ajis, 2013), which analysed the effect of socialisation on the employees’ infosec climate perceptions, have overlooked the distinction between selection and influence processes. this is due to the traditional research approach that solely conceptualises the employees’ socialisation as their perceptions and not in the form of networks. consequently, it remains unclear how infosec climate perceptions can be influenced by the workplace socialisation, although the relationship between these two constructs has been confirmed by existing research. figure 1. conceptual framework figure 1 presents the conceptual framework which summarises the key components in our research, i.e., the employees and their personal attributes, and the different network ties between them which represent their socialisation. the solid and dash lines represent existing and potential ties respectively, of which the latter is assumed to be the possible outcome of the former. specifically, we assume that employees would socialise with their colleagues in a network, e.g., seeking infosec advice, if they are already socialising in another network, e.g., seeking work advice. it would be more likely for employees to prefer interacting with colleagues whom they trust and know their expertise well because of their previous interactions. in line with the selection process described above, the occurrence of ties is also assumed to be motivated by employees who have certain demographics such as age and gender. the assumed effect of demographics on the employees’ socialisation is consistent with social australasian journal of information systems dang-pham, kautz, pittayachawan & bruno 2019, vol 23, selected papers from acis 2017 inf security climate & support network 7 influence and homophily theories (french & raven, 1959; ibarra & andrews, 1993; mcpherson et al., 2001), which posit that employees who have certain traits (e.g., seniority, tenure) are perceived more influential, and that people purposely choose to interact with similar others (e.g., having the same gender or working in the same department). consistent with infosec climate studies (chan et al., 2005; goo et al., 2014; jaafar & ajis, 2013), the emphasis of our conceptual framework is on the impact of the infosec support network on the employees’ infosec climate perceptions, particularly the perceptions of the infosec behaviours performed by their colleagues and direct supervisors. the up-and-down arrows next to the climate perceptions component in figure 1 highlight our research interest that focuses on the changing patterns of such perceptions as a consequence of the employees providing and receiving infosec support to/from each other. in line with the selection process and homophily effect discussed above, we also assume that the likelihood for the employees to establish ties is affected by having different levels of climate perceptions, i.e., high or low perceptions. 3 research method we employed a longitudinal sna method called saom which simultaneously analyses the employees’ perceptions of infosec climate and the patterns of socialisation in the form of networks. the saom method was developed especially for the purpose of detecting the selection and influence processes by separating them during the analysis (steglich et al., 2010). the following sections discuss the research context, the variables which we included in the questionnaire to collect data, the data collection process, and the preparation for the saom analysis. 3.1 research context we were approached by a large construction and manufacturing enterprise in vietnam (anonymised as ‘abc’) to provide advice and help improve their employees’ infosec awareness. the study reported here is part of the research project we subsequently performed together with the organisation. abc is one of the largest enterprises in vietnam, which employed a total of 311 office staff and more than 800 workers who worked at the manufacturing facility, at the time when our research was conducted. the company’s main services include designing, manufacturing and exporting high quality furniture, as well as delivering fitting projects for local and international clients. due to the increased infosec violations that had occurred, abc was motivated to improve the infosec environment by stimulating infosec-related interactions i.e., the sharing of infosec advice and troubleshooting support among the office staff. the collaboration with abc in this project on the diffusion of infosec knowledge provided a research opportunity to study the changing dynamics in an infosec environment. in line with the emerging theme of behavioural infosec in the literature, we advised abc to instigate, measure and analyse the changes in the employees’ infosec climate perceptions, which indicated the priority of infosec in the workplace, and the provisions of infosec resources between them by using sna methods. the adoption of sna techniques to design and implement organisational changes, which includes making use of opinion leaders to diffuse new ideas, has been applied in prior studies (cross, laseter, parker, & guillermo, 2006; valente, palinkas, czaja, chu, & brown, 2015). australasian journal of information systems dang-pham, kautz, pittayachawan & bruno 2019, vol 23, selected papers from acis 2017 inf security climate & support network 8 since abc’s objective was to increase the provisions of infosec resources among their employees, sna methods offer the analytical capabilities to quantitatively evaluate the improvements in the networks representing such provisions. we identified a number of infosec champions and conducted small infosec training sessions for these champions, who were then tasked to diffuse infosec knowledge to other colleagues in abc. 3.2 variables we organised the six forms of socialisation, the provisions of work advice, organisational updates, personal advice, trust for expertise, infosec advice and troubleshooting support in our study, into three types of networks: instrumental, expressive and infosec support. these networks represent the socialisation that facilitates sense-making activities and forms infosec climate perceptions. the questions for establishing these networks and their categories are summarised in table 5 appendix a. we used two sets of questions adapted from studies on infosec climate (chan et al., 2005; goo et al., 2014; jaafar & ajis, 2013) to measure the employees’ perceptions of their colleagues and direct supervisors’ infosec behaviours, which describe their perceptions of infosec climate. we employed a seven-point scale to ask 10 questions about the level of infosec behaviours performed by the employees’ colleagues and direct supervisors (see table 6 appendix a). we also included background characteristics of the employees as variables in our sao model. the employees’ demographics, namely age, gender, tenure (in years), seniority (operational, manager, and executive), champion status, and department membership were collected by an additional set of relevant questions. 3.3 data collection and preparation we collected our data in two waves by launching the same questionnaire twice. data collection of wave 1 was performed before the diffusion of infosec knowledge took place, and wave 2 took place three months after the diffusion. the questionnaire had two sections; the first section captured the six networks of interest by asking the employees to nominate a maximum of seven colleagues who engaged with them in a network (refer to table 5 appendix a). they could nominate multiple colleagues who socialised with them in the different networks. the second section asked the employees to answer the likert scale questions that measure their climate perceptions of colleagues and supervisors’ infosec behaviours (table 6 appendix a). questionnaires designed to collect data about whole networks require identifiable information such as the respondents’ real names, which can be a source of common method bias and affect the response rate since employees may not want to be identified and evaluated for their responses (borgatti et al., 2013; podsakoff, mackenzie, lee, & podsakoff, 2003). to alleviate this issue, the top management of abc published a memorandum of understanding that they will not access the identifiable data. we also explained to the employees in the questionnaire that their real names and nominations are collected for the purpose of mapping out the networks only. to further minimise common method bias, we mixed the order of questions so that they did not appear to measure the same constructs, as well as provided detailed definitions and examples to clarify potentially vague questions (podsakoff et al., 2003). missing data is problematic for longitudinal research and can result in biased findings without proper treatment (ripley, snijders, & preciado, 2011). whole-network research design, which focuses on a bounded environment, can have missing data due to the respondents’ refusal to participate in the survey or their exit of the bounded environment (borgatti et al., 2013). our australasian journal of information systems dang-pham, kautz, pittayachawan & bruno 2019, vol 23, selected papers from acis 2017 inf security climate & support network 9 questionnaire was sent to 311 office staff employees of abc in wave 1, which returned 264 responses, a response rate of 85 per cent. in wave 2, we collected 230 responses from a total of 288 employees, a response rate of 80 per cent. we trimmed the dataset by selecting only the employees who casted their nominations in both waves. this meant the exclusion of respondents who were only nominated in one of the two waves and of those who did not make any nominations, or casted their nominations only once in either wave 1 or wave 2. this resulted in a dataset consisting of 151 employees. performing saom analysis requires individuals’ perceptions and behaviours to be represented by a single-item, composite variable in integer format (ripley et al., 2011). in our study, this requirement also applies to the theoretical constructs of perceptions of infosec climate. we performed confirmatory factor analysis (brown, 2006) to fit measurement models for the infosec climate perceptions of colleagues (‘col’ model) and supervisors’ infosec behaviours (‘sup’ model). the two models were fitted under the assumption that perceptions in wave 1 influenced perceptions in wave 2. since our likert scale-based data violated the multivariate normality assumption, maximum likelihood estimation was deemed inappropriate and we used the bollen-stine bootstrapping method (bollen & stine, 1992) to evaluate whether the model was specified correctly and achieved adequate goodness-of-fit. both models exhibited acceptable goodness-of-fit with bollen-stine bootstrap p-values equal to 0.357 (col model) and 0.669 (sup model). table 1 further shows that there were no issues with convergent validity, except the removal of one item, sup5 due to its loadings at below the ±0.35 threshold (lewis, templeton, & byrd, 2005). these results supported the theoretical structure of the two constructs describing the employees’ perceptions of infosec climate. construct item loading–w1 α–w1 h–w1 loading–w2 α–w2 h–w2 sup sup1 0.94 0.94 0.95 0.90 0.95 0.95 sup2 0.94 0.95 sup3 0.89 0.88 sup4 0.83 0.90 sup5 dropped dropped col col1 0.93 0.94 0.96 0.89 0.94 0.96 col2 0.93 0.92 col3 0.71 0.69 col4 0.83 0.92 col5 0.93 0.94 acceptable criteria >±0.35 >0.70 >0.70 >±0.35 >0.70 >0.70 table 1. convergent validity (w1=wave 1; w2=wave 2) factor score weights from the fitted models were used to calculate composite scores of the latent constructs by calculating the sum products of these weights and the employees’ responses. next, we rounded the calculated composite scores to integers to meet the data requirement of saom method. 3.4 strategy of saom analysis we specified and estimated a stochastic actor-oriented (sao) model by using the r statistical package called ‘rsiena’ (ripley et al., 2011) to examine the conceptual framework shown in figure 1. snijders et al. (2010) provide a comprehensive and detailed introduction featuring the saom method, which involves specifying a mathematical model (i.e., the sao model) with parameters which describe the simultaneous changes in the employees’ selection australasian journal of information systems dang-pham, kautz, pittayachawan & bruno 2019, vol 23, selected papers from acis 2017 inf security climate & support network 10 patterns, i.e., the creation or maintenance of network ties over time and the influence effects, i.e., the changes in the employees’ infosec climate perceptions caused by the changing network ties. the mechanisms of these changes are then evaluated by using the markov chain monte carlo (mcmc) approach (dijkstra et al., 2010; ripley et al., 2011). since this research did not formulate hypotheses and proposed a conceptual framework instead of a theoretical model, its approach is exploratory in nature. the sao model specified and subsequently evaluated a large body of 109 parameters which described the possible forming mechanisms of the networks and infosec climate perceptions in accordance to the conceptual framework in figure 1. the 109 parameters are outlined as follows and elaborated on in appendix b. each of the three networks had six parameters included in the sao model. these 18 parameters describe the structural changes, such as the density of the network or the tendency of the nodes to reciprocate ties over time. in relation to the conceptual framework in figure 1, we were interested in exploring the impacts of the networks on each other. we specified in our sao model nine parameters to evaluate the mutual impacts between pairs of networks, i.e., between instrumental and expressive networks, instrumental and infosec support networks, and expressive and infosec support networks. there are eight attributes of the employees, namely (1) age, (2) gender, (3) tenure, (4) seniority, (5) department membership, (6) champion status, and climate perceptions of (7) colleagues and (8) supervisors’ infosec behaviours, each of which is assumed to impact on the formation of the networks in three different ways. three different parameters can be used to model the tendency of nodes to send or receive ties, especially when the sender or receiver have a high score for an attribute, e.g., older age, higher tenure or seniority, or when the sender and receiver have similar attributes. in the latter case, having similar attributes can be an exact match, e.g., same gender or same department membership, or approximate match, e.g., small age gap or tenure gap, which are described by another two parameters. a total of 66 parameters were included in the sao model to specify the individual attributes’ effects on network formation. to describe the impacts of the infosec network on the formation of two types of climate perceptions, two parameters were included in the sao model. four parameters were included in the model to describe the changing patterns of the two types of climate perceptions over time. the impacts of the five individual attributes, namely age, gender, tenure, seniority, and champion status, on the development of two types of climate perceptions were modelled with 10 parameters. consequently, a total of 16 parameters were included to describe the formation process of the employees’ climate perceptions. table 2 summarises the parameters that were included in the sao model to describe the changes in the networks and in the employees’ climate perceptions. australasian journal of information systems dang-pham, kautz, pittayachawan & bruno 2019, vol 23, selected papers from acis 2017 inf security climate & support network 11 parameter description number of parameters variables rate the number of model-based simulated opportunities for the nodes to add, remove or keep their ties between the two data collection waves 3 networks instrumental network, expressive network, infosec support network density the node’s tendency to establish ties 3 networks reciprocity the node’s tendency to reciprocate or return ties 3 networks gwespff the node’s tendency to close a triad, by sending a direct tie to another indirectly connected node 3 networks gwespfbmix the node’s tendency to close a triad, by sending a direct tie to another node that is indirectly connected by a different type of tie 3 networks crprod the tendency of different types of ties to co-occur 6 (i.e., six pairs) outactsqrt the tendency of a node to keep sending ties to other nodes, when that node already has many outgoing ties 3 networks inpopsqrt the tendency of a node to keep receiving ties from other nodes, when that node already has many incoming ties 3 networks egox the tendency of a node to send ties to other nodes, when that node has a high score for an attribute 21 (7 attributes x 3 networks) age, gender, tenure, seniority, champion status, col, sup altx the tendency of a node to receive ties from other nodes, when that node has a high score for an attribute 21 (7 attributes x 3 networks) samex the tendency of two nodes to establish a tie when they have the same attribute (categorical data) 12 (4 attributes x 3 networks) gender, seniority, department membership, champion status australasian journal of information systems dang-pham, kautz, pittayachawan & bruno 2019, vol 23, selected papers from acis 2017 inf security climate & support network 12 parameter description number of parameters variables simx the tendency of two nodes to establish a tie when they have the same attribute (continuous data) 12 (4 attributes x 3 networks) age, tenure, col, sup totsimw the tendency of nodes to change their individual attribute in accordance to their connected nodes who have attribute w. in this research, this parameter evaluates the tendency of employees (as nodes) to change their climate perceptions, to match with those of the connected colleagues who have the same department membership (as attribute w) 2 perceptions infosec support network, department membership, col, sup linear shape the tendency of climate perceptions to increase or decrease over time in a linear fashion 2 perceptions col, sup quadratic shape the tendency of climate perceptions to self-adjust to a higher/lower level when the current score is too low/high 2 perceptions efffrom the effect of an individual attribute on climate perceptions 10 (5 attributes x 2 perceptions) age, gender, tenure, seniority, champion status, col, sup total 109 key: col = climate perception of colleagues’ infosec behaviours; sup = climate perception of supervisors’ infosec behaviours table 2. parameters included in the sao model 4 analysis and findings the results of the saom analysis identified 36 parameters that achieved statistical significance, out of the 109 parameters included in the sao model. we present these parameters’ effects in the following sections; the statistical results concerning the effects which are captured by these parameters are reported in appendix c. 4.1 structural changes in the networks the parameter rate presents the mode-based, simulated number of opportunities for changing ties per actor in a network between the two points in time (ripley et al., 2011). the saom method simulates a series of mini-steps in such a period, based on characteristics of the empirical network, where the actors can add, remove, or maintain their ties in each mini-step. the infosec support network had the highest number of change opportunities, followed by the expressive and instrumental networks. this result shows that the diffusion of infosec knowledge had effectively stimulated the employees’ sharing of infosec advice and troubleshooting support. the stability of the expressive and instrumental networks indicates that the normal work routines at abc, as represented by the provisions of work and personal advice, remained unchanged. australasian journal of information systems dang-pham, kautz, pittayachawan & bruno 2019, vol 23, selected papers from acis 2017 inf security climate & support network 13 the outdegree effect for the instrumental, expressive and infosec support networks achieved statistical significance with negative values, indicating that these networks were sparse. network ties rarely occurred between employees, although more infosec support network ties were created over time. the reciprocity parameter, which describes the employees’ tendency to reciprocate ties, only achieved statistical significance for the instrumental and expressive networks but not for the infosec support network. the odds ratios–which indicates the likelihood of a phenomenon appearing–of the reciprocity parameters for instrumental and expressive networks were 2.3 (e0.84) and 4.7 (e1.55) respectively. this means that work advice and organisational updates were reciprocated 2.3 times more than only being unidirectional, and 4.7 times more likely for the network of provisions for personal advice and trust for expertise. network transitivity is represented through transitive closure which describes the tendency of a network actor to close their triads formed with two other actors, by establishing a direct connection with an actor who is indirectly connected via multiple intermediaries. our results, obtained through the positive estimate of the parameter gwespff (see appendix b for the details of the parameter’s meaning) indicated that the instrumental and expressive networks were transitive, whereas the infosec support network was not. in the infosec support network, sharing the same providers of infosec support did not create new interactions, which was consistent with the sparse and thin nature of this network. 4.2 forming mechanisms of socialisation for each of the background characteristics such as gender, age, and tenure, we evaluated parameters that described the socialising tendency of the senders, receivers and matching partners. evaluating the effects expressed by these parameters helps to determine the factors that influence the employees’ socialising choices in the instrumental, expressive and infosec support networks. for instance, we evaluated the tendency for male and female employees to seek work advice from their colleagues, and the tendency for employees who worked in the same department to seek infosec support from each other. gender was found to influence the employees’ decisions to seek work advice or organisational updates from their colleagues. we found that female employees tend to be sought more for instrumental resources compared to male employees; in addition, employees of the same gender tended to seek work advice and organisational updates from each other more. gender also determined whom the employees would trust for expertise, with female employees attracting more trust nominations by other employees than male employees. similarly, employees tended to trust colleagues of the same gender. in contrast, gender did not explain the patterns of seeking infosec support. with regard to the employees’ age, older employees tended to be sought for work advice and organisational support more, whereas younger employees were more likely to be sought for infosec support. seniority also played a role for increasing the employees’ received nominations in the instrumental network. we found that employees who held senior positions were nominated more to provide work advice and organisational updates. likewise, the provision of these instrumental resources occurred more between those having the same seniority. sharing the same department membership increased the employees’ nominations in the three networks. further, employees who served as infosec champions were 1.7 times more likely than non-champions to be sought for infosec support (with an odd ratio of e0.55). australasian journal of information systems dang-pham, kautz, pittayachawan & bruno 2019, vol 23, selected papers from acis 2017 inf security climate & support network 14 as our research objective was to detect the influence effect through separating selection and influence mechanisms, we included parameters that evaluated the employees’ tendency to socialise with those who were perceived to hold similar infosec climate perceptions. the results of these parameters did not achieve statistical significance. the insignificant results of these parameters indicated that the employees’ decision to socialise with other colleagues in the instrumental, expressive, and infosec support networks was not affected by having similar infosec climate perceptions. we detected that the three types of socialisation co-occurred with each other. the employees’ selection of colleagues to socialise with in one network influenced their socialising choice in other networks. the instrumental and expressive networks tended to co-occur with each other, indicating that employees were more likely to seek work advice and organisational updates from those whom they trusted and sought personal advice from, and vice versa. similarly, infosec support ties co-occurred with instrumental and expressive ties, in which the cooccurrence between infosec support and instrumental networks had a higher likelihood. this means that employees sought infosec support from colleagues who also provided instrumental and expressive resources to them. the instrumental and expressive networks exhibited different patterns of co-occurrence with the infosec support network which are shown in the results of their parameter gwespfbmix (see appendices b and c for the details of the meaning and results of this parameter). the found negative estimate of –0.62 implies that employees were less inclined to seek infosec support from those colleagues with whom they shared the same trusted people. an explanation for this intriguing phenomenon was that employees who shared personal matters with the same trusted people might prefer to avoid interacting with each other. consistent with the belief that knowledge represents power, resource seekers might minimise the possibility that their trusted contacts would know about their ignorance or dependence on others, by avoiding seeking resources from people who are close to their trusted contacts. in contrast, the positive effect of the same gwespfbmix parameter for the instrumental network suggested that employees who shared common providers of work advice were more likely to seek infosec support from each other. since employees who sought work advice from the same people had a high chance of working in similar job roles, this result might reflect the employees’ need to discuss relevant infosec-related matters with those who shared the same work duties. table 3 summarises the saom findings about the selection process, i.e., formation of the instrumental, expressive, and infosec support networks. australasian journal of information systems dang-pham, kautz, pittayachawan & bruno 2019, vol 23, selected papers from acis 2017 inf security climate & support network 15 formation mechanisms formation of instrumental network employees tended to seek work advice and organisational updates from: • female employees or those of the same gender • older employees • employees who held more senior positions or those having the same seniority • employees who worked in the same department • employees who gave personal advice to or trusted them for expertise formation of expressive network employees tended to seek personal advice from or trust the expertise of: • female employees or those of the same gender • employees who worked in the same department • employees who gave work advice or organisational updates to them formation of infosec support network employees tended to seek infosec advice and troubleshooting support from: • younger employees • employees who worked in the same department • infosec champions • employees who gave work advice or organisational updates to them • employees who gave personal advice to or trusted them for expertise • employees who received work advice and organisational updates from the same people employees tended to not seek infosec advice and troubleshooting support from: • employees who shared the same trusted people table 3. saom findings about selection process, i.e., formation of networks 4.3 forming mechanisms of infosec climate to understand the forming mechanisms of infosec climate which were caused by the influence process, we investigated how the employees’ infosec climate perceptions had changed over time under other factors’ effects. the saom results show that none of the background characteristics had an effect on the formation of the employees’ perceptions of infosec climate. since we calculated the scores for infosec climate perceptions of each employee, which ranged from 1 (non-visible climate) to 5 (highly visible climate), we were interested in examining the variation of those scores. the saom result showed a positive and statistically significant effect of the linear shape parameter (see appendix b for details of this parameter) for the climate perception of direct supervisors’ infosec behaviours, which indicated that the scores of these perceptions tended to increase over time. it means that the general perceptions of direct supervisors’ infosec behaviours became more favourable after the diffusion of infosec knowledge by the champions. the negative effects of the quadratic shape parameter (see appendix b for details of this parameter) for both types of climate perceptions suggested that the scores of these perceptions would be more likely to become low when they are high and vice versa. moreover, these scores tended to deviate around the average point rather than around the polarising ends which reflect a climate that is too non-visible or too visible. these results suggested that the employees in abc tended to adjust the level of their climate perceptions according to the majority of employees in the workplace. it is worth mentioning that the effects of social influence were tested differently compared to the other effects by using the score-type test. this test is recommended when there is a australasian journal of information systems dang-pham, kautz, pittayachawan & bruno 2019, vol 23, selected papers from acis 2017 inf security climate & support network 16 parameter producing an estimated value with an unusually large standard error, which suggests the result is unstable and the default estimation procedure is unsuitable for the parameter (ripley et al., 2011). by performing the score-type test, we observed a positive and significant one-sided test statistic of 3.0477 (with a chi-square value of 9.2887 and a p-value of 0.0023) for the influence effect on the climate perceptions of colleagues’ infosec behaviours. this result suggested that employees were inclined to adjust climate perceptions of colleagues’ infosec behaviours to match with those of their colleagues in the same department, whom they had nominated as capable of providing them with infosec support. the result also indicated that there were no clear patterns about the change in the employees’ climate perception of their direct supervisors’ infosec behaviours, caused by the influence exerted by their colleagues who provided infosec support to them. table 4 summarises the influence process, i.e., formation of the employees’ climate perceptions. formation mechanisms climate perceptions of colleagues’ infosec behaviours • self-adjusted to become more/less favourable when being too little/too much favourable • self-adjusted to match with the perceptions of the employees who gave infosec advice and troubleshooting support climate perceptions of direct supervisors’ infosec behaviours • became more favourable over time • self-adjusted to become more/less favourable when being too little/too much favourable table 4. saom findings about influence process, i.e., formation of climate perceptions 5 discussion the primary objective of this research was to seek a comprehensive understanding of the simultaneous formation of the employees’ infosec perceptions and the infosec support provisions, which represents the infosec-related socialisation. prior behavioural infosec research has explained the formation of infosec climate as an outcome of the employees’ socialisation, with socialisation facilitating discussions on infosec-related matters which as a consequence raises the employees’ awareness of the shared infosec practices in their workplace (ashforth, 1985; chan et al., 2005; schneider & reichers, 1983; weick, 1995). our findings indicate that employees tended to match their perceptions of colleagues’ infosec behaviours with those of their colleagues, who provided them with infosec support, and supported the findings of those prior studies. by analysing not only the changes in the employees’ infosec climate perceptions but also their socialisation in the form of networks, we further propose practical ways for organisations to facilitate positive improvements in their infosec climates through manipulating and exploiting the employees’ socialisation. 5.1 forming mechanisms of infosec perceptions and infosec support provisions our results established that the employees’ provisions of infosec support, which facilitated the social influence that shaped infosec climate, were affected by the employees’ sharing of department membership, similar ages and provisions of work-related resources and personal support. consistent with these results, we advise to stimulate the employees’ provisions of infosec support to develop a positive infosec climate, by devising strategies to exploit the employees’ background characteristics and interactions. australasian journal of information systems dang-pham, kautz, pittayachawan & bruno 2019, vol 23, selected papers from acis 2017 inf security climate & support network 17 one recommended strategy is to identify employees who are influential in the infosec domain in their department, based on their activeness in providing work advice, organisational updates and personal support to other employees. managers can ask employees to nominate their local champions, and sna tools can help identify these champions through network visualisations which depict the employees’ provisions of resources. the identified forming mechanisms of the instrumental and expressive networks provide additional cues to facilitate the provisions of these resources, which indirectly contribute to a positive infosec climate via stimulating the provisions of infosec support. in determining the employees’ personal attributes, e.g., gender, age, seniority, and socialisation, e.g., providing work and personal advice, that impacted on their provisions of infosec support, we contribute to the research area of selecting infosec champions for organisational infosec programs where empirical studies are scarce. infosec managers can use the findings of this study as cues to identify the employees who actively provide their colleagues with infosec support, and consider training these employees or appointing them to be infosec champions. in addition, the rotating of infosec champions across different departments may be an effective way to provide employees with new perspectives and knowledge about infosec. 5.2 methodological contributions to infosec research we demonstrate through this study the use of the saom method for simultaneously analysing perceptions of infosec climate, as a personal attribute of individuals, and the interactions that take place between these individuals. this saom method and the sna approach greatly complement the traditional approach employed by many behavioural infosec research studies, which solely focus on the individuals’ personal and cognitive attributes. this limitation is evident in the research area of infosec climate, where prior studies only detected employees to develop positive climate perceptions by perceiving the ongoing socialisation but not considering the actual infosec support communicated via such socialisation (see e.g., chan et al., 2005; goo et al., 2014; jaafar & ajis, 2013). this same argument also applies to other behavioural infosec studies which have examined the provisions of infosec support as products of the employees’ cognition such as perceptions and attitudes, but not within the network that are formed through the acts of sending and receiving infosec support (see e.g., safa et al., 2016; warkentin et al., 2011). the adoption of sna methods enables researchers to explore both the factors which trigger the individuals’ decisions to initiate or accept infosecrelated interactions, as well as the attributes of these interactions such as their intensity and type of content. we suggest further some applications of sna methods in the behavioural infosec field. behavioural infosec researchers may apply sna methods to investigate the impacts of networks on various infosec perceptions and behaviours provided by theories in the field. the literature reviews conducted by padayachee (2012), sommestad et al. (2014) and warkentin and mutchler (2014) have identified theoretical variables of prominent theories such as protection motivation theory (rogers, 1975), theory of planned behaviour (ajzen, 2011) and general deterrence theory (straub & welke, 1998); some of these variables have been consistently found to motivate infosec compliance and deter potential infosec violations. researchers can perform sna to evaluate the impacts of employees’ interactions on the important drivers of infosec compliance, such as attitude toward infosec, subjective norms, perceptions of cyber-threats and severity of infosec sanctions. we anticipate that the outcomes australasian journal of information systems dang-pham, kautz, pittayachawan & bruno 2019, vol 23, selected papers from acis 2017 inf security climate & support network 18 of such sna will help organisations to develop desirable infosec-related perceptions that lead to infosec compliance, while research findings about infosec-related networks can extend theories by providing knowledge of infosec processes. similarly, organisations can benefit from research which analyses organisational networks to deter infosec violations and mitigate negative perceptions such as disgruntlement or perceived organisational injustice (willison & warkentin, 2013). 5.3 directions for future research this research was conducted in vietnam, thus it is likely that the formation of the examined networks and the social influences facilitated by the employees’ interactions were influenced by the vietnamese culture. for instance, vietnam is considered as a collectivistic society that has large power distance and prefers flexibility and relaxing work environment (hofstede, 2001). such environment is conducive to social influence, since it fosters the formation of privileged groups and emphasises the development of strong relationships and consensus between people (hofstede, 2001). consequently, the employees at abc might have sought infosec support from or changed their infosec climate perceptions by accepting the social influence of the colleagues whom they wanted to establish effective work relationships with. in fact, our saom findings in relation to the self-adjustment tendency of the employees’ climate perceptions of the direct supervisors’ infosec behaviours is consistent with the collectivistic nature of the vietnamese culture. people who embrace cultures that have large power distance tend to accept a hierarchical order (hofstede, 2001). interestingly, our saom findings indicate that the employees tended to seek their senior colleagues for work advice and organisational updates, but not for infosec support. the analysis also confirmed the positive impact of seniority on the formation of infosec support network. however, these effects were overshadowed by the effects of other factors such as the employees’ department membership, age, and champion status. it is also worth noting that there are various cultural frameworks describing the dimensions of national cultures (see e.g., trompenaars’ (trompenaars & hampden-turner, 2012) model of national culture), and that the unique organisational culture of each workplace may have an impact on the employees’ infosec perceptions and behaviours (see e.g., padayachee, 2012; ruighaver, maynard, & chang, 2007). in relation to the cultural dimensions, we invite future researches to further investigate the impacts of national and organisational cultures on the formation of infosec support network and of infosec climate perceptions. sna methods can also support action research projects by providing the means to design interventions and quantitatively evaluate their outcome with network metrics. action research studies hold important implications in the behavioural infosec field since their findings can determine the solutions that work and explain why they work, through evaluating theoretically-based interventions in a real context (puhakainen & siponen, 2010). researchers outside of the behavioural infosec field have designed and implemented network-based interventions that induced practical organisational changes, including the use of opinion leaders to diffuse novel ideas, segmenting individuals into groups and delivering tailored training programs (see e.g., borgatti & cross, 2003; hatala & fleming, 2007; valente, 2012). provided the available tools and strategies to formulate network-based interventions, behavioural infosec researchers are encouraged to not only stop at analysing the impacts of networks on the employees’ infosec perceptions and behaviours, but also evaluate the potential of network-based interventions for creating transformational changes in organisational infosec. for example, practical measures that reflect adequate organisational australasian journal of information systems dang-pham, kautz, pittayachawan & bruno 2019, vol 23, selected papers from acis 2017 inf security climate & support network 19 infosec, which result from effective network-based interventions, may include the number of detected infosec violations or the employees’ scores for infosec awareness tests. since an improved infosec-related network implies an elevated level of sharing infosec resources among employees, it is reasonable to expect that the organisation having such a network will realise those positive outcomes. 6 conclusions current behavioural infosec studies have emphasised the importance of investigating the impacts of human factors on organisational infosec, since acquiring the knowledge of these impacts is critical for formulating effective strategies and measures for achieving infosec success. nevertheless, current research has predominantly focused on the employees’ cognition and psychological attributes which influence their infosec perceptions and behaviours, while overlooking their interactions and relationships. consequently, important insights for designing and implementing infosec improvements such as why employees seek or provide infosec support, as well as the unique structures of the networks reflecting different infosec workplaces, are unavailable to make informed decisions. the adoption of sna methods enables researchers to comprehensively analyse infosec environments, comprising both the employees’ personal attributes and their infosec-relevant interactions conceptualised in the form of networks. our research employed a longitudinal sna method called saom to examine the formation mechanisms of a network characterising the employees’ provisions of infosec support. the saom analysis also explained the formation of their infosec climate perceptions. the saom analysis identified the employees’ sharing of department membership and gender, small age gap, and seniority as indicators for the longitudinal changes in their provisions of instrumental and expressive resources and infosec support. the provisions of work advice and organisational updates, personal support and trust for expertise were found to increase the sharing of infosec advice and troubleshooting support that developed a shared infosec climate. specifically, the employees tended to match their climate perceptions of colleagues’ infosec behaviours with those of their colleagues who provided them with infosec support, whereas climate perceptions of direct supervisors’ infosec behaviours were unaffected. we offer practical recommendations to organisations to develop a positive infosec climate based on our research findings, while looking forward to future adoption of sna methods to extend theoretical knowledge of the behavioural infosec field. references ajzen, i. (2011). theory of planned behavior. in handbook of theories of social psychology: volume one (p. 438). ashforth, b. (1985). climate formation: issues and extensions. academy of management review, 10(4), 837–847. baskerville, r. l., park, e., & kim, j. (2014). an emote opportunity model of computer abuse. information technology & people, 27.2, 1–31. bollen, k. a., & stine, r. a. (1992). bootstrapping goodness-of-fit measures in structural equation models. sociological methods & research, 21(2), 205–229. https://doi.org/10.1177/0049124192021002004 australasian journal of information systems dang-pham, kautz, pittayachawan & bruno 2019, vol 23, selected papers from acis 2017 inf security climate & support network 20 borgatti , s. p., & cross, r. (2003). a relational view of information seeking and learning in social networks. management science , 49(4), 432–445. https://doi.org/10.1287/mnsc.49.4.432.14428 borgatti, s. p., everett, m. g., & johnson, j. c. (2013). analyzing social networks. sage publications ltd. brondino, m., pasini, m., & costa, s. (2013). development and validation of an integrated organizational safety climate questionnaire with multilevel confirmatory factor analysis. quality & quantity, 47, 2191–2223. https://doi.org/10.1007/s11135-011-9651-6 brown, t. a. (2006). confirmatory factor analysis for applied research. the guilford press. bulgurcu, b., cavusoglu, h., & benbasat, i. (2010). information security policy compliance: an empirical study on rationality-based beliefs and information security awareness. mis quarterly, 34(3), 523–548. chan, m., woon, i., & kankanhalli, a. (2005). perceptions of information security at the workplace: linking information security climate to compliant behavior. in perceptions of information privacy and security (vol. 1, pp. 18–41). cheng, l., li, y., li, w., holm, e., & zhai, q. (2013). understanding the violation of is security policy in organizations: an integrated model based on social control and deterrence theory. computers & security, 39, 447–459. retrieved from http://linkinghub.elsevier.com/retrieve/pii/s0167404813001387 cross, r., laseter, t., parker, a., & guillermo, v. (2006). using social network analysis to improve communities of practice. california management review, 49(1), 32–62. dang-pham, d., pittayachawan, s., & bruno, v. (2017). applying network analysis to investigate interpersonal influence of information security behaviours in the workplace. information & management, 54(5), 625–637. https://doi.org/10.1016/j.im.2016.12.003 delay, d., zhang, l., hanish, l. d., miller, c. f., fabes, r. a., martin, c. l., … updegraff, k. a. (2016). peer influence on academic performance: a social network analysis of social-emotional intervention effects. prevention science, 1–11. https://doi.org/10.1007/s11121-016-0678-8 dijkstra, j. k., lindenberg, s., veenstra, r., steglich, c., isaacs, j., card, n. a., & hodges, e. v. e. (2010). influence and selection processes in weapon carrying during adolescence: the roles of status, aggression, and vulnerability. criminology, 48(1), 187–220. https://doi.org/10.1111/j.1745-9125.2010.00183.x dourish, p., & anderson, k. (2006). collective information practice: exploring privacy and security as social and cultural phenomena. human–computer interaction, 21(3), 319–342. french, j. r. p., & raven, b. (1959). the bases of social power. in studies in social power (pp. 150–167). goo, j., yim, m., & kim, d. (2014). a path to successful management of employee security compliance: an empirical study of information security climate. ieee transactions on professional communication, 57(4), 1–24. australasian journal of information systems dang-pham, kautz, pittayachawan & bruno 2019, vol 23, selected papers from acis 2017 inf security climate & support network 21 guo, k. h., & yuan, y. (2012). the effects of multilevel sanctions on information security violations: a mediating model. information & management, 49(6), 320–326. https://doi.org/http://dx.doi.org/10.1016/j.im.2012.08.001 hatala, j.-p., & fleming, p. r. (2007). making transfer climate visible: utilizing social network analysis to facilitate the transfer of training. human resource development review, 6(1), 33–63. https://doi.org/10.1177/1534484306297116 herath, t., & rao, h. r. (2009). protection motivation and deterrence: a framework for security policy compliance in organisations. european journal of information systems, 18(2), 106– 125. hofstede, g. (2001). culture’s consequences: comparing values, behaviors, institutions, and organizations across nations (second). thousand oaks ca: sage publications. hogg, m. a., & reid, s. a. (2006). social identity, self-categorization, and the communication of group norms. communication theory, 16(1), 7–30. https://doi.org/10.1111/j.14682885.2006.00003.x ibarra, h., & andrews, s. b. (1993). power, social influence, and sense making: effects of network centrality and proximity on employee perceptions. administrative science quarterly, 38(2), 277. https://doi.org/10.2307/2393414 ifinedo, p. (2014). information systems security policy compliance: an empirical study of the effects of socialisation, influence, and cognition. information and management, 51(1), 69– 79. https://doi.org/10.1016/j.im.2013.10.001 jaafar, n. i., & ajis, a. (2013). organizational climate and individual factors effects on information security faculty of business and accountancy. international journal of business and social science, 4(10), 118–130. kines, p., lappalainen, j., lyngby, k., olsen, e., pousette, a., tharaldsen, j., … törner, m. (2011). nordic safety climate questionnaire (nosacq-50): a new tool for diagnosing occupational safety climate. international journal of industrial ergonomics, 41(6), 634–646. https://doi.org/10.1016/j.ergon.2011.08.004 kirlappos, i., parkin, s., & sasse, m. a. (2014). learning from “shadow security”: why understanding non-compliant behaviors provides the basis for effective security. in usec’14 workshop on usable security. lee, j., & lee, y. (2002). a holistic model of computer abuse within organizations. information management & computer security, 10(2), 57–63. https://doi.org/10.1108/09685220210424104 lewis, b. r., templeton, g. f., & byrd, t. a. (2005). a methodology for construct development in mis research. european journal of information systems, 14(4), 388–400. lingard, h. c., cooke, t., & blismas, n. (2009). group-level safety climate in the australian construction industry: within-group homogeneity and between-group differences in road construction and maintenance. construction management and economics issn:, 27(4), 419–432. https://doi.org/10.1080/01446190902822971 australasian journal of information systems dang-pham, kautz, pittayachawan & bruno 2019, vol 23, selected papers from acis 2017 inf security climate & support network 22 lowry, p. b., & moody, g. d. (2013). explaining opposing compliance motivations towards organizational information security policies. 2013 46th hawaii international conference on system sciences, 2998–3007. https://doi.org/10.1109/hicss.2013.5 mcknight, d. h. (2002). developing and validating trust measures for e-commerce: an integrative typology. information systems research, 13(3), 334–359. https://doi.org/10.1287/isre.13.3.334.81 mcknight, d. h., & chervany, n. l. (1996). the meanings of trust. measurement, 55455(612), 86. https://doi.org/10.1117/12.304574 mcpherson, m., smith-lovin, l., & cook, j. m. (2001). birds of a feather: homophily in social networks. annual review of sociology, 27(2001), 415–444. https://doi.org/10.1146/annurev.soc.27.1.415 morrison, e. w. (1993). newcomer information seeking: exploring types, modes, sources, and outcomes. academy of management journal, 36(3), 557–589. padayachee, k. (2012). taxonomy of compliant information security behavior. computers & security, 31(5), 673–680. https://doi.org/http://dx.doi.org/10.1016/j.cose.2012.04.004 podsakoff, p. m., mackenzie, s. b., lee, j.-y., & podsakoff, n. p. (2003). common method biases in behavioral research: a critical review of the literature and recommended remedies. the journal of applied psychology, 88(5), 879–903. https://doi.org/10.1037/00219010.88.5.879 puhakainen, p., & siponen, m. (2010). improving employee’ compliance through information systems security training: an action research study. mis quarterly, 34(4), 757–778. ripley, r. m., snijders, t. a. b., & preciado, p. (2011). manual for rsiena. university of oxford, department of statistics, nuffield college, 1. rocha flores, w., antonsen, e., & ekstedt, m. (2014). information security knowledge sharing in organizations: investigating the effect of behavioral information security governance and national culture. computers & security, 43, 90–110. https://doi.org/10.1016/j.cose.2014.03.004 rogers, r. w. (1975). a protection motivation theory of fear appeals and attitude change. journal of psychology, 91, 93–114. ruighaver, a. b., maynard, s. b., & chang, s. (2007). organisational security culture: extending the end-user perspective. computers & security, 26(1), 56–62. safa, n. s., solms, r. von, & von solms, r. (2016). an information security knowledge sharing model in organizations. computers in human behavior, 57, 442–451. https://doi.org/10.1016/j.chb.2015.12.037 saint-charles, j., & mongeau, p. (2009). different relationships for coping with ambiguity and uncertainty in organizations. social networks, 31(1), 33–39. https://doi.org/10.1016/j.socnet.2008.09.001 schneider, b., & reichers, a. (1983). on the etiology of climates. personnel psychology, (1934), 19–40. australasian journal of information systems dang-pham, kautz, pittayachawan & bruno 2019, vol 23, selected papers from acis 2017 inf security climate & support network 23 snijders, t. a. b., van de bunt, g. g., & steglich, c. e. g. (2010). introduction to stochastic actor-based models for network dynamics. social networks, 32(1), 44–60. https://doi.org/10.1016/j.socnet.2009.02.004 sommestad, t., hallberg, j., lundholm, k., & bengtsson, j. (2014). variables influencing information security policy compliance: a systematic review of quantitative studies. information management & computer security, 22(1), 42–75. https://doi.org/10.1108/imcs08-2012-0045 steglich, c., snijders, t. a. b., & pearson, m. (2010). dynamic networks and behavior: seperating selection from influence. sociological methodology, 8, 329–393. https://doi.org/10.1111/j.1467-9531.2010.01225.x straub, d. w., & welke, r. j. (1998). coping with systems risk: security planning models for management decision making. mis quarterly, 22(4), 441–469. https://doi.org/10.2307/249551 trompenaars, f., & charles hampden-turner, c. (2012). riding the waves of culture: understanding diversity in global business (3.ed). new york: mcgraw-hill. tsohou, a., karyda, m., kokalakis, s., & kiontouzis, e. (2013). managing the introduction of information security awareness programmes in organisations. european journal of information systems, 24(1), 1–21. https://doi.org/10.1057/ejis.2013.27 valente, t. w. (2012). network interventions. science, 337(6090), 49–53. https://doi.org/10.1126/science.1217330 valente, t. w., palinkas, l. a., czaja, s., chu, k.-h., & brown, c. h. (2015). social network analysis for program implementation. plos one, 10(6), e0131712. https://doi.org/10.1371/journal.pone.0131712 warkentin, m., johnston, a. c., & shropshire, j. (2011). the influence of the informal social learning environment on information privacy policy compliance efficacy and intention. european journal of information systems, 20(3), 267–284. https://doi.org/10.1057/ejis.2010.72 warkentin, m., & mutchler, l. (2014). behavioral information security management. in g. tucker & d.-h. topi (eds.), computing handbook (3rd ed., pp. 1–14). taylor & francis group. weick, k. e. (1995). sensemaking in organizations. thousand oaks, ca: sage publications. willison, r., & warkentin, m. (2013). beyond deterrence: an expanded view of employee computer abuse. mis quarterly, 37(1), 1–20. zohar, d., & luria, g. (2005). a multilevel model of safety climate: cross-level relationships between organization and group-level climates. journal of applied psychology, 90(4), 616–628. https://doi.org/10.1037/0021-9010.90.4.616 australasian journal of information systems dang-pham, kautz, pittayachawan & bruno 2019, vol 23, selected papers from acis 2017 inf security climate & support network 24 appendices appendix a: survey questions category network question instrumental seek work advice who do you usually ask for advice, e.g., look for or improve solutions, get referrals or confirmation about work? seek organisational updates from whom do you usually get the latest updates or changes, e.g., new policies, process, system that are happening or coming in abc? expressive seek personal advice when you want to discuss or ask for advice about personal life issues, whom would you talk to? trust for expertise who do you think would be most able because of education, experience, qualities to take over your work if you were too busy or absent? infosec support seek infosec advice who would explain the importance of infosec to you, and/or teach you how to perform security behaviours, and/or use security technologies? seek infosec troubleshooting support when you encountered a security problem e.g. lost or damaged data, computer virus infection etc., whom would you seek help from? note: employees may nominate a maximum of seven colleagues per question. table 1. network questions construct question (item) scale adapted sources perception of direct supervisor’s infosec behaviours (sup) how frequently do your direct supervisor(s) mention infosec matters to you and your coworkers? (sup1) never; very rarely; rarely; sometimes; occasionally; very frequently; always chan et al. (2005); goo et al. (2014); jaafar and ajis (2013) how much do your direct supervisor(s) ask that you and your co-workers in the work unit must perform infosec behaviours? (sup2) never; very little; little; somewhat; much; very much; a great deal kines et al. (2011); zohar and luria (2005) how frequently do your direct supervisor(s) discuss infosec threats with you and your coworkers? (sup3) never; very rarely; rarely; sometimes; occasionally; very frequently; always how serious, strict, or careful are your direct supervisor(s) when it comes to protecting infosec? (sup4) never; very little; little; somewhat; much; very much; a great deal zohar and luria (2005) how frequently do your direct supervisor(s) allow you and your co-workers to overlook infosec when rushing deadlines? (sup5) (reversed) never; very rarely; rarely; sometimes; occasionally; very frequently; always brondino, pasini and costa (2013); kines et al. (2011); zohar and luria (2005) australasian journal of information systems dang-pham, kautz, pittayachawan & bruno 2019, vol 23, selected papers from acis 2017 inf security climate & support network 25 construct question (item) scale adapted sources perception of colleagues’ infosec behaviours (col) how much do your co-workers perform infosec behaviours in their daily work? (col1) never; very little; little; somewhat; much; very much; a great deal kines et al. (2011) how much do your co-workers care about infosec? (col2) never; very little; little; somewhat; much; very much; a great deal jaafar and ajis (2013) how much training and updates about infosec do your co-workers receive? (col3) never; very little; little; somewhat; much; very much; a great deal goo et al. (2014) how much do your co-workers prioritise infosec when they are rushing deadlines? (col4) never; very little; little; somewhat; much; very much; a great deal chan et al. (2005); jaafar and ajis (2013); lingard, cooke and blismas (2009) how much do your co-workers pay attention to and perform infosec behaviours, even when they are not supervised? (col5) never; very little; little; somewhat; much; very much; a great deal brondino et al. (2013); chan et al. (2005); lingard et al. (2009) table 2. questions about infosec climate perceptions appendix b: parameters included in saom analysis the parameters that we evaluated in our sao model, which represented the mechanisms in the conceptual framework, to explain the simultaneous changes in the network structures and the employees’ perceptions of infosec climate were grouped into two categories, comprising those that explain: (1) the forming mechanisms of the networks, and (2) the forming mechanisms of infosec climate perceptions. parameters for modelling the formation mechanisms of networks the parameter density is included in almost every sao model to explain the likelihood of occurrence of network ties between random actors and to characterise the shape of a network (snijders et al., 2010). the estimate of this parameter is often negative, which describes the commonly sparse and thin structure of networks. another parameter frequently included in network models is reciprocity, which describes the tendency for actors to reciprocate ties (delay et al., 2016; snijders et al., 2010). we modelled the triadic effects of transitive closure and structural equivalence concerning the actors’ outgoing choices by including the parameters gwespff and gwespfbmix. gwesp stands for ‘geometrically weighted edgewise shared partners’, referring to the evaluation of these triadic effects by focusing on the intermediate actors’ contributions to a direct tie between two indirectly connected actors, where such contribution of having many intermediaries is downweighed, i.e., intermediaries that are geometrically far away from the indirectly connected pairs have less impact on their direct ties (ripley et al., 2011). the gwesp parameters can be distinguished by the two letters next to the abbreviation, which can be ‘f’ (‘forward’) or ‘b’ (‘backward’). these letters, in their order of appearance, refer to the directions of the ties which are sent from an actor to an intermediary, and from this intermediary to other actors who are indirectly connected to the former one. for example, the parameter gwespff describes the transitive closure effect which counts the number of triads australasian journal of information systems dang-pham, kautz, pittayachawan & bruno 2019, vol 23, selected papers from acis 2017 inf security climate & support network 26 comprising three actors i, j and multiple intermediaries k, where the tie between i and j is facilitated by the forward tie sent from i to k and by the forward tie from k to j. in the context of providing infosec support, the parameter gwespff evaluates the likelihood of employee i to directly seek infosec support from j, which increases when i indirectly seeks support from j via numerous intermediate employees k. the parameter gwespfbmix examines the phenomenon where the likelihood for i to have a tie with j increases, when i and j both have ties to multiple k in between. the term ‘mix’ in the parameter’s name indicates that the configuration of interest involves more than one type of network ties, which in our study are the combinations of ‘instrumental–infosec support’ and ‘expressive–infosec support’ ties. with this parameter we examined the likelihood for employee i to directly seek infosec support from employee j, when i and j both seek work or personal advice from multiple employees k between them. degree-related parameters were included in our sao model to improve goodness-of-fit (delay et al., 2016). we used these parameters to model the number of actors that received or sent many ties compared to the average number of ties, i.e., by using the parameters inpopsqrt and outactsqrt, or those that are not connected to any actors, i.e., are isolates (snijders et al., 2010). minimum inand out-degrees or the specific number of outgoing and incoming ties can also be modelled (ripley et al., 2011), which are useful when there are outliers in the network. parameters for modelling the changes in the employees’ selection of socialised colleagues explain why network ties are created or maintained between pairs of employees over time. background characteristics such as gender, department membership, age, tenure and seniority were modelled to evaluate the relationship between the actors’ characteristics, e.g., long tenure or high seniority and their tendency to socialise. homophily effects such as the tendency to socialise with actors having similar characteristics, e.g., same gender or same department, were modelled by the parameter simx for numeric variables (such as age and tenure) and samex for categorical variables (such as department membership and gender). since we calculated the scores measuring the employees’ infosec climate perceptions, we used these scores in modelling the homophily effect. since there are three different networks (instrumental, expressive and infosec support) in our model, we included the parameter crprod to study the co-changes in the employees’ selection patterns in multiple networks over time. for example, the tendency for two employees to provide infosec support while trusting each other’s expertise at the same time was modelled in that way. parameters modelling the formation mechanisms of infosec climate perceptions when performing saom analysis, the used rsiena tool includes the linear shape and quadratic shape parameters by default, where the former parameter describes the behaviour’s tendency to increase or decrease over time in a linear fashion, and the latter describes the tendency for the actor to self-adjust the behaviour when its score becomes too high or too low (ripley et al., 2011). in our study, these parameters examined the changes in the employees’ climate perceptions of colleagues and direct supervisors’ infosec behaviours that were independent of the external influence caused by interacting with other employees. we included the parameter totsimw to model the employees’ tendency to adjust infosec climate perceptions to match with those of the colleagues who provided infosec support to australasian journal of information systems dang-pham, kautz, pittayachawan & bruno 2019, vol 23, selected papers from acis 2017 inf security climate & support network 27 them, i.e., the assimilation effect. we also included the parameters efffrom which accounted for the effects of the employees’ background characteristics on their infosec climate perceptions (e.g., higher seniority leads to higher infosec climate perceptions). since we examined two types of climate perceptions i.e., of colleagues’ and direct supervisors’ infosec behaviours, each type of climate perceptions was modelled to have the same set of parameters explaining the mentioned effects. appendix c: saom results table 7 summarises the effects in the saom analysis that achieved statistical significance, as well as the parameters describing them and the estimated results. absolute t-statistics are calculated by dividing the estimate and the standard error to determine the parameters’ statistical significance, i.e., the estimate is twice as large as the standard error. although the parameters inpopsqrt, outactsqrt, simx and efffrom–as explained in appendix b–were initially included in the sao model, their results were neither reported in table 7 nor discussed in section 5 since they did not achieve statistical significance. effects for instrumental network parameter estimate std. error rate rate 7.09 –0.67 outdegree density –5.15 –0.74 reciprocity reciprocity 0.84 –0.24 transitivity gwespff 1.19 –0.23 gender of ties receiver altx 0.44 –0.13 gender of ties sender egox –0.44 –0.15 same gender samex 0.34 –0.12 same department samex 0.94 –0.12 age of ties receiver altx 0.03 –0.01 seniority of ties receiver altx 0.50 –0.17 same seniority samex 0.50 –0.18 expressive network crprod 1.90 –0.23 effects for expressive network parameter estimate std. error rate rate 5.46 –0.55 outdegree density –4.20 –1.01 reciprocity reciprocity 1.55 –0.19 transitivity gwespff 1.04 –0.19 gender of ties receiver altx 0.56 –0.15 same gender samex 0.66 –0.14 same department samex 1.05 –0.14 instrumental network crprod 1.75 –0.33 effects for infosec support network parameter estimate std. error rate rate 8.29 –1.05 outdegree (density) density –2.60 –1.05 same department samex 1.22 –0.15 age of ties receiver altx –0.03 –0.01 champion of ties receiver altx 0.55 –0.16 expressive network crprod 1.56 –0.4 gwespfbmix via expressive network gwespfbmix –0.62 –0.3 australasian journal of information systems dang-pham, kautz, pittayachawan & bruno 2019, vol 23, selected papers from acis 2017 inf security climate & support network 28 instrumental network crprod 1.25 –0.37 gwespfbmix via instrumental network gwespfbmix 0.56 –0.27 effects for perceptions of colleagues’ infosec behaviours parameter estimate std. error rate rate 2.07 –0.41 tendency for perceptions to increase/decrease over time linear shape 1.79 –1.26 tendency for perceptions to self-adjust quadratic shape –0.62 –0.19 assimilation of infosec climate perceptions caused by receiving infosec support from colleagues in the same department totsimw 8.40 na+++ effects for perceptions of supervisors’ infosec behaviours parameter estimate std. error rate rate 2.74 –0.59 tendency for perceptions to increase/decrease over time linear shape 2.57 –1.16 tendency for perceptions to self-adjust quadratic shape –0.37 –0.11 notes: 1. +++ score-type test was employed to examine this effect and the estimate was found statistically significant (chisquare=9.2887; p-value=0.0023; one-sided statistic=3.0477). 2. statistically significant effects have estimate > 2*standard error i.e., t-statistic > 1.96 3. refer to appendix b for the meanings of the included parameters table 3. saom results (statistically significant effects only) copyright: © 2019 dang-pham, kautz, pittayachawan & bruno. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 conceptual framework 2.1 infosec climate perceptions 2.2 socialisation and the formation of infosec climate perceptions 2.3 selection and influence processes 3 research method 3.1 research context 3.2 variables 3.3 data collection and preparation 3.4 strategy of saom analysis 4 analysis and findings 4.1 structural changes in the networks 4.2 forming mechanisms of socialisation 4.3 forming mechanisms of infosec climate 5 discussion 5.1 forming mechanisms of infosec perceptions and infosec support provisions 5.2 methodological contributions to infosec research 5.3 directions for future research 6 conclusions appendix a: survey questions appendix b: parameters included in saom analysis parameters for modelling the formation mechanisms of networks parameters modelling the formation mechanisms of infosec climate perceptions appendix c: saom results paper title (paper title style) australasian journal of information systems mehta, de & chattopadhyay 2022, vol 26, research article elucidating the role of emotion in privacy-concerns 1 elucidating the role of emotion in privacy-concerns: a text-convolutional neural network (text-cnn)-based tweets analysis of contact tracing apps mihir mehta indian institute of management raipur, india sourya joyee de indian institute of management raipur, india souryajoyee@iimraipur.ac.in manojit chattopadhyay indian institute of management raipur, india abstract the extant contact tracing privacy literature is yet to explore the significance of user emotions in privacy-related decision-making such as whether to use such potentially privacy-invasive apps. using social media analytics, the present study examines users’ privacy-related emotions stimulated by privacy-related aspects of contact tracing apps. a text-convolutional neural network (text-cnn)-based emotion analysis of tweets on the indian contact tracing app aarogya setu and its singaporean counterpart tracetogether conducted in the paper reveals that users expressed negative privacy-related emotions towards these apps indicating high levels of perceived privacy risks and the perceived lack of privacy protection. for tracetogether, users have also exhibited positive emotions to appreciate the steps taken by the government to protect their privacy. based on these findings, the government/data controllers can devise strategies to assuage users’ negative emotions and promote positive emotions to encourage the adoption of contact tracing apps. this work incorporates privacy related emotions as key informants about user privacy concerns within the privacy calculus theory. by relying on candid user opinions available through rich but inexpensive usergenerated content, the research provides a quick, reliable, and cost-effective approach to study potential app users’ emotions to gain insights into privacy concerns related to any egovernance platform. keywords: privacy calculus theory, text-convolutional neural network, emotion analysis of tweets, contact tracing apps, perceived privacy risks, perceived privacy protections. 1 introduction with the rapid worldwide spread of covid-19, digital contact tracing mobile applications have been rolled out in many countries to automatically track locations and movements of infected individuals and their social contacts with high levels of precision, quickly and efficiently (cho, ippolito, & yu, 2020). privacy researchers as well as users have expressed deep concerns regarding privacy risks resulting from the collection, processing, and sharing of personal data by these apps and the lack of suitable privacy protection measures (baumgärtner et al., 2020; rowe, 2020; fahey & hino, 2020; georgieva et al., 2021). profiling and tracking of specific users (wen et al., 2020; bengio et al., 2021; leith & farrell, 2020), deanonymization of infected persons leading to stigmatization (bäumgartner et al., 2020), data misuse by third parties (azad et al. 2020) and government surveillance (rowe, 2020; fahey australasian journal of information systems mehta, de & chattopadhyay 2022, vol 26, research article elucidating the role of emotion in privacy-concerns 2 & hino, 2020; georgieva et al., 2021) have been identified as some of the major privacy risks associated with contact tracing apps. the success of contact tracing applications is largely dependent on mass citizen acceptance (fox et al., 2021) and with greater level of privacy awareness, convincing citizens to install and use these applications presents a significant challenge (fahey & hino, 2020). emotion has been shown to have a significant impact on decision-making (lerner et al., 2015; zhang & xu, 2016). however, the existing privacy literature on contact tracing is yet to extensively consider privacy emotion, such as anger at privacy risks like surveillance and profiling and happiness at good privacy protection measures such as data protection regulations, as a prime element in privacy-related decision-making such as whether to use the app. privacy calculus theory suggests that a person's intention to disclose personal information is based on a balancing act in which potentially competing factors such as negative beliefs or privacy concerns are weighed against positive beliefs or benefits gained from information disclosure (culnan & armstrong, 1999; masur & scharkow, 2016). this paper is an early attempt to examine users’ privacy related emotions in the context of contact tracing apps, incorporating the role of privacy emotions as a key informant about user privacy concerns in the privacy calculus theory. negative privacy emotions can indicate higher privacy risk perception and less appreciation of existing protection measures, leading to hesitation or low willingness to rely on the app (barnard et al., 2014). positive privacy emotions, on the other hand, can indicate low privacy risk perception and higher appreciation of existing protection measures, leading to high readiness or willingness to rely on the app (li et al., 2008). prior works have explored how privacy concerns affect the willingness of citizens to adopt contact tracing apps in different countries such as ireland, france, australia, and belgium (fox et al., 2021; chan and saqib, 2021; walrave et al., 2020; horvath et al., 2020). these studies rely on user surveys for data collection and analysis. a limited number of studies have also been conducted to understand citizens’ opinion on contact tracing in different countries through the analysis of user-generated content (ugc) on social media such as twitter. while in ireland, citizens exhibited mostly positive opinion (rekanar et al., 2021), in some other countries such as india and brazil, citizens exhibited more negative opinions (praveen et al., 2020a; praveen et al., 2021). privacy concerns have been identified as one of the major reasons behind negative feelings (praveen et al., 2020a; praveen et al., 2021; crable & sena, 2020). recent works in the privacy literature (fiesler & hallinan, 2018; gonzalez et al., 2019a, 2019b) have also highlighted the limitations of survey-based methods in privacy and have begun to explore the potential of ugc on sm platforms such as facebook and twitter to reveal users’ privacy attitudes and opinions “in the wild”. our work incorporates privacy-related emotions as key informants of privacy concerns within the privacy calculus theory in the context of potentially privacy-invasive contact tracing apps. the study unveils that negative emotions are related to higher perceived privacy risks and the perceived lack of privacy protection measures, and these emotions are often the result of privacy-related stimuli around the app such as negative reports from news media, statements of privacy experts, ethical hackers, and activists. similarly, users exhibit positive emotions when they feel that the government has taken sufficient steps to protect their privacy, also a privacy-related stimulus. based on these findings, the government could devise suitable strategies to assuage users’ negative feelings and promote positive emotions australasian journal of information systems mehta, de & chattopadhyay 2022, vol 26, research article elucidating the role of emotion in privacy-concerns 3 to encourage the adoption of contact tracing apps. in contrast to traditional survey-based approaches, the methodological contribution of this research constitutes a quick, reliable, and cost-effective social media analytics-based approach to study potential app users’ emotions to gain insights into privacy concerns related to any e-governance technology or platform. in addition, the study reveals differences in user emotions around certain aspects across different contact tracing apps which prompts further future investigation into why such differences exist. in light of the above discussion, the following research questions are examined by this paper: • how can user-generated content (ugc) on social media be mined for insights into users’ perceived privacy risks and perceived privacy protections for contact tracing apps? • how do emotions on perceived privacy risks and perceived privacy protections for the indian and singaporean contact tracing apps differ? the rest of the paper is organized as follows. the first section introduced the paper. the second section discusses the theoretical background of this research. the third section describes the method adopted to analyse twitter data to gain citizen privacy insights. the fourth section highlights important findings. subsequently, the fifth section summarizes the theoretical and practical implications of the paper. in the final section, we conclude the paper with directions for future research. 2 theoretical background and review of literature in the face of a deadly pandemic such as covid-19, intuition dictates that people would be more concerned about their health leading to an increased willingness to adopt contact tracing apps, even at the potential cost of their privacy. however, experiments conducted across france, australia and the united states show that salient covid-19 concerns decrease the intentions of using contact tracing apps due to increased privacy concerns (chan & saqib, 2021). a longitudinal, survey-based examination of the competing influences of positive beliefs and privacy concerns on citizens’ acceptance of a contact tracing app shows that citizens’ privacy concerns demonstrate a negative influence on the willingness to rely on the application (fox et al., 2021). privacy concerns, informed by users’ perceived privacy risks and perceived privacy protection, relate to negative beliefs and users balance such beliefs with positive ones to decide whether to disclose their personal data, as suggested by the privacy calculus theory (pct). emotions have been found to play a key role in privacy-related decision making and users’ emotional responses to privacy aspects of contact tracing apps can be looked upon as key drivers of negative beliefs. therefore, in this study, we explore how privacy-related emotions can be extracted from user generated content (ugc) on social media. in the rest of this section, we discuss the theoretical background of our work and related prior works. 2.1 emotions, decision-making and privacy emotions have been defined as “felt tendency toward anything intuitively appraised as good (beneficial) or away from anything intuitively appraised as bad (harmful)” (arnold, 1960). emotions are intense, short-lived, and highly conscious affective states (frijda, 1994) and are relational or directed at a particular object (smith & kirby, 2000). australasian journal of information systems mehta, de & chattopadhyay 2022, vol 26, research article elucidating the role of emotion in privacy-concerns 4 emotions have been assumed to be the dominant driver of meaningful decisions (ekman, 2007; keltner et al., 2014; keltner & lerner 2010; loewenstein et al., 2001). integral emotions or emotions arising from the judgment or choice at hand strongly and routinely shape decision making (damasio, 1994; greene & haidt, 2002). the emotion-imbued choice (eic) model that account for both traditional (rational choice) inputs and emotional inputs depicts that the characteristics of the choice options as well as predicted emotions can influence emotions felt at the time of decision, ultimately impacting the decision (lerner et al., 2015). the affect-as-information model also explains how emotions may serve as affective feedback that guides people’s judgement, information processing and decisions (clore et al., 2001). feelings and emotions can be drivers of privacy decisions (kehr et al., 2013; li et al., 2011; wakefield, 2013; kehr et al., 2015). emotions of joy and fear, formed from website interactions, can significantly influence online consumers’ privacy protection beliefs and perceived privacy risks (li et al., 2011). the effect of the creepiness emotion which is a mixture of fear, anxiety, and strangeness, towards privacy harms associated with new technological features have also been discussed in the privacy literature (zhang & xu, 2016). people may find online behavioural targeting to be creepy (ur et al., 2012) which can discourage online consumers’ purchase intention (barnard et al., 2014). in the context of healthcare, anderson and agarwal (2011) show that emotions linked to one’s health condition play a significant role in the willingness to disclose health information. the interaction with a contact tracing app represents a novel privacy situation where users lack complete knowledge. in such a situation, they cannot rely only on cognition to evaluate the privacy situation and instead, as postulated by the feeling-as-information theory, emotions could provide important feedback about the app’s privacy character, further influencing the decision on whether to use the app (li et al., 2017). we explore users’ positive and negative emotions stimulated by privacy-related aspects of contact tracing apps. these stimuli may include the privacy features of the app itself (such as privacy policy, permissions, etc.) and interviews, news and/or other publications regarding the app’s privacy (such as discussions by privacy experts, news on regulatory measures on protection of data collected by the app, etc.). negative privacy emotions may be associated with a higher privacy risk perception and less appreciation of existing protection measures for the app, discouraging its use. positive privacy emotions, on the other hand, can indicate low privacy risk perception and higher appreciation of existing protection measures, encouraging the use of the app. 2.2 overview on privacy calculus theory the privacy calculus theory (pct) has been widely used to enhance the understanding of how users evaluate the fairness of disclosing personal information (keith et al., 2013; sun et al., 2015, xu et al., 2011; gutierrez et al., 2019). it suggests that a person's intention to disclose personal information is based on a calculus of behaviour, referred to as privacy calculus, in which potentially competing factors are weighed in the light of possible outcomes (dinevet al., 2008). it assumes that a rational decision-making process involves a cost-benefit analysis where privacy risks represent costs and the transaction itself gives rise to benefits and the resulting calculus determines willingness to disclose information (li, 2012; wigan, 2020; wildenauer, 2020). within this theory, factors such as perceived risks and vulnerability, computer anxiety and previous experience with privacy invasion have been shown to raise privacy concerns and discourage information disclosure whereas factors such as website australasian journal of information systems mehta, de & chattopadhyay 2022, vol 26, research article elucidating the role of emotion in privacy-concerns 5 reputation and vendor interventions such as privacy policies have been shown to mitigate privacy concerns and encourage information disclosure (li, 2012). in the context of healthcare, prior research has extended the privacy calculus to explicitly incorporate health status related emotion as a key driver of health information disclosure decision (anderson & agarwal, 2011). kehr et al. (2015) explored the role of an individual’s current mood, induced before the actual decisive situation occurred on privacy-related risk perceptions within a situational privacy calculus. while the benefits of contact tracing apps have been well-advertised by governments and the world health organization (who), privacy risks and measures taken to protect users from these risks remain much less discussed. therefore, taking the negative beliefs of privacy concerns as the base of our paper, we examine citizens’ candid expressions of emotions related to their perceived privacy risks and protections of contact tracing apps on the social media platform twitter. we thus look upon privacy emotions stimulated by privacy aspects of a technology as a key informant of user privacy concerns in privacy calculus theory, within the context of contact tracing apps. 2.3 overview on perceived privacy risk perceived privacy risk has been defined as the fear of the potential losses that would be incurred if personal information is disclosed without permission (featherman & pavlou, 2003) and the perceived risk of opportunistic behaviour related to the disclosure of personal information (dinev & hart, 2006). consistent with expectancy theory’s explanation that individuals are motivated to minimize negative outcomes, perceived privacy risk has been shown to negatively influence intention to disclose personal information (malhotra, kim, and agarwal, 2004). users of contact tracing apps fear the emergence of a surveillance state and the creation of detailed profiles that could lead to discrimination and stigmatization (praveen et al., 2020b; rowe, 2020; fahey & hino et al., 2020; georgieva et al., 2021). perceived privacy risks for non-commercial transactions, especially those within e-government, have received very less attention till date. risk perceptions can negatively affect user’s inclination to use e-government services (beldad et al., 2011). we represent perceived privacy risk through users’ negative emotions of fear and/or anger towards negative consequences due to opportunistic behaviour of the government. 2.4 overview on perceived privacy protection perceived privacy protection refers to users’ perception of the likelihood that the service provider with whom they share their personal data will adopt enough measures to protect such data from misuse such as undesirable disclosure or unauthorized use (kim et al., 2009; shaw & sergueeva, 2019). with increasing awareness, users are concerned about their privacy protection. they expect service providers to protect them from any potential risks, loss, or fraud (featherman et al., 2010). we represent perceived privacy protection as citizens’ perception of the effectiveness of the privacy protection measures being used by contact tracing apps. they may express positive emotion of happiness when they feel satisfied with a certain protection measure and may similarly express negative emotions of anger or fear when they feel dissatisfied with a protection measure. 2.5 social media analytics in opinion mining and sentiment analysis social media (sm) analytics combines, extends, and adapts methods for the analysis of social media data (stieglitz et al., 2014; zeng, et al., 2010, stieglitz et al., 2018) and involves the following three steps: 1) capturing relevant social media data by monitoring or listening, to australasian journal of information systems mehta, de & chattopadhyay 2022, vol 26, research article elucidating the role of emotion in privacy-concerns 6 various social media sources, archiving relevant data and extracting pertinent information; 2) understanding the data (after removing noise) by applying a key technique, such as sentiment analysis or social network analysis and finally, 3) presenting findings in a meaningful way (fan & gordon, 2014). the area encompasses a variety of modelling and analytical techniques such as topic modelling and opinion mining (fan & gordon, 2014). opinion mining or sentiment analysis is the “computational study of opinions, feelings and subjectivity in text” (pang & lee, 2008). it leverages computational linguistics, natural language processing, and other text analytics methods (fan and gordon, 2014). social media platforms have enabled users to easily express and share their thoughts and opinions. thus, sm platforms constitute a valuable resource for a variety of applications requiring insights on the public opinion about a concept. in recent years, sentiment analysis in the microblogging site twitter has attracted a lot of attention from researchers as many users share opinions, thoughts, and, in general, any kind of information about any topic of their interest on this platform (giachanou & crestani, 2016). opinions expressed in tweets can be useful for enterprises, governments as well as users of products and services (giachanou & crestani, 2016). for instance, based on the understanding of public views, expressed through tweets, regarding different social issues a government can initiate prompt actions. similarly, potential customers of a product can decide whether to buy a product or not based on opinionated information. emotion detection or identifying various emotions from text, is a problem related to twitter sentiment analysis. while sentiment reflects a feeling, emotion reflects an attitude (tsytsarau & palpanas, 2012). twitter posts have been analysed to understand the interplay between macroscopic socio-cultural events, such as the outcome of a political election, and the public’s mood state (bollen et al., 2011). much of the work on emotion analysis focuses on the six ekman emotions of joy, sadness, anger, fear, disgust, and surprise (ekman, 1992) whereas some others also focus on complex emotions such as politeness, rudeness, embarrassment, deception, confidence and confusion (mohammad, 2012). in this work, we analyse privacy-related positive and negative emotions around contact tracing apps to understand user perceptions on privacy risks and protection for such apps. 2.6 gaps in extant literature and the contributions of this research till date, researchers have captured privacy attitudes and concerns of users through conventional survey-based approaches using questionnaires (malhotra et al., 2004; balapour et al., 2020; xu et al., 2005; jarvenpaa et al., 2000; buchanan et al., 2007). survey-based methods in privacy have several limitations, including limited sample size, availability mostly in english, and costly to conduct in a multinational or global format (ur & wang, 2013; gonzalez et al., 2019b). thus, recent works (raber & kruger, 2018; gonzalez et al. 2019a, 2019b; fiesler & hallinan, 2018) have begun to explore text mining as an alternative approach. social media (sm) platforms are popular spaces for users to share their candid opinions and experiences and have large amounts of cross-cultural user-generated content (ugc) (rathore & ilavarsan, 2020; gonzalez et al, 2019b). as opposed to asking users directly about their privacy concerns as a part of a research study, this source of data, being a participant-driven response, can provide authentic and accurate reactions to real situations revealing issues that matter to the users (fiesler & hallinan, 2018). sm has the potential to capture users’ behavioural intentions by obtaining emotions (rathore & ilavarsan, 2020) related to various issues such as the introduction of contact tracing apps by the government. australasian journal of information systems mehta, de & chattopadhyay 2022, vol 26, research article elucidating the role of emotion in privacy-concerns 7 since emotions are short-term responses, compared to other traditional approaches (such as surveys and reviews in newspapers) to assess such responses, sm platforms can gauge emotions faster (rathore & ilavarsan, 2020). in addition, compared to traditional methods, sm provides information at a lower cost, in real-time and ugc is sufficient in terms of the amount of data, the relative lack of bias, the cost of the data and enables the unique opportunity to analyse cross-cultural, multi-language data (rathore and ilavarsan, 2020; gonzalez et al. 2019b). reference examination of user’s perceived privacy risks/ concerns examination of user’s perceived privacy protections comparison across geographies examination of ugc on social media analysis of users’ privacyrelated emotions context of digital contact tracing our work       fox et al. (2021)  × × × ×  chan & saqib (2021)  ×  × ×  lin et al. (2021)  × × × ×  walrave et al. (2020)  × × × ×  horvath et al. (2020)   × × ×  praveen et al., (2020a) × ×   ×  crable & sena (2020) × × ×  ×  praveen et al. 2021 × ×   ×  simko et al. (2020)  × × × ×  gonzalez et al. (2019a)  ×   × × gonzalez et al. (2019b)  ×   × × table 1. research gaps and our contributions several recent research works have highlighted numerous privacy concerns on contact tracing apps such as profiling and tracking of specific users (wen et al., 2020; bengio et al., 2021; leith & farrell, 2020), de-anonymization of infected persons (baumgärtner et al., 2020), data misuse by third parties (azad et al., 2020) and government surveillance (rowe, 2020; fahey & hino, 2020). some studies have explored how privacy concerns have affected the willingness of citizens to adopt contact tracing apps in different countries such as ireland, france, australia and belgium. within this literature, most works have adopted a surveybased approach (fox et al., 2021; chan & saqib, 202; walrave et al., 2020; horvath et al., 2020). citizens’ privacy concerns have been demonstrated to have a negative impact on their willingness to adopt contact tracing apps. a limited number of studies have also been conducted to understand citizen opinion on contact tracing in different countries. while in ireland, citizens exhibited mostly positive opinion about contact tracing (rekanar et al., 2020), in some other countries such as india and brazil, citizens exhibited more negative opinions (praveen et al., 2020a; praveen et al., 2021). privacy concerns have emerged as one of the major reasons behind negative feelings towards contact tracing (rekanar et al., 2020; praveen et al., 2020a; praveen et al., 2021; crable & sena, 2020) although such concerns have australasian journal of information systems mehta, de & chattopadhyay 2022, vol 26, research article elucidating the role of emotion in privacy-concerns 8 not been explored further by these works. our work builds on the existing literature but is different in the following ways. first, we show how emotional information embedded in user-generated content (ugc) from social media platform can be elicited to gain privacyrelated insights for a given technology or platform. specifically, through emotion analysis of tweets, we attempt to understand citizen privacy perceptions about the indian contact tracing app aarogya setu and the singaporean contact tracing app tracetogether which were among the first apps of this kind to be launched. secondly, we examine users’ privacyrelated emotions on perceived privacy risks and perceived privacy protections surrounding these apps thus assessing the negative beliefs (privacy concerns) of users in the privacy calculus model. in the broader context, this privacy calculus would directly affect the willingness to disclose personal data and hence the acceptance of contact tracing apps. thirdly, we compare the results for apps introduced by two different countries to understand how these emotions varied for these apps. to the best of our knowledge, we are the first to examine privacy-related emotions on contact tracing apps. we also extend the research on how ugc on sm can be mined using text-convolutional neural network (text-cnn) to derive privacy insights on contact tracing apps. we have organized and presented our contributions vis-à-vis key related research works in table 1. 3 method and data we develop a method enabling fine-grained extraction of user’s positive and negative emotions around possible privacy harms from contact tracing apps from ugc to understand their perceived privacy risks and perceived privacy protections without relying on traditional, survey-based methods. perceived privacy risk is represented by a negative emotion around a privacy harm related topic concerning the app. similarly, perceived protection is represented by a positive or a negative emotion around a privacy protection topic concerning the app. we adopt social media analytics for twitter data collection and analysis. sm analytics provides many cost-effective and real-time approaches to gain an understanding of user sentiments (rathore & ilavarsan, 2020; rathore et al., 2021). in twitter, ugc is mainly textual content in the form of short posts, called tweets. the character restriction on tweets makes them highly expressive compared to other social media posts (pak & paroubek, 2010; singh et al., 2019). in the remainder of this section, we present the approach we adopted to extract privacy related insights from twitter and specific methods used to analyse and visualise data. we develop a novel text-convolutional neural network (text-cnn) to perform emotion analysis using transfer learning on the collected dataset. a collocation analysis is then performed to retrieve relevant bigrams and trigrams and calculate their association measures using pmi and chi-square test. user emotions towards the app are identified through tweets relevant to unigrams, bigrams, and trigrams that fall in the vicinity of privacy related terms. 3.1 creation of target datasets data pre-processing begins with the selection of the event. in our case, it is the launch of contact tracing apps, aarogya setu in india, and tracetogether in singapore. enough data was available and could be extracted for our choice of event for further analysis. australasian journal of information systems mehta, de & chattopadhyay 2022, vol 26, research article elucidating the role of emotion in privacy-concerns 9 figure 1. a tweet on aarogya setu figure 2. a tweet on tracetogether in the first stage of data pre-processing, suitable keywords were selected. this was followed by the second stage where tweets were extracted, and the third stage consisted of tweet preparation. the data source for ugc is the micro-blogging site twitter. the keywords selected for extracting tweets related to the aarogya setu app include aarogya setu, aarogyasetu, and various other combinations of these terms. similarly, keywords were chosen for extracting tweets related to the tracetogether app. we extracted historical tweets from the twitter api. a list of individual tweet ids were first obtained and then hydrated using a custom hydrating algorithm. all the extraction and hydration processes were carried out using python 3 on google colab. all tweets that include the app’s name in their text, mention, or hashtag were collected for the period beginning with the launch of that app. for aarogya setu, tweets between the dates 2nd april, 2020 to 28th february, 2021 and for tracetogether, tweets between the dates 29th march, 2020 to 28th february, 2021 were collected. sample tweets for aarogya setu and tracetogether have been shown in figures 1 and 2 respectively. twitter data contains many languages, unstructured abbreviations, and grammar. therefore, after retrieving tweets, it is necessary to convert them into a meaningful format. firstly, nonascii characters were removed, followed by eliminating punctuations, numeric values, and urls. unnecessary words referred to as stop words (prepositions, determiners, conjunctions, etc.) that do not contribute to our analysis were filtered out using natural language toolkit (nltk). next, the entire corpus was lemmatized to keep only root words for analysis. we only considered tweets in english. ultimately, our first dataset consisted of 89005 tweets related to aarogya setu and the second dataset contained 10997 tweets related to tracetogether. these target datasets were annotated to evaluate the performance of our approach. each item is labelled into one of the nine emotion categories: anger, disgust, fear, guilt, sadness, shame, surprise, happiness, and neutral. the datasets were strongly labelled by six human annotators. the annotators were selected based on their domain knowledge, linguistic knowledge, and prior annotation experience. considering the huge size of the target datasets, the annotation task was divided into two sets of 50k tweets each. each set was labelled by three individual annotators. the intercoder reliability scores, calculated to measure the agreement among the annotators, accounted for 91% and 96% for aarogya setu and tracetogether, respectively. australasian journal of information systems mehta, de & chattopadhyay 2022, vol 26, research article elucidating the role of emotion in privacy-concerns 10 3.2 creation of auxiliary dataset we prepare an auxiliary sample set for training the proposed algorithm by combining publicly available datasets pertaining to the emotion analysis task. we combine eight publicly available text-emotion analysis datasets to form a single large auxiliary set. the datasets used in the combination are isear1, dailydialog2, emotionstimulus3, semeval 20194, emobank5, emoint6, crowdflower7, and emotion lines8. table 2 depicts the size and labels in each of these eight datasets. dataset size labels isear 7473 sentences joy (1081), sadness (1067), fear (1081), anger (1071), guilt (1052), disgust (1067) and shame (1054) dailydialog 102,879 dialogues extracted from conversations happiness (12885), sadness (1150), anger (1022), disgust (353), fear (74), surprise (1823), and others (85572) emotionstimulus 2414 sentences anger (483), disgust (95), fear (423), happy (479), sad (575), shame (146), surprise (213) semeval 2019 38,424 happy (4669), sad (5838), angry (5954), others (21,963) emobank 10k news headline, essays, blogs, newspapers, fiction, letters and travel guides dimensional labelling according to valence (v), arousal (a) and dominance (d) with numeric values in the range [1,5] relabeled as: anger (516), disgust (62), happy (3584), neutral (2629), sad (379), shame (119), surprise (2773) emoint 3960 items anger (941), fear (1257), joy (902), sadness (860) crowdflower 39,998 tweets anger (110), boredom (179), empty (827), enthusiasm (759), fun (1776), happiness (5209), hate (1323), love (3842), neutral (8638), relief (1526), sadness (5165), surprise (2187), worry (8459) emotion lines (el) 29,245 utterances obtained from dialogue in friends tv show and facebook messenger chats. friends: neutral (6531), joy (1710), sad (497), fear (247), anger (759), surprise (1658), disgust (331), non-neutral (2772) emotionpush: neutral (9855), joy (2101), sad (514), fear (40), anger (140), surprise (568), disgust (106), nonneutral (1418) table 2. details of datasets used in auxiliary set to ensure a definite scale of emotions, the auxiliary dataset is filtered to contain a specific set of nine emotion labels: happiness, sadness, neutral, fear, anger, disgust, guilt, shame, and surprise. these nine labels broadly cover the type of emotions perceived by a human being. however, the datasets constituting the auxiliary set contain varied labels. we map these items with multiple emotion labels to the desired nine label categories, as described in table 1 https://www.kaggle.com/shrivastava/isears-dataset (accessed on 20-feb-2021) 2 https://www.aclweb.org/anthology/i17-1099/ (accessed on 20-feb-2021) 3 https://www.site.uottawa.ca/~diana/resources/emotion_stimulus_data/ (accessed on 20-feb-2021) 4 https://www.humanizing-ai.com/emocontext.html (accessed on 15-feb-2021) 5 https://github.com/julielab/emobank (accessed on 10-feb-2021) 6 http://saifmohammad.com/webpages/emotionintensity-sharedtask.html (accessed on 10-feb-2021) 7 https://www.crowdflower.com/wp-content/uploads/2016/07/text_emotion.csv (accessed on 10-feb-2021) 8 https://sites.google.com/view/emotionx2019/datasets (accessed on 10-feb-2021) https://www.aclweb.org/anthology/i17-1099/ https://github.com/julielab/emobank http://saifmohammad.com/webpages/emotionintensity-sharedtask.html australasian journal of information systems mehta, de & chattopadhyay 2022, vol 26, research article elucidating the role of emotion in privacy-concerns 11 3. items labelled as joy in isear, emoint, and emotion lines are categorized as happiness in the auxiliary set. the remaining items labelled with emotions – anger, disgust, fear, guilt, sadness, shame, surprise, and neutral are used with the same labels in the auxiliary set. the emobank dataset originally contains dimensional labelling by providing a score between 1 to 5 for three attributes – valence (v), arousal (a), and dominance (d). its mapping on the emotional scale was developed by the annotators categorizing the items into anger, disgust, sadness, shame, surprise, happy, and neutral. the categorization of an item in each emotional category is carried out according to the rules described in table 3. the items labelled with emotions other than the nine categories that we use, such as boredom, empty, enthusiasm, fun, hate, love, relief, worry, others, and non-neutral were not used in the auxiliary set. datasets anger disgust fear guilt sadness shame surprise happiness neutral isear anger disgust fear guilt sadness shame joy dailydialog anger disgust fear sadness surprise happiness emotionstimulus anger disgust fear sad shame surprise happy semeval 2019 angry sad happy emobank 2.5≥v, d<3, a≥3 2.5≥v, d≥3, a<3 2.5≥v, d≥3, a≥3 2.5≥v, d<3, a<3 {v≥3, d≥3, a≤3} + {v≥3, d<3} {v≥3, d≥3, a≥3} {2.5> v>3} emoint anger fear sadness joy crowdflower anger sadness surprise happiness neutral el (friends) anger disgust fear sad surprise joy neutral el (emotionpush) anger disgust fear sad surprise joy neutral table 3. emotion mapping emotion category no. of items in auxiliary dataset no. of items in target datasetaarogyasetu no. of items classified by text-cnn in aarogyasetu no. of items in target datasettracetogether no. of items classified by text-cnn in tracetogether anger 10996 20894 21540 1699 1734 disgust 2014 1373 890 356 448 fear 3122 21886 22251 1642 1786 guilt 1052 325 89 138 22 sadness 16045 3221 2850 312 169 shame 1319 336 180 81 34 surprise 9222 4673 3735 300 277 happiness 32620 24656 25010 4940 5021 neutral 27653 11641 12460 1529 1506 table 4. number of items in each emotion category upon annotation and classification table 4 depicts the number of text items in the auxiliary dataset and the target datasets for each emotion category. it also illustrates the number of items in each emotion class according to manual annotation and text-cnn classification. it demonstrates that the auxiliary dataset is rich in items with ‘anger’, ‘sadness’. ‘happiness’, and ‘neutral’ labels. however, the remaining emotions on a fine-grained scale contain lesser instances. this data imbalance is australasian journal of information systems mehta, de & chattopadhyay 2022, vol 26, research article elucidating the role of emotion in privacy-concerns 12 due to low availability of annotated emotion classification datasets. however, the auxiliary dataset is sufficient to train a deep neural network effectively. 3.3 proposed method in this section, we discuss the method we adopted to derive privacy insights from user tweets. it consists of collocation analysis and emotion analysis. the method was applied separately for aarogya setu and tracetogether datasets. collocation analysis is performed to analyse linguistic, syntactic, and lexical relations in sentences to identify words that occur together more often than expected. we first analysed the meaningful bigrams (two words taken together) and trigrams (three words taken together) extracted from the collected dataset. collocations are retrieved using nltk’s bigram and trigram finders. the method returns all significant co-occurrences of terms with their frequencies or association score. for computing association score, pointwise mutual information (pmi) is calculated. mutual information is an information theoretic concept (church & hanks, 1990). if two points or words, x and y have probabilities p(x) and p(y), then their pmi, i (x, y) is defined as 𝐼𝐼(𝑥𝑥,𝑦𝑦) = 𝑙𝑙𝑙𝑙𝑙𝑙2 𝑃𝑃(𝑥𝑥,𝑦𝑦) 𝑃𝑃(𝑥𝑥)𝑃𝑃(𝑦𝑦) thus, pmi compares the probability of observing x and y together with the probabilities of observing x and y independently. if there is an association between x and y, then the joint probability p(x, y) will be much larger than the chance probability p(x)p(y) leading to i(x,y)≫0. on the other hand, if there is no such association between x and y, then p(x, y) ≈ p(x)p(y) and thus i(x, y) ≈ 0. chi-square test is used as another measure to calculate the association among bigrams and trigrams. using collocation analysis, we obtain bigrams and trigrams along with their frequency of usage, pmi score and chi-square test of co-occurrence. a positive pmi value signifies that the number of co-occurrences of the individual words as bigrams and trigrams is slightly less than the number of occurrences of those words individually or greater than expected cooccurrences. a negative pmi value means that the words in bigrams and trigrams occur together significantly less than they occur individually. we use the chi-sq test as a criterion for our analysis where chi-sq test statistic value is greater than 3.841 for bigrams and greater than 5.991 for trigrams holds the significance of their co-occurrence. the bigrams and trigrams with higher frequency, greater pmi score, and a higher chi-sq statistic value represent what users discussed the most about in the tweets. we employed transfer learning to perform emotion classification on our datasets. transfer learning is beneficial when it is required to transfer knowledge-based features to a target domain. this method is helpful in circumstances where data is not annotated, and transfer learning can be used to perform unsupervised classification on the target dataset by learning the feature representations from a similar auxiliary labelled dataset. the proposed method offers an advantage over traditional training and testing procedure of a machine learning algorithm on a labelled dataset. in supervised classification scenarios, data requires high quality manual annotations which is costly and resource expensive in the real-world scenario. also, in situations such as the recent pandemic, where credible research results are critically needed, it is not feasible to readily collect, annotate, and devise algorithms for early classification. the proposed transfer-learning-based method is advantageous given that the australasian journal of information systems mehta, de & chattopadhyay 2022, vol 26, research article elucidating the role of emotion in privacy-concerns 13 algorithm uses transferred domain knowledge obtained upon training a neural network on a large-sized auxiliary dataset which can be readily used to generate predictions for any unlabelled real-time testing data upon performing fine-grained emotion classification. figure 3. architectural representation of proposed text-cnn for emotion analysis the proposed architecture of text-based convolutional neural network (cnn) for emotion analysis is demonstrated in figure 3. initially, the proposed text-cnn classifier is trained on a large set of labelled samples, the auxiliary dataset. then, textual feature representations obtained from training on the auxiliary set are transferred to the target domain to label the target set. the auxiliary dataset is split in the ratio 8:1:1 for training, validation, and testing, respectively. the proposed convolutional neural network is trained on the auxiliary dataset for 20 epochs with a batch size of 64. the weighted instances from the auxiliary set are used to extract labels on the target dataset of tweets. the entire framework was built and implemented on google colab using python 3 with an allocated ram of 13.53 gb and nvidia tesla k80 gpu. 3.4 experimental performance we discuss the experimental performance of our transfer learning-based text-cnn approach to establish the efficacy of the proposed algorithm. the results are demonstrated for both aarogyasetu and tracetogether datasets through confusion matrices, and four evaluation metrics accuracy, precision, recall, and f1-score. a detailed error analysis is also performed to identify the possible causes of misclassification of tweets. tables 5 and 6 represent the confusion matrices for aarogyasetu and tracetogether datasets, respectively. table 7 displays the class-wise precision, recall and f1-scores for emotion classification for aarogyasetu and tracetogether datasets. it is evident from figure 5 that ~80% of the tweets from each category are correctly classified for aarogyasetu. for the tracetogether dataset, happiness and neutrality is correctly identified for most of the tweets, however guilt and shame are poorly classified due to the low number of tweets in these emotion categories. the testing accuracies on aarogyasetu and tracetogether are found to be 83.022% and 89.406%, respectively. this performance is exceptionally well given that the classification model uses domain transferrable knowledge for training instead of traditional supervised learning. the accuracy scores demonstrate the proposed method’s efficacy to r input text happiness sadness fear anger disgust guilt shame surprise d ro po ut so ftm ax embedding layer convolution + relu layers max pooling dense re lu a ct iv at io n k ar na ta ka h ig h co ur t b ar s c en tre fr om sh ar in g a ar og ya se tu ap p da ta w ith ou t u se r c on se nt australasian journal of information systems mehta, de & chattopadhyay 2022, vol 26, research article elucidating the role of emotion in privacy-concerns 14 generate fine-grained emotion labels on unseen real-world data. however, accuracy is not an appropriate measure when the evaluation dataset is imbalanced. to have a better picture of the model’s efficiency, we calculate the f1-scores for classification on both datasets. from table 5, we can analyse the class-wise performance observing that all the categories achieve f1-scores in the range of 75% to 87% for aarogyasetu. similarly, anger, fear, sadness, surprise, happiness, and neutrality fall in the f1-score range of 61% to 91% for tracetogether. low f1scores are obtained for the disgust, guilt, and shame categories with 57.69%, 26.96%, and 40.57%, respectively. anger disgust fear guilt sad shame surprise happy neutral anger 0.82 0.02 0.06 0.00 0.01 0.00 0.01 0.04 0.03 disgust 0.06 0.79 0.01 0.02 0.02 0.02 0.03 0.00 0.06 fear 0.05 0.01 0.83 0.00 0.01 0.00 0.02 0.06 0.02 guilt 0.02 0.04 0.02 0.80 0.04 0.02 0.02 0.00 0.02 sad 0.04 0.01 0.03 0.00 0.79 0.01 0.01 0.08 0.02 shame 0.06 0.04 0.00 0.02 0.01 0.82 0.00 0.02 0.02 surprise 0.04 0.01 0.04 0.00 0.01 0.00 0.84 0.04 0.02 happy 0.04 0.00 0.05 0.00 0.01 0.00 0.03 0.85 0.01 neutral 0.07 0.00 0.05 0.00 0.00 0.00 0.00 0.07 0.80 table 5. confusion matrix for aarogyasetu dataset anger disgust fear guilt sad shame surprise happy neutral anger 0.86 0.03 0.02 0.01 0.04 0.01 0.01 0.00 0.01 disgust 0.16 0.53 0.05 0.03 0.07 0.03 0.02 0.00 0.11 fear 0.05 0.01 0.86 0.02 0.02 0.00 0.01 0.01 0.02 guilt 0.09 0.05 0.14 0.23 0.27 0.14 0.00 0.00 0.09 sad 0.07 0.04 0.02 0.05 0.66 0.07 0.02 0.02 0.05 shame 0.03 0.12 0.06 0.32 0.06 0.32 0.03 0.00 0.06 surprise 0.07 0.05 0.08 0.01 0.00 0.01 0.71 0.06 0.00 happy 0.00 0.00 0.00 0.00 0.00 0.00 0.01 0.97 0.01 neutral 0.00 0.01 0.01 0.01 0.03 0.01 0.00 0.02 0.91 table 6. confusion matrix for tracetogether dataset class aarogya setu results (%) trace together results (%) precision recall f1-score precision recall f1-score anger 82.27 84.81 75.24 86.10 87.88 73.65 disgust 78.65 50.98 81.94 52.68 66.29 57.69 fear 83.07 84.46 79.33 86.39 93.97 77.27 guilt 79.78 21.85 86.18 22.73 3.62 26.96 sad 79.19 70.07 82.48 65.68 35.58 61.12 shame 82.22 44.05 86.99 32.45 13.58 40.57 surprise 84.34 67.41 85.72 70.76 65.33 78.70 happy 85.39 86.61 78.86 96.91 98.50 92.81 neutral 80.33 85.98 79.93 91.04 89.67 80.15 table 7. results of emotion classification on aarogyasetu and tracetogether datasets to analyse the potential causes of misclassification, we demonstrate the error analysis for aarogyasetu and tracetogether in table 8 and table 9, respectively. we calculate the total number of items misclassified for each category and list the count of misclassified tweets in australasian journal of information systems mehta, de & chattopadhyay 2022, vol 26, research article elucidating the role of emotion in privacy-concerns 15 each of them. class-wise error percentage is also calculated and listed in the tables. we also try to provide possible causes or hypothesis for some of the misclassifications. for aarogyasetu, a total of 3820 anger tweets out of 20,894 were misclassified with the maximum misclassification of 1331 tweets (34.84% of total errors in anger classification) in the fear category. it was followed by 807 tweets (21.13%) misclassified as happiness. taking these two numbers as matter of concern, we hypothesize that some parts of speech might deceive the algorithm to misclassify anger as fear, but some strongly used words in phrases like “joy turned downhill” were causes for misclassification of anger as happiness. other misclassification account to 16.94% as neutral, 9.4% as disgust, 8.25% as sadness, 1.36% as guilt, and 0.65% as shame. the surprise emotion can be both positive as well as negative accounting to 7.43% of the anger misclassification. out of 1373 disgust tweets, 190 misclassifications are present. 50 tweets each have been misclassified as anger and neutral. sadness and shame labels take 21 tweets each as misclassifications, whereas 23 tweets are misclassified as surprise followed by 15 tweets being misclassified as guilt, 8 as fear and 2 as happiness. for the fear emotion, the algorithm misclassified 32.94% of tweets as happiness followed by 28.30% as anger. similar reasons were observed for fear being misclassified as anger as was for the vice-versa. the algorithm cannot distinguish emotion mixed with sarcasm and thus tweets like “secularism is like aarogyasetu app. you have to declare yourself that you are sick or well irrespective of what actually you are” these are mapped as happiness instead of appropriate emotion, which in this case would be ‘fear’. other misclassifications were relatively insignificant in proportion and could not be attributed with a specific cause. the smaller quantity of tweets related to guilt and shame emotions makes them both difficult and unnecessary for error analysis. 39.46% of the tweets with sadness emotion were misclassified as happiness. this can be attributed to tweets that contained multiple positive words against lesser number of negative words despite the context of the tweet conveying a sad emotion. one such example is the tweet “i agree with you fully. in india, funding for r & d is driven by success, profit, negative mindset to serve vested interests. in west, it is driven by vision, mission. the sound by negativists on the aarogyasetu app is one such clear example.” this is followed by anger with 16.86% and fear with 14.84% as misclassification. surprise can be both positive and negative and hence find similar proportion of misclassification as fear (26.67%), happiness (25.98%) and anger (25.30%). tweets with happiness emotion are misclassified majorly misclassified as fear (32.89%), anger (26.32%) and surprise (20.27%). higher proportions of negative words with positive context, was found to be the key reason for such misclassification. some neutral tweets were randomly misclassified as anger, fear and happiness in majority amongst others. for tracetogether tweets, similar reasons can be attributed for the misclassifications. tweets with guilt, sadness and shame emotions were present in negligible proportions and are insignificant for error analysis. tweets with anger emotion were misclassified as sadness (31.54% of total misclassifications), followed by disgust (24.48%), and fear (13.69%) with other negligible misclassifications. tweets with disgust emotion were majorly misclassified as anger (33.96%), neutral (24.06%) followed by sadness (14.62). fear emotion tweets were misclassified as anger (33.33% of total misclassification in the fear category), guilt (18.11%), and neutral (15.23%). tweets with surprise and happiness emotions had higher accuracy and hence, less misclassification. although, a bias can be observed where tweets with surprise emotion are majorly misclassified as fear (25.93%), anger (24.69%) and happiness australasian journal of information systems mehta, de & chattopadhyay 2022, vol 26, research article elucidating the role of emotion in privacy-concerns 16 (22.22%) like the aarogyasetu tweets, which shows the positive and negative variation in surprise emotion. tweets with happiness emotion were majorly misclassified into surprise (41.94%). anger classified as disgust fear guilt sadness shame surprise happiness neutral count 359 1331 52 315 25 284 807 647 % of total errors 9.40 34.84 1.36 8.25 0.65 7.43 21.13 16.94 disgust classified as anger fear guilt sadness shame surprise happiness neutral count 50 8 15 21 21 23 2 50 % of total errors 26.32 4.21 7.89 11.05 11.05 12.11 1.05 26.32 fear classified as anger disgust guilt sadness shame surprise happiness neutral count 1066 148 102 330 46 390 1241 444 % of total errors 28.30 3.93 2.71 8.76 1.22 10.35 32.94 11.79 guilt classified as anger disgust fear sadness shame surprise happiness neutral count 2 4 2 4 2 2 0 2 % of total errors 11.11 22.22 11.11 22.22 11.11 11.11 0.00 11.11 sadness classified as anger disgust fear guilt shame surprise happiness neutral count 100 27 88 10 42 29 234 63 % of total errors 16.86 4.55 14.84 1.69 7.08 4.89 39.46 10.62 shame classified as anger disgust fear guilt sadness surprise happiness neutral count 10 8 0 4 2 0 4 4 % of total errors 31.25 25.00 0.00 12.50 6.25 0.00 12.50 12.50 surprise classified as anger disgust fear guilt sadness shame happiness neutral count 148 27 156 8 23 4 152 67 % of total errors 25.30 4.62 26.67 1.37 3.93 0.68 25.98 11.45 happiness classified as anger disgust fear guilt sadness shame surprise neutral count 962 65 1202 63 229 38 741 355 % of total errors 26.32 1.78 32.89 1.72 6.27 1.04 20.27 9.71 neutral classified as anger disgust fear guilt sadness shame surprise happiness count 836 35 615 0 40 10 54 861 % of total errors 34.11 1.43 25.09 0.00 1.63 0.41 2.20 35.13 table 8. error analysis for emotion classification on aarogyasetu data anger classified as disgust fear guilt sadness shame surprise happiness neutral count 59 33 21 76 14 11 4 23 % of total errors 24.48 13.69 8.71 31.54 5.81 4.56 1.66 9.54 disgust classified as anger fear guilt sadness shame surprise happiness neutral count 72 22 15 31 12 7 2 51 % of total errors 33.96 10.38 7.08 14.62 5.66 3.30 0.94 24.06 fear classified as anger disgust guilt sadness shame surprise happiness neutral count 81 19 44 28 8 14 12 37 % of total errors 33.33 7.82 18.11 11.52 3.29 5.76 4.94 15.23 guilt classified as anger disgust fear sadness shame surprise happiness neutral count 2 1 3 6 3 0 0 2 % of total errors 11.76 5.88 17.65 35.29 17.65 0.00 0.00 11.76 sadness classified as anger disgust fear guilt shame surprise happiness neutral count 12 7 4 9 11 3 3 9 % of total errors 20.69 12.07 6.90 15.52 18.97 5.17 5.17 15.52 shame classified as anger disgust fear guilt sadness surprise happiness neutral count 1 4 2 11 2 1 0 2 % of total errors 4.35 17.39 8.70 47.83 8.70 4.35 0.00 8.70 surprise classified as anger disgust fear guilt sadness shame happiness neutral count 20 13 21 4 1 3 18 1 australasian journal of information systems mehta, de & chattopadhyay 2022, vol 26, research article elucidating the role of emotion in privacy-concerns 17 % of total errors 24.69 16.05 25.93 4.94 1.23 3.70 22.22 1.23 happiness classified as anger disgust fear guilt sadness shame surprise neutral count 11 7 5 16 11 7 65 33 % of total errors 7.10 4.52 3.23 10.32 7.10 4.52 41.94 21.29 neutral classified as anger disgust fear guilt sadness shame surprise happiness count 7 10 9 13 46 12 3 35 % of total errors 5.19 7.41 6.67 9.63 34.07 8.89 2.22 25.93 table 9. error analysis for emotion classification on tracetogether data 4 results and discussion in this section, we present the results of our analysis of aarogyasetu and tracetogether datasets. to understand users’ opinions from tweets related to aarogyasetu and tracetogether app, we examined what they talked about the most and how they felt about it. for the first part, we used nlp-based collocation analysis, and for the latter part, we used deep learning-based emotion analysis. the four emotions anger, fear, happy and neutral account for 91.3% and 91.4% of tweets for aarogyasetu and tracetogether apps, respectively (see figure 4). the emotions of anger and fear together account for 49.2% of the aarogyasetu tweets whereas the happy emotion accounts for just 28.1% (figure 4). for tracetogether, the emotions of anger and fear together account for 32% of the tweets, whereas the happy emotion accounts for 45.7% of the tweets (figure 4). so, overall, tracetogether tweets show more happy emotions compared to aarogyasetu tweets. figure 4. emotion classification of aarogyasetu and tracetogether tweets tweets with neutral emotions hold negligible significance in the analysis, and therefore we end up with 76545 aarogyasetu tweets and 9491 tracetogether tweets for analysis. to 21540 1734 890 448 22251 1786 89 22 2850 169180 34 3735 277 25010 5021 12460 1506 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% aarogyasetu tracetogether pe rc en ta ge o f t w ee ts emotion classification of tweets anger disgust fear guilt sad shame surprise happy neutral australasian journal of information systems mehta, de & chattopadhyay 2022, vol 26, research article elucidating the role of emotion in privacy-concerns 18 simplify the study, we form two groups of reactions: positive and negative. tweets with anger and fear emotion belong to the negative reaction group, while tweets with happy emotion belong to the positive reaction group. the remaining five (5) emotions — disgust, guilt, sad, shame and surprise, constitute a tiny proportion of tweets and were excluded to avoid discrepancies in the analysis bigrams like ‘aarogya,’ ‘setu,’ and ‘use,’ ‘tracetogether’ appear more frequently than others because they resemble or contain the app's name. trigrams like ‘ravi,’ ‘shankar,’ ‘prasad,’ also reflected in the bigram as ‘ravi,’ ‘shankar,’ which refers to a person’s name used in aarogyasetu tweets have higher co-occurrences than their occurrence (frequency of ‘ravi,’ ‘shankar,’ ‘prasad’ is 1). in other words, such bigrams and trigrams are used more frequently together than used individually or used only together and therefore appear with higher pmi and chi-sq statistic value. table 10. pmi score and chi-sq test statistic value of relevant bigrams and trigrams (aarogyasetu) to eliminate this clutter of bigrams and trigrams from the analysis, we define sixteen words (keywords) to obtain insights into perceived privacy risks and perceived privacy protections of the users. these keywords are privacy, security, tracing, surveillance, risk, data, protection, steal, leak, sharing, policy, regulation, location, hacker, breach, and cybersecurity. fourteen bigrams and thirteen trigrams for tweets of aarogyasetu and five bigrams and five trigrams for the tracetogether app were selected based on these keywords. identifying the emotions of the tweets containing these keywords, bigrams, and trigrams would provide insight into public opinion regarding these contact tracing apps. 47.94% (36700 out of 76545) s.no category word / bigram / trigram pmi score chi-sq statistic 1 bigrams data protection 8.03 45543.46 2 data security 8.89 49722.38 3 data leak 8.52 46631.13 4 personal data 7.28 38550.22 5 privacy concern 8.46 46115.6 6 breach aarogya 6.91 21242.49 7 security breach 7.91 46805.46 8 contact trace 8.51 104729.3 9 cybersecurity cyberthreat 10.02 72836.47 10 personal information 9.33 28317.33 11 data breach 7.13 33266.90 12 privacy breach 6.75 25378.84 13 tracking app 5.03 1314.2 14 ethical hacker 9.84 98650.59 15 contact trace app 16.52 3014099 16 architecture indulge surveillance 22.09 98372189 17 trigrams claim cybersecurity activist 21.41 72891958 18 healthy person privacy 16.09 4064320 19 data breach aarogyasetu 11.64 129263.7 20 setu app useless 7.73 82488.55 21 hack aarogya app 7.52 260435.3 22 hacker claim privacy 16.06 860757.1 23 privacy first design 16.08 1385968 24 potential security issue 7.18 247359.5 25 hacker raise concern 16.06 248054.3 26 aarogya setu privacy 10.71 105678 27 assure ethical hacker 19.91 45394203 australasian journal of information systems mehta, de & chattopadhyay 2022, vol 26, research article elucidating the role of emotion in privacy-concerns 19 of the aarogyasetu unique tweets and 53.15% (5045 out of 9491) of the tracetogether unique tweets contained one amongst these keywords, bigrams, and trigrams. table 10 and table 11 present the pmi score and chi-sq test statistic value of relevant bigrams and trigrams for aarogyasetu and tracetogether respectively. we have used only those bigrams and trigrams that have a higher pmi score and chi-sq statistic value indicating their higher and frequent usage in the tweets. table 11. pmi score and chi-sq test statistic value of relevant bigrams and trigrams (tracetogether) 4.1 emotion analysis of aarogyasetu tweets tweets containing keywords such as privacy, security, surveillance, risk, protection, steal, leak, hacker, cybersecurity, and breach expressed mostly with anger and fear (see figure 5). similarly, bigrams such as data breach, privacy breach, privacy concern, security breach and breach aarogya and trigrams such as architecture indulge surveillance, healthy person privacy, data breach aarogyasetu and aarogyasetu privacy expressed negative emotions (see figure 6 and figure 7). table 12. aarogyasetu tweets containing selected keywords and bigrams with high negative reaction s.no category word / bigram / trigram pmi score chi-sq statistic 1 bigrams privacy concern 7.48 11720.4 2 security issue 5.52 100.50 3 contact tracing 6.71 43337.04 4 data breach 3.31 196.57 5 law enforcement 11.11 79726.38 15 contact trace app 11.03 636702.8 16 government technology agency 18.45 7540231 17 trigrams local law enforcement 21.82 81533074 18 criminal procedure code 19.20 21036631 19 for criminal investigation 16.02 9596831 keyword/ bigram tweet privacy it isn’t just about privacy with aarogya setu. the massive amounts of data as with any citizen info set, is prime target for profiling, cyber crime and even espionage. and the threats can come from some of the most formidable of threat actors. steal aarogyasetu is a private sector effort built by the private for the private and of private resources. nandan nilekani calls this stealing of your personal data, a data democracy. policy aarogyasetu is developed on a privacy-first policy. your data is never shared with anyone. the app is our protective shield against coronavirus during this critical time. watch this video to know more! #indiafightscorona @goi_meity @rsprasad @nicmeity @pib_india @mib_india https://t.co/ywyqdskip4 data karnataka high court bars centre from sharing aarogyasetu app data without user consent data breach ‘who is accountable for data breach?’: retd justice srikrishna to tnm on aarogyasetu privacy concern now that they banned chinese apps under privacy concern. i believe that we should also consider adding “aarogya set” to that list. ethical hacker how can you support this aarogyasetu farce despite ethical hackers saying in no uncertain terms that it is an intentional tool of mass surveillance! australasian journal of information systems mehta, de & chattopadhyay 2022, vol 26, research article elucidating the role of emotion in privacy-concerns 20 except for the bigram personal information, all other bigrams constitute negative reaction in the majority (>50%). emotions of anger and fear contribute the highest in tweets that share opinions regarding security breach, are related to information shared by ethical hacker, data breach possibilities in aarogyasetu app (breach aarogya) that lead to privacy breach, and on opinions related to contact tracing. on extending the keyword data to the bigram data breach, we observe an increase of fear emotion in tweets (see figure 5 and figure 6), also reflected by a sample tweet on this bigram (see table 12) where a user has re-iterated the question raised by retired justice srikrishna on who is going to be accountable for a data breach. the tweets also show that claims made by ethical hackers (see sample tweet in table 12) and news articles on privacy fuel negative emotions of anger and fear among citizens towards the app. trigrams cover topics of contact tracing (contact trace app), architecture (architecture indulge surveillance) of the app, claims made by cybersecurity activists (claim cybersecurity activist) and ethical hackers (hacker claim privacy), about the privacy of individuals (healthy person privacy) who have set up their profile in the app due to mandatory reasons, and about data breach (data breach aarogyasetu). tweets with these trigrams have a high percentage of negative reaction (see figure 7). the higher pmi and chi-sq statistic value of the trigrams make them topics of higher relevance (see table 10), and correspondingly, the negative reaction in the tweets holds high significance. significantly, the overall positive reaction is less than 20% even though the government claims the app to be of privacy first design via tweets. tweets, related to privacy protection status of the app, mostly originating from news media or the government, and which contain keywords such as data, sharing, policy and location have most of the emotions labelled as happy (see sample tweets with data and policy in table 12). figure 5. emotion distribution of keywords for aarogyasetu and tracetogether tweets australasian journal of information systems mehta, de & chattopadhyay 2022, vol 26, research article elucidating the role of emotion in privacy-concerns 21 overall, with respect to aarogyasetu, users have expressed negative emotions not only related to possible privacy harms such as surveillance and leak of sensitive information but also towards the lack of accountability and suitable privacy protection measures. overall, the negative emotion in these tweets either reflect user concerns towards potential privacy harms or the lack of protection measures and positive emotions towards protection measures considered sufficient by them. this re-iterates that user emotions can be seen as key informants of negative beliefs within the privacy calculus theory. ultimately, since more negative emotions were observed for aarogyasetu compared to positive ones, users who have expressed their opinions via tweets may have a net negative belief towards the app which could eventually mean lower willingness to rely on the app. figure 6. emotion distribution of bigrams for aarogyasetu and tracetogether tweets figure 7. emotion distribution of trigrams for aarogyasetu and tracetogether tweets australasian journal of information systems mehta, de & chattopadhyay 2022, vol 26, research article elucidating the role of emotion in privacy-concerns 22 4.2 emotion analysis of tracetogether tweets tweets with the keywords surveillance, data, hacker, and breach expressed higher negative emotions compared to the other keywords (see figure 5). tweets with bigrams such as privacy concern, data breach and law enforcement and trigrams such as government technology agency, local law enforcement and criminal procedure code express higher negative emotions compared to other bigrams and trigrams respectively (see figure 6 and figure 7). table 13. sample tweets containing selected keywords and bigrams with high negative reaction (tracetogether) users were concerned about the app’s vulnerabilities even when the application code was made open source, mainly because these concerns were backed by ethical hackers who claimed that government authorities were still logging information without the user’s approval or awareness (see sample tweet on data in table 13). they expressed fear and anger regarding their loss of privacy as claims have been made that data is accessible to the police ultimately reducing the app’s usage and reduction in trust on the government (see sample tweets on breach and data breach in table 13). tweets containing the keywords security, tracing, risk, protection, and location have a high positive reaction (see figure 5). people have appreciated some of the privacy protection measures of the app including no collection and usage of location data, sensible information security features along with clear explanations (see sample tweets in table 14). overall, for tracetogether, users have expressed positive emotions when they perceived a higher level of privacy protection and negative emotions when they perceived higher privacy risks as they came across information indicating that data is being shared without permission or that the protection measures are not sufficient. once again, this re-iterates that user emotions can be seen as key informants of negative beliefs within the privacy calculus theory. to summarize, higher positive emotions have been observed in tracetogether tweets compared to aarogyasetu tweets (figure 4. the bigram contact tracing and trigram contact trace app have received more positive emotion in case of tracetogether and more negative reaction in case of aarogyasetu (see figures 6 and 7). tracetogether has also garnered more keyword / bigram tweet surveillance the app cannot be examined easily using the conventional method. it is hard to verify the claims on privacy and raises concerns over #surveillance. being open source doesn.t always mean the actual code of #tracetogether either. #singapore #covid19 #coronavirus https://t.co/na65xkovit data coder posted analysis for tracetogether on github, saying he found pointers in the code to a government data-collection platform, raising the possibility that! authorities were logging the data without permission. https://t.co/51jrobhbr5 @wojciechklicki @apsalaar @bpaszcza breach #poll a huge debate going on whether the contact tracing app breaches privacy as it was claimed to be accessible by the police for crimes. since then, many have deleted the app. previously it was mandatory for most public places and facilities, including my office. #tracetogether data breach on reading more responses, the general direction is that people will trust the sg govt less on any future issues. was it worth having #tracetogether data by breaching the social contract? which #sg govt body is looking at things at a systems level? australasian journal of information systems mehta, de & chattopadhyay 2022, vol 26, research article elucidating the role of emotion in privacy-concerns 23 positive emotions than aarogyasetu around keywords such as security, tracing, risk, protection and sharing (see figure 5). both the apps have received negative emotions around keywords such as surveillance, steal and hacker. table 14. sample tweets containing selected keywords with high positive reaction (tracetogether) 5 theoretical contribution this study presents a methodological contribution wherein one can examine how usergenerated content can be utilized to understand users’ perceived privacy risks and perceived privacy protections towards a potentially privacy-invasive technology. so long, researchers have adopted survey-based methods to understand user’s negative beliefs related to technology within the privacy calculus theory. compared to this traditional method, the approach described in the paper contributes to gathering spontaneous and candid opinions of users. our approach can be extended to understand privacy perceptions of users for a variety of commercial as well as e-government applications. unlike most of the literature related to user privacy perceptions which have focused on commercial technologies such as ecommerce and commercial mobile applications (gu et al., 2017; li, 2014), our emphasis is on the privacy concerns of top-down government health surveillance apps like contact tracing. privacy calculus theory indicates that privacy-related decisions constitute rational anticipations of risks and benefits connected to data disclosure. we have incorporated privacy-related emotions as key informants of privacy concerns within the privacy calculus theory. our findings highlight users’ positive emotional response towards privacy protective measures and negative response towards potential privacy harms, further supporting studies which show that emotional states also impact privacy-related decision making. the insights from the findings for our second research question (rq2) reveal that there are indeed certain differences among user emotions across different contact tracing apps introduced by india and singapore. we also found under what scenarios such differences exist: tracetogether has garnered more positive emotions than aarogyasetu around keywords such as security, tracing, risk, protection and sharing. this revelation prompts further investigation on why such differences exist. cultural differences across the countries, differences in privacy-related features of the apps themselves and/or differences in each government’s attitude towards privacy could possibly explain the differences observed in user emotions. by answering rq2, we also show that our proposed methodology can be quickly extended to cover multiple countries and cultures. keyword tweet security lovely to see sensible information security and a very clear explanation in the singapore tracetogether app https://t.co/yf5cquyvuu risk and the benefits of many people using #tracetogether outweigh the possible risk it might hold. covid-19 is a big deal. location the tracetogether app can identify people who have been within 2m of coronavirus patients for at least 30 minutes, using wireless bluetooth technology. it does not collect or use location data. https://t.co/i09tiizcr3 australasian journal of information systems mehta, de & chattopadhyay 2022, vol 26, research article elucidating the role of emotion in privacy-concerns 24 6 practical implications our study has important practical implications for data controllers who determine the purposes for which and the means by which personal data is processed. as subsequent waves of the pandemic hit different nations incessantly and the possibility of other pandemics in the future where contact tracing may become necessary, it is important that data controllers pay close attention to users’ emotional responses. this could be the first step in understanding users’ privacy-related perceptions around the app. we show that potential users of contact tracing apps have expressed extensive negative emotions around privacy harms related to these apps. negative emotions can significantly influence users’ privacy protection beliefs and perceived privacy risks (li et al., 2011). according to privacy calculus theory, negative beliefs of privacy concerns may in turn lower chances of adoption of contact tracing apps. it is equally important for data controllers to understand what leads to these emotional responses. our study shows higher positive emotional responses such as happiness when users perceived a higher level of privacy protection and negative emotions such as fear when they perceived loss of control over data and insufficient protection measures. users’ positive appraisals about their experience interacting with an app (motive consistency) can trigger liking emotions (li et al., 2017). high levels of uncertainty, such as how data would be processed and shared in the absence of a comprehensive data protection regulation, are strongly associated with fear (for example, fear of surveillance) (watson & spence, 2007). similarly, accidental, negligent and intentional harm that is attributable to a known person(s), such the feeling that the government is responsible for not doing enough to protect personal data collected by the app, may lead to anger and outrage (watson & spence, 2007). based on these insights, data controllers can take one or more of the following actions to reduce negative emotions and fuel positive ones to improve the uptake of the app. 1) data controllers can focus on improving the overall experience in interacting with the app and the extent of privacy control including the availability of layered privacy policy to trigger positive emotions (li et al., 2017). 2) the introduction and enforcement of strong regulatory measures governing data processing and increasing transparency regarding the app’s privacy features can possibly lead to lower negative emotions from agency as well as uncertainty perspectives. 3) the government could also perform data protection impact assessment (dpia) for the app to minimize users’ privacy risks and publish the dpia report to improve stakeholders’ confidence in the app and improve its own accountability. further, we found differences in emotional reactions to different contact tracing apps. understanding these differences can help data controllers to comprehend the effects of privacy-related cultural characteristics of its user base on their emotional responses and fathom what types of app privacy features and protective measures are appreciated by users and which privacy risks lead to negative reactions. 7 conclusion this research provides insights regarding user emotions towards privacy aspects of contact tracing apps using social media analysis. it shows that users express negative emotions of australasian journal of information systems mehta, de & chattopadhyay 2022, vol 26, research article elucidating the role of emotion in privacy-concerns 25 anger and fear when they perceive privacy risks or when they perceive a lack of privacy protection measures. in many cases, such negative emotions have resulted from negative press. similarly, citizens express positive emotions when they feel satisfied about privacy protection measures. the study also shows that more negative emotions have been expressed towards the indian contact tracing app aarogyasetu compared to the singaporean app tracetogether. this research presents an effective way of analysing ugc in the context of privacy and compares emotions towards two contact tracing apps. however, there remains a lack of efforts in examining ugc to analyse privacy related emotions towards e-government products and platforms, in general, as well as cross-cultural examination of privacy emotions. future studies can therefore focus on understanding users’ privacy related emotions towards e-government initiatives, establish possible connections between such emotions and the adoption of such initiatives by users and compare privacy emotions towards e-government initiatives globally. the study indicated differences in privacy emotions towards the two apps, however, it did not engage in a rigorous quantitative analysis to validate the same. although twitter provides suitable user data, a lot of this data remains unexplored due to the limited possibilities to extract the data through apis. the study only explored tweets in english, excluding all other language data which may have led to overlooking of significant information related to what users feel about privacy of contact tracing apps. the present study has only examined twitter data. however, users may express their opinions on other sm platforms as well as app review forums and google/apple play stores. since the study only explores twitter posts, an element of selection bias could be involved, where people who have a negative opinion are more likely to tweet it than people who are happy. future research could address these issues. acknowledgement we thank the section editor and anonymous reviewers for their valuable suggestions and guidance to improve our article in each stage of revision. references anderson, c.l. & agarwal, r. (2011) the digitization of healthcare: boundary risks, emotion, and consumer willingness to disclose personal health information. information systems research, 22, 469–490. arnold, m. b. (1960). emotion and personality (vol. i & ii). new york. columbia university press. azad, m. a., arshad, j., akmal, s. m. a., riaz, f., abdullah, s., imran, m., & ahmad, f. (2020). a first look at privacy analysis of covid-19 contact tracing mobile applications. ieee internet of things journal. https://doi.org/10.48550/arxiv.2006.13354 balapour, a., nikkhah, h. r., & sabherwal, r. (2020). mobile application security: role of perceived privacy as the predictor of security perceptions. international journal of information management, 52, 102063. barnard, l. (2014). the cost of creepiness: how online behavioral advertising affects consumer purchase intention (doctoral dissertation, the university of north carolina at chapel hill). australasian journal of information systems mehta, de & chattopadhyay 2022, vol 26, research article elucidating the role of emotion in privacy-concerns 26 baumgärtner, l., dmitrienko, a., freisleben, b., gruler, a., höchst, j., kühlberg, j., ... & uhl, c. (2020). mind the gap: security & privacy risks of contact tracing apps. in 2020 ieee 19th international conference on trust, security and privacy in computing and communications (trustcom), 458-467. beldad, a., de jong, m., & steehouder, m. (2011). i trust not therefore it must be risky: determinants of the perceived risks of disclosing personal data for e-government transactions. computers in human behavior, 27(6), 2233-2242. bengio, y., ippolito, d., janda, r., jarvie, m., prud'homme, b., rousseau, j. f., ... & yu, y. w. (2021). inherent privacy limitations of decentralized contact tracing apps. journal of the american medical informatics association, 28(1), 193-195. bollen, j., mao, h., & pepe, a. (2011). modeling public mood and emotion: twitter sentiment and socio-economic phenomena. in proceedings of the international aaai conference on web and social media, 5(1). buchanan, t., paine, c., joinson, a. n., & reips, u. d. (2007). development of measures of online privacy concern and protection for use on the internet. journal of the american society for information science and technology, 58(2), 157-165. chan, e. y., & saqib, n. u. (2021). privacy concerns can explain unwillingness to download and use contact tracing apps when covid-19 concerns are high. computers in human behavior, 119, 106718. cho, h., ippolito, d., & yu, y. w. (2020). contact tracing mobile apps for covid-19: privacy considerations and related trade-offs. arxiv preprint arxiv:2003.11511. church, k., & hanks, p. (1990). word association norms, mutual information, and lexicography. computational linguistics, 16(1), 22-29. clore, g. l., gasper, k., & garvin, e. (2001). affect as information. handbook of affect and social cognition, 121-144. routledge, london, uk. crable, e., & sena, m. (2020). exploring sentiment towards contact tracing. in proceedings of the conference on information systems applied research, vol. 2167, p. 1508. culnan, m. j., & armstrong, p. k. (1999). information privacy concerns, procedural fairness, and impersonal trust: an empirical investigation. organization science, 10(1), 104-115. damasio, a. r. (1994). descartes’ error: emotion, rationality and the human brain. putman, new york, usa. dinev, t., & hart, p. (2006). an extended privacy calculus model for e-commerce transactions. information systems research, 17(1), 61-80. dinev, t., hart, p., & mullen, m. r. (2008). internet privacy concerns and beliefs about government surveillance–an empirical investigation. the journal of strategic information systems, 17(3), 214-233. ekman p. (2007). emotions revealed: recognizing faces and feelings to improve communication and emotional life. holt, new york, usa. ekman, p. (1992). an argument for basic emotions. cognition & emotion, 6(3-4), 169-200. australasian journal of information systems mehta, de & chattopadhyay 2022, vol 26, research article elucidating the role of emotion in privacy-concerns 27 fahey, r. a., & hino, a. (2020). covid-19, digital privacy, and the social limits on datafocused public health responses. international journal of information management, 55, 102181. fan, w., & gordon, m. d. (2014). the power of social media analytics. communications of the acm, 57(6), 74-81. featherman, m. s., & pavlou, p. a. (2003). predicting e-services adoption: a perceived risk facets perspective. international journal of human-computer studies, 59(4), 451-474. featherman, m. s., miyazaki, a. d., & sprott, d. e. (2010). reducing online privacy risk to facilitate e-service adoption: the influence of perceived ease of use and corporate credibility. journal of services marketing, 24(3), 219-229 fiesler, c., & hallinan, b. (2018). "we are the product" public reactions to online data sharing and privacy controversies in the media. in proceedings of the 2018 chi conference on human factors in computing systems, 1-13. fox, g., clohessy, t., van der werff, l., rosati, p., & lynn, t. (2021). exploring the competing influences of privacy concerns and positive beliefs on citizen acceptance of contact tracing mobile applications. computers in human behavior, 121, 106806. frijda, n. h. (1994). varieties of affect: emotions and episodes, moods, and sentiments. in p. ekman & r. j. davidson (eds.), the nature of emotion (pp. 59–67). new york: oxford university press. georgieva, i., beaunoyer, e., & guitton, m. j. (2021). ensuring social acceptability of technological tracking in the covid-19 context. computers in human behavior, 116, 106639. giachanou, a., & crestani, f. (2016). like it or not: a survey of twitter sentiment analysis methods. acm computing surveys (csur), 49(2), 1-41. gonzález, f., figueroa, a., lópez, c., & aragon, c. (2019a, november). information privacy opinions on twitter: a cross-language study. in conference companion publication of the 2019 on computer supported cooperative work and social computing,190-194. gonzález, f., yu, y., figueroa, a., lópez, c., & aragon, c. (2019b, may). global reactions to the cambridge analytica scandal: a cross-language social media study. in companion proceedings of the 2019 world wide web conference, 799-806. greene, j., & haidt, j. (2002). how (and where) does moral judgment work?. trends in cognitive sciences, 6(12), 517-523. gu, j., xu, y. c., xu, h., zhang, c., & ling, h. (2017). privacy concerns for mobile app download: an elaboration likelihood model perspective. decision support systems, 94, 19-28. gutierrez, a., o'leary, s., rana, n. p., dwivedi, y. k., & calle, t. (2019). using privacy calculus theory to explore entrepreneurial directions in mobile location-based advertising: identifying intrusiveness as the critical risk factor. computers in human behavior, 95, 295-306. horvath, l., banducci, s., & james, o. (2020). citizens’ attitudes to contact tracing apps. journal of experimental political science, 1-13. australasian journal of information systems mehta, de & chattopadhyay 2022, vol 26, research article elucidating the role of emotion in privacy-concerns 28 jarvenpaa, s. l., tractinsky, n., & vitale, m. (2000). consumer trust in an internet store. information technology and management, 1(1), 45-71. johnson, r., & zhang, t. (2015). effective use of word order for text categorization with convolutional neural networks. in proceedings of the 2015 conference of the north american chapter of the association for computational linguistics: human language technologies, naacl hlt 2015 (p. 103). kehr, f., kowatsch, t., wentzel, d., & fleisch, e. (2015). blissfully ignorant: the effects of general privacy concerns, general institutional trust, and affect in the privacy calculus. information systems journal, 25(6), 607-635. kehr, f., wentzel, d., & mayer, p. (2013). rethinking the privacy calculus: on the role of dispositional factors and affect. in proceedings of the thirty-fourth international conference on information systems, 1-10. keith, m. j., thompson, s. c., hale, j., lowry, p. b., & greer, c. (2013). information disclosure on mobile devices: re-examining privacy calculus with actual user behavior. international journal of human-computer studies, 71(12), 1163-1173. keltner, d., lerner j. s. (2010). emotion. in the handbook of social psychology, vol. 1, ed. gilbert, d. t., fiske, s. t., lindzey, g., pp. 317–52. wiley, hoboken, nj, usa. keltner, d., oatley, k., & jenkins, j. m. (2014). understanding emotions. wiley, hoboken, nj, usa. kim, d. j., ferrin, d. l., & rao, h. r. (2009). trust and satisfaction, two stepping stones for successful e-commerce relationships: a longitudinal exploration. information systems research, 20(2), 237-257. leith, d. j., & farrell, s. (2020, october). coronavirus contact tracing app privacy: what data is shared by the singapore opentrace app? in international conference on security and privacy in communication systems, 80-96. cham, switzerland: springer. lerner, j. s., li, y., valdesolo, p., & kassam, k. s. (2015). emotion and decision making. annual review of psychology, 66, 799-823. li, h., luo, x. r., zhang, j., & xu, h. (2017). resolving the privacy paradox: toward a cognitive appraisal and emotion approach to online privacy behaviors. information & management, 54(8), 1012-1022. li, h., sarathy, r., & xu, h. (2011). the role of affect and cognition on online consumers' decision to disclose personal information to unfamiliar online vendors. decision support systems, 51(3), 434-445. li, h., sarathy, r., & zhang, j. (2008). the role of emotions in shaping consumers’ privacy beliefs about unfamiliar online vendors. journal of information privacy and security, 4(3), 36-62. li, y. (2012). theories in online information privacy research: a critical review and an integrated framework. decision support systems, 54(1), 471-481. li, y. (2014). the impact of disposition to privacy, website reputation and website familiarity on information privacy concerns. decision support systems, 57, 343-354. australasian journal of information systems mehta, de & chattopadhyay 2022, vol 26, research article elucidating the role of emotion in privacy-concerns 29 lin, j., carter, l., & liu, d. (2021). privacy concerns and digital government: exploring citizen willingness to adopt the covidsafe app. european journal of information systems, 30(4), 1-14. loewenstein, g. f., weber, e. u., hsee, c. k., & welch, n. (2001). risk as feelings. psychological bulletin, 127(2), 267. malhotra, n. k., kim, s. s., & agarwal, j. (2004). internet users' information privacy concerns (iuipc): the construct, the scale, and a causal model. information systems research, 15(4), 336-355. masur, p. k., & scharkow, m. (2016). disclosure management on social network sites: individual privacy perceptions and user-directed privacy strategies. social media+ society, 2(1), 2056305116634368. mohammad, s. (2012). # emotional tweets. in * sem 2012: the first joint conference on lexical and computational semantics–volume 1: proceedings of the main conference and the shared task, and volume 2: proceedings of the sixth international workshop on semantic evaluation (semeval 2012), 246-255). pak, a., & paroubek, p. (2010, may). twitter as a corpus for sentiment analysis and opinion mining. in proceedings of the seventh international conference on language resources and evaluation (lrec 2010), 1320-1326. pang, b., & lee, l. (2009). opinion mining and sentiment analysis. computational linguistics, 35(2), 311-312. peng, h., li, j., he, y., liu, y., bao, m., wang, l., song, y., & yang, q., 2018, april. largescale hierarchical text classification with recursively regularized deep graph-cnn. in proceedings of the 2018 world wide web conference (pp. 1063-1072). praveen, s. v., & ittamalla, r. (2021). analyzing indian citizen's perspective towards government using wearable sensors to tackle covid-19 crisis—a text analytics study. health policy and technology, 10(2), 100521. praveen, s. v., ittamalla, r., & subramanian, d. (2020a). challenges in successful implementation of digital contact tracing to curb covid-19 from global citizen’s perspective: a text analysis study. international journal of pervasive computing and communications, 1-8. praveen, s. v., ittamalla, r., & subramanian, d. (2020b). how optimistic do citizens feel about digital contact tracing? –perspectives from developing countries. international journal of pervasive computing and communications, 1-9. raber, f., & krüger, a. (2018, july). privacy perceiver: using social network posts to derive users' privacy measures. in adjunct publication of the 26th conference on user modeling, adaptation and personalization, 227-232. acm, new york, usa. rathore, a. k., & ilavarasan, p. v. (2020). pre-and post-launch emotions in new product development: insights from twitter analytics of three products. international journal of information management, 50, 111-127. australasian journal of information systems mehta, de & chattopadhyay 2022, vol 26, research article elucidating the role of emotion in privacy-concerns 30 rathore, a. k., maurya, d., & srivastava, a. k. (2021). do policymakers use social media for policy design? a twitter analytics approach. australasian journal of information systems, 25. https://doi.org/10.3127/ajis.v25i0.2965 rekanar, k., o’keeffe, i. r., buckley, s., abbas, m., beecham, s., chochlov, m., ... & buckley, j. (2021). sentiment analysis of user feedback on the hse’s covid-19 contact tracing app. irish journal of medical science, 191(1), 103-112. rowe, f. (2020). contact tracing apps and values dilemmas: a privacy paradox in a neoliberal world. international journal of information management, 55, 102178. shaw, n., & sergueeva, k. (2019). the non-monetary benefits of mobile commerce: extending utaut2 with perceived value. international journal of information management, 45, 4455. simko, l., chang, j. l., jiang, m., calo, r., roesner, f., & kohno, t. (2020). covid-19 contact tracing and privacy: a longitudinal study of public opinion. arxiv preprint arxiv:2012.01553, 1-37. singh, j. p., dwivedi, y. k., rana, n. p., kumar, a., & kapoor, k. k. (2019). event classification and location prediction from tweets during disasters. annals of operations research, 283(1), 737-757. smith, c. a. & kirby, l. (2000) consequences require antecedents: toward a process model of emotion elicitation. in forgas, j. p. (ed.) feeling and thinking: the role of affect in social cognition. cambridge university press. stieglitz, s., dang-xuan, l., bruns, a., & neuberger, c. (2014). social media analytics-an interdisciplinary approach and its implications for information systems. business & information systems engineering, 6(2), 89-96. stieglitz, s., mirbabaie, m., ross, b., & neuberger, c. (2018). social media analytics– challenges in topic discovery, data collection, and data preparation. international journal of information management, 39, 156-168. sun, y., wang, n., shen, x. l., & zhang, j. x. (2015). location information disclosure in location-based social network services: privacy calculus, benefit structure, and gender differences. computers in human behavior, 52, 278-292. tsytsarau, m., & palpanas, t. (2012). survey on mining subjective data on the web. data mining and knowledge discovery, 24(3), 478-514. ur, b., & wang, y. (2013, may). a cross-cultural framework for protecting user privacy in online social media. in proceedings of the 22nd international conference on world wide web, 755-762. acm, new york, usa. ur, b., leon, p. g., cranor, l. f., shay, r., & wang, y. (2012, july). smart, useful, scary, creepy: perceptions of online behavioral advertising. in proceedings of the eighth symposium on usable privacy and security, 1-15. acm, new york, usa. wakefield, r. (2013). the influence of user affect in online information disclosure. the journal of strategic information systems, 22(2), 157-174. australasian journal of information systems mehta, de & chattopadhyay 2022, vol 26, research article elucidating the role of emotion in privacy-concerns 31 walrave, m., waeterloos, c., & ponnet, k. (2020). adoption of a contact tracing app for containing covid-19: a health belief model approach. jmir public health and surveillance, 6(3), e20572, 1-10. watson, l., & spence, m. t. (2007). causes and consequences of emotions on consumer behaviour: a review and integrative cognitive appraisal theory. european journal of marketing, 41(5/6), 487-511. wen, h., zhao, q., lin, z., xuan, d., & shroff, n. (2020, october). a study of the privacy of covid-19 contact tracing apps. in international conference on security and privacy in communication systems, 297-317. springer, cham, switzerland. wigan, m. (2020). rethinking it professional ethics: classical and current contexts. australasian journal of information systems, 24. https://doi.org/10.3127/ajis.v24i0.2851 wildenauer, m. (2020). the shared responsibility model: levers of influence and loci of control to aid regulation of ethical behaviour in technology platform companies. australasian journal of information systems, 24. https://doi.org/10.3127/ajis.v24i0.2797 xu, h., luo, x. r., carroll, j. m., & rosson, m. b. (2011). the personalization privacy paradox: an exploratory study of decision making process for location-aware marketing. decision support systems, 51(1), 42-52. xu, h., teo, h. h., & tan, b. c. (2005). predicting the adoption of location-based services: the role of trust and perceived privacy risk. in 26th international conference on information systems, icis 2005 (pp. 897-910). zeng, d., chen, h., lusch, r., & li, s. h. (2010). social media analytics and intelligence. ieee intelligent systems, 25(6), 13-16. zhang, b., & xu, h. (2016, february). privacy nudges for mobile applications: effects on the creepiness emotion and privacy attitudes. in proceedings of the 19th acm conference on computer-supported cooperative work & social computing (pp. 1676-1690). copyright: © 2022 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v26i0.3687 http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 theoretical background and review of literature 2.1 emotions, decision-making and privacy 2.2 overview on privacy calculus theory 2.3 overview on perceived privacy risk 2.4 overview on perceived privacy protection 2.5 social media analytics in opinion mining and sentiment analysis 2.6 gaps in extant literature and the contributions of this research 3 method and data 3.1 creation of target datasets 3.2 creation of auxiliary dataset 3.3 proposed method 3.4 experimental performance 4 results and discussion 4.1 emotion analysis of aarogyasetu tweets 4.2 emotion analysis of tracetogether tweets 5 theoretical contribution 6 practical implications 7 conclusion paper title (paper title style) australasian journal of information systems weerasinghe et al. 2023, vol 27, research article alignment of big data perceptions 1 alignment of big data perceptions across levels in healthcare: the case of new zealand kasuni weerasinghe massey university, new zealand w.m.k.g.weerasinghe@massey.ac.nz david pauleen national chung cheng university, taiwan nazim taskin bogazici university, turkey shane scahill university of auckland, new zealand abstract big data and related technologies have the potential to transform healthcare sectors by facilitating improvements to healthcare planning and delivery. big data research highlights the importance of aligning big data implementations with business needs to achieve success. in one of the first studies to examine the influence of big data on business-it alignment in the healthcare sector, this paper addresses the question: how do stakeholders’ perceptions of big data influence alignment between big data technologies and healthcare sector needs across macro, meso, and micro levels in the new zealand (nz) healthcare sector? a qualitative inquiry was conducted using semi-structured interviews to understand perceptions of big data across the nz healthcare sector. an application of a novel theory, theory of sociotechnical representations (tsr), is used to examine people’s perceptions of big data technologies and their applicability in their day-to-day work. these representations are analysed at each level and then across levels to evaluate the degree of alignment. a social dimension lens to alignment was used to explore mutual understanding of big data across the sector. the findings show alignment across the sector through the shared understanding of the importance of data quality, the increasing challenges of privacy and security, and the importance of utilising modern and new data in measuring health outcomes. areas of misalignment include the differing definitions of big data, as well as perceptions around data ownership, data sharing, use of patient-generated data and interoperability. both practical and theoretical contributions of the study are discussed. keywords: big data, new zealand healthcare, business-it alignment, theory of sociotechnical representations, business-it alignment taxonomy. 1 introduction in the past decade, with the advent of ever more sophisticated information technologies, and addressing increasing demands due to significant health issues the healthcare sector has undergone major changes targeting improved patient care (paré et al., 2008; roski et al., 2014). a wide range of clinical and operational information systems are used by healthcare systems around the world (menon et al., 2009). this growing use of information systems in the healthcare sector, on top of increasing patient populations, complex diseases (including public health emergencies such as covid-19), sophisticated medications and diagnostic testing, australasian journal of information systems weerasinghe et al. 2023, vol 27, research article alignment of big data perceptions 2 generates complex and unstructured data that have the characteristics of ‘big data’ (burns, 2014; ward et al., 2014; wyber et al., 2015). in general, big data refers to enormous amounts of unstructured and complex data produced by a wide range of sources, such as computer applications, mobile/wearable devices, and sensors (emani et al., 2015). until recent times data-driven approaches in healthcare were considered difficult, if not impossible, because technology itself was not mature enough to handle such complex data (wyber et al., 2015). however, recent technological developments around big data analytics have opened promising avenues for healthcare to make use of big-healthcare-data for improved healthcare delivery (herland et al., 2014; lv & qiao, 2020). some notable examples of application of big data technologies to healthcare include: precision medicine, discovering the most effective treatments, identifying patterns related to medication side effects and hospital readmissions, and advances in pharmaceutical research (groves et al., 2013; nash, 2014; tormay, 2015, weerasinghe et al., 2022b; lv & qiao, 2020; williams et al., 2018). although the healthcare sector has not been an early adopter of big data analytics (groves et al., 2013; ward et al., 2014), currently, developed countries demonstrate a great interest in the potential of big data to improve healthcare planning and service delivery (williams et al., 2018; he et al., 2017). similar to other developing countries, new zealand (nz) healthcare system is exploring the potential applications of emerging technologies, such as big data. the nz health strategy released in 2016 comprises two detailed documents: (i) the nz health strategy: future direction, and (ii) the nz health strategy: roadmap of actions 2016. nz health strategy: future direction identifies high-level strategies for the nz health system from 2016 to 2026. from an information and societal perspective, the strategy acknowledges that “[a]ll new zealanders live well, stay well, get well, in a system that is people-powered, provides services closer to home, is designed for value and high performance, and works as one team in a smart system” (minister of health, 2016). five key themes are identified which provide the needed direction for the desired future. these are: (i) people powered, (ii) closer to home, (iii) value and high performance, (iv) one team, and (v) smart system. the identification of “smart system” is a key starting point for technological advancements in the nz healthcare sector, including big data technologies. as such it is important to understand stakeholders’ perspectives of these technologies. research in the field of big data shows that the success of big data technologies depends on their alignment with business/sector needs (bean & kiron, 2013; watson, 2014; weerasinghe et al., 2018a). addressing this importance of alignment between big data technologies and healthcare sector needs, this paper presents a study exploring alignment of big data technologies across multiple levels (identified as macro-meso-micro) within the new zealand (nz) healthcare sector. the nz healthcare sector is led by the ministry of health (moh) (ministry of health, 2014) and the minister of health develops policy with input from the moh, cabinet and the government (ministry of health, 2017). district health boards1 (dhbs) and primary health organisations (phos) and their member general practices are the main organisations that are responsible for healthcare delivery. healthcare services are provided by these organisations to the nz population and are directed by the moh. 1 this research was conducted prior to nz health sector reforms in july 2022 which disestablished dhbs. australasian journal of information systems weerasinghe et al. 2023, vol 27, research article alignment of big data perceptions 3 due to the association of many different organisations, actors, and structural divisions in the nz healthcare system, it is understood to be a complex system. when studying complex systems, it is helpful to take an approach that incorporates the macro-meso-micro (mmm) perspective of the system to arrive at a holistic understanding (dopfer et al., 2004). within the nz healthcare sector macro has been identified as policy makers, meso as planners and funders, and micro as frontline care providers (cumming, 2011; scahill, 2012; weerasinghe et al., 2018a). studying the implications and applications of big data in a complex system can be done by exploring two types of elements: social and technical. as a technological phenomenon big data research often focuses on technical elements, such as analytic capabilities, security measures, infrastructure requirements and so on (e.g., chen et al., 2014; davenport, 2013; dhawan et al., 2014). the social elements around big data, such as understanding, commitment, value and perceived challenges are often given less attention in big data research (shin & choi, 2015). social elements involve the subjective understanding of a technological phenomenon that often affect its use (dulipovici & robey, 2013). thus, understanding how social elements impact big data implementation and use are crucial for its success (shin, 2015, weerasinghe et al., 2018a). political, organisational and managerial decisions concerning big data technologies are all greatly influenced by social elements (shin, 2015). the implementation of big data technologies is challenging and involves all healthcare sector levels, thus requiring the support of multiple stakeholders (weerasinghe et al., 2018a). for example, the implementation of big data technologies must accord with how different stakeholders implement health strategy. security measures, necessary funding, available skills and technology, and willingness to use are all considerations, alongside responsibilities, which reside at different stakeholder levels. it is important to note that perceptions about big data by stakeholders at different levels (mmm) may be different due to the range of roles they play, their experience and many other factors (moscovici, 1984). this notion of studying different perceptions and social elements aligns well with studying social dimension in a business-it alignment study (chan & reich, 2007). as such in this paper we explored the social dimension of alignment which refers to how stakeholders at different levels perceive the technological phenomenon and the alignment of such perceptions. based on this understanding the research question addressed in this paper is how do stakeholders’ perceptions of big data influence alignment between big data technologies and healthcare sector needs across macro, meso, and micro levels in the nz healthcare sector? business-it alignment refers to the fit between business (strategy or approach) and information technology (chan & reich, 2007; henderson & venkatraman, 1993; luftman, 1996; grant, 2010; jenkin & chan, 2010; el-mekawy et al., 2015). in this context it is defined as the fit between perceptions of big data and healthcare sector needs. studying big data as an emerging technology in the early stages of its introduction, implementation and adoption in the healthcare sector through a business-it alignment lens can provide a more comprehensive understanding, as well as identify potential issues (bean & kiron, 2013; watson, 2014). although the general understanding of business-it alignment concerns maintaining consistency between technology and formal documentation (such as policy, design documents, and the like), the successful implementation of technology also depends on stakeholders’ perceptions, understanding and commitment. these are the focus of the social dimension of alignment (dulipovici & robey, 2013; gal & berente, 2008). the social dimension australasian journal of information systems weerasinghe et al. 2023, vol 27, research article alignment of big data perceptions 4 of alignment fits well with the intent of the theory of sociotechnical representations (tsr), which is a novel theory developed to examine people’s sociotechnical representations of big data technologies and their applicability in their day-to-day work. tsr is a new theory (weerasinghe et al., 2022a) and this paper presents an early application of it. the next section provides a review of literature on big data in the context of health, identifying definitions and explaining potential opportunities and issues. the discussion in the theoretical foundations section is twofold: it explains tsr and its use and discusses a taxonomy derived from the alignment literature that guides an alignment study and identifies the scope and boundaries for the present study. the methodology section outlines the qualitative methods used in this study. the findings section reports on both alignments and misalignments around perceptions of big data in relation to each mmm level. the final section identifies important implications, limitations and directions for future work. 2 big data in health in general, big data refers to enormous amounts of unstructured and complex data produced by a wide range of sources such as computer applications, mobile devices, and sensors (emani et al., 2015; groves et al., 2013). there is no universally agreed upon definition for big data (herland et al., 2014) and phrases such as “massive amounts of data”, “enormous growth of data” and “large datasets” are typically seen across the literature as defining big data (chen et al., 2014; eynon, 2013). big data can be defined and distinguished from standard data based on three characteristics, known as the 3vs: volume (enormous amounts of data), variety (many different types and sources of data) and velocity (data that is generated and used at a high speed) (mcafee & brynjolfsson, 2012; russom, 2011). two additional vs – veracity (accuracy of data) and value (data that is able to create value) – are also commonly cited, extending the characteristics of big data to 5vs (emani et al., 2015; saporito, 2013; sathi, 2012). figure 1. types of big data in health big data in health refers to large and complex data across healthcare that may potentially be used to improve healthcare management and service delivery. typically, big data in health are data generated by health information systems such as patient management systems (pms), laboratory systems, radiology and imaging systems and the like, within the health system australasian journal of information systems weerasinghe et al. 2023, vol 27, research article alignment of big data perceptions 5 itself. genomics data (data obtained through genomic sequencing) and patient-generated data (data created by patients outside of the healthcare system) are also identified as other types of big data in health (roski et al., 2014). with this understanding, figure 1 illustrates the types of data that can be considered big data in the healthcare context. genomic data is used to understand a person’s genome through sequencing techniques2 . a genome (dna) has nearly three billion base pairs, which results in approximately 100 gigabytes worth of data for a single person (o'driscoll et al., 2013). due to its complexity and enormous volume, genomics data is categorised as big data (he et al., 2017). genomics in healthcare may enable precision medicine. traditionally medications are prescribed to patients on a trial and error basis because they do not cater to the individual or their genomic makeup. some medications work well for some patients but not for others. with genomics, precision medicine can more clearly define a disease and facilitate precise targeting of disease subgroups to allow better treatments (ashley, 2016). precision medicine uses big data technologies to facilitate personalised treatments by identifying a person’s genomic makeup (jameson & longo, 2015; weli, 2018). patient-generated data is data produced by patients outside of clinical settings in normal daily life (petersen & demuro, 2015; jim et al., 2020). such data is generated by mobile apps, wearable devices or other medical devices used by patients to monitor their health; for example, blood pressure monitors (shapiro et al., 2012). the use of such patient-generated data provides benefits including customised care plans, assessing patients’ functional status, understanding outcomes after surgery to predict the length of hospital stay, and so forth (petersen & demuro, 2015; williams et al., 2018). data from social media is also a form of patient-generated data that healthcare sectors can utilise for improved healthcare management and delivery. use of artificial intelligence technologies (such as machine learning/ml, natural language processing/nlp) have gained increased attention to improve efficiencies in healthcare practises (shailaja et al., 2018). healthcare research increasingly discusses the benefits of capturing patient experience through social media as opposed to getting patient feedback on the services through traditional methods (zadeh et al., 2019). although data with characteristics of big data have been generated by the healthcare sector for some time, historically data-driven approaches in healthcare were often considered complex, if not impossible, because technology was not mature enough use or analyse such data (wyber et al., 2015). for example, only around 15% of data from health records (which are in a structured form) is used at present, mostly through traditional analytics (roski et al., 2014). recent developments in big data analytics have opened up promising avenues for the healthcare sector to make better use of big-healthcare-data for improved healthcare delivery (tormay, 2015; wyber et al., 2015; lv & qiao, 2020). examples include: hadoop clusters, which can be used to economically store massive amounts of data; data science experts, who are capable of making sense of large and complex data generated nearly in real time; and advanced analytical capabilities, which allow for health data which is formatted in different ways (structured, semi-structured and unstructured) and found in different systems (e.g., electronic medical records, patient management systems, clinical systems, etc.) to be linked and analysed together (i.e., connected care). 2 sequencing techniques refer to technology (technologies) which allows inspection of dna. australasian journal of information systems weerasinghe et al. 2023, vol 27, research article alignment of big data perceptions 6 compared to other industries like retail merchandising and banking, the uptake of big data technologies in the healthcare sector has been limited and slow (groves et al., 2013; bakker et al., 2020). however, the healthcare sectors around the world is increasingly catching up (weerasinghe, 2019). the complex nature of the healthcare system, resistance to change by healthcare practitioners, uncertainty of returns, and privacy concerns are identified as possible reasons for this lag (groves et al., 2013). nonetheless, due to increasing it expenditures and the enormous amounts of under-utilised, complex data, the healthcare sector needs more efficient practices, research and tools to analyse and maximise the utility of big data (chawla & davis, 2013; guo & chen, 2023). health-it researchers highlight the importance of alignment and a comprehensive approach to the aligning, planning, executing and governing technology components in the healthcare sector (krey, 2018; weerasinghe et al., 2018a; fattah & arman, 2014). recently, developed countries have recognised the importance of big data analytics for healthcare (jim et al., 2020). as big data analytics allows for discovering associations and recognising patterns and trends, it has the potential to improve care, save lives and lower costs (raghupathi & raghupathi, 2014; lv & qiao, 2020). harnessing big data for enhanced applied knowledge could have significant implications for healthcare. some of these benefits include: improved clinical decision support, detecting gaps in care delivery, discovering the most effective treatments, identifying patterns related to medication side effects and hospital readmissions, delivery of personalised medicine through genomics, improving pharmaceutical research, reliance on patient-generated data for diagnostics and lastly, fraud detection (jim et al., 2020, williams et al., 2018, roski et al., 2014). although such benefits can be achieved, the very nature of big data catering to multiple different areas creates huge challenges on its own. some of these challenges include maintaining data quality, patient privacy, obtaining skills, changes to it infrastructure, and re-visiting policy (halamka, 2014; roski et al., 2014). technical elements (analytics, privacy and security measures, it infrastructure) involving big data implementations have been extensively researched (chen et al., 2014; davenport, 2013; bag et al., 2023). as a technological revolution itself, big data research naturally leans towards these technical elements. as a result, adequate research has not been carried out to investigate the social elements of big data (shin, 2015; shin & choi, 2015). social elements refer to users’ understanding, commitment, perceived value and challenges of big data in a given context. social elements reflect the actual and potential use of a technological phenomenon, and so investigating them is important. limited research on the effects of the social elements on big data technologies has shown that people’s understanding of big data technologies influences implementation and adoption (shin, 2015; weerasinghe et al., 2022). but empirical work identifying and discussing such social elements is limited. nonetheless, while uptake of big data is increasing in healthcare, social elements (e.g., a doctor’s understanding of an ai risk assessment tool) can play a significant role in its use in healthcare. thus, to address this, the research investigates social elements around big data in the nz healthcare context. 3 theoretical foundations the theoretical foundations of this paper are twofold. first, to investigate business-it alignment, the taxonomy of alignment conceptualisations (weerasinghe et al., 2018b) is used. the taxonomy helps identify and define the scope of the study. second, the theory of sociotechnical representations (tsr) developed by weerasinghe et al. (2022a) is used as a australasian journal of information systems weerasinghe et al. 2023, vol 27, research article alignment of big data perceptions 7 foundational theory to understand sociotechnical representations of big data in health to investigate its influence on alignment. 3.1 taxonomy of alignment for over 30 years business-it alignment has been a key concern in academia and industry, making it an important field of is research (chan & reich, 2007; jia et al., 2018). ‘alignment’ refers to the degree of fit between the business and information technology and involves strategy, structure and/or people (chan & reich, 2007; el-mekawy et al., 2015, henderson and venkatraman, 1992, luftman, 1996). the business-it alignment literature is vast and has many different conceptualisations as a result of being studied for years across many domains (chan & reich, 2007). weerasinghe et al. (2018b) created a taxonomy (table 1) that identifies existing conceptualisations of alignment and explains how they can be used to define the scope of an alignment study. classes properties of each class types bivariate fit cross-domain alignment strategic fit levels organisational operational system project individual sector dimensions strategic/intellectual structural (formal/informal) social cultural states end (result) process environments internal external table 1. taxonomy of alignment conceptualisations this taxonomy identifies five classes of alignment: types, levels, dimensions, states and environments. the properties of each type are identified, based on literature. the taxonomy encourages the study of at least one property within each class for an alignment study to be complete . the selected properties for this alignment study are shaded in the taxonomy (see table 2) and explained below. classes properties of each class types bivariate fit cross-domain alignment strategic fit levels organisational operational system project individual sector dimensions strategic/intellectual structural (formal/informal) social cultural states end (result) process environments internal external table 2. selected conceptualisations of alignment for the study strategic fit is selected as the type of alignment for this study (from the first row in the taxonomy). strategic fit investigates alignment across domains of business strategy, business structure, it strategy and it structure (henderson & venkatraman, 1992). as big data implementations demand change in business and it strategy and structure, it is necessary that all these domains are explored through strategic fit. as the research question relates directly to understanding the influence of big data perceptions across the healthcare sector, the level of alignment to be investigated is the sector level (second row of the taxonomy). by australasian journal of information systems weerasinghe et al. 2023, vol 27, research article alignment of big data perceptions 8 investigating sector level alignment the study will provide insights into understanding agreements and gaps in alignment across the whole healthcare sector under investigation. highlighting the lack of knowledge and the need to investigate the social elements around big data, the social dimension of alignment was selected as the lens through which to conduct this alignment study. it is important to note that the strategic/intellectual dimension here refers to exploring fit of documented strategies to the organisations needs and other elements such as people, and therefore is not the focus of this research. big data implementations involve the utilisation of a set of technological, social and organisational interactions. this could mean having to deal with groups of stakeholders with different interests, interpretations and perceptions (gal & berente, 2008), such as mmm levels. therefore, the social dimension of alignment is used to explore how big data is perceived by different players across the sector. this further fits with the identified gap in the literature – a lack of research around the social elements of big data (weerasinghe et al., 2022b). dulipovici and robey (2013) identify two aspects of technology use as intended use and situated use. intended use is the planned purpose of technology/is while situated use is the subjective understanding of technology/is. investigation through a social dimension lens will facilitate the study of social elements around big data, allowing an understanding of its situated use. because the implementation of big data is still in progress in the new zealand healthcare sector, it will continue to evolve; thus this study considered alignment as a process, as opposed to an end state. this definition of alignment as a process allows us to capture different stages of understanding of big data as well as big data initiatives across the sector. although many different organisations and healthcare bodies were examined (macro-meso-micro), because all the organisations are within the nz healthcare system itself, it is regarded as a study of the internal environment within the healthcare sector. 3.2 theory of sociotechnical representations the theory of sociotechnical representations (tsr) is a novel theory explained in weerasinghe et al. (2022a) which integrates key elements of two well-known theories: sociotechnical systems theory (sst) (emery, 1959) and social representations theory (srt) (moscovici, 1984). tsr combines is views of technology (through sst) with social psychology perspectives (through srt) to explore and understand individual perspectives of technology and the effects of these perspectives on technology implementation and use. tsr explains that given any technological phenomenon is co-created with the people around it (policy makers, planners, implementers and users): not just in what people do with it (e.g., tasks, responsibilities) but also in terms of how people perceive it (e.g., usefulness, commitment to use, value). as explained in sst by emery and trist (1965) the social subsystem (people) and the technical subsystem (technology) interact with each other and are interdependent with each other (as modelled in figure 2). therefore, the roles and responsibilities of people are affected by what technology is intended to do and can do. however, the theory of sociotechnical representations uses srt (moscovici, 1984) with sst, to explain that this interdependence of technology and people goes beyond concrete factors like roles, responsibilities, or pre-determined ways of usage, and posits that perceptions of technology (identified as sociotechnical representations) are critical to the success of technology. as shown in figure 2, tsr uses srt as a theoretical tool to deeply examine the technical subsystem. it does not ignore the social subsystem. instead, it looks into the interdependencies between the social and technical subsystems, highlighting that the social subsystem influences australasian journal of information systems weerasinghe et al. 2023, vol 27, research article alignment of big data perceptions 9 the representations of the technical subsystem. further, researchers such as orlikowski and scott (2008), leonardi (2011) and cecez-kecmanovic et al. (2014) have also acknowledged the importance of taking a social lens to technology adoption in their work on socio-materiality. because social representations theory (srt) is a tool to explore user perceptions of technology (the technical subsystem) using the potential social capabilities (of the social subsystem), the theory of sociotechnical representations (tsr) uses srt to inform sociotechnical systems theory (sst). the srt literature highlights two processes: anchoring and objectification. anchoring refers to identifying a phenomenon within a group based on the aspirations, perceptions, collective sense making by the group, while objectification supports the anchoring process through an individual’s mapping of the phenomenon. these two processes (anchoring and objectification) processes impacts/complements each other. for example, consider a healthcare organisation that has had cyberattacks in the past which compromised patient information. the employees may anchor their understanding of data security to strict access controls, encryption, and firewalls. they perceive data security as the need to protect sensitive information from unauthorized access, which becomes the common understanding in the organisation (anchoring). however, if there is an employee who is well versed about data privacy regulations and has studied cybersecurity, they may objectify data security as the need for data governance, proactive monitoring and employee awareness and training. this objectification is informed by their past experience/knowledge with cybersecurity. this then reshapes the groups anchored perception of data security. this is acknowledged in figure 2 by showing a recursive relationship between the two. also, the shape of representation is different to that of the actual technical subsystem highlighting that the representation may not always be the same as the technical subsystem as it is shaped by the social elements. figure 2. theory of sociotechnical representations (based on weerasinghe et al. (2022a)) australasian journal of information systems weerasinghe et al. 2023, vol 27, research article alignment of big data perceptions 10 srt acknowledges that representations are anchored and objectified within a group, which influences tsr to promote exploring stakeholder groups (moscovici, 1988). further, tsr highlights that the social subsystem influences anchoring and objectification of the technical subsystems’ representation. therefore, srt supports a deeper and richer understanding of sociotechnical systems. it does this by facilitating the examination of sociotechnical representations created by the social subsystem about the technical subsystem through anchoring and objectification. by understanding anchoring and objectification, tsr acknowledges that the representations of the technical subsystem are continuously evolving or being shaped because anchoring and objectification complementary each other. box 1 illustrates an application of tsr to explain anchoring and objectification. a health insurance company adopts a new system for project management. the system is the technical subsystem and the people (employees, management) are the social subsystem. the management team in an attempt to shape the representation of the new system begins by trying to anchor an understanding of this system by emphasizing its efficiency in automating tasks, streamlining workflows, and increasing productivity. they promote the idea that this system will revolutionize how projects are managed and lead to improved outcomes. this anchoring process happens within the organisation and is influenced by the social subsystem in accord with their responsibilities. however, individual employees may objectify the new system differently based on their own experiences and perspectives. for instance, some employees might objectify the technology as a source of surveillance or control, perceiving it as a means for management to monitor their performance. others may objectify it as a potential problem as they have experienced system failures in the past. this is the objectification process. as such, the representation the management wants to create may be shaped by social elements. as the implementation and adoption of the new project management system takes place, anchoring and objectification will continue to shape and re-shape the representation of the technological system. box 1. an example application of tsr using tsr, the present study focuses on the social elements of technology (i.e., how technology is perceived in terms of value and challenges, and the commitment of people involved). it contributes to understanding how the technical subsystem is perceived and appreciated (or not) by the stakeholders (e.g., mmm levels) themselves who are the social subsystem. this understanding helps researchers to identify issues and derive understandings about social interpretations and reasons why the technical subsystem influences sociotechnical interdependencies. the sst perspective explains that the social and technical subsystems are interdependent in the sense that peoples’ skills, values, and other humanistic characteristics are interrelated with the technological aspects, such as the systems, it infrastructure and tools table 3 explains the distinctions of sst, srt and tsr. tsr also argues that the social representation of the technical subsystem by the stakeholders of this subsystem plays an important part in the success and acceptance of technical systems. because a social representation of the technical subsystem is formed, tsr uses the term “sociotechnical representation” as opposed to the term “social representation” used in srt. a sociotechnical representation alludes to a technological phenomenon that is created through social interpretations of technology by an individual’s objectification and the groups’ process of anchoring. as such tsr explains that when people interact with technology, it may be that people are interacting with the representation of the technology. another important aspect of tsr is that it can explain why/how perceptions of technology can change over time. for example, as new actors/people get involved, their objectification can change group perceptions influencing the sociotechnical representation. use of tsr also has an important impact on the australasian journal of information systems weerasinghe et al. 2023, vol 27, research article alignment of big data perceptions 11 methodology and selection of participants compared to a typical sst-based study. because it focuses on sociotechnical representations, the use of tsr promotes the value of understanding the perceptions of people over physical documents or systems. sociotechnical systems theory social representations theory theory of sociotechnical representations people and technology are interdependent people create a representation of a phenomenon (e.g., technology) people and their representations of technology are interdependent social subsystem includes occupational roles, tasks and activities of people; technical subsystem includes technologies, technical capabilities and processes representation of a phenomenon (concept, object or situation) is created within the social group for the purpose of understanding and communicating and is influenced by thoughts, feelings and behaviours of the actors. the creation of the representation happens through anchoring and objectification. social subsystem (people) creates a representation of the technical subsystem which may or may not be similar to the actual technical phenomenon. this is impacted by anchoring and objectification. when using sst the focus of research questions/ objectives will be the use/adoption of technology when using srt the focus of research questions/ objectives will be the perceptions (representations) of technology and the factors that influence these perceptions. when using tsr the focus of a rq/research objectives will be the perceptions of technology and how it impacts use/adoption rq example: how are big data technologies used in the healthcare sector? rq example: how do people perceive big data in the healthcare sector? and what influence those perceptions? rq example: how do perceptions of big data influence the use of big data technologies in the healthcare sector? table 3. distinctions of sst, srt and tsr the use of a social dimension lens (based on the business-it alignment taxonomy) promotes the investigation of the “level of mutual understanding” (reich & benbasat, 1996) in a business-it alignment context. similarities and differences in sociotechnical representations around big data technologies and their use can inform big data’s influence on business-it alignment. the use of tsr is appropriate for a study that investigates the social dimension of alignment. in this study, alignment is defined as ‘the fit between perceptions of big data and healthcare sector needs at each subsector level (macro, meso, and micro)’. it is important that perceptions (social dimension of alignment) at each mmm level is investigated to understand similarities and difference between these levels within the sector (i.e. sector level alignment). 4 methodology the aim of this paper is to explore how big data analytics is perceived (represented) in the healthcare sector and how this representation influences business-it alignment. an exploratory qualitative approach is used (liamputtong & ezzy, 2005). the very nature of the healthcare sector and implementing and aligning big data technologies across this multi-level system potentially has unidentified complexities. considering social elements around big data adds a further level of complexity because social elements may vary from person to person as well as in each mmm level. accordingly, the research is not hypothesis testing in nature, but rather is guided by the research question: “how do stakeholders’ perceptions of big data influence alignment between big data technologies and healthcare sector needs across macro, meso, and micro levels in the nz healthcare sector?” australasian journal of information systems weerasinghe et al. 2023, vol 27, research article alignment of big data perceptions 12 the use of tsr promotes collecting data from individuals and interpreting it at a group level. previous literature has identified nz health sector to have three levels (macro, meso, and micro/mmm) and has emphasised the importance of exploring technology phenomena through these levels (cumming, 2011; scahill, 2012; weerasinghe et al., 2018a). these mmm levels were identified as the smallest unit of analysis. data was collected from individuals and was analysed and interpreted at each of the three subsector levels (mmm) to understand perceptions of big data at each subsector level. a cross-group analysis was then undertaken to understand influence on alignment across the nz healthcare sector. within the nz healthcare sector, the identified mmm levels have different tasks and responsibilities associated with big data initiatives, and are likely to construct different sociotechnical representations of big data. analysing the groups separately minimises the abstract level of analysis, allowing for examination of operational details within the sector (yin, 2014). the research design and procedure are shown in figure 3. the research question, literature and context along with the theoretical foundations influenced the selected research methodology. the data collection protocol was decided and three interview schemas for each of the mmm levels were piloted. data was collected from mmm groups and data from each individual level was separately analysed and individual summary tables were created (smith et al., 2009). using the three summary tables, a cross-level analysis was done to understand the situation across the healthcare sector. the cross-level summary table was used as a guide in reporting the findings and discussion identifying alignment and misalignment of big data in the nz healthcare context. figure 3. research procedure semi-structured interviews were conducted to gather rich data from participants at each mmm level (merriam, 2009) (interview schemas can be found in appendix 1). purposive sampling techniques were used as the research required gathering data from informants who were involved in constructing policies (macro), planning and implementing (meso), or using (micro) (current or future) big data technologies (miles et al., 2014; patton, 2015). a snowball sampling strategy was also used, asking informants to direct the researcher to other australasian journal of information systems weerasinghe et al. 2023, vol 27, research article alignment of big data perceptions 13 participants (miles et al., 2014). overall, 32 interviews were conducted, with six at macro level3, seventeen at meso and nine at the micro level. four participants at the meso level, who had clinical duties, also answered questions relating to the micro level. participant demographics are provided in appendix 2. as similar themes continue to emerge through data, theoretical saturation was reached, which allowed the researchers to determine the sample size (mason, 2010). interviews were audio recorded and transcribed verbatim. general inductive thematic analysis (thomas, 2006) was used to analyse data, supported by nvivo software. an inductive approach was adopted and its intent was to see what ideas emerged from the data rather than the literature around big data. the first step in the inductive analysis was to clean the raw data files and to bring all transcripts into a similar format using microsoft word. then reading and re-reading along with writing memos was done by the first author. when the first author was fully familiar with the data, transcripts were coded and themes were identified (example of coding themes and descriptions can be found in appendix 3). all themes at each mmm level were analysed separately to identify categories. these categories were re-analysed to remove less consequential categories and to merge similar ones. three summary tables were created for each of the mmm levels for analysis, which guided the report writing for each level. these findings were further validated through member checking (lincoln & guba, 1985). a summary of findings was sent to the participants. out of the twenty-six participants who could be contacted, four participants responded with feedback. the feedback confirmed that they agreed with the findings. in the next section, the findings are presented at each mmm level, and the cross analysis then identifies the alignment/misalignment issues which are presented in the discussion section. 5 findings: summary findings across mmm levels through a tsr lens this research investigates the social subsystem (people in mmm levels) and the technical subsystem (big data) and their interdependencies to business-it alignment in the context of nz healthcare. in this research tsr lens specifically allows us to understand big data (technical subsystem) from the perspective of people in mmm levels (social subsystem), and also allows us to understand the reasons behind such perceptions by evaluating anchoring and objectification processes (tools provided by srt). five key categories emerged from data analysis that influence the sociotechnical representations of big data. these are: (i) absence of a clear definition, (ii) understanding valued characteristics of big healthcare data, (iii) issues and challenges of big data, (iv) applications (current and future potential areas), and (v) influence of healthcare strategy and policy. this section describes findings of the three levels (mmm) separately, analysing how big data is socially represented in the nz healthcare context. 3 the data was collected between february 2016 and june 2018. during this period the government of nz changed, and the ministry of health went through a restructure and nz health strategy was revised. consequently, some of the participants who were interviewed in 2016 talk about things that are no longer in use. some of the business units that were thought to have a major role in the health system with regard to health-it were disestablished. australasian journal of information systems weerasinghe et al. 2023, vol 27, research article alignment of big data perceptions 14 5.1 macro level: policy makers big data in the macro level seem to have a sociotechnical representation of being a part of “evolving technology” (mac6) and is not deemed new. while a clear representation was not yet present (some participants being unclear about the term, some claiming it is only a buzz word, and some very positive about it), there was clear agreement that big data is about developments in technology around data creation, sharing, storage, and management. while they saw the role of big data as “continuing to grow” (mac2), one specific area they were interested in was precision medicine. they saw that precision medicine “at some point in time will provide useful tools for clinical decision making” (mac5). the macro level participants talked about their experience (relates to objectification in tsr) with problematic data quality across the healthcare sector with incomplete and inconsistent data followed by issues of accuracy that largely happen because of patient interactions with the health system. because of such experience, these macro level participants see it as an issue that has greater priority. with big data, policy makers see their role as promoting its use through sustainable policy and strategy – so that data can be used across many different fields including clinical care and decision making. the participants talked about the nz health strategy4 and how they saw the problems around big data being addressed in the strategy. they acknowledged that the health strategy provides opportunities for effective use of big health data through: (i) connected information, (ii) a well-defined national health index (nhi), and (iii) understanding of data collection settings. therefore, nz health strategy is expected to lead to improved accuracy and quality of big healthcare data that will later be used for big data analytics to undertake population health analytics, achieve and measure health outcomes, and make clinical decisions. 5.2 meso level: funders and planners a clear, universal representation of big data was not seen within the meso level. academics and vendors were able to clearly define big data while participants from district health boards (dhbs) and primary health organisations (phos) often voiced their confusion of how it differed from ‘small’ data. when asked about what influenced their understanding of big data academics spoke of their work (teaching or research) in a related field. similarly, because of vendor participants’ involvement in big data projects and their organisations constant promotion of big data related discussion (anchoring) they have a clear understanding of the term as used within their organisation. while the meso level participants did not think big data is new to healthcare, they identified that areas like measuring outcomes, population health analysis and clinical care can be improved with the use of more data. they also identified new areas of potential for health, such as precision medicine and cross government analysis through integrated data infrastructure (idi). although big data is not a novelty at the meso level, they did identify new types of data emerging within the healthcare sector such as genomics data and patientgenerated data that has great potential to improve clinical care. the it vendors and phos showed a great deal of interest in patient-generated data, but claimed that “it’s a pipe dream” 4 new zealand health system (including the health strategy) has undergone reforms after data collection took place. australasian journal of information systems weerasinghe et al. 2023, vol 27, research article alignment of big data perceptions 15 (mes9) to use such data in clinical care, specifically because there is lot more that needs to be improved before integrating patient-generated data. the planners and funders saw a lack of understanding by the micro level (clinicians) about the use of big data for clinical care and overall health system, as there is no professional discussion about the potential of big data across the sector. however, they explained that if clinicians were shown evidence of the potential benefits of big data, then they would get on board. participants emphasised that there are clinical level people who can influence their organisations to make better systems. on this basis, they highlighted that the government needs to promote a discussion of big data across the healthcare sector to enhance everyone’s understanding about it (anchoring). a majority of the participants at the meso level agreed that the government (macro level) is heading in the right direction with healthcare policy by identifying big data under ‘smart system’. since data is a key part of a smart system, this understanding of big data as a part of smart systems can be explained as anchoring of big data through health policy. however, some participants pointed out that health policy is not catering enough (if at all) to capturing patient-generated data, which they identified as being an important aspect of big data in healthcare, saying “no policy maker is talking about it [patient-generated data]” (mes13). 5.3 micro level: clinicians (general practitioners and hospital doctors) at the micro level, big data was represented as “large datasets” (mic5) or “national datasets” (mic2). there was no other understanding about what big data was but the participants had a very good idea of how data can be used in the healthcare context, particularly for clinical care and decision making. discussing the applications that used data, doctors expressed that a few tools are used such as health pathways (online tool that guides clinicians to manage health conditions of patients), and risk calculators. some doctors (specialists in hospitals) talked about how they still use manual disease risk calculators. the doctors raised concerns over data quality and hinted that because of their individual experience as well as experience of fellow colleagues with poor data quality (anchoring and objectification), they are reluctant to use tools for clinical decision making unless they are sure that the systems work. they explained that clinical decision making tools should be as rigorously tested as medications are if they are to be relied on in patient health care. clinicians also talked about how cumbersome the systems in use are in sharing data, being fragmented in nature and typically not effectively communicating with each other. for example, different general practitioner (gp) practices use different patient management systems (pmss) (e.g., medtech, mypractice) and hospitals use a completely different one. the gps showed a level of frustration with not being able to link real time to hospital data, and hospital doctors explained how it is an utter waste of money repeating tests because “i’m blind, i can’t see any of that data [data on reports done by the gp]” (mic6). gp’s also talked about patient-generated data and thought it was a good way to get to know the patients better. as the patients themselves can record data about their health with greater frequency (e.g., seven consecutive readings of blood pressure done at home verses one reading at the clinic) and share at (or bring along to) the consultation. australasian journal of information systems weerasinghe et al. 2023, vol 27, research article alignment of big data perceptions 16 6 discussion: alignment of big data perceptions across nz healthcare typically the result of an alignment study is identifying alignment and misalignment present through the investigated context/phenomenon. as a business-it alignment study, further analysis into the above findings through a cross level analysis showed areas of alignment and misalignment of big data technologies across the sector. it is important to iterate that this is not a traditional business-it alignment study (investigating the strategic dimension of alignment), but instead a study that investigated the social dimension of alignment. therefore, the study’s focus was on how participants perceived big data technologies and the technology’s potential/application for their day-to-day work. table 4 summarises alignment and misalignment issues found through further analysis of the findings. alignment in this study means that all three levels (macro, meso, micro/mmm) have similar perspectives. if only two of three levels aligned, this was considered misalignment. however, if one of the mmm levels did not talk about a certain issue/topic while the other two did and had similar views it was taken to be alignment. alignment misalignment • importance of data quality is well understood by all three levels. • privacy and security of data is seen as a challenge across the sector. • agreement around use of more data for improved measures of health outcomes • agreement by macro and meso around changes to skills and technology infrastructure to facilitate big data • aligned views (of macro and meso) around health policy and strategy as providing initial direction towards the future of big data • ambiguity and differences in defining big data within and across levels • differing views on velocity as a characteristic of big data • misaligned views around definitions of data ownership • disagreements around data sharing practices and privacy laws influencing data sharing • differing opinions around interoperability • misalignment around areas of application (precision medicine and clinical decision making) • invisibility of patient-generated data in health policy and strategy table 4. areas of alignment and misalignment of big data in new zealand healthcare these examples of alignment and misalignment are discussed below in light of the related literature. when discussing misalignment, tsr underpins the explanations and recommendations. 6.1 areas of alignment importance of data quality is well understood by all three levels of the healthcare sector. while big data brings many opportunities to healthcare, it also adds significant challenges around data quality that need to be addressed (halamka, 2014). perceptions of all three mmm levels showed that participants understood the importance of data quality. while arguing that data quality is not just about accuracy, participants across all levels identified factors that influence data quality such as relevance, completeness, timeliness, level of summarisation and availability of contextual information. the analysis also showed that those at the macro level are working on ensuring data accuracy through implementation of standards and policies, which will facilitate the capture of correct and complete data. those at the meso and micro levels agreed on the importance of ensuring standards through appropriate policy to maintain data quality. while this aligns with discussions in the literature around health policy (e.g., australasian journal of information systems weerasinghe et al. 2023, vol 27, research article alignment of big data perceptions 17 roski et al., 2014), this finding adds to the literature by identifying the agreements and acknowledgements at other levels around the importance of policy as a facilitator of data quality. another area of interest for improving accuracy is enabling the direct input of data to health information systems from digital devices (explained as the use of internet of things (iot) technologies) without the need for a human interface (e.g., blood pressure monitors entering the reading directly into patient records). iot technologies improve the reliability of data which increases data quality due to the connectedness brought through iot and sensor technologies (kyriazis & varvarigou, 2013). however, the iot literature also highlights data quality issues due to dropped readings, multi-source inconsistencies, and unreliable readings (karkouch et al., 2016). participants from the meso level emphasised their work around implementing proper measures to ensure the capturing of accurate data, through collecting correct and complete data. doctors at the micro level also talked about the value of data quality, explaining the importance of quality data for clinical and administrative decision making. all of the mmm levels have dealt with issues around poor data accuracy and therefore see it as important to tackle accuracy issues with modern technology around data. privacy and security around health data is seen as another major challenge by all of the participants across the sector. all participants agreed that privacy around personal health data must be secured. privacy and security are great concerns in the big data era, especially due to the amount of data being held by organisations, as well as potential use of cloud service providers (esposito et al., 2018). moreover, healthcare organisations have an added responsibility because of the sensitivity and the personal nature of healthcare data, which demands greater requirements around privacy and security measures. roski et al. (2014) argue that current practices, policies and security measures around the use of data need to be revisited by policy makers to facilitate better data security in the big data era, this does currently seem to be the case in new zealand. improved measures of health outcomes is facilitated by big data. when talking about the possible applications of big data technologies, all three levels talked about improvements to measuring outcomes within the healthcare sector. from clinicians who talked about the importance of getting a detailed view on how their patients are doing to policy makers wanting to see how they are achieving health targets (or not), there was a clear acknowledgement of how more data as well as new types of data can improve current practices of measuring outcomes. measuring health outcomes has been standard practice and the healthcare sectors are constantly looking for practices and technologies to improve these measurements (strome, 2014). globally, improvements to the measurement of outcomes are identified as a key area in which big data technologies can be utilised (groves et al., 2013). agreements around changes in skills and technology. another important challenge identified by macro and meso levels (and not by micro, as it is not relevant to them and their work) is changes to skills, it infrastructure, and it architecture. meso level also identified organisational structure changes around transformation of data. these show similarities with existing literature around big data transformation (davenport & dyché, 2013). at the macro level there was no data around organisational structure changes in the data collected in 2016, but there has been a recent restructure in late 2018 at moh to include a data and digital directorate. this is a significant step towards better policy, implementation, use and management of big health data. australasian journal of information systems weerasinghe et al. 2023, vol 27, research article alignment of big data perceptions 18 health strategy provides initial direction toward big data. macro and meso levels also identified and accepted that health strategy is providing initial direction toward big data technologies in the nz healthcare sector. the macro level believed the term “smart systems” (mac1) in health strategy is related to initiatives around big data, and the meso level agreed. further, meso level participants claimed that having this term included in strategy (which helped with anchoring and objectification, based on srt in tsr) provides a good platform to discuss big data technologies and their application across many domains. the literature also highlights the importance of health policy and strategy in providing direction around big data technologies as an important factor for big data success (blasimme et al., 2018; roski et al., 2014). 6.2 areas of misalignment in this section identified areas of misalignment are discussed, drawing on findings and literature. tsr is used to identify likely reasons for these misalignment issues. recommendations are made at the end of each point that can potentially overcome identified issues of misalignment. ambiguity and differences in defining big data within and across levels. the initial overall analysis showed that there was a lack of understanding and knowledge around defining the term ‘big data’ within all levels of healthcare. people across the sector defined big data in different ways. for some participants, big data is not seen as something new (e.g., mic6) while some saw big data as something they did not clearly understand (e.g., mes3), or were reluctant to use the term due to confusion around it (e.g., mac5). some participants saw big data as a buzzword (e.g., mes14), with a few exceptions who were able to clearly define big data (e.g., mac1). while big data has as of late turned into a buzzword, the literature highlights that there has to be a common understanding that big data concerns data that is too large, too fast and too complex to be dealt with through traditional/existing technologies (andreu-perez et al., 2015). there is an unanswered question as to whether big data is genuinely a new phenomenon, or whether large scale datasets consisting of data routinely collected for years are also classed as big data (collins, 2016). however, modern technologies developed around big data have increased the capabilities for making use of such large-scale datasets (collins, 2016). similarly, most participants acknowledged that evolving technology is what generates big data and new possibilities around health data. big data literature in the health domain explains that big data is not just about existing forms of health data but about utilising new forms of data that can be linked to health systems to improve healthcare management and delivery of care (ginsburg & phillips, 2018; zadeh et al., 2019). there was an understanding across sector levels that technology is changing how healthcare is delivered and data plays a prominent role. however, it was observed that these understandings are misaligned (e.g., some identified big data as new, some as a buzzword, some as not new) and are not heading in the same direction. while it is acceptable to have evolving representations as explained in tsr foundational theory (i.e., srt) (moscovici, 1984) and alignment as a process (jenkin & chan, 2010), ambiguity is not the same as evolving representations. ambiguity shows unawareness, lack of understanding and confusion. understandings through tsr can explain this ambiguity by drawing on data that shows there is a lack of anchoring of the term across the sector. many participants did not have anchoring experiences (group conversations, presentations, documentation such as policy) to understand big data and its possibilities. therefore, participants objectified the term ‘big data’ based on their background, past experience, australasian journal of information systems weerasinghe et al. 2023, vol 27, research article alignment of big data perceptions 19 understanding and other individual components (e.g., mac2, being an epidemiologist5 , has seen a lot of data for many years of her career and claims big data is not new). thus, they did not get to anchor the term through discussions within or across their levels. this results in many (mis)alignment issues across the sector. it is recommended that this gap needs to be addressed, mainly facilitated by macro and meso level organisations across the sector. such initiatives could include initiating discussions around the concept of big data and its applications. differing views on velocity as a characteristic of big data. in the big data literature, velocity is discussed as the real-time use of data (mcafee & brynjolfsson, 2012). however, one of the macro participants (mac56 ) emphasised that they opposed the views on real-time data, claiming “the real information comes through analysing both historical data and the most recent data”. while other macro participants talked about speed of data creation as velocity, there was a lack of explanation around using data in real time at the macro level. however, there was evidence at the meso level that participants saw timely use of data as a part of velocity and they saw using real-time data as desirable (but not currently being done in practice). these misaligned perceptions relating to the definition of big data can also be linked to the lack of anchoring across the sector. however, such alignment gaps may result in policies not capturing the potential use of data in real time as desired and valued by the other levels. similarly, to address unclear definitions of big data, in order to get everyone on the same page a common discussion is recommended. misaligned perceptions around data ownership. all three levels showed uncertainty around who owns patient data. at the macro level, it was highlighted that “primary care has a view that they own the patient information, and the patient information is a commercial asset” (mac4). meso participants also showed their confusion around data ownership. a senior technical specialist from a pho explained this confusion, saying “[t]he problem is there’s always the big question of ‘who owns the data?’ so if you ask this from a doctor, gp or a specialist or a dhb or the ministry of health i’m not sure they will answer you” (mes8). while the gps at the micro level were not sure whether they, the phos or the government owned patients’ data, hospital doctors did not have any comments about data ownership. because primary care (e.g. phos) has a responsibility to collect patient data, this may imply to those at the primary care level that they own this data. however, not having clear policies to facilitate anchoring and objectification of what data ownership means may result in confusion and a lack of clarity for all parties. data ownership has been a common concern in big data literature (kaisler et al., 2013); specifically in health, it is said that policy makers are required to redesign policies around data ownership when health systems are starting to utilise big data technologies (roski et al., 2014). therefore, it is highlighted that transparent guidelines through health policy are needed to facilitate clear understandings about data ownership. misalignment around data sharing practices and privacy laws influencing data sharing. while all three levels agreed on the importance of privacy and security around big health data, there seems to be disagreements around practices and privacy laws. macro level participants stated 5 a person who analyses populations to understand certain aspects of health (population health analysis) 6 mac5 is a senior executive leading a team at the macro level; he has had extensive experience in it and over 6 years of healthcare experience. australasian journal of information systems weerasinghe et al. 2023, vol 27, research article alignment of big data perceptions 20 that “in new zealand we have good privacy laws” (mac1). while micro and meso levels agreed that the privacy laws are protecting patient data, they highlighted that these laws may in fact be going too far, claiming privacy laws were hindering their ability to use data when it is required to help a patient. one meso level participant explained that “it [privacy law] would not allow me, as an interested party who had the capability, to help people who are disadvantaged at the moment [identified by the idi]” (mes4). tackling concerns around sharing big health data is often seen as a huge challenge to healthcare policy throughout the literature (blasimme et al., 2018). the literature highlights that policies around data sharing need to be updated and policies guiding data stewardship need to be adopted for better use of big data in the healthcare context (roski et al., 2014). similarly, meso and micro level participants recognised the need for flexible privacy laws along with clear ethical standards around sharing and use. tsr can explain this discrepancy through the definitions of tasks in different social subsystems at each level. because macro has a role in securing trust of patients through privacy laws, their understanding around the difficulties of using data is limited. on the contrary, meso and micro levels need to use data and they do report coming across these difficulties. to overcome these different perceptions, it is recommended that there is more open discussion around the importance of data sharing, requiring the policy level to be more open to revisiting policy, making necessary adjustments but also ensuring privacy of patients in the modern era. differing perceptions around interoperability. views around interoperability and the nature of the health system seem to have discrepancies. while all three levels identified the importance of interoperability, their thoughts and solutions around it showed misalignment issues. for example, the policy level acknowledged the semi-autonomous nature of the nz health system, claiming “it has always been like [semi-autonomous] that and it will probably always be like that” (mac3). they saw the semi-autonomous nature as allowing innovative organisations (phos or dhbs) to initiate new inventions without being driven by the government. however, the meso level participants saw this fragmented nature as something that created an “attitude of competitiveness” (mes14) between dhbs, causing dhbs to go in different directions and use “different systems and different methods” (mes3). the same was seen at the primary care level with phos (and their gps) using various systems (primarily pmss) that are provided by different software vendors (e.g., medtech, mypractice). doctors from hospitals reported the difficulties they go through on a daily basis due to the use of different health information systems. they also highlighted the amount of money and time wasted by having to repeat investigations, due to unlinked systems not providing them access to previous investigations done elsewhere. while implementing standards like hl7nz7 and snomed8 will facilitate interoperability, a few micro level participants felt that the government needs to mandate a single pms across the country as a starting point to fix issues around incompatibility. although the semi-autonomous nature may be an advantage, not managing it creates interoperability issues across the sector and becomes a larger problem to deal with. it was highlighted by the participants that while incompatible systems and 7 hl7nz is the new zealand affiliate of health level seven international. hl7 is the global developer of standards for health information systems to promote interoperability (http://www.hl7.org.nz/). 8 snomed international determines standards for clinical terminology (http://www.snomed.org/). snomed standards are used in electronic health records. australasian journal of information systems weerasinghe et al. 2023, vol 27, research article alignment of big data perceptions 21 interoperability issues are not issues specific to big data, moving forward into big data technologies will be difficult and may create more challenges if these issues are not dealt with in the traditional data environment. while the data suggested that there is a need for a countrywide pms, it is recommended that policy makers carefully consider this possibility alongside strengthening policies around interoperability before making changes. further research is needed to make informed recommendations. misalignment around areas of application. as identified by participants across the three levels, big data has multiple areas of application, such as: measuring outcomes (within-sector and crosssector analysis), precision medicine, population health, and clinical decision making. these areas of application were not seen in the same manner (in terms of potential and priority) by the three mmm levels. although several participants at macro and meso levels identified population health as an area with big data potential, there was little dialogue from across the sector to discuss alignment, and is not covered in this section. while perceptions about measuring outcome seemed to be in alignment as explained in the alignment section above, identified misalignment issues across precision medicine and clinical decision making are discussed below. precision medicine is a key interest of the big data area identified by the government. a precision medicine initiative by the moh is currently underway (at a research stage) in partnership with a dhb, a vendor and a university. as explained by one of the macro participants, health strategy through its identification of “smart systems” promotes fields like precision medicine. therefore, such initiatives align with overall objectives of nz healthcare. both macro and meso levels saw this initiative focussing on precision medicine as favourable. they explained that “precision medicine will at some point facilitate improved clinical care” (mac5) through understanding a person’s genomic structure. however, currently there was not enough information made available to clinicians about this precision medicine initiative, and they were not clear about the value of precision medicine. tsr can expose and elaborate this issue by explaining that the social subsystem at the macro level (being the policy makers) identifies that it is a macro level role to set futuristic strategy. as explained by mac1 and mac4, they are looking out for modern areas that nz health can benefit from. thus, because of their role, their thoughts about precision medicine are positively objectified and further anchored through discussions, allowing them to perceive the importance of precision medicine for improved healthcare in the future. similarly, those at the meso level (specifically participants from dhbs, vendors and universities) work closely with the ministry and have the opportunity to get involved in the project or in the discussion (objectification). further, because of their role (social subsystem) in planning and funding and implementing government policy (scahill, 2012), the social subsystem influences their objectification, allowing them to perceive precision medicine as a beneficial area of application for nz healthcare. in contrast, clinicians’ roles and responsibilities are around providing healthcare, and not many of them are involved in these discussions. unless they are informed, there is little opportunity for them to get information around new areas such as precision medicine. for example, mes5, who was interviewed for both meso and micro groups as they have a strategic role in a dhb while also practising as a specialist doctor in the hospital, talked about the potential of precision medicine – this shows a clear reference to tsr’s explanation of how the social subsystem influences the sociotechnical representations. therefore a robust plan for australasian journal of information systems weerasinghe et al. 2023, vol 27, research article alignment of big data perceptions 22 providing information to lower levels is important, and will facilitate a more positive environment in the future when precision-driven medicine becomes more available and applicable to front-line clinicians. clinical decision making is the other area of application discussed under misalignment. while literature has identified clinical decision making as an area that can greatly benefit from big data technologies (dang & mendon, 2015), differing perceptions were seen across the sector. the big data literature identifies clinical care and decision making as an ideal area to utilise big health data (roski et al., 2014). clinical decisions supported by data from health systems can assist decision makers to achieve gains in performance, reduce gaps between knowledge and practice, and improve patient safety (bates et al., 2003). however, macro level participants were very much focused on other areas (specifically measuring outcomes and population health) rather than looking into application of big data technologies for clinical decision making. several of the macro participants acknowledged that big data has potential in clinical care settings; they claimed “it’s not our [ministry’s] role” (mac2) to initiate the use of big data for clinical care and decision making. while the government and the moh has a broader role and is participating through its active role of understanding the overall health system and how it can be improved through modern data, clinical decision-making initiatives need support from policy for successful implementation (roski et al., 2014). identifying current clinical decision support and its use of data to be at a “rudimentary stage” (mes2), meso level participants identified the application of big data for clinical decision making as having great potential. at the micro level there seems to be confusion about the potential of big data tools to facilitate clinical decision making. these participants talked about tools like health pathways and atlas, and explained that they were wary of using any new tool without seeing evidence of its benefits. this echoes understandings of tsr, as tsr explains that experiencing something first-hand helps in terms of objectifying and anchoring it. as both the literature and the participants agree that clinical decision making can benefit from big data technologies, it is recommended that clinical decision making be made a priority and discussions initiated across the sector. prioritising clinical decision making as an important area of application will lead to development of tools; however, it will also require the greater involvement of clinicians. ignorance of patient-generated data from policy-making levels. patient-generated data (a source of big health data), while accepted and understood to have huge potential by meso and micro levels, did not seem to get much attention from the policy level. some clinicians at the micro level currently use patient-generated data through mobile apps; yet this presents many difficulties due to a lack of guidance from the policy-maker level as well as from their meso level dhbs and phos. the meso level identified the use of patient-generated data as important and something they are interested in (specifically phos). however, these meso level participants admitted that they are not doing anything in the area of patient-generated data yet, saying “on our priority list it’s probably well down” (mes9). the meso level participants explained that policy makers need to discuss patient-generated data, and need to provide better direction to the sector through policy around capturing and using patientgenerated data. understandings through tsr can explain that this voluntary identification of patient-generated data as an area of interest by meso and micro level participants is due to their direct experience in dealing with patients (a few gps explained how patients bring data recorded by them for consultations). at macro levels the absence of patient-generated data in australasian journal of information systems weerasinghe et al. 2023, vol 27, research article alignment of big data perceptions 23 policy may hinder their ability to objectify or anchor patient-generated data as beneficial. as one of the key action areas of health strategy is being ‘people-powered’ (minister of health, 2016), it is recommended that patient-generated data be incorporated into health policy for other levels to make effective use of it. 7 practical implications the practical implications of this study include implications for healthcare policy and practice. as reported the nz healthcare sector has issues around anchoring (as explained in tsr) of big data technologies which is the cause of some misalignment issues. participants objectified big data based on their experience and background as well as their roles, but a lack of anchoring through discussions across the sector around big data was causing ambiguity around this concept. as such, a key implication of this study is the importance of having a common discussion around the notion of big data, and identifying, agreeing on, and prioritising potential applications of big data technologies based on the needs of the sector. clear articulation of concepts of big data and related technologies across the sector needs to be facilitated by the macro and meso levels. through member checking the findings of alignment and misalignment was reported back to the participants and one macro level respondent commented that lack of understanding about the concept of data ownership at other levels is particularly surprising given how policy around data ownership clearly identifies the patient as the owner of health information. this further confirms the finding on misalignment around data ownership, and highlights the importance of clear guidelines through health policy to facilitate better understandings about data ownership across the sector. in addition the findings suggest the need for an open discussion around the importance of data sharing. policy makers need to be more open to revisiting policy, making required adjustments while also ensuring privacy of patients in this big data era. another implication of the study is the possibility of a whole of sector approach (e.g., the need for a countrywide electronic patient management system) as opposed to a fragmented approach. it is recommended that policy makers carefully consider this alongside strengthening policies around interoperability before making changes9 . based on the findings around issues in communicating/information flow of technology driven projects such as precision medicine, we highlight the importance of a robust plan for providing information to the lower levels. this will facilitate a more positive environment in the future when precision driven medicine becomes more available and applicable to front-line clinicians. further to this is the lack of prioritisation on the role of big data technologies for clinical decision making. prioritising clinical decision making as an important area of application will lead to development of e-tools; however, it will also require the greater engagement and involvement of clinicians. consultation with clinicians is important when developing and implementing tools as this leads to the successful adoption of such tools. 8 conclusions, limitations and future work this study set out to investigate perceptions around big data and how business-it alignment is influenced by such perceptions (identified as sociotechnical representations) in the nz 9 nz moving into a nation-wise health system as a part of sector reforms in 2022 is a step towards this direction. australasian journal of information systems weerasinghe et al. 2023, vol 27, research article alignment of big data perceptions 24 healthcare sector. the theory of sociotechnical representations (tsr) was used as the theoretical basis to investigate perceptions, identified as sociotechnical representations in tsr at multiple levels of macro, meso and micro. through investigating sociotechnical representations, this study identified the social elements influencing the sociotechnical representations around big data. the paper shows the applicability of the business-it alignment taxonomy (weerasinghe et al., 2018b)and uses tsr to conduct an alignment study. the paper identified areas of alignment and misalignment across the sector around perceptions of big data and its application. understandings generated through tsr are used to explain misalignment and provide recommendations where necessary. using the business-it alignment taxonomy (weerasinghe et al., 2018b), different lenses were used to frame the research. examining strategic fit allowed us to investigate people, strategy and policy, and technology (henderson & venkatraman, 1993) to understand how businessit alignment is influenced by sociotechnical representations of big data. the findings demonstrate that the sociotechnical representation of big data in the nz healthcare sector is formed through the influence of perceived definitions, valued characteristics, identified issues and challenges, identified areas of application, as well as direction provided through policy and strategy. while there was no clearly formed sociotechnical representation of big data, this was expected when investigating business-it alignment as a process (chan & reich, 2007). nz healthcare is an early adopter of electronic devices and computer systems in comparison to other parts of the world (protti & bowden, 2010). this habit of early adoptions has now resulted in huge amounts of data, and this will increase exponentially. with data being generated for nearly 30 years, along with datasets like the national minimum dataset of nz and other health datasets held by the ministry of health, traditionally collected health data is perceived to be big data. on top of this, new types of data like genomics and patient-generated data require big data technologies to be applied to facilitate improved healthcare delivery and management in nz. the implications of this paper are both theoretical and practical. theoretically the paper contributes specifically to tsr literature, making this one of the first applications of tsr and illustrating its potential contribution to is research. tsr provides a specific tool for investigating perceptions of technology and provides a way of understanding how perceptions of technology are formed at individual and group levels (objectification and anchoring). by extension, this understanding allows for interventions to shape objectification and anchoring. the practical implications of the study include the identification and discussion of areas of alignment and misalignment (which has been covered in detail in the previous section in detail). one of the key observations brought to light with tsr is that there is a lack of anchoring activities about big data across the sector. to create a common understanding about big data, its potential and application, it is important to initiate open discussions across the sector, possibly initiated by macro and meso levels. this was identified by several meso level participants who highlighted that there was no common discussion around what big data is, which was leading to confusion and possibly missed opportunities. while mmm levels were separately analysed before the cross-group analysis, some subgroups within levels were also observed. specifically, within the meso level, there were differing representations in some areas of application among subgroups such as dhbs, phos, vendors and academics. while presentation of such subgroups is accepted in tsr (through srt), australasian journal of information systems weerasinghe et al. 2023, vol 27, research article alignment of big data perceptions 25 prominent differences are noted in the discussion. it was identified that data from participants from dhbs show many similarities to those of the macro level. this can also be explained by tsr: because dhbs work closely with the government, many dhb participants were involved, or have been involved in government discussions around health-it. this could have allowed them to shape their perceptions (sociotechnical representations), through anchoring, to become similar to that of the government (and vice versa). participants at universities and vendors also had experience working with the government, but their roles seemed more independent (e.g., independent research funded by the ministry) – which could explain why their representations are not always as similar. one of the limitations of this study is that at the micro level, there are other clinicians that have not been included as participants (nurses). the researchers tried getting nurses involved but did not succeed, and due to time constraints around research, the decision was made to go ahead with hospital doctors and general practitioners. another study limitation is that pharmacy-related policy, funding, planning and use were not investigated. however, pharmacy is a different area and it is identified as a potential topic of study for the future. other future work could include investigations into specific issues around interoperability through a tsr lens, and investigating policy in more depth, existing use of standards and actual use in the clinical frontline. investigations into patient-generated data from a policy perspective are also suggested. quantitative studies would also be useful in generalising issues of alignment and misalignment within the nz healthcare sector and also for formalising the applicability of tsr. references andreu-perez, j., poon, c. c., merrifield, r. d., wong, s. t., & yang, g.-z. (2015). big data for health. ieee journal of biomedical and health informatics, 19, 1193-1208. ashley, e. a. (2016). towards precision medicine. nature reviews genetics, 17, 507-522. bag, s., dhamija, p., luthra, s., & huisingh, d. (2023). how big data analytics can help manufacturing companies strengthen supply chain resilience in the context of the covid-19 pandemic. the international journal of logistics management, 34, 1141-1164. bakker, l., aarts, j., uyl-de groot, c., & redekop, w. (2020). economic evaluations of big data analytics for clinical decision-making: a scoping review. journal of the american medical informatics association, 27, 1466-1475. bates, d. w., kuperman, g. j., wang, s., gandhi, t., kittler, a., volk, l., spurr, c., khorasani, r., tanasijevic, m., & middleton, b. (2003). ten commandments for effective clinical decision support: making the practice of evidence-based medicine a reality. journal of the american medical informatics association, 10, 523-530. bean, r., & kiron, d. (2013). organizational alignment is key to big data success. mit sloan management review big idea: data & analytics, 54. blasimme, a., fadda, m., schneider, m., & vayena, e. (2018). data sharing for precision medicine: policy lessons and future directions. health affairs, 37, 702-709. burns, w. a. 2014. healthcare data's perfect storm: why healthcare organizations are drowning in the data they are creating and why they need even more data to weather this storm [white paper]. available: https://s3.amazonaws.com/rdcmsaustralasian journal of information systems weerasinghe et al. 2023, vol 27, research article alignment of big data perceptions 26 himss/files/production/public/himssorg/content/files/hitachidatasystems_br458_perfectstorm.pdf [accessed 26 feb 2016]. cecez-kecmanovic, d., galliers, r. d., henfridsson, o., newell, s., & vidgen, r. (2014). the sociomateriality of information systems. management information systems quarterly, 38, 809-830. chan, y. e., & reich, b. h. (2007). it alignment: what have we learned? journal of information technology, 22, 297-315. chawla, n., & davis, d. (2013). bringing big data to personalized healthcare: a patientcentered framework. journal of general internal medicine, 28, 660-665. chen, m., mao, s., & liu, y. (2014). big data: a survey. mobile networks and applications, 19, 171-209. collins, b. (2016). big data and health economics: strengths, weaknesses, opportunities and threats. pharmaco economics, 34, 101-106. cumming, j. (2011). integrated care in new zealand. international journal of integrated care, 11, e138. dang, a., & mendon, s. (2015). the value of big data in clinical decision making. international journal of computer science and information technologies, 6, 3830-3835. davenport, t. h. (2013). analytics 3.0: in the new era, big data will power consumer products and services [online]. harvard business school press. available: https://hbr.org/2013/12/analytics-30 [accessed 12 jun 2016]. davenport, t. h., & dyché, j. (2013). big data in big companies. international institute for analytics. https://www.iqpc.com/media/7863/11710.pdf dhawan, r., singh, k., & tuteja, a. (2014). when big data goes lean. mckinsey quarterly, 24, 97-102. dopfer, k., foster, j., & potts, j. (2004). micro-meso-macro. journal of evolutionary economics, 14, 263-279. dulipovici, a., & robey, d. (2013). strategic alignment and misalignment of knowledge management systems: a social representation perspective. journal of management information systems, 29, 103-126. el-mekawy, m., rusu, l., & perjons, e. (2015). an evaluation framework for comparing business-it alignment models: a tool for supporting collaborative learning in organizations. computers in human behavior, 51, 1229-1247. emani, c. k., cullot, n., & nicolle, c. (2015). understandable big data: a survey. computer science review, 17, 70-81. emery, f. e. (1959). characteristics of socio-technical systems. london, uk: tavistock institute. emery, f. e., & trist, e. l. (1965). the causal texture of organizational environments. human relations, 18, 21-32. esposito, c., de santis, a., tortora, g., chang, h., & choo, k. r. (2018). blockchain: a panacea for healthcare cloud-based data security and privacy? ieee cloud computing, 5, 31-37. australasian journal of information systems weerasinghe et al. 2023, vol 27, research article alignment of big data perceptions 27 eynon, r. (2013). the rise of big data: what does it mean for education, technology, and media research? learning, media and technology, 38, 237-240. fattah, f., & arman, a. a. (2014). business-it alignment: strategic alignment model for healthcare (case study in hospital bandung area). 2014 international conference on ict for smart society (iciss), bandung, indonesia, 2014, 256-259, doi: 10.1109/ictss.2014.7013183. gal, u., & berente, n. (2008). a social representations perspective on information systems implementation: rethinking the concept of "frames". information technology & people, 21, 133-154. ginsburg, g. s., & phillips, k. a. (2018). precision medicine: from science to value. health affairs, 37, 694-701. grant, g. (2010). reconceptualizing the concept of business and it alignment: from engineering to agriculture. european journal of information systems, 19, 619-624. groves, p., kayyali, b., knott, d., & van kuiken, s. (2013). the 'big data' revolution in healthcare: accelerating value and innovation [online]. available: https://www.mckinsey.com/industries/healthcare-systems-and-services/ourinsights/the-big-data-revolution-in-us-health-care [accessed]. guo, c., & chen, j. (2023). big data analytics in healthcare. knowledge technology and systems: toward establishing knowledge systems science. springer. halamka, j. d. (2014). early experiences with big data at an academic medical center. health affairs, 33, 1132-1138. he, k. y., ge, d., & he, m. m. (2017). big data analytics for genomic medicine. international journal of molecular sciences, 18, 412. https://doi.org/10.3390/ijms18020412 henderson, j. c., & venkatraman, n. (1992). strategic alignment: a model for organizational transformation through information technology. in: kochan, t. a., & useem, m. (eds.) transforming organizations. new york, usa: oxford university press. henderson, j. c., & venkatraman, n. (1993). strategic alignment: leveraging information technology for transforming organizations. ibm systems journal 32, 472-484. herland, m., khoshgoftaar, t. m., & wald, r. (2014). a review of data mining using big data in health informatics. journal of big data, 1, 1-35. jameson, j. l., & longo, d. l. (2015). precision medicine personalized, problematic, and promising. obstetrical & gynecological survey, 70, 612-614. jenkin, t. a., & chan, y. e. (2010). is project alignment a process perspective. journal of information technology, 25, 35-55. jia, y., wang, n., & ge, s. (2018). business-it alignment literature review: a bibliometric analysis. information resources management journal, 31, 34-53. jim, h. s., hoogland, a. i., brownstein, n. c., barata, a., dicker, a. p., knoop, h., gonzalez, b. d., perkins, r., rollison, d., & gilbert, s. m. (2020). innovations in research and clinical care using patient-generated health data. ca: a cancer journal for clinicians, 70, 182-199. australasian journal of information systems weerasinghe et al. 2023, vol 27, research article alignment of big data perceptions 28 kaisler, s., armour, f., espinosa, j. a., & money, w. (2012). big data: issues and challenges moving forward. in proceedings of the 46th hawaii international conference on system sciences, 7-10 january 2013. 995-1004. karkouch, a., mousannif, h., al moatassime, h., & noel, t. (2016). data quality in internet of things: a state-of-the-art survey. journal of network and computer applications, 73, 57-81. krey, m. (2018). facing business-it-alignment in healthcare. in proceedings of 51st hawaii international conference on system sciences (hicss), waikoloa village hi, usa, 3-6 january 2018. 3090-3099. kyriazis, d., & varvarigou, t. (2013). smart, autonomous and reliable internet of things. procedia computer science, 21, 442-448. leonardi, p. m. (2011). when flexible routines meet flexible technologies: affordance, constraint, and the imbrication of human and material agencies. mis quarterly, 147-167. liamputtong, p., & ezzy, d. (2005). qualitative research methods, oxford, uk: oxford university press. lincoln, y. s., & guba, e. g. (1985). naturalistic inquiry, london, uk: sage publications. luftman, j. (1996) . competing in the information age : strategic alignment in practice, new york, usa: oxford university press. lv, z. & qiao, l. (2020). analysis of healthcare big data. future generation computer systems, 109, 103-110. mason, m. (2010). sample size and saturation in phd studies using qualitative interviews. forum: qualitative social research, 11, 1-19. mcafee, a., & brynjolfsson, e. (2012). big data: the management revolution. (cover story). harvard business review, 90, 60-68. menon, n. m., yaylacicegi, u., & cezar, a. (2009). differential effects of the two types of information systems: a hospital-based study. journal of management information systems, 26, 297-316. merriam, s. b. (2009). qualitative research: a guide to design and implementation, san francisco,, ca, usa: jossey-bass. miles, m. b., huberman, a. m., & saldana, j. (2014). qualitative data analysis: a methods sourcebook, thousand oaks, ca, usa: sage publications. minister of health (2016). new zealand health strategy: future direction. wellington, new zealand: ministry of health ministry of health (2014). briefing to the incoming minister 2014. wellington, new zealand. https://www.treasury.govt.nz/publications/bim/briefing-incoming-minister-health2014 ministry of health (2017). overview of the health system [online]. new zealand: ministry of health. available: http://www.health.govt.nz/new-zealand-health-system/overviewhealth-system [accessed 29 jan 2019]. australasian journal of information systems weerasinghe et al. 2023, vol 27, research article alignment of big data perceptions 29 moscovici, s. (1984). the phenomenon of social representation. in: farr, r. m., & moscovici, s. (eds.) social representations: european studies in social psychology. cambridge, uk: cambridge university press. moscovici, s. (1988). notes towards a description of social representations. european journal of social psychology, 18, 211-250. nash, d. b. (2014). harnessing the power of big data in healthcare. american health & drug benefits. o'driscoll, a., daugelaite, j., & sleator, r. d. (2013). ‘big data’, hadoop and cloud computing in genomics. journal of biomedical informatics, 46, 774-781. orlikowski, w. j., & scott, s. v. (2008). sociomateriality: challenging the separation of technology, work and organization. academy of management annals, 2, 433-474. paré, g., sicotte, c., jaana, m., & girouard, d. (2008). prioritizing clinical information system project risk factors: a delphi study. in proceedings of the 41st annual hawaii international conference on system sciences, waikoloa, hi, usa, 2008. doi: 10.1109/hicss.2008.354. patton, m. q. (2015). qualitative research & evaluation methods : integrating theory and practice, los angeles, ca: sage publications. petersen, c., & demuro, p. (2015). legal and regulatory considerations associated with use of patient-generated health data from social media and mobile health (mhealth) devices. applied clinical informatics, 6, 16-26. protti, d., & bowden, t. (2010). electronic medical record adoption in new zealand primary care physician offices. commonwealth fund: issues in international health policy, 96. raghupathi, w., & raghupathi, v. (2014). big data analytics in healthcare: promise and potential. health information science and systems, 2, 3. reich, b. h., & benbasat, i. (1996). measuring the linkage between business and information technology objectives. mis quarterly, 20, 55-81. roski, j., bo-linn, g. w., & andrews, t. a. (2014). creating value in health care through big data: opportunities and policy implications. health affairs, 33, 1115-1122. russom, p. (2011). big data analytics. tdwi best practices report, fourth quarter. https://tdwi.org/research/2011/09/best-practices-report-q4-big-dataanalytics.aspx?tc=page0&tc=assetpg&tc=page0&tc=assetpg&m=1 saporito, p. (2013). the 5 v's of big data: value and veracity join three more crucial attributes that carriers should consider when developing a big data vision [online]. a.m. best company, inc. available: https://www.thefreelibrary.com/the+5+v%27s+of+big+data%3a+value+and+veracity+j oin+three+more+crucial...-a0350676739 [accessed 7]. sathi, a. (2012). big data analytics: disruptive technologies for changing the game. boise, usa: mc press. scahill, s. l. (2012). the 'way things are around here' : organisational culture is a concept missing from new zealand healthcare policy, development, implementation, and research. new zealand medical journal (online). australasian journal of information systems weerasinghe et al. 2023, vol 27, research article alignment of big data perceptions 30 shailaja, k., seetharamulu, b., & jabbar, m. (2018). machine learning in healthcare: a review. in proceedings of the 2nd international conference on electronics, communication and aerospace technology (iceca), 2018. ieee xplore, 910-914. shapiro, m., johnston, d., wald, j., & mon, d. (2012). patient-generated health data [white paper]. rti international, april. https://www.rti.org/publication/patient-generatedhealth-data-white-paper. shin, d. h. (2015). demystifying big data: anatomy of big data developmental process. telecommunications policy, 40, 837-854. shin, d. h., & choi, m. j. 2015. ecological views of big data: perspectives and issues. telematics and informatics, 32, 311-320. smith, j. a., flowers, p., & larkin, m. 2009. interpretative phenomenological analysis : theory, method and research, london, uk: sage publications. strome, t. l. 2014. healthcare analytics for quality and performance improvement, hoboken, nj, usa: john wiley & sons. thomas, d. r. 2006. a general inductive approach for analyzing qualitative evaluation data. american journal of evaluation, 27, 237-246. tormay, p. (2015). big data in pharmaceutical r&d: creating a sustainable r&d engine. pharmaceutical medicine new zealand, 29, 87-92. ward, m. j., marsolo, k. a., & froehle, c. m. 2014. applications of business analytics in healthcare. business horizons, 57, 571-582. watson, h. j. (2014). tutorial: big data analytics: concepts, technologies, and applications. communications of the association for information systems, 34, 1247-1268. weerasinghe, k. (2019). transformation through big data analytics: a qualitative enquiry in health. 30th australasian conference on information systems. perth, australia. weerasinghe, k., pauleen, d., scahill, s., & taskin, n. (2018a). development of a theoretical framework to investigate alignment of big data in healthcare through a social representation lens. australasian journal of information systems, 22. weerasinghe, k., pauleen, d. j., scahill, s. & taskin, n. (2022a). theory of sociotechnical representations: a novel approach to understanding technology perspectives. pacific asia conference on information systems, 5-9 july 2022 taipei/sydney. association for information systems. https://aisel.aisnet.org/pacis2022/314 weerasinghe, k., scahill, s. l., pauleen, d. j., & taskin, n. (2022b). big data analytics for clinical decision-making: understanding health sector perceptions of policy and practice. technological forecasting and social change, 174, 121222. weerasinghe, k., scahill, s. l., taskin, n., & pauleen, d. j. (2018b). development of a taxonomy to be used by business-it alignment researchers. 22nd pacific asia conference on information systems. yokohama, japan: ais electronic library. weli, a. r. (2018). precision medicine. health affairs, 37, 687-687. williams, m. s., buchanan, a. h., davis, f. d., faucett, w. a., hallquist, m. l. g., leader, j. b., martin, c. l., mccormick, c. z., meyer, m. n., murray, m. f., rahm, a. k., schwartz, m. australasian journal of information systems weerasinghe et al. 2023, vol 27, research article alignment of big data perceptions 31 l. b., sturm, a. c., wagner, j. k., williams, j. l., willard, h. f., & ledbetter, d. h. (2018). patient-centered precision health in a learning health care system: geisinger’s genomic medicine experience. health affairs, 37, 757-764. wyber, r., vaillancourt, s., perry, w., mannava, p., folaranmi, t., & celi, l. a. (2015). big data in global health: improving health in lowand middle-income countries. bulletin of the world health organization, 93, 203-208. yin, r. k. (2014). case study research : design and methods, los angeles, ca: sage publications. zadeh, a. h., zolbanin, h. m., sharda, r., & delen, d. (2019). social media for nowcasting flu activity: spatio-temporal big data analysis. information systems frontiers, 1-18. appendices appendix 1: interview schemas 1.2. macro interview schema section a: demographics interview questions 1. how many years have you been working in the healthcare sector? 2. how many years have you been working in a policy making or advisory role? 3. what is your current position(s)? select which role to go forward with. 4. how long have you been in this position? 5. apart from working in the healthcare sector do you have a background in a. business b. it c. other: ……………………………………. section b: interview questions general information 6. looking from a health-it perspective, what are the responsibilities of your organisation towards the nz healthcare system? 7. can you describe your role and responsibilities within your organisation? to whom do you report? and who reports to you? (where do you fit in the company structure?) sociotechnical representation of big data (tsr) 8. there have been on-going discussions around the use of big data analytics in healthcare. in the literature there are different ideas relating to big data. i’m really interested in getting to know what your perception of big data is. what do you understand by the term big data? 9. i am interested in your view on the contribution that big data could make in the health care sector. do you think using big data and big data analytics could be used for better planning and delivery of healthcare? a. if so how? australasian journal of information systems weerasinghe et al. 2023, vol 27, research article alignment of big data perceptions 32 b. if not why not? c. not sure? note for interviewer: big data analytics refers to making use of tools and technologies to analyse large amounts of data that comes from a variety of sources. the sources could be a variety of healthcare information systems – clinical care, and administrative decision making as well as consumer generated data. 10. could you talk a bit about what might have influenced or informed your understanding of what big data is? prompt: do you think your understanding of big data was influenced by discussions you had with other board members or any other factors? 11. do you think that this (your) perceptions/ understandings are common across the board/organisation/department? or have you seen any different views in others? business-it alignment (through a social dimension lens) 12. i’ve looked at the health-it program 2015-2020 on the nhitb’s website/moh health strategy. so where do you see big data analytics within this program? 13. can you explain to me the reasons why big data is presented in that way (or not presented) in the health-it program/strategy? (what were the reasons for including big data in the health-it program/health strategy?) note: history of it success? big data analytics in healthcare success stories from other countries? industry pressure? need? 14. in your opinion is there anything missing or included which shouldn’t be included (related to big data)? 15. can you give me some examples of use of big data, big data analytics tools that you might know of? 16. i am interested in better understanding degrees of alignment in health care. do you think the big data analytics initiatives (outlined within the health-it plan/health strategy or the example you’ve given) align to the government’s healthcare objectives? if so can you elaborate, if not why not? to what extent do you think the big data analytics will actually facilitate the government objectives? 17. so far we have talked about big data as a concept and the involvement in it from a top level view. have you experienced a need for big data initiatives coming from regional or local level at healthcare provision as opposed to a strategic level? 18. we are looking at macro, meso and micro level alignment. what is your perspective of dhbs’ (meso) and healthcare providers’ (micro) role in successfully implementing such big data initiatives? 19. who do you think might be the potential beneficiaries of big data initiatives, and why? (sub groups? researchers? medical centres? consumers?) 20. big data initiatives are identified in the health-it program. who do you think are going to be running them? also, who are the potential users? 21. who else do you recommend i talk to at a policy making level about big data? are you able to introduce me to other high level people who are involved in health-it policy making? australasian journal of information systems weerasinghe et al. 2023, vol 27, research article alignment of big data perceptions 33 1.2. meso interview schema general information 1. can you tell me a bit about your educational background? 2. from a health-it perspective what are the responsibilities of your organisation towards nz healthcare? 3. how many years have you been working in the healthcare sector? 4. what is your current position(s)? (if many, select which role to go forward with.) 5. how long have you been in this position? 6. what is your role and responsibilities? 7. to whom do you report? who reports to you? 8. have you done any work with the moh or its business units? if so can you talk a bit about that? 9. do you interact with (other) phos/ dhbs/ other organisations? how? big data sociotechnical representations (tsr) 10. what does big data mean to you? 11. what contribution does it make to healthcare? do you think big data and big data analytics could be used for better planning and delivery of healthcare? 12. what might have influenced or informed your understanding? 13. do you think this view of big data and its use is common across your organisation and the people you work with? or have you seen any different views? 14. why is big data different from the normal data that we have? a. is it types of analytics? b. does it require new skills? c. does big data influence the organisation’s structure and roles? d. is it the change in it infrastructure? e. do you see it architecture changing with big data? (methods, models and technologies used) current situation – business-it alignment 15. are you aware of any current or planned big data analytics projects by your organisation? can you describe them a bit? are you involved? (clinical care, outcomes, precision medicine etc.) a. are you aware of the business objectives of these project/s? b. what healthcare objectives (overall heath objectives) are these projects catering to? what benefits does it bring to the patients? c. who benefits from these projects? d. who are the (potential) end users? how involved are they in these kinds of projects? e. in your view how does this/these project/s facilitate user objectives? or, what is the current position of your organisation’s use of big data? australasian journal of information systems weerasinghe et al. 2023, vol 27, research article alignment of big data perceptions 34 what is your understanding of the current situation of big data in nz health sector? 16. do you think big data can be used to improve services of your organisation? if so how? 17. do you have any concerns about big data use in health? 18. what do you think the policy-makers’ role is with regard to the success of big data initiatives? 19. do you think any improvement is needed with regard to health it policy for the successful use of big data? 20. do you see a need for any improvements by your organisation to cater to the big data hype? 21. who else do you suggest i talk to? 1.3. micro interview schema 1. can you tell me a bit about your educational background? 2. how long have you been a doctor? 3. do you have experience working in any other industry? do you have any it experience? 4. what is your current role? how long have you been in this role? 5. what are your responsibilities both clinical and administrative/managerial? 6. gps: do you own the practice or are you a salaried employee here? hospital doctors: to whom do you report? who reports to you? sociotechnical representations (tsr) 7. are you able to talk about the responsibilities of your organisation towards nz healthcare from a health it perspective? 8. what sort of it systems do you use at your practice/work? can you talk a bit about what they are and how they help your work? 9. how would you describe the use of data in these systems? how does the data help you do your daily tasks? 10. do you see any issues around using data in these systems? a. how can these issues be mitigated? what can you (doctors) do better? b. what can phos do to mitigate such issues? c. what can the government do to mitigate such issues? (data quality, privacy and security) 11. are you using any it systems for clinical decision making? can you explain? 12. would you prefer to have more information available to improve the consultation (or do you think the information you have is sufficient)? can you explain? 13. have you ever heard of the term big data? what does big data mean to you? if no, define – big data is data that’s large in volume, complex in the sense of lots of different varieties, so in health obviously things like text with scans, x-rays, other reports and even most modern things like data from patients’ fitbits maybe. and also there’s an element of real time in big data so something like collected now and used in near real time. the use according to international research says that this type of health data has a huge potential for things like measuring the performance of the health system and population health and even to be used in the clinical frontline to improve clinical care. what do you think about this in the nz context? australasian journal of information systems weerasinghe et al. 2023, vol 27, research article alignment of big data perceptions 35 14. what are your thought about using such data for clinical decision making? a. what are the issues that you see in using such big data in clinical decision making? b. is there anything that bodies like the pho, dhb or the government can do to mitigate such issues? (or improve the use?) 15. what are your thoughts on patient-generated data? i.e. collecting data from a blood pressure monitor or from a patient’s phone? (prompt: what about patient-generated data in huge volumes that constitutes big data not just own practice clinical data) 16. do you see any issues around using patient-generated data? a. how can these issues be mitigated? b. what can pho do to mitigate such issues? c. what can the government do to mitigate such issues? 17. are there any other technologies or information systems that you see or know of or have heard of which could improve your quality of work? 18. what do you think might have influenced you to think about data (both about data in systems and patient-generated data) in this manner? 19. have you seen any different perspectives about data from others around you? 20. can you explain your best and worst experience of using system generated data? (you might even talk about an experience of a colleague?) alignment 21. from a health-it perspective how would you describe the role of your pho? what do they do to help you (or not) do your work? 22. from a health-it perspective how would you describe the role of the ministry of health? how do they help you (or not) do your job better? 23. have you been involved in doing any work with the moh or the nhitb from a healthit perspective? if so can you talk a bit about that? 24. if gp only: how would you describe your interaction with the pho, from a health it perspective? 25. if gp only: how would you describe your interaction with the dhb, from a health-it perspective? if hospital doctor: can you talk about how the dhb administration communicate with you about health it? australasian journal of information systems weerasinghe et al. 2023, vol 27, research article alignment of big data perceptions 36 appendix 2: participant demographics participants organisation type organisation role main focus of the role (it or health) number of years of experience in healthcare ict experience (research) mac1 policy making body macro organisation x senior executive it 15 years yes mac2 policy making body macro organisation x general manager health > 20 years no mac3 policy making body macro organisation x general manager health > 16 years no mac4 policy making body macro organisation x manager health > 35 years no mac5 policy making body xyz board senior executive it 10 years yes mac6 policy making body macro organisation y manager health > 10 years no mes1 funding and planning (secondary care) dhb x clinical lead health 23 years yes mes2 funding and planning body (secondary care) dhb x clinical director health > 30 years yes mes3 funding and planning body (secondary care) dhb y manager it < 6 months yes mes4 funding and planning body (primary care) pho a senior manager health 26 years yes mes5 funding and planning body (secondary care) dhb z clinical director health 45 years yes mes6 funding and planning body (primary care) pho b manager it > 10 years yes mes7 funding and planning body (primary care) pho c c-level manager it < 1 year yes mes8 funding and planning body (primary care) pho d technical staff it > 4 years yes mes9 funding and planning body (primary care) pho e knowledge manager it 25 years yes australasian journal of information systems weerasinghe et al. 2023, vol 27, research article alignment of big data perceptions 37 mes10 funding and planning body (primary care) pho f c-level manager it < 2 years yes mes11 funding and planning body (primary care) pho f technical staff it < 2 years yes mes12 funding and planning body (primary care) pho c c-level manager it > 10 years yes mes13 university university x academic health-it 40 years yes mes14 university university x academic health-it > 15 years yes mes15 funding and planning body (secondary care) dhb x epidemiologist health-it 20 years yes mes16 vendor organisation vendor x manager it > 10 years yes mes17 vendor organisation vendor x general manager health > 20 years no mic1 hospital hospital x specialist doctor health 10 years no mic2 general practice gp w gp health > 35 years yes mic3 general practice gp x gp health 29 years yes mic4 hospital hospital y specialist doctor health 25 years no mic5 retired gp health 50 years yes mic6 hospital hospital z doctor health 29 years yes mic7 general practice gp y gp health 29 years no mic8 general practice gp y gp health 10 years no mic9 general practice gp z gp health 29 years no appendix 3: sample themes and data (data analysis) categories themes description of the theme representative quotes macro: use of big data for clinical decision making sociotechnical representation: significant potential big data has significant potential in clinical decision making. “i think the biggest potential for me is in the clinical care side. in the public health side, i think we've actually being doing a lot of what we do anyway.” (mac2) anchoring: low priority problems more serious than clinical decision making require immediate attention. “there’s not a lot of people who understand the potential of big data in a clinical environment. they are probably more interested in big data and the whole health system.” (mac1) australasian journal of information systems weerasinghe et al. 2023, vol 27, research article alignment of big data perceptions 38 macro: guidance of health strategy sociotechnical representation: opportunity health strategy provides more opportunity to use big data. “so the strategy is all about a person centred view of every person in nz which is the electronic health record, it's a summary view only. keeping with that key information which is available universally across the system, that the details of that drill down through links into electronic medical records and clinical data repositories which are scattered across the entire health system.” (mac5) meso: importance of patientgenerated data anchoring: policy policy makers are not thinking about the use of patientgenerated data. “…nobody is looking at it [patientgenerated data] and saying we need to think about what we can do with that data to transform the health system so we can handle the silver tsunami.” (mes14) micro: current point of care sociotechnical representations: fragmented systems systems in use do not talk to each other and it is difficult getting information needed. “…everyone’s got different systems and different platforms and different data management platforms which makes it very difficult if we want to compare say our data in christchurch with say a group in auckland. we’re not using the same structures.” micro: clinical profession objectification: medical training medical training does not include information analysis. “when you do medical training, you obviously you develop analytical thinking skills but in a different way. not so much in terms of information analysis or operations management which are important for managing the hospital but not part of our training. so that's all new to all of us. ” (mic1) copyright: © 2023 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v27i0.4067 http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 big data in health 3 theoretical foundations 3.1 taxonomy of alignment 3.2 theory of sociotechnical representations 4 methodology 5 findings: summary findings across mmm levels 5.1 macro level: policy makers 5.2 meso level: funders and planners 5.3 micro level: clinicians (general practitioners and hospital doctors) 6 discussion: alignment of big data perceptions across nz healthcare 6.1 areas of alignment 6.2 areas of misalignment 7 practical implications 8 conclusions, limitations and future work paper title (paper title style) australasian journal of information systems nguyen, nielsen & braa 2017, vol 21, research article scaling information infrastructures: medical licensing system 1 scaling information infrastructures: the case of the medical licensing system in a southeast asian country thanh ngoc nguyen university of oslo norway thanhng@ifi.uio.no petter nielsen university of oslo norway jorn braa university of oslo norway abstract scaling health information systems from small-scale pilots to national systems in developing countries poses a key challenge to systems designers and health managers. consequently, many projects dissolve and die before they reach the scale where they are useful for information management. the concept of bootstrapping, from the information infrastructure literature, has proven useful for discussing and understanding how to initiate and grow largescale, complex and networked information systems from scratch. we use the concept of bootstrapping to analyze and discuss an empirical case of building a large-scale medical licensing system in a southeast asian country. beyond describing the process leading up to the success of the licensing system, we contribute to the literature by identifying a range of factors influencing the bootstrapping process, and we suggest methods of making the bootstrapping strategy relevant in this context. keywords: information infrastructure, medical licensing, southeast asia, bootstrapping, scaling 1 introduction health information systems (his) are a key component of national public health infrastructure (abouzahr and boerma, 2005). timely and accurate data effectively support decision makers in making sound decisions relating to disease intervention, resource allocation, etc. (hinh and van minh, 2013). the role of his is even more critical in contexts of developing countries where resources are scarce and fatal diseases are rampant (braa et al., 2004). despite substantial investment, many his implementation initiatives have not delivered outcomes as expected, and many projects end before they become fully functioning. other initiatives are not able to achieve the necessary coverage to provide useful information for health managers. these challenges are related to processes of scaling (sahay and walsham, 2006), which means that a system must be expanded both technically and geographically to become sustainable (braa et al., 2004). the existing literature discussing the scaling of information systems focuses on what is scaled and how it is scaled (see for example: sahay and walsham 2006). there are few studies that attempt to address the issue of scaling strategically, i.e., how to scale an information system. to deal with large-scale and complex information systems, which also include his in developing countries, there is a stream of research which conceptualizes such systems as information infrastructures (hanseth, 2002). through this perspective, an information infrastructure (ii) is characterized as a shared, evolving, open, standardized, and heterogeneous network (hanseth and monteiro, 1998). the role and the significance of information infrastructures (iis) is getting attention from governments, academics, and the industry (aanestad and jensen, 2011; ciborra, 2000; cordella, 2006). unlike the isolated, australasian journal of information systems nguyen, nielsen & braa 2017, vol 21, research article scaling information infrastructures: medical licensing system 2 disconnected, and single-owner information system prevailing two and three decades ago, iis are complex networks comprising multiple information systems and actors with parallel and often conflicting interests (nielsen and sæbø, 2015; sahay et al., 2009). iis are described as assemblages of heterogeneous socio-technical components (rodon and hanseth, 2015) which relentlessly undergo stabilizing/destabilizing processes around innovation and what already exists—their installed base (hanseth and lyytinen, 2010). traditional approaches used in software engineering and information systems development are not sufficient for building iis due to the ambiguity, multiplicity and ever-changing nature of user requirements to be met, and the complex and inter-organizational relationships between multiple and different systems, developers and users (sommerville et al., 2012; star, 1999; star and ruhleder, 1996). these complexities challenge existing knowledge, organizational structures, and it governance, and are described as reasons for either total or partial failures of iis (ciborra, 2000). to tackle this situation, a growing body of research is trying to identify and understand the favourable conditions that enable successful ii growth (grisot et al., 2014; sanner et al., 2014) and the mechanisms that contingently drive the evolutionary trajectories of iis (henfridsson and bygstad, 2013). unlike that of a traditional information system, the trajectory of an ii cannot be fully controlled by one or a few actors based on planned and structured requirement engineering, implementation and governance. this is because an ii does not have a single owner and its boundaries are constantly redefined (hanseth and lyytinen, 2010). for example, jensen (2013), in her discussion of the case of building integrated healthcare information systems in denmark, highlights system flexibility, strategic modularity, and the role of actors as key mechanisms to consider in understanding and dealing with the evolution of iis. as an ii is highly complex and not in central control of a single actor (nielsen and aanestad, 2006), fully managing and controlling its trajectory of development is not possible. at early phases of ii development, convincing initial users to adopt an ii is a particularly challenging task since use typically cannot be mandated and the use value of an ii is directly related to the number of other users (hanseth and lyytinen, 2010). following this, another problem that ii designers have to deal with is how to grow the initial small user base to a critical mass where the network attracts enough users and starts to grow on its own (hanseth and aanestad, 2003). the process of continuously growing the user base of an ii with little or no external assistance is described by hanseth and aanestad (2003) as “bootstrapping”. with network technologies, the value attributed to the network by the user will depend on the number of other users. for example, the first user of a mobile phone network will see no value in the network until there are other users she would like to call or receive calls from. in settings where use is optional, but also where it is mandated, bootstrapping is about how to make such a network without value grow. it is about identifying the most adequate user for enrolment, and then building on and drawing upon the installed base of users and technology to grow an ii further. with bootstrapping, user preference is not treated as static but changing, and depends upon the preferences and actions of others. according to hanseth and aanestad (2003), users' knowledge and motivation, use situation, nature of the technology, and coordinating institutions are four factors that affect users' preferences and thus will shape their adoption decision. to understand how different factors influence the bootstrapping of iis in the context of hiss in developing countries and to tease out strategies and tactics for system developers, we conducted a longitudinal case study that involved the initiation, development, implementation and evolution of an ii used for managing medical licensing processes in a country in south east asia between 2012 and 2015. through a bottom-up and iterative process, a prototype was developed based on an open source software system specifically designed for the healthcare sector. this prototype was gradually and incrementally scaled to be fully functional in the whole country as the de facto medical licensing system (hereinafter the licensing system). by december 2015, the licensing system was adopted by all public hospitals (more than 1500) and all provincial health departments (63) and had processed nearly 300,000 health professionals’ licensing applications. australasian journal of information systems nguyen, nielsen & braa 2017, vol 21, research article scaling information infrastructures: medical licensing system 3 the rest of this paper is organized as follows. a review of extant literature on iis, bootstrapping and related concepts is provided in section 2. section 3 and section 4 present research methods and the case respectively. our analysis of the case is offered in section 5 followed by conclusions in section 6. 2 related literature the problem involved in building and scaling iis has received substantial focus in the extant ii literature (hanseth et al., 1996). recently, the problem has attracted the interest of researchers from the software engineering field (sommerville et al., 2012). this issue of building and scaling also exists in discussions related to information system architectures and platforms (woodard, 2008; herzhoff et al., 2010). due to the complexities and distributed control of iis, the building and scaling of iis are often described as a process which always unfolds in smallscale and evolutionary steps. iis develop because they have a life on their own without being controlled by human agencies, as described by dahlbom and janlert (1996): “the tomatoes themselves must grow just as the wound itself must heal” (p.6). due to the limited control of human agencies, the trajectories of ii growth are thus subject of unintended and unanticipated consequences and drift (ciborra, 2000). it becomes problematic in domains such as health information system (his) when an his can only become useful when it is expanded to reach a certain geographical coverage, i.e., a district or a province (sahay and walsham, 2006; sæbø, 2013). indeed, many pilot hiss have died because they could not provide data useful for health managers (braa et al., 2004). strategies to influence ii evolutionary growth in the situation of lack or limited control are thus crucial for ii building efforts. to address the problem of building up the user base from scratch, ii literature offers the concept of bootstrapping, which refers to the process of building a large network through initially using minimal resources, i.e., making an ii grow by itself (hanseth and aanestad, 2003). hanseth and aanestad (2003) use the logics of network to develop the concept of bootstrapping. network logics or network economics (shapiro and varian, 2013) is a concept from the field which focuses on studying large-scale and complex technical networks such as telecommunication, infrastructure, and standards. network economics refer to cases where the value of a network is increased according to the number of users who are using it. take, for example, the telephone network: it is useful only when its user can make a phone call to many other users—similar to email systems in the early days of the internet. the problem when building such a network is that it is difficult to convince users to start using it at the beginning when the number of users is very small. with a relatively high number of users, a network can start to grow by itself. to resolve this challenge, it seems to be fairly easy to identify the first group of users and incentivize and subsidize their use. when the network gains its critical mass, reaching the number of users necessary to make the next user adopt without particular incentives, the network can continue to expand without further support. hanseth and aanestad (2003) discuss the bootstrapping strategy as based on identifying those users who are willing to adopt technology and, in parallel, shape user preference through the design process. hanseth and aanestad (2003) apply this strategy in analyzing three empirical cases relating to building iis for telemedicine in norway. drawing on the three cases, hanseth and aanestad (2003) illustrate how cases that follow the bootstrapping strategy have a greater chance of succeeding. further, they discuss how the degree of success of the bootstrapping strategy is largely dependent on several contextual factors: user motivation and knowledge, use area and situation, nature of the technology, and coordinating institutions: australasian journal of information systems nguyen, nielsen & braa 2017, vol 21, research article scaling information infrastructures: medical licensing system 4 user motivation and knowledge the bootstrapping strategy should focus on users who are most positively inclined toward iis and who feel that iis could help them do their work better. user knowledge of iis is also an important factor to be considered in attracting the first users, as it could reduce the cost of adoption, e.g., by reducing training and support. however, user knowledge increases proportionally to the level of use and the exposure to iis. use area and situation the use area and situation are other important factors affecting the bootstrapping strategy. for example, telemedicine may be more effective in the case of radiology than in surgery, because the practice of advice in surgery tends to require co-presence in the operation room while advice from radiologists can easily be obtained and used remotely. nature of technology several dimensions of technology, including cost, flexibility, simplicity and availability, could have high impact on the users’ attitudes. coordinating institutions when iis grow, the need for coordinating structures is increased, which is not the case when only a few users exist. however, the introduction of governance structure at the beginning could impede the growth of iis, which brings more harm than good. table 1: contextual factors influencing bootstrapping these factors are developed based on telemedicine initiatives in a western country context where contextual conditions are significantly different from those of developing countries. for example, most his projects in developing countries are funded by international donors with their own agendas. furthermore, the centralized control of governmental agencies on selecting a technology or software provider also influences the users’ preferences. and in many cases, the use of a technical solution is mandated from the top. when it comes to hiss in the context of developing countries, therefore, political climate is one of the major contextual factors that shapes users’ adoption of an ii, as described in the work of sahay et al. (2009). further, the enactment of a law which regulates and enforces certain activities could spawn the need for an ii, creating favourable conditions for technology adoption. hanseth and aanestad (2003) also discuss the step-by-step approach to bootstrapping an ii and emphasize the need for “continued bootstrapping”. they argue for the following approach based on the example of scaling an electronic data interchange (edi) infrastructure:  improve and extend the existing infrastructure  generalize the local message formats to support more standards  make similar infrastructures for other areas (other kinds of lab reports, then other kinds of forms)  improve the solutions to support better and more efficient service delivery  go for national standards and use gateway to link standards of other countries tactics to enable bootstrapping are also discussed by, among others, hanseth and lyytinen (2010). examining the case of the evolution of the internet, hanseth and lyytinen (2010) generalize three design principles to address the problem of bootstrapping:  build functionality that offers direct usefulness  reuse what already exists rather than building from scratch  focus on expanding the user base rather than functionality iacucci et al. (2002) summarizes and generalizes a list of bootstrapping tactics, taking into account both the use and design side of the building of an electronic patient record system in a large and complex norwegian hospital: australasian journal of information systems nguyen, nielsen & braa 2017, vol 21, research article scaling information infrastructures: medical licensing system 5 use 1. target the users for which an ii is useful even with a small number (i.e., installed base) of other users. 2. start with motivated and knowledgeable users. 3. start from a use area which is low in complexity and criticality, and which does not require radical organizational change. design 4. design: design the solution so that it is useful for some users without an existing installed base. 5. design: an ii should be as simple, as cheap, and as flexible as possible. subsequently, move to more complex solutions. 6. design: build on an installed base, as much as possible. 7. design: increase usefulness of an ii by establishing gateways to already existing iis. table 2: bootstrapping tactics we will, in this paper, use the bootstrapping concept and the tactics to discuss our case. we will identify and discuss a range of factors influencing the bootstrapping process and we suggest extensions to make the bootstrapping strategy relevant in the context of our case. 3 research method and approaches this research was carried out under the aegis of the global health information system program (hisp), a network of north-south-south collaboration where the university of oslo, norway (hisp uio) has a key role. this project comprises people working in the health informatics domain with the ambition to strengthen health information systems in developing countries (braa et al., 2004). the first author of this paper is a citizen of a country in south east asia where the case unfolded. he joined the hisp network in 2004 and subsequently engaged in a variety of hisp activities including research, software development, customization, and implementation in various countries in africa and asia (including the country of the case). the second author, also a member of hisp uio, was involved in the analysis of the case and the discussion related to bootstrapping. the third author has worked in the case country on several projects including his most recent post as a consultant for the second phase of the medical licensing project. through his relationship with an it specialist from the ministry of health in the case country, the first author was encouraged to build a prototype assisting medical licensing processes based on the open source software platform, namely district health information software version 2 (dhis2). the prototype was subsequently endorsed by the moh, donors, and provincial health departments, and continued to grow to eventually become a national health ii. parts of the empirical data used in this research come from the daily interactions between the authors and their ongoing participation in the activities in this project. data were collected from formal and informal interviews with staff from the moh, provincial health departments, hospitals, and a donor. observations were made during project activities such as discussions, meetings, workshops, conferences, field visits to provinces/hospitals, user support, software coding, documents writing, and system deployment. consent to participate in these research activities from participants during meetings and workshops was explicitly sought. when meeting new participants, the first author clearly introduced himself as a researcher who was doing research for his phd as well as a participant in the software development process. emails were also used to inform about this participatory approach to research. sixsmith and murray (2001) have discussed how the ethical considerations on obtaining consent before using emails, forum postings, and other electronic internet archives australasian journal of information systems nguyen, nielsen & braa 2017, vol 21, research article scaling information infrastructures: medical licensing system 6 are still contentious in the research community. some argue that forums and other internet posts are in the public domain, and thus can be used for research purposes without consent. others believe that consent is also needed in such cases. however, seeking permission for using internet data can create other ethical issues, as participants might prefer not to discuss aspects of the previous data collected on them (sixsmith and murray, 2001). in our case study, emails and other electronic data used were mainly private communications among team members and external stakeholders. all of them were aware that the first author had a research agenda. such data, however, were also used with caution, and privacy and confidentiality of the participants were strictly protected. quotations from interviews or other sources such as email, sms, or opinions stated during workshops or meetings were anonymized to protect participants’ privacy and confidentiality. table 3 below summarizes data sources used in this research. sources of data collection tools quantity conferences and workshops notes 5 field trips notes 10 emails exchanged between the authors and other team members electronic 1000+ formal and informal interviews conducted during field trips to provinces. key informants included medical licensing officers at province (10) and central level (2) and the donor’s staff (3). each interview lasted between 30 minutes and 1 hour. notes 15 archived documents such as instant messages, bug tracking, system documents, user manuals, short message service (sms), phone logs, photos, server logs, software source code, code version control logs, and other project-related documents such as circulars, decisions, laws, technical reports, and evaluation reports. electronic n/a table 3: summary of data sources it was relatively challenging to manage the data coming from multiple sources, and the collection process spanned a long period of time (between 2012 and 2016). to get a sufficient overview of what we had, key events were arranged in chronological order and subsequently grouped by themes. the concept of bootstrapping described earlier was used to guide the analytical process. initially, critical factors causing substantial impact to ii trajectory were identified. key conditions and important factors leading up to the bootstrapping activities were sought to serve as a basis for further analysis. finally, strategies were derived to be reused in other contexts. 4 how the medical licensing system was scaled in this section, we chronologically describe our case and analyse the process of scaling the licensing system. we identify events that contributed to these changes in the user base and discuss contextual factors that triggered such changes. 4.1 building a prototype for the licensing system in january 2012, the first author of this paper was approached by a doctor (hereinafter doctors) working for the country’s administration of medical services (cams) at the ministry of health (moh). considering the author an expert in dhis2, a web-based and open source software platform designed for the collection, aggregation, and visualization of routine health indicators, doctors wanted to discuss the possibility of reconfiguring the software to build a centralized system to support medical licensing activities at the national level in the country. based on this discussion, the author spent about one week rapidly building a prototype based on the dhis2 platform. this licensing prototype was a very simple system with basic forms australasian journal of information systems nguyen, nielsen & braa 2017, vol 21, research article scaling information infrastructures: medical licensing system 7 and functionality to register the minimum of information needed from a health professional, her qualifications and work experience. doctors was impressed by the prototype (a screenshot is shown in figure 1 below) and immediately shared it with his friends working in various licensing offices in the country to get their feedback. a couple of licensing officers responded and started to look at the prototype. although no licensing office began to use the prototype in daily work, this initial feedback from the licensing officers was very useful for improving various aspects of the prototype. figure 1: screenshot of the first version of the prototype 4.2 piloting the prototype in the first province in a national workshop organized by cams in april 2012, the first author had the chance to again meet a doctor (hereinafter doctorp) who, four years prior, had attended several training classes on dhis2 in the country. in these classes, the first author had been one of the instructors. doctorp was at this time the head of the medical licensing office in one southern province (hereinafter southprovince). during a break in the workshop, there was a tripartite conversation between doctors, doctorp, and the first author about the licensing prototype. based on this, doctorp became eager to pilot the prototype in his province in parallel with a paper-based system. based on the prototype, he began to do some data entry with real licensing applications submitted to his province. through that process, he sent back a list of comments regarding the prototype. these comments were in a wide range of categories from polishing the licensing system with proper formats related to fonts, color, and text, to adding more reports such as the list of health professionals under review, other minor bug fixing, and missing functionality. based on the comments, the prototype was revised and improved. this comment-revision cycle was repeated several times. finally, doctorp was able to convince the director of the southprovince health services to approve the prototype and allow the use of for processing licensing applications in the province. since that time, the prototype has been an official system used in southprovince. doctorp played the role of a super user who was in charge of training his colleagues to use the licensing system for their work. 4.3 expanding the pilot to five more provinces the pilot in southprovince produced very good outcomes. based on this, doctors successfully persuaded the director of cams (moh) to write an official letter to six provinces (southprovince and 5 new provinces), encouraging them to pilot the licensing system. these provinces were selected based on two criteria: staff at the licensing offices had a good relationship with doctors, and the selected provinces represented the country geographically. australasian journal of information systems nguyen, nielsen & braa 2017, vol 21, research article scaling information infrastructures: medical licensing system 8 in a parallel process, a program supported by the asiabank was planning a comprehensive information system for the licensing of health providers and health facilities, as well as additional features such as a system for patient complaints. given the ongoing pilot activities of the licensing system now being extended to 6 provinces, integrating the two initiatives (one by the team and one by a firm which would be selected through a tender) was suggested. it was further suggested that the pilot should be exactly that: the licensing system built by the team would be only an interim system for testing out functionalities and gathering requirements and nothing more than that. consequently, in january 2013 asiabank agreed to offer a small grant to support the development and implementation of the licensing system and pilots in the selected provinces were initiated. the provision of the grant came after a consultant from asiabank evaluated the licensing system and gave a positive comment: “… [the licensing pilot] has made a lot of progress and i think your system is a good starting point for registration and licensing of health professionals” (email archives). from the start, there were conflicting views on the role of the prototype system within the more comprehensive system which was planned. but an agreement was reached to allocate some limited funds for a pilot under the condition that the piloted system should be a temporary solution and only run until the winner of the planned bid for a new system would take over and incorporate the licensing system into their software platform. the strategy the team adopted to counter the prospect of only being a temporary solution was to work hard to scale up and become as relevant and as useful as possible. furthermore, the more time that elapsed before the planned bid was to be awarded, the better their chances for making their own project indispensable. to support the rapid expansion of the implementation of the pilots, six people with diverse backgrounds (it, public health, and accounting) were recruited. together with the first author and doctors, the new members formed a project team (from now referred to as the team) to support the customization and implementation of the licensing system in the selected provinces. 4.4 limited national implementation a few months after the pilot system had been functioning in six provinces, asiabank sent a group of consultants to the pilot provinces to conduct an evaluation of the implementations. the reports by the consultants were positive about the licensing system; the consultants also recommended that the moh officially implement the licensing system nationwide. however, cams employed a more conservative strategy by sending a letter to request 63 licensing offices in the country to enter the data of about 150 applications into the licensing system for testing purposes. in two national conferences about medical licensing organized by cams in march 2013, the team was allocated about two hours per conference to introduce the licensing system to delegates coming from all provincial licensing offices. there were many complaints about the design. for example, one delegate commented: “the complicated design of the system meant that it took nearly 2 hours to enter a single application into the system. is it really feasible to use this in an official capacity?” to address this design issue, the team converted the existing combo boxes into a hybrid between free-text boxes and autocomplete boxes. users then had two options: select an item based on a previous entry or type a new value. following the two conferences, the team worked to make the licensing system more generic, so the provinces would be able to import their legacy from excel files, and they sent a letter to convince all provinces to adopt it. even though the letter was only about a pilot, the team cleverly managed to continuously improve the licensing system so that it could fully support licensing officers’ daily work processes and mitigate some of their work burden. eventually, the pilot system was widely accepted by provinces because it was useful for their work. 4.5 licensing system expanded to support hospital users in the implementation expansion process, the team was very keen to contact all provincial health departments to remind them to enter the 150 applications as requested by the moh. some provinces consented, but some refused due to various reasons. some provinces used excel to process applications and they did not see the need to migrate to the new system. the australasian journal of information systems nguyen, nielsen & braa 2017, vol 21, research article scaling information infrastructures: medical licensing system 9 excel-based approach was easy to learn and flexible to use for a variety of tasks, such as keeping track of applications and printing licensing certificates, thus making it the top choice of licensing officers. others were really busy with their routine work and did not have time to learn to use the new system. apart from convincing the provinces of the benefits of having a shared database, the team proactively built a conversion tool based on php to read data from excel and construct sql statements to transfer data to the database. this helped to automatically import legacy data from microsoft excel or word formats into the licensing system. before being fed into the conversion tool, excel (or word) files needed to be standardized in terms of number and order of columns, formats of values, etc. in this way, the provinces could seamlessly switch to the new system without any loss or re-entry of data. in may 2013, there was an event that triggered the adoption of the licensing system in all provinces. a feature that allowed hospitals to submit applications to provincial licensing offices by entering them directly into the licensing system was introduced. this feature was initially developed to meet a request from southprovince. after successfully being introduced in southprovince, it was incrementally improved and later adopted by other provinces. this was beneficial to both hospitals and licensing offices. on the one hand, licensing offices could focus on processing applications rather than entering data. on the other hand, hospitals could have their staff’s applications processed faster. when introduced, more and more provinces began to use the licensing system to process applications, and within four months it became a de facto national system (see figure 3 below). figure 2: the number of licensing applications processed by time (year-month) in the country, hospitals were previously not involved in the licensing processes. after the national implementation was attained, the growth in the user base stopped. when the team enabled hospitals to register applications for their staff into the licensing system, adoption picked up again rapidly. this growth was primarily due to the fact that hospitals were highly motivated to use the licensing system because their staff could get licenses more quickly when they registered the applications themselves. in august 2014, asiabank approved terms of reference (tor) for a bid to procure a new and comprehensive system (hereinafter bigsystem) comprising many modules such as selfregistration for health professionals, health facility licensing, continuing medical education (cme), patient complaints, and business intelligence. the bidding process for the bigsystem lasted many months and was restarted several times due to various procedural problems. these delays benefited the team significantly. eventually, in june 2015, a local company (hereinafter bigfirm) won the bid and subsequently got a contract to build the bigsystem. after the bid, the team continued to support the licensing system and work with the bigfirm on issues related to integration between the licensing system and the new modules. the integration process is still going on. australasian journal of information systems nguyen, nielsen & braa 2017, vol 21, research article scaling information infrastructures: medical licensing system 10 5 analysis and discussion 5.1 contextual factors that shaped the bootstrapping of the licensing system 5.1.1 user motivation and knowledge the first user of the licensing system prototype was the head of the licensing office in a southern province in the country (doctorp). when considering the evolution of the licensing system, the trajectory would have been different if he hadn’t decided to pilot the system in his province. prior to the birth of the licensing system, doctorp built an access-based system to support the licensing work in the province. there were some limitations in this system which did not make him completely satisfied. therefore, when he heard about the pilot licensing system, he was eager to be the first user. the licensing office that doctorp worked for is located in a newly established province which has many industrial parks and stands out with its economic growth rate. the open governance of this province encouraged change, proficiency, and innovation, forming a conductive environment for doctorp’s personal decision on using a new system. the further expansion of the pilot was supported by the donor and the moh, and the list of pilot provinces was based on recommendations from the team. the main criterion for selection was the relationship between doctors and the heads of licensing offices in pilot provinces. this is very different from the original bootstrapping model which emphasizes the need to enrol users based on their motivations and knowledge. in our case at this stage, personal trust and relationships was much more important. 5.1.2 use area and situation the bootstrapping strategy is largely influenced by the use area and use situation. initiatives that support and improve current practices will more easily enroll new users. in our case, the nature of the licensing practices could be improved significantly with a computerized information system. a centralized database for health professionals was critical for human resource planning and prohibiting fraud in medical practice. that vision was powerful when it came to convincing top leaders and provinces to follow and adopt a cloud-based system, which partially contributed to the success of the bootstrapping process. 5.1.3 the nature of technology availability refers to the readiness and accessibility of the ii designers and the supporting team. in our case, the use of cloud-based technology has significantly reduced the burden of support when the implementation took place across the country. with a centralized database, it was very easy for the supporting team to access data and, for example, verify and investigate if there was a problem with a user account. the users felt that the distance between them and the supporting team was just a matter of a phone call. the work practices described in our case vary from province to province. some provinces follow a full-stack approach which relies on the software system for data processing in every step of the licensing process. some provinces export data entered by hospitals to excel files and process licensing applications from there, and a few provinces strictly follow iso standards and track the status of applications through each step of processing. at the national level, the focus is mainly on data aggregation and reporting, which only becomes useful when full coverage of data is attained. thus, the licensing system appeared to be flexible and configurable and easily adaptable to different situations. information systems for professionals are often complicated and require extensive training. the challenge for designers who follow the bootstrapping approach is how to make the system easy and simple to use, while simultaneously maintaining its ability to support complex work practices. the simplified design of the licensing system enabled rapid scaling based on largescale adoption over a short time. however, it came with a cost. the use of the hybrid text field which allowed both selecting values from a combo list and entering free text ended up causing significant problems in later phases when summary and aggregation of data was needed. thus australasian journal of information systems nguyen, nielsen & braa 2017, vol 21, research article scaling information infrastructures: medical licensing system 11 there should be a balance between standards and flexibility which can both support data analysis over time and enable the bootstrapping process. the monetary cost for adoption was almost zero. this relates to training and flexibility as mentioned above, but also the fact that no fee was incurred to buy the software or pay the support team. all costs were covered by the donor. even the computers, internet lines, scanners, and printers were bought and supplied to provinces by the project. 5.1.4 coordinating institutions coordinating institutions are only needed when the network grows to a certain scale. initially, the licensing system was piloted informally and on a small scale. the coordinating institutions (i.e., the moh) played a bigger role when it was necessary to legitimize the full-scale implementation, which was the only way to collect the data of the entire country. this legitimacy was essential to resolve the problem of “all or nothing” in hiss in developing countries (braa et al., 2007). this was critical when a very big and important province refused to use the system. however, following the pressure from the moh’s letter, that province finally agreed to share their database in order to integrate with the national system. in table 4 below, we summarize how different factors affected our case’s bootstrapping strategy: original factors (hanseth and aanestad, 2003) how these factors operationalized in our case user motivation and knowledge first users were recruited primarily based on their established personal relationships with members of the team. user motivation was also dependent on the surrounding environment in which they were located, i.e., fast-growth economics and open governance in pilot provinces. use area and situation there was a need to meet requirements of users at different levels even at an early phase. some functionality provided direct usefulness for licensing officers. but there were other factors such as reporting and data analysis which were powerful in convincing top health managers of the increase in quality, transparency and accountability in the health system. nature of the technology some technologies, such as weband cloud-based ones, are easier to support than others because a supporting team does not have to go to the customers’ site to provide support. there was a need to find a balance between simplicity and complex business processes. cost was not an issue to most users. provincial health departments and hospitals did not have to pay for software; rather they received the system and many other things for free. coordinating institutions coordination from governmental agencies such as the moh was minimal at the outset and during the entire process of scaling. however, it was crucial when it came to assuring national scaling, in that it was the governmental agencies that requested provinces use the system, even at a limited scale. table 4: contextual factors influencing the bootstrapping in our case 5.2 the success of scaling the licensing system through bootstrapping based on the analysis of contextual factors that shape the bootstrapping, we identify three interrelated strategies that helped the team successfully employ the bootstrapping strategy: technicaland boundary-spanning competence, political manoeuvring, and the building of a australasian journal of information systems nguyen, nielsen & braa 2017, vol 21, research article scaling information infrastructures: medical licensing system 12 large user base. while the technical competence is vital for building a good system, the political manoeuvring is important to scale and create a strong user base. this in turn becomes a powerful means for an actor to establish its system and retain its role. 5.2.1 building technicaland boundary-spanning competence the team’s technicaland boundary-spanning competence was harnessed to quickly produce something small yet immediately useful. instead of building the system from scratch, the team decided to build on top of dhis2, a platform proven successful in healthcare settings. with this starting point, various components of the system were configured and reconfigured in a way that offered a good experience from the start and for the first user. the reconfiguration and assembling of existing components were based on architectural knowledge in combination with other competences including use context sensitivity and business models (andersson et al., 2008; baldwin, 2010; henderson and clark, 1990). the expertise of the individuals in the team and the success of the dhis2 platform were also well known to key people in the provinces, triggering them to approach the team and actively join the pilot. the technicaland boundary-spanning competences that the team was able to leverage to construct appropriate means beneficial to its plan were not taken for granted. one member of the team has worked with dhis2 for a long time as a developer and implementer. another member was a core developer of dhis2 between 2007 and 2010 before he left to work for a private company. the team was also joined by a member who had 2 years of experience working with dhis2 in another asian country. apart from that, one member of the team responsible for implementation had more than one year working on customizing dhis2 for a hospital reporting for the moh. also, the team was mixed with members with both it and medical backgrounds and competences. apart from the participation of doctors, there was also another member with public health background working in the team. the formation of the team with experienced and diverse members was important to combine, retain, and circulate the technicaland boundary-spanning competence that was vital for developing a good software system and understanding the potential in its future use. 5.2.2 manoeuvring and leveraging political support the team’s collective competence in itself would not be enough to create a useful system without the assistance of other means over and above that. the initialization of the pilot through personal contacts of a moh specialist was critical for this endeavour. additionally, although the letters from cams and the moh were limited in scope and effectiveness—i.e., they requested the registration of only 150 doctors—the team managed to leverage these political linkages to increase their control over the system. instead of supporting provinces only in their registration of 150 applications, the team worked to allow for conversion of legacy data from excel and developed functionality for hospital users. these extra steps were done to make the licensing system more supportive for daily routines of licensing offices, and to eventually win the users’ support. 5.2.3 building a large user base we can clearly see that technicaland boundary-spanning competences are important in the effort to bootstrap a system. however, such competences are not necessarily enough to succeed in the political game, which is common in the public sector, particularly in developing countries. for example, sahay et al. discuss a case when an innovation in the healthcare sector in india was easily overthrown when a new health secretary was elected in a state of india and this new secretary favoured another system (sahay et al., 2009). although technical competence helps, it is not a guaranty for success if there are no other supporting structures which can act as a counterforce to political reconfiguration. in our case, the team was successful in cultivating a large, sophisticated and nationwide user base, and covering all levels of the health system. the team used this strong means (user base and level of use) to protect the licensing system from substitution, especially when new software vendors (bigfirm) were selected to build additional modules for the licensing system. australasian journal of information systems nguyen, nielsen & braa 2017, vol 21, research article scaling information infrastructures: medical licensing system 13 5.3 bootstrapping tactics based on the analysis and discussion above, we now summarize in table 5 the step-by-step tactics of bootstrapping used in our case, in relation with seven tactics outlined earlier. the aim is to highlight how these tactics are similar and different from the tactics suggested by iaccuci et al. (2002) presented earlier in this paper. tactics suggested by iacucci et al. (2002) tactics from our case use 1. useful even with a small number of other users. addition: different parts of a system can also be relevant for different users. therefore, users should be targeted accordingly. 2. start with motivated and knowledgeable users. addition: personal relationships and networks can be used to motivate use. also, consider users not only as individuals but also the institutions, organisations and provinces to which they belong. 3. use area that does not require organizational change. addition: while the development is driven by users’ needs, which was not based on justification of complexity and criticality, it is important to mindfully convince users to start with prioritized use cases. new: pursue and exploit indirect means to motivate use, such as laws, decrees and official dispatches; free supply of computers, printers, and other stationaries; payment for data entry. design 4. useful even without an existing installed base. addition: while a full-scale solution may be required for full value, the value for lower-level organizational should be exploited initially. 5. start with simple solutions. addition: open source software should be considered due to its flexibility and availability. web-based solutions should be considered to reduce maintenance and support costs. 6. reuse installed base. addition: installed base should also refer to user experiences and skills, not only software system; i.e., ms excel skills 7. use gateways. addition: consider developing temporary gateways which could be thrown away easily to allow maximum experimentation. table 5: amendment to bootstrapping tactics 6 conclusion although bootstrapping is a powerful means of scaling an ii, many contextual factors related to technology, use situation, and other institutions may affect the outcome. in this paper, we have discussed various factors and their influence on the bootstrapping process of the licensing system in the southeast asian country. we have extended the original model by adding various nuances to the four aspects of bootstrapping, which involve political support, cost of technology, and personal relationships, arguing that they are important aspects to be considered when using a bootstrapping strategy. we also enhanced the bootstrapping tactics based on the empirical data of our case. we believe these extensions could be useful in analysing and pursuing the process of scaling hiss in other countries. practically, ii designers who are involved in his projects in developing countries should follow a contextualized bootstrapping strategy to guide design and implementation. references aanestad, m. & jensen, t. b. 2011. building nation-wide information infrastructures in healthcare through modular implementation strategies. the journal of strategic information systems, 20, 161-176. australasian journal of information systems nguyen, nielsen & braa 2017, vol 21, research article scaling information infrastructures: medical licensing system 14 abouzahr, c. & boerma, t. 2005. health information systems: the foundations of public health. bulletin of the world health organization, 83, 578-583. andersson, m., lindgren, r. & henfridsson, o. 2008. architectural knowledge in interorganizational it innovation. the journal of strategic information systems, 17, 19-38. baldwin, c. y. 2010. when open architecture beats closed: the entrepreneurial use of architectural knowledge [online]. available: http://hbswk.hbs.edu/item/when-openarchitecture-beats-closed-the-entrepreneurial-use-of-architectural-knowledge [accessed 01/05/2016]. braa, j., hanseth, o., heywood, a., mohammed, w. & shaw, v. 2007. developing health information systems in developing countries: the flexible standards strategy. management information systems quarterly, 31, 381-402. braa, j., monteiro, e. & sahay, s. 2004. networks of action: sustainable health information systems across developing countries. management information systems quarterly, 28, 337-362. ciborra, c. 2000. from control to drift: the dynamics of corporate information infastructures, oxford, oxford university press. cordella, a. 2006. information infrastructure in action, london school of economics and political science. dahlbom, b. & janlert, l.-e. 1996. computer future. department of informatics, university of gøteborg, unpublished manuscript. grisot, m., hanseth, o. & thorseng, a. a. 2014. innovation of, in, on infrastructures: articulating the role of architecture in information infrastructure evolution. journal of the association for information systems, 15, 197-219. hanseth, o. 2002. from systems and tools to networks and infrastructures-from design to cultivation. towards a theory of ict solutions and its design methodology implications (unpublished manuscript) [online]. available: http://heim.ifi.uio.no/~oleha/publications/ib_isr_3rd_resubm2.html [accessed 01/05/2016]. hanseth, o. & aanestad, m. 2003. design as bootstrapping. on the evolution of ict networks in health care. methods of information in medicine, 42, 384-391. hanseth, o. & lyytinen, k. 2010. design theory for dynamic complexity in information infrastructures: the case of building internet. journal of information technology, 25, 119. hanseth, o. & monteiro, e. 1998. understanding information infrastructure (unpublished book) [online]. available: http://heim.ifi.uio.no/~oleha/publications/bok.html [accessed 01/05/2016]. hanseth, o., monteiro, e. & hatling, m. 1996. developing information infrastructure: the tension between standardization and flexibility. science, technology & human values, 21, 407-426. henderson, r. m. & clark, k. b. 1990. architectural innovation: the reconfiguration of existing product technologies and the failure of established firms. administrative science quarterly, 9-30. henfridsson, o. & bygstad, b. 2013. the generative mechanisms of digital infrastructure evolution. mis quarterly, 37, 907-931. herzhoff, j. d., elaluf-calderwood, s. m. & sørensen, c. convergence, conflicts, and control points: a systems-theoretical analysis of mobile voip in the uk. mobile business and 2010 ninth global mobility roundtable (icmb-gmr), 2010 ninth international conference on, 2010. ieee, 416-424. australasian journal of information systems nguyen, nielsen & braa 2017, vol 21, research article scaling information infrastructures: medical licensing system 15 hinh, n. d. & van minh, h. 2013. public health in vietnam: scientific evidence for policy changes and interventions. global health action, 6. iacucci, e., nilsen, p. & berge, i. 2002. bootstrapping the electronic patient record infrastructure. proceedings of the iris 25th edition of iris, denmark. jensen, t. b. 2013. design principles for achieving integrated healthcare information systems. health informatics journal, 19, 29-45. nielsen, p. & aanestad, m. 2006. control devolution as information infrastructure design strategy: a case study of a content service platform for mobile phones in norway. journal of information technology, 21, 185-194. nielsen, p. & sæbø, j. i. 2015. three strategies for functional architecting: cases from the health systems of developing countries. information technology for development, 118. rodon, j. & hanseth, o. 2015. stabilization and destabilization processes at work in digital infrastructures: the co-functioning of architecture and governance. working paper, university of oslo. sahay, s., monteiro, e. & aanestad, m. 2009. configurable politics and asymmetric integration: health e-infrastructures in india. journal of the association for information systems, 10, 399-414. sahay, s. & walsham, g. 2006. scaling of health information systems in india: challenges and approaches. information technology for development, 12, 185-200. sanner, t. a., manda, t. d. & nielsen, p. 2014. grafting: balancing control and cultivation in information infrastructure innovation. journal of the association for information systems, 15, 220-243. shapiro, c. & varian, h. r. 2013. information rules: a strategic guide to the network economy, harvard business press. sixsmith, j. & murray, c. d. 2001. ethical issues in the documentary data analysis of internet posts and archives. qualitative health research, 11, 423-432. sommerville, i., cliff, d., calinescu, r., keen, j., kelly, t., kwiatkowska, m., mcdermid, j. & paige, r. 2012. large-scale complex it systems. communications of the acm, 55, 71-77. star, s. l. 1999. the ethnography of infrastructure. american behavioral scientist, 43, 377391. star, s. l. & ruhleder, k. 1996. steps toward an ecology of infrastructure: design and access for large information spaces. information systems research, 7, 111-134. sæbø, j. i. 2013. global scaling of health information infrastructures: circulating translations. phd thesis, faculty of mathematics and natural sciences, university of oslo. woodard, c. j. 2008. architectural control points. third international conference on design science research in information systems and technology (desrist 2008). atlanta ga, may 2008. copyright: © 2017 nguyen, nielsen & braa. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems nguyen, nielsen & braa 2017, vol 21, research article scaling information infrastructures: medical licensing system 16 paper title (paper title style) australasian journal of information systems de silva, burstein & vogel 2017, vol 21, research on health information systems editorial 1 participatory health information systems: theory and applications special section guest editors: daswin de silva la trobe university australia. d.desilva@latrobe.edu.au frada burstein monash university australia. frada.burstein@monash.edu doug vogel harbin institute of technology china. isdoug@hit.edu.cn editorial the graduation of healthcare delivery from its foundational mainstay into patient-centred care has widespread implications for health and wellbeing. information systems and technology are front-runners in necessitating this transition. in its ideal form, patient-centred care is respectful of and responsive to individual patient preferences, needs, values and ensures that patient values guide all clinical decisions (epstein et al. 2010). in both its ideal and actual forms, preferences, needs and values connote the significance of information in enabling patient-centred care. participatory health information systems (phis) are emerging platforms to capture, manage and understand this body of information. they aim to empower a network of support, consisting of healthcare institutes, practitioners, caregivers, family members and patients, to communicate, collaborate and contribute towards health and wellbeing objectives. besides healthy outcomes and resource usage optimisation, phis facilitate the accumulation of multidimensional information on a broad spectrum of healthcare topics ranging from disease variations to patient lifestyle choices and behaviours. this information provides an entire body of knowledge that has an immediate impact on the advancement of medical and clinical research. hence, it is in the strategic interest of policymakers, healthcare industry, as well as patients to endorse research, development and evaluation of participatory health information systems that enable patient-centred care. to better understand the ground roots underpinning phis, it is pertinent to present a brief account of participation theory as reported in information systems (is) literature. traditional is participation theory (figure 1) is founded on the causal structure of contingency theory. it is assumed a causal association between participation and is success exists and it is controlled by contingencies such as task uncertainty or system novelty. australasian journal of information systems de silva, burstein & vogel 2017, vol 21, research on health information systems editorial 2 figure 1: traditional is participation model (mckeen et al. 1994) markus and mao (2004) presented the updated is participation theory which overcomes the likelihood of a single causal process relating participation to outcomes. they redefine system success as two interdependent elements, system development success and system implementation success. they declare participation activities have varied effects on these two major outcomes and determine three further theories linking participation to system success (buy-in, system quality and emergent interactions). gaps identified among these links led to the updated theory. foundations of the updated theory lie in: 1) the distinction of system success into two concepts: system development success and system implementation success, with emergent reciprocal relations between them; 2) the description of groups of actors including stakeholders where participants are a subgroup, and change agents where is specialists are a subgroup; 3) a reformulated behavioural concept of participation activities, characterized in terms of type and richness, methods and conditions; and 4) the hypothesis of emergent causal processes (markus and mao 2004). updated is participation theory is illustrated in figure 2. figure 2: updated is participation theory (markus and mao 2004) taking into account that participatory information systems contribute towards increased stakeholder activity, increased systems adoption and enhanced health and wellbeing outcomes, this special section aims to advance research, development and evaluation of such systems by providing insights into emerging areas of research. the guest editors are pleased to announce a total of seven research articles focusing on a broad range of topics that reflect the diversity of participatory health information systems. markus and mao (2004) updated is participation theory is featured prominently across these research articles. the separation of system success into development and implementation success, expansion of participants into australasian journal of information systems de silva, burstein & vogel 2017, vol 21, research on health information systems editorial 3 stakeholders and change agents with stakeholders assuming a variety of roles in system success, are some of the notable endorsements of the updated theory. each article included in this special section focuses on a different stakeholder within the participatory healthcare delivery setting. a brief account of the seven articles follows. article 1: exploring mhealth participation for emergency response communities (david schwartz, abdelouahab bellou, luis garcia-castrillo, antonella muraro and nikolaos papadopoulos) this article explores a unique challenge in mhealth (mobile healthcare information systems); the active participation in emergency events by members of emergency response communities. as noted by authors, interventions in such environments are primarily short-term interactions which require clear and precise decision-making and constant tracking of potential participants in responding to an emergency medical event. the increasing prevalence of chronic conditions is a strong indication of the importance of creating emergency response communities with the support of mhealth applications to reshape society’s response to individuals in medical distress, specifically when conventional emergency medical services are not available immediately. article 2: can health 2.0 address critical healthcare challenges? insights from the case of how online social networks can assist in combatting the obesity epidemic (nilmini wickramasinghe, carolin durst and janine viol) a further exploration of the dynamics of mobile technology in healthcare, this article investigates the potential of online social networks and social media to address the obesity epidemic. authors design, implement and test the calorie cruncher, a health 2.0 application, to explore the influence of online social networks on individual’s health-related behaviour. the research unravels three forms of influence, an individual’s norms and value system, social control and pressure of social connections as well as social relationships that provide emotional support. it further provides a platform to understand the effect of obesity propagation in online social networks, identify potential online intervention strategies to disseminate health-related information to the right group of people, offer services fostering positive health-related behaviours and promote behaviour change advice. article 3: how do the participatory design features of health hackathons contribute to participatory medicine? (karen day, gayl humphrey and sophie cockroft) hackathon is a relatively new and emerging concept for participatory development of technology solutions or prototypes. in this article, authors present a descriptive case study approach focusing on the role of hackathons in participatory medicine. participation in design and development of a system/solution encompasses the strategic advantage of postimplementation participation. following interviews, observations, twitter feeds and presurveys during weekend-long hackathons, authors identify a number of success factors, group size, maturity of the idea, level of involvement of a mentor and involvement of students. furthermore, five skills of successful health hackathon participants have been recognised. they are knowledge, patient-focus, analytical skills, software design skills and professional perspective. article 4: a participatory model for multi-document health information summarisation (dinithi nallaperuma and daswin de silva) continuing on the importance of participation in design and development, this article presents a new model for multi-document health information summarisation that takes into account the role of end-user participation. retrieval of relevant and reliable health information is becoming a formidable challenge with increasing volumes of resources available for perusal via the internet. although information retrieval needs of health information consumers have been appropriately addressed by a substantial body of research literature, authors highlight the need for a multi-document information summarisation approach to effectively peruse the increasing volume of online content. the proposed model integrates both extractive and abstractive summarisation processes with continuous participatory inputs to each phase. a australasian journal of information systems de silva, burstein & vogel 2017, vol 21, research on health information systems editorial 4 prototypical implementation of the model was evaluated by both domain experts and health information consumers, with results confirming the generation of relevant and accurate summaries for diverse audiences. article 5: an examination of the mediating role for a nursing information system (lemai nguyen and nilmini wickramasinghe) mediation is a vital element in participatory healthcare. it is important for all healthcare practitioners to mediate/intervene as and when appropriate. in this paper, authors report findings from an examination of a nursing information system using activity theory. the nursing information system provides real-time nursing documentation in acute care hospital contexts. its objective is to enable superior nursing care by providing nurses with the opportunity to document patient care data into a tablet computer located at the patient bedside. the system was evaluated in a not-for-profit acute care hospital ward during its implementation. analysis of nurse interactions and perceptions showcase positive attitude towards the system and identified potential mediation capabilities and opportunities. article 6: the decision-making role of the patient in localised prostate cancer treatment (luke l wang and weranja k.b. ranasinghe) prostate cancer is one of the most common non-skin cancers affecting an increasing number of individuals worldwide. treatment selection for prostate cancer is a complex process involving multiple factors. authors conduct a systematic literature review of patient participation and decision-making in the treatment selection process for localised prostate cancer in order to evaluate capacity for improvement. a majority of patients prefer an active or collaborative role in decision-making, highlighting the importance of participation in the treatment selection process. personal aspects of the process and its influence on the final decision remain inadequately explored. authors recommend future research should be combined with ongoing study of multidisciplinary care and individualised risk based decisionmaking tools. article 7: the effectiveness of sms reminders and the impact of patient characteristics on missed appointments in a public dental outpatient clinic (emilia bellucci, lasitha dharmasena, lemai nguyen, and hanny calache) focusing on operational aspects of participation, this article reports on the effectiveness of reminders delivered to a mobile phone as well as patient demographics on missed appointments in a public dental outpatient clinic. a rich dataset collected over a time period of 46 consecutive months from the public outpatient dental clinic is analysed using descriptive statistics and inferential statistics techniques to determine if patient characteristics predict non-attendance and to examine the associations between different variables. pre-reminder and post-reminder attendance was found to be statistical significant. demographic and appointment profile of patients less likely to attend are also reported. authors recommend clinics focus on eliciting patient profiles to better understand non-attendance instead of adopting a generalised approach to encourage attendance. acknowledgement – the guest editors are sincerely thankful to the authors for their original contributions to this emerging interdisciplinary area of research. we kindly acknowledge and appreciate the time and effort expended by all reviewers. finally, we thank a/prof john lamp for his continuous guidance and support from inception to completion of this special section. references epstein, r. m., fiscella, k., lesser, c. s., and stange, k. c. 2010. "why the nation needs a policy push on patient-centered health care," health affairs (29:8), pp. 1489-1495. markus, m. l., and mao, j.-y. 2004. "participation in development and implementationupdating an old, tired concept for today's is contexts," journal of the association for information systems (5:11), p. 14. australasian journal of information systems de silva, burstein & vogel 2017, vol 21, research on health information systems editorial 5 mckeen, j. d., guimaraes, t., and wetherbe, j. c. 1994. "the relationship between user participation and user satisfaction: an investigation of four contingency factors," mis quarterly, pp. 427-451. copyright: © 2017 de silva, burstein & vogel. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems foshay, yeoh, boo, ong & mattie 2015, vol. 19, pp. s37-s54 framework for analysing effectiveness a comprehensive diagnostic framework for evaluating business intelligence and analytics effectiveness neil foshay schwartz school of business st francis xavier university nfoshay@stfx.ca william yeoh department of information systems and business analytics deakin university william.yeoh@deakin.edu.au yee-ling boo school of business it and logistics rmit university yeeling.boo@rmit.edu.au kok-leong ong la trobe business school la trobe university kok-leong.ong@latrobe.edu.au david mattie technology support group st francis xavier university dmattie@stfx.ca abstract business intelligence and analytics (bia) initiatives are costly, complex and experience high failure rates. organizations require effective approaches to evaluate their bia capabilities in order to develop strategies for their evolution. in this paper, we employ a design science paradigm to develop a comprehensive bia effectiveness diagnostic (biaed) framework that can be easily operationalized. we propose that a useful biaed framework must assess the correct factors, should be deployed in the proper process context and acquire the appropriate input from different constituencies within an organization. drawing on the biaed framework, we further develop an online diagnostic toolkit that includes a comprehensive survey instrument. we subsequently deploy the diagnostic mechanism within three large organizations in north america (involving over 1500 participants) and use the results to inform bia strategy formulation. feedback from participating organizations indicates that the bia diagnostic toolkit provides insights that are essential inputs to strategy development. this work addresses a significant research gap in the area of bia effectiveness assessment. keywords: business intelligence and analytics (bia); diagnostic framework; bia strategy formulation 1 introduction business intelligence and analytics (bia) can be defined as “the techniques, technologies, systems, practices, methodologies, and applications that analyze critical business data to help an enterprise better understand its business and market and make timely business decisions” (chen et al., 2012). a recent gartner worldwide survey of it spending revealed that bia technology is consistently in the top three priorities of chief information officers (gartner, 2014). moreover, global bia-related software spending exceeds us$14 billion (gartner, 2014). while effective bia capabilities are essential to the success of modern organizations, their deployment is complex, expensive, time consuming and laden with risk (isik et al., 2011; s37 australasian journal of information systems foshay, yeoh, boo, ong & mattie 2015, vol. 19, pp. s37-s54 framework for analysing effectiveness wixom and watson, 2010). expenditures on these systems include technical infrastructure, packaged software licenses, personnel and training. typically, full implementation costs are measured in millions of dollars (yeoh and popovic, 2015). a number of bia-related maturity models exist today. however, they typically provide insufficient depth to for evaluating an organization’s bia effectiveness on an on-going basis. this is a significant gap in the existing research as bia capability deployment is an on-going process – you are never ‘done’ delivering bia capabilities as they co-evolve with organizational strategy. a key question that must be addressed as bia capabilities evolve is: how effective are current capabilities and most importantly what areas need improvement? this question is exceptionally difficult to answer without a robust monitoring and evaluation mechanism. as the management saying goes: “if you can’t measure it, you can’t manage it.” therefore, the aim of this paper is to apply a design science paradigm to develop a useful and practical bia effectiveness diagnostics (biaed) framework. based on this framework, we further develop an online diagnostic toolkit that can be deployed in many organizational contexts. the remainder of this paper is structured as follows. in the next section we review foundational literature and present the relevance and significance of this research. in section 3 we present a framework and associated diagnostic toolkit for bia capability assessment. the fourth section outlines the design science-based methodology employed for this research. in section 5, we summarize feedback from bia champions from the organizations in which we deployed the biaed framework and diagnostic instrument. in the paper’s final section, we present our conclusions and an assessment of our research in terms of how it conforms to hevner et al.’s (2004) design science guidelines. 2 related work the concept of capability maturity models has received considerable attention within the bia field. a capability maturity model helps “integrate traditionally separate organizational functions, set process improvement goals and priorities, provide guidance for quality processes, and provide a point of reference for appraising current processes” (chrissis et al., 2011). as a consequence, the maturity model concept has been applied to bia initiatives to support organizations that seek to understand the current state of effectiveness or ‘maturity’ of bia capabilities as a key input into defining measures to help organizations evolve to higher levels of effectiveness (thamir and theodoulidis, 2013). the utility of bia-specific models has been demonstrated in several studies including hawking (2011), gudfinnsson et al. (2015), fedouaki (2013), hausladen and hass (2014) and rivera and shanks (2015). these studies have shown how such models can help organizations maximize the benefits provided by bia. in the following subsection, we provide a comprehensive review of the existing bia capability maturity models, highlight gaps in the current research and identify the contributions of the present research. 2.1 review of business analytics capability maturity models numerous capability maturity models exist in the bia literature. they span both practical and theoretical endeavors from practitioners and academics. to evaluate current models, we have leveraged the classification scheme developed by thamir and theodoulidis (2013), which consists of technical, organizational and human aspects, as discussed below. 2.1.1 technical aspect of bia as presented in table 1, virtually all existing models (22 out of 25 models) include an assessment of the technical aspect of bia, which includes technical infrastructure, business intelligence tools and data architecture. it is rational to assess bia maturity from this perspective as technology provides the foundation for bia. it is essential for organizations to invest in technical capabilities for managing data assets, delivering high quality information and providing knowledge workers with bi tools that provide the correct access and decision support functionality. this being said, it is important to note that the maturity model proposed by williams and williams (2007) excludes a technical assessment as the authors (who are s38 australasian journal of information systems foshay, yeoh, boo, ong & mattie 2015, vol. 19, pp. s37-s54 framework for analysing effectiveness business partners of the data warehouse institute (tdwi)), recommend the use of tdwi’s maturity assessment tool for this purpose (rajteric, 2010). the remaining four models in table 1 do not consider the technical infrastructure/tools or data architecture in their approach. specifically, the impact-oriented bi maturity model of lahrmann et al. (2011) and the american sap user group’s model (hawking et al., 2010) clearly place more weight on bi tools (“application architecture” in the hawking et al. (2010) model). neither model provides a clear and explicit description of data architecture or technical infrastructure. similarly, chuah and wong (2012) and sas (2009) focus on data or information related aspects such as management, quality, policy and utilization but only vaguely refer to technical infrastructure and tools in their models. 2.1.2 human aspect of bia the human aspect includes bia-related skills and knowledge, management sponsorship, training/education and organisational culture. the majority of existing models (13 out of 25) agree on the criticality of human factors such as technical competency, experience and knowledge of bia as well as the level of engagement, commitment and support from management in sponsoring bia initiatives. comparatively, the training and culture dimensions have less coverage in these models, (only 6 of 25 models). it is worth noting that only two models – davenport and harris’ (2007) analytical capability maturity model and the cosic et al. (2012) biacmm– cover all four dimensions of the human aspect. three models exclude the human aspect in their model constructs: the american sap user group (hawking et al., 2010), bimm (smc, 2009) and organic bi evolution model (russell et al., 2010). notably, the russell et al. (2010) organic bi evolution model of is biologically inspired and attempts to mimic the evolution of life. this model focuses entirely on the technical aspect of bia and consists of four stages (conception, coalescence, saturation and diversification) with each stage described by five levels. the other reviewed models present one to three human factors. for example, the ebim model (tan et al., 2011), data warehousing stages of growth (watson et al., 2009), bi maturity model (williams and williams, 2007), infrastructure optimization maturity model (microsoft, 2007), bi maturity hierarchy (deng, 2007) teradata maturity model (miller et al., 2009), bi development model (sacu and spruit, 2010) present one human factor. other models include two dimensions, for instance, bimm (raber et al., 2012), bi maturity model for healthcare (brooks et al., 2015), enterprise data management maturity model (fisher, 2007), hewlett packard bi maturity model (hp, 2009), the impact-oriented bi maturity model (lahrmann et al., 2011) and the bi maturity model (stock, 2013) have taken both skills and sponsorship into account in their models. three models cover three human aspect dimensions: gartner’s maturity model (rayner and schlegel, 2008), sas’ information evolution model (sas, 2009) and the data warehousing process maturity model (sen et al., 2006). 2.1.3 organizational aspect of bia the 25 models we reviewed reveal a total of four organizational dimensions. in particular, 16 of the reviewed models cover analytical processes, 13 models include organizational structure, 15 models capture governance and 10 models embrace the concept of cost-benefit analysis. tdwi’s bi maturity model (eckerson, 2009) and stock’s (2013) bi maturity model cover all four organizational dimensions. conversely, the organic bi evolution model (russell et al., 2010) and bi maturity model for healthcare (brooks et al., 2015) do not explicitly address organizational factors in their models. the brooks et al. (2015) model includes vision and strategy as well as management engagement and support, but group these factors under the human aspect of bia. the remaining models cover one to three organizational dimensions as depicted in table 1. there are seven models that capture only one dimension, namely the ebim model (tan et al., 2011), data warehousing stages of growth (watson et al., 2001), data warehousing process maturity (sen et al., 2006), enterprise data management maturity model (fisher, 2007), s39 australasian journal of information systems foshay, yeoh, boo, ong & mattie 2015, vol. 19, pp. s37-s54 framework for analysing effectiveness infrastructure optimization maturity model (microsoft, 2007), analytical capability maturity model (davenport and harris, 2007) and bi maturity hierarchy (deng, 2007). tan et al. (2011), sen et al. (2006) and fisher (2007) include governance as part of their model design. furthermore, deng (2007), microsoft (2007) and watson et al. (2001) include cost-benefit analysis as a maturity model construct. the models proposed by hagerty (2006), sas (2009), smc (2009) and sacu and sprui (2010) embrace two dimensions: analytical processes and organizational structure. the remaining models have considered three dimensions in their model design. to summarize, our review of existing maturity models provides us with many bia maturity evaluation dimensions. however, we have also identified shortcomings in existing bia maturity models and subsequently propose to address significant research gaps in this particular area. no . bi maturity models with references maturity aspects and dimensions organizational human technical a na ly ti ca l p ro ce ss es o rg an iz at io n st ru ct ur e g ov er na nc e c os tbe ne fit s sk ill s tr ai ni ng sp on so rs hi p c ul tu re te ch ni ca l in fr as tr uc tu re / t oo ls d at a a rc hi te ct ur e 1 the bi maturity model (stock, 2013) x x x x x x x x 2 enterprise business intelligence maturity model (chuah and wong, 2012) x x x x x 3 impact-oriented bi maturity model (lahrmann et al., 2011) x x x x x x 4 american sap user group (hawking et al., 2010) x x x x 5 business intelligence development model (sacu and spruit, 2010) x x x x x 6 teradata’s bi and dw maturity model (miller et al., 2009) x x x x x x 7 tdwi’s bi maturity model (eckerson, 2009) x x x x x x x x 8 hewlett packard bi maturity model (hp, 2009) x x x x x x x 9 bimm steria mummert consulting (smc, 2009) x x x x x 10 gartner maturity model for business intelligence and performance management (rayner and schlegel, 2008) x x x x x x x x 11 sas information evolution model (sas, 2009) x x x x x x 12 business intelligence maturity hierarchy (deng, 2007) x x x x s40 australasian journal of information systems foshay, yeoh, boo, ong & mattie 2015, vol. 19, pp. s37-s54 framework for analysing effectiveness table 1. review of existing bia capabilities maturity models (adapted from thamir and theodoulidis, 2013) 2.2 research gap and contributions while researchers have proposed numerous bia maturity models, to date there has been limited research on how they should be operationalized, i.e., clearly defining how organizations can deploy such models to guide the ongoing maturation of their bia capabilities in a comprehensive and systematic way. prior studies have focused exclusively on bia model substance – i.e., the aspects of maturity to be assessed. previous studies pay little attention to two important factors, specifically, the process context for maturity assessment, and the constituencies (i.e. stakeholders) whose input must be sought. considering the process context is critical because bia assessments must seamlessly align with it and business strategy development processes. if assessments are done in isolation, results cannot be effectively leveraged to organizational advantage, regardless of the quality of the framework employed. further, an organization’s bia capabilities will have strengths and weaknesses, thus there is a critical need to understand the nature of these strengths and weaknesses as perceived by key stakeholder groups: individuals who use (bia end-users), provide technical support for (it personnel) and fund (senior management) bia initiatives. it is surprising that none of the models reviewed consider the process context and the constituencies at the operationalization level. therefore this research fills a significant research 13 analytical capability maturity model (davenport and harris, 2007) x x x x x x x 14 infrastructure optimization maturity model (microsoft, 2007) x x x x 15 enterprise data management maturity model (fisher, 2007) x x x x x 16 business intelligence maturity model (williams and williams, 2007) x x x x 17 data warehousing process maturity (sen et al., 2006) x x x x x x 18 amr research’s business intelligence / performance management maturity model (hagerty, 2006) x x x x x x 19 ladder of business intelligence (cates et al., 2005) x x x x x x 20 data warehousing stages of growth (watson et al., 2001) x x x x 21 biacmm (cosic et al., 2012) x x x x x x x x x 22 organic evolution of bi (russell et al., 2010) x x 23 enterprise bi maturity (ebim) model (tan et al., 2011) x x x x 24 bimm (raber et al., 2012) x x x x x x x 25 bi maturity model for healthcare (brooks et al., 2015) x x x x total number 16 13 15 10 13 6 14 10 22 22 s41 australasian journal of information systems foshay, yeoh, boo, ong & mattie 2015, vol. 19, pp. s37-s54 framework for analysing effectiveness gap by providing a more comprehensive bia effectiveness diagnostic (biaed) framework that considers people and process in addition to the substance of the assessment. along these lines, we employ a design science paradigm and develop a biaed framework that explicitly considers assessment content, process context and key constituents. drawing on the biaed framework, we further develop an online diagnostic toolkit. specifically, the toolkit includes a custom-developed software application that allows the creation and administration of specialized online survey instruments and an extensive library of survey categories and over 300 questions (available at: https://goo.gl/bmp18d). subsequently, we deploy and iteratively evolve the framework and toolkit in three north america-based organizations in order to validate their utility and practicality. in terms of research contributions, this paper addresses an important topic in the field of business intelligence and analytics, that is, the operationalization of effectiveness/maturity analysis. we develop a framework that represents a practical and useful approach for assessing the current state of bia capabilities and for using the assessment results to drive strategy development. we identify the stakeholder groups to be consulted (i.e., bia end-users, technical personnel and senior management), and concisely specify the informational input required from each group. from a practical perspective, this research provides an efficient approach and toolkit that can be readily operationalized by any organization to assess bia effectiveness in order to drive moving-forward strategies. 3 developing the organizational bia effectiveness diagnostic mechanism the following sub-sections outline and present the details of the biaed framework that is composed of three components: assessment dimensions, constituencies to be consulted and process context. 3.1 assessment dimensions an effective assessment of bia capabilities requires that the proper elements be analyzed. as presented in figure 1, we view the dimensions of bia maturity as falling into two broad categories of what we term “management levers” – factors that can be controlled and manipulated to increase bia effectiveness. we submit that effectively manipulating these levers, i.e., through the development of bia-related strategies and associated tactics positively influences bia system users’ perceptions of the value of the bia system in terms of its support for analysis and decision-making. high levels of perceived value leads to extensive use of the system (as measured by frequency and duration use as well as the level of emersion), which in turn provides individual and ultimately organizational benefits. given the difficulty of quantifying the organizational benefits that result from effective bia capabilities (benefits tend to be second and third order effects), our framework focuses on assessing perceived individual benefits as well as level of bia system use. s42 australasian journal of information systems foshay, yeoh, boo, ong & mattie 2015, vol. 19, pp. s37-s54 framework for analysing effectiveness figure 1. bia effectiveness assessment model based on our review of the extant literature on bia maturity models, we propose technical characteristics (sec 3.1.1) and facilitating conditions (sec 3.1.2) as the two primary categories of management levers. technical characteristics assess the nature/quality of the data, metadata, bi tools and applications as well as the technical infrastructure supporting bia. facilitating conditions span and add granularity to the organizational and human maturity aspects proposed by thamir and theodoulidis (2013) and include the following dimensions: organizational culture and management support, process integration, delivery methodology, skills and knowledge and it support and training. the dimensions of each category as well as the rationale explaining their necessity are detailed in the following two sections. 3.1.1 technical characteristics data quality: high levels of data quality, as measured by dimensions such as accuracy, completeness, clarity and relevance is a fundamental building block of bia systems. the measurement of data quality should be based on user perceptions (vs. absolute measures). this is due to the fact that users must perceive the data presented to them as being of sufficient quality to be useful to them in carrying out job functions data must ‘fit its purpose’ (wang and strong, 1996). metadata quality: metadata is best described as information about the information resources available from bia systems. metadata is an essential support for users who must understand the information available in order to trust and use it. the presence of metadata improves decision process efficiency as well as decision quality (shankaranarayanan et al., 2006). when metadata is not available, users have been observed to avoid using a bi system (watson and haley, 1998). foshay et al. (2014) propose that bia systems should provide four types of metadata: definitional, quality, lineage and navigational and that metadata ensures that users are knowledgeable of and comfortable with the information supplied by bia systems. tools and applications: bia systems must support a wide array of bia ‘use cases’, including improving overall corporate performance, improving existing and enabling new processes and competencies and supporting collaboration both internally and externally (huang et al., 2012). as such, the tools and applications (e.g., reports, models, dashboards and scorecards) enabling bia must meet a wide array of needs. further, bi tools must provide the correct functionality, level of access, interactivity and performance required by users (verkooij and spruit, 2011; sharma and djiaw, 2011). s43 australasian journal of information systems foshay, yeoh, boo, ong & mattie 2015, vol. 19, pp. s37-s54 framework for analysing effectiveness technical infrastructure: a key characteristic of effective bia systems is their ability to adapt to ever changing user needs and to scale rapidly in the face of increasing activity and complexity (thamir and theodoulidis 2013). this necessitates careful planning and investment and the deployment of a robust data and system architecture. data architecture refers to the manner in which data is organized and stored to support bia applications. effective data architectures provide a balance of central management and user-oriented data provisioning as well as robust models for master data management (mdm). system architecture focuses on the flexibility, reliability, suitability, and effectiveness of the tools and processes leveraged to build and maintain bi system infrastructure and include extract, transform and load (etl), data cleansing, metadata management and mdm tools and processes (olszak and ziemba, 2007; sen et al., 2012). 3.1.2 facilitating conditions organizational culture and management support: in order for bia to be effectively disseminated throughout an organization, an ‘information oriented’ culture is essential (marchand et al., 2000). this means that a fact-driven approach to decision making is accepted as a corporate norm and that appropriate enterprise-level financial investments are made in people (training and education), processes and technology to ensure that this is the case. further, the active and visible support of senior management is a key element of a mature information-oriented culture (popovic et al., 2012). organizational alignment and process integration: bia systems must provide organizations with an effective means to sense and respond to what is happening in both external and operational environments (kettinger and marchand, 2011). as such, organizations require robust analytic processes to be in place and aligned with strategy development and performance management efforts (hawking and sellitto, 2015; schläfke et al., 2012). further, bia must be embedded in in organizational business processes in order to monitor and measure organizational performance at multiple levels (sharma and djiaw, 2011; bucher et al., 2009). bia practices/development approach: the nature of the systems that support bia differs significantly from other types of information systems given their informational (vs. functional) focus. further, they include elements of both infrastructure (i.e., etl process development and data warehouse deployment) and user-focused applications (e.g., reports, dashboards, scorecards and cubes) (lahrmann et al., 2011). as such, organizations must develop strong capabilities in bia-specific practices (i.e., a process framework as well as tools and techniques) that address the uniqueness of bia system delivery and to ensure effective, consistent, and repeatable implementations (couture, 2013). bia-related skills and knowledge: organizations require both strong technical skills for the delivery of bia systems as well as business analytics skills on the part of business personnel (chuah and wong, 2011). as such, it is essential to understand the degree to which specific bia roles are defined within an organization and the training and education that is available to ensure that appropriate skills and knowledge are developed and enhanced (stock, 2013). data governance: data quality is a cornerstone of bia. as such, organizations require effective data governance processes, people and tools (e.g., metadata management, data profiling, etc.) to ensure that important data assets are formally managed throughout the enterprise (eckerson, 2007). project governance: many organizations suffer from a large backlog of bia-oriented projects. as such, it is essential that the bia resources be deployed in a way that ensures alignment with organizational priorities. this implies that there exists effective bia project governance: processes, decision frameworks, roles and tools to identify, define and prioritize bia projects in a manner that aligns solution delivery with business strategy and priorities (barrett and barton, 2006). s44 australasian journal of information systems foshay, yeoh, boo, ong & mattie 2015, vol. 19, pp. s37-s54 framework for analysing effectiveness 3.2 constituencies in order to obtain a true picture of the current state of bia effectiveness, we propose that input must be sought from three distinct groups (i.e. constituencies) within an organization: bia users, the technical team, and senior management. each group has a unique perspective that must be considered in order to understand the degree to which bia is having the desired impact within an organization. bia users are the consumers of the information resources provided by bia systems. they are organizational decision makers and the knowledge workers who support them. the technical team is responsible for the architecture, design, delivery and ongoing support of bia solutions. senior management personnel (who may also be bia users) provide funding and, perhaps more importantly, establish the organization’s culture regarding the importance of bia and data driven decision-making. an effective stakeholder engagement process employs a variety of information gathering techniques including surveys, interviews/focus groups and workshops. the informational inputs that should be solicited from each group are presented in table 2. evaluation dimension users technical team senior management technical characteristics data quality x x metadata quality x bia tools and applications x technical infrastructure x facilitating conditions organizational culture and management support x x x organizational alignment and process integration x x bia practices/development approach x x bia-related skills and knowledge x x x data governance x x project governance x x outcomes perceived value x x level of use x x table 2. information required from each constituency group as highlighted by table 2, input for a given evaluation dimension may be sought from multiple constituencies – e.g., from both bia users and the technical team. there are a number of reasons for this, but one key purpose is to determine if there is significant misalignment of views between groups. for example, the technical team may believe that data quality in bia systems is good, based on their empirical analysis. however, bia users may have a different perception based on the fitness-to-purpose of this data. where misalignment is identified, measures can be established to address. 3.3 process context it is important for organizations to regularly assess the effectiveness of bia capabilities in order to ensure that they evolve at an optimal pace (hawking and sellitto, 2015). such assessments cannot be standalone or academic exercises – an organization must be positioned to take action on the insights gained. this means that organizations must perform assessments in the proper context – i.e., that provided by a strategy development process, as outlined in figure 2. we propose that the first step in such a process is a thorough assessment of bia capability effectiveness. as previously stated, the assessment exercise must evaluate all relevant capability dimensions and ensure that input is collected from the proper constituencies. s45 australasian journal of information systems foshay, yeoh, boo, ong & mattie 2015, vol. 19, pp. s37-s54 framework for analysing effectiveness figure 2. bia strategy development process the assessment serves as a critical input into the second step of the process – goal setting and gap analysis. in this step, insights gained from the current state assessment are factored along with bia best practices and the organization’s it and business strategy to identify and prioritize bia goals over a specific time horizon and clearly identify gaps in current capabilities (as identified in step 1) that may hinder their achievement. with goals and gaps identified, the organization can develop its bia strategy and identify and prioritize specific initiatives to operationalize the strategy (step 3). in the final step, the group(s) responsible for bia can begin executing initiatives that operationalize strategy and move the organization towards its goals. we propose that bia initiatives fall into two broad categories bia application delivery and bia process and infrastructure development. the former category involves delivering new or significantly enhancing bia applications while the latter may have an array of goals including improving technical infrastructure, addressing data quality issues, increasing the skill and knowledge level of technical and business personnel and enhancing analytic and decision processes. our proposed process is iterative and ongoing after a period of time, the effectiveness of the initiatives undertaken must be measured by starting the process again through a bia effectiveness assessment (step 1). the information collected during the assessment must be actionable to be useful. this has significant implications for its characteristics. specifically, the unit of analysis must be clearly defined and the information captured regarding the unit of analysis must be sufficiently granular as to be directional. the unit of analysis refers to the specific bia ‘ecosystem’ that is to be assessed (i.e., bia applications, underlying infrastructure and business areas supported). in many large organizations a number of bia systems may exist, each at a different level of maturity. as such, when performing a maturity assessment, the scope of the assessment must be clearly defined and communicated to the constituents who will be participating. granularity refers to the level of detail captured. the data collected must play two roles. first, it must clearly describe the maturity or effectiveness of each assessment dimension on a well-defined scale or continuum. this allows progress to be benchmarked and measured over time. second, the data collected must provide actionable insights; it must help explain why the current state of a given assessment dimension exists. consider the following example. the quality of data, as perceived by bia system users, is an important feature or characteristic of a bia system. to assess the current state, data regarding user perceptions of data quality must be collected, with a focus on its fitness to purpose. further, where there are perceived quality issues, the nature of the defects must be identified so that they can be addressed. figure 3 presents a set of questions that serve both roles. the likert style questions allow perceptions of quality in the unit of analysis (e.g. a data warehouse) to be quantified. the guided, openended questions permit specific issues, and the impact of these issues, to be described. s46 australasian journal of information systems foshay, yeoh, boo, ong & mattie 2015, vol. 19, pp. s37-s54 framework for analysing effectiveness figure 3. survey question example 4 methodology this research employs hevner et al.’s (2004) design science paradigm an approach to scientific inquiry in which researchers seek to address specific organizational issues through the creation and evaluation of innovative it artefacts. design science has been an important paradigm in decision support systems (dss) research (arnott and pervan, 2012), an area in which this paper also belongs. according to hevner et al., the objective of design science is to create artefacts (broadly defined to include constructs, models, methods, and instantiations) that have utility. our research develops and evaluates two such artefacts: an organizational bia effectiveness diagnostic framework and an associated online diagnostic toolkit. our objectives are aligned with those of design science – to provide a practical and useful means to evaluate an organization’s bia capabilities in order to improve them. as proposed by hevner et al. (2004), there are seven fundamental guidelines for design science research required to understand the problem to be solved and to design and deploy an artefact to address the problem. the approach dictates that a) a new artefact should be created and b) that it should provide a novel solution to an important business problem in particular domain (e.g., dss) (arnott and pervan, 2012). a valid artefact requires exhaustive evaluation of its utility, quality and effectiveness in addressing a problem. any design science research must make a clearly defined contribution to the literature in the form of a new artefact, creation of foundational elements or methodological advances and must be developed and evaluated utilizing rigorous methods. developing optimal artefacts must be an iterative process. finally, it is essential to effectively communicate research outcomes to both technical and managerial audiences. table 3 depicts hevner’s (2004) guidelines for design science research and describes how our research process successfully conforms to these guidelines. s47 australasian journal of information systems foshay, yeoh, boo, ong & mattie 2015, vol. 19, pp. s37-s54 framework for analysing effectiveness guideline hevner et al.’s (2004) description mapping to this research guideline 1: design as an artefact design-science research must produce a viable artefact in the form of a construct, a model, a method, or an instantiation. our project designs and delivers two artefacts: a biaed framework and a comprehensive online diagnostic toolkit. guideline 2: problem relevance the objective of design-science research is to develop technologybased solutions to important and relevant business problems. this paper addresses a significant business issue: assessing the current strengths and weaknesses of bia capabilities, thus providing key insights to drive future strategy / investment that increases bia effectiveness. guideline 3: design evaluation the utility, quality, and efficacy of a design artefact must be rigorously demonstrated via well-executed evaluation methods. the framework and the diagnostic toolkit were designed and subsequently deployed and evaluated in three organizations (involving over 1500 participants). feedback was solicited from technical and business personnel and incorporated into our artefact design. guideline 4: research contributions effective design-science research must provide clear and verifiable contributions in the areas of the design artefact, design foundations, and/or design methodologies. the two designed artefacts (i.e. the biaed framework and the associated online diagnostic toolkit) represent the research contributions. our work furthers research in the area of bia effectiveness/maturity assessment by providing operationalizable artefacts designed to support bia strategy development. guideline 5: research rigor design-science research relies upon the application of rigorous methods in both the construction and evaluation of the design artefact. the two artefacts developed were rigorously constructed based on an extensive literature review and subsequently tested, deployed and evaluated within three organizations. guideline 6: design as a search process the search for an effective artefact requires utilizing available means to research desired ends while satisfying laws in the problem environment. the artefacts were grounded in existing literature then iteratively pilot-tested within participating organizations taking into consideration the practical concerns and constraints of each organization. guideline 7: communication of research design-science research must be presented effectively both to technology-oriented as well as management-oriented audiences. the biaed framework, the diagnostic toolset and the information gained through assessments were presented to technical and business stakeholders within participating organizations. table 3. the mapping of biaed framework to design science research guidelines (adopted from hevner et al., 2004) we initiated our research by conducting a comprehensive review of the existing literature regarding bia maturity and effectiveness assessment, as detailed sections 2 and 3. based on our review, we conceived the biaed framework and developed a preliminary edition of our diagnostic instruments. we then determined that the optimal approach to evaluate and optimize our design was through a series of organizational case studies. to this end, we approached three north american organizations, a canadian municipal government (i.e., the halifax regional municipality in nova scotia), a canadian government agency known as the property valuation services corporation (pvsc), and a large multi-national automobile manufacturer (to comply with the agreement made with this organization, the name of the company is kept confidential). table 4 presents an overview of each organization, as well as their bia assessment goals. s48 australasian journal of information systems foshay, yeoh, boo, ong & mattie 2015, vol. 19, pp. s37-s54 framework for analysing effectiveness organization number of participants organizational overview international automobile manufacturer 1050 this large international automobile manufacturer sought to evaluate the bia capabilities supporting all of their north american business units. the organization had previously made large-scale investments in bia tools, technology infrastructure and personnel. the organization looked to gain an understanding of key factors currently limiting bia adoption across business units in order to develop moving-forward strategies. halifax regional municipality 300 this municipality based in eastern canada had made limited investment in bia capabilities and wished to understand a) issues stemming from the lack of bia capabilities in order to build a business case for investment and b) input required to develop a bia strategy and roadmap. property valuation services corporation (pvsc) 150 this governmental agency, based in eastern canada, sought to understand the strengths and weaknesses of existing bia capabilities in order to develop a business case for future investment and a long term bia strategy. table 4. participating organization profiles. we approached a number of organizations with the goal of finding bia “champions” senior management personnel willing to advocate for a formal bia assessment process within their organization. for organizations willing to undertake an assessment project, we struck oversight groups composed of both business and it personnel. we attempted to staff the working groups with managers who a) oversaw the design and deployment of bia capabilities or b) had oversight of business units with significant bia needs. the oversight groups were critically important to the success of assessments they defined and clarified the specific goals for the assessment, identified and recruited participants and worked, during the execution of each assessment, to ensure high participation rates within targeted groups. many of the members of oversight groups subsequently played key roles in bia business case and strategy development activities. once the specific goals for the organization were defined, we developed organization-specific versions of online surveys and invited a small group of pilot participants (technical staff, bia users and senior business leaders) to complete the instrument designed for their role. at the first organization we engaged, the automobile manufacturer, the pilot group consisted of approximately 50 individuals. we found the feedback from the pilot participants to be extremely useful as it allowed us to address issues related to understandability, relevance and survey completion time requirements. typically, the pilot exercise resulted in a moderate degree of rework to the instruments. once the instruments were finalized, they were sent to all targeted participants. on completion of the assessment, we analyzed the results and packaged them for subsequent use in strategy development activities. 5 feedback from participating organizations at the conclusion of each assessment project, we engaged bia champions, namely the cio of the municipal government, the ceo of the pvsc, and the bia strategy development project manager of the automaker in semi-structured interviews in order to gain an understanding of the relevance and utility of our biaed framework and diagnostic instruments. overall, the feedback received was instructive and clearly indicated the usefulness of our artefacts. the project champions were unanimous in the view that the framework proved useful for examining the current state of bia capabilities and the diagnostic toolkit delivered quantitative and qualitative results essential for building investment business cases and for developing bia strategies. as stated by the ceo of pvsc “as part of the implementation of bia in pvsc, we conducted a diagnostic survey with staff to examine the current state of information management and its challenges. this s49 australasian journal of information systems foshay, yeoh, boo, ong & mattie 2015, vol. 19, pp. s37-s54 framework for analysing effectiveness tool proved to be extremely valuable in building the necessary business case we required to formally proceed with a bia initiative. i would highly recommend this as a key step in any organization’s bia journey.” this view was supported by the cio of halifax regional municipality who asserted that “the diagnostic instrument provided the foundation for hrm’s bia strategy and business case. it effectively identified and quantified the impacts of issues in our current environment related to manual data integration, data quality and other key areas. it provided actionable information that allowed us to identify and prioritize our actions moving forward.” the champions pointed out that the online diagnostic toolkit was useful for gaining a multiplestakeholder perspective on bia capabilities in the organization. further, the compilation and synthesis of qualitative data (i.e. respondent commentary) was found to be very useful and directional. in fact, the bia strategy development manager for the large auto manufacturer commented that, “initially, many in the it group were sceptical of the value of using surveys to support bia strategy. however, on completion of the project, there was universal agreement that the surveys provided tremendous value. the findings were surprising and fundamentally altered our perception of what the focus of our future bia strategy should be. the level of insight provided by the bia assessment toolkit could not have been obtained any other way.” also, the champions remarked that the framework was useful because it considered the technical team’s perspective on what they think end-user perceptions are. they believed that comparing what the technical team views user perceptions to be to actual perceptions often yielded interesting insights and could be employed to improve business-it alignment. 6 concluding remarks as organizations continue to make ever-larger investments in bia capabilities to support an expanding array of organizational decision making activities, effective and efficient approaches for bia capability assessment becomes an essential tool. leveraging a design science approach, this paper presents a practical and useful framework and toolkit for this specific purpose. per hevner’s first design science guideline, we developed two artefacts – a comprehensive bia effectiveness diagnostic (biaed) framework (i.e., a model) and an online diagnostic toolkit (i.e., an instantiation). the framework encompasses three main perspectives: the coverage of effectiveness dimensions; the process context for assessment; and the constituencies from whom input must be sought. this research addresses the problem of efficiently diagnosing the effectiveness of current state bia capabilities across critical dimensions from the perspective of multiple stakeholder groups which is a critical (unaddressed) problem for many organizations today, thus conforming to hevner’s second guideline problem relevance. we designed novel artefacts that have been informed by, and incorporate elements of, previously developed maturity models practically test their efficacy through large-scale deployment and evaluation in three organizations (hevner et al.’s guideline 5 – research rigor). the utility of the artefacts was evaluated in two ways – by piloting the deployment of survey instruments within each organization and through interviews with bia champions at the conclusion of each engagement (hevner et al.’s guideline 3 – design evaluation and guideline 5 – research rigor). participant feedback was and will continue to be used to refine the designed artefacts for future deployments. as such, this iterative “build-and-evaluate” loop conforms to hevner et al.’s guideline 6 – design as a search process. in this paper we present the framework and representative sample of the survey questions associated with the online toolkit. we have presented the framework and toolkit within each of the organizations that we have collaborated with and have received feedback from both technical and managerial personnel that the artefacts are comprehensible and relevant. we s50 australasian journal of information systems foshay, yeoh, boo, ong & mattie 2015, vol. 19, pp. s37-s54 framework for analysing effectiveness therefore believe that the results of our research are presented in such a way as to be informative to both technical and management audiences, thus conforming to hevner et al.’s guideline 7 – communication of research. the two artefacts presented in this paper represent our research contribution (guideline 4 – research contributions). the biaed framework and associated diagnostic toolkit provide organizations with a practical, effective means to evaluate current bia capabilities and to incorporate evaluation results into strategy development processes. we thus provide a fully formed operational model and practical tools for bia effectiveness / maturity assessment, which have not been addressed by prior studies. as for future research, we plan to continue to refine the biaed framework and the associated toolkit by deploying them within a variety of organizational contexts. a key goal for our future research is to deploy and evaluate the artefacts in geographies outside of north america to determine if the framework requires modification in order to be effective in different geographies and cultures. references arnott, d., & pervan, g. (2012). design science in decision support systems research: an assessment using the hevner, march, park, and ram guidelines. journal of the association for information systems, 13(11), 923-949. barrett, d., & barton, n. (2006) best practices in building a data warehouse quickly, business intelligence journal, 11(4), 37-45. brooks, p., el-gayar, o. & sarnikar, s. (2015) a framework for developing a domain specific business intelligence maturity model: application to healthcare, international journal of information management, 35, 337-345. bucher, t., gericke, a., & sigg, s. (2009) process-centric business intelligence, business process management journal, 15(3), 408-429. cates, j. e., gill, s. s. & zeituny, n. (2005) the ladder of business intelligence (lobi): a framework for enterprise it planning and architecture, international journal of business information systems, 1(1), 220-238. chrissis, m. b., konrad, m. & shrum s. (2011) cmmi for development: guidelines for process integration and product improvement, addison-wesley professional. chuah, m-h. & wong, k-l. (2011) a review of business intelligence and its maturity models.” african journal of business management, 5(9), 3424–3428. chen, h., chiang, r. h., & storey, v. c. (2012). business intelligence and analytics: from big data to big impact. mis quarterly, 36(4), 1165-1188. chuah, m-h. & wong, k-l. (2012) construct an enterprise business intelligence maturity model (ebi2m) using an integration approach: a conceptual framework. in: mircea, m. (eds.), business intelligence – solution for business development, intech, 1-14. cosic, r., shanks, g. & maynard, s. (2012) towards a business analytics capability maturity model, proceedings of the 23rd australasian conference on information systems, geelong, victoria, australia, 1-11. couture, n. (2013) best practices for adopting agile methodologies for data warehouse development, business intelligence journal, 18, 8-17. davenport, t. h. & harris j. g. (2007) competing on analytics: the new science of winning, cambridge, ma: harvard business school press. deng, r. (2007) business intelligence maturity hierarchy: a new perspective from knowledge management, information management, available at: http://www.informationmanagement.com/infodirect/20070323/1079089-1.html s51 australasian journal of information systems foshay, yeoh, boo, ong & mattie 2015, vol. 19, pp. s37-s54 framework for analysing effectiveness eckerson, w. (2007) tdwi benchmark guide: interpreting benchmark scores using tdwi’s maturity model, tdwi research. available at: http://tdwiorg0000.web711.discountasp.net/content/tdwi_benchmark_final.pdf eckerson, w. (2009) tdwi’s business intelligence maturity model, the data warehousing institute, chatsworth. fedouaki, f., okar, c. & el alami, s. (2013) a maturity model for business intelligence system project in small and medium-sized enterprise: an empirical investigation, international journal of computer science issues, 10(6), 61-69. fisher, t. (2005) how mature is your data management environment? business intelligence journal, 10(3), 20-26. foshay, n., taylor, w., & mukherjee, a. (2014) winning the hearts and minds of business intelligence users: the role of metadata, information systems management, 31(2), 167-180. gartner press release (2014) gartner says worldwide business intelligence and analytics software market grew 8 percent in 2013, stamford, conn., april, 29, 2014, press release. available at: http://www.gartner.com/newsroom/id/2637615 gudfinnsson, k., strand, m. & berndtsson, m. (2015) analyzing business intelligence maturity, journal of decision systems, 24(1), 37-54. hagerty, j. (2006) amr research’s business intelligence/performance management maturity model, version 2, available at: http://www.eurim.org.uk/activities/ig/voi/amr_researchs_business_intelligence.pd f hausladen, i. & hass, a. (2014) a joint maturity model of bi-driven supply chains, proceedings of 14th international scientific conference on business logistics in modern management, osijek, croatia, 97-108. hawking, p., jovanovic, r. & sellitto, c. (2010) business intelligence maturity in australia, victoria university erp research group. hawking, p. (2011) business intelligence excellence: a company’s journey to business intelligence maturity, proceedings of the 17th americas conference on information systems, detroit, michigan, usa, 1-9. hawking, p., & sellitto, c. (2015) business intelligence strategy: a utilities company case study, international journal of enterprise information systems, 11(1), 1-12. hevner, a. r., march, s. t., park, j. & ram, s. (2004) design science in information systems research, mis quarterly, 28(1), 75-105. hewlett packard (2009) the hp business intelligence maturity model: describing the bi journey, available at: http://www.computerwoche.de/fileserver/idgwpcw/files/1935.pdf huang, t. c., liu, c., & chang, d. (2012) an empirical investigation of factors influencing the adoption of data mining tools, international journal of information management, 32(3), 257-270. isik, o., jones, m. c., & sidorova, a. (2011) business intelligence (bi) success and the role of bi capabilities, intelligent systems in accounting, finance and management, 18(4), 161-176. kettinger, w. j. & marchand, d. a. (2011) information management practices (imp) from the senior manager’s perspective: an investigation of the imp construct and its measurement, information systems journal, 21(5), 385-406. s52 australasian journal of information systems foshay, yeoh, boo, ong & mattie 2015, vol. 19, pp. s37-s54 framework for analysing effectiveness lahrmann, g., marx, f., winter, r. & wortmann, f. (2011) business intelligence maturity: development and evaluation of a theoretical model, proceedings of the 44th hawaii international conference on system sciences, poipu, kauai, hawaii, 110. marchand, d. a., kettinger, w. j. & rollins, j. d. (2000) information orientation: people, technology and the bottom line, sloan management review, 41(4), 69-80. microsoft (2007) business productivity infrastructure optimization campaign. miller, l., schiller, d. & rhone, m. (2009) data warehouse maturity assessment service, teradata, 1-15. available at: http://www.teradata.com/assets/0/206/276/3457d45f7327-4a36-b1dc-2e5daae3d269.pdf olszak, c. & ziemba, e. (2007) approach to building and implementing business intelligence systems, interdisciplinary journal of information, knowledge, and management, 2(1), 135-148. popovic, a., hackney, r., coelho, p. s., & jaklic, j. (2012) towards business intelligence systems success: effects of maturity and culture on analytical decision making, decision support systems, 54(1), 729-739. raber, d., winter, r. & wortmann, f. (2012) using quantitative analyses to construct a capability maturity model for business intelligence, proceedings of the 45th hawaii international conference on system sciences, grand wailea, maui, hawaii, 4219-4228. rajteric, i. h. (2010) overview of business intelligence maturity models, management international journal of contemporary management issues, 15(1), 47-67. rayner, n., schlegel, k. (2008) maturity model overview for business intelligence and performance management, gartner, stamford. available at: https://www.gartner.com/doc/842813/maturity-model-overview-business-intelligence rivera, d. s. & shanks, g. (2015) a dashboard to support management of business analytics capabilities, journal of decision systems, 24(1), 73-86. russell, s., haddad, m., bruni, m. & granger, m. (2010) organic evolution and the capability maturity of business intelligence, proceedings of 16th americas conference on information systems, lima, peru, 1-9. sacu, c. & spruit, m. (2010) bidm: the business intelligence development model, proceedings of the 12th international conference on enterprise information systems, funchal, madeira, portugal, 288-293. sas (2009) information evaluation model, available at: http://www.sas.com/software/iem/ schläfke, m., silvi, r., & möller, k. (2012) a framework for business analytics in performance management, international journal of productivity and performance management, 62(1), 110-122. sen, a., sinha, a. & ramamurthy, k. (2006) data warehousing process maturity: an exploratory study of factors influencing user perceptions, ieee transactions on engineering management, 53(3), 440-455. sen, a., ramamurthy, k., & sinha, a. p. (2012) a model of data warehousing process maturity, ieee transactions on software engineering, 38(2), 336-353. sharma, r. s., & djiaw, v. (2011) realising the strategic impact of business intelligence tools. vine, 41(2), 113-131. shankaranarayanan, g., even, a. d. i. r., & watts, s. (2006) the role of process metadata and data quality perceptions in decision making: an empirical framework and investigation. journal of information technology management, 17(1), 50-67. smc (2009), steria mummert consulting ag, available at: http://www.nomina.de/cognos/pdf/1s017_co.pdf s53 australasian journal of information systems foshay, yeoh, boo, ong & mattie 2015, vol. 19, pp. s37-s54 framework for analysing effectiveness stock, p. (2013) the business intelligence maturity model: describing the bi journey, available at: http://www.young-blood.co.za/index.php/2013-02-10-10-55-36/miningand-operations/item/20-bi-maturitymodel tan, c. s., sim, y. w., & yeoh, w. (2011). a maturity model of enterprise business intelligence. communications of the ibima, 2011(417812), 1-11. thamir, a., & theodoulidis, b. (2013). business intelligence maturity models: information management perspective. in: skersys, t., butleris, r. and butkiene, r. (eds), international conference on information and software technologies (icist), communications in computer and information sciences (ccis), 403, springer, 198221. verkooij, k., & spruit, m. (2013) mobile business intelligence: key considerations for implementations projects, journal of computer information systems, 54(1), 23-33. wang, r. y., & strong, d. m. (1996) beyond accuracy: what data quality means to data consumers, journal of management information systems, 12(4), 5-33. watson, h. j., & haley, b. j. (1998) managerial considerations, communications of the acm, 41(9), 32-37. watson, h., ariyachandra, t. & matyska, jr, r. j. (2001) data warehousing stages of growth, information systems management, 18(3), 42-50. watson, h. (2013) all about analytics, international journal of business intelligence research, 4(1), 1-16. wixom, b. & watson h. (2010) the bi-based organization, international journal of business intelligence research, 1(1), 13-28. williams, s. & williams, n. (2007) the profit impact of business intelligence, morgan kaufmann publishers, san francisco. yeoh, w., & popovic, a. (2015). extending the understanding of critical success factors for implementing business intelligence systems. journal of the association for information science and technology. doi: 10.1002/asi.23366 copyright: © 2015 foshay, yeoh, boo, ong & mattie. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. s54 paper title (paper title style) australasian journal of information systems mckay 2018, vol 22, research on educational technologies prescriptive training courseware 1 prescriptive training courseware: is-design methodology elspeth mckay rmit university elspeth.mckay@rmit.edu.au abstract information systems (is) research is found in many diverse communities. this paper explores the human-dimension of human-computer interaction (hci) to present is-design practice in the light of courseware development. assumptions are made that online courseware provides the perfect solution for maintaining a knowledgeable, well skilled workforce. however, empirical investigations into the effectiveness of information technology (it)-induced training solutions are scarce. contemporary research concentrates on information communications technology (ict) training tools without considering their effectiveness. this paper offers a prescriptive is-design methodology for managing the requirements for efficient and effective courseware development. to develop the methodology, we examined the main instructional design (id) factors that affect the design of it-induced training programs. we also examined the tension between maintaining a well-skilled workforce and effective instructional systems design (isd) practice by probing the current id models used by courseware developers since 1990. an empirical research project, which utilized this is-design methodology investigated the effectiveness of using it to train government employees in introductory ethics; this was a study that operationalized the interactive effect of cognitive preference and instructional format on training performance outcomes. the data was analysed using rasch item response theory (irt) that models the discrimination of people’s performance relative to each other’s performance and the test-items’ difficulty relative to each test-item on the same logit scale. the findings revealed that is training solutions developed using this is-design methodology can be adapted to provide trainees with their preferred instructional mode and facilitate cost effective etraining outcomes. keywords: information systems (is)-design; instructional design; instructional systems design (isd); prescriptive courseware is-design; workplace training; first principles of instruction; goal-based training education; component display theory (cdt); learning theory; training assessment; item response theory (irt); rasch model 1 introduction keeping up with information technology (it) and communications are key challenges facing the business sector (ma, fildes, and huang 2015) and leveraging it is critical to business success (benac 2015). even though substantial investments are continually being made, business leaders complain about the time it takes to implement efficient and effective it solutions. when their long-awaited training product finally arrives, it is not agile enough to keep pace with rapidly evolving strategic business goals (rohde et al. 2016). government agencies also have an interest in maintaining well-skilled and knowledgeable employees, and face the same problems (d'amours and legault 2013). training expenditure varies worldwide (görg and strobl 2006). for instance, in 2013 (north america) corporations and government spent us$141.7billion on training; this equates to us$7 for every us$1,000 in company revenues (harward 2014). north america and europe australasian journal of information systems mckay 2018, vol 22, research on educational technologies prescriptive training courseware 2 are amongst the top spenders, with asia and india outspending australia, south america and africa (harward 2014). the global market for training reached us$306.9billion in 2013 (harward 2014) (see table 1). country us$billion percentage of global market north america 141.7 46 europe 89.0 29 asia 31.0 10 india 21.5 7 australia 9.2 3 south america 6.3 2 africa 3.6 1 rest of world 4.6 1 table 1 global training expenditures, 2013 many factors influence organizations’ spending on training, but the most important are their size and the industry in which they operate. in 2013, organizations with fewer than 500 employees spent an estimated average of us$1,888 per employee, with midsize organizations (500 to 9,999 employees) spending us$838 and large organizations (10,000-plus employees) with similar training expenditure (miller 2014). it is more difficult to find accurate government agency training metrics. government agency training data typically appear in report appendices, with sketchy details of the organizations involved and little detail (shelly and jackson 2009). this is because such training practice metrics, are distributed among the partnerships that develop between technical and vocational education and training (tvet) organizations, private industry operators and government enterprises (spencer 2014);(acci 2011);(flag 2007 july). marketing companies link online it training environments to the currently popular concept of knowledge management, thereby attracting government agency sponsorship of their digital products. this understandable attraction results in the propensity for non-government training operators to cash in on the procurement of government web-services and media support (mena 2015). in the government sector, there is often no agreement amongst the various stakeholders as to which one takes on ict training resource ownership (mckay et al, 2007). finding the right solution for ict-based training is complicated by the multi-disciplinary mix of expertise that is required to synthesize elearning governance and training implementation. after all, these decisions involve the diverse nature of is practice such as: electronic commerce; global and societal; hci; information storage and retrieval; knowledge-based systems; management information systems; and systems and software engineering (burgess, grimshaw, and shaw 2017). firstly, training material should be made available to all employees and training providers who have access to a government agency's intranet. secondly, online training materials should be available continually, so that employees can access their training materials when needed. thirdly, the elearning/training solutions should have different trainee skill development pathways so that novice employees can go through the full learning program in a sequential (skill-building) order (mckay & izard, 2015a), and more experienced employees can select relevant training modules as refreshers (vann 2015);(chatzimouratidis, theotokas, and lagoudis 2012);(vignan et al. 2011). australasian journal of information systems mckay 2018, vol 22, research on educational technologies prescriptive training courseware 3 efficient delivery of effective training programs across a diverse and widely distributed workforce is a major challenge. for example, in the australian state of victoria in 2006/7, over 2000 department of human services staff members engaged in finance-related activities required training (mckay et al, 2007). providing such a large workforce with traditional faceto-face training sessions for a new procedure or legislative requirement is logistically difficult and very time-consuming. in addition, traditional modes of training service struggle to meet the growing requirement for flexibility in terms of immediacy of access and a more traineecentred approach. new employees, or someone recently transferred to a new position, frequently have an immediate need to learn specific administrative processes. however, government training courses are often filled several months in advance, forcing the employee to obtain one-on-one tuition or learn the new skill/procedure by trial and error, neither of which is efficient for the organization. conversely, an employee may be sent on a training course to meet organizational mandates or to meet the selection criteria for a future position, without immediately putting the new skills or procedures learned into practice; this means key competencies achieved during the training are forgotten, resulting in an unsatisfactory training outcome for both the trainee and the organization (chatzimouratidis, theotokas, and lagoudis 2012). often face-to-face training courses have difficulty catering for individual trainees’ needs due to short training timeframes and large volumes of instructional material; little wonder that understanding new administrative tasks overwhelms first-time (novice) learners (bagley and heltne 2003). as a consequence, they have difficulty in coping with technical details, limitations of the formalized training timeframe, and peer pressure. more experienced staff previously exposed to the subject matter (or instructional content) react unfavourably to the presentation of simple concepts and repetition of familiar content, becoming frustrated and bored. neither of these learner groups will achieve optimized learning outcomes for themselves or the organization (vignan et al. 2011). some australian government initiatives, such as tvet, successfully negotiate the it training services divide between the government and corporate sector. non-government training operators can realize their remunerative expectations in securing government training contracts. however, these lucrative arrangements appear to treat workforce training as oneoff stand-alone skills development exercises. there is a near-endless supply of off-the-shelf it training packages. deciding which one is suitable is not easy and there can be no guarantee of employees' reactions. as a standard goal of training is to boost workplace knowledge, ideally profiling people according to age, gender and experience would be used to benchmark expectations. however, generic it training programs are frequently put in place without adequate evidence of their suitability; one remedy is to design and build customised it training solutions which encourage learning while doing (kalraiya and buddhdev 2015);(schank 2002). schank (2002) says this approach may become a means to an end, and it works well if there is room to let the trainees fail without any significant negative consequences for the organization. however, learning by doing can be dangerous, for instance in a healthcare environment where health and safety are core organizational goals, and it can be expensive, because people learn at different rates and failing to provide sufficient relevant cases can prevent trainees from experiencing all they need to learn (schank 2002). australasian journal of information systems mckay 2018, vol 22, research on educational technologies prescriptive training courseware 4 rosenberg (2001) was an early advocate for organizations to adopt web-based business strategies to encourage strong partnerships with it and workplace learning (rosenberg 2001); more recent scholars argue that his ideas are equally relevant in today's business and government training environments (ballew et al. 2013). however, business adoption of webbased training solutions is negatively impacted by the instability of internet service providers. according to rosenberg (2001, p.152), even "...if you can get the web to everyone, but your infrastructure is unreliable (always going 'down'), or you're using an unreliable internet service provider (isp), your users will become very frustrated, very fast." to reiterate, there are not many is-design theories that have been developed for online communities per se (spagnoletti, resca, and gwanhoo 2015, schofield 2002). the is literature contains little research dealing with instructional systems design (isd) pertaining to it and government skills training courseware development. instead, most researchers address the creation of instructional experiences with it; using old id models to make their it training materials look appealing (appendix-1: id model review). in this paper we seek to fill this gap by exploring a prescriptive courseware is-design methodology, through a research study that investigated the interactive effect of instructional format and cognitive preference on government sector training performance outcomes. we examined the main instructional design (id) factors which affect the efficiency and effectiveness of it-induced training programs (docq 2015) (schank 2002). we argue that drawing on the first principles of instruction (merrill 2002a) will empower the courseware developer to design and implement effective online courseware that meets user/trainee expectations. simultaneously, we argue that the component display theory (cdt) (merrill 1987) remains relevant to implementing current it-induced training programs (sims 2014), as it supports our prescriptive courseware is-design methodology that novice courseware developers can follow to customize training courseware programs to target specific training requirements. this paper presents the theoretical background required to develop a prescriptive courseware is-design methodology, incorporating the first principles of instruction (merrill, 2002a), and goal-based training. then there is a review of current id practice and models which illustrate that the original id principles do not align well enough with modern ict and web 2.0 applications. to illustrate the effectiveness of the prescriptive courseware is-design methodology, there is an example of an empirical research study that used our is-design approach and quantified their instructional outcomes using the rasch item response theory (irt) which verified the results and provides the discussion and conclusion to complete this paper. 2 theoretical background courseware design has long been limited to the presentation of the digital artefacts in a rather myopic fashion. however, over 30 years ago david jonassen, a professor in learning technologies and an educational psychologist, was writing about the link between effective courseware design through electronic textual displays and computer-controlled environments (jonassen 1982). then, in a library and information science research study, six cases of information-seeking behaviours were examined, as students used perseus (a greek history information database). therefore, albeit sporadically, researchers have been trying to align it australasian journal of information systems mckay 2018, vol 22, research on educational technologies prescriptive training courseware 5 delivery platforms and pedagogy principles with a blended instructional approach towards elearning delivery modes for many decades. 2.1 prescriptive is-design methodology the review began by examining the commonly accepted prescriptive id models which have been published as id specifications or theoretical frameworks to facilitate the creation of instructional activities (culatta 2013). appraisal review criteria were: the model or theory; the inception date; originating researcher and further work; and evidential examples (government training, instructional systems design (isd), ict, and web 2.0) (see appendix-1). table 2 presents the results of the id model review; 25 models were listed by culatta (2013), with identification of the three objective taxonomies that align the knowledge domains of the id models (cognitive, affective and psychomotor). of the seminal 25 id models that originated through research, which largely focused on pedagogy (the process of teaching), there was no evidence that nine had survived into the new millennium. the advent of human-computer interaction (hci), however, renewed some researchers’ interest, to the extent that some of these early models form the basis of phd studies. seven models were used in government training programs, with the knirk and gustofson model only presented by researchers in an international conference on elearning for a knowledge-based society, showing no evidence of applicability to modern it (knirk and gustafson 1985). total number evidential examples prescriptive id models number found government training instructional systems design (isd) ict web 2.0 25 16 6 14 14 13 evidential examples objective taxonomies 3 3 1 2 3 3 table 2 prescriptive instructional design (id) models of the 14 models that show isd relevance, several models have been updated for modern usage. for instance, the dick and carey model, created in 1978, was adapted in the early 2000s (dick, carey, and carey 2004) and used in face-to-face training courses (schneider 2009). however, the literature reveals that researchers appear to have no appetite to adopt the dick and carey systematic design of instruction for ict and web 2.0 applications. in contrast, the skill acquisition strategies of the 4c-id model were adopted for ict and web 2.0 tools in a 2004 study (van merrienboer and jochems 2004). the assure model has also been adopted for isd with ict tools. so far as the objective taxonomies are concerned, the evidence shows researchers applying the cognitive domain to all four id model review categories (government training, isd, ict and web 2.0). no researchers have addressed the affective domain in government training environments, and no id researchers have addressed the psycho-motor domain in recent times (see appendix-1). australasian journal of information systems mckay 2018, vol 22, research on educational technologies prescriptive training courseware 6 prescriptive principles of id represent a set of theoretical propositions which focus on the relationships that remain true in appropriate conditions (sims 2014), regardless of the instructional program (set of prescribed activities) or specific instructional activity (merrill, barclay, and van-schaak 2008). merrill, barclay et al. (2008) attest that instructional programs will only succeed when these fundamental underlying principles are followed, including facilitating the instructional program in direct proportion to its implementation of these principles (instructional overview or fine detail), and reflecting the design of the learning/training environments and products rather than concentrating on how people will acquire knowledge and skills from these products. following these principles can deliver success, regardless of the delivery system or instructional architecture (clark 2003). however, a decrement in learning performance can occur if one or more of these underlying principles is not implemented (merrill 2002a). 2.2 first principles of instruction (merrill) following a comprehensive review of id theories, models and research, merrill (2009) abstracted a set of interrelated prescriptive id principles. merrill (2009) stated that: a principle is defined as a relationship that is always true under appropriate conditions regardless of the methods or models which implement this principle. principles are not in and of themselves a model or method of instruction, but rather relationships that may underlie any model or method. these principles can be implemented in a variety of ways by different models and methods of instruction. however, the effectiveness, efficiency, and engagement of a particular model or method of instruction is a function of the degree to which these principles are implemented. (p. 3) the following id principles (merrill 2009) are applicable to prescriptive is-design elements that promote successful training outcomes (see table 3). the demonstration principle: the trainees watch a demonstration, such as a screen-show and/or video clips. it is important for the is-design to provide guided navigation to general information or an organizing structure for a particular instance. further training enhancement is achievable through peer discussion and demonstration. the application principle: the trainees apply the new knowledge to a newly defined problem. this element describes the opportunity given within an online training module to provide solutions to a newly given case study example. it is important for the is-design to provide the trainee with corrective feedback. further enhancements can be achieved through peer collaborations. the task-centred principle: the trainees engage in a task-centred instructional strategy, such as providing more solutions to a given problem. further benefits can be expected when the isdesign sequences the training in a simple-to-complex task progression. the activation principle: the trainees can activate relevant prior knowledge or experience. the is-design needs to encourage the sharing of training activities with trainees who have previous experience. enhancements can be expected when trainees have opportunities for reflection. the integration principle: learning is promoted when learners integrate their new knowledge into their everyday world. this element amounts to offering circumstances of australasian journal of information systems mckay 2018, vol 22, research on educational technologies prescriptive training courseware 7 everyday life reflections that may require the trainee to defend their newly acquired skill/knowledge with another colleague. 2.3 goal-based training (schank) since roger schank introduced the concept of combining 'learning by doing' training with goal-based scenarios (schank et al. 1993/1994), many training programs have adopted this approach (biech 2015); (clark and mayer 2013). the theoretical basis for this work comes from the educational/social sciences fields, depicting human behaviour as devoid of the need to draw on is-design. for instance, schank (2002) could see the potential of web-based training, maintaining that the quality of training was improved because it offered the possibility of 'doing' over 'telling.’ schank's concepts were further developed into key elements of a goalbased scenario (hsu and moore 2011) as follows below (see table 3 for how these shank strategies were integrated into the prescriptive is-design methodology for managing the requirements for efficient and effective courseware development): learning/training goals: these refer to procedural or declarative knowledge (brinck 2016). to understand what participants thought about their training experience, there are four levels of learning evaluation: reaction; learning; behaviour; and results (kirkpatrick and kirkpatrick 2007). these data are often collected at the end of the training sessions and known in the field as ‘happy sheets’ (barlow 2009), and can provide enlightened insights into the training provider, the trainee's organization, and the training venue/technology. training mission: trainees need to buy into the training session. in other words, the mission statement must offer a realistic objective that motivates the trainee to make the decision to pursue the training goals (mcgarry 2010). this communiqué equates to an attitudinal state that enables an individual to choose taking an action towards some object, person, or event, as first described by robert gagné (gagné 1985). cover story: this element acts as a marketing tool (pallarito 2012) to identify a motivating story or textual example that explains the need for the training mission. the cover story needs to catch the trainee's eye. scenario operations: these are the instructional/training activities the trainees need to perform to fulfil the training mission. such training activities can involve anything the courseware designer believes will promote a trainee's comprehension. they may involve decision points, together with examples and non-examples (hsu and moore 2011) that reveal the consequences of following or not following the scenario examples described by merrill (2002). resources: these relate to the information that the trainee requires to complete the training session. a goal-based training session will involve two types of resources; the first being wellorganized information such as: textual explanations; video clips; voice-over narrations; graphics; or other resources accessible to the trainees. while the second type of resource relates to the embedded training (case-study) examples/stories (hsu and moore, 2011). feedback: training resources must provide two types of feedback (heron 2011). the first is known as formative feedback (shute 2007) and occurs during the training activities, whereby the trainee is given the chance to practice a skill/knowledge development. formative feedback must allow the trainee to repeat the training event until they understand the concept (merrill, 2002). the second type of feedback is known as summative feedback and occurs at the end of the training session. australasian journal of information systems mckay 2018, vol 22, research on educational technologies prescriptive training courseware 8 2.4 principles of instructional design 2017 so where is the integration of principles of id and isd in the year 2017? the id research community has responded rapidly and effectively to the advent of ict tools and smart web 2.0 media technologies. previous research had largely promoted pedagogical strategies that offer a common knowledge base, which provides for variation and detailed guidance for course designers when implementing merrill’s universal id principles. however, the advent of web 2.0 created an expansion of the way people use the internet, most noticeably the change from passive viewing of content to engagement with interactive sites (notably social media forums) which enable collaboration. the literature contains several courseware design theories providing guidance for id in 2017 (see the id model review in the appendix-1). they are briefly reviewed in the following paragraphs. the addie (analyse, design, develop, implement, evaluate) model comprises: high-quality designs; clearly identifiable learning objectives; carefully designed content; controlled facilitator/student workloads; well-integrated media; appropriate student activities; and student performance that is indicative of the expected instructional outcomes. addie works best with large numbers; it is ‘front-end’ loaded, meaning it concentrates most on the id process while leaving out interactivity issues between instructors and the student/trainees; and it lacks prescriptive advice for choosing technologies, or assessment strategies. the assure model was devised to implement gagné’s events of instruction: a – analyse learners; s – state the standards and objectives; s – select the instructional strategies, technology, media and materials; u – utilize technology, media and materials; r – require learner/trainee participation; and e – evaluate and revise (fontana 2014). assure represents an isd procedure that can be modified by teachers/trainers to suit the best instructional environment for their students/trainees. the ildf (integrative learning design framework for online learning) offers a practical means of shifting courseware design thinking from theory (anderson 2008) to the design of learnercentred instructional models in the digital age (spector 2010). the ildf represents a systematic id framework that supports the designing process and planning for customized instructional environments. the ildf is based on the premise that learning is a social process. the three id models described above advocate the pedagogies very well, but there is no evidence that they have been sufficiently tested for their ability to promote the intricacies of the web 2.0/id and social learning relationships which are so necessary for successful id in 2017. there are four other instances of id knowledge in the literature that have stood the test of time, and therefore are worthy of inclusion amongst these recent models. the first such instance of id knowledge is the discovery learning community (alfieri et al. 2011). they advocate for inquiry-based learning, as adopted by constructivist design models and championed by mayer (mayer 2004). the second are goal-based scenarios, which operate on the principle that people accept and apply explanations precisely if those explanations provide the information they need. the third and fourth instances of id knowledge relate to objective taxonomies: the psychomotor domain, and the affective domain. it remains curious that current courseware designers appear to bypass the psychomotor issues for id 2017. the affective domain provides a taxonomy that reflects the ontological hierarchy originally depicted by the principle of internalization. australasian journal of information systems mckay 2018, vol 22, research on educational technologies prescriptive training courseware 9 table 3 presents the prescriptive courseware is-design methodology that was used in the development of courseware to train government employees, which aligns with the merrill principles of instruction (merrill 2009) and his previous knowledge navigation strategies (merrill 2002b) (see appendix-2) and schank goal-based strategies (schank et al. 1993/1994). it articulates the sequence of courseware design steps required to create efficient and effective courseware training modules. merrill's principles schank’s strategies implications for is courseware design prescription demonstration learning/training goals • outline required outcome expectations. • provide mechanisms for participant feedback on training experience. training mission • provide communiqué to promote opportunity for trainee buy-in. show them what's in it for them. application cover story • create motivational tools to engage interest. • provide examples of how the training can be applied in new circumstances. • provide corrective feedback. task-centred scenario operations • provide an instances pool of examples and non-examples. • show the consequences of following a nonexample. activation resources • provide multiple instructional sequences to include: textual, static graphics, multimedia video clips, etc. • embed several case-study examples to highlight training activities in different environments. integration feedback • provide opportunity to engage with training colleagues to integrate new knowledge and personal experiences. • provide formative (during the training activities) feedback. • provide an embedded training quiz that allows reiterative participation. • provide summative (testing at the end of the training session) feedback. table 3 – combined strategies for prescriptive courseware is-design practice 3 research study background the prescriptive courseware is-design methodology was utilized in a funded research study that integrated the concepts of current it-enabled training practice within the context of the government sector. many providers offer training solutions to a wide range of local and global business and government customers. the theoretical frameworks that support this work largely emanate from well-established educational research (see the id-model review, appendix-1). though many training programs are available online, few show evidence of consideration of the theories that underlie courseware design. australasian journal of information systems mckay 2018, vol 22, research on educational technologies prescriptive training courseware 10 3.1 it-induced training practice in 2017 understanding the visual nature of our interactions with web-mediated training programs gives rise to knowing how different people deal with pictorial/textual content (riding 1991). this understanding has allowed researchers from previously disparate professions to combine their expertise in multidisciplinary research projects, including: deductive theory of human action; ethics in a government agency; and healthcare practice for junior doctors (kalraiya and buddhdev 2015). the resulting trans-disciplinary approach to it-induced training practice has a broadened audience of courseware design practitioners, with ‘design thinking’ emerging to champion the way forward with human-centred training courseware solutions (malamed 2011). malamed predicted that eight design trends would arise beyond 2015: design thinking; showing your work; learning experiences – not courses; micro-learning; digital badges; designing for data; crowd-sourced learning; and small private online courses (spocs) instead of massive open online courses (moocs). these elearning/training trends address three use and context dimensions of hci: the human-dimension, where the emphasis is on people’s need for language, communication and interaction and high-levels of tactility (solomon 2012); the computer or machine-dimension, where the technical means to provide the solution is foremost, and it and adaption are important; and the development-dimension, which relates to design approaches leading to implementation techniques and tools. in the human-dimension, the it-induced training proposition of ‘showing your work’ can be seen when people enact merrill’s ‘show me’ principle. it comes about through their propensity to be continually paying attention to their mobile devices and others become interested in what they are doing. encouraging this type of experiential sharing with it-induced training practices prevents critical intrinsic knowledge gaps, which occur when highly skilled employees leave their organizations. everyone benefits when people share their instructional experiences through an online (work) training environment (malamed 2015). boundaries for gaining new knowledge or refreshing a skill are now melting away (bradley 2006), meaning we have reached the point where the distinction between our professional and private lives are fully integrated. now, instead of relying upon formal training occurring in the workplace, people increasingly acquire knowledge from their mobile devices, at the time they need the information (sellers 2013). this socialized learning/training behaviour resembles the notion of experiential learning described initially by kolb (1984) and subsequently by hugg et al. (2013); it reveals a preference for ‘learning experiences’ rather than formalized training courses (tavangar 2014). in the machine-dimension, the proposition of it-induced micro-training shows we are living in an ever-shrinking digitally-enhanced world with ever-increasing connectivity expansion in which people are packing more into their daily lives and work practices (bradley 2006). instances of micro-learning and the consequent ‘digital badges’ have become commonplace (frederiksen 2013): the daily news is delivered to us in convenient sound-bites as staged performances, while many software applications post digital badges as motivational learning/training products. in the development-dimension of hci in training, crowd-sourced examples show how people around the world are combining individual efforts to leverage expertise from joint collaborations – from designing scalable technology solutions, to improving impoverished communities, to providing expert training resources for novice courseware designers. the final example presented here relates to the controversy over the newer id models that are australasian journal of information systems mckay 2018, vol 22, research on educational technologies prescriptive training courseware 11 appearing as successful it-induced training practices – the debate over using spocs instead of moocs. the latter are free generic training programs, with cohorts of learner/trainees numbering in the thousands and a high attrition rate (anderson 2008). in contrast, spocs (private and therefore likely to involve fees), typically implement a blended (or hybrid) instructional approach that involves the ‘flipped-classroom’ technique (van-praet 2016) and a small number (less than 100) of participants. this technique can vary according to the circumstances of both facilitator and learner/trainee groups. ict tools can be engaged on an individual basis for instructional activities prior to meeting in a classroom environment and without facilitation from a teacher/trainer. the facilitator and participants then come together so that the former can answer questions and the latter can give demonstrations of newly acquired knowledge or skills that are shared amongst the cohort. 3.2 research methodology the objectives of the example research study were to: • investigate ways to encourage skill development in government administrative tasks; • provide empirical evidence of the learning that occurred during an etraining session; • empower accurate vocational skills assessment; • evaluate the effectiveness of state government training programs; and • promote the inclusion of id principles into professional practice relating to webmediated courseware design. 3.3 research questions the main research question was: • are there significant differences between the levels of skill development generated through traditional face-to-face classroom training, a blended training approach combining computerized and face-to-face instructions, and computerized/online instructional modules? subsequent (secondary) research questions were: • does an individual's instructional training preference impact on their training outcome? • does the evaluation of etraining outcomes lead to more effective etraining strategies? • what are the effects of each specific instructional strategy on training outcomes in the corporate educational/training environment? 3.4 research design the example research study compared the change in knowledge of 36 government sector trainees as a result of participation in a training program delivered using one of three instructional modes (mckay & izard 2015a). the research had a quasi-experimental 3x3 design, involving three independent variables (training mode, instructional training preference and prior domain knowledge). cognitive performance outcomes (knowledge australasian journal of information systems mckay 2018, vol 22, research on educational technologies prescriptive training courseware 12 improvement from preto post-test) formed the dependent variable. there were three levels for each independent variable. the three instructional modes under investigation were traditional face-to-face with no computer-based materials (t-1), a blended approach involving traditional face-to-face learning and etraining tools (t-2), and computerized etraining (t-3). the instructional training preferences were categorised as spatial, imagery and verbal. the object-spatial imagery and verbal questionnaire (osivq) (blazhenkova and kozhevnikov 2009) was used to determine instructional mode preference for 25 participants (table 4). participants’ prior domain knowledge was assessed as novice, intermediate, and experienced (operationalized when each participant underwent the knowledge/skills screening questionnaire before their training session). cognitive media preference (osivq) instructional treatment face-toface t-1 blended t-2 (combination of face-to-face and computerized) fully computerized t-3 spatial 2 2 6 imagery 5 5 2 verbal 1 0 2 table 4 main experiment participant distribution of cognitive media preferences the researchers anticipated that cognitive processes would differ between a government employee with greater background experience and one without much experience (a novice trainee). according to bagley and heltne (2003), novice trainees will benefit more from instructional methods than more experienced trainees. 3.5 data analysis data was analysed using the quest interactive test analysis system (adams and khoo 1996), a rasch irt modelling tool. the trainee’s elearning/training performance was evaluated based on the magnitude of change in participant proficiency (civitello et al. 2015). it was anticipated that the analysis would identify the interactive dynamics of skill development with and without ict tools as training mediation techniques. 3.6 instrumentation validity and marking strategy to ensure homogeneity, accuracy and internal reliability of the preand post-testing instruments, it was necessary to calibrate all test-items. quest develops a uni-dimensional scale with equal intervals along each axis as it measures participants’ performances and testitems together (see figures 1 and 2). this rasch scaling approach provides a developmental sequence of instructional/training tasks, from simple to complex (rasch 1960). this rigorous scale identifies individuals at different skill levels, as well as locating each test-item with respect to the behaviour of the training content concepts being measured. each question (or test-item) was given a numeric label (question numbers), and there were two scoring categories used (dichotomous and partial credit). the dichotomous scores were recorded as 0 or 1, while the partial credit items were able to receive a 0, 1, 2, 3 or 4. scores were recorded in a data file to be submitted to the quest tool through a command file. an earlier calibration exercise was conducted in a pilot study (mckay & izard 2014), to validate the scaling australasian journal of information systems mckay 2018, vol 22, research on educational technologies prescriptive training courseware 13 characteristics and to reflect the developmental skills' sequencing on the scale to show the training events and instructional objectives from simple to the most complex. figure 1 quest variable map showing participants’ and test-items' performance responses to the preand post-test-items were analysed using the rasch model (adams and khoo 1996) which determines the probability of a person responding in a certain way to testitems. each individual response to every test-item was recorded in a matrix. where there were identical test-items on the preand post-tests, they were 'anchored' such that their scaled scores were comparable across the testing instruments. using this analytical process, it was possible to determine from the difference between pre-test and post-test scaled scores whether knowledge acquisition had occurred or not, and whether the relevant training strategy had resulted in improved training performance compared against the other two instructional training treatments. australasian journal of information systems mckay 2018, vol 22, research on educational technologies prescriptive training courseware 14 figure 2 quest kid map showing individual participant's performance comparisons of actual patterns of test-item interactions with participants and the rasch model pattern becomes the fit statistic, which enables further investigation of unusual patterns. the quest item fit map (figure 3) provides a visual check of the magnitude of the fit statistic for the test-items that lie within the two dotted lines. the asterisks represent the magnitude of the fit statistic for the test-items on the same line (adams and khoo 1996). australasian journal of information systems mckay 2018, vol 22, research on educational technologies prescriptive training courseware 15 figure 3 quest item fit map showing rasch model fit statistics unsuitable test-items (those outside the dotted lines) identified during the calibration exercise prompted modifications to the testing instruments for the final experiment. subsequently, it was anticipated that both the preand post-test questions would be answered within an acceptable range of expectation as determined by the quest estimate. in order to limit the effect of the trainees’ prior domain knowledge, their pre-test scores were subtracted from their post-test scores, enabling the experiment's main cognitive performance measure to be expressed as the quest difference logit value (qdlv). the mean of each training treatment sub-group’s score was then calculated and compared. 3.7 results the example research study looked at the various interactive effects of variables on performance outcomes, starting with the performance outcomes compared against treatment types, followed by the interactive effect of participants' instructional training preferences. 3.7.1 effect of treatment type on performance the three training program datasets resulted from traditional facilitator-led face-to-face sessions with no computer mediation (t-1); the blended approach that involved both face-toface training and etraining tools (t-2); and only the online etraining tools (t-3). these datasets were comparable because they involved the same training facilitator, whose knowledge of the business process and technical competence in elearning/training had been identified as of equally high standard to general workplace training competence. all participants were located on the logit-scale before and after the training session and were tracked for changes in status on the learning/training continuum. the resulting effect size estimates revealed gains in accomplishment for the main experiment, based on anchored scaled scores. when a test-item or a participant score is anchored, the anchor value or values apply to all subgroup australasian journal of information systems mckay 2018, vol 22, research on educational technologies prescriptive training courseware 16 combinations to which the test-item or participant belongs (adams and khoo 1996). table 5 shows the comparison between the performance gains of the blended treatment group (faceto-face and etraining tools t-2) and the computerized (etraining) only group t-3. blended (n=7) comparisons computerized (n=10) mean 1.741 1.408 mean difference 0.33 standard deviation 2.241 1.825 pooled standard deviation 2.00 effect size (standard error) 0.16 (0.49) descriptor small (izard, 2004) table 5 performance gains – blended group (t-2) versus computerized group (t-3) although the blended group (t-2) achieved a mean qdlv that was 23% higher than the computer only group (t-3), the effect size measure calculated for the gains in scaled score was judged to be 'small' (izard 2004). note that the associated standard error exceeded the effect size estimate. this means the range of the (estimate + standard error) to the (estimate standard error) includes zero. table 6 shows the comparison of the computerized (etraining) only treatment (t-3) with the face-to-face group (t-1). computerized (n=10) comparisons face-to-face (n=8) mean 1.408 1.631 mean difference -0.22 standard deviation 1.825 3.061 pooled standard deviation 2.44 effect size (standard error) -0.09 (0.47) descriptor very small (izard, 2004) table 6 performance gains – computerized group (t-3) vs face-to-face group (t-1) the face-to-face training participants (t-1) scored 16% higher on average than the computerized (etraining) only group (t-3). however, the effect size measure calculated for the gains in scaled score was judged to be 'very small' (izard 2004), based on anchored scaled scores. note that the associated standard error again exceeds the effect size estimate. table 7 shows the final comparison –the blended group (t-2) vs the face-to-face group (t-1). blended (n=7) comparisons face-to-face (n=8) mean 1.741 1.631 mean difference 0.11 standard deviation 2.241 3.061 pooled standard deviation 2.71 effect size (standard error) 0.04 (0.52) descriptor very small (izard, 2004) table 7 performance gains – blended group (t-2) vs face-to-face group (t-1) australasian journal of information systems mckay 2018, vol 22, research on educational technologies prescriptive training courseware 17 the blended treatment (t-2) produced a mean score only 7% higher than that of the face-toface group (t-1), and hence the effect size measure calculated for the gains in scaled score was judged to be 'very small' (izard 2004), based on anchored scaled scores. note that the associated standard error exceeds the effect size estimate. this training program on introductory ethics showed that the participants benefit most from a blended instructional treatment, where they received face-to-face instruction combined with self-exploration with an online training is. it also showed that participant performance was worst when they only had computerized instruction. 3.7.2 comparative effect of osivq on performance did the participants' responses to the osivq explain the magnitude gains in the training program (knowledge of ethics) and subsequent applications of that knowledge to the given case studies within the training materials? assuming that the three treatments were equally effective, then the osivq data may explain any gains from the pre-test to the post-test made by participants. if there was a positive relationship one would expect the highest performance gains to occur in the highest quintile on imagery, with subsequent quintiles showing lower gains. a negative correlation would expect the lowest gains to be associated with the highest quintile. however, this analysis does not show a systematic relationship between imagery and gains in participants' scores on knowledge of ethics and applications of that knowledge to the given case studies. similar explanations, if relevant, would apply to 'spatial' and 'verbal.' it appears that there was no systematic relationship between 'spatial' and gains in score on knowledge of ethics and applications of that knowledge to the case studies. yet it appears that there may be a systematic relationship between 'verbal' and gains in score on knowledge of ethics and applications of that knowledge to the given case studies. however, those in lower groups on the 'verbal' dimension appeared to derive greater benefit than those in the higher groups, thus a negative correlation. one has to ask why this would be so (mckay & izard 2015b). assuming that the three instructional treatments interact with the 'imagery,' 'spatial' and 'verbal' variables identified by use of the osivq, then the osivq data might explain the gains from pre-test to post-test made by participants in the separate training program treatment groups. table 8 presents the correlation index of each training program treatment group alongside the three osivq variables. performance /osivq correlation face-toface t1 blended (faceto-face/online t2 etraining t3 imagery -.589 .445 -.004 spatial -.149 .039 -.473 verbal -.550 .019 -.866 table 8 effect of object-spatial imagery and verbal questionnaire (osivq) on performance in terms of the interactive effects of osivq and training program treatment, on performance outcomes (knowledge of ethical concepts), the blended instructional treatment (t-2) involving both traditional face-to-face and etraining tools had a positive correlation with all three osivq variables. whereas there was a negative correlation between the face-to-face treatment (t-1) and the computerized etraining treatment (t-3) with all three osivq variables. the australasian journal of information systems mckay 2018, vol 22, research on educational technologies prescriptive training courseware 18 correlation coefficients illustrated some expected results, such as a strong negative correlation between the computerized etraining treatment (t-3) and the verbal osivq variable, and strong negative correlation between the face-to-face treatment (t-1) and the imagery osivq variable. however, there was a similar negative correlation between the face-to-face treatment (t-1) and the verbal osivq variable. 4 discussion government agencies and business organizations share the problem of how to maintain a wellskilled and knowledgeable workforce. modifying off-the-shelf it training packages is costly and takes a long time to organize and implement. the vexing issue for the training courseware designer is that the broad nature of the trainee demographics plays right into the hands of the non-government training providers, who offer ubiquitous programs that over-reach expectations and do not always provide effective results (sims 2014). although online training packages and training providers abound and, thanks to ict, are available worldwide, the optimal solution is to provide customised in-house training with an emphasis on flexible onthe-job experiential learning. our research indicates that part of this customization must include access to the training facilitator, either through face-to-face contact or through online interactivity. a review of existing id models (see appendix-1) highlighted a lack of direction for is courseware design strategies. is-design pertaining to courseware developers is either missing from the literature or relies on traditional id methodologies. however, the advent of smarter it has made customary id frameworks redundant. online training packages abound, due to the borderless context of accessible ict. there seems to be no limit to the number of training providers worldwide creating an online training plethora. and so, finding the right training package is very difficult. far better to make your own. new frameworks, such as goal-based training, are the future. goal-based training fits neatly into the do-it-yourself training courseware design realm. it embraces traditional id models to underpin online training strategies, use of new ict tools to develop courseware, and translation of training needs into successful experiential learning. the author set out to explore the human-dimension of hci to present is-design practice to innovate a prescriptive courseware is-design methodology. the methodology combines the merrill (2002a; 2002b; 2009) well-known id principles and schank’s goal-based strategies (schank et al. 1993/1994) to develop training modules. previous researchers have used this courseware is-design methodology to: test the interactive effect of instructional strategies and cognitive preferences (mckay 2000), in a web-mediated environment (mohamad 2012); to explore ict literacy assessment tools for trainee teachers in malaysia (mat-jizat 2012); and more recently to investigate mobile healthcare training practices in jordanian hospitals (bakkar 2016). to illustrate the effectiveness of the prescriptive courseware is-design methodology, an example research study was described. this research investigated the interactivity of training mode and instructional training preferences with respect to training program outcomes. gains in introductory ethics knowledge were measured in each of three training modes (traditional classroom facilitator-led (t-1), blended traditional face-to-face and online training (t-2), and online training only (t-3)). differences were detected in knowledge gains within the study australasian journal of information systems mckay 2018, vol 22, research on educational technologies prescriptive training courseware 19 groups associated with levels of government practice and demographics (job experience, gender and education). the principal findings of the example research study showed that changes in knowledge outcomes on introductory ethics achieved a substantial magnitude of effect (civitello et al. 2015);(cohen 1977), across the three training treatments. the t-1 group’s mean score improved by 1.63 (tables 6 and 7), the t-2 group’s by 1.74 (tables 5 and 7), and the t-3 group by 1.41 (tables 5 and 6). these findings are important because they vindicate the training strategies adopted by the courseware developers and strengthen the case for a customized approach to is training design. the is courseware design embraced the (merrill 2002b) principles of instruction, and the six elements of schank's goal-based training regime (schank et al. 1993/1994), confirming the prescriptive courseware is-design methodology for efficient and effective courseware development. furthermore, the study indicated that a blended instruction strategy delivers the best training performance outcomes for all learning preference groups. hence, the results demonstrate that implementing the flipped-classroom approach in a training context effectively combines the efficiencies and practicability of online training practice with the more traditional face-to-face approach. the analysis of the osivq variable interactive effect on performance also demonstrated that the blended instructional treatment (t-2) was the only treatment with positive correlations. the strongest correlation (0.445) was between the imagery variable and the blended instructional treatment (t-2). this result suggests that the images contained within the computerized etraining instruction, combined with face-to-face clarifications, had a positive effect on performance outcomes. as predicted, the imagery-preferenced trainees had a stronger performance correlation with the computerized etraining instructional mode (t-3) than face-to-face (t-1), and the verbal-preferenced trainees had a stronger performance correlation with the face-to-face training mode (t-1) than the online computerized etraining mode (t-3). the spatial-preferenced group had no strong correlation to any of the instructional treatments, however had a large negative correlation with the computerized etraining mode (t-3). spatial preference is a complex mix of cognitive processing that involves both object imagery (visual appearances) and spatial imagery (spatial relationships) (blazhenkova & kozhevnikov, 2009), therefore it is possible that the spatial-preferenced trainees experienced imagery overload when using the computerized training mode (t-3) and benefited from interactive training instruction with the facilitator. 5 conclusion a customized etraining program should seamlessly match each trainee’s learning profile to meet their individual needs allowing each trainee to define their own knowledge/skill development path. to customize an etraining program, a prescriptive courseware is-design methodology is required. this paper contributes to the field by proposing that a prescriptive training courseware isdesign methodology can generate an efficient and effective courseware development model. the model combines merrill’s (2002a) principles of id with schank's goal-based 'learning by doing' training id elements (schank et al. 1993/1994), to provide: training goals; a training mission; a cover story; scenario operations; resources; and the all-important formative feedback. australasian journal of information systems mckay 2018, vol 22, research on educational technologies prescriptive training courseware 20 the example project showed that the customized prescriptive courseware is-design methodology, combining instructional strategy, prior domain knowledge and preferred learning context could produce better outcomes than the one-size-fits-all approach to training solutions involving ict generally applied in the government/corporate sector. the example project proved this to be true, demonstrating significant knowledge acquisition by participants. improving government/corporate training outcomes, generating a more highly skilled workforce that will have a significant impact on business results. references acci. 2011. employers' commitment to training: key finding from the acci national workplace skills survey (2010) canberra, http://www.acci.asn.au/ australian chamber of commerce and industry. adams rj, khoo s-t (1996) quest:the interactive test analysis system. melbourne: software user manual. australian council for educational research. alfieri l, brooks pj, aldrish nj, tenenbaum jr (2011) does discovery-based instruction enhance earning? j. of educational psychology 103 (1):1-18 http://dx.doi.org/10.1037/a0021017. anderson t (2008) theory and practice of online learning: issues in distance education, canada: athabasca university. doi: 9781897425077. bakkar mn (2016) an investigation of mobile healthcare (mhealcare) training design for healthcare empoyees in jordan. doctoral dissertation (business information systems) total fulfilment rmit univ. school of business info. tech. & logistics. ballew p, castro s, claus j, kittur n, brennan l, brownson rc (2013) developing web-based training for public health practitioners: what can we learn from a review of five disciplies? health education research 28 (2):276-287. doi: 10.1093/her/cys098. barlow s (2009) happy sheet template. http://www.trainingzone.co.uk/develop/cpd/happysheet-template. benac dj (2015) keeping wind in your sale: keeping up with tools, techniques, and technology. j. of failure analysis and prevention 15:1-2 doi: 10.1007/s11668-014-9915-2. biech e (2015) 101 ways to make learning active beyond the classroom nj pfeiffer wiley. http://au.wiley.com/wileycda/wileytitle/productcd-1118971981.html. blazhenkova o, kozhevnikov m (2009) the new object-spatial-verbal cognitive style model: theory and measurement applied cognitive psychology 23 (5):638-663. doi: 10.1002/acp.1473. bradley g (2006) social and community informatics: humans on the net. ny: routledge: taylor&francis. brinck i (2016) developing an understanding of social norms and games: emotional engagement, nonverbal agreement, and conversation. theory & psychology 24 (6):737-754. burgess ft grimshaw p, shaw ne (2017) research commentary: diversity of the information research field: a governance perspective information systems research 28 (1):5-21. australasian journal of information systems mckay 2018, vol 22, research on educational technologies prescriptive training courseware 21 chatzimouratidis a, theotokas i, lagoudis in (2012) decision support systems for human resource training and development international j. of human resource management 23 (4):662-693. doi: 10.1080/09585192.2011.561235. civitello dj, cohen j, fatima h, halstead nt, liriano j, mcmahon ta, ortega cn, sauer el, sehgal t, young s, rohr jr (2015) biodiversity inhibits parasites: broad evidence for the dilution effect proceedings of the national academy of sciences of the united states of america (pnas) 112 (28):8667–8671. doi: 10.1073/pnas.1506279112. clark r (2003) six principles of effective e-learning: what works and why the elearning guild's learning solutions: practical applications of technology for learning e-magazine (sept.):1-10. clark rc, mayer re (2013) scenario-based e-learning: evidence-based guidelines for online workforce learning. ny. cohen j (1977) statistical power analysis for the behavioral sciences. new york: academic press. culatta r (2013) instructional design models http://www.instructionaldesign.org/models. d'amours m, legault m-j (2013) highly skilled workers and employment risks: role of instutions labor studies j. 38:89-109. dick wo, carey l, o'carey j (2004) the systematic design of instruction. 0205412742: allyn & bacon. docq f (2015) course design and delivery specifications as a tool for ensuring quality in an online training program canadian j. of learning and technology 41 (3.1). flag (2007 july) australian flexible learning framework 2005-2007 impact statement report to national senior officials comm. flexible learning advisory group: aust. govt dept of educ, sci. & training. fontana t (2014) the 1999 heinich, molenda, russell, & smaldina id model also known as assure https://prezi.com/uj4u_9y1ruwa/the-heinich-molenda-russel-smaldino-model-alsoknown-a/. frederiksen l (2013) best of the literature public services quarterly 9:321-325. doi: 10.1080/15228959.2013.842414. gagné rm (1985) the conditions of learning: and the theory of instruction ny: holt/rinehart/winston. görg h, strobl e (2006) do government subsidies stimulate training expenditure: microeconometric evidence from plant-level data sourthern economic journal 72 (4):860876. harward d (2014) how big is the training market? training industry. https://www.trainingindustry.com/blog/blog-entries/how-big-is-the-trainingmarket.aspx: heron g (2011) examining principles of formative and summative feedback british j. of social work 41 (2):276-295. hsu c-y, moore dr (2011) formative research on the goal-based scenario model applied to computer delivery and simulation j. of applied instructional design 1 (1):13-24. australasian journal of information systems mckay 2018, vol 22, research on educational technologies prescriptive training courseware 22 hugg pr, mcaulay jj, lorio kv, sarpy l (2013) experiential teaching on steriods: unexpected pedagogical discoveries loyola j. of public interest law loyola univ. new orleans college of law legal studies research paper 2014-01 http://ssrn.com/abstract=2418366. izard j (2004) impediments to sound use of formative assessment (and actions we should take to improve assessment for learning) paper presented at the australian association for research education (aare 2004): positioning education research melbourne http://www.aare.edu.au/04pap/iza04905.pdf. jonassen dh (1982) the technology of text. new jersey: educational technology. kalraiya a, buddhdev p (2015) the trojan project: creating a customized international orthopedic training program for junior doctors orthopedic reviews 7 (1): 6-8. kirkpatrick dl, kirkpatrick jd (2007) implementing the four levels: a practical guide for effective evaluation of training programs: berrett-koehler. knirk fg, gustafson kl (1985) instructional technology: a systematic approach to education: holt rinehard & winston. kolb da (1984) experiential learning: experience as the source of learning and development. nj: prentice hall. ma s, fildes r, huang t (2015) demand forecasting with high dimensional data: the case of sku retail sales forecasting with intra-and inter-category promotional information european j. of operational research 249 245-257. malamed c (2015) 8 learning design trends to watch in 2015 http://theelearningcoach.com/elearning_design/learning-design-trends-2015/. mat-jizat, je (2012) investigating ict-literacy assessment tools: developing and validating a new assessment instrument for trainee teachers in malaysia doctoral dissertation (business information systems). total fulfilment: rmit univ. school of business info. tech. & logistics. mayer re (2004) should there be a three-strikes rule against pure discovery learning? american psychologist 59 (1):14-19 http://dx.doi.org/10.1037/0003-066x.59.1.14. mcgarry d (2010) seven reasons people buy personal training sessions club industry. http://clubindustry.com/step-step/seven-reasons-people-buy-personal-trainingsessions. mckay e (2000) measurement of cognitive performance in computer programming concept acquisition: interactive effects of visual metaphors and the cognitive style construct. journal of applied measurement 1 (3):257-286. mckay e, axmann m, banjanin n, and howat a (2007) towards web-mediated learning reinforcement: rewards for online mentoring through effective human-computer interaction proceedings of the 6th iasted international conference on web-based education, chamonix, p:210-215, isbn:978-0-88986-650-8 http://www.iasted.org/conferences/pastinfo-557.html. mckay e and izard jf (2014) online training design: workforce reskilling in government agencies proceedings of the ic3e 2014 ieee conference on elearning, emanagement and services, melbourne, pp.70-75. australasian journal of information systems mckay 2018, vol 22, research on educational technologies prescriptive training courseware 23 mckay e and izard jf (2015a) evaluate online training effectiveness: differentiate what they do and do not know proceedings of the 8th international conference on ict, society and human beings 2015 (multi conference on computer science and information systems mccsis), las palmas de gran canaria. mckay e, and izard j (2015b) measurement of cognitive performance in introductory ethics: workforce reskilling in government agencies proceedings of the 4th ieee conference on elearning, e-management and e-services (ic3e2015), melaka. bagley ca, heltne m (2003) cooperative teams. collaboration conference, bloomington, mn. mena (2015) empanelment of independent voluntary/non-government organisation as capacity building and training providers [tender documents: t26398172]. in watershed development and soil conservation department, edited by a bawaba. rajasthan: proquest. merrill dm (2009) first principles of instruction. instructional design theories and models: building a common knowledge base iii. ny: routledge. merrill dm, barclay m, van-schaak a (2008) prescriptive principles for instructional design. in foundations of educational technology: integrative approaches and interdisciplinary perspectives, edited by j. michael spector, m.david merrill, m.f. j. van merriënboer and m.p. driscoll. ny: taylor & frances. merrill md (1987) the new component design theory: instructional design for courseware authoring instructional science 16:19-35. merrill md (2002a) first principles of instruction etr&d 50 (3):43-59 http://www.indiana.edu/~tedfrick/aect2002/firstprinciplesbymerrill.pdf. merrill md (2002b) pebble-in-the-pond model for instructional development performance measurement 41 (7):39-44 http://mdavidmerrill.com/papers/pebble_in_the_pond.pdf. miller l (2014) td 2014 state of the industry report: spending on employee training remains a priority. mohamad m (2012) the effects of web-mediated instructional strategies and cognitive preferences in the acquisition of introductory programming concepts: a rasch model approach doctoral dissertation (business information systems). total fulfilment rmit univ. school of business info. tech. & logistics. pallarito k (2012) cover story: head off the competition with marketing strategies that work hearing j. 65 (7):16,18,20,22. doi: 10:1097/01.hj0000416271.13977.8d. rasch g (1960) probabilistic models for some intelligence and attainment tests. copenhagen: nielsen & lydiche. riding rj (1991) cognitive styles analysis uk birmingham learning and training technology. rohde m, brödner p, stevens g, betz m, wulf v (2016) grounded design – a praxeological is research perspective j. of information technology. doi: 10.1057/jit.2016.5 rosenberg mj (2001) e-learning: strategies for delivering knowledge in the digital age. ny: mcgraw-hill. schank rc (2002) designing world-class e-learning: how ibm, ge, harvard business school, & columbia university are succeeding at e-learning. ny mcgraw-hill. australasian journal of information systems mckay 2018, vol 22, research on educational technologies prescriptive training courseware 24 schank rc, fano a, bell b, jona m (1993/1994) the design of goal-based scenarios j. of the learning sciences 3 (4):305-345. schneider p (2009) the redesign of face-to-face training courses and the role of participants' feedback within the redesign process partial fulfillment for a doctor of philosophy, educ, capella univ. (3379722). schofield k (2002) on-line learning: case studies of the corporate experience sydney: oval 192069871x. sellers a (2013) growing reliance on smart devices is influencing consumer behavior, according to research from euromonitor international http://finance.yahoo.com/news/growingreliance-smart-devices-influencing-080000376.html. shelly m, jackson m (2009) doing business with consumers online: privacy, security and the law international j. of law and information technology 17 (2):180-205. doi: 10.1093/ijlit/can003. shute v (2007) focus on formative feedback. nj: educational testing service, princeton https://www.ets.org/media/research/pdf/rr-07-11.pdf. sims r (2014) design alchemy. switzerland: springer. solomon m (2012) high-tech, high-touch customer service: inspire timeless loyalty in the demanding new world of social commerce saranac lake: amacom. https://www.micahsolomon.com/freechapters.html. spagnoletti p, resca a, gwanhoo l (2015) a design theory for digital plaforms supporting online communities: a multiple case study j. of information technology 30:364-380. spector j m (2010) learning and instruction in the digital age. edited by j.m. spector, d. ifenthaler, p. isaias, kinshuk and d.sampson. 1st ed. ny: springer. spencer j (2014) policy, business and ict: a study of the growth and decline of private vocational training providers in the education/migration industry in australia doctor of philosophy, global, urban and social sciences, rmit univ. (http://researchbank.rmit.edu.au/view/rmit:161511 ). tavangar h (2014) the out of eden walk: an experiential learning journey from the virtual to the real http://www.edutopia.org/blog/out-of-eden-experiential-learning-homa-tavangar. van-praet e (2016) business meeting training on its head: inverted and embedded learning business and professional communication quartely 79 (1):119-130. doi: 10.1177/2329490615610777. van merrienboer jjg, jochems w (2004) integrated e-learning: implications for pedagogy, technology and organization. london: routledgefalmer. vann ls (2015) demonstrating empathy: a phenomenological study of instructional designers making instructional strategy decisions for adult learners partial fulfillment of requirements for doctor of philosophy, capella univ. vignan s, senthilkumar g, marthi n, goeteti p (2011) integrating learning outcomes and bloom's taxonomy in web application development course: experiences from corporate training 2011 ieee students' technology sumposium iit kharagur. australasian journal of information systems mckay 2018, vol 22, research on educational technologies prescriptive training courseware 25 copyright: © 2018 mckay. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 theoretical background 2.1 prescriptive is-design methodology 2.2 first principles of instruction (merrill) 2.3 goal-based training (schank) 2.4 principles of instructional design 2017 3 research study background 3.1 it-induced training practice in 2017 3.2 research methodology 3.3 research questions 3.4 research design 3.5 data analysis 3.6 instrumentation validity and marking strategy 3.7 results 3.7.1 effect of treatment type on performance 3.7.2 comparative effect of osivq on performance 4 discussion 5 conclusion paper title (paper title style) australasian journal of information systems mazaheri, lagzian & hemmat 2020, vol 24, research article research directions in information systems field 1 research directions in information systems field, current status and future trends: a literature analysis of ais basket of top journals elham mazaheri department of management, faculty of economics and administrative sciences, ferdowsi university of mashhad, mashhad, iran. e.mazaheri@mail.um.ac.ir mohammad lagzian department of management, faculty of economics and administrative sciences, ferdowsi university of mashhad, mashhad, iran. zahra hemmat department of management, faculty of economics and administrative sciences, ferdowsi university of mashhad, mashhad, iran. abstract information system, as an emerging discipline which is dynamic and interdisciplinary, is constantly undergoing rapid changes; therefore, one of the most critical issues is to study the topics and methodological trends in this field. this study aimed to thoroughly investigate the topics and methodologies used in information system throughout a twelve-year period (20072018). in the present study, all the articles published in the top eight information system journals were investigated based on their topics and methodologies using a literature analysis approach, and the most common topics and methodologies in such studies were then detected. the main topics addressed in this field were electronic commerce/business, information system research, and is usage/adoption, and the survey was the dominant research methodology in this regard. furthermore, an increase in the use of the mathematical models indicates that more accurate quantitative research methods have made this field more mature. the research findings also show that the internet of things (iot), big data, social media, bitcoin and block chain, digital transformation, sharing economy, digital platform, methodology of machine learning, data mining, and text mining are the main research topics and methods which should be further studied. keywords: information systems research; research methodology; literature analysis; association for information systems; senior scholars’ basket of journals; academic journals 1 introduction compared to many disciplines in the field of social sciences and business, information system is a relatively new, emerging, and unique field undergoing constant changes for several reasons. on one hand, the changes in information technology, as one of the main areas of australasian journal of information systems mazaheri, lagzian & hemmat 2020, vol 24, research article research directions in information systems field 2 information system research, are occurring so quickly that researchers can hardly address a particular phenomenon or topic in-depth before dealing with the other new issues since the speed of technological advances in software technology and the other information and communication technologies such as social media, smart devices, and new delivery mechanisms is extremely high. another reason for the changes in this field is the emergence of new research paradigms, including interpretive and critical research, and new methodologies such as design science, action research and other methods, which have been extensively accepted. furthermore, new researchers with different backgrounds and experiences have been involved in this field. in addition, these researchers deal with a variety of theories, theoretical foundations, frameworks, and reference disciplines (namely management, organization sciences, and computer sciences). accordingly, the dynamic and interdisciplinary nature of the information system field and its diversity and rapid advances make it difficult for researchers to keep track of the modern-of-art topics and methodology trends; hence, having information about the current status of research in this filed and being aware of its change processes would help the researchers in the field of information systems. the present study investigated topics and methodology trends of research in the field of information systems throughout a 12-year period (2007-2018) using a literature analysis method. according to stemler (2001), the literature analysis enables researchers to study and navigate large knowledge bases using relatively simple and systematic methods. in this study, the data was gathered based on the review of eight major journals in the field of information systems as such a large database including 3250 articles was created. the collected data encompassed the journal name, the year of publication of a concerned article, and the topic and methodology of the research paper. an in-depth review of previous studies reveals that no comprehensive analysis has been carried out in recent years, and the most recent comprehensive research has been conducted during 2004-2013. furthermore, some of the most recent studies have concentrated on only one journal (palvia & kakhki, 2016; palvia et al., 2017). as a result, there are few comprehensive studies in this field. considering the significance of this subject, the present study analysed the topics and methodologies used in the ais basket of the top journals throughout a twelve-year period (2007-2018). this review study would contribute to the improvement of the research on information systems. the main objectives of this study are as follows: 1. determining the topics addressed by the researchers and analysing the research trends; 2. detecting the most common research methodologies and their trends; 3. identifying research gaps and predicting the future trends of information system research. in the following, a review of the relevant literature is presented. in addition to the research methodology, the frameworks used to analyse the articles along with the journal selection procedures and the coding methods are also described. furthermore, the analysis and coding results are also discussed. this paper also encompasses topic and methodological trends and is concluded with some discussions along with limitations and recommendations for future research. australasian journal of information systems mazaheri, lagzian & hemmat 2020, vol 24, research article research directions in information systems field 3 2 literature review since the early emergence of information systems, some researchers have published studies based on the analysis and review of the relevant literature. three types of analyses are found out in the literature: research topics and themes (avison, dwivedi, fitzgerald, & powell, 2008; claver, gonzález, & llopis, 2000; d'aubeterre, palvia, & steevens, 2005; dwivedi & kuljis, 2008; ji, min, & han, 2004; palvia & kakhki, 2016; palvia, kakhki, ghoshal, uppala, & wang, 2015; palvia et al., 2004; palvia, pinjani, & sibley, 2007; palvia et al., 2017; sinclaire, simon, & wilkes, 2004), methodologies and research paradigms (avison et al., 2008; chen & hirschheim, 2004; claver et al., 2000; dwivedi & kuljis, 2008; ji et al., 2004; liu & myers, 2011; myers & liu, 2009; palvia & kakhki, 2016; palvia et al., 2015; palvia et al., 2004; palvia, mao, salam, & soliman, 2003; palvia et al., 2007; palvia et al., 2017), and authors and universities (avison et al., 2008; claver et al., 2000; dwivedi & kuljis, 2008; palvia et al., 2007; sinclaire et al., 2004). some studies have also adopted a combination of two or three approaches. in addition, some other studies have also examined research models and paradigmatic approaches. table 1 presents the main research studies carried out on topics and methodologies. as shown in table 1, with the exception of a study by palvia et al. (palvia et al., 2015), which covered the period ranging from 2004 to 2013, no comprehensive and recent study has been published in the top journals in the concerned field. two other recent studies in 2016 and 2017 have reviewed specific journals (namely information & management and global information technology management). given the significance of this issue, the current study addressed this research gap as such this study analysed the topics and methodologies used in the top eight journals of information system and their trends over a 12-year period (2007-2018). 3 research methodology cumbie, jourdan, peachey, dugo, and craighead (2005) propose a three-stage process for literature analysis projects. in the first stage, the researcher collects a series of relevant articles. in the second stage, the researcher classified the articles into an appropriate framework. in the third stage, the researcher evaluates and combines the collected data. accordingly, all the articles published in the top eight journals during 2007 to 2018 were first reviewed and coded. in the second stage, a two-dimensional framework encompassing topics and methodologies was created in order to classify the research papers, and each paper was then categorized with regard to the thematic and methodological dimensions. finally, the analysis addressed practices such as comparing, connecting, and explaining the trends and results (levy & ellis, 2006). the last step was to identify the gaps in research literature and predict future trends (webster & watson, 2002). according to these three stages, the methodology framework of the present research is presented in figure1. australasian journal of information systems mazaheri, lagzian & hemmat 2020, vol 24, research article research directions in information systems field 4 second methodology first methodology second topic first topic reviewed journals period surveyed scholar field study empirical studies dss is development information & management mis quarterly 1997-1981 claver et al. 2000 frameworks and conceptual model survey top five journals of information systems 1993-1997 palvia et al. 2003 mathematical modeling survey is development information systems resource management top seven journals of information systems 1993-2003 palvia et al. 2004 case study survey top eight journals of information systems 1991-2001 chen and hirschheim 2004 internet resource management/is management issues top six journals of information systems 1998-2003 d'aubeterre et al. 2005 case study survey resource management/is management issues is usage information & management 1998-2005 palvia et al. 2007 case study library research literature analysis frameworks conceptual modeling is management is development information systems journal 1991-2007 avison et al. 2008 surveys and library research case study is development is management european journal of information systems 1997-2007 dwivedi and kuljis 2008 case study survey the basket of top journals ais (six journals) 1998-2007 myers and liu 2009 laboratory experiment survey is usage /adoption electronic commerce top seven journals of information systems 2004-2013 palvia et al. 2015 case study survey electronic commerce /businessit and culture is usage /adoption global information technology management 2006-2015 palvia et al. 2016 field study survey is usage /adoption electronic commerce/business information & management 2004-2014 palvia et al. 2017 table 1. summary of previous studies based on the topics and methodologies used and their application priorities australasian journal of information systems mazaheri, lagzian & hemmat 2020, vol 24, research article research directions in information systems field 5 figure 1. research methodology (adopted from [7] and [13]) 3.1 journal selection to create a pool of papers, the top and valid journals in the field of information systems were primarily selected; hence, the selected journals were a basket of top journals nominated introduced by the college of senior scholars. the college of senior scholars encourages colleagues, deans, and department chairs to consider a "basket" of top eight journals in the field of information system research. the college of senior scholars consists of professionals in the field of information systems, who have served as editors-in-chief of the concerned journals, current and former icis1 chairs, program chairs, and current and former heads of ais2. this list covers the subject diversity as it contains a variety of methods and geographical information. in addition, the review processes of the concerned journals are accurate, their editorial staff are well-known and well-appreciated, and the journals have international audiences and contributors. in addition, the sjr index ("scimago journal & country rank"), reflecting the scientific quality of a journal, were also extracted, and all the selected journals had high sjr index. therefore, the papers from these eight journals had been collected from 2007 to 2018. a total of 3252 articles were reviewed, as listed in table 2 by journal. 3.2 classification the researchers developed a classification framework of information system research based on several criteria. sjr index number of papers number of issues journal 2.04 438 66 european journal of information systems 2.32 288 70 information systems journal 3.48 542 48 information systems research 1.82 385 143 journal of ais 2.12 335 48 journal of information technology 2.39 471 48 journal of mis 1.43 227 48 journal of strategic information systems 4.21 566 48 mis quarterly 3252 519 total table 2. list of selected journals, sjr index and number of papers (2007-2018) 1 the international conference on information systems 2 association for information systems phase 1: preparing pool of papers: selecting journals phase 2: classification: developing taxonomy coding and assigning papers phase 3: synthesis and evaluation: -determining research gaps -forecasting the future -other appropriate outputs australasian journal of information systems mazaheri, lagzian & hemmat 2020, vol 24, research article research directions in information systems field 6 a majority of the studies using the literature analysis have focused on a combination of factors such as research topic, research model, research methodology, paradigmatic research approach, authors, and universities. in the previous section, some of the most important studies which have addressed these dimensions were mentioned. the present study focused on research topic and methodology, which are explained in the following sections. 3.2.1 research methodologies table 3 presents the research methodologies concerned to categorize the selected articles. the methodologies were adopted from palvia et al. (palvia et al., 2015). during the coding process, two titles "social network analysis" and "machine learning, data mining, text mining, and web mining" were added. according to the previous researchers, these two methods have not fit in the previous categories and need to be considered as new methodologies. further, it is necessary to provide a more detailed explanation about some of the concerned methodologies. first, it should be noted that the case study methodology is typically qualitative in nature; however, it can incorporate quantitative data as a part of the mixed-methods design. accordingly, the case studies may involve both qualitative and quantitative research methods as such we distinguished it from the qualitative method. regarding the difference between content analysis and text mining, it can be mentioned that content analysis is “a technique for gathering and analysing the content of a text. the content refers to words, meanings, pictures, symbols, ideas, themes or any message that can be communicated” (neuman, 2003). content is coded into various categories or concepts depending on selected criteria (weber, 1988), and coding can be manual or computer aided. text mining is a computer-assisted technique to extract information and trends from large amounts of textual data (feldman & sanger, 2007; fuller, biros, & delen, 2011; tan, 1999). it is based on data mining, machine learning, and natural language processing techniques used for the texts under study. it mainly relies on computer programs and algorithms; thus, it is supposed to resolve the problems associated with the reliability of content analysis (milne & adler, 1999). text mining is thus to analyse large amounts of texts using algorithms and computer programs. in contrast, content analysis is mostly done with or without computer software to examine the underlying meaning and constructs existing in the text being analysed. text mining is a quantitative approach, while content analysis is a qualitative one. in survey and mathematical modelling methods, survey data can be used for regression analysis and/or hierarchical linear modelling; however, in the present study, only articles that use structured questionnaires are included in this category, and the other papers adopting analytical (e.g., formulaic, econometric, or optimization model) or descriptive (e.g., simulation) models fall into the mathematical modelling category. table 3 shows the final classification of the methodologies. australasian journal of information systems mazaheri, lagzian & hemmat 2020, vol 24, research article research directions in information systems field 7 methodology definition 1 speculation / commentary research that derives from thinly supported arguments or opinions with little or no empirical evidence 2 frameworks and conceptual model research that intends to develop a framework or a conceptual model. 3 literature review research that is based mainly on the review of existing literature. 4 literature analysis research that critiques, analyzes, and extends existing literature and attempts to build new groundwork, e.g., it includes a meta-analysis. 5 case study study of a single phenomenon (e.g., an application, a technology, a decision) in an organization over a logical time frame. 6 survey research that uses predefined and structured questionnaires to capture data from individuals. normally, the questionnaires are mailed (now, fax and electronic means are also used). 7 field research study of single or multiple and related processes/ phenomena in single or multiple organizations. 8 field experiment research in organizational setting that manipulates and controls the various experimental variables and subjects. 9 laboratory experiment research in a simulated laboratory environment that manipulates and controls the various experimental variables and subjects 10 mathematical modeling an analytical (e.g., formulaic, econometric or optimization model) or a descriptive (e.g., simulation) model is developed for the phenomenon under study. 11 qualitative research qualitative research methods are designed to help understand people and the social and cultural contexts within which they live. these methods include ethnography, action research, interpretive studies, and examination of documents and texts. 12 design science “design science creates and evaluates it artifacts intended to solve identified organizational problems”. it involves a rigorous process to design artifacts to solve observed problems, to make research contributions, to evaluate the designs, and to communicate the results to appropriate audiences. such artifacts may include constructs, models, methods, and instantiations. 13 secondary data a study that utilizes existing organizational and business data, e.g., financial and accounting reports, archival data, published statistics, etc. 14 content analysis a method of analysis in which text (notes) are systematically examined by identifying and grouping themes and coding, classifying and developing categories. 15 social network analysis study of the relationships of humans, organizations, groups, or any other set of related entities based on graph theory. 16 machine learning, data mining, text mining and web mining machine learning (ml) is the scientific study of algorithms and statistical models that computer systems use to perform a specific task without using explicit instructions, relying on patterns and inference instead. it is seen as a subset of artificial intelligence. data mining is the process of discovery of unknown relationships and patterns within the data. text mining (text analysis) is the process of extracting high-quality information from the text. web mining is a process of discovery of unknown and useful information from web data. table 3. methodologies in the information system research (adopted from (palvia et al., 2015)) 3.2.2 topics alavi and carlson (1992) first detected the most frequent research topics in the literature on information systems, and barki et al. (barki, rivard, & talbot, 1993) developed a keyword classification scheme, which can be used as a basis for investigation is research topics. following the publication of these papers, the other researchers improved this topic-based classification. some authors have adopted the construct classification proposed by ais for australasian journal of information systems mazaheri, lagzian & hemmat 2020, vol 24, research article research directions in information systems field 8 topic analysis (levy & ellis, 2006). in their paper, palvia et al. (2004) used the top three-levels introduced by barki et al. (barki, rivard, & talbot, 1988) to develop their topics, and later in 2015, they added some topics to the list with regard to the research discussions and topics and new trends. the same category was used in the present study, and some changes were made during the coding process and some topics were added. for example, we added topics such as ‘iot’, ‘bitcoin, block chain, fintech’, ‘digital transformation’ and ‘sharing economy, digital platform’ as new topics, and it services/digital services as a topic not mentioned in the previous classification. some changes were made in the title of the categories (e.g., crowdsourcing and outsourcing categories were merged), and one of the titles was also removed from the list. table 4 shows the final classification of the most common research topics. big data 1 artificial intelligence/business intelligence/data analytics/expert system 2 business process 3 cloud computing 4 customer relationship management (crm) 5 databases 6 decision support system & executive is 7 e-government 8 electronic commerce/business 9 end user computing 10 enterprise resource planning (erp) 11 environment of it: internal or external 12 global information technology (git) 13 group support systems 14 health information technology 15 innovation/ digital innovation/ digital entrepreneurship 16 internet of things 17 internet/web 18 inter-organizational systems 19 is design and development 20 is education 21 is evaluation 22 is functional applications 23 is implementation 24 is management and planning 25 is research 26 is staffing 27 is usage/adoption 28 it and culture/ethic 29 it services/digital services 30 it value/ business value of it 31 knowledge management 32 media and communications 33 mobile computing 34 organizational design 35 outsourcing and offshoring/ crowdsourcing/ crowd funding 36 project management 37 security and privacy 38 social media and social computing 39 social networks 40 australasian journal of information systems mazaheri, lagzian & hemmat 2020, vol 24, research article research directions in information systems field 9 societal issues 41 software and programming languages 42 supply chain management 43 sustainability 44 team/virtual teams/collaboration 45 telecommunications and networking 46 bitcoin / block chain / fintech 47 sharing economy/digital platform 48 digital transformation 49 table 4. list of topic categories 3.3 coding while each article normally has a main topic and a dominant methodology, some papers deal with multiple topics and may use several methodologies. as a result, to include to such papers in the present study, in the coding file for each paper, three columns were designed for the topic(s) and two columns for methodology. accordingly, a total of 3252 papers published from 2007 to 2018, belonging to a basket of the top eight is journals, were reviewed and codified using the proposed framework. to determine the research topics, each paper was analysed by title, keywords, and abstract. this approach has been adopted by several authors (avison et al., 2008; farhoomand & drury, 1999; palvia et al., 2015). to determine the research methodologies, the abstract and content of the papers were reviewed. three experts (including the researcher and two assistants) coded the articles individually. to achieve reliable results and increase reliability of this process, the following points were considered by the coders: • in order to increase the coding uniformity and reduce the ambiguity, the coders had several meetings before coding and exchanged the necessary information. finally, they, to some extent, came to an agreement on the encoding process. • in the methodology section, standard definitions of the methodologies were provided for the coders to avoid coding mistakes. • to remove the effect of the chance from the calculations, the inter-coder reliability was estimated. inter-coder reliability was calculated in a two-phase process. in the first phase, the coders encoded 50 papers independently and then compared their results. table 5 shows the results of the first phase. coder 1 2 3 1 2 75% (topic) 63% (methodology) 3 78% (topic) 71% (methodology) 89% (topic) 69% (methodology) table 5. phase one of inter-coder reliability (by topic and methodology) australasian journal of information systems mazaheri, lagzian & hemmat 2020, vol 24, research article research directions in information systems field 10 as shown in table 5, inter-coder reliability is not at 90%, a level of significance recommended in the relevant literature (palvia et al., 2004); therefore, the authors discussed their findings in several sessions in the cases of inconsistency in order to reach a common understanding. in the second phase of the coding process, the coders coded 25 papers, and the results of intercoder reliability are presented in table 6. coder 1 2 3 1 2 93% (topic) 89% (methodology) 3 94% (topic) 91% (methodology) 98% (topic) 93% (methodology) table 6. phase two inter-coder reliability (by topic and methodology) as it can be noticed, all coders achieved reliability at an acceptable level in this phase. 4 results in this section, the results are presented based on the topic and methodology of the concerned articles, for which the main thematic points and methodology as well as the major trends are discussed separately. 4.1 the process of using information systems methodologies table 7 lists the most frequently used research methodologies in the top eight journals during a twelve-year period. since two methodologies were considered in coding for each paper (since a paper may use a combination of two methodologies, for example, survey and qualitative methods), two columns (namely primary and secondary methodologies) are devoted to methodology in the table. for each methodology in table 7, four values are presented: number of times a methodology was used as the primary methodology, number of times a methodology was used as a secondary methodology, total number of times a methodology was used, and the percentage of using a methodology. as it can be seen in table 7, in the journals studied during a twelve-year period (2007-2018), the survey method was the most common methodology (19.3%). the other frequently used methodologies were mathematical modelling (17%), case study (13.8%), qualitative research (11%) and literature review (7%), respectively. table 8 shows the significant trends by year and the priority of methodologies per year during a 12-year period. figure 2 also graphically illustrates the methodologies used during 2007 to 2018. australasian journal of information systems mazaheri, lagzian & hemmat 2020, vol 24, research article research directions in information systems field 11 methodology primary methodolo gy count secondary methodolo gy count total papers total (%) rank by total survey 609 62 671 19.3% 1 mathematical modeling 560 30 590 17% 2 case study 464 14 478 13.8% 3 qualitative research 354 29 383 11% 4 literature review 243 0 243 7% 5 laboratory experiment 197 15 212 6.1% 6 speculation/commentary 194 1 195 5.6% 7 frameworks and conceptual model 133 5 138 4% 8 secondary data 104 26 130 3.7% 9 field research 101 9 110 3.2% 10 field experiment 99 4 103 3% 11 design science 59 1 60 1/7% 12 literature analysis 55 0 55 1/6% 13 machine learning, data mining, text mining and web mining 44 2 46 1/3% 14 content analysis 21 14 35 1% 15 social network analysis 15 6 21 0/6% 16 total 3252 218 3470 100% table 7. rank of research methodology based on count and percentage of papers using a particular methodology australasian journal of information systems mazaheri, lagzian & hemmat 2020, vol 24, research article research directions in information systems field 12 r an k methodology 20 07 20 08 20 09 20 10 20 11 20 12 20 13 20 14 20 15 20 16 20 17 20 18 t o ta l 1 survey 61 46 51 61 74 59 49 53 69 45 56 47 671 2 mathematical modeling 31 26 30 44 42 77 55 55 47 50 61 72 590 3 case study 54 50 42 33 45 40 46 47 31 33 25 32 478 4 qualitative research 21 31 34 39 27 18 36 35 21 31 50 40 383 5 literature review 33 22 22 19 27 30 21 12 19 16 13 9 243 6 laboratory experiment 9 20 17 13 24 10 23 20 15 21 19 21 212 7 speculation / commentary 12 17 6 36 25 31 8 18 17 17 4 4 195 8 frameworks and conceptual model 15 11 15 10 10 17 13 9 7 8 8 15 138 9 secondary data 4 9 10 17 10 17 12 7 20 13 6 5 130 10 field research 7 6 5 12 9 18 10 13 5 5 12 8 110 11 field experiment 3 5 10 8 6 6 12 8 12 15 13 5 103 12 design science 1 9 3 2 1 7 4 6 8 8 9 2 60 13 literature analysis 5 5 5 4 4 6 6 4 5 5 2 4 55 14 machine learning, data mining, text mining and web mining 1 1 0 2 0 1 2 3 0 5 10 21 46 15 content analysis 4 1 4 4 0 3 2 3 5 5 2 2 35 16 social network analysis 2 1 2 2 2 1 1 2 0 4 1 3 21 total 263 260 256 306 306 341 300 295 281 281 291 290 3470 table 8 methodology trends over the years 2007-2018 survey method was the most-oft used research method in almost each year. the mathematical model also experienced a rising trend during this period and reached its peak in 2012. the other significant trend was a decline in the use of conceptual modelling and framework australasian journal of information systems mazaheri, lagzian & hemmat 2020, vol 24, research article research directions in information systems field 13 methodology. although the use of this methodology reached its peak in 2012, it was least used in 2015, 2016, and 2017. the use of this methodology, however, increased in 2018 once more. figure 2 the top 5 methodologies in the period (2007-2018) 4.2 topic trends of information systems table 9 shows the frequency and percentage of the topics covered in all papers published from 2007 to 2018 in the top eight journals by year. electronic commerce/business, information system research, and information system usage/adoption are the most frequently studied topics, with the “electronic commerce/business” being at the top of the list, having an 8.95% frequency. information system research and information system usage/adoption are ranked next with the frequencies of 7.82 and 7.36 percent, respectively. in addition to these three topics, seven other common topics are information system management and planning (6.89%), security and privacy (6.31%), software and programming language (4.72%), outsourcing and offshoring/crowdsourcing (4.54%), social media and social computing (4.41%), knowledge management (3.81%), and it value/ business value of it (3.31%). regarding the topic trends, there is an increasing attention to social media, social network, and big data. moreover, iot was also discussed for the first time in 2016. bitcoin / block chain / fintech, sharing economy/digital platform and digital transformation were also first presented as research topics in 2017 and 2018. figure 3 illustrates an analysis of these ten frequent topics. 0 10 20 30 40 50 60 70 80 90 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 survey mathematical modeling case study qualitative research literature review australasian journal of information systems mazaheri, lagzian & hemmat 2020, vol 24, research article research directions in information systems field 14 figure 3. the top ten topics in the period (2007-2018) 0 10 20 30 40 50 60 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 electronic commerce/business is research is usage/adoption is management and planning security and privacy software and programming languages outsourcing and offshoring/ crowdsourcing/ crowd funding social media and social computing knowledge management australasian journal of information systems mazaheri, lagzian & hemmat 2020, vol 24, research article research directions in information systems field 15 r an k topic 20 07 20 08 20 09 20 10 20 11 20 12 20 13 20 14 20 15 20 16 20 17 20 18 t o ta l p er ce n t 1 electronic commerce / business 33 30 27 41 21 34 26 31 15 26 29 30 343 8.95% 2 is research 17 27 19 16 36 53 26 25 27 22 21 11 300 7.82% 3 is usage/adoption 39 26 21 17 31 24 24 24 20 18 24 14 282 7.36% 4 is management and planning 17 12 17 22 33 21 29 28 23 20 19 23 264 6.89% 5 security and privacy 16 11 19 24 15 19 15 18 18 31 31 25 242 6.31% 6 software and programming languages 22 15 17 27 17 16 15 8 11 8 15 10 181 4.72% 7 outsourcing and offshoring / crowdsourcing / crowd funding 7 22 14 16 8 15 16 14 13 17 12 20 174 4.54% 8 social media and social computing 3 3 4 8 14 15 20 19 23 24 12 24 169 4.41% 9 knowledge management 16 13 7 11 15 18 9 9 14 17 5 12 146 3.81% 10 it value / business value of it 5 6 8 6 11 18 9 14 10 7 19 14 127 3.31% 11 social networks 2 6 0 14 5 6 10 10 17 12 19 13 114 2.97% 12 is design and development 13 8 11 16 9 13 13 5 2 7 10 6 113 2.95% 13 health information technology 20 3 5 3 17 5 5 8 9 11 18 9 113 2.95% 14 innovation / digital innovation / digital entrepreneurship 8 7 5 9 5 7 8 7 9 7 16 9 97 2.53% 15 internet / web 3 2 7 8 8 13 7 9 8 5 6 6 82 2.14% australasian journal of information systems mazaheri, lagzian & hemmat 2020, vol 24, research article research directions in information systems field 16 r an k topic 20 07 20 08 20 09 20 10 20 11 20 12 20 13 20 14 20 15 20 16 20 17 20 18 t o ta l p er ce n t 16 team / virtual teams / collaboration 5 6 12 8 4 11 2 3 6 2 4 12 75 1.96% 17 business intelligence / data analytics/expert system 2 2 3 2 3 13 3 8 4 10 12 9 71 1.85% 18 project management 5 5 7 8 2 2 9 9 3 6 5 3 64 1.67% 19 mobile computing 3 3 9 3 1 2 6 9 12 5 3 5 61 1.59% 20 is implementation 3 5 6 4 7 6 11 4 2 3 1 3 55 1.43% 21 media and communications 5 6 3 8 2 3 5 9 2 7 1 3 54 1.41% 22 decision support system & executive is 2 3 2 6 6 2 1 6 3 3 8 8 50 1.30% 23 is staffing 8 3 2 1 5 6 3 3 4 5 2 7 49 1.28% 24 telecommunications and networking 4 3 10 5 6 5 6 3 2 1 2 1 48 1.25% 25 e-government 13 7 1 5 7 3 3 2 1 3 0 2 47 1.23% 26 it services / digital services 0 3 4 1 2 3 5 1 7 3 7 10 46 1.20% 27 it and culture / ethic 7 3 3 3 4 3 3 5 8 1 0 5 45 1.17% 28 inter-organizational systems 3 7 4 6 5 2 4 4 1 1 5 2 44 1.15% 29 enterprise resource planning (erp) 6 6 4 6 3 3 2 0 4 3 1 0 38 0.99% 30 business process 4 5 2 6 2 4 2 3 4 3 2 0 37 0.97% 31 sustainability 0 0 1 4 8 0 4 2 1 5 9 1 35 0.91% australasian journal of information systems mazaheri, lagzian & hemmat 2020, vol 24, research article research directions in information systems field 17 r an k topic 20 07 20 08 20 09 20 10 20 11 20 12 20 13 20 14 20 15 20 16 20 17 20 18 t o ta l p er ce n t 32 societal issues 0 5 1 1 2 1 4 7 3 4 1 3 32 0.83% 33 big data 0 0 0 0 0 1 0 0 9 11 3 7 31 0.81% 34 cloud computing 0 0 2 0 1 1 3 2 4 6 2 8 29 0.76% 35 supply chain management 2 2 7 3 7 1 2 0 1 1 1 1 28 0.73% 36 end user computing 2 4 0 4 2 2 0 3 4 0 2 0 23 0.60% 37 bitcoin / block chain / fintech 0 0 0 0 0 0 0 0 0 0 6 11 17 0.44% 38 databases 2 1 1 3 2 3 1 1 0 1 0 1 16 0.42% 39 sharing economy / digital platform 0 0 0 0 0 0 0 0 0 0 3 13 16 0.42% 40 is education 2 1 2 6 0 0 1 1 0 0 0 1 14 0.37% 41 environment of it: internal or external 0 0 0 3 3 2 1 1 0 1 0 0 11 0.29% 42 group support systems 0 3 3 1 1 2 0 0 0 0 0 0 10 0.26% 43 customer relationship management (crm) 1 0 0 1 3 3 0 0 0 0 1 1 10 0.26% 44 global information technology (git) 1 2 1 2 0 1 0 0 0 1 1 0 9 0.23% 45 is functional applications 2 1 0 1 0 1 1 0 0 0 0 0 6 0.16% 46 internet of things 0 0 0 0 0 0 0 0 0 1 3 1 5 0.13% 47 is evaluation 1 0 1 1 0 0 0 1 0 0 0 0 4 0.10% australasian journal of information systems mazaheri, lagzian & hemmat 2020, vol 24, research article research directions in information systems field 18 r an k topic 20 07 20 08 20 09 20 10 20 11 20 12 20 13 20 14 20 15 20 16 20 17 20 18 t o ta l p er ce n t 48 organizational design 0 0 1 1 0 1 1 0 0 0 0 0 4 0.10% 49 digital transformation 0 0 0 0 0 0 0 0 0 0 0 3 3 0.08% total 304 277 273 341 333 364 315 316 304 321 341 347 3834 100% note: the total number of topics is more than the number of papers because of possible multiple topics per paper. table 9. ranking of research topics over years 2007-2018 by year australasian journal of information systems mazaheri, lagzian & hemmat 2020, vol 24, research article research directions in information systems field 19 4.3 publication patterns of journals although most information system journals accept and publish articles on all aspects of the information system, the journals may have their own specific orientations based on their specific conditions, including history, editors, reviewers, and even authors. in this section, the topics and research methods of each journal are examined and compared. table 10 shows the publication patterns of the top ten topics in the eight journals, and table 11 presents the research methods adopted in the studies published in the concerned journals. top topics (10) ejis isj isr jais jit jmis jsis misq electronic commerce / business 26 14 106 20 16 78 17 66 is research 58 26 17 53 69 9 7 61 is usage / adoption 45 28 34 48 22 31 15 59 is management and planning 50 23 40 25 32 26 33 35 security and privacy 30 13 43 27 6 62 12 49 software and programming languages 28 17 40 18 9 34 13 22 outsourcing and offshoring / crowdsourcing / crowd funding 13 9 33 7 25 26 30 31 social media and social computing 4 10 45 13 8 32 9 48 knowledge management 14 27 16 10 16 23 20 20 it value / business value of it 11 3 34 11 8 24 11 25 table 10. number of papers representing the top 10 topics, by journal as it can be seen in table 10, some topics have been of a greater interest in some journals. for example, the topic ‘electronic commerce’ is most frequently addressed in information systems research journal, and the topic ‘information system research’ is most frequently observed in misq and the journal of information technology. methodology ejis isj isr jais jit jmis jsis misq survey 123 67 90 71 41 118 60 101 case study 101 87 22 46 87 28 60 47 frameworks and conceptual model 21 13 6 38 14 7 10 29 qualitative research 78 75 26 44 40 31 36 53 speculation/ commentary 29 4 27 23 73 0 8 31 literature review 18 23 5 70 46 16 27 38 literature analysis 18 5 1 4 12 1 5 9 field research 10 8 24 11 7 16 3 31 field experiment 6 2 34 6 3 23 1 28 laboratory experiment 13 11 48 36 3 56 2 43 mathematical modeling 22 4 237 24 8 139 14 142 design science 8 3 3 16 0 14 2 14 secondary data 8 3 44 6 7 31 10 21 content analysis 9 5 4 6 2 5 0 4 social network analysis 2 2 8 2 3 3 1 0 machine learning, data mining, text mining and web mining 3 0 8 1 3 19 0 12 table 11. number of papers representing each methodology by journal australasian journal of information systems mazaheri, lagzian & hemmat 2020, vol 24, research article research directions in information systems field 20 regarding the research methods, the european journal of information systems and the journal of mis have published the largest number of papers with survey method; however, information systems research journal contains the largest number of papers using mathematical modelling. 5 discussion and conclusion information system is an emerging and unique discipline, which is constantly experiencing rapid changes as such understanding the topics and research methods used in this field and the trend of the changes in this regard can be useful for researchers in the field of information systems. accordingly, this study detected the trends in the field of information system research over a twelve-year period (2007-2018) using a comprehensive literature analysis approach. 5.1 topics and their trends during 2007-2018 the first objective of this study was to investigate the topics used by researchers and their trends during a 12-year period. to this end, 3252 articles from the top eight information system journals were reviewed and analysed by three coders. accordingly, several findings were achieved. first, the analyses showed that electronic commerce/business received higher attention among the other topics. this field of research dramatically caught the attention of many researchers with the advent of the internet. with revolutionary advances in internet technology and relevant software and applications to facilitate e-commerce, the research in this area continues to grow. this finding is in line with the findings of previous studies (e.g., palvia et al., 2015). information system research is another area which has been addressed frequently. in some other review studies (avison et al., 2008; palvia et al., 2015), this field was also ranked third. the present study refers to research papers on information system dimensions such as models, methods, paradigms and theories. considering the emerging and interdisciplinary nature of information system and the need to address paradigms, models, new methodologies and theories in this field, this topic seems to be of interest in future studies. the third area of research is the usage and acceptance of information systems. in a large number of previous studies, this area has been among the top three issues (palvia et al., 2015; palvia et al., 2004; palvia et al., 2007; sinclaire et al., 2004). the turning point of research on the usage and acceptance of information systems was emerged following davis' (davis, 1989) study on technology acceptance model. afterwards, many researchers have increasingly addressed this issue in various journals so that the research on this topic has remarkably grown. 5.2 methodologies and their trends during 2007-2018 as the second objective of the study, the researchers were to detect the most common research methodologies and their trends. the findings showed that most of the published papers are empirical studies and have used a survey method for data collection. this finding is consistent with the findings of some other studies (chen & hirschheim, 2004; myers & liu, 2009; palvia et al., 2015; palvia et al., 2004; palvia et al., 2003; palvia et al., 2007). information system researchers generally prefer to use survey method so that more than a quarter of the reviewed papers adopted this method to collect data. however, the results also revealed that that some recent researchers have also adopted qualitative and case study methods since these methods provide a deep, precise, and clear understanding of a phenomenon under study. the second australasian journal of information systems mazaheri, lagzian & hemmat 2020, vol 24, research article research directions in information systems field 21 most popular methodology was the mathematical model. in this study, a wide range of methods such as analytical models, formula-dependent models, optimization models, algorithms, multi-criteria decision-making, bayesian networks, hierarchical linear modelling, regression analysis, panel co-integration analysis, multiple indicators-multiple causes were also in this category. such an increased attention indicates that the authors are seeking to increase the accuracy of the research through adopting mathematical, quantitative, and complex methods. 5.3 research gaps and future trends the third objective of this paper was to identify research gaps and predict the future trends in the field of information system research. in this regard, a few points should be noted. first, the topics ‘iot’, ’bitcoin / block chain / fintech’, ‘sharing economy/digital platform’ and digital transformation have been recently raised in the field of information systems as such they have not been included in previous classifications; hence, the future researchers are more likely to focus on these issues. researchers have identified these issues based on the analysis of papers published during the research period. in other words, in recent years due to rapid innovation in information technology, new subject areas and issues emerging from these resulted in publications that did not fit previously identified subject areas and issues. researchers identified these issues by reviewing papers during the coding process. these categories were confirmed by attending conferences, typically, the first places where new research comes to light. evidence to supporting the identification of new topics was obtained by examining conference topic lists. for example, in the international conference on information systems (2020), in track descriptions section, the following topics were introduced as new research topics: ‘block chain, dlt, and fintech’, ‘sharing economy, platforms, and crowds’, ‘digital innovation, entrepreneurship, and new business models’, and ‘iot and the smart connected world’. in pacific asia conference on information systems (2020), in track descriptions section, the following topics were considered as new research topics: ‘block chain, fintech, and emerging technologies’, ‘sharing economy’, ‘digital transformation and social impact in the mena region’. second, the topic ‘big data’ was first posed in 2012 in the concerned journals and experienced a relatively large growth in 2015 and 2016. today, with the advent of technologies, various devices and equipment such as mobile phones, environmental sensors, various software logos, cameras, microphones, rfid detection devices, wireless sensor networks, social networks, and data production are increasing at a dazzling speed. accordingly, the large volume of information available to organizations, on the one hand, and the complexity of societies and organizations and the increasing need for decisive decisions based on this comprehensive information, on the other hand, require an established alignment between business needs and information system research in this field. this topic, thus, seems to be one of the would-be research areas. the other interesting trend is social media, which has also been of great interest over the past twelve years. given its high popularity among users and its widespread use, this topic is likely to be the focus of attention in the future. regarding the research methodology, a significant trend is to decrease the use of framework methods and conceptual models. although it was greatly used in 2012, it was less addressed australasian journal of information systems mazaheri, lagzian & hemmat 2020, vol 24, research article research directions in information systems field 22 in 2015, 2016, and 2017. once more, this topic became the hotbed of debate in 2018. these results suggest that the journals are more inclined to publish the real research more than the frameworks guiding the research studies. the other notable trend is the increasing use of machine learning and data mining over the last two years. according to the descriptions on big data and given the enormous amount of information available, these methods are suggested for the analyses as such they would be more widely used in the future. finally, the present study contributes to the existing literature by providing a detailed and relatively comprehensive overview of the information system studies in authentic journals and comparing the results. moreover, the novelty of this research lies in its being up-to-date and including a larger number of top journals. one of the limitations of the present research was that only eight journals were investigated. although a large number of the leading experts and authors in the field of information system have published their papers in these journals, there are some other specialized journals such as decision support systems, journal of global information technology management, ieee, information and management, which were not included in this study. these journals have their own rules and themes in selecting topics and methodologies; hence, the results of this study could be generalized to all the research information management systems with little caution. another limitation is concerned with the classification scheme for the research topics. despite the fact that a highly comprehensive list of topics was developed based on the previous research, the topic list turned out not to be completely comprehensive, and the topics of some papers were not included in these categories. accordingly, some of the issues were inevitably added during the coding process. the same limitation was noticed regarding the research methodology. one more limitation was the level of details for the topics. in order to preserve meaningfulness, topic categories should not be too broad or too limited. in this regard, we were to have an appropriate level of topic classification, to the best knowledge of the research team. however, the findings were at the same time affected by the type of thematic categories to some extent. in other words, the categories were affected by the topics grouped together or separated from each other. for example, a team, a virtual team, and collaboration were grouped together in one category; however, social networks and social media were classified separately. this came true for the level of methodological details. finally, since coding a large number of papers requires the division of tasks, and coding was carried out by three coders, there may be differences in codes. we spared our efforts to minimize such concerns through evaluating the coders using two sets of papers and reaching a common understanding. moreover, the inter-coder reliability was higher than the recommended values, indicating that there was a considerable degree of common understanding and agreement among the coders. considering the scope of the research, future researchers are recommended to consider other journals and compare them with the findings of the present research. furthermore, more comprehensive frameworks are suggested to be considered for future work. in other words, in addition to topics and methodologies, the future studies should include models and approaches, paradigms, writers, and universities. further, the future researchers can examine the adopted research methodologies to determine which methodology and topic are more consistent. australasian journal of information systems mazaheri, lagzian & hemmat 2020, vol 24, research article research directions in information systems field 23 references alavi, m., & carlson, p. (1992). a review of mis research and disciplinary development. journal of management information systems, 8(4), 45-62. avison, d. e., dwivedi, y. k., fitzgerald, g., & powell, p. (2008). the beginnings of a new era: time to reflect on 17 years of the isj. information systems journal, 18(1), 5-21. barki, h., rivard, s., & talbot, j. (1988). an information systems keyword classification scheme. mis quarterly, 299-322. barki, h., rivard, s., & talbot, j. (1993). a keyword classification scheme for is research literature: an update. mis quarterly, 209-226. chen, w., & hirschheim, r. (2004). a paradigmatic and methodological examination of information systems research from 1991 to 2001. information systems journal, 14(3), 197235. claver, e., gonzález, r., & llopis, j. (2000). an analysis of research in information systems (1981–1997). information & management, 37(4), 181-195. cumbie, b. a., jourdan, z., peachey, t., dugo, t. m., & craighead, c. w. (2005). enterprise resource planning research: where are we now and where should we go from here? jitta: journal of information technology theory and application, 7(2), 21. d'aubeterre, f., palvia, p., & steevens, j. (2005). a meta-analysis of current global information systems research. amcis 2005 proceedings, 48. davis, f. d. (1989). perceived usefulness, perceived ease of use, and user acceptance of information technology. mis quarterly, 319-340. dwivedi, y. k., & kuljis, j. (2008). profile of is research published in the european journal of information systems. european journal of information systems, 17(6), 678-693. farhoomand, a., & drury, d. h. (1999). a historiographical examination of information systems. communications of the ais, 1(5es), 4. feldman, r., & sanger, j. (2007). the text mining handbook: advanced approaches in analyzing unstructured data: cambridge university press. fuller, c. m., biros, d. p., & delen, d. (2011). an investigation of data and text mining methods for real world deception detection. expert systems with applications, 38(7), 8392-8398. the international conference on information systems. retrieved 12/15/2019, from https://icis2020.aisconferences.org/track-descriptions/ ji, s., min, q., & han, w. (2004). overview of information systems research in china: an empirical study. pacis 2004 proceedings, 13. levy, y., & ellis, t. j. (2006). a systems approach to conduct an effective literature review in support of information systems research. informing science, 9. liu, f., & myers, m. d. (2011). an analysis of the ais basket of top journals. journal of systems and information technology, 13(1), 5-24. milne, m. j., & adler, r. w. (1999). exploring the reliability of social and environmental disclosures content analysis. accounting, auditing & accountability journal, 12(2), 237-256. australasian journal of information systems mazaheri, lagzian & hemmat 2020, vol 24, research article research directions in information systems field 24 myers, m., & liu, f. (2009). what does the best is research look like? an analysis of the ais basket of top journals. pacis 2009 proceedings, 61. neuman, w. (2003). social research methods: qualitative and quantitative approaches . boston, ma: allyn & bacon. pacific asia conference on information systems. retrieved 12/15/2019, from https://www.pacis2020.org/conference-tracks/ palvia, p., & kakhki, m. d. (2016). methodological and topic trends in jgitm: a 10-year retrospect: taylor & francis. palvia, p., kakhki, m. d., ghoshal, t., uppala, v., & wang, w. (2015). methodological and topic trends in information systems research: a meta-analysis of is journals. cais, 37, 30. palvia, p., leary, d., mao, e., midha, v., pinjani, p., & salam, a. (2004). research methodologies in mis: an update. communications of the association for information systems, 14(1), 24. palvia, p., mao, e., salam, a., & soliman, k. s. (2003). management information systems research: what's there in a methodology? communications of the association for information systems, 11(1), 16. palvia, p., pinjani, p., & sibley, e. h. (2007). a profile of information systems research published in information & management. information & management, 44(1), 1-11. palvia, p., yk, c. p., kakhki, m. d., ghoshal, t., uppala, v., & wang, w. (2017). a decade plus long introspection of research published in information & management. information & management, 54(2), 218-227. scimago journal & country rank. retrieved 12/15/2019, from https://www.scimagojr.com/journalrank.php sinclaire, j., simon, j., & wilkes, r. (2004). research directions in mis: an assessment of current status. amcis 2004 proceedings, 529. stemler, s. (2001). an overview of content analysis. practical assessment, research & evaluation, 7(17), 137-146. tan, a.-h. (1999). text mining: the state of the art and the challenges. paper presented at the proceedings of the pakdd 1999 workshop on knowledge disocovery from advanced databases. weber, r. (1988). basic content analysis, sage university paper series on quantitative applications. the social science, series(07-049). webster, j., & watson, r. t. (2002). analyzing the past to prepare for the future: writing a literature review. mis quarterly, xiii-xxiii. copyright: © 2020 mazaheri, lagzian & hemmat. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems mazaheri, lagzian & hemmat 2020, vol 24, research article research directions in information systems field 25 which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v24i0.2045 paper title (paper title style) australasian journal of information systems zobair, sanzogni & sandhu 2020, vol 24, research article telemedicine adoption barriers 1 telemedicine healthcare service adoption barriers in rural bangladesh khondker mohammad zobair department of business strategy and innovation griffith university zobair@hotmail.com louis sanzogni department of business strategy and innovation griffith university kuldeep sandhu department of business strategy and innovation griffith university abstract this article investigates potential barriers to telemedicine adoption in centres hosted by rural public hospitals in bangladesh. little is known of the barriers related to telemedicine adoption in this context. analysis of data collected from rural telemedicine patients identified seven broad categories of barriers: lack of organisational effectiveness, information and communication technology infrastructure, quality of care, allocation of resources, health staff motivation, patient satisfaction and trustworthiness. their significance is explored. this research is based on the quantitative analysis of a data set of 500 telemedicine patients, from rural areas in bangladesh. a conceptual model showing the interaction of pre-determined classes of barriers was established and hypotheses set up and tested using partial least squares structural equation modelling. exemplary barriers to telemedicine adoption were identified and confirmed (p<.01) namely, lack of organisational effectiveness, health staff motivation, patient satisfaction, and trustworthiness collectively explaining 62% of the variance in barriers to adoption and providing for the first-time empirical support of their existence. these barriers offer considerable resistance to the adoption and maintenance of current telemedicine projects in rural bangladesh. further, lack of information and communication technology infrastructure, allocation of resources and quality of care are indirect barriers affecting successful deployment of telemedicine in rural settings. these findings illuminate adoption impediments faced by existing telemedicine projects and institutionalise favourable policy guidelines to improve bangladesh’s and similar emerging economies’ healthcare industries. policy interventions and recommendations are provided, including current research limitations leading to opportunities for future research. keywords: adoption, barriers, ict, pls-sem, rural and remote areas, telemedicine 1 introduction telemedicine can provide improved and more flexible primary healthcare support to rural and underprivileged people in developing countries. its primary objective is to improve healthcare services to geographically disadvantaged and medically underserved populations by providing high-quality care at reduced cost, offering greater convenience (lerouge & garfield, 2013). though there is high interest in telemedicine worldwide, the implementation rate is lagging (bullock, vehe, zhang, & correll, 2017) due to organisational, infrastructural, australasian journal of information systems zobair, sanzogni & sandhu 2020, vol 24, research article telemedicine adoption barriers 2 strategic, technological, financial, ethical, legal, and human resources barriers (lerouge & garfield, 2013). this article investigates potential barriers impeding the adoption of telemedicine healthcare services in centres hosted by rural bangladesh’s public hospitals serving medically disadvantaged rural populations. bangladesh’s healthcare system is highly pluralistic (ahmed et al., 2014), consisting of public, private and non-government organisations (ngos), and healthcare systems aided by international donor agencies (hamid, ahsan, begum, & asif, 2015). many urban and rural communities depend on non-institutional health facilities served by private health practitioners (paul, 1983). telemedicine must be integrated into healthcare industries to better manage and coordinate bangladesh’s healthcare systems (gibson et al., 2011). telemedicine could hold substantial promise (mars, 2013) to achieve bangladesh’s political health agenda (nissen & lindhardt, 2017), including facilitating better access to health services for rural communities (gibson et al., 2011). 1.1 research rationale there is growing consensus that telemedicine is a solution to delivering affordable healthcare support. telemedicine can improve health access, service quality, clinician and patient satisfaction (zhang, luo, nie, & zhang, 2017), and can link rural and remote communities with distant care reducing travel time and cost (jang-jaccard, nepal, alem, & li, 2014). due to a significant shortage of highly qualified health professionals, there has been growing demand for ict-based health services in low and middle income countries (ahmed et al., 2014). telemedicine holds much promise (mars, 2013). however, awareness among stakeholders as well as issues of privacy, confidentiality, defensive medical practices and fears (parimbelli et al., 2018), unfamiliarity with required skills and knowledge, and difficulties handling ict applications (koivunen & saranto, 2018) form complex barriers to broader telemedicine adoption in bangladesh clinical practices (parimbelli et al., 2018). these barriers give impetus to this research. 1.2 telemedicine in bangladesh the government of bangladesh is aggressively introducing telemedicine healthcare services to ensure national success in healthcare intervention. e-health services were initiated in 1998 under the ministry of health and family welfare (mohfw) to enhance public healthcare systems (m. r. hoque & bao, 2015). telemedicine was established in 1999 and the bangladesh telemedicine association in 2001 but due to insufficient attention, telemedicine has remained beyond the reach of much of the disadvantaged population (nessa, ameen, ullah, & kwak, 2008). in 2010, telemedicine was adopted in public hospitals under mohfw in partial fulfilment of the 2021 digital bangladesh vision. there are 84 active telemedicine centres in 57 of bangladesh’s 488 upazila (i.e., subdistrict) hospitals. the integration of 27 specialised, district-level and medical college hospitals that provide telemedicine support to existing rural telemedicine centres is considered a significant shift in bangladesh’s public healthcare sector. a central database system has been established to store data from health facilities, including data from all public health professionals (ahmed et al., 2014). despite growing demand for telemedicine, the adoption rate is slow and below expectations (menachemi, burke, & ayers, 2004). nevertheless, while telemedicine projects in bangladesh have faced challenges since 1999 (nessa et al., 2008), some have become fully operational, providing large-scale services and benefits meriting its continued consideration (williams, bhatti, alam, & nikolian, 2018), australasian journal of information systems zobair, sanzogni & sandhu 2020, vol 24, research article telemedicine adoption barriers 3 and giving strength to patients’ attitudes, awareness, and understandings of the viability of telemedicine projects. some empirical studies (ahmed et al., 2014; r. hoque & sorwar, 2017), have investigated the status of e-health and m-health initiatives in bangladesh. for example, government and nongovernment sectors (i.e., profit, non-profit and ngos) (m. r. hoque, karim, & amin, 2015), contribute to e-health and m-health services but are embryonic and need substantial research to overcome the challenges of successful adoption (m. r. hoque, 2016). in existing bangladeshrelated literature that focuses on e-health and m-health adoption issues, no studies were found on telemedicine adoption, nor have correlations been identified between providerreported barriers and telemedicine adoption. this represents a considerable knowledge gap duly addressed by this study. 2 review of the literature (hypotheses and conceptual model) the reviewed literature suggests that ict-based healthcare research is varied and influenced by researchers’ perspectives (vishwanath & scamurra, 2007). no conceptual or theoretical frameworks concerning the relationships between telemedicine barriers and adoption were found. robust prior studies in telemedicine, e-health, telehealth, and m-health (alaboudi et al., 2016; bigna, noubiap, plottel, kouanfack, & koulla-shiro, 2014; boonstra & broekhuis, 2010; bullock et al., 2017; cajita, hodgson, lam, yoo, & han, 2018; de bustos, moulin, & audebert, 2009; faber, van geenhuizen, & de reuver, 2017; freed et al., 2018; gravel, légaré, & graham, 2006; jang-jaccard et al., 2014; johnson, 2001; leaming, 2007; lerouge & garfield, 2013; lin, lin, & roan, 2012; lohmann, muula, houlfort, & de allegri, 2018; modi, portney, hollenbeck, & ellimoottil, 2018; moffatt & eley, 2011; ramtohul, 2015; rogove, mcarthur, demaerschalk, & vespa, 2012; scott kruse et al., 2018; scott & mars, 2013; van dyk, 2014; whitten, holtz, meyer, & nazione, 2009) were used to develop and test the hypotheses regarded as mutually exclusive and exhaustive when considered from a model parsimony perspective. these seven broad categories of barriers are encapsulated in the study’s conceptual model (see figure 1). figure 1. conceptual model. australasian journal of information systems zobair, sanzogni & sandhu 2020, vol 24, research article telemedicine adoption barriers 4 2.1 organisational barriers whittaker, aufdenkamp, and tinley (2009) and kifle, mbarika, tsuma, wilkerson, and tan (2008) have documented potential users, organisational infrastructure, and policy as significant barriers to telemedicine adoption. lack of organisational effectiveness is also considered to be a significant telemedicine adoption barrier (evans, 2015). van dyk (2014) reported that policy, change management, economic feasibility, perception, user-friendliness, and governance, have influence over the adoption of telehealth (i.e., telemedicine) services. legislation, organisational infrastructure, societal factors (boonstra & broekhuis, 2010; van dyk, 2014) and strategic planning (modi et al., 2018; scott & mars, 2013), have also been reported to have influence on e-health (i.e., telemedicine) adoption. bangladesh’s healthcare industries have found it difficult to enhance service quality and improve accessibility (s. akter, m. upal, & u. hani, 2008), partly because of the low rate of telemedicine deployment in rural settings. studies reveal that providing high-quality care (williams et al., 2018), leadership engagement (ross, stevenson, lau, & murray, 2016), and regulations and health guidelines are indispensable for building health seekers’ trust (oak, 2007). uncertainty about rules and clinical guidelines are considered dominant barriers (martin, probst, shah, chen, & garr, 2012) that may influence organisational trustworthiness and many e-health projects fail because of inadequate regulatory policy integration (gagnon et al., 2009). de bustos et al. (2009) argued that success in e-health adoption is possible only if the underpinning barriers can be identified and overcome. therefore, the following hypotheses are proposed: hypothesis 1: organisational effectiveness influences telemedicine adoption success. hypothesis 2: organisational effectiveness influences patient’ trustworthiness in telemedicine usage. 2.2 technological barriers telemedicine is a technology-based healthcare system that can contribute to better health management by integrating ict, clinics and the internet to provide teleconsultation between clinicians and patients (kiberu, mars, & scott, 2017). rural communities in developing countries are deprived of quality health services because of poor ict infrastructure (hossain, yokota, sultana, & ahmed, 2018). however, in bangladesh the ict sector has grown considerably in recent years, demonstrating an outstanding dominating position. most of the population (99%) have access to mobile networks that cover 92% of the geographical area, and 97% of households have at least one mobile phone (ahmed et al., 2014). islam et al. (2017) reported that 73.9% of participants share health information through social media, demonstrating a sophisticated level of engagement with ict-based health to make informed health decisions. in the public health sector, ict resources remain inadequate and reliability issues make it a potential technological barrier (van dyk, 2014). many telemedicine technologies appear unsuccessful because they have a significant low effect on healthcare practices and often disregard interdependencies between technology, human characteristics, and socioeconomic circumstances (van gemert-pijnen et al., 2011). hasan (2012) reported that in bangladesh, internet access and power supply are not sufficiently reliable to successfully adopt telemedicine. this is supported by m. r. hoque, mazmum, and bao (2014) who mentioned that in bangladesh, internet access and power supply are not sufficiently reliable to adopt successful telemedicine services. these are critical barriers to a secure ict infrastructure (lewis, synowiec, lagomarsino, & schweitzer, 2012), australasian journal of information systems zobair, sanzogni & sandhu 2020, vol 24, research article telemedicine adoption barriers 5 and they can impede quality of care, life and productivity (zanaboni & wootton, 2016). therefore, the following hypotheses are proposed: hypothesis 3: ict infrastructure influences organisational effectiveness. hypothesis 4: ict infrastructure influences quality of care. 2.3 resource allocation barriers as a developing country, bangladesh lacks the resources to deploy telemedicine (khan, hossain, hasan, & clement, 2012). kiberu et al. (2017) found that equipment costs, and insufficient human resources allocations are considered to be potential resources barriers to telemedicine adoption. the literature further reports that change resistance among patients and health staff, human resources, uncertainty of ownership, ict skills, policymaking, leadership and coordination, monitoring and evaluation, cultural and language barriers (fanta, pretorius, & erasmus, 2015), service delivery issues (hossain et al., 2018), hardware access (boonstra & broekhuis, 2010), financial resources allocation (e. miller, 2010), resources allocation (boonstra & broekhuis, 2010), legal and ethical factors (anderson, 2007), and appropriate equipment (kruse et al., 2018) are dominant resource barriers to telemedicine adoption. weinstein et al. (2014) mentioned that the cost, allocation of funding for equipment and the recruitment of physicians are considered the dominant barriers to telemedicine adoption. likewise, e. miller (2010) found that financial constraints, resources allocations, and inadequate supplies are dominant barriers to telemedicine adoption. therefore, the following hypothesis is proposed: hypothesis 5: allocation of resources influences the adequacy of ict infrastructure. 2.4 health staff motivation barriers health staff motivation is defined as the degree to which a staff member is willing to perform assigned tasks to attain organisational goals (franco, bennett, & kanfer, 2002) and is a vital issue facing many developing countries (jayasuriya, jayasinghe, & wang, 2014). financial incentives could be a crucial determinant of staff motivation because health services are entirely labour intensive (franco et al., 2002). if health workers do not perceive personal benefits from telemedicine, they will be unmotivated to change their working habits (vishwanath & scamurra, 2007). when health staff are rewarded with financial incentives based on job performance, their motivation increases, fostering e-health implementation (boonstra & broekhuis, 2010) and the provision of better health services that contribute to patient satisfaction in telemedicine services (ross et al., 2016). lack of clear benefits, incentives and support for physicians, poor patient–clinician communication, marketplace competition and privacy legislation (boonstra & broekhuis, 2010), shortcomings in facilities (jayasuriya et al., 2014), a strong sense of mission and observable organisational effectiveness are considered critical issues tied to staff motivation (e. miller, 2010). interpersonal relationships remain cornerstones of health staff motivation, provided they facilitate understanding and improve the exchange of health information between clinicians and patients (weiner & biondich, 2006). this translates into detailed health histories and better health treatment plans and outcomes (shaarani, taleb, & antoun, 2017). patients expect high-quality services at an affordable cost and providing poor-quality service (martin et al., 2012) encourages patients to bypass local health providers and travel to urban hospitals (nesbitt, marcin, martha, & cole, 2005). financial incentives could lead to quality australasian journal of information systems zobair, sanzogni & sandhu 2020, vol 24, research article telemedicine adoption barriers 6 improvement, intensifying telemedicine service adoption (r. h. miller & sim, 2004). therefore, the following hypotheses are proposed: hypothesis 6: health staff motivation influences quality of care. hypothesis 7: health staff motivation influences patient satisfaction. hypothesis 8: health staff motivation influences telemedicine adoption success. 2.5 quality of care barriers growing demands, spiralling costs, and inadequate and inconsistent service quality and delivery are risking ict-supported healthcare systems (lluch, 2011). the quality level of bangladesh’s healthcare services is a significant concern (chandler, chonya, mtei, reyburn, & whitty, 2009) and patient satisfaction regarding quality of care is consistently neglected (m. s. akter, m. upal, & u. hani, 2008). a recent study (de la torre díez, alonso, hamrioui, lópezcoronado, & cruz, 2018) claimed that in telemedicine provision, the quality of both service and experience is mandatory to maintain patient health. uncoordinated monitoring, inconsistent service quality, insufficient privacy and confidentiality, and poor coordination can be potential barriers to patient satisfaction in telemedicine services (lluch, 2011). physician–patient relationship interference is the most cited barrier to telemedicine healthcare service adoption (boonstra & broekhuis, 2010). therefore, the following hypothesis is proposed: hypothesis 9: quality of care influences patient satisfaction. 2.6 patient satisfaction barriers health client satisfaction is critical to healthcare quality assessment and establishes organisational loyalty—a core issue in the healthcare industry’s survival (elbeck, 1987). hage, roo, van offenbeek, and boonstra (2013) revealed that telemedicine services that do not meet demand and have poor availability or accessibility lead to significant patient distrust and service dissatisfaction. de la torre díez et al. (2018) described the service quality perceived by telemedicine patients as “quality of service experience”, while martin et al. (2012) claimed that patients expect high-quality services from providers. conversely, poor-quality services contribute to patient dissatisfaction and untrustworthiness. medical malpractice, unclear laws about telemedicine practices and inconsistent virtual medical devices at rural hospitals are dominant barriers continuing to plague telemedicine health provision (boonstra & broekhuis, 2010). studies report that patients may travel to other providers (call et al., 2015) if telemedicine is unable to satisfy them by improving access, facilitating patient education and engagement, and promoting clinical care (mathieson, leafman, & horton, 2017). therefore, the following hypotheses are proposed: hypothesis 10: patient satisfaction influences telemedicine adoption success. hypothesis 11: patient satisfaction influences trustworthiness in telemedicine usage. 2.7 patient trustworthiness barriers trustworthiness in telemedicine plays a critical role in forming patients’ confidence and ensuring their intentions to reuse the system (akter, d'ambra, & ray, 2011). trustworthiness incorporates three characteristics namely, ability, benevolence, and integrity (mayer & davis, 1999), and is critical in forming patient confidence and continuing intentions to reuse the system (akter et al., 2011). inadequate monitoring, lack of usefulness (vishwanath & australasian journal of information systems zobair, sanzogni & sandhu 2020, vol 24, research article telemedicine adoption barriers 7 scamurra, 2007) and lack of patient privacy, confidentiality and data security (chang, 2015), adds to users’ distrust about telemedicine service delivery. further significant barriers are a lack of management support, security standards, and clear patient benefits (vishwanath & scamurra, 2007). therefore, the following hypothesis is proposed: hypothesis 12: patients’ trustworthiness in telemedicine usage influences telemedicine adoption success. 3 research methodology 3.1 research instrument development initially a conceptual model with key constructs, corresponding indicators, and scales was synthesised from existing, established research (lankton & wilson, 2007; venkatesh, morris, davis, & davis, 2003), and extended/adapted to fit the proposed study (padovani & carvalho, 2016; webster & watson, 2002). it was then further refined through data collected from participant observation, group discussions, and in-depth interviews before finalising the constructs, indicators, and scales, leading to the establishment of a survey instrument to collect data to empirically validate the hypotheses and the model. partial least squares structural equation modelling (pls-sem; smartpls v.3.2.7) (marzagão & carvalho, 2016) was used for data analysis. the pls algorithm tool tested scales reliability, and a bootstrapping algorithm technique tested the hypothesis and structural model (zhang et al., 2017). a seven-point likert scale was used (i.e., 1 = very strongly disagree to 7 = very strongly agree) as recommended (lankton & wilson, 2007) for healthcare surveys. most scales used in this study were drawn from existing literature and were modified (lankton & wilson, 2007) to develop and validate appropriate scales for the present research. two management information systems experts examined the survey questionnaire for logical consistency, contextual relevance, terminology, and measurement content clarity (zhang et al., 2017). a focus group of 15 bangladeshi phd students were invited to evaluate the questionnaire (venkatesh et al., 2003) due to their academic skills, research experience, and knowledge about telemedicine services in rural bangladesh. the feedback from the evaluation was used to rectify/refine the questionnaire. in addition, the questionnaire (constructs, indicators, and scales) was tested for content validity and reliability in a pilot study involving n=25 telemedicine users representing 5% of the target sample (j. w. cresswell & clark, 2011). an iterative rectification process was performed until the questionnaire reached an acceptable range of criterion validity. the questionnaire was initially developed in english, translated into bangla, retranslated into english and adjusted so that both versions were comparable (akter et al., 2011). the survey questionnaires were distributed including informed consent forms to respondents who agreed to participate in the survey. the accompanying documentation reinforced that participation was voluntary and they could withdraw their involvement anytime. 3.2 sample and data collection the survey commenced after approval from the directorate of general health services, mohfw, as well as ethical clearance from the university. a pilot test was conducted according to j. w. cresswell and clark (2011) using a random selection of about 5% of the intended survey sample from three different study regions. high response rates were obtained. four university graduates were recruited to execute the survey. they were briefed australasian journal of information systems zobair, sanzogni & sandhu 2020, vol 24, research article telemedicine adoption barriers 8 on the study’s intent and instructed to read each survey question precisely, practice neutral tones, document participants’ answers accurately, and be non-judgemental (neuman, 2014). the data collection occurred from mid-january to mid-march 2017. a significant response rate (98.40%) was obtained and considered adequate to fulfil the research purpose. data were obtained from rural telemedicine patients from three subdistricts in bangladesh. each group was statistically representative of the telemedicine population, relying on commonalities in telemedicine infrastructure and comparable clinical methods across all bangladesh telemedicine centres. the telemedicine service facilities (i.e., operation standards) provided by the government were equal for all existing centres. costs, homogeneity, and a high concentration of the telemedicine population were also considered during subdistrict selection. a multistage random sampling method was deemed suitable for this study and is considered appropriate for ict-based medical research. burt and sisk (2005) applied this sampling technique for electronic health records in e-health research. probability random sampling techniques are ideal for achieving high accuracy, efficiency, and consistency in research outcomes (neuman, 2014). the telemedicine population was defined as those who had experienced telemedicine at least once in any rural telemedicine centre during the past 12 months. three districts where telemedicine centres existed were randomly selected. one district from the north and two from the south, east and west regions of bangladesh were excluded due to resource and time constraints, and reduced patient concentrations during the data collection period. the randomly selected districts were pubna, khulna and satkhira from which three upazila telemedicine centres were randomly selected. from these, a total of 500 samples were randomly designated for data collection, with each patient having an equal chance of being selected (akter et al., 2011). the literature indicates that with increasing sample size, pls-sem estimates improve with decreasing average error rates (marcoulides & saunders, 2006). from the patient lists obtained from the targeted telemedicine centres, 500 residential addresses were randomly selected, consisting of proportionate samples from dacope (n=242), devhata (n=205) and bera (n=53) upazila. representation at the addresses was based on the mallintercept technique (bush & hair jr, 1985). patients’ addresses and phone numbers were collected from rural telemedicine centres. they were contacted by phone and invited to participate in a one-hour survey at a telemedicine centre during office hours. those unable to travel to the centres were asked to participate according to their convenience. data were gathered through self-completion and interviewerfilled techniques to obtain high accuracy in the valid responses. each interviewer administered five survey questionnaires per day. the study met the sampling target by obtaining 500 valid responses. all questionnaires were scrutinised for completeness. eight samples were identified as incomplete and excluded; 492 samples were preserved for analysis. the survey’s demographics are presented in table 1. australasian journal of information systems zobair, sanzogni & sandhu 2020, vol 24, research article telemedicine adoption barriers 9 measure items frequency percentage (%) gender male 206 41.9 female 286 58.1 age 18 and ≤20 64 13.0 ≥21 and ≤30 158 32.1 ≥31 and ≤40 106 21.5 ≥41 and ≤50 88 17.9 >50 76 15.4 education illiterate 68 13.8 primary 104 21.0 secondary 178 36.2 higher secondary 64 13.0 bachelor 51 10.4 masters and above 27 5.5 table 1 demographic characteristics of the sample 4 data analysis and results pls-sem was employed to analyse the data set (muthupoltotage & gardner, 2018) as it largely consists of latent constructs which cannot be measured directly requiring a set of indicators that serve as proxies (hair jr, hult, ringle, & sarstedt, 2017). a pls-sem path model incorporates two sets of linear equations—a measurement model specifying a construct and its corresponding indicators and a structural model specifying the relationship between exogenous and endogenous constructs (henseler, hubona, & ray, 2017). in pls-sem, indicators are also called items or manifest variables that are directly measured proxy variables containing raw data (hair jr et al., 2017). the structural model focuses on the research questions and hypothesis, addressing the two major questions about the inclusion and interrelationships of the constructs in the model (henseler et al., 2017). therefore, the plssem technique is considered a more rigorous analysis for the proposed research model (muthupoltotage & gardner, 2018). 4.1 measurement model table 2 illustrates the summary assessment of the 27 indicators associated with the eight assigned latent constructs (i.e., barriers to adoption [ba], lack of health staff motivation [lhsm], lack of ict infrastructure [lict], lack of organisational effectiveness [loe], lack of quality of care [lqc], lack of resources [lr], lack of patient satisfaction [lsat] and lack of trustworthiness [lt] of the reflective measurement model that tested internal consistency reliability, and convergent and discriminant validity. the relationships between the indicators and corresponding latent constructs were all significant (p < .01). the standardised outer loadings of all indicators except one were higher than the threshold value (.70) (roldán & sánchez-franco, 2012), suggesting adequate, overall indicator reliability. indicator ba1 (.698) was considered comparable to the threshold value. the average variance extracted (ave) value of each construct exceeded the threshold value (>.5), thus indicating significant convergent validity (hair jr et al., 2017). these ave values demonstrate that each construct explains the variance of over half of their corresponding indicators, confirming acceptable convergent validity (roldán & sánchez-franco, 2012). both cronbach’s alpha and composite reliability exceeded their threshold values (>.70), confirming statistical significance of the model and demonstrating strong evidence of internal consistency reliability (hair jr et al., 2017). australasian journal of information systems zobair, sanzogni & sandhu 2020, vol 24, research article telemedicine adoption barriers 10 latent constructs indicators standardised loading ave cr cronbach’s alpha r2 adjusted r2 barriers to adoption ba1 .698 .556 .834 .737 .625 .622 ba2 .747 ba3 .773 ba4 .763 lack of health staff motivation lhsm1 .905 .786 .880 .730 lhsm4 .868 lack of ict infrastructure lict1 .840 .693 .900 .852 .668 .668 lict2 .839 lict3 .853 lict4 .797 lack of organisational effectiveness loe1 .851 .706 .923 .895 .730 .730 loe2 .864 loe3 .852 loe4 .839 loe5 .793 lack of quality of care lqc2 .913 .837 .912 .806 .538 .537 lqc4 .917 lack of resources lr1 .812 .684 .897 .846 lr2 .827 lr3 .819 lr4 .850 lack of patient satisfaction lsat1 .919 .826 .905 .790 .493 .491 lsat2 .898 lack of trustworthiness lt2 .803 .669 .890 .836 .432 .430 lt3 .826 lt4 .838 lt6 .803 table 2 convergent validity and reliability statistics of the sample the square root of each construct’s ave was higher than the correlation with other constructs except for lict, confirming the construct’s acceptable discriminant validity (see table 3) (sarstedt, wilczynski, & melewar, 2013). to verify the discriminant validity of lict, the crossloadings index was investigated and found to be higher than the cross-loading values of all other constructs, supporting acceptable discriminant validity (hair jr et al., 2017). the literature suggests that this test is highly recommended to ensure no indicators are incorrectly assigned to a construct (henseler et al., 2017). ba lhsm lict loe lqc lr lsat lt ba .746 lhsm .549 .887 lict .489 .251 .832 loe .497 .197 .855 .840 lqc .504 .727 .280 .245 .915 lr .436 .230 .818 .770 .242 .827 lsat .693 .656 .279 .251 .648 .232 .909 lt .539 .297 .582 .631 .292 .551 .337 .818 note. the square root of ave in bold. table 3 discriminant validity coefficients australasian journal of information systems zobair, sanzogni & sandhu 2020, vol 24, research article telemedicine adoption barriers 11 4.2 structural model the structural model was developed to describe the relationships among the latent constructs and examine their significance (roldán & sánchez-franco, 2012). a bootstrapping technique using 5000 iterations was used to test the statistical significance of the relationships between endogenous and exogenous latent constructs in the structural path models (hair jr et al., 2017). the structural model and hypothesised relationships between the endogenous and exogenous constructs were tested using a standardised path coefficient (β) and t-statistics (see tables 2 and 4) at p < .01, which conservative researchers have used in their research (hair jr et al., 2017). the threshold critical t-values (α; two-tailed test) at a significance level of α = .05, α = .01 and α = .10 were 1.96, 2.57 and 1.65, respectively (hair jr et al., 2017). the structural model revealed that all 12 hypotheses were supported by the relationships at the p < .01 significance level (see figure 2). the results (see tables 2 and 4) authenticate that the four proposed latent constructs had significant effects on barriers to telemedicine adoption. the findings show that the relationships between loe and ba (β=.228, t=4.587, p<.01), lhsm and ba (β=.126, t= 2.992, p< .01), lsat and ba (β=.488, t= 10.061, p<.01) and lt and ba (β=.193, t=3.536, p<.01) were statistically significant, confirming support for h1, h8, h10 and h12 (see table 4). an r2 ≈ .622 indicates that about 62% of the variance (i.e., ba) in the model was jointly explained by loe, lhsm, lsat and lt. the results were close to a substantial r2 value (i.e., 67%) (henseler, ringle, & sinkovics, 2009), suggesting high predictive capability of the model. further, the literature indicates that r2 demonstrates each construct’s significance and its associative contribution to the overall r2 (wilson, 2010). thus, the findings conclude that loe, lhsm, lsat and lt are four significant barriers impeding the adoption of current telemedicine projects in rural public hospital settings. hypothesis path coefficient (β) se t p decision h1 loe → ba .228*** .050 4.587 .000 supported h2 loe→lt .583*** .046 12.698 .000 supported h3 lict → loe .855*** .020 42.560 .000 supported h4 lict → lqc .104*** .038 2.712 .007 supported h5 lr → lict .818*** .023 35.733 .000 supported h6 lhsm → lqc .701*** .032 21.756 .000 supported h7 lhsm→ lsat .392*** .058 6.806 .000 supported h8 lhsm → ba .126*** .042 2.992 .003 supported h9 lqc → lsat .363*** .060 6.068 .000 supported h10 lsat → ba .488*** .048 10.061 .000 supported h11 lsat → lt .190*** .035 5.373 .000 supported h12 lt → ba .193*** .055 3.536 .000 supported note. *p < .10; **p < .05; ***p < .01 (two-tailed) confidence intervals for significance testing. table 4 structural estimation of t-values and p-values for the structural model australasian journal of information systems zobair, sanzogni & sandhu 2020, vol 24, research article telemedicine adoption barriers 12 figure 2. final pls-sem structural model for telemedicine adoption barriers. the structural model further revealed that eight other hypotheses were supported at the p < .01 significance level (see figure 2). the relationships between loe and lt (β = .583, t = 12.698, p < .01), lict and loe (β = .855, t = 42.560, p < .01), lict and lqc (β = .104, t = 2.712, p <. 01), lr and lict (β = .818, t = 35.733, p < .01), lhsm and lqc (β = .701, i = 21.756, p < .01), lhsm and lsat (β = .392, t = 6.806, p < .01), lqc and lsat (β = .363, t = 6.068, p < .01), and lsat and lt (β = .190, t = 5.373, p < .01) were statistically significant, confirming support for h2, h3, h4, h5, h6, h7, h9 and h11 (see table 4). however, some of these constructs were not significantly related to the barriers to telemedicine adoption. our findings revealed that these constructs have no direct effect on ba, but strongly affect their endogenous latent constructs. consistent with these findings, we can conclude that these constructs have a substantial indirect impact on the adoption of telemedicine projects in rural public hospital settings in bangladesh. further, r2 was significant for each these constructs and their contribution to the structural model’s overall r2 (see table 4) (wilson, 2010). these constructs may need further research into the significance of their contributions to barriers to telemedicine adoption. 5 discussion 5.1 summary of findings these findings are based on the opinions of patients who experienced telemedicine services. the survey excluded non-users because patients in bangladesh cannot access telemedicine services without physician referral. four substantial barriers offer considerable resistance to telemedicine adoption. discussion of these findings will elucidate on the concerns of how these barriers contribute to broader adoption complexities requiring appropriately guided australasian journal of information systems zobair, sanzogni & sandhu 2020, vol 24, research article telemedicine adoption barriers 13 action to make telemedicine sustainable. it is anticipated that identifying influential barriers to telemedicine adoption will provide insight for promoting efficiency in telemedicine clinical practices and advancing its sustainability in bangladesh’s healthcare industry. organisational barriers, health staff motivation, patient satisfaction and trustworthiness appeared to have a predominant adverse effect inhibiting telemedicine’s successful integration into bangladesh’s clinical practices. 5.2 organisational barriers lack of organisational effectiveness is a significant telemedicine adoption barrier, indicating that current telemedicine projects fail to meet their organisational goals regarding effectivity measures. effective tele-monitoring systems have been shown to reduce hospitalisations and mortality risk among patients with chronic heart failure (kim, gellis, bradway, & kenaley, 2018). many e-health project plans fail due to inadequate organisational policy integration into practice (gagnon et al., 2009). similar research (k. cresswell & sheikh, 2013) revealed that organisational efficiency is a major barrier to telemedicine adoption which needs to be quantified. evans (2015) suggested that successful telemedicine adoption relies on the organisational restructuring of internal processes, and the integration of new patient care policies and procedures which allow health seekers and health providers onto a telemedicine health care platform. the results authenticate that health organisations, strategic planning, infrastructural design, clear operational guidelines, and active management support have received insufficient attention in current telemedicine projects. therefore, implementation of these practices along with sufficient human resources could foster telemedicine effectiveness in rural public hospital settings. 5.3 health staff motivation barriers the second barrier is a lack of staff motivation, which the results demonstrate had a significant effect on telemedicine adoption barriers in rural public hospital settings. health staff motivation is vital to organisational effectiveness because they are the frontline user group of telemedicine service processes (ajami & bagheri-tadi, 2013). health staff performance determines how well an organisation functions (jayasuriya et al., 2014). the findings imply that present telemedicine projects do not adequately motivate health staff to enable an effective working environment, facilitate financial and professional benefits or allocate adequate resources for achieving patient health expectations. observations suggested that in an upazila rural hospital, physicians experienced heavy workloads and telemedicine patients were often an additional burden, causing professional frustration. physicians are reluctant to serve in rural hospitals due to limited support, insufficient resources, uncomfortable conditions and isolation from professional colleagues (henderson & tulloch, 2008). observations indicated that physicians in rural hospitals focused on general patients rather than telemedicine, resulting in increased wait times for telemedicine patients. present telemedicine projects do not dedicate physicians to telemedicine centres, and some telemedicine patients are serviced by ict support engineers who directly consult with specialised physicians. current projects have not focused on this issue in telemedicine practices. therefore, well-trained physicians should be dedicated to rural telemedicine centres and responsible for supervising telemedicine patients. the results show that a lack of health staff motivation had substantial effects on lack of quality of care and patient satisfaction. this suggests that health workers’ motivation is an influential australasian journal of information systems zobair, sanzogni & sandhu 2020, vol 24, research article telemedicine adoption barriers 14 determinant to enhance telemedicine service quality, contributing towards high patient satisfaction. poor service quality is of critical concern in developing countries and partly contributes to low healthcare staff motivation (chandler et al., 2009). financial incentives and personal and professional benefits can stimulate health staff motivation to perform their assigned tasks, resulting in higher service quality and patient satisfaction (lohmann et al., 2018). the findings suggest that the total service duration of telemedicine physicians in rural centres should be considered for those entitled to specialist medical degrees. similarly, telemedicine support engineers should be recognised to motivate them as enablers of organisational success (franco et al., 2002), but observations suggested they were frustrated due to low remuneration and were seeking opportunities with other providers. a primary concern in developing countries is financial accessibility, namely the government’s inability to provide sufficient funds to the public health system, low remuneration of health staff and lack of medical supplies (peters et al., 2008). to overcome this barrier, employer performance-based financial benefits are needed to convince health staff of the value of telemedicine services (e. miller, 2010). 5.4 patient satisfaction barriers lack of patient satisfaction is the third barrier. this significant finding indicates that current telemedicine projects fail to sufficiently satisfy their patients, suggesting that patients who perceive a lack of accessibility and poor service quality are more likely to be dissatisfied with telemedicine. patient satisfaction is a significant health outcome indicator (greenslade & jimmieson, 2011) and critical to organisational performance in hospital settings (koné péfoyo & wodchis, 2013). a satisfied patient will likely return to the services and share their positive experiences (hill & doddato, 2002). a recent study (williams et al., 2018) found that postoperative telemedicine patients have a satisfaction rate as high as 111% and prefer telemedicine follow-up. observations indicated that current telemedicine practices are non-committal in terms of systems accessibility. official opening hours are limited, contributing to patient dissatisfaction for those unable to access services after travelling long distances and paying additional travelling costs. the literature indicates that personnel, physical space and internet access are additional barriers in rural hospital settings (garrett et al., 2006). interestingly, to bolster patient satisfaction, current telemedicine does not offer an emergency telemedicine service facility, a specialised diagnosis facility in a rural centre or assurance of prescribed medicine availability in a local pharmacy. patients should have continual access to a telemedicine clinical team, promoting their health and wellbeing (freed et al., 2018). a study identified a positive relationship between service quality and patient satisfaction in healthcare interventions (lee & yom, 2007), suggesting that satisfaction is critical in healthcare quality assessment and organisational loyalty, reflecting the core issues affecting the longevity of current healthcare industries (elbeck, 1987). these findings are informative and relate to patient concerns regarding telemedicine accessibility and service quality. 5.5 patient trustworthiness barriers the fourth significant barrier is a lack of trustworthiness, which the findings suggest had a significant effect on telemedicine adoption. trustworthiness in telemedicine can be established by fulfilling four essential beliefs: ability, benevolence, integrity and predictability (akter et al., 2011; mayer & davis, 1999). the results indicate that current telemedicine projects fail to achieve these beliefs, indicating that patients do not sufficiently trust current service australasian journal of information systems zobair, sanzogni & sandhu 2020, vol 24, research article telemedicine adoption barriers 15 standards. ict-based healthcare services are often insufficiently funded, operate in a resourcepoor environment and, when accessible, are often dysfunctional or unresponsive (akter et al., 2011). these factors contribute to low trustworthiness of the health system (akter et al., 2011). observations suggested that existing telemedicine health facilities were inadequate regarding resource and performance measures. space, physical resources and medical and operational ict support were insufficiently allocated to telemedicine services. three to four patients were consistently present in the consultation room. from a sociocultural perspective, many patients are uncomfortable disclosing disease-related issues in front of other patients and so their disease-related privacy and confidentiality is unprotected. further, current telemedicine centres have not yet established convenient service hours, timely and easy referral systems, prompt responsiveness to patient needs or better patient supervision concerning patient care and supportiveness. since telemedicine entails the electronic transmission and storage of patient health-related data (chang, 2015), it is essential for health providers to keep secure data privacy and confidentiality. 5.6 other barriers the results show that a lack of ict support, resources and quality of care indirectly affect adoption barriers through a lack of organisational effectiveness, health staff motivation, patient satisfaction and trustworthiness. lack of ict infrastructure, allocation of resources and quality of care have no direct effect on telemedicine adoption barriers but strongly affect endogenous latent constructs. insufficient power supply and internet facilities, bandwidth expenses and telemedicine equipment might be primary challenges to telemedicine implementation in bangladesh (hasan, 2012). these are important factors in telemedicine implementation (lewis et al., 2012). equipment costs and maintenance, poor technical expertise and a lack of coordination, staffing and technical infrastructure are reported telemedicine adoption barriers (larsen, hudnall stamm, davis, & magaletta, 2004). researchers worldwide could test these factors and compare their findings to further understand telemedicine adoption barriers in rural settings. therefore, strong ict infrastructure, adequate financial and human resources, uninterrupted power supply and high internet speed and connectivity are required to expedite telemedicine adoption for the medically underprivileged rural community. 5.7 implications this unique study develops an original framework for a conceptual model that brings to light the relationships between telemedicine and its subsequent adoption barriers within the context of rural public hospital settings in bangladesh. this study has made several contributions to the literature. first, it has provided significant practical implications that can be applied to the health-related literature. second, this study investigated potential barriers impeding the adoption of telemedicine healthcare services in a developing country like bangladesh. third, it has identified four exemplary barriers to telemedicine adoption—lack of organisational effectiveness, health staff motivation, patient satisfaction and trustworthiness—that offer considerable resistance to the adoption of current telemedicine projects in rural bangladesh’s public hospitals settings. to the authors’ knowledge, no studies were available to guide research into factors causing the low rate of telemedicine adoption in bangladesh’s rural public hospitals. little research has investigated these barriers from a patient perspective, representing a considerable australasian journal of information systems zobair, sanzogni & sandhu 2020, vol 24, research article telemedicine adoption barriers 16 knowledge gap in the literature. the authors believe that the proposed model contributes significantly to the literature, particularly in the context of developing nations and provides practical guidelines for expediting telemedicine adoption in rural settings in emerging countries. the empirical findings may clarify the causative factors of existing telemedicine project adoption. the findings may also institutionalise favourable policy guidelines for successful telemedicine service adoption in rural settings, including the improvement of healthcare industries in both bangladesh and similar emerging economies in a global context. 5.8 conclusions this study explored a range of barriers to telemedicine adoption in bangladesh’s current public projects that have not been addressed in the literature. lack of organisational effectiveness, quality of care, trustworthiness and health staff motivation were identified as dominant barriers. this research is significant and hopes to build telemedicine into an institutionalised health infrastructure for providing better quality care with greater flexibility through modernised, specialised healthcare support for medically underprivileged rural communities in bangladesh. rural and remote populations still have inadequate access to dedicated public medical facilities and private after-hours medical practitioners. all barriers identified in this research are considerable threats to sustaining telemedicine projects but are believed to be manageable by bangladesh’s policymakers, stakeholders and implementers. an active telemedicine action plan and regulations are suggested to break down the adoption barriers and achieve organisational goals. although successful telemedicine adoption is considered a formidable task, especially in such a resource-poor country as bangladesh, the findings provide to the government, policymakers, implementers and stakeholders a clear vision regarding the causative adoption factors emerging from current telemedicine projects. the findings will also be useful in developing appropriate policy guidelines for the betterment of healthcare industries in bangladesh and similar emerging economies. 5.9 limitations and future research some limitations exist in this research. first, a small proportion of telemedicine centres were surveyed. future studies can include additional telemedicine centres to test findings’ generalisability. second, this study reflected only the views of health seekers. future research should consist of health providers’ views to understand more clearly telemedicine adoption barriers. third, only rural public health providers were included in this study. private telemedicine providers can be included in future research to understand their differences in service interests and telemedicine adoption barriers. finally, this study was limited to bangladesh. combining this study with cross-sectional data from similar developing countries can provide a broader view of telemedicine adoption barriers in a global context. 5.10 research highlights • the unprecedented data collection exercise involving months of field work in the most remote areas of bangladesh provides unique and valuable insights into remote people’s views regarding telemedicine services. • four significant barriers to telemedicine adoption were identified and confirmed namely, lack of organisational effectiveness, health staff motivation, patient satisfaction, and trustworthiness. a paucity of theses leads to significant resistance to the adoption of telemedicine projects in rural bangladesh’s public hospital's settings. australasian journal of information systems zobair, sanzogni & sandhu 2020, vol 24, research article telemedicine adoption barriers 17 • the structural equation modeling method used to validate a structural model showing interactions and persistence of interaction of telemedicine adoption barriers. • a validated, robust functional model that can be used to investigate telemedicine adoption barriers in emerging economies was confirmed. acknowledgements the authors gratefully acknowledge the participants in the study and the health bodies of directorate general of health services, mohfw of bangladesh, who helped in carrying out this research. references ahmed, t., bloom, g., iqbal, m., lucas, h., rasheed, s., waldman, l., bhuiya, a. (2014). ehealth and m-health in bangladesh: opportunities and challenges. (no. ids evidence report; 60). ids. ajami, s., & bagheri-tadi, t. (2013). barriers for adopting electronic health records (ehrs) by physicians. acta informatica medica, 21(2), 129. akter, m. s., upal, m., & hani, u. (2008). service quality perception and satisfaction: a study over sub-urban public hospitals in bangladesh. journal of services research, 125. akter, s., d'ambra, j., & ray, p. (2011). trustworthiness in mhealth information services: an assessment of a hierarchical model with mediating and moderating effects using partial least squares (pls). journal of the association for information science and technology, 62(1), 100-116. akter, s., upal, m., & hani, u. (2008). service quality perception and satisfaction: a study over sub-urban public hospitals in bangladesh. journal of services research, 8(s1), s125. alaboudi, a., atkins, a., sharp, b., balkhair, a., alzahrani, m., & sunbul, t. (2016). barriers and challenges in adopting saudi telemedicine network: the perceptions of decision makers of healthcare facilities in saudi arabia. journal of infection and public health, 9(6), 725-733. anderson, j. g. (2007). social, ethical and legal barriers to e-health. international journal of medical informatics, 76(5), 480-483. bigna, j. j. r., noubiap, j. j. n., plottel, c. s., kouanfack, c., & koulla-shiro, s. (2014). barriers to the implementation of mobile phone reminders in pediatric hiv care: a pre-trial analysis of the cameroonian more care study. bmc health services research, 14(1), 523. boonstra, a., & broekhuis, m. (2010). barriers to the acceptance of electronic medical records by physicians from systematic review to taxonomy and interventions. bmc health services research, 10(1), 231. bullock, d. r., vehe, r. k., zhang, l., & correll, c. k. (2017). telemedicine and other care models in pediatric rheumatology: an exploratory study of parents’ perceptions of barriers to care and care preferences. pediatric rheumatology, 15(1), 55. burt, c. w., & sisk, j. e. (2005). which physicians and practices are using electronic medical records? health affairs, 24(5), 1334-1343. australasian journal of information systems zobair, sanzogni & sandhu 2020, vol 24, research article telemedicine adoption barriers 18 bush, a. j., & hair jr, j. f. (1985). an assessment of the mall intercept as a data collection method. journal of marketing research, 158-167. cajita, m. i., hodgson, n. a., lam, k. w., yoo, s., & han, h.-r. (2018). facilitators of and barriers to mhealth adoption in older adults with heart failure. cin: computers, informatics, nursing, 36(8), 376-382. call, v. r., erickson, l. d., dailey, n. k., hicken, b. l., rupper, r., yorgason, j. b., & bair, b. (2015). attitudes toward telemedicine in urban, rural, and highly rural communities. telemedicine and e-health, 21(8), 644-651. chandler, c. i. r., chonya, s., mtei, f., reyburn, h., & whitty, c. j. m. (2009). motivation, money and respect: a mixed-method study of tanzanian non-physician clinicians. social science & medicine, 68(11), 2078-2088. doi:10.1016/j.socscimed.2009.03.007 chang, h. (2015). evaluation framework for telemedicine using the logical framework approach and a fishbone diagram. healthcare informatics research, 21(4), 230-238. cresswell, j. w., & clark, v. l. p. (2011). designing and conducting mixed methods resaerch. (2nd edition). cresswell, k., & sheikh, a. (2013). organizational issues in the implementation and adoption of health information technology innovations: an interpretative review. international journal of medical informatics, 82(5), e73-e86. de bustos, e. m., moulin, t., & audebert, h. j. (2009). barriers, legal issues, limitations and ongoing questions in telemedicine applied to stroke. cerebrovascular diseases, 27(suppl. 4), 36-39. de la torre díez, i., alonso, s. g., hamrioui, s., lópez-coronado, m., & cruz, e. m. (2018). systematic review about qos and qoe in telemedicine and ehealth services and applications. journal of medical systems, 42(10), 182. elbeck, m. (1987). an approach to client satisfaction measurement as an attribute of health service quality. health care management review, 12(3), 47-52. evans, p. r. (2015). an exploration of the perceptions of a us midwest acute care hospital organization regarding the adoption and acceptance of telemedicine. capella university, faber, s., van geenhuizen, m., & de reuver, m. (2017). ehealth adoption factors in medical hospitals: a focus on the netherlands. international journal of medical informatics, 100, 7789. fanta, g. b., pretorius, l., & erasmus, l. (2015). an evaluation of ehealth systems implementation frameworks for sustainability in resource constrained environments: a literature review. paper presented at the iamot 2015 conference proceedings, cape town. franco, l. m., bennett, s., & kanfer, r. (2002). health sector reform and public sector health worker motivation: a conceptual framework. social science & medicine, 54(8), 1255-1266. freed, j., lowe, c., flodgren, g., binks, r., doughty, k., & kolsi, j. (2018). telemedicine: is it really worth it? a perspective from evidence and experience. journal of innovation in health informatics, 25(1), 14-18. doi:10.14236/jhi.v25i1.957 australasian journal of information systems zobair, sanzogni & sandhu 2020, vol 24, research article telemedicine adoption barriers 19 gagnon, m.-p., shaw, n., sicotte, c., mathieu, l., leduc, y., duplantie, j., . . . légaré, f. (2009). users' perspectives of barriers and facilitators to implementing ehr in canada: a study protocol. implementation science, 4(1), 20. garrett, p., brown, c. a., hart-hester, s., hamadain, e., dixon, c., pierce, w., & rudman, w. j. (2006). identifying barriers to the adoption of new technology in rural hospitals: a case report. perspectives in health information management/ahima, american health information management association, 3. gibson, k. l., coulson, h., miles, r., kakekakekung, c., daniels, e., & o'donnell, s. (2011). conversations on telemental health: listening to remote and rural first nations communities. rural and remote health, 11(2), 1656-1674. gravel, k., légaré, f., & graham, i. d. (2006). barriers and facilitators to implementing shared decision-making in clinical practice: a systematic review of health professionals' perceptions. implementation science, 1(1), 16. greenslade, j. h., & jimmieson, n. l. (2011). organizational factors impacting on patient satisfaction: a cross sectional examination of service climate and linkages to nurses’ effort and performance. international journal of nursing studies, 48(10), 1188-1198. doi:10.1016/j.ijnurstu.2011.04.004 hage, e., roo, j. p., van offenbeek, m. a., & boonstra, a. (2013). implementation factors and their effect on e-health service adoption in rural communities: a systematic literature review. bmc health services research, 13(1), 19. hair jr, j. f., hult, g. t. m., ringle, c., & sarstedt, m. (2017). a primer on partial least squares structural equation modeling (pls-sem): sage publications. hamid, s. a., ahsan, s. m., begum, a., & asif, c. a. a. (2015). inequity in formal health care use: evidence from rural bangladesh. journal of international development, 27(1), 36-54. hasan, j. (2012). effective telemedicine project in bangladesh: special focus on diabetes health care delivery in a tertiary care in bangladesh. telematics and informatics, 29(2), 211-218. henderson, l. n., & tulloch, j. (2008). incentives for retaining and motivating health workers in pacific and asian countries. human resources for health, 6(1), 18-18. doi:10.1186/14784491-6-18 henseler, j., hubona, g., & ray, p. a. (2017). partial least squares path modeling: updated guidelines. in partial least squares path modeling (pp. 19-39): springer. henseler, j., ringle, c. m., & sinkovics, r. r. (2009). the use of partial least squares path modeling in international marketing. in new challenges to international marketing (pp. 277319): emerald group publishing limited. hill, m. h., & doddato, t. (2002). relationships among patient satisfaction, intent to return, and intent to recommend services provided by an academic nursing center. journal of cultural diversity, 9(4), 108-112. hoque, m. r. (2016). an empirical study of mhealth adoption in a developing country: the moderating effect of gender concern. bmc medical informatics and decision making, 16(1), 51. australasian journal of information systems zobair, sanzogni & sandhu 2020, vol 24, research article telemedicine adoption barriers 20 hoque, m. r., & bao, y. (2015). cultural influence on adoption and use of e-health: evidence in bangladesh. telemedicine and e-health, 21(10), 845-851. doi:10.1089/tmj.2014.0128 hoque, m. r., karim, m. r., & amin, m. b. (2015). factors affecting the adoption of mhealth services among young citizen: a structural equation modeling (sem) approach. asian business review, 5(2), 60-65. hoque, m. r., mazmum, m., & bao, y. (2014). e-health in bangladesh: current status, challenges, and future direction. int tech manag rev, 4(2), 87-96. hoque, r., & sorwar, g. (2017). understanding factors influencing the adoption of mhealth by the elderly: an extension of the utaut model. international journal of medical informatics, 101, 75-84. doi:10.1016/j.ijmedinf.2017.02.002 hossain, n., yokota, f., sultana, n., & ahmed, a. (2018). factors influencing rural end-users' acceptance of e-health in developing countries: a study on portable health clinic in bangladesh. telemedicine journal and e-health : the official journal of the american telemedicine association. doi:10.1089/tmj.2018.0039 islam, m. m., touray, m., yang, h. c., poly, t. n., nguyen, p. a., li, y. c., & syed, a. s. (2017). e-health literacy and health information seeking behavior among university students in bangladesh. paper presented at the 16th world congress of medical and health informatics: precision healthcare through informatics, medinfo 2017. jang-jaccard, j., nepal, s., alem, l., & li, j. (2014). barriers for delivering telehealth in rural australia: a review based on australian trials and studies. telemedicine and e-health, 20(5), 496-504. jayasuriya, r., jayasinghe, u. w., & wang, q. (2014). health worker performance in rural health organizations in lowand middle-income countries: do organizational factors predict non-task performance? social science & medicine, 113, 1-4. doi:10.1016/j.socscimed.2014.04.042 johnson, k. b. (2001). barriers that impede the adoption of pediatric information technology. archives of pediatrics & adolescent medicine, 155(12), 1374-1379. khan, m., hossain, s., hasan, m., & clement, c. k. (2012). barriers to the introduction of ict into education in developing countries: the example of bangladesh. online submission, 5(2), 61-80. kiberu, v. m., mars, m., & scott, r. e. (2017). barriers and opportunities to implementation of sustainable e-health programmes in uganda: a literature review. african journal of primary health care & family medicine, 9(1). kifle, m., mbarika, v. w., tsuma, c., wilkerson, d., & tan, j. (2008). a telemedicine transfer model for sub-saharan africa. proceedings of the 41st annual hawaii international conference on system sciences,. kim, e., gellis, z. d., bradway, c., & kenaley, b. (2018). key determinants to using telehealth technology to serve medically ill and depressed homebound older adults. journal of gerontological social work, 1-24. doi:10.1080/01634372.2018.1499575 australasian journal of information systems zobair, sanzogni & sandhu 2020, vol 24, research article telemedicine adoption barriers 21 koivunen, m., & saranto, k. (2018). nursing professionals' experiences of the facilitators and barriers to the use of telehealth applications: a systematic review of qualitative studies. scandinavian journal of caring sciences, 32(1), 24-44. koné péfoyo, a. j., & wodchis, w. p. (2013). organizational performance impacting patient satisfaction in ontario hospitals: a multilevel analysis. bmc research notes, 6(1), 509-509. doi:10.1186/1756-0500-6-509 kruse, c. s., atkins, j. m., baker, t. d., gonzales, e. n., paul, j. l., & brooks, m. (2018). factors influencing the adoption of telemedicine for treatment of military veterans with posttraumatic stress disorder. journal of rehabilitation medicine, 50(5), 385-392. lankton, n. k., & wilson, e. v. (2007). factors influencing expectations of e-health services within a direct-effects model of user satisfaction. e-service journal, 5(2), 85-112. larsen, d., hudnall stamm, b., davis, k., & magaletta, p. r. (2004). prison telemedicine and telehealth utilization in the united states: state and federal perceptions of benefits and barriers. telemedicine journal & e-health, 10(supplement 2), s-81-s-90. leaming, l. e. (2007). barriers to physician adoption of telemedicine and best practices for overcoming these barriers. (doctoral dissertation, medical university of south carolina). lee, m. a., & yom, y.-h. (2007). a comparative study of patients’ and nurses’ perceptions of the quality of nursing services, satisfaction and intent to revisit the hospital: a questionnaire survey. international journal of nursing studies, 44(4), 545-555. doi:10.1016/j.ijnurstu.2006.03.006 lerouge, c., & garfield, m. j. (2013). crossing the telemedicine chasm: have the us barriers to widespread adoption of telemedicine been significantly reduced? international journal of environmental research and public health, 10(12), 6472-6484. lewis, t., synowiec, c., lagomarsino, g., & schweitzer, j. (2012). e-health in low-and middleincome countries: findings from the center for health market innovations. bulletin of the world health organization, 90(5), 332-340. lin, c., lin, i.-c., & roan, j. (2012). barriers to physicians’ adoption of healthcare information technology: an empirical study on multiple hospitals. journal of medical systems, 36(3), 1965-1977. lluch, m. (2011). healthcare professionals’ organisational barriers to health information technologies—a literature review. international journal of medical informatics, 80(12), 849862. lohmann, j., muula, a. s., houlfort, n., & de allegri, m. (2018). how does performance-based financing affect health workers’ intrinsic motivation? a self-determination theory-based mixed-methods study in malawi. social science & medicine, 208, 1-8. marcoulides, g. a., & saunders, c. (2006). editor's comments: pls: a silver bullet? mis quarterly, iii-ix. mars, m. (2013). telemedicine and advances in urban and rural healthcare delivery in africa. progress in cardiovascular diseases, 56(3), 326-335. martin, a. b., probst, j. c., shah, k., chen, z., & garr, d. (2012). differences in readiness between rural hospitals and primary care providers for telemedicine adoption and australasian journal of information systems zobair, sanzogni & sandhu 2020, vol 24, research article telemedicine adoption barriers 22 implementation: findings from a statewide telemedicine survey. the journal of rural health, 28(1), 8-15. marzagão, d. s. l., & carvalho, m. m. (2016). critical success factors for six sigma projects. international journal of project management, 34(8), 1505-1518. mathieson, k., leafman, j. s., & horton, m. b. (2017). access to digital communication technology and perceptions of telemedicine for patient education among american indian patients with diabetes. journal of health care for the poor and underserved, 28(4), 1522-1536. doi:10.1353/hpu.2017.0131 mayer, r. c., & davis, j. h. (1999). the effect of the performance appraisal system on trust for management: a field quasi-experiment. journal of applied psychology, 84(1), 123. menachemi, n., burke, d. e., & ayers, d. j. (2004). factors affecting the adoption of telemedicine—a multiple adopter perspective. journal of medical systems, 28(6), 617-632. miller, e. (2010). telemedicine and the provider-patient relationship: what we know so far. report prepared for the nuffield council’s working party on medical profiling and online medicine. miller, r. h., & sim, i. (2004). physicians’ use of electronic medical records: barriers and solutions. health affairs, 23(2), 116-126. modi, p. k., portney, d., hollenbeck, b. k., & ellimoottil, c. (2018). engaging telehealth to drive value-based urology. current opinion in urology, 28(4), 342-347. moffatt, j., & eley, d. (2011). barriers to the up-take of telemedicine in australia-a view from providers. rural and remote health journal, 11(1581), 1-6. muthupoltotage, u. p., & gardner, l. (2018). analysing the relationships between digital literacy and self-regulated learning of undergraduates—a preliminary investigation. in advances in information systems development (pp. 1-16): springer. nesbitt, t. s., marcin, j. p., martha, m., & cole, s. l. (2005). perceptions of local health care quality in 7 rural communities with telemedicine. the journal of rural health, 21(1), 7985. nessa, a., ameen, m., ullah, s., & kwak, k. s. (2008). applicability of telemedicine in bangladesh: current status and future prospects. paper presented at the convergence and hybrid information technology, 2008. iccit'08. third international conference on. neuman, l. w. (2014). social research methods: qualitative and quantitative approaches. pearson. nissen, l., & lindhardt, t. (2017). a qualitative study of copd-patients’ experience of a telemedicine intervention. international journal of medical informatics, 107, 11-17. oak, m. (2007). a review on barriers to implementing health informatics in developing countries. journal of health informatics in developing countries, 1(1). parimbelli, e., bottalico, b., losiouk, e., tomasi, m., santosuosso, a., lanzola, g., . . . bellazzi, r. (2018). trusting telemedicine: a discussion on risks, safety, legal implications and liability of involved stakeholders. international journal of medical informatics, 112, 90-98. paul, b. k. (1983). a note on the hierarchy of health facilities in bangladesh. social science & medicine, 17(3), 189-191. australasian journal of information systems zobair, sanzogni & sandhu 2020, vol 24, research article telemedicine adoption barriers 23 peters, d. h., garg, a., bloom, g., walker, d. g., brieger, w. r., & hafizur rahman, m. (2008). poverty and access to health care in developing countries. annals of the new york academy of sciences, 1136(1), 161-171. ramtohul, i. (2015). the adoption of e-health services: comprehensive analysis of the adoption setting from the user׳ s perspective. health policy and technology, 4(3), 286-293. rogove, h. j., mcarthur, d., demaerschalk, b. m., & vespa, p. m. (2012). barriers to telemedicine: survey of current users in acute care units. telemedicine and e-health, 18(1), 48-53. roldán, j. l., & sánchez-franco, m. j. (2012). variance-based structural equation modeling: guidelines for using partial least squares. research methodologies, innovations and philosophies in software systems engineering and information systems, 193. ross, j., stevenson, f., lau, r., & murray, e. (2016). factors that influence the implementation of e-health: a systematic review of systematic reviews (an update). implementation science, 11(1), 146. sarstedt, m., wilczynski, p., & melewar, t. (2013). measuring reputation in global markets— a comparison of reputation measures’ convergent and criterion validities. journal of world business, 48(3), 329-339. scott kruse, c., karem, p., shifflett, k., vegi, l., ravi, k., & brooks, m. (2018). evaluating barriers to adopting telemedicine worldwide: a systematic review. journal of telemedicine and telecare, 24(1), 4-12. scott, r. e., & mars, m. (2013). principles and framework for ehealth strategy development. journal of medical internet research, 15(7). shaarani, i., taleb, r., & antoun, j. (2017). effect of computer use on physician-patient communication using a validated instrument: patient perspective. international journal of medical informatics. , 108, 152-157. van dyk, l. (2014). a review of telehealth service implementation frameworks. international journal of environmental research and public health, 11(2), 1279-1298. van gemert-pijnen, j. e., nijland, n., van limburg, m., ossebaard, h. c., kelders, s. m., eysenbach, g., & seydel, e. r. (2011). a holistic framework to improve the uptake and impact of ehealth technologies. journal of medical internet research, 13(4). vishwanath, a., & scamurra, s. d. (2007). barriers to the adoption of electronic health records: using concept mapping to develop a comprehensive empirical model. health informatics journal, 13(2), 119-134. weiner, m., & biondich, p. (2006). the influence of information technology on patientphysician relationships. journal of general internal medicine, 21(s1). weinstein, r. s., lopez, a. m., joseph, b. a., erps, k. a., holcomb, m., barker, g. p., & krupinski, e. a. (2014). telemedicine, telehealth, and mobile health applications that work: opportunities and barriers. the american journal of medicine, 127(3), 183-187. whittaker, a. a., aufdenkamp, m., & tinley, s. (2009). barriers and facilitators to electronic documentation in a rural hospital. journal of nursing scholarship, 41(3), 293-300. australasian journal of information systems zobair, sanzogni & sandhu 2020, vol 24, research article telemedicine adoption barriers 24 whitten, p., holtz, b., meyer, e., & nazione, s. (2009). telehospice: reasons for slow adoption in home hospice care. journal of telemedicine and telecare, 15(4), 187-190. williams, a. m., bhatti, u. f., alam, h. b., & nikolian, v. c. (2018). the role of telemedicine in postoperative care. mhealth, 4, 11-11. doi:10.21037/mhealth.2018.04.03 wilson, b. (2010). using pls to investigate interaction effects between higher order branding constructs. in handbook of partial least squares (pp. 621-652): springer. zanaboni, p., & wootton, r. (2016). adoption of routine telemedicine in norwegian hospitals: progress over 5 years. bmc health services research, 16(1), 496. zhang, m., luo, m., nie, r., & zhang, y. (2017). technical attributes, health attribute, consumer attributes and their roles in adoption intention of healthcare wearable technology. international journal of medical informatics, 108, 97-109. copyright: © 2020 zobair, sanzogni & sandhu. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v24i0.2165 http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 1.1 research rationale 1.2 telemedicine in bangladesh 2 review of the literature (hypotheses and conceptual model) 2.1 organisational barriers 2.2 technological barriers 2.3 resource allocation barriers 2.4 health staff motivation barriers 2.5 quality of care barriers 2.6 patient satisfaction barriers 2.7 patient trustworthiness barriers 3 research methodology 3.1 research instrument development 3.2 sample and data collection 4 data analysis and results 4.1 measurement model 4.2 structural model 5 discussion 5.1 summary of findings 5.2 organisational barriers 5.3 health staff motivation barriers 5.4 patient satisfaction barriers 5.5 patient trustworthiness barriers 5.6 other barriers 5.7 implications 5.8 conclusions 5.9 limitations and future research 5.10 research highlights paper title (paper title style) australasian journal of information systems whiteside, aleti, pallant & zeleznikow 2018, vol 22, research on recent advances in social media helpful or harmful? 1 helpful or harmful? exploring the impact of social media usage on intimate relationships naomi whiteside rmit university naomi.whiteside@rmit.edu.au torgeir aleti rmit university jason pallant swinburne university john zeleznikow victoria university abstract internet access in australia is increasing with the most popular online activity reported by australians being social media use. the literature is divided as to whether social media use is helpful or harmful to intimate relationships. recent research from an australian family counselling provider indicates that social media use is resulting in negative intimate relationship outcomes, echoing recent american research findings. however, other researchers note that social media use can have positive outcomes for intimate relationships and individuals. these different findings prompted this research which focuses on the impact of social media usage on intimate relationships using facebook as a case study. a survey of 518 australians examining facebook usage, relationship satisfaction and household income was undertaken. the results suggest that relationship satisfaction varies based on types of social media usage when controlled for household income. for most respondents, social media usage did not have a negative impact on their relationship. however, results indicated that where their partner was a ‘friend’ on social media, they were less likely to report concerns relating to social media usage impacting their relationship. it was also found that those living alone or with children on low incomes were more likely to be dissatisfied with their relationships and rely on social media to connect with others. finally, four types of facebook usage were identified and mapped against intensity of usage and relationship satisfaction, indicating that the way in which social media is used impacts relationship satisfaction. these findings indicate that social media use is helpful for some intimate relationships and harmful for others based on individual usage and their interaction with their partner via the platform. keywords: social media; social networking; intimate relationships; facebook; relationship satisfaction. 1 introduction internet usage has become a natural part of everyday life in australia. in 2015, 86% of all australian households had an internet connection (abs, 2016). one of the most popular activities online is social networking with 72% of australians indicating that they used the internet at home for social networking (abs, 2016). while there is hardly any difference in internet usage between women and men (84 and 85% respectively), individuals with a australasian journal of information systems whiteside, aleti, pallant & zeleznikow 2018, vol 22, research on recent advances in social media helpful or harmful? 2 personal income below au $40,000, and those who had not completed secondary school, were less likely to use the internet (abs, 2016). online social networking sites allows users to form, build and grow relationships in new ways. some of the more popular social media sites are those classified as 'social community' sites (tuten & solomon, 2015). social community sites are those made for users to share, socialise and converse; typical examples are facebook, twitter and linkedin (constantinides, 2014). further, these social communities are used for different reasons. users primarily turn to facebook for relationships, linkedin to build a professional identity, while twitter is more about conversations than relationships (kietzmann, hermkens, mccarthy, & silvestre, 2011). since the focus in this research is on intimate relationships, facebook was selected as the most suitable case for investigation. facebook is also the most used social media platform in australia, with sensis (2017) reporting that 94% of respondents indicated that they used the site. this figure is higher than american estimates which note that facebook usage sits at 71% of online adults (pew research center, 2016). ‘facebook, like other social media sites, is a platform where users can update their networks about how they are feeling by regularly posting status updates’ (carlson, farrelly, frazer, & borthwick, 2015, p5). facebook is particularly useful in terms of enabling users to ‘relate’ to one another whereby ‘two or more users have some form of association that leads them to converse, share objects of sociality, meet up, or simply just list each other as a friend…’ (kietzmann et al., 2011, p. 246). as mobile device ownership increases, so does the frequency of access to social media sites, with 35% of users reporting visiting a social media site more than five times a day (sensis, 2017). the average australian now spends almost half a day per week (10 hours) on facebook alone (sensis, 2017). social media is accessed anytime and anywhere, with 57% reporting checking in first thing in the morning, 84% using it in their living room, and 12% using it in the toilet (sensis, 2017). a study carried out by a leading relationship support and counselling provider in australia found that social media was increasingly having a negative impact on intimate relationships (relationships australia, 2011). some research indicates that social networking sites are being used for interpersonal electronic surveillance (ies) (r s tokunaga, 2010) or partner surveillance (fox & warber, 2014). recent american research indicates that social media usage is contributing to negative relationship outcomes such as divorce (baker & oswald, 2010; clayton, nagurney, & smith, 2013). in addition, us legal practitioners report that social media content is increasingly being used as evidence in divorce proceedings (aaml, 2010). from this perspective, it appears that an increased intensity of social media usage is linked to dissatisfaction with intimate relationships. however, other studies have suggested an opposite effect as social media allows users ‘…to share personal information, build and maintain social relationships, and have fun’ (hu, kettinger, & poston, 2015 p. 391). caers et al. (2013) point out that while many studies examine facebook usage, the changing nature of the platform and the diversity and limitations of the samples studied to date indicates a need for on-going research into the topic. ellison, steinfield, and lampe (2007 p. 631) note that there is potential for individuals to overuse social media and consequently ‘…the impact of such extreme usage patterns on individuals and their romantic relationships needs further exploration’. papp, danielewicz, and cayemberg (2012) also highlight the need for further research into how technology use impacts intimate relationships. australasian journal of information systems whiteside, aleti, pallant & zeleznikow 2018, vol 22, research on recent advances in social media helpful or harmful? 3 this research seeks to explore the impact that social media use has on intimate relationships in the australian context with a specific focus on understanding the experience of low income respondents. the paper is structured as follows: a background to the study is presented that highlights recent relevant research relating to social media and relationships. this is followed by a discussion of the methodology and results. the paper concludes with a summary of the research and a discussion of potential future work. 2 background since the launch of facebook in 2004 social media has inspired numerous studies into the impact that usage of platforms has on the lives and relationships of users (boyd & ellison, 2007). social media use has been found to have both negative (clayton, nagurney, et al., 2013; clayton, osborne, miller, & oberle, 2013; mod, 2010; muise, christofides, & desmarais, 2009; rahaman, 2015) and positive impacts on users' relations to others (baker & oswald, 2010; n b ellison et al., 2007). in contrast, a recent american study found that internet use had no impact on relationships for more than 60% of respondents (pew research center, 2014). these and other studies are explored here. 2.1 peer group identification in the context of facebook, a ‘friend’ is an individual, group or organisation that users have a connection with via the social media platform. users may communicate and interact with friends in the social media environment. this is quite different from the commonly accepted notion that ‘friends are individuals who have a close, personal relationship with one another that is characterized by mutual positive feelings’ (erdley, nangle, & burns, 2011 p. 669). indeed gilbert and karahalios (2009 p. 211) note that ‘social media treats all users the same: trusted friend or total stranger, with little or nothing in between’. however, individuals generally have a range of friendships that are characterized by varying degrees of closeness, self-disclosure and reciprocity (gilbert & karahalios, 2009). the friendship spectrum can be characterized in terms of tie-strength (granovetter, 1973 p. 1361), the ‘combination of the amount of time, the emotional intensity, the intimacy (mutual confiding), and the reciprocal services which characterize the tie’. weak ties can be considered acquaintances that you do not know well whereas strong ties are trusted friends that you are closely involved with (gilbert & karahalios, 2009; hacker, wickramasinghe, & durst, 2017). in the context of this research ‘friend’ means a social media friend who may sit anywhere on the spectrum of tie-strength. to narrow this range, we adopted peer group, ‘…a group of individuals who are approximately the same age, formed from common interests and affective bonds’ (hamit, 2011 p. 1073), to indicate social media friends that are more than just weak ties. this is in line with the use of the term peer in the context of social media users by wang, yu, and wei (2012), who created a peer group identification scale (see appendix one) used in this research. valenzuela et al. (2010), indicate that social media provides an avenue of social support whereby individuals who are experiencing difficulties in their offline relationship can find solace online via social media friends. while carlson et al. (2015, pp. 6-7) note that social media ‘…offers a platform where users could receive support and encouragement through interactions with friends and family’. this is supported by song et al.’s (2014) research into a ‘social augmentation hypothesis’ or a ‘social compensation hypothesis’ which posits that australasian journal of information systems whiteside, aleti, pallant & zeleznikow 2018, vol 22, research on recent advances in social media helpful or harmful? 4 people seek company and support online when their offline network is limited (berger, 1986 pp. 48-49; song et al., 2014 p. 447). as tokunaga (2011 p. 425) notes, social networking sites have unique affordances ‘…that support the maintenance of existing relationships and formation of new social ties’. social media also provides ‘utilitarian benefits’ as it allows users to keep up to date with the activities and preferences of friends or connections (hu et al., 2015 p. 394). as such, when social media is used to connect with friends and family, it may have a positive impact on relationships. social capital can be described as ‘the benefits individuals derive from their social relationships and interactions: resources such as emotional support, exposure to diverse ideas, and access to non-redundant information’ (nicole b. ellison, steinfield, & lampe, 2011 p. 873). indeed, social media use can provide positive benefits for users, including increased social capital (n. ellison, steinfield, & lampe, 2006; n b ellison et al., 2007; nicole b. ellison et al., 2011; lambert, 2016). in addition, baker and oswald (2010) found that using social media increased perceptions of friendship quality and provided social support for people who were shy. 2.2 facebook intensity: beyond frequency and duration of social media use facebook intrusion can be characterised as the usage of facebook that interferes with daily life and personal relationships (n b ellison et al., 2007). this intrusion has been characterised by turel and serenko (2012, p. 514) as ‘social networking website addiction’ which turel (2015) notes may lead users to experience feelings of guilt. ellison et al. (2007 p. 631) contend that facebook intrusion, ‘…may be associated with relationship dissatisfaction’. this position is reflected in the work of muise, christofides, and desmarais (2009 p. 443) whose research showed that there was ‘…a significant association between time spent on facebook and jealousy-related feelings and behaviours experienced on facebook’. recent research in australia also highlights that the heavy use of social media (defined in the study as accessing it five or more times a day), can increase the likelihood of ‘fomo’; the fear of missing out (aps, 2015). fomo is indicated where individuals feel anxious when disconnected from social media and experience increased concern about missing out on social interaction with peers as a result (aps, 2015). a swedish study (denti et al., 2012 p. 26) reported that … ‘those who used facebook more [in this instance women with low incomes and education levels] rated themselves as feeling less happy and content with their lives’. this resonates with the work from chou and edge (2012) who found that students with higher levels of facebook usage were more likely to perceive that others had happier lives. maier, laumer, eckhardt, and weitzel (2015, pp. 447-448) note that social media users may experience ‘social overload’: feeling exhausted by requests for social support from friends in their online social networks. thus, people may use social media differently, and that socio-economic status could play an important role in to what extent facebook functions as a positive or negative influence on relationships. while the aforementioned research focuses on frequency and duration of use, ellison et al. (2007 p. 1150) created the ‘facebook intensity’ scale (see appendix one) to provide a ‘better measure of facebook use than frequency and duration indices’. facebook intensity encompasses frequency and duration of use, but also an individual’s feelings about using facebook as well as their number of ‘friends’. when intensity is seen in this way, it includes the size of an individual’s network, frequency of use as well as the sentiments towards the site. as such, intensity is likely to impact relationship satisfaction. ellison et al. (2007 p. 1162) found australasian journal of information systems whiteside, aleti, pallant & zeleznikow 2018, vol 22, research on recent advances in social media helpful or harmful? 5 ‘that certain kinds of facebook use (articulated by [the] facebook intensity items) can help students accumulate and maintain bridging social capital’. 2.3 relationship satisfaction some studies point to social media use playing a role in relationship problems (aaml, 2010; clayton, nagurney, et al., 2013; pew research center, 2014; relationships australia, 2011; relationships australia victoria, 2012; valenzuela, halpern, & katz, 2014). a pew research (2014) study found that almost one third of respondents felt that social media had an impact on their relationship. uncertainty reduction theory (urt), may help to explain these relationships. developed by berger and calabrese (1975), urt ‘argues that relationships develop when participants are able to reduce uncertainty about each other and disintegrate when participants are unable to reduce uncertainty about each other’ (parks & adelman, 1983 p. 57). while the initial theory focused on the early stages of relationship development, over time the theory has been applied to ‘…the entire relational life cycle’ (parks & adelman, 1983 p. 56). parks and adelman (1983 p. 56) extended urt, examining the theory and its linkages to relationship stability. specifically, they posit that increasing the ‘amount of communication with a partner’s network’ will reduce uncertainty and enhance the stability of a relationship (parks & adelman, 1983 p. 57). urt was used by rahaman (2015) in exploring the impact of facebook use on relationships and it is used in this study as a theoretical lens to support the analysis and interpretation of results (stewart & klein, 2016). we propose that social media provides another channel via which partners can engage with one another and their extended group of family and friends. as such, social media has the potential to increase connections and information sharing with a partner’s network, reducing uncertainty in a relationship and potentially enhancing relationship satisfaction. for the purposes of this study relationship satisfaction is measured using the relationship assessment scale (ras) developed by hendrick et al. (1988) (see appendix one). social media use could have an impact on relationship satisfaction for some couples. a recent study noted that those with a household income lower than $30,000 usd were likely to argue about the time their partner spent online, or to find ‘...the online activity of their partner upsetting’ (pew research center, 2014 p. 16). use of social media can cause conflict and drive dissatisfaction with relationships, as the platform provides a means for partners to meet others, leading to potential infidelity (mcdaniel, drouin, & cravens, 2017). dang-pham, pittayachawan, and nkhoma (2015) point out that stalking and harassment are among the negative behaviours enabled by such social networking services. therefore, we propose that what people do and who they do it with on social media, is just as important as the intensity of use and peer connections when exploring the ramifications in terms of relationship satisfaction. we also posit that the impact of social media usage on a relationship is influenced by the way in which a partner uses (or doesn’t use) social media. for instance, overuse may not be considered problematic if both partners use social media to a similar degree and for similar things. based on existing literature, we propose that there are multiple types of social media users. these types differ in the reasons for each individual’s usage of social media, their intensity of usage, and the dynamics of usage relative to their relationship partner. that is, an individual’s reasons for use, intensity of use, and usage relative to their partner, distinguishes between types of users. we propose that membership in these user groups is dependent on a range of australasian journal of information systems whiteside, aleti, pallant & zeleznikow 2018, vol 22, research on recent advances in social media helpful or harmful? 6 individual characteristics, including gender, age, marital status, location, education, socioeconomic status, and peer group identification. for example, we predict that some types of users will be more likely male than female, and so on. finally, we propose that relationship satisfaction will differ across different types of users. based on these propositions, the conceptual model displayed in figure 1 is proposed. figure 1. conceptual model. based on this conceptual model, this study seeks to address three key research questions. firstly, the literature reviewed above indicate that income and education in particular were related to how people use social media. in particular, denati et al. (2012) suggested that low income and education could be related to higher intensity of usage – with negative implications on wellbeing. the intensified online relationships, may in turn reflect satisfaction with intimate relationships. as such, we propose the following research question: research question 1: are there any differences in relationship satisfaction, facebook intensity and identification with peer groups on social media when it comes to household income and living arrangements? secondly, we wanted to further investigate the relationship between how people use facebook and their relationship satisfaction. previous research has suggested that facebook usage can have positive as well as negative impacts on relationship satisfaction. further, as suggested by pew research (2014), those on a lower income were more likely to find the online activities of their partners upsetting. this leads us to the second research question: research question 2: controlling for household income, does the way people use facebook influence relationship satisfaction? finally, we wanted to investigate to what extent people with different characteristics and usage patterns could be described as particular groups of facebook users. urt suggests that people differ in their ability to reduce uncertainty about each other in relationships. it is possible that this ability can be enhanced or reduced based on how and why people use social media such as facebook. the literature reviewed here suggest that usage patterns and personal characteristics may impact relationship satisfaction, but it is unclear what patters and characteristics are most salient. this leads us to our final research question: research question 3: based on intensity of facebook usage, reasons for usage and online relations with partner, a); what groups of user types exist, b); what factors predict membership in these groups, and c); do these groups differ based on relationship satisfaction? 3 methodology a survey was used to examine the impact facebook usage has on intimate relationships. specifically, this study sought to explore the way in which facebook was used and if specific australasian journal of information systems whiteside, aleti, pallant & zeleznikow 2018, vol 22, research on recent advances in social media helpful or harmful? 7 usage impacted intimate relationships. the survey was distributed through a reputable australian market research company (the online research unit, http://www.theoru.com/). 518 responses were elicited from their panel. four respondent screening criteria were used; participants had to be above 18 years of age, currently in a relationship, current users of facebook and 40% of the sample had to belong to a low-income category. an australian government standard for determining access to services and support was used to identify respondents on low income (dhs, 2015). the relationship between patterns of facebook usage and key demographics was also examined. demographics relating to relationship and marital status, age, education and income were used. the survey instrument consisted of three existing validated scales developed to examine aspects of relationship satisfaction and facebook use. the ‘facebook intensity’ scale (n b ellison et al., 2007) was developed to examine the degree of facebook usage via measures encompassing emotional connection and integration into daily routines in addition to frequency and duration of use. this widely used scale was adopted to quantify intensity of facebook use in this study. wang et al. (2012) developed and validated the ‘identification with peer group online’ scale to investigate the influence of peer group identification and communication on purchase intention. this scale was adopted for the study to investigate the influence of peer group identification on relationship satisfaction. the widely used ‘relationship assessment scale’ is a seven-item generic measure of relationship satisfaction developed and validated by hendrick (1988) and hendrick, dicke, & hendrick (1998), which was used to quantify relationship satisfaction. all three scales are presented in appendix one. the survey also included a set of questions about reasons for using facebook. these questions were based on the sensis (2016 p.35) report ‘reasons for using social networking sites’, and were adapted to suit this particular study. finally, the survey asked about the frequency of facebook usage compared with a partner, whether the participants included or excluded their partner (as a friend) on facebook, and whether their partner was part of the peer group that they would normally interact with on facebook (see appendix one). this was of particular interest as it facilitated examination of a connection with a partner’s network via facebook, supporting analysis with respect to extended uncertainty reduction theory (urt) (parks & adelman, 1983). the relationships between variables were explored using analysis of variance (anova), regression analysis, and latent class analysis (lca). we use lca to categorise patterns of facebook usage. we then tested different factors that influence which type of user each individual is. finally, the groups identified by this model were then compared with respect to levels of relationship satisfaction using anova and post-hoc tests. the results were analysed using the extended urt as a lens. 4 results the demographic profile of respondents is displayed in table 1. the sample included equal proportions of males and females from a range of age groups, educations, locations and marital statuses. australasian journal of information systems whiteside, aleti, pallant & zeleznikow 2018, vol 22, research on recent advances in social media helpful or harmful? 8 variable n percent gender male 255 49.2% female 263 50.8% age 18-29 55 10.6% 30-39 128 24.7% 40-49 110 21.2% 50-64 129 24.9% 6596 18.5% marital status never married 34 6.6% defacto 68 13.1% married 402 77.6% separated / divorced / widowed 14 2.7% habitat metropolitan 393 75.9% rural / remote 125 24.1% education primary school or some high school 62 12.0% completed high school 76 14.7% tafe certificate 147 28.4% tertiary 233 45.0% personal income under $20,000 125 24.1% $20,001-$40,000 93 18.0% $40,001-$60,000 103 19.9% $60,001-$80,000 84 16.2% $80,000-$100,000 55 10.6% over $100,000 58 11.2% household income (all figures in aud) live alone, income <$30,000 18 3.5% live with a partner, income <$45,000 142 27.4% live with children, income <$60,000 48 9.3% income >$60,001 310 59.8% table 1. sample profile. rq 1: household income, facebook and relationship satisfaction the first research question investigated differences in relationship satisfaction, facebook intensity and identification with peer groups on social media when it comes to household income and living arrangements. to investigate this question, an anova analysis was conducted. the results are reported on table 2. australasian journal of information systems whiteside, aleti, pallant & zeleznikow 2018, vol 22, research on recent advances in social media helpful or harmful? 9 variable n mean f sig. relationship satisfaction live alone, income <$30,000 18 3.85 4.12 .007 live with a partner, income <$45,000 142 4.17 live with children, income <$60,000 48 3.83 income >$60,001 310 4.22 facebook intensity live alone, income <$30,000 18 3.12 2.26 .081 live with a partner, income <$45,000 142 2.69 live with children, income <$60,000 48 3.04 income >$60,001 310 2.83 identification with peer group live alone, income <$30,000 18 2.99 3.18 .024 live with a partner, income <$45,000 142 2.58 live with children, income <$60,000 48 3.01 income >$60,001 310 2.78 table 2. anova: household income. the results indicate significant differences for relationship satisfaction [f(3,514) = 4.12, p<0.01] and identification with peer group [f(3,514) = 3.18, p<0.05] across the income groups. interestingly, relationship satisfaction is at a similar level to the high-income group for couples living together on less than $45,000 per year. however, the level of satisfaction drops for those living alone (on less than $30,000) and those living with dependent children on less than $60,000. these relationships appear opposite when it comes to facebook intensity (not significant) and identification with peers on facebook. thus, those living alone or with children on low income are less satisfied with their relationships and may rely more on facebook to connect with others. rq 2: ways of using facebook, household income and relationship satisfaction the second research question addressed how the differences in the ways that people use facebook influenced relationship satisfaction while also controlling for household income. to address this question, we split the relationship satisfaction variable into two categories; unsatisfied and satisfied. based on the seven-point scale measuring satisfaction, participants who answered three or below on most the questions were regarded as unsatisfied. this led to 136 (26.3%) unsatisfied participants, and 382 (73.7%) satisfied participants. further, we combined the first three categories of household income investigated in rq1 as low income (40.2%), and compared this category with the high-income category (59.8%). from this, we compared the percentage of satisfied and unsatisfied participants based on reasons for using facebook and compared these results for low and high income groups. table 3 presents the results of this analysis. the most interesting results are those which deviate by more than 10% from the overall level of unsatisfied (26.3%) and satisfied (73.7%) participants (highlighted). in low-income households only, participants seemed less satisfied with their relationship when the partner is a heavier facebook user then them; 39.0% of low household income participants with heavy facebook using partners were unsatisfied with their relationships. another key difference in relationship satisfaction was found based on whether respondents used facebook to keep in touch with family and friends. in the low-income category 47.7% of respondents who did not use facebook in this way were dissatisfied with their relationships. australasian journal of information systems whiteside, aleti, pallant & zeleznikow 2018, vol 22, research on recent advances in social media helpful or harmful? 10 a similar trend was also observed in the high-income category with 40.0% dissatisfied when facebook was not used to keep in touch with family and friends. variable low income household high income household unsatisfied satisfied unsatisfied satisfied partner fb usage more 39.0% 61.0% 29.5% 70.5% same 27.9% 72.1% 20.2% 79.8% less 26.4% 73.6% 21.7% 78.3% partner part of fb peer group no 30.2% 69.8% 26.6% 73.4% yes 30.4% 69.6% 20.4% 79.6% partner friend on fb no 29.1% 70.9% 25.0% 75.0% yes 31.0% 69.0% 22.9% 77.1% keep in touch with fam. & friends no 47.7% 52.3% 40.0% 60.0% yes 25.6% 74.4% 21.8% 78.2% for work no 27.6% 72.4% 23.1% 76.9% yes 62.5% 37.5% 29.2% 70.8% to connect with organisations no 26.4% 73.6% 21.9% 78.1% yes 57.7% 42.3% 32.0% 68.0% to gather information no 30.0% 70.0% 23.2% 76.8% yes 31.6% 68.4% 24.7% 75.3% to share information no 30.6% 69.4% 24.0% 76.0% yes 28.6% 71.4% 21.9% 78.1% to meet new people no 30.4% 69.6% 23.3% 76.7% yes 28.6% 71.4% 27.8% 72.2% table 3. differences in facebook usage based on household income and relationship satisfaction. low income respondents who used facebook for different reasons than keeping in touch with family and friends were more likely to be unsatisfied with their relationships. 62.5% of those who used facebook for work, and 57.7% of those who use facebook to connect with organisations were dissatisfied with their relationship. that is, some low-income participants may seek to ‘escape’ though social media, which relates to higher dissatisfaction with their relationships. to further investigate the relationships between ways of using facebook and relationship satisfaction, we conducted a regression analysis with relationship satisfaction as the dependent variable (dv), and household income and the facebook usage variables as independent variables (iv). the results are shown in table 4. the model explains about 8% of the variation in relationship satisfaction (r2=.078). considering the vast number of variables that would affect a person’s relationship satisfaction, it is surprising to find any relationships here at all. however, the model shows statistically significant relationships. keeping in touch with friends and family is positively related to satisfaction, while using facebook to connect with organisations is negatively related to relationship satisfaction. excluding the partner from the peer group on facebook was negatively associated with relationship satisfaction and those with partners that were heavy facebook users were less satisfied with their relationships. finally, living alone with children on a low income also negatively affects relationship satisfaction. the regression largely supports the variations in relationship satisfaction uncovered in table 4. australasian journal of information systems whiteside, aleti, pallant & zeleznikow 2018, vol 22, research on recent advances in social media helpful or harmful? 11 item std. β t sig. (constant) 18.177 .000 partner more fb use -.107 -1.991 .047 partner same fb use -.030 -.602 .548 partner part of fb peer group -.109 -2.078 .038 partner friend on fb -.087 -1.620 .106 keep in touch with fam. & friends .108 2.308 .021 for work .027 .579 .563 to connect with organisations -.129 -2.746 .006 to gather information .001 .017 .986 to share information .013 .265 .791 to meet new people .023 .502 .616 facebook intensity -.091 -1.269 .205 identification with peer group -.008 -.110 .913 live alone, income <$30,000 -.069 -1.589 .113 live with a partner, income <$45,000 -.021 -.469 .639 live with children, income <$60,000 -.103 -2.256 .025 table 4. regression analysis of relationship satisfaction and facebook usage. rq 3: different types of facebook users the third research question related to whether different groups of facebook users could be identified based on the variables of facebook usage and the intensity of use, what factors may predict membership in these groups, and whether these groups differ based on relationship satisfaction. we tested this question through a latent cluster analysis (lca). the convergence criterion was set at 0.000001 (collins & lanza, 2010) and 50 random sets of starting parameters were used (masyn, 2013) to reduce the likelihood of convergence to a local maxima (mccutcheon, 2002). the variables in this lca were the same as in table 3, with the addition of the average score on the facebook intensity scale. the bayesian information criterion (bic) was used to compare relative model fit (collins & lanza, 2010; masyn, 2013). the 4-cluster solution resulted in the lowest bic value, suggesting this solution provided the best relative model fit (collins & lanza, 2010; masyn, 2013). finally, the segment profiles were considered to examine over-extraction (masyn, 2013), class separation (collins & lanza, 2010) and interpretability of the results (wedel & kamakura, 2000). the 4-cluster solution displayed no evidence of over-extraction, as the smallest cluster was 10%. additionally, these clusters led to meaningful interpretation as the clusters showed strong class separation. table 5 displays the profile of the developed clusters in the chosen 4cluster solution. cluster 1 (39%) use facebook only to keep in touch with family and friends and most of this cluster has their partner as a friend on facebook (91%). they are also relatively intense users of facebook with the second highest facebook intensity score. as such, this cluster is given the label connectors. the second cluster (29%) also primarily use facebook to keep in touch with family and friends (80%). however, in contrast to cluster 1, most respondents in cluster 2 do not have their partner as a friend on facebook (90%). in addition, they have the lowest intensity of use in the sample, and their partners are likely to be even lighter users than them (possibly non-users). this cluster is therefore labelled light users. cluster 3 (22%), display the highest intensity of facebook usage (mean = 3.65), and are the only cluster where most respondents use facebook to gather (60%) and share information (65%). based on this profile, the third cluster is labelled heavy users. finally, cluster 4 (10%) describes a category of australasian journal of information systems whiteside, aleti, pallant & zeleznikow 2018, vol 22, research on recent advances in social media helpful or harmful? 12 disconnectors, as most of this cluster do not use facebook for any of the listed tasks. although they have an account, their intensity of use remains low. additionally, this cluster was the most likely to state that their partner uses facebook more than them (88%), which further verifies the conscious disconnect from facebook. cluster 1 (connectors) cluster 2 (light users) cluster 3 (heavy users) cluster 4 (disconnectors) cluster size % 39 29 22 10 keep in touch with fam. & friends %no 00 20 09 68 %yes 100 80 91 32 for work %no 100 98 82 67 %yes 00 02 18 33 to connect with organisations %no 99 93 54 80 %yes 01 07 46 21 to gather information %no 92 88 40 76 %yes 08 12 60 24 to share information %no 95 93 35 92 %yes 05 07 65 08 to meet new people %no 100 99 76 95 %yes 00 01 24 05 partner fb usage %more 38 04 16 88 %same 35 11 30 11 %less 27 85 54 01 partner part of fb peer group %yes 69 02 62 59 %no 31 98 38 41 partner friend on fb %yes 91 10 86 94 %no 09 90 14 06 facebook intensity mean 2.78 2.37 3.65 2.44 table 5. cluster profiles based on preferred 4-cluster solution the next step of our analysis included investigating the factors predicting ‘cluster membership’. this was achieved by including demographic data as well as strength of peer group identification on social media as covariates in the lca. the results of this covariate analysis are displayed in table 6. australasian journal of information systems whiteside, aleti, pallant & zeleznikow 2018, vol 22, research on recent advances in social media helpful or harmful? 13 covariates connectors light users heavy users disconnectors wald p-value male 0.31 -0.76 0.37 0.08 12.79 0.01 female -0.31 0.76 -0.37 -0.08 age 0.00 0.06 -0.01 -0.05 12.63 0.01 never married 0.22 0.25 -0.61 0.14 5.36 0.80 defacto 0.19 -0.05 0.34 -0.47 married -0.36 -0.05 -0.43 0.84 separated/divorced/ widowed -0.05 -0.15 0.69 -0.50 metropolitan -0.03 -0.15 0.12 0.06 1.44 0.70 rural/remote 0.03 0.15 -0.12 -0.06 primary school or less 2.01 0.91 -2.09 -0.84 17.94 0.12 some high school -0.27 0.15 -0.17 0.29 completed high school -0.37 -0.01 0.47 -0.09 tafe certificate -0.99 -0.46 0.89 0.56 tertiary -0.38 -0.59 0.90 0.08 live alone, income <$30,000 -0.90 0.66 -0.41 0.65 12.53 0.18 live with a partner, income <$45,000 0.40 -0.55 -0.08 0.23 live with children, income <$60,000 -0.72 -0.31 -0.36 1.39 income >$60,001 1.22 0.21 0.85 -2.27 under $20,000 1.48 0.42 1.95 -3.85 13.66 0.55 $20,001-$40,000 0.46 0.51 0.04 -1.00 $40,001-$60,000 0.15 -0.07 -0.21 0.13 $60,001-$80,000 -0.48 -0.44 -0.42 1.34 $80,000-$100,000 -0.64 0.01 -0.61 1.24 over $100,000 -0.97 -0.42 -0.74 2.13 peer group identification 0.50 -0.81 1.54 -1.24 37.52 0.00 table 6. results of covariate analysis based on preferred 4-cluster solution. this covariate analysis displayed significant results for gender (wald = 12.79, p<0.05), age (wald = 12.63, p<0.05), and peer group identification (wald = 37.52, p<0.01). no differences were found regarding household income, marital status, location, education and income. this means that neither of these variables influences cluster membership in this analysis. for the variables with significant main effects, the specific coefficients were considered. a positive coefficient suggests that covariate increases the probability of membership in a cluster, while a negative coefficient implies the opposite. for gender, there was a positive impact on membership in the connectors (0.31), heavy users (0.37) or disconnectors (0.08) clusters for males, and a large negative impact on membership in the light users cluster (-0.76). that is, the light users are most likely women. in terms of age, increased age led to a higher probability of membership in the light users cluster (0.06) and lower likelihood of membership in the disconnectors cluster (-0.05). finally, peer identification had a positive impact on membership in either the heavy users (1.54) or connectors (0.50) clusters, but a negative impact on the disconnectors (-1.24) and light users (-0.81) clusters. this means that the heavy users and the connectors feel more connected to their peers on facebook than the light users and disconnectors. australasian journal of information systems whiteside, aleti, pallant & zeleznikow 2018, vol 22, research on recent advances in social media helpful or harmful? 14 with the developed clusters profiled and labelled, they were compared in terms of relationship satisfaction. this was achieved through a one-way anova where the dv was the average score on the relationship satisfaction scale, and the iv was cluster membership. tukey’s posthoc tests were used to compare differences between individual clusters. the results of this anova analysis are displayed in table 7. cluster n mean f sig. connectors 217 4.29 3.19 0.02 light users 145 4.06 heavy users 115 4.08 disconnectors 41 4.03 total 518 4.16 table 7. anova: clusters and relationship satisfaction. the results of the anova suggest that there are significant differences between the clusters of facebook users and relationship satisfaction. the connectors are the most satisfied (4.29) while the disconnectors are the least satisfied (4.03). light users (4.06) and heavy users (4.08) have medium relationship satisfaction comparatively. post-hoc analysis demonstrated the difference between connectors and light users was significant as p<0.05. based on the above analysis as well as the findings in the first two research questions, the matrix in table 8 classifies facebook users based on the intensity level of facebook usage and level of relationship satisfaction. user type type of use individual characteristics relationship satisfaction disconnectors (10%) have a facebook account, but do not use it for any particular reason, and have a low level of intensity of use. they are connected with their partners on facebook, but their partners are heavier users than them. more likely to be younger males with a low level of identification with their peer group. lowest light users (29%) use facebook to connect with friends and family, but without including their partner, and with very light intensity of use. strongly skewed towards females, and older users, with lower levels of peer group identification. 2nd lowest heavy users (22%) having the highest level of use intensity, these users connect with friends and family on facebook, but are also using it for a range of other activities including gathering and sharing information. they are friends with their partners, but their partners are lighter users. more likely to be males of average age, with high peer group identification. 2nd highest connectors (39%) use facebook primarily to connect with family and friends, are friends with their partner – who is also part of their peer group and have a relatively high intensity of facebook usage. more likely to be males (of any age). they also feel a stronger connection to their peers on facebook. highest table 8. facebook usage types ordered by relationship satisfaction (lowest to highest). australasian journal of information systems whiteside, aleti, pallant & zeleznikow 2018, vol 22, research on recent advances in social media helpful or harmful? 15 5 discussion and analysis the first research question prompted investigation of relationship satisfaction, facebook intensity and identification with peer groups with respect to household income and living arrangements. it was found that those living alone or with children and on a low income were less satisfied with their relationships and may rely more on social media to connect with friends and family. this echoes recent pew research (2014) findings, highlighting an area requiring further research and attention from support service providers. like the findings of denti et al. (2012), we found that facebook usage did not seem to assist in relationships satisfaction for those on low income with children or those living alone on limited means. as such, social media may not be used as a ‘remedy’ to assist those who struggle financially. the second research question explored if the way facebook was used influenced relationship satisfaction, while controlling for household income. the results indicated that there were differences in relationship satisfaction based on the way facebook was used. in instances where facebook was not used to keep in touch with family and friends, higher rates of relationship dissatisfaction were found. this was particularly associated with low income households. when a partner is a heavier user of facebook, low income respondents were more likely to be unsatisfied with their relationship. they were also less likely to be satisfied when they used facebook to connect with organisations or for work. excluding a partner from your group of ‘friends’ on facebook was also related to lower levels of relationship satisfaction. this finding aligns with uncertainty reduction theory (urt) which posits that increasing the ‘amount of communication with a partner’s network’ will reduce uncertainty and enhance the stability of a relationship (parks & adelman, 1983 p. 57). the final research question explored whether people could be grouped based on their facebook usage and whether these groups differed based on relationship satisfaction. the study found that people could be grouped and that relationship satisfaction did vary based on facebook usage. table 8 presents the results highlighting four key groups: disconnectors (low relationship satisfaction), light users and heavy users (medium relationship satisfaction) and connectors (high relationship satisfaction). the disconnector group is differentiated by light levels of facebook usage intensity and weak connections with their peer group on facebook. disconnectors are more likely to be younger and to have low income. connectors are moderate facebook users who primarily use facebook to connect with family and friends and are more likely to feel connected to their facebook peer group. the distinction between connectors and disconnectors can be framed through the lens of urt (parks & adelman, 1983), in that relationship satisfaction is enhanced through the maintenance of strong connections with family and friends and a partner on facebook. the light and heavy users on the other hand, differed in intensity of usage and connection with peers on facebook, but not in relationship satisfaction. while the light users tend to be older females, the heavy users are younger females. what distinguishes them from the connectors is the level of inclusion of their partner. again, this supports urt suggesting that facebook usage does not enhance one’s relationship when partners are excluded or less involved in the social media activities. recent australian and us research indicates that social media use is having a negative impact on users’ relations to others (clayton, nagurney, et al., 2013; clayton, osborne, miller, & oberle, 2013; mod, 2010; muise, christofides, & desmarais, 2009; rahaman, 2015). this is in contrast with studies that indicate that social media use can have a positive impact on users’ australasian journal of information systems whiteside, aleti, pallant & zeleznikow 2018, vol 22, research on recent advances in social media helpful or harmful? 16 relations to others (baker & oswald, 2010; n b ellison et al., 2007). this motivated our research to get a better sense of whether social media use was helpful or harmful in terms of its impact on intimate relationship satisfaction. the results uncovered, that for some users, social media use is harmful and for others it is helpful. however, it is not purely use or overuse that leads to social media being harmful to intimate relationships. rather, our findings indicate that users can be grouped into four types and that connection with one’s partner via social media plays a key role in maintaining relationship satisfaction. 6 theoretical and practical implications the findings of this research provide theoretical and practical implications. the study indicates that urt holds in the social media setting. that is, increasing the ‘amount of communication with a partner’s network’ reduces uncertainty and enhances the stability of a relationship (parks & adelman, 1983 p. 57). in addition this study contributes to the current literature which focuses on the us context, through an australian case study, building the body of knowledge relating to social media use and its impact on intimate relationships. one catalyst for the research was a recent finding of an australian counselling provider that social media is increasingly contributing to negative relationship outcomes for clients (relationships australia, 2011). our findings have the potential to inform counselling practice and support resource development through the identification of four usage types that experience differing levels of relationship satisfaction. importantly, our results indicate that intensity or overuse is not as critical to relationship satisfaction as how and who you use social media with. 7 limitations and agenda for future research an online panel was used to collect the sample for this study. online panels (especially amazon's mturk) have been criticised for being narrow in terms of demographics, as well as having users intentionally cheating with their answers (smith, roster, golden, & albaum, 2016). however, other studies have found the opposite, suggesting that online panels produce better results than face-to-face research approaches (casler, bickel, & hackett, 2013; hauser & schwarz, 2016). in our study, a reputable australian market research company (the online research unit, http://www.theoru.com/) was used. further, the sample was purposely selected to include a proportion of low-income participants. further research may wish to collect a sample using a different approach and compare to what extent the results varies. this research project relied on 'traditional' methods, using a 'small data' approach. the approach in itself has its limitations when the topic investigated relates to usage of a service that more than nine out of ten australians use in some form. to extend the current study, a big data approach could for example use an api to scrape social media data and then use the meta data together with a text analysis software such as linguistic inquiry and word count (liwc) (pennebaker, booth, boyd, & francis, 2015; pennebaker, boyd, jordan, & blackburn, 2015) to investigate what users talk about, who they talk with and how they express themselves linguistically. such a study could provide further insight into for example different emotional tones used as well as social status. thus, it would be possible to map different types of interactions on social media. however, a big data approach would have difficulties in capturing the level of satisfaction with user's offline intimate relationships. australasian journal of information systems whiteside, aleti, pallant & zeleznikow 2018, vol 22, research on recent advances in social media helpful or harmful? 17 these findings provide a snapshot of the impact that facebook usage has on relationship satisfaction in the australian context. the findings relating to low income groups are new in this context, augmenting recent findings from the united states. future research should further examine the impact that social media use has on low income groups. 8 conclusion this study examined the impact that facebook usage has on relationship satisfaction with a specific focus on exploring the impact on low income groups. findings indicate that relationship satisfaction varies based on the kind of facebook usage, not just the frequency and duration of facebook usage, as has been highlighted in previous studies. four key groups of facebook users were developed: disconnectors, light users, heavy users and connectors. connectors had the highest level of relationship satisfaction and were exemplified through using facebook to connect with family and friends, strong identification with their peer group on facebook and including their partner in their group of facebook ‘friends’. these findings echo urt which posits that maintaining close ties with a partner’s family and friends can lead to enhanced relationship outcomes. the literature indicates that social media use can have a helpful or harmful impact on intimate relationships. our findings extend this body of knowledge, showing that how you use social media and who you use social media to connect with plays a key role in relationship satisfaction. references aaml. (2010). big surge in social networking evidence says survey of nation's top divorce lawyers. retrieved from http://aaml.org/about-the-academy/press/press-releases/ediscovery/big-surge-social-networking-evidence-says-survey abs. (2016). 8146.0 household use of information technology, australia, 2014-15. retrieved from http://www.abs.gov.au/ausstats/abs@.nsf/mf/8146.0 aps. (2015). stress & wellbeing: how australians are coping with life. retrieved from http://www.psychology.org.au/assets/files/pw15-sr.pdf baker, l. r., & oswald, d. l. (2010). shyness and online social networking services. journal of social and personal relationships, 27(7), 873-889. doi:10.1177/0265407510375261 berger, c. r. (1986). uncertain outcome values in predicted relationships uncertainty reduction theory then and now. human communication research, 13(1), 34-38. doi:10.1111/j.1468-2958.1986.tb00093.x berger, c. r., & calabrese, r. j. (1975). some explorations in initial interaction and beyond: toward a developmental theory of interpersonal communication. human communication research, 1(2), 99-112. doi:10.1111/j.1468-2958.1975.tb00258.x boyd, d. m., & ellison, n. b. (2007). social network sites: definition, history, and scholarship. journal of computer-mediated communication, 13(1), 210-230. doi:10.1111/j.10836101.2007.00393.x caers, r., de feyter, t., de couck, m., stough, t., vigna, c., & du bois, c. (2013). facebook: a literature review. new media & society, 15(6), 982-1002. doi:10.1177/1461444813488061 australasian journal of information systems whiteside, aleti, pallant & zeleznikow 2018, vol 22, research on recent advances in social media helpful or harmful? 18 carlson, b., farrelly, t., frazer, r., & borthwick, f. (2015). mediating tragedy: facebook, aboriginal peoples and suicide. australasian journal of information systems, 19. doi:http://dx.doi.org/10.3127/ajis.v19i0.1174 casler, k., bickel, l., & hackett, e. (2013). separate but equal? a comparison of participants and data gathered via amazon’s mturk, social media, and face-to-face behavioral testing. computers in human behavior, 29(6), 2156-2160. doi:https://doi.org/10.1016/j.chb.2013.05.009 chou, h.-t. g., & edge, n. (2012). 'they are happier and having better lives than i am': the impact of using facebook on perceptions of others' lives. cyberpsychology, behavior & social networking, 15(2), 117-121. doi:10.1089/cyber.2011.0324 clayton, r. b., nagurney, a., & smith, j. r. (2013). cheating, breakup, and divorce: is facebook use to blame? cyberpsychology, behavior & social networking, 16(10), 717-720. doi:10.1089/cyber.2012.0424 clayton, r. b., osborne, r. e., miller, b. k., & oberle, c. d. (2013). loneliness, anxiousness, and substance use as predictors of facebook use. computers in human behavior, 29(3), 687-693. doi:http://dx.doi.org/10.1016/j.chb.2012.12.002 collins, l. m., & lanza, s. t. (2010). latent class and latent transition analysis: with applications in the social, behavioral, and health sciences. nj, usa: wiley. constantinides, e. (2014). foundations of social media marketing. procedia social and behavioral sciences, 148, 40-57. doi:10.1016/j.sbspro.2014.07.016 dang-pham, d., pittayachawan, s., & nkhoma, m. (2015). demystifying online personas of vietnamese young adults on facebook: a q-methodology approach. australasian journal of information systems, 19. doi:http://dx.doi.org/10.3127/ajis.v19i0.1204 denti, e., barbopoulos, i., nilsson, i., holmberg, l., thulin, m., wendeblad, m., . . . davidsson, e. (2012). sweden's largest facebook study. retrieved from http://hdl.handle.net/2077/28893 dhs. (2015). low income supplement. retrieved from http://www.humanservices.gov.au/customer/services/centrelink/low-incomesupplement ellison, n., steinfield, c., & lampe, c. (2006, 19-23 june). spatially bounded online social networks and social capital: the role of facebook. paper presented at the annual conference of the international communication association, dresden, germany. ellison, n. b., steinfield, c., & lampe, c. (2007). the benefits of facebook “friends:” social capital and college students’ use of online social network sites. journal of computermediated communication, 12(4), 1143-1168. doi:10.1111/j.1083-6101.2007.00367.x ellison, n. b., steinfield, c., & lampe, c. (2011). connection strategies: social capital implications of facebook-enabled communication practices. new media & society, 13(6), 873-892. doi:10.1177/1461444810385389 erdley, c. a., nangle, d. w., & burns, a. m. (2011). friends. in s. goldstein & j. a. naglieri (eds.), encyclopedia of child behavior and development. new york: springer. australasian journal of information systems whiteside, aleti, pallant & zeleznikow 2018, vol 22, research on recent advances in social media helpful or harmful? 19 fox, j., & warber, k. m. (2014). social networking sites in romantic relationships: attachment, uncertainty, and partner surveillance on facebook. cyberpsychology, behavior & social networking, 17(1), 3-7. doi:10.1089/cyber.2012.0667 gilbert, e., & karahalios, k. (2009). predicting tie strength with social media. paper presented at the chi '09 the sigchi conference on human factors in computing systems, boston, ma, usa. granovetter, m. s. (1973). the strength of weak ties. american journal of sociology, 78(6), 13601380. doi:doi:10.1086/225469 hacker, j., wickramasinghe, n., & durst, c. (2017). can health 2.0 address critical healthcare challenges? insights from the case of how online social networks can assist in combatting the obesity epidemic. australasian journal of information systems, 21. doi:http://dx.doi.org/10.3127/ajis.v21i0.1357 hamit, s. (2011). peer group. in s. goldstein & j. a. naglieri (eds.), encyclopedia of child behavior and development. new york: springer. hauser, d. j., & schwarz, n. (2016). attentive turkers: mturk participants perform better on online attention checks than do subject pool participants. behavior research methods, 48(1), 400-407. doi:10.3758/s13428-015-0578-z hendrick, s. s. (1988). a generic measure of relationship satisfaction. journal of marriage and family, 50(1), 93-98. doi:10.2307/352430 hendrick, s. s., dicke, a., & hendrick, c. (1998). the relationship assessment scale. journal of social and personal relationships, 15(1), 137-142. doi:10.1177/0265407598151009 hu, t., kettinger, w. j., & poston, r. s. (2015). the effect of online social value on satisfaction and continued use of social media. european journal of information systems, 24(4), 391-410. doi:10.1057/ejis.2014.22 kietzmann, j. h., hermkens, k., mccarthy, i. p., & silvestre, b. s. (2011). social media? get serious! understanding the functional building blocks of social media. business horizons, 54(3), 241-251. doi:10.1016/j.bushor.2011.01.005 lambert, a. (2016). intimacy and social capital on facebook: beyond the psychological perspective. new media & society, 18(11), 2559-2575. doi:doi:10.1177/1461444815588902 maier, c., laumer, s., eckhardt, a., & weitzel, t. (2015). giving too much social support: social overload on social networking sites. european journal of information systems, 24(5), 447464. doi:10.1057/ejis.2014.3 masyn, k. (2013). latent class analysis and finite mixture modeling. in t. d. little (ed.), the oxford handbook of quantitative methods in psychology (vol. 2, pp. 551-611). mccutcheon, a. l. (2002). basic concepts and procedures in single-and multiple-group latent class analysis. in j. a. hagenaars & a. l. mccutcheon (eds.), applied latent class analysis (pp. 56-88). mcdaniel, b. t., drouin, m., & cravens, j. d. (2017). do you have anything to hide? infidelityrelated behaviors on social media sites and marital satisfaction. computers in human behavior, 66, 88-95. australasian journal of information systems whiteside, aleti, pallant & zeleznikow 2018, vol 22, research on recent advances in social media helpful or harmful? 20 mod, g. b. b. a. (2010). reading romance: the impact facebook rituals can have on a romantic relationship. journal of comparative research in anthropology and sociology, 1(2), 61-77. muise, a., christofides, e., & desmarais, s. (2009). more information than you ever wanted: does facebook bring out the green-eyed monster of jealousy? cyberpsychology, behavior & social networking, 12(4), 441-444. doi:10.1089/cpb.2008.0263 papp, l. m., danielewicz, j., & cayemberg, c. (2012). 'are we facebook official?' implications of dating partners' facebook use and profiles for intimate relationship satisfaction. cyberpsychology, behavior & social networking, 15(2), 85-90. doi:10.1089/cyber.2011.0291 parks, m. r., & adelman, m. b. (1983). communication networks and the development of romantic relationships: an expansion of uncertainty reduction theory. human communication research, 10(1), 55-79. doi:10.1111/j.1468-2958.1983.tb00004.x pennebaker, j. w., booth, r. j., boyd, r. l., & francis, m. e. (2015). linguistic inquiry and word count: liwc2015. austin, tx: pennebaker conglomerates (www.liwc.net). pennebaker, j. w., boyd, r. l., jordan, k., & blackburn, k. (2015). the development and psychometric properties of liwc2015. austin, tx: university of texas at austin. pew research center. (2014). couples, the internet, and social media. retrieved from http://www.pewinternet.org/files/2014/02/pip_couples_and_technologyfin_021114.pdf pew research center. (2016). social networking fact sheet. retrieved from http://www.pewinternet.org/fact-sheets/social-networking-fact-sheet/ rahaman, h. m. s. (2015). romantic relationship length and its perceived quality: mediating role of facebook-related conflict. europe’s journal of psychology, 11(3), 395-405. doi:10.5964/ejop.v11i3.932 relationships australia. (2011). relationships indicators survey 2011. retrieved from http://www.relationshipsvictoria.com.au/assets/pdfs/indicatorsreportfinal2011.pdf relationships australia victoria. (2015). social media use impacts families and relationships. retrieved from https://www.relationships.org.au/what-we-do/research/onlinesurvey/february-2015-the-internet-and-relationships sensis. (2016). sensis social media report 2016: how australian people and businesses are using social media. retrieved from https://www.sensis.com.au/asset/pdfdirectory/sensis_social_media_report_2016.pdf sensis. (2017). sensis social media report 2017. retrieved from https://www.sensis.com.au/asset/pdfdirectory/sensis-social-media-report-2017.pdf smith, s. m., roster, c. a., golden, l. l., & albaum, g. s. (2016). a multi-group analysis of online survey respondent data quality: comparing a regular usa consumer panel to mturk samples. journal of business research, 69(8), 3139-3148. doi:https://doi.org/10.1016/j.jbusres.2015.12.002 song, h., zmyslinski-seelig, a., kim, j., drent, a., victor, a., omori, k., & allen, m. (2014). does facebook make you lonely?: a meta analysis. computers in human behavior, 36, 446452. doi:http://dx.doi.org/10.1016/j.chb.2014.04.011 australasian journal of information systems whiteside, aleti, pallant & zeleznikow 2018, vol 22, research on recent advances in social media helpful or harmful? 21 stewart, d., & klein, s. (2016). the use of theory in research. international journal of clinical pharmacy, 38, 615-619. tokunaga, r. s. (2010). social networking site or social surveillance site? understanding the use of interpersonal electronic surveillance in romantic relationships. computers in human behavior, 27, 705-713. tokunaga, r. s. (2011). friend me or you'll strain us: understanding negative events that occur over social networking sites. cyberpsychology, behavior & social networking, 14(7/8), 425-432. doi:10.1089/cyber.2010.0140 turel, o. (2015). quitting the use of a habituated hedonic information system: a theoretical model and empirical examination of facebook users. european journal of information systems, 24(4), 431-446. doi:10.1057/ejis.2014.19 turel, o., & serenko, a. (2012). the benefits and dangers of enjoyment with social networking websites. european journal of information systems, 21(5), 512-528. tuten, t. l., & solomon, m. r. (2015). social media marketing (2 ed.). thousand oaks, ca: sage. valenzuela, s., halpern, d., & katz, j. e. (2014). social network sites, marriage well-being and divorce: survey and state-level evidence from the united states. computers in human behavior, 36, 94-101. wang, x., yu, c., & wei, y. (2012). social media peer communication and impacts on purchase intentions: a consumer socialization framework. journal of interactive marketing, 26, 198-208. wedel, m., & kamakura, w. (2000). market segmentation: conceptual and methodological foundations. ny, usa: springer science & business media. australasian journal of information systems whiteside, aleti, pallant & zeleznikow 2018, vol 22, research on recent advances in social media helpful or harmful? 22 appendix one – survey items ellison et al.’s (2007) facebook intensity scale was adapted for use in the study. unless provided, response categories ranged from 1 = strongly disagree to 5 = strongly agree: o about how many total facebook friends do you have? (0-10=1, 11-50=2, 51-200=3, 201400=4, 401-=5) o in the past week, on average, approximately how many minutes per day have you spent on facebook? (0-10=1, 11-30=2, 31-60=3, 61-120=4. 121-=5) o facebook is part of my everyday activity o i am proud to tell people i’m on facebook o facebook has become part of my daily routine o i feel out of touch when i haven’t logged onto facebook for a while o i feel i am part of the facebook community o i would be sorry if facebook shut down hendrick’s (1988) scale was used to measure relationship satisfaction. categories ranged from 1 = low, 3=neutral to 5=high. o how well does your partner meet your needs? o in general, how satisfied are you with your relationship? o how good is your relationship compared to most? o how often do you wish you hadn't gotten into this relationship? o to what extent has your relationship met your original expectations? o how much do you love your partner? o how many problems are there in your relationship? wang et al.’s (2012 p. 203) identification with the peer group scale was adapted for use: o i am very attached to the peer group on facebook. o my peers on facebook and i share the same objectives. o the friendships i have with my peers on facebook mean a lot to me. o if my peers on facebook planned something, i'd think of it as something “we” would do rather than “they” would do. o i see myself as a part of the peer group on facebook. items used to measure facebook usage based on (sensis, 2016): o please select the options that best indicate why you use facebook (more than one allowed): o to keep in touch with family and friends o i use it for work o i use it to connect with organisations o i use it to gather information o i use it to share information o i use it to meet new people o my partner uses facebook: (more than me, less than me, about the same amount) o is your partner part of a group of friends that you most frequently interact with (your peer group) on facebook? yes/no australasian journal of information systems whiteside, aleti, pallant & zeleznikow 2018, vol 22, research on recent advances in social media helpful or harmful? 23 o is your partner your friend on facebook? yes/no copyright: © 2018 whiteside, aleti, pallant & zeleznikow. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 background 2.1 peer group identification 2.2 facebook intensity: beyond frequency and duration of social media use 2.3 relationship satisfaction 3 methodology 4 results 5 discussion and analysis 6 theoretical and practical implications 7 limitations and agenda for future research 8 conclusion paper title (paper title style) australasian journal of information systems shaik & makhecha 2019, vol 23, research on the role of tech in workforce mgt employee engagement in virtual teams 1 drivers of employee engagement in global virtual teams farheen fathima shaik indian institute of management tiruchirappalli farheen.f16006@iimtrichy.ac.in upam pushpak makhecha indian institute of management tiruchirappalli abstract global virtual teams (gvts) comprise geographically distributed groups of people collaborating with each other through technology-mediated communication. members of gvts are from different cultural backgrounds and time zones, who may (or may not) meet in person to take complex decisions or to deliver on the tasks that are of strategic importance. though technology has enabled gvts in almost all multinational organisations across all industries, keeping the members of gvts engaged over the duration of the team's task or project could still pose a challenge for organisations. employee engagement is defined as an employee's cognitive, behavioural and physical state directed towards organisational outcomes. while employee engagement has been researched in a collocated team context, it remains an under-researched area in the context of gvts. given that there are several characteristics of gvts which are distinct from the collocated team, it warrants a separate inquiry, which we undertake in this study. this study uses the job demands-resources theory of employee engagement to derive the drivers of employee engagement in gvts. through interpretive analysis of the lived experiences of members working in an organisation which extensively uses gvts for achieving its strategic goals, we conceptualise five drivers of employee engagement, namely, cultural intelligence, communication (formal and informal), technology, trust and individual maturity. keywords: employee engagement; geographically dispersed teams; global virtual teams; phenomenology; technology mediated communication. 1 introduction technological developments and global mobility have resulted in expansion of the scope of teams beyond the traditional collocated forms (matlay & martin, 2009) towards geographically dispersed and global forms of teams working virtually and connected through technology, commonly known as the global virtual teams (gvts) (jiminez, boehe, taras, & caprar, 2017). these types of teams are also known as “multinational and multicultural distributed teams” (connaughton & shuffler, 2007) or “transnational teams” (haas, 2006). gvts, as a form of organising work through technology-mediated communication and through a workforce spread across cultures and boundaries, has become a norm for all types and forms of organisations rather than being limited to large firms. such teams include not only full-time employees but also part-time employees, freelancers, suppliers and other forms of collaborations (jiminez et al., 2017). such teams require an appropriate choice of technology for interaction among team members for effective communication via technology (morgan, paucar-caceres, & wright, 2014). australasian journal of information systems shaik & makhecha 2019, vol 23, research on the role of tech in workforce mgt employee engagement in virtual teams 2 due to the increase in virtual forms of work mediated through technology, and multicultural nature of workforce that is becoming ubiquitous and indigenous to all types of firms, it is prudent for researchers and practitioners to explore and understand the challenges of working in gvts. dulebohn and hoch (2017) report the challenges faced by members of gvts as 1) understanding speech patterns in multicultural gvts; 2) leading efficiently; 3) combining and collaborating with different time zones; 4) developing personal rapport; 5) being on technology-mediated communication; and 6) selecting appropriate communication channels. these challenges are known to have a negative impact on results, including productivity, costs, deadlines, and retention as well as morale and stress of the employees (grenny & maxfield, 2017; tran, yang, & raikundalia, 2004). in general, one of the measure used by organizations to counter some of these negative impacts is to focus on increasing employee engagement levels (gilson, maynard, young, vartiainen, & hakonen, 2015). higher levels of employee engagement in organizations have been reported to increase the financial performance and employee productivity (harter et al, 2002). observations by consulting firms like aon, gallup, also suggest that employee engagement drives organizational results. especially aon hewitt (2005) emphasizes that they “have established conclusive, compelling relationship between engagement and profitability through higher productivity, sales, customer satisfaction, and employee retention”. however, it is alarming to note that the engagement levels are found to be lower in gvts. practitioner report such as kpmg (2012) specifies that though “costconscious businesses are turning to technology to become increasingly flexible and virtual….employee engagement measures from gvts are lower than those for on-site teams that work close to each other” (pg. 4). though the practitioner literature reports reduced levels of employee engagement in gvts, there is scant scholarly work on employee engagement in gvts. thus, in our view, due to the changed context of working in gvts which is characterised by virtuality, technology-mediated communication and multicultural interaction, and the ubiquitous presence of gvts in organisations, it is critical to understand the phenomenon of employee engagement in gvts through scholarly research and identify factors that can influence engagement of team members of gvts. this study is an interpretative analysis of employee engagement in gvts. as employee engagement in virtual teams is an underexplored area (gilson et al., 2015), as a starting point this research has used phenomenology as its methodological approach to understand employee engagement from the narratives and lived experiences of team members of gvts. from this interpretative analysis, we conceptualise the drivers of employee engagement that emerge from our study and compare these with antecedents of employee engagement researched in collocated teams as per extant literature. in a nutshell, this study tries to answer the question, “what drives employee engagement in the context of gvts?” in following sections, we first provide a brief overview of the challenges teams face in a virtual context. we then highlight the importance of employee engagement in organisations and theories used to explain employee engagement, with specific focus on job demands-resources (jd-r) theory, which we use in this study to understand employee engagement in gvts. this section also highlights reasons for the authors to anticipate change in drivers of employee engagement in gvts as compared to collocated teams. the third section of this paper discusses the relevance of phenomenology as a methodology in deriving the drivers of employee engagement in gvts. we then move on to describe evolved themes and their interpretations, followed by a discussion and end with a conclusion. australasian journal of information systems shaik & makhecha 2019, vol 23, research on the role of tech in workforce mgt employee engagement in virtual teams 3 2 virtuality and teams due to rapid changes in environment such as digitization, globalisation, and increased complexity, organisations today have turned towards team-based work structures in order to respond quickly to the dynamic environment to stay competitive. at the same time, with technology, communication, and competition, it has become necessary for organisations to unite globally distributed units with differences in culture and skill-sets. this has made information and communication technologies inevitable for a team to subside the spatial and temporal barriers, and meet the demands of work (alnuaimi, robert, & maruping, 2010), making gvts a preferred choice for managing work. gvts have been earlier defined by henry and hartzler (1998) as “groups of people who work closely together even though they are geographically separated by miles or even continents” (pg. 5) and later by powell, piccoli, and ives (2004) as “a group of geographically, organisationally and/or time dispersed workers brought together by information technologies to accomplish one or more organisation tasks” (pg. 7) to capture the temporary nature of gvts and the critical role of communication technologies across gvt members. the concept of virtual implies permeable interfaces and boundaries; teams that rapidly form, reorganise and dissolve when the needs of dynamic marketplace change; and individuals with differing competencies located across different time, space and cultures (bell & kozlowski, 2002). these teams also address new workforce demographics where the best employees may be located anywhere in the world and where workers increasingly are demanding technological sophistication and personal flexibility (burn & robins, 2002; townsend, demarie, & hendrickson, 1998). the main assumption behind formation of such teams is that people will contribute as much in gvt settings, as they do in a collocated team (alnuaimi, robert, & maruping, 2010). however, as per existing literature, members of gvts face certain challenges and this is reflected in low individual commitment; role overload; role ambiguity; absenteeism and social loafing; communication and collaboration difficulties; potentially lower team engagement by team members; difficulty in creating and maintaining trust; high levels of social and psychological isolation between members; and challenges in managing and monitoring teams (dulebohn & hoch, 2017; johansen, 1994; matley & martin, 2009). further, as organisations implement new technologies into team-based structures, researchers began to identify hurdles that these teams may need to overcome ( dulebohn & hoch, 2017; schaubroeck & yu, 2016) such as limitations in team identification (raghuram & wiesenfeld, 2004); managing diversity (gibson & gibbs, 2006); and determining leadership in gvts (bell & kozlowski, 2002; hoch & kozlowski, 2014). addressing and keeping such challenges in check has the potential to enable the advantages of working in a virtual environment (burgess, kerr, & houghton, 2013). in a global virtual context, a team assumes that its members set interdependencies and a common purpose that is unique to the team. the individual efforts are pooled and the need for interacting directly is limited. therefore, the quality of communication in a team becomes important, particularly in case of distributed teams compared to those that are not (morgan, paucar-caceres, & wright, 2014; schaubroeck & yu, 2016). in our view, the virtual context of working in gvts with dispersion of employees over multiple geographic locations, coupled with complete reliance on technology for communication and other such factors could potentially reduce the levels of employee engagement, as also reported by practitioner organisations (aon-hewitt, 2013; kpmg, 2012) therefore, in this study, we contend that as the challenges faced by members working in gvts are different when compared to a collocated australasian journal of information systems shaik & makhecha 2019, vol 23, research on the role of tech in workforce mgt employee engagement in virtual teams 4 team, we propose that dynamics of employee engagement in gvts might be different from those of collocated teams. the next section discusses employee engagement and its antecedents in the context of collocated teams, and also argues that existing antecedents might not completely predict employee engagement in a virtual, technology-mediated and multicultural context. 3 employee engagement the existing literature acknowledges that employee engagement is key to organisational success and competitive advantage (macey, schneider, barbera, & young, 2009; rich, lepine, & crawford, 2010; saks & gruman, 2014). also, it is one among the very few constructs which have captured the attention of academicians and practitioners alike (saks & gruman, 2014). according to kahn (1990), when employees are engaged, they bring cognitive, emotional and behavioural aspects of themselves at work towards performance, thus displaying their complete selves within their job roles (saks & gruman, 2014). and when employees are disengaged they decouple themselves from their work roles, and withdraw themselves cognitively, emotionally and physically during role performances (kahn, 1990, pg. 694). according to maslach, schaufeli, and leiter (2001) employee engagement involves vigour, dedication and absorption of employees in their jobs and is seen as an antithesis of burnout dimensions of exhaustion, cynicism, and inefficacy. recently, shuck and wollard (2010), define employee engagement as “an individual employee’s cognitive, emotional, and behavioural state directed toward desired organisational outcomes” (pg. 103). this definition conceptualises the three dimensions given by kahn (1990) and also relates them to organisational performance and hence we take this definition of employee engagement for this paper. the extant literature uses several models and theories to explain employee engagement. three major theories beingthe theory of psychological meaningfulness by kahn (1990), engagement as a positive antithesis of burnout (maslach et al., 2001) and the job demands-resources (jdr) model by bakker and demerouti (2007) (saks & gruman, 2014). the first theory of employee engagement by kahn (1990) found that employee engagement was a function of experiencing three psychological conditions, namely, psychological meaningfulness, psychological safety, and psychological availability. those employees who experience higher levels of psychological meaningfulness (being valuable, useful and worthwhile); psychological safety (being able to express one’s true self without the fear of negative consequences to one’s selfimage, status or career) and availability have been reported to engage more in their roles (saks & gruman, 2014). those social systems that are predictable, consistent and non-threatening provide higher levels of psychological safety (kahn, 1990). in the context of gvts, we question the sense of psychological safety due to the interactions among team members that are predominantly mediated through a technological medium. the expression of self through various forms of technology might differ in comparison to a collocated team where the emphasis is on face to face (ftf) communication (morgan et al., 2014). in context of gvts we anticipate that this can lead to either a reduced or alleviated form of psychological meaningfulness of work. psychological availability is when an employee is present psychologically, emotionally, and physically in their work, leading to higher levels of engagement. considering the contextual difference in gvts, the authors assume that expression of availability might also differ considerably in comparison to a collocated team. australasian journal of information systems shaik & makhecha 2019, vol 23, research on the role of tech in workforce mgt employee engagement in virtual teams 5 the second theory of engagement is based on burnout literature. maslach et al. (2001) discuss that engagement is the antithesis of burnout. according to them, burnout is an outcome of mismatch of six critical areas of organisational life, which are also reported to be antecedents of burnout: workload, control, rewards and recognition, community and social support, perceived fairness, and values. the greater alignment of these areas will lead to improved employee engagement, and it includes sustainable workload, feelings of choice and control, appropriate recognition and reward, supportive work community, fairness and justice, and meaningful and valued work (maslach & leiter, 2008). considering the context of gvts, the authors contend that among the six factors identified by maslach et al. (2001), contextual factors like perceived fairness, values, community and social support, and control in gvts might take newer meanings for the team members. changes in these factors may help us understand the ways in which employees feel engaged in a geographically dispersed and technologically mediated team. limiting factors of gvts like lack of ftf interactions, extensive geographic dispersion makes us assume that levels of engagement might differ across multiple forms of communication. job demands-resources (jd-r) theory is an extension of the burnout theory in which bakker and demerouti (2007) emphasise on working conditions to improve employee engagement. this was first used to explain burnout as a consequence of two processesone, high job demands which leads to exhaustion; and two, lack of job resources which leads to withdrawal of employees. these two broad categories of job demands and job resources have been used to explain employee engagement as well. proposed originally by demerouti, bakker, nachreiner, and schaufeli (2001), the jd-r model describes how job demands and resources trigger motivational and psychological health impairment processes. meta-analytic evidence shows that motivational processes through job resources such as autonomy, task variety, feedback, training and development, participation in decision-making, co-worker support, and supervisor support are positively related to employee engagement (crawford, lepine, & rich, 2010; halbesleben, 2010). there is also meta-analytic evidence which states that job demands such as work overload, role conflict and ambiguity, emotional demands, can adversely impact employee health, wellbeing and performance (albrecht & anglim, 2018; alarcon, 2011). we anticipate that job resources such as feedback, participation in decision making, co-worker support, and supervisor support could be different in gvts in comparison to a collocated teams. also, changes in job demands, especially emotional demands due to contextual differences of gvts can trigger changes in engagement level in gvts. this paper relies on the jd-r model as it focuses on the contextual factors of gvts that predict engagement and disengagement of employees rather than psychological factors as in other theoretical approaches. the change in context of gvts and the limitations that team members of gvts face can affect the drivers of their engagement. therefore, this study tries to answer the question, “what drives employee engagement in gvts?” 4 methodology in this study, we use phenomenology, a form of qualitative inquiry which is a philosophy (heidegger, 1962; husserl, 1962) and a methodology (schutz, 1967) which incorporates details of experiences in everyday life at a rather mundane level. in our view, phenomenological inquiry best suited the context of this study as we wanted to understand employee engagement in gvts from the team members’ perspective reflected in their experiences as it australasian journal of information systems shaik & makhecha 2019, vol 23, research on the role of tech in workforce mgt employee engagement in virtual teams 6 is an underexplored research area. the main goal of phenomenology is to critically reflect the conscious experience rather than subconscious motivation, by uncovering important and invariant features of those lived experiences of participants (jopling, 1996) and, thus, there is "fidelity to the phenomena as it is lived" (wilson, 2002). phenomenology emphasises on “careful study of individuals to discover deeper meaning of ‘lived’ experiences in terms of the individual relationship with time, space and personal history” (stern, 1994). the new context and lack of academic literature on employee engagement in gvts necessitated the need to first understand the phenomena as is, from the narratives of lived experiences of team members, and make meaning of their stories towards developing convergences and divergences according to the phenomenological accounts. in this study, interpretation of participants’ reflection on employee engagement in gvts emerged through a circular interplay between developing understanding of phenomenological interviews (thompson, locander, & pollio, 1989) and ongoing immersion in several key literatures relevant to participants’ reflections, namely, (1) jd-r model of employee engagement; (2) literature on gvts; (3) on multicultural implications; and, on (4) technology adaptation in organisations. the resulting interpretation seeks to articulate a theoretical interplay between drivers of employee engagement as articulated in academic literature, and phenomenological descriptions of meanings that events and experiences assume in life worlds of the members of gvts. the study was carried out at cyient ltd., a global organisation headquartered in hyderabad, india, with a presence in 47 countries and employing about 14,000 employees. cyient extensively uses gvts due to its geographical presence in many countries. at the outset of this investigation, a list of potential participants was developed from cyient ltd. purposive sampling criteria for phenomenological interviews (thompson, 1996) were based on the degree of virtuality of the team, and the experience of working in gvts. the degree of virtuality was based on three criteria as first given by o’leary and cummings (2007): (1) spatial, which is the geographic distance among team members; (2) temporal, which is the time difference among team members; and, (3) configurational, which is the locations where team members work alone or the uneven distribution of team members. these criteria provided a specific set of circumstances to situate the findings of this study and hence facilitate in-depth and localised accounts of the phenomena (sherry, 1991). the master data set of the employees was accessed to select teams based on the above selection criteria. three teams (26 members) were selected accordingly (see table 1). team managers of these three teams were contacted for consent to contact their respective team members. sixteen of the 26 members of the three gvts selected responded positively towards the research. the teams were selected where respondents belonged to managerial positions. this purposive selection was done as these members have experienced virtual teams as managers as well as team members, and the questionnaire was designed accordingly. it was expected that their experience of working in a virtual team can be more holistic leading to richer data for research. before the interviews, participants were assured of anonymity and they signed consent forms which explained the purpose and intended use of this investigation. australasian journal of information systems shaik & makhecha 2019, vol 23, research on the role of tech in workforce mgt employee engagement in virtual teams 7 table 1: teams selected for phenomenological interviews (nr: not responded) team designation gender service at cyient age location country continent interview interview medium (seating) duration team a general manager male 15.2 43 hyderabad manikonda india asia 0:55:00 telephone vice president (sales) male 6 56 london uk europe 0:57:00 telephone senior sales manager male 4 48 kuala lumpur malaysia southeast asia 0:45:00 webex assistant general manager male 3 43 sydney australia australia 0:43:00 telephone senior project manager male 16.7 65 paragould usa north america nr nr senior sales manager male 2.2 32 melbourne australia australia nr nr associate vice president male 14.4 52 dulles usa north america nr nr deputy general manager male 0.9 46 melbourne australia australia nr nr team b associate vice president male 16.1 56 hyderabad madhapur india asia 0:45:00 face to face assistant manager female 12.2 39 hyderabad manikonda india asia 1:03:00 face to face associate vice president male 6.7 45 east hartford usa north america 0:45:00 telephone senior marketing director female 2.3 45 london uk europe 0:55:00 webex senior project manager male 15.4 41 hyderabad madhapur india asia 0:40:00 face to face sales manager (singapore) male 1.1 53 seletar singapore southeast asia 0:37:00 telephone deputy general manager male 8.2 43 hyderabad – manikonda india asia 1:10:00 face to face senior project manager male 6 43 bangalore india asia 1:05:00 face to face senior project manager male 6.5 55 east hartford usa north america nr nr team c design engineer male 8.4 30 hyderabad manikonda india asia nr nr design engineer female 7.6 31 hyderabad manikonda india asia 1:10:00 telephone design engineer male 6.3 31 west perth australia australia 1:03:00 telephone design engineer male 8.4 29 peoria usa north america nr nr design engineer male 3.8 31 melbourne australia australia nr nr design engineer male 3 53 west perth australia australia nr nr manager a senior vice president & bu head male 12 49 hyderabad madhapur india asia nr nr manager b senior vice president & bu head male 8.6 43 hyderabad madhapur india asia 0:37:00 face to face manager c senior project manager male 3.7 41 hyderabad manikonda india asia 1:05:00 telephone australasian journal of information systems shaik & makhecha 2019, vol 23, research on the role of tech in workforce mgt employee engagement in virtual teams 8 as this research follows the tradition of qualitative inquiry we emphasised the development of highly in-depth analysis of life stories expressed by a small number of participants (levy 1981; mccracken, 1988; thompson, locander, & pollio, 1990). the purpose of using a restricted sample was to gain an in-depth understanding of the experiences and also assumptions behind those experiences through which the participants construe their world (mccracken, 1988). this becomes an important rationale for the objectives of phenomenological research based on hermeneutics. the interviews occurred either ftf or through telephone or webex calling, based on the availability and comfort of participants and we followed the conventions of phenomenological interviewing as given by thompson et al. (1989, 1990). three pilot interviews were conducted by the researcher to fine tune the interview schedule. the interviews were conducted largely through dialogue set by the participants and followed up by questions based on experiences described by them. the interview dialogues ranged from 45 to 70 minutes. nvivo pro 11 was used to transcribe audio tapes into text, leading to verbatim transcripts ranging from 15 to 25 single-spaced pages for each participant account. these transcripts of audio-taped interviews served as primary texts on which findings of this interpretative account is based. apart from the transcripts, researchers had also taken notes while in the organisation which amounted to about 150 pages of everyday diary writing and constant observation. interpretation in phenomenological analysis is proceeded by means of hermeneutic logic (thompson, pollio, & locander, 1994). an iterative process of reading, documenting, and systematising interview transcripts (thompson, 1996) was done. the transcripts were closely read to gain a sense of the whole and during this initial reading, the authors also started noting down key phrases, metaphors, and patterns of meanings. with every iteration, interpretation was developed and an effort towards getting a grasp of thematic similarities and meaningbased linkages was made. this process was undertaken for each participant and when the researcher felt that an internal structure is derived for a participant, attention was turned towards another and the process was repeated. an effort has also been made to note down similarities and differences from the interpreted interviews. as this iterative process continued, a larger pattern to capture essential dimensions through thematic relationships was derived. throughout the process, we developed a thematic structure by continuously modifying, questioning, and returning to the experiences described by each participant in the interviews, as suggested by thompson (1996). the iterative process was conducted using coding in nvivo pro 11. there were 240 nodes identified in the first round of the coding process. further, convergences led to the identification of 19 subordinate themes (appendix ii), from these themes nine emic interpretations were identified and finally these were mapped to five etic interpretations including: (1) cultural intelligence, (2) communication, (3) technology, (4) trust and (5) individual maturity. as pike (1967) states “the etic viewpoint studies behaviour as from outside of a particular system and is an essential initial approach to an alien system, and the emic viewpoint results from studying behaviour as from inside the system” (pg. 37). 5 a hermeneutic interpretation as a general overview, the interviews suggest that team members participating in this study had experience of working in gvts since the start of their career, and they shared a common set of work settings and social circumstances. further, awareness of belonging to gvts seemed to be closely related to their perception that they, and other team members of the gvt, australasian journal of information systems shaik & makhecha 2019, vol 23, research on the role of tech in workforce mgt employee engagement in virtual teams 9 were working in a context which is very different from that of a collocated team. all members were aware of the context that they were working in, and the typified views of being part of gvts. for example, throughout the interviews, participants spontaneously compared themselves to the working conditions and team members of a collocated team. in presenting the following hermeneutic account, our goal is to retain the holistic quality that characterised the participants’ descriptions while still providing analytic clarity. we first discuss the general structure of the thematic framework that emerged from hermeneutic analysis of interview transcripts. figure 1 portrays the system of thematic relations emerging from interpretation of interview texts. the representation of data in figure 1 follows the representation given by thompson (1996). the vertical continuum ranges from ‘experiencenear’ to ‘conceptually-abstract’ (thompson, 1996). this continuum suggests that concrete experiences and behaviours of participants manifest into higher order patterns of sociocultural meanings and processes. figure 1: levels of abstraction derived from phenomenological accounts of the participants, australasian journal of information systems shaik & makhecha 2019, vol 23, research on the role of tech in workforce mgt employee engagement in virtual teams 10 this continuum also represents interpretative movement from various levels of emic meanings to an etic framework that we use to situate the participants’ perceptions in a broader system of sociocultural meanings. the emic nomenclature has been derived from the vernacular of the participants. the levels of abstraction towards emic interpretations included, (1) participants’ perceptions about a specific experience; (2) reflective meanings of those experiences; and (3) meanings that emerge by the interpretation of larger patterns across all interviews (thompson, 1996). after presenting the emic account, an etic framework is conceptualised that links drivers of employee engagement in gvts. the placement of various themes of emic-focused interpretation (see figure 1) was done based on relationships that emerged from the narratives of the respondents’ lived experiences. it was found that gvt members faced challenges (demands) due to lack of goal clarity and coordination issues. and gvt members were trying to bridge these challenges by being sensitive to temporal and spatial distance among members and through constant communication for setting clear expectations on goals (resources). similarly, the issues of conflicts and informal communication (demands) were being bridged by building adequate interpersonal relationships among the members of gvts (resources) through various means. also, context of gvts implies that there is cultural diversity and extensive use of technology media for the team to be connected (demands). and, it was found that team members’ cultural intelligence, trust and maturity (resources) was a requirement for adequate engagement and functioning in a virtual team. each of the emic focused theme has been elaborated below. 5.1 focus on goal clarity senior marketing director (london): it is about setting the expectations and expected results upfront. it helps you and the individual to be clear on the deliverables. it is always easy to point somebody out when they are not doing the right thing. and in gvts, this takes high priority due to lack of personal and face to face interaction. this theme highlights the importance of being structured in setting expectations and deliverables, and on putting conscious efforts by team members towards structuring the forms of working. the team members also emphasise periodic meetings using technology like video/audio conferencing, or calls to discuss the progress of team work. increased frequency in regular meetings allows compensating for lack of physical interaction among team members of gvts. senior sales manager (kuala lumpur): we properly define challenges and action plan every week. and, every week we also have a meeting with the senior management and update them about the progress. so, no progress means i am sitting at home! senior project manager (india): as a manager, one has to ensure that the team is working on a single goal. it becomes very important while working in a virtual team. while in a collocated team it is also possible to work with less structure and do unplanned meetings on a need basis, participants emphasise that in gvts, unstructured and unplanned meetings and discussions cannot be undertaken due to the dispersion of the team temporally and spatially. thus periodic meetings and constant reiteration of goals help build clarity on goals which helps members of gvt to bring their cognitive and behavioural selves to work on goals important to the gvt. australasian journal of information systems shaik & makhecha 2019, vol 23, research on the role of tech in workforce mgt employee engagement in virtual teams 11 5.2 coordination issues: senior project manager (india): gvts face on-site and off-site issues. i was once working from new york, and my team was in india. due to the change in time zones, we faced a lot of difficulty in getting support from each other. and when i was back in india, the process of follow-up was also difficult. most participants stated that owing to contextual challenges of gvts, coordination is a challenge faced by team members and managers of gvts. vice president sales (london): most of the members get involved 7/24 with the team. some of my team is in far-off europe. it so happens sometimes that they forget other team members who are based out of their offices in their emails. these situations reduce coordination within the team. some team members feel left out. so, out of sight, out of mind situation often occurs in gvts. the members also emphasised that interdependency of team members in gvts is very high. even though members seem to be independently working from different geographical locations, sensitisation of members towards better coordination reduces emotional and behavioural fallouts and thus eases the functioning of gvts. 5.3 cultural diversity: senior vice president, business unit head (india): there was one person who i worked with in germany, she was a fantastic contributor who had taken significant responsibility and did that very well. during one of the feedback sessions, when i asked her what are the things we can do better and how is she coming along, one thing that she said was, during the initial days, she was actually thinking of leaving the organisation, because she felt so disrespected in the conference calls. she said that “everybody is talking over me, and that is not something that i am used to”. and also said, if somebody is talking, you finish that conversation, you hear that person, and then you make a point, and then the next person makes a point right, she is very used to that. she had not worked in a global team before but she said that it took time for her to understand that this is the way that people talk, (sic) even though she could never understand the reason behind it. the culture of ‘structure and rigidity’ in the west and that of ‘flexibility and chaos’ in the east has been highlighted by most participants, and they focus on the clutter that happens when these two cultures are to work along with each other in gvt context. our study found that to cope with cultural sensitivity issues, managers have their own adaptive strategieseither the manager goes for a detailed ‘background study’ or ‘contact switching’. contact switching is to speak the same language in the accent of the receiver. sales manager (singapore): the only thing i had to do was contact switching. by contact switching what i mean is even though i am speaking english, i speak in a different way, so they understand. so this is what is contact switching. and it works well. the majority of participants of this study emphasised challenges of cross-cultural environment in gvts. and, therefore, being sensitive to cultures, local festivals, diversity of a team, and being flexible to team members' cultures facilitates working in gvts. the participants also reflect that the multicultural nature of gvts makes driving change a challenging task as australasian journal of information systems shaik & makhecha 2019, vol 23, research on the role of tech in workforce mgt employee engagement in virtual teams 12 different cultures have different mindsets and have different factors for getting motivated. language and email etiquette have been stressed by the participants. being careful with words while responding to emails can lessen the perceptual differences and misinterpretations of expectations in a gvt. thus understanding and respecting cultural differences may enhance the cognitive, emotional and behavioural contributions of members of gvts. 5.4 team member trust and maturity the participants emphasised the role of trust in gvts. gvts require additional levels of trust due to lack of ftf interactions. it has been said that the need to build trust comes from the need to be dependent on the team members in gvts, especially so when members have not seen or met each other in person. general manager (india): in gvts, the relationships are built only on trust. everyone wants to be a part of a winning story. due to spatial dispersion, a lot of communication can go wrong due to lack of trust. design engineer (west perth): people take trust and give it back to you. i am here, and my team is in india. coordinating in different time zones is only possible if i can trust that my team is there to support me. further, several members emphasised the need for team members to be mature enough to be able to handle the nuances and challenges of gvts. all participants pinpointed the need to be self-managed rather than being micro-managed by managers in gvts. associate vice president (east hartford): there are some people who will find it difficult to work in a gvt because here at the end of the day you are not answerable to your manager, you are answerable to the goals, and you are answerable to the outputs that you have to come up with. thus when team members are mature and are able to build trust amongst each other, we expect that they would be able to bring their complete selves towards the performance in gvts. 5.5 technology medium senior vice president, bu head (india): technology is an integral part of even having gvts. if you are not connected on a very regular basis, gvts cannot work. all participants unanimously pointed out that the existence of gvts depends on technology. technology is the tool to collaborate, progress, enable and stay connected in gvts. the choice of virtual infrastructures such as video conferencing, skype calls, phone or email is dependent on context, and type of communication and task required for the team members. most connections are through email. vice president (london): i think a lot of this has to depend on what i call the virtual infrastructure. it is very important in gvts that you have the proper tools, means to connect to the company, means to extract the data and put it back, and means to talk to different people in your team. also, it has been noted that most participants mention reduced cost due to the existence of virtual forms of work. sales manager (singapore): technology has also enabled us to reduce the commuting through flights, back and forth etc. australasian journal of information systems shaik & makhecha 2019, vol 23, research on the role of tech in workforce mgt employee engagement in virtual teams 13 apart from being an enabler, technology has also surfaced as a deterrent in gvts, where it was noted that hiding information is easier in gvts compared to a collocated team. the functioning of gvts depends on a network of communication. the strength of connection determines the quality of communication. the lags in responses due to technology, the problems of transferring large files, and the problem of hiding behind text messages and email communication have emerged as challenges of technology through these interviews. alongside the technical issues, technology also demands more individual efforts by team members to build interpersonal connections among team members. the interviews highlight that in gvts the team members need to put extra efforts to connect, bond and understand each other. they need to overcome problems faced by team members due to lag in response, language misinterpretation, and bad networks. it has been noted that millennials are more comfortable and excited to work in gvts as they are also digital natives compared to their seniors. 5.6 conflicts: senior project manager (india): there is conflict, but the problem in a gvt is more difficult to identify. but, conflict does exist. and when you get into conflict, personally, i have not been successful in doing it while (sic) video conference. you have to get together and sort it up. most members interviewed view conflict management as one of the most important challenges as it just brews inside and does not surface. this, according to the participants, complicates the ways of working and compounds the challenges faced in gvts by withholding or hiding information and this we feel as hindrance in team members to contribute fully in gvts. design engineer (india): a lot of times, when conflict gets into gvts, that becomes unproductive for work, one has to have a face to face meeting with people in conflict and resolve it. it is very difficult to resolve it in a virtual environment. 5.7 interpersonal relationships vice president (london): [...] one is you get to know each other when you form gvts and you are talking for the first time. of course, through formal communication, you get to know each other, some of the details. but then informal communication happens when you send an email and you agree and put a small smiley. this helps in building interpersonal relationships in a virtual environment. the participants working in gvts mention that interpersonal skills are needed to adapt to conditions of gvts. skills that include writing emails, the etiquette of communication over technological media, and being sensitive in usage of words in text and email messaging have been mentioned. apart from the above skills, it has also been emphasized that having adequate knowledge regarding the cultural and national background of the team member helps in building better interpersonal relations. senior vice president (india): if one has done their homework and understand that there are circumstances and situations that could be impacting the other person differently, then, communication with the other person helps build better interpersonal relationships with the members who are geographically dispersed. this will in a way allow to reduce the distance between the locations (sic). australasian journal of information systems shaik & makhecha 2019, vol 23, research on the role of tech in workforce mgt employee engagement in virtual teams 14 5.8 informal communication it has been found in this study that informal communication among members of gvts allows them to get to know each other better and increases ‘team bonding’ given the challenge of not being able to meet on a daily basis. senior marketing director (singapore): i think it is important for a team to understand and also getting that additional information (through informal communication) is always helpful. design engineer (india): i think it is important for a team to understand and also getting that additional information (through informal communication) is always helpful. further, participants specify that informal communication is difficult in gvts as ftf interactions are reduced and, further, communications which occur outside the work boundaries in a collocated team are less possible in a virtual context. the participants specially mention that in gvts the social needs are seldom fulfilled. the getting together of teams or meeting team members personally during lunch and snack time are extensively reduced. the team members have been noted to feel isolated and left out, and also crave for belongingness. they also feel that positive energy is missing in teams while sharing concerns, and discussion of personal grievances are also reduced. such experiences by the participants lead to higher emotional exhaustion rather than physical exhaustion in gvts. the lesser experience of emotions, belongingness, and appreciation is a norm when one is part of gvts. 5.9 temporal distance according to participants, one of the biggest challenges in gvts is to understand and collaborate with people across time zones. it takes effort, patience, and persistence to coordinate the timings of all members of the team for a discussion, meeting, or a brainstorming session to be scheduled. if for any reason, the coordination fails for a particular time zone, team members would lose interest and disengage from subsequent discussions in teams. senior vice president (india): it sometimes happens that team members located outside of the headquarters, or outside of the location wherever the leader is located is, there is a tendency for teams who are in one location to just start brainstorming an idea, come up with something, some sort of a draft etc., and then go to gvts or teams outside of that headquarter or teams outside of that geography and ask them for their feedback. the teams which are (sic), they don't see anything wrong in that, we are soliciting inputs, we are making this part, but the problems with (sic) the teams outside see is that we want to be in the brainstorming meeting. even though the participants had reported having experienced challenges while working in gvt context, they have also stated that gvts give them autonomy of work, reduce distraction and allow higher immersion and dedication towards the job, and contribute towards advancing their career. a design engineer (india) shared an experience which involved high levels of commitment and loyalty towards the gvt she was working with, and another senior vice president (india) shared his view that in gvts, “the team members are loyal towards the leader or team more than the organisation due to lesser impact of organisational culture in gvts.” australasian journal of information systems shaik & makhecha 2019, vol 23, research on the role of tech in workforce mgt employee engagement in virtual teams 15 6 the conceptual model we use emic descriptions and concerns expressed by the participants to derive etic categorisations of drivers of employee engagement using the job demands-resources theory of employee engagement. the jd-r (bakker & demerouti, 2014, 2017, 2018) suggests that all job characteristics can be classified as either job demands or job resources. job demands are those aspects of work which cost energy of the employee; and job resources are those aspects of work which enable employees to deal with the job demands and achieve their goals (bakker & demerouti, 2018). job demands also are known to have an independent effect on employee well-being and, in contrast, job resources are known to initiate a motivational effect (scaufeli & bakker, 2004). job resources can also buffer the impact of job demands on the negative strain. even though each of them have an independent impact, they also work in concert. existing research indicates that employees actively interpret and influence their working conditions (bakker & demerouti, 2018). based on employee's occupational health, they can influence their job designs either into a loss cycle or a gain cycle. when employees are stressed, they influence their work environment negatively and are susceptible to lose even more resources leading to a loss cycle; and when motivated and engaged, they influence their environment in a positive way leading to gain cycles (bakker & demerouti, 2018). the job demands are increased with high levels of job strain (chronic exhaustion), poor communication, mistakes in structure, and conflicts among team members, and these behaviours have been termed as ‘self-undermining' as they lead to reduced performance (bakker & demerouti, 2018). the employees who experienced increased levels of job strain are noted to be engaged in self-undermining behaviours and report higher levels of emotional exhaustion, demands and lower performance (bakker & wang, 2016). those employees with positive job resources such as craftsmanship, pride in the profession, and positive feedback of their work have been said to positively influence work engagement which, in turn, positively impact future job resources (hakanen, perhoniemi, & toppinen-tanner, 2008). in case of gvts, academic literature, practitioner literature, and phenomenological accounts in this study identify the challenges which can be experienced as job demands by team members in comparison to collocated teams such as temporal and spatial dispersion, reduced interpersonal relationships, probability of higher conflicts, and reduced ftf interactions. the emic interpretations of this study also underline the additional challenges of a technological medium, need for higher trust and maturity levels of individuals involved, and the importance of cultural intelligence among team members. the emic interpretations are evident to the challenges faced by gvt members and the available job resources for similar job demands as a collocated team. this comparison may lead to an increase in loss cycles of team members compared to gain cycles. the mismatch in the form of higher job demands and limited job resources in the context of gvts necessitates a detailed understanding of employee engagement in gvts. our findings indicate that the challenges of employee engagement according to prior scholarly literature in the context of the collocated team is significantly different in comparison to challenges experienced by the participants in this study. we argue that the context of gvts is unique-it is virtual, technology-driven and multicultural and, hence, these contextual features could be perceived as job demands to be additionally met by members of gvts. if perceived as job demands these specific factors could negatively influence the employee's cognitive, emotional and behavioural contribution towards australasian journal of information systems shaik & makhecha 2019, vol 23, research on the role of tech in workforce mgt employee engagement in virtual teams 16 organisational goals. thus, it becomes prudent for organisations to understand and invest in creating job resources that would help gvt members to overcome the job demands and consequently increase employee engagement. the emic interpretations derived based on this study are mapped with the etic categorisations as shown in table 2. figure 2 is the derived conceptual framework based on these emic and etic interpretations which are discussed in the subsequent section. figure 2: drivers of employee engagement in gvts – etic interpretation. (framework: by authors) s.no emic interpretations etic categorizations 1 focus on goal clarity communication (formal) 2 conflicts communication (informal) 3 interpersonal relationships communication (informal) 4 informal communications communication (informal) 5 cultural diversity cultural intelligence 6 team member trust & maturity trust & maturity (separated) 7 technology medium technology 8 coordination issues technology 9 temporal distance technology table 2: mapping of emic interpretations into etic themes 6.1 cultural intelligence the findings of this phenomenological account reflect the need for cultural sensitivity or cultural awareness in gvts and, in our opinion, the cultural intelligence/ cultural quotient (cq) construct captures individual sensitivity towards the different cultures. as per ang et al. (2007), cq emphasises metacognitive, cognitive, motivational, and behavioural facets of an employee to adapt and function in a multicultural environment. thus, higher cultural intelligence of members in gvt act as a resource to overcome the barriers created due to cultural diversity as they would be more sensitive towards cultural differences. this has the potential to enhance emotional and behavioural contribution to the gvts, through higher engagement. in a traditional team setup, employees have sufficient physical interactions with their professional counterparts, which might lead to distinct and clearer differentiation between australasian journal of information systems shaik & makhecha 2019, vol 23, research on the role of tech in workforce mgt employee engagement in virtual teams 17 their work and non-work identities (national, gender, family). as the context of work changes in a gvt, and with blurring distinctions between work and non-work life identities, employees in such occupations tend to negotiate between work and non-work identities. in case of gvts, where team members belong to varied geographical locations, with a diverse set of cultural background, it is highly likely that due to lesser physical interactions and lesser rich mediums of communication, employees could be more aligned towards their non-work identities than the work identities (ramarajan & reid, 2013). in a gvt, it is possible that the impact of the cultural background of an employee is more pronounced and national culture prompts employees to form sub-groups within a team, as a process of self-categorisation, and, thus, social identification. individual well-being and engagement; occupational productivity and efficiency; and individual empowerment and autonomy have been mentioned as three direct consequences of alignment of work and non-work identities of an employee at the workplace (ramarajan & reid, 2013). due to distorted alignment, the employees in a gvt might not be able to express themselves cognitively, emotionally or physically as there is a lack of cultural understanding and confused identity. we propose that when team members express higher cultural intelligence, the alignment of non-work and work identities could create better understanding, fewer conflicts, and improved interpersonal relationships, (ang, van dye, koh, & ng, 2007). this has also been captured in the phenomenological accounts of this study. these could be perceived as job resources by members of gvts leading to higher levels of engagement. therefore, we propose that: proposition 1: cultural intelligence of team members positively influences employee engagement in gvts. 6.2 communication from excerpts in the emic descriptions, it is evident that in gvts, structured, frequent and formal communication on expectations and deliverables helps to build goal clarity among gvt members which could be perceived as a valuable job resource by employees. on the other hand, informal communication is less prevalent and, therefore, to build cultural awareness, bonding, and faith, it is very essential for a team to also have communication which is not work-related. this could help culturally diverse members of gvts to understand each other better. informal communication is something that happens spontaneously (kraut, fish, root, & chalfonte, 1990). and this could include communication which is unscheduled, interactive, spontaneous, optional, participant organised, and experience focused (flepp et al., 2017; hrastinski, 2010). earlier research has found that informal communication in a team is very essential for its successful collaboration (fish, kraut, root, & chalfonte, 2002). in gvts, accomplishment of team tasks inadvertently involves collaboration. thus, informal communication gains prominence. this study emphasises that informal communication helps the members’ understanding of their teamwork and learning about each other. and, by doing this, due to the underlying social relations, it gives critical facility and resources for engagement and for collaborators to jointly work, maintain and also drive to conclusions (fish et al., 2002). another added advantage for informal communication is that it helps to understand and empathise with time zone differences and different expectations of diverse team members. it helps in understanding other members’ challenges, and to structure the formal communication and scheduling when working in a virtual environment in such a way that participants converge with time zones differences and are flexible in their approach. therefore, appropriate communication, both formal and informal, act as resources to help australasian journal of information systems shaik & makhecha 2019, vol 23, research on the role of tech in workforce mgt employee engagement in virtual teams 18 overcome the challenges in gvts, and negate the demands of the job for gvt members to be cognitively, emotionally and physically immersed in their work. hence, it is proposed that: proposition 2: formal and informal communication among the team members positively influences employee engagement in gvts. 6.3 individual maturity our analysis in previous section indicate the advantage of having mature team members in gvts. according to argyris (1957), “as a person matures, changes occur in several continua which include passive to active, dependence to independence, behave in few ways to capable of behaving in many ways, erratic shallow interest to deep strong interest, short-term perspective to long-term perspective, subordinate position to equal (or) superordinate position, and, lack of awareness of self to awareness and control of self” (as cited in eldridge, 1991). such maturity on part of members of gvts would serve several benefits. further, for a gvt to work in coordination and alignment, it is necessary to have all team members as a single cohesive unit which allows them to use their skills and capabilities and motivate them towards concerted actions. team technical competency, team motivation, and virtual teamwork skills have been identified as dimensions of team maturity in gvts (zhang, tremaine, fjermestad, milewski, & o'sullivan 2006). in the analysis of our phenomenological accounts, we found that managers in gvts would prefer an individual who either has experience of working in a gvt or an individual with very high commitment levels. by the term ‘mature’, they meant that team members have high cognition towards their jobs. according to shuck (2010), employee engagement is “cognitive, emotional and behavioural”. we argue that the maturity of team members in gvts helps in team functioning, influencing the cognitive and emotional aspect of employee engagement. thus, we propose: proposition 3: maturity of team members positively influences employee engagement in gvts. 6.4 trust trust, openness, and honesty are seen as critical factors by team managers and also by team members when they are part of gvts. however, in gvts it may take time to build trust due to geographical dispersion, limited informal communication, diverse cultural background, language barriers and so on. most participants in the interviews pointed that at least one ftf conversation, or a video conference where the team has visual images of people they are talking to, will be helpful in building a relationship and extending collaboration in gvts. thus, building trust is one of the critical challenges faced in gvts (chatfield, shlemoon, redublado, & darbyshire, 2014). trust as a topic of research has been widely studied in the context of gvts (bell & kozlowski, 2002a; jarvenpaa & leidner, 1998; krebs, hobman & bordia, 2006). as discussed earlier, informal communication can be one of the means to build trust in gvts. in addition, structured and clear communication of goals allows the teams to be in alignment with their broader objective. trust enables alignment of common goals and reduces conflicts which can be highly prominent in gvts (panteli & sockalingam, 2005). the reduction of conflicts in gvts can lead to improved performance as team members share information across among team members which can be beneficial (panteli & sockalingam, 2005). the employees can thus focus more, both cognitively and emotionally, on their job rather than be in conflict with other team members. in view of the above, we propose that, proposition 4: trust among team members positively influences employee engagement in gvts. australasian journal of information systems shaik & makhecha 2019, vol 23, research on the role of tech in workforce mgt employee engagement in virtual teams 19 6.5 technology the very existence of gvts is based on the presence of technology media connecting team members across geography. technology plays a vital role in many forms such as text, email, instant messaging (im), video conferencing, webex, phone calling and so on (chatfield, shlemoon, redublado, & darbyshire, 2014). the selection of technology depends on the type of task. more complex tasks require more coordinated and interactive effort. team members’ roles become highly symbiotic and the need for deeply orchestrated teamwork, reciprocal communication and feedback are essential (bell & kozlowski, 2002). in complex tasks, information richness becomes critical and necessary due to collaboration demands (hollingshead, mcgrath, & o’connor, 1993). straus and mcgrath (1994) found that there was no difference between fff and technology-mediated groups in lower complexity tasks. however, ftf teams performed significantly better than their technology-mediated counterparts in more complex and judgmental tasks. bell and kozlowski (2002) opined that synchronous forms of communication are superior to asynchronous modes for complex tasks which require collaborating and a lot of information sharing. they suggest that these technologies in synchronous form maintain a higher degree of information richness and allow interactive decision-making mechanisms, leveraging collaboration and coordination. media effects are generally a combination of the technical capabilities of the communication media, the group's understanding of the medium and their usage according to the purpose (desanctis, poole, dickson, & jackson 1993). even though media effects depend on user experience as well as technical capabilities of the members, use of technology-mediated communication without ftf interaction can change the dynamics of team relationships, and misunderstandings, if any, might be hidden longer in gvts than in traditional settings (hansen, 2004). while diversity of a geographically distributed team might stimulate creativity allowing a wide variety of skills, it might also reduce cohesion in the team (kankanhalli, tan, & wei, 2007). the selection of technology, either synchronous or asynchronous, becomes important in the context of gvts. when appropriate technology is not selected there is a chance of misinterpretation of the message, and misunderstanding among the team members which can increase the risk of reduced levels of trust and increased conflicts (knight & burn, 2011; yigitbasioglu; 2014). the reflections on the phenomenological accounts also suggest that technology enables the working of gvts but the use of appropriate technology is important which enables smooth functioning of gvts by reducing the challenge of misinterpretation and misunderstanding among team members, facilitating team members to be cognitively and emotionally driven towards performance. hence, we propose: proposition 5: appropriate technology positively influences employee engagement in gvts. based on our findings, we conceptualise that employee engagement in gvts could be influenced by the cultural intelligence of team members, formal and informal communication in gvts, greater maturity of team members, higher trust levels in team members and appropriate technological support. 7 discussion and implications there has been an extensive study on employee engagement both in practitioner and academic literature, however, there is a scant literature which tries to understand employee engagement in the context of gvts. the purpose of this study was to understand the experiences of team members of gvts and then derive the drivers of employee engagement in gvts and compare australasian journal of information systems shaik & makhecha 2019, vol 23, research on the role of tech in workforce mgt employee engagement in virtual teams 20 these with traditional collocated team context. our findings and interpretive analysis at two levelsemic and eticreveal that in the context of gvts, the drivers of employee engagement take newer meaning and form. with the use of phenomenology, we make an attempt to understand the phenomenon of employee engagement through the narratives and stories of lived experiences of members of gvts. this is a seminal study (in our knowledge, it could be the first) to understand drivers of employee engagement in gvts using phenomenology. using the jd-r model, we argue that in the context of gvts, the job demands are unique and, hence, there is a need to transform some of these demands into resources or to negate the effects of few of the demands (e.g. cultural intelligence in gvt members can negate the negative influence of cultural misunderstandings). thus, by working on factors like technology, communications, trust, maturity, and cultural intelligence, gvts members’ perception of resources would increase, thus, making them increase their cognitive, emotional and behavioural contributions towards the team goals through higher levels of engagement. the importance of cultural intelligence (erez et al., 2013), technological competence (carlson, carlson, vaughn, & george, 2017; chatfield et al., 2014), individual maturity (friedrich, friedrich, & fetzer, 2017), informal communication (marlow, lacerenza, & salas, 2017) and trust among team members in gvt (alsharo, gregg, & ramirez, 2017; chatfield et al., 2014) have been studied earlier with their direct linkages to effective functioning of gvts. however, we conceptualise the relationship between the drivers and gvt success could be facilitated through the process of employee engagement. thus, we argue that for an employee to contribute towards gvt goals he/she needs to be engaged and for an employee to be engaged in the context of a gvt, the combination of these five drivers play an important role in helping members perceive the specific context of the gvt as an enabler, instead of a hindrance in performance in gvts. the results of this study demonstrate that when the context of team environment changes to being virtual, multicultural and technology-mediated, the challenges that the team members face in being engaged change. these challenges need to be addressed with adequate resources for the team members to be cognitively, emotionally and physically directing their energy towards their performance. this study involved understanding the lived experiences of employees as they engage in gvts and, thus, it adds to the academic literature by highlighting five possible factors that could drive employee engagement in gvts. on theoretical grounds, the study contributes to literature on gvts and the literature on employee engagement. this study also offers guidelines to practitioners on ways to improve employee engagement in a virtual context. as the practitioner literature, for example, aon hewitt (2013) or kpmg (2012), claim reduced levels of engagement in gvts, this study provides a direction towards improving engagement in gvts by providing appropriate technology support, and facilitating adequate informal and formal communications among team members. the gvts would gain by investing in improving the cultural intelligence of team members and providing appropriate opportunities to build trust among members. 8 conclusion and limitations this study was an attempt to understand employee engagement in gvts as recent times have witnessed a proliferous increase in globally distributed teams due to rapid growth of technology and globalisation. our interpretive analysis highlights that in order to increase engagement levels in gvts, organisations need to focus on appropriate team membership (members high in maturity and cultural intelligence) supported by appropriate technology, and adequate formal and informal communication. finally, facilitating teams to create a australasian journal of information systems shaik & makhecha 2019, vol 23, research on the role of tech in workforce mgt employee engagement in virtual teams 21 climate of trust among members might help in fostering an environment in gvts which is ultimately engaging. this study has several limitations which we propose to address in further studies. the drivers have been derived through the interpretive analysis of narratives by members of gvts while describing their experiences in gvts, and, hence, linkages with employee engagement have not been studied directly. we acknowledge the need to take this research forward by individually and collectively testing the variables conceptualised in this study through larger empirical study in multiple organisations. the subjective interpretations of the interpretations of the experiences of the participants by researchers add to the subjectivity in our conclusions and, hence, may not be infallible. we believe that an empirical testing of variables will help concretise conceptual variables with certainty, especially by showing full or partial mediation of employee engagement between the variables and effective functioning of the gvts. also, due to the use of many modes of technology for conducting interviews, the bias due to multiple media can be questioned. going forward, an empirical study will enable us to concretise the drivers of employee engagement in gvts conceptualized in this study. references alarcon, g. m. (2011). a meta-analysis of burnout with job demands, resources, and attitudes. journal of vocational behavior, 79(2), 549-562. albrecht, s. l., & anglim, j. (2018). employee engagement and emotional exhaustion of fly-infly-out workers: a diary study. australian journal of psychology, 70(1), 66-75. albrecht, s. l., bakker, a. b., gruman, j. a., macey, w. h., & saks, a. m. (2015). employee engagement, human resource management practices and competitive advantage: an integrated approach. journal of organizational effectiveness: people and performance, 2(1), 735. alnuaimi, o. a., robert, l. p., & maruping, l. m. (2010). team size, dispersion, and social loafing in technology-supported teams: a perspective on the theory of moral disengagement. journal of management information systems, 27(1), 203-230. alnuaimi, o. a., robert, l. p., & maruping, l. m. (2010). team size, dispersion, and social loafing in technology-supported teams: a perspective on the theory of moral disengagement. journal of management information systems, 27(1), 203-230. alsharo, m., gregg, d., & ramirez, r. (2017). virtual team effectiveness: the role of knowledge sharing and trust. information & management, 54(4), 479-490. ang, s., van dyne, l., koh, c., ng, k. y., templer, k. j., tay, c., & chandrasekar, n. a. (2007). cultural intelligence: its measurement and effects on cultural judgment and decision making, cultural adaptation and task performance. management and organization review, 3(3), 335-371. argyris, c. (1957) personality and organization: the conflict between system and the individual. new york: harper & row. bailey, c., madden, a., alfes, k., & fletcher, l. (2017). the meaning, antecedents and outcomes of employee engagement: a narrative synthesis. international journal of management reviews, 19(1), 31-53. australasian journal of information systems shaik & makhecha 2019, vol 23, research on the role of tech in workforce mgt employee engagement in virtual teams 22 bakker, a. b., & demerouti, e. (2007). the job demands-resources model: state of the art. journal of managerial psychology, 22(3), 309-328. bakker, a. b., & demerouti, e. (2014). job demands–resources theory. wellbeing: a complete reference guide, 1-28. bakker, a. b., & demerouti, e. (2017). job demands–resources theory: taking stock and looking forward. journal of occupational health psychology, 22(3), 273. bakker, a. b., & demerouti, e. (2018). multiple levels in job demands-resources theory: implications for employee well-being and performance. handbook of well-being. salt lake city, ut: def publishers. doi: nobascholar.com. bakker, a. b., & wang, y. (2016). self-undermining behaviour at work: evidence of construct and predictive validity. manuscript submitted for publication. bell, b. s., & kozlowski, s. w. (2002). a typology of virtual teams: implications for effective leadership. group & organization management, 27(1), 14-49. bhoola, v., & giangreco, a. (2018). hr activities and practices for project success: a multimethod approach from indian it firms. australasian journal of information systems, 22. burgess, k., kerr, d. v., & houghton, l. (2013). paradigmatic approaches used in enterprise resource planning systems research: a systematic literature review. australasian journal of information systems, 18(1). burn, j., & robins, g. (2002). a virtual organisation model for e-government. australian journal of information systems, 9(2), 104-112. carlson, j. r., carlson, d. s., hunter, e. m., vaughn, r. l., & george, j. f. (2017). virtual team effectiveness: investigating the moderating role of experience with computer-mediated communication on the impact of team cohesion and openness. in remote work and collaboration: breakthroughs in research and practice (pp. 687-706). igi global. chatfield, a., shlemoon, v. najem., redublado, w. & darbyshire, g. (2013). creating value through virtual teams: a current literature review. 24th australasian conference on information systems (pp. 1-11). australia: rmit university. christian, m. s., garza, a. s., & slaughter, j. e. (2011). work engagement: a quantitative review and test of its relations with task and contextual performance. personnel psychology, 64(1), 89-136. cole, m. s., walter, f., bedeian, a. g., & o’boyle, e. h. (2012). job burnout and employee engagement: a meta-analytic examination of construct proliferation. journal of management, 38(5), 1550-1581. connaughton, s. l., & shuffler, m. (2007). multinational and multicultural distributed teams: a review and future agenda. small group research, 38(3), 387-412. crawford, e. r., lepine, j. a., & rich, b. l. (2010). linking job demands and resources to employee engagement and burnout: a theoretical extension and meta-analytic test. journal of applied psychology, 95(5), 834. crawford, e. r., rich, b. l., buckman, b., & bergeron, j. (2013). the antecedents and drivers of employee engagement. in employee engagement in theory and practice (pp. 71-95). routledge. australasian journal of information systems shaik & makhecha 2019, vol 23, research on the role of tech in workforce mgt employee engagement in virtual teams 23 demerouti, e. (2014). design your own job through job crafting. european psychologist,19, pp. 237-247 demerouti, e., bakker, a. b., nachreiner, f., & schaufeli, w. b. (2001). the job demandsresources model of burnout. journal of applied psychology, 86(3), 499. desanctis, g., poole, m. s., dickson, g. w., & jackson, b. m. (1993). interpretive analysis of team use of group technologies. journal of organizational computing and electronic commerce, 3(1), 1-29. dulebohn, j. h., & hoch, j. e. (2017). virtual teams in organizations. human resource management review, 27(4), pp. 569-574. eldridge, w. d. (1991). the challenge of maturity: a comprehensive guide to understand and achieve psychological and social self-actualization as we grow older. university press of america. erez, m., lisak, a., harush, r., glikson, e., nouri, r., & shokef, e. (2013). going global: developing management students' cultural intelligence and global identity in culturally diverse virtual teams. academy of management learning & education, 12(3), 330-355. fish, r. s., kraut, r. e., root, r. w., & rice, r. e. (1993). video as a technology for informal communication. communications of the acm, 36(1), 48-61. flepp, c., imhof, m., meier, g., ryser, t., burkhard, r., schulze, h., & simon, a. (2017). designing rooms for virtual, informal communication: reciprocal awareness as a central criterion. in advances in ergonomic design of systems, products and processes (pp. 191-208). springer, berlin, heidelberg. friedrich, r., friedrich, r., & fetzer. (2017). virtual team maturity model. springer. gibson, c. b., & gibbs, j. l. (2006). unpacking the concept of virtuality?: the effects of geographic dispersion , electronic dependence , dynamic structure , and national diversity on team innovation author(s): cristina b . gibson and jennifer l . gibbs published by: sage publications , , 51(3), 451–495. gilson, l. l., maynard, m. t., jones young, n. c., vartiainen, m., & hakonen, m. (2015). virtual teams research: 10 years, 10 themes, and 10 opportunities. journal of management, 41(5), 1313-1337. grenny, j., maxfield, d. (2017, november). a study of 1100 employee found that remote workers feel shunned and left out. harvard business review, retrieved from blob:https://hbsp.harvard.edu/aece0f82-a45b-49ad-9482-0d2703e2dcf2 haas, m. r. (2006). acquiring and applying knowledge in transnational teams: the roles of cosmopolitans and locals. organization science, 17(3), 367-384. hakanen, j. j., perhoniemi, r., & toppinen-tanner, s. (2008). positive gain spirals at work: from job resources to work engagement, personal initiative and work-unit innovativeness. journal of vocational behaviour, 73(1), 78-91. hakanen, j. j., schaufeli, w. b., & ahola, k. (2008). the job demands-resources model: a three-year cross-lagged study of burnout, depression, commitment, and work engagement. work & stress, 22(3), 224-241. australasian journal of information systems shaik & makhecha 2019, vol 23, research on the role of tech in workforce mgt employee engagement in virtual teams 24 halbesleben, j. r. (2010). a meta-analysis of work engagement: relationships with burnout, demands, resources, and consequences. work engagement: a handbook of essential theory and research, 8(1), 102-117. hansen, m. (2004). virtual teams that work: creating conditions for virtual team effectiveness. personnel psychology, 57(1), 243. heidegger, m., macquarrie, j., & robinson, e. s. (1962). being and time. translated by john macquarrie & edward robinson.(first english edition.). london henry, j.e. and hartzler, m. 1998. tools for virtual teams. milwaukee, wi: asq quality press. hewitt, a. (2005). trends in employee engagement. aon hewitt corp, 18, retrieved from: http://aon.mediaroom.com/news-releases?item=62745, august 23, 2018. hoch, j. e., & kozlowski, s. w. (2014). leading virtual teams: hierarchical leadership, structural supports, and shared team leadership. journal of applied psychology, 99(3), 390. hollingshead, a. b., mcgrath, j. e., & o'connor, k. m. (1993). group task performance and communication technology: a longitudinal study of computer-mediated versus face-toface work groups. small group research, 24(3), 307-333. hrastinski, s. (2010). how do e-learners participate in synchronous online discussions? evolutionary and social psychological perspectives. in evolutionary psychology and information systems research (pp. 119-147). springer, boston, ma. husserl, e. (1962). ideas, trans. wrb gibson, new york, collier, 270. jarvenpaa, s. l., & leidner, d. e. (1998). communication and trust in global virtual teams. journal of computer-mediated communication, 3(4), jcmc346. jarvenpaa, s. l., & leidner, d. e. (1999). communication and trust in global virtual teams. organization science, 10(6), 791-815. jimenez, a., boehe, d. m., taras, v., & caprar, d. v. (2017). working across boundaries: current and future perspectives on global virtual teams. journal of international management, 23(4), 341-349. jopling, d. a. (1996). sub-phenomenology. human studies, 19(2), 153-173. kahn, w. a. (1990). psychological conditions of personal engagement and disengagement at work. academy of management journal, 33(4), 692-724. . kankanhalli, a., tan, b. c. y., & wei, k. k. (2007). managing conflict in global virtual teams. journal of management information systems, 24(3), 237-274. knight, s. , & burn, j. m. (2011). a preliminary introduction to the otam: exploring users’ perceptions of their on-going interaction with adopted technologies. australasian journal of information systems, 17(1), 5-41. kpmg (2012). rethinking human resources in a changing world. retrieved from https://home.kpmg.com/content/dam/kpmg/pdf/2016/06/pl-rethinking-humanresources-in-a-changing-world.pdf kraut, r. e., fish, r. s., root, r. w., & chalfonte, b. l. (1990). informal communication in organizations: form, function, and technology. in human reactions to technology: claremont symposium on applied social psychology (pp. 145-199). australasian journal of information systems shaik & makhecha 2019, vol 23, research on the role of tech in workforce mgt employee engagement in virtual teams 25 krebs, s. a., hobman, e. v., & bordia, p. (2006). virtual teams and group member dissimilarity: consequences for the development of trust. small group research, 37(6), 721-741. levy, s., (1981), "interpreting consumer mythology: a structural approach to consumer behavior," journal of marketing, 54 (summer), 49-61. macey, w. h., schneider, b., barbera, k. m., & young, s. a. (2009). employee engagement: tools for analysis, practice, and competitive advantage. malden, wa: wileyblackwell. macey, w. h., schneider, b., barbera, k. m., & young, s. a. (2011). employee engagement: tools for analysis, practice, and competitive advantage (vol. 31). john wiley & sons. marlow, s. l., lacerenza, c. n., & salas, e. (2017). communication in virtual teams: a conceptual framework and research agenda. human resource management review, 27(4), 575-589. maslach, c., leiter, m. p., & schaufeli, w. (2008). measuring burnout. in the oxford handbook of organizational well being. oxford maslach, c., schaufeli, w. b., & leiter, m. p. (2001). job burnout. annual review of psychology, 52(1), 397-422. matlay, h., & martin, l. m. (2009). collaborative and competitive strategies in virtual teams of e-entrepreneurs: a pan-european perspective. australasian journal of information systems, 16(1). mccracken, g., (1988), the long interview, newbury park, ca: sage. morgan, l., paucar-caceres, a., & wright, g. (2014). leading effective global virtual teams: the consequences of methods of communication. systemic practice and action research, 27(6), 607-624. mowshowitz, a. (2002). virtual organization: toward a theory of societal transformation stimulated by information technology. greenwood publishing group. o’hara-devereaux, m., r. johansen. 1994. global work: bridging distance, culture, and time. jossey-bass san francisco, ca o'leary, m. b., & cummings, j. n. (2007). the spatial, temporal, and configurational characteristics of geographic dispersion in teams. mis quarterly, 433-452. panteli, n., & sockalingam, s. (2005). trust and conflict within virtual inter-organizational alliances: a framework for facilitating knowledge sharing. decision support systems, 39(4), 599-617. peters, l., & karren, r. j. (2009). an examination of the roles of trust and functional diversity on virtual team performance ratings. group & organization management, 34(4), 479-504. pike, k. l. (1967). etic and emic standpoints for the description of behaviour. in k. l. pike, language in relation to a unified theory of the structure of human behaviour (pp. 37-72). the hague, netherlands: mouton & co. http://dx.doi.org/10.1037/14786-002 powell, a., piccoli, g., & ives, b. (2004). virtual teams: a review of current literature and directions for future research. acm sigmis database: the database for advances in information systems, 35(1), 6-36. australasian journal of information systems shaik & makhecha 2019, vol 23, research on the role of tech in workforce mgt employee engagement in virtual teams 26 raghuram, s., & wiesenfeld, b. (2004). work-nonwork conflict and job stress among virtual workers. human resource management, 43(2–3), 259–277. https://doi.org/10.1002/hrm.20019 ramarajan, l., & reid, e. (2013). shattering the myth of separate worlds: negotiating nonwork identities at work. academy of management review, 38(4), 621-644. rich, b. l., lepine, j. a., & crawford, e. r. (2010). job engagement: antecedents and effects on job performance. academy of management journal, 53(3), 617-635. saks, a. m. (2006). antecedents and consequences of employee engagement. journal of managerial psychology, 21(7), 600-619. saks, a. m., & gruman, j. a. (2014). what do we really know about employee engagement?. human resource development quarterly, 25(2), 155-182. salanova, m., & schaufeli, w. b. (2008). a cross-national study of work engagement as a mediator between job resources and proactive behaviour. the international journal of human resource management, 19(1), 116-131. schaubroeck, j., & yu, a. (2016). when does virtuality help or hinder teams? core team characteristics as contingency factors. human resource management review, 1–13. https://doi.org/10.1016/j.hrmr.2016.12.009 schaufeli, w. b., & bakker, a. b. (2004). job demands, job resources, and their relationship with burnout and engagement: a multi-sample study. journal of organizational behavior: the international journal of industrial, occupational and organizational psychology and behavior, 25(3), 293-315. schutz, a. (1967). the phenomenology of the social world. northwestern university press. sherry, j. f. (1991). postmodern alternatives: the interpretive turn in consumer research. handbook of consumer behaviour, 199, 548-591. shuck, b., & herd, a. m. (2012). employee engagement and leadership: exploring the convergence of two frameworks and implications for leadership development in hrd. human resource development review, 11(2), 156-181. shuck, b., & wollard, k. (2010). employee engagement and hrd: a seminal review of the foundations. human resource development review, 9(1), 89-110. snow, c. c., snell, s. a., davison, s. c., & hambrick, d. c. (1996). use transnational teams to globalize your company. organizational dynamics, 24(4), 50-67. stern, p. n. (1994). eroding grounded theory. critical issues in qualitative research methods, 212223 straus, s. g., & mcgrath, j. e. (1994). does the medium matter? the interaction of task type and technology on group performance and member reactions. journal of applied psychology, 79(1), 87. strauss, a., & corbin, j. m. (1990). basics of qualitative research: grounded theory procedures and techniques. sage publications, inc. australasian journal of information systems shaik & makhecha 2019, vol 23, research on the role of tech in workforce mgt employee engagement in virtual teams 27 thompson, c. j., locander, w. b., & pollio, h. r. (1989). putting consumer experience back into consumer research: the philosophy and method of existential-phenomenology. journal of consumer research, 16(2), 133-146. thompson, c. j., locander, w. b., & pollio, h. r. (1990). the lived meaning of free choice: an existential-phenomenological description of everyday consumer experiences of contemporary married women. journal of consumer research, 17(3), 346-361. thompson, c. j., pollio, h. r., & locander, w. b. (1994). the spoken and the unspoken: a hermeneutic approach to understanding the cultural viewpoints that underlie consumers' expressed meanings. journal of consumer research, 21(3), 432-452. townsend, a. m., demarie, s. m., & hendrickson, a. r. (1998). virtual teams: technology and the workplace of the future. academy of management perspectives, 12(3), 17-29. tran, m., yang, y., & raikundalia, g. (2004). consumption of multiple concurrent identities: the need from the instant messaging virtual community. australasian journal of information systems, 11(2). van den broeck, a., schreurs, b., de witte, h., vansteenkiste, m., germeys, f., & schaufeli, w. (2011). understanding workaholics' motivations: a self-determination perspective. applied psychology, 60(4), 600-621. wilson, t. d. (2002). alfred schutz, phenomenology and research methodology for information behaviour research. the new review of information behaviour research, 3(71), 1-15. yigitbasioglu, o. (2014). modelling the intention to adopt cloud computing services: a transaction cost theory perspective. australasian journal of information systems, 18(3). zhang, s., tremaine, m., fjermestad, j., milewski, a., & o'sullivan, p. (2006, october). delegation in virtual team: the moderating effects of team maturity and team distance. in global software engineering, 2006. icgse'06. international conference on (pp. 62-68). ieee. australasian journal of information systems shaik & makhecha 2019, vol 23, research on the role of tech in workforce mgt employee engagement in virtual teams 28 appendix i representative quotes (sic) underlying the emic interpretations. s.no representative quote emic theme interpretation etic theme mapping 1 so, we keep all the evidence mails and everything for clarity of work process and for future communication also. focus on goal clarity communication (formal) 2 i have been multi-tasking on different projects parallels. but when it comes to priority, i have to do one task on priority and send it to the customer or other team or other subordinate once it is done. focus on goal clarity communication (formal) 3 reporting involvement like the manager involvement will be less if it is a virtual team. focus on goal clarity communication (formal) 4 if it is planned, the things go on fine. but if it is unplanned, then there will be a problem. focus on goal clarity communication (formal) 5 where ever the team is available, we will be able to work well in such place. because, there will be no mental pressure, nothing will be there. we can able to do it as soon as possible. where the team is not available, our concentration is only on the work, so it is like, we will feel like somewhat stressed. focus on goal clarity communication (formal) 6 we properly define challenges and action plan every week. and, every week we also have a meeting with the senior management and update them about the progress. so, no progress means i am sitting at home! focus on goal clarity communication (formal) 7 as a manager, one has to ensure that the team is working on a single goal. it becomes very important while working in a virtual team. focus on goal clarity communication (formal) 8 there is conflict, but the problem in a gvt is more difficult to identify. but, conflict does exist. and when you get into conflict, personally, i have not been successful in doing it while (sic) video conference. you have to get together and sort it up. conflicts communication (informal) 9 a lot of times, when conflict gets into a gvt, that becomes unproductive for work, one has to have a face to face meeting with people in conflict and resolve it. it is very difficult to resolve it in a virtual environment. conflicts communication (informal) 10 informal communication is very less. everything we follow is formal communication. informal communication communication (informal) 11 regular team meetings and team lunch that is informal communication. informal communication communication (informal) 12 team building in a virtual team, like only on phone communication or else the manager has to visit the work office where his team is working, and or else he has to talk on phone or regular, wishes on some special days like birthdays, like we follow a general practice where some of the team members is having birthday we have celebrations. but in vt unless the person visits here or there, then only it can be like. informal communication communication (informal) australasian journal of information systems shaik & makhecha 2019, vol 23, research on the role of tech in workforce mgt employee engagement in virtual teams 29 s.no representative quote emic theme interpretation etic theme mapping 13 manager and team member communication is very important thing. a recent scenario we have implemented is, at least it is mandatory for him to meet the team weekly once. like apart from work i am saying. weekly once for team building apart from work mandatory. so that if there are any informal talks they can be discussed with the team members on team lunch, team tea or team practices or team outings. it will improve the interaction time with the manager and it will also have a friendly platform. informal communication communication (informal) 14 some people will be fine to explain everything with call, but others might not be comfortable. so, in that cases, we cannot communicate and it might not be helpful. informal communication communication (informal) 15 only the communication will be different. whenever we want to communicate with people, only that is the problem with virtual place. informal communication communication (informal) 16 interaction should be there and created by these people. you cannot sit for long hours and work without personal interaction. informal communication communication (informal) 17 when an associate wants to explain the personal reasons, then they feel that they cannot do it through video or audio calls. if i see the person, that will be definitely different, and if i am talking to the face whom i know there is a difference with whom i don’t know. informal communication communication (informal) 18 that difference has an impact on the work because it will change your mood. so when you are refreshed, you will have good productivity also. informal communication communication (informal) 19 only thing is that when we are isolated from the organization, from the family, then there will be personal sacrifices that you need to do. informal communication communication (informal) 20 i think it is important for a team to understand and also getting that additional information (through informal communication) is always helpful. informal communication communication (informal) 21 i think it is important for a team to understand and also getting that additional information (through informal communication) is always helpful. informal communication communication (informal) 22 it’s quite a different experience... it’s like six months we have been communicating this way and it’s really required. we are adapting to the situation. so, now we are almost on the same lines. so, now there is no problem with all this mails, conferences and videos. interpersonal relationships communication (informal) 23 team building in a virtual team, like only on phone communication or else the manager has to visit the work office where his team is working, and or else he has to talk on phone or regular, wishes on some special days like birthdays, like we follow a general practice where some of the team members is having birthday we have celebrations. but in vt unless the person visits here or there, then only it can be like. interpersonal relationships communication (informal) australasian journal of information systems shaik & makhecha 2019, vol 23, research on the role of tech in workforce mgt employee engagement in virtual teams 30 s.no representative quote emic theme interpretation etic theme mapping 24 but the person who is reporting them in a different location his involvement will be less unless he visits or they visit him. interpersonal relationships communication (informal) 25 some people will be fine to explain everything with call, but others might not be comfortable. so, in that cases, we cannot communicate and it might not be helpful. interpersonal relationships communication (informal) 26 we can find so many type of people at workplace, everyone has their own attitudes, and some people can be able to explain even a silly thing also without hesitating and without any domination. so, we cannot expect same thing from all people. interpersonal relationships communication (informal) 27 like in a collocated team, we have so many people to interact here. so, obviously you may have to spend some time to talk to others who ever are sitting around you. so, that is the only one thing. interpersonal relationships communication (informal) 28 then obviously normal working place only. because we can interact with other people for some time if you are not feeling well. then at least you will get some relaxation. interpersonal relationships communication (informal) 29 but, you know, compared to talking to associates and customers, there will be some difference. we don’t belong to that organization. interpersonal relationships communication (informal) 30 in my experience after working for seven years, i feel this small talk is a waste of time and somewhat like spoils your mood also to work at place. it is not always, because some people will talk very well when they see you and some people will talk in different way also. interpersonal relationships communication (informal) 31 like if you have seen the manager before, they will come to know what kind of issues they can share with the manager and cannot share with the manager. over technology you don’t know how the person might react. interpersonal relationships communication (informal) 32 our job depends on understanding. how the manager and how i get through. so when your reporting manager is face to face, this is what i say him directly that this is the required problem and if provided then it would be easy. so the direct communication in terms of time there will be a saving. interpersonal relationships communication (informal) 33 yeah, it feels a bit isolated. interpersonal relationships communication (informal) 34 you will not be able to see the organization happenings, day to day events when you are isolated. ya, that seems to be bad. but again when you are working in a project, you are a billable resource, you will see that your compensation will reach you. your salary is there. ya, so that’s there. interpersonal relationships communication (informal) australasian journal of information systems shaik & makhecha 2019, vol 23, research on the role of tech in workforce mgt employee engagement in virtual teams 31 s.no representative quote emic theme interpretation etic theme mapping 35 one is you get to know each other when you form a gvt and you are talking for the first time. of course, through formal communication, you get to know each other, some of the details. but then informal communication happens when you send an email and you agree and put a small smiley. this helps in building interpersonal relationships in a virtual environment. interpersonal relationships communication (informal) 36 if one has done their homework and understand that there are circumstances and situations that could be impacting the other person differently, then, communication with the other person helps build better interpersonal relationships with the members who are geographically dispersed. this will in a way allow to reduce the distance between the locations interpersonal relationships communication (informal) 37 there, the technical documents completely we read and their local language spanish, and we had a challenge to convert them into english. and, we have to learn spanish in order to learn some technical terminology. so, it was quite challenging task for us. cultural diversity cultural intelligence 38 we had different people working for different customers, spain, uk, but the work is similar. but the standards are different. because, for spain there are different standards for work and uk there are different standards. so, the work is similar but how you are going to get the output done the procedure is different. so i felt the platform like software that i am using for spain client is different from that of uk. cultural diversity cultural intelligence 39 so i felt like there are different ways of getting the output done for different clients and the other part is in spain also there are around 13 to 14 different standards. so, it is also challenging that to change from one task to another task, you have to go to different standards. so you have to go back to the technical documents and refresh everything before starting the work. cultural diversity cultural intelligence australasian journal of information systems shaik & makhecha 2019, vol 23, research on the role of tech in workforce mgt employee engagement in virtual teams 32 s.no representative quote emic theme interpretation etic theme mapping 40 there was one person who i worked with in germany, she was a fantastic contributor who had taken significant responsibility and did that very well. during one of the feedback sessions, when i asked her what are the things we can do better and how is she coming along, one thing that she said was, during the initial days, she was actually thinking of leaving the organisation, because she felt so disrespected in the conference calls. she said that “everybody is talking over me, and that is not something that i am used to”. and also said, if somebody is talking, you finish that conversation, you hear that person, and then you make a point, and then the next person makes a point right, she is very used to that. she had not worked in a global team before but she said that it took time for her to understand that this is the way that people talk, (sic) even though she could never understand the reason behind it. cultural diversity cultural intelligence 41 the only thing i had to do was contact switching. by contact switching what i mean is even though i am speaking english, i speak in a different way, so they understand. so this is what is contact switching. and it works well. cultural diversity cultural intelligence 42 in such case we have used one platform, cisco to get it solved, cisco like the desktop sharing option we had, so we used to call the client on the video conference, and using cisco, we used to get it solved. that platform helped us to get things done easily and in crucial times, when we had to meet some deadlines without any change. coordination issues technology 43 there will not be any problem when the teams are located in different places, the only problem comes how do you monitor the team members. because work point of view everything is being live updated on the server and the work status, everything is updated on the server so, getting the things done by people is the only thing you will be challenging, apart from the work here there will not be any problem when we are working in multiple locations also. coordination issues technology 44 we share them the task like this is the task for you, and this is the task for you, but when the person will come and get the take done, you don’t know. unless and until either we call he or he calls us or we drop us a text or he drops us a text. but, at what times he might come and leave, we don’t know. coordination issues technology 45 i will tell you the same thing if at all a customer, if we are sitting at a customer location in uk, like if we have multiple queries, we segregate everything and we will sit in front of the customer and the things become easy. so, the amount of time taken to solve the query is less when you are working in uk. coordination issues technology australasian journal of information systems shaik & makhecha 2019, vol 23, research on the role of tech in workforce mgt employee engagement in virtual teams 33 s.no representative quote emic theme interpretation etic theme mapping 46 but, when you are sitting in india, you consolidate everything and you send an email to the customer, call the customer on phone, some queries get solved and sometimes the customer says that you have to check back to his team members and get it done. so, the response time is less there. coordination issues technology 47 until the designer does not complete his task i don’t have an input to the checker. so, the designer has to close his task and only then the checker can have his input. so, if this is delayed, the other person's work will also be delayed. entirely it will affect my deadline. so, i have to closely monitor the person sitting remotely to get the design part done. coordination issues technology 48 for any change of policies we get an email communication, like regularly hr was available at floor, but if it is in remote location, unless and until the employee goes with an issue, the hr cannot respond in that. there is an issue, they will be responding for sure. so it is up to the employee to go and tell what the problem is then they will get the response if it is in the remote location. coordination issues technology 49 like, when you take the uk time example, the team sitting in uk working for some task, and the team sitting in india with similar task, the time frame is different, so there, the manager will be able to monitor only for a partial time, either on phone, email or any other communication. the other time frame after office hours it will be difficult for the manager and to contact the manager. coordination issues technology 50 when it comes with the communication, it is about the timings. that is something we cannot avoid. some things like if any activity has urgent delivery, definitely there will be some time gap. if we have sufficient time, there is no problem in communicate. if some problems we cannot explain them email, we can contact them by call or a video call. coordination issues technology 51 i was one among the two people in uk, and four people in hyderabad. so, the manager was in hyderabad itself. so whenever we need changes, then we request our employees from there only. so, they were doing on time and the manager also coordinates with them to deliver the work within the time. we could able to manage well. only the problem comes whenever there is an urgency. coordination issues technology 52 because, in a virtual place, if we have any concerns that we need to discuss with the manager, then we will be able to explain to him easily, and work well after that. when it comes to the technical matters there are so many senior people available in a normal set up. coordination issues technology australasian journal of information systems shaik & makhecha 2019, vol 23, research on the role of tech in workforce mgt employee engagement in virtual teams 34 s.no representative quote emic theme interpretation etic theme mapping 53 where ever the team is available, we will be able to work well in such place. because, there will be no mental pressure, nothing will be there. we can able to do it as soon as possible. where the team is not available, our concentration is only on the work, so it is like, we will feel like somewhat stressed. coordination issues technology 54 getting the right inputs at the right time. coordination issues technology 55 because we can ask their opinion. we can directly meet the experts available for the job. we can just do better things. coordination issues technology 56 gvts face on-site and off-site issues. i was once working from new york, and my team was in india. due to the change in time zones, we faced a lot of difficulty in getting support from each other. and when i was back in india, the process of follow-up was also difficult. coordination issues technology 57 most of the members get involved 7/24 with the team. some of my team is in far-off europe. it so happens sometimes that they forget other team members who are based out of their offices in their emails. these situations reduce coordination within the team. some team members feel left out. so, out of sight, out of mind situation often occurs in a gvt. coordination issues technology 58 we share them the task like this is the task for you, and this is the task for you, but when the person will come and get the take done, you don’t know. unless and until either we call he or he calls us or we drop us a text or he drops us a text. but, at what times he might come and leave, we don’t know. spatial distance technology 59 everything it is comfortable when everyone is located in the same location and it will be challenging while meeting the targets like crucial targets to control the people if they are at different location. spatial distance technology 60 amount of dedication when we are in a virtual team in different locations, we have to think more about the person who is sitting in the remote areas, about his task and when he can deliver his task, spatial distance technology 61 there are telecom where you need to have web-ex. technology medium technology 62 the earlier team experience when we used on software tools, which are completely out of scope, the tool errors were completely out of our scope, we have technical knowledge of how the system works, like what we have to do, we used to take that platform too, in order to do the testing, but the tool error was completely out of our scope. technology medium technology 63 in such case we have used one platform, cisco to get it solved, cisco like the desktop sharing option we had, so we used to call the client on the video conference, and using cisco, we used to get it solved. that platform helped us to get things done easily and in crucial times, when we had to meet some deadlines without any change. technology medium technology australasian journal of information systems shaik & makhecha 2019, vol 23, research on the role of tech in workforce mgt employee engagement in virtual teams 35 s.no representative quote emic theme interpretation etic theme mapping 64 if we have any queries or any challenges we regularly we have telephonic conversation. so, as of now what we are following is telephonic conversation. and for any clarification we are having rfi through email and technical queries, technology medium technology 65 so, we keep all the evidence mails and everything for that, and for future communication also. technology medium technology 66 when the team is located in one location, coming to the bottom level, like my team members, getting work done it is easy. so, i directly report to my manager on the status or else any updates through email. mostly i communicate with him over email. so, i don’t feel any challenge, due to remote location or at my location. technology medium technology 67 like if you have seen the manager before, they will come to know what kind of issues they can share with the manager and cannot share with the manager. over technology you don’t know how the person might react. technology medium technology 68 it’s quite a different experience... it’s like six months we have been communicating this way and it’s really required. we are adapting to the situation. so, now we are almost on the same lines. so, now there is no problem with all this mails, conferences and videos. technology medium technology 69 so whenever a need arises, we will exchange our interaction through mails, webex, phones as per the requirement. technology medium technology 70 technology is an integral part of even having a gvt. if you are not connected on a very regular basis, gvts cannot work. technology medium technology 71 i think a lot of this has to depend on what i call the virtual infrastructure. it is very important in a gvt that you have the proper tools, means to connect to the company, means to extract the data and put it back, and means to talk to different people in your team. technology medium technology 72 technology has also enabled us to reduce the commuting through flights, back and forth etc. technology medium technology 73 for us, and in uk, you might be knowing, they work until late evening. for us at morning they come at 12:00 or 1:00. so, we expect more mails in the second half. and it also continues in the evening hours. temporal distance technology 74 but, when you are sitting in india, you consolidate everything and you send an email to the customer, call the customer on phone, some queries get solved and sometimes the customer says that you have to check back to his team members and get it done. so, the response time is less there. temporal distance technology australasian journal of information systems shaik & makhecha 2019, vol 23, research on the role of tech in workforce mgt employee engagement in virtual teams 36 s.no representative quote emic theme interpretation etic theme mapping 75 like, when you take the uk time example, the team sitting in uk working for some task, and the team sitting in india with similar task, the time frame is different, so there, the manager will be able to monitor only for a partial time, either on phone, email or any other communication. the other time frame after office hours it will be difficult for the manager and to contact the manager. temporal distance technology 76 when it comes with the communication, it is about the timings. that is something we cannot avoid. some things like if any activity has urgent delivery, definitely there will be some time gap. if we have sufficient time, there is no problem in communicate. if some problems we cannot explain them email, we can contact them by call or a video call. temporal distance technology 77 but where the deliveries are a project need an immediate output and the schedule is so tight, so in that instances we can go with direct meetings or putting a report. it depends on the type of work you are in. temporal distance technology 78 it sometimes happens that team members located outside of the headquarters, or outside of the location wherever the leader is located is, there is a tendency for teams who are in one location to just start brainstorming an idea, come up with something, some sort of a draft etc., and then go to a gvt or teams outside of that headquarter or teams outside of that geography and ask them for their feedback. the teams which are (sic), they don't see anything wrong in that, we are soliciting inputs, we are making this part, but the problems with (sic) the teams outside see is that we want to be in the brainstorming meeting. temporal distance technology 79 dedication depends on individual responsibility. if he feels responsible and thinks that he is the one to get the task done, so you can get it done in limited time frame or you can also take extra amount of time. but, when he is at a different location and not taking ownership, it is always difficult to trace him even though they are calling him and he has any technical competency in that particular task which is lesser where we might have to mentor him, and give some support, then it will be a difficult task. team member maturity trust & maturity (separated) 80 because the amount of experience which he (experienced employee) has, is already high and he has to be ready to accept any kind of task for any kind of customer. his experience speaks, not anything. team member maturity trust & maturity (separated) 81 i don’t think so, because every individual as an employee has to take the ownership for his task and he has to definitely come back when the task is done to get the second task. so, ownership is the first quality where every individual has to own in a company. team member maturity trust & maturity (separated) australasian journal of information systems shaik & makhecha 2019, vol 23, research on the role of tech in workforce mgt employee engagement in virtual teams 37 s.no representative quote emic theme interpretation etic theme mapping 82 the basic thing that differs is that in a normal team, employee as i told you feels less responsible because there is some person who is monitoring him. but in a virtual team, there has to be completely full fledge 100% responsible for the task which is in hand, because it is being less monitored. team member maturity trust & maturity (separated) 83 reporting involvement like the manager involvement will be less if it is a virtual team. team member maturity trust & maturity (separated) 84 i was in uk last year and at that time, myself and one of my colleague, both was in uk. my manager was in hyderabad office. that time we came to know how to deal with the client and how to talk to them in case of official way, or whatever we need. team member maturity trust & maturity (separated) 85 in a virtual team, you need not waste much time talking to other people. it is a human tendency if we have surroundings, sometimes we try to talk to them, not always. if you are in virtual place, you don’t have chance to speak to so many people. so, obviously you can do your work faster. team member maturity trust & maturity (separated) 86 suppose if you take me and the company is not monitoring me, i will do my job and what are all the necessary things to complete the project. some people, unless manager comes to them and tells them, they won’t react to the work or timings and all. team member maturity trust & maturity (separated) 87 some people who are like they want to come late to office and go early from office, those type of people cannot be kept in a virtual team. team member maturity trust & maturity (separated) 88 it’s like you feel very determined. it’s kind of job responsibility. it’s like given my consent to the organization but i will be doing these roles whenever necessary. so i feel like it’s my job responsibility. team member maturity trust & maturity (separated) 89 you will not be able to see the organization happenings, day to day events when you are isolated. ya, that seems to be bad. but again when you are working in a project, you are a billable resource, you will see that your compensation will reach you. your salary is there. ya, so that’s there. team member maturity trust & maturity (separated) 90 they should have the skills, the fitting in, writing emails, basic corporate skill, then the people doing it, it should be easy for them to interact with such kind of people. team member maturity trust & maturity (separated) 91 mentoring will give him confidence. he should be under guidance. team member maturity trust & maturity (separated) 92 because they are away from the team, they will come to know what are all the things they have to do on their own. so, if they want to do something, no one will be there to guide them and they will face the situation on how to do it. and they will come to know on their own. team member trust and maturity trust & maturity (separated) 93 the relationships are built only on trust. everyone wants to be a part of a winning story. due to spatial dispersion, a lot of communication can go wrong due to lack of trust. team member trust and maturity trust & maturity (separated) australasian journal of information systems shaik & makhecha 2019, vol 23, research on the role of tech in workforce mgt employee engagement in virtual teams 38 s.no representative quote emic theme interpretation etic theme mapping 94 there are some people who will find it difficult to work in a gvt because here at the end of the day you are not answerable to your manager, you are answerable to the goals, and you are answerable to the outputs that you have to come up with. team member trust and maturity trust & maturity (separated) 95 then obviously normal working place only. because we can interact with other people for some time if you are not feeling well. then at least you will get some relaxation. trust trust & maturity (separated) 96 but, you know, compared to talking to associates and customers, there will be some difference. we don’t belong to that organization. trust trust & maturity (separated) australasian journal of information systems shaik & makhecha 2019, vol 23, research on the role of tech in workforce mgt employee engagement in virtual teams 39 appendix ii data structure – first order dimensions, second order themes, emic and etic theme mapping s. no. first order dimensions second order theme emic theme etic theme 1 personality, work requirement goals, how they operate, style of working is diff for diff team members understanding personality improved trust by understanding personality t ru st an d m atu rity 2 sensitivity/sentiment body language, how serious the customers are, have to judge through a con call 3 spend time and more effort to understand personality 4 understanding individual personality 5 use different tactics with different people depending on the feasible context 6 make adjustment comfortable to the other person 7 handling differing perceptions perception management and power relations managing perceptions to improve trust 8 gvts-perceptions management 9 don’t know how the other person will react 10 how sender sends the message and receiver perceives it 11 different means are comfortable for different people 12 power relations are underplayed in gvts 13 expert power and distributed leadership 14 need a mature team, self-managed, people should take initiatives to deliver team maturity team member maturity 15 need more mature people who take responsibility 16 high level of motivation to instinctively take ownership australasian journal of information systems shaik & makhecha 2019, vol 23, research on the role of tech in workforce mgt employee engagement in virtual teams 40 s. no. first order dimensions second order theme emic theme etic theme 17 gvts skills needed are adaptation, email, communication made easy team members skills needed and challenges for communication communication skills c o m m u n icatio n (f o rm al & in fo rm al) 18 writing mail will give limited and restricted communication, 19 informal communication opportunities are less 20 gvts team members interact limited time 21 expression of emotions and feelings not possible in gvts 22 informal communication difficult in gvts 23 human interactions are reduced 24 difference between explanation and understanding 25 informal communication raise questions outside work, updates, informal chats 26 gvts -not many people for suggestions, few people around to ask 27 the need to socialize does not get fulfilled. social needs not getting fulfilled by working remotely. unfulfilled social needs interpersonal relationships 28 less social get-togethers, not refreshed and not set to right mood 29 gvts feel isolated, miss out on local events 30 gvts social needs do not get satisfied 31 need for belongingness not satisfied 32 people who need face time and need to talk to their managers face difficulty 33 informal meets like lunch, discussion, going out etc. missing 34 no people around you to share your problems 35 gvtspositive energy from team members is missing 36 gvtscant share if you are not well, collocated team you can 37 gvts personal issues cannot be discussed 38 more emotional exhaustion than physical 39 gvts less experience in emotion, less appreciation of human experiences, 40 gvts no one to chase you or help you implications for managers 41 all forms of communication important in gvts 42 extend feelings to care about family, social media, birthday, small things for bonding 43 gvts skype, vc, whatsapp, non-visual communication 44 interaction deliberation 45 contribution to a purpose, then collaborate, communicate and support 46 sense of appreciation is less, and feel uncomfortable even when performing well 47 manage conflict in gvts conflicts conflict management 48 resolving conflicts 49 conflict management is high 50 conflict not so visible, hidden and be brewing inside australasian journal of information systems shaik & makhecha 2019, vol 23, research on the role of tech in workforce mgt employee engagement in virtual teams 41 s. no. first order dimensions second order theme emic theme etic theme 51 tasks specified disjunctive and conjunctive tasks task structure/goal clarity focus on goal clarity c o m m u n icatio n (f o rm a l & in fo rm al 52 conjunctive need to build on opinion/inputs of other members on projects 53 gvts clear kpis objectives and goals 54 need for clear, crisp and adjectives 55 conscious efforts, structured way of putting it to all stakeholders 56 structured, weekly biweekly call, updates and dedicated time 57 gvts meetings-structured pointed and formal 58 use regular meetings, frequency 59 lot of groundwork before meeting 60 gvts-employees not answerable to managers but to goals and objectives 61 set expectations right, set frequency meeting 62 role of manager to set clear goals and aims 63 how do team members get to work and how they achieve it 64 clear with goal setting/expectations/parameters/guidelines 65 setting expectations right 66 clean and correct messaging for each stakeholder 67 timely work reduces challenges 68 setting expectations right 69 make team members accountable 70 set objective and make people understand it, check output 71 once, twice in a month meeting will be with all team members, less frequent contact 72 discipline and be responsive and prompt 73 take responsibility of your decision australasian journal of information systems shaik & makhecha 2019, vol 23, research on the role of tech in workforce mgt employee engagement in virtual teams 42 s. no. first order dimensions second order theme emic theme etic theme 74 technology as a tool to collaborate and progress, enabler, stay connected technology as enabler coordination issues, temporal and spatial distance t ech n o lo g y 75 technology enable gvts online chats, discussions etc. 76 technology is enabler. technology selection plays role. 77 gvts-thrive on technology and flexibility 78 enjoy technology connections 79 millennials comfortable with technology, non-visual and non-verbal communication 80 use of technology 81 new technology advancement 82 gvts possible to hide info technology as deterrent 83 when network is down, it is a problem 84 delay in addressing challenges 85 can’t take up unplanned actions in vt 86 technology weak network in certain areas, big size file problem 87 gvts communication through technology, hide behind the email/text messages 88 in isolation disjunctive no inputs, lag/delay, not so much now due to advanced technology 89 most interaction on vc, skype, phone and less frequent visit virtual infrastructure 90 means of communication email, calls, webex 91 virtual infrastructure important, tools to connect, extract and tell different people in the team 92 depending on the type of task, technology is used. 93 gvts-interpersonal mechanism is email 94 tech advancement positive for gvts, reduces cost, vc, skype etc. increase interaction 95 different medium 96 gvtsmost connection through email 97 gvts-need extra time and efforts to connect, bond and understand efforts towards connecting at personal level coordination issues 98 building relations through technology 99 gvts can’t see body language so use skype and vc 100 on phone talk and open up for things you can’t write on email. 101 gvts technology bad link, lag in responding, language misinterpretation 102 engagement is high if they are happy working over technology 103 gvts connection becomes a priority and responsibility 104 use technology that the other person is comfortable with 105 set clear expectations and tasks, use technology to share info and empower people 106 customize + localize (tech, people process) 107 establishing connect through emails 108 millennials comfortable with emotions in technology but not bosses australasian journal of information systems shaik & makhecha 2019, vol 23, research on the role of tech in workforce mgt employee engagement in virtual teams 43 s. no. first order dimensions second order theme emic theme etic theme 109 no time lag, round the clock connection irrespective of time zone advantage of time zone differences temporal/spatial distance t ech n o lo g y 110 time zone diff incorporate flexibility 111 time gap to get issues solved 112 same time zone, not much frustration (0-3 hours) can initiate calls 113 time zone flexibility 114 vt coordination challenges coordination according to the time zone 115 set fixed meeting times 116 coordination issue to find appropriate time 117 more time on collaborating to set meeting 118 time zone differences, managing meetings 119 more challenging when you have to meet deadlines time zone matters challenges due to time zone 120 gvts-process affected by time zones 121 high time zone difference, more frustration, need more adjustment 122 time zone differences 123 information and use of emoticons, skype chat, sensitive to time zones, lunch time 124 can’t respond quickly to issues response time due to time zone 125 time zone challenges some decisions cannot wait for sleeping people, but they have valuable inputs for decisions 126 problem of getting right input at right time 127 time differences, long time taking, quick feedback not possible, allow delay or not take input of some people 128 time zone time gap is the bridge, issues in coordination 129 response to email introducing people, syncing calendars, time, difficult to involve all in brainstorming australasian journal of information systems shaik & makhecha 2019, vol 23, research on the role of tech in workforce mgt employee engagement in virtual teams 44 s. no. first order dimensions second order theme emic theme etic theme 130 driving change is challenging in cross cultural env cross cultural environment cultural diversity c u ltu ral in tellig en ce 131 train and plant people in different geographies 132 change in mindset 133 uniqueness and uniformity of org culture spreads across locations 134 culture sensitivity culture sensitivity 135 understanding diversity of culture 136 be sensitive to culture, local festivals, diversity 137 gvts-understanding cultural nuances help 138 culture-be sympathetic 139 diff cultures have diff motivation factors 140 understand diversity at deeper level to respect other culture, style and ethics. 141 cultures need to understand and adjust 142 cultural aspect important 143 language some cultures are very aggressive, some are comfortable etiquette, language, and preparedness across cultures 144 email etiquette is very important, interpretation of email depends on mood of reader 145 communication details, expectations and outcomes for all members across cultures 146 being careful with words while responding to emails 147 culture, contact switching, speak english in different way 148 training on cultural difference avoid misinterpretation of expectations 149 preparedness can reduce cultural difference preparedness about culture 150 do proper homework 151 consciousness will bring in adaptability, culture sensitivity and humility 152 understanding culture copyright: © 2019 shaik & makhecha. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems shaik & makhecha 2019, vol 23, research on the role of tech in workforce mgt employee engagement in virtual teams 45 paper title (paper title style) australasian journal of information systems perdana, robb, rohde & birt 2018, vol 22, research article sbr adoption in australia 1 standard business reporting (sbr) adoption in australia, critically acclaimed, box office flop: constructivist and ecological rationalities in information systems (is) adoption arif perdana singapore institute of technology arif.perdana@singaporetech.edu.sg alastair robb uq business school, university of queensland fiona rohde uq business school, university of queensland tc beirne school of law jacqueline birt uwa business school, university of western australia abstract is enables organizations to improve their productivity, streamline their business processes, and better understand the challenges and opportunities facing their business. these benefits can further accrue to individuals and organizations when they adopt and use the systems. while the benefits of is are multifarious, is adoption remains challenging. the far-reaching consequences of is motivate research examining the antecedents of successful is adoption both at individual and organizational levels. to examine the complexity of is adoption, we undertook an interpretive case study of sbr adoption in australia. we contend that sbr’s context in australia offers distinctive perspectives on the complexity of is adoption. we found that is adoption decisions can be based on both constructivist and ecological rationalities. our findings can provide insight in improving understanding of the benefits of sbr and have implications for companies, regulators, standard setters, and the accounting profession, more generally. 1 introduction is enables organizations to improve their productivity, streamline their business processes, and better understand the challenges and opportunities for their businesses. these benefits can further accrue to individuals and organizations when they adopt and use the systems (hirschheim, 2007). while the benefits of is are multifarious, is adoption remains challenging. the challenges emerging from managing the factors effecting is adoption are non-trivial. at the organizational level, the factors contributing to is adoption can be explained from two perspectives: rational assessments (i.e., economic rationality) (fichman, 2004) and institutionalism (teo, wei, & benbasat, 2003; mignerat & rivard, 2009; nielsen, mathiassen & newell, 2014). while both perspectives have been canvassed in the is adoption literature, when explaining is adoption at the organizational level potential gaps remain. in particular, australasian journal of information systems perdana, robb, rohde & birt 2018, vol 22, research article sbr adoption in australia 2 prior research largely applies institutionalism in the context of structural isomorphism (i.e., mimetic, normative, and coercive). two intriguing questions remain. first, why does structural isomorphism fail to fully explain organizational adoption decisions even though structural isomorphism exists (compagni, mele, & ravasi, 2015; troshani, parker & lymer, 2015)? second, why do rational assessments also fail to fully explain organizational adoption decisions even though the adoption decisions are economically beneficial (kaganer, pawlowski & wiley-patton, 2010; swanson & ramiller, 1997)? early adopters primarily rationalize their decisions based on the rational assessments, for example the technical, cost, and benefit assessments of the technology (fichman, 2004). this rationality, however, does not sufficiently explain adoption decisions because decisions may not be completely deterministic. structural isomorphism often complicates rationality at the organizational level, that is, the prevailing complex institutional and social influences (thornton, ocasio, & lounsbury, 2012). structural isomorphism has emerged as a complement to the rational assessments perspective. structural isomorphism itself, however, is problematic because it focuses on macro analysis at the organizational level, and thus tends to overlook the roles and actions of individuals (thornton et al. 2012; compagni et al. 2015). we argue that organizations neither simply follow structural isomorphism nor rational assessments instead the two concepts are interacting. for example, organizational is adoption may be affected by mimetic, normative or coercive isomorphism. at the individual level, however, individuals’ behaviour (e.g., powerful actors and influencers) internally and externally influence organizations’ decisions (swanson & ramiller 1997). individuals interactively connect and rationalize their day-to-day actions with organizations. organizations may also pursue adaptive behaviour to satisfice themselves rather than seeking to maximize their utility. our work is motivated by seeking to address the deficiencies in explaining organizational rationality in is adoption decisions. we use two conceptualizations of rationality to guide our study, namely, constructivist rationality and ecological rationality. both conceptualizations originate in the economics domain (smith, 2008). constructivist rationality aligns with the notion of the economic rationalistic approach as the dominant paradigm in is adoption (fichman, 2004), while ecological rationality is largely similar with the concept of institutional logics (nickerson & muehlen, 2006; smith, 2008; thornton et al. 2012). to examine the complexity of is adoption, we undertook an interpretive case study of sbr adoption in australia. the sbr context is relevant and provides scope for different conceptualizations, whereby, the context can neither be simply explained individually by structural isomorphism and/nor rational assessments perspectives. sbr should be evaluated holistically involving both constructivist rationality and ecological rationality. we contend that an investigation in the sbr context can complement findings from prior research and substantially increase our understanding of is adoption. when compared to other information systems (is) adoption cases with limited stakeholder influences, the complexity of sbr’s context in australia offers distinctive perspectives on the complexity of is adoption. sbr was introduced to reduce the costs associated with regulatory and business reporting via the use of extensible business reporting language (xbrl)-enabled reporting systems. the involvement of multiple stakeholders such as government and regulatory bodies, accounting professional bodies, vendors, businesses, and information users, in concert with its multi-use context, contribute to the complexity of sbr adoption in australia. given the efficiencies sbr can and does offer, its adoption as the preferred means to submit businesses’ financial reports australasian journal of information systems perdana, robb, rohde & birt 2018, vol 22, research article sbr adoption in australia 3 appears to make good economic sense, why then has its uptake been so limited? are australian companies “radical non-adopters” or “rational non-adopters”? the findings from this study yield theoretical and practical implications. first, we argue that is adoption decisions can be based on both constructivist rationality and ecological rationality. second, by considering both rationalities, we hope to shed light on factors that facilitate and hinder actual implementation of is. third, relevant to our study context, the findings provide timely feedback to regulators, preparers, and users of financial reports on reasons why widespread adoption of financial reporting via sbr has not occurred in australia despite claims of much improved regulatory reporting efficiency (nagy, jubb, rouse & cybulski, 2008; azam & taylor, 2011; 2016). fourth, the findings also provide some thoughts on improving understanding of the benefits of sbr adoption and, hence, its adoption rates. fifth, the findings from this study have implications for companies, regulators, standard setters, and the accounting profession, more generally. this paper proceeds as follows. section 2 explains the conceptual background which includes a review of extant research relevant to is adoption. section 3 contains our research context, data, and research method. section 4 presents our research findings. section 5 discusses our research findings. section 6 presents our research contributions and limitations. section 7 concludes the study. 2 background 2.1 two dominant paradigms in information systems adoption – economic rationalistic and institutionalism responding to the challenges of is adoption, is research has examined the relevant antecedents of is adoption and its applications. multiple theories and frameworks, many of which have been adapted from the psychology, sociology, economics, and management fields have been applied in is research to better understand is adoption and applications. two dominant paradigms emerged in studying is adoption and use, namely, economicrationalistic (fichman, 2004) and institutionalism (teo et al. 2003; mignerat & rivard, 2009; nielsen et al. 2014). the economic-rationalistic approach emerged from fichman’s (2004) explanation of the dominant paradigm in is adoption, whereas institutionalism evolved from the sociology field. economic-rationality suggests that the greater the resources organizations have, the greater the quantity of innovation they should generate (fichman, 2004). the underlying assumption of this paradigm is the amount of organizational resources is linearly correlated to the economic returns. when explaining is adoption, however, this assumption overlooks the complex multiple internal and external interactions to which organizations are subjected. this simplification necessitates research in is adoption that considers the complexities beyond the rational model (fichman, 2004). the institutionalism approach to explaining is adoption has gained support; particularly in light of the economic-rationalistic approach’s limited ability to explain the complexity of is adoption decisions. as north (1990, p.4), notes “in the jargon of the economist, institutions define and limit the set of choices of individuals”. in other words, institutionalism takes into account the rules, constraints, and the context and circumstances in which individuals and organizations operate. while is research has given this approach various terminologies (e.g., australasian journal of information systems perdana, robb, rohde & birt 2018, vol 22, research article sbr adoption in australia 4 institutional forces, organizing vision, and innovating mindfulness), its roots remain in institutionalism (nickerson & muehlen, 2006). the application of the institutionalism approach can generally be divided into macro and micro analyses. the macro analysis focuses on the organizations while the micro analysis suggests that individuals play important roles in businesses. the macro analysis largely uses dimaggio & powell’s (1983) conceptualization of three forces, namely, mimetic, normative and coercive. in the context of it adoption, the three forces can be explained as follows. mimetic isomorphism occurs when organizations imitate other organizational it adoption behaviours believing that such adoption could bring benefits. normative isomorphism refers to the pressures on organizations from institutions or customers arising from technological advancements, whereas, coercive isomorphism compels organizations to adopt technology as a result of governmental regulations (teo et al. 2003). critics question this conceptualization as it overlooks agentive behaviour, i.e., individuals’ actions that influence organizational behaviour. further, the robustness of the conceptualization of the three forces remains questionable (mignerat & rivard, 2009) as competitive and adaptive pressures may also necessitate businesses to adopt is (mignerat & rivard, 2009). in contrast, the micro analysis in the institutionalism approach suggests individuals, both as internal and external agents, have reciprocal relationships with the businesses. for example, the role of influential actors within particular businesses likely affects is adoption decisions or community discourses toward particular is features. thornton et al. (2012) conceptualized the macro and micro analyses in the institutionalism approach via institutional logics. institutional logics take into account the interrelationships between individuals and organizations and permits the institutionalism approach to consider, not just social actions, but also organizational ecology. for example, some phenomena might be better explained by businesses adapting to local environments rather than global settings in their quest to attain business stability (thornton et al. 2012). institutional logics, therefore, have advanced the institutionalism approach making it more encompassing. 2.2 rationalities in economics – constructivist rationality and ecological rationality constructivism, as described by smith (2003, p.468) “uses reason to deliberately create rules of action, and create human socioeconomic outcomes deemed preferable, given particular circumstances, to those produced by alternative arrangements.” that is, constructivist rationality seeks to arrive at optimum outcomes, rather than outcomes that satisfice, by relying on decision making that is deliberative, exercises cognitive processes, expectancies, and economic considerations. unlike constructivist rationality, ecological rationality contends that individual and organizational (business) decision making is not always deterministic, deliberative, and utility maximizing. ecological rationality further contends that the context in which individuals and organizations interact, and operate in, can influence their decision-making often compelling them to satisfice themselves and survive within their environment (smith, 2008; levin & aharon, 2014). in deference to ecological rationality, smith (p.468) then goes on to note, “…, it is important to remain sensitive to the fact that human institutions and most decision making is not guided primarily, if at all, by constructivism.” australasian journal of information systems perdana, robb, rohde & birt 2018, vol 22, research article sbr adoption in australia 5 2.3 two dominant paradigms versus rationalities in economics while both sets of explanations of is adoption arose from different fields, the economic-based is adoption paradigm conceptualizes is adoption like that of institutionalism, that is, via constructivist rationality and ecological rationality (smith, 2008). a similarity also exists between the economic-rationalistic approach and constructivist rationality (constructivism) in the economics (smith, 2008). thus, the two dominant paradigms in is adoption decisions, namely, the economic rationalistic approach and the institutionalism approach display similarities to the two economics-based rationalities. the economic rationalistic and constructivist rationality approaches have their roots in rational actions and economic considerations, while the institutionalism approach and ecological rationality have the similar core ideas, namely, institutional influences and adaptiveness relative to the surrounding environment. figure 1 presents the schema of the two dominant paradigms in is adoption decisions and illustrates their alignment with constructivist and ecological rationalities. figure 1. two dominant paradigms in is adoption both institutional logics and ecological rationality ostensibly have similar underlying assumptions, whereby collective individuals’ actions shape institutions and vice versa. overall, institutions are subject to institutional and environmental forces (nickerson & muehlen, 2006; smith, 2008; thornton et al. 2012). smith notes that his idea of ecological rationality is concerned with institutional adaptations with their surrounding environment such as markets, management, social, and other informal and formal systems. while institutional influence occurs via institutions following institutionalization processes (e.g., regulatory policies, social structures, technologies (mignerat & rivard, 2009), ecological rationality appears to consider both institutional and non-institutional influences. to make our conceptualization of rationalities comparable, we use the term ecological and constructivist rationality to guide our study. both rationalities align with the two dominant paradigms in is adoption. we submit that the use of two economics-based rationalities better articulates is adoption than the use of the current dominant is adoption paradigms. by is adoption decisions institutionalism approach (institutional logics) -social actions -institutional influence -adaptive economic-rationalistic -organizational resources -organizational expectancy -rational actions -economic considerations two dominant paradigms rationalities in economics constructivist rationality -rational actions -costs and benefits analysis -deliberative ecological rationality -institutional influence -non-institutional influence -organizational ecology -adaptive australasian journal of information systems perdana, robb, rohde & birt 2018, vol 22, research article sbr adoption in australia 6 considering both constructivist and ecological rationalities, we hope to add an environmental context to our investigation and uncover factors that facilitate or hinder actual implementation of sbr in businesses. 2.4 constructivist rationality, ecological rationality, organizational affordance, and organizational use of is relevant to is adoption, constructivist rationality can be defined as the deliberate use of reason to analyse is adoption and use based on the expectancy alternatives (smith, 2008). following this definition, we argue that when making is adoption decisions, individuals will evaluate the costs and benefits of the technology, as well as whether their organizations are willing to spend resources on the adoption process. in is literature, the aforementioned statement corresponds with performance and effort expectancy (venkatesh et al. 2003). while the two constructs were originally used at the individual level, at the organizational level, performance and effort expectancy are relevant in relation to the expectations of groups of people interacting with each other. in this study we defined performance expectancy as the organizational consideration given to whether or not is will enable and afford organizations to pursue their goals and obtain benefits (e.g., economic or financial) from is use. we define effort expectancy as the degree to which organizations are ready to use is in light of their resources. aside from considering the costs and benefits of adopting new technology, organizations are interested in what they can do with technology and how such technology can transform their business (markus & silver, 2008). this consideration depends on the features (i.e., functionality, ability, and quality) that the technology can offer to organizations. in this case, the scope of technology characteristics can include single systems and/or multiple systems. we argue that technological characteristics, therefore, contribute to is adoption decisions and how such technology is implemented in organization. to compliment the use of constructivist rationality, organizations may also use ecological rationality to adapt to their immediate environment such as supporting conditions, institutional influence, and organizational context. ecological rationality has its roots in ecological psychology suggesting that the surrounding environment (i.e., supporting environment, institutional influence, organizational context) in which people live, and interact with, plays a significant role in influencing the way they behave. this environment (e.g., objects, places, events, and circumstances) can help afford people to behave in a certain way. arising from this notion, the concept of affordance emerged in is literature. volkoff & strong (2013, p. 823), for example, define affordance as “the potential for behaviours associated with achieving an immediate concrete outcome and arising from the relation between an object (e.g., an it artefact) and a goal-oriented actor or actors”. at the organizational level, zammuto, griffith, majchrzak, dougherty & faraj (2007) specifically use the term affordance to capture the interrelationships between technology and business features that can conceivably affect businesses’ actions. multiple affordance concepts appear in the literature, for example, functional affordance (markus & silver, 2008) and organizational affordance (zammuto et al. 2007). taken together, we incorporate organizational affordance in our model to explain the possible behaviours arising from interactions between businesses and technology to afford their goals. this explanation aligns with zammuto’s et al. (2007) definition of organizational affordance reflecting the potential for businesses to coordinate actions to pursue their intended goal(s). australasian journal of information systems perdana, robb, rohde & birt 2018, vol 22, research article sbr adoption in australia 7 more specifically, organizational affordance emerges from the perceptions and actions of individuals within the business towards particular technologies and how those particular technologies should actually be used. while organizational affordance is a useful concept to help explain is adoption and use, few studies offer insight into the antecedents of organizational affordance. zammuto et al., (2007), for example, suggest that social settings such as, expertise, business processes and procedures, boundary-spanning approaches, and social capacities can determine organizational affordance (majchrzak & markus, 2012; seidel, recker & vom brocke, 2013; stendal, thapa & lanamäki, 2016; chatterjee, moody, lowry, chakraborty & hardin, 2015) to illustrate, businesses with high it experience would have different strategic implementations of enterprise resource planning (erp) software than businesses with little it experience (zammuto et al. 2007). while the features of the erp remain similar, the way businesses view and interact with the erp could lead to different strategic implementations. while the concept of affordance was originally derived from ecological psychology implying that affordance aligns with ecological rationality (markus & silver, 2008), the aforementioned examples also support the alignment of affordance with constructivist rationality (zammuto et al. 2007; leonardi, 2007). we speculate that businesses may develop such constructivist rationality via their expectancies toward particular technologies. alternatively, the characteristics of the technology per se can determine the way businesses should implement the technologies helping them to afford their goals. following this argument, we posit that constructivist rationality, ecological rationality, technology characteristics, and organizational affordance all play a role in the context of organizational is adoption and use decision. based on the two conceptualizations (i.e., constructivist rationality and ecological rationality) we proposed eight relevant constructs (i.e., performance expectancy, effort expectancy, technology characteristics, supporting environment, institutional influence, organizational context, organizational affordance, and organizational use of is) that can be used to explain is adoption decisions. of the eight categories, two categories (i.e., performance expectancy and effort expectancy) are subcategories of constructivist rationality, and three categories (i.e., supporting environment, institutional influence, and organizational context) are subcategories of ecological rationality. table 1 shows our proposed constructs. constructs definitions and explanations adapted from similarities to the ideas of prior studies difference between the ideas of prior studies constructivist rationality defined as the deliberate use of reason to analyze is adoption and use based on the expectancy alternatives. smith (2008) similar in meaning to the deliberate use of reasons when making decisions. the concept of constructivist rationality in the economic field is broader than in the current study. the definition of constructivist rationality proposed in this study is specific to organizational behavior towards is adoption use. the underlying australasian journal of information systems perdana, robb, rohde & birt 2018, vol 22, research article sbr adoption in australia 8 constructs definitions and explanations adapted from similarities to the ideas of prior studies difference between the ideas of prior studies assumption of constructivist rationality in this study is organizations develop their expectancies toward particular is. performance expectancy defined as the organizational consideration given to whether or not is will enable and afford organizations to pursue their goals and obtain benefits (e.g., economic or financial) from is use. venkatesh et al. (2003) similar to the concept of performance expectancy. the concept is adapted to the organizational level. effort expectancy defined as the degree to which organizations are ready to use is considering their resources. venkatesh et al. (2003) similar to the concept of effort expectancy. the concept is adapted to the organizational level. technology characteristics defined as the is’ features including functionality, ability, and quality. the scope of technology characteristics can include single systems and or multiple systems. goodhue & thompson (1995); goodhue (1995); markus & silver (2008) similar to the concept of it artifact. not applicable. ecological rationality defined as the organizational adaptations to their immediate environment such as, supporting conditions, institutional influence, and organizational context. smith (2008) similar in meaning to the environmental adaptations. the concept is specifically adapted to explain the environmental influences on organizational decision making. supporting environment defined as the extent to which organizations consider that technical infrastructure, venkatesh et al. (2003) similar in meaning to the facilitating conditions. different from the concept of facilitating conditions in that supporting environment also australasian journal of information systems perdana, robb, rohde & birt 2018, vol 22, research article sbr adoption in australia 9 constructs definitions and explanations adapted from similarities to the ideas of prior studies difference between the ideas of prior studies market and consumer demand exists to support is adoption and use. considers regulatory policies, and market and consumers' demand. institutional influence defined as the extent to which an organization considers that other organizations and policies can influence their is adoption and use decision. venkatesh et al. (2003) similar in meaning to the social influence. different from the concept of social influence in that the influential factor is organizations rather than social peers. we consider supporting environment, and organizational context differ from institutional influence in term of the institutionalizations processes. (tolbert and zucker, 1996; greenwood et al., 2002) organizational context defined as the organizational size, organizational type. not applicable not applicable. not applicable. organizational affordance potential behaviors arising from the collective views of individuals within organizations when interacting with technology to afford their goal oriented actions. markus & silver (2008); leonardi (2011); volkoff & strong (2013) similar in meaning to affordance concept. not applicable. organizational use of is the actual use of is. venkatesh et al. (2003) similar in meaning to use behavior the concept is adapted to the organizational level. table 1. explanatory constructs in is adoption decisions 3 research context, data and research method 3.1 research context: sbr adoption in australia sbr is a standard approach to online or digital record-keeping, and extensible business reporting language (xbrl) is the enabling technology for sbr for business transactions. the use of xbrl is expanding globally. in the us, the securities and exchange commission (sec hereafter) allowed companies to voluntarily file xbrl-based company reports starting 2005 and in 2009 mandating xbrl-based reporting for public companies (us securities and australasian journal of information systems perdana, robb, rohde & birt 2018, vol 22, research article sbr adoption in australia 10 exchange commission, 2016). countries that have adopted xbrl for business-to-government reporting include canada, uk, singapore, spain, and sweden (azam & taylor, 2011; chen, 2012). countries which have mandated xbrl for reporting to government agencies include japan, denmark, the netherlands, and the united kingdom. in australia, sbr was introduced in 2010 to reduce the regulatory reporting burden, and in consequence, also reduce the cost burden on businesses. one of the main benefits of sbr is its ability to create significant efficiencies in the regulatory reporting process (robb, rohde & green, 2016). in the context of business reporting, the costs arise due to the exchanges between those who prepare (data and report producers) and those who use (data and report consumers) financial reports. as noted in its sbr options paper, the premise of sbr is that businesses’ reporting information to government will be ‘recorded once, reported to many’ to help enhance business efficiency (treasury, 2012). despite sbr’s availability since 2010, its awareness and adoption has been limited in australia (nagy et al. 2008, azam & taylor, 2011; 2016). to help encourage uptake of sbr, since 2015 the australian securities and investments commission (asic) has permitted the use of improved digital reporting technology (ixbrl) removing the requirement for lodging entities to submit their financial reports as human readable pdfs (previously businesses using xbrl also had to submit human readable pdfs). reported benefits of using xbrl include improving the efficiency of financial reporting, accuracy of the financial data, timeliness and reliability of the data, and the ease with which data can be acquired and analysed (garner, henderson, sheetz, & trinkle, 2013). in australia, the purported benefits of using sbr include; the adoption of a common reporting standard (xbrl taxonomy), making financial reporting a by-product of natural business processes, and providing an electronic interface to agencies directly from accounting software to provide validation of reports (treasury 2012). prior research in the xbrl literature has primarily focused on the theoretical attributes and benefits associated with xbrl-based reporting from a user perspective (alles & piechocki, 2012; baldwin, brown & trinkle, 2006; janvrin & mascha, 2010; vasarhelyi, chan & krahel, 2012; muthusamy, bir & birt, 2017). recent research has found that the anticipated benefits of xbrl focus predominantly on efficiency within the realm of the data and report-producers and on effectiveness within the realm of the data and report-consumers (robb, et al. 2016). financial statements using xbrl, arguably, assist in increasing information transparency and ease of acquisition for financial analysis tasks (muthusamy et al. 2017; janvrin, pinsker, & mascha, 2013; steenkamp & nel, 2012). transparency of financial information is also claimed to be enhanced with the use of xbrl. transparency refers to the ability to identify and evaluate financial information in all parts of financial reports, including information contained in the disclosure notes (hodge, kennedy & maines, 2004). xbrl’s tagged information is clear and well documented and allows users to integrate information from various locations in the financial reports (muthusamy et al. 2017; hodge et al. 2004; alles & piechocki, 2012). further, xbrl allows information to be retrieved on a timely basis (azam & taylor, 2011). these features, particularly, the tagging permitted by xbrl-enabled reporting are absent in traditional financial reporting. while studies report the numerous advantages associated with the perceptions of users and the ease of preparation of reports, globally, voluntary xbrl adoption has been limited. possibly, this limited adoption is due to stakeholders’ concerns, e.g., filers, standard setters, regulators and/or software companies who have yet to promote the potential of sbr in terms australasian journal of information systems perdana, robb, rohde & birt 2018, vol 22, research article sbr adoption in australia 11 of data reliability, value-adding, and easily integrated consumption tools. there are other issues concerning the current “language” firms’ management information systems use and the on-going commitment of regulators (harris & morsfield, 2012). other matters associated with low adoption rates include, but are not limited to, perceptions of the low business value of xbrl for internal use, lack of education in the business community of the benefits of xbrl, and a lack of pressure to adopt xbrl from regulators (garner et al. 2013). globally xbrl-enabled reporting is gaining speed. for example, the european banking authority is mandating xbrl-enabled co-rep/fin-rep for regulatory reporting from the eu member states (european banking authority, 2013). the us sec now has mandatory xbrl-enabled financial reporting for all listed companies. increasingly, it appears that if australia wants to be part of the global capital market, it may find itself having to ensure much greater take-up of sbr – which is currently voluntary. australia is not unique in having limited sbr uptake (e.g., swedish bolagsverket), but somewhat unique in that during november 2012 treasury released an options paper calling for submissions on the use of sbr for financial reports. except for one confidential submission, all submissions to treasury have been made public. this paper examines stakeholder submissions offered in response to the three proposed sbr reporting options canvassed in treasury’s 2012 options paper. the first option suggested by treasury is the mandatory lodgement of financial reports using sbr. the second option is the voluntary lodgement of financial reports in ixbrl format, and the third option is to retain the status quo, i.e., mandatory pdf/paper reporting and voluntary xbrl. 3.2 data during the period november 2012 to march 2013, treasury sought stakeholders’ opinions on how sbr can be leveraged to enhance business reporting efficiency. treasury received 24 submissions from multiple stakeholders. to undertake cluster analysis as to whether australian sbr adoption matches with the two conceptualizations of rationality, we used the 23 publicly available sbr submissions to (australian) treasury and classified them into seven groups: regulator; professional body; company; accounting firm; developer; interest group; and a professional body’s australian business school representative. table 2 presents a summary of the submissions and stakeholder group. no. submissions stakeholder group 1 ernst & young accounting firm 2 grant thornton accounting firm 3 pitcher partners accounting firm 4 pricewaterhousecoopers accounting firm 5 westworth kemp consultants accounting firm 6 australian business school academic submission/cpaa academic representative 7 arkk solutions company 8 bhp billiton company 9 caseware developer australasian journal of information systems perdana, robb, rohde & birt 2018, vol 22, research article sbr adoption in australia 12 no. submissions stakeholder group 10 impact management group developer 11 muli management developer 12 villagemall operations developer 13 the group of 100 interest group 14 xbrl uk interest group 15 australian bankers' associations, inc. (aba) professional body 16 australian institute of superannuation trustees (aist) professional body 17 australian mutual provident (amp) professional body 18 australian shareholders' association (asa) professional body 19 business council of australia (bca) professional body 20 cpa australia and the institute of chartered accountants in australia/cpa icaa (now chartered accountants australia and new zealand/caanz) professional body 20 chartered secretaries australia professional body 21 institute of certified bookkeepers professional body 22 australian accounting standard board (aasb) regulator table 2. submissions to (australian) treasury relative to options for sbr adoption 3.3 research method this study uses an interpretive case study approach (klein & myers, 1999). we expect to provide theoretical insight derived from our case study (eisenhardt, 1989). we used cluster analysis as our analytical approach and synthesizing strategy of our interpretive case study (mills, durepos & wiebe, 2010). in this study, we have no predefined independent and dependent variables. we, however, use the two conceptualizations of rationality as a sensitizing device to guide us to derive a priori constructs of the coding scheme. this schema is necessary to make sense of the textual data and to guide us to examine the themes embedded within the sbr submission documents. table 3 presents our coding scheme. no. constructs use in coding scheme 1 performance expectancy 2 effort expectancy 3 technological characteristics 4 supporting environment 5 institutional influence 6 organizational context 7 technological affordance 8 organizational use of is table 3. coding scheme three authors were involved in the cluster analysis by manually coding the submission documents. each author analysed all 23 submission documents, evaluated the same contents australasian journal of information systems perdana, robb, rohde & birt 2018, vol 22, research article sbr adoption in australia 13 and coded sentences from the documents into the relevant proposed constructs. we then counted how many times each constructs appeared across the 23 documents. for the purpose of inter-rater reliability analysis, we avoided double counting for each category; therefore, the maximum numbers of variables appearing in the overall documents are 23. we employed interclass correlation (icc) to analyse the inter-rater reliability (shrout & fleish, 1979; mcgraw & wong, 1996). the interrater reliability analysis (icc=.817) shows strong agreement with the coding analysis. to store the data and to facilitate further analyses of the documents we corroborated the coded documents using nvivo. 4 case study findings we undertook cluster analysis of the 23 submission documents based on coding similarity to give us some indication of the proximity between categories. the result of the cluster analysis shows close proximity between constructivist and ecological rationalities (see figure 2). we contend that this proximity implies that these two rationalities form the basis of analysing is adoption decisions. along with technological characteristics, these two rationalities are subordinate to the branch of organizational affordance. this finding is logically consistent with prior literature whereby collective rationalities among individuals within organizations facilitate rationalities to act on affordances with the technologies (zammuto et al. 2007; leonardi, 2011; volkoff & strong, 2013). to deepen our cluster analysis, we examine relevant coded sentences or paragraphs from the 23 submission documents associated with constructivist and ecological rationalities, technological characteristics, organizational affordance, and organizational use of information systems. figure 2. cluster analysis based on coding similarity 4.1 constructivist rationality and ecological rationality recall, in our coding schemes, two categories, i.e., performance expectancy and effort expectancy embody constructivist rationality. in our definition, performance expectancy reflects organizational beliefs relative to the benefits of is adoption, whereas effort expectancy reflects organizational beliefs relative to the availability of the necessary resources to adopt is. as was anticipated these two expectancies appear in the documents. cpa australia and the institute are strong supporters of government programs that improve the efficiency and productivity of the australian economy. we believe that the option of the inline extensible business reporting language (ixbrl) for the lodgement of financial reports via standard business reporting (sbr) has the capacity to make a positive difference to the australian economy (cpa icca) australasian journal of information systems perdana, robb, rohde & birt 2018, vol 22, research article sbr adoption in australia 14 pwc supports initiatives that improve the transparency of financial reporting, the comparability of information in the global capital markets, and efficiencies in the systems and processes used by businesses. we believe the sbr initiative will provide significant benefits across all of these areas (pwc). aba members welcome initiatives to reduce the regulatory burden on business, but believe that mandatory lodgement at this time would require extensive and costly changes for all reporting systems, for all businesses, when other regulatory requirements are also driving major technology projects (aba). the cost to preparers was high and incremental as they often had to upgrade their financial reporting software to acquire a financial reporting tool with auto tagged templates and employ additional qualified staff to create tagged templates and complete the preparation of the tagged financial statements (bhp billiton). stakeholders were obviously aware of the potential consequences of sbr adoption including increased productivity and efficiency. at the same time, they were concerned with the costs and additional resources required to facilitate sbr adoption. these considerations reflect deliberate rationality involving costs and resources availability to pursue greater benefits from sbr adoption. this finding helps confirm our expectation that performance expectancy and effort expectancy are relevant proxies for constructivist rationality. in contrast to constructivist rationality where decision-making is deliberative, decisionmaking based on ecological rationality is adaptive. ecological rationality takes into consideration the environmental conditions under which decisions are made. organizations are inextricably linked with individuals, community, and their environment compelling them to be more adaptive and to create social constructions in which they produce and reproduce meanings, and rationalize their decisions (smith, 2008; thornton et al. 2012). we found stakeholders were concerned about the supporting environment (e.g., availability of technical infrastructure, market and consumer demand) as presented in the following excerpts. csa considers that mandatory implementation — through the push for consistent information from government agencies — will need to overcome several fundamental framework issues before sbr will be readily adopted by business. as noted above, the government must first communicate with business about sbr and how it interacts with other reporting frameworks, and also articulate the benefits of sbr to business. without such clarity as to the benefits, any attempt to mandate sbr is likely to be illreceived (csa). one of the most commonly quoted reasons for the slow take-up of sbr was the lack of a comprehensive accounting software available on the market and the fact that the key players (e.g., myob) had not released an sbr-enabled product at that point (australian business school academic submission/cpaa). we also found institutional influence. one such example occurs when government and their regulatory policy necessitate organizations to rationalize their sbr adoption decisions. without regulatory encouragement of sbr lodgement it is difficult to entice businesses to change their processes and means of lodging statutory reports such as the asic financial statements (impact management group). australasian journal of information systems perdana, robb, rohde & birt 2018, vol 22, research article sbr adoption in australia 15 if the real drivers for the use of sbr are the efficient operation of government bodies, the only way to achieve adoption at present will be through compulsion. that compulsion needs to be aligned with the creation of software accessible and affordable to the range of businesses that will be affected by sbr based lodgement (westworth kemp). aside from supporting environment and institutional influence, business context such as, businesses’ size and type were also considered when making sbr adoption decisions. we believe that a difference of 12 months is sufficient difference in mandatory adoption dates between larger and smaller businesses will be appropriate, based on the dates that will be used to implement the superstream reforms (aist). yes, we do believe there is justification for excluding certain entities from a mandatory requirement to lodge financial reports via sbr, should that proposal proceed. given there is limited interest and a small number of stakeholders in non-listed companies, we believe that the sbr requirement should be limited to listed public entities (bhp billiton). despite the clear distinction between constructivist and ecological rationalities, our cluster analysis also found common ground between them. the following texts show that government support and engagement with companies potentially increases their awareness of the costs and benefits of sbr adoption. a more comprehensive engagement strategy with corporate australia that is focused on understanding the costs and benefits of businesses adopting sbr to lodge their financial reports. although we acknowledge the government has already undertaken a consultation process with the business community on its proposals, we would encourage a more far reaching, comprehensive engagement strategy that aims to educate businesses on the potential benefits and costs of implementing sbr (pwc). supporting environment reflects the environmental influence, whereby businesses consider whether sufficient technical infrastructure, market, and consumer demand exists to support is adoption and use. similar to the supporting environment, institutional influences, such as organizations, government, and their regulatory policy influences constructivist rationality. use of sbr hitherto has been voluntary and the rate of adoption has been low. without some sort of incentive and compulsion, the take-up rate is likely to continue to be low and the potential benefits are unlikely to eventuate (westworth kemp). different types of businesses also apply organizational rationality towards the costs and benefits of sbr adoption. for example, the size of businesses determines the extent to which they consider their resources are sufficient to pursue the benefits of sbr adoption. we note that with any new mandated format, there are normally costs incurred in changing over. these costs are going to be proportionately larger for small businesses as opposed to larger businesses, and we propose that there be a sufficiently large transition period, in order to allow for small business to spread the costs of implementation over, where necessary (aist). australasian journal of information systems perdana, robb, rohde & birt 2018, vol 22, research article sbr adoption in australia 16 4.2 technological characteristics businesses’ decisions to adopt is are more complex than that of individuals’ decisions. more specifically, sbr’s features and capabilities are among the key themes in the submission documents as noted in the following text. we consider ixbrl to be more suitable than xbrl given that ixbrl enables rendering financial data in a more visually appealing format for people. xbrl was originally designed as a machine-to-machine language (ernst & young). the electronic interface has the ability to validate the reporting information and confirm that reports have been received by the relevant government agency (australian business school academic submission/cpaa). when contributors to the white paper expressed their opinions relative to sbr features and specifications, the context is intertwined with both constructivist and ecological rationalities. for example, the capabilities of sbr to deliver human and machine-readable information are associated with stakeholders’ performance expectancy. similarly, stakeholder awareness of sbr’s features should accord with the stakeholders’ environment. the following texts illustrate such relationships. the options paper notes an important benefit of financial reports lodged using ixbrl are their design for use by both machines and humans (cpa icaa1). allowing five years will enable listed entities to include sbr in any planned software upgrades that currently exist, rather than it having to be done as a completely separate compliance exercise (cpa icaa). it also needs to undertake a communication program that assists business to understand sbr, how it interacts with existing reporting frameworks and processes, and how business as well as government agencies will benefit from its implementation (csa). 4.3 organizational affordance and organizational use of information systems the sbr/xbrl adoption decision is challenging due to cost and benefit considerations and environmental factors. such considerations and factors include performance and effort expectancy, technological characteristics, and regulatory support, all of which critically determine the likelihood of xbrl adoption and use (dunne, helliar & lymer 2013; guilloux, locke, & lowe 2013; valentinetti & rea 2013). these considerations and factors can have multiple interdependencies and interactions, for example, the cost and effort of implementing xbrl into existing reporting systems may not make xbrl implementation straightforward. additionally, regulatory policy (mandatory vs. voluntary) can have different impacts on xbrl implementation: it is true that the first year can involve some additional effort on the part of finance teams and filers, but our experience is that for most clients the second year of ixbrl is far less painful than they had been led to expect, with widespread acceptance as filers have established their own xbrl routines (arkk solutions). 1 now caanz (chartered accountants of australia and new zealand). australasian journal of information systems perdana, robb, rohde & birt 2018, vol 22, research article sbr adoption in australia 17 a rush to mandate it could generate a negative perception of sbr as another prescriptive government reporting requirement. in the short-term this could lead to the outsourcing of report preparation, which would undermine the fundamental objectives and potential benefits of the sbr program (bca). this raises the issue as to whether the ifrs taxonomy has sufficient tags/elements. fewer amounts of tags would mean that entities will need to create their own specialized dictionary or extension taxonomy, which increases costs and decreases comparability of financial information between entities of the same industry (ernst & young). the current sbr capability uses xbrl and the documents produced by xbrl are not presented in an easily understandable format, hence they are generally only machinereadable. therefore, this means that if xbrl is used it would be necessary to continue with mandatory lodgement of financial reports in either paper or pdf form (amp). financial reports, which are already prepared under the regime of comprehensive international financial reporting standards, contain qualitative as well as quantitative data. sbr’s focus on quantitative data could detract from the usefulness of these reports to users (amp). we also found a relationship between organizational affordance and organizational use of is, as shown in the following text. it should be noted, however, based on the feedback provided by the practitioners that they tended to use sbr to simply lodge reports rather than collate data and create reports directly from their accounting software. the feedback indicated that the level of understanding of sbr among reporting practitioners was still rather limited and they did not take the benefit of all of the functionalities and features of sbr. (australian business school academic submission for cpaa). relative to the organizational use of is, the following texts attest to the limited sbr uptake in australia. of those respondents who were aware of sbr to a greater or lesser extent (in total around 58 per cent), only 15 per cent had used sbr to lodge their business reports to the government (australian business school academic submission for cpaa). the take-up of sbr among businesses and reporting practitioners has been significantly lower than anticipated by the sbr stakeholders. the initial business case developed by the treasury in 2008 predicted a take-up rate of 12 per cent in the year (australian business school academic submission/cpaa). 5 discussion our case study findings reveal themes embedded in the 23 publicly available submissions to the treasury. following the coding scheme, we find that the sbr adoption in australia reflects our a priori categories of the coding scheme (see, research method section). not only do the submissions reflect expectations, but they also adhere to the relationships between categories in our coding scheme. in effect, the context in which the stakeholders2 expressed their opinions 2 sentiments and views towards sbr adoption by stakeholder type are summarized in appendix a. australasian journal of information systems perdana, robb, rohde & birt 2018, vol 22, research article sbr adoption in australia 18 toward sbr adoption helped guide us to the appropriate categories in our coding scheme, as well as to their related categories in the scheme. in the following, we begin the discussion from the constructivist and ecological rationalities’ perspectives, then we describe the relationships between the two rationalities and their impact on sbr’s uptake. our study suggests that both performance and effort expectancy are the two essential bases for constructivist rationality. that is, businesses’ consideration of the costs and benefits reflect their constructivist rationality in any sbr adoption decisions. businesses are able to develop their constructivist rationality via available xbrl information, e.g., from government agencies, software vendors, and consulting firms. we found that stakeholders are aware that they have to have sufficient financial, technical, and human resource wherewithal to adopt sbr. stakeholders were also quite aware that sbr is capable of improving government efficiency and productivity, but did not seem to place any particular significance on this capability. constructivist rationality in sbr adoption is of little surprise, mirroring similar adoptions in other jurisdictions. for example, the rationale behind the voluntary filing programs in the us was to investigate the costs and benefits of xbrl use (debreceny et al. 2005). similarly, in the uk, xbrl adoption was initially introduced to reduce costs associated with regulatory compliance (troshani et al. 2015). because xbrl or sbr adoption requires involvement of multiple stakeholders and multifaceted interactions, constructivist rationality is not a sufficient explanation for sbr adoption. while the costs and effort are necessarily assessed for sbr adoption, the benefits of xbrl may not directly accrue to every organization, rather, the greater benefit may accrue to the entire economy as is the case of the netherlands sbr (chen, 2013). unlike constructivist rationality which refers to the deliberate use of reason to achieve the desired goals (smith, 2008), we assume that ecological rationality is less deliberately pursued. ecological rationality suggests that the immediate environment compels organizations to ecologically rationalize their decision making (smith, 2008). recall, in our study, ecological rationality is concerned with the supporting environment, institutional influence, and organizational size. in terms of supporting environment, stakeholders’ concerns are related to sbr’s alignment with the existing systems and existing reporting framework, as well as the availability of appropriate software to facilitate sbr. aligning with zammuto et al. (2007) and volkoff & strong (2013), the potential for xbrl adoption by organizations is influenced by their ability to make sense of the interdependencies and interactions between costs, benefits, and their environment. given that the setting for this study is australia, it is very likely that many large businesses have well established and fully compliant systems in place for their financial reporting. without either compulsion or appropriate (business) benefits they will likely opt to remain committed to those systems rather than adopt sbr-based financial reporting. considering institutional influence, when lodging financial reports in australia, companies have to comply with the corporations regulations 2001. section 1.0.07 general requirements for documents states: “unless asic otherwise approves, a document to be lodged must: (a) be on white or light pastel colour paper: (i) of international a4 size; and (ii) of medium weight and good quality; and (b) be clearly printed or written in black or dark blue in a manner that is permanent and will make possible a reproduction, by photographic, computerised or other electronic means that is satisfactory to asic… “ (australian government). while such regulation does not state that financial reports must be australasian journal of information systems perdana, robb, rohde & birt 2018, vol 22, research article sbr adoption in australia 19 human readable, the wording of the regulations implies that human readability is necessary – xbrl does not provide for human readability. therefore, australian xbrl filers who choose to lodge their financial reports via sbr, would still have to produce and lodge a pdf version of the reports to comply with the company regulations. clearly, almost no company would lodge an xbrl and a pdf version of their financial reports when a pdf only ensures their compliance with lodgement requirements. in 2015 asic approved the lodgement of ixbrl financial reports only as a permissible alternative. remaining with institutional influence, stakeholders apparently concur that regulation would play a significant role in hastening sbr adoption. this stakeholder view seems to imply that sbr adoption is likely to languish without compulsion. such compulsion, however, should take into account the supporting environment and organizational context, for example, the availability of affordable, firm-appropriate xbrl software. at present, accounting software in the australian market suitable for small to medium enterprises (sme) does not appear to have xbrl functionality. for example, govdirect a company providing software permitting smes to manage, process and submit government obligations electronically, was no longer in operation as of 18 december 2015. while large businesses may have the wherewithal (if not the will) to lodge financial reports via sbr, compelling smes to use sbr without appropriate software is an untenable situation. while prior studies appear to have dichotomized constructivist and ecological rationalities, our cluster analysis suggests that relationships apparently exist between constructivist and ecological rationalities. ecological rationalities can influence organizational constructivist rationalities. for example, government engagement with businesses contemplating sbr adoption can help them to better understand the costs and benefits of sbr and create a more supportive environment. we also found that both institutional influences like mandatory policy and the provision of incentives can enhance the potential benefits of sbr adoption. this finding accords with organizing visions theory (swanson & ramiller, 1997), whereby, key actors can help mobilize collective understanding of particular technologies or innovations via information diffusion. both rationalities cannot be separated from the characteristics of technologies. further, technology context and characteristics (i.e., relative advantage, compatibility, and complexity) can act as either inhibiting or driving factors of is adoption (tornatzky & fleischer, 1990). because sbr uses xbrl as the enabling technology, sbr’s inherits characteristics such as being machine-readable, capable of solving interoperability issues between multiple systems, and capable of leveraging information via interactive data visualization (perdana, robb & rohde, 2015). technological characteristics are among the contributing factors that interact with both constructivist and ecological rationalities. for example, the capability of sbr to deliver financial statements in interactive forms has the potential to increase performance expectancy. additionally, because sbr can interact with multiple systems, the supporting environment such as technical infrastructure, market and consumer demand, and appropriate government policy are necessary to achieve optimal sbr adoption. while in 2012 australian treasury released a sbr reporting options paper and received submissions from interested parties, aside from permitting ixbrl report lodgement in 2015, there has been silence ever since. we can only conclude that mandatory sbr reporting is no longer a high priority, and that the status quo will remain into the foreseeable future. however, should lodging financial reports via sbr become mandatory, constructivist and ecological australasian journal of information systems perdana, robb, rohde & birt 2018, vol 22, research article sbr adoption in australia 20 rationalities or technological characteristics no longer figure in lodgement choice decision making. imprudent mandatory policy, however, may impact negatively on perceptions of sbr and could inhibit the effective use of sbr in pursuing greater productivity. further, an xbrl taxonomy that accommodates companies’ needs, yet remains parsimonious, can increase the comparability of financial statements. should the taxonomy be insufficient, the costs associated with implementing taxonomy extensions are likely to increase, conversely, however, the comparability of financial reports will likely decrease. all of the aforementioned findings, that is, costs and benefits, the wherewithal related to finance, technology, and human resources, corporations regulations, and regulators have the potential to influence the adoption of sbr in australia. one influence that should not be underestimated is the uniqueness of the australian economy. australia has an advanced, diverse, first-world economy, however, 97% of all businesses are small businesses. further, 62% of small businesses are non-employing businesses and 75% of employing small businesses employ 1 to 4 employees (treasury, 2016). having such a concentration of small (and micro) businesses has, at least, three effects on sbr adoption in australia. first, the accounting software used by the majority of small businesses in australia (e.g., myob and reckon) almost certainly has no sbr capability, consequently, these small businesses have no means by which to access and use sbr-based financial reporting. arguably, and somewhat contrary to sbr’s raison d’etre, small businesses stand to benefit most from the reporting efficiencies offered by the ‘recorded once, reported to many’ benefits promoted by treasury. second, given the size of many small businesses there will be little, if any, demand from investors or other small businesses to compare the performance of such small businesses for, say, investment or financing purposes. third, and related to small businesses’ lower survival rates and volatile revenues, small businesses eschew using banks for debt finance, instead drawing on owners’ household assets for financing (connolly, norman & west, 2012). in short, given the characteristics of the australian economy, that is, the high proportion of small businesses, the decision not to adopt sbr is an example of an entire section of the economy pursuing ecological rationality relative to sbr adoption. taken together, our analysis confirmed that sbr uptake in australia is rather limited. while some stakeholders’ predicted that the uptake rate was 12 percent per year, in reality, businesses using sbr for financial reporting account for only a tiny fraction of that figure. as the australian business register’s (abr) report of the registrar notes for the period 2010-2013 “activity statements, pay as you go reports and tax file number declarations lodged to the ato accounted for the vast majority of sbr lodgements.” for the 2014-20153 period the report of the registrar notes, “along with its use for business-to-government transactions, sbr also underpins business-to-business transactions between employers and super funds under the superstream initiative.” in both post-sbr reports there is no mention of numbers or trends relative to companies submitting financial reports via sbr, rather the reports focus on mandatory superstream reporting and lodgement of government forms, for example business activity statements and tax file number declarations. given the stated benefits of lodging sbr based financial reports, one might assume that australian businesses are “radical non-adopters”, however, when viewed in light of 3 the 2014-15 report of the registrar is the most recent available on the australian business register’s website. australasian journal of information systems perdana, robb, rohde & birt 2018, vol 22, research article sbr adoption in australia 21 constructivist and ecological rationalities in concert with the australian economy’s characteristics, australian businesses’ actions simply reflect both constructivist and ecological rationalities and their environment. 6 contributions and limitations the findings from this study make three theoretical contributions to the current literature. first, our conceptualization guided by economics-based rationalities enriches our understanding of is adoption. while prior literature has explored is adoption, both from the economic-rationalistic and institutionalism perspectives, such perspectives have drawn criticism from the literature. in this current study, we show that is adoption involves both constructivist and ecological rationality. our findings show that organizations constructively develop their rationality based on the availability of information and resources. organizations, however, may not simply consider costs and benefits of is adoptions, but they are also adaptive to their environment. our analysis of the text confirms this circumstance, whereby, the alignment between existing systems, government support, software availability, and the economic environment, more generally, contribute to the likelihood of sbr’s uptake. businesses’ adaptation to the prevailing environment also appears to shape their economic considerations. we argue, therefore, that a focal point apparently exists between constructivist and ecological rationalities. second, our findings reveal themes embedded in the 23 submissions to the treasury reflecting our a priori categories of the coding scheme (see research method section). this finding indicates that the limited sbr adoption for financial reporting in australia can be explained from constructivist and ecological rationality perspectives. third, we use the concept of organizational affordance to further explain the relationship between the two rationalities and sbr (non) adoption in australia. volkoff & strong (2013) suggest that affordances are generative mechanisms which may or may not cause an event. organizational affordance offers organizations possibilities to use is for their desired goals, however, if those goals are already achieved (e.g., compliance with regulations), further affordances’ costs may outweigh any perceived benefits. organizational affordance will not be actualized unless the organizations have the wherewithal, capability, and motivation to do so (volkoff & strong 2013). to illustrate, multiple jurisdictions and organizations have taken various approaches to implementing xbrl-enabled technologies. in australia, for example, xbrl-enabled technologies were introduced as the backbone of sbr to reduce the cost burden of regulatory reporting on business. the netherlands uses xbrl via sbr to streamline administration works and information exchange. both are multi-purpose instantiations of xbrl-enabled technologies. in contrast, the us uses xbrl for financial report filing (10-k) only – a single purpose instantiation of xbrl-enabled technology. similarly, the uk uses ixbrl for mandatory corporate tax filing to her majesty’s revenue and customs (hmrc) as well as for optional filing of annual returns to companies house. while xbrl can potentially afford multiple possibilities, organizations may not realize those possibilities unless they actualize their affordance with xbrl. following on from the three theoretical contributions, we offer the following practical contribution. we contend that considering ecological rationality can help progress sbr adoption in australia. that is, we encourage actors to develop a shared understanding among australasian journal of information systems perdana, robb, rohde & birt 2018, vol 22, research article sbr adoption in australia 22 stakeholders relative to sbr adoption in australia. this shared understanding can heighten the legitimacy of sbr in australia helping turn optimal adoption into reality to gain the intended benefits, particularly for smes. further, well targeted incentives and ensuring that regulations governing lodgement of financial reports align with the aims of sbr could also help advance the sbr’s uptake. for example, incentives could have been made available to developers to incorporate the sbr taxonomy into popular sme accounting packages or the implied requirement for human-readable financial reports could have been revised at the outset of the sbr program. along with constructivist rationality and technological considerations, ecological rationality can help companies to substantiate and implement actions to adopt and use sbr. to illustrate, while the ability of xbrl-enabled technologies to streamline financial reporting and digital record-keeping is not in doubt, the implementation of xbrl in several jurisdictions remains challenging. for example, in france, a lack of institutional acceptance of xbrl implementations has helped undermine xbrl’s wide-ranging abilities (guilloux et al. 2013). in such circumstances, organizations may be reluctant to adopt xbrl-based technologies for fear of being left with a superseded technology. in contrast, the mandatory xbrl reporting policy in the us has unlocked multiple possibilities from xbrl use. aside from facilitating financial report filing, xbrl can afford organizations more ready compliance with regulations, leverage the value of financial data via business intelligence and analytics (bi&a), enable interactive data and information visualizations (idiv), and facilitate continuous auditing. we acknowledge the limitations of this study. while the authors remain informed of the current status of sbr adoption in australia, our interpretive case study relies on the data from 23 submission documents to the australian treasury. one might say that relying on these submissions alone may not generate a sufficiently broad palette of stakeholders’ views. several technological considerations that could impact sbr adoption such as security, trust and privacy issues were not exhibited in the submissions. since security, trust and privacy are important in is adoption decisions, future research could provide additional findings complementing this current research (carter, shaupp, hobbs & campbell, 2011; ali, soar & yong, 2016). future research may refine this study with broader data from multiple stakeholders (e.g., interviews, regulatory policy, and government documents). undertaking interview or survey to multiple stakeholders involved in sbr adoption in australia, for example, can provide insightful findings to complement or confirm this current research. further, future research may also employ grounded theory method to provide more insightful findings relative to sbr adoption in australia. 7 conclusion whilst acknowledging the above limitation, there appears to be themes emerging relative to constructivist and ecological rationalities. in australia, voluntary adoption of sbr-based financial reporting has not had its anticipated uptake. on the basis of cost – benefit analyses, from a constructivist rationality perspective, the case for sbr reporting appears quite attractive. to a large extent, however, australian companies have not yet adopted sbrenabled financial reporting. it seems that satisfying the constructivist requirements of an adoption choice is not sufficient motivation to actually adopt a particular technology. ecological rationality appears to play a substantial role in such adoption choices. in the case of sbr, it appears that without the right combination of the environmental, technological, and australasian journal of information systems perdana, robb, rohde & birt 2018, vol 22, research article sbr adoption in australia 23 regulatory strategies, affordances, and incentives, companies make their own rational decisions not to adopt technologies. while the choice to mandate the use of technologies such as sbr remains with governments, there still needs to be awareness that consideration must still be given to the aforementioned strategies, affordances, and incentives to ensure the desired adoption of the particular technology. references ali, o., soar, j., & yong, j. 2016. an investigation of the challenges and issues influencing the adoption of cloud computing in australian regional municipal governments. journal of information security and applications, 27-28, 19-34. alles, m. & piechocki, m. (2012), will xbrl improve corporate governance? a framework for enhancing governance decision making using interactive data. international journal of accounting information systems, 13(2). 91-108. australian government (2001). federal register of legislation. https://www.legislation.gov.au/details/f2013c00529/html/volume_1#_t oc362443230 accessed: [22 february 2017]. australia government (2013), report of the registrar https://abr.gov.au/about-us/report-ofthe-registrar-previous-years/report-of-the-registrar-2012-13/performance-andachievements/ accessed: [19 april 2018] australia government (2015), report of the registrar https://abr.gov.au/about-us/report-ofthe-registrar-previous-years/report-of-the-registrar-2014-15/overview/ accessed: [19 april 2018] azam, s., & taylor. d. (2011). standard business reporting in australia: voluntary take-up issues facing users, preparers and regulators of company financial and business reports. accountancy business and public interest, 10(1), 16-41. azam, s., & taylor. d. (2016). adopting standard business reporting in australia: are cfos persuaded by technology attributes? malaysian accounting review, 12 (1), 27-54. baldwin, a.a., brown, c. e. & trinkle, b. s. 2006. xbrl: an impacts framework and research challenge. journal of emerging technologies in accounting, 3, 97-116. carter, l., shaupp, l.c., hobbs, j., & campbell, r. 2011. the role of security and trust in the adoption of online tax filing. transforming government: people, process and policy, 5(4), 303-318. chatterjee, s., moody, g., lowry, p.b., chakraborty, s., & hardin, a. (2015). strategic relevance of organizational virtues enabled by information technology in organizational innovation. journal of management information systems, 32(3), 158-196. chen, y-c. (2012). a comparative study of e-government xbrl implementations: the potential of improving information transparency and efficiency. government information quarterly, 29 (4): 553– 563. chen, y-c. (2013). improving transparency in the financial sector: e-government xbrl implementation in the united states. public performance & management review, 37(2): 241262. australasian journal of information systems perdana, robb, rohde & birt 2018, vol 22, research article sbr adoption in australia 24 compagni, a., mele, v., & ravasi, d. (2015). how early implementations influence later adoptions of innovation: social positioning and skill reproduction in the discussion of robotic surgery. academy of management journal, 58(1), 242-278. connolly, e. , norman, d. & west, t. (2012). small business: an economic overview. reserve bank of australia. available at: https://www.rba.gov.au/publications/workshops/other/small-bus-finroundtable-2012/pdf/small-bus-fin-roundtable.pdf [accessed: 22 february 2017] debreceny, r., chandra, a., cheh, j.j., guithues-amrhein, d., hannon, n.j., hutchison, p.d., janvrin, d., jones, r.a., lamberton, b., lymer, a., mascha, m., nehmer, r., roohani, s., srivastava, r.p., travelsi, s., tribunella, t., trites, g., & vasarhelyi, m. a., (2005). financial reporting in xbrl on the sec’s edgar system: a critique and evaluation. journal of information systems, 19 (2), 191–210. dimaggio, p. j., & powell, w.w. (1983). the iron cage revisited: institutional isomorphism and collective rationality in organizational fields. american sociological review, 48 (2): 147– 160. dunne, t., helliar, c., lymer, a., & mousa, r. (2013). stakeholder engagement in internet financial reporting: the diffusion of xbrl in the u.k. the british accounting review, 45 (3), 167–182. eisenhardt, k.m. (1989). building theories from case study research. the academy of management review, 14(4), 532-550. european banking authority. (2013). update on the technical standards on supervisory reporting requirements. available at http://www.eba.europa.eu/-/update-on-thetechnical-standards-on-supervisory-reporting-requirements. fichman, r. g. (2004). going beyond the dominant paradigm for it innovation research: emerging concepts and methods. journal of the association for information systems, 5(8), 314-355. garner, d., henderson, d., sheetz, s.d., & trinkle, b.s. (2013). the different levels of xbrl adoption. management accounting quarterly, 14(2), 1-10. goodhue, d.l., & thompson, r.l. (1995). task-technology fit and individual performance. mis quarterly 19(2): 213–236. goodhue, d. 1995. understanding user evaluations of information systems. management science, 41(12), 1827–1844 guilloux, v., locke, j. & lowe, a. (2013). digital business reporting standards: mapping the battle in france. european journal of information systems, 22 (3), 257–277. harris, t. s., & morsfield, s. (2012) . an evaluation of the current state and future of xbrl and interactive data for investors and analysts. available at: https://www8.gsb.columbia.edu/ hirschheim, r. (2007). introduction to the special issue on “quo vadis tam – issues and reflections on technology acceptance research. journal of the association for information systems, 8(4): 203–205. australasian journal of information systems perdana, robb, rohde & birt 2018, vol 22, research article sbr adoption in australia 25 hodge, f.d., kennedy, j.j & maines, l.a. (2004). does search-facilitating technology improve the transparency of financial reporting. the accounting review 79(3): 687-703. janvrin, d., & mascha, m. f. (2010). the process of creating xbrl instance documents: a research framework. review of business information systems, 14(2), 11–34. janvrin, d. j., pinsker, r.e., & mascha, m.f. (2013). xbrl-enabled, spreadsheet, or pdf? factors influencing exclusive user choice of reporting technology. journal of information systems, 27 (2), 35–49. kaganer, e., pawlowski, s.d., & wiley-patton, s. (2010). building legitimacy for it innovations: the case of computerized physician order entry systems. journal of the association for information systems, 11(1), 1-33. klein, h. k. & myers, m. d. (1999). a set of principles for conducting and evaluating interpretive field studies in information systems, mis quarterly, 23(1), 67-93. leonardi, p.m. (2007). activating the informational capabilities of information technology for organizational change. organization science, 18(5), 813-831. leonardi, p.m. (2011). when flexible routines meet flexible technologies: affordance, constraint, and the imbrication of human and material agencies. mis quarterly, 35(1), 147-167. levin, y., & aharon, i. (2014). emotion, utility maximization, and ecological rationality. mind and society, 13, 227-245. majchrzak, a & markus, m. l. (2012). technology affordances and constraints in management information systems.in: kessler, e. (ed.), encyclopedia of management theory. sage publications, thousand oaks. markus, m. l., & silver, m.s. (2008). a foundation for the study of it effects: a new look at desanctis and poole’s concepts of structural features and spirit. journal of the association for information systems, 9(10), article 5. mcgraw, k.o., & wong, s.p. (1996). forming inferences about some intraclass correlation coefficients. psychological methods, 1(1), 30-46. mignerat, m., & s. rivard. (2009). positioning the institutional perspective in information systems research. journal of information technology, 24 (4): 369–391. mills, a.j., durepos, g., & wiebe, e. (2010). encyclopedia of case study research. london: sage. muthusamy, k. bir, p. & birt, j. (2017). xbrl and the qualitative characteristics of useful financial information. accounting research journal, 30 (1): 107-126. nagy, j., jubb, c., rouse, a. & cybulski, j. (2008). sbr/xbrl: a hot topic internationally with rising local prominence. deakin business review, 1 (2): 15.21. nickerson, j.v., & muehlen, m. (2006). the ecology of standards processes: insights from internet standard making. mis quarterly 30, 467-488. nielsen, j. a., l. mathiassen, & s. newell. (2014). theorization and translation in information technology institutionalization: evidence from danish home care. mis quarterly 38 (1): 165–186. australasian journal of information systems perdana, robb, rohde & birt 2018, vol 22, research article sbr adoption in australia 26 north, d.c. (1990). institutions, institutional change and economic performance, cambridge, cambridge university press perdana, a., robb, a. & rohde, f. (2015). an integrative review and synthesis of xbrl research in academic journals. journal of information systems, 29 (1): 115–153. robb, d., rohde, f. & green, p (2016) standard business reporting in australia: efficiency, effectiveness, or both? accounting & finance, 56 (2): 509–544. seidel, s; recker, j; & vom brocke, j. (2014). sensemaking and sustainable practicing: functional affordances of information systems in green transformations, mis quarterly, 37(4), 1275-1299. shrout, p.e., & fleish, j.l. (1979). interclass correlations: uses in assessing rater reliability. psychological bulletin, 86(2): 420-428. smith, v.l. (2003). constructivist and ecological rationality in economics. the american economic review, 93 (3): 465–508. smith, v.l. (2008). rationality in economics: constructivist and ecological forms. uk: cambridge university press. steenkamp, l.p. & nel, g.f. (2012). the adoption of xbrl in south africa: an empirical study. the electronic library, 30(3), 409 425. stendal, k., thapa, d., & lanamäki, a. (2016). analyzing the concept of affordances in information systems. in 2016 49th hawaii international conference on system sciences (hicss), ieee: 5270–5277. swanson, e. b., & ramiller, n.c. (1997). the organizing vision in information systems innovation. organization science, 8 (5): 458–474. teo, h.h., wei, k.k., & benbasat, i. (2003). predicting intention to adopt interorganizational linkages: an institutional perspective. mis quarterly 27(1), 19-49. the treasury. (2012). use of standard business reporting for financial reports. australian government. available at http://www.treasury.gov.au/consultationsandreviews/consultations/20 12/sbr-options-paper [accessed: 22 february 2017] the treasury. (2016). national small business statistics. release date: 23 september 2016. australian government. available at http://www.treasury.gov.au/publicationsandmedia/publications/2012/s ml-bus-data [accessed: 1 february 2017] thornton, p. h., ocasio, w., & lounsbury, m. (2012). the institutional logics perspective: a new approach to culture, structure, and process. new york, ny: oxford university press. tornatzky, l., & fleischer, m. (1990). the process of technology innovation. lexington: lexington books. troshani, i., paker, l.d., & lymer, a. (2015). institutionalising xbrl for financial reporting: resorting to regulation, accounting and business research, 45(2), 196-228. us security and exchange commission. (2016). structured disclosure at the sec: history and rulemaking. available at https://www.sec.gov/page/osdhistoryandrulemaking. australasian journal of information systems perdana, robb, rohde & birt 2018, vol 22, research article sbr adoption in australia 27 valentinetti, d. & rea, m. (2012). ifrs taxonomy and financial reporting practices: the case of italian listed companies. international journal of accounting information systems, 13(2), 163180. vasarhelyi, m. a., chan, d. y. & krahel, j. p. (2012). consequences of xbrl standardization on financial statement data, journal of information systems, 26(1), 155-167. venkatesh, v., morris, m.g., davis, g.b., & davis, f.d. (2003). user acceptance of information technology: toward a unified view. mis quarterly, 27(3), 425-478. volkoff, o., & strong, d.m. (2013). critical realism and affordance: theorizing it-associated organizational change processes. mis quarterly, 37(3), 819-834. zammuto, r.f., griffith, t.l., majchrzak, a., dougherty, d.j., & faraj, a. (2007). information technology and the changing fabric of organization. organization science, 18(5), 749-762. australasian journal of information systems perdana, robb, rohde & birt 2018, vol 22, research article sbr adoption in australia 28 appendix a: sentiments and views towards sbr adoption by stakeholder type sentiment/views category accounting firms companies professional bodies developers individuals constructivist rationality performance expectancy positive cautiously positive/ cost concerns positive effort expectancy positive cautiously positive/ cost concerns positive ecological rationality supporting environment supportive/ advocate engagement strategy supportive/ advocate engagement strategy institutional influence cautiously positive/ mandate recommended cautiously positive/ mandate recommended cautiously positive/ mandate recommended organizational context discern between large/small companies discern between large/small companies technological characteristics technology well-suited technology wellsuited/better explain benefits technology well-suited organizational affordance/ use of is cautiously positive/ taxonomy sufficiency concerns potentially negative/ cost and usefulness concerns cautiously positive/ better explain benefits the various stakeholders’ views reflect their roles relative to sbr and, to some extent, are likely are somewhat self-serving. for example, the accounting firms and professional bodies are generally view sbr’s adoption and possible mandatory use as australasian journal of information systems perdana, robb, rohde & birt 2018, vol 22, research article sbr adoption in australia 29 positive. the accounting firms and professional bodies both note the benefits to financial reporting transparency and efficiency. companies’ concerns, naturally enough, center on costs of software and staff upgrades, and that they, in fact, comply with current regulations. the developers, generally, advocate mandatory sbr reporting because their development efforts, from an investment perspective, become less risky. the individual’s views were from an academic perspective, accordingly they expressed confidence in the technology and that making sbr reporting mandatory, on balance, was desirable. overall, these views broadly align with robb et al.’s (2016) study on australian sbr adoption. copyright: © 2018 perdana, robb, rohde & birt. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 background 2.1 two dominant paradigms in information systems adoption – economic rationalistic and institutionalism 2.2 rationalities in economics – constructivist rationality and ecological rationality 2.3 two dominant paradigms versus rationalities in economics 2.4 constructivist rationality, ecological rationality, organizational affordance, and organizational use of is 3 research context, data and research method 3.1 research context: sbr adoption in australia 3.2 data 3.3 research method 4 case study findings 4.1 constructivist rationality and ecological rationality 4.2 technological characteristics 4.3 organizational affordance and organizational use of information systems 5 discussion 6 contributions and limitations 7 conclusion paper title (paper title style) australasian journal of information systems miah, shen, lamp, kerr & gammack 2019, vol 23, research on health information systems emerging insights of health informatics research 1 emerging insights of health informatics research: a literature analysis for outlining new themes shah j miah victoria university, melbourne, australia shah.miah@vu.edu.au jun shen university of wollongong, nws, australia john lamp deakin university, geelong, australia don kerr university of the sunshine coast, queensland, australia john gammack zayed university, abu dhabi, uae abstract this paper presents a contemporary literature review to provide insights into the current health informatics literature. the objective of this study is to identify emerging directions of current health informatics research from the latest and existing studies in the health informatics domain. we analyse existing health informatics studies using a thematic analysis, so that justified sets of research agenda can be outlined on the basis of these findings. we selected articles that are published in the science direct online database. the selected 73 sample articles (published from 2016 to 2018 in premier health informatics journals) are considered as representative samples of health informatics studies. the analysis revealed ten topic areas and themes that would be of paramount importance for researchers and practitioners to follow. the findings provide an important foundational understanding for new health informatics studies. keywords: health informatics; healthcare information systems; medical informatics; is research 1 introduction health informatics has experienced rapid research growth and application over the past few decades. as a subset of information systems research, the health informatics is a pivotal area, located at the intersection of information technologies, computer science, social science, behavioural science and health or medical care practices. these multidisciplinary fields are entangled in many forms of problem solving or information processing purposes, in order to meet the demands of various practitioners in the healthcare sectors. this applied research field focuses on key techniques, methods, resources, interventions, and devices that are required to improve data acquisition, retrieving, analysing, processing, displaying and use of information in health and medical decision making. health informatics is mainly powered by various forms of information technologies, internet enabled technologies, communication technologies, mobile or hand-held device based functionalities and telecommunication infrastructures, systems and their practices (mosa, yoo and sheets, 2012). this health information, as a multi-disciplinary field, is well-known due to its coverage as a wider australasian journal of information systems miah, shen, lamp, kerr & gammack 2019, vol 23, research on health information systems emerging insights of health informatics research 2 spectrum of its many sub-fields that combine the problem-solving skills of computer scientists or information systems specialists offering benefits to health care organisations and professionals or health care givers. the main aims of designing health informatics solutions are to deliver appropriate and innovative applications and technologies for improving healthcare information delivery. while the field is rapidly growing due health informatics has become an umbrella term that can be defined in the context of electronic health (e-health) research, clinical decision support systems (cdss), mobile health, electronic government for healthcare and telehealth (wang, turner and scott, 2003). this implies that scope of health informatics research is going beyond the health information products, systems and services, including tools for health practitioners and different users groups as well as personalised healthcare or monitoring systems for patients and citizens (european commission, 2004), as it can be related to any healthcare service delivery methods such as video conferencing, file sharing, blogging, and social media data exchange. health informatics has also been defined by world health organisations as an umbrella term and defined it to “encompass the rapidly evolving discipline of using computing, networking and communications – methodology and technology – to support the health related fields, such as medicine, nursing, pharmacy and dentistry” (al-shorbaji, 2001, p. 2). although vital research attention through a systematic literature review has been experienced over the past decades in the field, studies identifying themes as future research directions from existing research has been somehow limited. ikram, ghani and abdullah (2015) conduced a systematic literature review to analyse the application of health informatics in the four traditional medicine systems chinese medicine, ayurveda, traditional arabic and islamic medicine and traditional malay medicine. the study by ikram et al. (2015) utilised four databases (such as ebsco host, ieee explore, proquest and google scholar) finding the terms such as "traditional medicine", "informatics", "informatics infrastructure", "traditional chinese medicine", "ayurveda", "traditional arabic and islamic medicine", and "traditional malay medicine". based on the combination of the search terms, the study explored the involvement of informatics as a modern medicine system. deng et al (2018) produced a literature review that analysed existing medical informatics research by chinese academics on their articles published in international specialty journals. the main purpose of that study was to summarize the experiences and lessons learned in the medical informatics field. our aim differs from these works as we focus on revealing insights through exploring their themes that constitute essential knowledge for researchers and practitioners in the health informatics field. in our study, we analyse existing health informatics studies to reveal insights using a thematic analysis, so that we can outline research agenda for this special section, based on findings of that analysis. we selected articles that are published in elsevier’s science direct database1 (from 2016 to 2018) searching complete research and review articles from premier health informatics journals. the selected 73 sample articles are considered to be representative of health informatics field. this review paper is structured as follows: the next section describes the background of the study while the section 3 describes the methodological details. the section 4 presents the findings of the literature review through using the thematic analysis. the section 5 describes 1 https://www.sciencedirect.com/ australasian journal of information systems miah, shen, lamp, kerr & gammack 2019, vol 23, research on health information systems emerging insights of health informatics research 3 the selected articles published in the special section and further studies in the field. finally, the section 6 provides a discussion and summarises of the selected manuscripts. 2 background of the research field the term “health informatics” has been an umbrella concept used interchangeably with healthcare informatics, medical informatics, and biomedical informatics. this field has been defined as information engineering for healthcare or medical care benefits essentially the patient healthcare and its management, through the application of various information technologies. nadri et al. (2017) described that healthcare informatics is a multidisciplinary field that uses health information technologies in order to improve healthcare services via any combination of higher quality, higher efficiency as well as for creating new opportunities. the multidisciplinary field included information sciences, social, management, behavioural and other artificial sciences. o'donoghue and herbert (2012) argued that this multidisciplinary field deals with the resources, devices, and methods required to “optimize the acquisition, storage, retrieval, and use of information in health and bio-medicine. health informatics tools include computers, clinical guidelines, formal medical terminologies, and information and communication systems” (p.2). this implies that the research has been diversified in providing different healthcare or medical information support services. gardner et al. (2009) identified various concerns with the use of information technologies in healthcare by or for clinicians or practitioners. clinical or medical practitioners mainly aim to transform healthcare service by analysing, designing, implementing, and evaluating health information for better clinical and non-clinical outcome. the purposes of health informatics tools or applications were defined to 1) assess information and knowledge needs of practitioners, patients and communities; 2) characterize, evaluate, and refine clinical processes; 3) develop, implement, and refine clinical decision support systems; and 4) lead or participate in the procurement, customization, development, implementation, management, evaluation, and continuous improvement of clinical information systems. clinicians or medical practitioners collaborate with other health and information technology professionals to develop health informatics tools which promote patient care that is safe, efficient, effective, timely, patient-centred, and equitable. mobile health approaches are one of the popular effective provisions of health informatics research. m-health broadly refers to a mobile service or application for providing healthcare support to anyone, anytime, and anywhere (miah, gammack, hasan, hasan, 2016). utilising the processing power of mobile phones, gprs and internet technologies, m-health provides health professionals, patients, clinicians and other relevant users with support services to manage, disseminate, collect, administer, control and monitor healthcare information and improve health service delivery and quality of care support. the service provided by various m-health information systems (is) applications, eliminates geographical and temporal constraints while enhancing the coverage, quality, cost savings and other user provisions of healthcare (varshney, 2014; miah et al. 2017). as a sub-class of is, clinical decision support systems (cdss) are a type of specialised dss application that directly aids in “clinical decision making in which characteristics of individuals are matched to a computerized knowledge base for the purpose of generating patient-specific recommendations (hunt et al. 1998). combining these, mobile decision support applications have been developed for supporting decision making in clinical and non-clinical settings. these studies developed over the past decade have been viewed into three groups: 1) mobile based cdss for physicians and healthcare professionals (e.g. martínezaustralasian journal of information systems miah, shen, lamp, kerr & gammack 2019, vol 23, research on health information systems emerging insights of health informatics research 4 pérez et al. 2014; karim and bajwa, 2011); 2) mobile based decision support for outreach health workers (e.g. parveen et al. 2014; kuntagod and mukherjee, 2011); and 3) mobile based decision support for public use or improving public healthcare (e.g. ramesh et al. 2012; fung et al. 2014). the health informatics field has been extended over past decades to cover many directions in information systems. the sub-fields are healthcare service delivery technologies, data acquisition for healthcare or medical support; knowledge management for healthcare users, decision support systems for healthcare and medical purposes, healthcare modelling and simulation; data analytics, various health information systems, clinical information systems, integrated data repository, and computational intelligence for health care applications. hence, it is an essential task to provide a summary to determine the various paths of accumulated knowledge for is researchers within the area of health informatics using an application oriented perspective to provide a better understanding of the sub-field. a contemporary review discussion can offer such an overview on how the sub-field is creating an impact within the research field. it would also be an important reading for current practitioners and research students within the field. it can provide clearer understanding on the progressions of sub-fields within the field of health informatics. as such, the main aim of this study is to address the following research question: what are the current research themes in the existing health informatics studies that show promise for future knowledge growth? 3 study methodology in is research, a systematic literature review (slr) has been one of the popular methodologies used to reveal generalisable insights and themes that may help to create better understanding in shaping or developing an overview of a sub-field’s body of knowledge. webster and watson (2002) have indicated productive directions of conducting an slr that may contribute to is research and practice for preparing better future of the discipline. slrs not only organise cognate studies but can also sustain the evolution of evidence-based guidelines for practitioners (brereton et al. 2007). thus, we use the slr approach to derive deeper understanding of existing health informatics research following brereton et al.’s (2007) three phases of planning (defining research question and protocol), conducting (identify and assess relevant studies) and finally document review, where the synthesis of findings is written up. our main purpose is to gain insights related to the identification of themes in developing health informatics solutions both for organisations and practitioners. our scope includes health (healthcare, clinical, non-clinical or public healthcare delivery atmospheres) informatics research covering literature reviews, theoretical research, practical problem solving research, case studies, and/or any other forms of qualitative, quantitative and mixed methods or approaches in the relevant field. we have followed a bottom-up approach for collecting sample articles that took a multi-disciplinary perspective and were published as a full research article at the elsevier database driven by science direct. the search was performed across science direct (figure 1) identifying papers between 2016 to till 15 nov 2018. simultaneously we searched “health informatics” and “healthcare analytics” to capture articles that have the keyword in title, abstract, or body of the paper in order to australasian journal of information systems miah, shen, lamp, kerr & gammack 2019, vol 23, research on health information systems emerging insights of health informatics research 5 satisfy prisma2 conditions. search strings defined subjectively (jamshidi, ghafari, ahmad, and pahl, 2012) were applied to the database, comprising the terms {“health” or “healthcare”} and {“informatics” or “informatics”}, aiming to return the greatest volume of relevant articles. after that we excluded articles not in english, published in abstract, mini-reviews, editorial, news and also book chapters, newspaper articles, unpublished thesis papers and non-scientific articles. initially, we assessed titles and applied the pre-defined inclusion and exclusion criteria to each articles, obtaining a set of potentially relevant sample articles. following this, the full text of each article was obtained and their contents were critically evaluated by team members manually reading each article. 1,388 freely available articles were then selected for initial sorting. after removing duplicates, and those not satisfying our research foci, we came up with 112 articles. finally, after coding we excluded another 39 papers inappropriate to addressing our research question resulting in 73 papers selected for review. figure 2: slr methodology for sample collection 2 prisma provides guidance on evidence-based minimum sets of items required for conducting systematic literature reviews. prisma targets healthcare meta-analyses, but can also be used as a basis for reporting review findings. science direct 1,388 results found articles review articles (140), research articles (1,248), encyclopaedia (8), book chapters (115), conference abstracts (133), book reviews (2), correspondence (24), data articles (3), discussion (44), editorials (48), errata (4), mini reviews (4); news (11), practice guidelines (3), short communications (40), other (177) filtering samples exclude unrelated articles by reviewing titles, abstracts, keyword & full-text, we came up with 112 articles elected for fully review and interpretation:73 articles identifying article with title related to research objective screening through title, abstract, keyword, full text to build a list of terms related to the design research screening unclassified sample is the article related to topic? adding the article for selected sample yes no have all articles examined? eliminate paper finish screening yes no 73 articles were processed for the data synthesis & theme analysis selecting only complete articles, eliminating duplications, and article not in english, irrelevant research topics/themes. excluding book, chapters, mini reviews,encyclopaedia, newspaper, and incomplete text available. australasian journal of information systems miah, shen, lamp, kerr & gammack 2019, vol 23, research on health information systems emerging insights of health informatics research 6 figure 3: descriptions of sampling sources 3.1 thematic analysis thematic analysis is a technique for classifying, distinguishing, analysing and summarizing themes from the available data. it systematizes and defines the data set from details (boyatzis, 1998). one of the advantages of using thematic analysis is that it provides a flexible approach to identifying themes and thus provides a more accessible and useful form of analysis (nowell, norris, white, & moules, 2017). in addition, thematic analysis allows researchers to generate unanticipated insights based on the themes produced. nowell et al. (2017) described thematic analysis as a useful method for summarizing large sets of data through a well-structured approach to handling information. therefore we decided to adopt the approach of thematic analysis by braun and clarke (2006) in which the six-step phases seemed helpful for analysing the sample articles. these steps are: 1) familiarisation with the data; 2) generating initial codes; 3) searching for themes; 4) reviewing themes; 5) defining and naming themes, and 6) producing the report. in our study, as a part of the step 1, we found 73 sample articles published in the last five years regarding various directions of health informatics. during step 2 of the analysis, we coded each selected article. as part of step 3, we identified key themes and generated some descriptions around the themes from the target topic area. as part of reviewing the themes in step 4, we classified those themes with some additional details collected from those research articles. in step 5, we named ten themes as being key, and reviewed each theme three times to determine the aspects that each theme contributed. finally in step 6, to produce a concise and logical analysis oriented towards future developments, we present ten key themes in table 1. years and samples 2019 (24) 2018 (581) 2017 (538) 2016 (474) article types . review articles (140) . research articles (1,248) journal names and no. of articles procedia computer science (128) international journal of medical informatics (102) journal of biomedical informatics (64) the lancet (31) future generation computer systems (28) journal of the academy of nutrition and dietetics (28) ifac-papers online (25) neuro computing (24) journal of clinical epidemiology (17) computers in biology and medicine (16) australasian journal of information systems miah, shen, lamp, kerr & gammack 2019, vol 23, research on health information systems emerging insights of health informatics research 7 4 findings we found that health informatics research has been extended in application areas such as pathology, medical imaging, public healthcare, community care, home care, nursing, medical practices, consumer healthcare, clinical and bioinformatics, and clinical research domains. the application of the latest mature technologies are established in these domains but there are other technologies which are still in an embryonic stage. this implies that theory development research is still in demand to guide future application development research. further research should also be encouraged to assist with the maturing of existing areas in the field. we have consolidated ten key themes that show paths for the further growth and progress of research within the heath informatics field. a major concern of our thematic analysis was to determine underlying technological developments and their target application areas to identify sub-fields for improvement. this would identify the relevance or future research in the field of health informatics. in our presentation of these themes we include a description of the identified themes based on perceived significance. the first three topics are big data, telemedicine & telecare, and digital healthcare service delivery. table 1 illustrates the details of our initial findings. theme summary of findings comments big data the review found highest number of articles were published on big data. for instance, data analytics frameworks for prediction analysis is common in examining a big health dataset. mahmud, iqbal and doctor (2016) described a big data analytics framework that is based on a cloud computing integrated to gis and smart devices for capturing and visualising big data. the research was supported with a predictive model for health-shocks, using data from 1000 households in rural accessible regions of pakistan. nair et al. (2018) described a big data enabled system that aims at developing a health status prediction approach that using big data processing focusing on applying a machine learning model. choudharya and bajaj (2015) introduced data analytics using data mining techniques to be implemented on a rapid mining tool, so that diseases can be detected. the authors used a classification technique based on cross validation and decision trees to detect records before seeking advice from the doctor. also, rallapalli et al. (2016) described the impacts of data processing and analysing big healthcare data generated in the cloud computing environment for better decision making. to analyse the large-scale data they source automatically collected data from various healthcare applications and devices. to achieve better performance and to solve the scalability issues it is important to have a distributed big data system for handling data diversities of a cloud environment. our thematic analysis revealed that big data issues in health informatics have been an important area that show promise for future knowledge growth in the field. different analytics solutions using cloud computing as a method of solution, is one of the active existing themes, while other types of studies such as analysing various contexts of big data and their impacts for designing technological solutions are still at embryonic stage in development. australasian journal of information systems miah, shen, lamp, kerr & gammack 2019, vol 23, research on health information systems emerging insights of health informatics research 8 theme summary of findings comments telemedicine, telecare and telematics the review found the second highest number of articles was published on telemedicine and telecare themes. for instance, faiola and holden (2017) addressed ways people are increasingly moving to healthy living medicine (hlm) to mitigate health threats. hlm-seeker citizens rapidly adopted mobile technologies that enable control of personal healthcare as well as for collaboration with clinicians or other healthcare agents in order to have effective healthy living practices. these types of technologies are helpful with interventions such as medication and surgery with possible further risk of adverse effects, if there are risks from complex diseases and poor health habits. another example study is that of lilholt et al. (2015) in which a telehealth system was developed and its usability was evaluated for danish telecare north. the study used domain experts and principles, and compliance with jakob nielsen’s ten usability heuristics for interaction design. our thematic analysis revealed that telemedicine and telecare topics in health informatics have been another significant topic area that show promises for future knowledge growth in the field. digital health the review found the third highest number of articles published in relation to broader digital health topic areas. alshagathrh et al. (2018) described the experience of building an implantable medical device registry in saudi arabia. the work specifically addressed the early experiences of the saudi food and drug authority in developing a data sharing model for the implementation of a medical device registry at different hospitals. miah, hasan, hasan and gammack (2017) described a mobile health solution for the purpose of rural citizens’ healthcare decision support and information dissemination and also for diabetics management support care (miah, hasan and gammack, 2016). the other study by qi et al. (2015) explored the current situation and issues related to the development of the public healthcare workforce system for disease control and prevention in china. after identifying issues through the use of a survey, the study described corresponding strategies to address these issues for the future. our thematic analysis revealed that broader digital health topics in health informatics have been another significant topic area that shows promise for future knowledge growth in the field. education the review found the fourth highest number of articles were published in relation to education in health information systems areas. for example, chonsilapawit and rungpragayphan (2016) described the assessment of thai hospital pharmacists’ skills and knowledge of informatics, and their training needs, in order to improve curricular and professional development. the most current pharmacy curricula in thailand barely address the principles and skills concerned with informatics, and thai pharmacists usually acquire computer literacy and informatics skills through personal-interest, training and selfstudy. another example study by deng et al. (2018) analysed the quality and quantity of research by chinese academics in medical informatics based on their articles published in international specialty journals in recent years and examined medical informatics research hotspots in china. our thematic analysis revealed that health informatics has become an important subject of teaching and learning which is being developed in many asian countries. there is a need for developing research in relevant pedagogies for better practices in health informatics education. australasian journal of information systems miah, shen, lamp, kerr & gammack 2019, vol 23, research on health information systems emerging insights of health informatics research 9 theme summary of findings comments smart living the review found a number of articles published in relation to smart living topics in health informatics literature. this areas addressed smart technologies including internet of things based interventions. for instance, sun et al. (2018) introduced a data sensor model for an obesity identification, analysis, and estimation in which user is regarded as an online social network ‘sensor’ that can provide valuable health information. da costa et al. (2018) developed an approach (enabled by the internet of things) for allowing data from different sources to be combined in order to better diagnose patient health status and identify possible anticipatory actions. then other example study by phan et al. (2017) described research problems with human behaviour prediction and suggested explanations of healthcare intervention systems in health social networks, proposing an ontology-based deep learning model. yoshiura et al. (2018) proposed a health observatory conceptual model based on the semantic web in order to assist in the design, development and implementation processes of a health observatory. this shows that smart systems have experienced initial hype and later maturity periods in the path to better adoption and acceptance, with a focus on realistic solutions with combination of emerging technologies to improve it enabled health care. ethics, privacy, security and other concerns the review found a number of articles published in relation to ethics, privacy and security topic areas in health informatics. for instance, valdez and ziefle (2018) described the importance and utility of privacy-preserving techniques related to sharing of personal health data and differential privacy, analysing different sharing scenarios depending on the recipient of the data, the benefit of sharing data, the type of data, and parameterized privacy. nguyen et al. (2017) described a survey study to determine the prevalence and predictors of access to the internet and use of the internet to search for health information. shin et al. (2017) presented a study that validated a model using data collected in a time-series study of consumer use of online his in a user oriented context. richardson et al. (2017) described further determinants of usability, evaluating types of additional information gained from specific usability testing, considered as a part of the development phase of an integrated clinical prediction rule. roman et al. (2016) described a literature review study to assess the prevalence of navigation-related topics within the ehr usability and safety. this shows that the majority of the studies did pay attention to the importance of issues in ethics, privacy and security. health informatics is a major field where these concerns are extensively investigated, aged care the review found a number of articles published in relation to age care in health informatics topic areas. for example, faiola, papautsky and isolawe (2018) described an m-health lifestyle management model that utilises mobile health technology as a means to engage the baby boomer generation consumers in ways that establish the ageing population’s self-care. queiros et al. (2017) described a synoptic overview of current research into mobile health applications for ageing. the study found that a considerable number of mobile health applications for the ageing population have been developed for example supporting the care of patients with chronic medical conditions, promoting healthy lifestyles, minimizing loneliness and social exclusion, and ensuring home safety. this shows that aged care has been supported with mobile health services to meet real needs. this shows that the research on improving lifestyle of ageing populations has been growing and there is a need for improving literature on ageing care and support for their self-management and independent support. australasian journal of information systems miah, shen, lamp, kerr & gammack 2019, vol 23, research on health information systems emerging insights of health informatics research 10 theme summary of findings comments clinical and decision support (cdss) aspects the review found a number of articles published in relation to clinical decision support (cdss) topic areas. for example, kocbek et al. (2016) described a text mining based cdss for detecting admissions marked as positive for several diseases: lung cancer, breast cancer, colon cancer, secondary malignant neoplasm of respiratory and digestive organs, multiple myeloma and malignant plasma cell neoplasms, pneumonia, and pulmonary embolisma. the study used support vector machine classifiers for analysing the data from a combination of sources of medical records. robinson et al. (2014) identified the importance of patient portals for patients’ interaction with healthcare organizations and to view information. the purposes are categorised as informational, medical, logistical, or social. da costa et al. (2018) developed an approach (enabled by the internet of things) for allowing data from different sources to be combined in order to better diagnose patient health status and identify possible anticipatory actions. the dss research is also rapidly growing in other health oriented fields such as for healthcare supply chain (miah, ahsan msimangira, 2013; miah, 2008) this shows that the majority of the cdss studies focused both on clinical and nonclinical context for the purpose of decision making. organisational issues the review found a number of articles published in relation to organisational issues as topic areas. for example, garmann-johnsen (2015) compared two european regions on their use of assistive technologies. miah, hasan and gammack (2016) developed a cloud based e-heath solution for government to help bridging the digital divide between rural and urban healthcare services, enabling provision of suitable medical diagnoses and treatment consultations in a developing country. other example study by carney and kong (2017) investigated a phenomena of the complex problem of his design. the study suggested a collective response that may offer intelligence for an effective knowledge ecosystem, enhanced health informatics capacities, knowledge exchange, big-data, and situational awareness demonstrating the complex problems for the health informatics professionals. another study by miah, hasan, hasan and gammack (2017) introduced a social media based cdss to provide appropriate end-user technology (e.g. open media) that generally design based on real users’ demand using information systems development method that contribute principled specifications of design knowledge. this shows that the majority of the organisational issues studies focused both on clinical and nonclinical context for the purpose of healthcare information system development and implementation. evaluation and impact analysis the review found a number of articles published in relation to evaluation and impact analysis in health informatics. for example, brown et al. (2017) described an evaluation of a new his called “ea&f” for primary care (the performance improvement plan generator; pingr). the system covers the aspects of clinical performance, patient lists, patient-level data, and recommended actions and for evaluation, the study used iterative approach involving multiple evaluation cycles at different stages. another example study by craig and kodate (2018) contributed to literature of socio-technical theory and its value and/or relevance to health information in ireland. the study focused on three factors that can impact on health information; those of policy, infrastructure and people (pip), examining how ireland compares with other countries in relation to these factors. this shows that the majority of the evaluation and impact analysis in health informatics focused both on clinical and nonclinical context for the purpose of his implementation. table 1: findings of the health informatics literature review, with comments on each aspect. australasian journal of information systems miah, shen, lamp, kerr & gammack 2019, vol 23, research on health information systems emerging insights of health informatics research 11 5 special issues and further extended avenues from the ten key themes, we raised a call for new research papers that promote health informatics research. our aim was to stimulate the field by encouraging more contemporary studies that are needed to fill knowledge gaps. in september 2017 we published this call for a special section in the australasian journal of information systems. in total, sixteen articles were received. four were declined as they were irrelevant to the special section’s area of interest. twelve articles were selected for peer review. after conducting three review cycles, only six articles were finally accepted for publication in the special section. a key similarity of these articles’ focus is that each study examined effective health informatics research for organisations and stakeholders. the details of the published articles are given below. article 1: tele-monitoring technology as a tool for monitoring and management of patients with congestive heart failure telehealth interventions are designed to facilitate the remote exchange of information and data between patients and health care providers, improving the quality and safety of the patients and increasing efficiency and cost-effectiveness of health care providers. the development of telecommunications and virtual technology allowed a number of telehealth systems to be applied in different health care areas. these technologies can provide an alternative monitoring and solutions for decreasing the hospital readmission rates for patients with congestive heart failure (chf). in this paper, a systematic literature review regarding telemonitoring and its use in the management of patients with chf is conducted. the result indicated that standardized tele-monitoring design would reduce a length of hospitalization and re-hospitalization rate. the other factors that moderate the effectiveness of the telemonitoring intervention include quality of life, mortality rate, and disease-specific knowledge (health literacy). all these factors align with the consumer-centred principle of the australian safety and quality framework for health care. article 2: the experiences of three different user groups using personally controlled health record for multidisciplinary care team personally controlled health records (pchr) are patient-centric tools which allow individuals to own, manage, access and share their health information online from anywhere, at any time. patient-centric tools have received considerable interest and investment in recent years worldwide. these tools are thought to have the potential to increase individuals’ selfmanagement and involvement in their own health, as well as improve healthcare efficiency and delivery. the aim of this study is to report and analyse the experiences of three different user groups using pchr for multidisciplinary care team (mdct) including the advantages, disadvantages, barriers and obstacles, and the current state of pchr. in order to achieve the aim of this study, sixteen interviews with key informants from three different user groups were conducted in dunedin, new zealand. interviews were transcribed and analysed with thematic analysis. the key findings of this research showed that those who can benefit the most from pchrs are the least able to use it. it suits those who have basic knowledge about computers and the internet and those who can afford to use them. pchr is also best suited for individuals who are motivated about their health despite their health condition. however, more research is needed in the future with a larger sample, an easier to use pchr, different population other australasian journal of information systems miah, shen, lamp, kerr & gammack 2019, vol 23, research on health information systems emerging insights of health informatics research 12 than community health workers, patients with different chronic illnesses, and healthy patients. this research can be used as a basis and tested in future research on pchr adoption. article 3: the impact of telehealth technology on user perception of wellbeing and social functioning, and the implications for service providers the aim of the project was to evaluate the use of telehealth equipment in the homes of older community-dwelling people, and to review its social and economic impact. a mixed methods approach was adopted, involving interviews, observation and depression anxiety stress scales. overall, the greatest benefit was apparent in those participants with a low familiarity with technology and low digital literacy, where changes in behaviours to prevent an exacerbation of their condition was possible. the user interface design reduced concern about using the technology. changes achieved were through better compliance with medication and associated understanding of the impact on their vital signs and hence daily activities. this represented an improved health literacy and the economic benefits appear to be linked to that. less benefit was observed by those who had been self-monitoring previously. a greater focus on specific conditions and improved self-management could strengthen the evidence for targeted economic benefits. article 4: overcoming carer shortages with care robots: dynamic value trade-offs in run-time a rising elderly population and diminishing number of family and professional carers has led to calls for the intervention of care robots. this leaves the quality of robot-delivered care to be determined by designers, for profit companies, nursing codes of practice and conduct, potential user sample groups, etc. what is missing is the carer who consciously makes good ethical decisions during practice. good care is ‘determinative in practice’. that is, a carer can make good decisions because they are making them within the carer-patient relationship. if a robot is to be capable of good care ethics on the same level as humans, it needs to be conscious and able to make dynamic decisions in practice. moreover, a care robot must conduct patient interactions in appropriate ways, tailored to the person in its care, at run-time. this is because good care, as well as being determinative in practice, is tailored to the individual. the introduction of robotic care determined by limited stakeholders leaves customised care in danger and instead could potentially turn the quality of elderly care into ‘elderly management’. this study introduces a new care robot framework—the attentive framework— which suggests using care centred value sensitive design (ccvsd) for the design process, as well as a computationally conscious information system (is) to make practice-determinative decisions in run-time with extrinsic care value ordering. although vsd has been extensively researched in the is literature, ccvsd has not. the results of this study suggest that this new care robot framework, which is inspired by ccvsd, is competent in determining good, customised patient care at run-time. the contribution of this study is in its exploration of enduser willingness to trust known ai decisions and unwillingness to trust unknown ai decisions. moreover, this study signifies the importance of, and desire for, good, customised robot-delivered care. article 5: understanding the factors that influence the primary appraisal of mhealth tools in developing countries: an exploratory case-study in nigeria shortages of health workers, infrastructural deficiencies, limited access to medical care are just a few of the many barriers to care in developing countries. the integration of smartphones and mobile devices into healthcare systems has been proposed to address some of the physical australasian journal of information systems miah, shen, lamp, kerr & gammack 2019, vol 23, research on health information systems emerging insights of health informatics research 13 barriers to care and service delivery. these mhealth solutions extend the reach of medical care into rural areas of developing countries. however, it is not clear how mhealth solutions designed and tested in one developing region can be positively appraised for use in others. this study frames this problem using a coping theory approach based on an exploratory casestudy to understand the factors that influence primary appraisal of smartphone-enabled clinical guidelines (mhealth tool) for accessing, classifying and eliciting treatment recommendation for sick children under the age of five by rural healthcare workers (rhcws). findings identified a set of factors which are bound as an emerging explanatory positivity model that influence primary appraisal of an mhealth tool in a new context. these factors are the set of individual and social factors that governments, funding bodies and nongovernmental organisations should consider before embarking on the introduction of an mhealth tool in rural communities of developing countries. it is envisaged that by understanding the factors that influence primary appraisal, that is, either as an opportunity or a threat, practitioners and organisations will support positive appraisal and minimise the occurrence of negative ones when introducing mhealth tools. these findings have implications for theory, practice, and future research as explained in the concluding section of this paper. article 6: a design construct of developing approaches to measure mental health conditions mental health is an important determinant of communities’ well-being, influenced not only by individual attributes, but also by social and organisational environments in which people work and live. despite studies examining mental health status among specific populations, few attempts are evident that focus on solution designs for detecting and measuring impact of mental health conditions. in this study, we develop a construct utilising design science research principles for outlining common vocabulary around the problem, and solution design relevant to a mental health management system. for the case of it professionals, the developed construct is informed through a social-media based dataset containing more than 65,000 cells and 100 attributes potentially identifying influencing factors. machine learning techniques are applied to the dataset to discover new findings for this specific group. it is anticipated that the analysis reported in this study would contribute in developing other electronic health management systems both for communities and healthcare professionals. 6 overall discussion in this study, we conduct a systematic review of the literature focusing on determining the key themes of health informatics research. in order to answer the research question: what are the current research themes in the existing health informatics studies that show promise for future knowledge growth? thematic analysis was the functional choice to explore the major themes in the existing health informatics literature. this analysis method followed an inductive approach for outlining key themes from initial findings from the sample articles. after following the six steps of thematic analysis including re-reading, re-coding, re-focusing, and refining the content of each paper, ten main themes have been identified that were described in table 1. this type of research may offer benefits within the field as it offers an indication for future research growth. new or early career researchers can find in this research a landscape for building a foundation of their own study within the health informatics field. research in the health information domain is not a completely new field, but in this study we reinforce a need australasian journal of information systems miah, shen, lamp, kerr & gammack 2019, vol 23, research on health information systems emerging insights of health informatics research 14 for substantial exploration and improvement in various emerging sub-fields of the health informatics field, that may provide an insight which motivates researchers and practitioners in the field, by suggesting new research areas on innovative health systems, and their users and the context of their use (such as the key components of technologies, people and organisations). we consider that there is a need for more theoretical and applied research to contribute to the maturity of the health informatics field, which is a significant subset of information systems research. references alshagathrh, f., khanb, s.a., alothmany, n., al-rawashdeh, n., househ, m. (2015). building a cloud-based data sharing model for the saudi national registry for implantable medical devices: results of a readiness assessment, international journal of medical informatics 118, 113–119 al-shorbaji, n. (2001). health and medical informatics: technical paper who cairo, 2001, cited in hussein, s. and khalifa, a. (2011).biomedical and health informatics education and research at the information technology institute in egypt, yearb med inform, 20(01): 161-168 boyatzis, r. e. (1998). transforming qualitative information: thematic analysis and code development: sage. braun and clarke (2006) braun, v., & clarke, v. (2006). using thematic analysis in psychology. qualitative research in psychology, 3(2), 77-101. brereton, p., kitchenham, b., budgen, d., turner, m., and khalil, m. (2007). lessons from applying the systematic literature review process within the software engineering domain, journal of systems and software, 80(4), 571–583. brown, b., balatsoukas, p., williams, r., sperrin, m., and buchan, i. (2017). interface design recommendations for computerised clinical auditand feedback: hybrid usability evidence from a research-led system, international journal of medical informatics 94,191–206 chonsilapawit, t. and rungpragayphan, s. (2016). skills and knowledge of informatics, and training needs of hospital pharmacists in thailand: a self-assessment survey, international journal of medical informatics, 94, 255–262 choudharya, k. and bajaj, p. (2015). automated prediction of rct (root canal treatment) using data mining techniques: ict in health care, procedia computer science 46, 682 – 688 craig, s. and kodate, n. (2018). understanding the state of health information in ireland: a qualitative study using a socio-technical approach, international journal of medical informatics, 114, 1–5 da costa, c.a., pasluost, c.f., eskofier, b., da silva, d.b., and righia, r.d.r. (2018). internet of health things: toward intelligent vital signs monitoring in hospital wards, artificial intelligence in medicine, 89, 61–69 deng, h., wang, j., liu, x., liu, b., and lei, j. (2018). evaluating the outcomes of medical informatics development as a discipline in china: a publication perspective, computer methods and programs in biomedicine, 164, 75–85. australasian journal of information systems miah, shen, lamp, kerr & gammack 2019, vol 23, research on health information systems emerging insights of health informatics research 15 european commission (2004). ehealth making healthcare better for european citizens: an action plan for a european ehealth area, 2004 (available at http: europa.eu.int/eurlex/pri/en/dpi/cnc/doc/2004/com2004_0356en01.doc, accessed on 12 jun, 2018) faiola, p., and holdenb, r.j. (2018). consumer health informatics: empowering healthyliving-seekers through mhealth, progressincardiovascular diseases, 5 9, 4 7 9 – 4 8 6 fung, n. l. s., jones, v. m., bults, r. g. a., and hermens, h. j., (2014). guideline-based decision support for the mobile patient incorporating data streams from a body sensor network, in wireless mobile communication and healthcare (mobihealth), 2014 eai 4th international conference on, 2014, 312–315. gardner, r. m., overhage, j. m., steen, e. b., munger, b. s., holmes, j. h., williamson, j. j., & detmer, d. e. (2009). core content for the subspecialty of clinical informatics. journal of the american medical informatics association, 16(2), 153-157. garmann-johnsen, n. f., and roar eikebrokk, t. (2017). dynamic capabilities in e-health innovation: implications for policies. health policy and technology 6 (3): 292–301. hunt, d. l., haynes,r.b., hanna, s.e., and smith, k. (1998). effects of computer-based clinical decision support systems on physician performance and patient outcomes: a systematic review, jama, 280 (15), 1998,1339–1346. ikram, r.r.r., ghani, m.k.a., and abdullah, n. (2015) an analysis of application of health informatics in traditional medicine: a review of four traditional medicine systems, international journal of medical informatics 84(11), 988-996 jamshidi, p., ghafari, m..ahmad, a., pahl, c. (2012). a protocol for systematic literature review on architecture-centric software evolution research, technical report, lero the irish software engineering research centre, dublin city university, oct. 2012. karim, s. and bajwa, i. s. (2011). clinical decision support system based virtual telemedicine, in intelligent human-machine systems and cybernetics (ihmsc), 2011 international conference on, 1, 16–21. kocbek, s., cavedon, l., martinez, d., bain, c., manus, c.m., haffari, g., zukerman, i., and verspoor, k. (2016). text mining electronic hospital records to automatically classify admissions against disease: measuring the impact of linking data sources, journal of biomedical informatics, 64, 158–167 kuntagod, n. and mukherjee, c. (2011). mobile decision support system for outreach health worker, in e-health networking applications and services (healthcom), 13th ieee international conference on, 2011, 56–59. lilholt , p.h., jensena,m.h., hejlesen, o.k. (2015). heuristic evaluation of a telehealth system from the danish telecare north trial, international journal of medical informatics, 84, 319–326 mahmud, s., and iqbal, r., and doctor, f. (2016). cloud enabled data analytics and visualization framework for health-shocks prediction, future generation computer systems, 65, 169–181 australasian journal of information systems miah, shen, lamp, kerr & gammack 2019, vol 23, research on health information systems emerging insights of health informatics research 16 martínez-pérez, b., de la torre-díez, i., lópez-coronado, m., sainz-de-abajo, b., robles, m., and garcía-gómez, j.m. (2014). mobile clinical decision support systems and applications: a literature and commercial review. j med syst. 38(1), 4 miah, s.j., hasan, j., and gammack, j. g. (2016). on-cloud healthcare clinic: an e-health consultancy approach for remote communities in a developing country, telematics and informatics, 34 (1), 311-322 miah, s.j., hasan, n., hasan, r. and gammack, j.g. (2017). healthcare support for underserved communities using a mobile social media platform, information systems, 66, 1-12 miah, sj, ahsan, k. and msimangira, kab. (2013). an approach of purchasing decision support in healthcare supply chain management, operations and supply chain management 6 (2), 43-53 miah, sj. (2008). an ontology based design environment for rural decision support, unpublished phd thesis, griffith university, qld, australia miah, sj., gammack, j., hasan, n., hasan, h. (2017). a decision support framework for public healthcare: an approach to follow-up support service, australasian conference on information systems, dec 3-5, 2017, hobart, australia, available at: https://www.acis2017.org/wpcontent/uploads/2017/11/acis2017_paper_218_full.pdf, accessed on 23 july 2018. mosa, a. s. m., yoo, i., and sheets, l. (2012). a systematic review of healthcare applications for smartphones, bmc med information decision making, 12 (67), 2-31 nadri, h., rahimi b., timpka t., sedghi s., (2017). the top 100 articles in the medical informatics: a bibliometric analysis. j med syst, 41 (10) 150 nair, l.r., shetty, s. d., and shetty, s. d. (2018). applying spark based machine learning model on streaming big data for health status prediction, computers and electrical engineering, 65, 393–399 nguyen, a., mosadeghi, s., and almarioa, c.v. (2017). persistent digital divide in access to and use of the internet as a resource for health information: results from a california population-based study, international journal of medical informatics, 103, 49–54 nowell, l. s., norris, j. m., white, d. e., & moules, n. j. (2017). thematic analysis: striving to meet the trustworthiness criteria. international journal of qualitative methods, 16(1), 1609406917733847. o'donoghue, j. and herbert, j. (2012). data management within mhealth environments: patient sensors, mobile devices, and databases, journal of data and information quality (jdiq). 4 (1): 5 parveen, d. (2014). smarthealth india: development and field evaluation of a mobile clinical decision support system for cardiovascular diseases in rural india, jmir mhealth uhealth, 2, (4), 2014, e54. phan, n., dou, d., wang, h., kil, d., and piniewski, b. (2017). ontology-based deep learning for human behaviour prediction with explanations in health social networks, information sciences, 384, 298–313 australasian journal of information systems miah, shen, lamp, kerr & gammack 2019, vol 23, research on health information systems emerging insights of health informatics research 17 queirósa, a., cerqueira, m., santos, m., and rocha, p. (2017). mobile health to support ageing in place: a synoptic overview, procedia computer science, 121, 206-211 rallapalli, s., gondkar, rr, ketavarapu, u. p. k. (2016). impact of processing and analyzing healthcare big data on cloud computing environment by implementing hadoop cluster, procedia computer science, 85, 16 – 22 ramesh, m. v., anu, t. a., and thirugnanam, h. (2012), an intelligent decision support system for enhancing an m-health application, in 2012 ninth international conference on wireless and optical communications networks (wocn), 2012, 1–5. richardson, s., mishurisb, r., o’connella, a., feldstein, d., hessd, r., smith, p., mccullagh, l., mcginn, t., and mann, d. (2017). “think aloud”and“near live”usability testing of two complex clinical decision support tools, international journal of medical informatics, 106, 1-8 robinson, j. r., valentine, a., carney, c., fabbri, d., and jackson, g.p. (2014). complexity of medical decision-making in care provided by surgeons through patient portals, journal of surgical research, 2 1 4, 93-101 roman, l.c., ancker, j.s., johnson, s.b., and senathirajah, y. (2017) navigation in the electronic health record: a review of the safety and usability literature, journal of biomedical informatics 67, 69–79 shin, d.h., lee, s., and hwang, y. (2017). how do credibility and utility play in the user experience of health informatics services?, computers in human behavior, 67, 292-302 sun, q., wanga, s.n., lib, s., and zhoua, h. (2018) local spatial obesity analysis and estimation using online social network sensors, journal of biomedical informatics 83,54– 62 vaishnavi, v., and kuechler, w. (2008). design science research methods and patterns: innovating information and communication technology. auerbach publications, boston, ma. valdez, a.c. and ziefle, m. (2018). the users ’perspective on the privacy-utility trade-offs in health recommender systems, international journal of human-computer studies, 000 (2018) 1–14 wang, x., turner, l., and scott, re (2003). technology solutions – saving the day?, in the proceedings of the 6th annual can soc of telehealth conference, halifax. nova scotia 5-7 oct 2003 webster, j. and watson, r.t. (2002). analysing the past to prepare for the future: writing a literature review, management information systems quarterly, 26(2), xiii-xxiii. yoshiura, v. t., yamada, d.b., pellison, f.c., de lima, i.b., damiand, i.p. m., …et al. (2018). towards a health observatory conceptual model based on the semantic web, procedia computer science, 138, 131-136 copyright: © 2019 miah, shen, lamp, kerr & gammack. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems miah, shen, lamp, kerr & gammack 2019, vol 23, research on health information systems emerging insights of health informatics research 18 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 background of the research field 3 study methodology 3.1 thematic analysis 4 findings 5 special issues and further extended avenues 6 overall discussion paper title (paper title style) australasian journal of information systems hasan, connery & crawford 2018, vol 22, research article challenges of is research translations 1 challenges of is research translation: a study of tensions between ict innovation and conservative bureaucracies helen hasan university of wollongong hasan@uow.edu.au andrew connery university of wollongong kate crawford university of wollongong abstract in highly-structured public bureaucracies, such as universities, the need for stable and ordered administration conflicts with attempts at innovation and creativity. this is a particular dilemma for universities, which exist to create and share knowledge. the case presented here concerns the tensions generated where a university’s administration ignored innovative is research findings which could translate into innovations from which they could benefit. we employ leximancer to objectively analyse textual documents related to the case and invoke activity theory to make deeper sense of the phenomenon of our investigation. identifying several contractions behind the tensions uncovered in this case, we turn to the concept of dialectic as a tool for cooperatively resolving complex contradictions. the findings concern many cultural-historical factors that present barriers to the translation of is research into innovative practice in university administration. 1 introduction this paper is concerned with tensions associated with information technologies (it) innovation in conservative bureaucratic organizations, in particular universities. we posit this issue within the broader, but less studied, topic of research translation where the findings of information systems (is) research fail to translate to innovative use of ict within organizations (nielsen et al 2014). as it continue to evolve, new innovative is are developed. in many organizations, these systems rapidly become commonplace, transforming the structure, processes and culture of the organizations where they are implemented (serafeimidis & smithson 2003; van duivenboden & thaens 2007). rigid bureaucratic organizations can be averse to it-driven change and are slow to embrace new is. although, technology adoption has been an ongoing topic of is research, the findings in the area of is-driven innovation have had little impact on practice (e.g., jordan 1994; adler 2012). indeed, there are well-recognized barriers and challenges in translating the findings of is research into practical use in organizations (nielsen et al 2014; booker et al 2013). in order to improve our understanding of these barriers and challenges, we present the analysis of a case where they are evident. universities are in the business of creating and disseminating knowledge so that, among all bureaucracies, their managers should embrace innovation and have access to the latest is research findings, particularly those generated by their own academic staff and students. what australasian journal of information systems hasan, connery & crawford 2018, vol 22, research article challenges of is research translations 2 is more, the literature reveals that many is scholars choose their own universities as the site for their research (altbach 2014). surprisingly, we found no reports where the findings of such internal research have been used sustainably by the university where the research was conducted. moreover, among the literature on the changing landscape of university administration (deem 2004; whitchurch 2006; kevar & eckel 2000; ferlie et al. 2008; krishnaveni & meenakumari 2010), we found no indication that such change is informed by results of is research undertaken within that university. this paper reports the findings of an investigation into the tensions between the activities of bureaucratic administrators and those of researchers into ict innovations in the context of universities. using a specific case, we investigate the dialectic relationships between (a) technology and organization and (b) innovative research and bureaucratic stability. at a time when search engine optimization (seo) was a relatively new concept, our research aimed to evaluate the benefits of adding seo to the university’s website in order to improve the effectiveness of marketing activities. when the research began, the university’s marketers did not appreciate the potential of seo to revolutionize the marketing function by driving traffic to the website. their established practice relied on an attention-grabbing and professionallooking website design to reach prospective customers (i.e. prospective students). they strongly resented the disruption to this practice in the form of seo techniques suggested by the researcher, particularly as he was both a phd student and the owner of a business which had a long history of collaboration with the university. in our case, tensions arose from (a) the inability of university managers to appreciate and learn from the expertise of the seo researcher and (b) the frustration experienced by the researcher when his phd proposal, formally approved by an academic committee, was subsequently blocked by an administrative decision. to collect data, the approved research design required a single link to be added to the community engagement page of the university’s website, just as his business had in the past. as this link was core to the data collection, the refusal caused the project to be abandoned and initiated a subsequent dispute which had negative consequences for his business. for our investigation of the case, relevant confidential documents on the dispute between the university administration and the researcher were analysed using an impartial content analysis tool, leximancer (smith 2003). the language and interpretive lens of culturalhistorical activity theory (chat) (leontiev 1981) was used to interpret the concept maps produced by this tool. a chat model is employed to present the findings of the investigation to reveal dialectic contradictions and tensions between the activities of the case. the research question we investigate is “how can we reconcile the need for stability that comes with bureaucratic organizational processes with the need for innovation in organizational practices that comes with new technologies?” in seeking answers to this question, we aim to provide new insights into the challenges of translating the findings of is research into changes in organizational practice. the paper begins with a review of the literature relevant to the topic of our investigation and an examination of literature on those aspects of chat/activity theory, which justify its adoption for this study. this supports our assumption that the dialectic relationships between the activities identified in the study are complex and create tensions that can only be understood in terms of their cultural and historical contexts. australasian journal of information systems hasan, connery & crawford 2018, vol 22, research article challenges of is research translations 3 we then provide some background to the case and describe the data collection, as well as how the data was analysed before presenting the results of this analysis and its interpretation through the cultural-historical lens of activity theory. finally, we discuss the findings and their contribution to both the topic of the paper and the translation of is research to practice. 2 literature review our review of the literature begins with some background to our research in an examination of the role of ict in bureaucracies and then specifically in the context of university administration (ferlie et al 2008; senchez et al. 2007). this leads to an appraisal of current tensions that come from innovations in business practice involving new technologies in organizations (allen et al. 2001; van duivenboden & thaens 2007). recognising some of the contradictions behind these tensions we turn to the concept of dialectic as applied in chat as a tool for cooperatively resolving complex contradictions (trusson et al. 2014; collinson d 2014; langemeyer & roth 2006). in the is literature “studies have only paid scant attention to how field dynamics and organizational processes coevolve during information technology institutionalization” (nielsen et al 2014 p165). we welcome attention to dynamics and processes and our theoretical lens, chat, is consistent with this as it takes activity as the unit of analysis. we then describe how chat is employed to make sense of the phenomenon of our investigation where the unit of analysis, activity, is the dialectic relationship between subject and object mediated by physical and psychological tools in complex contexts (kaptelinin 2005). finally we provide a brief overview of ideas on knowledge and research translation as the broader concept within which we position our contribution. 2.1 bureaucracies and technology the growth in the complexity of the administrative tasks that came with the industrial revolution was a pre-condition for the emergence of bureaucracies (weber1922), which continue to be the most efficient and rational way of organizing (weber 1922, horton 2006, ader 2012). kallinikos (2004 p 13) portrays bureaucracy as a way of organizing “whose productive power resides in the way its formalized principles render organizational functioning independent of the personal qualities of the incumbents”. recent critical analysis reveals a decade-long “standoff between those who celebrated bureaucracy’s technical advantages and those who critiqued its human consequences” (adler 2012 p 244). graeber (2015) suggests that, with advances in is technology, bureaucracies have become components in an efficient and fused global system disempowering many people and reducing flexibility and agility. adler (2012 p 246), however, argues that bureaucracy is “simultaneously an enabling tool for organizing large-scale cooperation and a coercive weapon for exploitation”. these two functions coexist in a kind of paradox where organizations learn through both exploration for new knowledge and exploitation of existing knowledge (march 1991). in this paradox, new tensions are created between routine activities embedded in technical systems and creative activities carried out by smart people (collinson 2014; trusson et al.2014). this is a particular issue for institutions, such as universities, that have traditionally supported innovators and knowledge makers (deem 2004; krishnaveni & meenakumari 2010). ict has traditionally played a critical role in the exploitation of organizational data as developments in technologies over the years have enabled efficiencies through automation of australasian journal of information systems hasan, connery & crawford 2018, vol 22, research article challenges of is research translations 4 business operations. information systems have increased administrative efficiency and strengthened the administrative links within and between organizations (gallivan & depledge 2003). the use of ict to explore avenues for innovation has been more contentious. in a case study of efforts to effectively harness ict as an enabling force to meet the challenges of a digital age in government, allen et al. (2001) found that the necessary transformation, governance and accountability are likely to be blocked by an antiquated administrative culture that may be ill suited for a digital world. moreover, a review by murphy and rodriguezmanzanares (2008) of studies on contradictions in technology use in educational settings show that most focused on administration and a limited number focus on systems that support educational processes. 2.2 university administration and innovation extant research into the challenges of modernizing university management (deem 2004; whitchurch 2006; kevar & eckel 2000; ferlie et al. 2008) reveals that the highly-structured, hierarchical nature of university administrations has persisted over many centuries. among the mintzberg (1979) classification of organizational types, jordan (1994) identifies health services, legal practices and educational institutions, such as universities, as professional bureaucracies in which the people actually doing the productive work are highly skilled and highly trained. in the case of universities, traditionally nuanced and negotiated forms of administration through human interaction have been largely replaced by highly efficient automated systems that reduce decision making through human-system interaction to binary requirements that apply without discretion (hashim et al. 2010). the adoption of operational ict has helped to reduce the complexity and enhance the overall efficiency of administration of higher education in those functional areas common to most organizations in both the public and private sectors (krishnaveni & meenakumari 2010). some more specialized university is support systems have been reported (see e.g. hashim et al. 2010) as trials in individual institutions but few can be considered real innovations. university supporting infrastructures have historically included a clear boundary between the ‘administrators’ and the ‘academics’ where the identities and responsibilities of administrative and academic staff are clearly distinguished (whitchurch 2006). despite moves towards devolved management (whitchurch 2006) and changes in expectations about the roles of academics in leadership and management roles (deem 2004), this distinction was still found. comparing universities to other key public services, ferlie et al. (2008) note the historical standalone conservative nature of the higher education sector, where universities contain a mix of professional and bureaucratic elements within strongly institutionalized disciplinary silos. trust, integrity, and reputation in the academic community, may influence the allocation of privileges within these systems, but are no longer factors in most day-to-day transactions gallivan & depledge 2003). kevar and eckel (2000 p 1) list the following drivers for change in usa universities: “financial pressure, growth in technology, changing faculty roles, public scrutiny, changing demographics, competing values and the rapid rate of change in the world”. given the new pressures and demands of the rapidly changing environment, universities recognize the need to adapt their organizational structures and incorporate management systems to run their affairs more efficiently (whitchurch 2006). the view of students as customers or consumers has elevated the importance of identifying the commercial aspects of universities with courses seen as ‘products’ and an emphasis on marketing those products to customers (svensson & australasian journal of information systems hasan, connery & crawford 2018, vol 22, research article challenges of is research translations 5 wood 2007). kevar and eckel (2000 p 1) observe that “the array of challenges that higher education faces is virtually unparalleled when compared to any other point in u.s history”. higher education institutions are thus “immersed in important transformation processes that aim to make them more competitive, dynamic and transparent” sanchez et al. (2007 p 2). at the same time, academic staff and graduate students are expected to conduct research that is increasingly more focused on utilitarian problems related to perceived national interest and to apply outcomes to innovation (altbach 2014). in short, many research activities have become more controlled and there is less trust placed in academic self-determination (desouza et al. 2006). universities have thus adopted increasingly complex missions, which impose changes to the roles of academics, professional administrators and managers (whitchurch 2006). the traditional, more collaborative and participative decision making by faculty is often obscured by administrative systems. a review of the socio-cultural history of universities, suggests that: the traditional decision-making model of collaboration and participation by faculty has been overridden by a top down model of decision making which is often driven by the board of regents or administrators (powell 2013, p1), the question then arises as to the potential for innovation in administration processes to be derived from the knowledge of the people within bureaucratic systems. the australian knowledge management standard (as 5037) advises that innovation can begin with an internal audit of explicit and tacit knowledge, the latter being that held by members of the organization. as for explicit knowledge, we assume that most universities, like that of our case, have resumes of all academic staff with records of their research projects and publications as well as those of their research students. some researchers in business and is schools would have knowledge and expertise on topics that would inform innovation in university administration and management. we are not aware of any examples where this research knowledge is applied to university administration practices. 2.3 tensions, contradictions and dialectics the tensions between the need for rule-based stability and the desire for creative innovation are not new in large organizations with complex structures and processes (collinson 2014). there are inherent tensions and contradictions between bureaucratic administration and the potential for innovation from internal knowledge (blackler 1993; trusson et al. 2014). jordan (1994) promotes the need for both stability and change in complex settings and recommends that ict implementations must take into account different types of organizational structures as they affect methods of coordination and means of production. when describing the complications in allocating resources between exploitation and exploration activities march (1995) notes that most traditional change management processes can justify exploitation more rapidly than exploration. this is because activities of exploitation are likely to become effective in the short run but self-destructive in the long run. applying these ideas to contemporary university contexts, collinson (2014) describes dichotomies, dialectics, and dilemmas between the routine tasks of managing university budgets and the societal responsibility of universities as guardians and creators of knowledge. his conclusion is that the dialectical perspective can surface important questions about organizational power relations, paradoxes and contradictions that are typically under-explored within mainstream leadership studies. australasian journal of information systems hasan, connery & crawford 2018, vol 22, research article challenges of is research translations 6 we base our use of the term ‘dialectic’ on the hegelian triad (mueller 1958), which provides a workable way forward for the interpretation of our results. in essence, this says that truth, understanding and resolution of complex issues are found neither in the thesis nor the antithesis, but in an emergent synthesis which reconciles the two. on this basis, collinson (2014) states that dialectical perspectives view opposites and fixed binary poles as intrinsically interrelated concepts. he illustrates this by re-framing the descartes’ cartesian mind/body dualism in dialectical terms and criticizes the scientific method which required the separation of the rational from the ‘emotional. similarly, gallivan and depledge (2003) recognize that trust and control form a dialectic relationship, where each construct should be considered only in relationship to the other. the optimal productive balance between the two poles needs to be resolved according to the demands of any collective or organizational activity. paradoxically, in human terms, too little trust leads to alienation and loss of control (gallivan et al. 2003). 2.4 the cultural historical activity theory over the last two or three decades, is scholars have been attracted to chat/activity theory as a lens through which to analyse case studies involving ict innovation (nardi 1996; hasan 1998; sawchuk & stetsenko 2008; postholm 2008). this is predominantly because the chat concept of tool mediation provides a framework for the analysis of the ways tools shape, and are shaped by, complex human activity (allen et al. 2013; wertsch 1991). according to blackler (1993 p 875), “activity is a highly appropriate concept for organization theory. it draws attention to the social origins of motives and helps to explain the overall coherency of different actions.” we justify our choice of an activity theoretical approach by citing extant literature comparing it with actor network theory (miettinen 1999), critical realism (allen et al. 2013), and dewey’s sociocultural theory (postholm 2008). these authors demonstrate that chat tends to “emphasize the complexity and richness of the evaluation problem situation (or context) [and not allow methodologies which] tend to over-simplify the process through ‘cook-books’ that focus on the more measurable aspects of the outcome of is” (serafeimidis & smithson 2003 p 252). this allows for a diversity of subjective interpretations of the purpose of activities among different stakeholders (kaptellinin 2005; sestenko 2005). more recently, with the growth of social media and more complex systems, activity theory has also been used to augment complexity theory as an explanatory tool of complex phenomena in is research (hasan et al. 2010). because activity theory evolved within the russian culturalhistorical psychological tradition (sawchuk & stetsenko 2008; postholm 2008; wertsch 1991) it compels us to understand the historical and cultural context of our study (cole 1996; nardi 1996). this permits a holistic and contextual approach to data analysis and the presentation of findings (crawford & hasan 2006). this is particularly important in contested areas of rapid social and technological change (desouza et al. 2006). chat takes a holistic dialectic position which can allow a dialectic approach to contradiction (blackler 1993; sawchuk & stetsenko 2008; miettinin 1999; & roth, 2006, nardi 1996; murphy & rodriguez-manzanares 2008). in chat based research, dialectic relationships are evident in both topic and method (langemeyer & roth 2006). chat adopts the dialectic materialist view of activity and consciousness as being dynamically interrelated (leontiev 1981) and where learning and activity co-evolve. the dialectic view is also applied to the relationship between subject and object in an activity system (miettinen 1999) and to the reciprocal australasian journal of information systems hasan, connery & crawford 2018, vol 22, research article challenges of is research translations 7 relationship in the mediation of activities by tools where tools (such as is and bureaucratic processes) enable activities and activities demand new designs of tools. as described by sawchuk and stetsenko (2008) and many others, three generations of chat are widely recognized: ideas associated with vygotsky (1978) as the first generation, those with leontiev (1981) as the second generation, and engeström (1987, 2000) as the third generation most commonly adopted in is research such as that of chen et al (2013). the routine application of engestrom’s (1987) activity system framework is suitable for positivist studies in is (e.g. chen et al. 2013) while there is a need to invoke richer activity theory concepts for the study of unordered complex problems. blackler (1993) notes that this is particularly so in complex organizations, which embrace a plurality of divergent interest groups who favour a range of goals and priorities. our research analyses the dynamics of the tensions between the ordered and generic world of bureaucracies, it systems, and large organizations and the chaotic phenomena associated with specific instances of creative technological innovation and research. this distinction signifies what in information systems are variously referred to as different worldviews, epistemological stances or paradigms that make it difficult to present situations and problems from one coherent perspective (kazlauskas & hasan 2009). in this paper, although we use some language of third generation chat (engeström 1987), we emphasize the holistic nature of activity as a unit of analysis and ignore the sub-activity elements depicted in the well-known engeström triangle. instead we return to the coredefining notion of activity as it evolved through the work of vygotsky (1978), leontiev (1981) and blackler (1991) to be the dialectic relationship between subject (the doer of the activity) and object (encapsulating the purpose) mediated by physical and psychological tools (sestenko 2004). most significantly, an activity must always be understood in the context of its cultural and historical environment (kaptelin in 1996). in order to understand the dialectic nature of tensions between activities we focus on the diverse subjective interpretations of the purpose of an activity by all stakeholders. figure 1 the hierarchical structure of activity (leontiev 1981) leontiev (1981) described activity as a holistic, high-level, usually collaborative, construct such as undertaking a work project or doing a phd. his model depicted in figure 1 emphasizes the place of motives and purpose in the dialectic subject-object relationship that defines an activity. as the smallest unit that preserves unity, activity sits at the top of leontiev’s hierarchy which is composed of goal-oriented actions and underlying operations that are only meaningful in terms of the purpose of the activity. 2.5 knowledge translation from research to practice in is the terms knowledge translation and research translation are widely used in the medical literature, referring to the assessment, review, and utilization of scientific research in healthaustralasian journal of information systems hasan, connery & crawford 2018, vol 22, research article challenges of is research translations 8 care. although the notion of relevance, usually balanced against the need for rigor, is widely discussed among is scholars, the terms, or even the concepts of, knowledge translation or research translation are not so often applied (potthoff et al 2017). these authors seek a means of provided evidence to confirm that knowledge generated by is research is actually transferred to practice. in her widely cited article on is theory, gregor (2006) proposes a theory of design and action stating that work of this type occupies an important place in is. in this context she points out the need to design systems that translate expert knowledge into actionable knowledge for nonexpert users of the system. pawlowski et al. (2004 p645) use the term knowledge brokering where “it professionals see themselves as facilitating the flow of knowledge about both it and business practices across the boundaries that separate work units within organizations”. however, neither cover the transfer of expert is knowledge published as articles in is journals and conference into organizational practice. although this paper only describes the lack of, and potential benefits of, greater knowledge translation of is research in universities to practice within that university, we suggest that great attention to the broader topic of is research translation is warranted. rather than lay the blame on practitioners, we suggest that researchers can be proactive in this endeavour. useful resources we have found are (a) a commentary by meyer (2010) describes a role of knowledge broker for researchers in the social science which aims to promote a rise in research translation; and (b) a guide to research translation available from the university of melbourne’s sustainable society institute.1 3 methodology the research reported in this paper investigates tensions within bureaucratic organizations following an interpretivist paradigm involving dialectic contradictions. the confidential nature of legal action associated with the case, restricted the modes of data collection and analysis for which we could get ethics approval. unable to conduct interviews, we used available documented texts as our data sources some of which are confidential but rich in content that was not publicly available. to maintain confidentiality, our data analysis is mediated by the content analysis tool, leximancer, which produced maps of themes and concepts without revealing the text of the data itself. as described below, these maps are then systematically re-interpreted by the subjects of the research activity, i.e. the authors, to produce outcomes in the form of findings and conclusions. as advocated by walsham (1995), an appropriate methodology for complex multifaceted phenomena is an in depth, interpretive study of a single case observed within its real-life context. preliminary examination of the case led us to the recognition of the significance of the research problem it exemplified and the challenges of conducting research into a problem such as this where the boundaries between phenomenon and context are not clearly evident (yin 2002). as a consequence, the generalizability of the findings is difficult to establish (lee & baskerville 2003). moreover, at the core of the case are the experiences of one of the authors whom we regard as an engaged scholar (van de ven 2007) as both a businessman and a phd student. as such he brought different perspectives and complementary insights to our understanding of the case and our interpretation of the research findings. we saw this as 1 http://sustainable.unimelb.edu.au/sites/default/files/docs/part_two_guide_to_research_translation.pdf australasian journal of information systems hasan, connery & crawford 2018, vol 22, research article challenges of is research translations 9 enabling “a pluralistic approach to knowledge co-production among scholars and practitioners” (van de ven 2007 p 4). following an initial interpretation of the leximancer outputs by the engaged scholar, the other authors then re-interpreted the outputs of his analysis through the chat lens, using activity as the unit of analysis. emphasising the holistic nature of activity as championed by wertsch (1991). kaptelinnin (2005) and stetsenko (2005), we concentrate on tensions between activities rather that those between internal elements of activity systems as portrayed in engeström’s (1987) activity system. in particular, we rely on activity theory’s dependence on dialectic relationships (miettinen 1999) and its cultural-historical appreciation of the context of activities (vygotsky 1978). 3.1 background to the case the case we investigated took place over several years and concerns the tensions generated when a university’s administration withdrew approved support to an is research project whose results could have greatly benefitted the marketing function of the university’s website. the phd student who proposed the project was the owner of a small local social business “your online community” (yoc) which had a long-standing partnership with the university with which it had received a national community innovation award. he had completed an is master’s degree by research which investigated the unique business model of yoc as the business became commercially viable through its engagement with the community and the university. in 2008 he submitted a proposal to undertake part-time phd research aiming to investigate the efficacy of a search engine optimization (seo) algorithm that he has created to boost the online presence of local businesses registering on yoc. as yoc developed, the student had gained specialist expertise in search engine optimization (seo). he created and patented an innovative seo technique and was accepted as a phd student in the school of computer science rather than is to investigate its efficacy. the phd project proposal successfully passed the prescribed approval process which, in his case, included an agreement to put a temporary link to yoc on the university’s website’s list of local community partners for his data collection. he believed that this agreement would be honoured as, several years before, his business had been in an award-winning community project with the university in which a similar link had been created. the high page ranking of the university website would ensure sufficient traffic to the yoc site to test the seo’s performance. after spending two years part-time on a literature review and research design, the student was ready to collect data on the expected changes in traffic to the yoc site. at this point, the university’s web management unit refused permission for the link saying using the website for student research would conflict with their normal practice and set a risky precedent. as we will show, conflicts were created that lasted for another two years before formal proceedings were initiated to resolve the case in a court of law. the ensuing legal dispute between the student and the university endured for five years three years of which the student was officially on leave from his phd study. the case was eventually resolved in the student’s favour but too late for him to continue the phd on the topic of seo as advances in the field overtook the novelty of his original work. with the passage of time during the dispute, what was initially a novel seo approach worthy of academic investigation was superseded by other technical and social developments in the area. thus, a final resolution of the case in the student’s favour was meaningless as far as continuing the original australasian journal of information systems hasan, connery & crawford 2018, vol 22, research article challenges of is research translations 10 phd research in computer science was concerned. the student was still keen to do a phd and discussed this with his former master’s project supervisor in the school of is. eventually, the negative experience of the student was replaced with the realization that his dispute with the university was an instance of a much more significant issue, namely the complex problem of permitting emergent innovation in bureaucratic organizations. he re-enrolled with a new topic, “overcoming barriers when introducing perceived disruptive innovation into rigid efficient systems” and conducted a literature review on this new topic. 3.2 the data the data used in this study was chosen to give as balanced a picture as possible of different activities relevant to the case. the set of documents representing the stated university website policy and procedure over the relevant period were available on the staff internet. the student had retained all correspondence relating to the phenomenon under investigation including email communication and documents from the legal case. these have been analysed as objectively as possible using the automatic content analysis software leximancer (http://info.leximancer.com/). the findings are expressed as content maps of themes and concepts to avoid any identification of individuals involved or misrepresentation of the detailed contents. the agreement with the university was to embargo the student’s thesis for one year which is common practice when there is any chance of it containing sensitive material. this embargo has now expired. we have, however, been informed by the research committee that use of direct quotes from the student’s documents and emails would not be advisable. however, we have included among the results some excerpts from the public university policy documents and a list of contradictions identified by the student. the sets of extensive textual data included: • relevant university documents which represented the organization’s official position (vision, strategic plan and web management policy) over a ten-year period • student appeal documentation in the form of affidavits presented at court as a formal record of the case from the student’s perspective the documents, listed in the appendix, were analysed using leximancer (version 4); a software application for performing conceptual analysis of text data where words are mapped onto a set of automatically derived concepts. as described below, the concepts are developed solely from the independent algorithm and the text analysed; thus avoiding researcher bias (hewett et al., 2009). leximancer automatically groups the concepts into themes which can be used to identify activities and provide evidence of relationships between the activities of stakeholder actors. concepts, themes and relationships in the maps generated by the leximancer analysis can then be interpreted using activity as the unit of analysis. 3.3 content analysis with leximancer leximancer is a content analysis tool which uses novel robust artificial intelligence algorithms to develop and identify concepts and the relationships between them in free-form text (smith & humphreys, 2006; martin & rice, 2007). while the scientific basis of leximancer is beyond the scope of this paper detail can be found from smith (2003) and martin and rice (2007). in summary, leximancer develops a co-occurrence matrix of concept frequency and the cooccurrence of data, and then applies a statistical algorithm to derive a two-dimensional australasian journal of information systems hasan, connery & crawford 2018, vol 22, research article challenges of is research translations 11 concept map. the set of concept tags visualized on an interactive concept map, provides an intuitive graphical document exploration environment for the user (smith, 2003). during its development, an extensive evaluation was performed on real documents to compare the output of leximancer with the coding of domain experts (smith, 2003). researchers in various disciplines have found leximancer a valuable tool for processing transcripts of interviews (e.g., huber et al., 2007); drawing out common themes among sets of academic articles (e.g., liu & maddux, 2008; linger et al., 2005); comparing sets of documents over time or between sites. due to the validity and quality of leximancer’s in-built analytics it has the distinct advantages are its usability and objectivity. large amounts of text can be processed quickly in a quantitative, unsupervised manner that may identify concepts that were not envisaged by the researchers. after the first pass, users can inspect the source of the concepts in the text, and remove, merge or add concepts where appropriate. another pass can then quickly regenerate the map. text is quickly re-classified using machine learning that removes much of the need to revise thesauri as the domain vocabulary evolves. the procedure followed when using leximancer as a tool for document analysis in this project was as follows: 1. an initial scan was done with default settings to identify the top 30 concepts 2. the researcher inspected the automatically generated concepts to delete obviously spurious concepts, merge any concepts where appropriate and add extra concepts from below the 30 cut-off in some documents that relate to our study (e.g. the concept ‘innovation’ was ranked below 30 but was added) 3. the automatic scan was then run to re-generate the concept maps, figures 1 to 3 in the appendix. 4. the automatically generated theme size was reduced where appropriate so that themes most closely corresponded to identifiable activities. thus, these maps are fundamentally generated by the inbuilt leximancer algorithm using boolean logic and were thus free of researcher bias. 3.4 identification and interpretation of activities and relationships each of the resulting leximancer concept maps, depicted in figures 2-4, were systematically inspected by the three authors following a cycles process where the themes size was varied to identify potential activities. the concepts and themes were investigated and used to provide a label of the activities the themes represented. the activities identified were re-examined with reference to concepts to identify the subjects (doers), the cultural-historical context, and objects (purpose) as subjectively interpreted by the subject of the activity. thus activities were defined by the core subject-object relationship (kaptelinin 2005) and related to purpose (leontiev 1981). the student engaged in the case did a first pass. his results were then re-interpreted by the student’s supervisor then by the third author who had not been involved in the case at all. the three authors then met to produce a final version of the results as presented here in tables 2-3 and continue to review all findings for a deeper sense-making process of the tensions between key activities within the phenomenon of the study. this sense-making process is described in the section headed tensions and dialectics. australasian journal of information systems hasan, connery & crawford 2018, vol 22, research article challenges of is research translations 12 4 results of the data analysis to set the context for the choice of documents for the leximancer content analysis and activity theoretical interpretation we summarize the initial stages of the student’s part-time candidature. after one year’s study involving an intensive literature review, his research proposal was accepted which included the intention to place a link for 6 weeks on the university’s website to the student’s community business. four formal annual progress reports, required by the university, did not raise any problems or issues until the last of these when the university refused to allow the link and it was suggested that the student take leave to resolve. the complete absence of negative responses indicated to him that the problem would be eventually resolved, probably by the intervention of the faculty dean. this explains his perseverance with his initial research plan despite the on-going non-provision of much needed resources by the university. the student was also assured by the stated university vision: “to be a leader in ideas and solutions, a community of campuses and partners where discovery, learning and technology connect to transform people and the world we live in”. when it eventually became apparent that the link would not be provided, the student made a case directly to the university website administration asking them to support his request for a link to be put on the website for 6 weeks. he argued that the finding of his research could assist the university in improving the effectiveness of their web-based marketing with his innovative technical solutions (in particular, improving the seo of the university website and thus its ability to attract potential students/customers with its marketing material). with no success at this level he escalated his appeal to more senior management. in making his case, he relied on the stated university vision mentioned above that promoted the university as a place of new ideas and innovation. he also referred to the innovation award his community business, yoc, had received from its a long-standing partnership with the university, arguing that, in its own right, yoc would qualify for a link on the university’s community engagement web page. however, permission for the link was still denied. the mounting tensions then led to the prolonged court case mentioned above. in this section we describe how we analysed data from the documents listed in the appendix to explore the tensions among the various activities at play. 4.1 leximancer analysis and activity theoretical interpretation we used our leximancer analysis of two sets of university documents to objectively identify the university’s stated position on activities related to the case both among senior management (via their strategic planning documents) and in the website administration unit (via the website management policy documents). we then undertook a leximancer analysis of affidavits supporting the student’s subsequent court case against the university and use this to identify and examine the activities germane to the case from the student’s research perspective. in the three leximancer concept maps presented in figures 2-4, the coloured bubbles in the maps are themes, with coloured labels, and the smaller text nodes are concepts with smaller black labels. closeness on the map means that the concepts exist in similar contexts. following each leximancer map, is a table which summarizes the themes in the map and re-interprets them as activities identifying the subjects (doers), the cultural-historical context, and objects australasian journal of information systems hasan, connery & crawford 2018, vol 22, research article challenges of is research translations 13 (purpose) as subjectively interpreted by the subject of the activity. each of these is now described and interpreted as activities using the systematic process described above. 4.1.1 activities of senior managers (refers to figure 2 and table 2 in the appendix) figure 2 shows the results of the leximancer analysis of the various strategic planning documents created by senior managers of the university over the period of the study. table 2 contains the corresponding interpretation as activities. this analysis provides insight into the senior management’s perception of its responsibilities which appears to be managing the university as a business. the theme uow around the concept of quality is next to core business appears to be the activity of educating students. thus, academic staff development is one of the university’s strategic activities as the quality of academic staff is an important resource for the activity of teaching students. the strategic plan also aims to foster engagement and intellectual activities in the (external) university community. the strategic plan also aims to engage and foster the (internal) university community, people (individuals), and publicize newsworthy academic successes. further inspection of these documents reveals that the dominant strategic focus is external. the most recent strategic plan reinforces this predominance with research priorities including “incentive schemes to reward high citations, academic discipline-specific excellence (i.e. a* publications) and increased engagement with industry”; “visibility and discoverability of researchers”; “attract and leverage external funding” but nothing to indicate that management has any interest in the intellectual or practical outputs of its own researchers. our activity theoretical analysis of the concept map in figure 2 translates the university’s strategic plan into a series of objects (shared purposes in chat terms) that shape staff behaviour (see table 2). these include: conduct the university as a business with students as clients; market quality education to students; gain recognition among other universities through bureaucratic reporting processes, acknowledge research activities as part of the university brand, use technologies as tools that mediate both administrative and academic activities; broadcast information and communicate with people, often in a marketing capacity. the activity of strategic planning is revealed in figure 2 to be isolated from these operational activities. this seems to indicate that planning activities are not well integrated with other activities but are designed for their own ends to gain peer recognition and repute within formal evaluation systems of the tertiary education industry. in this context, research and academic work are seen as an indirect benefit and are reported as a way to validate the traditional ‘brand’ of the university as a place where knowledge is created but not where this knowledge is valued or applied. 4.1.2 activities of website administrators (refers to figure 3 and table 3 in the appendix) in universities, administrators have their own bureaucratic context and there is a clear boundary between the world views of ‘administrators’ and the ‘academics’ (whitchurch 2006). within this university context, figure 3 and table 3 present an analysis of the activities of those who administer the university’s website based on their formal policy documents. this analysis reveals the ways in which the administrative system shapes and mediates the activities within the university. as the digital ‘front door’ to the university the website is designed to enable activity to market the university and shape its image. activities also include oversight of content, australasian journal of information systems hasan, connery & crawford 2018, vol 22, research article challenges of is research translations 14 content management, research web presence, intranet document and calendar management and information broadcasts for staff and students. further inspection of these documents show that each opening section entitled “purpose of the policy begins with the goals “primarily focus on the informational needs of all its external users” and “support and promote the university vision, identity, activities and achievements”. as with the strategic plan there is no evidence that this unit acknowledges a role in supporting research students other than having a strict policy on what is allowed on the site and who controls this. each document has a section on “inclusion of external links on the university’s public website” and one of the allowable criteria is “organizations that have an approved business/community relationship” that would apply to the link to yoc requested in this case. the management style, and its role in the organization, has a focus on risk management, integrity and compliance with formal rights and privileges about access and content management. it reflects a hierarchical administration with a focus on marketing and administration rather than innovation and research development. it is generally accepted that the governance of the website and it system operates separately from the rest of the university activity. one might speculate that this is a cultural heritage from early it systems that were perceived by most staff as technical systems that can only be managed by specialists. 4.1.3 activities of the research student (refers to figure 4 and table 4 in the appendix) we have now established the university’s cultural historical context from our analysis of its strategic plans in figure 2 and table 2 and revealed the activities of the website management unit from our interpretation of figure 3 and table 3. this is now related to our study of the student researcher’s case emanating from a simple request to put one link on the university’s website for six weeks as stated in the student’s research proposal, which was accepted by the school. the only documented information on the case from the university’s perspective are two emails and very brief annual student reports advising the student to take leave while the dispute was settled. these came from academic managers who supervised the student and who assumed that university administration would support their decision to allow the link. in contrast, affidavits presented in court provide a detailed record of the case from the student’s perspective. our analysis of these documents in figure 4 and table 4 reveal that key activities are, firstly, the student’s phd research and his business yoc (both activities where he is the subject), secondly, his research supervision (an activity where academics in the school of computer science were the subjects), and thirdly, the activities of his dealings with the website management unit. the following is a list of contradictions identified by the student in his thesis: • as a businessman he viewed his provisional patents as extremely valuable intellectual property whereas the university considered it was of no commercial significance or value to the university. • administration was not perceived as part of the student’s community of practice. • administration and research were governed by different rules –regulations v common law. • reliance on different mediating tools for record keeping– the university’s legacy student record system v the student’s email system. australasian journal of information systems hasan, connery & crawford 2018, vol 22, research article challenges of is research translations 15 • the student expected the university to align its practices with its much-publicized strategic goals, namely to engage with the local business community and conduct innovative research. the administration relied on unwritten rules and historical practice. it is clear that the student researcher is attempting to act across otherwise unintegrated sectors of the university organization, principally the web-management marketing administration unit and the school of computer science, an academic unit. in addition, the student has a different cultural historical experience of website management, technical innovation and community collaboration which came from his own business experience. he had reason to assume that a similar culture existed at the university based on his success in innovative collaboration with the university before he became a research student. these experiences shaped his subjective interpretations of the situation and the purposes (objects) of activities with the university. these included to: conduct negotiated and formally approved research project in fulfilment of university requirements using agreed technical support, negotiate with supervising academic staff for an approved and resourced research project and to resolve technical needs; collaborate technically as a part of the research process, and enable local businesses through applied research in collaboration with the university. 4.2 tensions and dialectics we propose that, at the heart of this case, there is a dialect tension between the pressure on university management to run an efficient business using secured web-based technology and the latent resource for innovation represented by the findings of academic and student research. we have uncovered an instance of tensions that emerged when relevant internal research findings of the student’s seo project were ignored when they could have been applied to improve university business, in this case the effectiveness of its web-based marketing at to potential customer with seo which was innovative at the time (2008-2010). in summary, we identify the principal sources of tensions in this case as: • the ways in which the ordered university administrative system and culture now differs from traditional ideas of research and development and also from the increasingly volatile and unordered community context that is already highly influenced by interactive social media • the ways the formal system allocates student rights and privileges at the university, as mediated in the website management, obscure the academic decision-making and undertakings that initially approved the project and related technical support. the website managers appear unaware of the importance of student research at the university. • more generally the ways the management of the university as an ordered business conflicts with more collaborative traditional notions of academic research and decision making about research, knowledge creation and innovative development that require interpersonal trust, experimentation and ingenuity. • the cultural history of collaborative community business development using internet technologies, which forms the basis of the student’s local business experience (yoc), was at odds with the emerging risk-averse (more controlling and less trusting) cultural history of website use and management and its primary australasian journal of information systems hasan, connery & crawford 2018, vol 22, research article challenges of is research translations 16 role at the university. this meant that the possible benefits of the research findings on seo for website marketing at the university were not recognized. • the disconnect between the university’s relationship with a local businessman engaging as a partner in an award-winning community project and with the same person as a phd student doing related research. these tensions can be viewed as symptoms of complex problems whose potential positive resolution we now discuss in reference to our findings and the literature reviewed above. 5 discussion our specific case study concerned the disinclination of a university administrative unit to realize the benefits of learning through the translation of findings from a relevant internal phd research project. the study illustrates the organizational challenges of reconciling the tension between the exploitation of ordered ict processes and systems, and the exploration of the innovation that comes with advances in technology-based systems and un-ordered research processes (march 1991; snowden 2002). we posit this issue within the broader challenges of knowledge translation where the findings of is research fail to stimulate innovative practices in organizations, even those where the research is conducted within the organization. in the introduction to this paper we posed the research question: “how can we reconcile the need for stability that comes with bureaucratic organizational processes with the need for innovation in organizational practices that comes with new technologies?” in answering this question, we gained new insights into the challenges of translating the findings of is research into changes to organizational practice. applying chat principles we discovered dialectic tensions between activities identified throughout the case. these tensions were associated with both the attributes and circumstances of the subjects of activities and the object/purpose as described in tables 2-4. they were thus embedded in the subject-object relationships at the core of these activities. for example, in the student’ phd research activity (table 4) tensions arose from his dual role as student and businessman and the overlap between the research/business purpose of his phd activity. the activity was mediated by the culturalhistorical context of conflicting administration and academic cultures, as well as the need for both organizational stability and innovation. to answer the research question, we investigated a case where the orderly management of a university website, embedded in a bureaucratic risk-averse culture, was disrupted by a reasonable research student request for a temporary link on its community website. in ignoring this opportunity to learn about seo, the marketing unit failed to capitalize on internal activities aligned with innovation and creative intellectual capacity, despite the fact that these attributes are associated with core university values as stated in its mission. there appears to be a blind spot in the hierarchical power structures of universities which devalues new knowledge and creativity by its own research students, who are not accorded much authority. indeed, it appears from the analysis of university policy documents that management has never considered that results of research undertaken at the university could be of value to their operations or administration. policies and practice of administrative units within the university, such as website management, appear to echo the university vision for innovation but set and enforce rigid rules for their area of responsibility. australasian journal of information systems hasan, connery & crawford 2018, vol 22, research article challenges of is research translations 17 the case reported in this paper represents an example of what happens in a complex organization when human activity crosses boundaries between separate and culturally unintegrated activities. the situation is exacerbated by the form, function and administration of it services that rather simplistically mediate the interactions between the various complex sectors of the organization. rigid rules for security and order dominate over the potential for internal innovation that is ignored. in a knowledge-creating context, such as a university, this is particularly disappointing. the university context brings out the stark contrast between bureaucratic order and the unordered nature of human creative research and innovation. although this contrast generates tensions, a dialectic view of this situation is that both can be accommodated and provide mutual support. blackler (1993) applies activity theory to observe that “the effective functioning of an organization does not depend on people agreeing on why they are doing something; all that is required is that there is agreement on procedures for determining what should be done” (blackler 1993 p 876). our activity theoretical analysis reveals that organizational structures and processes are tools that inhibit many from making innovative decision involving the translation of is research into practice. in theorizing the need for both stability and innovation, we draw on the ideas of (snowden 2002), who describes the importance of both ordered and unordered activities for human learning and knowledge management, and march (1991) who demonstrates that exploitation and exploration of knowledge are both critical to organizational learning. our use of chat takes these ideas a step further by relying on the notion of dialectic relationships (miettinen 1999) within their cultural-historical context (vygotsky 1978). in the 21st century the productivity gains of the rapid automation of ordered administrative activity through is needs to be balanced against the need for more flexible, creative and ingenious exploration, experimentation and innovation by people (miller et al 2006). when situations involve activities with conflicting subject-objects relationships (i.e. purposes), we are dealing with complex problems that can never be completely solved. following hegelian ‘dialectic’ principles of thesis-antithesis-synthesis may provide a sensible way forward. as described above, dialectic implies that the subjective perceptions of those involved (the subjects of the activities) may differ but still all be valid. resolution of conflicting purposes of related activities should involve a mechanism in which different objects of the activities can all be accommodated in a negotiated synthesis. in this case, the level of control by it administrative services could have been productively relaxed to allow the approved research in exchange for shared knowledge about the potential benefits of the results. our view is that the occasional complex cases of conflict between the rules of the administration unit and innovative research projects should be recognized as a potential opportunity and differences would be best resolved through an independent ombudsman. reconciling the need for both stability and innovation can be easily dismissed as simply a matter of inadequate communication. however, entrenched power relationships often stand in the way of shared activity. silos exist within organizations and there is often limited flow of information, and much competition, between them. however, promoting meaningful, interactive communication and activity across boundaries can resolve tensions and create opportunities for mutual benefits. in our case, we saw that the university website largely broadcasts only management approved content, thereby creating tensions and fostering complex problems when is it used to mediate collaborative relationships between people e.g. for community engagement and industry partnerships. as an example of a research australasian journal of information systems hasan, connery & crawford 2018, vol 22, research article challenges of is research translations 18 translation problem with conflicting requirements, our university has twice attempted to create a “find an expert” system on the website allowing external access to researchers with relevant expertise. neither has been successful in either maintaining an accurate (traditional) database of internal expertise or designing a serviceable interface for end-users to search this database using non-academic language. this is an organizational problem where is research findings scholars could be allowed, with some guidance, to promote themselves, (see e.g.. the work of chasin et al. (2013) where corporate social media, such as yammer, have also been successfully used in such situations and could mediate more cooperative collaborations across boundaries). is research is exceptional in that it conducts a large amount of research in real-world settings. most such is research is either, (1) commissioned, and paid for, by an organization to solve a specific local problem; or (2) initiated by researchers seeking a relevant site to study a phenomenon of interest to them. research by university scholars on aspects of their own organization is a special case of the latter. in the first situation, the main research translation barriers are confidentiality agreements or lack of incentives to convert lessons learned to a form relevant to other contexts. in the second situation, there are research translation barriers between the output valuable to the researcher (a journal paper on the general phenomenon) and the organization (findings that are understandable and relevant to their specific context). the special case where researchers use their own university as a research site afford the opportunity to train researchers as knowledge brokers with the skills to be proactive in translating research findings in their discipline into a form accessible by practitioners. 6 conclusion this paper has theoretical, practical and methodological contributions. as our theoretical basis, activity theory is applied as a framework for understanding the ways cultural historical lived experience shapes human behaviour and how new technologies transform lived experience where ict innovation is attempted within a conservative bureaucracy. the case provides a pertinent example of how the activity theoretical dependence on dialectics at their core can be useful in explaining of tensions between subjective interpretations of activities with conflicting objects of both personal and organizational activities. in this paper, we have followed respected third generation activity theorists from different disciplines (engeström 1987; blackler1993; wertsch, 1991; stestneko 2005; kaptelinnen 2005) who adopt activity as their unit of analysis in the tradition of vygotsky (1978) and leontiev (1981). moving towards the building of new theory, our single case provides an understanding of tensions arising in a university, with a stated vision is “to be a leader in ideas and solutions”, that was unable to recognize internal innovation and apply its own knowledge. following walsham (2006), we offer some evidence of generalizability of the findings from this single case. in the first instance, although the case involved the application of new seo techniques to a website, the resulting tensions could apply to other innovative research activities which challenge the standard or authorized use of any entrenched information system. secondly, we suggest that our research site is typical of most universities. it is certainly consistent with those in the literature, reviewed above, where the historical bureaucratic australasian journal of information systems hasan, connery & crawford 2018, vol 22, research article challenges of is research translations 19 hierarchies and siloed structures (ferlie et al. 2008) are currently overlaid with increased financial pressure, growth in technology, changing faculty roles, and public scrutiny driving change towards a more commercial orientation (kevar & eckel 2000). our findings confirm those of several authors (whitechurch 2006; murphy et al. 2008; hashim et al. 2010) that tensions, such as those we have identified, are bound to emerge when new commercial imperatives, from outside the institution, combine with the remnants of the traditional bureaucratic processes to create a culture that no longer fosters the innovation and new knowledge generation that have historically been the hallmarks of universities. we thus suggest that the findings from this case could apply to problems in many universities. thirdly, we suggest that research conducted by is scholars within universities using aspects of the university as their object of study, could provide an opportunity to learn ways to better understand the mechanisms of research translations and ways to train research students in these mechanisms. practically there are lessons from the results of this research for all bureaucracies. in a volatile era of global social and economic change new technologies are arising from the more pressing human needs for agile adaptation. we speculate that social media have emerged in response to the impact of the excessive efficiency of automated administrative systems and their alienating impact on people. they are in turn transforming society, including business management and administration (coiné & babbitt 2014, hooper & genoff, 2015), in ways that challenge traditions. the challenge will be to reconcile these continuing tensions and contradictions between efficient exploitation and creative exploration into a prosperous and sustainable collective future to achieve a working synthesis in the dialectic relationship between efficiencies of automated control and the trust between people that enables new knowledge making and innovative applications of knowledge in society methodologically we have demonstrated a novel approach to data analysis and interpretation in a case where one of the authors is a key actor. a combination of chat principles and leximancer content mapping was employed as an objective tool for the activity of analysis of documentary data. the themes automatically generated in the concept maps were used to identify and understand the key activities of a contentious situation. references as 5037 (2005) australian standard knowledge management, standards australia, sydney. adler p. (2012) the sociological ambivalence of bureaucracy: from weber via gouldner to marx, organization science 23(1), 244–266 allen b., juillet l., paquet g., & roy j. (2001) e-governance & government on-line in canada: partnerships, people & prospects, government information quarterly 18, 93–104 allen, d.k., brown, a., karanasios, s., & norman, a. (2013) how should technologymediated organizational change be explained? a comparison of the contributions of critical realism and activity theory, mis quarterly, 37(3), .835-854. altbach p. (2014) the emergence of a field: research and training in higher education studies in higher education, 39( 8) 1306–1320, blackler f. (1993) knowledge and the theory of organizations; organizations as activity systems and the reframing of management, journal of management studies 30(6), 863-884 australasian journal of information systems hasan, connery & crawford 2018, vol 22, research article challenges of is research translations 20 booker, l., bontis, n., burstein, f., linger, h., & serenko, a. (2013). understanding the practical relevance of academic research in knowledge management: a lay epistemic theory approach. proceeding of amcis 2013. cairns, h. (1942). plato's theory of law. harvard law review, 56(3), 359-387. chasin f, gal u. & riemer k. (2013) the genealogy of knowledge: introducing a tool and method for tracing the social construction of knowledge on wikipedia, proceedings of the 24th australasian conference on information systems acis 2013, melbourne, australia, chen, r. sharman, r. rao, h. & upadhyaya, s. (2013) data model development for fire related extreme events: an activity theory approach, mis quarterly 37(1), 125-147. coiné,t. & babbitt, m (2014) a world gone social: how companies must adapt to survive, unites states of america, amacom cole, m. (1996). cultural psychology: a once and future discipline. cambridge, ma: the belknap press of harvard university press. collinson d (2014) dichotomies, dialectics and dilemmas: new directions for critical leadership studies? leadership 10(1), 36–55 crawford, k. & hasan, h. (2006) demonstrations of the activity theory framework for research in is. australasian journal of information systems, 13(2), 49-68. deem r. (2004) the knowledge worker, the manager-academic and the contemporary uk university: new and old forms of public management? financial accountability and management, 20(2), 107-128. desouza, k., el sawy, o., galliers, r., loebbecke ,c. & watson r. (2006) beyond rigor and relevance towards responsibility and reverberation: information systems research that really matters communications of the association for information systems, 17(1), #16. engeström, y. (1987). learning by expanding: an activity theoretical approach to developmental research, helsinki: orienta-konsultit. engeström, y. (1999). perspectives on activity theory, cambridge: cambridge uni press engeström, y. (2000) activity theory as a framework for analyzing and redesigning work. ergonomics 43(7), 960-974. ferlie e., musselin c. & andresani g. (2008) the steering of higher education systems: a public management perspective, high education 56, 325–348. finegan, a. d. (2010). making sense of wicked projects, proceedings of the australian institute of project management conference: creating a future, darwin, australia. gallivan m. & depledge g. (2003) trust, control and the role of interorganizational systems in electronic partnerships, information systems journal 13(2), 159–190. graeber, d. (2015) the utopia of rules, london, melville house gregor, s. (2006). the nature of theory in information systems. mis quarterly, 611-642. hasan h. (1998) activity theory: a basis for the contextual study of information systems in organisations. in hasan h, gould e. and hyland p. (eds) information systems and activity theory: tools in context wollongong university press, 19-38. australasian journal of information systems hasan, connery & crawford 2018, vol 22, research article challenges of is research translations 21 hasan h. crawford k. (2003) codifying or enabling: the challenge of knowledge management systems, journal of the operations research society. 54, 184-193. hasan, h, kazlauskas, a. & crawford, k (2010) blending complexity and activity frameworks for a broader and deeper understanding of is. proceedings of the international conference on information systems, st louis, missouri dec 13-15th. hashim, f., alam, g. m., & siraj, s. (2010). information and communication technology for participatory based decision-making-e-management for administrative efficiency in higher education. international journal of physical sciences, 5(4), 383-392. hooper, n. and genoff, r. (2015) new men, new women, new economy, sydney, federation press horton s. (2006) new public management: its impact on public servant's identity: an introduction to this symposium, international journal of public sector management, 19(6), 533 – 542 huber n, michael k & mccathie l (2007) barriers to rfid adoption in the supply chain. rfid eurasia, 1st annual, 5-6 september, 1-6. jordan e. (1994) information strategy and organization structure, information systems journal 4(4), 253.-270. kallinikos, j. (2004) the social foundations of the bureaucratic order. organization 11(1) 13–36. kaptelinin, v. (2005) the object of activity: making sense of the sense-maker. mind, culture, and activity, 12(1), 4-18. kazlauskas a. hasan h. (2009) web 2.0 solutions to wicked climate change problems. 16/2 australasian journal of information systems 16/2 special issue on climate change http://journal.acs.org.au/index.php/ajis/article/view/587 kevar a., & eckel p.(2000) the effect of institutional culture on change strategies in higher education: universal principles or culturally responsive, concepts? research report 143 eric clearinghouse on higher education, george washington univ., washington dc. krishnaveni, r., & meenakumari, j. (2010). usage of ict for information administration in higher education institutions-a study. international journal of environmental science and development, 1(3), 282. langemeyer i. & roth w. (2006) is cultural-historical activity theory threatened to fall short of its own principles and possibilities as a dialectical social science, outlines 2, 20-42 lee, a.s. & baskerville, r.l. (2003) generalizing generalizability in information systems research, information systems research 14(3) 221-243 leontiev, a. n. (1981) problems of the development of mind, progress, moscow. linger h, burstein f & hasan h (2005) articulating knowledge work: the contributions of activity theory and task-based knowledge management. in activity as the focus of information systems research (whymark k g and hasan h, eds), pp 71-90, knowledge creation press, rockhampton, queensland, australia. australasian journal of information systems hasan, connery & crawford 2018, vol 22, research article challenges of is research translations 22 liu l & maddux cd (2008) web 2.0 articles: content analysis and a statistical model to predict recognition of the need for new instructional design strategies. computers in the schools 25(3-4), 314-328. march, j.g. (1991). exploration and exploitation in organizational learning. organization science, 2(1), 71-87. martin n.j. & rice j. (2007) profiling enterprise risks in large computer companies using the leximancer software tool. risk management 9(3), 188-206. meyer, m. (2010). the rise of the knowledge broker. science communication, 32(1), 118-127. miettinen (1999) the riddle of things: activity theory and actor-network theory as approaches to studying innovations, mind, culture, and activity, 6(3), 170-195, miller, k. d., zhao, m., & calantone, r. j. (2006). adding interpersonal learning and tacit knowledge to march's exploration-exploitation model. academy of management journal, 49(4), 709-722. mintzberg, h. (1979) the structuring of organizations: a synthesis of the research. prenticehall, englewood cliffs, nj. mueller g. (1958) the hegel legend of "thesis-antithesis-synthesis" journal of the history of ideas. 19(3), 411-414. murphy e. & rodriguez-manzanares m. (2008), using activity theory and its principle of contradictions to guide research in educational technology australasian journal of educational technology 24(4), 442-457 nardi b (1996) context and conciousness: activity theory and human-computer interaction. mit press nielsen, j., mathiassen, l., & newell, s. (2014). theorization and translation in information technology institutionalization: evidence from danish home care. mis quarterly, 38(1), 165-186. pawlowski, s. d., & robey, d. (2004). bridging user organizations: knowledge brokering and the work of information technology professionals. mis quarterly, 645-672. periyakoil, v. s. (2007). taming wicked problems in modern health care systems. journal of palliative medicine, 10(3), 658-659. postholm m (2008) cultural historical activity theory and dewey’s idea-based social constructivism: consequences for educational research, critical social studies 1, 27-48 potthoff, t., meske, c., & köffer, s. (2017). assessing the knowledge transfer of is research to practice, knowledge transfer of is research to practice, 23rd americas conference on information systems, boston, 2017 powell, d.j. (2013) reflections on faculty participation in university decision making, universoitas, 8 p1 sanchez c., elena s., castrillo r (2007) the icu report: an intellectual capital proposal for university strategic behaviour, imhe “what works” conference paris. australasian journal of information systems hasan, connery & crawford 2018, vol 22, research article challenges of is research translations 23 sawchuk p. stetsenko a. (2008) sociological understandings of conduct for a noncanonical activity theory: exploring intersections and complementarities. mind, culture, and activity, 15(4), 339-360. serafeimidis v. & smithson s. (2003) information systems evaluation as an organizational institution – experience from a case study, information systems journal 13, 251–274 smith a (2003) automatic extraction of semantic networks from text using leximancer. in proceedings of hlt-naacl, 23-24, edmunton. smith ae & humphreys ms (2006) evaluation of unsupervised semantic mapping of natural language with leximancer concept mapping. behavior research methods 38(2) 262-279. snowden d 2002. complex acts of knowing: paradox and descriptive self-awareness. journal of knowledge management 6(2), 100-111. stetsenko, a. (2005). activity as object-related: resolving the dichotomy of individual and collective planes of activity. mind, culture and activity, 12(1), 70–88. svensson, g. & wood, g. (2007), are university students really customers? when illusion may lead to delusion for all!, international journal of educational management, 12(1), 17-28. trusson, c. r., doherty, n. f., & hislop, d. (2014). knowledge sharing using it service management tools: conflicting discourses and incompatible practices. information systems journal, 24(4), 347-371. van de ven, h.a. (2007) engaged scholarship a guide for organizational and social research. oxford university press, new york, usa. van duivenboden h. & thaens m. (2007) ict-driven innovation and the culture of public administration: a contradiction in terms? proceedings of the egpa conference, september 2007, madrid, spain 19-22 vygotsky, l. s. (1978) mind in society: the development of higher psychological processes, harvard university, cambridge. walsham, g. (2006) doing interpretive research european journal of information systems 15(3), 320-330 walsham, g. (1995) interpretive case studies in is research: nature and method european journal of information systems 4(2), 74-81. weber, m. (1922) economy and society retrieved july 2014 from https://archive.org/details/maxweber weber, m. (1958) the protestant ethic and the spirit of capitalism. translated by t. parsons. charles scribner’s sons, new york wertsch, j.v. (1991). voices of the mind. a sociocultural approach to mediated action. cambridge, ma: harvard university press. weston t. (2012) marx on the dialectics of elliptical motion historical materialism 20(4), 3–38 whitchurch, c. (2006). who do they think they are? the changing identities of professional administrators and managers in uk higher education. journal of higher education policy and management. 28(2): 159-171. australasian journal of information systems hasan, connery & crawford 2018, vol 22, research article challenges of is research translations 24 yin, r. k. (2002). case study research, design and methods, 3rd ed. newbury park, sage publications, copyright: © 2018 hasan, connery & crawford. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 literature review 2.1 bureaucracies and technology 2.2 university administration and innovation 2.3 tensions, contradictions and dialectics 2.4 the cultural historical activity theory 2.5 knowledge translation from research to practice in is 3 methodology 3.1 background to the case 3.2 the data 3.3 content analysis with leximancer 3.4 identification and interpretation of activities and relationships 4 results of the data analysis 4.1 leximancer analysis and activity theoretical interpretation 4.1.1 activities of senior managers (refers to figure 2 and table 2 in the appendix) 4.1.2 activities of website administrators (refers to figure 3 and table 3 in the appendix) 4.1.3 activities of the research student (refers to figure 4 and table 4 in the appendix) 4.2 tensions and dialectics 5 discussion 6 conclusion paper title (paper title style) australasian journal of information systems farrell 2017, vol 21, research on indigenous ict archiving the aboriginal rainbow 1 archiving the aboriginal rainbow: building an aboriginal lgbtiq portal andrew farrell university of wollongong farrella@uowmail.edu.au abstract indigenous lgbtiq people are a marginalised group that do not have the luxury of representation on a broad range of social and cultural issues. the development of various online projects aims to challenge that. this paper is a discussion of the development of a blog project which supports aboriginal and torres strait islander lgbtiq peoples through link sharing online on social media. the themes of identity, social media, and indigenous activism are rehearsed in this paper to demonstrate the potential for emerging projects that challenge oppressive politics keywords: blog; social media; lgbtiq; queer; indigenous; activism 1 introduction troy anthony baylis (2014) argues that since european contact indigenous australians have been stripped of their diverse and customary sexual and gendered practices through the imposition of a new social and cultural order or “cultural default”. the cultural default identifies the dominance and oppression of indigenous peoples through two distinct processes. firstly, baylis identifies the deliberate destruction of culture where indigenous people have been actively dispossessed of their lands, people, societies and cultures through violence. secondly, eurocentric ideas about gender and sexuality are exported and imposed onto indigenous peoples. the very fabric of culture is re-characterised by the recasting of indigenous peoples into new socio-cultural norms predetermined, generated, and controlled by the dominant culture. this discussion is highlighted by historical developments such as the enforcement of religion in indigenous communities. for example, traditional marriage systems were overhauled by religious doctrine replacing traditional marriage systems such as polygamy with christian and catholic monogamy. this introduced and enforced system assumed the holy distinction of man and woman in its monogamous form as a central pillar of civilisation as a kind of social engineering project erasing cultural traditions considered savage. mononormative and heteronormative restructuring of indigenous australians’ everyday lives persisted under eurocentric regimes which continue to dominate and shape our understanding of indigenous genders and sexualities today. recent developments, as explored in this paper, discuss indigenous voices of dissent that identify and challenge ideas about gender and sexuality beyond the scope of the western gender binary and its dominant sexual norms. the reclamation and self-determination of indigenous lgbtiq (lesbian, gay, bi-sexual, transgender, intersex, and queer) identity is crucial in these conversations as a ‘new frontier’ of identity politics that revives knowledge about our ancient diversities as well as asserting our identities as self-defined and contemporary. the development of strategies that enable, prioritise and make visible lgbtiq peoples is discussed in this paper through indigenous lgbtiq voices that speak up and become present online. this article will discuss the blog archiving the aboriginal rainbow and how it was developed as a way to promote and increase the representations of indigenous australians’ who identify as lgbtiq. following the theme of ‘indigenous peoples and activism on social media’ this paper will demonstrate how blogging is a form of activism for indigenous lgbtiq australians. this paper will first discuss the social phenomena of blogging and its various uses as it pertains to a broader social context. i will then turn my attention to the use of online platforms by both indigenous and lgbtiq communities. this will lead to a discussion of digital archives as social spaces where political discussions, arguments, and australasian journal of information systems farrell 2017, vol 21, research on indigenous ict archiving the aboriginal rainbow 2 developments are stored in a ‘portal’ designed to make indigenous lgbtiq standpoints accessible in the age of technology. i identify as indigenous and queer. i am a millennial. i am also active on social media and deeply invested in the political advancement of my communities online. youth culture is fast becoming synonymous with technology as we frequently engage online as both users and producers. burnwell (2010) argues that the demands of “participatory culture” online blurs the distinction of user and producer. participatory culture describes how users become active in identifying gaps in online markets and how they fill those gaps with self-produced content. outside of the digital space, research reveals that there is a severe gap in literature, research, and representation of indigenous lgbtiq australians (riggs & toone 2016). this dearth of research exists online as there are few projects which identify and share content focused on aboriginal and torres strait islander lgbtiq peoples. from this emerges an opportunity to innovate online projects that address this issue. archiving the aboriginal rainbow blog is a response to these issues. 2 blogging and social media archiving the aboriginal rainbow project was based on a personal blog about identity, research developments, and various reflections on gender and sexuality. blogging was first used to disengage with social media as a separate space to reflect and write. platforms such as facebook provide a place for constant and ongoing interaction with personal networks of people through daily personal updates called ‘statuses’, sharing, responding, commenting on a broad range of topics and issues, and interacting with family and friends. facebook is oversaturated with information and one can get lost in the daily flow. blogging provides an alternative platform that offers similar functions to facebook. however, a blog is less about constant interaction as it is centred on producing content. it is a place to write extensively as well as share photographs and embed content. it is a kind of online diary compared to facebook which centres more on social interaction. social media ‘pages’, or groups, were the first to influence the blog project. facebook groups such as black rainbow which is a facebook page dedicated to sharing indigenous lgbtiq articles demonstrates the effectiveness of social media link sharing. the creator of the page, dameyon bonson, self-identifies on the facebook page which has also since grown into a developing social enterprise (http://www.blackrainbow.org.au/). this inspired me, as the creator of archiving the aboriginal rainbow blog, to identify. i used a ‘pinned’ permanent post to assure the blogs potential audiences that the blog is run by a person who belongs to the indigenous lgbtiq community and someone who works and advocates for that community. the blog presents itself as an online portal that shares links to contemporary and historical audio, images, articles, art, and various other items found across the web (farrell 2014). as a doctoral student, my work has always been provoked by a growing sense of urgency and responsibility towards the idea that indigenous lgbtiq peoples must be heard, present, and accounted for. this call was also a catalyst for my emergent doctoral thesis. the blog project connects to my academic research as it was first used as a repository for information collected for the thesis. the blog is complimentary to my research and utilises the benefits of interaction and sharing online. independent research can be isolating, (percy 2014) and sharing materials in the blog is a way to circumvent those feelings. amassing articles and various online materials for the thesis would eventually be accessible however it was decided that these items should be shared as they are collected. this decision was of personal importance as much as it was a method adopted to assist research. the project developed further after critical discussions were presented at the third asia-pacific outgames human rights forum from the 14-16th of may 2013 in darwin (equal opportunity commission 2014). there, a call to cultivate strategies of lgbtiq inclusion and visibility for and by indigenous peoples from transnational, national, and local perspectives across the asiapacific region dominated the stage. the theme was “power though action”. an inspiring group of aboriginal transgender women who identified themselves as sistergirls spoke on their australasian journal of information systems farrell 2017, vol 21, research on indigenous ict archiving the aboriginal rainbow 3 experience and asserted that their voices must be heard by the lgbtiq community here in australia. from that, an even greater call to action came from brotherboys (aboriginal transgendered men) who strongly articulated their need to be known and heard. their calls for inclusion and solidarity reinvigorated the desire for me to ensure the archiving the aboriginal rainbow advocated alongside lgbtiq aboriginal peoples. prior to the outgames forum was the annual queer histories symposium. one of the sessions held during the symposium was based on how we can “queer” archival practices: how we can gather, store, promote, historicise, and cement lgbtiq history as a part of the australian “story”. this session highlighted the ways that lgbtiq people have developed a material history out of the uncommon objects and rarely heard stories of lgbtiq peoples. the panel included kevin hauniu of lesbian and gay archives of new zealand, eric julian manalastas lgbt policy archives project (manalastas), the centre for women’s studies, northern territory archives service, and graham willett from australian lesbian and gay archives (alga). these various archives, repositories, and research projects reiterated the importance of collecting, identifying, and building narratives from objects and resources. 3 archives archiving the aboriginal rainbow is informed by queer archives including three primary institutions and resources: australia’s largest australian lgbtiq archive (alga), and international projects such as the arizona queer archives (aqa), as well as various archives online including trove (national library of australia). after exploring these various archival collections there was a notable scarcity of content on indigenous lgbtiq peoples. this reflects a harsh reality for indigenous lgbtiq peoples who have not been and are not currently overly visible in institutions that store our nation’s histories. regardless of modern and progressive expressions to collect lgbtiq indigenous materials for storage these archives are often limited to smaller projects as indigenous lgbtiq artefacts are not so readily available for collection. under similar circumstances lgbtiq collections were lacking in indigenous archives. these issues are not always the fault of the archive but are a symptom of a broader historical silence. there are various logistical reasons that may shape this current state. this absence may be due to issues such as a lack of research funding. there is no such thing as an endless monetary supply geared towards indigenous inclusion in archives such as alga (australian lesbian and gay archive) who work on a largely voluntary basis. limitations are also generated out of the dearth of sources on, by, and from this group which may be collected and preserved. aboriginal and torres strait islander lgbtiq histories are not always complimented by literature, nor by the static and ephemeral objects that are catalogued in a traditional archive. social media has assisted in alleviating some of the limitations of research in the field of indigenous lgbtiq inquiry by making the community accessible. social networking connects professionals and emerging researchers with networks based on collective interests that otherwise may not have happened offline prior to the proliferation of such sites as myspace, msn, facebook, and twitter. archives such as alga demonstrate this as they utilise social networks as well as their own online domain to engage a broader audience in order to develop a network of sharing and appreciating intersectional histories. such archives also create the potential discovery of new items to collect. while archives such as alga have a physical institution, it is their use of social media that has mobilised their potential to seek indigenous materials for archival purposes. alga’s key principles include: …to collect and preserve historically significant items that reflect lgbt life in australia and, secondly, to disseminate this information back into the community through public exhibitions, publications, and programs (australian lesbian and gay archives). these principles have informed the archiving the aboriginal rainbow blog project. the blog mobilises alga’s rationale and deploys their principles to guide further investigation into australia’s lgbtiq indigenous history. the archive is a space where queer subjects put themselves together as historical subjects (marshal et. al. 2014, p. 2). archiving the aboriginal australasian journal of information systems farrell 2017, vol 21, research on indigenous ict archiving the aboriginal rainbow 4 rainbow is aimed at transforming the retrieval and revival of elements of emerging queer archives to address indigenous erasure and estrangement by cobbling together historical understandings of sexuality and gender through an appraisal of presences and absences in found materials (marshal et. al. 2014, p. 2). archiving the aboriginal rainbow “queers” the archive but it also indigenises queer approaches to collecting and presenting items by creating a space that centres on indigenous lgbtiq identities. 4 social media groups progressive online projects undertaken by aboriginal and torres strait islander lgbtiq people in social media pages, groups, and community networking are vital to the blog. these include facebook groups such as black rainbow, sistergirls and brotherboys australia, sisters and brothers nt “celebrating diversity”, anwernekenhe us-mob, moolagoo mob, and black lemons. these projects are made up of public and private collectives that include allies, families, and extended networks. on these group pages, you can find aboriginal and torres strait islander lgbtiq peoples, interact and consult, as well as share in the social atmosphere and access vital information such as emergency services and a place to advertise job prospects for the community. these spaces create and share social events and are often the first to share materials that may be collected by archiving the aboriginal rainbow. my involvement in these groups has largely remained with gender diverse subcultures of sistergirls, brotherboys, transgender, and queer identified aboriginal and torres strait islander peoples. the sistergirl and brotherboy facebook group is actively creating history by assisting and fostering a space that is largely made up of a minority who would otherwise be unable to connect and collectivise. this group is vital in mobilising and facilitating discussion and organisation of political calls to action. for example, the group and i have attended events such as anwernekenhe 2015 conference in alice springs as representatives from our collective (anwernekenhe 6 organising committee 2015). we utilised the facebook group to consult the broader aboriginal and torres strait islander gender diverse community on issues around hiv awareness which was then presented and discussed at the conference. the community has proven that it is not a page used for likes, popularity, or as a mere display. it is a living hub of activity that aims to benefit the broader community through direct political action. over time, this community has become integral to both my personal and professional life. i am now an administrator of that group and use the site to share articles of interest and assist those who need guidance with writing. i am still in search of my identity and place in these networks as a queer identified person but am able to negotiate and navigate my identity with assistance that is familiar and appropriate to my cultural heritage. the sistergirl and brotherboy facebook group benefits the archiving the aboriginal rainbow blog more directly as it is shared as a pinned post which receives feedback, engagement, support, and criticism from diverse community perspectives. 5 the blog the collection began with a handful of articles and now stands at approximately two hundred and fifty posts. wordpress.com hosts the project as it provided the best template to share the collection and use it as a repository for research findings that are a result of sifting and trawling the internet for links to share. on this website, the primary function of the blog set out to appropriate my research bibliographies and various found materials that were bookmarked online to form an online portal. over time the posts have formed an interactive collage of lgbtiq aboriginal and torres strait islander australia that is constantly evolving. the theoretical distinction of the term ‘archive’ is important for situating the blog. theimer (2012) claims that modern digital collections both misrepresent and depart from the traditional use of the term. she uses the term ‘digital humanist’ to describe those who appropriate the term ‘archive’ in the digital space. theimer emphasise the correct usage of ‘digital collection’ rather than ‘archive’ to characterise the grouping of materials purposefully selected to be studied and made accessible. instead of broadening the term theimer argues australasian journal of information systems farrell 2017, vol 21, research on indigenous ict archiving the aboriginal rainbow 5 that the maintenance of the definition should remain centred on the profession. however, the reality is that the term is now broadly appropriated and accepted to describe the storage of data in a web page. the blog utilises the term “archive” as per its online usage. it is not a traditional archive in the sense that it is a repository. rather, it is an innovative use of some of an archive’s traditional ideological frameworks and collecting methods that are appropriated to generate a space specifically designed to foreground representations of aboriginal and torres strait islander lgbtiq peoples. attributing value to the knowledge production of this collection is also what this project envisions by using the term. by using the term “archive” the blog signals the importance of lgbtiq indigenous australians contribution to social and cultural issues such as marriage equality, incarceration, homelessness, suicide, mental health, and so on. it is of the utmost importance that opinions and contributions to the site are expressed, valued and shared to maintain an equitable distribution of the ideas and realities articulated. this approach distinguishes the blog as it departs from the purpose of the traditional archive which retracts items from view for the purpose of preservation. rather than serving as just a repository of information this project is proactively collecting and asserting a range of politics where most archives would be depoliticised or, at least, claim to be. this project ties into the survival of the discourse surrounding indigenous lgbtiq issues and may also reinforce the survival of the community itself. when aboriginal and torres strait islander lgbtiq people access online resources, it is important that they see representations of people like themselves. when they go online, this project lets them know that they are seen and heard and that they are present and welcome. that kind of rationale would not come about without the blogs continued response to community or its emphasis and interaction with culturally sensitive gendered and sexual identity politics. 5.1 the “digital midden” in the broader aboriginal and torres strait islander communities, indigenous blogs such as deadly bloggers are important for shaping archiving the aboriginal rainbow. deadly bloggers, created by leesa watego, is the first and only directory of aboriginal and torres strait islander bloggers (http://deadlybloggers.com/). watego presented at the reterritorialising social media: indigenous people rise up symposium held at the university of wollongong on the 26th of november 2015. in a short but illuminating conversation afterwards, watego responded to how archiving the aboriginal rainbow might be placed amongst other blogging projects by aboriginal people and how it is situated as a modern cultural practice online. in her presentation at the reterritorialising social media: indigenous people rise up symposium watego discussed cultural conceptualisations of blogging online. she demonstrated how blog projects are significant in developing and maintaining cultural specificity. watego interpreted blogging as an example of a midden (watego 2015). middens are fossilised deposits of food waste that tell a story of how food cultures have developed throughout history. middens are familiar to me as my people were sea foragers who created middens out of seashells and seafood waste. this waste collected over millennia formed layers of history that, when unearthed, functions to inform and re-inscribe elements of past cultural practices into contemporary discourse. watego used the midden to describe how blogs are one of the many ways that we practice culture today. she describes blogs as middens that collect ideas which ultimately shape our understanding of modern indigenous cultures. similarly, carlson (2013, p. 145) argues a position on cultural continuity where aboriginal and torres strait islander peoples’ interaction online often mirror or adhere to offline expectations and regulations that change over time. both carlson (2013) and watego (2015) argue that the new forms of cultural practice we develop online have a correlation to the culturally distinct realities that we experience and practice outside of the web. in other words, our realities shape the digital footprint accumulated virtually as a midden would physically. the archiving the aboriginal rainbow blog functions as a contemporary midden of ideas generated by the lgbtiq indigenous community online as active, and ongoing. in a more australasian journal of information systems farrell 2017, vol 21, research on indigenous ict archiving the aboriginal rainbow 6 personal conversation watego interpreted the archiving the aboriginal rainbow blog as a curation – an organisation of materials that conveys a central idea or “story”. the broader narrative of the blog is situated in the diverse histories, experience, and futures of indigenous lgbtiq peoples with no discernible beginning or ending. it is a digital collection anchored in indigenous ways of sharing, preserving, and handing down stories as our cultures have done for tens of thousands of years. what is collected and shared facilitates stories shared from multiple cultural standpoints influencing how the broader audience may understand indigenous lgbtiq identity and diverse perspectives on a broad range of themes including community and individual achievement. 5.2 the black rainbow in 2015 dameyon bonson, the creator of black rainbow, was the first aboriginal lgbtiq person to be nominated for a human rights award for his contribution to advocacy and support for aboriginal and torres strait islander lgbtiq peoples (alexander 2015) and in 2016 he won the dr yunupingu award for human rights (hocking 2016). bonson’s achievements through his advocacy online has since led to the development of the black rainbow living well foundation for aboriginal & torres strait islander lgbtiq suicide prevention (bonson 2013) as well as bonson’s report voices from the black rainbow (bonson 2017). other milestones include the televised news special brotherboys on living black on nitv (nitv 2015) which developed out of the activism of the sistergirl and brotherboys facebook group. the blog also collects and displays the performances and careers of prominent aboriginal drag queens such as destiny haz arrived, who is a drag celebrity who has appeared at a range of community sporting and social events (ward & costello 2013, p. 18), and is an advocate for hiv awareness and prevention who was a headline act and poster-girl for sydney’s lesbian and gay mardi gras in 2015 (mears 2015). as material is generated around these important political, social, and cultural developments it is shared with pride by the archiving the aboriginal rainbow blog. 6 posting gathering and depositing into the “digital midden” is a simple process as follows: 1. search for article and materials to post. 2. copy and paste the title of the material as it is found. use original title as the title of the blog post. 3. embed link to original source into text “found here”. 4. invariably include a quote of the material in the post. 5. if audio/visual media is shared, the option to directly embed the materials sometimes exists. 6. create ‘tags’. 7. publish. the source is often just a click away and its accessibility depends on its host website. i do not claim ownership of the original content found in the external site. in this approach, the blog aims to reiterate the importance of context as emphasised by theimer (2012). she states that an archivist’s work is concerned with the preservation of context which is an essential aspect of demonstrating authenticity (2012). the blog collects without intent to intervene or misinterpret the source materials original context which is why precautions are taken in becoming too descriptive with the use of ‘tags’. tagging is an efficient and powerful aspect of the project. tagging is used to categorically describe the themes present in each article. in archiving the aboriginal rainbow, tags act as a kind of visual cartography of materials available across the blog. tags also grow larger in font as more articles are contributed to the category. the largest tags are currently “sistergirls”, australasian journal of information systems farrell 2017, vol 21, research on indigenous ict archiving the aboriginal rainbow 7 “brotherboys”, “transgender”, and “stories”. this may be subject to criticism as one tag dominates the space and may reflect an imbalance of representation of the lgbtiq aboriginal and torres strait islander community online whereas, for example, in the broader queer community articles relating to homosexuality may be dominant. tagging is useful for personal research as it identifies the potential for researching new sources to add to the “digital midden” for a more equitable representation of identities across the aboriginal and torres strait islander lgbtiq community. current materials sought for archiving the aboriginal rainbow include materials relating to indigenous peoples with intersex variations who are underrepresented in the project. some of the ‘tags’ found in the blog are uniquely indigenous as they reflect specific cultural ideas. “stories” is a unique tag based on the commonplace use of the term to describe biographical and experiential stories of aboriginal and torres strait islander peoples – often referred to as a “yarns” (geia et. al. 2013). one of the posts, for example, shares an audio project by yarns.org.au with three aboriginal lgbtiq stories from rural and remote regions (australian federation of aids organisations). our stories are an important guide to our cultures. archiving the aboriginal rainbow contains personal stories, stories of family, community, land, and so on. tagging centred on themes of gender and sexuality emphasises that our sexuality and our gendered stories are important to us and are not isolated from our experiences of self, family, community, and cultural identities. the blog is also multi-platform as it uses a connected twitter account to disseminate the information as it is created. this was purely to extend the reach of the blog and to obtain followers from other popular domains outside of wordpress and facebook. the twitter account has a modest following of 53 followers and the blog on wordpress.com has 41 followers. while facebook reigns as the place to widely share a link to archiving the aboriginal rainbow, twitter is used for audiences who may prefer one social platform to another. the blog is gaining increased international viewership and is often found in the first few pages when searching the combination of terms ‘aboriginal’ and ‘lgbtiq’. search engines like google and wordpress identify the growing numbers through statistical information gathered by the blog. this includes a map of what country the views are coming from, the number of views that pages have received across a range of time periods, and who has liked, subscribed, and commented on the blog. views naturally engender commentary. the blog attracts few comments and they are moderated as they are submitted. most of these comments ask questions about information not provided by the blog. a recent visitor asked a question in regards to an article shared on the blog. this visitor had taken issue with the art that accompanied an article. i referred them to approach the developers of the original source that shared this art. this is an interesting development that speaks to the perceived relationship of the blog to the links that are shared. the blog maintains its purpose as a project that does not generate its own content. rather, it is an information harvesting platform otherwise known as a “portal”. almasy (2005, p. 621) describes online portal’s as a “subject gateway” for specific research based resources in the form of a directory of online resources that arrange content by subject, contains current news stories based on the chosen field, and facilitates research. archiving the aboriginal rainbow blog is situated within this online genre. interaction in the form of comments on the blog reaches just beyond the parameters of a traditional portal to enable limited but potentially important opportunities of user and developer interaction. 7 limitations, identity, and authenticity materials sought for archiving the aboriginal rainbow blog are strictly about availability and accessibility online. there are materials beyond this scope that have the potential to be included in the project. however, from the outset, there have always been limitations as to the kinds of materials that can be shared. for example, peer-reviewed journal articles are typically only accessible through research institutions and others must be purchased. the blog is purely a mechanism that assists and directs people to where they may access articles, video, audio, australasian journal of information systems farrell 2017, vol 21, research on indigenous ict archiving the aboriginal rainbow 8 and so on. for example, the blog shares links to other websites where users can buy texts such as colouring the rainbow: blak queer and trans perspectives (hodge 2015). the blog encourages its audience to buy materials relating to indigenous lgbtiq peoples and, importantly, support lgbtiq indigenous authors such as alison whittaker’s collection of poetry in the award-winning lemons in the chicken wire (2016). historical materials are a challenge for the project. as noted, aboriginal and torres strait islander lgbtiq histories have largely been silenced due to the exclusion in research and mainstream discourse surrounding the existence of homosexuality in indigenous australian cultures throughout history. kleinert (2000) argues that early depictions of aboriginal sexualities by anthropologists are instrumental in the erasure of indigenous sexual diversity. their eurocentric lens, informed by imported ideas about gendered and sexual morality, has generated a mode of representation that continues to suppress the existence of sexually diverse practices in indigenous societies. this absence of representational focus is challenged by the rise of aboriginal and torres strait islander gay and lesbian voices which have seized upon opportunities now available to them to articulate their own lives and stories through a wide range of media (kleinert 2000, p. 295). today, transgender voices from within the indigenous lgbtiq community are pushing those boundaries further. recent texts argue for the visibility and inclusion of transgender people in aboriginal and torres strait islander communities by asserting that indigenous transgender identity pre-dates colonisation (canny & mccrumb 2010). this claim is reinforced through various contemporary texts and modern oral histories that follow a similar trajectory experienced by the gay and lesbian community against claims about aboriginal and torres strait islander peoples that silence our diverse past, present, and futures. this is a unique example of the ways that indigenous peoples reclaim their identity. interrogating and resisting silences about indigenous genders and sexualities revives important conversations pertaining to indigenous identity in australia that extend beyond the lgbtiq community. archiving the aboriginal rainbow blog is an agent of change in those arenas. the call to recognition promoted though link-sharing facilitates indigenous lgbtiq politics to benefit the wider indigenous community as it reconceptualises how aboriginal and torres strait islander communities know themselves, their histories and identities. the authenticity of aboriginal and torres strait islander lgbtiq identities is persistently challenged by those who participate in normative ideas about sexuality. the enduring impact of western cis-gendered and heteronormative domination has re-emerged in instances of lateral violence enacted on social media platforms such as facebook. the facebook page for nitv (national indigenous television) shared many lgbtiq articles throughout mardi gras season in early march 2017. they shared the article comment: the art of seeing aboriginal australia’s queer potential by artist and scholar troy-anthony baylis (2017). some responses were accepting and others were not. one person responded: this is absolute crap. homosexuality is not part of traditional aboriginal society or culture. trying to justify this is outrageous. how dare you try and create such a lie to justify modern day aboriginal homosexual homosexuality is a result of colonisation never have i herd [sic] men’s and women’s business be so disrespected by crazy delusional ideologies (unnamed facebook user). this comment demonstrates the kind of violence and denial directed towards aboriginal and torres strait islander lgbtiq people. comments such as these have created a need for online safety for indigenous lgbtiq people against those who assert that they do not exist, and when they do, are a perversion of a perceived traditional order that is authentically heterosexual. 8 safer spaces online the idea of safety is important for archiving the aboriginal rainbow. oppression, discrimination, and various forms of violence are issues that greatly affect aboriginal and australasian journal of information systems farrell 2017, vol 21, research on indigenous ict archiving the aboriginal rainbow 9 torres strait islander lgbtiq peoples. violence from within community lines is as prevalent as it occurs between lgbtiq peoples and wider society. kelly writes: bigotry within the queer community often goes by a different name – lateral violence, or, in other words, fighting against each other.… [which is often] directed at one’s peers rather than ones true adversaries (kelly 2017). kelly (2017) identifies the ways in which lgbtiq community engages in and perpetuates forms of oppression such as misogyny, queerphobia, transphobia, racism, ableism, and so on. kelly suggests that the very idea of a community itself is challenged by the existence of these ‘microaggressions’ as they persist across various lgbtiq sub-groups. for indigenous lgbtiq people, racism is an issue within the lgbtiq community. this is highlighted by the experience of racism on social media dating and “hook-up” apps such as grindr where indigenous peoples are confronted by racist taunts (donelly 2016). the establishment of safer spaces online is a response to racism within the lgbtiq community and is a necessary challenge for platforms such as grindr and facebook which utilise methods of safety such as providing options like blocking and reporting abuse. the term “safer spaces” is used here to acknowledge that there is no true safe space free from tension and all forms of violence (https://radicalassembly.wordpress.com/safer-spaces/). archiving the aboriginal rainbow has not received any abusive comments to date. it is however ready to moderate such events as they occur. as racism and homophobia persist on social media it is an anticipated challenge. although archiving the aboriginal rainbow has not been subjected to abuse, it has received criticism for sharing articles. the blog has been criticised for sharing articles under claims that i aim to speak for those whose voices are contained in the articles. ultimately, such criticism produces effects that are counterintuitive of the projects’ aims towards inclusion and visibility though the threat of community conflict. as the creator of the project, i must be accountable and adapt to the challenges that may occur. self-care, reflection, and formulating new approaches is how i, as producer, respond to community violence. while re-strategising is important sometimes the most effective use of technology is infrequent use. infrequent use can also be counter-productive to asserting and advocating for lgbtiq representations online however it is important to accept that reflexivity and listening must play a central role in the continuing development of the blog. this does not suggest acquiescence to all forms of criticism. rather, if the blog is to maintain its focus on inclusivity and political activism, it is important to continually aim for a balance that will allow for critical interaction while also promoting the political goals of the blog. 9 diversity archiving the aboriginal rainbow aims to reinforce the possibilities of, and for diversity. however, in some instances there are diversities that cannot be displayed and shared due to cultural sensitivity and protocol. indigenous australian cultures are not homogenous and the extent of their diversities is deeply complex. taboos surrounding gender and sexuality vary across australia’s aboriginal and torres strait islander cultural landscape and the protocol surrounding them are subject to change over time. i was made aware by private message on facebook that a blog post on archiving the aboriginal rainbow breached a sensitive protocol. the link shared on the blog contained information about traditional genital sub-incision which, as was informed, is highly sacred and taboo for transgender women who belong to the cultural groups who were named. sharing this article was a further violation of the protocol so the link was removed from public view. any further sharing of the article would also be potentially criticised including referencing it in this paper. issues such as these are a challenge for a blog about visibility and representations particularly of historical records that confirm diverse practices. while such cultural protocols are important to acknowledge, they do not demonstrably encroach on the blog’s objectives; in fact, they contribute to our knowledge of ourselves as vastly diversified cultures. australasian journal of information systems farrell 2017, vol 21, research on indigenous ict archiving the aboriginal rainbow 10 10 an indigenising project archiving the aboriginal rainbow is an ‘indigenising’ project. indigenising is a process of decolonising through deliberate shaping, subverting, and transforming of colonialist power systems to fund indigenous ways of knowing and being (tuhiwai smith 2012, p. 147). the blog is an indigenising project as it transforms the digital genre of a blog to support indigenous ways of knowing and being sharing the many ways that aboriginal and torres strait islander identities are expressed. strategies used to maintain archiving the aboriginal rainbow follows approaches and principles of healing, mobilisation, decolonisation (tuhiwai smith 2012, p. 121). these principles are informed by indigenous theorists, scholars, and leaders as a form of respect and connection to the broader aboriginal and torres strait islander community and indigenous communities across the globe. the ideas of these people constitutes an assertion of belonging for many and an acknowledgement that we are not alone in our efforts to reframe debates. tuhiwai smith argues that indigenous researchers may become complicit with colonial processes that alienate, exclude, or isolate activist aspirations (2012, p. 218). as a blogger, researcher, and community member, it is crucial to maintain a level of awareness about the ideas that place the blog as complicit and/or defiant to entrenched and structural imbalances. as discussed, lateral violence can affect intercommunity relationships that may result in exclusion. this is a sensitive issue for archiving the aboriginal rainbow as it is a project that aspires to combat exclusion. due to the absence of lgbtiq sub-groups online it may seem that they are excluded within the blog as they are not present or accounted for. however, this is a mere symptom of the broader absence of these peoples in accessible materials across the internet, regardless of original intentions to utilises ideas of resistance adopted from academic and political work aimed at intervening in the ways institutions conceive of indigenous genders and sexualities (driskill 2011). archiving the aboriginal rainbow faces challenges unique to terrains of social media. research and activist aspirations collide on this platform in ways that facilitate a discourse about why various groups are so underrepresented. 10.1 indigenous standpoint and the cultural interface to provoke discussions of diversity means to identify and challenge the codifying and unquestioning of introduced and imported systems of social organisation which may have become accepted as a fact of aboriginal life (hodge 2015). indigenous ideas about heterosexuality and the binary of man and woman are ultimately challenged by the material shared by the blog regardless of their unifying or conflicting nature. identity categories found within archiving the aboriginal rainbow are not vetted for the ways that they adhere to the dominant definition of certain terms. rather, tags such as “sistergirl” or “gay” are informed by the material that claim the use of the term. this approach is aligned with the notion of ‘standpoint’ defined by nakata (2007, p. 214) as an engagement with the social positions that contribute to the complex positionalities of values and ideas that situate aboriginal and torres strait islander peoples. archiving the aboriginal rainbow shares the perspectives of indigenous lgbtiq peoples as decolonising agents at the “cultural interface” (nakata 2007). nakata’s cultural interface theory is considered here to describe the ways that indigenous lgbtiq ideas filter through complex terrains of knowing and unknowing influencing how we see and know ourselves and others. i first introduced this discussion in my 2015 paper titled ‘can you see me? queer margins in aboriginal communities’ to describe indigenous lgbtiq agency (farrell 2015). the cultural interface, described by nakata, is a multi-layered and multi-dimensional space of dynamic relations constituted by… competing and contesting discourses… [which] inform, constrain or enable what can be seen or not seen, what can be brought to the surface or sutured over, what can be said or not said, heard or not heard, understood or misunderstood, what knowledge can be accepted, rejected, legitimised or marginalised (2007, p. 199). australasian journal of information systems farrell 2017, vol 21, research on indigenous ict archiving the aboriginal rainbow 11 nakata points to the complexity of competing systems that shape the existence of gender and sexual diversity within terrains of knowing and being. archiving the aboriginal rainbow functions at the cultural interface. the blog addresses the ways that aboriginal and torres strait islander lgbtiq people are excluded, silenced, and erased on a digital platform as a response to issues faced by indigenous lgbtiq peoples on and offline. nakata’s work sees the cultural interface as a site of productivity, in this instance, a site where negotiation and change are possible, where aboriginal and torres strait islander peoples are agents of provocation and change though making themselves visible, known, heard, understood, and legitimised. 11 the re-shaping and re-imagining of indigenous politics archiving the aboriginal rainbow is informed by what tuhiwai smith (2012) calls the ‘indigenous peoples project’. tuhiwai-smith describes the indigenous peoples’ project as a modern call to inclusiveness and diversification of indigenous politics reminding us that our needs and desires as indigenous peoples are always already diverse. this is strongly emphasised by the sexual and gender diverse members of our communities whose displays of identity become public calls to inclusion. the blog strengthens the re-conceptualisation of the indigenous people’s project by reinforcing calls for the inclusion of lgbtiq peoples in the advancement of indigenous politics in australia. the blog is built on the re-shaping and re-imagining of indigenous politics. driskill (2010) argues that diverse indigenous lgbtiq critiques are a part of the ongoing resistance to colonialism through separate and communal standpoints that imagine and perform survival, resistance, and continuity through the assertion of traditional and contemporary knowledge and oppositional politics. decolonisation does not have a fixed and finite goal (driskill 2010, p. 70). nor does archiving the aboriginal rainbow. while maintaining a political focus, having no fixed political destination or outcome is a strength of the blog as it becomes a space to share a wide range of standpoints that may seem to be self-contradictory, conflating, or fundamentally opposed. in addition, having no articulated telos, the blog is open to possibilities, to change, and to future development that may include data and stories not yet considered. diversity of ideas and contributions in this context means that contributions to the site inform, shape, and re-shape the terrain of knowledge that ultimately characterise the broader lgbtiq indigenous community in australia. the more contributors who speak enable the enrichment of the site and its potential for future development. while engendering solidarity across the lgbtiq community is a core objective of archiving the aboriginal rainbow, it is understood that solidarity does not necessarily connote a unanimous voice and that multiple standpoints are a central tenet of the blog’s expression of solidarity. the blog locates shared articles that contain potent and topical discussions about racism, antimarriage equality, homophobia, transphobia, and other social and political views that are important to aboriginal and torres strait islander lgbtiq peoples. the blog is not only for aboriginal and torres strait islander lgbtiq people. it is also catered for consumption by the broader community. by presenting a collection of lgbtiq content in a convenient and accessible way, archiving the aboriginal rainbow aims to shift some of the responsibilities and pressures experienced by indigenous lgbtiq people who are routinely required to justify themselves. this blog alleviates some of those pressures by directing potential allies to undertake their own online research and to be responsible for their own education and understanding of indigenous diversity. 12 conclusion archiving the aboriginal rainbow demonstrates how indigenous lgbtiq peoples assert their political standpoints online by vocalising our oppression, resisting denial, calling out and challenging violence, and taking ownership of our diversities. archiving the aboriginal rainbow will continue to develop well into the foreseeable future and adapt, when necessary, to accommodate the needs of the audience and the shifting political, social, and cultural grounds that are constantly in flux. its continuity depends on my continued passion to australasian journal of information systems farrell 2017, vol 21, research on indigenous ict archiving the aboriginal rainbow 12 advocate for these communities, a factor i see as a lifetime goal. the blog is a product of my life circumstances, my identity, and my role in community as a researcher and i am inspired to see it has become a beacon of the community functioning to facilitate indigenous lgbtiq representations in social media. to bring about political, social, and cultural change it is important to speak up, persist, and refuse to be invisible. archiving the aboriginal rainbow offers users, contributors, researchers and readers an opportunity to collaborate in a spirit of mutual generosity and with a view to understanding and addressing the historical silences that have oppressed our community for too long. references alexander, d. (2015) “a first for lgbti recognition at national indigenous human rights awards”, star observer. available at, http://www.starobserver.com.au/news/localnews/a-first-for-lgbti-recognition-at-national-indigenous-human-rightsawards/142097 [accessed 2sept 2016]. almasy, e. (2005) “tools for creating your own resource portal: cwis and the scout portal toolkit”, library trends, 53(4): 620-636. anwernekenhe 6 organising committee. (2015) anwernekenhe: remembering recognising responding: 21 years. available at, http://ana.org.au/wpcontent/uploads/documents/anwernekenhe_6_program_draft.pdf [accessed 5 jan 2016]. australian federation of aids organisations. available at, http://www.yarns.org.au/ [accessed 5 jan 2016]. australian lesbian and gay archives. available at, http://alga.org.au/ [accessed 4 jan 2016]. baylis, t. (2014) “the art of seeing aboriginal australia's queer potential”, the conversation, available at, http://theconversation.com/the-art-of-seeing-aboriginal-australias-queerpotential-25588 [accessed 1 march 2017]. baylis, t. (2017) “comment: the art of seeing aboriginal australia's queer potential”, sbs, available at, http://www.sbs.com.au/nitv/article/2017/03/01/comment-art-seeingaboriginal-australias-queer-potential [accessed 20 april 2017]. bonson, d. (2013) black rainbow. available at, https://www.facebook.com/blackrainbowaustralia [accessed 11 jan 2017]. bonson, d. (2017) voices from the black rainbow: the inclusion of the aboriginal and torres strait islander lgbqti, sistergirls and brotherboys in health, wellbeing and suicide prevention strategies. available at, https://dameyon.com/home/voicesfromtheblackrainbow/ [accessed 1 may 2017]. burnwell, c. (2010) “rewriting the script: toward a politics of young people's digital media participation”, review of education, pedagogy, and cultural studies, 32(4-5): 382-402. canny, a & mccrumb, d. (2010) sistergirl. matchbox pictures & plus films carlson, b. (2013) “the ‘new frontier’: emergent indigenous identities and social media”, in harris, m., nakata, m. & carlson, b. (eds.), the politics of identity: emerging indigeneity, university of technology sydney e-press, sydney. donelly, b. (2016) “gay minorities speak out against racist slurs on grindr”, sydney morning herald, available at, http://www.smh.com.au/national/gay-aboriginal-manpublishes-racist-slurs-on-dating-app-20160418-go8zov.html [accessed 20 april 2017]. driskill, q. (2010) “doubleweaving two-spirit critiques: building alliances between native and queer studies”, glq: a journal of lesbian and gay studies, 1(69): 69-92. australasian journal of information systems farrell 2017, vol 21, research on indigenous ict archiving the aboriginal rainbow 13 driskill, q. (2011) “queer indigenous studies: critical interventions in theory, politics, and literature”, university of arizona press, tucson. equal opportunity commission. (2013) “power through action”. available at, http://www.eoc.sa.gov.au/sites/eoc.sa.gov.au/files/attachments/hrf_flyer.pdf [accessed 5 jan 2016]. eric julian manalastas, “lgbt resources”. available at, http://pages.upd.edu.ph/ejmanalastas/lgbt-resources-0 [accessed 5 jan 2017]. farrell, a. (2014) archiving the aboriginal rainbow. available at, https://indigblackgold.wordpress.com/ [accessed 4 jan 2017]. farrell, a. (2015) “can you see me? queer margins in aboriginal communities”, journal of global indigeneity, 1(1). available at, http://ro.uow.edu.au/cgi/viewcontent.cgi?article=1010&context=jgi [accessed 15 jan 2016]. geia, l. k. hayes, b. & usher, k. (2013) “yarning/aboriginal storytelling: towards an understanding of an indigenous perspective and its implications for research practice”, contemporary nurse: a journal for the australian nursing profession, 46(1): 13-7. hocking, r. (2016) “human rights activists recognised at awards night”, sbs. available at, https://www.sbs.com.au/nitv/article/2016/06/10/human-rights-activists-recognisedawards-night [accessed 28 sept 2016]. hodge, d. (2015) “colouring the rainbow: blak queer and trans perspectives”, wakefield press, adelaide. available at, http://www.wakefieldpress.com.au/product.php?productid=1267 [accessed 5 jan 2016]. kelly, c. (2017) “fight amongst yourselves – bigotry within the queer community”, sbs. available at, http://www.sbs.com.au/topics/sexuality/agenda/article/2017/03/23/fight-amongstyourselves-bigotry-within-queer-community [accessed 20 april 2017]. kleinert, s. & neale, m. (2000) “renegotiating tradition: urban aboriginal art”, the oxford companion to aboriginal art and culture. oxford university press, melbourne. living black (2015) “brotherboys”, nitv, australia. available at, http://www.sbs.com.au/ondemand/video/451014723969/living-black-brother-boys [accessed 15 jan 2016]. mears, j. (2015) “drag queen destiny haz arrived to duet with jessica mauboy at mardi gras”, sydney morning herald. available at, http://www.smh.com.au/entertainment/music/drag-queen-destiny-haz-arrived-toduet-with-jessica-mauboy-at-mardi-gras-20150211-13at0e.html [accessed 20 march 2017]. marshall, d. murphy, k. p. & tortorici, z. (2014) “queering archives: historical unravelling’s”, radical history review, 120: 1-11. nakata, m. (2007) disciplining the savages: savaging the discipline, aboriginal studies press, canberra. national library of australia. available at, trove.nla.gov.au [accessed 5 january 2016]. percy, a. (2014) “studying a phd: don't suffer in silence”, the guardian. available at, https://www.theguardian.com/higher-educationnetwork/blog/2014/mar/25/studying-phd-dont-suffer-in-silence-seek-support [accessed 20 march 2017]. riggs, d. w. & toone, k. (2016) “indigenous sistergirls' experiences of family and community”, australian social work. doi: 10.1080/0312407x.2016.1165267 australasian journal of information systems farrell 2017, vol 21, research on indigenous ict archiving the aboriginal rainbow 14 theimer, k. (2012) “archives in context and as context”, journal of digital humanities, 1(2). available at, http://journalofdigitalhumanities.org/1-2/archives-in-context-and-ascontext-by-kate-theimer/ [accessed 20 march 2017]. tuhiwai smith, l. (2012) decolonizing methodologies: research and indigenous peoples, london, zed books. ward, j. & costello, m. (2013) “respect and resilience: shaping the response to hiv and stis among aboriginal and torres strait island communities”, hiv australia, 11(3). available at, https://www.afao.org.au/wp-content/uploads/2017/02/hiva-113_respect-and-resilience.pdf [accessed 20 march 2017]. watego, l. (2015) “reterritorialising social media”. available at, http://www.notquitecooked.com/2015/11/reterritorialising-social-media.html [accessed 15 jan 2017] copyright: © 2017 farrell. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems burmeister, scott, richards, adhikari 2016, vol 20, post publication review security & safety of mobile health apps post publication review scott, k, richards, d, adhikari, r. (2015) a review and comparative analysis of security risks and safety measures of mobile health apps. australasian journal of information systems, 19, doi:http://dx.doi.org/10.3127/ajis.v19i0.1210. review the above paper by scott et al 2015, is a timely contribution to the growing literature on mobile health apps. the authors stated their aim to be “to investigate the data security and privacy features of 20 popular free and paid health apps.” furthermore, as the title of their paper makes clear, consumer safety was also an important consideration in their study, which is further borne out in their methodology. thus their contribution draws attention to flaws in current health app designs and in international legislation and policies, which permit client data to be shared with third parties or to be stored insecurely. the background to the study, and the issues addressed in the discussion of their findings are applicable to a wide and international audience. overall i found it to be a thorough and engaging read. there are four areas that i wish to explore further, if the authors will respond to this review: (1) i found it curious that more was not made of the australian context, given that the authors are in australia and the journal targeted was australasian. instead the legal discussion mostly focuses on the usa, with a little also from europe. (2) whilst the security discussion was very thorough, and considered many viewpoints, it assumed all users are the same, when instead some are more vulnerable than others. (3) it appears to equate privacy with security. (4) client safety is seen as very important and, following on from (2), this is particularly the case for vulnerable clients. firstly, on page 2 the scott et al describe the difficulty for a client to keep abreast of legislative changes. this is indeed the case, as was recently shown not for clients, but health care providers in rural australia (burmeister, islam, dayhew, & crichton, 2015); one example therein was that of 104 changes to the nsw state privacy legislation in a single year – clearly beyond the scope of all but legal experts to keep abreast of. however, the last paragraph of section 2.4 is very usa centric and implies that the same is true elsewhere, when at least in australia that is not the case. for apps that store data in australian jurisdictions, there are clear laws that govern what can be shared with third parties, about encryption and more; seen particularly in changes to the commonwealth privacy act that were brought into play in mid-2014 and as a result of which there are now far greater requirements for transparency in the handling of personal information. secondly, all clients appear to be treated equally in this article. that in itself is curious, given that two of the apps reviewed were neuromind and stresscheck, apps which would be used by vulnerable consumers, many of whom could have cognitive challenges. people who are vulnerable, such as those with reduced cognitive capacity are not considered, as is evident at least by implication, by passages such as at the end of section 2.1, where it is assumed that rational choices are made by all mobile health app users. similarly table 3 assumes that all consumers who use the apps are rational and able to reason their way through the (sometimes bewildering) legal jargon and disclaimers. for instance, my work is mostly in areas of technology use of consumers of mental health services, where we have shown that such consumers, particularly older people, can be considered vulnerable users, whose vulnerability increases with increasing age (bernoth, dietsch, burmeister, & schwartz, 2014), and that their technology interaction needs can be different from those of younger age groups (burmeister, 2010). furthermore, as recently seen in this journal, people with suicidal ideation also need to be considered as not always making rational choices (carlson, farrelly, frazer, & borthwick, 2015). the third is evident in many places. for instance, none of the measures in table 1 are related to non-security privacy. whereas technical matters (eg encryption) are defined, privacy is not. 1 australasian journal of information systems burmeister, scott, richards, adhikari 2016, vol 20, post publication review security & safety of mobile health apps at times it is used in relation to confidentiality, at other times in relation to information privacy (cockcroft, 2006), and other uses. in fact such are all different types of privacy. the authors are not alone in failing to distinguish privacy. almost two decades ago i pointed out that the australian computer society had dropped ‘confidentiality’ from its code of ethics in the apparent mistaken assumption that it was the same as privacy (burmeister, 2000) and was involved in revising that misunderstanding (bowern, burmeister, gotterbarn, & weckert, 2006). finally, i’d have liked to see more in-depth discussion on consumer safety, which is even more critical for vulnerable consumers: the young, people with intellectual disabilities, and older people with neuro-degenerative diseases, as shown in many recent studies (pakrasi, burmeister, mccallum, coppola, & loeb, 2015; van wynsberghe, 2015). for instance, in discussing apps and other assistive devices for people with dementia teipel et al. (2016) claimed that the solution involves the balance of a complex negotiation of many factors (cultural, environmental and personal factors, social resources, and advancements in the intelligence and flexibility of the technological devices), that at its center has the safety of patients. oliver burmeister charles sturt university oburmeister@csu.edu.au author response we thank the reviewer for his comments on our paper, which have expanded our research findings into the security risks and safety measures of health apps and provided additional areas of future research. our study was aimed at general populations using health apps and we acknowledge the reviewer’s assertion that not all users are the same in their ability to follow the security and safety recommendations for app users. indeed, following such recommendations may be challenging for many classes of users due to a range of factors, including time, motivation, ability (cognitive, physical, psychological) and access to resources. we agree that vulnerable users, such as the elderly and those with mental health issues have specific needs and challenges in managing privacy and security concerns. we would argue that children are another vulnerable group (benassi, 1999; scott, gome, richards, & caldwell, 2015). however, it was, and is, beyond the scope of our investigation to suggest how our general recommendations could be modified for, or implemented by, specific groups of users. in other work, the second author has grappled with issues related to social media and social networks, such as adding friends and becoming a follower, for vulnerable populations (ruppert, richards, arnold, riches, & parmenter, 2010) and the (in)appropriate design and use of virtual characters as buddies or companions for those with psychosis, as in the work by bickmore, puskar, schlenk, pfeifer, and sereika (2010). our paper should have more clearly defined privacy and security. in a general sense, we use the word ‘privacy’ to refer to the rights of the individual to choose to reveal or disclose information about themselves and their property (agrawal & srikant, 2000), both tangible and intangible (warren & brandeis, 1890). in the australian health context, a range of federal, state and territory privacy regulations limit the collection, storage, access, use and disclosure of personal identifying information (nhmrc, 2004) (p. vii), and prohibit its use or disclosure without consent for purposes other than that for which it was collected, unless there is an emergency, law enforcement justification or administrative determination (nhmrc, 2004) (p. viii). we note that, in response to the increasing use of digitisation in contemporary society, the office of australian information commissioner is developing resources offering health privacy guidance for health service providers and consumers (oaic, 2015), in line with the privacy act 1988 (australian government, 1988). in our paper we treat ‘security’ as a higher-level concept covering more than just privacy, as evidenced in table 2. due to the sensitive nature of medical and personal data, in our paper we have discussed both privacy and security; however, we have used the word ‘security’ 2 australasian journal of information systems burmeister, scott, richards, adhikari 2016, vol 20, post publication review security & safety of mobile health apps throughout, notably in the paper title, to recognise that security is a bigger issue than just privacy. our study was targeted at an international rather than an australasian audience, so the international context we provided sought to identify the lack of legislation in much of the world, notably the usa, which is a leader in software app development. the reviewer’s comments and references are valuable and provide a more local context but give the impression that due to such legislation, there is less of a problem and that app developers should have clear guidelines to follow. however, our study, which involved the most frequently accessed health apps in australia, found that most of these apps failed to follow this legislation and in many cases may not have been bound by australian legislation, even though they were accessed and used by australians. this highlights the complexity and global reach of the problem, as well as issues around legal boundaries. further research in this area would clarify the issues involved and indicate possible means of addressing them. karen scott the children’s hospital at westmead university of sydney karen.scott@health.nsw.gov.au deborah richards macquarie university deborah.richards@mq.edu.au rajindra adhikari macquarie university rajindra.adhikari@students.mq.edu.au references agrawal, r., & srikant, r. (2000). privacy-preserving data mining. paper presented at the acm sigmod record. australian government. (1988). privacy act 1988. retrieved from https://www.legislation.gov.au/series/c2004a03712 benassi, p. (1999). truste: an online privacy seal program. communications of the acm, 42(2), 56-59. bernoth, m., dietsch, e., burmeister, o. k., & schwartz, m. (2014). information management in aged care: cases of confidentiality and elder abuse. journal of business ethics, 122, 453-460. doi: 10.1007/s10551-013-1770-7 bickmore, t. w., puskar, k., schlenk, e. a., pfeifer, l. m., & sereika, s. m. (2010). maintaining reality: relational agents for antipsychotic medication adherence. interacting with computers, 22(4), 276-288. bowern, m., burmeister, o. k., gotterbarn, d., & weckert, j. (2006). ict integrity: bringing the acs code of ethics up to date. australasian journal of information systems, 13(2), 168-181. burmeister, o. k. (2000). applying the acs code of ethics. journal of research and practice in information technology, 32(2), 107-120. burmeister, o. k. (2010). websites for seniors: cognitive accessibility. international journal of emerging technologies and society, 8(2), 99-113. burmeister, o. k., islam, m. z., dayhew, m., & crichton, m. (2015). enhancing client welfare through better communication of private mental health data between rural service providers. australasian journal of information systems, 19, 1-14. doi:http://dx.doi.org/10.3127/ajis.v19i0.1206 3 australasian journal of information systems burmeister, scott, richards, adhikari 2016, vol 20, post publication review security & safety of mobile health apps carlson, b. l., farrelly, t., frazer, r., & borthwick, f. (2015). mediating tragedy: facebook, aboriginal peoples and suicide. australasian journal of information systems, 19. doi:10.3127/ajis.v19i0.1174 cockcroft, s. (2006). information privacy: culture, legislation and user attitudes. australasian journal of information systems, 14(1). doi:10.3127/ajis.v14i1.7 nhmrc. (2004). national health and medical research council: the regulation of health information privacy in australia. retrieved from http://www.nhmrc.gov.au/_files_nhmrc/publications/attachments/nh53.pdf oaic. (2015). office of the australian information commissioner: health privacy guidance. retrieved from https://www.oaic.gov.au/engage-with-us/consultations/health-privacyguidance/ pakrasi, s., burmeister, o. k., mccallum, t. j., coppola, j. f., & loeb, g. (2015). ethical telehealth design for users with dementia. gerontechnology, 13(4), 383-387. doi:10.4017/gt.2015.13.4.002.00 ruppert, s., richards, d., arnold, s., riches, v., & parmenter, t. (2010). applying medicine 2.0 to the i-can-managing the needs and rights of end users. paper presented at the webist (2). scott, k. m., gome, g. a., richards, d., & caldwell, p. h. y. (2015). how trustworthy are apps for maternal and child health? health and technology, 4(4), 329-336. doi:10.1007/s12553-015-0099-x teipel, s., babiloni, c., hoey, j., kaye, j., kirste, t., & burmeister, o. k. (2016). information and communication technology solutions for outdoor navigation in dementia. alzheimer's & dementia: the journal of the alzheimer's association, 1-13. doi:10.1016/j.jalz.2015.11.003 van wynsberghe, a. (2015). healthcare robots: ethics, design and implementation: ashgate publishing, ltd. warren, s. d., & brandeis, l. d. (1890). the right to privacy. harvard law review, 193-220. copyright: © 2016 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. 4 http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems pham, pham, brennan & richardson 2017, vol 21, research article information security & people 1 information security and people: a conundrum for compliance hiep cong pham rmit university vietnam hiep.pham@rmit.edu.vn duy dang pham rmit university vietnam linda brennan rmit university joan richardson rmit university abstract this evaluation of end-users and it experts/managers’ attitudes towards performing it security tasks indicates important differences between their perspectives on what is and is not necessary to establish a secure corporate it environment. through a series of case studies, this research illustrates that making it easier for end-users to comply does not necessarily equate to enhanced implementation of security measures. end-users want to be autonomous, competent, self-motivated and active participants in the development of secure environments. however, managers and experts want to limit autonomy to ensure that procedures are followed closely, rather than permitting flexibility. this results in the creation of environments that are intrinsically de-motivating rather than motivating end-users to become self-determined and self-regulating co-creators of a secure it environment. the paper also discusses alternative approaches to developing a human system that works for end-users and experts. keywords: security compliance; security management; end user security behaviour 1 introduction information security management is designed to address four phases of organisational security risk management: deterrence, prevention, detection, and recovery (warkentin and willison 2009). end-user compliance plays a key role in ensuring the effectiveness of the “prevention” phase by avoiding risky practice. a majority of prior researches in the field have explored the motivation for security compliance, which consists of a wide range of internal and external factors to an individual. internal factors can be for example, self-efficacy (dang-pham and pittayachawan 2015, johnston and warkentin 2010, rhee et al. 2009) or security goal orientations (pham and nkhoma 2015), while external factors come from sanctions and rewards (d'arcy et al. 2014, herath and rao 2009a) , security culture (ruighaver et al. 2007) and climate (goo et al. 2014, dang-pham et al. 2015), as well as security demands and resources (pham et al. 2015, pham et al. 2016). security compliance researchers have conducted quantitative studies to evaluate the impact of specified factors on compliance (crossler et al. 2013, sommestad et al. 2014). however, there has been a paucity of qualitative studies designed to increase understanding of how users experience security-related issues. qualitative studies can yield rich perspectives in settings where human factors may intervene in planned outcomes. recent studies employing qualitative approaches have provided further insights into how the users experience security practices (albrechtsen and hovden 2009, pham et al. 2015, pham et al. 2016, dourish et al. 2004), and compared security perspectives between general users and security experts (albrechtsen and hovden 2009, posey et al. 2014). these studies have been useful in establishing that there are gaps between intentions and behaviours in security compliance contexts. however, it is not known if these gaps are because of (lack of) expertise or knowledge, australasian journal of information systems pham, pham, brennan & richardson 2017, vol 21, research article information security & people 2 or if something more calculated or deliberate is at the root of breaches in security. further, it is not known if end-users, experts or otherwise, share the concerns of it security managers when it comes to security compliance. a gap between manager expectations and end-user performance is unlikely to be conducive to a secure environment. our study takes a qualitative approach and explores the perceptions between average endusers and security experts/managers on five factors; namely risk evaluation, cost of compliance, formal compliance evaluation, self-efficacy, and social influences. the five studied factors were based on established behavioural theories including protection motivation theory (rogers 1975), the theory of planned behaviour (ajzen 1991) and rational choice theory (becker 1968). a number of previous studies have provided empirical evidence that the listed factors influenced security compliance (dang-pham and pittayachawan 2015, guo and yuan 2012, siponen et al. 2014, vance et al. 2012, ifinedo 2011). however, little research appears to have compared how end-users and security experts/managers perceive the impact of these factors on security compliance. understanding such diverse perspectives from both end-users and security managers can facilitate development of security programs that align with users’ expectations and improve end-users’ security compliance. 2 literature review earlier quantitative studies often employed well-founded behavioural theories to explain security compliance (sommestad et al. 2014). behavioural theories are widely applied to the information security context. factors that influence security compliance include security risk evaluation and cost of compliance from protection motivation theory (vance et al. 2012, ifinedo 2011, herath and rao 2009b), security compliance evaluation (sanctions and rewards) from rational choice theory (vance and siponen 2012, bulgurcu et al. 2010, li et al. 2010), selfefficacy and social influences from planned behaviour theories (ifinedo 2011). these theories have been found to affect security compliance to some extent in the aggregate. however, most quantitative studies did not examine whether average end-users and security experts/managers complied differently (sommestad et al. 2014). in this case aggregate data cannot adequately inform management decision making because the aggregate user is a nonexistent artefact of statistics, such as is derived when gender results are averaged to produce a mean that is neither male nor female. it is not uncommon for information security stakeholders to hold different perspectives about security issues. several studies have explored such perspectives. for instance, different perceptions of ownership over security issues amongst staff and managers were found to result in end-users’ rejection of security controls by forcefully breaking into the computer room (adams and blandford 2005). disagreements on the effectiveness of security controls and preferences between management and technical staff were also noted in mouratidis et al. (2008) where the management staff had doubts about the organisation’s network security system in spite of the security team’s confidence in the system’s effectiveness. in addition, albrechtsen and hovden (2009) found that security managers perceived end-users to be a major threat to security effectiveness, whereas the users were actually interested in contributing towards organisational security protection. security professionals and regular end-users were also found to evaluate security threats and coping measures differently in their workplaces (posey et al. 2014). for example, the managers saw the users as potential security threats whereas the users perceived security threats as external agents such as hackers and internet viruses. while these qualitative studies revealed the gap between different stakeholders, especially between end-users and security experts/managers’ perceptions of various information security issues, only posey et al. (2014) adopted protection motivation theory as a theoretical framework to guide their qualitative research. we contend that there has been a gap between the quantitative and qualitative studies of security compliance. quantitative studies have not addressed the different perceptions towards security factors among business roles of the stakeholders, while there is a paucity of established theories employed in qualitative studies and they sometimes fall into the category australasian journal of information systems pham, pham, brennan & richardson 2017, vol 21, research article information security & people 3 of description rather than theoretically founded research. hence, positing alternative strategies is problematic. this study aims to fill the gap by qualitatively examining how end-users and security experts/managers perceive five common factors that have been found to influence security compliance from several behavioural theories. the findings of the study would shed further insight into whether the two groups of users equally view these factors as effective in promoting security compliance. each of the five factors is now discussed the following sections. factor 1: security risk evaluation protection motivation theory (pmt) has been employed widely to explain how compliance could be motivated by fear of security threats (dang-pham and pittayachawan 2015, vance et al. 2012, ifinedo 2011, herath and rao 2009b). one of pmt’s main hypotheses is that adoption of protective measures is motivated by the cognitive process that evaluates the threats in terms of the degree of likelihood of an occurrence and the perceived severity of consequences (rogers 1975). in fact, this hypothesis has been supported by a number of information security studies (dang-pham and pittayachawan 2015, ifinedo 2011, vance et al. 2012). end-users would be motivated to respond to a security threat when it is evident and personally relevant to them. furthermore, end-users and security experts had different perceptions of security threats (posey et al. 2014, albrechtsen and hovden 2009). as a result, understanding how these stakeholders perceive security threats is important for security training and achieving compliance. factor 2: cost of security compliance the other main hypothesis based on pmt postulates that the end-user’s cognitive evaluations of protective security measures’ effectiveness and cost could impact on their adoption of the security compliance. employees often perceive performing security practices as an adjunct task, which is secondary to their primary work duties. hence, when there is a conflict between the need to perform security measures and primary tasks, end-users may ignore security requirements (adams and blandford 2005). the cost of compliance imposed on end-users has been identified as one of the key factors leading to non-compliance (furnell and rajendran 2012, leach 2003). furthermore, the complexity, vagueness, and overload of security tasks were found to negatively affect security compliance by causing stress and increasing moral disengagement with security programs (d'arcy et al. 2014). in other words, security compliance depends on the extent of the effort that an end-user is required to exercise and their effort depends their engagement with the requirements (e.g. moral, emotional, psychological, or behavioural). factor 3: formal security compliance evaluation general deterrence theory has been used to explain why users comply with information security policies (herath and rao 2009a). communicating the certainty and severity of sanctions for security non-compliance has been considered as an effective management strategy to prevent security non-compliance. however, inconsistent findings of the impact of sanctions have been reported (sommestad et al. 2014). for example, fear of penalties for noncompliance has been found to have a significant impact on security behaviour (herath and rao 2009a, kankanhalli et al. 2003). these studies found that if employees perceive high certainties of being caught for violating security policies, they were more likely to comply; moreover the certainty of being caught outweighs fear of the punishment severity. on the contrary, other studies found that sanctions did not have a significant impact on actual compliance (herath and rao 2009b, hu et al. 2011, pahnila et al. 2007). security compliance can also be motivated by external rewards (bulgurcu et al. 2010, padayachee 2012, ruighaver et al. 2007). there have also been inconsistent results regarding the effect of rewards on compliance. rewards were not considered a good predictor of intention to comply (sommestad et al. 2014). for instance, bulgurcu et al. (2010) found rewards, as a part of compliance’s benefits, could influence employees’ intention to comply. in contrast, pahnila et al. (2007) did not detect significant effects of rewards on actual compliance. australasian journal of information systems pham, pham, brennan & richardson 2017, vol 21, research article information security & people 4 factor 4: security self-efficacy the concept of self-efficacy is described in social cognitive theory (bandura 1977), and the theory of planned behaviour (ajzen 1991) as a component of perceived behavioural control. this construct refers to one’s self-confidence in their ability to mobilise motivation, cognitive resources, and actions needed to successfully complete a specific task within a given context. self-efficacy has been recognised as a key factor that positively influences security compliance (johnston and warkentin 2010, rhee et al. 2009). self-efficacy is also included in the pmt model and is hypothesised to affect motivation to take security actions (dang-pham and pittayachawan 2015, vance et al. 2012), and especially that knowledgeable and skilful employees are more amenable to take protective security tasks. self-efficacy is also strongly related to problem solving or solution seeking behaviours and therefore the development of competency. acquiring competency is a goal for self-determination alongside autonomy and relatedness (deci and ryan 2000). however much automation in security systems is extant in the design, at some point, human factors will intervene. consequently, a security system must be prepared for outcomes to be co-created between humans and the system that may not have been predicted at the outset, especially in a fast-paced changing environment. therefore, the design and creation of human systems to support the it system is also necessary for a secure environment. factor 5: social influences the impact of social influences on individual’s behaviours and beliefs have been widely acknowledged (cialdini and goldstein 2004, leenders 2002). social influences are often referred to as subjective norms (ajzen 1991) and can take the form of introjection motivation (gagné and deci 2005). for instance, subjective norms refer to the end-users’ beliefs about the normative expectations and social pressure that drive people’s intention to perform security behaviours, as posited in the theory of planned behaviour (ajzen 1991). similarly, selfdetermination theory (sdt) suggests that people complying with security requirements under introjection processes need to perform actions to maintain their ego which is associated with the surrounding social climate (gagné and deci 2005). the spectrum of motivating forces from amotivation through extrinsic, externally applied motivation, and then to self-motivating or intrinsic motivation (ryan and deci 2000) shifts the responsibility for compliance from ‘other’ to ‘self’. sdt argues that people can become self-motivating via a series of regulatory processes from non-regulation through external regulation and incrementally internalising the requirements of the task until they become self-motivated. in compliance, self-motivation is the ideal where people can be trusted to work within relevant parameters without surveillance, thereby decreasing costs of security. in the absence of self-motivation, extrinsic factors and other people (social influences and relatedness) motivate people. social influences can have normative and informative effects while taking forms of direct persuasions or indirect comparisons (brennan et al. 2014). it has been argued that people receive normative influence when they want to reduce ambiguity, whereas informative influence helps to clarify uncertainty (ashforth 1985). moreover, these influences can be achieved as people engage in direct communication, or indirectly compare their actions and beliefs with the others’ perspectives as they establish a social identity (leenders 2002). in an information security context, subjective norms have frequently been found to influence intention to comply with security policies and actual compliance (herath and rao 2009a). padayachee (2012) discussed that introjection could be created and maintained by an information security climate, which subsequently affected compliance. 3 research method 3.1 research design the main research question of this study was how average end-users and security experts in organisations perceive the impact of the following factors on end-user security compliance. 1. security risk evaluation australasian journal of information systems pham, pham, brennan & richardson 2017, vol 21, research article information security & people 5 2. cost of security compliance 3. formal security compliance evaluation 4. security self-efficacy 5. social influences a case study method was employed to investigate this question. the case study method is commonly described as an empirical enquiry suitable for investigating phenomena in their natural context, especially when the research topics are new and changing fast (dubé and paré 2003, yin 2009). this research investigated the management of information security as an important topic in the information systems research domain which has been evolving rapidly over past decades (dubé and paré 2003). moreover, the study involves the investigation of the complex relationship between multiple organisational stakeholders with different skills and knowledge. as a result, the case study method is appropriate for our research objective, which aims to examine in-depth the different perspectives towards security compliance between endusers and security experts/managers. our case study’s nature is descriptive and consists of multiple cases including two distinctive groups of end-users and security experts. rather than aiming at interpreting the phenomena, descriptive case studies present the phenomena as they are in an objective fashion (dubé and paré 2003) and multiple-case design allows increased generalisability of the results (yin 2009). our case study design is described in table 1 below. dimensions security perspectives cases security factor 1 … security factor 5 u n it security experts/managers security expert/manager 1–7 differences in perspectives amongst security experts/managers and end-users regarding each dimension end-users end-user 1–16 observations table 1: case study design 3.2 context to ensure the rigour of results derived from a descriptive case study method, there are criteria that need to be addressed in each stage of the research. the first criterion requires that the researchers to explicitly describe the context of the case study (dubé and paré 2003). our study took place in vietnam, a country in the south east asia region, which is rapidly transforming into the world’s it sourcing hub. the overall information security landscape in vietnam is still in its infancy. for instance, the recent information and communication technology white book of the vietnamese ministry of information and communication reveals that in 2013 only 27.5 per cent of companies had published their information security policies while 21.7 per cent had implemented prescribed processes to handle information security matters (vietnam-mic. 2014). funk and garnaeva (2013) listed vietnam in the top four countries in the world in terms of the highest risk of cyber threats. 3.3 data collection informative cases are critical for a case study method given the small sample available for study. to conform to the complex nature of information security management and add value to the study outcome, our cases were drawn from a pool of participants from diverse industries and it environments. participants were recruited by sending invitations to personal contacts, as well as professional forums on social platforms such as facebook and linkedin. the participants who agreed to take part in the research were screened to ensure that their companies had security policies and required security compliance at work (regardless of whether they had published it policies according to the ict white book. sixteen end-users and seven security experts and managers across a range of demographics took part in the australasian journal of information systems pham, pham, brennan & richardson 2017, vol 21, research article information security & people 6 interviews. the profiles of the participants are summarised in table 2. all 23 semi-structured interviews took one hour on average and were conducted in vietnamese or english, dependent on the native-speaking background of the participant and were audio recorded. end-user (u) security manager/expert (e) occupation industry u1–6 counter teller banking u7–8 accountant u9–12 university lecturers education u13–14 admin staff u15–16 marketing executive oil distribution e1 it auditor/consultant financial e2 it manager it services e3 security consultant banking e4 security officer it services e5 deputy it director banking e6 data security manager engineering e7 it director education table 2: profiles of interviewees 3.4 data analysis nvivo 10 was used to analyse interview transcripts by following yin's (2009) procedures, which include conducting within and cross case analysis in order to detect matching patterns to the predefined factors. the pre-determined five factors covered important components of data analysis, in which the interviewed data was classified to match with each identified factor description from prior literature and theories (strauss and corbin 1998). the five factors were then compared and discussed with coded data. to ensure reliability, a brief research report of the key findings was sent to the interviewees who were asked for their feedback, as well as verifying the coding with an it expert who has more than 15 years of teaching it disciplines. 4 findings of the study 4.1 factor 1: security risk evaluation security risk evaluation refers to the assessment of risk likelihood and severity. the security experts and end-users in our study held different perspectives about the significance of security risk evaluation. the majority of interviewed end-users reported that evaluating security risks consumed too much of their effort and time, as well as requiring skills that they did not have to perform the activity. “i can click on the warning message to cancel or run the application, but thinking about what security risks might happen if i run it would be too much to handle.” (u3, counter teller) “it does not matter to me much as i don’t have enough expertise and knowledge to assess the effectiveness of security tasks and the risks.” (u13, university admin staff) end-users explained their focus on the severity of security risks, which had direct and immediate impacts on their jobs. risks that affected the whole department or organisation were not obvious to these end-users. the security experts also shared this view such as e1 (it auditor) and e2 (it manager) who contended that only high-level executives had the ability to assess organisational risks and transform them into strategic and tactical directives. some security experts only expected employees to understand and comply with prescribed security procedures. e2 (it manager) reasoned that not many regular end-users would care about security risks that affected the company’s interests (e.g. reputation damage) because the impact of those high level threats were not relevant to their job. as a result, improving the australasian journal of information systems pham, pham, brennan & richardson 2017, vol 21, research article information security & people 7 relevance of the risk communication was suggested to be important in designing information security training: “the problem is like this, when you are the end-user and someone said: ‘if you don’t comply with information security then it will risk the reputation of the company.’ who cares? ‘in the worst case i’d just quit the company, not my business.’ that’s how most of them would think like.” (e2, it manager) both the interviewed experts and end-users agreed that communication of risks should explain clearly the impact on the target audience’s daily tasks. while the experts suggested using real life scenarios in security training sessions, the end-users agreed that they determined the risk’s likelihood according to experience of real incidents. in addition, both groups agreed that frequent reminders were necessary and useful for developing a risk aware climate and improving security compliance. the interviewed end-users also wanted to understand how implemented measures could prevent security threats, and suggested that having such knowledge would improve their risk evaluation. however, none of the interviewed experts mentioned explaining to the end-users the effectiveness of security measures. albrechtsen and hovden (2009) discussed that security managers may focus more on the risk’s likelihood while end-users reported paying more attention to the consequences of noncompliance. we found evidence that supported the different perceptions of security risks. the interviewed experts explained the opinions differed as a result of varied levels of perceived ownership of tasks in terms of their capacity to complete assigned tasks. this explanation supported findings from adams and blandford (2005) and posey et al. (2014) that there would be discrepancies in the security experts and end-users’ ownership of the security compliance activities, as well as their perceptions of the severity and likelihood of security threats. 4.2 factor 2: cost of security compliance both groups of participants agreed that compliance cost is inevitable and may not be avoided. a majority of the experts agreed that the only way to reduce response cost was to collaborate closely with department heads and incorporating their feedback when designing new security measures and procedures: “there is no way that information security is comfortable, it is simply a trade-off of being secure and other things.” (e2, it manager) “it can control what people can do with it resources, but if it control starts to affect productivity then it should open whatever the users need to do. it should not be limiting the productivity. if it needs to do something, it is business policy, not it intention to do that for their own benefits.” (e7, it director) likewise, the end-users acknowledged that the cost of compliance was in terms of consumed time, even when security tasks were automated (e.g. auto updates and backup) and it assistance was available. realisation of response cost was also found to be associated with low perception of risks: “for me as a user i understand that security compliance is only complying with organisation’s requirements. however, i do not see the problem–why i need to do that. for the users, i find it wastes too much time.” (u9, university lecturer) “checking laptop hardware and security audit could take half a day. it is annoying and it is really a burden for us. (u15, marketing executive) “the security task is time consuming. some require just a couple of minutes of time while others virtually take away my time. forgetting to change the password after the expiration date, teller is unable to log in the computer hence acquiring the assistance from it to unlock the account. due to the traffic of the bank, too crowded and sometimes multitasks, i could not change password immediately. (u3, counter teller) australasian journal of information systems pham, pham, brennan & richardson 2017, vol 21, research article information security & people 8 the other interviewed users explained they would comply with security requirements regardless of the cost since compliance was required as a part of their job. this is in contrast to posey et al. (2014)’s findings that the experts underestimated how much their end-users perceived compliance cost as an issue. the interviewed experts appeared to be more sympathetic to the end-users’ challenges of achieving work productivity bound by security constraints. however, the shared perceptions of compliance cost as a contributing factor to compliance emphasised its inevitable nature and suggested that its effects could only be reduced by other factors but not eliminated. 4.3 security compliance evaluation security compliance evaluation in our study refers to the weighting of rewards and sanctions for compliance versus non-compliance. we found discrepancies in how the interviewed experts and end-users perceived these rewards and sanctions. for instance, most end-users agreed that they would appreciate rewards such as recognitions or certificates as a result of performing compliant behaviour: “we need some announcements about what we did. maybe the it department, the general director, or department head can send out an announcement or letter of appreciation to acknowledge individuals who have done well in security. so that we can feel pride and we keep doing that.” (u15, marketing executive) “but in short term, there should be a clear “award – punishment”, gradually, it will become self-consciousness. in the long term, we will aim for the improvement in each individual’s consciousness as if we require that from the beginning, that would be very difficult.” (u7, bank accountant) on the contrary, only two experts recommended the use of rewards to encourage compliance, whereas the rest argued against it. experts who objected to the use of rewards claimed that compliance was a part of the job’s duties, which would be expected by default rather than a voluntary choice that required stimulus. second, expert e6 (data security manager) and e7 (branch manager) expanded that they would avoid making information security compliance a competition with a reward system, as that would overly motivate employees to take excessive security protections and could interfere with work productivity. “it is a commitment when you work somewhere and make sure that nothing… happen at the company you work for. if you ask for a reward for doing security, you are in the wrong mindset. the mindset is if you work somewhere, you are responsible and you should be compliant with any security requirements and use common sense to make sure that nobody steals your information. if you want a reward for that, i think it is wrong” and … “the reward also has a counter effect. when people are rewarded for their security effort, they would want to do more and more secure it. then it would reach a level where too much security and you can’t work anymore. or they try to secure many things, they may make mistakes and they may not work because they are not expert” (e7, it director) in fact, most of the experts believed that the only reward for compliance was being able to avoid punishments and liability when a security breach occurs. furthermore, e4 (security officer) justified the reward of employees’ compliance as mutually linked with the organisation’s interests: “all benefits of compliance should be explained as belonging to the company. the reason is that when the company receives the benefits, which would be shared with the employees. for example, the company could be trusted more by our clients, and they would give us more projects. the company’s revenue would subsequently increase, and so would the bonus of the employees, if that is considered their personal gain.” (e4, security officer) australasian journal of information systems pham, pham, brennan & richardson 2017, vol 21, research article information security & people 9 regarding the use of sanctions, most of our interviewed end-users acknowledged that their workplace did not implement any sanction schemes for non-compliance. on the one hand, one end-user who worked in the banking sector supported raising awareness of sanctions as he was concerned over financial risks that could result in serious consequences and should therefore be prevented. on the other hand, we found that all experts rejected using punishments for promoting compliance. however, respondents agreed that sanctions discouraged noncompliance. one expert explained that sanctions were more suitable to punish employees’ security breaches and not suitable for proactively minimising security mistakes. another expert (e3, security consultant) reasoned that white-collar workers would perceive sanction measures as a personal threat, which may result in even more resistance: “i always told my clients to never resort to sanctions. no matter how they look at it, the information security incidents that occurred are already in the past. the primary purpose of the security programs is to prevent incidents from happening, not to find ways to deal with things that already happened.” (e3, security consultant) unlike posey et al.'s (2014) finding that end-users thought more about intrinsic rewards while experts focused on extrinsic ones, we found that both of our interviewed groups recognised extrinsic rewards such as recognition and maintaining professional image as effective in motivating compliance. a potential explanation could be that information security has not yet been integrated into the organisational culture in our case studies, so compliance was still heavily dependent on the use of extrinsic incentives. in addition, our results reflected albrechtsen and hovden's (2009) findings that experts avoided imposing sanctions because they did not consider their role to be policemen or janitors. 4.4 security self-efficacy the possession of security knowledge and skills is essential to increase one’s ability to perform required security tasks. posey et al.'s (2014) study also reports that security experts were concerned about end-users’ lack of security knowledge and considered knowledge and skill shortage as a major threat to security programs. in our cases, most end-users perceived security compliance as simple and straightforward. end-users explained security compliance as following routines, such as changing passwords, locking computers, or not sharing computer accounts: “the security procedures are quite easy to follow. as long as i just follow instructions and not touch on the it parts or don’t make mistake on the it things.” (u15, marketing executive) all interviewed end-users reported that they were not required to perform any complex security tasks that needed special training. some end-users admitted that they did not know what skills they were lacking until they were asked to do complex security tasks, such as verifying potential spoofing attacks in email or malicious websites. most of them argued that security knowledge for regular end-users should be easy to understand and apply, whereas acquiring more advanced security skills ones should belong to it specialists and not in the endusers’ interest. similar to the opinions of end-users, the interviewed experts only expected their end-users to follow step-by-step instructions as prescribed in the policies. it was consistent between both interviewed groups that security skills were essential for security compliance, however, the challenge was how much security skill an end-user should acquire for effective compliance. the interviewed experts in our sample were concerned that highly skilful end-users may pose a deliberate threat, as they understand the information systems’ vulnerabilities. expert e7 (it director) was reluctant to educate end-users about the security infrastructure due to a fear that they could intentionally or by mistake breach the security systems. e7 (it director) also lacked trust in the end-users in regards to proper use of their security self-efficacy. australasian journal of information systems pham, pham, brennan & richardson 2017, vol 21, research article information security & people 10 “some end-users can set up own wi-fi network at work or home. there is no password, security protection on the wi-fi. anyone can access their own network. people think they know but actually they don’t.” (e7, it director) as a result, this raises questions about what types of security knowledge and system configurations should be included in training to ensure safe and secure information security systems in the organisation. as highlighted in albrechtsen and hovden (2009), certain types of knowledge should be excluded from security training. however, excluding security specific information from training may also lead to end-users having low confidence in their security response efficacy. 4.5 social influences while the interviewed experts confirmed the important role of social norms in affecting individuals’ attitudes and behaviours, they recommended not to rely on organisational norms to encourage compliance. expert e6 (data security manager) justified that employees with high information security awareness would be able to perform secure practices correctly even when the norms suggested alternative actions. another expert e3 (security consultant) was concerned that organisational norms may create strong subcultures that enact information security practices inconsistently among departments, especially in hierarchical organisations where autonomous work units were separate from top management executives’ reach: “sometimes in vietnamese firms the policy is announced from the higher level but enacted differently within the departments. especially when departments can spend their allocated budget however… people in those departments would just care about what their direct supervisors do but not the distant corporate levels.” (e1, it auditor) moreover, the experts agreed that only managers with formal authority would be suitable for persuading compliance. in their expert roles, designated managers were required to possess leadership and communication skills, and especially proficiency in information security: “we don’t need influential opinion leaders in communicating information security. because the nature of information security is being predominantly perceived as highly technical, so you need to have a technical person to gain people’s trust. then those people would listen to what that person says or trains them.” (e3, security consultant) nevertheless, some interviewed end-users acknowledged that they usually sought assistance regarding information security matters from nearby colleagues rather than from the information technology (it) department. a team’s common practices, access convenience and response timeliness were reported as the main reasons for choosing to request information from a colleague rather than the it staff. as team members operated on the same systems and handled similar security tasks, they formed local practices that all members followed. when dealing with unfamiliar security incidents, the end-users found it too time consuming and inconvenient to seek advice through formal channels, such as it help desk or policies. both university admin staff (u13 and u14) further suggested having area-specific champions who were active in updating security initiatives in the organisation and informing other team members of new tasks was advantageous. “because sometimes, it is like, we also know that it is important but there are so many things to do, and then, sometimes they (i.e. it staff) are not very available, so the more people know about that, the better. especially the one who is close to you, so you could come and see that person for security advice” (u13 and u14, university admin staff) 5 discussion overall the results of this study show an interesting conundrum for it security practice. on the one hand managers are mostly interested in compliance: passive and procedural. they also have concerns with end-users becoming too qualified and representing a danger to the system. australasian journal of information systems pham, pham, brennan & richardson 2017, vol 21, research article information security & people 11 as such their actions to enhance security are aimed at defining systems that limit expertise and decrease self-efficacy and skill development. as a method of illustrating this dilemma, figure 1 depicts ryan and deci's self-determination continuum, which we believe highlights the gap between experts’ views of compliance and end-users’ views. the experts are applying external regulation; extrinsic motivations and focusing on compliance by way of externally applied rewards and punishments. figure 1: self-determination theory (adapted from ryan and deci (2000)) on the other hand, the end-users are seeking to be self-determined; relying on intrinsic motivation and being engaged in it security for interest, enjoyment and inherent satisfaction. they want to be active participants in developing skills that enable and empower them, not passive receivers of procedural information. the challenge for those managing it security is to design a human system that allows for self-determination in compliance without increasing the risk to the organisation. 5.1 improving risk communications between information security stakeholders to enhance the likelihood that end-users will internalise security requirements, the regulatory processes need to shift from mere compliance (other directed) towards interest, enjoyment and inherent satisfaction (self-directed). security managers need to tailor risk communication to emphasise on the consequences of the risks on daily operations rather than merely their likelihood, thereby increasing congruence and awareness of consequences. outcomes of security breaches should be directly connected to the end-users’ daily work: increasing personal importance. in addition, the effectiveness of the security controls needs to be clearly explained to end-users since perceived response efficacy was found to reduce compliance cost significantly (dang-pham and pittayachawan 2015). we emphasise that educating the endusers about the security measures’ effectiveness needs more attention from security practitioners. this will increase the conscious valuing of the it system and improve the chances that the demands of it security will be accepted, synthesised and internalised. communication of security issues can come from departmental security champions. trained security champions are end-users who possess domain-specific security knowledge and can provide timely and accessible assistance to other users within a department. prior studies have found the availability of information security resources has a positive impact on security compliance (chan et al. 2005, goo et al. 2014, pham et al. 2015). while many organisations australasian journal of information systems pham, pham, brennan & richardson 2017, vol 21, research article information security & people 12 cannot afford running frequent training programs and sending reminders are not effective in improving end-users’ compliance, a team of dedicated security champions on site compensates for drawbacks, such as minimal reminders and a lack of training. self-determined people can and will help themselves if the need to do so is clear and evident and the resources are available to assist. as internal it security principally involves people not doing things, or doing everyday things safely, large-scale resources at the end-user level are not usually necessary. although, that assumes that the it infrastructure is secure in the first place. 5.2 implementing the right incentives in moving people along the spectrum towards self-determination, the incentives and rewards need to become less externally applied and more intrinsic. however, as our results show, experts believe that the only motivators that ‘work’ for compliance are external and punitive rather than rewarding. they do not believe that end-users are, or can be, intrinsically motivated, although the results of this study contest that view. as external regulations, such as tangible rewards and strict sanctions can have mixed impacts on security behaviour (siponen et al. 2014, vance and siponen 2012) and only produce short-term effects resulting from poorquality motivation (stone et al. 2008), other forms of incentives should be employed. for example, formal recognition of security skills development can intrinsically motivate end-users to maintain their efficacy and practice recommended security tasks. incentives for compliance should not promote over-protecting security activities but get the end-users interested and engaged with security practice on a regular basis. such incentives should make the end-users find security skills and practices are essential work skills, not just simply following prescribed steps in the security policies. 5.3 security training to provide value-added security skills in order to assist the transition to self-determined and self-motivated end-users, consideration of the various types of regulatory processes is necessary. using an externally derived, punitive system will not help people internalise the requirements and adopt them as a sustainable form of behaviour. as figure 1 illustrates, the gap between mere compliance and self-motivation is a large one. the intervening steps incrementally increase ‘self’ control and decrease the locus of ‘other’ when it comes to motivational stimuli (i.e., causality). in order to foster the required shifts, security training should provide adequate skills to end-users without encouraging risky security behaviour due to over confidence in their capability. management may just provide end-users with adequate skills to follow prescribed security policies; however, end-users want to see value in the acquisition of security skills beyond simple policy and process compliance. security training could clearly communicate security objectives and the significance of security protection to individual’s job and the whole organisation. thereby, focusing on why it is important and increasing the sense of value that compliance contributes to the individual’s daily activities. similarly, security skills should be portrayed as critical to one’s competence development and therefore seen as something desirable, as well as something that increases one’s enjoyment and job satisfaction; rather than an onerous distraction from the primary role at work. in addition, employees should be aware that higher security skills may go with higher responsibilities and they can significantly contribute to the success of overall security programs in the organisation. end-users should not be treated as a potential threat, which often results in cautious approach to security training (albrechtsen and hovden 2009). instead, they can be self-motivated allies in protecting the organisation in an insecure business environment. 6 conclusion our study employed a multi-case approach and described the perspectives of information security experts/managers and end-users on the impact of risk evaluation, rewards and sanctions, security self-efficacy and social influences on individuals’ security compliance. distinct sets of beliefs were found between the two groups, except both groups agreed on the inevitable nature of compliance cost. several practical lessons learned could be drawn from the findings. given that there is a gap between expert and managers’ views of security and endusers’ views, this study highlights that establishing compliance may need to take into account australasian journal of information systems pham, pham, brennan & richardson 2017, vol 21, research article information security & people 13 the different perspectives of end-users and experts or managers. firstly, risk communication and a reward system should be implemented to promote risk-aware culture and recognition of end-users’ security expertise that empowers and enables intrinsic self-regulation. secondly, security training needs to balance between providing necessary skills and discouraging overconfident risk taking behaviour while still permitting the development of competences that lead to satisfaction with the role. in addition, our findings clearly explained the five theoretical constructs from protection motivation theory, theory of planned behaviour and general deterrence theory in the context of behavioural security compliance. finally, this study contributes empirical findings in a non-western context to the current body of knowledge in the information security domain, although it is feasible that these issues exist in western it security contexts as well. references adams, a. & blandford, a. (2005) "bridging the gap between organizational and user perspectives of security in the clinical domain", international journal of human computer studies, 63(1-2):175–202 doi:10.1016/j.ijhcs.2005.04.022. ajzen, i. (1991) "theory of planned behavior", organizational behavior and human decision processes, 50(2):179–211. doi:10.1016/0749-5978(91)90020-t. albrechtsen, e. & hovden, j. (2009) "the information security digital divide between information security managers and users", computers & security, 28(6):476–490 10.1016/j.cose.2009.01.003. ashforth, b. (1985) "climate formation: issues and extensions", academy of management review, 10(4):837–847 bandura, a. (1977) "self-efficacy: toward a unifying theory of behavioral change", psychological review, 84(2):191–215 http://dx.doi.org/10.1037/0033-295x.84.2.191. becker, g. s. (1968) "crime and punishment: an economic approach", journal of political economy, 76(2):169 brennan, l., binney, w., parker, l., nguyen, d. & aleti, t. (2014). theories and their uses in social marketing, social marketing and behaviour change: models, theory and applications. cheltenham, glos gl50 2ja, uk: edward elgar publishing. bulgurcu, b., cavusoglu, h. & benbasat, i. (2010) "information security policy compliance: an empirical study of rationality-based beliefs and information security awareness", mis quarterly, 34(3):523-548 chan, m., woon, i. & kankanhalli, a. (2005) "perceptions of information security at the workplace: linking information security climate to compliant behavior", journal of information privacy and security, 1(3) doi: 10.1080/15536548.2005.10855772. cialdini, r. b. & goldstein, n. j. (2004) "social influence: compliance and conformity", annual review of psychology , 55:591–621 doi: 10.1146/annurev.psych.55.090902.142015. crossler, r. e., johnston, a. c., lowry, p. b., hud, q., warkentin, m. & baskerville, r. (2013) "future directions for behavioral information security research", computer & security, 32:90-101 doi:10.1016/j.cose.2012.09.010. d'arcy, j., herath, t. & shoss, m. k. (2014) "understanding employee responses to stressful information security requirements: a coping perspective", journal of management information systems, 31(2):285-318 doi: 10.2753/mis0742-1222310210. dang-pham, d. & pittayachawan, s. (2015) "comparing intention to avoid malware across contexts in a byod-enabled australian university: a protection motivation theory approach", computers & security, 48:281–297 doi:10.1016/j.cose.2014.11.002. australasian journal of information systems pham, pham, brennan & richardson 2017, vol 21, research article information security & people 14 dang-pham, d., pittayachawan, s. & bruno, v. investigating the formation of information security climate perceptions with social network analysis: a research proposal. 19th pacific asia conference on information systems (pacis), 2015 singapore. deci, e. l. & ryan, r. m. (2000) "the “what” and “why” of goal pursuits: human needs and the self-determination of behavior", psychological inquiry, 11:227-268 dourish, p., grinter, r. e., flor, j. d. d. l. & joseph, m. (2004) "security in the wild: user strategies for managing security as an everyday, practical problem", personal ubiquitous computing, 8:391-401 doi: 10.1007/s00779-004-0308-5. dubé, l. & paré, g. (2003) "rigor in information systems positivist case research: current practices, trends, and recommendations", mis quarterly, 27(4):597–635 funk, c. & garnaeva, m., (2013), kaspersky security bulletin 2013 [online]: securelist. available: https://securelist.com/analysis/kaspersky-securitybulletin/58265/kaspersky-security-bulletin-2013-overall-statistics-for-2013/ [accessed 10 may 2016]. furnell, s. & rajendran, a. (2012) "understanding the influences on information security behaviour", computer fraud & security, 2012(3):12-15 doi: 10.1016/s13613723(12)70053-2. gagné, m. & deci, e. l. (2005) "self-determination theory and work motivation", journal of organizational behavior, 26:331-362 goo, j., yim, m. & kim, d. (2014) "a path to successful management of employee security compliance: an empirical study of information security climate", ieee transactions on professional communication, 57(4):1–24 guo, k. h. & yuan, y. (2012) "the effects of multilevel sanctions on information security violations: a mediating model", information & management, 49:320-326 herath, t. & rao, h. r. (2009a) "encouraging information security behaviors in organizations: role of penalties, pressures and perceived effectiveness", decision support systems, 47:154-165 doi:10.1016/j.dss.2009.02.005. herath, t. & rao, h. r. (2009b) "protection motivation and deterrence: a framework for security policy compliance in organisations", european journal of information systems, 18:106-125 doi:10.1057/ejis.2009.6. hu, q., xu, z. c., dinev, t. & ling, h. (2011) "does deterrence work in reducing information security policy abuse by employees?", communications of the acm, 54(6):54-60 ifinedo, p. (2011) "understanding information systems security policy compliance: an integration of the theory of planned behavior and the protection motivation theory", computers & security, 31:83-95 doi:10.1016/j.cose.2011.10.007. johnston, a. c. & warkentin, m. (2010) "fear appeals and information security behaviors: an empirical study", management information systems quarterly, 34(3):549-566 kankanhalli, a., teo, h.-h., tan, b. c. y. & wei, k.-k. (2003) "an integrative study of information systems security effectiveness", international journal of information management, 23:139-154 leach, j. (2003) "improving user security behaviour", computers & security, 22(8) doi:10.1016/j.cose.2011.10.007. leenders, r. t. a. j. (2002) "modeling social influence through network autocorrelation: constructing the weight matrix", social networks, 24(1):21-47 doi:10.1016/s03788733(01)00049-1. li, h., zhang, j. & sarathy, r. (2010) "understanding compliance with internet use policy from the perspective of rational choice theory", decision support systems, 48:635-645 australasian journal of information systems pham, pham, brennan & richardson 2017, vol 21, research article information security & people 15 mouratidis, h., jahankhani, h. & nkhoma, m. z. (2008) "management versus security specialists: an empirical study on security related perceptions", information management & computer security, 16(2):187–205 padayachee, k. (2012) "taxonomy of compliant information security behavior", computer & security, 31:673-680 doi:10.1016/j.cose.2012.04.004. pahnila, s., siponen, m. & mahmood, a. (2007). employees’ behavior towards is security policy compliance. the 40th hawaii international conference on system sciences. pham, c. h., el-den, j. & richardson, j. influence of security compliance demands and resources on security compliance-an exploratory study in vietnam. pacific asia conference on information systems (pacis 2015), 2015 singapore. pham, c. h. & nkhoma, m. (2015) "security compliance-new insight from goal orientations and self-regulation theory", journal of systemics, cybernetics and informatics (jsci), 13(3):56-61 pham, h.-c., el-den, j. & richardson, j. (2016) "stress-based security compliance model-an exploratory study", journal of information and computer security, 24(4) posey, c., roberts, t. l., lowry, p. b. & hightower, r. t. (2014) "bridging the divide: a qualitative comparison of information security thought patterns between information security professionals and ordinary organizational insiders", information and management, 51(5):551–567 doi:10.1016/j.im.2014.03.009. rhee, h.-s., kim, c. & ryu, y. u. (2009) "self-efficacy in information security: its influence on end users’ information security practice behavior", computer & security, 28:816826 rogers, r. w. (1975) "a protection motivation theory of fear appeals and attitude change", journal of psychology, 91(1):93-114 doi:10.1080/00223980.1975.9915803. ruighaver, a. b., maynard, s. b. & chan, m. (2007) "organisational security culture: extending the end-user perspective", computer & security, 26:56-62 ryan, r. m. & deci, e. l. (2000) "self-determination theory and the facilitation of intrinsic motivation, social development, and well-being", american psychologist, 55(1):68-78 doi:10.1016/j.cose.2006.10.008. siponen, m., mahmood, m. a. & pahnila, s. (2014) "employee's adherence to information security policies: an exploratory field study", information & management, 51:217-224 doi:10.1016/j.im.2013.08.006. sommestad, t., hallberg, j., lundholm, k. & bengtsson, j. (2014) "variables influencing information security policy compliance: a systematic review of quantitative studies", information management & computer security, 22(1):42–75 http://dx.doi.org/10.1108/imcs-08-2012-0045. stone, d. n., deci, e. l. & ryan, r. m. (2008) "beyond talk: creating autonomous motivation through self-determination theory", journal of general management, 34(3):75–91 strauss, a. & corbin, j. (1998) basics of qualitative research, thousand oaks, ca, sage. vance, a. & siponen, m. (2012) "is security policy violations: a rational choice perspective", journal of organizational and end user computing, 24(1):21-41 doi: 10.4018/joeuc.2012010102. vance, a., siponen, m. & pahnila, s. (2012) "motivating is security compliance: insights from habit and protection motivation theory", information & management, 49:190-198 doi:10.1016/j.im.2012.04.002. vietnam-mic, 2014, vietnam information and data on information and communication technology whitebook 2014 [online], hanoi, vietnam. available: australasian journal of information systems pham, pham, brennan & richardson 2017, vol 21, research article information security & people 16 http://english.mic.gov.vn/upload/store/tintuc/vietnam/43/sach-trang-2014final.pdf [accessed 10 may 2016]. warkentin, m. & willison, r. (2009) "behavioral and policy issues in information systems security: the insider threat", european journal of information systems, 18(2):101– 105 yin, r. k. (2009) case study research: design and methods, thousand oaks, ca, sage. copyright: © 2017 pham, pham, brennan & richardson. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems karippur & balaramachandran 2022, vol 26, research article antecedents of effective digital leadership 1 antecedents of effective digital leadership of enterprises in asia pacific nanda kumar karippur s p jain school of global management, singapore pushpa rani balaramachandran s p jain school of global management, singapore pushparani.b@spjain.org abstract increasing awareness of digital transformation across asia pacific is putting a strong spotlight on how enterprises think of leadership and related key practices. the pace of transition to the new digital organisation creates even larger leadership gaps as most organisations have not moved rapidly enough to develop relevant leadership practices. with the support of literature, an analysis of related theories of transformational leadership, ambidexterity and dynamic capabilities are examined and the most suitable leadership attributes and practices for a digital enterprise is proposed. we test the proposed research model via a sample of leaders and senior managers from the asia pacific region. the results highlight the significant roles of leadership attributes, strategic priorities, organisational focus areas for exploration and digital governance practices for exploitation, in influencing effective digital leadership. the article reports valuable insights and relevant implications for leaders, enterprises, and researchers. keywords: ambidexterity, digital leadership, digital transformation, digital governance, dynamic capabilities, transformational leadership. 1 introduction the current era of ubiquitous digital transformation affects most business organisations. digital transformation involves leveraging emerging digital technologies such as mobile, cloud, data analytics, artificial intelligence, and internet of things (iot) technologies to enable business transformation and create new business models (warner & wäger, 2019). it is about envisioning how organisations bring together people, data, and processes and provide valueadded services to their customers and sustain a competitive advantage in the digital world (jimenez, lim, cheok, & ng, 2018). digital enterprises are organisations that harness the power of latest digital technologies to improve business performance, recognize the need to be agile, search and rethink new business models, foster digital talent and adopt a collaborative approach and culture (warner & wäger, 2019). the focus of this study is digital transformation in the asia pacific region which is expected to accelerate over the next three years and the region is poised to receive the economic rewards. asia pacific’s spending on tech was over us $380 billion in 2019 (huawei, 2020). with accelerated investment in digital transformation, international data corporation (2020) predicts over 65% of asia pacific gross domestic product (gdp) will be digitalised and spending to hit us$ 1.2 trillion between 2020 and 2023. the asia pacific region will account for over 50% of the global internet population and is expected to reach 72% of the regional population by 2023 (cisco, 2020). the mobile internet user penetration in the region was at 48% in 2019 with over two billion mobile internet users and is forecasted to reach 61% in 2025 (statista research department, 2021). seizing the full gamut of digital opportunities, however, australasian journal of information systems karippur & balaramachandran 2022, vol 26, research article antecedents of effective digital leadership 2 will require leadership commitment and dedication to a digital-first approach and a changed mindset that embraces collaboration, new skill acquisition and a rethinking of the manmachine relationship (bahl, 2016; tan, 2020). accelerating digital transformation in asia pacific is marked by enterprises' increasing initiatives to digitalize operations, empower employees, improve productivity, and engage customers (jimenez et al., 2018). over 43% of nearly 600 respondents in the asia pacific region indicated digital transformation as their top priority (tan, 2020). according to a 2018 asia pacific report (jimenez et al., 2018), it is expected that by 2021, digital transformation will add 0.8% annually to asia pacific's gdp, or us$1.16 trillion, with 60% of the region's gdp will be from digital products or services generated through digital transformation. jimenez et al. (2018) indicate that organisations can be categorised into two groups according to their digital transformation journeys; digital leaders and digital followers. digital leaders have a clearer holistic vision, digital strategy and achieve higher returns from their digital transformation projects. over 85% of asia pacific organisations surveyed in 2017 (jimenez et al., 2018) have embarked on their digital transformation journeys, of which, only 7% were classified as digital leaders. the study also showed that digital leaders reaped double the benefits of digital followers and identified five organisational traits of digital leaders; risktaking culture, flexibility and adaptability, collaboration with employees and partners, and alignment of organisation’s strategies. with the increased awareness of digital transformation in asia pacific, there is a need to focus on factors influencing effective digital leadership. digital transformation requires competent leaders, who, with their knowledge and experience, would be able to drive the change to integrate technology and business organisations toward innovation and business transformation to achieve successful outcomes (dahlström, desmet, & singer, 2017; world economic forum, 2018). organisations that are digital leaders strive to develop new digital capabilities and are making significant changes to their strategy-making process and organisational culture (della corte, del gaudio, & sepe, 2019). leadership theory has been evolving from a modern, static leadership theory to increased recognition of the expansive resources and the organisational relationships involved in leadership practices (raelin, 2016). digital enterprises require top to bottom organisational transformation (sow & aborbie, 2018). digital transformation is about competitiveness, new challenges, and innovation and how value can be created from digitalisation and establishing new ways of communication and collaboration (oberer & erkollar, 2018). hence, it is crucial for business leaders to adopt an effective leadership style and have the right capabilities to inspire their employees to innovate and sustain in the digital age. this research focused on the following question: what are the antecedents of effective digital leadership of enterprises in asia pacific? increasing awareness of digital transformation across asia pacific is putting a strong spotlight on how organisations think of leadership and related key practices. what is expected from successful leadership in the digital transformation era? there are significant research gaps in the areas of leadership attributes and practices leading to digital leadership. successful digital transformation requires top management support and digital transformation is intrinsically cross-functional and will require new leadership practices to transform the organisation (yokoi, shan, wade, & macaulay, 2019). the building of leadership capabilities for digital transformation has received limited scholarly attention (warner & wäger, 2019). not many empirical studies exist on leadership in the context of australasian journal of information systems karippur & balaramachandran 2022, vol 26, research article antecedents of effective digital leadership 3 digital business transformation. however, organisational-change leadership is a widely researched topic with vast literature that provides many useful ideas on digital business transformation (larjovuori, bordi, & heikkilä-tammi, 2018). the importance and scale of a digital transformation are so great that business leaders require to have certain attributes, capabilities, collaboration and support from top management and employees to effectively lead transformation successfully. this study focuses on the factors influencing effective leadership for digital enterprises such as leadership attributes, strategic priorities, organisational focus areas for exploration and digital governance practices. our research follows a quantitative approach with a strong empirical analysis to identify the key factors influencing effective leadership for digital enterprises. the next section will be a review of the related literature in order to develop a better understanding of the variables involved and their relation to the dependent variable. section three will explain the proposed research model and hypotheses development for each of the identified variable, while section four will describe the data collection and analysis followed by section five on the discussion of the findings. section six will elaborate on the implications for research and practice. finally, the limitations, scope for future research and conclusion are stated at the end of the paper. 2 literature review 2.1 digital enterprise a digital enterprise is an organisation that has a clear digital vision, high investment in digitalisation with focus on factors such as innovation culture, organisational agility, and nurturing talent to drive and deliver digital strategy (della corte et al., 2019). new digital organisations have evolved with the development of social media and mobile networking, big data, cloud computing and embracing of fourth industrial revolution (bongiorno, rizzo, & vaia, 2018). the most recent trends and directions of digital transformation are in the areas of artificial intelligence (ai) and machine learning (ml), internet of things (iot), analytics and augmented reality (ar). the technology industry is increasing its pace of implementing innovation, disruption and creating new ventures. organisations across asia pacific need to catch up with technological advancements and plan as digitalisation becomes a top business priority. companies across different sectors are converting their workplaces into digital workplaces (cortellazzo, bruni, & zampieri, 2019) by leveraging new technologies as an enabler of innovation, maximizing both the strategic and operational value and staying competitive in the dynamic business environment (cascio & montealegre, 2016). organisations need to invest early in digital technologies, and this involves organisation-wide changes that pose significant challenges and uncertainties to business leaders. digital transformation requires an organisation to set clear ambitious goals, have a digital vision and need capable leaders with innovative thinking and deep knowledge of the external market to drive the transformation and gain competitive advantage in the digital era (bongiorno et al., 2018; dahlström, desmet, & singer, 2017). digital transformation is challenging for organisations as they have to try to strike a balance in four main areas; building innovation capabilities along with existing product innovation, process and product innovations, collaboration between employees and external partners, and redesigning flexible and manageable governance structures (svahn, mathiassen, & lindgren, 2017). australasian journal of information systems karippur & balaramachandran 2022, vol 26, research article antecedents of effective digital leadership 4 2.2 dynamic capabilities there is an interesting fit between digital transformation as a phenomenon of interest and dynamic capabilities as a conceptual foundation. the nature of digital transformation, as a source of continuous change and disruption, creates an impetus for organisations to adopt digital technologies to gain or maintain their competitive advantage (vial, 2019). the dynamic capabilities framework helps to explain how enterprises respond to rapid technological and market change (warner & wäger, 2019). teece (2007, 2009) defines dynamic capabilities as an enterprises’ ability to sense opportunities (and threats), seize opportunities, and transform or reconfigure the organisation’s business model to adapt to the changing environment. with dynamic capabilities, an enterprise can sustain competitive advantage by leveraging and reconfiguring its existing competencies and assets to add value to its customers (o'reilly iii & tushman, 2011). however, there is limited research that examines how organisations build dynamic capabilities for digital transformation (warner & wäger, 2019). organisations need to sense disruptions that enable continuous, repeatable adaptation to cope with rapid changes to the external competitive environment, seize strategic opportunities and reconfigure the business model to sustain competitiveness (vial, 2019). leadership capability is critical to achieve digital maturity, higher organisational performance and plays a role in building the dynamic capabilities of an organisation. leaders need to sense the changes in market trends and prudently seize upon these trends by augmenting products with services and transforming the organisation. a digital leader takes on the role as a digital advisor with a clear vision, and ability to analyse environmental changes and disruptions of digital technologies and proactively reacts and invest resources to ensure the organisational transformation (kane et al., 2017; vial, 2019). 2.3 leadership in the digital age digitally successful enterprises have built strong leadership capabilities to envisage and drive transformation (zeike, bradbury, lindert, & pfaff, 2019). the role of leadership is crucial throughout the digital transformation journey of an organisation (larjovuori et al., 2018). the demands of digital disruption entail new skills and leadership capabilities to improve processes, engage talent and develop new value-added business models for organisations (kane et al., 2015). leadership capabilities and the operationalisation and implementation of digitalisation are important factors for enterprises to succeed with digital transformation (della corte et al., 2019). digital transformation is creating new challenges for leaders (cortellazzo et al., 2019). rogers (2016) highlighted that senior leadership teams must develop new ideas to take advantage on new business model innovations that augment customer needs and experiences. leaders must dexterously oscillate between long-term and short-term thinking, create trade-offs, and make decisions in imperfect situations quickly (swift & lange, 2018). leaders play a vital role to capitalise the value of digitalisation by retaining talent, engaging, and nurturing employees on the digital culture (harvard business review, 2017; world economic forum, 2018). leaders also need to be held accountable also for addressing new ethical concerns arising from the dark side of digital transformation such as unethical use of data. (cortellazzo et al., 2019). leader is a role figure who ensures digital maturity of an organisation with a digital vision and strategy (sainger, 2018). the rapid changes associated with digital disruption can be disorienting and thus organisations require strong leaders at the helm (kane, phillips, copulsky, & andrus, 2019). australasian journal of information systems karippur & balaramachandran 2022, vol 26, research article antecedents of effective digital leadership 5 with digital leaders making significant changes to their organisational culture and strategy to lead a digital business, it is important that they communicate a clear vision to the people and provide a business environment to facilitate digital maturity (della corte et al., 2019). digital leaders need to be creative visionaries and be prepared to accept and respond intuitively to continual disruption (tardieu et al., 2020). digital vision, empowerment, ability to manage diverse teams, are some of the competencies highlighted that are critical for digital transformation (imran, shahzad, butt, & kantola, 2020). though there is extensive research stating that leadership plays an important role for achieving success in enterprise systems, there is lack of research on how specifically leadership achieves business success through the adoption of technology (li et al., 2016). there are also very few studies that describe the impact of digital transformation on leadership (cortellazzo et al., 2019). 2.4 theories of leadership leadership style is a combination of traits, skills, and behaviours that leaders practise as they interact with followers (judge, bono, ilies, & gerhardt, 2002). there are various leadership theories such as behavioural leadership theories, leadership trait theories, and contingency leadership theories which explain the suitable leadership style based on the leader, followers, and organisational context (oberer & erkollar, 2018). traditional leaders use power and influence to lead, are goal-oriented, authoritative, and decisive (burley, 2019; kane et al., 2019). however, they lack the innovation, collaboration with employees and agility. hence, it is necessary to go beyond past leadership theories to explain the impact of digitalisation on leadership and leaders (kahai, 2013). some of the common characteristics of modern leadership theories include the role of social interaction and relational leadership practices (larjovuori et al., 2018). transactional and transformational leadership (burns, 1978; bass, 1985, 1990) are among the most influential behavioural leadership theories of the last decade (diaz-saenz, 2011). bass (1999) defined transformational leadership as motivating the follower beyond their selfinterest through inspiration, charisma, intellectual stimulation, or personalised consideration. transformational leadership motivates followers by changing employees’ values, manners, and behaviour, and, in turn, these followers propose new ideas and integrate such ideas into innovative activities. transformative leaders have a positive influence on enterprises promoting creativity and explorative innovation (kassotaki, 2019) and encourages employees to set and achieve their desired goals and put them into action to reach higher performance standards. in contrast, transactional leadership is task-oriented, uses various types of incentives to achieve goals and does not encourage experimentation or have positive influence on the organisation’s creativity and more likely to be associated with exploitative innovation (liu, yu, & wu, 2019). the transactional leadership style focuses on completing tasks and ensuring employees follow the rules and pursues efficiency. organisations may employ leaders with different leadership styles; transformational leaders, who motivate others to be successful and empower creativity, and transactional leaders, who sets the tone for others to follow without providing many opportunities for creativity or innovation (baškarada, watson, & cromarty, 2016). in a study by sow and aborbie (2018), it was found that a transformational leadership style, to a certain extent, contributed to the desired outcomes during a digital transformation as compared to other leadership styles. a australasian journal of information systems karippur & balaramachandran 2022, vol 26, research article antecedents of effective digital leadership 6 participative leadership style can involve employees in decision-making and induce positive outcomes for a successful change management strategy (fatima, majeed, & saeed, 2017). digital enterprises need transformative leaders with a vision for the future (sow & aborbie, 2018), a cross-hierarchical, team-oriented, and collaborative approach, with a strong focus on innovation (oberer & erkollar, 2018) and ability to embrace and drive change with a positive attitude (vey, fandel-meyer, zipp, & schneider, 2017). current business leaders must be able to combine both traditional and new digital skills to effectively guide their organisations into the future (kane et al., 2019) in order to stay competitive. literature on leadership, in the context of digital transformation, is still in its infancy and not many empirical studies exist. for the purpose of this study, we have chosen to focus on transformational leadership as the characteristics closely suit the attributes and traits of a digital leader. 2.5 ambidextrous leadership hess et al. (2020) report that firms in the current digital era face significant challenges despite the motivation and support by senior leadership for digital transformation. a major challenge is the need to balance the exploitation of existing capabilities while building new digital capabilities (svahn, mathiassen, & lindgren, 2017). tushman and o’reilly (1997) envisioned an ambidextrous organisation to be one where managers play two different roles simultaneously (pearce, 2013). ambidexterity is characterised by leaders with the ability to exploit current resources to improve the business operations and concurrently take risks by exploring new opportunities and implement best practices to the existing business model (finzi, firth, & lipton, 2018). exploitation is based on the refinement, optimisation of resources, improvement in processes, competencies, and leveraging technologies for effective and efficient implementation (popadiuk, luz, & kretschmer, 2018). for example, in the context of digital transformation, there is a need to ensure that a secure infrastructure is available to fully exploit the speed and agility of the digital technology services (heneghan, snyder, & symons, 2017). exploration refers to an organisation’s move to take risks to experiment and discover new ideas, alternatives, search for new opportunities and adopt innovation (march, 1991). however, by relying only on exploration, an organisation may face problems such as the inability to capture returns on innovation (o'reilly & tushman, 2008; o'reilly & tushman, 2013). o’reilly and tushman (2004) found organisations exhibit ambidextrous characteristics when they seize exploratory and exploitative innovation in business units, strategically integrated yet structurally independent to the top management, for competitive advantage. for effective digitalisation, business units need to collaborate more effectively with firms across geographies and encourage employees to be active on platforms where they can collaborate on the development of cutting-edge technologies with experts in other organisations and provide them with opportunities to develop their digital skills (kane et al., 2017; finzi, firth, & lipton, 2018; cortellazzo et al., 2019). ambidextrous leadership comprise the dynamic capabilities of a leader to articulate a strategic vision, adapt, integrate, and reconfigure organisational skills, and explore and exploit resources to sustain in the changing business environments (o'reilly iii & tushman, 2011; gentle & metselaar, 2020). ambidextrous leaders can both build and run the business. digital transformation is not just about digitising products and automating processes, it is also a new way of running the business. as customer expectations have changed, so must the approach australasian journal of information systems karippur & balaramachandran 2022, vol 26, research article antecedents of effective digital leadership 7 to acquiring and serving those customers: faster, more transparent, more effortless, and more personalized. as such, ambidextrous leaders are able operate interchangeably between build and run disciplines and are equally skilled leading a development program as managing a commercial review. they can take a new business from idea to market; optimise every part of the value chain once launched; and at the same time evolve the product to narrow in on efficiencies or widen access and appeal. this comes from an understanding that in fast-moving and uncertain competitive markets, active iteration and innovation of the business model is required to survive and win (takle, 2018). there is acknowledgement of an increased ambidexterity in the role of chief information officers (cios) and their role as gatekeepers and contributors (cortellazzo et al., 2019; kohli & johnson, 2011). 3 research model and hypotheses this section discusses the proposed research model based on the literature surveyed in the previous section and drawing on the theories of transformational leadership, dynamic capabilities, and ambidexterity. figure 1. the research model australasian journal of information systems karippur & balaramachandran 2022, vol 26, research article antecedents of effective digital leadership 8 leadership attributes are critical traits of effective leadership in a digital environment (kane et al., 2019; kreutzer, neugebauer, & pattloch, 2018). in ambidextrous leadership, leaders should not only be able to sense changes in their competitive environment but also potential changes in technology, customers, and competition (o'reilly iii & tushman, 2011) and set strategic priorities. exploration is often linked with emerging customer needs, unexploited markets, harnessing new technologies and innovation. in order to support the new value creation opportunities of digital transformation, an effective combination of explorative and exploitative capability development is required. leaders need to focus on a range of organisational areas for exploration (havermans, den hartog, keegan, & uhl-bien, 2015). digital governance practices for exploitation play an important role in supporting organisational change, enables collaborative communications across organisational boundaries and a shared digital decision-making culture of an organisation (bongiorno et al., 2018). figure 1 presents the proposed research model which theorises the role of these factors; leadership attributes, strategic priorities, organisational focus areas for exploration and digital governance practices for exploitation and simultaneously specifies the factors associated with them. the following hypotheses are proposed and explained in the subsequent sections. h1: the leadership attributes are positively associated with effective leadership for digital enterprises h2: strategic priorities of an organisation are positively associated with effective leadership for digital enterprises h3: organisational focus areas for exploration are positively associated with effective leadership for digital enterprises h4: strategic priorities are positively associated with organisational focus areas for exploration for digital enterprises h5: digital governance practices for exploitation is positively associated with effective leadership for digital enterprises 3.1 leadership attributes transformative vision, digital literacy and adaptability stand out as most critical traits of effective leadership in a digital environment (kane et al., 2019; kreutzer, neugebauer, & pattloch, 2018). leadership capabilities for digital enterprise involve creating a transformative digital vision, building technological leadership, engaging, and motivating employees and focusing on digital governance. (westerman, bonnet, & mcafee, 2014a, 2014b). change-agile leaders must have a future vision and be adaptable to changes and fluctuations in the business world. valentine and steward (2015) indicated that the competencies of an effective digital leader should encompass skills, knowledge, and experience to govern technology for strategic advantage. in addition, digital leaders should be involved in making technology decisions, governing risk, and effectively using technology to reap returns and add value to the organisation. a summary of key leadership attributes critical to digital leadership is listed in table 1 below. australasian journal of information systems karippur & balaramachandran 2022, vol 26, research article antecedents of effective digital leadership 9 leaderhip attributes description references digital visionary • leaders’ ability to create a vision that is clear and meaningful for the organisation’s digitalisation process and have the capability to execute strategies to actualize it • chief executive officer (ceo) must inspire employees to pursue a vision of the company as a digital business • digital leaders need an innovative spirit and unique ability to dream the big ideas swift & lange, 2018.; larjovuori et al., 2018; neubauer, tarling & wade, 2017; larjovuori, bordi, & heikkilätammi, 2016; el sawy, kræmmergaard, amsinck, & vinther, 2020; deloitte, 2016 smart risk taker • leaders should open the door to new possibilities, new markets, new customers and new innovations by taking risks • leaders must be smart about managing risks larjovuori et al., 2018; finzi, firth, & lipton, 2018; swift & lange, 2018 foster innovation • leaders needs to drive innovation to support digital transformation and remain competitive • leaders need to cultivate an innovation culture • leaders need to encourage employees to be creative, collaborate and think out of the box to sustain successfully as digital enterprises kane et al., 2019; cortellazzo et al., 2019; dubey, 2019; larjovuori et al., 2018; swift & lange, 2018; heneghan, snyder, & symons, 2017 agility • promote a flexible and innovative attitude in the organisation • regularly re-assesses the landscape and measuring results – with the agility to change course as needed • digital leaders require agility to have parallel focus on delivering both the big picture in the long run and monitor daily activities • change-oriented, open-minded, adaptable, and innovative. responsive to a fluid environment and adapt to changes in the technology and market environments hill, l.a., 2020; kane et al., 2019; cortellazzo et al., 2019; larjovuori et al., 2018; deloitte,2016; neubauer, tarling & wade, 2017; heneghan, snyder, & symons, 2017 tolerance of ambiguity • leaders need to be aware of uncertain or ambiguous information that prevents clarity of understanding or direction • need to be an agile leader and be able to manage uncertainty and ambiguity cortellazzo et al., 2019; bongiorno et al., 2018; swift & lange, 2018; finzi, firth, & lipton, 2018 natural collaborator • building partnerships with key stakeholders both inside and outside your organisation • leaders need the ability to connect to people and content—in a way that inspires innovation and ideas • collaborate with partners (e.g. platform companies) for new digital business opportunities • acquire essential expertise and solutions through collaboration • freely share their experience and expertise with others • encourage innovation by forming external partnerships hill, l. a., 2020 kane et al., 2019; cortellazzo et al., 2019; dubey 2019; larjovuori et al., 2018 builds digital talent • leaders are expected to source the talent/skills required to drive and deliver the digital strategy • reskill existing employees and look at new ways to expand the talent pool larjovuori et al., 2018; finzi, firth, & lipton, 2018; deloitte, 2016; geissbauer, vedso, & schrauf, 2016 australasian journal of information systems karippur & balaramachandran 2022, vol 26, research article antecedents of effective digital leadership 10 • c-suite leaders will need to attract, retain, and train other employees to work in a dynamic and digital ecosystem environment • encourage continuous learning, attract and retain talent • motivating and inspiring employees to achieve objectives of digitalisation table 1. leadership attributes hence, we hypothesise that the leadership attributes, as highlighted above, are positively associated with effective leadership for digital enterprises. 3.2 strategic priorities the digital era has not only accelerated the pace of change but also opened wholly new opportunities for firms to embrace the technology and reinvent business models. (hyek & nendick, 2014). responding to the opportunities and threats of digital is challenging (schadler et al., 2017) and each enterprise will need to prioritize its approach based on its unique market dynamics and relative digital maturity. ambidexterity facilitates new resource configurations that can pose as a competitive advantage to the firm (popadiuk et al., 2018). leaders need to set strategic priorities related to marketing and investments to be made, thus requiring the creation or selection of business models (popadiuk et al., 2018). ambidextrous leadership should be able to sense changes in their competitive environment, including possible shifts in technology, competition, customers (o'reilly iii & tushman, 2011) and set strategic priorities. the key strategic priorities for building a digital enterprise includes business model innovation, building competitive advantage, keeping pace with competitors and cost management (gottlieb & willmott, 2014). a company’s level of digital maturity has a significant influence on what the priorities of leaders should be (tardieu et al., 2020). a summary of the key strategic priorities is listed in table 2 below. strategic priority description references prioritise new business models • create new business models, enhancing organisation’s adaptation capabilities to changing conditions and business trends • seek high-level priorities for the organisation • prioritize organisation’s approach based on its unique market dynamics and relative digital maturity. • identify the digital opportunity and understand the new data-centric business paradigms tardieu et al., 2020; popadiuk et al., 2018; schadler et al., 2017; hyek & nendick, 2014 build competitive advantage • re-evaluate strategies, market position, competitive strengths and strategic changes to remain successful • review assets that give a potential competitive advantage and fully leverage them in strategies tardieu et al.,2020; popadiuk et al., 2018; hyek & nendick, 2014 keep pace with competitors • observe the nature of competition and discover what competitors are doing better and what can be learnt from them. • keep pace with competitors through enhancing, protecting, and reconfiguring the assets of the business enterprise • make thoughtful choices about adopting the right technologies to innovate the businesses donahue, 2020 popadiuk et al., 2018; deloitte, 2017 cost management • provide strategic lever to generate savings that can be invested in driving growth, create new opportunities, ittner, 2019; australasian journal of information systems karippur & balaramachandran 2022, vol 26, research article antecedents of effective digital leadership 11 and proactive ways to become more competitive in the global environment • maintain cost control • minimising transaction / operating costs • leverage digital technologies for cost management gottlieb, & willmott, 2014; hyek & nendick, 2014; table 2. strategic priorities hence, we hypothesise that strategic priorities of an organisation, as highlighted above, are positively associated with effective leadership for digital enterprises. 3.3 organisational focus areas for exploration exploration is often linked with emerging customer needs, unexploited markets, harnessing new technologies and innovation. in order to support the new value creation opportunities of digital transformation, an effective combination of explorative and exploitative capability development is required. leaders need to focus on a range of organisational areas for exploration (havermans, den hartog, keegan, & uhl-bien, 2015). to execute a digital strategy, organisations need transforming capabilities, which includes the continuous strategic renewal of assets and organisational structures (agarwal & helfat, 2009; teece, 2014), to respond to the dynamic environment and realize the full potential of strategic change (bharadwaj et al., 2013; karimi & walter, 2015; teece & linden, 2017). digital transformation uses platform technologies to create value and competitive advantage through offering new products and services, new business models, and new relationships (international data group, 2018). in order to transform their business to exploit the benefits of digitalisation, companies should focus on digital strategy, derive new business models, use data and analytics for decision-making and predicting future opportunities, adopt technology for operational efficiency, build an eco-system of partnerships, develop cross-functional talents and empower individuals and teams (world economic forum, 2018). for this study, the key areas to focus in an organisation to achieve successful digital transformation include diversifying business models through creative partnerships (fenwick & schadler, 2018), facilitate data sharing and data analytics (geissbauer, vedso, & schrauf, 2016; international data group, 2018), adopting new platforms to meet new business challenges (de la boutetière, montagner, & reich, 2018), identifying the roles and stakeholder partnership in business value chain, establishing digital teams to foster digital innovation (geissbauer, vedso, & schrauf, 2016) and empower employees to adopt digital technologies (westerman et al., 2014a, 2014b; de la boutetière et al., 2018). a summary of the organisational focus areas is listed in table 3 below. organisational focus areas for exploration description references diversify business models through creative partnerships • compete in fast-changing digital environments by making informed decisions about when to build, buy or partner to develop solutions that can accelerate the organisation’s own digital transformation • find ways to capitalize on new and unexpected model innovation that optimise customer needs and experiences • create, modify, significantly extend or replace the business model of the enterprise through strategic partnerships and alliances with businesses dealing with improved technology knight, 2020; della corte et al., 2019; fenwick & schadler, 2018; hyek & nendick, 2014 australasian journal of information systems karippur & balaramachandran 2022, vol 26, research article antecedents of effective digital leadership 12 facilitate data sharing and data analytics • capitalise on data sharing and data analytics to sense new customer-centric trends and enhance strategic decisionmaking • create a data-driven culture and accrue the competitive benefits that result from it • use analytics to derive actionable insights, make effective business decisions and add value to the business davenport & mittal, 2020; international data group, 2018; geissbauer, vedso, & schrauf, 2016 adopt new platforms to meet new business challenges • adopt emerging technology platforms such as cloud for better integration with business value chain partners, such as suppliers and key customers, to improve efficiencies significantly • adopting disruptive technologies to improve the customer experience and increase efficiency • collaborate with additional platform companies to jointly explore potential innovations • leveraging these partnerships to increase innovation finzi, firth, & lipton, 2018; de la boutetière et al., 2018 identify the roles and stakeholder partnership in business value chain • digitalising essential functions within the internal operations processes and with horizontal partners along the value chain • integration into digital ecosystems with value chain partners through end-to-end digitisation of all physical assets • collaborating and engaging with business stakeholders early in the process builds innovation and creates value from procurement chee & young, 2020; geissbauer, vedso, & schrauf, 2016 establish digital teams to foster digital innovation • setting up cross-functional teams to foster digital innovation and accelerate digital transformation • allocate time for employees to experiment or innovate • build multi-disciplinary teams, foster collaboration across departments, encourage brainstorming and shared decision making kane, 2019; link, 2018; geissbauer, vedso, & schrauf, 2016 empower employees to adopt digital technologies • giving employees the opportunity to generate their own ideas and propose where digitisation / digital technologies could and should be adopted in the business • establishing practices related to empower employees to work in new ways, play key roles in reinforcing changes and keep pace with the fast business • enabling and empowering employees to thrive in a digital environment by providing them with the resources and opportunities to carry out new initiatives kane et al., 2019; de la boutetière et al., 2018; westerman et al., 2014a, 2014b table 3. organisational focus areas for exploration hence, we hypothesise that organisational focus areas for exploration, as highlighted above, are positively associated with effective leadership for digital enterprises. additionally, we also theorise that strategic priorities are positively associated with organisational focus areas for exploration. the conflict between exploitation and exploration can be addressed by adopting a portfolio approach and setting strategic priorities. companies should focus on activities to support cash generation in the short run as well as have action plans aimed at creating new products, services, and/or business models (finzi, firth, & lipton, 2018). over a period, these activities may shift from an innovation focus to efficiency focus or vice versa, implying that the organisation’s aim or direction remains balanced at any point of time. hence, we hypothesise that strategic priorities are positively associated with organisational focus areas for exploration for digital enterprises. australasian journal of information systems karippur & balaramachandran 2022, vol 26, research article antecedents of effective digital leadership 13 3.4 digital governance practices for exploitation the digital transformation process affects the entire organisation and resistance may be faced from different departments. a study (matt, hess, & benlian, 2015) states that in order to deal with resistance, leadership skills are essential and active involvement of the different stakeholders affected by the transformation is required. resistance to change, poor communication amongst team members and lack of stakeholders’ involvement or buy-in are some of the challenges faced by companies undergoing digital transformation (solis, 2016). increased connectivity and information sharing eliminate the hierarchical barriers and organisational boundaries which leads to transforming task-based activities into more projectbased activities, where employees participate in the creation of new value-added services (cortellazzo et al., 2019). hence, there is a compelling need for leaders to build proper governance practices to overcome such challenges. according to tannou and westerman (2012), governance is one of the key determinants of success in digital transformation. information technology (it) governance has been widely discussed as a supporting function to reap the benefits of it in a business. although the traditional it governance has contributed to businesses in the past, it might not be able to reflect the realities of the current digital world. the current understanding of it would involve the need to be aware of latest developments in in this digitised era. firstly, products and services have become more digitalised, showing close integration of it with business processes (bharadwaj et al., 2013). hence, resulting in cross-functional decision-making in contrast to traditional, autonomous functional-level decision-making. secondly, the perception about it is evolving as it becomes more businessoriented involved in high-level strategic decision-making. finally, with the advancement and speed of technological development, companies are motivated to develop and revise governance measures in order to cope and align with the accelerating it-related decisionmaking processes. (bongiorno et al., 2018). therefore, the traditional view of it governance will gradually be ineffective in today’s digital enterprises. (bongiorno et al., 2018). digital governance (it governance in digital enterprises) is a methodology that establishes accountability for digital policy, strategy, and standards. digital governance plays an important role in supporting organisational change, enables collaborative communications across organisational boundaries and a shared digital decision-making culture of an organisation. governing this digital transformation requires leaders to determine the issues and challenges relating to the implementation of digital projects, and develop innovative and creative processes, stipulate new polices, roles and responsibilities (bongiorno et al., 2018) to manage these projects effectively. there has been research conducted in the areas of leadership, processes, people, strategy, technology, and organisational dynamics, which influence the state of digital governance in the organisations (dykes, 2012). effective digital governance processes help in digital development and clarify roles and responsibilities (welchman, 2015). study has also been done on it risk management and resource management, which improves the effectiveness of digital governance processes (lunardi, maçada, & becker, 2014). as per the study by devos and de ginste (2015), the digital governance concept is rapidly evolving in the business world, especially over the last few years (harvard business review, 2014). there are no best practices formally defined for digital governance. each industry and business should develop their own digital governance model based on its maturity and market dynamics. devos and de ginste (2015) study has shown that those organisations which have established effective digital governance were able to generate australasian journal of information systems karippur & balaramachandran 2022, vol 26, research article antecedents of effective digital leadership 14 higher returns to the extent of 40% on their it investments as compared to their competitors without digital governance. for this study, the digital governance practices for exploitation shown in table 4 were important to achieve successful digital transformation. digital governance practices for exploitation description references regulatory compliance • understand impact of new regulatory and compliance issues which demand action • provide direction to ensure legal or regulatory compliance • promote industry-wide standards-based approach to regulatory compliance and reporting • ensure compliance with regulation by providing a means of reporting infringements or mishandling of issues butler & o’brien, 2019; bongiorno et al., 2018 cyber security planning • senior management need to be aware of the growing cybersecurity risks and plan and address these risks accordingly • need to ensure security around organisational data assets and personal privacy, protecting them all from theft or external attacks. • establish a new balance between risk and opportunity management butler & o’brien, 2019; bongiorno et al., 2018; hyek & nendick, 2014 ethical data handling • recognize risks from unethical data practices and illegal use of insights and mitigating these threats is critical for a digital enterprise • proper management of personal data and compliance with data protection laws • organisations that display themselves as untrustworthy data custodians will face a major loss of brand equity. • employees advised to customise their privacy settings and respect other people’s privacy, monitor and track their digital footprint bongiorno et al., 2018; accenture, 2016 social media management • pay close attention to organisation’s online reputation • determine how the corporate brand is represented in social media such as facebook, twitter, or youtube or in authenticated or unauthenticated web sites • aware of unauthorised data sharing by social media platforms steele, allen, & watkins, 2019; bongiorno et al., 2018; tannou & westerman, 2012 employee collaboration • create space and opportunities for participation and engaging employees in the change process • expose employees to different ways of thinking and engaging, cultivate employee collaboration and allow them to embrace the digital culture larjovuori et al., 2018; kane et al., 2017; tannou & westerman, 2012 table 4. digital governance practices for exploitation hence, we hypothesise that digital governance practices for exploitation, as highlighted above, are positively associated with effective leadership for digital enterprises. australasian journal of information systems karippur & balaramachandran 2022, vol 26, research article antecedents of effective digital leadership 15 4 research method, data, and analysis 4.1 survey instrument a quantitative approach was taken for the research with a strong empirical analysis to identify the antecedents of effective digital leadership in asia pacific. through the identification of relevant measurements from a comprehensive literature review, a survey instrument was developed to test the research model and hypotheses in figure 1. content validity was ensured by adopting the scales for different measures from prior research (see appendix). to facilitate measurement, the survey instrument utilised items on a five-point likert scale rating ranging from strongly disagree, represented by 1, to strongly agree, represented by 5. the survey instrument comprised a total of 35 questions and had three sections with the first section on demographics, followed by a section introducing the research topic and the third section related to the constructs. 4.2 sampling process and data collection the target population for the survey were business leaders from the asia pacific region. we have chosen to test our proposed model across singapore, sydney (australia), mumbai (india) and dubai (uae) where there is large concentration of business leaders who are familiar with the asia pacific region. singapore is home to a large number of companies with regional headquarters (rhq). for example, 46% of rhq are based in singapore across a diverse range of industries (singapore economic development board, 2020). sydney has been the first choice for global companies within australia, with over 600 multinational firms running their asiapacific operations from this city (nsw treasury, 2019). india has been home to fastest-growing mature tech ecosystem. for example, mumbai is the sixth fastest growing tech-hub in the world (london & partners, 2021). dubai is the favoured city for the regional headquarters for the world’s largest 500 companies (rizvi, 2017). the questionnaire framed was then pilot tested with a sample size of 10 participants for comprehensibility and the ability to express the research intention. a total of five domain experts were consulted to validate the model and questionnaire. their inputs were integrated into the final model and questionnaire. a series of information technology management (itm) conclave events were organised in 2017 and 2018 targeting leaders and senior managers of enterprises focusing on the theme of digital transformation. these events were supported and promoted by singapore computer society (scs), australian computer society (acs), the national association of software and service companies (nasscom) and the indus entrepreneurs (tie) across the cities dubai, mumbai, singapore, and sydney with a total participation of over 200 leaders and senior managers. we reached out to these participants through the initial mailing list of companies who have also indicated digital transformation as their top priority. the companies were selected based on their interest in participating, as well as their diversity. (larjovuori et al., 2018). for this study, a random sample of business leaders and high-ranking managers involved in decision-making (such as ceos, cios, senior managers, or project managers) who were familiar or involved in digital transformation in asia pacific were targeted. these companies spanned across different industries from the region that included it, consulting, retail, banking and finance and healthcare. the participants were requested to participate in the survey through an online survey link. in the instructions given to the respondents, we asked them to respond to the questions by keeping their organisation in mind. a total of 120 australasian journal of information systems karippur & balaramachandran 2022, vol 26, research article antecedents of effective digital leadership 16 responses were received over a period of one month excluding the incomplete submissions. the demographic distribution of the sample is presented in table 5 (120 respondents). table 5. demographic profile of survey respondents (n= 120) 4.3 data analysis data analysis was conducted in two stages. in the first stage, the reliability and three types of validity examination; content validity, convergent validity, and discriminant validity, were conducted to assess the robustness of the measurement model. in the second stage, partial least squares (pls) path modelling was undertaken as it is preferred for research studies for its flexible modelling and identifying key drivers (hair, ringle, & sarstedt, 2011). the data analysis was carried out using adanco 2.0.1 to develop and evaluate the structural model and the causal relationships between the variables. 4.3.1 reliability the two variable reliability measurements used in this study are (1) cronbach's alpha (α) and (2) jöreskog's rho (ρc). a cronbach’s alpha coefficient which is greater than 0.6 and less than 1, is considered to be an acceptable level of reliability (pallant, 2007; hair, black, babin, & anderson, 2013). jöreskog’s rho evaluates composite reliability to appreciate the integrity and homogeneity of the model (werts, 1978). as shown in table 6, both cronbach’s alpha and composite reliability (jöreskog's rho) exceeded their threshold values (>0.70). this confirms the statistical significance of the model and demonstrates strong evidence of internal consistency reliability (hair, hult, ringle, & sarstedt, 2016). 4.3.2 validity convergent validity carlson (2012) mentioned that convergent validity evaluates the degree of correspondence between two measures of variables; composite reliability (cr) and the average variance extracted (ave). ave refers to the ratio of the construct variance to the total variance among indicators (hair et al. 2016; roldán & sánchez-franco, 2012). as shown in table 6, the average variance extracted (ave) value of each construct exceeded the threshold value of >0.5 (fornell demographic variable category frequency [n=120] percentage (%) size of firm less than 50 22 18% 50 to 200 employees 15 13% more than 200 employees 83 69% years associated with ict industry 1–5 years 15 13% 5–10 years 37 31% 10 15 years 29 24% more than 15 years 39 32% location of organisation singapore 53 44% india 23 19% australia 15 13% uae 26 22% others 3 2% australasian journal of information systems karippur & balaramachandran 2022, vol 26, research article antecedents of effective digital leadership 17 & larcker, 1981; hair, ringle, & sarstedt, 2011; henseler, hubona, & ray, 2016), thus indicating significant convergent validity (hair et al., 2016). hence, all constructs in the model have adequate reliability and convergent validity. construct average variance extracted (ave) jöreskog's rho (ρc) cronbach's alpha (α) leadership attributes 0.5916 0.9096 0.8832 effective digital leadership of enterprises 0.6149 0.9176 0.8948 digital governance practices for exploitation 0.6076 0.8853 0.8376 strategic priorities 0.5112 0.8023 0.6920 organisational focus areas for exploration 0.6862 0.9291 0.9083 table 6. convergent validity and reliability statistics discriminant validity discriminant validity ensures that the constructs that differ from each other are proven to be different (henseler, hubona, & ray, 2016). adanco 2.0.1 is used to test whether constructs are irrelevant or in truth random. as recommended by fornell and larcker (1981), the discriminant validity of the independent variables was examined by finding the square root of the ave. the ave measure of other variables should be lesser than the square root of ave obtained from a particular variable. this fornell-larcker (1981) method is used for evaluating the degree of distinction between the variables. the measures of the square root of the ave seen on the diagonal of table 7 are all greater than the inter-construct correlations, thus confirming discriminant validity (sarstedt, wilczynski, & melewar, 2013). this demonstrates satisfactory convergent and discriminant validity. construct leadership attributes effective digital leadership of enterprises digital governance practices for exploitation strategic priorities organisational focus areas for exploration leadership attributes 0.5916 effective digital leadership of enterprises 0.3541 0.6149 digital governance practices for exploitation 0.3762 0.3663 0.6076 strategic priorities 0.3496 0.2551 0.2307 0.5112 organisational focus areas for exploration 0.1238 0.3756 0.1629 0.1045 0.6862 table 7. overall discriminant validity for each construct note: squared correlations; ave in the diagonal. australasian journal of information systems karippur & balaramachandran 2022, vol 26, research article antecedents of effective digital leadership 18 figure 2. structural equation model with path coefficients australasian journal of information systems karippur & balaramachandran 2022, vol 26, research article antecedents of effective digital leadership 19 4.4 structured equation modelling / path analysis the structural equation model (sem) shown in figure 2 was developed to establish the relationship between the dependent and independent variables and examine their significance (roldán & sánchez-franco, 2012). path modelling is a special case of structural equation modelling (sem), in which single indicators are used in the model for each variable and the strength of each path is calculated as a product of the path coefficient along that path. the correlation is the equation of the sum of the contribution of all the paths through which the variables are connected. the strength of each path is calculated as a product of the path coefficient along that path. in our research, the results of the analysis in figure 2 shows that with its r2 value being 0.590, the model was acceptable, and all 5 hypotheses were supported (hooper, coughlan, & mullen, 2008). 4.4.1 hypothesis testing in our research, five hypotheses were postulated. the structural model and hypothesised relationships were tested using a standardised path coefficient (β) and t-statistics (table 8) at p< 0 .01. the threshold critical t-values (α; two-sided test) at a significance level of α = 0.01, α = 0.05 and α = 0.10 were 2.57, 1.96 and 1.65, respectively (hair et al., 2016). the structural model revealed that 3 hypotheses were supported by the relationships at the p < 0.01 significance level while 2 hypotheses was supported by the relationships at the p < 0.05 significance level. the results (table 8) authenticate that all five proposed latent constructs had significant effects on effective digital leadership of enterprises. our findings show that the relationships between leadership attributes and effective digital leadership of enterprises (β = 0.2353, t = 2.6242, p < 0.01), organisational focus areas for exploration and effective digital leadership of enterprises (β = 0.3922, t = 3.3078, p < 0.01), and digital governance practices for exploitation and on effective digital leadership of enterprises (β = 0.22442, t = 2.6646, p < 0.01), were statistically significant confirming the support for h1, h3 and h5 (table 8). the structural model further revealed that two other hypotheses were supported at the p < 0.05 significance level (see figure 2). the relationship between strategic priorities and effective digital leadership of enterprises (β = 0.2486, t = 2.5745, p < 0.05) and strategic priorities and organisational focus areas for exploration (β = 0.3233, t =2.3553, p < 0.05) were statistically significant, confirming support for h2 and h4 (table 8). 5 discussion the results of the analysis have shown that leadership attributes, strategic priorities, organisational focus areas for exploration and digital governance practices for exploitation, are significant antecedents of effective digital leadership, thus supporting the hypotheses, h1, h2, h3, h4 and h5. australasian journal of information systems karippur & balaramachandran 2022, vol 26, research article antecedents of effective digital leadership 20 table 8. outcomes of the hypothesis testing note: *p < 0.10; **p < 0.05; ***p < 0.01 (two-sided) confidence intervals for significance testing. australasian journal of information systems karippur & balaramachandran 2022, vol 26, research article antecedents of effective digital leadership 21 5.1 summary of findings the findings from this study have highlighted that leadership attributes have a positive impact on effective digital leadership for digital enterprises. various past studies examining the leadership traits and qualities suggests that state that digital leaders should be able pursue a clear and meaningful digital vision, inspire the employees, and have the capability to execute strategies to actualize the digitalisation process of the enterprise (swift & lange, 2018; larjovuori et al., 2018). leadership competencies that organisations need to develop in their leaders include digital vision, digital knowledge, empowerment, and managing diverse teams (imran et al., 2020). leaders would also be open to new possibilities, new markets, new customers, and new innovations by taking risks (larjovuori et al., 2018). this is consistent with studies on ambidextrous leadership which state that leaders embrace ambiguity, ambivalence, uncertainty, tension, and conflict (finzi, firth, & lipton, 2018). other studies state that a leader needs to be agile, adaptable to changes, open-minded (hill, 2020; kane et al., 2019; cortellazzo et al., 2019), drive and cultivate innovation (dubey, 2019; swift & lange, 2018) in a fluid and evolving market. another study mentioned the need for leaders to collaborate and build partnerships with key stakeholders for new digital business opportunities (larjovuori et al., 2018). in order to support the digital vision of the leader, leaders are to source, retain, train, and motivate employees to drive and deliver the digital strategy of the organisation (finzi, firth, & lipton, 2018). strategic priorities have a moderate impact on effective digital leadership for enterprises. based on previous studies, leaders are expected to seek high-level priorities for the organisation and create new business models and adapt to changing business trends (tardieu et al., 2020; popadiuk et al., 2018). in order to sustain and compete in the dynamic competitive environment, leaders will have to re-evaluate strategies, competitive strengths and review assets that give potential competitive advantage (tardieu et al., 2020; hyek & nendick, 2014). in addition, leaders must make thoughtful decisions about leveraging technologies to innovate the business (donahue, 2020; popadiuk et al., 2018). other studies state that leaders have to administer cost control measures and generate savings which can in turn be invested to drive growth, create new opportunities, and become more competitive in the global environment (ittner, 2019; gottlieb, & willmott, 2014). organisational focus areas for exploration have higher impact on effective digital leadership for digital enterprises. this is consistent with previous similar studies which stated that to compete in a fast-changing digital environment, an effective digital leader will need to focus on diversifying the organisation’s business model through building strategic partnerships (della corte et al., 2019; fenwick & schadler, 2018). another study mentioned that it was necessary for a business enterprise to create a data-driven culture by capitalising on data sharing and analytics (davenport & mittal, 2020; jimenez et al., 2018; geissbauer, vedso, & schrauf, 2016), adopt disruptive technologies to become better integrated with business value chain partners and improve efficiency (finzi, firth, & lipton, 2018; de la boutetière, montagner, & reich, 2018) and ensure end-to-end digitisation of all essential functions and integrate into digital ecosystems with value chain partners (chee & young, 2020; geissbauer, vedso, & schrauf, 2016). studies also stated that to accelerate digital transformation, digital leaders will need to set up cross-functional teams to foster digital innovation (kane et al., 2019; link, 2018) and empower employees to share new ideas and initiatives where digitisation or digital technologies can be adopted in the business enterprise (kane et al., 2019; de la boutetière et al., 2018). australasian journal of information systems karippur & balaramachandran 2022, vol 26, research article antecedents of effective digital leadership 22 it was observed that strategic priorities and organisational focus areas for exploration indicated a significant relationship. this is in line with the related findings (wrede, velamuri, & dauth, 2020) that states that strategic prioritisation influences the focus areas for exploration as the overall corporate strategy has integrated digitalisation and there is clear communication on the goals of digitalisation. past studies examining digital governance practices state the need for leaders to not only understand the impact of new regulatory and compliance but also to promote industry-wide standards-based approach to regulatory compliance and reporting (butler & o’brien, 2019). another study stated the need for leaders to be aware of the growing cybersecurity risks and unethical data practices and to enforce security controls to ensure privacy and compliance with data protection laws and practices (butler & o’brien, 2019; bongiorno et al., 2018, accenture, 2016). furthermore, with the widespread use of social media, leaders have to constantly pay attention to the organisation’s online reputation (steele, allen, & watkins, 2019). in order for employees to embrace the digital culture, leaders have to engage and create opportunities for employees to participate in the digitalisation change process (larjovuori et al., 2018; kane et al., 2017). 5.2 benefits of effective leadership the benefits of effective digital leadership include the capacity to sense, shape and seize digital opportunities, and maintain competitiveness through enhancing, combining, protecting, and, reconfiguring the business enterprise’s assets (teece, 2007; carcary, doherty, & conway, 2016). figure 3. effective digital leadership path coefficients in the above structural equation model, figure 3, the path coefficient for clear digital vision is 0.848, adept at aligning business and it strategy is 0.800, higher focus on innovation and growth is 0.832 and nurture the talent to derive and deliver digital strategy is 0.793. whereas the path coefficient for aggressive investment in agile and disruptive technologies is 0.728, australasian journal of information systems karippur & balaramachandran 2022, vol 26, research article antecedents of effective digital leadership 23 build on stable and secure infrastructure is 0.757, and build strategic partnerships is 0.721. this implies that clear digital vision, higher focus on innovation and growth and adept at aligning business and it strategy (heneghan, snyder, & symons, 2017), have the strongest impact on effective leadership, along with nurture the talent to derive and deliver digital strategy (tong, 2017) with a path coefficient of approximately 0.8. aggressive investment in agile and disruptive technologies, build on stable and secure infrastructure and build strategic partnerships have a moderate effect on effective leadership. leadership plays an important role in digital transformation. (zhang et al., 2011) study states that transformational leadership enhances team coordination, and this results in high team performance. transformational leadership improves the spirits, motivation, and performance of their followers. these include connecting the follower's sense of identity to the mission; inspire them; challenging followers to take greater responsibility and understanding their strengths and weaknesses to be able to assign the responsibilities accordingly. (garcía-morales et al., 2012) study suggests that transformational leadership, through organisational learning and innovation, is needed to improve an organisation’s performance in the current changing business environment. chen et al., 2012) further states that transformational leadership plays an important role in fostering innovation and is effective at the strategic business unit (sbu) level. from the survey findings, it shows that the transformational leadership style is more suited to be adopted by digital leaders of enterprises as the factors such as having a digital vision, foster innovation, agility, natural collaborator, building digital talent, are in accordance with the theory of transformational leadership (bass & riggio, 2006). holistically, the results highlight the importance of the factors considered and validate the implications of effective leadership for digital transformation. therefore, these factors must be taken into consideration by the business leaders of organisations who have embarked on the digital transformation journey to stay competitive in the digital economy. however, future research incorporating additional factors can be conducted to improve the accuracy and dynamism of the proposed model. 6 implications despite the positive attitude towards digital transformation and its benefits, there has not been much study linking leadership practices and effective digital leadership of enterprises (li et al., 2016). successful digital transformation starts from the top management and digital transformation is inherently cross-functional and will require new leadership practices to engage with and transform the organisation (yokoi, shan, wade, & macaulay, 2019). 6.1 implications for research firstly, this study has quantitatively affirmed the positive relation between leadership attributes, strategic priorities, organisational focus areas for exploration and digital governance practices for exploitation, and the effective leadership of enterprises in asia pacific. secondly, as research linking leadership and digital transformation is sparse, this paper will set a foundation for future studies related to effective digital leadership. thirdly, this is one of the initial studies to explore leadership practices based on theoretical lens of transformation leadership, ambidexterity, and dynamic capabilities. the authors extended the literature on various variables by proposing a conceptual model that sets a theoretical foundation to understand the antecedents of effective leadership. an in-depth australasian journal of information systems karippur & balaramachandran 2022, vol 26, research article antecedents of effective digital leadership 24 study to examine these variables can be carried out for future research. the model in this study (figure 2) can be used as a guide for future research involving the understanding of antecedents of effective leadership of enterprises. lastly, the results of the study can be further validated with other regions by examining the leadership attributes, strategic priorities, organisational and digital governance practices. 6.2 implications for practice this study has many important implications and recommendations for leaders, organisations, training institutes and government involved in digital transformation across asia pacific. firstly, given that only 7% of enterprises in asia pacific are identified as digital leaders (jimenez et al., 2018), there is a need for increased awareness of key leadership practices for effective digital leadership. the key factors identified such as leadership attributes, strategic priorities, organisational focus areas for exploration and digital governance practices for exploitation, will help enterprises to strategize and focus on the key factors to accelerate digital transformation and achieve effective digital leadership. there is a need to balance exploration and exploitation and put digital governance practices in place. the significant relationship of digital governance practices for exploitation indicates that senior management should work closely with the senior leaders and the business heads to develop digital business strategies that facilitate the strategic and secure use of technology across the enterprises to produce the desired business outcomes (heneghan, snyder, & symons, 2017). there is a need to ensure that businesses build their digital capabilities on top of a stable and secure infrastructure whether internally or externally sourced and are effective at selecting the most appropriate technologies and architectures (heneghan, snyder, & symons, 2017). secondly, with the increased focus on digital transformation, there is opportunity for leaders across asia pacific to develop and nurture the right leadership attributes and promote ambidextrous leadership. the digital leadership has to be constantly developed with top managers’ ambidexterity (della corte et al., 2019). the digital age requires rethinking the traditional leadership and command-and-control base governance that has been prevalent for so long in asia pacific. key attributes identified such as risk-taking, agility, tolerance of ambiguity, are critical for leaders for asia pacific to develop digital enterprises. it is important for them to accurately sense changes in their competitive environment, including potential shifts in technology, competition, customers and set strategic priorities and identify areas for exploration for the organisation. hence, leaders will need to seize opportunities in a timely manner by successfully innovating and implementing new systems in a changing business environment (popadiuk, luz, & kretschmer, 2018) and make significant investments in new technologies to achieve the full value and enterprise-wide, game-changing innovation (ghosh, burden, & wilson, 2019). thirdly, to successfully transition to the new digital economy, organisations need to find, recruit, and retain leaders and senior managers with digital skillsets and training institutes could play a critical role here. for example, business schools should accelerate collaborations with industry to develop and align the right curriculum and undertake more internships and digital transformation projects giving opportunity for students to enhance their leadership attributes and learn new practices for ambidextrous leadership. lastly, digital technologies offer opportunities for higher productivity growth and improvements in living standards. past studies indicate that of government incentives and regulatory support across asia pacific vary widely (kumar, 2017). consequently, there is a australasian journal of information systems karippur & balaramachandran 2022, vol 26, research article antecedents of effective digital leadership 25 need for governments across asia pacific to promote and incentivise leadership practices for effective digital leadership and accelerate organisational capability development in the areas of digital innovation and catalyse collaborations. there is a need for governments to put in place and update regulations in the areas of cybersecurity and policies and promote digital governance practices especially across small and medium-sized enterprises. 7 limitations of the study and future scope although this study makes contributions in terms of better understanding of the factors influencing effective leadership for digital enterprises, there are a few limitations. firstly, this research mainly focused on collecting data within the asia pacific region represented by data of sample size 120 from four main cities namely dubai, mumbai, sydney, and singapore. the observation and inferences were derived from a targeted survey of the asia pacific region. therefore, generalizing the results can be viewed as an issue. the research highlights the significant factors that influence effective leadership for digital transformation, but more research is required on this subject to uncover further dimensions. future research can include data collection from other regions such as america, europe and africa which also may have different economic, governance and infrastructural developments. secondly, even though we have identified a few variables that influence effective leadership for digital transformation, upcoming revisions may discover additional factors to enhance the model further. thirdly, the research model framed is cross-sectional in nature which means, with this research model, we have measured the perceptions at a single point of time. however, there is a likelihood of changes expected in the leadership style in future as organisations transform to adapt to industry and technological advancement. 8 conclusion this study aimed at identifying the antecedents of effective digital leadership of enterprises in asia pacific. the literature review conducted identified leadership attributes and practices that would be of great influence on digital leadership of enterprises. the proposed research model included five hypotheses and the results of the structural equation model analysis provide insights to the leaders and enterprises of asia pacific to help rethink the traditional leadership and governance practices. key attributes identified such as risk taking, agility, tolerance of ambiguity, are critical for leaders in asia pacific to develop digital enterprises. the conceptualisation of factors influencing effective leadership provides direction to organisations, technology professionals, business leaders and researchers to focus not only on leadership traits or qualities but also on other factors such as strategic priorities, organisational focus areas and digital governance practices. our study will help the enterprises to drive digital transformation projects and collaborate with the service providers to increase the success rate of digital transformation and create more digital leaders in asia pacific. acknowledgements the authors would like to thank the editor and the two anonymous reviewers of this journal for their constructive feedback and suggestions for improvement of this paper. references accenture. (2016). the ethics of data sharing: a guide to best practices and governance [white paper]. accenture. retrieved form https://www.accenture.com/_acnmedia/pdf35/accenture-the-ethics-of-data-sharing.pdf. australasian journal of information systems karippur & balaramachandran 2022, vol 26, research article antecedents of effective digital leadership 26 agarwal, r., & helfat, c. e. (2009). strategic renewal of organizations. organization science, 20(2), 281-293. https://doi.org/10.1287/orsc.1090.0423 bahl, m. (2016). the work ahead the future of businesses and jobs in asia pacific’s digital economy [white paper]. cognizant. retrieved form https://www.cognizant.com/whitepapers/thework-ahead-the-future-of-business-and-jobs-in-asia-pacifics-digital-economycodex2255.pdf baškarada, s., watson, j., & cromarty, j. (2016). leadership and organizational ambidexterity. journal of management development, 35(6), pp. 778-788. https://doi.org/10.1108/jmd-01-2016-0004 bass, b. m. (1985). leadership and performance beyond expectations. collier macmillan. london, uk. bass, b. m. (1990). bass & stogdill’s handbook of leadership: theory, research and managerial applications. new york: the free press. bass, b. m. (1999). two decades of research and development in transformational leadership. european journal of work and organizational psychology, 8(1), 9-32. https://doi.org/10.1080/135943299398410 bass, b.m., & riggio, r.e., (2006). transformational leadership, 2nd ed. l. erlbaum associates, mahwah, n.j. https://doi.org/10.1080/10887150701451312 bharadwaj, a., el sawy, o. a., pavlou, p. a., & venkatraman, n. (2013). digital business strategy: toward a next generation of insights. mis quarterly, 471-482. bongiorno, g., rizzo, d., & vaia, g. (2018). cios and the digital transformation: a new leadership role. in cios and the digital transformation (pp. 1-9). springer, cham. https://doi 10.1007/978-3-319-31026-8_1 burns, j. m. (1978). row. new york. burley, k. (2019, december 2). emergence of traditional leadership styles. retrieved from https://bizfluent.com/info-8469011-emergence-traditional-leadership-styles.html. butler, t., & o’brien, l. (2019). understanding regtech for digital regulatory compliance. in disrupting finance (pp. 85-102). palgrave pivot, cham. https://doi.org/10.1007/978-3030-02330-0_6 carcary, m., doherty, e., & conway, g. (2016, september). a dynamic capability approach to digital transformation: a focus on key foundational themes. in the european conference on information systems management (p. 20). academic conferences international limited. carlson, k. d., & herdman, a. o. (2012). understanding the impact of convergent validity on research results. organizational research methods, 15, 17–32. https://doi.org/10.1177/1094428110392383. cascio, w. f., & montealegre, r. (2016). how technology is changing work and organizations. annual review of organizational psychology and organizational behavior, 3, 349-375. https://doi.org/10.1146/annurev-orgpsych-041015-062352 chee, c. w., & young, h. k. (2020). procurement: making digital transformation work for you. retrieved from https://www.de.kearney.com/procurement/article?/a/procurementriding-the-transformative-digital-wave. chen, m. y. c., lin, c. y. y., lin, h. e., & mcdonough, e. f. (2012). does transformational leadership facilitate technological innovation? the moderating roles of innovative culture and incentive compensation. asia pacific journal of management, 29(2), 239-264. https://doi.org/10.1007/s10490-012-9285-9 cisco. (2020). cisco annual internet report (2018 2023). [white paper]. cisco. retrieved from https://www.cisco.com/c/en/us/solutions/collateral/executive-perspectives/ https://doi.org/10.1287/orsc.1090.0423 https://doi.org/10.1108/jmd-01-2016-0004 https://doi.org/10.1007/s10490-012-9285-9 australasian journal of information systems karippur & balaramachandran 2022, vol 26, research article antecedents of effective digital leadership 27 annual-internet-report/white-paper-c11-741490.html. cortellazzo, l., bruni, e., & zampieri, r. (2019). the role of leadership in a digitalized world: a review. frontiers in psychology, 10, 1938. https://doi.org/10.3389/fpsyg.2019.01938 cullen, d., hui, d., & kittle, g. (2019). transforming asia: the search for digital leaders. heidrick & struggles. retrieved from https://www.heidrick.com/knowledgecenter/publication/transforming_asia_the_search_for_digital_leaders. dahlström, p., desmet, d., & singer, m. (2017). the seven decisions that matter in a digital transformation: a ceo’s guide to reinvention. digital mckinsey article (feb 2017). davenport, t., & mittal, n. (2020). how ceos can lead a data-driven culture. harvard business review. retrieved from https://hbr.org/2020/03/how-ceos-can-lead-a-datadriven-culture. de la boutetière, h., montagner, a., & reich, a. (2018). unlocking success in digital transformations. mckinsey & company. retrieved from https://www.mckinsey.com/~/media/mckinsey/business%20functions/organization/o ur%20insights/unlocking%20success%20in%20digital%20transformations/unlockingsuccess-in-digital-transformations.ashx. della corte, v., del gaudio, g., & sepe, f. (2019). leadership in the digital realm: what are the main challenges?. in digital leadership-a new leadership style for the 21st century. intechopen. https://doi.org/10.5772/intechopen.89856 deloitte (2016). decoding digital leadership. deloitte. retrieved from https://www2.deloitte.com/content/dam/deloitte/uk/documents/public-sector/deloitteuk-decoding-digital-leadership.pdf. deloitte (2017). how do companies prepare for the opportunities and challenges for digitalisation? retrieved from https://www2.deloitte.com/content/dam/deloitte/ch/ documents/consumer-business/ch-cip-en-swiss-transformation.pdf. devos, j., & van de ginste, k. (2015). towards a theoretical foundation of it governance: the cobit 5 case. in ecime, 18 (2), pp. 95-103). academic publishing limited. diaz-saenz, h. r. (2011). transformational leadership. the sage handbook of leadership, 5(1), 299-310. donahue, c. (2020). how to keep pace with digital transformation and avoid becoming obsolete. retrieved from https://www.forbes.com/sites/theyec/2020/02/12/how-to-keeppace-with-digital-transformation-and-avoid-becoming-obsolete/#21b0d7984861. dubey, a. (2019). this is what great leadership looks like in the digital age. [online] world economic forum. retrieved from https://www.weforum.org/agenda/2019/04/ leadership-digital-age-leader/. dykes, b. (2012). how to create a data-driven dynasty. adobe. retrieved from https://smartcomm.net.au/wp-content/uploads/2016/11/how-to-create-a-data-drivendynasty.pdf. el sawy, o. a., kræmmergaard, p., amsinck, h., & vinther, a. l. (2020). how lego built the foundations and enterprise capabilities for digital leadership. strategic information management, 174–201. https://doi.org/10.4324/9780429286797-8 fatima, t., majeed, m., & saeed, i. (2017). does participative leadership promote innovative work behavior: the moderated mediation model. business & economic review, 9(4), 139156. https://doi.org/10.22547/ber/9.4.7 fenwick, n., & schadler, t. (2018). digital rewrites the rules of business. forrester, cambridge, massachusettes. australasian journal of information systems karippur & balaramachandran 2022, vol 26, research article antecedents of effective digital leadership 28 finzi, b., firth, v., & lipton, m. (2018). ambidextrous leadership: keystone of the undisruptable ceo. deloitte insights. fornell, c., & larcker, d. f. (1981). structural equation models with unobservable variables and measurement error: algebra and statistics. journal of marketing research, 18(3), 382– 388. https://doi.org/10.2307/3150980 garcía-morales, v. j., jiménez-barrionuevo, m. m., & gutiérrez-gutiérrez, l. (2012). transformational leadership influence on organizational performance through organizational learning and innovation. journal of business research, 65(7), 1040-1050. https://doi.org/10.1016/j.jbusres.2011.03.005 geissbauer, r., vedso, j., & schrauf, s. (2016). industry 4.0: building the digital enterprise. pwc. retrieved from https://www.pwc.com/gx/en/industries/industries-4.0/landingpage/industry-4.0-building-your-digital-enterprise-april-2016.pdf. gentle, c. & metselaar, j. (2020). achieving ambidextrous leadership: driving 'perform & transform' innovation for step-changed business results. retrieved from https://cmr.berkeley.edu/2020/04/ambidextrous-leadership/. ghosh, b., burden, a., & wilson, j. (2019). how to scale innovation and achieve full value with future systems. accenture. retrieved from https://www.accenture.com/_acnmedia/ thought-leadership-assets/pdf/accenture-future-systems-report.pdf. gottlieb, j., & willmott, p. (2014). the digital tipping point: mckinsey global survey results. mckinsey & company, mckinsey quarterly. hair, j. f., ringle, c. m., & sarstedt, m. (2011). pls-sem: indeed a silver bullet. journal of marketing theory and practice, 19(2), 139–152. https://doi.org/10.2753/mtp10696679190202 hair, j. f., black, w. c., babin, b. j., & anderson, r. e. (2013). multivariate data analysis. pearson new international edition. pearson higher ed. hair jr, j. f., hult, g. t. m., ringle, c., & sarstedt, m. (2016). a primer on partial least squares structural equation modeling (pls-sem). sage publications, usa. harvard business review (2014). delivering on the promise of digital transformation. harvard business review analytic services report 2014. retrieved from https://www.bmcsoftware.sg/content/dam/bmc/migration/pdf/hbr_bmc_report_au g2014-final.pdf. harvard business review (2017). operationalizing digital transformation: new insights into making digital work. harvard business review analytic services report 2017. harvard business school publishing. retrieved from https://hbr.org/resources/pdfs/comm/xl/ hbrasoperationalizingdigitaltransformation.pdf. havermans, l. a., den hartog, d. n., keegan, a., & uhl-bien, m. (2015). exploring the role of leadership in enabling contextual ambidexterity. human resource management, 54(1), 179-200. https://doi.org/10.1002/hrm.21764 heneghan, l., snyder, m., & symons, c. (2017). the route to digital business leadership. kpmg. retrieved from https://assets.kpmg/content/dam/kpmg/xx/pdf/2017/09/theroute-to-digital-business-leadership.pdf. henseler, j., hubona, g., & ray, p. a. (2016). using pls path modeling in new technology research: updated guidelines. industrial management & data systems. https://doi.org/10.1108/imds-09-2015-0382 hess, t., matt, c., benlian, a., & wiesböck, f. (2020). options for formulating a digital transformation strategy. in strategic information management (pp. 151-173). routledge. https://doi.org/10.4324/9780429286797-7 australasian journal of information systems karippur & balaramachandran 2022, vol 26, research article antecedents of effective digital leadership 29 hill, l. a. (2020). being the agile boss. mit sloan management review, 62(1), 7-10. hooper, d., coughlan, j., & mullen, m. r. (2008). structural equation modelling: guidelines for determining model fit. electronic journal of business research methods, 6(1), 53-60. huawei. (2020, august 6). huawei empowers asia pacific digital transformation, leading to adaptive enterprises in post-covid world. retrieved from https://e.huawei.com/sg/news/ebg/2020/asia-pacific-digital-transformation. hyek, p., & nendick, j. (2014). sustaining digital leadership. ernst and young. international data corporation (idc). (2020, november 4). idc predicts 65% of apac* gdp will be digitalized reaching us$ 1.2 trillion in spending by 2022. retrieved from https://www.idc.com/getdoc.jsp?containerid=prap46972820. international data group (idg). (2018). 2018 state of digital business transformation [white paper]. idg. https://www.idg.com/tools-for-marketers/2018-state-of-digital-businesstransformation-white-paper/. imran, f., shahzad, k., butt, a., & kantola, j. (2020, july). leadership competencies for digital transformation: evidence from multiple cases. in international conference on applied human factors and ergonomics (pp. 81-87). springer, cham.4. http://dx.doi.org/0.1007/978-3-030-50791-6_11 ittner, c. (2019). how digital technology is transforming cost management. retrieved from https://knowledge.wharton.upenn.edu/article/cost-management-in-the-digital-age/. jimenez, d. z., lim, v., cheok, l., & ng, h. (2018). unlocking the economic impact of digital transformation in asia pacific [white paper]. microsoft and international data corporation (idc) asia/pacific. https://news.microsoft.com/wpcontent/uploads/prod/sites/43/2018/11/unlocking-the-economic-impact-of-digitaltransformation.pdf. judge, t. a., bono, j. e., ilies, r., & gerhardt, m. w. (2002). personality and leadership: a qualitative and quantitative review. journal of applied psychology, 87(4), 765-780. https://doi.org/10.1037/0021-9010.87.4.765 kahai, s. s. (2013). leading in a digital age: what’s different, issues raised, and what we know. exploring distance in leader–follower relationships: when near is far and far is near. new york, routledge, s, 63-108. kane, g. c., palmer, d., phillips, a. n., kiron, d., & buckley, n. (2015). strategy, not technology, drives digital transformation. mit sloan management review and deloitte university press, 14(1-25). kane, g. c., palmer, d., nguyen-phillips, a., kiron, d., & buckley, n. (2017). achieving digital maturity. mit sloan management review, 59(1). kane, g. c., phillips, a. n., copulsky, j., & andrus, g. (2019). how digital leadership is (n't) different. mit sloan management review, 60(3), 34-39. karimi, j., & walter, z. (2015). the role of dynamic capabilities in responding to digital disruption: a factor-based study of the newspaper industry. journal of management information systems, 32(1), 39-81. https://doi.org/10.1080/07421222.2015.1029380 knight, w. (2020). the 9 dimensions of innovation for digital leaders. retrieved from https://digileaders.com/the-9-dimensions-of-innovation-for-digital-leaders. kohli, r., & johnson, s. (2011). digital transformation in latecomer industries: cio and ceo leadership lessons from encana oil & gas (usa) inc. mis quarterly executive, 10(4). kassotaki, o. (2019). ambidextrous leadership in high technology organizations. organizational dynamics, 48(2), 37-43. https://doi.org/10.1016/j.orgdyn.2018.10.001 https://news.microsoft.com/wp-content/uploads/prod/sites/43/2018/11/unlocking-the-economic-impact-of-digital-transformation.pdf https://news.microsoft.com/wp-content/uploads/prod/sites/43/2018/11/unlocking-the-economic-impact-of-digital-transformation.pdf https://news.microsoft.com/wp-content/uploads/prod/sites/43/2018/11/unlocking-the-economic-impact-of-digital-transformation.pdf australasian journal of information systems karippur & balaramachandran 2022, vol 26, research article antecedents of effective digital leadership 30 kreutzer, r. t., neugebauer, t., & pattloch, a. (2018). digital business leadership: digital transformation, business model innovation, agile organization, change management (pp. 263). springer, berlin. kumar, k. n. (2017). digital transformation in asia-pacific. cutter consortium. retrieved from https://www.cutter.com/article/digital-transformation-asia-pacific-2017-495011. larjovuori, r. l., bordi, l., mäkiniemi, j. p., & heikkilä-tammi, k. (2016). the role of leadership and employee well-being in organizational digitalization. tiziana russospenaand cristina mele, 1159. larjovuori, r. l., bordi, l., & heikkilä-tammi, k. (2018, october). leadership in the digital business transformation. in proceedings of the 22nd international academic mindtrek conference (pp. 212-221). https://doi.org/10.1145/3275116.3275122 li, w., liu, k., belitski, m., ghobadian, a., & o'regan, n. (2016). e-leadership through strategic alignment: an empirical study of small-and medium-sized enterprises in the digital age. journal of information technology, 31(2), 185-206. https://doi.org/10.1057/jit.2016.10 link, j. (2018). why organisations need digital leaders with these five key strengths. forbes media llc. retrieved from https://www.forbes.com/sites/forbeshumanresources council/2018/10/04/why-organizations-need-digital-leaders-with-these-five-keystrengths/#12b873567aee. liu, l., yu, b., & wu, w. (2019). the formation and effects of exploitative dynamic capabilities and explorative dynamic capabilities: an empirical study. sustainability, 11(9), 2581. https://doi.org/10.3390/su11092581 london & partners. (2021, january 14). global tech investors show confidence in london. retrieved from https://media.londonandpartners.com/news/global-tech-investorsshow-confidence-in-london. lunardi, g. l., maçada, a. c. g., & becker, j. l. (2014, january). it governance effectiveness and its antecedents: an empirical examination in brazilian firms. in 2014 47th hawaii international conference on system sciences (pp. 4376-4385). ieee. https://doi.org/10.1109/hicss.2014.540 march, j. g. (1991). exploration and exploitation in organizational learning. organization science, 2(1), 71-87. matt, c., hess, t., & benlian, a. (2015). digital transformation strategies. business & information systems engineering, 57(5), 339-343. https://doi.org/10.1007/s12599-015-0401-5 neubauer, r., tarling, a., & wade, m. (2017). redefining leadership for a digital age. global centre for digital business transformation, 1-15. nsw treasury. (2019). sydney facts. retrieved from https://invest.nsw.gov.au/whynsw/sydney-facts. o reilly, c. a., & tushman, m. l. (2004). the ambidextrous organization. harvard business review, 82(4), 74-83. o’reilly iii, c. a., & tushman, m. l. (2008). ambidexterity as a dynamic capability: resolving the innovator's dilemma. research in organizational behavior, 28, 185-206. https://doi.org/10.1016/j.riob.2008.06.002 o'reilly iii, c. a., & tushman, m. l. (2011). organizational ambidexterity in action: how managers explore and exploit. california management review, 53(4), 5-22. https://doi.org/10.1525/cmr.2011.53.4.5 https://media.londonandpartners.com/news/global-tech-investors-show-confidence-in-london https://media.londonandpartners.com/news/global-tech-investors-show-confidence-in-london australasian journal of information systems karippur & balaramachandran 2022, vol 26, research article antecedents of effective digital leadership 31 o'reilly iii, c. a., & tushman, m. l. (2013). organizational ambidexterity: past, present, and future. academy of management perspectives, 27(4), 324-338. https://doi.org/10.5465/amp.2013.0025 oberer, b., & erkollar, a. (2018). leadership 4.0: digital leaders in the age of industry 4.0. international journal of organizational leadership. https://doi.org/10.33844/ijol.2018. 60332 pallant, j. (2007). spss survival manual 3rd edition. new york: mcgraw hill. pearce, d. l. (2013). implementing innovation with ambidextrous leadership in small and medium enterprises. fisher digital publication, new york. popadiuk, s., luz, a. r. s., & kretschmer, c. (2018). dynamic capabilities and ambidexterity: how are these concepts related?. revista de administração contemporânea, 22(5), 639-660. https://doi.org/10.1590/1982-7849rac2018180135 raelin, j. a. (ed.). (2016). leadership-as-practice: theory and application. routledge. new york and london rizvi, m. (2017, march 22). wheel of fortune favours dubai. khaleejtimes. retrieved from https://www.khaleejtimes.com/business/corporate/wheel-of-fortune-favours-dubai. rogers, d. l. (2016). the digital transformation playbook: rethink your business for the digital age. columbia university press, new york. roldán, j. l., & sánchez-franco, m. j. (2012). variance-based structural equation modeling: guidelines for using partial least squares in information systems research. in research methodologies, innovations and philosophies in software systems engineering and information systems (pp. 193-221). igi global. https://doi.org/10.4018/978-1-4666-0179-6.ch010 sainger, g. (2018). leadership in digital age: a study on the role of leader in this era of digital transformation. international journal on leadership, 6(1), 1. sarstedt, m., wilczynski, p., & melewar, t. c. (2013). measuring reputation in global markets—a comparison of reputation measures’ convergent and criterion validities. journal of world business, 48(3), 329-339. https://doi.org/10.1016/j.jwb.2012.07.017 schadler, t., fenwick, n., gill, m., cazalet, a., & hartig, k (2017). the digital business imperative. forrester. retrieved from https://www.forrester.com/report/ the+digital+business+imperative/-/e-res115784. singapore economic development board. (2020, august 19). ride the wave of asian growth. retrieved from https://www.edb.gov.sg/en/our-industries/industries-and-keyactivities/headquarters.html. solis, b., and syzmanski, j., (2016). the race against digital darwinism: six stages of digital transformation. altimeter group. sow, m., & aborbie, s. (2018). impact of leadership on digital transformation. business and economic research, 8(3), 139-148. https://doi.org/10.5296/ber.v8i3.13368 statista research department. (2021, march 29). apac: mobile internet user penetration 20182025. retrieved from https://www.statista.com/statistics/201232/forecast-of-mobileinternet-penetration-in-asia-pacific/. steele, c., allen, l., & watkins, a. (2019). the ethical use of customer data in a digital economy. kpmg. retrieved from https://home.kpmg/content/dam/kpmg/uk/pdf/2019/04/ethicaluse-of-customer-data.pdf. svahn, f., mathiassen, l., & lindgren, r. (2017). embracing digital innovation in incumbent firms: how volvo cars managed competing concerns. mis quarterly, 41(1). https://doi.org/10.25300/misq/2017/41.1.12 australasian journal of information systems karippur & balaramachandran 2022, vol 26, research article antecedents of effective digital leadership 32 swift, m., & lange, d. (2018). digital leadership in asia pacific. korn ferry institute. retrieved from https://focus.kornferry.com/wp-content/uploads/2018/04/korn-ferry-digitalleadership-in-asia-pacific.pdf. tan, a. (2020). it priorities: 2020: apac. tech target. retrieved from https://www.computerweekly.com/ehandbook/it-priorities-2020-apac. takle, j. (2018). ambidextrous leadership transformational skills for a digital age. retrieved from https://www.linkedin.com/pulse/ambidextrous-leadership-transforming-banksdigital-age-jeremy-takle. tannou, m., & westerman, g. (2012). governance: a central component of successful digital transformation. mit center for digital business and capgemini consulting, london. tardieu, h., daly, d., esteban-lauzán, j., hall, j., & miller, g. (2020). leadership—what is required of leaders and leadership to achieve digital success?. in deliberately digital (pp. 95-105). springer, cham. https://doi.org/10.1007/978-3-030-37955-1_11 teece, d. j. (2007). explicating dynamic capabilities: the nature and microfoundations of (sustainable) enterprise performance. strategic management journal, 28(13), 13191350. https://doi.org/10.1002/smj.640 teece, d. j. (2009). dynamic capabilities and strategic management: organizing for innovation and growth. oxford university press on demand, new york. teece, d. j. (2014). the foundations of enterprise performance: dynamic and ordinary capabilities in an (economic) theory of firms. academy of management perspectives, 28(4), 328-352. https://doi.org/10.5465/amp.2013.0116 teece, d. j., & linden, g. (2017). business models, value capture, and the digital enterprise. journal of organization design, 6(1), 1-14. https://doi.org/10.1002/smj.640 tong, j. (2017). key components of digital governance in organisations. retrieved from https://www.joshtong.io/blog/2017/11/14/key-components-of-digital-governance-inorganizations. tushman, m. l., & o’reilly, c. a. (1996). ambidextrous organizations: managing evolutionary and revolutionary change. california management review, 38(4), 8– 29. https://doi.org/10.2307/41165852https://doi.org/10.2307/41165852 valentine, e., & stewart, g. (2015, january). enterprise business technology governance: three competencies to build board digital leadership capability. in 2015 48th hawaii international conference on system sciences (pp. 4513-4522). ieee. https://doi.org/10.1109/hicss.2015.539 vey, k., fandel-meyer, t., zipp, j. s., & schneider, c. (2017). learning & development in times of digital transformation: facilitating a culture of change and innovation. international journal of advanced corporate learning (ijac), 10(1), 22-32. https://doi.org/10.3991/ijac.v10i1.6334 vial, g. (2019). understanding digital transformation: a review and a research agenda. the journal of strategic information systems, 28(2), 118-144. https://doi.org/10.1016 /j.jsis.2019.01.003 warner, k. s., & wäger, m. (2019). building dynamic capabilities for digital transformation: an ongoing process of strategic renewal. long range planning, 52(3), 326-349. https://doi.org/10.1016/j.lrp.2018.12.001 welchman, l. (2015). managing chaos: digital governance by design. rosenfeld media. werts, c. e., rock, d. r., linn, r. l., & joreskog, k. g. (1978). a general method of estimating the reliability of a composite. educational and psychological measurement, 38(4), 933–938. https://doi.org/10.1177/001316447803800412 https://doi.org/10.2307%2f41165852 australasian journal of information systems karippur & balaramachandran 2022, vol 26, research article antecedents of effective digital leadership 33 westerman, g., bonnet, d., & mcafee, a. (2014a). leading digital: turning technology into business transformation. harvard business press, usa. westerman, g., bonnet, d., & mcafee, a. (2014b). the nine elements of digital transformation. mit sloan management review, 55(3), 1-6. world economic forum (2018). the digital enterprise: moving from experimentation to transformation. world economic forum in collaboration with bain and company. retrieved from: http://www3.weforum.org/docs/media/47538_digital%20enterprise_ moving_experimentation_transformation_report_2018%20-%20final%20(2).pdf wrede, m., velamuri, v. k., & dauth, t. (2020). top managers in the digital age: exploring the role and practices of top managers in firms' digital transformation. managerial and decision economics. https://doi.org/10.1002/mde.3202 yokoi, t., shan, j., wade, m., & macaulay, j. (2019). digital vortex 2019: continuous and connected change. zeike, s., bradbury, k., lindert, l., & pfaff, h. (2019). digital leadership skills and associations with psychological well-being. international journal of environmental research and public health, 16(14), 2628. https://doi.org/10.3390/ijerph16142628 zhang, x. a., cao, q., & tjosvold, d. (2011). linking transformational leadership and team performance: a conflict management approach. journal of management studies, 48(7), 1586-1611. https://doi.org/10.1111/j.1467-6486.2010.00974.x appendix constructs and measures summary of proposed indicators and their independent variables, as advised by past studies and journal articles. qno indicator literature independent variable la1 digital visionary (della corte, del gaudio, & sepe, 2019; cullen, hui, & kittle, 2019;larjovuori, bordi, & heikkilä-tammi, 2018; swift & lange, 2018; neubauer, tarling, & wade, 2017; heneghan, snyder, & symons, 2017; deloitte, 2016) leadership attributes (la) la2 smart risk taker la3 foster innovation la4 agility la5 tolerance of ambiguity la6 natural collaborator la7 builds digital talent sp1 prioritise new business models (schadler, fenwick, gill, cazalet, & hartig, 2017; popadiuk et al., 2018; gottlieb & willmott, 2014) strategic priorities (sp) sp2 build competitive advantage sp3 keep pace with competitors sp4 cost management australasian journal of information systems karippur & balaramachandran 2022, vol 26, research article antecedents of effective digital leadership 34 qno indicator literature independent variable ofa1 diversify business models through creative partnerships (world economic forum, 2018; . finzi, firth, & lipton, 2018; fenwick & schadler, 2018; de la boutetière, montagner, & reich, 2018; geissbauer, vedso, & schrauf, 2016; westerman, bonnet, & mcafee, 2014a, 2014b) organisational focus areas (ofa) for exploration ofa2 facilitate data sharing and data analytics ofa3 adopt new platforms to meet new business challenges ofa4 identify the roles and stakeholder partnerships in business value chain ofa5 establish digital teams to foster digital innovation ofa6 empower employees to adopt digital technologies dg1 regulatory compliance (bongiorno et al., 2018; solis, 2016; hyek & nendick, 2014; tannou & westerman, 2012; dykes, 2012) digital governance practices for exploitation (dg) dg2 cyber security planning dg3 ethical data handling dg4 social media management dg5 employee collaboration el2 clear digital vision (della corte et al., 2019; kane et al., 2019; swift & lange, 2018; popadiuk et al., 2018; neubauer, tarling, & wade, 2017; heneghan, snyder, & symons, 2017) benefits of effective digital leadership el2 adept at aligning business and it strategy el3 higher focus on innovation and growth el4 aggressive investment in agile and disruptive technologies el5 builds on stable and secure infrastructure el6 builds strategic partnerships el7 nurtured the talent to derive and deliver digital strategy australasian journal of information systems karippur & balaramachandran 2022, vol 26, research article antecedents of effective digital leadership 35 copyright: © 2022 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v26i0.2525 http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 literature review 2.1 digital enterprise 2.2 dynamic capabilities 2.3 leadership in the digital age 2.4 theories of leadership 2.5 ambidextrous leadership 3 research model and hypotheses 3.1 leadership attributes 3.2 strategic priorities 3.3 organisational focus areas for exploration 3.4 digital governance practices for exploitation 4 research method, data, and analysis 4.1 survey instrument 4.2 sampling process and data collection 4.3 data analysis 4.3.1 reliability 4.3.2 validity convergent validity discriminant validity figure 2. structural equation model with path coefficients 4.4 structured equation modelling / path analysis 4.4.1 hypothesis testing 5 discussion 5.1 summary of findings 5.2 benefits of effective leadership 6 implications 6.1 implications for research 6.2 implications for practice 7 limitations of the study and future scope 8 conclusion the australasian journal of information systems (ajis) began in 1993 and was based on the simple realisation that there was a need for an academic ‘outlet’ for is research in australia australasian journal of information systems volume 18 number 2 2014 3 editorial i have to admit that i feel a certain irony in taking on the position of editor-in-chief for ajis, given my well known views on the future of academic journals, or rather the lack of a future for them. i know that more than a few of my colleagues will also see this appointment as ironic. and yet, that appearance of irony is just that, an appearance. there is no question that every process in the chain of communicating the results of our research is currently subject to question, and it should be no surprise to us as information systems researchers that once again a disruptive technology is making its effects felt. i am not going to go into the details of my views on that particular issue, i am currently working on a paper looking at those technological changes and their effects. the point i would like to make, is that times of radical change are dangerous times. they offer huge potential for experimenting with new and different possibilities that are enabled by the changes. some of these experiments, such as the wide area information service (wais), fail as new technical advances are made, others fail as business models render them uncompetitive. arguably many of the paper based publishing models are falling into that category. if we look at academic publishing, even many of the open access electronic publication models simply translate existing models into the new online environment with minimal changes. in the 1990s we used to talk of “gold plating the cow paths” when this was done in many business implementations. in 1993, in the first issue of ajis, the foundation editor-in-chief, rob mcgregor said “there is a growing respect by the international community towards the research and scholarship that is emanating from australia.” i have no doubt that sentiment was a motivating factor in establishing ajis, and it is effectively disseminating that research and raising its profile in australasia and internationally that has to be the essence of our effort. to deliver this we have to focus on the best ways to achieve it, even if that requires changes to our processes in response to these developments. i believe that my background and understanding of these changes will help navigate ajis into this new era of academic publishing, in whatever form that may take. that’s all a long term perspective. in the short term, there are a number of other things i’m working on to improve and raise the profile of the journal. ebsco product managers have nominated the journal for promotion on ebsco research databases. we are also negotiating with cengage for a similar arrangement. this will bring greater exposure of articles published by ajis. we have a research notes section in ajis now. a research note is an article giving a brief description of a breakthrough is topic or idea. it may also refer to the modification of a methodology, method or associated tool. it should be timely and of interest to ajis readers. research notes provide an outlet for publication of information that is relevant and important, but may lack spatial or temporal replication. it should be no more than 2,500 words, and can include two figures or tables. it should have at least eight references. research notes will also be sent for peer review. research notes are not designed to serve as an outlet for publication of research that lacks appropriate scope or is better suited for other outlets. i also want to increase the number and coverage of themed issues. already craig parker has a team preparing papers focused on small and medium sized enterprises for issue 19(1). future themes include: • best papers from the foundations of information systems conference • indigenous information systems • intelligent decision making in information technology • the economic and societal impacts of broadband adoption in rural and regional communities australasian journal of information systems volume 18 number 2 2014 4 in this issue we have articles spanning decision support, gender, strategy and the like. some articles are empirical studies and some are reflective. they are examining issues within australia, and in the broader region. i hope you find at least one article to interest you. if not, then submit one! my sincere thanks to my head of school, dineli mather, for her support of my role as editor-in-chief, and to aais and acphis for the help and support that they have provided for this issue, and the acs for providing hosting services for the journal. not only is their help financial, but also in the support for the journal that they are fostering in the academic community. dr john lamp ajis editor-in-chief deakin university june 2014 paper title (paper title style) australasian journal of information systems shankar, jebarajakirthy & kumari 2021, vol 25, research on consumer evaluation special section editorial of mobile applications 1 editorial for the special section on research on consumer evaluation of mobile applications: does interactivity matter? amit shankar indian institute of management visakhapatnam, india ashankar@iimv.ac.in charles jebarajakirthy griffith business school, griffith university, australia pooja kumari indian institute of management jammu, india 1 background information technology advancement, internet revolution and mobile phone penetration have enabled organisations to revolutionize their service delivery patterns (shankar and jebarajakirthy, 2019; shankar et al., 2020). organisation are using a variety of innovative electronic channels to enhance the consumer reachability and achieve competitive advantages in a cost-effective manner (islam et al., 2021). among all electronic service delivery channels, mobile applications are one of the most promising channels with a great degree of ubiquity and localization (koenig-lewis et al., 2010; shankar, 2021). mobile applications are beneficial for both service providers and consumers. they enable consumers to avail customized services anytime from anywhere and allow organisations to provide cost-effective and time-efficient services to consumers (shankar and jain, 2021). due to the absence of face-to face interaction, the success of mobile commerce heavily depends on the interactivity of the mobile applications (jebarajakirthy and shankar, 2021). in the context of mobile commerce applications, interactivity refers to the layout of applications and contents that affect consumers’ experience while they interact with a mobile application (coursaris and sung, 2012; lee et al., 2015; islam et al., 2021). interactivity may be conceptualized as a multidimensional construct consisting of two major elements, namely feature orientation and perception orientation (gao et al., 2009). feature-oriented elements refer to some functional characteristics of a mobile application including, layout, navigation, content, and customer support that impact application interactivity (coursaris and sung, 2012). perception-oriented elements include playfulness, perceived hedonic benefits, user control, technology frustration, consumer passion and responsiveness that are concerned with how consumers perceive interactivity (cyr et al., 2009). bad design of the mobile applications leads to technology frustration which negatively impacts the consumer evaluation of the offered services (shankar et al., 2021). hence, marketers are keen to know how they can enhance feature-oriented interactivity and perception-oriented interactivity to reduce technology frustration and enhance consumer experience during the consumer journey on mobile application platforms. australasian journal of information systems shankar, jebarajakirthy & kumari 2021, vol 25, research on consumer evaluation special section editorial of mobile applications 2 this special section explores how marketers can enhance the consumer-brand relationship quality by enhancing mobile application interactivity (shankar and yadav, 2020). hence, the special section pursues the following objectives: it aims to • explore different theoretical models to investigate feature-oriented interactivity and perception-oriented interactivity elements of mobile applications • examine the impact of mobile application interactivity on consumer service evaluation • challenge the existing frameworks of mobile application interactivity • examine the impact of technology frustration and consumer passion on consumer experience, and finally, • explore the impact of mobile application interactivity on consumer perceived value. 2 articles in the special section the following four articles have been selected for inclusion in the special section: article 1: what’s behind a scratch card? designing a mobile application using gamification to study customer loyalty: an experimental approach in this study, the authors investigate how gamification elements in a mobile payment application affect hedonic and utilitarian motivations which in turn, lead to customer loyalty. the study also examines how the effects of motivations are moderated and mediated by different types of rewards (direct cash rewards vs indirect third party partnered rewards). a 2x2 experimental design with 385 respondents was conducted and the results indicate that direct cash rewards are more influential in a gamification scenario. article 2: why have consumers opposed, postponed, and rejected innovations during a pandemic? a study of mobile payment innovations using innovation resistance theory, the present study investigates consumer resistance towards mobile payment during the covid-19 pandemic. an artificial neural network analysis (ann) of data collected from 406 non-users of m-payments in india confirmed that usage barriers, value barriers, risk barriers, traditional barriers, and image barriers are major antecedents of resistance towards mobile payment. article 3: interactive app-based services: recovery evaluations and the mediating role of satisfaction in the relationship between customer-brand engagement and electronic word of mouth this study examines the differential impact of varying levels of customer participation in service recovery on recovery satisfaction, customer-brand engagement, and electronic-word of mouth by using co-creation theory and resource-based theory. the analysis of data collected from 495 customers of interactive food delivery apps in india showed that co-creation in recovery significantly influences customer-brand engagement and electronic-word of mouth and that recovery satisfaction mediated the relationship between customer-brand engagement and electronic-word of mouth. australasian journal of information systems shankar, jebarajakirthy & kumari 2021, vol 25, research on consumer evaluation special section editorial of mobile applications 3 article 4: does gamified interaction build a strong consumer-brand connection? a study of mobile applications the study analyses the relationship between the constructs of flow, brand engagement, selfbrand connection and brand usage intention among consumers of two gamified mobile applications in india. the research results suggest that flow formed by the five dimensions challenge, feedback, autonomy, immersion, and interaction positively associates with cognitive brand engagement and emotional brand engagement with both forms of engagement further strengthening consumers’ brand connection and usage intention. 3 conclusion the focus of the articles in this special section is on investigating the effects of mobile application interactivity on consumers’ behavioural intentions. the articles cover various aspects of interactivity such as, gamification, co-creation, and application design. the studies also examine how interactive features of mobile applications affect consumers’ behavioural intentions and behaviours such as, usage intention, loyalty, brand engagement, and electronic word of mouth. the special section enriches the mobile application interactivity literature and provides several suggestions for mobile commerce marketers how to enhance consumer experience by using interactivity features. acknowledgement we like to acknowledge the work of the reviewers for this special section who assessed the submissions and provided their valuable suggestions to improve the quality of the papers that were submitted and those that were ultimately accepted for publication in the special section. we would also like to thank associate professor john lamp and professor karlheinz kautz for their continuous guidance and support without which this special section would not have been possible. references coursaris, c. k., & sung, j. (2012). antecedents and consequents of a mobile website’s interactivity. new media & society, 14(7), 1128-1146. cyr, d., head, m., & ivanov, a. (2009). perceived interactivity leading to e-loyalty: development of a model for cognitive–affective user responses. international journal of human-computer studies, 67(10), 850-869. gao, q., rau, p. l. p., & salvendy, g. (2009). perception of interactivity: affects of four key variables in mobile advertising. international journal of human-computer interaction, 25(6), 479-505. islam, h., jebarajakirthy, c., & shankar, a. (2021). an experimental based investigation into the effects of website interactivity on customer behavior in on-line purchase context. journal of strategic marketing, 29(2), 117-140. koenig-lewis, n., palmer, a., & moll, a. (2010). predicting young consumers' take up of mobile banking services. international journal of bank marketing, 28(5), 410-432. lee, d., moon, j., kim, y. j., & mun, y. y. (2015). antecedents and consequences of mobile phone usability: linking simplicity and interactivity to satisfaction, trust, and brand loyalty. information & management, 52(3), 295-304. australasian journal of information systems shankar, jebarajakirthy & kumari 2021, vol 25, research on consumer evaluation special section editorial of mobile applications 4 shankar, a. (2021). how does convenience drive consumers' webrooming intention?. international journal of bank marketing, 39(2), 312-336. shankar, a., & jain, s. (2021). factors affecting luxury consumers’ webrooming intention: a moderated-mediation approach. journal of retailing and consumer services, 58, 102306. shankar, a., & jebarajakirthy, c. (2019). the influence of e-banking service quality on customer loyalty: a moderated mediation approach. international journal of bank marketing, 37(5), 1119-1142. shankar, a., & yadav, r. (2020). understanding the impact of csr domain on brand relationship quality. marketing intelligence & planning, 39(4), 559-573. shankar, a., jebarajakirthy, c., & ashaduzzaman, m. (2020). how do electronic word of mouth practices contribute to mobile banking adoption?. journal of retailing and consumer services, 52, 101920. shankar, a., tiwari, a. k., & gupta, m. (2021). sustainable mobile banking application: a text mining approach to explore critical success factors. journal of enterprise information management. https://doi.org/10.1108/jeim-10-2020-0426 copyright: © 2021 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v25i0.3635 https://doi.org/10.1108/jeim-10-2020-0426 http://creativecommons.org/licenses/by-nc/3.0/au/ 1 background 2 articles in the special section 3 conclusion microsoft word ajis_18(3)_v10.docx australasian journal of information systems volume 18 number 3 2014 193 modelling the intention to adopt cloud computing services: a transaction cost theory perspective ogan yigitbasioglu school of accountancy queensland university of technology ogan.yigitbasioglu@qut.edu.au abstract this paper uses transaction cost theory to study cloud computing adoption. a model is developed and tested with data from an australian survey. according to the results, perceived vendor opportunism and perceived legislative uncertainty around cloud computing were significantly associated with perceived cloud computing security risk. there was also a significant negative relationship between perceived cloud computing security risk and the intention to adopt cloud services. this study also reports on adoption rates of cloud computing in terms of applications, as well as the types of services used. keywords: cloud computing, vendor trust, security, reliability, saas, iaas, paas introduction despite the widely publicized benefits of cloud computing, the adoption of cloud-based solutions is not as high as one would expect according to evidence from studies in germany (benlian & hess, 2011), taiwan (low, chen, & wu, 2011), india, and southeast asia (gupta, seetharaman, & raj, 2013). also, a report by the australian communications and media authority revealed that 66% of smes did not utilize cloud computing services, and those that did use it mainly for webmail (australian, 2014). cloud computing has for many years attracted much attention from the business press, yet it is only more recently that empirical research has started to emerge in this field. research to date has focussed on the factors that affect the intention to adopt cloud computing services in terms of expected benefits such as cost advantages and strategic flexibility, and its risks which, among others, relate to security, performance, and the hidden costs (alshamaila, papagiannidis, & li, 2013; benlian & hess, 2011; lee, chae, & cho, 2013). other research has looked at a number of organizational factors that engender its adoption such as business process characteristics and top management support (low, et al., 2011; wu, cegielski, hazen, & hall, 2013). although, a body of knowledge on the factors that drive the intention to adopt cloud computing services is forming, evidence is scarce and is restricted to a handful of geographic regions. furthermore, a number of other factors remain relatively unexplored such as vendor trust, the uncertainty around cloud computing legislation, and the security and reliability of the service. also, there is little knowledge of the types of cloud computing solutions used in terms of applications and services outsourced. this study aims to address this gap using transaction cost theory (tce) (williamson, 1985). tce emphasizes the lack of trust in exchange relationships, as well as the various uncertainties that surround transactions, including technological uncertainty (sutcliffe & zaheer, 1998). these concepts may be of relevance in explaining organisations’ intention to adopt cloud computing services. the purpose of this paper is to extend our understanding of the factors that drive cloud computing adoption, as well as to provide some evidence on adoption rates from a new region, which is australia. from a tce point of view, i expect to find a relation between perceived uncertainty around the legislation of cloud computing and the perceived reliability and security of the service. also, vendor opportunism is predicted to determine the perceived reliability and security of the service. finally, i australasian journal of information systems volume 18 number 3 2014 194 expect to find an association between the perceived reliability and security of the service and the intention to adopt cloud computing. the study makes a number of contributions to the cloud computing literature. first, i examine the impact of legislative uncertainty on the perceived security and reliability of the service, whereas most studies do not focus on legislation. second, i test for the relationship between vendor opportunism and the perceived reliability and security of the service, which has received little attention in the cloud computing literature. third, a methodological contribution is made by developing a more comprehensive construct for cloud computing security that captures confidentiality, integrity, and availability of information. the structure of the paper is as follows. first, cloud computing is defined, which is followed by a review of the literature, particularly empirical papers on cloud computing from business and legal perspectives. next, hypotheses are developed and the methodology is presented. this is followed with the analysis of the data and the results. finally, the paper is concluded with a discussion of the results, as well as the implications for research and practice. literature review and hypotheses development cloud computing is a form of it outsourcing (ito) where it resources are purchased over the internet on a pay-per-use basis (marston, li, bandyopadhyay, zhang, & ghalsasi, 2011; ross & blumenstein, 2013). cloud computing services can be classified into three distinct groups: software as a service (saas), platform as a service (paas), and infrastructure as a service (iaas) (marston, et al., 2011). whereas saas enables the access to software, paas allows the remote development and deployment of applications. organizations also have the option to lease computer infrastructure without receiving any vendor support which is known as iaas (youseff, butrico, & da silva, 2008). two types of cloud computing configurations exist: private and public clouds (marston, et al., 2011). private clouds are similar to the traditional in-house it delivery model, whereby computing resources are either kept within the boundaries of the firm or alternatively, dedicated hardware are leased from a provider. in contrast, public cloud computing services use virtualization to host multiple organisations’ data on a common server and the data may be moved across different countries depending on the load and traffic (marston, et al., 2011). cloud computing adoption has been studied using a number of theories including the theory, organization, environment framework (toe) (depietro, wiarda, & fleischer, 1990), the theory of reasoned action (tra) (fishbein & ajzen, 1975), and the diffusion of innovation theory (doi) (rogers, 2010). drawing from the tra framework, a study by benlian and hess (2011) suggested that an organisations’ intention to adopt cloud computing services was associated with the perceived opportunities and risks of using the technology. opportunities of cloud computing included cost advantages, it quality improvements, and strategic flexibility (benlian & hess, 2011). on the other hand, perceived risks of cloud computing related to security, performance, loss of strategic capabilities, and the potential hidden costs of the service (benlian & hess, 2011). hidden costs can arise at any stage in the outsourcing life cycle, i.e. during contracting, transition, and, managing the effort. these unexpected costs can diminish the benefit of outsourcing or, in some cases, may make it more costly to outsource (barthelemy, 2001). low, et al. (2011) and alshamaila, et al. (2013) adopted the toe framework to study cloud computing adoption and found that, among others, relative advantage, top management support, competitive pressures and innovativeness were factors that encouraged the use of cloud computing. wu, et al. (2013) combined the doi theory with the information processing theory (march & simon, 1958) and reported that business process complexity and information system compatibility were negatively associated with the intention to adopt cloud services. on the other hand, entrepreneurial culture and application functionality were positively associated with cloud adoption, which also confirmed alshamaila, et al. (2013). other studies used pest analysis (lee, et al., 2013) or australasian journal of information systems volume 18 number 3 2014 195 no particular theory (gupta, et al., 2013) to study cloud computing adoption. these studies confirmed benlian and hess (2011) but also found that legal issues around cloud computing and the lack of understanding inhibited adoption. transaction cost economics although, tce has been the most frequently applied theory in the it outsourcing (ito) literature (lacity, khan, & willcocks, 2009), only few papers have adopted it specifically to study cloud computing adoption (i.e. benlian, 2009; benlian, hess, & buxmann, 2009). tce highlights the role of transaction costs and asset specific investments in market transactions. transaction costs are associated with the time and effort to search, negotiate, contract, and maintain a relationship with vendors or customers (williamson, 1985). when transaction costs become too high, tce posits that organizations are better off to vertically integrate. one of the key issues in tce is the distrust on the exchange partner. trust is defined as the probability that the result of an outcome is positive for an individual/group even if no influence is exerted on the involved parties (gamson, 1968). the problem of opportunism arises because of asset specific investments and the notion of incomplete contracts. asset specific investments are investments by either of the parties that have little value outside the relationship. contracts are incomplete because it is too expensive, if at all possible to specify all the possible contingencies in the contract, especially under high levels of uncertainty. thus the concept of uncertainty is central to tce. williamson (1985) distinguishes between two types of uncertainties, namely primary and behavioural uncertainty. behavioural uncertainty refers to opportunism, which is the result of incomplete contracts and the lack of trust. primary uncertainty includes exogenous sources of uncertainty such as natural events, consumer preferences, technology, as well as regulations (sutcliffe & zaheer, 1998). uncertainty according to the ito literature is considered a deterrent to ito (aubert, rivard, & patry, 2004; nam, rajagopalan, rao, & chaudhury, 1996). applying the tce logic, benlian, et al. (2009) found that application adoption uncertainty in terms of reliability, pricing and, contracting was negatively associated with the attitude toward saas adoption. in this study, i focus on both, behavioural uncertainty (opportunism) and a particular aspect of primary uncertainty: the uncertainty around the legislation of cloud computing; a concern that has been raised in the literature (janssen & joha, 2011; lee, et al., 2013; marston, et al., 2011). i do not focus on asset specificity in this study for several reasons. asset specificity is not relevant for iaas as there are low switching costs associated with switching providers (clemons & chen, 2011). with regard to saas, asset specificity has relevance in so far as customized software is concerned and this relationship has already been established (benlian, et al., 2009). legal concerns regarding the cloud cloud computing presents unique challenges for its adopters from a legal point of view. litigation may be problematic, potentially spanning multiple national borders, with the client in one nation, the vendor in another, the data centre in a third, and the internet service provider spanning many others (clemons and chen, 2011). it is important that the legal system at the vendor’s location is compatible with that of the client so that it meets local regulatory requirements (marston et al., 2011). for example, the australian privacy act limits the disclosure of customer data information to third parties, which might be difficult to enforce in foreign jurisdictions (apa, 2014). clemons and chen (2011) discuss a number of issues that makes cloud computing problematic from a legal point of view. also, the contractual agreements governing the various parties from the cloud provider to subcontractors and to the cloud user may complicate matters (martin, 2010). for example, it is not clear who would be liable when data is lost or compromised, especially when cross-border jurisdiction are involved (jaeger et al., 2008; egwutuoha et al., 2013). reliance on the contract for managing risk is also problematic as it is unable to “completely cover and specify the complexity of an outsourcing project” owing to the it subject matter of the contract being “a very volatile, fast-changing asset” (leimeister et al. 2010, p. 6). in australasian journal of information systems volume 18 number 3 2014 196 addition, cloud providers could include clauses in its terms of use agreements granting the cloud provider a licence to use end-user data without charge or to insert jurisdiction and choice of law clauses (svantesson and clarke, 2010). also, not all cloud providers permit clients to specify the location of the data (clemons and chen, 2011). therefore, the cloud may lead an organisation from compliance to noncompliance without any notice (isaca, 2012). security and reliability of cloud the security and reliability of the cloud computing service is important as it failure is often associated with significant impact on shareholder value as demonstrated by cases such as sobeys inc, sydney water, and tjx companies (parent and reich, 2009). the relevant literature has expressed some concerns around cloud computing security (grobauer, walloschek, & stocker, 2011; gupta, et al., 2013; marston, et al., 2011) which is known to affect its adoption (benlian & hess, 2011; lee, et al., 2013). security refers to the confidentiality, integrity, and availability of a system (gordon & loeb, 2002). there are advantages and disadvantages of cloud computing from a security point of view. firstly, reputable and third party audited cloud computing providers such as google and amazon are likely to have better it security than most other organisations. large cloud computing vendors invest millions, if not billions of dollars into their it architecture in terms of software and hardware to gain the trust of their clients. cloud providers’ data centres are also more likely to be better protected than an ‘average’ organisation’s it infrastructure. furthermore, many large cloud vendors have data centres scattered around the world and mirror the data for additional backup (egwutuoha, et al., 2013; wu, 2011). on the other hand, because of virtualization, when a virtual machine (vm) is compromised, access to other vms on the same physical server is highly likely (bizarro & garcia, 2013). also, from a confidentiality point of view, clients may be concerned that cloud computing vendors may be required to disclose information to their own governments as in the case of the patriot act (zhou, zhang, xie, qian, & zhou, 2010). thus, organisations may fail to comply with the australian privacy act, which can result in large fines, as well as reputational drawbacks. concerns with respect to the reliability and bandwidth associated with the provision of applications was also raised in the literature (benlian, et al., 2009; smith & kumar, 2004). reliability of an information system refers to the availability of the system (kim, lee, & ham, 2013). availability is important because organizations have become increasingly dependent on their it infrastructure. a temporary disruption to an organizations’ it infrastructure will in most cases result in financial loss, as well as damage to reputation. also, because cloud computing relies on the internet, connectivity problems could lead to service disruptions. this could be the internet third party provider or the cloud service provider experiencing connectivity problems, rendering the service unavailable. for example, in 2009, microsoft azure, which is microsoft’s iaas and paas service offering, went down for 22 hours. this happened again in 2013, when the services were unavailable for at least 8 hours (parnell, 2013). also, a large scale outage hit amazon in 2011, affecting amazon’s web services' elastic compute cloud. the outage took out popular social networking services such as foursquare, formspring, heroku, and reddit (winteford, 2011). australia is especially vulnerable to power/internet outages because of severe weather conditions. the security of a system also affects its availability (cooper, 2006; hanmer, mcbride, & mendiratta, 2007; kim, et al., 2013). systems that are vulnerable to distributed denial of service (ddos) attacks or malicious software are unlikely to perform the required operations when compromised, which will therefore affect availability. for example, connectwise, network solutions, and endnote experienced ddos attacks in recent times that prevented access to the applications (gurnee, 2013; westervelt, 2013). australasian journal of information systems volume 18 number 3 2014 197 given the lack of cloud client’s understanding of the service (lee, et al., 2013), as well as the perceived uncertainty or concerns around the legislation of cloud computing (egwutuoha, et al., 2013; marston, et al., 2011), i posit the following hypothesis. h1: perceived legal uncertainty is positively associated with perceived cloud computing security risk. vendor opportunism in cloud opportunism has been studied for decades by economists concerned with the risks of contracting (clemons & chen, 2011). there are three types of risks associated with contracting: (i) shirking and deliberate under performance (alchian & demsetz, 1972; aron, clemons, & reddi, 2005; chen & bharadwaj, 2009), (ii) poaching and the theft of intellectual property (aron, et al., 2005; chen & bharadwaj, 2009; clemons & hitt, 2004), and (iii) opportunistic repricing, client lock-in, and vendor lock in (aron, et al., 2005; clemons & row, 1992; willcocks, lacity, & kern, 1999). all of these risk are relevant for the cloud computing service (clemons & chen, 2011). for example, a cloud vendor may shirk by deliberately under investing in server capability, creating slow-downs that can be blamed on the network (clemons & chen, 2011) there is also the view that cloud vendors might have an incentive to provide less processing power than specified in the sla (zhang, ye, shi, du, & guizani, 2013). the role of trust has been extensively studied in the adoption of e-commerce (e.g. gefen, karahanna, & straub, 2003; hart & saunders, 1997; roca, garcía, & de la vega, 2009; teo & liu, 2007) and is known to affect the success of ito (lee, huynh, & hirschheim, 2008). trust is a critical aspect in many economic transactions and is considered a complexity reduction strategy (luhmann, davis, raffan, & rooney, 1979). the literature on trust posits that trust supports information sharing and adoption of e-commerce technologies (hart & saunders, 1997; kim, tao, shin, & kim, 2010; suh & han, 2003). trusted vendors are not believed to act opportunistically even if contracts are considered incomplete. research found a significant relationship between trust and security in e-commerce (pavlou, 2003; teo & liu, 2007), mobile commerce (siau & shen, 2003), online trading systems (roca, et al., 2009), and in e-payment systems (kim, et al., 2010). also suh and han (2003) suggests that website trust is positively associated with security. therefore, a vendor that is likely to act opportunistically and not in the interest of the client, is likely to cause a security risk. trust or the absence of trust does not only affect the purchase intentions directly (gefen, 2000; gefen, et al., 2003) but also through reduced risk (jarvenpaa, tractinsky, & saarinen, 1999; kollock, 1999). this leads to the following two hypotheses: h2: perceived vendor opportunism is positively associated with perceived cloud computing security risk. h3: perceived vendor opportunism is negatively associated with the intention to increase the adoption of cloud computing services. as discussed previously, an important aspect of a system is its reliability and security irrespective of its features, look and feel. if the service is perceived to be unavailable or insecure, then users will be unable to perform their tasks. security and reliability is therefore a determinant of the intention to use an online service (greenberg, li, & wong-on-wing, 2012; gupta, et al., 2013; luarn & lin, 2005; salisbury, pearson, pearson, & miller, 2001). h4: perceived cloud computing security risk is negatively associated with the intention to increase cloud computing adoption. the proposed research model is shown below. in addition to the below paths, i control for size in terms of employees and turnover as size tends to affect technology adoption (kimberly & evanisko, 1981). australasian journal of information systems volume 18 number 3 2014 198 figure 1: research model method data for this study was collected using an online survey. the questionnaire had two parts. the first part was applicable to both cloud computing users and non-users. the second part of the questionnaire was designed to capture the type of cloud computing services used. all the measures for testing the hypotheses of the study consisted of at least three indicators. the questions were answered using a 1-7 point likert scale, where 1 represented “completely disagree” and 7 represented “completely agree”. the measure ‘intention to increase the level of cloud computing adoption’ was adopted from benlian and hess (2011). the measure ‘perceived security risk’ captured the availability, confidentiality and integrity of the system (gordon & loeb, 2002). this measure was modelled as a formative construct as it was considered to ‘form’ the construct and because the items were not expected to correlate highly (bollen, 1998). for example, confidentiality of data was not expected to correlate with the uptime of the service or the internet speed. all the other measures were modelled as reflective measures. the measures ‘perceived legal uncertainty’ and ‘perceived vendor trust’ had three indicators each and were based on previous studies (bahli & rivard, 2003; benlian & hess, 2011; featherman & pavlou, 2003). partial least squares analysis (pls) was used to test the hypothesized relationships using the smartpls software (ringle, wende, & will, 2005). pls is a structural equation modelling technique that has a number of advantages over covariance-based techniques such as lisrel and amos. pls has fewer restrictions on the data in terms of sample size and normality (chin, 1998). generally, a sample size that is equal or larger than 10 times the number of indicators for the scale with the largest number of formative constructs (chin & newsted, 1999) is considered adequate, although smaller sizes are also acceptable (nicolaou & masoner, 2013; tabachnick & fidell, 2001). the sample consisted of 1,170 australian firms. the survey targeted it decision makers. contact details were purchased from a local e-mail list broker. a cover letter with the link to the online survey was emailed to the respondents in april 2014. to encourage participation, a one hundred dollar gift voucher was offered as a prize in a draw. two weeks after the delivery of the survey, a reminder was sent to the participants. in total, 125 questionnaires were completed. due to missing data, 5 responses were deleted resulting in 120 usable responses. the sample size was adequate based on the established rule of thumb (chin and newsted, 1999). the data was checked for the possibility of a response bias. the results indicated that there was not a significant difference between the early and late responders. i also tested for common method bias, which is a potential problem when the same person provides all the answers. harman’s single-factor test was carried out by entering all the measures into a factor analysis. common method bias is present when a single factor emerges from the factor analysis or when the majority of australasian journal of information systems volume 18 number 3 2014 199 covariance is explained by one factor. the test revealed that this was not the case as four factors emerged that explained most of the variance (podsakoff and organ, 1986). results the majority of the respondents were senior it managers with more than 10 years of experience in their current positions. more than 78% of the respondents’ age fell between 41 and 60 years old, indicating that they were highly experienced. many industries were represented in the final sample as evident from the distribution in table 1. the majority of the companies had a turnover of over 100 million aud. also, most companies had employees above 100. demographic characteristics of respondents (n=120) count respondents position cio (cto) it manager it decision maker other 28 46 17 29 sector manufacturing retail financial and insurance services construction wholesale information, media, and telecommunications utilities other services (real estate, professional, food etc.) 20 13 11 11 10 9 7 39 turnover less than 1 m 1 – 10 m 11 – 50 m 51 – 100 m more than 100 m 4 9 28 17 62 number of employees less than 50 51-100 101-500 501-1000 more than 1,000 12 13 41 11 43 table 1: demographic characteristics of respondents mean values, standard deviation and cronbach’s alpha values are shown in table 2. higher values than 0.7 for cronbach’s alpha confirmed that the scales were reliable (fornell and larcker, 1981). scale reliability is only applicable to reflective measures and therefore was not relevant to the measure ‘perceived security risks’, which was modelled as a formative construct (diamantopoulos and winklhofer, 2001). mean values for items measuring the intention to increase the level of cloud computing adoption were fairly high in relation to other measures suggesting that organizations viewed cloud computing services as a viable alternative to in-house it capabilities. furthermore, relative high mean scores on the legal measures indicated that the uncertainty around the legal issues of using cloud computing was a concern for some. internet reliability had the largest standard deviation indicating that internet speed or connectivity varied significantly from region to region. based on the qualitative australasian journal of information systems volume 18 number 3 2014 200 feedback, some of the respondents mentioned internet connectivity as the main impediment to the adoption of cloud services. also, one respondent mentioned that job security was a main concern if the organisation migrated to the cloud. measure item mean 1 -7 cronbach’s alpha intention to increase the level of cloud computing adoption if there are superior offers, cloud computing providers should be used for computing resources our company should increase the use of cloud computing services i support the further adoption of cloud computing 4.97 4.82 5.27 1.48 1.57 1.48 0.929 perceived vendor opportunism we trust cloud computing providers* cloud computing providers are unlikely to act opportunistically* cloud computing providers will exploit contractual loopholes to the detriment of your company it contracts with cloud providers are risky 4.39 3.83 4.04 4.05 1.23 1.43 1.22 1.34 0.855 perceived legal uncertainty in case of damage, present liability law is unclear about who will bear the damage the legal framework for the cloud in australia does not sufficiently safeguard the consumer we are unsure of how well we would be protected in the rule of law from cloud computing failure 4.63 4.69 4.93 1.05 1.13 1.27 0.854 perceived security risks the confidentiality and security of your business data are not guaranteed when adopting cloud computing solutions cloud computing is more vulnerable to information system attacks our company’s it resources are more secure when compared to the cloud 4.41 3.73 4.14 1.70 1.45 1.64 na cloud computing services aren’t as reliable as in house computing resources cloud computing services are unlikely to have the same uptime as local computing resources the internet is not as reliable or fast enough to support our computing needs 3.46 3.32 4.09 1.48 1.51 1.83 1 – highly disagree, 7 – highly agree na: not applicable *scale reverse coded table 2: measures and mean values australasian journal of information systems volume 18 number 3 2014 201 i also asked adopters of cloud computing the extent of their satisfaction with the services so far. the adopters indicated that cloud services met their expectations (mean: 5.34) and that they were satisfied as evident from the high mean value (mean: 5.32). cloud computing in one form or another was adopted by 85 organizations, compromising about 71% of the sample. the adoption rate was significantly higher than the german study of benlian and hess (2011), which was 40% for saas. furthermore, gupta, et al. (2013) reported in a predominantly asia pacific survey that only about a quarter of its respondents used iaas or saas. only three out of the 85 users used exclusively e-mail such as g-mail, suggesting that many adopters used a mixture of cloud services. figure 2 shows the types of applications used by respondents. figure 2: types of cloud applications used other applications included payroll, file share, various testing tools, e-mail filtering, travel management, content management, and human resource management systems. out of the 85 cloud computing (saas) adopters, 24 (28%) indicated that they also used iaas, whereas there were a total of 22 (26%) paas adopters. furthermore, 54 (64%) firms indicated that their data was stored in australia, although not exclusively. on the other hand 51 (60%) companies indicated that their data was stored abroad and only 12 (14%) had their data on premise. figure 3 shows the adoption intentions of non-adopters. the majority had no intention of adopting cloud computing services at the time of the survey. australasian journal of information systems volume 18 number 3 2014 202 figure 3: non-adopters’ adoption intention discriminant validity of the constructs was assessed by comparing the square root of variance (ave) extracted with inter-construct correlations. as per table 3, the square root of ave values in diagonal were larger than correlations and therefore discriminate validity was met (chin, 1998). there were weak to strong correlations, which indicated that at least some of the hypotheses were likely to be supported. perc. leg. perc. oppor perc. security intention perc. leg. 0.918 perc. oppor -0.131 0.840 perc. security 0.689 0.124 na intention -0.225 -0.364 -0.489 0.938 table 3: construct correlations and square root of ave convergent validity was satisfactory as all the indicator loadings were higher than 0.7 (see appendix 2) for reflective constructs (fornell and larcker, 1981). also, according to the pls analysis, all the hypothesized relationships were significant and the signs were as expected. r-squared for perceived security risk was 0.522, explaining more than half of the variance. perceived security risk and perceived vendor opportunism explained more than 30% of the variance in the intention to adopt cloud services. this was considered high as the model did not include many of the perceived benefits of cloud computing that drive its adoption (benlian and hess, 2011; lee et al., 2013; gupta et al., 2013). i also found a direct relation between perceived vendor opportunism and the intention to adopt cloud computing that was highly significant. therefore, all the hypotheses in the study were supported. in terms of the control variables, there was no significant relationship between the size of the firms and the intention to adopt cloud computing services. australasian journal of information systems volume 18 number 3 2014 203 figure 4: path coefficients, t-statistics, and r-squared discussion and conclusions this study applied tce logic to the cloud computing adoption decision of organisations, a theoretical framework that has received little attention from the current cloud computing adoption literature. tce highlights the adverse effects of uncertainty on transaction costs and differentiates between primary and behavioural uncertainty. in this study, i focused on a particular aspect of primary uncertainty, the perceived uncertainty around cloud computing legislation. users are likely to have concerns with cloud computing legislation because liabilities may not be as clearly defined as in traditional outsourcing relationships. this is due to the nature of cloud, which involves many parties and foreign legislation (egwutuoha, et al., 2013; jaeger, et al., 2008; marston, et al., 2011). the results of this study showed that the perceived legislative uncertainty around cloud computing was strongly associated with the perceived security risk of the cloud, confirming hypothesis 1. also, behavioural uncertainty in terms of perceived vendor opportunism was positively related to perceived security risk, although this relationship was not as significant as the direct relation to adoption. the highly significant and negative relation between perceived vendor opportunism and the intention to adopt cloud computing was consistent with previous studies (gefen, 2000; gefen, et al., 2003). therefore, hypothesis 2 and 3 were accepted, confirming a number of studies (gefen, 2000; gefen, et al., 2003; jarvenpaa, et al., 1999; knoll & jarvenpaa, 1994). there was also a significant negative relationship between perceived security risk and the intention to adopt cloud computing services as suggested in the literature (greenberg, et al., 2012; gupta, et al., 2013). therefore hypotheses 4 was also supported. this study makes a number of contributions. first, the results indicate that tce logic is a powerful theoretical framework to explain an organisations intention to adopt cloud computing services just as other theoretical frameworks such as toe, tra, and doi. the results also overall agree with the earlier ito literature on the adverse effects of uncertainty on ito (aubert, beaurivage, croteau, & rivard, 2008; nam, et al., 1996). this study might also be of value to practitioners who are considering the adoption of cloud computing services. the results are likely to be generalizable to other geographical regions as the impact of cultural factors was considered to be low. there are some implications of this study. firstly, potential cloud computing adopters might be discouraged from adopting cloud services because of the associated security risk of cloud computing. while the legislation around cloud computing is still evolving, organizations might consider it more ‘safe’ to adopt services with local providers and data-centres that are subject to only local laws. organisations may also want to consider hybrid solutions whereby only non-critical applications/data are outsourced (goscinski & brock, 2010; marston, et al., 2011). australasian journal of information systems volume 18 number 3 2014 204 contract negotiation and the sla is vital and adequate diligence must be exercised to secure a favourable outcome. clemons and chen (2011) discuss the issues specifically around cloud computing contracts. also, technology is improving that can automatically monitor compliance with slas (zhang, et al., 2013) and check the integrity of the data on the cloud (wang, chow, wang, ren, & lou, 2013). furthermore, cloud vendors need to engage in trust building activities. it is commendable that some cloud service providers undergo third party assurance audits. also, standards on cloud computing are being developed, although most of this effort does not address the problem of standardization and the vendor lock in problem (clemons & chen, 2011). there are some limitations of the study. the response rate, although acceptable, was rather low. hence, the results need to be interpreted with some caution despite the absence of a non-response bias. also, it would be fruitful to use other theoretical frameworks and methods to study cloud computing adoption. for example, the institutional theory (dimaggio & powell, 1983) might present additional insight into other factors that affect adoption such as the role of fashion and fad (abrahamson, 1991). future research would also benefit from case studies or longitudinal studies that report on specific experiences of cloud computing migration/adoption. references abrahamson, e. (1991). managerial fads and fashions: the diffusion and rejection of innovations. academy of management review, 586-612. alchian, a. a., & demsetz, h. (1972). production, information costs, and economic organization. the american economic review, 777-795. alshamaila, y., papagiannidis, s., & li, f. (2013). cloud computing adoption by smes in the north east of england: a multi-perspective framework. journal of enterprise information management, 26(3), 250-275. apa. (2014). australian privacy act. retrieved from office of the australian information commissioner, http://www.oaic.gov.au/privacy/privacy-resources/privacy-factsheets/other/privacy-fact-sheet-17-australian-privacy-principles aron, r., clemons, e. k., & reddi, s. (2005). just right outsourcing: understanding and managing risk. journal of management information systems, 22(2), 37-55. aubert, b. a., beaurivage, g., croteau, a. m., & rivard, s. (2008). firm strategic profile and it outsourcing. information systems frontiers, 10(2), 129-143. aubert, b. a., rivard, s., & patry, m. (2004). a transaction cost model of it outsourcing. information & management, 41(7), 921-932. australian. (2014). smes still scared of the cloud: acma. retrieved from http://www.theaustralian.com.au/business/latest/eighty-per-cent-of-aussies-using-the-cloudacma/story-e6frg90f-1226865028456 bahli, b., & rivard, s. (2003). the information technology outsourcing risk: a transaction cost and agency theory based perspective. journal of information technology, 18(3), 211-221. barthelemy, j. (2001). the hidden costs of it outsourcing. mit sloan management review, 42(3), 60+. retrieved from <go to isi>://000168173400012. benlian, a. (2009). a transaction cost theoretical analysis of software-as-a-service (saas)-based sourcing in smbs and enterprises. benlian, a., & hess, t. (2011). opportunities and risks of software-as-a-service: findings from a survey of it executives. decision support systems, 52(1), 232-246. benlian, a., hess, t., & buxmann, p. (2009). drivers of saas-adoption–an empirical study of different application types. business & information systems engineering, 1(5), 357-369. australasian journal of information systems volume 18 number 3 2014 205 bizarro, p. a., & garcia, a. (2013). virtualization: benefits, risks, and control. internal auditing, 28(4), 11-18. bollen, k. a. (1998). structural equation models: wiley online library. chen, y., & bharadwaj, a. (2009). an empirical analysis of contract structures in it outsourcing. information systems research, 20(4), 484-506. chin, w. w. (1998). the partial least squares approach for structural equation modeling. chin, w. w., & newsted, p. r. (1999). structural equation modeling analysis with small samples using partial least squares. statistical strategies for small sample research, 1(1), 307-341. clemons, e. k., & chen, y. (2011). making the decision to contract for cloud services: managing the risk of an extreme form of it outsourcing. in system sciences (hicss), 2011 44th hawaii international conference on (pp. 1-10): ieee. clemons, e. k., & hitt, l. m. (2004). poaching and the misappropriation of information: transaction risks of information exchange. journal of management information systems, 21(2), 87-107. clemons, e. k., & row, m. c. (1992). information technology and industrial cooperation: the changing economics of coordination and ownership. journal of management information systems, 9(2), 9-28. cooper, c. (2006). the true cost of outsourcing. charter, 77, 20-22. depietro, r., wiarda, e., & fleischer, m. (1990). the context for change: organization, technology and environment. the processes of technological innovation, 151-175. dimaggio, p. j., & powell, w. w. (1983). the iron cage revisited: institutional isomorphism and collective rationality in organizational fields. american sociological review, 147-160. egwutuoha, i. p., schragl, d., & calvo, r. (2013). a brief review of cloud computing, challenges and potential solutions. parallel & cloud computing, 2(1). featherman, m. s., & pavlou, p. a. (2003). predicting e-services adoption: a perceived risk facets perspective. international journal of human-computer studies, 59(4), 451-474. fishbein, m., & ajzen, i. (1975). belief, attitude, intention and behavior: an introduction to theory and research. fornell, c., & larcker, d. f. (1981). structural equation models with unobservable variables and measurement error: algebra and statistics. journal of marketing research, 382-388. gamson, w. a. (1968). power and discontent (vol. 124): dorsey press homewood, il. gefen, d. (2000). e-commerce: the role of familiarity and trust. omega, 28(6), 725-737. gefen, d., karahanna, e., & straub, d. w. (2003). trust and tam in online shopping: an integrated model. mis quarterly, 27(1), 51-90. gordon, l. a., & loeb, m. p. (2002). the economics of information security investment. acm transactions on information and system security (tissec), 5(4), 438-457. goscinski, a., & brock, m. (2010). toward dynamic and attribute based publication, discovery and selection for cloud computing. future generation computer systems, 26(7), 947-970. greenberg, r., li, w., & wong-on-wing, b. (2012). the effect of trust in system reliability on the intention to adopt online accounting systems. international journal of accounting and information management, 20(4), 363-376. grobauer, b., walloschek, t., & stocker, e. (2011). understanding cloud computing vulnerabilities. security & privacy, ieee, 9(2), 50-57. retrieved from http://ieeexplore.ieee.org/ielx5/8013/5739630/05487489.pdf?tp=&arnumber=5487489&isnum ber=5739630. australasian journal of information systems volume 18 number 3 2014 206 gupta, p., seetharaman, a., & raj, j. r. (2013). the usage and adoption of cloud computing by small and medium businesses. international journal of information management, 33(5), 861-874. gurnee, f. (2013). [update] “was not an outage” connectwise outage denial of service attack to blame? . retrieved from http://www.lookscloudy.com/2013/03/connectwise-outage-denial-ofservice-attack-to-blame/ hanmer, r. s., mcbride, d. t., & mendiratta, v. b. (2007). comparing reliability and security: concepts, requirements, and techniques. bell labs technical journal, 12(3), 65-78. hart, p., & saunders, c. (1997). power and trust: critical factors in the adoption and use of electronic data interchange. organization science, 8(1), 23-42. jaeger, p. t., lin, j., & grimes, j. m. (2008). cloud computing and information policy: computing in a policy cloud? journal of information technology & politics, 5(3), 269-283. janssen, m., & joha, a. (2011). challenges for adopting cloud-based software as a service (saas) in the public sector. jarvenpaa, s. l., tractinsky, n., & saarinen, l. (1999). consumer trust in an internet store: a cross cultural validation. journal of computer mediated communication, 5(2), 0-0. kim, c., tao, w., shin, n., & kim, k.-s. (2010). an empirical study of customers’ perceptions of security and trust in e-payment systems. electronic commerce research and applications, 9(1), 84-95. kim, h.-b., lee, d.-s., & ham, s. (2013). impact of hotel information security on system reliability. international journal of hospitality management, 35, 369-379. kimberly, j. r., & evanisko, m. j. (1981). organizational innovation: the influence of individual, organizational, and contextual factors on hospital adoption of technological and administrative innovations. academy of management journal, 24(4), 689-713. knoll, k., & jarvenpaa, s. l. (1994). information technology alignment or “fit” in highly turbulent environments: the concept of flexibility. in proceedings of the 1994 computer personnel research conference on reinventing is: managing information technology in changing organizations: managing information technology in changing organizations (pp. 1-14): acm. kollock, p. (1999). the production of trust in online markets. advances in group processes, 16, 99123. lacity, m. c., khan, s. a., & willcocks, l. p. (2009). a review of the it outsourcing literature: insights for practice. journal of strategic information systems, 18(3), 130-146. retrieved from <go to isi>://000271333200003. doi:doi 10.1016/j.jsis.2009.06.002 lee, j.-n., huynh, m. q., & hirschheim, r. (2008). an integrative model of trust on it outsourcing: examining a bilateral perspective. information systems frontiers, 10(2), 145-163. lee, s.-g., chae, s. h., & cho, k. m. (2013). drivers and inhibitors of saas adoption in korea. international journal of information management, 33(3), 429-440. low, c., chen, y., & wu, m. (2011). understanding the determinants of cloud computing adoption. industrial management & data systems, 111(7), 1006-1023. luarn, p., & lin, h.-h. (2005). toward an understanding of the behavioral intention to use mobile banking. computers in human behavior, 21(6), 873-891. luhmann, n., davis, h., raffan, j., & rooney, k. (1979). trust; and, power: two works by niklas luhmann: wiley chichester. march, j. g., & simon, h. a. (1958). organizations. australasian journal of information systems volume 18 number 3 2014 207 marston, s., li, z., bandyopadhyay, s., zhang, j., & ghalsasi, a. (2011). cloud computing the business perspective. decision support systems, 51(1), 176-189. retrieved from <go to isi>://000288519200017. doi:doi 10.1016/j.dss.2010.12.006 nam, k., rajagopalan, s., rao, h. r., & chaudhury, a. (1996). a two-level investigation of information systems outsourcing. communications of the acm, 39(7), 36-44. nicolaou, a. i., & masoner, m. m. (2013). sample size requirements in structural equation models under standard conditions. international journal of accounting information systems. parnell, b. a. (2013). microsoft's azure cloud down and out for 8 hours. retrieved from http://www.theregister.co.uk/2012/02/29/windows_azure_outage/ pavlou, p. a. (2003). consumer acceptance of electronic commerce: integrating trust and risk with the technology acceptance model. international journal of electronic commerce, 7(3), 101-134. podsakoff, p. m., & organ, d. w. (1986). self-reports in organizational research: problems and prospects. journal of management, 12(4), 531-544. ringle, c. m., wende, s., & will, s. (2005). smartpls 2.0 m3 beta. hamburg, 2005. roca, j. c., garcía, j. j., & de la vega, j. j. (2009). the importance of perceived trust, security and privacy in online trading systems. information management & computer security, 17(2), 96113. rogers, e. m. (2010). diffusion of innovations: simon and schuster. ross, p., & blumenstein, m. (2013). cloud computing: the nexus of strategy and technology. journal of business strategy, 34(4), 39-47. salisbury, w. d., pearson, r. a., pearson, a. w., & miller, d. w. (2001). perceived security and world wide web purchase intention. industrial management & data systems, 101(4), 165-177. siau, k., & shen, z. (2003). building customer trust in mobile commerce. communications of the acm, 46(4), 91-94. smith, m. a., & kumar, r. l. (2004). a theory of application service provider (asp) use from a client perspective. information & management, 41(8), 977-1002. suh, b., & han, i. (2003). the impact of customer trust and perception of security control on the acceptance of electronic commerce. international journal of electronic commerce, 7, 135-161. sutcliffe, k. m., & zaheer, a. (1998). uncertainty in the transaction environment: an empirical test. strategic management journal, 19, 1-23. svantesson, d., & clarke, r. (2010). privacy and consumer risks in cloud computing. computer law & security review, 26(4), 391-397. tabachnick, b. g., & fidell, l. s. (2001). using multivariate statistics. teo, t. s., & liu, j. (2007). consumer trust in e-commerce in the united states, singapore and china. omega, 35(1), 22-38. wang, c., chow, s. s., wang, q., ren, k., & lou, w. (2013). privacy-preserving public auditing for secure cloud storage. computers, ieee transactions on, 62(2), 362-375. westervelt, r. (2013). ddos attack behind latest network solutions outage. retrieved from http://www.crn.com/news/security/240158492/ddos-attack-behind-latest-network-solutionsoutage.htm willcocks, l. p., lacity, m. c., & kern, t. (1999). risk mitigation in it outsourcing strategy revisited: longitudinal case research at lisa. the journal of strategic information systems, 8(3), 285314. williamson, o. (1985). the economic institutions of capitalism. journal of economic. australasian journal of information systems volume 18 number 3 2014 208 winteford, b. (2011). growing pains: amazon ec2 suffers huge outage. itnews. retrieved from http://www.itnews.com.au/news/255549,growing-pains-amazon-ec2-suffers-hugeoutage.aspx/1 wu, w.-w. (2011). mining significant factors affecting the adoption of saas using the rough set approach. journal of systems and software, 84(3), 435-441. wu, y., cegielski, c. g., hazen, b. t., & hall, d. j. (2013). cloud computing in support of supply chain information system infrastructure: understanding when to go to the cloud. journal of supply chain management, 49(3), 25-41. youseff, l., butrico, m., & da silva, d. (2008). toward a unified ontology of cloud computing. gce: 2008 grid computing environments workshop, 42-51. retrieved from <go to isi>://000265406000006. zhang, h., ye, l., shi, j., du, x., & guizani, m. (2013). verifying cloud service level agreement by a third party auditor. security and communication networks. zhou, m., zhang, r., xie, w., qian, w., & zhou, a. (2010). security and privacy in cloud computing: a survey. in semantics knowledge and grid (skg), 2010 sixth international conference on (pp. 105-112): ieee. australasian journal of information systems volume 18 number 3 2014 209 appendix 1: list of abbreviations ave: average variance extracted crm: customer relationship management ddos: distributed denial of service doi: diffusion of innovation theory erp: enterprise resource planning iaas: infrastructure as a service ito: information technology outsourcing na: not applicable paas: platform as a service pls: partial least squares analysis saas: software as a service sla: service level agreement tce: transaction cost economics toe: theory, organization, environment framework tra: theory of reasoned action vm: virtual machine australasian journal of information systems volume 18 number 3 2014 210 appendix 2: loadings (weights) and composite reliability indicator loadings (weights) composite reliability* perc. leg. 1 0.905124 0.942007 perc. leg. 2 0.937834 perc. leg. 3 0.913027 perc. oppor. 1 0.934447 0.905261 perc. oppor. 2 0.903513 perc. oppor. 3 0.789799 perc. oppor. 4 0.717839 perc. security 1 (0.514163) na perc. security 2 (-0.018728) perc. security 3 (0.241093) perc. security 4 (0.319523) perc. security 5 (-0.176165) perc. security 6 (0.352393) intention 1 0.935844 0.958477 intention 2 0.944483 intention 3 0.941858 na: not applicable to formative constructs *threshold for composite reliability: 0.7 paper title (paper title style) australasian journal of information systems win & beydoun 2020, vol 24, selected papers from acis 2018 preface 1 towards the next generation of information systems: enhancing traceability and transparency khin than win school of computing and it, university of wollongong, wollongong win@uow.edu.au ghassan beydoun school of information, systems and modelling, university of technology sydney, australia abstract the information systems community plays a significant role in providing theories for guiding the creation of more effective information systems. this is nowadays most prominent in supporting the ongoing evolution of artificial intelligence (ai) and decision-making systems. as these systems are becoming increasingly reliable, and supplanting the presence of humans, the question of accountability and trust in these systems is increasingly coming to the fore. this editorial positions and showcases a collection of papers from the 2018 australasian conference on information systems within this context. keywords: traceability, decision making, transparency. 1 introduction information systems play a critical role in decision making processes for individuals and organisations. individuals and organisations have adopted different tools to process and analyse data to assist in gaining useful information and knowledge. with the extreme growth of ai, deployment of intelligent systems within organisations seeking further automation is increasingly being pursued. however, an emerging barrier to the acceptance of intelligent information systems is their current inability to adequately explain their decisions. to equip such systems with the ability to explain and justify, it is clear that understanding the nature of explanations and what constitutes good human justifications and communication in a given context is critical. from this vantage point of providing explainability to enhance trust, it is becoming increasingly noted that social science is required to support the continuing evolution of intelligent systems (miller 2019). the information systems community is best positioned to bridge social science to ai and it. hence, it is important that the is community is further sensitized to this role/responsibility moving forward with ai and decision support systems. in this short editorial paper, we first review the content of the section against the increasing importance of enhancing transparency and traceability of decision making in modern information systems. we further reflect on the role of the is community in that endeavour and conclude with further remarks on what items an agenda for the next wave of is efforts in this space could include. australasian journal of information systems win & beydoun 2020, vol 24, selected papers from acis 2018 preface 2 2 reviewing the content the contribution of is to enhance decision transparency and traceability is expressed in different forms within the community. many is researchers are enhancing analytics tools and potentials by enhancing the role of visualization of data. this in turn translates further visibility into intelligent systems processes to gain insights into how they arrived at their decision. a good number of is researchers are also concerned with modelling decision making in various applications areas, including the obvious area of information systems development itself. this collection of papers provides excellent exemplars of the breadth of information systems that can potentially bear on the challenge of enhancing trust and transparency of intelligent systems. four selected papers from acis (2018) presented information systems strategy in assisting decision-making. the best paper nominees from acis2018 were invited to expand their papers to the specifications of this journal and were accepted after two rounds of peer review and revisions. we thank all reviewers for their time spent on reviewing papers and providing feedback. the studies presented range from food safety and digital gaming to enterprise social networking adopting different information system theories. the paper by lam et al. (2020) presents the importance of information transparency in decision making related to food products. this work provides a compelling argument on how traceability can enhance trust in service delivery in the domain of food delivery. this paper concurs with the arguments presented in this preface and presents a good collection of is related efforts to support the notion that traceability is essential to ensure accountability and trust. the results of the paper from a quantitative study confirms this position in a food supply chain example. the paper by reimer et al. (2020) provides results of activities in enterprise social networks. they uncover hidden interaction patterns within the networks. by making these hidden patterns visible to management, they recommend their use to enable management to assess how the intent of the information system (i.e. the network) aligns with the outcome. this paper provides another powerful example of how uncovering hidden patterns and enhancing traceability of outcomes are critical to trust in the system. the paper by kautz et al. (2020) brings to the fore the importance of having well understood interactions in a distributed development environment. in other words, traceability of the development processes themselves is important to evaluate the success of the process. they present their studies in information system development in crowd sourcing using complex adaptive system theory. the study utilised distributed participatory design in a digital game development. the paper by faber et al. (2020) tackles the challenges of visualizing business ecosystems to enable visibility and transparency of the underlying complex processes. the paper reports the results of two recent case studies by adopting an action design research approach. again, this work addresses the key theme of this special issue in highlighting how traceability of the underlying processes in an information system is critical to trust its outcome. 3 concluding remarks this special section provides a forum for is academics and practitioners to identify and share the challenges, opportunities, and solutions that improve the transparency and traceability in australasian journal of information systems win & beydoun 2020, vol 24, selected papers from acis 2018 preface 3 information systems. a particular focus is on the viewpoint of increased insights into data in complex settings. we seek to provide a forum to disseminate work aiming to enhance trust in systems from the perspective that accountability cannot be compromised as our reliance on information systems increases. this special section aims to enhance the awareness of the role of is in that endeavour. is perspectives include enhanced visualisations of data patterns, additional insights into the underlying processes and their interplay with the users as well as the development process of it artefacts involved. future efforts of the is community along these three dimensions will continue to enhance the bridge between research undertaken in the social context of the system and research addressing their it technological challenges. references faber a., hernandez-mendez a., rehm s-v, matthes f. (2020), collaborative modelling and visualization of business ecosystems: insights from two action design research case studies, australasian journal of information systems 24, https://doi.org/10.3127/ajis.v24i0.2229 kautz k., bjerknes g., fisher j., jensen t. (2020), applying complex adaptive systems theory to understand distributed participatory design in contemporary, crowdsourced information systems development, australasian journal of information systems 24 https://doi.org/10.3127/ajis.v24i0.2225 lam t. k., heales j., hartley n., hodkinson c. (2020), consumer trust in food safety requires information transparency, australasian journal of information systems 24, https://doi.org/10.3127/ajis.v24i0.2219 miller, (2019), explanation in artificial intelligence: insights from the social sciences, artificial intelligence 267, 1-38 riemer k., lee l.l., kjaer c., haeffner a. (2020), identification of enterprise social network (esn) group archetypes in esn analytics – metrics selection and case application, australasian journal of information systems 24, https://doi.org/10.3127/ajis.v24i0.2355 copyright: © 2020 win & beydoun. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v24i0.2823 http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems win & beydoun 2020, vol 24, selected papers from acis 2018 preface 4 paper title (paper title style) australasian journal of information systems nallaperuma & de silva 2017, vol 21, research on health information systems participatory model for multi-document summarisation 1 a participatory model for multi-document health information summarisation dinithi nallaperuma la trobe university dinithi.mjayaratne@gmail.com daswin de silva la trobe university abstract increasing availability and access to health information has been a paradigm shift in healthcare provision as it empowers both patients and practitioners alike. besides awareness, significant time savings and process efficiencies can be achieved through effective summarisation of healthcare information. relevance and accuracy are key concerns when generating summaries for such documents. despite advances in automated summarisation approaches, the role of participation has not been explored. in this paper, we propose a new model for multidocument health information summarisation that takes into account the role of participation. the updated is user participation theory was extended to explicate these roles. the proposed model integrates both extractive and abstractive summarisation processes with continuous participatory inputs to each phase. the model was implemented as a client-server application and evaluated by both domain experts and health information consumers. results from the evaluation phase indicates the model is successful in generating relevant and accurate summaries for diverse audiences. keywords: summarization; multi-document; user participation; health informatics; mead, ontology, templates 1 introduction retrieval of relevant and reliable health information is becoming a formidable challenge with increasing volumes of resources available for perusal via the internet. this is significantly experienced in the healthcare domain as it is crucial for practitioners as well as researchers to have quick access to up-to-date information (afantenos et al., 2005). besides providers, healthcare consumers are also inundated with large volumes of information. their limited knowledge of relevance due to lack of contextual and discourse awareness entails a further degree of complexity. (moen, et al., 2016) (pivovarov and elhadad, 2015). for instance, health forums consist of many topics that would frequently repeat the same question with minor changes followed by a multitude of responses that reflect differences of opinion. in such cases, the automated generation of a summary of multiple documents relevant to a selected topic can contribute towards addressing information needs of consumers as well as healthcare providers and researchers. the information retrieval needs of health information consumers have been appropriately addressed by a substantial body of research literature on smart health information portals. the delivery of user-sensitive, relevant, timely and accurate health information to various user groups was the focus of these endeavours (burstein et al., 2006). notable research outcomes include, resource description quality criteria modelling (mckemmish et al., 2009), automated quality assessment (xie and burstein 2011) and decision support systems perspective on portals (burstein et al 2005). the need for a multi-document information summarisation approach for health information consumer has been indicated in these research endeavours. the main goal of a summary is to extract the important content in document(s) and present them in a short space (afanteno et al., 2005). aligned with radev et al., (2002) definition of a summary, afanteno et al. (2005) demonstrated that a document contains information bursts and an automated summary should distinguish these bursts of information from the rest of the australasian journal of information systems nallaperuma & de silva 2017, vol 21, research on health information systems participatory model for multi-document summarisation 2 document. thereby, identifying relevant information bursts is the key challenge in document summarisation. we posit that end-user participation is an important consideration to identifying such relevant information bursts in health document summarisation and has been consistently overlooked by all existing approaches. use of participatory design methods where ‘end users’ take part in service development processes is currently widespread in the health sector. with the complexity embedded in health information, specifically in health forums, system developers have to identify and manage social conflicts and technical issues (pilemalm & timpka, 2008). thursky and mahemoff (2007) and weng, et al. (2007) have shown promising results by using user participation in different sites such as hospital and clinical. it is mainly in response to the diversity of the problems encountered in the design processes that user participatory design principles have been used in health information system development. kappelman and mclean (1992) defined participation as the observable behaviour of system users such as involvement in development and implementation activities of information systems. markus and mao (2004) extended the traditional concept of system success into development success and implementation success, alongside explicit articulation of participants in each stage. zowghi et al. (2015) conducted a systematic review of the relationship between user involvement and system success to reveal a positive correlation between the two, alongside factors for effective management of user involvement. eichhorn & tukel (2015) present a further review of user involvement, taking into account multiple factors such as user roles and activities, selection of users, type of communications used, and timing and level of their involvement. both these reviews emphasise on the importance of user participation to effectuate relevance in design, development and implementation success. in this paper, we introduce a participatory model for automated summarisation of multidocument health information. the paper delineates the design, development and evaluation of the proposed model. the paper is organised as follows. section 2 reviews literature on document summarisation and user participation in information system design. section 3 presents overall design of the model and further discusses the participation model in order to validate the proposed participatory model for multi-document summarisation. section 4 discusses implementation; the three tier architecture of the summarisation model and the multi-document summarisation process. section 5 reports on both qualitative and quantitative evaluation. finally, concluding remarks are presented with a brief discussion on limitations and future work. 2 document summarisation jones (2007) conducted a comprehensive survey on document summarisation using several frameworks that evaluate key factors affecting summaries. he also evaluated statistical and symbolic methods that do not require model instantiation such as in-domain ontologies. saggion and poibeau (2012) have reported that extractive summarisation on multi-documents is challenging as it would lead to redundancies as well as inconsistent and/or hard to read summaries. lloret and palomar (2012) proposed sentiment based summaries where high degree of subjectivity is applicable and update summaries where only the most recent information on the topic would be considered for summarisation. aliguliyev (2009) reviewed salient notions and developments in summarisation such as partitioning-based clustering methods in generic document summarisation. he has shown that summarisation depends on optimising functions and on similarity measures which would be more useful in improving performance compared to traditional summarisation approaches. the primary methods of summarisation are extractive and abstractive (wan, 2008) (mani and maybury, 1999). these two methods are discussed in detail in the following subsections. 2.1 extractive summarisation most existing systems for summarisation use sentence extraction from the source. the advantage of extraction is that it does not require automated natural language generation. australasian journal of information systems nallaperuma & de silva 2017, vol 21, research on health information systems participatory model for multi-document summarisation 3 therefore, domain independence summaries are easy to implement as extraction does not require the vocabulary of the particular domain. however, poor linguistic coherence of the extracted summaries is the main disadvantage of the extraction based systems. a collection of sentences from one or more documents will be input to the extraction based summarisation process and the output would comprise of a subset of these sentences (radev et al., 2004) (salton et al., 1997). length of the required summary is taken into account when generating the subset. as shown by radev et al. (2004), ranking approaches based on similarity and centrality are used to rank sentences for inclusion in the summary. wan et al. (2007), have proposed a novel approach based on manifold ranking of sentences for query based multidocument summarisation. in their approach, firstly, the manifold ranking score for each sentence will be computed where biased information richness of the sentence is used. this is followed by greedy algorithm to penalize the sentences with the highest overall scores, which will indicate the sentence is more informative and is highly biased to the given query. mead is widely used open source summarisation method, it is an implementation of the centroidbased method for either single or multi-document summarizing (erkan and radev, 2004) (radev et al., 2004). 2.2 abstractive summarisation in direct contrast to extractive summarisation, abstractive summarisation emphasises on semantic structure of information content based on sentence compression and sentence reformulation (radev et al. 2002). summons (mckeown and radev, 1995) attempts abstractive summarisation using a set of heuristic rules that perform operations such as contradiction, change of perspective, refinement, etc. some of these operations require resolving conflicts such as information within different opinions. although summons generates accurate summaries when the domain is quite simple, a generalized system for more complex domains is problematic. mckeown et al. (1999) and barzilay et al. (1999) have reported an improvement where the input is raw text such as the text that is retrieved by a standard search engine in response to a query. the concept has been adapted by belkebir and guessoum (2016) in order to derive a more general representation where different concepts appear in the same context. 2.3 multi-document summarisation although summarising multiple documents could lead to overlap, the value generated by bursts of supplementing information is significant. the primary issues in multi-document summarisation are to identify and cope with redundancy across documents, recognize novelty and ensure that the final summary is coherent and complete. existing research on multidocument summarisation attempts to address these issues. making use of similarity measures between sets of sentences, different approaches such as clustering were used on implement extractive techniques. one approach identified common themes through clustering and then select one sentence to represent each cluster (mckeown et al., 1999; radev et al., 2000); another generates a composite sentence from each cluster (barzilay et al., 1999), while some approaches work dynamically by including each candidate passage only if it is considered novel with respect to the previous included passages, via maximal marginal relevance (carbonell and goldstein, 1998). multi-document summarisation has also been extended to multilingual environments (evans, 2005). 2.4 adaptation of abstractive and extractive techniques for participatory health information summarisation documents containing health information are exceptional due to volume and heterogeneity (radev et al., 2004). given the number and diversity of health information sources, methods must be found that will enable users to quickly understand and choose the relevant document. in our adaptation for health information summarisation, extraction is used to identify the information bursts using the key sentences from multiple health documents based on the centroid method. this is followed by abstractive summarisation based on templates that allow australasian journal of information systems nallaperuma & de silva 2017, vol 21, research on health information systems participatory model for multi-document summarisation 4 generalization of sentences. a health care ontology is used to find the generalized content to match the template from sentences captured through extraction. as accuracy is vital in healthcare, healthcare domain experts are required to impart their knowledge to determine accuracy. simultaneously, health information consumers specify and rank their information expectations in support of increased accuracy. therefore, user participation makes a significant contribution to the proposed multi-document health information summarisation model. past research endeavours on participative behaviour were limited to system development. however, markus and mao (2004) present participation as personally relevant and important which results in user participants being committed to the inputs given to the system development as well as adoption and usage. moreover, when system users’ need-based attitudes were coupled with participative behaviours, the relationship between system success and user participation appeared to be stronger compared to when the system researchers operationalized the participation construct in purely behavioural terms. 3 design of a participatory model for multi-document health information summarisation it is pertinent to extend markus and mao (2004) new is theory of participation for the proposed model as it successfully addresses logical gaps and gaps created by the practice of system design and development. it was also supportive to determine participants and their contributions towards multi-document health information summarisation. figure 1: adaptation of is user participation theory for multi-document health information summarisation (markus and mao, 2004) as illustrated in figure 1, the two key groups of participants can be identified as stakeholders (medical practitioners) and change agents (health information consumers). the participation activity for medical practitioners is the use of domain knowledge for template design whereas health information consumers contribute ratings and comments to convey their position on relevance of summarisation outcomes. as reported by markus and mao (2004), these activities would lead to the development and implementation success of a summarisation system that takes into account participatory activities. medical practitioners initiate their contribution of domain knowledge from template generation for the abstractive summarisation process. given the nature of health information, it is evident that different user groups seek different levels of information from a summary, for instance, a health information consumer requires a general understanding regarding the topic; a medical student requires an in-depth understanding of the topic and a medical practitioner requires a quick summary. therefore with the support of ten experts from the health domain, templates were generated for different types of information requirements (figure 2). these australasian journal of information systems nallaperuma & de silva 2017, vol 21, research on health information systems participatory model for multi-document summarisation 5 templates can be further extended to include other stakeholders such as carers, family and support providers. figure 2: templates generated for three distinct groups of end-users the following high level diagram provides an overview of complete model and the role and purpose of participation. figure 3: high-level schematic of the proposed model 4 development of the summarisation model this section provides a detailed account of each module illustrated in figure 3. the extraction module, abstraction module and nlp-based integration of the outcomes from these two modules are explicated below. 4.1 extraction module for the extraction module, mead (radev et al., 2004) an open-source framework for multi document extractive summarisation was extended to suit the requirements of the healthcare domain. hence, the adapted mead will be referred to as mead*. the mead framework decomposes sentence extraction into three steps. in the first step numerical features (positionbased, centroid-based, largest common subsequence and keywords) are calculated for each australasian journal of information systems nallaperuma & de silva 2017, vol 21, research on health information systems participatory model for multi-document summarisation 6 sentence. these features are assumed to correlate with how informative the sentence would be for summarising the given set of documents. for instance, one feature is based on the position of the sentence in the document, as sentences at the beginning (or at the end) of a document tend to be more general and informative. in mead*, the sentence feature was set to centroidbased which measures how similar the sentence is to the set of documents to be summarised. this is done by using a corpus where the set of documents will be represented as a centroid vector. as a result, each word is associated with the product of its term frequency (tf) in the documents times its inverse document frequency (idf) in the entire corpus. the centroid feature for a sentence can be then computed as the cosine overlap between the word occurrence vector for the sentence and the centroid vector of the set of documents. following feature calculation, a new classification phase was introduced to mead*. all the sentence features computed in the previous step are combined into a single numerical score for each sentence. finally, mead* performs re-ranking based on centroids where the numerical score for each sentence is adjusted relative to other sentences. this allows the system to avoid redundancy in the final set of sentences by lowering the scores of sentences that are similar to already selected sentences. 4.2 abstraction module the abstraction module aims to capture and summarise the semantics of content rather than statistical contribution of the text. abstraction consists of two major components: a content planner that selects information to be include in the summary through combination of input templates, and a sentence generator that selects appropriate words to express the information in a grammatically correct format. 4.2.1 content planning with user participation using a well-established domain ontology, snomed ct, content would be extracted to suit the templates above (figure 2). health information consumers will seek a generic template out of the sentences extracted whereas medical students will seek a more detailed template. once the connections are made using content planning component and the template is populated. keywords are retrieved from the extracted sentences and matched for meaning using the snomed ct ontology. for instance, the sentence 'pre-eclampsia is a disease' is identified as pre-eclampsia as a child node of disease in the ontology. table 1 presents a sample of sentences extracted for different templates. it can be observed that same content has been extracted for the criteria disease, reason, and relevance. even though, diagnosis of the health information consumer is stated as uncertain whereas for the medical student’s template diagnosis has a value. this is due to the sentences extracted for health information consumer where diagnosis has retrieved a low importance in the document. also, more details such as sections of a sentence was extracted for medical student and practitioner as it is critical not to lose information from the extracted sentences. moreover, in practitioner’s template (table 1) infection has three infections identified through the ontology. australasian journal of information systems nallaperuma & de silva 2017, vol 21, research on health information systems participatory model for multi-document summarisation 7 table 1: sentence extraction for different templates 4.2.2 nlp-based sentence generation sentence generation is a key component of the proposed model. the sentence generator gathers all the combined information and uses connective phrases to synthesize a summary, devised by adapting existing language generation tool; natural language tool kit (bird, klein, & loper, 2009). content were derived from the generated templates and diverse levels of healthcare terms were used to generate feature based sentences for different information consumers. sentence generation is carried out using feature structures which contains various kinds of information about grammatical entities. in the case of a verb, it is often useful to know what the semantic role played by the arguments, for example {'disease': obj, 'prevention': sub, 'prevent': v}. further, grammar is concerned with how words and sequences of words combine to form sentences. a distinct and complementary approach, dependency grammar, was used instead of how words relate to other words. therefore, {'disease': obj, ‘reason’: sub, cause: v} (table 1) feature structure was used to generate the sentence with the appropriate tense such as “high blood pressure and proteinuria cause pre-eclampsia” 5 results experiments were conducted using a test dataset of health documents from different domains. the two modules; extraction, abstraction (content planner and sentence generator). 5.1 extraction module module was tested with a test case which comprises three documents in order to check whether the relevant yet different level of information is extracted for different users. in the interest of space, results from three documents used to test the extraction module are shown below. out of three documents, two were in the domain of obstetrics and gynecology whereas the other was in the domain of cardiac defibrillators which is outside the domain of concern. difference in sentence extraction is highlighted. health information consumer disease: pre-eclampsia reason: high blood pressure and proteinuria whom: pregnancy diagnosis: uncertain medical student disease: pre-eclampsia whom: pregnancy diagnosis: no preventative measures or screening tools for early detection of the condition reason: one or more infectious agents present in the placenta, with a predominant organism triggering the condition impact: responsible for about 50,000 maternal deaths and 28% of all pregnancy complications worldwide annually medication: defective placentation medical practitioner disease: pre-eclampsia reason: high blood pressure and proteinuria infection: respiratory tract, female reproductive tract, periodontal cavity prevention: no preventative measures or screening tools for early detection of the condition australasian journal of information systems nallaperuma & de silva 2017, vol 21, research on health information systems participatory model for multi-document summarisation 8 user sentences extracted m ed ic a l d o ct o r [1] microbiome of the placenta in pre-eclampsia supports the role of bacteria in the multifactorial cause of pre-eclampsia [2] pre-eclampsia, a condition of high blood pressure and proteinuria in pregnancy, is a leading cause of maternal and infant morbidity and mortality. [3] it is thought that the presence of bacteria in the placenta could trigger increased release of anti-angiogenic factors, like soluble fms like tyrosine kinase 1 (sflt 1) and decreased release of pro-angiogenic factors, like placental growth factors (pigf) and vascular endothelial growth factor (vegf) from the placenta. [4] a meta-analysis for the association of infectious agents with pre-eclampsia reports a twofold increase in the risk of pre-eclampsia in the presence of infection (odd ratio [or] 2.1, 95% confidence interval [ci] 1.6 2.7). [5] on the other hand, pre-eclampsia may be triggered by the infectious burden of multiple organisms rather than the presence of a specific infectious agent that may not be adequate enough to cause a symptomatic infection but may activate the inflammatory cells and increase of anti-angiogenic factors. m ed ic a l in fo rm a ti o n c o n su m er [1] microbiome of the placenta in pre-eclampsia supports the role of bacteria in the multifactorial cause of pre-eclampsia. [2] pre-eclampsia, a condition of high blood pressure and proteinuria in pregnancy, is a leading cause of maternal and infant morbidity and mortality. [3] in the usa, pre-eclampsia is responsible for around 61% of pregnancy complications and accounts for 15.9%maternal deaths. [4] it is thought that the presence of bacteria in the placenta could trigger increased release of anti-angiogenic factors, like soluble fms like tyrosine kinase 1 (sflt 1) and decreased release of pro-angiogenic factors, like placental growth factors (pigf) and vascular endothelial growth factor (vegf) from the placenta. [5] a meta-analysis for the association of infectious agents with pre-eclampsia reports a twofold increase in the risk of pre-eclampsia in the presence of infection (odd ratio [or] 2.1, 95% confidence interval [ci] 1.6 2.7). [6] on the other hand, pre-eclampsia may be triggered by the infectious burden of multiple organisms rather than the presence of a specific infectious agent that may not be adequate enough to cause a symptomatic infection but may activate the inflammatory cells and increase of anti-angiogenic factors. [7] this could lead to a series of events, such as abnormal trophoblast function leading to endothelial dysfunction and impairment of oxygen nutrients to the placenta, which raises the maternal blood pressure causing pre-eclampsia. [8] the alternate, therefore, would be to examine the placenta of women with pre-eclampsia harvested at the time of delivery using culture independent highly sensitive methods that would enable one to detect traces of dna of infectious agents left behind by infection at the time of placentation. [9] researchers have discovered how one genetic and one sexual risk factor can combine to increase the risk of pre-eclampsia. m ed ic a l s tu d en t [1] microbiome of the placenta in pre-eclampsia supports the role of bacteria in the multifactorial cause of pre-eclampsia [2] pre-eclampsia, a condition of high blood pressure and proteinuria in pregnancy, is a leading cause of maternal and infant morbidity and mortality. [3] in sri lanka, pregnancy induced hypertension account for 7% of maternal and 24% of neonatal deaths due to premature delivery. [4] in the usa, pre-eclampsia is responsible for around 61% of pregnancy complications and accounts for 15.9%maternal deaths. [5] it is thought that the presence of bacteria in the placenta could trigger increased release of anti-angiogenic factors, like soluble fms like tyrosine kinase 1 (sflt 1) and decreased release of pro-angiogenic factors, like placental growth factors (pigf) and vascular endothelial growth factor (vegf) from the placenta. [6] a meta-analysis for the association of infectious agents with pre-eclampsia reports a twofold increase in the risk of pre-eclampsia in the presence of infection (odd ratio [or] 2.1, 95% confidence interval [ci] 1.6 2.7). [7] the first study that examined amniotic fluid of women with pre-eclampsia using both conventional microbial culture techniques and broad rage and group specific polymerase chain australasian journal of information systems nallaperuma & de silva 2017, vol 21, research on health information systems participatory model for multi-document summarisation 9 reaction (pcr) assays targeting the 16s ribosomal rna (rrna) gene reported the presence of lactobacillus iners, ureaplasma, sneathia, leptotrichia and streptococcus species. [8] on the other hand, pre-eclampsia may be triggered by the infectious burden of multiple organisms rather than the presence of a specific infectious agent that may not be adequate enough to cause a symptomatic infection but may activate the inflammatory cells and increase of anti-angiogenic factors. [9] this could lead to a series of events, such as abnormal trophoblast function leading to endothelial dysfunction and impairment of oxygen nutrients to the placenta, which raises the maternal blood pressure causing pre-eclampsia. [10] the current study was based on the hypothesis that pre-eclampsia is caused by one or more infectious agent present in the placenta, with a predominant organism triggering the condition. [11] the alternate, therefore, would be to examine the placenta of women with pre-eclampsia harvested at the time of delivery using culture independent highly sensitive methods that would enable one to detect traces of dna of infectious agents left behind by infection at the time of placentation. [12] researchers have discovered how one genetic and one sexual risk factor can combine to increase the risk of pre-eclampsia. [13] elizabeth triche and colleagues at brown university say their findings suggest there could be new ways for couples to plan pregnancy with improved awareness of the disorder, as well as improved management of the risk. table 2: sentences extracted from multiple documents therefore, it is evident that for different users’ different level and number of sentences are extracted. yet, all the sentences would be in the domain on obstetrics and gynecology. the content submitted from outside the domain was detached. 5.2 abstraction module the sentences extracted from the same documents for different users were used to test the abstraction module. first the content planner generated the following templates using the snomed ct ontology. 5.2.1 content planner table 1: demonstrates the templates generated for different users. it can be observed that same content has been extracted for the disease, reason, whom etc. even though, diagnosis of the medical information consumer is stated as uncertain whereas for the medical student’s template diagnosis has a value. this is due to the content extracted and the diagnosis has retrieved a low importance for the medical information consumer. also, more details such as sections of a sentence was extracted for medical student and medical doctor as it is critical not to lose information from the extracted sentences. moreover, in medical doctor’s template infection has three infections identified through the ontology and is concatenated using comma. 5.2.2 sentence generator using the templates generated for different users, following sentence were generated; medical information consumers will get a generic and more standard output, medical students will get a more detailed summary, whereas medical doctors will get a more compressed summary in order to give the medical doctor a basic idea on the content. australasian journal of information systems nallaperuma & de silva 2017, vol 21, research on health information systems participatory model for multi-document summarisation 10 user summary generated medical information consumer “high blood pressure and proteinuria causes pre-eclampsia. pregnancy gets pre-eclampsia. uncertain symptoms pre-eclampsia. detective placentation cures pre-eclampsia.” medical student “one or more infectious agents present in placenta, with a predominant organism triggering the condition causes preeclampsia. pregnancy gets pre-eclampsia. no preventative measure or screening tools for early detection of the condition is a symptom of pre-eclampsia. defective placentation cures pre-eclampsia.” medical doctor “respiratory tract, female reproductive tract, periodontal cavity infects pre-eclampsia. no preventive measures or screening tools for early detection of the condition prevents pre-eclampsia. “ table 3: summary generated for different user groups 6 evaluation online articles on obstetrics which are used by community (e.g. babycenter.com) was used for testing and evaluation of the developed multi-document summarisation system. the system was evaluated by 10 domain experts, obstetricians, and 50 healthcare information consumers in the obstetrics healthcare domain. the health information consumers were the users of online health forums such as http://www.babycenter.com/ and were willing to voluntarily participate in the study. 6.1 extraction module precision, recall, f-score and compression rate measures were used to evaluate the extraction module using the statistical values. figure 4: evaluation of extraction module as shown in figure 4, accuracy of extraction was high with an f-score close to 1 across all templates (0.73 for health information consumers, 0.69 for medical students and 0.86 for practitioners). figure 4 also presents compression rate of extractive summaries. compression rate (knight & marcu, 2000) depends on the details that are extracted from the content as well as the comprehensive rate of sentence generation. the evaluations show high compression rate values closer to 0 which denotes higher level of summarisation. australasian journal of information systems nallaperuma & de silva 2017, vol 21, research on health information systems participatory model for multi-document summarisation 11 6.2 abstraction module evaluation of the abstraction module required metrics of relevance of levels of sentences. in order to determine these metrics a qualitative study was required. the domain experts volunteered to compose manual summaries of the same documents used for this evaluation. the results are shown in table 4. manual analysis of the system generated summaries and expert generated summaries shows that same content and idea has been reflected in both the summaries. it is evident that the sentence generated could be more comprehensive although system generated summaries consist of many sentences than expert generated summaries. example 1 example 2 health information consumer system summarised: high blood pressure and proteinuria causes pre-eclampsia. pregnancy gets pre-eclampsia. uncertain symptoms pre-eclampsia. system summarised: contraceptive actions causes progesterone. women gets progesterone. bio-markers symptoms progesterone. expert summarised: 'pre-eclampsia is caused by high blood pressure and proteinuria in pregnancy which is diagnosed by uncertain and is medicated by defective placentation' expert summarised: ‘progesterone issues are caused by contraceptives taken by women and could be diagnosed with bio-markers.’ medical student system summarised: one or more infectious agents present in the placenta, with a predominant organism triggering the condition causes pre-eclampsia. pregnancy gets pre-eclampsia. no preventative measures or screening tools for early detection of the condition is a symptom pre-eclampsia. defective placentation cures pre-eclampsia. system summarised: menstrual abnormalities during the later reproductive years causes progesterone. women gets progesterone. bio-markers is a symptom of progesterone. pre-term delivery is an impact of progesterone. use of levonorgestrel instrauterine cures progesterone. expert summarised: 'pre-eclampsia is caused by high blood pressure and proteinuria in pregnancy. pre-eclampsia is diagnosed by no preventative measures or screening tools for early detection of the condition. pre-eclampsia is due to one or more infectious agents present in the placenta, with a predominant organism triggering the condition. pre-eclampsia is medicated by defective placentation.' expert summarised: ‘progesterone issues could be due to menstrual abnormalities during the later reproductive years of women which could be diagnosed with biomarkers. pre-term delivery could occur due to progesterone issues and should use levonorgestrel instrauterine to avoid that.’ healthcare practitioner system summarised: respiratory tract, female reproductive tract, periodontal cavity infects preeclampsia. no preventative measures or screening tools for early detection of the condition prevents pre-eclampsia. system summarised: contraceptive actions causes progesterone. effects on all uterine cell types infects progesterone. limit the damage from being born before organ maturity prevents progesterone. expert summarised: 'pre-eclampsia is infected by respiratory tract, female reproductive tract, periodontal cavity and is prevented by no preventative measures or screening tools for early detection of the condition' expert summarised: ‘progesterone issues are caused by contraceptives which would affect all uterine cell types and medication should be taken to avoid pre-term delivery.’ table 4: qualitative evaluation of system generated and expert generated abstraction summaries australasian journal of information systems nallaperuma & de silva 2017, vol 21, research on health information systems participatory model for multi-document summarisation 12 7 conclusion in this paper we have presented a novel participatory model for multi-document health information summarisation. although many summarisation techniques have been proposed in research literature, none consider the crucial participatory role of both end-users and domain experts. the updated is participation theory was extended to identify how user participation can be incorporated to a summarisation approach. the proposed model initiates with participatory template generation, followed by an abstractive and extractive summarisation approach. the generated summaries are integrated using nlp-based semantics integration and then presented based on the original template structure. the proposed model was developed using a client-server architecture and evaluated by 10 domain experts and 50 health information consumers in obstetrics. both quantitative and qualitative evaluation outcomes are indicative of an effective participatory model for summarisation. proposed model proves that user participation is essential in document summarisation as a summary depends on user preference. user participation is beneficial in health domain due to the diversity embedded in health information. health forums demonstrate the need for user participation where different users look for different levels of information. the proposed model extends user participation in is theory with different user group participation where medical practitioners participate with their domain knowledge and health information consumers participate by providing ratings and comments. the use of the model allows users to customize and generate relevant and accurate summaries for diverse expectations. as future work, we intend to extend the summarisation process to uses domain ontologies and semantic reasoning to generate a hybrid version of summaries with aggregate reasoning outcomes. moreover, the summarisation process will be expanded to capture social media inputs as part of the summaries as well as conduct evaluations into other healthcare domains. references afantenos, s., karkaletsis, v., & stamatopoulo, p., (2005). “summarisation from medical documents: a survey”, artificial intelligence in medicine, 33(2), p. 157-177. aliguliyev, r.m., (2009). “a new sentence similarity measure and sentence based extractive technique for automatic text summarisation”, expert systems with applications, (36), p. 7764–7772. barzilay, r., mckeown, k. and elhadad, m., (1999). “information fusion in the context of multi-document summarisation.”, in proceedings of acl '99. belkebir, r. and guessoum, a., (2016). concept generalization and fusion for abstractive sentence generation. expert systems with applications. bird, s., klein, e., & loper, e. (2009). natural language processing with python (1 edition). beijing ; cambridge mass.: o’reilly media. burstein, f., fisher, j. l., mckemmish, s. m., manaszewicz, r., malhotra, p. (2005). “user centred quality health information provision: benefits and challenges,” in: proceedings of the thirty-eighth annual hawaii international conference on system sciences burstein, f., mckemmish, s.m., fisher, j.l., manaszewicz, r., malhotra, p. (2006). “a role for information portals as intelligent decision support systems: breast cancer knowledge online experience”, in: intelligent decision-making support systems: foundations, applications and challenges, springer, london uk, pp. 359-383. burstein, f., mckemmish, s.m., fisher, j.l., manaszewicz, r., malhotra, p. (2006). “a role for information portals as intelligent decision support systems: breast cancer knowledge online experience”, in: intelligent decision-making support systems: foundations, applications and challenges, springer, london uk, pp. 359-383. australasian journal of information systems nallaperuma & de silva 2017, vol 21, research on health information systems participatory model for multi-document summarisation 13 collins, m., (1999). “head-driven statistical models for natural language parsing”, university of pennsylvania. de silva, d., burstein, f. and fisher, j., (2012). supporting personalised content management in smart health information portals. in proceedings of the 23rd australasian conference on information systems 2012 (pp. 1-11). eichhorn, b. r., & tukel, o. i. (2015). a review of user involvement in information system projects. int. j. inf. technol. proj. manag., 6(1), 26–53. erkan, g. and radev, d.r., (2004). “lexrank: graph-based lexical centrality as salience in text summarisation”, journal of artificial intelligence research, (22), pp. 457-479. eysenbach, g., (2000). “consumer health informatics”, bmj, (320), pp. 1713-1716. eysenbach, g. and köhler, c., (2002). “how do consumers search for and appraise health information on the world wide web? qualitative study using focus groups, usability tests, and in¬depth interviews”, bmj, (324), pp. 573-577. jadad, a.r. and gagliardi, a., (1998). “rating health information on the internet: navigating to knowledge or to babel”, jama, (279:8), pp. 611-614. jones, k.s., (2007). “automatic summarizing: the state of the art”, information processing and management, (43), pp 1-41. kappelman, leon. a., and mclean, ephraim. r., (1992). “promoting information system success: the respective roles of user participation and user involvement,” journal of information technology management, (3:1), p.1-12 knight, k., & marcu, d., (2000). “statistics-based summarisation step one: sentence compression”, in proceedings of the seventeenth national conference on artificial intelligence and twelfth conference on innovative applications of artificial intelligence, p. 703–710. lafferty, j.d. and mccallum, a., (2001). “conditional random fields: probabilistic models for segmenting and labeling sequence data”, in proceeding of the 18th international conference on machine learning, san francisco, p. 282-289. lloret, e. and palomar, m., (2012). text summarisation in progress: a literature review. artificial intelligence review, 37(1), pp.1-41. mani, i. and maybury, m.t., (1999). “advances in automated text summarisation”, cambridge: mit press. markus, m. l., and mao, j.-y. (2004). "participation in development and implementationupdating an old, tired concept for today's is contexts," journal of the association for information systems (5:11), p. 14. mckemmish, s.m., manaszewicz, r., burstein, f., fisher, j.l. (2009). “consumer empowerment through metadata-based information quality reporting: the breast cancer knowledge online portal,” journal of the american society for information science and technology (60:9), john wiley & sons, pp 1792-1807. mckeown, k., klavans, j., hatzivassiloglou, v., barzilay, r. and eskin, e., (1999). “towards multidocument summarisation by reformulation: progress and prospects”, in aaai/iaai, p.453-460. mckeown, k.r., and radev, d.r., (1995). “generating summaries of multiple news articles”, in proceedings of sigir '95, seattle, washington, p.74-82. moen, h., peltonen, l.-m., heimonen, j., airola, a., pahikkala, t., salakoski, t., & salanterä, s., (2016). “comparison of automatic summarisation methods for clinical free text notes”, artificial intelligence in medicine, (67), p. 25-37. australasian journal of information systems nallaperuma & de silva 2017, vol 21, research on health information systems participatory model for multi-document summarisation 14 pilemalm, s., & timpka, t. (2008). third generation participatory design in health informatics—making user participation applicable to large-scale information system projects. journal of biomedical informatics, 41(2), 327–339. pivovarov , r., & elhadad, o., (2015). "automated methods for the summarisation of electronic health records", jamai, p. 938-947. radev, d., hovy, e., and mckeown, k., (2002). “introduction to the special issue on text summarisation”, comput linguist, (28:4). radev, d.r., and fan, w., (2000). “automatic summarisation of search engine hit lists”, proceedings of the acl-2000 workshop on recent advances in natural language processing and information retrieval, hong kong, p.99-109. radev, d.r., jing, h., sty, m., and tam, d., (2004). “centroid-based summarisation of multiple documents”, international journal on information processing and management, (40:6), p. 919-938. saggion, h. and poibeau, t., (2013). automatic text summarisation: past, present and future (pp. 3-21). springer berlin heidelberg. sarkar, a., molla, d., and paris, c., (2013). “an approach for query focused text summarisation for evidence based medicine”, lecture notes in computer science, 7885:10), p. 295-304. thursky, k. a., & mahemoff, m. (2007). user-centered design techniques for a computerised antibiotic decision support system in an intensive care unit. international journal of medical informatics, 76(10), 760–768. wan, x., (2008). “using only cross-document relationships for both generic and topic-focused multi-document summarisations”, information retrieval, (11), p. 25-49. wan, x., yang, j., and xiao, j., (2007). “manifold-ranking based topic-focused multidocument summarisation”, in proceedings of the 20th international joint conference on artificial intelligence, hyderabad, p.2903-2908. weng, c., mcdonald, d. w., sparks, d., mccoy, j., & gennari, j. h. (2007). participatory design of a collaborative clinical trial protocol writing system. international journal of medical informatics, 76 suppl 1, s245-251. xiaoyan, c., and wenjie, l., (2011). “a spectral analysis approach to document summarisation: clustering and ranking sentences simultaneously”, information sciences, (181), p. 3816 3827. xie, j. and burstein, f., (2011). using machine learning to support resource quality assessment: an adaptive attribute-based approach for health information portals, proceedings of the 16th international conference on database systems for advanced applications international workshops, pp. 526-537 copyright: © 2017 nallaperuma & de silva. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems nallaperuma & de silva 2017, vol 21, research on health information systems participatory model for multi-document summarisation 15 paper title (paper title style) australasian journal of information systems poulsen 2020, vol 24, post publication review telehealth and user perceptions 1 burmeister, o. k., ritchie, d., devitt, a., chia, e., dresser, g., & roberts, r. (2019). the impact of telehealth technology on user perception of wellbeing and social functioning, and the implications for service providers. australasian journal of information systems, 23. doi:10.3127/ajis.v23i0.1501 review this post publication review discusses the article by burmeister et al. (2019) in which the authors present a study evaluating the social, economic, and wellbeing impacts of telehealth technology in the homes of community-dwelling older persons, highlighting its contribution and identifying areas for further research. the article centres on one particular project in which telehealth monitoring systems were used in homes in regional new south wales. the reviewed article is a timely contribution to the information systems (is) literature. the growing number of elderly and fewer number of caregivers to meet the needs of elderly in care is a trend observed globally (burmeister, 2016; draper & sorell, 2017; poulsen & burmeister, 2019). there are calls for healthcare is interventions, such as wearable assistive devices (fosch-villaronga & özcan, 2019), telehealth (shafiee hanjani, caffery, freeman, peeters, & peel, 2019), and care robots (poulsen, burmeister, & kreps, 2018; poulsen, burmeister, & tien, 2018) to help ease the impacts of this trend, particularly in regional or rural communities (smith, armfield, & caffery, 2019). the work reviewed here creates knowledge which helps to soften the introduction of telehealth technology into the homes of the older population. burmeister et al. (2019) further our understanding of the particular access needs of such communities, this is essential to improving user wellbeing (pakrasi, burmeister, coppola, mccallum, & loeb, 2015), social support (banbury et al., 2020), and health literacy (banbury et al., 2020) with telehealth technology. an important area highlighted in the reviewed article is good care with technology. for instance, beyond taking standard health measurements the technologies deployed in this study were adapted to obtain custom measurements related to each user’s health condition. the customisation of healthcare technology demonstrates good care as it aligns with aspects of good human-delivered care which is person-centred (abma, molewijk, & widdershoven, 2009) and determined in relationship with the user in context (upton, 2011). combi, pozzani, and pozzi (2016) highlight the lack of telehealth literature clarifying lessons learned, challenges, and recommendations related to design emerging from telehealth studies. while the authors of the reviewed article occasionally put the knowledge created throughout this study into several important learnings for future work in this area, a wider impression might have been made by setting out clear principles to guide the design, implementation, and ongoing support of telehealth technologies with older persons, much like greenhalgh et al. (2015) have done previously. for instance, in the discussion, to mitigate economic concerns associated with telehealth the authors suggested that less expensive personal monitoring equipment alternatives “could be used in conjunction with digital scales and automatic blood pressure monitors” (burmeister et al., 2019, p. 13), examples of alternative equipment and costs were given. as another example, in the concluding remarks, the authors noted that a nurse who assisted participants throughout the study was instrumental in improving the health literacy of participants. australasian journal of information systems poulsen 2020, vol 24, post publication review telehealth and user perceptions 2 although a variety of guiding statements, such as the two examples above, appear throughout the article reviewed, such insights ought to also have been consolidated into a set of principles for practitioners to refer to in future work and research utilizing telehealth technologies with older persons. furthermore, these principles should link back to the research questions, chiefly promoting wellbeing, social functioning, and positive economic impacts. on the study’s aim to review social functioning impacts, additional social dimensions could have been explored to reveal further social impacts and create an extensive understanding of the telehealth-older user social ecosystem. exemplary social dimensions include the social interactions between older persons and telehealth devices, those interactions between older users and telehealth systems capable of conversing or emotive actions, social effects on general practitioners using telehealth, social factors influencing user perceptions of telehealth technologies, and so on. overall, the study presented by burmeister et al. (2019) makes invaluable contributions to the literature on telehealth and the older population. in future work, further social functioning impacts ought to be explored and the lessons learned in this study should be formulated into a set of principles for addressing the design, implementation, and ongoing support of telehealth for older persons. adam poulsen charles sturt university apoulsen@csu.edu.au author response the authors were given the opportunity to respond, but chose not to. references abma, t., molewijk, b., & widdershoven, g. (2009). good care in ongoing dialogue. improving the quality of care through moral deliberation and responsive evaluation. health care analysis, 17, 217-235. doi:10.1007/s10728-008-0102-z. banbury, a., nancarrow, s., dart, j., gray, l., dodson, s., osborne, r., & parkinson, l. (2020). adding value to remote monitoring: co-design of a health literacy intervention for older people with chronic disease delivered by telehealth the telehealth literacy project. patient education and counseling, 103(3), 597-606. doi:10.1016/j.pec.2019.10.005. burmeister, o. k. (2016). the development of assistive dementia technology that accounts for the values of those affected by its use. ethics and information technology, 18(3), 185-198. doi:10.1007/s10676-016-9404-2. burmeister, o. k., ritchie, d., devitt, a., chia, e., dresser, g., & roberts, r. (2019). the impact of telehealth technology on user perception of wellbeing and social functioning, and the implications for service providers. australasian journal of information systems, 23. doi:10.3127/ajis.v23i0.1501. mailto:apoulsen@csu.edu.au australasian journal of information systems poulsen 2020, vol 24, post publication review telehealth and user perceptions 3 combi, c., pozzani, g., & pozzi, g. (2016). telemedicine for developing countries. a survey and some design issues. applied clinical informatics, 7(4), 1025-1050. doi:10.4338/aci2016-06-r-0089. draper, h., & sorell, t. (2017). ethical values and social care robots for older people: an international qualitative study. ethics and information technology, 19(1), 49-68. doi:10.1007/s10676-016-9413-1. fosch-villaronga, e., & özcan, b. (2019). the progressive intertwinement between design, human needs and the regulation of care technology: the case of lower-limb exoskeletons. international journal of social robotics. doi:10.1007/s12369-019-00537-8. greenhalgh, t., procter, r., wherton, j., sugarhood, p., hinder, s., & rouncefield, m. (2015). what is quality in assisted living technology? the archie framework for effective telehealth and telecare services. bmc med, 13, 91. doi:10.1186/s12916-015-0279-6. pakrasi, s., burmeister, o. k., coppola, j. f., mccallum, t. j., & loeb, g. (2015). ethical telehealth design for users with dementia. gerontechnology. gerontechnology, 13(4), 383387. doi:10.4017/gt.2015.13.4.002.00. poulsen, a., & burmeister, o. k. (2019). overcoming carer shortages with care robots: dynamic value trade-offs in run-time. australasian journal of information systems, 23. doi:10.3127/ajis.v23i0.1688. poulsen, a., burmeister, o. k., & kreps, d. (2018). the ethics of inherent trust in care robots for the elderly. in d. kreps, c. ess, l. leenen, & k. kimppa (eds.), this changes everything – ict and climate change: what can we do? (pp. 314-328). doi:10.1007/978-3319-99605-9_24 poulsen, a., burmeister, o. k., & tien, d. (2018). a new design approach and framework for elderly care robots. paper presented at the australasian conference on information systems, sydney, australia. shafiee hanjani, l., caffery, l. j., freeman, c. r., peeters, g., & peel, n. m. (2019). a scoping review of the use and impact of telehealth medication reviews. research in social and administrative pharmacy. doi:10.1016/j.sapharm.2019.12.014. smith, a. c., armfield, n. r., & caffery, l. j. (2019). telehealth a game changer: closing the gap in remote aboriginal communities. medical journal of australia, 211(1), 43-43.e41. doi:10.5694/mja2.50208. upton, h. (2011). moral theory and theorizing in health care ethics. ethical theory and moral practice, 14(4), 431-443. doi:10.1007/s10677-011-9295-6. copyright: © 2020 poulsen. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v24i0.2769 http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems poulsen 2020, vol 24, post publication review telehealth and user perceptions 4 paper title (paper title style) australasian journal of information systems ghandour & ghandour 2019, vol 23, research on health information systems using personally controlled health records 1 the experiences of three different user groups using personally controlled health record for multidisciplinary care team nadia ghandour senior analyst, royal brompton and harefield nhs foundation trust united kingdom nadia.ghan@gmail.com ahmad ghandour associate professor college of business al ain university of science & technology abu dhabi / uae abstract personally controlled health records (pchr) are patient-centric tools which allow individuals to own, manage, access and share their health information online from anywhere, at any time. patient-centric tools have received considerable interest and investment in recent years worldwide. these tools are thought to have the potential to increase individuals’ selfmanagement and involvement in their own health, as well as improve healthcare efficiency and delivery. the aim of this study is to report and analyse the experiences of three different user groups using pchr for multidisciplinary care team (mdct) including the advantages, disadvantages, barriers and obstacles, and the current state of pchr. in order to achieve the aim of this study, sixteen interviews with key informants from three different user groups were conducted in dunedin, new zealand. interviews were transcribed and analysed with thematic analysis. the key findings of this research showed that those who can benefit the most from pchrs are the least able to use it. it suits those who have basic knowledge about computers and the internet and those who can afford to use them. pchr is also best suited for individuals who are motivated about their health despite their health condition. however, more research is needed in the future with a larger sample, an easier to use pchr, different population other than community health workers, patients with different chronic illnesses, and healthy patients. this research can be used as a basis and tested in future research on pchr adoption. keywords: pchr, ehr, the diary, health informatics, mdct 1 introduction information technology is playing a vital role in attempts to improve healthcare systems and address current challenges in healthcare delivery. with the rise of chronic diseases and the major cost consumed on those with long-term conditions, there is an even greater need to find solutions that improve health care systems significantly. the emphasis on patient-centric and integrated healthcare systems is worldwide. patient-centric tools have the potential to improve patients’ satisfaction with their health, their clinical outcomes, and medical services efficiency. in recent years, personally controlled health records (pchrs) in particular have sparked significant interest in healthcare systems (pagliari, detmer, & singleton, 2007; tang, ash, australasian journal of information systems ghandour & ghandour 2019, vol 23, research on health information systems using personally controlled health records 2 bates, overhage, & sands, 2006). other terms have also been used in the literature and in organisations such as personal health records (phrs) and electronic health diary (ehd) (lober et al., 2006). for this current research, the term pchr will be used to fit any system that is webbased and allows patients to access, control, manage and track their own health. in the past few years, there has been a dramatic increase in the number of patients seeking health information online, on social media sites and other web-based resources (archer, fevrier-thomas, lokker, mckibbon, & straus, 2011; beard, schein, morra, wilson, & keelan, 2012). modern countries are striving to improve the way in which health services are delivered and managed. they are finding an effective way which will reduce the death rate among those with long-term conditions as well as reducing the cost. one approach is a person-centric shared plan which involves all health professionals that have a role in the patient’s care working to a common care plan and sharing information between them. the objective of this person-centric approach is to put the person at the centre of their care and create a collaborative/integrated environment for those with long-term conditions to manage their conditions at home, and for their caregivers who play a part in managing their conditions. this approach will get patients involved in managing their own health, improve efficiency of the usage of healthcare resources, improve time saving and safety, and improve patients’ relationships with their health providers. shared care planning allows patients access to their personal health information, treatment and follow-up plans, receive their laboratory results, and communicate electronically with their health providers. physicians hold health information on their patients and spend a considerable amount of time during an office visit to provide their patients with education and instructions regarding their health (tang et al., 2003). at the same time, there is a limited time for each visit, therefore patients are not always satisfied because they may not have enough time to get the information they need. also, being under time pressure could mean that they might forget to ask questions, or it could result in them leaving without understanding (levinson, lesser, & epstein, 2010; tang et al., 2003). for example, according to a survey of patients conducted in 2004 by the commonwealth fund in five countries (australia, canada, new zealand, the united kingdom, and the united states); respondents across all countries reported that their doctors are sometimes, rarely or never able to communicate well with patients, tell them about treatment plans or get them involved in making care decisions (ball, carla smith, & bakalar, 2007; schoen et al., 2004). patients need to be more engaged as partners in their health and their knowledge about their health needs to be satisfied (tang et al., 2003). electronic health records (ehrs) are not typically shared with patients (beard et al., 2012). thus, better methods of providing information to patients are needed (beard et al., 2012; zickmund et al., 2008). personally controlled health records (pchrs) represent an important and increasingly accepted health information technology to support self-management and patient-centred care (krist & woolf, 2011). moreover, the care of patients nowadays usually involves multidisciplinary care teams (mdcts) that all require to share and discuss information about patients. those mdcts are lacking communication and facing a major issue in delivering healthcare efficiently. there is a gap in communication between healthcare providers across primary care providers in the community and the hospital-based care providers. community healthcare providers are australasian journal of information systems ghandour & ghandour 2019, vol 23, research on health information systems using personally controlled health records 3 struggling in obtaining the information they need about their patients, thus, facing issues in delivering quality care (coiera, 2006). therefore, there is an increasing need for a system that bridges the gap between the complex mdcts as well as the patients and puts them all on the same page. it is also very important to make health information available at an efficient time for those who need it in order to ensure success of care (fichman, robert 2011). this research area is still at its inception and the literature is growing towards contributing valuable outcomes for the diffusion of pchr. no studies have looked specifically at the use of pchr by community groups nor at the use of pchr by different user groups. for the successful use of pchr by different groups, it is important to understand their views and the experiences of the use of this new phenomenon. it is important to learn: 1. if pchr will improve workflow and efficiency of current healthcare systems as has been reported in the literature. 2. the advantages and disadvantages of using the pchr based on the experiences of those groups who used it. 3. user’s concerns about pchr including privacy concerns. 4. identify the group of people that will most likely use and benefit from the pchr. 5. learn about the current state of pchr. this research will help understand the experiences of different user groups on how pchr will possibly benefit them, what are the disadvantages they experienced with using the new phenomenon, who is the best user for pchr from their point of view, what are their concerns on pchr and if they view pchr as a tool that will improve the efficiency of healthcare systems. the aim of this current research is to report and analyse the experiences of three different user groups using pchr for mdct. this article is structured as follows. first, pchr is reviewed with regards to its benefits, barriers adoption and current state, followed by the methodology of the study. subsequently, results and analysis section is presented followed by discussion. finally, a conclusion ends the study. 2 personally controlled health record (pchr) a personally controlled health record (pchr) is a special type or a sub-set of personal health records (phrs) distinguished by giving users more control over the access to and the content of their records (bourgeois, taylor, emans, nigrin, & mandl, 2008; mandl, simons, crawford, & abbett, 2007; weitzman, kaci, & mandl, 2009). markel foundation (2003) defined phr in their “connecting for health” report as: “an electronic application through which individuals can access, manage and share their health information, and that of others for whom they are authorized, in a private, secure, and confidential environment.” (p.14) in pchrs, patients do not only manage their health-related information, but they also own them. patients populate their records, decide on the data sources, control who is allowed to view/edit their records, and what to view/edit exactly (mandl et al., 2007; weitzman et al., 2009). australasian journal of information systems ghandour & ghandour 2019, vol 23, research on health information systems using personally controlled health records 4 it is generally assumed that increasing individuals’ control over their medical records will improve health, self-care preparation and reduce medication errors (mandl et al., 2007; weitzman et al., 2009). pchrs and phrs include the same information; they both allow individuals to capture, access, and manage their own health information. they can have a lifelong summary of their health information in one convenient place. a typical pchr/phr captures information such as: allergies, medical conditions, medication, medical treatments, immunisation, and test results (tang et al., 2006; weitzman et al., 2009; wynia, torres, & lemieux, 2011). there are several models of pchrs that have been described in the literature, but most are variations of tethered (ehr-based) systems and untethered (stand-alone) systems (archer et al., 2011; d. kaelber & pan, 2008; patel et al., 2011). electronic health record (ehr) refers to a full health record for a patient that is primarily held by a healthcare provider in electronic form (kim et al., 2009; shaw, kulkarni, & mador, 2011). over the past few years, the adoption of ehrs has been incredibly widespread. the ehr maintained by clinicians is a medico-legal requirement for clinicians; hard copy in the first instance but increasingly becoming electronic. ehrs are considered distinct from pchrs, due to that fact that ehrs are usually controlled by health providers and patients are seldom able to easily access these records and consequently are unable to fully engage in the management of their own health plans without access to the electronic information (hillestad et al., 2005). accessibility has been limited due to many constraints, such as; legal, ethical, logistical and technological issues. ehr is a term that is sometimes used interchangeably with the term “electronic patient record” and “electronic medical record” (emr)1, although emr could refer to hospital only medical records. an ehr-based system could be tethered to a provider’s electronic health record (ehr) and might include data from several sources such as; appointment systems, laboratories, pharmacies, health insurers and other healthcare entities (patel et al., 2011). while a stand-alone system could be completely free-standing and include data entered directly from patients, tethered system usually includes data from both self-entered and the electronic health record (ehr) (anoshiravani et al., 2011; patel et al., 2011). according to archer et al. (2011) a tethered pchr can convey to the patient much more than a stand-alone pchr; it enables patients to gather their fragmented health information in one place. also, integrating pchrs with ehrs appear to be critical for the successful use of pchrs (archer et al., 2011; detmer, bloomrosen, raymond, & tang, 2008; kim et al., 2009; tang et al., 2006). table 1 below shows a summary of pchr’s characteristics which were collected from the literature in comparison to an ehr. 1 https://www.ama-assn.org/ama/pub/physician-resources/health-information-technology/health-itbasics/emrs-ehrs.page australasian journal of information systems ghandour & ghandour 2019, vol 23, research on health information systems using personally controlled health records 5 pchr ehr access patients can choose who accesses their information patients do not have access to or control over their information availability available for patients from anywhere at anytime only available for healthcare providers, and can only be accessed on site control high patient control no patient control information source mainly from patients and could be from both patients and care providers if the pchr was tethered mainly from healthcare providers ownership patients healthcare providers services accessibility flexible access to test results and appointments if available not flexible for patients transparency clear about who edited or accessed what information usually not transparent table 1: pchr vs. ehr 2.1 pchr adoption: while pchrs have great potential to provide benefits not only for individuals, but for health professionals and health providers as well (blake & kerr, 2010; detmer et al., 2008; tang et al., 2006), the supporting evidence of clinical benefits and other specific benefits for pchrs adoption remains limited (d. kaelber & pan, 2008; patel et al., 2011; tang et al., 2006; tenforde, jain, & hickner, 2011). the following are the benefits and the barriers of adopting pchrs retrieved from the existing literature: 2.1.1 benefits of adopting pchrs: for individuals, pchrs offer various potential benefits (blake & kerr, 2010; tang et al., 2006). generally, patients’ involvement in their care seems to save time, improve care and safety as well as decrease cost (bourgeois et al., 2008; greenhalgh, hinder, stramer, bratan, & russell, 2010; hiller, mcmullen, chumney, & baumer, 2011). pchrs have the potential to improve the quality of care and change the way care is delivered to patients, and deliver better health outcomes (blake & kerr, 2010; caligtan & dykes, 2011; d. kaelber & pan, 2008). table 2 below represents a summary of the main potential benefits of the pchr which were reported in the literature. australasian journal of information systems ghandour & ghandour 2019, vol 23, research on health information systems using personally controlled health records 6 1 delivering better health outcomes (caligtan & dykes, 2011; d. kaelber & pan, 2008) 2 empowering patients and getting them involved in their own health (ball et al., 2007; fuji et al., 2012) 3 engaging patients with their health (caligtan & dykes, 2011; patel et al., 2011). 4 enhancing shared decision making (caligtan & dykes, 2011; tang et al., 2006; wunder, 2009) 5 improving the relationship between patients and their health professionals (archer et al., 2011; d. c. kaelber, jha, johnston, middleton, & bates, 2008; patel et al., 2011) 6 improving care delivery and health outcomes (caligtan & dykes, 2011; d. kaelber & pan, 2008) 7 improving memory and concordance of the patient (maeder & martin-sanchez, 2012) 8 improving clinical decision making (caligtan & dykes, 2011; tang et al., 2006; wunder, 2009) 9 increased patient control/ownership (caligtan & dykes, 2011; d. kaelber & pan, 2008) 10 increasing patients’ knowledge of their own health (chen, garrido, chock, okawa, & liang, 2009; tang et al., 2006) 11 increasing patient safety emont (2011); (heather et al., 2011; pagliari et al., 2007) 12 making health information accessible from anywhere at anytime (ball et al., 2007; caligtan & dykes, 2011; fuji et al., 2012; greenhalgh et al., 2010; pagliari et al., 2007; patel et al., 2011; tang et al., 2006) 13 reducing burden of care for clinicians and care providers (chen et al., 2009; greenhalgh et al., 2010; pagliari et al., 2007) 14 saving time and cost (bourgeois et al., 2008; dimick, 2008; greenhalgh et al., 2010; heather et al., 2011; hiller et al., 2011; d. c. kaelber et al., 2008; d. kaelber & pan, 2008; pagliari et al., 2007; steinbrook, 2008; tang et al., 2006) table 2: potential benefits of pchrs 2.1.2 barriers and obstacles with adopting pchrs: despite the superficial public interest in having pchrs and the beliefs in its’ potential benefits; there are also some possible impediments that have delayed its widespread adoption that have been discussed in the literature (lober et al., 2006; tang et al., 2006). some of these impediments were related to the following major factors; to the healthcare culture and system, technical factors, infrastructure and policy factors, as well as individual-level factors (detmer et al., 2008; kahn, aulakh, & bosworth, 2009; pagliari et al., 2007; tang et al., 2006). individual-level factors include: security and privacy concerns, attitudes and expectations of pchrs, desired benefits from pchr, and user characteristics such as; demographic factors, computer and technology literacy and health literacy (kahn et al., 2009; tang et al., 2006; weitzman et al., 2009). table 3 below represents a summary of the barriers to adopting pchrs which were reported in the literature. australasian journal of information systems ghandour & ghandour 2019, vol 23, research on health information systems using personally controlled health records 7 1 anxieties around security and confidentiality (hoerbst, kohl, knaup, & ammenwerth, 2010; d. c. kaelber et al., 2008; keselman et al., 2007; pyper, amery, watson, & crook, 2004) 2 digital divide (detmer et al., 2008; roblin, houston, allison, joski, & becker, 2009; sciadas, 2002). archer et al. (2011; patel et al., 2011). kim et al (2009); (miah, hasan, hasan, & gammack, 2017); 3 healthcare cultural and workflow issues (archer et al., 2011; detmer et al., 2008; patel et al., 2011; weitzman et al., 2009). 4 individual characteristics such as age, health status or computer literacy (lafky & horan, 2008; walker, ahern, le, & delbanco, 2009) 5 legal concerns (detmer et al., 2008; patel et al., 2011; tang et al., 2006). 6 technical issues i.e. fragmented healthcare systems (kahn et al., 2009; pagliari et al., 2007) table 3: barriers to adopting pchrs 2.2 patients with chronic illness self-managed care tools such as phrs and pchrs have large potential to improve healthcare services for patients regardless of their health conditions. however, according to steinbrook (2008) and dimarco et al. (2009), patients with chronic illnesses are the users who may benefit the most from pchrs. usually people with chronic conditions are on long-term medication or treatment which involves their general practitioner (gp) and at least one other healthcare provider. these people are the most frustrated and the group that interacts with the healthcare systems most frequently (steinbrook, 2008). research has estimated that about (40%) of the general population in a number of developed countries has a chronic condition (dimarco et al., 2009; tomlinson et al., 2009). if this percentage of patients were able to manage their own health, it would have a significant effect on producing effective healthcare at lower costs (dimarco et al., 2009; tomlinson et al., 2009). fragmented systems of storing and retrieving these patients’ data will impede their optimal care (tang et al., 2006). personally controlled health records create strong potential to help those patients with long-term conditions, including mentally ill patients to manage and monitor their clinical care (emont, 2011; ennis, rose, callard, denis, & wykes, 2011). furthermore, according to an online survey done by the “markle foundation” in 2003, individuals with chronic illnesses and caregivers for elderly parents reported the highest and the most urgent need for pchrs (sprague, 2006). also, according to both tang et al. (2006) and pagliari et al. (2007) patients with long term conditions who receive care from multidisciplinary care team (mdct) are the ones who will benefit the most from pchrs and are likely to be the most frequent users of it. despite this great potential of self-management tools to enhance chronic disease care and improve patient outcomes, the evidence to improve the clinical value remains limited (tenforde et al., 2011). 2.3 the current state of the pchr: several commercial companies around the world have already started to provide services that allow people to organise their personal health information, integrate health records from different providers, and share them with other people, including healthcare professionals (bourgeois et al., 2008). these services allow more convenient and patient-centred control of australasian journal of information systems ghandour & ghandour 2019, vol 23, research on health information systems using personally controlled health records 8 health records (dimarco et al., 2009). recently various pchr platforms have been launched: indivo, microsoft’s healthvault, and the dossia (mandl et al., 2007; weitzman et al., 2009). in usa for example, “my health manager” is a customisable portal that was fully deployed in 2007 by the us health maintenance organisation kaiser permanente, and it was linked to the ehr. this portal allows members to access parts of their centrally held record, including different health information such as diagnoses, laboratory results, previous clinic visits, medication, allergies and immunizations. it also allows patients to email their doctors through a secure connection and schedule appointments (greenhalgh et al., 2010; silvestre, sue, & allen, 2009). another internet accessible pchr was introduced around the world and in england particularly was healthspace. healthspace allows individuals to create accounts to manage their health. people can add information to their accounts such as their blood pressure, and record healthcare appointments on a calendar. also, individuals may have an advanced account in which they can gain access to their nationally stored medical summary drawn by a general practitioner, book appointments, and exchange emails with their clinician (greenhalgh et al., 2010). in australia, a new programme for internet-based person-controlled electronic health records (p-cehrs) was recently introduced by the australian government. the p-cehr plan is that from july 2012 those australian patients, who wish to, will have a secure portal through which they can access their health information that is stored on their different healthcare providers’ systems. this p-cehr will show a health summary which contains an individual’s index, demographic details, medical conditions, drugs, and allergies. it will also provide a search engine for accessing a variety of personal health information such as laboratory results, prescriptions, and referrals (morrison, robertson, cresswell, crowe, & sheikh, 2011; rudd, swan, & roxon, 2010). in summary, a pchr is considered distinct from an ehr; however there is a greater perceived value in connecting them together. pchr has received considerable interest and investment in new zealand and overseas recently, however, the adoption rates have been low. furthermore, the benefits and the barriers of adopting pchrs were explored vastly in the literature, despite the fact that the clinical evidence of these benefits and barriers remain limited. moreover, the literature has stressed the value of pchrs for chronic disease populations and has seen them as being the best users for it. 3 methodology research methodology is usually determined by the aim of the study, the nature of the problem and the setting (berg, 2004). thus, some problems require qualitative methods while other may be best addressed with quantitative methods. qualitative research methods are ideal for exploratory and descriptive studies. quantitative methods are well suited for explanatory studies that collect numerical and quantified data (schutt, 2016). given that this research is mostly descriptive and aimed at exploring and gaining insights through inquiry as opposed to statistical analysis, a qualitative approach was appropriate. while there are various qualitative approaches, key informant interviews thought to be more suitable to achieve the aim of the current study. a qualitative approach using key informant interviews will allow direct interaction between the researcher and the informants to create rich data (hughes, 2002). interviews were conducted on three different groups to see different views from different australasian journal of information systems ghandour & ghandour 2019, vol 23, research on health information systems using personally controlled health records 9 perspectives. it was important to seek new ideas and information from different groups/experiences. sixteen people were interviewed about the diary (pchr) from three different groups of users. the sample could not be any larger not only due to the fact that the diary is a new novel application that not many have used locally but also because of the time limitation when this study was undertaken. the diary is an innovative online pchr which will be used as a live example for this study. by the time the interviews were conducted, the diary was a beta incomplete version. it was also a stand-alone system that is not connected to any other healthcare system. the diary collects data such as lifestyle data, medication, symptoms and health conditions and repurposes them onto a single summary page with a graph of the user’s progress over their lifetime. the diary is owned by the patient/client with data uploaded by them. however, family members and other trusted caregivers may also upload data on patients’ behalf as appropriate. patients control who to share data with and they are able to invite their doctors or other healthcare professionals to their diary by email. patients can allow the invited people to either have read-only access or to be able to edit their diary. the data then is colour coded to show where it came from and who entered it. this will allow health professionals to evaluate data accordingly. the diary is also accessible from anywhere at any time as long as there is a device with an internet connection. the selection criterion of interviewees was based on those who have used the diary; five people from the company provided the diary (group one: the diary provider), seven from the community organizations that have used the diary (group two: community health workers), and four clients who have used the diary to manage their own health (group three: clients/patients). all people who have used the diary from both of group one and two have participated in the interviews, which resulted in variations in numbers from each group. only group three could have included a larger number of participants but unfortunately only four clients have agreed to participate for reasons which will be explained in the limitation section later on. the interviews lasted 30 minutes on average and were recorded to achieve validity and comprehensiveness (patton, 2001). in-depth interviews about the diary (pchr) were conducted with sixteen people from three different groups of users in the city of dunedin /new zealand. interview structure and predeveloped themes were also prepared based on the literature review which was sent to the participant prior to the interview (table 4). thematic analysis has been used widely in qualitative research and it can be used across a range of epistemology and research questions (king, 2004; miah & genemo, 2016). while there is no clear agreement about how rigorous a thematic analysis is, king (2004), suggested that direct quotes from participants to be embedded within the analytic narratives support the validity of the identified themes and merit of the analysis. extracts of raw data not only can produce trustworthiness of the identified themes but also insightful findings (braun & clarke, 2006). the interviews were recorded and transcribed. coding and detailed analysis using thematic content analysis for the conducted key informant interviews were guided by the seven themes table 4. shows the themes and the interview guiding questions for each group. australasian journal of information systems ghandour & ghandour 2019, vol 23, research on health information systems using personally controlled health records 10 4 results and analysis analysing the data that has been collected through the key informant interviews using thematic analysis 4.1 theme one: workflow and efficiency this theme examined the opinions of the sixteen key informant interview participants regarding their current workflow and how the diary could improve efficiency. opinions were examined among the three different groups of participants. participants from the group of the diary provider each identified different characteristics of the diary that distinguish it from a normal caregiver-controlled system or electronic health records (ehrs). “...it is owned by the patient; patients can enter their own information which they can’t usually do with ehr” “the diary captures medical, health and wellness info, temporarily correlates with it and longitudinally displays, which gives caregivers clear picture of a patient’s history” “the diary repurposes health, medical and wellness data and turns it into knowledge via easyto-understand graphical display”. they also identified problems that the diary has strong potential in solving. some participants felt that the diary would give patients access to their health information, and improve communication and information sharing between patients and their caregivers or clinicians. it enables health workers to access health information of their patients, and improves the family engagement with a patients’ health information. it also would empower patients and engage them better with their own health. moreover, it has the potential to solve the problem of “lack of health and medical data on a patient in a community setting”. the diary enhances the memory of a patient, as they cannot remember everything during their short visits to the doctor. patients can enter information whenever they remember, and caregivers can keep track of them as well. another problem the diary would solve is the difficulty of displaying a large amount of health data. the diary transforms raw data into meaningful information, and uses graphs and tables to summarize and present data. moreover, it breaks the traditional doctor-centric medical system. it solves the problem for people with chronic illnesses who need to be in touch with their caregivers on a daily basis, and who usually suffer from lifelong diseases such as diabetes, high cholesterol, or obesity. the diary may help with these issues, because it; “…provides a more balanced picture in the community for lifestyle and for the progress that the patient is making in between the visit to the doctor and the hospital. also, the information entered by the patient helps the doctor to diagnose them better.” australasian journal of information systems ghandour & ghandour 2019, vol 23, research on health information systems using personally controlled health records 11 # theme the diary provider community health workers patients/clients 1 workflow and efficiency 1. what makes the diary different than a normal electronic health record (ehr)? 2. problems or needs that the diary solves or meets 1. how patients’ information is currently stored 2. needs that attracted community health workers to the diary 3. community health workers’ expectations for the diary and the problems they expected it to solve 1. clients’ accessibility and desire to have more access to their health information and needs that pushed them to try the diary clients’ expectations of the diary 2 usefulness/ advantages/ disadvantages 1. the diary’s impact on healthcare delivery 2. the advantages of using the diary to manage patients’ health 3. the disadvantages of using the diary to manage patients’ health 1. did the diary meet community health workers expectations? 2. the diary’s role in improving healthcare delivery 3. the advantages of using the diary to manage patients’ health 4. disadvantages of the diary 1. advantages and disadvantages of the diary for clients 3 early adopters/ innovativeness 1. the new zealand market and where it is heading 1. community health workers’ attitude towards trying out new products in general 2. community health workers’ motivations to consider the diary 3. clients’ characteristics and motivation from the perspective of their community health workers 1. clients’ motivations to use the diary and attitudes in term of trying out new products 4 chronic diseases 1. the niche markets for the diary and characteristics of person who would most benefit from it. 1. characteristics of clients that would most benefit from the diary from community health workers’ point of views 1. characteristics of the person that would most benefit from using the diary from the clients’ perspective 5 security and privacy 1. community groups’ general attitude toward using the diary, privacy and security concerns, and any other concerns from the diary provider’s participants’ points of views. 1. patients’ general attitudes toward and concerns about the diary from community health workers’ points of view australasian journal of information systems ghandour & ghandour 2019, vol 23, research on health information systems using personally controlled health records 12 # theme the diary provider community health workers patients/clients 6 interoperability 1. is integrating the diary with an ehr going to produce better results from the perspective of the diary provider? 1. is integrating the diary with ehr going to produce better results from the perspective of community health workers? 1. is integrating the diary with ehrs going to produce better results from the perspective of clients? 7 improvements/ usability/ recommendations 1. the overall impression of the community groups’ workers about the diary from the diary provider’s points of views 1. community health workers’ overall impression of the diary and any other concerns 2. recommendations to make the diary better from community health workers’ perspective 3. would community health workers recommend the diary for other community organizations? 1. clients’ overall impression of the diary table 4: major themes and the interview guiding questions for each group australasian journal of information systems ghandour & ghandour 2019, vol 23, research on health information systems using personally controlled health records 13 another problem the diary has overcome is the ability to solve (which was pointed out by participants) the problem of communication among caregivers or clinicians. the diary puts them all on the same page and improves the sharing of health data between them, which means they could all work together toward better health for the patients. they believe that the diary connects desperate data silos across all layers of healthcare. the second group; a number of community health workers, all confirmed that they normally keep their patients’ health information electronic. only a few indicated that they use paper systems besides the electronic one. interviewees from this group also identified the needs that made them consider something such as the diary. this is due to the fact that the patients whom the community organization is serving are hard to deal with, as one participant stated: “we were not sure about doing that at first because we have hard enough time getting our own clients to engage with us. but i still wanted to use it because it has a lot of potential for the future, but it is not able to be used for our own agency’s clients at the moment.” another interesting point was raised by one of the participants, who believed in the existence of inequalities in health and looked at the diary as a tool that could reduce such inequalities and gaps in health. “i believe there have been massive gaps between the health of people with mental illnesses and intellectual disabilities, and the health management it. as a woman if you have a mental illness you will die 8 years earlier than a woman that doesn’t have a mental illness in nz. as a male if i have a mental illness, i’ll die 15 years earlier than somebody who doesn’t have a mental illness.” they finally identified the desired outcomes they expected from the diary. many participants believed in the diary as a tool that could help clients to analyse their lifestyle, empower patients to manage their own health information, show greater interest in their own health, and help them with their health behaviour especially, their physical health: “the diary is a tool that could help patients realizing themselves and their habits” one participant believed that it is a useful tool for staff such as support workers to use on behalf of the patients. this is “because some people are quite apathetic about their health”. this participant also expected a few proactive clients to take control and be able to fill in the diary without the need for support. some of these potential outcomes include: • patients ability to manage their own health • caregivers ability to track individual’s health development • patients are helped to become more aware of themselves • patients financial problems are solved • patients and especially mental illness patients are enhanced to become aware of their physical and mental health status. • patients are potentially encouraged to self-manage and show greater interest in their own health. • could reduce inequalities and gaps in health. australasian journal of information systems ghandour & ghandour 2019, vol 23, research on health information systems using personally controlled health records 14 the third group, clients/patients were interviewed showed their level of excitement about having more access to their health information. levels of excitement differed from one client to another; some were keen, and some were not interested at all. then, they were asked about their expectations of the diary. almost all participating clients did not have any expectations of the diary, they were all bored on the day that they tried it, and all wanted to do a new activity. “no expectations really, just something different to do that day” the general impression from clients was that they had been asked to use the diary from their health workers. some of the participating clients did not seem to have any basic knowledge about what the role of the diary in improving their health is. 4.2 theme two: usefulness/ advantages and disadvantages this theme examined the opinions of the sixteen key informant interviewees regarding the advantages that the diary could offer, as well as any disadvantages that may impact users. all participants have identified various advantages that the diary has strong potentiality in delivering better health outcomes. a few disadvantages were also identified by some of the interviewees. participants from the diary provider’s group all agreed strongly that the diary improves healthcare delivery. for one, the diary joins fragmented health data and puts it all in one central place. therefore, it saves clinicians’ and caregivers’ time and provides them with the information they need in a shorter amount of time, requiring less effort. it also enhances communication between patients and their caregivers; it puts them all on the same page, and helps them understand each other’s needs: “it can improve healthcare delivery by making siloed data more accessible and useful” “by joining disparate data silos and improving communication among all the members of a mdct” participants from this group stressed the value of joining all health information in one place; this unique feature helps health professionals make better clinical decisions and helps patients get involved in decision making: “in terms of healthcare delivery that better background allows the healthcare team to make better decisions with more comprehensive information”. they also listed many potential advantages that the diary provides. some of these potential advantages and ways of improving health delivery were: • saving the time of clinicians or caregivers • enhancing communications between patients and their caregiver • enhancing self-management and empowerment • helping health professionals to make better clinical decisions • helping patients to take part of the decisions making. • providing more comprehensive background of patients australasian journal of information systems ghandour & ghandour 2019, vol 23, research on health information systems using personally controlled health records 15 • allowing better diagnosis for patients, maybe better health outcomes at lower costs • having access from anywhere at any time in addition, this group of participants also recognized usability issues to be the main barrier for the diary. they also felt that the disadvantages of the diary in general include the amount of time taken to fill it in, and the motivation of either caregivers or patients themselves, this is a disadvantage not only for the diary in particular but also for any other pchr/diary. “yes, it requires a lot of data to be gathered and, unless much of that can be automated, many users will not bother with it” as for the community health worker, the diary did not meet most of their expectations, as they stated below: “no, it didn’t meet my expectations. however, i still see a lot of potential for it in the future”. “i found it very hard to use. the current form that it is in is frustrating”. “no it did not, unfortunately. i’m looking forward to seeing the improved version of the app though”. “no not really”. for the majority of this group of participants, one major problem was the difficulty of engaging their patients with the diary. the reason for this difficulty was not only because of the usability issues but also because of the clients and the community health workers themselves. participants felt that the diary interfered with their relationship with their patients, as they spend so much time showing them how to use the diary rather than talking about the patients’ health. also, getting patients to work with the diary was very hard which is: “because we work with such a high need population of people that is already hard to engage in itself, so the engagement needs to occur with us” although the diary only delivered a few real advantages for some of the community health worker interviewees, they all believed that it has strong potential in delivering many advantages in the future, such as; encouraging people to improve their health status, helping them understand their health status and educate them on how to improve it, helping the clients’ to become independent and self-managed, and improving patients’ relationship with their health professionals. yet, this group of participants (community health workers), also believed that the diary could not be beneficial and may actually provide disadvantages rather than advantages for community health patients specifically as follows: • these clients are usually not self-motivated and not interested in improving their health. • they usually rely on their health professionals and are not proactive at all. • they are hard to work with and do not engage with their health; their health professionals struggle in making them attend meetings or appointments. • most clients of community organizations do not have computer literacy. • most of them are low socio economic which means they are usually not able to afford computers, internet or even cell phones. australasian journal of information systems ghandour & ghandour 2019, vol 23, research on health information systems using personally controlled health records 16 • put an extra load on health professionals and could interfere negatively with the relationship rather than positively. the third group clients/patients generally did not understand what benefits the diary could offer for them, they did not have enough education about it and about the advantages they might achieve from using it. most of the clients were asked to use it by their support workers and did it as a result of boredom or in order to get the free gp voucher. at the same time, some other patients were interested in computers and willing to learn more and that is what attracted them to the diary. so, having the correct motivation to use the diary or pchr is important for update and sustainability. 4.3 theme three: early adopters and innovators this theme observed the different personalities of each interviewee. it examined the connections between using the diary and being an early adopter or an innovator. the first group of participants shared their opinions on the new zealand market’s direction and whether it is heading toward tools such as the diary. all interviewees from this group believed that new zealand is heading in the same direction as the diary, toward patientcentric and integrated systems. “the national health it board of new zealand has three priorities for health during the next 10 years, and the first one is making data standardized, the second one is making the data shared, and the third objective is having a patientcentric health record for every new zealander in 2014”. “the national health it board has stated that they want to have a patient portal in 2014, which is our diary if not something similar to it. everyone in new zealand will have access to their own health record by then”. “apps such as this are the first step on the path to a patient portal because they tie together clinicians and support workers but, in the long run, the patient will become in control of their own health”. whilst the second group of interviewees described themselves in terms of following innovation and new products. they also expressed their opinions on the clients who tried the diary and gave descriptions of the personalities of those clients. community health workers all seemed attracted to using new products and to be the first to use such products. the seven participating community health workers expressed different drivers and motives for their interest in the diary; some were interested in their own health, some were interested in finding something to improve their patients’ health and thought of the diary as a way of doing so, while others were just interested in new technology and wanted to try anything new. “100% open to new ideas and new thinking, especially around technology.” “i’m always curious about new things and always give them a try especially if it enhances my physical or mental health” “i’m always into using new technology” the participating community health workers also identified different motives that their clients might have had for using the diary, the following is a list of these different motives: • being proactive about their own health and wanting to learn more about it. australasian journal of information systems ghandour & ghandour 2019, vol 23, research on health information systems using personally controlled health records 17 • being keen on talking about their health problems. • curiosity about computers. • free gp voucher • interest in doing work with other people and work on projects. • being in a good relationship with their support worker. • being in a good mood on the day that they were asked to try the diary. the last group of interviewees also described their personalities and their motivations to use the diary. participating clients had different personalities and motives. most of them, even the clients who did not participate in the interviews, seemed to have a good relationship with their support workers and that is what mostly got them to give the diary a try. some of them also seemed to have an interest in learning more about computers. “keen about trying things out” “i’m interested in new things” “i’m pretty good, not with everything though it depends” “i’m okay” 4.4 theme four: chronic diseases in this theme, the opinions of the sixteen key informant interview participants concerning the type of person who would suit and benefit from using the diary the most were examined. each group of participants were asked different sets of questions to suit their role. all participants from all groups identified the characteristics of the person that is intended to benefit from the diary the most. the diary provider’s interviewees identified different niche markets for the diary and all agreed on that the diary was not designed for chronic illness patients only, although they are intended to benefit from it the most. as stated by participants below: ”the niche markets at the first instance are those who are costing the health market the most, these people with multi medications, chronic ill people such as diabetics, people who are difficult to manage because they are under a multi care basis, and people with mental health disabilities”. “the niche markets are chronic disease patients in the community, nursing and rest homes, community pharmacies, and community health centres”. participating community health workers had different opinions on who the diary could suit; some believed that the diary would not suit people who take many medications and have multiple health problems. this is due to the fact that “if those patients were interested in their health in the first place, they would not have got themselves into a mess with their health in the first place”. hence, those patients would not be proactive enough to track their health on the internet. “i can think of a lot of clients that we have that we didn’t suggest the diary for, because we knew it is hard enough to engage them to even taking their blood sugar or changing their lifestyle”. australasian journal of information systems ghandour & ghandour 2019, vol 23, research on health information systems using personally controlled health records 18 some believed that the diary could suit mentally ill clients as some of these clients are capable and aware of their own conditions and know how to manage them. on the other hand, mentally ill clients could also experience memory loss which means it is hard for them to remember things in the past or remember logins and passwords; this was the case with most of the mentally ill clients; they struggled with memorising logins and passwords which was part of the reason for them not to come back to the diary. in the third group one participating client felt that someone who is familiar with computers would most benefit from the diary, while another participant felt that anyone could get benefit out of using it. one other participant thought parents suit using the diary best. one participant believed that it suits mentally or physically depressed people, “as the diary helps you express yourself and have a clearer picture of your body. it puts you in the right path and helps you get better quickly. sometimes people rely on something (some type of food) and ignore many other things, the diary makes them realize these things and get more aware”. in conclusion, all participating community health workers, and the diary provider’s believed that the diary would only suit those who are interested in their own health, self-motivated clients, and those who are really keen on finding a way to improve their health. some of these people could be those who go to the gymnasium regularly. also, something like the diary would only suit clients who can afford internet and computers at their homes, and who have a level of literacy about computers and internet. 4.5 theme five: security and privacy in this theme, the privacy and security concerns in general, and attitudes of the sixteen key informant interview participants on the diary were observed. a moderate level of concern about privacy and security was observed. yet, the main concern for all participants from all of the groups was the difficulty of using the diary. group one’s (the diary provider) participants reported that 0% of community health workers showed concerns about privacy and security. “those who needed it didn’t really care and were happy to share their information, as those patients believed that the diary was supposed to help them get better” “the diary has been greeted with great enthusiasm. privacy and security issues have been discussed, but have not been one of the larger issues, populating the diary has been the largest issue”. “generally, in nz patients want their data to be shared with their healthcare providers, they believe it is a very good thing to do. patients usually only need to know that the company is doing everything they could to protect their data, they tend to be more open minded and believe in the benefits of the diary”. while, group two (community health worker) reported that 15% of patients expressed concerns about privacy and security. surprisingly, clients were not concerned about privacy and security and their general attitude toward the diary was positive; as one participant indicated: australasian journal of information systems ghandour & ghandour 2019, vol 23, research on health information systems using personally controlled health records 19 “those clients have been in the health system for so long, they are used to revealing their health information to their health professionals”. yet, the problem came after they tried to use the diary especially if it was on their own. as the diary was a beta unfinished version and had technical and functional issues, it was easy to put some clients off. clients also had concerns such as: • the diary takes too long to be filled in. • the diary uses the medical terminologies for health problems and for medications: “frustration around entering their medications; not finding the right name/brand for their medication, or they are not being there”. • the diary also has limitations on dosage selections; as well as limitations on the selection of minerals and vitamins. another significant struggle faced by almost every client especially those with mental illnesses was that it was hard for them to remember their login details, which also put most of them off and made it harder to convince them to come back for the diary. one other issue that was raised by one participant is that the clinical team usually has trouble engaging with their clients; they normally have enough difficulty trying to make them turn up to appointments or to convince them to make changes in their lives. thus, asking clients to work with the diary was really stressful for the clinical team especially as some of them felt over-protective of their clients and did not want to get them involved in something that they were not confident in, themselves. another issue that was blocking the way for those clients was that most of them are low socio economic, they do not have internet and computers at home as they do not have enough money to afford them. “so asking them to come in here (to the agency) to sit alongside us to put data on this website that they don’t really care about was just too big for us”. in regard to privacy and security, community health workers generally were comfortable. “paper based files are not as safe; they could just sit on someone’s desk and anyone could read them, i mean they are supposed to be locked away but when you are working on them, they’re not”. yet, only one participant was worried about the data being used in the wrong way such as feeding insurance companies. “i don’t believe anything that is placed on the internet is secure”. only 25% of group three (clients/patients) participants themselves were concerned about privacy and security. all participants’ clients had good attitudes in general toward the diary. “it was good and enjoyable; it was not as overwhelming as i expected it to be. i actually enjoyed doing it”. “was good” surprisingly, three out of four clients did not have any concerns about privacy and security, only one client: australasian journal of information systems ghandour & ghandour 2019, vol 23, research on health information systems using personally controlled health records 20 “was worried of hacking, i don’t think anything on the internet is secure really, anyone can hack into it these days” no other concerns were raised by clients at all. only one indicated that she did not have concerns because support workers explained to her that if she did not feel comfortable with something being in the diary, she would be able to take it out. 4.6 theme six: interoperability the opinion of the sixteen interviewees regarding connecting the diary with ehr and other health systems was examined in this theme. in general, participants across the three groups expressed interest in the concept of integrating the diary with electronic health records. also, some of them suggested other health systems other than ehrs that the diary should also have integration with, including the pharmacy system and laboratory results. the diary provider’s interviewees seemed to have an interest in the idea of integrating their diary with ehrs and other systems. they all believed that this is how their diary should look like in the future (integrated with ehrs and other health systems), as this will provide a full source of information for clients and health professionals therefore reducing inconsistencies, time, and cost for both clients and health providers. community health providers only know a little about their clients; they only know what patients tell them. sometimes they have to chase up after doctors to get the necessary information which also does not seem to be an easy thing to do. therefore, community health workers’ interviewees expressed high value in connecting all the health systems together which would help those community organizations have much more information about their clients, and help them in moving forward as one participant indicated: “we know very little about our community-based clients here” some participants emphasised the fact that the biggest challenge with the current form of the diary is that clients especially those with mental health issues would need help to fill it in; clients need motivation to make the diary successful. therefore, participants believed that there is a great reward from integrated system; “so, for us if their information comes through and goes to their file that would be very useful, but have not said that getting them to actually do it, that is the problem” surprisingly, some participants had concerns with the diary being integrated with other systems. one participant was concerned with privacy and security when any integration happens. another participant was concerned that the integration could possibly “diminish the feeling of control; the feeling that someone is populating your own diary”. one more participant also concerned that integration might cause confidentiality problems and health professionals might not be comfortable with these community health workers being able to access their records. “not a fully integration; because medical records need to have some objectivity and distance, it contains information that the clients should not necessarily be reading”. in the third group, it was hard for the participating clients to understand the idea of integration, therefore, little information was provided from them. only one participating client seemed to understand the idea, as indicated: australasian journal of information systems ghandour & ghandour 2019, vol 23, research on health information systems using personally controlled health records 21 “yes, especially if the doctor can write me a prescription every now and then through the internet, which i can print out here at this centre” 4.7 theme seven: improvements, usability and recommendations this theme examined the impressions of, and the attitudes towards, the diary from the sixteen key informant participants. the diary provider’s group described the impressions of community health workers toward the diary, whilst, community health workers and clients described their own impressions of it. from the point of view of the diary provider’s interviewees, community groups expressed good impressions of the diary. however, they also expressed their frustration with the difficulty of using it. “they really need it and it can tremendously help them”. “they loved the concept, but had trouble working out how to easily incorporate it in their daily roles”. yet, at the same time, there was strong agreement from the five of the diary provider’s interviewees that the community groups had a huge amount of difficulty using the diary. which is in fact due to usability and design issues; the diary had a complicated user interface and many functional and technical problems. another factor which made the diary difficult was indicated to by a few participants: “there was too much manual data entry, and they had trouble engaging some patients. the benefits were too far in the future”. “especially because most of the patients don’t have computer basics, don’t have email addresses, very frustrating” also, as the diary is still a new trend which has no statistics to prove the benefits of it, it is hard to convince new community groups to use it. for the diary to be a success, especially with community groups, as agreed by all participants, the diary needs ease of usability at first. one participant suggested touch screens as they are easier to control. community health workers participants’ almost all expressed a good attitude toward the diary; they believed that it had various potential benefits. however, it needs time, improvements, and both clients and health workers to buy-in and find motivation for it to be a success. participants recommended different ideas that might help with improving the adoption of the diary: “if it needs to be more successful here, more support workers need to be involved ensuring that their clients are here on a regular basis to update their information. there are more than 50 clients here every day, and we are only 2 support workers”. “it needs to have a lot of support behind it, such as political support from the wider health sector, things like support from the pho, dhb, and national health board” “as long as clients are aware of confidentiality, and who they should share their information with, the tool is great”. australasian journal of information systems ghandour & ghandour 2019, vol 23, research on health information systems using personally controlled health records 22 also, one of the participants suggested that if the diary is going to be used by mental health patients then it needs more directions on mental health. it needs to focus on the mood, hearing voices, behaviours and sleeping habits. also, participants all strongly insisted that the diary needs a lot of functional and technical improvements, and the user-interface should be a lot easier to use. some participants thought the diary is a great tool, but it is not for community groups: “why they chose us when there are people out there much more into using the diary than our clients! they could have more people to use that our harder to reach” one participant believed that the diary is not going to be successful amongst community groups, although it has great potential. almost all community health workers did not have any concerns about the diary. only one felt that “security is the main concern, and then the ease of use”. although, all of these participants seemed to believe that the diary is a good idea, each one of them had a different point of view on how to help the diary go forward. the following is a summary of all the different recommendations that were made by interviewees: • some believed it could be successful by only improving the user interface and sorting the technical and functional issues. • some others recommended that support workers of health givers are the main drivers for the tool. they need to motivate their clients and work closely with them. • meanwhile, some others believed that the diary needs political support such as support from the district health board or the national health board. • also, some recommended that if the diary is going to be used by mental illness clients then it needs more directions on mental health. • one recommended that the diary should be used by caregivers on the behalf of the patients to start with. • “in mental health, you have got limited opportunity to get clients attention, if they give it to you then you have to maximise that opportunity. if you can’t deliver what you said you were going to deliver, then you lose people’s interest at all”. last group of interviewees was a group of clients who did not mention many concerns. only one client mentioned the problem of constantly forgetting the password. “it is good, but i keep forgetting my password, so i don’t get to fill in my information every week”. however, participating community health workers believed that it was a problem for almost every client, and they also found huge difficulty in using computers. another client felt that the diary needs more about mental health. while one other client’s main intention of using the diary was to manage their weight. in summary, large amounts of data were obtained from interviewees in regard of the main aim of this study. data was all organized into seven major themes and analysed using a australasian journal of information systems ghandour & ghandour 2019, vol 23, research on health information systems using personally controlled health records 23 thematic qualitative technique as described in the previous section. the results of the sixteen key informant interviews will be discussed in the following section. 5 discussion this study explored the adoption of pchrs in the community within three different groups, that is, the diary provider, community health workers, and clients. results revealed that, participants from the three participating groups all expressed their views from different perspectives. the diary provider group, looked at the diary from a service/business provider perspective, therefore, their opinions were more theoretical. while, the second group of participants (community health workers), saw the diary from a different angle; more real and usable than the diary provider’s and provided more realistic positive and negative opinions. the third group of participants provided mixed views; this is due to different reasons including: • clients’ lack of understanding of the diary and its role in improving their health. • clients’ lack of experience with computers and technology. • clients’ perception of their health; being passive about it. • clients’ lower level of education. findings of this study must be interpreted carefully given the limitations of participants especially (clients) sample. although efforts were made to recruit a larger sample of clients to participate, the final sample was small and consisted mainly of people who had only used the diary few times and only did so because they were asked to by their support workers. in this section the main results are presented, discussed and combined with the scene setting literature. 5.1 workflow and efficiency results show that the diary has a strong possibility in improving healthcare systems efficiency. participants, especially those from the diary provider group have considered the diary different from a normal ehr. this has been likewise supported by literature, particularly by hillestad et al. (2005). participants have also outlined various characteristics that differentiate the diary from an ehr, all in which have been supported by the literature as summarized in table 1. participants from both of group one and group two reported several problems with the current healthcare systems that the diary will possibly solve. one major issue with current healthcare systems and with community health providers in particular is that they are usually out of the loop. community health providers are facing difficulty in collecting the necessary data for their patients. it also uses up a lot of time and resources. therefore, they thought that a patient-centric tool such as the diary will not only solve the breakdown between primary and secondary care, but will also keep community providers informed and in the loop about their patients’ health. this is in line with what coiera (2006) has also reported. participants emphasised on the importance of the diary in improving communication among mdct. the diary improves communications among mdct and provides real time access to patients’ information. it captures extra information from the community and from all different australasian journal of information systems ghandour & ghandour 2019, vol 23, research on health information systems using personally controlled health records 24 caregivers and creates a more comprehensive record of patients. this allows all mdcts to work together toward better health outcomes and improve the sharing of information among them. this is one of the significant differences between the diary and a normal ehr. this solves the problem for chronic ill patients particularly as they need to be in touch with their caregivers on a daily basis. this has been likewise emphasised by fichman et al. (2011). participants have also reported many different other problems within current healthcare systems that the diary is expected to solve. they thought that the diary will engage patients with their health and will make them more aware of their health problems. the diary also is expected to encourage patients to self-manage their health and show greater interest in their health. it also empowers them and gives them control. these finding have also been discussed within the literature as shown in table 2. however, looking at the diary more practically, community health workers participants thought that the diary would actually increase their work load. this is due to clients’ lack of experience with technology and their need of support from community health workers. this echoes the findings from the literature (archer et al., 2011; detmer et al., 2008; patel et al., 2011; weitzman et al., 2009), in which that healthcare professionals concern about increased responsibility; learning new technology might have been one of the impediments for the adoption of pchr. thus, participants stressed the value of integrating the diary to ehrs and other healthcare systems, as the current form of the diary requires effort and time from community health workers to spend with clients. the current form of the diary held little value to the clients and increases workers' work load, as will be discussed in section 5.7. surprisingly, results showed that 50% of participating clients are not willing to have more access to their health information. the small sample of clients seemed to be happy with the current workflow and the limited amount of data that they have access to, as one participating client thought that there was no need to access their own health information while they can ask their doctor. this result indicates that community groups are reliant on their caregivers and healthcare providers, which creates another barrier for the diary to be used by community groups. this is opposite to what was reported in the literature (ball et al., 2007; schoen et al., 2004; tang et al., 2003). 5.2 usefulness/advantages and disadvantages results show that in theory, the diary will possibly improve healthcare delivery and efficiency in various ways including joining forms of fragmented health data all in one place and putting all caregivers on the same page. participants also spoke of improving health care delivery by providing enough background and more comprehensive information about individuals’ health, therefore, clinicians will be more able to diagnose, assess, and make better decisions regarding patients’ health. the diary also improves communication between clinicians and patients and puts them on the same page. these results in line with previous studies are summarized on (table 2). yet, again when considering the diary on a more theoretical level, participants across the three different groups identified several advantages that the diary has strong potentiality in delivering, all of what was supported by the literature. participants saw the diary as being able to save time and cost; clinicians have all the information they need at one place and duplication such as blood test duplications are avoided. this result has been likewise supported in many other findings in the literature (heather et al., 2011; d. c. kaelber et al., australasian journal of information systems ghandour & ghandour 2019, vol 23, research on health information systems using personally controlled health records 25 2008; d. kaelber & pan, 2008; steinbrook, 2008). for participants, the diary enhances the memory of the patients as it helps them remember things they would not normally be able to remember in an interview situation, this is also supported by (maeder & martin-sanchez, 2012). the diary enhances patients’ self-management, empowers them, and gets them involved in decision making, which is also in line with the literature (ball et al., 2007; fuji et al., 2012). all of these perceived advantages are consistent with those previously reported in the literature (table 2). on the other hand, participants across three groups have also identified several disadvantages of using the diary. first of which is the difficulty of using the beta-version of the diary. 80% of group one’s participants agreed that usability issues are the biggest impediments for the diffusion of the diary. also, almost all of group two and group three participants’ have listed the difficulty of using the diary as the biggest disadvantage. a second disadvantage of the diary is the time it takes to be manually populated. this issue has been likewise reported in literature as one of the barriers to adopting pchr “technical issues” (kahn et al., 2009; pagliari et al., 2007). participants especially those from group two felt that the diary interferes negatively with their relationship with clients rather than positively. this is particularly because patients who belong to community organizations are unlikely to be able to use the diary by themselves. they are also not proactive about their health and tend to rely so much on their health providers which puts an extra load of work on health workers. this issue has also been highlighted by other studies within the literature (archer et al., 2011; detmer et al., 2008; patel et al., 2011; weitzman et al., 2009). while emont (2011), suggested that pchr would best suit people with low income, findings from this project suggest the opposite. results from this project are inline more with other studies which suggested “digital divide” to be one of the barriers for adopting pchr (archer et al., 2011; detmer et al., 2008; kim et al., 2009; miah et al., 2017; patel et al., 2011; roblin et al., 2009; sciadas, 2002). individuals with low income usually cannot afford computers or internet at their own homes. participants insisted that clients, who belong to community organization lack computer literacy, are low socio economic who cannot afford computers at their own homes. therefore, they suggested that the diary would suit someone who is familiar with computers and who at least can afford them at home. clients’ prior experience with technology and their motivation about their health will always play a big role in determining their individual perception of the diary or any similar patient-centric tool. a potential disadvantage of the diary that was not previously identified in the literature was that when patients especially those with chronic or mental illnesses have access to their records. they may begin to realize their bad health habits and their health issues which might affect them negatively and make them depressed rather than helping them move forward. another disadvantage of the diary that was not discussed in the literature is that the diary uses medical terminology for medication and health problems. clients usually do not know the medical terms for their conditions or medications and that was one of the reasons to put them off. 5.3 early adopters and innovators the diary provider emphasised that new zealand is heading in the same direction as the diary, and that was the main reason for the diary to be created in a place like new zealand. this is the main motive for the diary provider group. it also appeared that some of the australasian journal of information systems ghandour & ghandour 2019, vol 23, research on health information systems using personally controlled health records 26 community health workers, especially those who had a managerial role, recognized the importance of the diary in new zealand and the government plans on launching client portals in 2014. therefore, this project is particularly timely and could provide valuable results especially in new zealand. in terms of who they envisioned as early adopters of the diary, participants seemed to believe it would be clients who are interested in their health and who are looking for new ways to improve their health regardless of their health conditions. moreover, despite the fact that the diary is potentially more beneficial to clients with chronic diseases, participants suggested that only motivated clients and community health workers might be more likely to use it, as will be discussed in the next section. weitzman et al., (2009) suggested that the age of the user may affect their attitudes towards pchrs. however, referring to results, it appeared that the age was not a significant factor in determining early adopters. on the other hand, digital divide such as social status and users’ level of education acted as important factors on their attitude toward the diary. most of participating clients had only a secondary school level of education and had no access to the internet from their own homes. this result is in line with results from other studies (archer et al., 2011; detmer et al., 2008; patel et al., 2011; roblin et al., 2009; sciadas, 2002). one of the important findings is that motivation from both patients and health workers is the main aspect for the diary to work with community groups. health workers themselves need to support and educate their patients about the diary and get them to use it on a regular basis. it appeared that most of the clients used the diary without having a basic understanding of how it might help them improve their health. 5.4 chronic diseases patient-centred approaches are expected to facilitate care coordination among patients with chronic illnesses who usually receive their care from a number of different healthcare providers and caregivers (dimarco et al., 2009; pagliari et al., 2007; steinbrook, 2008). however, findings show that the diary is best for those who have different caregivers and who take many medications only in theory and ideally. participants believed that the diary can help these types of patients only if they were interested in doing so. patients’ buy-in and motivation is the main key for the diary to be useful for clients with long term conditions. the diary would perfectly suit people who are trying to make a difference in their life and are looking for something to help them manage their health such as those who attend the gym on a regular basis. results show that the diary would work perfectly for any client who is interested in their health regardless of their age or health status. also, participants identified the niche market for the diary as clients who can afford computers and internet, and who have a level of literacy about technology (knows how to use computers). in addition, findings show that the diary has a larger potential to suit caregivers and those who are caring for others with chronic illnesses than patients who themselves have chronic illnesses. the diary could be more beneficial for clinicians and caregivers themselves to be on the same page. it collects all the fragmented health data in one place for clinicians, which would reduce the time they need to spend on it and the effort they need to put in as well as offering comprehensive records for patients, helping them in decision making. this finding had also been emphasised in other findings in the literature (caligtan & dykes, 2011; pagliari et al., 2007; tang et al., 2006; wunder, 2009). australasian journal of information systems ghandour & ghandour 2019, vol 23, research on health information systems using personally controlled health records 27 furthermore, some participants believed that the diary could suit mentally ill clients as some of these clients are capable and aware of their own conditions. on the other hand, some others believed mentally ill clients could also experience memory loss which means it is hard for them to remember things in the past such as logins and passwords. in fact, this was the case with most of the clients; they struggled with memorising logins and passwords. moreover, those patients will most likely require assistance while working on the diary which will increase the workload of their health workers as discussed earlier. 5.5 security and privacy what was striking was that participants did not express many concerns with respects to security and privacy. only one community health worker believed that nothing on the internet is secure. however, all other participants simultaneously believed that these issues could easily be overcome, and are not initiating an issue nowadays; as evidenced of their point, they used the examples of online banking and the fact that paper records could also be affected by malicious access. also, as the diary complies with new zealand’s regulations, community groups especially on the management level were more confident with it. this is opposite to detmer et al. (2008)’s findings. on the other hand, in line with findings from (walker et al., 2009; weitzman et al., 2009); it appeared that the desire to have a full record of health information in an emergency situation fades away any concerns over privacy and security from those with chronic illnesses. 5.6 interoperability participants across the three different groups expressed a high value in integrating all the health systems together. they believed that integrated systems would help those community organizations have much more access to the necessary health information for individuals, which will also reduce cost and effort ultimately. they also believed that if the diary was integrated to the ehr and to all other health care systems it would have been easier. the diary as it is a stand-alone takes a lot of time to complete and is an error prone process. this has likewise been emphasised by other studies (archer et al., 2011; detmer et al., 2008; kim et al., 2009; tang et al., 2006). nevertheless, only a small percentage of community health worker participants were concerned with the diary being integrated to the ehr and that is for different interesting reasons. one of the significant reasons which were not previously discussed in the literature is that when someone else has access to your record. it diminishes the feeling of empowerment and ownership. also, patients should not know about everything that their health professionals write about them; it could be too revealing and health professionals might not feel comfortable with it. 5.7 improvements, usability and recommendations the current form of the diary was deemed not user friendly for almost all participants; most of whom were disappointed by the amount and the type of data that they needed to enter themselves. also, the registration process was laborious, took very long, and was complicated especially for clients. as well as fixing the technical issues with the diary and creating a more user-friendly interface, one of the necessary things to accelerate uptake of something like the diary by community groups would be it skills training along with providing them with wider access to computers and internet. another important factor to make the diary more successful with community australasian journal of information systems ghandour & ghandour 2019, vol 23, research on health information systems using personally controlled health records 28 groups is education. clients need to understand the anticipated benefits of the diary and how it might help in improving their health, not only the need to know how to use it. moreover, the diary needs motivation not only from the clients but also from clinicians and support workers. they need to motivate their clients and work closely with them. some suggested touch screens, as almost all clients struggled with using the mouse and they keyboard. 6 conclusions in summary, there was widespread consensus from almost all participants about the potential future value of the diary. they identified several advantages as well as challenges associated with adopting the diary. our key findings suggest that the diary suits motivated clients such as those who attend the gymnasium regularly despite their health status or age. also, the diary suits those who at least have basic knowledge about computers and internet and who can afford them at home. furthermore, there is a great value in integrating the diary with ehr and other health-related systems, otherwise the diary takes up a lot of time and effort and might be an error-prone process. furthermore, a number of factors will likely accelerate uptake, more widespread use of patient-centric tools were suggested by participants as discussed in the previous section (5.7). also, the following is a summary of the current barriers for clients to use the diary in which need to be considered in future development: • cost: community groups are low socio economically who cannot afford computers and internet at their own homes. • lack of access: limited number of computers available at the community centre & having no other place to access the internet from. • lack of computer literacy: do not have the computer literacy necessary to be able to use the diary. • typing problems • cognitive difficulties • community groups are hard people to deal with; health workers have enough trouble trying to get their patients to attend appointments. • community groups are usually not interested in improving their health and are not usually self-motivated. this project had several limitations. for one, the sample size was small in general. this is due to the fact that the topic of this project is a new and innovative area that not only did the literature not have many studies about, but also not many people are using it especially locally. although efforts were made to recruit more participants from the third group (clients/patients), the sample ended up very small. however, it was not possible to recruit any more participants from the other two groups. even though the sample size was small, interviews allowed in-depth conversation with participants unlike a survey and the themes were satiated. australasian journal of information systems ghandour & ghandour 2019, vol 23, research on health information systems using personally controlled health records 29 another limitation of this study was that the diary was beta unfinished version. that did not only affect community groups’ general attitude toward the diary, but it also was the reason for many clients to refuse taking part of the interviews. that is for a few reasons including: • clients struggled so much with using the diary and found it extremely hard to use. • clients lack computer literacy. they found it hard to take control of the mouse and the keyboard. • clients felt that the diary takes so long to be populated manually. • clients lack of education about the diary and how it might help them. although, the diary was a beta-version it was enough for participants to see how it works as it was almost a completed version. also, using the beta-version would help the provider to use the end users’ comments about the diary and improve it. also, as the results revealed, community clients are not willing to change the way they receive health information and are not looking for new ways to manage their health. this research could be expanded in the future and new directions could be pursued by: • recruiting a larger sample of participants • including other mdct members such as hospital-based clinicians and general practitioners. • including more patients with chronic illnesses and not only mentally ill patients • including healthy patients • using another pchr system other than the diary that is fully finished and easier to use. • possibly using other data collection methods such as focus groups references anoshiravani, a., gaskin, g. l., wilson, t., kopetsky, e., sandborg, c., & longhurst, c. a. (2011). implementing an interoperable personal health record in pediatrics: lessons learned at an academic children’s hospital. journal of participatory medicine, 3, e30. archer, n., fevrier-thomas, u., lokker, c., mckibbon, k. a., & straus, s. e. (2011). personal health records: a scoping review. journal of the american medical informatics association, 18(4), 515-522. ball, m. j., carla smith, n., & bakalar, r. s. (2007). personal health records: empowering consumers. journal of healthcare information management, 21(1), 77. beard, l., schein, r., morra, d., wilson, k., & keelan, j. (2012). the challenges in making electronic health records accessible to patients. journal of the american medical informatics association, 19(1), 116-120. berg, bruce lawrence. (2004). qualitative research methods for the social sciences (vol. 5): pearson boston. australasian journal of information systems ghandour & ghandour 2019, vol 23, research on health information systems using personally controlled health records 30 blake, jacqueline, & kerr, don. (2010). development of an online sleep diary for physician and patient use. knowledge management & e-learning: an international journal, 2(2), 188202. bourgeois, f. c., taylor, p. l., emans, s. j., nigrin, d. j., & mandl, k. d. (2008). whose personal control? creating private, personally controlled health records for pediatric and adolescent patients. journal of the american medical informatics association, 15(6), 737-743. braun, virginia, & clarke, victoria. (2006). using thematic analysis in psychology. qualitative research in psychology, 3(2), 77-101. doi: 10.1191/1478088706qp063oa caligtan, c. a., & dykes, p. c. (2011). electronic health records and personal health records. seminars in oncology nursing, 27(3), 218-228. chen, catherine, garrido, terhilda, chock, don, okawa, grant, & liang, louise. (2009). the kaiser permanente electronic health record: transforming and streamlining modalities of care. health affairs, 28(2), 323-333. coiera, enrico. (2006). communication systems in healthcare. clinical biochemist reviews, 27(2), 89. connecting for health. (2003). the personal health working group: final report: markle foundation. detmer, d., bloomrosen, m., raymond, b., & tang, p. (2008). integrated personal health records: transformative tools for consumer-centric care. bmc medical informatics and decision making, 8(1), 45. dimarco, c., wiljer, d., & hovy, e. (2009). self-managed access to personalized healthcare through automated generation of tailored health educational materials from electronic health records. association for the advancement of artificial intelligence, 4–7. dimick, chris. (2008). a cost-benefit model for phrs. journal of american health information management association, ahima. . emont, s. (2011). measuring the impact of patient portals: what the literature tells us. california healthcare foundation. ennis, l., rose, d., callard, f., denis, m., & wykes, t. (2011). rapid progress or lengthy process? electronic personal health records in mental health. bmc psychiatry, 11(1), 117. fichman, robert g, kohli, rajiv, & krishnan, ranjani. (2011). the role of information systems in healthcare: current research and future trends. information systems research, 22(3), 419428. fuji, k.t., abbott, a.a., galt, k.a., drincic, a., kraft, m., & kasha, t. (2012). standalone personal health records in the united states: meeting patient desires. health and technology, 2(3), 197-205. greenhalgh, t., hinder, s., stramer, k., bratan, t., & russell, j. (2010). adoption, nonadoption, and abandonment of a personal electronic health record: case study of healthspace. bmj: british medical journal, 341, 11. australasian journal of information systems ghandour & ghandour 2019, vol 23, research on health information systems using personally controlled health records 31 heather, c.o'donnell., patel, v., kern, l. m., barron, y., teixeira, p., dhopeshwarkar, r., & kaushal, r. (2011). healthcare consumers' attitudes towards physician and personal use of health information exchange. journal of general internal medicine 26(9), 1019-1026. hiller, j., mcmullen, m. s., chumney, w. m., & baumer, d. l. (2011). privacy and security in the implementation of health information technology (electronic health records): us and eu compared. buj sci. & tech. l., 17, 1. hillestad, r., bigelow, j., bower, a., girosi, f., meili, r., scoville, r., & taylor, r. (2005). can electronic medical record systems transform health care? potential health benefits, savings, and costs. health affairs, 24(5), 1103. hoerbst, a., kohl, c.d., knaup, p., & ammenwerth, e. (2010). attitudes and behaviors related to the introduction of electronic health records among austrian and german citizens. international journal of medical informatics, 79(2), 81. hughes, t.j. (2002). marketing principles in the application of e-commerce. qualitative market research: an international journal, 5(4), 252-260. kaelber, d. c., jha, a. k., johnston, d., middleton, b., & bates, d. w. (2008). a research agenda for personal health records (phrs). journal of the american medical informatics association, 15(6), 729-736. kaelber, d., & pan, e.c. (2008). the value of personal health record (phr) systems. amia annual symposium proceedings, 2008, 343–347. kahn, j.s., aulakh, v., & bosworth, a. (2009). what it takes: characteristics of the ideal personal health record. health affairs, 28(2), 369-376. keselman, a., slaughter, l., arnott-smith, c., kim, h., divita, g., browne, a., . . . zengtreitler, q. (2007). towards consumer-friendly phrs: patients’ experience with reviewing their health records. american medical informatics association, 2007, 399–403. kim, e. h., stolyar, a., lober, w. b., herbaugh, a. l., shinstrom, s. e., zierler, b. k., . . . kim, y. (2009). challenges to using an electronic personal health record by a low-income elderly population. journal of medical internet research, 11(4). king, nigel (ed.). (2004). using templates in the thematic analysis of text: sage publications. krist, a. h., & woolf, s. h. (2011). a vision for patient-centered health information systems. jama: the journal of the american medical association, 305(3), 300. lafky, db, & horan, ta. (2008). health status and prospective phr use. amia annual symposium proceedings, 1016. levinson, wendy, lesser, cara s, & epstein, ronald m. (2010). developing physician communication skills for patient-centered care. health affairs, 29(7), 1310-1318. lober, w. b., zierler, b., herbaugh, a., shinstrom, s. e., stolyar, a., kim, e. h., & kim, y. (2006). barriers to the use of a personal health record by an elderly population. amia annu symp proc., 2006, 514–518. maeder, a.j., & martin-sanchez, fj. (2012). health informatics: building a healthcare future through trusted information-selected papers from the 20th australian national health informatics conference (hic 2012) (f. j. m.-s. anthony j. maeder ed. vol. 178): ios pressinc. australasian journal of information systems ghandour & ghandour 2019, vol 23, research on health information systems using personally controlled health records 32 mandl, k. d., simons, w. w., crawford, w. c. r., & abbett, j. m. (2007). indivo: a personally controlled health record for health information exchange and communication. bmc medical informatics and decision making, 7(1), 25. miah, shah jahan, & genemo, hussein. (2016). a design science research methodology for expert systems development. 2016, 20. doi: 10.3127/ajis.v20i0.1329 miah, shah jahan, hasan, najmul, hasan, rashadul, & gammack, john. (2017). healthcare support for underserved communities using a mobile social media platform. information systems, 66, 1-12. doi: https://doi.org/10.1016/j.is.2017.01.001 morrison, z., robertson, a., cresswell, k., crowe, s., & sheikh, a. (2011). understanding contrasting approaches to nationwide implementations of electronic health record systems: england, the usa and australia. journal of healthcare engineering, 2(1), 25-42. pagliari, c., detmer, d., & singleton, p. (2007). potential of electronic personal health records. bmj: british medical journal, 335(7615), 330. patel, v. n., abramson, e., edwards, a. m., cheung, m. a., dhopeshwarkar, r. v., & kaushal, r. (2011). consumer attitudes toward personal health records in a beacon community. the american journal of managed care, 17(4), e104. patton, m.q. (2001). qualitative research & evaluation methods (m. q. patton ed. 3, illustrated, reprint ed.): sage publications, incorporated. pyper, c., amery, j., watson, m., & crook, c. (2004). patients' experiences when accessing their on-line electronic patient records in primary care. the british journal of general practice, 54(498), 38. roblin, d. w., houston, t. k., allison, j. j., joski, p. j., & becker, e. r. (2009). disparities in use of a personal health record in a managed care organization. journal of the american medical informatics association, 16(5), 683. rudd, k., swan, w., & roxon, n. (2010). a national health and hospitals network for australia's future delivering better health and better hospitals . australia: commonwealth of australia retrieved from http://www.health.gov.au/internet/yourhealth/publis hing.nsf/content/re port-redbook/$file/hrt_report3.pdf . schoen, c., osborn, r., huynh, p.t., doty, m., davis, k., zapert, k., & peugh, j. (2004). primary care and health system performance: adults' experiences in five countries. health affairs, 23, 283-283. schutt, russell k. (2016). understanding the social world: research methods for the 21st century: sage publications. sciadas, g. (2002). the digital divide in canada: science, innovation and electronic information division, statistics canada. shaw, n. t., kulkarni, a., & mador, r. l. (2011). patients and health care providers' concerns about the privacy of electronic health records: a review of the literature. electronic journal of health informatics, 6(1), e3. silvestre, a.l., sue, v.m., & allen, j.y. (2009). if you build it, will they come? the kaiser permanente model of online health care. health affairs, 28(2), 334-344. australasian journal of information systems ghandour & ghandour 2019, vol 23, research on health information systems using personally controlled health records 33 sprague, l. (2006). personal health records: the people's choice. nhpf issue brief, 820, 1-13. steinbrook, r. (2008). personally controlled online health data—the next big thing in medical care? new england journal of medicine, 358(16), 1653-1656. tang, p. c., ash, j. s., bates, d. w., overhage, j. m., & sands, d. z. (2006). personal health records: definitions, benefits, and strategies for overcoming barriers to adoption. journal of the american medical informatics association, 13(2), 121. tang, p. c., black, w., buchanan, j., young, c. y., hooper, d., lane, s. r., . . . turnbull, j. r. (2003). pamfonline: integrating ehealth with an electronic medical record system. tenforde, m., jain, a., & hickner, j. (2011). the value of personal health records for chronic disease management: what do we know? family medicine, 43(5), 351-354. tomlinson, m., swartz, l., officer, a., chan, k. y., rudan, i., & saxena, s. (2009). research priorities for health of people with disabilities: an expert opinion exercise. the lancet, 374(9704), 1857-1862. walker, j., ahern, d.k., le, l.x., & delbanco, t. (2009). insights for internists:“i want the computer to know who i am”. journal of general internal medicine, 24(6), 727-732. weitzman, e. r., kaci, l., & mandl, k. d. (2009). acceptability of a personally controlled health record in a community-based setting: implications for policy and design. journal of medical internet research, 11(2). wunder, s.s. (2009). pursuing the promise of patient portas. healthcare financial management association. wynia, m. k., torres, g. w., & lemieux, j. (2011). many physicians are willing to use patients’ electronic personal health records, but doctors differ by location, gender, and practice. health affairs, 30(2), 266. zickmund, s.l., hess, r., bryce, c.l., mctigue, k., olshansky, e., fitzgerald, k., & fischer, g.s. (2008). interest in the use of computerized patient portals: role of the provider– patient relationship. journal of general internal medicine, 23, 20-26. copyright: © 2019 ghandour & ghandour. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems fernandes, pereira & wiedenhöft 2021, vol 25, research article information technology governance 1 information technology governance and the individual's behavior: a cross-sectional study pedro fernandes university institute of lisbon, portugal pmvfs1@iscte-iul.pt rúben pereira university institute of lisbon, portugal guilherme wiedenhöft university of rio grande, brazil abstract information technology (it) is increasingly important to organizations and has become essential to the development of sustainable business growth. it is therefore necessary to adopt it governance (itg) mechanisms to ensure better solutions, sustainable growth, and better decision-making. as employee behavior is a competitive differential, this work focuses on the behavioral expression of itg. it aims at analyzing itg institutionalization's effect on the main dimensions of organizational citizenship behavior (ocb) in portugal. ocb describes individuals' voluntary commitment to an organization, including attitudes outside of their contractual functions. a descriptive-confirmative ex post facto research was conducted through survey research to 112 employees of it-related departments and divisions from companies of all sizes in portugal. the partial least square-structural equation modeling (pls-sem) method was used to test the overall model. the general hypothesis was confirmed, showing that itg institutionalization positively affects individuals' ocbs in portuguese organizations. in practical terms, it is possible to show organizations that by implementing their itg mechanisms, they are increasing employees' ocbs and, consequently, organizational effectiveness. keywords: it governance; itg mechanisms; organizational citizenship behavior; partial least squares (pls-sem). 1 introduction information technology (it) is increasingly important to organizations and has become essential to the development of sustainable business growth. furthermore, it is almost impossible for organizations in today's global digital economy to be competitive and innovative without investing in it (itgi, 2003; van grembergen & de haes, 2018). organizations must be conscious of the importance of governing it instead of only managing it. therefore, it is necessary to involve it governance (itg) processes (brandi & malheiro da silva, 2017). while it management involves short term aspects and focuses on the management of it operations, itg deals with long term and external aspects, performing and transforming it to meet current and future demands of the business's and stakeholders' expectations (brown & magill, 1994; de haes & van grembergen, 2004; wiedenhöft et al., 2019). therefore, itg is no longer considered a "nice-to-have" but a "must-have" (webb et al., 2006; pereira & mira da silva, 2012a). australasian journal of information systems fernandes, pereira & wiedenhöft 2021, vol 25, research article information technology governance 2 itg is how organizations strategically align it with business (schwarz & hirschheim, 2003; amali et al., 2018). it is part of enterprise governance and consists of leadership, organizational arrangements, patterns, and processes that ensure the development and maintenance of effective it control and accountability, performance, and risk management. plus, itg proposes that the organization's it sustains and extends its strategy and objectives (hardy, 2006; itgi, 2003; jacobson, 2009; sambamurthy & zmud, 1999; webb, 2006). finally, it is also part of itg's role to consider the organization's cultural differences since the national culture can influence its dimensions (pereira & mira da silva, 2012a; zhong et al., 2012). some recent studies, like those of juiz, guerrero, & lera (2014), juiz & toomey (2015), luciano, wiedenhöft, macadar, & pinheiro dos santos (2016), and wiedenhöft et al. (2017), suggest adopting itg mechanisms based in two orientations. on the one hand, the most widely studied is focused on the process side, which refers to crucial it strategic decisions and monitoring key roles and responsibilities (van grembergen et al., 2004; bartenschlager & goeken, 2010). on the other hand, the focus is on the behavioral side of itg, which is related to individuals dealing with it-related decisions and activities (huang et al., 2010; tiwana et al., 2013). as weill & ross (2004) argue, itg mechanisms must encourage individuals who use it to follow the desired behavior. since it has become vital to businesses' sustainability, support, and growth, organizations increasingly depend on it (pereira & mira da silva, 2012; pereira & mira da silva, 2012b; van grembergen & de haes, 2018). however, an advantageous competitive differential lies in its employees' behavior (p. m. podsakoff & mackenzie, 1997). it is important to clearly define the encouragement given to get a desirable behavior, ensuring effective governance (juiz et al., 2014). good behavior contributes to a more consistent and aligned relationship between business and it (van grembergen, 2004), while poor human behavior can defeat the best itg institutionalization process (juiz & toomey, 2015). this research focuses on the behavioral side of itg, acknowledging its importance. organizational citizenship behavior (ocb) works as a global measure of individual behavior at work, and it is used in this study to understand individuals' behavior (graham, 1991). organ (1989, 1997) defines and reviews ocb, describing it as a voluntary commitment and those individuals' behaviors that benefit the organization, though not being part of their contractual tasks and not formally rewarded by the organization's gratification system. based on itg and ocb concepts, this research aims to understand and test the effects of portuguese culture on the relationships between them, as well as to analyze how citizenship behavior can be affected by itg institutionalization. the cultural context is one of the main problems and a source of great motivation to this research, becoming important the study of its impact on itg and ocb concepts. according to p. m. podsakoff et al. (2000), the cultural context may have distinct impacts on citizenship behavior. in comparison, according to pereira & mira da silva (2012), contingency factors influence organizations' itg implementation. one of these factors is the "regional differences," where aspects such as language, local laws, and national information infrastructures impact the itg implementation (weisinger & trauth, 2003). as a part of the latin europe cluster, the portuguese culture is centered on the dimensions of femininity, high power distance, high collectivism, and low-performance orientation australasian journal of information systems fernandes, pereira & wiedenhöft 2021, vol 25, research article information technology governance 3 (hofstede, 1989; jesuino, 2002; a. a. rego et al., 2010; hofstede, 2011). according to ferreira, braun, & sydow (2013), the portuguese culture promises to relate well with citizenship behaviors. however, kowal, keplinger, & mäkiö (2019) and armenio rego & pina e cunha (2010) indicate a lack of studies on ocb in the portuguese context, describing it as an "understudied context," arguing that most of the studies were performed in the usa, asia, and northern europe. in order to contribute to this literature gap and to study the relationship between itg and ocb in portuguese organizations, the research question is: • does the adoption of itg mechanisms affect positively or negatively the individuals' behavior in portuguese organizations concerning ocb? to answer this question and achieve the proposed goal, it is essential to carry out in-depth research. a descriptive-confirmative ex post facto research will be developed during this study, materialized in a survey to be answered by workers of it-related departments and divisions from companies of all sizes in portugal. according to pereira (2014), portuguese organizations have already begun implementing their itg mechanisms, so now is the right moment to carry out this study using the research methodology mentioned above. this article has the following structure: an introductory section that introduces the research theme and its purpose. the theoretical background section reviews the literature about ocb and itg concepts and synthesizes what has previously been studied. the next section describes the theoretical model, constructs, and hypotheses tested in this research, and is followed by a section that presents the details of the applied research method. the data analysis section includes the evaluations and explanations of the survey results using the plssem method. finally, the discussion and conclusion section presents the main research findings. it also discusses the hypothesis tests' results and limitations and offers suggestions for future research. 2 theoretical background 2.1 it governance according to de haes et al. (2020), "governance of information technology" is a relatively new subject in the academic and professional literature. some pioneers in this matter, such as loh and venkatraman (1992), acknowledged the importance of itg as part of the organization's it strategy. since then, the topic has caught authors' attention, causing a considerable growth in itg literature. despite that, authors diverge on the definition of itg, proving a lack of shared knowledge in this subject (pereira & mira da silva, 2012b). in 2004, weill and ross (2004) defined itg as a way of encouraging individuals' desirable behavior in the use of it. this definition was widely used in the academic and professional world and recognized as the behavioral side of itg. however, it has been replaced over the years by a much more complete one, which links itg to corporate governance and business. one example is the definition of itg from de haes et al. (2020): "itg is an integral part of corporate governance for which, as such, the board is accountable. it involves the definition and implementation of processes, structures, and relational mechanisms that enable both business and it stakeholders to execute their responsibilities in support of business/it alignment, and the creation and protection of it business value." australasian journal of information systems fernandes, pereira & wiedenhöft 2021, vol 25, research article information technology governance 4 commonly seen as part of corporate governance (weill & ross, 2004), itg applies corporate governance concepts to strategically drive and control it on two key issues: the value that it provides to an organization and the control and mitigation of it-related risks (van grembergen, 2004; hardy, 2006). consequently, effective implementation of itg models can guarantee the organization's needs and objectives in leadership and accountability of the management board (amali et al., 2018). effective itg ensures return on it investments and improves organizational performance. since it investments are a significant part of corporate budgets, good itg models generate business value and mitigate the associated risks (jacobson, 2009; lunardi et al., 2009; pereira & mira da silva, 2012b; juiz & toomey, 2015). with regards to it, itg helps to define a direction for and control of it operations by specifying decision-making structures, processes, and relational mechanisms (sambamurthy & zmud, 1999). itg amplifies organizational it agility when aligned with it unit choices and their departments' peripheral knowledge (tiwana & kim, 2015). considering its strategic importance, managing it is insufficient; it is necessary to govern it (wiedenhöft et al., 2019). 2.2 organizational citizenship behavior ocb is characterized as an act of social exchange offered voluntarily by workers to organizations (organ, 1988). they are spontaneous gestures of collaboration and protective actions to safeguard the organization and everything related to it (a. rego, 2002). these voluntary actions are exempt from legal or contractual prescriptions. when executed, they allow a social relationship with the organization, initiating an exchange of functional extrapolitical acts that will possibly lead to a hypothetical future social, material, or economic remuneration. they are composed of employees' social acts which benefit the employer system and may or may not get rewards from the organization (smith et al., 1983; siqueira, 2003). besides ocb, there are other similar models built and validated that allow measuring individual behavior (matias siqueira, 2002), like prosocial behavior (brief & motowidlo, 1986), civic virtue (graham, 1991), and organizational commitment (borges-andrade, 1994). however, it is possible to find significant differences between those concepts (p. m. podsakoff et al., 2000). this study uses ocb like luciano et al. (2016) and wiedenhöft et al. (2017) did, taking into account the possible relationship with the desirable behavior encouraged by corporate governance and, consequently, itg. according to organ (1989), ocb is connected with job performance because citizenship behaviors are part of the spontaneous and innovative behaviors essential for effective organizations (katz & kahn, 1978). while there is a broad consensus about ocb as discretionary behaviors of individuals helping the organization, this consensus does not apply to all the ocb domains. thus, in the early 2000s, more than 30 ocb dimensions were reported in the literature (p. m. podsakoff et al., 2000). a few years later, this number had more than doubled, and it tends to increase (n. p. podsakoff et al., 2014). such dimensions are essential for measuring distinct ocb manifestations (graham, 1991). on this basis, a summary of ocb's main dimensions found in the literature and their definitions are presented below: • altruism: voluntary actions that help others with a work problem. • civic virtue: behaviors related to the individual involved or concerned about the company's life. australasian journal of information systems fernandes, pereira & wiedenhöft 2021, vol 25, research article information technology governance 5 • conscientiousness: an excellent posture of going well beyond minimally required levels of attendance, punctuality, housekeeping, conserving resources, and internal maintenance issues. • sportsmanship: good behavior of the individual who tolerates the inevitable inconveniences and demands of work without complaint. • helping behavior: focuses on helping coworkers in their jobs when it is needed. • courtesy: individuals' discretionary behavior aiming to prevent work-related problems from happening. • loyalty: identification with and loyalty to the organization's leaders and the organizations' individuals, workgroups, and departments' local interests. • individual initiative: refers to a proactive and spontaneous effort to solve problems and improve individual and group performance. • organizational compliance: the acceptance and respect for rules and procedures in the organization. • identification with the organization: shows how the individual defends the organization's image to outsiders. • interpersonal harmony: individuals' discretionary behavior aiming to prevent workrelated problems from happening. 3 theoretical model this section formulates and discusses the theoretical model, its constructs, and the hypotheses tested in this research, providing a roadmap for the remainder of the study. the study's general hypothesis demonstrates how itg institutionalization positively affects individuals' ocbs in portuguese organizations. previous research on ocb correlates good behaviors with a set of informal individual contributions that benefit the organization (organ, 1988; matias siqueira, 2002). this type of behavior is essential to guarantee conformance with the most delicate process models and to improve non-effective process models. furthermore, bad behavior can defeat even the best process models (juiz & toomey, 2015). itg and its mechanisms, an essential part of corporate governance regarding its it-related aspirations, are fundamental to encourage desired behavior from employees towards the use of it (weill & ross, 2004; de haes & van grembergen, 2008). wiedenhöft et al. (2017) stated that the itg institutionalization process acts in ocb's antecedents, like job satisfaction and rewards perception. this argument combines the influence of itg institutionalization processes on individuals' behavior with the organization's performance, sustaining a perspective of a recurrent positive effect on the organization. on this background, this study's research model is built upon an itg institutionalization model that has been derived from previous research (see luciano et al., 2016 and wiedenhöft et al., 2017), specified in table 1, in conjunction with arménio rego's (1999) ocb model presented in table 2. this ocb model was designed and operationalized for the portuguese australasian journal of information systems fernandes, pereira & wiedenhöft 2021, vol 25, research article information technology governance 6 context and has been used in previous research (wiedenhöft et al., 2017; a. rego, 2002; a. rego & pina e cunha, 2008; a. a. rego et al., 2010). the itg institutionalization model variable definition origin itg effectiveness perception (culturalcognitive institutionalization) variable related to the cultural-cognitive institutionalization of itg, which aims to analyze if the individuals perceive the efficiency of adopting itg mechanisms. institutionalization model built based on models of luciano et al. (2016), scott (2008) and wiedenhöft et al. (2017). itg mechanisms (regulatory institutionalization) variable related to the regulatory institutionalization of itg. the individual perceives the adoption of itg mechanisms to establish rules, monitoring, and sanctions. itg structure formalization (normative institutionalization) variable related to the normative institutionalization of itg. the individual realizes the formalization of the itg structure as a normative system of impositions on social behavior, authorizing and enabling social action. table 1. the itg institutionalization model variables the ocb model variable definition origin conscientiousness variable related to an excellent posture of going well beyond minimally required levels of attendance, punctuality, housekeeping, conserving resources, and internal maintenance issues. developed and validated by arménio rego (1999) in portugal. individual initiative variable connected to a proactive and spontaneous effort to solve problems and improve individual and group performance. identification with the organization the variable that shows how the individual seeks to defend the organization's image to outsiders. interpersonal harmony variable related to behaviors that aim to prevent workrelated problems from happening. table 2. the ocb model variables this research proposes that the resulting model relates the itg institutionalization model to the four ocb variables, as presented in figure 1. it also puts forward the following four hypotheses that will subsequently be tested: h1: the itg institutionalization model positively influences an individual's ocb behavior of conscientiousness. h2: the itg institutionalization model positively influences an individual's ocb behavior of interpersonal harmony. h3: the itg institutionalization model positively influences an individual's ocb behavior of identification with the organization. h4: the itg institutionalization model positively influences an individual's ocb behavior of individual initiative. australasian journal of information systems fernandes, pereira & wiedenhöft 2021, vol 25, research article information technology governance 7 figure 1. the research model 4 research method this research is performed with a functionalist epidemiology with the aim to understand a phenomenon in a way that can generate knowledge to be used by organizations (deetz, 1996). this study is an ex post facto research with a descriptive design and a confirmatory nature. the data collection and analysis were performed as a cross-sectional study (venkatesh et al., 2013). figure 2 presents the research design. figure 1. the research design model given the nature of the problem, as well as the context and purpose of this study, a survey research method was deemed suitable since it provides the possibility to study a population sample and generalize (glasow, 2005). a survey instrument works as a measurement object that links the objectives initially defined in a study with measurable variables that will help normalize and control the data to obtain the most accurate and reliable information (visser et al., 1986; ponto, 2015). therefore, a survey instrument was developed and administered in it-related departments and divisions from companies of all sizes in portugal from june to october 2019. no specific criteria, such as size or type of industry, were established for companies to participate in this study since the study's objective was to investigate individuals' organizational behavior. the only prerequisites were that the organization was located in portugal and had itg mechanisms implemented. as for employees, everyone who worked in the company for more than a year could participate to ensure respondents' familiarity with its itg institutionalization. australasian journal of information systems fernandes, pereira & wiedenhöft 2021, vol 25, research article information technology governance 8 this research adopted the questionnaire developed and validated by wiedenhöft et al. (2017) in brazil. it was adapted to portugal's portuguese linguistic terms, but it retained the original structure to enable the comparison of results. the questionnaire and its relation to all item codes are presented in the next chapter in table 5 and table 6. many benefits can emerge from using and adapting this questionnaire, starting with the fact that both studies used the same model, which was built considering the operationalization of the variables presented in table 1 and table 2. also, this questionnaire has the advantage of having already been tested and validated. therefore, a future test will only ensure that the questionnaire works and guarantee data compliance. finally, the use of this questionnaire still opens the door to a multicultural cross-study analysis, giving the readers the possibility to understand how this method might work in other countries and how the individual's behavior changes according to its national context. the distribution channel used to deliver the questionnaire to the workers of various organizations was the internet. the internet was the best communication channel to achieve this research's objectives with such a large and geographically dispersed population. the questionnaire was placed online using google forms. for this study's purpose and since it is used in scientific projects worldwide, this tool fulfilled all the requirements regarding information security and statistical functionalities (curts, 2013). at the end of the data collection procedure, a set of 116 surveys had been completed. for ethical reasons and by the organizations' request, the questionnaires were anonymous, making it impossible to identify the workers and organizations participating in the questionnaire. table 3 shows the respondents' profiles. social-demographic variables frequency percentage (%) gender male 80 71 female 32 29 age 18 35 years 59 52 36 45 years 51 46 > 55 years 2 2 education high school 21 19 bachelor's 66 59 master's 25 22 total employees in the organization 01 – 50 7 6 51 – 100 29 26 101 – 500 57 51 > 1000 19 17 total employees in the it department 01 – 10 26 23 11 – 50 23 20 51 – 100 32 29 > 100 31 28 table 3. respondents' profile a three-phase examination was carried out to make a selection from the data. first, the author had to ensure that the survey respondents were familiar with the theme of the research. second, to guarantee the answers' reliability, the respondents should have been employed in the organization for longer than a year or have at least two years of professional experience. australasian journal of information systems fernandes, pereira & wiedenhöft 2021, vol 25, research article information technology governance 9 finally, following joseph et al.'s (2014) statements, the data must be verified to exclude missing values, suspicious response patterns, and outliers. at the end of the data cleansing procedure, 112 good results were obtained, corresponding to 97% of the surveys. 5 data analysis this section evaluates the survey outcomes, presenting the results of the hypotheses proposed in the previous section. the partial least square (pls)-structural equation modeling (sem) (pls-sem) method was used to test the overall model. as the kolmogorov-smirnov test shows, it is not possible to determine the normality of the sample, so the pls-sem method was chosen because it is the one that best suits the needs of this study. for example, pls-sem fits well when sample sizes are not too large, the data are nonnormally distributed, and the purpose of the study is to explain the relationships between multiple variables in complex models (hair et al., 2011; hair jr. et al., 2014, 2018). figure 3 presents the three types of analysis and the tests that were performed for the study. figure 3 data analysis structure 5.1 factor analysis as stated in the previous section, a pre-test was not performed to validate the instrument for two reasons. on the one hand, wiedenhöft et al. (2017) have already validated the instrument in their study. on the other hand, the sample size is not big enough to be split in a pre-test analysis. despite this, the kaiser-meyer-olkin (kmo) and bartlett's sphericity test was run to measure the adequacy of the samples. the ibm spss (statistical package for the social sciences) statistics software was used to run the tests. according to the existing literature, a kmo statistics test should be above 0.5 (field, 2009). as table 4 shows, for the analysis, kmo = 0.858 ('great' according to field (2009)) and for bartlett's test of sphericity, the approximate chi-square is 2635.842 with 528 degrees of freedom, which australasian journal of information systems fernandes, pereira & wiedenhöft 2021, vol 25, research article information technology governance 10 is significant at this level (sig. < .05 according to field (2009)). therefore, it is credible to conclude that the data is appropriate for factor analysis. kaiser-meyer-olkin measure of sampling adequacy 0.858 bartlett's test of sphericity approximate chi-square 2635.842 degree of freedom 528 significance 0,000 table 4. kaiser-meyer-olkin and bartlett tests 5.2 measurement model analysis to evaluate the reliability and validity of the items and the constructs in the measurement model, the author used smartpls software, a friendly and prominent path modeling tool for pls-sem applications (wong, 2013; ringle et al., 2014). the models were built with this software, and it was used to verify the discriminant and convergent validity and the item and composite reliability. to simplify the interpretation of the model, table 5 and table 6 (see below) explain the variables used. the first-order model was constructed by linking each of the three latent variables of the itg institutionalization model to the four latent variables of the ocb model. it also connects the latent variables with the several observable variables that correspond to each one of the questions in the questionnaire. finally, after loading the data and completing the model's design, the pls algorithm was calculated. the "pls algorithm – setup" was configured according to ringle et al. (2014) and wong's (2013) recommended parameters: (1) weighting scheme: path weighting scheme (2) data metrics: (2.1) mean: 0 (2.2) variance: 1 (3) maximum iterations: 300 (4) abort criterion: 1.0e-5. (5) initial weights: 1.0. figure 4 (see below) shows the results of the calculations and the first-order model of the smartpls software constructs. as ringle et al. (2014) recommend, the critical values in this section are highlighted. the next step of the analysis consists of evaluating the reliability and validity of the items and constructs in the measurement model. first, it starts with discriminant validity, aiming to ensure that latent variables are independent of each other (hair jr. et al., 2014). this study follows two methods to measure discriminant validity. one must assess its validity by examining the observable variables' cross loads (hair jr. et al., 2014). table 7 (see below) shows the discriminant validity test, considering the cross loading analysis (chin, 1998). the results of the cross loading analysis show us that most of the factorial loads of observed variables are higher in the corresponding latent variables than the others. the exception occurs in the itgm variable, in which two factors (0.642 and 0.634) are slightly smaller (±0.032 or 3.2% of variance) than the other two factors (0.666 and 0.659) from the itg_form variable. these occur between them because they are related and are a cause and effect of each other. one way of solving this was removing the lower observable indicators from itgm and recalculating the model to ensure the independence of latent variables. however, the indicated values have slight differences, and it is expected that there is some degree of overlap between the itg australasian journal of information systems fernandes, pereira & wiedenhöft 2021, vol 25, research article information technology governance 11 institutionalization model variables' factors. thus, the instrument's discriminant validity was confirmed, considering the cross loading criterion (chin, 1998). table 5. explanation of the itg institutionalization model variables dimension question variable question intro question item it g in st itu tio na liz at io n m od el itg mechanisms (itgm) itgm_q1 regarding the adoption of the it governance model, i believe the organization i work for has: it structure or steering committee. itgm_q2 formalized it organizational structure. itgm_q3 risk analysis structure or committee. itgm_q4 structure or committee for managing it projects. itgm_q5 it performance indicators. itgm_q6 sets of it management, control, and evaluation practices. itgm_q7 set of practices for information security. itgm_q8 methods for assessing it strategic alignment levels. itgm_q9 physical space / office for it governance team or equivalent. itgm_q10 knowledge exchange practices. itg effectiveness perception (itg_efec_per) itg_efec_per_q1 regarding the it governance mechanisms in the organization where i work, i believe they: provide customer-focused it services. itg_efec_per_q2 provide integration between systems and processes. itg_efec_per_q3 promote collaborative and knowledge-sharing networks. itg_efec_per_q4 guarantee the optimization of resources in the it area. itg_efec_per_q5 focus the it governance mechanisms on customers' needs. itg_efec_per_q6 promote integration between the different organisms of the organization. itg structure formalization (itg_form) itg_form_q1 regarding the formalization of the it governance model: the organization where i work has an it governance model. itg_form_q2 the it governance model in the organization is formalized. australasian journal of information systems fernandes, pereira & wiedenhöft 2021, vol 25, research article information technology governance 12 table 6. explanation of the ocb model variables as the validity of our model continues to advance, table 8 (see below) shows the discriminant validity test, considering the fornell & larcker (1981) criterion. fornell & larcker (1981) suggest that the square root of the average variance extracted (ave) in each latent variable can be used to prove discriminant validity if this value is higher than other correlation values between the latent variables (wong, 2013). table 8 shows that each one of the latent variables shares better variance with its associated indicators than with any other indicator, which confirms the statement in table 7 about the discriminant validity of the model. finally, the values were analyzed in order to determine the internal consistency cronbach's alpha (ca), composite reliability (cr), and convergent validity (ave). following jr., c. black, et al. (2014) recommendations, table 9 (see below) shows the values and the reference values for each of the remaining validities. dimension question variable question intro question item o c b m od el individual initiative (ocb_ii) ocb_ii_q1 regarding the performance of my coworkers, i believe that: he/she keeps informed about what is going on in the organization. ocb_ii_q2 when there are problems to solve, he/she tries to solve them before presenting them to the manager. ocb_ii_q3 when something does not function, he/she attempts to find alternative solutions. ocb_ii_q4 voluntarily, he/she attempts to improve his/her knowledge, skills, and abilities. interpersonal harmony (ocb_ih) ocb_ih_q1 he/she is always complaining about trivial matters (that is, with little importance). ocb_ih_q2 he/she creates instability in the team (with tittle-tattle and intrigue). ocb_ih_q3 when he/she has difficult or unpleasant tasks to be done, he/she tries to pass the "hot potato" on to others. ocb_ih_q4 he/she is always referring to the negative side of things more than to the positive. ocb_ih_q5 when something does not work out for him/her, he/she justifies him/herself with others' mistakes. identification with the organization (ocb_io) ocb_io_q1 he/she makes an extra effort to benefit the organization, even with personal sacrifices. ocb_io_q2 he/she thinks about his/her duties in the first place, more than about his/her interests. ocb_io_q3 when he/she discovers business opportunities for the organization (even on weekends), he/she communicates this information to the managers. conscientiousness (ocb_co) ocb_co_q1 he/she is careless and thoughtless (for him/her, it does not matter whether the work is done correctly or not). ocb_co_q2 he/she is mentally fresh and ready to work as soon as he/she arrives at work. ocb_co_q3 he/she wastes time in matters unrelated to work. australasian journal of information systems fernandes, pereira & wiedenhöft 2021, vol 25, research article information technology governance 13 figure 4. the first-order model the instrument successfully passes both discriminant and convergent validity and shows internal consistency, which means that it is suitable for this study's expectations. to make it possible to test the researched hypotheses, a second-order measurement model was carried out following the recommendations of jr., m. hult, et al. (2014) and sanchez (2013). as a result of the first-order model's calculations, the values of the latent variables itgm, itg_efec_per, and itg_form were loaded into a new data file to represent the observable australasian journal of information systems fernandes, pereira & wiedenhöft 2021, vol 25, research article information technology governance 14 variables of the new latent variable itg_inst (itg institutionalization model). figure 5 (see below) presents the second-order model. table 7. discriminant validity cross loading analysis for the first-order mode itgm itg_efec_per itg_form ocb_co ocb_ih ocb_ii ocb_io itgm 0.748 itg_efec_per 0.591 0.748 itg_form 0.684 0.360 0.968 ocb_co -0.314 -0.322 -0.376 0.874 ocb_ih -0.199 -0.313 -0.282 0.764 0.871 ocb_ii 0.508 0.552 0.481 -0.661 -0.592 0.807 ocb_io 0.338 0.332 0.258 -0.223 -0.156 0.432 0.792 table 8. discriminant validity fornell-larcker criterion for the first-order model itgm itg_efec_per itg_form ocb_co ocb_ih ocb_ii ocb_io itgm_q1 0.794 0.381 0.537 -0.162 -0.106 0.346 0.249 itgm_q2 0.674 0.386 0.457 -0.149 -0.115 0.211 0.156 itgm_q3 0.699 0.297 0.450 -0.200 -0.089 0.298 0.200 itgm_q4 0.642 0.405 0.446 -0.186 -0.067 0.214 0.195 itgm_q5 0.794 0.434 0.487 -0.177 -0.075 0.309 0.268 itgm_q6 0.855 0.528 0.539 -0.301 -0.163 0.357 0.198 itgm_q7 0.754 0.289 0.556 -0.292 -0.163 0.353 0.125 itgm_q8 0.886 0.620 0.561 -0.245 -0.135 0.430 0.352 itgm_q9 0.697 0.301 0.613 -0.290 -0.258 0.416 0.281 itgm_q10 0.634 0.605 0.414 -0.251 -0.202 0.591 0.355 itg_efec_per_q1 0.244 0.693 0.132 -0.272 -0.265 0.357 0.206 itg_efec_per_q2 0.512 0.689 0.323 -0.322 -0.249 0.457 0.159 itg_efec_per_q3 0.546 0.731 0.337 -0.186 -0.134 0.438 0.329 itg_efec_per_q4 0.546 0.777 0.301 -0.126 -0.135 0.318 0.293 itg_efec_per_q5 0.338 0.822 0.263 -0.270 -0.356 0.498 0.271 itg_efec_per_q6 0.505 0.770 0.258 -0.233 -0.215 0.360 0.241 itg_form_q1 0.666 0.324 0.969 -0.353 -0.288 0.471 0.264 itg_form_q2 0.659 0.373 0.968 -0.376 -0.258 0.460 0.235 ocb_co_q1 -0.336 -0.311 -0.370 0.874 0.657 -0.630 -0.259 ocb_co_q2 -0.177 -0.197 -0.211 0.850 0.624 -0.536 -0.098 ocb_co_q3 -0.274 -0.305 -0.362 0.898 0.712 -0.556 -0.191 ocb_ih_q1 -0.187 -0.284 -0.227 0.599 0.815 -0.474 -0.178 ocb_ih_q2 -0.151 -0.262 -0.301 0.679 0.897 -0.555 -0.102 ocb_ih_q3 -0.244 -0.298 -0.280 0.735 0.903 -0.561 -0.136 ocb_ih_q4 -0.162 -0.262 -0.193 0.616 0.857 -0.454 -0.177 ocb_ih_q5 -0.122 -0.260 -0.211 0.690 0.880 -0.519 -0.098 ocb_ii_q1 0.509 0.591 0.381 -0.567 -0.483 0.788 0.315 ocb_ii_q2 0.405 0.400 0.472 -0.449 -0.404 0.795 0.322 ocb_ii_q3 0.320 0.358 0.346 -0.543 -0.543 0.809 0.403 ocb_ii_q4 0.361 0.374 0.334 -0.571 -0.487 0.837 0.371 ocb_io_q1 0.267 0.242 0.154 -0.209 -0.147 0.418 0.827 ocb_io_q2 0.138 0.171 0.084 -0.150 -0.151 0.218 0.672 ocb_io_q3 0.340 0.333 0.304 -0.175 -0.102 0.361 0.863 australasian journal of information systems fernandes, pereira & wiedenhöft 2021, vol 25, research article information technology governance 15 cronbach's alpha composite reliability average variance extracted (ave) itgm 0.912 0.926 0.559 itg_efec_per 0.843 0.884 0.560 itg_form 0.934 0.968 0.938 ocb_co 0.849 0.907 0.764 ocb_ih 0.920 0.940 0.759 ocb_ii 0.824 0.882 0.652 ocb_io 0.720 0.833 0.627 reference values ca > 0.70 cr > 0.70 ave > 0.50 table 9. convergent validity and internal model consistency for the first-order model in order to perform a second-order model, it is necessary to re-evaluate the instrument by repeating the validations performed for the first-order model in the second-order model. thus, table 10 shows the discriminant validity test, considering the cross loading analysis (chin, 1998), and table 11 shows the discriminant validity test, considering the fornell & larcker (1981) criterion. finally, table 12 shows the convergent validity and internal consistency model following the reference values that jr., c. black, et al. (2014) recommend. itg_inst ocb_co ocb_ih ocb_ii ocb_io itg_efec_per 0.904 -0.315 -0.203 0.508 0.340 itg_form 0.793 -0.322 -0.315 0.552 0.334 itgm 0.806 -0.377 -0.282 0.481 0.261 ocb_co_q1 -0.406 0.880 0.655 -0.630 -0.257 ocb_co_q2 -0.234 0.848 0.624 -0.535 -0.097 ocb_co_q3 -0.376 0.892 0.715 -0.556 -0.192 ocb_ih_q1 -0.281 0.598 0.820 -0.474 -0.177 ocb_ih_q2 -0.287 0.679 0.891 -0.555 -0.101 ocb_ih_q3 -0.330 0.733 0.910 -0.561 -0.135 ocb_ih_q4 -0.249 0.617 0.859 -0.454 -0.176 ocb_ih_q5 -0.240 0.690 0.873 -0.519 -0.097 ocb_ii_q1 0.596 -0.567 -0.486 0.787 0.316 ocb_ii_q2 0.510 -0.451 -0.400 0.796 0.323 ocb_ii_q3 0.410 -0.544 -0.542 0.809 0.400 ocb_ii_q4 0.429 -0.572 -0.488 0.837 0.372 ocb_io_q1 0.266 -0.211 -0.150 0.418 0.820 ocb_io_q2 0.159 -0.151 -0.152 0.218 0.662 ocb_io_q3 0.391 -0.175 -0.103 0.361 0.872 table 10. discriminant validity cross loading analysis for the second-order model itg_inst ocb_co ocb_ih ocb_ii ocb_io itg_inst 0.836 ocb_co -0.405 0.874 ocb_ih -0.322 0.764 0.871 ocb_ii 0.618 -0.662 -0.591 0.807 ocb_io 0.375 -0.223 -0.157 0.432 0.790 table 11. discriminant validity fornell-larcker criterion for the second-order model australasian journal of information systems fernandes, pereira & wiedenhöft 2021, vol 25, research article information technology governance 16 figure 5. the second-order model cronbach's alpha composite reliability average variance extracted (ave) itg_inst 0.782 0.781 0.699 ocb_co 0.849 0.887 0.764 ocb_ih 0.920 0.930 0.759 ocb_ii 0.824 0.833 0.652 ocb_io 0.720 0.821 0.624 reference values ca > 0.70 cr > 0.70 ave > 0.50 table 12. convergent validity and internal model consistency for the second-order model just as the first-order model, the second-order model successfully passes both discriminant and convergent validity and shows internal consistency, which means that it is suitable for this study's purpose. 5.3 analysis of the structural model with the assurance that the items and constructs are valid, the measurement model adjustments are finished. then it is possible to start the analysis of the structural model, or in other words, the analysis of the research model hypothesis with the pearson coefficient of australasian journal of information systems fernandes, pereira & wiedenhöft 2021, vol 25, research article information technology governance 17 determination (r2), the student's t-test, the stone-geisser indicator (q2), and the cohen indicator (f2). the analysis starts with the r2 (hair et al., 2011). the r2 evaluates the predictable proportion of variance that the independent variable has on the dependent variable, which measures the model's predictive accuracy (hair jr. et al., 2014). table 13 shows the values of the r2. ocb_io 0.141 ocb_ii 0.382 ocb_ih 0.104 ocb_co 0.164 table 13. coefficient of determination (r2) cohen (1988) suggests that, for the area of social and behavioral science, values such as r2 > 0.26, r2 > 0.13, and r2 > 0.0196 are described as having a substantial, moderate, or weak amount of correlation. for example, the coefficient of determination, r2, is 0.382 for the ocb_ii endogenous latent variable, which means that the latent variable itg_inst substantially explains 38.2% of the variance ocb_ii (wong, 2013). the next step of this analysis is to test the significance and the effects of the relationships pointed out between the latent variable itg_inst and the ocb model's four latent variables. these data resulted from the smartpls software, which used the bootstrapping module to calculate the student's t-test between the original values of the data and those obtained by the resampling technique (ringle et al., 2014). table 14 presents the path coefficients and t-test values. original sample (o) sample mean (m) standard deviation (stdev) t statistics itg_inst -> ocb_io 0.375 0.394 0.079 4.755 itg_inst -> ocb_ii 0.618 0.625 0.057 10.854 itg_inst -> ocb_ih 0.322 0.336 0.080 4.021 itg_inst -> ocb_co 0.405 0.415 0.071 5.695 table 14. test of significance of the relations between itg institutionalization and ocb looking at the last column of table 14, it is possible to verify that the values of the t-test are higher than 1.96, which corresponds to p-values ≤ 0.05 (a p-value is a number between 0 and 1 that helps to determine the significance of the results (rumsey, 2006)). this means that all the relations studied are significant. finally, table 15 shows the values of the last two adjustment quality indicators of the analyzed model: the predictive validity through the stone-geisser indicator (q2) and the effect size through the cohen indicator (f2) (ringle et al., 2014). stone-geiss (q2) cohen (f2) ocb_io 0.065 0.163 ocb_ii 0.222 0.617 ocb_ih 0.069 0.116 ocb_co 0.104 0.197 table 15. the model's predictive validity and constructs effects australasian journal of information systems fernandes, pereira & wiedenhöft 2021, vol 25, research article information technology governance 18 the q2 indicator examines the model's accuracy and predictive relevance and is obtained by the blindfolding procedure in smartpls. by analyzing the q2 indicator values, it is possible to verify that the values are above zero. this shows that the exogenous construct has predictive relevance for the endogenous construct under discussion (hair et al., 2011; hair jr. et al., 2014). finally, this section analyzes the f2 indicator values that evaluate each construct's contribution to the model's adjustment. according to cohen (1988), for the social and behavioral sciences, values such as f2 > 0.02, f2 > 0.15, and f2 > 0.36 are considered to have a small, medium, or large effect, respectively. thus, all the latent variables are essential for the model's application. 6 discussion and conclusion this study is intended to analyze the effects of adopting it governance mechanisms on individuals' behavior in portuguese organizations regarding their organizational citizenship behavior. table 14 shows the existence of significant relations (p-values ≤ 0.05) between itg institutionalization and the ocb variables. this confirms the study's general hypothesis that itg institutionalization positively affects individuals' ocbs in portuguese organizations. this study's general hypothesis is composed of four confirmed hypotheses. thus, for each one of them, based on the results of this study, it is possible to make predictions that itg institutionalization will increase: (1) conscientiousness behavior, (2) interpersonal harmony behavior, (3) identification with the organization behavior, and (4) individual initiative behavior. • h1: conscientiousness behavior increased up to 41% (β = 0.405; p-value ≤ 0.05). characterized as high-power distance and low-performance orientation, the portuguese culture is affected by the process's institutionalization and structure mechanisms. as a result, portuguese employees have greater compliance with organizational rules and issues related to the organization's internal maintenance. • h2: interpersonal harmony behavior increased up to 32% (β = 0.322; p-value ≤ 0.05). this behavior is characterized by the participation of individuals sharing knowledge and experiences. due to this, it was expected to work well with the itg institutionalization, which eventually happened. • h3: identification with the organization behavior increased up to 38% (β = 0.375; p-value ≤ 0.05). this indicator supports that itg institutionalization will instigate individuals to defend the organization's name with attitudes that dignify its image in front of people from outside the organization. • h4: individual initiative behavior increased up to 62% (β = 0.618; p-value ≤ 0.05). itg institutionalization supports relational mechanisms that promote communication with other people in the workplace to improve individual and group performance. this is reinforced by portuguese culture as portugal is characterized as a collectivist and feminine society, where people value equality, solidarity, and quality in their professional lives. several conclusions can be drawn from these results. on the one hand, looking at the theoretical path, it is possible to reinforce the validity of the model proposed by wiedenhöft et al. (2017). this reinforcement is also essential to separately fortify the validity of the ocb construct proposed by arménio rego (1999) and the itg institutionalization constructs proposed by luciano et al. (2016) and wiedenhöft et al. (2017), showing the existence of a australasian journal of information systems fernandes, pereira & wiedenhöft 2021, vol 25, research article information technology governance 19 significant and positive relationship between them. on the other hand, p. m. podsakoff et al. (1997) empirically demonstrates that organizations with better ocbs show better effectiveness indicators. in practice, this contributes to portuguese organizations showing that implementing their itg mechanisms increases employees' ocbs and, consequently, organizational effectiveness. 6.1 comparing results the conclusions drawn from this study of portuguese workers from public and private companies can be compared to wiedenhöft et al.'s (2017) results, which were obtained from the analysis of brazilian civil servants, by contrasting the values of the significance tests and effects of the relationships pointed out between the latent variable itg_inst and the ocb model's four latent variables as presented in table 14. the data presented in table 16 shows that the itg institutionalization model has different effect levels on individuals' ocbs from portugal (pt) and brazil (br). with this, it is possible to see how the cultural context impacts individuals' behavior. even if there is some parallelism between the brazilian and portuguese cultures regarding hofstede's (2011) variables, the cultural context indeed affects citizenship behavior dimensions. these results show that only the values related to the dimension "identification with the organization" are similar in both cultures. this is because portugal and brazil are two collectivist countries, where employees may improve behaviors that benefit the organization rather than behaviors that benefit themselves, typical in individualistic cultures. in contrast, but consistent with the data presented, the institutionalization of itg seems to have a better impact on the other three dimensions in the portuguese context since the remaining values are lower in the brazilian context. original sample (o) sample mean (m) standard deviation (stdev) t statistics pt br pt br pt br pt br itg_inst -> ocb_io 0.375 0.3571 0.394 0.3645 0.079 0.0590 4.755 6.0541 itg_inst -> ocb_ii 0.618 0.5410 0.625 0.5443 0.057 0.0538 10.854 10.592 itg_inst -> ocb_ih 0.322 0.2091 0.336 0.2262 0.080 0.0637 4.021 3.2815 itg_inst -> ocb_co 0.405 0.2429 0.415 0.2529 0.071 0.0647 5.695 3.7525 table 16. significance of the relations between itg institutionalization and ocb in this study in portugal (pt) and in wiedenhöft et al.'s (2017) study in brazil (br) 6.2 limitations and future research while relevant, these results should be interpreted with caution since this research was subjected to some overall constraints. on the one hand, the sample size is not especially large, making generalizations difficult and precluding a pre-test on the instrument used. despite this limitation, it was possible to meet the minimum requirements of the research techniques applied in the study. on the other hand, not all relevant industries are presented in the survey, and according to the contingency factors of itg institutionalization, this could be important. therefore, further studies must try to understand the role that itg institutionalization plays in ocb. future studies may collect more results to improve the conclusions of this study. it should be recognized that itg might impact differently on individual behaviors depending on the individual's role in an organizational hierarchy. this was the second time that the model australasian journal of information systems fernandes, pereira & wiedenhöft 2021, vol 25, research article information technology governance 20 relating itg institutionalization and the ocb were used and validated. in both studies, individual initiative behavior has a higher positive effect than the other behaviors. future research using the same model should consider if and how the model applies in other countries as well as look into specific types of organizations. if this relationship between itg institutionalization and individual initiative behavior remains significant, further research should investigate the main reasons for this outcome. references amali, l. n., hadjaratie, l., & suhada, s. (2018). the influence of organizational culture in application of information technology governance. journal of information systems engineering and business intelligence, 4(1), 1. https://doi.org/10.20473/jisebi.4.1.1-10 bartenschlager, j., & goeken, m. (2010). it strategy implementation framework bridging enterprise architecture and it governance. 16th americas conference on information systems 2010, amcis 2010, 1, 379–388. https://aisel.aisnet.org/amcis2010/400 borges-andrade, j. e. (1994). conceptualizing and measuring organizational commitment. 1. brandi, l. s. n., & malheiro da silva, a. (2017). contributions to a comparative study brazil and portugal: management of information systems and technologies. prisma.com, 33, 3– 32. https://doi.org/10.21747/16463153/33a1 brief, a. p., & motowidlo, s. j. (1986). prosocial organizational behaviors. the academy of management review, 11(4), 1689–1699. https://doi.org/10.1017/cbo9781107415324.004 brown, c. v., & magill, s. l. (1994). alignment of the is functions with the enterprise: toward a model of antecedents. mis quarterly, 18(4), 371. https://doi.org/10.2307/249521 chin, w. w. (1998). the partial least squares approach for structural equation modeling. in modern methods for business research. cohen, j. (1988). statistical power analysis for the behavioral sciences. in fibre chemistry (vol. 15, issue 6). https://doi.org/10.1007/bf00544941 curts, e. (2013). using google forms. creative commons attribution, 1–12. de haes, s., & van grembergen, w. (2004). it governance and its mechanisms. isaca. https://doi.org/citeulike-article-id:9755150 de haes, s., & van grembergen, w. (2008). analysing the relationship between it governance and business/it alignment maturity. proceedings of the annual hawaii international conference on system sciences, 1–10. https://doi.org/10.1109/hicss.2008.66 de haes, s., van grembergen, w., anant, j., & huygh, t. (2020). enterprise governance of information technology. achieving alignment and value in digital organizations. in springer. deetz, s. (1996). crossroads — describing differences in approaches to organization science : rethinking burrell and morgan and their legacy. organization science 7, 191–207. ferreira, a. i., braun, t., & sydow, j. (2013). citizenship behavior in project-based organizing: comparing german and portuguese project managers. international journal of human resource management, 24(20), 3772–3793. https://doi.org/10.1080/09585192.2013.777937 australasian journal of information systems fernandes, pereira & wiedenhöft 2021, vol 25, research article information technology governance 21 field, a. (2009). discovering statistics using spss (and sex and drugs and rock' n' roll). fornell, c., & larcker, d. f. (1981). evaluating structural equation models with unobservable variables and measurement error. journal of marketing research, 18(1), 39. https://doi.org/10.2307/3151312 graham, j. w. (1991). an essay on organizational citizenship behavior. employee responsibilities and rights journal, 4(4), 249–270. https://doi.org/10.1007/bf01385031 hair, j. f., ringle, c. m., & sarstedt, m. (2011). pls-sem: indeed a silver bullet. journal of marketing theory and practice, 19(2), 139–152. https://doi.org/10.2753/mtp10696679190202 hair jr., j. f., c. black, w., j. babin, b., & e. anderson, r. (2018). multivariate data analysis. in neuromarketing in india: understanding the indian consumer. https://doi.org/10.4324/9781351269360 hair jr., j. f., hult, g. t. m., ringle, c. m., & sarstedt, m. (2014). a primer on partial least squares structural equation modeling (pls-sem). hardy, g. (2006). using it governance and cobit to deliver value with it and respond to legal, regulatory and compliance challenges. information security technical report, 55–61. https://doi.org/10.1016/j.istr.2005.12.004 hofstede, g. (1989). organising for cultural diversity. european management journal, 7(4), 390– 397. https://doi.org/10.1016/0263-2373(89)90075-3 hofstede, g. (2011). dimensionalizing cultures: the hofstede model in context. online readings in psychology and culture, 2(1), 1–26. https://doi.org/10.9707/2307-0919.1014 huang, r., zmud, r. w., & price, r. l. (2010). influencing the effectiveness of it governance practices through steering committees and communication policies. european journal of information systems, 19(3), 288–302. https://doi.org/10.1057/ejis.2010.16 itgi. (2003). board briefing on it governance. in it governance institute. jacobson, d. d. (2009). revisiting it governance in the light of institutional theory. proceedings of the 42nd hawaii international conference on system sciences. http://scholarcommons.usf.edu/etd/6469 jesuino, j. c. (2002). latin europe cluster: from south to north. journal of world business, 37(1), 81–89. https://doi.org/10.1016/s1090-9516(01)00076-1 juiz, c., guerrero, c., & lera, i. (2014). implementing good governance principles for the public sector in information technology governance frameworks carlos. 2014(january), 9–27. juiz, c., & toomey, m. (2015). to govern it, or not to govern it? communications of the acm, 58(2), 58–64. https://doi.org/10.1145/2656385 katz, d., & kahn, r. l. (1978). the social psychology of organizations. new york: wiley. https://doi.org/10.1177/003803857200600313 kowal, j., keplinger, a., & mäkiö, j. (2019). organizational citizenship behavior of it professionals: lessons from poland and germany. information technology for development, 25(2), 227–249. https://doi.org/10.1080/02681102.2018.1508402 australasian journal of information systems fernandes, pereira & wiedenhöft 2021, vol 25, research article information technology governance 22 loh, l., & venkatraman, n. (1992). diffusion of it outsourcing: influence sources and the kodak effect. information systems research, 3(4). http://search.ebscohost.com/login.aspx?direct=true&db=epref&an=isr.c.ccd.loh.d ioisk&site=bsi-live dp ebscohost db epref luciano, e. m., wiedenhöft, g. c., macadar, m. a., & pinheiro dos santos, f. (2016). information technology governance adoption: understanding its expectations through the lens of organizational citizenship behaviour. international journal of it/business alignment and governance, 7(2), 22–32. https://doi.org/10.4018/ijitbag.2016070102 lunardi, g. l., becker, j. l., & macada, a. c. g. (2009). the financial impact of it governance mechanisms adoption: an empirical analysis with brazilian firms. nd hawaii international conference on system sciences, 1–10. https://doi.org/10.1109/hicss.2009.434 matias siqueira, m. m. (2002). measures of organizational behavior. 7, 11–18. http://www.scielo.br/pdf/epsic/v7nspe/a03v7esp.pdf organ, d. w. (1988). organizational citizenship behavior: the good soldier syndrome. in the academy of management review (vol. 14, issue 2). organ, d. w. (1997). organizational citizenship behavior: it's construct clean-up time. human performance, 10(2), 85–97. https://doi.org/10.1207/s15327043hup1002 pereira, r., almeida, r., & mira da silva, m. (2014). it governance patterns in the portuguese financial industry. hawaii international conference on system science it, 96(24), 13697– 13702. https://doi.org/10.1109/hicss.2014.541 pereira, r., & mira da silva, m. (2012). a literature review: guidelines and contingency factors for it governance. european, mediterranean & middle eastern conference on information systems 2012, 2012, 342–360. pereira, r., & mira da silva, m. (2012a). designing a new integrated it governance and it management framework based on both scientific and practitioner viewpoint article. international journal of enterprise information systems, october, 191–200. https://doi.org/10.1109/edoc.2012.30 pereira, r., & mira da silva, m. (2012b). it governance implementation: the determinant factors. communications of the ibima, september 2014, 1–16. https://doi.org/10.5171/2012.970363 podsakoff, n. p., podsakoff, p. m., mackenzie, s. b., maynes, t. d., & spoelma, t. m. (2014). consequences of unit-level organizational citizenship behaviors: a review and recommendations for future research. journal of organizational behavior, 35(s1), 87–119. https://doi.org/10.1002/job podsakoff, p. m., ahearne, m., & mackenzie, s. b. (1997). organizational citizenship behavior and the quantity and quality of work group performance. journal of applied psychology, 82(2), 262–270. http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1584824 podsakoff, p. m., & mackenzie, s. b. (1997). impact of organizational citizenship behavior on organizational performance: a review and suggestion for future research. human performance, 10(2), 133–151. https://doi.org/10.1207/s15327043hup1002 australasian journal of information systems fernandes, pereira & wiedenhöft 2021, vol 25, research article information technology governance 23 podsakoff, p. m., mackenzie, s. b., paine, j. b., & bachrach, d. g. (2000). organizational citizenship behaviors: a critical review of the theoretical and empirical literature and suggestions for future research. journal of management, 26(3), 513–563. https://doi.org/10.1177/014920630002600307 ponto, j. (2015). understanding and evaluating survey research. journal of the advanced practitioner in oncology, 6(2), 168–16871. rego, a. (1999). organizational citizenship behaviors: operationalizing a construct. psicologia, 13(1–2), 127–148. https://doi.org/10.17575/rpsicol.v13i1/2.563 rego, a. (2002). ethical climates and organizational citizenship behaviors. revista de administraçao de empresas, 42(1), 50–63. rego, a. a., pina, m., & pina e cunha, m. (2010). organizational justice and citizenship behaviors: a study in the portuguese cultural context. applied psychology: an international review, 59(3), 404–430. https://doi.org/10.1111/j.1464-0597.2009.00405.x3 rego, a., & pina e cunha, m. (2008). organisational citizenship behaviours and effectiveness: an empirical study in two small insurance companies. service industries journal, 28(4), 541–554. https://doi.org/10.1080/02642060801917695 ringle, c. m., da silva, d., & bido, d. (2014). structural equation modeling with the smartpls. remark – revista brasileira de marketing, 13, 56–73. https://doi.org/10.5585/remark.v13i2.2717 rumsey, d. (2006). statistics essentials for dummies. in wiley publishing (vol. 11, issues 1 p.1446). https://doi.org/10.2741/1775 sambamurthy, v., & zmud, r. w. (1999). arrangements for information technology governance: a theory of multiple contingencies. mis quarterly, 23(2), 261–290. sanchez, g. (2013). pls path modeling with r. schwarz, a., & hirschheim, r. (2003). an extended platform logic perspective of it governance: managing perceptions and activities of it. journal of strategic information systems, 12(2), 129–166. https://doi.org/10.1016/s0963-8687(03)00021-0 scott, w. r. (2008). approaching adulthood: the maturing of institutional theory. theory and society, 37(5), 427–442. https://doi.org/10.1007/s11186-008-9067-z siqueira, m. m. m. (2003). proposition and analysis of a model for organizational citizenship behaviors. revista de administração contemporânea, 7(spe), 165–184. https://doi.org/10.1590/s1415-65552003000500009 smith, c. a., organ, d. w., & near, j. p. (1983). organizational citizenship behavior: its nature and antecedents. journal of applied psychology, 68(4), 653–663. tiwana, a., & kim, s. k. (2015). discriminating it governance. information systems research, 26(4), 656–674. https://doi.org/10.1287/isre.2015.0591 tiwana, a., konsynski, b., & venkatraman, n. (2013). special issue: information technology and organizational governance: the it governance cube. journal of management information systems, 30(3), 7–12. https://doi.org/10.2753/mis0742-1222300301 australasian journal of information systems fernandes, pereira & wiedenhöft 2021, vol 25, research article information technology governance 24 van grembergen, w. (2004). strategies for information technology governance. https://doi.org/10.4018/978-1-59140-140-7 van grembergen, w., & de haes, s. (2018). introduction to the minitrack on it governance and its mechanisms. 51st hawaii international conference on system sciences (hicss), 4877– 4879. https://doi.org/10.1109/hicss.2014.535 van grembergen, w., de haes, s., & guldentops, e. (2004). structures, processes and relational mechanisms for it governance. october. venkatesh, v., brown, s. a., & bala, h. (2013). bridging the qualitative-quantitative divide: guidelines for conducting mixed methods research in information systems. mis quarterly, 37(1), 21–54. https://doi.org/10.1080/14767430.2018.1454705 visser, p. s., krosnick, j. a., & lavrakas, p. j. (1986). survey research. webb, p., pollard, c., & ridley, g. (2006). attempting to define it governance: wisdom or folly? proceedings of the 39th hawaii international conference on system sciences, 00(c), 661– 676. https://doi.org/10.1017/cbo9780511790423.026 weill, p., & ross, j. w. (2004). it governance how top performers manage it decision rights for superior results. it governance, boston: ha(harvard business school press boston, massachusetts), 1–10. https://doi.org/10.2139/ssrn.664612 weisinger, j. y., & trauth, e. m. (2003). the importance of situating culture in cross-cultural it management. ieee transactions on engineering management, 50(1), 26–30. https://doi.org/10.1109/tem.2002.808259 wiedenhöft, g. c., luciano, e. m., & pereira, g. v. (2019). information technology governance institutionalization and the behavior of individuals in the context of public organizations. information systems frontiers, 2017, 1453–1467. https://doi.org/10.1007/s10796-019-09945-7 wiedenhöft, g. c, luciano, e. m., & pereira, g. v.. (2017). institutionalization of information technology governance and the behavior of individuals in the public organizations context. ecis, 2017, 1453–1467. wong, k. k.-k. (2013). partial least squares structural equation modeling (pls-sem) techniques using smartpls. marketing bulletin, 24(1), 1–32. http://marketingbulletin.massey.ac.nz/v24/mb_v24_t1_wong.pdf%5cnhttp://www.researchgate.net/pro file/ken_wong10/publication/268449353_partial_least_squares_structural_equation_ modeling_(plssem)_techniques_using_smartpls/links/54773b1b0cf293e2da25e3f3.pdf zhong, x., aoun, c., & vatanasakdakul, s. (2012). does culture matter? cultural influences and it governance integration mechanisms. 18th americas conference on information systems 2012, amcis 2012, 2. copyright: © 2021 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems fernandes, pereira & wiedenhöft 2021, vol 25, research article information technology governance 25 doi: https://doi.org/10.3127/ajis.v25i0.3141 1 introduction 2 theoretical background 2.1 it governance 2.2 organizational citizenship behavior 3 theoretical model 4 research method 5 data analysis 5.1 factor analysis 5.2 measurement model analysis 5.3 analysis of the structural model 6 discussion and conclusion 6.1 comparing results 6.2 limitations and future research paper title (paper title style) australasian journal of information systems poulsen, ulhaq & miah 2019, vol 23, post publication review emerging insights of health informatics research 1 miah, s., shen, j., lamp, j., kerr, d., & gammack, j. (2019). emerging insights of health informatics research: a literature analysis for outlining new themes. australasian journal of information systems, 23. doi:10.3127/ajis.v23i0.2137 review this post publication review discusses the systematic review of health informatics by miah, shen, lamp, kerr, and gammack (2019), highlighting its contribution to the field and indicating areas for further research. the reviewed article is a sound contribution to the literature on information systems (is), adding timely insights into the current state of health informatics guided by a pressing research question: “what are the current research themes in the existing health informatics studies that show promise for future knowledge growth?” miah et al. (2019) adopted a thematic analysis approach for analysing and classifying the existing literature to summarise new insights that may have future research value. it extends previous health informatics publications in this journal (burmeister, islam, dayhew, & crichton, 2015; carlson, farrelly, frazer, & borthwick, 2015; poulsen, 2018; simpson, nevile, & burmeister, 2003) and other is journals (miah, hasan, & gammack, 2017; miah, hasan, hasan, & gammack, 2017). chiefly, the study identified ten research themes regarding existing health informatics by succinctly encapsulating the emerging aspects of this multidisciplinary is subset. to comment on the study, we have identified several sub-domains underneath the authors’ themes. we have done this to critique and complement the article. that is, we identify topics which are absent in the study, but we use the authors’ themes to do so. we hope that this contributes towards achieving the authors’ aim to “determine underlying technological developments and their target application areas” (miah et al., 2019). here, we contribute additional topics worthy of further investigation as they fall into the authors’ themes. those are, blockchain (themes: digital health and ethics, privacy, security and other concerns), fog/edge computing (themes: clinical and decision support aspects and digital health), and ethical healthcare robots (themes: telemedicine, telecare and telematics and smart living). health informatics with blockchain is being done by kuo, kim, and ohno-machado (2017) and gordon and catalini (2018). health applications of fog/edge computing are being explored by kraemer, braten, tamkittikhun, and palma (2017) and barik, dubey, and mankodiya (2017). ethical healthcare robots are being investigated by vanderelst and winfield (2018), poulsen and burmeister (2019), and anderson, anderson, and berenz (2017). to briefly expand on just one of these sub-domains and demonstrate the work being done in health informatics themes identified by the authors, consider ethical healthcare robotics. anderson et al. (2017) developed and empirically tested a value-driven medicine delivery robot which upholds rossian deontological ethics and uses situational variables to make dynamic ethical decisions. similarly, poulsen and burmeister (2019) are designing ethical care robots for elders which are dynamically responsive to care values as they emerge from human rights, professional ethics, and social groups such as ageing lgbtiq+ (poulsen et al., 2019; poulsen, burmeister, & tien, 2018). overall, the systematic review by miah et al. (2019) provides invaluable insights into the current state of health informatics by using a systematic methodology and thematic analysis. here, we have shared the reader and researcher perspective, identifying a few absent health australasian journal of information systems poulsen, ulhaq & miah 2019, vol 23, post publication review emerging insights of health informatics research 2 informatics topics and complementing the findings by identifying them here using the authors’ themes. adam poulsen charles sturt university apoulsen@csu.edu.au anwaar ulhaq charles sturt university response to the post publication review the prime author of the review study (miah et al. 2019) entirely agree to the blockchain based new technological prospects in improving healthcare information services. the heath informatics field will be enriched in particular for the solution design arena, what was suggested. the blockchain technology was invented by satoshi nakamoto in 2008, nearly a decade ago (nakamoto, 2008). since then, its applications have been widely introduced and discussed in many sectors however, it is remarkable in the financial sector where the security, privacy and authorization are the vital issues. despite this, the enormous potential of the application of blockchain is still under exploration and the realisation of its benefits to many organisations and individuals are still at its emergent stages. government as well as enterprises are showing their increasing interests in adopting blockchain based technologies realising that this technology may provide them effective support and further provisions for digital transformation. the world economic forum (wec, 2018) viewed that the blockchain would make industrial revolutions due to the facts that this technology can provide data security, transparency, as well as cost savings as the economic advantages. the main characteristics of the blockchain are: it works in a decentralised manner, for wider ranges of data and their transactions ensuring authorisations and compatibilities that could offer useful provisions for designing various healthcare information dissemination solutions. consequently, many recent studies discussed solution design using the blockchain based applications in healthcare such as for health record management (kar, 2016) and how blockchains and smart contracts operate for identifying the advantages as well as highlighting the ways the blockchains and other existing technologies such as iot can be used together (christidis and devetsikiotis 2016). for improving user access management and data quality and providing access to patients for managing their own data offer options to integrate the informed consent procedure while ensuring privacy of data transactions (razak, 2018). along these lines, we anticipate that blockchain based technology will make shift to transform the ways governments, businesses and communities use it or internet enabled platform as infrastructure for information collection, processing and analyzing that ensue effective delivery of information for better strategic decision support services. to capture the future research directions in the solution design research adopting blockchain, we offer the following future aspects: • strategic decision making services: further effective and robust decision support mechanisms development would be important study for the field of health informatics, in which structured and unstructured data are to be processed such as for setting out organisational vision and strategy as well as healthcare management to become more data-driven and vibrant solution applications ( for example australasian journal of information systems poulsen, ulhaq & miah 2019, vol 23, post publication review emerging insights of health informatics research 3 through the adoption of various specialised dss applications (e.g. in knowledge based dss (miah, 2008; miah, kerr, gammack and cowan, 2008)). • information disseminations services: further effective blockchain health informatics solutions oriented studies will be important for providing information distribution services to open up various development opportunities for individuals and organisations using user-centric and data-driven strategies (examples can be viewed in agricultural information services (miah, 2012) and e-government (miah, 2010)). • analytics support services: further effective blockchain health informatics solutions oriented studies will be important for developing data surveillance, business intelligence/analytics as well as predictive analytics for improving processes and practices as well as for better use of resources. let me conclude this brief commentary with a claim that for such solution design, studies may be benefited from a robust, broader and practice-focused design methodology in place. the design science research (dsr) paradigm (e.g. miah and gammack, 2014; miah and genemo 2016) provides better support in designing and evaluating specialised solutions over the past decades therefore, it would be showing promises for designing blockchain based healthcare solutions. this pragmatism is a school of thought that considers practical consequences or realworld effects to be vital components of both practical meaning and truth (hevner, 2007). anderson (2013), cited in tagle’s e-book (2019, p.8) suggested that the pragmatic paradigm for design and development research is focused on practice which better aligns with the dsr. based on these, it is anticipated that dsr would be a suitable methodological avenue for conducting blockchain based healthcare solution design studies. shah j miah victoria university, melbourne shah.miah@vu.edu.au references anderson, m., anderson, s. l., & berenz, v. (2017). a value driven agent: instantiation of a casesupported principle-based behavior paradigm. paper presented at the aaai 2016 workshop on ai, ethics & society. barik, r. k., dubey, h., & mankodiya, k. (2017, 14-16 nov. 2017). soa-fog: secure serviceoriented edge computing architecture for smart health big data analytics. paper presented at the 2017 ieee global conference on signal and information processing (globalsip). burmeister, o. k., islam, m. z., dayhew, m., & crichton, m. (2015). enhancing client welfare through better communication of private mental health data between rural service providers. australasian journal of information systems, 19, 1-14. doi:10.3127/ajis.v19i0.1206 carlson, b. l., farrelly, t., frazer, r., & borthwick, f. (2015). mediating tragedy: facebook, aboriginal peoples and suicide. australasian journal of information systems, 19. doi:10.3127/ajis.v19i0.1174 christidis, k. & devetsikiotis, m. (2016). blockchains and smart contracts for the internet of things, ieee access, 4, 2169-3536 australasian journal of information systems poulsen, ulhaq & miah 2019, vol 23, post publication review emerging insights of health informatics research 4 gordon, w. j., & catalini, c. (2018). blockchain technology for healthcare: facilitating the transition to patient-driven interoperability. computational and structural biotechnology journal, 16, 224-230. doi:10.1016/j.csbj.2018.06.003 hevner, a. r. (2007). a three cycle view of design science research, scandinavian journal of information systems, 19(2), 87-92. kar, i. (2016). estonian citizens will soon have the world's most hack-proof health-care records. [online]. available: http://qz.com/628889/this-eastern-european-country-ismoving-its-health-recordsto-the-blockchain/ kraemer, f. a., braten, a. e., tamkittikhun, n., & palma, d. (2017). fog computing in healthcare–a review and discussion. ieee access, 5, 9206-9222. doi:10.1109/access.2017.2704100 kuo, t.-t., kim, h.-e., & ohno-machado, l. (2017). blockchain distributed ledger technologies for biomedical and health care applications. journal of the american medical informatics association, 24(6), 1211-1220. doi:10.1093/jamia/ocx068 miah, s. j., shen, j., lamp, j., kerr, d., & gammack, j. (2019). emerging insights of health informatics research: a literature analysis for outlining new themes. australasian journal of information systems, 23. doi:10.3127/ajis.v23i0.2137 miah, s.j., & gammack, j. (2014). ensemble artifact design for context sensitive decision support, australasian journal of information systems, 18 (2), 5-20 miah, s.j., & genemo, h. (2016). a design science research methodology for expert systems development, australasian journal of information systems, 20, 1-29 miah, s.j. (2012). the role of end user in e-government application development: a conceptual model in the agricultural context, journal of organizational and end user computing, 24 (3), 69-85 miah, sj (2010). a new semantic knowledge sharing approach for e-government systems, in the proceedings of the 4th ieee international conference on digital ecosystems and technologies, miah, sj, kerr, d., gammack, j. and cowan, t. (2008). a generic design environment for the rural industry knowledge acquisition, knowledge-based systems, 21 (8), 892-899 miah, sj (2008). an ontology based design environment for rural decision support, unpublished phd thesis, griffith business school, griffith university, brisbane, australia miah, s. j., hasan, n., & gammack, j. (2017). extending the framework for mobile health information systems research: a content analysis. information systems, 69, 1-24. doi:10.1016/j.is.2017.04.001 miah, s. j., hasan, n., hasan, r., & gammack, j. (2017). healthcare support for underserved communities using a mobile social media platform. information systems, 66, 1-12. doi:10.1016/j.is.2017.01.001 nakamoto, s. (2008). bitcoin: a peer-to-peer electronic cash system. consulted, 1: 2012 poulsen, a. (2018). a post publication review of “threats to autonomy from emerging icts". australasian journal of information systems, 22. doi:10.3127/ajis.v22i0.1744 australasian journal of information systems poulsen, ulhaq & miah 2019, vol 23, post publication review emerging insights of health informatics research 5 poulsen, a., allen, m., burmeister, o. k., ul-haq, a., rehman, s., & tien, d. (2019). the role of care robots in managing loneliness among lgbt elderly. the australian nursing and midwifery journal. poulsen, a., & burmeister, o. k. (2019). overcoming carer shortages with care robots: dynamic value trade-offs in run-time. australasian journal of information systems, 23. poulsen, a., burmeister, o. k., & tien, d. (2018). a new design approach and framework for elderly care robots. paper presented at the australasian conference on information systems, sydney, australia. razak, o. (2018). revolutionizing pharma — one blockchain use case at a time. medium corporation, url: https://medium.com/linum-labs/revolutionizing-pharma-oneblockchain-use-case-at-a-time-e7922c8aa9a3 simpson, c., nevile, l., & burmeister, o. k. (2003). doing ethics: a universal technique in an accessibility context. australasian journal of information systems, 10(2). doi:10.3127/ajis.v10i2.159 tagle, b. (2019). a design science research methodology guide for graduate students, e-book published in url: https://books.google.com.au/books?isbn=0359611729, accessed on 6 june 2019 vanderelst, d., & winfield, a. (2018). an architecture for ethical robots inspired by the simulation theory of cognition. cognitive systems research, 48, 56-66. doi:10.1016/j.cogsys.2017.04.002 wec (2018). building block(chain)s for a better planet. in: environment, p. a. s. w. i. f. t. (ed.) fourth industrial revolution for the earth series. wef. copyright: © 2019 poulsen, ulhaq & miah. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems lanzl et al. 2023, vol 27, research article issues regarding it consumerization 1 issues regarding it consumerization: how mixed it portfolios of private and business it components cause unreliability julia lanzl university of hohenheim, branch business & information systems engineering of the fraunhofer fit & fim research center for information management, germany julia.lanzl@fim-rc.de manfred schoch university of hohenheim, branch business & information systems engineering of the fraunhofer fit & fim research center for information management, germany henner gimpel university of hohenheim, branch business & information systems engineering of the fraunhofer fit & fim research center for information management, germany abstract with increasing mobile work due to the covid-19 pandemic, the usage and relevance of consumer it for business purposes have substantially increased. in this light, an understudied area of it consumerization, the adverse outcomes for employees using consumer it for business purposes, is of major importance. we conduct a mixed-methods study to investigate the adverse outcomes of it consumerization. we build on prior studies and end-user interviews to draw connections between it consumerization and unreliability as one known technostressor. a quantitative survey of 162 full-time employees shows that it consumerization is associated with increased unreliability. the users’ general computer selfefficacy, instead, decreases unreliability, and unreliability leads to various job-related and health-related outcomes. we show that unreliability is driven by users’ experience while trying to integrate private and business it components for business purposes. we follow up on this observation through a qualitative analysis of open-ended survey questions to detail users’ experiences. our findings emphasize the need to examine the negative outcomes of it consumerization, despite its well-studied positive effects. we suggest that organizations should strive to integrate business and private it as much as it security constraints allow for reduced technostress. keywords: it consumerization, byod, technostress, self-efficacy, integration. 1 introduction today, many it users are responsible for their entire individual information system. such individual information systems include infrastructure (e.g., wi-fi, mobile data plans), devices (e.g., smartphones and laptops), as well as applications and services that run on these devices (e.g., instant messengers) (baskerville, 2011). many times, such it portfolios involve substantial amounts of consumer it that is mobile, fast, and innovative. user experience with such privately-owned it raises the bar for business it. it has long caused employees to bring their own it into the workplace, referred to as it consumerization (niehaves et al., 2012). it consumerization has been studied widely with an emphasis on organizational advantages, such as increased innovation and productivity (e.g., bautista et al., 2018; junglas et al., 2019) australasian journal of information systems lanzl et al. 2023, vol 27, research article issues regarding it consumerization 2 and organizational disadvantages, such as security and privacy risks (e.g., gewald et al., 2017; weeger et al., 2016), as well as end-user advantages and reasons for adoption (e.g., ortbach, 2015). adverse outcomes for the end-users, however, have been studied scarcely (köffer et al., 2014). ortbach, köffer, müller, and niehaves (2013), for example, find an increasing effect of it consumerization on stress – caused by increased reachability, lack of competence, workflow changes, and system redundancies. additionally, niehaves, köffer, ortbach, and reimler (2013) find qualitative evidence that it consumerization increases, for example, the individuals’ workload, which in turn leads to stress. two recent developments emphasize a renewed need to further engage with the adverse outcome of it consumerization. first and foremost, we have experienced a substantial increase in mobile work and work-fromhome, sparked by global social distancing measures in the wake of the covid-19 pandemic. particularly the need for digital communication and collaboration has risen during the pandemic. these circumstances have forced many end-users into mobile work regardless of individual it adoption decisions. furthermore, this development has caught many organizations off guard, and without adequate it to meet the needs of an entire organization working remotely. thus, many employees have felt the need to use their private infrastructure, devices, and applications to fill those voids. this increase in involuntary mobile work will undoubtedly highlight the disadvantages of it consumerization, particularly for its end users. two known disadvantages are increased workload and technostress (niehaves et al., 2012). we expect that in post-pandemic times, we will see higher levels of mobile work and workfrom-home than we saw before covid-19 as organizations today use more and more hybrid work as a mixture of working in the office (as before the pandemic) and work-from-home (as during the pandemic). second, a recent literature review and call for research has highlighted the need to understand better the creative and innovative use of it (tarafdar et al., 2019), which has been associated with it consumerization (junglas et al., 2019). while there are certainly opportunities for future research regarding positive psychological outcomes of it consumerization, researchers also need to understand the potential adverse outcomes to view such opportunities in the right light. one aspect relevant in the context is understanding the interplay between the technologies (in this case, consumer it) and the end-user. research has suggested that both the mitigation of adverse effects (e.g., shu et al., 2011; tarafdar et al., 2015) and the fostering of positive effects (salo et al., 2018) are related to the individual capabilities of the end-users. nevertheless, organizations have a responsibility to provide their employees with adequate it. related to the current increase in work-from-home and the associated increase in consumer it use, we take a detailed look at the negative side of unreliability (i.e., the “degree to which features and capabilities provided by the technology are [not] dependable,” ayyagari et al., 2011, p. 837) when using it portfolios consisting of both privately-owned and business-owned it components, which we call mixed it portfolios. to do that, we conduct a concurrent nested mixed-methods study with a dominant quantitative strand. with this, we aim to answer the following research questions: rq1: how does it consumerization use behavior affect unreliability? rq2: what other factors drive unreliability in the context of it consumerization? australasian journal of information systems lanzl et al. 2023, vol 27, research article issues regarding it consumerization 3 our research is structured as follows: first, we conduct a qualitative pre-study with the aim to check for applicability of the topic of it consumerization and unreliability. second, the main study is designed as a mixed-methods study, including a quantitative and a qualitative survey. thereby, we develop hypotheses that we test with the quantitative survey and then analyze the qualitative data for additional insights. we find that poor integration between the privately-owned and business-owned components of a mixed it portfolio is a major driver of unreliability, leading to adverse outcomes. these outcomes include switching exhaustion, transition costs, and dissatisfaction with the it portfolio. we further find that particularly users with low computer self-efficacy are prone to experiencing such issues. in our qualitative strand, we provide concrete categories of problems that may arise and that organizations should be aware of when designing their it consumerization policies. 2 theoretical background 2.1 it consumerization an information system (is) is a socio-technical system comprising technology, information, and social artifacts (a. s. lee et al., 2015). increasingly, individuals build, administrate, and use their own individual information systems (baskerville, 2011). because of this new autonomy, individuals can bring their own it components wherever they go, including the workplace. this phenomenon is known as it consumerization – using privately-owned it components for business purposes (niehaves et al., 2012). to illustrate what it consumerization is and is not, figure 1 displays the interplay. for one thing, it can be privately-owned or owned by the employer (business-owned), and secondly, it can be used for private purposes or business purposes. it consumerization refers to the upper right corner –the use of privately-owned it for business purposes. figure 1. interplay of related research streams australasian journal of information systems lanzl et al. 2023, vol 27, research article issues regarding it consumerization 4 building on harris et al. (2012), ortbach, köffer, bode, and niehaves (2013) introduced three possible types of it consumerization: (1) the organizationally approved adoption of consumer it, which includes byod strategies, (2) the usage of consumer it which is not formally permitted by the organization, and (3) the strategic inclusion of consumer it into the organizational is landscape. the latter cannot be directly influenced by the individual and, thus, is not within the scope of this paper on the digitization of the individual. regarding (1), the introduction of byod policies in organizations is increasing and has drawn much attention from researchers (e.g., crossler et al., 2014; köffer et al., 2015; j. lee et al., 2017). thereby, policies are not restricted to devices but have been extended to other it. baskerville and lee (2013), for example, call this bring your own system, and ortbach, köffer, bode, and niehaves (2013) mentioned applications and internet services as elements of such systems. regarding (2), ortbach (2015) points out that practitioners frequently report major issues with privatelyowned it that are brought into organizations without permission. this phenomenon contributes to shadow it, which is defined as devices and systems used by employees inside of an organization without formal it department approval (behrens, 2009; silic & back, 2014). there have been many studies investigating this phenomenon of it consumerization in the past years. research has covered four areas: advantages for employees, disadvantages for employees, organizational advantages, and organizational risks (niehaves et al., 2012). regarding employee advantages, studies have focused on the antecedents of use decisions. this focus has led to a thorough understanding of why employees participate in bring-yourown-device programs or bring their privately-owned devices to work. many of these studies build upon established technology acceptance and use literature, such as utaut, tam, and tpb (e.g., bautista et al., 2018; ortbach, 2015; weeger et al., 2016). these studies find that primarily the benefits for work purposes, such as increased usefulness and ease of use, drive adoption decisions. from an organizational perspective, outcomes of it consumerization use behavior are increased productivity and work quality (e.g., bautista et al., 2018). furthermore, advantages such as increased creativity, innovativeness, mobility, and flexibility (behrens, 2009; junglas et al., 2019; ortbach, 2015; weeger et al., 2016) are associated with it consumerization. many of these advantages are advantageous for organizations and the employees themselves. lastly, indirect organizational benefits, such as increased employer attractiveness (weeger et al., 2016) and organizational commitment (doargajudhur & dell, 2019), have been added to the list of benefits. there is also a stream of research that has focused on the negative sides of it consumerization from an organizational point of view. such risks mainly include it security and data privacy issues (e.g., gewald et al., 2017; weeger et al., 2016) and a loss of organizational control (behrens, 2009). direct connections between negative consequences for the individual and it consumerization have been scarcely investigated. these include that organizational encouragement for it consumerization increases work-to-life conflict (köffer et al., 2014). additionally, there are results on the issue using different data collection methods, such as user interviews (niehaves, köffer, ortbach, & reimler, 2013), case studies (niehaves, köffer, & ortbach, 2013; ortbach, köffer, müller, & niehaves, 2013), and analyses of practitioner literature (niehaves et al., 2012). all of these results point in the direction that it consumerization indeed leads to adverse outcomes for the individual. in that regard, the mentioned studies provide evidence for the increase in stress levels when using consumer it. an observation that is closely related to australasian journal of information systems lanzl et al. 2023, vol 27, research article issues regarding it consumerization 5 research on technostress. to the best of our knowledge, no empirical studies exist that have investigated the harmful effects of mixed it portfolios concerning actual use behavior quantitatively and in-depth. 2.2 negative outcomes of it consumerization studies have previously connected it consumerization research with technostress. early on, evidence in practitioner literature regarding it consumerization hints at stress as a possible outcome (niehaves et al., 2012). technological ubiquity and blurring of boundaries between private and work life were identified as potential reasons. others found that organizational encouragement for it consumerization does indeed translate to both work overload and workto-life conflict (köffer et al., 2014). it consumerization may lead to a higher workload due to ubiquitous access, resulting in stress (niehaves, köffer, & ortbach, 2013). this was echoed in an interview study that found increased reachability, lack of competence, workflow changes, and system redundancies to be drivers of adverse outcomes of it consumerization (ortbach, köffer, müller, & niehaves, 2013). system redundancies are described as frequent changes in systems, multi-system usage, and redundancy of data. in summation, extensive research has focused on reachability with aspects such as increased work-home conflict. aspects related to the mixed it portfolio itself, such as workflow changes and system redundancies, have seen little attention. there is also little evidence of how the mentioned technological factors interplay with a lack of competence in creating the perception of stress in relation to it consumerization. yet, with the growing relevance of it consumerization, it is important to understand when and how it has adverse humanistic outcomes, such as technostress. 2.3 negative psychological effects of it use the findings regarding the dark side of it consumerization can be closely related to research on psychological stress. stress results from the interplay between environmental demands and personal resources, in which the demands tax or exceed the person’s resources (lazarus & folkman, 1984). a theoretical lens frequently employed in the context of is research is the person-environment fit approach. ayyagari et al. (2011) have adapted the theory to reflect the person-technology fit (pt-fit). it states that both technological characteristics, as well as individual resources drive the perception of stress. this lens “provides insights into how technology characteristics influence stressors” (ayyagari et al., 2011, p. 836). a lack of pt-fit (a pt-misfit) is associated with various it-related strains (i.e., negative consequences). in this paper, we thus focus on the organizational context rather than the private one. our understanding of the phenomenon thus follows the definition of technostress as “stress that users experience as a result of their use of is in the organizational context” (tarafdar et al., 2015, p. 103). technostress is a consequence of technology use (ayyagari et al., 2011). thus, various forms of is use, such as email use, have been identified as drivers of stress (stich et al., 2019). further, the effect of online social network use on technostress has been investigated (maier et al., 2015), and is use has been included as a control variable in stress research (ayyagari et al., 2011). specific demanding conditions during it use create technostress. such conditions have been frequently associated with technology characteristics (tarafdar et al., 2019), and multiple research contributions have identified such characteristics (e.g., ayyagari et al., 2011; becker et al., 2020). according to pt-misfit, demands must be met using personal resources to avoid australasian journal of information systems lanzl et al. 2023, vol 27, research article issues regarding it consumerization 6 negative consequences. the theoretical lens suggests that these primarily include individual abilities (ayyagari et al., 2011). abilities refer to skills, training, time, and energy (edwards et al., 1998). previous research in the realm of is research has identified individual beliefs about capabilities (tams et al., 2018) and self-efficacy (shu et al., 2011) as such resources. pt-fit influences stressors due to it use (ayyagari et al., 2011). much research has focused on identifying technostressors, such as invasion, overload, complexity, uncertainty, insecurity (ragu-nathan et al., 2008), and unreliability (adam et al., 2017). others have drawn connections between pt-fit and work stressors (ayyagari et al., 2011). stressors derived from such a theoretical view are traditionally associated with an inherently negative connotation (tarafdar et al., 2019). thus, they cause negative outcomes often referred to as strains (ragunathan et al., 2008; tarafdar et al., 2019). strains associated with a lack of pt-fit are plentiful: they may comprise a negative impact on the organizational commitment of an individual (i.e., how strongly an employee is involved in the organization and how strongly he or she identifies with it), the identification with the employer’s values and goals, and ultimately the commitment to the workplace (continuance commitment, i.e., an employee’s attachment to an organization) (ragu-nathan et al., 2008). further, (techno)stress has adverse effects on the individual’s health and well-being, such as increased exhaustion and burnout (galluch et al., 2015). similarly, the impact of pt-fit on is-related outcomes has been investigated. among others, these include satisfaction with is (tarafdar et al., 2010) and discontinued usage intention (maier et al., 2015). in summary, while pt-fit and it consumerization are rich research streams, the overlap has been studied scarcely. however, with the recent advent of work-from-home, consumer technologies have seen renewed interest. thus, an investigation of the overlap is both topical and relevant to organizations and employees alike. 3 mixed-methods design and pre-study 3.1 mixed-methods design our research approach consists of different steps: in a first step and as a pre-study, we conducted semi-structured interviews with the aim of an applicability check of the question at hand (i.e., the effect of it consumerization on unreliability as well as further driving factors of unreliability in this context). after that, the main study followed a mixed-method approach (venkatesh et al., 2013; venkatesh et al., 2016) with the purpose of completeness. in the quantitative strand, we tested hypotheses derived from the literature. in the qualitative stream, we derived insights based on qualitative data collected as part of our survey. study pre-study main study data collection semi-structured interviews (n = 5) structured online survey (n = 161) strand qualitative (less dominant) quantitative (dominant) qualitative (less dominant) analysis method coding pls-sem coding aim and key inference stimulus and applicability check for hypothesis development statistical hypothesis testing further assessment of quantitative findings table 1. overview of mixed-methods design this stream helped us provide a more meaningful picture and richer explanations of the phenomenon, and more detailed insights for practice. the quantitative strand is the dominant australasian journal of information systems lanzl et al. 2023, vol 27, research article issues regarding it consumerization 7 part, using a structural equation model based on survey data. the qualitative part used coding principles from grounded theory to analyze the open-ended questions (strauss & corbin, 1990) – for additional details, see section 5.3. table 1 sketches the overall design. we draw meta-inferences over both the qualitative and quantitative strands in the discussion. 3.2 qualitative pre-study existing knowledge regarding issues with workflow changes and system redundancies in mixed it portfolios is scarce. thus, in our pre-study applicability check, we aimed to understand relevant influencing factors in the context of it consumerization. for that, we used the theoretical lens of pt-fit as a basis to derive interview questions and for our analysis of the interview data. this deductive-inductive approach is congruent with content analysis which has been widely used in is research (berg, 2009; salo et al., 2020). we interviewed users of a knowledge-intense service organization regarding their negative experiences related to issues while using consumer it for business purposes. upon receiving descriptions of stressful user experiences, we followed up with questions regarding stress creators, resources or individual characteristics, and technology characteristics. the interviews were recorded and transcribed. the transcripts were coded. contrary to grounded theory, this coding iteration was a deductive approach based on the authors’ domain knowledge. table 2 presents examples of direct quotes from the interviews. construct example based on the data unreliability “degree to which features and capabilities provided by the [digital] technology are [not] dependable” (ayyagari et al., 2011, p. 837) “i was working on a document on the business laptop, pressed the save button and closed the laptop. at home, i opened my private laptop, opened the file, and all my changes were unavailable. saving or synchronization [via the cloud technology] did not work; i have no idea.” “we were both working on the same presentation. the next day i opened it [on my private laptop], and part of the progress was gone. and then there was an error message.” it portfolio integration the ability to integrate data, communication and collaboration technologies, and other applications and services within one’s individual information system (based on rai & tang, 2010) “the problem that printing [from my private laptop] didn’t work was that there was no vpn connection and the printer only allows devices from the network.” “i had two computers that i worked with. the private computer was not integrated into the business it infrastructure, which means that i had no access to emails and data from the business computer and then always had to transfer everything from the business computer to the other computer.” general computer self-efficacy “an individual’s perception of efficacy in performing specific computer-related tasks” (marakas et al., 1998, p. 127) “i would describe myself as a tech-savvy person and able to deal with digital technologies. […] therefore, such situations are less stressful for me than for someone in whom these characteristics are less pronounced.” “i would not consider myself incompetent in it, but at that moment, i lacked the knowledge to deal with the specific situation without any problems. now i know better.” table 2. results of qualitative user interviews the findings show that the interviewees’ experience centered around the technostress creator unreliability. more precisely, switching between privately-owned and business-owned components of an it portfolio for business purposes was the most frequently reported source of stress related to it consumerization. the interviewees reported occasions where the integration of data collected or processed through privately-owned it caused problems when australasian journal of information systems lanzl et al. 2023, vol 27, research article issues regarding it consumerization 8 they were transferred to business it solutions and vice versa. for example, two interviewees mentioned that using a cloud solution was necessary to transfer data, which causes technical problems and may result in the loss of work progress. to help overcome such experiences of unreliability, our interviewees mentioned a seamless integration of the different components of their it portfolio to be one of the most important factors for effective it consumerization. they frequently reported issues regarding integration that led to the perception of unreliability. for example, barriers related to the transfer of data were reported, which created the need for workarounds to have the data on each necessary device, and one interviewee mentioned that they could only use the company’s printer from their private laptop and that they had to set up a vpn-connection to be able to print. as another influencing factor, our interviewees mentioned general computer self-efficacy to be important. in our interviews, respondents explained how they coped with stressful situations through creative solutions and workarounds. for example, one interviewee suggested that they lacked the technological competence to deal with experienced unreliability when using multiple components of heir mixed it portfolio. on the contrary, another employee stated that their background in it helps them be calmer and more resilient when it comes to overcoming issues with it. based on these results from our qualitative pre-study, we aim to further analyze the identified factors related to it consumerization. specifically, we want to further analyze how it consumerization may cause unreliability and how this is also dependent on it portfolio integration and general computer self-efficacy. thus, in the following section, we develop a research model incorporating these factors. 4 hypotheses development we aim to shed light on the relationship between it consumerization use behavior and its adverse consequences. for that, we use the theoretical lens of pt-misfit and emphasize the interplay between it consumerization, technology characteristics, personal resources, and outcomes. prior literature and our interviews show that unreliability plays an important role in that regard and is a driver of adverse outcomes of it consumerization. thus, the paper at hand focuses on unreliability as an important stressor but does not claim that it is the only issue related to it consumerization. two further relevant influencing factors were also identified in the interviews. first, the impact of the degree of integration between privatelyowned and business-owned components of the it portfolio used for business purposes. we define an it portfolio for business purposes as the composition of private and business it components (such as devices, applications, and services) used for business purposes (cf. junglas et al., 2019). second, users’ computer self-efficacy is considered a resource to mitigate the adverse impact of it consumerization. in addition, three adverse outcomes of unreliability in the specific context of it portfolios consisting of private and business it for business purposes were identified: dissatisfaction with the it portfolio, transition costs, and switching exhaustion. the following sub-sections develop our hypotheses in more detail. figure 2 summarizes the research model and emphasizes its relationship with the overarching pt-fit theory. australasian journal of information systems lanzl et al. 2023, vol 27, research article issues regarding it consumerization 9 figure 2: research model 4.1 it consumerization and unreliability system redundancies can be a source of stress in the context of it consumerization (ortbach, köffer, müller, & niehaves, 2013). we propose that this is because they cause the perception of unreliability (adam et al., 2017). unreliability is defined as the “degree to which features and capabilities provided by the technology are [not] dependable” (ayyagari et al., 2011, p. 837). this definition focuses on individual technologies and not necessarily their interplay. however, we extend this perception to it portfolios instead of single technologies and suggest that the use of privately-owned technologies for business purposes results in unreliability. thus, we hypothesize: h1: it consumerization use behavior increases unreliability. 4.2 effect of general computer self-efficacy personal resources play an essential role in the perception of stress. relevant personal resources are digital literacy (tarafdar et al., 2019) or technology competence (tarafdar et al., 2015). this role is echoed by it consumerization literature that finds an association with easier problem-solving for people with high technological competence (niehaves, köffer, ortbach, & reimler, 2013). further, the lack of competence is mentioned as the most frequent antecedent of technostress when engaging in it consumerization (ortbach, köffer, müller, & niehaves, 2013). in line with existing literature, we suggest a direct effect of general computer selfefficacy (shu et al., 2011) on technostress creators, such as unreliability: h2: general computer self-efficacy reduces unreliability. 4.3 effect of portfolio integration switching from privately-owned it to business-owned it is generally possible but associated with data transfer or information exchange between the different components, which we refer to as a lack of portfolio integration (rai & tang, 2010). if the privately-owned and businessowned it components are poorly integrated, users need to come up with workarounds to overcome problems that are caused by the lack of integration. such workarounds are a source of unreliability and cause frustration. nevertheless, when organizations provide seamless integration of the various privately-owned and business-owned components, such issues decrease. hence, we hypothesize: h3: it portfolio integration reduces unreliability. australasian journal of information systems lanzl et al. 2023, vol 27, research article issues regarding it consumerization 10 4.4 the effect of unreliability on outcomes as a first outcome of unreliability, we suggest dissatisfaction with the it portfolio, which is the user’s affective and cognitive evaluation of a consumption-related lack of need fulfillment experienced with the it portfolio (au et al., 2008). we argue that problems arising from integrating privately-owned and business-owned components of an it portfolio and the resulting perception of unreliability increase individuals’ dissatisfaction with their it portfolio (cf. au et al., 2008). hence, we hypothesize: h4a: unreliability increases dissatisfaction with the it portfolio. another result of switching between different technologies, such as online social networks, are heightened transition costs (maier et al., 2015). previous research finds that a lack of integration between different social media sites leads to high transition costs. transition costs reflect the time and effort required in such situations. as pointed out, switching from privately-owned it to business-owned it components is generally possible but associated with data transfer and communication efforts referred to as portfolio integration (rai & tang, 2010). thus, we propose the following: h4b: unreliability increases transition costs. technostress has several adverse outcomes regarding individuals’ well-being. particularly the effect on psychological strain has been studied extensively (e.g., ayyagari et al., 2011; galluch et al., 2015; maier et al., 2015). strain is one of the most important long-term problems that arise from stressful situations. to isolate the psychological effects of switching between different it, switching exhaustion has been conceptualized (maier et al., 2015). in other words, “the cause of the perception of exhaustion [is the] switching process” (maier et al., 2015, p. 291). while the construct was developed in the context of online social network use, we adapt it to the context of it consumerization and hypothesize: h4c: unreliability increases switching exhaustion. 5 empirical analysis 5.1 survey design and procedures to test the model empirically, we designed an online survey. the questionnaire started with an explanation of the survey’s scope and explained the use of private and business it for business purposes. participants indicated their level of it consumerization use behavior (junglas et al., 2019). we further measured unreliability (ayyagari et al., 2011) as well as its outcomes (au et al., 2008; maier et al., 2015). lastly, computer self-efficacy (marakas et al., 2007) and it portfolio integration (rai & tang, 2010) wee measured. all scales were reflective. we measured all items on a seven-point likert scale. where necessary, we adapted the items to the it consumerization context. appendix a provides an overview of all items. we furthermore asked an open-ended question regarding stressful or frustrating situations from switching between private it to business it and vice versa. we conducted the survey in april 2020 during the first wave of the covid-19 pandemic. for the recruiting of participants, we used the online crowdsourcing market amazon mechanical turk (mturk). thereby, we restricted participation to currently employed full-time workers in the us. internet-based platforms, such as mturk, allow the recruiting of participants for surveys and other tasks (steelman et al., 2014). mturk’s participant pool is closer to the us australasian journal of information systems lanzl et al. 2023, vol 27, research article issues regarding it consumerization 11 population than traditional university subject pools (paolacci et al., 2010). further, mturk has been frequently used in is research before (e.g., kehr et al., 2015). participants received a monetary reward of usd 1.30 for completing the survey. to ensure data quality, we implemented several measures. we only chose mturk workers that had participated in at least 50 tasks on mturk with an approval rate of 90% and more. further, we employed a commonly-used attention check (“if you are answering this survey cautiously, tick the second box from the left.”), an instructional manipulation check (oppenheimer et al., 2009), and we assessed open-ended questions to identify “unusual comments” (chmielewski & kucker, 2020). 5.2 results in total, 162 participants completed our survey and passed our quality gates, of which 32.7% are female and 65.4% male. three participants stated to be of another gender. the average age of respondents was 39 years. we assessed our research model through structural equation modeling (pls-sem) using smartpls 4 (ringle et al., 2022), which is a variance-based approach for sem (hair et al., 2017). we started with the evaluation of the measurement model before assessing the structural model and testing our hypotheses. 5.2.1 evaluation of the measurement model regarding internal consistency reliability (icr), all scales exceed the threshold of 0.708, with a minimum of 0.800 for cronbach’s alpha (alpha) and a minimum of 0.879 for composite reliability (cr). for convergent validity, we examined indicator reliability and average variance extracted (ave). convergent validity is satisfactory as the minimum of all indicators’ outer loadings is 0.780, and the minimum ave is 0.708. for discriminant validity, we first examined each indicator’s cross-loadings with all other constructs to check whether they are lower than the indicator’s outer loading on the construct. our data meets this criterion. second, each construct’s square root of the ave is higher than the highest correlation with other constructs (fornell-larcker criterion). thus, discriminant validity is supported. table 3 shows the means, standard deviations (sd), alpha, cr, and ave for all constructs. information on (cross-)loadings and the fornell-larcker criterion can be found in appendices b and c. # items mean sd loadings alpha cr ave it consumerization behavior 3 4.844 1.853 0.780-0.892 0.800 0.879 0.708 unreliability 3 5.287 1.272 0.951-0.966 0.957 0.972 0.920 general computer self-efficacy 6 6.069 1.295 0.824-0.934 0.944 0.954 0.777 it portfolio-integration 4 2.745 1.791 0.843-0.892 0.896 0.927 0.761 dissatisfaction with it portfolio 4 2.455 1.239 0.913-0.923 0.938 0.956 0.844 transition costs 3 3.434 1.712 0.926-0.957 0.941 0.946 0.962 switching exhaustion 4 3.091 1.850 0.939-0.953 0.962 0.963 0.973 table 3. descriptive statistics, main factor loadings, internal consistency, and average variance extracted 5.2.2 evaluation of structural model and hypotheses testing collinearity is not an issue since all variance inflation factors of the constructs are lower than 5.0 (max. of 1.262). figure 3 presents the path estimates for the model, including their significance level. australasian journal of information systems lanzl et al. 2023, vol 27, research article issues regarding it consumerization 12 figure 3. model results table 4 displays the corresponding r² and r² adjusted values as well as the q² values from plspredict, a procedure for assessing the predictive performance of the model (shmueli et al., 2019). as all q² values are above zero and since the majority of all indicators’ root mean squared errors (rmse) and mean absolute errors (mae) of the pls-sem analysis are smaller compared to the linear regression model (lm) (see appendix d), we assume predictive relevance (shmueli et al., 2019). we find that it consumerization use behavior is positively related to unreliability, which supports h1. also, general computer self-efficacy and it portfolio integration are found to have a significant and negative association with unreliability. this finding supports h2 and h3. lastly, h4a-h4c are supported as unreliability is positively related to dissatisfaction with the it portfolio, transition costs, and switching exhaustion. table 5 summarizes our hypotheses and their respective empirical results. r² r² adj. q² unreliability 0.255 0.240 0.223 dissatisfaction with it portfolio 0.075 0.069 0.077 transition costs 0.372 0.368 0.132 switching exhaustion 0.364 0.360 0.134 table 4. explained variance in the structural equation model theoretical hypotheses empirical results h1 pos. it consumerization behavior → unreliability supported ++ h2 neg. general computer self-efficacy → unreliability supported h3 neg. perceived it portfolio-integration → unreliability supported h4a pos. unreliability → dissatisfaction with it portfolio supported + h4b pos. unreliability → transition costs supported +++ h4c pos. unreliability → switching exhaustion supported +++ table 5: overview of hypotheses and empirical results note. plus signs indicate a significant and positive effect, minus signs a significant and negative effect, and n.s. would indicate a non-significant effect at the 5% level. for significant effects, +/indicates a small (f² ≥0.02), ++/-a medium (f² ≥0.15), and +++/--a large (f² >0.35) effect size. please see appendix e for the detailed f² values. australasian journal of information systems lanzl et al. 2023, vol 27, research article issues regarding it consumerization 13 we find that it consumerization use behavior is positively related to unreliability, which supports h1. also, general computer self-efficacy and it portfolio integration are found to have a significant and negative association with unreliability. this finding supports h2 and h3. lastly, h4a-h4c are supported as unreliability is positively related to dissatisfaction with the it portfolio, transition costs, and switching exhaustion. table 5 summarizes our hypotheses and their respective empirical results. 5.3 evaluation of the qualitative survey data for the qualitative strand of our main study, we asked the respondents to name stressful or frustrating situations from switching between privately-owned and business-owned it and vice versa. we collected 130 valid responses to this question. these answers were inductively coded using open coding (or initial coding), which refers to the beginning stages of the data analysis that aims to segment the data. initial codes were then reorganized into broader categories, a process that can be described as axial coding (strauss & corbin, 1990). our findings are presented in table 6. description example based on the data lack of reliable access difficulties, such as unreliability and slow speeds, with accessing business resources from private devices – particularly through secure connections via vpns. “accessing the shared server through vpn can be difficult if there are server problems or network issues. sometimes the server can be down for hours, and i cannot access files.” “i was unable to maintain a stable connect[ion] with my vpn for work, but a technician talked me through it, and it was eventually resolved.” issues primarily related to data transfer tedious file transfer transferring data from one device to another is perceived as tedious. lack of speed and time-consuming workarounds cause frustration. “i have found it difficult to have a particular software on my business it working. so, i used my private device to complete the job. but it is almost not possible to upload the results to the server of my company from a private it component without the approval of the it personnel [...]. so i have to forward this result to my business it before uploading, which turns out to be an extra effort.” “i have photoshop on my business laptop but use my personal desktop for most of my work. when i download a photo that needs editing, i have to email it to myself to my laptop so i can use photoshop.” data inconsistencies and redundancies unsuccessful syncing of applications between private and business it devices creates frustration, particularly where inconsistent data must be managed redundantly. “working from home, there have been some glitches where some applications are not talking to one another. there should be no time [gap] when i update one database, which should then carry over to other applications.” “i had some issues syncing up emails between three devices because one of the devices was on an older operating system. this laptop cannot be upgraded any further, so it’s causing me some issues.” loss of data while switching loss of data while transferring data from private to business component and vice versa, particularly when unreliable workarounds are necessary. “the last time i had to switch from one it component to the other, i ended up losing almost all my files because it was not well backed up. it was so frustrating that i had to call the company to help find a way to recover some of the important documents.” “i lost my flash drive on which i put the data [i wanted to transfer]. it was very frustrating.” issues primarily related to it usage incompatibility incompatibility of private and business it. mainly because of different operating systems and “sometimes, some files are not compatible across devices, especially between mac and pc. it is annoying to try to figure out how to convert them.” australasian journal of information systems lanzl et al. 2023, vol 27, research article issues regarding it consumerization 14 specific file types that cannot be accessed. “i have my work saved on my [private] laptop, and i want to access that work on my business phone, but not all of the data is fully transferable. i.e., excel documents are only on my laptops.” inability to use the same software problems with the installation of business applications on the private device or vice versa cause undesired barriers. this can be due to incompatible hardware and software, it policies, licenses, or a lack of admin rights. “i am not allowed to use email on my private it component. so when my manager uses instant messaging to alert me that i have to check my email, i need to use my business it component (laptop).” “trying to log on to our all-employee virtual meeting using microsoft teams – i couldn't get the native app on my pc to work, so switched to the app on my phone, and then the web app.” table 6. coding scheme of qualitative survey data we find several aspects that contribute to the perception of unreliability due to a lack of it portfolio integration. first, a lack of reliable access to company resources is reported to be a source of frustration. such a perception can be caused by slow vpn connections, unavailable business servers, or network issues. this creates problems while trying to cross the boundaries from private to business it. second, we find several aspects related to data and file transfer that are a source of unreliability for many employees. file transfer is perceived as tedious, particularly when direct options are unavailable and workarounds, such as file forwarding via email, have to be employed to switch from business to private solutions and vice versa. similarly, a lack of seamless integration between private and business it components can cause data inconsistencies or the need to manage data redundantly. this is particularly the case when options for automatic synchronization are missing or not working properly. in extreme cases, the need for workarounds or manual file transfer can result in data loss. this is a major source of frustration as productivity loss often accompanies it. lastly, incompatibility between different operating systems in a mixed it portfolio consisting of unaligned private and business it components creates issues. workarounds, such as file converters, have to be employed by the individuals, hindering their workflow. third, known routines can be adversely affected due to the inability to use the same software on private and business it. this can be caused by incompatibility between technologies or by deliberate decisions by organizations, such as it policies. it creates boundaries between the components, which take up additional time and are undesired by the employees. while these categories help gain a detailed understanding of users’ experiences, they are not without interrelations and interdependencies. for example, loss of data could result from a tedious file transfer workaround gone wrong, and incompatibility could result in an inability to use the same software. however, the categories can also occur independently. for example, the inability to use the same software can be due to it policies. despite these limitations, we consider the list a good overview of the underlying issues of unreliability due to a lack of integration between business and private it that may guide decisions in practice. 6 discussion our research was motivated by two major recent developments. first, the covid-19 pandemic and its social distancing measures force employees into more mobile work and work-from-home. this has specifically increased the need for digital communication and collaboration. the development was frequently accompanied by a need for it that was not provided by the employer, thus, resulting in increased it consumerization. second, prior research has stated the need to better understand innovative it use associated with it australasian journal of information systems lanzl et al. 2023, vol 27, research article issues regarding it consumerization 15 consumerization (tarafdar et al., 2019). however, adverse effects on the individual have been studied scarcely. this scarcity is particularly true for the role of mixed it portfolios and the integration of privately-owned and business-owned it components. thus, we built a theoretical model based on pt-fit and analyzed the effect of it consumerization on unreliability as well as associated outcomes and the influence of computer self-efficacy and it portfolio integration. we used a qualitative pre-study to inform our theorizing and tested the theoretical model quantitatively through survey data. the data supported our hypotheses. qualitative data from open-ended survey questions added richness to the understanding of the relationships. our mixed-methods approach was aimed at the primary purpose of completeness. however, it also allowed us to triangulate and corroborate our results. because our qualitative results provide various examples of the relationship between it consumerization behavior and unreliability, our meta-inferences are that the results from the two data types are convergent. the qualitative data reveals several issues that arise from using an it portfolio that consists of private and business components. this data allows us to pinpoint concrete problems and hurdles regarding it consumerization that may be mitigated by software providers (e.g., by emphasizing compatibility and easy data transferability via clouds) and it departments (e.g., through choose-your-own-device initiatives or by providing adequate remote access to company resources). we consider these results to be robust and to provide a comprehensive overview of the phenomenon under investigation and elaborate on the triangulation of quantitative and qualitative data in the following. our quantitative findings show that high it consumerization behavior comes with the perception that the it portfolio is unreliable (h1). our qualitative analysis allowed us to triangulate these findings by identifying various reasons for this relationship. we revealed that handling an it portfolio that consists of both private and business components may yield various compatibility issues (e.g., incompatibilities due to different operating systems or different software packages) and problems with data transfer between the two (e.g., data inconsistencies and redundancies or the loss of data while switching between two components). with extended it consumerization behavior, these issues appear more often and more of these issues may appear – thus leading to a higher perceived unreliablity as our quantitative analysis indicates. second, we found that confidence and skills in regard to technology (i.e., computer self-efficacy) may reduce the perception of unreliability (h2). we also found evidence for this relationship in our qualitative data. for example, users stated that converting data for it to work with incompatible it components requires effort, skills, and experience. third, we revealed a relationship between a higher it portfolio integration and a lower perception of unreliability (h3). we corroborated these findings through qualitative data that suggests that many issues that revolve around compatibility may be subject to specific it portfolios that are poorly integrated (e.g., portfolios consisting of mac and windows pcs). fourth, we also quantitatively investigated the negative outcomes of unreliability, which encompasses dissatisfaction with the it portfolio, switching exhaustion, and transition costs that hamper performance (h4a-c). while the outcomes were not the focus of our qualitative questions, we found some indications that yield the same direction as in the quantitative data, indicating plausibility of the quantitative findings. for example, when an it portfolio causes unreliability in the form of tedious data transfer that takes time and leads to frustration, switching between one device and another is perceived as exhausting and stressful. further, when individuals are unable to use the same software for communication because they are not australasian journal of information systems lanzl et al. 2023, vol 27, research article issues regarding it consumerization 16 allowed to or do not have access to their business-owned email on their private devices, they experience higher transition costs as it remains difficult for them to stay in touch with their coworkers when using their private devices. 6.1 theoretical contribution we found that it consumerization and the use of mixed it portfolios that are poorly integrated have multiple negative consequences. based on extant literature and qualitative interviews, unreliability is a major mediator of this relationship. this is important to notice since prior research on it consumerization has found higher usefulness and ease of use to be the key drivers for individuals’ it adoption decisions in general (venkatesh et al., 2012) and it consumerization in particular (ortbach, 2015). the perception of it as not behaving consistently and its features as not being dependable tend to go against classical antecedents of technology acceptance, like perceived ease of use and perceived usefulness. unreliability has been established as a technostress creator (adam et al., 2017; fischer et al., 2021). aspects of it, such as computer freezes, have been studied in laboratory experiments (weinert et al., 2020) and shown to cause biological consequences, such as the release of stress hormones (riedl et al., 2012). yet, there is little research that has drawn a connection between it consumerization and an increase in unreliability. here, we see a need for a deeper investigation to understand this paradox. as a first step to deepening this understanding, we found that it portfolio integration is a crucial factor for the seamless operation of it portfolios for business purposes, including private it components. to the best of our knowledge, this perception has not been considered in the it consumerization literature so far, and we contribute to it by raising this issue. we find that issues with integration between multiple components of a mixed it portfolio lead to a higher perception of unreliability. while the duality of it components certainly raises issues with integration, which we show in this study, such issues can also emerge between multiple heterogeneous it components provided by the business. for example, different manufacturers of it and their respective operating systems might cause such issues, which, thus, should be regarded in future research. this may also touch upon certain aspects of compatibility, which have been used as an exogenous variable that extends utaut and may explain user behavior (e.g., blut et al., 2022). yet, our study indicates that it may also be related to unreliability and, thus, technostress. while poor integration may be an issue with it portfolios consisting of only business-owned it components, their management is easier and lies in the hand of it departments. this is different for mixed it portfolios, where users are administrating their privately-owned components. it is apparent that the general computer self-efficacy of the users also plays a vital role in this relationship. we find that general computer self-efficacy influences the effect of it consumerization on technostress creators in several ways. this is congruent with the transactional model of stress that most technostress research is based on. here, stress emerges when external demands tax or exceed the resources an individual can use to meet the demands (lazarus & folkman, 1984). general computer self-efficacy is such a resource. several is studies have investigated the relationship between such it-related competencies that may lead to a perception of control in relation to technostress (shu et al., 2011; tams et al., 2018). our findings are congruent with these results and are particularly important for the individuals themselves when deciding whether to engage in it consumerization. this yields several practical implications that we discuss in the next section. australasian journal of information systems lanzl et al. 2023, vol 27, research article issues regarding it consumerization 17 we extend upon our quantitative findings through a qualitative analysis of users’ experience of poor integration of privately-owned and business-owned it. to the best of our knowledge, such in-depth insights are not covered by previous is-related research on unreliability or compatibility. we find a host of different problems that can arise, which center around universal access to company resources, data transfer, synchronization, and compatibility issues. these insights also extend our understanding of the technostress phenomenon and may guide future work on mitigating technostress related to it consumerization. 6.2 practical implications since employees tend to increasingly use privately-owned it for business purposes, it is crucial to understand one’s personal consequences of this adoption decision to make this decision rationally. our qualitative investigation reveals several issues that employees face when business it and private it are poorly integrated. some of these issues can be overcome by the organization: for example, seamless solutions for data transfer that employees can integrate onto their private devices may help with issues with data loss or data redundancies. many cloud-based office suites offer such integration that can be made available to private devices based on the organization’s it policies. several changes come as a tradeoff between it security and user-friendliness: with recent security incidents attracting global attention and home-office workers becoming an increasingly popular gateway for hackers, many companies tend to be cautious with their it policies. however, they should be aware that restricting access to company resources decreases the it portfolio integration and, thereby, causes unreliability for the employees. as our study shows, this stress, in turn, is associated with adverse effects on the employees and the company itself. our findings further show that such stressful situations are particularly problematic for it users that lack personal it-related resources, measured as general computer self-efficacy, to overcome such issues. thus, we conclude that it consumerization is only reasonable if individuals can handle the technologies of their it portfolio and the complexity introduced by the interplay of privately-owned and business it components. hence, it consumerization should be cautiously used by employees lacking the resources to meet the additional demands arising from it consumerization. furthermore, privately-owned it components usually receive less organizational support, an essential external resource for inhibiting technostress. in order to prevent the identified negative consequences of it consumerization, organizations would be well-advised to start offering support for privately-owned it. such support would allow both the organization and its employees to benefit from the advantages associated with it consumerization without risking their employees’ well-being and productivity. alternatively, to reduce it consumerization and its adverse outcomes in the first point, organizations can provide their employees with all relevant and easy-to-use it components they need for mobile work and work from home. yet, this naturally hampers its benefits. in summation, restrictive and laissez-faire it consumerization policies may adversely affect users that struggle with it. organizations should thus embrace it consumerization and offer technical support and adequate technological integration of private it devices into their portfolio to reap the benefits and mitigate the risks of it consumerization. australasian journal of information systems lanzl et al. 2023, vol 27, research article issues regarding it consumerization 18 7 limitations and conclusion our study has a number of limitations that leave room for further research. in the quantitative empirical part, we used data from a single cross-sectional survey in times of the covid-19 pandemic, and in our sample, male participants are more represented than in the us workforce in general. this leads to limitations in testing the robustness and generalizability of results. future research should follow up with generating additional data sets to test robustness and generalizability. particularly the relationship between it portfolio integration and technostress seems promising for future research and should be further validated. further, in our research, we emphasize the role of unreliability, which has been scarcely studied in the context of it consumerization. we show that it portfolio integration plays a significant role in this relationship. while we elicit several reasons why it consumerization creates such problems, for example, through a lack of strategic alignment of the components, we do not think that the outlined problems are limited to this domain. poorly managed or historically grown business it portfolios may have similar issues. future research could thus explore the impact of integration on technostress within other it portfolios. additionally, the pandemic-related conditions for organizations have changed. whereas during the pandemic, many organizations worked from home to a high degree, today, they ask their employees to come back to the office. even though this development might change the amount of it consumerization, we believe that in the current hybrid work settings (i.e., a mixture of working in the organization’s offices and at home), it consumerization still plays an important role. future research should thus investigate how the context of hybrid work may affect it consumerization. previous work has raised research questions regarding different types of stress, particularly challenge stress (tarafdar et al., 2019). one element mentioned to create challenging situations for users is innovative work behavior (tarafdar et al., 2019). this factor is said to be facilitated by it consumerization (junglas et al., 2019). while unreliability is likely a hindrance or a threat to most individuals, future research should look into ways that it consumerization can contribute to the bright side of technostress. despite these suggestions for future research, the paper at hand contributes to the scholarly discourse on both the effects of it consumerization and the antecedents of technostress. it provides several suggestions for practitioners to govern and manage it consumerization and mixed it portfolios. in conclusion, we add to the current understanding of it consumerization by shedding light on the adverse effects of it consumerization concerning technostress and its consequences. we find unreliability to be particularly relevant in this context. with a mixed-methods design, we detail why a mixed it portfolio of business and private it components creates a sense of unreliability. our research further contributes concrete issues that users experience and suggests how these effects can be attenuated on an organizational and individual level. we conclude that it consumerization needs to be adequately managed and integrated into the existing business-owned it landscape to reduce individual exhaustion, increase satisfaction with the it portfolio, and reduce transition costs that inhibit performance. we further suggest that it consumerization makes the most sense for users with a high level of it-related resources to successfully overcome the remaining boundaries between privately-owned and business-owned it. australasian journal of information systems lanzl et al. 2023, vol 27, research article issues regarding it consumerization 19 references adam, m. t. p., gimpel, h., maedche, a., & riedl, r [rene] (2017). design blueprint for stress-sensitive adaptive enterprise systems. business & information systems engineering, 59(4), 277–291. https://doi.org/10.1007/s12599-016-0451-3 au, n., ngai, e. w. t., & cheng, t. c. e. (2008). extending the understanding of end user information systems satisfaction formation: an equitable needs fulfillment model approach. management information systems quarterly, 32(1), 43–66. https://doi.org/10.2307/25148828 ayyagari, r., grover, v., & purvis, r. (2011). technostress: technological antecedents and implications. management information systems quarterly, 35(4), 831–858. https://doi.org/10.2307/41409963 baskerville, r. l. (2011). individual information systems as a research arena. european journal of information systems, 20(3), 251–254. https://doi.org/10.1057/ejis.2011.8 baskerville, r. l., & lee, a. s. (2013). individual-organizational bindpoints: a design theory for bring-your-own-system. pacis 2013 proceedings. http://aisel.aisnet.org/pacis2013/157 bautista, j. r., rosenthal, s., lin, t. t., & theng, y. l. (2018). predictors and outcomes of nurses’ use of smartphones for work purposes. computers in human behavior, 84, 360– 374. https://doi.org/10.1016/j.chb.2018.03.008 becker, j., berger, m., gimpel, h., lanzl, j., & regal, c. (2020). considering characteristic profiles of technologies at the digital workplace: the influence on technostress. icis 2020 proceedings. https://aisel.aisnet.org/icis2020/is_workplace_fow/is_workplace_fow/5/ behrens, s. (2009). shadow systems. communications of the acm, 52(2), 124–129. https://doi.org/10.1145/1461928.1461960 berg, b. l. (2009). qualitative research methods for the social sciences (7. ed.). boston, ma, usa: allyn & bacon. blut, m., chong, a. y. l., tsigna, z., & venkatesh, v. (2022). meta-analysis of the unified theory of acceptance and use of technology (utaut): challenging its validity and charting a research agenda in the red ocean. journal of the association for information systems, 23(1), 13–95. https://doi.org/10.17705/1jais.00719 chmielewski, m., & kucker, s. c. (2020). an mturk crisis? shifts in data quality and the impact on study results. social psychological and personality science, 11(4), 464–473. https://doi.org/10.1177/1948550619875149 crossler, r. e., long, j. h., loraas, t. m., & trinkle, b. s. (2014). understanding compliance with bring your own device policies utilizing protection motivation theory: bridging the intention-behavior gap. journal of information systems, 28(1), 209–226. https://doi.org/10.2308/isys-50704 doargajudhur, m. s., & dell, p. (2019). impact of byod on organizational commitment: an empirical investigation. information technology & people, 32(2), 246–268. https://doi.org/10.1108/itp-11-2017-0378 australasian journal of information systems lanzl et al. 2023, vol 27, research article issues regarding it consumerization 20 edwards, j. r., caplan, r. d., & harrison, r. v. (1998). person-environment fit theory. in c. l. cooper (ed.), theories of organizational stress (pp. 28–67). oxford university press. https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=97e0811e4c8b9394a 5d3cd049fcdfdb9a24920ab fischer, t., reuter, m., & riedl, r. (2021). the digital stressors scale: development and validation of a new survey instrument to measure digital stress perceptions in the workplace context. frontiers in psychology, 12, 607598. https://doi.org/10.3389/fpsyg.2021.607598 galluch, p. s., grover, v., & thatcher, j. b. (2015). interrupting the workplace: examining stressors in an information technology context. journal of the association for information systems, 16(1), 1–47. https://doi.org/10.17705/1jais.00387 gewald, h., wang, x., weeger, a., raisinghani, m. s., grant, g., sanchez, o., & pittayachawan, s. (2017). millennials’ attitudes toward it consumerization in the workplace. communications of the acm, 60(10), 62–69. https://doi.org/10.1145/3132745 hair, j. f., hult, g. t. m., ringle, c. m., & sarstedt, m. (2017). a primer on partial least squares structural equation modeling (pls-sem) (2nd edition). thousand oaks, ca, usa: sage publications. harris, j., ives, b., & junglas, i. (2012). it consumerization: when gadgets turn into enterprise it tools. mis quarterly executive, 11(3), 99-112. junglas, i., goel, l., ives, b., & harris, j. (2019). innovation at work: the relative advantage of using consumer it in the workplace. information systems journal, 29(2), 317–339. https://doi.org/10.1111/isj.12198 kehr, f., kowatsch, t., wentzel, d., & fleisch, e. (2015). blissfully ignorant: the effects of general privacy concerns, general institutional trust, and affect in the privacy calculus. information systems journal, 25(6), 607–635. https://doi.org/10.1111/isj.12062 köffer, s., junglas, i., chiperi, c., & niehaves, b. (2014). dual use of mobile it and work-tolife conflict in the context of it consumerization. icis 2014 proceedings. https://aisel.aisnet.org/icis2014/proceedings/societalimpacts/7 köffer, s., ortbach, k., junglas, i., niehaves, b., & harris, j. (2015). innovation through byod? business & information systems engineering, 57(6), 363–375. https://doi.org/10.1007/s12599-015-0387-z lazarus, r. s., & folkman, s. (1984). stress, appraisal, and coping. new york, usa: springer. lee, a. s., thomas, m., & baskerville, r. l. (2015). going back to basics in design science: from the information technology artifact to the information systems artifact. information systems journal, 25(1), 5–21. https://doi.org/10.1111/isj.12054 lee, j., warkentin, m., crossler, r. e., & otondo, r. f. (2017). implications of monitoring mechanisms on bring your own device adoption. journal of computer information systems, 57(4), 309–318. https://doi.org/10.1080/08874417.2016.1184032 maier, c., laumer, s., weinert, c., & weitzel, t. (2015). the effects of technostress and switching stress on discontinued use of social networking services: a study of facebook use. information systems journal, 25(3), 275–308. australasian journal of information systems lanzl et al. 2023, vol 27, research article issues regarding it consumerization 21 https://doi.org/10.1111/isj.12068 marakas, g. m., johnson, r. d., & clay, p. f. (2007). the evolving nature of the computer self-efficacy construct: an empirical investigation of measurement construction, validity, reliability and stability over time. journal of the association for information systems, 8(1), 16–46. marakas, g. m., yi, m. y., & johnson, r. d. (1998). the multilevel and multifaceted character of computer self-efficacy: toward clarification of the construct and an integrative framework for research. information systems research, 9(2), 126–163. https://doi.org/10.1287/isre.9.2.126 niehaves, b., köffer, s., & ortbach, k. (2012). it consumerization: a theory and practice review. amcis 2012 proceedings. https://aisel.aisnet.org/amcis2012/proceedings/enduseris/18 niehaves, b., köffer, s., & ortbach, k. (2013). the effect of private it use on work performance: towards an it consumerization theory. wirtschaftsinformatik 2013 proceedings. https://aisel.aisnet.org/wi2013/3 niehaves, b., köffer, s., ortbach, k., & reimler, s. (2013). boon and bane of it consumerization: the burnout-engagement-continuum. amcis 2013 proceedings. https://aisel.aisnet.org/amcis2013/enduseris/generalpresentations/11 oppenheimer, d. m., meyvis, t., & davidenko, n. (2009). instructional manipulation checks: detecting satisficing to increase statistical power. journal of experimental social psychology, 45(4), 867–872. https://doi.org/10.1016/j.jesp.2009.03.009 ortbach, k. (2015). unraveling the effect of personal innovativeness on bring-your-owndevice (byod) intention: the role of perceptions towards enterprise-provided and privately-owned technologies. ecis 2015 proceedings. https://aisel.aisnet.org/ecis2015_cr/141/ ortbach, k., köffer, s., bode, m., & niehaves, b. (2013). individualization of information systems: analyzing antecedents of it consumerization behavior. icis 2013 proceedings. https://aisel.aisnet.org/icis2013/proceedings/humanbehavior/25 ortbach, k., köffer, s., müller, c. p. f., & niehaves, b. (2013). how it consumerization affects the stress level at work: a public sector case study. pacis 2013 proceedings. https://aisel.aisnet.org/pacis2013/231 paolacci, g., chandler, j., & ipeirotis, p. g. (2010). running experiments on amazon mechanical turk. judgement and decision making, 5(5), 411–419. ragu-nathan, t. s., tarafdar, m., ragu-nathan, b. s., & tu, q. (2008). the consequences of technostress for end users in organizations: conceptual development and empirical validation. information systems research, 19(4), 417–433. https://doi.org/10.1287/isre.1070.0165 rai, a., & tang, x. (2010). leveraging it capabilities and competitive process capabilities for the management of interorganizational relationship portfolios. information systems research, 21(3), 516–542. https://doi.org/10.1287/isre.1100.0299 australasian journal of information systems lanzl et al. 2023, vol 27, research article issues regarding it consumerization 22 riedl, r., kindermann, h., auinger, a., & javor, a. (2012). technostress from a neurobiological perspective. business & information systems engineering, 4(2), 61–69. https://doi.org/10.1007/s12599-012-0207-7 ringle, c. m., wende, s., & becker, j.-m. (2022). smartpls 4 [computer software]. oststeinbek. salo, m., makkonen, m., & hekkala, r. (2020). the interplay of it users’ coping strategies: uncovering momentary emotional load, routes, and sequences. management information systems quarterly, 44(3), 1143–1175. https://doi.org/10.25300/misq/2020/15610 salo, m., pirkkalainen, h., makkonen, m., & hekkala, r. (2018). distress, eustress, or no stress? explaining smartphone users’ different technostress responses. icis 2018 proceedings. https://aisel.aisnet.org/icis2018/behavior/presentations/13 shmueli, g., sarstedt, m., hair, j. f., cheah, j.-h., ting, h., vaithilingam, s., & ringle, c. m. (2019). predictive model assessment in pls-sem: guidelines for using plspredict. european journal of marketing, 53(11), 2322–2347. https://doi.org/10.1108/ejm-02-20190189 shu, q., tu, q., & wang, k. (2011). the impact of computer self-efficacy and technology dependence on computer-related technostress: a social cognitive theory perspective. international journal of human–computer interaction, 27(10), 923–939. https://doi.org/10.1080/10447318.2011.555313 silic, m., & back, a. (2014). shadow it: a view from behind the curtain. computers & security, 45, 274–283. steelman, z. r., hammer, b. i., & limayem, m. m. (2014). data collection in the digital age: innovative alternatives to student samples. management information systems quarterly, 38(2), 355–378. https://doi.org/10.25300/misq/2014/38.2.02 stich, j.-f., tarafdar, m., stacey, p., & cooper, c. l. (2019). appraisal of email use as a source of workplace stress: a person-environment fit approach. journal of the association for information systems, 20(2), 132–160. https://doi.org/10.17705/1jais.00531 strauss, a. l., & corbin, j. m. (1990). basics of qualitative research: grounded theory procedures and techniques. thousand oaks, ca, usa: sage publications. tams, s., thatcher, j. b., & grover, v. (2018). concentration, competence, confidence, and capture: an experimental study of age, interruption-based technostress, and task performance. journal of the association for information systems, 19, 857–908. https://doi.org/10.17705/1jais.00511 tarafdar, m., cooper, c. l., & stich, j.-f. (2019). the technostress trifecta: techno eustress, techno distress and design: theoretical directions and an agenda for research. information systems journal, 29(1), 6–42. https://doi.org/10.1111/isj.12169 tarafdar, m., pullins, e. b., & ragu-nathan, t. s. (2015). technostress: negative effect on performance and possible mitigations. information systems journal, 25(2), 103–132. https://doi.org/10.1111/isj.12042 australasian journal of information systems lanzl et al. 2023, vol 27, research article issues regarding it consumerization 23 tarafdar, m., tu, q., & ragu-nathan, t. s. (2010). impact of technostress on end-user satisfaction and performance. journal of management information systems, 27(3), 303–334. https://doi.org/10.2753/mis0742-1222270311 venkatesh, v., brown, s. a., & bala, h. (2013). bridging the qualitative-quantitative divide: guidelines for conducting mixed methods research in information systems. management information systems quarterly, 37(1), 21–54. venkatesh, v., brown, s. a., & sullivan, y. w. (2016). guidelines for conducting mixedmethods research: an extension and lllustration. journal of the association for information systems, 17(7), 435–494. venkatesh, v., thong, j. y. l., & xu, x. (2012). consumer acceptance and use of information technology: extending the unified theory of acceptance and use of technology. management information systems quarterly, 36(1), 157–178. weeger, a., wang, x., & gewald, h. (2016). it consumerization: byod-program acceptance and its impact on employer attractiveness. journal of computer information systems, 56(1), 1–10. https://doi.org/10.1080/08874417.2015.11645795 weinert, c., maier, c., laumer, s., & weitzel, t. (2020). technostress mitigation: an experimental study of social support during a computer freeze. journal of business economics, 90(8), 1199–1249. https://doi.org/10.1007/s11573-020-00986-y appendices appendix a – measurement items it consumerization use behavior (source: junglas et al., 2019) itc01 i use my private devices (e.g., smartphone, ipad, or private laptop computer) for business purposes. itc02 i use mobile applications downloaded from the web for business purposes. itc03 i use my private services (e.g., skype, twitter, facebook, text messaging) for business purposes. unreliability (source: ayyagari et al., 2011) unr01 the features provided by components of my it portfolio for business purposes are not dependable. unr02 the capabilities provided by components of my it portfolio for business purposes are not reliable. unr03 components of my it portfolio for business purposes do not behave in a highly consistent way. it portfolio integration (source: rai & tang, 2010) pin01 my it portfolio for business purposes easily accesses data from its various components. pin02 my it portfolio for business purposes provides seamless connection between its various components (e.g., devices, services, data). pin03 my it portfolio for business purposes has the capability to exchange realtime information between its various components. pin04 my it portfolio for business purposes easily aggregates relevant information from its various data sources (e.g., file storage, messaging, email, office suite). general computer self-efficacy (source: marakas et al., 2007) cse01 i believe i have the ability to describe how a computer works. cse02 i believe i have the ability to install new software applications on a computer. cse03 i believe i have the ability to identify and correct common operational problems with a computer. cse04 i believe i have the ability to unpack and set up a new computer. cse05 i believe i have the ability to remove information from a computer that i no longer need. cse06 i believe i have the ability to use a computer to display or present information in a desired manner. dissatisfaction with it portfolio (source: au et al., 2008) dis01 i am very contented with my it portfolio for business purposes.* dis02 i am very pleased with my it portfolio for business purposes.* dis03 i feel delighted with my it portfolio for business purposes.* australasian journal of information systems lanzl et al. 2023, vol 27, research article issues regarding it consumerization 24 dis04 overall, i am very satisfied with my it portfolio for business purposes.* transition costs (source: maier et al., 2015) tc01 it takes a lot of time to maintain the level of information exchange with my business environment using different components of my it portfolio for business purposes. tc02 it takes a lot of time to maintain the level of communication with my business environment using different components of my it portfolio for business purposes. tc03 overall, it takes a lot of time to maintain the established level of socializing with my business environment when using different components of my it portfolio for business purposes. switching exhaustion (source: (maier et al., 2015) se01 switching from one component of my it portfolio for business purposes to one or more other components stresses me out. se02 i feel tired by switching from one component of my it portfolio for business purposes to one or more other components. se03 switching from one component of my it portfolio for business purposes to one or more other components is a strain for me. se04 i feel drained from activities involved in switching from one component of my it portfolio for business purposes to one or more other components. *reverse-coded appendix b – factor loadings (main loading in bold font) itc unr cse pin dis tc se itc itc01 0.780 0.114 0.279 0.281 -0.227 0.105 0.033 itc02 0.892 0.221 0.280 0.306 -0.186 0.150 0.170 itc03 0.848 0.183 0.068 0.168 -0.151 0.145 0.252 unr unr01 0.209 0.966 -0.267 -0.286 0.243 0.610 0.606 unr02 0.178 0.951 -0.327 -0.315 0.266 0.591 0.565 unr03 0.236 0.961 -0.228 -0.317 0.280 0.553 0.563 cse cse01 0.268 -0.144 0.824 0.328 -0.209 -0.121 -0.150 cse02 0.206 -0.313 0.934 0.360 -0.325 -0.207 -0.251 cse03 0.250 -0.217 0.871 0.406 -0.321 -0.150 -0.220 cse04 0.164 -0.194 0.844 0.315 -0.177 -0.231 -0.284 cse05 0.193 -0.278 0.899 0.322 -0.268 -0.222 -0.264 cse06 0.224 -0.293 0.913 0.423 -0.331 -0.236 -0.270 pin pin01 0.273 -0.312 0.384 0.882 -0.513 -0.208 -0.162 pin02 0.260 -0.272 0.322 0.873 -0.526 -0.220 -0.121 pin03 0.272 -0.295 0.388 0.892 -0.428 -0.242 -0.208 pin04 0.218 -0.217 0.317 0.843 -0.508 -0.183 -0.090 dis dis01 -0.205 0.271 -0.315 -0.524 0.917 0.206 0.201 dis02 -0.234 0.239 -0.367 -0.560 0.923 0.172 0.178 dis03 -0.157 0.209 -0.223 -0.450 0.913 0.105 0.146 dis04 -0.188 0.276 -0.256 -0.524 0.922 0.099 0.166 tc tc01 0.137 0.616 -0.213 -0.245 0.171 0.957 0.604 tc02 0.134 0.571 -0.212 -0.243 0.165 0.954 0.550 tc03 0.193 0.538 -0.219 -0.209 0.115 0.926 0.605 se se01 0.149 0.562 -0.333 -0.187 0.178 0.556 0.939 se02 0.203 0.596 -0.308 -0.197 0.199 0.622 0.948 se03 0.183 0.556 -0.220 -0.120 0.175 0.576 0.953 se04 0.214 0.571 -0.190 -0.145 0.166 0.595 0.952 note: itc = it consumerization use behavior, unr = unreliability, pin = it portfolio integration, cse = general computer self-efficacy, sat = dissatisfaction with it portfolio, tc = transition costs, se = switching exhaustion australasian journal of information systems lanzl et al. 2023, vol 27, research article issues regarding it consumerization 25 appendix c – inter-factor-correlations (square root of ave in the diagonal) itc unr cse pin dis tc se itc 0.841 unr 0.216 0.959 cse 0.241 -0.286 0.881 pin 0.296 -0.319 0.408 0.872 sat -0.215 0.274 -0.318 -0.563 0.918 tc 0.162 0.610 -0.227 -0.246 0.160 0.946 se 0.198 0.603 -0.277 -0.172 0.190 0.620 0.948 note: itc = it consumerization use behavior, unr = unreliability, pin = it portfolio integration, cse = general computer self-efficacy, sat = dissatisfaction with it portfolio, tc = transition costs, se = switching exhaustion appendix d – results of plspredict pls lm rmse mae rmse mae unr01 1.574 1.285 1.631 1.329 unr02 1.654 1.346 1.706 1.363 unr03 1.580 1.300 1.637 1.340 dis01 1.098 0.916 1.025 0.798 dis02 1.148 0.930 1.023 0.808 dis03 1.375 1.082 1.333 1.026 dis04 1.159 0.930 1.072 0.830 tc01 1.612 1.340 1.684 1.404 tc02 1.601 1.355 1.717 1.441 tc03 1.629 1.390 1.698 1.421 se01 1.786 1.522 1.819 1.497 se02 1.690 1.451 1.727 1.449 se03 1.755 1.491 1.799 1.514 se04 1.734 1.438 1.764 1.430 appendix e – effect sizes (f² values) relationship f² it consumerization use behavior → unreliability 0.166 general computer self-efficacy → unreliability 0.064 it portfolio integration → unreliability 0.116 unreliability → dissatisfaction with it portfolio 0.081 unreliability → transition costs 0.592 unreliability → switching exhaustion 0.572 australasian journal of information systems lanzl et al. 2023, vol 27, research article issues regarding it consumerization 26 copyright: © 2023 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v27i0.4121 http://creativecommons.org/licenses/by-nc/3.0/au/ keywords: it consumerization, byod, technostress, self-efficacy, integration. 1 introduction 2 theoretical background 2.1 it consumerization 2.2 negative outcomes of it consumerization 2.3 negative psychological effects of it use 3 mixed-methods design and pre-study 3.1 mixed-methods design 3.2 qualitative pre-study 4 hypotheses development 4.1 it consumerization and unreliability 4.2 effect of general computer self-efficacy 4.3 effect of portfolio integration 4.4 the effect of unreliability on outcomes 5 empirical analysis 5.1 survey design and procedures 5.2 results 5.2.1 evaluation of the measurement model 5.2.2 evaluation of structural model and hypotheses testing 5.3 evaluation of the qualitative survey data 6 discussion 6.1 theoretical contribution 6.2 practical implications 7 limitations and conclusion appendix a – measurement items appendix b – factor loadings (main loading in bold font) appendix c – inter-factor-correlations (square root of ave in the diagonal) appendix d – results of plspredict appendix e – effect sizes (f² values) paper title (paper title style) australasian journal of information systems day, humphrey & cockroft 2017, vol 21, research on health information systems health hackathon design & participatory medicine 1 how do the design features of health hackathons contribute to participatory medicine? karen day school of population health the university of auckland new zealand. k.day@auckland.ac.nz gayl humphrey national institute for health innovation the university of auckland new zealand sophie cockcroft uq business school university of queensland australia abstract the hackathon concept is attracting interest as a vehicle for participatory development in both health and information systems. publically available datasets, cloud based data storage, and increasingly sophisticated analytical methods, combined with user friendly development tools for mobile devices are inspiring innovation in the participatory medicine space. this has the potential to disrupt traditional methods and deliver solutions more rapidly, and in a form more likely to meet requirements. in health applications this involves putting the patient and their supports at the centre of design. this work contributes to solving the challenges involved in bringing a diverse cohort of designers, developers, problem owners, healthcare providers, patients, and citizens together to solve user-driven self-care problems using technology. we use a descriptive case study approach focussing on two weekend-long hackathons dubbed “health hackathon: solving self-care”. we gather thick data from multiple sources according to the process defined by geertz (1994) first, to provide a rich picture of the role of hackathons in participatory medicine and second, to contribute evidence to the practise of running a hackathon. some key originalities of our work include seeking more candid responses via selfserve interviews. through this, controversially, we noted a marked emphasis on the creative process over concerns for privacy and ethics around the personal data cloud created by hackathon products. we build on existing theories of participatory medicine and emerging methodologies for conducting hackathons to provide evidence of the efficacy of the hacking approach both in terms of outcome and team dynamics. through interviews, observation, twitter feeds and a pre-survey, we identify a number of success factors including (1) group size, (2) maturity of the idea, (3) level of involvement of a mentor, and (4) involvement of students. in addition we identify five skills identified by successful health hackathon participants; knowledge, patient focussed skills, analytical skills, software design skills and professional perspective. in common with previous studies we find that there are considerable social benefits that accrue in running a hackathon. participants meet new people and learn first-hand of the challenges and opportunities provided by the skill sets and work environments of others. this work builds on the existing body of research concerning hackathons and in particular work in the context of participatory medicine keywords hackathon; self-care; participatory medicine; information systems design 1 introduction health is everybody’s business. people are living longer than ever before, but more people are now dealing with health issues that persist and for which there is no current cure, such as australasian journal of information systems day, humphrey & cockroft 2017, vol 21, research on health information systems health hackathon design & participatory medicine 2 diabetes, asthma, and cardiovascular disease. models of care are emerging and evolving, the most recognised of which is the chronic care model (ccm) (wagner, davis, schaefer, von korff, & austin, 2002). the ccm claims that effective care is based on ‘productive interactions’ between the people providing care and those needing and accessing it. widespread use of this model has demonstrated improvements in the health of people with long term health issues (gee, greenwood, paterniti, ward, & miller, 2015; stellefson, 2013). the model has been adjusted to incorporate ehealth components (gee et al., 2015). participatory medicine takes the ccm a step further, and redefines the ‘productive interactions’ relationship between user and provider of healthcare services. frydman (2010) defines participatory medicine on the e-patients.net website as “… a movement in which networked patients shift from being mere passengers to responsible drivers of their health, and in which providers encourage and value them as full partners.” patients are no longer simply interacting productively with healthcare providers, many are ‘driving’ the interactions and taking charge of their health by using information sources on the internet and collecting and using data for self-care purposes (auffray, charron, & hood, 2010; hood & auffray, 2013). this is evidenced by the quantified self movement in which people collect data about themselves (often with tools they create) to improve their self-awareness and decision making (bowen et al., 2013; swan, 2012). swan (2012) argues that crowdsourcing of data from populations and self-tracking by individuals should become part of the foundation of the future of health care. the increased involvement of all stakeholders, and ubiquity of information systems as integral parts of the health care system, raise opportunities for participatory medicine. how do hackathons fit in this new ecosystem? health hackathons bring together participants from all parts of the healthcare system as described in the ccm, as well as giving ‘patients’ 1 an opportunity to lead and fully participate in software innovation development. hackathons are weekend/week long events in which teams of people with mixed skills and backgrounds form groups and work intensively on a technology solution to a problem (chowdhury, 2012). our research question is, ‘how do the design features of health hackathons contribute to participatory medicine?’ the paper proceeds as follows. we discuss participatory medicine and its pivotal role in modern healthcare. we then explore how information systems for self-care are impacted by this. focussing on the health domain we then examine the evolving methodology implicit in hackathons. in the remaining sections the research design is described we present the findings via an in depth analysis of the thick data (geertz, 1994) generated by the process. finally, we discuss our hackathon process in the context of prior studies and make suggestions for success factors and in the conclusion describe the hackathon process can contribute to participatory design and participatory medicine in the future. 2 background 2.1 participatory medicine participatory medicine puts citizens in charge of their health (auffray et al., 2010; frydman, 2010; hood & auffray, 2013), i.e. as active participants in data collection and decision making about their own health. this is more than the relationship implied in the ccm, of proactive providers and informed responsive patients (gee et al., 2015; wagner et al., 2002). it extends the patient-clinician relationship from productive interactions to incorporate activities outside 1 in the ccm people interacting with providers of healthcare services are called ‘patients’. for the purpose of this article, they will be referred to as ‘citizens’ in the spirit of participatory medicine, were the person accessing and using healthcare services is actively involved in the design, delivery, and use of such services. australasian journal of information systems day, humphrey & cockroft 2017, vol 21, research on health information systems health hackathon design & participatory medicine 3 the clinician’s workplace, e.g. access to vast amounts of health information on the internet that citizens use to make their own healthcare decisions (bragazzi, 2013; flores, glusman, brogaard, price, & hood, 2013). data is collected about lifestyle, personal health indicators, e.g. blood pressure monitoring, and any other data of interest, as evidenced by the range of data collected by members of the quantified self. this extends clinicians’ roles to include supervision and coaching of citizens as they learn about their own diagnostic tests, treatments, and responses to treatments, in order to make informed decisions about their health (hood & auffray, 2013). the context of participatory medicine is systems. systems medicine incorporates predictive, preventive, personalised, and participatory principles (also referred to as ‘p4’) (hood & auffray, 2013). bragazzi (2013) proposes two additional principles psychocognitive and public describing the importance of personal responses to health, and also how people make their data public in pursuit of benefits for others. he includes psychocognitive and public components of p6 medicine as ‘community, collaboration, self-caring, co-creation, coproduction, and co-development using technologies delivered via the internet’ (pg 357). once diseases are seen in the context of a biological and social system, they can be predicted and prevented. this requires big data analytics, i.e. the analysis of very large, often unstructured data from multiple sources. consequently, diseases and patients can be stratified, and early and accurate diagnosis and prevention are more accessible. people are deemed to have a ‘personal data cloud’ about their own health profile (flores et al., 2013), which can be pooled with the clouds of other people and mined for health and care planning, and service delivery. diversity, multiple dimensions of data, sourced from many people, provide ‘thick data’ (geertz, 1994) for diagnosis and care planning, enabling the potential for citizens to plan their own healthcare. this cloud enables personalisation of healthcare, in the form of accurate stratified diagnosis and intervention. the ability to collect, collate, and analyse large amounts of highly detailed data, has enabled the view of ‘network of networks’ of biological phenomena. this ‘network of networks’ concept is in turn applied socially to people using the internet. social media and other facilities on the internet, such as google and facebook, have enabled networks of people to form and collectively respond to healthcare needs and thereby be collectively responsive to disease and wellness, e.g. peer to peer groups such as patientslikeme (bragazzi, 2013; flores et al., 2013; hood & auffray, 2013). citizens are becoming ‘active informed consumers interacting in digitally powered social networks’ (flores et al., 2013, pg 566). consequently, citizens design and co-produce innovative technologies with or without healthcare and information systems professionals to collect and respond to data. 2.2 involving citizens in information systems design the software design process is complex, involving many stakeholders with different points of view and needs. developing information systems typically involves four phases; planning, analysis, design, and implementation. the heart of the analysis phase is requirements elicitation: this is critical in health applications where user acceptance and usability are key issues. in recent years there has been an emphasis on user-centric design (andersen, bansler, kensing, moll, & nielsen, 2014; pilemalm & timpka, 2008). in health care, the users are typically identified as clinicians and others who work in health services (bahlol rahimi, vimarlund, & timpka, 2009). traditionally the requirements of stakeholders (usually health and care professionals) are assessed using interviews, observation, document analysis and questionnaires (dennis, wixom, & roth, 2014; shelly & rosenblatt, 2011). often, to be expedient at project sign off, the final agreement on requirements places more weight on the inputs of budget holders and high level decision makers. the practice team (clinicians and managers in healthcare services) becomes somewhat distanced from the final design decisions of the information system they will be using in the future. as a middle layer between healthcare management and consumers, information systems design in health has traditionally focussed on clinicians, such as doctors and nurses, and their information needs in carrying out their duties. citizens are often overlooked as key australasian journal of information systems day, humphrey & cockroft 2017, vol 21, research on health information systems health hackathon design & participatory medicine 4 stakeholders, even though the system development process can include consumer representatives (citizens). citizens are usually outnumbered in the context of userdriven/focussed design. nunes, verdezoto, fitzpatrick, kyng, gr, et al. (2015) found that many studies concerning the development of systems for self-care privileged a medical perspective. leading on from this nunes et al developed themes via a grounded theory literature review method that suggested further focus was needed on citizens’ day to day lived experience, and in using the existing frameworks and networks of carers and support people that citizens have around them. participatory design is a type of sociotechnical design that advocates involvement of users throughout the development process and whilst it has not been widely adopted in healthcare environments it is gaining currency (andersen et al., 2014; ballegaard, hansen, & kyng, 2008; bowen et al., 2013; cole-lewis et al., 2016; de silva, burstein, stranieri, williams, & rinehart, 2013; teixeira, saavedra, ferreira, santos, & ieee, 2011). in a participatory medicine context, the inclusion of citizens in information systems design for self-care is becoming essential. 2.3 the nature of hackathons health hackathons have gained significant traction as sources of medical innovation globally (depasse et al., 2014; mantzavinou & ranger, 2014). hackathons have the potential to deliver solutions where there are health discrepancies among citizens and where resources are limited. they are seen as a valuable way of discovering innovation for health care (walker & ko, 2016). chowdhury (2012) states that hackathons are a way for people (who would normally not be collaborating on a healthcare project) to collaborate and develop a solution, as a bottom-up form of change. in this way, citizens, healthcare professionals, and technology developers can be partners in the ‘productive interactions’ characteristic of the ccm. as the interactions are outside of the personal and private doctor-patient relationship, citizens are incorporated into the design space as equals. hackathons are inclusive and are not bounded by the conventions of traditional system development. the inclusive spirit of hackathons gives citizens a place in the design space. a hackathon is an event at which people create software solutions to real life problems, meeting and working with people and expertise otherwise not available (chowdhury, 2012). the events are not always called hackathons, e.g. sometimes they are called boot camps. the event is announced (usually on the internet), and people are invited to participate. there is no screening to select people with certain skills. there is no attempt to control who attends and what problems they want to solve, or what skills they bring (jones, semel, & le, 2015; komssi, pichlis, raatikainen, kindström, & järvinen, 2015). the only common attempt at control is to define the theme for a hackathon (komssi et al., 2015). briscoe and mulligan (2014) describe different types of hackathon, classifying them broadly as techno-centric and focus-centric. the techno-centric hackathons are typified by the development of open source software. the focuscentric hackathon is usually socially oriented and demography specific. focus-centric hackathons target software development to address or contribute to a social issue or a business objective, and are considered applied hackathons. hackathon event planning and the event itself follows a process. komssi et al (2015) describe three stages: pre-hackathon, hackathon, and post-hackathon. the pre-hackathon period involves participants creating ideas and preparing pitches of their ideas, and building teams to bring to the hackathon. the hackathon stage involves intense hacking and building of a prototype, while the post-hackathon stage is the period in which the new technology matures and new relationships are developed. de passe et al (2014) describe three stages that fit the komssi et al (2015) description of the hackathon stage. in the first stage pain points are identified for solutions to be developed. this is followed by the development of teams who brainstorm the problem and possible solution/s, build a prototype (or minimum viable product), and present it for judges to determine the best product prototype. anyone who is interested in an idea (pain point) that has been pitched is able to join/form a group. people discuss the idea with the person who presented it, make decisions about what skills are australasian journal of information systems day, humphrey & cockroft 2017, vol 21, research on health information systems health hackathon design & participatory medicine 5 required, and what they have to offer, and then decide (tentatively at first) to join a group (jones et al., 2015). hackathons have characteristics that identify the events and associated activities uniquely as hackathons. hackathons are intensive weekend or day-long events. the primary purpose of a hackathon is to solve problems (chowdhury, 2012; komssi et al., 2015). software is usually developed, although some organisations are using the hackathon model to achieve nonsoftware goals, e.g. the ‘transformathons’ used by the uk’s national health service to stimulate rapid and deep change (nhs, 2016). anyone can pitch an idea to solve a problem and anyone can offer to help design and develop a solution, i.e. hackathons are egalitarian (chowdhury, 2012; jones et al., 2015). individuals usually pitch the ideas and self-organised groups usually work on the solutions (jones et al., 2015). the development of a solution is treated as a competition in which the group with the best prototype wins (sometimes a monetary prize, sometimes mentoring and assistance with taking the product to market) (angelidis et al., 2016; depasse et al., 2014; komssi et al., 2015). hackathons typically operate on an open invitation for people to develop solutions to problems in the form of open software, thus enabling open participation for anyone with an interest in a problem and or its solution. people bring skills, experience, and expertise that is usually not available in the everyday workplace, thus enabling new ways of thinking and doing that are not accessible in everyday work. health care has historically shifted from a paternalistic approach (where clinicians and other healthcare providers were privileged and patients were passive recipients of care) to a systems approach (bragazzi, 2013). similarly, the design of healthcare information systems has privileged clinicians and other healthcare providers, essentially ignoring the idea that the consumer/patient/citizen is also an end-user. to enable the cultural shift to the new systems medicine approach, patients (citizens) are encouraged to participate in and take charge of their own healthcare. participatory medicine is part of systems medicine, which includes collecting data and using data analytics to explore so called “big data” (de silva, burstein, jelinek, & stranieri, 2015) in order to provide preventive, predictive, participatory, public, and psychocognitive health care. innovation involving information systems is essential for systems medicine to succeed. hackathons have no rules – anyone who is interested can identify pain points and work in self-organised groups to solve problems with information systems innovations. 3 ‘health hackathon: solving self-care’ as case study we present a case study of a real world application of participatory health information system development. the case study consists of two hackathons collaboratively hosted by the university of auckland, new zealand, in collaboration with hinz (health informatics new zealand), an organisation that represents health informaticians. the hackathons occurred on 13 – 15 february, 2015, and 18 20 march, 2016. the topic for both hackathons was ‘health hackathon: solving self-care’ focussing on the citizen’s (or patient’s) perspective. ethics approval for research on the hackathons was granted by the university of auckland human participants ethics committee on 16/12/2014, reference number 013391. a run sheet was designed for the weekend’s schedule, based on the process described by komssi et al (2015) and depasse et al (2014) and hackathon hosts whom kd interviewed while planning the 2015 event, e.g. two uk nhs hackathon hosts ("nhs hack day," 2015). after the event opening, people were invited to pitch their ideas (pain points), groups formed during the course of the evening, and worked on their solution over the course of the weekend. all groups presented their prototype to a panel of judges and audience on sunday afternoon. the hackathon was considered to be over after the four judges selected and announced the winner. the hackathons began at 6.00 pm on the friday and continued until 4.00 pm on the sunday. although the venue closed at 10.00 pm each evening, participants took tasks home and australasian journal of information systems day, humphrey & cockroft 2017, vol 21, research on health information systems health hackathon design & participatory medicine 6 continued working. the hackathon activities, schedule, and group dynamics and processes lent themselves to data collection for research purposes. 4 research design a hackathon follows the same process regardless of the topic (depasse et al., 2014; komssi et al., 2015), making available several opportunities for research data to be collected. table 1 shows how these opportunities were used to gather data. our multi-method approach aimed at triangulation, in which results overlaps from the different methods confirmed findings or presented different facets of the same phenomena (morse, 2015). we chose multiple methods to give us thick multi-faceted descriptions (geertz, 1994) especially if there was a low number of responses. we aimed to attract up to 120 participants for each hackathon. we collected data about the people and about the event. from the people about the people 2015 2016 about the event questionnaires: 1 how healthy do you feel, rand 36-item health survey 1.0 questionnaire items (rand health, 2016) 2 patient assessment of care received for a long term health issue (glasgow et al., 2005) 3 how confident are you about your health (smith, wallston, & smith, 1995) registration details (skills on offer, mentor offers) 110 people 81 people blog about each group (who, problem, innovation) 42 people in 7 groups 27 people in 6 groups evaluation photos of group work 33 31 twitter trail self-serve interview sets 11 5 observations and researcher reflections table 1: data sources the questionnaires were designed to elicit information about chronic conditions of participants (‘how healthy do you feel’ questionnaire) and perceptions of what it is like to be a patient in the nz health system (‘patient assessment of care received’ questionnaire) and how a person was coping with a long term health issue (‘how confident are you about your health’ questionnaire). it was assumed that the hackathon topic of ‘solving self-care’ would attract people with long term health issues that they wanted to solve, or at least influence solutions because of a lived experience with a health issue. the registration form was designed to ask for name (stripped for research purposes, but used for registration and name badge), day job, skills offered for use at a hackathon, offer to be mentor, offer to work in a team/group, dietary requirements (for catering purposes and not used for the research). the registration data were analysed for skills patterns. people took photos of one another and tweeted about the event using them, and the hackathon host (kd) took photos for her blog and research. a professional photographer donated his time and skill and took 186 photos of the sunday presentations and judges announcing the winner for the 2015 event. kd took 119 photos of the 2016 presentations and judging. there were two self-serve interviews: (1) working in my group, and (2) our innovation in 2050. participants were alone when completing each interview, which took between five and ten minutes each to complete. table 2 contains the interview questions. australasian journal of information systems day, humphrey & cockroft 2017, vol 21, research on health information systems health hackathon design & participatory medicine 7 interview 1: working in my group interview 2: our innovation in 2050 please start the recorder and tell us about your day job and why you decided to come to this hackathon, i.e. what work do you do and what are you hoping to do at this hackathon? what idea is your group working on? why did you choose that team? was there another idea you were also interested in? if so, why did you not join that team? from your point of view, what are the people in your group contributing to the development of the group’s idea? for example, what are their day jobs and what roles are they playing in the group? what is the most important aspect of the solution you’re working on? if your solution gets finished and people are able to use it one day, how and where would people use it? if you remove an aspect of your solution that stops it from working, what would that aspect be? what does success look like for your team? please start the recorder and tell us about your day job and what idea your group is working on. if you have a long term health issue, please briefly describe it. tell us about a health experience you’ve had that influences how you’re thinking about your group’s idea? if you don’t have a health issue, what brought you to this hackathon? how do you imagine how healthcare will work in 2050? if there were no risk of failure, how would your team’s idea have to change to be useful in 2050? table 2: self-serve interview questions we chose self-administered (self-serve) structured interviews to explore the hackathon experience. self-administered validated interviews have been designed for witness statements from children (gawrylowicz, memon, & scoboria, 2014), but self-administration usually applies to questionnaires. instructions are given to the respondent who then records answers to structured questions. since self-administered interviews are not frequently reported in the literature, we used the strengths and weaknesses of self-administration for questionnaires to assist our decision to use self-administered interviews. self-administration is perceived to improve the honesty and candidness of responses in the absence of the interviewer (rodriguez, sana, & sisk, 2015). the tone of hackathons is usually voluntariness and generosity; we wanted to preserve this tone in the ethics of our data gathering. the self-organising group dynamic that typically has fragile commitment that changes over the weekend (jones et al., 2015), influenced our decision to set up a quiet and private interview space, invite people to complete the interviews at a time that suited them during the hackathon, and be available to support them if they had any questions. self-administered interviews and questionnaires alike, especially if voluntary, are at risk of attracting fewer responses than assisted questionnaires and in-person interviews (rodriguez et al., 2015). the benefits of honesty and openness in a voluntary activity that fitted the dynamics of a hackathon weekend outweighed the risk of low response rates. triangulation and the use of thick description connected the data from many sources to create a rich picture. people responded to the interview invitation as per table 3. it was difficult for people to set aside time at the hackathon to complete the interviews. the first half of the weekend was too early for answers to our questions, and the commitment to the competition was so strong from australasian journal of information systems day, humphrey & cockroft 2017, vol 21, research on health information systems health hackathon design & participatory medicine 8 the saturday afternoon onwards that people only set aside time just before sunday lunch and after the judging session to complete some interviews. interview 1: working in my group 2: our innovation in 2050 year 2015 2016 2015 2016 number of interviews 9 0 11 5 table 3: uptake of self-serve interviews the blog (day, 2015) recorded aspects of the hackathon, particularly the judges’ choice of winner, the people’s vote after the hackathon, and any stories and photos that were written about. the twitter trail for both events contained comments and photos that could be used for understanding group dynamics and processes. researcher observations were noted in field notes. the data analysis was primarily qualitative. the registration data were analysed to find patterns of skills for the hackathon as a whole. they were also used to attribute skills (as defined by day jobs) to groups to demonstrate how skill/role mixes gravitated towards different topics associated with the groups. the photos were visually analysed to determine group dynamics and progress at certain times during each event. the blog was analysed to compare and contrast what the groups worked on during the two events. the content of researcher observation notes was analysed for references to how the groups worked and how the dynamics changed over time during the two events. 5 findings 5.1 what they achieved most of the 2015 problems were presented by clinicians, i.e. paediatrician (winning group’s leader), pharmacists, a gp, hospital doctor, and labsmart was presented by a primary service’s information systems manager. the spread was equal in 2016, where three of six groups were led by a public health professional (winning group leader), a pharmacist, and a clinical researcher. the balance were led by a postgraduate health informatics student, a person with a rare genetic disorder, and the general manager of a software company. table 4 indicates the problems and solutions worked on in groups. all the solutions required data collection from patients, and use of data by patients and their healthcare professionals. not all the solutions met the brief about self-care, but they all appear to meet the requirements of participatory medicine, e.g. citizen feels in charge of their health by gathering and using data. 2015 group name problem and solution fight the fever (winner) problem: reducing occurrence of rheumatic fever by increasing compliance of secondary prophylaxis. solution: a mobile app, encouraging self and whanau management (whanau ora approach (ministry of health, 2015)). talking pillbox problem: medication adherence for visually impaired people and those with language barrier. solution: talking label. simply press the label to hear your name, drugs, and instruction from the pharmacist in the language you understand. read my drugs problem: poor understanding of medical labels due to bad vision, language barrier, poor literacy. solution: qr codes printed on medication labels at pharmacy. patient scans the qr code using our app on their smart phone and will be presented with an enlarged version of the label, translated into their preferred language and with the option for the medication instructions to be read out audibly. australasian journal of information systems day, humphrey & cockroft 2017, vol 21, research on health information systems health hackathon design & participatory medicine 9 2015 lab smart problem: ensuring type 2 diabetes patients get the lab requisition forms and results prior to their 3-monthly gp visit to collect their medication prescription so that they can ask questions and be responsible in managing their health. solution: automate the form from gp systems (validated by gp) to the patient by email in the short term and directly to the lab and notification to the patient in the long term. similar process (long term) for: lab results, medicine prescription, referral to podiatrist. got pain problem: self-management of angina chest pain. aim to improve empowerment, conduct remote monitoring, reduce hospital admissions. solution: all with binary options based on american heart association guidelines and algorithm. hunger minder problem: global misunderstanding of the correct cues for hunger; physical hunger is not easily recognised. solution: hungerminder: a mobile phone app designed to give people the ability to recognise and be aware of hunger symptoms and to support them in their ability to make better eating decisions meal by meal. healthy me, healthy you problem: information mismatch between information and support people typically receive (too little, too much, wrong level) after a consultation of admission to hospital and what they want or need to self-care optimally. solution: a wiki-style mobile app that can tailor information and resources to needs of patient and they can choose what and when to access it. this includes a top tips library where patients and providers can share through a vetted approved process their ideas and top tips for living well with a range of conditions and issues, building an interactive community of interest. 2016 project bump (winner) problem: paper pamphlets with no way of measuring of health literacy solution: a web based mobile friendly platform scalable to long term conditions and the global market, an immersive experience for users loop problem: patients not being fully informed of their test results of understanding what the results mean. solution: website/app which allows patient to track the full journey of their test results. also incorporates decision support algorithm which provides patients with a suggested action to follow up their result. this will also boost health literacy. divert myself problem: lack of self-monitoring and self-management tools for diverticular disease solution: interactive mobile application for self-management, monitoring and wellbeing of people who have a lived experience of diverticular disease medlook problem: poor health literacy, high medication error, and poor medication adherence to prescriptions solution: care support applications such as medication reminders, medication tracking, utilizing a crowd sourced (from pharmacists) medication database healtheid problem: lack of verified and credentialed authentication available to systems which operate in health care solution: verified id, linked to credentialing available to systems as a service dotdotdot problem: people with rare conditions take longer to diagnose. they find it difficult to know what to expect (symptoms to diagnosis, surgeries, jargon). solution: smart profiles in which user comparisons can be make against aggregated data to match profiles. table 4: the problems and solutions of the groups for the two hackathons 5.2 who came and what they contributed hackathons are about people, the problems they want to solve, and the skills and experience they bring to the solutions they choose to work on. as per table 1, there were 110 registrations for the 2015 hackathon and 81 in 2016. the questionnaires revealed a health profile, and the registration data provided a day jobs and skills profile. australasian journal of information systems day, humphrey & cockroft 2017, vol 21, research on health information systems health hackathon design & participatory medicine 10 5.2.1 health profile from the questionnaires we determined that most of the 53 respondents (from both years) felt healthy; were confident about their health and how they managed it; 60% of respondents had long term health issues; and 36% knew someone with a long term health issue. they listed a variety of conditions ranging from food allergies to overweight, mental health issues, diabetes, and glaucoma, among others. in response to the positive questions in the ‘how confident do you feel’ questionnaire, the respondents were mostly positive and felt confident about managing their health (figure 1). this was supported in the questions that were negatively framed. 56% of the participants were in the age range of 18 – 39, one person was 15 years old, no-one was older than 69, and 53% were female. figure 1 ‘how confident do you feel about your health’ questionnaire there were only eight responses to the other two questionnaires. seven respondents indicated that they felt well and were mostly satisfied (agree rather than strongly agree) with the health services that they used. one person felt let down by the health system, which was confirmed in volunteered discussion during the course of the 2015 hackathon. this health profile is confirmed in the interviews. since the interviews were anonymous we cannot match them to the questionnaire respondents. the question, ‘tell us about a health experience you’ve had that influences how you’re thinking about your group’s idea’ revealed that some people came to the hackathon with an agenda to solve a personal health issue, while others did not link their health experience to their hackathon intentions. one person indicated that the process of lab tests and gp (general practitioner) visits was not logical – personal experience made it illogical to visit the gp to discuss diabetes progress if the lab tests are done after the appointment. another person had lost a significant amount of weight and became interested in learning how the body signals hunger, hence her interest in the hunger minder group. someone from the fight the fever group was taking medications regularly and, although was doing well, was interested in helping other people who may struggle to take their medications regularly. three participants from different groups did not have a long term health issue, and indicated that there was no personal experience that influenced their thinking. a doctor who had had an injury resulting in surgery and hospitalisation indicated that the experience had influenced her thinking, saying, ‘and when i became a patient i was so surprised at the incredible disempowerment i felt.’ she wanted to influence the development of a product that met the needs she had experienced, and from the citizen’s point of view. 0.00% 10.00% 20.00% 30.00% 40.00% 50.00% 60.00% 70.00% 80.00% 90.00% 100.00% i feel confident about my health i handle myself well with respect to my health i succeed in the projects i undertake to improve my health i'm generally able to accomplish my goals with respect to my health i am able to do things for my health as well as most other people r es p o n se s n =5 2 questions agree & strongly agree not sure disagree and strongly disagree australasian journal of information systems day, humphrey & cockroft 2017, vol 21, research on health information systems health hackathon design & participatory medicine 11 5.2.2 day jobs and skills profile their ‘day jobs’ and the skills they offered at the hackathon were varied. both hackathons attracted people whose roles fitted into groups that repeated in 2016. the types of groups and the differences between the two hackathons are shown in figure 2. figure 2: day jobs of people who registered the business and clinical roles show little difference between 2015 and 2016. the high number of academic people registering for the 2016 hackathon is a result of strong marketing at universities that did not happen in 2015. the 2015 hackathon’s date had been set for before semester because the campus would have unused space that the hackathon could take advantage of. we realised that students should have been given an opportunity to attend the hackathon, and so the 2016 date was set for three weeks after the beginning of semester to include students. more technical people registered in 2015 and slightly more clinical people registered in 2016. this resulted in some 2016 groups not having a software developer to assist them. some roles on the registration forms did not fit into any of the groupings that occurred from analysing the registration data, and they formed the groups ‘other’ and ‘combination’. the ‘other’ group for 2015 consisted of a student working part time in a job unrelated to health care, and a journalist. this group in 2016 consisted of a graphic designer, a forecourt attendant at a fuel station, and a glass installer. the ‘combination’ group in 2015 was made up of four people with previous clinical roles (a nurse, two pharmacists, and a doctor), who were now doing user experience design, business analysis, decision support systems, and software development respectively. two people had two concurrent roles; one was a company director and part time physiotherapist and the other was it support and planetarium director. in 2016 this group of four people each had two concurrent roles. two were medical education fellows and rmos (registered medical officers enrolled in specialisation study programme), one was a developer and a university student, and the other was a computer programmer who is also doing a master of health informatics. the day jobs showed up in the groups unpredictably as can be seen in table 5. each person chose to join a group on the basis of their interests and what they felt they could contribute. the mean number of people in a group was 6, and the mode 5. group size ranged from 2 to 10. the smallest groups could not leverage group dynamics, and the largest groups (8 and above) struggled with complex group dynamics. the two winning groups, fight the fever and bump, had six and seven members respectively, close to the mean group size. the success of this group size (6-7) is consistent with recommendations in raatikainen et al (2013). 0.0 5.0 10.0 15.0 20.0 25.0 30.0 35.0 40.0 p ar ti ci p an ts roles 2015 2016 australasian journal of information systems day, humphrey & cockroft 2017, vol 21, research on health information systems health hackathon design & participatory medicine 12 the leaders of the groups had a vested interest in their group’s progress because the group’s work was based on those people’s ideas pitched on the friday. in the interviews about their innovation in 2050, those who were group leaders indicated that their idea would have solved the problem it was currently responding to and therefore would not exist in 2050. in contrast software developers asked to be assigned to groups, as indicated by a young coder saying, ‘tell me which group to join so that i can write code.’ r o le t y p e s c o n su m er a ca d em ic t ec h n ic a l b u si n es s c li n ic a l o th er c o m b in a ti o n n o t in d ic a te d t o ta l g r o u p m e m b e r s 2015 fight the fever 1 2 1 2 6 talking pillbox 1 1 2 read my drugs 1 1 1 1 4 lab smart 1 5 1 1 8 got pain 2 2 3 7 hunger minder 3 3 3 1 10 healthy me, healthy you 1 3 1 2 1 1 9 2016 bump 2 1 2 1 1 7 loop 3 1 1 5 divert myself 2 2 1 5 medlook 2 2 4 healtheid 1 3 1 5 dotdotdot 2 2 table 5: roles in the groups not all people who registered answered the question in the registration form about the skills they brought to the hackathon. several left the answer blank, 17 indicated their role (e.g. nurse, doctor, pharmacist, software developer), seven stated their interest (present an idea; interest in telemonitoring, neuroscience, health promotion and behaviour change, working on an interactive app, ‘bring innovative solutions to health care’), 24 described experience, and 48 listed skills. the skills fell into four themes: clinical (e.g. medical knowledge, self-care programme development), it (e.g. software development, database analysis), business (e.g. process analysis, patent, marketing), and design (e.g. creative thinking, user experience design), as can be seen in table 6. australasian journal of information systems day, humphrey & cockroft 2017, vol 21, research on health information systems health hackathon design & participatory medicine 13 clinical it business design knowledge 9 3 12 7 patient focussed skills 7 3 analytical skills 1 2 11 software design skills 25 4 perspective 2 2 7 2 table 6 thematic breakdown of skills listed in registration form these themes in turn fell into five types of skills. people brought knowledge (e.g. medical/business/technical/health informatics knowledge) and their own perspective (e.g. junior doctor, project management, creative thinker). there is no overlap between clinical and business people with software designers, as would be expected. the overlap between clinical and design skills took the form of user experience design and patient centred design. 5.3 why they came and how their health issues influenced innovation (or not) the reasons interviewees gave for attending the hackathon were mostly about meeting people and seeing how they use their different skills to solve problems. they felt they had something to contribute but did not frame their contribution as citizens who access and use healthcare services. one person encapsulated the hackathon experience, saying, ‘and i thought, well i first wanted to see how it works, what it is, how i could have fun with unknown people suddenly forming into a team and trying to develop something in this short time frame. and to see if there is, if it can really be done.’ the interviewees described their groups’ dynamics by privileging certain roles, depending on the role and perspective of the person talking. clinicians privileged clinical contributions and then indicated that they had attended the hackathon to leverage technical skills. a developer from the same group talked first about other developers and technically skilled people and then about clinicians. this implies that people work and think from their perspective first, and if not reminded of other perspectives may overlook the role of people living with long term health issues in innovation development. of the interviewees working in the labsmart group, one person did not have a health issue, one had arthritis, and the other had type 2 diabetes. the way the two with health issues talked about their contribution to the group’s work was couched in personal experience tones rather than the skills they brought to the group’s solution. the attraction of the hackathon was not about the event’s theme, but an opportunity to develop something that put citizens in ‘the driver’s seat’ (frydman, 2010). one group ignored the brief and focussed on a problem that their leader was passionate about, e.g. healtheid, arguing that big data requires a different approach to identity authentication and management, especially if citizens collect, analyse, and use their own data, or personal data clouds as described by flores et al (2013). the question about what healthcare could be like in 2050 raised insights such as the ability to use technology to make healthcare processes, activities, and decision making much more transparent. some participants focused on efficiency gains, reduction in costs of care, automation, and the use of more sophisticated technology to simplify care and make it easier to access. others indicated that citizens should take more responsibility for their health, be more accountable, and become more involved. 5.4 the hackathon process and dynamics ideas that were pitched on the friday evening ranged from well-formed, thoughtful innovations, to fuzzy possibilities. fifteen ideas were pitched in 2015, one person told a story about a catastrophic stroke she had experienced and offered a patient’s perspective, and one australasian journal of information systems day, humphrey & cockroft 2017, vol 21, research on health information systems health hackathon design & participatory medicine 14 person simply offered software development skills saying that she had no healthcare experience but wanted to help. in 2016 seven people pitched ideas, and no-one took the stage for other reasons. there appeared to be a pattern of interpersonal dynamics at both hackathons that contributed to the spirit of the ‘hack way’ as described by chowdhury (2012) and described as social benefits by raatikainen et al (2013). people arrived on the friday, keen to share their ideas and listen to others. the dynamics shifted during the weekend as described by jones (2015), as people explored the ideas of others, tentatively indicated interest, which grew into tentative commitment. by saturday morning the commitment was confirmed, as noted in a mentor’s twitter comment, ‘day 2 groups formed – stormed and in deep’. in both hackathons there was a group that completely restructured the next day (talking pillbox and loop). in 2015 several pharmacists pitched ideas about people taking their medicines reliably and with insight. one group formed to address this problem, but two members of the group disagreed with the proposed solution and formed a new group. loop had a good foundation but some determined that it needed to change in focus to meet the brief of the hackathon topic. by noon on saturday these two groups had appeared to resolve their differences and were committed to their collective goals. this dip in morale and recovery confirms this finding of previous hackathon case studies (raatikainen et al., 2013). all the groups spent the morning brainstorming and exploring a broad range of ways to solve the problem they had chosen. mentors challenged the content that was emerging from the brainstorming activities. at this point, all the groups demonstrated a shift in the problem definition (it was less clear than in the pitching session), and possible solution (new ideas were being explored). groups were ‘failing early, and pivoting fast’ as described by depasse et al (2014), changing direction to what could be more successful. the challenges from the mentors appeared to unsettle group members. there was lively debate. conflicts in perspective, content, and purpose surfaced and groups attempted to resolve them by accommodating, modifying, and/or discarding suggestions. mentors focussed groups on what their minimum viable product would be on sunday afternoon, which appeared to sharpen the intensity of the groups’ activities. the challenge about how they planned to commercialise their product appeared to increase their anxiety about being able to complete a prototype in time. one participant commented on twitter saying, ‘was a super stressful weekend but extremely rewarding!’ the challenges did not appear to divert them from their goals, but did help them pivot to more useful ways of achieving their goals. by the saturday evening the groups had become quiet and there was evidence of hard work in post-it notes and butcher’s paper with brainstormed ideas posted on the walls. some groups were using ‘lean canvas’ charts (supplied by a mentor). the white boards had mind maps, diagrams, and notes, which had been written and rewritten during the course of the day. group members took on individual tasks or paired up to complete tasks. on the final day, declining mentor interactions, and with the requirements relatively stable, groups focussed on refining their prototypes and presentations. groups appeared to resent interruptions and were focussed on single-mindedly completing their minimum viable product sufficiently to present it to the judges after lunch. the stress was palpable until after the winners were announced by the judges. 6 discussion and conclusion our research question was, ‘how do the design features of health hackathons contribute to participatory medicine?’ we hosted two hackathons titled, ‘health hackathon: solving selfcare’ and gathered data to form a thick description of how health hackathons enable participatory medicine as a form of participatory information systems design. people did not register for the hackathon on the basis of a lived experience with a long term health issue. however, many of those who attended did have personal experience of a health australasian journal of information systems day, humphrey & cockroft 2017, vol 21, research on health information systems health hackathon design & participatory medicine 15 issue or knew someone who had experienced a long term health issue. those who responded to the questionnaires were confident about their ability to take care of their health, were generally satisfied with the healthcare services they received, and felt healthy. the interviews revealed that people did not specifically use their health issue to influence how a group worked on a solution but found that the influence was present in their contributions and decisions. their day jobs and skills mixes mirrored those that are carefully selected in the usual information systems design process, i.e. healthcare stakeholders include clinicians and consumers and the rest of the team consists of developers and analysts. in contrast, hackathon design teams form spontaneously. it is not typical for consumers who have long term health issues to be included in software development requirements gathering or be directly involved in the development process (nunes, verdezoto, fitzpatrick, kyng, grönvall, et al., 2015). hackathons offer an unprecedented opportunity to include citizens and enable them to drive the requirements. citizens being in charge of their health and how they use services is integral to participatory medicine (flores et al., 2013; hood & auffray, 2013), and hackathons offer opportunities for this kind of citizen investment in their own health. hackathons are a radical departure from standard information systems design processes. requirements gathering, iterative design to meet the requirements, and testing are not part of the hackathon process. that is not to say that the process is at odds with information system development. we propose the contrary that hackathons, and their focus on the minimum viable product, are complimentary to newer agile methods in development with a focus on synergy between stakeholder groups and creativity. most of the problems and associated solutions proposed at our hackathons represented pain points experienced by clinicians, e.g. fight the fever was presented and led by a paediatrician, and hunger minder arose from research conducted by a general practitioner. the self-organised groups, however, were not predetermined and people became committed to them out of either a personal experience of the problem or an interest in it (usually a personal or clinical interest), or to use skills to build a solution. anyone could join a group. people described the group dynamics according to their own world view, rather than privileging one person’s skill set or point of view or requirements over another. all who joined a group found meaningful ways to contribute to the group’s product design. they brought their own perspective and privileged that perspective. typically clinicians are considered to be end users (b. rahimi & vimarland, 2007). in hackathons citizens and clinicians share the end user role. the minimum viable products of software that emerged from our health hackathons were designed for data collection about health issues and the associated self-care activities and requirements. it appears from our findings that hackathon members were not concerned about the personal data cloud that would be collected as a result of their solutions. in contrast, one group, healtheid, was concerned about interoperability issues, and identity authentication and management for big data collections from multiple sources. however, even the judges did not perceive this new role of identity management and disqualified the team because they did not meet the brief of self-care. the participatory medicine potential of big data and personal data clouds as described by flores et al (2013) was not perceived by anyone other than the healtheid group. the solutions that were presented to the judges in both hackathons relied heavily on mobile phone app designs. clinicians use smartphone apps for drug guides, medical calculations and billing (franko & tirrell, 2012). apps are also available for citizens to use for monitoring aspects of health and wellbeing, and the most commonly addressed health conditions are diabetes, diet and exercise, smoking cessation and mental health (boulos, brewer, karimkhani, buller, & dellavalle, 2014; martínez-pérez, de la torre-díez, & lópez-coronado, 2013) and the most common intervention strategy is tracking. amongst apps used to monitor health, bennett (2015) notes the difference between apps that protect health and encourage good health behaviours and those that purport to monitor physical parameters and possibly detect the presence of disease. since the topic of our hackathons was self-care, the latter group of apps featured. australasian journal of information systems day, humphrey & cockroft 2017, vol 21, research on health information systems health hackathon design & participatory medicine 16 it is not surprising that mobile phone apps lend themselves to the hackathon environment, where rapid development of a minimum viable product and intensive work on construction of a solution constrain the scope of and platform for a solution. it is frequently stated in the literature that there is a glut of health apps on the market, and further that there is a significant gap between consumer products and scientific validation with developers seeing success in terms of revenue rather than improvement in patient outcome (boulos et al., 2014). these apps are often developed without citizen and/or clinician involvement and are often not evidence based. with this problem in mind stoyanov et al (2015) devised a score card to empirically evaluate the quality of health apps. this comprehensive tool includes issues of design, security, functionality and quality of information. beyond app quality, the risk that poor apps pose to users was considered by lewis and wyatt (2015) who developed an algorithm for statistically modelling the risks of adverse events associated with using health apps. the need for consumer and patient and practitioner involvement identified by krebs et al (2004) lends weight to the hackathon approach. we recommend that future hackathons make such evaluation tools available to enable groups to think through their solution in the context of safety from the outset, and how much future work is needed to prevent possible adverse events. the group dynamics shifted during the weekend from enthusiastic interest in a topic, through the ‘forming, storming, norming, and performing’ phases from tuckman’s model of group dynamics (bonebright, 2010). commitment to the group’s final minimum viable product grew as time passed, and the intensity of the contributions and discussions grew until they were ready to present to the judges. challenging discussions from mentors at the beginning were welcomed but interruptions were resented in the hours preceding the judging. the mentors played a crucial role in focussing the groups, and challenging individuals to think and work creatively. this is supported in the literature, where trainer et al (2014) have indicated that without mentors novice groups of developers and innovators would flounder. a hackathon is a micro-network of people from different walks of life, some of whom have experience of long term health issues, which influence how they think and act about solutions to problems pitched by others. the hackathon gives citizens, the ‘practice team’ of clinicians and managers, and software developers space to create innovative solutions to problems. the group dynamic shifts as different aspects of the solution are developed, and the citizen has the opportunity to be ‘in the driving seat’ as and when necessary. there were some limitations to this study. the events were limited to a weekend each, with small numbers of people contributing to the questionnaires and self-serve interviews. the hackathon process limited the number of opportunities for people to respond to the interviews and questionnaires. consequently, there was a low response rate and the interviews were inconsistently completed. to ensure rigour we used multiple methods to derive a thick description as outlined by geertz (1994) and triangulated data that overlapped between methods (morse, 2015). hackathons add a dimension to traditional participatory information systems design processes. due to the short time for developing a response to a problem (one weekend) the usual rigour is not present in terms of requirements gathering before software development and testing and evaluation afterwards. this should fall into the preand post-hackathon periods described by komssi et al (2015). hackathons offer opportunities for citizens with long term health issues to share the end-user role that clinicians usually occupy, or to simply occupy the role fully by providing solutions to their own pain points, thereby taking charge of their own health (auffray et al., 2010; flores et al., 2013; hood & auffray, 2013) or occupying the driver’s seat of their healthcare situation (frydman, 2010). hackathons offer opportunities for information systems development in creating the future of healthcare, as described by swan (2012). the big data that results from participatory medicine becomes part of public and population and personal health in ways not yet perceived, and citizens take charge of their health in unprecedented ways using solutions arising from prototypes developed collaboratively in hackathons. australasian journal of information systems day, humphrey & cockroft 2017, vol 21, research on health information systems health hackathon design & participatory medicine 17 references andersen, t., bansler, j., kensing, f., moll, j., & nielsen, k. d. (2014). alignment of concerns: a design rationale for patient participation in ehealth. in r. h. sprague (ed.), 2014 47th hawaii international conference on system sciences (pp. 2587-2596). angelidis, p., berman, l., casas-perez, m. d. l. l., celi, l. a., dafoulas, g. e., dagan, a., lopez, j. n., otine, c., paik, k, rojas-potosi, l., symeonidis, a. l., winkler, e. osorio-valencia, j. s. (2016). the hackathon model to spur innovation around global mhealth. journal of medical engineering & technology, 1-8. auffray, c., charron, d., & hood, l. (2010). predictive, preventive, personalized and participatory medicine: back to the future. genome medicine, 2(8), 57. ballegaard, s. a., hansen, t. r., & kyng, m. (2008). healthcare in everyday life: designing healthcare services for daily life. paper presented at the proceedings of the sigchi conference on human factors in computing systems. bennett, d. b. (2015). importance of distinguishing between different types of health app. bmj, 350, h2334. bonebright, d. a. (2010). 40 years of storming: a historical review of tuckman's model of small group development. human resource development international, 13(1), 111-120. 10.1080/13678861003589099 boulos, m. n. k., brewer, a. c., karimkhani, c., buller, d. b., & dellavalle, r. p. (2014). mobile medical and health apps: state of the art, concerns, regulatory control and certification. online journal of public health informatics, 5(3) 10.5210/ojphi.v5i3.4814 bowen, s., mcseveny, k., lockley, e., wolstenholme, d., cobb, m., & dearden, a. (2013). how was it for you? experiences of participatory design in the uk health service. codesigninternational journal of cocreation in design and the arts, 9(4), 230-246. 10.1080/15710882.2013.846384 bragazzi , n. l. (2013). from p0 to p6 medicine, a model of highly participatory, narrative, interactive, and" augmented" medicine: some considerations on salvatore iaconesi's clinical story. patient preference & adherence, 2013 (7), 353 359. https://doi.org/10.2147/ppa.s38578 chowdhury, j. (2012). hacking health: bottom-up innovation for healthcare. technology innovation management review, 2(7), 31. cole-lewis, h. j., smaldone, a. m., davidson, p. r., kukafka, r., tobin, j. n., cassells, a., mynatt, e. d., hripcsak, g. mamykina, l. (2016). participatory approach to the development of a knowledge base for problem-solving in diabetes self-management. international journal of medical informatics, 85(1), 96-103. 10.1016/j.ijmedinf.2015.08.003 day, k. (2015). day by day. retrieved 5 may 2016, from www.kdayinformatix.wordpress.com de silva, d., burstein, f., jelinek, h. f., & stranieri, a. (2015). addressing the complexities of big data analytics in healthcare: the diabetes screening case. australasian journal of information systems, 19, s99-s115. de silva, d., burstein, f., stranieri, a., williams, k., & rinehart, n. (2013). a participatory information management framework for patient centred care of autism spectrum disorder. paper presented at the 24th australasian conference on information systems (acis). dennis, a., wixom, b. h., & roth, r. m. (2014). systems analysis and design: john wiley & sons. depasse, j. w., yost, a., carroll, r., santorino, d., ippolito, a., chu, z., & olson, k. r. (2014). less noise, more hacking: how to deploy principles from mit's hacking medicine to australasian journal of information systems day, humphrey & cockroft 2017, vol 21, research on health information systems health hackathon design & participatory medicine 18 accelerate health care. international journal of technology assessment in health care, 30(3), 260-264. 10.1017/s0266462314000324 flores, m., glusman, g., brogaard, k., price, n. d., & hood, l. (2013). p4 medicine: how systems medicine will transform the healthcare sector and society. personalized medicine, 10(6), 565-576. 10.2217/pme.13.57 franko, o. i., & tirrell, t. f. (2012). smartphone app use among medical providers in acgme training programs. journal of medical systems, 36(5), 3135-3139. frydman, g. a. (2010). a patient-centric definition of participatory medicine. retrieved from http://e-patients.net/archives/2010/04/a-patient-centric-definition-of-participatorymedicine.html gawrylowicz, j., memon, a., & scoboria, a. (2014). equipping witnesses with transferable skills: the self-administered interview©. psychology, crime & law, 20(4), 315-325. http://dx.doi.org/10.1080/1068316x.2013.777961 gee, p. m., greenwood, d. a., paterniti, d. a., ward, d., & miller, l. m. s. (2015). the ehealth enhanced chronic care model: a theory derivation approach. journal of medical internet research, 17(4) 10.2196/jmir.4067 geertz, c. (1994). thick description: toward an interpretive theory of culture. in m. martin & l. c. mcintyre (eds.), readings in the philosophy of social science (pp. 213-231): mit press. glasgow, r. e., wagner, e. h., schaefer, j., mahoney, l. d., reid, r. j., & greene, s. m. (2005). development and validation of the patient assessment of chronic illness care (pacic). medical care, 43(5), 436-444. hood, l., & auffray, c. (2013). participatory medicine: a driving force for revolutionizing healthcare. genome medicine, 5(12), 1-4. doi: 10.1186/gm514 jones, g. m., semel, b., & le, a. (2015). “there's no rules. it's hackathon.”: negotiating commitment in a context of volatile sociality. journal of linguistic anthropology, 25(3), 322-345. http://dx.doi.org/10.1111/jola.12104 komssi, m., pichlis, d., raatikainen, m., kindström, k., & järvinen, j. (2015). what are hackathons for? ieee software, 32(5), 60-67. 10.1109/ms.2014.78 krebs, a. m., dorohonceanu, b., & marsic, i. (2004). supporting collaboration in heterogeneous environments. journal of management information systems, 20(4), 199 228. cle: http://dx.doi.org/10.1080/07421222.2004.11045787 lewis, t. l., & wyatt, j. c. (2015). app usage factor: a simple metric to compare the population impact of mobile medical apps. journal of medical internet research, 17(8) 10.2196/jmir.4284 mantzavinou, a., & ranger, b. j. (2014). health hackathons drive affordable medical technology innovation through community engagement. retrieved 6/9/2016 from http://cooperation.epfl.ch/ martínez-pérez, b., de la torre-díez, i., & lópez-coronado, m. (2013). mobile health applications for the most prevalent conditions by the world health organization: review and analysis. journal of medical internet research, 15(6), e120. 10.2196/jmir.2600 ministry of health. (2015). whanau ora programme. retrieved 6/9/2016, from http://www.health.govt.nz/our-work/populations/maori-health/whanau-oraprogramme morse, j. m. (2015). critical analysis of strategies for determining rigor in qualitative inquiry. qualitative health research, 25(9), 1212-1222. 10.1177/1049732315588501 australasian journal of information systems day, humphrey & cockroft 2017, vol 21, research on health information systems health hackathon design & participatory medicine 19 nhs hack day. (2015). retrieved july, 2016, from http://nhshackday.com/about/team/ nhs, u. (2016). nhs transformathon. retrieved 1/9/2016, from http://theedge.nhsiq.nhs.uk/transformathon/ nunes, f., verdezoto, n., fitzpatrick, g., kyng, m., gronvall, e., storni, c. (2015). self-care technologies in hci: trends, tensions, and opportunities. acm trans. comput.-hum. interact., 22(6), 1-45. 10.1145/2803173 nunes, f., verdezoto, n., fitzpatrick, g., kyng, m., grönvall, e., & storni, c. (2015). self-care technologies in hci: trends, tensions, and opportunities. acm transactions on computer-human interaction (tochi), 22(6), 33. 10.1145/2803173 pilemalm, s., & timpka, t. (2008). third generation participatory design in health informatics making user participation applicable to large-scale information system projects. journal of biomedical informatics, 41(2), 327-339. 10.1016/j.jbi.2007.09.004 raatikainen, m., komssi, m., dal bianco, v., kindstöm, k., & järvinen, j. (2013). industrial experiences of organizing a hackathon to assess a device-centric cloud ecosystem. paper presented at the computer software and applications conference (compsac), 2013 ieee 37th annual. rahimi, b., & vimarland, v. (2007). methods to evaluate health information systems in healthcare settings: a literature review. journal of medical systems, 31, 397 432. 10.1007/s10916-007-9082-z rahimi, b., vimarlund, v., & timpka, t. (2009). health information system implementation: a qualitative meta-analysis. journal of medical systems, 33(5), 359. 10.1007/s10916008-9198-9 rand health. (2016). medical outcomes study: 36 -item short form survey instrument. retrieved 5 may 2016, from http://www.rand.org/health/surveys_tools/mos/mos_core_36item_survey_print.htm l rodriguez, l. a., sana, m., & sisk, b. (2015). self-administered questions and interviewer– respondent familiarity. field methods, 27(2), 163-181. 10.1177/1525822x14549315 shelly, g., & rosenblatt, h. j. (2011). systems analysis and design: cengage learning. smith, m. s., wallston, k. a., & smith, c. a. (1995). the development and validation of the perceived health competence scale. health education research, 10(1), 51 64. stellefson, m. (2013). the chronic care model and diabetes management in us primary care settings: a systematic review. preventing chronic disease, 10, 120180. doi: http://dx.doi.org/10.5888/pcd10.120180 stoyanov, s. r., hides, l., kavanagh, d. j., zelenko, o., tjondronegoro, d., & mani, m. (2015). mobile app rating scale: a new tool for assessing the quality of health mobile apps. jmir mhealth and uhealth, 3(1) swan, m. (2012). health 2050: the realization of personalized medicine through crowdsourcing, the quantified self, and the participatory biocitizen. journal of personalized medicine, 2(3), 93-118. 10.3390/jpm2030093 teixeira, l., saavedra, v., ferreira, c., santos, b. s., & ieee. (2011). using participatory design in a health information system 2011 annual international conference of the ieee engineering in medicine and biology society (pp. 5339-5342). trainer, e., chaihirunkarn, c., & herbsleb, j. (2014). the big effects of short-term efforts: mentorship and code integration in open source scientific software. journal of open research software, 2(1), e18. http://doi.org/10.5334/jors.bc australasian journal of information systems day, humphrey & cockroft 2017, vol 21, research on health information systems health hackathon design & participatory medicine 20 wagner, e. h., davis, c., schaefer, j., von korff, m., & austin, b. (2002). a survey of leading chronic disease management programs: are they consistent with the literature? journal of nursing care quality, 16(2), 67-80. walker, a., & ko, n. (2016). bringing medicine to the digital age via hackathons and beyond. journal of medical systems, 40(4), 1-3. 10.1007/s10916-016-0461-1 copyright: © 2017 day, humphrey & cockroft. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems sultana, teoh & karanasios 2022, vol 26, research article the impact of blockchain on supply chains 1 the impact of blockchain on supply chains: a systematic review jakia sultana rmit university, melbourne, australia say yen teoh rmit university, melbourne, australia sayyen.teoh@rmit.edu.au stan karanasios university of queensland, brisbane, australia abstract supply chains face many challenges around coordination, information asymmetry, quality assurance, complex disruptions, and traceability. blockchain is arguably a technology that can address these challenges and make a significant impact. to shed light on the impact of blockchain, we undertake a cross-discipline systematic literature review on blockchain and supply chains. this review focused on identifying blockchain’s current and proposed impacts on the supply chain at three levels: organisational, inter-organisational, and industry. the findings identified twelve core supply chain themes across pre-implementation, postimplementation, and emerging tensions associated with adopting blockchain. these findings extend knowledge by going beyond understanding blockchain and its application and articulating multi-levels of impacts. based on our review, we propose future research directions. by providing an overview of the current impact of blockchain, the review also offers insights to help managers to make informed decisions around the implementation and use of blockchain in supply chains. keywords: blockchain, impact, levels, supply chain, systematic review. 1 introduction there has been rapid growth in blockchain research across various academic disciplines. within the field of information systems (is), researchers view blockchain as a ‘revolution’ (gomber et al., 2016) or as ‘transformative’ (beck et al., 2016; kshetri, 2018; puschmann et al., 2018), echoing trends in the technology-practitioner, media and the wider academic community. most early research focused on blockchain in banking and capital markets (beck et al., 2016; egelund-müller et al., 2017; guo & liang, 2016). recently academics, media and industry have discussed blockchain in different settings, such as in business (attaran & gunasekaran, 2019; li et al., 2018), academia (treiblmaier et al., 2020), healthcare (kumari & saini, 2020), supply chain (di vaio & varriale, 2020; korpela et al., 2017) and government (allen et al., 2019). one area that has received much attention is the supply chain (bai & sarkis, 2020; kshetri, 2018; treiblmaier, 2018). supply chains face challenges such as coordination, information asymmetry, quality assurance, the complexity of disruptions, and traceability (chang et al., 2020; etemadi et al., 2021). organisations in supply chains also struggle with counterfeited products and illegal practices (de boissieu et al., 2021). numerous examples of such practices exist. for instance, the toxic industrial chemical melamine was found in milk powder in china, australasian journal of information systems sultana, teoh & karanasios 2022, vol 26, research article the impact of blockchain on supply chains 2 horse meat was found in beef labelled packaging in europe, and corn syrup was mixed with honey in australia (bedo, 2018; behnke & janssen, 2020). according to the fbi’s intellectual property rights unit, fraudulent imitations reportedly result in organisations losing almost $600 (usd) billion yearly (schlesinger & day, 2019). such illegal practices trade approximately 3.3% of global trade and continuously growing (kapadia, 2019). according to the international chamber of commerce, by 2022, the counterfeited product trade will reach $4.2 trillion and risk 5.4 million jobs (borrell, 2020). blockchain is arguably critical to addressing these issues (chang et al., 2019; kshetri, 2018; min, 2019; tönnissen & teuteberg, 2020). existing studies have predominantly examined how blockchain could benefit supply chains (akram & bross, 2018; korpela et al., 2017; sternberg & baruffaldi, 2018) or focused on the design of blockchain solutions (arena et al., 2019; chen et al., 2020; yong et al., 2020). however, a lack of understanding of blockchain’s impact on supply chains remains. this is evident in previous reviews conducted, which, while useful (summarised in table 1), are limited in understanding impacts across the supply chain in three main ways. first, they focused on the suitability of blockchain to supply chains. because of this, a second limitation is that they tend to offer a techno-optimistic view because they consider potential impact rather than actual impact. third, they zoom in on limited aspects of blockchain of the supply chain (such as information sharing) and hence do not provide an overarching picture of impact across the supply chain network. study focus blossey et al. (2019) explored different applications of blockchain in the supply chain gurtu and jestin (2019) the potential value of blockchain in supply chain management pranto et al. (2019) understanding the applicability of blockchain technology queiroz et al. (2019) blockchain applications in supply chain management, challenges of adoption and plan for future research wang et al. (2019) blockchain impact on future supply chain practices and policies chang and chen (2020) potential application and future directions pournader et al. (2020) co-citation analysis in four clusters and future directions wan et al. (2020) impact of blockchain technology for information sharing within a supply chain lim et al. (2021) exploring research on blockchain-based supply chains in terms of value, scholarly attractions, methodologies and industries table 1. previous literature reviews in particular, existing research and reviews do not provide a comprehensive picture of the impact of blockchain at different levels of the supply chain (e.g., organisational level, interorganisational level, industry level). understanding blockchain impact on the supply chain and its succession of impact in a multitiered supply chain is crucial (i) to obtain a deeper understanding of the blockchain and supply chain landscape and realise the value of blockchain, and (ii) to guide future blockchain research. hence, guided by this motivation, we undertake a systematic review of blockchain and supply chain papers to address the research question: what is the impact of blockchain on supply chains? by addressing this research question, we address calls to go beyond understanding what blockchain is and explore its applications (risius & spohrer, 2017). while studies tend to focus on the leading business blockchain solution, including ethereum, ibm blockchain, hyperledger fabric, hyperledger sawtooth, r3 corda, quorum, etc. (lawton, 2021); our review is agnostic in terms of blockchain solutions, instead focuses on blockchain in general. australasian journal of information systems sultana, teoh & karanasios 2022, vol 26, research article the impact of blockchain on supply chains 3 the remainder of the paper is structured as follows. first, we outline our review process and then we present the findings of our systematic review. our discussion focuses on the impact of blockchain in supply chains at the organisational, inter-organisational, and industry levels, and we outline implications for research and practice. the paper concludes by summarising the key takeaways and identifying the limitations of our research. 2 the review a systematic literature review can be described as “a systematic, explicit, and reproducible method for identifying, evaluating, and synthesising the existing body of completed and recorded work” (fink, 2005, p. 3), which stands on its own as a complete research activity (okoli & schabram, 2010). systematic literature reviews have been crucial for (i) analysing the research progress of a particular phenomenon, (ii) advancing the depth of understanding of a research topic; (ii) synthesising conceptual debate and empirical evidence; (iii) developing theories; and (iv) identifying and providing conceptual pillar for subsequent topics or research domains that need more attention (okoli, 2015). they help become oriented in an emerging domain and aid in teaching, research, informing policy and supporting practice (okoli & schabram, 2010). the purpose of the systematic review in this paper is to analyse research progress on blockchain in supply chains by synthesising themes to offer an overarching understanding of the impact of blockchain. such knowledge helps to set the foundation to outline future directions of blockchain research relevant to supply chains. figure 1. prisma framework of the selection process, based on moher et al. (2009) our review followed the “preferred reporting items for systematic review and metaanalyses (prisma)” guidelines for systematic reviews (moher et al. 2009). a rigorous and explicit review methodology ensured quality and transparency throughout the review process (okoli, 2015). it also helped mitigate biases in the selection and review process. prisma has australasian journal of information systems sultana, teoh & karanasios 2022, vol 26, research article the impact of blockchain on supply chains 4 been applied in other reviews of is phenomenon (frizzo-barker et al., 2020; litchfield & khan, 2019), as well as in blockchain (chukwu & garg, 2020; kuo et al., 2019) and related management reviews (paliwal et al., 2020; rocha et al., 2021). prisma consists of four phases: identification, screening, eligibility and inclusion. figure 1 outlines how these phases were applied in this review. subsequently, each phase is described in detail. 2.1 identification for journal papers, we began our search in critical online academic databases: scopus, ebsco, proquest, web of science, springer open, ieeexplore and ais e-library. our research focus demanded a broad search framing to help avoid the risk of missing relevant articles (pranto et al., 2019). for instance, some databases, such as the ais e-library, were more specific in their orientation, although we recognise there is overlap across these databases. additionally, for conference papers, we considered top is proceedings amcis, pacis, ecis, icis, and hicss from the ais e-library. for conference papers, we limited the search from 2017 because conference articles from 2016 would likely appear as journal articles 3 to 4 years later (günther et al., 2017). based on an initial literature review and external consultation with researchers and practitioners, we selected a set of keywords related to blockchain and supply chain, as listed in table 2. the search keywords were used primarily on ‘title’ and in the paper’s abstract/keywords/body (dependent on the database search interface) from 2016 to june 2021 (see appendix a for search queries of different databases). this period was set because even though the introduction of blockchain can be traced to 2008 and the emergence of bitcoin (nakamoto, 2008), its application in the supply chain has only attracted industry attention since 2016. after this phase, a total of 1,546 papers were returned. search keywords "blockchain” and “supply chain" "blockchain” and “supply chain management" "blockchain technology” and “supply chain management" "blockchain" and "scm” "distributed ledger technology" and "supply chain" "dlt" and "supply chain” "dlt" and "scm" "blockchain" and "supply chain" and (“impact” or "functional" or "issues" or "integration" or "information sharing" or "performance") table 2. search keywords 2.2 screening because we searched multiple databases, we undertook a further phase of refining the sample papers. duplicates were identified by exporting all references into endnote to locate and eliminate duplicates. furthermore, we screened paper titles to remove papers that were not relevant to blockchain in the supply chain and didn’t mention the keywords in the title. research-in-progress, emergent research forum, and treo talks were also removed. in total, 1,369 references were removed. appendix b provides examples of excluded papers of each stage. the next step was filtering papers based on the assessment of eligibility. australasian journal of information systems sultana, teoh & karanasios 2022, vol 26, research article the impact of blockchain on supply chains 5 2.3 eligibility in this phase, we further set the conditions for our review. we first limited the results to highquality publications. we relied on the australian business deans council journal quality list, and selected journals ranked a*, a, or b1 . this list includes all the is senior scholars’ basket of journals and leading supply chain journals such as the journal of operations management, journal of supply chain management, supply chain management journal, international journal of operations and production management and international journal of production and economic. then, we read each remaining paper abstract to examine if the paper would be suitable for further review. we eliminated papers that did not connect blockchain (or related keywords) with the supply chain or mentioned it less than two times in the abstract. in other words, one of the paper’s key focuses had to be on blockchain and the supply chain. often omitted key papers mentioned blockchain en passant, such as an emerging technology or a growing trend (see appendix b for examples of excluded papers). 2.4 inclusion we aimed to arrive at a set of papers for this systematic review focusing on blockchain in the supply chain. we went beyond the title/abstract/keyword in the final phase and examined the remaining papers for deeper relevance. papers that were technical or did not consider blockchain application or its impact on the supply chain were excluded. this led to excluding papers that proposed technical solutions and frameworks. whereas the included papers focused on different blockchain features or studied various blockchain issues through the model, analysis of case studies, or empirical data. as a result, 44 papers were included for analysis. among these 44 papers, 38 are journal articles from 23 different journals, and 6 are conference papers from two conferences. of these papers, 75% are conceptual papers, and 25% are empirical. the list of identified papers and a list of outlets with the number of papers are given in appendix c and appendix d, respectively. 2.5 data analysis procedures we followed a multidimensional approach to analyse the papers and provide an in-depth understanding of the impact of blockchain across the supply chain network. we first classified papers based on whether they were concerned with the organisational, inter-organisational and industry levels. we then identified major themes within these levels. this analysis approach helped address our research question in three ways. firstly, the organisational level review identifies how blockchain impacts organisations and their activities within the supply chain. secondly, the inter-organisational level review identifies how blockchain impacts the interactions between organisations in the supply chain network. lastly, the industry level review identifies a more holistic understanding of the blockchain impact from the supply chain industry level. figure 2 outlines the data analytical process. to guide the analysis, a review framework (table a, in appendix e) was used to guide our analysis. this help to focus the research on the method, findings, limitations and implications 1 the 2019 list endorses 2,682 journal entries with the following classifications: a* 7.41% (199), a 24.27% (651), b 31.69% (850), c 36.61% (982) https://abdc.edu.au/research/abdc-journal-quality-list/ australasian journal of information systems sultana, teoh & karanasios 2022, vol 26, research article the impact of blockchain on supply chains 6 of the identified papers. appendix e provides a guide on how we examine the paper and examples of themes with representative data provided. figure 2. data analytical process at the same time, we examined whether the themes represent pre-implementation or postimplementation or focused on emerging tension driven. ‘pre-implementation driven’ focuses on the impact of blockchain related to preparations/ conditions that demand before implementing blockchain in the supply chain. ‘post-implementation driven’ focuses on the impact of blockchain related to changes blockchain could bring in the supply chain after implementation. ‘emerging tension’ focuses on the tensions arising from blockchain implementation in supply chain networks. 3 findings this section will present the analysis of the papers and address the research question by understanding the impact of blockchain on the supply chain. after reviewing 44 papers, 12 core themes were identified from three-level of the supply chain. the findings are presented in table 3 by organisational, inter-organisational and industry levels. column one categorises the papers by the level, column two shows the main themes, column three summarises the sub-themes, whilst columns four to six indicate the focus on impact. while some of the research that we reviewed cuts across these levels, our categorisation demonstrates the research and progress within each category before making links across these levels of analysis. australasian journal of information systems sultana, teoh & karanasios 2022, vol 26, research article the impact of blockchain on supply chains 7 identified core themes, themes, and aspects of the impact l ev el s core themes (areas blockchain impact supply chain) themes current aspects of impact preimplementation postimplementation emerg -ing tensio n o rg an is at io n al supply chain process and business model redesign of supply chain process (chang et al., 2019; tönnissen & teuteberg, 2020) * * changes and innovations to business models (tönnissen & teuteberg, 2020) * transformation of business process (sunny et al., 2020) * efficiency and risk reduce the average time of orders (martinez et al., 2019) * reduce transaction costs (roeck et al. 2020; schmidt & wagner 2019; van hoek 2019, bumblauskas et al. 2020; zheng et al. 2019) * reduce the risk and cost of product recalls (bumblauskas et al., 2020) * reduce security and operational risk (choi, 2020; min, 2019) * reduce supply chain risk (lai et al., 2021) * optimise supply chain management (lai et al., 2021) * reduce errors (rao et al., 2021) * visibility, transparency, resilience enhance supply chain resilience (min, 2019) * improve visibility (martinez et al., 2019; van hoek, 2019) * efficient and transparent transactions (bai & sarkis, 2020; schmidt & wagner, 2019) * enhance supply chain transparency and traceability (ali et al., 2021; kohler & pizzol, 2020) * enable selective transparency, authenticity and trust (röck, 2020) * profit and performance valid and effective measurement of outcomes (kshetri, 2018) * increase profit (choi, 2020; hayrutdinov et al., 2020; zheng et al., 2019) * improve supply chain performance (hald & kinra, 2019; wamba et al., 2020) * reduce supply chain performance (hald & kinra, 2019) * ensure product quality (bai & sarkis, 2020) * protect brand images (howson, 2020) * in te ro rg an is at io n al integration and collaboration pre-conditions to adopt blockchain (behnke & janssen, 2020; fan et al., 2020) * simplify b2b, supply chain and technology integration (ali et al., 2021; korpela et al., 2017) * supply chain actors' recursive impact on adoption and integration (sternberg et al., 2020) * enhance collaboration (ali et al., 2021) * required collaboration of actors (sharma, 2021) * australasian journal of information systems sultana, teoh & karanasios 2022, vol 26, research article the impact of blockchain on supply chains 8 integration with supply chain objectives and existing technology is required (van hoek, 2020) * requirement for blockchain-based collaboration (herm & janiesch, 2021) * challenges in the integration of logistic activities(sternberg & baruffaldi, 2018) * equality in information, data sharing and management data management, safety and decentralisation (kohler & pizzol, 2020; yadav & singh, 2020) * improve information sharing (hayrutdinov et al., 2020; zheng et al., 2019) * challenges in providing transparency and obtaining tamperproof data (ghode et al., 2021) * challenges related to coordination and data sharing (ghode et al., 2021) * addressing societal changes (ghode et al., 2021) * supply chain parties’ willingness to share(straubert et al., 2021) * questioning blockchain information vs realworld information(straubert et al., 2021) * interorganisational trust enhance inter-organisational trust among stakeholders (howson, 2020; kumar et al., 2020) * challenges for inter-organisational trust (ghode et al., 2020; ghode et al., 2021) * enable mutual trust (röck, 2020) * network provenance, relational governance, interdependency no disintermediation and new reintermediation (tönnissen & teuteberg, 2020) * * provenance in the network (kumar et al., 2020; rogerson & parry, 2020) * relational governance (ghode et al., 2020) * change in power distribution (roeck et al., 2020) * reduce dependency (roeck et al., 2020) * governance challenges (ghode et al., 2021) * network profit and cost choosing a profit-earning product for blockchain (ghode et al., 2021) * improve the overall profit of the chain (hayrutdinov et al., 2020; zheng et al., 2019) * reduce transaction costs of members (zheng et al., 2019) * in d u st ry industry performance and sustainability enhance industry performance and reduce costs (zheng et al., 2019) * questioning sustainable performance (chen et al., 2020; di vaio & varriale, 2020) * improve supply chain quality management (li et al., 2020) * fix and transform the world’s food system (bumblauskas et al., 2020) * improve economic sustainability (niu et al., 2021) * industry coordination, traceability and digital model coordination among industry players (di vaio & varriale, 2020) * * end-to-end digitisation is required (rogerson & parry, 2020) * access to complete information, traceability and transparency (hew et al., 2020; kamble et al., 2020) * australasian journal of information systems sultana, teoh & karanasios 2022, vol 26, research article the impact of blockchain on supply chains 9 industry standards and regulations defining an open and standard data format (chen et al., 2020) * essential for regulatory assurance and end-user desirability (sharma et al., 2019) * lack of government regulation and trust (yadav et al., 2020) * lack of regulatory forum (howson, 2020) * table 3. identified core themes, themes, and aspects of the impact 3.1 impact of blockchain at the organisational level based on our review, we found that at the organisational level, the impact of blockchain emerged based on the four core themes. they are supply chain process and business model; efficiency and risk; visibility, transparency and resilience; and profit and performance. 3.1.1 supply chain process and business model blockchain implementation impacts the supply chain as it demands redesigning supply chain processes and business models to improve efficiency (chang et al., 2019). redesigning is required to address frictions associated with manual processes, centralised authentication and interference from intermediaries. with blockchain, these frictions can be overcome through process automation, peer-to-peer authentication and disintermediation (chang et al., 2019). at the same time, such transformations are fraught with challenges, especially in large supply chains (sunny et al., 2020). additionally, ongoing blockchain application in the supply chain process could lead to unanticipated changes to the business model (tönnissen & teuteberg 2020). for instance, tönnissen & teuteberg (2020) found that while initially, there are no intermediaries in blockchain applications (i.e., agri-food and cognizant retail applications), over time, blockchain service providers could become established intermediaries. 3.1.2 efficiency and risk blockchain can impact supply chain efficiency, including time savings (martinez et al., 2019) and transaction cost (roeck et al., 2020; schmidt & wagner, 2019; van hoek, 2019, bumblauskas et al., 2020; zheng et al., 2019). efficiency is achieved with blockchain as it replaces intermediaries with the crowd of nodes on the system to avoid dependence on intermediaries. for instance, enabling peer-to-peer authentication introduces disintermediation and decentralisation, which eventually optimises the supply chain activities (lai et al., 2021). such optimisation reduces intermediaries’ fees, the required number of operations and order processing time (martinez et al., 2019), and operational and security risks related to intermediaries’ intervention, privacy and other compliance issues (choi, 2020; min, 2019). the level of automation blockchain introduces in data exchange and interconnection helps to reduce risks, including errors caused by manual interventions (rao et al., 2021) that occurred during planning, sourcing, making deliveries, and returns (lai et al., 2021). this includes overproduction or underutilisation risks during planning, trust issues related risk due to unknown supply sources, defects and disruption risk in the making, transport-level tampering and misplacing risk in the delivery, intermediaries, manipulation and verification error risk in returns (lai et al., 2021). for many organisations, this is a crucial attraction of blockchain as it means that risk and expenses related to health and recalls in the food supply chain are reduced australasian journal of information systems sultana, teoh & karanasios 2022, vol 26, research article the impact of blockchain on supply chains 10 (bumblauskas et al., 2020). hence, blockchain can help organisations to be more efficient in supply chain processes, enhance transactions (martinez et al., 2019; schmidt & wagner, 2019) and reduce transaction costs, which are also considered the main motivators for organisations to adopt blockchain (van hoek 2019). 3.1.3 visibility, transparency and resilience organisational impacts also include visibility, transparency and resilience (ali et al., 2021; martinez et al., 2019; min, 2019; schmidt & wagner, 2019; van hoek, 2019). to impact visibility, blockchain demands new information sharing and information processing capabilities from organisations (martinez et al., 2019). to safeguard confidentiality and sharing of valuable information organisations can predetermine the conditions and access rights on which blockchain solutions will work. by doing so, organisations can control over-sharing information and unintentional information disclosure while enforcing the desired level of transparency (röck, 2020). blockchain-enabled transparency impacts organisations in multiple ways. transparency through blockchain allows organisations to better plan and forecast to reduce supply-demand uncertainty cost-effectively. for instance, information about the origins of raw materials making their way to a final product or service, including all the touchpoints, can be recorded and tracked (bai & sarkis 2020). this technical attribute of blockchain facilitates an environment for transparency (kohler & pizzol, 2020; schmidt & wagner, 2019). information transparency provided by blockchain also helps to strengthen resilience positively within the supply chain. especially during times of uncertainty and risk by allowing better forecasting with information sharing, enabling strong security measures against fraud and modification, and detecting tangible and invisible risks such as cyberattacks, computer hacking, credit failures, and contract frauds (min, 2019). these beneficial impacts of blockchain drive organisations to continue blockchain adoption within the supply chain network (van hoek, 2019). however, a critical concern in ensuring transparency is the continuous feeding of information into the blockchain by organisations in the supply chain network (schmidt & wagner, 2019). 3.1.4 profit and performance profit and performance are the end consequence of different efforts through blockchain. so far, blockchain has shown to offer a positive impact on organisations by enhancing organisational profit and reducing transaction costs (zheng et al., 2019) through disintermediation, especially in certain supply chains where the intermediaries’ fees are high (choi, 2020). blockchain can also impact profitability by facilitating valid and effective performance measures (kshetri, 2018) and influencing price sensitivity through effective information sharing (hayrutdinov et al., 2020). through information sharing with stakeholders, organisations (especially large organisations) can ensure transparency to protect organisational brand image (howson, 2020) and ensure quality and sustainable practices (bai & sarkis, 2020) in the supply chain network. this led to two possible impacts on organisation performance. first is positive organisational supply chain performance (hald & kinra, 2019; wamba et al., 2020). second, blockchain might impede organisational supply chain performance by challenging the adaptability of organisations with current resources and reducing organisational human resource competencies through automation (hald and kinra, 2019). australasian journal of information systems sultana, teoh & karanasios 2022, vol 26, research article the impact of blockchain on supply chains 11 3.2 impact of blockchain at the inter-organisational level at the inter-organisational level, five core themes emerged: integration and collaboration; equality in information, data sharing and management; inter-organisational trust; network provenance, interdependency, relational governance; network profit and cost (refer to table 3). 3.2.1 integration and collaboration studies’ findings show that blockchain simplifies the impact of technology integration and collaboration with actors in the network by providing security, peer-to-peer authentication, smart contract, and transparent information sharing (ali et al., 2021; korpela et al., 2017). the critical factors for facilitating successful blockchain-based inter-organisational integration are the shared understanding of supply chain data, fundamental functionality of the current system embedded in the blockchain, and standards for inter-operability (korpela et al., 2017) and collaboration from supply chain partners (sharma, 2021). however, achieving this is not straightforward (sternberg & baruffaldi, 2018), especially in inter-organisational settings. studies also found that collaboration requirements in the supply chain network are multidimensional (herm & janiesch, 2021). it includes data exchange, data privacy, platform implementation, administration, certification authorities etc. (herm & janiesch, 2021). the reason is that one supply chain actor’s adoption and integration decision would trigger recursive effects on other partners in the supply chain network (sternberg et al., 2020). scholars recognise that blockchain should not be seen as a solution for all types of supply chain problems but rather for achieving specific objectives (van hoek, 2020). blockchain can offer inter-organisational transparency and traceability impact on the supply chain, but to achieve that, the supply chain needs to fulfil some boundary conditions before using blockchain, such as standardisation of processes and interfaces, having a joint platform and independent governance are prime boundary conditions (behnke and janssen, 2020). other conditions, such as end-customer awareness, costs of supplier and manufacturer, and usage cost of blockchain in the network, would also require attention to determine should organisations need to adopt blockchain (fan et al., 2020). 3.2.2 equality in information, data sharing and management blockchain can facilitate equality in information and data sharing between organisations in the supply chain network by maintaining the information-sharing effort throughout the product life-cycle (hayrutdinov et al., 2020; zheng et al., 2019), ensuring data safety, quality and accessibility (yadav & singh, 2020) and improving data management (kohler & pizzol, 2020). sharing information in the network (such as cost-revenue) helps build long-term relationships among the supply chain actors (hayrutdinov et al., 2020). however, providing transparency and obtaining tamperproof data concerning shielding confidentiality is challenging. tamper resistance is subjective to technical attributes like smart contract coding, as bad coding on smart contracts hampers the tamperproof feature of the blockchain (ghode et al., 2021). at the same time, blockchain information does not necessarily always match realworld information. since faulty sensors and intentional manipulation of the input can impact transparency (straubert et al., 2021). regardless of blockchain’s functionality, if organisations resist sharing data in the supply chain network, it will impact the transparency in the supply chain (straubert et al., 2021). australasian journal of information systems sultana, teoh & karanasios 2022, vol 26, research article the impact of blockchain on supply chains 12 3.2.3 inter-organisational trust inter-organisational trust enhances the inter-organisational relationship. however, improving inter-organisational trust in the network is challenging (ghode et al., 2020; ghode et al., 2021). kumar et al. (2020) proposed that blockchain should be deployed in a supply chain network lacking inter-organisational trust. this is because blockchain is argued to mend trust issues within the supply chain (howson, 2020) and enable mutual trust (röck, 2020). for example, the global seafood network struggles with trust issues regarding the funds from charitable causes. these trust issues can be solved with blockchain-based fundraising and data sharing (howson, 2020). 3.2.4 network provenance, relational governance, and interdependency blockchain can influence network provenance, relational governance and interdependency in the supply chain network. enabling provenance in the network is one of the attractions of blockchain. the exchange of reliable information within the network helps enhance network provenance (kumar et al., 2020; rogerson & parry, 2020) and maintain relationships among partners (ghode et al., 2020). in a supply chain network, the long-term and trusted relationship among partners is maintained through relational governance. relational governance can influence the integration, scope of flexibility and control using authority. however, cooperation management among the members in the network for relational governance is challenging, and this could further impact the adoption of blockchain in the network (ghode et al., 2020). obeying the governance rules of organisations is also challenging to implement blockchain in the supply chain (ghode et al., 2021). interdependency within the supply chain network can be reduced due to the transparency enabled by blockchain. this is because the distribution of information reduces the need for intermediation, as intermediaries lose their information sovereignty or monopoly. this also has a subsequent impact on power shift and change in power distribution in the network. while reducing dependence seems to have a positive impact, the change in power distribution might cause disadvantages (roeck et al., 2020). however, tönnissen and teuteberg (2020) suggest that blockchain creates re-intermediation instead of disintermediation, which questions the dependence reducing impact of blockchain on the third party. 3.2.5 network profit and cost in addition to organisational level financial benefits, blockchain enhances the overall profit in the supply chain network by reducing the participation cost (zheng et al., 2019), minimising the risk of partners in the network (zheng et al., 2019), and facilitating information sharing (hayrutdinov et al., 2020; zheng et al., 2019). research shows that network profit will be higher for the participants in the supply chain network who adopted blockchain than those who did not adopt blockchain technology when participants were risk-neutral (zheng et al., 2019). another study investigated information sharing efforts and found that blockchain enhances the network’s profit through proper information sharing, reducing transaction costs and decreasing price sensitivity in the network (hayrutinov et al., 2020). however, choosing a profit-earning product for adopting blockchain is challenging (ghode et al., 2021). hence, blockchain adoption displays a promising impact on network profit and cost through a wise selection of products, proper information sharing, and a certain risk environment. australasian journal of information systems sultana, teoh & karanasios 2022, vol 26, research article the impact of blockchain on supply chains 13 3.3 impact of blockchain at the industry level three core themes emerged at the industry level: industry performance and sustainability; industry coordination, traceability and digital model; industry standards and regulations (refer to table 3). 3.3.1 industry performance and sustainability blockchain influences industry performance and sustainability by reducing costs (zheng et al., 2019) and enhancing supply chain quality management through sharing information properly with the correct partner (li et al., 2020). focusing on the food supply chain, bumblauskas et al. (2020) argue that creating a transparent and traceable supply chain by blockchain also helps to solve issues such as safety, contamination, economic loss etc., and transform the world’s food system for the better. information transparency allows individual organisations to monitor the movements of the products with the consent of other organisations in the supply chain network. this helps to minimise possible corruption practices and counterfeit products, which enhance industry performance. as such investigating the aircraft industry, di vaio and varriale (2020) also argued that blockchain helps improve efficiency, effectiveness, and sustainability. managing disruptions, stakeholders’ collaborations and involvements are key to improving efficiency, effectiveness, and sustainable performance (di vaio & varriale 2020). a lack of stakeholders’ collaboration and involvement (di vaio & varriale 2020), privacy issues, and intolerance of blockchain immutability can impact performance and sustainability (chen et al. 2020). furthermore, blockchain can impact economic sustainability but is subject to factors like product quality, retailers’ brand image and power, supplier variance of supply uncertainty and customer surplus (niu et al., 2021). overall, to bring sustainability in performance, managers and policymakers need to step up to create a forum for facilitating collaboration (di vaio & varriale 2020), participation (niu et al., 2021) and discussing issues upfront. 3.3.2 industry coordination, traceability and digital model blockchain promotes industry coordination by reducing fragmentation, discoordination and inefficient operation (di vaio & varriale, 2020). blockchain can impact end-to-end digitisations requirements to facilitate coordination and visibility in the network (rogerson & parry, 2020). without that requirement, human data entry onto blockchains remains open to doubt as the occurrence of potential error or corruption is still possible. however, with complete digitisation within the network, difficulty could arise in terms of masking sensitive data, endangering the identity of actors and the quantity of data produced (ibid). additionally, by capturing and allowing stakeholders complete access to supply chain information, blockchain can help ensure traceability and transparency at the industry level (hew et al., 2020). blockchain is highly acknowledged for enabling and providing traceability in different industries, such as the halal food industry, healthcare, etc. traceability is also considered one of the core enablers of blockchain implementation (kamble et al., 2020). therefore, blockchain implementation impacts access to information and digitisation requirements to enable coordination and traceability at the industry level. 3.3.3 industry standards and regulations research emphasises the need to establish industry standards and regulations to scale up blockchain implementation and adoption in supply chains. the lack of established government regulations (yadav, et al., 2020), data standardisation (chen et al., 2020) and trust australasian journal of information systems sultana, teoh & karanasios 2022, vol 26, research article the impact of blockchain on supply chains 14 among stakeholders (yadav et al., 2020) have meant that blockchain hadn’t been adopted widely at the industry level. industry standards and regulations are still embryonic across different countries. some argue that regulatory requirements for blockchain will help to record information on product’s touchpoints and facilitate the information to the regulatory authorities, authorised actors and the end-users (sharma et al., 2019). regulatory forums can be attractive for those keen on adhering to compliance (howson, 2020). challenges will remain for those who operate illegally and disobey compliance (howson, 2020). 4 discussion this systematic review set out to obtain a deeper understanding of the blockchain and supply chain landscape, realise blockchain’s value, and guide future blockchain research. in addressing the research question, we found core areas of the supply chain impacted by blockchain and aspects of impact on those core areas from three different levels. these stress the impact of blockchain at different levels, which contribute to scholarship by (i) enriching knowledge on the current state of blockchain’s impact on the supply chain; (ii) going beyond current research focus to understanding the broader impact of blockchain technology in the organisation, inter-organisation and industry level; and (iii) laying the foundation for further substantial blockchain impact research in the areas of the supply chain. each of the levels and key findings is discussed below. following the discussion on summarised findings, we also articulate research questions to guide future research. the summarised findings of blockchain’s impact on the supply chain are provided in table 4. firstly, one of the critical challenges of using technology is to match its’ specifications with organisational or business needs (davenport, 1998). the case of blockchain is not different. at the organisational level, most pre-implementation impacts are related to evaluating organisational technology needs or organisational readiness for using technology. this includes identifying inefficiency, the need for technology, understanding downstream and upstream impacts etc. at the same time, at the inter-organisational level and industry level, pre-implementational impact focuses on understanding and embracing network needs for implementing blockchain and industry requirements respectively. this is important as the successful implementation of blockchain as an inter-organisational technology depends on the understanding needs of actors in the supply chain network with support from industry level initiatives. in the practitioner literature, there are arguments that most of the current implementations of blockchain solutions are political moves in the organisation, championed by powerful actors in the supply chain, or a consequence of technological hype around the industry (furlonger & valdes, 2017). the pre-implementation impact shows that there are different aspects at different levels that require attention and consideration before implementing blockchain. thus, blockchain implementation should not be based on the decision of one organisation and influenced by hype only, but rather based on the consideration of technical, organisational and the perspective of multiple actors. secondly, with the emergence of new technology, there is a tendency in is literature to study its applications or view it as a tool for solving business issues (rossi et al., 2019). similarly, blockchain’s post-implementation impact at different levels shows mostly optimistic views of blockchain. extensive exploration of blockchain’s potential impact, application and benefit could be the reason. the realisation of impact based on empirical evidence is yet to be fully developed. australasian journal of information systems sultana, teoh & karanasios 2022, vol 26, research article the impact of blockchain on supply chains 15 thirdly, implementing new technology (like blockchain) within complex organisational and social systems can raise practical tensions (treiblmaier, 2019). however, so far emerging tensions impact of blockchain at the organisational level are mostly proposed and not realised in detail. this may be linked to a lack of operational blockchain use cases (clohessy & acton, 2019; verhoeven et al., 2018) and the lack of empirical studies (cole et al., 2019; frizzo-barker et al., 2020). at the inter-organisational level, emerging tensions are mostly similar to typical technology barriers (i.e., standards for inter-operability or fundamental functionality of the current system), obvious inter-organisation system barriers (i.e., collaboration or cooperation) or linked with participating organisations’ attitudes (i.e., resistance to share or obeying governance rules). also, the tensions are mostly derived from the pre-implementation impact, and less from post-implementation impact. whilst industry level tensions communicate more on how the industry has been slow in taking initiative, and less on blockchain impact on the emergence of tension. this is because there is still lack of detailed guidance on how the industry should prepare, incentivise, and facilitate blockchain adoption. tensions from different levels are also linked, as such handling tension of one level could help to solve other level tensions as well. core -themes pre-implementation impact post-implementation impact emerging tension impact o rg an is at io n al supply chain process and business model identification of inefficient process need for blockchain changes in business models unanticipated changes in the business model re-intermediation efficiency and risk automation in data exchange increased efficiency cost structure optimise supply chain activities visibility, transparency, and resilience predetermine the conditions and access rights organisational reaction during uncertainty better forecasting with information sharing solve traceability issues continuous information feeding in the blockchain profit and performance evaluating current intermediaries' impact influence price sensitivity protecting brand image facilitate sustainable practice questioning costeffectiveness reducing supply chain competencies questioning longterm sustainability in te ro rg an is at io n al l ev el integration and collaboration require organising existing supply chain consideration for effective integration fulfilling boundary conditions recursively impact adoption decision large-scale collaboration becomes easier establishment of reputation missing fundamental functionality of the current system standards for interoperability analysing collaboration requirements equality in information, data information sharing efforts change in the current mode of data obtaining tamperproof data australasian journal of information systems sultana, teoh & karanasios 2022, vol 26, research article the impact of blockchain on supply chains 16 sharing and management sharing and interaction information ownership sustainability in building long-term relationships inconsistency in blockchain information vs realworld information intentional manipulation in the input resistance to share inter-organisational trust identifying trust issues acceptance of blockchain enhance the interorganisation relationship the success of blockchain in an untrusted network network provenance, relational governance, and interdependency understanding the need for provenance reliable data exchange integration scope of flexibility control in the use of authority reduce information monopoly change in power distribution cooperation management dependence reducing obeying the governance rules network profit and cost risk attitude of participants wise selection of products for adopting blockchain facilitate information-sharing reduce participation cost decrease price sensitivity choosing a profitearning product in d u st ry industry performance and sustainability create a forum for collaboration and discussion reduce fragmentation, inefficiency, and uncoordinated operations management of disruption reduce economic loss lack of stakeholders’ collaboration and involvement privacy issue intolerance of blockchain immutability industry coordination, traceability and digital model digitisation requirement access to information change in the economic model masking sensitive data endangering the identity of actors industry standards and regulations regulatory forums facilitating the information to the regulatory authorities, authorised actors and the end-users lack of established government regulation data standardisation disobeying compliance table 4. summarised findings based on these findings, this review proposes possible several research directions (in the form of research questions) that consider the implications of blockchain in the supply chain. (i) how does blockchain implementation impact cost structure and cost-effectiveness? australasian journal of information systems sultana, teoh & karanasios 2022, vol 26, research article the impact of blockchain on supply chains 17 (ii) how does the change in the business model by blockchain impact organisational supply chain activities? (iii) how does re-engineering the supply chain process with blockchain introduce new tensions in the organisations while solving current tensions? (iv) how do supply chain competencies impact blockchain implementation? (v) how does blockchain impact socio-organisational elements and activities? (vi) how agile is the blockchain solution in solving traceability issues during an uncertain situation? (vii) what factors need to consider before blockchain implementation at the organisational level? (viii) how can blockchain impact organisations’ long-term performance and sustainable practices? (ix) how to realise the business value of blockchain within a supply chain network? (x) how does blockchain impact inter-organisational trust and interaction? (xi) how does integrating and adopting blockchain by one actor impact other actors in the network? (xii) how does governance facilitate blockchain deployment in the supply chain network? (xiii) how do blockchain impact power distribution and information monopoly? (xiv) how blockchain mediated change in information sharing and interaction impact information ownership? (xv) how could blockchain change information ownership by challenging actors’ relationships in the supply chain? (xvi) how can blockchain facilitate sustainability in relation-building, collaboration and reputation management? (xvii) how should the need for blockchain in the network be evaluated in terms of provenance and trust? (xviii) how does blockchain facilitate information symmetry toward a circular economy? (xix) how established government regulation impact organisational trust? (xx) how should the industry level blockchain initiative be introduced and how will it impact digitising the supply chain? (xxi) how do industry level incentives facilitate the adoption of blockchain? (xxii) how practical is the blockchain for wide industry level adoption? this study’s findings enhance the existing literature on the blockchain-based supply chain. the proposed future agenda on blockchain-based supply chain studies will help academics with research directions toward substantial research in the future. in practice, this research sets a foundation for managers to understand the current impact of blockchain within the supply chain. it allows them to prepare appropriately and strategically to adequately realise the blockchain value in the supply chain. acknowledging the different dimensions of impact will help them recognise blockchain business value and prepare to adopt blockchain solutions in the supply chain. understanding the pre-implementation impacts will help managers to prepare for blockchain implementation. moving forward, knowledge of the postimplementation impacts will allow them to recognise how much value blockchain could bring to the supply chain. finally, sensing the emerging tensions and subsequent impacts will support managers to act proactively on the emerging tensions and facilitate blockchain adoption. australasian journal of information systems sultana, teoh & karanasios 2022, vol 26, research article the impact of blockchain on supply chains 18 to further support managers in understanding the interaction between the impact of blockchain at different levels and phases, we propose a blockchain impact framework below (figure 3) that illustrates the impact of blockchain in three critical phases, from preparation to blockchain adoption. it also shows how understanding different types of impacts helps to get prepare, recognise the value and adopt blockchain in the supply chain network and what understanding managers need to have in each phase. thus, this framework is useful in guiding managers’ understanding and actions on key impacts at three critical phases. figure 3. blockchain impact framework 5 conclusion this systematic review investigates the academic research on the impact of blockchain on supply chains and presents directions to guide future research focus based on the current research landscape. the study provides a clearer view of how blockchain impacts supply chains and how it has triggered a succession of organisational, inter-organisational, and industry level impacts. we categorise the existing research into twelve core themes across preand post-implementation phases and tensions that blockchain introduces. by bringing together this research, we go beyond understanding what blockchain is and its possible applications and provide a deeper understanding of the blockchain and supply chain literature. a contribution of our work is a model that brings together three critical phases: preparation, implementation and adoption, types of impact and where the impact takes place. our insights also lay the foundation for further blockchain impact studies in the supply chain, whereby we presented 22 questions that can drive future research. like all reviews, this review comes with several precautions and certain inescapable compromises, such as the reliance on database searches and the underlying assumption that they provide coverage of blockchain and supply chains (boell & cecez-kecmanovic, 2015). while this study undertook a broad search in a range of databases, there remains the possibility that papers may have been omitted. in addition, given that blockchain is a new australasian journal of information systems sultana, teoh & karanasios 2022, vol 26, research article the impact of blockchain on supply chains 19 technology, it is essential to consider that the impact of blockchain is likely to evolve over time. this review finds the early impacts of the technology. this review also identifies that most papers remain conceptual, which signals the embryonic nature of blockchain applications. we believe there should be more empirical research as this technology matures that can be informed by the directions for research we articulated. references akram, a., & bross, p. (2018). trust, privacy and transparency with blockhain technology in logistics. paper presented at the mediterranean conference on information systems (mcis). https://aisel.aisnet.org/mcis2018/17 ali, m. h., chung, l. n., kumar, a., zailani, s., & tan, k. h. (2021). a sustainable blockchain framework for the halal food supply chain: lessons from malaysia. technological forecasting and social change, 170 (2021), 120870. https://doi.org/10.1016/j.techfore.2021.120870 allen, d. w., berg, c., davidson, s., novak, m., & potts, j. (2019). international policy coordination for blockchain supply chains. asia & the pacific policy studies, 6(3), 367-380. https://doi.org/10.1002/app5.281 arena, a., bianchini, a., perazzo, p., vallati, c., & dini, g. (2019). bruschetta: an iot blockchain-based framework for certifying extra virgin olive oil supply chain. paper presented at the 2019 ieee international conference on smart computing (smartcomp). 173-179. https://doi.org/ 10.1109/smartcomp.2019.00049. attaran, m., & gunasekaran, a. (2019). applications of blockchain technology in business: challenges and opportunities. cham: springer international publishing, cham. bai, c., & sarkis, j. (2020). a supply chain transparency and sustainability technology appraisal model for blockchain technology. international journal of production research, 58(7), 21422162. https://doi.org/10.1080/00207543.2019.1708989 beck, r., stenum czepluch, j., lollike, n., & malone, s. (2016). blockchain–the gateway to trustfree cryptographic transactions. paper presented at the european conference on information systems, istanbul. https://aisel.aisnet.org/ecis2016_rp/153 bedo, s. (2018). dodgy food items passed off as the real thing are making their way onto supermarket shelves. retrieved from https://rb.gy/0mclvs behnke, k., & janssen, m. f. w. h. a. (2020). boundary conditions for traceability in food supply chains using blockchain technology. international journal of information management, 52(2020), 101969. https://doi.org/10.1016/j.ijinfomgt.2019.05.025 blossey, g., eisenhardt, j., & hahn, g. (2019). blockchain technology in supply chain management: an application perspective. paper presented at the hawaii international conference on system sciences 2019 (hicss-52). https://aisel.aisnet.org/hicss-52/os/impact_of_blockchain/6/ boell, s. k., & cecez-kecmanovic, d. (2015). on being ‘systematic’ in literature reviews. in l. p. willcocks, c. sauer, & m. c. lacity (eds.), formulating research methods for information systems: volume 2, 48-78. london: palgrave macmillan uk. https://doi.org/10.1002/app5.281 https://rb.gy/0mclvs https://doi.org/10.1016/j.ijinfomgt.2019.05.025 australasian journal of information systems sultana, teoh & karanasios 2022, vol 26, research article the impact of blockchain on supply chains 20 borrell, d. v. (2020). the knockoff effect: the hidden environmental and social impact of counterfeit goods. retrieved from https://www.mytotalretail.com/article/the-knockoffeffect-the-hidden-environmental-and-social-impact-of-counterfeit-goods/ bumblauskas, d., mann, a., dugan, b., & rittmer, j. (2020). a blockchain use case in food distribution: do you know where your food has been? international journal of information management, 52(2020), 102008. https://doi.org/10.1016/j.ijinfomgt.2019.09.004 chang, s. e., & chen, y. (2020). when blockchain meets supply chain: a systematic literature review on current development and potential applications. ieee access, 8, 6247862494. https://doi.org/10.1109/access.2020.2983601. chang, s. e., chen, y. c., & lu, m. f. (2019). supply chain re-engineering using blockchain technology: a case of smart contract based tracking process. technological forecasting and social change, 144, 1-11. https://doi.org/10.1016/j.techfore.2019.03.015 chang, y., iakovou, e., & shi, w. (2020). blockchain in global supply chains and cross border trade: a critical synthesis of the state-of-the-art, challenges and opportunities. international journal of production research, 58(7), 2082-2099. https://doi.org/10.1080/00207543.2019.1651946 chen, s., liu, x., yan, j., hu, g., & shi, y. (2020). processes, benefits, and challenges for adoption of blockchain technologies in food supply chains: a thematic analysis. information systems and e-business management, 19, 909–935. https://doi.org/10.1007/s10257-020-00467-3 choi, t. m. (2020). supply chain financing using blockchain: impacts on supply chains selling fashionable products. annals of operations research, 1-23. https://doi.org/10.1007/s10479020-03615-7 chukwu, e., & garg, l. (2020). a systematic review of blockchain in healthcare: frameworks, prototypes, and implementations. ieee access, 8, 21196-21214. https://doi.org/10.1109/access.2020.2969881. clohessy, t., & acton, t. (2019). investigating the influence of organizational factors on blockchain adoption: an innovation theory perspective. industrial management & data systems. 119 (7), 1457-1491. https://doi.org/10.1108/imds-08-2018-0365 cole, r., stevenson, m., & aitken, j. (2019). blockchain technology: implications for operations and supply chain management. supply chain management, 24(4), 469-483. https://doi.org/10.1108/scm-09-2018-0309 davenport, t. h. (1998). putting the enterprise into the enterprise system. harvard business review, 76(4). de boissieu, e., kondrateva, g., baudier, p., & ammi, c. (2021). the use of blockchain in the luxury industry: supply chains and the traceability of goods. journal of enterprise information management, 34(5), 1318-1338. https://doi.org/10.1108/jeim-11-2020-0471 di vaio, a., & varriale, l. (2020). blockchain technology in supply chain management for sustainable performance: evidence from the airport industry. international journal of information management, 52. https://doi.org/10.1016/j.ijinfomgt.2019.09.010 https://www.mytotalretail.com/article/the-knockoff-effect-the-hidden-environmental-and-social-impact-of-counterfeit-goods/ https://www.mytotalretail.com/article/the-knockoff-effect-the-hidden-environmental-and-social-impact-of-counterfeit-goods/ https://doi.org/10.1108/imds-08-2018-0365 australasian journal of information systems sultana, teoh & karanasios 2022, vol 26, research article the impact of blockchain on supply chains 21 egelund-müller, b., elsman, m., henglein, f., & ross, o. (2017). automated execution of financial contracts on blockchains. business & information systems engineering, 59(6), 457-467. https://doi.org/10.1007/s12599-017-0507-z etemadi, n., borbon-galvez, y., strozzi, f., & etemadi, t. (2021). supply chain disruption risk management with blockchain: a dynamic literature review. information, 12(2), 70. https://doi.org/10.3390/info12020070 fan, z. p., wu, x. y., & cao, b. b. (2020). considering the traceability awareness of consumers: should the supply chain adopt the blockchain technology? annals of operations research, 309(2), 837-860. https://doi.org/10.1007/s10479-020-03729-y fink, a. (2005). conducting research literature reviews: from the internet to paper (2nd ed.). thousand oaks, california: sage publications. frizzo-barker, j., chow-white, p. a., adams, p. r., mentanko, j., ha, d., & green, s. (2020). blockchain as a disruptive technology for business: a systematic review. international journal of information management, 51, 102029. https://doi.org/10.1016/j.ijinfomgt.2019.10.014 furlonger, d., & valdes, r. (2017). practical blockchain: a gartner trend insight report. retrieved from https://www.gartner.com/document/3628617?ref=solrall&refval=182700378&qid=a8931 8613984fd5789d202a1f0d1f680 ghode, d., yadav, v., jain, r., & soni, g. (2020). adoption of blockchain in supply chain: an analysis of influencing factors. journal of enterprise information management, 33(3), 437456. https://doi.org/10.1108/jeim-07-2019-0186 ghode, d. j., yadav, v., jain, r., & soni, g. (2021). blockchain adoption in the supply chain: an appraisal on challenges: ims. [blockchain adoption in the supply chain]. journal of manufacturing technology management, 32(1), 42-62. http://dx.doi.org/10.1108/jmtm-112019-0395 gomber, p., kauffman, r. j., & weber, b. w. (2016). financial is, underlying technologies, and the fintech revolution. journal of management information systems, call for papers. günther, w. a., rezazade mehrizi, m. h., huysman, m., & feldberg, f. (2017). debating big data: a literature review on realizing value from big data. the journal of strategic information systems, 26(3), 191-209. https://doi.org/10.1016/j.jsis.2017.07.003 guo, y., & liang, c. (2016). blockchain application and outlook in the banking industry. financial innovation, 2(1), 24. https://doi.org/10.1186/s40854-016-0034-9 gurtu, a., & jestin, j. (2019). potential of blockchain technology in supply chain management: a literature review. international journal of physical distribution & logistics management, 49(9), 881-900. http://dx.doi.org/10.1108/ijpdlm-11-2018-0371 hald, k. s., & kinra, a. (2019). how the blockchain enables and constrains supply chain performance. international journal of physical distribution and logistics management, 49(4), 376-397. https://doi.org/10.1108/ijpdlm-02-2019-0063 https://doi.org/10.3390/info12020070 http://dx.doi.org/10.1108/jmtm-11-2019-0395 http://dx.doi.org/10.1108/jmtm-11-2019-0395 https://doi.org/10.1016/j.jsis.2017.07.003 http://dx.doi.org/10.1108/ijpdlm-11-2018-0371 australasian journal of information systems sultana, teoh & karanasios 2022, vol 26, research article the impact of blockchain on supply chains 22 hayrutdinov, s., saeed, m. s. r., & rajapov, a. (2020). coordination of supply chain under blockchain system-based product lifecycle information sharing effort. journal of advanced transportation, 2020, 10. http://dx.doi.org/10.1155/2020/5635404 herm, l.-v., & janiesch, c. (2021). towards an implementation of blockchain-based collaboration platforms in supply chain networks: a requirements analysis. paper presented at the hawaii international conference on system sciences 2021 (hicss-54). https://aisel.aisnet.org/hicss-54/st/blockchain_engineering/5/ hew, j. j., wong, l. w., tan, g. w. h., ooi, k. b., & lin, b. (2020). the blockchain-based halal traceability systems: a hype or reality? supply chain management. 25 (6), 863-879. https://doi.org/10.1108/scm-01-2020-0044 howson, p. (2020). building trust and equity in marine conservation and fisheries supply chain management with blockchain. marine policy, 115, 103873. https://doi.org/10.1016/j.marpol.2020.103873 kamble, s. s., gunasekaran, a., & sharma, r. (2020). modeling the blockchain enabled traceability in agriculture supply chain. international journal of information management, 52, 101967. https://doi.org/10.1016/j.ijinfomgt.2019.05.023 kapadia, s. (2019). counterfeit goods total $509b, 3.3% of global trade and growing. retrieved from https://www.supplychaindive.com/news/counterfeits-rising-globaltrade-oecd/550798/ kohler, s., & pizzol, m. (2020). technology assessment of blockchain-based technologies in the food supply chain. journal of cleaner production, 269, 122193. https://doi.org/10.1016/j.jclepro.2020.122193 korpela, k., hallikas, j., & dahlberg, t. (2017). digital supply chain transformation toward blockchain integration. paper presented at the proceedings of the 50th hawaii international conference on system sciences. https://aisel.aisnet.org/hicss50/in/digital_supply_chain/2/ kshetri, n. (2018). 1 blockchain's roles in meeting key supply chain management objectives. international journal of information management, 39, 80-89. https://doi.org/10.1016/j.ijinfomgt.2017.12.005 kumar, a., liu, r., & shan, z. (2020). is blockchain a silver bullet for supply chain management? technical challenges and research opportunities. decision sciences, 51(1), 8-37. https://doi.org/10.1111/deci.12396 kumari, k., & saini, k. (2020). data handling &amp; drug traceability: blockchain meets healthcare to combat counterfeit drugs. international journal of scientific and technology research, 9(3), 728-731. retrieved from https://www.scopus.com/inward/record.uri?eid=2-s2.085082617820&partnerid=40&md5=8ef619d1e11158b5c0481701f77f907c kuo, t.-t., zavaleta rojas, h., & ohno-machado, l. (2019). comparison of blockchain platforms: a systematic review and healthcare examples. journal of the american medical informatics association, 26(5), 462-478. https://doi.org/10.1093/jamia/ocy185 http://dx.doi.org/10.1155/2020/5635404 https://aisel.aisnet.org/hicss-54/st/blockchain_engineering/5/ https://www.supplychaindive.com/news/counterfeits-rising-global-trade-oecd/550798/ https://www.supplychaindive.com/news/counterfeits-rising-global-trade-oecd/550798/ https://www.scopus.com/inward/record.uri?eid=2-s2.0-85082617820&partnerid=40&md5=8ef619d1e11158b5c0481701f77f907c https://www.scopus.com/inward/record.uri?eid=2-s2.0-85082617820&partnerid=40&md5=8ef619d1e11158b5c0481701f77f907c australasian journal of information systems sultana, teoh & karanasios 2022, vol 26, research article the impact of blockchain on supply chains 23 lai, j. y., wang, j. t., & chiu, y. h. (2021). evaluating blockchain technology for reducing supply chain risks. information systems and e-business management, 19(4), 1089-1111. https://doi.org/10.1007/s10257-021-00533-4 lawton, g. (2021). top 9 blockchain platforms to consider in 2021. retrieved from https://www.techtarget.com/searchcio/feature/top-9-blockchain-platforms-to-consider li, j. y., maiti, a., springer, m., & gray, t. (2020). blockchain for supply chain quality management: challenges and opportunities in context of open manufacturing and industrial internet of things. international journal of computer integrated manufacturing, 33(12), 1321-1355. https://doi.org/10.1080/0951192x.2020.1815853 li, y., marier-bienvenue, t., perron-brault, a., wang, x., & paré, g. (2018). blockchain technology in business organizations: a scoping review. paper presented at the proceedings of the 51st hawaii international conference on system sciences. https://aisel.aisnet.org/hicss-51/os/blockchain_and_fintech/2/ lim, m. k., li, y., wang, c., & tseng, m.-l. (2021). a literature review of blockchain technology applications in supply chains: a comprehensive analysis of themes, methodologies and industries. computers & industrial engineering, 154,107133. https://doi.org/10.1016/j.cie.2021.107133 litchfield, a. t., & khan, a. (2019). a review of issues in healthcare information management systems and blockchain solutions. paper presented at the conf-irm. https://aisel.aisnet.org/confirm2019/1 martinez, v., zhao, m., blujdea, c., han, x., neely, a., & albores, p. (2019). blockchain-driven customer order management. international journal of operations and production management, 39, 993-1022. https://doi.org/10.1108/ijopm-01-2019-0100 min, h. (2019). blockchain technology for enhancing supply chain resilience. business horizons, 62(1), 35-45. https://doi.org/10.1016/j.bushor.2018.08.012 moher, d., liberati, a., tetzlaff, j., & altman, d. g. (2009). preferred reporting items for systematic reviews and meta-analyses: the prisma statement. plos med, 6(6: e1000097). https://doi.org/10.1371/journal.pmed1000097 nakamoto, s. (2008). bitcoin: a peer-to-peer electronic cash system. niu, b., shen, z., & xie, f. (2021). the value of blockchain and agricultural supply chain parties' participation confronting random bacteria pollution. journal of cleaner production, 319. https://doi.org/10.1016/j.jclepro.2021.128579 okoli, c. (2015). a guide to conducting a standalone systematic literature review. communications of the association for information systems, 37(1), 43. https://doi.org/10.17705/1cais.03743 okoli, c., & schabram, k. (2010). a guide to conducting a systematic literature review of information systems research. sprouts: working papers on information system, 10(26). retrieved from http://sprouts.aisnet.org/867/1/okolischabram2010sproutslitreviewguide.pdf https://www.techtarget.com/searchcio/feature/top-9-blockchain-platforms-to-consider https://doi.org/10.1016/j.cie.2021.107133 https://doi.org/10.17705/1cais.03743 http://sprouts.aisnet.org/867/1/okolischabram2010sproutslitreviewguide.pdf australasian journal of information systems sultana, teoh & karanasios 2022, vol 26, research article the impact of blockchain on supply chains 24 paliwal, v., chandra, s., & sharma, s. (2020). blockchain technology for sustainable supply chain management: a systematic literature review and a classification framework. sustainability, 12(18), 7638. https://doi.org/10.3390/su12187638 pournader, m., shi, y., seuring, s., & koh, s. l. (2020). blockchain applications in supply chains, transport and logistics: a systematic review of the literature. international journal of production research, 58(7), 2063-2081. https://doi.org/10.1080/00207543.2019.1650976 pranto, s., jardim, l., oliveira, t., & ruivo, p. (2019). literature review on blockchain with focus on supply chain. paper presented at the capsi 2019 proceedings, portugal. https://aisel.aisnet.org/capsi2019/37 puschmann, t., damsgaard, j., & westland, j. c. (2018). financial technology (fintech), blockchain and the internet of value. european conference on information systems, call for papers. queiroz, m. m., telles, r., & bonilla, s. h. (2019). blockchain and supply chain management integration: a systematic review of the literature. supply chain management: an international journal. 25 (2), 241-254. https://doi.org/10.1108/scm-03-2018-0143 rao, s. k., gulley, a., russell, m., & patton, j. (2021). on the quest for supply chain transparency through blockchain: lessons learned from two serialized data projects. journal of business logistics, 42(1), 88-100. https://doi.org/10.1111/jbl.12272 risius, m., & spohrer, k. (2017). a blockchain research framework: what we (don’t) know, where we go from here, and how we will get there. business & information systems engineering, 56(6), 385-409. https://doi.org/10.1007/s12599-017-0506-0 rocha, g. d. s. r., de oliveira, l., & talamini, e. (2021). blockchain applications in agribusiness: a systematic review. future internet, 13(4), 95. https://doi.org/10.3390/fi13040095 röck, d. (2020). the foundation of distributed ledger technology for supply chain management. paper presented at the proceedings of the 53rd hawaii international conference on system sciences. https://aisel.aisnet.org/hicss-53/in/digital_supply_chain/4/ roeck, d., sternberg, h., & hofmann, e. (2020). distributed ledger technology in supply chains: a transaction cost perspective. international journal of production research, 58(7), 2124-2141. http://dx.doi.org/10.1080/00207543.2019.1657247 rogerson, m., & parry, g. c. (2020). blockchain: case studies in food supply chain visibility. supply chain management, 25(5), 601-614. https://doi.org/10.1108/scm-08-2019-0300 rossi, m., mueller-bloch, c., thatcher, j. b., & beck, r. (2019). blockchain research in information systems: current trends and an inclusive future research agenda. journal of the association for information systems, 20(09), 1390-1405. https://doi.org/10.17705/1jais.00571 schlesinger, j., & day, a. (2019). here’s how the trade war could lead to a boom in counterfeit goods. retrieved from https://www.cnbc.com/2019/03/13/heres-how-the-trade-warcould-lead-to-a-boom-in-counterfeit-goods.html https://doi.org/10.3390/su12187638 https://doi.org/10.1108/scm-03-2018-0143 https://doi.org/10.3390/fi13040095 http://dx.doi.org/10.1080/00207543.2019.1657247 https://www.cnbc.com/2019/03/13/heres-how-the-trade-war-could-lead-to-a-boom-in-counterfeit-goods.html https://www.cnbc.com/2019/03/13/heres-how-the-trade-war-could-lead-to-a-boom-in-counterfeit-goods.html australasian journal of information systems sultana, teoh & karanasios 2022, vol 26, research article the impact of blockchain on supply chains 25 schmidt, c. g., & wagner, s. m. (2019). blockchain and supply chain relations: a transaction cost theory perspective. journal of purchasing and supply management, 25(4). https://doi.org/10.1016/j.pursup.2019.100552 sharma, m. g. (2021). supply chain, geographical indicator and blockchain: provenance model for commodity. international journal of productivity and performance management. ahead-ofprint (ahead-of-print ). https://doi.org/10.1108/ijppm-05-2021-0288 sharma, r. s., wingreen, s., kshetri, n., & hewa, t. m. (2019). design principles for validating use cases of blockchain for food supply chains. paper presented at the amcis 2019 proceedings. https://aisel.aisnet.org/amcis2019/green_is_sustain/green_is_sustain/4 sternberg, h., & baruffaldi, g. (2018). chains in chains–logic and challenges of blockchains in supply chains. paper presented at the 51st hawaii international conference on system sciences. https://aisel.aisnet.org/hicss-51/in/digital_supply_chain/3/ sternberg, h. s., hofmann, e., & roeck, d. (2020). the struggle is real: insights from a supply chain blockchain case. journal of business logistics. 24(1), 71-87. https://doi.org/10.1111/jbl.12240 straubert, c., sucky, e., & mattke, j. (2021). blockchain technology for tracking and tracing in supply chains: a critical viewpoint. paper presented at the hawaii international conference on system sciences 2021 (hicss-54). https://aisel.aisnet.org/hicss54/os/blockchain/5/ sunny, j., undralla, n., & pillai, v. m. (2020). supply chain transparency through blockchainbased traceability: an overview with demonstration. computers & industrial engineering, 150. https://doi.org/10.1016/j.cie.2020.106895 tönnissen, s., & teuteberg, f. (2020). analysing the impact of blockchain-technology for operations and supply chain management: an explanatory model drawn from multiple case studies. international journal of information management, 52, 101953. https://doi.org/10.1016/j.ijinfomgt.2019.05.009 treiblmaier, h. (2018). the impact of the blockchain on the supply chain: a theory-based research framework and a call for action. supply chain management: an international journal. 23 (6), 545-559. https://doi.org/10.1108/scm-01-2018-0029 treiblmaier, h. (2019). toward more rigorous blockchain research: recommendations for writing blockchain case studies. frontiers in blockchain, 2. https://doi.org/10.3389/fbloc.2019.00003 treiblmaier, h., french, a. m., & risius, m. (2020). cultural feasibility as a moderator of blockchain acceptance in academia. paper presented at the european conference on information systems (ecis) 2020, research-in-progress papers. https://aisel.aisnet.org/ecis2020_rip/6 van hoek, r. (2019). unblocking the chain–findings from an executive workshop on blockchain in the supply chain. supply chain management: an international journal. 25 (2), 255-261. https://doi.org/10.1108/scm-11-2018-0383 van hoek, r. (2020). developing a framework for considering blockchain pilots in the supply chain – lessons from early industry adopters. supply chain management, 25(1), 115-121. https://doi.org/10.1108/scm-05-2019-0206 https://doi.org/10.1108/scm-01-2018-0029 https://doi.org/10.1108/scm-11-2018-0383 australasian journal of information systems sultana, teoh & karanasios 2022, vol 26, research article the impact of blockchain on supply chains 26 verhoeven, p., sinn, f., & herden, t. t. (2018). examples from blockchain implementations in logistics and supply chain management: exploring the mindful use of a new technology. logistics, 2(3), 20. https://doi.org/10.3390/logistics2030020 wamba, s. f., queiroz, m. m., & trinchera, l. (2020). dynamics between blockchain adoption determinants and supply chain performance: an empirical investigation. international journal of production economics, 229(2020), 107791. https://doi.org/10.1016/j.ijpe.2020.107791 wan, p. k., huang, l., & holtskog, h. (2020). blockchain-enabled information sharing within a supply chain: a systematic literature review. ieee access, 8, 49645-49656. https://doi.org/ 10.1109/access.2020.2980142. wang, y., jeong, h. h., & beynon-davies, p. (2019). understanding blockchain technology for future supply chains: a systematic literature review and research agenda. supply chain management, 25(1), 62-84. http://dx.doi.org/10.1108/scm-03-2018-0148 yadav, s., & singh, s. p. (2020). an integrated fuzzy-anp and fuzzy-ism approach using blockchain for sustainable supply chain. journal of enterprise information management, ahead-of-print (ahead-of-print ). https://doi.org/10.1108/jeim-09-2019-0301 yadav, v. s., singh, a. r., raut, r. d., & govindarajan, u. h. (2020). blockchain technology adoption barriers in the indian agricultural supply chain: an integrated approach. resources, conservation and recycling, 161. https://doi.org/10.1016/j.resconrec.2020.104877 yong, b., shen, j., liu, x., li, f., chen, h., & zhou, q. (2020). an intelligent blockchain-based system for safe vaccine supply and supervision. international journal of information management, 52. https://doi.org/10.1016/j.ijinfomgt.2019.10.009 zheng, k., zhang, z., chen, y., & wu, j. (2019). blockchain adoption for information sharing: risk decision-making in spacecraft supply chain. enterprise information systems, 1-22. https://doi.org/10.1080/17517575.2019.1669831 https://doi.org/10.3390/logistics2030020 https://doi.org/10.1016/j.ijpe.2020.107791 australasian journal of information systems sultana, teoh & karanasios 2022, vol 26, research article the impact of blockchain on supply chains 27 appendices appendix a search queries examples scopus ( title-abs-key ( “blockchain” and “supply chain”) and pubyear > 2015 and pubyear < 2022 ) and ( limit-to ( srctype, "j" ) or limitto ( srctype, "p" ) or exclude ( srctype, "p" ) ) and ( limit-to ( pubstage, "final" ) ) and ( limit-to ( doctype, "cp" ) or limit-to ( doctype, "ar" ) or exclude ( doctype, "cp" ) ) and ( limit-to ( subjarea, "busi" ) ) and ( limit-to ( language, "english" ) ) ebsco boolean/phrase: ti “blockchain” and “supply chain” expanders * apply equivalent subjects limiters * full text, * peer reviewed * published date: 20160101-20210630 * publication type: academic journal * document type: article * language: english proquest ti(“blockchain” and “supply chain”) date: from 01 january 2016 to 30 june 2021, additional limits-source type: scholarly journals, document type: article, language: english web of science ti= (“blockchain” and “supply chain”) | timespan: 2016-01-01 to 2021-06-30 (publication date) refined by: document type: article, language: english springer open “blockchain” and “supply chain” (date cannot be specified, year 2016 to 2021) ieeexplore ("document title”: “blockchain" and “document title”: “supply chain”) filters applied: journals, 2016 2021 ais e-library title:( “blockchain" and “supply chain”) (date: from 01 january 2016 to 30 june 2021)-for journal title:( “blockchain" and “supply chain”) (date: from 01 january 2017 to 30 june 2021)for conference note* the blue colour is the place for different search keywords from table 2 appendix b example of excluded papers prisma stages exclusion rationale examples of papers excluded screening papers not mentioning “blockchain” and “supply chain” in the title, rather focusing only on the blockchain or only supply chain are screened. • ma, c., kong, x., lan, q. et al. (2019). the privacy protection mechanism of hyperledger fabric and its application in supply chain finance. cybersecur, 2(5), https://doi.org/10.1186/s42400-019-0022-2 • zwitter, a., boisse-despiaux, m. (2018). blockchain for humanitarian action and development aid. int j humanitarian action 3(16), https://doi.org/10.1186/s41018018-0044-5 • park, j. (2021). promises and challenges of blockchain in education. smart learn. environ. 8(33), https://doi.org/10.1186/s40561-021-00179-2 • zhao, j.l., fan, s. & yan, j. (2016). overview of business innovations and research opportunities in blockchain and introduction to the special issue. financ innov, 2(28), https://doi.org/10.1186/s40854-016-0049-2 • guo, y., liang, c. (2016). blockchain application and outlook in the banking industry. financ innov, 2(24). https://doi.org/10.1186/s40854-016-0034-9 • pedersen, asger b.; risius, marten; and beck, roman (2019). a ten-step decision path to determine when to use australasian journal of information systems sultana, teoh & karanasios 2022, vol 26, research article the impact of blockchain on supply chains 28 blockchain technologies, mis quarterly executive, 18 (2). https://doi.org/10.17705/2msqe.00010 eligibility papers keywords stated less than two times in abstract or studying blockchain as en passant, such as an emerging technology or a growing trend around the topic are excluded. • cai, yj, et al. (2021). platform supported supply chain operations in the blockchain era: supply contracting and moral hazards, decision sciences, 52 (4), 866-892. https://doi.org/10.1111/deci.12475 • juan, ihs 2020, 'the blockchain technology and the regulation of traceability: the digitization of food quality and safety', european food and feed law review, 15(6), 563570. https://doi.org/10.13140/rg.2.2.11987.20008 • xu, xp, et al. 2021, 'coordination of a supply chain with an online platform considering green technology in the blockchain era', international journal of production research, ahead-of-print (ahead-of-print), 1-18 https://doi.org/10.1080/00207543.2021.1894367 papers that don’t meet the quality standard that this paper focuses on likepublished in c ranked in abdc ranking, or not ranked at all. • abelseth, b (2018). blockchain tracking and cannabis regulation: developing a permissioned blockchain network to track canada's cannabis supply chain, dalhousie journal of interdisciplinary management, 14. • ada, n, et al. (2021). blockchain technology for enhancing traceability and efficiency in automobile supply chain—a case study, sustainability, 13 (24), 13667. https://doi.org/10.3390/su132413667 • azizi, n, et al. (2021). iot–blockchain: harnessing the power of internet of thing and blockchain for smart supply chain, sensors, 21(18), 6048. https://doi.org/10.3390/s21186048 • bannor, rk & kyire, skc (2021). a review on understanding blockchain technology in global supply chains; opportunities and challenges for agribusinesses, indian journal of economics and development, 17 (4), 917-927. https://doi.org/10.35716/ijed/21283 • clasby, d & wollega, ep (2020). blockchain technology key to veracity in supply chain transaction data, journal of management & engineering integration, 13(2), 1-7. inclusion principally technical papers focusing on, for example, proposing models, technical solutions, frameworks and systems on different blockchain platforms are excluded • agrawal, tk, et al. (2021). blockchain-based framework for supply chain traceability: a case example of textile and clothing industry, computers & industrial engineering, 154, 107130. https://doi.org/10.1016/j.cie.2021.107130 • dolgui, a, et al. (2020). blockchain-oriented dynamic modelling of smart contract design and execution in the supply chain, international journal of production research, 58 (7), 2184-2199. https://doi.org/10.1080/00207543.2019.1627439 • helo, p & hao, y (2019). blockchains in operations and supply chains: a model and reference implementation, computers & industrial engineering, 136, 242-251. https://doi.org/10.1016/j.cie.2019.07.023 • reddy, krk, et al. (2021). developing a blockchain framework for the automotive supply chain: a systematic review, computers & industrial engineering, 157. https://doi.org/10.1016/j.cie.2021.107334 https://doi.org/10.17705/2msqe.00010 http://dx.doi.org/10.13140/rg.2.2.11987.20008 https://doi.org/10.1080/00207543.2021.1894367 https://doi.org/10.3390/su132413667 https://doi.org/10.3390/s21186048 http://dx.doi.org/10.35716/ijed/21283 https://doi.org/10.1016/j.cie.2021.107130 https://doi.org/10.1080/00207543.2019.1627439 https://doi.org/10.1016/j.cie.2019.07.023 https://doi.org/10.1016/j.cie.2021.107334 australasian journal of information systems sultana, teoh & karanasios 2022, vol 26, research article the impact of blockchain on supply chains 29 • yong, b, et al. (2020). an intelligent blockchain-based system for safe vaccine supply and supervision, international journal of information management, 52, 102024 https://doi.org/10.1016/j.ijinfomgt.2019.10.009 appendix c list of papers no papers (journals) 1 ali, mh, et al. (2021). a sustainable blockchain framework for the halal food supply chain: lessons from malaysia, technological forecasting and social change, 170, 120870. https://doi.org/10.1016/j.techfore.2021.120870 2 bai, c & sarkis, j (2020). a supply chain transparency and sustainability technology appraisal model for blockchain technology, international journal of production research, 58(7), 2142-2162. https://doi.org/10.1080/00207543.2019.1708989 3 behnke, k., & janssen, m. f. w. h. a. (2020). boundary conditions for traceability in food supply chains using blockchain technology. international journal of information management, 52(2020), 101969. https://doi.org/10.1016/j.ijinfomgt.2019.05.025 4 bumblauskas, d., mann, a., dugan, b., & rittmer, j. (2020). a blockchain use case in food distribution: do you know where your food has been? international journal of information management, 52(2020), 102008. https://doi.org/10.1016/j.ijinfomgt.2019.09.004 5 chang, s. e., chen, y. c., & lu, m. f. (2019). supply chain re-engineering using blockchain technology: a case of smart contract based tracking process. technological forecasting and social change, 144, 1-11. https://doi.org/10.1016/j.techfore.2019.03.015 6 chen, s., liu, x., yan, j., hu, g., & shi, y. (2020). processes, benefits, and challenges for adoption of blockchain technologies in food supply chains: a thematic analysis. information systems and ebusiness management, 19, 909–935. https://doi.org/10.1007/s10257-020-00467-3 7 choi, t. m. (2020). supply chain financing using blockchain: impacts on supply chains selling fashionable products. annals of operations research, 1-23. https://doi.org/10.1007/s10479-020-036157 8 di vaio, a., & varriale, l. (2020). blockchain technology in supply chain management for sustainable performance: evidence from the airport industry. international journal of information management, 52. https://doi.org/10.1016/j.ijinfomgt.2019.09.010 9 fan, z. p., wu, x. y., & cao, b. b. (2020). considering the traceability awareness of consumers: should the supply chain adopt the blockchain technology? annals of operations research, 309(2), 837-860. https://doi.org/10.1007/s10479-020-03729-y 10 ghode, d., yadav, v., jain, r., & soni, g. (2020). adoption of blockchain in supply chain: an analysis of influencing factors. journal of enterprise information management, 33(3), 437-456. https://doi.org/10.1108/jeim-07-2019-0186 11 ghode, d. j., yadav, v., jain, r., & soni, g. (2021). blockchain adoption in the supply chain: an appraisal on challenges: ims. [blockchain adoption in the supply chain]. journal of manufacturing technology management, 32(1), 42-62. http://dx.doi.org/10.1108/jmtm-11-2019-0395 12 hald, k. s., & kinra, a. (2019). how the blockchain enables and constrains supply chain performance. international journal of physical distribution and logistics management, 49(4), 376-397. https://doi.org/10.1108/ijpdlm-02-2019-0063 13 hayrutdinov, s., saeed, m. s. r., & rajapov, a. (2020). coordination of supply chain under blockchain system-based product lifecycle information sharing effort. journal of advanced transportation, 2020, 10. http://dx.doi.org/10.1155/2020/5635404 14 hew, j. j., wong, l. w., tan, g. w. h., ooi, k. b., & lin, b. (2020). the blockchain-based halal traceability systems: a hype or reality? supply chain management. 25 (6), 863-879. https://doi.org/10.1108/scm-01-2020-0044 15 howson, p. (2020). building trust and equity in marine conservation and fisheries supply chain management with blockchain. marine policy, 115, 103873. https://doi.org/10.1016/j.marpol.2020.103873 https://doi.org/10.1016/j.ijinfomgt.2019.10.009 https://doi.org/10.1016/j.techfore.2021.120870 https://doi.org/10.1080/00207543.2019.1708989 http://dx.doi.org/10.1108/jmtm-11-2019-0395 http://dx.doi.org/10.1155/2020/5635404 australasian journal of information systems sultana, teoh & karanasios 2022, vol 26, research article the impact of blockchain on supply chains 30 16 kamble, s. s., gunasekaran, a., & sharma, r. (2020). modeling the blockchain enabled traceability in agriculture supply chain. international journal of information management, 52, 101967. https://doi.org/10.1016/j.ijinfomgt.2019.05.023 17 kohler, s., & pizzol, m. (2020). technology assessment of blockchain-based technologies in the food supply chain. journal of cleaner production, 269, 122193. https://doi.org/10.1016/j.jclepro.2020.122193 18 kshetri, n. (2018). blockchain's roles in meeting key supply chain management objectives. international journal of information management, 39, 80-89. https://doi.org/10.1016/j.ijinfomgt.2017.12.005 19 kumar, a., liu, r., & shan, z. (2020). is blockchain a silver bullet for supply chain management? technical challenges and research opportunities. decision sciences, 51(1), 8-37. https://doi.org/10.1111/deci.12396 20 lai, j. y., wang, j. t., & chiu, y. h. (2021). evaluating blockchain technology for reducing supply chain risks. information systems and e-business management, 19(4), 1089-1111. https://doi.org/10.1007/s10257-021-00533-4 21 li, j. y., maiti, a., springer, m., & gray, t. (2020). blockchain for supply chain quality management: challenges and opportunities in context of open manufacturing and industrial internet of things. international journal of computer integrated manufacturing, 33(12), 1321-1355. https://doi.org/10.1080/0951192x.2020.1815853 22 martinez, v., zhao, m., blujdea, c., han, x., neely, a., & albores, p. (2019). blockchain-driven customer order management. international journal of operations and production management, 39, 993-1022. https://doi.org/10.1108/ijopm-01-2019-0100 23 min, h. (2019). blockchain technology for enhancing supply chain resilience. business horizons, 62(1), 35-45. https://doi.org/10.1016/j.bushor.2018.08.012 24 niu, b., shen, z., & xie, f. (2021). the value of blockchain and agricultural supply chain parties' participation confronting random bacteria pollution. journal of cleaner production, 319. https://doi.org/10.1016/j.jclepro.2021.128579 25 rao, s. k., gulley, a., russell, m., & patton, j. (2021). on the quest for supply chain transparency through blockchain: lessons learned from two serialized data projects. journal of business logistics, 42(1), 88-100. https://doi.org/10.1111/jbl.12272 26 roeck, d., sternberg, h., & hofmann, e. (2020). distributed ledger technology in supply chains: a transaction cost perspective. international journal of production research, 58(7), 2124-2141. http://dx.doi.org/10.1080/00207543.2019.1657247 27 rogerson, m., & parry, g. c. (2020). blockchain: case studies in food supply chain visibility. supply chain management, 25(5), 601-614. https://doi.org/10.1108/scm-08-2019-0300 28 schmidt, c. g., & wagner, s. m. (2019). blockchain and supply chain relations: a transaction cost theory perspective. journal of purchasing and supply management, 25(4). https://doi.org/10.1016/j.pursup.2019.100552 29 sharma, m. g. (2021). supply chain, geographical indicator and blockchain: provenance model for commodity. international journal of productivity and performance management. ahead-of-print (aheadof-print). https://doi.org/10.1108/ijppm-05-2021-0288 30 sternberg, h. s., hofmann, e., & roeck, d. (2020). the struggle is real: insights from a supply chain blockchain case. journal of business logistics. 24(1), 71-87. https://doi.org/10.1111/jbl.12240 31 sunny, j., undralla, n., & pillai, v. m. (2020). supply chain transparency through blockchainbased traceability: an overview with demonstration. computers & industrial engineering, 150. https://doi.org/10.1016/j.cie.2020.106895 32 tönnissen, s., & teuteberg, f. (2020). analysing the impact of blockchain-technology for operations and supply chain management: an explanatory model drawn from multiple case studies. international journal of information management, 52, 101953. https://doi.org/10.1016/j.ijinfomgt.2019.05.009 33 van hoek, r. (2020). developing a framework for considering blockchain pilots in the supply chain – lessons from early industry adopters. supply chain management, 25(1), 115-121. https://doi.org/10.1108/scm-05-2019-0206 34 van hoek, r. (2019). unblocking the chain–findings from an executive workshop on blockchain in the supply chain. supply chain management: an international journal. 25 (2), 255-261. https://doi.org/10.1108/scm-11-2018-0383 http://dx.doi.org/10.1080/00207543.2019.1657247 https://doi.org/10.1108/scm-11-2018-0383 australasian journal of information systems sultana, teoh & karanasios 2022, vol 26, research article the impact of blockchain on supply chains 31 35 wamba, s. f., queiroz, m. m., & trinchera, l. (2020). dynamics between blockchain adoption determinants and supply chain performance: an empirical investigation. international journal of production economics, 229(2020), 107791. https://doi.org/10.1016/j.ijpe.2020.107791 36 yadav, s., & singh, s. p. (2020). an integrated fuzzy-anp and fuzzy-ism approach using blockchain for sustainable supply chain. journal of enterprise information management, ahead-of-print (ahead-of-print). https://doi.org/10.1108/jeim-09-2019-0301 37 yadav, v. s., singh, a. r., raut, r. d., & govindarajan, u. h. (2020). blockchain technology adoption barriers in the indian agricultural supply chain: an integrated approach. resources, conservation and recycling, 161. https://doi.org/10.1016/j.resconrec.2020.104877 38 zheng, k., zhang, z., chen, y., & wu, j. (2019). blockchain adoption for information sharing: risk decision-making in spacecraft supply chain. enterprise information systems, 1-22. https://doi.org/10.1080/17517575.2019.1669831 papers (conference) 39 herm, l-v & janiesch, c (2021). towards an implementation of blockchain-based collaboration platforms in supply chain networks: a requirements analysis, in hawaii international conference on system sciences 2021 (hicss-54), https://aisel.aisnet.org/hicss-54/st/blockchain_engineering/5/ 40 korpela, k, hallikas, j & dahlberg, t (2017). digital supply chain transformation toward blockchain integration, in proceedings of the 50th hawaii international conference on system sciences, https://aisel.aisnet.org/hicss-50/in/digital_supply_chain/2/ 41 röck, d (2020). the foundation of distributed ledger technology for supply chain management, in 53rd hawaii international conference on system sciences, https://aisel.aisnet.org/hicss53/in/digital_supply_chain/4/ 42 sharma, rs, et al. (2019). design principles for validating use cases of blockchain for food supply chains, in amcis 2019 proceedings, https://aisel.aisnet.org/amcis2019/green_is_sustain/green_is_sustain/4 43 straubert, c, et al. (2021). blockchain technology for tracking and tracing in supply chains: a critical viewpoint, in hawaii international conference on system sciences 2021 (hicss-54), https://aisel.aisnet.org/hicss-54/os/blockchain/5/ 44 sternberg, h & baruffaldi, g (2018). chains in chains–logic and challenges of blockchains in supply chains, in 51st hawaii international conference on system sciences, 3936-3943, https://aisel.aisnet.org/hicss-51/in/digital_supply_chain/3/ appendix d list of outlets no outlets (journal) no. of papers 1 technological forecasting and social change 2 2 international journal of production research 2 3 international journal of information management 6 4 information systems and e-business management 2 5 annals of operations research 2 6 journal of enterprise information management 2 7 journal of manufacturing technology management 1 8 international journal of physical distribution and logistics management 1 9 journal of advanced transportation 1 10 supply chain management 4 11 marine policy 1 12 journal of cleaner production 2 13 decision sciences 1 14 international journal of computer integrated manufacturing 1 15 international journal of operations and production management 1 16 business horizons 1 17 journal of business logistics 2 18 journal of purchasing and supply management 1 19 international journal of productivity and performance management 1 20 computers & industrial engineering 1 https://doi.org/10.1016/j.ijpe.2020.107791 https://aisel.aisnet.org/hicss-54/st/blockchain_engineering/5/ australasian journal of information systems sultana, teoh & karanasios 2022, vol 26, research article the impact of blockchain on supply chains 32 21 international journal of production economics 1 22 resources, conservation and recycling 1 23 enterprise information systems 1 outlets (conferences) 24 hawaii international conference on system sciences (hicss) 5 25 americas conference on information systems (amcis) 1 appendix e table a: review framework focus of the paper • what research question do they address? • what was the objective of the paper? method • is it empirical or conceptual? • is it qualitative or quantitative? • type and amount of data analysed. • how is the data analysed? findings of the paper • what conclusions are drawn? limitations and future research • what limitations are expressed? • what future scope is outlined? implication • what are the stated implications? table b: theme identification levels quotes from studies’ focus and findings themes identified o rg an is at io n al this study focuses on the feasibility and inceptive application of supply chain processes. we proposed a blockchain-based framework along with the use of an affiliated technology, i.e., smart contracts, to derive the feasible benefits of the supply chain process design. (chang et al. 2019, p. 2) > redesign supply chain process supply chain activities are among the ones that are most likely to be transformed by blockchain. among other things, blockchain facilitates valid and effective measurement of outcomes and performance of key supply chain processes (kshetri 2018, p. 88) > valid and effective measurement of outcomes > supply chain processes blockchain improves the efficiency of the process: it reduces the number of operations, reduces the average time of orders in the system, reduces workload, shows traceability of orders and improves visibility to various supply chain participants (martinez et al. 2019, p. 993) > efficiency of the process > reduces the average time of orders > reduces workload > shows traceability > improves visibility the results of our research are based on 10 different applications from the logistics industry. ……… in addition, our investigation is based on a snapshot of the use cases. especially the blockchain technology shows a high dynamic and leads to permanent changes in the business models of the companies. (tönnissen & teuteberg 2020, p. 9) > changes in the business models australasian journal of information systems sultana, teoh & karanasios 2022, vol 26, research article the impact of blockchain on supply chains 33 our study reveals that the effects of dlt on supply chain transactions are two-sided. we found six effects of dlt solutions that have a cost-reducing or cost avoidance impact on supply chain transactions. in addition, we found two effects that change the power distribution between buyers and suppliers in transactions and a single effect that reduces the dependency of supply chain transactions on third parties. while cost reducing and avoidance, as well as dependency-reducing effects, are positive effects, the change in power distribution might come with disadvantages. (roeck et al. 2020, p. 2124) > supply chain transactions cost reduction > reduce dependency > change in power distribution blockchain is projected to be the latest revolutionary technology and is gaining increasing attention from academics and practitioners. blockchain is essentially a distributed and immutable database that enables more efficient and transparent transactions. blockchain reduces transaction costs, as it allows for transparent and valid transactions. (schmidt & wagner 2019, p. 1) > efficient and transparent transactions. > reduces transaction costs workshop participants reported that adoption of blockchain in the supply chain today is very limited but actively considered by many. drivers for this consideration include achieving greater transparency and visibility, as well as, improving processes and reducing costs (van hoek 2019, p. 255). > achieving greater transparency and visibility > improving processes and reducing costs by creating traceable and transparent supply chains for food, consumers can attain the information they need to make informed choices about the food they buy and the companies they support. for stakeholders in the food supply chain, having traceability and transparency builds better relationships with their customers, increases efficiency, and reduces the risk and cost of food recalls, fraud, and product loss. (bumblauskas et al. 2020, p. 1) > traceability and transparency > increases efficiency > reduces the risk and cost of food recalls, we have shown that if the service fees by banks are sufficiently high, adopting blockchain technology is a mean-risk dominating policy which brings a higher expected profit and a lower risk for the supply chain, (choi 2020, p. 1) > increased profit > lower risk the blockchain is a peer-to-peer network of information technology that keeps records of digital asset transactions using distributed ledgers that are free from control by intermediaries such as banks and governments. thus, it can mitigate risks associated with intermediaries’ interventions, including hacking, compromised privacy, vulnerability to political turmoil, costly compliance with government rules and regulation, instability of financial institutions, and contractual disputes. this article unlocks the mystique of blockchain technology and discusses ways to leverage blockchain technology to enhance supply chain resilience in times of increased risks and uncertainty. (min 2019, p. 1) > reduce security and operational risk > enhance supply chain resilience the analysis identified four different blockchain identities which enable scm and improve supply chain performance: information lighthouse, exploitation technology, exploration technology and relationship-building technology. however, the analysis also identified three different blockchain identities which constrain scm and reduce supply chain performance: domination technology, straitjacket and deskiller (hald & kinra 2019, p. 15) > improve supply chain performance > reduce supply chain performance australasian journal of information systems sultana, teoh & karanasios 2022, vol 26, research article the impact of blockchain on supply chains 34 accordingly, we administered a survey in order to review the opinions and views of supply chain practitioners. the results support the model and indicate that blockchain applications can improve supply chain performance. (wamba et al. 2020, p. 1) > improve supply chain performance based on the implementation of a blockchain technology evaluation, supply chains can identify and prioritise opportunities for improvement which may lead to the supply chain transparency impacts that are associated with their blockchain technology, resulting in potentially ensuring product quality and reducing natural environmental concerns (bai & sarkis 2020, p. 2159) > supply chain transparency > ensuring product quality blockchain technology is enabling larger companies to cost effectively protect their brand images. (howson 2020, p. 5) > protect brand images the study results show that the product lifecycle information sharing effort under the blockchain system increases the profit of the whole chain (hayrutdinov et al. 2020, p. 1) >increase profit >information sharing effort we have shown that if the service fees by banks are sufficiently high, adopting blockchain technology is a mean-risk dominating policy which brings a higher expected profit and a lower risk for the supply chain, (choi 2020, p. 1) > higher expected profit > lower risk for the supply chain the use of blockchain technology can reduce transaction costs among spacecraft supply chain members and fulfil information sharing, thus improving the overall profit of spacecraft supply chain system (zheng et al. 2019, p. 16) > reduce transaction costs > information sharing > improving the overall profit our findings indicated that blockchain, as disruptive technology, can help halal food smes achieve food sc transparency (ali et al. 2021, p. 11) >enhance transparency our findings indicated that blockchain, as disruptive technology, can help halal food smes achieve food sc transparency (ali et al. 2021, p. 11) >enhance transparency the results highlight how blockchain is not a stand-alone technology, but rather one element in a system of technologies. while blockchain-based technologies are expected to bring a variety of impacts, only some are directly attributable to the blockchain element: increased transparency, traceability, and trust. (kohler & pizzol 2020, p. 1) >increased transparency, traceability, and trust this study…...find that bct can be used to reduce scrs and optimize scm in various processes of planning, sourcing, making, delivery, and returns. scrs may be reduced through the implementation of bct, including decentralization, transparent and open data, traceability, non-tamperability, and encryption security, enabling companies to more accurately and securely operate their sc activities. (lai et al. 2021, p. 1106) >reduce supply chain risk >optimise supply chain management we found evidence to suggest that a substantial majority of errors in the process were indeed manual (consistent with craig et al., 2015), and enhancing sct through automated, serialized data exchange may provide a path to reduce said errors …….(rao et al. 2021, p. 97) >reduce error in order to adopt blockchain applications into an ongoing business process, a severe transformation is needed in the work culture. this transformation will be extremely difficult for large supply chains as they hold many participants.(sunny et al. 2020, p. 11) >transformation of the business process required the study indicated that dlt enables steady data availability, selective transparency, high authenticity and mutual trust in supply chains. (röck 2020, p. 4532) >enables selective transparency, authenticity and trust australasian journal of information systems sultana, teoh & karanasios 2022, vol 26, research article the impact of blockchain on supply chains 35 in te ro rg an is at io n al our findings imply that supply chain systems have first to be modified and organizational measures need to be taken to fulfil the boundary conditions, before blockchain can be used successfully (behnke & janssen 2020, p. 1) >fulfil boundary condition the findings in the article endorse that the tscm can be made efficient by integrating the bc technology considering five most driving characteristics, namely, data safety and decentralization, accessibility, documentation, data management, and quality. (yadav & singh 2020, p. 1) > data management, safety and decentralization the use of blockchain technology can reduce transaction costs among space craft supply chain members and fulfil information sharing, thus improving the overall profit of spacecraft supply chain system (zheng et al. 2019, p. 16) > reduce transaction costs of members > information sharing > improve the overall profit of the chain our investigation into ten blockchain logistics applications show that there is no disintermediation. in one case, existing intermediaries are replaced by a new intermediary so that reintermediation takes place. (tönnissen & teuteberg 2020, p. 8) > disintermediation and reintermediation marine conservation and global seafood production networks have a trust problem. ……this short commentary has considered how blockchain technology is being leveraged to mend trust issues, enabling new forms of resourcing and fundraising for healthy oceans as well as more transparent fish supply chains. (howson 2020, p. 5) >inter-organizational trust we identified and prioritized key factors: inter-organizational trust and relational governance as organizational challenge that influences adoptability of bt in sc. (ghode et al. 2020, p. 437) > absence of existing interorganizational trust > relational governance the open source blockchain technology seems to offer functionalities beyond those of current legacy technologies; additionally, this technology offers data security and costeffective transmission of transactions security and cost-effective transmission of transactions in peer-to-peer networks with no central system. in this way, blockchain technology simplifies b2b integration and enables micro level iot integration (korpela et al. 2017, p. 10) >b2b and technology integration in this vertical context, the adoption and integration decision of one supply chain actor recursively affects the adoption and integration decisions of the other supply chain actors. (sternberg et al. 2020, p. 1) p-1 > supply chain actors' recursive impact on adoption and integration our analysis in this article suggests that blockchain technology should be deployed selectively, mainly for interorganizational transactions among untrusted parties, and in applications that need high levels of provenance and visibility (kumar et al. 2020, p. 31) > inter-organizational trust > provenance and visibility the paper shows that blockchain can be utilised as part of a system generating visibility and trust in supply chains. from live case studies, we provide empirical evidence that blockchain provides visibility of exchanges and reliable data in fully digitised supply chains. this provides provenance and guards against counterfeit goods. (rogerson & parry 2020, p. 601) > visibility and trust > provenance in network australasian journal of information systems sultana, teoh & karanasios 2022, vol 26, research article the impact of blockchain on supply chains 36 our study reveals that the effects of dlt on supply chain transactions are two-sided. we found six effects of dlt solutions that have a cost-reducing or cost avoidance impact on supply chain transactions. in addition, we found two effects that change the power distribution between buyers and suppliers in transactions and a single effect that reduces the dependency of supply chain transactions on third parties. while cost reducing and avoidance as well as dependency-reducing effects are positive effects, the change in power distribution might come with disadvantages. (roeck et al. 2020, p. 2124) > supply chain transactions cost reduction > reduce dependency > change in power distribution the study results show that the product lifecycle information sharing effort under the blockchain system increases the profit of the whole chain (hayrutdinov et al. 2020, p. 1) >increase profit of the whole chain > information sharing effort from the findings, this research postulates extensive sc integration and regulatory intervention (internal and external competencies respectively) as universal enablers, the fostering of sc integration between firms is mandatory. blockchain remains a technology that aims to simplify and enhance the collaboration between two parties in the sc. (ali et al. 2021, p. 10) >enhance collaboration >supply chain integration the results show that it is conditional for the supply chain to adopt the blockchain technology, and the condition is related to the traceability awareness of consumers, the production costs of the supplier and manufacturer, and the cost of using the blockchain technology. we also find that under a certain condition, the revenue sharing contract can realize a pareto improvement for the supply chain that adopts the blockchain technology.(fan et al. 2020, p. 837) >pre condition to adopt this study presents some key challenges, i.e. to develop trust within sc management system, obey the governance rules and regulations of institution while implementation of bt in sc, provide transparency of data, obtain tamperproof data to avoid fraud, improve coordination and information sharing among the sc partners, choose a profit-earning product sc for adoption of bt, adopt societal changes and train the participants to strengthen behavioral intention towards adoption of bt in sc.(ghode et al. 2021, p. 56) > governance challenges >provide transparency of data and obtain tamperproof data > inter-organizational trust, >improve coordination and information sharing > choosing profit earning product for blockchain >adopt societal changes other impacts such as improved data management are a sideeffect of digitizing non-digital processes. (kohler & pizzol 2020, p. 1) >improved data management to implement de-commodification, disintermediation and adoption of blockchain requires both financial and collaboration of all the actors in the supply chain. (sharma 2021, p. 14) >required collaboration of actors blockchain needs to fit into supply chains and existing technology and may not be the right technology for all supply chain problems at all. in fact, blockchain adoptions may remain targeted on very specific local context and supply chain objectives (van hoek 2020, p. 120) >alignment with existing technology >alignment with context and supply chain objective the use of the blockchain within scns can thus make fraud attempts and manipulations more difficult…. the objective of our contribution was to identify requirements for such a blockchain-based collaboration platform. the requirements should enable a transfer into practice. (herm & janiesch 2021, p. 6872) >requirement for blockchainbased collaboration australasian journal of information systems sultana, teoh & karanasios 2022, vol 26, research article the impact of blockchain on supply chains 37 the logic of blockchains in supply chains give us that we either need a mechanism to establish physical trust, such as a reputation system or a central authority creating trust. industrial experience and our experiences in the relog project reveal that the integration of logistics activities and adoption of supply chain technology may not be straightforward (sternberg & baruffaldi 2018, p. 3941) >challenges integration of logistic activities the study indicated that dlt enables steady data availability, selective transparency, high authenticity and mutual trust in supply chains. (röck 2020, p. 4532) >enable mutual trust blockchains currently do not seem to be the holy grail for tnt in scs. an often-touted advantage of blockchains is that they represent a single point of truth. …. regardless of the technology used, the sc parties must be willing to collect and share trackand-trace data. therwise, transparency and traceability in supply chains are not possible.(straubert et al. 2021, p. 5585) >supply chain parties’ willingness to share >questioning blockchain information vs real-world information in d u st ry the use of blockchain technology can reduce transaction costs among space craft supply chain members and fulfil information sharing, thus improving the overall profit of spacecraft supply chain system (zheng et al. 2019, p. 16) > information sharing >overall sc profit all four of our cases demonstrate kshetri’s (2018) assertion that blockchain offers both immutability and transparency, while wwftraseable, techrock and demeter all support the position that unit-level visibility is possible. all of our cases highlight the need for end-to-end supply chain digitisation (rogerson & parry 2020, p. 608) (food industry) > immutability and transparency, > visibility > end-to-end supply chain digitisation this is one of the main blockchain technology applications in the airport industry. it promotes cooperation between the main players in the aviation industry and the air traffic controllers (atcs) to reduce fragmentation, inefficiency, and uncoordinated operations (di vaio & varriale 2020, p. 1) it also allows information and data sharing, but it is still not possible to observe a high level of sustainable performance (di vaio & varriale 2020, p. 1) > cooperation among industry players >questioning sustainable performance the transparency of information in blockchain-based systems causes issues of private protection. the non-reversibility mechanisms of blockchain result in error intolerance. the theme standardization of data presents benefits of blockchain adoption, which facilitates supply chain data management in complex food systems. however, all the stakeholders face great challenges to define an open and standard data format so as to obtain the benefits (chen et al. 2020, p. 22) >privacy issue and error intolerance in performance > data standardization >defining standard data format blockchain, a technology that could ensure security and transparency, resembles a sound solution. with the participation from all stakeholders along the halal supply chain, a blockchain-based halal traceability system allows end consumers to access complete supply chain information from the origin to their hands (hew et al. 2020, p. 14) > ensure security and transparency >access complete sc information the findings from the study suggest that, among the identified enablers, traceability was the most significant reason for bt implementation in asc followed by auditability, immutability, and provenance (kamble et al. 2020, p. 1) > traceability >auditability, >immutability, >provenance the results reveal that “lack of government regulation and lack of trust among agro-stakeholder to use blockchain” are significant adoption barriers of blockchain in indian asc (yadav et al. 2020, p. 1) > lack of government regulation >lack of trust among stakeholders australasian journal of information systems sultana, teoh & karanasios 2022, vol 26, research article the impact of blockchain on supply chains 38 for stakeholders in the food supply chain, having traceability and transparency builds better relationships with their customers, increases efficiency, and reduces the risk and cost of food recalls, fraud, and product loss. the blockchain technology and this business are creating a case for fixing and transforming the world’s food system. (bumblauskas et al. 2020, p. 1) > fix and transform the world’s food system challenges for promoting greater equity between fisheries stakeholders in the global south will likely prove difficult where compliance attracts unaffordable costs. regulatory forums may prove attractive to fishing outfits keen to prove their compliance. they are unlikely to attract the interest of fishers who operate illegally. (howson 2020, p. 5) > lack of regulatory forum the focus of this paper is on a potential application of blockchain in scqm with detailed analysis of its technical feasibility and the methods of transitioning between the current approaches to a blockchain-oriented platform. …. the survey concludes that blockchain along with iiot can improve scqm in an open manufacturing model (li et al. 2020, p. 1321) >improve supply chain management we identify the industry requirement for the application of blockchain for agriculture. ..application of blockchain technology will be important for regulatory assurance and enduser desirability (sharma et al. 2019, p. 7) > important for regulatory assurance and end-user desirability we find that the powerful retailer and the supplier selling highquality products are better off in blockchain. we further find that blockchain improves the supply chain’s economic sustainability,….(niu et al. 2021) > improves the supply chain’s economic sustainability copyright: © 2022 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v26i0.3755 http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems wathuge & sedera 2022, vol 26, research article saving the environment from the internet 1 saving the environment from the internet: a polynomial mitigation model of reducing individual internet consumption through internet pricing and environmental awareness ayodhya wathuge southern cross university, gold coast, queensland, australia ayodhyawathuge@gmail.com darshana sedera southern cross university, gold coast, queensland, australia abstract the ever-increasing internet usage purports to make substantial damage to the environment as a result of the emissions arising from the internet supporting infrastructures like data centers. however, there has been much less discussion on creating an awareness of the damage that the internet does to the environment. the purpose of this study is to investigate whether the internet use can be reduced at the individual level by calibrating internet price and one’s environmental awareness. the study employs a population-based survey experiment and conducts a polynomial regression analysis using a sample of 326 individuals to understand the conjoined relationship between internet price and environmental awareness. the results indicate that internet price affordability moderated the relationship between environmental awareness and internet usage. although, before inducing awareness, pricing plays a major role in changing consumption, after inducing awareness the significance of pricing decreases, and awareness tends to determine the consumption. moreover, the multi-group analysis infers that age does not show any significance on price affordability and environmental awareness, although males display a high price sensitivity and responsiveness towards awareness than their female counterparts. keywords: environmental awareness, internet use, price, polynomial regression, longitudinal survey. 1 introduction scientists have demonstrated that there is little time left for the planet earth to cross the 'pointof-no-return' concerning climate change (aengenheyster, feng, van der ploeg, & dijkstra, 2018), due to the dangers of increasing temperatures resulting from human actions. a ‘code red for humanity’ has been issued by the united nations following a report of the intergovernmental panel on climate change (ipcc, 2021). environmental scientists, policymakers, and politicians overwhelmingly agree that creating an awareness of the environmental damage is one of the hardest challenges in the process of creating a sustainable environment (ghansah, owusu-manu, ayarkwa, edwards, & hosseini, 2021). for example, some studies deny climate change as a ‘real’ phenomenon (haltinner & sarathchandra, 2021), and some are reluctant to attribute the negative effects of climatic change to human behaviors (hallegatte, 2009; jylhä, tam, & milfont, 2020). as such, there has been a wide range of campaigns to create public awareness of the environmental damage such as earth hour (olexsak & meier, 2014) and nowadays green campaign in malaysia (islam, saidur, rahim, & solangi, 2010) that exemplify the importance of creating public environmental awareness. in australasian journal of information systems wathuge & sedera 2022, vol 26, research article saving the environment from the internet 2 creating such awareness, much of the global attention is focused on the damage caused by transportation, electricity, and fossil fuel combustion (luís, vauclair, & lima, 2018). although there have been few discussions on the purported damage that the rising consumption of information and communication technologies (ict) does to the environment (salahuddin & alam, 2015; yang, zhang, & li, 2019), studies on how the internet can cause environmental damage are scarce (hill, widdicks, & hazas, 2020; widdicks, 2020). yet, in recognising the potential harm of internet use on the environment, countries, and organisations have taken important steps to reduce the impact of the internet by optimising internet support infrastructures (goiri, katsak, le, nguyen, & bianchini, 2013; un, 2019). for example, companies with large data centers are taking steps towards using renewable energy, instead of using fossil fuels to operate their data centers (goiri et al., 2013). major corporations like apple, facebook, and google have dedicated themselves to sourcing 100 % of their energy from renewables in the future (un, 2019). further, providing low-latency mobile services with limited battery power, caching data chunks at optimal edge servers to maximise the integrated utility value, has attracted more attention (li, tang, tang, & luo, 2019). although technological progress makes information technology (it) equipment more energyefficient, many studies acknowledge that the rapid growth in internet consumption outperforms the energy efficiency of such technologies, devices, and algorithms (briscar, 2017; milne, 2014). for example, the total internet usage in the year 2019 equals to 1.2 billion years, considering the average of six hours of internet use per day (kemp, 2019). such high usage contributes to increasing the levels of carbon dioxide (co2). it is expected that 23% of total co2 emissions to come from the internet by 2030 (andrae & edler, 2015). the increasing number of internet-based mobile phone usage/addiction (chi, hong, & chen, 2020; kuss, griffiths, karila, & billieux, 2014), increasing internet-based platforms (kenney & zysman, 2020), narrowing of the digital divide (grishchenko, 2020) and increasing affordability of internet (itu, 2020) lead to high usage that eventually leads to high levels of pollution. finally, the global pandemic of covid-19 has further increased the use of the internet, as more and more individuals and organisations rely on digital engagements. moreover, the majority of internet users are oblivious to the environmental damage caused by internet use (aldossary & djemame, 2016; aldossary et al., 2019). considering the aforementioned factors, this study is designed to investigate how individual internet consumption can be usefully minimised, thereby reducing the potential damage that it has on the environment. more specifically, this study explores the concomitant roles of (i) awareness and (ii) financial disincentives in reducing individual internet use. it draws the theoretical foundations of the theory of value-belief-norms, where it posits that environmental behaviors are basically generated due to biospheric, altruistic, and egoistic values. according to the literature on pro-environmental behavior (eriksson, garvill, & nordlund, 2006; jakovcevic & steg, 2013), such values can be awakened using variables such as awareness and financial disincentives. this study draws its inspirational foundations from the past studies of analogous disciplines such as anti-smoking, drug and alcohol, and road safety that have demonstrated the effectiveness of employing awareness and financial disincentives to modify human behaviors (petrunoff, rissel, wen, & martin, 2015; pope et al., 2020). the use of the internet is conceptually comparable to these disciplines, given that they provide heightened sensations and are addictive in nature (curry, wagner, & grothaus, 1991; lee, han, kim, & renshaw, australasian journal of information systems wathuge & sedera 2022, vol 26, research article saving the environment from the internet 3 2013; thombs & osborn, 2019). however, we observed that past is studies, regardless of the context and certainly not in green-it (chen, hui, su, fang, & hui, 2017; shuja et al., 2017), rarely employed both awareness and financial disincentives together. studies suggest that the simultaneous employment of both awareness and financial disincentives (like pricing) could lead to substantial attitudinal changes, compared to using only one of them (haque, ahmend, & rahman, 2013; shuja et al., 2017; verdonk-kleinjan, candel, knibbe, willemsen, & de vries, 2011). in this paper, we investigate the combined and simultaneous approach of using awareness and financial disincentives to lower internet use (i.e., human behavior). as mentioned, such strategies are commonly employed in anti-smoking, drug and alcohol, and road safety campaigns (chen et al., 2020; izquierdo, ramírez, mcwilliams, & ayuso, 2011; verdonk-kleinjan et al., 2011). drawing parallels with such contexts, we develop a competing model of awareness and financial disincentives to reduce internet use. we argue that combining awareness and financial disincentives would make necessary changes to the individual behavior concerning internet use – thereby reducing its damage to the environment (hobman & frederiks, 2014; stern, 1999). however, such interventions must be carefully employed (kollmuss & agyeman, 2002; whitmarsh & o'neill, 2010). typically, in consumer marketing and economics, consumption is manipulated through a delicate combination of price (of the product or service) and awareness (henriksen, 2012). however, with the reducing prices of internet service provision (itu, 2020), we are yet to determine the combined role of financial disincentives (i.e., internet price) and increased awareness in changing human behaviors (in this case, reducing internet use). therefore, the primary objective of this study is to determine the combined effect of internet price sensitivity and environmental awareness to minimise one’s internet usage. the study further assesses the effects of age and gender on the aforementioned relationship. this paper is structured as follows. the paper commences with a literature review, followed by hypotheses development. next, the theoretical model, experiment design and sample, and methods of analysis are explained. subsequently, the results of the analysis are provided followed by a discussion on the application of the framework identified. finally, the paper concludes with the conclusion, limitations, and future research implications. 2 literature review this section of the paper provides succinct information about the literature the study is based on. first, it provides a brief on the environmental damage internet usage causes. then, a discussion takes place on how awareness and financial disincentives can reduce internet usage thereby minimising the environmental damage. this discussion is done by referring to the related literature from other similar contexts. pricing and awareness have been used in various contexts such as sales, retail, and supply chain for a long time to alter the behaviors of consumers. however, we looked at contexts where a substantial change of behaviors has taken place through manipulation of pricing and awareness strategies, because we consider the usage of the internet as an environmentally harmful behavior that resembles certain aspects of pervasive behaviours, we considered such as addiction. the section ends by summarising the literature on awareness and pricing as the two intervention pathways to reduce internet usage. australasian journal of information systems wathuge & sedera 2022, vol 26, research article saving the environment from the internet 4 2.1 environmental damage of the internet the following explanation provides a short introduction to the primary energy consumption areas of the internet. further, it introduces some internet applications and their future usage predictions that create alarming consequences on the environment. internet is mainly composed of three infrastructures: data centers to store and process data, network equipment to carry data, and end-user devices to view and give commands (un, 2019). each data center is a service provider that provides data and related information services including applications, facilities, networks, servers, and storage resources making it the core infrastructure of the internet (schulz, 2016). by 2019, the number of data centers increased to 8.4 million (liu, 2019), supporting internet services through undisturbed, zerodowntime operations (khalaj, scherer, siriwardana, & halgamuge, 2014). the power required for data centers is generated through electricity (zimmermann et al., 2012), while cooling is done using both water and electricity (zhang, wang, & wang, 2012). data center electricity consumption is growing 30% annually due to the drastic demand increases in internet services, making it the fastest-growing end-use of electricity (han, coles, price, gadgil, & tschudi, 2019). therefore, through data center operations, a huge amount of co2 is emitted as electricity is mainly generated through non-renewable sources. while companies like google (2019) have committed to renewable energy for data centers, it has only a small percentage of the total global internet usage. moreover, internet applications too are gaining high ubiquity. there are approximately 4.48 billion internet users in the world, reaching nearly 60% of the global population (clement, 2019). it is expected that in a few years, the internet will be available to the total global population. moreover, the emergence of technologies like smac-iot (social media like facebook; mobile applications; analytics available even to the consumer; cloud services that offer near-zero or free storage and the predicted growth of internet-of-things) further increase the negative impact of the internet on the environment. according to research done by statista (2019), by 2025 the total iot connected devices are expected to reach 75.44 billion worldwide, which is a fivefold increase in ten years. such developments will further increase the individual consumption of the internet and internet services. as such, the awareness of its potential damage to the environment becomes ever more important. for example, a simple behavioral change like downloading vs. streaming, frequency of checking of social media, unnecessary and spam emails beg the question of the impact of individual internet practices. therefore, it is worth reducing individual internet consumption. in doing so, we looked at previous studies that have used pricing and awareness to reduce socially confined behaviors, to identify types of pricing and awareness strategies. 2.2 the role of awareness in changing human behavior among many factors that generate behaviour change, awareness appears as one of the most primary and important change agents. many behaviour change frameworks consider creating awareness as the first step in modifying behaviour (de vries et al., 2003; jenkin, webster, & mcshane, 2011). therefore, in contexts such as anti-smoking, drink driving, and speed driving awareness is being used as a prominent strategy to change behaviour. past studies in the anti-smoking context use different types of awareness campaigns to reduce the initiation and continuation of smoking. counter industry media campaigns raise awareness of persuasive intent of protobacco messages empowering individuals to be less australasian journal of information systems wathuge & sedera 2022, vol 26, research article saving the environment from the internet 5 receptive to tobacco promotions (hershey et al., 2005; syu, huang, & huang, 2010). in a longitudinal study, hershey et al. (2005) show that teens became less receptive to tobacco promotions with a counter-industry media campaign. they show that teens who received a higher level of campaign exposure displayed more negative beliefs about tobacco industry practices and more negative attitudes towards the tobacco industry leading to low receptivity to protobacco advertising which showed less progression towards smoking intentions and behaviour. moreover, past studies show that creating awareness of the benefits of quit smoking along with the consequences of smoking leads to reduce adult smoking depicting that people in higher age brackets are responsive to awareness (brewer et al., 2016; farrelly et al., 2017). studies show that awareness is a significant factor in reducing drink driving (lewis, watson, & white, 2008; nathanail & adamos, 2013; sheehan et al., 1996). sheehan et al. (1996) show that interactive educational programs that provide confidence to face pressures by replying to persuasive arguments and alternative strategies to avoid drink driving helped improve current social attitudes and behaviour change. however, employing an experimental approach nathanail and adamos (2013) show that drivers who were exposed to communication campaigns that create awareness of the permissible alcohol level while driving and risks of drink driving showed no significant effect in controlling the behaviour. moreover, significant behaviour changes can be observed in speed driving contexts (mckenna, 2007; poulter & mckenna, 2010; ranjit, snyder, hamilton, & rimal, 2017). poulter and mckenna (2010) and mckenna (2007) using a sample of school students show that educational interventions improved pre-driver road safety beliefs including speed driving. this summary concludes that awareness creates a significant impact on behaviour change although a few studies do not. however, the positive behaviour change outperforms the neutrality in behaviour for awareness. although there are studies in other contexts incorporating awareness, the studies conducted related to individual internet usage and sustainability are very limited. studies have developed several frameworks related to green it/s, and they acknowledge the importance of creating an overall awareness of energy conservation and carbon emissions facilitating green is practices (elliot & binney, 2008), contextual factors affecting green assimilation (cooper & molla, 2014), green it readiness models (molla & cooper, 2010) and trends and patterns related to green it (sedera, lokuge, tushi, & tan, 2017). moreover, studies have developed instruments to measure the environmentally sustainable it performance drivers (molla, 2013). however, the studies based on these frameworks address only the national and organisational level strategies to minimise internet damage. past studies suggest that creating awareness at the societal level or the individual level is critical and however, is sparse (jenkin, webster, et al., 2011). further, studies suggest that even organisations are in ‘infancy stages of awareness’ of green it/is practices, implying that awareness of environmental pollution of the internet is very low (jenkin, mcshane, & webster, 2011). few studies have been conducted in information systems literature promoting eco-sustainable behaviours by employing gamification techniques (seidler et al., 2020), human-computer interaction techniques (hill et al., 2020; widdicks, 2020) and an awareness strategy (wathuge & sedera, 2021) to reduce the individual internet usage to minimise the climate change. however, there are no studies that investigate the combined effect of awareness and another strategy. therefore, leaning on literature we australasian journal of information systems wathuge & sedera 2022, vol 26, research article saving the environment from the internet 6 investigate the effect of awareness on internet usage to test whether it creates an impact on behaviour. the hypothesis is; h1: increased environmental damage awareness creates a positive impact on reducing internet usage. 2.3 the role of price in changing human behaviour pricing also plays a major role in changing human behaviour. it is being used to motivate as well as demotivate individuals in engaging in certain behaviours (aebli, volgger, & taplin, 2021; ball et al., 2011). pricing acts as a deterring agent in the contexts of anti-smoking, drinkdriving, and speed driving. past studies show that pricing creates an immediate impact on behaviours. moreover, anti-smoking literature summarises that imposing new taxes on cigarette packets as well as increasing existing taxes lead to reduce smoking (ferrer, orehek, scheier, & o'connell, 2018; pope et al., 2020; verdonk-kleinjan et al., 2011). in a study to decrease the cigarette demand, while increasing the demand for a substitute, pope et al. (2020) show that taxes on cigarettes play a major role in reducing the demand for cigarettes while increasing the tendency towards using the substitute. moreover, investigating tax increases of cigarette packets over a long period, ferrer et al. (2018) conclude that increase in taxes has a substantial effect on the consumption of cigarettes among smokers low in behavioural disengagement. price regulations related to drink driving and speed driving are mainly imposed as fines. according to past studies, the amount of fine is determined by the blood alcohol level (nagata, setoguchi, hemenway, & perry, 2008) for drink driving, and exceeded speed range or offence history (delhaye, 2007) for speed driving. nagata et al. (2008) show that by decreasing the permissible blood alcohol level and increasing penalties, large and immediate public health benefits measured through factors such as alcohol-impaired traffic injuries, alcohol-impaired severe traffic injuries, and alcohol-impaired traffic fatalities per billion kilometers driven can be observed. similarly, izquierdo et al. (2011) demonstrate that introduction of a penalty point system based on the exceeded speed limit to control road accidents due to irresponsible driving resulted in decreasing the number of fatalities. these studies extracted from the past literature show that pricing strategies employed according to the severity of offence were effective in changing behaviour resembling prohibitive behaviour interpretations. when studying the types of pricing strategies, we could observe that past studies have used different types of pricing models. the primary pricing models identified are flat-rate, blockrate, and volume-based pricing (chen et al., 2020). in relation to internet pricing, researchers have investigated economic (bateni, hajiaghayi, jafarpour, & pei, 2011; lv & rouskas, 2010) and network congestion (nevo, turner, & williams, 2016; sen, joe-wong, ha, & chiang, 2015) specific pricing strategies to maximise profits and to increase resource utilisation which lacks the environmental perspective. near zero internet prices increase the internet demand exponentially, and more importantly, it creates irresponsible internet practices such as unattended music or video playing, addictive social media usage, and addictive online gaming behaviours. therefore, it is of high necessity to determine the effect of internet price changes on internet usage to eventually reduce its negative environmental impact. therein, owing to the past studies we test the hypothesis, h2: increased internet pricing creates a positive impact in reducing internet usage. australasian journal of information systems wathuge & sedera 2022, vol 26, research article saving the environment from the internet 7 as described earlier, awareness is an important factor that creates changes in the behaviours of smokers, drink and speed drivers. however, some studies show that the effect awareness alone creates is not significant in modifying the aforementioned behaviours (nathanail & adamos, 2013; schaap et al., 2008; steptoe et al., 2002). steptoe et al. (2002) show that risk awareness inconsistently relates to smoking behaviour in countries that have lower education and economic levels (verdonk-kleinjan et al., 2011). moreover, highly addicted substance users show low or no response to awareness. to enhance the relationship between awareness and behaviour change in instances such as the ones mentioned above, scholars suggest using more than one strategy (frieden et al., 2005; giesbrecht & greenfield, 2003; verdonk-kleinjan et al., 2011). studies have used pricing as a strategy to enhance the effects that awareness creates on human behaviour (oguchi, 2016). similarly, to enhance the impact of awareness on internet usage we use the price variable as a moderator of the relationship between awareness and internet usage. if financial disincentives significantly influence behaviour change, it follows that financial disincentives will have a significant moderating role in the relationship between awareness of environmental damage and behaviour change. therefore, the hypothesis is; h3: increased internet pricing moderates the relationship between environmental damage awareness and internet usage. as observed above, changes to human behaviour in smoking, drink driving, and speeding have been achieved through a combined effort of both pricing and awareness. as stipulated in hypothesis 3, we also argue that the combined effect of internet pricing and environmental awareness would minimise the use of the internet. however, we recognise that such a combination in this context is likely to yield a spectrum (or a continuum) of types of behaviours, rather than a unified behaviour. past studies show that behaviours concerning price can be (i) resource exploitive, where the consumer engages in maximising his utility by exploiting the resources (watts & zimmerman, 1978), (ii) principle and value-based, where principles and values acquire a major proportion in an individual's cognition than existing practical conditions (kivetz & tyler, 2007), (iii) logical, where the behaviour is logically true to existing logical interpretations priem and butler (2001) and (iv) prohibitive, where external prohibitive factors like price discourage intention to behave (liang, farh, & farh, 2012; lin & johnson, 2015). 2.4 theoretical foundations a fundamental premise of consumer behaviour is that a consumer develops certain attitudes towards products or services that would lead to actual behaviours (luchs & mooradian, 2011). multiple approaches have emerged to explain the progression from attitudes, such as awareness (brounen, kok, & quigley, 2013) and price consciousness (alford & biswas, 2002), that lead to behaviours. for example, the theory of planned behaviour, social identity theory, means-end theory, and norm activation model, which was later extended to create the valuebelief norm (vbn) theory can be named as a few. vbn theory is selected as the theoretical premise of this study. vbn theory is a theory of environmentalism which is built on the causality of five variables: human values (biospheric, altruistic, and egoistic) which influence environmental cognitions, new ecological paradigm— belief that human actions deteriorate the environment quality, awareness of consequences of human actions on the environment, the ascription of responsibility that oneself should take actions to protect the deteriorating australasian journal of information systems wathuge & sedera 2022, vol 26, research article saving the environment from the internet 8 values, and building up a sense of personal obligation to act upon the ascription of responsibility, which leads to actual behaviour (stern, dietz, abel, guagnano, & kalof, 1999). vbn theory has been extensively employed in studies on the socio-psychological perspectives of environmentalism (liobikienė & poškus, 2019; stern, 2000), where defined behaviourally as the propensity to take actions with pro-environmental intent. the importance of choosing vbn theory as the philosophical lens of our study lies in a few factors. firstly, unlike other theories, it precisely recognises that individual behaviour may also be rooted in altruistic or biospheric values. this helps in profoundly grasping our study’s core idea well. that is the mitigation of internet consumption despite economic benefits that occur from one's egoistic values. the theory supports the ascription of responsibility for things an individual value personally, and shows the causal relationship of how personal values, beliefs, and norms shape pro-environmental behaviour. secondly, the vbn theory postulates a mechanism: norm activation, which can decrease ‘rational utility calculus’ considered in most ‘subjective utility logic’: the economic advantage of an opportunity, on which most other theories are based (stern, 1999). in proving that stern states, processes that bypass subjective utility are important for understanding behaviours that lack egoistic justifications, whereby an individual act selflessly without expecting any personal benefit like such we expect from the internet users. therefore, we chose the vbn theory which most accurately provides the philosophical lens to our study. the vbn theory which the study is based on has been used in several instances to investigate the effect of awareness and pricing on behaviour (eriksson et al., 2006; jakovcevic & steg, 2013). pricing is a measure of sacrifice for individuals in purchasing a product/service (zeithaml, 1988). as such, it provides market signals to change the demand or supply. vbn theory shows that monetary aspects which determine egoistic values at times prevent the execution of pro-environmental actions. for example, (lópez-mosquera & sánchez, 2012) show that egoistic values negatively affected the willingness to pay for conservation. however, the same value can be used to restrict individuals from performing environmentharming actions (eriksson et al., 2006; sahin, 2013). on the other hand, awareness can be used to heighten the biospheric and altruistic values of an individual (eriksson et al., 2006; jakovcevic & steg, 2013; pooley & o’connor, 2000). it would change the attitudes, activate personal norms, and moral obligations towards the target behaviour (ebbs, dahlhaus, barton, & kandra, 2018; pooley & o’connor, 2000). therefore, using a combination of pricing and awareness would heighten the values of individuals, and as they are complementary in nature and not substitutes, putting them together would generate environment-friendly results. through his studies, stern (2000) makes several recommendations, where he proposes investigating the causal relationships of ‘personal financial capabilities’, ‘general environmentalist predispositions’, and ‘change in lifestyle’ around which our study is based. the vbn theory has been used in several instances to investigate the effect of awareness and pricing on behaviour (eriksson et al., 2006; jakovcevic & steg, 2013). there are different types of pricing strategies for the internet such as flat-rate strategy, block-rate strategy, and volumetric rate strategy that are specifically location-wise popular. through the study, we observe the behaviour outcome of internet users by considering the affordability of internet pricing and environmental awareness. the research model, embedded in the study design depicted in figure 1 shows how the effects of personal financial capabilities (measured using ‘price affordability’ (fan, 2005)) and australasian journal of information systems wathuge & sedera 2022, vol 26, research article saving the environment from the internet 9 environmental predisposition (measured using the ‘environment awareness’ (arlt, hoppe, & wolling, 2011)) changes the lifestyle through pro-environmental intent measured using the ‘internet usage’ (abrahamse & steg, 2011). therein, the independent variables are price affordability and awareness while the dependent variable is internet usage. figure 1. study design 3 the experiment design and sample a population-based survey experiment was used as the method that allows the combination of the causal power of experiments and generalisability of population-based samples (mutz, 2011). population-based survey experiments possess the design of an experiment, while a survey is embedded to capture the data from a representative population-based sample. the experiment was conducted online as the method allows to conduct it remotely. to test the model, a survey consisting of questions related to before and after effects of awareness, pricing, and internet usage was employed. the survey consisted of questions related to demographic characteristics of respondents, awareness of internet damage on the environment: measured using the perception they hold on to environmental damage their internet usage cause, and price affordability: measured using perception about prices of internet data packages and their affordability. further, we included questions related to internet usage: measured using time spent on the internet, the probability to change current usage pattern, frequency of use, and addiction to it. a seven-point likert scale ranging from 1strongly disagree to 7strongly agree was utilised to capture the responses. after the study-1 control group survey data collection was completed, we induced the subjects with a tailor-made video of four minutes about the damage that internet consumption does to the environment. to test the effects of the treatment, subjects were provided with the same survey again. it included questions to assess the attitudinal change that occurred after watching the video. there, the current intention to use the internet was questioned. therein, we test the simultaneous roles of the targeted awareness campaigns (maibach, 1993) and pricing (fan, 2005), which are purported to be altering behaviours. the experiment was fielded within 15 days (mcginty, goldman, pescosolido, & barry, 2015; mcginty, webster, & barry, 2013; pedulla & thébaud, 2015). treatment effects were measured by the difference between the level of awareness, price sensitivity, and internet usage of the control group and experimental group. 3.1 treatment through keen observation of several videos, we selected a video published by a public news channel of france named france 24 on youtube (france24, 2017), as it contained relevant content proved in the literature that could create awareness, and also the video length was appropriate. the video provided information about the amounts of co2 emissions of most australasian journal of information systems wathuge & sedera 2022, vol 26, research article saving the environment from the internet 10 popular internet-based activities such as circulating emails and web searches. moreover, it defined the concept of data centers and their energy requirements. overall, the video provides a succinct awareness of the damage internet does to the environment. to capture the number of people who have watched the video before and after the questionnaire was provided, we kept a count on the number of youtube views. figure 2 frames provide three captures to show the content of the video. figure 2. screen captures of the video 3.2 measures we adapted survey questions, by reviewing literature related to the constructs and identifying different ways that they can be conceptualised to best capture the aspects of price affordability, awareness, and internet usage. further, we assured that the questionnaire includes questions that acquire necessary inputs that we need to measure in the analysis acquiring content and face validity (lawshe, 1975). questions related to measuring awareness were adapted from studies such as (bulgurcu, cavusoglu, & benbasat, 2010; morgil, seçil, seçken, yavuz, & oskay, 2004; yucedag, kaya, & cetin, 2018). questions related to pricing were derived and adapted from (kuruzovich, viswanathan, agarwal, gosain, & weitzman, 2008; zettelmeyer, morton, & silva-risso, 2001), and questions related to internet usage were derived from (mathwick & rigdon, 2004; venkatesh, brown, maruping, & bala, 2008). 3.3 sample the population considered for the study was global internet users. to test the model presented in figure 1, a representative sample was drawn from the sample frame of ordinary internet users of sri lanka. according to kothari (2004), the sample frame should be represented in a "list". however, lists that contain details of general internet users are not available (fricker, 2008). furthermore, list-based sampling frames such as university email lists, email lists of organisations may restrict the sample to specific characteristics such as highly educated or financially strong subjects (fricker, 2008). therefore, we distributed the link of the questionnaire randomly among the internet users we meet and through social media accounts such as facebook, linkedin, and instagram to reduce the sample biasness. moreover, we encouraged the receivers of the questionnaire to distribute it randomly among the internet users they meet. in selecting the sample, we used stratified sampling technique to stratify the population according to gender. the global internet user gap of male vs female is 17% as per the international telecommunication union (itu, 2019). that means males who use the internet are approximately higher by 17%. henceforth, we randomly recruited male and female respondents that match the above statistic by distributing the survey link among the sri lankan internet users. finally, we acquired a sample of 326 internet users. according to our sample, the gap between internet using males (58.3%) and females (41.7%) was 16.6% which approximately equaled the global internet user gender gap. australasian journal of information systems wathuge & sedera 2022, vol 26, research article saving the environment from the internet 11 a larger sample would be more appealing to the generalisation of the results. however, the current sample does well represent the global internet users who use the internet on average 6 hours and 42 minutes a day (kemp, 2019) which approximately resembles sri lanka's average internet usage time (kemp, 2019). the majority of the respondents in the sample are young adults. therefore, the sample represented a trustworthy number of major internet users, because global internet usage is more skewed towards young adults and their general awareness is low (clement, 2020; thilina, guruge, & nanayakkara, 2020). it was necessary that the sample comes from a single geographical area, have a similar educational background, and have similar economic conditions to increase the homogeneity and to control the effect of extraneous variables. therefore, internet users based in sri lanka were derived. moreover, in selecting the sample, we looked at a location where people are more price sensitive and literate as we observe price sensitivity and awareness. due to price fluctuations, price sensitivity can be easily identified. moreover, the price model is also sought in the sample. developed countries have flat-rate pricing and price-sensitivity varies based on the price model. based on these factors we selected sri lanka which is a developing country that has a high literacy rate. further, the country offers block-rate pricing where price sensitive internet users are prominent. the demographics and categorical values derived through the survey are summarised in table 1 below. the categories were created by using the original input of each survey respondent, without making any adjustments or modifications. age # % internet access # % below 25 75 23 limited 266 82 26-40 240 74 unlimited 60 18 above 41 11 3 cost of the internet package gender below 1000 lkr1 91 28 male 190 58.3 1001 – 2000 lkr 149 46 female 136 41.7 above 2001 lkr 86 26 internet connections mobile 68 21 home 26 7 both 236 72 table 1. demographics and categorical values note: 1lkr is the currency of sri lanka when doing the analysis, we considered several control variables, that are extraneous variables not linked to the hypotheses and theories being tested (spector & brannick, 2011). they are the geographical location and price model. by selecting sri lanka, we have controlled the effects of geographical location and price model on the hypotheses being tested. the effects of age and gender on internet usage are calculated in the multi-group analysis in a later section. 4 analysis and results for the analysis, we employed three types of tests: partial least squares (pls) structural equation modeling (sem) method (benitez-amado, henseler, & castillo, 2017; benitezamado, llorens-montes, & fernandez-perez, 2013), multi-group analysis (mga), and polynomial regression (atwater, waldman, ostroff, robie, & johnson, 2005; edwards & parry, 1993) with response surface methodology (box & draper, 1987; cornell & khuri, 1987). the pls analysis was employed to validate the constructs and derive the mga, while the australasian journal of information systems wathuge & sedera 2022, vol 26, research article saving the environment from the internet 12 polynomial regression and response surface analysis were used to understand the interplay between price and awareness towards internet usage in the control and experimental groups of the study. the pls-sem is well suited for highly complex predictive models, which supports the mapping of formative observed variables (henseler & sarstedt, 2013; sedera, gable, & chan, 2003; wold, 1989). when conducting polynomial regression and response surface methodology, guidelines of (shanock, baran, gentry, pattison, & heggestad, 2010) were followed. 4.1 pls analysis the smart-pls software was used to do the pls analysis. to test the overall fit of the proposed research models (figure 1), we conducted a confirmatory composite analysis by following the guidelines (henseler et al., 2014) and (benitez, henseler, & castillo, 2017). the models were tested using adanco 2.0.1 software (dijkstra & henseler, 2015), with the bootstrap resampling method (4,999 resamples). it followed the guidelines of (dijkstra, 2010), and subconstructs were estimated by using the regression weights (mode b). therein, we observed the standardised root mean squared residual (srmr), unweighted least squares (uls) discrepancy (duls), and geodesic discrepancy (dg), which let us assess the appropriateness of the composite model. the srmr of the a-priori model was 0.033 for the control group (study-1) and 0.041 for the experimental group (study-2)– well below the recommended threshold of less than 0.080 – at the 0.05 alpha level (dijkstra & henseler, 2015; hu & bentler, 1999). the duls = 0.103 and dg = 0.017 for the control group indicated that we could confirm with a probability of 5% that the measurement structure of our proposed model was correct, demonstrating good model fit (dijkstra & henseler, 2015; hu & bentler, 1999). next, we established construct validity using the average variance extracted (ave). therein, we observed that for each construct the ave was greater than the variance shared between the construct and the other constructs in the model, thus indicating strong construct and discriminant validity. moreover, the strong and significant path coefficients for both pricing and awareness were shown (study-1 for pricing was (0.401) and study-2 (0.522), and the path coefficients of awareness for study-1 was 0.376 and for study-2 it was 0.608). finally, the r2 was determined for study-1 and study-2, revealing 0.22 and 0.31 respectively for study-1 and 2. the results confirm the overarching theoretical foundations (figure 1), encouraging further investigation into the associations with pricing, awareness, and internet usage. 4.2 multi-group analysis a multi-group analysis permits to test whether the pre-defined subpopulations generate significant heterogeneous observations (henseler, 2007). therein, it tests the moderating effect of the subpopulations on the relationship between the observed independent and dependent variables (henseler & chin, 2010; henseler, ringle, & sinkovics, 2009). the multi-group analysis commenced with the micom (measurement invariance of composite models) procedure, which assessed the invariance (henseler, ringle, & sarstedt, 2016). henseler et al. (2016) recommend the use of micom, employing a three-step approach to analyse (i) configural invariance, (ii) compositional invariance, and (iii) the equality of composite mean values and variances. the results confirmed the three types of invariance, which implied that measurement invariance holds, and that a multi-group analysis is therefore possible (hair et al., 2018; henseler et al., 2016; schlagel & sarstedt, 2016). meaning that the moderating variable has no effect on the loadings of the elements in the model. after confirming the existence of invariance, the next step was to apply the multi-group analysis, comparing the explained australasian journal of information systems wathuge & sedera 2022, vol 26, research article saving the environment from the internet 13 variance for each group using appropriate categorical variables. in this case, age and gender analyses were carried out to identify if they create any impact. the last step of the multi-group analysis is to analyse the differences between coefficients in different paths. if these differences are significant (if the p-value is smaller than 0.05 or larger than 0.95 for the difference of groupspecific path coefficients (see (henseler et al., 2009))), the categorical variable has a moderating effect. the age variable was divided into two subpopulations: below 25 years and above 25 years. researchers suggest that younger persons are more environmentally perturbed than people of older age because environmentalism is a channel that shows younger persons’ comparatively low contribution to the social order and the dominant value system (abdul-wahab & abdo, 2010; arcury & christianson, 1993; bell, greene, fisher, & baum, 1996). however, some literature suggests that environmental awareness grows with age as people get more mature and exposed to knowledge (aminrad, zakaria, & hadi, 2011; ziadat, 2010). nevertheless, the mga showed no significant differences between the age groups. further, past studies suggest that age also affects price sensitivity (chaloupka & grossman, 1996). younger a person, their disposable income is comparatively lower than those of higher age brackets, indicating that they are more sensitive towards price changes (chaloupka & grossman, 1996; ding, 2003; hollingworth et al., 2006). however, price sensitivity might affect comparatively less on younger aged groups, if they are dependent on their parent’s or guardian’s income or if they are conscious of certain brands or products (muratore, 2016). the two age groups depicted no differences in the analysis for the price (see table 2). group 1 age = < 25 = 75 group 2 age > 25 = 251 group 1 vs. group 2 β age = < 25 β age > 25 | (β age = < 25 β age > 25) | tvalue significance p-value price affordability >> internet usage 0.571 0.518 0.053 .863 not significant 0.471 env: awareness >>internet usage 0.489 0.416 0.073 .881 not significant 0.437 table 2. multi-group analysis of sub-samples based on age these results could be due to the high literacy rate prevailing in sri lanka, where even younger persons have the maturity to grasp awareness, and on the other hand due to cheaper data prices of sri lanka. results of the mga depicted in table 2 demonstrate that there were no significant differences between the age groups for both relationships. gender is another factor that we considered in the mga. the literature proposes that males are more aware of environmental damage and engage in more pro-environmental behaviours than females because of their comparatively high engagement in the community and political issues (abdul-wahab & abdo, 2010; mcevoy, 1972). in the notion of price sensitivity, the literature also suggests that males are more price sensitive, and hence more responsive to price changes than females (andreoni & vesterlund, 2001). results of the mga based on gender revealed significant differences in how price affordability and environmental awareness affected their respective internet use. it is observed (see table australasian journal of information systems wathuge & sedera 2022, vol 26, research article saving the environment from the internet 14 3) that men were more sensitive to both price affordability and environmental awareness showing an impact on their internet use, as compared to our female sample. group 1 male = 190 group 2 female = 136 group 1 vs. group 2 β male = 190 β female = 136 | (β male = 190 β female = 136) | tvalue significance p-value price affordability >> internet usage 0.613 0.586 0.027 3.744 significant 0.001 env: awareness >> internet usage 0.629 0.515 0.114 5.193 significant 0.001 table 3. multi-group analysis of sub-samples based on gender in the notion of price sensitivity, the literature also suggests that males are more price sensitive, and hence more responsive to price changes than females (andreoni & vesterlund, 2001). results of the mga based on gender revealed significant differences in how price affordability and environmental awareness affected their respective internet use. it is observed (see table 3) that men were more sensitive to both price affordability and environmental awareness showing an impact on their internet use, as compared to our female sample 4.3. polynomial regression polynomial regression is a technique to model the relationship between multiple independent variables (x and y) concerning a dependent variable (z) through a non-linear relationship (shanock et al., 2010). similar to polynomial regression (edwards & parry, 1993), response surface methodology – abbreviated as ‘pr and rsm’ – (box & draper, 1987; cornell & khuri, 1987) is a common technique that analyses non-linearities. pr and rsm allow researchers to examine the extent to which the combinations of two predictor (independent) variables relate to an outcome (dependent) variable. according to venkatesh and goyal (2010), linear models do not bring out the theory projected complications appropriately. the pr and rsm are ideal to investigate the purported relationship between price and awareness and its hypothesised relationship with the internet use of the consumer. as such, we labelled price as x variable, awareness as the y variable, whilst internet use was labeled as the outcome variable z, where the equation was, 1. internet use = ƒ (price*, awareness**) the complete empirical specification was: 2. internet use = β0 + β1 price* + β2 awareness** + β3 price2 + β4 (price x awareness) + β5 awareness2 + e we followed the procedure outlined in (shanock et al., 2010) to run the polynomial regression analysis on the data collected from 326 consumers to create the response surface as shown in figure 3. panel (a) of figure 3 represents the data of study-1 and panel (b) of figure 3represents data of study-2. australasian journal of information systems wathuge & sedera 2022, vol 26, research article saving the environment from the internet 15 figure 3. response surfaces of study-1 and study-2 according to table 4, a significant a1 indicated that the line of the agreement had an additive relationship to the outcome variable. a positive a1 indicated that the outcome effect study phase coefficient std error test stat (t) significance a1: slope along x=y (as related to z) study-1 -0.21 0.09 -9.226 significant study-2 -0.47 0.03 -12.988 significant a2: curvature on x=y (as related to z) study-1 -0.48 0.02 -12.014 significant study-2 0.34 0.03 5.534 significant a3: slope along x=-y (as related to z) study-1 0.15 0.06 7.13 significant study-2 0.07 0.02 0.73 non-significant a4: curvature on x=-y (as related to z) study-1 0.09 0.18 1.11 non-significant study-2 -0.31 0.73 -11.682 significant table 4. longitudinal assessment of study 1 and study 2 variable increased as both predictor variables increased, whereas a negative a1 indicated that the outcome variable decreased as both predictor variables increased. a significant a2 for study-1 indicated a non-linear slope of the line of perfect agreement (agreement in the two predictor variables related to the outcome in a nonlinear way) (shanock et al., 2010). the significant and negative values of a2 in study-1 suggested that the line of perfect agreement as it relates to internet use was positive and a convex surface (upward curving). for study-2 we observed a concave surface with downward curving. next, we interpret how the degree of discrepancy between price and awareness relates to the outcome variable internet usage, by assessing the curvature along the line of incongruence (x = -y) as it relates to internet usage with a4. we observed, significant negative a4 for study-2, and non-significant for the data of study-1 showed a slight concave surface (see figure 3), that is, internet use would increase slightly as the degree of discrepancy between breadth and depth decreases. finally, we considered a3 to establish the direction of the discrepancy related to the outcome, as indicated by the slope of the x = -y line (line of incongruence) as it relates to internet usage. a significant negative a3 for study-2 (see table 4) indicated that internet usage was higher when the discrepancy was such that price was higher than awareness and vice versa. australasian journal of information systems wathuge & sedera 2022, vol 26, research article saving the environment from the internet 16 5 application of the framework encouraged by the polynomial regression results and consistent with the notions presented in the literature review, we see a continuum of internet user behaviours relating to the combined effects of internet pricing and environmental awareness. figure 4 presents four types of internet user behaviours, arranged in a 2x2 matrix associated with price and awareness. the framework is intended as a practical tool to guide future directions in relation to this research. therein, we created four possible self-explanatory behaviour types: opportunistic, idealistic, tautological, and prohibitive – each representing an internet use category for the four 'highlow' combinations of awareness and price. in figure 4, the position where price and awareness are low is termed as 'opportunistic'. it indicates that the consumer engages in maximising his utility by exploiting the resources (watts & zimmerman, 1978). our analysis showed it as the category that represents the highest intention to use the internet, asserting to the characteristics found in the literature of exploitive usage of resources. rivera and de leon (2004) in their study show that ski-riding for free and lack of monitoring has made riders exploit the resources. these instances show that when there are fewer restrictions people tend to exploit resources for their advantage. similarly, the behavioural intention identified through internet consumers in study1 is consistent with the opportunistic assumptions. however, in study-2, even after inducing awareness, the opportunistic behaviour of consumers remains as it is. this implies that for behavioural changes of opportunistic consumers inducing awareness will not create any changes in personal values and attitudes when the price remains low. figure 4. the application framework that summarises findings using a 2x2 matrix of price and awareness on internet usage the combination of high-awareness and low-price in figure 4 is denoted as ‘idealistic’. idealism is recognised as behaviour that emerges when principles and values acquire a major proportion in an individual’s cognition than existing practical conditions (kivetz & tyler, 2007). in a pro-environmental context, this can be interpreted as being environmentally and socially responsible (kivetz & tyler, 2007). agreeing to reduce internet consumption due to high awareness of the damage even when the prices are low is assumed as the idealistic behaviour in our study. the literature suggests that awareness programs like anti-alcohol and road safety campaigns have resulted in a reduction of destructive and unhealthy behaviour even when there are no other benefits to the consumer, confirming the idealistic behaviour (nemtsov, 1998; rundmo & iversen, 2004). replicating the same characteristics, in the combination of low-price and high-awareness, our study showed the idealistic behaviour australasian journal of information systems wathuge & sedera 2022, vol 26, research article saving the environment from the internet 17 pattern in study-2. it elucidates that, after the inducement of awareness users tend to restrict their usage even more because awareness has created a substantial impact on the consumer. next, we define the combination of high-price and high-awareness in figure 4 as ‘tautological’, logically indicating that internet usage should be the lowest as of the tautological assumptions. tautology as explained by priem and butler (2001) is a statement that is always logically true to the prepositional interpretations. aligning to the tautological assumptions, the intention of internet consumption decreased in study-2 after inducing awareness. previous studies on cigarette consumption also show similar behaviour where tax-induced pricing and awareness campaigns reduced cigarette consumption (cantrell et al., 2013). therefore, we can assume that awareness has a notable impact on the intention to use the internet in the tautological consumer category. as the last combination, we term the high-price and low-awareness pair as ‘prohibitive.’ although awareness is less, external prohibitive factors such as high price discourage intention to use the internet, according to the characteristics of prohibitive behaviour in previous studies (liang et al., 2012; lin & johnson, 2015). study-1 perfectly brings out this phenomenon in which the prohibitive category showed the lowest intention to use the internet. since only price has played the prohibitive role in study-1, the impact of awareness has become unimportant there. however, according to study-2, the effect of the prohibitive factor does not create such an effect as the intention to consume the internet increases. our observations highlight the complexity of human behaviour, where the intended behaviour deviates from the accepted 'prohibitive' assumptions. overall, it is clear that 'prohibitive' (high price – low awareness) is the most effective combined strategy, before increasing environmental awareness (study-1), while 'idealistic' behaviour marks the lowest point of internet use (low price – high awareness) after awareness. 6 summary the purpose of conducting this study was to examine the effect of pricing and awareness of environmental damage, on the internet usage of individuals. as per the results of the analysis of 326 respondents, a statistically significant relationship was observed between pricing and awareness with internet use. the following main findings were extracted from the analysis. three observations were made for study-1 (before inducing awareness): (i) internet usage was highest, where both price and awareness were at their lowest. (ii) internet use was at the lowest when the price was at the highest and the awareness was at the lowest. (iii) the impact of awareness was insignificant, with price increases. however, after inducing awareness (study-2), the following three observations were made: (iv) internet usage was at the lowest, when awareness was the highest, with the lowest price, (v) internet usage remained low (though not the lowest) for high awareness, even when the price increases, (vi) price diminished its significance on usage, with increasing awareness. moreover, the results parallel with other contexts that have used pricing and awareness as strategies to control human behaviour. the study outcomes show that awareness creates a significant impact on reducing internet usage. moreover, the price creates a significant impact on internet usage. finally, the moderating effect of pricing on the relationship between awareness and internet usage also shows significant results. however, it should be noted that australasian journal of information systems wathuge & sedera 2022, vol 26, research article saving the environment from the internet 18 the impact awareness alone creates is higher than the impact both pricing alone creates, and pricing and awareness together create on internet usage. the mga of age and gender on the relationships between price affordability and awareness on internet usage reveals interesting results. the study shows that the two age groups (above and below 25 years), did not yield any significant differences in the relationships observed. that means, they did not indicate any significant moderating effect on the relationship between environmental awareness and internet usage, or price affordability and internet usage. however, the gender variable indicated differences in the relationships observed where males were more sensitive to both price and awareness than the female sample. that means, the observed relationships were moderated by the variable gender. our study contributes to the vbn theory by observing a causal relationship that entails ‘personal financial capabilities,’ ‘general environmentalist predispositions’ and ‘change in lifestyle.’ on one hand, it showed that the impact of awareness was nonsignificant with the increase in price before inducing awareness. that is, before inducing awareness, the egoistic values of a person generated beliefs like ecological world view, awareness of consequences, and ascription of responsibility. as a result, personal norms activated, and according to an individual's norms, he altered his intention to behave. on the other hand, the study showed that with the increase of awareness, the significance of price on behavioural intention reduced. that is, after inducing awareness, biospheric and altruistic values of consumers generated, and it led to the generation of beliefs, personal norms, and finally the intention to behave. the polynomial hypothesis of the co-existence of such factors makes the study findings closer to reality. figure 4 provided a practical framework to understand the continuum of internet use behaviours. the classification can be used to identify how behaviour patterns of individuals change when awareness and price change. accordingly, it can be used to decide which and when to employ the strategies to reduce internet consumption. if awareness strategies that increase motivation to reduce internet usage are not employed, then an increase in internet service prices that creates prohibitive behaviours can be introduced as the best option. however, idealistic behaviours that reduce internet usage significantly, can be observed when awareness is increased which manipulates people's values, principles, and attitudes. henceforth, after inducing awareness, internet price can be decreased. this research was tested using data only from sri lanka which would restrict to gain an overall global picture of the changes in internet usage in manipulating independent variables. therefore, we encourage further validation of results by gathering data from other countries that have different internet infrastructure and cultural values. furthermore, the study measures general internet usage of individuals which cannot be reduced at times based on the user’s purpose of using the internet (e.g.: sending a work email). therein, we identify that changes individuals do to their internet usage depend on the need of the user, and the study does not specifically investigate internet usage based on the purpose of the user. we encourage the study of internet use based on specific purposes such as internet use for leisure and work. moreover, we have considered internet use in only block-rate pricing strategy, which is the most popular pricing strategy in sri lanka. results are true to the studied pricing structure. however, we recognise that some countries provide unlimited internet data through fixed pricing. therefore, further generalisability is planned through extended studies of all pricing models. moreover, we gathered data during a global pandemic that may have augmented australasian journal of information systems wathuge & sedera 2022, vol 26, research article saving the environment from the internet 19 internet use across the globe. as such, future studies must consider the possible adverse effect that such high use may have added to the study findings. further, the consumer’s willingness to pay premium pricing and its effect on consumers’ intention to use the internet could be tested through a future study. premium pricing is the price that a customer is willing to pay extra for the environment for the same usage of the internet. premium pricing associates with uniqueness and quality as well as emotional engagement, and hence, a consumer might tend to pay more if they feel satisfied or contributed towards environmental wellbeing. this phenomenon can be tested with the inducement of environmental awareness. moreover, as another future implication, more inclusive measurements of additional vbn variables can be used to test how the intentions of consumers change. for that, variables like altruistic feelings towards nature, governmental policies, the maturity level of consumers can be tested in seeking the expected output. references abdul-wahab, s. a., & abdo, j. (2010). the effects of demographic factors on the environmental awareness of omani citizens. human and ecological risk assessment: an international journal, 16(2), 380-401. doi:https://doi.org/10.1080/10807031003670410 abrahamse, w., & steg, l. (2011). factors related to household energy use and intention to reduce it: the role of psychological and socio-demographic variables. human ecology review, 18(1), 30-40. doi:http://www.jstor.org/stable/24707684 aebli, a., volgger, m., & taplin, r. (2021). a two-dimensional approach to travel motivation in the context of the covid-19 pandemic. current issues in tourism, 1-16. doi:https://doi.org/10.1080/13683500.2021.1906631 aengenheyster, m., feng, q. y., van der ploeg, f., & dijkstra, h. a. (2018). the point of no return for climate action. earth system dynamics, 9(3), 1085-1095. doi:https://doi.org/10.5194/esd-9-1085-2018 aldossary, m., & djemame, k. (2016, september). energy consumption-based pricing model for cloud computing. paper presented at the 32nd uk performance engineering workshop, university of bradford, uk. retrieved january 20, 2020, from http://eprints.whiterose.ac.uk/104975/ aldossary, m., djemame, k., alzamil, i., kostopoulos, a., dimakis, a., & agiatzidou, e. (2019). energy-aware cost prediction and pricing of virtual machines in cloud computing environments. future generation computer systems, 93, 442-459. doi:https://doi.org/10.1016/j.future.2018.10.027 alford, b. l., & biswas, a. (2002). the effects of discount level, price consciousness and sale proneness on consumers' price perception and behavioral intention. journal of business research, 55(9), 775-783. doi:https://doi.org/10.1016/s0148-2963(00)00214-9 aminrad, z., zakaria, s., & hadi, a. s. (2011). influence of age and level of education on environmental awareness and attitude: case study on iranian students in malaysian universities. the social sciences, 6(1), 15-19. doi:10.3923/sscience.2011.15.19 andrae, a., & edler, t. (2015). on global electricity usage of communication technology: trends to 2030. challenges, 6(1), 117-157. doi:10.3390/challe6010117 arcury, t. a., & christianson, e. h. (1993). rural-urban differences in environmental knowledge and actions. the journal of environmental education, 25(1), 19-25. doi:10.1080/00958964.1993.9941940 australasian journal of information systems wathuge & sedera 2022, vol 26, research article saving the environment from the internet 20 arlt, d., hoppe, i., & wolling, j. (2011). climate change and media usage: effects on problem awareness and behavioural intentions. international communication gazette, 73(1-2), 4563. doi:https://doi.org/10.1177/1748048510386741 atwater, l., waldman, d., ostroff, c., robie, c., & johnson, k. m. (2005). self–other agreement: comparing its relationship with performance in the us and europe. international journal of selection and assessment, 13(1), 25-40. doi:https://doi.org/10.1111/j.0965075x.2005.00297.x ball, k., mcnaughton, s. a., mhurchu, c. n., andrianopoulos, n., inglis, v., mcneilly, b., le, h. n., leslie, d., pollard, c., & crawford, d. (2011). supermarket healthy eating for life (shelf): protocol of a randomised controlled trial promoting healthy food and beverage consumption through price reduction and skill-building strategies. biomed central public health, 11(1), 715. doi:https://doi.org/10.1186/1471-2458-11-715 bateni, m. h., hajiaghayi, m. t., jafarpour, s., & pei, d. (2011, april). towards an efficient algorithmic framework for pricing cellular data service. paper presented at the ieee international conference on computer communications, shanghai, china. retrieved february 20, 2020, from https://ieeexplore.ieee.org/document/5935231 bell, p. a., greene, t. c., fisher, j. d., & baum, a. (1996). environmental psychology. new york: harcourt brace. benitez-amado, j., henseler, j., & castillo, a. (2017, july). development and update of guidelines to perform and report partial least squares path modeling in information systems research. paper presented at the 21st pacific asia conference on information systems, langkawi, malaysia. retrieved march 02, 2020, from https://core.ac.uk/download/pdf/301372855.pdf benitez-amado, j., llorens-montes, f. j., & fernandez-perez, v. (2013, august). the relationship between it infrastructure leveraging, talent management and operational sustainability, and their effects on the business value of the operations strategy. paper presented at the americas conference on information systems, chicago, illinois. retrieved from march 02, 2020 from https://core.ac.uk/download/pdf/301359353.pdf box, g. e., & draper, n. r. (1987). empirical model-building and response surfaces. new york: john wiley & sons. brewer, n. t., hall, m. g., noar, s. m., parada, h., stein-seroussi, a., bach, l. e., hanley, s., & ribisl, k. m. (2016). effect of pictorial cigarette pack warnings on changes in smoking behavior: a randomized clinical trial. the journal of the american medical association internal medicine, 176(7), 905-912. 10.1001/jamainternmed.2016.2621 briscar, j. r. (2017). data transmission and energy efficient internet data centers. american university law review, 67(1), 233-268 brounen, d., kok, n., & quigley, j. m. (2013). energy literacy, awareness, and conservation behavior of residential households. energy economics, 38, 42-50. doi:https://doi.org/10.1016/j.eneco.2013.02.008 bulgurcu, b., cavusoglu, h., & benbasat, i. (2010). information security policy compliance: an empirical study of rationality-based beliefs and information security awareness. management information systems quarterly, 34(3), 523-548. doi:10.2307/25750690 cantrell, j., kreslake, j. m., ganz, o., pearson, j. l., vallone, d., anesetti-rothermel, a., xiao, h., & kirchner, t. r. (2013). marketing little cigars and cigarillos: advertising, price, and associations with neighborhood demographics. american journal of public health, 103(10), 1902-1909. doi:10.2105/ajph.2013.301362 australasian journal of information systems wathuge & sedera 2022, vol 26, research article saving the environment from the internet 21 chaloupka, f. j., & grossman, m. (1996). price, tobacco control policies and youth smoking. national bureau of economic research (working paper no. 5740). doi:10.3386/w5740 chen, h., hui, h., su, z., fang, d., & hui, y. (2017). real-time pricing strategy based on the stability of smart grid for green internet of things. mobile information systems, 2017, 111. doi:https://doi.org/10.1155/2017/5039702 chen, t., sze, n., saxena, s., pinjari, a. r., bhat, c. r., & bai, l. (2020). evaluation of penalty and enforcement strategies to combat speeding offences among professional drivers: a hong kong stated preference experiment. accident analysis & prevention, 135, 105-366. doi:https://doi.org/10.1016/j.aap.2019.105366 chi, x., hong, x., & chen, x. (2020). profiles and sociodemographic correlates of internet addiction in early adolescents in southern china. addictive behaviors, 106, 106-385. doi:https://doi.org/10.1016/j.addbeh.2020.106385 clement, j. (2019). global digital population as of october 2019. retrieved march 05, 2020, from https://www.statista.com/statistics/617136/digital-population-worldwide/ clement, j. (2020). age distribution of internet users worldwide 2019. retrieved march 05,2020, from https://www.statista.com/statistics/272365/age-distribution-of-internetusers-worldwide/ cooper, v. a., & molla, a. (2014). absorptive capacity and contextual factors that influence green it assimilation. australasian journal of information systems, 18(3), 271-288. doi:https://doi.org/10.3127/ajis.v18i3.1099 cornell, j. a., & khuri, a. i. (1987). response surfaces: designs and analyses. new york: marcel dekker, inc. curry, s. j., wagner, e. h., & grothaus, l. c. (1991). evaluation of intrinsic and extrinsic motivation interventions with a self-help smoking cessation program. journal of consulting and clinical psychology, 59(2), 318. doi:https://doi.org/10.1037/0022006x.59.2.318 de vries, h., mudde, a., leijs, i., charlton, a., vartiainen, e., buijs, g., clemente, m. p., storm, h., navarro, a. g., & nebot, m. (2003). the european smoking prevention framework approach (efsa): an example of integral prevention. health education research, 18(5), 611-626. doi:https://doi.org/10.1093/her/cyg031 delhaye, e. (2007). the enforcement of speeding: should fines be higher for repeated offences? transportation planning and technology, 30(4), 355-375. doi:https://doi.org/10.1080/ 03081060701461758 dijkstra, t. k. (2010). latent variables and indices: herman wold’s basic design and partial least squares. in handbook of partial least squares (pp. 23-46). berlin, heidelberg: springer. dijkstra, t. k., & henseler, j. (2015). consistent partial least squares path modeling. management information systems quarterly, 39(2), 297-316. doi:https://www.jstor.org/ stable/26628355 ding, a. (2003). youth are more sensitive to price changes in cigarettes than adults. the yale journal of biology and medicine, 76(3), 115-124. ebbs, d., dahlhaus, p., barton, a., & kandra, h. (2018). an unexpected decrease in urban water demand: making discoveries possible by taking a long-term view. water policy, 20(3), 617-630. doi:https://doi.org/10.2166/wp.2018.096 edwards, j. r., & parry, m. e. (1993). on the use of polynomial regression equations as an alternative to difference scores in organizational research. academy of management journal, 36(6), 1577-1613. doi:https://doi.org/10.5465/256822 australasian journal of information systems wathuge & sedera 2022, vol 26, research article saving the environment from the internet 22 elliot, s., & binney, d. (2008). environmentally sustainable ict: developing corporate capabilities and an industry-relevant is research agenda. paper presented at the pacific asia conference in information systems, suzhou, china. retrieved march 01, 2020, from https://aisel.aisnet.org/cgi/viewcontent.cgi?article=1239&context=pacis2008 eriksson, l., garvill, j., & nordlund, a. m. (2006). acceptability of travel demand management measures: the importance of problem awareness, personal norm, freedom, and fairness. journal of environmental psychology, 26(1), 15-26. doi:https://doi.org/10.1016/j.jenvp.2006.05.003 fan, q. (2005). regulatory factors influencing internet access in australia and china: a comparative analysis. telecommunications policy, 29(2-3), 191-203. doi:https://doi.org/10.1016/j.telpol.2004.11.007 farrelly, m. c., duke, j. c., nonnemaker, j., macmonegle, a. j., alexander, t. n., zhao, x., delahanty, j. c., rao, p., & allen, j. a. (2017). association between the real cost media campaign and smoking initiation among youths—united states, 2014–2016. 66(2), 47-50. retrieved from https://www.ncbi.nlm.nih.gov/pmc/articles/pmc5657653/ ferrer, r. a., orehek, e., scheier, m. f., & o'connell, m. e. (2018). cigarette tax rates, behavioral disengagement, and quit ratios among daily smokers. journal of economic psychology, 66, 13-21. doi:https://doi.org/10.1016/j.joep.2018.03.005 france24 (producer). (2017). the hidden pollution of the internet. retrieved from https://www.youtube.com/watch?v=gx8sorz_-fg&t=18s fricker, r. d. (2008). sampling methods for web and e-mail surveys. in the sage handbook of online research methods (pp. 195-216). london: sage publications ltd. frieden, t. r., mostashari, f., kerker, b. d., miller, n., hajat, a., & frankel, m. (2005). adult tobacco use levels after intensive tobacco control measures: new york city, 2002–2003. american journal of public health, 95(6), 1016-1023. doi:10.2105/ajph.2004.058164 ghansah, f. a., owusu-manu, d.-g., ayarkwa, j., edwards, d. j., & hosseini, m. r. (2021). assessing the level of awareness of smart building technologies (sbts) in the developing countries. journal of engineering, design and technology. doi:https://doi.org/10.1108/ jedt-11-2020-0465 giesbrecht, n., & greenfield, t. k. (2003). preventing alcohol-related problems in the us through policy: media campaigns, regulatory approaches and environmental interventions. journal of primary prevention, 24(1), 63-104. doi:https://doi.org/10.1023/a:1025027701426 goiri, í., katsak, w., le, k., nguyen, t. d., & bianchini, r. (2013). parasol and greenswitch: managing datacenters powered by renewable energy. association for computing machinery special interest group on programming languages notices, 48(4), 51-64. doi:https://doi.org/10.1145/2499368.2451123 google. (2019). google environmental report. retrieved january 08, 2020, from https://services.google.com/fh/files/misc/google_2019-environmental-report.pdf grishchenko, n. (2020). the gap not only closes: resistance and reverse shifts in the digital divide in russia. telecommunications policy, 44(8), 102004. doi:https://doi.org/ 10.1016/j.telpol.2020.102004 hair, j. f., ringle, c. m., gudergan, s. p., fischer, a., nitzl, c., & menictas, c. (2018). partial least squares structural equation modeling-based discrete choice modeling: an illustration in modeling retailer choice. business research. doi:10.1007/s40685-018-0072-4 hallegatte, s. (2009). strategies to adapt to an uncertain climate change. global environmental change, 19(2), 240-247. doi:10.1016/j.gloenvcha.2008.12.003 australasian journal of information systems wathuge & sedera 2022, vol 26, research article saving the environment from the internet 23 haltinner, k., & sarathchandra, d. (2021). the nature and nuance of climate change skepticism in the united states. rural sociology, 1-30. doi:https://doi.org/10.1111/ ruso.12371 han, t., coles, h., price, p. n., gadgil, a., & tschudi, w. (2019). corrosion coupons may not be useful for predicting data center equipment failure rates. center of expertise for energy efficiency in datacenters, united states department of energy. retrieved march 22, 2020, from https://libguides.scu.edu.au/apa6/government/statistical/technicalpublications haque, m. m., ahmend, a., & rahman, a. (2013). impacts of water price and restrictions on water demand: a case study for australia. in water conservation: practices, challenges and future implications (pp. 127-145). new york: nova science publishers. henriksen, l. (2012). comprehensive tobacco marketing restrictions: promotion, packaging, price and place. tobacco control, 21(2), 147-153. doi:10.1136/tobaccocontrol-2011-050416 henseler, j. (2007, september). a new and simple approach to multi-group analysis in partial least squares path modeling. paper presented at the the 5th international symposium on pls and related methods, oslo, norway. retrieved april 10, 2020, from https://repository.ubn.ru.nl/handle/2066/160875 henseler, j., & chin, w. w. (2010). a comparison of approaches for the analysis of interaction effects between latent variables using partial least squares path modeling. structural equation modeling, 17(1), 82-109. doi:https://doi.org/10.1080/10705510903439003 henseler, j., dijkstra, t. k., sarstedt, m., ringle, c. m., diamantopoulos, a., straub, d. w., ketchen jr, d. j., hair, j. f., hult, g. t. m., & calantone, r. j. (2014). common beliefs and reality about pls: comments on rönkkö and evermann (2013). organizational research methods, 17(2), 182-209. doi:https://doi.org/10.1177/1094428114526928 henseler, j., ringle, c. m., & sarstedt, m. (2016). testing measurement invariance of composites using partial least squares. international marketing review, 33(3), 405-431. doi:https://doi.org/10.1108/imr-09-2014-0304 henseler, j., ringle, c. m., & sinkovics, r. r. (2009). the use of partial least squares path modeling in international marketing. in r. r. sinkovics & p. n. ghauri (eds.), new challenges to international marketing (vol. 20, pp. 277-319). bingly: emerald group publishing limited. henseler, j., & sarstedt, m. (2013). goodness-of-fit indices for partial least squares path modeling. computational statistics, 28(2), 565-580. doi:https://doi.org/10.1007/s00180-0120317-1 hershey, j. c., niederdeppe, j., evans, w. d., nonnemaker, j., blahut, s., holden, d., messeri, p., & haviland, m. l. (2005). the theory of" truth": how counterindustry campaigns affect smoking behavior among teens. health psychology, 24(1), 22-31. doi:10.1037/02786133.24.1.22 hill, j., widdicks, k., & hazas, m. (2020, june). mapping the scope of software interventions for moderate internet use on mobile devices. paper presented at the 7th international conference on ict for sustainability, bristol, united kingdom. retrieved august 20, 2020, from https://dl.acm.org/doi/pdf/10.1145/3401335.3401361 hobman, e. v., & frederiks, e. r. (2014). barriers to green electricity subscription in australia:“love the environment, love renewable energy… but why should i pay more?”. energy research & social science, 3(1), 78-88. doi:https://doi.org/10.1016/j.erss.2014.07.009 australasian journal of information systems wathuge & sedera 2022, vol 26, research article saving the environment from the internet 24 hollingworth, w., ebel, b. e., mccarty, c. a., garrison, m. m., christakis, d. a., & rivara, f. p. (2006). prevention of deaths from harmful drinking in the united states: the potential effects of tax increases and advertising bans on young drinkers. journal of studies on alcohol, 67(2), 300-308. doi:https://doi.org/10.15288/jsa.2006.67.300 hu, l. t., & bentler, p. m. (1999). cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. structural equation modeling: a multidisciplinary journal, 6(1), 1-55. doi:https://doi.org/10.1080/10705519909540118 ipcc. (2021). summary for policymakers. in v. masson-delmotte, p. zhai, a. piran, s. l. connors, c. péan, s. berger, n. caud, y. chen, l. goldfarb, m. i. gomis, m. huang, k. leitzell, e. lonnoy, j. b. r. matthews, t. k. maycock, t. waterfield, o. yelekçi, r. yu, & b. zhou (eds.), climate change 2021: the physical science basis. contribution of working group i to the sixth assessment report of the intergovernmental panel on climate change: cambridge university press. in press. retrieved september 05, 2020, from https://www.ipcc.ch/report/ar6/wg1/downloads/report/ipcc_ar6_wgi_full_report_ smaller.pdf islam, m., saidur, r., rahim, n., & solangi, k. (2010). usage of solar energy and its status in malaysia. engineering e-transaction, 5(1), 6-10. itu. (2019). the digital gender gap is growing fast in developing countries. retrieved april 05, 2020, from https://itu.foleon.com/itu/measuring-digital-development/gender-gap/ itu. (2020). affordability. retrieved november 20, 2020, from https://www.itu.int/en/ mediacentre/backgrounders/pages/affordability.aspx izquierdo, f. a., ramírez, b. a., mcwilliams, j. m., & ayuso, j. p. (2011). the endurance of the effects of the penalty point system in spain three years after. main influencing factors. accident analysis & prevention, 43(3), 911-922. doi:https://doi.org/10.1016/ j.aap.2010.11.014 jakovcevic, a., & steg, l. (2013). sustainable transportation in argentina: values, beliefs, norms and car use reduction. transportation research part f: traffic psychology and behaviour, 20, 70-79. doi:https://doi.org/10.1016/j.trf.2013.05.005 jenkin, t. a., mcshane, l., & webster, j. (2011). green information technologies and systems: employees’ perceptions of organizational practices. business & society, 50(2), 266-314. doi:https://doi.org/10.1177/0007650311398640 jenkin, t. a., webster, j., & mcshane, l. (2011). an agenda for ‘green’information technology and systems research. information and organization, 21(1), 17-40. doi:https://doi.org/10.1016/j.infoandorg.2010.09.003 jylhä, k. m., tam, k. p., & milfont, t. l. (2020). acceptance of group-based dominance and climate change denial: a cross-cultural study in hong kong, new zealand, and sweden. asian journal of social psychology. doi:10.1111/ajsp.12444 kemp, s. (2019). digital around the world in 2019. retrieved feburary 20, 2020, from https://www.juancmejia.com/wp-content/uploads/2019/03/digital-2019-wearesocial-yhootsuite.pdf kenney, m., & zysman, j. (2020). covid-19 and the increasing centrality and power of platforms in china, the us, and beyond. management and organization review, 16(4), 747752. doi: https://doi.org/10.1017/mor.2020.48 khalaj, a. h., scherer, t., siriwardana, j., & halgamuge, s. (2014, december). increasing the thermal efficiency of an operational data center using cold aisle containment. paper presented at the 7th international conference on information and automation for sustainability, colombo, sri lanka. doi:10.1109/iciafs.2014.7069633 australasian journal of information systems wathuge & sedera 2022, vol 26, research article saving the environment from the internet 25 kivetz, y., & tyler, t. r. (2007). tomorrow i’ll be me: the effect of time perspective on the activation of idealistic versus pragmatic selves. organizational behavior and human decision processes, 102(2), 193-211. doi:https://doi.org/10.1016/j.obhdp.2006.07.002 kollmuss, a., & agyeman, j. (2002). mind the gap: why do people act environmentally and what are the barriers to pro-environmental behavior? environmental education research, 8(3), 239-260. doi:https://doi.org/10.1080/13504620220145401 kothari, c. r. (2004). research methodology: methods and techniques (2 ed.). new delhi: new age international. kuruzovich, j., viswanathan, s., agarwal, r., gosain, s., & weitzman, s. (2008). marketspace or marketplace? online information search and channel outcomes in auto retailing. information systems research, 19(2), 182-201. doi:https://doi.org/10.1287/isre.1070.0146 kuss, d., griffiths, m., karila, l., & billieux, j. (2014). internet addiction: a systematic review of epidemiological research for the last decade. current pharmaceutical design, 20(25), 4026-4052. lawshe, c. h. (1975). a quantitative approach to content validity. personnel psychology, 28(4), 563-575. doi:10.1111/j.1744-6570.1975.tb01393.x lee, y. s., han, d. h., kim, s. m., & renshaw, p. f. (2013). substance abuse precedes internet addiction. addictive behaviors, 38(4), 2022-2025. doi:https://doi.org/10.1016/ j.addbeh.2012.12.024 lewis, i., watson, b., & white, k. m. (2008). an examination of message-relevant affect in road safety messages: should road safety advertisements aim to make us feel good or bad? transportation research part f: traffic psychology and behaviour, 11(6), 403-417. doi:https://doi.org/10.1016/j.trf.2008.03.003 li, c., tang, j., tang, h., & luo, y. (2019). collaborative cache allocation and task scheduling for data-intensive applications in edge computing environment. future generation computer systems, 95, 249-264. doi:https://doi.org/10.1016/j.future.2019.01.007 liang, j., farh, c. i., & farh, j.-l. (2012). psychological antecedents of promotive and prohibitive voice: a two-wave examination. academy of management journal, 55(1), 71-92. doi:https://doi.org/10.5465/amj.2010.0176 lin, s. h., & johnson, r. e. (2015). a suggestion to improve a day keeps your depletion away: examining promotive and prohibitive voice behaviors within a regulatory focus and ego depletion framework. journal of applied psychology, 100(5), 1381-1397. doi:10.1037/apl0000018 liobikienė, g., & poškus, m. s. (2019). the importance of environmental knowledge for private and public sphere pro-environmental behavior: modifying the value-beliefnorm theory. sustainability, 11(12), 3324. doi:https://doi.org/10.3390/su11123324 liu, s. (2019). global number of data centers 2015-2021. retrieved january 28, 2020, from https://www.statista.com/statistics/500458/worldwide-datacenter-and-it-sites/ lópez-mosquera, n., & sánchez, m. (2012). theory of planned behavior and the value-beliefnorm theory explaining willingness to pay for a suburban park. journal of environmental management, 113, 251-262. doi:https://doi.org/10.1016/j.jenvman.2012.08.029 luchs, m. g., & mooradian, t. a. (2011). sex, personality, and sustainable consumer behaviour: elucidating the gender effect. journal of consumer policy, 35(1), 127-144. doi:10.1007/s10603-011-9179-0 luís, s., vauclair, c.-m., & lima, m. l. (2018). raising awareness of climate change causes? cross-national evidence for the normalization of societal risk perception of climate change. environmental science & policy, 80, 74-81. doi:https://doi.org/10.1016/ australasian journal of information systems wathuge & sedera 2022, vol 26, research article saving the environment from the internet 26 j.envsci.2017.11.015 lv, q., & rouskas, g. n. (2010). an economic model for pricing tiered network services. annals of telecommunications-annales des télécommunications, 65(3-4), 147-161. doi:https://doi.org/10.1007/s12243-009-0149-3 maibach, e. (1993). social marketing for the environment: using information campaigns to promote environmental awareness and behavior change. health promotion international, 8(3), 209-224. doi:https://doi.org/10.1093/heapro/8.3.209 mathwick, c., & rigdon, e. (2004). play, flow, and the online search experience. journal of consumer research, 31(2), 324-332. doi:https://doi.org/10.1086/422111 mcevoy, j. (1972). the american concern with the environment. in j. w. r. burch, j. n. h. cheek, & l. taylor (eds.), social behavior, natural resources and the environment (pp. 214236). new york: harper and row. mcginty, e. e., goldman, h. h., pescosolido, b., & barry, c. l. (2015). portraying mental illness and drug addiction as treatable health conditions: effects of a randomized experiment on stigma and discrimination. social science & medicine, 126, 73-85. doi:https://doi.org/10.1016/j.socscimed.2014.12.010 mcginty, e. e., webster, d. w., & barry, c. l. (2013). effects of news media messages about mass shootings on attitudes toward persons with serious mental illness and public support for gun control policies. american journal of psychiatry, 170(5), 494-501. doi:https://doi.org/10.1176/appi.ajp.2013.13010014 mckenna, f. p. (2007). do attitudes and intentions change across a speed awareness workshop? paper presented at the behavioural research in road safety seminar, london, united kingdom. retrieved march 20, 2020, from https://citeseerx.ist.psu.edu/viewdoc/ download?doi=10.1.1.514.1615&rep=rep1&type=pdf milne, j. e. (2014). storms ahead: climate change adaptation calls for resilient funding. vermont law review, 39(4), 819-866. molla, a. (2013). identifying it sustainability performance drivers: instrument development and validation. information systems frontiers, 15(5), 705-723. doi:10.1007/s10796-013-9415z molla, a., & cooper, v. (2010). green it readiness: a framework and preliminary proof of concept. australasian journal of information systems, 16(2), 5-23. doi:https://doi.org /10.3127/ajis.v16i2.545 morgil, i̇., seçil, a., seçken, n., yavuz, s., & oskay, ö. ö. (2004). the influence of computerassisted education on environmental knowledge and environmental awareness. chemistry education research and practice, 5(2), 99-110. doi:10.1039/b3rp90032k muratore, i. (2016). teens as impulsive buyers: what is the role of price? international journal of retail & distribution management, 44(11), 1166-1180. doi:https://doi.org/10.1108/ ijrdm-08-2015-0120 mutz, d. c. (2011). population-based survey experiments. princeton: princeton university press. nagata, t., setoguchi, s., hemenway, d., & perry, m. (2008). effectiveness of a law to reduce alcohol-impaired driving in japan. injury prevention, 14(1), 19-23. doi:http://dx.doi.org/10.1136/ip.2007.015719 nathanail, e., & adamos, g. (2013). road safety communication campaigns: research designs and behavioral modeling. transportation research part f: traffic psychology and behaviour, 18, 107-122. doi:https://doi.org/10.1016/j.trf.2012.12.003 australasian journal of information systems wathuge & sedera 2022, vol 26, research article saving the environment from the internet 27 nemtsov, a. v. (1998). alcohol-related harm and alcohol consumption in moscow before, during and after a major anti-alcohol campaign. addiction, 93(10), 1501-1510. doi: https://doi.org/10.1046/j.1360-0443.1998.931015016.x nevo, a., turner, j. l., & williams, j. w. (2016). usage-based pricing and demand for residential broadband. econometrica, 84(2), 411-443. doi: https://doi.org/10.3982/ecta11927 oguchi, t. (2016). achieving safe road traffic—the experience in japan. international association of traffic and safety sciences, 39(2), 110-116. doi:https://doi.org/10.1016/ j.iatssr.2016.01.003 olexsak, s. j., & meier, a. (2014). the electricity impacts of earth hour: an international comparative analysis of energy-saving behavior. energy research & social science, 2(1), 159-182. doi:https://doi.org/10.1016/j.erss.2014.04.014 pedulla, d. s., & thébaud, s. (2015). can we finish the revolution? gender, work-family ideals, and institutional constraint. american sociological review, 80(1), 116-139. doi:https://doi.org/10.1177/0003122414564008 petrunoff, n., rissel, c., wen, l. m., & martin, j. (2015). carrots and sticks vs carrots: comparing approaches to workplace travel plans using disincentives for driving and incentives for active travel. journal of transport & health, 2(4), 563-567. doi:https://doi.org/10.1016/j.jth.2015.06.007 pooley, j. a., & o’connor, m. (2000). environmental education and attitudes: emotions and beliefs are what is needed. environment and behavior, 32(5), 711-723. doi:https:// doi.org/10.1177/0013916500325007 pope, d. a., poe, l., stein, j. s., kaplan, b. a., dehart, w. b., mellis, a. m., heckman, b. w., epstein, l. h., chaloupka, f. j., & bickel, w. k. (2020). the experimental tobacco marketplace: demand and substitutability as a function of cigarette taxes and e-liquid subsidies. nicotine and tobacco research, 22(5), 782-790. doi:https://doi.org/10.1093/ntr/ ntz116 poulter, d. r., & mckenna, f. p. (2010). evaluating the effectiveness of a road safety education intervention for pre-drivers: an application of the theory of planned behaviour. british journal of educational psychology, 80(2), 163-181. doi: https://doi.org/10.1348/ 014466509x468421 priem, r. l., & butler, j. e. (2001). tautology in the resource-based view and the implications of externally determined resource value: further comments. academy of management review, 26(1), 57-66. doi:https://doi.org/10.5465/amr.2001.4011946 ranjit, y. s., snyder, l. b., hamilton, m. a., & rimal, r. n. (2017). self-determination theory and risk behavior in a collectivistic society: preventing reckless driving in urban nepal. journal of health communication, 22(8), 672-681. doi:https://doi.org/10.1080/10810730. 2017.1341569 rivera, j., & de leon, p. (2004). is greener whiter? voluntary environmental performance of western ski areas. policy studies journal, 32(3), 417-437. doi:https://doi.org/10.1111/j.15410072.2004.00073.x rundmo, t., & iversen, h. (2004). risk perception and driving behaviour among adolescents in two norwegian counties before and after a traffic safety campaign. safety science, 42(1), 1-21. doi:https://doi.org/10.1016/s0925-7535(02)00047-4 sahin, e. (2013). predictors of turkish elementary teacher candidates' energy conservation behaviors: an approach on value-belief-norm theory. international journal of environmental and science education, 8(2), 269-283. doi:10.12973/ijese.2013.204a australasian journal of information systems wathuge & sedera 2022, vol 26, research article saving the environment from the internet 28 salahuddin, m., & alam, k. (2015). internet usage, electricity consumption and economic growth in australia: a time series evidence. telematics and informatics, 32(4), 862-878. doi:10.1016/j.tele.2015.04.011 schaap, m. m., kunst, a. e., leinsalu, m., regidor, e., ekholm, o., dzurova, d., helmert, u., klumbiene, j., santana, p., & mackenbach, j. p. (2008). effect of nationwide tobacco control policies on smoking cessation in high and low educated groups in 18 european countries. tobacco control, 17(4), 248-255. doi:http://dx.doi.org/10.1136/tc.2007.024265 schlagel, c., & sarstedt, m. (2016). assessing the measurement invariance of the fourdimensional cultural intelligence scale across countries: a composite model approach. european management journal, 34(6), 633–649. schulz, g. (2016). the green and virtual data center. boca raton, florida, united states: crc press. sedera, d., gable, g., & chan, t. (2003, july). survey design: insights from a public sector erp success story. paper presented at the 7th pacific asia conference on information systems, university of south australia. retrieved march 05, 2020, from https://eprints.qut.edu.au/24794/1/24794.pdf sedera, d., lokuge, s., tushi, b., & tan, f. (2017). multi-disciplinary green it archival analysis: a pathway for future studies. communications of the association for information systems, 41(1), 674-773. doi:https://doi.org/10.17705/1cais.04128 seidler, a.-r., henkel, c., fiedler, m., kranz, j., ixmeier, a., & strunk, k. s. (2020). promoting eco-sustainable behavior with gamification: an experimental study on the alignment of competing goals. paper presented at the forty-first international conference on information systems, india. retrieved january 10, 2021, from https://aisel.aisnet.org/icis2020/societal_impact/societal_impact/3 sen, s., joe-wong, c., ha, s., & chiang, m. (2015). smart data pricing: using economics to manage network congestion. communications of the acm, 58(12), 86-93. doi:10.1145/2756543 shanock, l. r., baran, b. e., gentry, w. a., pattison, s. c., & heggestad, e. d. (2010). polynomial regression with response surface analysis: a powerful approach for examining moderation and overcoming limitations of difference scores. journal of business and psychology, 25(4), 543-554. doi:https://doi.org/10.1007/s10869-010-9183-4 sheehan, m., schonfeld, c., ballard, r., schofield, f., najman, j., & siskind, v. (1996). a three year outcome evaluation of a theory based drink driving education program. journal of drug education, 26(3), 295-312. doi:https://doi.org/10.2190/rprv-7gp1-xh7f-3lhn shuja, j., ahmad, r. w., gani, a., ahmed, a. i. a., siddiqa, a., nisar, k., khan, s. u., & zomaya, a. y. (2017). greening emerging it technologies: techniques and practices. journal of internet services and applications, 8(1), 1-11. doi:https://doi.org/10.1186/s13174017-0060-5 sinha, i., & batra, r. (1999). the effect of consumer price consciousness on private label purchase. international journal of research in marketing, 16(3), 237-251. doi:https://doi.org /10.1016/s0167-8116(99)00013-0 spector, p. e., & brannick, m. t. (2011). methodological urban legends: the misuse of statistical control variables. organizational research methods, 14(2), 287-305. doi:https://doi.org/10.1177/1094428110369842 statista. (2019). internet of things (iot) connected devices installed base worldwide from 2015 to 2025. retrieved february 15, 2020, from https://www.statista.com/statistics/ 471264/iot-number-of-connected-devices-worldwide/ australasian journal of information systems wathuge & sedera 2022, vol 26, research article saving the environment from the internet 29 steptoe, a., wardle, j., cui, w., baban, a., glass, k., pelzer, k., tsuda, a., & vinck, j. (2002). an international comparison of tobacco smoking, beliefs and risk awareness in university students from 23 countries. addiction, 97(12), 1561-1571. doi: https://doi.org/10.1046/j.1360-0443.2002.00269.x stern, p. c. (1999). information, incentives, and proenvironmental consumer behavior. journal of consumer policy, 22(4), 461-478. doi:https://doi.org/10.1023/a:1006211709570 stern, p. c. (2000). new environmental theories: toward a coherent theory of environmentally significant behavior. journal of social issues, 56(3), 407-424. doi:10.1111/0022-4537.00175 stern, p. c., dietz, t., abel, t., guagnano, g. a., & kalof, l. (1999). a value-belief-norm theory of support for social movements: the case of environmentalism. human ecology review, 6(2), 81-97. syu, f.-k., huang, m.-y., & huang, j.-j. (2010). a successful intervention to reduce cigarillo use among baltimore youth. fooyin journal of health sciences, 2(3-4), 72-84. doi:https://doi.org/10.1016/s1877-8607(11)60002-7 thilina, d., guruge, m., & nanayakkara, n. (2020). a descriptive analysis on digital behaviour of young adults in sri lanka. international journal of business and management invention, 9(6), 58-67. doi:10.35629/8028-0906055867 thombs, d. l., & osborn, c. j. (2019). introduction to addictive behaviors (3 ed.). new york: the guilford press. un. (2019). what's the cost of your search? retrieved january 06, 2020, from https://www.unenvironment.org/news-and-stories/story/whats-cost-your-search venkatesh, v., brown, s. a., maruping, l. m., & bala, h. (2008). predicting different conceptualizations of system use: the competing roles of behavioral intention, facilitating conditions, and behavioral expectation. management information systems quarterly, 32(3), 483-502. doi:https://doi.org/10.2307/25148853 venkatesh, v., & goyal, s. (2010). expectation disconfirmation and technology adoption: polynomial modeling and response surface analysis. management information systems quarterly, 34(2), 281-303. doi:https://doi.org/10.2307/20721428 verdonk-kleinjan, w. m., candel, m. j., knibbe, r. a., willemsen, m. c., & de vries, h. (2011). effects of a workplace-smoking ban in combination with tax increases on smoking in the dutch population. nicotine & tobacco research, 13(6), 412-418. doi:https://doi.org/10.1093/ntr/ntr014 wakefield, k. l., & inman, j. j. (2003). situational price sensitivity: the role of consumption occasion, social context and income. journal of retailing, 79(4), 199-212. doi:https://doi.org/10.1016/j.jretai.2003.09.004 wathuge, a., & sedera, d. (2021). an awareness-based model to minimize the environmental damage of the internet usage: a longitudinal study. paper presented at the pacific asia conference on information systems, dubai. retrieved august 20, 2021, from https://aisel.aisnet.org/pacis2021/130/ watts, r. l., & zimmerman, j. l. (1978). towards a positive theory of the determination of accounting standards. accounting review, 53(1), 112-134. whitmarsh, l., & o'neill, s. (2010). green identity, green living? the role of proenvironmental self-identity in determining consistency across diverse proenvironmental behaviours. journal of environmental psychology, 30(3), 305-314. doi:https://doi.org/10.1016/j.jenvp.2010.01.003 widdicks, k. (2020). understanding and mitigating the impact of internet demand in everyday life. lancaster university. https://doi.org/10.17635/lancaster/thesis/951 https://www.research.lancs.ac.uk/portal/en/people/kelly-widdicks(e8ed1c40-2af5-468b-b94a-638194ee6dfb).html https://www.research.lancs.ac.uk/portal/en/publications/understanding-and-mitigating-the-impact-of-internet-demand-in-everyday-life(8a3c8dc3-c1dc-4ca6-970d-29b9ae94af94).html https://doi.org/10.17635/lancaster/thesis/951 australasian journal of information systems wathuge & sedera 2022, vol 26, research article saving the environment from the internet 30 wold, h. (1989). theoretical empiricism. new york, usa: paragon house. yang, s., zhang, d., & li, d. (2019). a calculation model for co2 emission reduction of energy internet: a case study of yanqing. sustainability, 11(9), 2502. doi:https://doi.org/10.3390/su11092502 yucedag, c., kaya, l. g., & cetin, m. (2018). identifying and assessing environmental awareness of hotel and restaurant employees’ attitudes in the amasra district of bartin. environmental monitoring and assessment, 190(2), 60. doi:https://doi.org/10.1007/s10661017-6456-7 zeithaml, v. a. (1988). consumer perceptions of price, quality, and value: a means-end model and synthesis of evidence. journal of marketing, 52(3), 2-22. doi:https://doi.org/10.1177/ 002224298805200302 zettelmeyer, f., morton, f. s., & silva-risso, j. (2001). cowboys or cowards: why are internet car prices lower? national bureau of economic research. working paper no. 8667. doi:https://www.nber.org/papers/w8667 zhang, y., wang, y., & wang, x. (2012, september). electricity bill capping for cloud-scale data centers that impact the power markets. paper presented at the 41st international conference on parallel processing, pittsburgh. retrieved march 06, 2020, from https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=6337605&casa_token=psxleixp tjaaaaaa:pl6_gwzfioa5zw9axooai1rxkbxjvlg4xwt3ddoeqjlxrjecii7nfd4ahn irjvbnqtwwedy9lg ziadat, a. h. (2010). major factors contributing to environmental awareness among people in a third world country/jordan. environment, development and sustainability, 12(1), 135-145. doi:https://doi.org/10.1007/s10668-009-9185-4 zimmermann, s., meijer, i., tiwari, m. k., paredes, s., michel, b., & poulikakos, d. (2012). aquasar: a hot water cooled data center with direct energy reuse. energy, 43(1), 237-245. doi:https://doi.org/10.1016/j.energy.2012.04.037 appendix – questionnaire items environmental awareness adapted from: (bulgurcu et al., 2010), (morgil et al., 2004): my internet usage can be damaging to the environment. i am aware of the cost that data centers have on the environment. i would have liked my internet providers and services to use renewable energy sources. i save internet data by downloading a song, rather than listening to it online frequently. compared to my friends, i spend more time on the web. price affordability adapted from: (zeithaml, 1988), (wakefield & inman, 2003), (sinha & batra, 1999): i am conscious about the price of my internet. i find the internet services that i use to be affordable. i am not willing to exert extra effort to find the cheapest internet package. the money saved by finding a cheaper internet package is not worth the time and effort. increase of telecommunication tax will never affect my internet usage pattern. australasian journal of information systems wathuge & sedera 2022, vol 26, research article saving the environment from the internet 31 internet usage adapted from: (mathwick & rigdon, 2004), (venkatesh et al., 2008): i spend several hours a day on the web. compared with most of my friends, i think i spend a lot of time on the web. outside of the time i spend with e-mail, i consider myself to be a "heavy internet user." i frequently have multiple devices connected to my internet. i try limiting my time using the internet, but found it challenging. copyright: © 2022 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v26i0.3239 http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 literature review 2.1 environmental damage of the internet 2.2 the role of awareness in changing human behavior 2.3 the role of price in changing human behaviour 2.4 theoretical foundations 3 the experiment design and sample 3.1 treatment 3.2 measures 3.3 sample 4 analysis and results 4.1 pls analysis 4.2 multi-group analysis 5 application of the framework 6 summary paper title (paper title style) australasian journal of information systems alhassan, li, reddy & duppati 2020, vol 24, research article consumer acceptance and continuance of mobile money 1 consumer acceptance and continuance of mobile money: secondary data insights from africa using the technology acceptance model abidin alhassan waikato management school university of waikato, new zealand leon li associate professor of finance waikato management school university of waikato, new zealand leonli@waikato.ac.nz krishna reddy senior lecturer of finance management and legal studies toi ohomai institute of technology, rotorua, new zealand geeta duppati senior lecturer of finance waikato management school university of waikato, new zealand abstract this research provides empirical results on context-based factors that determine the use of mobile money in africa by employing a unique secondary data which provides a new perspective on the technology acceptance model (tam) application. the hypothesised research model tests the context-based constructs to determine how these constructs affect peoples' intentions and attitudes towards the continued use of mobile money. exploring these constructs provides an understanding of the insights into the marketing campaigns and growth of mobile money providers. the empirical results suggest that the availability of electricity is an important factor for mobile phone functionality and ongoing use of mobile money in the long run. there is also a correlation between regulations that are perceived to be enabling and the intentions of individuals to continue using mobile money. however, there is a negative correlation between rural dwellings and the intentions of individuals to adopt the use of mobile money which finds its basis in the inadequate mobile networks and national grid systems in rural areas. keywords: mobile money, technology acceptance model, ubiquity, perceived rural data availability: data analysed in the study are collected from public sources. 1 introduction mobile money is considered by many (de koker and jentzsch, 2013; gosavi, 2018; ngugi et. al, 2010; porteous, 2009) as the new service frontier for financial inclusion, especially in africa. this innovation has created tremendous opportunities for both service providers and users. prior research states that only 23% of adults in sub saharan africa (ssa) living below 2usd/day possess a formal bank account (demirgüç-kunt et al., 2015). the uptake of mobile australasian journal of information systems alhassan, li, reddy & duppati 2020, vol 24, research article consumer acceptance and continuance of mobile money 2 phones in africa offers opportunities for financial access to previously unbanked segments of the population. however, the very trend of increased penetration of mobile money among the poor of africa has concerned policymakers and development experts (aker and mbiti 2010). for mobile money providers (mobile network operators, mnos), acquiring new customers and promoting their usage is only the initial step. they also need to retain existing users and facilitate their ongoing custom, which represents better security for the long-term existence of providers. reichheld and schefter (2000) have contended that the cost of acquiring a new user is five times that of retaining an existing user. according to gsma (2019), there are 395.7 million registered mobile money accounts in the sub-saharan african region as of december 2018, representing 45.6% of the global figures. further, out of the 395.7 million registered mobile money accounts, 145.8 million are active users. what accounts for this fall in active usage? although competition among providers is intense (okeleke, rogers, and pedros, 2017; gsma, 2016), and service provisions are almost homogenous, mobile money providers need to retain users in order to achieve a competitive advantage. the costs of switching from one provider to another are also relatively low for customers which means that active, ongoing usage levels should also remain high. however, factors such as the limited availability of power, constraining regulations, poor infrastructure within rural dwellings, and low incomes still remain, and these cut across all provider networks. for providers to be able to maintain and retain active customers in the mobile money industry, it is important that they overcome these structural barriers. to date, no research has investigated the relationship between mobile money adoption/retention and the exigencies of power supply and government regulations. despite the fact that consumers’ initial adoption of mobile money is an important first step towards successful financial inclusion (the inception of the kenyan m-pesa in 2007 provides assurance that consumers are willing to engage), many scholars note that the long-term viability of the use of information technology for financial inclusion and its eventual success depends on its continued use rather than its initial adoption (zheng, zhao, & stylianou, 2013, bhattacherjee, 2001). this is especially critical with regard to the success of mobile money (san-martin and lópez-catalán, 2013, hung, yang, and hsieh, 2012). however, the ongoing use of mobile money has garnered little research attention to date. the question that emerges is this: “are consumers in africa able to continue using mobile money beyond the initial stage of adoption or do current structural bottlenecks preclude their ongoing usage?”1 this study contributes to the literature by providing empirical evidence on critical contextbased factors (modelled using tam) that leads to long-term ongoing usage of mobile money in rural africa. given the high practical relevance of prior empirical work (schierz, schilke, and wirtz, 2010), this research develops and tests an integrative model of factors determining consumers’ adoption and use of mobile money services. service pricing, the supply of electricity for mobile phone functionality, mobile network infrastructure development, government regulations, and income disparities are all key determinants of ongoing usage by consumers. presumably, if ongoing use of mobile money proves to be less costly for consumers 1 empirical evidence of poor uptake and minimal levels of continuous use of bank accounts in randomised control trials (rcts) highlight the problems that arise if context-based factors are not addressed before interventions and innovations are rolled out. more details can be found in dupas et al., (2012) and karlan, ratan, and zinman (2014). australasian journal of information systems alhassan, li, reddy & duppati 2020, vol 24, research article consumer acceptance and continuance of mobile money 3 than traditional alternatives -for example, money transfer through banks and informal mechanisms using bus drivers as middlemen -there will be greater consumer acceptance of this innovative technology in the long run (schierz et al, 2010). our empirical results show that the availability of electricity is a significant factor for consumers initially adopting and intending to continue using mobile money in africa. . our study finds that a second significant factor in facilitating the adoption and ongoing use of mobile money in africa is government regulations that are enabling. conversely, income disparities appear to be an insignificant factor in this regard.2 the rest of this paper is as follows: section 2 reviews the relevant literature and addresses the theoretical perspectives; section 3 develops the research hypotheses; section 4 describes the data, measurement of the variables and the research method; section 5 presents the empirical results; section 6 provides a discussion and addresses policy implications; and finally, the conclusions of this study and future research directions are summarized in section 7. 2 theoretical perspectives davis (1985) argues that a person’s motivation to adopt a technological system can predict how that person makes use of the system, which is also directly influenced by external stimuli such as the system’s features and capabilities. he proposed the technology acceptance model (tam) which has been considered the most influential extension of the theory of reasoned action (tra) and the theory of planned behaviour (tpb) (fishbein and ajzen, 1975). accordingly, tam has five attributes that help to predict the acceptance of a system by an individual: perceived usefulness, perceived ease of use, attitudes toward use, intention to use, and actual use. davis (1989) defined perceived usefulness as “the degree to which a person believes that using a particular system would enhance his or her job performance” (davis, 1989 p. 320). perceived ease of use is defined as “the degree to which a person believes that using a particular system would be free of efforts” (davis, 1989 p. 323). fishbein and ajzen (1975) argue that attitudes toward the use of an information system is “the individual’s positive or negative feelings or value about performing the target behaviour” (fishbein and ajzen, 1975 p.216). the intention to use is described as “the strength of one’s intention to perform a specified behaviour” (fishbein and ajzen, 1975 p.288). according to bagozzi (2007), the tam model is too parsimonious and should be expanded by factors relevant to the specific technology under investigation. integrating variables from related theoretical perspectives can also provide a better understanding of consumer acceptance (nysveen, pedersen, and thorbjørnsen, 2005). thus, consistent with other studies (luarn and lin, 2005; schierz, schilke et al., 2010 ), we use the main constructs of tam as the foundational point of our research and extend it with additional constructs important to mobile money acceptance in the african context (schierz, schilke et al., 2010). 2 this is true when we reflect on the number of mobile phones in africa that appear to belie the poverty of african people. see aker and mbiti (2010) for more information on this. australasian journal of information systems alhassan, li, reddy & duppati 2020, vol 24, research article consumer acceptance and continuance of mobile money 4 3 development of research hypotheses 3.1 attitudes to adopt and intentions to keep using mobile money attitude is defined as the degree to which using technology is positively or negatively valued by an individual. attitude in turn positively affects consumers’ adoption of technology and their usage intentions (davis, 1989; davis, bagozzi, & warshaw, 1989). personal experience plays a strong role in forming an individual’s attitude and social influence is another important determinant of consumer intentions and behaviour (venkatesh et al., 2003). in our model, we represent attitude by the percentage of subscribers per 100 inhabitants of mobile money users. this proxy is important in the sense that the percentage of people who actually subscribe to any mobile company will determine their attitudes to adopt and use the system. it represents individual willingness to choose and use mobile money as opposed to other alternatives (if any) by actively using it on an ongoing basis. gsma (2019) reports that most african countries' mobile subscriptions are in the range of 120% to 189%, which is higher than in some developed countries. in line with this, we propose the following hypothesis: h1 : there is a positive relationship between an individual’s attitude towards adoption of mobile money and their ongoing use of it. 3.2 perceived usefulness, ease of use, and adoption of mobile money davis (1989) refined the tam by suggesting that a user’s motivation can be explained by three factors: perceived usefulness, perceived ease of use, and attitude towards using the system. the relationship between perceived ease of use and perceived usefulness and their effects on a user’s behaviour to adopt and continue using the technology have been examined in it/is literature (venkatesh & davis, 1996; horton & lipsitz, 2001; davis 1989). while the above studies used tam wholly as the theoretical model, others have extended the tam by including additional constructs. for instance, lu et al., (2003) extended the tam by studying mobile wireless internet acceptance. they argue that individual differences, technological complexities, facilitating conditions, social influences, and trust in the wireless environment have a positive effect on long-term and short-term perceived usefulness and on perceived ease of use. mao et al., (2005) developed a research model to test the adoption of advanced mobile phone services (e.g. mobile internet access, e-mail, and payments) using tam. in addition to perceived usefulness and perceived ease of use, they found support for variables measuring efficacy and personal innovativeness. finally, kwon and chidambaram (2000) suggested that the adoption and acceptance of cellular phones was influenced directly and indirectly by individual characteristics, perceived ease of use, perceived usefulness, enjoyment, social pressure, and apprehensiveness. based on these studies, we propose our generic tam hypotheses as follows: h2 : there is a positive relationship between perceived usefulness and attitudes to adopt mobile money. h3 : there is a positive relationship between perceived ease of use and attitudes to adopt mobile money. 3.3 effects of ubiquity/power supply and rural dwelling on perceived ease of use and perceived usefulness of mobile money this study suggests that ubiquity of mobile phones, the availability of power, and rural dwellings have an influence on consumer perceptions of the usefulness and convenience of australasian journal of information systems alhassan, li, reddy & duppati 2020, vol 24, research article consumer acceptance and continuance of mobile money 5 mobile money. this is in accordance with previous literature which highlights the importance of identifying antecedents of tam (agarwal & prasad, 1999; benbasat and barki, 2007) based on particular is and geographical contexts. perceived usefulness and perceived ease of use are hypothesised as antecedents of individual attitudes towards adoption which, in turn, affects intentions to use mobile money on an ongoing basis. kleinrock (1996) argues that most people are “nomads” when it comes to technology and this can be identified with mobile phones and africans who are always on the “go” for economic activities. a person who leads a busy life such that he/she is always on the move will be more likely to adopt mobile money technology compared to one who leads a sedentary lifestyle. the effectiveness of mobile money rests on its ubiquity, which enable transactions to place anytime, anywhere (kleinrock 1996), particularly the extent to which mobile money is perceived to provide personalised and uninterrupted banking and payments needs between the individual and other individuals, businesses, and governments, (p2p, p2b, p2g, b2b, and b2g). however, this can only happen if the functionality of the mobile phone -such as having a reliable source of electricity to charge it -is supported. as electricity generation is well below full capacity in most parts of africa (rubin, 2017), individuals, especially rural dwellers, may struggle to find a reliable power source to charge their mobile devices. there are multiple documentations of the extreme lengths to which rural people resort in order to access a power source to charge their mobile phones and to access mobile phone network services (eriksson, 2008; gyimah-brempong & karikari, 2007; jacobson, 2007). rural dwellers who have no electricity will find it more difficult to adopt and use mobile money compared to those in urban areas with a reliable source of electricity. therefore, if the national grid is unavailable, the mobility-related needs that serve as the driver for adopting and using mobile money (kakihara and sorensen, 2002) in the first place may affect perceptions of its usefulness and convenience. we therefore propose our third and fourth hypothesis: h4 : there is a positive relationship between ubiquity of mobile phones/reliable power supply and perceived ease of use of mobile money. h5 : there is a negative relationship between rural dwelling and perceived usefulness of mobile money. 3.4 effects of regulation and costs on perceived usefulness and perceived ease of use of mobile money enabling regulations and the associated costs with using mobile money can also affect consumer perceptions of the usefulness and convenience of mobile money. those who adopt the use of mobile money may adapt their behaviour to the new technology if they perceive it is useful to their everyday life (agarwal and prasad, 1999). liu, kauffman, and ma (2015) argue that market competition, cooperation, and regulations act as key accelerators of industry changes, while innovations in mobile payments have the potential to transform it. in the financial services sector, financial institutions are closely connected to consumer welfare, so regulators are extremely cautious about how disruptive technological innovations may change the market (liu et al., 2015). enabling regulations facilitate acceptance and uptake of new systems (zacharis, 2012) in providing assurance that sanctions will follow should there be any breach of contracts. inter-operability is another reason why regulation is important. because of competition and financial sector harmonisation, inter-operability that does not have the necessary regulation could be as any form of informal system, which can have a negative impact on the financial system of african countries. australasian journal of information systems alhassan, li, reddy & duppati 2020, vol 24, research article consumer acceptance and continuance of mobile money 6 in looking at how the costs of mobile money services can affect consumer perceptions of its usefulness, we take inspiration from the work of swanson (1982), who provided evidence that perceptions of technology as both useful and easy to use were important behavioural determinants. swanson demonstrated that potential users will select and use information reports based on a trade-off between perceived information quality (perceived usefulness) and the associated cost of access. mathieson, peacock, and chin (2001) and luarn and lin (2005) also find that perceived costs influence one’s intentions to use an information service and mobile banking, respectively. similarly, venkatesh, thong, and xu (2012) argue that the cost and pricing structure of it/is have a significant impact on technology use by consumers. for mobile money to be perceived as useful, potential users must see that the benefits of the system outweigh the costs. consequently, we propose the next hypotheses as follows: h6 : there is a positive relationship between regulation and perceptions of mobile money as easy to use h7 : there is a positive relationship between costs and perceptions of mobile money as useful 3.5 effects of education and gender on perceptions of mobile money as easy to use venkatesh et al. (2003) argue that social influence and individual characteristics are important determinants of user intention and behaviour. arguably then, social influence and individual characteristics will influence one’s perceptions of how easy it is to use mobile money. venkatesh et al. (2012) contend that mobile device consumers are highly diverse and factors such as age, gender, and experience (education) (agarwal and prasad, 1999; porter and donthu, 2006; yang, 2005) will have a direct impact on user perceptions of mobile money, particularly in terms of perceived ease of use. females may face greater difficulties in processing information or using technology and thus may require more support, compared to their male counterparts (venkatesh et al., 2012). however, greater experience (education) can moderate perceptions as the user becomes more familiar with using the technology and will be less inclined to rely on external support (venkatesh et al., 2012). in line with these views, we hypothesise the following: h8 : there is a positive relationship between education level (experience) and perceptions of mobile money as easy to use. h9 : there is a negative relationship between female users and perceptions of mobile money as easy to use. 3.6 control variables to fully account for the differences among individuals and the organisations that provide mobile money, we include four control variables that characterise our unit of analysis (liang et al., 2007). these control variables are referred to as facilitating conditions (absorptive capacity), and comprise age, income, provider size, and marketing advertisement. facilitating conditions involve the individual’s ability to use is when conditions are conducive to their adoption. it is similar to the capacity of organisations to assimilate new it/is systems into their existing processes. for the purposes of this paper, we suggest that facilitating conditions also involves the link between the mno’s capacity to make their mobile money system userfriendly and the ability of the user to use it with minimum effort. our study controls for age and income differences between individual users and provider size and marketing strategies between mobile money providers as each of these factors may impact on user adoption and australasian journal of information systems alhassan, li, reddy & duppati 2020, vol 24, research article consumer acceptance and continuance of mobile money 7 ongoing use of mobile money systems. older consumers may struggle more to use new systems, such as mobile money while regulations may also restrict very young individuals from using it (venkatesh et al., 2012). meanwhile, the purchasing power of affluent consumers may increase their likelihood of adopting new technologies compared to poorer consumers as they have the ability to select those which are most suitable to their needs. large provider organisations may have a greater capacity to market product and services which, again, may influence levels of uptake and ongoing usage. figure 1 provides the hypothesised research model, indicating the direction of mediation. figure 1 research mode 4 data, measurement of variables and research method 4.1 data the data for the study is obtained from gsma. the gsma is a subsidiary of groupe speciale mobile (gsm), which was formed in 1982 by the confederation of european posts and telecommunications (cept) to design a pan-european mobile technology. the gsma mobile for development foundation, incorporation was created in 2007 to demonstrate the positive social impact of mobile technology, bringing together mobile operators, donors and the international development community. they also collect data known as the global mobile money dataset. this is a comprehensive set of global metrics for the mobile money industry, measured on a quarterly or monthly basis depending on the metric. among the dataset is the mobile money regulatory index, the mobile money deployment tracker that comprises live services, registered agents, registered accounts, and the volume and value of transactions. these datasets represent one of the most current sources of information about the mobile money industry. our study used these datasets to understand the drivers of mobile money adoption and ongoing usage. for the regulation variable, we used the mobile money australasian journal of information systems alhassan, li, reddy & duppati 2020, vol 24, research article consumer acceptance and continuance of mobile money 8 regulatory index, which scores countries based on the extent to which their regulatory framework enables widespread mobile money adoption. scores range between 0-100, with a higher score associated with more enabling regulations. each mobile money deployment (live service) is associated with a regulatory index, so account owners are able to discern if enabling regulations have had an impact on their attitudes and intentions to adopt and use mobile money. the gsma defines a mobile money service as a service that includes transferring money and making payments using the mobile phone. for a mobile money service to be included in the mobile money deployment tracker, it must meet the following criteria: available to the unbanked; offers a network of physical transactional points which can include agents; separate from bank branches and atms, making the service widely accessible to everyone; and separate from payment services such as apple pay and google wallet. the global mobile money dataset also has data on mobile cellular subscribers and the percentage of rural dwellers with ict infrastructure. the percentage of mobile cellular subscribers for every 100 inhabitants and the percentage of population covered by the mobile cellular network were taken from the international telecommunications unit (itu). these data were cross-checked with data from the world bank, and the differences are negligible. data on the percentage of the total population with electricity consists of the percentage of the country’s population that is connected to the national grid, also taken from itu and the world bank. the data for the cost of mobile money was taken from the research ict africa (ria) network, a non-profit organisation based in south africa. ria researches ict policy and regulations and facilitates evidence-based policies that aim to improve access, use, and applications of ict for social development and economic growth. the quarterly data on prices are gathered from mobile network operators across africa y and standardised using the oecd mobile pricing measurement (cheapest prepaid mobile voice product by country (in usd)). data on the individual characteristics for mobile money customers (a proxy for facilitating conditions) are taken from the global findex database. these comprise age, gender (female), income and education. to control for the variations in size of mobile money provider organisations, we use sales revenue and marketing budgets as two measures of organisational size, obtained from the financial statements and websites of these mnos. overall, 40 countries that offer mobile money operations are included in this study, giving us a monthly dataset of 480 data points. 4.2 measurement of variables the measurement items were formulated based on theory, review of the literature and modified to reflect the assumptions, propositions and hypotheses of context-based themes. the items used to measure attitudes towards adoption and ongoing intentions to use mobile money were taken from davis (1989), davis, bagozzi, and warshaw (1989), and luarn and lin (2005) and proxied in order to understand mobile money in the african context. australasian journal of information systems alhassan, li, reddy & duppati 2020, vol 24, research article consumer acceptance and continuance of mobile money 9 construct name description measurement coded as data source related studies intention to use mobile money given the opportunity, the respondent will use mobile payment services. percentage of the population with a mobile network in a country 1= 50% and above 0= 49% and below itu (2018), gsma (2018), world bank (2018) venkatesh et al. (2003), davis (1989) attitude towards adopting mobile money the degree to which using technology is positively or negatively valued by an individual. percentage of mobile subscribers per 100 inhabitants 1= 50% and above 0= 49% and below itu (2018), gsma (2018), world bank (2018) venkatesh et al. (2003), davis (1989) perceived usefulness the degree to which a person believes that using a particular system would enhance his or her job performance. percentage of people who used the internet to pay bills or to buy something online in the past year, primary education or less (% age 15+) 1 = 50% and above 0 = 49% and below global findex (2018) venkatesh et al. (2003), davis (1989) perceived ease of use the degree to which a person believes that using a particular system would be free of efforts. mobile money account ownership, primary education or less (% age 15+) 1 = yes 0 = otherwise global findex (2018) venkatesh et al. (2003), davis (1989) antecedents of perceived usefulness (pu) rural dwelling lack of mobile network infrastructure and electricity. rural population as the percentage of the total population 1 = 50% and above 0 = 49% and below gsma (2018) venkatesh, thong, and xu (2012) cost the extent to which a person believes that using mobile money will cost money. price of prepaid mobile voice and sms. 1 = 10 cent and above 0 = 9 cent and below ria (2018) luarn and lin (2005) antecedents of perceived ease of use (peou) ubiquity/power how well consumers believe mobile money can be integrated into their daily mobile activities (ability to use mobile percentage of the population with electrification (national grid) 1= 50% and above 0= 49% and below itu (2018), world bank (2018) kakihara and sorensen (2002), rubin (2017) australasian journal of information systems alhassan, li, reddy & duppati 2020, vol 24, research article consumer acceptance and continuance of mobile money 10 phone and mobile money at anytime and anywhere). regulation mobile money regulatory index which scores countries based on the extent to which their regulatory framework enables widespread mobile money adoption. scores range between 0-100, with a higher score associated with more enabling regulation. mobile money regulatory index which scores countries based on the extent to which their regulatory framework enables widespread mobile money adoption. scores range between 0-100, with a higher score associated with more enabling regulation. 1 = 50% and above 0 = 49% and below gsma (2018) liu, kauffman, and ma (2015) education better knowledge structures to facilitate user learning of mobile money. primary school completion rate as a percentage of the total population 1 = primary 2 = others global findex (2018) venkatesh, thong, and xu (2012), gender the likelihood that females may require more support to adapt to technology use than men. percentage of women with mobile money accounts 1= 50% and above 0= 49% and below global findex (2018) venkatesh, thong, and xu (2012), control variables age the likelihood that older consumers may turn to face difficulty in processing information or using technology (mobile money). percentage of population aged 65 and above as the percentage of total population 1 = age 65 and above 0 = age 64 and below global findex (2018) venkatesh, thong, and xu (2012) income willingness to pay (wtp) for mobile money due to price value. percentage of mobile account ownership 1 = ownership 0 = otherwise global findex (2018) venkatesh, thong, and xu (2012) provider size total assets or total market share controlled by an mno total assets or total market share controlled by an mno 1 = 20% and above 0 = 19% and below mnos websites and annual financial statements (2018) marketing budget total advertising budget committed for marketing mobile money in a year total advertising budget committed for marketing mobile money in a year 1 = us$ 1 million and above 0 = below us$ 1 million mnos websites and annual financial statements (2018) table 1: variable description, measurement and data source australasian journal of information systems alhassan, li, reddy & duppati 2020, vol 24, research article consumer acceptance and continuance of mobile money 11 as discussed earlier, perceived ease of use is the proxy for the percentage of people with primary education or lower who have mobile money accounts in a country (demirguc-kunt et al., 2018). perceived ease of use is a latent variable with a number of items (ubiquity/power, regulation, education and gender) as its antecedents and which are taken from previously validated studies and modified to fit the specific technology under study (rubin, 2017; luarn and lin, 2005; venkatesh and davis, 1996; liu, kauffman and ma, 2015; venkatesh et al., 2012). facilitating conditions include individual covariates such as age, gender (female), income disparities and experience (education), all of which affect the main tam construct, perceived ease of use and the control variables. for the other controls, provider size and marketing strategies, we obtained data from financial statements and other information on the websites of various mnos. perceived usefulness, also a latent variable (proxied as the percentage of people who used the internet to pay bills or to buy something online in the past year), (demirguc-kunt et al. 2018), is determined by the adoption features comprising the constructs of rural dwelling and cost (fishbein and ajzen, 1975; venkatesh et al., 2012; swanson, 1982). the hypothesised model (figure 1) comprised 12 observed variables and two latent constructs (perceived usefulness, perceived ease of use) are used to determine intentions to adopt and to continue using mobile money in africa. the full list of items used in this study and their modifications are listed in table 1. 4.3 assumptions for secondary data as an alternative to traditional tam traditionally, tam is grounded in primary data since constructs are built based on predetermined survey questions that enquire about user intentions to adopt and use a particular is. the limitations of the context in which this study was conducted compelled the use of secondary data as proxies for the traditional tam constructs. the following assumptions guide the use of secondary data: 1. the choice of the secondary data approach is governed by both the research questions and data availability. the research question approach is used because the authors have a priori hypotheses that needed verification through suitable datasets. the datasets used in this study were deemed to be the most appropriate proxies because of the unique nature of the phenomenon under investigation. 2. the authors generated operational definitions of the proxies used, and the independent, control, and outcome variables. this helped us to identify the best analytical approach to use. 3. the data is recorded from the original variables. this is done in order to transform them to meet the assumptions of the statistical model of analysis, structural equation modelling (sem). using secondary data as an alternative to tam, simulation analysis has some advantages, including its cost-effective nature. secondary datasets may be regarded as ‘clean data’, already scrutinised by primary data collectors who normally document the data collection and filtration process. the hypotheses for this study would otherwise require primary datasets are potentially costly to acquire. using datasets that are already available saves money, time, and serves as a preliminary study for future undertakings. secondary data also provides us with the ability to cross-link information from different data sources that are key to studying ongoing usage of mobile money systems for which the original tam model lacks. this is australasian journal of information systems alhassan, li, reddy & duppati 2020, vol 24, research article consumer acceptance and continuance of mobile money 12 necessary as some researchers agree that tam is more parsimonious (bagozzi, 2007) and therefore requires additional constructs to help understand the adoption of mobile money in africa. the use of the proxies has, therefore, opened up another window from which tam researchers can approach with confidence. finally, the use of secondary data in this study allows us to study real-life cases in conjunction with existing theories like tam. our approach enables us to identify potential new interventions to current problems that can subsequently be tested in prospective studies. a particular case in point is the ownership of mobile money in africa, a phenomenon that defies logic in light of the region’s widespread poverty (aker and mbiti, 2010). 4.4 advantages of sem over regression analysis the two most widely used statistical methods to test mediated effects in secondary data are multiple regression analysis and structural equation modelling (sem) analysis. multiple regression analysis uses methods such as stepwise regression and progressive adjustment to detect mediation effects. baron and kenny (1986) argue that the inclusion of variables in multiple regression analysis rests on the assumptions that variables should be normally distributed, given the explanatory variables, and that measurement errors and multicollinearity are absent when dealing with two or more explanatory variables. furthermore, all variables included in a multiple regression analysis must be observed measures. these assumptions cannot be supported by the type of data used for this study. because not all of the data are directly observed in a survey, we cannot guarantee that the data is free from measurement errors or that the explanatory indicators are free of multicollinearity. for the above reasons, we use sem for the empirical estimation. sem analysis is arguably the most effective way to test mediated effects when data requirements are met. as multiple indicators are used, sem is ideal in constructing measurement models for the theoretical concepts hypothesised (li, 2011) and allows for multiple indicators of latent variables which are a more realistic representation of the variables under study (schierz et al., 2010). when measurement error is a concern, as is the case with our data, sem uses multiple indicators to overcome it. in contrast to ols regression, which is based on the assumption that variables are measured perfectly (bollen 1987), sem operates on a rationale that each explanatory and dependent variable may be associated with measurement errors. 4.5 assumptions for using sem in tam analysis following bailey and pearson (1983), we postulate that the adoption of mobile money is a desire to be satisfied. legris et al. (2003) argue that satisfaction is considered as the sum of one’s feelings or attitudes toward a variety of factors affecting the situation. they defined it as the sum of m user’s weighted reactions to a set of n factors. satisfaction = ∑ 𝑊𝑖𝑗 𝑅𝑖𝑗 (𝑗 = 1, … . . , 𝑛, 𝑖 = 1, … . . , 𝑚) (1) where rij is the reaction to factor j by individual i and wij is the importance of factor j to individual i. attitudes toward adoption and intentions to use mobile money on an ongoing basis are directly related to the reactions of individuals i, who see the usefulness of mobile money in their everyday life. to understand these reactions and to measure their importance as factors that satisfy the desire of individuals’ attitudes and continuance intentions to adopt and use mobile money, certain variables must trigger them. these variables are the exogenous variables that play crucial roles for people in their decisions about whether to adopt mobile australasian journal of information systems alhassan, li, reddy & duppati 2020, vol 24, research article consumer acceptance and continuance of mobile money 13 money. this can be illustrated as paths of structural coefficients (dawn, 2009) in sem as in equation (2) below: 𝑦𝑖𝑗 = φ𝑦𝑖𝑗 + π𝑋𝑖𝑗 + ϐ (2) where yij is a vector of endogenous dependent variables, φ is a matrix of coefficients of the yij’s on other y_ij’s (part of the structural relationships), π is a matrix of coefficients of xij’s on the yij’s (also part of the structural relationship). xij is a vector of the independent latent variables, exogenous constructs (the predictor variables), and ϐ is the error disturbance. yij 's and xij’s depict the one-on-one mapping between the variables and constructs, and the matrix of measurement error ϐ, (dawn, 2009). this path model for user intentions to adopt and use mobile money is represented in figure 1. for this study, all the constructs were estimated using sem with maximum likelihood. 5 empirical results to begin with, we conducted an analysis of a hybrid model of the structural model, incorporating the measurement models and all of the paths specified in figure 1. a set of goodness-of-fit indices (gfis) was generated, indicating that the theoretical model predicted well the observed and latent input matrix (li, 2011). the goodness of fit measures for the structural model show satisfactory values as follows: goodness of fit index (gfi) = 0.89, root mean square error of approximation (rmsea) = 0.060, the bentler-bonett normed fit index (nfi) = 0.91, the tucker-lewis index (tli) = 0.88, the comparative fit index (cfi) = 0.85, the incremental fit index (ifi) = 0.90, and the coefficient of determination (cd) = 0.91, which are within accepted values (bollen, 1987). following goodhue, lewis, and thompson (2011), (as cited in henseler, ringle, and sarstedt, 2015), we examine the vif values between the constructs. this is to avoid high collinearity between constructs that may lead to an exaggerated estimation of path coefficients. any construct that exhibits a vif number above 5.00 shows collinearity. the vif values (see table 2) for the study fell between 1 and 4, all below the recommended values of below 5, indicating that multicollinearity is not an issue in our study. satisfied that the model is fit, we now show the results of the context-based factors that determine the initial uptake and ongoing usage of mobile money in africa. following bollen's (1987) approach regarding direct, indirect and total models in sem, we show the direct paths among the key theoretical constructs in which items or factors in the hypothetical model in figure 1 are either confirmed or rejected. we show the standardised coefficients in figure 2 and table 3 for easy comparison of magnitudes, as the measurement of the variables was done on different scales. the standardised coefficients for each path closely approximate the effective magnitude usually shown by beta weights in regression. thus, low coefficients have limited substantive effect (hair et al., 1995). to ensure the stability of the results, we test the correlation of the exogenous variables (see table 2) to deduce the presence of multicollinearity. berry et al. (1985) argue that for any specific empirical analysis, multicollinearity, which is a correlation among explanatory variables, can affect the stability of the empirical results. to avoid this, they suggested that, as a rule of thumb, the correlation among the explanatory variables should be below 0.8. as shown in table 4, all the correlations are within the cut-off value of 0.8, reconfirming that the multicollinearity is not an issue in our study. australasian journal of information systems alhassan, li, reddy & duppati 2020, vol 24, research article consumer acceptance and continuance of mobile money 14 item intention attitude pu rural pu cost pu ubiquity pu regulation peou education peou gender age income provider size marketing vif intention 1.00 .21 attitude .68 1.00 .30 pu rural -.41 -.23 1.00 .17 pu cost -.35 -.74 -.57 1.00 .39 peou ubiquity .74 .51 -.25 -.58 1.00 .27 peou regulation .72 .57 -.24 -.19 .72 1.00 .32 peou education .58 .49 -.70 -.44 .67 .54 1.00 .41 peou gender -.34 -.31 -.36 .23 -.16 .61 .14 1.00 .09 age .46 -.01 -.42 .52 -.63 .31 -.31 .08 1.00 .28 income .61 .21 -.29 .27 -.65 .16 -.52 -.21 .25 1.00 .22 provider size .06 .21 -.34 -.49 .38 .54 .63 .05 .06 .12 1.00 .33 marketing .10 .13 .09 .72 .64 .42 .41 .07 .23 .30 .65 1.00 .29 notes: 1. this table lists the correlations matrix of all the tested variables in the study. first, the positive and significant correlation between peou ubiquity/power and continuous intention to adopt mobile money indicates that the supply of electricity is important to the adoption of mobile money. second, the correlation between peou ubiquity/power and attitudes towards adoption of mobile money is also positive and significant, which supports a similar argument as above. surprisingly, the values of pu rural and continuous intention to adopt mobile money is negative and significant, which indicate the challenges of rural dwellers on the adoption of mobile money in africa. the correlation matrix largely supports the estimation results in table 4 and figure 2. 2. the correlation estimates shown in this table suggest that multicollinearity is not a problem among the constructs. all constructs are below the cut-off figure of .80, and .50 for correlation and vif respectively. 3. pu is perceived usefulness and peou is perceived ease of use. table 2 correlation matrix and vif among the exogenous variables australasian journal of information systems alhassan, li, reddy & duppati 2020, vol 24, research article consumer acceptance and continuance of mobile money 15 since measurement scales can affect the results of the estimation of the sample if items are found to be highly correlated with each other, we test the scale of reliability for the consistency of the data and items in our sample by looking at the detailed information of the cronbach’s alpha of the 12 items used. as shown in table 3, the item-test correlation and the item-rest correlation all produce figures that are within limits of 0.8. the overall α of 0.8 is an indication that items in the model are reliable and consistent with providing stable results. item obs sign item-test correlation item-rest correlation inter item correlation alpha perceived intention 40 + .79 .72 .23 .75 perceived attitude 40 + .72 .76 .23 .75 antecedents of perceived usefulness rural 40 .52 .56 .28 .74 cost 40 .58 .46 .26 .78 antecedents of perceived ease of use ubiquity/power 40 + .71 .75 .23 .75 regulation 40 + .64 .54 .25 .77 education 40 + .78 .73 .23 .75 gender 40 .24 .08 .31 .82 control variables age 40 .30 .14 .30 .81 income 40 .32 .17 .30 .81 provider size 40 + .68 .58 .25 .77 marketing 40 + .72 .68 .24 .74 test scale .26 .80 notes: this table presents the estimation results of the scale reliability test. the standardised cronbach’s alpha for the scale reliability is found through the equation 𝛼𝑠𝑡𝑎𝑛𝑑𝑎𝑟𝑑𝑖𝑠𝑒𝑑 = 𝐾𝜗 (1+(𝐾−1)𝜗) , where k is the number of items, 𝜗 is the average inter-item correlation, that is the mean of k (k-1)/2 coefficients in the upper triangular correlation matrix. the table measures the internal consistency of the 12 items, showing how closely related the items are as a group. item-test and item-rest correlations show values below .8 are reliable and consistent. the alpha of .8 suggests that the items have high internal consistency. the standardised values are important as it helps in supporting the hypotheses tested. table 3. scale reliability test of standardised items as shown in table 4, all our hypotheses are supported in the model. for the control variables, age and income are not significant, but the size and marketing strategies of mobile money providers show significant values. the r2 values are substantial for perceived usefulness (r2 = 0.677), attitudes towards adoption (r2 = 0.691), perceived ease of use (r2 = 0.785), and intentions to continue using mobile money (r2 = 0.683). the results show that, together, rural dwelling and costs explained 67.7% of the total variance in perceived usefulness, while ubiquity/ power supply, regulation, education and gender explained 78.5% of the variance in perceptions of mobile money as easy to use. moreover, 69.1% of the variance in attitudes towards adoption is explained. 68.3% of the variance in intentions to adopt and to keep using mobile money is explained. the results of these r2 statistics indicate that the structural model developed in this study has an acceptable level of predictive accuracy as seen in figure 2. australasian journal of information systems alhassan, li, reddy & duppati 2020, vol 24, research article consumer acceptance and continuance of mobile money 16 examined variable sem coeff chi-statistics remark 𝐻1: attitude towards adoption .51*** χ2= .72 accepted (4.21) df = 1, p = .000 𝐻2: pu .42*** χ2= .72 accepted (6.73) df = 1, p = .000 𝐻3: peou .54*** χ2= .72 accepted (3.45) df = 1, p = .000 𝐻4: peou ubiquity/power .54*** χ2= .79 accepted (6.13) df = 1, p = .000 𝐻5: pu rural dwelling -.39*** χ2= .49 accepted (-5.08) df = 1, p = 000 𝐻6: peou enabling regulation .33*** χ2= .59 accepted (7.20) df = 1, p = .000 𝐻7: pu cost .17** χ2= .68 accepted (4.09) df = 1, p = .005 𝐻8: peou education .24*** χ2= .61 accepted (3.55) df = 1, p = .000 𝐻9: peou gender (female) -.02* χ2= .37 accepted (-0.57) df = 1, p = .641 control variables age -.01ns not accepted income -.03ns not accepted provider size .09*** accepted marketing .08*** accepted overall 𝑅2 .64 table 4 mediation effects on individual attitudes towards the adoption of mobile money in africa. figure 2 results of the path relations in the research mode australasian journal of information systems alhassan, li, reddy & duppati 2020, vol 24, research article consumer acceptance and continuance of mobile money 17 6 discussion and implications 6.1 theoretical implications this study examines the factors that influence the initial adoption of, and ongoing intentions to use, mobile money by employing tam that includes perceived usefulness and perceived ease of use as moderators to attitudes towards adoption. the existing literature provides limited guidance on how tam is perceived within the context of african is adoption. the importance of context cannot be overstated (agawal and prasad, 1997); in another context, there may be differences in the variables identified to understand is use and acceptance. equally, the meaning of variables such as perceived usefulness and perceived ease of use may be drastically influenced by the type of is being studied. the use of proxies to study adoption of mobile money in africa is, therefore, a justifiable enterprise. the most important determinant of mobile money usage in africa is perceived ease of use. context-based variables such as ubiquity/power supply, enabling regulations, education and gender are positively associated with individual attitudes towards the adoption of mobile money. perceived usefulness is another significant determinant of the uptake of mobile money in africa. these findings support prior studies on mobile payment adoption and acceptance (bailey et al, 2017; he, chen, and kitkuakul, 2018; polites and karahanna, 2018). the findings also open a new window on the use secondary data in tam that is parsimonious. 6.2 methodological implications from a methodological point of view, our study offers a new perspective on how tam can be analysed, incorporating secondary data. our novel approach is applied to a new is (mobile money) that has, until now, received minimal examination in relation to tam. our use of proxies highlights how available datasets can be leveraged to study tam. the outcomes of using these proxies warrant some discussion here. ubiquity and power supply are shown to be particularly important; simply put, if people lack a reliable supply of power to charge their phones, they cannot use mobile money systems. african people are known to be mobile (kakihara and sorensen 2002; kleinrock, 1996), in search of economic opportunities, and mobile money supports this mobility. limited access to electricity affects the number of people who can adopt and sustain their use of mobile money (rubin, 2017). in africa, the costs involved in securing access to electricity are high, especially for inhabitants of towns and villages that are not connected to the national grid. such mobile phone users are compelled to travel long distances in order to access electricity just to keep their phones functional. although the availability of mobile money systems in africa should, in theory, allow people to increase the quality of their lifestyles, the lack of basic infrastructure, such as limited access to electricity3, prevents a great number of people from reaping these benefits. regulations are another significant factor in consumer intentions to adopt and to keep using mobile money. when mobile money systems were first introduced in africa (m-pesa), regulations were not considered to be of high importance (jack and suri, 2011). however, consumer reports of abuse by agents and operators, and most recently fraud, have made 3 the countries with the lowest levels of public access to electricity are as follows: burundi (6.5% of the population), malawi (9.8% of the population), liberia (9.8% of the population), central african republic (10.8% of the population), burkina faso (13.1% of the population) (world bank, 2014). australasian journal of information systems alhassan, li, reddy & duppati 2020, vol 24, research article consumer acceptance and continuance of mobile money 18 regulations an important factor in perceptions of mobile money as easy to use since consumers are assured of some means of recourse if contracts are breached. furthermore, as with other traditional financial systems (gsma, 2016), mobile money requires proper regulations to be in place as it is a form of financial provision. mnos also require regulations to ensure that their investments are secured and to enable them to conduct legitimate and competitive business. regulations also ensure that government revenue generation, in terms of taxes, is adequately secured. they are also a mechanism for effective competition between mobile money providers, ensuring that prices are kept low for everybody. previous user experience of mobile money platforms and brand reputations will also influence one’s intentions to adopt and to keep using mobile money. prior experience with the technology reduces one’s dependence on external support to use mobile money (venkatesh et al., 2012). the intuitive interface of mobile money platforms (mcgrath and lonie 2013) can lead to positive attitudes towards adoption. according to wu and wang (2005), the extent to which a new system is consistent with those previously used by the user strongly influences perceptions of that new system’s ease of use. the use of mobile money by women is found to be negative and moderately significant. this confirms the hypothesis that females tend to depend on men for engaging with new information systems, which is in line with the findings of venkatesh et al. (2012). our findings with regard to rural dwelling and the perceived usefulness of mobile money supports our hypothesis and existing literature in that mobile money adoption and usage is concentrated in cities and urban centres (aker et al., 2016; aker and mbiti, 2010; asongu, 2015; donovan, 2012). although adoption of mobile money is starting to grow in rural areas, there remains a great disparity between rural uptake and adoption levels in urban centres, which reflects the underdevelopment of infrastructures in rural areas of africa (mas and kumar, 2008, rubin, 2017). a comparison between the transaction costs associated with mobile money services and those of traditional money transfer systems in africa indicates that mobile money has advantages (aker and mbiti, 2010; jack and suri, 2014). the fast and secure nature of mobile money services also contribute to the reduction in costs, further reinforcing its perceived usefulness (aker et al., 2016; gsma, 2016; jack and suri, 2011; mas and kumar, 2008). individual attitudes towards mobile money are an important factor for its adoption and use in africa. we defined this factor with a proxy -the percentage of mobile subscribers per 100 inhabitants -and found a positive and significant correlation. the data also provides evidence to this effect. for instance, the data for attitude towards mobile money for south africa, mauritius and ghana are 147.13%, 143.73% and 135.8% respectively. the positive attitudes of african people towards mobile money reflects its convenient, secure and fast nature, but it also exposes the inadequate and underdeveloped nature of other payment systems in africa. for the controls, we find negative and insignificant coefficients for age and income but positive and significant values for provider size and ability to market mobile money products and services. the result is consistent with the literature that adults aged 65 years and over are less likely to adopt mobile money because of the technical knowledge associated with it (zimmerman and arnold, 2013). also, most african countries restrict people below the age of 18 from purchasing sim cards which may hinder this group from using mobile money even if they may have access to mobile phones (zimmerman and arnold, 2013). the use of income to australasian journal of information systems alhassan, li, reddy & duppati 2020, vol 24, research article consumer acceptance and continuance of mobile money 19 gauge uptake of mobile money needs further analysis as the number of mobile phones per head belies the widespread poverty in africa. provider size and marketing is important, and each mno’s strategy is to capture the greatest market share in order to enjoy economies of scale. 6.3 managerial implications this study reveals that organisational support is required to promote consumer adoption and ongoing usage of mobile money. this can be done through the provision of incentives and through strengthening end-user support so as instil confidence among users. it also revealed that mnos use marketing communications like advertisements to encourage consumers to adopt and to continue using mobile money. managers of mobile money services can leverage the positive impact of mobile money in africa to expand their services to enhance financial inclusion, which is a necessary component for the region’s growth and development. it is further envisaged that mnos will, in the long run, remain competitive in terms of service pricing as ongoing increases in price will dissuade consumers from continuing their use of mobile money. 6.4 policy implications our study has important policy implications for mobile money services in africa. it demonstrates that the availability of electricity is absolutely crucial for the uptake of mobile money. people need a reliable source of power in order to charge their mobile phones so that these phones can be used anytime and anywhere. basic infrastructure is a necessary component for the expansion of mobile money services into areas that are yet to be adequately connected. if power or energy is available, this cuts the cost of travel for rural dwellers who may intend to use mobile money but who do not have the means to maintain the functionality of their phones in their homes. rubin (2017) emphasises that policy frameworks are gathering momentum internationally around the build-once approach to infrastructure development. an example of the build-once approach for africa through the public private partnerships (ppps) would be electricity and mnos sharing towers for their wires. another example involves the coordination of mnos to lay wires under new roads for electricity. this policy framework will direct future ict and energy transformations that can further lower the costs of electricity. in turn, this may allow mnos to increase their customer base if consumers have access to basic infrastructure that enables them to keep using mobile money. enabling regulations are important for the growth of mobile money services. since the introduction of m-pesa in 2007, mobile money services in africa have undergone major transformations in terms of service provision, competition, consumer protection protocols, monetary policy frameworks by the government and central banks, and tax payments. enabling regulations also imply that mnos are regulated which gives consumers the assurance that their rights are protected. central banks have an obligation to ensure that regulations are sufficiently enabling in order to sustain the evolution and expansion of mobile money. the interoperability of mobile money remains a challenge for several african countries (particularly within the eight countries of the west african economic monetary union (waemu) as existing regulations are often inadequate to address the changes and complexities that arise as technology keeps evolving. policymakers and regulators need to undertake broader consultation involving a wide range of stakeholders in order to progress forward. australasian journal of information systems alhassan, li, reddy & duppati 2020, vol 24, research article consumer acceptance and continuance of mobile money 20 7 conclusion and future research this study seeks to understand the context-based factors that influence the adoption and continuous use of mobile money in africa. using data from different sources, we presented the preliminary results for 40 african countries in relation to consumer intentions to use mobile money. our study used tam with modifications made for local context factors. we use the model to determine how these factors affect the initial adoption and ongoing use of mobile money. using sem, our results show that access to electricity is crucial to consumers’ ability to use mobile money. partnerships between mnos and the government are critical to improving the provision of electricity to rural inhabitants which, in turn, enables mnos to grow and sustain their customer base. mnos can influence the attitudes of consumers through their marketing and agent networks. our preliminary empirical results for the attitude construct (proxied by the percentage of mobile network subscribers per 100 inhabitants) indicate that peoples’ attitudes play a major role in the uptake and ongoing use of mobile money in africa. mnos should identify reference groups who can play an important role in the diffusion of mobile money. in particular, if users are identified in the early stages of their adoption of mobile money, and they continue to enjoy the benefits of it, they may influence their social networks to also adopt mobile money. our results highlight the importance of enabling regulations as a mechanism for increasing the uptake of mobile money. as people become increasingly aware of regulations that guide mnos in their mobile money operations, people are more inclined to trust the system of mobile money. conversely, a lack of proper regulations will hinder the inroads made by mobile money in africa thus far. finally, the current lack of infrastructure (mobile networks and electricity) continues to hinder the diffusion of mobile money in rural areas of africa. given that the majority of africans continue to live in rural areas, it is only prudent that mnos collaborate with governments to improve these infrastructures for shared prosperity. there are several directions for future research that this research facilitates. first, the data used is secondary, with its attended measurement issues. future studies could use primary data to test the model further and confirm the preliminary empirical results gathered in this research. second, there may exist other context-based factors that could further explain the adoption and ongoing use of mobile money by african consumers. a fruitful direction for additional research would be to examine the earlier stages of the adoption process and to capture the responses of individuals who eventually stop using mobile money. finally, the results of this study are valid only for the african context, which is characterised by a very high penetration rate of mobile phones in spite of widespread poverty and limited infrastructure. more empirical studies should be carried out in other cultural settings with similar characteristics of widespread poverty, limited infrastructure and minimal regulations, to widen our knowledge. australasian journal of information systems alhassan, li, reddy & duppati 2020, vol 24, research article consumer acceptance and continuance of mobile money 21 references agarwal, ritu, and jayesh prasad. 1999. "are individual differences germane to the acceptance of new information technologies?" decision sciences 30 (2):361-391. agarwal, r prasad, j. 1997. the role of innovation characteristics and perceived voluntariness in the acceptance of information technologies. decision science, 28,(3), 557-582. aker, jenny c, rachid boumnijel, amanda mcclelland, and niall tierney. 2016. "payment mechanisms and anti-poverty programs: evidence from a mobile money cash transfer experiment in niger." economic development and cultural change 65 (1):1-37. aker, jenny c, and isaac m mbiti. 2010. "mobile phones and economic development in africa." journal of economic perspectives 24 (3):207-32. asongu, simplice. 2015. "the impact of mobile phone penetration on african inequality." international journal of social economics 42 (8):706-716. bagozzi, richard p., 2007. "the legacy of the technology acceptance model and a proposal for a paradigm shift." journal of the association for information systems 8 (4):243-254. bailey, a.a.; pentina, i.; mishra, a.s.; mimoun, m.s.b. (2017). mobile payments adoption by us consumers: an extended tam. int. journal of retail distribution management, 45, 626– 640. baron, reuben m, and david a, kenny. 1986. "the moderator-mediator variable distinction in social psychological research: conceptual, strategic, and statistical considerations." journal of personality and social psychology 51 (6):1173-1182. benbasat, izak, and henri barki. 2007. "quo vadis tam?" journal of the association for information systems 8 (4):7. berry, william d, william d berry, stanley feldman, and dr stanley feldman. 1985. multiple regression in practice: sage. bhattacherjee, anol, 2001. "understanding information systems continuance: an expectationconfirmation model." mis quarterly, 351-370. bollen, kenneth a., 1987. "total, direct, and indirect effects in structural equation models." sociological methodology:37-69. doi: url: https://www.jstor.org/stable/271028. davis, fred d., 1985. "a technology acceptance model for empirically testing new end-user information systems: theory and results." massachusetts institute of technology, boston. davis, fred d. 1989. "perceived usefulness, perceived ease of use, and user acceptance of information technology." mis quarterly (13(3)):319-340. davis, fred d, richard p bagozzi, and paul r warshaw. 1989. "user acceptance of computer technology: a comparison of two theoretical models." management science 35 (8):9821003. dawn iacobucci,. (2009). everything you always wanted to know about sem (structural equations modeling) but were afraid to ask. journal of consumer psychology (19) 673–680. de koker, l., & jentzsch, n. (2013). financial inclusion and financial integrity: aligned incentives?. world development, 44, 267-280. https://www.jstor.org/stable/271028 australasian journal of information systems alhassan, li, reddy & duppati 2020, vol 24, research article consumer acceptance and continuance of mobile money 22 demirgüç-kunt, asli, leora f klapper, dorothe singer, and peter van oudheusden. 2015. "the global findex database 2014: measuring financial inclusion around the world." the world bank. demirguc-kunt, asli, leora klapper, dorothe singer, saniya ansar, and jake hess. 2018. the global findex database 2017: measuring financial inclusion and the fintech revolution: the world bank. donovan, kevin. 2012. "mobile money for financial inclusion." information and communications for development 61 (1):61-73. dupas, pascaline, sarah green, anthony keats, and jonathan robinson. 2012. challenges in banking the rural poor: evidence from kenya's western province. national bureau of economic research. eriksson, evanjeline. 2008. a case study about cell phone use by people in rural kenya. divaportal.org. fishbein, martin, and icek ajzen. 1975. belief, attitude, intention and behaviour: an introduction to theory and research. goodhue, dale l, william lewis, and ronald l thompson. 2011. "a dangerous blind spot in is research: false positives due to multicollinearity combined with measurement error." amcis. gosavi, a. (2018). can mobile money help firms mitigate the problem of access to finance in eastern sub-saharan africa? journal of african business, 19(3), 343-360. gsma. 2016. "the mobile economy-africa 2016." london: gsma, g intelligence london. gsma. 2019. "mobile money metrics,." gsma intelligence, london. uk. gyimah-brempong, kwabena, and john agyei karikari. 2007. "telephone demand and economic growth in africa." oecd economic studies, 38 (1). hair, joseph f, rolph e anderson, ronald l tatham, and william c black. 1995. "multivariate data analyses with readings." englewood cliffs, new jersey. he, y.; chen, q.; kitkuakul, s. (2018). regulatory focus and technology acceptance: perceived ease of use and usefulness as efficacy. cogent business management, 5, 1–22. henseler, jörg, christian m ringle, and marko sarstedt. 2015. "a new criterion for assessing discriminant validity in variance-based structural equation modelling." journal of the academy of marketing science 43 (1):115-135. horton, nicholas j, and stuart r lipsitz. 2001. "multiple imputations in practice: comparison of software packages for regression models with missing variables." the american statistician 55 (3):244-254. hung, ming-chien, shih-ting yang, and ting-chu hsieh. 2012. "an examination of the determinants of mobile shopping continuance." international journal of electronic business management 10 (1):29. j.e. bailey, s.w. pearson. (1983). development of a tool for measuring and analysing computer user satisfaction, management sciences 29 (5), pp. 530–545. australasian journal of information systems alhassan, li, reddy & duppati 2020, vol 24, research article consumer acceptance and continuance of mobile money 23 jack, william, and tavneet suri. 2011. mobile money: the economics of m-pesa. national bureau of economic research. nber.org. jack, william, and tavneet suri. 2014. "risk-sharing and transactions costs: evidence from kenya's mobile money revolution." american economic review 104 (1):183-223. jacobson, arne, 2007. "connective power: solar electrification and social change in kenya." world development, 35 (1):144-162. kakihara, masao, and carsten sorensen. 2002. "mobility: an extended perspective.", proceedings of the 35th annual hawaii international conference on system sciences. karlan, dean, aishwarya lakshmi ratan, and jonathan zinman. 2014. "savings by and for the poor: a research review and agenda." review of income and wealth 60 (1):36-78. kleinrock, leonard. 1996. "nomadicity: anytime, anywhere in a disconnected world." mobile networks and applications, 1 (4):351-357. kwon, hyosun stella, and laku chidambaram. 2000. "a test of the technology acceptance model: the case of cellular telephone adoption." proceedings of the 33rd annual hawaii international conference on system sciences. li, spencer d. 2011. "testing mediation using multiple regression and structural equation modelling analyses in secondary data." evaluation review 35 (3):240-268. liang, huigang, nilesh saraf, qing hu, and yajiong xue. 2007. "assimilation of enterprise systems: the effect of institutional pressures and the mediating role of top management." mis quarterly:59-87. lim, w. m. (2018). dialectic antidotes to critics of the technology acceptance model. australasian journal of information systems, vol 22, research note lim, w, a, lim and c, phang. 2019. toward a conceptual framework for social media adoption by non-urban communities for non-profit activities: insights from an integration of grand theories of technology acceptance. australasian journal of information systems, vol 23 liu, jun, robert j kauffman, and dan ma. 2015. "competition, cooperation, and regulation: understanding the evolution of the mobile payments technology ecosystem." electronic commerce research and applications 14 (5):372-391. lu, june, chang lu, chun-sheng yu, and james e yao. 2003. "exploring factors associated with wireless internet via mobile technology acceptance in mainland china." communications of the iima, 3 (1):9. luarn, pin, and hsin-hui lin. 2005. "toward an understanding of the behavioural intention to use mobile banking." computers in human behaviour 21 (6):873-891. mao, en, mark srite, jason bennett thatcher, and onur yaprak. 2005. "a research model for mobile phone service behaviours: empirical validation in the us and turkey." journal of global information technology management 8 (4):7-28. mas, ignacio, and kabir kumar. 2008. "banking on mobiles: why, how, for whom?" papers.ssrn.com. australasian journal of information systems alhassan, li, reddy & duppati 2020, vol 24, research article consumer acceptance and continuance of mobile money 24 mathieson, kieran, eileen peacock, and wynne w chin. 2001. "extending the technology acceptance model: the influence of perceived user resources." acm sigmis database: the database for advances in information systems 32 (3):86-112. mcgrath, fionán, and susie lonie. 2013. "platforms for successful mobile money services." gsma mobile money for the unbanked, june 27. ngugi, b., pelowski, m., & ogembo, j. g. (2010). m‐pesa: a case study of the critical early adopters’ role in the rapid adoption of mobile money banking in kenya. the electronic journal of information systems in developing countries, 43(1), 1-16. nysveen, herbjørn, per e pedersen, and helge thorbjørnsen. 2005. "intentions to use mobile services: antecedents and cross-service comparisons." journal of the academy of marketing science 33 (3):330. okeleke, kenechi, m rogers, and x pedros. 2017. "the mobile economy 2017." gsma intelligence, london. the uk. paul legris, john ingham, and pierre collerette. 2003. why do people use information technology? a critical review of the technology acceptance model. information & management 40, 191–204. polites, g.; karahanna, e. (2018). shackled to the status quo: the inhibiting effects of incumbent system habit, switching costs, and inertia on new system acceptance. mis quarterly, 36, 21–42 porteous, d. (2009). mobilizing money through enabling regulation. innovations: technology, governance, globalization, 4(1), 75-90. porter, constance elise, and naveen donthu. 2006. "using the technology acceptance model to explain how attitudes determine internet usage: the role of perceived access barriers and demographics." journal of business research 59 (9):999-1007. reichheld, frederick f, and phil schefter. 2000. "e-loyalty: your secret weapon on the web." harvard business review 78 (4):105-113. rubin, nilmini. 2017. without energy, the internet is just a black hole: creating energy solutions for information and communications technology. washington dc.: alliance for affordable internet. san-martin, sonia, and blanca lópez-catalán. 2013. "how can a mobile vendor get satisfied customers?" industrial management data systems, 113 (2):156-170. schierz, paul gerhardt, oliver schilke, and bernd w wirtz. 2010. "understanding consumer acceptance of mobile payment services: an empirical analysis." electronic commerce research and applications 9 (3):209-216. swanson, e burton. 1982. "measuring user attitudes in mis research: a review." omega 10 (2):157-165. syed, m., & nelson, s. c. (2015). guidelines for establishing reliability when coding narrative data. emerging adulthood, 3(6), 375-387 venkatesh, viswanath, and fred d davis. 1996. "a model of the antecedents of perceived ease of use: development and test." decision sciences 27 (3):451-481. australasian journal of information systems alhassan, li, reddy & duppati 2020, vol 24, research article consumer acceptance and continuance of mobile money 25 venkatesh, viswanath, michael g morris, gordon b davis, and fred d davis. 2003. "user acceptance of information technology: toward a unified view." mis quarterly:425-478. venkatesh, viswanath, james yl thong, and xin xu. 2012. "consumer acceptance and use of information technology: extending the unified theory of acceptance and use of technology." mis quarterly:157-178. world bank, iea. 2014. "electric power consumption (kwh per capita) in africa." wu, jen-her, and shu-ching wang. 2005. "what drives mobile commerce?: an empirical evaluation of the revised technology acceptance model." journal of information management, 42 (5):719-729. yang, kenneth cc, 2005. "exploring factors affecting the adoption of mobile commerce in singapore." telematics informatics, 22 (3):257-277. zacharis, nick z, 2012. "predicting college students' acceptance of podcasting as a learning tool." interactive technology smart education, 9 (3):171-183. zheng, yiming, kexin zhao, and antonis stylianou. 2013. "the impacts of information quality and system quality on users' continuance intention in information-exchange virtual communities: an empirical investigation." decision support systems 56:513-524. zimmerman, jamie m, and julia arnold. 2013. "hope or hype? five obstacles to mobile money innovations for youth financial services." innovations: technology, governance, globalization 8 (1-2):233-246. copyright: © 2020 alhassan, li, reddy & duppati. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v24i0.2579 http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems freeman, park, middleton & allen 2016, vol 20, research on rural and remote ict broadband for socio-economic development the importance of broadband for socio-economic development: a perspective from rural australia julie freeman deakin university julie.freeman@deakin.edu.au sora park university of canberra catherine middleton ryerson university matthew allen deakin university abstract advanced connectivity offers rural communities prospects for socio-economic development. despite australia’s national broadband infrastructure plans, inferior availability and quality of rural internet connections remain persistent issues. this article examines the impact of limited connectivity on rural socio-economic opportunities, drawing from the views of twelve citizens from the boorowa local government area in new south wales. the available fixed wireless and satellite connections in boorowa are slow and unreliable, and remote regions in the municipality are still without any internet access. participants identified four key areas in their everyday lives that are impacted by insufficient connectivity: business development, education, emergency communication, and health. rural citizens often already face challenges in these areas, and infrastructure advancements in urban spaces can exacerbate rural-urban disparities. participants’ comments demonstrated apprehension that failure to improve connectivity would result in adverse long-term consequences for the municipality. this article suggests that current broadband policy frameworks require strategic adaptations to account for the socio-economic and geographic contexts of rural communities. in order to narrow australia’s rural-urban digital divide, infrastructure developments should be prioritised in the most underserved regions. keywords: rural connectivity; national broadband network; socio-economic development; policy; digital divide; digital inclusion 1 introduction access to reliable and fast digital connectivity is a perennial problem in rural australia despite large-scale investments and infrastructure developments under various iterations of the national broadband network (nbn) and several specific-purpose rural access programs in earlier years (middleton & park, 2014; alizadeh, 2013; tiwari, lane & alam, 2015). the nbn was once heralded as offering rural communities prospects for development and inclusion in the digital economy by overcoming some of the disadvantages associated with physical distances (department of broadband, communications and the digital economy, 2011). while slow and uneven network rollout (middleton & park, 2014) was to be expected with such a large-scale complex project, ongoing disparities in broadband access continue to jeopardise rural socio-economic opportunities (alizadeh, 2013). this article offers evidence of the impacts of limited connectivity (including fixed-line, mobile and satellite internet) on a rural community drawn from discussions with twelve citizens from the boorowa local government area in new south wales. the availability and quality of internet access in boorowa significantly lags behind urban centres. access in this region suffers from variable signal strengths, ongoing latency issues, significant costs, and fails to support high-bandwidth services. additionally, there are especially remote areas in the municipality 1 australasian journal of information systems freeman, park, middleton & allen 2016, vol 20, research on rural and remote ict broadband for socio-economic development that still lack any internet access. participants identified four key areas in their community that are affected by limited connectivity: business development, education, emergency communication, and health. digital connectivity has been recognised to be a driver of rural socio-economic prosperity (townsend, sathiaseelan, fairhurst & wallace, 2013; prieger, 2013; katz & suter, 2009). the internet is increasingly becoming a prerequisite to complete everyday tasks, but the apparent lack of urgency in extending effective broadband connectivity to rural australia is reinforcing and extending existing rural-urban discrepancies (alizadeh, 2013). the risk in this situation is that current disparities faced by rural communities are likely to be exacerbated as urban spaces receive advanced connectivity, and rural citizens become less able, relatively speaking, to take advantage of digital business, learning, health and safety opportunities. this article suggests that the conditions of rurality require greater consideration in nbn developments to account for communities’ socio-economic and geographical contexts. if australia is to narrow its ruralurban digital divide, then underserved rural regions should be prioritised for government investment in broadband infrastructure developments, as they were in the period before the nbn was first commissioned. 2 literature review advanced connectivity facilitates positive economic effects, digital citizenship practices and increased societal engagement (prieger, 2013). broadband has widely been attributed to fostering economic growth through improved business productivity, building human capital through online education and digital civic engagement, creating opportunities for knowledgebased telework, as well as providing greater access to essential services such as tele-healthcare (griffith, heydon, lamb, lefort, taylor, trotter & wark, 2013; townsend et al., 2013; prieger, 2013; steele & lo, 2013). however, the activities pursued online are fundamentally shaped by the availability and quality of internet connections, and those with broadband engage in more online activities (hale, cotton, drentea & goldner, 2010). in rural areas, distance to major services and low population densities increase the costs of building network infrastructure. areas with small and geographically dispersed populations provide low demand and high financial outlay (tiwari et al., 2015; eardley, bruce & goggin, 2009). while australia has a range of regional telecommunications policies, rural areas continue to suffer from lower internet penetration, poorer broadband connectivity, and higher costs for these services (regional telecommunications independent review committee, 2015). internet and broadband penetration decrease as remoteness increases. in very remote areas, only 41.4% of the population have broadband and 49.6% have internet access at home. in contrast, the figures for major cities are 70.8% and 77.3%, respectively (abs, 2013). despite efforts to narrow the digital divide between rural and urban areas, this gap has not been significantly reduced and the effects of the gap are exacerbated by rural areas’ overall lower educational and income levels as well as aging populations (park, freeman, middleton, allen, eckermann & everson, 2015). moreover, absence of rural competition diminishes the provision of innovative services and prospects for reduced prices, with limited consumer choice resulting in increased cost to access inferior infrastructure (eardley et al., 2009). as potential rural users are often already financially disadvantaged, additional costs for internet access may prove to be unnecessary expenses (notara, 2011). in addition to poor availability, the quality and reliability of rural broadband is inferior to metropolitan areas, with many communities “severely limited” (department of communications, 2013a, p4). rural broadband access under nbn plans will predominantly be achieved through fixed wireless and satellite technologies, rather than fibre-optic cabling. both fixed wireless and satellite connections offer significantly slower speeds, particularly for uploading, than fixed-line broadband (middleton & park, 2014; tiwari et al., 2015; martin & rice, 2012). the first and foremost issue therefore is uneven access to quality internet services in rural areas. 2 australasian journal of information systems freeman, park, middleton & allen 2016, vol 20, research on rural and remote ict broadband for socio-economic development rural areas already face socio-economic challenges, such as limited access to healthcare and education facilities as well as reduced opportunities for business expansion (jones, cardoso, hüttner, oliveira, dos santos, lopes & russomano, 2012; townsend et al., 2013). rural infrastructure investments have demonstrated positive socio-economic impacts and increases in productivity and participation (katz & suter, 2009). for example, communities with highquality broadband are more economically competitive (townsend et al., 2013; alizadeh, 2013) and, for businesses, the internet has become a requirement for transactional and transformative change, improved performance and new opportunities (mazzarol, 2015; townsend et al., 2013). mazzarol (2015) argues that for small and medium enterprises the decision to engage in the digital economy is no longer an option. the concern here then is what happens to rural businesses that are willing but unable to digitally participate. beyond business and economic prospects, advanced connectivity enables essential rural services (townsend et al., 2013). in the area of healthcare, for example, broadband facilitates: improved quality of rural medical practices and patient care, including continuity of health services and additional medical resources; better access to specialist services; digital consultations with cost savings for patients and public health systems (decreased travel and fewer admissions); greater opportunities for practitioners to participate in professional training and development activities; and recruitment and retention of medical staff in the rural health workforce (jang-jaccard, nepal, alem & li, 2014; jones et al., 2012; steele & lo, 2013). in terms of infrastructure, most tele-health applications, such as virtual consultations, require stable high-speed connections to support real-time interactions (steele & lo, 2013). wired broadband supports these types of practices, but the fixed, wireless, mobile and satellite services in rural australia are often too unreliable and unstable with insufficient bandwidth, particularly if affected by weather conditions, distance from towers, or bottlenecks during peak usage times (steele & lo, 2013; martin & rice, 2012; tiwari et al., 2015). there are, however, growing advantages of mobile broadband, which has proved to be an important addition to the australian digital economy by contributing $33.8 billion in 2013 (acma, 2014). businesses that have adopted mobile broadband report quicker growth, reduced costs, saved time, improved quality of goods/services, increased sales, and greater access to new markets and suppliers (acma, 2014). mobile broadband offers opportunities for rural communities’ economic development through, for example, use of gps and sensor data to help maximise production and distribution of agricultural goods (griffith et al., 2013). advancing the availability and quality of broadband in rural areas holds promise for addressing disparities in access to essential services and socio-economic opportunities (townsend et al., 2013). however, there are risks amid large-scale infrastructure developments of prioritising commercially compelling urban and regional centres, with other areas left behind (middleton & park, 2014). moreover, when some rural localities receive improved connectivity in the early stages of network rollout and other areas are left waiting, socio-economic divisions and digital gaps between regional areas can escalate (alizadeh, 2013). alizadeh’s (2013) examination of abs socio-economic and accessibility/remoteness indexes for the 60 early release nbn sites found that infrastructure advancements can intensify existing social divisions. regional areas with nbn access hold a competitive advantage over other localities without nbn infrastructure. communities at the lower end of the indexes are at the greatest risk of falling further behind, particularly if scheduled in the late stages of rollout (alizadeh, 2013). highspeed broadband infrastructure and mobile internet play pivotal roles in the future of the rural digital economy, as well as civic access to fundamental information and services. this article highlights the challenges faced by a rural community while waiting for improved infrastructure. 3 case study background and methodology this article presents a case study of the impact of limited connectivity in boorowa, a rural municipality located in the southern inlands inner regional area of new south wales. the southern inlands region surrounds the nation’s capital city, canberra, and is predominantly 3 australasian journal of information systems freeman, park, middleton & allen 2016, vol 20, research on rural and remote ict broadband for socio-economic development agricultural land (79%) with only 1% classified as urban (park et al., 2015). boorowa covers over 2,500km2 and is located approximately 120km south of the regional centre of orange and 120km north of canberra. the council (www.boorowa.nsw.gov.au) is located in the boorowa township, which is also home to approximately half of the municipality’s 2,399 residents (abs, 2011a). the local government area includes smaller villages such as frogmore, rugby, reids flat and rye park. like the rest of the southern inlands region, boorowa is largely a farming and grazing area, with 33.5% of local employment in the agriculture, forestry and fishing industries (abs, 2011b). local residents also commute to canberra (the nearest major city) and beyond for work, with the 2011 census data showing the main categories of occupation as managers (29.9%), labourers (15.3%), and technicians and trades workers (12.8%). the municipality had 341 businesses in 2011, and 167 people worked from home. the median age of boorowa residents is 45 years (australia’s median age is 37), which is distributed over 965 households. the average household size is 2.3 persons, with a median total weekly household income of $778. the proportion of all occupied private dwellings that had access to the internet was 62.8% (abs, 2011b). the region is not included in any immediate nbn plans, and connectivity is currently restricted to adsl (digital connections through the copper telephone network) and 3g mobile coverage in some townships, as well as remote connections to the interim nbn satellite service. this research is a follow-up project to a rural connectivity workshop that was held in canberra in may 2014 and involved participants from seven rural new south wales local governments (including boorowa), the department of communications, regional development australia, and australia’s information and communication technology research centre of excellence (park et al., 2015; freeman & park, 2015). this workshop sought to uncover issues faced by local authorities when seeking to expedite rural digital inclusion. the current research aimed to explore civic experiences of connectivity and its impact on local socio-economic development. in august 2014 citizens of boorowa were invited to participate in a structured dialogue about rural internet connectivity. adapting flanagan’s (2008) method, the goal was to combine the narratives from disparate individuals into a joint reflection and collaboration on priority areas necessary for development. dialogue is encouraged so that participants openly discuss and debate ideas, develop clearer understandings of how problems affect different segments of the community, and why addressing the problems is important. the narratives that emerge through these discussions help develop a coherent and collective vision on an issue, which can be used to guide actions and promote change. it is a useful method when there are diverse stakeholders involved, if the group cannot meet regularly for ongoing discussions, or if the issue being addressed is complex (flanagan, 2008). two researchers and one assistant were present during the event, each of whom took notes, and the event was audio recorded and transcribed. one researcher acted as a moderator to ensure that each participant had ample opportunity to speak his/her views and ask questions of other participants. the boorowa council assisted with participant recruitment, including providing advertising space in its newsletter and identifying key community figures from different backgrounds. among the twelve citizens who participated (table 1), two had adsl internet connections at home, seven used 3g wireless connections, two used the interim satellite network, and one person had no home internet access. the range of participants enabled broad insight into the different social and economic activities impacted by limited connectivity, and helped to uncover areas of priority for future action. 4 australasian journal of information systems freeman, park, middleton & allen 2016, vol 20, research on rural and remote ict broadband for socio-economic development pseudonym: gender: age: sector: maureen female 20s public sector barbara female 20s student adam male 30s public sector susan female 30s education alicia female 30s professional arthur male 30s professional trevor male 50s agriculture rachel female 50s student winona female 50s public sector carl male 60s agriculture mark male 60s local business louise female 60s agriculture table 1: participants’ demographic profile prior to the event, participants were given a ‘trigger question’ (the question used was: ‘how can high-speed internet benefit boorowa?’). they prepared up to three responding statements, and took turns to elaborate on their ideas in group discussion on the day. in total, 33 statements were offered by the participants. all participants had the opportunity to seek further clarification of the suggestions made and contemplate the issues involved. by taking turns at the onset of the discussion, this method encouraged the transformation of participants’ ideas through consideration of others’ opinions. a synthesis, discussion and voting process to determine priority areas resulted in the 33 initial statements being sorted and distilled by the participants into four categories: business development; education; emergency communication; and health. these areas emerged as the key community concerns arising from the inaccessibility and unreliability of broadband in boorowa but, according to the participants, are also where the greatest benefits from improved connectivity can be achieved. participants continued with more explicit discussion of the challenges and opportunities within the four categories. this enabled greater insight into their understandings, experiences and expectations of digital connectivity, and captured the everyday challenges and frustrations faced by these rural citizens. the four categories were used as the themes for analysis of the event’s transcript, in addition to direct connection challenges (that is, the availability and quality of participants’ internet access). the transcript was analysed by keyword coding within these themes (without the use of analysis software) to facilitate the organisation and retrieval of the most meaningful pieces of data (coffey & atkinson, 1996). there are benefits and limitations to this research method. some of the advantages of undertaking a qualitative approach include: gaining understanding into the experiences and perspectives of socially-situated participants (lindlof & taylor, 2002) to reveal nuances that would otherwise be masked within larger quantitative datasets (wolff & andrews, 2010); generating a large amount of data targeted to the research topic in a short timeframe (morgan, 1997); and enabling comparisons between participants’ viewpoints to facilitate greater insight into divergences and consistencies between the group (flanagan, 2008; morgan, 1997). while the dialogue included a diverse range of participants, the small sample group is by no means representative of the entire boorowa community or reflective of all rural australian areas. moreover, group discussions can be unnatural social settings, which can hinder dialogue and offer less details than other methods as they rely upon group interactions to shape the data gathered (morgan, 1997). with these limitations in mind, the research was intended only as a pilot study to identify areas impacted by rural connectivity that are in need of further in-depth research and analysis. the following section draws from participants’ insights into how limited connectivity shapes their everyday experiences, and to highlight how broadband-enabled opportunities may benefit the municipality’s socio-economic development. 5 australasian journal of information systems freeman, park, middleton & allen 2016, vol 20, research on rural and remote ict broadband for socio-economic development 4 findings participants’ overarching goal was to realise improved digital connectivity in boorowa so that local citizens would have opportunities for socio-economic advancement. as indicated above, an outcome from the research was the identification of four priority areas where development of high-speed rural broadband would bring the greatest benefits to the region: business development, education, emergency communication, and health. table 2 details participants’ statements on how improved connectivity could benefit the community within each of these themes. the table shows the level of support for each statement (participants voted for up to five priorities from among their initial statements; only those which received support are listed here). ten of the twelve participants voted for a single statement about community safety, which suggests it is a significant point of concern. also worth noting is that, as a category, statements relating to local business development received the most votes (23) overall, indicating that broadband access is increasingly important for rural businesses. category: participants’ statements: votes: category votes: business to ensure economic development for the local area. 7 23 ability for local businesses to interact with external markets and become competitive. 5 better access to technology for farmers on a day-to-day basis. 2 better connectivity would allow opportunities for more employment in a more diverse economy. 2 better for business, opportunities to work from home. 2 encourage businesses to come to boorowa. 2 being able to access essential services from remote locations, leading to less travel and more efficient use of time. 1 better mobile coverage for visitors to the town. 1 prospects for new and existing businesses – tourism. 1 education better [general] education opportunities. 6 12 young people having equitable access to education opportunities. 6 emergency communication community safety – effective communication with ambulance and rural fire service. 10 10 health health services require high-speed internet to communicate with colleagues, patients and participate in education. 5 5 table 2: thematic categories and statements receiving participant votes 4.1 connection challenges: the importance of access varying levels of access mean that digital gaps exist not only between rural and urban areas, but within rural regions: we’ve got the dual issue here where in the town there is good access and we want that access improved up to the level of urban communities. but aside from that, we’ve got the ‘no access at all’ issue. (adam) lack of, or unstable, access prevents rural citizens from capitalising on digital opportunities, even in comparison to residents in nearby townships. improving connectivity in townships to be on par with urban speeds would be welcomed, but a matter of urgency is obtaining at least some form of internet access for residents in more remote areas: “just to get better coverage in outlying areas, for mobile coverage, as well as internet. i think we must be rated as secondclass citizens, because nothing ever happens” (louise). arthur noted that rural internet does not need to be the 100mbps fibre-to-the-premises nbn once promised to urban citizens: 6 australasian journal of information systems freeman, park, middleton & allen 2016, vol 20, research on rural and remote ict broadband for socio-economic development so many points just come down to this digital divide and the broadband we’re talking about is not the broadband the cities are talking about. we’re just wanting some basic access. this idea of broadband, it’s not encapsulated as one big thing. while often associated with fibre-optics in urban spaces, the term ‘broadband’ rarely denotes fibre-optics in rural and remote areas. instead, these citizens are referring to high-speed connections through the 3g telecommunications network and satellite technologies. there was a strong desire for additional mobile towers to be built as these would solve dual issues of poor telecommunications coverage and internet availability in more remote areas: they are starting to realise i think that nbn optic-fibre everywhere is not the solution, particularly when mobile solutions now are at the top rung technically; the point where it’s providing as good a broadband as the cable… they should put towers up. (arthur) overall, the form of connectivity was less important to participants than its capacity to enable comparable digital opportunities to their urban counterparts: our needs are no different from people in metropolitan centres and i certainly can’t imagine anyone having a meeting like this in the middle of canberra or the middle of sydney about why you need these services. so i struggle with that. i think it’s a little bit insulting. (alicia) participants believed they should not be considered as separate to the rest of the australian population due to their geographical location. they felt there should be a duty to ensure access to broadband – as an essential service for participation in the digital economy and potential socio-economic prosperity – regardless of location or the increased cost of establishing infrastructure in remote areas: it’s about equal opportunities. everything in this world is online now… it’s just a god given right that we should be able to access it and be able to achieve what we can, as a community, as individuals. (winona) the following sections explore the categories participants identified as impacted by the inaccessibility and unreliability of local broadband. each section briefly outlines the role of connectivity in rural business, education, emergency communication and health services, respectively, before providing evidence from participants’ comments of the challenges faced when trying to digitally-enable these practices in the boorowa community. while the categories have been separated to aid analysis, the participants viewed them as interrelated and suggested that developments to any one area would bring benefits to the others. 4.2 business development agriculture is the primary industry in the boorowa region (abs, 2011b). farming is the heart of small businesses in rural australia, although the industry has a critical skill shortage mainly due to its aging workforce (lamb, 2012). access to and use of digital technologies is necessary to support rural farming growth and sustainability, with innovations already used in connected areas to help ensure resilience (lamb, 2012). sensor networks can monitor water trough levels, yields and soil moisture/temperature, gates can be fitted with auto alarms and (un)locking mechanisms, animals can be tracked through digital ear tags and gps, and drones can monitor sizeable areas (griffith et al., 2013; lamb, 2012). however, the potential of digital technologies to contribute to australian farming is largely unrealised, with uneven nbn developments and limited mobile telephony contributing factors (griffith et al., 2013; lamb, 2012). for example, mobile coverage may reach the bulk of the australian population but it covers significantly less of the landmass (reaching approximately 25%), and nbn plans to connect homes and businesses to fixed broadband overlooks its use in non-premised locations such as farmland (griffith et al., 2013; department of communications, 2013b). trevor, a local farmer, noted that “day-by-day there’s something new and something better and faster and quicker. i think if farmers can’t keep up with what’s going on in that, the rest of the world is going to go past us”. improved connectivity could offer local businesses 7 australasian journal of information systems freeman, park, middleton & allen 2016, vol 20, research on rural and remote ict broadband for socio-economic development opportunities to engage in wider markets and remain competitive. without such opportunities, participants felt that future business operations may be jeopardised: it [broadband] would increase the ability for businesses to interact with external markets. i know that a lot of market activity happens online now, particularly in the areas of auctions and that type of thing. if you’re not there in the market within the last minute or two of an auction, you’re going to miss out, basically. i think that’s a big issue that probably already is affecting, and will have more potential in the future to affect, our community. (adam) local business owners from other industries also spoke of the challenges of operating with insufficient internet access. in particular, participants mentioned issues using government websites for taxation purposes. without sufficient access, online taxation can be an inefficient task: “i spent four and a half hours on the internet the other day trying to log in to get my auskey so i could do my tax… i must be able to save at least a couple of hours out of it” (trevor). rachel highlighted psychological stress and potential compliance matters arising from the compulsory use of digital taxation systems: they [government] are the worst internet sites ever and they’re slow as all hell. i’ve got to do my tax stuff for our business and i’m terrified, because you can’t send it in by disk anymore, that i’m not going to be able to do it by the date because we’re going to have trouble with the internet. given the australian government’s recent commitment to implementing ‘digital first’ services by the end of 2017, there are evident challenges for individuals and businesses in rural areas trying to access such services (freeman & park, 2015). forcing small and medium enterprises to lodge their quarterly business activity statements and taxation information online does not accommodate those with barriers to connectivity. participants noted ways in which broadband could improve business operations including increased efficiency and competitiveness. for example, arthur, an it consultant, and susan, stated that advanced connectivity would allow remote access to computers in other locations, saving time currently spent in transit and costs for clients. rachel suggested that broadband may encourage internet-based businesses to the area as well as improve the operations of existing businesses, such as having the capacity to sell goods online. greater access may also encourage teleworkers to relocate to the area, particularly people who work in canberra but seek a rural lifestyle, and only travel when necessary: i think the ability to work from home allows people to get involved in the higher income careers – boorowa is sort of a low income area. so i would like to see more people on a high income, just because it would be great for the town. (carl) teleworking has been shown to have monetary value for both businesses and employees, with the former saving operational costs and the latter experiencing travel gains from reduced physical attendance (townsend et al., 2013). connectivity also holds promise for tourism and promotional purposes; participants suggested they could stream local events for marketing and interact with people outside of the community. as it stands, “tourists coming out are expecting to have the same access that they have [in urban centres]; to be able to be on holiday and still be able to communicate with the rest of the world” (maureen). boorowa’s local government and business association have been attempting to obtain greater connectivity for the region for several years. disheartened that their efforts have been unsuccessful, winona suggested (in jest) that perhaps the local area could capitalise on its lack of connectivity: i’ve just had an idea for economic development. we can actually say that we’re the dark ages and you know how people get fatigued with it stuff? they could come to boorowa and surrounds and be disconnected from the world. 8 australasian journal of information systems freeman, park, middleton & allen 2016, vol 20, research on rural and remote ict broadband for socio-economic development potential marketing prospects aside, participants believed that advanced connectivity holds value for the local economy, could offer greater opportunities for rural businesses, and contribute to sustainable regional development: strengthening and diversifying our economy and making sure that we’re sustainable into the future. by having the better connection, it’s going to allow that… diverse economy and being able to attract different industries, expanding industries and being more sustainable. (winona) participants suggested that advanced communication infrastructure is central to “ensure the viability of our community in the long-term” (alicia). 4.3 education the internet has become a necessary part of education in australia, and there are significant differences in digital technology use in educational settings (such as cloud computing and online lectures) and everyday life (such as social networking and emails) (waycott, bennett, kennedy, dalgarno & gray, 2010). lack of digital connectivity presents a learning challenge. li and ranieri (2013) examined digital imbalances between rural and urban school children aged between 10 and 14. they concluded that rural children have fewer opportunities to access computers and the internet at home or school, less support with new technologies, and decreased frequencies of internet activities. they also found that lower internet self-efficacy amongst rural students correlated directly with poor academic performance. when learnt in childhood, the capacity to use technology for independent learning forms a necessary precondition for many higher education and workplace opportunities (katsinas & moeck, 2002; townsend et al., 2013). lack of connectivity also shapes technology integration into school learning, with tools such as electronic white boards, desktops and tablets significantly less likely to be used in rural areas (wang, 2013). boorowa’s local kindergarten to year 12 (k-12) school has altered its curriculum so students are not required to access the internet from home: my original assumption was that we had students within our local government area that were being disadvantaged by the fact that they didn’t have access to the internet and couldn’t do the research that, for example, the kids in the town could – just by lack of access. what i found out was our school here actually doesn’t set that expectation of students, that they will have any access at all. so it’s not only the students outside of the town area. the policies of the school are structured so that everyone is equally disadvantaged, which is the way they’re structuring things here in our community. but you compare that to everywhere else and we’re way behind. (adam) altering curriculum so students are not required to access the internet for homework impacts children’s learning development as well as their social and technical capacities (katsinas & moeck, 2002). these areas are, of course, encouraged within the school; the choice to restrict options such as online assessments is something the school undertakes to ensure fair and equitable student development. the k-12 school is also considering a ‘bring your own device’ program as occurs in areas with advanced connectivity, however many participants expressed concerns that the existing network access will be inadequate to cater for this increased activity. commenting on one of the other smaller and more remote primary schools in the region (with eight students enrolled in 2014), alicia indicated that “they have a smart board at rugby public but they don’t have [it connected to the] internet.” all of the participants felt that local children have fewer educational opportunities than urban children and are fundamentally disadvantaged because of poor connectivity. moreover, structuring education so that ‘everyone is equally disadvantaged’ may bring other long-term adverse ramifications to the region, such as hindering boorowa’s youth when entering the workforce or higher education. winona stated: i keep on hearing that kids who are going to uni[versity] from boorowa actually either are deferring or dropping out after their first year… i can’t believe that they would miss 9 australasian journal of information systems freeman, park, middleton & allen 2016, vol 20, research on rural and remote ict broadband for socio-economic development an opportunity like that. but you start to wonder now why that’s happening. why are they not adapting? in boorowa’s case, the comment above may be an indicative consequence from rural students’ inexperience of access, technical competencies, and social development in the online context. participants spoke of the way that higher education is rapidly changing to integrate technological advances but rural internet infrastructure remains largely stagnant. the capacity of local infrastructure makes learning online no small feat: “working on and downloading course materials while you’re not at university… can take a very long time, which is very frustrating. or it [the connection] will just fall out and then you’ve got to start again” (rachel). professional development opportunities are also difficult to undertake online: we have a small number of staff who try to participate in professional development opportunities online and access to professional forums to further their knowledge. they have a lot of trouble with that because of the speed of the internet and [they] aren’t able to do it. so we therefore pay staff members to attend courses in places. it’s a lot more expensive for small businesses who are trying to access professional help. (susan) here, insufficient connectivity is tied with unnecessary financial expenses for rural businesses. if employers are unable to fund travel for education, then an indirect consequence is that poor internet access contributes to the gap in rural skills. this is problematic as it is often rural citizens who require frequent and diverse re-skilling to remain viable employees in areas suffering income loss in the digital economy (townsend et al., 2013). 4.4 emergency communication emergency situations require effective information and communication technology integration for real-time organisation, management and coordination of mobile public safety response efforts, often across multiple governance agencies (wamba, edwards & sharma, 2012; baldini, karanasios, allen & vergari, 2014). the new south wales state emergency service, for example, equips all of its staff with smartphones in an effort to support field operations (wamba et al., 2012). the use of digital interactive data, sensor networks, and location services aid effective information exchange, situational awareness and the coordination and mobility of response efforts. however, rural areas often lack the required bandwidth or mobile network coverage to facilitate use of necessary response tools including interactive voice and data communications (baldini et al., 2014). often it is the geographical terrain of rural areas that presents a challenge to effective emergency communication (nosketurner, tacchi, horst & papoutsaki, 2014) and limited connectivity in rural australia has been shown to create significant delays in information-sharing regarding emergencies. martin and rice (2012), for example, illustrate that the lack of high-speed infrastructure between alice springs and canberra has resulted in delays of more than a day in the release of emergency information, and the infrastructure that is in place is insufficiently robust to withstand adverse events. subsequently, radio and word-of-mouth remain the most useful forms of communication during emergencies in rural areas (ryan, 2013). boorowa’s current network capacity presents community safety concerns, both in everyday isolated incidents and during times of crisis when emergency service communication and information is critical. many participants were anxious about the substantial geographical areas that are not covered by either fixed broadband or mobile coverage, reporting their inability to phone for assistance from farmland and other areas outside of townships including lachlan valley way, a state highway connecting the municipality to the hume (interstate) freeway, and noting that mobile services through providers other than telstra often fail within the municipality’s townships. emergency services such as the ambulance and fire brigade are often unable to be contacted (either by individuals or dispatch stations/hospitals) while en route to situations, which presents challenges both for individuals requiring assistance and service personnel: we had a local emergency group meeting on during the week and the local ambulance rep, his biggest gripe was trouble with providing their service once they got in that dark 10 australasian journal of information systems freeman, park, middleton & allen 2016, vol 20, research on rural and remote ict broadband for socio-economic development [unconnected] area down to the north and the north-east… it could be a life threatening problem… [but] if there’s no service [coverage] they’re basically back to the basics with the ambulance service. (trevor) alicia spoke of an instance when she called for an ambulance and decided it would be quicker to meet the ambulance half way between the hospital and her home. however, the ambulance was uncontactable for more than half of its 40 minute journey. susan suggested that: “what’s more concerning too is that the ambulance drivers were saying they’re even having trouble with satellite phones… the satellite’s getting so full that we’re not even getting that now”. trevor, who is a local captain in the rural fire service (rfs), commented that the “rfs are talking about all these great systems of communication they’ve got now, but not many of them work in an area where you don’t have mobile or internet coverage”. he indicated that the local brigade is trialling digital trackers to help identify the location of personnel during fires, with their locations synced to his ipad. but poor coverage means he can lose track of people’s locations, which impedes the coordination of emergency service personnel during times of crisis. for boorowa citizens, it’s not only the contacting in times of emergency, but it’s accessing information. so okay, you know there are fires in the area, i can’t get online to see what the latest reports are or whatever, so i can’t contact anyone as well. (alicia) participants’ comments indicated an irony surrounding current emergency communication initiatives. for example, the australian government has contributed $59.5 million towards the establishment of an emergency alert system that works through both landline and mobile services (australian government, 2015). however, landline services in areas like boorowa are often unreliable as repairs and upgrades to older copper networks are suspended due to anticipated nbn developments (park et al., 2015). as carl noted, “the copper wire’s had its day. every time it rains it does stop [working]”. in terms of mobile alerts, text messages sent to residents in areas under threat are reliant upon sufficient coverage. the government acknowledges that messages will not be received if users are in areas without mobile coverage (www.emergencyalert.gov.au). participants were frustrated that many of “the community safety issues could be solved by more phone towers” (mark). 4.5 health rural australia is faced with a significant shortage in medical facilities and professionals, which results in rural citizens being admitted to hospital for preventable conditions more often than urban citizens (jang-jaccard et al., 2014). lack of access to appropriate rural health services contributes to decreased life spans and a doubled mortality rate for rural residents under 65 years of age (jang-jaccard et al., 2014). moreover, people living in dispersed areas often face issues of social isolation that can affect their wellbeing, which is exacerbated through poor access to communication networks (hale et al., 2010; townsend et al., 2013). digital infrastructure and initiatives can offer access to services for rural populations marginalised from healthcare systems (jones et al., 2012). hale et al. (2010) outline four types of healthrelated practices enabled by the internet: finding health information; accessing healthcare resources (including communication with physicians/providers); purchasing medical/health products; and increased contact with social support networks. boorowa’s poor connectivity inhibits basic health practices such as contact between patients, physicians and colleagues. alicia, a medical doctor, argued: i think [rural] health services would benefit enormously from effective mobile and internet services, particularly to communicate with their patients. everyone relies on mobile phones these days; it’s part of our culture and the way we do things. so i think that’s a big thing. communicating with colleagues on our internet service currently, i certainly can’t even receive a ct scan via an email because it’s just too big to download. alicia relocated to boorowa from sydney, and it would be difficult for her to set up a local practice to service boorowa and its surrounds until connectivity is improved. a better 11 australasian journal of information systems freeman, park, middleton & allen 2016, vol 20, research on rural and remote ict broadband for socio-economic development connected local clinic would offer rural patients greater access to essential health services as well as the capacity “to be able to go to the [local] doctor and hook up with your doctor in canberra or sydney and they’d be able to have a conversation with all the files and that in front of them” (barbara). in the meantime, it is common for rural patients to travel more than 100km for any type of specialist advice (adam). participants stated that they are unable to use information and communication technologies for social opportunities, including networking (social media), accessing entertainment (itunes; iview), and video contact with distant family members (skype). susan, who has an adsl connection, stated that, “my dad is in the states... for us to have access and be able to talk to people on the other side of the world, the software [skype] doesn’t always run so well”. local citizens using the 3g or satellite services are even less likely to be able to use skype due to its reliance on high-speed and bandwidth. another participant suggested that poor connectivity impacts the development of necessary social skills amongst rural youth: a lot of our kids don’t have access to the social opportunities that everyone else does… with the younger generations coming through, it becomes more noticeable – and i guess probably more acute – the effect of not having that social interaction. most people in this room would have grown up in an environment where the way they socialise was very different. whereas young kids now, it’s completely different. (adam) poor social interaction can affect mental health. rural social and mental health services are already limited, which results in decreased help-seeking practices in rural areas (30.8% in comparison to 36.9% in cities) and increased rates of self-harm and suicide – suicide, for example, is 66% higher in australian country areas than in cities (national rural health alliance, 2015). while the internet can aid access to healthcare, information, and social support, disparities in digital access in rural areas inhibit potential healthcare improvements. 5 discussion as was the case with previous communication infrastructure developments (goggin, 2002), poor digital connectivity in rural australia may create the emergence of a technological underclass of individuals who are left behind, with less privileged and vulnerable groups facing further marginalisation (notara, 2011; eardley et al., 2009). the experiences of the boorowa community indicate the extensive socio-economic impacts that limited or no access to broadband can have in rural areas. significant concerns have been raised in relation to the need for high-speed connectivity to stimulate rural economies through new business possibilities, address disparities in rural education and learning opportunities, prevent breakdowns in communication and information sharing during emergency situations, and empower rural citizens through improved access to health services. there was apprehension amongst participants for the long-term negative ramifications that insufficient broadband access may bring the community in these areas, such as boorowa’s youth being disadvantaged when entering higher education or the workforce. there is an urgent need to prioritise broadband infrastructure developments for rural australia as poor connectivity prevents many rural communities from capitalising on digital opportunities. in the few years of the nbn’s existence, it has already created, reinforced and extended socio-economic disparities between those with access to it and those without (alizadeh, 2013), and future developments could again reinforce digital disadvantages. the observations in this article therefore confirm that conditions of rurality – including the geographic and socio-economic contexts of communities – require greater consideration in planning policies (morrison, lane & hibbard, 2015; broadband for the bush, 2013). to help facilitate digital opportunities in rural communities, infrastructure priority should be given to those who remain unconnected. while the department of communications (2013a) requested nbn co. prioritise developments in underserved regions, the company was only required to ‘consider’ this request in its rollout, which can ultimately be undertaken in the most logistically and commercially feasible manner. the 2015 regional telecommunications 12 australasian journal of information systems freeman, park, middleton & allen 2016, vol 20, research on rural and remote ict broadband for socio-economic development review committee recommends that the rollout to areas with poor or no coverage be monitored carefully to allow informed assessment of persistent gaps and development of timely responses. for areas where deployment is less feasible, it would be beneficial to enable greater flexibility in both policy and infrastructure developments. localised approaches to rural digital inclusion have effectively reduced the digital divide and associated socio-economic disparities amongst marginalised and disempowered groups in other countries. for example, walterova and tveit’s (2012) research into europe’s digital local agenda demonstrates that policy frameworks can be strategically translated to account for specific local conditions such as geographical contexts, socio-economic circumstances, and different market environments. infrastructure developments can be tailored to suit particular areas and community needs to ensure effectiveness, but this requires policy approaches and development plans that can be adapted and targeted to the contexts of place-based communities to facilitate greater socio-economic inclusion (hogan & young, 2013; alizadeh, 2013). such an approach privileges the development of “self-sufficient and socio-economically viable regional communities” (hogan & young, 2013, p326), allowing infrastructure developments to aid the sustainability of rural locales. there are many alternative approaches that have been proposed in this area. amongst others, these include: the development of custom user-generated or wireless community networks (middleton & bryne, 2010; goggin, 2007); local public broadband as a civic infrastructure (mcshane, 2013); extending universal service obligations to include broadband (morsillo, 2013); increased subsidies from metropolitan areas to rural regions (dobson, jackson & gengatharen, 2011); greater public investment (li, 2012); and public-private partnerships (given, 2010; falch & henten, 2010). the appropriateness of different forms of broadband connection will vary by locales and users. the community participants in this research suggested that building additional mobile towers would be the most suitable for the boorowa municipality – they would help overcome the dual issue of poor telephony coverage as well as broadband access, would be useful for both residents and visitors, and they offer decreased costs in comparison to satellite coverage. this reflects findings of the two most recent regional telecommunications reviews (2012, 2015), which concluded that mobile coverage was the most pressing issue for rural australia. other suggestions were put forward by the participants, such as fibre-optics to nodes with copper conduits to homes or communal wireless services, but participants also recognised the limitations of these types of approaches including longer installation times, higher development and maintenance costs, and reliance upon an already failing copper network. even with reduced speeds, mobile wireless was preferred so that all residents could benefit from some form of connectivity, and participants suggested municipal sites where additional towers would have the greatest reach. the challenge now lies in identifying the best and fastest way in which mobile towers can be built. it is here where adaptation to current policies is needed, and government intervention would be useful to encourage the development of alternative local infrastructure models and funding arrangements for rural australia. 6 conclusion this article explores the impact of poor broadband connectivity on socio-economic opportunities in the rural municipality of boorowa. broadband access in rural australia is significantly limited in terms of both availability and quality (tiwari et al., 2015). to reiterate the point made earlier by the research participant dubbed arthur: “the broadband we’re talking about is not the broadband the cities are talking about. we’re just wanting some basic access.” the boorowa case demonstrates that divides in access exist not only between rural and urban areas but within rural regions, with connectivity discrepancies between people living in townships and those in more remote locations. insufficient access to high-speed broadband limits boorowa’s socio-economic development, with significant short and long-term concerns in the areas of business development, education, emergency communication, and health. local 13 australasian journal of information systems freeman, park, middleton & allen 2016, vol 20, research on rural and remote ict broadband for socio-economic development residents already face challenges in these areas, but they are also where the greatest benefits from improved connectivity can be realised. townsend et al. (2013) describe this as the paradox of rural broadband – insufficient access is a major socio-economic problem faced by rural communities but, at the same time, it is rural communities that can gain the greatest socio-economic benefits from broadband. ironically, broadband policies often identify rural communities as exemplars for why developments are needed and who can benefit the most, while deployment is prioritised in urban spaces (townsend et al., 2013). as digital technologies are increasingly embedded within socioeconomic practices, issues will continue to arise for those with poor connectivity. first and foremost, government resources should be directed towards providing access for disenfranchised citizens who would otherwise be unable to participate in the digital economy. there is an urgent need for greater consideration of the conditions of rurality in nbn developments so that rural citizens are not marginalised (alizadeh, 2013). the only way to narrow the rural-urban digital divide is to start with infrastructure developments in the most underserved regions. privileging fibre-optic connections for large proportions of the population may fail rural communities by increasing disparities in broadband availability and quality (townsend et al., 2013). as more everyday activities transition into online formats, rural inequities will be amplified. the participants in this research sought comparable digital opportunities to their urban counterparts so they are not disadvantaged. the government should ensure all citizens have access to essential services and can participate in the digital economy (broadband for the bush, 2013; townsend et al., 2013). this will likely require strategic adaptations to current nbn policy and planning frameworks by accounting for the socio-economic contexts of rural communities to help ensure their sustainability. references 2011–12 regional telecommunications independent review committee. (2012) 2011–12 regional telecommunications review – regional communications: empowering digital communities, commonwealth of australia, canberra. alizadeh, t. (2013) “towards the socio-economic patterns of the national broadband network rollout in australia”, in ruming, k., randolph, b. & gurran, n. (eds.), state of australian cities conference: refereed proceedings, soac research network, sydney, http://www98.griffith.edu.au/dspace/bitstream/handle/10072/58738/90792_1.pdf?s equence=1 australian bureau of statistics (abs). (2011a) 2011 census quickstats: boorowa, abs, canberra, http://www.censusdata.abs.gov.au/census_services/getproduct/census/2011/quicksta t/lga11050?opendocument&navpos=220 australian bureau of statistics (abs). (2011b) national regional profile: boorowa (local government area) 2007-2011, abs, canberra, http://www.abs.gov.au/ausstats/abs@nrp.nsf/lookup/lga11050main+features12007 -2011 australian bureau of statistics (abs). (2013) national regional profile, 2007-2011, abs, canberra, http://www.abs.gov.au/ausstats/abs@.nsf/detailspage/1379.0.55.0012007%20to% 202011?opendocument australian communications and media authority (acma). (2014) the economic impacts of mobile broadband on the australian economy, from 2006 to 2013, centre for international economics, canberra. australian government. (2015) emergency alert, attorney general’s department, barton, act, http://www.em.gov.au/emergency-warnings/pages/emergencyalert.aspx 14 australasian journal of information systems freeman, park, middleton & allen 2016, vol 20, research on rural and remote ict broadband for socio-economic development baldini, g., karanasios, s., allen, d. & vergari, f. (2014) “survey of wireless communication technologies for public safety”, ieee communications surveys & tutorials, 16(2):619641. broadband for the bush. (2013) better telecommunications for rural and remote australians, desert knowledge australia, http://broadbandforthebush.com.au/wpcontent/uploads/2013/05/broadband-for-the-bush-alliance-policy-june-2013.pdf coffey, a. & atkinson, p. (1996) making sense of qualitative data: complementary research strategies, sage, thousand oaks, california. department of broadband, communications and the digital economy (dbcde). (2011) national digital economy strategy: leveraging the national broadband network to drive australia’s digital productivity, dbcde, canberra. department of communications. (2013a) broadband availability and quality report, department of communications, canberra, http://www.communications.gov.au/__data/assets/pdf_file/0018/212535/broadband _availability_and_quality_report.pdf department of communications. (2013b) mobile coverage programme – discussion paper, department of communications, canberra, http://www.communications.gov.au/__data/assets/pdf_file/0003/204069/mobile_c overage_programme_-_discussion_paper.pdf dobson, p., jackson, p. & gengatharen, d. (2011) “examining rural adoption of broadband – critical realist perspectives”, the pacific asia conference on information systems proceedings, ais electronic library, qut, http://aisel.aisnet.org/pacis2011/56 eardley, t., bruce, j., & goggin, g. (2009) telecommunications and community wellbeing: a review of the literature on access and affordability for low-income and disadvantaged groups (sprc report 9/09, prepared for the telstra low income measures assessment committee (limac)), social policy research centre, university of new south wales, sydney. falch, m. & henten, a. (2010) “public private partnerships as a tool for stimulating investments in broadband”, telecommunications policy, 34(9):496-504. flanagan, t.r. (2008) “scripting a collaborative narrative: an approach for spanning boundaries”, design management review, 19(3):80-86. freeman, j. & park, s. (2015) “rural realities: digital communication challenges for rural australian local governments”, transforming government: people, process and policy, 9(4):465-479. given, j. (2010) “take your partners: public private interplay in australian and new zealand plans for next generation broadband”, telecommunications policy, 34(9):540-549. goggin, g. (2002) “rural line of flight: telecommunications and post-metro dreaming”, transformations, 2:1-13. goggin, g. (2007) “an australian wireless common?”, media international australia, 125:118-130 griffith, c., heydon, g., lamb, d., lefort, l.d., taylor, k, trotter, m. & wark, t. (2013) smart farming: leveraging the impact of broadband and the digital economy, csiro and university of new england, nsw. hale, t.m., cotton, s.r., drentea, p. & goldner, m. (2010) “rural-urban differences in general and health-related internet use”, american behavioural scientist, 53(9):13041325. hogan, a. & young, m. (2013) “visioning a future for rural and regional australia”, cambridge journal of regions, economy and society, 6(2):319-330. 15 australasian journal of information systems freeman, park, middleton & allen 2016, vol 20, research on rural and remote ict broadband for socio-economic development jang-jaccard, j., nepal, s., alem, l. & li, j. (2014) “barriers for delivering telehealth in rural australia: a review based on australian trials and studies”, telemedicine and e-health, 20(5):496-504. jones, c.r., cardoso, r.b., hüttner, e., oliveira, h.w., dos santos, m.a., lopes, m.h.i & russomano, t. (2012) “towards designing for equity: active citizen participation in ehealth”, transforming government: people, process and policy, 6(4):333-344. katsinas, s.g. & moeck, p. (2002) “the digital divide and rural community colleges: problems and prospects”, community college journal of research and practice, 26(3):207-224. katz, r. & suter, s. (2009) estimating the economic impact of the broadband stimulus plan, columbia institute for tele-information, new york. lamb, d. (2012) “thoughts on capturing opportunities and overcoming obstacles for australian agriculture related to electronic agriculture”, in yunusa, i. (ed.), proceedings of the 16th australian agronomy conference, armidale, nsw, http://www.regional.org.au/au/asa/2012/plenary/8549_lamb.htm li, g. (2012) “the return of public investment in telecommunications: assessing the early challenges of the national broadband network policy in australia”, computer law & security review, 28(2):220-230. li, y. & ranieri, m. (2013) “educational and social correlates of the digital divide for rural and urban children: a study on primary school students in a provincial city of china”, computers and education, 60(1):197-209. lindlof, t.r. & taylor, b.c. (2002) qualitative communication research methods (2nd edition), sage, thousand oaks, california. martin, n. & rice, j. (2012) “emergency communications and warning systems: determining critical capacities in the australian context”, disaster prevention and management: an international journal, 21(5):529-540. mazzarol, t. (2015) “smes engagement with e-commerce, e-business and e-marketing”, small enterprise research, 22(1):79-90. mcshane, i. (2013) “local public broadband – the missing link in australia’s broadband debate”, proceedings of the 3rd national local government researchers’ forum (pp.109-119), uts epress, adelaide. middleton, c.a. & bryne, a. (2011) “an exploration of user-generated wireless broadband infrastructures in digital cities”, telematics and informatics, 28(3):163-175. middleton, c. & park, s. (2014) “waiting for the national broadband network: challenges of connectivity in rural australia”, 20th its biennial conference – the net and the internet: emerging markets and policy, international telecommunications society, rio de janeiro, http://econstor.eu/bitstream/10419/106893/1/816841047.pdf morgan, d.l. (1997) focus groups as qualitative research (2nd edition), sage, thousand oaks, california. morsillo, r. (2013) “a shared value approach to digital inclusion”, in freeman, j. (ed.), proceedings of the emerging issues in communication research and policy conference, 2013 – refereed papers (pp. 1-11), news & media research centre, university of canberra, canberra. morrison, t.h., lane, m.b. & hibbard, m. (2015) “planning, governance and rural futures in australia and the usa: revisiting the case for rural regional planning”, journal of environmental planning and management, 58(9):1601-1616. national rural health alliance. (2015) mental health in rural and remote australia, national rural health alliance, deakin west, act, 16 australasian journal of information systems freeman, park, middleton & allen 2016, vol 20, research on rural and remote ict broadband for socio-economic development http://ruralhealth.org.au/sites/default/files/publications/fact-sheet-mental-health2015.pdf noske-turner, j., tacchi, j., horst, h. & papoutsaki, e. (2014) “locating disaster communication in changing communicative ecologies across the pacific”, australian journal of telecommunications and the digital economy, 2(4):66.1-66.16. notara, d. (2011) another barrier? regional consumers, not-for-profit organisations, and the nbn in the northern rivers region, australian communications consumer action network, sydney. park, s., freeman, j., middleton, c., allen, m., eckermann, r. & everson, r. (2015) “the multilayers of digital exclusion in rural australia”, proceedings of the 48th annual hawaii international conference on system sciences (pp. 3631-3640), computer society press, kauai, hawaii. prieger, j.e. (2013) “the broadband digital divide and the economic benefits of mobile broadband for rural areas”, telecommunications policy, 37(6-7):483-502. regional telecommunications independent review committee. (2015) 2015 regional telecommunications review, commonwealth of australia, canberra. ryan, b. (2013) “information seeking in a flood”, disaster prevention and management: an international journal, 22(3):229-242. steele, r. & lo, a. (2013) “telehealth and ubiquitous computing for bandwidth-constrained rural and remote areas”, personal and ubiquitous computing, 17(3):533-543. tiwari, s., lane, m. & alam, k. (2015) “the challenges and opportunities of delivering wireless high speed broadband services in rural and remote australia: a case study of western downs region (wdr)”, the 26th australasian conference on information systems proceedings, university of south australia, adelaide, https://acis2015.unisa.edu.au/wp-content/uploads/2015/11/acis_2015_paper_205.pdf townsend, l., sathiaseelan, a., fairhurst, g. & wallace, c. (2013) “enhanced broadband access as a solution to the social and economic problems of the rural digital divide”, local economy, 28(6):580-595. walterova, i. & tveit, l. (2012) “digital local agenda: bridging the digital divide”, transforming government: people, process and policy, 8(4):345-357. wamba, s.f., edwards, a. & sharma, r. (2012) “big data as a strategic enabler of superior emergency service management: lessons from the new south wales state emergency service”, presented at society for information management and mis quarterly executive pre-icis 2012 sim academic workshop (pp. 1-3), university of wollongong, nsw, http://ro.uow.edu.au/cgi/viewcontent.cgi?article=1153&context=buspapers wang, p. (2013) “examining the digital divide between rural and urban schools: technology availability, teachers’ integration level and students’ perception”, journal of curriculum and teaching, 2(2):127-139. waycott, j., bennett, s., kennedy, g., dalgarno, b. & gray, k. (2010) “digital divides? student and staff perceptions of information and communication technologies”, computers & education, 54(4):1202-1211. wolff, r.s. & andrews, e. (2010) “broadband access, citizen enfranchisement, and telecommunications services in rural and remote areas: a report from the american frontier”, ieee communications magazine, 48(5):128-135. copyright: © 2016 freeman, park, middleton & allen. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 17 http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems freeman, park, middleton & allen 2016, vol 20, research on rural and remote ict broadband for socio-economic development australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. 18 http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 literature review 3 case study background and methodology 4 findings 4.1 connection challenges: the importance of access 4.2 business development 4.3 education 4.4 emergency communication 4.5 health 5 discussion 6 conclusion paper title (paper title style) australasian journal of information systems burmeister 2020, vol 24, post publication review effects of misuse of personal details on older people 1 watson, j., lacey, d., kerr, d., salmon, p., & goode, n. (2019). understanding the effects of compromise and misuse of personal details on older people. australasian journal of information systems, 23. https://doi.org/10.3127/ajis.v23i0.1721 review this post publication review discusses the article by watson, lacey, kerr, salmon, and goode (2019). that article raises important considerations for older people when they interact online. it is a thorough review of the non-financial considerations related to cyber safety for older people in australia. the authors raise a few issues to consider for future research, but of course like everyone else in the world, they could not foresee the covid-19 pandemic. covid-19 has forced older people who previously had shied away from going online, into the online environment. they have been the most vulnerable in this crisis and the self-isolation has affected them more than other segments of the australian population. particularly for older people new to online interaction, they are potentially even more at risk of falling victim to cybercriminals. the australian computer society recently warned all australians that there has been a significant increase in cyber scams and criminal activity since the start of covid19 related self-isolation. i am interested therefore to explore with the authors of this excellent article what the implications of their study might be for vulnerable older australians, struggling with online interactions, often for the first time, and how they can best avoid becoming victims of criminal activities online. i know from my own research that legislated protections can be difficult to enforce (burmeister, islam, dayhew, & crichton, 2015; thomas, burmeister, & low, 2019), and that certain groupings of people can be more vulnerable than others, due to societal discriminations that pre-exist, such as based on gender identity, culture, or chronic illness (bennett et al., 2017; burmeister & kreps, 2018; poulsen & burmeister, 2019; sayers, cleary, hunt, & burmeister, 2017; teipel et al., 2016). what might the authors of the (watson et al., 2019) article propose to help particularly vulnerable segments of the older population, as opposed to treating all older people equally? particularly for older people venturing into the online world for the first time due to covid-19, how can we better educate them to ensure safe online interactions? some of the impacts discussed in the (watson et al., 2019) article are about older people choosing not to engage online. however, that is not really an option in the covid-19 environment. instead, older people in particular are forced to interact, socially, financially an in all other ways. furthermore, the authors raise the issue of the impacts on the mental health of older people, and arguably it is not just their mental health but also their social and emotional wellbeing (burmeister et al., 2019; sum, mathews, pourghasem, & hughes, 2008), given that a cessation of online activity would further isolate them in the present pandemic era. what can be done by older people themselves and wider society to overcome the negative impacts of cyber-criminal activities? oliver burmeister charles sturt university oburmeister@csu.edu.au australasian journal of information systems burmeister 2020, vol 24, post publication review effects of misuse of personal details on older people 2 author response the authors would like to thank the reviewer for his comments on our article. we agree that covid-19 has implications for the elderly’s use of the internet and that vulnerable older australians are particularly at risk. this is an important research area and there is a pressing need to educate vulnerable sections of society on how to avoid becoming victims of criminal activities in online environments. unfortunately, this is beyond the scope of our current work which was undertaken as part of a phd program focusing on the structure, function and constraints of the system with which victims interact following an incident of identity compromise and misuse. consequently, we are not in a position to make recommendations about how older adults can protect themselves online. judy watson centre for human factors and sociotechnical systems university of the sunshine coast jwatson@usc.edu.au david lacey university of the sunshine coast don kerr university of the sunshine coast paul salmon centre for human factors and sociotechnical systems university of the sunshine coast natassia goode worksafe victoria, melbourne centre for human factors and social technical systems references bennett, e., berry, k., emeto, t. i., burmeister, o. k., young, j., & shields, l. (2017). attitudes to lesbian, gay, bisexual and transgender parents seeking health care for their children in two early parenting services in australia. journal of clinical nursing, 26(7-8), 1021– 1030. doi:10.1111/jocn.13595 burmeister, o. k., islam, m. z., dayhew, m., & crichton, m. (2015). enhancing client welfare through better communication of private mental health data between rural service providers. australasian journal of information systems, 19, 1-14. doi:10.3127/ajis.v19i0.1206 burmeister, o. k., & kreps, d. (2018). power influences upon technology design for age-related cognitive decline using the vsd framework. ethics and information technology, 20(3). burmeister, o. k., ritchie, d., devitt, a., chia, e., dresser, g., & roberts, r. (2019). the impact of telehealth technology on user perception of wellbeing and social functioning, and the implications for service providers. australasian journal of information systems, 23. doi:10.3127/ajis.v23i0.1501 poulsen, a., & burmeister, o. k. (2019). overcoming carer shortages with care robots: dynamic value trade-offs in run-time. australasian journal of information systems, 23. doi:10.3127/ajis.v23i0.1688 australasian journal of information systems burmeister 2020, vol 24, post publication review effects of misuse of personal details on older people 3 sayers, j. m., cleary, m., hunt, g. e., & burmeister, o. k. (2017). service and infrastructure needs to support recovery programmes for indigenous community mental health consumers. international journal of mental health nursing, 26, 142–150. doi:10.1111/inm.12287 sum, s., mathews, m. r., pourghasem, m., & hughes, i. (2008). internet technology and social capital: how the internet affects senior's social capital and wellbeing. journal of computermediated communication, 14, 202-220. teipel, s., babiloni, c., hoey, j., kaye, j., kirste, t., & burmeister, o. k. (2016). information and communication technology solutions for outdoor navigation in dementia. alzheimer's & dementia: the journal of the alzheimer's association, 12(6), 695-707. doi:10.1016/j.jalz.2015.11.003 thomas, g., burmeister, o., & low, g. (2019). the importance of ethical conduct by penetration testers in the age of breach disclosure laws. australasian journal of information systems, 23. doi:10.3127/ajis.v23i0.1867 watson, j., lacey, d., kerr, d., salmon, p., & goode, n. (2019). understanding the effects of compromise and misuse of personal details on older people. australasian journal of information systems, 23(0). doi:10.3127/ajis.v23i0.1721 copyright: © 2020 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v24i0.2839 http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems goyal, pillai & chauhan 2021, vol 25, research article e-governance using mobile applications 1 e-governance using mobile applications: a case study of india during the covid-19 pandemic sandeep goyal thapar institute of engineering and technology, india sandy2u@gmail.com anandan pillai publicis media, india sumedha chauhan o. p. jindal global university, india abstract healthcare initiatives backed by electronic-governance (e-governance) principles have contributed well to the extant literature and practice. governments and healthcare systems across the world were taken aback by the swamping impact of the covid-19 pandemic. however, they reacted quickly by developing contact-tracing mobile applications (apps) for creating awareness, providing information about various healthcare initiatives, and helping citizens to use the required information in case of emergency. the major challenge was to develop such e-governance interventions in a short time and ensure their quick adoption among the masses. hence, it is worthwhile to investigate the factors leading to the adoption of such e-governance initiatives, especially in the context of a widespread pandemic situation. the present study is an attempt to analyze the factors driving the intention to use contact tracing mobile apps launched by governments globally during the covid-19 pandemic. we have conducted the study in the context of india, where the government launched a community-driven contact tracing mobile app for its citizens during the covid-19 pandemic in april 2020. the study adopted an empirical approach to test how epistemic value, convenience value, conditional value, functional value, and privacy concerns influenced the intention to use this approach. the study found that intention to use such an app was positively influenced by functional value, which in turn was positively influenced by convenience and conditional values. it suggests that the convenience of using the app, perceived seriousness of the pandemic (i.e., conditional value), and utilitarian benefits (i.e., functional value) of the contact-tracing mobile app enhanced its acceptance. however, its novelty (i.e., epistemic value) and privacy concerns are not significant predictors of intention to use. the study recommends that the government should place more emphasis on improving the functional value which is driven by convenience and context-specific features to push the use of an e-governance initiative during the crisis. keywords: intention to use; covid-19; pandemic; contact tracing mobile application; theory of consumption values; technology adoption; e-governance 1 introduction the outbreak of the covid-19 pandemic resulted in a major risk to the health and well-being of individuals across the globe (pan & zhang, 2020). the adverse impact of the covid-19 pandemic was expected to be relatively more severe for densely populated developing nations like india, china, brazil, etc. developing countries are characterized by high population density, heterogeneity in terms of socio-economic level, and under-resourced health australasian journal of information systems goyal, pillai & chauhan 2021, vol 25, research article e-governance using mobile applications 2 infrastructure in terms of hospital beds, doctors, nurses, diagnostics, and testing facilities for their population (chib, van velthoven, michelle, & car, 2015). for example, india is one of the leading developing nations, comprising 17.7% of the global population, and was expected to face the worst impact during the covid-19 pandemic (worldometer, 2021; ghosh, nundy, & mallick, 2020). therefore, it became critical for the government of india (goi) to take appropriate measures while dealing with under-resourced healthcare infrastructure issues, such as the shortage of hospital beds, testing kits and ventilators during the covid-19 pandemic (iqbal & dar, 2020). to deal with the pandemic, several countries launched the community-driven contact tracing mobile application (app) to trace the covid-19 infected people and their contact history. the use of the contact tracing mobile app for real-time tracking of the infected people and their interactions helped these countries in isolating and facilitating the fast recovery of the infected people as well as parallel profiling and recovery of contacts who interacted with these people during the onset of coronavirus symptoms. examples of covid-19 community-driven contact tracing mobile apps used by different countries include immuni in italy (immuni website, 2021), covidsafe in australia, stopcovid in france, corona app in germany, mytrace in malaysia, covidradar in mexico, protego in poland, trace together in singapore, swiss contact tracing app in switzerland, hayat eve sigar in turkey, nhs covid-19 app in the uk (o'neill, ryan-mosley, & johnson, 2020). the scarcity of resources and not-so-robust healthcare infrastructure, as well as the internet and telecom setup in developing countries, especially those having high population density like india, makes them a robust case for launching e-governance-focused contact tracing mobile apps to enable better coordination, communication, prevention, and self-control during pandemic scenarios (garg, bhatnagar, & gangadharan, 2020). for example, during the covid-19 pandemic, the goi launched the community-driven contact tracing mobile app called aarogya setu on 2nd april 2020 as an e-governance initiative to enable prevention and control of this pandemic among the people of india (the times of india, 2020). within two weeks of its launch in india, the number of contact-tracing mobile app downloads reached 50 million (banerjea, 2020). during the early days post-launch, several citizens expressed scepticism about this mobile app, due to privacy and security issues highlighted by ethical hackers and some political opposition leaders in the country (the economic times, 2020). however, despite the criticism, the number of downloads has increased steadily for this contact tracing mobile app in india and reached more than 169 million in february 2021 (aarogya setu website, 2021). similarly, contact tracing mobile apps have seen increasing adoption and usage numbers in various countries despite similar scepticism over security and privacy issues. another example relates to the launch of a contact-tracing mobile app named "immuni” by the government of italy on 1st june 2020, amidst privacy concerns (reuters, 2020). despite scepticism over privacy and security, the adoption of the contact tracing mobile app in italy increased steadily after the launch. from june 2020 to february 2021, the app has been downloaded by more than 10.27 million users. looking at growth trends, the number of downloads has increased from 10.245 million on 29th january 2021 to 10.272 million on 4th february 2021 (immuni website, 2021). however, in percentage terms, the adoption rate of contact tracing mobile apps in india and italy is still only around 12% and 17%, respectively, australasian journal of information systems goyal, pillai & chauhan 2021, vol 25, research article e-governance using mobile applications 3 considering the approximate population of india at 1380 million and italy at 60.46 million (worldometer, 2021). though these contact tracing mobile apps have shown significant traction and facilitated realtime direct connections, thereby strengthening the impact of e-governance, there is still significant scope for increasing the penetration of these e-governance mobile apps among people in different countries. this implies that governments need to devise appropriate strategies, tactics and measures to enhance the adoption rate without enforcing compliance. at the same time, this also provides an opportunity for researchers to undertake future studies in this direction, in which they can focus on understanding the key enablers leading to the mass adoption of e-governance initiatives. the gravity of the pandemic situation and governments’ efforts to deal with such an unprecedented situation, presents an interesting and crucial question: what factors drive the citizens’ intention to use the e-governance-initiated contact-tracing mobile app? to answer the above question, we took the case of a contact tracing mobile app launched by the goi during the covid-19 pandemic. further, we undertook the positivist approach to understand the influence of consumption factors, such as epistemic value, functional value, conditional value, convenience value, as well as privacy concerns on the intention to use. the extant literature has looked into the influence of consumption values and other related factors in driving the intention to use mhealth app services (lee, han, & jo, 2017); mobile health services (lee & han, 2015); and mobile apps in general (wang, liao, & yang, 2013) among the public. similarly, there is existing literature which has evaluated the influence of privacy concerns and related factors in driving the behavioral intention to use mobile apps and related technology in healthcare services (dhaggara, goswami, & kumar, 2020; gu, xu, xu, zhang, & ling, 2017). research articles like lee et al. (2017) recommended the study of additional context and content values during future research in the context of intention to use mobile healthcare services across other countries. lee et al. (2017) argued that context-content-related values can vary across countries, as cellular network availability, as well as healthcare systems, vary in terms of healthcare provider density. therefore, based upon a review of the extant literature, we realized that, if consumption values and privacy concerns are studied together, they can provide more meaningful and valuable insights into the intention to use a mobile app, especially in the context of health-related services. for this reason, we decided to use consumption and privacy values together in the research model to study the intention to use the mobile app. an epistemic value construct measured how the mobile app helped to satisfy users' curiosity about covid-19 cases around them in a novel manner. a conditional value construct helped to measure the perceived seriousness of the covid-19 risk amongst app users. convenience value measured the extent to which the app was convenient for users to remain updated about the situation around them. the functional value construct highlighted the utilitarian benefits one could achieve by using the app. finally, the privacy concerns construct helped in understanding whether mobile app users were concerned about their privacy exposure due to its usage. the extant literature has ample proof of the relevance of these constructs to the adoption of a technology (gu et al., 2017; lee & han, 2015; lee et al., 2017; pihlström & brush, 2008; sheth, newman, & gross, 1991). australasian journal of information systems goyal, pillai & chauhan 2021, vol 25, research article e-governance using mobile applications 4 overall, the present study has the potential to make some unique contributions. first, it has examined how different factors influence the intention to use the mobile app launched by the government for electronic-governance (e-governance) and its functional value. as mentioned above, the influence of multiple constructs was investigated, each construct having a varying potential influence. therefore, our research captures a comprehensive picture of mobile app adoption. second, this study attempts to enrich insights by testing alternate models along with the main research model. this can help in gaining an in-depth understanding of the relationships among the constructs. finally, though there have been several studies on the adoption of mobile apps, including health apps, their findings cannot be generalized to pandemic situations. therefore, this study has the potential to add significant value to the existing literature and provide recommendations to policymakers on how to enhance the acceptance of a technology-oriented e-governance effort at the time of crisis. the present study offers significant theoretical and practical implications. in the context of theoretical considerations, this is one of the first research studies that has integrated the theoretical constructs of privacy concerns and consumption values while investigating public intention to use the contact-tracing mobile app during the pandemic scenario. another key theoretical implication involves the determination of factors contributing to the functional value, adoption, and usage of government-backed e-governance initiatives in a pandemic situation. finally, this is one of the first e-governance-related research studies which has tested alternate models to explore other possible relationships between the constructs having a significant impact on the intention to use the contact-tracing mobile app. in the context of practical considerations, this is one of the first research studies, which has presented a unique perspective regarding the factors driving the adoption of a digital mobile app as an egovernance initiative in one of the most populated countries in the world, having huge socioeconomic diversity. another key recommendation by this study involves focusing on functional value and context-specific features, rather than the novelty aspect, while launching e-governance initiatives among the general public in a heterogeneous country like india. the research paper is structured as follows. section 2 develops the theoretical background and hypotheses. the research methodology is explained in section 3. section 4 analyses the data and presents the results. the results are discussed in detail in section 5. finally, section 6 presents the concluding remarks, theoretical and practical implications, limitations and future research directions. 2 theoretical background and hypothesis development rapid advancements in information and communication technologies (ict) have resulted in inexpensive and pervasive technologies. such technologies have enabled the governments of various nations to design and implement innovative solutions for better e-governance and value offerings to the citizens of their countries (chauhan & kaushik, 2016). e-governance seeks to utilize ict to provide government services digitally to organizations, citizens, and various levels of government to achieve the purpose of promoting good governance (singh & sahu, 2018; saxena, 2005). e-governance provides government information and public services on time, empowers the citizens to retrieve information without bureaucracy, and improves managerial effectiveness (stemberger & jaklic, 2007). thus, e-governance plays a vital role in streamlining public administration, which enables the establishment of seamless, citizencentric, polycentric, and cooperative governance (leitner, 2003). hence, the adoption of eaustralasian journal of information systems goyal, pillai & chauhan 2021, vol 25, research article e-governance using mobile applications 5 governance initiatives has been studied by several scholars (chauhan & kaushik, 2016; khanra & joseph, 2019). a review of extant research studies related to adoption, acceptance, or use of technology brings out two perspectives. first, a majority of the studies related to the adoption, acceptance, or use of technology designed their conceptual model based on prevalent information system theories like the technology acceptance model (tam), the theory of reasoned action (tra), and the unified theory of acceptance and use of technology (utaut) (lim, 2018a, 2018b; lim, lim, & phang, 2019). second, few studies highlighted the need to look beyond traditional information system theories of technology adoption (pihlström & brush, 2008; shroff & keyes, 2017; okazaki, 2008). on similar lines, this study investigates the intention to adopt the contact-tracing mobile app, based on consumption values along with privacy concerns, instead of the tam and utaut. the study focuses on understanding the significance of consumption values and privacy concerns as predictors for the adoption of technology, which is expected to add value in the context of the e-governance setup, accessibility to e-healthcare, or access to higher education online. with growing internet and smartphone penetration, the launch and use of the mobile app or m-app is gaining importance as a business phenomenon for service providers and as a differentiated value offering (content, information, or service-oriented) for users (alhassan, li, reddy, & duppati, 2020; gu et al., 2017; lee, 2010; lu, yao, & yu, 2005; sarkar, chauhan, & khare, 2020; singh, 2020). many existing research studies have focused on understanding people’s consumption, use, or adoption behavior in the context of the specific product or service offering. there are two elements in these types of studies. one of the elements involves the determinants which influence the outcome of the study. another element relates to the outcome itself. in this study, the first element relates to the role of personal values, especially consumption values and privacy concerns in influencing the adoption or use behavior of consumers towards a particular product or service offering online (gu et al., 2017; sheth et al., 1991). the second element in this study corresponds to the intention to use. understanding the individual's intention to use different technology-based offerings has been observed as one of the key research themes across many studies published during 1990 – 2020. scholars have extensively used these theories to study the factors that influence users’ technology usage intentions, such as the intention to use smartphones, mobile apps, and different modes of elearning scenarios, such as web-based learning, internet-based learning, and massive open online courses (moocs) (chow & shi, 2014; lee, 2010). the expression ‘intention to use’ is derived from behavioral intention studies and is defined as the strength of an individual’s intention to perform specified behavior (fishbein & ajzen, 1975), which subsequently predicts the actual adoption of technology and its usage by the user (agrebi & jallais, 2015). tam related research studies have shown that intention to use has a positive impact on adoption behavior (lu, lin, & chen, 2017). there have been similar research studies that have observed a direct correlation between behavioral intention to use and actual usage, resulting in behavioral intention to use as a significant predictor of the action (davis, bagozzi, & warshaw, 1989; hill, smith, & mann, 1987). in reference to the above background information, this section further explains the theory of consumption value and privacy theory. these theories have been applied to this study as australasian journal of information systems goyal, pillai & chauhan 2021, vol 25, research article e-governance using mobile applications 6 appropriate in the context of the contact-tracing mobile app as an e-governance, communication, and preventive healthcare application during the covid-19 pandemic situation. 2.1 the theory of consumption values researchers in the social sciences stream, especially those involved in marketing-related studies are interested in analyzing the determinants of personal values which influence the adoption or intention to use among consumers of different kinds of products and service offerings (lee & han, 2015; sheth et al., 1991). rokeach (1968, p.16) defined personal value as a "centrally held, enduring belief which guides actions and judgments across specific situations and beyond immediate goals to more ultimate end-states of existence.” understanding personal values, especially consumption values, helps in gaining an understanding of the behavioral pattern of the consumers, such as why certain consumers behave in a certain way (sheth et al., 1991; vinson, scott, & lamont, 1977). researchers argue that value cannot be a one-dimensional construct. rather, it is considered to be a multidimensional phenomenon because people perceive value differently depending upon personal beliefs, needs, types of offering, mode of delivery, and preferences (kumar, sachan, & kumar, 2020; lee & han, 2015, sheth et al., 1991). the theory of consumption values postulates that a consumer’s positive attitude and buying decision are formed on the basis of their assessments of several value components, such as epistemic, conditional, functional, emotional, and social values (sheth et al., 1991). consumers’ intention to buy a service or a product is enhanced or motivated by these values (yang & lin, 2017). this theory is built upon three key axiomatic propositions: a) a consumer’s choice is the function of multiple consumption values; b) the consumption values make distinct contributions in any buying situation; and c) these values are independent. this theory is applicable to various types of product classifications, such as durable and nondurable products and services (gonçalves, lourenço, & silva, 2016) and its predictive validity has been established in over 200 applications (sheth et al., 1991). hence, a decision might be affected by any or every consumption value. values help in guiding actions, judgments, attitudes, and comparisons between particular items and settings (long & schiffman, 2000). hence, there exists substantial support for the role of values in developing attitudes, beliefs, and behaviors (kilbourne & pickett, 2008). kumar et al. (2020) studied the behavioral intention of users towards internet banking and argued that consumption values regarding internet banking services have a moderating effect on customer satisfaction and subsequently influence their behavioral intention towards such services. there have been research studies in a similar context, which applied different consumption value dimensions to understand the context behind specific consumer or user behavior, especially in the context of healthcare-related offerings (cengiz & kirkbir, 2007; zainuddin, russell-bennett, & previte, 2013). cengiz and kirkbir (2007) identified the significance of functional, social, and affective values in the determination of perceived value for healthcare services. zainuddin et al. (2013) investigated the role of functional and emotional values in the value creation process for preventive healthcare services. however, there is a dearth of research that has analyzed the role of different dimensions of personal values in the determination of usage behavior towards mobile apps, especially in a pandemic situation. australasian journal of information systems goyal, pillai & chauhan 2021, vol 25, research article e-governance using mobile applications 7 2.2 privacy theory information privacy especially refers to the control of personal information, e.g., name, employment history, ethnicity, etc. (lah, 2008). smith, dinev, and xu (2011) described it as someone’s state of restricted access to information about himself/herself. westin (2003) argued that privacy is the psychological condition in which people can take a decision on what, how, and when information about them can be shared with others. altman (1976) theorized that privacy is a boundary control process that can open and close the boundary between the self and one’s social surroundings to achieve “the selective control of access to the self”’. therefore, the principle behind privacy is the control of information. regarding privacy concerns, it is becoming increasingly important and relevant in influencing the intention-to-use behavior of consumers, especially in the context of technology-based products or service offerings. as technology is becoming all-pervasive in the lives of human beings, there is growing concern or scepticism that the majority of online technology platforms, especially mobile apps, are privacy-invasive. many online or mobile apps collect personal details which are not required for the functionality of the application and use those details for unauthorized purposes (gu et al., 2017). 2.3 conceptual model constructs 2.3.1 epistemic value according to sheth et al. (1991, p. 162), epistemic value is defined as, “the perceived utility acquired from an alternative’s capacity to arouse curiosity, provide novelty, and satisfy a desire for knowledge seeking.” any new experience or an alternative that is being chosen for avoiding boredom, or fulfilling curiosity, or experiencing new learning provides epistemic value (sheth et al., 1991). for example, the adoption of online games or mobile apps can be triggered by the curiosity or desire to experience the new technology-based offering (okazaki, 2008; pihlström & brush, 2008). people try a new product or service because they get bored with the continued use of the existing brand and want to try something new in their life out of curiosity or variety-seeking behavior (chen & kotz, 2000; steenkamp & baumgartner, 1992). thus, epistemic value can play an essential role in the intention to use something different. however, sheth et al. (1991) also highlighted that, due to epistemic value, customers who go for a new or alternative experience may return to their original consumption patterns after experiencing the change. thus, epistemic value may or may not provide a long-lasting impact on the use of a new or alternative product unless and until the user finds high utility value while using it. the contact-tracing mobile app was primarily launched to provide real-time utilitarian value to mobile users during the pandemic situation of covid-19. this app holds functional value because it provides the users the opportunity to get real-time information regarding covid19 statistics, infected cases around them, and steps to be taken in case of any related health issue. this app carries detailed audio-visual messages featuring prominent personalities from bollywood (hindi movie industry). these messages help the users understand the precautions they should take during the covid-19 pandemic, besides giving health and hygiene tips, etc. apart from the continuous awareness that goi created about covid-19 through traditional media channels like television, newspapers, and radio, the launch of the aarogya setu app as a novel technology-based real-time interface is considered a commendable initiative. based on the above discussion, we propose the following hypothesis: australasian journal of information systems goyal, pillai & chauhan 2021, vol 25, research article e-governance using mobile applications 8 h1: epistemic value positively impacts the functional value of the e-governance-initiated contact tracing mobile app. 2.3.2 conditional value sheth et al. (1991) define conditional value as “the perceived utility acquired by an alternative as the result of the specific situation or set of circumstances facing the choice maker.” a consumer's decision to use a particular product or service depends upon the context or situation around him or her. for example, people take a significant interest in buying wedding-related products and services during the time leading up to their wedding event. similarly, the christmas tree sale is maximum during the christmas season, rather than the whole year. conditional value represents the social or functional value which is experienced in a specific context or situation and is independent of time and location (pihlström & brush, 2008; sheth et al., 1991). according to pihlström and brush (2008), the majority of users associate high conditional value with a specific product or service offering if it adds value to them in specific scenarios, like unplanned situations, unfamiliar locations, or during the state of movement from one place to another. this also implies that the extent of conditional value derived from the product or service depends, to a certain extent, on the mental state and location of the user. the rationale explained here fits well into the context of the conditional value derived from information services or mobile apps during a pandemic situation like covid-19. during covid-19, several countries launched a contact-tracing mobile app to collect and share realtime information with app users regarding covid-19 cases, symptoms, preventive measures, and self-control actions anytime and anywhere. this indicates that the contact-tracing mobile app has high conditional value during pandemic situations, because it enables real-time access to time-critical information for the citizens anytime and anywhere. once citizens start using the contact-tracing mobile app, they find it useful in getting timely access to required information, thereby safeguarding themselves from infection by the coronavirus. this creates a positive impact in terms of the functional appeal of this app. based on this discussion, we hypothesize a positive relationship between conditional and functional value dimensions. h2: conditional value positively impacts the functional value of the e-governance-initiated contact-tracing mobile app. 2.3.3 convenience value convenience value relates to the ease and speed of effectively and conveniently achieving a particular task (anderson & srinivasan, 2003). convenience value for a product or service is determined based on the value addition that it provides to the consumer in terms of minimizing the time and effort required for completing a task (colwell, aung, kanetkar, & holden, 2008; pihlström & brush, 2008; singh, 2020). pagani (2004) argues that the speed of use is one of the key determinants in the adoption of multimedia mobile by the users. in the healthcare context, zhang, yu, and shen (2012) highlight how the electronic health records system can lead to an increase in convenience value due to quick and easy data input, storage, distribution, and retrieval. in the context of information services and mobile apps, convenience value is considered to be one of the key dimensions for enhancing utilitarian value because it promotes self-service, thereby australasian journal of information systems goyal, pillai & chauhan 2021, vol 25, research article e-governance using mobile applications 9 improving the speed of access to information in optimal time and with minimal efforts (pura, 2005; rintamäki, kanto, kuusela, & spence, 2006; singh, 2020). singh (2020) reviewed the post-adoption behavior of users towards mobile payment systems and argued that effort expectancy or user-friendliness of the mobile payment system promotes initial acceptance as well as continuance intention towards its use. in this study, the convenience value of the contact tracing mobile app in the context of covid-19 lies in its ease of use and 24x7 self-service access to real-time information regarding the overall count of pandemic cases in the affected countries, as well as the count of infected patients in adjoining areas, and details regarding nearby hospitals and health centers for any emergency. these apps provide an opportunity for self-service and real-time access to covid-19 data to millions of smartphone users across the affected countries (lohchab, 2019). for example, during the covid-19 pandemic, the government of india increased the penetration of the contact-tracing mobile app by setting up an interactive voice response system (ivrs) for feature phone users in india (agarwal & sharma, 2020). this ivrs setup was expected to help goi reach 900 million citizens of india. this setup enabled self-service access in 24x7 mode to feature phone users, who started calling on the ivrs toll-free number to discuss their medical condition and get timely advice regarding the appropriate medical intervention. based on the above discussion, we formulate the following hypothesis regarding the relationship between convenience and functional value dimensions in the context of using a contact-tracing mobile app during the pandemic situation. h3: convenience value positively impacts the functional value of the e-governance-initiated contact tracing mobile app. 2.3.4 functional value functional value relates to the utilitarian functions which are offered by a specific product or service. tzeng (2011) argues that functional value involves the manifestation of the product's quality and feature attributes, resulting in utilitarian performance. sheth et al. (1991) define functional value as the product's ability to perform a functional, utilitarian, or physical purpose. singh (2020) argues that the utility value of the mobile payment system enhances the performance expectancy belief of the users, which in turn significantly influences their continuance intention. this value dimension is considered to be a primary driver in the choice and usage behavior of the consumer. another way of looking at functional value is that it involves the value derived from efficient task fulfillment in terms of monetary value and superiority, compared to available alternatives, availability, reliability, durability, and usability (holbrook, 1994; pihlström & brush, 2008). in this study, the contact-tracing mobile app’s functional value is driven by the bundle of value-added features offered by the app to its users during the pandemic situation. during the pandemic situation of covid-19, the key offerings of the contact-tracing mobile app launched by the governments in different countries involve educating/alerting the user anytime and anywhere, about their contact with a covid-19 patient and corrective measures in case of any symptoms. the app also offers access to telemedicine, real-time information regarding nearby e-pharmacy, and diagnostic services (firstpost, 2020). moreover, at least in the case of the indian app, using the app does not get counted in the daily mobile data usage limit (firstpost, 2020). geographically stranded citizens of the country can apply for an e-pass through the australasian journal of information systems goyal, pillai & chauhan 2021, vol 25, research article e-governance using mobile applications 10 contact-tracing mobile app to seek permission to travel across district and state borders. all these features of the contact-tracing mobile app have significantly enhanced its functional value, which in turn strengthens its usability among the people of india. we propose the following hypothesis regarding the relationship between functional value and intention to use the contact-tracing mobile app during the pandemic situation. h4: functional value positively impacts the intention to use the e-governance-initiated contact-tracing mobile app. 2.3.5 privacy concerns there is increasing concern for privacy in this digital world. gu et al. (2017) define privacy concerns in the context of an android app as the extent to which users feel concerned about the possible loss of privacy when they determine whether to download an app. many research studies look at the relationship between privacy-invasive mobile apps or web applications visà-vis users' intention to use those mobile apps or web applications (gu et al., 2017). the dominant view in these research studies implies that privacy concerns negatively affect the intention of the individual towards the use of digital technologies like mobile apps or web applications for online information, electronic-content, electronic-learning, or buy-sell transactions (angst & agarwal, 2009; dinev & hart, 2006). there has been a gradual shift in research studies from general towards context-specific privacy concerns over a period. it has been observed that the extent of privacy concerns depends upon the context in which a particular digital technology is supposed to be used (xu, teo, tan, & agarwal, 2012). the authors argue that context-specific privacy concerns involve the linkage between an individual's assessment of privacy concerns and a specific context with a specific external agent. this implies context-specific privacy concerns vary across individuals and are driven by the individual's assessment of the situational context. many experts have raised privacy concerns regarding the contact-tracing mobile apps launched by different countries as an e-governance initiative during the covid-19 pandemic for the following reasons. first, the contact-tracing mobile app launched in countries like india uses gps tracking and bluetooth data to track users' movement and share real-time information about the infected cases within a close radius (the quint, 2020). second, it has raised concerns among the experts due to below-par performance on data minimization in some countries (the hindu, 2020; the quint, 2020; firstpost, 2020). this means the contacttracing mobile app in some countries is collecting more data than needed for it to work. moreover, there is no clarity on how long the details of the infected and healthy users will be kept in the system. third, the app has attracted criticism for lack of transparency in some countries like india, as the app was not open-sourced at the time of launch and so third parties could not review its code and methods (the hindu, 2020; the quint, 2020; firstpost, 2020). fourth, governments in countries like india have not defined a public sunset clause for the app (firstpost, 2020). this implies there is no clarity as to when this app will cease to be employed. to add complexity and concerns in the minds of the potential users, some countries, like india, initially pushed the use of the contact-tracing app as a mandatory step for smartphone users in some states of india (the hindu, 2020). the fact that many states in india have tried to make its use mandatory is likely to raise privacy concerns among users despite its functional value. australasian journal of information systems goyal, pillai & chauhan 2021, vol 25, research article e-governance using mobile applications 11 while taking into consideration the view of the experts on contact-tracing mobile apps launched in different countries and rising concerns about their privacy attributes, we hypothesize that privacy concerns have a significant negative impact on the use of this app. h5: privacy concerns negatively impact the intention to use the e-governance-initiated contacttracing mobile app. table 1 provides a summary of the consumption value and privacy dimensions which act as key determinants towards intention to use the aarogya setu app, and the corresponding research model is highlighted in figure 1. dimension description sources epistemic value perceived utility is acquired from an alternative's capacity to arouse curiosity, provide novelty, and satisfy a desire for knowledge seeking. chen and kotz (2000); okazaki (2008); pihlström and brush (2008); sheth et al. (1991); steenkamp and baumgartner (1992) conditional value perceived utility is acquired by an alternative as the result of the specific situation or set of circumstances facing the choice maker. pihlström and brush (2008); sheth et al. (1991) convenience value ease and speed of achieving a particular task effectively and conveniently anckar and d'incau (2002); anderson and srinivasan (2003); colwell et al. (2008); pihlström and brush (2008); pagani (2004); pura (2005); rintamäki et al. (2006); zhang et al. (2012) functional value the ability of the product to perform the functional, utilitarian, or physical purpose holbrook (1994); sheth et al. (1991); pihlström and brush (2008); tzeng (2011) privacy concerns the extent to which users feel concerned about the possible loss of privacy when they are deciding whether to download an app angst and agarwal (2009); dinev and hart (2006); gu et al. (2017); xu et al. (2012) table 1. description of the value and privacy dimensions figure 1. research model australasian journal of information systems goyal, pillai & chauhan 2021, vol 25, research article e-governance using mobile applications 12 3 research methodology 3.1 the aarogya setu mobile app aarogya setu in the sanskrit language meant, "bridge to health" (clarance, 2020). launched in april 2020, this community-driven contact-tracing mobile app became the world’s fastest downloaded mobile tracing app, as it was downloaded over 50 million times in just 13 days from its launch (banerjea, 2020). during april – june 2020, the aarogya setu app was downloaded over 100 million times (clarance, 2020). aarogya setu mobile app was designed, developed, and hosted by national informatics centre, under the guidance of the ministry of electronics and information technology. as of may 17, 2020, this app was made available in eleven vernacular languages of india, as well as in english. amidst growing privacy and security concerns, goi made the source code of this mobile app public on 26th may 2020. the total download figures for this contact-tracing mobile app have been growing steadily and surpassed 167.7 million and 169 million in december 2020 and february 2021, respectively, with daily active users in the range of around 4 – 5% (aarogya setu website, 2021). a user would access the aarogya setu app an average of four times a day, with average daily time spent on the aarogya setu app around eight minutes. during 2020-21, the consistent upswing in the intention to use this community-driven contact tracing mobile app signifies the relevance and usefulness of this e-governance initiative among the public and the government for real-time information, monitoring, tracking, and self-control. after downloading the contact-tracing app, the user gets a notification regarding the device’s bluetooth and location sharing data capture. the user registers himself/herself on the app by responding to a self-administered questionnaire. the questionnaire focuses on travel history, medical history, smoking habits, and whether he or she has had any covid-19 symptoms. the app tracks the interaction between the user and other individuals with the help of bluetooth and location. this creates a social graph that helps the citizens and the government / medical officers in identifying contact-tracing opportunities for further steps. though the aarogya setu app received global visibility due to its exceptional success, some leftists were busy maligning the product, rather than supporting the country’s invention (o’neill, 2020). aarogya setu’s success was considered to have resulted due to the compulsory adoption strategy imposed by the indian government on frontline covid warriors, government officials, etc. (o’neill, 2020). journalists failed to report the importance of these covid warriors who were fighting the battle without worrying about their health and it was essential to safeguard and support them. on the other hand, some myopic journalists missed the fact that there were several developed countries that adopted the compulsive adoption strategy for contact tracing apps (farronato, iansiti, bartosiak, denicolai, ferretti, & fontana, 2020). in developed countries like italy, even private sector employees were forced to download the contact-tracing app. in a vast country like india, if the government had waited for the contact-tracing app to be downloaded voluntarily, contact tracing would have been much more difficult and there would have been many more infections. hence, it was the right strategy adopted by the indian government to first safeguard its frontline covid warriors and thereby reach sufficient critical mass that would be required to ensure the contact tracing strategy was effective. australasian journal of information systems goyal, pillai & chauhan 2021, vol 25, research article e-governance using mobile applications 13 3.2 measurements the measurements of the principal constructs such as privacy and intention of use were acquired from existing studies (gu et al., 2017; kijsanayotin, pannarunothai, & speedie, 2009; lu et al., 2005; shroff & keyes, 2017) and tailored to fit the context of our research. these measurements were content validated based on the opinion of a panel of academicians in the management discipline. the measurements of other principal constructs, namely functional value, conditional values, epistemic value, and convenience value were self-developed following the approach suggested by previous studies (hoehle & venkatesh, 2015). first, the preliminary items were created by using the existing literature (pihlström & brush, 2008; sheth et al., 1991; zandifar & badrfam, 2020) and discussion with a panel of academicians. we further invited two faculty members and four doctoral scholars who were acquainted with the present research subject to conduct a face validity check of the preliminary items. the items were further modified based on their opinion. second, we conducted a card-sorting test to assess the content validity of the items. we invited ten business school students to participate in this activity. the meaning of each construct was explained to these students. we constructed a matrix containing the definition of each construct in the top row and all the items in the first column. every participant was provided with the matrix and asked to choose only one construct for each item. later, the proportion of substantive agreement was calculated (anderson & gerbing, 1991). the outcomes showed that the likelihood of every item being accurately classified was more than 60%, meeting the conditions for content validity (mackenzie, podsakoff, & podsakoff, 2011). third, a pre-test was conducted to further validate the instrument. we measured the items using a seven-point likert scale, ranging from strongly disagree to strongly agree. the participants of the pre-test were informed about the purpose of the present research. we collected 150 valid responses for the pre-test. the reliability of the measurement scale was analyzed by assessing the cronbach’s alpha values, which were found to be greater than 0.7 for each construct, signaling adequate reliability. our next step was to conduct exploratory factor analysis. we performed the kmo and bartlett spherical tests to assess predictive validity. we found the kmo value as 0.775 (> 0.7), the approx. chi-square value as 1299.161, the degree of freedom as 171, and the significance level of p as 0.000 (< 0.05). these results suggested that the measurement instrument was appropriate for exploratory factor analysis (churchill, 1979). as a result of exploratory factor analysis, the measurement items were categorized into four factors, together explaining 68.808% of the total variation. ultimately, the measurement instrument of four constructs demonstrating good reliability and validity was ready for further use. table 2 presents the list of constructs and their measurements. 3.3 data collection we created an online survey form. the link to the survey form was emailed to university students, faculty members, and professionals working across different organizations in india. through the email, the potential respondents were informed about the purpose of this academic research and assured of their anonymity. the data for the present study was collected from may 1, 2020 to may 30, 2020. australasian journal of information systems goyal, pillai & chauhan 2021, vol 25, research article e-governance using mobile applications 14 construct items measurements source functional value fv1 aarogya setu app helps in understanding how to self-isolate. self-developed fv2 aarogya setu app helps in understanding what needs to be done in case someone develops covid-19 symptoms. fv3 aarogya setu app alerts immediately if someone in proximity develops covid-19 symptoms and tests positive. fv4 aarogya setu app helps in identifying the nearest hospital/health center. fv5 aarogya setu app helps in remaining updated about covid19 active cases in india. conditional value cv1 the unpredictability of the covid-19 situation stresses me. self-developed cv2 the seriousness of the covid-19 risk stresses me. epistemic value ev1 i am curious to stay informed regarding the covid-19 cases around me from the aarogya setu app. self-developed ev2 aarogya setu app is a novel way to know about covid-19 situation around me. convenience value conv1 aarogya setu app helps me to stay informed about covid19 situation anytime and anywhere. self-developed conv2 aarogya setu app does not help me to stay informed about covid-19 situation more easily. (reverse coded) privacy concerns pc1 i think the aarogya setu app over-collects my personal information. gu et al. (2017) pc2 i worry that the aarogya setu app may leak my personal information to irrelevant third parties. pc3 i am concerned that the aarogya setu app may violate my privacy. pc4 i am concerned that the aarogya setu app may misuse my personal information. intention to use iu1 using the aarogya setu app is worthwhile. kijsanayotin et al. (2009); lu et al. (2005); shroff and keyes (2017) iu2 i intend to use the aarogya setu app in the future. iu3 i intend to use the aarogya setu app frequently. iu4 i expect to continue using the aarogya setu app in the future. table 2. constructs and measurements the survey questionnaire had two questions at the beginning to screen out respondents who either did not download the aarogya setu app or who downloaded it but did not complete the self-assessment test on the app. in both scenarios, respondents were asked to submit the questionnaire without completing other questions in the survey. the link to the online survey questionnaire was sent to 1,350 indian citizens of different age-groups. about 850 respondents clicked on the link and submitted the survey. among the 850 responses received, 359 respondents had not downloaded the aarogya setu app, and 206 respondents had downloaded the app but did not attempt the self-assessment test. that left us with a final list of 285 respondents who had experienced the aarogya setu app themselves. this process was followed to ensure that the people in our survey were actual users of the aarogya setu app. australasian journal of information systems goyal, pillai & chauhan 2021, vol 25, research article e-governance using mobile applications 15 as the country was under full lockdown during the data collection phase, the researchers did not have the opportunity to administer the survey questionnaire to respondents face-to-face. instead, the researchers took the option of electronic distribution of survey questionnaires. the respondents were informed about the goal of the research so that they could fill up the survey questionnaire appropriately. the questions in the survey used a seven-point likert scale ranging from strongly disagree to strongly agree. the demographics of these respondents are presented in table 3. the sample represents the fact that india has one of the world’s largest young populations (statistics times, 2020), while more than 50% of india’s population was over 25 years old (livemint, 2020). the sample selected for the study is adequately representative of the overall country’s population trends with respect to age groups. this sampling does have a bias towards respondents who were well-educated. however, given the external constraints, researchers could not get access to a larger set of respondents with varied educational backgrounds. characteristics % characteristics % gender education male 63% graduate 59% female 37% post-graduate 32% doctorate 6% other 4% age region 20-29 14% north india 51% 30-39 41% south india 16% 40-49 27% east india 5% 50-59 10% west india 12% > 60 7% central india 16% table 3. demographics of respondents 4 data analysis and results we applied the structural equation modeling (sem) approach to examine the relationships present in the proposed research model. the survey data was assessed using variance-based partial least squares (pls) sem. we undertook the pls sem approach as it can handle complex predictive models. pls structural modelling is a robust method to examine the causal model encompassing many constructs (chin & newsted, 1999). we used the smartpls 3 software to examine the measurement model and the structural model in our research. to assess the measurement model, we evaluated the reliability and the validity of the measurements in the questionnaire. on the other hand, relationships present in the research model were examined using the structural model. 4.1 common method bias (cmb) we used procedural as well as statistical treatments for assessing the cmb issue (podsakoff, mackenzie, lee, & podsakoff, 2003). under the procedural treatment, respondents were informed that no response was right or wrong so they should remain candid while completing the survey. additionally, they were notified that their identity would remain anonymous. australasian journal of information systems goyal, pillai & chauhan 2021, vol 25, research article e-governance using mobile applications 16 under the statistical treatment, harman’s single factor test was applied to assess the presence of cmb, and the result indicated that only 23.462% of the variance was explained. thus, we concluded that the cmb issue did not exist (hew & kadir, 2017). 4.2 measurement model to evaluate the measurement model, we tested reliability, convergent validity and discriminant validity. the reliability of the constructs was assessed via cronbach's alpha and composite reliability values. as shown in table 4, the value of every cronbach’s alpha and composite reliability score was greater than 0.7, indicating a satisfactory level of reliability of scales (fornell & larcker, 1981). table 4 also shows the values of average variance extracted (ave) to be greater than 0.5, indicating convergent validity. convergent validity was also evaluated through the standard loadings of the measurement items. reliability convergent validity cronbach's alpha composite reliability average variance extracted (ave) cv 0.768 0.896 0.812 conv 0.739 0.884 0.792 ev 0.828 0.919 0.850 fv 0.869 0.905 0.657 iu 0.859 0.904 0.702 pc 0.847 0.888 0.667 table 4. reliability and convergent validity of constructs construct item loading standard deviation t statistics conditional value cv1 0.895 0.015 59.047 cv2 0.907 0.014 65.144 convenience value conv1 0.905 0.021 43.461 conv2 0.875 0.026 33.086 epistemic value ev1 0.895 0.216 4.139 ev2 0.947 0.208 4.564 functional value fv1 0.814 0.021 38.337 fv2 0.853 0.015 56.006 fv3 0.779 0.025 31.515 fv4 0.781 0.023 34.238 fv5 0.823 0.019 43.892 intention to use iu1 0.835 0.021 38.985 iu2 0.882 0.013 67.783 iu3 0.786 0.026 30.753 iu4 0.846 0.018 47.064 privacy concerns pc1 0.905 0.192 4.708 pc2 0.724 0.242 2.993 pc3 0.849 0.201 4.231 pc4 0.777 0.205 3.788 table 5. item loadings table 5 shows that every loading was greater than 0.6, demonstrating an adequate level of convergent validity for all the constructs (hair, black, babin, anderson, & tatham, 2010). furthermore, discriminant validity was assessed by comparing the square roots of aves with the values of inter-construct correlation. table 6 indicates that the values of the square root of australasian journal of information systems goyal, pillai & chauhan 2021, vol 25, research article e-governance using mobile applications 17 aves (diagonal values) were greater than the values of inter-construct correlation, signifying discriminant validity (fornell & larcker, 1981). cv conv ev fv iu pc cv 0.901 conv -0.046 0.89 ev 0.023 0.022 0.922 fv 0.455 0.340 0.060 0.811 iu 0.289 0.099 -0.065 0.465 0.838 pc 0.035 -0.024 0.044 -0.017 -0.089 0.817 table 6. discriminant validity of constructs 4.3 structural model the structural model was employed to evaluate the hypotheses developed in the present research. we computed the path coefficients and the r2 values to examine the structural model (chin & newsted, 1999). path coefficients act as the measurement for the statistical significance of the hypotheses, whereas r2 depicts the research model’s capability to explain the variation in the dependent variables. the results of the structural model are presented in table 7. we found support for hypotheses h2, h3, and h4, while no support was found for h1 and h5. it was also revealed that the model explains 22.3% of the variation in intention to use and 34% of the variation in functional value. hypothesis relationship path co-efficient t statistics significance h1 ev→fv 0.042ns 0.805 not significant h2 cv→fv 0.471* 11.712 significant h3 conv→fv 0.361* 8.545 significant h4 fv→iu 0.464* 11.724 significant h5 pc→iu -0.08ns 1.228 not significant table 7. structural model results 5 discussion this research paper is based on the theory of consumption values and used its multiple dimensions, including functional, social, epistemic, and conditional values. the element of privacy concerns in technology adoption was also included. the primary model was based on various studies in the literature (gu et al., 2017; kijsanayotin et al., 2009; lu et al., 2005; pihlström & brush, 2008; sheth et al., 1991; shroff & keyes, 2017; zandifar & badrfam, 2020). the scale items were adapted for the research context of the e-governance-related launch of a contact-tracing mobile app in india known as aarogya setu. also, the researchers attempted alternate models as this is considered in the literature an effective way to understand a new context (boon-itt & wong, 2016; falbo, kim, & chen, 2009). table 8 indicates a summary of the main model and four alternate models. figure 2 then depicts the four alternate models and the significance of the relationship between different constructs in each of these models. as the pandemic research context is severe and new to the indian audience, we decided to test the alternate models and find if any alternative australasian journal of information systems goyal, pillai & chauhan 2021, vol 25, research article e-governance using mobile applications 18 relationships could be explored. convenience value and conditional value constructs showed consistent significant relationships with functional value and intention to use across the models in which they were tested. also, the functional value had a significant relationship with the intention to use. it was clearly established that the stress and urgency of the situation (measured by conditional value) along with the convenience with which the technology could be adopted, led to strong functional value, which further led to intention to use. relationships main model alternate model 1 alternate model 2 alternate model 3 alternate model 4 ev→fv cv→fv 0.471 0.472 0.474 0.472 conv→fv 0.361 0.364 0.364 0.362 pc→fv ev→iu cv→iu 0.326 0.305 conv→iu 0.121 0.119 pc→iu fv→iu 0.464 0.469 table 8. relationships summary across various alternate models tested note: highlighted cells indicate significant relationships. numbers within cells indicate the path coefficients as shown by the results of the four alternate models tested, conditional and convenience values were found to have a significant positive impact on functional value and the intention to use. thus, users of the contact-tracing mobile app had easy access to utilitarian benefits during stressful times. the apparent reasons behind these findings are as follows. first, contact-tracing mobile apps were launched by the governments of different countries, including india, as an e-governance initiative during the covid-19 pandemic situation. considering the covid-19 context, the intention to use this mobile app was not driven by the epistemic value or novelty aspect. the contact-tracing mobile app is looked upon primarily from the functional value perspective when it comes to intention to use. second, again considering the pandemic context, intention to use among citizens of countries like india is driven by the functional value-addition rather than privacy features. as highlighted in the literature, there is no denying the fact that users value privacy more than many other aspects while using any mobile app (angst & agarwal, 2009; dinev & hart, 2006; gu et al., 2017; xu et al., 2012). however, the covid-19 pandemic subdued the privacy concerns, as making use of online contact-tracing mobile apps has become a necessity for the people in this pandemic situation, rather than an optional choice to remain connected with the relevant information. the insignificant relationship between privacy concerns and intention to use was commonly found across all the alternate models. however, this comes as a huge surprise and is in contrast to the extant literature where privacy concerns were established to be a key determinant in technology adoption (angst & agarwal, 2009; dinev & hart, 2006; gu et al., 2017; xu et al., 2012). with the background of multiple instances when the push for the adoption of contact tracing mobile app in countries like india was criticized publicly by opposition political parties and some ethical hackers for bugs with respect to privacy, its insignificance as a contributor to australasian journal of information systems goyal, pillai & chauhan 2021, vol 25, research article e-governance using mobile applications 19 the intention to use was surprising. the reason for such a relationship could be that, in the case of a pandemic scenario, users probably do not worry much about their privacy, but attach more importance to the utilitarian benefits of the technology that would come to their rescue in this stressful scenario. this result opens up a plethora of discussions and needs stronger evaluation in the future. another surprise element from the main and alternate models is the insignificant relationship of epistemic value with functional value and intention to use. it simply means that novelty and curiosity of the contact-tracing mobile app neither enhance the functional value of the app nor result in people’s intention to use it. the apparent reason behind this finding is that the novelty feature does not seem to influence technology adoption among the users during an emergency. during the covid-19 pandemic situation, users in countries like india have preferred contact-tracing mobile apps primarily due to the conditional and convenience attributes. figure 2. alternate models and relationship details australasian journal of information systems goyal, pillai & chauhan 2021, vol 25, research article e-governance using mobile applications 20 6 conclusion this research study examines the determinants, especially related to consumption values and privacy concerns in the behavioral use intention among mobile phone users towards egovernance-initiated mobile app adoption during the covid-19 pandemic. among the determinants, it has been found that convenience value, conditional value, and functional value constructs have a significant impact on intention to use the mobile app. however, privacy concerns and epistemic value do not have a significant impact on intention to use it. additionally, epistemic value does not have a significant impact on the perceived functional value of the mobile app. the research model offers insights from two perspectives. first, it showcases how the government in any country can leverage the internet and mobile app technology to exercise the e-governance initiative, connect with its population, and share relevant information in realtime. this argument is in sync with the proposition made by hasan and linger (2020). hasan and linger (2020) argued how the practices of government bureaucracies can be overcome by the open nature of electronic-democracy (e-democracy). according to toots (2019, p547), egovernance or e-democracy involves “...socio-technical information systems [acting] as instruments for democratic dialogue between citizens and governments.” globally, government organizations can look at leveraging digital technologies to enhance community engagement during exceptional scenarios, which in turn will build trust and acceptance among the general public towards e-governance initiatives. the use of the contact tracing mobile app emerged as a very popular e-governance tool for the goi by enabling it to maintain real-time connectivity and communication with more than 100 million people spread across india within 30 days of its launch. a similar trend is expected across other nations globally. second, it elaborates on how technology can be leveraged to promote the self-behavior orientation among the public in real-time. this generalization could be limited to an audience cohort that is largely well educated and internet-savvy. however, the behavioral intention of another cohort that was educated lower than degree level and was not internet-savvy might be needed to ascertain the generalizability of this research. next, we discuss the critical implications for theory and practice, followed by limitations and future research directions. 6.1 theoretical implications from the theoretical point of view, this is one of the first research studies that has looked into the intention to use technology-based offerings from the perspective of both theory of consumption values and privacy theory. the article combined the theoretical constructs of consumption values with privacy concerns to investigate the intention to use among the users of the government-backed contact-tracing mobile app during the pandemic scenario. the use of an integrated approach in studying the intention to use technology-based offerings is a value addition to the existing research, since the majority of the existing research articles in this area have reviewed separately the influence of consumption values and privacy concerns on intention-to-use behavior towards technology-based offerings (gu et al., 2017; kumar et al., 2020; xu et al., 2012; zainuddin et al., 2013). second, it determines the factors constituting the functional value of e-governance initiatives in challenging situations. third, the study contributes significantly to understanding the role of mobile apps in e-governance initiatives by the government, especially when there is a sense australasian journal of information systems goyal, pillai & chauhan 2021, vol 25, research article e-governance using mobile applications 21 of urgency, which requires time-critical updates and communication to the country's citizens. this argument is in sync with the proposition made by hasan and linger (2020), who highlighted the role of internet and mobile technologies in gaining community engagement and mass mobilization while driving e-governance initiatives during exception scenarios. fourth, the study highlights the significance of testing alternative models on the same data-set in a particular context. this addition is important, especially in exceptional or new scenarios like the covid-19 context. this alternative model approach is important to look beyond the dominant mindset or perception towards the specific research model, and thereby enhances learning about the relationship between different constructs (boon-itt & wong, 2016; falbo et al., 2009). as the covid-19 pandemic research context is severe and new to the indian audience, we decided to test the alternate models and find if any alternative relationships could be explored. 6.2 practical implications the present study also has significant implications from the perspective of practitioners. first, the study signifies the role of technology in triggering systemic behavior-change orientation among individuals in a specific region, society, or country. contact-tracing mobile apps were launched by governments globally to provide timely updates to the citizens of their country during the covid-19 pandemic situation. in this way, the governments discharged their responsibility of connecting people on a common technology platform for real-time communication, information, and guidelines during the exceptional situation of the covid19 pandemic. at the same time, by doing this, governments expected the citizens of their country to exhibit alert and self-responsive behavior during the pandemic scenario, thereby minimizing the adverse impact during this exceptional situation. second, the study suggests that governments need to focus more on enhancing the functional value driven by convenience and context-specific features, rather than novelty aspects, to drive the willing adoption and use of an e-governance initiative among the citizens during a crisis. this finding is in synchronization with the argument made by lee and han (2015) as well as lee et al. (2017). these articles highlighted the significance of context-driven usefulness and convenience values in stimulating the intention to use mhealth apps or services. third, during the covid-19 pandemic, the success of e-governance initiatives undertaken by the governments of various nations in the context of adoption and intention to use for community-driven contact-tracing mobile apps by millions of people globally points at the scope for wider implications in different areas, especially for the people at the lower levels of the socio-economic pyramid. there is increasing scope for the community-oriented egovernance model, which is characterized by citizen engagement and public participation in real-time (lee-geiller & lee, 2019). e-governance initiatives in the form of government-tocitizens solutions enable bridging the institutional voids and empower the underserved population by providing real-time access to information in different areas (khanra & joseph, 2019; soma, termeer, & opdam, 2016). for example, there is an opportunity to launch an e-governance community-driven mobile app in the context of agriculture, which is linked to providing real-time access to vital information for farmers on soil quality, seeds, weather forecast, seasonal crops, expected demand levels, best pricing options, etc. from agriculture experts and community members. similarly, e-governance mobile apps have huge scope for enabling access to real-time australasian journal of information systems goyal, pillai & chauhan 2021, vol 25, research article e-governance using mobile applications 22 information on skill development, employability options, etc. for the underserved segment across the country. 6.3 limitations and future research directions the current covid-19 pandemic scenario has emerged as a fairly new and unexpected situation for the world. in view of this context, there are multiple areas in which research can be further enhanced for alternative theories and learning. future research could be conducted in the following areas. first, the current research has been conducted only amongst subjects that had installed the contact tracing mobile app on their smartphones in india. however, lately, as the goi has facilitated contact-tracing mobile app services for feature phone owners through an ivrs service, future research could examine the influence of constructs like epistemic value, conditional value, convenience, privacy, and functional value on the intention to use in the context of subjects who are accessing the contact tracing mobile app service through feature phones in india and other developing nations. second, in the current research, india has been assumed to be a homogeneous cluster of citizens. however, india is a vast country with high diversity in terms of cultural, educational, religious, and societal factors. future research could examine the moderating effect of these external factors and check if the relationship between the constructs tends to change. third, the sample set in the current research is focused on just the end-users. in future research, academicians could focus on incorporating other stakeholders, such as policymakers or technology evangelists and explore the phenomenon from a multi-stakeholder perspective. fourth, as various countries have launched such mobile app initiatives to keep their respective citizens updated with the latest scenario about the covid-19, multi-country research could be attempted to understand what differences existed between countries. this could lead to a much wider generalization of the model, hence leading to wider acceptability. fifth, this study has focused just on the behavioral intentions of the end-user. in the future, researchers, especially from the technology domain, could further explore the technology architecture and privacy-related factors in a much-detailed manner. sixth, the sample set for this study was restricted to a cohort that had at least a university degree as an education background, as the researchers could not go for face-to-face survey questionnaire administration. the behavioral intention of another cohort who was less educated could be studied in the future to understand if there was a difference in their intention to adopt any contact-tracing mobile app. finally, this research did not consider the social status of the respondents (e.g., income). in the future, researchers can study the uptake of such apps among people of varying social status. references aarogya setu website (2021). retrieved january 9, 2021, from https://aarogyasetu.gov.in/ agarwal, s., & sharma, y. s. (2020). aarogya setu coming on feature phones; to cover the entire country. retrieved september 21, 2020, from https://economictimes.indiatimes.com/tech/software/aarogya-setu-coming-on-featurephones-to-cover-the-entirecountry/articleshow/75493148.cms?utm_source=contentofinterest&utm_medium=text &utm_campaign=cppst australasian journal of information systems goyal, pillai & chauhan 2021, vol 25, research article e-governance using mobile applications 23 agrebi, s., & jallais, j. (2015). explain the intention to use smartphones for mobile shopping. journal of retailing and consumer services, 22, 16-23. alhassan, a., li, l., reddy, k., & duppati, g. (2020). consumer acceptance and continuance of mobile money. australasian journal of information systems, 24. https://doi.org/10.3127/ajis.v24i0.2579 altman, i. (1976). privacy: a conceptual analysis. environment and behavior, 8(1),7-28. anckar, b., & d'incau, d. (2002). value creation in mobile commerce: findings from a consumer survey. journal of information technology theory and application, 4(1), 43-64. anderson, j. c., & gerbing, d. w. (1991). predicting the performance of measures in a confirmatory factor analysis with a pretest assessment of their substantive validities. journal of applied psychology, 76(5), 732-740. anderson, r.e., & srinivasan, s.s. (2003). e-satisfaction and e-loyalty: a contingency framework. psychology & marketing, 20(2), 123-138. angst, c. m., & agarwal, r. (2009). adoption of electronic health records in the presence of privacy concerns: the elaboration likelihood model and individual persuasion. mis quarterly, 33(2), 339-370. banerjea, a. (2020). india's aarogya setu becomes world's highest downloaded app in just 13 days. retrieved september 21, 2020, from https://www.livemint.com/technology/technews/india-s-aarogya-setu-becomes-world-s-highest-downloaded-app-in-just-13-days11586954392024.html boon-itt, s., & wong, c. y. (2016). empirical investigation of alternate cumulative capability models: a multi-method approach. production planning & control, 27(4), 299-311. cengiz, e., & kirkbir, f. (2007). customer perceived value: the development of a multiple item scale in hospitals. problems and perspectives in management, 5(3), 252-268. chauhan, s., & kaushik, a. (2016). evaluating citizen acceptance of unique identification number in india: an empirical study. electronic government, an international journal, 12(3), 223-242. chen, g., & kotz, d. (2000). a survey of context-aware mobile computing research. dartmouth computer science technical report tr2000-381. retrieved september 21, 2020, from https://digitalcommons.dartmouth.edu/cgi/viewcontent.cgi?article=4201&context=faco a chib, a., van velthoven, m. h., & car, j. (2015). mhealth adoption in low-resource environments: a review of the use of mobile healthcare in developing countries. journal of health communication, 20(1), 4-34. chin, w.w., & newsted, p.r. (1999). structural equation modeling analysis with small samples using partial least squares. statistical strategies for small sample research, 1(1), 307-341. chow, w. s., & shi, s. (2014). investigating students’ satisfaction and continuance intention toward e-learning: an extension of the expectation–confirmation model. procedia-social and behavioral sciences, 141, 1145-1149. australasian journal of information systems goyal, pillai & chauhan 2021, vol 25, research article e-governance using mobile applications 24 churchill jr, g. a. (1979). a paradigm for developing better measures of marketing constructs. journal of marketing research, 16(1), 64-73. clarance, a. (2020). aarogya setu: why india's covid-19 contact tracing app is controversial. retrieved september 21, 2020, from https://www.bbc.com/news/world-asia-india52659520 colwell, s. r., aung, m., kanetkar, v., & holden, a. l. (2008). toward a measure of service convenience: multiple-item scale development and empirical test. journal of services marketing, 22(2), 160-169. davis, f. d., bagozzi, r. p., & warshaw, p. r. (1989). user acceptance of computer technology: a comparison of two theoretical models. management science, 35(8), 982-1003. dhaggara, d., goswami, m., & kumar, g. (2020). impact of trust and privacy concerns on technology acceptance in healthcare: an indian perspective. international journal of medical informatics, 141, 104164. dinev, t., & hart, p. (2006). an extended privacy calculus model for e-commerce transactions. information systems research, 17(1), 61-80. falbo, t., kim, s., & chen, k. y. (2009). alternate models of sibling status effects on health in later life. developmental psychology, 45(3), 677-687. farronato, c., iansiti, m., bartosiak, m., denicolai, s., ferretti, l., & fontana, r. (2020). how to get people to actually use contact-tracing apps. harvard business review digital articles. retrieved september 18, 2020, from https://hbr.org/2020/07/how-to-get-peopleto-actually-use-contact-tracing-apps fishbein, m., & ajzen, i. (1975). belief, attitudes, intention, and behavior: an introduction to theory and research. reading, ma: addison-wesley. firstpost (2020). aarogya setu: lack of data privacy laws, transparent policies make app worrisome, say mit researchers. retrieved september 21, 2020, from https://www.firstpost.com/tech/news-analysis/aarogya-setu-the-mandatory-contacttracing-app-of-india-gets-reviewed-by-mit-university-here-is-what-they-think8354661.html fornell, c., & larcker, d. f. (1981). evaluating structural equation models with unobservable variables and measurement error. journal of marketing research, 18(1), 39-50. garg, s., bhatnagar, n., & gangadharan, n. (2020). a case for participatory disease surveillance of the covid-19 pandemic in india. jmir public health and surveillance, 6(2), e18795. ghosh, a., nundy, s., & mallick, t. k. (2020). how india is dealing with covid-19 pandemic. sensors international, 1, 100021. gonçalves, h. m., lourenço, t. f., & silva, g. m. (2016). green buying behavior and the theory of consumption values: a fuzzy-set approach. journal of business research, 69(4), 14841491. gu, j., xu, y. c., xu, h., zhang, c., & ling, h. (2017). privacy concerns for mobile app download: an elaboration likelihood model perspective. decision support systems, 94, 19-28. australasian journal of information systems goyal, pillai & chauhan 2021, vol 25, research article e-governance using mobile applications 25 hair, j. f., black, w. c., babin, b. j., anderson, r. e., & tatham, r. l. (2010). multivariate data analysis (7th ed.). nj: pearson prentice hall, uppersaddle river. hasan, h., & linger, h. (2020). letting the public in: dialectic tensions when governments use ict to engage citizens. australasian journal of information systems, 24. https://doi.org/10.3127/ajis.v24i0.1897 hew, t. s., & kadir, s. l. s. a. (2017). applying channel expansion and self-determination theory in predicting use behaviour of cloud-based vle. behaviour & information technology, 36(9), 875-896. hill, t., smith, n. d., & mann, m. f. (1987). role of efficacy expectations in predicting the decision to use advanced technologies: the case of computers. journal of applied psychology, 72(2), 307-313. hoehle, h., & venkatesh, v. (2015). mobile application usability: conceptualization and instrument development. mis quarterly, 39(2), 435-472. holbrook, m. b. (1994). the nature of customer value: an axiology of services in the consumption experience. service quality: new directions in theory and practice, 21(1), 21-71. immuni website (2021). retrieved february 7, 2021, from https://www.immuni.italia.it/ dashboard.html iqbal, n., & dar, k. a. (2020). coronavirus disease (covid-19) pandemic: furnishing experiences from india. psychological trauma: theory, research, practice, and policy, 12(s1), s33-s34. khanra, s., & joseph, r. p. (2019). adoption of e-governance: the mediating role of language proficiency and digital divide in an emerging market context. transforming government: people, process and policy, 13(2), 122-142. kijsanayotin, b., pannarunothai, s., & speedie, s. m. (2009). factors influencing health information technology adoption in thailand's community health centers: applying the utaut model. international journal of medical informatics, 78(6), 404-416. kilbourne, w., & pickett, g. (2008). how materialism affects environmental beliefs, concern, and environmentally responsible behavior. journal of business research, 61(9), 885–893. kumar, r., sachan, a., & kumar, r. (2020). the impact of service delivery system process and moderating effect of perceived value in internet banking adoption. australasian journal of information systems, 24. https://doi.org/10.3127/ajis.v24i0.1923 lah, f. (2008). are ip addresses “personal identifiable information”? i/s: a journal of law and policy for the information society, 4(3), 681-706. lee, e., & han, s. (2015). determinants of adoption of mobile health services. online information review, 39(4), 556-573. lee, e., han, s., & jo, s. h. (2017). consumer choice of on-demand mhealth app services: context and contents values using structural equation modeling. international journal of medical informatics, 97, 229-238. lee, m. c. (2010). explaining and predicting users’ continuance intention toward e-learning: an extension of the expectation–confirmation model. computers & education, 54(2).506516. australasian journal of information systems goyal, pillai & chauhan 2021, vol 25, research article e-governance using mobile applications 26 lee-geiller, s., & lee, t. d. (2019). using government websites to enhance democratic egovernance: a conceptual model for evaluation. government information quarterly, 36(2), 208-225. leitner, c. (2003). egovernment in europe: the state of affairs. european institute of public administration, maastricht. lim, w. m. (2018a). dialectic antidotes to critics of the technology acceptance model: conceptual, methodological, and replication treatments for behavioural modelling in technology-mediated environments. australasian journal of information systems, 22. https://doi.org/10.3127/ajis.v22i0.1651 lim, w. m. (2018b). revisiting concepts and theories in information systems and technology. australasian journal of information systems, 22. https://doi.org/10.3127/ajis.v22i0.1733 lim, w. m., lim, a. l., & phang, c. s. c. (2019). toward a conceptual framework for social media adoption by non-urban communities for non-profit activities: insights from an integration of grand theories of technology acceptance. australasian journal of information systems, 23. https://doi.org/10.3127/ajis.v23i0.1835 livemint (2020). more than 50% of india’s population 25 yrs or older: survey. retrieved september 15, 2020, from https://www.livemint.com/news/india/more-than-50-of-indias-population-25-yrs-or-older-survey-11593793054491.html lohchab, h. (2019). overall, india handset market growth to fall in 2020. retrieved september 21, 2020, from https://economictimes.indiatimes.com/tech/hardware/overall-indiahandset-market-growth-to-fall-in-2020/articleshow/72950192.cms?from=mdr long, m.m., & schiffman, l.g. (2000). consumption values and relationships: segmenting the market for frequency programs. journal of consumer marketing, 17(3), 214-232. lu, h. k., lin, p. c., & chen, a. n. (2017). an empirical study of behavioral intention model: using learning and teaching styles as individual differences. journal of discrete mathematical sciences and cryptography, 20(1), 19-41. lu, j., yao, j. e., & yu, c. s. (2005). personal innovativeness, social influences and adoption of wireless internet services via mobile technology. the journal of strategic information systems, 14(3), 245-268. mackenzie, s. b., podsakoff, p. m., & podsakoff, n. p. (2011). construct measurement and validation procedures in mis and behavioral research: integrating new and existing techniques. mis quarterly, 35(2), 293-334. o’neill, p.h., ryan-mosley, t., & johnson, r. (2020). a flood of coronavirus apps are tracking us. now it’s time to keep track of them. retrieved september 21, 2020, from https://www.technologyreview.com/2020/05/07/1000961/launching-mittr-covid-tracingtracker/ o’neill, p. h. (2020). india is forcing people to use its covid app, unlike any other democracy. retrieved september 18, 2020, from https://www.technologyreview.com/2020/05/07/ 1001360/india-aarogya-setu-covid-app-mandatory/ okazaki, s. (2008). exploring experiential value in online mobile gaming adoption. cyberpsychology & behavior, 11(5), 619-622. australasian journal of information systems goyal, pillai & chauhan 2021, vol 25, research article e-governance using mobile applications 27 pagani, m. (2004). determinants of adoption of third generation mobile multimedia services. journal of interactive marketing, 18(3), 46-59. pan, s. l., & zhang, s. (2020). from fighting covid-19 pandemic to tackling sustainable development goals: an opportunity for responsible information systems research. international journal of information management, 55, 102196. pihlström, m., & brush, g. j. (2008). comparing the perceived value of information and entertainment mobile services. psychology & marketing, 25(8), 732-755. podsakoff, p. m., mackenzie, s. b., lee, j. y., & podsakoff, n. p. (2003). common method biases in behavioral research: a critical review of the literature and recommended remedies. journal of applied psychology, 88(5), 879-903. pura, m. (2005). linking perceived value and loyalty in location-based mobile services. managing service quality, 15(6), 509-538. reuters (2020). italy launches covid-19 contact-tracing app amid privacy concerns. retrieved february 7, 2021, from https://www.reuters.com/article/us-health-coronavirus-italyapp-idinkbn2383ew rintamäki, t., kanto, a., kuusela, h., & spence, m. t. (2006). decomposing the value of department store shopping into utilitarian, hedonic and social dimensions. international journal of retail & distribution management, 34(1), 6-24. rokeach, m. (1968). a theory of organization and change within value-attitude systems. journal of social issues, 24(1), 13-33. sarkar, s., chauhan, s., & khare, a. (2020). a meta-analysis of antecedents and consequences of trust in mobile commerce. international journal of information management, 50, 286-301. saxena, k. b. c. (2005). towards excellence in e-governance. international journal of public sector management, 18(6), 498-513. sheth, j. n., newman, b. i., & gross, b. l. (1991). why we buy what we buy: a theory of consumption values. journal of business research, 22(2), 159-170. shroff, r. h., & keyes, c. j. (2017). a proposed framework to understand the intrinsic motivation factors on university students’ behavioral intention to use a mobile application for learning. journal of information technology education: research, 16(1), 143168. singh, m., & sahu, g. p. (2018). study of e-governance implementation: a literature review using classification approach. international journal of electronic governance, 10(3), 237-260. singh, s. (2020). an integrated model combining ecm and utaut to explain users’ postadoption behavior towards mobile payment systems. australasian journal of information systems, 24. https://doi.org/10.3127/ajis.v24i0.2695 smith, h. j., dinev, t., & xu, h. (2011). information privacy research: an interdisciplinary review. mis quarterly, 35(4), 989-1016. soma, k., termeer, c. j., & opdam, p. (2016). informational governance–a systematic literature review of governance for sustainability in the information age. environmental science & policy, 56, 89-99. australasian journal of information systems goyal, pillai & chauhan 2021, vol 25, research article e-governance using mobile applications 28 statistics times (2020). population of india. retrieved september 15, 2020, from http://statisticstimes.com/demographics/country/india-population.php steenkamp, j. b. e., & baumgartner, h. (1992). the role of optimum stimulation level in exploratory consumer behavior. journal of consumer research, 19(3), 434-448. stemberger, m. i., & jaklic, j. (2007). towards e-government by business process change—a methodology for public sector. international journal of information management, 27(4), 221232. the economic times (2020). rahul gandhi raises security, privacy concerns over aarogya setu app. retrieved september 21, 2020, from https://economictimes.indiatimes.com/news/politics-and-nation/rahul-gandhi-raisessecurity-privacy-concerns-over-arogya-setu-app/articleshow/75508771.cms the hindu (2020). data | how safe is aarogya setu compared to covid-19 contact tracing apps of other countries? retrieved september 21, 2020, from https://www.thehindu.com/data/how-safe-is-aarogya-setu-compared-to-contacttracing-apps-of-other-countries/article31618852.ece the quint (2020). mit researchers downgrade aarogya setu app to one star in review. retrieved september 21, 2020, from https://www.thequint.com/tech-and-auto/technews/aarogya-setu-app-gets-one-star-out-of-five-in-mit-review the times of india (2020). government launches covid-19 tracking app aarogya setu. retrieved september 21, 2020, from http://timesofindia.indiatimes.com/articleshow/ 74946627.cms?utm_source=contentofinterest&utm_medium=text&utm_campaign= cppst toots, m. (2019). why e-participation systems fail: the case of estonia's osale.ee. government information quarterly, 36(3), 546-559. tzeng, j. y. (2011). perceived values and prospective users’ acceptance of prospective technology: the case of a career eportfolio system. computers & education, 56(1), 157-165. vinson, d. e., scott, j. e., & lamont, l. m. (1977). the role of personal values in marketing and consumer behavior. journal of marketing, 41(2), 44-50. wang, h. y., liao, c., & yang, l. h. (2013). what affects mobile application use? the roles of consumption values. international journal of marketing studies, 5(2), 11-22. westin, a. f. (2003). social and political dimensions of privacy. journal of social issues, 59(2), 431-453. worldometer (2021). countries in the world by population (2021). retrieved february 20, 2021, from https://www.worldometers.info/world-population/population-by-country/ xu, h., teo, h. h., tan, b. c., & agarwal, r. (2012). research note—effects of individual selfprotection, industry self-regulation, and government regulation on privacy concerns: a study of location-based services. information systems research, 23(4), 1342-1363. yang, h.-l., & lin, r.-x. (2017). determinants of the intention to continue use of solomo services: consumption values and the moderating effects of overloads. computers in human behavior, 73, 583-595. australasian journal of information systems goyal, pillai & chauhan 2021, vol 25, research article e-governance using mobile applications 29 zainuddin, n., russell-bennett, r. & previte, j. (2013). the value of health and wellbeing: an empirical model of value creation in social marketing. european journal of marketing, 47(9), 1504-1524. zandifar, a., & badrfam, r. (2020). iranian mental health during the covid-19 epidemic. asian journal of psychiatry, 51, 101990. zhang, y., yu, p., & shen, j. (2012). the benefits of introducing electronic health records in residential aged care facilities: a multiple case study. international journal of medical informatics, 81(10), 690-704. copyright: © 2021 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v25i0.3129 http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 theoretical background and hypothesis development 2.1 the theory of consumption values 2.2 privacy theory 2.3 conceptual model constructs 2.3.1 epistemic value 2.3.2 conditional value 2.3.3 convenience value 2.3.4 functional value 2.3.5 privacy concerns 3 research methodology 3.1 the aarogya setu mobile app 3.2 measurements 3.3 data collection 4 data analysis and results 4.1 common method bias (cmb) 4.2 measurement model 4.3 structural model 5 discussion 6 conclusion 6.1 theoretical implications 6.2 practical implications 6.3 limitations and future research directions paper title (paper title style) australasian journal of information systems shi & ma 2025, vol 29, research article ai empowerment of asian-australian migrant workers received on 15 april 2024, accepted on 18 november 2024 after 4 rounds of review with karlheinz kautz as accepting editor. 1 ai empowerment of asian-australian migrant workers: progress, potentials, and patterns yingnan shi centre for business data analytics, business school university of western australia perth, australia email: yingnan.shi@uwa.edu.au chao ma research school of management australian national university canberra, australia abstract as artificial intelligence (ai), particularly generative ai, becomes integral to organizational practices, its capacity to augment human capabilities presents opportunities and challenges. we focus on the integration of ai into the workplace, emphasizing its impact on asianaustralian migrant workers—a group frequently transitioning from developing to developed economies and facing unique workplace challenges. we explore how ai may not only enhance job performance and integration by overcoming cultural and linguistic barriers but also influences perceptions of overqualification among these workers. employing psychological empowerment theory and the information systems fusion framework, alongside time-lag surveys and k-means clustering, we introduce a novel ai empowerment scale and investigate the nuanced effects of ai on immigrant workers. our findings reveal that while ai-driven psychological empowerment increases technology infusion use and overall job performance, it also underscores significant variations in how different demographic groups experience these benefits, offering new insights into the complex interplay between ai empowerment and employee perceptions. the study advances psychological empowerment and perceived overqualification research by revealing ai's varied impacts across workforce clusters. it underscores the need to manage ai carefully to avoid workplace inequalities and calls for further exploration of ai's dynamics in diverse settings. keywords: ai empowerment, ai-human collaboration, artificial intelligence, migrant workers, technology adoption. 1 introduction after years of hype and speculation, new generations of intelligence systems, especially generative ai software—as defined by dwivedi et al. (2023) as advanced technologies capable of understanding and generating human natural language using large-scale machine learning models—has finally left the realm of science fiction to become a clear organizational priority (ozturkcan & gashi nulleshi, 2024). in 2020, schwartz et al. (2020) projected that, with its unprecedented computational power and efficiency, ai would add us$13 trillion to the global economy over the next decade, redefining job roles and operations across various sectors. based on precedence research (2024), in 2024, the global artificial intelligence (ai) market was valued at usd 638.23 billion and is projected to grow to approximately usd 3,680.47 billion by 2034, reflecting a compound annual growth rate (cagr) of 19.1% over this period. its australasian journal of information systems shi & ma 2025, vol 29, research article ai empowerment of asian-australian migrant workers 2 growing influence is particularly evident in information systems (is), where ai's promise of augmenting human capabilities is being tested in real-world applications (amankwah-amoah et al., 2024; dwivedi et al., 2023; hughes et al., 2021). however, a thorough understanding of how ai applications can be effectively promoted to empower workers, especially in terms of inducing adequate behavioral responses, remains unclear (enholm et al., 2022; perifanis & kitsios, 2023). addressing this challenge is crucial for harnessing the full potential of ai within diverse workforces, particularly among migrant workers. as lindström et al. (2020) note, migrant workers transitioning from developing to developed economies find ai to be a crucial lever for economic, social, and environmental integration in their host countries. among these migrant workers, ai’s transformative opportunity is particularly significant for asian workers. first, according to hanson (2022), migrant workers, particularly ones born in east and south asia, have contributed significantly to job growth in ai-related occupations since 2000, underscoring the technology's relevance for these migrant labourers. second, recent studies demonstrate that asian individuals exhibit a higher receptivity toward ai than other ethnic groups. for instance, ho et al. (2023) found that asian individuals place a higher level of trust in emotional ai1—systems that read, classify, and interact with human emotions— compared to their western counterparts. this trust suggests a smoother integration for asian workers, marked by increased receptivity to ai-driven feedback and quicker adaptation to aihuman collaboration initiatives (mantello et al., 2023). given these cultural variations in relatedness, trust and receptivity, it is critical for ai designers to consider these preferences when developing tools that augment the capabilities of immigrant workers in ai-driven environments. we have seen discussions suggesting that ai's advanced capabilities provide substantial benefits to migrant workers. these technologies, according to hillmann (2022) and hussain et al. (2024), help bridge language barriers, offer access to previously unreachable training programs, aid in navigating complex work and residency regulations, and capitalize on unique skills and experiences to boost job performance. as emphasized by wang et al. (2023), ai-enabled technological supports are both functional and transformative, enabling immigrant workers to fully integrate and succeed in their new environments. furthermore, ai is fostering empowerment beyond productivity and efficiency gains by encouraging organizational citizenship behaviors (ocbs) like constructive voicing, which are crucial for organizational adaptability and improvement (karimikia, 2017; karimikia et al., 2020). we further argue ai-driven empowerment also reaches into the psychological domain, offering benefits beyond mere material enhancements of productivity and efficiency. to understand this empowerment, we applied spreitzer (1995)’s framework of psychological empowerment, a motivational construct manifesting through four cognitions: meaning, competence, self-determination, and impact. this framework reflects an individual’s 1 emotional ai systems can dynamically adjust responses and interactions based on real-time emotion recognition, context awareness, and personalized feedback mechanisms, enhancing usability and effectiveness by tailoring the user experience to individual emotional states and thus improving user satisfaction across various applications and industries, see ho, m. t., mantello, p., & ho, m. t. (2023). an analytical framework for studying attitude towards emotional ai: the three-pronged approach. methodsx, 10, 102149. doi.org/https://doi.org/10.1016/j.mex.2023.102149 . australasian journal of information systems shi & ma 2025, vol 29, research article ai empowerment of asian-australian migrant workers 3 perceptions of available resources and their active orientation toward their work role. we applied this construct to the context of workers' interactions with ai systems. this application helps us understand the nuanced empowerment journey ai provides: on one hand, as eilers et al. (2022) suggest, psychological empowerment driven by ai can significantly enhance employee effectiveness, transforming not only workflows but also the psychological relationship between employees and organizations, fostering a more engaged and proactive workforce. as hassandoust and techatassanasoontorn (2022) also suggest, the psychologically empowered infusion use of new technologies may not only simplify routine tasks but also deepen engagement with and utilization of new technological applications, thereby enriching the work experience and facilitating more profound interactions with technology. on the other hand, while ai can significantly improve job performance and workplace integration, it may also lead to adverse effects. some workers may feel inadequate in adapting to the changes brought by new intelligent systems (chen et al., 2022), while others may experience overqualification, resulting the improvement in job performance could paradoxically lead reduced job satisfaction and cause related issues. the above discussion reveals significant complexities in managing technological advancements to ensure they benefit rather than inadvertently harm those they are intended to aid: despite the opportunities ai presents for enhancing the integration and effectiveness of migrant workers, careful management is necessary to avoid exacerbating disparities. to achieve this, a sufficient understanding of ai’s empowerment effects on asian immigrant workers is essential. specifically, several research gaps remain unaddressed: there is a scarcity of research on how ai empowerment can trigger behavioral changes, such as infusion use, constructive voicing, job performance, and perceived overqualification. additionally, there is limited research identifying how different groups respond differently to ai empowerment. although research on ai empowerment is growing, the field still lacks a robust, validated ai empowerment scale. finally, there is a shortage of research examining the micro-level impacts of ai on ethnic minorities, which impedes the development of practical implications for ai designers and developers. these gaps must be addressed to create more inclusive, empowering, and effective ai systems that can be applied in diverse workplace contexts. using an australia-wide survey, we employ psychological empowerment theory and the is fusion framework, alongside methodologies like three-wave time-lag surveys and k-means clustering, to uncover the complex dynamics of ai in workplace settings. specifically, we address the research question: "how does psychological empowerment, facilitated by aienabled intelligent systems, influence the infusion use of ai, job performance, constructive voicing behaviors, and perceived overqualification among asian-australian migrant workers?" our study also analyzes how demographic factors and job roles intersect with ai empowerment to produce varied outcomes among different worker clusters. this study thus significantly advances the literature on ai empowerment by validating a novel ai empowerment scale and exploring how ai influences behavioral changes, such as constructive voicing and job performance, as well as its nuanced effects on perceived overqualification. additionally, we extend the discussion on how ai empowerment can shape the experiences of marginalized groups, particularly asian australian immigrants. our research offers new insights into how ai ethics can promote diversity and inclusion, providing a framework for understanding ai’s micro-level impacts on ethnic minorities in the workplace. furthermore, we provide practical guidance for implementing ai empowerment in organizational it strategies, highlighting the need for tailored support mechanisms to address the diverse australasian journal of information systems shi & ma 2025, vol 29, research article ai empowerment of asian-australian migrant workers 4 reactions of employees to ai, ensuring it serves as a unifying rather than dividing tool. we also emphasize the importance of personalized managerial interventions for successful ai integration. for ai designers and developers, we underscore the importance of incorporating ai empowerment into ux design, in addition to technical functionalities, which have been overemphasized in current literature. by providing a foundational understanding of these interactions, this research sets the stage for further exploration of how ai strategies can be tailored to enhance inclusivity and effectiveness within diverse organizational contexts. 2 literature review structured across three thematic sections, our review begins by exploring how ai, as part of intelligent systems, augments human capabilities, reshaping traditional work practices and decision-making processes. it then transitions to analyzing how psychological empowerment, catalyzed by ai, translates into various behavioral outcomes such as job performance and so on. finally, it addresses the specific context of asian australian immigrants, evaluating how ai impacts their workplace experiences and intersects with issues of ethnic identity and cultural integration. this comprehensive review aims to delineate the multifaceted effects of ai on employee empowerment, behavior, and adaptation within diverse organizational settings. 2.1 artificial intelligence and psychological empowerment we situate our study in the context of intelligent systems—in our case, artificial intelligence (ai)-enabled systems—in augmenting human capabilities and transforming human work behaviors. intelligent systems are commonly defined as a computer-assisted system that uses computational tools such as learning algorithms and statistical models to provide knowledge for problem solving and decision making (von krogh, 2018). these systems, particularly those enhanced by ai, are now not merely tools for task execution but are capable of learning from and interacting with their environment in complex, often probabilistic ways that extend beyond traditional deterministic programming (wu-gehbauer & rosenkranz, 2024). this positions these systems as active collaborators in the workplace, capable of augmenting human intelligence by managing complex and ambiguous problem-solving tasks traditionally handled by humans. this augmentation, where humans and machines collaborate closely to perform tasks (jarrahi et al., 2023), ensures that humans retain responsibility for critical decision-making even when supported by ai systems (chen et al., 2022; cheng et al., 2022; shi & deng, 2024). granted, using ai to augment human capabilities across various industries exemplifies a dual narrative of opportunities for enhancement and adaptation challenges. for instance, benbya et al. (2020) illustrate how ai-based systems enhance diagnostic accuracy by aiding physicians in interpreting complex digital images. despite these advances, there remains a significant learning curve as medical professionals reconcile these new tools with conventional microscope-based diagnostics. furthermore, ibm (2023) reports that ai facilitates a shift in business processes by simplifying routine tasks, thereby allowing employees to engage in more strategic activities that enhance business efficiency. nonetheless, this shift requires employees to develop new skills and adapt to changing roles within ai-enhanced workflows. among these cases, as sarker et al. (2024) argue, ai has the potential to augment human intellectual capacities, promoting effective collaboration within teams. yet, the integration of ai into daily operations involves adjustments in decision-making processes and team australasian journal of information systems shi & ma 2025, vol 29, research article ai empowerment of asian-australian migrant workers 5 dynamics, presenting both a strategic advantage and a challenge, which underscores the complex interaction between ai’s transformative potential and the need for adaptive strategies in its practical implementation. we attempt to understand this adaptation by borrowing and appropriating spreitzer (1995)’s "psychological empowerment" construct framework. this construct was conceptualized as an employee’s experienced psychological state based on cognitions about themselves in relation to their work role. it is a motivational construct manifested in four key cognitions: meaning, competence, self-determination, and impact. together, these dimensions create an overall sense of empowerment that reflects an individual's orientation to their work role. specifically, meaning refers to the value that an individual attaches to their work, based on their own standards and ideals. this concept was originally initiated by thomas and velthouse (1990). when employees find their work meaningful, they are more likely to be engaged and committed. developed from gist (1987) and bandura (1990)’s self-efficacy, competence is the belief in one's ability to perform tasks skillfully. this belief in personal capability is crucial for taking on challenges and persisting in the face of difficulties. self-determination was adapted from deci and ryan (2008) and reflects the degree of autonomy and freedom employees feel they have in their work. it involves having control over how tasks are carried out and the ability to make decisions regarding one's work. impact refers to the extent to which an individual feel they can influence organizational outcomes (i.e., employees who feel their actions make a difference are more likely to be proactive and involved in their work and thus be more empowered). it is adapted from ashforth (1989)’s research regarding experience of powerlessness and martinko and gardner (1982)’s research about learned helplessness. we appropriated this construct for a specific and increasingly important context—employees, particularly asian australian migrant workers, using intelligence systems (especially with the recent ai boom) in their workplace. our construct also examines this motivational perspective and adapted the aforesaid four dimensions2. moreover, structurally, this construct, as outlined by spreitzer (1995), is a second-order construct, meaning that psychological empowerment is not directly measured but is instead understood through four underlying dimensions. spreitzer (1995) also offers a structured validation process for later researchers to follow. these guidelines help subsequent researchers, like us, extend, evolve, adapt, and develop the construct, and we have seen many good instances in recent years. for instance, seibert et al. 2 particularly, the meaningfulness dimension captures the extent to which the ai system’s knowledge is perceived as personally significant and relevant to the worker’s tasks. for asian-australian immigration workers, our research suggests that the ai system provides crucial insights that are highly relevant to their work, especially in areas like decision-making, case analysis, and improving human-ai collaboration in immigration processes. competence refers to the workers' belief in their ability to use the ai system effectively to enhance their research or job-related tasks. specifically, for immigration workers, this includes feeling confident in using ai to navigate complex immigration issues and applying ai-driven insights to make informed decisions. the self-determination dimension reflects the autonomy and freedom workers experience when using ai systems. in our research on asian-australian immigration workers, this involves having the freedom to choose how they integrate ai into their workflows, including deciding on methodologies and approaches that best suit their tasks. impact refers to the perceived influence that the ai system’s knowledge has on the worker’s ability to affect outcomes in their work environment. in our context, this means how the ai system helps workers better understand human-ai collaboration dynamics, control the analysis process, and influence decisions that shape immigration policies or case outcomes. australasian journal of information systems shi & ma 2025, vol 29, research article ai empowerment of asian-australian migrant workers 6 (2004) introduced the concept of an “empowerment climate” as a higher-level construct that transformed the individual-level construct of psychological empowerment. many of these studies are in the information systems field. for instance, kim and gupta (2014) explore the role of psychological empowerment in the successful infusion of information systems into users' work lives. hassandoust and techatassanasoontorn (2022) operationalized psychological empowerment within the is context, relating it specifically to it usage. notably, their research involved end-users of crm systems from organizations in new zealand, with all measures adapted to fit the crm context. oetting (2009) adapted spreitzer (1995)’s psychological empowerment construct to the context of consumer marketing processes, establishing a new construct called empowered involvement (emi). ng and kim (2009) reinterpreted psychological empowerment by introducing the concept of "user empowerment," which reflects an individual’s orientation towards system usage. chen et al. (2022) appropriated the concept of psychological empowerment by adapting it to the knowledge context, resulting in a new construct called intelligent system knowledge empowerment. albrecht et al. (2021) introduced the concept of "pro-environmental meaningful work" (pemw), and they tested how it acts as a mediator between corporate environmental responsibility and pro-environmental employee outcomes in the workplace. zaza and junglas (2016) established a new construct called "it empowerment," focusing on how employees engage with technology in the workplace. we referenced these prior studies when implementing our appropriation of the construct. notably we also separated our constructs from other potentially similar constructs, such as ai use, ai integration, ai collaboration, and ai transformation. relevant details are provided in appendix 5 to ensure a smooth reading experience. 2.2 behavioral outcomes of empowerment: infusion use, job performance, perceived overqualification, and constructive voicing psychological empowerment leads to behavioral changes in the workplace. research has consistently shown that empowered individuals are more likely to demonstrate persistence and resourcefulness, facilitating easier adaptation during the implementation of intelligent systems. these systems tend to introduce significant shifts in business processes and employee routines (beaudry & pinsonneault, 2005), and ai-enabled systems are not an exception (afiouni & pinsonneault, 2023). while ai augmentation presents opportunities for upskilling, it also necessitates substantial changes as employees must adapt to new technologies and integrate them into their daily workflows (jaiswal et al., 2022). one significant outcome is the infusion use of information systems, which is defined by schwarz (2003) as the extent to which a system's capabilities are fully utilized, seamlessly integrated into work processes, and maximized to enhance job performance. employees are more inclined to fully utilize complex systems, such as enterprise systems, when they are confident in their skills and have autonomy in determining how to comprehensively use the system (kim & gupta, 2014). sehgal (2007) notes that when users perceive the system's usage as aligning with their personal needs and desires, they consider such use important and personally relevant. this alignment motivates users to engage deeply with the system, utilizing its features extensively to perform tasks (hsieh & wang, 2007). achieving optimal job performance is also a critical goal in management information systems research. according to methot et al. (2016), job performance is defined by the effectiveness and productivity with which an individual completes specific tasks and responsibilities associated australasian journal of information systems shi & ma 2025, vol 29, research article ai empowerment of asian-australian migrant workers 7 with their role. empowerment not only reflects an employee's beliefs based on meaning, competence, self-determination, and impact but also captures the interplay between technology and work, which includes the belief an employee holds about their work role and their motivation to engage in behaviors that enhance job performance (ng & kim, 2009). strong beliefs in competence can enhance employees' confidence in managing changes and viewing challenging tasks as attainable (herold et al., 2007; vardaman et al., 2012). similarly, as bankins et al. (2024) summarize, the integration of ai systems into organizational practices can empower employees by providing new tools and resources, reducing work demands, and enhancing job satisfaction, ultimately leading to psychological empowerment that increases job performance constructive voice, according to krefft et al. (2024), consists of workers’ voluntary expressions of specific, constructive-intended, and future-change-oriented messages in the workplace. by viewing challenges as surmountable, employees become more involved with their colleagues, reducing the likelihood of discrepancies with their role expectations and minimizing role conflict. reduced role conflict alleviates psychological strain, allowing employees to focus more on problem-solving and decision-making for work objectives (keith & frese, 2005; muraven & baumeister, 2000). a robust sense of empowerment encourages employees to anticipate and address issues proactively and cooperatively, fostering organizational citizenship behaviors such as voicing behaviors, which are crucial for organizational adaptability and improvement (karimikia, 2017; karimikia et al., 2020). notably, the aforementioned four dimensions of psychological empowerment—meaning, competence, self-determination, and impact—could significantly influence behavioral outcomes like job performance and constructive voicing. while spreitzer describes these dimensions as “collectively” fostering an overall sense of empowerment, each aspect of ai psychological empowerment plays a unique role in shaping employee engagement with tasks and interactions with ai systems, thereby driving improvements in performance and voicing behaviors: first, since the meaning dimension refers to how perceivably valuable or relevant an employee finds their work in relation to their ideals or standards, when ai systems provide perceivably meaningful insights, employees are more likely to engage deeply with the technology and view its integration into their workflow as purposeful. as employees find greater meaning in their work, they become more motivated to invest effort, thus enhancing job performance, particularly during a digital transformation process (yildiz et al., 2024). in ai-augmented environments, where the system’s outputs could directly influence decisionmaking and problem-solving, employees who attach high personal value to these systems may be more inclined to maximize their utility, leading to infusion use which, in turn, leads to better job outcomes as employees are more thorough and effective in their work. constructive voicing could be similarly impacted by meaning, as employees who perceive their work as meaningful are more likely to feel a sense of responsibility to improve processes, provide feedback, and engage in change-oriented behaviors that benefit the organization. this pathway aligns with hwang et al. (2023), who found that coaching leadership can promote employees' sense of meaningfulness in their work, though our focus here is on how ai systems facilitate this sense of purpose. second, the sense of compentency, once heightened, could promote job performance, as employees become more resilient, willing to experiment with ai tools, and capable of navigating complex tasks. immigrant workers often face various career barriers, such as social and political pressures, economic constraints, and the stress of cultural differences between their home and host countries (schultheiss, 2015). enhanced perceived australasian journal of information systems shi & ma 2025, vol 29, research article ai empowerment of asian-australian migrant workers 8 competence may help them address these barriers. when immigrant workers feel capable, they are also more likely to engage in voicing behaviors (gardner, 2010), making this a key area where large language models can particularly exert a beneficial effect. third, in ai-augmented environments, the self-determination dimension plays a critical role in empowering employees to decide how and when to use ai tools. when employees have the freedom to integrate ai into their workflows as they see fit, they experience greater satisfaction and engagement (kim & gupta, 2014), which improves performance. furthermore, selfdetermination can positively influence constructive voicing; employees with greater autonomy feel empowered to suggest process improvements and contribute to decisionmaking, fostering a work environment where they feel free to express ideas. finally, the impact dimension relates to employees’ belief that they can influence organizational outcomes (spreitzer, 1995), which is essential for fostering proactive behaviors. management literature generally agrees that employees’ self-assessment of impact significantly influences constructive voicing behaviors (burris et al., 2013): when employees feel that their input matters—and assuming other factors, such as employee-supervisor alignment, are met—they are more likely to speak up. an increased sense of impact also enhances job performance, as employees observe the direct effects of their contributions on organizational success. prior management research supports this pathway. for instance, a sense of impact, often linked to positive emotional affect, improves job performance (côté, 1999). for another, job satisfaction and organizational commitment are shown to correlate with work engagement and performance, indicating that employees’ sense of impact significantly enhances their performance (tadampali & hadi, 2017). our research adopts a new angle by examining ai empowerment, with a consideration of the impact dimension within this empowerment. perceived overqualification refers to a subjective appraisal where individuals consider themselves to possess more education, experience, or skills than required by their jobs (ma et al., 2020). research has shown that many highly educated immigrant workers often find themselves in entry-level positions or are forced to pursue new career paths upon relocating, leading to the phenomenon of overqualification (dolan et al., 2022; mackey et al., 2022). this places immigrant workers in a situation where they feel their skills and qualifications exceed the demands of their jobs. as ai technology becomes more sophisticated, workers who experience higher levels of psychological empowerment—feeling more competent, impactful, and autonomous in their roles—are more likely to let ai enable them to take on more complex tasks, fostering a sense of mastery and meaningful contribution to their work, and offering these technologies that enhance perceived competence, impact, and autonomy allows workers to reshape their roles to better align with their strengths, motivations, and passions. as demir et al. (2024) suggest, this can help employees feel more adequately challenged and aligned with their job responsibilities, thereby reducing the perception of overqualification. in this sense, ai becomes a tool for workers to reclaim a sense of value and fulfillment in their roles, mitigating the negative effects of overqualification. conversely, recent reports highlight a rise in ai-induced overconfidence (sison et al., 2023; underwood, 2024), where individuals feel overly capable due to substantial assistance from advanced intelligence systems, which could be particularly problematic when those with limited ai knowledge or experience overestimate their understanding of ai’s capabilities in augmenting their work (horowitz & kahn, 2024; zhang, 2023). furthermore, as ai increasingly takes over both routine and complex tasks previously managed by employees (huang & rust, 2018), some workers may develop a sense of insecurity and underutilization (dou, 2023). witnessing ai’s ability to handle these tasks australasian journal of information systems shi & ma 2025, vol 29, research article ai empowerment of asian-australian migrant workers 9 can amplify the perception that their own skills and qualifications are no longer fully utilized, thereby increasing feelings of overqualification. this dual effect highlights the critical need for thoughtful ai implementation, as it has the potential to either empower workers or exacerbate feelings of underutilization, depending on the specific tasks involved. 2.3 behavioral outcomes in asian australian context and ethnic identity asian australian immigrants face unique challenges in the workplace, often viewed through the lens of the "invisible model minority" stereotype (yip et al., 2021). some researchers and practitioners use the term 'bamboo ceiling,' first introduced by hyun (2005) in her book breaking the bamboo ceiling: career strategies for asians, to describe the invisible barriers that prevent qualified asian individuals from attaining leadership positions in the u.s., analogous to the 'glass ceiling' that impedes the advancement of women. some researchers have given the 'bamboo ceiling' a narrower definition: lu (2023) describes it as a phenomenon where, despite notable educational and economic achievements, asians, particularly east asian migrants, are disproportionately underrepresented in leadership roles. this underrepresentation is often attributed to pervasive stereotypes that east asians lack creativity, which is deemed essential for leadership in u.s. culture. the 'bamboo ceiling' thus encapsulates the barriers and subtle biases that prevent asians from reaching the highest ranks of professional leadership, despite their competence and qualifications. however, as ho (2020) and sum et al. (2023) have noted, this phenomenon is not exclusive to america nor limited to east asian communities. in fact, the impact of the 'bamboo ceiling' on asian-australians in the professional realm is also evident but remains poorly understood. for instance, evans (2019) noted that although asian-australians constitute 12% of the population, they hold only about 3% of senior leadership positions in public institutions and major corporations, a disparity that is exacerbated by racial discrimination, cultural stereotypes, and a lack of institutional commitment to cultural diversity. these figures contrast with 7% and 6% in the u.s., as reported by hemmige et al. (2023) and ruiz et al. (2023), respectively. liu et al. (2024) provide a detailed analysis of these nuances through the lens of "precarious multiculturalism," which illuminates the intermittent and conditional nature of racial inclusion in australia. as morris (2008) suggests that while the stereotype (e.g., the "model minority"—quiet, hardworking, studious, and compliant) has been praised for demonstrating the success of asian communities, it also masks a variety of difficulties including navigating cultural differences and confronting career stagnation. despite the perception of asians excelling academically and professionally, they remain underrepresented in leadership roles due to cultural stereotypes and racial discrimination (ruiz et al., 2023). notably, ai has the potential to significantly influence immigrant workers by helping them overcome these cultural barriers and triggering beneficial work behavioral changes. studies by sumi et al. (2024) and rude and giesing (2022) demonstrate how ai aids immigrants in utilizing their unique skills, leading to better integration. additionally, yan and grossman (2023) discuss the role of icts in building social capital among immigrants, facilitating their adaptation and integration by enhancing communication and building networks. such technological support is crucial as it not only helps immigrants integrate into new environments but also maximizes their potential to contribute effectively within their workplaces. adding to the complexity is the notion of ethnic identity, which is defined as increasing an individual's receptivity to social influence from members of the same ethnic group while australasian journal of information systems shi & ma 2025, vol 29, research article ai empowerment of asian-australian migrant workers 10 decreasing their receptivity to social influence from non-group members (hekman et al., 2009). some research highlights that high ethnic identity among employees can foster positive behavioral outcomes such as self-affirmation values and psychological empowerment, influencing how immigrant workers perceive and control their environment, including their interaction with technology (henze et al., 2002; molix & bettencourt, 2010). workers who feel a strong connection to their ethnic identity may derive confidence and empowered feelings, perceive a higher status, and are empowered to engage more proactively with technological tools, overcome adaptation challenges, and enact more inter-ethnic group and in-group helping behaviors (abad merino, 2014). overall, the empowerment afforded by ai warrants thorough investigation due to its nuanced distinctions, which remain underexplored in systematic and detailed research. ai's role in the workplace is evolving, and its impact on psychological empowerment and infusion use needs deeper examination, particularly as it relates to job performance, organizational citizenship behaviors (ocbs), such as constructive voicing behaviors, and perceived overqualification. the role of ai in the workplace is still a complex interplay of enhancing capabilities while also potentially reinforcing existing inequalities, requiring a nuanced understanding and proactive management to harness its benefits while mitigating its challenges. 3 method 3.1 participants in partnership with octopus group, a reputable australian survey firm, we secured a participant pool validated through the following mechanisms, ensuring reliability for our data collection: first, octopus group's dedicated team and systematic recruitment processes guarantee that participants meet the required demographic and psychographic criteria. they mandate australian-based mobile numbers for participation and require sms and email confirmation, ensuring the authenticity of responses. second, their high reward rates attract a large and diverse panel, resulting in balanced representation across age, gender, education, and ethnicity. moreover, to further ensure data quality, we implemented several filtering methods. we screened out subjects who did not complete all three waves of our study and those who completed the survey in less than two minutes. additionally, we included a question regarding participants' interaction with ai, derived from the study by sowa et al. (2021), which defines human-ai interaction across four tiers. at the basic level 1, the interaction between humans and ai is minimal, where human operators either compete against or operate separately from ai. in level 2, described as complementary collaboration, humans and ai each handle tasks suited to their specific strengths. at level 3, termed dependent collaboration, there is a symbiotic relationship where ai sometimes relies on human input for making decisions. level 4, termed as hybrid collaboration, sees ai as an integral part of human cognitive processes, working in unison in what is referred to as a ‘centaur’ setup. we introduced an initial level 0 (i.e., individuals who do not engage with ai at all); participant who chose “level 0” were not included in our analysis. detailed procedure checks were included in appendix 4. we issued 600 questionnaires and, following screening and participant drop-off, ended up with 525 valid responses, equating to a retention rate of 87.5%. this sample size is sufficiently larger than other recent similar research by chen et al. (2022) and hassandoust and australasian journal of information systems shi & ma 2025, vol 29, research article ai empowerment of asian-australian migrant workers 11 techatassanasoontorn (2022). all respondents were asian migrant workers at a company based in australia. demographic factor mean range coding age 2.51 [0, 6] 1 = less than 21; 2 = 22 – 30; 3 = 31 – 40; 4 = 41 – 50; 5 = more than 50. gender -0.13 [-1, 1] -1 = female, 0 = others (including “prefer not to say”), 1 = male. education 3.02 [0, 5] 0 = high school and lower; 1 = diploma; 2 = bachelor’s degree; 3 = master’s degree; 4 = doctoral degree. tenure 41.17 [0, 390] measured in months. table 1 demographics statistics 3.2 procedure we employed a three-wave time-lag survey design in this research. this design has the potential to provide causal insights that are comparable to those derived from experimental designs. as wang et al. (2017) explains, the time lag between measurements is crucial for addressing the “issues of causality” (if the lags are neither too long nor too short), similar to how experimental designs utilize time precedence to strengthen causal inferences. additionally, this design facilitated the tracking of trends over time (cohen et al., 2013) and helped mitigate common method bias (hedman, 1972; maynard et al., 2014; podsakoff et al., 2003; vomberg & klarmann, 2022). collaborating with octopus, we acquired data from a diverse and australian nationwide sample. there was an approximate one-month interval between each wave of data collection. this time interval strikes a critical balance by being far enough separated to reduce common method bias but close enough to maximize retention of participants. and it has been adopted by prior studies in both management and information systems fields, such as y. li et al. (2022), fiori et al. (2015), gorostiaga manterola et al. (2022), and bhattacherjee and premkumar (2004). our sample exhibited considerable diversity in terms of their demographic characteristics (see table 1). the timing for distributing the key measurements can be found in table 3. demographic variables were collected across the three time points. note that we tolerated a certain level of missing data, and, when that happens to a numeric variable, we imputed them with its mean. 3.3 measures prior to elaborating on our primary metrics, we ascertained the reliability and validity of the core variable, "ai empowerment (aiemp)," through the utilization of python (version 3.9), with the aid of the factor_analyzer and semopy packages. for this construct, we adapted the psychological empowerment framework developed by spreitzer (1995) to suit the context of ai empowerment. the original framework, which comprises four dimensions, has seen widespread application across numerous pertinent studies within the field of management information systems. initially, we conducted a kaiser-meyer-olkin (kmo) test on our data, yielding an adequacy value of 0.9469. based on criteria posited by shrestha and statistics (2021), this is adequate. australasian journal of information systems shi & ma 2025, vol 29, research article ai empowerment of asian-australian migrant workers 12 we then tested the cronbach’s α values for all dimensions of our adapted construct and found they all were high, exceeding 0.86 (meaningfulness: 0.864; competence: 0.907; selfdetermination: 0.889; impact: 0.900): all well above the recommended threshold of 0.707 as suggested by nunnally (1978). we subsequently used an exploratory factor analysis (efa) to further explore the adapted construct. specifically, we used the oblique version of geomin rotation with minimum residual methods. we chose these methods is because our data did not pass normality tests (see appendix 2) and these methods do not assume normality and can be more robust in smaller samples compared to other methods such as maximum likelihood and maximum likelihood estimation. four components collectively accounted for 80.71% of the total variance. all items loaded more highly on their own constructs than on other constructs, supporting a four-factor solution for aiemp. overall, the results suggest adequate psychometric properties for measurement of the dimensions. we reported the loadings for the proposed four factors in table 2. we further conducted a two-step efa to explore the hierarchical factor structure underlying the aiemp items. we calculated the average score for items under each dimension and performed efa again. a single overarching component emerged that explains 78.02% of the variance in the scores derived from the second-step efa components, supporting that a second-order structure for overarching construct (i.e., aiemp). variable factor 1 factor 2 factor 3 factor 4 aiemp_meaningfulness_1 0.699 0.093 0.105 0.016 aiemp_meaningfulness_2 0.719 0.121 0.081 0.096 aiemp_meaningfulness_3 0.536 0.184 0.037 0.160 aiemp_competence_1 0.220 0.574 0.201 0.033 aiemp_competence_2 0.116 0.738 0.087 0.099 aiemp_competence_3 0.079 0.518 0.095 0.310 aiemp_sd_1 0.103 0.249 0.228 0.402 aiemp_sd_2 0.087 0.053 0.121 0.718 aiemp_sd_3 0.031 0.174 0.288 0.528 aiemp_impact_1 0.096 0.142 0.596 0.193 aiemp_impact_2 0.069 0.092 0.711 0.114 aiemp_impact_3 0.111 0.084 0.451 0.332 note: n = 525, missing variables (n=6) were imputed with mean. table 2 cross-loadings of the aiemp items we summarised the factors in table 3. note that, besides ai empowerment, other measurements that we employed in this study had been used in prior management and information system researches, including jones et al. (2002), schwarz (2003), sundaram et al. (2007), norton et al. (2015) and babalola et al. (2019). australasian journal of information systems shi & ma 2025, vol 29, research article ai empowerment of asian-australian migrant workers 13 factor description adapted from ai empowerment (aiemp) [time 1] ai empowerment (aiemp) is measured on a two-level scale, with a set of items assessing how fully individuals utilize ai to enhance their job performance, integrate ai into their work processes, and maximize the potential of ai to support and enhance their work activities (i.e., the four dimensions, including meaningfulness, competence, self determination, and impact). spreitzer (1995) ethnic identity (ei) [time 1] ethnic identity refers to an individual’s sense of belonging, attachment, and identification with a specific ethnocultural group (hekman et al., 2009) and is measured through five items that explore personal identification with asian ethnicity, assessing how criticism or praise of asians feels personal, the use of inclusive language when referring to asians, a shared sense of achievement with other asians, and feelings of embarrassment in response to negative media portrayal of asians. besco (2015) and mael and tetrick (1992) infusion use (iu) [time 2] infusion use is the extent to which individuals utilize the system's capabilities to their fullest potential, seamlessly integrate it into their work processes, and maximize its potential to support and enhance their job performance. there were 3 items used to measure this capability. chen et al. (2022), sundaram et al. (2007) and schwarz (2003) job performance (jp) [time 3] job performance is defined as the effectiveness and productivity with which an individual completes the specific tasks and responsibilities associated with their work role. it is measured using self-reported ratings of five items that assess task-focused contributions, including the completion of assigned duties and the fulfillment of responsibilities specified in the job description. these items are designed to evaluate both the quality and quantity of work performed, reflecting how well an employee aligns with and contributes to organizational goals. methot et al. (2016) perceived overqualification (po) [time 3] perceived overqualification is a subjective appraisals where individuals consider themselves possessing more education, experience, or skills than required by their jobs (ma et al., 2020). perceived overqualification was measured with the nine-item scale and has been used frequently in various overqualification research studies (liu et al., 2015). sample item includes “my job requires less education than i have.” maynard et al. (2006) constructive voicing (cv) [time 3] constructive voice is defined as the voluntary expression of specific, constructive-intended and future-changeoriented messages in workplaces; there are 5 items and, similarly, all the items are measured on a 7-point likert scale. krefft et al. (2024) table 3 measurement summary 3.4 analyses & results the clustering method that we used in this study was based on k-means clustering algorithm. recent advancements in data analysis and clustering algorithms have significantly impacted various scientific and industrial fields by enabling more sophisticated data handling and decision-making processes. among the various clustering algorithms, the k-means clustering australasian journal of information systems shi & ma 2025, vol 29, research article ai empowerment of asian-australian migrant workers 14 algorithm stands out due to its simplicity, efficiency, and broad applicability, and the sample size also makes this study be suitable for conducting a k-mean clustering analysis (ikotun et al., 2023). figure 1 the elbow method plot we firstly adhered to the traditional approach (d’silva & sharma, 2020) by generating an elbow method plot. as observed in figure 2, the plot illustrates the inertia scores for various numbers of clusters, ranging from 1 to 10. the "elbow point," where the curve begins to flatten, appears around 3, 4, 5, or 6 clusters. subsequently, we compared the silhouette scores among these configurations and identified that clusters 3, 4, and 6 were potentially viable. after a more detailed examination of the principal component analysis (pca) processed scatter plots for these three options (refer to appendix 1), we determined that the 6-cluster solution was the most logical choice. figure 2 silhouette scores comparisons among 1-10 clusters australasian journal of information systems shi & ma 2025, vol 29, research article ai empowerment of asian-australian migrant workers 15 we calculated and presented the distances between cluster centres and the anova results across clusters in table 4 and table 5, respectively. the anova results show significant differences in the clustering variables across the clusters models, indicating that the clusters are distinct in terms of the variables used. cluster 0 1 2 3 4 5 n 0 0.00 114 1 2.68 0.00 99 2 2.96 2.90 0.00 104 3 3.83 4.15 3.06 0.00 68 4 2.95 3.24 2.03 2.64 0.00 106 5 2.34 2.53 2.28 2.73 2.35 0.00 34 table 4 distances between final cluster centers factor ss t p-value aiemp 214.89 66.43 <0.001 ei 221.76 80.27 <0.001 iu 317.17 78.89 <0.001 po 352.43 72.14 <0.001 pe 257.51 133.79 <0.001 cv 356.91 85.27 <0.001 note: n =525; degree of freedom=5; ss = sum of square; aiimp = ai empowerment, ei = ethnic identity, iu = infusion use, po = perceived overqualification, jp = job performance, cv = constructive voicing table 5 anova results cluster aiemp ei iu po jp cv age gend er educ atio n tenu re 0 4.93 5.45 4.92 5.64 6.19 5.50 2.58 -0.08 3.16 35.71 1 5.23 5.22 5.20 3.02 6.26 5.79 2.51 -0.16 3.04 37.66 2 4.44 4.66 4.14 3.95 4.45 4.30 2.25 -0.09 2.96 38.05 3 2.91 4.16 2.38 4.29 6.30 4.68 3.02 0.04 2.81 63.21 4 4.13 4.80 4.12 4.24 6.13 3.26 2.49 -0.32 3.00 41.85 5 4.33 3.57 4.56 4.47 6.12 5.18 2.44 -0.12 3.04 41.08 table 6 means of 6-cluster model we reported the results of the clustering analysis in table 6, in which the model delineates profiles of asian immigrant workers in terms of their responses to ai empowerment and its perceived impact on their work lives. with this model, we observe a complex landscape where ai-related variables and self-perceptions vary significantly across clusters. the clusters illuminate differences in perceived overqualification, ai empowerment, and behavioral outcomes, indicating that the relationship between ai integration and worker perception is multifaceted. in cluster 0, these individuals perceive a relatively high level of overqualification and moderate ai empowerment. their use of technology and performance are quite high, suggesting they are successfully adapting and utilizing ai in their work. they engage actively in constructive voicing, indicative of a proactive workplace attitude. their ethnic identity is strongly felt, which might contribute to their confidence and status, empowering australasian journal of information systems shi & ma 2025, vol 29, research article ai empowerment of asian-australian migrant workers 16 them further. they are younger on average, with a slight female skew, well-educated, and have moderate tenure. per cluster 1, this group reports a high levels of ai empowerment, infusion use, and constructive voicing behaviors, suggesting that they may feel very enabled by ai to enhance their work roles, leading to high performance. their relatively strong ethnic identity could also be providing a significant sense of empowerment. they are relatively younger, more likely to be female and have also a moderate tenure, similar to cluster 0 but with a significantly lower perceived overqualification. per cluster 2, members of this cluster show relatively low level of perceived overqualification, ai empowerment, and infusion use, paired with relatively lower performance and constructive voicing behaviors. this suggests a less proactive engagement with ai tools or possible adaptation challenges. their ethnic identity is strongly felt, which may influence their work experience and adaptation process. they tend to be slightly less educated and have a lower gender skew towards females compared to clusters 0 and 1, with similar age and tenure. individuals in cluster 3, which is a small-size group, show moderate levels of perceived overqualification, the lowest ai empowerment, and infusion use among all clusters, yet their performance is quite high. their engagement in constructive voicing is strong, suggesting that while they may not feel as empowered by ai, they still perform well and contribute to the workplace. they have the highest average age and tenure, which could indicate more experience in the workforce or possibly a resistance or refrain from ai adaptation. their ethnic identity is also felt less strongly compared to other clusters. per cluster 4, this cluster has moderate perceived overqualification and ai empowerment, with average infusion use and performance. however, their constructive voicing behaviors are lower, indicating a potential hesitance or barriers to fully engaging in workplace dialogue or technology adaptation. they are the oldest group on average, have a notable female presence, are well-educated, and possess the highest tenure, suggesting a matured workforce. cluster 5 has the lowest number of members. it exhibits moderate perceived overqualification and ai empowerment, good infusion use, and very high performance. their constructive voicing is also high, reflecting effective workplace communication and engagement with ai. they have a moderate age and tenure and are likely to be educated, with a balanced gender distribution. the six clusters reflect a spectrum of adaptation and empowerment experiences among asian immigrant workers, influenced by their interaction with ai tools, their sense of ethnic identity, and control variables such as age, gender, education, and tenure. these findings could guide targeted interventions and supports to enhance ai integration and overall satisfaction in the workplace. we believe conducting post-hoc analysis is a crucial step in exploring and understanding the differences between clusters. this method validates the initial clusters and provides deeper insights into the variability within the data that may not be apparent from the primary clustering. by examining these differences, researchers can confirm the distinctiveness of clusters, uncover hidden patterns, and draw more informed conclusions. this approach is commonly used in empirical studies, such as those by balaban et al. (2023) and stewart et al. australasian journal of information systems shi & ma 2025, vol 29, research article ai empowerment of asian-australian migrant workers 17 (2006), who utilized post-hoc analysis to identify meaningful groupings and inform future research directions. 3.5 post-hoc analysis figure 3 means of ai-human collaboration levels according to clusters in the first post hoc analysis, we tried to find a further explanation on the nuanced difference between cluster 0 and 1. although the key variables that we used for clustering and the demographical variables of the two cluster groups were very similar, significant variations were observed between the two identified clusters regarding their ai-human collaboration levels at time 1, which have been categorized into four stages as defined by sowa et al. (2021)3. a one-way anova test was conducted to compare the ai-human collaboration levels between the clusters, revealing a statistically significant difference (mean difference = 0.027, p = 0.019). this analysis suggests that individuals in cluster 0 predominantly engage in lower levels of ai-human collaboration, where the role of ai is less integral. interestingly, this relatively superficial level of ai interaction tends to coincide with a stronger perception of overqualification. conversely, individuals in cluster 1, who experience enhanced levels of aihuman collaboration, heightened ai empowerment, more effective technology infusion, and superior self-reported job performance, exhibit a lower level of perceived overqualification. this correlation may be attributed to several factors. firstly, the enhanced ai empowerment in cluster 1 likely results in more positive performance feedback. such feedback can affirm an 3 please note that the attention-checking choice (level 0) is not considered a formal level in this test. australasian journal of information systems shi & ma 2025, vol 29, research article ai empowerment of asian-australian migrant workers 18 employee's perception of the suitability of their role and the adequacy of their skill utilization. the positive reinforcement and tangible outcomes of their efforts serve to align their skills with job demands more closely. additionally, heightened ai empowerment may imply that employees are not merely users of ai tools but are actively involved in their development, optimization, or detailed interaction. this deep engagement may increase their perception of being integral to critical aspects of organizational operations and innovation, enhancing their sense of skill utilization and ongoing development, thereby mitigating feelings of overqualification. this distinguished from cluster 3 which presents a unique case wherein moderate levels of perceived overqualification are observed, despite the lowest levels of ai empowerment and infusion use among all clusters. this cluster also displays the lowest level of ai-human collaboration and is characterized by the highest average age and tenure. these attributes may suggest a greater amount of workforce experience, which could concurrently indicate a potential resistance to, or a slower adaptation of, ai technologies. additionally, the less strongly felt ethnic identity in this cluster could influence their integration and interaction within the technological and social fabric of the organization, potentially impacting their engagement with ai technologies. overall, the advanced integration of ai in cluster 1 appears to create a work environment where employees perceive their roles as adequately challenging and commensurate with their skills, thus reducing the incidence of perceived overqualification. this dynamic underscores the importance of strategic ai deployment in aligning job demands with employee capabilities, fostering job satisfaction, and enhancing organizational performance. 4 discussion ai empowerment is increasingly becoming a focal point of discussion within both academic and industrial circles. this shift is propelled by the advancements in new waves of ai applications, especially generative ai technologies, that have reached a threshold where their potential for mass and effective collaboration with humans in general tasks unveils new vistas of efficiency and possibility (benbya et al., 2024; kanbach et al., 2023; shi & deng, 2024). the interplay of ai's prowess in automation and data analytics with human creativity and intuition offers fertile ground for innovation. through ai collaboration, the streamlining of workflows, acceleration of creative processes, and amplification of productivity in creative professions are feasible (amankwah-amoah et al., 2024). however, the potential of ai to automate tasks within these industries also brings into sharp focus the risks of job displacement and necessitates a thorough exploration of how human roles can evolve to work symbiotically with ai technologies (mogaji et al., 2024). as echoed by scholars like shukla (2023), davenport and miller (2022), and ghaffari et al. (2024), the field is currently fraught with uncertainties, including the balance of automation with the human touch, ethical use, and the impact on job roles (amankwah-amoah et al., 2024). this study, therefore, contributes a timely exploratory discussion to this burgeoning discourse. adding to the importance of our research is the focus on the cohort of asian australian immigrants. despite being considered a model minority with a significant representation in stem and computing and algorithmic-related areas (rude & giesing, 2022; yan & grossman, 2023), asian australian immigrant workers often encounter the 'bamboo ceiling' phenomenon (hwang & beauregard, 2022; lee & zhou, 2015; lu, 2022; yip et al., 2021). they are susceptible australasian journal of information systems shi & ma 2025, vol 29, research article ai empowerment of asian-australian migrant workers 19 to becoming the invisible workforce within their professional environments (jun et al., 2023). the psychological empowerment afforded by ai has the potential to assist these workers in breaking through the transparent glass barriers, especially when these barriers are entrenched in domains where ai, notably generative ai, excels (sumi et al., 2024). nonetheless, there exists the possibility that while job performance improves, employees might concurrently experience a sense of perceived overqualification. this perception can engender job dissatisfaction (kaymakcı et al., 2022), among other subsequent issues. should this be true, such a phenomenon deserves our vigilant attention. underpinned by the psychological empowerment theory, the is fusion framework, and utilizing time-lag surveys alongside k-means clustering methods, our exploratory study ventures into these significant discussion points. we first extended the classical psychological empowerment scale to formulate our novel ai empowerment scale, which we have rigorously tested for validity and reliability. subsequent to our time-lagged survey, we validated the classical resource-based view that ai-enabled psychological empowerment can incite infusion use among asian australian migrant workers. the data, segmented through time-lag research, revealed that groups with higher ai-enabled psychological empowerment also reported higher infusion use, and vice versa. similarly, we confirmed the anticipated effect of infusion use on job performance. on another note, our findings uncovered a potential link between perceived overqualification and the diversity in ai empowerment. our data revealed two distinct clusters of participants who were comparable in various dimensions (age, education level, ai empowerment, high infusion use levels, and significant job performance), yet one cluster felt overqualified, while the other did not report heightened overqualification-related perception. further scrutiny suggests that the differential in ai-human collaboration levels could explain the reasonable differences in perceived overqualification observed. moreover, we identified a smaller-sized cluster of individuals predisposed to stronger perceived overqualification tendencies alongside higher self-reported job performance, independent of ai-enabled empowerment. our results also indicated that these individuals tend to be older, predominantly male, and relatively less educated. intriguingly, they also reported higher levels of constructive voicing behavior, which we speculate may be associated with the specific industries they are concentrated in. by contrast, cluster 2 and cluster 4 present varied profiles. cluster 2, with the youngest average age across all clusters, reported moderate ai empowerment levels but did not exhibit high job performance or perceived overqualification. cluster 4 reported relatively higher job performance, yet their voicing behaviors were markedly lower, and cluster 4 comprised a higher proportion of female members. this also attracts future further research. lastly, the connection between ethnic identity and the three outcome variables measured at time 3 (i.e., job performance, constructive voicing, perceived overqualification) appeared to be complex. this suggests that ethnic identity may have a negligible impact, or there may exist intricate moderated mediation effects, which warrant further empirical investigation. 4.1 theoretical contributions this study makes a significant contribution to the literature on empowerment, particularly ai empowerment, by validating a potentially widely-applicable scale that clarifies the nuanced roles ai can play in enhancing the capabilities of the human workforce. while ai empowerment is an emerging research area, and recent studies have explored this concept, we identified certain gaps that our study addresses. for example, t. j.-j. li et al. (2022) australasian journal of information systems shi & ma 2025, vol 29, research article ai empowerment of asian-australian migrant workers 20 examined ai empowerment in the context of gig workers, proposing a bottom-up approach where ai-enabled intelligent assistants empower workers with task planning and decisionmaking. although their study offers valuable insights into ai-driven empowerment, it does not establish a measurable construct for ai empowerment, nor does it delve into the psychological foundations of empowerment. for another, reis et al. (2021) introduced "aibased user empowerment" in the context of visual big data analysis, aiming to enhance system usability and boost users' self-confidence through adaptive interfaces. while their focus on interface design is important, their research lacks an explicit framework to quantitatively capture empowerment. our paper expands on these ideas in a more theoretically robust manner by adding a more systematic, empirically-valid research paradigm. this study also breaks new ground by linking ai empowerment to higher levels of job performance in different ways, suggesting that empowerment goes beyond subjective perception and has tangible impacts on operational outcomes. our research extends the literature on empowerment-induced behavioral change in two keyways. first, we confirm prior studies, such as hassandoust and techatassanasoontorn (2022), which demonstrate the effect of psychological empowerment on information system infusion use and job performance, and show that ai empowerment can similarly induce comparable behavioral changes. moreover, we offer a nuanced discovery by exploring how micro-level interactions with ai technologies influence perceptions of overqualification. previous studies in this field have primarily focused on macro-level factors leading to overqualification (liu et al., 2022; sparreboom & tarvid, 2016; syed, 2008). however, as highlighted by liao et al. (2024) research on overqualification is evolving to recognize that it is not only caused by macro-level misfits but also shaped by the day-to-day technological empowerment of employees. in line with this, our findings suggest that individuals’ai empowerment plays a pivotal role in shaping perceptions of overqualification, following two distinct paths: some workers felt overqualified while reporting higher-than-average job performance, while others reported lower perceptions of overqualification. this aligns with theoretical literature: one stream suggests ai can enable workers to take on more complex tasks, fostering a sense of mastery and meaningful contribution to their work. technologies that enhance perceived competence, impact, and autonomy allow workers to reshape their roles to better align with their strengths, motivations, and passions, as suggested by demir et al. (2024); some may indeed experience ai-induced overconfidence (sison et al., 2023; underwood, 2024), and thus they perceive more overqualification. furthermore, our results revealed the group of respondents who experience higher-than-average levels of perceived overqualification tends to be older, predominantly male, and relatively less educated. interestingly, they also reported higher levels of constructive voicing behavior. we speculate that this behavior may be influenced by underlying factors such as cultural norms, which foster environments where speaking up and providing input are encouraged for such asian migrant workers as pre-described, regardless of formal qualifications. these individuals may come from sectors where proactive communication and engagement are highly valued, motivating them to voice their opinions—an area that remains underexplored in the existing literature. our empirical results also suggest employees’ current level of ai-human collaborations, as measured by sowa et al. (2021)’s framework, may also have a critical role on affecting the perceived overqualification, thus we also introduce another promising theoretical direction to link a hot information system topic with a promising human resource topic organically together. australasian journal of information systems shi & ma 2025, vol 29, research article ai empowerment of asian-australian migrant workers 21 third, with the literature on ai ethics in ethnic research, our study made two key advances. first, ai empowerment, through its transformative capabilities, holds significant potential in promoting diversity and inclusion. our research extends this stream of work, such as almufareh et al. (2024) (ai-enabled inclusive technologies), austin and holloway (2022) (aienabled assistive technologies), and pritiprada et al. (2024) (the role of new technologies, including ai, blockchain, and the metaverse, in empowerment of promoting diversity), by showing how ai can help move asian australian immigrants from marginalized "invisible" actors and break through the "bamboo ceiling." in this regard, we believe our study serves as a starting point for a deeper understanding of how ai, while functioning as a tool for inclusion, also poses ethical dilemmas that could further marginalize already 'invisible' groups. thus, we build on chi et al. (2021)’s work by offering new insights into how ai ethics is being reconfigured to make diversity and inclusion more actionable for practitioners, ensuring that marginalized ethnic groups are not sidelined. moreover, we observed the complex nature of ai’s role in the workplace empowerment of asian australians. not all cohorts benefit equally and fairly from the advances in newer generations of ai systems. some individuals, as we also observed, may be significantly left behind, which, as lin and chen (2022) also noted, suggests that ai can function both as a tool for empowerment and as a source of potential marginalization. this dual perspective contributes to the ongoing discourse on ai ethics, as our results demonstrate the nuance that, while ai has the capacity to enhance the roles of some marginalized ethnic groups, its benefits are not equally experienced. although certain groups may gain empowerment, there is a danger that others may remain excluded or further marginalized. furthermore, another stream of research (preminger, 2020), has expressed concerns that ai systems may mask underlying ethnic hierarchies and deny collective rights, further marginalizing already disadvantaged groups. our findings help to mitigate this fear to some extent, as we observed that ai empowerment, particularly in a large cluster of young asian migrant workers, triggers positive behavioral changes in the workplace (e.g., voicing behaviors). this may help these workers break the invisible ceiling and counter stereotypes of being quiet, passive, and non-confrontational (ruttiman, 2009). our research places a spotlight on the ethical imperative for inclusivity within ai development and implementation. thus, this theoretical advancement bridges a gap in the current literature by emphasizing the microlevel impact of ai on ethnic minorities, particularly in how it may enhance or diminish their visibility and agency in service sectors. 4.2 practical contributions the first practical contribution of this study is our study helps the introduction of ai empowerment in designing organizational it strategies that effectively promote ai applications. prior research by monod et al. (2024), reis et al. (2021), gladden et al. (2022) and usmani et al. (2023) has already emphasized the importance of incorporating empowerment in practice. however, our research goes further by offering companies a concrete ai implementation framework. specifically, to promote ai empowerment, four key components should be prioritized: meaningfulness, competence, self-determination, and impact. additionally, our findings suggest that the success of such ai empowerment strategies depends on the specific cohort of individuals it targets and how the ai implementation project is introduced and supported. this underscores the need for personalized support paths and psychological mechanisms to ensure employees feel empowered by ai, rather than alienated by its integration, as mittal et al. (2023) emphasize. australasian journal of information systems shi & ma 2025, vol 29, research article ai empowerment of asian-australian migrant workers 22 second, employees' behavioral reactions to ai empowerment are diverse, and distinct clusters exhibiting different reactions are likely to emerge as new ai strategies are implemented. due to this high degree of variability, companies should tailor their strategies and provide not only technical but also psychological support mechanisms to help employees feel more empowered by ai. current research often focuses on the role of managers in fostering ai-human collaboration by conducting needs analyses and offering tailored technical ai literacy training (dixit & maurya, 2021). our study further argues that incorporating a tailored psychological empowerment approach can help mitigate the risk of creating segments within the company where ai becomes a dividing factor rather than a unifying and empowering tool. building on the initial point about the necessity for a strategic approach to ai promotion, it becomes clear that merely adopting new technologies does not guarantee an augmentation of it-human collaboration. this is corroborated by prior management information systems literature (harper et al., 2004) and is clearly applicable to ai applications. in fact, as demonstrated by sowa et al. (2021) and also the empirical results of this study, low levels of ai-human collaboration might lead to unintended side-effects. therefore, appropriate managerial intervention becomes critical, and this intervention, as sowa et al. (2021) introduce, should focus on several key areas, such as adequate training, user experience (ux) design, etc., to foster effective ai-human collaboration. beyond the aforementioned training program designs, an ai-empowerment-driven it strategy can also guide user-centric design approaches. our ai empowerment framework can thus be also applied to participatory approaches, leading to the creation of user-friendly interfaces that align with employees' specific needs and workflows. this approach recommends that future generations of workplace ai tools significantly enhance users'—particularly migrant workers'—perceived meaningfulness, competence, self-determination, and impact on their work. moreover, interventions should consider the emotional and psychological aspects of ai integration. as gladden et al. (2022) and ghaffari et al. (2024) argue, we suggest managers can introduce change management strategies that address the common fears and misconceptions about ai, such as job displacement and loss of control. they should emphasize the value of ai as a tool that enhances employee capabilities and productivity rather than a replacement for human intelligence. to minimize the ai divide within a company, managers should also develop differentiated strategies that cater to the diverse reactions and readiness levels across employee clusters. this may include personalized support paths, such as mentoring for those less comfortable with ai and advanced projects for those who are more tech-savvy. finally, management should establish clear guidelines and ethical standards for ai use to ensure that the technology is applied responsibly and inclusively, considering the impact on all stakeholders. regular assessments of ai's impact on job roles and organizational structure will help to identify areas where human roles can evolve alongside ai advancements, thereby promoting a culture of continuous learning and adaptation (ibrahim & rashad, 2024; wilson & daugherty, 2018). in this regard, our study also provides a foundation for understanding the complex interplay between ai advancements and the inclusivity of ethnic minorities, highlighting the need for digital corporate responsibility. it calls for service organizations to actively engage in ethical decision-making processes that recognize the dignity and rights of all employees, particularly those who are at risk of remaining 'invisible' in rapidly digitizing environments. australasian journal of information systems shi & ma 2025, vol 29, research article ai empowerment of asian-australian migrant workers 23 4.3 limitation and future suggestions a potential limitation of our study is the exclusive reliance on self-reported measures for all variables. since our research focused specifically on asian racial employees rather than a broader employee sample across business organizations, we employed a self-report approach to data collection. this method was chosen due to the nature of our participants and the longitudinal design of our study, which aimed to minimize participant dropout over multiple timepoints. although our analysis indicated that common method bias was not a major issue, we recognize that future research could benefit from a multi-source design to gain a deeper understanding of the studied constructs. for example, adding evaluations from immediate supervisors regarding performance outcomes could help offset the biases inherent in selfreport methods and enhance the validity of our findings. employing such a multi-source approach in future studies could provide a more thorough evaluation of the relationships between variables and reduce the limitations tied to sole reliance on self-reported data. additionally, while our research model outlines general mechanisms by which ai empowerment supports employees' adaptation, leading to enhanced performance and organizational citizenship behaviors (ocbs), we acknowledge that the effectiveness of these constructs may vary across different contexts. higher-level factors and constructs could play a significant role as well. although we employed empowerment as an individual-level construct in alignment with its original design, we believe that future research should explore high-level constructs, such as empowerment climate—a group-level adaptation of empowerment proposed by researchers such as seibert (2004) to establish further investigation and contextualization of empowerment-related constructs. recently more and more studies have begun to illuminate that perceived overqualification, traditionally seen as a disadvantageous condition for both employees and organizations, may actually hold positive implications under certain circumstances (liao et al., 2024). these findings suggest that overqualification does not solely lead to dissatisfaction and turnover intentions as previously thought. instead, it can potentially drive employees to pursue further personal development and innovation within their roles, contributing to organizational growth and adaptability. this emerging perspective warrants deeper investigation, particularly in how organizations might harness the potential of overqualified employees. further research should explore the conditions under which perceived overqualification leads to positive outcomes, such as enhanced creativity, greater organizational commitment, or proactive engagement in tasks beyond assigned roles. moreover, it is crucial to understand the specific organizational cultures, leadership styles, and job design factors that can convert the challenges associated with overqualification into valuable opportunities. while the aim of our exploratory research is to generate new insights rather than to formulate formal hypotheses, we suggest that future studies should focus on developing precise hypotheses for empirical testing. these hypotheses can then be employed to investigate the long-term impacts of perceived overqualification on career trajectories and organizational outcomes. such investigations may lead to a reevaluation of hiring and management practices, facilitating a more effective integration and utilization of the skills and potential of overqualified employees. despite its popularity, the k-means algorithm that we primarily used in this study is not without limitations. the need to pre-specify the number of clusters and the sensitivity to initial centroid placement can lead to suboptimal clustering performance, particularly with complex or high-dimensional datasets (ikotun et al., 2023). additionally, the traditional implementation australasian journal of information systems shi & ma 2025, vol 29, research article ai empowerment of asian-australian migrant workers 24 of k-means struggles with scalability issues in the era of big data, where datasets are not only large but also often come from diverse data sources and contain noise and outliers. this will not be an issue for this study as the sample is reasonably small. however, in the future, if a population-wide or multi-centered survey to be conducted, this issue embedded in k-means algorithm may become a series issues. in this regard, we suggest using improved initialization techniques, such as k-means++, and adaptations that allow for better handling of outliers and noisy data, are among the key developments that enhance the robustness and accuracy of the clustering results (ailon et al., 2009; liu et al., 2023). lastly, many researchers have expanded the concept, psychological empowerment, from individual to group-level constructs. for instance, seibert et al. (2004) appropriated the original psychological empowerment construct and introduced an "empowerment climate", characterized by information sharing, autonomy, and team responsibility. while our research continues to employ empowerment at the individual level, we recognize the potential of these higher-level constructs and suggest further exploration in diverse contexts. references abad merino, s. (2014). dynamics of prosocial behaviour: subtle bias toward racial and gender groups. universidad de córdoba. afiouni, r., & pinsonneault, a. (2023, december 10-13). human-in-control: a human-centered model of adaptation to ai augmentation. in proceedings of the 45th international conference on information systems (icis), hyderabad, india. ailon, n., jaiswal, r., & monteleoni, c. (2009). streaming k-means approximation. in proceedings of the 22nd international conference on neural information processing systems (neurips), vancouver, canada. alabed, a., javornik, a., & gregory-smith, d. (2022). ai anthropomorphism and its effect on users' self-congruence and self–ai integration: a theoretical framework and research agenda. technological forecasting and social change, 182, 121786. doi.org/10.1016/j.techfore.2022.121786 albrecht, s. l., bocks, a., dalton, j., lorigan, a., & smith, a. (2021). pro-environmental employee engagement: the influence of pro-environmental organizational, job, and personal resources. sustainability, 14(1), 43. doi.org/10.3390/su14010043 almaiah, m. a., alfaisal, r., salloum, s. a., hajjej, f., shishakly, r., lutfi, a., alrawad, m., al mulhem, a., alkhdour, t., & al-maroof, r. s. (2022). measuring institutions’ adoption of artificial intelligence applications in online learning environments: integrating the innovation diffusion theory with technology adoption rate. electronics, 11(20), 3291. doi.org/10.3390/electronics11203291 almufareh, m. f., kausar, s., humayun, m., & tehsin, s. (2024). a conceptual model for inclusive technology: advancing disability inclusion through artificial intelligence. journal of disability research, 3(1), 20230060. doi.org/10.57197/jdr-2023-0060 amankwah-amoah, j., abdalla, s., mogaji, e., elbanna, a., & dwivedi, y. k. (2024). the impending disruption of creative industries by generative ai: opportunities, challenges, and research agenda. international journal of information management, 102759. doi.org/10.1016/j.ijinfomgt.2024.102759 australasian journal of information systems shi & ma 2025, vol 29, research article ai empowerment of asian-australian migrant workers 25 ashforth, b. e. (1989). the experience of powerlessness in organizations. organizational behavior and human decision processes, 43(2), 207–242. doi.org/10.1016/07495978(89)90051-4 austin, v., & holloway, c. (2022). assistive technology (at), for what? societies, 12(6), 169. doi.org/10.3390/soc12060169 babalola, m. t., ren, s., kobinah, t., qu, y. e., garba, o. a., & guo, l. (2019). negative workplace gossip: its impact on customer service performance and moderating roles of trait mindfulness and forgiveness. international journal of hospitality management, 80, 136–143. doi.org/10.1016/j.ijhm.2019.02.007 balaban, i., filipović, d., & zlatović, m. (2023). post hoc identification of student groups: combining user modeling with cluster analysis. education and information technologies, 28(6), 7265–7290. doi.org/10.1007/s10639-022-11468-9 bandura, a. (1990). perceived self-efficacy in the exercise of control over aids infection. evaluation and program planning, 13(1), 9–17. doi.org/10.1016/0149-7189(90)90004-g bankins, s., ocampo, a. c., marrone, m., restubog, s. l. d., & woo, s. e. (2024). a multilevel review of artificial intelligence in organizations: implications for organizational behavior research and practice. journal of organizational behavior, 45(2), 159–182. doi.org/10.1002/job.2735 beaudry, a., & pinsonneault, a. (2005). understanding user responses to information technology: a coping model of user adaptation. mis quarterly, 29(3), 493–524. doi.org/10.2307/25148693 benbya, h., davenport, t. h., & pachidi, s. (2020). artificial intelligence in organizations: current state and future opportunities. mis quarterly executive, 19(4). doi.org/10.2139/ssrn.3741983 benbya, h., strich, f., & tamm, t. (2024). navigating generative artificial intelligence: promises and perils for knowledge and creative work. journal of the association for information systems, 25(1), 23–36. doi.org/10.17705/1jais.00872 besco, r. (2015). rainbow coalitions or inter-minority conflict? racial affinity and diverse minority voters. canadian journal of political science/revue canadienne de science politique, 48(2), 305–328. doi.org/10.1017/s0008423915000396 bhattacherjee, a., & premkumar, g. (2004). understanding changes in belief and attitude toward information technology usage: a theoretical model and longitudinal test. mis quarterly, 28(2), 229–254. doi.org/10.2307/25148634 burris, e. r., detert, j. r., & romney, a. c. (2013). speaking up vs. being heard: the disagreement around and outcomes of employee voice. organization science, 24(1), 22– 38. doi.org/10.1287/orsc.1110.0732 chen, f. f., sousa, k. h., & west, s. g. (2005). teacher's corner: testing measurement invariance of second-order factor models. structural equation modeling: a multidisciplinary journal, 12(3), 471–492. doi.org/10.1207/s15328007sem1203_7 chen, l., hsieh, j. p.-a., & rai, a. (2022). how does intelligent system knowledge empowerment yield payoffs? uncovering the adaptation mechanisms and contingency australasian journal of information systems shi & ma 2025, vol 29, research article ai empowerment of asian-australian migrant workers 26 role of work experience. information systems research, 33(3), 1042–1071. doi.org/10.1287/isre.2021.1097 cheng, x. s., zhang, x. p., cohen, j., & mou, j. (2022). human vs. ai: understanding the impact of anthropomorphism on consumer response to chatbots from the perspective of trust and relationship norms. information processing & management, 59(3), 102940. doi.org/10.1016/j.ipm.2022.102940 chi, n., lurie, e., & mulligan, d. k. (2021). reconfiguring diversity and inclusion for ai ethics. in proceedings of the 2021 aaai/acm conference on ai, ethics, and society, virtual event, usa. doi.org/10.1145/3461702.3462622 cohen, j., cohen, p., west, s. g., & aiken, l. s. (2013). applied multiple regression/correlation analysis for the behavioral sciences (3rd ed.). routledge. doi.org/10.4324/9780203774441 côté, s. (1999). affect and performance in organizational settings. current directions in psychological science, 8(2), 65–68. doi.org/10.1111/1467-8721.00016 czarnitzki, d., fernández, g. p., & rammer, c. (2023). artificial intelligence and firm-level productivity. journal of economic behavior & organization, 211, 188–205. doi.org/10.1016/j.jebo.2023.05.008 d’silva, j., & sharma, u. (2020). unsupervised automatic text summarization of konkani texts using k-means with elbow method. international journal of engineering research and technology, 13(9), 2380–2384. doi.org/10.37624/ijert/13.9.2020.2380-2384 davenport, t. h., & miller, s. m. (2022). working with ai: real stories of human-machine collaboration. mit press. doi.org/10.7551/mitpress/12127.001.0001 deci, e. l., & ryan, r. m. (2008). self-determination theory: a macrotheory of human motivation, development, and health. canadian psychology/psychologie canadienne, 49(3), 182–185. doi.org/10.1037/a0012801 demir, m., dalgıç, a., & yaşar, e. (2024). how perceived overqualification affects task performance and proactive behavior: the role of digital competencies and job crafting. international journal of hospitality & tourism administration, 25(3), 552–574. doi.org/10.1080/15256480.2022.2135665 dixit, s., & maurya, m. (2021, february 17–19). equilibrating emotional intelligence and aidriven leadership for transnational organizations. in proceedings of the 2021 international conference on innovative practices in technology and management (iciptm), uttar pradesh, india. dolan, m., epstein, a., lipson, r., & winiski, e. (2022). building the new high road: immigrants and workforce development. labor markets & employment. https://www.pw.hks.harvard.edu/post/new-high-road dou, y. (2023). does the application of artificial intelligence technology affect employees' turnover intention? in proceedings of the 2023 6th international conference on information management and management science (imms), chengdu, china. doi.org/10.1145/3625469.3625488 dwivedi, y. k., kshetri, n., hughes, l., slade, e. l., jeyaraj, a., kar, a. k., baabdullah, a. m., koohang, a., raghavan, v., & ahuja, m. (2023). “so what if chatgpt wrote it?” australasian journal of information systems shi & ma 2025, vol 29, research article ai empowerment of asian-australian migrant workers 27 multidisciplinary perspectives on opportunities, challenges and implications of generative conversational ai for research, practice and policy. international journal of information management, 71, 102642. doi.org/10.1016/j.ijinfomgt.2023.102642 eilers, k., peters, c., & leimeister, j. m. (2022). why the agile mindset matters. technological forecasting and social change, 179, 121650. doi.org/10.5465/ambpp.2021.13110abstract enholm, i. m., papagiannidis, e., mikalef, p., & krogstie, j. (2022). artificial intelligence and business value: a literature review. information systems frontiers, 24(5), 1709–1734. doi.org/10.1007/s10796-021-10186-w evans, g. (2019). the 'bamboo ceiling' in australia is real. the sydney morning herald. https://www.gevans.org/opeds/oped208.html fiori, m., bollmann, g., & rossier, j. (2015). exploring the path through which career adaptability increases job satisfaction and lowers job stress: the role of affect. journal of vocational behavior, 91, 113–121. doi.org/10.1016/j.jvb.2015.08.010 gardner, k., & liu, r. (2010). claiming their voice: sociolinguistic factors affecting immigrant workers' ability to speak up. the catesol journal, 21(1), 45–65. ghaffari, s., yousefimehr, b., & ghatee, m. (2024). generative-ai in e-commerce: use-cases and implementations. in proceedings of the 20th csi international symposium on artificial intelligence and signal processing (aisp), babol, iran. gist, m. e. (1987). self-efficacy: implications for organizational behavior and human resource management. academy of management review, 12(3), 472–485. doi.org/10.5465/amr.1987.4306562 gladden, m., fortuna, p., & modliński, a. (2022). the empowerment of artificial intelligence in post-digital organizations: exploring human interactions with supervisory ai. human technology, 18(2), 98–121. doi.org/10.14254/1795-6889.2022.18-2.2 gorostiaga manterola, m. a., balluerka lasa, m. n., & rodríguez lópez, a. (2022). assessment of adaptive performance and its role as a moderator of the relationship between personorganization fit and organizational citizenship behaviors. psicothema. doi.org/10.7334/psicothema2021.310 hair, j. f., anderson, r. e., tatham, r. l., & black, w. c. (2006). multivariate data analysis with readings (6th ed.). pearson prentice hall. hanson, g. (2022). immigration and regional specialization in ai. in robots and ai (pp. 180– 231). routledge. doi.org/10.4324/9781003275534 harper, l. d., gertner, a. s., & guilder, j. a. v. (2004). perceptive assistive agents in team spaces. in proceedings of the 9th international conference on intelligent user interfaces (iui), funchal, madeira, portugal. doi.org/10.1145/964442.964496 hassandoust, f., & techatassanasoontorn, a. (2022). antecedents of is infusion behaviours: an integrated it identity and empowerment perspective. behaviour & information technology, 41(11), 2390–2414. doi.org/10.1080/0144929x.2021.1928287 hekman, d. r., steensma, h. k., bigley, g. a., & hereford, j. f. (2009). effects of organizational and professional identification on the relationship between administrators' social australasian journal of information systems shi & ma 2025, vol 29, research article ai empowerment of asian-australian migrant workers 28 influence and professional employees' adoption of new work behavior. journal of applied psychology, 94(5), 1325–1335. doi.org/10.1037/a0015315 hemmige, h., ellison, r., & chen, a. (2023). how companies can bring more asian americans into the c-suite. boston consulting group. https://www.bcg.com/publications/2023/howcompanies-can-promote-asian-american-executives henze, r., norte, e., sather, s. e., walker, e., & katz, a. (2002). leading for diversity: how school leaders promote positive interethnic relations. corwin press. herold, d. m., fedor, d. b., & caldwell, s. d. (2007). beyond change management: a multilevel investigation of contextual and personal influences on employees' commitment to change. journal of applied psychology, 92(4), 942–951. doi.org/10.1037/0021-9010.92.4.942 hillmann, f. (2022). new technologies, migration and the future of work. journal of culture, politics and innovation, 2022(3), 2283–7949. doi.org/10.12893/gjcpi.2022.3.7 ho, c. (2020). aspiration and anxiety: asian migrants and australian schooling. melbourne university publishing. ho, m.-t., mantello, p., & ho, m.-t. (2023). an analytical framework for studying attitude towards emotional ai: the three-pronged approach. methodsx, 10, 102149. doi.org/10.1016/j.mex.2023.102149 holmström, j. (2022). from ai to digital transformation: the ai readiness framework. business horizons, 65(3), 329–339. doi.org/10.1016/j.bushor.2021.03.006 horowitz, m. c., & kahn, l. (2024). bending the automation bias curve: a study of human and ai-based decision making in national security contexts. international studies quarterly, 68(2). doi.org/10.1093/isq/sqae020 hsieh, j. j., & wang, w. (2007). explaining employees' extended use of complex information systems. european journal of information systems, 16(3), 216–227. doi.org/10.1057/palgrave.ejis.3000663 huang, m.-h., & rust, r. t. (2018). artificial intelligence in service. journal of service research, 21(2), 155–172. doi.org/10.1177/1094670517752459 huang, m.-h., & rust, r. t. (2021). engaged to a robot? the role of ai in service. journal of service research, 24(1), 30–41. doi.org/10.1177/1094670520902266 hughes, r. t., zhu, l., & bednarz, t. (2021). generative adversarial networks–enabled human–artificial intelligence collaborative applications for creative and design industries: a systematic review of current approaches and trends. frontiers in artificial intelligence, 4, 604234. doi.org/10.3389/frai.2021.604234 hussain, r., sabir, a., lee, d.-y., zaidi, s. f. a., pedro, a., abbas, m. s., & park, c. (2024). conversational ai-based vr system to improve construction safety training of migrant workers. automation in construction, 160, 105315. doi.org/10.1016/j.autcon.2024.105315 hwang, c. y., kang, s. w., & choi, s. b. (2023). coaching leadership and creative performance: a serial mediation model of psychological empowerment and constructive voice behavior. frontiers in psychology, 14, 1077594. doi.org/10.3389/fpsyg.2023.1077594 australasian journal of information systems shi & ma 2025, vol 29, research article ai empowerment of asian-australian migrant workers 29 hwang, s., & beauregard, t. a. (2022). contextualising intersectionality: a qualitative study of east asian female migrant workers in the uk. human relations, 75(4), 609–634. doi.org/10.1177/0018726721989790 hyun, j. (2005). breaking the bamboo ceiling: career strategies for asians: the essential guide to getting in, moving up, and reaching the top. harper business. ibm. (2023). best practices for augmenting human intelligence with ai. ibm ai ethics board. https://www.ibm.com/blog/best-practices-for-augmenting-human-intelligence-with-ai/ ibrahim, t., & rashad, h. (2024). the evolving role of healthcare professionals in the age of ai: impacts on employment, skill requirements, and professional development. journal of artificial intelligence and machine learning in management, 8(2), 14–21. doi.org/10.1093/inthealth/ihaa007 ikotun, a. m., ezugwu, a. e., abualigah, l., abuhaija, b., & heming, j. (2023). k-means clustering algorithms: a comprehensive review, variants analysis, and advances in the era of big data. information sciences, 622, 178–210. doi.org/10.1016/j.ins.2022.11.139 jaiswal, a., arun, c. j., & varma, a. (2022). rebooting employees: upskilling for artificial intelligence in multinational corporations. the international journal of human resource management, 33(6), 1179–1208. doi.org/10.1080/09585192.2021.1891114 jarrahi, m. h., askay, d., eshraghi, a., & smith, p. (2023). artificial intelligence and knowledge management: a partnership between human and ai. business horizons, 66(1), 87–99. doi.org/10.1016/j.bushor.2022.03.002 jones, e., sundaram, s., & chin, w. (2002). factors leading to sales force automation use: a longitudinal analysis. journal of personal selling & sales management, 22(3), 145–156. doi.org/10.1080/08853134.2002.10754303 jun, s., phillips, l. t., & foster-gimbel, o. a. (2023). the missing middle: asian employees’ experience of workplace discrimination and pro-black allyship. journal of applied psychology, 108(2), 225–240. doi.org/10.1037/apl0001068 kanbach, d. k., heiduk, l., blueher, g., schreiter, m., & lahmann, a. (2023). the genai is out of the bottle: generative artificial intelligence from a business model innovation perspective. review of managerial science, 18, 1–32. doi.org/10.1007/s11846-023-00696-z karimikia, h. (2017). the impact of positive behaviours on the effectiveness of information systems departments [doctoral dissertation, auckland university of technology]. karimikia, h., safari, n., & singh, h. (2020). being useful: how information systems professionals influence the use of information systems in enterprises. information systems frontiers, 22, 429–453. doi.org/10.1109/hicss.2016.585 kaymakcı, r., görener, a., & toker, k. (2022). the perceived overqualification's effect on innovative work behaviour: do transformational leadership and turnover intention matter? current research in behavioral sciences, 3, 100068. doi.org/10.1016/j.crbeha.2022.100068 keith, n., & frese, m. (2005). self-regulation in error management training: emotion control and metacognition as mediators of performance effects. journal of applied psychology, 90(4), 677–691. doi.org/10.1037/0021-9010.90.4.677 australasian journal of information systems shi & ma 2025, vol 29, research article ai empowerment of asian-australian migrant workers 30 kim, h.-w., & gupta, s. (2014). a user empowerment approach to information systems infusion. ieee transactions on engineering management, 61(4), 656–668. doi.org/10.1109/tem.2014.2354693 krefft, a., marcus, b., & weigelt, o. (2024). facets of constructive voice behavior: construct clarification, theoretical refinement, and measurement. journal of business psychology, 1– 21. doi.org/10.1007/s10869-024-09932-z lee, j., & zhou, m. (2015). the asian american achievement paradox. russell sage foundation. li, t. j.-j., lu, y., clark, j., chen, m., cox, v., jiang, m., yang, y., kay, t., wood, d., & brockman, j. (2022). a bottom-up end-user intelligent assistant approach to empower gig workers against ai inequality. in symposium on human-computer interaction for work '22, durham, us. li, y., li, y., yang, p., & zhang, m. (2022). perceived overqualification at work: implications for voice toward peers and creative performance. frontiers in psychology, 13, 835204. doi.org/10.3389/fpsyg.2022.835204 liao, m., zhang, m. j., carnevale, j. b., huang, c., & wang, l. (2024). capable fish or deficient ponds? a meta-analysis of consequences, mechanisms, and moderators of perceived overqualification. journal of management. doi.org/10.1177/01492063241239298 lin, t.-a., & chen, p.-h. c. (2022). artificial intelligence in a structurally unjust society. feminist philosophy quarterly, 8(3/4). doi.org/10.5206/fpq/2022.3/4.14191 lindström, n., koutsikouri, d., stier, j., & arvidsson, m. (2020). migrant employment integration and artificial intelligence (ai). in proceedings of the 32nd annual workshop of the swedish artificial intelligence society (sais), stockholm, sweden. liu, h., yu, k.-h., & wright, c. f. (2024). precarious multiculturalism: the racialized experience of asian in/exclusion in australia. american behavioral scientist. doi.org/10.1177/00027642241231319 liu, s., luksyte, a., zhou, l., shi, j., & wang, m. (2015). overqualification and counterproductive work behaviors: examining a moderated mediation model. journal of organizational behavior, 36(2), 250–271. doi.org/10.1002/job.1979 liu, y., du, x., & ma, s. (2023). innovative study on clustering center and distance measurement of k-means algorithm: mapreduce efficient parallel algorithm based on user data of jd mall. electronic commerce research, 23(1), 43–73. doi.org/10.1007/s10660024-09837-2 liu, y., liu, d., du, h., liu, s., & zhou, x. (2022). the multilevel study on the impact of highperformance human resource practices on employees’ voice behavior: a moderated mediation model. frontiers in psychology, 13, 792329. doi.org/10.3389/fpsyg.2022.792329 lu, j. g. (2022). a social network perspective on the bamboo ceiling: ethnic homophily explains why east asians but not south asians are underrepresented in leadership in multiethnic environments. journal of personality and social psychology, 122(6), 959–977. doi.org/10.1037/pspa0000292 australasian journal of information systems shi & ma 2025, vol 29, research article ai empowerment of asian-australian migrant workers 31 lu, j. g. (2023). a creativity stereotype perspective on the bamboo ceiling: low perceived creativity explains the underrepresentation of east asian leaders in the united states. journal of applied psychology, 109(2), 238–256. doi.org/10.1037/apl0001135 ma, c., ganegoda, d. b., chen, z. x., jiang, x., & dong, c. (2020). effects of perceived overqualification on career distress and career planning: mediating role of career identity and moderating role of leader humility. human resource management, 59(6), 521– 536. doi.org/10.1002/hrm.22009 mackey, w., coates, b., & sherrell, h. (2022). migrants in the australian workforce: a guidebook for policy makers. grattan institute. https://grattan.edu.au/report/migrants-in-theaustralian-workforce/ mael, f. a., & tetrick, l. e. (1992). identifying organizational identification. educational and psychological measurement, 52(4), 813–824. doi.org/10.1177/0013164492052004002 mantello, p., ho, m.-t., nguyen, m.-h., & vuong, q.-h. (2023). bosses without a heart: sociodemographic and cross-cultural determinants of attitude toward emotional ai in the workplace. ai & society, 38(1), 97–119. doi.org/10.1007/s00146-021-01290-1 martinko, m. j., & gardner, w. l. (1982). learned helplessness: an alternative explanation for performance deficits. the academy of management review, 7(2), 195–204. doi.org/10.2307/257297 maynard, d. c., joseph, t. a., & maynard, a. m. (2006). underemployment, job attitudes, and turnover intentions. journal of organizational behavior, 27(4), 509–536. doi.org/10.1002/job.389 maynard, m., luciano, m., d'innocenzo, l., mathieu, j., & dean, m. (2014). modeling timelagged reciprocal psychological empowerment-performance relationships. journal of applied psychology, 99(4), 737–749. doi.org/10.1037/a0037623 mcelheran, k., li, j. f., brynjolfsson, e., kroff, z., dinlersoz, e., foster, l., & zolas, n. (2024). ai adoption in america: who, what, and where. journal of economics & management strategy, 33(2), 375–415. doi.org/10.3386/w31788 methot, j. r., lepine, j. a., podsakoff, n. p., & christian, j. s. (2016). are workplace friendships a mixed blessing? exploring trade-offs of multiplex relationships and their associations with job performance. personnel psychology, 69(2), 311–355. doi.org/10.1111/peps.12109 mittal, p., jora, r. b., sodhi, k. k., & saxena, p. (2023). a review of the role of artificial intelligence in employee engagement. in proceedings of the 2023 9th international conference on advanced computing and communication systems (icaccs), tamil nadu, india. mogaji, e., viglia, g., srivastava, p., & dwivedi, y. k. (2024). is it the end of the technology acceptance model in the era of generative artificial intelligence? international journal of contemporary hospitality management. doi.org/10.1108/ijchm-08-2023-1271 molix, l., & bettencourt, b. a. (2010). predicting well-being among ethnic minorities: psychological empowerment and group identity. journal of applied social psychology, 40(3), 513–533. doi.org/10.1111/j.1559-1816.2010.00585.x australasian journal of information systems shi & ma 2025, vol 29, research article ai empowerment of asian-australian migrant workers 32 monod, e., mayer, a.-s., straub, d., joyce, e., & qi, j. (2024). from worker empowerment to managerial control: the devolution of ai tools' intended positive implementation to their negative consequences. information and organization, 34(1), 100498. doi.org/10.1016/j.infoandorg.2023.100498 morris, r. (2008). looking through the twin lens of race and gender: a new politics of surveillance in asian australian and asian canadian women's writing. muraven, m., & baumeister, r. f. (2000). self-regulation and depletion of limited resources: does self-control resemble a muscle? psychological bulletin, 126(2), 247–259. doi.org/10.1037/0033-2909.126.2.247 ng, e. h., & kim, h. w. (2009). investigating information systems infusion and the moderating role of habit: a user empowerment perspective. in proceedings of the 30th international conference for information systems (icis), phoenix, arizona, us. norton, t. a., parker, s. l., zacher, h., & ashkanasy, n. m. (2015). employee green behavior: a theoretical framework, multilevel review, and future research agenda. organization & environment, 28(1), 103–125. doi.org/10.1177/1086026615575773 nunnally, j. (1978). an overview of psychological measurement. in clinical diagnosis of mental disorders: a handbook (pp. 97–146). doi.org/10.1007/978-1-4684-2490-4 oetting, m. (2009). empowered involvement. in ripple effect: how empowered involvement drives word of mouth (pp. 73–81). gabler. doi.org/10.1007/978-3-8349-8372-5_5 ozturkcan, s., & gashi nulleshi, s. (2024). generative artificial intelligence (gen ai) as a cocreator: exploring the collaboration between human and machine entrepreneurs. in proceedings of the nordic academy of management, reykjavik, iceland. perifanis, n.-a., & kitsios, f. (2023). investigating the influence of artificial intelligence on business value in the digital era of strategy: a literature review. information, 14(2), 85. doi.org/10.3390/info14020085 podsakoff, p. m., mackenzie, s. b., lee, j.-y., & podsakoff, n. p. (2003). common method biases in behavioral research: a critical review of the literature and recommended remedies. journal of applied psychology, 88(5), 879–903. doi.org/10.1037/0021-9010.88.5.879 preminger, j. (2020). meritocracy in the service of ethnocracy. citizenship studies, 24(2), 247– 263. doi.org/10.1080/13621025.2020.1720604 pritiprada, p., satpathy, i., patnaik, b., & thaichon, p. (2024). fostering women's empowerment in europe: case studies of innovation in ai, blockchain, and the metaverse. in technology and luxury hospitality (pp. 80–92). routledge. reis, t., bruchhaus, s., vu, b., bornschlegl, m. x., & hemmje, m. l. (2021). towards modeling ai-based user empowerment for visual big data analysis. ceur workshop proceedings, 2863, 50–58. https://ceur-ws.org/vol-2863/paper-07.pdf precedence research. (2024). artificial intelligence (ai) market – 2024-2034 (global industry analysis, size, share, growth, trends, regional outlook, and forecast). https://www.precedenceresearch.com/artificial-intelligence-market australasian journal of information systems shi & ma 2025, vol 29, research article ai empowerment of asian-australian migrant workers 33 rude, b., & giesing, y. (2022). technological change and immigration – a race for talent or of displaced workers. in beiträge zur jahrestagung des vereins für socialpolitik 2022: big data in economics, kiel, germany. ruiz, n. g., noe-bustamante, l., & shah, s. (2023). demographic profile of asian american adults. in diverse cultures & shared experiences shape asian american identities. pew research center. https://www.pewresearch.org/2023/05/08/asian-american-identityappendix-demographic-profile-of-asian-american-adults/ ruttiman, j. (2009). breaking through the “bamboo ceiling” for asian american scientists. science, 29(09). sarker, s., susarla, a., gopal, r., & thatcher, j. b. (2024). democratizing knowledge creation through human-ai collaboration in academic peer review. journal of the association for information systems, 25(1), 158–171. doi.org/10.17705/1jais.00872 schultheiss, d. e., & davis, b. l. (2015). immigrant workers: career concerns and barriers. in apa handbook of career intervention, volume 1: foundations (pp. 259–277). american psychological association. doi.org/10.1037/14438-015 schwartz, j., denny, b., mallon, d., durme, y. v., hauptmann, m., yan, r., & poynton, s. (2020). superteams: putting ai in the group. in paradox as a path forward. deloitte. https://www.deloitte.com/global/en/our-thinking/insights/topics/talent/human-capitaltrends/2020/human-ai-collaboration.html schwarz, a. h. (2003). defining information technology acceptance: a human-centered, managementoriented perspective [doctoral dissertation, university of houston]. segars, a. h., & grover, v. (1998). strategic information systems planning success: an investigation of the construct and its measurement. mis quarterly, 22(2), 139–163. doi.org/10.2307/249393 sehgal, r. (2007). user empowerment: an enabler of enterprise systems success [doctoral dissertation, queensland university of technology]. seibert, s. e., silver, s. r., & randolph, w. a. (2004). taking empowerment to the next level: a multiple-level model of empowerment, performance, and satisfaction. academy of management journal, 47(3), 332–349. doi.org/10.5465/20159585 shahapure, k. r., & nicholas, c. (2020). cluster quality analysis using silhouette score. in proceedings of the 2020 ieee 7th international conference on data science and advanced analytics (dsaa), piscataway, us. shi, y., & deng, b. (2024). finding the sweet spot: exploring the optimal communication delay for ai feedback tools. information processing & management, 61(2), 103572. doi.org/10.1016/j.ipm.2023.103572 shrestha, n. j. (2021). factor analysis as a tool for survey analysis. american journal of applied mathematics and statistics, 9(1), 4–11. dx.doi.org/10.12691/ajams-9-1-2 shukla, s. (2023). creative computing and harnessing the power of generative artificial intelligence. journal of environmental sciences and technology, 2(1), 556–579. doi.org/10.5281/zenodo.10847103 australasian journal of information systems shi & ma 2025, vol 29, research article ai empowerment of asian-australian migrant workers 34 sison, a. j. g., daza, m. t., gozalo-brizuela, r., & garrido-merchán, e. c. (2023). chatgpt: more than a “weapon of mass deception” – ethical challenges and responses from the human-centered artificial intelligence (hcai) perspective. international journal of human–computer interaction, 1–20. doi.org/10.1080/10447318.2023.2225931 sowa, k., przegalinska, a., & ciechanowski, l. (2021). cobots in knowledge work: human– ai collaboration in managerial professions. journal of business research, 125, 135–142. doi.org/10.1016/j.jbusres.2020.11.038 sparreboom, t., & tarvid, a. (2016). imbalanced job polarization and skills mismatch in europe. journal for labour market research, 49, 15–42. doi.org/10.1007/s12651-016-0196-y spreitzer, g. m. (1995). psychological empowerment in the workplace: dimensions, measurement, and validation. academy of management journal, 38(5), 1442–1465. doi.org/10.5465/256865 stewart, k. j., darcy, d. p., & daniel, s. l. (2006). opportunities and challenges applying functional data analysis to the study of open source software evolution. statistical science, 21(2), 167–178. doi.org/10.1214/088342306000000141 sum, n., lahiri-roy, r., & belford, n. (2023). hope theory as resistance: narratives of south asian scholars in australian academia. equality, diversity and inclusion: an international journal. doi.org/10.1108/edi-03-2023-0085 sumi, e. j., kabir, m. s., & alam, m. n. (2024). artificial intelligence in migrant labour management: a comprehensive review. international journal for multidisciplinary research, 6(1). doi.org/10.36948/ijfmr.2024.v06i01.13041 sundaram, s., schwarz, a., jones, e., & chin, w. w. (2007). technology use on the front line: how information technology enhances individual performance. journal of the academy of marketing science, 35, 101–112. doi.org/10.1007/s11747-006-0010-4 syed, j. (2008). employment prospects for skilled migrants: a relational perspective. human resource management review, 18(1), 28–45. doi.org/10.1016/j.hrmr.2007.12.001 tadampali, a. c. t., & hadi, a. (2017). the effect of job satisfaction and organizational commitment on work engagement and performance. in proceedings of the 2nd international conference on education, science, and technology (icest 2017) (pp. 19). doi.org/10.2991/icest-17.2017.19 taherizadeh, a., & beaudry, c. (2023). an emergent grounded theory of ai-driven digital transformation: canadian smes’ perspectives. industry and innovation, 30(9), 1244–1273. doi.org/10.1080/13662716.2023.2242285 thomas, k. w., & velthouse, b. a. (1990). cognitive elements of empowerment: an “interpretive” model of intrinsic task motivation. academy of management review, 15(4), 666–681. doi.org/10.2307/258687 underwood, t. (2024). will ai make us overconfident? https://tedunderwood.com/2024/08/31/will-ai-make-us-overconfident/ usmani, u. a., happonen, a., & watada, j. (2023). human-centered artificial intelligence: designing for user empowerment and ethical considerations. in proceedings of the 2023 australasian journal of information systems shi & ma 2025, vol 29, research article ai empowerment of asian-australian migrant workers 35 5th international congress on human-computer interaction, optimization and robotic applications (hora), istanbul, turkey. vardaman, j. m., amis, j. m., dyson, b. p., wright, p. m., & van de graaff randolph, r. (2012). interpreting change as controllable: the role of network centrality and self-efficacy. human relations, 65(7), 835–859. doi.org/10.1177/0018726712441642 venkatesh, v. (2022). adoption and use of ai tools: a research agenda grounded in utaut. annals of operations research, 308(1), 641–652. doi.org/10.1007/s10479-020-03918-9 vichi, m., & saporta, g. (2009). clustering and disjoint principal component analysis. computational statistics & data analysis, 53(8), 3194–3208. doi.org/10.1016/j.csda.2008.05.028 vomberg, a., & klarmann, m. (2022). crafting survey research: a systematic process for conducting survey research. in c. homburg, m. klarmann, & a. vomberg (eds.), handbook of market research (pp. 67–119). springer international publishing. doi.org/10.1007/978-3-319-57413-4_4 von krogh, g. (2018). artificial intelligence in organizations: new opportunities for phenomenon-based theorizing. academy of management discoveries, 4(4), 404–409. doi.org/10.5465/amd.2018.0084 wang, m., beal, d. j., chan, d., newman, d. a., vancouver, j. b., & vandenberg, r. j. (2017). longitudinal research: a panel discussion on conceptual issues, research design, and statistical techniques. work, aging and retirement, 3(1), 1–24. doi.org/10.1093/workar/waw033 wang, s. m., scott, k. m., artemenko, m., miceli, m., & berendt, b. (2023). “we try to empower them” – exploring future technologies to support migrant jobseekers. in proceedings of the 2024 acm conference on fairness, accountability, and transparency (facct), rio de janeiro, brazil. wilson, h. j., & daugherty, p. r. (2018). collaborative intelligence: humans and ai are joining forces. harvard business review, 96(4), 114–123. wong, l.-w., tan, g. w.-h., ooi, k.-b., & dwivedi, y. (2024). the role of institutional and self in the formation of trust in artificial intelligence technologies. internet research, 34(2), 343–370. doi.org/10.1108/intr-07-2021-0446 wu-gehbauer, m., & rosenkranz, c. (2024). towards a framework for developing artificial intelligence systems for human-artificial intelligence augmentation. in proceedings of the 32nd european conference on information systems (ecis), paphos, cyprus. yan, l., & grossman, g. m. (2023). robots and ai: a new economic era. taylor & francis. yildiz, k., akoğlu, h. e., emir, e., angosto, s., & garcía-fernández, j. (2024). the perceived effect of digital transformation and resultant empowerment on job performance of employees in the fitness family business. journal of family business management. doi.org/10.1108/jfbm-02-2024-0038 yip, t., cheah, c. s., kiang, l., & hall, g. c. n. (2021). rendered invisible: are asian americans a model or a marginalized minority? american psychologist, 76(4), 575–586. doi.org/10.1037/amp0000857 australasian journal of information systems shi & ma 2025, vol 29, research article ai empowerment of asian-australian migrant workers 36 zaza, i., & junglas, i. (2016). it self-service engagement: a theory of trying perspective. in proceedings of the thirty-seventh international conference on information systems (icis), dublin, ireland. zhang, p. (2023). taking advice from chatgpt. arxiv preprint arxiv:2305.11888. dx.doi.org/10.48550/arxiv.2305.11888 appendix 1 aiemp supplementary analysis results table 7 and figure 4, figure 5 show that the four dimensions of aiemp factor did not follow a normal distribution. variable shapiro-wilk w p-value meaningfulness aiemp_meaningfulness_1 0.909*** <0.001 aiemp_meaningfulness_2 0.908*** <0.001 aiemp_meaningfulness_3 0.912*** <0.001 competence aiemp_competence_1 0.911*** <0.001 aiemp_competence_2 0.909*** <0.001 aiemp_competence_3 0.906*** <0.001 self-determination aiemp_sd_1 0.912*** <0.001 aiemp_sd_2 0.923*** <0.001 aiemp_sd_3 0.904*** <0.001 impact aiemp_impact_1 0.913*** <0.001 aiemp_impact_2 0.901*** <0.001 aiemp_impact_3 0.910*** <0.001 table 7 shapiro-wilk normality test results note: * p < .05, ** p < .01, *** p < .001. australasian journal of information systems shi & ma 2025, vol 29, research article ai empowerment of asian-australian migrant workers 37 figure 4 q-q plots for the four dimensions in aiemp australasian journal of information systems shi & ma 2025, vol 29, research article ai empowerment of asian-australian migrant workers 38 figure 5 principal component analysis (pca) processed clusters scatter plot australasian journal of information systems shi & ma 2025, vol 29, research article ai empowerment of asian-australian migrant workers 39 figure 6 silhouette plots for the solutions australasian journal of information systems shi & ma 2025, vol 29, research article ai empowerment of asian-australian migrant workers 40 in accordance with the evaluation criteria suggested by hair et al. (2006) and chen et al. (2005), our confirmatory factor analysis (cfa) results (see table 8) indicate that the second-order model performs comparably to three alternative first-order factor models: a unidimensional model, an uncorrelated model, and a correlated model. specifically, model 1 posits that a single first-order factor explains the variance among the 12 items. model 2 assumes that these items form four uncorrelated first-order factors—meaningfulness, competence, selfdetermination, and impact. model 3 allows these four factors to correlate freely. model 4 is our second-order factor model which accounts for the relationships among the four first-order factors. as chen et al. (2005) say, a second-order model is preferable for several reasons: it estimates fewer parameters and thus offers more degrees of freedom, enhancing parsimony. furthermore, it simplifies the interpretation of complex models by distilling the core variances of first-order factors and effectively isolating the measurement errors and unique variances associated with these factors. this ability to clarify the underlying structure without compromising the model's integrity makes the second-order factor model particularly valuable in our analysis. thus the second order model is able to better quantify and understand how specific dimensions (first-order constructs) contribute to a broader, more comprehensive construct (second-order construct) particularly beneficial in fields like psychology or business, where complex constructs are common and comprise various underlying factors (segars & grover, 1998), and thus by using a second-order model, researchers and practitioners can more accurately assess these composite constructs, leading to better decision-making and more targeted interventions. model dof chi-square cfi gfi nfi rmsea first order unidimensional 54 628.879 0.882 0.872 0.872 0.143 uncorrelated 54 1626.049 0.677 0.670 0.670 0.236 correlated 48 130.945 0.983 0.973 0.973 0.057 second-order 50 190.491 0.971 0.961 0.961 0.073 table 8 cfas for the alternative measurement models for aiemp note: cfi, comparative fit index; gfi, goodness-of-fit index; nfi, normed fit index; rmsea, root mean square error of approximation appendix 2 cluster supplementary materials we used principal component analysis (pca), a dimensionality-reduction method, to prepare the visualization of the clusters (vichi & saporta, 2009). we standardized the range of the continuous initial variables so that each one of them has a mean of 0 and a standard deviation of 1. then we computed the covariance matrix of the data, and, from the covariance matrix, the eigenvalues and eigenvectors are computed. the eigenvectors were then sorted by decreasing eigenvalues to rank the corresponding principal components in order of significance. then, the data can be projected onto the new feature space, so we could develop the clustering visualization as in figure 5. note in the figure, the x-axis is the first principal axis and captures the maximum variance in the data. it effectively represents a weighted combination of the australasian journal of information systems shi & ma 2025, vol 29, research article ai empowerment of asian-australian migrant workers 41 original variables that accounts for the largest amount of variability in the dataset. the specific weights (or loadings) for each original variable can give insight into which variables contribute most to this component. the y-axis is the second principal axis and captures the maximum remaining variance that is orthogonal (i.e., uncorrelated) to the first principal component. it provides the second most significant way of seeing the variability in the data. we conducted another analysis to compare the silhouette scores following prior research (shahapure & nicholas, 2020). the silhouette widths for the 3, 4 and 6-cluster solutions were shown in , indicating no major differences and all have a silhouette value higher than 1. the 3 and 6-cluster solutions were slightly better than the 4-cluster solution. appendix 3 concepts correlations the correlation table is presented in table 9. age gender education tenure aiemp iu jp cv ei age 1.00 gender 0.04 1.00 education 0.12 -0.05 1.00 tenure 0.35 -0.01 -0.07 1.00 aiemp -0.14 0.07 0.09 -0.12 1.00 iu -0.08 -0.02 0.09 -0.09 0.49 1.00 jp 0.13 -0.09 0.01 0.09 -0.03 0.07 1.00 cv 0.07 0.00 0.08 -0.04 0.21 0.26 0.28 1.00 ei 0.00 -0.05 -0.03 -0.07 0.26 0.22 0.02 0.15 1.00 table 9 correlations matrix note: age, quantified as a categorical variable with ranges from 1 (less than 21 years) to 5 (more than 50 years); gender, coded as -1 for female, 0 for others (including “prefer not to say”), and 1 for male; education, ranging from 0 (high school and lower) to 4 (doctoral degree); tenure, indicating the length of current employment in months; ai empowerment (aiemp) [time 1], which measures how individuals utilize ai to enhance job performance, with aspects like meaningfulness, competence, selfdetermination, and impact; infusion use (iu) [time 2], assessing the comprehensive utilization and integration of system capabilities; job performance (jp) [time 3], australasian journal of information systems shi & ma 2025, vol 29, research article ai empowerment of asian-australian migrant workers 42 evaluated through self-report ratings of task-focused contributions; constructive voicing (cv) [time 3], involving expressions of constructive and change-oriented messages; and ethnic identity (ei) [time 1], exploring personal identification with asian ethnicity through reactions to societal feedback and shared achievements. australasian journal of information systems shi & ma 2025, vol 29, research article ai empowerment of asian-australian migrant workers 43 appendix 4 methods for attention checks and filtering of unqualified participants to ensure the reliability of our data, we employed two primary methods for attention checks and filtering out unqualified participants: one utilizing the octopus group's methods4, and the other one is based on our own tailored approach. this appendix provides a detailed account of our own procedures. 1. mechanistic filtering methods our first approach involved mechanistic filtering to exclude unqualified participants. we implemented the following criteria: completion of all study waves: each participant was assigned a unique id that they were required to use consistently across all three waves of data collection. by merging data from the three waves based on these ids as key, we were able to identify and exclude participants who failed to complete at least one of the waves. survey completion time: we monitored the total duration each participant spent on the survey. participants who completed the survey in less than two minutes were excluded, as this indicated insufficient time for thoughtful responses. 2. interaction with an ai question to check attention in addition to mechanistic filtering, we assessed participants' familiarity with ai, as well as their attention, by incorporating a question derived from the study by sowa et al. (2021). this question was designed to categorize human-ai interaction into four tiers (separation (no collaboration), complementary collaboration, dependent collaboration, and hybrid collaboration). specifically, the question is: please choose the level of collaboration that best describes the current state of human-ai interaction in your work environment, considering the descriptions below: option a: level 1: separation (no collaboration) human workers compete with or work completely separately from ai machines. option b: level 2: complementary collaboration humans and ai complement each other by focusing on tasks they are individually good at. option c: level 3: dependent collaboration ai and humans become dependent on each other, with ai sometimes needing human guidance for decision-making. option d: level 4: hybrid collaboration ai becomes an extension of the human brain, and the two work together collaboratively as a 'centaur.' this is also an attention checking mechanism because we introduced an additional e. level 0: no engagement with ai. participants who selected level 0 were presumed to have overlooked the survey requirements, as all participants were required and informed to have prior experience with ai in their work. this oversight was considered indicative of a lack of attention, leading to the exclusion of these individuals from our analysis. by employing these multi-faceted filtering techniques, we validated our data and ensured its reliability for our study. 4 we briefly introduced the methods in the main body. for details, please see: https://support.octopusgroup.com.au/ australasian journal of information systems shi & ma 2025, vol 29, research article ai empowerment of asian-australian migrant workers 44 appendix 5 aiemp supplementary analysis results in recent years, there has been a growing body of research exploring various ai-related concepts such as ai use/adoption, ai integration, ai transformation, and ai collaboration. while these topics are closely related to our core concept of ai empowerment, they are distinguishable in several key aspects. the details of these distinctions are outlined below. concept summary comments on difference ai use/adoption ai adoption normally refers to the decisionmaking process that individuals or organizations undergo when deciding to adopt ai technologies. it focuses on the initial acceptance and willingness to integrate ai into the work environment, influenced by factors such as perceived usefulness, ease of use, and external pressures. commonly, ai use and ai adoption are discussed together, as seen in venkatesh (2022). nonetheless, as in mcelheran et al. (2024) and (wong et al., 2024), ai use in some research can be more broadly understood as the practical application of ai technologies within firms, varying in intensity and scope from basic, exploratory use to advanced, widespread integration across different aspects of the business. our ai empowerment concept centres on the idea that employees feel empowered when they perceive that an external intelligence system supports them in achieving their work goals and overcoming work-related challenges. this concept primarily operates at the individual level, focusing on how ai influences personal outcomes such as job satisfaction, performance, and proactive behaviours in the workplace. unlike ai use/adoption, which often involves simpler metrics like a dummy variable indicating ai usage or a single-level scale (see venkatesh (2022), czarnitzki et al. (2023) and almaiah et al. (2022)), our ai empowerment employs a two-level scale to measure how individuals utilize ai in their jobs, integrating psychological empowerment theories with ai-specific behaviours. ai integration/ self–ai integration according to alabed et al. (2022), ai integration, or self–ai integration, can be viewed as a more advanced stage than mere adoption. it involves reconfiguring work processes and enhancing capabilities through ai by incorporating an anthropomorphized ai agent into one's selfschema. this integration occurs through a cognitive match between ai and the self, known as self-congruence, where ai becomes a part of the user's identity, thereby enhancing their sense of identity and social belonging (huang & rust, 2021). although both ai integration and ai empowerment involve the utilization of ai technologies to improve work outcomes, ai empowerment focuses on how employees feel empowered by the presence and use of ai technologies but does not emphasize the psychological process where individuals perceive ai as an extension of themselves. empowerment literature typically addresses how technologies empower employees by making them feel more competent, meaningful, determined, and impactful in their roles, as seibert et al. (2004) guided, rather than integrating ai into their self-schema. australasian journal of information systems shi & ma 2025, vol 29, research article ai empowerment of asian-australian migrant workers 45 ai transformation ai transformation, or ai-driven digital transformation (aidt), according to taherizadeh and beaudry (2023), , refers to the profound and comprehensive changes that ai technologies bring to an organization's activities, boundaries, and goals. this concept extends beyond the mere adoption or use of ai, involving a holistic reconfiguration of organizational processes, structures, and strategies to fully leverage ai's potential. it evolves from the broader concept of digital transformation. ai transformation drives significant shifts in how organizations operate, compete, and deliver value, often leading to the creation of new business models and restructuring organizational boundaries (holmström, 2022). ai empowerment, on the other hand, focuses on the individual's experience and perception of how ai technologies enhance their ability to achieve workrelated goals and overcome challenges. it is concerned with how ai influences personal job satisfaction, performance, and proactive behaviors by making employees feel more competent, autonomous, and impactful in their roles. ai collaboration this concept discusses the interdependence between ai systems and employees in performing tasks together to achieve shared goals. ai collaboration can range from ai assisting with routine tasks to ai and humans co-creating solutions, making decisions, and enhancing productivity in complex work environments(davenport & miller, 2022). sowa et al. (2021) provide a framework for measuring ai collaboration across different levels, from minimal interaction to fully integrated collaboration where ai and humans work together as a 'centaur.' both ai-human collaboration and ai empowerment involve leveraging ai technologies to enhance human capabilities in the workplace. however, ai-human collaboration focuses on the dynamic levels of interaction between ai systems and humans. in contrast, ai empowerment is more specifically focused on the psychological and motivational aspects of ai usage, concentrating on how ai makes individuals feel more empowered in their specific roles. copyright copyright © 2025 yingnan shi & chao ma. this is an open-access article licensed under a creative commons attribution-non-commercial 4.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v29.5189 paper title (paper title style) australasian journal of information systems alhelaly et al. 2024, vol 28, research article mobile identity protection 1 mobile identity protection: the moderation role of selfefficacy yasser alhelaly nova information management school, nova university lisbon, lisbon, portugal yalhelaly@novaims.unl.pt gurpreet dhillon g. brint ryan college of business, university of north texas, usa university of kwazulu-natal, durban, south africa tiago oliveira nova information management school, nova university lisbon, lisbon, portugal abstract the rapid growth of mobile applications and the associated increased dependency on digital identity raises the growing risk of identity theft and related fraud. hence, protecting identity in a mobile environment is a problem. this study develops a model that examines the role of identity protection self-efficacy in increasing users’ motivation intentions to achieve actual mobile identity protection. our research found that self-efficacy significantly affects the relationship between users’ perceived threat appraisal and their motivational intentions for identity protection. the relation between mobile users’ protection, motivational intentions, and actual mobile identity protection actions was also found to be significant. additionally, the findings revealed the considerable impact of awareness in fully mediating between selfefficacy and actual identity protection. the model and its hypotheses are empirically tested through a survey of 383 mobile users, and the findings are validated through a panel of experts, thus confirming the impact of self-efficacy on an individual’s identity protection in the mobile context. keywords: mobile identity protection, identity theft, self-efficacy, threat appraisal, identity protection awareness, mobile context, motivation. 1 introduction mobile security has gained paramount importance, necessitating our diligent efforts to comprehend its implications and develop effective measures for mitigating mobile security risks (yao et al., 2018). as mobile devices become increasingly prevalent and digital identities more integral, protecting individual mobile identities has become a pressing issue. furthermore, the growing adoption of mobile applications like mobile banking and digital wallets in recent years has drawn the focus of cybercriminals toward targeting mobile users. a mastercard study1 found that 76% of consumers prefer mobile wallets for convenience. juniper research2 predicts over 5.2 billion global digital wallet users by 2026, indicating substantial growth. this potential revenue has led to increased targeting of mobile identities 1 https://www.mastercard.com/news/press/2021/april/mastercard-new-payments-index-consumerappetite-for-digital-payments-takes-off/ 2 https://www.juniperresearch.com/press/digital-wallet-users-exceed-5bn-globally-2026 https://www.mastercard.com/news/press/2021/april/mastercard-new-payments-index-consumer-appetite-for-digital-payments-takes-off/ https://www.mastercard.com/news/press/2021/april/mastercard-new-payments-index-consumer-appetite-for-digital-payments-takes-off/ https://www.juniperresearch.com/press/digital-wallet-users-exceed-5bn-globally-2026 australasian journal of information systems alhelaly et al. 2024, vol 28, research article mobile identity protection 2 by hackers and fraudsters. according to a 2022 cybercrime report by lexisnexis solutions3 , a portentous 75% of potential cyberattacks focus on mobile digital transactions. this trend underscores that hackers are actively aiming at users’ mobile identities, particularly in financial transactions, which are perceived as easier and more lucrative. the report's analysis of 35.5 billion mobile financial transactions further reveals a 32% increase worldwide in automated attacks using bots and a remarkable 50% year-over-year rise in human-initiated attacks using social engineering since 2019. mobile devices are integrated information systems with unique characteristics. the national institute of standards and technology (nist) highlights that, unlike pcs, mobile devices have unique characteristics that require diverse security for organizations and individuals, including bringing your own device (byod) against various threats (souppaya & scarfone, 2013). similarly, zambrano & rafael (2018) emphasized that companies adopting mobile devices in byod setups face heightened security threats. while mobile devices share attack vectors with pcs, they are more susceptible to risks like an advanced persistent threat (apt), social engineering, malware, loss, and theft. these vulnerabilities expose employees to identity theft and data loss, partly due to mobile devices lacking dedicated it control and their unique characteristics differing from pcs. these characteristics span software and hardware such as small size, mobility, wireless connectivity, local data storage, full-fledged operating systems, web/mobile apps, gps, sensors, cameras, biometric authentication (souppaya & scarfone, 2013), and mobile payment features through near-field communication (nfc) or quick response code (qrc) technologies (gong et al., 2020). despite having diverse security measures compared to pcs, souppaya & scarfone (2013) also noted that mobile devices encounter greater vulnerabilities, including limited physical controls due to their size and mobility, utilization of untrusted devices (such as jailbroken or rooted devices), exposure to untrusted networks, usage of location services, and interactions with untrusted apps. in the same line, wu et al. (2020) reveal limited user app security awareness. mobile apps often request broad data access, risking malicious access/sharing. due to poor mobile security awareness, users trust app stores (e.g., google play, app store), granting all apps data access. this aspect leads to ignoring security controls/alerts, which increases security risks and mobile identity threats (wu et al., 2020). recent research in the information systems (is) field has also shown a keen interest in various facets of mobile security. notably, scholars have delved into topics like security beliefs during mobile shopping (venkatesh et al., 2017), trust in mobile payment services (gong et al., 2020), the influence of mobile security notification design on users’ security perceptions (wu et al., 2020), the usability of security features in mobile apps and users’ perceptions of security (sanyal et al., 2021), safeguarding employee security and privacy through enterprise mobile systems (choudhary et al., 2022), and more recently, the foundational principles for crafting secure mobile health systems (lin et al., 2023) and the impact of trust and perceived risks in using mobile payments in micro business transactions (mombeuil, 2023). while existing studies have touched upon various aspects of mobile security, the area of mobile identity protection remains underexplored. given the practical and theoretical underpinnings discussed above regarding the problem of mobile identity protection, it becomes evident that with the growing prevalence of mobile 3 https://risk.lexisnexis.com/insights-resources/research/cybercrime-report australasian journal of information systems alhelaly et al. 2024, vol 28, research article mobile identity protection 3 devices, they have become increasingly susceptible to phishing and identity theft attacks. consequently, mobile users must possess the efficacy to implement diverse security measures to counter these threats effectively (verkijika, 2019). this dimension is particularly crucial because, despite technological advancements, individuals continue to be targeted as the weakest links in systems and network security (khando et al., 2021). this determinant highlights the pressing need to motivate individuals to adopt protection measures against technology-related threats (ogbanufe & baham, 2023). in this context, the significance of information security awareness emerges as a critical factor in enhancing individuals' protective behaviour (khando et al., 2021). therefore, our study aims to delve deeper into this area by employing the protection motivation concepts (rogers, 1975; 1983) to comprehensively understand the motivational dynamics related to identity protection in mobile environments. specifically, our study investigates three pivotal concepts. first, individuals' perception of their efficacy in safeguarding their mobile identities. second, how the perceived threats of identity theft impact individuals' protective behaviour. third, how security awareness can improve the relationship between self-efficacy and motivational behaviour toward mobile identity protection. this research makes four contributions. first, we present a mobile identity protection motivation model that combines the self-efficacy theory of motivation and the protection motivation theory. second, we examine the proposed conceptual model to see how selfefficacy and threat appraisal play a role in individual motivation intentions. third, we look at how self-efficacy influences both the relationship between threat appraisals and motivational intentions for identity protection among mobile users, as well as the relationship between motivational intentions and actual identity protection. finally, we scrutinize the mediation impact of protection awareness on the relationship between self-efficacy and actual identity protection. our initial model is developed from the literature, but we undertook a qualitative phase to evaluate the findings' validity, efficacy, and relevance. 2 literature and theoretical framing four bodies of literature inform our research – how individual identity protection is accomplished in mobile environments, the role of self-efficacy in individual mobile identity protection, how individual threat appraisal for mobile identity protection unfolds, and the role awareness plays in explaining mobile identity protection. the following subsections discuss each of the bodies of literature. 2.1 mobile identity protection how individual identity protection is accomplished in mobile environments is a topic of immense interest. several scholars have worked on developing measures, defining different conceptualizations of identity, and implications of technological identity on human behaviour. (carter & grover, 2015), for instance, examined the everyday use of technology by individuals (e.g., smartphones, wi-fi hotspots, social media, etc.) and how such technology use becomes part of an individual’s identity. they proposed that the strength of technological identity rests on the degree to which its past enactment is associated with embeddedness. the arguments are based on the original work on identity management by (mccall george & simmons, 1978; stryker & serpe, 1994), and (mccall george & simmons, 1978; stets & cast, 2007). other scholars have conceptualized identity along similar lines, albeit in different contexts (for instance, see ogbanufe & gerhart (2020)). the consensus view of identity, however, gravitates australasian journal of information systems alhelaly et al. 2024, vol 28, research article mobile identity protection 4 to the degree to which people regard their use of technology as vital to their sense of self as an emergent form of identity. in the digital world, an individual’s identity is constantly exposed to various security threats and cyberattacks, ranging from identity misuse and personal data theft to privilege escalation, social engineering, hacking, security breaches, and identity theft itself (ogbanufe & pavur, 2022; camp, 2004). among these identity-related threats, identity theft and fraud stand out as the most frequently reported attacks (ftc, 2020). identity theft involves another person's unauthorized use of digital identity information, leading to the fraudulent acquisition of victims' personal details (bose & leung, 2019). in response to the escalating menace of identity theft, researchers have delved into understanding the nuances of online identity theft and devising innovative countermeasures. these have ranged from email authentication, identification systems (bose & leung, 2019), multi-factor authentication (ogbanufe & baham, 2023), evoking fear and threats to push individuals toward protecting their identities (ogbanufe & pavur, 2022), and enhancing security self-efficacy (verkijika, 2019). 2.2 identity protection self-efficacy self-efficacy is defined as the expectation that one will be able to act successfully (bandura, 1997). self-efficacy has been widely used in is literature. scholars have investigated and operationalized self-efficacy in different contexts (schunk dale & pajares, 2009). notably, it has been employed to probe into topics like executives’ decisions to adopt anti-malware software in small and medium businesses (lee & larsen, 2009), the impact of fear appeals on users' security compliance (johnston & warkentin, 2010), the influence of organizational commitment on information asset protection (posey et al., 2015), situational factors impacting intent to violate information security policies (isp) (johnston et al., 2016), cognitive-affective drivers of employees toward isp compliance (d’arcy & lowry, 2019), and mobile technology identity and intentions to use mobile health applications (balapour et al., 2019). more recently, employees’ work productivity and technostress (loh et al., 2023) and the role of metacognitive skills in developing low-coded applications (matook et al., 2023). most of these studies adopted self-efficacy from the commonly used computer self-efficacy measure by compeau & higgins (1995). yet, compeau et al. (2022) highlight the need for context-specific it-related self-efficacy constructs to capture evolving it phenomena effectively. the is literature introduces diverse self-efficacy concepts such as deterrence efficacy (willison & warkentin, 2013), email screening self-efficacy (herath et al., 2014), phishing detection self-efficacy (wang li & rao, 2016), privacy self-efficacy (belanger & crossler, 2019; crossler & bélanger, 2019; liu et al., 2023; soh et al., 2022), and security selfefficacy (silic & lowry, 2020; wang et al., 2023). in mobile security, mobile-specific constructs like mobile-computing self-efficacy (keith et al., 2015), mobile privacy protection self-efficacy (belanger & crossler, 2019), and mobile anti-phishing self-efficacy (verkijika, 2019) emerge. this study suggests mobile identity protection self-efficacy to understand individuals' capabilities in safeguarding mobile identities. while most studies have looked at how self-efficacy directly affects outcomes, only a few researchers have explored its role in moderating or mediating relationships. for instance, reychav et al. (2019) examined the reliability of using mobile apps as a point of contact between patients and healthcare providers. their findings reveal the significant moderation australasian journal of information systems alhelaly et al. 2024, vol 28, research article mobile identity protection 5 impact of self-efficacy on mobile technology identity and self-report reliability. similarly, soh et al. (2022) emphasized how privacy self-efficacy moderates the connection between social capital bridging/bonding and social media addiction. recently, yazdanmehr et al. (2023) highlighted the moderation impact of isp-related self-efficacy on security-related stress (srs) and coping responses. hampel et al. (2023) also utilized self-efficacy to improve digital tech attitudes, showing robot-specific self-efficacy mediating vicarious experience and tech enthusiasm. in the mobile security context, verkijika (2018) defined self-efficacy as an individual's perception regarding their skills and ability to perform a given security behavior. verkijika (2019) argues that while security literature acknowledges self-efficacy as a potential tool for preventing security threats, its role in influencing security behaviours has yielded mixed results. moreover, the shift from desktop to mobile devices in the last years has captured hackers' attention, prompting the need for security research that explores the connection between self-efficacy and other security aspects tailored to the mobile environment. in this study, we conceptualize the self-efficacy concept in the context of mobile security by suggesting identity protection self-efficacy as a key predictor of an individual's motivational intentions toward actual identity protection. we also examine the moderation impact of identity protection self-efficacy on threat appraisal, motivation, intentions, and actual behavior toward mobile identity protection. table a1 in the appendix summarizes selfefficacy conceptualization in information systems literature. 2.3 threat appraisal and protection motivation the third stream of literature delves into individual threat appraisal and its exploration of motivation-behavioural intention toward mobile identity protection. threat appraisal reflects people's perceptions of their susceptibility to a threat's perceived severity (chen & zahedi, 2016). in the context of mobile identity threats, perceived vulnerability relates to how likely someone believes their identity is to be targeted by a theft threat, while perceived severity relates to the perceived seriousness of the potential consequences of that threat. both vulnerability and severity are factors that contribute to an individual's overall mobile identity threat appraisal and subsequently influence their protective intentions and behaviours toward identity protective measures. to explain the influence of threat invocations on an individual’s behaviours, rogers (1975) developed the protection motivation theory (pmt), which examines how people assess threats and the subsequent influence on their behavior. rogers based his theory on two cognitive processes: threat assessment and handling assessment (bada & sasse, 2014). notably, ogbanufe & baham (2023) emphasize the importance of choosing an adequate theoretical framework that aligns with the specific system type and objectives. they argued that while technology adoption and use theories center around systems that improve individual productivity and performance, security systems primarily address the prevention of pain, damage, risks, and threats associated with technologies. in light of this, ogbanufe & baham (2023) suggest using pmt for security studies that aim to explore how motivation drives the adoption of protective measures against technology-related threats. along the same line, the seminal work of bandura (1978) suggested that the principle of self-efficacy is given a central function in evaluating improvements in fearful and threat avoidance behavior. both threat appraisal and self-efficacy have been correlated and used in pmt to study information security behavior (maddux & rogers, 1983; rogers, 1975; wittee, 1996). over the australasian journal of information systems alhelaly et al. 2024, vol 28, research article mobile identity protection 6 years, pmt has become widely employed in information security research to aid in theorizing about individual motivation to change security-related behaviours to protect themselves and their organizations (boss et al., 2015). recently, many researchers have deployed pmt to understand the impact of threat appraisal on protection motivation behavior. for instance, ogbanufe & pavur (2022) used perceived threats to explore adaptive protection motivation for identity theft protection. however, their study did not investigate the relationship between motivation intentions and actual behavior in mobile identity protection. in this context, verkijika (2019) emphasizes the significance of examining real protective behaviours to fully comprehend how security intentions translate into actions. in the same vein, thompson et al. (2017) encourage deploying models of security intentions to extend and examine the link between intentions and actual security behaviours. motivated by the backdrop of the preceding arguments and drawing from the framework of the pmt, this study focuses on exploring perceived threat appraisal to gain a deeper understanding of motivational intentions and subsequent actions related to mobile identity protection against threats and attacks. in the upcoming section, we will discuss the role of identity protection awareness in shaping individuals' behaviours and decisions within the realm of mobile security. 2.4 identity protection awareness weixun li et al. (2023) highlighted the crucial role of information security awareness as the linchpin for integrating comprehensive security solutions. information security awareness is defined as a unified awareness of security threats and countermeasures, functioning as a foundational element for robust security maintenance (weixun li et al., (2023). in the information security context, jaeger & eckhardt (2021) recommend using awareness of situations to promote personal security behavior. this strategy empowers individuals to effectively promote identity theft protection and security-related actions. in line with this perspective, khando et al. (2021) emphasized the significant impact of awareness on information security behavior, underscoring its priority in both the research and practice domains. as discussed earlier, this stems from the recognition that humans frequently represent the most vulnerable aspect when it comes to securing systems, especially mobile devices (khando et al., 2021). similarly, jaeger (2018) noted the evolving nature of research on information security awareness, highlighting the unexplored territories within the field of information systems. despite the proliferation of studies on information security awareness, a comprehensive understanding of its scope and its interactions with other constructs remained lacking (jaeger, 2018). given these considerations, our study finds significance in exploring the role of awareness in elucidating actual identity protection behavior within the mobile environment. in essence, our study aligns with the current stream of research in is, shedding light on the intricate dimensions of mobile security. by exploring the nexus between self-efficacy, threat appraisal, awareness, and protection motivation, we strive to advance the comprehension of user intentions and actual behavior toward identity protection within the evolving landscape of mobile security. consequently, recognizing the critical significance of this intersection, we have undertaken a gap analysis to examine the identity protection behavior of individuals in the mobile environment. this gap analysis addresses a pivotal question: what are the lacunae in our understanding of the roles of self-efficacy, threat appraisal, and protection awareness in motivating individuals to achieve mobile identity protection? australasian journal of information systems alhelaly et al. 2024, vol 28, research article mobile identity protection 7 our gap analysis revealed three gaps in the existing literature. first, we discovered that the context of identity protection is often overlooked in mobile security literature. second, despite a large body of existing research about self-efficacy in the information systems field (please see table a2 in the appendix), we discovered that the moderation impact of self-efficacy is also underexplored. finally, the role of protection awareness as a mediator has been overlooked in the extant literature, especially on the relationship between self-efficacy and actual protection behavior. table a2 in the appendix summarizes the gap analysis for the four bodies of informing literature. 3 conceptual model and hypotheses development two theories inform our conceptualization of mobile identity protection – the self-efficacy theory (bandura, 1994) and the protection motivation theory (rogers, 1975). integrating both theories allows us to enhance the explanatory power of each model (robert, 2002). identity protection self-efficacy was drawn from kim & kankanhalli (2009). perceived severity and perceived vulnerability were drawn from boss et al. (2015), the identity protection motivational intentions construct was based on taylor & todd (1995), and identity protection awareness was informed by bulgurcu et al. (2010). finally, actual identity protection was based on pavlou & fygenson (2006). figure 1 shows our conceptual research model. we propose that while threat appraisal only has a positive direct effect on mobile identity motivational intents, self-efficacy has both a direct and a moderating effect. furthermore, we hypothesize that protection awareness serves as a full mediator between self-efficacy in identity protection and actual mobile identity protection. figure 1. conceptual motivation model for mobile identity protection 3.1 the role of threat appraisal the pmt theory was designed to explain how risky behavior can be manipulated and how important it is to understand the components of a motivating message (rogers, 1975; rogers, 1983). when faced with a threat, individuals create their perceptions of threat severity and vulnerability (rogers, 1975; witte, 1992; wittee, 1996). furthermore, fear-inducing persuasive messages are effective in changing people's attitudes, behavioural intent, and actions. threat appraisal is represented in our model as a second-order reflective-formative type construct. the construct is composed of perceived vulnerability and perceived severity. perceived vulnerability describes the user's sense of susceptibility to losing mobile personal information, australasian journal of information systems alhelaly et al. 2024, vol 28, research article mobile identity protection 8 as well as the chance of becoming a victim of identity theft attacks. perceived severity describes the compromised identity’s perceived repercussions due to mobile loss or identity theft attacks. hence: h1: threat appraisal is a second-order construct of perceived threat appraisal and vulnerability. threat appraisal positively influences identity protection motivational intentions. 3.2 the role of identity protection self-efficacy prior research has shown that self-efficacy beliefs are reliable predictors of behavioural outcomes such as performance (ouweneel et al., 2013) and are positively related to the motivational effects of goals (bandura & cervone, 1983). self-efficacy has proven to be a more consistent predictor of behavioural outcomes than any other motivational construct (graham & weiner, 1996). self-efficacy has been discovered to be a significant predictor of motivational behaviours (chen & zahedi, 2016). in this study, we posit that identity protection self-efficacy will affect identity protection motivational intentions, protection awareness, and actual identity protection. hence: h2: identity protection self-efficacy positively influences identity protection motivational intentions. h3: identity protection self-efficacy positively influences actual identity protection. h4: identity protection self-efficacy positively influences protection awareness. we also posit that self-efficacy moderates the relationship between threat appraisal and identity protection motivational intentions, the relationship between identity protection motivational intentions and actual identity protection, and the relationship between identity protection awareness and actual identity protection. thus: h2a: identity protection self-efficacy negatively moderates the effect of threat appraisal on identity protection motivational intentions. h3a: identity protection self-efficacy positively moderates the effect of identity protection motivational intentions and actual identity protection. h3b: identity protection self-efficacy negatively moderates the effect of identity protection awareness on actual identity protection. 3.3 the role of identity protection motivational intentions many social psychology researchers employ the theory of expected action (tpb) paradigm to model planned behaviours (ajzen, 1991). motivating behavioural intentions measures how hard people are willing to carry out a task. according to tpb, behavioural intention is the most powerful predictor of behaviours; after all, people do what they intend to do (pavlou & fygenson, 2006). according to early & chaiken (1998), behavioural intentions are a pivotal component in the model relating to the connection between attitudes and behaviours. the immediate cause of conduct is thought to be behavioural intention (fishbein & ajzen, 1980). behavioural intention is a measure of a person’s willingness to complete the acts required to attain specific objectives, and it reflects the motivational factors that underpin the actions (guillon et al., 2004). in the mobile identity protection context, it is therefore critical to measure mobile users’ intentions to protect their identity using different protection features such as face, fingerprint, passwords, or pin. hence, we propose the following hypothesis based on the preceding arguments: h5: identity protection motivational intention positively influences actual identity protection. australasian journal of information systems alhelaly et al. 2024, vol 28, research article mobile identity protection 9 3.4 the role of identity protection awareness the most significant factor influencing user behavior in terms of steps taken to defend themselves against threats is an individual's awareness (dhillon & chowdhuri, 2013; johnston & warkentin, 2010). the primary goal of awareness is to raise consciousness and to make people more open to change (dhillon et al., 2020; manke & winkler, 2013). psychologically, a person who is bombarded with too many warnings and recommendations could be tempted to give up all attempts to defend himself and not be concerned about any threat (fisher & rost, 1986). threatening or intimidating messages are ineffective because they increase the individual's stress level to the point that the individual can become repulsed or deny the existence of any threat (bada & sasse, 2014). for the same reason, we found that it is crucial in this study to investigate and measure users’ awareness that may influence the actual behavior of protecting their mobile against identity threats. although smartphones have received considerable attention in the security literature, the relevant studies on the security awareness of mobile identity are currently somewhat limited. our proposed model includes identity protection awareness not only as a direct predictor of actual identity protection but also as a mediator between the relationship between identity protection self-efficacy and actual identity protection (kraiger et al., 1993). therefore, we hypothesize that: h6: identity protection awareness positively influences actual identity protection. 4 methods 4.1 measurement we first developed our measurement tool and subsequently crafted a questionnaire through the qualtrics online survey platform. subsequently, we seamlessly integrated the survey with a prolific crowdsourcing platform for efficient data collection. our approach to selecting evaluation items was guided by meticulously considering established constructs within the information systems literature (bagozzi, 2011). we examined previously validated measures from reputable sources exhaustively to ensure the credibility and relevance of our chosen items. perceived appraisal items encompassing perceived severity and perceived vulnerability were adopted from boss et al. (2015). identity protection self-efficacy items were derived from kim & kankanhalli (2009), while identity protection awareness was drawn from bulgurcu et al. (2010). the measures of identity protection motivation intention were adapted from taylor & todd (1995), and finally, the measures of actual identity protection were adopted from pavlou & fygenson (2006). by adopting these established measures, we aimed to capture the nuances of each construct accurately, contributing to the overall validity of our research. finally, a consistent seven-point likert scale, ranging from ‘strongly disagree’ to ‘strongly agree,’ was employed across all measurement items, as illustrated in table 1. a panel comprising five experts in information systems (is) was convened to ensure the robustness of our measurement approach for the meticulous evaluation and refinement of our model’s content and validity. among the panel members, two accomplished professionals in the is security field hold prominent positions. the first individual boasts over two decades of comprehensive experience in information systems, focusing on specialized areas like mobile security and privacy. the second expert, well-versed in information management, contributes a thorough understanding of identity management systems. the remainder of the panel comprises skilled researchers: two ph.d. students immersed in the nuances of mobile security research and a post-doctoral researcher actively engaged in a research project centered on australasian journal of information systems alhelaly et al. 2024, vol 28, research article mobile identity protection 10 mobile security and privacy. the panel’s expertise spans continents, with the professionals representing the us and the researchers stationed in europe. table 1. the measurement items we adopted a structured facilitation approach to mitigate the impact of dominant panel members (o’sullivan et al., 2015; o’sullivan et al., 2022), allowing for balanced participation. anonymous contributions were utilized to ensure unbiased input. discussion leadership construct items instrument reference threat appraisal (ta) secondorder construct perceived severity (ps) ps1 if my personal information were stolen from my mobile due to an identity theft attack, i would suffer a lot of pain. boss et al. (2015) ps2 if my mobile identity were compromised, it would be severe. ps3 if my mobile identity were compromised, it would be serious. ps4 if my mobile identity were compromised, it would be significant. perceived vulnerability (pv) pv1 i am unlikely to lose personal information from my mobile in the future pv2 my mobile is at risk of becoming a victim of identity theft attacks. pv3 my mobile will likely become a victim of an identity theft attack. pv4 my mobile may become a victim of an identity theft attack. identity protection self-efficacy (ipse) se1 based on my knowledge, skills, and abilities, protecting my mobile identity would be easy for me. kim & kankanhalli (2009) se2 i can protect my mobile identity without the help of others se3 i can protect my mobile identity reasonably well on my own identity protection awareness (ipaw) ipaw1 overall, i am aware of the mobile identity threats and their negative consequences. bulgurcu et al. (2010) ipaw2 i have sufficient knowledge about the cost of potential identity theft problems. ipaw3 i understand the concerns regarding mobile identity protection and the risks they pose in general. identity protection motivation intention (ipmi) bi1 i intended to protect my mobile identity taylor & todd (1995) bi2 i intend to use my mobile using (face, fingerprint, passwords, or pin) identity protection features. bi3 i intend to protect my mobile identity frequently. actual identity protection (aip) aip1 i expect my mobile identity to be protected when i apply identity protective actions (strongly disagree/agree) pavlou & fygenson (2006) aip2 actual protection of my mobile identity information would make it much more (difficult/easier) for me to apply identity-protective actions. aip3 i feel secure that my personal information is kept private when i protect my mobile identity (strongly disagree/agree). aip4 feeling secure that my mobile identity is protected would make it much more (difficult/easier) for me to apply identity protection actions. australasian journal of information systems alhelaly et al. 2024, vol 28, research article mobile identity protection 11 rotated among members, minimizing consistent dominance. we encouraged diverse viewpoints and dissenting opinions to foster inclusive debates. regular feedback checkpoints ensured everyone's input was considered, maintaining a collaborative environment and preventing undue influence. in response to this panel's insights, we fortified our questionnaire by enhancing item quality in terms of clarity and context. the ensuing questionnaire results not only validated the integrity of our data but also furnished compelling evidence of the instrument's robust validity and reliability. 4.2 data collection data collection occurred between march and april 2022, facilitated by prolific data collection services. our approach began with an exploratory pilot involving 80 participants, which yielded valuable insights into the reliability and validity of the employed scales. building upon the successful pilot, we expanded our sample to encompass 303 additional respondents, resulting in a comprehensive dataset of 383 responses. the online survey, designed to be completed within 10–15 minutes, garnered an average completion time of approximately 12 minutes per participant. for our sample size, we followed partial least squares structural equation modelling (plssem) principles and acknowledged the importance of robust analysis (hair et al., 2014). considering our scenario, our study encompasses two dependent variables. one of these variables entails three paths (r2= 20.6%), while the other involves five paths (r2= 51.1%). we followed the guidelines of cohen (1992) for the minimum sample size recommendation to establish the adequacy of our sample size. for significance levels of 5% and r² values around 0.25, the recommended minimum sample size is 59 observations. similarly, for r² values around 0.50, the suggested minimum is 42 observations. notably, our sample size surpasses these thresholds. in this study, we selected 383 responses to increase the precision (i.e., consistency) of pls-sem estimations and ensure informed and meaningful results, as well as substantiated inferences (hair et al., 2021). table 2. profile of the respondents our survey comprised a diverse participant pool, offering insights into our sample's composition. guided by the hair et al. (2021) guidelines, our sample size of 383 responses with zero missing values ensured robust statistical power and enhanced estimation precision. gender distribution was balanced, with 48% females and 52% males. participants ranged from 18 to 68 years, reflecting varying ages. educational backgrounds varied, with 34% holding high school degrees, 42% possessing bachelor's degrees, and 21% having post-graduate qualifications. notably, 3% reported incomplete schooling. geographically, our participants represented a global presence, with 80% from europe, 9% from the usa and canada, and 11% from diverse regions. these characteristics underscore the strength of our statistical characteristics frequency (%) respondents (n = 383) characteristics frequency (%) respondents (n = 341) gender female 48% 185 educati onal degree no school 3% 11 male 52% 198 high school 34% 130 age 18-24 43% 164 bachelor 42% 160 25-44 46% 178 master 19% 73 44-68 11% 41 ph.d. 2% 9 australasian journal of information systems alhelaly et al. 2024, vol 28, research article mobile identity protection 12 methodology, samples’ diversity, and amplifying the relevance of our findings within our research context. table 2 above displays the respondent profile. recent work in is has highlighted the importance of evaluating the influence of common method bias (cmb) on the results of statistical analysis (chin et al., 2012). cmb occurs when the estimates of the relationships between two or more constructs are biased because they are measured with the same method (podsakoff & organ, 1986). we assess cmb with two methods: (1) harman's one-factor test to identify common method variance (podsakoff et al., 2003), and (2) the pls marker variable approach to analyse data contaminated with method (lindell & whitney, 2001). the former confirmed that none of the variables alone accounts for the bulk of the variation. the latter included a potentially unrelated marker variable in the study model, resulting in a maximum mutual variance with other variables of 0.008 (0.8 percent), which is considered a low value (johnson et al., 2011). as a result, no substantial cmb was discovered. 5 results of data analysis we used pls-sem regression using smart pls 3.0 to conduct our analysis in this study. the literature has suggested that the approach works best when: 1) the model has never been tested before (ke et al., 2009), 2) the latent constructs are moulded with formative indicators (goo et al., 2009), and 3) evaluating path coefficients that are substantially different from zero, it is important to avoid restrictive distributional assumptions (fornell & bookstein, 1982). 5.1 measurement model we assessed the construct reliability, convergent validity, indicator reliability, and discriminatory validity of scales for reflective constructs through a measurement model. given the nature of our approach utilizing pls analysis, we opted for composite reliability (cr) as our metric for assessing construct reliability. however, it is noteworthy that some researchers may prefer cronbach's alpha due to its tendency to provide higher reliability estimates. however, a major limitation of cronbach’s alpha is that it assumes all indicator loadings are the same in the population. according to hair et al. (2021), violating this assumption could result in lower reliability values than those produced by composite reliability. both constructs provided cr values greater than 0.7 (table 3), indicating that the constructs were internally consistent and acceptable (henseler et al., 2009; straub, 1989). the average variance extracted (ave) was used to demonstrate convergent validity. all constructs' ave values are higher than 0.50 (table 3). as a result, the measurement model's convergent validity is established (fornell & larcker, 1981; hair et al., 2012). for indication reliability, the loading should be more than 0.7 (churchill jr, 1979; henseler et al., 2009). all loadings in table 4 are more than 0.7, suggesting that the reliability indicator has been satisfied. we used three criteria to evaluate discriminatory validity: (1) fornell-larcker criteria, (2) cross-loadings, and (3) heterotrait-monotrait ratio (htmt) (henseler et al., 2015). the fornell-larcker criteria assess discriminant validity in structural equation models. it compares the square root of average variance extracted (ave) with correlations between constructs (hair et al., 2021). ave should exceed correlations to ensure distinctiveness. this requirement safeguards against overlapping or redundant constructs and establishes a clear distinction between constructs. australasian journal of information systems alhelaly et al. 2024, vol 28, research article mobile identity protection 13 table 3. descriptive statistics, correlation, composite reliability (cr), and average variance extracted (ave) table 4. loadings and cross-loadings notably, this distinction is established in our model in the case of threat assessment (ta), which operates as a second-order construct comprising perceived vulnerability (pv) and perceived severity (ps). as demonstrated in table 4, the diagonal elements, representing the square root of ave for each construct, surpass the correlations between these constructs. this element substantiates the fulfillment of the first criterion for ensuring discriminant validity, as outlined by fornell & larcker (1981). the second criterion is cross-loading, a crucial element for evaluating discriminant validity in structural equation modelling. it involves indicators of one construct loading on other construct mean sd cr ipse pv ps ta ipmi ipa aip identity protection self-efficacy (ipse) 4.563 1.472 0.940 0.916 perceived vulnerability (pv) 3.547 1.194 0.895 -0.290 0.825 perceived severity (ps) 4.682 1.417 0.936 0.021 0.242 0.886 threat appraisal (ta) 4.239 1.076 0.865 -0.109 0.628 0.907 0.676 identity protection mutational intention (ipmi) 5.670 1.128 0.908 0.295 0.013 0.352 0.288 0.876 identity protection awareness (ipa) 4.821 0.963 0.891 0.282 -0.061 0.308 0.221 0.441 0.820 actual identity protection (aip) 5.325 0.946 0.860 0.234 -0.042 0.319 0.238 0.662 0.532 0.778 construct items ipse pv ps ipmi ipa aip identity protection self-efficacy (ipse) ipse1 0.889 -0.291 0.008 0.272 0.251 0.218 ipse2 0.926 -0.261 0.029 0.274 0.260 0.217 ipse3 0.933 -0.244 0.037 0.264 0.265 0.208 perceived vulnerability (pv) pv1 -0.232 0.756 0.237 0.041 -0.086 0.012 pv2 -0.236 0.881 0.221 0.010 -0.088 -0.058 pv3 -0.245 0.836 0.173 -0.024 -0.042 -0.070 pv4 -0.243 0.822 0.162 0.013 0.024 -0.020 perceived severity (ps) ps1 -0.011 0.184 0.826 0.309 0.279 0.291 ps2 0.041 0.215 0.908 0.317 0.304 0.313 ps3 0.043 0.224 0.917 0.324 0.257 0.258 ps4 -0.002 0.231 0.889 0.298 0.252 0.271 identity protection mutational intention (ipmi) ipm1i 0.278 0.017 0.362 0.884 0.454 0.623 ipmi2 0.255 -0.016 0.235 0.811 0.301 0.494 ipmi3 0.242 0.027 0.316 0.929 0.388 0.609 identity protection awareness (ipa) ipa1 0.239 -0.009 0.285 0.389 0.819 0.460 ipa2 0.140 -0.035 0.165 0.332 0.780 0.388 ipa3 0.280 -0.077 0.253 0.374 0.827 0.452 actual identity protection (aip) aip1 0.193 -0.096 0.262 0.487 0.281 0.758 aip2 0.279 -0.180 0.228 0.512 0.439 0.769 aip3 0.121 0.062 0.267 0.535 0.329 0.803 aip4 0.128 0.093 0.309 0.527 0.377 0.787 australasian journal of information systems alhelaly et al. 2024, vol 28, research article mobile identity protection 14 constructs, possibly indicating construct ambiguity (hair et al., 2021). in this criterion, the loadings (in bold) should be greater than cross-loadings to prove measurement validity (chin, 1998). our analysis consistently reveals that loadings (in bold) significantly outperform crossloadings, as showcased in table 4. this robust pattern underscores the distinct nature of our indicators, reinforcing the reliability and accuracy of our measurements. our third method for evaluating discriminant validity is the heterotrait-monotrait (htmt) ratio of correlations. this approach compares correlations between different constructs to those within the same construct, determining their distinctiveness (henseler et al., 2009). table 5 shows that all htmt values are below the 0.9 threshold, signifying robust discriminant validity. this technique ensures the accurate capture of unique construct variances, reinforcing the measurement model's validity and bolstering confidence in interpreting construct relationships and effects in subsequent analyses. table 5. heterotrait-monotrait ratio (htmt) note. values in diagonal (bolt) are the ave square root we modelled threat appraisal (ta) as a reflective-formative second-order construct (hair et al., 2012), with reflective perceived severity (ps) and perceived vulnerability (pv). these constructs are ta formative measures. for the formative construct, a measurement model was used to determine multicollinearity, as well as the importance and sign of weights, for the formative build. the variance inflation factor (vif) statistic is used to assess multicollinearity. table 6 shows that the vif values are below the 3.3 mark, implying that the variables are not multicollinear (lee & xia, 2010). the two constructs are statistically significant (p 0.01) and have a positive sign. as a result, the structural model can be tested using the formative construct. table 6. formative measurement model evaluation note. *** p < 0.01; ** p < 0.05; * p < 0.10 5.2 structural model we calculated the multicollinearity of all constructs using the variance inflation factor (vif). the vif is 1.062, which is less than the threshold of 3.3, meaning that the variables are not multicollinear (lee & xia, 2010). figure 2 presents that structure model that explains the variation and the path coefficients. bootstrapping with 5000 resamples was used to determine the statistical significance levels of the hypothesized construct. construct ipse pv ps ta ipmi ipa aip identity protection self-efficacy (ipse) perceived vulnerability (pv) 0.333 perceived severity (ps) 0.034 0.277 threat appraisal (ta) 0.228 0.922 0.928 identity protection mutational intention (ipmi) 0.337 0.035 0.397 0.295 identity protection awareness (ipa) 0.318 0.089 0.348 0.301 0.514 actual identity protection (aip) 0.276 0.171 0.380 0.367 0.807 0.652 formative construct (second-order construct) constructs (first-order reflective) weights vif threat appraisal (reflective-formative type) perceived vulnerability (pv) 0.440*** 1.062 perceived severity (ps) 0.802*** 1.062 australasian journal of information systems alhelaly et al. 2024, vol 28, research article mobile identity protection 15 figure 2. structural model for mobile identity protection note: *** p < 0.01; ** p < 0.05; * p < 0.10 the research model explains 20.6% of identity protection motivational intentions (ipmi). both perceived severity (�̂�𝛽= 0.802; p < 0.01) and perceived vulnerability (�̂�𝛽= 0.434; p < 0.01) are statistically significant to form threat appraisal (ta), confirming that ta is a second-order reflective-formative type construct (ringle, sarstedt, and straub 2012). the hypotheses of identity protection self-efficacy (�̂�𝛽= 0.323; p < 0.01), and threat appraisal (�̂�𝛽= 0.316; p < 0.01) are statistically significant and explain identity protection motivational intentions (ipmi). hence, h1 and h2 are supported. also, the hypothesis of identity protection self-efficacy (�̂�𝛽= 0.282; p < 0.01) is statistically significant in predicting awareness protection. hence, h5 is supported. only the hypothesis (ipse) is statistically insignificant (�̂�𝛽= -0.011; p > 0.01). hence, h3 is not supported for explaining actual identity protection (aip). the structure model explains 51.5 % of the variation in actual identity protection (aip). hypothesis identity protection motivational intentions (�̂�𝛽= 0.546; p < 0.01), and identity protection awareness (�̂�𝛽= 0.296; p < 0.01) are statistically significant to explain actual identity protection (aip). thus, h4 and h6 are supported. the moderating effect between threat appraisal and identity protection motivational intentions is confirmed (�̂�𝛽= 0.106; p < 0.10); and between identity protection motivation intentions and actual identity protection is also confirmed (�̂�𝛽= 0.075; p < 0.05); hence h2a and h3a are confirmed. however, identity protection self-efficacy is not a statistically significant moderator (�̂�𝛽= 0.013; p > 0.10) between identity protection motivational awareness and actual identity protection. hence, h3b is not supported. 6 implications our research has four important theoretical implications: (1) perceived appraisal of mobile identity threats motivates users to protect their mobile identity. (2) low identity protection self-efficacy and high perceived threats result in high motivation to engage in identity protection. (3) high identity protection self-efficacy and high perceived threats result in low motivation to engage in identity protection. the protection awareness fully mediates the relationship between identity protection self-efficacy and actual identity protection. our findings undoubtedly contribute to reframing the self-efficacy construct in the context of mobile identity protection. however, to ensure what our study revealed, we engaged our australasian journal of information systems alhelaly et al. 2024, vol 28, research article mobile identity protection 16 panel of experts to discuss the findings. validation can occur before an instrument is deployed; it can also be done after identifying the quantitative results. as straub et al. (2004) have argued, “validation guidelines are owned by communities of practice” (p. 417). venkatesh et al. (2013) discuss mixed methods research and explicitly call for a qualitative assessment following the quantitative phase. during the interviews, we initially presented our study objectives, the measurement tool, and the significant findings to the panel members. we then asked the panellists to evaluate the findings with the content of the corresponding items. in the paragraphs below, we discuss our findings and their theoretical contributions and implications in light of what our panellists considered. perceived appraisal of mobile identity threats and identity protection. our study enhances the understanding of modelling the role of self-efficacy in motivation as an important concern for identity protection (tubbs, 1994). we do so by assessing the moderating effect of identity protection self-efficacy on motivating mobile users to achieve actual identity protection (o’leary-kelly et al., 1994). furthermore, we find that self-efficacy is also significant in the relationship between identity protection motivation intention and actual mobile identity protection. we presented our findings to the expert panel. all the participants confirmed the results and found them consistent with the resultant constructs, items, and the related content of each item. they all agreed that threat appraisal is a strong positive predictor of the motivational intentions of mobile identity protection. our findings align with the (khan et al., 2023) study results, suggesting that pmt-based training effectively increases threat knowledge and individuals' self-efficacy, which significantly predicts cybersecurity behavioural intention. one of the consultants who advises corporate clients about mobile identity protection noted: when i am discussing identity protection issues with my clients, a recurring theme is how well they are versed in the prevalent threats. i usually find that the more the individuals know about the threats, the greater their motivation to protect. i found this to be such an important issue, particularly when many employees of my client organizations are working remotely. while the finding may sound intuitive, individuals tend to take identity protection for granted in a mobile environment. as one of our user respondents noted: it is so difficult to work with individual settings of different apps. i just trust the device and the operating system. i am sure there are threats, but ios and android have probably figured it out. a similar sentiment is found in the extant literature. maddux & rogers (1983), for example, posit that a threat’s occurrence has a positive effect on an individual’s intentions to adopt preventive behavior. recently, menard et al. (2017) found that both perceived threat severity and perceived threat susceptibility positively influence an individual's behavioural intention to undertake secure behaviours in their study on user motivations in protecting information security. in a similar vein, johnston et al. (2023) suggest that to enhance the effectiveness of fear appeals; these should incorporate both threat appraisal and efficacy elements to evoke fear and exert influence on individuals' protective behavioural intentions. identity protection self-efficacy and identity protection motivation. our research found an interesting relationship between identity protection self-efficacy and identity protection motivation. low self-efficacy results in higher identity protection motivation. conversely, with high self-efficacy, motivation to engage in identity protection is low. figure 3 indicates that in a mobile environment with low identity protection self-efficacy, threat appraisal is more important to explain the identity protection motivational intentions. this aspect means that australasian journal of information systems alhelaly et al. 2024, vol 28, research article mobile identity protection 17 the perceived threat appraisal of mobile identity threats motivates users with low self-efficacy to protect their mobile identity. in such cases, the perceived threats to individuals with low identity protection self-efficacy increase their motivational intentions to protect their mobile identities. all members of our panel found the findings to be relevant. low self-efficacy and high perceived threat appraisal are indeed strong motivators to achieve identity protection in the mobile context. as one of the respondents noted: in my line of work, i see this all the time. i often find people who have no faith in their capabilities, but when they are made aware of the persistent threats and consequences of their identity loss, they suddenly become motivated to protect their identity. once, i was conducting a workshop at a large telecommunication firm, and it was surprising to see how even the more technologically advanced individuals had no faith in their ability to protect themselves. when i introduced the range of threats that they were exposed to, their motivation to protect increased. the extant literature also suggests a similar orientation. for instance, studies conducted by (maddux & rogers, 1983; rogers, 1975) found perceived threats and self-efficacy to impact users' behavioural intentions for protective actions. johnston & warkentin (2010) also posit that threat appraisal impacts individuals' protective behavioural intentions, which are not uniform across all individuals. however, it is influenced by self-efficacy, threat severity, and threat vulnerability, among other factors. figure 3. moderation effect of self-efficacy on the relationship between ta and ipmi one of the interesting findings in this study shows that in an environment with mobile users with high identity protection self-efficacy, threat appraisal is not so important in explaining identity protection motivational intention. this dimension is in contrast to individuals with low mobile identity protection self-efficacy, where mobile users with high self-efficacy are less motivated to engage in identity protection when they perceive high identity threats (figure 4). australasian journal of information systems alhelaly et al. 2024, vol 28, research article mobile identity protection 18 figure 4. moderation effect of self-efficacy on the relation between ipmi and aip our panel of experts found no general disagreement. one panellist, for instance, noted: the high self-efficacy of mobile users leads to overconfidence in their efficacy in actual identity protection. in my workplace, overconfidence became a real challenge for our cybersecurity team. in one case, employees claim they have enough knowledge to keep their mobile identity protected. yet, when they were queried, few employees knew what multi-factor authentication was, and even fewer knew how to use it to protect their identity. it is interesting to note that this phenomenon is also pointed out in the literature. stone (1994) discusses the negative effect of overconfidence on individuals' self-efficacy and performance. according to stone, mild underconfidence can have a more positive motivational impact than overconfidence and heavy underconfidence. however, one of the panellists had a different explanation. he said: from a psychological perspective, mobile users with high self-efficacy may generate a relaxed attitude that reduces their motivational intentions to protect their mobile identity. high selfefficacy users feel less stressed regarding perceived identity threats, which may eventually affect their intentions to carry out the actual protective behavior. i believe age has something to do with this. i think that youngsters are more relaxed about identity protection practices than the older group. this possibly makes them more vulnerable to mobile identity theft and related threats. it is noteworthy that extant literature in education supports the assertion about the negative impact of high self-efficacy on individuals’ subsequent behavior. for example, vancouver et al. (2001) concluded that high self-efficacy creates relaxation in the student’s behavior regarding exams, which impacts their performance in later examinations over time. perhaps a future experimental study can provide a deeper insight. threat appraisal, identity protection self-efficacy, and identity protection motivation intention. our research suggests that self-efficacy positively impacts individual intention for identity protection. prior studies have indicated that self-efficacy enhances complex task performance and impacts context-oriented dynamic tasks (e.g., mobile identity protection) by australasian journal of information systems alhelaly et al. 2024, vol 28, research article mobile identity protection 19 increasing individuals' motivation (thangavelu et al., 2021). self-efficacy has been shown to boost motivation and affect task success (ballout, 2009). individuals with high self-efficacy beliefs put in more effort, which leads to the achievement of their goals (bandura, 1982; bandura et al., 1999). we also found appraisal of existing threats helps individuals to protect their mobile identity. both perceived threat vulnerability and threat severity significantly influence an individual’s intention to protect mobile identity. prior research relates perceived vulnerability and perceived severity and antecedents to threat appraisal (khan & das, 2016). this phenomenon suggests that threat appraisal affects individual behavioural intentions to perform recommended individual behaviours, which are partly influenced by self-efficacy perceptions (johnston & warkentin, 2010). our findings are consistent with extant literature, highlighting the importance of threat appraisal in motivating individual behavioural intentions toward mobile identity protection. the role of protection awareness in mobile identity protection. while the present study affirms the importance of protection awareness in boosting the individual’s actual mobile identity protection, our finding revealed the mediation impact of protection awareness on the relationship between an individual’s identity protection self-efficacy and actual identity protection in a mobile environment. there were different interpretations of the findings. one panellist suggests that: low self-efficacy in mobile identity protection may lead to protection procrastination, which indicates that mobile users with low capabilities in safeguarding their mobile identity may delay or put off protection tasks until they detect an identity threat, or their identity being compromised. this explains why self-efficacy has a minor impact on mobile users' real identity protection while having a substantial impact on their motivational intentions. from my experience, awareness of mobile identity threats and their negative consequences may encourage individuals to focus on protection tasks and alleviate the tendency to procrastinate. procrastination is the act of delaying the completion of tasks that must be finished within a particular amount of time (kirst-ashman, 2016). in the information security context, procrastination can act as a motivational mechanism, emerging from stress or threat, leading to postponed security protection actions, reduced security behavior, avoidance of security tasks (xu & guo, 2019), and increased vulnerability to privacy and security risks (xiao & spanjol, 2021). the interpretation of procrastination tendency is consistent with multiple research that implies a link between procrastination and self-efficacy, given the self-regulation concerns that define trait procrastination (ferrari, 1992; tuckman, 1991). self-efficacy is important in the self-regulation of behavior because of its effects on intention formation and strength, as well as action persistence in the face of obstacles (bandura, 1977; 1986). the relationship between procrastination and self-efficacy is consistent with bandura’s (1986; 1977) theory that strong efficacy beliefs encourage the behavior to start and persist, whereas poor efficacy beliefs contribute to action avoidance. a recent study on the relationship between mindfulness and procrastination (cheung & ng, 2019) posits that awareness could minimize people's procrastination by encouraging them to focus on activities they're working on rather than succumbing to distractions. more recently, ali & dominic (2022) suggested implementing security awareness through seta programs to mitigate procrastination behavior and enhance the adoption of protection measures. another respondent suggested: using another motivational concept will be useful to examine the effect of the motivational aspect on an individual’s protective behavior in the mobile context. australasian journal of information systems alhelaly et al. 2024, vol 28, research article mobile identity protection 20 identity protection motivation intention and actual identity protection. our research also found a significant and positive moderating role of self-efficacy on the relationship between individuals’ motivational intentions and actual mobile identity protection. as shown in figure 4, in an environment with high ipse, the importance of ipmi in explaining actual identity protection is higher than in a low ipse environment. thus, an individual with high selfefficacy in mobile identity protection will be intentionally more motivated to achieve actual identity protection. our finding complements and confirms the extant literature showing that the actual behavior is affected by behavioural intention (davis, 1989; davis et al., 1989). the existing literature also demonstrates similar outcomes. for instance, ogbanufe & pavur (2022) investigated the influence of fear on individuals’ motivation to safeguard against identity theft, employing concepts such as self-efficacy and protection motivation. however, their inquiry did not explore how self-efficacy impacts protective behaviours, nor did it delve into the predictive role of awareness in fostering motivation against identity theft. our study contributes valuable insights by addressing these gaps. our findings reveal that self-efficacy serves as a moderator in the relationship between perceived threats and individuals’ intentions to secure their mobile identities. furthermore, our research uncovers that protection awareness acts as a full mediator, connecting self-efficacy to the actual practice of identity protection. these newly revealed insights contribute to a more nuanced comprehension of the dynamics that shape mobile identity protection. moreover, our study significantly enriches the landscape of information security research. for instance, the recent work of weixun li et al. (2023) emphasizes the critical role of security awareness as a bridge to implementing comprehensive security measures. their perspective highlights that an improved grasp of security threats enhances cooperation with security policies, fostering proactive measures to safeguard the security environment. the absence of individual information security awareness also exerts an impact on organizational security. khando et al. (2021) underscore this issue, highlighting that organizations often struggle to safeguard information assets as they rely mainly on technical solutions. nevertheless, they contend that human vulnerabilities are a major factor in many security incidents. consequently, individuals’ information security awareness is pivotal in countering undesirable security behavior. in our study, identity protection awareness is shown to influence an individual’s actual identity protection. despite the vital role of selfefficacy in protecting mobile identity, individuals too often do not realize that they must ensure their mobile protection by themselves. therefore, the knowledge of mobile identity threats that individuals may face, and the use of protective measures are essential. knowing the threats leads to the use of necessary mobile identity protection measures. however, ignorance in this field casts doubt on an individual's expertise and ability to take appropriate protective measures (markelj & bernik, 2015). hence, the finding of this study complements previous awareness studies and highlights the practical significance of fostering mobile identity protection to enhance overall information security posture. building upon these insights, our findings highlight the tangible benefits of enhancing individuals' awareness regarding various identity threat techniques and corresponding protective measures, thus bolstering their mobile identity protection. furthermore, our study underscores the crucial interplay of motivational-behavioural factors in shaping the effectiveness of mobile identity protection awareness programs. as our results suggest, this australasian journal of information systems alhelaly et al. 2024, vol 28, research article mobile identity protection 21 provides a solid foundation for the development of training initiatives and awareness programs aimed at motivating employees to adopt the requisite measures for safeguarding their mobile identities in their workplace. in essence, our study not only underscores the practical implications of security awareness and self-efficacy in bolstering mobile identity protection but also lays the groundwork for actionable strategies to enhance information security in the mobile realm. 7 conclusions this study investigates the gaps in mobile identity protection and the roles of self-efficacy and threat appraisal on an individual’s motivational intentions. we also study the moderating role of individual identity protection self-efficacy on motivational intentions. moreover, we examined the mediation impact of awareness on the relationship between self-efficacy and actual identity protection. our research extends the current understanding of the concept of self-efficacy in the mobile identity protection context. while acknowledging the importance of technical measures for identity protection, we discuss the threat appraisal motivational aspects of mobile identity protection. our findings offer organizations suggestions for managing mobile identity to protect individuals from identity threats and the associated security attacks, particularly using motivational strategies and techniques. for future research, we suggest using the theory of expectancy-value motivation (wigfield & eccles, 2000) in addition to studying mobile identity protection. we also believe that adding expectancy and value constructs to the model could explain the protection motivation behavior in more depth by analyzing the individuals’ expected success in protecting their mobile identity based on their protection perceived values. references ajzen, i. (1991). the theory of planned behavior. organizational behavior and human decision processes, 50(2), 179–211. https://doi.org/10.1016/0749-5978(91)90020-t ali, r. f., & dominic, p. (2022). investigation of information security policy violations among oil and gas employees: a security-related stress and avoidance coping perspective. journal of information science, 016555152210876. https://doi.org/10.1177/01655515221087680 ameen, n., tarhini, a., shah, m. h., madichie, n., paul, j., & choudrie, j. (2021). keeping customers’ data secure: a cross-cultural study of cybersecurity compliance among the gen-mobile workforce. computers in human behavior, 114, 106531. https://doi.org/10.1016/j.chb.2020.106531 bada, m., sasse, a. m., & nurse, j. r. (2019). cyber security awareness campaigns: why do they fail to change behaviour? international conference on cyber security for sustainable society. coventry university, united kingdom. https://doi.org/10.48550/arxiv.1901.02672 bagozzi, r. p. (2011). measurement and meaning in information systems and organizational research: methodological and philosophical foundations. mis quarterly, 35(2), 261–292. https://www.jstor.org/stable/23044044 australasian journal of information systems alhelaly et al. 2024, vol 28, research article mobile identity protection 22 balapour, a., reychav, i., sabherwal, r., & azuri, j. (2019). mobile technology identity and self-efficacy: implications for the adoption of clinically supported mobile health apps. international journal of information management, 49, 58–68. https://doi.org/10.1016/j.ijinfomgt.2019.03.005 ballout, h.i. (2009), "career commitment and career success: moderating role of self-efficacy", career development international, 14(7), 655-670. https://doi.org/10.1108/13620430911005708 bandura, a. (1977). self-efficacy: toward a unifying theory of behavioral change. psychological review, 84(2), 191–215. https://doi.org/10.1037/0033-295x.84.2.191 bandura, a. (1978). self-efficacy: toward a unifying theory of behavioral change. advances in behaviour research and therapy, 1(4), 139–161. https://doi.org/10.1016/0146-6402(78)900024 bandura, a. (1982). self-efficacy mechanism in human agency. american psychologist, 37(2), 122–147. https://doi.org/10.1037/0003-066x.37.2.122 bandura, a. (1986). national inst of mental health. (1986). social foundations of thought and action: a social cognitive theory. hoboken, nj, usa: prentice-hall, inc. bandura, a. (1997). self-efficacy: the exercise of control. new york, ny, usa: w h freeman/times books/ henry holt & co. bandura, a., freeman, w. h., & lightsey, r. (1997). self-efficacy: the exercise of control. journal of cognitive psychotherapy, 13, 158-166. https://doi.org/10.1891/0889-8391.13.2.158 bandura, a., & cervone, d. (1983). self-evaluative and self-efficacy mechanisms governing the motivational effects of goal systems. journal of personality and social psychology, 45(5), 1017–1028. https://doi.org/10.1037/0022-3514.45.5.1017 bandura, a., freeman, w. h., & lightsey, r. (1999). self-efficacy: the exercise of control. journal of cognitive psychotherapy, 13(2), 158-166. https://doi.org/10.1891/08898391.13.2.158 bandura, albert. (1994). self-efficacy. in v. s. ramachaudran (ed.), encyclopedia of human behavior (vol. 4, issue 01, pp. 71–81). (reprinted in h. friedman [ed.]), encyclopedia of mental health. san diego, ca, usa: academic press, 1998. http://happyheartfamilies.citymax.com/f/self_efficacy.pdf belanger, f., & crossler, r. e. (2019). dealing with digital traces: understanding protective behaviors on mobile devices. the journal of strategic information systems, 28(1), 34–49. https://doi.org/10.1016/j.jsis.2018.11.002 bose, i., & leung, a. c. m. (2019). adoption of identity theft countermeasures and its shortand long-term impact on firm value. mis quarterly, 43(1), 313–327. https://doi.org/10.25300/misq/2019/14192 boss, s. r., galletta, d. f., lowry, p. b., moody, g. d., & polak, p. (2015). what do systems users have to fear? using fear appeals to engender threats and fear that motivate protective security behaviors. mis quarterly, 39(4), 837–864. https://doi.org/10.25300/misq/2015/39.4.5 australasian journal of information systems alhelaly et al. 2024, vol 28, research article mobile identity protection 23 bulgurcu, b., cavusoglu, h., & benbasat, i. (2010). information security policy compliance: an empirical study of rationality-based beliefs and information security awareness. mis quarterly, 34(3), 523–548. https://doi.org/10.2307/25750690 camp, l. j. (2004). digital identity issues. ieee technology and society magazine, 23(3), 34–41. https://doi.org/10.1109/mtas.2004.1337889 carter, m., & grover, v. (2015). me, myself, and i (t) conceptualize information technology identity and its implications. mis quarterly, 39(4), 931–958. https://www.jstor.org/stable/26628658 chen, y., & zahedi, f. m. (2016). individuals’ internet security perceptions and behaviors: polycontextual contrasts between the united states and china. mis quarterly, 40(1), 205– 222. https://doi.org/10.25300/misq/2016/40.1.09 cheung, r. y. m., & ng, m. c. y. (2019). being in the moment later? testing the inverse relation between mindfulness and procrastination. personality and individual differences, 141, 123– 126. https://doi.org/10.1016/j.paid.2018.12.015 chin, w. w. (1998). commentary: issues and opinion on structural equation modeling. mis quarterly, 22(1), vii–xvi. http://www.jstor.org/stable/249674 chin, w. w., thatcher, j. b., & wright, r. t. (2012). assessing common method bias: problems with the ulmc technique. mis quarterly, 36(3), 1003–1019. https://doi.org/10.2307/41703491 choudhary, p. k., routray, s., upadhyay, p., & pani, a. k. (2022). adoption of enterprise mobile systems–an alternative theoretical perspective. international journal of information management, 67, 102539. https://doi.org/10.1016/j.ijinfomgt.2022.102539 churchill jr, g. a. (1979). a paradigm for developing better measures of marketing constructs. journal of marketing research, 16(1), 64–73. https://doi.org/10.1177/002224377901600110 cohen, j. (1992). statistical power analysis. current directions in psychological science, 1(3), 98– 101. https://doi.org/10.1111/1467-8721.ep10768783 compeau, d., correia, j., & bennett thatcher, j. (2022). when constructs become obsolete: a systematic approach to evaluating and updating constructs for information systems research. mis quarterly, 46(2), 679–711. https://doi.org/10.25300/misq/2022/15516 compeau, d. r., & higgins, c. a. (1995). computer self-efficacy: development of a measure and initial test. mis quarterly, 19(2), 189–211. https://doi.org/10.2307/249688 craig, k., thatcher, j. b., & grover, v. (2019). the it identity threat: a conceptual definition and operational measure. journal of management information systems, 36(1), 259–288. https://doi.org/10.1080/07421222.2018.1550561 crossler, r. e., & bélanger, f. (2019). why would i use location-protective settings on my smartphone? motivating protective behaviors and the existence of the privacy knowledge–belief gap. information systems research, 30(3), 995–1006. https://doi.org/10.1287/isre.2019.0846 d’arcy, j., & lowry, p. b. (2019). cognitive-affective drivers of employees’ daily compliance with information security policies: a multilevel, longitudinal study. information systems journal, 29(1), 43–69. https://doi.org/10.1111/isj.12173 australasian journal of information systems alhelaly et al. 2024, vol 28, research article mobile identity protection 24 davis, f. d. (1989). perceived usefulness, perceived ease of use, and user acceptance of information technology. mis quarterly, 13(3), 319–340. https://doi.org/10.2307/249008 davis, f. d., bagozzi, r. p., & warshaw, p. r. (1989). user acceptance of computer technology: a comparison of two theoretical models. management science, 35(8), 982–1003. https://doi.org/10.1287/mnsc.35.8.982 deng, g., & fei, s. (2023). exploring the factors influencing online civic engagement in a smart city: the mediating roles of ict self-efficacy and commitment to community. computers in human behavior, 143, 107682. https://doi.org/10.1016/j.chb.2023.107682 dhillon, g., abdul talib, y. y., & picoto, w. n. (2020). the mediating role of psychological empowerment in information security compliance intentions. journal of the association for information systems, 21(1), 5. https://doi.org/10.17705/1jais.00595 dhillon, g., & chowdhuri, r. (2013). individual values for protecting identity in social networks. international conference on information systems (icis 2013): reshaping society through information systems design, 3, 2177–2192. https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=18dbe95c34ce00ed0 17e1b5689c6291add214589 dinev, t., & hu, q. (2007). the centrality of awareness in the formation of user behavioral intention toward protective information technologies. journal of the association for information systems, 8(7), 23. https://doi.org/10.17705/1jais.00133 eagly, a. h., & chaiken, s. (1998). attitude structure and function. in d. t. gilbert, s. t. fiske, & g. lindzey (eds.), the handbook of social psychology (pp. 269–322). new york, ny, usa: mcgraw-hill. https://psycnet.apa.org/record/1998-07091-007 edwards, j., miles, m. p., d’alessandro, s., & frost, m. (2022). linking b2b sales performance to entrepreneurial self-efficacy, entrepreneurial selling actions. journal of business research, 142, 585–593. https://doi.org/10.1016/j.jbusres.2021.12.074 ferrari, j. r. (1992). psychometric validation of two procrastination inventories for adults: arousal and avoidance measures. journal of psychopathology and behavioral assessment, 14(2), 97–110. https://doi.org/10.1007/bf00965170 fishbein, m., & ajzen, i. (1980). understanding attitudes and predicting social behavior. englewood-cliffs, nj, usa: prentice-hall. fisher jr, e. b., & rost, k. (1986). smoking cessation: a practical guide for the physician. clinics in chest medicine, 7(4), 551–565. https://europepmc.org/article/med/3539471 fornell, c., & bookstein, f. l. (1982). two structural equation models: lisrel and pls applied to consumer exit-voice theory. journal of marketing research, 19(4), 440–452. https://doi.org/10.1177/002224378201900406 fornell, c., & larcker, d. f. (1981). structural equation models with unobservable variables and measurement error: algebra and statistics. journal of marketing research, 18(3), 382– 388. https://doi.org/10.2307/3150980 ftc. (2020). consumer sentinel network. federal trade commission, data book 2019. https://www.ftc.gov/system/files/documents/reports/consumer-sentinel-network-databook-2019/consumer_sentinel_network_data_book_2019.pdf australasian journal of information systems alhelaly et al. 2024, vol 28, research article mobile identity protection 25 gao, w., liu, z., guo, q., & li, x. (2018). the dark side of ubiquitous connectivity in smartphone-based sns: an integrated model from information perspective. computers in human behavior, 84, 185–193. https://doi.org/10.1016/j.chb.2018.02.023 gong, x., zhang, k. z. k., chen, c., cheung, c. m. k., & lee, m. k. o. (2020). what drives trust transfer from web to mobile payment services? the dual effects of perceived entitativity. information & management, 57(7), 103250. https://doi.org/10.1016/j.im.2019.103250 goo, j., kishore, r., rao, h. r., & nam, k. (2009). the role of service level agreements in relational management of information technology outsourcing: an empirical study. mis quarterly, 33(1), 119–145. https://doi.org/10.2307/20650281 graham, s., & weiner, b. (1996). theories and principles of motivation. in d. c. berliner, & r. calfee (eds.), handbook of educational psychology (pp. 63-84). new york, ny, usa: macmillan. https://www.scirp.org/(s(351jmbntvnsjt1aadkposzje))/reference/referencespapers.aspx ?referenceid=1209774 guillon, v., dosnon, o., esteve, m.-d., & gosling, p. (2004). self-efficacy and behavioral intention: a mediational analysis of the effects of commitment on career counseling. european journal of psychology of education, 19(3), 315–332. https://doi.org/10.1007/bf03173226 hair, j. f., ringle, c. m., & sarstedt, m. (2012). partial least squares: the better approach to structural equation modeling? long range planning, 45(5–6), 312–319. https://doi.org/10.1016/j.lrp.2012.09.011 hair, j. f., sarstedt, m., ringle, c. m., & mena, j. a. (2012). an assessment of the use of partial least squares structural equation modeling in marketing research. journal of the academy of marketing science, 40(3), 414–433. https://doi.org/10.1007/s11747-011-0261-6 hair jr, j. f., hult, g. t. m., ringle, c. m., sarstedt, m., danks, n. p., & ray, s. (2021). partial least squares structural equation modeling (pls-sem) using r: a workbook. cham, switzerland: springer nature. https://doi.org/10.1007/978-3-030-80519-7 hair jr, j. f., sarstedt, m., hopkins, l., & kuppelwieser, v. g. (2014). partial least squares structural equation modeling (pls-sem): an emerging tool in business research. european business review, 26(2), 106–121. https://doi.org/10.1108/ebr-10-2013-0128 hair jr, j., hair jr, j. f., hult, g. t. m., ringle, c. m., & sarstedt, m. (2021). a primer on partial least squares structural equation modeling (pls-sem). thousand oaks, ca, usa: sage publications. https://doi.org/10.1007/978-3-030-80519-7 hampel, n., sassenberg, k., scholl, a., & ditrich, l. (2023). enactive mastery experience improves attitudes towards digital technology via self-efficacy–a pre-registered quasiexperiment. behaviour & information technology, 1–14. https://doi.org/10.1080/0144929x.2022.2162436 henseler, j., ringle, c. m., & sarstedt, m. (2015). a new criterion for assessing discriminant validity in variance-based structural equation modeling. journal of the academy of marketing science, 43(1), 115–135. https://doi.org/10.1007/s11747-014-0403-8 australasian journal of information systems alhelaly et al. 2024, vol 28, research article mobile identity protection 26 henseler, j., ringle, c. m., & sinkovics, r. r. (2009). the use of partial least squares path modeling in international marketing. in a. m. rugman (ed.), new challenges to international marketing (pp. 14-33). bingley, uk: emerald group publishing ltd. https://doi.org/10.1108/s1474-7979(2009)0000020014 herath, t., chen, r., wang, j., banjara, k., wilbur, j., & rao, h. r. (2014). security services as coping mechanisms: an investigation into user intention to adopt an email authentication service. information systems journal, 24(1), 61–84. https://doi.org/10.1111/j.1365-2575.2012.00420.x herath, t., & rao, h. r. (2009). protection motivation and deterrence: a framework for security policy compliance in organisations. european journal of information systems, 18(2), 106– 125. https://doi.org/10.1057/ejis.2009.6 huang, g., & ren, y. (2020). linking technological functions of fitness mobile apps with continuance usage among chinese users: moderating role of exercise self-efficacy. computers in human behavior, 103, 151–160. https://doi.org/10.1016/j.chb.2019.09.013 jaeger, l. (2018). information security awareness: literature review and integrative framework. in proceedings of the 51st hawaii international conference on system sciences. the usa. https://doi.org/10.24251/hicss.2018.593 jaeger, l., & eckhardt, a. (2021). eyes wide open: the role of situational information security awareness for security-related behaviour. information systems journal, 31(3), 429–472. https://doi.org/10.1111/isj.12317 johnson, r. e., rosen, c. c., & djurdjevic, e. (2011). assessing the impact of common method variance on higher order multidimensional constructs. journal of applied psychology, 96(4), 744–761. https://doi.org/10.1037/a0021504 johnston, a. c., & warkentin, m. (2010). fear appeals and information security behaviors: an empirical study. mis quarterly, 34(3), 549–566. https://doi.org/10.2307/25750691 johnston, a. c., warkentin, m., mcbride, m., & carter, l. (2016). dispositional and situational factors: influences on information security policy violations. european journal of information systems, 25(3), 231–251. https://doi.org/10.1057/ejis.2015.15 johnston, a., di gangi, p. m., bélanger, f., crossler, r. e., siponen, m., warkentin, m., & singh, t. (2023). seeking rhetorical validity in fear appeal research: an application of rhetorical theory. computers & security, 125, 103020. https://doi.org/10.1016/j.cose.2022.103020 ke, w., liu, h., wei, k. k., gu, j., & chen, h. (2009). how do mediated and non-mediated power affect electronic supply chain management system adoption? the mediating effects of trust and institutional pressures. decision support systems, 46(4), 839–851. https://doi.org/10.1016/j.dss.2008.11.008 keith, m. j., babb, j. s., lowry, p. b., furner, c. p., & abdullat, a. (2015). the role of mobilecomputing self-efficacy in consumer information disclosure. information systems journal, 25(6), 637–667. https://doi.org/10.1111/isj.12082 khan, h., & das, a. (2016). security behaviors of smartphone users. information management & computer security, 24, 116. https://doi.org/10.1108/ics-04-2015-0018 australasian journal of information systems alhelaly et al. 2024, vol 28, research article mobile identity protection 27 khan, n. f., ikram, n., murtaza, h., & javed, m. (2023). evaluating protection motivation based cybersecurity awareness training on kirkpatrick’s model. computers & security, 125, 103049. https://doi.org/10.1016/j.cose.2022.103049 khando, k., gao, s., islam, s. m., & salman, a. (2021). enhancing employees information security awareness in private and public organisations: a systematic literature review. computers & security, 106, 102267. https://doi.org/10.1016/j.cose.2021.102267 kim, h.-w., & kankanhalli, a. (2009). investigating user resistance to information systems implementation: a status quo bias perspective. mis quarterly, 33(3), 567–582. https://doi.org/10.2307/20650309 kirst-ashman, k. k. (2016). empowerment series: introduction to social work & social welfare: critical thinking perspectives. (5th ed.). boston, ma, usa: cengage learning. kraiger, k., ford, j. k., & salas, e. (1993). application of cognitive, skill-based, and affective theories of learning outcomes to new methods of training evaluation. journal of applied psychology, 78(2), 311 –328. https://doi.org/10.1037/0021-9010.78.2.311 lee, g., & xia, w. (2010). toward agile: an integrated analysis of quantitative and qualitative field data on software development agility. mis quarterly, 34(1), 87–114. https://doi.org/10.2307/20721416 lee, y., & larsen, k. r. (2009). threat or coping appraisal: determinants of smb executives’ decision to adopt anti-malware software. european journal of information systems, 18(2), 177–187. https://doi.org/10.1057/ejis.2009.11 liang, h., & xue, y. (2009). avoidance of information technology threats: a theoretical perspective. mis quarterly, 33(1), 71–90. https://doi.org/10.2307/20650279 liang, h., & xue, y. l. (2010). understanding security behaviors in personal computer usage: a threat avoidance perspective. journal of the association for information systems, 11(7), 394–413. https://doi.org/10.17705/1jais.00232 lin, w., xu, m., he, j., & zhang, w. (2023). privacy, security, and resilience in mobile healthcare applications. enterprise information systems, 17(3). https://doi.org/10.1080/17517575.2021.1939896 lindell, m. k., & whitney, d. j. (2001). accounting for common method variance in crosssectional research designs. journal of applied psychology, 86(1), 114–121. https://doi.org/10.1037/0021-9010.86.1.114 liu, j., skoric, m. m., & li, c. (2023). disentangling the relation among trust, efficacy and privacy management: a moderated mediation analysis of public support for government surveillance during the covid-19 pandemic. behaviour & information technology, 1–20. https://doi.org/10.1080/0144929x.2023.2178830 loh, x.-m., lee, v.-h., hew, j.-j., tan, g. w.-h., & ooi, k.-b. (2023). the future is now but is it here to stay? employees’ perspective on working from home. journal of business research, 167, 114190. https://doi.org/10.1016/j.jbusres.2023.114190 maddux, j. e., & rogers, r. w. (1983). protection motivation and self-efficacy: a revised theory of fear appeals and attitude change. journal of experimental social psychology, 19(5), 469– 479. https://doi.org/10.1016/0022-1031(83)90023-9 australasian journal of information systems alhelaly et al. 2024, vol 28, research article mobile identity protection 28 malodia, s., mishra, m., fait, m., papa, a., & dezi, l. (2023). to digit or to head? designing digital transformation journey of smes among digital self-efficacy and professional leadership. journal of business research, 157, 113547. https://doi.org/10.1016/j.jbusres.2022.113547 manke, s., & winkler, i. (2013). the habits of highly successful security awareness programs: a crosscompany comparison. securementem, retrieved april 12, 2016, from https://www.securementem.com/wp-content/uploads/2013/07/habits_white_paper.pdf markelj, b., & bernik, i. (2015). safe use of mobile devices arises from knowing the threats. journal of information security and applications, 20, 84–89. https://doi.org/10.1016/j.jisa.2014.11.001 martens, m., de wolf, r., & de marez, l. (2019). investigating and comparing the predictors of the intention towards taking security measures against malware, scams, and cybercrime in general. computers in human behavior, 92, 139–150. https://doi.org/10.1016/j.chb.2018.11.002 matook, s., maggie wang, y., koeppel, n., & guerin, s. (2023). metacognitive skills in lowcode app development: work-integrated learning in information systems development. journal of information technology, 02683962231170238. https://doi.org/10.1177/02683962231170238 mccall george, j., & simmons, j. l. (1978). identities and interactions: an examination of human associations in everyday life. new york, ny, usa: free press. menard, p., bott, g. j., & crossler, r. e. (2017). user motivations in protecting information security: protection motivation theory versus self-determination theory. journal of management information systems, 34(4), 1203–1230. https://doi.org/10.1080/07421222.2017.1394083 menard, p., warkentin, m., & lowry, p. b. (2018). the impact of collectivism and psychological ownership on protection motivation: a cross-cultural examination. computers & security, 75, 147–166. https://doi.org/10.1016/j.cose.2018.01.020 mombeuil, c. (2023). consumers’ willingness to use mobile payments in micro business transactions: differences in demographic factors. information systems frontiers, 1–14. https://doi.org/10.1007/s10796-023-10421-6 mullins, j., & sabherwal, r. (2022). just enough information? the contingent curvilinear effect of information volume on decision performance in is-enabled teams. mis quarterly, 46(4), 2197–2228. https://doi.org/10.25300/misq/2022/17290 ogbanufe, o., & gerhart, n. (2020). the mediating influence of smartwatch identity on deep use and innovative individual performance. information systems journal, 30(6), 977–1009. https://doi.org/10.1111/isj.12288 ogbanufe, o. m., & baham, c. (2023). using multi-factor authentication for online account security: examining the influence of anticipated regret. information systems frontiers, 25(2), 897–916. https://doi.org/10.1007/s10796-022-10278-1 australasian journal of information systems alhelaly et al. 2024, vol 28, research article mobile identity protection 29 ogbanufe, o., & pavur, r. (2022). going through the emotions of regret and fear: revisiting protection motivation for identity theft protection. international journal of information management, 62, 102432. https://doi.org/10.1016/j.ijinfomgt.2021.102432 o’leary-kelly, a. m., martocchio, j. j., & frink, d. d. (1994). a review of the influence of group goals on group performance. academy of management journal, 37(5), 1285–1301. https://doi.org/10.5465/256673 o’sullivan, t. l., corneil, w., kuziemsky, c. e., & toal-sullivan, d. (2015). use of the structured interview matrix to enhance community resilience through collaboration and inclusive engagement. systems research and behavioral science, 32(6), 616–628. https://doi.org/10.1002/sres.2250 o’sullivan, t., tracey, s., & corneil, w. (2022). structured interview matrix (sim) facilitators’ guide. https://ruor.uottawa.ca/bitstream/10393/44813/1/sim%20facilitator%27s%20guide_o% 27sullivan%20et%20al_%202022.pdf pavlou, & fygenson. (2006). understanding and predicting electronic commerce adoption: an extension of the theory of planned behavior. mis quarterly, 30(1), 115. https://doi.org/10.2307/25148720 podsakoff, p. m., mackenzie, s. b., lee, j.-y., & podsakoff, n. p. (2003). common method biases in behavioral research: a critical review of the literature and recommended remedies. journal of applied psychology, 88(5), 879–903. https://doi.org/10.1037/00219010.88.5.879 podsakoff, p. m., & organ, d. w. (1986). self-reports in organizational research: problems and prospects. journal of management, 12(4), 531–544. https://doi.org/10.1177/014920638601200408 posey, c., roberts, t. l., & lowry, p. b. (2015). the impact of organizational commitment on insiders’ motivation to protect organizational information assets. journal of management information systems, 32(4), 179–214. https://doi.org/10.1080/07421222.2015.1138374 reychav, i., beeri, r., balapour, a., raban, d. r., sabherwal, r., & azuri, j. (2019). how reliable are self-assessments using mobile technology in healthcare? the effects of technology identity and self-efficacy. computers in human behavior, 91, 52–61. https://doi.org/10.1016/j.chb.2018.09.024 rheea, h.-s., kimb, c., & ryuc, y. u. (2009). self-efficacy in information security: its influence on end users’ information security practice behavior. computers & security, 28(8), 816– 826. https://doi.org/10.1016/j.cose.2009.05.008 ringle, c. m., sarstedt, m., & straub, d. w. (2012). editor’s comments: a critical look at the use of pls-sem in" mis quarterly". mis quarterly, 36(1), iii–xiv. https://doi.org/10.2307/41410402 rogers, r. w. (1975). a protection motivation theory of fear appeals and attitude change1. the journal of psychology, 91(1), 93–114. https://doi.org/10.1080/00223980.1975.9915803 rogers, r. w. (1983). cognitive and physiological processes in fear appeals and attitude change: a revised theory of protection motivation. in j. r. cacioppo, & r.e. & petty australasian journal of information systems alhelaly et al. 2024, vol 28, research article mobile identity protection 30 (eds.), social psychology: a sourcebook, pp. 153-176. new york, ny, usa: guilford. https://search.gesis.org/publication/zis-rogers1983cognitive sanyal, p., menon, n., & siponen, m. (2021). an empirical examination of the economics of mobile application security. mis quarterly, 45(4), 2235–2260. https://doi.org/10.25300/misq/2021/15315 schunk, d. h., & pajares, f. (2009). self-efficacy theory. in k. r. wenzel & a. wigfield (eds.), handbook of motivation at school (pp. 35–53). milton park, uk: routledge/taylor & francis group. https://repository.stkipjb.ac.id/index.php/lecturer/article/download/2935/2473#page=48 silic, m., & lowry, p. b. (2020). using design-science based gamification to improve organizational security training and compliance. journal of management information systems, 37(1), 129–161. https://doi.org/10.1080/07421222.2019.1705512 soh, f., smith, k., & dhillon, g. (2022). the relationship between social capital and social media addiction: the role of privacy self-efficacy. australasian journal of information systems, 26. https://doi.org/10.3127/ajis.v26i0.3367 souppaya, m., & scarfone, k. (2013). guidelines for managing the security of mobile devices in the enterprise. national institute of standards and technology. https://doi.org/10.6028/nist.sp.800-124r1 stets, j. e., & cast, a. d. (2007). resources and identity verification from an identity theory perspective. sociological perspectives, 50(4), 517–543. https://doi.org/10.1525/sop.2007.50.4.517 stone, d. n. (1994). overconfidence in initial self-efficacy judgments: effects on decision processes and performance. organizational behavior and human decision processes, 59(3), 452–474. https://doi.org/10.1006/obhd.1994.1069 straub, d., & gefen, d. (2004). validation guidelines for is positivist research. communications of the association for information systems, 13. https://doi.org/10.17705/1cais.01324 straub, d. w. (1989). validating instruments in mis research. mis quarterly, 13(2), 147. https://doi.org/10.2307/248922 stryker, s., & serpe, r. t. (1994). identity salience and psychological centrality: equivalent, overlapping, or complementary concepts? social psychology quarterly, 57(1), 16–35. https://doi.org/10.2307/2786972 tang, x., & wei, s. (2022). how do ambidextrous leadership and self-efficacy influence employees’ enterprise system use: an empirical study of customer relationship management system context. information technology & people, 35(4), 1443–1465. https://doi.org/10.1108/itp-07-2020-0479 taylor, s., & todd, p. a. (1995). understanding information technology usage: a test of competing models. information systems research, 6(2), 144–176. https://doi.org/10.1287/isre.6.2.144 thangavelu, m., krishnaswamy, v., & sharma, m. (2021). impact of comprehensive information security awareness and cognitive characteristics on security incident australasian journal of information systems alhelaly et al. 2024, vol 28, research article mobile identity protection 31 management–an empirical study. computers & security, 109, 102401. https://doi.org/10.1016/j.cose.2021.102401 thompson, n., mcgill, t. j., & wang, x. (2017). “security begins at home”: determinants of home computer and mobile device security behavior. computers & security, 70, 376–391. https://doi.org/https://doi.org/10.1016/j.cose.2017.07.003 tubbs, m. (1994). commitment and the role of ability in motivation: comment on wright, o’leary-kelly, cortina, klein, and hollenbeck (1994). journal of applied psychology, 79(6), 804–811. https://doi.org/10.1037/0021-9010.79.6.804 tuckman, b. w. (1991). the development and concurrent validity of the procrastination scale. educational and psychological measurement, 51(2), 473–480. https://doi.org/10.1177/0013164491512022 vancouver, j. b., thompson, c. m., & williams, a. a. (2001). the changing signs in the relationships among self-efficacy, personal goals, and performance. journal of applied psychology, 86(4), 605–620. https://doi.org/10.1037/0021-9010.86.4.605 vedadi, a., & warkentin, m. (2020). can secure behaviors be contagious? a two-stage investigation of the influence of herd behavior on security decisions. journal of the association for information systems, 21(2), 3. https://doi.org/10.17705/1jais.00607 venkatesh, v., aloysius, j. a., hoehle, h., & burton, s. (2017). design and evaluation of autoid enabled shopping assistance artifacts in customers’ mobile phones. mis quarterly, 41(1), 83–114. https://doi.org/10.25300/misq/2017/41.1.05 venkatesh, v., brown, s. a., & bala, h. (2013). bridging the qualitative-quantitative divide: guidelines for conducting mixed methods research in information systems. mis quarterly, 37(1), 21–54. https://doi.org/10.25300/misq/2013/37.1.02 verkijika, s. f. (2018). understanding smartphone security behaviors: an extension of the protection motivation theory with anticipated regret. computers & security, 77, 860–870. https://doi.org/10.1016/j.cose.2018.03.008 verkijika, s. f. (2019). “if you know what to do, will you take action to avoid mobile phishing attacks”: self-efficacy, anticipated regret, and gender. computers in human behavior, 101, 286–296. https://doi.org/10.1016/j.chb.2019.07.034 wang, j., li, y., & rao, h. r. (2016). overconfidence in phishing email detection. journal of the association for information systems, 17(11), 759–783. https://doi.org/10.17705/1jais.00442 wang, x., li, y., khasraghi, h. j., & trumbach, c. (2023). the mediating role of security anxiety in internet threat avoidance behavior. computers & security, 134, 103429. https://doi.org/10.1016/j.cose.2023.103429 warkentin, m., johnston, a. c., & shropshire, j. (2011). the influence of the informal social learning environment on information privacy policy compliance efficacy and intention. european journal of information systems, 20(3), 267–284. https://doi.org/10.1057/ejis.2010.72 li, w., leung, a., & yue, w. (2023). where is it in information security? the interrelationship among it investment, security awareness, and data breaches. mis quarterly, 47(1), 317– 342. https://doi.org/10.25300/misq/2022/15713 australasian journal of information systems alhelaly et al. 2024, vol 28, research article mobile identity protection 32 wigfield, a., & eccles, j. s. (2000). expectancy–value theory of achievement motivation. contemporary educational psychology, 25(1), 68–81. https://doi.org/10.1006/ceps.1999.1015 willison, r., & warkentin, m. (2013). beyond deterrence: an expanded view of employee computer abuse. mis quarterly, 37(1), 1–20. https://doi.org/10.25300/misq/2013/37.1.01 witte, k. (1992). putting the fear back into fear appeals: the extended parallel process model. communication monographs, 59(4), 329–349. https://doi.org/10.1080/03637759209376276 witte, k. (1996). predicting risk behaviors: development and validation of a diagnostic scale. journal of health communication, 1(4), 317–342. https://doi.org/10.1080/108107396127988 wu, d., moody, g. d., zhang, j., & lowry, p. b. (2020). effects of the design of mobile security notifications and mobile app usability on users’ security perceptions and continued use intention. information & management, 57(5), 103235. https://doi.org/10.1016/j.im.2019.103235 xiao, y., & spanjol, j. (2021). yes, but not now! why some users procrastinate in adopting digital product updates. journal of business research, 135, 685–696. https://doi.org/10.1016/j.jbusres.2021.06.066 xu, z., & guo, k. (2019). it ain’t my business: a coping perspective on employee effortful security behavior. journal of enterprise information management, 32(5), 824–842. https://doi.org/10.1108/jeim-10-2018-0229 yao, m.-l., chuang, m.-c., & hsu, c.-c. (2018). the kano model analysis of features for mobile security applications. computers & security, 78, 336–346. https://doi.org/10.1016/j.cose.2018.07.008 yazdanmehr, a., li, y., & wang, j. (2022). does stress reduce violation intention? insights from eustress and distress processes on employee reaction to information security policies. european journal of information systems, 32(6), 1033–1051. https://doi.org/10.1080/0960085x.2022.2099767 yoo, c. w., goo, j., & rao, h. r. (2020). is cybersecurity a team sport? a multilevel examination of workgroup information security effectiveness. mis quarterly, 44(2), 907–931. https://doi.org/10.25300/misq/2020/15477 zahedi, f., abbasi, a., & chen, y. (2015). fake-website detection tools: identifying elements that promote individuals’ use and enhance their performance. journal of the association for information systems, 16(6), 448–484. https://doi.org/10.17705/1jais.00399 zambrano, f. r. r., & rafael, g. d. r. (2018). bring your own device: a survey of threats and security management models. international journal of electronic business, 14(2), 146–170. https://doi.org/10.1504/ijeb.2018.094862 zhou, q., li, b., scheibenzuber, c., & li, h. (2023). fake news land? exploring the impact of social media affordances on user behavioral responses: a mixed-methods research. computers in human behavior, 148, 107889. https://doi.org/10.1016/j.chb.2023.107889 australasian journal of information systems alhelaly et al. 2024, vol 28, research article mobile identity protection 33 appendix table a1. the self-efficacy conceptualization in the information systems literature author theory self-efficacy antecedents self-efficacy type dependent variable findings yazdanmehr et al. (2023) tmsc; set security related stress (srs) isp-related selfefficacy (ispse) inward emotionfocused coping (iefc); outward emotion-focused coping (oefc); problem focused coping (pfc) ispse mod: srs → iefc. ispse mod: srs → oefc. ispse mod: srs → pfc. liu et al. (2023) set perceived cost (pc); perceived benefits (pb); perceived threat (pt) privacy selfefficacy (pse); political efficacy (pe) privacy trust (pt); public support for government surveillance (psgs) pse med: pc;pb;pt → psgs pe → pt matook et al. (2023) set self-efficacy (se) metacognitive skills (ms) se → ms loh et al. (2023) pmt; tts self-efficacy (se) work productivity (wp); technostress (ts) se → wp se → ts wang et al. (2023) ttat security selfefficacy (sse); response-selfefficacy (rse) avoidance of internet threats (ait) sse → ait rse → ait zhou et al. (2023) fit perceived autonomy (pa) skepticism selfefficacy (sse) fact authentication (fa); counter arguing (ca); instant sharing (is) pa → sse sse → fa sse → ca hampel et al. (2023) sct; set intervention type (it); vicarious experience (ve); enactive mastery (em) robot-specific selfefficacy (rsse) technology enthusiasm (te) it; ve; em → rsse rsse → te rsse med: it; ve; em → te deng & fei (2023) set; slt information content; responsiveness environment quality ict self-efficacy (ictse) online sevic behaviour (osb) ictse → osb malodia et al. (2023) set digital self-efficacy (dse) for sme digital transformation (dt) dse → dt australasian journal of information systems alhelaly et al. 2024, vol 28, research article mobile identity protection 34 yazdanmehr et al. (2022) tmsc; hsm positive responses (pr); negative responses (nr) information security policies self-efficacy (ispse) playful problem solving (pps); wishful thinking (wt) ispse mod: pr →pps ispse mod: nr →wt tang & wei (2022) at ambidextrous leadership performance selfefficacy (pse); creative selfefficacy (cse) exploitative use (eu); explorative use (eu) pse → eu cse → eu edwards et al. (2022) set entrepreneurial self-efficacy (ese) creative selling (cs); sales innovation (si) ese → cs ese → si soh et al. (2022) sct; set social capital bridging (scbr.); social capital bonding (scbo.) privacy selfefficacy (pse) social media addiction (sma) pse mod: scbr. → sma pse mod: scbo. → sma mullins & sabherwal (2022) ipt; st information value (iv) computer selfefficacy (cse) decision performance (dp) cse mod: iv → apm ogbanufe & pavur (2022) pmt perceived efficacy (pe) adaptive protection motivation (apm); maladaptive message rejection (mmr) pe → apm pe → mmr jaeger & eckhardt (2021) pmt situational is awareness perceived coping efficacy (pce) protection motivation (pm) pce → pm silic & lowry (2020) hmsam learning security selfefficacy behavioral intention (bi) to follow security policies sse → bi to follow security policies yoo et al. (2020) sct; dt; ct; tpb; it; pmt security knowledge coordination individual efficacy (ie); workgroup collective efficacy (wse) workgroup information security effectiveness (wise) effectiveness ie → wise effectiveness wse → wise effectiveness huang & ren (2020) sct; set perceived usefulness (pu) exercise selfefficacy (ese) continuance use intention (cui) of fitness mobile apps ese mod: pu → ci balapour et al. (2019) itit, set; tam related it experience (ite) self-efficacy (se) intention to use mobile health apps (iumha) ite → se itsee → iumha reychav et al. (2019) set mobile technology identity (mti) self-efficacy (se) self-report reliability (srr) se → srr se mod: mti → srr australasian journal of information systems alhelaly et al. 2024, vol 28, research article mobile identity protection 35 verkijika (2019) ttat anti-phishing selfefficacy (apse) avoidance motivation (am); avoidance behavior (ab) apse → am apse → ab belanger & crossler (2019) tpb mobile privacy protection selfefficacy (mppse) mobile privacy protection (mpp) intention / behaviour mppse → mppi/mppb d’arcy & lowry (2019) rct, tpb compliance selfefficacy (cse) compliance behavior cse → compliance behavior crossler & bélanger (2019) set; imb model personal motivation, social motivation privacy selfefficacy (pse) individuals’ usage of smartphone privacy settings pse → privacy setting usage wang et al. (2016) sct phishing detection self-efficacy (pdse) overconfidence marginal effect (ome) pdse → ome johnston et al. (2016) pmt; gdt. big five personality traits self-efficacy (se) intention to violate is security policies (ivisp) se→ ivisp posey et al. 2015) pmt self-efficacy (se) protection motivation (pm) se → pm zahedi et al. (2015) pmt detector accuracy, detector speed coping selfefficacy (cse) reliance on the detector (rod) cse → rod keith et al. (2015) sct mobile-computing self-efficacy (mcse) trusting belief (tb), disclosure mcse → tb mcse → disclosure herath et al. (2014) tam; ttat email screening self-efficacy (esse) coping motivation\behavior (cmb) esse → cmb willison & warkentin (2013) st, pmd; tpb information systems (is) controls deterrence efficacy (def) is policy violation intention (ispvi) de → ispvi warkentin et al. (2011) slt; set situational support; verbal persuasion; vicarious experience privacy policy compliance selfefficacy (ppcse) privacy compliance learning behavioral intention (pclbi) ppcse → pclbi liang & xue (2010) ttat self-efficacy (se) avoidance motivation (am) se → am bulgurcu et al. 2010) tpb, pmt, rct, dt self-efficacy to comply (setc) intention to comply (itc) setc → itc johnston & warkentin, (2010) pmt, fam perceived threat & perceived vulnerability self-efficacy (se) behavioral intentions (bi) se → bi herath and rao (2009) gdt, pmt, tpb, dtpb, oc. response efficacy (re) security policy attitude (spa) re→ spa australasian journal of information systems alhelaly et al. 2024, vol 28, research article mobile identity protection 36 lee & larsen (2009) tpb; tam; idt, pmt self-efficacy (se) intention to adopt malware (itam) by sme se → itam by sme liang & xue (2009) ttat self-efficacy (se) perceived avoidability (pa) se → pa rheea et al., (2009) tpb computer experience; security breach incidents; general controllability self-efficacy in information security (seis) information security practice behavior (ispb) seis→ ispb dinev & hu (2007) tpb self-efficacy (se) perceived behavioral control (pbc); behavioral intention (bi) se → pbc se → bi note. tts: transactional theory of stress; fit: feelings-as-information theory; tmsc: the transactional model of stress and coping; hsm: holistic stress model; at: ambidexterity theory ipt: information processing theory; st: schema theory tpb: theory of planned behavior; ttat: technology threat avoidance theory; tam: technology acceptance model; idt: innovation diffusion theory; pmt: protection motivation theory; gdt: general deterrence theory; sct: social cognitive theory; dtpb: decomposed theory of planned behavior; oc: organizational commitment; fam: fear appeal model; rct: rational choice theory; dt: deterrence theory; slt: social learning theory; set: social exchange theory; set: self-efficacy theory; pmd: protection motivation deterrence; imb: information motivation behavioral skills model; ct: communication theory; it: institutional theory; hmsam: hedonic-motivation system adoption model; itit: it identity theory. table a2. the gap analysis of mobile identity protection in information systems literature protective motivation literature a dapted theories protection context se ta pa protection environment authors objective findings inform ation protection privacy protection identity protection d irect im pact m oderation threat severity threat v ulnerability d irect im pact m ediation m obile c om puter n etw ork it ogbanufe & baham (2023) understanding the impact of regret and online security threats response on individuals' usage of multifactor authentication (mfa) the emotion of regret heightens threat appraisals and consequentl y motivates protection and use of mfa pm t australasian journal of information systems alhelaly et al. 2024, vol 28, research article mobile identity protection 37 ogbanufe & pavur (2022) explore the impact of regret and fear on individuals' protection motivation toward the threat of identity theft. while fear impacts individuals only on low threats, regret influences protective behavior in low and high threats of identity theft attacks. pm t ameen et al. (2021) understanding of employees' informationsecurity behavior cybersecurit y policies and cultural differences predict different threats specific to smartphone use. pm t; g d t; tra g ap1 g ap2 g ap3 vedadi & warkentin (2020) investigated how providing popular information can trigger individuals’ behavior and can subsequently influence security behaviors users develop a higher protection motivation when they become aware of the widespread use of a certain security technology ed t bose and leung (2019) exploring the effects of adopting identity theft countermeasur es on firm value adopting identity theft countermea sures increases the short and long-term market of adopting firms. c u t verkijika (2019) investigated the impact of selfefficacy, anticipated regret, and gender on avoidance motivation antiphishing self-efficacy, regret, and gender influence mobile phishing avoidance behavior. tta t australasian journal of information systems alhelaly et al. 2024, vol 28, research article mobile identity protection 38 bélanger & crossler (2019) understanding protective behavior in mobile devices mobile privacy protection self-efficacy affects the intention toward mobile information protection. tpb crossler & bélanger (2019) examining the privacy knowledge– belief gap and the protective behavior in a mobile context. individual privacy skills influence their motivational behavior toward mobile privacyprotective behavior. im bs martens, de wolf, & de marez (2019) investigate motivations to protect oneself against scams, malware, and cybercrime. findings suggest significant differences when protecting oneself against ‘technical’ cybercrimes (malware) compared to more ‘social’ cybercrimes (scams) pm t gao et al. (2018) explore the dark side of ubiquitous connectivity enabled by smartphonebased sns. ubiquitous connectivity increases sns users’ discontinuo us usage intention by raising privacy concerns and protection motivation. pm t; ipt australasian journal of information systems alhelaly et al. 2024, vol 28, research article mobile identity protection 39 craig, thatcher, & grover (2019) developing it identity threat as a new construct to understand employees; resistance behavior to it use it identity protection behavior may develop resistance response to identity threat sources, such as intergroup conflict, verification prevented, and meaning change. id t verkijika (2018) understanding the anticipated regret of not adhering to a security behavior in smartphones. anticipated regret mediating the relationship between mobile security intention and behavior pm t menard, warkentin, & lowry (2018) evaluate individual cultural values to motivate employees to perform secure behaviors. an individual's orientation toward collectivism has an impact on the intention not to perform secure behaviors. pm t thompson, mcgill, & wang (2017) understand the security behavior of mobile users at home both selfefficacy and perceived threats have a positive impact on users’ security behavior. pm t note. se: self-efficacy; ta: threat appraisal; pa: protection awareness; tpb: theory of planned behavior; pmt: protection motivation theory; usp: unified security practices; abm: awareness boundary model; set: self-efficacy theory; imbs: information–motivation–behavioral skills model.; grey boxes indicate a gap in the literature. cut: cue utilization theory; edt: expectation-disconfirmation theory; ttat: technology threat avoidance theory (ttat); ccm: cognition change model; idt: identity theory. ipt: information processing theory; gdt: general deterrence theory; tra: theory of reasoned action australasian journal of information systems alhelaly et al. 2024, vol 28, research article mobile identity protection 40 acknowledgments: the authors would like to thank fundação para a ciência e a tecnologia (fct) regarding the program uidb/04152/2020-centro de investigação em gestão de informação—magic (nova ims). copyright: © 2024 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v28.4397 http://creativecommons.org/licenses/by-nc/3.0/au/ https://doi.org/10.3127/ajis.v28.4397 1 introduction 2 literature and theoretical framing 2.1 mobile identity protection 2.2 identity protection self-efficacy 2.3 threat appraisal and protection motivation 2.4 identity protection awareness 3 conceptual model and hypotheses development 3.1 the role of threat appraisal 3.2 the role of identity protection self-efficacy 3.3 the role of identity protection motivational intentions 3.4 the role of identity protection awareness 4 methods 4.1 measurement 4.2 data collection 5 results of data analysis 5.1 measurement model 5.2 structural model 6 implications 7 conclusions paper title (paper title style) australasian journal of information systems carlson, jones, harris, quezada & frazer 2017, vol 21, research on indigenous icttrauma, shared recognition & indigenous resistance on social media 1 trauma, shared recognition and indigenous resistance on social media bronwyn carlson macquarie university bronwyn.carlson@mq.edu.au lani v. jones university at albany, state university new york michelle harris university at albany, state university new york nelia quezada university at albany, state university new york ryan frazer macquarie university abstract this paper investigates the ways in which indigenous australians respond individually, and collectively, to racial vilification by means of social media sites. introducing the concept of “shared recognition” this paper describes the collective sense of anger and frustration experienced by indigenous people when traumatic events in the public domain act as reminders of ongoing colonialism. three examples are explored to demonstrate collective trauma as a result of racist and discriminatory acts that are made public, and the ways in which social media is utilised by indigenous australians to make sense of and cope with trauma. firstly, the four corners program on abc television entitled ‘australia’s shame’. secondly, a cartoon produced by the editorial cartoonist for the australian newspaper, bill leak depicting indigenous fathers as neglectful. finally, the social media movement, #indigenousdads, that emerged in response to these events and demonstrates ongoing resistance to colonial narratives. drawing on sara ahmed’s (2004) notion of “affective economies” this paper unpacks the politics of pain, shame and pride in the aftermath of both the four corners program and the bill leak cartoon. keywords: shared recognition; social media; indigenous; trauma; resistance; linked fate 1 introduction this paper investigates the ways in which indigenous australians respond individually, and as members of broad and diverse communities, to racial vilification by means of social media sites. we introduce the concept of “shared recognition”—a term we have coined—to describe the collective sense of anger and frustration experienced by indigenous people when traumatic events in the public domain act as reminders of the enduring-ness of colonial violence. it refers to the ways in which we continue to internalise white domination, and also in this instance, to our responses and reactions to it on social media. the term “shared recognition” acquires meaning through the widely held understanding that colonialism is not finished business, and that indeed, its effects continue to permeate the political and social domains in australia in ways that adversely affect the health and well-being of all indigenous subjects. “shared recognition” takes note of the communal acknowledgement that is felt when all forms of racism toward indigenous people are publicly displayed through a variety of social media events, some of which will be discussed here. we build on the concept of “linked fate” as described by michael dawson (1995) who applies this particular usage to black american subjects. dawson (1995) argues that race still has a strong influence on the life chances of african americans and as a result there is a connection between their interests as individuals and the collective fate of the group. “shared recognition” australasian journal of information systems carlson, jones, harris, quezada & frazer 2017, vol 21, research on indigenous icttrauma, shared recognition & indigenous resistance on social media 2 by contrast refers specifically to australian indigenous people’s shared understanding and recognition of race-related discriminatory events, their knowledge of the effects of colonialism, both past and current, and their response to it by way of social media forums. we are interested in how in instances of public iterations of racism, collective trauma and hurt manifest on social media, and how platforms are accessed, deployed, and used as sites of resistance. in order to flesh out the ideas that support and substantiate claims to “shared recognition”, we use the following three examples as case studies: a broadcast of the four corners program on abc television entitled ‘australia’s shame’ aired on july 25th 2016. this program covered the violent mistreatment of indigenous youth at the don dale juvenile justice centre in darwin. secondly, we draw from a cartoon produced by the editorial cartoonist for the australian newspaper, bill leak. this cartoon raised, and has continued to generate considerable public debate about depictions of aboriginal people. finally, we examine the social media movement, #indigenousdads, that emerged in response to both of these events. we contend that these events and reactions may demonstrate that as a result of “shared recognition”, indigenous australians suffer a deep sense of collective trauma when racist and discriminatory acts committed against their group are made public, and that they utilise public media spaces to make sense of and cope with trauma. we draw on sara ahmed’s (2004) notion of “affective economies” to unpack the politics of pain, shame and pride in the aftermath of the screening of this episode of the four corners program. we believe that future empirical studies can establish firm links between traumatic events, social media’s treatment of such events and the implications for emotional and mental states that users experience as a result of interactions with social media, as well as the coping mechanisms they may employ to deal with the effects of online trauma. 2 background explicit images of tear-gassing and the violent abuse of young aboriginal boys being held in the don dale youth detention centre in darwin were screened on the australian broadcasting corporation's, four corners program on july 25th 2016. the program showed a series of leaked images and video footage that revealed that aboriginal boys as young as 13 had been subject to treatment described by aboriginal journalist, jack latimore and colleague clare land as “torture comparable to the abuses committed against inmates of the notorious abu ghraib prison” (2016, np). social media (facebook, instagram and twitter, among others) was flooded with posts about the program and images of the abuse were shared widely, nationally and internationally, in fact, the image of one of the young boys, 17 year old dylan voller, made national and international headlines (see figure 1). the image showed voller handcuffed, hooded and strapped to a mechanical restraint chair. other footage and images showed voller being thrown across his cell, kneed and knocked to the ground, repeatedly stripped naked and kept in solitary confinement for lengthy periods of time. the public was repeatedly exposed to this story and images over the following weeks. figure 1: national post in the vancouver sun, july 27th 2016 australasian journal of information systems carlson, jones, harris, quezada & frazer 2017, vol 21, research on indigenous icttrauma, shared recognition & indigenous resistance on social media 3 the prime minister immediately announced that a royal commission would be established to investigate the treatment of indigenous youth in the juvenile justice system. indigenous lawyer, noel pearson, responded by reminding the prime minister that a royal commission was held 25 years ago into aboriginal deaths in custody with little changing despite a raft of recommendations, and that indigenous people are still over-represented in the criminal and juvenile justice system. pearson commented that more needs to be done to empower indigenous communities and stated that, blackfellas have got to take charge and take responsibility for their own children. that part of the message really struggles to get traction…we're united in relation to the outrage about the end consequences, but we're divided over the question over whether indigenous responsibility is a crucial part of the solution. and i say it is (lateline 2016). shortly after the interview with pearson, the australian published a cartoon by bill leak depicting an aboriginal boy being returned by a police office to his father who is holding a beer can, and asks the police officer, “yeah righto, what’s his name then?”. like the four corners story and images of young aboriginal boys being abused, the cartoon was repeatedly shared on social media accompanied by commentary that was often racist and derogatory about aboriginal parents, and fathers particularly. bill leak’s depiction of aboriginal fathers was defended at the highest levels with the prime minister malcolm turnball and former prime minister john howard stating publicly that the cartoon was not racist. figure 2: pedestrian tv @pedestriandaily, october 27th 2016 in response, aboriginal father and twitter user, joel bayliss tweeted an image of himself and his children with a comment “to counter the bill leak cartoon here is a pic of me & my kids. i am a proud aboriginal father”. the backlash against the cartoon saw the rise of a social media campaign using the hashtag #indigenousdads, where indigenous children and parents posted images documenting how their fathers had raised them and how their families have inspired them. bill leak defended his cartoon claiming his objective was to promote discussion. the press council who are responsible for establishing standards of practice for print and online publications, received over 700 complaints however, decided that the cartoon did not breach press standard that caused “substantial offence, distress or prejudice” (meade 2016a, np). australasian journal of information systems carlson, jones, harris, quezada & frazer 2017, vol 21, research on indigenous icttrauma, shared recognition & indigenous resistance on social media 4 figure 3: tweet by bayliss, j. @jb_bays, 5th august 2016 contrary to this decision many indigenous people expressed “substantial offence” via twitter and other social media platforms. dameyon bonson, founder of black rainbow an advocacy group for indigenous lbgtqi people and communities said he felt “gut punched” when he saw the cartoon. he stated that he “felt crippled by it” and particularly as it was published on national aboriginal and torres strait islander children’s day (davey 2016, p. np). roy ahsee, chairperson of the nsw aboriginal lands council said he was disappointed that australians did not recognise the hurt and humiliation the cartoon had caused (davey 2016, np). journalist allan clarke tweeted that the bill leak cartoon drew on outdated stereotypes and seeing it made him sick. figure 4: tweet by clarke, a. @allanjclark, august 15th 2016 australasian journal of information systems carlson, jones, harris, quezada & frazer 2017, vol 21, research on indigenous icttrauma, shared recognition & indigenous resistance on social media 5 many of the responses revealed a significant level of distress as expressed by gerry moore, ceo of the secretariat of national aboriginal and islander child care (snaicc), who stated that, this cartoon was never going to foster a constructive debate about intergenerational trauma, or issues of alcoholism and family breakdown that are born from it, or the lack of support for frontline services that work to address these issues … personally, as an aboriginal father, the cartoon offended me. it saddened me to see such tired old stereotypes trotted out in a national publication (meade 2016a, np). following joel bayliss, indigenous people took to twitter expressing widespread outrage about the way the media represented indigenous men and families. using the hashtag “#indigenousdads” they posted images celebrating their relationships with their fathers and other significant role models in their families. luke pearson, founder of the rotating twitter account indigenousx stated that the #indigenousdad campaign, was an important demonstration aimed at countering racist depictions and stereotypes, an essential reminder in any ‘national conversation’ that is going to take place. it sets the conversation tone and reminds us and reinforces the importance of our collective strength and humanity (2016, np). #indigenousdads trended on social media demonstrating a “shared recognition” and response from indigenous australians in a show of solidarity countering stereotypical media representation that continues to plagues indigenous people (bogle 2016, np). figure 5: tweet by national indigenous radio @natindigradio 7th august 2016 3 social media and trauma although indigenous australians are enthusiastic social media users (carlson 2016a) there is a paucity of research that examines the individual and collective effects of being subject to ongoing and often consistent racial discrimination and traumatic events on social media (carlson 2016b). online hate and harassment are very real issues for indigenous people and constitute a form of “shared recognition” whereby trauma is understood as a consequence of colonialism and the continued subjugation and vilification of indigenous people. such expressions of hatred are not always confined solely to online environments. in fact, online expressions of intolerance can be correlated with violent actions offline. recently, for example, in kalgoorlie, western australia, online hate preceded the tragic death of a young aboriginal boy. prior to his death several comments were posted on a community site. in one example, a australasian journal of information systems carlson, jones, harris, quezada & frazer 2017, vol 21, research on indigenous icttrauma, shared recognition & indigenous resistance on social media 6 woman posted about aboriginal youth breaking into a vehicle. the comments that followed urged people to hunt them down and run aboriginal youth off the road referring to them as “sub human mutts” (purtill 2016, np). two days after the posts, a 14-year-old aboriginal boy was struck by a utility truck while riding his dirt bike. following his death further posts were made celebrating the death and expressing hope that more aboriginal youth are killed and calling for an “annual cull” (purtill 2016, np). subject to a continual barrage of hate and discrimination, the levels of distress and anguish amongst indigenous australians are understandably high (carlson 2016b). an extensive literature exists that documents a litany of real and perceived discrimination in the lives of other historically oppressed groups and the mental and physical health consequences they suffer as a result of this reality (carter 2007; clark et al 1999; krieger et al 1993; laveist 1996; williams & harris-reid 1999; williams 1996). william smith, tara yosso and daniel solorzano (2011) liken the everyday extreme stress people of colour face to the stress soldiers face in battle. “racial battle fatigue” they argue, refers to the “psychophysiological symptoms resulting from living in mundane extreme racist environment” (2011, p. 302). they suggest that the stress of constantly battling racism can be “mentally, emotionally, and physically draining and/or lethal” (2011, p. 302). a recent report conducted by the charity organisation, mission australia, revealed that indigenous australians suffer psychological distress at twice the rate of non-indigenous people and are much more likely to be admitted to hospital for selfharm (medhora 2016, np). according to feminist researchers jane bailey and sara shayan, online hate and harassment can lead to depression, anxiety and in some cases suicide (2016, pp. 333-334). carlson argues that the rise of social media has meant that indigenous australians are “privy to sadness in unprecedented and unique ways” (2016b, np). injustice, discrimination and violence against indigenous people make regular appearances in our news feed on social media resulting in further trauma. combined with lived experiences of racism, exposure to confronting and violent news content on social media can lead to psychological problems similar to ptsd. according to psychologist monnica williams, a collective trauma can be experienced by groups of people who are subject to such conditions (williams, 2015). while social media has some very tangible positive aspects, we contend that participating on social media—especially exposure to racist or discriminatory content—can be overwhelming and can have serious negative impacts on people’s wellbeing. 3.1 trauma trauma and the experiences of trauma can shatter basic assumptions and beliefs about personal identity and about the social fabric of the world (ainsenburg & ell 2005, p. 217). to understand the ways in which trauma impacts one’s life through a wide range of health, mental health and social problems, we have to understand what it is (the world health organization (who) 2013). trauma has been defined as “a sudden, unexpected, overwhelming intense emotional blow or a series of blows that assaults the person(s) from externally. traumatic experiences are external, but they quickly become incorporated into the mind” (terr 1990, p. 8). van der kolk (2015) makes a point about the complicated nature of trauma arguing that, “traumatization occurs when both internal and external resources are inadequate to cope with external threat” (p.393). as mentioned, it is not the trauma itself that does the damage; rather, it is how the individual’s mind and body reacts in its own unique way to the traumatic experience in combination with the unique response of the individual’s social group. trauma is transmitted through several pathways, which may include psychological, physical, social and cultural conditions (sotero 2006). intergenerational trauma is one such way that transcends each of these pathways and includes a wide range of experiences where people are physically threatened, hurt or violated, or when they witness others in similar situations. intergenerational trauma impacts the entire person or people including the way we think, the way we learn, the way we remember things, the way we feel about ourselves, the way we feel about other people, and the way we make sense of the world are. these conditions are all australasian journal of information systems carlson, jones, harris, quezada & frazer 2017, vol 21, research on indigenous icttrauma, shared recognition & indigenous resistance on social media 7 profoundly altered by intergenerational trauma (bacciagaluppi 2011). intergenerational trauma, also referred to as historical trauma, is described as the complex experience of various populations that have been subjected to tribulation in the form of war, genocide, colonialism, among other distressing incidents (gone 2013, p. 687). intergenerational trauma is also connected to the impact that these experiences have had throughout the lifespan of those feeling violated and in the lives of generations that follow through re-traumatization via television, video, news, and other forms of media (twitter, facebook, and instagram) (denham 2008; carlson et al. 2015). 3.2 trauma and indigenous people: shared recognition intergenerational trauma has been increasingly utilised within community activism, as well as in scholarly work, to discuss the health and overall well-being of indigenous people. sotero (2006) connects trauma and deep emotional and psychological scarring to a public health issue, as growing health disparities are connected to intergenerational trauma and ongoing distress caused by natural disasters, racism, physical and sexual assault, among other issues of social injustice experienced by indigenous populations. sotero (2006) also discusses a correlation between the collective distressing experience of trauma and how it impacts the psychological, emotional and physical health outcomes of those affected. the collective experience of trauma emanates from a “shared recognition” of the continuity of colonial practices. this manifests often from the deep seated internalising of inferiority that comes with the repeated onslaught of racism; indigenous people, whether politically motivated or not in the ‘white’ sense of that term, have a deep and abiding knowledge of how we have been, and continue to be perceived, as evidenced by the previous examples, which are but a smattering of random events used here to demonstrate recent articulations of racism; there are, unfortunately many more. “shared recognition” ‘kicks in’ immediately as we ‘see’ ourselves depicted, mis-represented and vilified in the public domain (sotero 2006). like all other ethnic and racial groups in society, we have vastly differing views on many things, but we do hold a “shared recognition” of the historical and continuing application of colonial rule and its perpetually violent effects on our communities. among indigenous peoples, intergenerational trauma, as an effect of “shared recognition” is often linked to the innumerable deaths of many peoples through violence, mass genocide, and the spread of disease (kirmayer et al., 2014). intergenerational trauma is also related to the impact of loss in connection to the loss many indigenous people endure of their homes, lands and for some generations their culture (dudgeon & kelly, 2014; ranzijn, mcconnochie & nolan, 2009). health disparities contribute to the elevated mortality and morbidity rates of indigenous subjects, adding to the traumatic experience and in many cases, the unresolved psychological impacts resulting from ongoing trauma (brave heart et al., 2011). brave heart et al. (2011), discuss the collaborative reaction that a group of peoples may experience as a result of trauma as historical trauma response (htr). one aspect of this response is historical unresolved grief, which brave heart describes as the “profound unsettled bereavement resulting from the cumulative devastating losses” (2011, p. 283). entire communities are affected, resulting in responses such as social malaise, dysfunctional family and social structures and high rates of suicide (carlson & frazer, 2015). bombay, matheson and anisman (2009) have suggested that chronic exposure to trauma can manifest in individual symptoms such as anxiety, depression, grief, addiction, self-harm and suicide. experts have suggested that indigenous populations experience higher levels of stressors associated with post traumatic stress disorder (ptsd) (evans-campbell 2008). however, according to bryant-davis (2007) a diagnosis of ptsd is too narrow for the intergenerational trauma experienced by indigenous people. bryant-davis suggests complex post traumatic stress disorder (cptsd) is more accurate. menzies (2010) likewise argues that ptsd diagnoses ignore the role of culture and intergenerational trauma and fail to give insight into the relationship between historical and contemporary trauma, between policy and practice and the on-going effects of colonial rule. australasian journal of information systems carlson, jones, harris, quezada & frazer 2017, vol 21, research on indigenous icttrauma, shared recognition & indigenous resistance on social media 8 3.3 theorising collective trauma and linked fate through “shared recognition” while intergenerational trauma speaks to how trauma is experienced across generations, collective or cultural trauma refers to the social and political process through which a group— a collective—comes to see themselves as traumatised by both historical and current events, and how collective identities attempt to understand and represent this trauma, and ultimately, to seek a remedy or compensation for the trauma (alexander et al., 2004). the key to cultural trauma is the social process through which members of the collective begin to construct events as traumatic: an active process of meaning making in which multiple agents such as social commentators, academics, community leaders, and spokespersons, for example, begin to filter particular events or a single event through the lens of trauma, therefore creating a shared understanding within the group (alexander et al., 2004). the process includes several steps including identifying the cause of the trauma, who suffered harm, and who/what perpetrated the harm, as well as ascertaining the extent of the harm. in order to share a sense of collective trauma through “shared recognition”, individuals must first share a sense of belonging to a group. this group consciousness is defined as the politicised in-group identification based on one’s identity as a member of a racial/ethnic minority group that is disadvantaged and that this status can only be overcome through collective action (gurin, miller & gurin 1980; jackman & jackman 1973; miller et al., 1981; garcia 2003). the concept of “shared recognition” emphasises this point: because of the centrality of racial stratification in many industrialised countries with heterogeneous populations, individual people of colour often perceive that their fates are tied to those of racial/ethnic group to which they belong. in the united states, where much of the literature on “linked fate” has focused on the experiences of african americans and other black ethnic populations, studies report a common consciousness amongst blacks because of skin colour, experiences with being a victim of discrimination, and a sense that their group is disadvantaged and therefore they too, are likely to be disadvantaged (wright-austin, middleton & yon 2012). evidently, factors such as racial discrimination have implications for the formation of a pan-ethnic identity—meaning that even if a traumatic event happens in the african american community, latinos and black caribbean americans are likely to also feel traumatised by the event because of group consciousness to a superordinate identity; blackness or ‘minority status’ in such instances are understood as shared modes of oppression, despite obvious differences in histories and cultural status. studies have shown this sense of linked fate to exist even as groups may state preference for being acknowledged by their sub-group identities (crisp, stone, & hall, 2006; dovidio, gaertner, & saguy, 2009; hornsey & hogg 2000; masuoka 2006). according to hurwitz, peffley, and mondak (2015) linked fate may serve to “heighten sensitivity of problems associated with disadvantage, not just problems associated with the group to which one feels linked” (p. 516). many scholars also claim that in-group consciousness leads to increased political participation (sanchez 2006; sanchez & vargas 2016), and support for coalitions with other racial/ethnic groups (kaufman 2003; sanchez 2008; uhlaner 1991). therefore, even as one sees differences between one’s own group victimisation and another’s, if similarities in victimisation or disadvantage exist, groups can feel a sense of linked fate to the group to which one does not belong thereby invoking a kind of group-based victim consciousness and more importantly, a sense of collective traumatisation (volhardt 2009; volhardt 2010; volhardt 2012). this form of consciousness, embedded in the knowledge of a broad based oppression to a range of minority groups, can have both positive and negative consequences. indeed, collective recognition of racism and oppression can engender a feeling of solidarity, but also, importantly, it negates the specificity of histories that individual groups experience thereby obfuscating an understanding of the inherent causes and effects of race hatred, their particularities and historical precedents, and the manifold ways racism plays out in the public domain for specific groups and individuals. solidarity in this context, while providing a sense australasian journal of information systems carlson, jones, harris, quezada & frazer 2017, vol 21, research on indigenous icttrauma, shared recognition & indigenous resistance on social media 9 of commonality among many racially vilified groups, serves the interests of dominant forces precisely as it obscures racialised histories and lumps together all “minorities”. although a dearth of similar studies involving indigenous groups inside or outside the us context exist, one can surmise that similar processes of group consciousness are at play amongst colonised peoples, and all socially disadvantaged groups. recent worldwide outpouring of support from blacks throughout the diaspora and by other groups who understand themselves to occupy similar positions in their own social contexts as blacks occupy in the us—groups like indigenous australians—for the #blacklivesmatter social media driven campaign, illustrate the group-based victim consciousness and activists throughout the world not only joined in solidarity against racial discrimination and violence, but also saw themselves as experiencing variations of the same struggle. therefore, indigenous australians marching in australia’s largest cities holding #blacklivesmatter banners was not only an act of support and allegiance, but also an indicator of “shared recognition”. in this context, solidarity can be a positive and affirming consequence of “shared recognition”, but as suggested, individual groups need to be wary and develop a sound consciousness of the particular historical causes and consequences of their oppression. 4 the affective economies of collective trauma we draw on sara ahmed’s (2004) notion of “affective economies” to unpack the politics of pain, shame and pride. ahmed rejects both the “inside–out” and “outside–in” models of emotions. in the first instance, “psychological” accounts of emotions, such as emotional pain or distress, reduce emotions to internal bodily experiences. they posit emotions as that which is purely private and personal; emotions are fully contained within an individual’s body, which may then be expressed outwards (hence, the “inside–out” model). on the other hand, the second “sociological” account posits emotions as social but impersonal. individuals are seen as passive victims of emotions that are generated outside of the body. emotions “do not reside in subjects or objects,” ahmed writes. rather, she challenges the very distinction between a pre-existing “inside” and “outside”, instead arguing, “emotions create the very effect of the surfaces and boundaries that allow us to distinguish an inside and an outside in the first place” (2004 p.10). it is through the circulation of particular emotions that the surfaces, skins and borders of both individual and collective bodies are intensified. in short, bodies themselves (and therefore their “insides” and “outsides”) are an effect of the circulation of emotions. ahmed talks about this social circulation of emotions in terms of “affective economies” (2004, p. 10). she shows how pain and pride, for instance, through their circulation, are entangled in the very achievement of bodies—both individual and collective. it is through the circulation of emotions such as pain, shame and pride that bodies come into being, what she calls “surfacing”. following ahmed, then, and in the context of this paper, we ask not “what are emotions?” but “what do emotions do?” by framing the question in this way, we look towards the politics of emotions, rather than the experience (i.e. the ‘feelingness’) of emotions. because they are implicated in the production of bodies, emotions are always already political. we raise the question, how do the emotions of pain, shame and pride “work to shape the ‘surfaces’ of individual and collective bodies” (ahmed 2004, p.10) in the context of indigenous people responding to racial vilification. we, too, reject both the “inside–out” and “outside–in” models of emotions. rather than reducing collective trauma to internal bodily processes, whereby the body registers physical or psychological damage which the individual may then express as “pain”; and rather than seeing collective trauma as pre-existing/historical pain that is passively received by subjects, we understand collective trauma as implicated in the very production of political, politicised bodies. in this way, we will track the “affective economies” of twitter in the wake of the don dale detention centre report. australasian journal of information systems carlson, jones, harris, quezada & frazer 2017, vol 21, research on indigenous icttrauma, shared recognition & indigenous resistance on social media 10 5 the circulation of pain in this section, we ask: what does the online expression of collective trauma do? the four corners report generated a huge public response. the images—of boys hooded, shirtless, tied to chairs alongside stories of them being tear gassed while playing cards and of being left in isolation for weeks on end—provoked an immediate debate about juvenile detention in australia. as stated, the harrowing accounts, videos and images were circulated widely on social media in the days and weeks following the airing of the episode producing a wide range of responses and commentary. through the tweets that followed the don dale episode, we witnessed the mass expression of pain by indigenous australians. by conceptualising this as the circulation of an event that produces, collective trauma, we can see that indigenous people are able to recognise the continuity of colonial power structures contained within these events, constituting a “shared recognition” of the specificities of our own colonial history and continued subjugation. figure 6: finlay, s. m. @ontopic, october 1st 2016 figure 7: brennan, b. @bridgeyb, july 25th 2016 figure 8: goat, s. @senor_goat, july 25th 2016 this case is an example of collective trauma and not simply an isolated incident of racial vilification based its political and historical precedents. this includes its links to the discourses produced to justify the removal of children. i.e. the stolen generations, indigenous incarceration, and the aboriginal deaths in custody government inquiry and its subsequent report, represent the longstanding colonial tactic of holding indigenous people accountable for the violent effects of colonialism. australasian journal of information systems carlson, jones, harris, quezada & frazer 2017, vol 21, research on indigenous icttrauma, shared recognition & indigenous resistance on social media 11 figure 9: clarke, a. @allanjclarke, july 25th 2016 6 pain: the re/production of collective bodies the don dale episode made visible the colonial mechanisms of indigenous incarceration, indigenous peoples’ over-representation in prison, and also, the trivialisation of attitudes towards indigenous life and culture, despite the gesture of a national inquiry. indigenous people already know, following the previous national inquiry into aboriginal deaths in custody that a national inquiry into our welfare is meaningless insofar as any expectations of governmental action. for indigenous people, the story of dylan voller and other child prisoners was about more than state violence against children in detention. this was about state violence specifically aimed at indigenous children. indigenous children make up more than half of the children currently in juvenile detention (soldani 2016). such statistics clearly reflect the continuity of colonial practices. we argue that such practices constitute for many of us a reactivation of what was the attempted genocide of our people during the early colonial era. through a “shared recognition” of the continuity of colonial violence documented in the episode, indigenous people express their pain vociferously on twitter; there was an immediate response that can be tracked which demonstrates acute anguish, extreme sadness, and in many cases, rage by indigenous social media users. these various manifestations of pain are both visceral and palpable. figure 11: bungajee, s. @bunbajee, 4th august 2016 ahmed argues it is often through pain that social collectives are formed and, that simultaneously, pain becomes a vehicle that identifies both victims and survivors of colonial histories. such histories are inscribed on the bodies of colonised subjects. ahmed notes this is “bound up with how we inhabit the world, how we live in relationship to the surfaces, bodies and objects that make up our dwelling places” (2004, p.27). more specifically, ahmed points out that, “[p]ain involves the violation or transgression of the border between inside and outside, and it is through this transgression that i feel the border in the first place” (2004, p.27). thus, we contend that collective trauma is implicated in the production of political collectives. this pain creates an “inside” and “outside” of an indigenous body politic that is situated in a subaltern position to the colonial state. in this case, the circulation of pain intensifies the border between a white colonial state and its indigenous colonial subjects. it is australasian journal of information systems carlson, jones, harris, quezada & frazer 2017, vol 21, research on indigenous icttrauma, shared recognition & indigenous resistance on social media 12 through pain that socio-political structures are realised: colonised nations become nation states through the maintenance of divisions between white colonisers and ‘others’ that function to sustain colonial power relations. 7 shame and strategies of containment colonial power, although omnipresent, is subject to fragility, however. the images of children being subject to violent carceral practices in the don dale detention facility and the expressed pain circulated online afterwards, threaten to destabilise the dominant narrative of the benevolent nation state that is reproduced by the prime minister’s reaction and call for a “national inquiry”. to borrow from ahmed, in relation to the report into the stolen generations of indigenous australians: the pain represented the “broken skin of the nation” (2004, p.35). an injustice is being committed, children being abused, the situation demands resolution through the laying of blame, invariably returned to the victims of violence. ahmed argues that pain is implicated in the production of shame and that shame works to expose a failure to fulfil an ideal such as a “benevolent state” or “compassionate nation”. in doing so, shame deprives one the ability to feel pride: “what makes […] injustice unjust is that it has taken pride away; it has deprived white australia of its ability to declare its pride in itself to others,” ahmed writes (2004, p.112; italics in original). as national pride is central to any idea of a coherent nation-state, it must be reclaimed. this can only happen if shame is ‘shifted’ from white australia to indigenous australia. and indigenous people know from past experience that any gesture towards a national inquiry is likely to be lip-service based on a transient feeling of shame that will be replaced by efforts to reconstitute national pride. figure 12: williams, m. @indigenousx, 7th august 2016 in the first move prime minister turnbull announced a royal commission into youth justice in the northern territory the morning after the episode aired—the swiftness of action by the state was unprecedented. through this move, the federal government maintained legitimacy by demonstrating a firm resolve to find justice—and also made it clear that if fault was found anywhere, it would be with the territory government, not the federal government. this is an important political manoeuvre because it shifts the onus of responsibility from the broad body politic to the domain of the northern territory who can then be found responsible and ‘brought into line’. while turnbull recognised the seriousness of the injury, and displayed appropriate horror at the four corners report, his silence since this occurred, and his shifting of the onus of responsibility to the territory government represent a deflection of responsibility on a national level and an acknowledgement of the effects that such brutality will have on the nation state as a whole. australasian journal of information systems carlson, jones, harris, quezada & frazer 2017, vol 21, research on indigenous icttrauma, shared recognition & indigenous resistance on social media 13 8 the leak cartoon soon after the airing of the episode, the australian published a cartoon by bill leak depicting an aboriginal boy being returned by a police office to his father, who is holding a beer can and asks the police officer, “yeah righto, what’s his name then?”. the obvious implication was that incarceration of indigenous youth—such as those in don dale—is a direct consequence of neglectful, alcoholic parents. the leak cartoon, and the colonial discourse it reproduced, thus worked to contain the political potential of national outrage by redirecting its source back onto indigenous people. the reading of this image by many indigenous people was that if indigenous people were hurt and traumatised by the images of the don dale detention centre, it was their own fault. while what happened to dylan voller, and many other indigenous children, is abhorrent, the image worked discursively to reinforce the notion that the fault rests in the hands of indigenous people: parents, communities and their sympathisers. in this way, and following ahmed, the location of shame, returned to indigenous people, seeks to mollify the pain felt by indigenous people by replacing it with feelings of shame. 9 resisting the colonial narrative through the circulation of pride framing the online response in terms of the “politics of trauma” (instead of simply the experience of trauma) we are able to make positive links from this what is known as the #indigenousdads campaign. the online circulation of pain produced a politicised collective that, as we will now discuss, can then coordinate a response that troubles leak’s colonial discourse. leak’s image reduces all indigenous people to incapable and uncaring parents, unable or unwilling to acquiesce to the dictates of white parenting practices. the image is thus a reproduction of the discourse that produced the stolen generations and continues to see indigenous kids being removed, incarcerated, or violated for being indigenous. defending his cartoon, leak stated that, “if you think things are pretty crook for children locked up in…don dale youth detention centre, you should have a look at the homes they come from” (meade 2016b, np). the fact that leak and his supporters attempted to refute any intentional harm is evidence of the enduringness of this discourse. notwithstanding leak’s protestations regarding his intent that the cartoon was to generate much needed national discussion, indigenous social media users, through a “shared recognition” of the colonial discourse being deployed in the image, recognised the colonial discourses that attempted to deflect blame for indigenous disadvantage. through twitter, users were provided a platform to contest the placement of shame on indigenous people. although the democratic potential of social media is heavily contested, it is clear that social media platforms are often ‘seized’ by users as sites for oppositional politics and new forms of political engagement. the backlash against the cartoon saw the rise of a social media campaign using the hashtag #indigenousdads, where indigenous children and parents posted images documenting how their fathers had raised them and how their families have inspired them. these images and stories stood in direct contrast to the colonial stereotype of the neglectful indigenous parent while also promoting culturally specific parenting practices that involve wider familial networks. in this way, the #indigenousdads campaign worked to undermine leak’s totalising colonial discourse and to provide a platform for indigenous dads, grandparents, uncles, family friends and elders who often collectively share the responsibilities of raising and nurturing children and ensuring their safety. australasian journal of information systems carlson, jones, harris, quezada & frazer 2017, vol 21, research on indigenous icttrauma, shared recognition & indigenous resistance on social media 14 figure 13: aboriginal health, @nachoaustralia 10 shared recognition our research suggests a commonality of responses by indigenous respondents to social events that indicate the extent of knowing that comes from “shared recognition”. indigenous people in australia are far better informed, as a consequence of social media (carlson 2016c) about the historical events surrounding colonialism, but also, about the current effects, the intergenerational trauma, and the deep seated depression, anxiety and general ill health that comes from public perceptions and on-going vilification. our access to, and increasing use of social media as a viable means of response to “shared recognition” is evidence of our understanding of the codes of racial vilification, but also, our resistance to it. in addition, social media provides a cultural framework where indigenous people can speak to, and about, the various familial and socio-cultural formations that constitute the broad spectrum of indigenous life in australia. 11 conclusion this paper is a starting point for continued discussion regarding the utility of social media sites for indigenous people in australia. we have demonstrated that the effects of racism in the public sphere, often normalised or marginalised, have a considerable, detrimental and ongoing impact on us. a “shared recognition” of the impact of colonisation and its continuity in the public sphere enables us to understand and respond to the collective responses to public utterances of racism, and also, to their often disastrous effects. we are aware that having a “shared recognition” of the historical precedents of racism does not imply we all react in the same way, but that indigenous people who use social media to respond to racism do so as agents of resistance with the knowledge that their opposition will often bring about further articulations of racism. there is much to be done, we feel, in the area of understanding the use of social media as a political tool for dissent. the case studies we offer here testify that social media sites can be powerful vehicles of opposition for indigenous politics, and can form a basis for discursive shifts, and potential change. australasian journal of information systems carlson, jones, harris, quezada & frazer 2017, vol 21, research on indigenous icttrauma, shared recognition & indigenous resistance on social media 15 references aisenerg, e. & ell, k. (2005) “contextualizing community: violence and its effects an ecological model of parent-child interdependent coping”, journal of interpersonal violence, 20(7): 855-871. ahmed, s. (2004) the cultural politics of emotion, edinburgh university press, edinburgh. alexander, j., eyerman, r., giesen, b., smelser, n. j., & sztompka, p. (2004) cultural trauma and collective identity, university of california press, berkley, california. bacciagaluppi, m. (2011) “the study of psychic trauma”, journal of the american academy of psychoanalysis and dynamic psychiatry, 39(3):525-538. bailey, j. & shayan, s. (2016) “missing and murdered indigenous women crisis: technological dimensions”, canadian journal of women and the law, 28(2): 321-341. bogle, a. (2016) #indigenousdads trends as australia rejects ‘racist’ cartoon, mashable available at, http://mashable.com/2016/08/06/indigenous-dads-trendstwitter/#16b6y4ueh5qf [accessed 10 december 2016]. bombay, a., matheson, k., & anisman, h. (2009) “intergenerational trauma: convergence of multiple processes among first nations people in canada”, journal of aboriginal health, 5:1-42. brave heart, m. y., chase, j., elkins, j., & altschul, d. b. (2011) “historical trauma among indigenous peoples of the americas: concepts, research, and clinical considerations”, journal of psychoactive drugs, 43(4):282-290. bryant-davis, t. (2007) “healing requires recognition: the case for race-based traumatic stress”, the counseling psychologist, 35:135. carlson, b. (2016a) 12 deadly indigenous social media users to follow, the conversation, available at: https://theconversation.com/12-deadly-indigenous-australian-socialmedia-users-to-follow-66479 [accessed 12 january 2017]. carlson, b. (2016b) for aboriginal and torres strait islander people, what are the health impacts of social media, croakey, available at: https://croakey.org/for-aboriginal-andtorres-strait-islander-people-what-are-the-health-impacts-of-social-media/ [accessed 12 january 2017]. carlson, b. (2016c) “aboriginal identity, community and social media” in carlson, b. the politics of identity: who counts as aboriginal today?. aboriginal studies press, canberra. carlson, b. & frazer, r. (2015) “’it’s like going to a cemetery and lighting a candle’: aboriginal australians, sorry business and social media”, alternative: an international journal of indigenous peoples, 11(3):211-224. carlson, b., farrelly, t., frazer, r., borthwick, f. (2015) “mediating tragedy: facebook, aboriginal people and suicide”, australian journal of information systems, 29:1-15. carter, r.t. (2007) racism and psychological and emotional injury: recognizing and assessing race-based traumatic stress. counseling psychologist, 35. 13-105. clark, r., n.b anderson, v.r. clark, & d.r williams. (1999) racism as a stressor for african americans: a biopsychosocial model. american psychologist, 54, 805-816. stone, c. h., & hall, n. r. (2006) “recategorization and subgroup identification: predicting and preventing threats from common ingroups”, personality and social psychology bulletin, 32:230-243. dawson, m. (1995) behind the mule: race and class in african-american politics. princeton university press, princeton. australasian journal of information systems carlson, jones, harris, quezada & frazer 2017, vol 21, research on indigenous icttrauma, shared recognition & indigenous resistance on social media 16 davey, m. (2016) #indigenousdads and their children take to twitter over bill leak cartoon, the guardian. available at, https://www.theguardian.com/australianews/2016/aug/07/indigenousdads-and-their-children-take-to-twitter-over-bill-leakcartoon [accessed 11 january 2017]. denham, a. r. (2008) “rethinking historical trauma: narratives of resilience”, transcultural psychiatry, 45(3):391-414. dovidio, j., gaertner, s., & saguy, t. (2009) “commonality and the complexity of “we”: social attitudes and social change”, personality and social psychology review, 13:3-20. dudgeon, p., & kelly, k. (2014) “contextual factors for research on psychological therapies for aboriginal australians”, australian psychologist, 49(1):8-13. evans-campbell, t. (2008) “historical trauma in american indian/ native alaska communities: a multilevel framework for exploring impacts on individuals, families and communities”, journal of interpersonal violence, 23(3):316-338. garcia, j. a. (2003) latino politics in america: community, culture, and interests, rowman and littlefield, lanham, md. gurin, p., miller, a. h., & gurin, g. (1980) “stratum identification and consciousness”, social psychology quarterly, 43:30-47. hornsey, m. j., & hogg, m. a. (2000) “intergroup similarity and subgroup relations: some implications for assimilation”, personality and social psychology bulletin, 26:948-958. hurwitz, j., peffley, m., & mondak, j. (2015) “linked fate and outgroup perceptions: blacks, latinos, and the us criminal justice system”, political research quarterly, 68:505520. jackman, m. r., & jackman, r. w. (1973) “an interpretation of the relation between objective and subjective social status”, american sociological review, 38:569-582. kaufman, k. m., (2003) “cracks in the rainbow: group commonality as a basis for latino and african-american political coalitions”, political research quarterly, 56(2):199-210. kirmayer, l. j., gone, j. p., & moses, j. (2014) “rethinking historical trauma”, transcultural psychiatry, 51(3):299-319. krieger, n., d.l. rowley, a.a. herman, b. avery & m.t. phillips. (1993) racism, sexism, and social class: implications for studies of health, disease, and well-being. american journal of preventative medicine, 9 (6) 82-122. lateline (2016) noel pearson: royal commission won’t fix the problems facing indigenous australia, lateline. available at, http://www.abc.net.au/news/2016-08-02/royalcommission-noel-pearson-says-won't-fix-problems/7683602 [accessed 15 january 2017]. latimore, j. & land, c. (2016) why another royal commission when the recommendations of the last one continue to gather dust?, indigenousx. available at, http://indigenousx.com.au/why-another-royal-commission/#.wh1pabz95um [accessed 18 december 2016]. masuoka, n. (2006) “together they become one: examining the factors of panethnic group consciousness among asian americans and latinos” social science quarterly, 87(5):993-1011. meade, a. (2016a) press council declines to sanction the australian for bill leak cartoon, the guardian. available at, https://www.theguardian.com/media/2016/sep/06/presscouncil-declines-to-sanction-the-australian-for-bill-leak-cartoon [accessed 18 december 2017]. australasian journal of information systems carlson, jones, harris, quezada & frazer 2017, vol 21, research on indigenous icttrauma, shared recognition & indigenous resistance on social media 17 meade, a. (2016b) bill leak defends ‘racist’ cartoon amid widespread criticism, the guardian. available at, https://www.theguardian.com/media/2016/aug/05/bill-leak-defendsracist-cartoon-amid-widespread-criticism?cmp=share_btn_twl [accessed 18 december 2016]. medhora, s. (2016) “zero out of ten”: indigenous youth rate low levels of happiness, abc. available at, http://www.abc.net.au/triplej/programs/hack/indigenous-youth-ratetheir-happiness-low/7820300 [accessed 18 december 2016]. menzies, p. (2010) “intergenerational trauma from a mental health perspective”, native social work journal, 7:68-85. miller, a. h., gurin, p., gurin, g., & malanchuk, o. (1981) “group consciousness and political participation”, american journal of political science, 25(3):494-511. pearson, l. (2016) #indigenousdads – combating stereotypes and reclaiming the conversation, indigenousx. available at, http://indigenousx.com.au/indigenousdadscombating-stereotypes-and-reclaiming-the-conversation/ [accessed 2 january 2017]. purtill, j. (2016) racist. violent. deleted: the facebook posts dividing kalgoorlie, abc. available at, http://www.abc.net.au/triplej/programs/hack/the-facebook-postsdividing-kalgoorlie/7805346 [accessed 19 december 2016]. ranzijn, r., mcconnochie, k. r., & nolan, w. (2009) psychology and indigenous australians: foundations of cultural competence. palgrave macmillan, australia. sanchez, g. r. (2006) “the role of group consciousness in latino public opinion”, political research quarterly, 59(3):435-446. sanchez, g. r. (2008) “latino group consciousness and perceptions of commonality with african americans”, social science quarterly, 89(2):428-444. sanchez, g. r. & vargas, e. d. (2016) “taking a closer look at group identity: the link between theory and measurement of group consciousness and linked fate”, political research quarterly, 69(1):160-174. smith, w., yosso, t., & solorzano, d. (2011) “challenging racial battle fatigue on historically white campuses: a critical race examination of race related stress” in christine stanley (ed.) faculty of color teaching in predominantly white colleges and universities, anker publishing company, massachusetts, pp. 299-327. soldani, b. (2016) the facts about indigenous youth detention in australia, sbs. available at, http://www.sbs.com.au/news/thefeed/article/2016/07/26/facts-about-indigenousyouth-detention-australia [accessed 12 january 2017]. sotero, m. m. (2006) “a conceptual model of historical trauma: implications for public health practice and research”, journal of health disparities research and practice, 1(1):93-108. terr, l. (1990) too scared to cry: psychic trauma in childhood, harper & row: new york. the world health organization (who), (2013) who releases guidance on mental health care after trauma, 6 august, geneva, available at, http://www.who.int/mediacentre/news/releases/2013/trauma_mental_health_20130 806/en/ [accessed 12 january 2017]. uhlaner, c. j. (1991) “perceived discrimination and prejudice and the coalition prospects of blacks, latinos, and asian americans” in bryan o. jackson and michael b. preston, (eds.) racial and ethnic politics in california, institute for governmental studies press: berkeley, california. van der kol, b. (2015) the body keeps the score: brain, mind, and body in the healing of trauma, viking publishing: new york. australasian journal of information systems carlson, jones, harris, quezada & frazer 2017, vol 21, research on indigenous icttrauma, shared recognition & indigenous resistance on social media 18 vollhardt, j. r. (2009) “the role of victim beliefs in the israelipalestinian conflict: risk or potential for peace?”, peace and conflict: journal of peace psychology, 15:135–159. vollhardt, j. r. (2010) “enhanced external and culturally sensitive attributions after extended intercultural contact”, british journal of social psychology, 49:363-383. vollhardt, j. r. (2012) “collective victimization”. oxford handbook of intergroup conflict, 136157. williams, d.r. (1996) racism and health: a research agenda. ethnicity and disease. 6 (1-2) 1-6. williams, d.r. & m. harris-reid. (1999) race and mental health: emerging patterns and promising approaches. in a. v. horowitz and t. l. scheid (eds.), the sociology of mental illness. new york: cambridge university press. williams, m. (2015) “the links between racism and ptsd”, psychology today. available at, https://www.psychologytoday.com/blog/culturally-speaking/201509/the-linkbetween-racism-and-ptsd [accessed 18 december 2016]. wright-austin, s., middleton, r., & von, r. (2012) “the effect of racial group consciousness on the political participation of african-americans and black pan-ethnics in miami-dade county, florida”, political research quarterly, 63:629-641. copyright: © 2017 carlson, jones, harris, quezada & frazer. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems indulska, tuunainen & riemer 2019, vol 23, selected papers from acis 2017 preface 1 preface to the special section of selected papers from the 28th australasian conference on information systems marta indulska uq business school, the university of queensland m.indulska@business.uq.edu.au virpi kristiina tuunainen school of business aalto university kai riemer discipline of business information systems the university of sydney business school it is our privilege as the program co-chairs of the 28th australasian conference on information systems (acis) to have continued with the tradition of australasian journal of information systems (ajis) special section focused on papers presented at acis. the acis conference is dedicated to providing an annual forum for local and international researchers to exchange ideas related to information systems topics. the conference, much like the ajis journal, has a broad information systems scope, which is also reflected in the papers presented in this special section. the represented topics range from topical contributions focused on information systems security and also improving information systems related education, to methodologically related contributions providing guidance on measurement approaches, design science research, and a contribution providing guidance on adequate framing and argumentative writing of information systems research. the papers we invited for potential publication in this special section underwent a rigorous and double-blind review process. overall, we invited 7 papers and accepted 5 papers after 12 rounds of review. these 7 papers were selected on the basis of the views of the acis program committee and acis track chairs, from a total of 67 full-length papers accepted into the conference out of an original pool of 169 full-length submissions. all five papers included in this special section present an extension of the original acis paper publication. we are thankful to the reviewers for their time and effort in shaping the selected papers, the authors for their expert revisions, and the ajis editor for his assistance in creating this special section. we hope you will find this selection of papers of interest and enjoyable to read. sincerely, marta, virpi and kai ajis special section co-editors copyright: © 2019 indulska, tuunainen & riemer. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems indulska, tuunainen & riemer 2019, vol 23, selected papers from acis 2017 preface 2 paper title (paper title style) australasian journal of information systems hossain, hasan, chan & ahmed 2017, vol 21, research article predicting user acceptance and continuance behaviour towards lbs 1 predicting user acceptance and continuance behaviour towards location-based services: the moderating effect of facilitating conditions on behavioural intention and actual use mohammad alamgir hossain rmit university mahripon@yahoo.com mohammad imran hasan latrobe university caroline chan rmit university jashim uddin ahmed north south university bangladesh abstract the purpose of this study is to establish and examine the significance of a consumer acceptance and continuance model for location-based services (lbss) through the integration of perceived entertainment gratification (peg) and perceived application quality (paq) with the technology acceptance model (tam). by arguing that behavioural intention (bi) does not automatically lead to actual use (au), we investigated the moderating effect of facilitating conditions (fc) on the relationship between bi and au. a quantitative study was conducted in australia and bangladesh; data were obtained from multiple sources by systematic sampling of the distribution of questionnaires. for data analysis we applied the partial least square (pls) method. the results indicate that, in both australia and bangladesh, perceived usefulness (pu), peg and paq have significant influence on user attitude (ua), which has a subsequent effect on bi. interestingly, perceived ease of use (peou) does not have a direct effect on ua but indirectly influences it through pu—confirming the mediating effect of pu. further, fc has a moderating effect between bi and au. the implications of these findings and directions for future research directions are also discussed. keywords: location-based service, tam, consumer acceptance and continuance, entertainment gratification, application quality 1 introduction with the proliferation of mobile technologies—as a consequence of cost and usability—mobile devices are carried by most of the people at most of the time. as geographic location of mobile devices can be identified, location specific information can be poured-in. consequently, people are exposed to a number of location-based services (lbss) using mobile technologies. lbss are defined as the “services that depend on and are enhanced by positional information of mobile device[s]” (dhar and varshney 2011, p.122). the australian mobile telecommunications association defined them as “any information, entertainment or social media service or application available on a mobile device that makes use of the device’s geographic position” (amta 2010, p. 4). lbss have been evolved after the federal communications commission in the usa mandated mobile telecommunication companies to acquire precise information about the location of wireless callers who called 911. afterwards, mobile phone operators, mobile phone companies and software firms started investing significantly in the development and integration of lbs applications that could be accessible from mobile devices (bellavista, kupper, and helal 2008). today, lbs is an unprecedented way of providing information services to the community as well as an effective means of advertising for businesses. australasian journal of information systems hossain, hasan, chan & ahmed 2017, vol 21, research article predicting user acceptance and continuance behaviour towards lbs 2 the overall market of lbs is growing exponentially; the revenue generated worldwide from lbs in 2005 was around $2.6 billion and this figure had increased to almost $10 billion by 2010 (steinfield 2004). the four most commonly used services of lbs are emergency services, including roadside assistance, police, fire and hospital and search and rescue missions; content-and-lbss, including weather reports, train or bus schedules and nearest petrol stations; tracking and navigation services, such as fleet tracking and locating pets or children; and location-based advertising, including promotions and customer notifications (steinfield 2004). based on customer locations (or preferences) information is typically supplied by a lsp in the form of text, image, map or directions. the time saving aspects of lbs along with the convenience it offers can be very valuable to the user (pura 2005). a reasonable number of academic studies have been conducted that contribute empirical insights into the behavioural aspects of lbs use. investigating the intentions of customers to adopt lbs, zhou (2012) found that privacy, trust and perceived risk are important; whereas, in another study, the same sample suggested that other factors such as contextual offerings and ubiquitous connections are similarly important (zhou 2015). likewise, predicting behavioural intention (bi) of potential and experienced customers to adopt lbs, xu and gupta (2009) found performance expectancy, effort expectancy, personal innovativeness and privacy concerns to be significant. assessing the effect of the value perceived by customers on bi, pura (2005) identified conditional, monetary and convenience-related factors but found epistemic value to be likewise significant. prior studies also investigated consumers’ continuance intentions and found that perceived risk, usefulness and justice (zhou 2013), perceived privacy and service provider interventions to manage security and information quality (often considered as system quality) are important (chin 2012, xu and gupta 2009, xu, teo, and tan 2005). intention does not necessarily always confirm the actual acceptance of a technology. likewise, in spite of existing efforts in the industry, “lbs has not been widely adopted by users” (zhou 2013, p. 142). in europe it has gained only 1% in data traffic (patel 2004) and the trend is even worse in developing countries (dano 2003, rainio 2003). rather than examining user attitude and intention, ascertaining actual usage criteria is important because usage is considered ‘as a surrogate measure for information systems success’ (taylor and todd 1995, p. 144). by way of a solution to this problem, this study is based upon tam, which “has emerged as a powerful and parsimonious way to represent the antecedents of system usage through behavioural beliefs” (taylor and todd 1995, p. 145). another reason for using tam in this research is because it better explains information-systems-related behaviour than other competitive theories including the theory of planned behaviour (tpb) (taylor and todd 1995, p. 148). our study incorporates perceived entertainment gratification (peg) and perceived application quality (paq) with tam to provide a new integrated model for consumer acceptance and continuance of lbs founded on the following rationales: (i) perceived usefulness (pu) and perceived ease of use (peou) are used to predict consumer acceptance of lbs applications; (ii) most individual it users (not including employees coerced to use it applications as a part of their job requirements) do not expect merely utilitarian benefits but also hedonic benefits from using an application. they may accept an application out of an urgent need but decide to continuance if it is enjoyable to use (lim and ting 2014); (iii) given the importance of positive initial experience for accepting an application, the overall quality of the application is crucial for its continued use. it must address a variety of needs for a variety of people with effective privacy and security features; otherwise, people will uninstall it after having the initial use for a specific occasion; (iv) further, in order to convert ‘attitude’ into ‘intention’, an external locus of control over extraneous (e.g., technological) resources is needed (bhattacherjee, perols, and sanford 2008)—leading to the investigation of the moderating effect of facilitating conditions (fc). it can be reemphasised that pu and peou are the variables related to ‘acceptance’ while peg and paq are related with ‘continuance’ of lbs applications. these four variables are considered as the antecedents of ‘attitude towards use of lbs’. this approach is used by prior studies in information systems (e.g., lim and ting 2014). australasian journal of information systems hossain, hasan, chan & ahmed 2017, vol 21, research article predicting user acceptance and continuance behaviour towards lbs 3 most existing studies in the lbs domain were conducted using a single sample but prior information systems (is) studies have cautioned that the same behaviour of a given model cannot be guaranteed across cultures (aleke, ojiako, and wainwright 2011, tan et al. 2010). for instance, comparing between hong kong and canada, bailey and caidi (2005) found that differences in cultural notions of privacy may affect the acceptance of it-based innovations. privacy is a serious concern in australia but less so in bangladesh (hossain 2014, hossain and dwivedi 2014). hossain, dwivedi, and piercy (2015) explained that: tam has been tested across technologies, persons, contexts and times and has been found as a robust model with satisfactory predictive validity; however, it is not universal in terms of culture. generally speaking, tam is more consistent in north america and europe but has got some reservations in asia. hence, we applied the same model in two different cultures to understand its applicability across cultures. the model is validated by survey data using pls modelling. the organisation of the rest of this paper is as follows. in the next section we present the theoretical background and research framework used for the current study. we then describe the empirical aspects of the study, explaining the development of measurement instruments, data collection and analyses. in the final section we interpret the results and offer concluding remarks. 2 theoretical background to understand the acceptance behaviour of individuals in relation to technological innovations, tam is the most accepted and applied theory (hossain, dwivedi, and piercy 2015). tam signifies two determinants of technological acceptance: pu and peou. these directly influence user attitudes (ua) towards the use of technology that, in turn, lead to behavioural intention (bi) that ultimately drive actual use (au). although initially developed to investigate computer acceptance behaviours, tam has been applied in numerous applications including e-mail, the internet, e-commerce, software applications, knowledge management systems and modelling grammar—just to name a few (hossain, dwivedi, and piercy 2015). in spite of the mass acceptance of tam, the model still lacks some important and essential aspects for predicting consumer behaviour specific to ubiquitous technologies, such as mobile phones (ahmad et al. 2013). hence, many studies have extended tam, arguing that other contextualised factors and their correlations possess better capacity to predict the bi of consumers (mathieson, peacock, and chin 2001)—still the original two constructs, pu and peou, have always been used. 2.1 perceived usefulness pu, which accounts for user perceptions of the benefits of it usage, has been consistently identified in literature as a significant variable explaining the behaviour of it users. pu is defined as the degree to which an individual believes that using lbs will reduce the time and effort required to search and obtain results from a location-based query (adapted from davis 1989, xu, teo, and tan 2005). lbs can help an individual to find locations (positioning), train or bus schedules, and weather reports (barnes 2003, junglas and watson 2008). in most cases, information provided by lbs is faster as well as more accurate, convenient and precise than that gained by other means such as (street) directories (hurson and gao 2009, pura 2005). the effect of pu on ua has been validated by prior is research. in fact, pu has emerged as one of the most significant determinants of ua as evidenced by studies in mobile services (nysveen, pedersen, and thorbjørnsen 2005b) and mobile banking (bhattacherjee 2000). prior studies have found that, as a consequence of their various advantages, lbs applications develop positive attitudes in users (e.g., junglas, abraham, and watson 2008). therefore, we propose that: h1. perceived usefulness will have a positive influence on user attitude to accepting a location-based service australasian journal of information systems hossain, hasan, chan & ahmed 2017, vol 21, research article predicting user acceptance and continuance behaviour towards lbs 4 2.2 perceived ease of use ‘[e]ven if an application is perceived as useful, it will only be used if it is perceived as easy to use’ (davis 1989, p.320). peou is defined as the degree to which a person believes that using a technology will be free from effort (davis 1989). peou has been a widely-used variable in is/it studies, including those on the internet (bradley 2012, jebeile and reeve 2007, seyal and rahman 2003), m-commerce (yang 2005), and mobile services (lópez-nicolás, molinacastillo, and bouwman 2008, nysveen, pedersen, and thorbjørnsen 2005b). peou has also been identified as a significant factor for location tracking and location awareness services (junglas and watson 2008). usabilityand complexity-related issues, such as using complicated navigation or requiring too many inputs from users, form non-positive attitudes to using applications. among the available lbs applications, people tend to use those that require less mental effort and time; even for customised or advanced searches, applications that complete most inputs or provide easily selectable options are more preferable to users than those that request detailed inputs. following prior studies, our next hypothesis is as follows: h2. perceived ease of use will have a positive influence on user attitude to accepting a location-based service according to tam, pu is influenced by peou because, in the absence of other affecting variants, the easier a system is to use the more useful it can become. extensive empirical evidence can be observed in is literature that suggests that peou is significantly linked to user attitude, both directly and indirectly, via its impact on pu (mediating effect) (e.g., venkatesh and davis 2000, jebeile and reeve 2007). our study examines the effects of peou on pu because, in the absence of other modifying influences, the less effort required to use an lbs application, the more its perceived performance will be increased. hence: h3. perceived ease of use will have a positive influence on the perceived usefulness of lbss bhattacherjee, perols, and sanford (2008) argue that the initial adoption of technology is important in justifying its acceptance; however, its ultimate success depends on its continued use, as users may initially accept it, but may discontinue using it at a later stage. in its current context, despite the numerous benefits of lbs, there are growing concerns about the success of lbs applications. following other studies, we have used ‘continuance’ as a dummy for the success of lbs technology; perceived entertainment gratification (peg) and perceived application quality (paq) comprise the continuance criteria for this study (lim and ting 2014). 2.3 perceived entertainment gratification peg has received much attention in is literature in recent time. often referred to by other names such as hedonic/emotional benefits, intrinsic motivation or perceived enjoyment, peg refers to the degree to which lbs applications are perceived to be fun and entertaining to use and arouse positive feelings (lim and ting 2014). prior studies found that users wish to experience pleasure; they rank an online service with higher value if the user experience is entertaining and more enjoyable than dull (nysveen, pedersen, and thorbjørnsen 2005a). particularly because of improvements in technology that ensure high quality mobile devices in terms of speed, graphics and intelligence, lbs applications have become more attractive than before. the inclusion of animation, graphical views of location and sound as well as features such as ‘remember a search’ or ‘suggest to others’ offer entertainment advantages for lbs applications. numerous empirical studies relating to is continuance identified peg as an influential antecedent of attitude (e.g., huang, lin, and chuang 2007, leung and wei 2000). therefore we assume that: h4. perceived entertainment gratification will have a positive influence on user attitude to continued use of lbss australasian journal of information systems hossain, hasan, chan & ahmed 2017, vol 21, research article predicting user acceptance and continuance behaviour towards lbs 5 2.4 perceived application quality in is research, especially research related to internet and e-commerce, system quality is a commonly-applied variable. for lbs applications, system quality is related to the application itself. functionalities such as internet speed, the closest telecom tower and fit to screen are beyond the scope of individual applications. in general, perceived application quality (paq) is associated with particular applications and refers to user perceptions regarding technical levels of performance of lbs applications for two aspects: security and privacy (lim and ting 2014, akter, d’ambra, and ray 2013). application security is defined as the degree to which lbs applications are perceived to provide security by protecting data and information to ensure transactions without financial loss. similarly, privacy has always been considered an important parameter for gaining confidence in lbs applications, corresponding to the respect of lbs applications for user privacy in terms of collecting, storing, using and sharing personal information (li 2011). as lbs applications are location specific, they can link locations to user interests or at least to generic characteristics (e.g., gender and age). further, to some extent, user movements (e.g., to casinos, brothels or suspected terrorist hubs) and browsed services (steinfield 2004) can be tracked or traced-back (dhar and varshney 2011). prior research demonstrates that paq has considerable influence on web based services and lbs use (li 2011, zhou 2013b, steinfield 2004). therefore, we propose the following hypothesis: h5. perceived application quality will have a positive influence on user attitude to continued use of lbss 2.5 user attitude, behavioural intention and usage behaviour prior studies have established that ua is an essential variable explaining human behaviour or bi. initially, theory of reasoned action and tpb authenticated a direct association between ua and bi. later, tam re-established bi’s direct dependence on ua. several is studies have provided empirical evidence supporting such a relationship (seyal and rahman 2003). for example, kuo and yen (2009) found that ua is the most influential factor for bi to use 3g mobile value-added services, while xu, oh, and teo (2009) established the relationship between ua and bi in the context of location-based advertising and mobile services (lópeznicolás, molina-castillo, and bouwman 2008, nysveen, pedersen, and thorbjørnsen 2005a). based on these theoretical lenses, this study posits that users with positive attitude towards lbs applications are more likely to intend to use them than users who possess negative attitudes. therefore: h6. user attitude will have a positive influence on user behavioural intention to use lbss in tam, actual usage behaviour (au) is modelled as a direct function of bi. extensive empirical studies have validated this notion. for example, investigating in the m-commerce field, wu and wang (2005) found bi to be a factor significantly affecting au. such evidence has been established in electronic services for e-consumers (hsu and chiu 2004) and mobile services (kargin, basoglu, and daim 2008). assuming that positive intention towards lbs applications eventually drives au, the following hypothesis is developed: h7. behavioural intention will have a positive influence on user’s actual usage behaviour of lbss 2.6 facilitating conditions triandis (1980) claimed that positively intentioned behaviour cannot occur if fcs in the environment prevent it. he argued that fc is an objective measure, whereas other studies hold fc to be a perception-based variable (e.g. bhattacherjee 2000, limayem and hirt 2003). bhattacherjee (2000) considers fc to be an external constraint, demonstrating beliefs about availability of resources as facilitating specific behaviour. he suggested that internet-enabled services such as e-banking will become more feasible once the required technological infrastructure becomes available and effectual. in an lbs context, fc can be defined as the degree to which technical and infrastructural resources, such as sophistication of devices australasian journal of information systems hossain, hasan, chan & ahmed 2017, vol 21, research article predicting user acceptance and continuance behaviour towards lbs 6 (equipped with location abilities, e.g., smartphones) and internet connectivity, are available for access within a reasonable response time (amta 2010, bhattacherjee, perols, and sanford 2008). prior is studies (e.g., triandis 1980) have considered fc as an important variable and direct antecedent for explaining au, although thompson, higgins, and howell (1991) could not establish such a relation for personal computer use. later studies claim that fc should be considered a direct antecedent of bi: ‘[t]he absence of facilitating resources represents barriers to usage and may inhibit the formation of intention and usage’ (taylor and todd 1995, p. 153). however, contrary to triandis (1980), limayem and hirt (2003) found that the link is strong between fc and bi but not between fc and au. interestingly, limayem and hirt (2003) implied a moderating effect of fc on bi and au but did not test it. “facilitating conditions moderate the link between behavioural intentions and behaviour …. even if a person has the intention to perform a particular behaviour or habitually performs the behaviour, the behaviour may not occur when the facilitating conditions do not permit it. for example, a student who intends to use an ibct [internet-based communication tools] to communicate with his/her peers will not be able to do so if his or her internet connection doesn’t work.” (limayem and hirt 2003, p. 71) based on previous is research that finds mixed influences of fc on bi and au and addressing limayem and hirt (2003) research agenda, this current study investigates the moderating effects of fc on the relationship between bi and au as hypothesised below: h8. the relationship between bi and au is moderated by the level of available facilitating conditions; that is, the relationship is weaker with insufficient fc and stronger with greater fc. a conceptualised structural model demonstrating the variables and their associated relationships considered by this study is represented diagrammatically in figure 1. figure 1. the research model 3 research methods 3.1 data collection and validation data was collected from australia and bangladesh using ‘mall-intercepted’ and online surveys. the face-to-face surveys were conducted in melbourne (australia) and dhaka (bangladesh) in universities, cafés and shopping centres. these cities were chosen because they comprise both urban and suburban communities of people from different cultural, educational and professional backgrounds. moreover, these two cities occupy respectively the top and bottom positions of the ‘world’s most liveable city’ scale (the economist 2014). telecommunication infrastructure is one of the 30 factors considered in this ranking, where melbourne scored the highest at 100 and dhaka the lowest at 26.8. therefore, considering these two cities, this study sought to discover whether availability of infrastructure really matters for lbs use. in order to australasian journal of information systems hossain, hasan, chan & ahmed 2017, vol 21, research article predicting user acceptance and continuance behaviour towards lbs 7 reduce any systematic bias, the face-to-face surveys were conducted in shopping malls of different suburbs and different universities of the two cities at different days and times. each potential respondent was approached randomly. before answering the questionnaire, each respondent was asked if they use at least one lbs application. moreover, an online survey was conducted using social networking groups in facebook and twitter. six groups – three from each country – have been approached while only one group from each country has agreed to distribute the questionnaire among their members. overall, the response from online survey was disappointing; only 69 responses were collected. after exhausting several attempts, a total of 219 responses were collected, but six of these were unusable because more than 10% of their values were missing. standard demographic measures were analysed to characterise the sample (see table 1). categories distribution (%) categories distribution (%) gender education male 56 high school 3.51 female 44 college 45.96 age undergraduate 38.3 18-25 58.16 postgraduate 12.23 26-33 23.21 internet experience 34-41 10.58 <1 year 2.67 42-49 5.91 1-3 years 26.45 50+ 2.14 4-5 years 24.67 5+ years 46.21 table 1: demographic statistics (n = 213) we validated the data in three ways. first, we tested whether the responses from two samples (103 from australia and 110 from bangladesh) were significantly different. a mann-whitney u test showed that there was no significant difference in the data from two independent samples. second, we tested for non-response bias using a kolmogorov-smirnov (k-s) test. using one indicator from each construct, we applied a two-sample k-s test. the non-significant k-s test1 suggested that the sample was drawn from the same distribution. finally, we checked the common method variance (cmv), although prior ‘empirical tests of tam have shown that it explains much of the variance in usage intention and self-reported usage’ (taylor and todd 1995, p. 145). as a remedy for item-characteristic effects that can arise when common scale formats are used, we employed three open-ended measures to ascertain au and likert scale for the other measures (sharma, yetton, and crawford 2009). we then applied one of the most prescribed methods—the marker variable (mv) technique (lindell and whitney 2001). we added an mv by deliberately including the question ‘lbs are flexible to meet variety of my needs’ that was theoretically unrelated to the research variables. the highest correlation2 between the mv and the latent variable (lv) was observed as bi 0.176—the maximum shared variance with other variables in the model is around 3%, indicating that cmv was rather limited in our research. pls, particularly smartpls (version 3.2.3, www.smartpls.de) were employed for data analysis. 3.2 measures this study consists of eight constructs. the constructs were measured using multiple-item scales drawn from pre-validated measures in is (see appendix a). all the measures operationalizing the constructs in the research model were considered as ‘reflective’ because they were interchangeable and shared one theme. specifically, pu and peou were measured 1 p values: 0.000, 0.000, 0.000, 0.012, 0.034, 0.000, 0.000, 0.031 for the indicators taken each from the derived constructs. 2 the correlating values are: pu, peou, eg, sq, fc, att, int, use and are equal to 0.106, 0.141, 0.155, 0.098, 0.138, 0.153, 0.176, 0.119 respectively. australasian journal of information systems hossain, hasan, chan & ahmed 2017, vol 21, research article predicting user acceptance and continuance behaviour towards lbs 8 by an instrument developed by davis (1989); ua, peg and bi were measured using a method adapted from lim and ting (2014) and limayem and hirt (2003). paq was ascertained using an approach taken from akter, d’ambra, and ray (2013), lim and ting (2014), and xu and gupta (2009), and ub was established by adapting bhattacherjee, perols, and sanford (2008). four items that measure fc were adapted from various studies: item 1 denoted ease of access (taylor and todd 1995, bhattacherjee 2000); items 2 and 3 indicated reliability and efficiency of services (limayem and hirt 2003); and item 4 signified the levels of assistance provided to users when they encountered difficulties (thompson, higgins, and howell 1991). 4 results 4.1 assessment of measurement properties assessing the measurement properties of the research model following standard pls techniques we focused on the reliability and validity of the measures representing each construct and their convergent and discriminant validities. the first set of discriminant validity tests proves that a construct is more strongly related to its own measures than to any other construct in the model. to test this, we first calculated the average variance extracted (ave) and compared its square root with correlations among constructs. as depicted in table 2, all constructs met the acceptable criterion for ave (as greater than 0.5) and thus established the reliability of the latent variables (chin 2010). moreover, the square root of ave when compared to construct correlations proves that each construct is more closely related to its own measures than to other constructs (as shown on the right-hand side of table 2). then, the internal consistency for each block of indicators was examined by applying two measures: cronbach’s alpha provided the lower boundary of internal consistency reliability and composite reliability provided the upper boundary for the (unknown) true reliability (hair jr et al. 2017); both measures needed to exceed 0.7. table 2 shows that all constructs met the threshold values. chin (2010) recommended that ‘not only [should] each measure be strongly related to the construct it attempts to reflect, but it should not have a stronger connection with another construct’ (p. 671). therefore, we performed two tests of discriminant validity to examine how each item relates to each construct. first, the loading of each item was calculated. referring to igbaria, guimaraes, and davis (1995)’s argument, we set 0.6 as the minimum cut-off level for item reliability; following this rule, one item (att 2: ‘i am comfortable using lbs’) was discarded. it appears that for many respondents this indicator may have been indistinguishable from peou. item loading checks indicated that all items effectively imitated their respective constructs (as shown by the bold values of each construct in table 3). later, we compared the correlations of each item to all other constructs obtained from a cross-loading matrix. table 3 shows that no item loads higher value onto other constructs than onto the construct it represents and thus we claim that discriminant validity has been obtained at item level. australasian journal of information systems hossain, hasan, chan & ahmed 2017, vol 21, research article predicting user acceptance and continuance behaviour towards lbs 9 measures of discriminant validity inter-correlations of the latent variables cr ave α construct ua peg bi fc peou paq pu au 0.805 0.508 0.711 ua 0.713 0.851 0.589 0.768 peg 0.577 0.767 0.888 0.667 0.832 bi 0.526 0.701 0.816 0.864 0.614 0.790 fc 0.532 0.634 0.669 0.784 0.881 0.650 0.820 peou 0.502 0.636 0.747 0.628 0.806 0.849 0.584 0.762 paq 0.577 0.734 0.696 0.690 0.695 0.764 0.889 0.668 0.834 pu 0.571 0.618 0.608 0.599 0.553 0.572 0.817 0.860 0.676 0.759 au 0.434 0.615 0.658 0.626 0.702 0.603 0.615 0.822 cr – composite reliability; ave average variance extracted; α – chronbach’s alpha; ua – user attitude, peg – perceived entertainment gratification, bi – behavioural intention, fc facilitating conditions, peou – perceived ease of use, paq – perceived application quality, pu – perceived usefulness, au – actual use; bold diagonal values are square root of respective construct table 2: psychometric properties of the constructs then, we examined convergent validity to see ‘how high … each of the loadings [were] and [whether] they [were] more or less similar’ (chin 2010, p. 674). although there was no set value prescription for the range, the observation ‘the narrower the range and [the] higher the lowest loading’ was taken as guide to ensure convergent validity’ (hair jr et al. 2017). in this study we achieved relatively higher average loadings and a narrower range (from 0.674–0.892), confirming no issues with convergent validity. australasian journal of information systems hossain, hasan, chan & ahmed 2017, vol 21, research article predicting user acceptance and continuance behaviour towards lbs 10 user attitude perceived entertainment gratification behavioural intention facilitating conditions perceived ease of use perceived application quality perceived usefulness actual use ua1 0.736 0.562 0.475 0.508 0.464 0.584 0.523 0.456 ua3 0.674 0.323 0.269 0.29 0.241 0.303 0.385 0.204 ua4 0.715 0.306 0.324 0.31 0.317 0.311 0.312 0.25 ua5 0.725 0.371 0.374 0.338 0.348 0.352 0.353 0.245 peg1 0.427 0.738 0.492 0.517 0.484 0.597 0.268 0.368 peg2 0.45 0.751 0.446 0.724 0.366 0.429 0.542 0.432 peg3 0.367 0.778 0.552 0.819 0.545 0.583 0.496 0.544 peg4 0.506 0.801 0.65 0.673 0.559 0.641 0.573 0.542 bi1 0.441 0.575 0.864 0.569 0.68 0.577 0.509 0.59 bi2 0.455 0.595 0.851 0.608 0.604 0.552 0.533 0.578 bi3 0.409 0.537 0.804 0.494 0.562 0.574 0.445 0.483 bi4 0.41 0.585 0.74 0.507 0.589 0.575 0.494 0.481 fc1 0.396 0.595 0.582 0.809 0.547 0.56 0.458 0.518 fc2 0.473 0.656 0.507 0.779 0.491 0.581 0.389 0.458 fc3 0.45 0.751 0.446 0.724 0.366 0.429 0.542 0.432 fc4 0.367 0.778 0.552 0.819 0.545 0.583 0.496 0.544 peou1 0.327 0.417 0.508 0.496 0.745 0.491 0.441 0.531 peou2 0.41 0.497 0.557 0.436 0.831 0.599 0.469 0.559 peou3 0.442 0.589 0.659 0.577 0.85 0.582 0.448 0.612 peou4 0.426 0.529 0.666 0.515 0.795 0.56 0.429 0.559 paq1 0.47 0.486 0.408 0.479 0.508 0.709 0.37 0.39 paq2 0.444 0.654 0.614 0.568 0.518 0.777 0.448 0.46 paq3 0.416 0.575 0.555 0.518 0.538 0.764 0.482 0.528 paq4 0.424 0.527 0.554 0.542 0.558 0.804 0.453 0.463 pu1 0.505 0.586 0.564 0.57 0.49 0.491 0.788 0.538 pu2 0.467 0.552 0.532 0.508 0.485 0.538 0.834 0.495 pu3 0.443 0.451 0.371 0.436 0.351 0.431 0.852 0.424 pu4 0.445 0.414 0.506 0.431 0.47 0.401 0.793 0.548 au1 0.383 0.556 0.575 0.579 0.663 0.56 0.545 0.892 au2 0.405 0.557 0.637 0.548 0.612 0.556 0.529 0.881 au3 0.262 0.381 0.366 0.401 0.431 0.332 0.446 0.678 table 3: outer model loadings and cross-leadings 4.2 assessment of the structural model for the purpose of assessing the structural model, the direction of path coefficient (β value) and the magnitude of t-statistics were checked. the results are illustrated in figure 2 and summarised in table 4. the results found that all our proposed hypotheses were supported except h2. as a criterion of the predictive power of the model we calculated the coefficient of determination (r2) values of the endogenous constructs (henseler, ringle, and sinkovics 2009). our model explains 43.10% of au, 27.70% of bi and 43.80% of ua (r2 values of 0.67, 0.33, and 0.19 are considered as substantial, moderate, and weak, respectively; henseler, australasian journal of information systems hossain, hasan, chan & ahmed 2017, vol 21, research article predicting user acceptance and continuance behaviour towards lbs 11 ringle, and sinkovics 2009, p. 303). moreover, we examined the effect size3 (f2). the addition of two variables (namely, paq and peg) into the tam resulted in the effect size (f2) of 0.105 which is almost ‘medium’ (f2 of 0.02, 0.15, and 0.35 represent small, medium, and large effect, chin 2010). therefore, it can be suggested that the model developed in this study has better explanatory power than traditional tam for explaining lbs use. path coefficient standard deviation t statistics p values supported ? h1: perceived usefulness → user attitude 0.289*** 0.074 3.927 0.000 yes h2: perceived ease of use → user attitude 0.057 ns 0.091 0.629 0.529 no h3: perceived ease of use → perceived usefulness 0.558*** 0.059 9.518 0.000 yes h4: perceived entertainment gratification → user attitude 0.194* 0.086 2.315 0.021 yes h5: perceived application quality → user attitude 0.229** 0.086 2.645 0.008 yes h6: user attitude → behavioural intention 0.526*** 0.06 8.789 0.000 yes h7: behavioural intention → actual use 0.658*** 0.044 15.017 0.000 yes significance level *p < 0.05, **p < 0.01, ***p < 0.001 table 4: structural properties of the constructs 4.3 assessing the mediating effect the results show that peou does not affect bi but has an indirect effect through pu. this indicates the potential mediating role of pu that is strongly supported by the literature.(e.g., venkatesh and davis 2000). to substantiate this assertion we performed further analysis. mediation can be assessed by examining the relationship of the indirect link via the potential mediating variable—path a from the predictor (peou) to the mediator (pu) and path b from the mediator to the endogenous variable (ua), and the direct link between predictor and endogenous variable (ua; path c) (eberl 2010). for a mediation effect to be proven, the z value4 has to be significant. in our case, z equals 3.593 which is significant at p < 0.005, but b value is insignificant and thus the full mediation effect is accepted. further, we calculated the size of the effect5 establishing that 73.88% of the total effect of peou on ua is explained by pu. 4.4 assessing the moderating effect to examine the moderating effect of fc on the relationship between bi and au, we used the pls product-indicator approach. this process proposes building product terms between the indicators of the exogenous variable (i.e., bi) and the indicators of the latent moderator variable (i.e., fc). consequently, the interaction term consists of 16 items (4 bi x 4 fc) (see henseler and fassott 2010 for detail). to analyse the moderating effect, we estimated the influence of bi and fc as well as the interaction term on ua. to claim a moderation effect, the 3 𝑓2 = 𝑅𝑖𝑛𝑐𝑙𝑢𝑑𝑒𝑑 2 −𝑅𝑒𝑥𝑐𝑙𝑢𝑑𝑒𝑑 2 1−𝑅𝑖𝑛𝑐𝑙𝑢𝑑𝑒𝑑 2 4 𝑍 = 𝑎∗𝑏 √𝑏2𝑠.𝑒.𝑎 2 +𝑎2𝑠.𝑒.𝑏 2+𝑠.𝑒.𝑎 2 𝑠.𝑒.𝑏 2 where a, b, c = path coefficient from peou to pu, pu to ua and peou to attitude respectively, s.e. = standard error 5 𝑉𝐴𝐹 = 𝑎∗𝑏 (𝑎∗𝑏)+𝑐 australasian journal of information systems hossain, hasan, chan & ahmed 2017, vol 21, research article predicting user acceptance and continuance behaviour towards lbs 12 path coefficient between the interaction term and ua has to be significant (regardless of the values of the other two relations), which was true in our case (β = 0.537, sd = 0.245, t = 2.192, r2 = 0.502) and significant at p < 0.05. to assess the strength of the moderating effect, the literature suggests using effect size (f2).6 the effect size is ‘moderate’ when it has an f2 value of 0.15. thus, we confirm that fcs do moderate the relationship between bi and au, affirming hypothesis h8. attitude towards use of lbs behavioral intention to use lbs facilitating conditions 0 .5 3 7 * continuance perceived application quality 0.194* 0. 22 9* * 0.526*** actual usage behavior 0.658*** acceptance perceived ease of use perceived usefulness 0 .5 5 8 ** * 0.057ns 0.289*** perceived entertainment gratification significance level *p < 0.05, **p < 0.005, ***p < 0.001; the values are β and parenthesis are t-values figure 2. the structural evaluation of the model 4.5 multi-group analysis to check whether respondent perceptions were different based on their culture (we used geographical location as a dummy), we performed multi-group analysis, which reveals significantly different group-specific pls-sem results for the relationships in the structural model. we compare pls-sem results of lbs users from australia (group 1) to those from bangladesh (group 2). henseler, ringle, and sarstedt (2016) suggested that it is important to assess the measurement invariance of composite model (micom) in variance-based sem using pls. in our assessment, we apply the three steps recommended in micom procedure. first, in order to establish configural invariance, we ensure that the following three aspects are identical for both groups: setup of the measurement models and the structural model; data treatment for the model estimation using the full set of data and each group of data; and algorithm settings for all model estimations. in the second step, we assess the compositional invariance using the smartpls 3 software (ringle et al., 2015). in principle, compositional invariance requires that c equals one. a permutation test reveals if the correlation c is significantly different from one or not. table 5 shows the results of 5,000 permutations. the permutation test substantiates that the c values are close to 1 and none of them are significantly different from one. we therefore conclude that compositional invariance has been established for all composites in our model. finally, in step 3, we assess the constructs’ equality of mean values and variances across groups. the permutation test results show that the mean value and the variance of a composite in group 1 do not significantly differ from the results in group 2, thus full measurement invariance is established. 6 𝑓2 = 𝑅𝑚𝑜𝑑𝑒𝑙 𝑤𝑖𝑡ℎ 𝑚𝑜𝑑𝑒𝑟𝑎𝑡𝑜𝑟 2 −𝑅𝑚𝑜𝑑𝑒𝑙 𝑤𝑖𝑡ℎ𝑜𝑢𝑡 𝑚𝑜𝑑𝑒𝑟𝑎𝑡𝑜𝑟 2 1−𝑅𝑚𝑜𝑑𝑒𝑙 𝑤𝑖𝑡ℎ 𝑚𝑜𝑑𝑒𝑟𝑎𝑡𝑜𝑟 2 australasian journal of information systems hossain, hasan, chan & ahmed 2017, vol 21, research article predicting user acceptance and continuance behaviour towards lbs 13 composite c value (=1) 95% confidence interval composite invariance? pu 0.999 [0.996; 1.000] yes peou 0.998 [0.996; 1.000] yes peg 0.995 [0.988; 1.000] yes paq 0.996 [0.987; 1.000] yes ua 0.989 [0.971; 1.000] yes bi 0.999 [0.997; 1.000] yes au 0.997 [0.988; 1.000] yes composite difference of the composite’s mean value (=0) 95% confidence interval equal mean values? pu -0.009 [-0.367-0.375] yes peou -0.009 [-0.349-0.345] yes peg -0.012 [-0.331-0.409] yes paq -0.010 [-0.334-0.355] yes ua -0.008 [-0.387-0.413] yes bi -0.010 [-0.368-0.385] yes au -0.008 [-0.316-0.321] yes table 5. micom results now, in order to assess if the relationships are different in the two different samples, we run the multi-group analysis between australian and bangladeshi users. we applied the permutation test as it has advantages statistical properties and recommended by the scholars (e.g., hair jr et al. 2017). the results, presented in appendix b, reject variation in the two groups. that means, the hypothesised relationships developed in this research remain same in two different counties namely australia and bangladesh. 5 discussion 5.1 discussion of the main effects the results established that, overall, customers accept lbs applications when they perceive them to be useful; however, to engage in continued use, customers expect applications to offer hedonic benefits and with good quality in terms of privacy and security features. this proposed and validated framework paves the way for application developers to conduct integrated analyses and design of lbs use, while underscoring that the provision of a useful service, although important, is not sufficient on its own; developers must address other associated factors, such as quality of apps, when designing lbs applications. now we will discuss the results of each hypothesis along with their practical implications. supporting the first hypothesis our results found that pu influences attitudes to lbs use (β = 0.289; t = 3.927; p = 0.000). our study suggests that consumers perceive that lbs applications will improve their performance and productivity when finding locations and other related information. in other words, customers form positive attitude towards lbs applications when these are perceived to be use-enhancing search initiatives (e.g., saving time and effort or providing current and trustworthy information). given this scenario, lbs applications should be designed applying an extensive range of tools related to spatial technologies to provide optimised solutions to users. in this regard unbiased and current information would be important. in addition, customised services could be offered to users based on their search patterns. as a minimum, remembering past user searches could further improve the pu of lbs. additionally, by adapting the frequently used visiting features of users and refining their searches accordingly, lbs applications could integrate user information with updated versions, helping users to save time and effort on their next visits. australasian journal of information systems hossain, hasan, chan & ahmed 2017, vol 21, research article predicting user acceptance and continuance behaviour towards lbs 14 in contrast to the expectations of our second hypothesis (h2), peou is unable to significantly predict uas (β = 0.057; t = 0.629; p = 0.529). in the context of lbs, zhou (2012) likewise found that ‘effort expectancy’ does not influence usage intention. this is likely because lbs applications are not perceived as difficult to use and so peou does not necessarily play a role in forming (positive) attitudes. indeed, as a consequence of unprecedented advancement in technologies (e.g., smartphones) and software, it is observed that uas to technology are becoming positive. even toddlers can now install and use applications (e.g., games) without asking for help from elders, a phenomenon that is relatively recent. at the same time, selfefficacy is increasing among users who use computer applications regularly; therefore, peou does not create any variance in attitude. nonetheless, lbs applications should be built so as to require minimal effort for users to understand and use (junglas and watson 2008). however, the significance of peou should not be underestimated because peou positively and strongly affects pu (β = 0.558; t = 9.518; p = 0.000). moreover, our mediation effect statistically proved that peou has an indirect effect on ua through pu (β = 0.537, t = 2.192). this finding is consistent with prior studies. hossain, dwivedi, and piercy (2015) claimed that “an increasing number of tam studies have found that pu has greater influence on bi than peou; whereas, peou has stronger effect through pu than as a direct effect on bi” (p. 806). our results claim that, when other things remain constant, pu is influenced by peou, while ua is dependent on pu. this is implied as follows. the easier an lbs application is to use, the more users perceive it as useful and a useful application encourages users to form positive attitudes to using it. consistent with our findings, prior is studies recommend that peou is significantly and indirectly associated with ua through pu (e.g., venkatesh and davis 2000). the fourth hypothesis posited that peg would have a positive effect on attitudes to lbs use; the results revealed a significant contribution (β = 0.194; t = 2.315; p = 0.021) and thus h4 was confirmed. as contemporary customer expectations are no longer confined to utilitarian benefits but are also directed towards hedonic pleasures, there is a need for lbs applications to be entertaining and fun and to create enjoyment for users. features such as light and sound effects (using flash, for example) and the potential to share searches on social media, could be considered as ways of converting attitudes from dull to lively. nonetheless, developers should not forget information quality at the cost of focusing on entertainment—rather, a blend of both concerns in equal measure might be useful. our fifth hypothesis was accepted (β = 0.229; t = 2.645; p = 0.008); as expected, a positive association exists between paq and ua. this finding emphasises that users do not consider the complexity of lbs applications as an issue and demonstrate comparatively little concern for enjoyment of the application; conversely, they are more concerned with the quality of the application itself, mostly as related to security and privacy. from a systems perspective, this is obviously an important issue that continually needs to be addressed. therefore, application developers should take measures such as blocking third-party advertisements or affiliated sites, securing users from potential hazards and risks associated with privacy and security and informing users about privacy and security measures. similarly, lsps can encourage customers to sample trial versions and provide demonstrations and information on security and privacy mechanisms (e.g., through frequently asked questions and video tutorials). our next hypothesis (h6) predicted that ua to lbs use would have a positive influence on bi; the findings of this study confirm such a relationship (β = 0.526; t = 8.789; p = 0.000). this means that consumers who possess favourable attitudes to lbs use will have a higher tendency to use it. one practical implication of this finding is that providers of lbs applications should conduct proper marketing initiatives to convince potential customers about their (positive) functionalities, assisting them to form positive attitudes. as the prior hypothesised relationships explain, disseminating useful features along with benefits related to enjoyment while demonstrating how applications are secure and privacy-enabled would instil confidence and help to form positive attitudes. strategic emphases and resource allocation are of great importance during the development of lbs applications (lim and ting 2014). australasian journal of information systems hossain, hasan, chan & ahmed 2017, vol 21, research article predicting user acceptance and continuance behaviour towards lbs 15 5.2 effect of moderating variables in an attempt to extend the existing literature on user intentions and au that represents a critical but understudied area in is, we investigated the effect of fcs. our data analysis established that user’s bi to use lbs applications would drive them to au, strongly supporting hypothesis h7 (β = 0.658; t = 15.017; p = 0.000). further, the moderating effect calculations demonstrate that fc has a positive and significant moderating influence on the relationship between bi and au. this means that the relation between intention and au is more prominent under the influence of fc. in other words, when fc is low, bi will have less effect on au than when fc is high. the implications of this finding are enormous. the results suggest that for people intending to use lbs applications, factors including pu, peou or peg are important but, when they decide to actually use lbs applications, fc is of greater concern—increased fc will lead to greater usage. in terms of practical implications, the supply-side stakeholders of lbs applications (e.g., mobile carriers and internet providers) should ensure steady infrastructure that permits and encourages people to use lbs. the positive impact of fc should also alert system developers to seriously address potential barriers, including how downloads (of maps or addresses, for example) can be optimised in the presence of slow internet speeds. developers can also educate users. for instance, as the accuracy of lbs applications varies considerably depending on the settings chosen by the user, amta (2010) guidelines demonstrating the proper use of lbss would benefit their usage. ‘however, the presence of facilitating resources may not, per se, encourage usage’ (taylor and todd 1995, p. 153) if other antecedents are missing or poorly managed. 5.3 implication of cross-country differences one of the major objectives of this study was to understand the cross-cultural differences in user perceptions towards lbs use. our multi-group analysis shows that, in general, there is no significant difference between two cultural groups (australia and bangladesh, representing an individualistic and a collectivistic society, respectively). the main implication of this finding is that a major reservation concerning the limitation of the applicability of the tam in asian culture is not necessarily be true as long as contextual factors are incorporated—our model worked well across two different cultures. in other words, the tam behaves well for a technology (e.g., internet) which has almost same nature of applications across societies (fusilier and durlabhji 2005). for general/universal technologies, the market is progressively getting truly global. as mentioned earlier, anyone can (un)install mobile applications without asking help from others – this is not necessarily the case exclusively in the developed countries. the ease of installation, use, and uninstallation of an application thus deny the impact of several historically-proven variables such as self-efficacy, (general and computer) literacy, prior experience with technology, etc. on user acceptance and continuance. in fact, in some cases, people in developing countries use more mobile applications, or the extent of use of a particular application is much higher, than that in the developed countries (e.g., mobile health applications, braw 2014, gable 2015). as a result, the applicability of the tam variables is getting universal. for example, in many instances, perceived complexity does not impact the adoption of several technologies in both developed and developing countries [for example, online banking in uk (yousafzai, foxall, and pallister 2010) and india (roy, kesharwani, and singh bisht 2012)]. in other words, the unique socio-economic and demographic profiles of the end-users in underdeveloped communities do not actually have significant impact shaping the perceptions of the users toward the use of universal technologies. hence, it is not recommended to make significant difference in the design of the functional and interface characteristics (from users’ perspectives) of mobile applications based on the cultural or socioeconomic differences. however, a more detailed look at the results (appendix b) indicates some differences in perceptions—though not statistically significant ones. the three statistically supported antecedents (pu, peg and paq) have stronger effects on australian users, indicating that australian users expect lbs applications to be useful, pleasant to use and safe in security and privacy measures. conversely, although bangladeshi users agree with these factors, they are australasian journal of information systems hossain, hasan, chan & ahmed 2017, vol 21, research article predicting user acceptance and continuance behaviour towards lbs 16 less likely to be concerned about peg; comparatively, bangladeshi users rank peou higher than australian users do. this finding is supported by cultural practice. the addressing of the mechanisms of physical properties is non-uniform (i.e., different throughout different parts of a city) and quite hard to comprehend in bangladesh. therefore, following the instructions provided by an lbs application is not always easy. moreover, updating mechanisms for a change of address is unscientific and can sometimes complicate the uses of applications. further, lbs applications are still at infancy stage; only a handful of services are currently available. hence, bangladeshi users still consider lbs applications as a privilege and as already too much of a luxury to expect entertainment, although the applications are required to be privacyand security-enabled. on the contrary, australian users have been enjoying almost every possible lbs application that they can imagine. they expect that lbs applications should be useful (e.g., providing built-in information as to whether a store is open or closed, along with general business hours), secure enough to transact (e.g., to order a drink from the nearest coffee parlour before arrival) and not susceptible to privacy violations—while, at the same time, the whole virtual journey should be enjoyable. thus, one practical suggestion derived from this multi-group analysis is that lbs developers should be attentive when designing applications, especially those targeting individualistic societies. 5.4 theoretical contributions the primary contribution of this study is its integration of variables associated with user acceptance (i.e., pu and peou) and continuance (i.e., peg and paq) in a single model in the context of lbss. based on tam, this study incorporated three context-based constructs and examined their relationships with behavioural variables, namely, attitudes, intentions and use. predicting system use behaviours, this study provides an invitation for future research to consider acceptance and continuance factors using a single model, tested on two different cultures and found to have consistent results. in addition, the model makes a contribution to is literature by introducing debate as to how user intention converts into au. finally, the demonstrated application of pls path modelling techniques provides functional guidelines to emerging researchers about how to systematically analyse quantitative data obtained from two samples with moderating and mediating variables and multi-group analysis. 5.5 limitations and future research direction despite contributing new and valuable insights to lbs literature, this study has been faced with some limitations that may inform future research. first, we used a self-reported survey that may have resulted in self-selection bias. although the cmv tests did not expose any concerns regarding data quality, it is still not possible to claim definitively that the data are free from self-reported bias in relation to use behaviour. future research could use actual objective usage data from lbs providers. second, the sample used in the survey was limited to two countries assumed to represent different cultures and such an assumption could result in limited generalisability. future studies could investigate this model in different cultures and locate differences, which may assist in developing a generic acceptance and continuance model. third, this study analysed the model using longitudinal data aiming to understand the behaviour of users at a specific time. future studies could examine the model using crosssectional data. further, the effect of control variables including age, education and previous experience using ubiquitous technologies would enhance the understanding of lbs behaviour. fourth, prior studies have approached fcs by categorising them into technologyand resource-based constraints; we applied a single construct using indicators from both. further research investigating their individual effects may help to better understand the effects of fcs. finally, lbss can be of two types: active and passive. using ‘active services’ a user himself/herself actively locates his/her position (i.e., the location of the mobile device) when seeking information contextualised to that location. on the contrary, ‘passive services’ enable a user to be located by someone else—‘once the user has consented to and initiated use of the service, identified third parties may view the location of that user’s device without explicitly notifying the locatee each time their device is being located’ (amta 2010, p. 5). hence, passive services raise more concerns about potential privacy and security abuse. this current study australasian journal of information systems hossain, hasan, chan & ahmed 2017, vol 21, research article predicting user acceptance and continuance behaviour towards lbs 17 focused only on active services, whereas future research could examine passive service-related issues and investigate this model in a different contextual setting. references ahmad, norita, noha tarek amer, faten qutaifan, and azza alhilali. 2013. "technology adoption model and a road map to successful implementation of itil." journal of enterprise information management 26 (5):553-576. akter, shahriar, john d’ambra, and pradeep ray. 2013. "development and validation of an instrument to measure user perceived service quality of mhealth." information & management 50 (4):181-195. aleke, bartholomew, udechukwu ojiako, and david w wainwright. 2011. "ict adoption in developing countries: perspectives from small-scale agribusinesses." journal of enterprise information management 24 (1):68-84. amta. 2010. amta guidelines: location based service providers. australia. bailey, stuart gm, and nadia caidi. 2005. "how much is too little? privacy and smart cards in hong kong and ontario." journal of information science 31 (5):354-364. barnes, s, j. 2003. "location-based services." e-service journal 2 (3):59-77. bellavista, paolo, axel kupper, and sumi helal. 2008. "location-based services: back to the future." pervasive computing, ieee 7 (2):85-89. bhattacherjee, anol. 2000. "acceptance of e-commerce services: the case of electronic brokerages." systems, man and cybernetics, part a: systems and humans, ieee transactions on 30 (4):411-420. bhattacherjee, anol, johan perols, and clive sanford. 2008. "information technology continuance: a theoretic extension and empirical test." journal of computer information systems 49 (1):17-26. bradley, joseph. 2012. "if we build it they will come? the technology acceptance model." in information systems theory, 19-36. springer. braw, elisabeth. 2014. "technology-based mobile health takes off in developing countries." the guardian accessed 18 january. https://www.theguardian.com/sustainablebusiness/technology-mobile-health-developing-countries. chin, natalie jun pei. 2012. "critical success factors of location-based services." chin, wynne w. 2010. "how to write up and report pls analyses." in handbook of partial least squares, 655-690. springer. dano, m. 2003. "location-based apps get wheels in western europe." rcr wireless news 22 (8):4. davis, fred d. 1989. "perceived usefulness, perceived ease of use, and user acceptance of information technology." mis quarterly 13 (3):319-340. dhar, subhankar, and upkar varshney. 2011. "challenges and business models for mobile location-based services and advertising." communications of the acm 54 (5):121-128. eberl, markus. 2010. "an application of pls in multi-group analysis: the need for differentiated corporate-level marketing in the mobile communications industry." in handbook of partial least squares, 487-514. springer. fusilier, marcelline, and subhash durlabhji. 2005. "an exploration of student internet use in india: the technology acceptance model and the theory of planned behaviour." campuswide information systems 22 (4):233-246. gable, eryn. 2015. "developing countries lead the way in mobile-health technologies." centre for health market innovations accessed 17 january. australasian journal of information systems hossain, hasan, chan & ahmed 2017, vol 21, research article predicting user acceptance and continuance behaviour towards lbs 18 http://healthmarketinnovations.org/blog/developing-countries-lead-way-mobilehealth-technologies. hair jr, joseph f, g tomas m hult, christian ringle, and marko sarstedt. 2017. a primer on partial least squares structural equation modeling (pls-sem): sage publications. henseler, jörg, and georg fassott. 2010. "testing moderating effects in pls path models: an illustration of available procedures." in handbook of partial least squares, 713-735. springer. henseler, jörg, christian m ringle, and marko sarstedt. 2016. "testing measurement invariance of composites using partial least squares." international marketing review 33 (3):405-431. henseler, jörg, christian ringle, and rudolf sinkovics. 2009. "the use of partial least squares path modeling in international marketing." advances in international marketing (aim) 20:277-320. hossain, mohammad alamgir. 2014. "exploring the perceived measures of privacy: rfid in public applications." australasian journal of information systems 18 (2). hossain, mohammad alamgir, and yogesh k dwivedi. 2014. "what improves citizens’ privacy perceptions toward rfid technology? a cross-country investigation using mixed method approach." international journal of information management 34 (6):711-719. hossain, mohammad alamgir, yogesh k. dwivedi, and niall c. piercy. 2015. "technology acceptance model (tam)." in the sage encyclopedia of quality and the service economy, edited by su mi dahlgaard-park, 806-807. thousand oaks: sage. hsu, meng-hsiang, and chao-min chiu. 2004. "internet self-efficacy and electronic service acceptance." decision support systems 38 (3):369-381. huang, jen-hung, yu-ru lin, and shu-ting chuang. 2007. "elucidating user behavior of mobile learning: a perspective of the extended technology acceptance model." the electronic library 25 (5):585-598. hurson, ali r, and xing gao. 2009. "location-based services." encyclopedia of information science and technology:2456-2461. igbaria, magid, tor guimaraes, and gordon b davis. 1995. "testing the determinants of microcomputer usage via a structural equation model." journal of management information systems 11 (4):87-114. jebeile, sam, and robert reeve. 2007. "explaining intention to use an information technology innovation: an empirical comparison of the perceived characteristics of innovating and technology acceptance." australasian journal of information systems 15 (1): 137-152 junglas, iris a, and richard t watson. 2008. "location-based services." communications of the acm 51 (3):65-69. junglas, iris, chon abraham, and richard t watson. 2008. "task-technology fit for mobile locatable information systems." decision support systems 45 (4):1046-1057. kargin, banu, nuri basoglu, and tugrul daim. 2008. "factors affecting the adoption of mobile services." international journal of services sciences 2 (1):29-52. kuo, ying-feng, and shieh-neng yen. 2009. "towards an understanding of the behavioral intention to use 3g mobile value-added services." computers in human behavior 25 (1):103-110. leung, louis, and ran wei. 2000. "more than just talk on the move: uses and gratifications of the cellular phone." journalism & mass communication quarterly 77 (2):308-320. australasian journal of information systems hossain, hasan, chan & ahmed 2017, vol 21, research article predicting user acceptance and continuance behaviour towards lbs 19 li, yuan. 2011. "empirical studies on online information privacy concerns: literature review and an integrative framework." communications of the association for information systems 28 (1): 453-496. lim, weng marc, and ding hooi ting. 2014. "consumer acceptance and continuance of online group buying." the journal of computer information systems 54 (3):87-96. limayem, moez, and sabine gabriele hirt. 2003. "force of habit and information systems usage: theory and initial validation." journal of the association for information systems 4 (1): 65-97. lindell, michael k, and david j whitney. 2001. "accounting for common method variance in cross-sectional research designs." journal of applied psychology 86 (1):114-121. lópez-nicolás, carolina, francisco j molina-castillo, and harry bouwman. 2008. "an assessment of advanced mobile services acceptance: contributions from tam and diffusion theory models." information & management 45 (6):359-364. mathieson, kieran, eileen peacock, and wynne w chin. 2001. "extending the technology acceptance model: the influence of perceived user resources." acm sigmis database 32 (3):86-112. nysveen, herbjørn, per e pedersen, and helge thorbjørnsen. 2005a. "explaining intention to use mobile chat services: moderating effects of gender." journal of consumer marketing 22 (5):247-256. nysveen, herbjørn, per e pedersen, and helge thorbjørnsen. 2005b. "intentions to use mobile services: antecedents and cross-service comparisons." journal of the academy of marketing science 33 (3):330-346. patel, n. 2004. "practical apps preferred over entertainment in cell phones, firm reports." electronic news (north america) 50 (10):1. pura, minna. 2005. "linking perceived value and loyalty in location-based mobile services." managing service quality 15 (6):509-538. rainio, a. 2003. navi program report on personal navigation. edited by liikennejaviestinta¨ministerio¨. helsinki. roy, sanjit kumar, ankit kesharwani, and shailendra singh bisht. 2012. "the impact of trust and perceived risk on internet banking adoption in india: an extension of technology acceptance model." international journal of bank marketing 30 (4):303-322. seyal, afzaal, and mohd noah abd rahman. 2003. "student use of the internet: an extension of tam in technical and vocational institutions in brunei darussalam." australasian journal of information systems 10 (2) :91-104. sharma, rajeev, philip yetton, and jeff crawford. 2009. "estimating the effect of common method variance: the method—method pair technique with an illustration from tam research." mis quarterly 33(3):473-490. steinfield, charles. 2004. "the development of location based services in mobile commerce." in e-life after the dot com bust, 177-197. springer. tan, khong sin, siong choy chong, binshan lin, and uchenna cyril eze. 2010. "internetbased ict adoption among smes: demographic versus benefits, barriers, and adoption intention." journal of enterprise information management 23 (1):27-55. taylor, shirley, and peter a todd. 1995. "understanding information technology usage: a test of competing models." information systems research 6 (2):144-176. the-economist. 2014. a summary of the liveability ranking and overview. thompson, ronald l, christopher a higgins, and jane m howell. 1991. "personal computing: toward a conceptual model of utilization." mis quarterly 15(1):125-143. australasian journal of information systems hossain, hasan, chan & ahmed 2017, vol 21, research article predicting user acceptance and continuance behaviour towards lbs 20 triandis, harry c. 1980. "values, attitudes, and interpersonal behavior." nebraska symposium on motivation, 1979. venkatesh, viswanath, and fred d davis. 2000. "a theoretical extension of the technology acceptance model: four longitudinal field studies." management science 46 (2):186204. wu, jen-her, and shu-ching wang. 2005. "what drives mobile commerce?: an empirical evaluation of the revised technology acceptance model." information & management 42 (5):719-729. xu, heng, and sumeet gupta. 2009. "the effects of privacy concerns and personal innovativeness on potential and experienced customers’ adoption of location-based services." electronic markets 19 (2-3):137-149. xu, heng, lih-bin oh, and hock-hai teo. 2009. "perceived effectiveness of text vs. multimedia location-based advertising messaging." international journal of mobile communications 7 (2):154-177. xu, heng, hock-hai teo, and bernard tan. 2005. "predicting the adoption of location-based services: the role of trust and perceived privacy risk." icis 2005 proceedings:71. yang, kenneth cc. 2005. "exploring factors affecting the adoption of mobile commerce in singapore." telematics and informatics 22 (3):257-277. yousafzai, shumaila y, gordon r foxall, and john g pallister. 2010. "explaining internet banking behavior: theory of reasoned action, theory of planned behavior, or technology acceptance model?" journal of applied social psychology 40 (5):1172-1202. zhou, tao. 2012. "examining location-based services usage from the perspectives of unified theory of acceptance and use of technology and privacy risk." journal of electronic commerce research 13 (2):135-144. zhou, tao. 2013. "examining continuous usage of location-based services from the perspective of perceived justice." information systems frontiers 15(1):141–150. zhou, tao. 2015. "understanding user adoption of location-based services from a dual perspective of enablers and inhibitors." information systems frontiers 17(2):413–422. copyright: © 2017 hossain, hasan, chan & ahmed. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems hossain, hasan, chan & ahmed 2017, vol 21, research article predicting user acceptance and continuance behaviour towards lbs 21 appendix a: measurement items perceived usefulness (pu) (1: strongly disagree; 5: strongly agree) pu1. lbs reduce my searching time pu2. lbs reduce my searching efforts pu3. lbs provide me access to useful information/services pu4. overall, lbs are useful perceived ease of use (peou) (1: strongly disagree; 5: strongly agree) peou1. my interaction with the lbs applications would be clear peou2. i find the lbs applications easy to use peou3. learning to use lbs applications is easy for me peou4: it would be easy for me to become skillful at using lbs applications perceived entertainment gratification (peg) (1: strongly disagree; 5: strongly agree) peg1. lbs applications are fun to use peg2. i find lbs applications entertaining peg3. lbs applications provide me with lots of enjoyment peg4. lbs an exciting way to find information and services perceived application quality (paq) (1: strongly disagree; 5: strongly agree) sq1. lbs applications protect my personal information sq2. lbs applications do not share information with others sq3. lbs applications use sufficient security measures sq4. lbs applications are flexible to meet variety of needs facilitating conditions (fcs) (1: strongly disagree; 5: strongly agree) fc1. the lbs platform is available for most of my queries fc2. reliability of lbs connection is reasonable fc3. response time of lbs applications is reasonable fc4. guidance is available to me for assistance with difficulties user attitude (1: strongly disagree; 5: strongly agree) att1. i feel happy to use lbs applications att2. i am comfortable using lbs applications att3. i like to explore through lbs applications att4. i like upgrade myself with the latest information/service (e.g. new restaurant) using lbs att5. i think using lbs is a good idea behavioural intention (bi) (1: strongly disagree; 5: strongly agree) int1. i intend to use lbs applications regularly int2. my intention is using lbs applications than use any alternative means (e.g., street directory) int3. i intend to increase using lbs australasian journal of information systems hossain, hasan, chan & ahmed 2017, vol 21, research article predicting user acceptance and continuance behaviour towards lbs 22 int4. i intend to recommend lbs to others actual use (au) au1. number of lbs applications i use per day: 0 | 1-3 | 4-6 | 7-9 | other (specify): _____ au2. number of times i currently use lbs per day: 0 | 1-3 | 4-6 | 7-9 | other (specify): _____ au3: percentage of location-based tasks that you currently process using lbs: 0% | 1-20% | 21-40% | 41-60% | other (specify): _____ appendix b: result of the multi-group analysis australian sample (n=103) bangladeshi sample (n=110) hypotheses path coefficient sd t-values path coefficient sd t-values permutation p-values pu → ua 0.318*** 0.090 3.529 0.238* 0.120 1.989 0.827ns peou → ua 0.01ns 0.121 0.080 0.104 ns 0.131 0.792 0.899ns peou → pu 0.456*** 0.094 4.860 0.616*** 0.750 8.201 0.106ns peg → ua 0.264* 0.107 2.475 0.046 ns 0.140 0.328 0.407ns paq → ua 0.203* 0.102 2.183 0.281* 0.142 1.978 0.912ns ua →bi 0.503*** 0.083 6.090 0.475*** 0.083 5.687 0.758ns bi → au 0.561*** 0.070 8.071 0.704*** 0.058 12.081 0.248ns note. sd, standard deviation; significance level *p<0.05, **p<0.01, ***p<0.001; ns, not-significant paper title (paper title style) australasian journal of information systems gupta, tan, ee & phang 2018, vol 22, research note resource-based view of information systems 1 resource-based view of information systems: sustainable and transient competitive advantage perspectives gaurav gupta indian institute of management, calcutta gauravg13@email.iimcal.ac.in kevin tee liang tan swinburne university of technology yaw seng ee swinburne university of technology cynthia su chen phang swinburne university of technology abstract the resource-based view (rbv), or resource-based theory, is one of the oldest and most influential theories in the field of information systems. this paper contends that it is timely to revisit, reflect on, and reposition rbv to ensure its continued disciplinary relevance and progress. in doing so, this paper (i) provides a succinct and sharp evaluation of the conventional rbv of information systems that firms use to establish sustainable competitive advantage, and (ii) makes an original contribution by introducing a contemporary rbv of information systems that firms can use to establish transient competitive advantage. both these contributions should advance the current and future understanding of information systems as (a) an internal firm resource, (b) a source of competitive advantage, and (c) a driver of firm performance. keywords: firm performance, firm resource, information systems, resource-based view (rbv), sustainable competitive advantage, transient competitive advantage. 1 introduction the resource-based view (rbv), or resource-based theory, is a classical and influential theory in the field of information systems. the theory, derived from edith penrose’s (1959) theory of firm growth, was introduced by birger wernerfelt (1984) and popularized by jay barney (1991). since then, the seminal articles by the two rbv theorists—wernerfelt and barney— have been cited more than 85,000 times (as of december 20, 2017; see google scholar). while most studies that use rbv are empirical, some scholars have chosen to review and propose ways forward for using rbv in future studies. the present paper is similar to the latter in the sense that it reviews the extant literature on the rbv of information systems. at the same time, it differs from the latter as it adopts and takes advantage of a resource-based approach by combining insights from existing reviews and recent studies on the rbv of information systems. it does so in order to accelerate its endeavour to offer an overview of the theory’s usage in the field and to chart new directions for advancing the understanding of information systems from the rbv perspective. australasian journal of information systems gupta, tan, ee & phang 2018, vol 22, research note resource-based view of information systems 2 2 rbv the essence and logic of rbv can be clearly understood when broken down and organized as follows: a. firms possess heterogeneous resources. b. firm resources can be used to conceive, choose, and implement firm strategies. c. firm strategies are likely to be different, if not similar, but not identical. d. firm strategies contribute to and account for differences in firm performance. e. firm performance is predicated on firm resources. f. firm resources that are valuable, rare, inimitable, non-substitutable, durable, appropriable, and mobile (among others) can create and sustain competitive advantages. g. firms with competitive advantages can enjoy myriad benefits, such as premium and rent yields (amit & schoemaker, 1993; barney, 1991; collis & montgomery, 1995; wade & hulland, 2004; wernerfelt, 1984). 3 current rbv of information systems information systems represent both a product and a discipline. this duality in form can be distinguished using the rbv. specifically, an information system is the resource of focus when it takes a product form—that is, the interrelated components of hardware, software, data, people, and process—in studies situated in the information systems discipline. by drawing on existing reviews and recent studies of the rbv of information systems, this paper provides a general overview of the discipline’s current understanding of the rbv of information systems, which takes the form of three major subsections, as follows (see figure 1). australasian journal of information systems gupta, tan, ee & phang 2018, vol 22, research article resource-based view of information systems 3 figure 1 current and future rbv of information systems australasian journal of information systems gupta, tan, ee & phang 2018, vol 22, research article resource-based view of information systems 4 3.1 information systems as an internal firm resource firm resources can be internal (or inside-out) and external (or outside-in) to the firm. internal resources are assets owned and controlled by the firm, such as financial, human, physical, and technological resources; whereas external resources are assets that may be earned and controlled, to a certain extent depending on various factors like industry attractiveness and structural autonomy, but not necessarily owned by the firm, such as customers, competitors, and suppliers, among others (anggraeni, 2014; dierickx & cool, 1989; hulland et al., 2007; wade & hulland, 2004). this suggests that information systems are an internal firm resource because the acquisition, integration, and usage of its interrelated components are owned and controlled by firms, be it in the form of access (e.g. lease, subscription), progressive ownership (e.g. hire purchase), or total ownership. 3.2 information systems as a strategic and sustainable competitive advantage competitive advantage is the advantage that a firm (a) has over its competitors, (b) develops using its resources, and (c) uses to drive superior performance (barney, 1995; peteraf, 1993). generally, information systems, as an internal firm resource, are considered to be more powerful than an external firm resource in establishing competitive advantage (mahoney & pandian, 1992; ravichandran & lertwongsatien, 2005). this may be attributed to the chain (or multiplier, spillover) effect that comes from having an information advantage, as an information advantage facilitates the creation of other competitive advantages, such as cost and differentiation advantages (lubit, 2001; porter & millar, 1985). more importantly, the sustainability of information systems as a competitive advantage is difficult but rewarding. it allows firms to detect and respond to market opportunities and threats, such as to counter and protect against resource imitation, transfer, or substitution (wade & hulland, 2004). the keys to sustaining the competitive advantage of information systems can come from greater alignment between alternative (e.g. business) and information system strategies (choe, 2003; liang & you, 2009), complementary synergies with other internal and external firm resources (hulland et al., 2007; ravichandran & lertwongsatien, 2005), innovation (grant, 1999; shin, 2006), which may be incremental, or even better, ground-breaking (coulthard & keller, 2016; lim, 2016), and systematic and timely upgrades of organizational (e.g. expertise, skills) and functional capabilities (e.g. hardware, software) (kraaijenbrink et al., 2010; liang & you, 2009; ravichandran & lertwongsatien, 2005; seddon, 2014). 3.3 information systems as a driver of firm performance the concept of firm performance is simple—it relates to how well the firm performs (bi et al., 2015; cosic et al., 2015; someh & shanks, 2016). however, the extent to which information systems can effectively play a role as a firm resource that drives firm performance is rather complex due to its inconsistency, as demonstrated by wade and hulland (2004) in their seminal review and by liang and you (2009) in their meta-analysis of the rbv of information systems. specifically, in some cases, direct effects are reported, where the relationship between information systems and firm performance tends to be positively rather than negatively related; in other cases, contingent effects have been reported, where information systems ought to interact with other internal and external firm resources in order to effectively drive firm performance, especially in the long run (e.g. organizational commitment and investment; environmental complexity, munificence, and turbulence). such effects, according to wade and hulland, are predicated upon clemons and row’s (1991) “strategic necessity hypothesis,” australasian journal of information systems gupta, tan, ee & phang 2018, vol 22, research article resource-based view of information systems 5 which suggests that whether or not information systems will effectively play a role in driving firm performance, and the extent of performance as a result of that drive, is dependent on the strategic role of information systems in relation to the firm and its competitors. in other words, information systems may be crucial for a firm’s operations, but that does not imply that this firm resource is automatically translated into a competitive advantage, especially when firms face competitors whose strategies are substantially different from theirs. nevertheless, the means to sustain firm performance through information systems is similar to that dedicated to sustaining competitive advantages (e.g. innovation, upgrades of capabilities; cooper & molla, 2014; liang & you, 2009; ravichandran & lertwongsatien, 2005). 4 future rbv of information systems notwithstanding the extant contributions of the existing rbv of information systems, specifically the theorization of information systems as (a) an internal firm resource, (b) a strategic and sustainable competitive advantage, and (c) a driver of firm performance, this paper asserts that the continued relevance and usefulness of rbv for information systems research will require a fresh injection of thought-provoking ideas and further extensions to existing evaluation criteria to accommodate alternative and future rbvs of information systems. the rationale behind this contention is that the omnipresence and proliferation of ground-breaking technologies spearheaded by technological revolutions (e.g. artificial intelligence, blockchain, crowdsourcing, and the internet of things) may erode the sustainability of existing competitive advantages held by firms. this may be seen especially in firms that are innovative but not agile enough to detect and respond to change, and therefore they fail to build and sustain a rich and dynamic portfolio of competitive advantages (mcgrath, 2013). such propositions herein are supported by the marshallian quasi-monopoly and equilibrium argument that competitive advantages will continue to emerge but eventually become obsolete over time. given the above, firms operating in today’s competitive environments can expect to encounter and learn about the diminishing luxury of time and stability, where change is the only constant. in that process, firms should recognize and work to bring in innovations in information systems that can provide them with a portfolio of transient competitive advantages (instead of sustainable competitive advantages). in that sense, firms that are information-intensive will seek to develop and grow their information systems with different competitive resource attributes at two stages: introductory (short run) and maturity (long run) (see figure 2). high information intensity low information intensity transient competitive advantage in the short run (introductory) critical resource attribute: timeliness critical resource attribute: completeness transient competitive advantage in the long run (maturity) critical resource attribute: accuracy critical resource attribute: consistency figure 2 t-a-c-c framework for information intensity and transient competitive advantage australasian journal of information systems gupta, tan, ee & phang 2018, vol 22, research article resource-based view of information systems 6 in the short run, transient competitive advantages can be acquired in two ways, depending on the information intensity of the firm. specifically, high-information-intensity firms are reliant on the speed of information to gain transient competitive advantages, which occurs when information systems deliver information that excels in timeliness (e.g. for bitcoin and forex traders). in contrast, low-information-intensity firms place greater emphasis on information systems to furbish decision makers with information that excels in completeness (e.g. for marketing intelligence and research agencies), and in doing so, establish transient competitive advantages. in the long run, existing firms strengthen their competitive positioning of existing competitive advantages, and simultaneously develop and pursue new avenues and opportunities for competition. highly information-intensive firms will create opportunities to acquire additional transient competitive advantages, which can be carried out by channelling other internal and external resources in order to further improve the accuracy of the information generated by their information systems (on top of timeliness and many of the other critical resource attributes)—for example, fake news is more of a problem for mature social media than for newer ones that inherently lack credibility due to their newness. finally, lowinformation-intensity firms will expand their portfolio of transient competitive advantages by investing their resources to strengthen the consistency of information produced by their information systems (on top of completeness and many of the other critical resource attributes). consistency ensures reproducibility, establishes standardization, and offers the flexibility to expand that anchors on solid knowledge foundations. 5 conclusion the rbv of information systems is entrenched in the idea that information systems are (i) an internal firm resource, (ii) a source of competitive advantage, and (iii) a driver of firm performance. the conventional rbv of information systems can be applied to comprehend and build sustainable competitive advantages for firms that operate in relatively stable environments, whereas the contemporary rbv of information systems can be used to understand and develop transient competitive advantages for firms that operate in relatively agile and dynamic environments. more importantly, future research on the rbv in information systems should consider exploring new and disruptive technologies to enrich our understanding of and the theory’s value to information systems. researchers should especially focus on those technologies that challenge pre-existing notions of the rbv. for instance, big data, cloud computing, crowdsourcing platforms, and the internet of things have all proven to be essential in generating competitive advantages. yet, they challenge the conventional notions of both the rbv and information systems. specifically, big data is a recognized source of competitive advantage. yet, it fails to adhere to the value, rarity, inimitability, and non-substitutable (vrin) assumptions of the rbv theory (braganza et al., 2017). cloud resource, though developed to possess unique competencies, is neither rare nor inimitable (mitra et al., 2017). the internet of things and crowdsourcing, on the other hand, are highly reliant on external sources of information (santaro et al., 2017; venanzi et al., 2016). meanwhile, big data will also concede on this dependence, acknowledging that it’s roles will continue to morph as a long-term investment (sedera et al., 2016). australasian journal of information systems gupta, tan, ee & phang 2018, vol 22, research article resource-based view of information systems 7 nevertheless, these ground-breaking sources of information can further enrich the contemporary understanding of the rbv with a common recognition of the need for open innovation regarding both strong internal systems and external sources of knowledge (sedera et al., 2016; santoro et al., 2017). any independent reliance would lead to limited success and, potentially, even to an erosion of any competitive advantages that existing information systems may have once yielded (braganza et al., 2017). ultimately, the impact that new technologies have on a firm’s information systems and performances are largely dependent on their internal competencies (e.g. managerial and staff capabilities) in adapting to these technological changes within a dynamic business environment in order to continuously develop transient competitive advantages (wamba et al., 2017). thus, exploring how contemporary disruptions affect firm’s resources, competitive advantages, and performances will provide valuable insight into the future of the rbv of information systems. to this end, the authors hope that the insights herein will stimulate greater research on rbv dedicated to advancing and supporting the current and future understanding of information systems. acknowledgement the authors wish to convey their deepest appreciation to weng marc lim, the editors, and the anonymous reviewers for the constructive feedback to improve the quality of the paper. references amit, r., & schoemaker, p. j. (1993). strategic assets and organizational rent. strategic management journal, 14(1), 33-46. doi: https://doi.org/10.1002/smj.4250140105 anggraeni, e. (2014). the impact of internal and external resources, and strategic actions in business networks on firm performance in the software industry. delft, netherlands: delft university of technology doi: https://doi.org/10.4233/uuid:906fbade-bd8c-487d-a6b58be924478b4a barney, j. (1991). firm resources and sustained competitive advantage. journal of management, 17(1), 99-120. doi: https://doi.org/10.1177/014920639101700108 barney, j. b. (1995). looking inside for competitive advantage. academy of management executive, 9(4), 49-61. doi: https://doi.org/10.5465/ame.1995.9512032192 bi, r., davison, r. m., & smyrnios, k. x. (2015). it and fast growth small-to-medium enterprise performance: an empirical study in australia. australasian journal of information systems, 19. doi: https://doi.org/10.3127/ajis.v19i0.1012 braganza, a., brooks, l., nepelski, d., ali, m., & moro, r. (2017). resource management in big data initiatives: processes and dynamic capabilities. journal of business research, 70, 328-337. doi: https://doi.org/10.1016/j.jbusres.2016.08.006 choe, j. m. (2003). the effect of environmental uncertainty and strategic applications of is on a firm's performance. information & management, 40(4), 257-268. doi: https://doi.org/10.1016/s0378-7206(02)00008-3 clemons, e. k., & row, m. c. (1991). sustaining it advantage: the role of structural differences. mis quarterly, 15(3), 275-292. doi: https://doi.org/10.2307/249639 australasian journal of information systems gupta, tan, ee & phang 2018, vol 22, research article resource-based view of information systems 8 collis, d., & montgomery, c. a. (1995). competing on resources: strategy in the 1990s. harvard business review, 73 (july–august), 118-128. cooper, v. a., & molla, a. (2014). absorptive capacity and contextual factors that influence green it assimilation. australasian journal of information systems, 18(3). doi: https://doi.org/10.3127/ajis.v18i3.1099 cosic, r., shanks, g., & maynard, s. b. (2015). a business analytics capability framework. australasian journal of information systems, 19. https://doi.org/10.3127/ajis.v19i0.1150 coulthard, d., & keller, s. (2016). publication anxiety, quality, and journal rankings: researcher views. australasian journal of information systems, 20. doi: https://doi.org/10.3127/ajis.v20i0.1262 dierickx, i., & cool, k. (1989). asset stock accumulation and sustainability of competitive advantage. management science, 35(12), 1504-1511. doi: https://doi.org/10.1287/mnsc.35.12.1504 grant, r. m. (1999). the resource-based theory of competitive advantage: implications for strategy formulation. california management review, 33(3), 114-135. doi: https://doi.org/10.2307/41166664 hulland, j., wade, m. r., & antia, k. d. (2007). the impact of capabilities and prior investments on online channel commitment and performance. journal of management information systems, 23(4), 109-142. doi: https://doi.org/10.2753/mis0742-1222230406 kraaijenbrink, j., spender, j. c., & groen, a. j. (2010). the resource-based view: a review and assessment of its critiques. journal of management, 36(1), 349-372. doi: https://doi.org/10.1177/0149206309350775 liang, t.-p., & you, j. j. (2009). resource-based view in information systems research: a meta-analysis. proceedings of pacific asia conference on information systems 2009. available at doi: https://aisel.aisnet.org/pacis2009/72 lim, w. m. (2016). a post publication review of "publication anxiety, quality, and journal rankings: researcher views". australasian journal of information systems, 20. doi: https://doi.org/10.3127/ajis.v20i0.1487 lubit, r. (2001). the keys to sustainable competitive advantage. organizational dynamics, 29(3), 164-178. https://doi.org/10.1016/s0090-2616(01)00026-2 mahoney, j. t., & pandian, j. r. (1992). the resource-based view within the conversation of strategic management. strategic management journal, 13(5), 363-380. doi: https://doi.org/10.1002/smj.4250130505 mcgrath, r. g. (2013). transient advantage. harvard business review, 91(6), 62-70. mitra, a., o'regan, n., & sarpong, d. (2017). cloud resource adaptation: a resource based perspective on value creation for corporate growth. technological forecasting and social change. doi: https://doi.org/10.1016/j.techfore.2017.08.012 penrose, e. t. (1959). the theory of the growth of the firm. new york: wiley. peteraf, m. a. (1993). the cornerstones of competitive advantage: a resource-based view. strategic management journal, 14(3), 179-191. https://doi.org/10.1002/smj.4250140303 australasian journal of information systems gupta, tan, ee & phang 2018, vol 22, research article resource-based view of information systems 9 porter, m. e., & millar, v. a. (1985). how information gives you competitive advantage. harvard business review, 63(july–august), 149–160. ravichandran, t., & lertwongsatien, c. (2005). effect of information systems resources and capabilities on firm performance: a resource-based perspective. journal of management information systems, 21(4), 237-276. doi: https://doi.org/10.1080/07421222.2005.11045820 santoro, g., vrontis, d., thrassou, a., & dezi, l. (2017). the internet of things: building a knowledge management system for open innovation and knowledge management capacity. technological forecasting and social change. doi: https://doi.org/10.1016/j.techfore.2017.02.034 sedera, d., lokuge, s., grover, v., sarker, s., & sarker, s. (2016). innovating with enterprise systems and digital platforms: a contingent resource-based theory view. information & management, 53(3), 366-379. doi: https://doi.org/10.1016/j.im.2016.01.001 seddon, p. b. (2014). implications for strategic is research of the resource-based theory of the firm: a reflection. journal of strategic information systems, 23(4), 257-269. doi: https://doi.org/10.1016/j.jsis.2014.11.001 someh, i. a., & shanks, g. (2016). emergence of business value from complementary interactions between informational and transactional it systems. australasian journal of information systems, 20. doi: https://dx.doi.org/10.3127/ajis.v20i0.1122 venanzi, m., guiver, j., kohli, p., & jennings, n. r. (2016). time-sensitive bayesian information aggregation for crowdsourcing systems. journal of artificial intelligence research, 56, 517-545. wade, m., & hulland, j. (2004). the resource-based view and information systems research: review, extension, and suggestions for future research. mis quarterly, 28(1), 107-142. doi: https://doi.org/10.2307/25148626 wamba, s. f., gunasekaran, a., akter, s., ren, s. j. f., dubey, r., & childe, s. j. (2017). big data analytics and firm performance: effects of dynamic capabilities. journal of business research, 70, 356-365. doi: https://doi.org/10.1016/j.jbusres.2016.08.009 wernerfelt, b. (1984). a resource-based view of the firm. strategic management journal, 5(2), 171-180. doi: https://doi.org/10.1002/smj.4250050207 copyright: © 2018 gupta, tan, ee & phang. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems gupta, tan, ee & phang 2018, vol 22, research article resource-based view of information systems 10 1 introduction 2 rbv 3 current rbv of information systems 3.1 information systems as an internal firm resource 3.2 information systems as a strategic and sustainable competitive advantage 3.3 information systems as a driver of firm performance 4 future rbv of information systems 5 conclusion paper title (paper title style) australasian journal of information systems gregor et al. 2023, vol 27, research article indigenous knowledge-sharing 1 indigenous knowledge-sharing interventions in australia and the use of information and communication technology: a scoping review shirley gregor australian national university (anu), canberra, australia shirley.gregor@anu.edu.au maya gunawardena university of canberra, australia ahmed imran university of canberra, australia safiya okai-ugbaje australian national university (anu), canberra, australia catherine page jeffery university of sydney, australia rhonda wilson university of newcastle, australia abstract indigenous peoples in australia are subject to significant disadvantages both socioeconomically and in health, education and service provision. knowledge-sharing interventions, including those with an information and communication technology (ict) base, have the potential to address these challenges. interventions occur against a background of an ancient culture with distinctive ways of knowing and doing, including storytelling, art and performance. this study documents the results of a scoping review of interventions that have been undertaken in this context. it considers the outcomes of these interventions, the extent to which indigenous ways of knowing were accounted for and whether ict was involved. our review of the peer-reviewed literature located two prior reviews and seven primary studies. all of the primary studies were about health interventions; of these, all those that reported positive outcomes only had incorporated indigenous ways of knowing, some in innovative ways. only two studies used ict as their main vehicle. this article provides a base for further work by documenting the current status of the field and identifying gaps, such as the scarcity of non-health and ict-based studies. the cases identified provide useful insights for those with an interest in developing future initiatives. keywords: indigenous knowledge, knowledge transfer, knowledge sharing, aboriginal, information and communication technology, intervention. 1 introduction indigenous peoples make up six per cent of the world population, estimated at 476 million people (world bank, 2022). they are holders of traditional knowledge linked to land, natural resources, ancestry and culture (world bank, 2022). many indigenous peoples have endured eviction from their land and relocation through colonization and, currently, indigenous people account for 19% of the extreme poor globally (world bank, 2022). as with minority groups australasian journal of information systems gregor et al. 2023, vol 27, research article indigenous knowledge-sharing 2 generally, indigenous peoples face many challenges, particularly from institutional racism affecting access to and equitable distribution of resources and services, leading to socioeconomic, health, educational, vocational and judicial disadvantages (elias & paradies, 2021). the position of indigenous australians1 is, in many respects, similar to indigenous peoples globally, with the same disadvantages further amplified by the geographical scale of australia that means that some communities are very remote (wilson & waqanaviti, 2021). the council of australian governments in 2007 identified substantial disadvantages experienced by indigenous australians in many areas, including life expectancy, children’s mortality, education and employment. the australian government is continuing to address these challenges through the “closing the gap” initiative (australian government, 2020). the rise of networked technologies such as the internet presents new challenges for equitable access to and distribution of digital capacity and resources for minority groups, including indigenous people (hunter & radoll, 2020). on the one hand, digital communications offer additional capacity to improve knowledge-sharing and information-brokerage; on the other hand, digital resources and infrastructure are not consistently available within remote communities and can also present certain risks to vulnerable members of the community. further, the use of ict and non-indigenous approaches to knowledge-sharing incorporating ict may not align with traditional indigenous ways of sharing knowledge, rendering such efforts to address pressing social challenges ineffective. it has been argued, for example, that ict design methods in australia should follow a paradigm that encompasses the place, relationships, agency and indigenous ways of knowing, disrupting structures of learning, design and research based on colonialism (ibinarriaga & martin, 2021). some on-the-ground studies indicate that knowledge-sharing strategies (interventions)2 can assist indigenous people to address contemporary challenges in some circumstances. one such example is a program aimed at developing suicide prevention expertise within australian aboriginal communities using ict and multimedia approaches (gorman & ward, 2008). however, the challenge for those intending to manage, design, develop or utilize knowledge-sharing interventions with indigenous australians, with or without ict, is that there are very few high-level research insights documenting the nature of such interventions and their efficacy. some prior reviews exist (see, for example, brusse et al. (2014)) but they are limited to specific areas or technologies. this challenge had relevance for this paper’s authors, as our research team is engaged in a wider project in which we are investigating issues and possible interventions employing ict to address knowledge gaps in relation to children’s online safety amongst australian 1 the focus of the paper is on indigenous people in australia. the terms “indigenous people in australia” “indigenous australians” and “indigenous” and “aboriginal” without qualification denote these people and are used interchangeably. further clarification is provided when the reference is to indigenous peoples globally. 2 an intervention is defined as “an action or provision of a service to produce an outcome or modify a situation” (dudgeon, luxford & cox, 2016, p. x). the term “intervention” should not be seen as having any negative connotation (e.g. as shown also in brusse at al., 2014). the indigenous community may itself institute and carry out the intervention. alternative terms used are “approach”, “program”, “project” or “implemented strategy”. australasian journal of information systems gregor et al. 2023, vol 27, research article indigenous knowledge-sharing 3 indigenous parents and caregivers3. one of the paper’s authors is indigenous and the project is guided by an indigenous reference group. against this background, the current article documents a scoping review that addresses the following research questions: (1) what interventions have been developed, in what circumstances and with what outcomes, to help facilitate knowledge-sharing with indigenous communities in australia? (2) how did these interventions take account of indigenous ways of knowing, being and doing, if at all? (3) how was ict involved in these interventions, if at all? the literature review was limited to interventions focusing on indigenous australian populations. indigenous australian culture and history have commonalities with indigenous populations elsewhere, but differences do exist and so a restricted focus is appropriate. studies were included in the review if they focused on a knowledge-sharing intervention, whether or not the intervention was ict based. our reasoning was that knowledge about non-ict-based interventions is useful in its own right, but also that these interventions could provide evidence and ideas for future ict-based projects. further in some cases, ict might play a subsidiary role in an intervention without being the primary vehicle. the scope of the review was limited to peer-reviewed literature. our work’s main contribution is a review of the evidence base for knowledge-sharing interventions involving indigenous australians. thus, the work identifies potentially effective strategies that can be embedded in ict-based innovations. further, the research has the potential to assist with the design and development of effective interventions that take indigenous traditions into account. the review provides a base for further cumulative knowledge consolidation in the area of knowledge-sharing with indigenous communities. the paper proceeds by first giving an overview of the situation faced by indigenous australians, their knowledge-sharing practices and use of ict and different approaches proposed for developing interventions in this context. it then describes our scoping review approach and presents results followed by a discussion. 2 indigenous australians and knowledge-sharing 2.1 overview indigenous australians, comprised of aboriginal and torres strait islander peoples, accounted for 3.2 per cent (812,300) of the australian population of 25,500,000 in 2021 (australian bureau of statistics, 2022). ninety-one per cent of the indigenous population identified as aboriginal, about 5 per cent as torres strait islander, and the others both aboriginal and torres strait islander (australian institute of health and welfare, 2021). australia’s indigenous people are said to be the world’s oldest civilisation, with 65,000 years of sustained cultural traditions (althaus, 2020). indigenous people live in all parts of the country, including in major cities and remote and desert areas. the states of new south wales and queensland have sizeable indigenous populations, as well as the northern territory where indigenous people account for one-third of the territory’s population (australian institute of health and welfare, 2021). 3 relevant issues and research by the australian government are described in esafety commissioner (2022). australasian journal of information systems gregor et al. 2023, vol 27, research article indigenous knowledge-sharing 4 the sorry history of colonization for indigenous people in australia since 1788 is documented in texts such as that produced by the australian institute of aboriginal and torres strait islander studies (aiatsis, 2021). such works describe the dispossession of land, conflicts, restrictions on indigenous people through protection acts, removal of children from their parents and other discriminatory treatment. in some regions, indigenous people were moved to specially created reserves, taking them away from their country, food, sacred sites and families. despite this history, indigenous people celebrate many achievements, including but not limited to music, theatre and dance, visual art, film and television, literature, media and sport. although the effects of colonization on indigenous australians are many, here we focus on aspects that appear most relevant to the current work, particularly those concerning indigenous culture and ways of knowing, being and doing. while indigenous australians have many similarities, including a shared colonist story, each of many groups has a unique cultural identity with distinct histories, traditions and languages (whitehouse et al., 2014). of the 250 languages at the time of european settlement, only 150 have remained post-colonization (applebee, 2020). culture and land are integral to indigenous identity. hence, for aboriginal and torres strait islander peoples, land is not a commodity or asset that is owned but an entity that encompasses nature, animals and plants, with which they have a deep spiritual, physical, social and cultural connection (whitehouse et al., 2014). country in this instance is not just a place of origin or geographical area, but an interconnection of land and sea areas that epitomises the existence, ways of life and spirituality of indigenous people (whitehouse et al., 2014). for indigenous people, “country is family, culture, identity. country is self” (kwaymullina, 2005, p 12), and kinship reflects a deep sense of connection to country, family and community (bourke et al., 1998). it is believed that indigenous peoples’ relationships with land and country and undertaking traditional rituals is why the indigenous culture and identity have survived over the years despite colonizing forces (applebee, 2020; schillhorn, 1999). these cultures have continued to be passed on from generation to generation through unique indigenous ways of knowing. older people are seen as custodians of culture in indigenous communities. being old, however, does not automatically ensure status as an elder, with the position coming from respect for someone’s moral authority and cultural knowledge. elders play a strong role in the decision-making of a community. people given the status of elder may be referred to as “aunty” or “uncle” as a sign of respect (behrendt, 2021). further, specific roles in the community may be assigned to women and men, especially with respect to cultural stories and spiritual life (behrendt, 2021). sharing based on reciprocity is a strong value within indigenous communities. people are expected to share resources and food and help others when needed and will expect help in return when in need themselves (behrendt, 2021). indigenous people today live in diverse circumstances across australia, from remote communities to urban areas. distinctive cultural values are still evident, including living communally, knowing where you are from, restrictions on who can know what, an equal place for women, culture dependent on storytelling, and connecting to the environment (behrendt, 2021). 2.2 indigenous ways of knowing and communicating australian indigenous knowledge is considered sacred – a spirit that transforms the holder. the person transferring knowledge does so not as an individual but as part of a process in which the holder has a responsibility rather than a right to ownership (brady, 1997). in australasian journal of information systems gregor et al. 2023, vol 27, research article indigenous knowledge-sharing 5 scientific studies, it is common practice to have a definition of the concepts, attributes or terminologies used to provide the scope and context of the study. indigenous scholars caution against a definition for ‘indigenous knowledge’ as, although it is a widely accepted term amongst aboriginal and torres strait islander peoples (weiss et al., 2013), indigenous people have diverse groups with distinct cultures, traditions and ceremonies that constrain definition. nonetheless, berkes et al.’s (2000) analysis of traditional indigenous knowledge in the international literature has been widely accepted (weiss et al., 2013) and describes indigenous knowledge as comprised of locally situated factual or empirical management systems, norms, values and beliefs held by communities that are shared, transformed and passed on from generation to generation. intergenerational transfer of knowledge and cultural traditions occurs via several means, including orally through stories. storytelling is a long-standing tradition and knowledge transfer strategy of indigenous people. iseke (2013) describes it as an invaluable tool that sustains indigenous communities worldwide, nurtures relationships, and validates people’s lived experiences. storytelling can take several forms and includes casual conversations during daily interactions and while undertaking activities such as farming, hunting, food-sharing and traditional ceremonies with elders, where children are taught with love and recognised for their achievements (blenkinsop, 2017). in australia, some traditional healers report the use of daily early-morning stories (alpiri) as a mechanism to prepare the mind for the day ahead and to organise the tasks to be achieved (ngaanyatarra pitjantjatjara yankunytjatjara women’s council aboriginal corporation, 2021). indigenous storytelling also takes more structured conversational approaches, such as engoori, dadirri and yarning. engoori is a strength-based conversational approach with roots found in the mithaka-tjimpa people of south-west queensland, traditionally used as a diplomatic approach to resolving misunderstandings between conflicting ideologies and groups (spillman, 2018). the engoori process now extends to other areas, including as an epistemological framework and research methodology, a deep learning approach in the classroom and an organisational tool to strengthen processes (waller et al., 2018). dadirri is a form of deep listening to the stories told by elders which originated from the ngangikurungkurr people of the daly river in the northern territory (coombes & ryder, 2020). yarning is widely used by indigenous people from around the world to build respectful relationships and to preserve and pass on traditional knowledge (bessarab & ng’andu, 2010; osmond & phillips, 2019). these conversational storytelling techniques have been extended to research methods as they facilitate a respectful and honest interaction between researchers and participants, enabling researchers to learn about the issues being discussed (applebee, 2020; coombes & ryder, 2020), while breaking formalities and creating a conducive interaction atmosphere of equal partnership where researchers and participants are both knowers and learners (bessarab & ng’andu, 2010). indigenous visual art and crafts are symbolic and creative means of communication that have been transmitted over generations (ngaanyatarra pitjantjatjara yankunytjatjara women’s council aboriginal corporation, 2021). these symbols hold historical, spiritual and cultural significance (cameron, 2015). the arts and the often collaborative act of making these works demonstrate how indigenous people value thinking, dreaming and imagining in a slow, respectful and careful manner as they seek to communicate deeply spiritual and practical lore and cultural messages (ngaanyatarra pitjantjatjara yankunytjatjara women’s council aboriginal corporation, 2021). thus, semiotics play a significant role in indigenous ways of communication and carry deep meanings, such as the relationships between people, animals, australasian journal of information systems gregor et al. 2023, vol 27, research article indigenous knowledge-sharing 6 the earth, the moon and the sun. some authors show that indigenous dot art is believed to be used in healing and ceremonial processes (cameron, 2015). however, colonization has led to diminished respect for traditional knowledge (ngaanyatarra pitjantjatjara yankunytjatjara women’s council aboriginal corporation, 2021) and misrepresentations of indigenous art are common. for example, sim (1966) found that circles on the rocks in dharug country in nsw did not represent the sun and the moon as misinterpreted by western artists but, rather, they depict the sacred connections between people and places. the misrepresentation of indigenous knowledge continues to be problematic and is of concern to many indigenous people (ngaanyatarra pitjantjatjara yankunytjatjara women’s council aboriginal corporation, 2021). music and dance are powerful mediums for expressing indigenous cultural identity and maintaining a connection to the home country. the term ‘corroboree’ implies the participatory public performance of songs and dances, an equivalent to modern theatre. however, it is much more than just a celebration or entertainment, where songwriting, singing, and dance continue to build pride on a broad communal level (fuary, 1993). aboriginal people claim that dance rituals function to bring dreaming power and presence into the time and places and bodies of the performers, according to anthropologists (e.g. ellis, 1985). indigenous dance performances can include performers wearing ochre faces and body paint to portray a connection to a totem, family or other cultural elements. costumes or dress are created using various types of australian animal fur, feathers combined with coloured fabric, traditional tools and small branches from local trees and plants. dances sometimes involve a ceremonial smoking ceremony to evoke cleansing (ngaanyatarra pitjantjatjara yankunytjatjara women’s council aboriginal corporation, 2021). dances can tell stories using mimicking of various animal motions and elements of nature or depict everyday activities. didgeridoo, yidaki, rhythm, clapping sticks and a singer are frequently used as dance accompaniments. henry (2000) explores “dance as a form of corporeal politics” (p.322) for indigenous australians where people from cultural groups perform in the public arena to express their opposition to the state’s bureaucracy. in summary, indigenous ways of knowing and communicating are diverse and expressed in many creative formats unlike westernised communication of knowledge. we have been able to provide only a brief overview here and recognise that a richer description is beyond the scope of this manuscript. 2.3 indigenous australians’ use of information and communications technology several scholars report that indigenous australians are enthusiastic users of digital technologies and use them in diverse ways for transferring knowledge, amongst other activities (brady & dyson, 2015; du et al., 2015; dyson, 2004). despite considerable barriers to ict adoption such as cost, lack of adequate infrastructure and low levels of digital literacy (du & haines, 2017; radoll, 2010), the use of ict has become an important everyday activity for indigenous australians (dyson, 2015) across work, educational and social environments (radoll, 2010). research shows that indigenous people across the globe are typically early adopters of digital technology (carlson & frazer, 2018; latimore et al., 2017) and use ict for a diverse range of purposes, including information-seeking, playing online games, using social media, listening to music, watching videos, as well as uploading content such as music, photos australasian journal of information systems gregor et al. 2023, vol 27, research article indigenous knowledge-sharing 7 and videos to platforms for sharing within their own communities (du & haines, 2017; shaw et al., 2014; dyson, 2015). the embodiment of western values and ideologies in contemporary ict, and the potential for ict to represent a new medium of colonization for indigenous users has, however, been a topic of some debate. icts are not neutral but are the products of a particular culture and therefore embody the ideologies of that culture (dyson, 2004). this raises the question as to whether indigenous australians can harness the benefits of icts while retaining their own culture. dyson (2004) highlights the complexity of the issue, with, on the one hand, low ict adoption rates, but on the other hand a positive response to technologies by many indigenous australian users. despite some active resistance to the use of digital technology in indigenous knowledge practices (verran & christie, 2007), there remains little evidence that australian indigenous people are rejecting ict and the western values embodied within them (dyson, 2004). however, dyson (2004) argues that icts must reflect indigenous culture and be used to “service individual and community goals and aspirations” (p. 65). she further argues: rather than view icts as inherently loaded with western values, it might be more accurate to say that they are only capable of furthering the agenda of the dominant culture if used to that end. excluding indigenous people from icts on this basis is itself a form of cultural imperialism which seeks to maintain their existing disempowerment and marginalisation. in an age when everything runs on computer technology, indigenous people, as much as any other, have the need for full access to that technology and power over it. (p. 69) there are many examples of digital technologies being put to use in ways that remain true to indigenous culture, enhance indigenous cultural practices and the transfer of knowledge, and which promote the interests of their traditional groupings, clan lands, histories, connections and place (verran & christie, 2007, p 214; christie and verran, 2013; dyson, 2004). aboriginal elders are using computer technologies to help teach the younger indigenous generation to learn and understand the community’s knowledge traditions (verran & christie, 2014); web applications are being used to transfer traditional knowledge to younger generations and for community development (corbett et al., 2009, as cited in mayerhofer & taylor, 2010); multimedia archives are being used to store indigenous knowledge; traditional languages spoken by just a few are being taught using talking books (mayerhofer & taylor, 2010); and facebook is being used to maintain and learn languages and teach cultural traditions and protocols to younger generations (carlson, 2019). additionally, indigenous australians have demonstrated an interest in using digital technologies in their struggle to develop sustainable livelihoods on their own lands (verran & christie, 2007). social media has been instrumental in providing a platform for indigenous people to actively contribute to public discourse on topics relevant to indigenous society, particularly for illuminating the challenges of colonization and its consequences more broadly (for example, walter, 2022). portable devices, especially mobile phones, have become the technology of choice for many indigenous communities (shaw et al., 2014; brady & dyson, 2015) and mobile technologies have been enthusiastically adopted in remote indigenous communities where previous attempts to introduce other icts have been unsuccessful (brady & dyson, 2015). many indigenous communities choose mobiles over fixed-line ict (dyson, 2015) despite a lack of sufficient mobile coverage in many remote areas (shaw et al., 2014). this choice is due to the relatively low cost of mobile devices and the convenience of being able to communicate while away from home. mobile phones form a ‘conduit of connection’ within indigenous australasian journal of information systems gregor et al. 2023, vol 27, research article indigenous knowledge-sharing 8 communities helping to facilitate connections between family members, maintain cultural and language links, and support indigenous people’s mobility (brady & dyson, 2015, p. 29). additionally, indigenous people value mobile phones and other portable devices such as mp3 players and ipods for entertainment purposes such as listening to music, playing games, and watching videos, as well as sharing, storing and viewing content such as photos (shaw et al., 2014; brady & dyson, 2015). brady and dyson (2015) observed the close fit of the multi-media features of mobile devices and traditional cultural strengths, namely oral and audio practices and pictorial expression. consistent with indigenous values of sharing and reciprocity, devices are often shared amongst community members rather than individually owned (shaw et al., 2014). social media use has become an everyday, typical activity, especially for young indigenous australians. more than 60 per cent of indigenous people within remote communities are active users of social media – a rate that is higher than the general population (callinan, 2014). in sum, icts, particularly mobile technologies, are an important part of many indigenous people’s daily lives. indigenous people appreciate the various benefits of ict and believe that they have the capacity to improve their lives in terms of access to online government services, health consultations, online education, and job opportunities (du & haines, 2017). icts have the potential to address the continuing disadvantages experienced by indigenous communities by improving access to educational, governmental, financial and health services (radoll, 2010). significant barriers remain, however, which stand in the way of realising the full suite of benefits of ict, including knowledge transfer and furthering indigenous people’s cultural aims and objectives. these include its cost (initial adoption as well as ongoing retention); insufficient infrastructure in remote communities; and low levels of digital literacy in some areas (hunter & radoll, 2020; dyson, 2004; du & haines, 2017). 2.4 knowledge sharing interventions the prior discussion has dealt with the culture and distinctive ways of knowing, being and doing of indigenous people, particularly those in australia. it has also been shown that indigenous australians have been able to adapt and use ict developed in other cultures for their own purposes to a considerable extent. however, questions remain that motivated this study: namely, how should interventions that deal with the development of ict specifically for purposes meant to benefit indigenous australians be undertaken (i.e. the process); and what forms should they take (i.e. the content or product)? the first question is a matter of some debate, while our initial investigation showed that the extant literature yielded little in regard to the second question at an overview level. research approaches that address the development of new artifacts or new ways of doing something are referred to as design research or design science research in fields including information system and management (e.g. see gregor & hevner, 2013; denyer et al., 2008). these approaches deal with the development of prescriptive “how to” knowledge rather than the descriptive “what is” knowledge that has typified disciplinary areas such as the natural sciences. the term design alone may be used more at the level of professional practice. akama et al. (2019), for example, uses the label design for methods such as those practised in humancentred design at the stanford school. a variant on design science research is action design research (sein et al., 2011), which pays attention simultaneously to building innovative it artifacts in context and interwoven learning from the intervention. at the professional practice level consideration of an indigenous context could occur through user involvement in a australasian journal of information systems gregor et al. 2023, vol 27, research article indigenous knowledge-sharing 9 traditional systems development approach. as pointed out, however, in a critical case study of an ict4d intervention with indigenous people in taiwan, there were a number of unfortunate outcomes of this approach including the aboriginal people feeling marginalized and without a voice (lin et al., 2015). all the approaches and methods referred to above were developed with little questioning of the limits to their applicability across cultures, and subsequently the universality of such design approaches and methods has been questioned. for example, the anthropologist escobar (2015) sees that a euro-american perspective on design has been “exported to many world regions over the past few hundred years through colonialism, development and globalization” (p. 14). akama et al. (2019) provide critical discussion of problems relating to universality in design education and the reinforcement of modernist ideologies in relation to first peoples in australia and new zealand. these authors, however, see a nascent but growing movement towards design’s decolonization. they provide personal, reflexive stories and propose: respectful, reciprocal and relational approaches as an ontology of co-designing social innovation. this ontology requires a sensitivity to design’s location within multi-layered sites of power, knowledge, practices, cultural values, and precarious asymmetries as condition of collaboration (p. 2). further approaches are discussed by brereton et al. (2014) in relation to human-computer interaction. one approach identified is ethnography combined with action research and rapid ethnography. here, ethnography can be seen as a form of data collection. brereton et al. (2014) however, consider that such approaches can lead to a false sense of expertise on the part of the researcher and possibly arrogance. further, as a result of the history of ethnographic research, which was employed by colonial administrators interested in managing others (the native peoples of colonies), there can be wariness in aboriginal communities about being investigated. these authors also mention the use of participatory design and co-design, which they see as concerned with the bringing together of participants who can bring their own knowledge and expertise to a project. they suggest, however, that participatory methods may need to be revised in the context of remote aboriginal communities, so as to be compatible with their socio-cultural habits. a number of specific approaches have been proposed for projects involving indigenous australians, although different terms are employed: for example, an eight ways of knowing framework (yunkaporta & mcginty, 2009); indigenous leadership and partnership (dudgeon et al., 2016); co-design (ibinrriaga & martin, 2021); community-led development (mitchell et al., 2021); and country-centred design (janke, 2021, p. 301). wilson (2008) developed a relationships based methodology based on work with indigenous scholars in canada and australia. the approaches above provide some guidance with respect to methods and processes for those seeking to undertake interventions for knowledge sharing in indigenous communities. the work reported shows ways in which a number of stakeholders, both from within and without these communities, can work together respectfully and with reciprocation in knowledge sharing. interventions may be initiated from within a community as well as from an external agency. for example, brereton et al. (2014, p. 1184), who emphasize “engagement and reciprocity first!”, describe how their relationships with communities on groote eylandt developed in a project on cane toad invasion and then led to a “digital noticeboard” project that was initiated by the community. australasian journal of information systems gregor et al. 2023, vol 27, research article indigenous knowledge-sharing 10 although the literature reviewed here show some commonalities in thinking about approaches to knowledge sharing interventions with indigenous communities, we were unable to locate prior reviews or cross-case analyses of outcomes of the use of the different approaches. neither did the extant literature yield reviews that showed analysis of the mechanisms or content that was employed in cases of interventions. these gaps provide justification for the work described further below. 3 research approach 3.1 method the method used is that of a scoping review (arksey & o’malley, 2005; levac et al., 2010; peters et al., 2020) which is suitable for a study that aims to review evidence for treatment or intervention when the topic of enquiry is relatively broad and where many different study designs have been used. paré et al. (2015) identify a scoping review as one that is appropriate for information systems research in their typology of reviews. one author of the review is a well experienced indigenous researcher and the project of which it forms part has an indigenous reference group that advises on all work performed within the scope of the project. the review, even in draft form, has been useful in discussions with the reference group on how to approach the next stages of design in our project, in conjunction with what has arisen from yarning group discussions. the scoping study followed the five steps proposed by arksey and o’malley (2005): (1) identify the research question; (2) identify relevant studies; (3) study selection; (3) chart the data; and (5) collate, summarise and report the results. the recent update for scoping studies by peters et al. (2020) was taken into account. peters et al. (2020) note that the sources used in scoping reviews are not likely to lend themselves to meta-analysis and a descriptive approach to reporting is appropriate. basic coding of data in categories may be useful, particularly when “the purpose is to identify or clarify concepts or definitions within a field or to identify key characteristics related to a concept” (p. 2122). 3.2 search strategy the process began with a comprehensive search of scopus, proquest, web of science and science direct. these databases were chosen because they are popular and believed to have relevant studies for the review. to ensure the search was exhaustive, it was extended to informit (a database with a distinct focus on australian and new zealand indigenous research collections). only peer-reviewed studies from the five databases were considered. table 1 shows the search string developed to select studies based on the research questions provided in the introduction. the final search terms resulted from an iterative process where the researchers considered the results of different combinations of terms and refined the search so it was neither too broad nor too narrow. the search process was managed using the covidence software. concept search terms context “indigenous” and “australia” knowledge sharing and (“knowledge transfer” or “ways of knowing” or “communication strategy” or “awareness raising” or “advice giving” intervention and (“intervention” or “program” or “project” or “technology” or “strategy”) table 1. search terms australasian journal of information systems gregor et al. 2023, vol 27, research article indigenous knowledge-sharing 11 3.3 exclusion criteria studies were excluded if: (a) they did not focus on experiences in australia, or if results for australia could not be separated from a study that included other regions; (b) they were not aimed directly at indigenous people (e.g., if aimed at teachers or medical workers who then interact with indigenous people); (c) the knowledge-sharing aspect could not be separated from some other intervention (e.g., advice could be given on dental care, but dental treatment was also given and outcomes were not evaluated separately, as in parker et al. (2012); (d) the focus was on methodology or research practice and there was limited attention to evidence of the outcomes of the intervention; (e) they were speculative articles or feasibility studies. 3.4 study selection the initial search returned 144 studies, reduced to 98 after duplicates were removed. at least two researchers assessed the titles and abstracts of these 98 studies to select articles that represented original empirical research (primary studies) as well as review studies (nonprimary studies), taking the exclusion criteria into account. figure 1: …. figure 1. prisma flowchart of search process the abstract screening process resulted in 30 studies being selected. at least two members of the team then checked the full text of the articles against the exclusion criteria and other team members were consulted until a sample of eight studies remained, including one review and seven primary studies. forward and backward secondary searching was carried out to check for further relevant articles. as a result of this step, the review by brusse et al. (2014) was included as it was cited by adams et al. (2017). thus, the final sample included two reviews and seven primary studies. figure 1 above illustrates the search process in a prisma flowchart. records identified from databases scopus, proquest, web of science, science direct, informit (n = 144) records after duplicates removed (n = 98) duplicates removed (n = 46) records excluded after title and abstract screening (n = 68) full articles assessed for eligibility (n = 30) records excluded with reasons (n = 22) • not aimed directly at indigenous people • knowledge sharing not separate from other interventions • focus was on methodology/research practice final studies included (n = 9) *the ninth study was found after a forward and backwards secondary search australasian journal of information systems gregor et al. 2023, vol 27, research article indigenous knowledge-sharing 12 3.5 data extraction the studies extracted are first presented at an overview level to show the nature and extent of the studies in general (see kuckertz & block, 2021). the data extracted (charted) for primary studies are then further analysed using a schema based on the context-interventionsmechanisms-outcome (cimo) framework (denyer et al., 2008). the cimo framework is used in fields including information systems, management, and organisation studies to present practice-based evidence (e.g., suchowerska et al., 2020) and as a base for design principles (see boucharas et al., 2010). 4 results 4.1 overview of included studies the search process identified a sample of nine articles that were published between 2014 and 2021, including two review studies and seven primary studies. table 2 gives an overview of the review studies selected and table 3 of the primary studies. further analysis of the primary studies is provided in the following section. the brusse et al. (2014) review concluded that although social media technologies have the capacity to address health promotion in indigenous communities, the evidence for their capacity to achieve results is limited. the authors recognise that one issue that may help explain the lack of evidence for project results could be that the nature of the technologies used (social media and mobile apps) do not allow for clear outcomes or implementation measures. they suggested that health promotion organisations need to gain a more thorough understanding of technologies, their users and their use. the olsen et al. (2014) review analysed their sample of studies for developing a response to the hepatitis b virus in terms of five priority areas: building partnerships and strengthening community action, diagnosis and screening, preventing transmission, clinical management and health maintenance and support. the last of these priority areas was highlighted as a concern for knowledge sharing. the authors concluded that there was a need “to develop culturally appropriate health promotion interventions for people with chb [chronic hepatitis b] and their families to strengthen health literacy” (p. 140). this review did not report any studies where outcomes were reported from a knowledge-sharing intervention. the seven primary studies were published between 2015 and 2021. a variety of study methods were used: four were mixed methods (adams et al., 2017; berends & halliday, 2018; fagan et al., 2015; thomas et al., 2010), two were mainly qualitative (cox et al., 2021; mitchell et al., 2021) and one was a confessional-type account with rich data (welsh, 2018). four of the seven studies employed external independent analyses, in that they were undertaken by people who were at least some distance from the intervention being evaluated (berends & halliday, 2018; cox et al., 2021; fagan et al., 2015; thomas et al., 2010). this overview shows that a number of studies appearing since 2014 included some evaluation of outcomes for knowledge-sharing interventions with indigenous communities, although the methods varied. the two relevant review articles from 2014 did not find any relevant primary studies with an evaluation of outcomes. one possible conclusion is that this area of study is becoming more mature, as there is more attention to the evaluation of outcomes. australasian journal of information systems gregor et al. 2023, vol 27, research article indigenous knowledge-sharing 13 study details/aim discipline/ region type of study indicative results brusse, et al. (2014). social media and mobile apps for health promotion in australian indigenous populations: scoping review. journal of medical internet research, 16(12), e3614. to study social media and mobile apps used in health promotion: (1) evidence worldwide, (2) their use in australian indigenous communities. health/ (1) world-wide (2) australia 1) scoping review of peer-reviewed evidence, (2) review of relevant projects in australia. 1) found seven reviews, 17 primary studies, (2) found five social media projects, four relevant mobile phone apps. no publicly available evaluation for any of these projects. olsen, et al. (2014). responding to australia’s national hepatitis b strategy 2010-13: gaps in knowledge and practice in relation to indigenous australians. australian journal of primary health, 20(2), 134–142 “map the main sources and types of evidence available on the epidemiology of hbv [hepatitis b virus] as well as public health responses” 2001-2013 for indigenous australians (p. 134). health/ australia scoping review “significant gaps in knowledge and practice were identified in relation to each of the five national hepatitis b strategy priority action areas” (p. 134). table 2. review studies study details/ aim discipline/ region type of evaluation (internal/ external) indicative results 1) adams et al. (2017). how’s your sugar? evaluation of a website for aboriginal people with diabetes. jmir diabetes, 2(1) “to describe the construction and utilization of the website “how’s your sugar,” a website for aboriginal people with type 2 diabetes” (p. 1). health/ all victoria mixed method: content analysis of documents, google analytics, survey. (internal) “inclusion of aboriginal ways of knowing and being alongside other theoretical and evidence models in web design is possible. aboriginal people do utilize web-based health promotion” (p. 1). 2) berends et al. (2018). capacity building and social marketing promotes healthy lifestyle behaviour in an australian aboriginal community. australian journal of rural health, 26(4), 279–283 to describe a social marketing initiative targeting a regional aboriginal australian community and to determine whether a specific campaign produced positive outcomes (p. 280). healthy lifestyle behaviour/ regional victoria mixed method: interviews, facebook insights, survey. (external) “social marketing engaged the community and prompted positive behaviour change” (p. 279) 3) cox et al. (2021). closing the gap in aboriginal health disparities: is there a place for elders in the neoliberal agenda? australian health review, 46(2), 173-177. to examine how “elders consider the closing the gap programs for improving community health outcomes” (p. 173) community health/ remote community in tasmania participatory action research, thematic analysis of individual interviews and yarning circles. (external) “the closing the gap programs were seen by elders as having instrumental value for addressing aboriginal community disadvantage. however, the programs also represented a source of ongoing dependency that threatened to undermine the community’s autonomy, self-determination and cultural foundations” (p. 173) australasian journal of information systems gregor et al. 2023, vol 27, research article indigenous knowledge-sharing 14 4) fagan et al. (2015). successes in sexual health communications development, programmatic implementation and evaluation in the torres strait region 2006 to 2012. australian and new zealand journal of public health, 39(3), 270–276 “to evaluate the indigenous sexual health promotion program in the torres strait 2006–2012 that culminated in an education-entertainment radio drama, kasa por yarn (kpy)” (p. 270) health/ remote mixed method: prior publications, document review, survey. (external) “there was improvement in sexual health knowledge scores (p<0.00) in the 15–19year-old torres strait islander population between 2007 and 2012. the 2012 15–24year-old population exposed to kpy had higher sexual health knowledge scores compared with those unexposed (p=0.02)” (p. 270) 5) mitchell et al. (2021). using community-led development to build health communication about rheumatic heart disease in aboriginal children: a developmental evaluation. australian and new zealand journal of public health, 45(3), 212–219. to address low understanding by families of acute rheumatic fever (arf) and rheumatic heart disease suffered by children (rhd) in remote nt through health communication and to increase opportunities for prevention through awareness. health/ remote analysis of qualitative data collected throughout process. (internal) “some children showed new responses and attitudes to skin infections and their rhd treatment. language teachers used natural social networks to disseminate new information. a community interagency collaboration working to prevent rhd commenced” (p. 212). 6) thomas et al. (2010). lessons for aboriginal tobacco control in remote communities: an evaluation of the northern territory ‘tobacco project’. australian and new zealand journal of public health, 34(1), 45-49. “to evaluate a northern territory (nt) government-led pilot ‘tobacco project’ in six remote communities” (p. 45). some activities included were not knowledge transfer: e.g. changing stock in stores. health/ remote surveys of staff, interviews. observation of intervention, review of project documents, sales monitoring of tobacco consumption.(external) “more tobacco control activity was consistently associated with a greater reduction in tobacco consumption” (pp. 45). 7) welsh, b. (2018). the hepatitis c trilogy: a case for indigenous theatre as a contemporary manifestation of traditional healing business. australasian drama studies, 73, 20-41. an indigenous theatre company, ilbijerri, presented shows focused on the treatment of hepatitis c within the indigenous community health/ three communities in victoria, including a prison rehearsal observation/ yarning circles (internal) concluded that the theatre work was a contemporary form of healing ritual. table 3. primary studies 4.2 charting analysis of the primary studies was undertaken using a charting method to address our research questions. table a1 in the appendix shows the data charted for one study as an example against the cimo-based schema. the detailed schema was developed by two of the researchers after the study of four articles, with a third researcher helping resolve differences (following peters et al., 2020). a spreadsheet showing the charting of all primary studies in the sample will be made available online. australasian journal of information systems gregor et al. 2023, vol 27, research article indigenous knowledge-sharing 15 4.3 nature of interventions developed tables 2 and 3 show that the two reviews and seven primary studies identified were all related to health. one study was at the state level (victoria), one was regional, one was conducted with three communities within a state and four with remote communities. the different aspects of health addressed were type 2 diabetes, healthy lifestyles, community health, sexual health, rheumatic heart disease, tobacco consumption and hepatitis c. from table 3, it can be seen that the primary interventions reported were a website, a community facebook page, closing the gap programs, an education-entertainment radio drama and other communication strategies, a curriculum for rheumatic heart disease, a tobacco control project and an indigenous theatre performance. all studies reported positive outcomes. only one study also reported negative outcomes, namely cox et al. (2021) in their independent analysis of the closing the gap program with a small group of elders in tasmania. the elders felt the program threatened their community’s autonomy and cultural foundations, despite yielding results of instrumental value. to address the nature of the interventions developed, it is of interest to look at the development approach for the intervention and the “mechanisms” employed to achieve desired outcomes. table 4 gives a short account of the interventions. 1) intervention: the how’s your sugar website for aboriginal people with type 2 diabetes in northern australia and torres straits islands (adams et al., 2017) approach: collaborative, mix of indigenous and non-indigenous team members. an initial review of other websites targeted at indigenous people informed the design. consultations with aboriginal people with diabetes and aboriginal health workers helped to determine what would be appealing to the target audience specifying requirements such as entertainment and humour, storytelling, and aboriginal health workers. mechanisms and tools: the website had an opening page with a graphic depicting a kitchen with four peers around a table. visitors could navigate around the kitchen and click to link to appropriate resources, including videos and a nutrition game. fridge magnets were used for promotion. 2) intervention: deadly sports facebook page in gippsland (reported on by berends & halliday, 2018) approach: partnership between the project and local stakeholders (note the berends et al. study is mainly concerned with evaluation). mechanisms and tools: facebook page with calendar of local events, healthy lifestyle information, regular posts of local people engaged in healthy activities including videos, newsletter. the immediacy of the media included was seen as important to engagement and promotion. 3) intervention: australian government closing the gap programs (reported on by cox et al., 2021) approach: the authors say that the closing the gap programs lack an acknowledgement that policy-making is a value-laden process “reflecting a suite of dominant western neoliberal assumptions” (p. 173). mechanisms and tools: this study was primarily evaluation and did not provide details of the closing the gap programs. the eight elders interviewed suggested three possible approaches to ensure the community’s cultural foundations were not undermined: 1) embedding cultural values in the programs; 2) separating the programs from cultural values; 3) ensuring culture is kept as core business. 4) intervention: sexual health promotion activities 2006-2012 in far north queensland (reported on by fagan et al., 2015) approach: partnership between public health unit, district health service and communities. mechanisms and tools: numerous promotion activities including school-based health education, arts-based strategies and a radio drama kasa por yarn as well as communication activities through sms, information nights, newspaper and radio announcements, mass media campaign including tv, website and posters, and a hiphop project. 5) intervention: curriculum for rheumatic heart disease (rhd) lessons in northern australia (mitchell et al., 2021) australasian journal of information systems gregor et al. 2023, vol 27, research article indigenous knowledge-sharing 16 approach: the multidisciplinary team included aboriginal members. the development process included “both-ways learning”, with careful iterative discussions leading to the development of the “story-line”, a narrative for information dissemination. exposure to rhd data was a catalyst. analysis showed adherence to the principles of community-led development in the work. mechanisms and tools: conceptual and language issues meant meaning-based translation methods. indigenous knowledge and metaphors were used (e.g white blood cells as warriors). information posters in four languages were produced for the community. 6) intervention: tobacco project in six remote communities in the northern territory 2007-2008 (reported on by thomas et al., 2010) approach: communities were asked to prioritize and plan activities for which a northern territory government department could provide support. activities varied across the six communities, with examples “of both successful and poor collaboration and conflict between project partners” (p. 48). mechanisms and tools: most of the project’s efforts involved community education and awareness raising, but also included activities such as cessation of cigarette sales at a supermarket and establishment of smoke-free areas. this study was primarily for evaluation and did not provide detailed accounts of the project’s activities. 7) intervention: series of theatre performances viral on hepatitis b in victoria (welsh, 2018) approach: solely indigenous members. when making viral, the first exercise that theatre makers were asked to do was to write down what they knew about hepatitis c. the development process began with gum leaves brought in to sweep the space and where the cast and crew sat around the freshly made stage. mechanisms and tools: the performances included drama, dance, story and humour. the rehearsal process and the final production of viral provided a relationally accountable and responsible portrayal of the lives, experiences and challenges for indigenous people living with hepatitis c. yarning circles afterwards discussed the performance. table 4. intervention design and development 4.4 indigenous ways of knowing our second research question asked how interventions take account of indigenous ways of knowing, being and doing, if at all. four of the seven studies in table 4 gave detailed accounts of the development of an intervention that explicitly acknowledged indigenous ways of knowing, being and doing. these studies noted consultation with aboriginal people and inclusion of aboriginal self-determination pathways (adams et al., 2017); use of culturally appropriate storyline and radio dramas (fagan et al., 2015); language-based and culturally relevant health communication centring on indigenous epistemologies (mitchell et al., 2021); and storytelling: culturally safe and partially decolonised rehearsal space, respectful collaborative practice and responsibly drawing from lived experiences and community voices (welsh, 2018). three other studies in table 4 focused on evaluation and had relatively little detail on the intervention itself. berends and halliday (2018) noted that the local community was involved in social media content production and dissemination. cox et al. (2021) did not describe the closing the gap programs evaluated in detail but used yarning in their evaluation. these authors ascribed problems seen in the gap programs to inadequate recognition in the neoliberal governmental approach to aboriginal cultural values, particularly in respect of the role of elders, who “are recognised cultural custodians of social, kinship, ecological, language and spiritual knowledge systems developed over millennia” (p. 174). thomas et al. (2010) noted that storytelling was used to reinforce anti-tobacco messages. a number of the interventions included performative means to enable knowledge sharing. some studies showed how this type of work arose during the development processes, without australasian journal of information systems gregor et al. 2023, vol 27, research article indigenous knowledge-sharing 17 necessarily being based on explicit theoretical knowledge of indigenous knowledge-sharing practices. for example, the development of the how’s your sugar website (adams et al., 2017) included an initial review of other websites aimed at diabetes management. the team found that these websites commonly featured static materials, such as images, texts and downloadable pdfs. they were written from the perspective of a non-aboriginal health professional as an expert addressing a client. a team member commented: “who do they think they are talking to? it’s like they stuck some aboriginal artwork on it and said ‘that’ll do’, it’s patronizing” (p. 4). the website that resulted from this project had an opening page of a kitchen with four peers, a familiar informal space for yarning, and visitors could navigate around this space. each peer had a link to a short video about how they managed their own diabetes. the curriculum developed in the rheumatic heart disease program (mitchell et al., 2021) developed analogies and metaphors in a storyline that included showing white blood cells defending the body as a traditional war party defending the territory against invading outsiders. school students produced additional materials such as t-shirts with graphics depicting a white blood cell fighting a bad germ and a fish trap depicting how white blood cells know good germs from bad. a number of the studies mentioned the importance of making activities fun and including humour. for example, the production of the viral theatre performances deployed humour through the depiction of three characters representing documentary filmmakers being presented as “very naïve and very, very white” (welsh, 2018, p. 28). the indigenous producers included these characters as comic relief. they were meant to illustrate what happens when non-indigenous people are given a small amount of cultural safety training and then start using inappropriate terms. in sum, six of the seven primary studies reported only positive outcomes and all of these showed some recognition of indigenous ways of knowing, doing and being in some way, including storytelling, drama, dance, graphic artwork and humour. these studies also showed, where detail was available, that indigenous people were involved in development activities mostly in appropriate ways, with terms used such as collaborative (adams et al., 2017), partnership (berends & halliday, 2018; fagan et al., 2015), and solely indigenous development (welsh, 2018). 4.5 use of ict in interventions our third research question asked how ict was involved in these interventions, if at all. only two of the seven studies had interventions that were primarily ict-based, involving a website (adams et al., 2017) and a facebook page (berends & halliday, 2018). other studies incorporated ict as a component of the intervention. the sexual health promotion, although mainly using a radio drama as a vehicle, also included a website (fagan et al., 2015). the intervention to address rheumatic heart disease used data from a medical database showing the high incidence of the disease as a catalyst to initiate change (mitchell et al., 2021). 5 discussion and conclusions our scoping review identified two review articles and seven primary studies relevant to our research objectives, published from 2014-2021. all were in the area of health. six of the seven studies recognised indigenous ways of knowing, being and doing to some extent and all australasian journal of information systems gregor et al. 2023, vol 27, research article indigenous knowledge-sharing 18 reported some positive outcomes. the seventh study attributed some negative outcomes to a lack of respect for indigenous cultural values (cox et al., 2021). unlike earlier reviews (brusse et al., 2014; olsen et al., 2014), the final sample of primary studies located in our review all had some form of evaluation and four were evaluations undertaken independently of the primary development project. this finding may indicate a growing maturity in the field and more attention being paid to the importance of evidence for interventions to show what works under which circumstances. on the other hand, only seven studies were found, which appears a relatively small number. the selection process for the review excluded a number of studies that were located with our search string and described interventions, particularly in relation to the methodology that was used but did not describe outcomes to any extent. this observation indicates there is still room for improvement in building an evidence base. it is not expected that scoping reviews, which address a broad question and include studies with varying methods of gathering evidence, can provide definitive guidance for the efficacy of different strategies. nevertheless, our study has shown that for seven studies where results were evaluated, the six that reported only positive outcomes had all given consideration to indigenous ways of knowing to some extent. this observation is congruent with the arguments reviewed in our introduction that propose that attention to indigenous ways of knowing and doing is vital in projects with indigenous communities. the seven studies also show the broad range of stakeholders who can be involved in interventions, including indigenous people, government departments, community leaders, and local health organizations. the impetus for the interventions varied. in one case the project originated in an indigenous theatre company (welsh, 2018), in one spontaneous emergence was observed amongst a school’s aboriginal language and culture teachers (mitchell et al., 2021), while others were initiated by government departments (e.g. thomas, 2010). the detailed accounts of projects with positive outcomes provide striking examples of how verbal, visual and performative expression can be incorporated into knowledge-sharing interventions. imagination and creativity were evident in the developmental processes. for example, welsh (2018) documented the power of theatre to influence indigenous audiences on issues that affect their lives, such as the trauma of living with hepatitis c. indigenous performances that incorporate humour, dance and song – elements that have long been integral to indigenous cultural practice – can thus be harnessed by an indigenous creative team, or team members, to good effect. the cases provide examples of how indigenous team members were directly responsible for elements of the design of an intervention, in ways that non-indigenous people appear unlikely to have envisaged. for example, websites designed from a non-indigenous viewpoint for indigenous people were seen as patronising and highlight the need for engaging and interactive websites, as recognized by indigenous team members in the “how’s your sugar” project (adams et al., 2017). the theatre performance designed by a solely indigenous dance company had a storyline that poked fun at “very, very white” documentary makers with only a smattering of cultural safety training (welsh, 2018, p. 28). it is perhaps surprising that our review located only two studies where ict was the primary vehicle for knowledge sharing. this finding may be a result of ict-based projects either not including an evaluation of outcomes or being left out of the peer-reviewed literature. such a finding would be congruent with the review by brusse et al. (2014) of social media and mobile australasian journal of information systems gregor et al. 2023, vol 27, research article indigenous knowledge-sharing 19 apps used in health promotion which found nine relevant projects that were non-peerreviewed and had no publicly available evaluation of the projects. although the reporting of ict-based interventions in our study context is limited, the two studies that do use ict (adams et al., 2017; berends & halliday, 2018) should be carefully considered by those planning ict-based work. the non-ict-based studies also provide interesting ideas that could be built on or adapted in future work. the oral traditions and performative aspects of design that have been incorporated could be used further in ict-based innovations that use multimedia-based modes of communication. our initial background review shows that indigenous people embrace the use of digital technologies and are putting them to use in ways that enhance their cultural practices (e.g., verran & christie, 2007). it is critical, however, that any ict-based interventions, particularly those primarily designed by non-indigenous people, are designed in such a way so as to not merely reproduce western values, but rather facilitate use amongst indigenous communities that accords with their cultural practices (dyson, 2004). in addition to involving indigenous peoples in the design of such interventions, this may also entail ensuring that ict-based interventions are mobilefriendly, facilitate easy sharing with others, and utilise multimedia forms of expression including videos, pictures and music. our study has some limitations. the search process was limited to academic peer-reviewed literature while governmental reports and similar documents were not searched. thus, projects undertaken by governmental bodies and the projects reported by fogarty et al. (2018) relating to health promotion strategies addressing gambling are not necessarily included. in addition, although the search process was performed with care, some relevant studies may have been excluded through our selection of search terms. this study makes a valuable contribution to the body of knowledge by reviewing what is currently known about interventions to assist in knowledge sharing in australian indigenous communities where an evaluation of outcomes has occurred. it supports the view that it is vital to acknowledge and utilise indigenous ways of knowing in such interventions. the cases reviewed also suggest that indigenous team members are essential for creative aspects of interventions that are appropriate for an indigenous audience. the study reveals few accounts of ict-based interventions with empirical evaluation but the case studies reviewed, both ict and non-ict based, offer rich detail to inform future design and development work. although this study has focussed on interventions for knowledge sharing with australian indigenous people, it should have relevance more broadly. the finding concerning the need to utilize indigenous ways of knowing in the australian context can be related to the methods proposed for research with other indigenous people worldwide, such as wilson’s (2008) framework of relational accountability that was developed in both australia and canada (referenced in the indigenous theatre company study (welsh, 2018)). the study also indicates directions for future research. all the studies selected for the review were in health-related disciplines – an unexpected finding. this highlights the need for similar studies in other areas such as the other problem areas identified in the closing the gap initiative, including education, vocational training and services provision. sustainability and environmental management are other potential areas for further work in australia and are being researched internationally (e.g. tschirhart et al., 2016). while some projects are occurring in the non-health areas, they appear to lack the level of maturity that has been reached in reporting outcomes, as found in health fields. indigenous australians continue to face a range australasian journal of information systems gregor et al. 2023, vol 27, research article indigenous knowledge-sharing 20 of structural and institutional challenges that affect their well-being. much more needs to be done to address these challenges in ways that incorporate indigenous ways of knowing, being and doing so as to bring about meaningful change for groups experiencing substantial disadvantages. acknowledgements we would like to acknowledge the traditional lands on which our authors live and work, and pay our respect to elders, past, present and emerging. one author, a descendent of the wiradjuri people, would like to acknowledge and pay respect to her country, culture and elders. funding was received from the university of canberra collaborative indigenous research initiative. references adams, k., liebzeit, a., browne, j., & atkinson, p. (2017). how’s your sugar? evaluation of a website for aboriginal people with diabetes. journal of medical internet research diabetes, 2(1). doi: 10.2196/diabetes.6930 the australian institute of aboriginal and torres strait islander studies (aiatsis) (2021). the little red yellow black book. (4th edn.). aboriginal studies press.acton, act, australia akama, y., hagen, p., & whaanga-schollum, d. (2019). problematizing replicable design to practice respectful, reciprocal, and relational co-designing with indigenous people. design and culture. doi: 10.1080/17547075.2019.1571306 althaus, c. (2020). different paradigms of evidence and knowledge: recognising, honouring, and celebrating indigenous ways of knowing and being. australian journal of public administration,79(2), 187-207. applebee, w. (2020). from homo sacer to subaltern: becoming aboriginal online. doctoral dissertation, university of canberra, australia. https://researchsystem.canberra.edu.au/ws/portalfiles/portal/47143146/uc_final_the sis_submission_wayne_applebee_10th_december_2020.pdf arksey, h., & o'malley, l. (2005). scoping studies: towards a methodological framework. international journal of social research methodology, 8(1), 19-32. australian bureau of statistics (2022). national, state and territory population. retrieved from: https://www.abs.gov.au/statistics/people/population/national-state-and-territorypopulation/latest-release. australian government (2020). closing the gap report. https://ctgreport.niaa.gov.au/sites/default/files/pdf/closing-the-gap-report-2020.pdf australian institute of health and welfare. (2021). profile of indigenous australians. https://www.aihw.gov.au/reports/australias-welfare/profile-of-indigenousaustralians behrendt, l. (2021). indigenous australians for dummies. (2nd edn.). wiley, milton, qld. berends, l., & halliday, r. (2018). capacity building and social marketing promotes healthy lifestyle behaviour in an australian aboriginal community. australian journal of rural health, 26(4), 279–283. doi: 10.1111/ajr.12421 australasian journal of information systems gregor et al. 2023, vol 27, research article indigenous knowledge-sharing 21 berkes, f., colding, j., & folke, c. (2000). rediscovery of traditional ecological knowledge as adaptive management. ecological applications, 10(5), 1251-1262. bessarab, d., & ng’andu, b. (2010). yarning about yarning as a legitimate method in indigenous research. international journal of critical indigenous studies, 3(1), 37-50. doi: 10.5204/ijcis.v3i1.57 blenkinsop, l. m. (2017). generational perspectives on community knowledge transfer in nipissing first nation. doctoral dissertation, university of guelph, ont, canada. https://atrium.lib.uoguelph.ca/server/api/core/bitstreams/f213c372-9a66-45cb-91f110898bdf0e46/content boucharas, v., van steenbergen, m., jansen, s., & brinkkemper, s. (2010, november). the contribution of enterprise architecture to the achievement of organizational goals: a review of the evidence. in international workshop on trends in enterprise architecture research (pp. 1-15). springer, berlin, heidelberg, germany. bourke, c. & edwards, w. h. & bourke, e.. & tv open learning. (1998). aboriginal australia: an introductory reader in aboriginal studies. university of queensland press, st lucia, qld, australia. brady, f., & dyson, l. e. (2015). why mobile? indigenous people and mobile technologies at the edge. in l. e dyson,s. grant, m., & hendriks (eds), indigenous people and mobile technologies (pp. 39-58). routledge, milton park, uk. brady, w. (1997). indigenous australian education and globalisation. international review of education, 43(5), 413-422. brereton, m., roe, p., schroeter, r., & lee hong, a. (2014, april). beyond ethnography: engagement and reciprocity as foundations for design research out here. in proceedings of the sigchi conference on human factors in computing systems (pp. 1183-1186). toronto, canada. brusse, c., gardner, k., mcaullay, d., & dowden, m. (2014). social media and mobile apps for health promotion in australian indigenous populations: scoping review. journal of medical internet research, 16(12), e3614. callinan, t. (2014). remote indigenous australians rely on facebook to stay in touch, sbs. retrieved from: http://www.sbs.com.au/news/article/2014/08/26/remote-indigenousaustralians-rely-facebook-stay-touch cameron, e. (2015). is it art or knowledge? deconstructing australian aboriginal creative making. arts, 4 (2), 68-74). doi: 10.3390/arts4020068 carlson, b., & frazer, r. (2018). social media mob: being indigenous online. macquarie university, sydney, australia. carlson, b. (2019). disrupting the master narrative: indigenous people and tweeting colonial history. griffith review, 6), 224-234 christie, m., & verran, h. (2013). digital lives in postcolonial aboriginal australia. journal of material culture, 18(3), 299–317. doi: 10.1177/1359183513492081 coombes, j., & ryder, c. (2020). walking together to create harmony in research: a murri woman’s approach to indigenous research methodology. qualitative research in https://www.taylorfrancis.com/search?contributorname=stephen%20grant&contributorrole=editor&redirectfrompdp=true&context=ubx https://www.taylorfrancis.com/search?contributorname=max%20hendriks&contributorrole=editor&redirectfrompdp=true&context=ubx australasian journal of information systems gregor et al. 2023, vol 27, research article indigenous knowledge-sharing 22 organizations and management: an international journal, 15(1), 58-67. https://doi.org/10.1108/qrom-07-2018-1657 corbett, j., singleton, g., & muir, k. (2009). web 2.0 for aboriginal cultural survival: a newaustralian outback movement. participatory learning and action, 59(1), 71-78 cox, t., hoang, h., mond, j., & cross, m. (2021). closing the gap in aboriginal health disparities: is there a place for elders in the neoliberal agenda? australian health review, 46(2), 173-177. doi: 10.1071/ah21098 denyer, d., tranfield, d., & van aken, j. e. (2008). developing design propositions through research synthesis. organization studies, 29(3), 393-413. du, j. t., haines, j., sun, v. q., partridge, h., & ma, d. (2015). understanding indigenous people’s information practices and internet use: a ngarrindjeri perspective. pacis 2015 proceedings. paper 183. https://aisel.aisnet.org/pacis2015/183/ du, j. t., & haines, j. (2017). indigenous australians' information behaviour and internet use in everyday life: an exploratory study. information research: an international electronic journal, 22(1). https://informationr.net/ir/22-1/paper737.html. dudgeon, p., luxford, y., & cox, a. (2016). solutions that work, what the evidence and our people are telling us. aboriginal and torres strait islander suicide prevention evaluation project report. https://www.atsispep.sis.uwa.edu.au/__data/assets/pdf_file/0006/2947299/atsispepreport-final-web.pdf dudgeon, p., calma, t., milroy, j., mcphee, r., darwin, l., von helle, s., & holland, c. (2018). indigenous governance for suicide prevention in aboriginal and torres strait islander communities: a guide for primary health networks. https://www.blackdoginstitute.org.au/wp-content/uploads/2020/04/designed-finalcultural-framework-guide-v4.pdf dyson, l. e. (2004). cultural issues in the adoption of information and communication technologies by indigenous australians. in f. sudweeks and c. ess (eds.) proceedings cultural attitudes towards communication and technology, pp. 58-71. murdoch university, wa, australia. dyson, l. e. (2015). framing the indigenous mobile revolution. in l. e dyson, s. grant, & m. hendriks (eds), indigenous people and mobile technologies (pp. 15-36). routledge, milton park, uk. elias, a., & paradies y. (2021). the costs of institutional racism and its ethical implications for healthcare. journal of bioethical inquiry, 18(1):45-58. doi: 10.1007/s11673-020-10073-0 ellis, c. (1985). aboriginal music: education for living. university of queensland press, st lucia, qld, australia. esafety commissioner (2022). cool, beautiful, strange and scary: the online experiences of aboriginal and torres strait islander children and their parents and caregivers. australian government canberra, australia. https://www.esafety.gov.au/sites/default/files/202303/cool_beautiful_strange_and_scary_report.pdf https://www.taylorfrancis.com/search?contributorname=stephen%20grant&contributorrole=editor&redirectfrompdp=true&context=ubx https://www.taylorfrancis.com/search?contributorname=max%20hendriks&contributorrole=editor&redirectfrompdp=true&context=ubx https://www.taylorfrancis.com/search?contributorname=max%20hendriks&contributorrole=editor&redirectfrompdp=true&context=ubx australasian journal of information systems gregor et al. 2023, vol 27, research article indigenous knowledge-sharing 23 escobar, a. (2015). transiciones: a space for research and design for transitions to the pluriverse. design philosophy papers, 13(1), 13-23. fagan, p. s., robertson, h. k., pedrana, a. e., raulli, a., & crouch, a. a. (2015). successes in sexual health communications development, programmatic implementation and evaluation in the torres strait region 2006 to 2012. australian and new zealand journal of public health, 39(3), 270–276. doi: 10.1111/1753-6405.12356 fogarty, m., coalter, n., gordon, a., & breen, h. (2018). proposing a health promotion framework to address gambling problems in australian indigenous communities. health promotion international, 33(1), 115-122. fuary, m. (1993). torres strait cultural history. in n. loos & t. osanai (eds.), indigenous minorities and education: australian and japanese perspectives on their indigenous peoples, the ainu, aborigines and torres strait islanders (pp. 165-186). sanyusha, tokyo, japan gorman, d., & ward, r. (2008). developing suicide prevention expertise with aboriginal communities. mental health nursing – a broad canvas: the art of mental health nursing in the age of technology and science, 6-10 oct 2008, australian college of mental health nurses, melbourne, australia. gregor, s., & hevner, a. r. (2013). positioning and presenting design science research for maximum impact. mis quarterly, 37(2), 337-355. henry, r. (2000). dancing into being the tjapukai aboriginal cultural park and the laura dance festival. the australian journal of anthropology, 11(2), 322-332. hunter, b. h., & radoll, p. j. (2020). dynamics of digital diffusion and disadoption. australasian journal of information systems, 24. doi:10.3127/ajis.v24i0.1805 ibinarriaga, h. d., & martin, b. (2021). critical co-design and agency of the real. design and culture, 13, 253 276. iseke, j. (2013). indigenous storytelling as research. international review of qualitative research, 6(4), 559-577. janke, t. (2021). true tracks: respecting indigenous knowledge and culture. unsw press ,sydney, nsw, australia. kuckertz, a., & block, j. (2021). reviewing systematic literature reviews: ten key questions and criteria for reviewers. management review quarterly, 71(3), 519-524. kwaymullina, a. (2005). seeing the light: aboriginal law, learning and sustainable living in country. indigenous law bulletin, 6(11), 12-15. latimore, j., nolan, d., simons, m., & khan, e. (2017). reassembling the indigenous public sphere. australasian journal of information systems, 21. levac, d., colquhoun, h., & o'brien, k. k. (2010). scoping studies: advancing the methodology. implementation science, 5(1), 1-9. lin, c. i., kuo, f. y., & myers, m. d. (2015). extending ict4d studies.mis quarterly, 39(3), 697712. australasian journal of information systems gregor et al. 2023, vol 27, research article indigenous knowledge-sharing 24 mayerhofer, p., & taylor, a. (2010). the influence of ict on indigenous mobility. population studies research brief, no. 2010059. https://digitalcollections.cdu.edu.au/nodes/view/4625?keywords=&type=all&lsk=3171 09e096a532720b98cdcf41d264bc mitchell, a. g., diddo, j., james, a. d., guraylayla, l., jinmarabynana, c., carter, a., rankin, s. d., djorlom, g., coleman, c., scholes, m., haynes, e., remenyi, b., yan, j., & francis, j. r. (2021). using community-led development to build health communication about rheumatic heart disease in aboriginal children: a developmental evaluation. australian and new zealand journal of public health, 45(3), 212–219. doi: 10.1111/17536405.13100 ngaanyatarra pitjantjatjara yankunytjatjara women’s council aboriginal corporation., (2021). traditional healers of central australia: ngangkari. magabala books, broome, australia. olsen, a., wallace, j., & maher, l. (2014). responding to australia’s national hepatitis b strategy 2010-13: gaps in knowledge and practice in relation to indigenous australians. australian journal of primary health, 20(2), 134–142. doi: 10.1071/py12162 osmond, g., & phillips, m. g. (2019). yarning about sport: indigenous research methodologies and transformative historical narratives. the international journal of the history of sport, 36(13-14), 1271-1288. paré, g., trudel, m. c., jaana, m., & kitsiou, s. (2015). synthesizing information systems knowledge: a typology of literature reviews. information & management, 52(2), 183-199. parker, e. j., misan, g., chong, a., mills, h., roberts-thomson, k., horowitz, a. m., & jamieson, l. m. (2012). an oral health literacy intervention for indigenous adults in a rural setting in australia. bmc public health, 12(1), 1-7. peters, m. d., marnie, c., tricco, a. c., pollock, d., munn, z., alexander, l., ... & khalil, h. (2020). updated methodological guidance for the conduct of scoping reviews. jbi evidence synthesis, 18(10), 2119-2126. radoll, p. (2010). stone chips to silicon chips: a grounded theory of information and communication technology adoption in australian indigenous households, rural, urban and remote. unpublished doctoral dissertation, australian national university, canberra, australia. schillhorn, k. (1999). the status and rights of indigenous peoples in australia. heidelberg journal of international law, 59, 443-462. https://www.zaoerv.de/59_1999/59_1999_2_a_443_462.pdf. sein, m. k., henfridsson, o., purao, s., rossi, m., & lindgren, r. (2011). action design research. mis quarterly, 35(1), 37-56. shaw, g., brereton, m., & roe, p. (2014, december). mobile phone use in australian indigenous communities: future pathways for hci4d. in proceedings of the 26th australian computer-human interaction conference on designing futures: the future of design (pp. 480-483). https://doi.org/10.1145/2686612.2686688 sim, i. m. (1966). rock engravings of the macdonald river district, nsw (no. 7). australian institute of aboriginal studies, canberra, act, australia. australasian journal of information systems gregor et al. 2023, vol 27, research article indigenous knowledge-sharing 25 spillman, l. (2018). meta-organization matters. journal of management inquiry, 27(1), 16–20. doi: 10.1177/1056492616688856 suchowerska, r., barraket, j., qian, j., mason, c., farmer, j., carey, g., ... & joyce, a. (2020). an organizational approach to understanding how social enterprises address health inequities: a scoping review. journal of social entrepreneurship,11(3), 257-281. thomas, d., johnston, v., & fitz, j. (2010). lessons for aboriginal tobacco control in remote communities: an evaluation of the northern territory ‘tobacco project’. australian and new zealand journal of public health,34(1), 45-49. doi: 10.1111/j.1753-6405.2010.00472.x. pmid: 20920104. tschirhart, c., mistry, j., berardi, a., bignante, e., simpson, m., haynes, l., ... & jafferally, d. (2016). learning from one another: evaluating the impact of horizontal knowledge exchange for environmental management and governance. ecology and society, 21(2), article no. 41. verran, h., & christie, m. (2007). using/designing digital technologies of representation in aboriginal australian knowledge practices. human technology: an interdisciplinary journal on humans in ict environments, 3(2), 214-227. waller, l., mccallum, k., & gorringe,s. (2018).resisting the truancy trap: indigenous media and school attendance in ‘remote’ australia. postcolonial directions in education, 7(2), 122–147 walter, m., (2022). weaving stories and data to decolonise methodologies. in tebrakunna country., lee, e., & evans, j. (eds.), indigenous women’s voices, 189-203. zed books, london uk. weiss, k., hamann, m., & marsh, h. (2013). bridging knowledges: understanding and applying indigenous and western scientific knowledge for marine wildlife management. society & natural resources, 26(3), 285-302. welsh, b. (2018). the hepatitis c trilogy: a case for indigenous theatre as a contemporary manifestation of traditional healing business. australasian drama studies, 73, 20-41. whitehouse, h., watkin lui, f., sellwood, j., barrett, m. j., & chigeza, p. (2014). sea country: navigating indigenous and colonial ontologies in australian environmental education. environmental education research, 20(1), 56-69. wilson r. l., & waqanaviti k. (2021). navigating first nations social and emotional wellbeing in mainstream mental health services. yatdjuligin: aboriginal and torres strait islander nursing and midwifery care, 281–306. doi: 10.1017/9781108894166.015 wilson, s. (2008). research is ceremony indigenous research methods. fernwood, winnipeg, man, canada. world bank. (2022). indigenous peoples. https://www.worldbank.org/en/topic/indigenouspeoples yunkaporta, t., & mcginty, s. (2009). reclaiming aboriginal knowledge at the cultural interface. the australian educational researcher, 36(2), 55-72. australasian journal of information systems gregor et al. 2023, vol 27, research article indigenous knowledge-sharing 26 apenndix a table a1 example showing charting for one primary study charting schema data extracted* general information title mitchell, a. g., diddo, j., james, a. d., guraylayla, l., jinmarabynana, c., carter, a., ... & francis, j. r. (2021). using community-led development to build health communication about rheumatic heart disease in aboriginal children: a developmental evaluation. australian and new zealand journal of public health, 45(3), 212-219. year of publication 2021 discipline area health type of study (primary or nonprimary) primary funding/sponsor body australian government under the strong communities for children program context aim of the study/problem addressed health rheumatic heart disease state/region of australia (remote/reserve, regional, urban) remote community town, nt underpinning theory (if any)/policy/perspective principles of community-led development specified indigenous ways of knowing, being and doing (if any yes “cultural ways of learning” intervention (project/activity) nature/type (overarching) curriculum for rheumatic heart disease lessons ict-based (yes/no) no who was involved in development (indigenous, non-indigenous, mix) multidisciplinary team including aboriginal members beneficiaries (target audience) every school-aged child in remote community mechanisms (process steps) how was the intervention/ project/ activity done (i.e., the design, development, implementation process) (i) “both-ways learning” with careful iterative discussions led to development of the “story-line”, a narrative for information dissemination [story telling] (ii) exposure to rhd data was a catalyst; (iii) conceptual and language issues meant meaningbased translation methods; (iv) indigenous knowledge and metaphors used (e.g white blood cells as warriors) [metaphor/analogy] australasian journal of information systems gregor et al. 2023, vol 27, research article indigenous knowledge-sharing 27 what artifacts/tools were involved/produced school children produced t-shirts with graphics, information posters did any of the artifacts used/produced involve ict yes data from rhd database outcomes positive outcomes some children showed new responses and attitudes to skin infections and rhd treatment. teachers used natural social networks to disseminate new information. community interagency collaboration to prevent rhd commenced. negative outcomes none noted evaluation evaluation methods used developmental evaluation with qualitative data gathered throughout project who did the evaluation (internal/external) internal * secondary coding is shown in square brackets copyright: © 2023 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v27i0.4217 http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 indigenous australians and knowledge-sharing 2.1 overview 2.2 indigenous ways of knowing and communicating 2.3 indigenous australians’ use of information and communications technology 2.4 knowledge sharing interventions 3 research approach 3.1 method 3.2 search strategy 3.3 exclusion criteria 3.4 study selection 3.5 data extraction 4 results 4.1 overview of included studies 4.2 charting 4.3 nature of interventions developed 4.4 indigenous ways of knowing 4.5 use of ict in interventions 5 discussion and conclusions paper title (paper title style) australasian journal of information systems grünloh et al. 2022, vol 26, research article opportunities, ethical challenges, and value implications 1 opportunities, ethical challenges, and value implications of pervasive sensing technology for supporting older adults in the work environment christiane grünloh roessingh research and development, enschede, the netherlands & biomedical signals and systems group, university of twente, enschede, the netherlands c.grunloh@rrd.nl miriam cabrita roessingh research and development, enschede, the netherlands & biomedical signals and systems group, university of twente, enschede, the netherlands carina dantas innovation department, cáritas diocesana de coimbra, coimbra, portugal sofia ortet innovation department, cáritas diocesana de coimbra, coimbra, portugal abstract responding to the challenges of demographic change, a growing number of ehealth solutions are appearing on the market, aiming to enable age-friendly living and working environments. pervasive sensing and monitoring of workers' health-, behavioural-, emotionaland cognitive status to support their health and workability enable the creation of adaptive work environments and the provision of personalised interventions. however, this technology also introduces new challenges that go beyond user acceptance and privacy concerns. based on a conceptual investigation and lessons learnt within the smartwork project (h2020-826343), this paper outlines opportunities and ethical challenges of pervasive sensing technology in the work environment that aims to support active and healthy ageing for office workers in a holistic way, including their values and preferences. only by identifying those challenges, implicated values and value tensions is it possible to convert them into design opportunities and find innovative ways to address identified tensions. the article outlines steps taken within the project and closes with a reflection on the limits of technological responses to societal problems and the need for regulations and changes on a societal level. keywords: ethical challenges, smart work environment, human values, healthy ageing, personalised interventions, digital health. 1 introduction the challenges of demographic change and an ageing population are manifold, such as the increasing number of people living with chronic conditions (world health organization, 2010b), of workers with functional impairments (liang et al., 2008), of people taking the role of informal caregivers along with their job (papastavrou, charalambous, & tsangari, 2012), and also the gradually increased retirement age (komp, 2018), and rising costs in healthcare (harbers & achterberg, 2012). older workers tend to remain longer at work than previously, however, involuntary early retirement occurs and is associated with negative effects, such as health, financial status, and adjustment to retirement (topa, depolo, & alcover, 2018). a recent meta-analysis recommended that organisations should avoid forcing workers into earlier australasian journal of information systems grünloh et al. 2022, vol 26, research article opportunities, ethical challenges, and value implications 2 retirement, because “the adverse effects on the individuals, their families, and health and social protection systems are disproportionate in comparison with the economic gain of this measure” (topa et al., 2018). simultaneously, the large proportion of older adults gives rise to opportunities, for example, opportunities connected to making use of all the knowledge, experience and wisdom of older workers. these could be shared with or transmitted to younger generations through adequate tools, facilitating a smooth transition to the retirement age. to make sure that the experience and know-how of older adults can be applied in the working environment until their retirement age, there is a need to create age-friendly working environments. as adopting and maintaining a healthy lifestyle can be challenging, ehealth – also referred to as digital health – solutions can play an important role in supporting older adults to stay active and healthy for as long as possible. technological interventions in healthcare, consumer health, and the lifestyle market have the potential to support individuals in improving their lifestyle, monitoring their progress, and taking action. the idea of “self-knowledge through numbers” is at the heart of the quantified self movement which started in 2007 (sharon, 2017). the availability of affordable wearable devices that track physical activity, sleep, standing time and other aspects made self-tracking even more accessible. furthermore, ehealth solutions can support the individual in the difficult process of adopting and maintaining healthy behaviour, for example, by visualising the progress and providing personalised coaching (jansen-kosterink et al., 2020). ehealth tools for monitoring and coaching of healthy lifestyle are now more important than ever before. this is reflected in public health policies urging people to take care of their health, and the healthcare sector moving from the more traditional approach with the healthcare professional as expert to the empowerment approach, which considers patients as active and equal partners in their care (funnell, arnold, donnelly, & taylor-moon, 1991). the shift in healthcare from treatment to prevention also addresses challenges in the workplace, as experienced workers who are able to maintain healthy and active can stay longer in their job and their valuable knowledge is not lost. as some biological changes are inevitable with increasing age, the work environment can adapt to this and, for example, not add any additional workload on an individual who is already working at the capacity limit. however, bringing digital health solutions to the workplace does not come without ethical challenges (e.g., the tension between workers' privacy and the employers' need to control and monitor the use of technology (iavicoli, valenti, gagliardi, & rantanen, 2018). given the sensitive nature of health data, employees are concerned about sharing personal health data as it can have unexpected consequences (e.g., social pressure, social expectation management and crossing boundaries between private life and workplace (gorm & shklovski, 2016). furthermore, the introduction of technology and information systems have changed working conditions and shifted the balance of power towards the employer and away from employees (dantas, van staalduinen, machado, jegundo, & rodrigues, 2019). adding another monitoring layer via pervasive sensing to this power dynamic might negatively impact the work environment and impede technology acceptance. information systems research has contributed for decades to the literature of health information systems (chen, baird, & straub, 2019), work systems (alter, 2013), ethical information systems (mingers & walsham, 2010), and information systems that support human values (friedman, kahn jr, & borning, 2006). this article contributes to the field of information systems by conceptually investigating human values and tensions that are australasian journal of information systems grünloh et al. 2022, vol 26, research article opportunities, ethical challenges, and value implications 3 implicated when developing and introducing pervasive health sensing technology to promote work sustainability among older adults. 2 smartwork project the european research and innovation action project smartwork is part of the horizon 2020 programme and funded under the topic ‘adaptive smart working and living environments supporting active and healthy ageing’. the ultimate aim of the smartwork project is to build a worker-centric system to support the sustainability of workability, in other words, to help the older adult to work for as long as desired. to achieve this, the smartwork system integrates unobtrusive sensing and modelling of the worker state using a suite of novel services for contextand worker-aware adaptive work support. the system consists of six separated services; four targeting the older worker (healthyme, myworkability, ubiwork, workcoach), one targeting the manager/supervisor (digiteam) and the final one targeting formal/informal carers (icare): • healthyme: continuous, unobtrusive, and ubiquitous monitoring of physiological and behavioural parameters to support self-management of chronic conditions, promote behaviour change and improve quality of life of the older worker; • myworkability: continuous assessment of the psycho-physical capacity of the office worker to provide ai-based decision support by predicting short and long-term changes in capabilities and abilities of the office worker and translating such changes into evolving work requirements; • ubiwork: support on-the-fly work flexibility through a computer work environment available anywhere and anytime; • workcoach: on-demand training support and new skills acquisition to support the older worker prolong his/her functional workability; • digiteam: smart and flexible management of the workforce to increase efficiency and productivity of teams working on specific tasks, and to optimize training and knowledge management activities; • icare: efficient continuous care management and health risk assessment to be provided to caregivers. an unobtrusive sensor network containing multiple sensing components facilitates the collection, storage and distribution of health, behaviour, cognitive and emotional information to all six smartwork services. given its ambition and complexity, the work performed within the smartwork project and, in particular, some of the discussions that arose between end-users and technical partners, are taken as starting points to explore the opportunities and ethical challenges of pervasive sensing technology in the work environment. 3 theoretical background the workplace has been established by the world health organisation (who) as one of the priority settings for health promotion in the 21st century (world health organization, 2010a). as most individuals spend a large part of their awake time at work, the workplace offers the setting and infrastructure for health promotion activities. personal health technology (e.g., wearables) are increasingly introduced in the workplace setting, for example, as part of health australasian journal of information systems grünloh et al. 2022, vol 26, research article opportunities, ethical challenges, and value implications 4 promotion campaigns or insurance discount programs (gorm & shklovski, 2016). health promotion in the workplace brings numerous benefits to the employee, namely reduced stress, increased job satisfaction and enhanced self-esteem, which in turn translates to benefits to the organisation, such as reduced staff turnover, reduced absenteeism, and increased productivity (world health organization, 2010a). workplace health promotion programs support prevention efforts that target the generally healthy population who do not maintain good health (e.g., encourage exercise, fitness, healthy nutrition, weight, stress management) or those that are already at high risk due to certain lifestyle practices (e.g., smoking, sedentary lifestyle, poor nutrition, high stress) (goetzel & ozminkowski, 2008). a recent review of workplace health programs between 2000 and 2020 showed that these mostly address physical activity, followed by interventions to promote mental health and stress management, smoking cessation and ergonomics and musculoskeletal disorders (maría-ángeles, maqueda, francisco, maría-jesús, & maría-dolores, 2021). for the ageing population, health promotion at the workplace brings specific opportunities by supporting the sustainability of workability through digital interventions, as addressed in the following three sections. this is followed by a section outlying the data protection principles in europe, as this is the regulatory framework within which the smartwork project operates when collecting and processing large amounts of various types of data. the section closes with a brief overview of approaches that account for human values in technology design. 3.1 supporting the sustainability of workability ageing is only loosely associated with a person’s age in years and there is a wide range of needs and experiences of older adults (world health organization, 2021). there is a set of common age-related changes often starting around the age of 50 to 55, with implications to the daily life of the individual, such as vision-, hearingand psycho-motor decline (liang et al., 2008). according to the world health organization (who), by 2030, 1 in 6 people in the world will be aged 60 years and older, and by 2050, 80% of the world’s population over 60 years will be living in lowand middle-income countries (world health organization, 2021). while the proportion of older people is continuously growing and with it the burden of chronic conditions and the occurrence of multimorbidity (marengoni et al., 2011), chronic health problems also increasingly affect young and middle-aged people (world health organization, 2010b). ischaemic heart disease, stroke, and diabetes emerged as major contributors to the burden of disease in the age group 25-49 (vos et al., 2020). hence, in this work we do not provide an age threshold for the “older worker”, but instead leave it open to individuals experiencing a health-related decline that impacts their working ability. chronic diseases have a great impact on the working population and society. for instance, in the european union, more than half a million people aged between 25 and 64 years died from major non-communicable diseases in 2013, representing a loss of around 3.4 million potentially productive life years across the former 28 eu countries (oecd / european union, 2016). however, in most cases, people do not die from chronic diseases but live with the disease for long periods of their life. according to the latest global burden of diseases, injuries, and risk factors study (gbd), despite steadily improved global health, half of the burden of disease was related to non-communicable diseases and injuries (vos et al., 2020). prevention and selfmanagement of chronic diseases are now becoming central in health promotion activities, also at the workplace. an active lifestyle, healthy nutrition, no smoking and limited alcohol consumption are some of the lifestyle factors which impact the course of chronic disease. australasian journal of information systems grünloh et al. 2022, vol 26, research article opportunities, ethical challenges, and value implications 5 digital health solutions provide the means to deploy interventions taking a holistic perspective of health, bridging the gap between the work and home environment, while respecting a person’s preferences and values. in this sense, to the older worker, health promotion at the workplace using digital health solutions has the specific aim of maintaining workability for a longer period of time, allowing individuals to sustain their working activities until the retirement age. 3.2 digital behaviour change interventions in the context of this article, digital behaviour change interventions are a particular type of digital health solution that aims to support an individual in maintaining or achieving a healthier lifestyle. a specific characteristic of this type of intervention is that they allow communication with the user at any time, being at work, on the go or at home. digital behaviour change interventions have three core components: sensing, analysis, and coaching (cabrita, op den akker, tabak, hermens, & vollenbroek-hutten, 2018). the sensing component concerns the acquisition of the data. the data acquisition can be unobtrusive – through the use of wearable or environmental sensing – but can also be achieved by selfreported input from the individual. at the workplace, one can think of assessing the workload by monitoring cardiac parameters with a low-cost webcam (bousefsaf, maaoui, & pruski, 2014), the stress detection with physiological and sociometric sensors (mozos et al., 2017), or a pressure-sensitive keyboard and use of a capacitive mouse to discriminate between stressful and relaxed conditions (hernandez, paredes, roseway, & czerwinski, 2014). the second component of digital behaviour change interventions is the analysis of the data and deals with the question “how to make sense of the data collected?”. usually, digital behaviour change interventions deal with heterogeneous data, which originate from various sources at different sampling frequencies and sometimes even seem to provide contradictory information. all-inall, several challenges arise for algorithms aiming at the analysis of health-related data, with methods from data science taking growing attention in the field. the third and final component, coaching, concerns the techniques used to motivate the user to change behaviour, often based on the data collected and all information available. while the first two components are mostly determined by state-of-the-art technology, coaching deals with the question of how to motivate the user to change the behaviour. digital behaviour change interventions aim to persuade users to alter their routines in a way that sometimes users themselves are not willing or motivated to do. the first step of the behaviour change process is often convincing the individual about the need to change, followed by support on how to achieve the desired behaviour and, finally, to provide adequate feedback to motivate the user to stay in the desired behaviour and not to fall back into old habits. 3.3 digital health interventions at the workplace so far, most digital health interventions at the workplace have targeted improvements in the mental health of employees. stratton and colleagues performed a meta-analysis investigating the effectiveness of digital health interventions targeting the mental health of employees (stratton et al., 2017), in which most made use of digital health interventions in the form of education or exercises (e.g., guided meditation), with minimal objectively or subjectively measured input from the participant. digital health interventions also have a great potential in improving physical health at the workplace, for example, by targeting the management of occupational physical activity. while for office workers the aim is on reducing sedentary time, for manual workers the focus should be on preventing or managing musculoskeletal australasian journal of information systems grünloh et al. 2022, vol 26, research article opportunities, ethical challenges, and value implications 6 disorders, for example, promoting management of physical activity between occupational and leisure time (holtermann, hansen, burr, søgaard, & sjøgaard, 2012). commercial solutions are also appearing with the target of health promotion at the workplace. one example is fitbit health solutions from fitbit inc., which provides corporate programs to promote health and well-being in organisations 1. fitbit health solutions combines the use of tracking devices to measure sleep and heart rate with self-reported information (e.g., on diet habits), and offers specialised coaching programs to promote healthy lifestyles. another example is the minddistrict solution, which provides an ehealth platform to promote mental health at the workplace 2. in minddistrict, the employee is invited to choose from a set of predefined programs based on positive psychology principles, such as “building confidence”, “finding balance” and “living an active and healthy life”. fitbit health solutions and minddistrict are two examples of ehealth solutions facilitating health promotion at the workplace initially motivated by the consumer market and academic research, respectively. 3.4 data protection principles and regulations in europe the appreciation of the holistic nature of a person requires the collection of large amounts of various types of data. smartwork’s unobtrusive sensing network processes a considerable amount of newly generated personal data and also involves the secondary processing of previously gathered personal data from relevant big data registries (e.g., the english longitudinal study of ageing, elsa, phelps et al., 2020). the key legislation under which the smartwork project must operate is the general data protection regulation 2016/679 (gdpr) (general data protection regulation, 2016). the gdpr is the branch of human rights protecting the rights of the study participant and concerns personal data that are processed in the eu (where processing includes very broadly, any action on personal data). although multiple parties might potentially claim data (subject, researchers, enterprises), it should be acknowledged that the individual has the right to be clearly informed on why and how data is collected, about potential direct implications to the user (including risks and gains), as well as on its protection procedures on data collecting, storing, processing, transfer, and reporting (including the eventual use of secondary data). the gdpr makes clear several rights and principles that must apply to the processing of personal data (art. 5). the data must be processed “lawfully, fairly, and in a transparent manner” (art. 5.1a), and for specific purposes, and not further processed in an incompatible manner (art. 5.1b). article 6.4 provides that it is possible to process data for further compatible purposes, but with safeguards. the principle of data minimisation requires that only personal data sufficient for the purpose can be processed (article 5.1c) and that it should be accurate (art. 5.1d). and under article 5.1f, “personal data shall be processed in a manner that ensures appropriate security of the personal data, including protection against unauthorised or unlawful processing and accidental loss, destruction or damage, using appropriate technical or organisational measures”. furthermore, chapter 3 outlines the rights of the data subject, for example, right of access by the data subject (article 14), right to erasure (also called ‘right to 1 https://healthsolutions.fitbit.com/ 2 https://www.minddistrict.com/ https://healthsolutions.fitbit.com/ https://www.minddistrict.com/ australasian journal of information systems grünloh et al. 2022, vol 26, research article opportunities, ethical challenges, and value implications 7 be forgotten’, article 17), right to restriction of processing (article 18), or the right to object (article 21). it can be seen that the data collected and processed in the smartwork project falls squarely within the gdpr definitions and thus poses challenging questions. an essential activity within the smartwork project is to reflect on the ethical challenges the potential solutions pose and on how to conform with principles and regulations (dantas et al., 2019). any system that supports workability needs to observe both the spirit and letter of these principles and the way that they are interpreted and developed into implementation. 3.5 accounting for human values in design ehealth technology can be conceptualised as a tool that supports people to perform certain tasks (i.e., it realizes functional requirements, for example, scheduling an appointment with a healthcare professional). this conceptualisation of technology as being merely a tool might imply that a specific design is not inherently moral, that morality depends on its use and hence technology would be value-neutral. however, over the past decades, there is growing consensus that technological systems cannot be considered value-neutral and that design as activity is value-laden (friedman & kahn jr, 2007; jafarinaimi, nathan, & hargraves, 2015; miller, 2020). depending on the particular design or how the technology is used, the ehealth system may support or hinder human values (e.g., supporting trust and transparency through patient accessible clinical notes (cromer et al., 2017)). different approaches emerged that support human values through systems design, for instance, computer ethics, social informatics, computer supported cooperative work and participatory design (friedman & kahn jr, 2007). other approaches focus on values more in the economic sense, such as value-based software engineering (whittle, ferrario, simm, & hussain, 2021) or worth-centred design (cockton, 2020). value sensitive design (vsd) employs a broader meaning of values, which refer to “what is important to people in their lives, with a focus on ethics and morality” (friedman & hendry, 2019). vsd is a theoretically grounded approach that accounts for human values in a principled and comprehensive way throughout the design process by conducting empirical, conceptual, and/or technical investigations (friedman et al., 2006). although some values have been identified to be basic or culturally universal (schwartz, 2012), empirical investigations are crucial to examine which values are affected in what way in the concrete setting (i.e., what people, in a particular context, consider important in life and how they solve or prioritise value tensions that may arise in a particular situation). the importance of empirical investigations is also reflected in the core commitment of value sensitive design, namely taking an interactional perspective in which values are not embedded in technology but implicated through engagement with it (davis & nathan, 2015). this means that “although the features or properties that people design into technologies more readily support certain values and hinder others, the technology’s actual use depends on the goals of the people interacting with it” (friedman et al., 2006). technology related to health and well-being in the workplace combines two complex applications domains: healthcare and the work environment. each domain is represented by a multitude of stakeholders who have different interests and values that might be affected by the design. as this theoretical background section shows, ehealth technology approaches for workability sustainability and behaviour change interventions, as well as data protection principles, are available to support the design and development of pervasive health sensing technology in the workplace. however, little is known about the ethical challenges, implicated australasian journal of information systems grünloh et al. 2022, vol 26, research article opportunities, ethical challenges, and value implications 8 values, and value tensions when it comes to the actual shaping of functions and features and the technology usage in practice. the ethical challenges identified and discussed in this paper reveal human values and tensions that may arise due to different interests and perspectives of stakeholders involved, as well as technological and contextual factors. 4 method a conceptual investigation of human values and tensions that are implicated was conducted in the context of developing and introducing pervasive health sensing technology in the workplace for older adults. this analysis was performed from the backgrounds of the different authors involved in the project (biomedical engineering, human-computer interaction and media informatics, psychology, research and innovation project management, law and ethics), utilizing available literature, empirical investigations conducted within the project and discussions and decisions made in the course of the project. furthermore, writing was also part of the analysis, as writing is considered in most qualitative research as an “integral element of the analytic process” (clarke & braun, 2013) and “the process through which the analysis develops in its final form” (braun & clarke, 2013). 5 results every new technological innovation has the potential to solve problems, but at the same time may introduce new challenges and problems, as these technologies are implemented in a socio-technical context that has its own rules and constraints. this section outlines a nonexhaustive list of opportunities, ethical challenges, human values and value tensions in the context of a smart work environment, that we identified within the smartwork project. 5.1 human well-being and privacy in the smart workplace the value of human well-being was defined by brey (2015) as “a state of persons which designates that they are happy or flourishing and that their life is going well for them.” depending on the underlying philosophical theory, this well-being can mean the absence of pain and the presence of pleasure (hedonist theories), the fulfilment of a person’s desires (desire-fulfilment theories), or well-being is conceptualised in terms of a list of objective goods (objective list theories) (brey, 2015). other concepts that are often used in healthcare (e.g., quality of life) or the workplace (e.g., work-life balance) can be subsumed also under the value of human well-being. the value of privacy refers to “a claim, an entitlement, or a right of an individual to determine what information about himself or herself can be communicated to others” (friedman et al., 2006). warnier, dechesne, and brazier (2015) identified three aspects that capture the main points associated with the concept of privacy: freedom from intrusion (the right to be left alone), control of information about oneself, and freedom from surveillance (the right not to be tracked, followed, or watched in one’s own private space). the ubiwork service of smartwork aims to facilitate flexible workplaces, meaning that older workers can engage in work-related activities at the place of their best convenience. flexible workplaces have many advantages that may contribute to a person’s well-being. for example, it can contribute to a better work-life balance and accommodate the personal life situation of workers (e.g., allowing them to take care of their children, family members, or others). nowadays, borders between work and private life are continuously crossed, especially as technology has spread from the workplace to our homes and everyday lives and culture australasian journal of information systems grünloh et al. 2022, vol 26, research article opportunities, ethical challenges, and value implications 9 (bødker, 2015). recently, working from home has increased significantly across occupational groups due to the covid-19 pandemic where many had to shift towards working from home for the first time, demanding them to quickly adapt to this new arrangement (kramer & kramer, 2020). mobile devices (laptops, tablets, smartphones, smartwatches) allow employees to be available around the clock and to work from home. thus, in a smart work environment that extends to a home office, data has to be collected while they are in their home environment which increases the chance that also non-work-related data will be collected. in addition, digital behaviour change interventions within the healthyme service require sensing of lifestylerelated data in the home environment. take the example of stress management interventions by improving sleep quality and sleep duration, which naturally requires the sensing of sleeprelated data. this, however, introduces a privacy challenge to work-related health promotion activities, as the data collection has to take place also in very private areas of people’s lives. next to blurring the boundary between work and private life, data collection for a smart work environment also blurs the boundary between healthand productivity-related data. in smartwork, the myworkability service aims to infer the workability of the older worker at any point in time. the workability is determined, among other factors, by the health status of the worker. this service takes into account previous and existing (chronic) conditions which, for example, due to an exacerbation, might impact the workability of an individual at a certain moment. the aim is to achieve a good balance between the physical and mental strain caused in the job and the employee’s working capacity and therefore contributes to the person’s wellbeing. this leads us to what is likely one of the strongest ethical challenges of pervasive sensing in a smart work environment: sharing health-related data with the manager or employer. healthrelated data is always sensitive and very personal, therefore well protected in ethical and privacy regulations. however, when it comes to a smart work environment, the line between health and productivity-related data can become blurred, especially the ones related to stress and the ways of measuring stress and workload. this creates a tension between the values of well-being (which would be supported by tailoring the workload to the levels of stress) and privacy (in case a worker does not want to have their health data collected and processed). to illustrate this challenge, it is worth mentioning that this issue was iteratively discussed within the smartwork project and was the basis for the decision of not sharing any data that may be associated with health or personal performance with the employers. if the work environment is supposed to adapt flexibly to the needs of older office workers, information on the health status and their workability has to be shared in some shape or form. how can a solution determine the personal needs in a non-invasive way? moreover, how to ensure any data collected cannot be used as an asset in the power relationship between employer and employee for a different reason than the improvement of the working conditions? defining a system where these two challenges are overcome will bring a major step forward for the effective implementation of holistic working and living environments. 5.2 data ownership, control, access, and use another major challenge to the smartwork system implementation is connected to the values of ownership: who owns the data? who is allowed to access and make use of information? ownership and property have been defined as “a right to possess an object (or information), australasian journal of information systems grünloh et al. 2022, vol 26, research article opportunities, ethical challenges, and value implications 10 use it, manage it, derive income from it, and bequeath it” (friedman, kahn, borning, & huldtgren, 2013). concerning intellectual property, lipinski & britz discuss the dual nature of information ownership, which supports the right to control (i.e., ownership) as well as the right of access to information (lipinski & britz, 2000). this means that although ownership involves the right to control the access to information, others may possess the right of access to that information as well. as mentioned previously, having control of information about oneself is key and freedom from surveillance (the right not to be tracked, followed, or watched in one’s own private space) are important aspects of privacy (warnier et al., 2015). furthermore, aspects that are related to people having control over their information are also covered within the gdpr (see section 3.4). one challenge concerning the right to control data access is closely related to the difficulties of data anonymisation or pseudonymisation. employees might agree to have their data collected for personal reflection and self-management of their health but might prefer this to be accessible only to themselves (lidynia, brauner, & ziefle, 2018). whether a worker consents to sharing information with their supervisor is likely to be also influenced by cultural factors. in a survey conducted among danish (n=49) and portuguese (n=50) workers as part of smartwork’s requirements elicitation activities, the two groups showed different attitudes towards the potential functionality that allows their employer to be informed about their performance or ability. danish workers considered informing the employer on worker performance (84%) or workability (68%) not very or not at all useful. however, more than half of the portuguese workers in the survey considered these functions useful or even very useful (62% informing on worker performance, 66% informing on workability) (smartwork, 2019). considering that the digiteam service of smartwork intends to enable flexible management of the workforce, to increase efficiency and productivity (from an employer’s perspective), and also to optimise training and knowledge management activities, data related to productivity must somehow be made available to managers. besides the potential underlying ethical and data protection issues, there is also a clear and challenging overlap, since the data that contribute to stress assessment is not only related to the productivity of workers but also their health and well-being. in other words: although health data will not be shared with the managers as such, they are an important factor to assess the current stress level and hence workability. as previously referred, within the smartwork project, privacy concerns were prioritised leading to the decision not to share this data with employers, but this is clearly a shortcoming to the full exploitation of digiteam’s potential. related to the ownership of data, another challenge concerns where that data will be stored. an exploratory study by lidynia and colleagues showed that storing data on a company server was not favourable and participants preferred to keep the data to themselves (lidynia et al., 2018). within the aforementioned smartwork survey with dutch and portuguese workers, only a small percentage of the respondents (11%) revealed some apprehension towards their organisation’s privacy and data protection policy (smartwork, 2019). in a recent study in the netherlands, around seven in 10 workers reported having some or a great deal of trust in their employer’s ability to keep their personal health information secure (mercer, 2020). this might, however, be viewed differently in other countries. 5.3 autonomy and positive health interventions to support well-being autonomy has been defined as the ability of people “to decide, plan, and act in ways that they believe will help them to achieve their goals“ (friedman et al., 2013). within the medical australasian journal of information systems grünloh et al. 2022, vol 26, research article opportunities, ethical challenges, and value implications 11 domain, patient autonomy encompasses the right to self-determination (i.e., the right to make their own decisions without interference from others), and psychological autonomy (i.e., the capacity of persons to exercise the right to self-determination) (katz, 2002). the main aim of positive health interventions is to support human well-being (see definition of the value of human well-being in section 5.1). health can be seen as one aspect of well-being. within the smartwork project, health is not just seen as the state of complete physical, mental and social well-being (as defined by the world health organization, 1948), but adopts the more recent “positive health” concept by huber and colleagues. this new concept of health as “the ability to adapt and to self-manage, in the face of social, physical and emotional challenges” acknowledges the fact that living and ageing with a chronic disease has become a norm (machteld huber et al., 2011; m huber et al., 2016). this holistic concept of positive health encompasses six dimensions: bodily functions, mental functions and perception, spiritual/existential dimension, quality of life, social and societal participation, and daily functioning (m huber et al., 2016). similar to the acknowledgement of health being multidimensional, the smartwork project adopts a transdisciplinary approach and addresses six main intervention domains: 1. healthcare interventions for health self-management; 2. behavioural interventions to promote physical activity, healthy nutrition and wellbeing; 3. cognitive function interventions to maintain cognitive capacity; 4. work management interventions to reduce work-related stress; 5. work-related training interventions, to facilitate explicit/ implicit knowledge; acquisition and intergenerational transfer; 6. work environment adaptations. within each intervention domain, pervasive monitoring of the older workers’ health, their behaviour, and their cognitive and emotional status will take place through the unobtrusive sensor network. in addition, older workers will be asked to report or validate their status on well-being parameters at work regularly. that way, the smartwork system can assess or validate the risks of functional or cognitive decline, which may lead to absence from work or early retirement. by detecting those risks, the system will recommend to the worker possible interventions (e.g., behaviour change) in form of advice and guidance, as the smartwork system aims to complement the user’s own observations rather than providing diagnosis, prevention or cure. balancing the number of interventions that contribute to a person’s health in the long run with their perceived well-being, values, and preferences in the here and now, while respecting their autonomy presents another challenge. adopting the holistic “positive health” concept by huber implies that potentially several interventions could be useful for a given employee at the same moment to account for the multi-dimensional aspects of active and healthy ageing. however, the number of interventions towards older adults’ ability and sustainability at work may bring as many benefits as potential threats. smartphone or digital notifications, for instance, have been recognised as a major distracting factor, due to the risk of constantly interrupting the ongoing workflow (ortet et al., 2019). additionally, combined with the repetitive interruptions from the data collection (e.g., by asking a worker to do selfaustralasian journal of information systems grünloh et al. 2022, vol 26, research article opportunities, ethical challenges, and value implications 12 assessments and report their experiences), these interventions are likely to create a spiral effect of additional anxiety and stress (ortet et al., 2019) and counteract their well-being. in other words, the same intervention that is supposed to contribute to positive health in one dimension (e.g., bodily functions) might have a detrimental effect on a person’s well-being in another dimension (e.g., mental functions and perception). furthermore, not all intervention needs can be addressed at the same time and some domains are strongly connected to and influence each other (e.g., exercise, sleep, and nutrition habits). some interventions might objectively contribute to a person’s physical well-being but interfere subjectively with a person’s perceived quality of life (for example, because even though a certain behaviour might be considered unhealthy, it’s important for that person). behaviour change interventions aiming at positive health can therefore be in tension with a person’s autonomy. in establishing ethical principles of persuasive design, the golden rule of persuasion has been defined as: “the creators of a persuasive technology should never seek to persuade a person or persons of something they themselves would not consent to be persuaded to do” (berdichevsky & neuenschwander, 1999). thus, one of the challenges for the smartwork system will be to unveil the most promising and adequate type of interventions for older office workers that are in line with their values and preferences, in an integrated and efficient way, without overburdening them while respecting their autonomy. 5.4 division of responsibility between employer and worker technology can be designed for the value of responsibility. the design of technology can affect a person’s responsibility in that a user assumes responsibility or is responsible for something, or that it distributes responsibility between individuals or groups (fahlquist, doorn, & van de poel, 2015). whether an individual is considered responsible for something or this is considered a collective responsibility might depend on the cultural context. for example, when it comes to stress at work, cultural differences have been identified whether the responsibility of moderating life stressors is placed on the individual or shared with the environment (i.e., the emphasis being placed on social causes of illness) (karasek & theorell, 1990). a smart work environment that takes the health of employees into account and supports them in living an active and healthy life can be seen as an extension of electronic performance monitoring (epm). epm includes the continuous collection, analysis, and reporting of information about an employee’s productive activities (smith, carayon, sanders, lim, & legrande, 1992). the suite of services in the smartwork project goes beyond epm by including monitoring of physiological and behavioural parameters that might, or might not, be directly related to the productivity of the worker. the type and mechanism of data collection (sensors and short-questionnaires) can put quite a burden on the worker. in addition to that, as the worker is also the recipient of interventions (coaching, notifications, reminders), this implies a responsibility that is put mainly on the individual. however, the personal behaviour of the worker is only one side of the story, as also the organisation or company has a responsibility to create an environment and design jobs in a way that enables healthy working in the first place (karasek & theorell, 1990). for example, while workplace wellness programs may lead to reported better health behaviours, the effect on clinical measures of health and economic outcomes may be limited (song & baicker, 2019). from the perspective of the worker, psycho-social factors such as job design, ability to make australasian journal of information systems grünloh et al. 2022, vol 26, research article opportunities, ethical challenges, and value implications 13 decisions and control over work are not only positive contributors to health in the work setting but seem even more relevant than individual lifestyle issues (dugdill, 2000). finally, the extensive monitoring of health and performance might also be in tension with an important aspect of privacy: freedom from surveillance, which as mentioned before is the right not to be tracked, followed, or watched in one’s own private space (warnier et al., 2015). 5.5 power relations at work, accountability, and trust power has been defined in different ways, for example, by max weber as “the probability that one actor within a social relationship will be in a position to carry out his own will despite resistance, regardless of the basis on which this probability rests” (weber, 1978). this rather negative perspective on the concept of power has been challenged, critiqued, amended, and extended since then (hardy & clegg, 2006). galinsky and colleagues define power as asymmetric control over valuable resources and outcomes within a specific situation and set of social relations, which implicitly involves both control over and independence from others in obtaining important outcomes (galinsky, magee, gruenfeld, whitson, & liljenquist, 2008). in other words, power is the capacity to be uninfluenced by others (galinsky et al., 2008). accountability refers to “the properties that ensures that the actions of a person, people, or institution may be traced uniquely to the person, people, or institution” (friedman et al., 2013). accountability is closely related to transparency, where accountability provides evidence to justify past actions to others and transparency is the tendency to be open in communication (hulstijn & burgemeestre, 2015). accountability/transparency and privacy are values that are considered typical for value tensions to occur (friedman et al., 2013). trust has been defined as a psychological state that represents the trusted person or object as being trustworthy and (because this may not actually be the case) involves accepting one’s vulnerability to others (nickel, 2015). in the context of online interaction, interpersonal trust means that “we are vulnerable to harm from others yet believe these others would not harm us even though they could” (friedman, khan, & howe, 2000). a smart work environment can support office workers in a flexible work setting to account for the work they did, even if not in the office and hence enables accountability and transparency of their contribution to the workforce. it may, for example, show that certain workers are very efficient in the morning vs. the evening, motivating accordingly an adaptation of work schedule and expected availability. a smart work environment that supports some form of performance monitoring can also help workers to reflect on their work, for example, in terms of how long certain tasks take them, which could lead to improving their planning skills. furthermore, workers receive a more objective measurement of how much they actually work, so that they might be able to set some adjustments (e.g., setting guilt-free boundaries by not checking emails at the weekend considering they already worked overtime). transparent and objective performance measures might increase the perception of fairness with regard to the division of workload. even in companies with rather flat hierarchies, control over resources by people in power still remains and so does the dependence of powerless to those in power. adding to this dependency health and productivity monitoring tools, which are part of the smartwork suites, can potentially have negative effects on relationships at work, job satisfaction and attitude. accountability/transparency can bring some negative effects, for instance, on the trust chain between management and employee. in a non-monitored work environment, management australasian journal of information systems grünloh et al. 2022, vol 26, research article opportunities, ethical challenges, and value implications 14 and employee have to establish trust in each other and in a balanced give-and-take. the opportunity to check in on a detailed level and the perception of monitoring as being unfair, abusive, or infringing on an employee’s rights (‘big brother’ is watching) can create an atmosphere of mistrust (alder, noel, & ambrose, 2006). the perception of monitoring, therefore, stands in contrast to the value of privacy. working conditions in which performance is electronically monitored can be perceived as more stressful and workers can experience an increased level of anxiety, depression, anger, health complaints and fatigue (smith et al., 1992), hence counteracting their well-being. a lack of trust that the technology is used to their benefit might lead to workers cheating the system (e.g., by pretending to be more active than they actually are) to avoid repercussions. following from this, another important aspect to consider is the way a smart work environment might affect the relationship between the worker and their colleagues and management. meaningful work and meaningful relationships are important aspects of the “positive health” concept (m huber et al., 2016) and hence a person’s well-being. another aspect related to power and control refers to the idea of opting-out (i.e., that participation in performance and health monitoring is voluntary), which would support the value of autonomy (see section 5.3). this concerns less the design as such and more how a system like smartwork would be introduced and used in the workplace. depending on the culture of the organisation, power dynamics and the relationship between co-workers, voluntary participation might be undermined by peer pressure. peer pressure is understood as “the social influence of members of a peer group leading others to take certain actions or adopt certain practices” (kieslinger, 2015). especially when opting-in is the norm, nonparticipation especially with regard to performance monitoring might be perceived as suspicious by co-workers. as described in the essay by solove (2007), the ‘nothing to hide’ argument is very popular in the public discourse when privacy is balanced against security. statements like “if you aren’t doing anything wrong, then what do you have to hide?” or “if you’ve got nothing to hide, what do you have to fear?” illustrate how accountability/transparency can be in conflict with privacy. however, this conflict or tension might be based on a very narrow view of privacy as a form of concealment or secrecy (solove, 2007). the negative perception might depend not only on functionalities but also on how the tools have been introduced and implemented in the particular workplace. for example, perceived organisational support and advanced notice can mitigate this negative view and even increase employee trust (alder et al., 2006). another factor that can reduce the negative effect of monitoring felt by employees is a focus on feedback and performance appraisal (chalykoff & kochan, 1989). the empirical study by chalykoff and kochan was one of the first to investigate employees’ viewpoints and responses to monitoring. their study showed that not the monitoring in itself, but how it is used in practice has significant effects on office workers’ general attitudes and behaviours, and that “managerial attention to recognised standards for performance appraisal, feedback, and good supervision can significantly reduce the otherwise negative effects of monitoring” (chalykoff & kochan, 1989). one should note, however, that employees who initially oppose monitoring per se might not change their minds even if the monitoring and feedback process is managed well (chalykoff & kochan, 1989). australasian journal of information systems grünloh et al. 2022, vol 26, research article opportunities, ethical challenges, and value implications 15 6 discussion this paper describes the conceptual investigation of human values and tensions present when introducing pervasive health sensing technology in the workplace. the work was conducted in the context of a european research and innovation action project and reflects the perspective of consortium members from different backgrounds, supported by literature and empirical investigations. we have identified and discussed eight key human values and value tensions that arose when designing and developing a system that promotes workability among older adults, namely human well-being, privacy, ownership, autonomy, responsibility, power, accountability, and trust. while the results are supported by literature and empirical investigations during the project, further empirical investigation with stakeholders after the technology has been implemented is needed to examine which values are affected when the smartwork services are used in the work context. the project targets older office workers in europe. it is unclear, whether the results (challenges, implicated values, and value tensions) are applicable in other countries or age groups that might also benefit from health promotion programs. however, the results can be used as a starting point for such conceptual investigations in other contexts. ethical challenges can be seen as a design constraint that might inhibit, but can also enable creativity (rosso, 2014). hence, a design constraint can also be re-framed as a design opportunity. the aim to support older workers holistically introduces the challenge of balancing human well-being and privacy in the smart workplace and issues considering data ownership. the initial conceptualisation of the smartwork crossed boundaries between private life and workplace; sensitive health-related and productivity-related data. however, as the smartwork project places a premium on human values, the decision was made to prioritise privacy and autonomy at the expense of fully exploiting the potential of some services. another challenge concerns balancing autonomy and positive health interventions. within the smartwork project, respecting the autonomy of the older worker and their values is of utmost importance. given their context, preferences and values, workers can prioritise their interventions so that they can accomplish their self-determined goals. it can be helpful, however, to support individuals in identifying which interventions might be more urgent than others concerning their health and work to make an informed decision. hereby, the person’s ‘psychological autonomy’ is supported, as it increases the capacity of the older worker to exercise their right to self-determination (katz, 2002). to facilitate the prioritisation of interventions, smartwork has worked closely with a local occupational oriented health centre that advises companies and people on how to become and remain healthy, vital, and employable. professionals from various disciplines (e.g., psychologist, physiotherapist, lifestyle coach) shared their expertise from consultations with workers from various backgrounds on how to approach prioritisation of interventions. this informed the smartwork project also concerning the limits of digital interventions and where a blended approach between professionals and smartwork might be beneficial. one important challenge in a smart work environment that makes use of pervasive sensing is to balance the responsibilities put on the employer and the worker. the services within smartwork such as ubiwork that supports work flexibility, workcoach for on-demand training, and digiteam for flexible team management are a response to this challenge by acknowledging the organisation’s responsibility of creating a supportive work environment, whereas the services healthyme and myworkability address the lifestyle and behaviour of the individual. australasian journal of information systems grünloh et al. 2022, vol 26, research article opportunities, ethical challenges, and value implications 16 the smartwork project aims to address the challenges brought by a growing ageing population by supporting active and healthy ageing. hence, smartwork can be seen as a technological response to a societal problem (i.e., the demographic change and the growing number of chronic conditions prevalent in the population). however, socio-technological theories pose that technology affects the behaviour of people and people also affect the working of the technology (klein, 2014). when we consider the first intention under a project such as smartwork, we see indeed the aim for a greater societal good – to prevent older employees to retire earlier, namely by promoting healthier behaviours and lifestyles and to provide a better exchange of knowledge and skills amongst different generations. however, this same tool can collect data for these purposes, which when misused may justify the dismissal of specific older employees due to lack of productivity. in other words: the challenge concerning power relationships at work might be intensified through tools that allow monitoring and shift the balance of power even more towards the employer. the old discussion of atomic energy – produced for a higher good and forever connected to the atomic bomb and its horrendous effects – seems to always haunt the decision of developing these disruptive technologies. however, progress will always imply risks and increased responsibility. one must not avoid progress but search for better tools to secure a correct use. in that sense, smartwork, as similar systems to promote workability, shall not live off its good intentions but instead ensure its good use – and keep this objective in all of the development and testing phases still to come. 7 conclusion the ageing population and the effects on healthcare, workforce and society is a challenge that has widespread implications for the social structure. the development and implementation of technology can be one way to support active and healthy ageing and at the same time facilitate older adults to stay working as desired. every new technology, however, potentially introduces new challenges, as technology affects the behaviour of people and people also affect the working of the technology. before a smart working environment with pervasive sensing and health coaching is implemented and used for a longer time, it remains a question of how it will affect the behaviour of people, organisations, and even society. a technology that accounts for the values and needs of older employees while supporting an adaptive workplace can also be misused to lay off exactly those who need flexibility or support. regulations to prevent discrimination against older employees might be needed for those challenges that cannot be answered by a smart design. within smartwork, the aim is to design better tools and prevent misuse as much as possible, which in turn may bring about innovations that might not have been envisioned without facing those constraints and challenges. however, there is a limit as to how far a technological response to a societal problem can go. despite the potential of digital health solutions to support active and healthy ageing, society might have to change in terms of how people with chronic conditions or of older age are viewed. too often, the ageing population is described in terms of a “burden”, especially on healthcare systems due to increasing costs. the underlying view of smartwork, however, is a positive one, considering the ageing workers as experienced and valuable. thus, investing in employees in the later phase of the working life in terms of flexible work settings, on-demand training and health support should not be seen as an additional cost factor in an economic sense, but as a means to keep valuable employees as long as possible. australasian journal of information systems grünloh et al. 2022, vol 26, research article opportunities, ethical challenges, and value implications 17 acknowledgement this work is supported by the smartwork project, funded by the european commission within h2020-dth-2018 (grant agreement: 826343). since the conclusion of the project, three of the authors have changed affiliation. miriam cabrita is currently affiliated with innovation sprint sprl, brussels, belgium, carina dantas with shine 2europe, coimbra, portugal, and sofia ortet with escola superior de enfermagem de coimbra / nursing school of coimbra, coimbra, portugal. references alder, g. s., noel, t. w., & ambrose, m. l. (2006). clarifying the effects of internet monitoring on job attitudes: the mediating role of employee trust. information & management, 43(7), 894-903. doi:https://doi.org/10.1016/j.im.2006.08.008 alter, s. (2013). work system theory: overview of core concepts, extensions, and challenges for the future. journal of the association for information systems, 14(2). doi:10.17705/1jais.00323 berdichevsky, d., & neuenschwander, e. (1999). toward an ethics of persuasive technology. commun. acm, 42(5), 51–58. doi:10.1145/301353.301410 bødker, s. (2015). third-wave hci, 10 years later---participation and sharing. interactions, 22(5), 24–31. doi:https://doi.org/10.1145/2804405 bousefsaf, f., maaoui, c., & pruski, a. (2014). remote detection of mental workload changes using cardiac parameters assessed with a low-cost webcam. computers in biology and medicine, 53, 154-163. doi:https://doi.org/10.1016/j.compbiomed.2014.07.014 braun, v., & clarke, v. (2013). successful qualitative research: a practical guide for beginners: sage. brey, p. (2015). design for the value of human well-beingwell-being. in j. van den hoven, p. e. vermaas, & i. van de poel (eds.), handbook of ethics, values, and technological design: sources, theory, values and application domains (pp. 365-382). dordrecht: springer netherlands. cabrita, m., op den akker, h., tabak, m., hermens, h. j., & vollenbroek-hutten, m. m. r. (2018). persuasive technology to support active and healthy ageing: an exploration of past, present, and future. journal of biomedical informatics, 84, 17-30. doi:https://doi.org/10.1016/j.jbi.2018.06.010 chalykoff, j., & kochan, t. a. (1989). computer-aided monitoring: its influence on employee job satisfaction and turnover. personnel psychology, 42(4), 807-834. doi:https://doi.org/10.1111/j.1744-6570.1989.tb00676.x chen, l., baird, a., & straub, d. w. (2019). an analysis of the evolving intellectual structure of health information systems research in the information systems discipline. journal of the association for information systems, 20(8). doi:10.17705/1jais.00 561 clarke, v., & braun, v. (2013). teaching thematic analysis: overcoming challenges and developing strategies for effective learning. the psychologist, 26(2), 120-123. cockton, g. (2020). worth-focused design, book 2: approaches, context, and case studies. synthesis lectures on human-centered informatics, 13(2), i-203. cromer, r., denneson, l. m., pisciotta, m., williams, h., woods, s., & dobscha, s. k. (2017). trust in mental health clinicians among patients who access clinical notes online. psychiatric services, 68(5), 520-523. doi:https://doi.org/10.1176/appi.ps.201600168 dantas, c., van staalduinen, w., machado, n., jegundo, a., & rodrigues, f. (2019). older workers, technology and the balance of power: an ethical review. paper presented at the proceedings of the 12th acm international conference on pervasive technologies https://doi.org/10.1016/j.im.2006.08.008 https://doi.org/10.1145/2804405 https://doi.org/10.1016/j.compbiomed.2014.07.014 https://doi.org/10.1016/j.jbi.2018.06.010 https://doi.org/10.1111/j.1744-6570.1989.tb00676.x https://doi.org/10.1176/appi.ps.201600168 australasian journal of information systems grünloh et al. 2022, vol 26, research article opportunities, ethical challenges, and value implications 18 related to assistive environments petra '19, rhodes, greece. https://doi.org/10.1145/3316782.3322770 davis, j., & nathan, l. p. (2015). value sensitive design: applications, adaptations, and critiques. handbook of ethics, values, and technological design: sources, theory, values and application domains, 11-40. doi:https://doi.org/10.1007/978-94-007-6970-0_3 dugdill, l. (2000). developing a holistic understanding of workplace health: the case of bank workers. ergonomics, 43(10), 1738-1749. doi:https://doi.org/10.1080/001401300750004140 fahlquist, j. n., doorn, n., & van de poel, i. (2015). design for the value of responsibility. handbook of ethics, values, and technological design: sources, theory, values and application domains, 473-490. doi:https://doi.org/10.1007/978-94-007-6970-0_18 friedman, b., & hendry, d. g. (2019). value sensitive design: shaping technology with moral imagination. cambridge, ma: mit press. friedman, b., & kahn jr, p. h. (2007). human values, ethics, and design. in a. sears & j. a. jacko (eds.), the human-computer interaction handbook (2nd ed., pp. 1241-1266): crc press. friedman, b., kahn jr, p. h., & borning, a. (2006). value sensitive design and information systems. in p. zhang & d. galletta (eds.), human-computer interaction management information systems: foundations (pp. 348-372). armonk, ny, usa: me sharpe. friedman, b., kahn, p. h., borning, a., & huldtgren, a. (2013). value sensitive design and information systems. in n. doorn, d. schuurbiers, i. van de poel, & m. e. gorman (eds.), early engagement and new technologies: opening up the laboratory (pp. 55-95). dordrecht: springer netherlands. friedman, b., khan, p. h., & howe, d. c. (2000). trust online. commun. acm, 43(12), 34–40. doi:10.1145/355112.355120 funnell, m. m., arnold, m. s., donnelly, m., & taylor-moon, d. (1991). empowerment: an idea whose time has come in diabetes education. the diabetes educator, 17(1), 37-41. doi:https://doi.org/10.1177%2f014572179101700108 galinsky, a. d., magee, j. c., gruenfeld, d. h., whitson, j. a., & liljenquist, k. a. (2008). power reduces the press of the situation: implications for creativity, conformity, and dissonance. journal of personality and social psychology, 95(6), 1450. doi:https://doi.org/10.1037/a0012633 general data protection regulation. (2016). regulation (eu) 2016/679 of the european parliament and of the council of 27 april 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing directive 95/46/ec. retrieved from http://data.europa.eu/eli/reg/2016/679/oj goetzel, r. z., & ozminkowski, r. j. (2008). the health and cost benefits of work site healthpromotion programs. annual review of public health, 29(1), 303-323. doi:10.1146/annurev.publhealth.29.020907.090930 gorm, n., & shklovski, i. (2016). sharing steps in the workplace: changing privacy concerns over time. paper presented at the proceedings of the 2016 chi conference on human factors in computing systems, san jose, california, usa. https://doi.org/10.1145/2858036.2858352 harbers, m., & achterberg, p. (2012). europeans of retirement age: chronic diseases and economic activity. bilthoven: rivm. retrieved from https://ec.europa.eu/health/sites/health/files/major_chronic_diseases/docs/rivm_report _retirement_en.pdf https://doi.org/10.1145/3316782.3322770 https://doi.org/10.1007/978-94-007-6970-0_3 https://doi.org/10.1080/001401300750004140 https://doi.org/10.1007/978-94-007-6970-0_18 https://doi.org/10.1177%2f014572179101700108 https://doi.org/10.1037/a0012633 http://data.europa.eu/eli/reg/2016/679/oj https://doi.org/10.1145/2858036.2858352 https://ec.europa.eu/health/sites/health/files/major_chronic_diseases/docs/rivm_report_retirement_en.pdf https://ec.europa.eu/health/sites/health/files/major_chronic_diseases/docs/rivm_report_retirement_en.pdf australasian journal of information systems grünloh et al. 2022, vol 26, research article opportunities, ethical challenges, and value implications 19 hardy, c., & clegg, s. (2006). some dare call it power. in s. clegg, c. hardy, t. lawrence, & w. nord (eds.), the sage handbook of organization studies (2nd ed.). london: sage publications. hernandez, j., paredes, p., roseway, a., & czerwinski, m. (2014). under pressure: sensing stress of computer users. paper presented at the proceedings of the sigchi conference on human factors in computing systems, toronto, ontario, canada. https://doi.org/10.1145/2556288.2557165 holtermann, a., hansen, j., burr, h., søgaard, k., & sjøgaard, g. (2012). the health paradox of occupational and leisure-time physical activity. british journal of sports medicine, 46(4), 291-295. doi:http://dx.doi.org/10.1136/bjsm.2010.079582 huber, m., knottnerus, j. a., green, l., van der horst, h., jadad, a. r., kromhout, d., . . . smid, h. (2011). how should we define health? bmj, 343, d4163. doi:https://doi.org/10.1136/bmj.d4163 huber, m., van vliet, m., giezenberg, m., winkens, b., heerkens, y., dagnelie, p., & knottnerus, j. (2016). towards a ‘patient-centred’operationalisation of the new dynamic concept of health: a mixed methods study. bmj open, 6(1), e010091. doi:http://dx.doi.org/10.1136/bmjopen-2015-010091 hulstijn, j., & burgemeestre, b. (2015). design for the values of accountability and transparency. in j. van den hoven, p. e. vermaas, & i. van de poel (eds.), handbook of ethics, values, and technological design: sources, theory, values and application domains (pp. 303-333). dordrecht: springer netherlands. iavicoli, s., valenti, a., gagliardi, d., & rantanen, j. (2018). ethics and occupational health in the contemporary world of work. international journal of environmental research and public health, 15(8), 1713. retrieved from https://www.mdpi.com/1660-4601/15/8/1713 jafarinaimi, n., nathan, l., & hargraves, i. (2015). values as hypotheses: design, inquiry, and the service of values. design issues, 31(4), 91-104. retrieved from http://www.jstor.org/stable/43830434 jansen-kosterink, s., bulthuis, r., ter stal, s., van velsen, l., pnevmatikakis, a., kyriazakos, s., . . . op den akker, h. (2020). the results of an iterative evaluation process of an mhealth application for rewarding healthy behaviour among older adults. 62-78. doi:https://doi.org/10.1007/978-3-030-52677-1_4 karasek, r., & theorell, t. (1990). healthy work: stress, productivity, and the reconstruction of working life. new york: basic books. katz, j. (2002). the silent world of doctor and patient: johns hopkins university press. kieslinger, b. (2015). academic peer pressure in social media: experiences from the heavy, the targeted and the restricted user. first monday, 20(6). doi:https://doi.org/10.5210/fm.v20i6.5854 klein, l. (2014). what do we actually mean by ‘sociotechnical’? on values, boundaries and the problems of language. applied ergonomics, 45(2, part a), 137-142. doi:https://doi.org/10.1016/j.apergo.2013.03.027 komp, k. (2018). shifts in the realized retirement age: europe in times of pension reform and economic crisis. journal of european social policy, 28(2), 130-142. doi:https://doi.org/10.1177%2f0958928717709174 kramer, a., & kramer, k. z. (2020). the potential impact of the covid-19 pandemic on occupational status, work from home, and occupational mobility. journal of vocational behavior, 119, 103442. doi:https://doi.org/10.1016/j.jvb.2020.103442 https://doi.org/10.1145/2556288.2557165 http://dx.doi.org/10.1136/bjsm.2010.079582 https://doi.org/10.1136/bmj.d4163 http://dx.doi.org/10.1136/bmjopen-2015-010091 https://www.mdpi.com/1660-4601/15/8/1713 http://www.jstor.org/stable/43830434 https://doi.org/10.1007/978-3-030-52677-1_4 https://doi.org/10.5210/fm.v20i6.5854 https://doi.org/10.1016/j.apergo.2013.03.027 https://doi.org/10.1177%2f0958928717709174 https://doi.org/10.1016/j.jvb.2020.103442 australasian journal of information systems grünloh et al. 2022, vol 26, research article opportunities, ethical challenges, and value implications 20 liang, j., bennett, j. m., shaw, b. a., quiñones, a. r., ye, w., xu, x., & ofstedal, m. b. (2008). gender differences in functional status in middle and older age: are there any age variations? the journals of gerontology series b: psychological sciences and social sciences, 63(5), s282-s292. doi:https://doi.org/10.1093/geronb/63.5.s282 lidynia, c., brauner, p., & ziefle, m. (2018). a step in the right direction – understanding privacy concerns and perceived sensitivity of fitness trackers. paper presented at the advances in human factors in wearable technologies and game design (ahfe 2017), cham. https://doi.org/10.1007/978-3-319-60639-2_5 lipinski, t. a., & britz, j. (2000). rethinking the ownership of information in the 21st century: ethical implications. ethics and information technology, 2(1), 49-71. doi:https://doi.org/10.1023/a:1010064313976 marengoni, a., angleman, s., melis, r., mangialasche, f., karp, a., garmen, a., . . . fratiglioni, l. (2011). aging with multimorbidity: a systematic review of the literature. ageing research reviews, 10(4), 430-439. doi:https://doi.org/10.1016/j.arr.2011.03.003 maría-ángeles, s., maqueda, j., francisco, m., maría-jesús, s., & maría-dolores, s. (2021). evaluation of the effectiveness of workplace health promotion programs from 2000 to 2020: literature review. open journal of preventive medicine, 11(4), 113-131. mercer. (2020). health on demand netherlands report. retrieved from https://www.mercer.nl/content/dam/mercer/attachments/europe/netherlands/healthon-demand-netherlands-report-final.pdf miller, b. (2020). is technology value-neutral? science, technology, & human values, 46(1), 5380. doi:10.1177/0162243919900965 mingers, j., & walsham, g. (2010). toward ethical information systems: the contribution of discourse ethics. mis quarterly, 34(4), 833-854. doi:10.2307/25750707 mozos, o. m., sandulescu, v., andrews, s., ellis, d., bellotto, n., dobrescu, r., & ferrandez, j. m. (2017). stress detection using wearable physiological and sociometric sensors. international journal of neural systems, 27(2), 1650041-1650041. doi:https://doi.org/10.1142/s0129065716500416 nickel, p. j. (2015). design for the value of trust. in j. van den hoven, p. e. vermaas, & i. van de poel (eds.), handbook of ethics, values, and technological design: sources, theory, values and application domains (pp. 551-567). dordrecht: springer netherlands. oecd / european union. (2016). health at a glance: europe 2016: state of health in the eu cycle. doi:https://doi.org/10.1787/23056088 ortet, s., dantas, c., machado, n., tageo, v., quintas, j., & haansen, s. (2019). pervasive technologies applied to the work environment: implications for end-users: the foreground for smartwork concerns and requirements. paper presented at the proceedings of the 12th acm international conference on pervasive technologies related to assistive environments, rhodes, greece. https://doi.org/10.1145/3316782.3322769 papastavrou, e., charalambous, a., & tsangari, h. (2012). how do informal caregivers of patients with cancer cope: a descriptive study of the coping strategies employed. european journal of oncology nursing, 16(3), 258-263. doi:https://doi.org/10.1016/j.ejon.2011.06.001 phelps, a., marmot, m., oskala, a., clemens, s., banks, j., rogers, n., . . . oldfield, z. (2020). english longitudinal study of ageing: waves 0-9, 1998-2019. retrieved from: http://doi.org/10.5255/ukda-sn-5050-18 https://doi.org/10.1093/geronb/63.5.s282 https://doi.org/10.1007/978-3-319-60639-2_5 https://doi.org/10.1023/a:1010064313976 https://doi.org/10.1016/j.arr.2011.03.003 https://www.mercer.nl/content/dam/mercer/attachments/europe/netherlands/health-on-demand-netherlands-report-final.pdf https://www.mercer.nl/content/dam/mercer/attachments/europe/netherlands/health-on-demand-netherlands-report-final.pdf https://doi.org/10.1142/s0129065716500416 https://doi.org/10.1787/23056088 https://doi.org/10.1145/3316782.3322769 https://doi.org/10.1016/j.ejon.2011.06.001 http://doi.org/10.5255/ukda-sn-5050-18 australasian journal of information systems grünloh et al. 2022, vol 26, research article opportunities, ethical challenges, and value implications 21 rosso, b. d. (2014). creativity and constraints: exploring the role of constraints in the creative processes of research and development teams. organization studies, 35(4), 551585. doi:https://doi.org/10.1177%2f0170840613517600 schwartz, s. h. (2012). an overview of the schwartz theory of basic values. online readings in psychology and culture, 2(1). doi:https://doi.org/10.9707/2307-0919.1116 sharon, t. (2017). self-tracking for health and the quantified self: re-articulating autonomy, solidarity, and authenticity in an age of personalized healthcare. philosophy & technology, 30(1), 93-121. doi:https://doi.org/10.1007/s13347-016-0215-5 smartwork. (2019). first version of codesign methodology, user requirements and use cases. retrieved from http://www.smartworkproject.eu/wpcontent/uploads/2019/07/smartwork_d2.2-co-design-method_reqs_use-cases.pdf smith, m. j., carayon, p., sanders, k. j., lim, s. y., & legrande, d. (1992). employee stress and health complaints in jobs with and without electronic performance monitoring. applied ergonomics, 23(1), 17-27. doi:https://doi.org/10.1016/0003-6870(92)90006-h solove, d. j. (2007). i've got nothing to hide and other misunderstandings of privacy. san diego law review. retrieved from https://ssrn.com/abstract=998565 song, z., & baicker, k. (2019). effect of a workplace wellness program on employee health and economic outcomes: a randomized clinical trial. jama, 321(15), 1491-1501. doi:https://doi.org/10.1001/jama.2019.3307 stratton, e., lampit, a., choi, i., calvo, r. a., harvey, s. b., & glozier, n. (2017). effectiveness of ehealth interventions for reducing mental health conditions in employees: a systematic review and meta-analysis. plos one, 12(12), e0189904. doi:https://doi.org/10.1371/journal.pone.0189904 topa, g., depolo, m., & alcover, c.-m. (2018). early retirement: a meta-analysis of its antecedent and subsequent correlates. frontiers in psychology, 8. doi:10.3389/fpsyg.2017.02157 vos, t., lim, s. s., abbafati, c., abbas, k. m., abbasi, m., abbasifard, m., murray, c. j. l. (2020). global burden of 369 diseases and injuries in 204 countries and territories, 1990– 2019: a systematic analysis for the global burden of disease study 2019. the lancet, 396(10258), 1204-1222. doi:https://doi.org/10.1016/s0140-6736(20)30925-9 warnier, m., dechesne, f., & brazier, f. (2015). design for the value of privacyprivacy. in j. van den hoven, p. e. vermaas, & i. van de poel (eds.), handbook of ethics, values, and technological design: sources, theory, values and application domains (pp. 431-445). dordrecht: springer netherlands. weber, m. (1978). economy and society: an outline of interpretive sociology (vol. 1): univ of california press. whittle, j., ferrario, m. a., simm, w., & hussain, w. (2021). a case for human values in software engineering. ieee software, 38(1), 106-113. doi:https://doi.org/10.1109/ ms.2019.2956701 world health organization. (1948). preamble to the constitution of the world health organization as adopted by the international health conference, new york, 19-22 june, 1946; signed on 22 july 1946 by the representatives of 61 states (official records of the world health organization, no. 2, p. 100) and entered into force on 7 april 1948. the definition has not been amended since 1948. http://www.who.int/governance/eb/ who_constitution_en.pdf. retrieved from https://www.who.int/about/who-we-are/ frequently-asked-questions https://doi.org/10.1177%2f0170840613517600 https://doi.org/10.9707/2307-0919.1116 https://doi.org/10.1007/s13347-016-0215-5 http://www.smartworkproject.eu/wp-content/uploads/2019/07/smartwork_d2.2-co-design-method_reqs_use-cases.pdf http://www.smartworkproject.eu/wp-content/uploads/2019/07/smartwork_d2.2-co-design-method_reqs_use-cases.pdf https://doi.org/10.1016/0003-6870(92)90006-h https://ssrn.com/abstract=998565 https://doi.org/10.1001/jama.2019.3307 https://doi.org/10.1371/journal.pone.0189904 https://doi.org/10.1016/s0140-6736(20)30925-9 australasian journal of information systems grünloh et al. 2022, vol 26, research article opportunities, ethical challenges, and value implications 22 world health organization. (2010a). healthy workplaces: a model for action: for employers, workers, policy-makers and practitioners. retrieved from https://www.who.int/occupational_health/publications/healthy_workplaces_model_act ion.pdf world health organization. (2010b). tackling chronic disease in europe: strategies, interventions and challenges. observatory studies series: 20. retrieved from https://apps.who.int/iris/handle/10665/326484 world health organization. (2021). ageing and health. retrieved from https://www.who.int/news-room/fact-sheets/detail/ageing-and-health copyright: © 2022 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v26i0.3133 https://www.who.int/occupational_health/publications/healthy_workplaces_model_action.pdf https://www.who.int/occupational_health/publications/healthy_workplaces_model_action.pdf https://apps.who.int/iris/handle/10665/326484 https://www.who.int/news-room/fact-sheets/detail/ageing-and-health http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 smartwork project 3 theoretical background 3.1 supporting the sustainability of workability 3.2 digital behaviour change interventions 3.3 digital health interventions at the workplace 3.4 data protection principles and regulations in europe 3.5 accounting for human values in design 4 method 5 results 5.1 human well-being and privacy in the smart workplace 5.2 data ownership, control, access, and use 5.3 autonomy and positive health interventions to support well-being 5.4 division of responsibility between employer and worker 5.5 power relations at work, accountability, and trust 6 discussion 7 conclusion paper title (paper title style) australasian journal of information systems islam, miah, kamal & burmeister 2019, vol 23, research on health information systems approaches to measure mental health conditions 1 a design construct of developing approaches to measure mental health conditions md rafiqul islam department of computer science & engineering, islamic university of technology (iut), bangladesh shah jahan miah information system discipline college of business, footscray park campus victoria university shah.miah@vu.edu.au abu raihan m. kamal department of computer science & engineering, islamic university of technology (iut), bangladesh oliver burmeister school of computing and mathematics charles sturt university abstract mental health is an important determinant of communities’ well-being, influenced not only by individual attributes, but also by social and organisational environments in which people work and live. despite studies examining mental health status among specific populations, few attempts are evident that focus on solution designs for detecting and measuring impact of mental health conditions. in this study, we develop a construct utilising design science research principles for outlining common vocabulary around the problem, and solution design relevant to a mental health management system. for the case of it professionals, the developed construct is informed through a social-media based dataset containing more than 65,000 cells and 100 attributes potentially identifying influencing factors. machine learning techniques are applied to the dataset to discover new findings for this specific group. it is anticipated that the analysis reported in this study would contribute in developing other electronic health management systems both for communities and healthcare professionals. keywords: healthcare is; design research; mental disorder; diagnosis; occupational stress; ict professionals. 1 introduction professionals in many organizations often handle relatively stressful tasks within daily operations, affecting mental wellbeing. mental health problems have an impact on professional communities directly through increased absenteeism; creating a negative influence on productivity and profits, as well as an increase in costs. around half of individuals with a diagnosis of mental illness have moderate or severe occupational impairment, leading to social exclusion, poor self-esteem, and financial hardship (de australasian journal of information systems islam, miah, kamal & burmeister 2019, vol 23, research on health information systems approaches to measure mental health conditions 2 choudhury et al. 2013). standard, symptom-based treatments, in isolation, have inadequate effects on occupational outcomes that has led many to conclude that work outcomes for people with mental illness can be improved by increased involvement by workplaces. this involvement requires a systematic support that enables appropriate analysis, management, and exploring of solution strategies such as developing rehabilitation plans for individuals. mental health condition is one of the main causes of occupational dissatisfaction and sickness (french et al. 1982; boyce et al. 2008) in the workplace (bell & klein, 2001). in workplaces where information technology (it) professionals work, mental stress triggers when external demands and expectations exceed professionals’ physical capacity, skills, and knowledge for comfortably managing problem solving and relevant tasks (lands bergiset et al., 1998; joyce et al., 2009; brohan et al. 2010). for the it professionals, chronic exposure to stressful situations such as work overload, poor supervisory support and low input into decision-making links to a range of debilitating health outcomes (both physical and mental health issues), including depression, anxiety, emotional exhaustion, immune deficiency disorders and cardiovascular disease (noblet & la montagne, 2006). although these understandings benefit social awareness and suggest treatment options, holistic solution strategies in relation to developing mental health management systems are required, and guidance for systematic solution development is still lacking. research in healthcare is addresses issues regarding organizational, operational, managerial, technical and social aspects of is in different healthcare settings. for developing systematic solutions and design knowledge much healthcare is research takes a positivist stance (lin et al., 2017). design science research approaches promote views, methods, guidelines, approaches, and experiences for developing innovative is solution artefacts such as constructs (in terms of constituting the language to specify problems and solutions vocabulary), models (in terms of abstractions for representing solutions), methods (in terms of algorithms of solutions), and instantiations (in terms of system prototypes) (hevner et al. 2004). in this study, to address the issue of discovering factors affecting mental health conditions we outline here a construct for detecting a particular mental health condition (e.g. depression based on age group, sex, occupation, and country of the worker) among it professionals. the proposed construct also informs a solution language by utilizing various machine learning approaches that provide an innovative concept in designing more formal systematic mental health management solutions. stressful working conditions impact indirectly on professionals’ well-being by limiting an individual’s ability to make positive changes to their lifestyle behaviours (e.g. smoking, sedentary behaviour) or by directly contributing to their negative health behaviours (michie and williams, 2003). at a broader scale, occupational mental issues of workforce could result in poor efficiency and throughput in the economies of countries, even impacting wider issues such as poverty and other social calamities (cox et al. 2004; danziger, 2009; laitinen-krispijn, 2000). it professionals in particular can experience considerable occupational stress due to real time demands, equipment malfunctions, intrusion and other security threats system, system cutover problems and unanticipated disasters (al-saggaf et al. 2017). as it professionals work in all sectors and in all countries, they provide a good population around which to develop a general construct. australasian journal of information systems islam, miah, kamal & burmeister 2019, vol 23, research on health information systems approaches to measure mental health conditions 3 in considering mental health of it professionals, the following research issues are important for developing both generic problem significance and provisions for solution strategies regarding a mental health management system: 1) what exogenous and endogenous workplace factors affect mental health among it professionals? 2) what are the attributes of it professionals who are more vulnerable to mental health issues? 3) what are the common mental health issues diagnosed in the it professionals and how prevalent is mental disorder? 4) what is the relationship between mental health issues and attitudes toward mental health based on cultural norms (geographic location)? 5) what is the most influential attribute contributing toward diagnosis of mental health conditions? 6) why are people with severe mental illness excluded from work? and does it matter if people with severe mental illness work? understanding such concerns are prerequisite to selecting technologies for decision support solution frameworks that may better handle mental health issues. social media sites are abundant, offering common platforms where people express and share their personal issues using comments, photos, icons and other content. social media data are imbued with useful information about personal well-being and life-style and many social media or organization media sites provide publicly available large datasets that contain relevant attributes of professionals. this provides a source of big data to address our questions. our aim in this study is to provide understanding around mental health problems and possible data-driven decision support solution approaches developed through the use of machine learning (ml) techniques. this has been shown to be effective in other areas of mental health research (teipel et al., 2016). in dsr, research in relation to designing constructs has a significant impact on the way in which tasks and problems are comprehended for fully functional solution design (boland, 2002; hevner et al. 2004). problem representation has a profound impact on design work, and theoretical constructs shape the development of models or representations of the problem domain. the remainder of the paper is organized as follows: section 2 presents background to the study while section 3 describes methodological details. section 4 reports on the proposed design construct that informs mental health issue detection and guides solution design, followed by a discussion and evaluation in section 5. finally, section 6 provides a conclusion and future directions of the study. 2 related works today’s social media technologies allow people to express their views freely online which provides rich and valuable content that can enable opportunities for new insights or support information (mislove et al., 2007). as a result, a growing number of researchers are using social media data for a range of online health, medical, and psychosocial studies (de choudhury et al. 2013). however, it is important to carefully consider the online content of target population australasian journal of information systems islam, miah, kamal & burmeister 2019, vol 23, research on health information systems approaches to measure mental health conditions 4 for the assessment of generalisability (choi et al. 2017). joyce et al. (2016) identified that depression is a most common factor of mental disorders in most developed countries while carrying out a systematic meta-review examining the effectiveness of the interventions of workplace mental health. the study by joyce et al. (2016) concluded that there are empirically supported interventions that workplaces can utilize systematic solutions to aid in the prevention of common mental illness. cousins et al. (2004) described the development of health and safety executive (hse) new stress management standards—that offer organizations’ continuous improvement through a three-phase stress preventative process—and the development of a supporting ‘indicator tool’ (a two-phase questionnaire to assess employee perceptions of working conditions). they mentioned the hse's aim to provide an associated methodology that will enable organizations to effectively tackle work-related stress, and subsequently reduce both its incidence and prevalence. (lamontagne et al. 2007) discussed how job related stress has a wide range of adverse effects on mental, physical, and organizational health. these studies also emphasised how developing systems approaches for reducing the adverse impact of job stress, prevalent practice is dominated by the worker. these studies imply a clear need for designing computerbased solutions for measuring and detecting mental health conditions. sharma et al. (2009) developed the global mental health assessment tool – primary care (gmhat/pc) to assist general practitioners and other health professionals to make a quick, convenient, and comprehensive standardized mental health assessment. they mentioned that it may help for detecting and managing mental disorders in primary care and general health settings more effectively. gabriel and liimatainen (2000) represented the importance of mental health of the workplace in enhancing the economic and social integration of people with mental health difficulties. they analysed the scope and impact of mental health problems in the labour markets in five countries including finland, germany, poland, united kingdom, and united states and their work to advance and to advocate for employment for people with mental health difficulties. cox and cox (1993) considered the traditional approach to occupational health which points up the importance of psychosocial and organizational issues. they focused the development of a new three factors framework of concern, that is, organizational health based on health, work and organization. they addressed the organizational health problems and managed effectively within the framework that proved successful for the more tangible workplace hazards. although some of these studies addressed mental health issues at the workplace in general, none have considered mental stress in it workplaces, nor have any had a focus to develop a solution design understanding, for instance, by applying data-driven methods for the purpose of mental stress detection. this paper collects social media/open sourced data to examine various attributes of tech workers and to diagnose their mental health conditions by applying ml techniques, ultimately to suggest a solution design approach for better management of mental health issues. 3 design methodology the research adopted a design science research (dsr) methodology for designing the proposed artefact. dsr has received increasing attention by information systems (is) design researchers in recent years and can produce distinct types of artefacts: constructs, models, methods, instantiations and theories (gregor & hevner, 2013). baskerville et al. (2015) australasian journal of information systems islam, miah, kamal & burmeister 2019, vol 23, research on health information systems approaches to measure mental health conditions 5 described design science as representing (a) a design-science research project, (b) an artefact “build and evaluate” project such that a research project may entail, (c) the production of new knowledge from design-and-development, and (d) the creation of reports or articles describing this design-science research project. our study follows this model to produce a new construct in the form of an artefact design and its evaluation. baskerville et al. (2015) also suggested that dsr is an approach that enables structural guidance for researchers to create modernized or innovative system artefacts: in our study, a design construct forming a language of problems and solutions that can be applied for a real solution development. 3.1 dsr framework the study adopted a dsr framework for the design construct comprising problem definitions and in principle solution strategies, informed by ml techniques intended to address the decision making around mental health conditions for their appropriate management. hevner et al.’s (2004) seven guidelines provide useful criteria for defining a dsr study problem space, specifying a design based solution artefact, implementing the design solution, evaluating the design artefact and communicating study details and results. in addition, our description here is practically guided by gregor and hevner (2013), explicating the level of contribution to artefact abstraction and knowledge for the production of a publication (e.g. artefact design through conceptualization of ensemble artefact view defined in miah & gammack, 2014). figure 1: our methodological steps for identifying mental health issues we utilise four design phases namely data collection and processing, data classification, problem statement and diagnosis of mental health issues (illustrated in figure 1). this complements the approach of miah, vu, gammack and mcgrath (2017) in which three phases (problem identifications, artefact development and communication about the artefact) were recreated from hevner’s et al. (2004) dsr framework. we mapped these design guidelines as follows: guideline 1 defining the type of artefact as construct, guideline 2 is to signify the mental health problems, guideline 3 conducting an experiment to demonstrate utility of the australasian journal of information systems islam, miah, kamal & burmeister 2019, vol 23, research on health information systems approaches to measure mental health conditions 6 technology’s provisions, guideline 4 identifying requirements of decision support for mental health issues management, guideline 5 comparing and contrasting on the selection of technologies, guideline 6 selecting evaluation techniques and guideline 7 communicating the studies among academic and industry audiences. 3.2 data collection we used a publicly available dataset (https://www.kaggle.com/osmi/mental-health-in-tech-2016) containing attributes of workers working in it/tech workplaces. the dataset contains the number of attributes of it/tech workplace workers. in this dataset, the total number of records in each part is approximately 10,000. we used part 1 to analyse the situation in it workplace and part 2 to detect the percentage of it workplace workers who are suffering from mental stress. this dataset contains 100 columns where each column represents a unique piece of information about a tech worker. although each of the individual columns provides valuable information for diagnosis of tech workplace workers mental health conditions, in this paper, we selected 23 columns because the other 77 columns are not strictly necessary for detecting mental health conditions of tech workplace workers. we divided our dataset into two parts to detect and examine the mental stress of it workplace workers. overall, these columns contain both mentally “sick” and mentally “sound” workers information. part 1 contains 13 columns (including demographic indicators) and part 2 contains 10 columns based on the following questions which are shown below. overview of part 1 • have you heard of or observed negative consequences for co-workers who have been open about mental health issues in your workplace? • do you believe your productivity is ever affected by a mental health issue? • if yes, what percentage of your work time (time performing primary or secondary job functions) is affected by a mental health issue? • do you think that discussing a mental health disorder with previous employers would have negative consequences? • would you be willing to bring up a physical health issue with a potential employer in an interview? • would you bring up a mental health issue with a potential employer in an interview? • do you think that team members/co-workers would view you more negatively if they knew you suffered from a mental health issue? • how do you describe your work position? • how willing would you be to share with friends and family that you have a mental illness? • what is your age? • what is your gender? • what country do you live in? australasian journal of information systems islam, miah, kamal & burmeister 2019, vol 23, research on health information systems approaches to measure mental health conditions 7 overview of part 2 • have you observed or experienced an unsupportive or badly handled response to a mental health issue in your current or previous workplace? • have your observations of how another individual who discussed a mental health disorder made you less likely to reveal a mental health issue yourself in your current workplace? • do you have a family history of mental illness? • have you had a mental health disorder in the past? • do you currently have a mental health disorder? • if yes, what condition(s) have you been diagnosed with? • if maybe, what condition(s) do you believe you have? • have you been diagnosed with a mental health condition by a medical professional? • have you ever sought treatment for a mental health issue from a mental health professional? • if you have a mental health issue, do you feel that it interferes with your work when being treated effectively? 3.3 ground truth dataset we now discuss how we constructed our dataset with ground truth information (on whether the it/tech workers conditions are mentally sick). we use our dataset and divided into two sets (1) for the positive (yes) class (mentally sick) and (2) for the negative (no) class (mentally sound). we justified the information in each set manually using three experts. we analysed approximately 10,000 records of it workplace workers where 58% obtained yes and 42% obtained no. table 1 illustrates the overview of mental condition of it/tech workers. total number of it/tech workers 1,433 mentally sick workers 933 mentally sound 500 table 1: overview of mental condition of it/tech workers 3.4 measuring mental illness we acknowledge that mental health can be a variable condition, and the spectrum is not as binary as the classification of “sick and “sound” suggests, and moreover that “mental illness” is itself controversial in psychology since at least szasz (1960). nonetheless as with any fuzzily defined categorisation, particular classifications can be usefully distinguished and evidentially supported. as experts were used to determine the class level distinction, the detection has an authorised basis, so we preface our discussion of measurement by noting this design choice. figure 2 represents a measuring procedure for detecting mental health conditions using a ml technique. in this procedure, we collect data for finalizing the features of the dataset and then applied the ml technique for detecting the mental health indicators associated with anxiety, depression, bipolar disorder, personality and mood as the initial focus for our investigation. australasian journal of information systems islam, miah, kamal & burmeister 2019, vol 23, research on health information systems approaches to measure mental health conditions 8 figure 2: detecting procedures that were used in this study we first analyse a set of demographic attributes including age group, sex, occupation, and country that were used to represent the range of mentally sick people in it/tech workplace (table 2). attributes such as family problems, past condition, present condition, and workplace environment condition were used for detecting the mental disorder in the workplace (table 3). variable total anxiety disorder mood disorder other disorder age group 20-30 222 134 53 35 31-40 239 150 69 20 41-50 76 38 29 9 >50 27 13 10 4 sex male 363 207 102 54 female 177 114 52 11 occupation back-end developer 139 72 44 23 supervisor/team lead 75 49 18 8 executive leadership 28 13 11 4 australasian journal of information systems islam, miah, kamal & burmeister 2019, vol 23, research on health information systems approaches to measure mental health conditions 9 variable total anxiety disorder mood disorder other disorder devops/sysadmin 55 34 16 5 dev evangelist/advocate 27 16 11 0 support 26 17 6 3 one-person shop 47 20 15 12 front-end developer 76 55 16 5 designer 24 16 5 3 other 66 40 19 7 hr 3 3 0 0 country united states of america 384 228 115 41 united kingdom 62 43 17 2 australia 16 9 6 1 canada 23 15 3 5 germany 10 4 5 1 netherland 22 8 5 9 france 3 2 0 1 denmark 2 1 0 1 other country 45 26 11 8 table 2: categorization of it/tech workers in various mental disorders mental health disorder in the past? sex anxiety mood other total yes m 137 22 13 172 f 82 15 3 100 no m 70 14 6 90 f 33 2 1 36 family history of mental illness? yes m 86 13 8 107 f 57 9 1 67 no m 121 23 11 155 f 58 8 3 69 currently have a mental health disorder? yes m 124 22 15 161 f 72 14 2 88 no m 83 14 4 101 f 43 3 2 48 australasian journal of information systems islam, miah, kamal & burmeister 2019, vol 23, research on health information systems approaches to measure mental health conditions 10 mental health issue in current or previous workplace? sex anxiety mood others total yes m 107 15 9 131 f 58 11 2 71 no m 100 21 10 131 f 57 6 2 65 diagnosed with a mental health condition by a medical professional? yes m 93 16 9 118 f 61 12 3 76 no m 114 20 10 144 f 54 5 1 60 if you have a mental health issue, do you feel that it interferes with your work when being treated effectively? na m 103 20 10 133 f 58 6 1 65 rarely m 46 5 3 54 f 21 4 1 26 some times m 47 7 5 59 f 30 6 2 38 often m 11 4 1 16 f 6 1 0 7 table 3: number and proportion of it/tech workplace workers mental disorder into various conditions 4 artefact description the proposed design construct comprises two aspects: identifying factors that affect mental health conditions at the workplace and design and evaluation of a solution approach for a particular group to showcase a solution strategy as represented in figure 3. for identifying affecting factors, we first collected open access data operating a modified api open-sourced programming code (source: https://developers.facebook.com/docs/graph-api/using-graphapi/). the collected data was in the form of textual content. we proposed an analytics methodology as illustrated in figure 4. secondly, we used a machine learning technique for classifying the collected textual data for showcasing some findings in terms of solution design. figure 3: general structure of the proposed construct nunamaker et al. (1991) argued that the process of constructing solution artefacts and offering understanding of problems are important tasks for design researchers. supporting this argument, hevner et al. (2004) described that in design research it is imperative to provide factors affecting mental health conditions at work design construct of mental health conditions machine learning approach as a solution australasian journal of information systems islam, miah, kamal & burmeister 2019, vol 23, research on health information systems approaches to measure mental health conditions 11 “the language in which problems and solutions are defined and communicated” that can inform design of “cutting-edge” is applications. figure 4: method of a ml based analytics the proposed analytics methodology (figure 4) consists of steps such as initial conditions of workers (healthy worker in it workplace), assessed by a set of methods (e.g. personalized risk profiling). next, when the workers engage their workload, their mental illness can be assessed by methods such as examining physical or mental conditions. then the third step is to assess the mental issues, by finding the factors affecting their individual conditions. finally the ml has been applied to validate the factors through measuring the comparison between ground truth response and ml response. the ground truth response created by three domain experts in which a binary conversion from textual data to numeric data was taken to test the ml technique’s acceptability. machine learning in the most general conception is a set of tools and methods with the aim of modelling and understanding data. ml algorithms can learn how to perform important tasks by generalizing from examples (oleksy, 2017). this is often feasible where manual programming is not. as more data becomes available, more ambitious problems can be tackled. these algorithms have the ability to extract information and infer structure from collections of measurements, data streams such as sound/video signal and human language (yin et al. 2017, burmeister et al. 2015). learning in general is mainly performed in two ways: knowledge acquisition and skill refinement. thus, ml contributes to decision support systems (dss) in two major realms: through supporting the acquisition and refinement of problem-solving knowledge and solution strategies, and through enabling dss to adapt and evolve to be more efficient and effective in decision-making support. in this aspect we gained knowledge for investigating how ml techniques can be suitable for good decision support and useful for knowledge acquisition in their ability to adapt. australasian journal of information systems islam, miah, kamal & burmeister 2019, vol 23, research on health information systems approaches to measure mental health conditions 12 5 discussion and evaluation the paper so far has presented a construct design for informing important mental health issues and possible solution provision so a fully-functional decision support application in future can be developed. we promoted an idea of using online available data to assess potential of the factors affecting mental health at it workplace and machine learning based solution design. we discussed the performance of different ml classifiers in detecting mentally ill workers in a shorter time. the experiment was conducted using matlab (matlab is a productive software environment for researchers to analyse and visualize data with latest techniques. we used the latest version of 2016b, for better outcome). we applied four major ml classifiers such as decision trees (it is method for using a tree-like graph or model of decision aspects), knearest neighbour (knn), support vector machine (support vector machine (svm) is a discriminative classifier, is one of the most popular machine learning algorithms), and ensemble as mentioned earlier. each ml classifier technique has sub-classifiers such as decision treessimple tree, medium tree, and complex tree; svmlinear, quadratic, cubic, fine gaussian, medium gaussian, and coarse gaussian; knnfine, medium, coarse, cosine, cubic and weighted, ensemble boosted tree, bagged tree, subspace discriminant, subspace knn, rusboosted tree. performances of these classifiers have been computed by using the evaluation matrices parameters such as precision, recall, and f-measure (f-measure has a common use in data mining projects. it combines precision and recall for the harmonic mean of precision and recall). all the evaluation metrics that we used are defined as follows. precision is the ratio of true positives to the cases that are predicted as positive. it is the percentage of selected cases that are correct. 𝑃𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 (𝑃) = 𝑇𝑟𝑢𝑒 𝑃𝑜𝑠𝑖𝑡𝑖𝑣𝑒 𝑇𝑟𝑢𝑒 𝑃𝑜𝑠𝑖𝑡𝑖𝑣𝑒 + 𝐹𝑎𝑙𝑠𝑒 𝑃𝑜𝑠𝑖𝑡𝑖𝑣𝑒 recall is the ratio of true positives to the cases that are actually positive. it is the percentage of corrected cases that are selected. 𝑅𝑒𝑐𝑎𝑙𝑙 (𝑅) = 𝑇𝑟𝑢𝑒 𝑃𝑜𝑠𝑖𝑡𝑖𝑣𝑒 𝑇𝑟𝑢𝑒 𝑃𝑜𝑠𝑖𝑡𝑖𝑣𝑒 + 𝐹𝑎𝑙𝑠𝑒 𝑁𝑒𝑔𝑎𝑡𝑖𝑣𝑒 f-measure is the mean of precision and recall. it takes both false positives and false negatives into an account. f-measure is calculated as: 𝐹𝑀𝑒𝑎𝑠𝑢𝑟𝑒 = 2 𝑃𝑅 𝑃 + 𝑅 the experiment was carried out by using 10-fold cross-validation on all testing data sets. for every ml classifier, we found the value of its sub-classifier which persists to precision, recall and f-measure (refer to table 4). the result has been graphically represented in figures 5 and 6. australasian journal of information systems islam, miah, kamal & burmeister 2019, vol 23, research on health information systems approaches to measure mental health conditions 13 algorithm precision recall f-measure accuracy complex tree 0.994623 0.991425 0.993022 99 medium tree 0.994577 0.982851 0.988679 98.5 simple tree 0.957974 0.952840 0.955400 94.1 fine knn 0.997851 0.995712 0.996781 99.5 medium knn 0.988159 0.983922 0.986036 98.1 coarse knn 0.933537 0.978563 0.955520 94 cosine knn 0.997826 0.983922 0.990825 98.7 cubic knn 0.988159 0.983922 0.986036 98.1 weighted knn 0.997851 0.995712 0.996781 99.5 linear svm 0.997860 1 0.998929 99.9 quadratic svm 0.996794 1 0.998394 99.8 cubic svm 0.996794 1 0.998394 99.8 fine gaussian svm 0.997851 1 0.998924 99.5 medium gaussian svm 0.996788 0.998927 0.997856 99.7 coarse gaussian svm 0.996791 0.998928 0.997858 99.7 ensemble boosted tree 0.996784 0.996784 0.996784 99.5 ensemble bagged tree 0.996791 0.998928 0.997858 99.7 ensemble subspace discriminant 0.95010 0.979635 0.964643 95.3 ensemble subspace knn 0.904244 0.981779 0.941418 92 ensemble rusboosted tree 0.995694 0.991425 0.993555 99.1 table 4: precision and recall corresponding to best f-measure of the different classifier figure 5 represents the result of precision and recall values of different classifiers where the xaxis represents the different sub-classifiers of decision tree, knn, svm and ensemble techniques. we mentioned (1-20) as a symbol of decision treessimple tree, medium tree, and complex tree; svmlinear, quadratic, cubic, fine gaussian, medium gaussian, and coarse gaussian; knnfine, medium, coarse, cosine, cubic and weighted, ensemble boosted tree, bagged tree, subspace discriminant, subspace knn, rusboosted tree respectively and y-axis represents the range of value. australasian journal of information systems islam, miah, kamal & burmeister 2019, vol 23, research on health information systems approaches to measure mental health conditions 14 figure 5: precision-recall curve for the 20 sub-classifiers of machine learning techniques similarly, figure 6 showed the result of f-measure where the x-axis represents the name of different sub-classifiers and y-axis represents the percentage of the sub-classifier result. figure 6: performance comparisons among the classifiers using the above ml classification techniques, we examined detection performance of mentally ill it workplace workers, so as to comprehend the significance of different feature types. we already demonstrated the results of various characterizations with various proportions of four features. the outcome showed that, in our test set, the best performing model is svm. in addition, for all of the features precision, recall and f-measure calculation, svm gives the most outstanding outcome relating to the class of mental disorder indicative of it workplace workers. moreover, in figures 7 and 8, we have shown the various range of mental disorders values based on age group and sex, where x-axis represents the various names of the mental disorders and y-axis represents the number of people who are feeling anxious and moody. australasian journal of information systems islam, miah, kamal & burmeister 2019, vol 23, research on health information systems approaches to measure mental health conditions 15 figure 7: the range of age group in it/tech workers with mental disorders figure 8: the number of male and female in it/tech workers with mental disorders similarly, figures 9 and 10 demonstrate the result of the range of mental disorder values based on occupation/positions such as back-end developer, supervisor/team leader, executive, system admin, dev evangelist, one person shop officer, font-end developer, designer, and hr, and country such as usa, uk, australia, canada, germany, netherland, and france, where the x-axis represents the designation of it workplace workers and their country respectively and the y-axis represents the number of people who are feeling anxiety, mood etc. in figure 9, we examined all of the occupational positions in the it workplace for calculating the mental disorders. it was observed that for all of the mental disorders, back-end developers are suffering the most. we trust that our present study has laid the ground for future research on deductions and revelation of new data in view of cause-event correlation. a limit in the data set used is the small number of occupation classifications. the it workplace is much more diverse than these few classifiers suggest, and they vary between countries (al-saggaf et al., 2017). australasian journal of information systems islam, miah, kamal & burmeister 2019, vol 23, research on health information systems approaches to measure mental health conditions 16 figure 9: the number of people in the different sections of the it/tech workplace with mental disorders. figure 10 shows the populations of some countries who are suffering from different mental disorders in the it/tech workplace. the country with the largest population (67%) is the usa with anxiety, mood and other disorders, whereas denmark has the smallest population. apart from the usa, the largest countries are uk, australia, and canada respectively. however, interpretation of the figure can be misleading, because it is population based and based upon the limited sample in the publicly available data set. for instance, anxiety and depression are the two most prevalent mental illnesses experienced in australia (birch, 2015; sayers et al., 2017). similarly, in figure 8 anxiety is the most prevalent in it workers. mood in this data set includes, but is not limited to, depression. a more representative data set would therefore be likely to show different results to what is depicted in figure 10. figure 10: the number of people in the different countries of the it/tech workplace with mental disorders australasian journal of information systems islam, miah, kamal & burmeister 2019, vol 23, research on health information systems approaches to measure mental health conditions 17 6 conclusion the paper represents basic requirements of developing a systematic is solution for establishing common vocabulary of problems and solution technique selection. in our study we used a representative case of it professionals. for promoting an understanding on mental health issues and possible technology selection for decision support solution design, we have exhibited the capability of using it/tech workplace survey data for measuring and detecting mental disorders among tech workers. to give a clear understanding of our work, the research challenges were stated at the start of this paper. the proposed analytics performed on the selected dataset, provide some insight on the research challenges. below is the summary of our findings: • what exogenous and endogenous workplace factors affect the mental health of tech worker? it was revealed that there are four key factors that significantly contribute toward the mental health condition of tech workers. these factors include ‘minimal risk’, ‘workplace concern’, ‘activity concern’, and ‘emotional stresses. • what are the attributes of tech workers who are more vulnerable to mental health issues? although, most of the workers working in technical departments are prone to mental health issues, our study found that the worker with ‘family history of mental stress’, ‘past condition of mental stress’, are more likely to develop mental health issues. in addition, the current workplace environment can act as a catalyst to further magnify the chances of mental health of already vulnerable workers. • what are the common mental health issues diagnosed in the tech workers and how many mental disorder episodes they had experienced? proposed analytics performed on the selected dataset suggest various mental health issues that tech workers could possibly be presented with. common mental health issues include ‘anxiety disorder’ (generalized, social, phobia, etc.), ‘mood disorder’ (‘depression’, ‘bipolar disorder’, etc.) and other kinds of disorders such as (‘personality disorder’, ‘seasonal affective disorder’, ‘burn out’, ‘pdd-nos’, ‘combination of physical impairment’ (strongly nearsighted) with a possibly mental one (mcd / "adhd"), ‘substance use disorder’ , ‘stress response syndromes’, ‘sexual addiction’, ‘obsessive-compulsive disorder’, and ‘post-traumatic stress’). these results showing a relationship between mental illness and physical impairment are recognised more widely, as seen in the equally well initiative recently launched by the australian national mental health commission to better address the physical needs of people experiencing mental illness (roberts et al. 2018). analytics presented in table 2 above suggested that workers in the age bracket of 20-30 years are more likely to get anxiety. this finding is supported by an oecd report (oecd, 2012) on the relationship between work and mental health, which reported that mental health workplace challenges tend to arise at an early age. this could possibly due to lack of their experience of dealing with high-stress tech environment and other pressures at the early stages of their professional career. the situation could be better with the age as workers get more experience and find ways to deal with work related anxiety and depression. australasian journal of information systems islam, miah, kamal & burmeister 2019, vol 23, research on health information systems approaches to measure mental health conditions 18 • what is the relationship between mental health issues and attitudes toward mental health based on the geographic location in the global workplace? the relationship of mental health issues with geographic locations varies country to country. table 3 presents information on mental health condition of workers working in various countries. • what is the most influential attribute contributing toward diagnosis of mental health condition in the tech workplace? although we considered a total of 23 attributes for diagnosis of mental health conditions of tech workers, the most significant attributes are found to be family history of mental illness, personal history of mental illness, present situation of mental health and the working environment. we trust that these attributes provide valuable information on the influential attributes contributing toward the diagnosis of mental health condition in the tech workplace. in this paper, we make an attempt to get an insight of mental health condition of tech workers by applying various ml techniques. our findings suggest that all the classifiers results are almost 99%. in future work, we will extend the literature in the area of growing attention called ‘mental health mining’, in particular in health related knowledge discovery. the suitability of mental illness data for disease surveillance will be demonstrated along with its potential limitations in a case study of mental illness around the world. additionally, the state-of-art and empirically validated machine learning algorithm will be employed in order to gain insight into the most common mental illnesses around the world. we will use further techniques to extract paraphrases from more types of mental disorder features. also, we will be using more datasets to verify the efficiency and effectiveness of ml techniques to subsequently improve the predictive accuracy. this will prove the way for a larger scale investigation into the topic in the upcoming research. as a result, the benefits of high quality input data as well as the innovative visualization tool for patterns exploration will considerably facilitate knowledge discovery. finally, the literature on social media data mining for mental health applications will be extended with a new practical application to the current matters of concern, including depression and anxiety. references al-saggaf, y., burmeister, o. k., & schwartz, m. (2017). qualifications and ethics education: the views of ict professionals. australasian journal of information systems, 21, doi: 10.3127/ajis.v21i0.1365. baskerville, r. l., kaul, m., & storey, v. c. (2015). genres of inquiry in design-science research: justification and evaluation of knowledge production. mis quarterly, 39(3), 541-564. bell, b. s., & klein, k. j. (2001). effects of disability, gender, and job level on ratings of job applicants. rehabilitation psychology, 46(3), 229. bender, a., & farvolden, p. (2008). depression and the workplace: a progress report. current psychiatry reports, 10(1), 73-79. birch, j . (2015). beyondblue newaccess demonstration independent evaluation, summary of findings, reporting to august 2015, available at https://www.beyondblue.org.au/docs/default -source/research-projectaustralasian journal of information systems islam, miah, kamal & burmeister 2019, vol 23, research on health information systems approaches to measure mental health conditions 19 files/bw0353_beyondblue-newaccess-demonstration-independentevaluation.pdf?sfvrsn=7e1050ea_0 accessed on 6 june 2018 boland, r. (2002). design in the punctuation of management action. managing as designing: creating a vocabulary for management education and research, in: boland, r. (ed.) frontiers of management workshop. weatherhead school of management, june 14-15, 106-112. boyce, m., secker, j., johnson, r., floyd, m., grove, b., schneider, j., & slade, j. (2008). mental health service users’ experiences of returning to paid employment. disability & society, 23(1), 77-88. brohan, e., henderson, c., little, k., & thornicroft, g. (2010). employees with mental health problems: survey of uk employers' knowledge, attitudes and workplace practices. epidemiology and psychiatric sciences, 19(4), 326-332. burmeister, o. k., islam, m. z., dayhew, m., & crichton, m. (2015). enhancing client welfare through better communication of private mental health data between rural service providers. australasian journal of information systems, 19, 1-14. doi:http://dx.doi.org/10.3127/ajis.v19i0.1206 bzdok, d., & meyer-lindenberg, a. (2017). machine learning for precision psychiatry. arxiv preprint arxiv:1705.10553. choi, i., milne, d. n., glozier, n., peters, d., harvey, s. b., & calvo, r. a. (2017). using different facebook advertisements to recruit men for an online mental health study: engagement and selection bias. internet interventions, 8, 27-34. cousins, r., mackay, c. j., clarke, s. d., kelly, c., kelly, p. j., & mccaig, r. h. (2004). ‘management standards’ work-related stress in the uk: practical development. work & stress, 18(2), 113-136. cox, t., & cox, s. (1993). occupational health: control and monitoring of psychosocial and organisational hazards at work. journal of the royal society of health, 113(4), 201-205. cox, t., leka, s., ivanov, i., & kortum, e. (2004). work, employment and mental health in europe. work & stress, 18(2), 179-185. danziger, s., frank, r. g., & meara, e. (2009). mental illness, work, and income support programs. american journal of psychiatry, 166(4), 398-404. de choudhury, m., counts, s., & horvitz, e. (2013). social media as a measurement tool of depression in populations. paper presented at the proceedings of the 5th annual acm web science conference. french, j. r., caplan, r. d., & van harrison, r. (1982). the mechanisms of job stress and strain (vol. 7): chichester [sussex]; new york: j. wiley. gabriel, p., & liimatainen, m.-r. (2000). mental health in the workplace: introduction, executive summaries, cornel university ilr school, digitalcommons, available: http://digitalcommons.ilr.cornell.edu/gladnetcollect/223/, accessed on 8 june 2018. gregor, s., & hevner, a. r. (2013). positioning and presenting design science research for maximum impact. mis quarterly, 37(2). australasian journal of information systems islam, miah, kamal & burmeister 2019, vol 23, research on health information systems approaches to measure mental health conditions 20 hatch, s. l., harvey, s. b., dandeker, c., burdett, h., greenberg, n., fear, n. t., & wessely, s. (2013). life in and after the armed forces: social networks and mental health in the uk military. sociology of health & illness, 35(7), 1045-1064. hu, h., li, j., wang, h., & daggard, g. (2006). combined gene selection methods for microarray data analysis. paper presented at the international conference on knowledge-based and intelligent information and engineering systems. hu, h., li, j., wang, h., daggard, g., & shi, m. (2006). a maximally diversified multiple decision tree algorithm for microarray data classification. paper presented at the proceedings of the 2006 workshop on intelligent systems for bioinformatics-volume 73. huang, j., peng, m., wang, h., cao, j., gao, w., & zhang, x. (2017). a probabilistic method for emerging topic tracking in microblog stream. world wide web, 20(2), 325-350. joyce, s., modini, m., christensen, h., mykletun, a., bryant, r., mitchell, p. b., & harvey, s. b. (2016). workplace interventions for common mental disorders: a systematic metareview. psychological medicine, 46(4), 683-697. joyce, t., mcmillan, m., & hazelton, m. (2009). the workplace and nurses with a mental illness. international journal of mental health nursing, 18(6), 391-397. kabir, m. e., wang, h., & bertino, e. (2012). a role-involved purpose-based access control model. information systems frontiers, 14(3), 809-822. khalil, f., wang, h., & li, j. (2007). integrating markov model with clustering for predicting web page accesses. paper presented at the proceeding of the 13th australasian world wide web conference (ausweb07). laitinen-krispijn, s., & bijl, r. (2000). mental disorders and employee sickness absence: the nemesis study. social psychiatry and psychiatric epidemiology, 35(2), 71-77. lamontagne, a. d., keegel, t., & vallance, d. (2007). protecting and promoting mental health in the workplace: developing a systems approach to job stress. health promotion journal of australia, 18(3), 221-228. landsbergis, p. a., schnall, p. l., deitz, d. k., warren, k., pickering, t. g., & schwartz, j. e. (1998). job strain and health behaviors: results of a prospective study. american journal of health promotion, 12(4), 237-245. lin, y.-k., chen, h., brown, r. a., li, s.-h., & yang, h.-j. (2017). healthcare predictive analytics for risk profiling in chronic care: a bayesian multitask learning approach. mis quarterly, 41(2). miah, s.j., vu, qh, gammack, j. & mcgrath, m. (2017). a big-data analytics method for tourist behaviour analysis, information and management, 54, 771-785 miah, s.j. & gammack, j. (2014). ensemble artifact design for context sensitive decision support, australasian journal of information systems 18(2), 5-20 michie, s., & williams, s. (2003). reducing work related psychological ill health and sickness absence: a systematic literature review. occupational and environmental medicine, 60(1), 39. australasian journal of information systems islam, miah, kamal & burmeister 2019, vol 23, research on health information systems approaches to measure mental health conditions 21 mislove, a., marcon, m., gummadi, k. p., druschel, p., & bhattacharjee, b. (2007). measurement and analysis of online social networks. paper presented at the proceedings of the 7th acm sigcomm conference on internet measurement. noblet, a., & lamontagne, a. d. (2006). the role of workplace health promotion in addressing job stress. health promotion international, 21(4), 346-353. nunamaker jr, j. f., chen, m., & purdin, t. d. (1990). systems development in information systems research. journal of management information systems, 7(3), 89-106. oecd. (2012). sick on the job? myths and realities about mental health and work. paris: oecd publishing retrieved from http://dx.doi.org/10.1787/9789264124523-en. oleksy, t. a. (2017). machine learning methods for mood disorder decision support. the university of bergen. roberts, r., ryan, k., burmeister, o., and maylea, c. (2018) a national priority: the physical health of people living with mental illness, australian nursing & midwifery journal, 25(10), 42. sayers, j. m., cleary, m., hunt, g. e., & burmeister, o. k. (2017). service and infrastructure needs to support recovery programmes for indigenous community mental health consumers. international journal of mental health nursing, 26, 142–150. sharma, v. k., & copeland, j. r. (2009). detecting mental disorders in primary care. mental health in family medicine, 6(1), 11. sun, x., wang, h., li, j., & zhang, y. (2011). injecting purpose and trust into data anonymisation. computers & security, 30(5), 332-345. szasz, t. s. (1960). the myth of mental illness. american psychologist, 15(2), 113-118. http://dx.doi.org/10.1037/h0046535 teipel, s., babiloni, c., hoey, j., kaye, j., kirste, t., & burmeister, o.k. (2016). information and communication technology solutions for outdoor navigation in dementia. alzheimer's & dementia: the journal of the alzheimer's association, 12(6), 695-707. doi:10.1016/j.jalz.2015.11.003 umanandhini, m. d., & kalpana, g. (2017). survey on stress types using data mining algorithms. stress, 4(04). von alan, r. h., march, s. t., park, j., & ram, s. (2004). design science in information systems research. mis quarterly, 28(1), 75-105. wang, y., li, h., wang, h., zhou, b., & zhang, y. (2015). multi-window based ensemble learning for classification of imbalanced streaming data. paper presented at the international conference on web information systems engineering. yin, x.-x., hadjiloucas, s., & zhang, y. (2017). pattern classification pattern classification of medical images: computer aided diagnosis (pp. 93-128): springer. yin, x., hadjiloucas, s., & zhang, y. (2016). classification of thz pulse signals using twodimensional cross-correlation feature extraction and non-linear classifiers, computer methods and programs in biomedicine, 127, 64-82. australasian journal of information systems islam, miah, kamal & burmeister 2019, vol 23, research on health information systems approaches to measure mental health conditions 22 copyright: © 2019 islam, miah, kamal & burmeister. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems feddema, harrigan & wang 2021, vol 25, research on user involvement the dark side of social media engagement 1 the dark side of social media engagement: an analysis of user-generated content in online wildlife trade communities kim feddema the university of western australia, australia paul harrigan zayed university, united arab emirates the university of western australia, australia paul.harrigan@uwa.edu.au shasha wang queensland university of technology, australia abstract there is a growing academic interest on the dark side of engagement on social media and the role of user-generated content (ugc). the illicit trade of wildlife online is a major contributor to global species loss and, thus, strategies to reduce demand for wild species and consumer engagement in the market and are of paramount importance. we first conduct qualitative analyses on a large data set of ugc (n=14,729 words from 1060 comments from 12 facebook groups) from a biodiversity hotspot, the republic of indonesia. we use automatic machinelearning lexical software to explore the discourse that occurs in comments of posts that promote behaviour change and demand reduction. then, to examine the efficacy of these posts, we test an extended elaboration likelihood model to determine the nature of information processing that leads viewers to agree with wildlife conservation content. our results show that motivation, opportunity and ability factors moderate the relationship between information processing and comment valence, as well as influencing whether comments indicate attitude change. this work extends the use of theory from information systems and marketing to conservation, and provides both conceptual and practical recommendations to encourage behaviour change and reduce the harmful effects of engagement. keywords: consumer engagement, wildlife trade, ugc, moa, elm 1 introduction consumer engagement has received attention across multiple disciplines, from marketing to education to social psychology (alvarez-milán et al., 2018; beckers, van doorn & verhoef, 2018; harmeling et al., 2017). according to brodie et al. (2013) “consumer engagement in a virtual brand community involves specific interactive experiences between consumers and the brand, and/or other members of the community… [it] is a multidimensional concept comprising cognitive, emotional, and/or behavioural dimensions, [which] plays a central role in the process of relational exchange” (brodie et al., 2013 p. 107). in practice, it is characterised by interactive user behaviours, such as writing a review on social media, commenting on an instagram post, responding to questions in a discussion forum, or using a brand’s hashtag (beckers, van doorn & verhoef, 2018; eigenraam et al., 2018; schivinski, christodoulides & dabrowski, 2016). this user-generated content (ugc) is defined as content that i) is made australasian journal of information systems feddema, harrigan & wang 2021, vol 25, research on user involvement the dark side of social media engagement 2 publicly available over the internet, ii) reflects a certain amount of creative effort, and iii) is created outside of professional routines and practices (vickery & wunsch-vincent, 2007). recently, consumer engagement research has begun considering the potential negative impacts of consumer interactions online. baccarella et al. (2018) identified seven building blocks of the “dark side” of social media, (i) sharing inappropriate content, (ii) location tracking, (iii) misinformation, disinformation and aggressive engagement, (iv) exploitation of the online self (v) threat, coercion, abuse and intimidation, (vi) in-group out-group bias and (vii) shaming and defamation. this is supported by research that has found controversial and/or negative content is overrepresented on social media platforms and that ugc can result in the proliferation of hate speech, pornography, and defamation, and create communities that encourage violence, self-harm and invasion of privacy (george & scerri, 2007; shelton & skalski, 2014). more recently, research has explored ugc that stretches to criminal activity, including radicalisation and the spread of terrorism (lara-cabrera, gonzalez-pardo, & camacho, 2019; yar, 2018). traditionally, illicit or taboo contexts can be difficult, or even dangerous, to research. therefore, investigating ugc online may be critical to understanding previously inaccessible phenomena (mackey, et al., 2018; skålén, aal, & edvardsson, 2015). one such phenomenon is the trade of wild animals, which contributes to the decline of many species across the globe (esmail et al., 2019). within online wildlife trade communities, species are bought and sold by social media users, who either post as individual traders or who represent brands. as a result, there is growing desire from conservation practitioners to utilise social media as a tool to reduce the supply and demand of wildlife products to protect both animal populations and public health (blundell & mascia, 2005; martin et al., 2018; nijman, 2010; nijman et al., 2012; veríssimo & wan, 2018; yar, 2018). such initiatives primarily provide messages aimed at creating attitude change and encouraging pro-environmental actions (moorhouse et al., 2020; veríssimo & wan, 2018). in our context, these messages take the form of demand reduction posts that explain relevant laws governing the breeding and ownership of wildlife, or that use emotive imagery and examples to show the negative impacts of wildlife ownership on welfare and conservation. note that not all social media users within trade groups will view these messages. therefore, in our paper we distinguish between the term, ‘users’ (i.e. all group members), and ‘viewers’ (i.e. those who have looked at the post). additionally, we do not have access to data around the reach or number of views of posts, but rather ugc, which represents only the opinion of ‘commenters’ (i.e. those viewers who have commented on the post). this is a higher-order form of behavioural engagement. few attempts have been made to examine the efficacy of these conservation programs in shifting stakeholder attitudes and behaviours online within a real-world context (moorhouse, et al., 2017; nuno et al., 2018). from an information systems perspective, ugc posted in direct response to such content presents a unique opportunity to inform and improve strategies to alter consumer behaviour. in this paper, we first provide an overview of the literature on social media engagement, online wildlife trade communities, and two behaviour change theories – the elaboration likelihood model (elm) and motivation-opportunity-ability theory (moa) previously, used to elicit detailed insights into attitude formation in social media communities (abid, harrigan, & roy, 2019; pee & lee, 2016; roy, balaji, & nguyen, 2020). underpinning our research in these theories enables us to leverage the opportunity that ugc provides to explore consumer australasian journal of information systems feddema, harrigan & wang 2021, vol 25, research on user involvement the dark side of social media engagement 3 sentiment, information sharing, and the psychological processes that are required to convince consumers to shift their attitudes and behaviours. then, across two studies, we assess the responses uploaded by group members on demand reduction posts within online wildlife trade communities across indonesia. over a period of 10 months we conducted passive observations in 12 facebook groups from across java and sumatra that advertise native indonesian species for sale. we then collected and analysed a final dataset totalling 36 posts and 1060 comments. study 1 uses the semantic analysis program leximancer 4.5.1 to complete a detailed and objective textual analysis of the data and investigate comments responding to demand reduction posts. we highlight salient concepts and operationalise the dimensions of moa present in the data. study 2 builds on this by combining moa theory with the elm in a quantitative model of information processing. this was used to test the role of cognitive routes and behavioural constructs on comment valence, defined here as the extent to which commenters agree or disagree with the content (positive and negative valence respectively). the contributions of this research are four-fold. first, we exemplify the dark side of social media engagement in a unique context. we apply semantic analysis to shed light on the community response to demand reduction posts that aim to educate viewers on the ethics and legality of their behaviours, and shift consumers to adopt pro-conservation attitudes. second, we extend the elm with moa theory, and apply it in a unique context to explore how post attributes and group members’ attitudes influence their behaviours upon viewing conservation messages. third, we collect and analyse a large ugc dataset and use it to operationalise key concepts, which is especially meaningful given the dominant role of ugc in the dark side of engagement where traditional self-report surveys may not be possible, or may not reflect real-world behaviours. finally, we provide actionable recommendations for conservation marketers and is managers using social media posts for behaviour change initiatives, such as effective post length, content types and monitoring strategies. 2 literature review 2.1 the dark side of engagement research has shown how online engagement can have negative impacts on business success (hollebeek & chen, 2014; juric, smith, & wilks, 2016) and users’ mental health and wellbeing, with active engagement with peers on social media sites, such as facebook, leading to significant feelings of stress, jealousy and anxiety in young people (fox and moreland, 2015). for example, in some circumstances the negative emotions that arise due to comparing oneself to others can result in increased negative body image in young women (hogue & mills, 2019) or depressive symptoms (feinstein et al., 2013). on a wider scale, pearce (2015) demonstrated the use of social media by an authoritarian power to coerce the public and maintain political control. finally, aswani, kar & ilavarasan (2019) highlight the devastating consequences for both individuals and institutions if users spread misinformation through ugc on social platforms, particularly where that information pertains to medical, criminal or political content. 2.2 online wildlife trade few studies have extended the engagement literature to the consequences that arise when engagement on social media fosters demand for illicit products, other than limited research on australasian journal of information systems feddema, harrigan & wang 2021, vol 25, research on user involvement the dark side of social media engagement 4 the trade of drugs and alcohol (alhabash et al., 2015; lavorgna, 2016; van de ven & koenraadt, 2017). in the wildlife trade context, illicit behaviour includes the sale of protected species, the sharing of animal cruelty images and misinformation, and privacy issues (feddema et al., 2020; morcatty et al., in press). engagement with ugc in particular can lead to harmful behaviours not only for the content creator, due to health and legal risks posed from handling dangerous and illegal wildlife, but also the animals at the centre of the trade, and indeed the environmental ecosystem. the use of online social media platforms for the trade of wildlife has facilitated the sale of millions of plants and animals and is a major contributor to global species loss, (esmail et al., 2019; kareiva et al., 2002; krishnasamy & stoner, 2016; phassaraudomsak & krishnasamy, 2018). social media significantly increases the capacity for vendors to engage with potential consumers and has serious and widespread consequences, both for the global conservation of species as well as for human health, as the current covid-19 pandemic has illustrated. morcatty et al. (in press) showed that covid-19 was used as by some vendors to increase sales of wild animals online and that few precautions were taken to reduce the risk of disease transfer between animals and humans. across south east asia, thousands of endangered species are sold per month over facebook groups and instagram posts. as one example, in 2016, 12 thai facebook groups were monitored for just 30 minutes per day and within a single 23-day period researchers had observed over 1521 animals from over 200 species being sold (krishnasamy & stoner, 2016; phassaraudomsak & krishnasamy, 2018). the scale and severity of south east asian trade is therefore a current research priority for both academics and practitioners (broad, 2020; esmail et al., 2019; kareiva et al., 2002). 2.3 indonesian context in this study, we specifically look at the trade in the indonesian context. indonesia experiences over-harvesting and illicit trade across all animal taxa and is the global epicentre of the songbird crisis (nijman et al., 2017; nijman et al., 2017a; nijman, shepherd, & sanders, 2012). the two main governing bodies are i) the convention on international trade of endangered species of wild fauna and flora (cites), which governs international trade of species between member nations, and (ii) the directorate general of the conservation of natural resources and ecosystems (ksdae) which falls under the jakarta ministry of environment and forestry (klhk) and is responsible for national legislation. programs and permits are then implemented and enforced on the ground by the office of conservation of natural resources (bksda). additionally, the vulnerability of extant species is measured by the international union of the conservation of nature (iucn) and placed on a scale between the most at risk of extinction (critically endangered) and the least (least concern). on the 1st of june 2018 indonesia updated their list of protected species (peraturan menteri lingkungan hidup dan kehutanan republik indonesia nomor p.20) for the first time since 1999, adding hundreds of new species to the protected species list (gokken, 2018). additionally, during the period in which data were collected there were ongoing reports that the government may be considering revising the main regulation that governs wildlife trade (undang-undang nomor 5 tahun 1990) for the first time in 29 years (icel, 2019). in many cases, however, even legal species are subject to maximum harvesting or ownership quotas that are routinely flouted, making it impossible to determine the actual legality of individual animals. finally, there may be species that are currently legal to own, yet are vulnerable to over-harvesting and population decline. therefore, for this research we do not consider it australasian journal of information systems feddema, harrigan & wang 2021, vol 25, research on user involvement the dark side of social media engagement 5 practically relevant to classify species traded according to their legality under national or international law. 2.4 elaboration likelihood model the elaboration likelihood model (elm) has been used in is research over several decades to understand how people process information during attitude formation and behaviour change (petty, cacioppo, & schumann, 1983). it posits that there are two processing routes within the brain (i) the central route, in which the viewer personally considers the information they feel is most important to establish their attitudinal position; (ii) the peripheral route, in which the viewer does not use their own judgement but is persuaded by simple cues that are associated either positively or negatively with the content. literature suggests that when information processing occurs through the central route, attitudinal change is more likely to be enduring and predictive of future behaviours, whereas when information is processed through the peripheral route it is more likely to be temporary and susceptible to influence (petty, cacioppo, & schumann, 1983). understanding the processing of information is therefore paramount to combatting the dark side of engagement and creating conservation education campaigns that lead to long-lasting behaviour change and positive conservation outcomes. the elm framework has seen limited use in conservation projects, and has predominantly been used to evaluate the success of specific education campaigns in zoos or outreach programs (gore et al., 2008; jacobson et al., 2018; macdonald, milfont, & gavin, 2016). in marketing, previous research has used the elm to investigate what leads users to engage with mobile services (narang, jain, & roy, 2012), facebook second-hand marketplaces (chang, lu, & lin, 2020), and even online health interventions (short et al., 2015). it is also a well-accepted model in the is research for use in testing the influence of cues on information technology acceptance (bhattacherjee & sanford, 2006; li, 2013), web personalisation (ho & bodoff, 2014), and behaviour on social media (abid, harrigan & roy, 2019; chang, yu, & lu, 2015; teng, khong, & goh, 2014). 2.5 motivation-opportunity-ability theory motivation-opportunity-ability (moa) provides a detailed framework to understand the mechanisms behind information processing and the formation of attitudes (macinnis & jaworski, 1989). the theory posits that information processing occurs when the viewer is motivated and has both the ability and the opportunity to process information (poiesz & robben, 1996). this differs from elm as it distinguishes between ability and opportunity and provides a more in-depth exploration of the determinants behind antecedents of behaviour (petty et al., 2004). moa theory can therefore complement the elm in explaining the drivers of attitudes in our context. previous engagement research has used moa theory to explain the user engagement process with social media information sharing (pee & lee, 2016), c2c exchanges (gruen, osmonbekov, & czaplewski, 2005; gruen, osmonbekov, & czaplewski, 2007) and hotel and travel marketing (leung & bai, 2013). while it is not yet widely used within is research, recent work by roy, balaji & nguyen (2020) uses the model to explore in-store smart technology adaptation. moa theory has yet to be applied to the wildlife trade context. australasian journal of information systems feddema, harrigan & wang 2021, vol 25, research on user involvement the dark side of social media engagement 6 3 data set we accessed a range of indonesian wildlife trade groups on facebook and searched within them for all demand reduction posts. these posts included messages designed to reduce the demand for wildlife and limit the trade of animals within the groups, and/or providing information on the connection between trade behaviours, wild population decline and species conservation. to do this, we conducted a word search of the terms “bksda” as well as “dilindungi”, “di lindungi”, “konservasi” and “apendik”. we also included their english translations “threatened”, “protected”, “conservation” and “appendix” respectively and downloaded all posts that were returned. these search terms were selected using snowball sampling in which the main terms associated with demand reduction efforts in both bahasa indonesia and english were initially used and further searches were then conducted with spelling variants and terms discovered within the returned results. this was repeated until no more posts were returned and we had reached saturation. data were downloaded directly from facebook to give a final dataset of 36 posts that presented the negative aspects of wildlife trade and 1060 associated comments from 12 facebook groups. seven groups each had one post and five groups each had 2, 4, 6, 8 and 9 posts, respectively. these groups predominantly represented themselves as groups that facilitated the buying, selling or ownership of wildlife across all taxa, although two groups specialised in the trade of primates (n(posts) = 9) and songbirds (n(posts) = 11). comments that were deemed to be ‘noise’ were removed from the dataset leaving a total of 188 comments comprised of 14,729 words for analysis. this data set was used for analysis in studies 1 and 2. 3.1 ethical considerations we acknowledge that our data contains potentially sensitive content, and significant effort was made to ensure all ethical considerations were met. while the content analysed was not intended to be used for research purposes, previous research suggests that there can be a reasonable expectation by users that social platforms are considered public spaces and that content uploaded to the sites may be viewed as such (burkell et al., 2014). all data was passively collected from the groups without communicating with any group members about trade behaviours or interfering with any behaviour. where we required permission from group administrators to gain access to groups, this was requested and granted on average within 24-48 hours, in-line with the experience of siriwat, nekaris & nijman (2019). to reduce the risk to participants, all data were anonymised and no profile information was collected on group members. however, it should be noted that many profiles observed operated under pseudonyms, further suggesting an awareness that they may be observed. all work was completed with approval from the relevant university’s human ethics research committee, under case number ra/4/20/5008. 4 study 1 4.1 semantic analysis study 1 examined the text pattern of demand reduction posts and their comments, achieved through a semantic analysis with leximancer. online communities create large volumes of text-based data that can be analysed using ‘text mining’ software and semantic analysis to create a visualisation of the data and elaborate on common concepts and patterns within the australasian journal of information systems feddema, harrigan & wang 2021, vol 25, research on user involvement the dark side of social media engagement 7 data1. leximancer is an automated semantic mapping tool that uses statistical algorithms to analyse blocks of text and identify themes and concepts using frequencies and co-occurrences of words. while leximancer does have the capacity to conduct sentiment analysis, this was not possible to complete in our case due to the nature of the unique sentence structure used in social media posts (leximancer, 2017). as leximancer is fully enabled to process multiple languages, our analysis was conducted without having to first translate the data, reducing the risk of bias or error through mistranslation or misinterpretation by the research team (leximancer, 2017). however due to the use of slang and dialects, data was cleaned and formatted prior to running the program to maximise the accuracy of interpretation. due to ethical considerations, the researchers did not fully immerse themselves ethnographically in the groups, not taking part in any behaviours or discussions, but were able to passively follow the groups for ten months prior to the data analysis. this, along with experience in previous research, provided a level of fluency in the slang being used and thus an ability to recode words into official bahasa indonesia. during this process, the coder was careful not to change grammar or to add or remove any words that had the potential to change the meaning of the text. an illustrative example of this style of recoding is given below in english: commenter 1 – mi? dnt want td, can gv 30? commenter 1 (recoded) – how much is it? don’t want trade, can give 30? after running the analyses, the data were sorted into files to allow for comparison between the groups and between the species clades and the pdf ‘data exports’ and ‘dashboard reports’ were downloaded (wilk, soutar & harrigan, 2017). all outputs were reported in bahasa indonesia but have been translated for the purpose of this paper. 4.2 results we first provide context by reporting descriptive findings of the demand reduction messages to provide a full understanding of the information and the message framing that the commenters received. our initial descriptive findings showed the top 10 concepts in order of relevance were protected, fauna, bird, species, wiratno, captivity, year, flora, nature and community. figure 1 shows the leximancer distance map, which indicates that the three most prominent themes (written in coloured font) are protected, wiratno and criminality. the protected theme contains the concepts flora and fauna (written in black font), and these terms (or more specifically their bahasa indonesia equivalents) are more often used in ‘scientific’ or ‘official’ terminology, distinguishing them from the less significant theme ‘animal’ which is used more broadly. the theme also includes the concept year and this can be explained by the number of posts that contained direct quotes from the regulations, for example – “jakarta, ministry of environment and forestry (klhk), monday, august 6, 2018. 1 a key benefit of using leximancer is its ability to generate an impartial concept map using self-seeded constructs without requiring input from the researcher (wilk, soutar, & harrigan, 2017). these maps can be read using a “heat map” approach in which the warmer the colour (i.e. towards the red/orange spectrum), the more central the theme is to the group. additionally, the higher the number of concepts within a theme indicates that is it more prevalent in the text, however the size of the circle itself is not indicative of importance. the more overlap the themes have with one another the more linked the two are within the text and direct lines between concepts suggests that they are tightly linked within the text. australasian journal of information systems feddema, harrigan & wang 2021, vol 25, research on user involvement the dark side of social media engagement 8 ….there is an additional list of protected species of plants and animals in p.20 / 2018, which is 241 species or 26% of the list listed in the attachment to government regulation no. 7 of 1999”. posts that directly quote official regulations also account for the second most prominent theme wiratno, who is the director general of the ksdae, as well as the theme klhk. the term protected specifically refers to legal protection and is not used to mean protected on an individual level or to be protected from immediate physical danger. this term is used both within the quotes from official ksdae documentation, as above, but also by individual traders and commenters as illustrated by the following quotes – “buying, selling, possessing, raising endangered or protected species is against the law!!” and “it’s better to delete maybe the posts with protected animals”. figure 1. leximancer distance map for all demand reduction posts the next strongest theme, criminality, shows that within these trade groups demand reduction messages are typically presented through the lens of rules and regulations, rather than moral or ecological arguments. this is further supported by the fact that all terms typically associated with ecological and biological conservation efforts are clustered within the theme nature, which is a much cooler temperature on the heat map (i.e. less important within the discourse) and directly connected to the wiratno theme. this is reflected in quotes such as “every person is prohibited from releasing protected animals from one place in indonesia to another place in or outside indonesia”. the only biological clade strong enough to create a concept within the data is the bird clade. this concept is also directly linked to the term community and this is possibly due to the cultural role that birds play within indonesian society or specifically in relation to the ‘kicau mania’ or songbird competition community that is highly prevalent in java. all comments on these posts were then analysed separately and a second distance map was created (figure 2). this was used to determine the most salient concepts to the commenters, as well as to operationalise the moa dimensions for study 2. the ranked concepts show the top ten concepts in order of relevance were protected, animal, bird, sell, bksdae, forest, cat, appendix, know and want. the heat map in figure 2 indicates three dominant themes within the australasian journal of information systems feddema, harrigan & wang 2021, vol 25, research on user involvement the dark side of social media engagement 9 comments, being protected, species and forest cat in that order. it should be noted that in this context, the term bksdae was used as a port-manteau of the ksdae and bksda. the protected theme differs in composition to the previous map of the demand reduction posts, encompassing a greater number of concepts, including the terms appendix and sell. the appendix concept is associated with the categorisation of species protected by cites legislation into different appendices depending on the severity of punitive action. this is particularly interesting given that the initial distance map (of the demand reduction posts) did not identify any themes referring to the cites regulations, instead only identifying wiratno. finally, the lack of overlap between the themes was highly apparent within the map, with ‘protected and forest cat’ and ‘protected and know’ having the only overlap. figure 2. shows the leximancer distance map from all response comments. the protected theme differs in composition to the previous map of the demand reduction posts, encompassing a greater number of concepts, including the terms appendix and sell. the appendix concept is associated with the categorisation of species protected by cites legislation into different appendices depending on the severity of punitive action. this is particularly interesting given that the initial distance map (of the demand reduction posts) did not identify any themes referring to the cites regulations, instead only identifying wiratno. finally, the lack of overlap between the themes was highly apparent within the map, with ‘protected and forest cat’ and ‘protected and know’ having the only overlap. 4.3 motivations, opportunity and ability the leximancer maps were used to explore the way potential moa constructs were discussed within the comments and to operationalise them. to explore how these terms were used in context, the prominence values for all paired concepts was downloaded and is presented in table 1 to further indicate the most frequent terms used in association with our motivation, opportunity and ability variables. prominence values reflect the co-occurrence between concepts and a value greater than 1.0 suggests that the co-occurrence is more often than by chance and a score of over 3.0 is deemed sufficient to identify compound concepts (leximancer, 2017; wilk, soutar & harrigan, 2017) australasian journal of information systems feddema, harrigan & wang 2021, vol 25, research on user involvement the dark side of social media engagement 10 4.3.1 motivation concepts demonstrating the commenters’ attitudes towards conservation and their desire to engage in trade behaviours were considered to demonstrate the ‘motivation’ variable. initially we intended only to identify terms that indicated motivation to complete pro-conservation behaviours, however the leximancer output identified both the don’t concept as well as three concepts that were determined to represent motivation to continue participating in trade behaviour, sell, buy, want. these were included as we consider them to indicate a negative motivation towards conservation action. the prominence values for the concepts most frequently paired with the four ‘motivation’ terms showed that the term buy was most frequently paired with sell, bird, forest cat, wild and appendix; the term sell was most frequently paired with buy, appendix, species, animal, wild, forest, cat and forest cat, and the term most frequently paired with don’t was infant (table 1). the term don’t was also used to express an attitude toward behaviours, as evidenced by the quotes “don’t just sit by and watch as animals are taken from their habitat, fed rice and domesticated” and “if you have a pet that’s over 2 months old you’re less likely to get bloating, don’t adopt fresh ones from the forest”. the term want showed no significant correlation with other terms despite ranking as one of the top concepts. this suggests that use of the term may vary considerably across the comments. comments were then further explored using quotes from the output to determine in what context these terms were used and to establish the level of variation among the discourse, illustrated by quotes including “i bought [the animal] to keep it, not to abandon it *smiley emoticon*”, “how are you selling appendix and other protected species?”, “never again buy birds from the market stalls, such as the bar winged prinia, flower pecker bird, sunbirds or other species” and “i just said please report it if anyone is selling appendix animals”. 4.3.2 opportunity our intention was to include two measures of opportunity within the study model; the length of the post in words and the posters’ perception of their own opportunity to carry out the intended action without consequence. as facebook content is static and not time sensitive, allowing readers to spend as much time as they wish to comprehend the post, the length of the post represents the length of time that the reader has to be influenced or persuaded by the message within the post. within the mapping results of the initial demand reduction posts we can see that the term ‘criminal’ is identified as a key theme, although it is of lower importance to the discussion than many of the other themes. no terms relating to punitive action or legal consequences were found to be concepts or themes within the discussion threads of the comments and therefore we did not use their perception of consequence as a measure of opportunity in study 2. we did, however, explore the comments themselves through illustrative quotes. we found that some commenters did not believe legal action would reduce their ability to carry out trade behaviours safely, saying “it all depends on if we react, uncle, if you think about the purple-throated sunbird, or all species of sunbirds, they’ve always been protected but in fact there are still many market stalls that sell them” or encouraging the sale of animals only to trusted people “meaning sell it to your family/friends, trading is under the clause (referring to legislation)”, as well as comments from those who were very concerned by the legal risks such as “do not buy an appendix species or keep one as a pet. sooner or later, the police and local specialists will find out”. australasian journal of information systems feddema, harrigan & wang 2021, vol 25, research on user involvement the dark side of social media engagement 11 4.3.3 ability within the ‘subjective knowledge’ variable, the intention was to establish whether it could be operationalised by differentiating between the knowledge of national regulations, cites regulations and/or the iucn threat status of the animals. it is important to reiterate that subjective knowledge only refers to the participant’s own belief that they understand the concept, and therefore it was not assessed whether the statements that were made by commenters were accurate or not. table 1. shows the prominence values for all paired concepts in the comment data. australasian journal of information systems feddema, harrigan & wang 2021, vol 25, research on user involvement the dark side of social media engagement 12 concepts in bold and marked with an asterisk are significant. the mapping results showed prominent discussion of cites international regulations, such as “at least we already know the rules, as much as possible try not to keep, kill or trade appendix species and species protected by the law” and “why do they want to make the permissions so complicated? … whose fault is that?” as well as discussions about confusion surrounding the ksdae regulations “the forest rangers should give us a list of names of all the species that are protected or not, so ordinary people like me know which ones can be kept”. this contrasts with the demand reduction posts, where the only theme or concept representing a governing body was wiratno, representing the ksdae. initially we considered using three variables to measure the ability construct (i) subjective knowledge of ksdae regulations, (ii) subjective knowledge of cites regulations and (iii) subjective knowledge of iucn regulations. however, the only discussion of iucn status that appeared within either of leximancer outputs was the term ‘extinct’(see figure 2). therefore, we determined that iucn threat status was not a useful indicator of subjective ability for our model. as such, we refined our model to include only the subjective knowledge of cites regulations and ksdae regulations as indicators of ability. these data show how discourse analysis of ugc can provide detailed insight into the way that communities respond to demand reduction messages. through the qualitative analysis of comment data, we identified key concepts that commenters feel are important in their understanding of conservation and wildlife trade laws. while this approach is useful in identifying group patterns, it cannot provide evidence of individual attitude formation or change, thus we also used study 1 to refine and confirm our variables for a complementary quantitative study. our variables are as follows: motivation as attitude towards conservation (macinnis & jaworski, 1989); opportunity as the post length (abid, harrigan & roy, 2019); and ability as the subjective knowledge of both cites and ksdae regulations (poiesz & robben, 1996). 5 study 2 our second study seeks to assess attitude formation in response to demand reduction messages posted within social media groups that trade wildlife. to do this, we have created an extension of the model that synthesises the traditional central and peripheral processing routes with moa theory. specifically, we will assess how the dual routes of information processing and the viewer’s existing knowledge, attitudes and values towards conservation and trade influence the valence of comments by consumers, that is ugc, towards the demand reduction posts (figure 3). as in elm, the model considers two routes of information processing, the central and the peripheral route. central route processing, in this case, involves a cognitive assessment of the quality of the argument presented in the post’s text, while the peripheral route consists of cues based on post vividness, popularity and source credibility. argument quality is one of the most discussed elements of social media message processing and denotes the persuasiveness of the information presented (teng, khong & goh, 2014). in this study, we define the argument quality of each message as either strong or weak, based upon work by petty, cacioppo & schumann (1983) and petty & cacioppo (1984). a weak message is one that may have the same number of arguments as a strong message, but the australasian journal of information systems feddema, harrigan & wang 2021, vol 25, research on user involvement the dark side of social media engagement 13 arguments are less persuasive, or that includes fewer (strong) brand assertions. additionally, we posit that emotionally charged messages will tend to be processed via the peripheral route, and those that promote negative mood will reduce the efficacy of message processing (fu & chen, 2012; mundorf, zillmann, & drew, 1991). figure 3. shows the proposed research model for study 2 many peripheral cues have been identified and used in the literature, particularly within the social media context. we have chosen to focus on post vividness, popularity and source credibility. source credibility has been found to play a critical role where the argument quality cannot be assessed by the viewer or where the messages are deemed irrelevant (teng, khong & goh, 2014). likewise, post popularity is measured by the number of likes, comments and shares on posts and has been found to be one of the most crucial determinants of post persuasiveness (chang, lu & lin, 2020). finally, post vividness, used to illustrate post aesthetics, has been shown to influence the preferences of viewers and the endurance of attitude change, particularly on social media (chang, yu & lu, 2015; coyle & thorson, 2001; lin, swarna & bruning, 2017). based on this literature, our first hypotheses are as follows: h1a – argument quality will have a positive impact on comment valence. h1b – peripheral cues will have a positive impact on comment valence. 5.1 motivation, opportunity, ability theory constructs the consumer’s motivation is their desire and readiness to complete a behaviour (macinnis, moorman, & jaworski, 1991). motivation is often conceptualised as ‘personal relevance’; in other words, the extent to which the message is seen to ‘apply to them’ (bhattacherjee & sanford, 2006; pee & lee, 2016). in our study, commenters who show strong positive or australasian journal of information systems feddema, harrigan & wang 2021, vol 25, research on user involvement the dark side of social media engagement 14 negative attitude towards conservation are presumed to assess the content as appropriate and relevant to them. those who have a neutral opinion on conservation are seen to have a low motivation to process and therefore are more likely to process the information via the peripheral route (tang, jang & morrison, 2012). additionally, those who believe that trade actions benefit conservation (through captive breeding or protection of rare individuals) are seen to have low motivation and those who believe that trade is harmful for conservation are seen to have high motivation. this is predicted to moderate the influence of both the central and peripheral routes as follows: h2a– motivation will have a positive influence comment valence. h2b– low motivation (i.e., neutral attitude towards conservation) will strengthen the relationship between peripheral cues and comment valence. h2c– high motivation (i.e., negative or positive attitude towards conservation) will strengthen relationship between argument quality and comment valence. post length has been studied previously as a peripheral cue (teng, khong & goh, 2014) and is associated with source style and aesthetics on social media (de vries, gensler & leeflang, 2012), however it is accepted that peripheral cues do not always act solely through the peripheral route (petty et al., 2004). additionally, as social media content is not time-restricted, and can be viewed at any time, post exposure cannot accurately be measured as a unit of time. finally, visible content is replaced as new posts are uploaded to the site and there is a tendency of social media users to consume content very quickly. we therefore suggest that the length of the post is an appropriate metric of ‘opportunity’. therefore, we hypothesise as follows: h3a – opportunity (i.e., longer posts) will positively influence comment valence. h3b – opportunity (i.e., longer posts) will strengthen the relationship between peripheral cues and comment valence. h3c – opportunity (i.e., longer posts) will strengthen the relationship between argument quality and comment valence. we do not define ‘ability’ by the commenters’ objective knowledge of law and regulation in indonesia as these regulations and enforcement vary considerably across the archipelago. nor do we consider their perceived ability to house and care for animals as, in many cases, husbandry is undertaken on a small, ad-hoc scale that requires few resources and does not abide by the welfare standards that a researcher would typically advise. finally, it would not be suitable to use objective truth as a measure of intent since the commenter can only base their own actions on what they believe to be the case. for these reasons, we determined the most appropriate measure of ability for our study would be the creators’ subjective knowledge, in other words, the extent to which the commenter is confident that they know the national and international protections for the species and the impact of trade behaviour on species populations (poiesz & robben, 1996). we hypothesise as follows: h4a – ability will positively influence comment valence. h4b – ability (i.e., subjective knowledge of the legal regulations around trade) will weaken relationship between peripheral cues and comment valence. h4c – ability (i.e., subjective knowledge of the legal regulations around trade) will weaken the relationship between argument quality and comment valence. australasian journal of information systems feddema, harrigan & wang 2021, vol 25, research on user involvement the dark side of social media engagement 15 5.2 methods most variables are coded from posts as twoto three-level categorical variables (see table 2). continuous variables, such as popularity and length of the posts, were transformed into threelevels categorical variables based on the 33.3% and 66.6% cut off points. the moderating relationship of ability was tested as two dimensions, first using the cites variable and then again using the ksdae variable to determine any salient differences. a binary logistic regression (blr) was used to test the main effect in hypothesis 1. hierarchical binary logistic regressions were used to test the moderation effects in the following hypotheses. all analyses were run in spss v25. a regression approach was used to test the moderators because each of the constructs was measured by one item/variable, and the dependent variable is binary. more importantly, it allows researchers to compare the effects before and after adding a moderator (ro, 2012). variables code results dependant variable comment valence (n=188) 3agree positive (59%) 2no opinion neutral (0%) 1disagree negative (41%) independent variables argument quality (n=188) 3strong, many high (72.3%) 1weak, none low (27.7%) source credibility (n=188) 3credible high (42.6%) 1no source low (57.4%) vividness (n=188) 3video 2image high (62.8%) 0% 1text low (37.2%) popularity (n=188) 3range (138-4316) high (40.4%) sum of likes, comments, 2range (34-119) moderate (27.7%) shares 1range (5-32) low (31.9%) moderators motivation (cons) (n=164) attitude towards conservation efforts 0 – not mentioned 0% 3 positive positive attitude (51.8%) 2 neutral neutral attitude (15.9%) 1 negative negative attitude (32.3%) opportunity (length) (n=155) 3range (157-816) long (38.7%) total number of words in 2range (28-103) moderate (25.8%) post 1range (3-25) short (35.5%) ability (cites) (n=114) 3high high knowledge (49.1%) level of subjective knowledge 2medium moderate knowledge (17.5%) 1low low knowledge (33.3%) ability (ksdae) (n=127) 3high high knowledge (52.8%) level of subjective knowledge 2medium moderate knowledge (17.3%) 1low low knowledge (29.9%) noise comments with emoticons only comments that did not reflect the topic of conversation multiple identical comments by a single commenter comments that only provide an image or link comments that only tag other accounts comments that were ambiguous or did not make sense comments that were unable to be confidently translated by any team members and were assumed to be ambiguous table 2. descriptive analysis of variables australasian journal of information systems feddema, harrigan & wang 2021, vol 25, research on user involvement the dark side of social media engagement 16 assumptions (e.g., no outliers, binary dependent variable, little to no multicollinearity issues) for blr were tested and satisfied (field, 2009). please note that the effective sample in the moderation analysis may be smaller than 188 for some constructs, as moderators were coded from posts containing a certain number of 'missing values'. this operation reflects the reality of field data, but we suggest future researchers to include more posts in their analysis of similar data. 5.3 results and discussion 5.3.1 h1 test: dual routes of information processing the model had a good fit with a significant χ² statistic (χ² (4) = 10.045, p<0.05; hosmerlemeshow p>0.05). the model explained between 7.5% (cox & snell r2) and 10.1% (nagelkerke r2) of the variance in comment valence, and correctly classified 62.2% of cases. as shown in appendix a, argument quality was negatively related to comment valence (b=0.621, p<0.01). a strong argument was 0.537 times less likely to enhance comment valence than a weak argument. consistent with the blr, a cross tab analysis also found that posts that hold a strong argument (52.9%) were less likely to show a positive comment valence than those with a weak argument (75%, χ² (1)= 7.569, p<0.01). this result is significant but contradictory to h1a, and h1a was not supported. none of the peripheral cues showed significant relationships (p>0.05), thus h1b was not supported. to explain these findings, we draw on psychological reactance theory in which viewers of persuasive messages become aversely psychologically aroused if they feel their freedoms are being diminished (brehm, 1966; steindl et al., 2015, shen, 2015). previous research in choicelimiting messages for advertising and health communication has indicated that the stronger the level of persuasion, the more psychological pushback the viewer may experience, leading to a distrust of the information and a rejection of the message (gardner & leshner, 2016; weiger, hammerschmidt & wetzel, 2018). commenters may therefore reject demand reduction posts with high argument quality due to a belief that their freedom and autonomy to make their own choices is being curtailed. in study 1, the differences we observed between the discourse in the demand reduction posts as opposed to the comments, further supports that there remains a disconnect between the message-framing used and its interpretation by group members. specifically, we found that demand reduction posts predominantly focused upon punitive control measures for reducing wildlife trade, as opposed to ecological or moral arguments for behaviour change. indeed, rather than providing information that encourages group members to change their attitudes, the posts overwhelming aim to inform them of punitive consequences that limit their ability to carry out behaviours. we recommend that future research empirically test psychological reactance as a result of individual demand reduction campaigns using the measurement constructs put forth by dillard and shen (2005), where reactance theory is fundamentally concerned with specific, context-based behaviours (miron & brehm, 2004). we also recommend that future research explore links between wildlife trade and other illicit behaviours, as the intensity of psychological reactance has been linked to the perceived risk of future limitations on associated behaviours (i.e. if they take away one freedom, what stops them taking away others?) (brehm & sensenig, 1966). australasian journal of information systems feddema, harrigan & wang 2021, vol 25, research on user involvement the dark side of social media engagement 17 this is particularly relevant given the literature on reactance restoration, in which viewers of psychologically arousing messages seek to re-assert their control by either performing the restricted behaviour, or illicit behaviours perceived as similar (quick & kim, 2009).we therefore recommend that trials take place to determine the most appropriate measures to use for each target audience, as it is likely they may differ according to culture, beliefs and other personality factors (steindl & jonas, 2012; woller, buboltz & loveland, 2007). our study 2 results did not support hypothesis 1b, suggesting that there was no significant direct impact of peripheral cues on comment valence. however, our subsequent analyses showed that this was moderated by the moa dimensions, as discussed below. 5.3.2 h2 test: motivation the moderating effects were tested by a hierarchical blr analysis. the hierarchy 1 model had a good fit with a significant χ² statistic (χ² (5) = 96.44, p<0.001; hosmer-lemeshow p>0.05). the model explained between 44.5% (cox & snell r2) and 59.8% (nagelkerke r2) of the variance in comment valence, and correctly classified 84.1% of cases. as shown in appendix a, commenters’ attitude towards conservation was added to the baseline model as a test of the motivation construct, however this did not change the relationships between argument quality or peripheral cues and comment valences. motivation itself was positively associated with comment valence (b=2.120, p<0.001). commenters who were more positive towards conservation were 8.329 times more likely to have a positive comment valence than those who were more negative. consistent with the blr, a cross tab analysis also found that more commenters who hold a positive attitude (80%) tend to show a positive comment valence, compared to those who hold a neutral (12%) or negative (7.4%) attitude towards conservation (χ² (2)= 79.541, p<0.001). therefore, h2a was supported. these findings are in-line with past literature that has demonstrated the link between motivation and attitude formation (clark, abela, & ambler, 2005; gruen, osmonbekov & czaplewski, 2007). it may seem an obvious result that those who hold a positive attitude towards conservation agree with demand reduction posts, however, given the prevalence of behaviours that are antithetical to conservation within the groups such as animal cruelty, over-harvesting, discussions of illegal consumption of endangered species etc. it is perhaps a more complicated relationship than it appears. in study 1, we found that some commenters within multi-species trade groups consider their own actions as pro-environmental, while judging the behaviour of others to be unethical. as an example, it is possible that captive breeders view themselves as morally superior to hunters, pet owners view themselves as morally superior to wild meat consumers etc. on a post detailing the hunting of an orangutan (pongo abelii) in sumatra, commenters remarked “curse the perpetrator!”, “very cruel” and “i hope that the perpetrator receives immediate guidance from god, as no-one can avoid the laws of the creator”. however the same commentators later openly remarked about consuming the meat of endemic surilis (presbytis spp.) and selling a southern mitered langur (presbytis mitrata), listed on appendix 2 of cites and deemed vulnerable to extinction by the iucn. therefore, we recommend creating multiple, specific campaigns targeting different trade activities to ensure wide coverage of motivation and beliefs and reduce the perception that the information does not apply to all traders. the hierarchy 2 model had a good fit with a significant χ² statistic (χ² (9) = 105.819, p<0.001, hosmer-lemeshow p>0.05). the model explained between 47.5% (cox & snell r2) and 63.9% (nagelkerke r2) of the variance in comment valence, and correctly classified 84.1% of cases. australasian journal of information systems feddema, harrigan & wang 2021, vol 25, research on user involvement the dark side of social media engagement 18 the interaction between vividness and motivation (b=-0.760, p=0.05) and the interaction between popularity and motivation (b=-0.868, p=0.05) were both marginally significant. figure 4a shows that a neutral attitude towards conservation strengthens the relationship between vividness and comment valence compared to a negative or positive motivation level. figure 4b shows that when people held a negative attitude towards conservation, the popularity of the post could lead to a switch towards a positive comment valence. this result is in line with our prediction. however, popularity did not influence people who had a positive attitude towards conservation. more interestingly, when people were less motivated or had a neutral attitude towards conservation, popularity led to a negative comment valence. this result is contradictory to our prediction. these results indicate potential moderations and therefore h2b was partially supported. as shown in appendices b and f, the moderation on argument quality and comment valence was not significant, therefore, h2c was rejected. figure 4a. h2 moderations by motivation: motivation moderates vividness → comment valence figure 4b. h2 moderations by motivation: motivation moderates popularity → comment valence 0% 17% 18% 60% 44% 29% 90% 91% 89% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 0.5 1 1.5 2 2.5 3 3.5 c o m m e n t v al en ce popularity negative attitude neutral attitude positive attitude low moderate high australasian journal of information systems feddema, harrigan & wang 2021, vol 25, research on user involvement the dark side of social media engagement 19 the moderating effect of motivation on the relationship between peripheral cues and comment valence indicates that when commenters have a low motivation to process information, the impact of vividness is strengthened and becomes significant, with comments becoming more positive in general (h2b). this supports our belief that, under conditions in which the commenter does not have strong opinions on conservation either way, they will look to peripheral cues to assist their information processing (tang, jang & morrison, 2012). this result however was complicated our contradictory findings that an increase in the popularity of the post had a negative impact on comment valence. by contrast, where viewers hold a negative attitude towards conservation, the popularity of the post could lead to positive comments. it is unclear at this stage what may be causing these results, however, the influence of social peers and post popularity has been shown to have a strong influence for social media users generally and in particular for collectivist cultures such as indonesia (de vries, gensler & leeflang, 2012; lin, swarna & bruning, 2017). 5.3.3 h3 test: opportunity the hierarchy 1 model did not have a good fit with an insignificant χ² statistic (χ² (5) = 4.975, p>0.05; hosmer-lemeshow p>0.05). the model explained between 3.2% (cox & snell r2) and 4.2% (nagelkerke r2) of the variance in comment valence, and correctly classified 61.3% of cases. source credibility was the only significant factor, after adding post length, i.e. the opportunity to process the message. there was no significant relationship between opportunity and comment valence (p>0.05). therefore, h3a was not supported. outside of controlled experimental design, it is difficult to control the exposure of any commenter to a message in order to increase opportunity. this is particularly true of social media, which is not time-limited and presents information that can be viewed retrospectively (chu, kamal & kim, 2019). it is possible that this structure explains the lack of relationship between the opportunity construct and comment valence. the hierarchy 2 model had a good fit with a significant χ² statistic (χ² (9) = 26.128, p<0.001) and insignificant hosmer-lemeshow test statistic (p>0.05). the model explained between 15.5% (cox & snell r2) and 20.8% (nagelkerke r2) of the variance in comment valence, and correctly classified 62.6% of cases. as shown in appendix c, the opportunity (post length) moderated the relationships between source credibility (b=0.917, p<0.01), vividness (b=-0.858, p<0.05), popularity (b=-1.588, p<0.05) and comment valence. figure 5a shows the positive relationship between source credibility and positive valence only happened when the post was longer. figure 5b shows the positive relationship between vividness and comment valence only happened when the post was longer. figure 5c shows the positive relationship between popularity and comment valence only happened when the post was moderate or longer. therefore, h3b was supported, but h3c was not supported. those sharing posts from credible sources should therefore ensure that their posts are no shorter than 157 words long if they want to boost positive comments, while those sharing posts that lack sources deemed credible by the viewers should keep them short. the use of long posts does not appear to strengthen the negative impact of argument quality and therefore, we would predict no harmful effects of using longer posts. where possible, our findings recommend increasing the length of all posts to at least 28 words. additionally, it has been found that increasing redundancy of information by repeating key messages can create a greater opportunity for both brand recognition and information processing within the duration of the ad or post (macinnis, moorman & jaworski, 1991). australasian journal of information systems feddema, harrigan & wang 2021, vol 25, research on user involvement the dark side of social media engagement 20 figure 5a. h3 moderations by opportunity (post length): opportunity (length) moderates source credibility → comment valence figure 5b. h3 moderations by opportunity (post length): opportunity (length) moderates vividness → comment valence 54% 56% 63% 62% 9% 63% 0% 10% 20% 30% 40% 50% 60% 70% 0.5 1 1.5 2 2.5 c o m m en t v al e n ce source credibility low length moderate length high length low high 55% 55% 74% 63% 47% 53% 0% 10% 20% 30% 40% 50% 60% 70% 80% 0.5 1 1.5 2 2.5 c o m m en t v al e n ce vividness low length moderate length high length low high 73% 54% 55% 67% 50% 63% 0% 10% 20% 30% 40% 50% 60% 70% 80% 0.5 1 1.5 2 2.5 c o m m en t v al e n ce popularity low length moderate length high length low high australasian journal of information systems feddema, harrigan & wang 2021, vol 25, research on user involvement the dark side of social media engagement 21 figure 5c. h3 moderations by opportunity (post length): opportunity (length) moderates popularity → comment valence 5.3.4 h4 test: ability although study 1 found that the ‘ksdae’ concept was not as salient in the comments as cites was, in study 2, slightly more group members show high level of knowledge in ksdae laws (52.8%) than cites laws (49.1%, see table 2). this finding was unexpected, as it was predicted that wildlife traders’ knowledge of the national regulations may have been impacted by the changes to the laws that were occurring in the lead up to data collection, see section 2.3. in fact, as some of the posts that were included in our study were focused on detailing these new laws, it is possible that this biased our results to suggest the viewers had a greater level of knowledge than in reality due to the fact they were reacting to these informative posts. none of the posts, by contrast, aimed to inform the group members about the correct cites appendix list for traded species. hypothesis 4 was tested in two formats; knowledge of cites regulations and knowledge of ksdae regulations, respectively. the hierarchy 1 model for ability (cites) had a good fit with a significant χ² statistic (χ² (5) = 32.799, p<0.001; hosmer-lemeshow p>0.05). the model explained between 25.0% (cox & snell r2) and 33.4% (nagelkerke r2) of the variance in comment valence, and correctly classified 72.8% of cases. as shown in hierarchy 1 of appendix d, commenters’ knowledge of cites regulations, or ability (cites), was added to the baseline model. adding this new variable did not change the relationships between argument quality and comment valences, but source credibility became a significant and positive predictor of comment valence (b=0.576, p<0.05). ability (cites) increased comment valence (b=1.217, p<0.001). commenters that had a higher-level ability (cites) were 3.337 times more likely to show a positive comment valence than those who had a lower ability. therefore, when the model was run using the cites variable, h4a was supported. the hierarchy 1 model for ability (ksdae) had a good fit with significant χ² statistic (χ² (5) = 23.849, p<0.001; hosmer-lemeshow p>0.05). the model explained between 17.1% (cox & snell r2) and 22.8% (nagelkerke r2) of the variance in comment valence, and correctly classified 69.3% of cases. as shown in appendix e, apart from argument quality (b=-0.856, p<001), source credibility (b=0.462, p<0.05) was also a significant factor after adding ability (ksdae). in addition, ability (ksdae) had a positive main effect on comment valence (b=0.915, p<0.001). commenters who had a higher-level of knowledge about ksdae regulations had 2.497 times higher tendency to drive a positive comment valence compared to those who had a lower level of such knowledge. therefore, when the model was run with the ksdae variable, h4a was also supported. this reflects the importance of consumers’ ability to process information (in this case subjective knowledge of regulations) in forming their attitudes to demand reduction posts and supports previous research around the moderating role of the ability construct within moa (binney, hall & oppenheim, 2007; poiesz & robben, 1996). this relationship suggests that an increase in education about both national and international regulations would greatly improve the likelihood of viewers to agree with demand reduction posts. future education campaigns should consider group members’ existing perceptions of cites and ksdae legislation, law enforcement practices, corruption, and economic opportunity. simplifying messages to only contain relevant information will improve message clarity and prevent misunderstanding and misrepresentation. for example, education programs explaining the australasian journal of information systems feddema, harrigan & wang 2021, vol 25, research on user involvement the dark side of social media engagement 22 variance in species nomenclature, in particular across dialects, would ensure group members understand which regulations relate to the species they are likely to encounter in their networks. as discussed, our findings relate only to subjective knowledge i.e. group members understanding. however, as mentioned above, variation in naming conventions can create ambiguity in species identification. additionally, there is significant complexity across provincial, national, international and social restrictions on hunting and trade. therefore, it is likely that there are traders who feel confident that they understand the laws but who are unwittingly engaging in illegal trade behaviours. therefore, it is critical that education campaigns verify that their audience has accurately interpreted their information and are confident they will be able to recall it in their daily lives. the hierarchy 2 model (cites) had a good fit with a significant χ² statistic (χ² (9) = 46.969, p<0.001; hosmer-lemeshow p>0.05). the model explained between 33.8% (cox & snell r2) and 45.0% (nagelkerke r2) of the variance in comment valence, and correctly classified 78.1% of cases. as shown in appendix d, while the moderation on source credibility and positive comment valence was not significant, the ability (cites) variable moderated the relationships between argument quality (b=-1.151, p<0.05), vividness (b=-1.476, p<0.01), popularity (b=1.419, p<0.01) and comment valence. figure 6a shows both a moderate and high level of ability (cites) weaken the positive relationship between vividness and positive valence. figure 6b shows the positive relationship between popularity and positive valence only occurs when commenters hold a high level of knowledge (cites). therefore, when the model was run using the cites variable, h4b was partially supported. figure 6c shows a moderate level of ability (cites) weakens the negative relationship between argument quality and comment valence, compared to the two extreme low or high level of knowledge. therefore, when the model was run using the cites variable, h4c was supported. figure 6a. h4 moderations by ability (cites): ability (cites) moderates vividness → comment valence 7% 44% 40% 20% 77% 73% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 0.5 1 1.5 2 2.5 p o si ti ve v al e n ce vividness low knowledge moderate knowledge high knowledge low high australasian journal of information systems feddema, harrigan & wang 2021, vol 25, research on user involvement the dark side of social media engagement 23 figure 6b. h4 moderations by ability (cites): ability (cites) moderates popularity → comment valence figure 6c. h4 moderations by ability (cites): ability (cites) moderates argument quality → comment valence the hierarchy 2 model (ksdae) had a good fit with a significant χ² statistic (χ² (9) = 29.140, p<0.01; hosmer-lemeshow p>0.05). the model explained between 20.5% (cox & snell r2) and 27.4% (nagelkerke r2) of the variance in comment valence, and correctly classified 69.3% of cases. as shown in appendix d, after adding interaction terms, the only significant interaction term was ability (ksdae) and argument quality (b=-0.908, p<0.05). the negative relationship between argument quality and comment valence was stronger among people who had a high level of knowledge about ksdae (figure 7). therefore, when the model was run using the ksdae variable, h4c was partially supported. 70% 40% 33% 29% 14% 50% 0% 10% 20% 30% 40% 50% 60% 70% 80% 0.5 1 1.5 2 2.5 c o m m en t v al e n ce popularity low knowledge moderate knowledge high knowledge low high 60% 24% 40% 27% 100% 68% 0% 20% 40% 60% 80% 100% 120% 0.5 1 1.5 2 2.5 c o m m e n t v al en ce argument quality low knowledge moderate knowledge high knowledge low high australasian journal of information systems feddema, harrigan & wang 2021, vol 25, research on user involvement the dark side of social media engagement 24 figure 7. h4 moderations by ability (bksdae) our findings support our hypothesis that the strength of the relationship between argument quality and comment valence would be weakened by the ability dimension but only under conditions were commenters had a moderate level of knowledge about cites regulations. when knowledge is very low, commenters lack the ability to process information and when knowledge is very high, they experience psychological reactance (grandpre et al., 2003). if viewers are confident that they are already cognisant of the content presented, they may feel they have no requirement to process additional information, increasing the aversion when they perceive their autonomy or choices are being limited (spingarn, 1988). the results from the analysis of the knowledge of ksdae support this theory, as they show that high levels of knowledge about the ksdae strengthen the negative effect of argument quality, further confirming that high perceived knowledge leads to a rejection of the information. in contrast, where viewers had moderate knowledge, they were in a ‘psychological goldilocks’ wherein having just the right amount of knowledge or motivation enables them to both process and be persuaded by information. where campaigns seek to inform group members on legislation and enforcement, work must be done to ensure that local communities are made fully aware that actions taken are in the best interest of the community, and that punitive measures are enforced fairly and consistently. this would require consultation with local people and law enforcement, ethnographic research, and regular review by species specialists. additionally, to reduce psychological resistance, practitioners could frame messages to encourage a feeling of similarity and trust in the messenger, as well as suggesting to the viewer that they have a moderate level of knowledge about regulations. for example, messages such as “thank you all for teaching me about the bksda regulations were any of you aware that there are harvest quotas for many species that limit the number of animals that can be traded legally?”. we found that when viewers had low levels of knowledge of cites regulations, commenters were attracted by high vividness of posts to provide positive comments, supporting elm research that suggests that those with low ability to process information tend to be more influenced by peripheral cues (petty et al., 2004). conversely, when posts were popular, only group members that had high knowledge of cites regulations tended to post positive 50% 29% 40% 35% 100% 62% 0% 20% 40% 60% 80% 100% 120% 0.5 1 1.5 2 2.5 c o m m en t v al en ce argument quality low knowledge moderate knowledge high knowledge low high australasian journal of information systems feddema, harrigan & wang 2021, vol 25, research on user involvement the dark side of social media engagement 25 comments. it is possible that as posts gained likes and comments, group members felt that more eyes were upon them, and there was an increased pressure to conform to social norms. previous research into environmental messaging has shown a complex relationship between peer comparison, social norms, and personal attitudes when messages are viewed on social media and this effect should not be discounted (hynes & wilson, 2016). where it is determined that most group members have low subjective knowledge of regulations, posts should be designed with a greater focus on the visual aesthetics and interactivity to gain higher numbers of positively valanced comments. as group members will not process the information via the central route, it will be more efficient to maximise the potential of the peripheral cues to create behaviour change. this can be done by using high quality videos or photos combined with texts and links, as was shown by yousaf et al. (2020). our results also suggest that, to be most effective, vivid posts should provide as much opportunity for information processing as possible, either through having long videos or adding large text sections alongside their image content. 7 implications and conclusion user-generated content (ugc) in online wildlife trade communities is predominantly dark, in that it promotes the trade of species that are over-harvested, threatened with extinction and treated with poor animal welfare. across two studies, this research has investigated the impact of demand reduction posts as they attempt to shine a light on these issues within communities. these posts have the potential to lessen the negative impact of consumer engagement, yet globally they remain under-studied and the behavioural responses of trade participants is relatively poorly understood. consumer behaviours are specific and context-dependent and future research will be able to test our findings in other communities and contexts, in and outside of wildlife trade. to that end, we have carried out an extensive data collection and analysis of big data to draw recommendations for one trade segment: the online wildlife communities for indonesian species. our research synthesises techniques from marketing, information systems and conservation to assess ugc content in online wildlife trade groups. it is therefore uniquely placed to provide insights in all three fields. first, we contribute to consumer engagement research by applying the elm and moa theory in parallel to explain the antecedents to, and moderators of, information processing. combining these two theories has allowed us explain behaviours that have previously not been explored due to the difficulty in accessing and interpreting datasets from traditionally inaccessible communities. second, we collect big data from current online user communities, operationalising and analysing key theoretical elm and moa constructs through real-world ugc, rather than traditional self-report surveys, offering opportunities for future information systems research to replicate and advance our work in other contexts. specifically, one of our studies applies a novel semantic analysis technique to a large ugc dataset. finally, this multidisciplinary study should lay the foundation for future research in conservation to draw on information systems and marketing models to explain key behaviours, and information systems and marketing researchers should be encouraged to expand the scope of their engagement research beyond traditional commercial settings. practically, we provide advice for conservation practitioners using social media and big data to pay careful attention to both the argument and emotional qualities of their posts, but also to be aware of the specifics around viewer psychology, cultural attitudes around persuasion australasian journal of information systems feddema, harrigan & wang 2021, vol 25, research on user involvement the dark side of social media engagement 26 and identity and level of knowledge around legal frameworks. as one of the group members from our study eloquently stated “you have to be self-aware; when changing the mindset of apathetic people who don’t want to know, it will be very difficult as their brains and feelings are hard as rocks and they don’t want to accept the truth”. acknowledgments we would like to thank dr vincent nijman and dr. kai nekaris for their insights into indonesian trade culture and karlina anggarendra, amank and rifqi hendrik for their assistance with translation. makasih banyak. we would also like to thank the two reviewers for their useful feedback and assistance with the manuscript. finally, we wish to acknowledge the traditional owners of the land on which the manuscript was written, the whadjuk people of the noongar nation, and pay our respects to the elders past, present and emerging. references abid, a., harrigan, p., & roy, s.k. (2019). online relationship marketing through content creation and curation. marketing intelligence & planning. 38(6), 699-712. doi:10.1108/mip-04-2019-0219 alhabash, s., mcalister, a.r., quilliam, e.t., richards, j.i., & lou, c. (2015). alcohol's getting a bit more social: when alcohol marketing messages on facebook increase young adults’ intentions to imbibe. mass communication and society, 18(3), 350-375. doi: 10.1080/15205436.2014.945651 alvarez-milán, a., felix, r., rauschnabel, p.a., & hinsch, c. (2018). strategic customer engagement marketing: a decision making framework. journal of business research, 92, 61-70. doi: 10.1016/j.jbusres.2018.07.017 aswani, r., kar, a.k., & ilavarasan, p.v. (2019). experience: managing misinformation in social media—insights for policymakers from twitter analytics. journal of data and information quality (jdiq), 12(1), 1-18. doi: 10.1145/3341107 baccarella, c.v., wagner, t.f., kietzmann, j.h., & mccarthy, i.p. (2018). social media? it's serious! understanding the dark side of social media. european management journal, 36(4), 431-438. doi: 10.1016/j.emj.2018.07.002 beckers, s.f., van doorn, j., & verhoef, p.c. (2018). good, better, engaged? the effect of company-initiated customer engagement behavior on shareholder value. journal of the academy of marketing science, 46(3), 366-383. doi: 10.1007/s11747-017-0539-4 bhattacherjee, a., & sanford, c. (2006). influence processes for information technology acceptance: an elaboration likelihood model. mis quarterly, 30(4), 805-825. doi:10.2307/25148755 binney, w., hall, j., & oppenheim, p. (2007, january). the moa framework and behavioural response. in anzmac 2007: 3rs, reputation responsibility relevance: proceedings of the 2007 australian and new zealand marketing academy conference, 1144-1151. university of otago, school of business, dept. of marketing. http://hdl.handle.net/10536/dro/du:30008150 blundell, a.g., & mascia, m.b. (2005). discrepancies in reported levels of international wildlife trade. conservation biology, 19(6), 2020-2025. doi:10.1111/j.15231739.2005.00253.x australasian journal of information systems feddema, harrigan & wang 2021, vol 25, research on user involvement the dark side of social media engagement 27 brehm, j.w. (1966). a theory of psychological reactance. academic press. brehm, j. w., & sensenig, j. (1966). social influence as a function of attempted and implied usurpation of choice. journal of personality and social psychology, 4(6), 703–707. doi:10.1037/h0023992 broad, s. (2020). wildlife trade, covid-19 and zoonotic disease risks: shaping the response. retrieved from https://www.traffic.org/publications/reports/wildlife-trade-covid-19and-zoonotic-disease-risks-shaping-the-response/ brodie, r. j., ilic, a., juric, b., & hollebeek, l. (2013). consumer engagement in a virtual brand community: an exploratory analysis. journal of business research, 66(1), 105-114. doi: 10.1016/j.jbusres.2011.07.029 burkell, j., fortier, a., wong, l & simpson, j (2014) facebook: public space, or private space?, information, communication & society, 17(8), 974-985, doi: 10.1080/1369118x.2013.870591 cacioppo, j. t., & petty, r. e. (1984). the elaboration likelihood model of persuasion. in na advances in consumer research. eds: kinnear, t, provo, ut: association for consumer research, 11, 673-675. chang, h., lu, y, & lin, s.c. (2020). an elaboration likelihood model of consumer respond action to facebook second-hand marketplace: impulsiveness as a moderator. information & management, 57(2), 103171. doi: 10.1016/j.im.2019.103171 chang, y, yu, h., & lu, h. (2015). persuasive messages, popularity cohesion, and message diffusion in social media marketing. journal of business research, 68(4), 777-782. doi:10.1016/j.jbusres.2014.11.027 chu, s. c., kamal, s., & kim, y. (2019). re-examining of consumers’ responses toward social media advertising and purchase intention toward luxury products from 2013 to 2018: a retrospective commentary. journal of global fashion marketing, 10(1), 81-92. doi: 10.1080/20932685.2018.1550008 clark, b.h., abela, a.v., & ambler, t. (2005). organizational motivation, opportunity and ability to measure marketing performance. journal of strategic marketing, 13(4), 241259. doi:10.1080/09652540500338014 coyle, j.r., & thorson, e. (2001). the effects of progressive levels of interactivity and vividness in web marketing sites. journal of advertising, 30(3), 65-77. doi:10.1080/00913367.2001.10673646 de vries, l., gensler, s., & leeflang, p.s. (2012). popularity of brand posts on brand fan pages: an investigation of the effects of social media marketing. journal of interactive marketing, 26(2), 83-91. doi: 10.1016/j.intmar.2012.01.003 dillard, j.p., & shen, l. (2005). on the nature of reactance and its role in persuasive health communication. communication monographs, 72(2), 144-168. doi: 10.1080/03637750500111815 eigenraam, a.w., eelen, j., van lin, a., & verlegh, p.w. (2018). a consumer-based taxonomy of digital customer engagement practices. journal of interactive marketing, 44, 102-121. doi: 10.1016/j.intmar.2018.07.002 australasian journal of information systems feddema, harrigan & wang 2021, vol 25, research on user involvement the dark side of social media engagement 28 esmail, n., wintle, b., athanas, a., beale, c., bending, z., dai, r., harrington, l. a. (2019). emerging illegal wildlife trade issues in 2018: a global horizon scan. conservation letters, 13 (4), e12715. doi:10.1111/conl.12715 feddema, k., harrigan, p., nekaris, k. a. i., & maghrifani, d. (2020). consumer engagement behaviors in the online wildlife trade: implications for conservationists. psychology & marketing, 37(12), 1755-1770. doi:10.1002/mar.21423 feinstein b., hershenberg r., bhatia v., latack j., meuwly n. & davila j. (2013) negative social comparison on facebook and depressive symptoms: rumination as a mechanism. psychology of poplur media culture, 2(3), 161–70. doi: 10.1037/a0033111 field, a. (2009). discovering statistics using spss:(and sex and drugs and rock'n'roll). sage. fox, j., & moreland, j. j. (2015). the dark side of social networking sites: an exploration of the relational and psychological stressors associated with facebook use and affordances. computers in human behavior, 45, 168-176. doi: 10.1016/j.chb.2014.11.083 fu, j.r., & chen, j. (2012). an investigation of factors that influence blog advertising effectiveness. international journal of electronic business management, 10(3), 194-203. https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.654.7872&rep=rep1&type=p df gardner, l., & leshner, g. (2016). the role of narrative and other-referencing in attenuating psychological reactance to diabetes self-care messages. health communication, 31(6), 738-751. doi: 10.1080/10410236.2014.993498 george, c.e., & scerri, j. (2007). web 2.0 and user-generated content: legal challenges in the new frontier. journal of information, law and technology, 2. available at ssrn: https://ssrn.com/abstract=1290715 gokken, b (2018, 3 august) indonesia adds hundreds of birds to protected species list, mongabay news. article retrieved from: https://news.mongabay.com/2018/08/ indonesia-adds-hundreds-of-birds-to-protected-species-list/ gore, m.l., knuth, b.a., scherer, c.w., & curtis, p.d. (2008). evaluating a conservation investment designed to reduce human–wildlife conflict. conservation letters, 1(3), 136145. doi: 10.1111/j.1755-263x.2008.00017.x grandpre, j., alvaro, e. m., burgoon, m., miller, c. h., & hall, j. r. (2003). adolescent reactance and anti-smoking campaigns: a theoretical approach. health communication, 15(3), 349-366. doi: 10.1207/s15327027hc1503_6. gruen, t., osmonbekov, t., & czaplewski, a. (2005). how e-communities extend the concept of exchange in marketing: an application of the motivation, opportunity, ability (moa) theory. marketing theory, 5(1), 33-49. doi:10.1177/1470593105049600 gruen, t., osmonbekov, t., & czaplewski, a. (2007). customer-to-customer exchange: its moa antecedents and its impact on value creation and loyalty. journal of the academy of marketing science, 35(4), 537-549. doi:10.1007/s11747-006-0012-2 harmeling, c. m., moffett, j. w., arnold, m. j., & carlson, b. d. (2017). toward a theory of customer engagement marketing. journal of the academy of marketing science, 45(3), 312-335. doi: 10.1007/s11747-016-0509-2 australasian journal of information systems feddema, harrigan & wang 2021, vol 25, research on user involvement the dark side of social media engagement 29 ho, s.y., & bodoff, d. (2014). the effects of web personalization on user attitude and behavior. mis quarterly, 38(2), 497-520. doi: 10.25300/misq/2014/38.2.08 hogue, j. v., & mills, j. s. (2019). the effects of active social media engagement with peers on body image in young women. body image, 28, 1-5. doi: 10.1016/j.bodyim.2018.11.002 hollebeek, d & chen, t. (2014), "exploring positivelyversus negatively-valenced brand engagement: a conceptual model". journal of product & brand management, 23(1), 6274. doi: 10.1108/jpbm-06-2013-0332 hynes, n., & wilson, j. (2016). i do it, but don't tell anyone! personal values, personal and social norms: can social media play a role in changing pro-environmental behaviours?. technological forecasting and social change, 111, 349-359. doi: 10.1016/j.techfore.2016.06.034 icel (2019, 27 november) revisi undang konservasi sumber daya alam hayati digulirkan lagi, article retrieved from: https://icel.or.id/berita/icel-dalam-berita/revisi-undangundang-konservasi-sumber-daya-alam-hayati-digulirkan-lagi/ jacobson, s. k., morales, n. a., chen, b., soodeen, r., moulton, m. p., & jain, e. (2018). love or loss: effective message framing to promote environmental conservation. applied environmental education & communication, 18(3), 252-265. doi:10.1080/1533015x.2018.1456380 juric, b., smith, s., & wilks, g. (2016). negative customer brand engagement: an overview of conceptual and blog-based findings. in r. j. brodie, l. d. hollebeek, & j. conduit (eds.), customer engagement: contemporary issues and challenges (pp. 272). singapore: routledge. kareiva, p., marvier, m., west, s., & hornisher, j. (2002). slow-moving journals hinder conservation efforts. nature, 420(6911), 15-15. doi: 10.1038/420015a krishnasamy, k., & stoner, s. (2016). trading faces: a rapid assessment on the use of facebook to trade wildlife in peninsular malaysia. retrieved from petaling jaya, selangor, malaysia lara-cabrera, r., gonzalez-pardo, a., & camacho, d. (2019). statistical analysis of risk assessment factors and metrics to evaluate radicalisation in twitter. future generation computer systems, 93, 971-978. doi: 10.1016/j.future.2017.10.046 lavorgna, a. (2016). ‘how the use of the internet is affecting drug trafficking practices’, the internet and drug markets (european monitoring centre for drugs and drug addiction: insights 21, publications office of the european union, luxembourg. available from: https://www.emcdda.europa.eu/publications/insights/internet-drug-markets_en leung, x. y., & bai, b. (2013). how motivation, opportunity, and ability impact travelers' social media involvement and revisit intention. journal of travel & tourism marketing, 30(1-2), 58-77. doi:10.1080/10548408.2013.751211 leximancer (2017) leximancer manual version 4.5. available from http://info.leximancer.com li, c.y. (2013). persuasive messages on information system acceptance: a theoretical extension of elaboration likelihood model and social influence theory. computers in human behavior, 29(1), 264-275. doi:10.1016/j.chb.2012.09.003 australasian journal of information systems feddema, harrigan & wang 2021, vol 25, research on user involvement the dark side of social media engagement 30 lin, h. c., swarna, h., & bruning, p. f. (2017). taking a global view on brand post popularity: six social media brand post practices for global markets. business horizons, 60(5), 621633. doi: 10.1016/j.bushor.2017.05.006 macdonald, e., milfont, t., & gavin, m. (2016). applying the elaboration likelihood model to increase recall of conservation messages and elaboration by zoo visitors. journal of sustainable tourism, 24(6), 866-881. doi: 10.1080/09669582.2015.1091464 macinnis, d. & jaworski, b. (1989). information processing from advertisements toward an inegrative framework. journal of marketing, 53(4), 1-23. doi:10.2307/1251376 macinnis, d., moorman, c., & jaworski, b. (1991). enhancing and measuring consumers' motivation, opportunity. journal of marketing, 55(4), 32-53. doi:10.2307/1251955 mackey, t., kalyanam, j., klugman, j., kuzmenko, e., & gupta, r. (2018). solution to detect, classify, and report illicit online marketing and sales of controlled substances via twitter: using machine learning and web forensics to combat digital opioid access. journal of medical internet research, 20(4), e10029. doi: 10.2196/10029. martin, r. o., senni, c., apos, & cruze, n. c. (2018). trade in wild-sourced african grey parrots: insights via social media. global ecology and conservation, 15, e00429. doi:10.1016/j.gecco.2018.e00429 miron, a. m., & brehm, j. w. (2006). reactance theory-40 years later. zeitschrift für sozialpsychologie, 37(1), 9-18. doi:10.1024/0044-3514.37.1.9. moorhouse, t. p., balaskas, m., d'cruze, n. c., & macdonald, d. w. (2017). information could reduce consumer demand for exotic pets. conservation letters, 10(3), 337-345. doi:10.1111/conl.12270 moorhouse, t. p., coals, p. g. r., d'cruze, n. c., & macdonald, d. w. (2020). reduce or redirect? which social marketing interventions could influence demand for traditional medicines? biological conservation, 242, 108391. doi:10.1016/j.biocon.2019.108391 morcatty, t. q., feddema, k., nekaris, k. a. i., & nijman, v. (in press). online trade in wildlife and the lack of response to covid-19. environmental research. doi:10.1016/j.envres.2020.110439 mundorf, n., zillmann, d., & drew, d. (1991). effects of disturbing televised events on the acquisition of information from subsequently presented commercials. journal of advertising, 20(1), 46–53. doi: 10.1080/00913367.1991.10673206 narang, s., jain, v., & roy, s. (2012). effect of qr codes on consumer attitudes. international journal of mobile marketing, 7(2), 52-64. doi: 10.1177/1847979018812323 nijman, v. (2010). an overview of international wildlife trade from southeast asia. biodiversity and conservation, 19(4), 1101-1114. doi:10.1007/s10531-009-9758-4 nijman, v., sari, s. l., siriwat, p., sigaud, m., & nekaris, k. a. (2017). records of four critically endangered songbirds in the markets of java suggest domestic trade is a major impediment to their conservation. birdingasia, 27, 20–25, available from: https://www.orientalbirdclub.org/birdingasia-27 nijman, v., shepherd, c. r., & sanders, k. l. (2012). over-exploitation and illegal trade of reptiles in indonesia. the herpetological journal, 22(2), 83-89. australasian journal of information systems feddema, harrigan & wang 2021, vol 25, research on user involvement the dark side of social media engagement 31 nijman, v., spaan, d., rode-margono, e. j., wirdateti, & nekaris, k. a. i. (2017a). changes in the primate trade in indonesian wildlife markets over a 25-year period: fewer apes and langurs, more macaques, and slow lorises. american journal of primatology, 79(11). e22517. doi:10.1002/ajp.22517 nuno, a., blumenthal, j. m., austin, t. j., bothwell, j., ebanks-petrie, g., godley, b. j., & broderick, a. c. (2018). understanding implications of consumer behavior for wildlife farming and sustainable wildlife trade. conservation biology, 32(2), 390-400. doi:10.1111/cobi.12998 pearce, k.e. (2015). democratizing kompromat: the affordances of social media for statesponsored harassment. information, communication & society, 18(10), 1158-1174. doi: 10.1080/1369118x.2015.1021705 pee, l., & lee, j. (2016). trust in user-generated information on social media during crises : an elaboration likelihood perspective. asia pacific journal of information systems, 26, 1-22. doi:10.14329/apjis.2016.26.1.1 petty, r., rucker, d.d, bizer, g.y & cacioppo, j.t (2004) the elaboration likelihood model of persuasion, in perspectives on persuasion, social influence, and compliance gaining. eds. seiter, j & gass, r. boston, ma: allyn & bacon petty, r. and cacioppo, j. (1984), the effects of involvement on responses to argument quantity and quality: central and peripheral routes to persuasion. journal of personality and social psychology,46, 69-81. doi: 10.1037/0022-3514.46.1.69 petty, r., cacioppo, j., & schumann, d. (1983). central and peripheral routes to advertising effectiveness: the moderating role of involvement. journal of consumer research (pre1986), 10(2), 135-146. doi:10.1086/208954 phassaraudomsak, m., & krishnasamy, k. (2018). trading faces: a rapid assessment on the use of facebook to trade wildlife in thailand. retrieved from petaling jaya, selangor, malaysia poiesz, t., & robben, h. (1996). advertising effects under different combinations of motivation, capacity, and opportunity to process information. in na advances in consumer research. eds. corfman, k. & lynch j., provo, ut: association for consumer research, 23, 231-236. quick, b. l., & kim, d. k. (2009). examining reactance and reactance restoration with south korean adolescents: a test of psychological reactance within a collectivist culture. communication research, 36(6), 765-782. doi: 10.1177/0093650290346797 ranjan, k.r., & read, s. (2016). value co-creation: concept and measurement. journal of the academy of marketing science, 44(3), 290-315. doi: 10.1007/s11747-014-0397-2 ro, h. (2012). moderator and mediator effects in hospitality research. international journal of hospitality management, 31(3), 952-961. doi: 10.1016/j.ijhm.2011.11.003 roy, s.k., balaji, m.s., & nguyen, b. (2020). consumer-computer interaction and in-store smart technology (ist) in the retail industry: the role of motivation, opportunity, and ability. journal of marketing management, 36(3-4), 299-333. doi:10.1080/0267257x.2020.1736130 australasian journal of information systems feddema, harrigan & wang 2021, vol 25, research on user involvement the dark side of social media engagement 32 schivinski, b., christodoulides, g., & dabrowski, d. (2016). measuring consumers' engagement with brand-related social-media content: development and validation of a scale that identifies levels of social-media engagement with brands. journal of advertising research, 56(1), 64-80. doi: 10.2501/jar-2016-004 shelton, a. k., & skalski, p. (2014). blinded by the light: illuminating the dark side of social network use through content analysis. computers in human behavior, 33, 339-348. doi: 10.1016/j.chb.2013.08.017 shen, l. (2015). antecedents to psychological reactance: the impact of threat, message frame, and choice. health communication, 30(10), 975-985. doi: 10.1080/10410236.2014.910882 short, c., rebar, a., plotnikoff, r., & vandelanotte, c. (2015). designing engaging online behaviour change interventions: a proposed model of user engagement. the european health psychologist, 17(1), 32-38. available from: https://www.ehps.net/ehp/index.php/contents/article/view/763 siriwat, p., nekaris, k. a. i., & nijman, v. (2019). the role of the anthropogenic allee effect in the exotic pet trade on facebook in thailand. journal for nature conservation, 51, 125726. doi: 10.1016/j.jnc.2019.125726 skålén, p., aal, k. a., & edvardsson, b. (2015). cocreating the arab spring: understanding transformation of service systems in contention. journal of service research, 18(3), 250264. doi: 10.1177/1094670514559700 spingarn, j. a (1988) the pseudo self-sufficient patient. the psychotherapy patient, 4(3-4), 111-118, doi: 10.1300/j358v04n03_11 steindl, c., & jonas, e. (2012). what reasons might the other one have?—perspective taking to reduce psychological reactance in individualists and collectivists. psychology, 3, 11531160. doi: 10.4236/psych.2012.312a170 steindl, c., jonas, e., sittenthaler, s., traut-mattausch, e., & greenberg, j. (2015). understanding psychological reactance. zeitschrift für psychologie, 223(4), 205-214. doi: 10.1027/2151-2604/a000222 tang, l. r., jang, s. s., & morrison, a. (2012). dual-route communication of destination websites. tourism management, 33(1), 38-49. doi: 10.1016/j.tourman.2011.01.021 teng, s., khong, k. w., & goh, w. w. (2014). conceptualizing persuasive messages using elm in social media. journal of internet commerce, 13(1), 65-87. doi: 10.1080/15332861.2014.910729 van de ven, k., & koenraadt, r. (2017). exploring the relationship between online buyers and sellers of image and performance enhancing drugs (ipeds): quality issues, trust and self-regulation. international journal of drug policy, 50, 48-55. doi: 10.1016/j.drugpo.2017.09.004 veríssimo, d., & wan, a. (2018). characterizing efforts to reduce consumer demand for wildlife products. conservation biology, 33(3), 623-633. doi: 10.1111/cobi.13227 vickery, g., & wunch-vincent, s. (2007). web and user-created content: web 2.0, wikies and social networking. oecd publishing, paris, doi: 10.1787/9789264037472-en. australasian journal of information systems feddema, harrigan & wang 2021, vol 25, research on user involvement the dark side of social media engagement 33 weiger, w. h., hammerschmidt, m., & wetzel, h. a. (2018). don’t you dare push me: how persuasive social media tactics shape customer engagement. journal of the association for consumer research, 3(3), 364-378. doi: 10.1086/698713 wilk, v., soutar, g. n., & harrigan, p. (2017). tackling social media data analysis: comparing and contrasting qsr nvivo and leximancer. qualitative market research, 22(2), 94-113. doi: 10.1108/qmr-01-2017-0021 woller, k. m., buboltz, w. c., & loveland, j. m. (2007). psychological reactance: examination across age, ethnicity, and gender. the american journal of psychology, 15-24. doi: 10.2307/20445379 yar, m. (2018). a failure to regulate? the demands and dilemmas of tackling illegal content and behaviour on social media. international journal of cybersecurity intelligence & cybercrime, 1(1), 5-20. available at: https://vc.bridgew.edu/ijcic/vol1/iss1/3 yousaf, a., amin, i., jaziri, d., & mishra, a. (2020). effect of message orientation/vividness on consumer engagement for travel brands on social networking sites. journal of product & brand management. vol. ahead-of-print. doi: 10.1108/jpbm-08-2019-2546 appendices appendix a. h1 test results dv: comment valence b std. error exp(b) 95% ci sig. constant 1.616 1.167 5.035 0.083 h1a argument quality -0.621 0.237 0.537 0.337-0.855 0.005 h1b source credibility 0.255 0.178 1.290 0.911-1.828 0.076 vividness -0.005 0.197 0.995 0.676-1.463 0.489 popularity -0.078 0.234 0.925 0.585-1.463 0.369 appendix b. h2 test results – moderation by motivation dv: comment valence b std. error exp(b) 95%ci sig. hierarchy 1 argument quality -1.200 0.637 0.301 0.086 1.05 0.030 source credibility 0.395 0.261 1.484 0.89 2.476 0.066 vividness 0.321 0.287 1.379 0.786 2.418 0.132 popularity 0.004 0.328 1.004 0.528 1.91 0.495 h2a motivation 2.120 0.294 8.329 4.679 14.827 0.000 hierarchy 2 h2b motivation by source credibility -0.436 0.348 0.646 0.327 1.279 0.105 motivation by vividness -0.760 0.471 0.467 0.186 1.176 0.053 motivation by popularity -0.868 0.526 0.420 0.15 1.177 0.050 h2c motivation by argument quality -0.829 0.559 0.437 0.146 1.306 0.069 australasian journal of information systems feddema, harrigan & wang 2021, vol 25, research on user involvement the dark side of social media engagement 34 appendix c. h3 test results – moderation by opportunity appendix d h4 test results – moderation by ability (cites) dv: comment valence b std. error exp(b) 95%ci sig. hierarchy 1 argument quality -0.959 0.354 0.383 0.191 0.768 0.004 source credibility 0.576 0.284 1.778 1.019 3.104 0.022 vividness -0.038 0.287 0.962 0.548 1.69 0.447 popularity -0.299 0.378 0.742 0.354 1.555 0.215 h4a ability (cites) 1.217 0.277 3.377 1.96 5.817 0.000 hierarchy 2 h4b ability (cites) by source credibility 0.551 0.373 1.735 0.835 3.603 0.070 ability (cites) by vividness -1.476 0.467 0.229 0.092 0.57 0.001 ability (cites) by popularity -1.419 0.544 0.242 0.083 0.703 0.005 h4c ability (cites) by argument quality -1.151 0.580 0.316 0.101 0.986 0.024 appendix e h4 test results – moderation by ability (ksdae) dv: comment valence b std. error exp(b) 95%ci sig. hierarchy 1 argument quality -0.856 0.332 0.425 0.222 0.814 0.005 source credibility 0.462 0.256 1.587 0.961 2.623 0.036 vividness -0.110 0.254 0.896 0.544 1.474 0.332 popularity -0.302 0.343 0.739 0.377 1.448 0.189 h4a ability (ksdae) 0.915 0.240 2.497 1.56 3.997 0.000 hierarchy 2 4b ability (ksdae) by source credibility 0.203 0.303 1.225 0.676 2.218 0.252 ability (ksdae) by vividness -0.542 0.338 0.581 0.3 1.127 0.054 dv: comment valence b std. error exp(b) 95%ci sig. hierarchy 1 argument quality -0.363 0.271 0.695 0.409 1.183 0.090 source credibility 0.386 0.214 1.471 0.967 2.239 0.036 vividness -0.086 0.234 0.918 0.581 1.451 0.357 popularity -0.222 0.322 0.801 0.426 1.505 0.245 h3a opportunity (length) -0.077 0.325 0.925 0.49 1.749 0.406 hierarchy 2 h3b opportunity (length)by source credibility 0.917 0.347 2.502 1.268 4.935 0.004 opportunity (length)by vividness -0.858 0.503 0.424 0.158 1.137 0.044 opportunity (length) by popularity -1.588 0.750 0.204 0.047 0.889 0.017 h3c opportunity (length) by argument quality -12.038 8125.766 0.000 0 . 0.500 australasian journal of information systems feddema, harrigan & wang 2021, vol 25, research on user involvement the dark side of social media engagement 35 ability (ksdae) by popularity -0.525 0.439 0.591 0.25 1.398 0.116 h4c ability (ksdae) by argument quality -0.908 0.495 0.403 0.153 1.064 0.034 appendix f. shows the effect of moderations on comment valence argument quality vividness popularity source credibility low high δ low high δ low high δ low high δ moti vation negative 11 15 4 0 18 18 neutral 27 60 33 60 29 -31 positive 85 92 7 90 89 -1 opport unity low 55 55 0 73 54 -19 54 56 2 moderate 74 63 -11 55 67 12 63 62 -1 high 47 53 6 50 63 13 9 63 54 ability (cites) low 60 24 -36 7 44 37 14 50 36 moderate 40 27 -13 40 20 -20 33 29 -4 high 100 68 -32 77 73 -4 70 40 -30 ability (ksdae) low 50 29 -21 moderate 40 35 -5 high 100 62 -38 this table presents the change in percentage of comments with a positive valence (represented by delta (δ)) under each condition. copyright: © 2021 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v25i0.2987 http://creativecommons.org/licenses/by-nc/3.0/au/ https://doi.org/10.3127/ajis.v25i0.2987 paper title (paper title style) australasian journal of information systems badreddine et al. 2023, vol 27, selected papers from acis 2022 quality of cancer-related podcasts 1 integrating users’ perceptions to identify features indicating the quality of cancer-related podcasts provided by non-profit cancer organisations basma badreddine macquarie business school, sydney, australia yvette blount deakin university, melbourne, australia alireza amrollahi macquarie business school, sydney, australia ali.amrollahi@mq.edu.au abstract while cancer podcasts are valuable for support and information, there is a significant gap in understanding their quality features from users’ perspectives. understanding quality features from users’ perspectives is important to ensure that cancer-affected people receive the support they need. this study addresses this gap by combining multiple theoretical perspectives: a. the iq assessment framework, source credibility theory, two-factor theory of website design to assess the quality of podcasts, and b. reception theory to highlight listeners’ perception of quality. these perspectives, together, enrich the concept of information systems quality and provide a comprehensive understanding of podcasts’ quality. through semi-structured interviews with 17 cancer-affected individuals, the research found that credibility, content, and design were essential quality features, with the visual appearance serving as a motivational factor. the integration of reception theory highlights users’ active role in shaping quality perceptions, offering new insights into the effective design of cancer-related podcasts. this novel approach bridges a critical research gap, illuminating the complex interplay of technical and human factors in assessing podcast quality from the perspective of users. keywords: cancer podcasts, information systems’ quality, reception theory, content, credibility, design. 1 introduction the usage of a range of information and communication technology tools can contribute to the enhancement of the quality of life for users (ractham et al., 2022). with the convenience of the internet, cancer-affected people can obtain healthy lifestyle advice, build a greater understanding of health issues, and make informed decisions about treatments (lleras de frutos et al., 2020). in a study that explored internet search patterns by cancer-affected people, mcleod et al. (2017) found that 85% of patients diagnosed with cancer use the internet to access information about their disease. however, health information on the internet is of varying quality (hamzehei et al., 2018). in a study that examined the quality of 100 websites relating to pancreatic cancer, de groot et al. (2017) found that many websites lacked up-to-date information and appropriate readability levels. similarly, 32% of patients with thyroid cancer found that online thyroid cancer information could be barely understood (chang et al., 2019). australasian journal of information systems badreddine et al. 2023, vol 27, selected papers from acis 2022 quality of cancer-related podcasts 2 uncertainty about the trustworthiness and accuracy of online health information presents a barrier to health information use by users (zhao et al., 2022). previous studies on website usage patterns showed that among the different sources of online cancer information, cancer-affected people tend to access information produced by non-profit cancer organisations (npcos) more than commercial (such as pharmaceutical companies) and government sites (chang et al., 2019). online resources provided by npcos cover a range of cancer topics including understanding cancer, emotions at early stages of diagnosis, cancer treatments, managing side effects, living well after cancer, complementary therapies, cancer questions and myths, and cancer stories (mills & millward, 2019). many of these services and support are provided using different online support resources. cancer podcasts are one type of resource provided by npcos to support cancer-affected people via audio in the form of interviews, conversations, and panel discussions (scott et al., 2023). evaluation frameworks assessing the quality of health-related podcasts focused on examining podcasts’ sustainability as supplementary tools for providing clinical and palliative care guidance for healthcare professionals (nwosu et al., 2017), and supporting education on emergency practices such as paediatric (zaver et al., 2016), neurologic (grock et al., 2016), and respiratory medicine (min et al., 2018). however, there is a dearth of studies researching the use of cancer-related podcasts by considering the perspectives of patients, survivors, and caregivers. given that online information influences health-related decisions of canceraffected people seeking online advice and confirmation of decisions (gandamihardja et al., 2023), understanding the quality of cancer-related podcasts provided by npcos is important, because cancer-affected people are more likely to seek online support from npcos than from other sources (chang et al., 2019). moreover, less satisfaction with the information obtained about cancer is linked to reduced quality of life for cancer-affected people (faller et al., 2017). this study used the reception theory to gain a more comprehensive understanding of canceraffected people’s perspectives about cancer-related podcasts. unlike traditional technological perspectives, the reception theory allows the research to explore the intricate details of human perception regarding these podcasts. by examining how the audience receive, interpret, and respond to the content (hall, 2007), reception theory can complement information quality (iq) framework and provide more comprehensive explanation. this approach transcends mere technological assessment and considers the listeners’ diverse backgrounds, expectations, and emotional responses (holub, 2013). in the context of cancer, where the information can have profound impacts on decision-making and well-being, understanding human perception is critical as it ensures that the information provided is not only accurate and appealing but also resonates with the intended audience. thus, the application of reception theory in this research provides a new perspective that aligns with the human-centric nature of healthcare information, allowing for a more comprehensive evaluation of the quality of cancer-related podcasts. therefore, this paper aims to identify and examine features that indicate the quality of cancerrelated podcasts provided by npcos from the perspective of cancer-affected people in their specific context. hence, the research question that guides this study is: what features indicate the quality of cancer-related podcasts provided by npcos and how do they resonate with cancer-affected people? australasian journal of information systems badreddine et al. 2023, vol 27, selected papers from acis 2022 quality of cancer-related podcasts 3 this study contributes to the literature by providing insights into the quality indicators of podcasts provided by npcos from cancer-affected people’s perspectives. the integration of reception theory with information quality enriches our understanding of the user’s perception and communication strategies. furthermore, the study provides evidence-based findings to inform the development of cancer-related podcasts by npcos that need to allocate scarce resources efficiently and effectively when designing and selecting the content of those podcasts, enhancing the podcasts’ usability. 2 literature review health-related podcasts are commonly used in a variety of health contexts to provide information to support patients and caregivers (gopal, 2019), educate policymakers, researchers, and the public on palliative care (nwosu et al., 2017), and provide continuous medical education for health practitioners (berk et al., 2020). medical practitioners working in emergency medicine use podcasts for the most up-to-date information and research for caring for patients (grock et al., 2016; min et al., 2018; thoma et al., 2015). with the increased use of podcasts in emergency medicine, the need for evaluating podcasts’ currency and quality emerged (thoma et al., 2015). thoma et al. (2015) used experts in emergency medicine to identify quality indicators of emergency medicine podcasts. the taxonomy for quality used in that study was informed by the work of paterson et al. (2015), which involved an international cohort of health educators refining a list of 151 quality indicators for medical education podcasts to 13 relevant quality indicators subcategorised into three major themes: credibility, content, and design. however, none of the previous studies examined the quality of cancer-related podcasts used by cancer-affected people for obtaining emotional and informational support. studies evaluating users’ satisfaction with podcast technology focused on the opinions of medical practitioners using podcasts for medical education. for example, in a study that interviewed 16 users of emergency medicine podcasts, usability and convenience were the most reported features satisfying users followed by a sense of connectedness to the broader professional community (riddell et al., 2020). other perceived advantages of medical podcasts included increased comprehensibility and the acquisition of more knowledge when compared to traditional learning methods such as textbooks (back et al., 2017). podcasts are convenient online resources for cancer-affected people to obtain information in different locations, such as while waiting at clinics for treatment, in the car, on public transportation or anywhere else. however, people are more likely to integrate the information from the internet into their health decisions if the quality and value of the health information meet their needs (leung, 2008). the quality of online information can play an important role in positive outcomes. for example, relevant and sufficient information shared on a breast cancer website helped women with breast cancer make informed decisions relating to cancer treatments (bruce et al., 2015). therefore, it is important to understand features cancer-affected people consider essential when evaluating the quality of cancer-related podcasts provided by npcos. 3 theoretical framework previous studies used different measures to assess the quality of online health-related resources. for example, koo et al. (2011) classified the dimensions of quality into three main australasian journal of information systems badreddine et al. 2023, vol 27, selected papers from acis 2022 quality of cancer-related podcasts 4 streams: information quality, information presentation, and website attractiveness. in another study, dubowicz and schulz (2015) developed a framework that included the following dimensions: trustworthiness, competence, interference, layout, textual deficits, usability, and suitability. a more comprehensive framework is one developed by paterson et al. (2015) who conducted a thematic analysis of existing studies on health-related podcasts and blogs to identify quality indicators for podcasts and blogs used by learners in the health profession. the study classified quality indicators into three main themes: the credibility of the resource, quality of the shared content, and the design/layout of the website displaying the online resource. the three quality themes were further used in studies that investigated different audience groups, such as content producers to identify quality indicators for blogs and podcasts in emergency medicine and critical care (thoma et al., 2015), and health educators to identify quality indicators for social-media-based medical education podcasts and blogs (lin et al., 2015). due to its comprehensiveness and frequent use in the context of health systems, this study adopted paterson et al. (2015) taxonomy to explore the podcasts’ quality dimensions from the perspective of cancer-affected people. the conceptual framework was developed using a multi-theory perspective that integrated the information quality (iq) assessment framework (stvilia et al., 2007), the source credibility model (hovland & weiss, 1951), the twofactor theory for website design (zhang et al., 2000), and reception theory (hall, 2007). the components of the conceptual framework are discussed in the following sections. 3.1 podcasts’ quality dimensions 3.1.1 content this study used the iq assessment framework by wang and strong (1996) to establish a model for quality indicators of podcast content. the iq assessment framework consists of four main constructs capturing the following aspects: intrinsic, contextual, representational, and accessibility (wang & strong, 1996). within the health context, stvilia et al. (2009) extended the iq assessment framework to establish an evaluation tool for assessing health information quality that included five basic criteria: accuracy, completeness, authority, usefulness, and accessibility. incorporating dimensions from the iq assessment framework and following a literature review on health information quality, this study included accuracy, relevancy, usefulness, completeness, richness, currency, and interpretability as proposed dimensions of the podcast content quality. 3.1.1.1 accuracy accuracy refers to the degree to which the information is correct and free of errors (koo et al., 2011). koo et al. (2011) defined the accuracy of the shared cancer information in terms of the extent the information is consistent, believable, meaningful, unambiguous and correct. this study sought to determine whether accuracy is considered important by cancer-affected people when evaluating the quality of cancer-related podcasts. therefore, we need to understand the extent to which quality is impacted if cancer-affected people feel misled by false or conflicting opinions or statements (bernstam et al., 2008) shared on cancer podcasts. 3.1.1.2 relevancy relevancy refers to the extent to which the shared information covers topics of interest pertinent to the informational needs and questions of people with health-related concerns (kanthawala australasian journal of information systems badreddine et al. 2023, vol 27, selected papers from acis 2022 quality of cancer-related podcasts 5 et al., 2016). in a study designing an online patient education resource to manage chronic diseases, patients reported preferences for information tailored to their symptoms, guidance on self-care strategies and advice on personal treatments (win et al., 2016). however, the lack of sufficient resources can constrain npcos from providing personalised advice to online users. this study examined the extent to which it is important that information shared on podcasts covers topics relating to the interests and needs of cancer-affected people. 3.1.1.3 usefulness usefulness refers to the extent to which the information is able to provide a better understanding of specific health issues, enabling users to make informed and accurate health decisions (li et al., 2018). cancer-affected people found that useful information obtained from cancer forums offered help, advice and emotional support (hargreaves et al., 2018). in a study evaluating the effectiveness of a web-based educational program for breast cancer, the usefulness of information was the most influential factor for users’ satisfaction compared to other factors such as system design, system convenience, adequacy of information and system efficiency (yi et al., 2008). given the positive impact of the perceived usefulness of health websites on users’ intentions to use those websites (li et al., 2018), this study included usefulness in the conceptual framework for quality indicators of podcasts. 3.1.1.4 completeness completeness refers to the extent to which the resource provides the right scope and amount of information that is sufficient (boon-itt, 2019; fadahunsi et al., 2019) to fulfil the specific needs of cancer-affected people. online health information is incomplete when it is unable to provide sufficient explanation to cover all aspects of a particular issue and lacks logical links (duttabergman, 2004). even if online health resources are accurate, this does not necessarily indicate that the information is complete. for example, in a study assessing the completeness of online pancreatic cancer information, de groot et al. (2017) assigned low completeness scores for cancer websites that did not necessarily provide inaccurate or incorrect data, but that did not demonstrate the full picture by failing to include important information. therefore, completeness of the information is included in the conceptual framework as one component of the quality content to examine whether accessing sufficient information on podcasts associated with in-depth explanations of cancer topics sought by cancer-affected people is essential for using those online resources. 3.1.1.5 richness richness refers to the extent to which the resource covers a wide range of information explained in an in-depth manner (kim & mrotek, 2016). zheng et al. (2013) defined richness of information in terms of depth, diversity and volume. in the kim and mrotek (2016) study, two factors were used to assess the richness of information shared in an online health community: topic breadth and topic depth. topic breadth refers to the extent the online community provides a wide range of topics or includes several categories related to a specific health topic (kim & mrotek, 2016). topic depth refers to the extent to which the online community discusses topics in an in-depth manner or provides links redirecting health consumers to external websites that offer rich information on a given topic (kim & mrotek, 2016). therefore, richness is included in the conceptual framework for quality indicators of podcasts to examine australasian journal of information systems badreddine et al. 2023, vol 27, selected papers from acis 2022 quality of cancer-related podcasts 6 whether richness is vital for cancer-affected people seeking a wide range of cancer information and emotional support from those resources. 3.1.1.6 currency currency refers to the extent the information is up-to-date and available when needed (almutiry et al., 2013). currency is one of the core quality dimensions of health-related information (liaw et al., 2013). given the advances in cancer therapies, the progress of research on cancer screening and prevention, and information shared at a rapid pace (heymach et al., 2018), obtaining up-to-date information is very important for cancer-affected people because health-related decisions are time-sensitive and influenced by this data (huber et al., 2017). it is not clear to what extent cancer-affected people require the content shared via podcasts to be up-to-date and provide cutting-edge knowledge on cancer. therefore, this study includes currency in the conceptual model to understand how users judge the timeliness of the shared data and whether currency indicates the quality of those resources. 3.1.1.7 interpretability interpretability refers to the extent the information meaning is sufficiently explained by using terms that are easily comprehended (almutiry et al., 2013; laranjeiro et al., 2015), and whether it is communicated at an appropriate understandable level, free of jargon and complicated medical terminology (win et al., 2016). the readability of cancer websites has been extensively researched by scholars and health experts to determine the level of understandability of cancer information shared on these websites (arif & ghezzi, 2018; lynch et al., 2017). monton et al. (2019) used an online analysis tool that provides several indices to indicate the comprehension level of online cancer resources for family caregivers and found that 93% of the resources require a ninth-grade educational level to understand the content. in a different study, bates et al. (2007) found that improving the comprehension level of online health information had no impact on users’ evaluation of the quality of health information. therefore, we need to understand whether cancer-affected people perceive interpretability as an essential feature indicating the quality of cancer podcasts. 3.1.2 credibility according to the source credibility theory, the receiver’s acceptance of the message is determined to a large extent by the communicator’s characteristics and level of trustworthiness in the resource (hovland & weiss, 1951). the two major elements of the source credibility theory are trustworthiness and expertise (hovland & weiss, 1951). these two dimensions are integrated into our established framework as shown in figure 1 and explained further below. 3.1.2.1 trustworthiness trustworthiness refers to the user’s perception of the level of honesty, integrity, transparency and decency of the resource (choi, 2015; lowry et al., 2014). hooda et al. (2023) revealed that enhanced quality of technological tools, encompassing service quality, information quality, and system quality, has the potential to augment trust in online resources. previous studies have assessed the trustworthiness of online health information by asking online users to evaluate the extent to which they perceive the resource to be informative, convincing, trustworthy, truthful, transparent and decent (choi, 2015; dubowicz & schulz, 2015; mun et al., 2013). the study by khan et al. (2021) revealed that perceived trust plays a vital role within australasian journal of information systems badreddine et al. 2023, vol 27, selected papers from acis 2022 quality of cancer-related podcasts 7 technological platforms, reducing healthcare professionals’ concerns and risks associated with communicating personal and vulnerable health information. the trustworthiness of online health information is influenced by the origin of the operator (the organisation maintaining the health website); its ultimate purpose (reasons for running the website); and commercial implications (committed to profit or social benevolence) (choi, 2015). for example, participants in the study by choi (2015) reported that health information offered by non-commercial organisations was more credible than information shared by organisations driven by profit purposes. previous research showed that the level of trust in online health resources has a positive impact on behavioural, affective and cognitive health outcomes for users of the information (robertson-lang et al., 2011). this indicates that trust may constitute a quality indicator of podcasts. therefore, further investigation is required to understand whether cancer-affected people identify trustworthiness as a critical dimension for using those resources. 3.1.2.2 expertise expertise refers to the communicator’s competencies, skills, knowledge and level of performance within a specific domain (choi, 2015). prior research has shown that the expertise of the knowledge contributor is a key factor impacting users’ judgement of the believability of the shared information (jung et al., 2016; ma & atkin, 2017). in a study identifying the quality indicators of health-related podcasts, experts believed that transparency is a key dimension of the source credibility and includes items such as listing communicators’ qualifications and affiliations, providing details of the authors who created the resource, mentioning the origin of information by citing references, and referring listeners to additional resources (thoma et al., 2015). this study aims to understand cancer-affected people’s perceptions of the importance of the source expertise and how this influences their decisions to continue using podcasts. hence, expertise is included in the conceptual framework for quality indicators of those resources. 3.1.3 design zhang et al. (2000) developed the two-factor model for website design to identify the design factors that enhance the usability and serviceability of the website and increase users’ satisfaction and motivation to revisit the website. the two-factor model for website design identified several categories and core features classified as either hygiene or motivational (zhang & von dran, 2000). the presence of hygiene factors encourages users to stay with the website and the absence of those features may lead to users’ dissatisfaction and potential abandonment of the website (zhang et al., 2000). motivational factors, on the other hand, make the website look more appealing and increase users’ motivation to visit the website, yet the absence of those motivational features would not discourage users to access and revisit the website (zhang & von dran, 2000). this study included navigation, accessibility, and aesthetics as proposed dimensions of the design factor of cancer-related podcasts and the webpages displaying them. 3.1.3.1 navigation navigation refers to the extent to which an online resource provides tools for users to facilitate the location of information (brower, 2004; zheng et al., 2013). the navigational structure of an online resource includes the links and hyperlinks the resource offers, navigational instructions australasian journal of information systems badreddine et al. 2023, vol 27, selected papers from acis 2022 quality of cancer-related podcasts 8 and menu, site maps, help functionality, search engine and information organisation (hajli et al., 2015; thoma et al., 2015; whitten et al., 2008; zhang et al., 2015; zhang, 2010). to improve the functionality of navigation tools, such as the search engine, islam and rahman (2017) suggested tracking users’ most browsed activities and upgrading search engines by enabling the most searched items and favoured topics to be viewed first. therefore, navigation is examined as one dimension of the design factor in the conceptual framework for quality indicators of cancer podcasts. 3.1.3.2 accessibility accessibility refers to the extent to which the information can be easily accessed without exerting relatively high effort (zheng et al., 2013). zheng et al. (2013) assessed accessibility by examining whether elements of an online resource (graphics, texts, etc.) can be quickly downloaded and visible. in a study on quality indicators of health educational podcasts, accessibility was measured by the degree to which the podcast loads and responds quickly, does not undergo significant downtimes by being available most of the time, can be accessed using multiple devices, is compatible with multiple operating systems and browsers, and is well-maintained to support the functionality of multimedia components and hyperlinks (thoma et al., 2015). therefore, integrating cancer-affected people’s experiences and opinions about the accessibility of online resources is crucial to understand whether accessibility indicates the quality of podcasts. this is because the viewpoint of scholars and experts can be different from the viewpoint of cancer-affected people, who are the direct users of those resources. accordingly, accessibility is included as one dimension of the design factor in the conceptual framework for quality indicators of podcasts. 3.1.3.3 aesthetics aesthetics refers to the extent to which the online resource feels and looks good by having an appealing and easy-to-follow layout and appropriate and relevant images (zhang et al., 2015). different criteria have been used in studies assessing the aesthetic state of online health resources. for example, koo et al. (2011) assessed the aesthetic state of an online cancer information site by measuring the attractiveness of the screen layouts, use of colours, screen background and patterns, and multimedia content to understand the impact on users’ satisfaction and perceived usefulness. in another study, kim and mrotek (2016) used the modified version of the aesthetic scale that includes four dimensions: craftmanship, colourfulness, diversity and simplicity (moshagen & thielsch, 2013) to assess the aesthetic state of an online health community. cancer websites with poor aesthetic states are dissatisfying to cancer-affected people. for example, small font sizes of an online cancer resource added difficulty for reading the text by older cancer patients, who reported several shortcomings in the layout of the cancer website (bolle et al., 2016). accordingly, aesthetics is included as one dimension of the design factor in the conceptual framework for quality indicators of podcasts. all dimensions mentioned above for content, credibility, and design factors are illustrated in figure 1. australasian journal of information systems badreddine et al. 2023, vol 27, selected papers from acis 2022 quality of cancer-related podcasts 9 figure 1. quality dimensions of cancer-related podcasts 3.2 audience reception although assessing information quality by focusing on the information and information systems can be beneficial, it cannot provide a comprehensive perspective without considering audience perceptions (salaün & flores, 2001), in this study, users. considering audience or users’ perceptions is particularly important in the context of this study as cancer-affected people are more vulnerable, with variable information needs, and exposed to various types of risks and misinformation (badreddine et al., 2023). the inclusion of reception theory provides a more nuanced understanding of the perception of cancer-affected people using podcasts. reception theory is a critical approach in literary studies and sociology that focuses on the audience’s experience and interpretation of a text or piece of media (hall, 2007). rather than emphasising the author’s intent or the text’s inherent meaning, reception theory centres on the reader’s role in constructing meaning. it acknowledges that audience bring their unique backgrounds, experiences, and perspectives to a content, and these elements shape their interpretation. by examining how different audiences may respond at different times and in various cultural contexts, reception theory offers a dynamic understanding of how meaning is created and negotiated (willis, 2021). over time, the application of the theory has been extended to other areas and disciplines including information systems (ma, 2002). in the context of this study, cancer-affected users perceive the quality of podcasts from different perspectives depending on their specific settings. reception theory focuses on the processes of transferring meaning from creator to audience of creative work. this process includes encoding, where the sender crafts a message with shared rules and symbols, and decoding, where the receiver must understand the message as it was intended by the encoder (hall, 2007). furthermore, by considering their demographics and cultural background, the theory identifies three categories of audiences: i. dominant (who interprets a text or media work in the way that the creator intended); ii. negotiated (who accepts the creator message despite being against their personal convictions); and iii. australasian journal of information systems badreddine et al. 2023, vol 27, selected papers from acis 2022 quality of cancer-related podcasts 10 oppositional (who interprets a content in a way that is fundamentally in conflict with the creator’s intended meaning or the dominant cultural values). previous studies used the reception theory to examine podcasts as a form of creative work and investigated various types of reception in survivors of domestic violence (boling, 2022) and students (kitamura, 2021). within the domain of health communication, the literature has also placed emphasis on the examination of perceptions held by diverse vulnerable populations, exploring the multifaceted perspectives and experiences of these individuals (chen, 2020; rennick-egglestone et al., 2019). reception theory is a good fit for this study because it provides a more nuanced understanding of perceptions of the three quality factors of cancerrelated podcasts from cancer-affected people who need access to resources and support depending on their individual context, for example, where they are on their cancer journey. consequently, the integration of the three theories of information quality with reception theory to examine how diverse audiences perceive various dimensions of information quality constitutes the conceptual framework that guides this study. figure 2 illustrates this integrated model. figure 2. conceptual framework on podcast quality 4 methodology this study adopted the qualitative research design by using the narrative approach to extrapolate raw data from spoken stories of users’ experiences as a source of empirical knowledge (butina, 2015). the narrative approach includes diverse viewpoints from a varied range of participants that may result in different interpretations and new themes when analysing the narrative context (butina, 2015). the qualitative methodology was suitable to allow the researcher to acquire a depth of understanding of cancer-affected people’s subjective experiences of using podcasts. australasian journal of information systems badreddine et al. 2023, vol 27, selected papers from acis 2022 quality of cancer-related podcasts 11 semi-structured interviews were conducted with 17 women affected by breast cancer who are users of the podcast series by breast cancer network australia (bcna)1 . participants were recruited via an advertisement published on the bcna website. cancer-affected people were eligible to participate if they understood english, were at least 18 years of age, and had used or are still using the podcast series. the interview questions were adapted from previous studies that examined the quality of online health information. the interviews ranged between 30 and 45 minutes, were conducted via zoom or phone, and were digitally recorded with the interviewee’s permission. the demographic data of the study participants are shown in table 1 below. demographic variable n % demographic variable n % gender age female 17 100% 35 – 44 years old 3 17.5% cancer status 45 – 54 years old 3 17.5% cancer patient 9 53% 55 – 64 years old 9 53% post treatment 7 41% 64 years old or above 2 12% carer 1 6% length of usage time diagnosis < 1 year 8 47% dcis 5 29% 1 2 years 5 29% early breast cancer 10 59% > 2 years 4 24% metastatic breast cancer 2 12% table 1. demographic data of the study participants collecting data from 17 participants was sufficient to reach data saturation when variation in the collected data levelled off and no new interpretations and ideas were added (francis et al., 2010). the interview material was transcribed and loaded into nvivo12 for additional processing and analysis. thematic analysis was used to analyse the interview data, comprehend all parts of the interview texts, and describe themes and patterns in this study (pope & mays, 2019). after analysing the interview transcripts, the researchers were able to reconstruct and reduce the data by filtering out unnecessary conversations and selecting the perceptions that may corroborate key interpretations and findings (miles et al., 2014). the data set was then reduced to meanings to generate abstract findings and extract themes (pope & mays, 2019). the data was first analysed using the iq assessment framework, source credibility model, and two-factor theory for website design. secondly, the data was analysed using reception theory for the thematic analysis and categorisation of interviewees. 1 breast cancer network australia is a not-for-profit organisation that supports australians affected by breast cancer. bcna aims to ensure that australians affected by breast cancer receive support, information, treatment and care appropriate to their needs. with the opportunities that digital innovation is creating, bcna has launched podcast series named “upfront about breast cancer” including several episodes featuring different cancer topics to help cancer-affected people during treatment and beyond. listeners can also refer to transcripts of podcasts if they are having trouble understanding a topic or if they want to refer to a piece of information later by scanning the transcript rather than listening to the audio. each episode is labelled with a title that describes the discussed topic and lists all the related resources that offer the appropriate support. australasian journal of information systems badreddine et al. 2023, vol 27, selected papers from acis 2022 quality of cancer-related podcasts 12 5 findings the analysis of qualitative data offers a rich and nuanced understanding of cancer-affected people’s perceptions on various dimensions of podcast quality. also, through in-depth analysis of transcribed data, we were able to uncover new themes and patterns that shed light on the complexities of the quality of podcasts. each interview was analysed to classify interviewees into three categories of audience. dominant listeners are those individuals comprising a substantial portion of the study participants who consistently conveyed strong positivity regarding the quality aspects of podcasts and satisfaction with their usage. negotiated listeners are those individuals who held a somewhat favourable view of podcast quality but suggested that their reuse would be contingent upon improvements in specific dimensions. in contrast, oppositional listeners held a negative view of most of the podcasts’ quality dimensions. they either harboured mistrust or found the content and design to be subpar. this section presents the main findings of the study, organised according to the key themes under investigation. accuracy: participants reported that obtaining flawless and reliable information is an important factor when deciding to use podcasts. participants agreed that accurate data is the key to effective decision-making, and that inaccurate data places patients at risk of obtaining inadequate care and worsening their health. users tended to compare what they knew about the medical condition and what their doctors informed them with information available through podcasts to confirm the accuracy of information. users who believed that the shared data was inaccurate intended to stop listening to the podcasts. for example, a cancer survivor mentioned that: there are a lot of grey areas. i felt that it wasn't validated from my experience... i think it was biased data and not a true picture, as the speaker put the experience in a box with no room for other people's experiences. i would not refer any of my friends to listen to this series of podcasts. relevancy: users felt well-informed and more confident discussing knowledge obtained through podcasts with their doctors, particularly when the information was relevant and directly related to their situations. participants preferred listening to podcasts that gave them a sense of their own treatments and diagnosis. those users are more willing to listen to podcasts that share information tailored to their symptoms and particular conditions. participants believed that off-topic and irrelevant podcast messages were less useful in making health decisions and would discourage them from listening to those podcasts. to illustrate this point, a cancer patient said: that's very important to me because i'm triple negative so a lot of information about breast cancer is about hormone receptors and postmenopausal women, and circumstances that don't match with my diagnosis. so, when i find stuff that is relevant, i absolutely love it and delve into it. usefulness: participants stated that they are more inclined to use podcasts that can help them expand their knowledge and make better health-related decisions. users thought that using the podcast to validate information obtained from medical specialists made its usefulness more apparent. users are more likely to use podcasts offering valuable advice, stress-relieving remedies, and dietary guidance. participants responded that if the shared knowledge via the podcasts becomes less useful and does not provide appropriate practical tips, they will less likely listen to the podcasts. for example, a cancer patient noted that: australasian journal of information systems badreddine et al. 2023, vol 27, selected papers from acis 2022 quality of cancer-related podcasts 13 … certainly, the last series of podcasts with charlotte tottman was useful, perhaps because she was a psychologist, i found that particularly helpful… and even for some of the strategies she mentioned, i would use going forward…. if the info was not helpful, i just wouldn’t bother to listen. completeness: participants believe a podcast is incomplete if it does not provide sufficient explanation to cover all aspects of the discussed topic. participants stated that even if the podcast shared factual data, it might not be complete since it didn't show the entire picture by excluding key information. one listener regarded the podcast's material as "incomplete" since it concentrates on the speaker's unpleasant experiences while ignoring the positive aspects. this participant believed that the shared material is harmful to women with breast cancer since it does not depict the entire picture which should include both positive and negative experiences. another cancer patient said: … that's why i kept listening to it and that's because it gave me so much. it just helped me to know the many aspects of my cancer, you know everything from treatment and dealing with treatment and the side effects of treatment... and everything. richness: some participants reported that detailed explanations of the podcast topics were beneficial and valuable, especially since discussions with doctors focused on a narrow range of cancer-related topics as opposed to a broad range. another participant, on the other hand, complained that the podcast series lacked diversity because they all shared similar themes. as a result, they were less interested in the quality of podcasts that covered narrow subjects. to illustrate this point, a cancer patient stated: when i was first diagnosed, i was greedy for information. i prefer a wide range of cancer topics … every time i've been worried about something like my nails started falling off, i'd be able to go to the one that talks about side effects from radiation or chemo and be able to get that bit to reassure me. if there was a link under the podcast that provides additional data on this, that’ll be fantastic. currency: the study participants felt that getting the most recent information is crucial for cancer-affected people because cancer therapies are improving, research on cancer screening and prevention is advancing, and information is being shared at a rapid rate. they held the opinion that having access to current information can facilitate making timely health-related decisions that are consistent with the most recent developments in cancer care and treatment. for example, one cancer survivor said: i just don't think it's worthwhile listening to outdated information if the studies they’re citing are really old because research is moving so quickly and there are so many new things... when my mom had breast cancer 30 years ago she was told to go home and rest after treatment, whereas now they'd say try and keep moving as much as you can and so if you're told to go home and rest when the current research is to keep moving, it's kind of worthless. interpretability: participants expressed a preference for podcasts that succinctly deliver the subject issues and share information that can be comprehended with less effort. participants thought that individuals without a background in medicine would benefit from having a glossary of terms. according to one participant, it is challenging to have authoritative information without complicated medical terms and jargon because they form an essential component of it, but the information must be clearly communicated using simple terms. participants claimed that they would quit listening to podcasts that are difficult to understand. to illustrate this viewpoint, a cancer survivor said: australasian journal of information systems badreddine et al. 2023, vol 27, selected papers from acis 2022 quality of cancer-related podcasts 14 i just would not listen to it if it's totally incomprehensible to me. i don't mind people using medical terms, because if you are in a medical situation, you probably should know them, but they need to be explained …. when it's an early diagnosis thing, where you haven't heard these terms or you have heard them, but you can't remember what they mean, a glossary of terms would be very useful. because there's so much to take in at the beginning and you're tired of usually you got stuck on the fact that you got cancer and can't remember all the other stuff ... within the content dimension and focusing on reception theory, the data was mapped to the various types of audiences highlighted in this theory, that showed: dominant or hegemonic listeners: those individuals formed a significant portion of those who were interviewed (11 out of 17). they generally indicated a full acceptance of the podcasts’ content as they believed in the accuracy of whatever coming from a reputable source: anything that bcna does is usually very good very well researched, and they also usually use really prominent people in the fields. dominant listeners believed that the content of the podcast enhanced their ability to cope with cancer and gain more knowledge: i guess it made me feel empowered that i knew so many little things now i have all of these tools that i can use to make myself feel better. negotiated listeners: four interviewees provided less favourable assessments for specific dimensions of the content quality. while they agreed with certain dimensions, they also expressed reservations or recommended adjustments. in particular, they identified situations where the podcast's content omitted crucial information and occasionally lacked clarity in its interpretation. it doesn't tell you about the good stuff and how you can change your life and how great it is to change your life to be more healthy. it's certainly had a similar theme every episode and i’d probably like to see more maybe diversity. the results of our analysis revealed that three individuals within this category come from a medical background and possess extensive experience in dealing with cancer. this background and experience influenced their evaluation of the content quality of these podcasts: i’ve got a science background, i like to know the science behind things so during my treatment, i wanted to know exactly what was going on and so that's vital. i wouldn't be dialling in if it was a winch fist of people who didn't know what they're talking about. oppositional or counter-hegemonic listeners: two interviewees held a negative view of the podcast content and expressed reservations about most dimensions of the content quality. the predominant reason for this oppositional perspective stems from their emotive reaction to the auditory material they were exposed to. these users indicated their intention to avoid podcast episodes that exhibited bias and conveyed information perceived as less accurate and less useful. for example, a listener criticised the reliance of the host on her own experience by saying: i took offense to that her experience was a blanket experience and everybody is dying to experience what i did; and it is going to be hard, and i think that is a myth; it doesn't have to be a difficult hard experience! australasian journal of information systems badreddine et al. 2023, vol 27, selected papers from acis 2022 quality of cancer-related podcasts 15 trustworthiness: many participants stated that they would generally have faith in the information supplied via podcasts if those resources were run by reputable institutions including charities and the government. participants said they trusted podcasts that offered objective medical advice, directed listeners to other credible sources, included information on the data originator, and disclosed any conflicts of interest. for example, one cancer patient said: i think i trust it because it is coming from bcna which is a reputable source of information. it's probably the premier website that breast cancer patients in australia would use. so in fact, there's probably quite a lot of faith and expectation there that their information is accurate and trustworthy and that you're not going to be led astray. expertise: participants noted that the calibre of the podcast host has an impact on how much users trust the information presented, especially when it comes from licenced medical professionals like doctors. according to users, it is crucial to introduce the podcast guests and outline their credentials at the beginning to demonstrate their suitability to discuss the subject at hand. when the communicator is subjective, listeners have expressed intentions to stop using the podcast. for instance, one cancer survivor said that because podcasts were constrained by the values of the physician or the person delivering the message, they are unlikely to be listened to in the future. another cancer patient said: well, i mean when i’m listening to someone, i want them to either have experience of breast cancer themselves or have medical knowledge about cancer. because that's the type of information that i would trust. i think that's why specific podcasts really appealed to me because the speaker had that level of experience herself, but she also had an original angle. by considering the three audience categories, an analysis of the podcast users' perceptions of the credibility dimensions was conducted, revealing the following results: dominant or hegemonic listeners: 12 out of 17 listeners mentioned that they place trust in the podcasts’ content. they believe that reputable sources, such as bcna, diligently seek the opinions and advice of experts when selecting speakers to convey messages through their podcasts: i’ve got full faith that bcna are kind of doing that due diligence prior to getting their speakers involved. negotiated listeners: three interviewees held neutral views regarding the podcast's credibility dimensions. for example, one individual preferred to apply their own validation of data before trusting it blindly even if it were generated by a reputable source: i guess it's more about the multiple levels of checks and balances which are useful, rather than one source of information. oppositional or counter-hegemonic listeners: two interviewees lacked trust in the podcasts’ content and perceived the speakers as biased, primarily focusing on their own experiences. this tendency sometimes made them feel that the content is unsettling and less reassuring. for instance, one listener critiqued the host’s attitude, stating: well, one of them, that really made me angry was the one on and i can't remember it was on sexuality and intimacy. i think that was cold and it was the speaker's experience and she quoted statistics to say that after treatment and maybe during while you're having hormone blockers … you don't orgasm anymore. australasian journal of information systems badreddine et al. 2023, vol 27, selected papers from acis 2022 quality of cancer-related podcasts 16 i suppose the podcast scared me. i got really scared and i didn't want to believe that this is true, so i got a naturopath and i started taking b12 and it helped yeah. navigation: participants believed that the podcast webpage is well-designed if it has a navigation system that works consistently across the entire website and offers tools for finding the content quickly. for instance, one participant mentioned that the authority and reliability of the podcast can be determined by how well-designed the navigation system is. participants advised that a title that clearly describes the podcast’s subject is very helpful. users acknowledged losing interest if the links and hyperlinks of the podcasts were not adequately maintained and expressed frustration with less powerful search engines. to illustrate this point, a cancer survivor said: if it was too hard to navigate the website to find them, i would just give up, especially if it took longer than a minute, then i wouldn't be actually keen. everyone wants things to be easy, particularly when you are first diagnosed and you're looking for information. you know there's a lot of stuff going on as well, you just want it to be easy enough. accessibility: podcasts that load and respond quickly were favoured by participants. they expressed dissatisfaction with podcasts that undergo lengthy outages and are not constantly available. they claimed that listening to podcasts using various devices is convenient, especially since they can do so from any place. participants favoured podcasts that worked with a variety of operating systems and browsers and were well-maintained to support the functionality of multimedia elements and hyperlinks. to illustrate this point, a cancer patient noted that: basically, accessing them from various devices is very important so you can listen to them anywhere on your mobile phone or tablet… if you are very eager and keen to listen to a certain podcast and then it just went down, yeah, i'd be pretty annoyed. aesthetic: users thought that a visually appealing website is one that contains convenient pictures, fonts and colours that complement the concept of the website and link better to the theme. although the inclusion of those features would boost users' satisfaction with the website, users said that the aesthetic appeal of the website has no bearing on their choices to continue using the podcasts. the participants found the podcast transcripts to be helpful and preferred the audio speech to be at a reasonable pace and speed. one participant, for instance, described how they gave up and stopped listening to podcasts issued by another charity because the audio was delivered by a very rapid speaker who was attempting to convey a complicated topic using complex medical terms. additionally, some participants mentioned how crucial it is that the voice can be clearly heard without exerting too much effort to continue listening. to illustrate this point, a cancer patient stated: the pace and speed of speech are very important. i think if you feel rushed through it that doesn't really work. i tend to talk quickly when i’m nervous and i think a lot of people do and it's irritating as a listener, to listen to somebody who's talking quickly because you keep missing stuff. whereas people that are experienced podcasters sort of measured and slower than normal talking and it just seems to work a lot better. also, a clear voice recording is critical to my understanding of it, to be able to make sure that you are following along with the information that comes through. the perceptions of the three categories of audience regarding the design dimensions of the podcasts were as follows: australasian journal of information systems badreddine et al. 2023, vol 27, selected papers from acis 2022 quality of cancer-related podcasts 17 dominant or hegemonic listeners: 12 out of 17 individuals were pleased with the podcasts’ design, finding the podcast website easy to navigate and user-friendly. they held a neutral stance regarding the visual aesthetics of the website: everyone wants things to be easy and, particularly when you are first diagnosed and you're looking for information. you know there's a lot of stuff going on as well, you just want it to be easy enough that's, the only thing that's easy in your day it just makes a difference. i wouldn't say that the colours and images are as important as the audio quality. negotiated listeners: two interviewees expressed less satisfaction with the podcast's navigation system and accessibility. for instance, those individuals expressed a preference for more detailed descriptions of the podcast topics and found the website somewhat challenging as they encountered difficulties in locating specific podcast. they noted that the visual aesthetics of the podcast website held minimal significance and did not influence their intentions to continue using it. for example, they mentioned: i did find it a little bit hard so when i’m looking for a specific topic, you know look up peeling nails or something, and you get you know 5 million hits and so i tend to sort of stop using those resources. i found it very hard to find it as a resource to begin with, so it was not made kind of find out that bcna even existed. oppositional or counter-hegemonic listeners: similar to the opinion of the other two categories of audience, three oppositional listeners did not place significant importance on the visual appearance of the podcast website. this aligns with the finding that the aesthetic aspect in terms of the colour and theme of the podcast website serves as a motivator, as its absence does not deter users from listening to the podcasts. nevertheless, within this group, two individuals were reluctant to access podcasts that required a substantial amount of time to navigate or featured an unacceptable pace of audio speech: … fairly important because i’ve listened to a few where they're really slow, so i have to speed them up because it's so annoying to listen to and have lots of big pauses in the podcast. it's not actually important to make the aesthetics with the podcast when you're listening to it. 6 discussion the study discovered that the podcast information’s accuracy is a key factor within the domain of the content factor. the results demonstrated that cancer-affected people are less likely to listen to podcasts that share misleading information and include erroneous and contradictory beliefs. cancer-affected people generally verify experiential information with their doctors more frequently than the medical information shared by medical practitioners via podcasts. therefore, the accuracy aspect that constitutes one parameter of the intrinsic iq of the iq assessment framework (wang & strong, 1996) is an essential quality element of cancer-related podcasts. the accuracy of content is notably emphasised when it comes to oppositional listeners, who are more inclined to reject podcasts entirely if the content is less precise or conveys misleading and conflicting information. users felt more knowledgeable and confident in sharing information acquired through podcasts with their doctors, particularly when the content was relevant and directly linked to their experience. according to the users, applying the information shared via the podcasts australasian journal of information systems badreddine et al. 2023, vol 27, selected papers from acis 2022 quality of cancer-related podcasts 18 when making health decisions depends on the relevance of the podcast content. cancer-affected people emphasised the importance of podcasts that provide insightful advice and practical guidance, indicating that the podcast’s usefulness is another important aspect under the content factor. the podcast listeners also sought comprehensive podcasts incorporating all the current information on the cancer topic under discussion. this focus on comprehensiveness was especially pronounced among negotiated listeners, who displayed an increased concern with podcasts that omitted vital information. the diversity, volume, and depth of the shared information were attributes cancer-affected people perceived as essential for listening to the podcasts. those attributes align with zheng et al. (2013) classification of the richness dimension. having access to current information can help podcast listeners make prompt medical decisions in line with the most recent developments in cancer treatment and care. this is important especially that finding the latest information about cancer can be challenging because cancer information disseminated over the internet may not be regularly updated (doubleday et al., 2020). questioning the quality of the podcast content has elicited emotional responses from oppositional listeners, many of whom have voiced strong opinions about discontinuing their engagement with the content because of perceiving a lack of relevancy, usefulness, completeness, richness, and currency. users who believed that sophisticated medical terminology affected the podcasts' interpretability were more likely to seek podcasts that provide content at an adequate understanding level and utilise simple language. the findings showed that podcasts that included transcripts, examples and scenarios, and a glossary of terms made it easier for listeners to comprehend the shared content. the comprehensiveness of the data is paramount, especially considering that negotiated listeners with medical backgrounds have highlighted the challenges they encounter when dealing with data that is less interpretable or lacks a complete context. while such data may be more easily grasped by negotiated listeners who share a related background or possess extensive experience with cancer, it may be interpreted differently by other listeners. therefore, interpretability within the parameters of the representational iq of the iq assessment framework is an essential quality component of cancer-related podcasts. podcasts are more credible if they are managed by a well-reputed npco and are clear on who was involved in creating the podcast content. dominant listeners often regard this as a primary factor influencing their trust in and sustained engagement with the podcast content. users exhibit a greater inclination to listen to podcasts when they are hosted by well-qualified authors who also disclose their affiliations. therefore, trustworthiness and expertise underpinning the credibility factor were crucial dimensions of podcasts, indicating that the source credibility model fits well within the podcast quality context. an important theme that emerged related to users’ concern about the level of the speaker’s objectivity. users reported that some podcasts concentrated on the speaker's perspective and made little effort to integrate the experiences of other cancer-affected people. though users were interested in listening to personalised experiences, the process of inviting comments from others who may wish to endorse or refute the shared content can help address this issue and increase the objectivity of the shared content (thoma et al., 2015). negotiated listeners raised doubts about the credibility of certain transmitted messages and leaned towards applying their own validation criteria before accepting any advice. conversely, oppositional listeners australasian journal of information systems badreddine et al. 2023, vol 27, selected papers from acis 2022 quality of cancer-related podcasts 19 rejected trust in the information, deeming it biased and subjective due to its emphasis on personal experiences, which they found controversial when compared to knowledge grounded in science and collective experience. citing sources, referring users to additional good quality resources, and listing any conflicts of interest tend to increase health-related podcasts’ credibility (thoma et al., 2015). therefore, the objectivity of the speaker was an emerging theme cancer-affected people viewed as important for using podcasts. this finding is consistent with wang and strong’s (1996) study that identified objectivity as one of the metrics of the intrinsic iq of the iq assessment framework and defined it as the extent to which the content was impartial, unprejudiced, and unbiased. a navigation system featuring clear descriptions of links, logical subject labels, and a powerful search engine is crucial for users when navigating podcast websites. users are less likely to spend time and effort searching the website to find a particular podcast, and therefore, navigation is one of the hygiene factors within the design factor’s domain. the process of accessing podcasts using different devices and operating systems was another hygiene factor such that the absence of this functionality would discourage users from listening to cancerrelated podcasts. furthermore, within the accessibility feature, users were less inclined to access slow-loading and unresponsive podcasts. figure 3. conceptual framework on cancer-affected people’s perceptions of the dimensions of information quality in cancer-related podcasts except for one feature identified by all the three categories of users as a motivating factor within the aesthetics dimension that related to the colours, visuals, and themes used for the podcast series website, all other features were hygiene factors. those features included the pace and speed of the podcast speech and the clarity of the voice recording. in conclusion, australasian journal of information systems badreddine et al. 2023, vol 27, selected papers from acis 2022 quality of cancer-related podcasts 20 using the two-factor theory of website design (zhang & von dran, 2000) to establish quality indicators of podcast design helped researchers better understand how podcast listeners feel about using cancer-related podcasts for support and evaluating the design of those podcasts. figure 3 above illustrates the integration of findings from the two different perspectives. 7 conclusion this study delved into various facets of the quality of cancer-related podcasts and their influence on how users perceive and engage with them. the research findings showed that accuracy and comprehensiveness in podcast content, particularly for users navigating the challenging landscape of cancer-related information, are of utmost importance. canceraffected individuals prioritise podcasts that offer up-to-date guidance, practical advice, and rich contextual information. furthermore, the study showed that interpretability is crucial, as listeners who encounter difficulties with complex medical terminology value podcasts that provide content in understandable language and support it with transcripts, examples, scenarios, and glossaries. this consideration is vital, as it ensures that a broader audience can benefit from the content, not just those with a medical background or extensive experience in dealing with cancer. the study revealed that all dimensions of the credibility factor are critical. the source and expertise of podcast content creators significantly influence listeners' trust in the information presented. objectivity is an emerging aspect of podcast quality, as users expressed concern about the level of speaker objectivity and the need to integrate diverse perspectives. this theme suggests that involving comments from others who may endorse or challenge the content can enhance objectivity and increase the credibility of shared information. within the realm of design, clear navigation systems, speedy loading times, responsive design, and cross-device compatibility are essential for user-friendly podcast experiences. hygiene factors such as efficient website navigation and accessibility across various devices and operating systems play a pivotal role in user engagement. additionally, factors related to aesthetics, like colours, visuals, and themes, although important, are more motivating factors rather than hygiene factors in the overall podcast design. this study provided valuable insights into how podcast quality influences user perceptions and engagement. the integration of these diverse dimensions and perspectives illustrates the multifaceted nature of podcast quality and its impact on users seeking support and information related to cancer. this study contributes to a deeper understanding of podcast quality and offers practical guidance for creators and providers in optimising the design and content of cancer-related podcasts. nonetheless, this study is not without limitations. first, this study was based in australia and used data from only one npco that provides cancer-related podcasts. further studies could recruit respondents experiencing cancer-related podcasts offered by other organisations in different countries to verify the quality factors found in this study. second, this study assumes that the demands of everyone affected by cancer are similar; thus, it does not differentiate between the views of each patient, survivor, and caregiver when evaluating the quality elements. therefore, future studies need to investigate differences in perspectives when examining quality indicators of podcasts. lastly, even though understanding quality indicators of podcasts can help npcos effectively design and manage cancer-related podcasts, australasian journal of information systems badreddine et al. 2023, vol 27, selected papers from acis 2022 quality of cancer-related podcasts 21 the psychosocial outcomes, and the impact of using cancer-related podcasts on users’ wellbeing are still under-researched. therefore, further research into the impact of cancer-related podcasts on the well-being of cancer-affected people is also needed. references almutiry, o., wills, g., alwabel, a., crowder, r., & waiters, r. (2013). toward a framework for data quality in cloud-based health information system. international conference on information society (i-society 2013). ieee. https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6636362 arif, n., & ghezzi, p. (2018). quality of online information on breast cancer treatment options. the breast, 37, 6-12. https://doi.org/10.1016/j.breast.2017.10.004 back, d. a., von malotky, j., sostmann, k., hube, r., peters, h., & hoff, e. (2017). superior gain in knowledge by podcasts versus text-based learning in teaching orthopedics: a randomized controlled trial. journal of surgical education, 74(1), 154-160. https://doi.org/10.1016/j.jsurg.2016.07.008 badreddine, b., blount, y., & amrollahi, a. (2023). quality indicators of online cancer communities from the perspective of cancer-affected people. proceedings of the 56th hawaii international conference on system sciences. maui, hi, usa, january 3-6, 2023. https://hdl.handle.net/10125/103042 bates, b. r., romina, s. m., & ahmed, r. (2007). the effect of improved readability scores on consumers’ perceptions of the quality of health information on the internet. journal of cancer education, 22(1), 15. https://doi.org/10.1007/bf03174369 berk, j., trivedi, s. p., watto, m., williams, p., & centor, r. (2020). medical education podcasts: where we are and questions unanswered. journal of general internal medicine, 1-3. https://doi.org/10.1007/s11606-019-05606-2 bernstam, e. v., walji, m. f., sagaram, s., sagaram, d., johnson, c. w., & meric-bernstam, f. (2008). commonly cited website quality criteria are not effective at identifying inaccurate online information about breast cancer. cancer: interdisciplinary international journal of the american cancer society, 112(6), 1206-1213. https://doi.org/10.1002/cncr.23308 boling, k. s. (2022). “it’s that ‘there but for the grace of god go i’piece of it”: domestic violence survivors in true crime podcast audiences. mass communication and society, 26(6), 1-23. https://www.tandfonline.com/doi/full/10.1080/15205436.2022.2061359 bolle, s., romijn, g., smets, e. m., loos, e. f., kunneman, m., & van weert, j. c. (2016). older cancer patients’ user experiences with web-based health information tools: a think-aloud study. journal of medical internet research, 18(7), e208. https://doi.org/10.2196/jmir.5618 boon-itt, s. (2019). quality of health websites and their influence on perceived usefulness, trust and intention to use: an analysis from thailand. journal of innovation and entrepreneurship, 8(1), 4. https://doi.org/10.1186/s13731-018-0100-9 brower, s. m. (2004). academic health sciences library website navigation: an analysis of fortyone websites and their navigation tools. journal of the medical library association, 92(4), 412. https://www.ncbi.nlm.nih.gov/pmc/articles/pmc521512/ https://hdl.handle.net/10125/103042 australasian journal of information systems badreddine et al. 2023, vol 27, selected papers from acis 2022 quality of cancer-related podcasts 22 bruce, j. g., tucholka, j. l., steffens, n. m., & neuman, h. b. (2015). quality of online information to support patient decision-making in breast cancer surgery. journal of surgical oncology, 112(6), 575-580. https://doi.org/10.1002/jso.24046 butina, m. (2015). a narrative approach to qualitative inquiry. clinical laboratory science, 28(3), 190-196. https://doi.org/10.29074/ascls.28.3.190 chang, k., berthelet, e., grubbs, e., hamilton, s., karvat, a., tran, e., . . . ingledew, p.-a. (2019). websites, websites everywhere: how thyroid cancer patients use the internet. journal of cancer education, 35, 1-7. https://doi.org/10.1007/s13187-019-01576-5 chen, c.-m. (2020). public health messages about covid-19 prevention in multilingual taiwan. multilingua, 39(5), 597-606. choi, w. (2015). a new framework of web credibility assessment and an exploratory study of older adults' information behavior on the web. doctoral thesis, florida state university, fl, usa. https://diginole.lib.fsu.edu/islandora/object/fsu%3a253213 de groot, l., harris, i., regehr, g., tekian, a., & ingledew, p.-a. (2017). quality of online resources for pancreatic cancer patients. journal of cancer education, 34(2), 223-228. https://doi.org/10.1007/s13187-017-1290-8 doubleday, a. r., novin, s., long, k. l., schneider, d. f., sippel, r. s., & pitt, s. c. (2020). online information for treatment for low-risk thyroid cancer: assessment of timeliness, content, quality, and readability. journal of cancer education, 36, 1-8. https://doi.org/10.1007/s13187-020-01713-5 dubowicz, a., & schulz, p. j. (2015). medical information on the internet: a tool for measuring consumer perception of quality aspects. interactive journal of medical research, 4(1), e8. https://doi.org/10.2196/ijmr.3144 dutta-bergman, m. j. (2004). the impact of completeness and web use motivation on the credibility of e-health information. journal of communication, 54(2), 253-269. https://doi.org/10.1111/j.1460-2466.2004.tb02627.x fadahunsi, k. p., akinlua, j. t., o’connor, s., wark, p. a., gallagher, j., carroll, c., . . . o’donoghue, j. (2019). protocol for a systematic review and qualitative synthesis of information quality frameworks in ehealth. bmj open, 9(3), e024722. https://doi.org/10.1136/bmjopen-2018-024722 faller, h., brähler, e., härter, m., keller, m., schulz, h., wegscheider, k., . . . richard, m. (2017). unmet needs for information and psychosocial support in relation to quality of life and emotional distress: a comparison between gynecological and breast cancer patients. patient education and counseling, 100(10), 1934-1942. https://doi.org/10.1016/j.pec.2017.05.031 francis, j. j., johnston, m., robertson, c., glidewell, l., entwistle, v., eccles, m. p., & grimshaw, j. m. (2010). what is an adequate sample size? operationalising data saturation for theory-based interview studies. psychology and health, 25(10), 1229-1245. https://doi.org/10.1080/08870440903194015 gandamihardja, t. a., liyanage, s., coutee, t., peled, a. w., & masannat, y. a. (2023). the role of social media and breast cancer: how does it impact patients? breast care, 18(3), 203-208. https://doi.org/10.1159/000530433 australasian journal of information systems badreddine et al. 2023, vol 27, selected papers from acis 2022 quality of cancer-related podcasts 23 gopal, d. p. (2019). why i prescribe podcasts to my patients. british journal of general practice, 69(689), 621-621. https://doi.org/10.3399/bjgp19x707009 grock, a., joshi, n., swaminathan, a., rezaie, s., gaafary, c., & lin, m. (2016). blog and podcast watch: neurologic emergencies. western journal of emergency medicine, 17(6), 726. https://doi.org/10.5811/westjem.2016.9.31010 hajli, m. n., sims, j., featherman, m., & love, p. e. (2015). credibility of information in online communities. journal of strategic marketing, 23(3), 238-253. https://doi.org/10.1080/0965254x.2014.920904 hall, s. (2007). encoding and decoding in the television discourse. in a. gray, j. campbell, m. erickson, s. hanson, & h. wood (eds.) cccs stencilled occasionalpapers (1st ed., vol. 2, pp. 386-398). milton park, uk: routledge. http://epapers.bham.ac.uk/2962/ hamzehei, r., ansari, m., rahmatizadeh, s., & valizadeh-haghi, s. (2018). websites as a tool for public health education: determining the trustworthiness of health websites on ebola disease. online journal of public health informatics, 10(3). https://doi.org/10.5210%2fojphi.v10i3.9544 hargreaves, s., bath, p. a., duffin, s., & ellis, j. (2018). sharing and empathy in digital spaces: qualitative study of online health forums for breast cancer and motor neuron disease (amyotrophic lateral sclerosis). journal of medical internet research, 20(6). https://doi.org/10.2196/jmir.9709 heymach, j., krilov, l., alberg, a., baxter, n., chang, s. m., corcoran, r. b., . . . dreicer, r. (2018). clinical cancer advances 2018: annual report on progress against cancer from the american society of clinical oncology. journal of clinical oncology, 36(10), 1020-1044. https://doi.org/10.1200/jco.2017.77.0446 holub, r. c. (2013). reception theory. milton park, uk: routledge. hooda, a., gupta, p., jeyaraj, a., & dwivedi, y. (2023). clarifying the role of e-government trust in e-government success models: a meta-analytic structural equation modeling approach. australasian journal of information systems, 27, 1-22. https://doi.org/10.3127/ajis.v27i0.4079 hovland, c. i., & weiss, w. (1951). the influence of source credibility on communication effectiveness. public opinion quarterly, 15(4), 635-650. https://doi.org/10.1086/266350 huber, j., maatz, p., muck, t., keck, b., friederich, h.-c., herzog, w., & ihrig, a. (2017). the effect of an online support group on patients׳ treatment decisions for localized prostate cancer: an online survey. urologic oncology: seminars and original investigations, 35(2), 37. e19-37. e28. https://doi.org/10.1016/j.urolonc.2016.09.010 islam, j. u., & rahman, z. (2017). the impact of online brand community characteristics on customer engagement: an application of stimulus-organism-response paradigm. telematics and informatics, 34(4), 96-109. https://doi.org/10.1016/j.tele.2017.01.004 jung, e. h., walsh-childers, k., & kim, h.-s. (2016). factors influencing the perceived credibility of diet-nutrition information web sites. computers in human behavior, 58, 3747. https://doi.org/10.1016/j.chb.2015.11.044 australasian journal of information systems badreddine et al. 2023, vol 27, selected papers from acis 2022 quality of cancer-related podcasts 24 kanthawala, s., vermeesch, a., given, b., & huh, j. (2016). answers to health questions: internet search results versus online health community responses. journal of medical internet research, 18(4), e95. https://doi.org/10.2196/jmir.5369 khan, i., loh, j., saleh, a., quazi, a., & talukder, m. (2021). effect of barrier related factors on perceived usefulness and ease of use of social media applications in the australian healthcare sector. australasian journal of information systems, 25. https://doi.org/10.3127/ajis.v25i0.2625 kim, h.-s., & mrotek, a. (2016). a functional and structural diagnosis of online health communities sustainability: a focus on resource richness and site design features. computers in human behavior, 63, 362-372. https://doi.org/10.1016/j.chb.2016.05.004 kitamura, c. (2021). the life and times of the berlin secession podcast. master thesis, lindenwood university, st. chares, mo, usa. https://digitalcommons.lindenwood.edu/theses/7/ koo, c., wati, y., park, k., & lim, m. k. (2011). website quality, expectation, confirmation, and end user satisfaction: the knowledge-intensive website of the korean national cancer information center. journal of medical internet research, 13(4), e81. https://doi.org/10.2196/jmir.1574 laranjeiro, n., soydemir, s. n., & bernardino, j. (2015). a survey on data quality: classifying poor data. 2015 ieee 21st pacific rim international symposium on dependable computing (prdc. zhangjiajie, china, november 18-20, 2015. https://ieeexplore.ieee.org/document/7371861 leung, l. (2008). internet embeddedness: links with online health information seeking, expectancy value/quality of health information websites, and internet usage patterns. cyberpsychology & behavior, 11(5), 565-569. https://doi.org/10.1089/cpb.2007.0189 li, y., wang, x., lin, x., & hajli, m. (2018). seeking and sharing health information on social media: a net valence model and cross-cultural comparison. technological forecasting and social change, 126, 28-40. https://doi.org/10.1016/j.techfore.2016.07.021 liaw, s.-t., rahimi, a., ray, p., taggart, j., dennis, s., de lusignan, s., . . . talaei-khoei, a. (2013). towards an ontology for data quality in integrated chronic disease management: a realist review of the literature. international journal of medical informatics, 82(1), 10-24. https://doi.org/10.1016/j.ijmedinf.2012.10.001 lin, m., thoma, b., trueger, n. s., ankel, f., sherbino, j., & chan, t. (2015). quality indicators for blogs and podcasts used in medical education: modified delphi consensus recommendations by an international cohort of health professions educators. postgraduate medical journal, 91(1080), 546-550. https://doi.org/10.1136/postgradmedj2014-133230 lleras de frutos, m., casellas-grau, a., sumalla, e. c., de gracia, m., borràs, j. m., & ochoa arnedo, c. (2020). a systematic and comprehensive review of internet use in cancer patients: psychological factors. psycho-oncology, 29(1), 6-16. https://doi.org/10.1002/pon.5194 lowry, p. b., wilson, d. w., & haig, w. l. (2014). a picture is worth a thousand words: source credibility theory applied to logo and website design for heightened credibility and australasian journal of information systems badreddine et al. 2023, vol 27, selected papers from acis 2022 quality of cancer-related podcasts 25 consumer trust. international journal of human-computer interaction, 30(1), 63-93. https://doi.org/10.1080/10447318.2013.839899 lynch, n. p., lang, b., angelov, s., mcgarrigle, s. a., boyle, t. j., al-azawi, d., & connolly, e. m. (2017). breast reconstruction post mastectomy-let's google it. accessibility, readability and quality of online information. the breast, 32, 126-129. https://doi.org/10.1016/j.breast.2017.01.019 ma, t. j., & atkin, d. (2017). user generated content and credibility evaluation of online health information: a meta analytic study. telematics and informatics, 34(5), 472-486. https://doi.org/10.1016/j.tele.2016.09.009 ma, y. (2002). a design analysis model for developing world wide web sites. journal of the american society for information science and technology, 53(7), 531-535. https://doi.org/10.1002/asi.10074 mcleod, j., yu, i., & ingledew, p.-a. (2017). peering into the deep: characterizing the internet search patterns of patients with gynecologic cancers. journal of cancer education, 32(1), 85-90. https://doi.org/10.1007/s13187-016-1002-9 miles, m. b., huberman, a. m., & saldana, j. (2014). qualitative data analysis: a methods sourcebook. (3rd ed.). thousand oaks, ca, usa: sage. mills, e., & millward, h. (2019). emotional well-being for people with cancer: a narrative review of charity sector resources and peer-reviewed articles for effects of complementary interventions. https://eprints.lancs.ac.uk/id/eprint/139375/1/the_landuu_company_emotional_well _being_project_report.pdf min, a. a., morley, e. j., rezaie, s. r., fox, s. m., & grock, a. (2018). academic life in emergency medicine blog and podcast watch: respiratory emergencies. cureus, 10(6), e2812. https://doi.org/10.7759/cureus.2812 monton, o., lambert, s., belzile, e., & mohr-elzeki, d. (2019). an evaluation of the suitability, readability, quality, and usefulness of online resources for family caregivers of patients with cancer. patient education and counseling, 102(10), 1892-1897. https://doi.org/10.1016/j.pec.2019.05.010 moshagen, m., & thielsch, m. (2013). a short version of the visual aesthetics of websites inventory. behaviour & information technology, 32(12), 1305-1311. https://doi.org/10.1080/0144929x.2012.694910 mun, y. y., yoon, j. j., davis, j. m., & lee, t. (2013). untangling the antecedents of initial trust in web-based health information: the roles of argument quality, source expertise, and user perceptions of information quality and risk. decision support systems, 55(1), 284-295. https://doi.org/10.1016/j.dss.2013.01.029 nwosu, a. c., monnery, d., reid, v. l., & chapman, l. (2017). use of podcast technology to facilitate education, communication and dissemination in palliative care: the development of the amipal podcast. bmj supportive & palliative care, 7(2), 212-217. https://doi.org/10.1136/bmjspcare-2016-001140 australasian journal of information systems badreddine et al. 2023, vol 27, selected papers from acis 2022 quality of cancer-related podcasts 26 paterson, q. s., thoma, b., milne, w. k., lin, m., & chan, t. m. (2015). a systematic review and qualitative analysis to determine quality indicators for health professions education blogs and podcasts. journal of graduate medical education, 7(4), 549-554. https://doi.org/10.4300/jgme-d-14-00728.1 pope, c., & mays, n. (2019). qualitative research in health care. hoboken, nj, usa: john wiley & sons. https://doi.org/10.1002/9781119410867 ractham, p., techatassanasoontorn, a., & kaewkitipong, l. (2022). old but not out: social media use and older adults' life satisfaction. australasian journal of information systems, 26. https://doi.org/10.3127/ajis.v26i0.3269 rennick-egglestone, s., ramsay, a., mcgranahan, r., llewellyn-beardsley, j., hui, a., pollock, k., . . . roe, j. (2019). the impact of mental health recovery narratives on recipients experiencing mental health problems: qualitative analysis and change model. plos one, 14(12), e0226201. https://doi.org/10.1371/journal.pone.0226201 riddell, j., robins, l., brown, a., sherbino, j., lin, m., & ilgen, j. s. (2020). independent and interwoven: a qualitative exploration of residents’ experiences with educational podcasts. academic medicine, 95(1), 89-96. https://doi.org/10.1097/acm.0000000000002984 robertson-lang, l., major, s., & hemming, h. (2011). an exploration of search patterns and credibility issues among older adults seeking online health information. canadian journal on aging/la revue canadienne du vieillissement, 30(4), 631-645. https://doi.org/10.1017/s071498081100050x salaün, y., & flores, k. (2001). information quality:: meeting the needs of the consumer. international journal of information management, 21(1), 21-37. scott, c., campbell, p., nemirovsky, a., loeb, s., & malik, r. (2023). prostate cancer and podcasts: an analysis and assessment of the quality of information about prostate cancer available on podcasts. prostate cancer and prostatic diseases, 1-2. https://doi.org/10.1038/s41391-023-00694-y stvilia, b., gasser, l., twidale, m. b., & smith, l. c. (2007). a framework for information quality assessment. journal of the american society for information science and technology, 58(12), 1720-1733. https://doi.org/10.1002/asi.20652 stvilia, b., mon, l., & yi, y. j. (2009). a model for online consumer health information quality. journal of the american society for information science and technology, 60(9), 1781-1791. thoma, b., chan, t. m., paterson, q. s., milne, w. k., sanders, j. l., & lin, m. (2015). emergency medicine and critical care blogs and podcasts: establishing an international consensus on quality. annals of emergency medicine, 66(4), 396-402. e394. https://doi.org/10.1016/j.annemergmed.2015.03.002 wang, r. y., & strong, d. m. (1996). beyond accuracy: what data quality means to data consumers. journal of management information systems, 12(4), 5-33. https://doi.org/10.1080/07421222.1996.11518099 whitten, p., smith, s., munday, s., & laplante, c. (2008). communication assessment of the most frequented breast cancer websites: evaluation of design and theoretical criteria. journal of computer-mediated communication, 13(4), 880-911. australasian journal of information systems badreddine et al. 2023, vol 27, selected papers from acis 2022 quality of cancer-related podcasts 27 https://doi.org/10.1111/j.1083-6101.2008.00423.x willis, i. (2021). reception theory, reception history, reception studies. in oxford research encyclopedia of literature. oxford, uk: oxford university press. https://doi.org/10.1093/acrefore/9780190201098.013.1004 win, k. t., hassan, n. m., oinas-kukkonen, h., & probst, y. (2016). online patient education for chronic disease management: consumer perspectives. journal of medicalsystems, 40(4), 88. https://doi.org/10.1007/s10916-016-0438-0 yi, m., kim, j., noh, d.-y., lee, j.-l., yoo, k.-y., hwang, k.-t., & chung, h.-d. (2008). evaluation of the satisfaction and usefulness of a web-based educational program for breast cancer patients. the open medical informatics journal, 2, 129. https://doi.org/10.2174/1874431100802010129 zaver, f., hansen, m., leibner, e., little, a., & lin, m. (2016). blog and podcast watch: pediatric emergency medicine. western journal of emergency medicine, 17(5), 513. https://doi.org/10.5811/westjem.2016.6.30193 zhang, p., small, r. v., von dran, g. m., & barcellos, s. (2000). a two factor theory for website design. in proceedings of the 33rd annual hawaii international conference on system sciences (pp. 10-pp). ieee. zhang, p., & von dran, g. m. (2000). satisfiers and dissatisfiers: a two-factor model for website design and evaluation. journal of the american society for information science, 51(14), 1253-1268. https://doi.org/10.1002/1097-4571(2000)9999:9999<::aid-asi1039>3.0.co;2-o zhang, y., sun, y., & xie, b. (2015). quality of health information for consumers on the web: a systematic review of indicators, criteria, tools, and evaluation results. journal of the association for information science and technology, 66(10), 2071-2084. https://doi.org/10.1002/asi.23311 zhang, z. (2010). feeling the sense of community in social networking usage. ieee transactions on engineering management, 57(2), 225-239. https://doi.org/10.1109/tem.2009.2023455 zhao, y. c., zhao, m., & song, s. (2022). online health information seeking behaviors among older adults: systematic scoping review. journal of medical internet research, 24(2), e34790. https://doi.org/10.2196/34790 zheng, y., zhao, k., & stylianou, a. (2013). the impacts of information quality and system quality on users' continuance intention in information-exchange virtual communities: an empirical investigation. decision support systems, 56, 513-524. https://doi.org/10.1016/j.dss.2012.11.008 australasian journal of information systems badreddine et al. 2023, vol 27, selected papers from acis 2022 quality of cancer-related podcasts 28 copyright: © 2023 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v27i0.4435 http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 literature review 3 theoretical framework 3.1 podcasts’ quality dimensions 3.1.1 content 3.1.1.1 accuracy 3.1.1.2 relevancy 3.1.1.3 usefulness 3.1.1.4 completeness 3.1.1.5 richness 3.1.1.6 currency 3.1.1.7 interpretability 3.1.2 credibility 3.1.2.1 trustworthiness 3.1.2.2 expertise 3.1.3 design 3.1.3.1 navigation 3.1.3.3 aesthetics 3.2 audience reception 4 methodology 5 findings 6 discussion 7 conclusion paper title (paper title style) australasian journal of information systems fernando & scholl 2020, vol 24, selected papers from the 2019 aice conference value tensions and the value of data 1 towards using value tensions to reframe the value of data beyond market-based online social norms anisha t j fernando lecturer, saibt university of south australia kathleen lumley college anisha.fernando@unisa.edu.au lesa scholl head of college kathleen lumley college abstract making sense of data, its value, and impact is imperative for individuals, organisations, and societies to function in the cyber age. the online interactions through which data flows present many benefits. however, the consumption of data and its value is problematic due to an overreliance on market norms as a substitute for values-based online social norms and practices, creating value tensions. understanding the implications of data is further complicated due to the complex contextual nature of online interactions. these challenges are addressed through efforts from technology organisations and policy initiatives. largely absent from these efforts is an understanding of the values needed to ground healthy online social interactions, and processes that nurture and afford the practice of these values in contextual community settings. value tensions as an ethics tool can surface and clarify these interpersonal needs in understanding data and its impact, while its exploration can be grounded by the value sensitive design methodology. communities may be appropriately placed to grapple with these value tensions given the contextual nature of interactions and the collective benefits and harms at stake. this discussion paper presents a preliminary research agenda raising questions on uncovering value tensions and understanding the values at stake to transform data practices and develop healthy online social norms, to reframe the value of data beyond market-based online social norms. keywords: value sensitive design, value tensions, social values, online social norms, data flows, data literacy, data ethics. 1 introduction: purpose and context online interactions are fundamental to 21st century life and offer many benefits for people and organisations alike such as effectiveness, efficiencies, convenience, and cost-savings. these interactions and behaviours are influenced by online social norms such as: checking facebook or instagram while on the bus, at work or when interacting with others; or, asking google for instant answers when mildly curious. these pervasive activities may be harmless, but people seem to trust the information consumed even when it is misinformed (thorson 2016). these new sources of data diets indicate a growing cultural trend where people may prefer consuming information via search engines over the need to access specialist knowledge sources. by contrast, these activities have non-transparent and hidden possible traps of engaging online, unless values ground these interactions. for example, decisions can be made with australasian journal of information systems fernando & scholl 2020, vol 24, selected papers from the 2019 aice conference value tensions and the value of data 2 misleading or misinformed non-specialist opinions presented as facts (thorson 2016). algorithms controlling the visibility and access to information may be primarily incentivised to generate monetary value rather than evaluate the trustworthiness of information. recent changes made to google’s algorithms to focus on expertise, authoritativeness and trustworthiness measures for ymyl (your money your life) websites (i.e. content such as legal, financial, health-related etc. which may impact satisfaction and well-being of information consumers), indicate plausible risks and underscore the importance of healthy information consumption (ray 2019). in this context, values refer to social values, i.e. moral guiding principles of life and motivating behaviours practiced by people. these values and practices impact decisions people take in the pursuit of how to live well, shaping their identities and beings. the nature of the roles played by people in various societal units such as families, communications and organisations are influenced by these social values (bruce 1999). when people use technology, their values and practices are mediated by technology and its design (verbeek 2011). decisions people take in socio-technical contexts are technomoral choices shaped by the affordances of technologies (vallor 2018). these values do not exist in isolation, but underpin online interactions. value sensitive design (vsd) is a methodology which considers the values at stake when designing technology. in the vsd context, a value is “what is important to people in their lives, with a focus on ethics and morality” (friedman & hendry 2019 p. 24). value tensions, a vsd tool, offers an opportunity to consider values that may be in contrast, but hold importance in its interconnected relationship (friedman & hendry 2019). the specific value tension that is pertinent to this discussion on understanding data is the tension between marketand social-based norms at play through online interactions. given the focus on social values and the collective nature of its impact, the scope, and notions of values in the context of this exploration is at the broader societal level. within popular media and the broader society, there are questions around the ways in which online interactions are challenging established traditional family, cultural, or social values. conventionally social norms are established through interactions in social structures such as families, communities, schools, but technology-based spaces present a new dynamic. social norms in these spaces are mediated by technology (verbeek 2011). values embedded in the design of technology may afford certain practices, influencing the establishment of social norms. general societal cues that signpost behaviour such as body language and practices like accountable relationships with people are not as visible in the online sphere. for example, the notice and consent approach or the default opt-in approach used to express privacy expectations on online websites does not recognise that people may not completely understand the details involved when consenting to use of the website and to transfer of their personal data (nissenbaum 2010). valuing data through the lens of marketbased norms may drive and increase the appetite for monetary value, risking the social nature and the integrity of online interactions. every time people are online, they generate data, use data, leak data, make decisions based on data and trust data. this data diet is increasingly becoming a frenzy-driven, adrenalin-fuelled activity where individuals seem to have the freedom to act through this lens (eyal 2014). such behaviour is often devoid of traditional norms such as respect for others or norms get drowned out or lost. then what becomes easier and transparent are outrageous morally unacceptable australasian journal of information systems fernando & scholl 2020, vol 24, selected papers from the 2019 aice conference value tensions and the value of data 3 behaviours experienced online such as trolling and cyber-bullying, and the normalising of carefully curated social worldviews leading to even instagram or twitter-driven suicides. other subtle unacceptable behaviours people think they can get away include photoshopping, creating deep fakes, manipulating other people’s data to obscure the truth, hacking attractive honeypots, and social engineering sophisticated phishing campaigns to target people with poor awareness and data literacy skills. for example, comparing different notions of value where norms are distorted: creating and concealing fake money is forgery and money laundering, which are prosecutable crimes; while creating actual fake news to fuel misinformation or disinformation campaigns breaks social contracts, harms the integrity of interactions, and diminishes trust, but is usually dismissed as the internet’s next evolving fad. the boundaries between socio-technical practices that are considered appropriate and inappropriate are becoming blurred. hence, the consumption of data and its value, however, is problematic because people are more likely to propagate polarised views, are exposed to misinformation and untrustworthiness of sources, encounter disinformation campaigns, and experience effects of social isolation, mistrust and technology addiction (policy department for citizens' rights and constitutional affairs 2019; badawy, lerman & ferrara 2018; pasquale 2015; center for humane technology 2019; courtwright 2019). current efforts by organisations and governments are largely framed from an institutional standpoint, whereas data ethics issues usually affect individuals personally. institutions are seen to not take accountability, while individuals are absolved of their responsibility to take action due to the siloed nature of sociotechnical interactions. for example, the ethical impacts of big data use and the complex nature of organisational needs are well documented (boyd & crawford 2012; crawford, miltner & gray 2014). community-based initiatives may reinvigorate exploration of the values and the value tensions at play in relation to societal needs. this discussion paper proposes the need to rethink or reframe the value of online data and the need to explore possibilities to encourage people to create healthy online spaces and practices using value tensions as an ethics tool to aid this process in community settings. examples of common online interactions are used to describe market-based online social norms experienced and arguments are presented to empathise the need for online social norms, need to also be rooted in social values to sustain healthier ways of interacting online. there is a need to uncover a deeper understanding of how surfacing this value tension can help people understand the contextual nature of their online interactions and how social values are formed and practice. this exploration of value tensions as an ethics tool can also provide useful input to designers of these technologies in imbuing ethical social values in technologies and mediating healthier online social norms. 2 motivation and impact: understanding the value tension at stake social vs. market based online norms in this section, the specific value tension of social vs. market norms influencing online interactions and the subsequent impact of data and its value is discussed. an overreliance on market-based online social norms has the potential to disrupt the integrity of people’s online interactions. in this cyber age, market incentives drive data creation and use. technology organisations are heavily influenced by advertising, where data is monetised. this data is freely acquired from people’s interactions with proprietary technology, and its complexity is australasian journal of information systems fernando & scholl 2020, vol 24, selected papers from the 2019 aice conference value tensions and the value of data 4 oblivious to the everyday person. monetary value is manifested in the aggregation of data and the creation of behavioural surplus (benkler 2006; ariely 2008; zuboff 2019). these technology organisations face competing incentives, where their strategic business goals compete with their stated intentions for ethical practices. social norms will not hold in a market exchange because these are not grounded in social values (ariely 2008). for example, through social production, data is freely created by people in exchange for online services (benkler 2006). the value of data (i.e. information goods in this context) appreciates with time and increases with aggregated use compared to traditional market goods that depreciate over time (ariely 2008). the authorship of content is lost, and people are not rewarded in a typical market-based version of social production (lanier 2014). it is harder for technology organisations to verify the trustworthiness of data when the incentive is to publish fast with sensationalism because these indicators drive user clicks, which drive advertising revenue and increase market-based value. traditional gatekeepers are replaced or devalued, and efforts to introduce gatekeepers, as done by facebook, fall short because within this new medium the norms and impacts are in flux and the rate of data production is more than what is manageable by a single entity regardless of its size (zuboff 2019). hence, organisations are not best placed to foster nurturing healthy online social norms because their structures may not lend to effective accountability (as seen in the debate to define the big technology organisations as media or technology companies). these market-based online social norms propagate data being valued in monetary terms, driven by market incentives. for example, facebook generated us55.8 billion in revenue for 2018, using people’s attention, data and interactions as raw materials in this data processing chain that drives surveillance capitalism (statista 2019; zuboff 2019). established information providers use advertising to drive network effects, reduce operating costs and enable greater efficiency. this framing is risky, as data communicates details about people – it reveals our identities and preferences; our personas become public sources with limited control and open to influencing. the value of data is being framed as mostly transactional and lies in aggregation from a market-based perspective. statistically significant inferences guide the development of online interactions like personalised search results. it is not profitable to solely target an individual person’s preferences and habits with higher degrees of precision. a person’s identity is intertwined with their experiences. data is also human, containing characteristics about people. is the social association with data about people and their connections lost when viewing it through this marked-based frame? people ascribe meaning to information derived based on data processed through these interactions (thorson 2016, casanovas et al. 2017; dawson et al. 2019). through these misplaced intentions and unethical practices in creating and releasing data, value is placed on data by external systems. this absence of adequate agency by people leaves vulnerabilities for data misuse and misinterpretation. this distorted, manipulated, and fictious data accumulates value through interactions and sharing as opposed to value created because of its reliability, credibility, and trustworthiness (pasquale 2015; zuboff 2019). these market-based norms add to the complexity of decision-making involved in online interactions. for example, consenting to privacy is termed conventionally as trade-offs exploring the nature and the extent to which people are willing to release personal data for benefits (financial or otherwise). however, humans are cognitively limited in their ability to make complex decisions such as the decisions required in online privacy (ariely 2008). low australasian journal of information systems fernando & scholl 2020, vol 24, selected papers from the 2019 aice conference value tensions and the value of data 5 levels of technical literacy and aliteracy inhibit contextual decision-making in online contexts (naughton 2012). technical literacy concerns the levels of awareness and understanding around risks associated with data flows. aliteracy occurs when people have the ability to understand but are not interested or find it difficult to convey their expectations due to complexities in implementing and using alternate mechanisms. for example, a survey of australian consumers reported that people found it difficult to distinguish the provenance and operation of different elements of a search engine results page, particularly with integrated vertical search services and less so with distinguishing ads from search results (scardamaglia & daly 2016). the disconnect between social and market values around technology innovation triggers a norm, where the market value take precedence over the social value. traditionally, human interactions are premised on social values. in online interactions, social values will not hold in a market exchange because the interaction is not premised on social values, but largely influenced by market-based values (ariely 2008). for example, standardised testing introduced mechanisms of measuring student performance in education. however, it has become the de-facto measure of student learning, where the incentive is to perform better on tests as opposed to creating a stable culture focused on learning where performances achieving good learning outcomes will naturally emerge (ariely 2008). both students and schools are measured against this indicator, which influences their roles in a learning context, which is usually termed as a social activity. the effects of social values, whether positive or negative, impact human beings, not technology. social cohesiveness is lost in these market-based online exchanges. market-based norms and values stems from commercial motivations. the cost of interactions is considered free by people, largely influenced by the ethos stemming from academic activities that kickstarted the development of the internet. being on the internet is viewed as a social exchange. for example, on social media, friendships are measured by the amount of likes a photo gets or the traction a post gets. these measures of association show a quantitative dimension of popularity, while the qualitative sentiments are lost, losing that richness and depth normally associated with friendships. people are accustomed to encountering new functionality and form new habits through their online interactions. people may be incentivised to provide inaccurate information to avoid loss of social values like privacy and such falsification of data is a dis-benefit to organisations because data quality diminishes too. market-based norms tend to commodify social values, whereas people see services as being free on the internet. is there a place for social values in a medium driven by market norms? for example, the country fire service organise themselves around a community purpose for safety and wellbeing. in open source software development, many amateur and expert programming enthusiasts spent countless hours problem-solving and helping each other out. these communities are widely popular and sustain themselves without any significant market incentive, and prioritise social values over market-based values. community-facilitated initiatives may help people make sense of data, its value and impact beyond experiencing and habituating to largely market-based values and norms. a community setting can make visible the importance of protecting social values and the integrity of online interactions in technology-mediated spaces. the benefits and harms imbued by technology are not just limited to individuals, but have a collective societal impact and scope. australasian journal of information systems fernando & scholl 2020, vol 24, selected papers from the 2019 aice conference value tensions and the value of data 6 3 current efforts and new challenges initiatives to address making sense of data, its value and impact from an ethical lens in the cyber age focus mainly on technological and legal perspectives. technology initiatives which include efforts to create empathetic and ethically minded technologies are growing, for example, educational programs focus on equipping technologists and it professionals with data ethics principles (massachusetts institute of technology 2019). these efforts are codified in organisational settings through professional codes of conduct, policies, and data ethics pledges. these professionals are the focus of educational efforts because their actions in their professional roles determine how personal data of people is used. such education is focused on how organisations can ethically use the data for the purpose for which it is collected. even in this organisational context, questions arise in terms of vested competing interests, with commercial market incentives influencing ethical practices and shaping online social norms. other initiatives addressing online ethics are policy programs and legal reframing. visible policy mechanisms include eu regulations around legal compliance measures to protect data across transnational boundaries, creating independent ethics advisory boards, training programs; antitrust measures and lawsuits (casanovas et al. 2017; dawson et al. 2019; stiglitz 2019). efforts to divide large technological companies are suggested as they act as monopolies with disproportionate market effects, creating power and information asymmetries (stiglitz 2019; zuboff 2015; nissenbaum 2010). while expected behaviours and appropriate values to be upheld are codified for it professionals in organisational or societal bodies through codes of conduct, nothing equivalent exists for appropriate norms and data practices for individuals and members of communities. it is unclear how online social values and norms are nurtured given the strong influence of technology-mediation in interactions (verbeek 2011). tools are extensions of self and afford us value in the act of interacting (gibson 1979; heidegger 1977). if the core value is not embedded in interactions, the value cannot be exercised. individuals who use these technologies are absolved from their personal responsibility because of the isolating, siloed, contextually-public nature of online interactions. missing from this discourse is a discussion around community-centred initiatives to address these challenges. a person’s data is about that individual. often the best forms of privacy protection or effectiveness of ethical practices is in the act of data creation, because this is where the risk is at a minimum. it is important to develop proactive data protection initiatives because privacy loss is often instantaneous and reputational, and may affect more than just an individual, given its interconnected nature. while advertisers need to take responsibility for the secondary collection of data and could benefit from education around incentives and ethical practices, community education initiatives are also needed. there is a lack of community initiatives to teach people basic data skills and data ethics practices to nurture and guide their socio-technical interactions. 4 conclusion: future steps and research agenda given that data consumption is engineered, and unhealthy online social norms and its effects are becoming more evident in society, the authors propose a preliminary research agenda to address these questions: • what are the values at stake in different market-influenced online contexts? australasian journal of information systems fernando & scholl 2020, vol 24, selected papers from the 2019 aice conference value tensions and the value of data 7 • what can be done to reframe the valuing of data and transform data practices? • what can be done to develop healthy online social norms? to address these questions, the proposed research methodology needs to be cognisant of the values at play. value sensitive design (vsd) considers the application of human values in the design of technology through a rigorous process (friedman, kahn & borning 2009). value tensions as a vsd-based ethics tool could help unpack contrasting values of data influenced largely by market norms and offer opportunities to ask how social values may help reframe or understand data and its value. this work grounded in vsd, would contribute towards extending current understandings of online social values and norms, and would explore how broader societal structures are impacted by these socio-technical value tensions. future work would include conducting a systematic literature review on the application of value tensions as an ethics tool to identify mechanisms through which value tensions was prototyped. following this conceptual exploration, the proposed value tensions tool will be co-designed with community-based participants and rapidly prototyped to create network learning resources which will propagate understanding data, reframing its value, and surface social values for online interaction contexts. investigations exploring these questions underpinned by vsd will provide a useful lens through which to study the value tensions driven by social vs. market-based norms in online contexts. unhealthy online social norms driven by market incentives influence people’s values and practices around data. for example, misinformation propagated by algorithms incentivised by monetary value risks the trustworthiness of information and the integrity of the interactions through which people consume information. market-based norms create value tensions with the social nature of online interactions. value tensions as an ethics tool is proposed as a method to facilitate an understanding of data and the social values at stake in online contexts through community-based initiatives. it is important to understand how healthy online social norms, practices and values can be established to reframe the value attached to data and sustain the integrity of socio-technical interactions. acknowledgements the authors would like to thank joseph hall for discussions on the early versions of this paper for the aice 2019 conference, and monica behrend for stimulating critique and reviewing early drafts. the authors would also like to thank the anonymous reviewers who have taken the time to review and provide insightful comments that helped develop this paper. references ariely, d. (2008). predictably irrational: the hidden forces that shape our decisions. new york: harpercollins publishers ltd. badawy, a., lerman, k., & ferrara, e. (2018, august). who falls for online political manipulation? retrieved from https://arxiv.org/abs/1808.03281 benkler, y. (2006). the wealth of networks: how social production transforms markets and freedom. new haven, connecticut yale: university press. boyd, d., & crawford, k. (2012). critical questions for big data: provocations for a cultural, technological, and scholarly phenomenon. information, communication & society. 15(5), 662-679. australasian journal of information systems fernando & scholl 2020, vol 24, selected papers from the 2019 aice conference value tensions and the value of data 8 bruce, s. (1999). sociology: a very short introduction. great britain: oxford university press. casanovas, p., de koker, l., mendelson, d., & watts, d. (2017). regulation of big data: perspectives on strategy, policy, law and privacy. health and technology, 7(4), 335-349. center for humane technology. (2019, june). the problem: the extractive attention economy is tearing apart our shared social fabric. retrieved from https://humanetech.com/problem/ courtright d. t. (2019). the age of addiction: how bad habits become big business. cambridge, massachusetts: the belknap press of harvard university. crawford, k., gray, m., & miltner, k. (2014). big data critiquing big data: politics, ethics, epistemology special section introduction. international journal of communication. 8(10), 1663 -1672. dawson d., schleiger e., horton j., mclaughlin j., robinson c., quezada g., scowcroft j., & hajkowicz s. (2019, april). artificial intelligence: australia’s ethics framework. data61 csiro, australia. retrieved from https://data61.csiro.au/en/our-work/ai-framework policy department for citizens' rights and constitutional affairs. (2019, february), disinformation and propaganda – impact on the functioning of the rule of law in the eu and its member states. european parliament. retrieved from https://www.europarl.europa.eu/regdata/etudes/stud/2019/608864/ipol_stu(2019) 608864_en.pdf eyal n. (2014). hooked: how to build habit-forming products. london, england: penguin. friedman, b., kahn, p. & borning, a himma, k. (2009). value sensitive design and information systems. in k.e. himma & h.t. tavani (eds) the handbook of information and computer ethics (pp. 69-101). hoboken, nj, usa: john wiley & sons. friedman, b. & hendry, d. g. (2019). value sensitive design: shaping technology with moral imagination. cambridge, ma: mit press. gibson, j. (1979). the ecological approach to visual perception: classic edition. boston, massachusetts: psychology press. heidegger, m. (1977). the question concerning technology and other essays. new york: harper & row. lanier, j. (2014). who owns the future? london, england: penguin books ltd. massachusetts institute of technology. (2019). professional education ethics of ai: safeguarding humanity. retrieved from https://professional.mit.edu/programs/shortprograms/ethics-ai naughton, j. (2012). what you really need to know about the internet: from gutenberg to zuckerberg. great britain: quercus. nissenbaum, h. (2010). privacy in context, stanford, california: stanford university press. pasquale, f. (2015). the black box society: the secret algorithms that control money and information. cambridge, massachusetts: harvard university press. ray, l. (2019). google’s algorithm undergoes enormous shifts to prioritise content focused on expertise, authoritativeness and trustworthiness. journal of digital & social media marketing. 7(2), 128-136. australasian journal of information systems fernando & scholl 2020, vol 24, selected papers from the 2019 aice conference value tensions and the value of data 9 scardamaglia, a. & daly, a. (2016). google, online search and consumer confusion in australia. international journal of law and information technology. 24 (3), 203-228. statista. (2019). facebook's annual revenue and net income from 2007 to 2018 (in million u.s. dollars). retrieved from https://www.statista.com/statistics/277229/facebooks-annual-revenueand-net-income/ stiglitz j.e. (2019). people, power and profits: progressive capitalism for an age of discontent. great britain: allen lane, penguin random house uk. thorson, e. (2016). belief echoes: the persistent effects of corrected misinformation. political communication. 33(3), 460-480, retrieved from doi: 10.1080/10584609.2015.1102187 vallor, s. (2018). technology and the virtues: a philosophical guide to a future worth wanting. new york: oxford university press. verbeek, p. p. (2011). moralizing technology: understanding and designing the morality of things, chicago: university of chicago press. zuboff, s. (2015). big other: surveillance capitalism and the other prospects of an information civilization. journal of information technology, 2015(30), 75-89. zuboff, s. (2019). the age of surveillance capitalism: the fight for the future at the new frontier of power. new york: public affairs. copyright: © 2020 fernando & scholl. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v24i0.2793 http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems mehdi darban 2021, vol 25, research article understanding the perception of is niche 1 understanding the perception of information system niche: conceptualization, measurement, and validation mehdi darban department of management & marketing, louisiana state university-shreveport, shreveport, la, usa mehdi.darban@lsus.edu abstract the number of niche technologies is growing, and they are getting noticed. however, the usage dynamics of such systems have not been explored. this study introduces the “niche” information systems (is) construct to differentiate between mainstream and niche technologies and to illustrate how such differences potentially influence a person’s is usage behavior. we posit that niche-ness is perceptual, and a measurement to gauge an individual’s niche perception in the domain of systems usage is desired. resting on optimal distinctiveness theory, the study develops a novel conceptualization and operationalization of niche technology perceptions, focusing on social networking systems. our main contribution includes the examination of the typology of the perceived niche, which results in the development and validation of an eight-item instrument for perceived technology niche through three studies. in addition, by placing the construct in a nomological network in a fourth study, we demonstrate that the perceived technology niche construct is related to is continuance intention. keywords: niche systems, perceived niche, conspicuousness, uniqueness, digital extended self, scale development. 1 introduction in the past two decades niche products, that is, highly individualized and tailored products designed to address the needs of select user groups instead of the mass market, have increasingly attracted people (fogliatto, et al., 2012; schaefers, 2014; tan et al., 2017). information systems (is) artifacts have experienced such a trend in some cases. the advent and successful introduction of niche social networking services (snss), such as clubhouse, behance, ello, flixster, and researchgate, serve as good examples of this phenomenon. the users of mainstream snss, like facebook and instagram, are being attracted by these niche alternatives. for instance, recent reports reveal that people are leaving facebook to join other less mainstream snss (whitehead, 2020). niche systems are designed to lure a select cluster of users dedicated to a particular activity or topic and gather them together. the highly specialized and specific feature characteristics of such systems target smaller masses of people for the advantage of less rivalry and higher prices. however, niche systems have the potential to grow and develop into a broader market over time (kolter & keller, 2006). therefore, a successful niche product can be in a better competitive position when the niche market expands. for instance, apple introduced the ipad as a niche product, which ultimately grew into a mass-market product (macmillan, 2012). is literature has mainly examined the adoption dynamics of mainstream systems and applications. the growth in the eminence of niche systems versus mainstream alternatives calls for an exploration of the concept of a technology niche. to better illustrate the usage australasian journal of information systems mehdi darban 2021, vol 25, research article understanding the perception of is niche 2 patterns of niche systems, it is important to examine how users perceive the is niche. however, an is research agenda on niche systems has yet to emerge, and the concept perceived as is niche has not received empirical attention. this study addresses this gap by focusing what is meant by “niche technology,” explicitly in the snss’ settings, and develops an instrument to measure a person’s perception of the is niche. in doing so, we delineate the domain of the technology niche and offer a multidimensional conceptualization and operationalization for the new construct. therefore, this research contributes to the is literature by developing a platform for future exploration of how niche characteristics of a technology influence success and acceptance of the system. thereby, we unified the domains of consciousness, uniqueness, and digital extended self with is consumption to improve our understanding of the usage dynamics of niche systems. the scale would benefit systems designers in the creation and continuous monitoring of niche systems. as for practitioners, how niche users view a product is both a main factor of differentiation from others (kapferer, 2012) and a central driver of usage (vigneron & jouhnson, 2017). presently, there is no instrument rating the perceived niche of an is, or a clear description of what constitutes a niche versus a non-niche is. we begin by providing a brief review of the perceived niche construct, its conceptualization, and dimensions. then, through four studies, we develop and validate our proposed scale for the technology niche at the individual level. finally, the key findings of the research are reviewed and discussed. 2 background: perceived niche optimal distinctiveness theory (odt) advocates that individuals reconcile conflicting needs for differentiation and assimilation through group memberships (brewer, 1991). this literature suggests that people have a fundamental need to defend their distinctiveness and individual identities while also having a desire to belong to social groups. individuals apply strategies to settle these opposing needs. one common strategy is to form high perceptions regarding a product they are using or a community they are joining. for instance, they may perceptually enhance how niche an sns is to convey their membership to a select group of sns users. hence, the niche-ness of an is is not automatically ingrained in its structural reality but may be perceived in the mind of individuals and be sensitive to the social setting (berger & heath, 2008). research on conspicuous consumption—people’s propensity to acquire and exhibit exclusive goods—provides further explanation for the above argument. for instance, schaefers (2014) posits that using niche products, defined as products possessing a higher degree of specificity than corresponding mass-market alternative products, is a means to address one’s need for a promoted social standing. individuals are more likely to use niche products to express themselves than mainstream ones, which are unlikely to allow a person to express her/his identity (berger & heath, 2008). hence, there is agreement in the literature to regard niche products as products that, as they are acquired and consumed, bring reverence to the person, apart from their functional utility. put differently, niche products enable users to fulfill both psychological and functional needs. extending this notion to the context of snss, we may contend that niche snss satisfy the functional needs of a specific group of users (e.g., establishing collaborative research for researchers and scientists through researchgate) and australasian journal of information systems mehdi darban 2021, vol 25, research article understanding the perception of is niche 3 may also address users’ psychological desire to stay distinct from other professionals (i.e., a researcher rather than a typical professional (brewer, 1991)). the extant literature considered that an is artifact is either niche or not niche, that is, mainstream. while, as we cited above, odt implies that the sense of niche-ness of an is product could be perceptual (berger & heath, 2008). it is sensible to argue that not all niche products are equally niche. likewise, the same product’s level of niche attribute may be rated differently by different individuals. for instance, as kemp (1998) mentions “water” could be perceived by some observers as a niche product and by some as a necessity, contingent upon who wants water or why. interestingly, in different situations, the same product could be either a niche or a necessity for the same person (vigneron & johnson, 2017). accordingly, a user might perceive a system as highly niche while another person may view it as mainstream. also, we may observe a difference between niche is associated with an upper range and those associated with the lower range of niche. for example, some snss may be outlined as niche systems, but all those snss may not be perceived niche, equally. linkedin could be a niche sns across the general population as it is for professionals and relatively specific social activities (e.g., finding a job). while researchgate, compared to linkedin, may be perceived as more niche, since it is for a particular cluster of professionals (i.e., researchers). 2.1 perceived niche dimensions to the best of our knowledge, there is no investigation into niche is. hence to study the domain of niche systems, focusing on snss, we started with reviewing the relevant literature and conducting a series of qualitative research efforts, including focus group interviews and subsequent validation studies. reviewing psychometric works niche marketing strategies, status-seeking, and conspicuous consumptions (dogan et al., 2018; echols & tsai, 2005; parrish, 2010; vigneron & johnson, 2017), we proposed a conceptual framework to measure perceived system niche (figure 1). the extant research argues that apart from personal motivations of consumption and usage behaviors, the interpersonal and external factors, such as social membership groups, derive those behaviors (darban & polites, 2020; roux et al., 2017). this conceptualization of perceived niche that the construct has two orientations, i.e., personal and interpersonal, is in accordance with the central tenet of odt. the theory argues that in defining their identity in relation to others, people strive to achieve a balance between the interpersonal and personal motives derived from ingroup-outgroup categorization (brewer & pickett, 2014). accordingly, we incorporated the personal and interpersonal dimensions in our developed description of the construct. it helped us establish a balance between these two groups of niche consumption motives for our conceptualization of niche construct, which reflects both ingroup and outgroup needs. this approach is consistent with previous research on status-directed consumption, as it demonstrated that consumption behavior varies between people depending on their susceptibility to personal and interpersonal influences (vigneron & johnson, 2017). following the literature, we included “consciousness” and “uniqueness,” which represent interpersonal values (e.g., amaldoss & jain, 2008) and “digital extended self,” which reflects personal motivations (e.g., villanova, 2019). hence, our subsequent qualitative and quantitative scale development efforts provide initial support to the proposed conceptualization of the perceived niche. australasian journal of information systems mehdi darban 2021, vol 25, research article understanding the perception of is niche 4 figure 1. framework of perceived niche index 2.2 conspicuousness conspicuousness refers to a product’s value-expressive aspect (bajac et al., 2018). people develop desires for conspicuous consumption, such as using niche products, to influence others’ perceptions about themselves (brewer, 1991; patsiaouras & fitchett, 2012). therefore, sns users would prefer niche alternatives to signal their status. as niche snss might be seen as less known and elite products, unlike mainstream snss, their use could be a mechanism for satisfying user’s interpersonal motives, such as yearning for status-signaling (schaefers, 2014). therefore, the use of niche products can address the users’ functional needs better than corresponding mainstream alternatives and also transmit a symbolic meaning (gierl & huettl, 2010). thus, the conspicuousness aspect of a niche system lets the user be noticed and expressed. 2.3 uniqueness uniqueness refers to the extent to which a product is distinct from products in the same category (cho & suh, 2020). prior studies, including research on uniqueness (snyder & fromkin, 2012), intergroup differentiation (jetten et al., 2004), and optimal distinctiveness (brewer, 1991), argue that people deviate from similar others. specifically, the uniqueness literature posits that individuals may resist conformity as too much similarity leads to negative emotional reactions (snyder & fromkin, 2012). to mitigate this unwelcome state, they tend to differentiate themselves. hence, people susceptible to interpersonal influences may prefer to drop mainstream products in favor of more niche ones to enhance their social images (wan et al., 2018). schaefers (2014) points out that the use of niche products might be a strategy to improve a person’s social image and communicate her/his uniqueness. prior research found that the scarcity, as opposed to the popularity, of a product enhances the perceived uniqueness of it, which may drive user’s preferences for that product (pantzalis, 1995). similarly, in the context of snss, the uniqueness dimension of niche refers to the concept of exclusivity, which may attract the sns user who loses interest in mainstream and popular australasian journal of information systems mehdi darban 2021, vol 25, research article understanding the perception of is niche 5 products (vigneron & johnson, 2017). we observe that niche snss are designed for an exclusive group of users, rather than for the general population. for instance, while facebook is designed for (almost) everyone, ello is for sns users who have interest in digital art. 2.4 digital extended self sns users may prefer a niche system because of its symbolic meaning (wang & lee, 2020). a product must have a symbolic meaning to enable users to express status clearly. such systems offer the possibility for a user to extend the self by integrating symbolic meanings into their identities, which results in the creation and enhancement of the self (belk, 1988). specifically, the identity-relevant products (vs. neutral ones), such as snss (wang et al., 2020), provide greater capacity to signal one’s identity (ferraro, et al., 2011). the concept of extended self indicates that individuals view their group membership, or being part of a users’ cluster of specific snss, as part of their identity (parkinson et al., 2018). belk (2018) updated the concept of extended self to the is realm by introducing digital extended self. in the pre-digital age, we could try out new identities by owning specific clothes or cars, while in the present digital era, our online virtual appearance offers a more accessible and safer context for such self-experimentation. belk noted that an affiliation with certain virtual communities might be perceived as an extension of a person’s real self. similarly, adopting a specific system, like a niche sns, may create a context for a person to fully represent his/herself and form the sense of virtual self. therefore, the creation and the communication of the desired extended self, play a significant role in motivating a person to use a niche product. stage of scale development sample details results study 1 preliminary qualitative data collection business students (n = 27) literature review & focus group sessions list of 22 candidate items item generation expert judges (n =4) face validity 22 items reduced to 11 items qsort expert judges (n = 8) item replacement ratios and kappa (construct validity) 11 items reduced to 10items study 2 -initial longitudinal validation study amazon’s mt (n = 180) at two time period, 8 weeks apart efa, reliability, discriminant and convergent validity, cfa 10 items reduced to 8 items study 3 -longitudinal validation and establishment study amazon’s mt (n = 353) at two time periods, 8 weeks apart reliability, discriminant and convergent validity, cfa final list of 8 items study 4 – nomological validity testing amazon’s mt (n = 308) at three time periods, 8 weeks apart predictive validity, structural equation modelling establishment of nomological validity table 1. summary of the scale development process 3 scale development the study developed and validated the scale, basing on grounded theory approach and following the procedure recommended by moore and benbasat (1991). as pointed out by nayal and pandey (2020) grounded theory facilitates the domain exploration and scale development of the novel phenomenon. in study 1, we started with a review of the relevant literature followed by a qualitative data collection. then, we created an initial set of items in the item generation step. later, we conducted q-sort analysis. finally, via three empirical studies, we validated the instrument. table 1 provides an overview of the entire process. australasian journal of information systems mehdi darban 2021, vol 25, research article understanding the perception of is niche 6 3.1 study 1 preliminary qualitative data collection: focus groups since there has been limited research on niche technology, we conducted a series of interviews to explore the domain of individuals’ niche perceptions. participants were from a large midwestern university (27 students: 12 females and 15 males, median age 24 years old), and they were active users of niche technologies (e.g., augmented reality apps, researchgate, behance). at the first phase in the focus group sessions, we employed open-ended questions to capture personal description of niche from the perspective of our sample. some examples are: “how does a niche app satisfy your needs differently from a less niche one?”, “why do you feel some snss are more niche than others?” and “what does compel you to use more niche is?.” then, the researchers and assistant moderators met to discuss the transcripts to detect preliminary themes and aspects related to niche technology perceptions. it initially resulted in 25 statements, which were reviewed and reduced to 22, reflecting the 3 aspects of the perceived niche construct (7 reflecting conspicuousness, 8 reflecting uniqueness, and 7 reflecting digital extended self) (figure 1). we did not find any clear and distinct theme reflected by the eliminated three statements. therefore, we concluded that those three statements were not directly and accurately representing any aspect of the perceived niche construct. according to the results of study 1 and building on porter’s (1980) generic strategic approach and other relevant studies which we reviewed in previous sections (e.g., echols & tsai, 2005; parrish, 2010; vigneron & johnson, 2017), we conclude that niche technologies represent three dimensions of consciousness, uniqueness, and digital extended self. these three aspects refer to the main defining criterion for a niche technology, that is “specificity.” hence, the perceived is niche is defined as a user’s belief about the degree of specificity of the attributes of technology. 3.2 item generation and expert review we intended to develop a short, easy-to-administer instrument. we sought to conceptualize perceived niche as a reflective, second-order construct. it is because we were interested in a predictive and simple scale for the construct, rather than in the unique properties of its different manifestations. hence, we developed an instrument in which items reflecting the dimensions of the construct were aggregated into a single score. this conceptualization corresponds to what bagozzi and heatherton (1994) call the total aggregation model of perceptual construct measurements. formative, rather than reflective, conceptualization may cause overall scores on the scale to be psychologically unclear, and the relations between perceived niche scores and other variables become challenging to interpret (see bagozzi & heatherton,1994). besides, prior research reveals that there should be high levels of covariation across the identified dimensions of perceived niche (e.g., franke & schreier, 2008), which supports that the construct is reflective (jarvis et al., 2003). to generate the items, we followed a multistep procedure to develop the instrument (straub et al., 2004). firstly, a pool of potential items (22 items) for measuring perceived niche was created based on our qualitative focus group interviews, the definition of the construct, and extant literature. then, four is and management academics evaluated the instrument and deleted some items with poor face validity. in total, 11 items: three items reflecting conspicuousness, four items reflecting uniqueness, and four items reflecting digital extended self, were selected. australasian journal of information systems mehdi darban 2021, vol 25, research article understanding the perception of is niche 7 3.3 q-sort analysis to map items dimensions, four judges (doctoral students of is) in each round participated in two-step q-sorting procedures (moore & benbasat, 1991). in the first round, the judges could create as many groups as they want but were required to name the resulting groups. in the second round, four different judges were given the name and description of the focal construct dimensions and a fourth “too ambiguous/does not fit” category. then, they assigned the cards to those four categories. to assess construct validity, we examined the item-placement ratios, as described by moore and benbasat (1991). if an item is consistently placed into its intended category, we may be confident that it has high construct validity. to further evaluate the reliability of the sorting by the judges for each pair of judges in each sorting step, we measured their level of agreement in categorizing by calculating cohen’s kappa (1968). kappa scores higher than 0.65 are considered acceptable. 3.3.1. q-sort results in the first round, two judges created three categories, while the other two had four, resulting in an overall item placement ratio of 64% and an average kappa score of 0.63. in this round, one item was dropped from the item pool because it was found to be too ambiguous (fitting in an unintended category) by two judges. a second card sorting confirmed 10 items mapped to their corresponding dimensions of perceived niche. the overall item placement ratio within the target construct for the second round was improved to 76%, and the average kappa score increased to 0.88. thus, we concluded that the developed scale demonstrated construct validity, with a high potential for acceptable reliability coefficients. 4 assessment of the scale 4.1 research technology we conducted two longitudinal and one nomological studies to assess the developed scale. ello (studies 2 and 3) and behance (study 4), two snss, were the research technologies. based on their published manifestos, in sharp contrast to other mainstream snss (e.g., facebook), these two social networking tools do not perform data mining to turn their users into products (ello manifesto, 2014). additionally, they are designed to attract a specific segment of sns users interested in digital art. for instance, the creator of ello, paul budnitz, expressed: “we don’t want everyone on ello. that’s not what we’re building” (shea, 2018). also, they offer some features for purchase, such as the ability to have multiple profiles. hence, we argue that these two, relatively new snss, qualify as niche technologies with the potential for observing acceptable levels of variance in their users’ niche perceptions. 4.2 study 2: initial validation study this study assesses the discriminant and convergent validity of the generated measurement. we recruited 203 participants from amazon’s mechanical turk (mt), resulting in a usable sample of 180 participants. fifty six percent of the respondents were female, and the median age was 33 years old, and the median education was college degree. the participants were provided with the url of the first survey questionnaire (i.e., demographic questions). then, they were asked to create an account on ello to participate in australasian journal of information systems mehdi darban 2021, vol 25, research article understanding the perception of is niche 8 the second phase of data collection. they provided their user ids for later screening. eight weeks later, the respondents to the first survey were invited to participate in the second survey. they confirmed they had used the system. additionally, we checked the provided user ids and their activities on ello to ensure the participant had interacted with the system. we dropped 23 responses for not using the system during the previous eight weeks and data quality (e.g., speeding through a survey). then, the respondents completed the second survey, which included the developed scale of perceived niche. we used 7-point likert scales, with 1 representing “strongly disagree,” 4 “neutral,” and 7 “strongly agree.” 4.2.1 analyses and results an exploratory (principle components) factor analysis (efa) using varimax rotation in spss was conducted to assess the reliability of the scale (moore & benbasat, 1991). then, we estimated a confirmatory factor analysis (cfa) to further confirm the validity, reliability, dimensionality, and second-order factor structure of the instrument. to enhance the rigor of our assessments and ensure that items were strictly measuring niche perceptions, we included measures for related variables such as network effect (e.g., the more people use ello, the more valuable it is to users), confirmation (e.g., compared to my initial expectations, the ability of ello to improve my performance was much better than expected), task-technology fit (e.g., ello’s functions are very sufficient). conspicuousness (cn) cn1 ello is designed for an elite cluster of sns users. cn2 i use ello because it enhances my social image. cn3 ello’s content reflects its high status. uniqueness (uq) uq1 ello is distinct from other snss. uq2 ello can be used in a different way compared to other snss. uq3 there are characteristics that are specific to ello. uq4 ello’s content is different from other snss’. digital extended self (dg) dg1 i can relate ello to my identity. dg2 i can express myself fully using ello. dg3 ello allows me to express myself the way i could not by other snss. table 2. sns perceived niche instrument note: items uq3 and dg1 were dropped due to low item loadings. our factor analysis confirmed the factor structure, discriminant, and convergent validities of the instrument (see table a1 in appendix a). loadings greater than 0.707 are considered adequate (chin, 1998). two items with low loadings (one reflecting “uniqueness” loading at 0.61, and one reflecting “digital extended self,” loading at 0.54) were removed with the content validity in mind. the subsequent factor analysis indicated that the selected eight items loaded on the relevant dimensions. the cfa results of the initial 10-item and the purified 8-item measurements further supported removing those two items (see table a2 in appendix a). in table 2, we provide a complete listing of all items and their corresponding dimensions of perceived niche retained for the final version of the scale. to evaluate the second-order conceptualization, we estimated three models (table 3): unidimensional, and three freely correlated factors, and a second-order model. the results of comparisons across models indicate that the second-order model that considers sns niche as australasian journal of information systems mehdi darban 2021, vol 25, research article understanding the perception of is niche 9 a second-order construct with three reflective first-order factors better illustrates the relationships in our data (thatcher, et al., 2018). model χ 2, df cfi rmsea unidimensional (1st order factor) 415.19, 20 0.68 0.217 three freely correlated (1st order factors) 36.62, 17 0.95 0.085 second-order construct 21.87, 17 0.98 0.040 table 3. second-order model comparisons for perceived niche moreover, we assessed the reliability and average variance extracted (ave) values, following fornell and larcker (1981) (see table a3 in appendix a). the cronbach’s alpha and composite reliability of the scale exceeded the 0.70 thresholds (nunnally & bernstein, 1994). although the dimensions of perceived niche were found to be correlated, each dimension’s ave exceeded the squared correlations between all other dimensions. it indicates that dimensions form conceptually related, yet discriminant, portions of a superordinate second-order construct (polites & karahanna, 2012; thatcher et al., 2018). overall, study 2 developed an eight-item scale of perceived sns niche and provided preliminary confirmation of scale reliability and validity. we used the final version of the scale in a large-scale validation study (study 3). 4.3 study 3: validation and establishment study a validation study was conducted to confirm the measurement model. like study 2, we conducted a longitudinal (i.e., two surveys) data collection through amazon’s mt. ello was the research technology. respondents at time1 were invited to participate in the second survey. out of initial 420 participants, 353 participants provide data suitable for analysis. forty three percent of the respondents were female, and the median age was 33 years old, and the median education was college degree. 4.3.1 analysis and results the cfa results confirm that all items’ loadings on their respective constructs were greater than the suggested threshold of 0.707 (see table a2 in appendix a). the composite reliability estimates ranged from 0.83 to 0.91, while the aves ranged from 0.61 to 0.76, indicating acceptable convergent validity (bagozzi & yi, 1988). also, discriminant validity was established based on the values for the square root of ave for each construct, exceeding its correlations with other constructs in the model (chin, 1998). the measurement model offered good fit to the data (χ 2 = 39.70, df = 17, cfi = 0.98, rmsea = 0.05). 4.4 study 4: assessing the nomological net we evaluated the predictive validity of the developed perceived niche scale vis-à-vis postadoption system use. specifically, we tested the association between sns perceived niche and system continuance use intention (bhattacherjee, 2001). consistent with game theory research, the conspicuousness and uniqueness of a product can promote its value and consumers’ usage intentions (snyder & fromkin, 2012). similarly, based on belk’s (1988) theory of the extended self, ferraro et al. (2010) posit that identity-relevant products, such as snss, promote usage motivations. these arguments have been validated in the context of online shopping and reviews (e.g., lee et al., 2015). therefore, an examination of system use is relevant to further assess the predictive validity of the scale. we expect that perceived niche positively influences australasian journal of information systems mehdi darban 2021, vol 25, research article understanding the perception of is niche 10 a person’s continuance use intention, defined as whether a person plans to use a familiar technology in the future. we followed the same procedures as studies 2 and 3 in collecting our data. as discussed earlier, behance was the research technology. we added a third data collection phase, time 3 survey, to measure continuance intention adapting items from bhattacherjee (2001). moreover, we controlled for the two constructs linked to system use by including perceived usefulness, perceived ease of use, adapted from davis (1989), and measured in time 2 (see table a4 in appendix a). out of the initial 360 respondents, 308 provided usable responses to all three surveys. forty six percent of the respondents were female, median age 29 years old, and median education college degree. we employed structural equation modeling. 4.4.1 analyses and results the results for the measurement model support the validity and reliability of the measures (see table a5 in appendix a). although, the longitudinal nature of the study reduces the concern with common method bias (cmb), we statistically addressed the possible effect of cmb (sharma et al., 2009). we included an unmeasured latent method factor to cfa and let the items to load on both their theoretical constructs and the third factor (bagozzi, 2011). it resulted in only nine percent of the common variance value. we used amos 25 to test the structural model, and the results confirmed the structural model. perceived niche has significant impact on the dependent variable (β = 0.33, p < .001). this finding verifies the relevance of perceived niche to sns usage behavior and supports our argument (see figure 2). figure 2. results of model testing 5 discussion this paper is a conceptual and empirical attempt to understand users’ perceptions of is niche. the conducted four studies address the requirements to develop and validate a measurement for the perceived niche construct. specifically, we validated perceived niche against related concepts such as task-technology fit, confirmation, and network effect, in study 2. we confirmed this finding in study 3 with larger sample size. moreover, in study 4, using a different research technology, we showed that perceived niche significantly predicted australasian journal of information systems mehdi darban 2021, vol 25, research article understanding the perception of is niche 11 continuance usage intention. our results suggest that domain-specific technology niche measurement provides an acceptable level of explanatory power. our multi-stage and multi-study research design, in which each study investigated perceived technology niche scale from different aspects, should promote confidence in our findings. the results of the conducted studies supported the proposed second-order conceptualization of perceived technology niche, which reflects the robustness of our operational definition and theoretical foundations for this new construct. 5.1 theoretical implications the findings of this research contribute to the broader literature on is consumption and, in particular, to the growing literature on niche is. prior research exploring is consumption has failed to recognize the niche characteristic of technology, whereas characteristics of a system, as experienced by individuals, would potentially impact their adoption decisions (darban & polites, 2020; rogers, 2003). hence, our findings extend research on is adoption and usage by providing an updated and comprehensive investigation into the “technology niche” concept. an important contribution of this study is to delineate the domain of the technology niche. our research is the first dedicated effort to propose and validate a multidimensional scale of is niche. the adopted multi-study design should inspire confidence in our findings. we conceptualized the perceived niche as a phenomenon with three latent dimensions reflecting on interpersonal-oriented perceptions, that is, conspicuousness and uniqueness, and on personal-oriented perception, or the digital extended self. each of the three empirical studies supported our subordinate second-order conceptualization of the technology niche. it illustrates the robustness of our operational and theoretical descriptions of this new construct and demonstrates the implication of the findings to future research. it is important to note that the perception of niche dimensions discussed and identified here represents a comprehensive set of factors that may influence individuals’ is behaviors (e.g., adoption and usage). moreover, we contribute to theory by developing a scale that potentially will assist researchers in investigating the role of niche perceptions in technology, and more specifically, in the snsusage settings. the forthcoming demise of some mainstream snss and recognition of more niche systems (e.g., flixster and letterboxd), is a phenomenon, which is worthy of further examinations via measuring the effect of relevant variables, such as “perceived niche.” due to the advent of such niche snss, it would be interesting to examine the dynamics of individuals’ adoption of those technologies. to the best of our knowledge, niche technology usage behaviors of individuals (e.g., adoption and post-adoption outcomes) have not been studied in prior is research. by developing and validating a short, but accurate, measurement, this study provides the is researchers with a usable and suitable tool for further exploration of the implications of the niche concept for technology use. finally, the developed scale has the potential to further increase the theoretical and practical understanding of the influence of technological characteristics, that is, the degree of technology niche, on an adopter’s is usage behavior. in other words, theoretical models aimed at understanding why people prefer niche systems, rather than mainstream alternatives, can be empirically analyzed through the developed scale. 5.2 practical implications the perceived technology niche instrument is a valid and reliable tool for measuring the influence of this neglected, however emerging, technology attribute on user’s interaction with australasian journal of information systems mehdi darban 2021, vol 25, research article understanding the perception of is niche 12 a system. the scale introduced in this study would allow system designers to measure the niche-ness of their current or prototype products. armed with an enhanced understanding of how their systems are being perceived by the users, they could revise their prototype system or adopt more effective marketing actions. for instance, we showed in study 4 that perceived niche is positively associated with continuance intention; therefore, by influencing how the users view the system in terms of nichness, in other words, improving the niche perceptions among the users, managers can extend the lifecycle of their systems. besides, the second-order operationalization of the construct offers a finer-grain understanding of how dimensions of technology niche shape niche-ness perception. our conceptualization of the construct suggests that system designers should be mindful in addressing all three aspects of technology niche-ness in order to position their products as a niche, rather than a mainstream system, if desired. moreover, future researchers, by incorporating the role of perceived niche in their models, may help system designers identify what characteristics of the system lead to higher levels of perceived niche and what initiatives can lead to higher perceived niche levels during the implementation of the new information systems. it is particularly important for snss, as such systems are susceptible to abandonment (samhan, 2018). hence, sns designers should demonstrate and convey its niche-ness to its consumers, effectively. 6 limitations and future research our study has several limitations. we studied perceived niche using two different snss and separate samples. future studies should consider exploring perceived niches in more complex technologies and the broader settings of social systems. further research is encouraged to assess the utility of the scale in other types of technologies with different characteristics and technology settings. for instance, our focus was on the voluntary use of a system, so results may contrast if examining mandatory technology usage, for example, as is the case with yammer, an organizational enterprise system (venkatesh et al., 2003). also, our research context was snss. therefore, the developed scale was tested on this specific type of system. future research is needed to assess the utility of the scale in other and more complex technology use contexts and software types, such as enterprise resource planning (erp) systems, including other sample populations, like working adults versus amazon’s mt. researchers should examine the implications of these boundary conditions on the power of perceived niche regarding different types of is use. another potentially fruitful avenue of research would be further examining the link between technology niche-ness and different types of post-adoptive system use, for example, abandonment, deep use, and active use. for instance, as samhan (2018) pointed out, some systems, including snss, are prone to be abandoned by their users after an initial success, due to the stress associated with using an sns (maier et al., 2015), and it would be interesting to explore how the system’s perception of niche can influence such post-adoptive outcomes. 7 conclusion our studies contribute to is research by conceptualizing “perceived niche” and establishing and validating a scale to operationalize the construct. is research on niche technologies has not emerged, despite the recent advent and popularity of niche systems. our research is the maiden study in the area of system niche consumption. we explored the concept of “niche australasian journal of information systems mehdi darban 2021, vol 25, research article understanding the perception of is niche 13 technology,” focusing on the sns context, and developed a measurement. in doing so, we integrated the domains of consciousness, uniqueness, and digital extended self with is consumption, to better illustrate the usage dynamics of niche systems. we identified the above-mentioned three dimensions of perceived niche, and, across four empirical studies, we supported this conceptualization and developed the instrument. first, we conducted qualitative preliminary focus group interviews. then, we refined the scale through two longitudinal validation studies (studies 2 and 3), which provided evidence of the scale’s reliability and validity. to assess the predictive and nomological validity of the refined scale, we conducted study 4. this research will potentially help is scholars explore the implications of the emerging and important concept of technology niche for usage behaviors. references amaldoss, w., & jain, s. (2008). research note—trading up: a strategic analysis of reference group effects. marketing science, 27(5), 932-942. bagozzi, r. p., & heatherton, t. f. (1994). a general approach to representing multifaceted personality constructs: application to state self-esteem. structural equation modeling: a multidisciplinary journal, 1(1), 35-67. bagozzi, r. p., & yi, y. (1988). on the evaluation of structural equation models. journal of the academy of marketing science, 16(1), 74-94. bagozzi, r. (2011). measurement and meaning in information systems and organizational research: methodological and philosophical foundations. mis quarterly, 35(2), 261-292. bajac, h., palacios, m., & minton, e.a. (2018). consumer-brand congruence and conspicuousness: an international comparison. international marketing review, 35(3), 498-517. belk, r. w. (1988). possessions and the extended self. journal of consumer research, 15(2), 139168. belk, r. (2018). ownership: the extended self and the extended object. in psychological ownership and consumer behavior (pp. 53-67). springer, cham. berger, j., & heath, c. (2008). who drives divergence? identity signaling, outgroup dissimilarity, and the abandonment of cultural tastes. journal of personality and social psychology, 95(3), 593-614. bhattacherjee, a. (2001). understanding information systems continuance: an expectationconfirmation model. mis quarterly, 25(3), 351-370. brewer, m. b. (1991). the social self: on being the same and different at the same time. personality and social psychology bulletin, 17(5), 475-482. brewer, m. b., & pickett, c. l. (2014). the social self and group identification. in: forgas, j. p. & williams, k. d. (eds.) the social self: cognitive, interpersonal, and intergroup perspectives. hove, east sussex: psychology press. chin, w. w. (1998). the partial least squares approach to structural equation modeling. modern methods for business research, 295(2), 295-336. australasian journal of information systems mehdi darban 2021, vol 25, research article understanding the perception of is niche 14 cho, j. y., & suh, j. (2020) spatial color efficacy in perceived luxury and preference to stay: an eye-tracking study of retail interior environment. frontiers in psychology,11 (2), 115. cohen, j. (1968). weighted kappa: nominal scale agreement provision for scaled disagreement or partial credit. psychological bulletin, 70(4), 213. darban, m., & polites, g. l. (2020). why is it hard to fight herding? the roles of user and technology attributes. acm sigmis database: the database for advances in information systems, 51(4), 93-122. davis, f. d. (1989). perceived usefulness, perceived ease of use, and user acceptance of information technology. mis quarterly, 13(3), 319-340. dogan, v., ozkara, b. y., & dogan, m. (2018). luxury consumption tendency: conceptualization, scale development and validation. current psychology,39, 934-952. echols, a., & tsai, w. (2005). niche and performance: the moderating role of network embeddedness. strategic management journal, 26(3), 219-238. ello manifesto (2014). retrieved april 20, 2018, from https://ello.co/wtf/about/ello-manifesto/ ferraro, r., escalas, j. e., & bettman, j. r. (2011). our possessions, our selves: domains of selfworth and the possession–self link. journal of consumer psychology, 21(2), 169-177. fogliatto, f. s., da silveira, g. j., & borenstein, d. (2012). the mass customization decade: an updated review of the literature. international journal of production economics, 138(1), 1425. fornell, c., & larcker, d. f. (1981). evaluating structural equation models with unobservable variables and measurement error. journal of marketing research, 18(1), 39-50. franke, n., & schreier, m. (2008). product uniqueness as a driver of customer utility in mass customization. marketing letters, 19(2), 93-107. gierl, h., & huettl, v. (2010). are scarce products always more attractive? the interaction of different types of scarcity signals with products' suitability for conspicuous consumption. international journal of research in marketing, 27(3), 225-235. jarvis, c. b., mackenzie, s. b., & podsakoff, p. m. (2003). a critical review of construct indicators and measurement model misspecification in marketing and consumer research. journal of consumer research, 30(2), 199-218. jetten, j., spears, r., & postmes, t. (2004). intergroup distinctiveness and differentiation: a meta-analytic integration. journal of personality and social psychology, 86(6), 862-879. kapferer, j. n. (2012). abundant rarity: the key to luxury growth. business horizons, 55(5), 453462. kemp, s. (1998). perceiving luxury and necessity. journal of economic psychology, 19(5), 591-606. lee, y. j., hosanagar, k., & tan, y. (2015). do i follow my friends or the crowd? information cascades in online movie ratings. management science, 61(9), 2241-2258. macmillan, d. (2012). lenovo ceo says tablets make up niche market dominated by apple. retrieved april 20, 2018, from www.businessweek.com/news/2012-01-13/ lenovo-ceo-says-tablets-make-up-niche-market-dominated-by-apple.html. australasian journal of information systems mehdi darban 2021, vol 25, research article understanding the perception of is niche 15 maier, c., laumer, s., weinert, c., & weitzel, t. (2015). the effects of technostress and switching stress on discontinued use of social networking services: a study of facebook use. information systems journal, 25(3), 275-308. moore, g. c., & benbasat, i. (1991). development of an instrument to measure the perceptions of adopting an information technology innovation. information systems research, 2(3), 192-222. nayal, p., & pandey, n. (2020). digital coupon redemption: conceptualization, scale development and validation. australasian journal of information systems, 24(2020), 1-23. nunnally, j. c., & bernstein, i. h. (1994). psychometric theory (3rd ed.). new york: mcgrawhill, inc. pantzalis, i. (1995). exclusivity strategies in pricing and brand extension. unpublished doctoral dissertation. university of arizona, tucson, az. parkinson, b., millard, d. e., o’hara, k., & giordano, r. (2018). the digitally extended self: a lexicological analysis of personal data. journal of information science, 44(4), 552-565. parrish, e. (2010). retailers’ use of niche marketing in product development. journal of fashion marketing and management, 14 (4), 546-561. patsiaouras, g., & fitchett, j.a. (2012). the evolution of conspicuous consumption. journal of historical research in marketing, 4(1), 154-176. polites, g. l., & karahanna, e. (2012). shackled to the status quo: the inhibiting effects of incumbent system habit, switching costs, and inertia on new system acceptance. mis quarterly, 36 (1), pp. 21-42. porter, m. e. (1980). competitive strategy. free press: new york. rogers, e. m. (2003). diffusion of innovations (5th ed.). free press: new york. roux, e., tafani, e., & vigneron, f. (2017). values associated with luxury brand consumption and the role of gender. journal of business research, 71, 102-113. samhan, b. (2018). revisiting technology resistance: current insights and future directions. australasian journal of information systems, 22(2018), 1-11. schaefers, t. (2014). standing out from the crowd: niche product choice as a form of conspicuous consumption. european journal of marketing, 48(9), 1805-1827. sharma r., yetton p., & crawford j. (2009). estimating the effect of common method variance: the method—method pair technique with an illustration from tam research. mis quarterly, 33(3), 473-490. shea, c. (2018). is a social network that doesn’t share user data possible? we asked someone who’s trying. retrieved from https://www.vox.com/conversations/2018/3/27/17168790/ ello-facebook-alternative-data-privacy-cambridge-analytica-deletefacebook. snyder, & fromkin, h. (2012). uniqueness: the pursuit of difference. plenum: new york, ny. straub, d., boudreau, m. c., & gefen, d. (2004). validation guidelines for is positivist research. communications of the association for information systems, 13(1), 24, 380-427 australasian journal of information systems mehdi darban 2021, vol 25, research article understanding the perception of is niche 16 tan, t. f., netessine, s., & hitt, l. (2017). is tom cruise threatened? an empirical study of the impact of product variety on demand concentration. information systems research, 28(3), 643-660. thatcher, j. b., wright, r. t., sun, h., zagenczyk, t. j., & klein, r. (2018). mindfulness in information technology use: definitions, distinctions, and a new measure. mis quarterly, 42(3), 831-848. venkatesh, v., morris, m. g., davis, g. b., & davis, f. d. (2003). user acceptance of information technology: toward a unified view. mis quarterly, 27(3), 425-478. vigneron, f., & johnson, l. w. (2017). measuring perceptions of brand luxury. in advances in luxury brand management (pp. 199-234). palgrave macmillan, cham. villanova, d. (2019). the extended self, product valuation, and the endowment effect. ams review, 9(3), 357-371. wan, x., wang, t., & wu, j. (2018). i'll follow the minority: the effects of sales level on purchase intention of self-expressive products. frontiers in psychology, 9(1), 1135-1147. wang, c., & lee, m. k. (2020). why we cannot resist our smartphones: investigating compulsive use of mobile sns from a stimulus-response-reinforcement perspective. journal of the association for information systems, 21(1), 175-200. whitehead, m. (2020). why people leave facebook – and what it tells us about the future of social media. retrieved from https://theconversation.com/why-people-leave-facebookand-what-it-tells-us-about-the-future-of-social-media-128952 appendix a table a1. items and factor loadings (study 2) cnf conspicuousness uniqueness digital extended self ttf ne cnf1 .785 .051 .111 .204 .082 .011 cnf2 .884 .018 .089 .221 .173 .030 cnf3 .870 .108 .231 .198 .103 .085 cnf4 .709 .185 .031 .201 .215 .151 nch-cn1 -.026 .859 .249 .218 -.053 .011 nch-cn2 .060 .852 .261 .235 -.087 -.041 nch-cn3 .009 .830 .311 .161 -.121 -.012 nch-uq1 .201 .234 .831 .200 -.102 -.077 nch-uq2 -.012 .261 .873 .222 -.089 .121 nch-uq4 .098 .238 .829 .207 -.101 .098 nch-dg2 .012 .183 .214 .822 -.203 .171 nch-dg3 -0.09 .267 .130 .871 .098 .113 ttf1 .085 -.025 -.037 .201 .719 .045 ttf2 .051 -.076 .121 .012 .803 .027 ttf3 .114 -.130 .091 .090 .839 .048 ttf4 .077 -.107 .120 .230 .824 .106 ttf5 .102 -.067 -.122 .201 .808 .045 ttf6 .037 -.026 .231 -.121 .785 .080 ttf7 .066 -.043 .101 .099 .739 .121 australasian journal of information systems mehdi darban 2021, vol 25, research article understanding the perception of is niche 17 ttf8 .177 -.125 -.091 .120 .723 .091 ne1 .105 -.010 .201 .056 .099 .779 ne2 .175 .051 .089 -.034 .017 .711 ne3 .046 -.103 -.100 .089 .031 .841 ne4 .058 -.067 -.009 .209 .038 .805 ne5 .191 -.250 -.121 .221 .090 .735 notes: principal components with varimax rotation analysis conducted using spss 16.0 cnf: confirmation; nch = perceived niche, cn = conspicuousness, uq = uniqueness, dg = digital extended self, ttf = task-technology fit; ne = network effect. table a2. item loadings, means and standard deviations item study 2 study 3 standardized item loading mean s.d. standardized item loading mean s.d. nch-cn1 0.86 4.54 1.83 0.83 5.02 1.10 nch-cn2 0.88 4.81 1.97 0.87 4.78 1.05 nch-cn3 0.87 4.37 1.79 0.81 5.01 0.99 nch-uq1 0.83 4.49 1.74 0.87 4.87 1.61 nch-uq2 0.92 4.50 1.38 0.87 4.91 1.32 nch-uq4 0.86 4.71 1.77 0.76 5.03 1.06 nch-dg2 0.81 4.02 1.43 0.83 5.11 1.31 nch-dg3 0.80 4.30 1.40 0.77 4.70 1.20 notes: nch = perceived niche, cn = conspicuousness, uq = uniqueness, dg = digital extended self table a3. construct means, standard deviations, reliabilities, and correlations (study 2) construct mean s.d. ave cr cronbach’s alpha 1 2 3 construct mean s.d. ave cr cronbach’s alpha 1 2 3 1. nch-cn 4.35 1.23 0.76 0.90 0.89 0.87 2. nch-uq 4.54 1.15 0.71 0.91 0.89 0.53 0.84 3. nch-dg 5.07 1.30 0.62 0.83 0.79 0.47 0.45 0.79 notes: ave = average variance extracted, square-roots of the aves appear in bold along the diagonal of the correlations. nch = perceived niche, cn = conspicuousness, uq = uniqueness, dg = digital extended self table a4. item measures (study 4) construct & item mean s.d. standardized item loading perceived niche: conspicuousness (cn)a cn1 behance is designed for an elite cluster of sns users. 4.98 1.87 0.84 cn2 i use behance because it enhances my social image. 4.43 2.00 0.88 cn3 behance’s content reflects its high status. 4.27 2.01 0.86 perceived niche: uniqueness (uq)a uq1 behance is distinct from other snss. 4.15 1.85 0.72 uq2 behance can be used in a different way compared to other snss. 4.46 1.71 0.89 uq4 behance’s content is different from other snss. 4.44 1.68 0.88 perceived niche: digital extended self (dg)a dg2 i can express myself fully using ello. 4.11 1.46 0.88 australasian journal of information systems mehdi darban 2021, vol 25, research article understanding the perception of is niche 18 dg3 behance allows me to express myself the way i could not by other snss. 4.30 1.42 0.87 continuance intentionb con1 i intend to continue using ello rather than discontinue its use. 5.05 1.23 0.78 con2 my intentions are to continue using ello rather than use any alternative means. 4.98 1.50 0.86 con3 if i could, i would like to discontinue my use of ello.c 4.82 1.39 0.77 perceived ease of usea peu1 my interaction with ello is clear and understandable. 3.78 2.94 0.82 peu2 interacting with ello does not require a lot of mental effort. 3.45 1.55 0.73 peu3 i find it easy to get ello to do what i want it to do. 3.52 2.11 0.91 peu4 i find ello to be easy to use. 3.29 1.88 0.81 perceived usefulnessa pu1 ello helps me to accomplish tasks more quickly. 3.78 1.68 0.73 pu2 ello improves the quality of the tasks i do. 3.75 1.85 0.79 pu3 ello gives me greater control over my tasks. 3.81 1.77 0.85 pu4 ello enhances my effectiveness in my tasks. 3.89 1.66 0.83 notes: acollected in time 2 bcollected in time 3 creverse coded items were measured on a 1 = strongly disagree, 7 = strongly agree format table a5. avs, reliabilities, and correlations (study 4) item mean s.d. cr ave 1 2 3 4 5 6 1. nch-cn 4.26 1.23 0.90 0.75 0.86 2. nch-uq 5.80 1.31 0.87 0.70 0.53 0.83 3. nch-dg 4.83 1.09 0.87 0.78 0.50 0.56 0.88 4. con 5.49 1.23 0.80 0.57 0.39 0.09 0.37 0.76 5. peu 4.81 1.40 0.90 0.70 0.08 0.17 0.16 0.37 0.83 6. pu 4.77 1.10 0.86 0.62 -0.02 -0.10 0.05 0.31 0.51 0.79 notes: the correlations between all pairs of first-order constructs of perceived niche fall below 0.90 (bagozzi et al. 1991), indicating distinct dimensions. ave = average variance extracted square-roots of the aves appear in bold along the diagonal of the correlations. nch = perceived niche, cn = conspicuousness, uq = uniqueness, dg = digital extended self, con = continuance intention, peu = perceived ease of use, pu = perceived usefulness. measurement model fit statistics chi-square 196.40 with d.f. 137, cfi = .98, rmsea .38 copyright: © 2021 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems mehdi darban 2021, vol 25, research article understanding the perception of is niche 19 doi: https://doi.org/10.3127/ajis.v25i0.2951 1 introduction 2 background: perceived niche 2.1 perceived niche dimensions 2.2 conspicuousness 2.3 uniqueness 2.4 digital extended self 3 scale development 3.1 study 1 preliminary qualitative data collection: focus groups 3.2 item generation and expert review 3.3 q-sort analysis 3.3.1. q-sort results 4 assessment of the scale 4.1 research technology 4.2 study 2: initial validation study 4.2.1 analyses and results 4.3 study 3: validation and establishment study 4.3.1 analysis and results 4.4 study 4: assessing the nomological net 4.4.1 analyses and results 5 discussion 5.1 theoretical implications 5.2 practical implications 6 limitations and future research 7 conclusion paper title (paper title style) australasian journal of information systems mirkovski, doyle & gaskin 2025, vol 29, research article practitioners’ use of information systems research received on 02-feb 2024, accepted on 22 december 2024 after 2 rounds of review with karlheinz kautz as accepting editor. 1 investigation of practitioners’ attitudes and intentions to use information systems research: a mixed method study of the research format kristijan mirkovski deakin business school deakin university 221 burwood highway burwood, melbourne, australia email: kmirkovski@deakin.edu.au cathal doyle school of information management victoria university of wellington rutherford house, 23 lambton quay wellington, new zealand james gaskin information systems department byu marriott school of business brigham young university provo, utah, usa abstract balancing research relevance and rigour has been a long-standing concern in the information systems (is) discipline, crucial for advancing academic understanding and disseminating knowledge to practitioners. this paper addresses the need for effective knowledge product translation and dissemination by proposing the use of animated research briefs with visual storytelling cues. grounded in the cognitive theory of multimedia learning and critical narrative theory, these briefs aim to make research more accessible and engaging for is practitioners. a mixed-method approach is deployed: (1) a survey-based experiment to assess how research format influences practitioners' attitudes, which in turn affect their intentions to use academic research to solve work-related problems in the near term, and (2) in-depth interviews to explore practitioners’ perceptions of different research formats and their impact on the intention to use academic research in the near term. the paper offers new insights into the effectiveness of multimedia formats in enhancing practitioner engagement and understanding, thereby fostering a more effective dissemination and translation of research knowledge for practitioner audiences. keywords: research relevance, research knowledge dissemination, research knowledge translation, is practitioners. 1 introduction balancing research relevance and rigour has long been a topic of discussion in the information systems (is) academic community (dennis, 2001; lee et al., 2021; rosemann & vessey, 2008). as an academic discipline, is research seeks to advance scholarly understanding and disseminate knowledge to practitioners such as systems and business analysts, it consultants mailto:kmirkovski@deakin.edu.au australasian journal of information systems mirkovski, doyle & gaskin 2025, vol 29, research article practitioners’ use of information systems research 2 and advisors, is developers and engineers, is security and compliance specialists, and is executives, who are often informed by applied research. dennis (2001) emphasised that creating new research knowledge and disseminating it to practice should coexist and not be viewed as substitutes. disregarding research relevance during knowledge creation can cause the original purpose of is research to be lost or diverged. therefore, balancing research relevance with knowledge creation is crucial to fostering both innovative advancements and implications in the is discipline. benbasat and zmud (1999) identified four key characteristics of is research relevance linked to its content (i.e., interesting, applicable, and current) and style (i.e., accessible). they proposed four critical questions for evaluating the relevance of published research content: (1) does it address pertinent concerns (interesting)? (2) can it be utilised (applicable)? (3) is it timely (current) for practitioners? and (4) can practitioners understand the research content (accessibility)? when these characteristics are met, research output is considered to have potential relevance for practitioners (benbasat & zmud, 1999). subsequent examinations of research relevance have further refined this construct. for example, klein et al. (2006) consolidated the characteristics into three broader ones: importance, accessibility, and applicability. rosemann and vessey (2008) further introduced suitability, focusing on the fit of research to meet the needs of practice. although research relevance has attracted much attention since the early 2000s, recent work highlighted the ongoing challenge of balancing rigour and relevance due to unclear conceptualisation and a lack of comprehensive, actionable insights. in a review of the practical relevance in the is discipline, moeini et al. (2019) shifted from merely examining the critical characteristics of relevance to eliciting four specific perspectives: (1) potential practical relevance of research, (2) perceived research relevance by practitioners, (3) use of research in practice, and (4) realised research impact. this updated notion emphasises that research knowledge, including its content and style, is not the sole determinant of relevance. relevance should not be seen as unidirectional but should consider the perspectives of both researchers and practitioners (fox & groesser, 2016). importantly, moeini et al. (2019) argued that researchers can control determining the potential practical relevance of their work by focusing on four key dimensions: (1) topic selection, (2) knowledge product creation, (3) knowledge product translation, and (4) knowledge product dissemination. effective knowledge product translation involves communicating research outcomes rather than processes and tailoring messages to practitioners’ needs. knowledge product translation can be achieved through offering contextualised examples, avoiding academic jargon, and creating compelling storylines (moeini et al., 2019). knowledge product dissemination involves publishing in practitioner-focused journals, leveraging digital technologies, and using interactive formats like videos and blog posts (moeini et al., 2019). despite the focus on improving knowledge product translation and dissemination approaches, gaps remain in actionable recommendations for researchers and the effectiveness of alternative formats for translating and disseminating research. for example, toffel (2016) and moeini et al. (2019) suggested leveraging blog posts and videos as concise and interactive formats to disseminate knowledge products in a manner that is more comprehensible for practitioners but did not offer any explicit guidance on designing these formats effectively. hence, this paper builds on the extant efforts advocating for the dissemination of is research through multimedia-enabled formats (e.g., gaskin et al., 2016; hull et al., 2019; mirkovski et al., 2019) to develop theoretically driven, australasian journal of information systems mirkovski, doyle & gaskin 2025, vol 29, research article practitioners’ use of information systems research 3 yet actionable recommendations for knowledge product translation and dissemination based on empirical evidence. in this paper, we propose using animated research briefs with visual storytelling features to impact potential research relevance. animated research briefs are short videos using animated graphics, characters, and text to enhance accessibility compared to conventional text-based reports (avital et al., 2017). visual storytelling conveys research knowledge through narratives and visual elements, facilitating comprehension and engagement (williams, 2019). our approach is grounded in the cognitive theory of multimedia learning (ctml) (mayer, 2005) and critical narrative theory (cnt) (voithofer, 2004) based on which we develop an immersive, animated research brief that is a particularly suitable format for communicating practice-oriented academic research to practitioners who are not accustomed to reading academic manuscripts (nunamaker jr et al., 2015). we then aim to assess the effectiveness of this format with practitioners by examining their attitudes and short-term intentions to use academic research knowledge in their work (i.e., apply research findings to their work in the near future). we focus on short-term intention to use in order to assess specific and immediate applicability, rather than uncertain potential applicability. in this paper, we employ a mixed-method approach. study 1 conducts a survey-based experiment to investigate the impact of research format on practitioners’ attitudes to use is academic research and the effect of these attitudes on their intentions. study 2 involves indepth interviews with is practitioners to further explore their perceptions about traditional, text-based and multimedia-enabled research formats for conveying research knowledge and how they shape their short-term intentions to use academic work in their professional practice. accordingly, we attempt to address the following research questions: (rq1) how does the format of academic research (e.g., text-based vs. multimedia-enabled) influence is practitioners' attitudes and their intentions to use the research for solving practical business problems? and (rq2) how do is practitioners perceive text-based and multimedia-enabled formats for translating and disseminating research knowledge? by addressing these research questions, we aim to contribute to the is community’s understanding of how to further improve knowledge product translation and dissemination. the paper is structured as follows: we begin by reviewing current work on research relevance, followed by outlining our theoretical underpinnings. next, we present our research model and hypotheses. the research methodology then details the data collection and analysis. after presenting the findings of both studies, we elaborate on these findings and how they contribute to the extant work on research relevance in the discussion section. based on the findings of our work, we also provide practical recommendations for researchers about how to effectively translate and disseminate their knowledge products based on our findings. finally, the conclusion highlights the paper's limitations and suggests avenues for future research. 2 background work achieving a balance between research relevance and rigour has been an ongoing debate in our discipline (dennis, 2001; lee et al., 2021; rosemann & vessey, 2008). while research relevance is often linked to securing funding (mcclements et al., 2024), there is disagreement about whether the primary purpose of research is to produce knowledge that is directly applicable to practical problems. some researchers argue that the primary goal of academia is to advance australasian journal of information systems mirkovski, doyle & gaskin 2025, vol 29, research article practitioners’ use of information systems research 4 theoretical knowledge, regardless of its immediate practical implications (van de ven, 2007). concerns about securing funding from practitioners are also not universal. some researchers worry that relying on practitioner funding could compromise the independence and freedom of academic inquiry, especially in the social sciences (bourdieu, 1988). these concerns have led to funding models in some business schools where grants are primarily obtained from government sources rather than the industry, to preserve research integrity and independence (etzkowitz & leydesdorff, 2000). importantly, dennis (2001) emphasised that generating new research knowledge and making it relevant to practitioners are both crucial elements of research relevance and should not be considered mutually exclusive. seminal work identified four key characteristics of is research relevance based on two perspectives: content (interesting, applicable, and current) and style (accessible) (benbasat & zmud, 1999). klein et al. (2006) then built upon the work of benbasat and zmud (1999) to consolidate the four characteristics of research relevance into three more broad ones— importance, accessibility, and applicability—with importance signifying content that is currently of interest to the real world. further to this work, rosemann and vessey (2008) introduced the characteristic of suitability emphasising the fit to meet the needs of practice, and together with importance serving as critical steps before considering applicability in practice. in a more recent work, moeini et al. (2019) highlighted that the ongoing challenge of balancing rigour and relevance stems from unclear conceptualisation and a lack of comprehensive, actionable insights on how to address specific aspects of relevance. they examined four specific perspectives that guide the progression of achieving research relevance: (1) potential practical relevance of research, (2) perceived research relevance by practitioners, (3) use of research in practice, and (4) realised research impact. moeini et al. (2019) proposed that researchers can enhance the potential practical relevance of their work by focusing on (1) topic selection, (2) knowledge product creation, (3) knowledge product translation, and (4) knowledge product dissemination. to gain a better understanding of the current state of the literature, we reviewed and categorised existing studies on enhancing research relevance. we mapped these studies and their proposed measures across the four dimensions identified by moeini et al. (2019), as detailed in table 1. the dimension of topic selection has received significant scholarly attention. when selecting topics with potential value for practitioners, current work proposed specific measures that individual researchers should consider aligning their perspectives with the ones of practitioners (moeini et al., 2019). these measures include establishing industry sponsorships or collaborations (moeini et al., 2019; pearson et al., 2005; robey & markus, 1998; toffel, 2016; van de ven & johnson, 2006), conducting regular surveys with it executives to identify current issues and provide research guidelines (srivastava & teo, 2005), and placing researchers in industry settings to gain exposure to professional experiences (gill & bhattacherjee, 2009; toffel, 2016). further measures have been proposed to enhance the dimension of knowledge product creation in which the practice orientation is central. some of these measures include adopting research models that integrate applied theory with evaluation or policy research (robey & markus, 1998), designing studies that accommodate the complex nature of practical applications (fox & groesser, 2016; van de ven & johnson, 2006), and co-producing knowledge with practitioners throughout all stages of the research process (moeini et al., 2019; rosemann & vessey, 2008). australasian journal of information systems mirkovski, doyle & gaskin 2025, vol 29, research article practitioners’ use of information systems research 5 the dimension of knowledge product translation highlights effective communication on the contextualised usability of research knowledge with practitioners (moeini et al., 2019). the proposed measures for this dimension are twofold, addressing content and style of translation. from a content perspective, one of the key emphases to enhance potential research relevance is to focus on communicating research outcomes rather than research processes, such as dedicating and extending a practical implication section (robey & markus, 1998). current work suggested that the implications derived should be tailored to more detailed segments of practitioners to meet their diverse needs (fox & groesser, 2016; gill & bhattacherjee, 2009). moeini et al. (2019) further emphasised the importance of leveraging contextualised examples when adapting the conceptual knowledge for practitioners and associating the research knowledge with usable solutions in practice. from another perspective, considerations have been extended to the communication styles while translating research knowledge. robey and markus (1998) argued that academic reports indeed require rhetoric but still need a compelling storyline that is novel, critical, and constructive to motivate practitioners to continue reading and signal usability. gill and bhattacherjee (2009) further recommended tailoring research messages into stories that align with practitioners’ knowledge structures, cognitive schemas, and communication patterns, which aims at mitigating the inherent differences between researchers and practitioners and facilitating more effective communication. the dimension of knowledge product dissemination refers to the effective transmission of research findings to the targeted audience (moeini et al., 2019). although researchers have limited control over the low engagement of practitioners with senior is journals and conferences, adopting practice-oriented formats and mediums has been identified as a means to enhance dissemination within the researchers' purview. for example, current work has recommended publishing in practitioner-focused journals and conferences, as well as in news columns (gill & bhattacherjee, 2009; pearson et al., 2005; toffel, 2016). toffel (2016) and moeini et al. (2019) advocated for leveraging dominant digital technologies, such as social media, as dissemination platforms, which are considered more accessible channels for practitioners. it was further emphasised that consumable reports are not the only effective formats and more concise and interactive formats, such as videos and blog posts, could also be employed to convey knowledge in a manner that is more comprehensible for practitioners. based on the synthesis of proposed measures for addressing research relevance, we identified two gaps in the current work. first, although considerable attention has been devoted to improving the style of translating research knowledge, there is a notable lack of effort in identifying actionable recommendations for researchers. for example, when tailoring communication messages to align with practitioners’ communication patterns, existing measures primarily focused on the elements to omit, such as statistical methods, testing procedures, and academic jargon (gill & bhattacherjee, 2009; robey & markus, 1998; toffel, 2016). minimal guidance has been suggested on the elements and techniques required to effectively adapt to a more comprehensible style or to develop an engaging storyline for practitioners. second, there is limited work on the alternative dissemination formats and how to effectively create content for these emerging formats. as well, there has been little effort to examine the effectiveness of these alternative dissemination formats compared to traditional ones. for example, current work suggested leveraging blog posts and videos, which differ significantly from traditional text-based reports, as effective means for communicating research australasian journal of information systems mirkovski, doyle & gaskin 2025, vol 29, research article practitioners’ use of information systems research 6 knowledge. nevertheless, there remains a lack of explicit guidance on how to design and develop these visual-oriented formats effectively. moreover, even if researchers successfully create relevant knowledge based on guidance from current work, the absence of actionable measures to ensure practitioners' comprehension and accessibility may still hinder its full potential for relevance in practice. to address these gaps, we develop theoretically driven yet actionable recommendations for the dimensions of knowledge product translation and dissemination. we extend existing measures by incorporating animated research briefs with visual storytelling cues to enhance the potential research relevance of the created knowledge. the content and style of our animated research briefs are grounded in theoretical frameworks, respectively the ctml and cnt. importantly, adhering to the notion of research relevance proposed by moeini et al. (2019), this paper moves beyond merely framing recommendations for promoting potential research relevance. we consider a broader scope of research relevance by examining the effectiveness of our proposed approach with practitioners, encompassing the downstream perspectives of how practitioners perceive the potential relevance and their attitudes and intentions to adopt the research knowledge. dimension measures for addressing potential research relevance topic selection • pursuing practitioner sponsorship (robey & markus, 1998) • focus on the interests, needs, concerns and/or questions of practitioners (pearson et al., 2005; robey & markus, 1998; rosemann & vessey, 2008; srivastava & teo, 2005; van de ven & johnson, 2006) • developing working and/or professional relationships with is/it practitioners (pearson et al., 2005) • placing researchers in practice (gill & bhattacherjee, 2009) • working as a practitioner (toffel, 2016) • attending crossover workshops (toffel, 2016) • co-authoring with practitioners (toffel, 2016) knowledge product creation • adopting new models of research (applied theory, evaluation research, policy research) (robey & markus, 1998) • designing the research project to be a collaborative learning community (van de ven & johnson, 2006) • designing the study for an extended duration of time to building closer relationships and collaborations (van de ven & johnson, 2006) • employing multiple models and methos to study the complex reality (van de ven & johnson, 2006) • conducting applicability checks as an integral part to the entire research process for ensuring interests and importance to practitioners (rosemann & vessey, 2008) • relating new research findings to the prior, and to extant theory, to provide coherent (fox & groesser, 2016) • developing a sounding board of practitioners (toffel, 2016) • co-authoring with practitioners (toffel, 2016) • conducting field visits and practitioner interviews (toffel, 2016) knowledge product translation • producing reports that are shorter and more concise (robey & markus, 1998) • creating novel critical and constructive storyline (robey & markus, 1998) • ensuring creditable evidence (robey & markus, 1998) • writing in a different style and avoiding academic jargon (pearson et al., 2005; toffel, 2016) • avoiding long literature reviews, extended explanations of methodologies, and abstract statistical analysis (gill & bhattacherjee, 2009; pearson et al., 2005; robey & markus, 1998) australasian journal of information systems mirkovski, doyle & gaskin 2025, vol 29, research article practitioners’ use of information systems research 7 • developing different access, perceptual filters for different audiences (fox & groesser, 2016; gill & bhattacherjee, 2009; robey & markus, 1998) • clarifying what to do and how to apply the findings for addressing issues in practice (fox & groesser, 2016) knowledge product dissemination • submitting for practitioner-oriented journals (gill & bhattacherjee, 2009; pearson et al., 2005; robey & markus, 1998; toffel, 2016) • presenting to practitioners at conferences (toffel, 2016) • disseminating in newspaper op-eds, blogs and social media (moeini et al., 2019; toffel, 2016) • encouraging other to convey to practitioners (moeini et al., 2019; toffel, 2016) table 1. summary of proposed measures for addressing potential research relevance based on the four dimensions by moeini et al. (2019) 3 theoretical underpinnings in our pursuit of identifying effective and actionable measures for addressing research relevance beyond topic selection and knowledge creation, we ground this paper in the fundamental cognitive theories that help us with aligning the translation and dissemination of research knowledge with human cognitive structures, particularly taking into account practitioners’ communication styles and patterns. in the is discipline, leveraging multimedia, particularly video formats, to complement traditional text-based reports has been widely discussed and underscored with additional values such as open access and enhanced visibility (gaskin et al., 2016). building on this foundation, we employ ctml (mayer, 2005) and cnt (voithofer, 2004) to guide the development of animated research briefs with visual storytelling cues for communicating research knowledge with is practitioners. ctml is recognised as a comprehensive framework that underscores the value of integrating text and visuals, leveraging multimedia technologies to enhance knowledge comprehension and learning outcomes (mayer, 2005). ctml is underpinned by three core assumptions (mayer, 2005): (1) limited capacity, which posits that humans have a restricted capacity in working memories for processing current information; (2) active processing, which asserts that meaningful learning can only be achieved when learners are involved in the active selection, organisation and interaction with content and information; (3) dual channel, which suggests that learners are engaged in two distinct channels for processing verbal (e.g. words) and visual (e.g. images) information respectively. following the three assumptions, ctml outlines ten principles—coherence, signalling, redundancy, spatial contiguity, temporal contiguity, segmenting, pre-training, modality, and personalisation—that inform the design of multimedia instructional materials to optimise learning based on how individuals process information from both visual and auditory channels (mayer, 2008). ctml and its evidence-based principles have been widely served as guidelines for designing educational materials in research, tailored for specific user groups such as students with learning disability (greer et al., 2013) and for specific learning tasks such as foreign language vocabulary acquisition (immanuel & a, 2023) or assessment (kirschner et al., 2023). existing work in is has also adopted the ctml to design videos that effectively support is knowledge education, translation and dissemination. hull et al. (2019) and mirkovski et al. (2019) outlined the potential of ctml-based visual storytelling in enhancing the comprehension and perceived utility of research knowledge among students and academics, thereby positively affecting their intentions to cite, share, and accept is academic research. moreover, ctml and its principles have been employed in designing animated videos to support business australasian journal of information systems mirkovski, doyle & gaskin 2025, vol 29, research article practitioners’ use of information systems research 8 stakeholders in comprehending complex artefacts and dynamic business process models, which illuminates the opportunity of leveraging ctml-based aminated content to effectively accommodate the diverse cognitive needs and knowledge expertise levels when communicating with practitioners (aysolmaz & reijers, 2021). therefore, we adopt ctml and its multimedia design principles to inform the design of the animated research briefs, and to improve the translation style and dissemination format of research knowledge for is practitioners. we provide brief definitions of the ten ctml principles in table 2. robey and markus (1998) argued that a compelling storyline is also crucial for maintaining practitioner engagement with research knowledge. given the minimal measures for constructing an effective storyline, we integrate cnt to guide the design of narratives and structures when translating research knowledge. cnt posits that story structures are not static, implying that stories evolve over time as social, cultural, historical and technical factors (e.g., media convergence) emerge, change, dissipate or merge (voithofer, 2004). cnt provides a conceptual framework for designing and evaluating multimedia materials, explicitly focusing on eight elements of the story structure: genre, story, plot and subplot, space, place and settings, time, charter and characterisation, point of view/focalisation, complication/crisis and resolution and coda. in this paper, cnt offers an additional theoretical perspective to enhance the multimedia learning experiences by optimising the story structure as well as translating the research knowledge specific to the context and communication patterns of is practitioners. cnt has been extensively employed in improving the delivery structure and style of educational materials (diamond, 2011). extant work has leveraged cnt to improve the course structure for delivering ill-defined knowledge to engineering students (mccarthy, 2012). by enabling students to impose their own cognitive order on the course materials, the cnt-based intervention has significantly influenced students’ engagement and motivation in addressing engineering problems and deriving solutions for given assignments. furthermore, mirkovski et al. (2019) also discussed that future research should adhere to evidence-based guidelines, such as principles outlined in cnt, to effectively communicate the story's meaning, rather than relying on mere paraphrasing, when conveying is research knowledge through visual storytelling. we adopt cnt and its principles to inform the content of the animated research briefs, enhancing the storytelling aspect. as a result, the underlying is research knowledge can be perceived as more comprehensible by the targeted audience. we provide explanations and examples of the eight cnt principles in table 2. in our paper, we propose to integrate cnt with ctml to develop animated research briefs to complement the existing textual-based research articles for translating and disseminating is research knowledge. we propose that animated research briefs designed according to the ten principles of ctml for multimedia instructional materials and the eight elements of cnt for story structure would further encourage is practitioners to perceive the disseminated research knowledge as being engaging, relatable, and impactful to their work. we thus develop an animated research brief with a design and a story informed by ctml and cnt for a published article by ebel et al. (2016)1 to improve the dissemination and translation of research 1 readers can access the animated research brief at the following link: https://www.youtube.com/watch?v=0whqxvatca (10:21). we selected ebel et al. (2016) because their framework provides valuable support for developing and managing new business models, which is particularly relevant for is practitioners in the it industry focused on product and market development. the framework's systematic methodologies help roles like product owners, it developers, and business analysts create innovative, scalable it solutions that align with evolving business strategies and drive future growth. https://www.youtube.com/watch?v=0whqx-vatca https://www.youtube.com/watch?v=0whqx-vatca australasian journal of information systems mirkovski, doyle & gaskin 2025, vol 29, research article practitioners’ use of information systems research 9 knowledge for is practitioners. in table 2, we outline how our two theoretical foundations were applied in developing the animated research brief. refer to appendix 1 for more detailed information about the animated research brief. theoretical foundation principle definition application ctml coherence reduce use of irrelevant material. the five activities of business model development projects were highlighted using voice narration, visual graphs, and printed text. signalling highlight essential material. important information about each of the five activities of business model development projects was presented using a combination of narrative voice and visual graphics. redundancy reduce (if possible) onscreen text to narrated animation only. capturing the voice narration in the form of on-screen text (e.g., subtitles) was included, despite the suggested omission, as some of the learners might not be native english speakers. spatial contiguity place corresponding text and graphics in proximity. on some occasions, highlighted printed words were appropriately positioned in the proximity of graphics. temporal contiguity present corresponding narration and animation concurrently. visual graphics and voice narration were synchronised. segmenting present animation in learner-paced segments. the video narrative was divided four sections that included 53 viewer-paced segments, each consisting of two to three sentences with a duration of 12 to 15 seconds. pre-training provide pre-training of essential material for learners. a brief overview of the selected article was provided at the beginning of the video narrative, mentioning its key research objectives. modality present words as narration instead of text. visual graphics were guided with narrative voice rather the printed words. multimedia present words and graphics together rather than words separately. visual graphics and narrative voice were used simultaneously. personalisation present words in conversational style rather than formal style. a more personalised and less formal tone for the narrative voice was used.2 cnt genre the category or type of narrative with specific conventions and expectations. the script was presented in a casual, yet professional settings using a young practitioner, named steve, as a virtual tour guide. story, plot, and subplot the central narrative, additional interconnected narratives, and the main storyline’s complexity. the script followed the structure of the dsr framework, including four arc-story sections: introduction, motivation and research approach, problem formulation, and iteration cycles and research output. 2 in our attempts to use a more personal and less formal language, we were cautious not to alter the original and intended meaning; thus, we restrained ourselves from overly using buzzwords and acronyms. australasian journal of information systems mirkovski, doyle & gaskin 2025, vol 29, research article practitioners’ use of information systems research 10 space, place, and settings the physical and geographical aspects, including locations and environments. simulation of multiple virtual spaces (e.g., office meetings) were used. time the sequencing, pacing, and chronological order of events in the narrative. various visual graphics were used to illustrate overlapping events with various meanings and interpretations (e.g., individual processes of the proposed framework). character and characterisation the central figures in the story, their traits, motivations, and development. a virtual tour guide, steve, who was characterised as a young practitioner, led learners through the video narrative. point of view /focalisation the perspective from which the narrative is narrated, such as first-person, thirdperson, or omniscient. the script was narrated by a virtual tour guide, who discussed the key points from the perspective of practitioners. complication/ crisis the turning point introducing conflict and tension in the narrative. the script was aligned with the structure of the proposed framework, including problem-cantered instructions (e.g., problem identification and solution development) in the four sections. resolution and coda the point where conflicts are resolved and the narrative’s conclusion or reflection. in the last section (i.e., iteration cycles and research output), visual graphics are used to illustrate the proposed framework for designing business model development tools. table 2. definition and application of ctml’s ten principles and cnt’s eight guidelines, adapted from mayer (2008) and voithofer (2004) 4 research model and hypotheses development based on the theoretical underpinning of ctml and cnt, we develop our research model, as illustrated in figure 1. while the potential relevance in is academic research needs to be upheld by the researchers themselves, it is also imperative to assess whether the potential relevance can be perceived and embraced by practitioners (moeini et al., 2019). we conduct two studies to address these aspects. in study 1, we investigate how the format of research affects practitioners' attitudes to use is academic research (h1) and how these attitudes before (h2a) and after consumption (h2b) of the academic research influence their short-term intentions to use it. in study 2, we explore practitioners' perceptions of different research formats and their shape their short-term intentions to use academic research for their work. figure 1. research model australasian journal of information systems mirkovski, doyle & gaskin 2025, vol 29, research article practitioners’ use of information systems research 11 in this paper, we investigate the effect of theory-driven, animated research briefs by assessing the positive influence on practitioners’ attitudes, which refer to practitioners’ favourable evaluations or appraisals to use is academic research (ajzen, 1991). we focus solely on the attitude construct because our proposed measure—supplementing traditional text-based reports with animated research briefs—primarily aims to improve practitioners' internal perceptions and intrinsic motivations to use is academic research at an individual level. other factors influencing behavioural intention, such as subjective norms and perceived behavioural control, which relate to social and organisational influences (ajzen, 1991) are beyond the scope of this paper. having a positive attitude to use is academic research not only signifies a positive evaluation of its potential relevance but also indicates inner acceptance among practitioners (ajzen, 1991). this underlying acceptance is crucial because, although practitioners can acknowledge the perceived relevance of is academic research, a lack of agreement or motivation to use it diminishes the value of the potential relevance created by researchers or perceived by practitioners. importantly, we posit that ctmland cnt-informed animated research briefs can foster positive attitudes to use is academic research through three key mechanisms: (a) comprehension and retention, (b) personalisation, and (c) engagement. each mechanism may also be associated with positive appraisals, further embracing practitioners’ positive attitudes to use is academic research. we further elaborate on the actualisation of these three mechanisms in the following paragraphs. first, ctml emphasises that multimedia content enhances both comprehension and retention (mayer, 2005). by presenting is research knowledge in a manner that combines textual and visual elements following ctml design principles, animated research briefs facilitate a deeper understanding and memory. for example, as outlined by the signalling principle, animated research briefs utilise engaging narrative voice and alerting visual graphics (mayer, 2008) to highlight and explain complex system architectures, such as microservices or cloud computing frameworks, which in turn can help to demystify abstract concepts like data modelling or system integration. in addition, adhering to the principle of temporal contiguity, these briefs can present voice narration and corresponding visual graphics simultaneously to enhance the dual-channel processing of information (mayer, 2008). this approach is particularly useful for conveying detailed processes such as software development methodologies or cybersecurity protocols, ensuring that is practitioners can better understand and retain the critical information needed for practical application. hence, compared with other formats for translating and disseminating research knowledge, ctml-guided animated research briefs serve as a powerful supplement to explain complex concepts, which encourages is practitioners to evaluate and perceive the potential research relevance that researchers intend to convey. furthermore, the increase in comprehension and retention can strengthen the inner feeling of competence of is practitioners while engaging with the research content, leading to a positive inclination and appreciation for the academic research, subsequently further enhancing the positive attitudes. second, ctml and cnt also emphasise a more personalised dissemination and translation of is research knowledge. theory-driven, animated research briefs attend to the communication patterns of practitioners when presenting the research knowledge, aiming to mitigate the barriers caused by the inherent differences between researchers and practitioners (gill & bhattacherjee, 2009). for instance, in accordance with the personalisation principle outlined in australasian journal of information systems mirkovski, doyle & gaskin 2025, vol 29, research article practitioners’ use of information systems research 12 the ctml, animated research briefs employ a less formal communication style with more content tailored to specific audience groups (mayer, 2008). this could include simplifying technical jargon related to it systems, cybersecurity, or data analytics, making the information more accessible. furthermore, as suggested by the character and characterisation principle of cnt, these briefs can feature an animated narrator who embodies the role of an it professional or systems analyst, mirroring the audience's real-life roles (voithofer, 2004). this narrator can guide practitioners through complex topics such as cloud computing strategies or enterprise software deployment in a relatable manner, similar to how they learn and apply practical knowledge within their organisations. this approach not only makes the content more engaging but also helps practitioners see the relevance of the research to their everyday work. in such a way, animated research briefs do not only assist with comprehension through a more personalised translation of is research knowledge but also enable a higher sense of relatedness for practitioners compared with traditional forms of academic reports, enhancing the positive attitudes of practitioners regarding is academic research. third, ctml-based animated research briefs emphasise engagement. ctml principles, such as coherence, spatial contiguity, and multimedia, enhance engagement and reception when practitioners review is research knowledge (mayer, 2008). for example, traditional text-based research on topics, such as database management or emerging technologies, can often be dense and unengaging, leading to boredom or frustration among is practitioners. animated research briefs can alleviate this by using engaging visuals and interactive elements to illustrate complex concepts, making them more accessible and enjoyable. this approach can keep is practitioners interested and help them see the potential relevance of is academic research, fostering positive attitudes and increasing the likelihood of applying these insights in their work. in summary, we posit that: h1: a text-based article supplemented with an animated research brief will result in a larger positive change in practitioners’ attitudes to use is academic research (after consuming research) when compared to (1) an extended abstract only, (2) an extended abstract supplemented with an animated research brief, (3) an animated research brief only, or (4) a full text-based article only. positive attitudes are crucial in predicting behavioural intentions, a relationship that has been extensively discussed in is research (bhattacherjee & premkumar, 2004; karahanna et al., 1999). extant work highlights the pivotal role of positive attitudes in fostering intentions to engage in certain behaviours in the workplace (e.g., moeini & rivard, 2019; ormond et al., 2019; titah & barki, 2009). building on this understanding, we also propose that positive attitudes toward is academic research, both before and after consumption, can lead practitioners to have positive short-term intentions to use research knowledge through three key mechanisms: (a) comprehension and retention, (b) personalisation, and (c) engagement. we elaborate on the actualisation of these mechanisms in the following paragraphs. first, ctml’s modality principle highlights the importance of using multiple sensory channels (visual and auditory) for more effective learning (mayer, 2008). animated research briefs can simplify complex concepts, such as data analytics, cybersecurity protocols, or software development methodologies, by integrating text, visuals, and audio. for example, animated diagrams can break down intricate processes, such as machine learning algorithms or cloud infrastructure setups, while voiceovers can clarify key points and contextualise information. is practitioners who engage with these multimedia-enhanced materials can develop positive australasian journal of information systems mirkovski, doyle & gaskin 2025, vol 29, research article practitioners’ use of information systems research 13 attitudes as they find the content easier to understand and retain. this clear and engaging presentation helps them grasp critical information more effectively, making them more likely to incorporate is research knowledge into their professional practices, whether it involves implementing new technologies or optimising it workflows. similarly, the coherence principle of ctml ensures that materials are logically structured, making it easier for practitioners to follow and understand the content. when is practitioners perceive research materials as well-structured and easy to understand, they are more likely to develop a favourable attitude toward the research. this enhanced comprehension and retention foster a positive attitude to use is research knowledge, making practitioners more inclined to integrate this knowledge into their professional practices. second, the personalisation mechanism focuses on tailoring research materials to resonate with practitioners' unique perspectives and experiences. for instance, by incorporating elements that directly address the audience's professional context and needs, such as cybersecurity threats, efficient data management techniques, and the adoption of ai, the content becomes more relatable and engaging. this approach mirrors the cnt’s character and characterisation element (voithofer, 2004), where is practitioners are seen as central figures in the narrative of the research application. when research materials are personalised, is practitioners can see themselves reflected in the content, making it easier for them to connect with and internalise the information. this connection fosters positive attitudes to use research, as it feels more relevant and accessible. as a result, these positive attitudes increase practitioners' intentions to apply the research knowledge in their professional practices. third, the engagement mechanism is essential in motivating practitioners to use is academic research. cnt's complication/crisis element introduces turning points in a narrative, creating conflict and tension that capture the audience's attention (voithofer, 2004). for example, when is practitioners encounter challenges in the research, such as data security breaches or system downtime, their engagement increases as they seek solutions. by presenting research findings within engaging narratives that highlight these relevant issues, animated research briefs can captivate practitioners, making the research more relevant and interesting. this heightened engagement fosters positive attitudes to use research, as practitioners perceive it as directly addressing their professional challenges. consequently, these positive attitudes enhance their intention to apply the research in their work, emphasising the importance of engagement in effectively utilising is academic research. in summary, we propose that: h2a: practitioners’ attitude to use is academic research (before consumption) has a positive influence on their short-term intention to use is academic research. h2b: practitioners’ attitude to use is academic research (after consumption) has a positive influence on their short-term intention to use is academic research. 5 research method we adopted a mixed-method approach3 to comprehensively address our research questions (creswell & clark, 2018), as shown in figure 2. to address rq1, study 1 employs a survey 3 this research has received approval from the victoria university of wellington (new zealand) human ethics committee, with the reference number 0000027590. for any questions regarding the ethical aspects of this research, you may contact the human ethics committee convenor at hec@vuw.ac.nz. mailto:hec@vuw.ac.nz australasian journal of information systems mirkovski, doyle & gaskin 2025, vol 29, research article practitioners’ use of information systems research 14 based experiment to quantitatively examine (1) the extent to which research format affects practitioners’ attitudes to use is academic research and (2) the effect of practitioners’ attitudes, before and after consumption, on their behavioural short-term intentions to use is academic research. we use anova to assess the impact of research format on practitioners’ attitudes (h1) and regression analysis to investigate the influence of practitioners’ attitudes on their short-term intention to use academic research for their work (h2a and h2b). by comparing five distinct research formats—an extended abstract only (thereafter labelled as “abstract”), an animated research brief followed by an extended abstract (thereafter labelled as “abstract video”), a full academic article only (thereafter labelled as “article”), an animated research brief followed by a full academic article (thereafter labelled as “video article”), and an animated research brief only (thereafter labelled as “video”)—we identify specific influences on practitioners’ attitudes which then influence their behavioural intentions. in study 1, we further qualitatively explore practitioners’ expected use cases of academic research through thematic analysis of the open-ended questions in the survey-based experiment. to address rq2, study 2 employs in-depth interviews to qualitatively explore practitioners' perceptions of text-based and multimedia-enabled research formats for translating and disseminating research knowledge and how they shape their short-term intentions to use academic research in their work. we use thematic analysis to derive further nuanced insights into practitioners’ experiences and suggestions for improvement about the five distinct research formats, thereby enriching our understanding of how to best translate and disseminate research knowledge for practitioner audiences. by combining these quantitative and qualitative methods (johnson & onwuegbuzie, 2004), we aim to gain a more comprehensive understanding of how is practitioners engage with various research formats, potentially enhancing the validity and depth of our findings about translating and disseminating research knowledge. we provide details about the measurements, data collection, and data analysis for study 1 and study 2 in the following sections. figure 2. research method australasian journal of information systems mirkovski, doyle & gaskin 2025, vol 29, research article practitioners’ use of information systems research 15 5.1 measurement 5.1.1 study 1 the research model includes two adapted measures4. attitude to use academic research is the practitioner’s positive or negative feelings about using peer-reviewed academic research in their work practice. it was measured by attitude before and after exposing practitioners to the treatment (i.e., consumption of is research in different formats). behavioural short-term intention to use academic research is the extent to which the practitioner feels he/she would use the peerreviewed academic material for their work practice. we also measured practitioners’ specificity to use academic research, referring to the detailed and contextualised ways in which practitioners anticipate using academic research findings in their work, by collecting qualitative data in our survey-based experiment. this measurement encompasses the expected use cases and situations where they can apply these findings to solve practical problems. refer to table a1 in appendix 2 for details on the measurement operationalisation. 5.1.2 study 2 to capture the perceptions of perceptions of text-based and multimedia-enabled research formats for translating and disseminating research knowledge, this study adopted a semistructured interview protocol. interviews were informal and adaptable with prompts to gather data on a variety of issues and perspectives based on interviewees’ work tenure and occupational work experience in the it industry. additional data were also collected on practitioners’ perspectives and sentiments about using academic research in different formats, as well as their concerns and necessary improvements for translating and disseminating research knowledge to practitioner audiences. refer to appendix 2 for details on the interview protocol. 5.2 data collection 5.2.1 study 1 we used qualtrics to collect anonymous responses from highly filtered demographics for our survey-based experiment. we followed the latest procedures for effectively collecting data from third-party providers (e.g., mturk, dynata, and qualtrics), which involved using screeners and attention traps to improve the data quality (lowry et al., 2016). we outline the data collection procedures followed for our survey-based experiment in the following paragraphs. in the early stages of the study design, it was acknowledged that is practitioners have busy working schedules and therefore a financial reimbursement (au$40) for their time was deemed as necessary to increase the participation in and completion of the experiment. the participants had to meet six initial criteria: 1) work in the it industry (since this was the target population for our theoretical agenda); 2) use a tablet, laptop or desktop device to complete the experiment (rather than a mobile phone, which might inhibit proper viewing of the content); 3) be at least 18 years old (due to irb constraints and to be justifiably within our 4 we used attitude and behavioural intention measures from general technology use (davis, 1989; venkatesh et al., 2003) to assess practitioners’ attitudes and intentions to use academic articles, given their versatility and adaptability. mirkovski et al. (2019) adapted the behavioural intention variable to measure academics' intentions to cite, share, and accept research. griffioen (2019) explored undergraduates' research attitudes and intentions for future professional use, while hsiao and chen (2015) investigated physicians' intentions to use clinical guidelines. this demonstrates the applicability of the general technology use measures in diverse professional contexts. australasian journal of information systems mirkovski, doyle & gaskin 2025, vol 29, research article practitioners’ use of information systems research 16 target population); 4) have at least high school diploma (since they were being treated with scientific literature); 5) have at least one year of working experience in the it industry (again, to fall within our target population); and 6) have an it related occupational role with experience in product/market development, including product owner, it developer, product manager, software engineer, business analyst, it analyst, it project manager, solution tester, it consultant, solution architect, and other (to be a relevant target for our research question). we further screened participants to ensure they met two additional criteria: (1) whether examining and learning from academic research had been part of their job and (2) whether they had consumed the academic article before today. these screeners were important to verify that participants had relevant experience with academic research and to eliminate any prior familiarity with the specific article, ensuring unbiased responses. these screeners filtered out 382 of the 652 pre-screened participants, leaving us with 270 participants who met all the eight criteria. besides meeting the eight criteria, we used multiple attention traps to filter out unengaged participants. first, we included five objective questions about the content covered in the consumed academic article to ensure credibility and quality of responses: (1) “what are the main phases of the authors' research approach?”, (2) “which company initiated the formulation of the problem and why?”, (3) “which theory was selected as the core theory to support the framework development for digital tools in new business model design?”, (4) “what artefacts were created as part of the functional requirements of the proposed framework, and what are their objectives?”, and (5) “how was the developed framework evaluated in the second iteration cycle?” second, we employed a time-to-complete exclusion criterion by asking, “how long did it take to review the provided materials?” to filter out unengaged responses further. third, we excluded responses that completed the experiment in an unreasonably short time, based on these criteria: (1) abstract video/video article > 12 mins (exclude treatments less than 12 minutes in duration) and (2) video/abstract/article > 12 mins (exclude treatments less than 12 minutes in duration). fourth, since attitude was measured at t1 and t2 with the same questions, we inverted the scales for attitude at t2 as an attention trap. these comprehensive attention traps filtered out 40 participants, leaving us with 230 valid responses. the final set of participants in our experiment covered all three ais academic regions: australia (n=51), the us (n=62), the uk (n=53), and india (n=64). the final sample was comprised of participants with an average age of 37 years, 66% of whom were male. the majority (89%) had a bachelor’s degree or higher. the breakdown of the top 3 occupational roles was 33% project managers, 26% it developers, and 10% it/business analysts. the average number of years of experience in the it industry was 11.8 with 62% of respondents operating in senior positions (e.g., it project manager, it consultant, solution architect), 23% in (self-employed) executive positions (e.g., product manager, product owner), and 15% in junior positions (e.g., it developer, software engineer, solution tester, it analyst). in the first section of the experiment, participants answered demographic and screening questions. before being exposed to a treatment, participants were asked to report their attitude to use academic articles in their everyday professional work. in the second section, participants completed the treatment. participants were randomly assigned to one of the five treatments: 1) abstract (n=42); 2) abstract video (n=47); 3) article (n=43); 4) video article (n=47); and 5) video (n=51). in the third section, participants were asked to report their attitude and short-term australasian journal of information systems mirkovski, doyle & gaskin 2025, vol 29, research article practitioners’ use of information systems research 17 intention to use academic articles post-treatment. they were also asked to elaborate on how and in which situations they could use the findings of the article in their work. in this section, the experiment participants were also exposed to the attention traps. 5.2.2 study 2 a call for interviews was posted on various professional associations and meet-ups, including it professionals (new zealand), nztech, pmi new zealand, and so on. a call for participation, offering a financial reimbursement of nz$40, was posted on multiple university alumni associations in new zealand. the call clearly outlined our research objectives, interview expectations, and the financial compensation for participants. convenience sampling was undertaken which included utilising geographic locations and resources that make participant recruitment convenient. hence, interviews were conducted with 12 practitioners from new zealand, focusing on one academic ais region convenient to one of the authors, as depicted in table 3. the interview participants had to meet the same selection criteria and undergo screening as the ones in the survey-based experiment and were also exposed to the same set of attention traps, which in turn ensured the quality of the recruited participants. those who met the initial selection criteria, passed all the filters from the screeners and attention traps, and accepted to participate in the interview were invited to consume the selected academic research in one of the formats from the five treatments of the survey-based experiment, assigned at random. after completing this task, participants took part in a one-hour, face-to-face in-depth interview (myers & newman, 2007) to share their insights on their experiences with and suggestions for improving the different research formats, as well as how these formats impact their short-term intention to use academic research; thus, helping us develop a better understanding of how to best translate and disseminate research knowledge for practitioner audiences. we documented interviews’ observational notes within 24 hours to capture contextual details that complemented the verbal data, providing deeper insights into participants' attitudes, feelings, and reactions to the different research formats. these notes enabled us to interpret the interview responses more accurately, identify patterns or inconsistencies, and enhance the richness of our analysis (myers & avison, 2002). our data collection yielded 12 hours of indepth interviews and 1,400 words of interview notes, creating a rich qualitative dataset on how is practitioners perceive text-based and multimedia-enabled formats for translating and disseminating research knowledge. ps (sex) education degree working experience current it roles position level use of academic research treatment a (m) vocational training 16 years service delivery manager senior not at all video b (f) master’s degree or above 18 years it consultant senior to some degree video c (m) vocational training 11 years it service designer senior not at all abstract video d (f) bachelor’s degree 2 years user experience designer junior to some degree video australasian journal of information systems mirkovski, doyle & gaskin 2025, vol 29, research article practitioners’ use of information systems research 18 e (m) master’s degree or above 2 years it consultant junior to some degree video article f (m) bachelors 13 years it security consultant senior not at all article g (f) master’s degree or above 15 years business analyst senior to large extent video article h (m) bachelor’s degree 3 years business analyst junior to some degree abstract i (m) bachelor’s degree 13 years it consultant senior not at all abstract j (m) bachelor’s degree 22 years enterprise systems manager senior to some degree video k (f) bachelor’s degree 18 years hardware engineer senior to some degree abstract l (m) master’s degree or above 7 years it developer middle to some degree abstract video table 3. demographics on interview participants 5.3 data analysis 5.3.1 study 1 we conducted our analysis of the survey-based experiment data in three stages. first, we utilised anova (tabachnick & fidell, 2007) to analyse the quantitative experiment data and determine the extent to which research format influences practitioners’ attitudes to use is academic research to solve practical work-related problems. second, we applied regression analysis (cohen et al., 2002) to further analyse the quantitative experiment data and examine the effect of practitioners’ attitudes on their short-term intentions to use academic research. third, we used a thematic analysis (king et al., 2004) to analyse the qualitative experiment data and explore practitioners’ expected use cases of academic research and situations where they can apply the findings to solve practical problems. we started with open coding and allowed codes to emerge naturally from the data, which were constantly modified based on their usefulness and suitability (corbin & strauss, 2014). this step resulted in 28 descriptive first-order themes about the context in which is practitioners can use the findings of the article, including leveraging virtual business model innovations, solving issues by applying researched and validated approaches, customising and streamlining work processes, evaluating and optimising technology implementation and others. to derive conceptually more abstract themes about the use case domains, we then started integrating qualitatively similar codes into higher-level categories via data reduction (corbin & strauss, 2014). this step resulted in nine second-order aggregate themes: business model development and innovation, problem-solving and decision-making, efficiency and customisation, collaboration and teamwork, data organisation and analysis, is development and innovation, learning and skills development, application of research in practice, and customer relationship management. we have coded, analysed, and visualised the open-ended questions with nvivo™ 11. refer to table a2 in appendix 3 for details on the final coding scheme from the thematic analysis in study 1. australasian journal of information systems mirkovski, doyle & gaskin 2025, vol 29, research article practitioners’ use of information systems research 19 5.3.2 study 2 our goal was to gain a deeper understanding of the phenomenon through qualitative data, so we employed a structured yet creative approach to data analysis (miles et al., 2014). we analysed the interview data in three stages. first, we integrated all our data to create rich narratives for each research format used to convey research knowledge, with a particular focus on text-based and multimedia-enabled formats. this step was crucial not only for preparing the data for subsequent analysis but also for validating our findings by triangulating data from interviews and observational notes. it also helped us form an initial understanding of how these formats impacted practitioners’ short-term intentions to use academic research in their professional practice (miles et al., 2014). second, we employed a thematic analysis (king et al., 2004) to analyse the transcribed interviews and observational notes. we began to inductively code the different research formats with a focus on their characteristics to translate and disseminate research knowledge (corbin & strauss, 2014). our first step of the coding process was open coding, during which we allowed first-order codes to naturally emerge from the data and continuously modified, refined, and integrated them based on their usefulness and suitability. this step led to the identification of 48 relatively descriptive first-order codes about is practitioners’ perspectives of different research formats, such as article’s content does not make sense, video’s content is approachable, visual storytelling is a valuable practice, abstract’s content lacks applicability, article’s format is disengaging, and others. to develop a more abstract understanding of their perceptions of the formats and how they impact their short-term intention to use academic research, we began grouping similar codes into higher-level categories through data reduction (corbin & strauss, 2014). to ensure the internal consistency and distinctness of the emerging categories, we evaluated each of the codes not only in terms of whether it was qualitatively similar to other codes within a category but also whether it was qualitatively different from codes within other categories. this approach led to the identification of four distinct second-level themes: content, format, information processing and recall, and storytelling. we used nvivo™ 11 to code, analyse, and visualise the interview data. refer to table a3 in appendix 3 for the final coding scheme from the thematic analysis in study 2. 6 findings 6.1 study 1 6.1.1 anova this study assessed whether practitioners’ attitudes to use is academic research varied depending on the format they were assigned. to investigate these differences (rq1), we conducted a one-way anova, using the post-treatment (t2) attitude variable as the dependent variable and the research format as the grouping variable. we ensured that our data met the anova assumptions for normality and homogeneity of variance by performing a shapiro-wilk test (p = 1.48) and levene’s test (p = 0.770). a one-way anova was then conducted, followed by bonferroni pairwise comparisons to adjust for multiple comparisons. the anova did not reveal any significant differences in attitudes based on the research format (f (4, 225) = 0.819, p = 0.514). refer to tables a4 and a5 in appendix 4 for details on bonferroni pairwise comparisons. australasian journal of information systems mirkovski, doyle & gaskin 2025, vol 29, research article practitioners’ use of information systems research 20 the attitude variables measured participants' perceptions of using academic articles, including whether they viewed it as a good idea (att1), an activity they would enjoy (att2), a wise choice versus a foolish one (att3), and a pleasant activity (att4). we averaged these scores to capture practitioners' overall attitudes both before (t1) and after (t2) consuming academic research. this approach allowed us to calculate attitude change by subtracting the average score at t1 from the average score at t2, as shown in figure 3. we then conducted one-way anovas, with research format as the grouping variable and attitude change as the dependent variable, which yielded non-significant results (f(4, 225) = 1.249, p = 0.291). figure 3. mean attitude reported at t2 in conclusion, the one-way anovas did not reveal a significant impact of specific research formats on is practitioners’ attitudes towards using academic research. consequently, the hypothesis (h1), which suggested that different research formats would influence practitioners' attitudes at t2, was not supported. although the differences between conditions were not statistically significant, it is noteworthy that attitude scores decreased the least in the “abstract video” condition. there are several possible explanations for the insignificant relationship observed in this hypothesis. first, practitioners may already have strong, established views about the value of academic research, which could make their attitudes more resistant to change based solely on the format of the research presentation. these pre-existing attitudes might overshadow any potential influence of the format, making it difficult to detect significant changes. second, the novelty of new formats, such as multimedia presentations, could have led to cognitive overload, distracting practitioners and preventing them from fully engaging with the content. this distraction might have limited the format's impact on their attitudes. third, practitioners might be resistant to changing their established work practices, preferring familiar and traditional research formats. this resistance could have reduced the effectiveness of new formats in influencing their attitudes. replicating the study with a larger sample size and considering different formats or longer-term attitude assessments could help to better understand these results. fourth, this particular article might not have resonated with each scholar, when another article may have on a case-by-case basis; though we did try to mitigate australasian journal of information systems mirkovski, doyle & gaskin 2025, vol 29, research article practitioners’ use of information systems research 21 this by selecting an article we anticipated would be most relevant to those we solicited to participate in this study. 6.1.2 regression analysis this study evaluated the impact of practitioners’ attitudes on their short-term intention to use is academic research both before (t1) and after (t2) exposure to the experimental treatment. to examine the relationship between practitioners’ attitudes and their short-term intentions to use academic research in their work (rq1), we conducted a regression analysis. the results from the correlations and linear regression showed a strong and positive association between practitioners’ attitudes towards using academic research and their behavioural short-term intention to use it, both before (t1) and after (t2) exposure to the treatment, depicted in tables 4 and 5. intend_to_use attitude _t2 attitude _t1 intend_to_use 1 0.387*** 0.646*** att_t2 0.387*** 1 0.454*** att_t1 0.646*** 0.454*** 1 *** correlation is significant at the 0.001 level (2-tailed). table 4. correlations b se beta t p (constant) 0.140 0.280 0.501 0.617 att_t1 0.693 0.066 0.553 10.504 0.000 att_t2 0.241 0.053 0.240 4.565 0.000 dependent variable: intend_to_use table 5. coefficients the boxplots displaying the distribution of practitioners’ attitudes at t1 revealed that the short-term intention to use academic research was lowest among those with negative attitudes and highest among those with positive attitudes, as shown on the left side of figure 4. interestingly, the short-term intention to use academic content increased among participants who initially reported negative attitudes at t2, as illustrated on the right side of figure 4. this suggests that a pragmatic short-term intention to use academic research can persist even when personal attitudes are less favourable. however, there was significant variability in intention among participants who reported negative attitudes at t2. figure 4. boxplots of intention to use and attitude sentiment at t1 and t2 australasian journal of information systems mirkovski, doyle & gaskin 2025, vol 29, research article practitioners’ use of information systems research 22 the histograms showing response frequencies in figure 5 indicate that the distribution of mean attitude scores is skewed to the right at both t1 (left side) and t2 (right side). this suggests that practitioners generally agreed with the positive attitude statements at both measurement points. furthermore, the histogram depicting attitude change on the right side of figure 6 shows a range of responses, with some indicating a positive change (between 0 and a 1-point increase) and several reflecting a downward adjustment in attitude at t2. when examining the short-term intention to use is academic research and its relation to attitude, we observed a consistent pattern: short-term intention to use remained overall positive and aligned closely with attitude scores at both t1 and t2, as depicted on the left side of figure 6. figure 5. histogram for practitioners’ attitude to use academic research at t1 and t2 figure 6. histogram for practitioners’ intention to use academic research and attitude change the paired samples t-test revealed a statistically significant difference (t(229) = 4.752, p < 0.001), indicating that the mean attitude reported at t2 (3.959) was lower than the mean attitude at t1 (4.238). to further analyse this change, we conducted a multiple regression analysis, using research format as the predictor variable and intention as a control variable. this analysis found that research format was not a significant predictor of attitude change: r²_adj = 0.003, f (5, 224) = 1.156, p = 0.33. additionally, the considerable variability observed in the boxplot whiskers on the left side and the error bars on the right side of figure 7 may explain why statistically significant differences between the research formats could not be detected. australasian journal of information systems mirkovski, doyle & gaskin 2025, vol 29, research article practitioners’ use of information systems research 23 figure 7. mean change in practitioners’ attitude from t1 to t2 across formats in conclusion, the regression analysis showed a significant impact of practitioners’ attitudes on their short-term intentions to use is academic research both before (t1) and after (t2) exposure to our experimental treatment. this supports our hypotheses (h2a and h2b) that practitioners’ attitudes at t1 and t2 would correlate with their behavioural short-term intentions to use is academic research. the negative average change in attitude between t1 and t2 suggests that mere exposure to academic content might have been challenging for practitioners, despite most participants in our sample reporting at least some prior use of is academic research. although we observed a change in attitude, we could not identify differential effects based on the format. we also cannot determine whether this change in attitude is attributable to a high baseline due to social norms: i.e., scientific research is supposed to be a good thing and is smart and clever, so as a businessperson, i should have a positive attitude towards it (even if i’ve never seen any). but then exposure to it brought the ideal expectation (attitude) toward a realistic (grounded) evaluation. one possible explanation for the lack of differential effects based on research format is that the currently available research formats for disseminating is academic research may not offer distinct advantages that resonate with practitioners. while practitioners may be open and willing to engage with is academic research, the formats they are presented with might not effectively bridge the gap between academic content and practical application. this could mean that the traditional textual formats, even when supplemented with video content, are still perceived as detached from the real-world contexts in which practitioners operate. practitioners, especially those who are not deeply embedded in academic settings, might find it challenging to extract actionable insights from these formats, leading to a disconnect between their attitudes and their short-term intentions to use the research. the results suggest that despite their openness to utilising academic research, practitioners might not see a clear, immediate benefit in the formats provided. this highlights the need for more innovative and practically oriented formats that can directly address the specific needs and constraints of practitioners, making the research more accessible, relatable, and immediately applicable. interestingly, we found the highest short-term intention to use is academic research when practitioners were exposed to the “abstract video” and “video article” formats. the drop in attitude at t2 was also smaller in conditions featuring video content than those with textual academic articles. video formats, such as "abstract video" and "video article," likely offer a australasian journal of information systems mirkovski, doyle & gaskin 2025, vol 29, research article practitioners’ use of information systems research 24 more dynamic and visually engaging way to present research findings, making complex academic content more accessible and easier to digest. this is particularly important for practitioners who may be pressed for time or who might find traditional textual formats dense or challenging to navigate. video content can convey key messages more directly, use visuals to simplify complex ideas, and incorporate storytelling elements that resonate with practitioners' real-world experiences. furthermore, the ability of video content to maintain a more stable attitude over time may indicate that these formats are not only more engaging initially but also leave a lasting impression that textual formats struggle to achieve. the interactive and multisensory nature of video content might help reinforce the material, leading to better retention and a stronger intention to apply the research in practice. nevertheless, further investigation with a larger sample is necessary to understand better whether specific formats have a greater influence on practitioners’ attitudes and short-term intentions to use is academic research. 6.1.3 thematic analysis this study investigated practitioners’ specificity to use is academic research. we conducted a thematic analysis to explore how practitioners anticipate using the findings of ebel et al. (2016) in their work, as presented in the experimental treatment. they anticipated using academic research findings in various detailed and contextualised ways to enhance their professional practices. a prominent theme is business model development and innovation, where practitioners aim to leverage academic insights to innovate and manage new business models, implement improvements, and develop efficient frameworks for problem-solving. they see academic research as a valuable resource for generating new business strategies and enhancing business processes systematically. another significant theme is problem-solving and decision-making, with practitioners recognising the potential of research to offer validated approaches for more effective problem-solving and decision-making in their daily work. efficiency and customisation is another area where research findings are expected to play a role, particularly in streamlining and customising work processes, reducing complexity, and optimising tasks. practitioners also anticipate improvements in collaboration and teamwork, seeing research as a tool to enhance team communication, coordination, and collaborative project development. additionally, learning and skill development is a key area where practitioners expect to use research findings to facilitate learning, enhance research skills, and apply new management tools effectively. in conclusion, these findings suggest that practitioners see practical value in academic research and are inclined to apply its insights to their professional activities. the detailed ways in which they plan to use these findings indicate a strong intention to integrate academic knowledge into real-world applications, demonstrating the relevance and applicability of academic research in solving practical problems. 6.2 study 2 6.2.1 thematic analysis to explore practitioners' perceptions of text-based and multimedia-enabled research formats for translating and disseminating research knowledge and how these formats shape their short-term intentions to use academic research in their work (rq2), we conducted a thematic analysis. the key themes emerging from the thematic analysis are (1) research content, (2) research format, information processing and recall, and research storytelling, which provided australasian journal of information systems mirkovski, doyle & gaskin 2025, vol 29, research article practitioners’ use of information systems research 25 insights into practitioners' experiences and suggestions for improving the five distinct research formats. refer to table a6 in appendix 5 for exemplary evidence for the content theme of the thematic analysis. research content practitioners who received academic content through the “video” format noted that, while the tone, visuals, and storytelling were engaging, the content lacked definite conclusions, working examples, and further details, making it less applicable. one participant mentioned, “i agreed with everything in principle with what was being said in it; i just found it very hard to get excited about it because it didn’t really give you a lot of detail about exactly what was found” (participant a). another practitioner noted that while the “video” format presented concepts that appeared relevant, the recommendations provided felt "somewhat generic." this participant acknowledged the potential applicability of these concepts but highlighted a critical limitation: the need for adaptation to fit the specific needs of their organisation. the practitioner remarked, "i can see how the concepts might apply, but i would need to adapt them to fit the specific needs of my organisation" (participant d). the precise academic language used in the video, not commonly found in professional circles, was also a concern, with one participant stating, “some of these words get a bit complicated” (participant d). despite these issues, the narrative aspects of the video were appreciated, with a participant adding, “i liked that there was some kind of personification in the video, [...] the way that the storytelling worked, i felt like i was starting to get to know a host character in the video” (participant d). for those exposed to the “abstract” format, the lack of visuals, structure, and practical examples led to perceptions that the content was abstract and lacked direct relevance. as participant h pointed out, “if it has a user story or example, it would be great [...] it gives me a vivid picture that i can really know when and where i can apply this theory.” the abstract style and overly specific language were viewed as barriers to the content’s applicability in professional practice. similarly, practitioners assigned to the “article” format found it dense, overly theoretical, and difficult to translate into practical action without significant effort. participant f expressed this sentiment by saying, “it will be difficult to pick up the article, run with it, and say that these are the three or five things that i as a customer need to do. you’d have to read the entirety of the article, try and summarise it yourself, and make it relatable to the problems that you have.” in contrast, the “abstract video” format was generally perceived as relevant and applicable. however, the extended abstract’s lack of details and definite conclusions diminished its direct relevance. as participant c explained, “i would definitely need more details before considering using that thing [...] i’m not sure i would quote it, i would need to think about it a little bit more.” despite these shortcomings, is practitioners appreciated the engaging visuals of the short video, which participant c found “really good as key takeaways,” though they expressed a need for more substantial reference materials: “maybe some reference materials or something afterwards that people could download.” respondents indicated a preference for consuming research from more accessible and credible sources, such as suppliers' white papers and consultants' reports, over traditional academic formats. participant j remarked, "i usually turn to sources like gartner magic quadrant or sas white papers when i need reliable and practical insights. they’re more straightforward and relevant to what i’m working on, unlike some academic articles that are too theoretical and hard to apply directly." australasian journal of information systems mirkovski, doyle & gaskin 2025, vol 29, research article practitioners’ use of information systems research 26 research format regarding the “video” format, practitioners noted that while it provided a good overview and was engaging, it was not effective as a stand-alone tool for comprehensive information transfer. as participant a noted, “the 10-minute video was great for a very brief introduction, but you’d want that to be your starting piece.” the format was less effective for deep content comprehension due to its brevity and distractions like subtitles. participant b added, “generally, what i expect is if there is a printed version, that can really help because then you can [...] highlight the keywords.” one practitioner expressed a clear preference for more visual and interactive research formats, specifically highlighting the advantages of the video article format. she explained, "i prefer something more visual because it’s easier to digest and keeps my attention." this participant contrasted this with text-based formats, noting that "text articles are too dense and timeconsuming for me to go through during a busy workday." (participant g) the “abstract” format was criticised for being disengaging, with participants finding it difficult to extract clear overviews or practical takeaways from the content. participant k described it as “a lot of information and [...] all the words i feel are a little bit boring.” the “article” format was viewed as time-consuming and difficult to digest. participant f mentioned that “it’s quite off-putting to look at the two-column text format,” suggesting that pairing it with more interactive formats like videos could improve its accessibility and engagement. one practitioner shared their intentions to apply insights from a research format that combined a short video with a full academic article, stating, "i haven’t had a chance to implement any of the key takeaways from video and article, but i do plan to use some of the insights in an upcoming project." he appreciated the video for providing a quick and accessible overview but acknowledged the need to revisit the full article for more detailed information before he could effectively apply the insights, saying, "the video was great for giving me a quick overview, but i’ll need to dive back into the article for more details before i can put anything into practice" (participant e). information processing and recall and research storytelling a practitioner expressed concerns about the practicality of implementing recommendations from the “abstract” format, stating, "one challenge i see is that the extended abstract didn’t provide enough practical examples, so i’m not entirely sure how to implement the recommendations" (participant k). this lack of concrete examples left them uncertain about how to apply the insights effectively in their work. she indicated that further effort would be required to bridge this gap, saying, "i’ll need to do some additional research or maybe consult with colleagues to figure out the best way to apply these insights." practitioners who engaged with the “video” format found it challenging to recall key findings and implications due to the simultaneous use of animation, audio, and subtitles, which demanded focused attention. however, they were able to recall the narrative and story arches effectively. participant d stated, “i can’t remember parts of it because i haven’t been able to read it, but i remember, i can kind of visualise the diagram where they had, start with this thing, do some of that thing, and then while you are doing it, do the other thing concurrently.” similarly, the “abstract” treatment presented challenges in recalling details, though participants could recall the narrative structure. is practitioners who were assigned to australasian journal of information systems mirkovski, doyle & gaskin 2025, vol 29, research article practitioners’ use of information systems research 27 consume the “article” format mentioned that they had to read the text multiple times to understand the content and evaluate its relevance. as participant f explained, “i would have to re-read it and probably read it a third time to get any actual action points out of it.” when exposed to the “video article” or “abstract” formats, participants had better information recall but reported information processing overload due to needing to switch between materials. despite this, the narrative structure of the animated research brief was acknowledged as an effective method for conveying complex academic content to practitioners. participant g noted, “as a sort of takeaway and a way to engage my memory, it would be quite good to have a middle ground between the graphics that were presented and the script [...] that would be a nice way to tie them together.” in conclusion, these findings reveal that practitioners have varied experiences with different research formats. practitioners generally find multimedia formats, particularly those with videos, more engaging and accessible than traditional text-based formats. however, they often seek more practical examples and clearer conclusions, which are typically lacking. text-based formats are familiar but are frequently criticised for being dense, time-consuming, and difficult to apply in practice due to inaccessible language, lack of examples, and theoretical content. while they acknowledge the relevance of academic research in addressing current technological and business issues, they struggle to assess the applicability of traditional is articles to their challenges. as a result, they tend to favour more accessible sources like suppliers’ white papers and consultants’ reports over traditional academic formats. thus, a blended approach that combines the engaging aspects of multimedia with the depth of traditional academic articles may be most effective for practitioners. for example, visual formats, such as animated research briefs are engaging and easier to digest, making them ideal for quick overviews. however, to support practical application, these should be paired with detailed, text-based formats that include specific examples and actionable insights. this combination ensures that practitioners can both quickly grasp the main concepts and have access to the depth of information needed for implementation. this highlights the importance of considering both content and format when translating and disseminating academic research to ensure it is both applicable and relevant to practitioners’ needs. 7 discussion 7.1 contributions and implications this paper contributes to the ongoing discourse in the is research discipline concerning the balance between research rigour and relevance, building on existing arguments made by benbasat and zmud (1999); dennis (2001); lee et al. (2021); rosemann and vessey (2008). our findings reveal the complexities of translating and disseminating research to practitioners, highlighting the challenges in bridging the gap between academic insights and practical application (moeini et al., 2019). our findings challenge the notion that merely modifying the format of research dissemination, such as incorporating multimedia elements, can substantially alter practitioners' attitudes towards using academic research. the non-significant results for h1 suggest that practitioners' attitudes are deeply ingrained and may not be easily swayed by superficial changes in presentation. this emphasises the importance of focusing on the substance and relevance of australasian journal of information systems mirkovski, doyle & gaskin 2025, vol 29, research article practitioners’ use of information systems research 28 the research itself rather than solely relying on multimedia formats to enhance its perceived value, as argued by benbasat and zmud (1999) who identified the content of research as a key characteristic of its relevance. our qualitative findings further emphasise the need for research to be relatable and engaging, addressing the specific challenges and contexts that practitioners face in their daily work and supporting the notion that research relevance should consider the perspective of practitioners (fox & groesser, 2016). we also found that practitioners' attitudes, both before and after consuming research, play a role in shaping their short-term intentions to use it. the significant results for h2a and h2b highlight the importance of fostering positive attitudes towards academic research among practitioners. this aligns with existing arguments by bhattacherjee and premkumar (2004), karahanna et al. (1999), moeini and rivard (2019), ormond et al. (2019), and titah and barki (2009) who have all discussed the pivotal role of positive attitudes in fostering intentions to engage in certain behaviours in the workplace. this can be achieved by ensuring that research is both relevant and presented in a clear, concise, and accessible manner, focusing on practical implications and actionable insights. this echoes the recommendations of robey and markus (1998) and gill and bhattacherjee (2009) regarding the importance of clear communication and tailoring research messages to practitioners' needs. our exploration of practitioners' perceptions of different research formats offers valuable insights into the strengths and weaknesses of various approaches. while multimedia formats, such as animated research briefs, are generally perceived as more engaging and accessible, aligning with the suggestions of toffel (2016) and moeini et al. (2019), they can lack the depth and practical examples that practitioners require for effective application. in contrast, traditional text-based formats, while valued for their comprehensiveness, can be overwhelming and difficult to navigate, resonating with the concerns raised by gill and bhattacherjee (2009) about the inherent differences between researchers and practitioners. hence a blended approach, combining various formats, may be the most effective way to bridge the gap between is research and practice. this approach supports the notion that, while textual articles are valuable, more concise and interactive formats can also effectively convey knowledge in a manner that is more accessible to practitioners. given the disconnect between research format and practitioner engagement, the is research discipline should reconsider its traditional approaches to knowledge translation and dissemination. given the disconnect between research format and practitioner engagement, the is research discipline should reconsider its traditional approaches to knowledge translation and dissemination. our findings advocate for more accessible, engaging, and practically oriented research dissemination strategies. embracing a broader range of formats, including videos, animations, and interactive tools, could enhance the impact of is research while maintaining its academic integrity. this approach aligns with the concept of the open artefact, where is researchers are encouraged to disseminate models, methods, and frameworks outside the traditional academic paper, thus fostering a stronger connection between academia and practice (doyle et al., 2019). in conclusion, practitioners' attitudes toward academic research are complex and multifaceted. while different formats, such as the above, can enhance engagement and accessibility, they should complement rigorous and relevant research that addresses practitioners' practical needs. by focusing on clear communication, actionable insights, and relatable narratives, researchers can foster positive attitudes and increase the likelihood that their work will be australasian journal of information systems mirkovski, doyle & gaskin 2025, vol 29, research article practitioners’ use of information systems research 29 used to inform and improve practice, contributing to the effective knowledge product translation and dissemination advocated by moeini et al. (2019). however, this requires researchers to think beyond the academic paper as the only output of their research and consider other formats that practitioners would appreciate (doyle, 2021). by framing research findings within narratives that resonate with practitioners' real-world experiences and challenges, and disseminating them through the right format, is researchers can make their work more engaging and impactful. this shift could foster a more robust collaboration and knowledge exchange between academia and practice, leading to more impactful solutions to real-world problems. based on these findings, we present table 6, which outlines practical recommendations for translating and disseminating research knowledge products. each recommendation is supported by detailed explanations and practical examples to ensure effective implementation. dimensions (moeini et al., 2019) recommendation explanation knowledge product translation prioritise substance over superficial format changes emphasise the importance of the research content itself in addressing practitioners' challenges. superficial changes in presentation, such as adding multimedia elements, should not overshadow the substance of the research. by focusing on the relevance and practicality of the research, practitioners are more likely to engage with and apply the findings (benbasat & zmud, 1999). incorporate practical, context-specific examples integrate detailed, real-world examples or case studies that resonate with the specific industry or organisational context to which the research is being applied. this approach helps practitioners see the direct applicability and benefits of the research in their own work environments, making it more likely that they will adopt the recommendations (fox & groesser, 2016). customise research for practitioner audiences tailor the language, structure, and content of the research to fit the specific needs and preferences of different practitioner groups. this involves avoiding academic jargon and instead using terminology that is familiar and accessible to practitioners. additionally, the content should be structured in a way that aligns with the practical decision-making processes within the targeted industry (gill & bhattacherjee, 2009). highlight actionable takeaways clearly clearly identify and emphasise actionable insights at the beginning of research outputs. practitioners often have limited time, so highlighting the practical steps they can take based on the research findings will increase the likelihood of implementation. this can include executive summaries, bullet points, or other concise formats that make it easy for practitioners to extract the key messages (robey & markus, 1998). knowledge product dissemination adopt a blended dissemination strategy use a combination of traditional text-based articles and modern multimedia formats, such as videos, podcasts, and infographics, to reach a broader and more diverse audience. this approach recognises that different practitioners have different preferences for how they consume information. by offering multiple formats, researchers can ensure that their findings are accessible to all relevant audiences (toffel, 2016). australasian journal of information systems mirkovski, doyle & gaskin 2025, vol 29, research article practitioners’ use of information systems research 30 actively involve practitioners in the dissemination process engage practitioners directly in the dissemination process by inviting them to participate in events such as workshops, seminars, or webinars. this active involvement not only helps to communicate the research findings but also allows practitioners to discuss and explore practical applications in a collaborative environment. such interactions can lead to greater buy-in and more effective implementation of the research outcomes (ormond et al., 2019). create visual summaries for quick reference develop visual tools such as animated video briefs, infographics, or one-page summaries that distil complex research findings into easily digestible formats. these tools are particularly useful for busy practitioners who need to quickly understand the key points of the research and how they can be applied. visual summaries can serve as quick reference guides that practitioners can revisit when implementing the research in their work (moeini et al., 2019). ensure research artefacts are openly accessible make research artefacts, such as frameworks, models, or tools, available on open access platforms. these artefacts should be accompanied by detailed guides or tutorials that explain their practical application in real-world scenarios. open accessibility ensures that a wider audience of practitioners can benefit from the research, while the supporting materials help them effectively implement the findings in their own contexts (doyle, 2021; doyle et al., 2019). table 6. practical recommendations for researchers how to effectively translate and disseminate their knowledge products 7.2 limitations and future research this paper is not without limitations. first, only one is article within the design science research methodological domain was used in both the survey-based experiment and interviews. this is particularly problematic because, while this article may not be relevant and useful to a specific practitioner, another article may be highly relevant. the body of is literature is not generic or homogenous. this limitation affects both attitudes towards academic research and short-term intention to use academic research. our measures were generic (e.g., “using peer-reviewed academic articles for my professional work is a good idea”), but our operationalisation used only one specific study. thus, our findings must be considered within this limitation that the practitioners’ attitude and intentions are anchored by a specific study, despite asking about academic research in general. results may have differed if given multiple articles or asked specifically about this paper. we tried to mitigate this issue by focusing our recruiting for study 2 on is practitioners dealing with product/market development in the it industry (and therefore familiar with business model development). to generalise these findings across the is community, future studies should include multiple is articles representing various subfields and methodologies. second, the small sample size of 12 practitioners from new zealand limits the generalisability of our qualitative findings. future research should involve a larger, more diverse sample from different regions. moreover, this paper focused exclusively on is practitioners; future research should explore the dissemination of is research to other stakeholders, such as government agencies and students, to better understand their attitudes and intentions regarding the use of australasian journal of information systems mirkovski, doyle & gaskin 2025, vol 29, research article practitioners’ use of information systems research 31 is academic research. while some of the results may apply to other fields, this is beyond the scope of the current study. third , while we do control for age, gender, occupation, rank, and years of it work experience, we fail to account for some known confounds, such as subjective norms and perceived behavioural control (moeini et al., 2019). thus, our findings might shift if these confounds were included in our models. our multi-method approach helps to alleviate this limitation to some extent by providing alternative corroborating perspectives on the quantitative findings. 8 conclusion this paper offers important insights into how the format of academic research influences is practitioners' attitudes and short-term intentions to use research in practice. it adds to the ongoing discussion about research relevance and rigour in the is domain. the findings highlight the challenges and opportunities associated with translating and disseminating research knowledge to practitioners. by understanding the factors that influence practitioners' attitudes and short-term intentions, researchers can develop more effective strategies for bridging the gap between academia and practice, ensuring that their work has a meaningful and lasting impact on the real world, ultimately fulfilling the dual purpose of is research (dennis, 2001). regarding rq1, the findings show that while h2 is supported—indicating that practitioners' attitudes significantly impact their short-term intentions to use is academic research—h1 is not significant, meaning the format of the research did not influence practitioners' attitudes as hypothesised. however, despite this lack of direct influence, there was a general decline in attitude after exposure to academic content, suggesting challenges in engaging with the material, regardless of format. for rq2, the study reveals that practitioners generally prefer multimedia-enabled formats, such as animated research briefs, due to their engaging and accessible nature. however, these formats often lack the depth and practical examples that practitioners require for effective application. in contrast, traditional text-based formats, while valued for their comprehensiveness, are frequently criticised for being dense and difficult to digest. these findings suggest that a blended approach, combining the engagement of multimedia with the thoroughness of traditional formats, may better serve practitioners' needs. reference ajzen, i. (1991). the theory of planned behavior. organizational behavior and human decision processes, 50(2), 179-211. doi.org/10.1016/0749-5978(91)90020-t avital, m., mathiassen, l., & schultze, u. (2017). alternative genres in information systems research. european journal of information systems, 26(3), 240-247. doi.org/10.1057/s41303-017-0051-4 aysolmaz, b., & reijers, h. a. (2021). animation as a dynamic visualization technique for improving process model comprehension. information & management, 58(5). doi.org/10.1016/j.im.2021.103478 benbasat, i., & zmud, r. w. (1999). empirical research in information systems: the practice of relevance. mis quarterly, 23(1), 3. doi.org/10.2307/249403 australasian journal of information systems mirkovski, doyle & gaskin 2025, vol 29, research article practitioners’ use of information systems research 32 bhattacherjee, a., & premkumar, g. (2004). understanding changes in belief and attitude toward information technology usage: a theoretical model and longitudinal test. mis quarterly, 28(2), 229. doi.org/10.2307/25148634 bourdieu, p. (1988). homo academicus. stanford university press. doi.org/10.1093/sf/68.4.1317 cohen, j., cohen, p., west, s. g., & aiken, l. s. (2002). applied multiple regression/correlation analysis for the behavioural sciences (3rd edition ed.). routledge. doi.org/10.4324/9780203774441 corbin, j., & strauss, a. (2014). basics of qualitative research: techniques and procedures for developing grounded theory (4th edition ed.). sage. doi.org/10.4135/9781452230153 creswell, j. w., & clark, v. l. p. (2018). designing and conducting mixed methods research. sage publications. davis, f. d., bagozzi, r. p., & warshaw, p. r. (1989). user acceptance of computer technology: a comparison of two theoretical models. management science, 35(8), 9821003. doi.org/10.1287/mnsc.35.8.982 dennis, a. r. (2001). relevance in information systems research. communications of the association for information systems, 6. doi.org/10.17705/1cais.00610 diamond, m. e. (2011). the role of narrative in multimedia learning university of nevada, las vegas]. https://digitalscholarship.unlv.edu/thesesdissertations/1231 doyle, c. (2021). how can i share my work? a review of the open access policies of is journals. hawaii international conference on system sciences (hicss), hawaii, usa. doyle, c., luczak-roesch, m., & mittal, a. (2019). we need the open artefact: design science as a pathway to open science in information systems research. extending the boundaries of design science theory and practice: 14th international conference on design science research in information systems and technology, desrist 2019, worcester, ma, usa, june 4–6, 2019, proceedings 14, ebel, p., bretschneider, u., & leimeister, j. m. (2016). leveraging virtual business model innovation: a framework for designing business model development tools. information systems journal, 26(5), 519-550. doi.org/10.1111/isj.12103 etzkowitz, h., & leydesdorff, l. (2000). the dynamics of innovation: from national systems and “mode 2” to a triple helix of university–industry–government relations. research policy, 29(2), 109-123. doi.org/10.1016/s0048-7333(99)00055-4 fox, s., & groesser, s. n. (2016). reframing the relevance of research to practice. european management journal, 34(5), 457-465. doi.org/10.1016/j.emj.2016.07.005 gaskin, j., lowry, p., & hull, d. (2016). leveraging multimedia to advance science by disseminating a greater variety of scholarly contributions in more accessible formats. journal of the association for information systems, 17(6), 413-434. doi.org/10.17705/1jais.00430 gill, g., & bhattacherjee, a. (2009). whom are we informing? issues and recommendations for mis research from an informing sciences perspective. mis quarterly, 33(2), 217. doi.org/10.2307/20650290 greer, d. l., crutchfield, s. a., & woods, k. l. (2013). cognitive theory of multimedia learning, instructional design principles, and students with learning disabilities in computer-based and online learning environments. journal of education, 193(2), 41-50. doi.org/10.1177/002205741319300205 hull, d. m., lowry, p. b., gaskin, j. e., & mirkovski, k. (2019). a storyteller's guide to problem‐based learning for information systems management education. information systems journal, 29(5), 1040-1057. doi.org/10.1111/isj.12234 australasian journal of information systems mirkovski, doyle & gaskin 2025, vol 29, research article practitioners’ use of information systems research 33 immanuel, j., & a, m. s. h. m. (2023). cognitive theory of multimedia learning powered code-switching interface for vocabulary acquisition of esl students. journal of language teaching and research, 14(6), 1439-1448. doi.org/10.17507/jltr.1406.01 johnson, r. b., & onwuegbuzie, a. j. (2004). mixed methods research: a research paradigm whose time has come. educational researcher, 33(7), 14-26. doi.org/10.3102/0013189x033007014 karahanna, e., straub, d. w., & chervany, n. l. (1999). information technology adoption across time: a cross-sectional comparison of pre-adoption and post-adoption beliefs. mis quarterly, 23(2), 183-213. doi.org/10.2307/249751 king, n., cassell, c., & symon, g. (2004). using templates in the thematic analysis of texts. in c. cassell & g. symon (eds.), essential guide to qualitative methods in organizational research (pp. 256-270). sage. doi.org/10.4135/9781446280119.n21 kirschner, p. a., park, b., malone, s., & jarodzka, h. (2023). toward a cognitive theory of multimedia assessment (ctmma). in learning, design, and technology (pp. 153-175). doi.org/10.1007/978-3-319-17461-7_53 klein, g., jiang, j., & saunders, c. (2006). leading the horse to water. communications of the association for information systems, 18. doi.org/10.17705/1cais.01813 lee, j. k., park, j., gregor, s., & yoon, v. (2021). axiomatic theories and improving the relevance of information systems research. information systems research, 32(1), 147171. doi.org/10.1287/isre.2020.0958 lowry, p. b., d’arcy, j., hammer, b., & moody, g. d. (2016). “cargo cult” science in traditional organization and information systems survey research: a case for using nontraditional methods of data collection, including mechanical turk and online panels. the journal of strategic information systems, 25(3), 232-240. doi.org/10.1016/j.jsis.2016.06.002 mayer, r. e. (2005). cognitive theory of multimedia learning. in r. e. mayer (ed.), the cambridge handbook of multimedia learning (2nd edition ed., pp. 31-48). cambridge university press. doi.org/10.1017/cbo9780511816819.004 mayer, r. e. (2008). applying the science of learning: evidence-based principles for the design of multimedia instruction. american psychologist, 63(8), 760-769. doi.org/10.1037/0003-066x.63.8.760 mccarthy, m. (2012). applying narrative to the delivery of the ill-defined domain of manufacturing systems to undergraduate students. australasian journal of engineering education, 18(1). doi.org/10.7158/d11-140.2012.18.1 mcclements, d. j., mcclements, j., & mcclements, i. f. (2024). securing funding: oiling the wheels of research. in how to be a successful scientist: a guide for graduate students, postdocs, and professors (pp. 239-261). springer. doi.org/10.1007/978-3-031-51402-9_11 miles, m. b., huberman, a. m., & saldaña, j. (2014). qualitative data analysis: a methods sourcebook (3rd edition ed.). sage. doi.org/10.1080/10572252.2015.975966 mirkovski, k., gaskin, j., hull, d., & lowry, p. (2019). visual storytelling for improving the comprehension and utility in disseminating information systems research: evidence from a quasi-experiment. information systems journal, 29(6), 1153-1177. doi.org/10.1111/isj.12240 moeini, m., rahrovani, y., & chan, y. e. (2019). a review of the practical relevance of is strategy scholarly research. the journal of strategic information systems, 28(2), 196-217. doi.org/10.1016/j.jsis.2018.12.003 australasian journal of information systems mirkovski, doyle & gaskin 2025, vol 29, research article practitioners’ use of information systems research 34 moeini, m., & rivard, s. (2019). responding or not to information technology project risks: an integrative model. mis quarterly, 43(2), 475-500. doi.org/10.25300/misq/2019/14505 myers, m. d., & avison, d. (2002). qualitative research in information systems: a reader. sage. doi.org/10.4135/9781849209687 myers, m. d., & newman, m. (2007). the qualitative interview in is research: examining the craft. information and organization, 17(1), 2-26. doi.org/10.1016/j.infoandorg.2006.11.001 nunamaker jr, j. f., briggs, r. o., derrick, d. c., & schwabe, g. (2015). the last research mile: achieving both rigor and relevance in information systems research. journal of management information systems, 32(3), 10-47. doi.org/10.1080/07421222.2015.1094961 ormond, d., warkentin, m., & crossler, r. e. (2019). integrating cognition with an affective lens to better understand information security policy compliance. journal of the association for information systems, 1794-1843. doi.org/10.17705/1jais.00586 pearson, m., pearson, a., & shim, j. p. (2005). the relevancy of information systems research: the practitioner's view. information resources management journal, 18(3), 50-67. doi.org/10.4018/irmj.2005070104 robey, d., & markus, m. l. (1998). beyond rigor and relevance: producing consumable research about information systems. information resources management journal, 11(1), 7-16. doi.org/10.4018/irmj.1998010101 rosemann, m., & vessey, i. (2008). toward improving the relevance of information systems research to practice: the role of applicability checks. mis quarterly, 32(1), 1-22. doi.org/10.2307/25148826 srivastava, s. c., & teo, t. (2005). relevance in information systems research: an empirical analysis. pacis proceedings. tabachnick, b. g., & fidell, l. s. (2007). experimental designs using anova. thomson/brooks/cole taylor, s., & todd, p. a. (1995). understanding information technology usage: a test of competing models. information systems research, 6(2), 144-176. doi.org/10.1287/isre.6.2.144 titah, r., & barki, h. (2009). nonlinearities between attitude and subjective norms in information technology acceptance: a negative synergy? mis quarterly, 33(4). doi.org/10.2307/20650329 toffel, m. w. (2016). enhancing the practical relevance of research. production and operations management, 25(9), 1493-1505. doi.org/10.1111/poms.12558 van de ven, a. h. (2007). engaged scholarship: a guide for organizational and social research. oxford university press, usa. van de ven, a. h., & johnson, p. e. (2006). knowledge for theory and practice. academy of management review, 31(4), 802-821. doi.org/10.5465/amr.2006.22527385 venkatesh, v., morris, m. g., davis, g. b., & davis, f. d. (2003). user acceptance of information technology: toward a unified view. mis quarterly, 27(3), 425-478. doi.org/10.2307/30036540 voithofer, r. (2004). teaching computers to tell learning stories: using critical narrative theory to frame design and evaluation strategies for online educational experiences. journal of educational multimedia and hypermedia, 13(1), 47-72. australasian journal of information systems mirkovski, doyle & gaskin 2025, vol 29, research article practitioners’ use of information systems research 35 williams, w. r. (2019). attending to the visual aspects of visual storytelling: using art and design concepts to interpret and compose narratives with images. journal of visual literacy, 38(1-2), 66-82. doi.org/10.1080/1051144x.2019.1569832 appendix 1: animated research brief to address the audience and content considerations for translating and disseminating is research knowledge, we have deliberately chosen a design science research article. one of the main tenets of design science research is to build a usable artefact that is relevant to the practice. it adopts an action design research methodology to (1) explore, induce, and deduct a real-world, practical problem or concern (i.e., the process of identifying new markets and developing suitable business models within a traditional workshop setting at sap was being deemed as too costly and not feasible); (2) design and test a solution for the identified problem (i.e., proposing a new business model development tool by reviewing the existing literature and conducting expert interviews, and then piloting and evaluating the proposed tool in use setting at sap); and (3) validate and generalise the findings to other applications outside the immediate context (i.e., applying the learnings from designing the new business model development tool to a broader class of problems and identifying its contributions to the literature on the development and management of new business models). figure a1. visual overview of the animated research brief for the published academic article by ebel et al. (2016) australasian journal of information systems mirkovski, doyle & gaskin 2025, vol 29, research article practitioners’ use of information systems research 36 this article presents a framework for developing a tool for supporting the development and management of a new business model (ebel et al., 2016), which is relevant to is practitioners dealing with product/market development in the it industry. the proposed framework is particularly relevant for roles tasked with the creation, implementation, and management of innovative it solutions that align with evolving business strategies. by leveraging virtual business model innovation, as outlined in the framework, practitioners (e.g., product owners, it developers, product managers, software engineers, business analysts, it analysts, it project managers, solution testers, it consultants, and solution architects) can enhance their capacity to develop robust, scalable, and market-responsive it products. this framework for designing business model development tools provides systematic methodologies for assessing and integrating innovative business models, thus ensuring that it solutions not only meet current market demands but also drive future growth, underscoring its practical utility for is practitioners operating in the it industry. appendix 2: measurement operationalisation and interview protocol construct (source) items/questions definition attitude to use academic research before consumption (att_t1) attitude to use academic research after consumption (att_t2) (ajzen, 1991; davis et al., 1989; taylor & todd, 1995) please indicate how much the statements below are applicable to you: strongly disagree … strongly agree • using peer-reviewed academic articles for my professional work is a good idea (att1). • i like the idea of using peer-reviewed academic articles to help my work (att2). • (r) using peer-reviewed academic articles for my work is a foolish idea (att3). • i find use of peer-reviewed academic articles for my professional work to be pleasant (att4). the practioner’s positive or negative feelings about using peerreviewed academic research in his/her work practice. behavioural short-term intention to use academic research (biu) (venkatesh et al., 2003) please indicate how much you agree with the statements below: strongly agree … strongly disagree • i intend to use peer-reviewed academic research for my work practice in the next 3-6 months (biu1). • i predict to use peer-reviewed academic research for my work practice in the next 3-6 months (biu2). • i plan to use to peer-reviewed academic research for my work practice in the next 3-6 months (biu3). extent to which the practitioner feels he/she would use the peerreviewed academic material for his/her work practice in the immediate future. practitioners’ specificity to use academic research • how could this academic material help you? • briefly describe a situation in which you could use the main findings from this academic material and how the findings could help you deal with the situation. the detailed and contextualised ways in which practitioners anticipate using academic research findings in their work. table a1. operationalisation of items for study 1 australasian journal of information systems mirkovski, doyle & gaskin 2025, vol 29, research article practitioners’ use of information systems research 37 interview protocol for study 2 opening prompts: • could you start by telling me about your current role in the it industry? how long have you been working in this position? • how many years of experience do you have in the it field overall? have you worked in similar roles before, or has your career evolved into your current position? • can you describe some of the key responsibilities in your role? how do these responsibilities align with your experience in it? • throughout your career, what significant changes or developments in the it industry have impacted your work the most? • in your current role, how frequently do you engage with academic research or new industry developments to inform your work practices? how does your experience shape the way you approach this information? main interview questions with prompts: 1. what are your overall thoughts regarding the academic article and/or research brief you have watched? • prompt 1a: was the information presented in a way that was easy to follow and understand? what aspects did you find particularly clear or confusing? • prompt 1b: how do you feel about the depth of information provided in the academic article/research brief? do you find this format effective for understanding complex concepts? • prompt 1c: did the content meet your expectations based on your experience with similar research materials? why or why not? 2. what are your overall thoughts regarding the use of a story to translate and disseminate academic research? tell me more. please explain. • prompt 2a: do you think the storytelling approach made the research findings more memorable? can you give an example? • prompt 2b: do you think a story or narrative format could make the content more relatable or actionable? could you see yourself using a similar approach to communicate complex information in your own work? 3. what are your overall thoughts regarding the visuals that were used in the research article/research brief? tell me more. please explain. • prompt 3a: did the visuals aid your understanding of the research findings? which visual elements were the most impactful for you? • prompt 3b: do you think the use of multimedia elements (e.g., video, animations) could help clarify the research findings? how would they impact your engagement with the material? • prompt 3c: were there any visuals that you found distracting or unhelpful? why? 4. was this relevant to your work practice? can you use the recommendations to inform your work practice? how current is this? please explain. • prompt 4a: how adaptable do you find these recommendations to the unique aspects of your role or organisation? would any modifications be needed? australasian journal of information systems mirkovski, doyle & gaskin 2025, vol 29, research article practitioners’ use of information systems research 38 • prompt 4b: when considering how you consume research for your work, do you prefer a text-based format or something more visual and interactive? why? 5. did you action some of the key takeaways or do you plan to do so? please explain. • prompt 5a: how soon do you think you’ll apply the key takeaways in your work? are there specific projects or tasks where you see them being particularly useful? • prompt 5b: were there any barriers or challenges that might prevent you from acting on these takeaways? how might you overcome them? 6. what are the main takeaways in this research brief? can you recall any of them? please explain. • prompt 6a: which takeaway stood out to you the most, and why do you think it was memorable? • prompt 6b: did the way the information was presented (e.g., textual with figures, visually through storytelling) help you retain the key points? how so? 7. do you think you will use any of the main takeaways from this research brief to inform your work practice? please explain. • prompt 7a: if you were to share this research with a colleague, which format would you choose and why? what do you think would resonate more with them? • prompt 7b: do you foresee any long-term changes in your work practice based on what you’ve learned from this research brief? what might those changes look like? appendix 3: final coding schemes second-order aggregate themes first-order codes business model development and innovation leveraging virtual business model innovations developing and managing new business models implementing and improving business models creating new ideas and business strategies enhancing business processes through systematic approaches creating efficient frameworks for business problem-solving problem-solving and decision making solving problems more effectively analysing and solving business-related issues solving issues by applying researched and validated approaches efficiency and customisation customising and streamlining work processes evaluating and optimising work processes reducing complexity in tasks collaboration and teamwork collaborative project development improving team collaboration enhancing team communication and coordination data organisation and analysis organising raw data for better results is development and innovation building it development models integrating new technologies evaluating and optimising technology implementation learning and skill development enhancing research skills facilitating learning and problem-solving learning management tools effectively application of research in practice applying academic material for practical solutions applying findings to real-world scenarios using academic research to enhance business practices customer relationship management improving customer satisfaction gathering and using feedback australasian journal of information systems mirkovski, doyle & gaskin 2025, vol 29, research article practitioners’ use of information systems research 39 gathering and analysing feedback from stakeholders table a2. final coding scheme from thematic analysis for study 1 second-order aggregate themes first-order codes research content article’s content does not make sense article’s content is complex video article’s content has good applicability abstract video’s content has good relevance video’s content has language specificity video’s content is accessible video’s content is approachable article’s content is too abstract and theoretical article’s content is too condensed article’s content lacks accessibility abstract’s content lacks applicability article’s content lacks definite conclusions abstract’s content lacks direct relevance video’s content lacks further details article’s content lacks highlights article’s content lacks relevance abstract’s content lacks structure abstract’s content lacks visuals abstract’s content lacks working examples video article’s content provides sufficient details video article’s content has language specificity video article’s content has timeliness video’s content visuals are good article’s content visuals are not good video’s tone was good video article is reliable source video article has well-structured content research format article’s format does not provide good overview article video’s format is effective to convey information video’s format is accessible article’s format is disengaging article video’s format is engaging video’s format is good overview article’s format is time-consuming article’s format lacks accessibility abstract’s format is not effective in isolation abstract’s format is not suitable for comprehension video’s format subtitles are distracting video articles’ format is suitable for content comprehension video article’s format subtitles are useful information processing and recall alternative sources of is knowledge difficulty of information recall information processing overload reasonable information recall professionals lack of time research storytelling limited use of storytelling storytelling is valuable practice visual storytelling is valuable practice table a3. final coding scheme from thematic analysis for study 2 australasian journal of information systems mirkovski, doyle & gaskin 2025, vol 29, research article practitioners’ use of information systems research 40 appendix 4: anova (i) format (j) format mean difference (i-j) std. error sig. 95% confidence interval lower bound upper bound video video article -0.13371 0.17867 1.000 -0.6402 0.3728 article -0.05392 0.18294 1.000 -0.5726 0.4647 abstract 0.19013 0.18412 1.000 -0.3318 0.7121 abstract video -0.04860 0.17867 1.000 -0.5551 0.4579 video article video 0.13371 0.17867 1.000 -0.3728 0.6402 article 0.07979 0.18647 1.000 -0.4488 0.6084 abstract 0.32383 0.18762 0.857 -0.2081 0.8557 abstract video 0.08511 0.18228 1.000 -0.4316 0.6019 article video 0.05392 0.18294 1.000 -0.4647 0.5726 video article -0.07979 0.18647 1.000 -0.6084 0.4488 abstract 0.24405 0.19170 1.000 -0.2994 0.7875 abstract video 0.00532 0.18647 1.000 -0.5233 0.5339 abstract video -0.19013 0.18412 1.000 -0.7121 0.3318 video article -0.32383 0.18762 0.857 -0.8557 0.2081 article -0.24405 0.19170 1.000 -0.7875 0.2994 abstract video -0.23873 0.18762 1.000 -0.7706 0.2932 abstract video video 0.04860 0.17867 1.000 -0.4579 0.5551 video article -0.08511 0.18228 1.000 -0.6019 0.4316 article -0.00532 0.18647 1.000 -0.5339 0.5233 abstract 0.23873 0.18762 1.000 -0.2932 0.7706 table a4. bonferroni’s pairwise comparison average attitude at t2 (i) format (j) format mean difference (i-j) std. error sig. 95% confidence interval lower bound upper bound video video article 0.04141 0.17986 1.000 -0.4685 0.5513 article 0.13121 0.18416 1.000 -0.3909 0.6533 abstract 0.00455 0.18535 1.000 -0.5209 0.5300 abstract video -0.26179 0.17986 1.000 -0.7717 0.2481 video article video -0.04141 0.17986 1.000 -0.5513 0.4685 article 0.08981 0.18772 1.000 -0.4424 0.6220 abstract -0.03685 0.18888 1.000 -0.5723 0.4986 abstract video -0.30319 0.18350 0.999 -0.8234 0.2170 article video -0.13121 0.18416 1.000 -0.6533 0.3909 video article -0.08981 0.18772 1.000 -0.6220 0.4424 abstract -0.12666 0.19298 1.000 -0.6738 0.4204 abstract video -0.39300 0.18772 0.374 -0.9252 0.1392 abstract video -0.00455 0.18535 1.000 -0.5300 0.5209 video article 0.03685 0.18888 1.000 -0.4986 0.5723 article 0.12666 0.19298 1.000 -0.4204 0.6738 abstract video -0.26634 0.18888 1.000 -0.8018 0.2691 abstract video video 0.26179 0.17986 1.000 -0.2481 0.7717 video article 0.30319 0.18350 0.999 -0.2170 0.8234 article 0.39300 0.18772 0.374 -0.1392 0.9252 abstract 0.26634 0.18888 1.000 -0.2691 0.8018 table a5. bonferroni’s pairwise comparison (average attitude change t2-t1) australasian journal of information systems mirkovski, doyle & gaskin 2025, vol 29, research article practitioners’ use of information systems research 41 appendix 5: exemplary evidence experiment treatment codes interview evidence video video’s content visuals are good “i guess with a cartoon you just take it at face value, don’t you? you don’t look terribly closely at the cartoon, not like a person’s face; you examine a person’s face a bit more than you would a cartoon, so probably a cartoon is probably good for that sort of message, to be honest” – participant a “i liked that there was some kind of personification in the video but then it kind of felt like, the way that the story telling worked, i felt like i was starting to get to know a host character in the video” – participant d “ah, obviously really, i would say practical and, ah, self-explanatory. so, helps, ah, to improve the understanding” – participant j abstract abstract’s content lacks visuals “and usually if we are going to use some kind of research material thing, it could be more like well-done research. something like, we would prefer to see a well-organized results or finding first, and maybe more visualized way and with some kind of an easier to understand” – participant h “it is quite a lot of information and well i think it would be more like efficient, if we can have some kind of a workflow or process flow that we can follow so it gives you some kind of a preface, some kind of an introduction that i can just in ten seconds, and i can know what it is talking about” – participant h “ah, yeah, it would’ve been great to see just a high-level diagram of the model and the factors in it that presents everything all at once – that would’ve been really nice” – participant i “and for example, i noticed, um, there is, ah, it introduced, did some approaches, if oh if there is a flowchart to show the approaches, how the approaches are going, i think it would be better to, to read” – participant k article article’s content is too abstract and theoretical “talking about concepts in the abstract at quite a technical level often tends not to be communicated quite well down to the customer” – participant h “so, in terms of tone and style they tend, these papers tend to adhere to quite an academic tone and style” – participant h abstract video abstract video’s content has good relevance “[authors] done all the research and them talking about the best, summarising the best ways that you’ve found through all those studies and all the evidence and going through and looking at how customers were involved and how to look at existing solutions as well and the development later with the customer was really good as key takeaways” – participant c “i think it's always useful when someone else has already done the research and you can come across it for a start because it saves people a lot of time” – participant c video article video article’s content has good applicability “so, there were some process elements in the design phase where that absolutely could've been applicable, but it's very much a settled process now and we're overlaying the technology to support that at this point” – participant e “if i was doing a business process job, i could definitely believe that it's something that would be absorbed and reapplied” – participant e “yeah. no, it seemed, yeah, modern and applicable” – participant g table a6. exemplary evidence for the content theme of the thematic analysis in study 2 copyright © 2025 mirkovski, k., doyle, c., gaskin, j. this is an open-access article licensed under a creative commons attribution-noncommercial 4.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi.org/10.3127/ajis.v29.5047 https://creativecommons.org/licenses/by-nc/4.0/deed.en https://creativecommons.org/licenses/by-nc/4.0/deed.en paper title (paper title style) australasian journal of information systems pham, ulhaq, nguyen & nkhoma 2021, vol 25, research article knowledge sharing and cyber security behaviour 1 an exploratory study of the effects of knowledge sharing methods on cyber security practice hiep cong pham rmit university vietnam viet nam hiep.pham@rmit.edu.vn irfan ulhaq rmit university vietnam viet nam minh nhat nguyen rmit university vietnam viet nam mathews nkhoma rmit university vietnam viet nam abstract in a networked global economy, cyber security threats have accelerated at an enormous rate. the security infrastructure at organisational and national levels are often ineffective against these threats. as a result, academics have focused their research on information security risks and technical perspectives to enhance human-related security measures. to further extend this trend of research, this study examines the effects of three knowledge sharing methods on user security practices: security training, social media communication, and local security experts (non-it staff). the study adopts a phenomenological method employing in-depth focus group interviews with 30 participants from eight organisations located in ho chi minh city, vietnam. the study expands on understanding factors contributing to self-efficacy and security practice through various knowledge sharing channels. current methods of periodical training and broadcast emails were found to be less effective in encouraging participants to develop security self-efficacy and were often ignored. security knowledge sharing through social media and local experts were identified as supplementary methods in maintaining employees’ security awareness. in particular, social media is suggested as a preferred channel for disseminating urgent security alerts and seeking peer advice. local security experts are praised for providing timely and contextualised security advice where member trust is needed. this study suggests that provisions of contemporary channels for security information and knowledge sharing between organisations and employees can gain regular attention from employees, hence leading to more effective security practices. keywords: knowledge sharing, social media, cyber security, security compliance 1 introduction the security risks to an organisation’s sensitive information are constantly growing. both external and internal attacks are becoming more sophisticated and persistent (sindiren & ciylan, 2018). juniper research (2017) predicts data breaches will cost $8 trillion globally by 2022. technical measures have been effective and robust in preventing cyber risks from australasian journal of information systems pham, ulhaq, nguyen & nkhoma 2021, vol 25, research article knowledge sharing and cyber security behaviour 2 information security breaches (rocha et al., 2014). however, research also shows that a majority of organisational security incidents are directly or indirectly caused by employees who violate or neglect the information policies of their organisations (ashenden, 2008; sindiren & ciylan, 2018). employee compliance choices are therefore critical to organisational security (sommestad et al., 2014), and overcoming this challenge requires more than technical measures. prior studies have established that users’ personal factors such as attitude, self-efficacy, and perceived response costs associated with security tasks can affect their intention to comply with information security policies and practices (sommestad et al., 2015). additionally, knowledge about cyber security and motivation to protect from cyber risks are necessary to enhance cyber security practice. security self-efficacy, which is the combination of an individual’s security knowledge, skills, and expertise, enables the individual to perform security tasks and cope with changing security requirements. a lack of knowledge about information security leads to low levels of employee engagement in cyber security practice, jeopardising the organisation’s information security (pham et al., 2016). knowledge about cyber security is often achieved through training (puhakainen & siponen, 2010). training often consists of formal dissemination of security information (e.g. policies), as well as procedural, technical knowledge (park et al., 2017). current training methods are often seen as inefficient at disseminating security knowledge to non-technical employees, mainly because of the complexity and technical aspects of information security knowledge (safa and von solms, 2016). lengthy policies and training materials often lack simplicity for an average user and technical information in policy documents remains complex (pham et al, 2019). another way of developing security knowledge can be facilitated through collaborative sharing among employees (hwang & kim, 2007; wang & noe, 2010). the effectiveness of collaborative networks for knowledge sharing is evident in other areas such as medicine (oh, 2012) and hospitality (yang, 2009). building an employee culture of proactively raising awareness and sharing knowledge of information security matters has recently been debated across disciplines (schlienger & teufel, 2002). since humans are key assets of every organisation, and most security concerns are human-based, a knowledge-sharing culture helps organisations build trustworthy partnerships among internal communities (schlienger & teufel, 2002). however, insecure or inappropriate sharing of information assets could also lead to the potential loss of data or information or could put people into vulnerable situations. in such scenarios where individuals are facing a problematic situation, trustworthy advice and clear guidelines for knowledge sharing becomes significant (tamjidyamcholo et al., 2013). although proper security practice can be learned through formalised procedures, there are certain tasks which require more practical instructions and timely advice of information security. an employee might unintentionally violate the security policy of an organisation, which could place them and the organisation at risk. to address this, researchers place a high stake on workplace knowledge sharing through social exchanges and informal peer discussions. safa & von solms (2016) discuss the potential benefits of an information sharing culture and how it can improve staff efficacy regarding security awareness, promote sharing knowledge on policies and procedures, and lead to compliance improvements in security programs. however, few research studies have examined how employees practice knowledge sharing in the context of cyber security (rocha et al., 2014). furthermore, it is not known whether and how such practices affect security behaviour in the workplace. australasian journal of information systems pham, ulhaq, nguyen & nkhoma 2021, vol 25, research article knowledge sharing and cyber security behaviour 3 consequently, this study explores how employees acquire and share security information at work and assesses the impacts of such knowledge sharing practices on employees’ security self-efficacy and practice. the study employed a qualitative phenomenological method of indepth focus group interviews with employees in various organisations located in ho chi minh city, vietnam. the findings contribute to both theoretical and practical aspects of employing various security knowledge sharing methods to influence employees’ efforts to maintain security awareness and practice. 2 significance of knowledge sharing to security awareness and practice organisations manage information security through a combination of administrative and technical measures. administrative measures aim to encourage safe security practices and deter malicious computer abuses through security policies and procedures, awareness training, and security compliance supervision (crossler et al., 2013). however, the effectiveness of administrative measures relies greatly on the levels of awareness and compliance among it system users. technical measures ensure safe security practice through the use of authentication, authorisation, data encryption, and antivirus software, as well as a myriad of hard and soft system elements designed to prevent breaches and encourage desired behaviours (powell, 2018). previous research has shown that knowledge sharing among users within organisations is an effective way to increase awareness of and compliance with information security policies (mallinder & drabwell, 2013; safa & von solms, 2016). given the increasing number of cyber risks, effective real-time knowledge sharing could help employees protect the organisation against potential security risks (torres & gupta, 2018). to decrease cyber risks from users, the risks need to be apparent to users and they need to understand their role in decreasing these risks. if users are not aware of cyber security issues, and if they do not have sufficient self-efficacy to respond, security threats cannot be addressed (safa et al., 2016). in addition to awareness of security risks, knowledge about appropriate responses is necessary if cyber risks are to be reduced. awareness of the risk is necessary, but is insufficient at reducing risk alone; people must also know how to act in situations of security breaches (popovac & fine, 2018). security risk awareness and skills are generally referred as self-efficacy, which encompasses “a belief in one’s capability to protect information and information systems from unauthorised disclosure, modification, loss, destruction, and lack of availability” (rhee et al., 2009). high levels of self-efficacy mean knowledgeable and skilful employees are more likely to take protective security tasks. a lack of self-efficacy has also been found to be a major contributor to cyber security compliance disengagement (pham et al., 2016; pham et al., 2019). recent literature on knowledge sharing advocates a socio-technical perspective and posits that knowledge does not only reside in documents and systems but also in people’s minds (brown & duguid, 1999; nonaka & takeuchi, 1995). furthermore, knowledge sharing is a joint approach between sender and receiver, resulting in individual learning, and positively impacts organisational learning (ipe, 2003). it is important for organisations to understand that tacit knowledge (or ‘know-how’) is a sticky concept, hard to share completely, and is not always easy for a receiver to understand at full scale (nonaka et al., 2006). another factor that influences cyber security compliance is that of task complexity. complex tasks required to secure information assets cannot always be accomplished without an australasian journal of information systems pham, ulhaq, nguyen & nkhoma 2021, vol 25, research article knowledge sharing and cyber security behaviour 4 effective knowledge sharing process (jafari & charband, 2016; zhang et al., 2012). complexity becomes an issue in cyber security when the risk is difficult to comprehend by a lay-user (zhang & costa, 2018). in such cases, peer networks and social learning are useful in decreasing the overall fear of engagement that often pervades a high-risk it environment (warkentin et al., 2011). by applying a social learning theoretical lens to the healthcare sector, warkentin et al. (2011) found that informal sharing methods such as support from colleagues, sharing materials, informal verbal discussions, feedback sessions, and observations helped individuals to improve their security behaviours and compliance with policy. 3 the effect of three knowledge sharing methods on security practice knowledge sharing enables employees to develop ideas, share information about security concerns, and collaborate within the workplace environment about cyber information threats (rocha et al., 2014). cummings (2004) describes two approaches to knowledge sharing: formal methods, including disseminating information security policies and formal training (höne & eloff, 2002), and informal methods, comprising of conversations, peer to peer discussions, and advice-seeking and sharing. given the starting point of multiple methods of knowledge sharing, this study focuses on assessing the effects of three forms of knowledge sharing: formal training programs, social media communication, and local security experts on employees’ security awareness and practice. 3.1 formal security training training programs are a popular form of disseminating knowledge and developing safe information security behaviour in employees (puhakainen & siponen, 2010). training is an efficient way to deliver the theoretical explanations that are necessary for users to understand why and how security compliance can help users protect the information assets of an organisation (clark, 2008). training can also improve self-efficacy and self-regulation of information security behaviour (clark, 2008). however, security training can also change attitudes and behaviours towards a specific issue without users actively thinking or deeply analysing and considering the issue (puhakainen & siponen, 2010). furthermore, training builds on prior understandings and abilities, and ensuring learners’ security knowledge levels are at a commensurate level can be challenging (park et al., 2017). consequently, training on its own is insufficient to completely ameliorate cyber security risks, and other forms of knowledge sharing are necessary. training offers not only a better chance to directly transfer knowledge to the employees, but it also allows the organisations to develop the content with the strong relation to the organisation’s policies (brandl, 2012). knowledge sharing through training and close collaboration can build trust among employees and can further develop a secure environment (liu et al., 2011; safa & von solms, 2016). trust is an important mechanism in enhancing knowledge sharing between employees and is therefore a necessary precursor to securing the cyber environment (feledi et al., 2013). the effectiveness of programs on changing cyber security practices depends on how the information has been shared with users. khan et al (2011) noted that face-to-face training is more effective than computer-based training. this is because users’ cyber security actions can australasian journal of information systems pham, ulhaq, nguyen & nkhoma 2021, vol 25, research article knowledge sharing and cyber security behaviour 5 be built on understanding the consequences of a compromised security system (rhee et al., 2009) and the reasons for developing security policies and procedures (pham et al., 2019), thus face-to-face training can raise cyber security awareness of employees. furthermore, face-toface education is also more economically viable than computer-based training, which usually requires more physical and technical resources. computer-based training, however, has the advantage of one-on-one communication. it allows users to practice at their own pace, which prevents users from becoming overwhelmed by information (khan et al, 2011). since the nature of computer-based training is ready-made, it often lacks direct interactions between organisations and employees, and is difficult to reflect the organisation’s policies (brandl, 2012; khan et al, 2011). 3.2 social media communication of security updates with the dramatic development of communication technologies, virtual communities on social media platforms have emerged as a new way to share knowledge. these communities allow people to share information and experience without meeting face-to-face (chang et al., 2015). furthermore, these communities are available when and where the user needs them, via any device and in a format they can access given their skills and capabilities. rather than providing information passively, social media offers more cooperative and open communication, where a large number of people are free to share any of their thoughts, experiences, opinions, feedback, and perspectives (kaplan & haenlein, 2010). the increased mobility of social media and the popularity of smartphones has made social media use a daily activity (kwahk & park, 2016). social media platforms offer an effective way to acquire new knowledge from peers and networks (wasko & faraj, 2000). social media tools therefore operate as a knowledge management system, as they permit a flexible mode of dialogue and sharing information in several formats (kwahk & park, 2016; oostervink et al., 2016). each of these capabilities enables real-time problem solving by it professionals and others within the community, potentially minimising any damage from cyber threats. in addition to flexible social structures and immediacy, social media enables the dissemination of knowledge and information in multiple forms such as videos, photos, audio, comments, or blog posts. this permits community members to share knowledge in whatever form is most natural for them at the time (kwahk & park, 2016). the role of social media as a knowledge-sharing platform is well recognised, but research on the use of social media in communicating cyber security information is scarce (aloul, 2012; gupta and brooks, 2013; hajli & lin, 2016). this is potentially due to the security risks associated with social media in general (e.g. pattabiraman et al., 2018; zhang & costa, 2018). however, there has been research in a university setting that has shown that social media can be effective in a knowledge sharing context (aloul, 2012). hence, it is timely to evaluate the influence of knowledge sharing through social media on cyber security practice. 3.3 local security experts (non-it professional staff) providing domainspecific knowledge when it comes to delivering cyber security knowledge, time and place of the support is critical. having at least one information security expert or ‘champion’ in each department has been shown to be effective (ashenden, 2008; cherdantseva et al., 2016). a local security expert is responsible for answering security concerns of others, identifying potential threats, and reminding others to comply with security policies. in order to carry out such responsibilities, australasian journal of information systems pham, ulhaq, nguyen & nkhoma 2021, vol 25, research article knowledge sharing and cyber security behaviour 6 the local expert is normally recognised as having more experience and skills than the ‘average’ user. by facilitating knowledge sharing within a specific work community, local experts can also reduce waiting times and costs associated with computer down-time (safa et al., 2016) by being immediately available to assist. these ‘champions’ may not be formally appointed but may be evidenced merely by their contributions to their departments. there are two types of local experts: formal, such as managers, and informal, such as any individual employee who is respected by others when it comes to cyber security. manager’s support has been found to be an important factor in facilitating knowledge sharing. it increases knowledge sharing among employees by reinforcing positive attitudes of employees towards cyber security measures (shafiq et al., 2013). such support can be expressed implicitly through the reactions of supervisors when managing mistakes, consistent and equal treatment of employees, competences of supervisors that relate to technical aspects, and the supervisors’ willingness to support and protect employees in front of other managers (shafiq et al., 2013). managerial support can therefore positively influence the efforts required to adapt to change, user-responsibility towards security issues, levels of collaboration with others, and knowledge sharing (raineri & paillé, 2016). furthermore, employees’ belief in their managers’ knowledge and expertise was found to positively affect employees’ self-reported security breaches (liao, 2008; mittal & dhar, 2015), demonstrating employees’ willingness to co-create a secure environment. with the control and discretionary power to make decisions, line managers are able to allocate training schedules, develop training strategies, build competence programs to increase the relevant knowledge of their employees, provide valuable feedback, create an open working environment, enhance engagement levels, and promote information sharing among employees (kettinger et al., 2015; ramus & steger, 2000). finally, with high information security knowledge and skills, local security experts can be a potential channel for advice on specific security problems and facilitating security knowledge exchange among employees. in summary, many previous studies support the important role of user self-efficacy in achieving safe security practice and efforts to employ security technologies (bulgurcu et al., 2010; johnston and warkentin, 2010; rhee et al., 2009). improving users’ self-efficacy through knowledge and practice sharing is a common approach to any security program to achieve more effective security practice. however, organisations often lack a clear understanding of how various knowledge sharing practices can influence users’ effort to acquire security skills and their resulting practice. this study therefore sets out to ascertain how employees perceive the effectiveness of the three identified knowledge-sharing methods on enhancing their selfefficacy which subsequently improves security practice. the arguments put forward in this study is summarised in figure 1. australasian journal of information systems pham, ulhaq, nguyen & nkhoma 2021, vol 25, research article knowledge sharing and cyber security behaviour 7 figure 1: conceptual model of cyber security knowledge sharing 4 study methodology we focus on developing a deeper understanding of several individuals’ common experiences of acquiring security information through various methods and how such knowledge impacts their and their own daily practice at work. such insights help develop more focused training programs and policies. exploring lived experiences of a phenomenon such as security practice to gain in-depth understanding of underlying reasons contributing to such practice requires a phenomenological method (cilesiz, 2011; creswell, 2007). the purpose of a phenomenological study is to establish essential elements of common experiences among participants (moustakas, 1994). it is important to note that a phenomenological approach acknowledges the key common experience can change and be incomplete (moustakas, 1994). the results of this study therefore do not claim to represent a universal truth, but rather the shared security practices at a specific time and place, as manifest in the participants’ experiences and as seen from the perspective of an individual researcher. another requirement of phenomenological research is that there are no prior assumptions about the correctness or falsity of a participant’s experience of the phenomenon (ashworth, 1999). hence before starting the interviews, we did not start with a set of well-formed hypotheses, but instead looked for common practices that emerged through a series of interviews and used other participants to validate our findings. the participants were encouraged to freely share their own perspectives to provide insight into their security experiences. criterion sampling was employed so that only participants who share or receive relevant security information at work would participate in the study. this sampling technique is recommended for phenomenological study in creswell (2007). for data collection, in-depth interviewing was used to ask the participants how they managed their day-to-day lives and personal experiences in relation to cyber security. in depth interviewing is well suited to collecting personal experience descriptions (creswell, 2007). we approached 12 organisations with whom we had worked in the past for suitable employees to interview. eight organisations agreed to join the study. participants were selected from a range of departments, who could provide diverse perspectives of their security practices. thirty participants participated in eight in-depth focus group interviews, ranging from 60 to 90 minutes (see table 1). the number of interviews was considered sufficient when the participants’ responses provided a clear picture of the three knowledge sharing practices and australasian journal of information systems pham, ulhaq, nguyen & nkhoma 2021, vol 25, research article knowledge sharing and cyber security behaviour 8 their effectiveness on security practice. as little significant new insights were acquired in the last few interviews, the number of interviews was considered to meet the theoretical saturation criterion for qualitative research established by dworkin (2012), which indicates the point where no new information or themes are observed in the data. organisation number of participants position and pseudonyms 1. software retailer 3 software designer (3.1, 3.2, 3.3) 2. financial organisation 1 4 auditors (4.1, 4.2, 4.3, 4.4) 3. financial organisation 2 4 financial specialist (3.1, 3.2, 3.3) 4. financial organisation 3 1 marketing staff (4.1) 1 compliance officer (4.2) 4 financial specialist (4.3, 4.4, 4.5, 4.6) 5. financial organisation 4 3 financial specialist (5.1, 5.2, 5.3) 1 market researcher (5.4) 6. agriculture exporting organisation 1 investor relation associate (6.1) 7. university 2 lecturer (7.1, 7.2) 2 professional staff (7.3, 7.4) 8. marketing organisation 2 advertising designer (8.1, 8.2) 2 marketing staff (8.3, 8.4) table 1: profiles of participants participants’ responses were analysed following the three-step phenomenal analysis outlined in moustakas (1994) – phenomenological reduction, imaginative variation, and synthesis – with the help of text analysis software nvivo 11. in the first step, all interviews were conducted face-to-face in both english and vietnamese (dependent upon the language capabilities of the participants), recorded, transcribed, and translated into english. researchers then went through the transcripts several times, treating all relevant statements as having equal value. the pre-identified themes of information sharing methods and potential impacts on security practice were used to classify coded responses to enable the researchers to discover, comprehend, and gain insights of the participants. this step was designed to group the responses, eliminate repetitions and overlaps, and produce a list of meaning units (words/phrases that represent only one meaning) across participants. individual participants’ descriptions of their experience of security practice were recorded as quotes together with the researchers’ supplementary statements, creating individual textual descriptions. next, individual structural descriptions or the underlying reasons of participants’ day-to-day security experiences were established and transforming participants’ statements into security terminology. the final step of phenomenological analysis consisted of synthesising individual textual and structural descriptions of security practice among participants, and constructing textual-structural synthesis. this composite synthesis comes in the form of a final narrative detailing an in-depth description of security practice of the whole group. the interviews consisted of two parts. the first part asked participants about their security knowledge sharing practices. this part explored how employees discuss and share information related to cyber security within their work environment and their preferred way to share this information. the second part assessed how participants view the effectiveness of australasian journal of information systems pham, ulhaq, nguyen & nkhoma 2021, vol 25, research article knowledge sharing and cyber security behaviour 9 specific knowledge sharing practices on their own security practices. the interview questions are included in appendix a. 5 results 5.1 overview of the roles of security self-efficacy and peer knowledge sharing in security practice when participants were asked to describe their views of the impacts of knowledge sharing on their security practice, there were mixed responses regarding the necessity of security selfefficacy if security tasks are simple or less obvious to them. participants from the university highlighted the insignificant role of security skills in their jobs due to task simplicity or unclear expectations: the it department does all security tasks and they do not require anything from me. so, i am not clear that i must be responsible for the security tasks. (lecturer 7.1) i do not think security training is required. because all security tasks here are quite simple and rare. if users spot a problem, they can report directly to it department. it does not resemble a scenario that can come up in the training. i have not experienced any security incident that needs a training. (professional staff 7.4) other participants thought that security tasks are purely for it professionals or too complex for normal users to assess the risks and take proper care, hence they did not consider it necessary to put effort in maintaining security knowledge and practice: we do not normally update security knowledge, it is necessary to it professionals. we hardly do that because it is not one of our concerns. (financial specialist 4.4) to some participants, the purposes of enhancing security skills and exercising safe practice are not clear, which means they view it as a waste of time: i see that security compliance is only that you have to comply with organisation’s requirements. but i do not see the problem, much on my side why i need to do that. it is much bureaucratic you have to do this, to do that. for the users, i find it too much a waste of time. (lecturer 7.1) many participants, however, acknowledged they can contribute towards security and that self-efficacy is critical and needs to be learned: matching computer skills will make me willing to comply. security tasks could be personalised based on different requirements from different positions in the company. that will support your work better. (market researcher 5.4) other participants also suggested that to encourage effective security information sharing, organisations could develop a collaborative and open communication culture within teams, fostering the free flow of knowledge, including security knowledge, amongst employees: culture of the organisation is important. if people are friendly, the information about the risks will be shared quickly to other staff members in the team. (financial specialist 3.1) most participants agreed they normally did not view sharing security information with their colleagues as an effective way to maintain security awareness or skill building. they did, however, share such information with it professionals. sharing security topics was deemed unnecessary in everyday experiences at work. there emerged several reasons for this assessment. firstly, the rarity of security issues and the relatively low impact of cyber security australasian journal of information systems pham, ulhaq, nguyen & nkhoma 2021, vol 25, research article knowledge sharing and cyber security behaviour 10 on daily work activities meant that it was simply not a topic of conversation. secondly, even when there was talk about it security, the information was complex and difficult to understand without technical expertise. thirdly, participants were not convinced that it security was urgent enough to resolve the issue through peer-support or independently; they felt that notifying it department about the issue would be ‘enough’. fourthly, it security breaches often encompass confidential information and participants felt that it professionals are better placed to protect information assets than their colleagues. finally, even if an employee could solve the security issues, since they have different roles and are not dedicated to supporting it issues, asking for help was seen as a disruption to others. 5.2 formal training for security awareness many participants agreed that training by security professionals is a common method of providing security knowledge and skill enhancement. security training can equip users with basic knowledge of organisational policies and relevant risks. one participant emphasised the relevance and authenticity of the security cases used in the training that urged them to take security matters more seriously. this view is consistent with findings of rhee et al (2009), which indicates that security breach experience can affect perceived self-efficacy and how users exert effort in their security practice. one participant confirmed this: security training needs to show how users’ security compliance is important to protect the business. also, we need mock up exercises through which we can easily understand the severity of the risks and effectiveness of the recommended actions. then we will follow and support security guidelines actively. (marketing staff 8.4) in addition to security training, the efficacy of it professionals is important to ensure participants take security advice. a few participants explained that it demonstrates the organisation can be effective to protect information systems: i will take it staff advice if they show competency, capable of managing the it security risks, and they give us some knowledge if we follow them. they have to prove that they can do something with the risks for my computer first. from that i will take their advice into account, it should come from a qualified it department. (marketing specialist 8.2) it was a consensus among participants that training contents need to be customised for different work contexts due to the specific nature of the risks: people might think that cyber security is something that is not related to their job and do not understand about it. therefore, i think it is necessary to get every employee to engage with the training program. furthermore, the training should be designed to relate to the jobs’ natures to be more relevant and interesting. (software designer 1.2). participants suggested that formal training programs should be offered periodically to reinforce security knowledge and practice, and that the content formats of such trainings should be more diverse, which many organisations may not consider: we do not have much security training at work. i think security was mentioned briefly during the induction for new staff. after that, there is no further training or updates on what security requirements and skills that staff need to know. (financial specialist 3.2). participants who had attended security trainings before often suggested more engaging and interactive formats, not simply a reinforcing of instructions. traditional didactic training australasian journal of information systems pham, ulhaq, nguyen & nkhoma 2021, vol 25, research article knowledge sharing and cyber security behaviour 11 methods were seen as providing too much technical and plain policy information in an unattractive format: everything about rules and regulations needs to be educated in a creative, smart, and attractive way. training should provide new knowledge, excite people so that compliance becomes voluntary, not something that rigid, compulsory so ‘you have to do it’ way. (financial specialist 4.6) sharing the same view of creating more interesting and engaging training programs, a few participants wanted training that was fun to undertake, including interactive game-like activities that would stimulate more interest and facilitate easy security skills acquisition: i think training programs incorporating game-format competitions will attract more engagement from employees. game-format interactions make the learning more fun and realistic rather than having to read the policies that company provide. (auditor 2.3) this request is consistent with research into negative emotions and behaviour change, whereby behaviours are more sustained over time if they are associated with positive emotions (brennan & binney, 2010). 5.3 social media for sharing of information security participants from financial organisations 2, 3, 4, and 5 reported the use of several social media applications at work as an unofficial information sharing channel. groups of stock traders or advisers from these organisations used social media tools such as facebook messenger, skype, and zalo (a vietnamese chat application) to share market-related information. they used these tools because they are popular, provide instant knowledge sharing, and support multimedia content, including visual and audio files. some of these social professional groups comprised more than fifty people from both inside and outside an organisation, although the details of these communications were kept confidential from the researchers. when asked to comment on how it would be useful for the organisation to communicate security topics through social media channels, especially in urgent situations, two participants liked the idea: sending security warnings through skype is very useful in my company because all staff can join and discuss about the problems easily. if the risk is very important and serious, the organisation can send the information through emails and everybody can spread the news further on skype. (finance specialist 3.3) most people in my company use and check notifications on the zalo, therefore, it is easier to get people’s attention by using this social media application rather than through formal email channel, which i normally skip reading. if the messages are sent to my zalo account, maybe i will look at it because it is much shorter, more visual and easier to remember. (auditor 2.1) using social media applications for knowledge sharing was preferred by these participants because of its convenience, timeliness, and because it can influence their security practice. for example, when someone shared a personal experience of a security incident, such as losing information due to a hard disk crash, losing a social account due to poor password usage, or not carefully checking spoof websites, members in the social group took the advice very quickly and personally. participants reported formal training that employed traditional methods and hypothetical scenarios did not help them to fully appreciate the serious consequences of poor security behaviour: australasian journal of information systems pham, ulhaq, nguyen & nkhoma 2021, vol 25, research article knowledge sharing and cyber security behaviour 12 when i heard someone in my work group lost their facebook account or all data on the organisation’s provided computer due to a hard disk malfunction, i immediately changed my password or asked it staff how to back up my data. (marketing staff 8.3) furthermore, participants raised the point that messages shared on social media should be brief (due to the small screens of mobile devices) and visual engaging, to depict the contents more clearly, and to avoid ‘tl;dr syndrome’ (too long; didn’t read). additionally, social media information needs to be framed to directly relate to each group’s interests, to avoid flooding their professional social media with irrelevant updates. too much irrelevant information can lead to messaging being ignored or avoided: we do not want to receive too many security notices on our social channels. only very significant and urgent notifications should be sent to these channels. otherwise we will block or ignore future messages from the it department. (financial specialist 3.3) a few participants suggested that email communications were not as effective as social media as a means of sharing knowledge and improving awareness. this was because the email content was considered plainer and more tedious to read than the visually rich content of social media platforms. while most participants preferred using social media to share important security information to organisational emails, many of them were not aware of the security risks of using social media. they were also not aware of the consequences of disclosing financial information on potentially open and unsecure channels, which may include people outside the organisation. however, if people are using social media anyway to make their information lives easier, organisations may be able to take this into consideration when designing cyber security knowledge sharing systems. 5.4 local security experts (non-it professional staff) for sharing contextspecific security knowledge many participants suggested that new employees often lacked adequate knowledge when dealing with minor security issues, particularly when they were unfamiliar with the issues or the organisation’s requirements. other sources of security information such as training may not be readily available at the time of need and formal policies do not cover all jobs’ security risks. some participants shared that their local security experts, who could be senior staff or simply someone who has more experience of the security context, can be a valuable resource for assisting staff with security protection. for example, one participant explained: at previous workplaces, i did not have any training course on cyber security. however, from starting at this organisation, my senior was the one who trained and gave me advice about the information protection. (university staff 7.4) most participants agreed that a local security expert can be an alternative for employees seeking security advice or solutions, rather than relying solely on the it department. moreover, because a local security expert is perceived as part of the employees’ peer group, they can often provide better and more relevant advice to their colleagues (e.g. based on their experience of the job requirements) than it staff. trust and respect between employees and the local experts were also identified as important. a high level of trust among colleagues and local security experts could initiate open discussions on security issues, in both non-critical and critical incidents. australasian journal of information systems pham, ulhaq, nguyen & nkhoma 2021, vol 25, research article knowledge sharing and cyber security behaviour 13 some participants mentioned that they had taken immediate actions to change passwords and perform backup of their laptops when they heard of security breaches experienced by colleagues, as it was both relevant to their day-to-day activities and genuine enough that it required urgent action. a participant explained: during a lunch, my colleague told her laptop got a hard disk crash and all data was lost. it staff was trying to recover it. i was shocked to think it could happen to me a s well. so, i contact it department to assist with data protection immediately. (professional staff 7.3) others agreed that peer sharing of security topics is important and helpful for their security practice. peer sharing is also both authentic and relevant to solving cyber security problems among employees. for example, one participant stated: information shared from other colleagues is important because it directly relates to my job. furthermore, staff information sharing such as during lunch will help me to understand more about the problems. (auditor 2.4) furthermore, commonly available knowledge through peer sharing can quickly address security issues and enhance the self-efficacy of employees. for example, a market researcher commented: i expect to solve the problem in timely manner. therefore, updating knowledge from surrounding colleagues is important and necessary to prevent risky situations. it is more on the spot support that formal it channel cannot match. (market researcher 5.1) a participant explained the benefit of having a local expert for security advice: because each department has a different policy – for example, a finance department cares more about personal trading policy than the marketing department – having an expert who has experience and knowledge about cyber security in our department is good idea, since they know what problems we usually face during work and we can trust them to ask. (compliance officer 4.2) another participant added how he paid more attention to security warnings from his colleagues than to general security updates from the it department: in my opinion, reminders from colleagues who have high security knowledge are more important to me, which will affect more on organisation’s security effectiveness. (market researcher 5.1) some organisations did not conduct formal orientation for new staff members, who thus did not have efficacy to respond to security issues as expected. therefore, direct senior staff members’ advice can provide new users with specific and unique job-related security knowledge and requirements. sharing knowledge between a designated local security expert and other colleagues is therefore a supplementary approach to enhance security knowledge of employees, which helped the participants to feel more competent in dealing with unknown security issues. table 2 summarises the key findings from the interviews regarding the effects on security information sharing methods on the participants’ security practice. australasian journal of information systems pham, ulhaq, nguyen & nkhoma 2021, vol 25, research article knowledge sharing and cyber security behaviour 14 knowledge sharing methods current sharing practice effects on security awareness and practice email communication, instructor-led security training emails are commonly used as the main communication channel but are often ignored by employees because they are not customised to a specific audience. security trainings are conducted only for new staff; follow-up sessions rarely happen. these methods can be complex and unengaging, and it is difficult to include contextualised security risks. provide good coverage of organisational security policies and procedures. create initial necessity of security awareness, however, the effect does not last without regular repeat sessions. less effective for on-going, urgent security updates. social media communication no interviewed organisations had clear policies on the use of social media at work, though its use was popular among stock trading organisations. suitable for organisations where staff members are using social media for work-related communications. organisations can send quick updates, customise information to targeted recipients, and use more attractive contents. members in the social group tend to know and trust each other to share security confidential information. use of smart phones and social media applications are very common in vietnam. attract immediate attention and caution from staff members because security risks are shared by someone they know. effective in spreading urgent warnings because users use social applications more often than emails. concern that confidential security information can easily spread out outside of organisational controls. local security experts senior staff members with good experience and knowledge of technology and applications used in respective departments can provide highly relevant and domain-specific advice. more convenient and timely to seek advice than through formal it support channels. highly effective for timely, jobspecific security advice when trust is important. staff can respect and follow their advice. provide effective feedback to novices or end-users without attending formal trainings. can act as facilitator/whistle blower on security practices. table 2: summary of study findings 6 discussion this study set out to explore how three methods of knowledge sharing – formal security training, social media, and local security experts (non-it professional staff) – impacted employees’ security practices. thirty employees from eight organisations in finance, marketing and higher education in ho chi minh city, vietnam participated in eight in-depth focus group interviews. the findings contribute to both theoretical development in terms of self-efficacy and communication impact on security practice and practical steps that organisations can implement to support and encourage employees’ information security compliance. 6.1 theoretical contributions our study expands existing understandings of how self-efficacy can be developed and how its development can influence security practice. it does this through examining the impacts of australasian journal of information systems pham, ulhaq, nguyen & nkhoma 2021, vol 25, research article knowledge sharing and cyber security behaviour 15 three knowledge sharing methods on participants’ efforts to acquire security self-efficacy and to take recommended actions. rhee et al (2009) put forward and found support of three determinants of security self-efficacy – general computer experience, security breach experience, and general controllability – which in turn affect security practice (technology use and care behaviour) and efforts to strengthen security. this study elaborates further on these findings, exploring how such security breach experiences and the controllability of security risks can be acquired and built up through knowledge sharing through social media and local security experts. knowledge obtained from these methods enhances employees’ confidence to deal with security threats and motivates them to be committed to proper security practice. in addition, social media applications facilitate social group discussions, where people can comment and contribute to the information security problems in an informal setting, and modelling of appropriate behaviour is shared amongst peers. we posit that an employee’s information security practices can be enhanced through learning in everyday activities from peers and local security experts. given the complexity of security issues and the apparent lack of timely training, local security experts can also provide task-specific and timely advice in response to an immediate threat (raineri & paillé, 2016; shafiq et al., 2013). as a point of close contact, local security experts can facilitate regular exchanges of best practices on similar issues and develop a communal approach to mitigating security challenges. our study answers rhee et al (2009)’s call to investigate the impact of indirect learning (learning from peers) and social persuasion (group discussion and sharing of proper practice) on security practice. furthermore, findings of this study confirm the arguments made in earlier studies that alternative communication methods are required to disseminate security information that can instil the importance of security practice and the needs to enhance self-efficacy (willison et al., 2018). alternatively, short and regular communications should complement annual training to maintain employees’ ongoing security awareness (barlow et al, 2018). we argue that different kinds of security information need different communication methods to attract users’ attention and actions. security policies and procedures can be shared via formal emails, and urgent security risk warnings should be delivered to specific user groups in media-rich and short messages via social media. work-specific security issues can be initiated from local security experts who know the users and their specific requirements. 6.2 practical implications from a practical perspective, the findings suggest organisations should look at how security trainings and communication methods can be designed to attract employees’ attention, justify their effort, enable employees to acquire necessary skills, and promote the notion of peer sharing of security issues and knowledge that may not be timely and adequately covered through formal channels. security professionals should use a mix of communication channels, such as emails for general and periodical security updates and sms and social media for urgent and critical risk updates. particularly, our findings show that organisations can adopt social media tools to communicate security issues, taking advantage of their availability and accessibility on mobile devices (kwahk & park, 2016). while there are some caveats to this (e.g. public versus private sphere usage), people carry their mobile devices with them most of the time and disseminating urgent security messages through social media can therefore reach most people almost instantly. australasian journal of information systems pham, ulhaq, nguyen & nkhoma 2021, vol 25, research article knowledge sharing and cyber security behaviour 16 organisations should promote an open and trusted information security sharing culture and establish an infrastructure for employees to freely and conveniently share or report security risks at the workplace. lacking a culture that encourages peer knowledge sharing can lead to higher risks, as found by tamjidyamcholo et al. (2014). for example, learning about authentic security incidents from peers can have greater impacts on one’s precautions in protecting organisational information assets (furnell & rajendran, 2012). however, to facilitate sensitive security information sharing, appropriate protocols need to be established providing guidance on what can be shared, in what formats, and on which channels. this study found employees considered sharing security topics among peers may not be appropriate due to a lack of trust and clear procedures. providing true-to-life security incidents in periodical security trainings is also important to demonstrate the real effects of security threats and taking recommended security measures (willems & meinel, 2012). as found by rhee (2009), experience of previous security incidents can reduce one’s perceived self-efficacy; hence those who have not experienced such incidents may underestimate the seriousness of security risks. organisations should consider using gamification, simulation, and virtual scenarios to more effectively convey security information. gamification approaches enable the creation of an active and collaborative working environment with higher motivation and enthusiasm, leading to better performance of employees (burke, 2016). concomitant to concerns about the format of training is the timing of delivery. it trainings should be organised for new hires and reinforced periodically. lacking regular professional trainings is clearly evident in the organisations in this study, making the employees feel uninformed about the criticality of security knowledge and practice in their daily work. 7 conclusion and future research maintaining security awareness is an important element in ensuring employees’ compliance with security practice and an organisation’s overall information security. using in-depth interviews with users in eight organisations in vietnam, our study explored how knowledge sharing through security trainings, social media, and local non-it experts impacted employees’ security awareness. while relatively narrow in focus, the study’s findings contribute to both theoretical and practical aspects of improving employees’ security practices. this study reaffirms the importance of developing self-efficacy through acquiring security awareness in users’ security practices. it adds peer knowledge sharing as another dimension of self-efficacy, providing an additional way to improve security awareness. this study supports the role of sharing and giving security advice among peers, such as social groups, local experts, and professional it staff. for security practitioners, the study’s findings highlight required changes in their training and communication methods to ensure employees’ continued interest in and attention to security practice. it is important for organisations to develop robust knowledge sharing systems with supporting channels that utilise the latest mobile and social technologies and the expertise among employees. employees normally take short-term precautions regarding security risks (e.g. are on high alert after a major security incident then ignore them soon afterwards). through regular social media sharing and local experts, security risks and measures can be spread out to targeted audiences in a timely and personalised fashion. however, precautions need to be taken when sensitive security information is shared using social media. australasian journal of information systems pham, ulhaq, nguyen & nkhoma 2021, vol 25, research article knowledge sharing and cyber security behaviour 17 this study has some limitations due to the nature of the research methodology. the small sample size of this study may not generalise the findings on a larger scale or establish a tested model of knowledge sharing methods and their impacts on security practice. the findings presented in this study mainly came from the employees’ views toward security practice. acquiring comparable responses from the security practitioners in each organisation would assess the practicality of new security communication channels, such as using social media for distributing security updates, which does carry additional risks. future studies should conduct quantitative surveys to validate the findings of this study and establish a model of knowledge sharing, for better replication of results across different organisational contexts. such research would quantitatively ascertain the impacts of different information sharing methods on employees’ security practice. subsequently, more studies should investigate the impact of different antecedents of self-efficacy on users’ security behaviour. finally, it is important to acknowledge the inherent risks from sharing confidential information on social media. it is important to develop secure and private social media tools to protect the organisation. further research could also investigate and validate the contradicting impacts of social media on employees’ security behaviour and corporate confidentiality. references aloul, f. a. (2012). the need for effective information security awareness. journal of advances in information technology, 3, 176-183. doi:10.4304/jait.3.3.176-183 ashenden, d. (2008). information security management: a human challenge? information security technical report, 13, 195-201. doi:10.1016/j.istr.2008.10.006 ashworth, p. (1999). ‘‘bracketing’’ in phenomenology: renouncing assumptions in hearing about student cheating, international journal of qualitative studies in education, 12(6), 707–721. doi: 10.1080/095183999235845 barlow, j., warkentin, m., ormond, d. & dennis, a. (2018). don't even think about it! the effects of antineutralization, informational, and normative communication on information security compliance. journal of the association for information systems, 19, 689-715. doi:10.17705/1jais.00506 brandl, d. (2012). 3 pillars of industrial cyber security. control engineering, 59, 8 brennan, l. & binney, w. (2010). fear, guilt and shame appeals in social marketing. journal of business research, 63, 140-146. doi:10.1016/j.jbusres.2009.02.006 brown, j.s. & duguid, p. (1999), balancing act: how to capture knowledge without killing it, harvard business review, 78, 3, 73-80 bulgurcu, b., cavusoglu., h. & benbast, i. (2010). information security policy compliance: an empirical study of rationality-based beliefs and information security swareness, mis quarterly, 34, 3, 523-548. doi: 10.1016/j.cose.2020.101963 burke, b. (2016). gamify: how gamification motivates people to do extraordinary things, routledge. australasian journal of information systems pham, ulhaq, nguyen & nkhoma 2021, vol 25, research article knowledge sharing and cyber security behaviour 18 chang, c. m., hsu, m. h. & lee, y. j. (2015). factors influencing knowledge-sharing behavior in virtual communities: a longitudinal investigation. information systems management, 32, 331340. doi:10.1080/10580530.2015.1080002 cherdantseva, y., hilton, j., rana, o. & ivins, w. (2016). a multifaceted evaluation of the reference model of information assurance & security. computers & security, 63, 45-66. doi:10.1016/j.cose.2016.09.007 clark, r. c. (2008). building expertise: cognitive methods for training and performance improvement, san francisco, ca, john wiley & sons cilesiz, s. (2011). a phenomenological approach to experiences with technology: current state, promise, and future directions for research, education tech research dev, 59, 487–510. doi:10.1007/s11423-010-9173-2 creswell, j. w. (2007). qualitative inquiry and research design: choosing among five approaches (2nd ed.), thousand oaks, ca: sage. crossler, r. e., johnston, a. c., lowry, p. b., hud, q., warkentin, m. & baskerville, r. (2013). future directions for behavioral information security research. computer & security, 32, 90-101. doi:10.1016/j.cose.2012.09.010 cummings, j. n. (2004). work groups, structural diversity, and knowledge sharing in a global organization. management science, 50, 352-364. doi:10.1287/mnsc.1030.0134 dworkin, s. l. (2012). sample size policy for qualitative studies using in-depth interviews. archives of sexual behavior, 41, 1319. doi:10.1007/s10508-012-0016-6 feledi, d., fenz, s. & lechner, l. (2013). toward web-based information security knowledge sharing. information security technical report, 17, 199-209. doi:10.1016/j.istr.2013.03.004 furnell, s. & rajendran, a. (2012). understanding the influences on information security behaviour. computer fraud & security, 2012, 12-15. doi:10.1016/s1361-3723(12)70053-2 gupta, r. & brooks, h. (2013). using social media for global security, john wiley & sons hajli, n. & lin, x. (2016). exploring the security of information sharing on social networking sites: the role of perceived control of information. journal of business ethics, 133, 111-123. doi:10.1007/s10551-014-2346-x höne, k. & eloff, j. h. p. (2002). information security policy — what do international information security standards say? computers & security, 21, 402-409. doi:10.1016/s01674048(02)00504-7 hwang, y. & kim, d. j. (2007). understanding affective commitment, collectivist culture, and social influence in relation to knowledge sharing in technology mediated learning. ieee transactions on professional communication, 50, 232-248. doi: 10.1109/tpc.2007.902664 ipe, m. (2003), knowledge sharing in organizations: a conceptual framework, human resource development review, 2, 4, 337-359. doi: 10.1177/1534484303257985 australasian journal of information systems pham, ulhaq, nguyen & nkhoma 2021, vol 25, research article knowledge sharing and cyber security behaviour 19 jafari, n. n. & charband, y. (2016). knowledge sharing mechanisms and techniques in project teams: literature review, classification, and current trends. computers in human behavior, 62, 730742. doi:10.1016/j.chb.2016.05.003 johnston, a. c. & warkentin, m. (2010). fear appeals and information security behaviors: an empirical study. management information systems quarterly, 34, 549-566. doi:10.2307/25750691 juniper research. (2017). cybercrime & the internet of threats [online]. available: https://www.juniperresearch.com/document-library/white-papers/cybercrime-the-internet-ofthreats-2017 [accessed 30 may 2018] kaplan, a. m. & haenlein, m. (2010). users of the world, unite! the challenges and opportunities of social media. business horizons, 53, 59-68. doi:10.1016/j.bushor.2009.09.003 kettinger, w. j., li, y., davis, j. m. & kettinger, l. (2015). the roles of psychological climate, information management capabilities, and it support on knowledge-sharing: an moa perspective. european journal of information systems, 24, 59-75. doi:10.1057/ejis.2013.25 khan, b., alghathbar, k. s., nabi, s. & khan, m. (2011). effectiveness of information security awareness methods based on psychological theories. african journal of business management, 5, 10862-10868. doi:10.5897/ajbm11.067 kwahk, k. y. & park, d. h. (2016). the effects of network sharing on knowledge-sharing activities and job performance in enterprise social media environments. computers in human behavior, 55, 826-839. doi: 10.1016/j.chb.2015.09.044 liao, l. f. (2008). impact of manager's social power on r&d employees' knowledge-sharing behaviour. international journal of technology management, 41, 169-182. doi:10.1504/ijtm.2008.01599 liu, d., ji, y. & mookerjee, v. (2011). knowledge sharing and investment decisions in information security. decision support systems, 52, 95-107. doi:10.1016/j.dss.2011.05.007 mallinder, j. & drabwell, p. (2013). cyber security: a critical examination of information sharing versus data sensitivity issues for organisations at risk of cyber attack. journal of business continuity & emergency planning, 7, 103. doi:retrieved from: https://www.henrystewartpublications.com/jbcep mittal, s. & dhar, r. l. (2015). transformational leadership and employee creativity: mediating role of creative self-efficacy and moderating role of knowledge sharing. management decision, 53, 894-910. doi:10.1108/md-07-2014-0464 moustakas, c. 1994. phenomenological research methods. thousand oaks, ca: sage nonaka, i. & takeuchi, h. (1995), the knowledge creation company: how japanese companies create the dynamics of innovation, new york: oxford university press nonaka, i., von krogh, g. & voelpel, s. (2006), organizational knowledge creation theory: evolutionary paths and future advances, organization studies, 27, 8, 1179-1208. doi: 10.1177/0170840606066312 oh, s. (2012). the characteristics and motivations of health answerers for sharing information, knowledge, and experiences in online environments. journal of the american society for information science and technology, 63, 543-557. doi:10.1002/asi.21676 australasian journal of information systems pham, ulhaq, nguyen & nkhoma 2021, vol 25, research article knowledge sharing and cyber security behaviour 20 oostervink, n., agterberg, m. & huysman, m. (2016). knowledge sharing on enterprise social media: practices to cope with institutional complexity. journal of computer‐mediated communication, 21, 156-176. doi:10.1111/jcc4.12153 park, s.-k., lee, s.-h., kim, t.-y., jun, h.-j. & kim, t.-s. (2017). a performance evaluation of information security training in public sector. journal of computer virology and hacking techniques, 13, 289-296. doi: 10.1007/s11416-017-0305-7 pattabiraman, a., srinivasan, s., swaminathan, k. & gupta, m. (2018). fortifying corporate human wall: a literature review of security awareness and training. information technology risk management and compliance in modern organizations. igi global pham, c. h., el-den, j. & richardson, j. (2016). stress-based security compliance model-an exploratory study. journal of information and computer security, 24, 326-347. doi:10.1108/ics-10-2014-0067 pham, c.h, brennan, l and furnell, s., (2019). information security burnout: identification of sources and mitigating factors from security demands and resources. journal of information security and applications, 46, 96-107. doi:10.1016/j.jisa.2019.03.012 popovac, m. & fine, p. (2018). an intervention using the information-motivation-behavioral skills model: tackling cyberaggression and cyberbullying in south african adolescents. reducing cyberbullying in schools: international evidence-based best practices, 225 powell, j. (2018). an introduction to systems theory: from hard to soft systems thinking in the management of complex organizations. complexity and healthcare organization. crc press puhakainen, p. & siponen, m. (2010). improving employees’ compliance through information systems security training: an action research study. mis quarterly, 34, 757–778. doi:10.2307/25750704 raineri, n. & paillé, p. (2016). linking corporate policy and supervisory support with environmental citizenship behaviors: the role of employee environmental beliefs and commitment. journal of business ethics, 137, 129-148. doi:10.1007/s10551-015-2548-x ramus, c. & steger, u. (2000). the roles of supervisory support behaviors and environmental policy in employee "ecoinitiatives" at leading-eege european companies. academy of management journal, 43, 605. doi:10.2307/1556357 rhee, h.-s., kim, c. & ryu, y. u. (2009). self-efficacy in information security: its influence on end users' information security practice behavior. computers & security, 28, 816-826. doi:10.1016/j.cose.2009.05.008 rocha flores, w., antonsen, e. & ekstedt, m. (2014). information security knowledge sharing in organizations: investigating the effect of behavioral information security governance and national culture. computers & security, 43, 90-110. doi:10.1016/j.cose.2014.03.004 safa, n. s. & von solms, r. (2016). an information security knowledge sharing model in organizations. computers in human behavior, 57, 442-451. doi: 10.1016/j.chb.2015.12.037 safa, n. s., von solms, r. & furnell, s. (2016). information security policy compliance model in organizations. computers & security, 56, 70-82. doi:10.1016/j.cose.2015.10.006 https://protect-au.mimecast.com/s/mcsdcd1jzpuly8lbhbh7_w?domain=apac01.safelinks.protection.outlook.com https://protect-au.mimecast.com/s/mcsdcd1jzpuly8lbhbh7_w?domain=apac01.safelinks.protection.outlook.com australasian journal of information systems pham, ulhaq, nguyen & nkhoma 2021, vol 25, research article knowledge sharing and cyber security behaviour 21 schlienger, t. & teufel, s. (2002). information security culture. security in the information society. springer shafiq, m., zia-ur-rehman, d. m. & rashid, m. (2013). impact of compensation, training and development and supervisory support on organizational commitment. compensation & benefits review, 45, 278-285. doi:10.1177/0886368713515965 sindiren, e. & ciylan, b. (2018). privileged account management approach for preventing insider attacks. international journal of computer science and network security, 18, 33-42 sommestad, t., hallberg, j., lundholm, k. & bengtsson, j. (2014). variables influencing information security policy compliance: a systematic review of quantitative studies. information management & computer security, 22, 42–75. doi:10.1108/imcs-08-2012-0045 sommestad, t., karlzén, h. & hallberg, j. (2015). the sufficiency of the theory of planned behavior for explaining information security policy compliance. information and computer security, 23, 200-217. doi:10.1108/ics-04-2014-0025 tamjidyamcholo, a., bin baba, m. s., tamjid, h. & gholipour, r. (2013). information security – professional perceptions of knowledge-sharing intention under self-efficacy, trust, reciprocity, and shared-language. computers & education, 68, 223-232. doi:10.1016/j.compedu.2013.05.010 tamjidyamcholo, a., bin baba, m. s., shuib, n. l. & rohani, v. a. (2014). evaluation model for knowledge sharing in information security professional virtual community. computers & security, 43, 19-34. doi:10.1016/j.cose.2014.02.010 torres, h. g. & gupta, s. (2018). the misunderstood link: information security training strategy wang, s. & noe, r. a. (2010). knowledge sharing: a review and directions for future research. human resource management review, 20, 115-131. doi:10.1016/j.hrmr.2009.10.001 warkentin, m., johnston, a. c. & shropshire, j. (2011). the influence of the informal social learning environment on information privacy policy compliance efficacy and intention. european journal of information systems, 20, 267-284. doi:10.1057/ejis.2010.72 wasko, m. m. & faraj, s. (2000). “it is what one does”: why people participate and help others in electronic communities of practice. the journal of strategic information systems, 9, 155-173. doi: 10.1016/s0963-8687(00)00045-7 willems, c. & meinel, c.(2012) online assessment for hands-on cyber security training in a virtual lab. global engineering education conference (educon), 2012 ieee, 1-10 willison, r., warkentin, m. & johnston, a. c. (2018). examining employee computer abuse intentions: insights from justice, deterrence and neutralization perspectives. information systems journal, 28, 266-293. doi:10.1111/isj.12129 yang, j.-t. (2009). individual attitudes to learning and sharing individual and organisational knowledge in the hospitality industry. the service industries journal, 29, 1723-1743. doi:10.1080/02642060902793490 australasian journal of information systems pham, ulhaq, nguyen & nkhoma 2021, vol 25, research article knowledge sharing and cyber security behaviour 22 zhang, s. & costa, s. (2018). mobile phone usage patterns, security concerns, and security practices of digital generation. international journal of mobile human computer interaction (ijmhci), 10, 23-39. zhang, x., pablos, p. o. d. & zhou, z. (2012). effect of knowledge sharing visibility on incentive-based relationship in electronic knowledge management systems: an empirical investigation. computers in human behavior, 29, 307-313. doi:10.1016/j.chb.2012.01.029 copyright: © 2021 pham, ulhaq, nguyen & nkhoma. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v25i0.2177 http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems pham, ulhaq, nguyen & nkhoma 2021, vol 25, research article knowledge sharing and cyber security behaviour 23 appendix a topic 1: overall view of cyber security knowledge sharing 1. what do you think about the importance of updating cyber security knowledge in regard to your current job? 2. how often do you share information related to cyber security knowledge with your colleagues? 3. who is the one that you usually share cyber security information or experience? why? 4. what are your expectations from the it teams (abilities, skills, enthusiasm and engagement)? topic 2: effectiveness of knowledge sharing methods 1. which method of communication makes you feel more interested in cyber security topics? 2. how can the company’s spending on cyber security training programs affect your attitude/effort in putting effort to assist? 3. how cyber security training has been employed in your company? how do you think about its effectiveness? 4. how often do you use social media to share information with your colleagues at work? do you usually share cyber security information to your colleagues via social media? 5. how do you think about the effectiveness of using social media on sharing cyber security information? 6. what do you think about staff sharing and support on cyber security topics? would you be more interested if cyber security is shared/communicated through colleagues rather than a formal channel like security policies/it policies? 7. what do you think about having someone who has rich knowledge of cyber security as one of your colleagues? does it facilitate your willing to share cyber security information at work? 8. how do you think about the effectiveness of having a local it expert (non-it staff) as your colleagues on solving cyber security problem and improving your cyber security knowledge? paper title (paper title style) australasian journal of information systems balakrishnan, das & chattopadhyay 2020, vol 24, research article implementing data strategy 1 implementing data strategy: design considerations and reference architecture for data-enabled value creation radhakrishnan balakrishnan indian institute of management raipur, india radhakrishnan.efpm2016@iimraipur.ac.in satyasiba das indian institute of management raipur, india manojit chattopadhyay indian institute of management raipur, india abstract with the arrival of big data, organizations have started building data-enabled customer value propositions to increase monetizing and cost-saving opportunities. organizations have to implement a set of guidelines, procedures, and processes to manage, process and transform data that could be leveraged for value creation. this study has approached the journey of an organization towards data-enabled value creation through four levels of data processing, such as data extraction, data transformation, value creation, and value delivery. this study has critical inferences on using data management solutions such as rdbms, nosql, newsql, big data and real-time reporting tools to support transactional data in internal systems, and other types of data in external systems such as social media. the outcome of this study is a methodological technology independent data management framework an organization could use when building a strategy around data. this study provides guidelines for defining an enterprise-wide data management solution, helping both the academicians and practitioners. keywords: data management; data management reference architecture; data strategy; design considerations for big data; governance for big data; data-enabled value creation; 1 introduction organizations across industries consider data as an essential factor of production similar to other critical assets such as labour, capital, and land. in the past, organizations were primarily focusing on enterprise-specific structured data, and of late, organizations have started collecting data regardless of the size, structure, and the speed at which it is created (manyika et al., 2011). big data is no longer a buzz word used by a select number of technology companies and consultants. today, most organizations have to deal with large volumes of data as their interactions with stakeholders are recorded with increasing level of granularity. the initial definition of big data confining only to volume has expanded to include velocity at which the data gets created, variety such as structured, semi-structured and unstructured formats, veracity or accuracy of data, and the value that gets generated through data insights (t. h. davenport, 2014). there is value hidden within big data collected by organizations, and the effective extraction and application of that value have become a key differentiator in the battle for supremacy in the marketplace. australasian journal of information systems balakrishnan, das & chattopadhyay 2020, vol 24, research article implementing data strategy 2 traditionally, organizations have been collecting and analysing data from various transactional systems to identify patterns and arrive at inferences with which they have enhanced operational efficiency and reduced cost. now there is a plethora of additional data, such as extensive customer and supplier information, detailed product catalogues, more information surrounding online transactions, and an increase in the volume of communication with third parties, including email and voice recording systems. this is further supplemented with data available on social media (sm) that exist outside of the organization. with this barrage of data, organizations have to identify the right means to store, retrieve and process data to arrive at actionable inferences. the study on data management has received extensive attention in the recent past, and some of the topics covered in data management are summarized in table 1. in the literature related to data management, ‘how do organizations build a robust data architecture to support different types of data management systems, and what are the critical elements of the proposed architecture that enable data-driven value generation?’ has been identified as a less researched topic. to this end, the proposed study attempts to arrive at a reference architecture, including different types of data management solutions, and how these systems can combine to drive value-creation for customers. there are two objectives of this study: (i) to identify how organizations support a variety of data management solutions from traditional rdbms to newsql, and the recommended data architecture for supporting data-enabled value creation, and (ii) to understand the design considerations in implementing the proposed architecture. this study is organized as follows: in section-2, the study presents a literature review on the need for data management solutions to create data-enabled value creation. in section-3, the study covers the research methodology, research setup, and data analysis associated with the investigation. the findings are presented in section-4, followed by a discussion on the findings in section-5, and the conclusion in section-6. the study has identified a methodological technology independent framework an organization could use when building a strategy around data-enabled value creation and delivery. aspects of data management notable contributions fundamentals on data analytics (t. h. davenport, harris, long, & jacobson, 2001) data analysis on web 3.0 (hendler, 2009; spivack, 2011) data management – value creation framework (lim et al., 2018) data management – big data based value creation (mazzei & noble, 2017) data management – building blocks for a data-driven culture (anderson & li, 2017) data strategy – guidelines for development (bowen & smith, 2014) data management – usage of relational database management systems (rdbms) and nosql (link & prade, 2019; pokorny, 2013) data management – migration from rdbms to nosql (rocha, vale, cirilo, barbosa, & mourao, 2015) data management – features of nosql systems (mohan, 2013; pavlo & aslett, 2016) data management – use of newsql systems (kaur & sachdeva, 2017; simsek, 2019) table 1: aspects of data management and notable research contributions australasian journal of information systems balakrishnan, das & chattopadhyay 2020, vol 24, research article implementing data strategy 3 2 literature review the first section of the literature review discusses how data strategy is critical for an organization, and the design considerations for data-enabled value creation. the second section discusses the use of traditional relational database management systems (rdbms) to store structured data and provides a brief on the issues related to rdbms. the third section provides an overview of how nosql systems could be used to overcome some of the problems related to rdbms. the fourth section discusses newsql systems and how they could be leveraged to build data management solutions. the last section discusses the need for data governance and critical elements of data governance. 2.1 data strategy – a critical component of web 3.0 john markoff of the new york times had coined the term web 3.0 to refer to the intelligent web such as the “applications using semantic web, natural language search, data mining, machine learning (ml), and artificial intelligence (ai) technologies that emphasize machine-facilitated understanding of information to provide a more productive and intuitive user experience” (spivack, 2011). web 3.0 technologies enable firms to respond to external changes quickly by integrating data and applications and providing the ability to “infer relationships between data available in different applications or different parts of the same application” (hendler, 2009). web 3.0 comprises of technologies such as ubiquitous connectivity, network computing, open technologies, and the intelligent internet (spivack, 2011). the intelligent web consists of a variety of technology solutions associated with data stores and data processing using natural language processing (nlp), ml and ai. in the existing literature on data management, the terms data and information have often been used interchangeably. this study differentiates data and information based on the widely popular data-information-knowledge hierarchy (braganza, 2004; t. h. davenport et al., 2001). data are facts and observations, and in a given context, processed data becomes information (zack, 1999). the information that leads to results becomes part of the knowledge. data analytics is defined as the transformation of quasi-finished data into useful insights and knowledge (t. h. davenport et al., 2001). the quasi-finished data goes through a process of cleansing, amending, and augmenting before going through further processing by statistical techniques, computer programs, and manual analysis. organizations consider knowledge as a strategic asset, and organizations have to create, capture, and share knowledge to remain competitive (zack, 1999). organizations implement knowledge management mechanisms by building repositories of explicit knowledge, defining roles to execute and manage the knowledge accumulation process, and developing solutions using information technologies to support knowledge repositories and processes (zack, 1999). an organization achieves data-enabled value creation by focusing on four areas, such as data collection, information creation, value creation, and distribution through provider networks (lim et al., 2018). organizations use data in a variety of ways, such as for exploring new opportunities, for arriving at insightful analysis on various internal and external factors, and for improving the organization’s data eco-system by compiling all possible data from end customers (mazzei & noble, 2017). organizations could build innovative business models by building data-enabled assets and resources. organizations, by developing analytical capabilities, could improve product development, customer service, and other value-chain activities (mazzei & noble, australasian journal of information systems balakrishnan, das & chattopadhyay 2020, vol 24, research article implementing data strategy 4 2017). organizations look at data and analytics as tools for enhancing innovation and operational efficiency, thus leading to additional revenue streams. organizations could reap huge rewards by implementing effective means of data management solutions. data-driven organizations have higher output and productivity than their less data-driven counterparts. organizations could facilitate data-driven culture by implementing various measures such as building a repository that provides a single version of truth, building a repository containing data dictionary to understand what the data means, giving access to information for all relevant roles in the organization, and enabling managers to support data-driven decision making (anderson & li, 2017). technology is the critical enabler for big-data aided value generation. however, the quality and integrity of data used for arriving at inferences depend on business processes and rules rather than on technology (bowen & smith, 2014). firms generate value by analysing data available at their disposal, and the value derived from analysis helps on both internal operations front, and external market services front. the dataenabled value generation options range from managerial actions such as implementing datadriven process management to managing operations through data-driven workforce management and creating new market offerings based on data analytics, ml and ai techniques. for an organization to build data-enabled value creation, it has to deploy the right set of data management processes, tools, and methods for facilitating the building blocks necessary to create a data-driven culture. this study is an attempt in that direction to provide insights on a reference data architecture for handling all types of data in a corporate environment such as data in traditional transactional systems, data warehouses, big data repositories, schema-less data stores, and a newer variant of the scalable transactional system such as newsql. the next few sections deal with various types of data, an orientation on the usage, shortcomings, and alternative options. 2.2 relational database management systems (rdbms) rdbms store and represent data in two-dimensional tables. this is a simple and effective way of automated storage and retrieval of structured data. while using rdbms, programmers employ query-based scripting languages to perform data management tasks. this continues to be useful when the data is in a structured format, and the scale of the data is manageable. rdbms are suitable for online transaction processing (oltp) applications and characterized by atomicity (a), consistency (c), isolation (i) and durability (d). the acid properties of rdbms ensure that the related transactions get completed as one unit – all or nothing, transactions happen independently, transactions that are written to the storage result in consistent data stores, and data stays durable irrespective of system failures (pokorny, 2013). rdbms have been used for decades and are mature systems, yet, in the face of ever-changing types of data and the complicated relationships among them, they become difficult to use. in rdbms, especially in oltp systems, data gets stored in normalized forms following codd’s rules, eliminating redundancy and making sure that all changes in the data are consistent across the models (link & prade, 2019). the more complex the collection of data is, the more levels of hierarchy and cross relationships, the less possible it is to represent the collected data within the simple tabular structures of an oltp relational database system. the issues associated with normalized data resulting in inefficient retrievals and lack of support for complex structures such as hierarchies are eliminated in data-warehouses. datawarehouses are relational in nature and store aggregate information that can readily be used for analysis and reporting. rdbms supporting data warehouses feature non-normalized or australasian journal of information systems balakrishnan, das & chattopadhyay 2020, vol 24, research article implementing data strategy 5 de-normalized data stores to facilitate easy retrieval for analysis and reporting. data warehouses are built with star-schemas or snowflake schemas. data warehouses comprise of dimensional stores through which hierarchies can be supported. also, data warehouses enable storage of business metrics associated with various dimensional stores. when it comes to performance, rdbms could scale up very well, but always with an associated cost. the initial investment needed for rdbms is high as one has to pay for both software systems such as oracle, and high-end servers with excellent processing capabilities. if one uses rdbms for storing some reasonably common forms of data like storing the content from channels such as sm, articles, blogs, forums, to identify prospects in various domains, it will result in processing heavy-duty data. while using rdbms for this purpose, one would run into issues on cost, performance, scalability, and even on building complex programming logic for text processing. the applications using a massive amount of unstructured data streaming on a real-time basis would need a scalable system with extensive storage and high availability (rocha et al., 2015). rdbms support scaling-up vertically by moving the database to a machine with better hardware, resulting in additional cost. the sources of data such as real-time feeds and sm warrant a new type of system called nosql that facilitates scaling-up horizontally (or scaling-out) by adding more low-cost machines or commodity servers (pavlo & aslett, 2016). 2.3 ‘not only sql or nosql’ – a schema-less database solution with the arrival of web 2.0 and web 3.0, different types of data formats and various data channels emerged, and this new paradigm warranted for a different approach for data storage and retrieval. the modified approach to storing data in nosql is called cap. cap deals with data consistency data being correct all the time, data availability reading and writing data all the time, and partition tolerance no complete failure of the system, and the system becomes consistent when it comes online. the type of systems based on cap is not acid and called base systems (pokorny, 2013). base reflects the features of nosql systems that include basic-availability, soft-state, and eventual-consistency. the critical feature of nosql system is that it sacrifices robust transactional guarantees and data models of rdbms in favour of eventual consistency and more flexible data models such as key-value pairs, document-stores, columnar stores, graph databases and likes (pavlo & aslett, 2016). performance and flexibility are the two critical reasons for firms to move to nosql systems (lourenço, cabral, carreiro, vieira, & bernardino, 2015). performance is focused on sharing and managing distributed data, and flexibility is the ability to handle different types of data such as unstructured and semi-structured data that may arise on the web. in the base model, the main emphasis is on data availability despite multiple system failures. nosql databases spread data across many storage systems with a high degree of replication. nosql solutions operate similarly as other distributed systems, providing critical features such as scalability, concurrency, data fail-over and recovery, and data security. the storage failures do not necessarily result in a complete outage of the system. in rdbms, data is organized in the form of database tables, whereas nosql databases use different approaches for storing data (sareen & kumar, 2015). for example, a key-value nosql database such as mongodb stores values for each key and distributes those values across the database to allow for easy and efficient retrieval. the nosql database does not have a standard schema or physical model that stores the meta-data for the data elements. in other words, there is no schema available for nosql databases, and they could be categorized as schema-less databases. australasian journal of information systems balakrishnan, das & chattopadhyay 2020, vol 24, research article implementing data strategy 6 nosql systems are highly optimized for storing high volumes of data and efficient data retrieval. nosql system could be a better fit for storing blogs, articles, sm content and any unstructured data that does not fit into a relational database table. for nosql systems, developers do not create logical data models, and hence, data is unconstrained (stantic & pokorny, 2014). there are query tools such as xquery and jsoniq developed to support data retrieval from nosql systems. however, they are not widely used. in cases where nosql query tools are not used, the nosql query options are built into the application layer. nosql systems are inherently huge in nature because of the type of data that gets saved. the usage of unconstrained data in nosql systems is generally supported by metadata such as data catalogues for facilitating efficient and faster data retrievals. nosql systems complement rdbms, and hence these systems co-exist. the issues associated with rdbms, such as data model change management, and data integrity enforcement, are eliminated in nosql systems. nosql systems are useful when the data’s nature does not require a relational data model (sareen & kumar, 2015). nosql systems present a different set of issues such as lack of designer tools to manage nosql systems, and lack of routines or options to migrate from one nosql system to another nosql system (mohan, 2013). nosql systems do not have any standardized way of storing and retrieving data like rdbms and sql based systems, and hence, facilitating change management on nosql based systems could be frightening (mohan, 2013). nosql systems are not recommended for applications requiring functionality, such as data integrity and security. however, they could be used for analysing high volume and real-time data such as web-log data, and click-stream data (stantic & pokorny, 2014). nosql cannot be used as the standard data store for all types of data, and organizations have to use nosql systems for the right use cases to facilitate efficient data storage, retrieval, and change management. this study attempts to understand use cases relevant for nosql systems and how these systems could work with existing traditional rdbms supporting online transaction processing (oltp) solutions. 2.4 newsql systems newsql systems represent a newer version of the already existing rdbms technologies. newsql systems are the ones that combine the critical features of both rdbms and nosql systems (pavlo & aslett, 2016). newsql systems are based on distributed architecture operating on shared-nothing resources and contain components to support multi-node concurrency control, fault tolerance through replication, and distributed query processing (pavlo & aslett, 2016). this includes cloud computing providers that offer newsql databaseas-a-service (dbaas) products. newsql dbaas products are hosted on cloud infrastructure, and the customers are provided with a universal-resource-locator (url) to connect to the system. dbaas enables the pay-per-use of computing resources. newsql systems support horizontal scaling by adding new servers, and support partitioning, sharding, and clustering. newsql systems, by combining features from both nosql and rdbms, could support both acid transactions and enable high-availability systems. newsql systems meet many of the requirements for data management in cloud environments, combined with the benefits offered by traditional rdbms (grolinger, higashino, tiwari, & capretz, 2013). newsql could be considered as an alternative to nosql systems for new oltp applications (kaur & sachdeva, 2017). some of the newsql systems, such as voltdb, support in-memory processing of complex acid transactions in a distributed manner. newsql systems are developed from scratch using shared-nothing architecture, or australasian journal of information systems balakrishnan, das & chattopadhyay 2020, vol 24, research article implementing data strategy 7 built by augmenting the existing rdbms, enabling them to scale-out. through augmentation, the features of newsql could be extended on top of existing rdbms solutions, and this feature would be useful for organizations that are not willing to migrate to new database solutions. the traditional one-size-fits-all approach of rdbms does not hold good anymore, and newsql empowers organizations to move towards specialized database designs catering to their specific data storage and processing needs (simsek, 2019). 2.5 data governance information is a critical resource for organizations, and it forms the basis for business intelligence and data analytics (newman & logan, 2009). information takes on new importance in it-related strategies such as service-oriented architecture. data governance is defined as an organizational approach to data and information management through a set of policies and procedures, to manage the life cycle of data from acquisition to use to disposal (korhonen, melleri, hiekkanen, & helenius, 2013). with data governance, organizations could define guidelines and standards for data quality management and assure compliance with laws governing data (weber, otto, & osterle, 2009). data or information governance is a mechanism through which data that is considered a liability, get converted into a trusted strategic asset (kooper, maes, & lindgreen, 2011) the organization’s use of information technology (it) is primarily focused on operational efficiency; however, the evolving complex it landscape has resulted in redundant and inconsistent data in enterprise environments (korhonen et al., 2013). incorrect and erroneous data add additional costs and risks and lead to poor managerial decisions. ever since the inception of big data and streaming data, organizations have started considering data they collect from various internal systems and external sources, as assets. information assets are facts that have value or potential value that are documented (khatri & brown, 2010). the critical business benefit of information governance is the realization of data as a valuable and manageable organizational asset, and other business benefits include improved business decisions due to accurate data, and increased user trust within the organizations’ data sources (marco, 2006). data governance strategy results in improved reliability, traceability, and authenticity of data, thus providing the backbone communication platform across all stakeholders. some of the critical elements of data governance include data quality, meta-data management, and data life-cycle management. having high-quality information is a pre-requisite to building data-driven customer value propositions (weber et al., 2009). hence, it is critical for organizations to focus on data quality so that they could make effective managerial and strategic decisions. data quality management helps an organization to plan, provision, organize, use, and retire high-quality data (weber et al., 2009). for a firm to implement the right governance program, it has to manage various aspects of data management such as information security, risk management, data privacy, data security, data loss, and business continuity. organizations in the digital era have to adopt a holistic approach to manage the three critical challenges associated with data management such as data security, data privacy and compliance obligations (salido, 2010). these problem areas of data management have to be adeptly supported by three capabilities of firms such as people, process and technology. organizations design three kinds of governance mechanisms such as decision-making structures, alignment processes, and formal communications (weill & ross, 2005). australasian journal of information systems balakrishnan, das & chattopadhyay 2020, vol 24, research article implementing data strategy 8 successful information governance warrants accountability, and accountability is executed through a formalized process called stewardship (newman & logan, 2009). the people aspect of data management deals with identifying data-stewards people collectively responsible for data quality in the organization by defining policies and procedures for the classification, protection, use, and management of data (salido, 2010). the data stewards engage in information governance by implementing managerial activities such as setting standards, enforcing policies around data, resolving intra-team issues and implementing day-to-day operational procedures (newman & logan, 2009). the process aspect of data management deals with defining a set of processes to manage elements such as information security needs and compliance needs of the organization. the process aspect of information governance deals with monitoring and reconciling data at critical points along data’s path through the organization’s it systems, and this includes initial data entry points and data aggregation points (griffin, 2005). at the data-entry points, the it systems could check for valid values or required values. at data aggregation points, the process steps could include eliminating duplicate data entries and checking for valid crossreferences to strengthen the authenticity of data. also, the process options include features for manual corrections to exceptions and errors that cannot be fixed automatically (griffin, 2005). the technology aspects of information governance and data management deal with activities such as providing secure infrastructure, implementing identity and access control, strengthening information protection features, and implementing robust auditing and reporting functionalities (salido, 2010). the software solution considered for big data-related data management solutions should scale to accommodate large datasets, leverage the underlying hardware platforms efficiently, and bridge the increasing gap between growing data and computing power (kambatla, kollias, kumar, & grama, 2014). the terms data governance and data quality are often used interchangeably, but both are different in theory and practice. data quality happens at both the front-end through data validations and back-end through databases. the process of data quality is defined, structured, and implemented through a well-designed data governance framework. in the data governance framework, rules and policies related to data ownership, data processes, and data technologies are clearly defined. in other words, data governance provides a framework for managing data quality (ghosh, 2019). the existing literature on data management covers a variety of topics, as mentioned in table 1. however, in the current research of data management, queries such as ‘how do organizations support variety of data management solutions ranging from traditional rdbms to newsql, and how do organizations integrate them for facilitating data collection, aggregation, and analysis?, ‘what is the recommended data architecture for supporting dataenabled value creation?’, and ‘what are the design considerations in implementing the proposed architecture, in terms of data governance, data security, disaster management, and business continuity?’, have been identified as less explored, and recommended for research. this study attempts to delve into various aspects of data management, to arrive at inferences on reference architecture and design considerations to enable data strategy for customer value creation. australasian journal of information systems balakrishnan, das & chattopadhyay 2020, vol 24, research article implementing data strategy 9 3 research methodology 3.1 research setting this study is based on an explorative approach to understand data management solutions used by organizations, and the inferences are generic in nature. the study is executed through interviews with three data architects focusing on building solutions using the latest data management tools and technologies. all three data architects have significant industry experience, worked with multi-national corporations, and architected complex data management solutions for global clients. one of the architects – a1, has been working on building solutions for clients in north america. another interviewee – a2, has been working with clients in western europe and the americas, and the third one – a3 has experience in building solutions for e-commerce, banking and financial services clients in north america. this research is conceptual in nature, and there are no control variables, moderating, and mediating factors considered for this study. this study includes interviews that have enabled the researchers to understand the data architecture used for creating a robust data management solution covering all design aspects, end-user aspects, and governance aspects. the interviewees have provided documents on processes used while building data warehouses and presented architecture diagrams that accommodate various data management solutions. the study involves interviewing three candidates, and for insightful qualitative research, the number of interviewees may not be considered sufficient, as interviewing three people does not satisfy the condition of having a representative sample. however, the three senior people found for this study a1, a2 and a3 are playing leadership roles in their respective teams. a1, as a team lead, manages fifteen data architects, a2, manages twenty-five data architects, and a3 manages fifteen data architects. all interviewees have exposure to variety of business challenges in data management domain, and they have provided solutions to complex business problems, with help from team members. they have a collective knowledge base on various data management techniques and worked on all tools and technologies related to data management. this criterion qualifies them as subject matter experts (sme) who could provide insights on all aspects of data management. the sme has profound knowledge in the subject, and their experience was evident as they could comment and voice their opinion on all prominent and latest methodologies and technologies used for data management. all the chosen interviewees are from different organizations, and hence their comments and opinions are independent, and not influenced by other members. in addition to these discussions, the researchers have met with the team members to have informal discussions on finer aspects of data management. during those meetings, the team members had shared artifacts covering the data management solutioning and implementation details for various clients. the researchers have used the transcripts, documentation, and artifacts to arrive at the inferences and contributions for this paper. 3.2 research design and data collection the scope of the research is limited to four dimensions of data-enabled value creation, as suggested by lim et al. (2018), and the dimensions include data collection, information creation, value creation, and distribution or information delivery. the scope is technical in nature, and covers governance aspects of data management such as information security, data privacy, and data quality. the queries covering these four dimensions are created in australasian journal of information systems balakrishnan, das & chattopadhyay 2020, vol 24, research article implementing data strategy 10 consultation with faculties teaching courses on data management for under-graduate engineering students. the study does not focus on covering the aspects related to hardware or infrastructure components associated with data management. the study comprises of interviews with subject matter experts, and the research is exploratory in nature. the researchers have avoided any leading queries to interviewees and included probing queries to offer more clarity on the base queries when needed. the questions are openended, and some of the queries used for this study are, “how are various data management systems integrated? from your area of expertise, could you provide insights on why the integration is needed, and how it was implemented?”, “how would you handle exceptions while loading different types of data?” “what is the new customer value proposition that comes with the data exploration tools?” “how does your new value proposition get monetized? could you give specific scenarios on how you helped your client monetize on data offerings?”, and “what kind of data governance mechanisms you have implemented? what are the design considerations for business continuity?”. the study is conducted without any underlying assumptions, and the researchers have personally met all the three data architects, conducted interviews and transcribed the audio files. the transcripts are reviewed with the interviewees for approval, thus enhancing the accuracy of collected data (huber & power, 1985). the study comprises of 150 minutes of audio recordings, and 16000 words in transcripts. also, the authors have referred to various process documentation, and discussions with critical team members through informal question-and-answer sessions to answer queries, for a period of another 140 minutes; 3.3 data analysis the researchers have transcribed the recorded conversations and validated the transcribed content through the maker-checker process, a dual-approval process typically used in the financial services industry (commercial bank, 2018). in this dual-approval process, a maker enters the transactions into a system, and a checker reviews these transactions and approves the same. for this study, an author analysing the transcript files, gets his analysis reviewed and approved by the other author. the researchers have analysed the data, verified each other’s work, and frequently referred to both the data and existing literature when developing the research report. this study has identified findings across four aspects of data-enabled value creation, and these inferences are summarized in the next section. 4 research findings the study proposes a methodological technology independent framework an organization could implement to support data-enabled value creation. all the critical elements of the structure supporting data-enabled value creation are portrayed in fig.1. these components are discussed at length in subsequent sections. the first section provides details on different types of data analytics solutions implemented by organizations. the second section contains the findings on data integration methods adopted by firms, explicitly covering all types of data such as structured data, semi-structured data, and unstructured data. the third section summarizes the findings on data transformation methods used by firms to create information that can be part of the knowledge. the fourth section lists the findings on the value creation resulting from such an architecture, and the last section summarizes the findings on other business critical considerations such as data quality, data security, and business continuity. australasian journal of information systems balakrishnan, das & chattopadhyay 2020, vol 24, research article implementing data strategy 11 fig.1 reference architecture for creating big data enabled value creation 4.1 use cases for various data management technologies one of the data architects has spelled out the use cases of these data management technologies, and opined, “anything that is transactional in nature – where you have structured schema and transactions …things like erp and stuff – rdbms is the preferred choice; … when you have aspects related to data integrity and things, that is where rdbms comes into play; …. the challenge comes as (when) the data volume increases; the only way you can handle volumes is to grow vertically; like you have to increase the processing capability of the server. ….we had influence of social media data coming; influence of semi-structured data coming; semi-structured in the form of xml, json, and other files have come; these things resulting in change in data structures as well; that is how we got nosql; …lots of vital data that needs to be stored and does not need to be in an integrated manner (data integrity need not be there) then, we have to go for nosql systems … a new generation data management solution – you could handle big volumes and scaling like nosql – that is where newsql comes into (the) picture; low latency data retrievals – huge storage, scaling up horizontally, with very low latency;” (a2, data and solution architect) data management solutions are implemented for descriptive, predictive and prescriptive analytics solutions. the following table lists the suggestions as indicated by the interviewees, on the type of data management solutions possible with different types of data. - internal sources crm retail sales erp ecommerce inventory external sources twitter facebook e t l data lake edw hr order inventory customer nosql or big data environment s e n ti m e n t m e tr ic s p ro d u c t a ff in ity m e tr ic s s e rv ic e q u a lit y m e tr ic s enterprise functional metrics social media metrics nosql or big data environment edw fail-over/secondary instance streaming analysis business analytics real time trends sentiment metrics other social media metrics passive replication active replication e l t .... data marts australasian journal of information systems balakrishnan, das & chattopadhyay 2020, vol 24, research article implementing data strategy 12 type of data management technology descriptive analytics solutions predictive and prescriptive analytics solutions rdbms (supports structured data and acid transactions) various reporting solutions based on customer relationship management (crm) and enterprise resource planning (erp) applications; management information systems (mis) generating daily reports; retail: recommendation solutions banking: fraud monitoring solutions health-care: identifying the right solution based on given health conditions (prescriptive solution) nosql (supports unstructured and semi-structured data) statistics information such as tag-clouds, sentiments on social-media content, and visual reporting on hierarchical information arriving at the sentiment trend based on the tone of sentiments collected over a period of time; predicting the most-preferred features in a new product or service based on the comments collected from semi-structured and unstructured data sources newsql (structured data) similar to rdbms; however, newsql is mostly used for real-time analytics and predictive analytics streaming analytics – on customer sentiments; identifying trends in user perceptions; real-time recommendations on product purchases; streaming analytics on data from iot; market basket analysis in retail industry; solutions supporting big data (hdfs based systems) all types of data; mainly used as ‘data staging’ area and data archival area; helpful to deal with ‘save now and analyze later’ kind of data; staging area for extract-load-transform (elt) kind of data extraction; storage for archived-data wherever needed; system can support all type of data to support descriptive, predictive and prescriptive analytics solutions table 2: data management technologies and use cases supported 4.2 data integration data integration involves combining data from disparate and heterogeneous source systems to arrive at a unified view of the data. data integration consolidates applications within one firm to provide a unified view of the firm’s data assets (“data integration,” 2018). data integration enables businesses to provide users with a real-time view of business performance by combining data residing in different sources, and inclusion is the first step towards transforming data into meaningful insights such as business metrics and key-performanceindicators (kpi) (doyle, 2017). organizations use various means to extract data from source systems, integrate them, arrive at different business metrics, and store those kpi in a relational data warehouse that could drive further analysis. organizations could use either etl (extract-transform-load) or elt (extract-load-transform) to extract data from the source systems. when using etl, the data is extracted from source systems, transformational rules are applied to the data, all necessary business metrics are calculated, and the transformed information is loaded into the target database. in the case of elt, the extracted data is loaded into a temporary work area called ‘staging area’, transformation rules are applied to arrive at business metrics, and the australasian journal of information systems balakrishnan, das & chattopadhyay 2020, vol 24, research article implementing data strategy 13 transformed information or business metrics is loaded into the target data warehouse. to answer a specific query on the right use cases for using either of etl or elt, a3 claims, “if the systems you extract data from (source systems) are all local (within your internet/intranet), then etl would be good; there won’t be much of latency issues here as the data is internal; the data you have on external systems – especially social media, etc – you extract data, you load them as-is into some sort of temporary store (data staging area) – and then transform and load/persist into data warehouse; (use elt) “ (a3, data architect) etl is the standard way of data extraction when sourcing from internal transaction processing systems. however, data from sm such as twitter and facebook get loaded first into a data store like nosql, necessary metrics such as sentiment scores are calculated, and the results are loaded into a relational data warehouse. sm data is usually extracted through scripts, software programs, and crawler programs. an etl or elt process might throw exceptions or errors for data that cannot be processed completely. these exceptions are sent back to source systems, and alert messages are raised to business users. business users view error records through user interfaces or applications wherein they would have options to fix the data, and resubmit for processing. the data extraction and load procedures maintain data integrity when loading data into target systems, and reconcile target data with source systems to load all relevant information into target systems. the data extraction process, depending on the business rules, could be set as an incremental-refresh or full-refresh procedures. in the case of incremental-refresh, new transactions since the last incremental-refresh, will get extracted from the source system, and loaded into the target system. the incremental transactions are usually identified through a timestamp field in transactions. in the case of full-refresh, the entire target data gets overwritten by the source data. the following quote summarizes how incremental-refresh and full-refresh data extraction procedures work: “when you do data extraction, and if you know that the data (source data) is changing, then, you will have to set that as a full refresh; if the data doesn’t change much, it has to be incremental refresh; if i have to quote example from the credit card expense transactions, the transactions happen on a daily basis, and no need to do full refresh (on those daily transactions); likewise, your banking transactions – no need to do full refresh as all the transactions like deposits, withdrawals, etc – are incremental in nature; and also, once you do a transaction you hardly change anything; maybe there is a credit onto your account, but that is a new transaction – which can again be incremental; full refreshes happen when you set up the target (data) for the first time; also – things like billing for one of your corporate clients: there, may be billing has to be done as a full refresh on a periodic basis – say 3 months or 6 months or even a year;” (a3, data architect) data transformation is a process through which the data extracted from source systems is transformed to reflect the precise business metrics used for end-user analysis. the data transformation happens either at the source system or at the target system after the load to the staging area is completed. for data extraction and transformation, one could choose to transform at the source, provided the source systems are internal to the firm or available within the firm’s intranet. data extracted from sm and external sites is brought inside the firm’s intranet, loaded into the staging area, transformed to arrive at metrics such as sentiment scores and product affinity scores, and loaded into a target data warehouse for further analysis. australasian journal of information systems balakrishnan, das & chattopadhyay 2020, vol 24, research article implementing data strategy 14 organizations could implement data transformation using in-memory techniques that are part of new data management solutions called newsql systems. newsql technologies offer inmemory computing features that support real-time analysis of data. in this case, the data that comes in real-time such as internet-of-things (iot) sensor data and real-time twitter feeds, could be loaded into the newsql database, and analysed on a real-time basis. the new generation analytics tools such as spotfire and tableau provide in-memory-computing features through which the entire data used for reporting could be cached into the reporting server, and cached data in memory would expedite information delivery and visualization. one of the subject matter experts has mentioned the following while discussing in-memory computing: “it (in-memory analysis) is something that we typically use it for visualization; when you want to write a query and load lots of data; it is supposed to be available for visualization very quickly – then you can use in-memory; …in-memory computing will be beneficial for sporting, casino kind of businesses where the update should be in seconds and decisions should be taken in seconds.” (a1, senior data architect) organizations store business metrics and kpi data in a data warehouse that is usually represented by a star-schema wherein the business-relevant kpi are stored in one or more fact tables (or a data store), and all reporting entities are stored as dimensions. for an organization, there would be an enterprise-wide data warehouse (edw) that hosts business metrics relevant for all functional teams within the organization. in addition to this edw, there would be many subject-area data warehouses one for each functional unit within the organization. these subject-area data warehouses are otherwise known as data-marts. the data-marts are built by extracting subject-specific data from edw, or in some cases, an organization could build edw from the underlying subject-specific data-marts. a1 has suggested, “the other important thing is for all data warehouses to have subject areas (on their own); it would help in semantic analysis; you put customer data into ‘customers’ store, sales into a separate store; you can put that into (create) order, inventory; likewise, you can create subject areas, and keep the data into different subject areas; anybody who want to do customer analysis can easily do it from the subject-area data-mart (data base); it is always better to persist in subject-matter data store and do further analysis; “ (a1, senior data architect) data transformation process is preceded with the pre-processing phase during which data from disparate systems get cleansed, validated and modelled to support transforming and loading the data into target database. the processes of extracting data from source systems, transforming data, and loading necessary business-relevant metrics into edw are portrayed in fig.2. australasian journal of information systems balakrishnan, das & chattopadhyay 2020, vol 24, research article implementing data strategy 15 fig.2 edw built using etl and elt processes the findings from this study are in line with inferences on big data extraction methods identified by paakkonen and pakkala (2015). in addition, this study provides inferences on extraction options for both internal and external systems, and outlines the need for datastaging to handle data from external systems. 4.3 data lakes and dataspaces data lakes enable large-scale data storage at a relatively low cost. it is cheap and persistent storage that can support huge volumes of data (t. davenport & verma, 2018). the data stewards have to identify all possible sources from where the data has to be brought into the enterprise data lake. once the causes are identified, the architects have to identify the right means to get the incremental and full dump of data from the source, and these mechanisms would include techniques such as writing a crawler job, developing a program to call api function to get data, configuring an elt tool to extract data, and using vendor (source) provided adapters or scripts. the architects have to define an integrated semantic layer as mentioned by a2, “to give a specific example – assume you are getting social media comments on your product, service quality and product ideation; these are three different things – or you could call it ‘subject areas or domains’. when you bring it over (the data over to staging), you put it (the extracted data) on a staging (temporary) data store; a semantic layer integrating all these three domains have to be modelled – to be precise, integrating your products through product-id and relating those products to the comments you get on service quality (related to those products), and user feedback on new features that could be added to the product through product ideation. internal systems package applications (sap, oracle erp, ms-crm) legacy (mainframe, as400) oia (other internal “homegrown” applications) • business rules driven data extraction • restructure data as needed • calculate business metrics • load into target edw hr marketing sales social media metrics • business rules driven extraction • load data into staging area loading area external systems online forums social media e t l edw e l t data lake enterprise functional metrics r e s tr u c tu re d a ta a s n e e d e d c a lc u la te b u s in e s s m e tr ic s l o a d i n to t a rg e t e d w … data marts australasian journal of information systems balakrishnan, das & chattopadhyay 2020, vol 24, research article implementing data strategy 16 building this integrated model is a complex process as the relationships are not defined in a concrete way; you will have to do lot of trial and error things (modelling); one need to create ontologies, tag-clouds for often-used words, classifications on the type of comments used, etc – to arrive at this integrated (semantic) layer; data analytics and your predictive stuff (analytics) can be driven off of this (integrated layer); all these things constitute your data lake” (a2, data and solution architect) hadoop based big data environments provide flexibility to organizations when dealing with high volumes of data from different source systems. since systems supporting big data are hosted on low-cost commodity hardware, organizations could choose to save data from less popular social forums, newspapers and online magazines into a big-data environment, without worrying about how it could be leveraged for customer value creation. in the words of an architect, “we always tell them (clients) – ‘forget about what you want to do with data; have a data-lake in hadoop; any data, you come across – make sure that you store in hadoop (big data environment)’; we are not saying that high-volume data only should go to hadoop; all other data – streaming, etc -more like a ‘staging’ area; dump everything into one store; we then figure out a way to use it, build analysis and store the inferences;” (a2, data and solution architect) all the subject matter experts handling data management solutions feel that data lakes could be used to archive all types of data that may or may not be used later. this would result in a data store that is humongous in nature, and hence retrieving data for further processing might be time-consuming and cumbersome in nature. dataspaces would help overcome these issues. dataspaces are not part of a data integration layer, and they provide base functionality over all data sources, regardless of how integrated they are (franklin, halevy, & maier, 2005). dataspace comprises of data models with a collection of relationships between data repositories. one of the basic dataspace services is cataloguing data elements from all participating data sources (franklin et al., 2005). dataspace platform should provide means for searching and querying the metadata. the metadata forms the basis for all retrieval operations, and the metadata in dataspace could be configured at various levels such as timebound data, source-specific data, and certainty of data. dataspace is considered as a hybrid of a database management system storing data from variety of sources, a search engine, a datasharing system and an information integration system (mirza, chen, & chen, 2010). in the words of a team member from a2, “any data that you get into data lake, one has to create a model or a catalogue supporting all sources of data, where the data is retrieved from, time-stamp associated with data, all the metadata associated with data like which user (if possible to identify) has created it, geo (geography) associated with the data, relationship between data elements from various sources – if any, etc. we could build more intelligence in the meta data by providing filtering elements such as the key tags that are being referred, the impact the data have – in terms of decision making, etc. all this could be part of the meta data ; when we retrieve data, usually you retrieve it through an app(application) layer; there, you query the meta-data to identify the right source we should query from, and initiate our retrieval; by having an effective cataloguing, you would be able to isolate which sources need to be queried, and also – how impactful the inference could be. if you don’t have catalogue information or namespace kind of setup, querying from a data lake might australasian journal of information systems balakrishnan, das & chattopadhyay 2020, vol 24, research article implementing data strategy 17 be impossible as no one knows where to get it (your search results) from” (team-a2, data and solution architect) the inferences presented in this study that are related to data spaces is similar to the concepts suggested by franklin et al. (2005) and mirza et al. (2010). additionally, this study emphasizes the need for having subject-area specific data stores along with catalogue and meta-data for each subject-area. business intelligence solutions built on this premise enable combining data across various subject-areas to arrive at valuable inferences that could drive business decisions. 4.4 customer value creation the organizations of the new digital era use data as a means by which they could create customer value proposition and leverage the same for monetization. a2 has opined, “data monetization is an area where every organization wants to get into; data is very valuable and not that easy to get; now organizations look at – we have data now, and how they can make money out of it? something we are looking into – of late is ‘benchmarking’; benchmarking is very relevant for people in health-care industry; benchmarking gives an idea on where an organization stands with respect to various parameters that are standardized for the industry – and gives an indication of where the company should focus on; there are organizations in healthcare industry, who collect data (from various firms), these people collect non-sensitive data, gender data, demographic data – and start creating benchmarks with data; the data can be federated and third-party people can bank on the data for benchmarking; that is one way of monetizing; this probably would fall into ‘data-as-a-service’ model as well..” (a2, data and solution architect) one of the architects has opined on the data-enabled value creation saying, “(by listening to data in sm and other channels), businesses get more informed in terms of what products have to be developed, and how they have to be positioned … if you provided targeted discounts, people tend to buy more; ..people tend to buy more as there is a discount which in turn increases sales. if you look at a data warehouse, it helps – and gives directions to salespeople on what to do (which product to develop), when and where (positioning in different markets)?” (a1, senior data architect) firms have deployed self-service business intelligence tools for doing explorative data analysis. the information visualization tools support a variety of analysis such as what-if analysis, scenario analysis, correlation analysis, and enable business users to understand business kpi in a better way. the new generation data explorative tools are more tuned for business use than for it use, and engage customers more efficiently than the previous generation reporting tools. the inferences on value creation and value delivery are aptly summarized by one of the interviewees: “you provide value-add like self-service reports (to business users), real-time analysis, help customers in their product launch, product positioning, enhancing your target market reach those are your monetization opportunities; internally, you use data-driven decisions to improvise your operational metrics; that is your cost saving opportunities; any data-driven initiatives (value propositions) help you on two fronts: open (enables) new avenues for revenue (additional revenue generation), and cost-saving opportunities by process optimization; and by doing your data management the right way – you get to benefit from both ends (revenue generation and cost cutting);” (a3, data architect) australasian journal of information systems balakrishnan, das & chattopadhyay 2020, vol 24, research article implementing data strategy 18 the findings from this study resonate with the ideas of using data-driven decisions for process optimization and value creation that are presented in various other studies. furthermore, this study highlights options for data-enabled new value creation and revenue generation opportunities such as building data-as-a-service model. 4.5 other critical considerations: data quality, data security and business continuity the organization’s ability to create a data-driven value proposition is severely impacted by low-quality data. organizations strive hard to strengthen the quality of data used for building edw. a slang expression ‘garbage in, garbage out (gigo)’ (techopedia, 2011) refers to inaccurate results due to erroneous input data. data governance mechanisms make sure that the data entered into both transactional systems and reporting systems are valid, valuable, safe and could be used effectively for decision making. one of the subject matter experts has to say this, when queried on the aspects of data governance: “implementing master data management, cleansing rules for data, business rules for data validation, implementing maker-checker for enforcing transactional integrity, data backups and archival based on business rules at stipulated time periods – all are part of data governance mechanisms; an organization has to look at these things holistically – to set up their governance mechanisms for ensuring ‘clean data’;” (a3, data architect) organizations have to comply with stringent data security mandates, and one of the rules imposed by sarbanes-oxley (sox) considers encryption as a critical security control, and one of the best practices, for both data-at-rest, and data-in-transit (mcafee, 2019). this implies that all types of data residing in enterprise systems within the organization, have to be encrypted. data encryption happens at the database level or the hardware level. organizations could choose to encrypt entire databases such as mongodb or oracle hosting transactional applications. the encryption at the database level is preferred over incorporating security at the code level because database-level encryption provides uniformity across all functional elements of the application. data centres hosting servers running enterprise applications could be protected by the use of firewall software and access control mechanisms. for encrypting data-in-transit, firms could use a secured protocol such as https instead of http. for datatransfers from the source system to the target system, firms could use secured-ftp instead of regular ftp. organizations implement data replication to support recovery from data failures and business continuity. data replication happens through two modes such as active-replication and passivereplication. the databases are replicated on a real-time basis during active-replication, and during passive-replication, databases are replicated when they are offline. organizations usually keep primary server hosting data and a secondary, failover server in case of system failures. in the case of system failures or data loss, the fail-over server takes over automatically. organizations define business rules for configuring data replication frequencies between primary and fail-over sites. for all critical transactional systems, replication happens frequently, or on a real-time basis. for all non-critical transactions, replication could occur during off-hours. the non-critical data such as facts and dimensions in an edw are replicated less-frequently, usually once a day. in case of data loss, this incremental data in edw could be built from the data in transactional systems. the objective of replication is to keep the data loss at a very minimal level. australasian journal of information systems balakrishnan, das & chattopadhyay 2020, vol 24, research article implementing data strategy 19 5 discussion this study on data architecture related to using diverse data management solutions has inferences on using data-access concepts such as data virtualization, implementing alternate options for real-time analytics, and defining general guidelines for building a robust data architecture for enterprise data lake and edw. these inferences are discussed at length in subsequent sections. the last section discusses the implications for both practitioners and academicians and the limitations of this study. 5.1 options available for data extraction while extracting data from source systems, organizations could choose either etl or elt as an option to transform and load data into target systems. for both etl and elt data extraction, organizations create necessary scripts or use commercial-off-the-shelf tools to complete the data extraction and load jobs. creating these jobs is time-consuming, and one needs to go through rigorous testing to cover all possible data scenarios. one of the options available for an organization to extract data from source systems instead of setting up etl or elt jobs is to build data virtualization on top of underlying databases. data virtualization integrates data from disparate source systems without replicating the data, to create a single virtual data layer that delivers unified data to multiple applications and end-users (patrizio, 2019). when answering a query on specific scenarios for using data virtualization over etl or elt, a1 has opined, “data virtualization is a case where you want to do ad-hoc-queries (for reporting); when your business wants to do adhoc-querying, and you want to run something which is not already existing. they (business users) can look at the data, define some queries on it, and explore the data further; so, data virtualization helps with ad-hoc queries which are not existing already; and it adds value without adding a long etl process…. i would rather say – use data virtualization for all pocs (proof of concepts) that you want to run; find the value of it; and for better performance (if it needs to be repeated), you make it into etl or elt (for further extraction, processing, and analysis);” the suggestion is to use the data virtualization option to query directly from underlying systems when working on proof-of-concept or demo-version of applications. organizations could choose to go with etl or elt jobs when they want to implement the proof-of-concept version as a full-fledged application. data virtualization helps a firm to increase productivity when building edw, brings down the cost as the data does not get replicated through etl, and lessens the data governance needs (patrizio, 2019). 5.2 real-time analytics organizations could use streaming analytics or real-time analytics to query continuous data streams and detect conditions instantly, within a small period from the time the data is received (perera, 2018). real-time analysis is the right choice in scenarios where the data insights are more valuable only when the data is processed shortly after it is received, and the value diminishes fast with time (perera, 2018). some of the typical use cases for real-time analytics are applications such as providing real-time personalization options, detecting anomalies and frauds in real-time, providing real-time emergency health-care services, implementing smart device applications, and building algorithmic trading applications (lawton, 2019). options such as an e-commerce portal recommending a product to buy, or offering a discount based on the transactions one has made in the last few hours, are made australasian journal of information systems balakrishnan, das & chattopadhyay 2020, vol 24, research article implementing data strategy 20 possible through real-time analytics. organizations could use technologies such as newsql to drive real-time analytics. one of the architects has commented on the popularity and relevance of newsql systems, “i have my apprehensions about newsql; if you look at the way it has come up, it has been there for almost 8 years now; 2011 is when it came first; – not much of adoption happened in the industry; that is what i have seen; even though it (newsql) happened, it happened in a small way; in places like ours (e-commerce portals), it is difficult to adopt (a tool or technology) which is not previously adopted or proven (by others); … the maturity of newsql is still a question; (a1, senior data architect) the architects feel that newsql systems handle real-time analysis at a higher cost compared to other streaming analytics tools that are open-source or offered at a significantly lower cost. one of the subject matter experts used for this study, feel that tools such as kafka, provided by apache foundation, could listen to messages streaming from sm, and synchronously do analysis such as trends and sentiments. information visualization tools could directly connect to streaming analytics tools and plot real-time reports on inbound comments. the messages coming through kafka are cached in kafka for a duration that could be configured. developers could write scripts to move the streaming messages coming through kafka, and persist them in a data-store if needed (yang, 2016). a1 has opined, “you will have a lot more channels kafka listens at (to); you could push messages to various channels – the product could be one, and so on. … if they (firms) want to do that (real-time analysis), we will have (a tool like) tableau listening to kafka; then, you can do real-time analytics; so, kafka would be your data source for tableau; it is like messaging queue; the data gets persisted for a period of time (in kafka); then, you could refresh that also; if you do not want to keep that data (in kafka) for a long time, you could set the configuration to ‘delete the messages’, or you could move the messages (to your data warehouse or transactional systems) also” (a1, senior data architect) the big data and data management reference architecture depicted in fig.1 covers all the components of edw and data lake. the streaming-analytics part in fig.1 is explained in fig.3, which contains high-level architecture and process flow associated with kafka enabled stream-processing. fig.3 stream processing using kafka – high-level architecture and process flow persist data to disk(optional) data request api twitter facebook python tableau batch data streaming data data sources extraction python script kafka storage & messaging storage kafka connect hdfs hive store analysis & report australasian journal of information systems balakrishnan, das & chattopadhyay 2020, vol 24, research article implementing data strategy 21 5.3 guidelines for building a robust data architecture organizations have predicaments on the validity of the data management solutions discussed in this paper, and the chances of these solutions becoming obsolete. the solutions mentioned in this study such as rdbms, nosql, and newsql or real-time analysis tools, complement each other and will continue to co-exist in an enterprise. the choice of using an appropriate data management solution is dependent on the type of data that needs to be managed, the way the data has to be integrated and transformed, and the method the data has to be analysed. in the interest of time, effort and cost, organizations tend to standardize the usage of data management solutions. the standardization of solutions is considered a good initiative, but it may not be the right choice. as one could notice, data keeps on evolving. as the data evolves, the data management solutions have to evolve as well. with that, the adage of one-size-fits-all would not work anymore, when discussing data management solutions. it is suggested that organizations should extract all types of data available on external systems, load them into a low-cost big data setup, regardless of whether it could be leveraged or not. when organizations arrive at a need to use the saved data from external sources, they could implement processes to transform the staged data into meaningful business metrics and persist into an edw. organizations have to be mindful of data quality, as the data quality determines the effectiveness of data analytics. organizations have to implement a business-rule driven data cleansing, data profiling, and data validation procedure to make sure that the data is analysis-ready before it is transformed and loaded into edw. for transactional systems, to strengthen the data quality, organizations could implement maker-checker, a widely implemented, successful process implemented in the financial services industry (commercial bank, 2018). this process mandates a transaction entered by a maker to be verified and approved by a checker, to make sure that the data is valid and useable. the data extracted from external source systems such as sm, forums, blogs, and newsletters get inserted into data lakes, and data governance enables the data to be tagged. in other words, data governance mechanisms warrant architects to catalogue the data that gets into data lakes by building a meta-data repository. the data has to be governed in terms of ownership, accountability, sharing, and usage, to avoid accumulating unwanted data in the form of data swamps (ghosh, 2019). the organization’s success in building a robust data management system depends on implementing solutions such as master data management (mdm) solution, implementing data privacy, and maintaining data portability to other environments when needed (addagada, 2019). master data is defined as an organization’s core data, containing the necessary information that is critical to run the business, and the data is relatively stable in nature (foote, 2019a). some of the master data domains include customers, products, and locations. in the words of a1, “the centralized master data – used by all systems in the enterprise; generally, one (the firm) stores only one instance of master data – that gets replicated or federated to other systems; the data in master data should reflect ‘the single version of your most important entities in the enterprise – like customer and part/product details’. a well-defined master data of customers and products can enable the firm to get a unified view of customer-wise product sales across various channels – in an effective way in very short span of time; mdm increases data analytics efficiency” (a1, senior data architect) australasian journal of information systems balakrishnan, das & chattopadhyay 2020, vol 24, research article implementing data strategy 22 as the type of data evolves, data modelling has become even more important (foote, 2019b). the benefits associated with modelling nosql data include improved data quality, lot more identifiable information, and enhanced business intelligence. architects could start with simple models, and make the models agile, keeping them evolve as new data sources and types of data get added into data lakes (foote, 2019b). in the words of a2 and his team, “data models are very important regardless of the data you use; all of us are familiar with oltp and olap related data models (normalization and relational databases); now, with all the evolving database types, we need to constantly add meta-data repositories and models to define and hold that data; this (modelling nosql data) will play a critical role when you do analysis; to give an example, if you want to monitor twitter feeds for a given trending topic, the data could be stored in a relational model that store, topic-id, topic-name, comment or tweet, and some meta-data on the person posted the tweet including the date-time stamp; once, you have it in this form, assuming that it is saved in bigdata hadoop cluster, querying this data using tools like hive would be much easier; you could query, do your aggregates and analytics on top of this; as and when new types and sources evolve, one should keep adding flexible data models in the form of your meta-data, and also tables to host your data.. this is very important to have a control over what comes into your system” (a2, data and solution architect) good data governance mechanisms, coupled with data management best practices contribute to the successful implementation of data-driven value creation. the data governance issues related to data security, data privacy, and data quality could be managed effectively through data-stewardship, the right set of processes for extraction, transformation and loading, and the right tools and technologies. in this section, as mentioned earlier, the aspects to related to technology governance are included, and mechanisms related to people and process aspects of governance are not dealt with. 5.4 implications and limitations of this study this study has implications for both practitioners and academicians. for the practitioners, this study provides an overview of building an edw by encompassing data from disparate sources and external systems. this study provides insights to managers on the choices of data management solutions available, and the ideal use cases for the same. with this information, managers would be able to choose the right data management tool for a specific business purpose. this study provides inferences to academicians on the methods used for creating value propositions from data available in internal systems and external systems. academicians would be able to understand the components involved in building an edw, and the use of various data management solutions. this study provides extensive coverage of the processes needed for creating an edw. the subject matter experts used for the study have offered insights on generic processes used in data analysis. however, the insights offered on data architecture are oriented more towards practitioners. this study proposes research to be conducted in academic settings, to understand the concepts and theory behind implementing data management solutions. this suggested research would be able to bring perspective on the nature of transactions, properties of data conducive for certain analysis, extraction and transformation methods, and options available for analytics. 6 conclusion many of the findings from this study are already available as part of technical white papers, research articles, and conference proceedings. however, in this paper, the researchers have australasian journal of information systems balakrishnan, das & chattopadhyay 2020, vol 24, research article implementing data strategy 23 attempted to collate all those findings to arrive at a reference architecture for all types of data management solutions. the critical contribution from this paper is to provide a consolidated reference material that would provide input to practitioners on: a) using the right technology and methods for data analysis, b) implementing the right use cases for various data management technologies, c) recommending a reference architecture for managing various data management solutions and design considerations for implementing the solution, d) providing a literature review and overview of all types of data management solutions, their pros and cons, and how they are part of the data management eco-system, and e) providing a consolidated view on various technologies used for data management such as data lakes, dataspaces, virtualization, and real-time monitoring tools. this study has approached the journey of an organization towards data-enabled value creation through four levels of data processing, such as data extraction, data transformation, data exploration or information visualization, and value creation. the study has critical observations with respect to using rdbms, nosql, and in-memory data management solutions. the query on organizations supporting a variety of data management solutions, and the recommended data architecture for supporting the same, is aptly answered by one of the subject matter experts. the entire process of building an edw that enables further data analysis is succinctly quoted here: “the traditional data in various transactional systems, you store in relational databases; you have data from social media, and whatever other external sources – you want to bring it over to a nosql kind of environment which is hosted on a big data environment, or, you could use any other form of big data systems (systems supporting big data provided by various third parties), and then you use some level of processing and transformation to take all metrics that are relevant for your business needs, and then bring it (those metrics and kpi) into a structured kind of data warehouse, and you drive your information visualization (reporting and analysis) on top of that” (a1, senior data architect) organizations could create value by providing real-time analysis on data retrieved from sm, and an architect, when discussing the technology used for real-time analytics, has mentioned, “any data that does not have to be persisted, and you need real-time analysis, then go for newsql; here, what you do is – you stream the data into newsql in-memory database, enable all your analysis on the same; … newsql is not proven, and a bit expensive, and you have other options available as well – for doing real-time analysis (like kafka)” (a3, data architect) organizations manage the data-enabled value creation through governance mechanisms implemented in a holistic way to address challenges associated with data security, data privacy, data loss and recovery, and business continuity. the design considerations should include elements such as encryption at the database and hardware-level for data-at-rest, security through firewall at the data-centre level, data transfer over secured protocols for datain-transit, access controls at various levels, scheduled back-ups for data, data storage through both primary and fail-over instances, and data replication through active and passive modes. organizations have to implement robust information-governance mechanisms across data handling processes to achieve higher level of quality. one of the architects has commented on the aspect of data governance, “’ clean and good data’ enables good metrics, and good metrics enable great inferences – and hence, these things (data governance mechanisms and best practices) are very critical for australasian journal of information systems balakrishnan, das & chattopadhyay 2020, vol 24, research article implementing data strategy 24 implementing quality data management solutions that would drive your business;” (a3, data architect) in summary, data is considered as the gold of the 21st century (aviza, 2017), and using it the right way would yield economic returns at an epic scale. organizations could harvest the value embedded in data by implementing the right data management technologies, coupled with appropriate information governance mechanisms. acknowledgements we thank our interviewees rajendran subramanian, narayanan kandanchatha, mukundakumar kg, brijesh madhavan, shiju jalaludin, murali avanamuthu, and their team members for sharing their pearls of wisdom during the course of this research. we are immensely grateful to dr. s. varadhaganapathy for helping us in framing our research queries. we extend our sincere thanks to latha venkitachalam and baiju ambadan for providing comments that have greatly improved the manuscript. please note that any errors in the manuscript are our own, and the errors should not tarnish the reputations of these esteemed persons. references addagada, t. (2019). customer data protection: deriving value and ownership. retrieved may 24, 2019, from https://www.dataversity.net/customer-data-protection-deriving-valueand-ownership/ anderson, c., & li, m. (2017). five building blocks of a data-driven culture. retrieved may 27, 2019, from https://techcrunch.com/2017/06/23/five-building-blocks-of-a-data-drivenculture/ aviza, e. (2017). data is the gold of the 21st century. retrieved may 20, 2019, from https://www.cloudbakers.com/blog/data-is-the-gold-of-the-21st-century bowen, r., & smith, a. r. (2014). developing an enterprisewide data strategy. healthcare financial management, 68(4). braganza, a. (2004). rethinking the data – information – knowledge hierarchy : towards a casebased model. international journal of information management, 24, 347–356. https://doi.org/10.1016/j.ijinfomgt.2004.04.007 commercial bank, c. g. (2018). why dual approval matters. retrieved may 6, 2019, from https://businessaccess.citibank.citigroup.com/basprod/citiiwt/images/why_dual_appr oval_matters.pdf data integration. (2018). retrieved may 20, 2019, from https://www.dataintegration.info/dataintegration davenport, t. h. (2014). big data at work: dispelling the myths and uncovering the opportunities. harvard business review press. davenport, t. h., harris, j. g., long, d. w. de, & jacobson, a. l. (2001). data to knowledge to results: building an analytic capability. california management review, 43(2). davenport, t., & verma, a. (2018). it’s time to modernize your big data management techniques. retrieved may 18, 2019, from australasian journal of information systems balakrishnan, das & chattopadhyay 2020, vol 24, research article implementing data strategy 25 https://www2.deloitte.com/us/en/insights/topics/analytics/data-managementtechniques-approaches-tools.html doyle, m. (2017). the importance of a “data integration first” strategy. retrieved may 15, 2019, from https://www.dqglobal.com/2017/07/25/importance-data-integration/ foote, k. d. (2019a). a brief history of master data. retrieved may 12, 2019, from https://www.dataversity.net/a-brief-history-of-master-data/ foote, k. d. (2019b). data modeling in an agile world. retrieved may 15, 2019, from https://www.dataversity.net/data-modeling-in-an-agile-world/ franklin, m., halevy, a., & maier, d. (2005). from databases to dataspaces: a new abstraction for information management. sigmod record, 34(4), 27–33. https://doi.org/10.1145/1107499.1107502 ghosh, p. (2019). data governance and data quality use cases. retrieved may 12, 2019, from https://www.dataversity.net/data-governance-and-data-quality-use-cases/ griffin, j. (2005). data governance: a strategy for success. dm review, 15(8), 15, 70. grolinger, k., higashino, w. a., tiwari, a., & capretz, m. a. m. (2013). data management in cloud environments : nosql and newsql data stores. journal of cloud computing. https://doi.org/https://doi.org/10.1186/2192-113x-2-22 hendler, j. (2009). web 3.0 emerging. ieee computer society, 42(january), 111–113. huber, g. p. ., & power, d. j. . (1985). retrospective reports of strategic-level managers : guidelines for increasing their accuracy. strategic management journal, 6(2), 171–180. kambatla, k., kollias, g., kumar, v., & grama, a. (2014). trends in big data analytics. journal of parallel and distributed computing, 74(7), 2561–2573. https://doi.org/10.1016/j.jpdc.2014.01.003 kaur, k., & sachdeva, m. (2017). performance evaluation of newsql databases. in international conference on inventive systems and control (pp. 1–5). https://doi.org/10.1109/icisc.2017.8068585 khatri, v., & brown, c. v. (2010). designing data governance. communications of the acm, 53(1). https://doi.org/10.1145/1629175.1629210 kooper, m., maes, r., & lindgreen, r. e. (2011). information governance as a holistic approach to managing and leveraging information prepared for ibm corporation. international journal of information management, 31. korhonen, j. j., melleri, i., hiekkanen, k., & helenius, m. (2013). designing data governance structure : an organizational perspective. gstf journal on computing, 2(4), 11–17. https://doi.org/10.5176/2251-3043 lawton, g. (2019). 7 enterprise use cases for real-time streaming analytics. retrieved may 20, 2019, from https://searchbusinessanalytics.techtarget.com/feature/7-enterprise-use-cases-forreal-time-streaming-analytics lim, c., kim, k., kim, m., heo, j., kim, k., & maglio, p. p. (2018). from data to value : a ninefactor framework for data-based value creation in information-intensive services. australasian journal of information systems balakrishnan, das & chattopadhyay 2020, vol 24, research article implementing data strategy 26 international journal of information management, 39(january 2017), 121–135. https://doi.org/10.1016/j.ijinfomgt.2017.12.007 link, s., & prade, h. (2019). relational database schema design for uncertain data relational database schema design for uncertain data $. information systems. https://doi.org/10.1016/j.is.2019.04.003 lourenço, j. r., cabral, b., carreiro, p., vieira, m., & bernardino, j. (2015). choosing the right nosql database for the job: a quality attribute evaluation. journal of big data, 2(1), 1–26. https://doi.org/10.1186/s40537-015-0025-0 manyika, j., chui, m., brown, b., bughin, j., dobbs, r., roxburgh, c., & byers, a. h. (2011). big data: the next frontier for innovation , competition , and productivity. mckinsey global institute, (may). marco, d. (2006). understanding data governance and stewardship, part 1. dm review, 16(9), 28. mazzei, m. j., & noble, d. (2017). big data dreams : a framework for corporate strategy. business horizons, (60), 405–414. mcafee. (2019). overview of serbanes-oxley. retrieved may 20, 2019, from https://www.skyhighnetworks.com/cloud-compliance/sarbanes-oxley-encryptioncompliance-requirements/ mirza, h. t., chen, l., & chen, g. (2010). practicability of dataspace systems. international journal of digital content technology and its applications, 4(3), 233–243. https://doi.org/10.4156/jdcta.vol4.issue3.23 mohan, c. (2013). history repeats itself : sensible and nonsensql aspects of the nosql hoopla. ibm alamaden research center, 11–16. newman, d., & logan, d. (2009). governance is an essential building block for enterprise information management. gartner research, (may 2006). pääkkönen, p., & pakkala, d. (2015). big data research reference architecture and classification of technologies , products and services for big data systems. big data research, 2(4), 166–186. https://doi.org/10.1016/j.bdr.2015.01.001 patrizio, a. (2019). what is data virtualization? retrieved may 20, 2019, from https://www.datamation.com/big-data/what-is-data-virtualization.html pavlo, a., & aslett, m. (2016). what is really new with newsql ? sigmod record, 45(2), 45– 55. perera, s. (2018). a gentle introduction to stream processing. retrieved may 20, 2019, from https://medium.com/stream-processing/what-is-stream-processing-1eadfca11b97 pokorny, j. (2013). nosql databases: a step to database scalability in web environment. international journal of web information systems, 9(1), 69–82. https://doi.org/10.1108/17440081311316398 rocha, l., vale, f., cirilo, e., barbosa, d., & mourao, f. (2015). a framework for migrating relational datasets to nosql ∗. proceedia computer science, 51, 2593–2602. https://doi.org/10.1016/j.procs.2015.05.367 australasian journal of information systems balakrishnan, das & chattopadhyay 2020, vol 24, research article implementing data strategy 27 salido, j. (2010). data governance for privacy, confidentiality and compliance: a holistic approach. isaca journal, 6, 1–7. sareen, p., & kumar, p. (2015). nosql database and its comparison with sql database. international journal of computer science & communication networks, 5(5), 293–298. simsek, g. (2019). what is new about newsql? retrieved june 7, 2019, from https://softwareengineeringdaily.com/2019/02/24/what-is-new-about-newsql/ spivack, n. (2011). web 3.0: the third generation web is coming. retrieved may 18, 2019, from https://lifeboat.com/ex/web.3.0 stantic, b., & pokorny, j. (2014). opportunities in big data management and processing. databases and information systems viii. https://doi.org/10.3233/978-1-61499-458-9-15 techopedia. (2011). garbage in, garbage out (gigo). retrieved may 15, 2019, from https://www.techopedia.com/definition/3801/garbage-in-garbage-out-gigo weber, k., otto, b., & osterle, h. (2009). one size does not fit all — a contingency approach to data governance. acm journal of information quality, 1(1). https://doi.org/10.1145/1515693.1515696.http weill, p., & ross, j. (2005). a matrixed approach to designing it governance. mit sloan management review, 46(2), 26–34. yang, f. (2016). building a streaming analytics stack with apache kafka and druid. retrieved may 20, 2019, from https://www.confluent.io/blog/building-a-streaming-analytics-stackwith-apache-kafka-and-druid/ zack, m. h. (1999). managing codified knowledge. sloan management review, 40(4), 45–58. copyright: © 2020 balakrishnan, das & chattopadhyay. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v24i0.2541 http://creativecommons.org/licenses/by-nc/3.0/au/ http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems lai & liew 2021, vol 25, research article towards a cashless society 1 towards a cashless society: the effects of perceived convenience and security on gamified mobile payment platform adoption p. c. lai university malaya, malaysia ewilly j.y. liew monash university malaysia, malaysia ewilly.liew@monash.edu abstract integrating gamification into mobile payment platforms incentivizes people to use digital alternatives for payment and could spur user-centric, platform-mediated interactions. this study examines the relationship between perceived convenience and perceived security on individual users’ intention to use a gamified mobile payment platform in malaysia; a developing country envisioned to build a cashless society. the partial least square structural equation modeling (pls-sem) technique is employed on a final sample of 388 online users. the results show that perceived convenience has a strong but indirect effect on the intention to use. perceived security has a strong and direct effect on intention to use and mediates the relationship between perceived convenience and intention to use. furthermore, the reliability aspect of security is a top priority concern for users interested in using mobile payment. the multi-functional aspect of convenience is a top priority concern to attract users who are not interested in using mobile payment at first. the study discusses theoretical and practical implications for developing a dual strategy of ‘ensuring convenience’ and ‘assuring security’ to encourage the gamified mobile payment platform adoption in developing countries. keywords: mobile payment, gamification, perceived convenience, perceived security, developing country 1 introduction the emergence of digital platforms and new technologies has created a new trajectory for technology-mediated payment transactions and multichannel financial services delivery. unlike the highly conservative and regulated banking industry, the non-banking service industry has been more fluid, flexible, and customizable (alhassan, li, reddy, & duppati, 2020). mobile payment in the service marketing industry provides consumers a cost-effective alternative to pay for products, services, and information transactions digitally via wireless technologies (lai & scheela, 2018). integrating gamification into the mobile payment platform as deliberate incorporation of user experience (ux) design element could spur a multichannel mobile marketing experience (högberg, shams, & wästlund, 2019; lemon & verhoef, 2016). platform-mediated interactions between service providers and users play a central role in creating value (mukerji & roy, 2019). gamification on mobile payment platform capitalizes on user-centric interaction to encourage consumer engagement (hofacker, de ruyter, lurie, manchanda, & donaldson, 2016), motivate purchase decisions (huotari & hamari, 2017), and ultimately, increase sales revenue (ramadan & farah, 2017). thus, there emerged a potential australasian journal of information systems lai & liew 2021, vol 25, research article towards a cashless society 2 paradigm shift in the way businesses convert consumer engagement to actual sales anytime, anywhere. the increasingly complex and diverse digital ecologies on the internet have a significant impact on individuals and organizations living in a connected society (linger & hasan, 2020). within the platform ecosystem, businesses face pressures to diversify their marketing channels and make optimal pricing decisions (zhang, xu, & bai, 2020) for gaining competitive advantages and larger market shares. mobile payment platforms open a new avenue for businesses to create an interactive marketing strategy that would reduce consumers’ buying time and improve shopping quality. consumer-centric convenience is a significant contributing factor to advance the service economy (cao & zhu, 2019) and accelerate mobile services such as in-app advertising (truong, nkhoma, & pansuwong, 2019), internet banking (kumar, sachan, & kumar, 2020), and mobile payment (de kerviler, demoulin, & zidda, 2016; teo, tan, ooi, hew, & yew, 2015). perceived convenience from a mobile payment perspective refers to consumers’ perceived expenditures of time and effort to conduct payment transactions more readily (berry, seiders, & grewal, 2002; gensler, verhoef, & böhm, 2012). however, what businesses understood as consumercentric convenience, thereby influencing their marketing strategy decision, may not necessarily coincide with what consumers perceive as convenient. furthermore, a meta-analysis review conducted by liu and colleagues (2019) found that consumers living in cash-centric developing countries are particularly positive towards adopting mobile payment. plausible reasons could be the rapid proliferation of affordable smartphones and the rising awareness among tech-savvy consumers to catch up with fastpaced urban lifestyles (poushter, bishop, & chew, 2018). due to these reasons, developing countries have a very high potential for using mobile payment platforms despite the relatively lower broadband internet penetration (lai & scheela, 2018; mukerji & roy, 2019). nevertheless, not all developing countries have successfully implemented mobile payment solutions (panhwer, pitafi, memon, & memon, 2020), subject to the country-level idiosyncratic risk and volatility. also, unlike the developed counterparts, developing countries tend to have different development stages (liew, vaithilingam, & nair, 2014; mukerji & roy, 2019; sharma & gupta, 2009) that might affect consumer perception and intention to adopt mobile payment differently. moreover, the regulatory environment is an essential aspect of mobile payment adoption that calls for attention, particularly among the developing countries. the unesco (zelezny-green, voslon, & conole, 2018) reported that developing countries face four major barriers to increasing digital inclusion due to lack of infrastructure, affordability, user capabilities, and incentives (schmida, bernard, zakaras, lovegrove, & swingle, 2017). these barriers cause consumers to be sceptical about using “online money” and affect their interest in adopting mobile payment into their daily lifestyle. perceived security refers to the degree to which consumers believe that using mobile payment procedures will be secure (lai, 2017). mobile payment consumers should feel certain and unthreatened that their personal and bank information provided for the transaction remains safe (de kerviler et al., 2016; kumar et al., 2020; lai, 2017). there has been a growing literature on mobile payment (chen & li, 2017; de kerviler et al., 2016; miao & jayakar, 2016; o’reilly, duane, & andreev, 2012; wu, liu, & huang, 2017; xin, techatassanasoontorn, & tan, 2015). meta-analysis results from sixty-one information systems (is) studies highlighted the importance of usage intention and ascertained that perceived australasian journal of information systems lai & liew 2021, vol 25, research article towards a cashless society 3 usefulness and perceived ease of use are among the top contributing factors to mobile payment system adoption (liu et al., 2019). most previous studies have employed dominant theoretical approaches to investigate mobile payment (liu et al., 2019) such as the technology acceptance model (tam) (davis, 1989; venkatesh & davis, 2000), the theory of planned behavior (tpb) (taylor & todd, 1995), or unified theory of acceptance and use of technology (utaut) (venkatesh, morris, gordon, & davis, 2003). apart from the above well-worn theoretical frameworks, there is an increasing need for incorporating consumer-centric measures (pwc, 2019) such as perceived value in terms of benefits and risks associated with mobile payment (de kerviler et al., 2016). perceptions of benefit and risk are inversely related and psychologically linked to consumers’ overall affective evaluation of activity or technology (alhakami & slovic, 1994). value perceptions are crucial for success in the service industry (boksberger & melsen, 2011). nonetheless, systematic literature reviews have revealed a lack of focus on the effects of perceived convenience (benefit) and perceived security (cost of sacrifice) on user intention to adopt mobile payment (liu et al., 2019; ström, vendel, & bredican, 2014). little is also known about the relationship between perceived convenience and perceived security when incorporating gamification onto the mobile payment platform. the motivation for conducting this study comes from our inquiry about the potential tradeoff between convenience and security concerns perceived by individual users interacting on the mobile platforms (kim, chan, & gupta, 2007). the potential trade-off could impede the effectiveness of gamified mobile payment platform adoption, resulting in detrimental effects to the overall success of e-commerce development in a country (cao & zhu, 2019; chaudhry, farash, naqvi, & sher, 2016). to address this gap in the literature, this study hopes to respond to liu et al.’s (2019) call for considering ‘the place where consumers live’ as a potential factor that could affect consumer mobile payment adoption, thus contextualizing the current study to malaysia. drawing from a value-based theoretical approach, this study seeks to examine the effects of perceived convenience and perceived security on consumers’ intention to use a gamified mobile payment platform in malaysia. part of the novelty of this study is the use of blockchain technology and the incorporation of gamification as a user experience (ux) design element on a gamified mobile payment platform. we also explore multigroup differences based on user interest for mobile payment adoption (chauhan, mukhopadhyay, & jaiswal, 2018; yuniarta & purnamawati, 2021). from an interactive marketing perspective, gamification on a mobile payment platform would encourage user engagement to act on incentives gained on mobile application (or app) towards making the actual purchase (högberg et al., 2019) and meet the expectations of least digitally-skilled consumers (colby & parasuraman, 2003). consumers should feel comfortable and safe and be willing to use mobile payment to conduct transactions readily in their daily lives. the rest of the paper is structured as follows. section 2 outlines mobile payment development in malaysia. software architecture and design of a gamified mobile payment platform are outlined in the next section. section 4 presents a review of relevant literature and hypotheses development. the methodology and empirical results are presented in sections 5 and 6. theoretical and practical implications are discussed in the following sections 7 and 8. the paper concludes with summary remarks, limitations, and future research directions. australasian journal of information systems lai & liew 2021, vol 25, research article towards a cashless society 4 2 mobile payment development in malaysia in malaysia, the government envisioned to build a cashless society by introducing the interoperable credit transfer framework (ictf). major mobile payment platform providers under the ictf would eventually operate on a shared real-time retail payments platform (rpp) (lee & khaw, 2018). the central bank has issued forty-nine (49) licenses to platform providers generating a volume of 2.2 million mobile banking transactions, 7.59 million internet banking transactions, and 1.92 million transactions to date (bank negara malaysia, 2020). with about fifty mobile payment providers serving 32.4 million user populations in malaysia, these providers compete for competitive intelligence to differentiate themselves in disrupting consumers’ spending patterns and payment habits faster than their competitors (chen, zhu, & xie, 2004). meanwhile, the global covid-19 pandemic has stirred unprecedented safety concerns about coronavirus transmission via cash (auer, cornelli, & frost, 2020). world health organization (who) encourages contactless payment to prevent the spread of covid-19 with physical cash notes handling (huang, 2020). the malaysian government has urged people to use cashless payment during covid-19 by allocating myr750 million (usd186 million) to drive mobile payment adoption (fong, 2020) and introducing initiatives such as epenjana e-wallet stimulus, shop malaysia online initiative, and the micro, small and medium enterprises (msme) e-commerce campaign (the star, 2020). the top five popular mobile payment apps that have recently gained traction in malaysia are grabpay, touch n’ go ewallet, boostpay, fave, and bigpay based on a combined apple and android phone monthly active users’ statistics (chew, 2019). increasing acceptance of these emerging platforms demonstrates a certain level of technology convergence in the national digital ecosystem that would afford successful implementation and commercialization of other similar platforms in malaysia (lai & scheela, 2018). mobile payment is gaining widespread acceptance in developing countries, accounting for 35% of global growth with a projected compound annual growth rate (cagr) of 23.5% (capgemini, 2019). more specifically, the global consumer insight survey (pwc, 2019) showed that amongst the top ten are developing countries in southeast asia, such as thailand, vietnam, indonesia, philippines, and malaysia. despite having the lowest mobile payment adoption rate amongst all, malaysia has witnessed a promising year-over-year growth of 17% in 2019 (pwc, 2019). indications of possible consumer-centric acceptance and resistance provide an avenue for this study to investigate factors that encourage or discourage users’ intention to use mobile payment in malaysia. this study would also provide insights for other countries that share a similar developmental trajectory of consumer perception and national digital infrastructure. 3 software architecture and design of a gamified mobile payment platform this study focuses on a gamified mobile payment platform that can integrate payment cards, the internet, and mobile applications in one solution (lai, 2017; lai, toh, & alkhrabsheh, 2019). the platform’s software architecture was developed using blockchain technology akin to electronic coins in a smartphone-based shopping system deployed in japan (tashiro, 2018). blockchain-based applications leverage the distributed peer-to-peer network such that each member in the blockchain network can openly access and directly verify peer-to-peer australasian journal of information systems lai & liew 2021, vol 25, research article towards a cashless society 5 transactions without involving an intermediary or a central authority (harrison, hair, & ajjan, 2021). each transaction is cryptographically secured and visibly stored on a blockchain as chronological, immutable, and auditable records to protect against fraud or malicious activities that compromise security aspects of the platform (casino, dasaklis, & patsakis, 2019). figure 1 depicts a general multi-tier software architecture of the gamified mobile payment platform. the primary layer is built on public blockchain technology using open-source cryptocurrencies such as bitcoin and ethereum. smart contracts functionality of these cryptocurrencies will automatically execute peer-to-peer payment transactions that meet predetermined terms and conditions and electronically verify the chain of ownership using pairs of public keys (for identification) and private keys (authentication and encryption) (nakamoto, 2008). the secondary layer comprises private blockchain technology for the database security protocol (casino et al., 2019) and proprietary technology for scalability and payment tracing protocol (lai, 2017). the third application layer contains business logic that determines how the platform will operate with the local context's rules and regulations. user identification will be verified in compliance with the national bank of malaysia’s electronic know-your-customer (e-kyc) guideline stipulated under the national anti-money laundering policy (bank negara malaysia, 2019). the presentation layer of a gamified mobile payment platform runs on the user’s mobile device. gamification enhances the mobile-based presentation layer with game-style rewards, which have found to positively attract user engagement (högberg et al., 2019; yang, asaad, & dwivedi, 2017), trigger actual purchase decision (huotari & hamari, 2017), and further increase the brand equity of participating stores (xi & hamari, 2020). gamification is a valueadding extension to any compatible software architecture of a mobile payment platform. the “game on app” feature has a twofold function of engaging and motivating users for purchase (alsawaier, 2018). in return, participating stores can analyze the shopping data of their customers who are also app users. figure 1. software architecture of a gamified mobile payment platform australasian journal of information systems lai & liew 2021, vol 25, research article towards a cashless society 6 figures 2–4 illustrate the design mechanism of a gamified mobile payment platform. loyalty point or stamp collection is typically used in incentive design for the retail or service industry (huotari & hamari, 2017). loyalty program that incorporates gamified elements could drive consumer motivation and engagement value (hollebeek, das, & shukla, 2021). the app allows users to convert the loyalty points earned into monetary tokens that can be used as digital money to pay for actual purchases transacted in any participating online or offline stores. figure 2. a user experience in earning loyalty points through game on app (own illustration) figure 3. converting loyalty points into blockchain monetary tokens (own illustration) figure 2 showed a user experience in collecting loyalty points or stamp through three different sets of gamification activities across three participating stores (shop a, shop b, and shop c). the user encounters an augmented reality or virtual reality experience on the app upon visiting the physical shop a. the user plays a game on the app designed for shop b and responds to a survey at shop c. the user earns loyalty points through a series of mobile games across the participating stores. figure 3 shows that users can choose to convert the loyalty points earned into monetary tokens, i.e., electronic coins issued solely on the gamified mobile australasian journal of information systems lai & liew 2021, vol 25, research article towards a cashless society 7 payment app. following this, users can use the monetary tokens expressed in the local currency (i.e., ringgit malaysia) to pay for products, services, and information transacted at the participating stores (see figure 4). users can also share monetary tokens using peer-to-peer online money transfer for shopping purchases, commonly seen in developed countries but not as common in malaysia. figure 4. using blockchain monetary tokens to pay for various transactions (own illustration) the mobile payment platform ecosystem encompasses consumers’ mobile devices, internet data, payment details, and the retailers’ point-of-sale (pos) system in a physical store or the checkout cart system on an online store (de kerviler et al., 2016; lai et al., 2019). consumers have the site option to scan qr codes directly from the mobile payment app. they could also ‘tap and pay’ at the physical pos kiosk using mobile devices enabled with near field communication (nfc) technology. the fastest straightforward approach would be performing payment transactions remotely from the store and directly on a multi-functional mobile payment platform (lai & scheela, 2018). 4 literature review and hypotheses development to date, davis’ (1989) technology acceptance model (tam) has transpired as one of the most employed models to study information technology adoption behavior, including mobile payment adoption and use at the individual level (liu et al., 2019). perceived usefulness (pu) and perceived ease of use (peu) are two core constructs in tam postulated for influencing users’ attitudes and intentions towards various types of technology adoption (davis, bagozzi, & warshaw, 1989). there had been a high level of consensus among the mobile payment literature that pu and peu have demonstrated significant positive impacts on consumers’ intention to adopt and use mobile payment (liu et al., 2019). under such a circumstance, the inclusion of pu and peu in the conceptual model might be overly generalized (lim, 2018). an alternative approach would be kim, chan, and gupta’s (2007) value-based adoption model (vam) from the value maximization perspective. the original vam examines mobile internet adoption among individuals who play a dual role of technology users and service australasian journal of information systems lai & liew 2021, vol 25, research article towards a cashless society 8 consumers, similar to the context of mobile payment adoption (kim et al., 2007). due to the voluntary nature of adoption, users tend to evaluate the trade-off between total benefits received and total costs of sacrifice involved in using mobile payment services based on value perceptions (de kerviler et al., 2016; kim et al., 2007). from the utilitarian perspective of perceived value theory, individual will maximize positive utility (perceived benefits) and minimize negative utility (perceived costs of sacrifice) to assess the overall net utility of a product or service in the service industry (boksberger & melsen, 2011; sánchez-fernández & iniesta-bonillo, 2007). convenience exemplifies perceived utilitarian value that appears particularly important for influencing mobile payment adoption (de kerviler et al., 2016; teo et al., 2015). in contrast, security risks represent perceived costs of sacrifice associated with consumption that negatively connote potential sacrifices in terms of monetary and psychological losses (de kerviler et al., 2016). studies in malaysia have extended tam to include convenience and security as external stimuli in influencing pu and peu for mobile payment adoption (lai, 2017; lai et al., 2019). however, liu et al.’s (2019) meta-analysis of sixty-one (61) information systems (is) studies found that neither perceived convenience nor perceived security correlates with pu and peu for mobile payment adoption. this study decided not to include pu and peu as the de facto variables in our conceptual model as we are more interested in examining the effects of convenience and security on adoption intention from the perceived value perspective (see figure 5). vam is preferred over tam to appropriately evaluate the cost (i.e., perceived security) and benefit (i.e., perceived convenience) associated with adoption intention among consumers (nayal, pandey, & paul, 2021; vishwakarma, mukherjee, & datta, 2020). drawing from perceived value theory, this study proposed a research model that excludes tam’s core constructs, namely pu and peu. instead, our proposed model directly examines the effect of perceived convenience and perceived security on individual users’ behavioral intention to use a gamified mobile payment platform in the service marketing industry. the proposed model also investigates the relationship between convenience and security; both were previously conceptualized as external constructs in tam (lai, 2017; lai et al., 2019). figure 5. proposed research model perceived convenience perceived security intention to use h1 h2 h3 multigroup analysis: overall, interested users, and non-interested users australasian journal of information systems lai & liew 2021, vol 25, research article towards a cashless society 9 4.1 convenience the concept of perceived convenience has been popular in conventional shopping (farquhar & rowley, 2009) and online shopping (jiang, yang, & jun, 2013) as essential for customer satisfaction and intention (colwell, aung, kanetkar, & holden, 2008; seiders, voss, godfrey, & grewal, 2007). convenience is a crucial element of perceived value for e-commerce adoption (weissman, 2012) and particularly mobile payment adoption (de kerviler et al., 2016; teo et al., 2015). copeland (1923) first introduced the term convenience, referring to the degree to which little cognitive efforts are required for decision making, including time-buying or timesaving efforts. a product or service is deemed to be convenient if it saves time and reduces cognitive, emotional, and physical burdens for users (chang & polonsky, 2012). in managing consumer behavior, kaufmann (2014) argued that a consumer-centric definition of convenience is preferred over the researcher’s definition. furthermore, consumers’ perceived convenience is a context-based concept that could vary from one context to another (jiang et al., 2013), hence relevant to this study in examining the ‘place’ factor. the design of a gamified mobile payment platform should have consumers in mind – by understanding convenience from the consumers’ perspective and where consumers live. past studies (de kerviler et al., 2016; kumar et al., 2020; lai et al., 2019; mukerji & roy, 2019) showed that consumers prefer convenient and practical mobile payment channels to pay for transactions anytime and anywhere on mobile phones. time-poor consumers seek valueadding mobile payment services by evaluating benefits offered versus costs involved in making a mobile payment transaction (de kerviler et al., 2016; ström et al., 2014). this study conceptualizes perceived convenience as encompassing six aspects of utilitarian value: saving time, carrying less physical cash, an emergency need of funds, multi-functional design, customizes applications, and a single platform (chang & polonsky, 2012; c. kim, mirusmonov, & lee, 2010; lai et al., 2019). consistent with the concept of consumer-centric convenience, consumers will decide ‘when’ and ‘where’ to shop in their best interest. mobile payment becomes appealing to consumers living in a cash-centric culture, typically among the developing countries (liu et al., 2019). as users’ desire for convenience mounts to the extent of frequently diverting to mobile payment as an alternative payment option, this study proposes that user intention to adopt mobile payment will increase. gamification features on the mobile payment platform would incentify and attract the use of mobile payment. thus, our first hypothesis is as follows: h1: perceived convenience positively impact intention to use gamified mobile payment. 4.2 security security is an increasing concern in the platform ecosystem following the rapid growth of technology advancement and e-commerce offerings (kumar et al., 2020; lai, 2017; liew et al., 2014; morosan & defranco, 2016). the cryptography of blockchain technology guarantees privacy and security protocol against malicious activities that alter the chain of peer-to-peer transactions (casino et al., 2019). unlike privacy that refers to user control over personal information, security connotes protection over the mobile payment platform from threats such as unauthorized intrusion, illegal theft, or destruction (gupta & dubey, 2016; lai, 2017; liu et al., 2019; shin, 2009). this study conceptualizes security as encompassing six aspects of the blockchain guarantees: reliability, privacy, authentication and authorization, integrity, nonrepudiation, and confidentiality (casino et al., 2019; lai, 2017). this study focuses on user australasian journal of information systems lai & liew 2021, vol 25, research article towards a cashless society 10 perception of the level of security exposure distinctly different from the platform's inherent risks. past studies found that users’ intention to use mobile payment is susceptible to the level of security exposure they perceive at the point of adoption (de kerviler et al., 2016; lai, 2017). generally, users tend to accept a new payment alternative if they perceive lower security exposure (such as feeling reliable, safe, and unthreatened) that outweighs the old payment methods (kumar et al., 2020; lai, 2017). a reliable and trustworthy gamified mobile payment platform will incorporate sufficient security measures to assure users peace of mind in performing their transactions. users hence do not need to worry about fraud, identity theft, or information leakage during the mobile payment process. this study proposes that favorable perceived security will assure users of the reliability, privacy, and safety concerns of using a gamified mobile payment platform. thus, our second hypothesis is as follows: h2: perceived security positively impact intention to use gamified mobile payment. 4.3 convenience to security the relationship between perceived convenience and perceived security for mobile payment adoption has been sidelined or subsumed under other variables of concern in past research (liu et al., 2019). on the one hand, high profile service providers such as apple, google, alibaba, grab, airasia have sparked renewed interest around gamified mobile payment platforms that might trigger large-scale adoption towards a promising future of convenience (johnson, kiser, washington, & torres, 2018). on the other hand, security and privacy have been significant barriers to adoption (chutikulrungsee, 2016; senarathna, yeoh, warren, & salzman, 2016), often subject to the country-level stages of development and digital infrastructure readiness, particularly in the developing context (liew et al., 2014; schmida et al., 2017). the gamification on a mobile payment platform serves as a design of convenience that would encourage mobile payment use. in contrast, its monetary tokenization serves as a security measure to safeguard from unauthorized breaches. while studies seem to suggest that perceived convenience contributes to mobile payment adoption and perceived security requires further investigation (lai et al., 2019; weir, douglas, carruthers, & jack, 2009), burmeister (2015) stressed the importance of security over convenience. still, little is known whether perceived convenience and perceived security should co-exist for mobile payment adoption or would be a trade-off with an inverse relationship. the relationship might also vary across different groups of interested and non-interested users. this study proposes that perceived security would mediate the relationship between perceived convenience and intention to use a gamified mobile payment platform. the relationship between perceived convenience and perceived security might be akin to the effect from peu to pu in the tam. users who perceive that little cognitive efforts are required to perform transactions using mobile payment (i.e., favorable perceived convenience) will form favorable perceived security that encourages intention to use a gamified mobile payment platform. thus, our third hypothesis is as follows: h3: perceived convenience positively impacts perceived security. australasian journal of information systems lai & liew 2021, vol 25, research article towards a cashless society 11 5 methodology 5.1 data data was collected using an online questionnaire survey based on a non-probabilistic purposeful sampling method. the study targeted only malaysian users who have had experience purchasing goods or using services online over the past one (1) year using a gamified mobile payment platform. an email blast was sent to a list of 1358 users obtained from a series of ict investor exhibition events held in the klang valley region of malaysia. the questionnaire items for perceived convenience (6 items, c1-6), perceived security (6 items, s16), and intention to use (4 items, int1-4) were adopted from past studies developed and validated in malaysia (lai, 2017; lai et al., 2019). 5.2 sample statistics the final sample contained 388 usable responses for further analysis after removing nonresponses and incomplete responses. male users (230, 59.3%) were more than female users (158, 40.7%), mostly in the age groups of 26-40 (181, 46.6%) and 41-55 (102, 26.3%). there were 87 (22.4%) young users below 25 years old and 18 (4.6%) elderly users above the retirement age of 56. most of the respondents were single (204, 52.6%) than married (184, 47.4%) and well educated with 264 of them (68.0%) graduated from colleges or universities, 78 (20.1%) from secondary or high school, and 46 (11.9%) from graduate schools. besides the 78 respondents still studying, most respondents were working in the ict/ manufacturing sector (172, 44.3%), followed by other sectors (67, 17.3%), banking/ finance (37, 9.5%), and retail/ hypermarket (34, 8.8%). 165 respondents were holding middle management positions (42.5%), 56 junior management positions (14.4%), 46 professional positions (11.9%), 24 top management positions (6.2%), and 97 other positions (25.0%). in particular, there were more interested users in using gamified mobile payment platform (240 (61.9%) strongly interested; 14 (3.6%) interested) compared to users who were just neutral (106, 27.3%) or not interested (26, 6.7%) or strongly not interested (2, 0.5%). 5.3 statistical analysis a multivariate analysis based on the partial least squares structural equation modeling (plssem) method was conducted using the smartpls 3 software (ringle, wende, & becker, 2015). the pls-sem emphasizes a causal-predictive approach to sem that provides causal explanations without imposing distributional assumptions (hair, sarstedt, ringle, & gudergan, 2018). in comparison, covariance-based cb-sem has numerous restrictive assumptions and focuses on model fitting for theory confirmation (hair, risher, sarstedt, & ringle, 2019). in this study, pls-sem was preferred as a non-parametric, variance-based approach to (i) explore a value-based theoretical framework that is lesser-known than the dominant tam, (ii) explain the causal relationship between perceived convenience and perceived security on adoption intention, and (iii) predict individual users’ intention to use a gamified mobile payment platform. 6 empirical results this study followed hair et al.’s (2019) research guidelines for evaluating the measurement and structural models in pls-sem. the measurement model satisfied all of the evaluation criteria for internal consistency reliability, convergent validity, and discriminant validity. the australasian journal of information systems lai & liew 2021, vol 25, research article towards a cashless society 12 hypotheses of this study were tested in the structural model. statistical inferences were based on the associated t-statistics above ±1.96 for a two-tailed test at the 5% level of significance. 6.1 measurement model for construct reliability and validity, all constructs met the evaluation requirements with composite reliability (cr) greater than the threshold of 0.70 and average variance extracted (ave) greater than the threshold of 0.50 (see table 1). almost all of the item loadings were greater than 0.708, with most of the items above 0.800. only one item (s4) had a loading of 0.646 but did not significantly threaten the internal consistency and convergent validity of the related construct (i.e., security) (hair et al., 2018); cr=0.917, ave=0.691 (excluded) versus cr=0.914, ave=0.643 (included). hence item s4 was retained in the model. the latent variable correlation between constructs was also reported in table 1. cr ave convenience security intention to use convenience 0.961 0.806 1.000 security 0.914 0.643 0.516 1.000 intention to use 0.909 0.714 0.426 0.573 1.000 table 1. construct reliability, validity, and latent variable correlation discriminant validity of the constructs was established based on henseler, ringle, and sarstedt’s (2015a) heterotrait-monotrait ratio of correlations (htmt). table 2 showed that all inter-construct htmt values were below the htmt.85 criterion threshold of 0.85, and the twotailed 95% bias-corrected confidence intervals of htmt were within the unit interval. the model fit was achieved based on the standardized root means square residual (srmr) of 0.088 below the threshold of 0.10 (henseler et al., 2014). table 3 showed that the measurement model did not have a multicollinearity issue, as all inner vif values were below the threshold of 3.3 (hair, hult, ringle, & sarstedt, 2014). convenience security intention to use convenience security 0.545*** [0.387, 0.666] intention to use 0.463*** [0.318, 0.578] 0.649*** [0.491, 0.757] table 2. discriminant validity assessment based on htmt criterion note: significance level (bootstrapped) ***p < 0.01, **p < 0.05, *p<0.10, n.s. non-significant. the values in brackets represent the 95% bias-corrected confidence interval of the htmt values obtained from a bootstrapping procedure with 5000 resamples. convenience security convenience security 1.000 intention to use 1.362 1.362 table 3. inner vif values 6.2 structural model for hypotheses testing, the structural model was run with a bootstrapping procedure of 5000 resamples based on a distribution-free sampling method. based on r2 statistics, the model explained a 35.2% variance of users’ intention to use a gamified mobile payment platform. in australasian journal of information systems lai & liew 2021, vol 25, research article towards a cashless society 13 particular, perceived convenience explained a 26.6% variance of perceived security. all three hypotheses were supported. from table 4, the results showed that perceived convenience had a strong direct positive effect on perceived security (β= 0.516, p<0.000), but a weak direct positive effect on users’ intention to use (β=0.178, p<0.002). the latter effect was weaker than the former, based on the magnitude of the path coefficients. perceived convenience had a moderate indirect effect (β=0.248, p<0.000) but a strong total effect (β= 0.426, p<0.000) on users’ intention to use. compared to the total effect of perceived convenience, perceived security still had a stronger direct positive effect on the intention to use (β=0.482, p<0.000). the total effect of perceived security is the same as its direct effect. based on cohen’s (1988) f 2 statistics, perceived convenience had a large effect size on security (f 2 = 0.362) but very small effect size on the intention to use (f 2 = 0.036), whereas perceived security had a medium effect size on the intention to use (f 2 = 0.263). hypothesized relationships direct effect effect size f 2 indirect effect total effect h1 convenience  intention to use 0.178 (3.168)*** 0.036 (1.315) n.s. 0.248 (6.285)*** 0.426 (7.436)*** h2 security  intention to use 0.482 (7.987)*** 0.263 (2.855)*** 0.482 (8.021)*** h3 convenience  security 0.516 (9.108)*** 0.362 (2.819)*** 0.516 (9.148)*** table 4. direct, indirect, total effects of hypothesized relationships note: significance level (bootstrapped) ***p < 0.01, **p < 0.05, *p<0.10, n.s. non-significant. cohen’s f^2 effect size threshold: large > 0.35, medium > 0.15, small > 0.02. the blindfolding procedure showed that all construct cross-validated redundancy q2 values were above the threshold of zero; q2=0.152 for security and q2=0.230 for intention to use. hence sufficient predictive relevance of the structural model was established. predicted indicators pls-sem prediction linear model (lm) prediction pls-sem versus lm rmse mae mape q2 rmse mae mape q2 rmse mae mape int1 0.491 0.403 18.895 0.196 0.484 0.381 17.877 0.221 lm lm lm int2 0.562 0.489 22.839 0.098 0.566 0.492 22.930 0.084 pls pls pls int3 0.565 0.478 22.690 0.036 0.565 0.488 23.130 0.036 lm pls pls int4 0.502 0.397 18.491 0.132 0.518 0.408 18.831 0.077 pls pls pls s1 0.491 0.326 17.308 0.219 0.497 0.310 16.533 0.198 pls lm lm s2 0.568 0.475 23.613 0.236 0.580 0.464 23.026 0.203 pls lm lm s3 0.638 0.433 16.842 0.066 0.646 0.437 17.219 0.041 pls pls pls s4 0.687 0.558 19.988 0.034 0.681 0.544 19.824 0.051 lm lm lm s5 0.561 0.482 22.888 0.179 0.565 0.481 22.913 0.166 pls lm pls s6 0.508 0.428 19.419 0.177 0.514 0.428 19.478 0.156 pls pls pls table 5. comparing information criteria between pls-sem and lm predictions note: the intention to use indicators (int1-4) were predicted using convenience indicators (c1-6) on hypothesis h1 and security indicators (s1-6) on hypothesis h2. the security indicators (s1-6) were predicted using convenience indicators (c1-6) on hypothesis h3. comparisons were made based on which model has the lower value of information criteria – root mean squared error of predictions (rmse), mean absolute error (mae), mean absolute percentage error (mape), and blindfolding-based q2. using a plspredict procedure, this study assesses the predictive relevance of the empirical findings. the pls predict procedure was conducted on 10-fold cross-validation using training estimates from the pls-sem model to predict the testing sample (shmueli et al., 2019). table australasian journal of information systems lai & liew 2021, vol 25, research article towards a cashless society 14 5 shows that all indicators predicted in the pls-sem model had q2 values greater than the zero thresholds. moreover, the pls-sem model outperformed the linear regression model (lm) with lower values of prediction errors (rmse, mae, and mape) for the majority of predicted indicators (i.e., 18 out of 30). the results indicated that the pls-sem model has medium predictive power and is better than a linear model prediction. observed heterogeneity might likely present in this study, depending on the types of users. users with different levels of prior interest in using a gamified mobile payment platform may perceive convenience and security differently (chauhan et al., 2018; yuniarta & purnamawati, 2021). first, the study needs to ascertain if a meaningful multigroup comparison exists across interested users, non-interested users, and all users. a measurement invariance of the composite models (micom) procedure was conducted to assess the measurement invariance of the empirical model (henseler, ringle, & sarstedt, 2015b; henseler et al., 2015). based on the results, the empirical model established configural invariance (identical pls models, data treatment, and algorithm settings) but not the composite invariance. the results of 1000 permutations in table 6 shows that convenience is the only construct with between-group correlation c (0.999) not significantly different from 1 (p = 0.121 > 0.05). as such, a multigroup comparison using the empirical model would not be meaningful. however, this study could still provide a separate analysis of each user group without comparing the results between interested users and non-interested users. constructs correlation, c 95% confidence interval p-value compositional invariance? convenience 0.999 [0.999, 1.000] 0.121 yes intention to use 0.923 [0.991, 0.997] 0.000 no security 0.987 [0.994, 0.998] 0.006 no table 6. measurement invariance assessment finally, ringle and sarstedt’s (2016) importance-performance map analysis (ipma) was conducted to gain additional insights into the effects of perceived convenience and perceived security on users’ intention to use a gamified mobile payment platform. ipma contrasted the importance of hypothesized relationships (based on unstandardized total effects of the pls estimates) with the performance of related constructs (based on average latent variable scores rescaled on 0-100). ipma aims to identify priority areas of constructs and indicators that exhibit high-importance but low-performance scores to guide managerial prioritization for performance improvement (ringle & sarstedt, 2016). based on the ipma results at the construct level, table 7 shows that security could be a priority area for performance improvement among interested users and all users. in contrast, noninterested users would prioritize convenience for performance improvement. the constructlevel results found that the security construct shows higher importance (0.46 for all users, 0.483 for interested users) but also higher performance (40.59 for all users, 37.31 for interested users). prioritizing security is still possible (ringle & sarstedt, 2016) and preferred as the pls-sem results have previously shown that security is strongly contributing to the total effect of convenience. thus, improving perceived security will indirectly improve perceived convenience, even though the latter has lower performance scores than the former. however, for non-interested users, the convenience construct is clearly a priority area with higher importance (0.258) but lower performance (37.44) scores. australasian journal of information systems lai & liew 2021, vol 25, research article towards a cashless society 15 importance (based on total effects) performance (based on mean latent variable scores) construct level: all (n=388) interested (n=254) noninterested (n=132) all (n=388) interested (n=254) noninterested (n=132) convenience 0.444 0.451 0.258 31.88 28.62 37.44 security 0.460 0.483 0.111 40.59 37.31 47.34 table 7. ipma construct-level priority areas for performance improvement note: importance based on total effects (x-axis), performance scores on the intention to use (y-axis). on average, a 1-unit increase in security would increase users’ intention to use (performance) by 0.46% for all users and 0.483% for interested users (importance). security improvement would affect interested users slightly more than all users. in contrast, among the noninterested users, a 1-unit increase in convenience would increase users’ intention to use (performance) by 0.258%. the results show that the impact of improving security concerns for interested users is larger than improving convenience concerns for non-interested users. based on the ipma results at the indicator level, table 8 shows the top 5 priority sub-areas for performance improvement that have indicators with high importance (total effects ≥ 0.08) but below-average performance (intention to use ≤ 0.50). among all users and the interested users, the top 5 priority sub-areas were mostly concerning security issues though in different order of importance. four of these indicators reflect the security aspects of reliability (s1), privacy (s2), non-repudiation (s5), and confidentiality (s6), except for one indicator representing the convenience of carrying less cash (c1). reliability (s1) topped the list with the highest importance (0.093) but the lowest performance score (28.54) for all users. although reliability (s1) was ranked third in terms of importance (0.093), it too had the lowest performance score (23.819) for the interested users. hence, the indicatorlevel results showed that the reliability (s1) aspect of a gamified mobile payment platform should have the top priority for performance improvement among interested users and all users. all (n=388) interested (n=254) non-interested (n=132) ind. imp perf ind. imp perf ind. imp perf s1 0.093 28.54 s6 0.098 39.08 c2 0.053 32.77 s6 0.092 42.59 c1 0.096 33.20 c3 0.045 36.74 s2 0.085 35.05 s1 0.093 23.82 c5 0.043 37.31 c1 0.084 37.08 s2 0.084 29.53 c4 0.042 38.07 s5 0.083 39.63 s5 0.083 34.84 c6 0.039 38.26 table 8. impa indicator-level priority areas for performance improvement note: ind = indicator. importance is based on unstandardized total effects (imp); performance is based on rescaled average latent variable scores (perf) on the other hand, non-interested users were mostly concerned with the convenience of having multi-function (c2), an emergency need of funds (c3), time-saving (c4), customized application (c5), and a single platform (c6); security was not listed among the top priorities. multi-function (c2) had the highest importance (0.053) but the lowest performance score (32.77). hence, the multi-functional aspect of a gamified mobile payment platform should have the top priority for performance improvement among the non-interested users. australasian journal of information systems lai & liew 2021, vol 25, research article towards a cashless society 16 7 theoretical contributions the empirical results supported the hypotheses in this study that users’ value perceptions of convenience and security positively impact their intention to use a gamified mobile payment platform. perceived security has a strong and direct influence on intention to use, while perceived convenience has a strong but indirect influence on intention to use via perceived security. the direct effect size from perceived convenience to intention to use was not significant. the results shed light on the relationship between convenience and security and how the strong bond of this relationship positively impacts mobile payment adoption. hence, platform providers and policymakers should focus on the dual strategy of ‘ensuring convenience’ and ‘assuring security’ to encourage the use of a gamified mobile payment platform. the results have implications for developing countries that share a similar development trajectory like malaysia. the sample demographic profile showed that more users were interested (n=254) and in favor of using a gamified mobile payment platform than users who are neutral or not interested (n=132). when users’ interests were analyzed separately because a multigroup analysis was not feasible, this study found two different results. for all users, particularly the interested users, the ‘reliability’ aspect of security should be prioritized for enhancing their intention to use a gamified mobile payment platform. however, for the non-interested users, the ‘multifunctional’ aspect of convenience should be prioritized for encouraging their adoption intention. the results provide insights on developing strategies that could ensure the multifunctional design of convenience to attract non-interested users and assure the reliability aspect of security to retain interested users. 8 practical implications malaysia has witnessed increasing digital readiness among individual and business users due to various technological reforms and initiatives led by the government. in collaboration with private telecommunication companies, the malaysian government has rigorously encouraged high-speed broadband and 4g mobile internet in urban and rural areas (economic planning unit, 2015). the success of mobile payment implementation in a volatile and competitive platform market calls for both demand-side and supply-side preconditions to work in harmony. for individual users, their intention to use a gamified mobile payment platform is dependent on the synergy between marketers’ design strategies and the country’s digital infrastructure. among the business users, informal institutions of co-investing and networking (e.g., business angel ecosystem) are supporting investors and small-medium enterprises (smes) to contend with political, legal, and financial challenges in malaysia (harrison, scheela, lai, & vivekarajah, 2018). our gamified mobile payment platform has proven to be a step forward towards providing empirical support to the conceptualization of blockchain-based cryptocurrency and mobile payment. xu and colleagues (2020) considered this combination as attractive in developing countries that have barriers to digital adoption in general. additionally, incorporating gamification as a design element in our platform has positively fostered user engagement for using cryptocurrency mobile payment services in malaysia. our study provides empirical evidence that encourage the usage of a gamified mobile payment platform in a developing country (putri et al., 2019) and mitigate some human-related concerns in the emerging decentralized blockchain ecosystem (parizi & dehghantanha, 2018). australasian journal of information systems lai & liew 2021, vol 25, research article towards a cashless society 17 in ensuring the multi-functional aspect of convenience, various gamification activities or reward programs by the participating stores can be designed into the app architecture for user perusal. users could earn loyalty points from a series of e-stamp collection, converting the points earned into monetary tokens of real currency to pay for transactions in a single gamified mobile payment platform. furthermore, users could also top up monetary tokens through online banking, internet payment (e.g., debit card, credit card, paypal), and mobile payment. in malaysia, boost introduced a rewarding cashless experience through various features like boostup ‘shake’ rewards, cashup cashback savings, qr payment services for users to earn ecoins or ‘shake’ for cashbacks and rewards (boost, 2019). the rapid advancement of affordable smartphones and 5g technology could accelerate the multi-functional, multichannel financial services delivery (kumar et al., 2020) in a single platform. in assuring the reliability aspect of security, features like the pin, picture, one-time password (otp), finger, or facial recognition can be designed onto the app architecture for user authentication. within the mobile payment platform ecosystem, various security standards and protocols are available to reduce the associated risks with blockchain technology and mobile payment (casino et al., 2019; lai, 2017). they safeguard authentication, encryption, integrity, and non-reputability within the blockchain ecosystem (casino et al., 2019) and have become more common across the mobile platform industry. in malaysia, local regulations such as the iso 27000 series of standards are formulated to assure information security. to unlock the full features of any mobile wallet platform, malaysian users must verify their identity via electronic know-your-customer (ekyc) verification using a registered malaysia mobile number, a selfie photo, and a matching national identity card or passport (bank negara malaysia, 2019). platform service providers or marketers should incorporate security compliance into the software architecture and design of a gamified mobile payment platform to meet the standard national security and data protection regulation even in the non-banking service industry. understanding users’ value perceptions over the platform’s design and security concerns is a step towards mitigating a common misperception that users would naturally switch to any new mobile payment option in the market (lai & scheela, 2018). moreover, businesses need to strategize engagement towards targeting the right type of users by understanding user interests under their perception of adopting and using a gamified mobile payment platform. 9 conclusion, limitations, and future directions this study contributes to the literature by explicitly examining the relationship between perceived convenience and perceived security on users’ intention to use a gamified mobile payment platform. perceived convenience had a strong but indirect positive effect on intention to use, while perceived security had a strong and direct positive effect on intention to use. perceived security mediates the relationship between perceived convenience and intention to use. the study also contextualizes the empirical model to malaysia, among developing countries that have witnessed a rapid proliferation of digital growth but are vulnerable to idiosyncratic risk and volatility underlying the different stages of technological advancement. two different results were found when examining the empirical model by the types of users. interested users and the general public were more concerned about the reliability aspect of security, whereas non-interested users were more concerned about the multi-functional aspect of convenience. the empirical results showed that the dual strategy of ‘ensuring convenience’ and ‘assuring security’ could encourage mobile payment platform adoption in malaysia. australasian journal of information systems lai & liew 2021, vol 25, research article towards a cashless society 18 therefore, businesses and platform service providers should take appropriate measures in designing the reliability and multi-functional aspects of a gamified mobile payment platform. this study is not without limitations. one major limitation was that the survey targeted users who were assumed to have internet or mobile data access with at least one-year self-reported experience using a gamified mobile payment platform. based on the sample collected from an urban region of malaysia, this study may not be generalized to other parts of the country. moreover, as users gain experience and move to more advanced mobile payment platforms, the empirical results based on cross-sectional data may no longer apply at a later point in time. future studies should consider including non-internet or non-mobile users in the sample using a probabilistic sampling method for better sample representation in malaysia. a longitudinal study would also be useful to track changes in user perception over time. acknowledgments special thanks to collaborators at blockchain japan, blockchain investors, globalclas technology, and monash global asia in the 21st century (ga21) multidisciplinary platform. both authors contributed equally to this work. references alhakami, a. s., & slovic, p. (1994). a psychological study of the inverse relationship between perceived risk and perceived benefit. risk analysis, 14(6), 1085–1096. https://doi.org/10.1111/j.1539-6924.1994.tb00080.x alhassan, a., li, l., reddy, k., & duppati, g. (2020). consumer acceptance and continuance of mobile money: secondary data insights from africa using the technology acceptance model. australasian journal of information systems, 24. https://doi.org/10.3127/ajis.v24i0.2579 alsawaier, r. (2018). the effect of gamification on motivation and engagement. international journal of information and learning technology, 35(1), 56–79. retrieved from https://doi.org/10.1108/ijilt-02-2017-0009 auer, r., cornelli, g., & frost, j. (2020). covid-19, cash, and the future of payment. retrieved from https://www.bis.org/publ/bisbull03.pdf bank negara malaysia. (2019). electronic know-your-customer (e-kyc): exposure draft. in know your customer. retrieved from https://knowyourcustomer.com/bank-negaramalaysia-issues-e-kyc-policy/ bank negara malaysia. (2020). payment statistics. retrieved from https://www.bnm.gov.my/index.php?ch=ps&pg=ps_stats&lang=en. berry, l. l., seiders, k., & grewal, d. (2002). understanding service convenience. journal of marketing, 66(3), 1–17. https://doi.org/10.1509/jmkg.66.3.1.18505 boksberger, p. e., & melsen, l. (2011, may). perceived value: a critical examination of definitions, concepts and measures for the service industry. journal of services marketing, vol. 25, pp. 229–240. https://doi.org/10.1108/08876041111129209 boost. (2019). boost introduces new features for a more rewarding cashless experience. retrieved from https://www.myboost.com.my/news/boost-introduces-new-featuresfor-a-more-rewarding-cashless-experience/ australasian journal of information systems lai & liew 2021, vol 25, research article towards a cashless society 19 burmeister, o. k. (2015). improving professional it doctorate completion rates. australasian journal of information systems, 19, 55–70. https://doi.org/10.3127/ajis.v19i0.1073 cao, c., & zhu, x. (2019). strong anonymous mobile payment against curious third-party provider. electronic commerce research, 19(3), 501–520. https://doi.org/10.1007/s10660018-9302-2 capgemini. (2019). world payments report 2019. retrieved from https://www.capgemini.com/news/world-payments-report-2019/ casino, f., dasaklis, t. k., & patsakis, c. (2019, march 1). a systematic literature review of blockchain-based applications: current status, classification and open issues. telematics and informatics, vol. 36, pp. 55–81. https://doi.org/10.1016/j.tele.2018.11.006 chang, y.-w., & polonsky, m. j. (2012). the influence of multiple types of service convenience on behavioral intentions: the mediating role of consumer satisfaction in a taiwanese leisure setting. international journal of hospitality management, 31, 107–118. https://doi.org/10.1016/j.ijhm.2011.05.003 chaudhry, s. a., farash, m. s., naqvi, h., & sher, m. (2016). a secure and efficient authenticated encryption for electronic payment systems using elliptic curve cryptography. electronic commerce research, 16(1), 113–139. https://doi.org/10.1007/s10660-015-9192-5 chauhan, s., mukhopadhyay, s., & jaiswal, m. (2018). the adoption of mobile app for b2c transaction in platform marketplace: an empirical examination of key drivers. journal of information technology case and application research, 20(1), 9–22. https://doi.org/10.1080/15228053.2018.1453338 chen, j., zhu, z., & xie, h. y. (2004). measuring intellectual capital: a new model and empirical study. journal of intellectual capital, 5(1), 195–212. https://doi.org/10.1108/ 14691930410513003 chen, x., & li, s. (2017). understanding continuance intention of mobile payment services: an empirical study. journal of computer information systems, 57(4), 287–298. https://doi.org/10.1080/08874417.2016.1180649 chew, j. (2019). which e-wallet app do malaysians use frequently? retrieved april 13, 2020, from iprice malaysia website: https://iprice.my/trends/insights/best-ewallet-malaysia/ chutikulrungsee, t. t. (2016). post publication review on security and privacy concerns for australian smes cloud adoption: empirical study of metropolitan vs regional smes. australasian journal of information systems, 20. https://doi.org/10.3127/ajis.v20i0.1369 colby, c., & parasuraman, a. p. (2003). technology still matters. marketing management, 12(4), 28–33. retrieved from https://www.researchgate.net/publication/ 279697463_technology_still_matters colwell, s. r., aung, m., kanetkar, v., & holden, a. l. (2008). toward a measure of service convenience: multiple-item scale development and empirical test. journal of services marketing, 22(2), 160–169. https://doi.org/10.1108/08876040810862895 copeland, m. (1923). relation of consumers’ buying habits to marketing methods. harvard business review1, 1(april), 282–289 australasian journal of information systems lai & liew 2021, vol 25, research article towards a cashless society 20 davis, f. d. (1989). perceived usefulness, perceived ease of use, and user acceptance of information technology. mis quarterly, 13(3), 319–339. https://doi.org/10.2307/249008 davis, fred d., bagozzi, r. p., & warshaw, p. r. (1989). user acceptance of computer technology: a comparison of two theoretical models. management science, 35(8), 982–1003. https://doi.org/10.1287/mnsc.35.8.982 de kerviler, g., demoulin, n. t. m., & zidda, p. (2016). adoption of in-store mobile payment: are perceived risk and convenience the only drivers? journal of retailing and consumer services, 31, 334–344. https://doi.org/10.1016/j.jretconser.2016.04.011 economic planning unit. (2015). eleventh malaysia plan, 2016-2020. retrieved from https://www.mea.gov.my/en/rmk/eleventh-malaysia-plan-2016-2020 farquhar, j. d., & rowley, j. (2009). convenience: a services perspective. marketing theory, 9(4), 425–438. https://doi.org/10.1177/1470593109346894 fong, v. (2020, june 8). malaysia to spend rm 1.2 billion promoting e-wallets in 2020. fintech malaysia. retrieved from https://fintechnews.my/23971/e-wallets-malaysia/e-walletmalaysia-pejana/ gensler, s., verhoef, p. c., & böhm, m. (2012). understanding consumers’ multichannel choices across the different stages of the buying process. marketing letters, 23(4), 987– 1003. https://doi.org/10.1007/s11002-012-9199-9 gupta, m. p., & dubey, a. (2016). e-commercestudy of privacy, trust and security from consumer’ s perspective. computer science and mobile computing hair, j.f., hult, g. t. m., ringle, c. m., & sarstedt, m. (2014). a primer on partial least squares structural equation modeling (pls-sem). thousand oaks, ca: sage publication hair, j. f., risher, j. j., sarstedt, m., & ringle, c. m. (2019). when to use and how to report the results of pls-sem. european business review, 30(1), 2–24. https://doi.org/10.1108/ebr11-2018-0203 hair, j. f., sarstedt, m., ringle, c. m., & gudergan, s. p. (2018). advanced issues in partial least squares structural equation modeling (pls-sem). thousand oaks, ca: sage publication harrison, d. e., hair, j. f., & ajjan, h. (2021). essentials of marketing analytics (1st ed.). mcgraw-hill education harrison, r., scheela, w., lai, p. c., & vivekarajah, s. (2018). beyond institutional voids and the middle-income trap: the emerging business angel market in malaysia. asia pacific journal of management, 35(4), 965–991. https://doi.org/10.1007/s10490-017-9535-y henseler, j., dijkstra, t. k., sarstedt, m., ringle, c. m., diamantopoulos, a., straub, d. w., … calantone, r. j. (2014). common beliefs and reality about partial least squares: comments on rönkkö & evermann (2013). organizational research methods, 17(2), 182–209. https://doi.org/https://doi.org/10.1177/1094428114526928 henseler, j., ringle, c. m., & sarstedt, m. (2015). a new criterion for assessing discriminant validity in variance-based structural equation modeling. journal of academic of marketing science, 43(1), 115–135. https://doi.org/10.1007/s11747-014-0403-8 australasian journal of information systems lai & liew 2021, vol 25, research article towards a cashless society 21 henseler, j., ringle, c. m., & sarstedt, m. (2015). testing measurement invariance of composites using partial least squares. international marketing review, 33(3), 405–431. https://doi.org/10.1108/imr-09-2014-0304 hofacker, c. f., de ruyter, k., lurie, n. h., manchanda, p., & donaldson, j. (2016). gamification and mobile marketing effectiveness. journal of interactive marketing, 34, 25–36. https://doi.org/10.1016/j.intmar.2016.03.001 högberg, j., shams, p., & wästlund, e. (2019). gamified in-store mobile marketing: the mixed effect of gamified point-of-purchase advertising. journal of retailing and consumer services, 50, 298–304. https://doi.org/10.1016/j.jretconser.2018.07.004 hollebeek, l. d., das, k., & shukla, y. (2021). game on! how gamified loyalty programs boost customer engagement value. international journal of information management, 102308. https://doi.org/10.1016/j.ijinfomgt.2021.102308 huang, r. (2020, march 9). who encourages use of contactless payments due to covid-19. forbes. retrieved from https://www.forbes.com/sites/rogerhuang/2020/03/09/whoencourages-use-of-digital-payments-due-to-covid-19/?sh=59ac5feb41eb huotari, k., & hamari, j. (2017). a definition for gamification: anchoring gamification in the service marketing literature. electronic markets, 27(1), 21–31. https://doi.org/10.1007/s12525-015-0212-z jiang, l. (alice), yang, z., & jun, m. (2013). measuring consumer perceptions of online shopping convenience. journal of service management, 24(2), 191–214. https://doi.org/10.1108/09564231311323962 johnson, v. l., kiser, a., washington, r., & torres, r. (2018). limitations to the rapid adoption of m-payment services: understanding the impact of privacy risk on m-payment services. computers in human behavior. https://doi.org/10.1016/j.chb.2017.10.035 kaufmann, h. r. (2014). handbook of research on managing and influencing consumer behavior. in handbook of research on managing and influencing consumer behavior. https://doi.org/10.4018/978-1-4666-6547-7 kim, c., mirusmonov, m., & lee, i. (2010). an empirical examination of factors influencing the intention to use mobile payment. computers in human behavior, 26(3), 310–322. https://doi.org/10.1016/j.chb.2009.10.013 kim, h. w., chan, h. c., & gupta, s. (2007). value-based adoption of mobile internet: an empirical investigation. decision support systems, 43(1), 111–126. https://doi.org/10.1016/j.dss.2005.05.009 kumar, r., sachan, a., & kumar, r. (2020). impact of service delivery system process and moderating effect of perceived value in internet banking adoption. australasian journal of information systems, 24. https://doi.org/10.3127/ajis.v24i0.1923 lai, p. c. (2017). security as an extension to tam model: consumers’ intention to use a single platform e-payment. asia-pacific journal of management research and innovation, 13(3–4), 110–119. https://doi.org/10.1177/2319510x18776405 lai, p. c., & scheela, w. (2018). convergence of technology in the e-commerce world and venture capital landscape in south east asia. in global entrepreneurship and new australasian journal of information systems lai & liew 2021, vol 25, research article towards a cashless society 22 venture creation in the sharing economy (pp. 149–268). retrieved from https://www.igi-global.com/chapter/convergence-of-technology-in-the-e-commerceworld-and-venture-capital-landscape-in-south-east-asia/186365 lai, p. c., toh, e. b. h., & alkhrabsheh, a. a. (2019). empirical study of single platform epayment in south east asia. in r. c. ho (ed.), strategies and tools for managing connected consumers (pp. 252–278). https://doi.org/10.4018/978-1-5225-9697-4.ch015 lee, z. w., & khaw, d. p. t. (2018). transforming mobile phones into e-wallets in malaysia. retrieved from https://www.bnm.gov.my/files/publication/qb/2018/q2/p7.pdf. lemon, k. n., & verhoef, p. c. (2016). understanding customer experience throughout the customer journey. journal of marketing, 80(6), 69–96. https://doi.org/10.1509/jm.15.0420 liew, e. j. y., vaithilingam, s., & nair, m. (2014). facebook and socio-economic benefits in the developing world. behaviour & information technology, 33(4), 345–360. https://doi.org/10.1080/0144929x.2013.810775 lim, w. m. (2018). dialectic antidotes to critics of the technology acceptance model: conceptual, methodological, and replication treatments for behavioural modelling in technology-mediated environments. australasian journal of information systems, 22. https://doi.org/10.3127/ajis.v22i0.1651 linger, h., & hasan, h. (2020). making is relevant in a connected world. australasian journal of information systems, 24. https://doi.org/10.3127/ajis.v24i0.2509 liu, z., ben, s., & zhang, r. (2019). factors affecting consumers’ mobile payment behavior: a meta-analysis. electronic commerce research, 19(3), 575–601. https://doi.org/10.1007/s10660-019-09349-4 miao, m., & jayakar, k. (2016). mobile payments in japan, south korea and china: crossborder convergence or divergence of business models? telecommunications policy, 40(2–3), 182–196. https://doi.org/10.1016/j.telpol.2015.11.011 morosan, c., & defranco, a. (2016). it’s about time: revisiting utaut2 to examine consumers’ intentions to use nfc mobile payments in hotels. international journal of hospitality management, 53, 17–29. https://doi.org/10.1016/j.ijhm.2015.11.003 mukerji, m., & roy, p. s. (2019). platform interactions and emergence of an organizational field: case study on ola. australasian journal of information systems, 23. https://doi.org/10.3127/ajis.v23i0.2113 nakamoto, s. (2008). bitcoin: a peer-to-peer electronic cash system. retrieved november 20, 2020, from https://www.klausnordby.com/bitcoin/bitcoin_whitepaper_document_ hd.pdf. nayal, p., pandey, n., & paul, j. (2021). examining m-coupon redemption intention among consumers: a moderated moderated-mediation and conditional model. international journal of information management, 57, 102288-. https://doi.org/10.1016/ j.ijinfomgt.2020.102288 o’reilly, p., duane, a., & andreev, p. (2012). to m-pay or not to m-pay—realising the potential of smart phones: conceptual modeling and empirical validation. electronic markets, 22(4), 229–241. https://doi.org/10.1007/s12525-012-0105-3 australasian journal of information systems lai & liew 2021, vol 25, research article towards a cashless society 23 panhwer, p., pitafi, a., memon, m. s., & memon, a. (2020). awareness and reason towards slow adoption of e-payment system: study of hyderabad. annals of contemporary developments in management & hr, 2(1), 6–21. https://doi.org/10.33166/acdmhr.2020.01.002 parizi, r. m., & dehghantanha, a. (2018). on the understanding of gamification in blockchain systems. proceedings 2018 ieee 6th international conference on future internet of things and cloud workshops, w-ficloud 2018, 214–219. https://doi.org/10.1109/wficloud.2018.00041 poushter, j., bishop, c., & chew, h. (2018). social media use continues to rise in developing countries but plateaus across developed ones. retrieved from https://www.pewresearch.org/global/2018/06/19/social-media-use-continues-to-rise-indeveloping-countries-but-plateaus-across-developed-ones/. putri, m. f., hidayanto, a. n., negara, e. s., budi, n. f. a., utari, p., & abidin, z. (2019). gratification sought in gamification on mobile payment. icicos 2019 3rd international conference on informatics and computational sciences: accelerating informatics and computational research for smarter society in the era of industry 4.0, proceedings. https://doi.org/10.1109/icicos48119.2019.8982424 pwc. (2019). it’s time for a consumer-centred metric: introducing “return on experience”: global consumer insights survey. retrieved from https://www.pwc.com/gx/en/ consumer-markets/consumer-insights-survey/2019/ report.pdf ramadan, z. b., & farah, m. f. (2017). the pokémonisation of the first moment of truth. international journal of web based communities, 13(2), 262–277. https://doi.org/10.1504/ijwbc.2017.084417 ringle, c. m., & sarstedt, m. (2016). gain more insight from your pls-sem results the importance-performance map analysis. industrial management and data systems, vol. 116, pp. 1865–1886. https://doi.org/10.1108/imds-10-2015-0449 ringle, c. m., wende, s., & becker, j.-m. (2015, january 5). smartpls 3. retrieved from http://www.smartpls.com sánchez-fernández, r., & iniesta-bonillo, m. á. (2007). the concept of perceived value: a systematic review of the research. marketing theory, 7(4), 427–451. https://doi.org/10.1177/1470593107083165 schmida, s., bernard, j., zakaras, t., lovegrove, c., & swingle, c. (2017). connecting the next four billion: strengthening the global response for universal internet access. retrieved february 12, 2020, from usaid, dial digital impact alliance and ssg advisors website: www.usaid.gov/sites/default/files/documents/15396/connecting_the_next_four_billio n-20170221_final.pdf seiders, k., voss, g. b., godfrey, a. l., & grewal, d. (2007). servcon: development and validation of a multidimensional service convenience scale. journal of the academy of marketing science, 35(1), 144–156. https://doi.org/10.1007/s11747-006-0001-5 senarathna, i., yeoh, w., warren, m., & salzman, s. (2016). security and privacy concerns for australian smes cloud adoption: empirical study of metropolitan vs regional smes. australasian journal of information systems. https://doi.org/10.3127/ajis.v20i0.1193 australasian journal of information systems lai & liew 2021, vol 25, research article towards a cashless society 24 sharma, s. k., & gupta, j. n. d. (2009). identifying factors for lack of e-commerce in developing countries. in k. rouibah, o. khalil, & a. hassanien (eds.), emerging markets and e-commerce in developing economies (pp. 70–88). hershey, pa: information science reference. shin, d. h. (2009). towards an understanding of the consumer acceptance of mobile wallet. computers in human behavior, 25(6), 1343–1354. https://doi.org/10.1016/ j.chb.2009.06.001 shmueli, g., sarstedt, m., hair, j. f., cheah, j. h., ting, h., vaithilingam, s., & ringle, c. m. (2019). predictive model assessment in pls-sem: guidelines for using plspredict. european journal of marketing, 53(11), 2322–2347. https://doi.org/10.1108/ejm-02-20190189 ström, r., vendel, m., & bredican, j. (2014). mobile marketing: a literature review on its value for consumers and retailers. journal of retailing and consumer services, 21(6), 1001– 1012. https://doi.org/10.1016/j.jretconser.2013.12.003 tashiro, m. (2018). major japanese corporation driving cryptocurrency usage » brave new coin. retrieved february 24, 2020, from https://bravenewcoin.com/insights/majorjapanese-corporation-driving-cryptocurrency-usage taylor, s., & todd, p. (1995). understanding information technology usage: a test of competing models. information systems research, 6(2), 144–176. teo, a. c., tan, g. w. h., ooi, k. b., hew, t. s., & yew, k. t. (2015). the effects of convenience and speed in m-payment. industrial management and data systems, 115(2), 311–331. https://doi.org/10.1108/imds-08-2014-0231 the star. (2020, june 12). boost supports government’s e-wallet initiatives. retrieved from https://www.thestar.com.my/business/business-news/2020/06/12/boost-supportsgovernments-e-wallet-initiatives truong, v. n. x., nkhoma, m., & pansuwong, w. (2019). an integrated effectiveness framework of mobile in-app advertising. australasian journal of information systems, 23. https://doi.org/10.3127/ajis.v23i0.1971 venkatesh, v., & davis, f. d. (2000). a theoretical extension of the technology acceptance model: four longitudinal field studies. management science, 6, 144–176. venkatesh, v., morris, m. g., gordon, b. d., & davis, f. d. (2003). user acceptance of information technology: toward a unified view. mis quarterly, 27(3), 425–478. retrieved from http://www.jstor.org/stable/30036540 vishwakarma, p., mukherjee, s., & datta, b. (2020). travelers’ intention to adopt virtual reality: a consumer value perspective. journal of destination marketing and management, 17. https://doi.org/10.1016/j.jdmm.2020.100456 weir, c. s., douglas, g., carruthers, m., & jack, m. (2009). user perceptions of security, convenience and usability for ebanking authentication tokens. computers and security, 28(1–2), 47–62. https://doi.org/10.1016/j.cose.2008.09.008 australasian journal of information systems lai & liew 2021, vol 25, research article towards a cashless society 25 weissman, a. (2012). convenience: the third essential of a customer-centric business. retrieved february 24, 2020, from ux magazine website: https://uxmag.com/articles/convenience wu, j., liu, l., & huang, l. (2017). consumer acceptance of mobile payment across time antecedents and moderating role of diffusion stages. industrial management and data systems, 117(8), 1761–1776. https://doi.org/10.1108/imds-08-2016-0312 xi, n., & hamari, j. (2020). does gamification affect brand engagement and equity? a study in online brand communities. journal of business research, 109, 449–460. https://doi.org/10.1016/j.jbusres.2019.11.058 xin, h., techatassanasoontorn, a. a., & tan, f. b. (2015, june 1). antecedents of consumer trust in mobile payment adoption. journal of computer information systems, vol. 55, pp. 1–10. https://doi.org/10.1080/08874417.2015.11645781 xu, l., chen, l., gao, z., carranco, l., fan, x., shah, n., … shi, w. (2020). supporting blockchain-based cryptocurrency mobile payment with smart devices. ieee consumer electronics magazine, 9(2), 26–33. https://doi.org/10.1109/mce.2019.2953734 yang, y., asaad, y., & dwivedi, y. (2017). examining the impact of gamification on intention of engagement and brand attitude in the marketing context. computers in human behavior, 73, 459–469. https://doi.org/10.1016/j.chb.2017.03.066 yuniarta, g. a., & purnamawati, g. a. (2021). psychological dimensions and practical strategies: msme and mobile payment adoption. management science letters, 11, 577– 586. https://doi.org/10.5267/j.msl.2020.9.009 zelezny-green, r., voslon, s., & conole, g. (2018). digital inclusion for low-skilled and lowliterate people: a landscape review unesco digital library. retrieved from https://unesdoc.unesco.org/ark:/48223/pf0000261791 zhang, j., xu, n., & bai, s. (2020). the optimal pricing decisions for e-tailers with different payment schemes. electronic commerce research, 1–28. https://doi.org/10.1007/s10660020-09396-2 copyright: © 2021 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v25i0.2809 http://creativecommons.org/licenses/by-nc/3.0/au/ 1  introduction 2 mobile payment development in malaysia 3 software architecture and design of a gamified mobile payment platform 4 literature review and hypotheses development 4.1 convenience 4.2 security 4.3 convenience to security 5 methodology 5.1 data 5.2 sample statistics 5.3 statistical analysis 6 empirical results 6.1 measurement model 6.2 structural model 7 theoretical contributions 8 practical implications 9 conclusion, limitations, and future directions paper title (paper title style) australasian journal of information systems hooda et al. 2023, vol 27, research article clarifying the role of e-government trust 1 clarifying the role of e-government trust in egovernment success models: a meta-analytic structural equation modeling approach apeksha hooda o.p. jindal global university, sonipat, haryana, india parul gupta management development institute gurgaon, india anand jeyaraj wright state university, oh, usa anand.jeyaraj@wright.edu yogesh dwivedi swansea university, wales, uk & symbiosis institute of business management, pune & symbiosis international (deemed university), pune, maharashtra, india abstract e-government implementation success is of critical importance for nations. prior information systems (is) success models emphasize the effects of information quality, service quality, system quality, and user satisfaction but do not consider e-government trust. this study incorporates e-government trust into the is success model and empirically tests the model on empirical findings reported in 67 prior studies using meta-analysis methods and structural equation modeling. our analysis shows that: a) information quality, service quality, system quality, and user satisfaction influence e-government trust, and b) system use mediates the effect of e-government trust on intention to use e-government systems in the future. keywords: information system success, e-government trust, meta-analysis, sem, metaanalytic structural equation modeling, masem. 1 introduction advances in information and communication technology (ict) and the global penetration of the internet have significantly contributed to the proliferation of e-government systems in recent decades. the success of e-government systems is estimated to be around 15% in developing countries (anthopoulos et al., 2016) despite the recognition that they may provide transparency, accountability, participation, and collaboration in public services (maclean & titah, 2022). e-government success refers to the satisfactory e-government system adoption and use by its beneficiaries (teo et al., 2008). prior research has proposed various models and factors that impact system use (su) or behavioral intention (bi) to use e-government systems. for instance, the technology acceptance model (tam) proposed two technology attributes—perceived usefulness and perceived ease of use—to impact su and bi (davis, 1989) whereas the unified theory of acceptance and use of technology (utaut) posited performance expectancy, effort expectancy, social influence, and facilitating conditions to impact su or bi (venkatesh et al., 2003). while these models offer useful insights, they do not address specific challenges in egovernment settings such as the quality of service or users’ satisfaction with systems. the australasian journal of information systems hooda et al. 2023, vol 27, research article clarifying the role of e-government trust 2 information systems success (iss) model proposed service quality, information quality, system quality, and user satisfaction as influential factors in su or bi (delone & mclean, 2003). however, the risks and uncertainties experienced by e-government users are not theorized in the iss model. since e-government systems involve online transactions and access to personal information, users face risks and uncertainties in adopting and using such systems. the quality attributes of the technology (delone & mclean, 2003) or users’ perceptions of technology usefulness (venkatesh et al., 2003) are not sufficient to deal with risks and uncertainties. rather, psychological factors such as trust are crucial to foster the initial adoption and subsequent long-term use of e-government systems (alkraiji, 2020). e-government maturity models have identified various stages in deploying e-government systems (layne & lee, 2001) and trust plays a significant role at every stage (bannister & connolly 2011). trust has received research attention in e-government settings, but has not been explicitly included in iss models (rana et al., 2013; teo et al., 2008) although prior studies have considered trust and quality together. this study has two related objectives. first, we develop an integrated research model incorporating e-government trust into the iss model and theorize paths involving trust and iss constructs such as service quality, system quality, information quality, and user satisfaction on su and bi. the integrated model offers a richer perspective since it simultaneously considers quality and trust factors in e-government su. second, we use metaanalytic structural equation modeling (masem) methods to empirically test the integrated research model (jeyaraj & dwivedi, 2020). masem methods enable the synthesis of effect sizes for bivariate relationships reported in prior studies, obtain the matrix of correlations among variables, and apply sem methods to examine the interrelationships among constructs. in doing so, this study reconciles inconsistent findings in prior studies for the relationships examined in our model. for instance, trust had mixed effects on bi (al-omairi et al., 2020; alsulami & hashim, 2018); information quality had mixed effects on bi (mellouli et al., 2020; stefanovic et al., 2016); and system quality had mixed effects on bi (sharma & mishra, 2017; mensah, 2019). thus, our study offers both theoretical and empirical contributions in examining e-government systems. 2 theoretical background 2.1 e-government systems an e-government system is an ict-based solution used by governments to deliver public services to its beneficiaries. it may manifest as different types of digital technologies including web sites and e-service portals that enable users to interact with governments (undesa, 2004). public services include information provision in response to citizens’ questions (e.g., age of retirement) or transaction processing to cater to specific needs (e.g., an application for building permit) through e-government systems (nielsen, 2021). both types of services may require users to provide personal or sensitive data through the e-government system, which is a significant reason for consideration of e-government trust. 2.2 technology adoption and use prior research on the adoption and use of e-government systems has yielded useful insights. models of is acceptance (davis, 1989; venkatesh et al., 2003) have been used in examining egovernment adoption and use. these models found performance expectancy, effort australasian journal of information systems hooda et al. 2023, vol 27, research article clarifying the role of e-government trust 3 expectancy, social influence, and facilitating conditions to impact e-government su (alryalat et al., 2015). models based on expectation confirmation theory (ect) and social cognitive theory (sct) found satisfaction, outcome expectations, prior experience, and social influence to impact users’ bi to use e-government (alruwaie et al., 2020). such models typically do not consider quality or trust as influential in technology adoption and use. 2.3 is success and trust the iss model emphasizes different dimensions of quality in technology use. three dimensions—service quality, information quality, and system quality—have been proposed to influence user satisfaction and su (delone & mclean, 2003). the model has been empirically tested in various studies in the e-government domain (wang & liao, 2008). service quality represents the superiority of service provided by a system; information quality refers to the correctness, usefulness, and completeness of the system output; and system quality represents the technical quality of the system such as availability, reliability, and access (wang & liao, 2008; wang et al., 2010), all of which exert an influence on user satisfaction. further, the effects of the three quality factors and user satisfaction on e-government su have received considerable empirical support (wang & liao, 2008). the iss models thus underline the importance of the various quality attributes unlike the technology adoption and use models. similar to technology adoption and use models, the iss models do not portray the role of trust in su. however, trust can be a significant consideration in e-government settings due to the impersonal nature of electronic interactions with governmental systems and the need to divulge private and sensitive data (carter & bélanger, 2005; undesa, 2018; santa et al., 2019; j. zhang et al., 2021). when a system is perceived to be safe and secure, which may be considered as quality attributes, trust may be enhanced, which may result in su (teo et al., 2008; singh et al., 2020). thus, trust may be closely linked with the iss models than technology adoption models since the tam constructs may not necessarily increase trust since they do not provide ways for individuals to overcome negative thoughts about safety and security. since trust needs to be addressed to enhance e-government maturity (nielsen, 2016) and trust needs to be developed rather than be automatically transferred from other settings (luhmann, 1988), e-government systems could be designed to increase users’ trust by improving the three quality dimensions and adhering to security and privacy expectations (teo et al., 2008). 3 research model figure 1 shows the integrated research model with e-government trust and the iss model examined in this study. table 1 shows the definitions of the model constructs. construct definition reference system quality e-government systems’ efficiency, availability, and reliability teo et al. (2008) information quality completeness, usefulness, and correctness of information provided by the e-government system wang et al. (2010) service quality e-government systems’ quality of services and support in answering queries, entertaining requests, and offering solutions kumar et al. (2007); stefanovic et al. (2016) user satisfaction users’ feelings about the e-government system after using it konradt et al. (2016); sachan et al. (2018) australasian journal of information systems hooda et al. 2023, vol 27, research article clarifying the role of e-government trust 4 e-government trust users’ confidence in the e-government system to provide reliable and efficient service without technical errors and opportunistic behaviors of service providers almaiah & nasereddin (2020); warketin et al. (2002) system use users’ actual utilization of the e-government system to avail public services and complete transactions singh et al. (2020); nurdin et al. (2010) behavioral intention users’ willingness to utilize e-government systems in the future fishbein & ajzen (1980); zhao & khan (2013) table 1. construct definitions figure 1. initial research model 3.1 e-government trust e-government trust represents the users’ expectations that e-government systems provide authentic, correct, and timely information in a secured online environment without technical errors and opportunistic behaviors of service providers (papadopoulou et al., 2010; warkentin et al., 2002). it includes beliefs that governments are equipped with the necessary technical and managerial resources to provide services in a secure online environment (warkentin et al., 2002). actions involving investment of trust are based on trustors’ (e-government users) estimate of trustees’ (e-government) credibility (sztompka, 1998). it encompasses beliefs about the government’s goodwill, information privacy, data authenticity, service quality, and transaction security (papadopoulou et al.,2010; piehler et al., 2016). e-government trust may be impacted by the systems’ quality attributes. system quality represents users’ perceptions regarding the efficiency, availability, and reliability of the e-government system and the accuracy of the communication (teo et al., 2008). the technical capability of e-government systems depicted by speed and reliability plays a crucial role in winning users’ confidence by improving their trust with e-government systems. for instance, users are likely to have greater levels of trust in e-government systems if they can rely on the availability of the system whenever they need to use them to obtain information or complete transactions and they know that the systems will not fail them. thus, system quality is of considerable importance to e-government trust. australasian journal of information systems hooda et al. 2023, vol 27, research article clarifying the role of e-government trust 5 h1: system quality is positively related to trust in e-government systems. when users perceive that e-government systems provide information that can suitably and appropriately meet their requirements, they are likely to demonstrate higher level of trust towards both the medium of service (internet) and the provider (government) (mun et al., 2013). however, poor information quality may obstruct e-government success since inaccurate, incomplete, and ambiguous information provided by e-government systems may shake users’ trust adversely affecting e-government initiatives (lee & levy, 2014). information quality thus remains crucial to e-government success as it is a strong determinant of user’s trust. h2: information quality is positively related to trust in e-government systems. service quality is the users’ evaluation of received services towards their service expectations from the system (wang et al., 2010). service quality is assessed as the system’s capability to provide desired services that help users in efficiently completing the governmental transactions. this quality dimension covers the speed at which users’ queries receive responses from the e-government system (kumar et al., 2007). by appropriately meeting their expectations, the quality of e-government systems’ service can strengthen users’ beliefs about the service provider (government) competence and the medium (internet). since face-to-face interaction with public service providers are not always possible in e-government contexts, service quality plays a crucial role in shaping users’ trust. h3: service quality is positively related to trust in e-government systems. 3.2 user satisfaction anwer et al. (2016) contend that user-centric service options, process performance, and security features of e-government systems significantly influence users’ satisfaction and overall success of e-government systems. users have reported dissatisfaction despite the benefits offered by e-government systems (kumar et al., 2018; united nations, 2016) and only small proportions of users in countries such as brazil, mexico, and uruguay use e-government systems (fairas et al., 2017; roseth et al., 2017; nielsen, 2019). poorly designed interfaces, inconvenient access, inappropriate information, slow responsiveness, untrustworthiness of technology, system malfunction (united nations, 2016; gupta & suri, 2017) and usability issues such as sub-optimal presentation structure (nielsen, 1999) are cited as reasons for the low levels of adoption and satisfaction among users. poor system quality of e-government systems reflected in incompatible technical standards, poor system design, inflexibility and poor integration of e-government systems and databases generate user dissatisfaction (angelopoulos et al., 2010; savoldelli et al., 2014). when systems are deemed to be convenient, reliable, and user-friendly, users may become more satisfied with the general efficiency and effectiveness of e-government systems (sachan et al., 2018). prior studies provide empirical support for the effect of system quality on user satisfaction (weerakkody et al., 2016; veeramootoo et al., 2018). h4: system quality is positively related to user satisfaction. information quality, perceived in terms of accessibility, usability, accuracy, and understandability, is a significant contributing factor of users’ satisfaction (al-rawahna et al., 2018). these features determine the extent to which users find the system helpful. hence, if egovernment systems fail to provide the desired information quality, users experience australasian journal of information systems hooda et al. 2023, vol 27, research article clarifying the role of e-government trust 6 dissatisfaction and frustration (al-rawahna et al., 2018). prior e-government studies contributed empirical evidence to support a positive significant impact of information quality on user satisfaction (weerakkody et al., 2016; wang & liao, 2008). h5: information quality is positively related to user satisfaction. service quality of e-government systems represented by reliable, responsive, and empathetic service, increases user satisfaction levels (al-rawahna et al., 2018). e-government systems which are responsive to users’ needs by providing the necessary help for them to use systems elevate user satisfaction. prior e-government studies provide empirical support for the effect of service quality on user satisfaction (wang & liao, 2008; stefanovic et al., 2016). h6: service quality is positively related to user satisfaction. 3.3 e-government system use trust in e-government reduces the perceived risk and uncertainty associated with online interactions and transactions, which encourages users to use e-government systems. trust is crucial in assuring users that the government will not engage in opportunistic behaviors by using its venerable position in governmental transactions to disadvantage the users (fukuyama, 1995; lewis & weigert, 1985). thus, trust can minimize the negative impacts of perceived risks and uncertainties, which can impact the use of e-government systems to a greater extent. h7: trust in e-government systems is positively related to system use. user satisfaction is a strong indicator of how well e-government systems have performed in accordance with users’ expectations and needs. a significant obstacle to e-government success is user dissatisfaction with e-government systems (nawafleh, 2018; sachan et al., 2018). this implies that the challenge of guaranteeing repeated use and consequent success of egovernment systems may depend on satisfied users (lai & pires, 2010). a satisfied user may choose e-government systems over traditional methods of interacting with governments. prior studies have reported significant effects for user satisfaction on e-government su (rana et al., 2015). h8: user satisfaction is positively related to system use. 3.4 behavioral intention bi explains the degree to which users formulate conscious plans to demonstrate specified behaviors to engage in or not engage in a particular activity (fishbein & ajzen, 1980; warshaw & davis, 1985). bi is viewed as the probability that users will choose to utilize e-government systems for their future needs. bi is largely driven by an individual’s internal evaluation results based on prior su (ajzen & fishbein, 1980) and has considerable empirical support in prior is literature (law et al., 2016; van slyke et al., 2007). h9: system use is positively related to behavioral intention. 4 research methods 4.1 sample to empirically test our research model, we gathered studies published between 2000 and 2021 using multiple online databases including scopus, google scholar, and digital government reference library (dgrl). keywords for the search included “electronic government,” “eaustralasian journal of information systems hooda et al. 2023, vol 27, research article clarifying the role of e-government trust 7 government,” “digital government,” “internet government,” “mobile government,” and “online government” along with phrases such as “success,” “adoption,” “usage,” “satisfaction,” “trust,” “quality,” and “intention” (and its variants such as “behavioral intention” and “use”). the search resulted in more than 500 articles. duplicate articles and others such as teaching notes, editorials, qualitative studies, and reviews were excluded first, followed by articles that did not examine e-government success, adoption, usage, or intention (kumar et al., 2018; rana et al., 2015; valle-cruz, 2019). finally, studies that did not examine a relationship in our integrated research model or report correlations required for meta-analysis were excluded. our final meta-analysis sample included 67 studies1. 4.2 coding data were uniformly coded from the identified studies. for each study, we coded author names, journal name, and year of publication. to facilitate meta-analysis, we also coded data for the constructs and relationships in our model. the mean, standard deviation (sd), reliability, and likert scale anchors were coded for the constructs while the pearson correlation and sample size were coded for the relationships. the coded data was screened to assess consistency with analysis requirements. first, differently worded constructs across studies were combined as a single construct. for instance, delivery system quality, perceived service quality, online service quality, and quality of service were coded as service quality (idoughi & abdelhakim, 2018; mensah, 2019; wang et al., 2020; santa et al., 2019) while perceived system quality, web site quality, and quality of system were coded as system quality (ibrahim & zakaria, 2016; santa et al., 2019; stefanovic et al., 2016). this categorization was based on the definitions and/or measurement scales used in prior studies. for instance, delivery system quality was defined as the users’ perception of technical performance of the system (idoughi & abdelhakim, 2018) while initial trust (azam et al., 2013) was measured using the same scale as e-government trust. table 2 shows details of the categorization effort. second, the coded data was assessed to determine if the observations were independent, i.e., only one finding was allowed from the same study for a bivariate relationship. if multiple findings could be coded for any relationship from the same study, we computed the mean of the correlations (azam et al., 2013; mensah, 2019; teo et al., 2008). the exception was when the same study reported results of multiple samples (e.g. gonzalez et al., 2010) . finally, reliabilities of constructs were not reported in all studies and hence the average reliability was substituted for missing values (gilbert et al., 2004). overall, we coded 226 findings from the studies. 1 the studies were published in: 2004 (1), 2007 (2), 2008 (2), 2009 (1), 2010 (1), 2011 (2), 2012 (2), 2013 (2), 2015 (2), 2016 (5), 2017 (9), 2018 (3), 2019 (13), 2020 (19), and 2021 (3). four studies each were published in government information quarterly and international journal of electronic government research, three studies each in journal of theoretical and applied information technology and international journal of information management, and, two studies each in information & management, transforming government: people, process and policy, electronic government, international journal of business information systems, international journal of interactive mobile technologies, computers in human behavior, and sage open, and one each in 37 other journals. complete citations for the studies available from the authors upon request. australasian journal of information systems hooda et al. 2023, vol 27, research article clarifying the role of e-government trust 8 research model construct construct in prior study reference system quality perceived system quality (ibrahim & zakaria, 2016) website quality (santa et al., 2019) quality of system (mensah, 2019; mensah & mi, 2017) information quality information quality (alruwaie et al., 2020) perceived information quality (alkraiji, 2021) service quality online service quality (wang & teo, 2020) quality of service (santa et al., 2019; alqaralleh et al., 2020) delivery system quality (idoughi & abdelhakim, 2018) perceived service quality (mensah & mi, 2017) user satisfaction citizen satisfaction (obaid & ahmad, 2021) user satisfaction (mellouli et al., 2020; al-zahrani, 2020) satisfaction (piehler et al., 2016) public satisfaction (hariguna, lai, hung, chen, et al., 2017) e-government trust perceived trust (eid et al., 2020) trust (kamarudin et al., 2021) initial trust (azam et al., 2013) e-governance trust (mansoori et al., 2018) system use actual use (albar & hddas, 2018) eservice utilization (hossan & ryan, 2016) usability (noor et al., 2011) usage behavior (weerakkody et al., 2013) use behavior of e-government services (ibrahim & zakaria, 2016) e-government system usage (alshaher, 2021) behavioral intention adoption (shuib et al., 2019) behavioral intention (rana & dwivedi, 2015) intention to adopt (alharbi et al., 2017) intention to use (shahzad et al., 2020) use of e-government (qasim nidawy et al., 2020) public intention use (hariguna, lai, hung, & chen, 2017) willingness to use (mensah & mi, 2018) table 2. coding and classifying constructs in prior studies table 2 shows details of the categorization effort. second, the coded data was assessed to determine if the observations were independent, i.e., only one finding was allowed from the same study for a bivariate relationship. if multiple findings could be coded for any relationship from the same study, we computed the mean of the correlations (azam et al., 2013; mensah, 2019; teo et al., 2008). the exception was when the same study reported results of multiple samples (e.g. gonzalez et al., 2010) . finally, reliabilities of constructs were not reported in all studies and hence the average reliability was substituted for missing values (gilbert et al., 2004). overall, we coded 226 findings from the studies. 4.3 analysis we employed hunter & schmidt (2004) meta-analysis methods to compute the corrected correlation for each relationship in our model. measurement errors were corrected by factoring in the reliabilities of the constructs. specifically, the observed correlation was divided by the square root of the product of the reliabilities of the two constructs in the relationship. sampling errors were corrected by computing the sum of the product of the sample size and observed correlation and dividing by the sum of the sample sizes across studies. table 3 shows the corrected correlations, the cumulative sample size, and the number of observations for each australasian journal of information systems hooda et al. 2023, vol 27, research article clarifying the role of e-government trust 9 relationship. further, the table shows the mean, standard deviation, and reliability for each construct. we used the linear transformation x7 = (x5 -1)*(6/4) + 1, where x7 and x5 represent the 7-point and 5-point likert scale values respectively. further, the failsafe-n and credibility interval for each relationship are also shown on table 3. since the credibility intervals do not include 0, the effects in our model are positive (whitener 1990). failsafe-n, i.e., the number of non-significant findings needed to invalidate the results (sabherwal et al., 2006), ranges from 25 (i.e., information quality to su relationship) to 336 (i.e., e-government trust to bi relationship) with the average of 111 across the 21 relationships. we conclude that publication bias is not a major problem. the masem analysis was done in amos 21 using the corrected correlations and the minimum sample size (435). construct mean (sd) cr bi iq rq sq tr us su behavioral intention (bi) 5.361 (1.26) 0.85 [0.309, 0.583] 3200 [0.033, 0.895] 11464 [0.395, 0.756] 7213 [0.269, 0.879] 23145 [0.157, 1.04] 9980 [0.496, 0.812] 8456 information quality (iq) 5.33 (1.62) 0.85 0.440 (9, 2960) [0.040, 1.11] 8607 [0.297, 0.929] 21328 [0.343, 0.866] 9975 [0.279, 0.846] 17148 [0.249, 0.682] 1461 service quality (rq) 5.70 (1.90) 0.82 0.473 (15, 6361) 0.534 (12, 3360) [0.238, 0.745] 4449 [0.286, 0.874] 5827 [0.141, 1.09] 8610 [0.093, 0.766] 4234 system quality (sq) 5.40 (1.63) 0.85 0.595 (8, 2609) 0.597 (20, 9268) 0.503 (9, 2404) [0.296, 0.869] 7178 [0.218, 0.913] 15750 [0.044, 0.805] 1480 e-government trust (tr) 5.11 (1.54) 0.88 0.569 (29, 10435) 0.625 (10, 5239) 0.600 (8, 2879) 0.593 (9, 4432) [0.211, 0.998] 8594 [0.528, 0.763] 4228 user satisfaction (us) 5.289 (1.63) 0.87 0.623 (10, 3851) 0.547 (18, 8816) 0.615 (12, 3486) 0.557 (17, 7910) 0.622 (10, 4590) [0.282, 0.888] 2835 system use (su) 5.29 (1.13) 0.89 0.675 (6, 3546) 0.471 (3, 485) 0.428 (4, 2597) 0.510 (3, 485) 0.672 (3, 1734) 0.616 (3, 485) note. cr: construct reliability; sd: standard deviation correlation, number of observations, and sample size in the lower triangle failsafe-n and low and high thresholds of 90% credibility interval in the upper triangle table 3. results of meta-analysis 5 results the research model showed reasonable fit: χ2 (df) = 159.27 (9), p < 0.01, tli = 0.808, cfi = 0.918, rmsea = 0.094, and srmr = 0.186. all paths hypothesized in the research model were supported. the χ2 / df ratio for the model was above 3 and not acceptable (sabherwal et al., 2006). while cfi was acceptable (> 0.90), tli was lower than the recommended 0.90 level (bentler & bonett, 1980). rmsea and srmr were above the recommended 0.08 (sabherwal et australasian journal of information systems hooda et al. 2023, vol 27, research article clarifying the role of e-government trust 10 al., 2006; browne & cudeck, 1993). the results showed modification indices (mi > 10) for inclusion of other paths that may yield better fit. we first added the path from system quality to bi (mi = 56.12). the resultant model showed better fit than the research model: χ2 (df) = 77.277 (8), p < 0.01, tli = 0.961, cfi = 0.963, rmsea = 0.134, and srmr = 0.049. the χ2/df ratio did not meet the recommended threshold. both cfi and tli were acceptable. srmr was acceptable but rmsea did not meet the recommended threshold. results showed adding other paths could improve fit. the path from user satisfaction to bi (mi = 16.35) was included next. the resulting model showed better fit: χ2 (df) = 46.085 (7), p > 0.05, tli = 0.938, cfi = 0.979, rmsea = 0.107, and srmr = 0.036. the χ2/df ratio did not meet recommended thresholds although cfi and tli were acceptable. rmsea was above recommended while srmr was acceptable. results showed other paths could improve model fit. we added the path between user satisfaction and e-government trust (mi = 15.38). the resultant model was a better fit with the data: χ2 (df) = 15.489 (6), p < 0.01, tli = 0.982, cfi = 0.995, rmsea = 0.057; srmr = 0.018. the χ2/df ratio and the other fit indices were acceptable. the emergent model is shown in figure 2. notes. figure 2. final emergent model 6 discussion 6.1 findings an integrated model of iss success and e-government trust along with su and bi was empirically examined in this research. the masem analysis showed significant results for all the hypothesized paths, indicating strong support for the integrated model. in addition, a few paths emerged in the analysis, providing new insights into the iss relationships. r2 for trust = 0.557 r2 for satisfaction = 0.481 r2 for use = 0.516 r2 for intention = 0.565 χ2 = 15.538, df = 6, p < 0.01 cfi = 0.995, tli = 0.982 rmsea = 0.057, srmr = 0.018 hypothesized path emergent path australasian journal of information systems hooda et al. 2023, vol 27, research article clarifying the role of e-government trust 11 all three technology quality attributes had significant effects on e-government trust as hypothesized. system quality (h1, β = 0.26***), information quality (h2, β = 0.21***), and service quality (h3, β = 0.16***) had positive impact on e-government trust. in addition, user satisfaction (emergent, β = 0.21***) influenced e-government trust. collectively, the three quality attributes and user satisfaction explained 55.7% of variance in e-government trust. these results are consistent with prior findings (e.g., santa et al., 2019) that found the accuracy of communication facilitated by high quality design of e-government system to be instrumental in winning users’ e-government trust. users place greater trust in systems that are efficient and effective in providing the desired information in a timely manner and enable users to easily handle their needs. this emphasizes a customer-oriented approach with a focus on customizing the e-government systems’ attributes for users’ needs and priorities (kelly & vidgen, 2005). the three technology quality attributes also had significant effects on user satisfaction as hypothesized. system quality (h4, β = 0.20***), information quality (h5, β = 0.17***), and service quality (h6, β = 0.32***) had positive impact and explained 48.1% of the variance in user satisfaction. our study finds that superior system quality yields higher user satisfaction (e.g. sachan et al., 2018). system quality reflects the operational efficiency of e-government system in terms of smooth and error-free governmental transactions facilitated by the system. such pleasant experiences improve user satisfaction (sachan et al., 2018; anwer et al., 2016). this study also finds support for information quality, which stresses the importance of information appropriateness, accuracy, and comprehensiveness (salaun & flores, 2001), which increases user satisfaction. similarly, the reliability, accuracy, and usefulness of information provided by e-government systems will satisfy user needs and be instrumental in elevating user satisfaction with e-government systems (gotoh, 2009). this study also found service quality to influence user satisfaction, emphasizing the impact of promptness and accuracy of e-government service on user satisfaction. the user-centric capabilities of e-government systems and a great service experience will improve user satisfaction. both e-government trust (h7, β = 0.33***) and user satisfaction (h8, β = 0.21***) had significant effects on su as hypothesized and explained 51.6% of the variance in su. the results suggest that when they trust e-government systems for authenticity of information, privacy of personal data, security of online transaction, and quality of service (papadopoulou et al., 2010), users may be more inclined to utilize e-government systems for future transactions. users’ egovernment trust is thus reflected in their repeated su (sternstein, 2010). further, satisfied users possess a positive overall perception of e-government which will translate into future use (sachan et al., 2018). su had a significant effect on bi (h9, β = 0.46***) as hypothesized. bi was influenced by system quality (emergent, β = 0.22***) and user satisfaction (emergent, β = 0.15***). these three constructs explained 56.5% of the variance in bi. these results confirm that prior e-government su explains bi. if users found the e-government system technically efficient, reliable, and fast during their previous engagements, they are likely to use it (wang & liao, 2008). users satisfied with and confident about the -government systems’ capabilities may be more inclined to hold stronger intentions for future use (west, 2004). 6.2 limitations the study limitations may be considered in interpreting the study findings. first, the study did not use primary data but relied on statistics reported in prior studies. this study cannot australasian journal of information systems hooda et al. 2023, vol 27, research article clarifying the role of e-government trust 12 verify the accuracy of the statistics reported and assumes that prior studies were of high quality. second, the study aggregated prior findings despite their theoretical and empirical differences. third, the meta-analysis included prior studies published in journals but not in conferences proceedings or other sources such as book chapters. the average failsafe-n across all relationships in the model is high, which can mitigate this limitation. fourth, only those studies that reported zero-order pearson correlations were included in the sample while other studies were excluded. finally, the coding process included assumptions when necessary statistics could not be coded. 6.3 implications for research first, our research contributes to the literature on information system success by extending it with e-government trust. the original iss model was situated in a work environment involving employees (delone & mclean, 1992) while the revised iss model was contextualized in an e-commerce setting relating to customers (delone & mclean, 2003)—in both cases, the iss model included quality dimensions and satisfaction but not trust. since the iss model has been used to investigate su in different contexts, the inclusion of trust makes it more applicable in contexts that involve service-based activities such as e-government. all three dimensions of quality depicted in the iss model were helpful in establishing egovernment trust. second, our research clarifies the potential roles of su and bi at least in the context of iss models. while prior literature on technology acceptance has theoretically argued for su→bi (law et al., 2016) and bi→su (venkatesh et al., 2003), the revised iss model had not specified a relationship between su and bi although both constructs were included in the model (delone & mclean, 2003). this study theorized su→bi rather than bi→su since users may determine their future bi to use e-government based on their past use of the system. the empirical support for the su→bi relationship has implications for modeling su and bi in future studies of technology acceptance. third, our research also highlights new paths between the constructs in iss models. two such new paths involve bi—where system quality and user satisfaction influenced bi. since iss models typically model system quality and user satisfaction to impact su (delone & mclean, 1992), the empirical tests demonstrating the impacts on bi provides new insights. in doing so, this study also showed that e-government trust and user satisfaction mediate the effect of system quality on su, thus emphasizing the role of e-government trust in iss models. in addition, trust was influenced by user satisfaction, showing that trust was influenced by several iss constructs. further, su mediated the effect of user satisfaction on bi. finally, our research does not handle the reciprocal relationships that may be possible between different constructs in our model. for instance, h7 argued that e-government trust influenced su. however, it is possible that continued su over time may influence trust either positively or negatively. similarly, h8 examined the effect of user satisfaction on su. but su can also influence user satisfaction with continue use. further, h9 proposed that su had an effect on bi, but a reciprocal effect is possible since bi has the potential to influence su over time. future research may consider these avenues to examine the reciprocal relationships. 6.4 implications for practice this study provides many useful takeaways for public service providers. first, our investigation suggests service quality, information quality, and system quality of eaustralasian journal of information systems hooda et al. 2023, vol 27, research article clarifying the role of e-government trust 13 government are not only important, but play a vital role in improving users’ overall perception of trust and satisfaction towards e-government system. while designing e-government systems, government agencies must ensure that the system is capable of generating accurate, complete and timely information in a safe and convenient online environment without any technical glitch. e-government managers may analyze clickstream data of first-time users to understand their peculiar requirements and challenges of using e-government system (park et al., 2013). for example, to warrant accuracy and completeness, e-government system designers may deploy auto-test procedures, hyperlinks and guidelines to help users efficiently complete desired tasks. an interactive user-friendly user interface, round-the-clock availability, and a fast query resolution system may foster ease of using e-government (meuter et al., 2000). the importance of service quality is also evident; users exhibit higher levels of satisfaction and trust with high-quality e-government systems. while system quality and information quality may improve functional efficiency, service quality that establishes a partnership between government agencies, technology suppliers, and consultants is necessary to achieve overall operational efficiency (kafaji, 2013). the technology suppliers and consultants must be encouraged to deliver beyond functional features and focus on personalization features. this will enable government agencies to accomplish its goals and increase users’ trust and satisfaction with e-government systems. second, our study validates the significant direct impact of satisfaction and trust on egovernment su. these are two effective means of reducing uncertainty associated with egovernment systems. while efficient and fully secured systems are prerequisites for gaining trust and satisfaction, user skills to optimally and cautiously use the system are equally important. the overall value of e-government systems is co-created by the e-government service provider and the users. awareness and training programs should be conducted to educate users on how to successfully use e-government systems and provide useful feedback for further improvement. such initiatives portray that government agencies care about users’ requirements and are capable of delivering high-quality services. this can positively alter users’ perceptions towards and engagements with e-government systems. third, our study suggests that su is a key determinant of users’ bi to use e-government systems in the future. for the success of e-government initiatives, users should be encouraged to plan their upcoming transactions on e-government systems. users can experience and recognize the benefits of e-government su only when they actually use it (norris & moon, 2005). government agencies should motivate users to use e-government platforms to influence their future use behaviors. incentive schemes and promotion campaigns may be announced to attract users to try e-government platforms, which may facilitate future use of e-government systems. fourth, our analysis showed the direct impact of system quality on bi. this implies that egovernment system designers should prioritize the technical efficacy of the system to serve user needs in terms of efficiency, accuracy, and ease of using the system. e-government system managers should consider the evolving nature of user expectations and requirements and communicate them to the system designers. this calls for update and upgrade of egovernment systems at regular intervals so that users do not feel neglected. by doing so, egovernment agencies can involve users in improving system quality. high-quality systems will attract users to try e-government systems and result in future intentions to use the system should they find the systems technically superior. australasian journal of information systems hooda et al. 2023, vol 27, research article clarifying the role of e-government trust 14 finally, our results revealed the direct impact of user satisfaction on e-government trust and bi. efforts should be made to elevate user satisfaction levels since a satisfied user is likely to trust the efficiency and capability of e-government systems meeting their requirements (sachan et al., 2018). e-government system managers should convince the system designers to not only focus on the functional aspects but also consider the personalization and safety features of the system. since face-to-face interaction with service providers and front-office help may not be possible in e-government settings, such features will be helpful to users and increase their satisfaction. a satisfied user may not be concerned about the inherent uncertainty and risks involved in online interactions and may be more willing to use egovernment service platforms (teo et al., 2008). 7 conclusion this study extended the iss model to include e-government trust as an antecedent of su leading to bi. based on a meta-analysis and structural equation modeling approach, the prominent role of e-government trust in information system success was uncovered in this study. specifically, e-government trust influences su, which in turn, impacts bi; e-government trust is influenced by service quality, information quality, system quality, and user satisfaction; and e-government trust mediates the effects of service quality, information quality, system quality, and user satisfaction on su. these findings offer new theoretical insights and practical implications in dealing with e-government systems. references adams, d. a., nelson, r. r., & todd, p. a. (1992). perceived usefulness, ease of use, and usage of information technology: a replication. mis quarterly, 16(2), 227-247. agarwal, r., & karahanna, e. (2000). time flies when you're having fun: cognitive absorption and beliefs about information technology usage. mis quarterly, 24(4), 665-694. agarwal, r., & prasad, j. (1997). the role of innovation characteristics and perceived voluntariness in the acceptance of information technologies. decision sciences, 28(3), 557-582. albar, a. m., & hddas, m. a. (2018). exploring saudi citizens’ acceptance of mobile government service. international journal of advanced computer science and applications, 9(11), 397–400. https://doi.org/10.14569/ijacsa.2018.091156 alharbi, n., papadaki, m., & dowland, p. (2017). the impact of security and its antecedents in behaviour intention of using e-government services. behaviour & information technology, 36(6), 620–636. al-hujran, o., al-debei, m. m., chatfield, a., & migdadi, m. (2015). the imperative of influencing citizen attitude toward e-government adoption and use. computers in human behavior, 53, 189–203. alkraiji, a. i. (2020). citizen satisfaction with mandatory e-government services: a conceptual framework and an empirical validation. ieee access, 8, 117253–117265. alkraiji, a. i. (2021). an examination of citizen satisfaction with mandatory e-government services: comparison of two information systems success models. transforming government: people, process and policy, 15(1), 36–58. https://doi.org/10.1108/tg-01-20200015 australasian journal of information systems hooda et al. 2023, vol 27, research article clarifying the role of e-government trust 15 almarabeh, t., & abuali, a. (2010). a general framework for e-government: definition maturity challenges, opportunities, and success. european journal of scientific research, 39(1), 29–42. al-omairi, l., al-samarraie, h., alzahrani, a. i., & alalwan, n. (2020). students’ intention to adopt e-government learning services: a developing country perspective. library hi tech, 39(1), 308–334. alqaralleh, b. a. y., al-omari, a. h., & alksasbeh, m. z. (2020). an integrated conceptual model for m-government acceptance in developing countries: the case study of jordan. international journal of interactive mobile technologies, 14(6), 115–136. https://doi.org/10.3991/ijim.v14i06.11449 al-rawahna, a. s. m., chen, s.-c., & hung, c.-w. (2018). the barriers of e-government success: an empirical study from jordan. international journal of managing public sector information and communication technologies, 9(2), 1–18. alruwaie, m., el-haddadeh, r., & weerakkody, v. (2020). citizens’ continuous use of egovernment services: the role of self-efficacy, outcome expectations and satisfaction. government information quarterly, 37(3), 101485. alryalat, m. a. a., rana, n. p., & dwivedi, y. k. (2015). citizen’s adoption of an e-government system: validating the extended theory of reasoned action (tra). international journal of electronic government research, 11(4), 1–23. alshaher, a. (2021). it capabilities as a fundamental of electronic government system success in developing countries from users perspectives. transforming government: people, process and policy, 15(1), 129–149. https://doi.org/10.1108/tg-05-2020-0080 al-sulami, z. a., & hashim, h. s. (2018). measuring the success of e-government systems: applying the success model of the delone and mclean information system. journal of theoretical and applied information technology, 96(22), 7654–7670. al-zahrani, m. s. (2020). integrating is success model with cybersecurity factors for egovernment implementation in the kingdom of saudi arabia. international journal of electrical and computer engineering, 10(5), 4937–4955. https://doi.org/10.11591/ijece.v10i5.pp4937-4955 angelopoulos, s., kitsios, f., & papadopoulos, t. (2010). new service development in egovernment: identifying critical success factors. transforming government: people, process and policy, 4(1), 95-118. anthopoulos, l., reddick, c. g., giannakidou, i., & mavridis, n. (2016). why e-government projects fail ? an analysis of the healthcare.gov website. government information quarterly, 33(1), 161–173. https://doi.org/10.1016/j.giq.2015.07.003 anwer, m. a., esichaikul, v., rehman, m., & anjum, m. (2016). e-government services evaluation from citizen satisfaction perspective: a case of afghanistan. transforming government: people, process and policy, 10(1), 139–167. azam, a., qiang, f., & abdullah, m. i. (2013). determinants of e-government services adoption in pakistan: an integrated model. electronic government, an international journal, 10(2), 105–124. australasian journal of information systems hooda et al. 2023, vol 27, research article clarifying the role of e-government trust 16 bannister, f., & connolly, r. (2011). trust and transformational government: a proposed framework for research. government information quarterly, 28(2), 137–147. bentler, p.m. & bonett, d.g. (1980). significance tests and goodness of fit in the analysis of covariance structures. psychological bulletin, 88(3), 588-606. bhattacherjee, a., & sanford, c. (2009). the intention-behavior gap in technology usage: the moderating role of attitude strength. behaviour & information technology, 28(4), 389-401. browne, m.w. & cudeck, r. (1993). single sample cross-validation indices for covariance structures. multivariate behavioral research, 24(4), 445-455. carter, l., & bélanger, f. (2005). the utilization of e-government services: citizen trust, innovation and acceptance factors. information systems journal, 15(1), 5–25. davis, f. d. (1989). perceived usefulness, perceived ease of use, and user acceptance of information technology. mis quarterly, 13(3), 319–340. davis, f. d., bagozzi, r. p., & warshaw, p. r. (1989). user acceptance of computer technology: a comparison of two theoretical models. management science, 35(8), 9821003. delone, w. h., & mclean, e. r. (1992). information systems success: the quest for the dependent variable. information systems research, 3(1), 60–95. delone, w. h., & mclean, e. r. (2003). the delone and mclean model of information systems success: a ten-year update. journal of management information systems, 19(4), 9-30. eid, r., selim, h., & el-kassrawy, y. (2020). understanding citizen intention to use mgovernment services: an empirical study in the uae. transforming government: people, process and policy, 15(4), 463-482. farias, p., goldsmith, s., flumian, m., mendoza, g., wiseman, j. & zanabria, g. (2017). governments that serve: innovations that improve service delivery to citizens. interamerican development bank, washington d.c, usa. https://publications.iadb.org/en/publication/17464/governments-serve-innovationsimprove-service-delivery-citizens fishbein, m., & ajzen, a. (1980). understanding attitudes and predicting social behaviour. prentice-hall. inc., englewood cliffs, nj, usa. fukuyama f. (1995). the social virtues and the creation of prosperity. free press, london, uk. gilbert, d., balestrini, p., & littleboy, d. (2004). barriers and benefits in the adoption of egovernment. the international journal of public sector management, 17(4/5), 286–301. gonzalez, p., adenso-diaz, b., & gemoets, l. a. (2010). a cross-national comparison egovernment success measures: a theory-based empirical research. in proceedings of the 16th americas conference on information systems (amcis 2010), pp. 1–10. lima, peru. https://aisel.aisnet.org/amcis2010/354 gotoh, r. (2009). critical factors increasing user satisfaction with e-government services. electronic government, an international journal, 6(3), 252-264. gupta, p. j., & suri, p. (2017). measuring public value of e-governance projects in india: citizens’ perspective. transforming government: people, process and policy, 11(2), 236-261. australasian journal of information systems hooda et al. 2023, vol 27, research article clarifying the role of e-government trust 17 hariguna, t., lai, m.-t., hung, c.-w., & chen, s.-c. (2017). understanding information system quality on public e-government service intention: an empirical study. international journal of advanced manufacturing technology and maintenance and reliability. human behavior, 11(3), 271–290. hossan, c., & ryan, j. (2016). factors affecting e-government technology adoption behaviour in a voluntary environment. international journal of electronic government research, 12(1), 24–49. https://doi.org/10.4018/ijegr.2016010102 hunter, j. e., & schmidt, f. l. (2004). methods of meta-analysis: correcting error and bias in research findings. sage publishing, thousand oaks, ca, usa. ibrahim, o. a., & zakaria, n. h. (2016). e-government services in developing countries: a success adoption model from employees perspective. journal of theoretical and applied information technology, 94(2), 383–396. idoughi, d., & abdelhakim, d. (2018). developing countries e-government services evaluation identifying and testing antecedents of satisfaction case of algeria. international journal of electronic government research (ijegr), 14(1), 63–85. jackson, c. m. (1997). toward an understanding of the behavioral intention to use an information system. decision sciences, 28(2), 357-389. jeyaraj, a., & dwivedi, y. k. (2020). meta-analysis in information systems research: review and recommendations. international journal of information management, 55(december), 102226. kafaji, m. a. (2013). evaluating the roll of service quality as a mediator on user satisfaction in e-government applications. problems of management in the 21st century, 8, 55. http://oaji.net/articles/2014/450-1397905708.pdf kamarudin, s., omar, s. z., zaremohzzabieh, z., bolong, j., & osman, m. n. (2021). factors predicting the adoption of e-government services in telecenters in rural areas: the mediating role of trust. asia-pacific social science review, 21(1), 20–38. karavasilis, i., vrana, v., & zafiropoulos, k. (2016). an extended model of e-government adoption by civil servants in greece. international journal of electronic government research, 12(1), 1–23. kelly, b., & vidgen, r. (2005). a quality framework for web site quality: user satisfaction and quality assurance. paper presented at the 14th international conference on world wide web 2005, may 10-14, chiba, japan, pp. 930–93. https://doi.org/10.1145/1062745.1062803 konradt, u., nerdinger, f. w., & ellwart, t. (2016). a model for usability in e-commerce services: theoretical concept and empirical evidence. in encyclopedia of e-commerce development, implementation, and management, pp. 2332-2343, igi global, hershey, pe, usa. kumar, v., mukerji, b., & butt, i. (2007). factors for successful e government adoption: a conceptual framework. electronic journal of e-government, 5(1), 63-76. kumar, r., sachan, a., mukherjee, a., & kumar, r. (2018). factors influencing e-government adoption in india: a qualitative approach. digital policy, regulation and governance, 20(5), 413–433. https://dl.acm.org/doi/proceedings/10.1145/1062745 https://dl.acm.org/doi/proceedings/10.1145/1062745 australasian journal of information systems hooda et al. 2023, vol 27, research article clarifying the role of e-government trust 18 lai, c. s. k., & pires, g. (2010). testing of a model evaluating e-government portal acceptance and satisfaction. electronic journal of information systems evaluation, 13(1), 35-46. law, m., kwok, r. c., & ng, m. (2016). an extended online purchase intention model for middle-aged online users. electronic commerce research and applications, 20, 132-146. layne, k., & lee, j. (2001). developing fully functional e-government: a four stage model. government information quarterly, 18(2), 122–136. lee, a., & levy, y. (2014). the effect of information quality on trust in e-government systems' transformation. transforming government: people, process and policy, 8(1), 76-100. lewis, j. d., & weigert, a. (1985). trust as a social reality. social forces, 63(4), 967-985. limayem, m., hirt, s. g., & cheung, c. m. k. (2007). how habit limits the predictive power of intention: the case of information systems continuance. mis quarterly, 31(4), 705737. luhmann, n. (1988). trust: making and breaking cooperative relations in gambetta, d. (ed.) familiarity, confidence, trust: problems and alternatives, pp. 94–107, blackwell, new york, usa. maclean, d., & titah, r. (2022). a systematic literature review of empirical research on the impacts of e-government: a public value perspective. public administration review, 82(1), 23-28. https://doi.org/10.1111/puar.13413 mansoori, k. a. a., sarabdeen, j., & tchantchane, a. l. (2018). investigating emirati citizens’ adoption of e-government services in abu dhabi using modified utaut model. information technology & people, 31(2), 455–481. https://doi.org/10.1108/itp-12-2016-0290 mellouli, m., bouaziz, f., & bentahar, o. (2020). e-government success assessment from a public value perspective. international review of public administration, 25(3), 153–174. https://doi.org/10.1080/12294659.2020.1799517 mensah, i. k. (2019). predictors of electronic government services adoption: the african students’ perspective in china. international journal of public administration, 42(12), 997– 1009. mensah, i. k., & mi, j. (2017). electronic government services adoption: the moderating impact of perceived service quality. international journal of electronic government research, 13(3), 38–54. https://doi.org/10.4018/ijegr.2017070103 mensah, i. k., & mi, j. (2018). an empirical investigation of the impact of demographic factors on e-government services adoption. international journal of e-services and mobile applications, 10(2), 17–35. https://doi.org/10.4018/ijesma.2018040102 meuter, m. l., a. l. ostrom, r. i. roundtree, & m. j. bitner (2000). self-service technologies: understanding customer satisfaction with technology-based service encounters. journal of marketing, 64(3), 50-64. morgeson, f. v., vanamburg, d., & mithas, s. (2011). misplaced trust? exploring the structure of the e-government-citizen trust relationship. journal of public administration research and theory, 21(2), 257–283. australasian journal of information systems hooda et al. 2023, vol 27, research article clarifying the role of e-government trust 19 mun, y. y., yoon, j. j., davis, j. m., & lee, t. (2013). untangling the antecedents of initial trust in web-based health information: the roles of argument quality, source expertise, and user perceptions of information quality and risk. decision support systems, 55(1), 284-295. nawafleh, s. (2018). factors affecting the continued use of e-government websites by citizens: an exploratory study in the jordanian public sector. transforming government: people, process and policy,17 (1), 1-17. nielsen, j. (1999). designing web usability: the practice of simplicity. new riders publishing, thousand oaks, ca, usa. nielsen, m. m. (2016). e-governance and stage models: analysis of identified models and selected eurasian experiences in digitising citizen service delivery. electronic government, an international journal, 12(2), 107–141. https://doi.org/10.1504/eg.2016.076132 nielsen, m.m. (2019).tutorial: how to create an ecosystem and framework for digital public sector service delivery. sixth international conference on edemocracy & egovernment (icedeg), quito, ecuador, pp. 6-9, doi: 10.1109/icedeg.2019.8734336. nielsen, m.m. (2021). the untapped potential: the inclusive, personal and co-created public service experience in europe. in musiał-karg, m., luengo, ó.g. (eds). digitalization of democratic processes in europe. studies in digital politics and governance. springer publishing company, cham, switzerland. https://doi.org/10.1007/978-3-030-71815-2_13 noor, z. m., kasimin, h., aman, a., & sahari, n. (2011). an adoption model of electronic government services in malaysia: electronic labor exchange (elx). jurnal pengurusan, 33, 87–97. https://doi.org/10.17576/pengurusan-2011-33-07 norris, d. f., & moon, m. j. (2005). advancing e-government at the grassroots: tortoise or hare?. public administration review, 65(1), 64-75. obaid, q. m. s., & ahmad, m. f. (2021). the linkage between e government and citizens’ satisfaction in uae. annals of the romanian society for cell biology, 25(1), 3118–3130. papadopoulou p., nikolaidou m. & martakos d. (2010).what is trust in e-government? a proposed typology. proceedings of 43rd hawaii international conference on system sciences, honolulu, hi, usa, 2010, pp. 1-10, doi: 10.1109/hicss.2010.491. park, s., choi, y. t., & bok, h. s. (2013). does better e-readiness induce more use of egovernment? evidence from the korean central e-government. international review of administrative sciences, 79(4), 767-789. piehler, r., wirtz, b., & daiser, p. (2016). an analysis of continuity intentions of egovernment portal users. public management review, 18(2), 163–198. https://doi.org/10.1080/14719037.2014.965270 qasim nidawy, b. al, sidek, s., ali al-shami, s., & elzamly, a. (2020). e-government trust model (egov-trust) that enhances the usage of e-government services among users in iraq. journal of theoretical and applied information technology, 15(3), 402-418. rana, n. p., dwivedi, y.k. , & williams, m. d. (2013). evaluating the validity of is success models for the electronic government research: an empirical test and integrated australasian journal of information systems hooda et al. 2023, vol 27, research article clarifying the role of e-government trust 20 model. international journal of electronic government research, 9 (3), 1–22. https://doi.org/10.4018/jegr.2013070101. rana, n. p., & dwivedi, y. k. (2015). citizen’s adoption of an e-government system: validating extended social cognitive theory (sct). government information quarterly, 32(2), 172–181. https://doi.org/10.1016/j.giq.2015.02.002 rana, n. p., dwivedi, y. k., & williams, m. d. (2015). a meta-analysis of existing research on citizen adoption of e-government. information systems frontiers, 17(3), 547–563. roseth, b., & reyes, a.(2017). wait no more: citizens, red tape, and digital government: caribbean edition. inter-american development bank, washington d.c., usa. http://dx.doi.org/10.18235/0002080 sabherwal, r., jeyaraj, a., & chowa, c. (2006). information system success: individual and organizational determinants. management science, 52(12), 1849-1864. sachan, a., kumar, r., & kumar, r. (2018). examining the impact of e-government service process on user satisfaction. journal of global operations and strategic sourcing, 11(3), 321336. salaun, y., & flores, k. (2001). information quality: meeting the needs of the consumer. international journal of information management, 21(1), 21-38. santa, r., macdonald, j. b., & ferrer, m. (2019). the role of trust in e-government effectiveness, operational effectiveness and user satisfaction: lessons from saudi arabia in e-g2b. government information quarterly, 36(1), 39–50. https://doi.org/10.1016/j.giq.2018.10.007 savoldelli, a., codagnone, c., & misuraca, g. (2014). understanding the e-government paradox: learning from literature and practice on barriers to adoption. government information quarterly, 31, s63-s71. shahzad, f., xiu, g., shafique khan, m. a., & shahbaz, m. (2020). predicting the adoption of a mobile government security response system from the user’s perspective: an application of the artificial neural network approach. technology in society, 62(june 2019). https://doi.org/10.1016/j.techsoc.2020.101278 sharma, r., & mishra, r. (2017). investigating the role of intermediaries in adoption of public access outlets for delivery of e-government services in developing countries: an empirical study. government information quarterly, 34(4), 658-679. shuib, l., yadegaridehkordi, e., & ainin, s. (2019). malaysian urban poor adoption of egovernment applications and their satisfaction. cogent social sciences, 5(1), 1–18. https://doi.org/10.1080/23311886.2019.1565293 singh, h., grover, p., kar, a. k., & ilavarasan, p. v. (2020). review of performance assessment frameworks of e-government projects. transforming government: people, process and policy, 14(1), 31-64. https://doi.org/10.1108/tg-02-2019-0011 stefanovic, d., marjanovic, u., delić, m., culibrk, d., & lalic, b. (2016). assessing the success of e-government systems: an employee perspective. information & management, 53(6), 717–726. australasian journal of information systems hooda et al. 2023, vol 27, research article clarifying the role of e-government trust 21 sternstein, a. (2010). study links online transparency efforts, trust in government. nextgov, february 16, 2010. https://www.nextgov.com/cxo-briefing/2010/02/study-links-onlinetransparency-efforts-trust-in-government/45965/ sztompka, p. (1998). trust, distrust and two paradoxes of democracy. european journal of social theory, 1(1), 19-32. taylor, s., & todd, p. (1995). understanding information technology usage: a test of competing models. information systems research, 6(2), 144-176. teo, t. s. h., srivastava, s. c., & jiang, l. (2008). trust and electronic government success: an empirical study. journal of management information systems, 25(3), 99–132. undesa (2004). global e-government readiness report 2004 towards access for opportunity. https://publicadministration.un.org/egovkb/en-us/reports/un-e-government-survey2004 (accessed: 1 january 2022). undesa (2016) e-government in support of sustainable development. united nations. available at: https://publicadministration.un.org/egovkb/en-us/reports/un-egovernment-survey-2016 (accessed: 3 january 2022). undesa (2018). gearing e-government to support transformation towards sustainable and resilient societies. available at: https://publicadministration.un.org/egovkb/enus/reports/un-e-government-survey-2018 (accessed: 1 january 2022). valle-cruz, d. (2019). public value of e-government services through emerging technologies. international journal of public sector management, 32(5), 473–488. van slyke, c., ilie, v., lou, h., & stafford, t. (2007). perceived critical mass and the adoption of a communication technology. european journal of information systems, 16, 270-283. veeramootoo, n., nunkoo, r., & dwivedi, y. k. (2018). what determines success of an egovernment service? validation of an integrative model of e-filing continuance usage. government information quarterly, 35(2), 161–174. venkatesh, v., morris, m. g., davis, g. b., & davis, f. d. (2003). user acceptance of information technology: toward a unified view. mis quarterly, 27(3), 425–478. venkatesh, v., thong, j. y. l., chan, f. k. y., hu, p. j. h., & brown, s. a. (2011). extending the two-stage information systems continuance model: incorporating utaut predictors and the role of context. information systems journal, 21(6), 527–555. wang, c., & teo, t. s. h. (2020). online service quality and perceived value in mobile government success: an empirical study of mobile police in china. international journal of information management, 52(january), 102076. https://doi.org/10.1016/j.ijinfomgt.2020.102076 wang, y. s., & liao, y. w. (2008). assessing egovernment systems success: a validation of the delone and mclean model of information systems success. government information quarterly, 25(4), 717–733. wang, y. j., hernandez, m. d., & minor, m. s. (2010). web aesthetics effects on perceived online service quality and satisfaction in an e-tail environment: the moderating role of purchase task. journal of business research, 63(9-10), 935-942. australasian journal of information systems hooda et al. 2023, vol 27, research article clarifying the role of e-government trust 22 warkentin, m., gefen, d., pavlou, p. a., & rose, g. m. (2002). encouraging citizen adoption of e-government by building trust. electronic markets, 12(3), 157-162. warshaw, p. r., & davis, f. d. (1985). disentangling behavioral intention and behavioral expectation. journal of experimental social psychology, 21(3), 213-228. weerakkody, v., el-haddadeh, r., al-sobhi, f., shareef, m. a., & dwivedi, y. k. (2013). examining the influence of intermediaries in facilitating e-government adoption: an empirical investigation. international journal of information management, 33(5), 716–725. weerakkody, v., irani, z., lee, h., hindi, n., & osman, i. (2014). a review of the factors affecting user satisfaction in electronic government services. international journal of electronic government research, 10(4), 21–56. weerakkody, v., irani, z., lee, h., hindi, n., & osman, i. (2016). are u.k. citizens satisfied with e-government services? identifying and testing antecedents of satisfaction. information systems management, 33(4), 331–343. west, d. m. (2004). e-government and the transformation of service delivery and citizen attitudes. public administration review, 64(1), 15-27. whitener, e.m. (1990). confusion of confidence intervals and credibility intervals in metaanalysis. journal of applied psychology, 75(3), 315-321. wu, j., & lederer, a. (2009). a meta-analysis of the role of environment-based voluntariness in information technology acceptance. mis quarterly, 33(2), 419-432. wu, j., & du, h. (2012). toward a better understanding of behavioral intention and system usage constructs. european journal of information systems, 21(6), 680-698. copyright: © 2023 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v27i0.4079 http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 theoretical background 2.1 e-government systems 2.2 technology adoption and use 2.3 is success and trust 3 research model 3.1 e-government trust 3.2 user satisfaction 3.3 e-government system use 3.4 behavioral intention 4 research methods 4.1 sample 4.2 coding 4.3 analysis 5 results 6 discussion 6.1 findings 6.2 limitations 6.3 implications for research 6.4 implications for practice 7 conclusion paper title (paper title style) australasian journal of information systems pye, holt, salzman, bellucci & lombardi 2015, vol. 19, research article engaging students in blended environments 1 engaging diverse student audiences in contemporary blended learning environments in australian higher business education: implications for design and practice graeme pye deakin university graeme.pye@deakin.edu.au dale holt deakin university dale.holt@deakin.edu.au scott salzman deakin university scott.salzman@deakin.edu.au emilia bellucci deakin university emilia.bellucci@deakin.edu.au luisa lombardi deakin university luisa.lombardi@deakin.edu.au abstract this research reports on a student audience engaging in an australian university’s undergraduate commerce program information systems core unit that is offered across three separate geographic campus locations and online. the research extends upon work undertaken on student engagement within the located domain of blended learning environments and juxtaposes the online perspective in an australian higher education business context. findings, inter alia, are presented across seven major student engagement dimensions as applied to the interplay between online and located/campus learning (i.e. online active learning, online social interaction, online collaboration, online teaching, online assessment, online academic relevance and online contact with staff). implications for blended learning design, elearning and practice in such complex online environments are examined. keywords: higher education; technology; student engagement 1 introduction student engagement is an important predictor of student academic success, satisfaction and retention rates within undergraduate courses and university more broadly. this research represents the initial findings of a two year project investigating student engagement experiences of learning utilising a contemporary online blended learning management system (lms) environment. the research investigates student engagement in a foundational disciplinary business subject offered in a large faculty of business and law in a major australian university using such an lms. the university in question operates across three campuses in three separate cities in metropolitan, regional and rural settings in the australian state of victoria. it has a long and distinguished history in the provision of distance and online education. australian and international universities continue to strategically seek competitive advantage by exploiting information and communication technologies (ict) to service students, irrespective of their study location, through extending the options for flexible education, distance education, elearning and online blended learning environments. the site of this research, deakin university, has committed to the aspiration of pushing the digital frontier, with its own blended cloud (online) and located (physically in australasian journal of information systems pye, holt, salzman, bellucci & lombardi 2015, vol. 19, research article engaging students in blended environments 2 situ) learning multi-modal vision. located learning settings encompass primarily the university’s various distributed campuses, although its learning centres and designated workplace settings of relevance to its work-integrated learning programs can also be considered as part of an expanded view of where valued located learning occurs. the research aims to inform, enhanced delivery and the quality of business education in australian higher education by eliciting the student engagement factors and the dynamic interplay of student learning in the cloud, and physically located in situ on campus by: • developing, piloting and refining a student engagement survey instrument related to cloud (online) and located (in situ) learning; • ascertaining the nature and extent of student engagement in such blended learning environments; and • developing a set of design principles and practices to enhance student engagement. this paper presents findings from the administration of a student engagement survey to students enrolled in a large, first year core subject on business information systems offered in the faculty’s commerce program. the findings focus on the online student engagement factors of students, as opposed to staff teaching experiences in a blended learning environment. the unit is offered physically in situ across three located campuses and offcampus in the cloud. the survey was based on significant international work reported in the higher education literature relating to the key dimensions of student engagement in the use of online learning systems in the context of their increasing adoption in campus-based settings. the research focuses on ubiquitous lms environments and not on the more specialised educational technology field of 2-dimensional and 3-dimensional gaming and simulation spaces, including, for example, student engagement in 3-d platforms like second life (see holt, segrave & cybulski, 2011). implications for design and practice enhancements in such mainstream lms environments are considered along with future developments in the research. 2 background 2.1 new blended learning environments in higher education the positioning of this research is distinctive, yet indicative of the multi-dimensional blended learning environments employed in the australian higher education sector, with a specific focus on the business education domain. in particular, deakin university’s faculty of business and law offers most of its programs in multi-modal form both in the cloud and located at multiple campus locations. program design, delivery and teaching practices have evolved to provide multi-modal study options and flexibility in students’ choice of units and delivery modes of blended learning environments. ict have been designed and implemented to further facilitate and enable these various study options. ict along with its now mobile forms, driven by a flexibility imperative is leading to the convergence of all modes of learning around contemporary online blended learning environments. traditional synchronous and asynchronous modes of distance education has been a major global force over the last forty or more years, as manifested in the establishment of a large number of dedicated national open universities, using successive waves of ict innovation and development (daniel, 1996). the most recent manifestation of distance education in the form of elearning can be found in the global reach of providers of massive open online courses (moocs) which are being originated by a broad range of higher education institutions. some of their offerings are being used to enrich the on-campus learning experience. (for example, see the special edition of distance education on research and development of moocs in global higher education, 2014.) running in parallel with the expansion of global distance education offerings, has been the emergence of more flexible and blended forms of provision emerging out of the growing use australasian journal of information systems pye, holt, salzman, bellucci & lombardi 2015, vol. 19, research article engaging students in blended environments 3 of ict in traditional campus, residential-based universities, in school programs and through major workforce training provision around the world (bonk & graham, 2006; latcham & jung, 2010; kristin, 2013). these developments have revolved around the reconfiguration of the provisions of classroom (located) teaching and online learning. this reconfiguration involving both the nature and amount of both types of provision has a special focus on attempts to reinvent campus/site-based education in the wake of ict developments sweeping over the entire educational and training sectors. garrison and vaughan (2008) and shea (2007) highlight the need for blended approaches to be well integrated and mutually reinforcing in value to student learning. distance education remains attractive to matureaged students, often in employment and/or with significant personal responsibilities, wishing to have an accessible and quality distance education experience. likewise, blended approaches to learning has been grounded in the need to provide a better quality and more flexible learning experience for younger students who are predominantly studying in campus-based (located) environments. both located and online learning are relatively discrete with learning provisions targeted to two reasonably well defined and different student cohorts. this study focuses on the major convergence of these two types of learning modes where traditional conceptions of on-campus, off-campus, classroom and online, have begun to dissolve in the face of the imperative for maximum flexibility in the ways students organise their studies in their increasingly hectic lives of work and play. this notional convergence or blending of the located and online learning environment, is exemplified at deakin university as clouddeakin. this learning management system (lms) seeks to deliver a single blended learning environment that is both engaging and relevant to each respective located (in situ on-campus) and cloud (online) student cohorts, as per figure 1. figure 1. clouddeakin lms structure 2.2 investigating blended and online learning through the lens of student engagement various lines of conceptual framing have been developed to promote students learning online. some of these conceptual frameworks have been grounded in elearning in distance education. notably, for instance, the work of salmon (2011) and her five-stage, scaffold model for enabling collaborative learning, and that of garrison and anderson (2003) and their three interrelated presences (cognitive, social and teaching) for enabling robust online communities of inquiry. bonk and zhang (2008) developed a practical read, reflect, display and do (r2d2) model for online learning which is primarily directed at using different sensemaking modalities, and with broader applicability for designing both online and blended learning. the current research aims to examine student perceptions of their experiences of evolving online and blended learning environments through the ‘student engagement’ frame of reference. the study of student engagement can be most directly related to the empirically australasian journal of information systems pye, holt, salzman, bellucci & lombardi 2015, vol. 19, research article engaging students in blended environments 4 well-grounded work of garrison and anderson (2003) who draw heavily on various dimensions of interactivity promoted by educational technologies in developing their conceptual framework, namely: teacher-student interaction; student-student interaction; student-content interaction; teacher-teacher interaction; and content-content interaction. they also featured prominently the importance of social and teaching presence in building effective online learning communities. the research contributes to the large body of work on promoting student learning in online and blended learning settings in two important respects. first, it adds to the research undertaken on student as opposed to staff experiences of blended learning and teaching (see graham & robison, 2007; kaleta, skibba & joosten, 2007). second, the aim here is to seek student perceptions so as to investigate how students are practising engagement at university, based on a strong line of higher education research in this domain and to extend these investigations into the particular field of students engaging with online and blended environments. student engagement focuses on the time and effort students put into purposively designed educational activities directed at achieving desired learning outcomes. student engagement is seen as a proxy for quality student experiences and outcomes. it has been well conceptualised, theorised, researched and actioned in the us (kuh, 2003; kuh, kinzie, schuh, whitt & associates, 2005). it has been adapted to australian conditions through the work of scott (2006) in analysing course experience questionnaire (ceq) open-ended comments, the australasian survey of student engagement (ausse, see acer website) and the more recent university experience survey (ues). a major leader in the development of these surveys has been hamish coates, formerly at the australian council of educational research (acer), then the l h martin institute and followed by being director of the centre for the study of higher education (cshe) at the university of melbourne (see krause & coates, 2008; coates, 2010). coates (2006) investigated how learning technologies, as particularly prominent through the large-scale implementation of learning management systems (lmss) throughout higher education, were impacting campus-based student engagement. taking account of the more general set of dimensions constituting student engagement, coates (2006) developed a set of student engagement constructs for online learning environments (as related to online learning engagement in campus-based settings). this set of dimensions are directed at finding out what students self-report that they actually do online, as opposed to what teachers might think they should be doing. coates’ approach elicits student views on what and how much they have done of relevance to their academic pursuits. these constructs of online student learning inform the development of a student engagement survey for cloud (online) and located (campus) learning in this research relating to a business education context. the research adapts the work of coates (2006) to blended learning in contemporary educational technology contexts, and represents a unique niche in the myriad of investigations on student learning online. the only line of research identified which has investigated the intersection of blended learning and student engagement is reported by george-walker, hafeez-baig, gururajan and danaher (2010). two of their cases related to postgraduate business education in the field of information systems and information technology. while the prior work of george-walker et al. (2010) was evaluated, their work was not directly informed by the student engagement literature or the survey method of data collection in the student engagement domain. 2.3 key characteristics of blended learning environments for the purposes of this research, 24 elements constituting developments in blended learning have been identified by the research team. these are the factors that can be chosen and varied in the design and practical operation of blended learning environments. these factors will be discussed further under the findings implications. australasian journal of information systems pye, holt, salzman, bellucci & lombardi 2015, vol. 19, research article engaging students in blended environments 5 curriculum and assessment design elements: 1. unit aim; 2. course learning outcomes (clos); 3. relevant discipline-based threshold learning outcomes (tlos); 4. unit learning outcomes (ulos); 5. relevant graduate learning outcomes (glos); 6. formal assessment requirements. curriculum delivery elements around cloud and located learning: 7. online assessment rubric(s); 8. interface design for presentation of learning resources; 9. prescribed text(s), including purpose, use & electronic availability; 10. other publisher learning resources, e.g. online quiz banks, podcasts, interactive exercises, simulations; 11. readings – number, type, use; 12. live class/lecture recordings & powerpoint slides; 13. live class/lecture webinars (linking in students off-site); 14. pre-recorded class/lecture resources & powerpoint slides; 15. other in-house recordings, e.g. concepts, models, interviews; 16. face-to-face, in-person lectures/classes presentations – number, hours, purposes; 17. face-to-face, in-person tutorials/seminars – number, hours, purposes; 18. online discussion forums – purposes; 19. electronic tutorials/seminars – number, hours, purposes, type of technology used ; 20. online discussion forums – purposes; 21. electronic consultation drop-in hours – purpose, type of technology used; 22. eportfolio use; 23. planned integration of social media – purpose, type of social media used; 24. knowledge of unplanned use of social media. table 1: key characteristics of blended learning environments 3 research methodology 3.1 scope the research is being conducted over two years (2014-2015) involving surveying students undertaking undergraduate business education studies in business information systems, which is a first trimester, first year, core unit in both the commerce and information systems undergraduate programs. business information systems is a very large commencement study unit and plays an important role in helping students in transitioning into studies in higher education in the business realm. the unit chosen is seen to be representative of major developments in cloud and located learning in business education, in the faculty. 3.2 theoretical frame of reference citing coates (2006, p.69), student surveys are structured conversations between learners and universities. this research uses a survey design to ascertain student views on how, and how much, they are engaging with the cloud and located learning environments. the online survey method used is self-selecting and gives the most number of students the opportunity to share their views on what they have personally experienced. drawing on an already developed, trialled and validated student engagement survey related to online learning in campus contexts by coates (2006), the survey developed for this research is relevant to the online student experiences to be elicited, and efficient to administer with large student cohorts. the six key dimensions of the survey, cover: australasian journal of information systems pye, holt, salzman, bellucci & lombardi 2015, vol. 19, research article engaging students in blended environments 6 1. online active learning: key ways in which students use online systems to construct and enhance learning; 2. online social interaction: use of online learning systems to support general forms of social interaction; 3. online collaboration: students’ use of online systems to work collaboratively with peers; 4. online teaching: whether teachers use online learning systems to promote effective learning; 5. online academic relevance: use of online systems to enhance the relevance and context of study; 6. online contact with staff: extent and ease of use of online systems by students in contacting staff. 3.3 online student engagement survey design the student engagement survey used the preceding six key dimensions identified by coates (2006) and associated activities indicating quality and quantity of time and effort related to each. a seventh student engagement dimension was added relating to various activities associated with online assessment. the assessment management features of learning management systems are seen as key benefits in supporting teaching and student learning online. assessment review and improvement is seen a major means of enhancing the student learning experience in higher education (boud, 2010; boud & molloy, 2013; nicol, 2010). focus on assessment has been a major driver in reviewing and renewing curriculum design and delivery in australian higher education, as advanced by senior university educational leadership (see lawson, 2015; oliver, 2013). it was seen as highly pertinent to student engagement with contemporary curricula and enabling learning environments to foreground assessment as a key dimension of student engagement in this research. in addition to the seven dimensions of student engagement, several other pertinent factors were surveyed as summarised in table 2. other survey factors: • background information: gender, primary campus of study, age, student status (domestic or international); • accessing and downloading learning resources to mobile devices; • use of social media for collaborative purposes; • effectiveness of integration of cloud (online) and located (campus) environments (on-campus students only); • amount of time studying unit on average per week; • achievement of unit learning outcomes; • amount of time spent studying unit online; • amount of study time spent studying unit away from campus (on-campus students only); • learning benefits of online environment; • most and least effective aspects of unit blended learning environment. table 2: other survey factors 3.4 data collection and response rate this paper reports on summary data collected from students studying a major first year unit, business information systems (bis), which is a core unit in the undergraduate bachelor of commerce degree at deakin university. data collection was undertaken anonymously using self-selection sampling via an online survey of student participants utilising an online survey australasian journal of information systems pye, holt, salzman, bellucci & lombardi 2015, vol. 19, research article engaging students in blended environments 7 tool (called “opinio”) administered through the strategic intelligence and planning unit at deakin university during weeks 8 to 11 of trimester 1, 2014. the research project and survey instrument received ethics approval from the faculty’s ethics committee. the total number of enrolled student invitee emails was 1345. the response rate is shown in table 3. table 3: survey response rates for completions and incompletes comparisons between sample and population demographic and background factors is presented in table 4. students sample population no. of respondents 104 1345 response rate (completions) 7.73% — gender female 44 (42%) 503 (38%) male 60 (58%) 842 (62%) mode of enrolment located/on-campus 83 (80%) 1194 (89%) cloud/online/offcampus 21 (20%) 151 (11%) campus melbourne burwood 60 (58%) 898 (67%) geelong waterfront 15 (14%) 254 (19%) warrnambool 8 (8%) 42 (3%) cloud/online/offcampus 21 (20%) 151 (11%) domestic/international domestic 95 (91%) 1218 (91%) international 9 (9%) 127 (9%) age 18-21 71 (68%) 1001 (74%) 22-25 8 (8%) 223 (17%) 26-30 6 (6%) 54 (4%) 31-36 12 (11%) 41 (3%) 37-42 1 (1%) 13 (1%) above 42 6 (6%) 13 (1%) table 4: sample and population comparisons australasian journal of information systems pye, holt, salzman, bellucci & lombardi 2015, vol. 19, research article engaging students in blended environments 8 the survey system saves responses after each page is complete, so some partially completed responses exist. in the survey analysis, the maximum applicable number of useful responses is used. in the sample, 58% were female and 42% were male. these proportions are not significantly different. the sample statistics indicate that females make up somewhere between 48% and 67% of the student population, and males make up somewhere between 33% and 52% of the population. however, the university student information system (sis) database unit population records indicate that there are 504 females and 802 males enrolled, representing 37% females and 63% males. gender is, therefore, disproportionately represented in the sample. whilst data on the gender of participants is not the focus of this research, the results do conform to other surveys conducted online. just as it is well known that pleasure seekers are less likely to respond to an online survey (sax, gilmartin & bryant, 2003), research also indicates that unmarried people, females and older students are more likely to respond to online surveys (dey, 1997; johnson, o’rourke, burris & owens, 2002; cull, o'connor, sharp & tang, 2005). more specifically, females are more likely to respond in online surveys (rogelberg & luong, 1998). moreover, whilst we recognise the likelihood of a degree of selfselection bias, we also recognise the potential for committed self-selecting participants. 3.5 data analysis all data was managed using microsoft xl 2013 and analysed using xl statistics version: 09.09.14. it is noted that the inherent bias often present in online surveys implies that it is possible that some groups might be disproportionately represented and therefore, generally numerical summaries are presented and discussed. however, where applicable, to assess for differences between two means, the t-test was used, and where hypotheses tests applied, critical alpha was set at 0.05. 4 findings a summary of findings is outlined under key survey headings. 4.1 study time, allocation and achievement of unit learning outcomes considering the bis unit is a commencing unit for students’ tertiary studies, their perceptions of amount of time spent studying the unit, the amount of time spent studying online and whether they believed they had achieved the unit learning outcomes becomes critically important. twenty-two percent of respondents spend no more than 4 hours studying each week, and possibly as many as 36% of students spend at least 6 hours studying each week. forty-eight percent of responding students spend more than 50% of their study time studying away from campus (where relevant) but online. seventy-five percent of responding students indicated that they dedicate at least 10% of their study time to activities online. eighty-three percent of responding students believe that they achieved the unit learning outcomes to at least some extent. the normal expectation is that students would be expected to spend 8-10 hours a week studying the unit. clearly, a significant number are below this expectation as shown in figure 2. australasian journal of information systems pye, holt, salzman, bellucci & lombardi 2015, vol. 19, research article engaging students in blended environments 9 figure 2. the amount of time students indicate they spent studying each week it would be expected that most students would master the unit learning outcomes fully or to a large extent. again, students’ perceptions fell below that benchmark as shown in figure 3. figure 3: perceived extent of learning outcomes met as a function of the amount of time students indicate that they spent studying implications for designing and operating blended learning environments to reinforce and challenge students to spend more, better and the appropriate amount of time on the unit to achieve its objectives is addressed at the end of the paper. in relation to cross-tabulating study time with the student engagement constructs significant differences were evident in the following: australasian journal of information systems pye, holt, salzman, bellucci & lombardi 2015, vol. 19, research article engaging students in blended environments 10 • those who indicate that they are studying 10 to 12 hours per week, also indicate that they will score less on questions themed in the social interaction construct (3.09 compared to 3.74, 3.88 and 4.05). • those who indicate that they are studying 10 to 12 hours, are more likely to score higher on the teaching construct items than those who indicate that they are only study for less than 3 hours per week (5.02 compared to 4.62). • those who indicate that they are studying 10 to 12 hours per week, are more likely to score higher on the staff contact items than those who indicate that they are only study for less than 3 hours per week (4.67 compared to 3.74). • in amount of time spent studying between onand off-campus students. that is, when considering both onand off-campus cohorts, off-campus students indicate that they are studying more. for example, 55% of the off-campus students in the sample indicate that that they are studying more than 6 hours, whereas only 30% of the on-campus students indicated that they were making this kind of effort. 4.2 perceived integration of cloud and located learning environments the proportion of the student population who believe that (1) ‘cloud and located learning enhanced each other’ and (2) ‘cloud and located learning were relevant to each other’ was 74.7%. similarly, the proportion of the student population who believe that (3) ‘the connection between located and cloud learning was not always clear’ and (4) ‘there was little or no connection between located and cloud learning’ was 25.3%. indicating classroom dimensions of complex blended learning environments is a complex and ongoing challenge. 4.3 perceived learning benefits fifty-three percent of the student population believed that the online system maximised their learning from on-campus classes (conversely, it could also be true that 20% of the student population did not believe that the online system maximised their learning from oncampus classes); 84.27% of the student population believed that the online system allowed them to organise their study time more effectively (conversely, 7% of the student population did not believe that clouddeakin allowed them to organise their study time more effectively); and 89.25% of the student population believed that the online environment allowed them to learn the subject materials more conveniently (conversely, 5% of the student population did not believe that the online environment allowed them to learn the subject materials more conveniently). the youngest age group (18 – 21 year olds) are more likely to think that the online system aided in maximising their learning from on-campus classes. suggesting, the older a student gets, the less likely they are to think that the online system maximises their on-campus (in class) learning experience. 4.4 online active learning eighty-eight percent of students believe that online content materials could be used to improve learning; 76% of students indicate that online materials make classroom (located) learning more meaningful; 91% indicate they would identify expected academic standards using the online environment; 59% indicate that online materials challenged them to learn; 29% reveal that they would never use a mobile device to access the online environment; and 53% state they would never use a mobile device to access and download resources from the online site. compared to any other age group, younger people are significantly less likely to be challenged with online learning materials. further, younger students (18-21 year olds) are less likely to think that staff used the online system to clarify what was required to do well. australasian journal of information systems pye, holt, salzman, bellucci & lombardi 2015, vol. 19, research article engaging students in blended environments 11 4.5 online social interaction seventy-eight percent of the respondents indicate that teaching staff participated in online discussions; 57% indicate it was easy to explain their ideas in online discussions; 49% indicate that they had helpful online discussions with other students; 28% reveal they met new people when using the online environment. these findings suggest a moderating of value attributed to traditional asynchronous discussion boards on lms-type environments. the majority of students surveyed indicated facebook was their preferred technology for social interactivity and collaborative purposes. reasons why facebook was preferred related to ready access to notification alerts and privacy of conversations. a cross-tabulation by mode of enrolment showed a significant difference as evident. that is, off-campus students are likely to score higher on questions themed in the social interaction construct (4.3 for off-campus compared to 3.7 for on-campus students). 4.6 online collaboration thirty-three percent of the on-campus student population used the online system with other students around the campus. conversely, 21% of the on-campus population did not use the online system with other students around the campus. twenty percent of the on-campus student population used the online system to work with other students outside the class. conversely, 37% of the on-campus population did not use the online system to work with other students outside the class. thirty-three percent of the population used the online system to do academic work with other students. conversely, 33% of the student population did not use the online system to do academic work with other students. thirty-nine percent of the student population used the online system to communicate with other students. conversely, it could also be true that as many as 32% of the student population did not use the online system to communicate with other students. a cross-tabulation by gender showed a significant difference was evident as shown in figure 4. figure 4: gender difference regarding the construct “collaboration” australasian journal of information systems pye, holt, salzman, bellucci & lombardi 2015, vol. 19, research article engaging students in blended environments 12 figure 4 illustrates that males are likely to, on average, score higher on questions themed in the collaboration construct (3.49 for males compared to 3.14 for females). a cross-tabulation by mode of enrolment also showed a significant difference was evident. that is, off-campus students are likely to score higher on questions themed in the collaboration construct (2.7 for off-campus compared to for on-campus students). compared to any other age group, younger people (18-21 year olds) studying on-campus are significantly more likely to use the online system with other students around the campus. 4.7 online teaching eighty-two percent of the student population believes that staff used the online environment to clarify what was required assessment-wise to do well; 82% of respondents believe that staff used the online environment in ways that improved overall teaching; 92% of respondents believe that academic staff made an effort to communicate with students online; 82% of responding students believe that academic staff used the online environment to provide students with extra assistance; 94% believe that teaching staff used the online site to tailor activities for students; and 69% believe that teaching staff used the online environment to discuss interesting issues. on the latter result, younger students (for example, those under the age of 30 years) are less likely to think that staff used the online system to discuss interesting issues. in cross-tabulating with mode of enrolment a significant difference was evident. that is, off campus students who responded are likely to score higher on questions themed in the teaching construct (5.05 for off-campus compared to 4.71 for on-campus students). offcampus students indicated that they received better clarity regarding what was required, compared to any other cohort in the unit. the results overall indicate that teaching staff used the online environment solidly to advance their teaching intentions, and that students in turn saw it as a major component of their blended learning experience. 4.8 online assessment eighty-five percent of the student population believes that assignments are easy to locate using the online environment; 90% of them believe tests are easy to locate online; 95% of responding students believes that assignments are easy to submit online, while five percent of responding students believe that assignments are not easy to submit online; 81% indicate that online assignment submission gives them more time to complete their work; 54% of responding students believe that online assignment feedback improved their understanding of the topic. (conversely 21% of respondents indicate that online assignment feedback did not improve their understanding of the topic.) 60% of responding students believe that online assignment feedback helped them prepare for subsequent assessable work. (twentyfour percent of responding students believe that online assignment feedback did not help them prepare for subsequent assessable work.) evidence suggests online assessment facilitates a more convenient and effective means of engaging with unit requirements. a technical glitch in the operation of the system did compromise the value students perceived relating to assignment feedback through online rubric use. this is instructive to the extent that technical failures in systems of great importance to student learning can adversely affect student engagement factors. 4.9 online academic relevance sixty-four percent of responding students believe that using the online system made study seem more relevant (conversely, 17% of responding students do not believe that using the online system makes study seem more relevant.); 75% of students believe that using the online system makes them feel like part of the university (conversely, 18% of students do not believe that using clouddeakin makes them feel like part of the university.); and 64% of them believe that using online materials within the online environment helped them put their australasian journal of information systems pye, holt, salzman, bellucci & lombardi 2015, vol. 19, research article engaging students in blended environments 13 study in a real-world context (conversely, 19% of students do not believe that using the online environment helps them put their study in a real-world context). a cross-tabulation by gender showed that a significant difference was evident as depicted in figure 5. figure 5: gender difference regarding the construct “academic online relevance” females are more likely to, on average, score higher on questions themed in the relevance construct (4.69 for females compared to 4.43 for males). compared to the 22 to 25 year olds, the 3136 year olds (n=12) are less likely to think that the online environment made study more relevant. 4.10 online contact with staff fifty-nine percent of responding students at least agree that they used the online system to contact staff. (conversely, 19% of students do not agree that they used the online system to contact academic staff.) sixty-seven percent of responding students found it easy to communicate with staff online. (conversely, 11% of responding students did not find it easy to communicate with staff online.) forty-six percent of participating students had individual contact with academic staff using the online system. (conversely, 26% of them did not have individual contact with academic staff using the online system.) not surprisingly, off-campus students are more likely to use the online system to contact staff, than the other campus-based cohorts. compared to the other three campus-based student cohorts, off-campus students found it easier to communicate with staff online, perhaps given their greater level of maturity. australasian journal of information systems pye, holt, salzman, bellucci & lombardi 2015, vol. 19, research article engaging students in blended environments 14 figure 6: mode of study difference regarding the construct “contact with staff” cross-tabulating with mode of enrolment revealed that a highly significant difference was evident (figure 6). off-campus students are likely to score significantly higher on questions themed in the contact with staff construct (4.79 for off-campus compared to 3.94 for oncampus students). 4.11 most and least effective aspects of the clouddeakin environment from an analysis of open-ended comments it was found: • the structure of the online environment was predominately well presented for easy navigation and as the unit’s only repository for materials. the only negatives were related to better notification of updated material and the use of rules/widgets to distinguish between read and unread material. • the use of the system to submit assignments online was preferred to paper-based submission since it was easy and there was no printing involved. • students found the online system useful for flexible access to unit material 24/7. this enabled students to structure their learning around busy schedules, classroom clashes and part-time/full-time work. • access to online lectures was mentioned mainly as a positive, since it allowed students to catch up on missed classes, although not all students appreciated the system recordings, noting bad lecture recording quality and a negative perspective of the online offering due to comparisons made with other online course mooc providers. • the effectiveness of tutorials were mixed based on the location of students (located/campus, cloud/online-based). located/on-campus students found tutorials quite effective, particularly in helping students to summarise their weekly classes and hence keep on top of the material. the lack of synchronous australasian journal of information systems pye, holt, salzman, bellucci & lombardi 2015, vol. 19, research article engaging students in blended environments 15 communication recordings of tutorials was a common complaint among cloud/online-based students, who noted that in previous classes cloud/onlinebased students have found these recordings very useful. • the majority of students found the discussion board on the lms as very useful in preparing for assignments and would often use this area to uncover ‘hidden tips’. a couple of students felt there was a delayed response to questions via the discussion boards and even after using the alert email system. one student found that not all the discussions he was alerted to were of interest, and hence was annoyed by the excessive number of email alerts. one student suggested questions should be moderated and deleted if they are more relevant to another thread to eliminate duplicated responses. 5 implications for blended learning design and practice 5.1 framing the design and practice of blended learning around learning outcomes and assurance of learning irrespective of the mode of delivery, the learning outcomes must be clearly established, and formal assessment directed at motivating students towards this achievement (biggs & tang, 2011). student engagement does in part relate to productive study time on tasks engendering quality learning and tasks must be aligned with the relevant learning outcomes. students self-reported notional average weekly study time may under-represent the actual time spent on all aspects of their study. it is possible students were only considering contact time within the lecture, tutorial, consultation or online in clouddeakin rather than the entire time spent studying, researching and reading. it might also be surmised that many students remain fundamentally results-driven focusing primarily on the next set of assessment activities to be undertaken and completed. hence, they may not be immediately concerned with or paid little attention to, the achievement of the unit learning outcomes. therefore, blended learning educators need to highlight learning expectations in the first lecture relating to the hours students should dedicate to unit study per week, to reasonably achieve the unit learning outcomes. in this study, it seems a significant number of students were not on average spending the expected hours to achieve the desired learning. this is not an uncommon finding in student study patterns in australian higher education. however, it is notable that the cloud (online/off-campus) student cohort perceived that they tended to work harder in the unit compared to their located (on-campus) student cohort counterparts. it is clear that a significant number of students spend a substantial amount of their study time studying online, irrespective of their mode of enrolment. the incentive not to physically attend located classes for those students enrolled in this mode is strong given the provision of all resources (including recorded lectures) online. the results, however, did not necessarily delineate separately between located and cloud learners. convenient and ready access to all available learning resources to all students irrespective of mode of enrolment creates an ongoing dilemma in the design and operation of such learning environments. for the predominately younger school-leaver located cohort, this form of remote engagement can be deleterious to the development of their communication, collaborative and networking skills. furthermore, external course accreditation requirements in business education can demand that evidence of learning thresholds is provided to assure the development and assessment of specific professional capabilities amongst identifiable student cohorts. the removal of certain incentives for withdrawing into individualistic online study like removing the provision of recorded live lectures to the located cohort can run up against faculty and institutional commitments to providing maximum options for flexible study. australasian journal of information systems pye, holt, salzman, bellucci & lombardi 2015, vol. 19, research article engaging students in blended environments 16 5.2 maximising the value proposition in designing blended learning environments from this analysis and discussion it is clear that it is a complicated challenge to maximise the teaching/learning value proposition in designing blended learning environments involving multi-campuses, multiple student cohorts and different types of student cohorts. the number and diversity of student needs and circumstances means that design and practice trade-offs are required. irrespective of the amount and range of learning resources provided online, the investigation has revealed that thoughtful online systems’ interface design including social media is essential in allowing students to easily locate and use the resources in achieving the desired learning. learning resources and assessment tasks must be well organised around the determined curriculum structure and available in formats which make them easy to study online or download and use offline on various types of desktop and mobile computer devices. the structural and discretionary variation in unit sites can lead to quite different student interpretations of the ease of usability of each unit site with which they engage. online learning system interfaces can be less web intuitive and facebook-like than the information systems that the students may be more comfortable and familiar with in their own lives. adopting a small set number of different templates that streamline the interface between units would develop familiarity and limit usability issues around locating relevant/specific learning resources and information, based on this investigation. the consistent and easy-touse interfaces of the major global massive open online course (mooc) platforms have much to commend in this respect compared to the more limited interface design options of proprietary learning management systems (lmss). the investigation revealed the more limited downloading and use of learning resources on mobile devices than may have been expected, and this finding should be set in the context of a course being studied on ict-enabled business information systems. a tentative conclusion reached, in this context, was that students perhaps use mobile devices to keep track of dates and discussions, rather than downloading resources for offline learning. in relation to online communication and moderation practices, the study highlighted the need for a more active approach to the moderation of online discussions. in the case of this course’s online learning system, the discussion postings were not easily 'search-able' and the message/posting settings can be seen as restrictive and counter-intuitive, requiring time for students to develop a familiarity with the online discussion features. most students appear to use the online discussion forums as they would facebook and post a message expecting a direct response from the lecturer, rather than reviewing the other postings first. there seems to be a challenge in generating student collaboration and collegiality online within traditional asynchronous discussion boards. initially it takes time for the students to see the benefit of using discussion boards given that to them they must seem like an old technology solution to communication demands. there now appears to be a trend in online distance education to move more from asynchronous online discussion forums to synchronous forms of discussion like etutorials and eseminars. after the completion of this study in this course, a 'pilot' trial consisting of using three synchronous communication sessions with cloud students was only seen to have been somewhat successful. these gave the remote students and opportunity to speak directly with the unit chair in three, one hour long sessions. the use of such synchronous communication sessions requires far greater development and investigation. while the provision of such direct opportunities for teaching staff-student interactions might engender stronger student engagement they might also carry the cost of reduced perceived student study flexibility to the detriment of their motivation to continue studying. in this study, there was some evidence that older students are perhaps more set in their learning styles and expectations and are less adaptable or accepting of new learning environments than younger students. this finding should not be discounted, although, on balance, adaptability and change should continue to be encouraged generally throughout the cohorts, australasian journal of information systems pye, holt, salzman, bellucci & lombardi 2015, vol. 19, research article engaging students in blended environments 17 particularly off-campus and mature-age students, in order to equip all students with the skills required for contemporary collaborative workplace practices. in addition to institutionally supported lms and other synchronous communication tools, evidence from previous studies suggests the increasing use of social media applications in supporting peer exchanges and cooperative endeavors (evans 2014; kelm 2011; graham 2014). evans (2014) found a strong causal relationship between twitter use and student engagement in university activities and class based discussions, and similarly graham (2014) found business students regarded the social dimension to learning and discussion among their team members made projects more enjoyable. the reasons why facebook was preferred over other platforms, particularly compared to our lms discussion board, was related to ready access of notifications and the privacy of conversations. ready access to notifications is somewhat related to the evan’s survey item on their empirical study: “twitter is a good way to post things as soon as i think of them” (evan, 2014, p.912), which positively inferred, together with other factors, strong use of social media for student engagement. the second finding related to the privacy of conversations is quite understandable among first year students who may be shy or embarrassed to share with unknown people in the class. anecdotal evidence suggests the increasing use of social media applications in supporting peer exchanges and cooperative endeavors. the unit investigated is about information systems and the message to students is around ‘practising what you preach’ and utilising ict as the basis of communication via discussions, email, the microsoft lync synchronous discussion tool etc. the unit also has an active twitter feed since its implementation and, interestingly, a facebook page for the unit was implemented more recently as a means of providing an additional channel of communicating class news and information more generally and directly into a student facebook feed if they choose to 'like' the page. however, the institutional online learning system remains the default means of communicating all teaching and learning materials, and communications to all student cohorts with both social media communication channels accessible from the lms unit home page. again, whatever the purported benefits of social media use for enhancing student engagement, more trialing and evaluation work is required to evidence learning gains as related to large mainstream blended learning developments. finally, there is one crucial element of blended learning practice that cannot necessarily be standardised and that is the behaviours of teaching staff who operate in their own local campus-based contexts and who might contribute in different ways to practice in the online environment. student engagement and satisfaction may be dependent on the popularity and status of the teaching staff member on the respective campus, or servicing the needs of the cloud student cohort, and the level of attention to their needs that they perceive. having a located, constant teaching staff member being the perceived leader on each campus, and for the cloud cohort, can alleviate concerns around inconsistent service provision. so too can an inclusive leadership approach giving all key teaching staff an opportunity to create and contribute to the major learning resources made available to all student cohorts in a course. in whatever circumstance of this nature, it is important for the teaching team to maintain a strict focus on a prompt and consistent use and response to student questions online. maintaining a consistent message is critical, which requires a small number of designated teaching staff members to be involved only in such activities. otherwise, contradictory messages can abound that only serves to confuse students. 6 future developments the research continues to investigate the effects of design interventions on student engagement on particular subjects offered at different points of time over the two-year project period. the research will also report on more generalised blended learning design and practice activities which contribute to higher student engagement irrespective of the discipline, year of offer, and subject-based designs and practices in the business education domains. a specific point of interest will relate to a further and deeper investigation of any australasian journal of information systems pye, holt, salzman, bellucci & lombardi 2015, vol. 19, research article engaging students in blended environments 18 identified differences in patterns of student engagement based on key demographic and background factors of cohorts surveyed. 7 conclusion multi-modal, multi-campus, diverse audience blended learning designs and practices is a hallmark of australian higher education in the realm of business education. these types of blended learning environments raise pressing challenges in providing all students, irrespective of location, mode of delivery, background and motivation, will the best possible opportunity of mastering unit and course-wide learning outcomes. assuring the mastery of desired learning outcomes is now central to university strategic curriculum design and course enhancement efforts. it is also a growing imperative in meeting external regulatory and professional accreditation requirements. designing for student engagement provides a theoretically and conceptually grounded approach to enhancing and assuring student learning. the study has trialled the use of an online student engagement survey, with a dual focus on cloud and located learning experiences, and the relationship between the two. the findings have highlighted the importance of lms interface designs to maximise the use and value of learning resources, the ways current lms communication features are used by students, the perceived value of assessment functions operating effectively in online systems, and the emergence of alternative forms of communication and collaboration, which must be considered through the prism of social media. the balancing act still remains the development of online environments that cater for the needs of students who rely almost entirely on them to achieve the required learning outcomes, and those who best benefit from dual engagement with both cloud and located learning opportunities for learning achievement. references acer (2014): http://www.acer.edu.au/ausse biggs, j. & tang, c. (2011). teaching for quality learning at university. what the student does. buckingham, england: society for research into higher education & open university press. bonk, c.j. & graham, c.r. (2006). (eds.). handbook of blended learning: global perspectives, local designs. san francisco, ca: pfeiffer. bonk, c.j. & zhang, k. (2008). empowering online learning: 100 + activities for reading, reflecting, displaying, & doing. san francisco: john wiley & sons. boud, d. (2010). student assessment for learning in and after courses. final report for senior fellowship. australian learning and teaching council (altc): http://www.olt.gov.au/resource-student-assessment-learning-and-after-courses-uts2010 boud, d. & molloy, e. (2013). (eds.). feedback in higher and professional education understanding it and doing it well. london: routledge. coates, h. (2006). student engagement in campus-based and online education: university connections. routledge: london. coates, h. (2010). development of the australasian survey of student engagement (ausse). higher education, 60:1-17. cull, w. l., o’connor, k. g., sharp, s. & tang, suk-fong s. (2005). response rates and response bias for 50 surveys of pediatricians. health services research, 40(1):213-226. daniel, j.s. (1996). mega-universities and knowledge media: technology strategies for higher education. london: kogan page. dey, e. l. (1997). working with low survey response rates: the efficacy of weighting adjustments. research in higher education, 38(2), 215-227. australasian journal of information systems pye, holt, salzman, bellucci & lombardi 2015, vol. 19, research article engaging students in blended environments 19 evans, m. (2014). twitter for teaching: can social media be used to enhance the process of learning? british journal of educational technology, 45(5):902–915. garrison, d.r. & anderson, t. (2003). e-learning in the 21st century. a framework for research and practice. london: routledge falmer. garrison, d.r. & vaughan, n.d. (2008). blended learning in higher education: framework, principles and guidelines. san francisco: jossey-bass. george-walker, l. de., hafeez-baig, a., gururajan, r. & danaher, p.a. (2010). experiences and perceptions of learner engagement in blended learning environments: the case of an australian university in y. inoue (ed.). cases on online and blended learning technologies in higher education: concepts and practices. information science reference, 23-43. graham, c.r. & robison, r. (2007). realizing the transformational potential of blended learning: comparing cases of transforming blends and enhancing blends in higher education. in a.g. picciano & c.d. dziuban (eds.) blended learning: research perspectives. the sloan consortium, 84-110. graham, m. (2014). social media as a tool for increased student participation and engagement outside the classroom in higher education. journal of perspectives in applied academic practice, 2(3):16–24. holt, d., segrave, s. & cybulski, j. (2011). (eds.) professional education using esimulations: benefits of blended learning design. hershey, us: igi global. johnson, t. p., o'rourke, d., burris, j., & owens, l. (2002). culture and survey nonresponse. in r. m. groves, d. a. dillman, j. l. eltinge & r. j. a. little (eds.), survey nonresponse. new york: wiley, 55-69. isbn: 978-0-471-39627-7 kaleta, r., skibba, k. & joosten, t. (2007). discovering, designing, and delivering hybrid courses. in a.g. picciano & c.d. dziuban (eds.) blended learning: research perspectives. the sloan consortium, 112-43. kelm, o. r. (2011). social media: it’s what students do. business communication quarterly, 74(4):505-520 krause, k-l & coates, h. (2008). students’ engagement in first-year university, assessment & evaluation in higher education, 33(5):493-505. kristin, k. (2013). teaching on the education frontier instructional strategies for online and blended classrooms. jossey-bass: san francisco. kuh, g.d. (2003). what we’re learning about student engagement from nsse. change, 35(2):24-31. kuh, g.d., kinzie, j., schuh, j.h., whitt, e.j., & associates (2005). student success in college: creating conditions that matter. san francisco: jossey-bass. latcham, c. & jung, i. (2010). distance and blended learning in asia. open & flexible learning series. london: routledge. nicol (2010). re-engineering assessment practices in higher education: http://www.reap.ac.uk/home.aspx lawson, r. (2015). curriculum design for assuring learning – leading the way. final report for senior fellowship. office for learning and teaching (olt): http://www.olt.gov.au/resource-curriculum-design-assuring-learning-leading-way2015 oliver, b. (2013). graduate attributes as a focus for institution-wide curriculum renewal: innovations and challenges. higher education research & development, 32(3):45063. australasian journal of information systems pye, holt, salzman, bellucci & lombardi 2015, vol. 19, research article engaging students in blended environments 20 salmon, g. (2011). e-moderating. the key to teaching and learning online. 3rd edition. new york: routledge. sax, l.j., gilmartin, s.k. & bryant, a.n. (august 2003) assessing response rates and nonresponse bias in web and paper surveys. research in higher education, 44(4):409-432. scott, g. (2006). accessing the student voice using cequery to identify what retains students and promotes engagement in productive learning in australian higher education. barton, australian capital territory: department of education, science and training. shea, p. (2007). towards a conceptual framework for learning in blended environments. in a.g. picciano & c.d. dziuban (eds.) blended learning: research perspectives. the sloan consortium, 19-35. special issue, distance education (2014). moocs: emerging research, 35(2), august. rogelberg, s. g. & luong, a. (1998). nonresponse to mailed surveys: a review and guide. current directions in psychological science, 7(2):60-65, april. university experience survey (ues): https://www.education.gov.au/university-experiencesurvey copyright: © 2015 pye, holt, salzman, bellucci & lombardi. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems bellucci, dahrmesana, nguyen & calache 2017, vol 21, research on health information systems effectiveness of sms reminders 1 the effectiveness of sms reminders and the impact of patient characteristics on missed appointments in a public dental outpatient clinic emilia bellucci deakin business school deakin university emilia.bellucci@deakin.edu.au lasitha dharmesena deakin business school deakin university lemai nguyen deakin business school deakin university hanny calache school of health and social development deakin university abstract this paper reports on the failure to attend (fta) rate of appointments as well as patients following the implementation of sms reminders in a public dental outpatient service. given the ineffectiveness of the intervention and a highly representative patient’s profile, this paper identifies the demographic characteristics of patients who miss all of their appointments. data on appointment attendance, patient demographics and dental service type was collected over a time period of 46 consecutive months. using descriptive and inferential statistics (chisquare, two sample tests and marascuilo procedure) we found the sms intervention was ineffective in reducing the fta rates. further, patients associated with high rates of nonattendance exhibited one or more of the following characteristics: male; age 26 – 44; nonconcession card holders; a person of indigenous, local, asian or african descent, and of refugee status, persons living in low socio-economic areas; and appointments in general care and student clinics. whilst the literature overwhelmingly attributes sms reminders to improving the attendance rate of patients in outpatient clinics, our contradictory findings suggest a more targeted approach in settings whose patients exhibit strong characteristics associated with non-attendance. keywords: non-attendance; sms reminder service; public outpatient clinics; dental health services; patient demographic and appointment characteristics. 1 introduction patients failing to attend appointments is a very prevalent problem for outpatient clinics around the world. non-attendance can result in the underutilization of resources and extended waiting lists (geraghty et al. 2008), contribute to the loss of clinician time (bos, hoogstraten & prahl-andersen 2005) and delayed access to care for other patients (stubbs et al. 2012). patient failure to attend appointments also results in lower healthcare outcomes for patients, often due to missed opportunities to continuous healthcare (cashman et al. 2004). the financial expense of missed appointments can be quite significant and can increase the cost of the service (stubbs et al. 2012; milne 2010; perron et al. 2010). when a patient fails to attend an appointment, the outpatient service must still pay for the room, equipment, the salaries of clinicians and administrative staff for setting up the appointment. in addition, missed appointments contribute to already prolonged waiting periods for primary care australasian journal of information systems bellucci, dahrmesana, nguyen & calache 2017, vol 21, research on health information systems effectiveness of sms reminders 2 appointments and surgery. these costs are particularly concerning for outpatient clinics in the public health system as these measures are often used to calculate future funding. in australia, public dental services are funded by the government. access to dental services are usually restricted to adults in the public welfare system and to children. during 2007 – 2008, the australian government spent approximately $6.1 billion (6.2% of the total health expenditure) on dental health services (dental health services victoria 2011). given this substantial investment, it is imperative we attempt to limit opportunities for wasted resources such as those attributed to unattended appointments. in addition, research over the last 20 years has drawn significant links between oral health and serious health conditions including chronic diseases such as cardiovascular disease, diabetes, respiratory diseases, stroke, kidney disease, dementia and other serious aliments (dental health services victoria 2011). therefore, ensuring the public maintain good oral health is not only important to help prolong life expectancies and to protect good quality of life, but it may have financial implications for increased expenditure across the whole health services network. whilst failure to attend fta (also referred to as non-attendance rates, missed appointments, did not attend dna or no-shows) vary significantly within and between outpatient specializations, the literature overwhelmingly finds the intervention of sms reminders as major contributors to higher attendance rates. there are however, instances where sms reminders have had no effect. in this article, we refute the universal application of sms reminders, with the view that if the technology does not arrest the non-attendance rate, its’ adoption further impinges on limited financial resources. this study reports on the non-attendance rates of patients at a major dental outpatient clinic co-located on the same site as a public dental hospital. it is located within three kilometers of the cbd. the dental outpatient service provides very affordable dental services to persons on low incomes and to the socially marginalized that is, to those who cannot afford private dental care. further, the clinic is a key provider of dental services to the victorian public. given the highly representative demographic profile of its patients, this study also investigated a patient non-attendance score and whether a profile of characteristics common in patients who fail to attend appointments can be proposed. 2 background 2.1 literature review interventions to tackle non-attendance have included postal, telephone and electronic reminders such as sms. outpatient clinics often use sms to remind patients of their appointments, which are sent close to the date and time of the appointment. sms text messages are found to be more effective or as effective as other forms of intervention (milne 2010; perron et al. 2010; chen et al. 2008; leong et al. 2006; hasvold & wootton 2011). sms reminders are preferred due to lower costs and easy delivery of the intervention (koshy, car & majeed 2008; mciver et al. 2015) in comparison telephone calls and mailed letters. sms is a common mode of communication, particularly as the prevalence of mobile phone ownership is high in australia, with penetration rates of 81% among the australian public (deliotte 2014). the literature overwhelmingly supports the adoption of sms reminders to improve attendance rates at medical clinics around the world (hasvold &wootton 2011; guy et al. 2012; boksmati et al. 2016). a recently published meta-analysis and systematic review by boksmati et al. (2016) shows an increased odds ratio compared to a previous meta-analysis of sms reminder effectiveness in appointment adherence by guy et al. (2012). table 1 lists papers reporting on fta rates by comparing patient cohorts sent sms reminders (sms) and those not sent a sms reminder (control) in medical outpatient settings. given some studies suggest an analysis of patient characteristics could help to increase the effectiveness of appointment reminders (cashman et al. 2004; perron et al. 2010; norton et al. 2014), the table also distinguishes any patient or appointment characteristics analysed and whether an association between such characteristics and the effectiveness of sms reminders was reported. australasian journal of information systems bellucci, dahrmesana, nguyen & calache 2017, vol 21, research on health information systems effectiveness of sms reminders 3 the association between non-attendance and visit type (such as new or follow up appointments) has been confirmed by five studies; whilst the other characteristics reflect mixed findings. in addition to the studies listed in the table, other studies analysing patient demographics of gender and age (irrespective of a reminder intervention) show an association between these demographic characteristics and general non-attendance (cashman et al. 2004; perron et al. 2010). other characteristics mentioned in the non-attendance literature include associations with socio-economic group (cashman et al. 2004; lalloo & mcdonald 2010; mcinnes et al. 2014) and asylum seeker status (perron et al. 2010). from table 1, we can see the majority of articles report sms reminders are helpful in reducing the non-attendance rate of patients, with a small number of studies reporting sms reminders have no effect, suggesting the universal application of sms as an effective measure to counter non-attendance should not be assumed. equally, this table shows patient and appointment characteristics may help to predict the success of an intervention such as sms reminders on patient non-attendance. australasian journal of information systems bellucci, dahrmesana, nguyen & calache 2017, vol 21, research on health information systems effectiveness of sms reminders 4 author, year out-patient clinic country sms successful? non-attendance/attendance rates characteristics associated with non-attendance within sms groups adankin et al. 2014 post natal nigeria yes non-attendance rate = 42.85 (control); 21.3% (sms). none tested bos et al 2005 orthodontic netherlands no change change did not reach statistical significance age, gender, insurance type were not associated with non-attendance brannan et al. 2011 ophthalmology uk yes non-attendance rate = 12% (control); 5.5 % (sms) none tested chen et al. 2008 hospital health promotion china yes attendance rate: 80.5% (control); 87.5% (sms) none tested da costa et al 2010 general practice brazil yes non-attendance rate = 25.57% (control); 19.42% (sms) none tested downer et al. 2005 pediatric hospital australia yes non-attendance rate = 23.4 % (control); 14.2 % (sms) none tested downer et al. 2006 pediatric hospital australia yes attendance rate: 80.5% (control); 90.2% (sms). non-attendance rate fell by 5.5% in new appointments and 11.6% in follow up appointments visit type were associated with nonattendance fairhurst & sheikh, 2008 primary care scotland no change change did not reach statistical significance none tested farmer et al. 2013 sexual health and hiv uk yes and no overall non-attendance rate = 28% (control); 24% (sms). male clinic nonattendance rate = 28% (control); 18 (sms). no change in hiv clinic. gender and clinic type were associated with non-attendance foley, 2009 pediatric dental scotland yes and no overall non-attendance rate: 23.9% (control); 10.4% (sms). clinic type was associated with nonattendance geraghty et al 2008 ent ireland yes non-attendance (mean) rate: 33.6% (control); 22% (sms). none tested koshy et al 2008 ophthalmology uk yes non-attendance rates = 18.1% (control); 11.2% (sms) none tested leong et al 2006 primary care malaysia yes attendance rates: 48.1% (control); 59.6% (sms) none tested liew et al. 2009 primary care malaysia yes non-attendance rate = 23.0 (control); 15.6 (sms) none tested mcinnes et al. 2014 urban veterans primary care us yes 19% downward change in sms group, compared to non-sms group none tested australasian journal of information systems bellucci, dahrmesana, nguyen & calache 2017, vol 21, research on health information systems effectiveness of sms reminders 5 author, year out-patient clinic country sms successful? non-attendance/attendance rates characteristics associated with non-attendance within sms groups milne et al. 2006 hospital outpatient scotland yes and no lower attendance rates for new appointments. no change in follow up appointments visit type was associated with non attendance milne, 2010 hospital scotland yes new appointments non-attendance rate = 6.8 (control); 3.7 (sms). return appointments non-attendance rate = 5.2 (control); 4.0 (sms). visit type was associated with nonattendance narring et al. 2013 hospital youth switzerland no change change did not reach statistical significance none tested norton et al. 2014 hiv us no change change did not reach statistical significance none tested parikh et al. 2010 university us yes non-attendance rate = 23.1% (control); 17.3% (sms) age, visit type were associated with nonattendance perry, 2011 dental scotland yes non-attendance rate = 31% (control); 14% (sms) none tested prasad & arnand, 2012 dental university india yes rate of attendance: 35.5% (control); 79.2% (sms) none tested sims et al. 2012 mental health england yes attendance rates were significantly higher for the 2009 and 2010 samples (sms) than for the 2008 sample (control) delivery time was not associated with non-attendance. taylor et al. 2012 physical therapy not stated yes non-attendance rate = 16% (control); 11% (sms) age, clinic type, visit type and day was associated with non-attendance youssef 2013 hospital saudi arabia yes non-attendance rate = 39.8% (control); 26.3% (sms) age, gender, nationality were not associated with non-attendance. first appointment was found associated with non-attendance. table 1: empirical studies assessing non-attendance /attendances rate in sms reminder trials. australasian journal of information systems bellucci, dahrmesana, nguyen & calache 2017, vol 21, research on health information systems effectiveness of sms reminders 6 2.2 setting the dental outpatient clinic used in our study is located on the same site as a public dental hospital located in the melbourne city centre. in the clinic, all patient appointment records are captured electronically using the titanium patient management system of spark dental technology (https://www.spark-dental.com/). the sms reminder system was programmed and integrated into the titanium system by the hospital’s internal it department. sms reminders are sent in english one day prior to the appointment. the text reminds patients of the appointment time and date and contains instructions on how to cancel the appointment if required: “reminder [firstname]: appointment at [clinicname] on [appointmentdate] at [appointmenttime]. call [clinicphone] only if you cannot attend.” in australia, dental care services are provided by the public and private sector, with the large majority of dentists working in the private sector. to be eligible for public dental care, certain eligibility criteria must be met. for adults to be eligible, they must be recipients of either a healthcare card or pensioner concession cards. these cards are offered to residents who are on low incomes and to refugees, asylum seekers and all indigenous (aboriginal and torres strait islander) persons. the benefits of eligibility are affordable dental care, with most cardholders paying a small co-payment for services. the value of the co-payment varies according to whether patients attend a general service or a specialist service. in addition, if healthcare cardholders are being treated by dental students, then the treatment is free. the hospital also runs an emergency service for all persons. 3 objective the demographic of patients attending the clinic is typical of persons who cannot afford private dental care. with a large proportion of the dental outpatient services’ patients falling into this group, it is important to investigate possible associations between patient demographics and non-attendance in addition to assessing the effectiveness of sms reminders. in light of the literature review, the objective of the paper is two-fold: 1) to determine the effectiveness of an sms reminder system to decrease the fta rate of a public dental outpatient clinic. 2) to develop a patient profile with characteristics which are significantly associated with non-attendance. 4 materials and methods prior to the commencement of the study human research ethics approvals for use of secondary de-identified data were obtained from the health service and affiliated university. patient appointment data was obtained for a 46-month period from september 2009 to june 2013. during this time, not all appointments were sent sms reminders, with text reminders sent according to clinic type, part of a trial rotation of dental rooms. over the time period, all dental rooms included in the study were subjected to the trial. records were automatically updated by a “1” in the “sms sent” field if the sms message was sent successfully to the registered mobile number. data records included patient numbers, the date and time of the appointment, patient characteristics and the dental service type. data was retrieved from the one site. demographic factors collected include gender, country region, suburb region, age groups, indigenous status, refugee (and asylum seeker) status, concession card holders and dental service type. the data attached to each appointment was extracted from the system and analysed. adults (persons aged above 18) were included in the study and no identifying data such as names, addresses or phone numbers was collected. patients were separated into age groups australasian journal of information systems bellucci, dahrmesana, nguyen & calache 2017, vol 21, research on health information systems effectiveness of sms reminders 7 representing distinct adult stages: young adult (18 – 25), adult (26 – 44), mature adult (45 – 65), seniors (65 +). our groupings are consistent with cashman et al (2004). country regions (indicating a patient’s identifying cultural background) were organized by continent, the only exception was australasia where local patients (predominately identifying as australian or new zealanders) were separated from the rest of australasia, such as the pacific islands. suburb regions were aggregated according to regions set by the victorian electoral office.1 dental service types were grouped into five categories: general care, special needs, specialist care, surgery and student clinics. general dental care refers to check ups and other minor procedures such as teeth cleaning associated with primary care. special needs appointments are for patients with disabilities. student clinics refer to student clinicians treating patients. specialist care clinics refer to procedures such as prosthodontics, periodontics and orthodontics, while surgery refers to theatre procedures. table 2 provides a breakdown of demographic factors of patients in our sample. the number of appointments made during this time was 862,745 and of these, 139,924 appointments had a sms reminder sent to the patient’s mobile phone. the number of patients sent sms reminders was 34415. demographic factor levels missing data gender male (40%), female (59%) 1% identifying cultural background africa (3.39%), america (0.15%), asia (12.39%), australasia (0.33%), australia & new zealand (56.14%), europe (8.26%), north america (0.32%), south america (0.46%) 18.55% suburb region eastern metropolitan (9.72%), eastern rural victoria (8.32%), northern metropolitan (28.59%), northern rural victoria (5.73%), south eastern metropolitan (10.71%), southern metropolitan (7.78%), western metropolitan (21.91%), western rural victoria (6.89%) 0.35% age groups 18-25yrs (25%), 26-44yrs (40%), 45-65yrs (24%), 65+yrs (11%) indigenous indigenous (2.1%), non-indigenous (83.5%) 14.4% refugee (including asylum seeker status) yes (0.2%), no (99.8%) concession card type veterans pensioner concession card (0.03%), health care card (55.97%), no card (3.93%), pensioner concession card (40.08%) dental service type general care (18.13%), special needs (3.34%), specialist care (47.85%), surgery (13.65%), student clinics (17.03%) 0.01% table 2: demographic factors of patients (number of patients=34415) sent an sms and frequency distributions 1 https://www.vec.vic.gov.au/electoralboundaries/stateregionprofiles.html australasian journal of information systems bellucci, dahrmesana, nguyen & calache 2017, vol 21, research on health information systems effectiveness of sms reminders 8 the collected data were analysed using the minitab (version 17) statistical software. all assumptions of statistical tests were examined and met before the analysis. the analysis was conducted at the significance level of 5%. the next section presents the findings in response to the two research objectives formulated above. 5 results 5.1 effectiveness of sms reminders to decrease fta rates to answer the question of whether sms reminders improved patient attendances, we used the appointment as the unit of analysis to determine: a) the non-attendance rate of the appointment before (πpre) and after the sms implementation (πpost) b) following the implementation, the non-attendance rate of the appointment population sent an sms reminder (πpost, sms) against the population not sent an sms reminder (πpost, sms1). table 3 reports a) non-attendance rate of the scheduled appointments before sms implementation from sep 2007 to oct 2009 and after sms implementation from oct 2009 to sep 2011. to determine whether there is a difference between the two attendance rates (πpre i.e. failure to attend – yes % before sms implementation and πpost i.e. failure to attend – yes % after sms implementation), we formulated the following hypotheses: h0: πpre ≥ πpost h1: πpre < πpost. the p-value was 0.016 smaller than 5%, leading us to conclude that the data provide statistically significant evidence to infer that the non-attendance rate before sms implementation is lower than the one after sms implementation. prior sms reminder service post sms reminder service did not attend 28,848 (9.3%) 31,456 (9.5%) total appointments 309,147 331,533 table 3: summary of non-attendance rates – yes % prior to and after introducing sms reminder service table 4 reports sample statistics in regards to b) the non-attendance rate of the appointment population sent an sms reminder (π post, sms) against the population not sent an sms reminder (π post, sms1). appointments where sms was sent (sms) appointments where sms was not sent (sms1) totals attended 125122 654297 779419 did not attend 14802 68524 83326 total 139924 722821 862745 % nonattendance 10.58% 9.48% 9.66% table 4. proportion of non-attendance vs attended appointments (sent an sms vs not sent an sms australasian journal of information systems bellucci, dahrmesana, nguyen & calache 2017, vol 21, research on health information systems effectiveness of sms reminders 9 the following hypotheses were formulated: h0: πpost, sms ≤ πpost, sms1 h1: πpost, sms > πpost, sms1 we found statistical evidence (p value = 0.000) for the ineffectiveness of the sms reminder service in reducing non-attendance. 95% confidence interval for difference (πpost, sms πpost, sms1): 0.93%, 1.28%. given this result, we explored explanations from an analysis of patients’ characteristics highlighted in the literature and available in the data set. this analysis provides impetus for the second research objective. 5.2 patient profile associated with high non-attendance the second research question asked to explore the potential relationship between patient nonattendance and their demographics. the patient was used as the unit of analysis. step 1: an analysis was conducted to explore the relationship between the number of missed appointments and the scheduled appointments per patient. to enable analysis on a patient level, we calculated a fta score2 for each patient in the dataset as follows: patient fta score = (number of missed appointments /number of appointments scheduled)*100%. table 5 shows patient fta scores grouped into non-overlapping intervals and the corresponding frequency distribution used to examine the relationship between missed appointments and demographic factors. findings in table 5 reveal amongst the patients who received an sms reminder, 69% of patients attended all of their scheduled appointments (fta score = 0 %). about 9.5% of patients failed to attend all of their appointments (fta score = 100%). fta score number of patients % patients 100% 3281 9.5336% 90-99% 3 0.0087% 80-89% 76 0.2208% 70-79% 213 0.6189% 60-69% 62 0.1802% 50-59% 1891 5.4947% 40-49% 1377 4.0012% 30-39% 1053 3.0597% 20-29% 1629 4.7334% 10-19% 1057 3.0713% 0% 23773 69.0774% 34415 100.0000% table 5: percentage distribution of fta score per patient during the 46-month study period, the number of appointments scheduled ranged from 1 to 72, with the modal number of appointments scheduled per patient being 1 (38.7%) and the arithmetic mean being 4. covering 77% of the patients in the sample had no more than 4 2 a “fta score” is a calculation assessing a patient’s level of non-attendance at multiple appointments. australasian journal of information systems bellucci, dahrmesana, nguyen & calache 2017, vol 21, research on health information systems effectiveness of sms reminders 10 appointment, 90% had nine or less appointments, 94.4% had 14 or less appointments and about 97% had 21 or less appointments. ()* fta score table 6: percentage distribution of patients across missed appointments and number of scheduled appointments per patient percentages in table 6 were calculated based on actual data, for example in the cell marked $, 0.3% of patients do not attend 8-9 appointments out of ten appointments. as per table 6, 38.68% of the patients made a single appointment, and of these, 77.4% attended and 22.6% missed the appointment. a number of observations can be made. firstly, the percentage of patients whose number of missed appointments is none (0) i.e. (fta = 0%), decreases as the number of scheduled appointments increases. from this observation we can conclude that patients are increasingly failing to attend all their scheduled appointments (fta score=0%) as they are given more appointments to attend. secondly, the percentage of ‘never show up’ patients (fta score =100%) declines considerably as the number of appointments per patient increases, suggesting some patients will try not to miss all their scheduled appointments when they are given more appointments to attend. moreover, regardless of the total number of visits scheduled per patient, patient percentage for a single missed dental appointment is concentrated around 25%. for a given number of scheduled appointments per patient, patient percentages for multiple missed dental appointments scheduled (fta score = 20%, 30%, … 100%) decrease significantly as the number of scheduled appointments increase. thus, appointment keeping appears to be related to the number of dental appointments scheduled. percent (%) 0 1 2 3 4-6 7-9 10-12 13-14 1 77.4% (fta=0%) 22.6% (fta=100%) 13313 38.68 2 73.2% (0%) 23.2% 3.6% (100%) 6738 19.58 3 70.1% (0%) 24.30% 4.90% 0.7% (100%) 4095 11.9 4 60.9% (0%) 27.5% 9.0% 2.4% 0.2% (100%) 2383 6.92 5 59.8% (0%) 26.4% 10.1% 2.9% 0.7% (≥90%) 1471 4.27 6 59.1% (0%) 25.4% 10.6% 3.8% 1% (≥75%) 1069 3.11 7 59.5% (0%) 23.8% 11.8% 2.9% 2% (≥57%) 790 2.3 8 57.7% (0%) 25.2% 10.4% 3.6% 3.1% (≥50%) 636 1.85 9 54.3% (0%) 26.0% 11.5% 5.5% 2.7% (≥44%) 477 1.39 10 53.2% (0%) 26.7% 9.9% 7.2% 2.7% 0.3% (≥80%)$ 374 1.09 11-20 51.3% (0%) 24.0% 12.7% 6.2% 5.1% 0.6% 0.1% 2047 5.95 21-30 38.1% (0%) 23.1% 15.7% 7.9% 11.9% 2.4% 0.7% 0.1% 706 2.05 31-40 30.6% (0%) 23.1% 15.7% 8.7% 16.5% 3.7% 1.2% 0.4% 242 0.7 41-50 16.4% (0%) 29.5% 21.3% 3.3% 18.0% 6.6% 1.6% 3.3% 61 0.18 51-60 33.3% (0%) 25.0% 8.3% 0.0% 25.0% 8.3% 0.0% 0.0% 12 0.03 72 100% 1 0 34415 100 number of scheduled appointments per person number of missed appointments patient total australasian journal of information systems bellucci, dahrmesana, nguyen & calache 2017, vol 21, research on health information systems effectiveness of sms reminders 11 furthermore, we examined the relationships between number of ‘no show’ appointments and number of 'no show' appointments by those that never show up. year % "no show" appts. % patients "never show up" (fta = 100%) % no show appts. by those that never show up3 sep dec 2009 (3 months) 10.32% 8.30% 43.02% jan dec 2010 10.16% 8.60% 29.32% jan dec 2011 10.61% 9.08% 31.35% jan dec 2012 10.56% 9.85% 36.60% jan jun 2013 (6 months) 11.55% 10.85% 48.86% table 7: comparison between ‘no show appointments’ % and “never show up patients’% table 7 demonstrates a consistent observation that at least 30% of missed appointments are by patients who never attend their appointments. this finding suggests further analysis into the demographic profile of patients who are at high risk of not making all scheduled appointments, patient fta score = 100%. step 2: a chi-square test (table 8) was conducted to detect associations between the patient fta score and demographic factors i.e. to test whether different patient groups in terms of a demographic factor have the same proportion of fta score. demographics factors include gender, identifying cultural background, suburb region, age groups, indigenous, refugee or asylum seeker status, concession card type, and dental service type. the two hypotheses were formulated as: h0: there is no association between fta score levels and the demographic factor (e.g. gender), i.e. they are independent. h1: there is an association between fta score levels and the demographic factor (e.g. gender), i.e. they are dependent. variable p-value gender 0.011 identifying cultural background 0.000 suburb region 0.000 age in years 0.000 indigenous 0.000 refugee or asylum seeker status 0.000 concession card type 0.000 dental service type 0.000 table 8 association between fta score and demographic factors: results of pearson chisquare test 3 (% no show appointments by those that never show up =number of 'no show' appointments by those that never show up / number of "no show" appts.*100%). australasian journal of information systems bellucci, dahrmesana, nguyen & calache 2017, vol 21, research on health information systems effectiveness of sms reminders 12 results evident in table 8 revealed that h0 was rejected for all demographic factors, including p-value for all variables 0.00 as well as for gender (p-value of 0.011). therefore, all associations were statistically significant at 5% level of significance (p-value < 0.05). rejecting the null hypothesis of equality allows us to conclude that not all patient groups (grouped by demographic characteristics) are equal with respect to the proportion of patient fta scores. in addition, it does not tell us which patient group or patient groups cause the rejection. thus, a two sample t-test was conducted to test the difference between two patient groups and when there are several patient groups under investigation; the marascuilo procedure was employed to simultaneously test the differences of all pairs of proportions. step 3: patients with an individual fta score 100% were analysed to isolate profiles of high risk patients. marascuilo procedure or two sample ttest were employed to test the differences of all pairs of patient proportions across different levels of patient demographic factors. variable proportion sample size difference in proportions p-value†/ critical value‡ gender female (f) male (m) 8.69% 9.78% 20,463 13,615 p(f)-p(m)* 0.011† identifying cultural background europe (eur) asia australia, new zealand (aus&nz) north america africa australasia south america america 4.96% 8.25% 8.81% 9.09% 10.54% 10.62% 10.83% 11.54% 2,843 4,265 19,322 110 1,167 113 157 52 p(eur)p(africa)* p(eur)-p(asia)* p(eur)p(aus&nz)* 0.037‡ 0.022‡ 0.017‡ suburb region eastern metropolitan (em) northern rural victoria (nrv) eastern rural victoria (erv) south eastern metropolitan (esm) southern metropolitan (sm) western rural victoria (wrv) western metropolitan (wm) northern metropolitan (nm) 7.02% 8.26% 8.74% 8.95% 9.56% 10.00% 10.33% 10.34% 3,346 1,973 2,862 3,687 2,679 2,370 7,539 9,839 p(em)-p(nm)* p(em)-p(wm)* p(em)-p(wrv)* 0.020‡ 0.021‡ 0.028‡ age in years 1: 18-25 2: 26-44 3: 45-65 4: 65+ 10.07% 12.61% 6.56% 3.62% 9,767 12,884 8,370 3,394 p(2)-p(1) p(2)-p(3) p(2)-p(4) p(1)-p(3) p(1)-p(4) p(3)-p(4) 0.0118‡ 0.0114‡ 0.0124‡ 0.0111‡ 0.0121‡ 0.0117‡ indigenous yes (y) no (n) 15.96% 9.24% 664 28,737 p(y) – p(n)** 0.000† refugee (and asylum seeker) yes no 27.54% 9.50% 69 343,46 p(y) – p(n)** 0.000† australasian journal of information systems bellucci, dahrmesana, nguyen & calache 2017, vol 21, research on health information systems effectiveness of sms reminders 13 variable proportion sample size difference in proportions p-value†/ critical value‡ concession card type pensioner concession card (pcc) health care card (hc) no card (nc) 7.83% 10.05% 19.60% 13,792 19,271 1,352 p(nc)-p(hc)** p(nc)-p(pcc)** p(hc)p(pcc)** 0.000† 0.000† 0.000† dental service type 1.general care 2.student clinics 3.surgery 4.specialist care 5.special needs 13.58% 13.05% 8.33% 7.49% 3.92% 6,238 5,861 4,696 16,467 1,149 p1-pi †† i = 2,3,4 p2-pj †† j = 3,4,5 p3-p5 †† p4-p5 †† 0.022‡,0.015‡ ,0.018‡ 0.019‡, 0.22‡,0.022‡ 0.015‡ 0.018‡ * test for difference =0 (vs < 0) ** test for difference =0 (vs > 0) †p-value ‡ critical value †† list of differences in patients’ proportions that are statistically significant table 9 characteristics of patients who failed to keep their appointments (fta score = 100%) as per table 9, female patients were significantly more likely to attend their appointments than male patients. older patients were less likely than younger patients to miss appointments. interestingly, patients who were in 26-44 age group were more likely than other patients to miss all of their appointments. the group (18-24 years) was the second most likely to miss their appointments. these differences across age groups were found to be statistically significant. patients from socially disadvantaged groups, such as identifying as indigenous (aboriginal, torres strait islander), refugees and asylum seekers were significantly more likely to miss appointments. patients who had pensioner concession card were more likely than others to keep their appointments. patients who must pay the full fee were more likely to miss appointments than concession and healthcare cardholders. proportions of fta score =100% varied markedly by cultural background. europeans were more likely to keep appointments and the percentage differences between them and australians and new zealanders, asia, africa were statistically significant. patients living in low income and disadvantaged regions such as western rural victoria, western metropolitan and northern metropolitan, were most likely to miss appointments whereas those from eastern metropolitan (and the most affluent) were less likely. furthermore, the differences in proportions between eastern metropolitan and three regions including western rural victoria, western metropolitan, northern metropolitan were statistically significant. patients with appointments in general care and student clinics were more likely to miss appointments, while special needs patients were more likely to keep appointments. in addition, the differences in fta score =100% proportions between patients’ scheduled for general care and each service type of student clinics, surgery, and specialist care; between student clinics and each of surgery, specialist care and special needs; and between surgery and special needs specialist care and special needs were statistically significant. australasian journal of information systems bellucci, dahrmesana, nguyen & calache 2017, vol 21, research on health information systems effectiveness of sms reminders 14 6 discussion 6.1 discussion of findings in relation to previous research this section discusses the findings in relation to previous work in the literature. our reported non-attendance rate of 9.48% (no sms sent) is on par and compares well against rates reported in medical specializations and dental literature. non-attendance rates range from 12% in ophthalmology clinics (brannan et al. 2011), 33.6% in an ent outpatient clinic (geraghty et al. 2008) and 23.4% in four specialized outpatient clinics (downer, meara & da costa 2005), while in dental outpatient clinics, non-attendance rates range from 11% for tooth scaling reminders (cheng et al. 2013), 31% and 12% in pediatric dental (parikh et al. 2010; cheng et al. 2013) respectively and 21.3% in general dental clinics (lalloo & mcdonald 2010). our statistical analysis found insufficient evidence to infer that the sms reminder intervention improved the non-attendance rate for the outpatient clinic. this result contravenes the majority of literature which overwhelmingly reports sms reminders’ effectiveness in lowering non-attendance rates. similar to our finding, a small number of studies reported no change in the non-attendance rate from a sms intervention (table 1), suggesting sms reminders may not be appropriate for all patient cohorts, particularly for low socio-economic groups who may experience additional barriers to accessing health care, particularly dental care (norton et al. 2014). our study found the percentage of patients not attending appointments appears to increase significantly as the number of scheduled appointments increases, and reaffirms the finding that multiple sms reminders did not help patients who repeatedly fail to attend their general practice appointments (fairhurst &sheikh 2008). this finding suggests that technology by itself does not solve the patient non-attendance problem. our finding that missed appointments were significantly associated with persons aged 25 – 44 may be due to work and other priorities of adults in this age group, such as a taking care of children (lalloo & mcdonald 2010). we also found older patients are more likely than younger patients to keep their appointments, as similarly reported in the literature (cashman et al. 2004; perron et al. 2010; parikh et al. 2010). sms reminders were found ineffective in a primarily youth (aged 12 – 24) clinic in a similar study (narring et al. 2013), where it was reported most patients did not initiate appointments as they were often referred by a healthcare professional or their parents. in our study, the second age group most likely to miss appointments was the younger 18 – 24 age group. further research is warranted into understanding how young adults make appointments and took responsibility for appointments made on their behalf. previous studies reported mixed findings regarding gender. males are more likely to miss appointments (perron et al. 2010; hamilton, round & sharp 2002), while in another study women are most likely to miss appointments (neal et al. 2001). yet other studies concluded there is no significant relationship between gender and non-attendance (bos, hoogstraten & prahl-andersen 2005; cashman et al. 2004; parikh et al. 2010). whilst our study found men are more likely to miss appointments, this association was the least in statistical significance compared to other patient characteristics. males may tend to prioritize work schedules before health related appointments, particularly if the appointment is not absolutely necessary. more studies are required to investigate whether a strong association exists. we also found patients with appointments in general care or in student clinics contribute to the non-attendance rate. clinical general care includes appointments for non-complicated dental care and routine check-ups. patients may tend to miss appointments if symptoms are self-resolving (parikh et al. 2010) and may fail to attend routine check-ups if they feel well, particularly if they are busy with other priorities such as paid work or caring for children. our study suggests patients tend to miss appointments with students clinicians. similar findings were reported where patients were likely to miss appointments if they were cared for by a junior doctor rather than by a senior doctor (perron et al. 2010). the dental outpatient australasian journal of information systems bellucci, dahrmesana, nguyen & calache 2017, vol 21, research on health information systems effectiveness of sms reminders 15 service in our study provides training facilities and supervising clinicians for student dentists from the university of melbourne. appointments with student clinicians can be lengthy as students must have each stage of the procedure checked before they can move on, with supervisors in charge of 5 – 8 students at a time. patients are aware of the increased time required for appointments in student clinics when appointments are made. even though appointments with student clinicians are free, patients may not comprehend the high quality of supervision provided or are concerned with the longer time required to complete a dental procedure. patients identifying with a cultural background of australian, new zealand, asian or african, are of indigenous descent or have refugee or asylum seeker status were associated with higher rates of consistently missing appointments (that is, a higher proportion of individual patient fta score =100%) compared to others. riggs et al. (2012) found disadvantaged communities such as refugees and migrants may lack oral health knowledge usually accepted in a developed country, preferring to use anecdotal care conventional in their culture. similarly, some migrant groups do not value dental health, for example the chinese view of dental disease as ‘normal’ (kwan & holmes 1999). aboriginals have a concept of time which differs from a system based on setting specific appointments, which may explain aboriginal non-attendance (lalloo & mcdonald 2010). in addition, perron et al. (2010) found asylum seeker status is related to high non-attendance rates and in this case cites poor socio-economic status as an indicator of non-attendance status (perron et al. 2010). persons living in low socio-economic regions in melbourne and victoria, in particularly western victoria, western metropolitan and northern metropolitan tend to be associated with non-attendance. these areas of melbourne are identified as low socio-economic by the latest abs census data (abs, 2011). this finding concurs with similar conclusions relating low socioeconomic status to non-attendance (cashman et al. 2004, norton et al. 2014; neal et al. 2001). the finding that non-concession card holders (patients who are ineligible for discounted dental care) are the most likely to miss appointments is supported by parikh et al. (2010) which found patients who must pay for their treatment tend not to attend. an explanation may be related to the clinic in our study treating emergency patients, many of whom would not normally attend the clinic as an eligible patient. patients treated in emergency are often booked into a follow up appointment at the outpatient clinic. these patients, for example, may prefer to attend a private dentist who is located closer to home, and may do so without cancelling the follow up appointment. our results suggest patients living in low socio-economic regions of victoria contribute to the fta rate. seemingly contravening these results are our findings suggesting non-concession card holders (predominately patients in high socio-economic groups) also contribute to the non-attendance rate. the proportion of non-concession holders however is very small, and we suggest a most likely explanation for this high fta rate is due to emergency services connected to the clinic, as explained above. as evidenced in table 2, the clinic predominately services patients holding health card and pensioner concession cards. 6.2 practice implications firstly, sms reminders should not be implemented in outpatient clinics simply because the intervention is considered low cost. we have shown patient and appointment characteristics are strongly associated with non-attendance, and suggest clinics review the characteristics of their patient cohort to inform a targeted approach to reducing non-attendances. other implications of this work includes revising assumptions into how the sms reminder service supports the patient appointment system and its interactions with patients. the sms system in our study requests a patient contacts the service only if they are not attending the appointment. if no response is received, the outpatient service assumes the patient will be attending. in doing so, the clinic also assumes the patient saw the message and has acted upon it. in contrast, if the emphasis for responding to an sms is on confirming attendance, the service will also capture patients who are cancelling their appointment. another suggested australasian journal of information systems bellucci, dahrmesana, nguyen & calache 2017, vol 21, research on health information systems effectiveness of sms reminders 16 revision to the sms reminder service includes provision for patients to automatically confirm attendance through touch screen technology. for example, patients may confirm or cancel their appointment by pressing a button labelled “yes” or “no” respectively, whilst at the same time management can ensure the sms has been received and acted upon. these software changes may assist patients to better communicate their intentions directly to the clinic and hence contribute to the re-scheduling of appointments. targeted interventions should be considered for people with the characteristics we have found contribute to the non-attendance rate. reminders targeted to such persons shortly before the appointment (including telephone calls) may assist in rescheduling the appointments that patients cannot attend. other recommendations include double booking appointments for multiple non-attending patients and implementing a fee-based system to act as a deterrent. policy makers may consider increasing dental health promotion for the socially marginalized (including indigenous communities, refugees, and asylum seekers) and for communities commonly associated with low income. for people who do not attend appointments for service types such as general care and with student clinicians, these groups may benefit from health promotion programs on the importance of preventive and continuous care. likewise patients may not be aware of the adequate level of care provided by student clinicians who are supervised by senior dentists. targeted interventions for patients who do not hold a concession card should be considered. we have found paying for an appointment as a predictor of non-attendance, and hence recommend targeted reminders or even a fee based system for missed appointments by non-concession cardholders. 6.3 limitations limitations of the study include restricted generalizability as it reports on a single outpatient clinic, even though it is common for studies in this area to draw conclusions based on populations from the one site4. outpatient clinics often see numerous patients who present with different conditions and are attended to by various health professions, hence resembling a diversity which is representative of multiple site data collection. given the consistency of our data sampling method and findings with previous studies reporting on low-income and disadvantaged patients, it is reasonable to assume our findings may be replicated in clinics serving similar patient groups. the study was also limited in the range of criteria predicting non-attendance. the analyses in this study assess some of the factors present in the literature that have been shown to be associated with patient appointment keeping. others such as the time and day of the appointment were not included. finally, we could only make inferences from the data to explain why patients missed appointments. other factors such as occupation, highest education level, marital status, nature of the job (part time, casual, fulltime etc.), number of dependents, owner of the mobile, mode of transport, travel time to clinic etc will be considered in our future work. 7 conclusion based on a large data set of 862,745 appointment records over the period of 46 months, our findings in relation to the ineffectiveness of sms and the confirmed associations between patient characteristics and non-attendance strongly suggest patient characteristics are important factors to consider in the adoption of sms reminders. this study is of particular importance to public health care settings, and can be used to inform interventions that will have a high likelihood of reducing missed appointments. future work will involve a follow up study to investigate our interpretations of demographic and patient characteristic predictors, such as differences in patient location, country of origin, age and gender. the new concept of a patient fta score will be used to conduct different types 4 for example, of the 25 papers listed in table 1, 18 of these used data retrieved from a single site. australasian journal of information systems bellucci, dahrmesana, nguyen & calache 2017, vol 21, research on health information systems effectiveness of sms reminders 17 of analytics to further investigate non-attendance. in addition, a qualitative research study is planned to understand the reasons for patient non-attendance. interviews will be conducted with patients who fit the above discussed profile and miss appointments, to understand why they miss appointments and how we can best assist them in taking care of their health. such a qualitative study will contribute to the information systems literature by examining contextual and human factors affecting the effectiveness of technology when introduced to healthcare service provision. such factors can be explored and analyzed using socio-technical theories such as activity theory (vygotsky 1978; engeström 2000) or actor network theory (callon 1986; latour 2005). implications for practice from these studies include strategies to increase attendance at outpatient clinics. future research into the relationships between patient demographics and non-attendance is vital to providing underprivileged communities with realistic opportunities for improved healthcare. author contributions emilia bellucci and lemai nguyen collaborated on the protocol for the research, reviewing the statistical results and developed the full manuscript. lasitha dharmasena contributed to the statistical analysis of the data and hanny calache provided access to the data and clinical expertise required in interpreting our results. acknowledgements many thanks to the anonymous reviewers and this journal’s editors for their comments on earlier drafts of the manuscript. we also thank the participating dental outpatient clinic’s administrative staff for their contributions to this project. statement of conflict of interest there was no conflict of interest relevant to this research. summary table what is already known of the topic: • the literature overwhelmingly attributes sms reminders to improving the attendance rate of patients in outpatient clinics • there are limited studies reporting sms reminders have no effect on the nonattendance rate, suggesting universal and successful application of sms reminders cannot be assumed. • patient and appointment demographics may help predict patient non-attendance. key results: • the fta rate before the implementation of the sms reminder system is lower than the fta rate following implementation. • sms reminders were ineffective among the population sent an sms. • patients who miss all their appointments exhibited one or more of the following characteristics: male; age 26 – 44; non-concession card holders; a person of indigenous, local, asian or african descent, and of refugee status, persons living in low socio-economic areas; and have appointments in general care and student clinics. contributions: • sms reminder systems have a limited effect on certain demographics and appointment characteristics, including those synonymous with the public health system. • a demographic and appointment profile of patients who never attend appointments is proposed. australasian journal of information systems bellucci, dahrmesana, nguyen & calache 2017, vol 21, research on health information systems effectiveness of sms reminders 18 • outpatient clinics should not assume a cost effective intervention such as sms reminders will be successful in their context, and suggests clinics focus on eliciting patient profiles to better understand patient non-attendance. references abs. (2011) census of population and housing: socio-economic indexes for areas (seifa). australian bureau of statistics. adanikin, a.i., awoleke, j.o & adeyiolu, a. (2014) “role of reminder by text message in enhancing postnatal clinic attendance”. international journal of gynecology & obstetrics 126(2):179-180. boksmati, n., butler-henderson, k., anderson, k. & sahama, t. (2016) “the effectiveness of sms reminders on appointment attendance: a meta-analysis”. journal of medical systems 40:90. bos, a., hoogstraten, j. & prahl-andersen, b. (2005) “failed appointments in an orthodontic clinic”, american journal of orthodontics and dentofacial orthopedics 127(3), 355-357. brannan, s.o., dewar, c., taggerty, l. & clark, s. (2011). the effect of short messaging service text on nonattendance in a general ophthalmology clinic. scottish medical journal 56(3):148-150. callon, m. (1986). the sociology of an actor-network: the case of the electrical vehicle. in m. callon, j. law, & a. rip (eds.), mapping of the dynamics of science and technology. basingstoke: the macmillan press lt. cashman, s.b., savageau, j. a., lemay, c. a. & ferguson, w. (2004) “patient health status and appointment keeping in an urban community health center” journal of health care for the poor and underserved 15(3): 474-488. chen, z-w., fang, l-z., chen, l-y., dai, h-l. (2008) “comparison of an sms text messaging and phone reminder to improve attendance at a health promotion center: a randomized controlled trial”. journal of zhejiang unversity science b 9(1):34-38. cheng c-cj, li c-y, hu y-j, shen, h-c. & huang, s-m. (2013) “effects of tooth scaling reminders for dental outpatients”. journal of telemedicine & telecare 19(4):184-89. da costa, t.m., salomão, p.l., martha, a.s. pisa, i.t. & sigulem, d. (2010) “the impact of short message service text messages sent as appointment reminders to patients' cell phones at outpatient clinics in são paulo, brazil”. international journal of medical informatics 79(1):65-70. deloitte. (2014) media consumer survey 2014: australian media and digital preferences – 3rd edition. dental health services victoria. (2011) links between oral health and general health the case for action , available at: https://www.dhsv.org.au/__data/assets/pdf_file/0013/2515/links-between-oralhealth-and-general-health-the-case-for-action.pdf [accessed 10th september 2016]. downer, s.r., meara, j.g. & da costa, a.c. (2005) “use of sms text messaging to improve outpatient attendance”. medical journal of australia 183(7):366-68. downer, s.r., meara, j.g., da costa, a.c. & sethuraman, k. (2006) “sms text messaging improves outpatient attendance”. australian health review 30(3), 389-396. engeström, y. (2000). "activity theory and the social construction of knowledge: a story of four umpires". organization, 7(2), 301-310. doi: 10.1177/135050840072006 fairhurst, k., sheikh, a. (2008) “texting appointment reminders to repeated non-attenders in primary care: randomised controlled study”. quality and safety in health care 17(5):373-376. australasian journal of information systems bellucci, dahrmesana, nguyen & calache 2017, vol 21, research on health information systems effectiveness of sms reminders 19 farmer, t., brook, g., mcsorley, j., murphy, s. & mohamed, a. (2013) “using short message service text reminders to reduce 'did not attend' rates in sexual health and hiv appointment clinics”. international journal of std & aids 0(0), 1-5. foley, j. & o'neill, m. (2009) “use of mobile telephone short message service (sms) as a reminder: the effect on patient attendance”. european archives of paediatric dentistry: official journal of the european academy of paediatric dentistry 10(1):15-18. geraghty, m., glynn, f., amin, m. & kinsella, j. (2008) “patient mobile telephone 'text' reminder: a novel way to reduce non-attendance at the ent out-patient clinic” journal of laryngology and otology, 122(3): 296-298. guy, r., hocking, j., wand, h., stott, s., ali, h. & kaldor, j. (2012) “how effective are short message service reminders at increasing clinic attendance? a meta-analysis and systematic review”. health services research 47(2):614-632. hamilton, w., round, a. & sharp, d. (2002) “patient, hospital, and general practitioner characteristics associated with nonattendance: a cohort study”. british journal of general practice 52: 317-19. hasvold, p.e. & wootton, r. (2011) “use of telephone and sms reminders to improve attendance at hospital appointments: a systematic review”. journal of telemedicine and telecare 17(7): 358-364. koshy, e., car, j., & majeed, a. (2008) “effectiveness of mobile-phone short message service (sms) reminders for ophthalmology outpatient appointments: observational study”. bmc ophthalmology 8:9. kwan, s.y.l. & holmes, m.a.m. (1999) “an exploration of oral health beliefs and attitudes of chinese in west yorkshire: a qualitative investigation”. health education research 14(4):453-60. lalloo, r. & mcdonald, j. m. (2010) “appointment attendance at a remote rural dental training facility in australia”. bmc oral health 13(1):1-8. latour, b. (2005). reassembling the social: an introduction to actor-network theory. oxford, uk: oxford university press. leong, k.c., chen, w.s., leong, k.w., mastura, i., mimi, o., sheikh, m.a., zailinawati, a.h., ng, c.j., phua, k.l. & teng, c.l. (2006) “the use of text messaging to improve attendance in primary care: a randomised controlled trial”. family practice 23: 699– 705. liew, s.m., ng, c.j., tong, s.t., lee, v.k.m., teng, c.l. & leong k.c. (2009) “text messaging reminders to reduce non-attendance in chronic disease follow-up:a clinical trial”. british journal of general practice, 59: 916–920. mcinnes, d.k.k, petrakis, b.a., gifford, a.l, rao, s.r., houston, t.k., asch, s.m. & o’toole, t.p. (2014) “retaining homeless veterans in outpatient care: a pilot study of mobile phone text message appointment reminders”. american journal of public health 104(s4):s588-s94. mciver, r., dyda, a., mcnulty, a.m., knight, v., wand, h. c. and guy, r. j. (2015) “text message reminders do not improve hepatitis b vaccination rates in an australian sexual health setting”, journal of the american medical informatics association 0:1-5 milne, r.g. (2010) “reducing non-attendance at specialist clinics: an evaluation of the effectiveness and cost of patient-focussed booking and sms reminders at a scottish health board” international journal of consumer studies 34(5): 570-580. milne, r.g., horne, m., & torsney, b. (2006) “sms reminders in the uk national health service: an evaluation of its impact on "no-shows" at hospital out-patient clinics”. health care management review 31(2):130-136. australasian journal of information systems bellucci, dahrmesana, nguyen & calache 2017, vol 21, research on health information systems effectiveness of sms reminders 20 narring, f.o., perron, n.j., dao, m.d., righini, n.c, humair, j-p., broers, b., gaspoz, j-m. & haller, d.m. (2013) “text-messaging to reduce missed appointment in a youth clinic: a randomised controlled trial”. journal of epidemiology & community health 67(10):888-891. neal, r.d., lawlor, d.a., allgar, v., colledge, m., ali, s., hassey, a., portz, c. & wilson, a. (2001) “missed appointments in general practice: retrospective data analysis from four practices”. british journal of general practice 51:830-832. nelson, t.m., berg, j.h., bell, j.f., leggot, p.j. & seminario, a.l. (2011) “assessing the effectiveness of text messages as appointment reminders in a pediatric dental setting”. journal of the american dental association 142(4):397-405. norton bl, person ak, castillo c, pastrana, c., subramanian, m. & stout, j. e. (2014) “barriers to using text message appointment reminders in an hiv clinic”. telemedicine and ehealth 20(1):86:89 parikh, a, gupta, k, wilson, a.c., fields, k., cosgrove, n.m., & kostis, j.b. (2010) “the effectiveness of outpatient appointment reminder systems in reducing no-show rates”. american journal of medicine 123(6):542-48. perron, n.j., dao, d. m., kossovsky, m.p., miserez, v., chuard, c., calmy, a. & gaspoz, j-m. (2010) “reduction of missed appointments at an urban primary care clinic: a randomised controlled study”. bmc family practice 11:79 87. perry, j.g. (2011) “a preliminary investigation into the effect of the use of the short message service (sms) on patient attendance at an nhs dental access centre in scotland”. primary dental care: journal of the faculty of general dental practitioners (uk) 18(4):145-149. prasad, s., & anand, r. (2012) “use of mobile telephone short message service as a reminder: the effect on patient attendance”. international dental journal 62(1):21-26. riggs, e., van gemert, c., gussy, m., waters, e., & kilpatrick, n. (2012) “reflections on cultural diversity in oral health promotion and prevention”. global health promotion 19(1): 6063. stubbs, n.d., geraci, s.a., stephenson, p.l., jones d.b. & sanders, s. (2012) “methods to reduce outpatient non-attendance” american journal of the medical sciences 344(3): 211-219. taylor, n.f., bottrell, j., lawler, k. & benjamin, d (2012) “mobile telephone short message service reminders can reduce nonattendance in physical therapy outpatient clinics: a randomized controlled trial”. arch phys med rehabil 93(1):21-26. vygotsky, l.s. (1978). mind in society: the development of higher psychological processes. cambridge: harvard university. sims, h., sanghara, h., hayes, d., wandiembe, s., finch, m., jakobsen, h., tsakanikos, e., okocha, c.i. & kravariti, e. (2012) “text message reminders of appointments: a pilot intervention at four community mental health clinics in london”. psychiatric services 63(2):161-168. youssef, a. (2014) “use of short message service reminders to improve attendance at an internal medicine outpatient clinic in saudi arabia: a randomized controlled trial”. eastern mediterranean health journal 5:317. copyright: © 2017 bellucci, dahrmesana, nguyen & calache. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems bellucci, dahrmesana, nguyen & calache 2017, vol 21, research on health information systems effectiveness of sms reminders 21 paper title (paper title style) australasian journal of information systems kautz, bjerknes, fisher & jensen 2020, vol 24, selected papers from acis 2018 complex adaptive system theory in contemporary isd 1 applying complex adaptive systems theory to understand distributed participatory design in contemporary, crowdsourced information systems development karlheinz kautz school of business it and logistics, rmit university, melbourne, australia karlheinz.kautz@rmit.edu.au gro bjerknes faculty of it, monash university, melbourne, australia julie fisher school of business it and logistics, rmit university, melbourne, australia tomas jensen communication, knowledge & change (formerly united nations children's fund), cairns, australia abstract contemporary information systems development (isd) takes place in a dynamic environment; it is generally acknowledged as a complex activity. it has been proposed that isd projects should be viewed as a complex adaptive system (cas) and that these projects are better understood through the application of cas. distributed participatory design (dpd) is an approach to contemporary isd where different, geographically dispersed stakeholders, often called the crowd, participate voluntarily and typically unaffiliated with the development organisation in the development and design of information systems in distributed design teams which are mostly online on internet, web-based, and social media platforms. going beyond individual methods, techniques, and practices, the objective of this research is twofold as we answer the two research questions: how are dpd projects in crowdsourced isd managed and performed in their entirety and how does cas theory provide plausible explanations and contribute to an understanding of contemporary isd? for this purpose, we present a case of dpd in crowdsourced isd in an intergovernmental, not-for-profit environment in the context of a dpd project which engaged pacific youth in the development of a digital game where we applied cas theory to better understand and gain insights for isd theory and practice. keywords: contemporary information systems development, distributed participatory design, complex adaptive systems theory. 1 introduction contemporary information systems development (isd) takes place in a dynamic environment; it is generally acknowledged as a complex activity (truex et al., 1999; highsmith 2000; meso and jain 2006, kautz 2012). benbya and mckelvey (2006) present an analysis of sources of australasian journal of information systems kautz, bjerknes, fisher & jensen 2020, vol 24, selected papers from acis 2018 complex adaptive system theory in contemporary isd 2 complexity in contemporary isd, and recognise as such: changing user requirements, organizational needs, and external competitive conditions; increased interdependencies among the involved individuals, organizations and technologies; and the rapid evolution of is and it. they purport that isd is often viewed as a complex top-down process and lament the fact that such a perspective falls short in dealing with the often unexpected contingencies of the isd process. as an alternative, they put forward a conception of isd based on complexity theory (kauffman 1993; holland 1995; holland 1998) and propose that isd projects should be viewed as a complex adaptive system (cas) and that these projects are better understood through the application of cas where complexity generally refers to an emergent property of systems that are made of large numbers of self-organising agents that interact in a dynamic and non-linear fashion. cas theory studies how a complex system can adapt to its environment and how innovative properties of a system emerge from the interactions of its components (vidgen and wang 2009). cas theory has been extensively applied in management studies, with benbya and mckelvey (2006) and vidgen and wang (2009) providing two comprehensive summaries of this literature. there are also writings on the topic in the is domain (jacucci et al. 2006; merali and mckelvey 2006), as well as specific research on cas and isd (highsmith 2000; jain and meso 2004; benbya and mckelvey 2006; meso and jain 2006; vidgen and wang 2006; vidgen and wang 2009; kautz and madsen 2010; kautz 2012). in isd, the work has mostly been either conceptual or in the area of agile software development. vidgen and wang (2009) lament a scarcity of empirical research on cas in the is field which they relate to the difficulty of turning the quite abstract concepts of cas theory into sufficiently concrete case study research and thus call for more research that on a cas theory background examines closely how development practices are actually implemented and performed. with the work presented here we provide such an empirical case study of an isd project. distributed participatory design (dpd) is one form of contemporary isd. it has its roots in participatory design (pd) which is an isd and design approach whose central tenet is the participation of people, users as equal design partners in the co-design of the information systems and technologies (is/it) that they are supposed to use themselves (kensing and blomberg 1998; lukyanenko et al. 2016). it originally developed in the 1970s in scandinavia and focused on user participation in internal organisational settings in the development of dedicated, tailor-made is/it which were to be used in the workplace. thus, most studies of pd examine the development of a single, customised information system that typically supports workflows within a single client organisation (obendorf et al. 2009). since its beginnings pd has progressed to also consider non-organisational as well as community-driven contexts (lukyanenko et al. 2016). participation by less formally organised communities and the crowd in the development of content producing community-based service systems has challenged existing isd and pd approaches (öberg et al. 2009; kazman and chen 2009; disalvo et al. 2013; lukyanenko et al. 2016). the nature of participation in such design and development processes is emergent and cannot be fully controlled (markus and mao 2004). to cope with these new contexts and forms of work and participation – such as communities and virtual networks – traditional pd expanded to deal with the diversification of stakeholders and with settings where stakeholders are distributed across various dimensions of time, space and organisational structures (obendorf et al. 2009). the concept of australasian journal of information systems kautz, bjerknes, fisher & jensen 2020, vol 24, selected papers from acis 2018 complex adaptive system theory in contemporary isd 3 distributed participatory design (dpd) refers to the participation of different stakeholders in distributed design teams, mostly online, through internet, web-based, and social media platforms where user participation in online projects is primarily voluntary and the participants are typically unaffiliated with the development organisation 1(lukyanenko et al. 2016). declaring a dearth of research clement et al. (2008) as well as disalvo and disalvo (2014) request more studies on how to conduct pd in new environments. much of the existing research on dpd focusses on individual methods, techniques, and practices such as mediated feedback, commented case studies, and surveys (gumm et al. 2006) as well as probe blogs, idea, concept, and feature postings, commenting, voting and online testing (näkki and koskela-huotari 2012). to go beyond these individual methods, techniques, and practices markus and mao (2004) ask to extend research on participatory approaches to isd transcending conventional settings, roles and types of participants and contributors. following these calls for more investigations into cas and dpd the objective of our research presented here is therefore twofold and we pose the following questions: how are dpd projects in crowdsourced isd managed and performed in their entirety and how does cas theory provide plausible explanations and contribute to an understanding of contemporary isd? to pursue our research objectives and to answer the research questions, we studied a case of dpd in crowdsourced isd in an unconventional setting in an intergovernmental, notfor-profit environment in the context of an unicef (pacific)2 initiated dpd project which engaged pacific youth in the development of a facebook based digital game which had the objective to raise awareness about climate change challenges in that region (fisher 2012). thus, we validate and confirm existing theory in a new setting and provide practical advice for performing contemporary isd. the remainder of the article is structured as follows: the next section introduces our theoretical background and analytical framework in more detail. our research approach is explained in section 3 and a case narrative is provided in section 4. section 5 includes the analysis of the dpd and isd process in the case setting; it also discusses our findings and their implications for research and practice. we finish with our conclusions and a summary of our contributions in section 6. 2 theoretical background: cas theory as stated above cas theory has been recognized as a valuable approach to understanding contemporary isd. as one of the first authors highsmith (2000, 2002) reflecting on his extensive experience as isd consultant and practitioner put forward that agile development approaches as an instance of isd draw on cas theory as their foundation. subsequently several research teams have applied the theory and provided organising principles and suggestions for best practice for the isd process which we in the following briefly review here. 1 in this context, it has been argued that the free/libre/open source software development (floss) approach represents a specific form of dpd (titlestad et al. 2009); however, with its focus on technical development of source code by mostly highly tech-savvy software developers and programmers we will here not consider this otherwise very interesting strand of work further. 2 the united nations children’s fund (unicef) is a united nations (un) non-government, non-forprofit organisation and program that provides humanitarian and development assistance to children and mothers in developing countries. for the remainder of this article, we will refer to unicef (pacific islands countries) as unicef (p). australasian journal of information systems kautz, bjerknes, fisher & jensen 2020, vol 24, selected papers from acis 2018 complex adaptive system theory in contemporary isd 4 benbya and mckelvey (2006) present some conceptual work, focusing on the co-evolutionary aspect of cas, and put forward seven rather abstract principles of adaptive success, and discuss their bearings on isd problems. their advice is quite abstract and, in parts, not easy to implement. meso and jain (2006) identify seven principles of cas which, to some extent, overlap with those described by benbya and mckelvey (2006), but whose meanings are more intuitive and easier to grasp. they are the principles of open systems, interactions and relationships, transformative feedback loops, emergent behaviour, distributed control, shallow structure, and growth and evolution. the authors provide a conceptual argument that they support with various examples from the literature. they map the principles to agile development practices, and from this mapping derive more concrete, yet generic recommendations for best practices for isd. vidgen and wang (2009) with a focus on co-evolution and agile development develop a framework and summarise the key concepts of cas in three guiding principles for the organization of agile software development as matching co-evolutionary change rate, optimizing self-organisation, and synchronising concurrent exploitation and exploration. based on an empirical case, they identify six capabilities as enablers of agility in agile teams grounded in cas, namely, co-evolution of it team and customer to create business value; sustainable working with rhythm; collective mindfulness; sharing and team learning; process adaptation and improvement; product innovation. kautz and madsen (2010) provide similar findings, also basing their research on the objective of understanding agile software development. extending this work kautz (2012) shows how cas and cas principles are advantageous for comprehending and organising isd in general, beyond any particular development approach chosen for the execution of a project such as agile development. in an empirical account of contemporary isd he identifies a number of practices and provides practical advice to cope with complexity in isd in an adaptive manner. the practices are similar to those depicted by meso and jain (2006) and vidgen and wang (2009), but he argues that rather than being enablers for agility, these practices are properties of complex adaptive systems. there is however no single and fully shared definition of cas and no unifying cas theory, therefore our theoretical background is based on vidgen and wang’s (2009) literature review which takes both the original cas literature and its applications in managerial, organisational and is studies into account. a complex adaptive system consists of a large number of loosely interconnected and interacting autonomous parts or agents, each of which behaves according to some set of, sometimes rather simple, rules. these rules require agents to adjust their behaviour to that of other agents with whom they interact. the resulting system behaviour can be very complex (vidgen and wang 2009). interaction is a significant concept in this context, as “the behaviour of the system is determined by the nature of these interactions, not by what is contained within the components. since the interactions are rich, dynamic, nonlinear, and are fed back, the behaviour of the system as a whole cannot be predicted from the inspection of its components. the notion of "emergence" is used to delineate this aspect” (jain and meso 2004 citing cilliers 2000). cas theory rests on the idea that order emerges through the interaction of the agents (benbya and mckelvey 2006). emergence is the property of cas which creates some greater property of the whole, the system behaviour results from the interaction of the parts and the australasian journal of information systems kautz, bjerknes, fisher & jensen 2020, vol 24, selected papers from acis 2018 complex adaptive system theory in contemporary isd 5 agent behaviours; however, the emergent system behaviour cannot be predicted or fully explained from the measured behaviours of the agents (highsmith 2000). interaction and emergence are closely related, and link together other generally acknowledged properties of cas. beyond the interconnected autonomous agents, a number of concepts are frequently used when discussing cas. these concepts which we briefly introduce below are selforganisation, co-evolution, the poise at the edge of chaos, time pacing, and the poise at the edge of time. we have described and applied these concepts earlier in a detailed analysis of an isd project which used agile systems development and project management practices (kautz 2012). interconnected autonomous agents, human or non-human, have the ability to independently intervene and determine what action to take, given their perception of their environment. yet, they are interconnected and interact in such a way that they collectively or individually are responsive to change around them, but not overwhelmed by the information flowing to them by this connectivity (mitleton-kelly 2003). self-organisation is the capacity of interconnected autonomous agents to evolve into an optimal organised form without external force. it results from the agents’ interaction in a disciplined manner within locally defined and followed rules and, as such, requires a departure from command and control management (volberda and levin 2003; vidgen and wang 2009). co-evolution relates to the fact that a cas and/or its parts alter their structures and behaviours at a sustainable change rate in response to the interactions of its parts and to the interaction with other cas that co-exist in an ecosystem where adaptation by one system affects the other systems. this leads to further adaptations and reciprocal change where the systems do not evolve individually, but concertedly (kauffman 1993; vidgen and wang 2009). poise at the edge of chaos describes the ability of a cas to be at the same time stable and unstable, to never quite lock into place, yet never quite fall apart. the edge is the place that provides not only the stimulation and freedom to experiment and adapt as well as for novelty to appear, but it also allows for the sufficient frameworks and structures to avoid disorderly disintegration. this gives a competitive advantage: cas that are driven to the edge of chaos out-compete those that are not (brown and eisenhardt 1998; anderson 1999; stacey 2003; mcmillian 2004; vidgen and wang 2009). time pacing in this context indicates that a cas creates an internal rhythm that drives the momentum of change. change in a cas is triggered by the passage of time rather than the occurrence of events; this stops them from changing too often or too quickly (brown and eisenhardt 1998; vidgen and wang 2009). poise at the edge of time conceptualizes a cas’s attribute of simultaneously being rooted in the present, yet being aware of the future and its balance as well as synchronization between engaging enough exploitation of existing resources and capabilities to ensure current viability with engagement of enough exploration of new opportunities to ensure future viability (brown and eisenhardt 1998; volberda and levin 2003; vidgen and wang 2009). all these core concepts are heavily intertwined and mutually reinforcing (vidgen and wang 2006). thus, cas can be characterized through the emergence of co-evolutionary, selforganised behaviour, structure and order through the interaction of interconnected autonomous agents in a time-paced rhythm balanced at the edge of time. at the heart of cas australasian journal of information systems kautz, bjerknes, fisher & jensen 2020, vol 24, selected papers from acis 2018 complex adaptive system theory in contemporary isd 6 theory’s relevance for isd is the concept of emergence which appears in relation to all key concepts (highsmith 2000; kautz 2012): (1) as emergent order resulting from self-organising agent interactions (kauffman 1993; holland 1995), (2) as the emergence of team learning as a result of the interaction and capabilities of interconnected autonomous agents (mitleton-kelly 2003), (3) as truly emergent self-organisation in contrast to a premeditated or deliberately implemented one by management (stacey 2003), (4) as co-evolutionary emergent behaviour and structure (benbya and mckelvey 2006) or even emergent co-evolution (kelly 1994), (5) as the region of emergent complexity, the edge of chaos (benbya and mckelvey 2006), (6) as rhythm considered an emergent state of working in the context of time pacing (vidgen and wang 2009), and (7) as emergent balance between exploitation and exploration at the edge of time (bocanet and ponsiglione 2012). 3 research approach and method our research follows the interpretive paradigm and as such takes a hermeneutic stance; accordingly we perform and secure the validity of our work through the application of an established set of principles for conducting and evaluating interpretive studies in information systems that suggests that all human understanding is achieved by iterating between considering the interdependent meaning of the parts and the whole that they form (klein and myers 1999). these principles are the fundamental principle of the hermeneutic circle, the principle of contextualisation, the principle of interaction between the researchers and the subjects, the principle of abstraction and generalisation, the principle of dialogical reasoning, the principle of multiple interpretations, the principle of suspicion. this approach covers and extends other measures as for example put forward by venkatesh et al. (2013) to obtain design as well as analytical and inferential validity of qualitative research in general. in the following will we describe how we have applied the listed principles in this research. in line with our hermeneutical perspective, we also applied a hermeneutic approach for conducting literature reviews that recognises a literature review as an iterative process of developing understanding that is emergent, unpredictable and creative, rather than straightforward, orderly and strictly prescribed (boell and cecez-kecmanovic 2014). it has, as baiyere (2017) has put it “the advantage of a terse and pithy presentation of prior research and gives room for more conceptual reasoning over the tendency to re-list articles.” this fits well with the overarching aim of our research that is to challenge the widely accepted knowledge about isd to advance theory in that area. alvesson and sandberg (2011, 2013) have described this approach to research as assumption-challenging rather than ‘gap-spotting’ and have labelled it as generating research questions through problematisation (alvesson and sandberg 2011). given the limited literature concerning dpd in crowdsourced isd and how it unfolds, our investigation is based on an exploratory, qualitative case study (creswell 2003) of an isd australasian journal of information systems kautz, bjerknes, fisher & jensen 2020, vol 24, selected papers from acis 2018 complex adaptive system theory in contemporary isd 7 project. the project utilised a social media platform and involved a number of different organisational units and stakeholder groups including numerous youths as participants. while it is often stated that it is not possible to theorise and to generalise from a single case study, walsham (1995) suggests that it is possible to generalise case study findings among others in the form of a contribution of rich insight. in contrast to a representative case selection we selected a revelatory case which has not been described in the is literature before; such cases are considered to be prototypical of the phenomenon of interest (gerring 2007) and by studying such a case we were able to illuminate key aspects of the phenomenon under investigation. on this background we used the key concepts of cas theory in a new setting for our data analysis and followed the principle of abstraction and generalisation. as such, our findings exhibit an instance of theorising through idealisation (ghazawneh and henfridsson 2013) which needs to be further validated. the roles and length of stay in the field varied for the four authors of this paper. the fourth author has been involved in the project as a reflective practitioner (schön 1983) throughout the whole period. as the unicef (p) communications specialist and project sponsor, he was involved as the overall project co-coordinator at all stages of the project. he shared correspondence and provided reflections on the process. as an employee and insider, he enhanced the depth and breadth of understanding the case setting that may not be accessible to a non-native researcher (kanuha 2000). the third author also participated during the whole project, as an involved, accompanying researcher (walsham 1995) and academic consultant. initially involved in the establishment of the technical development team and its coordination and communication protocol, and although later only occasionally providing advice to the project team and otherwise observing the development process she had an impact on the design and development of the game. given the background of these authors the purpose of the research presented here was to investigate in a less unbiased manner how dpd takes place in practice. thus, the first and the second author acted as outside observers (walsham 1995) and were included in the reflective process. they conducted interviews with the involved researcher and academic consultant and independently analysed all available empirical material and as such took the principle of interaction between the researchers and the subjects into account. during the analysis the two first authors regularly discussed their emerging results with the two others and incorporated their feedback. through this process our study followed the principles of multiple interpretations, dialogical reasoning, and suspicion. the combination of intervention, interpretation, and collaboration between the three academic researchers and the fourth author was chosen to increase the interpretive rigor of our analysis. in line with the research topic and the interpretive approach, our understanding of dpd in the game development project has come about through an iterative process of interpretation, comparison and connecting of prior research and empirical data which is in line with the fundamental principle of the hermeneutic circle. applying the principle of contextualisation in this process as described in the introduction we took the social and historical background of the research setting into consideration and included it in the critical reflections of our case. given the distributed location of the project participants the extensive email trail between them was the main data source. lee (1994) argues that email communication provides a rich understanding of what has occurred. these emails contained status information, reflections before, during and after the development and implementation of the game, conceptual australasian journal of information systems kautz, bjerknes, fisher & jensen 2020, vol 24, selected papers from acis 2018 complex adaptive system theory in contemporary isd 8 feedback, reflections and recollections concerning input into the design of the game, the elements of climate change which it was addressing, test results as well as technical feedback. the empirical data also comprised social media postings by the four fiji adolescents who served as facilitators between the technical development team and the juvenile pacific islander crowd and their responses to the request for input. table 1 includes a list of all participants, their roles and their locations; more detail about the different actors, their relationship and their locations will be provided in the next section. project documentation such as the unicef (p) strategic plan for digital engagement, its project description, brief and evaluation as well as a terms of reference document were included as valuable data sources as were the field notes by the sponsor and the academic consultant. further empirical data for the study was collected through semi-structured, openended interviews conducted by the academic consultant with the three members of the technical development team and by the outside researchers with the academic consultant and the project sponsor concerning their role and experiences during the project. the developers were interviewed for about 45 minutes in length with the interviews focusing on the issues around the dpd process and their reflections on the project. the issues included how they undertook the development process, how they managed the interactions with other participants, the mechanisms for communication and how they incorporated new ideas and change requests. the interviews also explored how the developers generated and refined their ideas particularly in relation to the sponsor’s brief and delved into their motivations for becoming involved apart from the modest amount they were paid. we wished to achieve an interaction between the existing literature and our observations from the case setting to explain interrelationships and contribute to theory with new insight from practice that might be useful for scholars and practitioners. in case study research identifying the boundaries of an investigated case is considered important (miles et al. 2014). our analysis takes its starting point in september 2010 when the project was conceived and ends in august 2011. following with what miles et al. (2014) describe as data condensation and based on sarker et al. (2006) and our earlier work (kautz 2009) our first step was to produce a list of the involved participants (see table 1), a timeline spanning that period (see figure 1) and a case narrative which is included in the following section in a condensed form. the narrative provides a progress or sequence of events and serves as a frame of reference for further analysis and interpretation of the data (fincham 2002). the next stage involved revisiting the narrative and the empirical data as we then we mapped the data onto the cas concepts and categorised our findings (see section 5) accordingly. using cas theory helped us to increase our understanding of dpd and isd practice in the case setting. before providing a more detailed analysis, we next present a narrative account of the investigated project. australasian journal of information systems kautz, bjerknes, fisher & jensen 2020, vol 24, selected papers from acis 2018 complex adaptive system theory in contemporary isd 9 figure 1. the timeline of the game development project 4 a case narrative we identified the following five phases of the game development project: 1 initiation of the idea and funding; 2 establishment of the team; 3 conceptual design of the game; 4 development of the consolidated game; 5 launch of the consolidated game. 4.1 phase 1 – initiation of the idea and funding mid 2010 the communications specialist at unicef (p) proposed a project to the organisation. he was concerned that although unicef (p) had a strong social media presence and was regularly communicating with their audience via social media, two-way interaction was very limited. his major objective was to ensure that pacific islander youth engaged more with unicef. his vision was to engage youth through encouraging them to participate in a project via social media. given the threats posed to small pacific islands from climate change the proposal was to develop a game which would also help youth to learn more about how to respond to climate change. he put this proposal to the commonwealth of learning (col) in canada, an intergovernmental organisation from the commonwealth of nation in november 2010 (for more information see www.col.org). the col subsequently provided modest funding in early january 2011 for hiring developers. the communications specialist who was located on the pacific islands immediately approached an information systems professor in melbourne, australia who was known to him with a request to join the project as an involved, accompanying researcher in the role of an academic consultant to help establish, manage, and support, when need arises, a development team. this led to the second phase. australasian journal of information systems kautz, bjerknes, fisher & jensen 2020, vol 24, selected papers from acis 2018 complex adaptive system theory in contemporary isd 10 project participants roles locations unicef communications specialist project sponsor, project coordinator, overall decision maker fiji, pacific islands information systems professor involved researcher as academic consultant melbourne, australia three research students developers in the core development team hongkong, melbourne, and regional victoria, australia four fiji adolescents social media facilitators fiji, pacific islands pacific islander youth requirements contributors pacific islands commonwealth of learning (col) project funder through financial support canada unicef staff expertise providers, feedback providers, functional and technical testers pacific islands, new york, usa three chinese youth functional testers china international climate experts expertise providers globally distributed table 1. project participants, roles, and locations 4.2 phase 2 – establishment of the team the academic consultant in january 2011 approached three young research students who fulfilled the position requirements; they accepted the invitation and were in the same month appointed as the developers for a period of 30 working days with an original project runtime from february 1 to april 15, 2011. two of them were chinese by birth and one was from bangladesh. one developer was living in hong kong, another lived in regional victoria, australia and the third in melbourne; the latter two knew each other, but they did not know the third developer beforehand, nor did they meet this developer in person during the project. the sponsor’s first email to the development team including the academic consultant described his vision and what he wanted to achieve, the game was not to be about climate change but how people could respond to the impact of climate change. in january 2011, the sponsor identified and contacted four adolescents from fiji to be social media facilitators for soliciting and gathering ideas from pacific islander youth about the game. the social media facilitators posted a photo with a message inviting input on the game and launched this as a facebook album with text encouraging unicef (p) facebook fans to participate and to contribute to the design of the game. initial input and comments came from 16 fans, as well as 15 fans hitting the ‘like’ button. during the same period the academic consultant facilitated a process among the members of the core development team and the sponsor who also acted as project co-ordinator where protocols for how the development team would operate were agreed. the academic consultant played no further significant role in the development process after the communication protocols had been agreed, but as stated above provided occasional advice and observed the course of the project. the sponsor was happy for the developers to manage the project themselves in terms of ideas for the game and how the work was undertaken. the developers’ first meeting was a telephone conversation about how they would manage the process given they were geographically dispersed. they agreed that they would email each other every couple of days to cater for the quite short timeline for finalising the game. they also decided to use skype to talk and instant messaging and chat to communicate. although there was no formal team leader, the student from bangladesh very quickly took charge of managing how australasian journal of information systems kautz, bjerknes, fisher & jensen 2020, vol 24, selected papers from acis 2018 complex adaptive system theory in contemporary isd 11 things would work, she kept meeting minutes including the decisions that were taken, the next discussion topics and who would be responsible for determining what the tasks would be. the tasks were reviewed at each meeting confirming what had been done and establishing the next tasks and responsibilities. at the end of each meeting an email summarising progress was sent to the sponsor by the informal leader. he reviewed the progress and if he thought there was something that needed to be changed or wanted to provide feedback, he would email the informal leader or alternatively he called her using skype. brief notes were taken from the skype meetings focusing on any requested changes. 4.3 phase 3 – conceptual design of the game the first stage of development was to reach agreement on what the game would be and its look and feel. one developer researched relevant aspects of climate change, another looked at different approaches to and types of facebook games and the third investigated appropriate technologies, tools and development approaches. as the development of the ideas for the game progressed the sponsor was an intermediary sharing these ideas with a range of people from the funding organisation, climate change experts and unicef staff to receive input concerning the direction of the game. further information on climate change was also provided on a regular basis by the relevant experts to the sponsor. the sponsor handed the feedback and the ideas of the involved pacific islander youth provided through the facebook page and facilitated by the four adolescents from fiji to the developers. the requirements of the sponsor and ideas of the key stakeholders, pacific islander youth, and unicef staff, guided the developers. the team used the following process to decide on their final game: at the very beginning the sponsor asked the developers to think about some ideas. they gave themselves a week to brainstorm and then collected their ideas to see which of them could be combined. this led to three major ideas; each with a particular focus from one developer which reflected what they individually thought what the youth and unicef (p) should concentrate on. this resulted in a game which consisted of three games in one (see figure 2). each game was quite different in the way the players would interact; the co2 reducer challenge requires players to identify potential co2 emitters; the evacuate life challenge requires players to understand the climate change threats and initiate action, e.g. to evacuate or rebuild before there are serious consequences; the flood tales challenge highlights the causes of floods and the need for flood mitigation. an important design principle was to ensure that each game was not too complicated. the developers found the fan page postings very helpful; the responses from the pacific islander youth had suggested that the game needed to be very interactive, interesting and colourful; it should have graphics, be fun and focused on action, something which promoted to be positive and to make change. 4.4 phase 4 development of the consolidated games after the developers and the sponsor had agreed on the consolidated game’s design, development proper, including detailed design, coding, testing and evaluation could begin. concerning the management and coordination of the process, one developer commented: “[the development process proper] was very challenging because we would not face each other and sit together, this was a challenging part.” the team took an active role in ensuring input in the form of further information and feedback was managed effectively and encouraged further participation by the sponsor and unicef (p) staff. as there was no opportunity to discuss, elaborate and clarify ideas and concerns face to face all information and communication had to be, and eventually was, very concise. as the team members were working independently australasian journal of information systems kautz, bjerknes, fisher & jensen 2020, vol 24, selected papers from acis 2018 complex adaptive system theory in contemporary isd 12 and each component of the game was developed separately, several issues concerning the build and layout of the consolidated game arose during this phase as one developer highlighted: “the game came in three different formats, totally different interfaces. the developing process of the three people was quite different. it came as three totally different styles of game, different user interface, different colour, a lot of things were different. there was no standard look to the three different games. fortunately, finally we got this sorted out the three games now look quite similar.” figure 2: an illustration of the games (adopted from fisher 2012) the sponsor and unicef (p) staff reviewed the first version of the consolidated game and provided feedback; this included the colours, fonts and graphics, the text and help function provided with the game. the sponsor highlighted that further work was needed on standardisation and how the three components linked together to be one game. the sponsor also reinforced the need for links to further information be embedded in each game. technical testing and evaluation were iterative. the developers each first conducted technical unit and system testing to uncover programming errors and for this purpose identified a set of criteria in particular to test the features of the game, to ensure the various games linked internally, that the colour schemes, text size and files et cetera were correct and consistent. each developer tested the work of the other two and provided feedback through their regular phone and skype meetings and email. while the developers tested for programming errors the game was functionally tested by unicef (p) staff who played the game and provided feedback to the sponsor. a technical person within unicef also tested the consolidated game and provided technical feedback once the team had incorporated the earlier feedback. the developers were also asked to find a platform to run the game and after investigation identified google which had a free service. further user evaluation similar to user acceptance testing was undertaken by three friends of the developers in china who were young and used facebook. they played the game and provided advice suggesting that the graphics and artwork needed to be still more attractive. they thought players would be encouraged to play longer if the game was even more interesting. the social media facilitators also provided feedback along these lines, suggesting the game be more colourful and easier to play. all feedback was considered, further changes made, and the final version of the game was ultimately accepted by the sponsor. australasian journal of information systems kautz, bjerknes, fisher & jensen 2020, vol 24, selected papers from acis 2018 complex adaptive system theory in contemporary isd 13 4.5 phase 5 launch of the consolidated game an email to various international unicef groups announced the launch of the game in july 2011. the game had a favourable reception as many positive comments on what had been achieved were made by unicef worldwide, pacific islander youth and facebook fans. a press release issued shortly after the launch showed unicef’s positive assessment of the initiative. postings on the unicef (p) fan page highlighted how successful the game was with requests for the game to be translated into pacific islands’ languages and to include it on the madagascar unicef page. voices of youth, a unicef organisation designed to support young people and to give them the opportunity to learn about their world requested that they embed the game on their website which the developers then did. lastly, the launch event marked the end of the project for the development team and sparked the developers’ pride about their achievement. the consolidated game is now in use and distributed through three facebook sites: unicef (p), voices of youth and unite for climate. 5 analysis and discussion: the game development project as a complex adaptive system the dpd in the game development project can be analysed and discussed from many theoretical perspectives. such perspectives could be the low degree of it use, notably of social media, by intergovernmental, non-governmental or not-for-profit organisations (chang and chang 2011; cokerill 2013), their challenges of engagement with youth (henderson and bowley 2010) and of the empowerment of this specific user group (clement 1994), markedly through gamification (hamari et al. 2014); another focus could be to examine the project as a particular instance of open source software development (titlestad et al. 2009) or to take the general discourse on crowdsourcing (lukyanenko et al. 2016) or on crowdsourcing in isd for value co-creation (kazman and chen 2009) into account as we have done elsewhere (kautz et al. 2019). our focus here however is on dpd as an approach to isd and how dpd projects in crowdsourced isd unfold and are managed and performed in their entirety. as contemporary isd is generally acknowledged as a complex activity we apply complex adaptive systems (cas) theory to better understand and make recommendations for isd practice. benbya and mckelvey (2006) based on conceptual work, meso and jain (2006) with support from examples from the literature, and vidgen and wang (2009) backed up through an empirical case, all focus on certain cas concepts for the organisation of isd projects, mostly agile development, without any consideration of distributed activities. the same is valid for kautz (2012) whose research went beyond agile development but stayed within the boundaries of a nondistributed project organisation. our work is based on these predecessors but centred around the idea of emergence we chose for our analysis of a distributed development endeavour a set of interrelated concepts that characterize cas: interaction, interconnected autonomous agents, self-organisation, poise at the edge of chaos, co-evolution, time pacing, and poise at the edge of time. 5.1 interactions interactions are a significant characteristic of a complex adaptive system. based on the initial project organisation which had been established by the sponsor with support from the academic consultant and which consisted of a first project plan and vision, an early requirements specification, and a formal contract for the developers and continuing with the enrolment of various other participant groups and emerging organisational structures, australasian journal of information systems kautz, bjerknes, fisher & jensen 2020, vol 24, selected papers from acis 2018 complex adaptive system theory in contemporary isd 14 interactions in the dpd and crowdsourced game development project took mainly place through various digital technologies and involved the different participant groups throughout the whole project in multiple forms, formal and informal, and at numerous occasions organised and spontaneous. they could among others be found during the scheduled meetings and feedback sessions between the sponsor and the developers and between the developers themselves. the narrative of the case which describes the project participants and their roles and relationships provides ample evidence of the rich, intense, dynamic, and nonlinear interactions which were fed back into the project and determined its performance and its emerging innovative properties resulting in the consolidated game. it indicates that project’s progression could not be predicted. the difference between the planned three months for developing the game and the actual seven months illustrates this and the concept of emergence describes this characteristic of cas. in the following, we revisit the game development project and accentuate the different facets of emergence and of the other key concepts of cas theory to provide a better understanding of dpd in crowdsourced isd and to derive some insights for the organisation of such isd projects. our analysis shows how all these core concepts of cas are strongly entwined and reciprocally reinforce each other. table 2 contains, as a summary of our findings, the cas concepts, their characteristics and the emerging effects in the game development project. cas concept cas characteristics in the project emerging effect interaction multiple forms of interaction: formal & informal, organised & spontaneous, at numerous occasions, among others during the scheduled meetings & feedback sessions between the various participant groups, through mainly digital technologies unpredictable, rich, intense, dynamic, non-linear interactions fed back into the project & determined its performance & emerging innovative properties of the resulting consolidated game inter connected, autonomous agents interconnected, autonomous agents in distributed groups of participants organised in a kernel, periphery & as masses; sponsor and facilitators as social mechanisms coordinated by simple rules, digital technologies as technical mechanisms; developer autonomy when distributing & selecting tasks & manifested in their individual, autonomous design decisions contributions; social media facilitator autonomy when filtering requirements; autonomy at the periphery & in the masses when participants joined & left as they saw fit sharing of project-relevant knowledge in feedback sessions & meetings; all kernel members’ involvement in project management & design decisions; developers’ task self-assignment; involvement of other participants in design and evaluation activities; emergence of collective learning among these participants emergent self-organisation emergent self-organisation: through egalitarian management with short communication paths; sponsor & facilitators acted like peers; developer self-organisation through shared responsibility for governance/decision making, task self-assignment, self-appointed team manager; social media facilitator self-organisation through the way requirements & feedback were solicited & filtered emergence of self-organisation of autonomous project members, in particular developers & social media facilitators; emergence of order with individual & team discipline as vital elements of selforganisation australasian journal of information systems kautz, bjerknes, fisher & jensen 2020, vol 24, selected papers from acis 2018 complex adaptive system theory in contemporary isd 15 cas concept cas characteristics in the project emerging effect poise at the edge of chaos state of bounded instability: stability through initial project plan & vision, overall requirements specification, formal contract, organisation of participants in kernel; instability through different participant groups spread over continents/time zones & continuous flow of ideas, requirements, change requests, feedback; balanced by emergent organisation in stable kernel, dispersed periphery & independent masses appearance of a region of emergent complexity with steady, swift handling of frequently incoming input through short iterations of manageable size & regular planning/feedback sessions in the kernel emergent co-evolution co-evolution of people, processes, products through multiple forms of interactions, knowledge sharing & mutual learning about ideas, requests, early game versions & feedback reinforcement of emergent structures of interaction & collaboration & unpredictable emerging structures & behaviours through unplanned inclusion/departure of participants resulting in changes of the game time pacing time pacing: internally set through short iterations, early test versions, as well formal weekly & regular within a week planning meetings/feedback sessions, based on rules for coordination & engagement; mechanisms for selecting ideas, requests, requirements; manageable requirements size emergence of a working rhythm with appropriate intervals to handle changes resulting in changes in the game not occurring too frequently & too fast poise at the edge of time poise at the edge of time: focus on present through attention on current iteration, requirements & design proposals taking into account existing version & design of future extensions; focus on past & future through overall project plan, project vision, frequent planning/feedback sessions; focus on present & future by sponsor & developers reflecting on development process emergent balance of exploitation & exploration manifested in current versions of the game, the consolidated game with standardised interface, developed formats for management & communication, inclusion of further projects participants in testing & evaluations activities table 2: cas characteristics of the game development project 5.2 interconnected, autonomous agents and the emergence of team learning in the game development project, all participants acted as autonomous, interconnected agents. the different distributed groups of participants were interconnected through various social and technical mechanisms. technologies, mostly digital ones, such as telephone, email, skype, instant messaging and chat were mainly used to connect in what kazman and chen (2009) call the kernel of a crowdsourced project, while email and primarily facebook were used in the periphery and by the masses in the crowd (kazman and chen 2009). the sponsor and the social media facilitators acted as social mechanisms and implemented simple rules of engagement to coordinate design proposals and other input from the periphery and the masses. the sponsor and the developers had developed another simple set of rules for the communication between them and within the development team. the participants’ autonomy was expressed in numerous ways. at the periphery the pacific islander youth requirements contributors, the various unicef staff as well as the testers voluntarily joined the project, provided their input, and left the project as they saw fit. the social media facilitators australasian journal of information systems kautz, bjerknes, fisher & jensen 2020, vol 24, selected papers from acis 2018 complex adaptive system theory in contemporary isd 16 autonomously communicated with the requirements contributors and filtered their input for the sponsor and the developers. the autonomy of the developers showed when they contributed to the functional design where the sponsor then made the final decisions. their autonomy became even more apparent in the liberty that the developers had when distributing and picking tasks. further, when implementing the games, the developers acted as self-governing with regard to the technical design decisions they made. despite that autonomy, the project participants were highly interlinked and maintained their relationships through the above described structures and measures, which supported the various described forms of interactions to achieve interconnectivity. a result of autonomy, the different capabilities of autonomous, interconnected agents and their interactions is the emergence of team learning (mitleton-kelly 2003). the sharing of project-relevant knowledge in the scheduled feedback sessions and frequent meetings, combined with all the kernel team members’ involvement in project management and design decisions, in addition to the developers’ self-assignment of tasks based on competence and interest, led to the emergence of collective learning among the participants in the kernel of the project. they also learned about the crowdsourced development and utilization of digital games and the issue at hand, climate change. through the involvement of other participants in design and evaluation activities the emergent learning spread to, and in, these other involved groups of participants as well. 5.3 emergent self-organisation and the emergence of order the concept of self-organisation departs from the command and control philosophy of traditional organisations. it places emphasis on increased autonomy, delegated decision making, more interactions with other individuals and the environment. individuals and teams must still define and follow local rules and allow these rules to evolve over time in the course of self-organisation. self-organisation is closely related to the concept of interconnected, autonomous agents (volberda and levin 2003). the game development project showed all these characteristics. the project was not led and controlled top-down but managed in a rather egalitarian manner. the sponsor acted primarily as a facilitator and co-ordinator. after having negotiated the communication protocol with the developers and the social media facilitators he created an environment with short communication paths that fostered self-organisation of the developers which was characterised by task self-assignment and largely autonomous decision making and joint responsibility. one of them, self-appointed and accepted by the other two, co-ordinated their work internally in their team. in fact, all those in facilitator roles appeared more like peers and were part of a very flat organisational structure as they contributed to the environment that nurtured self-organisation. self-organisation was further evidenced by the way the social media facilitators solicited and filtered the requirements which were provided by the other youth, and the way these youth offered their ideas. the implementation of the communication protocols and the introduction of the role of the social media facilitators illustrate the emergence of self-organisation of autonomous participants and the subsequent emergence of order in the design and development process of the project. it also shows that individual and team discipline are not in conflict with, but a vital element of, self-organisation. in this context, the way feedback beyond the self-organised regular feedback sessions was provided and gathered from the other participants and subsequently handled in orderly form, as part of planning activities, also reflects the emergence of order. australasian journal of information systems kautz, bjerknes, fisher & jensen 2020, vol 24, selected papers from acis 2018 complex adaptive system theory in contemporary isd 17 5.4 poise at the edge of chaos and the region of emergent complexity the game development project poised at the edge of chaos as it was constantly in a state of bounded instability which means that it, paradoxically, was simultaneously stable and unstable (stacey 2003). the sponsor’s initial project plan and vision as well as his specification of the overall requirements, the developers’ formal contract, and the organisation of a kernel consisting of the sponsor and the developers, acted as super-ordinate structuring mechanisms that created a relative stable space within which the development process and the various versions of the game could unfold. however, it also had to deal with the instability brought about by the continuous flow of ideas, requirements, change requests, and feedback caused by the involvement of the different participant groups which were spread over several continents and time zones. these made up the edge of chaos where complexity emerged. the project balanced this complexity and coped with the ‘chaos’ (benbya and mckelvey 2006) by emergent organisation of the participants in (1) the before mentioned separate stable kernel which had a decision mandate and performed the functional and technical design as well as the technical development tasks, (2) a dispersed periphery which provided, facilitated and filtered requirements and feedback and (3) independent and even less stable masses who provided ideas and requirements. this organisational form allowed for the steady and flexible handling of, and swift reaction on the frequently incoming input through short iterations of manageable task size, as well as through the regular planning and feedback sessions of the participants in the kernel of the project. it illustrates how the project manoeuvred in a region of emergent complexity, and balanced at the edge of chaos as these measures at the same time supported the necessary flexibility and provided a frame for stability, as they structured the project participants’ activities and helped those in the kernel to know what to do, when to do it, and what to expect from others. 5.5 emergent co-evolution and the emergence of behaviour and structure co-evolution emerged in the game development project through the above described multiple forms of interactions in which the distributed participants shared knowledge and learned from each other. the mutual learning had the reciprocal effect of reinforcing the emerging structures of collaboration and interaction, as well as the behaviour of the individual project participants. in particular, the continuous provision, filtering, and handling of ideas, requests, requirements, and other feedback, along with the frequent availability of early versions of the game fuelled this process. it kept the participants informed about the current status of the game and provided opportunities to explore, evaluate, and learn how to use it, as well as to create new ideas that were then fed back to the development team to become part of the next version. thus, the game co-evolved with the distributed participants. together, this demonstrates the co-evolution of people, processes and products (meso and jain 2006). the project also exhibited the unpredictable emergent behaviour and structure of the different entities of a cas based on the described co-evolution during the distributed design and development process. the organisational structure of the dispersed participants was not planned on beforehand nor was the inclusion of certain participants. the youth requirement contributors, the unicef staff and the climate experts joined the project voluntarily when being called upon and left it when they had decided to do so without any notice; they were largely unknown to each other and the other participants. further, the idea of involving other young people such as the testers or the requirements contributors in further feedback cycles on the design and early versions of the game emerged during the developers’ interactions. when implemented, the feedback changed the game and its behaviour accordingly. australasian journal of information systems kautz, bjerknes, fisher & jensen 2020, vol 24, selected papers from acis 2018 complex adaptive system theory in contemporary isd 18 5.6 time pacing and the emergence of rhythm in the game development project, the overall project plan and the developers’ contract set the time frame for the dpd and development process. the short iterations made early versions of the games available. the emergence of a lasting working rhythm and the setting of the pace for the project were supported by the virtual planning meetings and feedback sessions, held weekly or more often, that handled the continuously incoming input from the participants at the periphery and the masses. this was achieved through the agreed rules for coordination, communication, and engagement, as well as mechanisms for selecting the ideas, requests, and requirements from the different dispersed participants. as a result, the versions of the game did not change too frequently or too fast. together with the manageable size of the requirements, it provided the appropriate intervals to match and handle the changes. in this way, time pacing, as reported by vidgen and wang (2009) and the emerging internal rhythm drove change in the project in accordance with the passage of time and, at the same time, allowed for stability and flexibility. 5.7 poise at the edge of time and the emergent balance of exploitation and exploration in cas theory a focus on the present while keeping the past in mind and preparing for the future is regarded as poising at the edge of time (brown and eisenhardt 1998). in the game development project, the centre of attention was always the current iteration and the current requirements and design proposals while also taking into account the existing version of the game and the design for future extensions. the developers built the game through several iterations where they at any point took the available knowledge and requirements concerning the development process, the game development, and the issue at hand, climate change into consideration, while investigating further options, receiving new ideas, requirements and feedback. the developers exploited existing knowledge by including links to other information resources, by using accessible code from other games, and not least by sharing code between themselves. they used ideas, requirements and proposals with their roots in the presence and simultaneous awareness of the past and the future approved by the sponsor to produce a current version of the game whilst investigating prospective options with information and feedback from, and in consultation with, the sponsor, but also the social media facilitators, the youth requirements contributors, local and headquarter unicef staff, international climate change experts, and the three testers. in doing so the project concurrently balanced the exploitation of existing knowledge and the exploration of new knowledge at the edge of time (bocanet and ponsiglione 2012). this emerging balance was supported through the frequent virtual meetings and feedback sessions between the developers themselves and between them and the sponsor as well as the other coordination and filtering mechanisms. the overall project plan and vision, as well as the frequent planning sessions structured around ‘releases’ and iterations of current versions of the game, supported a focus on, and constituted a manifestation of, both the past and the future. the frequent feedback sessions with the sponsor were also used by the developers to think about their own behaviour and to review and improve the development process. this is reflected in the developed formats for management and communication in the core team, the consolidated game with a standardised interface as well as the inclusions of further youth into the distributed testing and evaluations activities. australasian journal of information systems kautz, bjerknes, fisher & jensen 2020, vol 24, selected papers from acis 2018 complex adaptive system theory in contemporary isd 19 6 conclusion prior research into pd centred to a large extent on industrial environments with a focal organisation with some exceptions notably in open source software development in the notfor-profit arena (kazman and chen 2009; lukyanenko et al. 2016). most of the work on dpd is based on action research projects with significant intervention by researchers and only to a lesser extent on actual empirical practice studies without or only marginal direct influence by a research team on the course of the dpd project; dpd research also focusses heavily on individual methods, techniques, and practices (gumm et al. 2006; obendorf et al. 2009; titlestad et al. 2009; näkki and koskela-huotari 2012; lukyanenko et al. 2016). in contrast, ours is a case study of genuine dpd through an intergovernmental, not-for-profit organisation – an organisation that is neither a part of a government nor a conventional for-profit business – and mainly youth in an isd project of a digital game. our analysis provides an in-depth understanding of how the project was managed and performed in its entirety in a not-forprofit environment. it reveals a complex network of geographically dispersed actors in a transient project organisation. we demonstrate that the game development project can be understood as an example of dpd in contemporary, crowdsourced isd. furthermore, by applying cas theory we show that the project can be understood as a cas and that cas theory provides explanations for how and why dpd as an approach to contemporary isd worked in the investigated case. the validation of our empirical results through the application of cas theory contributes to the growing literature that acknowledges cas theory as a relevant theoretical foundation for understanding contemporary dpd and isd. researchers can use cas theory to perform, analyse, present and compare longitudinal case studies of how dpd, and more general isd unfolds in practice over time. this is crucial as social science and is researchers (see e.g. eisenhardt 1989; walsham 1995; van de ven 2007) highlight that the complexity of practice is such that a theory and an explicit framework of ideas are necessary as a guide for data collection and identification of important research findings. detailed studies of practice and subsequent formulation of empirically grounded theories serve to enhance researchers and practitioners’ knowledge and to introduce new concepts that both groups can bring to their respective practice (madsen et al. 2006). our work has practical bearings, too. it shows how actual dpd can be organised in a project to result in a process and outcome that all stakeholder groups appreciate. in practice, while recognising that the actual course of an isd project will evolve with the situation, cas can be used for: (1) managing and performing dpd during the development process by providing an understanding of dpd as an approach to isd, and (2) after-the-fact reflection and collection of lessons learnt. in conclusion, we acknowledge that our study is an exploratory, single case study and that the game development project belongs to a special class of development project which may limit the generality of our findings. but, like walsham (1995) and klein and myers (1999), we contend that this does not mean that it does not contribute to in our case a sound empirical practice study and rich insight about dpd as a vital approach to isd the collective body of knowledge, both academic and practical, of a discipline. while our research provides a link between the otherwise often disconnected research areas and research communities of dpd, isd and cas, still more studies are necessary to allow for more theorising and for a viable theory of dpd in isd. to accomplish a more exhaustive explanatory theory, to answer why australasian journal of information systems kautz, bjerknes, fisher & jensen 2020, vol 24, selected papers from acis 2018 complex adaptive system theory in contemporary isd 20 dpd in isd played out the way it did in the presented case and to draw more general lessons learnt, further research is needed. references alvesson, m., and sandberg, j. 2011. “generating research questions through problematization,” academy of management review (36:2), pp. 247-271. alvesson, m., and sandberg, j. 2013. “has management studies lost its way? ideas for more imaginative and innovative research,” journal of management studies (50), pp. 128-152. anderson, p. 1999. “complexity theory and organization science,” organization science (10:3), pp. 216–232. baiyere, a. 2017. “the interface of it capabilities and disruptive innovation,” in proceedings of the 28th australasian conference on information systems (acis 2017), hobart, tas, australia, december 4 6, 2017. benbya, h., and mckelvey, b. 2006. “toward a complexity theory of information systems development,” information technology & people (19:1), pp. 12-34. boell, s. k., and cecez-kecmanovic, d. 2014. “a hermeneutic approach for conducting literature reviews and literature searches,” communications of the association for information systems (34), article 12. brown, s., and eisenhardt, k. 1998. competing on the edge: strategy as structured chaos, boston, ma: harvard business school press. bocanet, a., and ponsiglione, c. 2012. “balancing exploration and exploitation in complex environments,” vine the journal of information and knowledge management systems (42:1), pp. 15-35. chang, y.-j., and chang, y.-s. 2011. “investigation of organizational interaction and support in an ngo through computer-mediated discussions,” journal of educational technology & society (14:3), pp. 130-140. cilliers, p. 2000. “what can we learn from a theory of complexity?” emergence (2:1), pp. 2333. clement, a. 1994. “computing at work: empowering action by ‘low-level users’,” communications of the acm (37:1), pp. 52-63. clement, a., costantino, t., kurtz, d., and tissenbaum, m. 2008. “participatory design and web 2.0: the case of pipwatch, the collaborative privacy toolbar,” in proceedings of the 10th conference on participatory design, bloomington, in, usa, acm, pp. 51-60. cockerill, c.h. 2013. “exploring social media obstacles and opportunities within public agencies: lessons from the ohio division of wildlife,” international journal of business and social science (4:2), pp. 39-44. creswell, j. 2003. research design qualitative, quantitative and mixed methods approaches, thousand oak, ca: sage publications. disalvo, c., clement, a., and pipek, v. 2013. communities: participatory design for, with and by communities. in j. simonsen and t. robertson, t. (eds.), routledge international handbook of participatory design, ny: routledge, pp. 218-248. australasian journal of information systems kautz, bjerknes, fisher & jensen 2020, vol 24, selected papers from acis 2018 complex adaptive system theory in contemporary isd 21 disalvo, b., and disalvo, c. 2014. “designing for democracy in education: participatory design and the learning sciences,” in proceedings of the 11th international conference of the learning sciences, boulder, cl, usa. eisenhardt, k. 1989. “building theories from case study research,” the academy of management review (14:4), pp. 532-550. fincham, r. 2002. “narratives of success and failure in systems development,” british journal management (13), pp. 1-14. fisher, j. 2012. “engaging pacific youth through a facebook game,” acm inroads (3:4), pp.7985. gerring, j. 2007. case study research: principles and practices, cambridge: cambridge university press. ghazawneh, a., and henfridsson, o. 2013. “balancing platform control and external contribution in third-party development: the boundary resources model,” information systems journal (23:2), pp. 173-192. gumm, d.c., janneck, m., and finck, m. 2006. “distributed participatory design – a case study,” in proceedings of the nordichi workshop on distributed participatory design, vol. 2, oslo, norway. hamari, j., koivisto, j., and sarsa, h. 2014. “does gamification work? – a literature review of empirical studies on gamification,” paper presented at the 47th annual hawaii international conference on system sciences. henderson, a., and bowley, r. 2010. “authentic dialogue? the role of "friendship" in a social media recruitment campaign,” journal of communication management (14:3), 237-257. highsmith, j. 2000. adaptive software development: a collaborative approach to managing complex systems, ny: dorset house publishing. highsmith, j. 2002. agile software development ecosystems, boston, ma: addison-wesley. holland, j. h. 1995. hidden order: how adaptation builds complexity, reading, ma: addisonwesley. holland, j. h. 1998. emergence: from chaos to order, cambridge, ma: perseus publishing. jacucci, e., hanseth, o., and lyytinen, k. 2006. “taking complexity seriously in is research. introduction to the special issue,” information technology & people (19:1), pp. 5-11. jain, r., and meso, p. 2004. “theory of complex adaptive systems and agile software development,” in proceedings of the 10th americas conference on information systems, new york, ny, pp. 1661-1668. kanuha, v. k. 2000. ““being native" versus “going native”: conducting social work research as an insider,” social work in health care (45:5), pp. 439-447. kauffman, s. 1993. the origins of order: self-organisation and selection in evolution, ny: oxford university press. kautz, k. 2009. “improvisation in information systems development practice,” journal of information technology case and application research (11:4), pp. 30-59. australasian journal of information systems kautz, bjerknes, fisher & jensen 2020, vol 24, selected papers from acis 2018 complex adaptive system theory in contemporary isd 22 kautz, k. 2012. “beyond simple classifications: contemporary information systems development projects as complex adaptive systems,” in proceedings of the 33rd international conference on information systems, orlando, fl. kautz, k., and madsen, s. 2010. “understanding agile software development in practice,” in proceedings of the 2010 international conference on information resources management, rose bay, jamaica. kautz, k., bjerknes, g., fisher, j., and jensen, t. 2019. “the process of co-creation in information systems development: a case study of a digital game development project,” in andersson, b. et al. (eds), advances in information systems development: designing digitalization, lecture notes in information systems and organisation, springer publishing company, cham, switzerland, vol. 34, pp .187-206. kazman, r., and chen, h. 2009. “the metropolis model a new logic for development of crowdsourced systems,” communications of the acm (52:7), pp. 76-84. kelly, k. 1994. out of control the new biology of machines, social systems, and the economic world, new york, ny: addison-wesley. kensing, f., and blomberg, j. 1998. “participatory design: issues and concerns,” computer supported cooperative work (7:3), pp. 167-185. klein, h., myers, m.d. 1999. “a set of principles for conducting and evaluating interpretive field studies in information systems,” mis quarterly (23:1), pp. 67-93. lee, a. s. 1994. “electronic mail as a medium for rich communication: an empirical investigation using hermeneutic interpretation,” mis quarterly (18:2), pp. 143-157. lukyanenko, r., parsons, j., wiersma, y. f., sieber, r., and maddah, m. 2016. “participatory design for user-generated content: understanding the challenges and moving forward,” scandinavian journal of information systems (28:1), pp. 37-70. madsen, s., kautz, k., and vidgen, r. 2006. “a framework for understanding how a unique and local is development method emerges in practice,” european journal of information systems (15:2), pp. 225-235. markus, m. l., and mao, j.-y. 2004. “participation in development and implementationupdating an old, tired concept for today’s is contexts,” journal of the association for information systems (5:11), pp. 515–544. mcmillan, e. 2004. complexity, organizations, and change, london, uk: routledge. merali, y., and mckelvey, b. 2006. “using complexity science to effect a paradigm shift in information systems for the 21st century,” journal of information technology (21:4), pp. 211-215. meso, p., and jain, r. 2006. “agile software development: adaptive systems principles and best practices,” information systems management (23:3), pp.19-30. miles, m. b., huberman, m. a., and saldana, j. (2014). qualitative data analysis (3rd ed.), london: sage. mitleton-kelly, e. 2003. “ten principles of complexity and enabling infrastructures,” in mitleton-kelly, e. (ed.), complex systems and evolutionary perspectives on organisations: the application of complexity theory to organisations, oxford, uk: elsevier science ltd, pp. 3-20. australasian journal of information systems kautz, bjerknes, fisher & jensen 2020, vol 24, selected papers from acis 2018 complex adaptive system theory in contemporary isd 23 näkki, p., and koskela-huotari, k. 2012. “user participation in software design via social media: experiences from a case study with consumers,” ais transactions on humancomputer interaction (4:2), pp. 129-152. obendorf, h., janneck, and finck, m. 2009. “inter-contextual distributed participatory design: communicating design philosophies and enriching user experience,” scandinavian journal of information systems (21:1), pp. 50-76. öberg, k.d., gumm, d., and naghsh, a.m. 2009. “a special issue editorial distributed pd: challenges and opportunities,” scandinavian journal of information systems (21:1), pp. 2326. sarker, s., sarker, s., and sidorova, a. 2006. “understanding business process change failure: an actor-network perspective,” journal of management information system (23:1), pp. 5186. schön, d.a. 2003. the reflective practitioner. how professionals think in action, ny: basic books. stacey, r. d. 2003. strategic management and organisational dynamics: the challenge of complexity (4th ed.), harlow, uk: financial times, prentice hall. titlestad, o.h., staring, k., and braa, j. 2009. “distributed development to enable user participation: multilevel design in the hisp network,” scandinavian journal of information systems (21:1), pp. 27-50. truex, d.p., baskerville, r., and klein, h. 1999. “growing systems in emergent organizations,” communications of the acm (42:8), pp. 117-123. van de ven, h.a. 2007. engaged scholarship a guide for organisational and social research, ny: oxford university press. venkatesh, v., brown, s.a, bala, h. (2013). “bridging the qualitative–quantitative divide: guidelines for conducting mixed methods research,” mis quarterly (37:1), pp. 21-64. vidgen, r., and wang, x. 2006. “organizing for agility: a complex adaptive systems perspective on agile software development process,” in ljungberg j., and andersson, m. (eds.) proceedings of the 14th european conference on information systems, gothenburg, sweden, pp. 1316-1327. vidgen, r., and wang, x. 2009. “coevolving systems and the organisation of agile software development,” information systems research (20:3), pp. 355-376. volberda, h. w., and levin. a. y. 2003. “guest editors’ introduction: coevolutionary dynamics within and between firms: from evolution to co-evolution,” journal of management studies (40:8), pp. 2111-2136. walsham, g. 1995. “interpretive case studies in is research: nature and method,” european journal of information systems (4:2), pp.74-81. copyright: © 2020 kautz, bjerknes, fisher & jensen. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems kautz, bjerknes, fisher & jensen 2020, vol 24, selected papers from acis 2018 complex adaptive system theory in contemporary isd 24 which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v24i0.2225 1 introduction 2 theoretical background: cas theory 3 research approach and method 4 a case narrative 4.1 phase 1 – initiation of the idea and funding 4.2 phase 2 – establishment of the team 4.3 phase 3 – conceptual design of the game 4.4 phase 4 development of the consolidated games 4.5 phase 5 launch of the consolidated game 5 analysis and discussion: the game development project as a complex adaptive system 5.1 interactions 5.2 interconnected, autonomous agents and the emergence of team learning 5.3 emergent self-organisation and the emergence of order 5.4 poise at the edge of chaos and the region of emergent complexity 5.5 emergent co-evolution and the emergence of behaviour and structure 5.6 time pacing and the emergence of rhythm 5.7 poise at the edge of time and the emergent balance of exploitation and exploration 6 conclusion paper title (paper title style) australasian journal of information systems cronholm, gӧbel & shrestha 2024, vol 28, research article collaboration in action design research received 7 sep 2023, accepted 4 july 2024 after 1 round of review with karlheinz kautz as accepting editor 1 researcher-practitioner collaboration in action design research stefan cronholm university of borås, sweden stefan.cronholm@hb.se, hannes göbel university of borås, sweden anup shrestha school of business university of southern queensland, australia abstract action design research (adr) is a well-known research method within design science research (dsr). an essential characteristic of the adr method is the need for researcherpractitioner collaboration (rpc). while there is abundant research on rpc regarding information systems projects in general concerning explanatory and normative knowledge, there is very limited prescriptive knowledge on how to execute rpc in adr projects. successful collaboration in adr projects is imperative since the development of sociotechnical it artefacts requires frequent interaction in organisational contexts. however, rpc can be hard to manage due to competing interests. therefore, the purpose of this paper is to present prescriptive knowledge for how to manage rpc. we have analysed a collaborative adr project consisting of several researchers and practitioners. based on a grounded theory approach, we have developed theoretical models based on challenges identified in an adr project. the models provide prescriptive knowledge regarding: shape the it artefact based on organisational intervention, exploit the mutual dependency between developing design principles and it artefacts, and contextualise and generalise learning. each model involves logical relationships between: conditions for the challenges, actions taken to address the challenges and consequences of the actions taken. the guidelines were deducted from the models and consist of recommendations that could be considered in future adr projects. keywords: researcher-practitioner collaboration, action design research, adr projects, design science research, academy-industry collaboration. 1 introduction design science research (dsr) has gained a lot of interest and has been established as a widely accepted research approach in information systems (is) (e.g., gregor & hevner, 2013; vaishnavi & kuechler, 2015). one purpose of dsr is to design it artefacts which are broadly defined as constructs (vocabulary and symbols), models (abstractions and representations), methods (algorithms and practices), and instantiations (implemented and prototype systems) (hevner et al., 2004). this increasing interest has created a need for specific dsr methods (venable et al., 2017; cronholm & göbel, 2019). one of the most popular research methods within dsr is action design research (adr) which draws on both design research (dr) and action research (ar) (sein et al., 2011). with over a thousand citations indexed in scopus and over 2000 citations according to google scholar, the adr method is one of the most cited dsr methods. mailto:stefan.cronholm@hb.se australasian journal of information systems cronholm, gӧbel & shrestha 2024, vol 28, research article collaboration in action design research 2 our analysis of the adr method has revealed three significant characteristics. the first characteristic is the idea of combining dr and ar, which involves that the design of it artefacts should be based on intervention in authentic organisational settings. the second characteristic is to respond to the dual mission of making theoretical contributions and assisting in solving the problems of practitioners. this double task stresses the importance of developing design knowledge such as design principles (as a form of nascent theory) and it artefacts in parallel. it also accentuates the necessity of planning adr projects to meet this requirement. the third characteristic of the adr method is the strong encouragement for the generalisation of qualitative studies. sein et al. (2011) state that problems, solutions and design principles should be generalised. this means that a contextual problem should be cast as an instance of a class of problems and that the designed artefact should be regarded as an instance of a solution class (ibid.). one purpose of generalisation is to reconceptualise learning to be valid more broadly outside a single-case study. the three characteristics target practitioners and the practitioners’ organisations, and hence several scholars regard the adr method as a highly collaborative researcher-practitioner approach (e.g., petersson & lundberg, 2016; haj-bolouri et al., 2018; cronholm & göbel, 2019). it is evident that sein et al. (2011) share this view when they state that “researchers bring their knowledge of theory and technological advances, while the practitioners bring practical hypotheses and knowledge of organizational work practices” (p.43). this collaborative aspect of the adr method aims to increase the organisational relevance of the designed artefact and encourage interaction between researchers and practitioners. against this backdrop, we can conclude that a) the adr method relies heavily on successful researcher-practitioner collaboration (rpc) in context and b) the three adr characteristics influence how rpc is organised in adr projects. based on an analysis of an adr project (see section 5), we have found that the adr method provides excellent guidance regarding rpc on why to collaborate and what to collaborate on. however, the method provides limited literature support on how to collaborate, i.e., prescriptive knowledge. the lack of prescriptive knowledge could have a negative impact on project effectiveness and efficiency, the relationship between researchers and practitioners, and research goals (segal, 2022). our observation is also supported by other scholars who state that there is a need for prescriptive guidelines on rpc in adr projects (collatto et al., 2017; haj-bolouri et al., 2018; cronholm & göbel, 2019; henriques & o’neill, 2021). for example, henriques & o’neill (2021) emphasise that rpc collaboration is paramount in design-oriented projects in order to find the best solutions for socio-technical problems and that additional methods and guidance are required. as a complement to the existing guidance on why to collaborate and what to collaborate on, the purpose of this study is to suggest prescriptive knowledge (how to do) for rpc in adr projects. prescriptive knowledge is presented as a) theoretical models consisting of structures that explain relationships between conditions for rpc challenges, actions taken to address the challenges, and consequences of the actions taken and as b) guidelines that could be used in future adr projects. we define a challenge as something that is regarded, either by researchers or practitioners, as a threat to successful collaboration (cronholm & göbel, 2022). a guideline is defined as practical information intended to advise people on how something should be done or what should be done about something (cambridge dictionary, 2023). we state that the theoretical models and guidelines advance existing knowledge concerning rpc in adr australasian journal of information systems cronholm, gӧbel & shrestha 2024, vol 28, research article collaboration in action design research 3 projects. the models and guidelines can support future adr projects and possibly other research dsr or is projects involving collaboration between researchers and practitioners. therefore, they target both researchers and practitioners. our research question reads: how can rpc challenges in adr projects be managed? understanding challenges related to researcher-practitioner collaboration is critical to project planning, realisation and evaluation. therefore, we have analysed an empirical research project guided by the adr method, which we refer to as “the adr project” in this article, in order to identify critical themes concerning rpc. this adr project was chosen due to its composition of several researchers and practitioners who frequently interacted in order to fulfil the project goals. the following section presents a literature review concerning researcher-practitioner collaboration in adr projects. after that, we describe the analysed adr project. next, we explain our research design. then, we present the identified themes concerning researcherpractitioner collaboration. next, we describe the theoretical models, followed by a presentation of the guidelines. after that, we discuss the findings. finally, we draw conclusions and describe implications of our work to theory and practice. 2 literature review in order to identify existing knowledge on rpc within adr projects, we searched the scopus database, which is the largest within is. the keywords used were “action design research” and “researcher-practitioner collaboration”. the reason for explicitly including these keywords was that we wanted to identify previous knowledge about the significant rpc characteristics of the adr method (see section 1). the search within the scopus database returned only seven articles. out of these seven articles, three neither discussed collaboration nor adr projects, and one of the articles reflected upon researcher-industry collaboration but not in adr projects. only three articles involved reflections on researcher-practitioner collaboration in adr projects. therefore, we expanded our literature search to include backward reference searching (i.e., snowball sampling, e.g., naderifar et al., 2017), which meant that we reviewed relevant articles cited in the returned articles from the scopus database. the expanded literature search resulted in four additional articles regarding rpc in adr projects. the total return of seven articles limits the pool of extant wisdom that could be drawn upon to suggest ways to conduct rpc (see section 2.1). as mentioned in section 1, the adr method draws on action research. therefore, we extended the literature review further to also include experiences from action research (e.g., rapoport, 1970; susman & evered, 1978), (see section 2.2). as mentioned in section 1, the adr method draws on both action research and design research. 2.1 knowledge about rpc identified in adr projects first, bilandzic and venable (2011) have proposed an adapted research method called participatory action design research, which emphasises that the practitioner’s involvement should be extensive from the initial design to the final presentation of results. their view of practitioners is that they are regarded as both subjects and co-researchers, which is similar to the one by argyris and schön (1989, p.613), who state that “causal inferences about the behaviour of human beings are more likely to be valid and enactable when the human beings in question participate in building and testing them”. second, matzner et al. (2018) have used australasian journal of information systems cronholm, gӧbel & shrestha 2024, vol 28, research article collaboration in action design research 4 the adr method in order to develop a framework consisting of core rpc concepts in service research. the framework consists of an iterative process involving the following phases: a) formulate research needs, societal goals and political goals, b) formulate research and business goals, c) carry out design, analysis, development and full launch, and d) describe research and business outcomes. third, haj-bolouri et al. (2016) have extended the adr method by adding a front-end component, which is inspired by participatory action research and participating design. one conclusion is that learning should be focused on all adr stages. in addition, a participatory approach involving both researchers and practitioners co-creating knowledge should be explicit at each step of the adr method. fourth, haj-bolouri et al. (2018) have further analysed how scholars use the adr method. they have identified three topics concerning rpc: a) how to balance the competing interests of organisational stakeholders with the interests of a research community, b) how to balance the practitioner need of a situated implementation of the designed itartefact against the research need to produce generalised knowledge and c) how to balance the findings between specific and generalisable research outcomes. based on the three topics, haj-bolouri et al. (2018) have identified 11 themes that describe the use of adr in practice. the themes are primarily problem-oriented, although some offer practical rpc advice such as a) presenting careful mutual elaboration of the key research issues, and b) organising workshops to involve stakeholders and identify essential priorities. furthermore, haj-bolouri et al. (2018, p.144) state that “due to reason that the methodology [the adr method] is described in broad terms in the original paper by sein et al. (2011), applications of the methodology in projects have revealed recurring issues that researchers and practitioners face in practice”. fifth, vom brocke et al. (2021) argue that dsr (including adr) should be conducted within a continuous dialogue of research and practice using a dance metaphor. they state that academia and practice have to move together according to a joint “rhythm”; sometimes closer together, sometimes more on their own; sometimes more active, sometimes more passive; but in any given situation, there should be a mutual commitment towards a joint performance and the accomplishment of the project. they have suggested four guidelines to maximise the practical relevance: establishing a continuing relationship of mutual trust and respect, studying design where it happens, seeking and embracing opportunities to make an impact, and performing the dsr dance “on the fly”. finally, cronholm and göbel (2022) have analysed an adr project in order to identify how rpc challenges could be tackled. the paper is presented as a conference paper and is therefore regarded as research in progress. the identified challenges in that paper were: researcher intervention in practitioner contexts, reciprocal shaping between artefacts and design principles that emerged from theory and practice, and researcher and practitioner learning. the research presented in our study extends the previous research-in-progress publication through: a) a renewed analysis of prior knowledge of rpc in adr projects, b) a re-analysis of data, and c) enhanced theoretical models demonstrating rpc challenges including actions taken. 2.2 knowledge about rpc identified in action research first, rapoport (1970, p.499) defines action research as a research method that “aims to contribute both to the practical concerns of people in an immediate problematic situation and to the goals of social science by joint collaboration within a mutually acceptable ethical framework”. moreover, he identifies three dilemmas: ethical dilemmas, goal dilemmas and australasian journal of information systems cronholm, gӧbel & shrestha 2024, vol 28, research article collaboration in action design research 5 dilemmas of initiatives. he concludes that the action researcher seeks to optimise ta balanced realisation of both the practical affairs of humans and the intellectual interests of the scientific community. second, susman and evered (1978) present a method for action research that is proposed as an alternative to procedures emanating from positivist science. the cyclical method consists of five phases: diagnosing, action planning, action taking, evaluating and specifying learning. one purpose of the method is to support collaboration between researchers and practitioners and to generate theory grounded in action. furthermore, the authors state that action research is future-oriented, collaborative, implies systems development, generates grounded theory, is agnostic and situational. third, avison (2001) discusses control mechanisms for action research projects. the primary purpose of the article is to suggest how the "double challenge" of combining both action and research can be overcome. he suggests three aspects of control that action researchers need to focus on: initiation (did the problem discover the research or vice versa?), authority (who is really in charge of the research project?) and formalisation (the degree of formal control structures such as contracts and letter of agreements). fourth, mathiassen (2002) reports from a research project in which researchers and practitioners worked together to understand, support, and improve systems development practices in four organisations. he proposes collaborative practice research to organise and conduct research into systems development practice based on close collaboration between researchers and practitioners throughout the whole process. moreover, he presents four lessons learnt: organise collaborations as a loosely coupled system of related agendas; implement full learning cycles of understanding; supporting and improving practice; combine action research; experiments and practice studies; and establish basic documentation systems to support longitudinal practice studies. finally, te’eni et al. (2017, p.542) discuss how research and practice could be made more productive. for example, the authors suggest “… for researchers to influence practice, they need to engage in dialog with practitioners. it is as simple as that”. furthermore, they argue that practitioners must hear and be heard when it comes to discussions about research of meaningful problems. the authors suggest a new procedure to bring practitioners into the discussion by developing and disseminating reviews of selected research articles that are tailored to the needs and the language of practitioners. one purpose of the procedure is to support researchers to influence practice and to engage researchers and practitioners in a structured dialogue. 2.3 summary of literature review our analysis of the existing literature has provided valuable insights regarding rpc. however, we can conclude that there is a need for developing complementary prescriptive knowledge about rpc supporting adr projects. we have found that: a) our literature review only revealed seven articles presenting rpc challenges in adr projects. consequently, there is limited knowledge on this topic; b) all the reviewed articles have developed models, methods or principles that involve different aspects of rpc. the type of knowledge identified is mainly explanatory (why to do something) and normative (what to do) but not prescriptive (how to do); c) the literature review also uncovers that several scholars agree on the need to develop additional knowledge concerning rpc and intervention in organisational contexts australasian journal of information systems cronholm, gӧbel & shrestha 2024, vol 28, research article collaboration in action design research 6 (e.g., haj-bolouri et al., 2018; cronholm & göbel, 2022) and d) there is a lack of guidelines concerning actions to take in order to address rpc challenges. 3 the adr project we have analysed a research project guided by the adr method (sein et al., 2011). the adr method consists of four stages which are: a) problem formulation: to identify and conceptualise research opportunities based on existing theories and technologies; b) building, intervention and evaluation: to realise the design of the artefact and articulate the design principles; c) reflection and learning: to move conceptually from building a solution for a particular instance to applying that learning to a broader class of problems; and d) formalising of learning: the situated learning from an adr project should be further developed into general solution concepts, (see sein et al., 2011 for a detailed description of the adr method) (see figure 1). figure 1. the stages of the adr method (sein et al., 2011) the point of departure for this adr project was that digital innovation is increasingly contributing to the growth of business. the awareness of the importance of service perspective and data analytics for increased innovation capability has encouraged organisations to search for digital tools to support their innovation efforts. however, the adr project identified a lack of existing useful digital tools that assist towards achieving innovation. consequently, the purpose of the adr project was to develop design principles and a digital tool supporting australasian journal of information systems cronholm, gӧbel & shrestha 2024, vol 28, research article collaboration in action design research 7 organisations in their utilisation of data to improve the digital innovation process. the adr project involved four researchers and nine organisations in sweden. the organisations belonged to multiple sectors, were of different sizes (small, medium, and large), and were classified as service providers or service customers in the private or public sector. the industry sectors represented were the automobile industry, telecommunications, and it. the roles of the participating practitioners involved were the following: it quality manager, head of architecture and solutions, it consultants, manager of consumer services, business manager, ceos, it process framework manager, software developers, and manager consumer sales. the researchers consisted of two professors and two phd students from the field of is. the project was conducted over a period of three years. the adr project collected data from all the organisations by conducting interviews and organising workshops. each interview lasted for approximately 1 hour each and each workshop lasted for 2-3 hours. the interviews were recorded and transcribed. all researchers and practitioners were deeply involved in the adr project during the organisational interventions and in the design and evaluation of the digital tool. during the adr project, we realised that the adr project also provided an excellent opportunity to analyse rpc. this meant that we identified rpc challenges and actions taken to address the challenges in parallel with the fulfilment of the objectives of the adr project. the reasons for utilising the adr project for analysing rpc were: a) it consisted of frequent interactions between researchers and practitioners, which preferably took place in the participating organisations’ contexts, b) it provided access to organisational settings and c) it provided access to rich data from interventions in nine organisations. initially, there was a rift between the researchers’ and practitioners’ interests in the adr project. based on this observation, we realised that rpc is more difficult in “practice than on paper”. conboy (2009, p.329) states that it is essential to distinguish between the “documented method” and the “method-in-action”. the documented method is intended to serve as an ideal model for the development process (iivari & maansaari, 1998), while the method-in-action describes the way it may be used in a particular situation (fitzgerald et al., 2002). our study consists of an empirical analysis of an instantiation of the adr method. 4 research design in this paper, the result of the adr project is placed in the background, and the focus is on the rpc challenges that occurred in the adr project and actions taken to address these challenges. our overall research design has been both inductive and deductive. we commenced with an inductive analysis of an adr project which resulted in three themes regarding rpc. these themes were then generalised into three corresponding theoretical models. thereafter, the three models were integrated into an overarching model (see section 6). the models represent abstractions from the adr project. in the next step, we deducted three guidelines from the themes and models with the purpose of developing prescriptive knowledge that could help future adr projects to manage rpc challenges (see section 7). moreover, we have also compared our findings with previous knowledge (see section 8). we collected data from both a researcher and practitioner perspective. we used the following sources: a) rpc statements in the adr method, b) notes taken from discussions among the researchers and practitioners during project meetings and workshops, c) documentation of australasian journal of information systems cronholm, gӧbel & shrestha 2024, vol 28, research article collaboration in action design research 8 collaborative actions taken and d) project documentation. in total, data were collected from 15 project meetings and 23 interventions in the practitioners’ organisations (workshops). in order to collect data from practitioners, we conducted nine group interviews involving several representatives from each organisation. in order to collect data from a researcher perspective, we interviewed the researchers who intervened in the organisations and were active when designing and evaluating the digital tool. the rpc challenges were identified and analysed by the authors of this paper, who also participated in the adr project. this meant that the authors of this paper had dual roles: a) developing design principles and a digital tool supporting organisations in their utilisation of data to improve digital services and b) reflecting upon the adr project in order to create prescriptive knowledge concerning researcher-practitioner collaboration. in order to ensure that relevant data regarding rpc collaboration was collected, the three characteristics of the adr method described in section 1 were used as a lens for collecting data, i.e., a) the design of it artefacts should be based on intervention in authentic organisational settings, b) respond to the dual mission of making theoretical contributions and assisting in solving the problems of practitioners and c) encouragement for generalisation of qualitative studies. in order to analyse the collected data, we applied the grounded theory method (gtm) to analyse rpc in the adr project. grounded theory is a qualitative research method that seeks to develop theories grounded in data. there were several reasons for selecting gtm. urquhart et al. (2010) state that gtm “… has proved to be extremely useful in developing context-based, process-oriented descriptions and explanations of information systems phenomena” (p.358). charmaz (2017, p.34) views gtm from a pragmatic perspective and adds that gtm is “interrogating the taken-for-granted” and “taking a deeply reflexive stance”. moreover, gtm provides explicit, sequential guidelines for conducting qualitative research strauss and corbin, 1990). gtm exist in several types and versions (chun et al., 2019; goldkuhl & cronholm, 2019). we have followed the open, axial and selective coding steps suggested by strauss and corbin (1990). one reason for selecting this version is that it strongly supports the application of an action-oriented paradigm in the axial coding step. finally, in order to be transparent and claim rigour, we have followed the principles for qualitative research suggested by klein and myers (1999). in particular, we have applied the fundamental principles of the hermeneutic circle in order to understand the relationship between parts and the whole (i.e., the parts of the theoretical models and the core theme, see figures 3-6), the principle of contextualisation when identifying organisation-specific conditions, actions and consequences, the principle of abstraction and generalisation when developing the theoretical models, and the principle of multiple interpretations when interpreting the collected data. next, we describe how the three coding steps and the development of guidelines were implemented in this study. 4.1 open coding – generation of categories in interpretive approaches such as gtm, the analyst makes various decisions about how to comprehend the data (walsham, 1995). risks concerning biased interpretation can be reduced by involving two or more researchers when searching for and analysing data. therefore, two of the authors of this paper analysed the collected data individually with the purpose of creating categories and subcategories by grouping similar concepts. in the following step, the result of the individual analyses was jointly compared and reconciled. moreover, we also australasian journal of information systems cronholm, gӧbel & shrestha 2024, vol 28, research article collaboration in action design research 9 made comparisons between data collected from the nine participating organisations in order to identify similarities and differences. axial coding – identification of relationships between categories the purpose of the axial coding was to create theoretical models by identifying vertical and horizontal relationships between the categories. the vertical relationships in our analysis consist of superior categories and subcategories. the creation of horizontal relationships was supported by the action paradigm model consisting of three meta-categories: conditions, actions and consequences (strauss & corbin, 1990). a condition defines a specific circumstance and corresponds to the questions “why” or “what” (e.g., conditions for rpc challenges to appear). an action describes something that occurs under particular circumstances and corresponds to the question “how” (e.g., actions taken to address the condition). a consequence is a result dependent on the conditions and actions. it corresponds to the question of “what happens” if specific actions are carried out (e.g., consequences of the actions taken). the creation of horizontal relationships meant that the superior categories were additionally categorised in accordance with one of the meta-categories. we refer to units consisting of conditions, actions and consequences as themes. the themes were identified by iteratively moving back and forth between the open, axial, and selective coding steps. each theme has been illustrated as a theoretical model. selective coding – development of theory and identification of core category during the selective coding step, we developed an overarching theoretical model (core theme) that connected all the theoretical models created in the axial coding step. the purpose of the overarching model was to integrate the findings into a whole that represented the main topic of the study. finally, we checked the content of the theoretical models and the relationships between the themes in order to ensure consistency and coherency. this meant that we reanalysed the theoretical model in terms of structure, logic and uniformity. development of guidelines regarding rpc finally, we have developed prescriptive knowledge in terms of guidelines. gregor (2009) states that developing prescriptive knowledge involves creativity and imagination. in our study, this meant that we derived guidelines from the theoretical models developed during the steps of axial and selective coding. in order to formulate the guidelines, we have in particular focused on categories related to the meta-category “actions”. this meant that the actions have been further refined in order to provide detailed guidelines for rpc management. the formulation of the guidelines has been inspired by hollnagel (2012), who suggests a structure consisting of a description of why something is important, what needs to be done, how to apply the guideline and who contributes with what. besides these generic questions, we have added when in order to clarify the adr stage(s) the guidelines apply to. the purpose of the guidelines is to support rpc in future adr and dsr projects by providing clear examples and techniques. 5 themes – conditions, actions and consequences the purpose of this section is to presented findings identified in the analysed adr project. we have found rpc challenges oriented towards the three specific characteristics of the adr method (see section 1) and more general rpc challenges (e.g., trust, balancing researcher and practitioner interests, establishing a researcher-practitioner agreement). in this paper, we are australasian journal of information systems cronholm, gӧbel & shrestha 2024, vol 28, research article collaboration in action design research 10 predominantly interested in adr-specific rpc challenges. consequently, we have omitted rpc challenges which are general and well-described in existing literature (e.g., mathiassen, 2002; te’eni et al., 2017). our analysis of the adr project has identified three themes concerning rpc: a) shape the it artefacts based on organisational intervention, b) exploit the mutual dependency between developing design principles and it artefacts, and c) contextualise and generalise learning. moreover, our analysis of the three themes involved different aspects of reflection and learning, which supported us in suggesting prescriptive knowledge for rpc. in table 1, we demonstrate how the themes relate to the three adr characteristics presented in section 1. themes adr characteristics shape the it artefacts based on organisational intervention. the combination of dr and ar, which involves that the design of it artefacts should be based on intervention in authentic organisational settings. exploit the mutual dependency between developing design principles and it artefacts. the dual mission of making theoretical contributions and assisting in solving the problems of practitioners. contextualise and generalise learning. generalisation of qualitative studies. table 1. suggested themes related to adr characteristics. the presentation of each theme follows the structure: a) description of conditions (support) provided by the adr method for addressing a challenge experienced in the adr project. b) presentation of actions taken in the adr project to address the challenge. the actions taken were inspired by normative statements (what to do) in the adr method and creative problem-solving discussions among researchers and practitioners in the adr project. c) descriptions of consequences of the action taken. d) confirming quotes from researchers and practitioners to justify the themes. 5.1 theme 1: shape the it artefacts based on organisational intervention the adr method reflects the premise that it artefacts are ensembles shaped by the organisational context during development and use. in order to find guidance for shaping the it artefact, the adr project examined the adr method. the following statements were identified: a) “the emerging artifact, as well as the theories ingrained in it, are continuously instantiated and repeatedly tested through organizational intervention and subjected to participating members’ assumptions, expectations, and knowledge” (sein et al., 2011, p.42). b) “the adr method simultaneously supports the building of innovative it artifacts in an organizational context and learning from the intervention while addressing a problematic situation” (sein et al., p.38). unquestionably, the statements above clearly indicate that organisational intervention is vital when shaping it artefacts. the statements inform, in an excellent way, why intervention is essential and what should be done. unfortunately, the adr method does not inform how intervention can be organised in collaborative adr projects. at the beginning of the adr project, the researchers and practitioners realised that they had different motives for participating in the project. therefore, the adr project needed to reduce australasian journal of information systems cronholm, gӧbel & shrestha 2024, vol 28, research article collaboration in action design research 11 or eliminate possible tensions between researchers and practitioners, which could obstruct organisational intervention. the main objective for the practitioners was to develop a digital tool supporting them to improve data utilisation. the main objective of the researchers was to make theoretical contributions. however, both parties recognised that the fulfilment of the project goals was highly dependent on successful rpc. in order to develop a shared understanding of different motives and eliminate possible barriers to interventions, the adr project identified individual and shared goals by organising a workshop where all the members of the adr project participated. an example of a practitioner goal was to develop a digital tool supporting a structured innovation process, while an example of a researcher goal was to develop design principles supporting the development of such tools. these goals uncovered underlying motives for participation, and they guided the actions of the researchers and practitioners throughout the adr project. another challenge in the adr project related to intervention was the question of collecting qualitative data for generalisation. sein et al. (2011) state that the artefact represents a solution that could be generalised and that the “move from the specific-and-unique to generic-andabstract” is a critical component of the adr method. however, the design of an it artefact always reflects the circumstances in the context where it is to be used. therefore, the adr project realised that it is not just a matter of moving from the specific to the generic; it is also a matter of creating a qualitative basis which can provide accurate and substantial conditions for generalisation. in the adr project, this meant that the intervention in the organisations’ settings was regarded as a preparatory part of the generalisation process since the purpose was to collect contextual data that later could be used for abstraction and generalisation in the adr stages “reflection and learning” and “formalisation of learning”. consequently, the adr project carefully collected contextual factors when intervening in organisational contexts and used method triangulation to increase the probability that generalisations could be made in accordance with contextual findings. we identified that the rigorous collection of contextual data from all the organisations resulted in a large and varied data quantity which described each organisation’s specific problems and solutions requirements. furthermore, the observation that intervention was regarded as an essential part of the generalisation process created a better understanding of the adr project as a research project. it could be argued that each organisation was only interested in its own contextual data, individual goals and problems, and solutions, and not in generalisations of data. however, the adr project discovered that the organisations found it valuable to interact with each other in order to learn more about how other organisations tackled challenges related to innovation. therefore, the adr project organised workshops involving all the organisations with the purpose of sharing knowledge across organisations. finally, the members of the adr project realised that evaluating organisational intervention could be time-consuming and ad-hoc since there were no guidelines in the adr method. therefore, the adr project developed an intervention strategy that was initiated by the researchers. however, the strategy was agreed upon collectively and formulated jointly by the researchers and the practitioners in a workshop involving all nine organisations. the strategy included the following steps: 1) define the context, 2) decide on intervention approach, and 3) describe anticipated and unanticipated consequences. all decisions made about the planning of the adr project were carried out in consensus between the researchers and practitioners. next, the steps will be presented in detail. australasian journal of information systems cronholm, gӧbel & shrestha 2024, vol 28, research article collaboration in action design research 12 step 1 define the context the context was defined by describing the characteristics of the practitioners working place and included descriptions of: what is the nature of the work that is conducted, what roles (practitioners) should be involved in the intervention, what process should be analysed, what data are relevant and what digital systems were currently used. examples of contextual characteristics collected during the adr project were a specific focus on data related to the innovation process and adoption of a service-oriented perspective. the latter included value co-creation between service customers and providers. these characteristics were jointly defined by researchers and practitioners by identifying, documenting and agreeing upon boundaries. the context definition supported the adr project to focus on what was considered as essential and what was regarded as being outside the scope of the project. the adr project continuously revised the context definition due to project results and learning. step 2 decide on intervention approach the adr project decided to use naturalistic intervention sessions, which meant that the interventions took place in the organisations’ settings (venable et al., 2016). the purpose of the intervention sessions was to formatively evaluate the digital tool. moreover, the sessions were influenced by sprint demonstrations (schwaber & sutherland, 2011), which is a short timeboxed period when a team works to complete a set amount of work. in addition, the adr project also used techniques for root-cause analysis (wilson et al. 1996) in order to identify problem causes and effects. both these techniques were familiar to the practitioners. all the sessions involved both managers and operational staff in order to increase the possibility of collecting data from several perspectives. one intervention session involved the evaluation of the digital tool and follow-up interviews. the purpose was to collect feedback that could be used for a redesign of the digital tool and the development of the design principles. the adr project conducted three iterations of the intervention sessions. in total, 23 sessions were carried out (2-3 sessions per organisation). the reason for ending the intervention session after three iterations was saturation in data, i.e., the marginal benefit of one more iteration was judged to be low. the result from each intervention session was considered as highly contextual (organisationalspecific). in order to formulate meta-requirements (class of problems), a more generalised digital tool and generalised design principles, the adr project organised workshops involving all the nine companies. in these highly interactive workshops, researchers and practitioners jointly agreed upon formulations on higher abstraction levels. step 3 describe anticipated and unanticipated consequences for each intervention session, anticipated and unanticipated consequences from evaluating the tool were identified (sein et al., 2011). in order to keep track of the consequences through the iterations, the adr project created a table consisting of three columns: anticipated consequences, unanticipated consequences and comments. an anticipated consequence corresponded to positive feedback while suggestions for improvement were considered as unanticipated consequences. description of unanticipated consequences and comments were used for a redesign of that particular function of the digital tool. furthermore, unanticipated consequences were again evaluated in the following iteration. the table of the anticipated and unanticipated consequences supported the adr project to follow how the digital tool and design principles emerged over time. australasian journal of information systems cronholm, gӧbel & shrestha 2024, vol 28, research article collaboration in action design research 13 a consequence of the collaborative planning of the intervention strategy strengthened the convergence of shared understanding of different motives between the researcher and practitioners. finally, the adr project found that the intervention strategy supported efficient and effective interventions. we end the description of this theme by presenting quotes from researchers and practitioners: “the systematic intervention strategy guided us to organise the intervention in a way that encouraged learning between practitioners and researchers, and between practitioners themselves” (researcher) “the systematic intervention strategy supported transparency of the process and it also made the expectations of each other more transparent” (practitioner) “we appreciated the opportunity to evaluate the tool in our organisation since it meant that we had a real impact on the development” (practitioner) “we are happy to include ideas from researchers. in our systems development projects, we would never consider merging design requirements derived from theory with our own requirements” (practitioner) 5.2 theme 2: exploit the mutual dependency between developing design principles and it artefacts the adr method stresses the importance of developing design principles and it artefacts. the design principles should emerge from the reconceptualising of the specific solution instance into a class of solutions, while the it artefacts (the solutions) evolve as a result of iterative design and evaluation. we have identified helpful support in the adr method for developing it artefacts. however, we have not, to the same extent, found support for developing design principles. consequently, there is an imbalance concerning the support for developing design principles and it artefacts. the following support was found in the adr method for developing design principles (sein et al. 2011): a. “during bie [building, intervention, and evaluation], the problem and the artifact are continually evaluated, and the design principles are articulated for the chosen class of systems”. (p.42) b. “the design principles capture the knowledge gained about the process of building solutions for a given domain, and encompass knowledge about creating other instances that belong to this class”. (p.42) c. “articulate outcomes as design principles”. (p.45) we recognise that there is an implicit relationship between some of the principles in the adr method and theme 2. however, the added value of proposing this theme is that it helps to identify a detailed level of the relationships between the design principles, it artefact, theory and empirical data (see figure 2). we also acknowledge the informative and well-structured case description at the end of the paper presented by sein et al. (2011). the case description involves support for developing design principles and describes in an excellent way how design principles emerge over time. however, the case was conducted as an action research project and was later reconstructed as an adr case. this means that the case was not conducted as a design-oriented adr project and therefore it is unclear how the significant characteristics of the adr method affected the development of the design principles. during the bie stage, the adr project discovered a mutual dependency between the development of the design principles and the digital tool. as mentioned above, the adr method provides excellent support for developing the it artefact and some support for developing design principles. however, there is no explicit support for exploiting the mutual australasian journal of information systems cronholm, gӧbel & shrestha 2024, vol 28, research article collaboration in action design research 14 dependency between the it artefact and the design principles. this mutual dependency can be exploited in order to fulfil the project goals. the purpose of the emerging design principles was to guide the development of the digital tool. at the same time, the digital tool was a carrier of design knowledge that could be tested. this insight was based on feedback from the intervention sessions that involved the organisations’ evaluations of the digital tool. we realised that the evaluation of the digital tool not only affected the re-design of the digital tool but also provided valuable knowledge to refine the design principles. this understanding resulted in a decision that the digital tool and the design principles needed to be developed in parallel, which was something that the adr project had to consider in the project planning. the mutual dependency between the development of the design principles and the digital tool consisted of: • the development of design principles was guided by empirical feedback from evaluating the digital tool. this meant that the digital tool provided a platform for evaluating the design principles. • the development of the digital tool was guided by the emerging design principles. this meant that enhancements of the design principles were used to shape the digital tool. the adr project utilised the mutual dependency in order to balance competing interests. as mentioned in section 5.1, the researchers’ primary interest was to develop design principles while the practitioners’ primary interest was to design a digital tool. this meant that a significant action in the adr project was to articulate the close relationship between the researchers’ primary interest in developing design principles and the practitioner’s primary interest in making the mutual dependency transparent. this transparency supported an increased understanding of how the researcher and practitioner interests could benefit from each other. moreover, the feedback from evaluating the digital tool also resulted in new insights prompting the researchers to consult complementing theories such as resource-based theory (e.g., penrose, 1959) and service-dominant logic (e.g., vargo & lusch, 2004. this meant that the adr project utilised the empirical feedback as a guide for identifying complementing theories, which provided further knowledge for developing the design principles and the digital tool. consequently, the formulation of the design principles and the development of the digital tool were based on theoretical insights and empirical evidence. our analysis of the mutual dependency of the design principles and the digital tool (it artefact) is illustrated in figure 2. one example of this mutual dependency is related to resource liquefying. in the adr project, resource liquefying was defined as the mobilisation and decoupling of resources from their related physical form or device (e.g., lusch and nambisan 2015). that is, more decoupled resources could increase the capability for service innovation. based on this service-oriented theoretical statement, researchers formulated an initial design principle which inspired the adr project to develop functionality that supports the liquefaction of resources. however, during the evaluation of the tool, the adr project identified that there was a need to extend the functionality to include the possibility for users to communicate and share knowledge related to specific characteristics in the context, i.e., a communication protocol. consequently, the adr project redesigned the digital tool and modified the initial formulation of the design principles. australasian journal of information systems cronholm, gӧbel & shrestha 2024, vol 28, research article collaboration in action design research 15 due to the increased understanding of the mutual dependency of the design principles and the digital tool, the collaboration between the researchers and practitioners was further motivated. this insight increased the acceptance of solving both the scientific problem and the problem that existed in practice, creating a better willingness to share knowledge. moreover, the constant use of theories and empirical feedback created a solid grounding of the design principles and the artefact. figure 2. mutual dependencies between the design principles and the it artefact. another challenge in the adr project was how to formulate design principles. the design principles were of interest to both researchers (as a form of theoretical knowledge) and practitioners (as practical guidance when designing it artefacts). this means that the design principles had to be jointly verified by the researchers and practitioners. the examples of design principles provided by the adr method consist of a title and a short description of attributes of the artefact (i.e., what components the artefact should include). the adr project experienced a need for expressing design principles in a more advanced way that also involved action-oriented support (i.e., what actions the artefact should allow for) and justificatory support (arguments). consequently, the adr project conducted a literature review in order to identify standards for formulating design principles. the adr project identified several valuable suggestions such as: march and smith (1995), chandra et al. (2015), cronholm and göbel (2018), goldkuhl (2004), van aken (2004) and van den akker (1999). the adr project decided to follow the recommendation suggested by van den akker (1999) because it supported a structured description consisting of where to do (context), what to do, how to do and why to do. it also provided a logical structure and supported that design principles be formulated on the same abstraction level. these advantages were considered supportive because some members of the adr project were novices regarding the development of design principles. the suggestion reads: if you want to design intervention x [for the purpose/function y in context z], then you are best advised to give that intervention the characteristics a, b, and c [substantive emphasis], and to do that via procedures k, l, and m [procedural emphasis], because of arguments p, design principles it artefact (digital tool) theory evaluation sessions governs the design of provides an evaluation platform for provides insights insights for consulting complementing theory provides empirical evidence provides insights australasian journal of information systems cronholm, gӧbel & shrestha 2024, vol 28, research article collaboration in action design research 16 q, and r. one example of a design principle formulated in the adr project reads: “ if you want to design a digital resourcing system for the purpose of enabling actors in a service-oriented context to co-create novel value propositions in the discovery stage of the digital innovation process, then you are best advised to design the digital resourcing system to support actors in service-ecosystems so they can mobilise and decouple resources, and to do that via contextualised communication protocols, defined boundary objects and to store those resources as digital resources because it creates a sound data source fostering digital innovation.” finally, we present some confirming quotes from researchers and practitioners regarding our findings: “the integration of theory into the it artefact helped us to justify design solutions and in a cumulative way build further on existing knowledge” (researcher) “in the beginning of the project we were interested in tools for data utilisation, but now we are equally interested in the design principles” (practitioner) “the feedback from practitioners guided us when consulting theories that we initially did not think were relevant. it meant that we expanded our literature search in order to ingrain additional theories into the it artefact” (researcher) 5.3 theme 3: contextualise and generalise learning sein et al. (2011) explicitly state that adr projects should move from the specific-and-unique to generic-and-abstract. they define three levels for this conceptual move: generalisation of the problem instance, generalisation of the solution instance, and derivation of design principles from the project results. the following support for generalisation was found in the adr method: a. “the reflection and learning stage move[s] conceptually from building a solution for a particular instance to applying that learning to a broader class of problems”. sein et al. (2011, p.44) b. “the situated learning from an adr project should be further developed into general solution concepts for a class of field problems” sein et al. (2011, p.44) c. “the action design researcher should generate knowledge that can be applied to the class of problems that the specific problem exemplifies” (sein et al. 2011, p. 40) d. “reconceptualizing the learning from the specific solution instance into design principles for a class of solutions” sein et al. (2011, p.45). all these quotes convincingly explain why it is important to generalise and what should be done. however, the adr project identified insufficient support for what actions to take in order to meet the generalisation requirement. this meant there was a need for guidelines due to the highly contextual nature of the organisational interventions. moreover, the adr project realised that it was important that the findings contribute to each organisation’s specific requirements (contextualisation) but also be valid to other organisations outside the project (generalisation). as mentioned in section 4, the adr project was carried out as a qualitative study. the problem of generalisation from qualitative studies is well-known (e.g., baskerville et al., 2015). in order to meet this challenge, the adr project utilised the fact that nine organisations participated in the project. the adr project implemented two mechanisms to support generalisation: a) arrangements of dyadic researcher-practitioner meetings with all the participating organisations. one dyadic meeting meant that 1-2 researchers interacted with one organisation (2-3 practitioners) at a time to jointly reflect on how the goals were fulfilled and to decide the next step in the process, australasian journal of information systems cronholm, gӧbel & shrestha 2024, vol 28, research article collaboration in action design research 17 b) multi-organisational workshops that involved all the nine organisations and the researchers. in addition, we identified an interplay between the dyadic meetings and the multi-organisational workshops. for example, the contextual problems and requirements for developing the it artefact were used as input for generalisation. at the same time, vague and unclear generalisations regarded as incomplete governed further collection of contextual data. this means that the generalisation process was conducted as a mix of a bottom-up and topdown approach. it also meant an iterative process of describing contextualised and generalised learning. more specifically, the adr project was inspired by object-oriented techniques (e.g., date 2006) such as uml diagrams (e.g., blaha & rumbaugh, 1991) supporting the formulation of a problem class and problem instances. these techniques supported the practitioners’ acceptance to move from the contextual instances of the problems to the general understanding of the problem class. the object-oriented technique also supported the generalisation of design principles. another identified consequence was that the adr project experienced the generalisation process as positive because it legitimises research as a profession. it also advises against a type of it consultant behaviour that consists of merely solving an instance of a problem. besides generalisation, the adr method should support researcher and practitioner learning. sein et al. (2011) state that principle 4, mutually influential roles, particularly addresses the importance of mutual learning among the different project participants. however, the adr project discovered that learning in the adr method is mainly discussed in relation to researchers. the organisation of the dyadic meetings and multi-organisational workshops also contributed to practitioner learning. furthermore, learning was also supported by a mix of researcher and practitioner presentations at a seminar and joint publications of scientific and technical papers. the process of co-authoring papers formalised insights and learning for both researchers and practitioners. lastly, we present a few quotes from researchers and practitioners regarding our findings. “to identify a class of problems increases the possibility that the problem is true, new and interesting” (researcher) “one reason to participate in the project is to interact and learn from other organisations” (practitioner) “applying object-oriented techniques when casting problem instances to a class was a critical success factor since the technique was well understood by both practitioners and researchers” (researcher) 6 theoretical models the purpose of section 6 is to generalise the three project-specific themes, presented in section 5, into corresponding theoretical models that are aimed to be valid to other adr and dsr projects. thereafter, we have created an integrated abstract wholeness illustrating how the three models are related to each other. glaser and strauss (2017) state that a substantive theory emerges from conceptual categories grounded in data. in our case, the substantive theory corresponds to theoretical models, which are induced from the analysis of the adr project (see section 5). the development of the models involved the application of inductive reasoning, which means that the development of theory is based on the collected data, the observations made, and logical thinking (strauss & corbin, 1990). furthermore, the generation of the models utilised theoretical sampling, australasian journal of information systems cronholm, gӧbel & shrestha 2024, vol 28, research article collaboration in action design research 18 which is a process of developing theory as it emerges (ibid.). the models also built further on previous knowledge (see discussion in section 8.2). as mentioned in section 4, the structure of the theoretical models follows the action-oriented paradigm consisting of three meta-categories: conditions, actions, and consequences. this means that they explicitly explain the relationships between the conditions for a challenge, the action taken to address the challenge, and the consequences of the actions taken (see figures 3-5). the models should be interpreted in the following way: a) the conditions provide information about rpc challenges (i.e., the adr statements informs about what to do but not how to do something), b) the collaborative actions taken provide information about how arrangements can be made to address the challenges and c) the consequences provide information about what will probably happen when the collaborative actions have been carried out. detailed information about the actions, including supporting techniques, will be given when the guidelines are presented (see section 7). figure 3. model of theme 1: shape the it artefact based on organisational intervention. conditions “the emerging artifact, as well as the theories ingrained in it, are continuously instantiated and repeatedly tested through organizational intervention and subjected to participating members’ assumptions, expectations, and knowledge” (sein et al., 2011, p.42). “the adr method simultaneously supports the building of innovative it artifacts in an organizational context and learning from the intervention while addressing a problematic situation” (sein et al., p.38). actions collaboratively develop an intervention strategy: define the context. prioritise contextual characteristics. reciprocally shape the it artefact and the organisational context. reflect on the implemented contextual characteristics. consequences deepened understanding of the problem. organisational problems are viewed as knowledgecreation opportunities. agreed, systematic and transparent intervention. reduced tension between researchers and practitioners. the functionality of the artefact is based on organisational relevance. a solid basis for generalisation. conditions “during bie [building, intervention, and evaluation], the problem and the artifact are continually evaluated, and the design principles are articulated for the chosen class of systems”. (sein et al., 2011, p.42) actions collaboratively exploit the mutual dependency: create a protocol that describes how the mutual relationship between the design principles and it artefact is utilised. describe anticipated and unanticipated consequences systematically. choose appropriate guidelines when formulating design principles. consequences strengthened validation of design principles and it artefact. articulated design principles for a chosen class of systems. increased practitioner acceptance of developing a solution as an instance of the problem and a class of problems. improved understanding of design principles. australasian journal of information systems cronholm, gӧbel & shrestha 2024, vol 28, research article collaboration in action design research 19 figure 4. model of theme 2: exploit the mutual dependency between design principles and it artefacts. in the next step, we identified an overarching theoretical model that integrated the three themes. the purpose of the overarching model is to illustrate a wholeness showing how the three themes presented in figure 3-5 are related to one another. therefore, the overarching model constitutes an abstraction of these themes and is entitled “researcher-practitioner collaboration: overarching adr challenges and solutions”, which corresponds to the central topic of our study (see figure 6). the overarching theoretical model was identified by figure 5. model of theme 3: contextualise and generalise learning. analysing the relationships between the three themes. the first one, “shape the it artefact based on organisational intervention”, was regarded as a condition for the other two themes since it establishes a critical point of departure for rpc. the second theme, “exploit the mutual dependency between developing design principles and it artefacts.”, is regarded as a central action since it constitutes a primary concern for rpc and is affected by the theme “shape the it artefact based on organisational intervention”. finally, the third theme, “contextualise and generalise learning”, is considered a consequence since the likelihood of learning will increase if the first and second themes are adequately addressed. conversely, “contextualise and generalise learning” will probably be obstructed if adr projects fail to manage the first and second main themes. 7 guidelines the purpose of this section is to complement the theoretical models and themes presented above with three guidelines for rpc. the main difference between the theoretical models and the guidelines is that the theoretical models illustrate a wholeness consisting of conditionsactions-consequences while the purpose of the guidelines is to deepen the action-part of the theoretical models in order to offer concrete recommendations. that is, the guidelines aim to conditions “the situated learning from an adr project should be further developed into general solution concepts for a class of field problems” sein et al. (2011, p.44). “reconceptualizing the learning from the specific solution instance into design principles for a class of solutions” sein et al. (2011, p.45). actions collaboratively contextualise and generalise learning: up-cast and down-cast between instances and classes. specify learning outcomes on two abstraction levels: contextualised and generalised. utilise the interplay between dyadic meetings and multi-organisational workshops as a means to move from the specific-and-unique to genericand-abstract. identify an appropriate generalisation level. consequences problem and solution instances are cast as classes. transparent move from contextualised and generalised learning. combined researcher and practitioner learning. learning between companies. australasian journal of information systems cronholm, gӧbel & shrestha 2024, vol 28, research article collaboration in action design research 20 make the theoretical models more actionable through providing more details, examples and techniques. the need for developing guidelines regarding rpc is supported by several figure 6. the overarching theme on rpc challenges and solutions. scholars (e.g., collatto et al., 2017; haj-bolouri et al., 2018; cronholm & göbel, 2018; henriques & o’neill, 2021). for example, haj-bolouri et al. (2018) state that the adr method is a highlevel framework and that there is a need for more concrete ways of working with the methodology. as mentioned in section 4, we have structured the guidelines according to the five generic components: why something is important to do, what needs to be done, how to apply the guideline, when something should be done (what adr stage) and who can contribute. 7.1 guideline 1: strategize organisational intervention to identify contextual characteristics why: context awareness is crucial when using the adr method because of the reciprocal relationship between the organisational context and an it artefact. this means that the context affects the design of the it artefact, and the it-artefact affects the context. sein et al., 2011, p.43) state that “researchers bring their knowledge of theory and technological advances, while the practitioners bring practical hypotheses and knowledge of organizational work practices”. therefore, the primary purpose of guideline 1 is to help researchers and practitioners to strategize organisational intervention collaboratively by systematically and simultaneously building and evaluating innovative it artifacts and learning from the intervention while addressing a problem in a contextual situation. what: collaboratively develop an intervention strategy. how: action 1define the context. we define context as “any information that can be used to characterise the situation of an entity. an entity is a person, place, or object that is considered relevant to the interaction between a user and an application, including the user and applications themselves” (dey, 2001, p. 5). based on the definition, we recommend that researchers and practitioners broadly identify contextual characteristics (quantitative and qualitative) that could potentially impact on the design. action 2 prioritise contextual characteristics. in the second activity, we recommend that researchers and practitioners jointly prioritise the identified characteristics by assessing their perceived importance in influencing the it artefact. the output from condition: it artefacts shaped by organisational intervention. action: exploit the mutual dependency between design principles and it artefacts. consequence: contextualised and generalised learning. researcher-practitioner collaboration: overarching adr challenges and solutions australasian journal of information systems cronholm, gӧbel & shrestha 2024, vol 28, research article collaboration in action design research 21 activity 2 consists of a structured list of characteristics systematically described at different priority levels. action 3 reciprocally shape the it artefact and the organisational context. organisational intervention is always naturalistic (venable et al., 2016), which can be carried out as a formative or summative evaluation (scriven, 1996). in order to support reciprocal shaping, we recommend the adr project to carry out evaluation episodes consisting of four steps: a) the researchers present the current status of the it artefact to the practitioners, b) the practitioners test the it artefact when working on an actual task while researchers are aware of contextual characteristics and take notes, c) the practitioners provide feedback and d) the researchers conduct individual interviews with the participating practitioners based on evaluation properties and contextual characteristics. action 4 reflect on the implemented contextual characteristics: since the process for organisational intervention is iterative, we recommend that researchers and practitioners, based on learnings from step 3, jointly review the implemented contextual characteristics and the potential need for changes of the context definition. examples of questions that could be asked are: how does the it artefact align with the context? what effects did the artefact have on the context? did the context affect the use of the it-artefact? when: primary adr stages are problem formulation, bie, reflection and learning. who: researchers and practitioners contribute with their unique competences. 7.2 guideline 2: capitalise on the mutual dependency between the it-artefact and design principles to support guided emergence why: the primary purpose of capitalising on the mutual dependency between the emergent it-artefact and the design principles is to increase the transparency and optimise validation of both the it-artefact and design principles. what: utilise the interplay between the emergent it artefact and design principles. the knowledge gained from developing the design principles can be utilised to build the it artefact and vice versa. how: action 1 create a protocol that describes how the mutual relationship between the design principles and it artefact is utilised. we recommend adr projects to view the it artefact as a platform for evaluation of the design principles and letting the emergent design principles govern the ongoing development of the artefact. this means to a) thoroughly describing how well-received functionality regarding the it artefact affects the formulation of the design principles and b) informing on how emergent design principles affect the design of the it artefact. this activity means creating explicit relationships between the it artefact and a design principle. in order to support transparency of the mutual dependency, we recommend adr projects to document the relationships in a table consisting of three columns: functionality of the it artefact, design principles and comments. in order to support the transparency of the progress of the emergent it artefact and design principles, we strongly advocate that the table includes both graphical and textual descriptions. australasian journal of information systems cronholm, gӧbel & shrestha 2024, vol 28, research article collaboration in action design research 22 action 2 describe anticipated and unanticipated consequences systematically. we recommend that researchers and practitioners systematically analyse anticipated and unanticipated consequences during each bie iteration. anticipated consequences correspond to functionality in the it artefact that is well received by the practitioners. unanticipated consequences correspond to functionality that is in need of further development in the following bie iteration. in order to keep track of anticipated and unanticipated consequences, we recommend adr projects use a table consisting of three columns: anticipated consequences, unanticipated consequences and actions to be taken. the table is used when researchers and practitioners are planning the next bie iteration. this iterative process ends when the researchers and practitioners agree that no further iterations are necessary. this systematic way of documenting anticipated and unanticipated consequences for each bie iteration will support transparency, communication and learning between researchers and practitioners. action 3 choose appropriate standards when formulating design principles. design principles target both researchers and practitioners which means that they should be collaboratively formulated. in order to ensure that design principles are homogenously formulated, we recommend that guidelines are selected. there exist several standards for formulating design principles such as: march and smith (1995), van aken (2004), goldkuhl (2004), van den akker (1999), chandra et al. (2015) and cronholm and göbel (2018). all these suggestions build, in one way or another, on the formula “perform action a in order to obtain goal b”. we recommend that adr projects scrutinise different alternatives and make choice according to the complexity of the circumstances. when: primary adr stages are bie, reflection and learning, formalisation of learning. who: researchers contribute with design knowledge, standards for formulating design principles and technical skills. practitioners contribute with suggestions for improvements (feedback). 7.3 guideline 3: exploit up-casting and down-casting to maximise contextualised and generalised learning why: the primary purpose of recommending up-casting and down-casting is to support contextualised and generalised learning and to identify appropriate generalisation levels by traversing classes and instances both bottom-up (up-casting) and top-down (down-casting). what: exploit contextual learning as a means of generalised learning. contextual learning involves learning about instances such as specific characteristics (e.g., goals, needs and problems). generalised learning means to up-cast the specified learning into learning about a class. how: action 1 up-cast and down-cast between instances and classes. while guideline 1 focuses on the context, guideline 3 focuses on the process of moving from contextual findings to generalised learning. this means exploiting previous contextual learning to acquire generalised learning. in order to cast the problems and solutions as instances of a classes, we recommend adr projects to utilise object-oriented techniques. (e.g., date 2006). using object-oriented techniques involves creating class diagrams and, for example, identifying properties, operations and processes of an instance (context) that also could be encapsulated within a class. the class acts as a blueprint and defines the australasian journal of information systems cronholm, gӧbel & shrestha 2024, vol 28, research article collaboration in action design research 23 structure and behaviour of instances. we recommend an up-casting (bottom-up) approach when creating classes of problems and solutions and their relationships to instances. however, the creation of classes also requires a down-casting (top-down) approach. the design knowledge created from a previous bie iteration can be seen as a tentative class which can be down-casted and tested in the following bie iteration. this means that new findings in the following iteration is tested against the tentative class to verify that it belongs to the class. the iterations consist of a continuous interplay between upand down-casting. in addition, we recommend that theoretical sampling is conducted when a class is vague or unclear (glaser & strauss, 2017). action 2 specify learning outcomes on two abstraction levels: contextualised and generalised. we recommend that learning is specified on both contextualised and generalised levels. the idea is that the contextualised learning works as a stepping stone for achieving generalised learning. this means that the specifics of a context need to be properly described before abstractions can be made. moreover, a thorough description of contextual learning should be of extreme interest to the practitioners. action 3 utilise the interplay between dyad meetings and multi-organisational workshops as a means to move from the specific-and-unique to generic-and-abstract. if the adr project involves several organisations (or has the possibility to confirm results with other organisations outside the project), we recommend that the contextual data gathered from dyadic meetings are used as input to multi-organisational workshops. as mentioned in section 5.3, a dyadic meeting involves researchers and practitioners from one organisation. a multi-organisational workshop involves all the participating organisations and can be viewed as a tool for generalising problem and solution instances into problem and solution classes. this means that different problem and solution instances are jointly discussed in order to arrive at class descriptions that are based on consensus. a multi-organisational workshop is an arena for supporting generalised learning by creating abstract representations of design knowledge. action 4 identify an appropriate generalisation level. finding an appropriate generalisation level means being able to balance universality and utility (e.g., lincoln & guba, 1985). we recommend adr projects to apply the framework developed by winter (2013). the framework consists of four abstraction levels: situated knowledge (one of a kind), configurable knowledge (adaptations are needed), archetypical knowledge (generalisation is applicable for all problem situations that share specific properties and generic knowledge (one size fits all), (see section 8.3 for more information). when: problem formulation, bie, reflection and learning, formalisation of learning. who: researchers contribute with techniques regarding generalisation. practitioners contribute with confirmation and suggestions for improvements regarding the formulation of instances and classes. 8 discussion our discussion involves three parts: a) an argumentation for why we view the three themes as predominantly adr specific, b) a comparison of what is new in our paper and previous knowledge, and c) a possible generalisation of our findings. australasian journal of information systems cronholm, gӧbel & shrestha 2024, vol 28, research article collaboration in action design research 24 8.1 why are the findings predominantly adr-specific? we argue that the theoretical models and guidelines are predominantly specific to adr projects because their assembling is based on the three significant characteristics of the adr method discussed in section 1: • the design of it artefacts should be based on intervention in authentic organisational settings. • respond to the dual mission of making theoretical contributions and assisting in solving the problems of practitioners. • encouragement for generalisation of contextual design studies. we state that these characteristics are not prominent to the same extent in other dsr approaches/methods, or more general is research methods. for example, the framework proposed in the seminal paper by hevner et al. (2004) focuses on it artefact development and does not consider organisational intervention and the role of the organisational context to an extent similar to that of the adr method. furthermore, peffers et al. (2007) have suggested a promising dsr process model for developing it-artefacts. however, the process model does not a) emphasise that it artefacts emerge through intervention in organisational settings and does not b) not include support for generalisation or theory development. we can conclude that neither hevner et al. (2004) nor peffers et al. (2007) provide explicit prescriptive knowledge for rpc. there are also a few dsr methods that are oriented towards participatory dsr. for example, soft design science methodology (sdsm) (pries-heje et al., 2014) focuses on socio-technical aspects of is and their integration into organisations and society. the stages in sdsm effectively guide the user of the methodology between the real world and the abstract world. however, the method does not consider the development of innovative it artefacts to the same extent as the adr method. another dsr method is participatory action design research (padr) (bilandzic & venable, 2011). similar to adr, padr combines action research and design science research. however, padr is limited to supporting urban informatics. moreover, padr does not promote generalisation of contextual design studies. the last identified method that supports participatory dsr is also called participatory action design research (padre) (haj-bolouri et al., 2016). one purpose of padre is to suggest how reflection and learning can be integrated into each stage of the adr method. this means that padre can be seen as an extension of the adr method and thus involves the same three significant characteristics as the adr method. in summary, none of these methods appear to offer explicit prescriptive knowledge for rpc. if we look at more general research methods such as action research (ar) (e.g., susman & evered, 1978; avison et al., 2001; cronholm & goldkuhl, 2004), it is evident that ar supports intervention and knowledge emergence in authentic settings but pays little attention to abstraction and generalisation. moreover, ar does not provide explicit support for it artefact development. against this backdrop, we argue that the theoretical models and guidelines developed in our study are highly relevant to consider when managing rpc in adr projects. 8.2 what is new in our study? the purpose of this section is to compare theoretical models and guidelines (what is new) with the literature review (what is known) and statements in the adr method in order to highlight our knowledge contribution. australasian journal of information systems cronholm, gӧbel & shrestha 2024, vol 28, research article collaboration in action design research 25 8.2.1 comparison with the literature review our literature review revealed that only a few studies present some form of advice or guidelines. the most interesting study was conducted by haj-bolouri et al. (2018) who identified three essential topics that researchers experienced when working with the adr method: a) how to balance the competing interests of the organisational stakeholders with the interests of a research community, b) how to balance the practitioner need of a situated implementation of the designed itartefact against the research need to produce generalised knowledge and c) how to balance the findings between specific and generalisable research outcomes. the purpose of their paper was to conduct a retrospective analysis of how adr has been used in practice, not to provide guidelines for managing challenges regarding rpc. our guidelines also suggest specific tools to be used when when moving from problem instances to a class of problems and supporting contextualised and generalised learning. moreover, henriques and o’neill (2021) advocate that focus groups should be used in adr projects in order to provide traceability between problems, requirements, solutions and artefacts. in addition, henriques and o’neill (2021) present statements such as “rigorous and committed stakeholder engagement is a critical success factor in complex projects” and “the collaboration of stakeholders is paramount to find the best solutions for socio-technical problems”. these statements are clearly oriented towards what to do, while our study is oriented towards how to do something. our literature review also uncovered a few dilemmas and models regarding rpc. for example, rapoport (1970) discusses goal dilemmas between researchers and practitioners, which have been an important aspect when formulating our three themes. furthermore, susman and evered (1978) argue that specifying learning is vital, which has contributed to the creation of theme 3 (contextualise and generalise learning). in addition, the research project conducted by mathiassen (2002), which involved several researchers and organisations, inspired us to organise dyad meetings and multi-organisational workshops (see theme 3). when we compare these dilemmas and models with our study, we can conclude that our models extend previous knowledge with a structure consisting of the meta-categories: conditions, actions and consequences (see figures 3-7). this means we have suggested a logic consisting of relationships between the categories identified in our study. it is important to note that we do not state that the conditions and the consequences are new. on the contrary, these are reasonably well-described in the existing literature. however, we argue that the actions per se are novel and also the actions viewed in a context consisting of conditions, actions and consequences. based on our comparison between our study and previous knowledge, we state that our findings extend previous knowledge with guidelines and theoretical models that aim to support users of the adr method when managing rpc. 8.2.2 comparison with the adr method in order to shed light on what is new in our study regarding the adr method, we have compared statements and principles of the adr method with the three guidelines suggested in our study. firstly, sein et al. (2011) state that “the emerging artifact, as well as the theories ingrained in it, are continuously instantiated and repeatedly tested through organizational intervention and subjected to participating members’ assumptions, expectations, and knowledge” (sein et al., 2011, p.42). this informative statement describes what to do regarding organisational intervention and has inspired us to develop the guideline “strategize organisational intervention for guided emergence”. the purpose of the guideline is to support australasian journal of information systems cronholm, gӧbel & shrestha 2024, vol 28, research article collaboration in action design research 26 organisational intervention through an intervention strategy collaboratively developed by researchers and practitioners. we state that the guideline complements the statements in the adr method by informing about how organisational intervention could be conducted. our guideline should primarily be viewed as a complement to the adr principle 1: 3: reciprocal shaping. secondly, sein et al. (2011) state that the outcomes of an adr project should consist of an it artefact and design principles. for example, they emphasise that “during bie [building, intervention and evaluation stage], the problem and the artifact are continually evaluated, and the design principles are articulated for the chosen class of systems” (p.42). this statement clearly describes what should be done and has inspired us to develop the guideline “capitalise on the mutual dependency between the it-artefact and design principles”. the purpose of the guideline is to support efficient validation of it-artefacts and design principles by utilising their interplay. in addition, several scholars have reported that too much freedom regarding the formulation of design principles has resulted in that reusability cannot be taken for granted (e.g., kruse & seidel, 2016; chandra kruse & seidel, 2017; cronholm & göbel, 2018). therefore, our guideline also involves the recommendation to formulate homogenous design principles. our recommendation specifically targets novice dsr/adr scholars. the second guideline should primarily be viewed as a complement to the adr principle 6: guided emergence. thirdly, sein et al. (2011, p.44) state: “the situated learning from an adr project should be further developed into general solution concepts for a class of field problems”. this is a conceptual move that involves contextualised and generalised learning. this statement has constituted a base for developing the guideline “organise dyadic and multi-organisational interaction for contextualised and generalised learning”. the purpose of the guideline is to contribute to maximised contextualised and generalised learning. we argue that this guideline extends the statements in the adr method regarding learning by illustrating the interplay between dyad meetings and multi-organisational workshops when casting the problem instance into a class of problems, reconceptualising the specific solution instance into a class of solutions, and specifying learning into design principles for a class of solutions. the third guideline should primarily be viewed as a complement to the adr principle 7: generalised outcomes. finally, all the guidelines build on existing statements and principles in the adr method. we state that the guidelines complement existing adr statements primarily oriented towards describing what to do and why to do something. our guidelines also involve suggestions for explicit techniques that could be applied to support the management of rpc. 8.3 generalisation in section 7.1, we have argued that the identified themes are predominantly adr-specific. by using the term predominantly, we mean that they are likely to appear in adr projects but can also appear in other rpc projects involving similar goals, purposes and processes. that is, the themes are primarily but not necessarily exclusively associated with adr projects. as mentioned above, generalisation is a common requirement for both quantitative and qualitative research studies. our study is not an exception. in order to discuss the degree of generalisation of our theoretical models and guidelines, we have applied a framework developed by winter (2013). the framework consists of four australasian journal of information systems cronholm, gӧbel & shrestha 2024, vol 28, research article collaboration in action design research 27 abstraction levels: “level 0 situated knowledge”, “level 1 configurable knowledge”, “level 2 archetypical knowledge” and “level 3 generic knowledge”. “level 0 situated knowledge” is defined as one of a kind, i.e., the solution is applicable only in a specific organisation for a specific process at a specific point in time. in our case, this refers to findings in the described adr project (see section 5). “level 1 configurable knowledge” means that, based on either a refinement of archetypes (such as it artefacts) or on a classification of real-world problems, many problem configurations are differentiated whose solutions are created from reusable modules by configuration or aggregation. if we compare this description to our study, we state that our theoretical models and guidelines are applicable to other adr projects. in order to support the reuse of our findings in future research adr projects, we have provided transparent descriptions and explanations. however, contextual adaptions might be needed. “level 2 archetypical knowledge” is defined as models applicable for all problem situations that share certain properties (e.g., process type, organisation type, project type/goals, available resources and/or skills). for example, level 2 could refer to other dsr projects using other dsr methods. in particular, we argue that our findings should be considered by other participatory dsr projects using other participatory dsr methods such as soft dsr methodology (pries-heje et al., 2014), participatory action design research (padr, bilandzic and venable 2011), and participatory action design research (padre, haj-bolouri et al, 2016). “level 3 generic knowledge” is defined as one size fits all, i.e., the models are applicable to all organisations in all existing or possible worlds. in the context of generalising the results of our study, level 3 could denote is projects in general. as stated earlier, the identified themes are predominantly adr-specific but not necessarily exclusively related to adr projects. however, regarding levels 3 and 4, we argue that our theoretical models and guidelines should be considered on these higher abstraction levels if they face similar rpc challenges. to conclude, we consider the theory projectable to other contexts with similar characteristics to the studied adr project. the concept of projectability is introduced by baskerville and pries-heje (2019) as an alternative to the more backwards-looking generalisability concept. 9 conclusion in section 1, we stated that there was a lack of prescriptive support for rpc in the adr method. however, we also recognised that the support in responding to the questions of why to collaborate and what to collaborate on is excellent. thus, our study aimed to present prescriptive knowledge in terms of theoretical models and guidelines that can support rpc collaboration in adr projects. based on three central adr characteristics, we have developed theoretical models consisting of three themes: shape the it artefact based on organisational intervention, exploit the mutual dependency between developing design principles and it artefacts, and contextualise and generalise learning. the structure of the models follows an action paradigm model consisting of the meta-categories conditions, actions and consequences, which provide explanatory relationships. we consider the models projectable to other contexts with similar characteristics to the studied adr project. moreover, we have developed three guidelines to support rpc. the three guidelines read: strategize organisational intervention to identify contextual characteristics, capitalise on the mutual dependency between the it-artefact and design principles to support guided emergence, and exploit up-casting and down-casting to australasian journal of information systems cronholm, gӧbel & shrestha 2024, vol 28, research article collaboration in action design research 28 maximise contextualised and generalised learning. we can conclude that the actions taken in the analysed adr project supported a) the adr project to successfully manage the challenges in order to fulfil the project goals and b) researchers and practitioners to collaborate successfully. we regard our knowledge contribution as a complement to prior knowledge of rpc. we view the theoretical models and guidelines as prescriptive knowledge contributions to both practice and theory. the practice in our case consists of research projects involving researchers and practitioners. thus, our knowledge contribution addresses adr project management in particular and dsr and is project management in general. the theoretical models and guidelines also constitute a theoretical contribution because they add new prescriptive knowledge about rpc. our conclusions are based on the analysis of an adr project and previous knowledge regarding rpc. this means that we have built further on previous knowledge on the management of rpc in adr projects. to further strengthen the theoretical models and guidelines created in this study, we suggest that they will be evaluated in future projects facing rpc challenges. references argyris, c., & schön, d. a. (1989). participatory action research and action science compared: a commentary. american behavioral scientist, 32(5), 612-623. http://dx.doi.org/10.4135/9781412985383.n6 avison, d., baskerville, r., & myers, m. (2001). controlling action research projects. information technology & people, 14(1), 28-45. http://dx.doi.org/10.1108/09593840110384762 baskerville, r., & pries-heje, j. (2019). projectability in design science research. journal of information technology theory and application (jitta), 20(1), 3. bilandzic, m., & venable, j. r. (2011). towards participatory action design research: adapting action research and design science research methods for urban informatics. journal of community informatics, 7(3). http://dx.doi.org/10.15353/joci.v7i3.2592 blaha m. & rumbaugh j. (1991). object-oriented modeling and design with uml. upper saddle river, nj, usa: prentice-hall. cambridge dictionary (n.d.). guideline. retrieved may 14, 2024. https://dictionary.cambridge.org/dictionary/english/guideline. chandra kruse, l., & seidel, s. (2017). tensions in design principle formulation and reuse. in proceedings of design science research in information systems and technology. chandra, l., seidel, s., & gregor, s. (2015). prescriptive knowledge in is research: conceptualizing design principles in terms of materiality, action, and boundary conditions. in proceedings of the hawaii international conference on system sciences. http://dx.doi.org/10.1109/hicss.2015.485 charmaz, k. (2017). the power of constructivist grounded theory for critical inquiry. qualitative inquiry, 23(1), 34-45. http://dx.doi.org/10.1177/1077800416657105 chun tie, y., birks, m., & francis, k. (2019). grounded theory research: a design framework for novice researchers. sage open medicine, 7, 1-18. http://dx.doi.org/10.1177/2050312118822927 http://dx.doi.org/10.4135/9781412985383.n6 http://dx.doi.org/10.1108/09593840110384762 http://dx.doi.org/10.15353/joci.v7i3.2592 http://dx.doi.org/10.1177/1077800416657105 http://dx.doi.org/10.1177/2050312118822927 australasian journal of information systems cronholm, gӧbel & shrestha 2024, vol 28, research article collaboration in action design research 29 collatto, d. c., dresch, a., lacerda, d. p., & bentz, i. g. (2018). is action design research indeed necessary? analysis and synergies between action research and design science research. systemic practice and action research, 31(3), 239-267. conboy, k. (2009). agility from first principles: reconstructing the concept of agility in information systems development. information systems research, 20(3), 329354. cronholm s, & göbel h. (2018). guidelines supporting the formulation of design principles. in proceedings of the 29th australasian conference on information systems. sydney, australia, dec 3-5. http://dx.doi.org/10.5130/acis2018.ak cronholm, s. & göbel, h. (2019). evaluation of action design research. scandinavian journal of information systems, 31(2), 35-82. cronholm, s., & göbel, h. (2022). action design research–models for researcher-practitioner collaboration. in proceedings of the international conference on design science research in information systems and technology, pp. 393-404. http://dx.doi.org/10.1007/978-3-03106516-3_29 cronholm, s., & goldkuhl, g. (2004). conceptualising participatory action research–three different practices. electronic journal of business research methods, 2(2), 47-58. date, c. j. (2006). an introduction to database systems. pearson education. dey, a. k. (2001). understanding and using context. personal and ubiquitous computing, 5, 4-7. http://dx.doi.org/10.1007/s007790170019 fitzgerald, b., russo, n., & stolterman, e. (2002). information systems development: methods in action. london: mcgraw-hill. glaser, b. g., & strauss, a. l. (2017). discovery of grounded theory: strategies for qualitative research. routledge. http://dx.doi.org/10.4324/9780203793206 goldkuhl, g. (2004). design theories in information systems – a need for multi-grounding, journal of information technology theory and application, 6(2), 59-72. goldkuhl g & cronholm s. (2019). grounded theory in information systems research – from themes in is discourse to possible developments. in proceedings of the international conference on information systems, dec 15-18, munich germany. gregor, s. (2009). building theory in the sciences of the artificial. in proceedings of the 4th international conference on design science research in information systems and technology, pp. 1-10. http://dx.doi.org/10.1145/1555619.1555625 gregor, s., & hevner, a. r. (2013). positioning and presenting design science research for maximum impact. mis quarterly, 37(2), 337-355. http://dx.doi.org/10.25300/misq/2013/37.2.01 haj-bolouri, a., bernhardsson, l., & rossi, m. (2016). padre: a method for participatory action design research. in proceedings of the international conference on design science research in information system and technology (pp. 19-36). springer. http://dx.doi.org/10.1007/978-3-319-39294-3_2 haj-bolouri, a., purao, s., rossi, m., & bernhardsson, l. (2018). action design research in practice: lessons and concerns. in proceedings of the european conference on information systems (ecis). http://dx.doi.org/10.5130/acis2018.ak http://dx.doi.org/10.1007/978-3-031-06516-3_29 http://dx.doi.org/10.1007/978-3-031-06516-3_29 http://dx.doi.org/10.1007/s007790170019 http://dx.doi.org/10.4324/9780203793206 http://dx.doi.org/10.1145/1555619.1555625 http://dx.doi.org/10.25300/misq/2013/37.2.01 http://dx.doi.org/10.1007/978-3-319-39294-3_2 australasian journal of information systems cronholm, gӧbel & shrestha 2024, vol 28, research article collaboration in action design research 30 henriques, t.a. & o’neill, h. (2021), design science research with focus groups – a pragmatic meta-model, international journal of managing projects in business, 16(1), 119140. http://dx.doi.org/10.1108/ijmpb-01-2020-0015 hevner, a. r., march, s. t., park, j., & ram, s. (2004). design science in information systems research. mis quarterly, 28(1), 75-105. http://dx.doi.org/10.2307/25148625 hollnagel, e. (2012). task analysis: why, what, and how. handbook of human factors and ergonomics, 383-396. http://dx.doi.org/10.1002/9781118131350.ch13 iivari, j., & maansaari. j., (1998). the usage of systems development methods: are we stuck to old practices? information and software technology, 40(9), 501-510. http://dx.doi.org/10.1016/s0950-5849(98)00077-9 klein, h. k. & myers. m.d. (1999). a set of principles for conducting and evaluating interpretive field studies in information systems, mis quarterly, 3(1), 67-88. http://dx.doi.org/10.2307/249410 lincoln, y. s., & guba, e. g. (1985). naturalistic inquiry, newbury park, ca: sage publications. http://dx.doi.org/10.1016/0147-1767(85)90062-8 lusch, r. f., & nambisan, s. (2015). service innovation. mis quarterly, 39(1), 155-176. march s t & smith g. (1995). design and natural science research on information technologies, decision support systems, 15(4), 251-266. http://dx.doi.org/10.1016/01679236(94)00041-2 mathiassen, l. (2002). collaborative practice research. information technology & people, 15(4), 321-345. http://dx.doi.org/10.1108/09593840210453115 matzner, m., plenter, f., chasin, f., betzing, j. h., & von hoffen, m. (2018). new service development through action design research in joint research projects. in proceedings of the 26th european conference on information systems (ecis). naderifar, m., goli, h., & ghaljaie, f. (2017). snowball sampling: a purposeful method of sampling in qualitative research. strides in development of medical education, 14(3), 1-6. http://dx.doi.org/10.5812/sdme.67670 peffers, k., tuunanen, t., rothenberger, m. a., & chatterjee, s. (2007). a design science research methodology for information systems research. journal of management information systems, 24(3), 45-77. http://dx.doi.org/10.2753/mis0742-1222240302 penrose, e. t. (1959). the growth of the firm. white plains, ny: sharpe. http://dx.doi.org/10.4324/9780429056604-4 petersson, a. m., & lundberg, j. (2016). applying action design research (adr) to develop concept generation and selection methods. procedia cirp, 50, 222-227. http://dx.doi.org/10.1016/j.procir.2016.05.024 pries-heje, jan, john r. venable, and richard baskerville (2014) soft design science methodology. in o. e. hansen, j. simonsen, c. svabo, s. malou strandvad, k. samson & m. hertzum (eds.), situated design methods. mit press, cambridge, ma, usa. http://dx.doi.org/10.7551/mitpress/9936.003.0008 http://dx.doi.org/10.5812/sdme.67670 australasian journal of information systems cronholm, gӧbel & shrestha 2024, vol 28, research article collaboration in action design research 31 rapoport, r. n. (1970). three dilemmas in action research: with special reference to the tavistock experience. human relations, 23(6), 499-513. http://dx.doi.org/10.1177/001872677002300601 schwaber, k., & sutherland, j. (2011). the scrum guide. scrum alliance, 21(19), 1. http://dx.doi.org/10.1002/9781119203278.app2 scriven, m. (1996). types of evaluation and types of evaluator. evaluation practice, 17(2), 151162. http://dx.doi.org/10.1177/109821409601700207 segal t. (2022, dec 31). what is prescriptive analytics? investopedia. https://www.investopedia.com/terms/p/prescriptive-analytics.asp sein, m.k., henfridsson, o., purao, s., rossi, m., & lindgren, r. (2011). action design research. mis quarterly, 35(1), 37-56. http://dx.doi.org/10.2307/23043488 strauss, a., & corbin, j. (1990). basics of qualitative research. sage publications. susman, g., & evered, r. (1978). an assessment of the scientific merits of action research. administrative science quarterly, 23(4), 582-603. http://dx.doi.org/10.2307/2392581 te’eni, d., seidel, s., & brocke, j. v. (2017). stimulating dialog between information systems research and practice. european journal of information systems, 26(6), 541-545. http://dx.doi.org/10.1057/s41303-017-0067-9 urquhart, c., lehmann, h., & myers, m. (2010). putting the ‘theory’ back into grounded theory: guidelines for grounded theory studies in information systems. information systems journal, 20(4), 357–381. http://dx.doi.org/10.1111/j.1365-2575.2009.00328.x van aken, j. (2004), management research based on the paradigm of the design sciences: the quest for field-tested and grounded technological rules, journal of management studies, 41(2), 219-246. http://dx.doi.org/10.1111/j.1467-6486.2004.00430.x van den akker, j. (1999). principles and methods of development research. in j. j. h. v. d. akker, r. branch, k. gustafson, n. m. nieveen, & t. plomp, t. (eds.), design approaches and tools in education and training (pp. 1-14). springer. http://dx.doi.org/10.1007/97894-011-4255-7_1 vargo, s. l., & lusch, r. f. (2004). evolving to a new dominant logic for marketing. journal of marketing, 68(1), 1–17. http://dx.doi.org/10.1509/jmkg.68.1.1.24036 vaishnavi, v. k., & kuechler, w. (2015). design science research methods and patterns: innovating information and communication technology. crc press. http://dx.doi.org/10.1201/b18448 venable, j., pries-heje, j., & baskerville, r. (2016). feds: a framework for evaluation in design science research. european journal of information systems, 25(1), 77-89. http://dx.doi.org/10.1057/ejis.2014.36 venable, j. r., pries-heje, j., & baskerville, r. l. (2017). choosing a design science research methodology. in proceedings of the 28th australasian conference on information systems (acis). hobart, australia, dec 4-6. vom brocke, j. v., weber, m., & grisold, t. (2021). design science research of high practical relevance. in proceedings of the engineering the transformation of the enterprise (pp. 115135). springer, cham. http://dx.doi.org/10.1007/978-3-030-84655-8_8 http://dx.doi.org/10.2307/2392581 australasian journal of information systems cronholm, gӧbel & shrestha 2024, vol 28, research article collaboration in action design research 32 walsham, g. (1995). interpretive case studies in is research: nature and method, european journal of information systems, 4(2), 74-81. http://dx.doi.org/10.1057/ejis.1995.9 wilson, p. f., dell, l. d., & anderson, g. f. (1996). root cause analysis: a tool for total quality management. the journal for health care quality, 18(1). http://dx.doi.org/10.1111/j.19451474.1996.tb00823.x winter, r. (2013). towards a framework for evidence-based and inductive design in information systems research. in proceedings of the european design science symposium (pp. 1-20). springer, cham. http://dx.doi.org/10.1007/978-3-319-13936-4_1 copyright: © 2024 cronholm, gӧbel & shrestha. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v28.4281 http://dx.doi.org/10.1007/978-3-319-13936-4_1 http://creativecommons.org/licenses/by-nc/3.0/au/ https://doi.org/10.3127/ajis.v28.4281 paper title (paper title style) australasian journal of information systems johnstone & tate 2017, vol 21, research article improving it project governance 1 improving it project governance: a reflective analysis based on critical systems heuristics david johnstone victoria university of wellington david.johnstone@vuw.ac.nz mary tate queensland university of technology abstract it project governance involves establishing authority structures, policies and mechanisms for it projects. however, the way governance arrangements are implemented can sometimes exclude or marginalise important stakeholders. in this paper, we use critical systems thinking, and the notions of boundary critique and entrenched structural conflict, to inform a critical reanalysis of a case study where the governance proved relatively ineffective. we use the ‘twelve questions’ from the critical systems heuristics (csh) approach to diagnose problems with governance arrangements and suggest solutions. based on this, we suggest the csh approach has theoretical and practical efficacy for improving it project governance in general. keywords: it project governance; paradigms; critical systems heuristics; boundary critique “nothing about us without us” (anon)1 1 introduction it projects typically involve a range of stakeholder groups. adapting freeman’s (1984) original, generic definition, a project stakeholder is any group or individual who can affect, or is affected by, the conduct and outcomes of the project (mitchell, agle, & wood, 1997). different stakeholder interests generate different agendas, and stakeholder conflict within it projects is common because “…independent people must work together to achieve multiple objectives in an environment of flux, ambiguity, and scarce resources.” (hammer, 1997, p.165). while it is true that many published guidelines on the management of organisational projects, such as the project management body of knowledge (pm-bok), the open group architecture forum (togaf) and prince2 now have more comprehensive sections on stakeholder management and engagement, there is still a sense that conflict should be resolved by dealing with troublesome stakeholders, rather than seeing the conflict as potentially constructive (richet, ngwenyama, & rowe, 2016). we argue, following roberts, cheney, and sweeney (2002), that a broader, more nuanced understanding of stakeholders’ perspectives is required to address their concerns, incorporate their views into the decision-making process, and reduce damaging project conflict. this means examining the framework under which project performance monitoring, decision-making, and managerial action takes place. that is the role of project governance (muller, 2009; pmi, 2013; turner, 2009). this has already been recognised at a corporate governance level: “the critical governance tasks … are to ensure effective negotiations, coordination, cooperation, and conflict resolution to maximize and distribute the joint gains among multiple parties of interest” (kochan & rubinstein, 2000, p. 370). we examine how this can be enacted at an it project governance level. 1 this is a slogan and credo that originated in disability rights groups, and has subsequently been adopted by other interest groups to communicate the idea that no policy should be decided by any representative without the full and direct participation of members of the group(s) affected by that policy. (http://en.wikipedia.org/wiki/nothing_about_us_without_us) australasian journal of information systems johnstone & tate 2017, vol 21, research article improving it project governance 2 we applied ulrich’s (1983) critical systems heuristics (csh), employing boundary critique and his ‘twelve questions’ (ulrich & reynolds, 2010) to perform a critical analysis of the governance of an it project. we emphasise that this project was not conducted from its inception as critical research (myers & klein, 2011) per se. our contributions are to demonstrate how csh can be used as a reflective and diagnostic tool to generate fresh insights in an it project context, and to suggest how critical principles can be incorporated into our understanding of it project governance. to our knowledge, this is the first study to offer a critical analysis of it project governance. csh is based on critical systems approaches used by social scientists for framing, analysing and intervening in complex social situations involving multiple, interconnected factors that are frequently characterised by significant conflict between stakeholder groups. the notion of boundary critique, and the application of csh questioning, are tightly linked: “in csh, the process of questioning boundary judgements is crucial” (ulrich & reynolds, 2010, p.263). it projects can be characterised as complex social situations involving multiple stakeholder groups, sometimes with incompatible perceptions of success (davis, 2014), goals and values. it project governance can be characterised as a framework for setting and managing boundaries and interactions between various stakeholder groups in which project management is carried out. we therefore propose csh as a technique for informing it project governance design. our research questions are: rq1) what insights can csh, as a diagnostic tool, contribute to our understanding of it project governance in this case study? rq2) how can csh be used effectively as a tool for improving it project governance? the rest of this paper is structured as follows. we offer a literature review, then we introduce the original case context and the methodology used in this study. next we present the results, followed by a discussion and a conclusion. 2 critical systems heuristics and boundary critique this section introduces a brief background to csh, boundary setting, boundary critique, and csh literature. 2.1 boundaries, social systems and conflict analysis of boundaries in social systems begins with the premise that in complex social systems there are a wide range of interconnected forces undergoing continuous interaction; everything is directly or indirectly connected to everything else. “however, you can’t have a god’s eye view…so there are inevitable limits to understanding, and it is those limits that we call boundaries” (midgley & richardson, 2007, p.167). boundaries start out as cognitive boundaries; as both midgley (1992) and ulrich (2000) point out, internal perceptions of the boundaries of a problem or situation are a general cognitive prerequisite for identifying and understanding a system of concern to the respective decision-maker. in social systems, boundaries may be conceptual or social constructs; that is, they are perceived, negotiated or created by the participants in the system, rather than being ‘real’ characteristics that are present in the environment (churchman, 1970). boundaries “mark the [real or perceived] inclusion or exclusion of stakeholders, people and issues. they demarcate what is relevant to an analysis” (midgley & richardson, 2007, p.172), but they start as beliefs about the importance or legitimacy of various groups. in many social situations, there may be multiple, sometimes conflicting, views of the appropriate boundaries for inclusion and exclusion of issues and stakeholders. this type of mismatch is conceptualised in figure 1 (midgley, 1992). the inner ellipse represents a boundary judgment (a) by one group, and the next ellipse is a boundary judgment (b) by a second group. midgley called the area in between these two boundaries the marginal area. he suggested that each of the boundaries reflects a value judgement about inclusions and exclusions. when these value judgements are not shared between the stakeholder groups, australasian journal of information systems johnstone & tate 2017, vol 21, research article improving it project governance 3 damaging conflicts can arise. further, if the value conflicts are not recognised and addressed, they can ‘stabilise’ and perpetuate over longer periods of time, so that although immediate conflict situations may be resolved, divisions between stakeholder groups can continue for an extended period, generating further instances of conflict (midgley & pinzon, 2011). wider system marginalised elements primary elements a b figure 1. marginalisation through boundary setting (adapted from midgley, 1992). 2.2 boundary critique using critical systems heuristics ulrich (1987) developed a set of questions which he called 'critical systems heuristics' (csh). these questions invite stakeholder participants to reflect on what they believe the current situation is, and what, in their view, it ought to be. the questions focus on four areas: motivation, control, expertise, and legitimacy. motivation focuses on the purpose and beneficiaries of the system and whose interests are being served. control establishes who has decision-making authority and what resources they have at their disposal. expertise describes what forms of knowledge are necessary, and where that knowledge resides. legitimacy considers the worldview and potential sources of oppression inherent in a social system. ulrich suggests that the csh approach can be used as a reflective and diagnostic tool by comparing the ‘descriptive mapping’ of a situation (also called the ‘is’ analysis) with the ‘ideal mapping’ (identifying the ‘ought’) (ulrich & reynolds, 2010, p.266). we have adapted ulrich’s twelve questions as ‘was’ and ‘ought’ questions for coding and reflecting, respectively, on the case data. these are listed in table 1. focussing question questions for coding the case data motivation: focus on the beneficiary 1. who was/ought to have been the beneficiary of the project? 2. what was/ought to have been the purpose of the project? 3. what was/ought to have been the project’s measure of success? authority: focus on the decision-maker 4. who was/ought to have been the decision-maker that controlled the conditions for project success? 5. what resources were/ought to have been controlled by the decisionmaker? 6. what resources were/ought to have been outside the control of the decision-maker? knowledge: focus on the expert / professional 7. who did/ought to have provided the relevant knowledge and skills? 8. what were/ought to have been the necessary knowledge and skills? 9. what were/ought to have been regarded as assurances of successful implementation? australasian journal of information systems johnstone & tate 2017, vol 21, research article improving it project governance 4 focussing question questions for coding the case data legitimation: focus on those affected, but not involved 10. who was/ought to have been representing the interests of those negatively affected by, but not involved with the project? 11. what were/ought to have been the opportunities for the interests of those negatively affected to have expression and freedom from the worldview of the project? 12. what space was/ought to have been available for reconciling differing worldviews regarding the project among those involved and affected? table 1. twelve focussing questions for coding and analysing the case data. the csh method falls generally within the stream of critical systems thinking (cst) research, which has many variants and has been the subject of intense methodological and paradigmatic debates (jackson, 2003; midgley, 1997; ulrich, 2012). we remain largely agnostic in these discussions; in this, we align ourselves with the recent thinking of ulrich (2012) himself, who lamented that “the discussion about cst has remained largely an insider discussion” (p. 1233). ulrich (2012) emphasised that his method is intended to be used and to be usable and therein “resides the basic idea and value of ‘critical’ systems thinking: it can support reflective practice with special regard to contextual selectivity” (p.1233). nevertheless, we acknowledge one criticism which may affect the practical efficacy of the method; that it is utopian (flood and jackson, 1991), and is impractical when there is a clear power imbalance: “why should the involved bother to take account of the views and interests of those who are affected but not involved?” (jackson, 2003, p 227). we aim to explore the practical value of csh for informing it project governance arrangements, rather than contributing to the theoretical and paradigmatic debates. 2.3 it project governance and it project management the concepts of governance and management, within an organisational context, are tightly linked, though distinct in purpose. “the governance system defines the structures used by the organization, allocates rights and responsibilities within those structures, and requires assurance that management is operating effectively and properly within the defined structures. the role of management is to manage the organization within the framework defined by the governance system; this applies particularly to the governance and management of projects.” (too & weaver, 2014, p.1385, emphasis added). more specific conceptualisations of project governance, however, continue to offer a diversity of opinion (biesenthal & wilden, 2014). nevertheless, there is some commonality across most definitions of project governance arrangements. these include the allocation of decision-making authority and responsibility (e.g. pinto, 2014; too & weaver, 2014); development of policies that guide and constrain decision-making (e.g. klakegg, williams, magnussen, & glasspool, 2008); and implementation of mechanisms that support decision-making and implementation, including processes, procedures, and methodologies (e.g. those described in practitioner guidelines such as the pm-bok). this provides a coherent definition of it project governance (itpg), where these arrangements “…specify why, where and when decisions are made (policy), who should make them (authority), and how they should be made and implemented (mechanisms).” (johnstone, huff, & hope, 2006). project governance may operate at different levels within the organisation, depending on the project context (bekker, 2014). for example, biesenthal and wilden (2014) identified three distinct levels: the organisational level (overseeing portfolio direction, project sponsorship, and high-level reporting), the project management office level (overseeing portfolio/ programme management, project evaluation and prioritisation), and project level (overseeing project management, delivery, objectives, expectations and outcomes). however, how these project governance levels are realised in practice may vary considerably across different australasian journal of information systems johnstone & tate 2017, vol 21, research article improving it project governance 5 organisations. a commonly used authority structure for itpg at the project level is a project steering committee (van grembergen & de haes, 2009), which is responsible not only for project oversight, but high-level decision-making where issues that could not be resolved by the project manager were escalated (jiang, klein, & carr, 2002). 3 method for this study, we analysed previously collected case data using a boundary critique lens and abductive reasoning. we briefly describe the method used to capture the original case data, and the method used to perform the analysis for this study. 3.1 the method used for the original case study research originally, the research initiative employed a case research method (eisenhardt, 1989) investigating it project governance and stakeholder conflict (johnstone, 2010). data was collected across five cases, where each case represented recently completed it project(s) for a different organisation. semi-structured interviews were conducted with key participants involved with the it project for each case. archival material was used to augment and triangulate the interview data. this paper is based on data collected from one of these cases. 3.2 the method used for this study we started with an abductive approach based on a critical reading of the original case transcripts to arrive at some initial propositions which we developed based on critical systems thinking literature. the viewpoint represented in the analysis is therefore that of the research team, adopting a critical stance with regard to the original case data. abductive reasoning can re-examine previously observed phenomena: abduction…works through interpreting or recontextualizing individual phenomena within a contextual framework, and aims to understand something in a new way, from the perspective of a new conceptual framework” (kovács & spens, 2005, p.138). abduction can be used to link existing theory to case analysis by creating “patterns of expectation” (propositions) which can be confirmed, or disconfirmed, by the detailed case data (evers & echo, 2006). our high-level propositions were that some stakeholder groups were marginalised, the project governance arrangements failed to adequately address this, and project conflict ensued. we asked ourselves “if one stakeholder group were genuinely marginalised, or felt marginalised, in this project, how well would this explain the case data?” this informed the ‘is’ perspective in the analysis. to do the detailed analysis, initially, the entire set of transcripts for the original case, and all relevant archival material was re-read. we then used pattern matching (campbell, 1966; yin, 2013). a pattern is a characteristic of qualitative data analysis that focuses on patterns of relationships between variables in a holistic way. patterns can be derived from literature, theory, or even reflection and experience (almutairi, gardner, & mccarthy, 2014). in our case, the twelve csh questions formed the ‘patterns’. for example, for the questions associated with authority, we would scan for a description of a stakeholder group (e.g. ‘finance’, ‘head office’, ‘frontline staff’) in combination with terms associated with authority such as ‘decision’, ‘control’, ‘authority’ ‘resources’ or their synonyms. where multiple informants commented on the same issue, we compared data across informants to understand the convergence or divergence between their viewpoints. this analysis was then compared with our initial expectations and high-level propositions to see if the evidence supported the assertions (evers & echo, 2006). following that, we asked “if the it project governance arrangements had been configured differently, and more inclusively, what might the project have looked like?” this formed the ‘ought’ perspective. we note we were not able to perform a direct intervention in the project in question. however, the ‘ought’ perspective from the csh questions provided important insights into the type of interventions that could usefully have been carried out, and yielded lessons for future it project governance arrangements. australasian journal of information systems johnstone & tate 2017, vol 21, research article improving it project governance 6 4 the case background 4.1 the organisation and the project2 the government organisation that established the context for the project is responsible for providing emergency services on a national basis, involving trained professionals who carry out the frontline duties. they are operationally managed through stations geographically dispersed throughout the country, and are co-ordinated hierarchically through districts, then regions, and finally the centrally-located head office (ho). the organisation had a considerable investment in the equipment (assets) required to carry out the emergency services, most of which was located on specialist vehicles or housed in the stations. these assets had to be managed both physically and financially. each station was responsible for regular maintenance checks and repairs of their equipment, using their own asset management system. well-maintained equipment was critical to front-line personnel safety and job effectiveness. ten years earlier, as part of controversial economic reforms, a businessman with no prior experience in emergency services was appointed ceo of the organisation. efficiencies were sought through cost reduction, primarily through reduction of staff and equipment, which caused acrimony among frontline staff, who saw it as a threat to their professionalism, safety and ability to perform effectively. there was extensive conflict at the time, culminating in national strike action. the experiment did not last long, the ceo was removed, and the ‘efficiencies’ largely reversed, but the memory of negative emotions and conflict between ho and the stations regarding equipment (assets) remained very strong. the project currently in question formed part of the roll-out of a new erp system. the first module, a financial management information system (fmis), had been installed successfully. the implementation of the second erp module – an asset management information system (amis) – was more problematic. the manual systems struggled to provide the financial data required by the finance department (‘finance’) at ho. the new amis system was meant to standardise both physical and financial asset management electronically, from the station level through to the national level. following a similar structure to the previous erp module, governance was established for the implementation of the amis system. the ho executive team, comprising the chief executive officer (ceo), chief information officer (cio), chief financial officer (cfo) and other organisational executives, assumed responsibility for project governance at the organisation level. a key role of the executive team, from a governance perspective, is establishing and allocating the membership of the project steering committee (psc). psc membership for both the fmis and amis projects included mostly the same people, mostly from ho, including the cio and a strong representation from finance (who were seen by ho as the primary erp client). the psc hired external consultants for two key roles: the project manager and a quality assurance (qa) consultant, who were both added to the psc. there was little by way of policy set by the executive team, leaving the psc to act relatively autonomously. despite occasional reporting to the executive team, there were no formal mechanisms available (beyond the psc) to resolve the issues that ensued with amis. the psc met once a month, being responsible for itpg at the project level. they delegated extensive decision-making authority to the project manager, who had responsibility for day-to-day management regarding the conduct of the project, delivery of outcomes, allocation of resources, and stakeholder management. the psc had oversight and final responsibility for these functions and received regular status reports from the project manager. they also got involved directly in decision-making from time to time, when they were unhappy with the performance of the project manager. the only other non-ho psc member was a regional manager, co-opted onto the psc only after problems with amis appeared, because of his 2 the project names and position titles have been altered to help preserve anonymity. australasian journal of information systems johnstone & tate 2017, vol 21, research article improving it project governance 7 knowledge of the business processes in the organisation at the station, district and regional levels. he was the only member of the psc with any operational, frontline experience. 4.2 the research participants a brief description of those interviewed is provided in table 2. there were two project managers externally contracted for the amis project, one for the early part of the project, where the majority of events related to this paper occurred, and her replacement for the latter part of the project. unfortunately, the original amis project manager had left the country at the time of interviewing, while the replacement did not wish to be interviewed. interviewee position it project governance body membership responsibility and stakeholder representation chief information officer (cio) ho executive team; project steering committee (amis) executive team representative and the most senior manager of the it department national finance manager project steering committee (amis) second most senior representative of the finance department (after the cfo), also from ho senior vendor consultant – project team (amis) [none] responsibility for client management qa consultant project steering committee (amis) independent advisor reporting on both system and process quality regional manager project steering committee (amis) co-opted later onto the psc for his asset knowledge at the station level table 2. case study interviewees. 4.3 the events the amis psc agreed from the outset that a critical pre-requisite for an integrated, national asset management module was to conduct a thorough inventory and classification of the organisations’ assets. the intention was to collect the data from stations and vehicles across the country, and use this to populate the new amis database. finance already possessed highlevel financial data, but was unable to break it down. consequently, finance developed a classification system for the assets, and contracted an external firm to work with the organisation and visit each station across the country to collect the necessary data. in practice, this meant a team of two (one contractor, and one frontline staff member from the relevant district) would arrive at a station and ask the personnel there to strip down the vehicles of equipment, as well as note what was in the station, and the collection team would record the information. this was a time-consuming, unpopular exercise: “...we were pushing for accuracy, [and so frontline staff] had to go off and check their assets … it’s just an extra burden for them. … [capturing] information that is of importance and value to the head office people who make these policy decisions. … but not to the people on the outside who are actually going to do all the work." [cio] the collection took longer than expected; it was not seen as a high priority at station level. furthermore, feedback from the data collection teams suggested finance’s classification system for the asset information was poorly designed, and they struggled to determine how the asset information was to be recorded using it. at the end of the exercise, the station-level data was aggregated and compared with ho’s national figures. it was clear the collected data was far from accurate, leaving the psc puzzled as to why the collection process failed: "all the processes were there to make sure we got the best data that we could. …so, in theory, we should have got full cooperation and really good data out of it." [national finance manager] australasian journal of information systems johnstone & tate 2017, vol 21, research article improving it project governance 8 consequently, the psc decided to repeat the data collection process, this time with an asset classification scheme that was revised with the help of the regional manager on the psc, who called on his frontline experience and knowledge. however, resistance was still an issue as many stations were now even more reluctant to cooperate. the collected asset data, although more accurate than before, was still regarded by the psc as inadequate. after considerable debate within the psc, despite a likely budget and schedule blowout for the amis project, a third round of data collection was conducted. the psc also fired the amis project manager at this point. the resulting aggregated figures had improved again, though many on the psc were still unhappy with the discrepancies. after considerable debate, the psc decided this time they would not pursue a fourth attempt. as the cio was to observe, there was a “collective weariness” over the issue within the psc, and they were unsure how frontline operations staff would react to yet another round of data collection. 5 results based on the analysis of the re-coded data, responses to the 12 questions adapted from ulrich (1987) are presented. the information provided was sourced from two or more participants. no factual inconsistencies between participants were apparent. 5.1 csh analysis we employ the 12 questions as an analytical tool in order to make sense of the situation of interest. in particular, we aim to identify the differences between stakeholder views, representing these multiple perspectives as boundary judgements. 5.2 sources of motivation 5.2.1 question 1: who ought to have been/was the beneficiary of the project? the project was proposed by finance to improve their ability to manage asset financial information. this was supported by the executive because this improved capability would also support the organisation’s strategic objectives to utilise government funding more effectively. because amis would standardise physical asset management electronically, it was assumed (by ho generally, and finance specifically) that station managers would also benefit. however, station staff did not see themselves as beneficiaries of amis, as they already had manual systems of their own, which generally worked well at a local level, and issues around data aggregation at higher levels did not affect them: “…it [asset classification] would just be done late … and skimpily. we would get figures down that were not trustworthy. it became, to [the stations], a low priority task.” [cio]. a gap exists between ho, who believed the stations will benefit, and the stations, who felt they would not only fail to benefit, but would incur additional work for someone else’s benefit. frontline staff were responsible for initially recording and classifying the physical assets, and for updating the system on an ongoing basis, and so ought to have received some reward or recognition as an incentive, although how this would work in practice is unclear. 5.2.2 question 2: what ought to have been/was the purpose of the project? amis was meant to provide finance with direct financial control over asset tracking/ purchase/value at all breakdown levels (individual asset, station, district, region, and national). more broadly, the standardised asset processes would integrate electronically with other organisational processes as part of the overall erp system, which enable better strategic planning and improve high-level reporting to government. again, there was a difference between this official ho perspective and the station perspective, who apparently assumed that the purpose was to reduce costs by rationalising the distribution of assets based on (ho’s) perceived need. given the historical bad feeling associated with ho initiated cost reduction, the stated purpose ought to have included an explicit assurance that this rationalisation was not the intention of ho. australasian journal of information systems johnstone & tate 2017, vol 21, research article improving it project governance 9 5.2.3 question 3: what ought to have been/was the measure of project success? the measures of success for the project as a whole were based on the traditional project parameters of budget, time, and functionality, including overall integration with the fmis module. the asset data collection exercise, while included as part of the amis project, was seen as a sub-project with its own measures of success: all station managers signed off on their data provision, and the success measure was that the aggregated station-based data was expected to be ‘acceptably’ close (as decided by the amis psc) to the national totals already possessed by finance. achieving this proved difficult. first, requiring station managers to sign off on the data they provided would only ensure success if they cared sufficiently about the accuracy, and many did not, particularly as the effort required to collect the station-level data was onerous. second, the psc had not agreed, prior to the audit, what level of aggregated totals would count as acceptable. consequently, there was considerable debate within the psc after each data collection round, and unanimity was never achieved. the qa consultant consistently advised the psc to agree on acceptable levels prior to each audit round. further, he urged the psc to treat the asset audit as an independent project, with a different project manager and separate resources, so that the amis project did not fall further and further behind in both budget and schedule. this advice was not taken. 5.3 sources of control 5.3.1 question 4: who ought to have been/was the decision-maker that controlled the conditions for project success? the authority structures were centred on ho, the psc, and the project manager. the measures of success for the amis project were set by the executive team at ho, but ongoing decisionmaking as to how ‘success’, based on these measures, was to be achieved was delegated to the psc. the project manager was responsible for operational decision-making regarding the project team and working with the vendor, but only within the decision-making framework provided by the psc (the project activities were evaluated by the psc through regular project reporting). thus, project control was exerted both directly (through decision-making) and indirectly (through governance arrangements) by the ho/finance dominated psc: "…when you see the make-up of the senior management team, it’s weighted towards the people who manage the business at national office.” [vendor consultant]. however, the amis project in general, and the audit sub-project in particular, required direct involvement from frontline staff at the station level. but, as demonstrated 10 years earlier, the frontline staff had the power to resist and overturn directives from ho. there were no mechanisms available to resolve the conflict over the audit process, and so the situation was handled on an ad hoc basis. after the second round of data collection still failed to satisfy the psc, the issue was briefly escalated to the ceo, who asked all regional managers to enforce co-operation from station managers. this did not, apparently, appreciably improve the situation: “...[and the stations] were like separate governments. …[they were commanded to cooperate]... . but then [they responded]…‘no, you aren’t coming onto my patch’…‘we do it this way’ or ‘we do it differently’…a lot of that sort of talk [ensued].” [vendor consultant] 5.3.2 question 5: what resources ought to have been/were controlled by the decision-maker? the psc had full control of the resources (via the budget) allocated by the executive team for the project, who hired the relevant staff, paid for vendor involvement, etc. the project manager had control of the operational resources, within the constraints set by the psc. the project manager was also meant to have control over access to the assets, but was unable to achieve this, and this was one of the primary reasons for her removal. however, escalation to higher authorities did not improve matters. given how onerous the actual audit was for the stations, it could be argued that they should have been provided with additional resources to help – although it is not clear what these additional resources could have been, because the audit required specialist knowledge and skills held by a limited pool of professionals. australasian journal of information systems johnstone & tate 2017, vol 21, research article improving it project governance 10 5.3.3 question 6: what resources ought to have been/were outside the control of the decision-maker? the organisation owned the assets, and finance was responsible for their financial management. however, the assets were physically resident in the stations and under direct control of the station managers. further, the station staff had the knowledge, expertise and labour required to conduct the asset audit effectively. for these reasons, it could be argued that the physical control of the assets should remain outside the control of the finance-dominated psc. 5.4 sources of knowledge / expertise 5.4.1 question 7: who ought to have/did provide the relevant knowledge and skills? most of the necessary skills and knowledge were provided through the psc and project team: the vendor and it dept provided software development and implementation; the finance dept. provided financial asset management; the amis project manager, project management; and the qa consultant, independent quality assurance of project-related practices. nevertheless, there were three areas of concern. the first was a relative lack of experience with major, complex it projects. until the decision to implement an erp system had been made, it-related change had mostly been on a smaller scale. the qa consultant had the requisite experience and had offered advice regarding the separation of the asset audit as an independent project and the agreement on what constituted satisfactory audit data – but this advice was not heeded. second, the original project manager was made responsible for gaining cooperation from frontline staff, but did not seem to possess the ability to do so. two interviewees suggested that being female, she may have found it difficult to achieve rapport in a very ‘blokey’, male-dominated culture. another problem was that she, as a contractor, was seen as an outsider. third, as the need to co-opt experience with frontline work into the psc demonstrated, the relevant knowledge about the physical assets was not initially available: "there was not a great communication line to tell people what they were doing, to encourage them to help, and so the data [the collection team] brought back was, frankly, compromised." [regional manager] 5.4.2 question 8: what ought to have been/were the necessary knowledge and skills? as noted above, most of the requisite skills and knowledge to successfully complete an it project were available. however, there was a failure of the psc to recognise the need for the asset knowledge and asset handling skills possessed by the frontline staff. there was also, it seems, a failure to recognise the value of experience with large it projects, as offered by the qa consultant. 5.4.3 question 9: what ought to have been/were regarded as assurances of successful implementation? the psc were responsible for monitoring and agreeing upon the budgetary and scheduling objectives, with the help of the qa consultant. the cfo and other members of finance were responsible for ensuring the required level of financial asset management functionality was achieved. it was not clear, from the data collected, who was responsible for the equivalent physical asset management functionality. given the initially poor classification system design provided by finance, it is possible they were initially the primary arbiters of success for the asset audit, before this role eventually fell to the psc. in spite of possessing the greatest knowledge about physical asset management on the psc, the co-opted regional manager arguably ought to have had a major role, but his experience was not harnessed. 5.5 sources of legitimation 5.5.1 question 10: who ought to have been/was representing the interests of those negatively affected by, but not involved with the project? frontline staff were negatively affected, first, by being required to invest considerable time and effort into the asset audit, and second by losing much of the control they had over their own australasian journal of information systems johnstone & tate 2017, vol 21, research article improving it project governance 11 asset management. they were not provided with representation in either the governance or management bodies regarding key decisions relating to the design of the system and the conduct of the project. although they were involved with the project, this consisted primarily of being told what to do, with little apparent consultation. it can be argued that with representation on the psc, frontline staff would have had greater, likely positive, influence on the events that caused project delays. although co-opted onto the psc for different reasons, the regional manager did at least try to explain the station-level perspective, but with limited success. furthermore, because a regional manager is relatively high in the authority hierarchy, the frontline staff may have preferred someone closer to their level and correspondingly further from ho. 5.5.2 question 11: what ought to have been/were the opportunities for the interests of those negatively affected to have expression and freedom from the worldview of the project? ho, especially through their representation on the psc, held virtually all legitimacy. emancipation for frontline staff did not occur. as problems with the audit ensued, the psc were aware the frontline staff were unhappy, yet still assumed that coercion, as opposed to consultation, was the answer: “when it was brought to their attention that they had an issue, [the psc] tried to do everything that needed to be done to bring it back on track. but that was based on a misunderstanding about just how difficult it was. the issue was essentially allowed to drag on." [qa advisor]. “everything that needed to be done” did not include the psc recognizing the structural conflict and the underlying marginalisation of a major stakeholder group. if anything, their attempts at resolving the issue appeared to further entrench the conflict. 5.5.3 question 12: what space ought to have been/was available for reconciling differing worldviews regarding the project among those involved and affected? ho-based interviewees suggested frontline staff were seen by ho as resistant to change, and many of them computer illiterate. it was assumed that, in a command-and-control environment, operational staff would do what they were told. frontline staff had a worldview based on safety, public service and autonomy at the station level; they saw amis as interfering with that autonomy and forcing them to adopt a system devised by ho. the worldview of ho was managerial and militaristic; it privileged the agenda of the head office managers, and adopted a militaristic command-and-control approach towards frontline staff. the worldview ought to have recognised the dependence of the project on the professional expertise and co-operation of frontline staff and adopted a more consultative approach. 5.6 boundary critique as both midgley (1992) and ulrich (2000) point out, boundary setting is a general cognitive prerequisite for identifying a system of interest to the respective decision-makers. the frontline community were not represented in the governance structure for the project. in fact, until the regional manager was co-opted, the psc did not include anyone who had ever been involved in frontline operations. establishing governance structures is an explicit boundary setting process. it establishes decision-making rights – frontline staff were not part of the decision-making as to how assets would be coded or data would be captured. it legitimises some stakeholders and their value systems – by controlling the governance processes, ho (and particularly finance) legitimised their values and world-view. in the minds of the decisionmakers, the frontline staff were ‘outside’ the authority and decision-making structures. the boundary decision, and the value conflicts embedded in the project governance structure, are shown in figure 2. the value conflicts over asset management were considerable, and understandable when the respective roles of the two groups were considered. frontline staff were extremely engaged with the operational maintenance of their assets, keeping them work-ready, safe, and in good australasian journal of information systems johnstone & tate 2017, vol 21, research article improving it project governance 12 order. they were also very engaged in the correct, professional use of highly specialised and very expensive equipment. lives, literally, depended on this. they were much less concerned with ensuring that the assets were accurately described in a system that added no value to the day-to-day conduct of their work, and they did not need a system to remind them to maintain or replace their assets; their safe use was motivation enough. head office representatives, by comparison, did not have the visceral involvement with equipment that frontline staff did, and would probably never be able to fully understand this worldview. their worldview was based on the responsible financial management of a large asset pool. a boundary critique suggests that some level of values conflict was almost inevitable. 5.6.1 entrenched structural conflict we can see in figure 2 that the establishment of governance structures created real boundaries that reflected the cognitive boundaries, value conflicts, and different worldviews of people involved in the project. however, theorists of boundary conflicts suggest that these conflicts can have impacts beyond the boundaries of the immediate situation and life-span of the system (or project) being considered. if the value conflicts are not recognised and addressed, they can ‘stabilise’ and perpetuate over longer periods of time, so that although immediate conflict situations may be resolved, divisions between stakeholder groups can continue for an extended period, generating further instances of conflict (midgley & pinzon, 2011). this structural conflict, also known as relationship or affective conflict, “…is detrimental to individual and group performance, member satisfaction, and the likelihood a group will work together in the future.” (jehn & mannix, 2001, p.239). a major insight from applying the boundary critique approach to the case study was the identification of entrenched structural conflict between stations and ho which arose from the previous history of intense conflict between ho and stations over assets. while the amis project was seen by ho as a continuation of the phased erp implementation, participants pointed out that the issue of asset management was organisationally sensitive for historical reasons. this conflict was resistant to regular forms of communication, and was sufficiently entrenched that the good efforts of individuals had no impact: “you tell them there is no sinister, hidden purpose … we’re not trying to take your assets. we’re just trying to get a fair reflection of what they actually are. then everyone delves back into the past, and their memories are [long]…. so there is that general mistrust." [national finance manager]. this mistrust highlighted just how different the worldview of the frontline staff was: "[this was] because the people out there think these people in [ho], in the main, they’re just a bunch of selfish idiots… particularly because they wear a suit – they don’t wear a uniform.” [regional manager]. the project was starting with an atmosphere of mistrust, suspicion, and entrenched structural conflict between two major stakeholder groups which was not recognized or addressed. frontline staff head office values: values assets as essential tools maintaining public & personal safety values: values assets as financial resources boundary: entrenched conflict based on historical mistrust and differing values figure 2. marginalisation of frontline staff by the executive team australasian journal of information systems johnstone & tate 2017, vol 21, research article improving it project governance 13 6 discussion in this section, we discuss the implications of our application of csh principles and boundary critique in the context of our wider research questions. we recall these were: rq1) what insights can csh, as a diagnostic tool, contribute to our understanding of it project governance in this case study? and rq2) how can csh be used effectively as a tool for improving it project governance? earlier, it project governance was defined as a set of arrangements based around three components: the allocation of decision-making authority, establishing rights and responsibilities; the creation of relevant policies that constrain and guide decision-makers; and the establishment of mechanisms, such as processes, procedures and methodologies that can be used by decision-makers as aids to the conduct of the project. 6.1 descriptive mapping of the case we address rq1 by examining the ‘descriptive mapping’ of the case situation, based on the diagnostic ‘was’ analysis of the 12 questions. there were two levels of itpg in operation. at the project level, the psc had allocated operational decision-making authority to the project manager, had policies in place constraining and guiding her decisions regarding allocation of resources, frequency of reporting, working with the project team (including those from both the it department and the vendor), and provided mechanisms for reporting on progress, including budget, schedule and functionality measures. however, problems arose when issues that could not readily be resolved by the project manager (especially regarding working relationships with frontline staff) were escalated to the psc. it was then that the psc were acting as decision-makers within the higher level governance framework established by the executive team. at the higher level, the executive team allocated decision-making authority to a psc almost exclusively comprised of ho personnel, with a strong bias towards finance, who were seen as the main beneficiaries of the new system. the lack of clear policy on how to deal with issues affecting the project only emphasised the lack of experience the psc membership had with larger, more complex it projects, and their tendency to act on an ad hoc basis. furthermore, no formal mechanisms for consultation and communication with frontline staff were available. thus, while it can be argued that the psc were making many of the mistakes, the root problems appear to go back to the governance at the organisational level. the executive team allocated authority to a committee in its own likeness, disregarding (albeit unconsciously) the cultural gap (“suits” vs. “uniforms”) and entrenched conflict between ho and the front line. the csh questioning highlighted the dominance of ho in terms of system benefits, control and knowledge recognition, and the lack of legitimacy held by those at the lower end of the command-and-control hierarchy. 6.2 ideal mapping of the case we address rq2 by examining the ‘ideal mapping’ of the case situation, based on the ‘ought’ analysis of the 12 questions. here we present an alternative narrative for the project, as it might have unfolded if the governance were configured differently. the purpose is to extrapolate the likely implications of a more balanced and inclusive governance structure; to derive insights for this case, and for the design of it project governance arrangements in general. it was argued that many of the problems experienced in amis, while exacerbated by some of the decision-making by the psc, could ultimately be traced back to the governance at the organisational level. csh revealed the failure to identify, respect and legitimise an important stakeholder by the executive team, instead ensuring project level governance continued to reflect the worldview and values of ho. allocating decision-making authority to a stakeholder representative body like a project steering committee needs to be fully representative, not only to be fair, but to help prevent, mitigate or resolve damaging conflict. australasian journal of information systems johnstone & tate 2017, vol 21, research article improving it project governance 14 organisational itpg could also include mechanisms (including the project methodology), and policies requiring their use, to: enable open, consultative communication channels with affected parties; ensure sources of knowledge and skills are both recognised and utilised; resources are allocated to those that need them to do their jobs (beyond the needs of ho); and, importantly, formally recognise and address conflict between stakeholders. as midgley and pinzon (2011) note, “a good participative process can often support people in exploring their differences without them ever experiencing this as a conflict.” (p.1545). one of our contributions is to demonstrate that it project governance is inherently and by its nature a boundary-setting exercise. if we understand it as such, and apply a csh lens, we see that establishing and applying it project governance frameworks can never be value neutral but will always contain embedded worldviews. in the vast majority of cases, these are those of the system sponsor. dissenting stakeholders are seen as a problem to be overcome, not as legitimate proponents of alternative world-views and value-systems. marginalisation or hostility towards some project stakeholders can be understood as a direct outcome of the worldview of those who put it project governance arrangements in place. since frontline staff were professionally, intimately involved with the physical assets they used, the ho worldview also needed to account for the expertise and knowledge they possessed, and the benefits that should accrue for their direct involvement with the project. rather than attempting to move control of the assets to higher levels of decision-making, the controllers of the assets should have been moved to higher levels of decision-making. this raises the issue of coercion – if there is a power imbalance, how can csh help if the powerful simply close debate? in the case discussed here, the power imbalance was evident with a strong command-and-control culture; an expectation that frontline staff will simply do what they are told. but the affected were involved in this case, and they did have (or find) the power to negatively impact the project. by creating a policy requiring full identification and representation of stakeholders when membership of a steering group is being established destabilising conflict can be prevented. while this provides no guarantee of an emancipation, even where stakeholders may genuinely be powerless and affected but not involved, policy supporting their involvement in high-level decision-making may still lead to a better outcome. another contribution is the observation, through boundary critique, that conflict can become entrenched over time, if the marginalisation through differing worldviews is not addressed. in this case, the marginalisation occurred in a dramatic fashion ten years earlier, and distrust of ho by frontline staff had remained ever since. while projects may be temporary structures, in many organisations, business and social situations, the same stakeholder groups are required to interact over periods of time that may extend over decades. this could easily apply in other situations: operational staff and management; professional groups, like teachers and doctors, and the government agencies that fund them; welfare clients and welfare providers; telecommunications users and telecommunications companies. a mechanism for conflict resolution (for example, an issue register) will likely only address the symptoms of structural conflict, not its underlying cause. those responsible for project oversight should be encouraged to think in broader terms, including the potential for marginalisation and the role of the organisational machinery in perpetuating these boundary mismatches and the continued risk of conflict. unlike other types of organisational conflict, structural conflict can be harder to eradicate and potentially more damaging over the long term (jehn & mannix, 2001). finally, there is the question as to how csh can be used in the future to improve it project governance. the csh questions were described by their originator as a “framework for reflective practice based on practical philosophy and systems thinking.3” ulrich emphasises that the csh approach is not only intended as a critical method. “despite its emancipatory implications (the aspect for which it is best known), csh should not be misunderstood and used as an emancipatory systems approach only; for its principle of systematic boundary 3 http://wulrich.com/csh.html australasian journal of information systems johnstone & tate 2017, vol 21, research article improving it project governance 15 critique is vital for sound professional practice in general4”. the use of critical systems thinking techniques in it projects has been rare (e.g. cordoba & midgley, 2008). we believe the csh approach can make multiple contributions to it project governance. first, as a governance design tool. establishing high-level it project governance has a critical role in addressing potential conflicts in worldviews, before projects begin. the csh questions are clear, intuitive and usable by practitioners designing it project governance structures. this includes appointing fully representative steering committees, adopting appropriate policies that guide and constrain decision-making, and providing inclusive, consultative processes and methodologies that can help prevent, mitigate or reduce conflict. second, as a diagnostic tool, for remediating at-risk projects or as part of a post-implementation review, particularly where stakeholder conflict may have occurred. finally to understand projects in a wider organizational context, identify entrenched structural conflicts that may cause recurring problems in projects, and put governance mechanisms in place to address them. 7 further research one of the leading project management frameworks, the pmbok, did not introduce a dedicated section on stakeholders until its 5th edition, and the leading it service management framework, itil, does not include any specific knowledge area for stakeholder management. while stakeholders are better represented in togaf, the type of questions asked by midgley and ulrich are entirely absent in frameworks like pm-bok, prince2 and itil, which tend to focus instead on ‘controlling’ stakeholders. further research could investigate how a critical examination of project boundaries and stakeholder expertise, legitimacy and authority could be incorporated effectively into leading industry methodologies and best practice frameworks. 8 limitations a major limitation of this study is it was conducted post-hoc on previously collected case data. in some respects this lends legitimacy to the results, as we did not start with an a priori assumption that there were issues with stakeholder boundaries, inclusion, authority and legitimacy, and these findings rather emerged from the data and a post-hoc literature search. in other respects, this is a limitation, as we were not able to fully explore these issues through further questioning. for example, no frontline staff were directly interviewed, although insights into their perspective were provided by the regional manager (as an ex-frontline officer), and the qa consultant (as an independent observer). 9 conclusions in this paper, we argue that by: (i) incorporating a critical perspective in our conceptualisations of what it project governance involves; and (ii) using a critical systems heuristics (csh) lens for developing, implementing and reviewing governance arrangements, we can improve the effectiveness of it governance arrangements in practice and reduce conflicts between stakeholder groups. we exhort it professionals and managers to engage critically and constructively with the opportunities offered by different worldviews and marginalised constituencies; and to adopt the credo of “nothing about us without us”, rather than seeing differences in perspective from different stakeholder groups as something that it projects need to ‘manage’ and ‘control’. references almutairi, a., gardner, g., & mccarthy, a. (2014). practical guidance for the use of pattern matching technique in case study research: a case presentation. nursing and health sciences, 16(2), 239-244. 4 ibid australasian journal of information systems johnstone & tate 2017, vol 21, research article improving it project governance 16 bekker, m. (2014). project governance:" schools of thought. south african journal of economic and management sciences, 17(1), 22-32. biesenthal, c., & wilden, r. (2014). multi-level project governance: trends and opportunities. international journal of project management, 32, 1291-1308. campbell, d. (1966). pattern matching as an essential in distal knowing. in k. r. hammond (ed.), the psychology of egon brusnvick (pp. 81-106). new york: holt, rinchart, and winston. churchman, c. (1970). operations research as a profession. management science, 17(2), b37b53. cordoba, j., & midgley, g. (2008). beyond organisational agendas: using boundary critique to facilitate the inclusion of societal concerns in information systems planning. european journal of information systems, 17, 125-142. davis, k. (2014). different stakeholder groups and their perceptions of project success. international journal of project management, 32(2), 189-201. eisenhardt, k. (1989). building theories from case research. academy of management review, 14(4), 532-550. evers, c., & echo, h. (2006). on generalizing from single case studies: epistemological reflections. journal of philosophy in education, 40(4), 511-526. freeman, r. (1984). strategic management: a stakeholder approach. boston, usa: pitman. hammer, m. (1997). beyond reengineering: how the process-centered organization is changing our work and our lives harper collins. jackson, m. (2003). systems thinking: creative holism for managers. chichester, uk: wiley. jehn, k., & mannix, e. (2001). the dynamic nature of conflict: a longitudinal study of intragroup conflict and group performance. academy of management journal 44(2), 238-251. jiang, j., klein, g., & carr, c. (2002). measuring information systems service quality: servqual from the other side. mis quarterly, 26(2), 145. johnstone, d. (2010). it project governance and stakeholder conflict resolution. phd, victoria university, wellington. johnstone, d., huff, s., & hope, b. (2006). it projects: conflict, governance, and systems thinking. paper presented at the hawaii international conference on systems sciences (hicss), hawaii. klakegg, o., williams, t., magnussen, o., & glasspool, h. (2008). governance frameworks for public project development and estimation. [klakegg, ole jonny, et al. "governance frameworks for public project development and estimation." project management journal 39.s1 (2008).]. project management journal, 39(s1). kochan, t., & rubinstein, s. (2000). toward a stakeholder theory of the firm: the saturn partnership. organization science, 11(4), 367. kovács, g., & spens, k. (2005). abductive reasoning in logistics research. international journal of physical distribution & logistics management 35(2), 132-144. midgley, g. (1992). the sacred and profane in critical systems thinking. systemic practice and action research systemic practice and action research, 5, 5-16. midgley, g. (1997). dealing with coercion: critical systems heuristics and beyond. systems practice, 10(1), 37-57. australasian journal of information systems johnstone & tate 2017, vol 21, research article improving it project governance 17 midgley, g., & pinzon, l. (2011). boundary critique and its implications for conflict prevention. journal of the operational research society, 62, 1543-1554. midgley, g., & richardson, k. (2007). systems thinking for community involvement in policy analysis. emergence: complexity and organization, 9(1/2), 167-183. mitchell, r., agle, b., & wood, d. (1997). toward a theory of stakeholder identification and salience: defining the principle of who and what really counts. academy of management review, 22(4), 853-887. muller, r. (2009). project governance (fundamentals of project management): ashgate publishing group. myers, m., & klein, h. (2011). a set of principles for conducting critical research in information systems. mis quarterly, 35(1), 17-36. pinto, j. (2014). project management, governance, and the normalization of deviance. international journal of project management, 32(3), 376-387. pmi. (2013). guide to the project management body of knowledge (pm-bok) (5th ed.): project management institute. richet, j.-l., ngwenyama, o., & rowe, f. (2016). key stakeholders dissent and their influences in reframing of strategic intent in is-enabled organizational transformation. paper presented at the international conference in information systems (icis), dublin, ireland. roberts, t., cheney, p., & sweeney, p. (2002). assessing group interaction during hands-on lan projects. engineering management journal 14(2), 13-19. too, e., & weaver, p. (2014). the management of project management: a conceptual framework for project governance. international journal of project management, 32, 1382-1394. turner, j. (2009). the handbook of project-based management: leading strategic change in organizations: mcgraw-hill. ulrich, w. (1983). critical heuristics of social planning: a new approach to practical philosophy. chichester, uk: wiley. ulrich, w. (1987). critical heuristics of social systems design. european journal of operational research, 31(3), 276-283. ulrich, w. (2000). reflective practice in the civil society: the contribution of critically systemic thinking. reflcetive practice, 1(2), 247-268. ulrich, w. (2012). operational research and critical systems thinking an integrated perspective. part 1: or as applied systems thinking. journal of the operational research society, 63, 1228-1247. ulrich, w., & reynolds, m. (2010). critical systems heuristics. in m. reynolds & s. holwell (eds.), systems approaches to managing change: a practical guide (pp. 243-292). london: springer-verlag. van grembergen, w., & de haes, s. (2009). enterprise governance of information technology: achieving strategic alignment and value: springer yin, r. (2013). case study research: design and methods: sage. copyright: © 2017 johnstone & tate. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems johnstone & tate 2017, vol 21, research article improving it project governance 18 permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. paper title (paper title style) australasian journal of information systems li et al. 2024, vol 28, research article machine learning based decision-making 1 machine learning based decision-making: a sensemaking perspective jingqi (celeste) li the university of queensland, brisbane, queensland, australia morteza namvar the university of queensland, brisbane, queensland, australia m.namvar@business.uq.edu.au ghiyoung p. im university of louisville, louisville, kentucky, usa saeed akhlaghpour the university of queensland, brisbane, queensland, australia abstract the integration of machine learning (ml), functioning as the core of various artificial intelligence (ai)-enabled systems in organizations, comes with the assertion that ml models offer automated decisions or assist domain experts in refining their decision-making. the current research presents substantial evidence of ml’s positive impact on business and organizational performance. nonetheless, there is a limited understanding of how decisionmakers participate in the process of generating ml-driven insights and enhancing their comprehension of business environments through ml outcomes. to enhance this engagement and understanding, this study examines the interactive process between decision-makers and ml experts as they strive to comprehend an environment and gather business insights for decision-making. it builds upon weick’s sensemaking model by integrating ml’s pivotal role. by conducting interviews with 31 ml experts and ml end-users, we explore the dimensions of sensemaking in the context of ml utilization for decision-making. consequently, this study proposes a process model which advances the organizational ml research by operationalizing weick’s work into a structured ml-driven sensemaking model. this model charts a pragmatic pathway, outlining the interaction sequence between decision-makers and ml tools as they navigate through recognizing and utilizing ml, exploring opportunities, assessing ml model outcomes, and translating ml models into action, thereby advancing both the theoretical framework and its practical deployment in organizational contexts. keywords: machine learning (ml), decision-making, sensemaking. 1 introduction artificial intelligence (ai) is reshaping the foundation of business operations, fundamentally altering how companies operate and compete, as it facilitates smart services and automates tasks traditionally carried out by humans (cui et al., 2022). machine learning (ml) serves as the driving force for decision-making in a variety of ai systems (namvar et al., 2022). by training on vast datasets, learning intricate patterns, and generating predictive models, these ai systems autonomously analyze new data, recognize trends, and make accurate decisions based on their ‘learned knowledge’. as an illustration, ml is widely used for targeting prospective customers (simester et al., 2020), making inventory replenishment decisions (m. li & li, 2022), or predicting and selecting hedge fund returns (wu et al., 2021). decisions australasian journal of information systems li et al. 2024, vol 28, research article machine learning based decision-making 2 stemming from ai systems, where insights are derived from applying ml to large datasets, might introduce an entirely innovative approach to solving business problems (van den broek et al., 2021). consequently, both practitioners and researchers have emphasized the necessity of further research on the utilization of ml in organizational decision-making procedures (enholm et al., 2021). unlike conventional decision support tools, ml is distinguished by its unique characteristics (collins et al., 2021). ml systems are noted for their advanced learning capacity (van den broek et al., 2021), self-sufficiency (teodorescu et al., 2021), and a remarkable level of opacity (lebovitz et al., 2021), which sets them apart even from other intelligent technologies (baird & maruping, 2021). these attributes contribute not only to ml’s effectiveness but also present certain challenges. for instance, the opacity of ml systems can raise issues regarding transparency and accountability in decision-making (de laat, 2018) leading to potentially suboptimal decisions based upon various, unrecognized biases (manyika et al., 2017). according to the mckinsey global institute report (2017) within ai systems, ml is being adopted for tasks ranging from data analysis to customer service, and it projects potential job displacement and creation scenarios. the report highlighted the transformative potential of ml in industries such as manufacturing, healthcare, finance, and more. the way in which ai systems and their decision support engine, ml, enhance organizational decision-making differs significantly from traditional information technology (it) approaches. conventional decision support tools rely on a set of data to provide suggestions for organizational decisionmaking (chen et al., 2012). while ml systems do automate decision-making, they also iteratively optimize their approaches using additional data to enhance outcomes (collins et al., 2021). nonetheless, creating ml systems solely driven by a purely technical perspective can lead to ineffective use and unintended repercussions (namvar et al., 2022) and challenges which we have already listed above. these challenges underscore the imperative for a thorough evaluation of the implications of decisions made by ml and the potential ramifications these decisions might exert on businesses and end-users. in developing and introducing these systems, ml experts should engage in collaboration with ml end-users, encompassing both decision-makers and internal and external clients (van den broek et al., 2021). however, ml initiatives inherently entail challenges (ligon & sim, 2000). for example, organizational decision-making might anticipate outcomes that are impractical considering the available data, or ml experts might worry about the future uses of these systems in ways that are not aligned with their value systems. psychological research (see f.ex. kahneman & klein, 2009) suggests that proficient decision-making within intricate and disputatious social contexts hinges on more than mere technical knowledge, logic, or algorithms. the existing knowledge on how ml operates at a micro, detailed, granular level in decision-making processes is limited or incomplete, underscoring the need for more indepth research to achieve a comprehensive understanding of how ml functions in decisionmaking contexts (namvar et al., 2022). this research delves into the ways human reasoning processes can enhance individuals’ sensemaking by adeptly tailoring decisions to the specific context. given the idiosyncrasies of ml use for decision-making, our research examines how decision-makers can approach these challenging situations and therefore, this study asks, “how do managers use ml models to gain insight for their decision-making?” to address the aforementioned research question, we conducted 31 interviews with ml end users and experts and aimed to present a process model of ml-driven sensemaking, grounded australasian journal of information systems li et al. 2024, vol 28, research article machine learning based decision-making 3 in weick’s sensemaking model (weick, 1995; weick et al., 2005). sensemaking is defined as “the process through which individuals work to understand novel, unexpected, or confusing events” (maitlis & christianson, 2014, p. 57). it involves structuring the process to eliminate ambiguity in decision-making (weick, 1995) and assigning significance to past events (boland, 2008). weick’s (1995) model of sensemaking provides an apt theoretical foundation for this study as it elucidates the sequential activities undertaken by decision-makers when employing ml. the connection between sensemaking and decision-making is reciprocal (maitlis & christianson, 2014): sensemaking precedes decision-making by providing descriptions of the surroundings, while it follows decision-making when the latter generates ambiguity. this model is particularly suitable for studying ml-driven decision-making due to its focus on the interpretive cycles of understanding complex environments. it mirrors the iterative learning and application inherent in ml, making it ideal for exploring how managers interact with and derive actionable insights from ml outputs. on this background we: (1) identify the properties of sensemaking in the context of ml, and (2) develop a process model to show how sensemaking occurs in organizations using ml. the process model portrays the sensemaking process of decision-makers, from identifying business problems and opportunities to finding actionable solutions through interactions with ml experts. the process model captures theoretical constructs using a sequence of events that take place over time and focus on explaining how and why certain outcomes occur at the end of the process (newman & robey, 1992). this research constitutes a cutting-edge exploration by intertwining weick’s sensemaking model with ml practices in decision sciences. by doing so, it uncovers the nuanced processes by which decision-makers interact with ml algorithms, offering a novel model for ml-driven sensemaking. this model not only delineates the cognitive and practical steps involved in integrating ml into decision-making but also sheds light on the transformative impact of ml on organizational strategy and outcomes. the insights provided are set to challenge conventional narratives and inspire a rethinking of decision-making paradigms in the age of ai and ml. the ensuing sections first present the theoretical background of the study, followed by the details of the research method and its design, as well as a description of the data collection process. then, we present the thematic analysis of the collected data along with the proposed process model of the study. finally, the paper concludes and elaborates on the limitations of this study and opportunities for further research. 2 theoretical background ml is defined as the process of extracting patterns from data through learning algorithms, which then generate models for addressing real-world issues (jordan & mitchell, 2015). employing learning algorithms, like artificial neural networks, k-nearest neighbors, or a blend thereof, is integral to ml (collins et al., 2021). hyperparameters are calibrated to tailor these learning algorithms to the specific problem. the training of an ml model, guided by both the algorithm and the data, culminates in the creation of a model that accurately encapsulates the patterns inherent in the data. ml experts set up ml systems: they narrow their focus to a specific business problem, prepare and provide data, and train the ml algorithms. this initial setup depends on ml experts’ proficiency and influences their data selection, preparation, and optimization, based on their australasian journal of information systems li et al. 2024, vol 28, research article machine learning based decision-making 4 understanding of the application domain in question (khakabi et al., 2010), in our case business organizations and their environments. a deeper grasp possessed by those engaged in the ml system’s development empowers an efficient setup of ml models (namvar et al., 2022), amplifying the impact of ml in improved decisions (stetson et al., 2012). conversely, uninformed or biased engagement in the process of ml model development can inadvertently infuse flawed perspectives about the problem into the ml models (fügener et al., 2021). the input data for ml systems reflects a restricted viewpoint of a business problem, which might yield a less precise portrayal in comparison to how decision-makers or domain experts might present it (namvar & cybulski, 2014). in contrast, accounting for the perspective or concerns of decision-makers can offer a more comprehensive understanding of business issues within their genuine real-world settings (namvar et al., 2022) which is not limited to the narrow problem represented in data. to unpack the problem, next, we discuss the existing research on ml and decision-making. 2.1 ml and decision-making the majority of research on ml and decision-making seeks to build a theory by using a variance approach that focuses on macro-level constructs (trieu, 2017). most studies have addressed the value-creation aspect of big data. typically, studies investigate the sequence of analytics investments, analytics assets and capabilities, analytics impacts, and organizational performance (grover et al., 2018; trieu, 2017). although there has been significant development in ml research, studies have primarily concentrated on macro-level constructs, often overlooking the micro-level phenomena (namvar et al., 2022). a focus on the micro-level is vital; it uncovers the individual dynamics that are critical to the broader application of ml, particularly how decision-makers interact with and interpret ml outputs. at the micro-level, some research examined the use aspect of ml. deng and chi (2012) identified patterns of postadoptive use of big data in general with a focus on the problems that prevent its successful use. dinter (2013) examined the impacts of system quality and adequate information supply on effective use. li et al. (2013) and surbakti et al. (2019) investigated motivational and organizational elements for analytics use. other research has evaluated the decision-making aspect of ml. for example, ghasemaghaei and her colleagues (2019; 2017) studied decisionmaking quality with respect to data use, data quality, data diagnosticity, and analytics competency. still, other research recognized the essential role of sensemaking in the decisionmaking process (sharma et al., 2014) and identified the features of visual representation that facilitate sensemaking (baker et al., 2009). while these studies have touched on aspects of ml use, it is important to understand the decision-making processes in relation to specific ml attributes to grasp the complex cognitive mechanisms decision-makers employ when utilizing data-driven insights. by focusing on sensemaking processes, we offer such a nuanced view of how managers interpret and implement ml in decision-making. 2.2 sensemaking for decision-making sensemaking is a core organizational activity for managers looking to understand their business environment and business problems (weick, 1995), forming mental models that depict reality and causality, thereby laying the foundation for informed decision-making (seidel et al., 2018; woodside, 2005). in particular, sensemaking endeavors can tackle the complexities linked to the utilization of data, and ml, for decision-making (hasan & gould, 2001). weick’s (1995) sensemaking model provides a robust foundation for comprehending australasian journal of information systems li et al. 2024, vol 28, research article machine learning based decision-making 5 how individuals navigate these processes within business settings, highlighting it as an organizational process centred on human cognition to interpret data. the mechanism connecting sensemaking and decision-making is sequential: sensemaking constructs the mental models required for comprehending complex ml data, which then guides managers in evaluating and selecting appropriate actions during decision-making (maitlis & christianson, 2014). decision-making and sensemaking are discrete yet interconnected processes, where decision-making typically follows sensemaking (maitlis & christianson, 2014). decision-making entails evaluating a spectrum of feasible courses of action and opting for the optimal alternative. in contrast, sensemaking pertains to how managers comprehend and construe information (hekkala et al., 2018; namvar et al., 2018), while decision-making primarily centers on the prospective ramifications of decisions, actions, and outcomes (boland, 2008). thus, informed decision-making is predicated on effective sensemaking, where the latter shapes the context within which decisions are made. weick conceives of sensemaking as a principal organizing process, unfolding through a sequence of ecological change, enactment, selection, and retention (see figure 1). in this process, individuals organize to interpret unclear inputs and subsequently implement this understanding into the world, enhancing its structure (weick et al., 2005). enactment is triggered by ecological changes and includes sensemaking activities of noticing and bracketing to change flux into order by individuals. they bring into existence a variety of structures via reciprocal exchange with other actors in the organization. in selection, the number of possible interpretations for the information is reduced. individuals identify a plausible, tentative interpretation through mental engagement and articulation. a plausible interpretation is finalized in the process of retention. retention reflects on experience, formulates identities, and guides future interpretation. retention cycles back to selection and enactment. figure 1. weick’s model of sensemaking (jennings & greenwood, 2003 adapted from weick, 1979, p. 132) weick’s sensemaking model is suitable for understanding ml-driven decision-making because it aligns with the iterative and interpretive nature of ml processes. ml algorithms often resemble weick’s notion of ecological changes in the form of new data, requiring managers to enact sensemaking through noticing and bracketing, which is converting data into structured, understandable formats. selection happens as decision-makers reduce vast potential interpretations to plausible ones, resonating with ml’s pattern identification. finally, australasian journal of information systems li et al. 2024, vol 28, research article machine learning based decision-making 6 retention allows for the experience with ml to inform future interpretations and decisions, much like training and fine-tuning an ml system with new data strengthens its predictive capability as we will in detail elaborate on in the discussion section. this model in nature aptly addresses the nuances of ml by mapping the sensemaking stages to how ml insights are generated, interpreted, and applied in decision-making. 3 research method and data collection this study investigates the process of ml-driven decision-making during which managers use ml to gain insight into their problem-solving. managers are frequently involved in sensemaking when they have business data and want to enhance their understanding of the environment. this process is necessary and essential for making quality decisions. due to the absence of previous research on ml and sensemaking, this study conducted interviews with ml experts to assess the applicability of sensemaking in optimizing ml’s role in decisionmaking. interviews are selected for their effectiveness in revealing the complex practices and cognitive processes of the professionals in the ml field, which is crucial for our study’s focus on sensemaking. such interviews facilitate an in-depth understanding of individual perceptions and practices, aligning with the qualitative inquiry's goals in is research (conboy et al., 2012). thematic analysis (braun & clarke, 2006) was employed to scrutinize the collected data. the data analysis followed a six-step process: becoming familiar with the data, applying initial codes, recognizing themes, reviewing and refining themes, solidifying themes, and producing the final report. we interviewed participants representing diverse organizations, with a specific focus on elucidating their individual experiences in utilizing ml for decision-making. overall, we conducted interviews with 31 participants. table 1 presents brief profiles of study participants, along with their backgrounds and experience (dickson, 2015). the study participants include ml consultants, ml system developers, data analysts, and decision-makers who all are engaged with or rely on ml in their daily practices and decisions. primarily, they originate from sizable enterprises, each encompassing over 1000 employees. only five participants came from small companies with fewer than 100 employees. the participants are from different sectors, including finance, government, it, banking, healthcare, retail, and education. selecting participants from various industries and sizes enhances the research’s transferability. we utilized semi-structured interviews as a framework to facilitate open and flowing discussions, rather than adhering to a rigid sequence of prompts and responses. after an informal introduction with all involved parties, we presented complex business scenarios to the study participants, requiring substantial business data. participants shared how they used ml to address these scenarios. some talked about their successes, while others shared instances of failure. for instance, one of our study participants recounted a situation where she presented an ml model to stakeholders. within minutes, she realized she had lost the audience because her presentation style did not suit them, and they did not understand the outcomes. furthermore, the interviewees addressed a series of inquiries regarding their use of ml for comprehending business landscapes and operations. australasian journal of information systems li et al. 2024, vol 28, research article machine learning based decision-making 7 table 1. the background of the study participants note. to preserve confidentiality and protect the identities of the individuals who participated in the interviews, pseudonyms have been assigned to all interviewees. these pseudonyms have been used consistently throughout the document to ensure anonymity while maintaining the integrity of the research findings. 4 empirical results 4.1 thematic analysis the cyclical process of data collection helped us prepare the transcripts and their analysis. we dubbed concepts related to the use of ml in organizations with preliminary codes by reading sector pseudonym position retail emily senior delivery analyst khin it service manager finance ruofan data infrastructure, senior business analyst madison enterprise intelligence dale data analyst, enterprise intelligence emma total rewards manager, director information technology alfred data management & integration, director julian planning & financial governance, director daniel data analyst hill managing director jane senior consultant myla senior consultant nathan marketing & sales, director robert channel technical manager ross data visualization company, founder roy software sales manager scott development, general manager shane technology strategist shaun executive director healthcare andrew diagnostic imaging, manager arnaldo business intelligence developer glenn data analysis, manager ian diagnostic imaging, operations manager kim revenue manager government clark operations, director rachel analytics & research, director heetai it architecture & strategy, manager edmond analysis & financial reporting, national manager education jordan senior data analyst matt business intelligence, manager jennifer customer services, manager australasian journal of information systems li et al. 2024, vol 28, research article machine learning based decision-making 8 the interview transcripts. these codes include the concepts associated with using ml in organizations for better decision-making and understanding of business environments. in total, we uncovered 56 preliminary codes. codes with akin connotations within pertinent contexts were combined to formulate final codes. subsequently, 24 initial concepts (common codes) were formulated, encompassing diverse facets of ml that held relevance in its application for sensemaking within organizational contexts. figure 2 presents these concepts, along with the data structure of our analysis. for reduction and elimination, and with reflection on each transcript, we reviewed each transcript, excising statements that exhibited vagueness, repetition, or overlap. following this, we collated statements bearing the same code to effectively showcase study participants' perspectives on distinct matters (see table 2 for the identified first order concepts and representative quotes). next, we organized the first-order concepts into eight coherent themes, following the methodology outlined by gioia et al. (2013). this interpretive process involved iterative discussions among our research team to ensure a shared understanding and agreement on the thematic groupings. for instance, as depicted in figure 2 the “integration” theme emerges through an analysis of three key first-order concepts: 1) data source, 2) business rules, and 3) business processes. figure 2 contains the eight themes which we categorized as second-order themes and their constituting first-order concepts. these themes and their firstorder concepts are explained in the following. in the remainder of the paper, ids refer to the second column of table 2 and are used to relate to the concepts. integration is the consolidation of data sources found in different business processes and external databases while taking various business rules into account. since the value of data explodes when it can be linked and fused with other data, integration is critical to recognizing the promise of ml. the back-end of ml systems is, therefore, a data integration pipeline that extracts data from distributed and usually heterogeneous sources and processes [e.g., id3]. integration demands a significant investment of labor, often constituting a substantial portion of the endeavor to modify business processes and ensure data quality [id1]. it requires working with several business processes and engaging different ml stakeholders [id2]. recognizing is understanding what ml can do for the business. it remediates an existing fallacy or limitation [id4], or pursues an opportunity to enhance business processes or attract customers. our study participant hill, highlighted that we need to consider both data opportunities and business problems when using ml. one of the major impediments to using ml in organizations is unclear requests from decision-makers at the start [e.g., id5]. decisionmakers need to engage with ml experts to articulate requests for the extraction of business drivers [id6]. interactivity regards providing decision-makers with the opportunity to engage and interact with the preliminary data visualization. it can be in the form of an interactive data visualization with an option to feed various inputs and adjust the parameters. decisionmakers prefer to investigate the stories in data [id8]. stories assist decision-makers in exploring the available data, applying their business understanding, and making their interpretations. in complex business and data scenarios, analysts need to visualize the integrated data to engage decision-makers in finding pieces of data that require further investigation. our study participants recognized intuitive and user-friendly [id7] self-service interfaces as a significant effort in a substantial domain as they help decision-makers explore additional insights and know the drivers of business functions. australasian journal of information systems li et al. 2024, vol 28, research article machine learning based decision-making 9 figure 2. the developed concepts and themes australasian journal of information systems li et al. 2024, vol 28, research article machine learning based decision-making 10 concept id representative quote data sources 1 the complexity comes when you have more than one system, just trying to integrate them. you’ve got enough data quality issues with one system, let alone two. business rules 2 there is always some communication with the end-users, and then the next step is to understand because that conversation creates your business rules [ruofan]. business processes 3 we recently did some benchmarking for financial services companies and on the delivery of the report – so most of the time we were working with procurement and finance; at the end of the report, we were working with marketing [shaun]. business problems 4 quite often, people define a problem very clearly, but what they expect from the analytics is completely different from what they requested [rachel]. opportunity 5 there was a company who engaged oracle or sap but did not enable the spend analytics module or reporting modules and did not train anybody for that purpose. so, it is the scope, and then it is articulating what they believe the deliverables will be from that [shaun]. engagement 6 the end-users are not clear at the start, which i would say most organizations are not crystal clear when they look for a solution or a service. engagement clarifies that and allows us to make sure that by the time we get to the end, it is going to be a yes [shaun]. ease of use 7 an awful lot of the executives want to carry an ipad and take the information into board meetings and use it live. a lot of the work is going into how to make the reports user-friendly to operate [robert]. storytelling 8 we try to tell a story. we analyze the data and decide what the key things the data is telling. then we tell them: “these are the things that are most relevant, so it depends on you now” [madison]. feedback 9 data analysts start building and delivering, and then they receive feedback. they iterate and go through and eventually break the work up into bits and deliver as much as they can [matt]. analytical capability 10 our end-users were saying the data was wrong, and i said: “no, the data is not wrong; you are not asking it the right question.” so, once i showed them how to ask the right question, how to tick the right boxes, they said, “yeah exactly it is” [ian]. data awareness 11 data visualization helps you find questions that you didn’t know you wanted to ask. it is about a way of exploring data and finding what insights or points that decisionmakers then want to feed into a more regular business intelligence system [ross]. details 12 we get information from the data warehouse, put it in a paper, publish it as a pdf. if it is a starred item, decision-makers review it. otherwise, it wouldn't even get read [matt]. personalizati on 13 you need to be able to have the tools and be able to manipulate and explore the possibilities yourself, in some way, visually. on the other side of the coin is the ability to engage and communicate with an audience and to convey your message to that particular audience [ross]. cues (insights) 14 while decision-makers recognize the business environment is unsettled, and there are always changes and external factors, it is a cliché, but they see the big picture in that. they often get very pointy and succinct information, above or below the line. it is not grayed [shaun]. attitude 15 few say we are looking for a tool that will enable us to do decision-making; to be honest, i rarely hear that language, which is what they are looking for [shaun]. acceptance 16 we have decision-makers with some strong opinions and beliefs about what is going on. if you are going to be developing a model which is contrary to these positions, or long-held standing positions, you have to present it very plausible and easy-tounderstand [rachel]. validation 17 you would not make a recommendation without running it past the decision-makers. even if they have the standard implementation, they are going to do their things to it, and they have got their quirks about why they do things. so, they will always be a point of validation [madison]. alerts 18 our event management system gives us alerts, notifications, and monitoring of our kpis. it could be our sales forecast numbers, our budget’s due, or it could be from predictive analytics, which provides an alert that could happen [robert]. australasian journal of information systems li et al. 2024, vol 28, research article machine learning based decision-making 11 concept id representative quote options 19 generally, decision-makers will want from a data analyst or consultant to demonstrate the options and then a recommendation [madison]. plausibility 20 not all reports have to be 100 percent confident; you can still have a report which you can say there is only 30 percent confidence, and that is better than nothing at all [heetai]. access 21 the idea of the executive having a mini ipad on the golf course, and getting a report sent to them, analyzing them and making some decision on the report, is very real today [robert]. time 22 the business should wait for various pieces of data that some data supplier can provide. and there is processing time required for technology. also, a long time is needed for data analysts to bring this data in and analyze it [ruofan]. justifying 23 there is a culture of discovery and justifying your position based on evidence, which is probably a bit harder than just teaching someone how to generate an invoice differently [nathan]. ad hoc requests 24 over time, once decision-makers are comfortable with understanding what they want from analytics, they apply the existing functions. then, again, they’ll come with the same problem but for a different data set [daniel]. table 2. the first-order concepts exploring is about decision-makers investigating the preliminary data visualization and drilling into interactive visuals. decision-makers are often unaware of the available data and opportunities; working with interactive data visualization helps them leverage their understanding of available data and explore data opportunities [id11]. this exploration helps decision-makers provide constructive feedback to ml experts about how much the preliminary data visualization [id9] helps them pursue data opportunities or re-formulate business problems. the analytical skills and capabilities of decision-makers determine the effectiveness of their data exploration [e.g., id10]. ml model presentation by ml experts is about providing decision-makers with the results of their analysis. from an ml expert’s point of view, information presentation stands between interactive data visualization, discussed in interactivity themes and actionable reports. while interactive data visualization aims to assist in exploring opportunities, information presentation is used to improve the validation of the ml models by decision-makers and increase their acceptance. our study participants cautioned about saturating decision-makers in busy business environment settings with a lot of information from ml [id12]. they underscored the necessity for concise and focused ml models that offer optimal choices to decision-makers at the board level, customized to suit the intended audience [id13]. the study participants expanded on the significance of simplicity and trends in ml outcomes, as decision-makers exhibit a preference for identifying anomalies above or below trend lines in charts [id14]. assessing is about investigating the validity of the presented ml models so that trust in using them can be developed. decision-makers are sceptical about ml outcomes when the outcomes are not validated [id17]. this scepticism creates even more resistance toward reports with recommended solutions for decision-makers [id15]. for decisions at the operational level, ml can usually provide some straightforward numbers about business processes. however, at the strategic level, more effort is required to justify ml outcomes as decision-makers have strong opinions, beliefs and long-held standing positions [id16]. australasian journal of information systems li et al. 2024, vol 28, research article machine learning based decision-making 12 actionable ml models provide a validated analysis in the form of options and recommendations from which the final business decision can be made, and actions taken. although intensive work on ml can be ongoing, our study participants’ repeated complaints about ml concerned its effective and sustainable use. this theme reflects a demand for actionable ml and involves several features that enable the significant use of ml in an organization. this theme identifies four features that support the use of analytical reports in action, namely, alerts, options, plausibility, and access. timely alerts are generated via automated reports and notify decision-makers of business events [id18]. a variety of platforms are used to generate timely automated reports [id21]. reports are more effective when they are available on portable devices when decision-makers need them. decisionmakers further embrace and engage with ml models that present plausible outcomes [id20] and imprecise alternatives [id19]. for instance, the statistical examination of predictive models can aid decision-makers in gauging the model’s efficacy with a certain degree of certainty. such ml models afford decision-makers a chance to exercise their discernment. turning into action is about using validated reports in practice and applying them to business decisions. our study participants comprehended that decision-making is intertwined with risks and unstructured scenarios and that the ambiguity of outcomes significantly impacts fact-based decision-making. they highlighted that in certain instances, intuitive decisionmakers seek validation in ml models to justify preexisting notions. [id23]. they discussed the lengthy process of generating ml models [id22] and cautioned against using aged reports. shaun also added that the decision-making process is not nimble, contributing to report aging. 4.2 the process of ml-driven sensemaking our analysis has identified eight themes related to using ml in organizations, but so far considered each of them only individually. to further advance the understanding of the use of ml in sensemaking, we now move beyond the individual themes to examine how individual actors and the themes related across the process of sensemaking. figure 3 shows the cyclic, infused by feedback loops, process of ml-driven sensemaking as a sequence of encounters and episodes that over time that leads to turning the outcome of the process into action and to applying it in business decision-making. in using the concepts of encounter and episode to underline the processual nature of the sensemaking phenomenon we follow newman and robey (1992, p. 253) who explain that “an episode refers to a set of events that stand apart from others, thus signifying the end of one sequence of activities and the beginning of another … encounters mark beginnings and ends of episodes.” we recognize the four themes recognizing, exploring, assessing, and turning into action as the episodes and the four themes integration, interactivity, ml model presentation, and actionable ml models as encounters and thus operationalize and further extend weick’s model of sensemaking to the context of ml. integration presents the first encounter in the proposed process model. ml experts typically start an ml project by integrating various data sources from different business processes. recognizing them is the first episode triggered by integration and is about understanding what ml can do for a business (woerner & wixom, 2015). the engagement between decisionmakers and ml experts is very important in this episode and ml experts assist decisionmakers so that they can understand data sources and opportunities [id5]. failure in this engagement limits the use of ml in everyday operational decisions (günther et al., 2017; torres et al., 2018). australasian journal of information systems li et al. 2024, vol 28, research article machine learning based decision-making 13 figure 3. process model of ml-driven sensemaking exploring is the second episode in the process model. an interactivity encounter facilitates the transition from recognizing to exploring. ml experts develop interactive interfaces to help decision-makers explore available data. visualized data is available via maps, charts, and diagrams and empowers decision-makers to perceive important patterns in data and to identify areas for further examination. the manner in which decision-makers perceive and engage with visualizations can markedly shape their exploration and comprehension of the accessible data. the next episode is assessing the presented information. decision-makers are often sceptical about ml outcomes, especially when using them for strategic decision-making. they feel confident when they can effectively integrate validated insights into their decision-making (shollo & galliers, 2016), and make use of the potential of ml (bhimani, 2015). an ml model presentation encounter facilitates the transition from exploring to assessing. the nature of insights unearthed from the accessible data hinges on the diversity and level of detail within the various data sources (krishnamoorthi & mathew, 2018; yoo, 2015). presenting the insights is, therefore, a challenge for ml experts. our thematic analysis identified three concepts related to ml model presentation (i.e., cues, details, and personalization). after ml experts and decision-makers reach a shared understanding and recognize data opportunities or business problems, decision-makers start exploring the preliminary ml models. ml experts also analyze data and discover insights and cues related to the identified opportunities or problems. ml experts personalize and present the developed ml models based on the skills, capabilities, and preferences of the decision-makers [id13]. they also need to avoid saturating decision-makers with details and highlight the star cues [id14]. a deeper understanding of data and stronger consent is then achieved through assessing the presented information. the final episode is turning into action. the application of ml outcomes occurs after decisionmakers have carefully explored and assessed the presented information and the insights. decision-makers supplement the ml outcomes with their experiences, common sense, and contextual knowledge (shollo & galliers, 2016), to solve problems for which the conditions are often unknown (shollo & galliers, 2016). an actionable ml model encounter enables the transition from assessing to turning into action. reports as the outcomes of ml summarize the discovered insights. our data shows that decision-makers prefer ml models that show levels of certainty with possible options [id19], especially in situations where the reports influence australasian journal of information systems li et al. 2024, vol 28, research article machine learning based decision-making 14 strategic decision-making. the reports, therefore, should beneficially include recommendations and the plausibility associated with each option. the process model also comprises three feedback loops: (1) from exploring to recognizing, 2) from turning into action to exploring, and (3) from turning into action to assessing. first, during exploring, decision-makers reduce the equivocality of the data by iteratively cycling through interactive data visualizations. they improve their understanding of the available data by simplifying business-data rules and applying their experience. along with ml experts, decision-makers then continuously refine business problems and data opportunities. decision-makers offer feedback about their new understanding and how recognizing can be repeated and improved. next, turning into action loops back to future rounds of exploring and assessing. depending on the trust decision-makers gain from their experience of turning ml models into action, the feedback loops can be positively or negatively reinforcing. when decision-makers have developed trust through validation of reports [id17], the volume and elaboration of ml outcomes in their business decision will increase. the opposite is true when they have not built trust, and they continue to seek new data sets or methods to deal with business problems. a key point of the proposed process model is that turning into action is always based on the earlier exploring [id11] and assessing [id17]. it also has a lagged effect on the next episode of recognizing as the engagement of end-users provides clarity for ml experts, resulting in their understanding of the business problem [id6]. our process model provides a decision-making model based on and in the context of applying ml systems. it demonstrates the interplay between decision-makers and ml experts. first, the model recognizes the role of decision-makers in four episodes, i.e., recognizing, exploring, assessing and turning into action. in particular, it introduces the crucial role of decisionmakers in recognizing what ml can do for a business and elaborates on the capabilities of decision-makers in identifying business problems and opportunities that can be addressed by ml. decision-makers’ preconceptions and business understanding assist in creating some variation among the business patterns discovered by ml experts from ml technologies. decision-makers structure the business-data environment and enact ml environments by participating in the exploring episode. next, the process model shows how ml experts contribute to the process of ml-driven sensemaking. they initiate this process through the integration encounter. also, they play a crucial role in triggering the transition between episodes through three encounters (i.e. interactivity, ml model presentation, and actionable ml models). in each of these encounters, a different ml outcome is generated based on ml experts’ understanding of the data-business environment and the input provided by the decision-makers during the progression of the sensemaking process. 5 discussion in this study, we applied weick's sensemaking model to explore how ml is used in organizational decision-making. our analysis of interviews with ml experts and decisionmakers led to a process model that illustrates the interplay of various elements in ml-driven sensemaking. this model includes integration of ml, recognition of patterns, interactivity in data exploration, presentation of ml models, assessment of ml outputs, and the transformation of insights into actionable decisions. these findings offer theoretical advancements in understanding the cognitive aspects of ml in organizations. in the australasian journal of information systems li et al. 2024, vol 28, research article machine learning based decision-making 15 subsequent sections we discuss the theoretical and practical implications, providing insights for scholars and practitioners on effectively implementing ml in decision-making processes. 5.1 theoretical implications the results of this study have several theoretical implications for ai-driven decision science. first, the study extends ml research by explaining how decision-makers actually use ml models for their decision-making. past research has focused on the economic and social values of ml at a macro level. a few studies have examined the use and decision-making aspects of ml (namvar et al., 2022). as such, the understanding of ml use at the micro-level is rather limited. this study contributes to such understanding by elucidating the actual process of ml use and its roles in decision-making. drawing on the sensemaking theory by weick (1995), we identified eight themes via thematic analyses and developed a process model of ml-driven sensemaking. our process model of data-driven sensemaking (figure 3) enriches weick’s process model of organized sensemaking by identifying the crucial role of ml and ml experts during the process of sensemaking. to further elucidate the alignment between our identified themes and weick’s sensemaking model, we can draw direct parallels between the stages of ecological change, enactment, selection, and retention within weick’s model, and the themes identified through our thematic analysis. each theme represents an element of the ml sensemaking process, reflecting the corresponding stages in weick’s model, thus providing a theoretical grounding for our findings. as figure 1 shows, in weick’s model sensemaking is triggered by ecological change in the organizational environment. similar to this, our process model is triggered by recognizing what data and ml can do for decision-making. decision-makers and ml experts evaluate the significance of data in developing a strategy toward either mitigating the risk or pursuing opportunities associated with these changes. in the enactment phase, decision-makers enact a more ordered environment to understand the meaning of the ecological changes, and to examine more, they isolate and bracket some portions of the ecological changes (weick, 2005). our model suggests that enactment in the ml context can be effectively accomplished by decision-makers exploring the available data. decision-makers attend to a certain portion of data provided in an interactive format by ml experts. they selectively filter data that represent the bracketed changes, and look for relationships. selection is made by obtaining a tentative, plausible interpretation of the information from multiple possible meanings (weick, 1995). our model indicates that selection can be accomplished by assessing the presented ml models. in the assessing episode, decision-makers validate the presented ml models by ml experts and select reasonable interpretations among various possible relationship structures. similar to retention in weick’s model, our model recognizes retention in the form of turning into action. decision-makers retain validated ml models and apply them to business practices. validated ml models guide their current actions and future interpretations. in our process model the resulting ml models serve as a map of relationships between events and actions that guide decisions. similar to the loops in weick’s model, our model has the loopback from turning into action to extra rounds of exploring and assessing. after decision-makers select some provided reports for use in action, they may recognize the need for additional data and methods for further exploration and better assessment of their decision-making situation. last, our study demonstrates that ml may not only support decision-making but also sensemaking. our process model identifies a sequence of episodes and encounters of australasian journal of information systems li et al. 2024, vol 28, research article machine learning based decision-making 16 sensemaking as interchanges between decision-makers and ml experts that take place over time before decision-makers select actionable solutions. this last finding suggests that decision-makers should recognize both, the beneficial role of such interchanges with ml experts and the significant role of ml tools, and consciously and actively engage with both to reach any actionable and favorable solutions. 5.2 practical implications our study also provides several other practical implications. first, beyond for decision-making proper, decision-makers may also advantageously use ml technologies to improve their general understanding of the business environment they are acting in. to this end, decisionmakers should willfully engage in the sensemaking episodes, recognizing what ml can do, exploring preliminary ml models, assessing ml outcomes, and turning ml outcomes into action, and collaborate where necessary with ml experts in the encounters which mark the beginnings and ends of the sensemaking episodes to educate ml experts about the business environment to improve the latter’s understanding of the available business data. in particular, decision-makers should embrace the provided interactivity and self-service features of ml tools and consider them an opportunity to enhance their business understanding among other through customization of the reports that are created and made available. second, ml experts should collaborate with decision-makers and engage with them during ml model development. they should not only think of ml as a solution for an existing business problem but also consider business opportunities that can be identified from the available data. ml experts should offer preliminary ml models to decision-makers in an easyto-use format and engage decision-makers in their exploration so that the latter can discover further opportunities and, in this context, new data needs. ml results should be presented in a way that facilitates easy assessment and subsequently improves decision-makers' trust in using ml outcomes. we also suggest practice-relevant guidance for integrating ml-driven sensemaking into organizational processes. decision-makers makers may incorporate ml insights into strategic planning sessions, ensuring that sensemaking outcomes are translated into actionable strategies. additionally, ml experts and decision-makers should jointly conduct periodic review meetings to assess the ongoing effectiveness of ml applications in decision-making and adjust approaches as necessary. decision-makers could conduct regular knowledgesharing workshops with ml experts, create comprehensive documentation on business processes, and involve them in strategic meetings for a practical understanding of business challenges and objectives. these strategies would ensure that ml experts gain an in-depth understanding of the business environment, enhancing the development of relevant and impactful ml solutions. to put these recommendations into practice, for instance, in healthcare, our findings can guide clinicians in their sensemaking processes when they utilize ml systems for decision-making, particularly in interpreting complex patient data to improve treatment outcomes and in the finance industry, the insights could assist risk officers in understanding the nuances of ml models for credit risk assessment, enhancing their ability to make sense of financial predictions and data. australasian journal of information systems li et al. 2024, vol 28, research article machine learning based decision-making 17 6 conclusion this study aimed to understand how managers use ml models to gain insights for their decision-making. to accomplish this goal, we identified the properties of sensemaking and developed a process model to explain how sensemaking takes place in organizations that use ml models for decision-making. our research is a response to calls for a micro-level understanding of the sensemaking phenomenon associated with ml. to achieve this objective, we conducted interviews with 31 study participants, to assess both the strengths and limitations of ml within the context of the sensemaking process these were reorganized through the derivation of a ml-driven sensemaking process model which includes the decision-maker episodes of recognizing, exploring, assessing, and turning into action and the mlf expert encounters of integration, interactivity, ml model presentation and actionable ml models. our study provides valuable insights but also acknowledges certain limitations, which pave the way for prospective research. similar to any qualitative study, this research design does not ascertain additional interdependent factors that impact ml productivity, even though several of these were explored during the conducted interviews. notably, we did not investigate decision-makers’ degree of domain knowledge and experience, nor how this affects their interactions with ml. additionally, ml experts might introduce challenges or biases during the sensemaking process. future research could undertake a detailed exploration of these elements to deepen our understanding of ml in decision-making. for instance, conducting case studies to examine how decision-makers’ domain knowledge improves their sensemaking within specific organizational settings. investigations might also assess the conditions conducive to generating insights via ml and the impact of outsourcing ml functions, which introduces additional challenges such as the need for frequent knowledge updates. future research can also open a discussion on the variability of ai experts’ involvement in business contexts and how this variability affects the effectiveness of ai model optimization. while our proposed model draws from a diverse array of industries and suggests broad applicability, we recognize that unique characteristics of different industries and organizational contexts may affect the model's generalizability and transferability. thus, we invite further studies to validate and test our model across various settings. moreover, while our study is anchored in weick’s sensemaking model, an integration of perspectives from other disciplines could yield additional theoretical richness. future studies might employ agency theory, stakeholder theory, or concepts from cognitive psychology and humancomputer interaction to provide a more comprehensive understanding of the nuances involved in ml and decision-making. finally, in our study, we focused on interviews for data collection. while this method provided rich insights, we acknowledge the limitation of not incorporating other methods such as document analysis or quantitative approaches. these methods could offer different perspectives and enhance the robustness of the findings. future research could benefit from employing a mixed-methods approach, which would allow for a more comprehensive analysis and strengthen the overall research design. australasian journal of information systems li et al. 2024, vol 28, research article machine learning based decision-making 18 acknowledgments we acknowledge professor karlheinz kautz's significant role in the development of our manuscript. his constructive feedback and support as editor-in-chief have been instrumental in the evolution and success of our research. we also extend our thanks to the review team for their valuable input. references baird, a., & maruping, l. m. (2021). the next generation of research on is use: a theoretical framework of delegation to and from agentic is artifacts. mis quarterly, 45(1), 315–341. https://doi.org/10.25300/misq/2021/15882 baker, j., jones, d., & burkman, j. (2009). using visual representations of data to enhance sensemaking in data exploration tasks. journal of the association for information systems, 10, 533–559. https://doi.org/10.17705/1jais.00204 bhimani, a. (2015). exploring big data’s strategic consequences. journal of information technology, 30(1), 66–69. https://doi.org/10.1057/jit.2014.29 boland, r. (2008). decision making and sensemaking. in handbook on decision support systems (pp. 55–63). berlin, heidelberg, germany: springer publishing. braun, v., & clarke, v. (2006). using thematic analysis in psychology. qualitative research in psychology, 3(2), 77–101. https://doi.org/10.1191/1478088706qp063oa chen, h., chiang, r. h. l., & storey, v. c. (2012). business intelligence and analytics: from big data to big impact. mis quarterly, 36(4), 1165–1188. https://doi.org/10.2307/41703503 collins, c., dennehy, d., conboy, k., & mikalef, p. (2021). artificial intelligence in information systems research: a systematic literature review and research agenda. international journal of information management, 60, article 102383. https://doi.org/10.1016/j.ijinfomgt.2021.102383 conboy, k., fitzgerald, g., & mathiassen, l. (2012). qualitative methods research in information systems: motivations, themes, and contributions. european journal of information systems, 21(2), 113–118. https://doi.org/10.1057/ejis.2011.57 cui, r., li, m., & zhang, s. (2022). ai and procurement. manufacturing & service operations management, 24(2), 691–706. https://doi.org/10.1287/msom.2021.0989 de laat, p. b. (2018). algorithmic decision-making based on machine learning from big data: can transparency restore accountability? philosophy & technology, 31(4), 525–541. https://doi.org/10.1007/s13347-017-0293-z deng, x., & chi, l. (2012). understanding postadoptive behaviors in information systems use: a longitudinal analysis of system use problems in the business intelligence context. journal of management information systems, 29(3), 291–326. https://doi.org/10.2753/mis0742-1222290309 australasian journal of information systems li et al. 2024, vol 28, research article machine learning based decision-making 19 dickson, s. a. (2015). stuck in the middle: generation xers lived experiences in the workforce while navigating between baby boomers and millennials, and perceived impact on their job satisfaction. [doctoral dissertation, colorado state university]. mountain scholar. https://api.mountainscholar.org/server/api/core/bitstreams/27b6c9fd-3f9e-4bd9-8a14bb402d7d6d0f/content dinter, b. (2013). success factors for information logistics strategy—an empirical investigation. decision support systems, 54(3), 1207–1218. https://doi.org/10.1016/j.dss.2012.09.001 enholm, i. m., papagiannidis, e., mikalef, p., & krogstie, j. (2021). artificial intelligence and business value: a literature review. information systems frontiers. 24(5), 1709–1734. https://doi.org/10.1007/s10796-021-10186-w fügener, a., grahl, j., gupta, a., & ketter, w. (2021). will humans-in-the-loop become borgs? merits and pitfalls of working with ai. mis quarterly, 45(3), 1527–1556. https://doi.org/10.25300/misq/2021/16553 ghasemaghaei, m. (2019). does data analytics use improve firm decision making quality? the role of knowledge sharing and data analytics competency. decision support systems. 120, 14–24. https://doi.org/10.1016/j.dss.2019.03.004 ghasemaghaei, m., ebrahimi, s., & hassanein, k. (2017). data analytics competency for improving firm decision making performance. the journal of strategic information systems, 27(1), 101–113. https://doi.org/10.1016/j.jsis.2017.10.001 gioia, d. a., corley, k. g., & hamilton, a. l. (2013). seeking qualitative rigor in inductive research: notes on the gioia methodology. organizational research methods, 16(1), 15–31. https://doi.org/10.1177/1094428112452151 grover, v., chiang, r. h. l., liang, t.-p., & zhang, d. (2018). creating strategic business value from big data analytics: a research framework. journal of management information systems, 35(2), 388–423. https://doi.org/10.1080/07421222.2018.1451951 günther, w. a., mehrizi, m. h. r., huysman, m., & feldberg, f. (2017). debating big data: a literature review on realizing value from big data. the journal of strategic information systems, 26(3), 191–209. https://doi.org/10.1016/j.jsis.2017.07.003 hasan, h., & gould, e. (2001). support for the sense-making activity of managers. decision support systems, 31(1), 71–86. https://doi.org/10.1016/s0167-9236(00)00120-2 hekkala, r., stein, m.-k., & rossi, m. (2018). metaphors in managerial and employee sensemaking in an information systems project. information systems journal, 28(1), 142– 174. https://doi.org/10.1111/isj.12133 jennings, p. d., & greenwood, r. (2003). constructing the iron cage: institutional theory and enactment. in r. westwood & s. clegg (eds.), debating organization: point-counterpoint in organization studies (pp.195–207), hoboken, nj, usa: wiley-blackwell. jordan, m. i., & mitchell, t. m. (2015). machine learning: trends, perspectives, and prospects. science, 349(6245), 255–260. https://doi.org/10.1126/science.aaa8415 kahneman, d., & klein, g. (2009). conditions for intuitive expertise: a failure to disagree. american psychologist, 64(6), 515–526. https://doi.org/10.1037/a0016755 australasian journal of information systems li et al. 2024, vol 28, research article machine learning based decision-making 20 khakabi, s., gholamian, m. r., & namvar, m. (2010, january 27–29). data mining applications in customer churn management. 2010 international conference on intelligent systems, modelling and simulation. liverpool, united kingdom. https://doi.org/10.1109/isms.2010.49 krishnamoorthi, s., & mathew, s. k. (2018). business analytics and business value: a comparative case study. information & management, 55(5), 643–666. https://doi.org/10.1016/j.im.2018.01.005 li, m., & li, t. (2022). ai automation and retailer regret in supply chains. production and operations management, 31(1), 83–97. https://doi.org/10.1111/poms.13498 li, x., po-an hsieh, j. j., & rai, a. (2013). motivational differences across post-acceptance information system usage behaviors: an investigation in the business intelligence systems context. information systems research, 24(3), 659–682. https://doi.org/10.1287/isre.1120.0456 ligon, j., & sim, j.-s. (2000). which companies are more profitable when using data mining techniques? americas conference on information systems, long beach, california, august 10–13, 2000. http://aisel.aisnet.org/amcis2000/381?utm_source=aisel.aisnet.org%2famcis2000%2f381 &utm_medium=pdf&utm_campaign=pdfcoverpages. maitlis, s., & christianson, m. (2014). sensemaking in organizations: taking stock and moving forward. the academy of management annals. the academy of management annals, 8(1), 57–125. https://doi.org/10.1080/19416520.2014.873177 manyika, j., lund, s., chui, m., bughin, j., woetzel, j., batra, p., ko, r., & sanghvi, s. (2017). jobs lost, jobs gained: workforce transitions in a time of automation. https://epic.org/wpcontent/uploads/foia/epic-v-ai-commission/epic-19-09-11-nscai-foia-20200529-5thproduction-pt4-outside-reports-resources.pdf [accessed march 13, 2022]. namvar, m., & cybulski, j. (2014). bi-based organizations: a sensemaking perspective. international conference on information systems, auckland, new zealand, december 14–17. https://aisel.aisnet.org/icis2014/proceedings/isstrategy/39. namvar, m., cybulski, j., phang, c., wee, c.-y., & tan, k. y. (2018). simplifying sensemaking: concept, process, strengths, shortcomings, and ways forward for information systems in contemporary business environments. australasian journal of information systems, 22. https://doi.org/10.3127/ajis.v22i0.1654 namvar, m., intezari, a., akhlaghpour, s., & brienza, j. p. (2022). beyond effective use: integrating wise reasoning in machine learning development. international journal of information management, 69, article 102566. https://doi.org/10.1016/j.ijinfomgt.2022.102566 newman, m., & robey, d. (1992). a social process model of user-analyst relationships. mis quarterly, 16(2), 249–266. https://doi.org/10.2307/249578 australasian journal of information systems li et al. 2024, vol 28, research article machine learning based decision-making 21 seidel, s., kruse, l. c., székely, n., gau, m., & stieger, d. (2018). design principles for sensemaking support systems in environmental sustainability transformations. european journal of information systems, 27(2), 221–247. https://doi.org/10.1057/s41303-017-0039-0 sharma, r., mithas, s., & kankanhalli, a. (2014). transforming decision-making processes: a research agenda for understanding the impact of business analytics on organisations. european journal of information systems, 23(4) 433–441. https://doi.org/10.1057/ejis.2014.17 shollo, a., & galliers, r. d. (2016). towards an understanding of the role of business intelligence systems in organisational knowing. information systems journal, 26(4), 339– 367. https://doi.org/10.1111/isj.12071 simester, d., timoshenko, a., & zoumpoulis, s. i. (2020). targeting prospective customers: robustness of machine-learning methods to typical data challenges. management science, 66(6), 2495–2522. https://doi.org/10.1287/mnsc.2019.3308 stetson, p. d., bakken, s., wrenn, j. o., & siegler, e. l. (2012). assessing electronic note quality using the physician documentation quality instrument (pdqi-9). applied clinical informatics, 3(02), 164–174. https://doi.org/10.4338/aci-2011-11-ra-0070 surbakti, f. p. s., wang, w., indulska, m., & sadiq, s. (2019). factors influencing effective use of big data: a research framework. information & management, 57(1), article 103146. https://doi.org/10.1016/j.im.2019.02.001 teodorescu, m. h., morse, l., awwad, y., & kane, g. c. (2021). failures of fairness in automation require a deeper understanding of human-ml augmentation. mis quarterly, 45(3), 1483–1499. https://doi.org/10.25300/misq/2021/16535 torres, r., sidorova, a., & jones, m. c. (2018). enabling firm performance through business intelligence and analytics: a dynamic capabilities perspective. information & management, 55(7), 822–839. https://doi.org/10.1016/j.im.2018.03.010 trieu, v.-h. (2017). getting value from business intelligence systems: a review and research agenda. decision support systems, 93, 111–124. https://doi.org/10.1016/j.dss.2016.09.019 van den broek, e., sergeeva, a., & huysman, m. (2021). when the machine meets the expert: an ethnography of developing ai for hiring. mis quarterly, 45(3), 1557–1580. https://doi.org/10.25300/misq/2021/16559 weick, k. e. (1979). the social psychology of organizing. (2nd edn.). new york, usa: mcgrawhill. weick, k. e. (1995). sensemaking in organizations (vol. 3). thousand oaks, ca, usa: sage publications. weick, k. e., sutcliffe, k. m., & obstfeld, d. (2005). organizing and the process of sensemaking. organization science, 16, 409–421. woerner, s. l., & wixom, b. h. (2015). big data: extending the business strategy toolbox. journal of information technology, 30(1), 60–62. https://doi.org/10.1057/jit.2014.31 australasian journal of information systems li et al. 2024, vol 28, research article machine learning based decision-making 22 woodside, a. g. (2005). opening up decision making: making sense of entrepreneur and reseller business-to-business strategies. journal of business & industrial marketing, 20(7), 347–354. https://doi.org/10.1108/08858620510628588 wu, w., chen, j., yang, z., & tindall, m. l. (2021). a cross-sectional machine learning approach for hedge fund return prediction and selection. management science, 67(7), 4577–4601. https://doi.org/10.1287/mnsc.2020.3696 yoo, y. (2015). it is not about size: a further thought on big data. journal of information technology, 30(1), 63–65. https://doi.org/10.1057/jit.2014.30 copyright: © 2024 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v28.4781 http://creativecommons.org/licenses/by-nc/3.0/au/ https://doi.org/10.3127/ajis.v28.4781 1 introduction 2 theoretical background 2.1 ml and decision-making 2.2 sensemaking for decision-making 3 research method and data collection 4 empirical results 4.1 thematic analysis 4.2 the process of ml-driven sensemaking 5 discussion 5.1 theoretical implications 5.2 practical implications 6 conclusion paper title (paper title style) australasian journal of information systems riemer, lee, kjaer & haeffner 2020, vol 24, selected papers from acis 2018 enterprise social network archetypes in analytics 1 identification of enterprise social network (esn) group archetypes in esn analytics: metrics selection and case application kai riemer university of sydney australia kai.riemer@sydney.edu.au laurence lock lee swoop analytics pty ltd australia cai kjaer swoop analytics pty ltd australia annika haeffner ulm university germany abstract with the proliferation of enterprise social networks (esn), the measurement of esn activity becomes increasingly relevant. the emerging field of esn analytics aims to develop metrics and models to measure and classify user activity to support organisational goals and outcomes. in this paper we focus on a neglected area of esn analytics, the classification of activity in esn groups. we engage in explorative research to identify a set of metrics that divides an esn group sample into distinct types. we collaborate with sydney-based service provider swoop analytics who provided access to actual esn meta data describing activity in 350 groups across three organisations. by employing clustering techniques, we derive a set of four group types: broadcast streams, information forums, communities of practice and project teams. we collect and reflect on feedback from esn champions in fourteen organisations. for esn analytics research we contribute a set of metrics and group types. for practice we envision a method that enables group managers to compare aspirations for their groups to embody a certain group type, with actual activity patterns. keywords: enterprise social network, esn groups, esn analytics, social media analytics, esn metrics. 1 introduction enterprise social networks (esn), online services that allow employees to connect and converse with each other in a secure space, have made fast inroads into organisations with the promise to foster collaboration and enable new work practices (leonardi 2015). according to a report by mckinsey (2012), effective use of such services can result in a 20-25 percent improvement in the productivity of knowledge workers. in the case of one large company forrester research found a return on investment of 365 percent on their esn investment over three years (dodd 2011). more generally, with more organisations adopting esn, questions about how to measure benefits and success abound (bughin 2015). australasian journal of information systems riemer, lee, kjaer & haeffner 2020, vol 24, selected papers from acis 2018 enterprise social network archetypes in analytics 2 the emerging sub field of esn analytics (e.g. schwade and schubert 2017), which is part of social media analytics (behrendt et al. 2014b; stieglitz et al. 2018), aims to develop metrics and models to examine esn log file and content data to gain a better understanding of actual esn usage pattern. this comprises both metrics for measuring the health and structural properties of the overall network, as well as metrics to characterise and classify individual esn user behaviour and user roles (hacker et al. 2017a). while existing work has presented metrics at the network and individual level, the intermediate, or group level of analysis, has not been covered so far. yet, groups play an important role within esn in organising communication and collaboration practices. in this paper we are concerned with identifying and developing metrics to distinguish between esn group types. we ask: which metrics will discriminate esn groups into distinct types? we utilise an esn activity meta-data set, sourced from the yammer networks of three organisations, provided by australian analytics company swoop analytics. we engage in explorative research, using cluster analysis, employing a range of general social network and esn-specific activity metrics to see which ones divide a sample of 350 groups into distinct group types. we find that a combination of three metrics best divided our group sample into clusters that are not only well-interpretable, but relate to, and extend the classification used by yammer itself: 1) evenness of user participation in the group, 2) the degree to which messages elicit responses from others (reciprocity), and 3) network density, the extent to which users interact with all other users in a group. clustering with these metrics resulted in four distinct esn group types: 1) broadcast streams, 2) information forums, 3) communities of practice and 4) project teams. a brief evaluation with feedback solicited from esn champions across 14 swoop client organisations shows that these group types are a good reflection of esn activity more broadly. we present some ideas for extending the typology. for the emerging field of esn analytics our study serves to demonstrate the feasibility of group type identification with typical esn-specific and more general social network analysis (sna) metrics. we do not claim generalisability for our group archetypes, though we would expect our group archetypes to be somewhat typical of esn group proliferation more broadly and thus of use to broader research into understanding the role of groups in esn. for practitioners, such as network managers and group leaders, we outline the scaffold of a method for visualising the discrepancies between aspiration and actual activity in esn groups. this will help group leaders understand how their group is tracking against the patterns of a particular group type that they envision their group to embody. given its pioneering nature, the study points to various promising avenues for future research. 2 background: enterprise social networking enterprise social networks (esn) are part of the enterprise 2.0 phenomenon (mcafee 2009) which describes the application of social technologies such as social networking, blogs, wikis, or microblogging services within organisations (razmerita et al. 2014). as such, esn present a variant of public social networking services (richter et al. 2011). typical services are microsoft yammer, facebook workplace, ibm connections, chatter by salesforce, and smaller services such as mumbacloud, jive or tibbr. australasian journal of information systems riemer, lee, kjaer & haeffner 2020, vol 24, selected papers from acis 2018 enterprise social network archetypes in analytics 3 2.1 definitions and characteristics common to all social media is that they facilitate user participation, interaction, and the generation of content by users (boyd and ellison 2007). specifically, esns are services, accessed through a web browser or mobile app, that allow people to (1) communicate with their co-workers or broadcast messages to everyone within the organisation; (2) explicitly indicate or implicitly reveal particular co-workers as communication partners; (3) post, edit, and sort text and files linked to themselves or others; and (4) view the messages, connections, text, and files posted, edited and sorted by anyone else in their organisation at any time of their choosing (leonardi et al. 2013). another defining characteristic of esn is their malleability (richter and riemer 2013b). unlike more traditional information systems that are employed to solve a concrete problem and are thus associated with a concrete task or purpose, esn are best understood as infrastructures that are intended to create potentials for new ways of communicating and working (riemer et al. 2009). hence, the proliferation of esn often follows a bottom-up approach of implementation, a more inclusive and egalitarian process (schneckenberg 2009). as a result, esn have been associated with a variety of organisational practices such as communication, collaboration (riemer et al. 2010), knowledge management (levy 2009) crowdsourcing (schlagwein and bjorn-andersen 2014), open innovation (dahlander and gann 2010), or open strategy (tavakoli et al. 2015). this renders esn both an interesting and important context for is research, and a challenge for organisations as they have to keep track of the emerging activity in their esn. the research presented here aims to help organisations understand their own enterprise social networks. 2.2 prior research prior research on esn typically falls within one of four existing streams. the first stream captures conceptual work outlining typical esn characteristics (leonardi et al. 2013; treem and leonardi 2012), comparing esn with traditional ways of relationship building in organisations (kane et al. 2014), envisioning high-level benefits, such as for knowledgesharing (fulk and yuan 2013; majchrzak et al. 2013) or interacting in the workplace more generally (dimicco et al. 2008; zhang et al. 2010). the second stream comprises concrete, explorative, usually qualitative case studies investigating usage patterns of esn. typical findings reveal benefits of esn for information sharing and discovery (zhao and rosson 2009), for creating awareness within the organisation (zhang et al. 2010), or for knowledge creation and sharing in professional service contexts (riemer and scifleet 2012). the third stream represents studies that measure the benefits of esn from different angles and in different contexts; these studies are typically quantitative in nature, employing either survey-based approaches, e.g. to measure individual esn benefits for knowledge workers (mantymaki and riemer 2016), or they use data obtained from the esn services directly, such as for studying the connection between social capital and employee performance (riemer et al. 2015). finally, the fourth stream, to which this study contributes, contains generative works that aim to identify metrics and develop analytics frameworks for measurement of esn activity in organisations. studies in this context focus on the application of social network analysis (sna) to reveal emerging informal networks between esn users (behrendt et al. 2014a), identify australasian journal of information systems riemer, lee, kjaer & haeffner 2020, vol 24, selected papers from acis 2018 enterprise social network archetypes in analytics 4 particular value-adding users (berger et al. 2014), characterise individual user activity profiles (holtzblatt et al. 2013), or develop a comprehensive framework for identifying user types in knowledge work contexts (hacker et al. 2017b). 2.3 esn analytics and metrics development as an emerging stream of research, esn analytics is a sub field of social media analytics (stieglitz et al. 2014; stieglitz et al. 2018). sometimes called social collaboration (or social software) analytics, it is “a specialized form of examination of log files and content data, to gain a better understanding of the actual usage of ess” (schwade and schubert 2017, 401). here, we define esn analytics as methods and practices for the identification and utilisation of metrics and models for measuring different aspects of user activity in enterprise social networks, including user activity levels and user profiles, network activity levels, structural network characteristics, and network health indicators, in support of support organisational goals and outcomes. esn analytics becomes relevant for organisations for two main reasons. first, due to their malleability, concrete implementations of esn in organisations typically emerge in quite different ways, supporting a range of different use cases and activities (richter and riemer 2013a). it is thus important for organisations to be able to track and understand user activity on their own particular esn. second, esns are becoming more integral to organisational communication and collaboration practices, which renders gaining an understanding for the activity on the platform and ways in which to support individuals and social groups within the esn more important (schwade and schubert 2017). at the same time, there are opportunities in esn analytics for both academia and industry. first, the particular nature of esn affords detailed analysis of user activity, because all (or most) user interactions are logged and, in principle, available for analysis. second, most esn platforms currently do not provide sophisticated analytics capabilities, leaving room for new offerings by third-party providers and for academia to explore new ways of measuring and accounting for various aspects of esn activity. given the newness of this field the number of studies contributing to establishing metrics or models to support esn analytics is still limited (cf. schwade and schubert 2017). so far two main areas of application for analytics exist. the first area focuses on metrics characterising the social network as such. this is where traditional social network analysis (sna) techniques are brought to bear (wasserman and faust 1994). for example, riemer et al. (2015) have shown how social capital metrics can be utilised to link certain network characteristics to employee performance. behrendt et al. (2014b) provide an overview of sna metrics and studies for use in esn contexts. the second area aims to develop new dedicated esn metrics to characterise individual user behaviours and to generate models that classify user populations into distinct user types. most notably is the research program by hacker and colleagues (e.g. hacker et al. 2017b). other works include a study by cetto et al. (2018) who classified users by knowledge sharing and seeking behaviours, and frank et al. (2017), who utilised log data from exchange, microsoft office 365 and sharepoint to identify user roles (frank et al. 2017). 2.4 analytics of esn groups what is lacking so far are works that engage with esn groups, the intermediate level of analysis between network and individuals. groups play an important role in esn as they allow for the creation of dedicated spaces for conversation and information exchange between australasian journal of information systems riemer, lee, kjaer & haeffner 2020, vol 24, selected papers from acis 2018 enterprise social network archetypes in analytics 5 a subset of users. given their usefulness many companies find that the number of groups tends to proliferate over time, with some groups very active and many others abandoned. at the same time groups are used for different purposes, and they exist in different shapes and forms, from very small ones to large behemoths. we suggest that a better understanding of different group types, their structural features and activity patterns, will be useful for decision-makers in better harnessing their esn for value. however, we are aware of only one study engaging in detail with esn user activity at the group level, classifying groups in the context of knowledge work (riemer and tavakoli 2013). however, this study is not useful in the context of esn analytics, since the classification was based on a manual coding of user messages, which is impractical as the basis for analytics practices. accordingly, we investigate the following question: which set of metrics discriminates best a population of esn groups such that it results in a set of meaningful group types characterised by different activity patterns? 3 study overview we utilise esn activity meta-data from three yammer networks, obtained from australian analytics company swoop analytics pty ltd (in the following just: swoop). we collect a range of esn metrics from extant literature and operationalise these based on the swoop data set. we then test each metric to see which ones divide the sample of groups in our data set into distinct types. we briefly introduce our research setting and data set, before we outline our method and research approach, utilising actual esn meta data describing activity in 350 groups across three organisations. 3.1 research setting and esn data set swoop offers a cloud-based platform that provides analytics for organisations’ yammer, facebook workplace and microsoft teams networks. when given permission by an organisation to integrate with its network, swoop “provide[s] access to more than 30 measurement indicators giving organisations and individuals deep insights into collaboration across the enterprise.” it uses these metrics to provide user profiles, in the form of a typology that classifies each user. generally, any action performed by an esn user is stored in the backend database of the esn system and available in the form of digital traces, “digitally stored, event-based, chronological records of activities of actors, which result in direct or indirect actor relations or content in different data formats” (behrendt et al. 2014a, 4). we distinguish usage data, or meta-data from user-generated data, or content, which contains what was posted. in order to ensure confidentiality swoop does not collect any content from organisations, only meta-data. metadata is data about activities or interactions that indicates how, when and where an esn activity was performed, what kind of interaction was performed, and who was involved. whereas the yammer data model is organised around messages, swoop provides esn activity data already organised as interactions between users. moreover, swoop collects from an organisation’s esn information that is not included in the yammer database, such as information on ‘likes’ or ‘mentions’ of other users (tagging), each of which are represented in the swoop data model as particular interactions. swoop distinguishes the following interaction types: post, reply, notification, mention or like. table 1 shows what meta data is available for each interaction. australasian journal of information systems riemer, lee, kjaer & haeffner 2020, vol 24, selected papers from acis 2018 enterprise social network archetypes in analytics 6 id unique identifier for each interaction class type of interaction: post, reply, notification, mention or like from user-id of user initiating the interaction to user-id of targeted user (not relevant if class equals post, as post is undirected) thread id unique id for every thread, every interaction belongs to a thread, “post” creates new thread date timestamp of the interaction group id unique id of the group in which an interaction takes place (if empty, not in group) table 1: meta-data for each interaction in the swoop data model for this study we had access to data from yammer networks of three firms (two financial service firms and one professional service firm). the data provided by swoop (with the firms’ permissions) contained meta-data of all interactions in the various groups across these networks for a representative 10-week period. to protect user privacy swoop only shared anonymised meta-data, which was stripped of user and group names. users, groups and all interactions remain traceable however through their unique ids. in total, the data set contained 683,733 interactions by 40,304 users in 350 groups. 3.2 data preparation: construction of the esn social graph any analytics of esn user activity has to begin with the construction of the social network graph. generally, a social network “consists of a finite set or sets of actors and the relation or relations defined on them” (wasserman and faust 1994, 20). whereas in public social networks, such as twitter or facebook, networks can be inferred from explicit friend or follower relationships, in esns relationships have to be constructed from user activity, as follower relationships either do not exist or are inconsequential to communication on the platform (behrendt et al. 2014a). at the most basic level a dyadic relationship between two individuals is said to exist when one user responds to another's message (ahuja et al. 2003). this is in line with social network theory, which asserts that relationships emerge from interactions (granovetter 1973; krackhardt 1992). esn meta-data can thus be utilised to infer the ensuing network (behrendt et al. 2014b). for our study, swoop provided various types of interactions between users that can be utilised to construct network graphs for each group in our sample. at the same time, the inclusion of different interaction types in graph creation has implications for calculating and interpreting metrics; for example, does liking someone’s post constitute a relationship with that person, or should a relationship only be considered based on a reply to a message, as this suggests that the respondent has actually read (and not merely seen) the message and found it stimulating enough to interact? 3.3 iterative research process utilising cluster analysis our aim was to identify those metrics that best discriminate the sample of esn groups in a way that results in certain archetypes describing groups regarding their activity patterns. much like individual user profiles and archetypes already provided by swoop, the question we explore in this study is thus, can we identify a set of metrics that provides a similar set of group archetypes? given the explorative nature of this question, our research approach needed to be ‘creative’ and iterate between identification and calculation of metrics and a clustering of groups based on varying sets of metrics. hence, the steps in this process were: australasian journal of information systems riemer, lee, kjaer & haeffner 2020, vol 24, selected papers from acis 2018 enterprise social network archetypes in analytics 7 1) identification of metric candidates, describing both the network structure of a group and user activity, 2) selection of metrics for inclusion in cluster analysis, 3) calculation of metrics for each group, 4) selection of a cluster algorithm and proximity measure, 5) performing of cluster analysis, 6) interpretation of results, and 7) soliciting feedback from esn user organisations. a tool was implemented using the software package matlab to facilitate iterating on steps 2 to 6 until a result emerged that a) discriminated well into distinct group clusters, and b) was interpretable in a way that corresponds with typical esn use, i.e. that made sense from a practical point of view. a cluster analysis is a method for semi-automated grouping of large numbers of objects based on their similarity described by a vector of quantified characteristics (hartigan 1975). previous research already demonstrated that clustering techniques are useful for classifying complex networks of different kinds (newman and girvan 2004; strogatz 2001). for this study we experimented with a number of clustering algorithms (song et al. 2012). ultimately agglomerative clustering, in particular the complete-linkage algorithm (defays 1977; krznaric and levcopoulos 1998) with a standardised euclidean distance measure (pandit and al. 2011) produced the most useful results. cluster analysis is ‘semi-automated’ because it is up to the researchers to determine whether or not a clustering was successful. according to everitt (1993) success is given when the researcher, who is familiar with the data, can sensibly interpret the resulting clusters. a good set of clusters shows homogeneous and clearly separable clusters. to identify clusters we used dendrograms, plotting of metrics and a three-dimensional plot of group locations according to their metric values. in turn, the requirement to judge and interpret the clustering result in each instance, meant that it turned out not to be feasible to include more than three metrics in each clustering attempt. each clustering was thus done on the basis of triplets of metrics. this allowed surfacing first which individual metrics, and second which metric combinations discriminated the group sample most distinctively (given that some metrics correlate and didn’t discriminate in distinct ways). a large number of individual clustering runs were performed in a semi-automated way using matlab, from which first suitable metrics emerged and which in a second step converged on the most suitable group clustering. we refrain here from reporting the details of the individual cluster analyses as this goes beyond the scope of this paper. 4 findings: key metrics and four esn group archetypes in this section we present our findings in three steps. first, we briefly introduce a list of key metrics candidates derived from existing esn analytics literature; in a second step discuss those three metrics in more detail that were shown to be the ones best discriminating our data set into distinct and interpretable groups. third, we introduce and discuss the four esn group archetypes that resulting from the cluster analyses. australasian journal of information systems riemer, lee, kjaer & haeffner 2020, vol 24, selected papers from acis 2018 enterprise social network archetypes in analytics 8 4.1 catalogue of key metrics candidates drawing on existing research we identified a list of metrics candidates: 1) esn group activity metrics describe different aspects of communication in each group, such as how many users post, how many interactions are carried out, how responsive users are in replying, how many replies each post elicits, how many users engage in each discussion. our list (see table 2) was adapted from the metrics catalogue provided by hacker et al. (2016). 2) social network metrics characterize structural properties of the social graph of a group, such as how densely users in a group are linked, how diverse the external links of users to other groups are, or to what extent the network is dominated by particular users, as measured by the gini coefficient (yakovenko and rosser jr 2009) (see appendix 1 for gini calculation). metric measurement interpretation esn group activity metrics # active users number of users who performed at least one interaction inside a group within a timeframe allows comparing groups according to different levels of user involvement # interactions number of interactions inside a group within a timeframe allows comparing groups regarding different activity levels response rate (threads and posts) the share of threads/posts with at least one reply (likes are not counted as reply) measure the level of engagement in a group response rate (includes likes) modified response rate that includes also likes measures level of recognition, not just actual responses. replies per thread average number of replies per message thread measures extent to which group engages in longer discussions. passivity number of likes divided by number of replies measures the level of mere recognition relative to actual engagement users per thread average number of different users that contribute to one thread an alternative measure of engagement. group social network metrics density of directed & undirected graph number of actual edges divided by the number of possible edges between nodes measures how evenly group members interact with each other. user diversity (external links) average number of groups in which the users of a particular group are active measures how diverse the user population of a group is in terms of membership in other groups gini coefficient general measure of equality applied to number of interactions per user (0=all users contribute equally, 1=all contributions by one user). measures how equal the contributions in a group are distributed among its users. table 2: overview of key metrics candidates considered during the exploratory analysis 4.2 metrics that best discriminate the groups sample our explorative analysis ‘tested’ varying triplet combinations of metrics by running separate cluster analyses on the sample of 350 groups each time. the analyses surfaced a set of three metrics that not only discriminate well within the group sample, but also differentiate the groups into four distinct clusters that are well interpretable and that correspond with known uses of esn groups in organisations: 1. density of directed graph: for each group a directed graph is created by adding a node for each active user and a directed edge between all node pairs whose userids appear as “from” and “to” in one or more transactions inside the group; the australasian journal of information systems riemer, lee, kjaer & haeffner 2020, vol 24, selected papers from acis 2018 enterprise social network archetypes in analytics 9 edge points to the node whose user-id appears as “to”. the density of this graph is defined as the number of existing edges divided by the number of possible edges. density is a measure of the degree to which members of the group are connected, resulting from people talking directly to each other. 2. gini coefficient: this metrics stems from economics and was originally intended to measure wealth inequality, that is the unevenness of wealth distribution. in the esn context, it measures how evenly activity in a group is distributed. the higher the gini coefficient, the more uneven is the activity distributed in a group. a gini of 1 means that only one person is responsible for all activity, a gini of 0 means everyone contributes exactly the same amount of activity. 3. thread reciprocity: thread reciprocity measures the share of all posts with at least one reply. it is thus akin to a response rate measure. groups with a high thread reciprocity are more conversational. note that a like is not regarded as a reply; rather, a genuine response post is required. 4.3 group types resulting from the cluster analysis from these metrics the clustering algorithm derived a total of initially five clusters (chosen after visual inspection of the resulting dendrogram). after a further detailed analysis of the five clusters we decided to merge the two smallest of the clusters (shown as clusters 3 and 5 in figure 1, and in red and green in figure 2) as they turned out to be quite similar in terms of metrics. figure 1 demonstrates for each of the three metrics separately how they discriminate between the clusters; figure 2 provides a three-dimensional plot which visually locates all 350 groups; and table 3 names and summarizes the metrics for each of the four clusters. in the following we interpret each of the clusters. figure 1: metrics values for each of the resulting clusters australasian journal of information systems riemer, lee, kjaer & haeffner 2020, vol 24, selected papers from acis 2018 enterprise social network archetypes in analytics 10 figure 2: cluster locations [shape of markers = company; marker size = group size] cluster metrics # of active users # colour name particip. (gini) density reciprocity avg min max 1 blue broadcast streams uneven low low 80.5 13 352 2 light blue information forums even low low 59.0 36 107 4 orange community of practice uneven low high 125.9 9 1018 3/5 green/red project teams even med/high high 9.7 7 13 table 3: overview of group classification according to the three metrics broadcast streams: these groups are quite large in terms of active users (those who interacted at least once in the 10-week period), yet they show only low levels of interaction and participation across the user population. rather, they feature many single messages written by a small number of participants, and a large number of people who mostly read and only occasionally post. in addition, people are not well-connected with each other. such characteristics are typical of groups used for announcements and the broadcasting of information. typical uses are corporate communications or hr departments and business divisions pushing information to users in ways that resemble one-to-many ‘intranet’ use. such communication does not require responses from (reciprocity), or interaction among users (density). the relatively large number of active users is explained by ‘likes’ acknowledging posts. information forums: significant about this group type is that, while it shows rather even participation among users posting into the group, these posts do not solicit many replies from other users, or lead to interactions among users to build relationships. such properties are typical of information forums, in which people post information, questions or requests for other users, but which are not home to many conversations or actual work interactions. 0.1 1 0.2 0.3 0.4 0.8 1 0.5 g in i to ta l a c ti o n s 0.6 0.6 0.8 0.7 thread reciprocity 0.8 0.6 density directed 0.4 0.9 0.4 0.2 0.2 0 0 australasian journal of information systems riemer, lee, kjaer & haeffner 2020, vol 24, selected papers from acis 2018 enterprise social network archetypes in analytics 11 community of practice: these groups show uneven participation but high reciprocity. this means that, while many posts receive replies from other users, these initial messages are written by a core group of members. in addition, the overall network density is low in that people are not well connected among each other. the latter is partly explained by the fact that these groups are the largest on average in our sample. we term these groups ‘communities of practice’ (cop). cops are groups of loosely connected members which often congregate around a particular topic and a core group of leaders or experts in the context of organisational learning and knowledge exchange, while a rather large number of group members follow the conversation as an audience and only occasionally participate. project teams: these groups are by far the smallest in our sample and show significantly higher levels of connection between the group members than groups in the other three clusters. they are also highly interactive and conversational with even participation. such properties are typical of project teams in which all group members are actively involved in performing joint work and all group members interact and converse with each other on a daily basis. 4.4 user feedback on group archetypes we had the opportunity to present this research and the resulting esn group archetypes to a selection of swoop client organisations utilising microsoft yammer, in order to solicit feedback. a brief online questionnaire was set up for participants of one of swoop’s user group events to fill in. the survey comprised two free-text questions: 1. what are your thoughts on the groups presented? do they make sense? are they useful? why/why not? 2. if you think there is a particular type of group missing, please tell us about it. we solicited responses from yammer champions across fourteen organisations. overwhelmingly the responses were in support of the group typology with 13 of the respondents clearly signalling that they were useful or mostly useful, commenting that they “make sense”, or “resonate” with their own networks in that they themselves “have a mix of them all” or that the typology covers the groups that they had seen in their own yammer network. given the overall agreement, we suggest that the archetypes exhibit a certain universality or applicability across organisations. additionally, a few respondents named further group types as complements to the typology; the most relevant ones were: 1) groups for non-work-related content (4 mentions: “work social info group”, “common interest groups”, “informal groups”, or “non-work groups”) 2) groups for distributing corporate resources (1 mention), akin to an intranet site that allows users to ask questions and discuss content. 3) groups for large, short-term engagement events (1 mention), such as a “hackathon”. we note firstly that we did not investigate and discriminate our group sample by content, which means that a distinction between work or non-work content cannot be made, but we would expect that such groups could show interaction patterns of all four archetypes above, as one can imagine non-work initiatives of all different shapes. this would not justify adding australasian journal of information systems riemer, lee, kjaer & haeffner 2020, vol 24, selected papers from acis 2018 enterprise social network archetypes in analytics 12 a special group type on those ground. secondly, we would expect that groups for distributing corporate content which are quite interactive will manifest as an outlier of the broadcast cluster, in that reciprocity and density are slightly higher, but not at the levels that would warrant classification as one of the other group types. finally, large temporary groups with high levels or density, reciprocity and even participation might well exist (as in project teams), but did not show up in our sample as all groups of this type were quite small, likely because they are rare and thus none fell into our time-bounded sample. 5 discussion we set out to investigate which set of metrics discriminates best in a sample of esn groups, using cluster analysis, to derive a set of meaningful group types characterised by different activity patterns. our explorative analysis, utilising activity meta-data from 350 groups from three organisations, converged on three metrics that measure reciprocity in terms of the proportion of messages eliciting replies, evenness of user participation, and density in terms of user connectedness in the group. those metrics in turn distinguish four distinct group types, which we named broadcast streams, information forums, communities of practice and project teams. our brief evaluation with client organisations lends further support to the typology. we note that these groups correspond to, but also extend, the group types that esn providers such as microsoft yammer or facebook workplace provided as templates for their users when creating new groups (see figure 3). 5.1 comparison of our group typology with esn provider templates in the case of yammer, two of our group types, project teams (‘project’) and broadcast streams (‘my organisation’) have direct equivalents, while yammer subsumes all other use cases under a broad category ‘community’, intended for users to “share best practices, learn new skills and connect around shared interests.” yammer’s recent decision to suspend the group classification feature, after feedback from users, indicates that the typology was not granular enough and thus unhelpful. our finding suggests that a distinction should be made between communities of practices and information forums to differentiate those groups that are intended to focus on learning and sharing best practices from those that revolve around common interests and information sharing. the former, communities of practice, require more interaction and conversations between users (as measured by reciprocity), but at the same time will show a certain unevenness in participation (as measured by gini), given that sharing of best practices and learning come with a differentiation in roles between experts/teachers and a broader audience of learners. this distinction is further supported by earlier, content-based studies that classify esn use cases, where a strong distinction is made between communication genres that generate ‘discussion and conversation’ and those that are mainly one-way for ‘providing input’ for others (riemer et al. 2011). in the case of facebook workplace, we note that this esn similarly operates with three main categories, ‘teams & projects’, ‘open discussions’ and ‘announcements’. in addition, workplace uses a group category ‘social & more’ to separate out non-work-related communication, much like recommended by four of the swoop client organisation, and also adds two more specialised groups (‘multi-company’ and ‘buy & sell’) that are out of scope in our context as they stray from the purpose of an esn. we note that separating ‘social’ conversations from work-related ones, while appealing to certain executive managers, might australasian journal of information systems riemer, lee, kjaer & haeffner 2020, vol 24, selected papers from acis 2018 enterprise social network archetypes in analytics 13 send the questionable signal to employees that non-work-related conversations, while tolerated, are somehow ‘second rate’. prior research has shown however that healthy esns exhibit about 40% communication that are not necessarily work-related but form the basis for any network community to exist in the first place (riemer et al. 2011). figure 3: group template for yammer (left) and facebook workplace (right) 5.2 usefulness of the esn group typology the feedback from swoop and its client base suggests that our typology will be helpful for esn group leaders and community managers for managing their esn networks. specifically, we suggest that measurement of group characteristics will allow group leaders to compare their aspiration for what a particular group intends to become with actual usage patterns. for example, a group that intends to support a project team might, upon application of our metrics, be classified as a community of practice, indicating a lack of density, which comes with unhelpful network fragmentation in the project team. similarly, an intended cop might be classified as a broadcast stream, indicating a lack of engagement (reciprocity) among its members. finally, an intended information forum that lacks even participation becomes lopsided with a lack of diversity in contributions and perspectives (see figure 4). we suggest that knowledge of such discrepancies will allow group leaders to manage and counteract accordingly. figure 4: examples of possible discrepancies between group aspiration and measured types project teams: how even is participation distributed? how well connected are users with each other? broadcast streams: only few people post. most people read. but: maybe could achieve more engagement? communities of practice: will be content with a number of passive users. but what is the level of engagement? information forums: will be content with rather low engagement but how even is participation to ensure diverse input? lack of connection uneven participation la ck o f en ga ge m en t australasian journal of information systems riemer, lee, kjaer & haeffner 2020, vol 24, selected papers from acis 2018 enterprise social network archetypes in analytics 14 6 conclusion our study contributes to esn research in general, and the emerging field of esn analytics more specifically, by extending esn analytics approaches to the group level. specifically, we contribute initial metrics and a typology of esn groups according to activity patterns, as the basis for broader research into understanding the role of groups in esn networks. furthermore, our study contributes to esn practice a method for esn group leaders and network managers to measure group activity in a meaningful way, to visualise discrepancies between group aspiration and actual user activity, as measured by our metrics, and thus to improve group communication to achieve intended communication patterns. we envision that our metrics and classification could suitably be implemented in platforms such as that provided by swoop. every research study is circumscribed by certain design choices and limitations and ours is no exception. firstly, our study was driven by curiosity and a practical interest in learning about user behaviour in esn groups. as theorising in this space is still in its infancy, future studies will aim to consolidate empirical findings into more generalisable insights. secondly, our study is merely a first, necessarily limited step in a broader research endeavour to extend analytics to the esn group level. while we had access to a unique and relevant data set, we only derived one typology. without doubt, future research is needed to corroborate our findings. we envision that future analyses will apply similar explorative analyses to different esn networks to replicate our results, unearth additional useful metrics for discriminating group activity and extending our typology. thirdly, we focused on the group level alone, but did not measure any interaction effects between individual behaviour and collective group behaviour. hence, it will be worthwhile investigating the link between group-level and individual-level metrics and types, such as those identified by (hacker et al. 2017b). for example, will groups of certain types benefit from the presence of certain individual user types among its members? fourthly, while we did not set out to explore the best clustering technique to identify group types, we acknowledge that there might be clustering methods that could potentially provide equally useful or better results. finally, we only carried out a limited survey-based study to corroborate our results with user organisations. additional qualitative research, utilising interviews with network managers and group leaders, might investigate the usefulness of our proposed typology, whether it captures all intended use cases for groups, and how it might be used to support decision-making. it would be particularly interesting to see how the implementation of group metrics in platforms such as swoop will shape user behaviour. references ahuja, m. k., galletta, d. f., and carley, k. m. 2003. "individual centrality and performance in virtual r&d groups: an empirical study," management science (49:1), pp. 21-38. behrendt, s., richter, a., and riemer, k. 2014a. "conceptualisation of digital traces for the identification of informal networks in enterprise social networks," proceedings of the 2014 australasian conference on information systems. behrendt, s., richter, a., and trier, m. 2014b. "mixed methods analysis of enterprise social networks," computer networks (75), pp. 560-577. australasian journal of information systems riemer, lee, kjaer & haeffner 2020, vol 24, selected papers from acis 2018 enterprise social network archetypes in analytics 15 berger, k., klier, j., klier, m., and richter, a. 2014. "who is key?-value adding users in enterprise social networks," proceedings of the 2014 european conference on information systems (ecis). boyd, d. m., and ellison, n. b. 2007. "social network sites: definition, history, and scholarship," journal of computer‐mediated communication (13:1), pp. 210-230. bughin, j. 2015. "taking the measure of the networked enterprise," mckinsey quarterly (51:10), pp. 1-4. dahlander, l., and gann, d. m. 2010. "how open is innovation?," research policy (39:6), pp. 699-709. defays, d. 1977. "an efficient algorithm for a complete link method," the computer journal (20:4), pp. 364-366. dimicco, j., millen, d. r., geyer, w., dugan, c., brownholtz, b., and muller, m. 2008. "motivations for social networking at work," proceedings of the 2008 acm conference on computer supported cooperative work: acm, pp. 711-720. dodd, l. 2011. "study reveals huge roi when using yammer," strategic communication management (15:6), p. 7. everitt, b. s. 1993. cluster analysis (third edition). london melbourne auckland: edward arnold. frank, l., gimpel, h., schmidt, m., and schoch, m. 2017. "emergent user roles of a digital workplace: a network analysis based on trace data," 38th international conference on information systems, seoul, south korea. fulk, j., and yuan, y. c. 2013. "location, motivation, and social capitalization via enterprise social networking," journal of computer‐mediated communication (19:1), pp. 20-37. granovetter, m. 1973. "the strength of weak ties," american journal of sociology (78:6), pp. 1360-1380. hacker, j. v., bernsmann, r., and riemer, k. 2017a. "dimensions of user behavior in enterprise social networks," in social knowledge management in action: applications and challenges, r. helms, j. cranefiels and j. van reijsen (eds.). cham, switzerland: springer international publishing, pp. 125-146. hacker, j. v., bodendorf, f., and lorenz, p. 2016. "a framework to analyze enterprise social network data," in enterprise big data engineering, analytics, and management, m. atzmueller, s. oussena and t. roth-berghofer (eds.). hershey: igi global, pp. 84-107. hacker, j. v., bodendorf, f., and lorenz, p. 2017b. "a framework to identify knowledge actor roles in enterprise social networks," journal of knowledge management (21:4), pp. 817838. hartigan, j. a. 1975. clustering algorithms. new york: john wiley & sons. holtzblatt, l., drury, j., and weiss, d. 2013. "evaluating the uses and benefits of an enterprise social media platform," journal of social media for organizations (1:1), pp. 1-21. australasian journal of information systems riemer, lee, kjaer & haeffner 2020, vol 24, selected papers from acis 2018 enterprise social network archetypes in analytics 16 kane, g. c., alavi, m., and labianca, g. j. 2014. "what's different about social media networks? a framework and research agenda," management information systems quarterly (38:1), pp. 274-304. krackhardt, d. 1992. "the strength of strong ties: the importance of philos in organizations," networks and organizations: structure, form, and action (216), p. 239. krznaric, d., and levcopoulos, c. 1998. "fast algorithms for complete linkage clustering," discrete & computational geometry (19:1), pp. 131-145. leonardi, p. m. 2015. "ambient awareness and knowledge acquisition: using social media to learn “who knows what” and “who knows who” " mis quarterly (39:4), pp. 747762. leonardi, p. m., huysman, m., and steinfield, c. 2013. "enterprise social media: definition, history, and prospects for the study of social technologies in organizations," journal of computer‐mediated communication (19:1), pp. 1-19. levy, m. 2009. "web 2.0 implications on knowledge management," journal of knowledge management (13:1), pp. 120-134. majchrzak, a., faraj, s., kane, g. c., and azad, b. 2013. "the contradictory influence of social media affordances on online communal knowledge sharing," journal of computer‐ mediated communication (19:1), pp. 38-55. mantymaki, m., and riemer, k. 2016. "enterprise social networking: a knowledge management perspective," international journal of information management (36:6), pp. 1042-1052. mcafee, a. 2009. enterprise 2.0: new collaborative tools for your organization's toughest challenges. boston: mcgraw-hill professional. mckinsey. 2012. "the social economy: unlocking value and productivity through social technologies." from http://www.mckinsey.com/insights/high_tech_telecoms_internet/the_social_economy newman, m. e. j., and girvan, m. 2004. "finding and evaluating community structure in networks," physical review (69:2), pp. 26-113. pandit, s., and al., e. 2011. "a comparative study on distance measuring approaches for clustering," international journal of research in computer science (2:1), pp. 29-31. razmerita, l., kirchner, k., and nabeth, t. 2014. "social media in organizations: leveraging personal and collective knowledge processes," journal of organizational computing and electronic commerce (24:1), pp. 74-93. richter, a., and riemer, k. 2013a. "the contextual nature of enterprise social networking: a multi case study comparison," 21st european conference on information systems ecis 2013, utrecht university, netherlands. richter, a., and riemer, k. 2013b. "malleable end-user software," business & information systems engineering (5:3), pp. 195-197. richter, d., riemer, k., and vom brocke, j. 2011. "internet social networking," business & information systems engineering (53:2), pp. 89-103. australasian journal of information systems riemer, lee, kjaer & haeffner 2020, vol 24, selected papers from acis 2018 enterprise social network archetypes in analytics 17 riemer, k., altenhofen, a., and richter, a. 2011. "what are you doing? enterprise microblogging as context building," european conference on information systems (ecis), helsinki, finnland, pp. 1-13. riemer, k., finke, j., and hovorka, d. s. 2015. "bridging or bonding: do individuals gain social capital from participation in enterprise social networks?," international conference on information systems icis 2015, fort worth, united states. riemer, k., richter, a., and böhringer, m. 2010. "enterprise microblogging," business & information systems engineering (2:6), pp. 391-394. riemer, k., and scifleet, p. 2012. "enterprise social networking in knowledge-intensive work practices: a case study in a professional service firm," proceedings of the 23rd australasian conference on information systems 2012: acis, pp. 1-12. riemer, k., steinfield, c., and vogel, d. 2009. "ecollaboration: on the nature and emergence of communication and collaboration technologies," electronic markets (19:4), pp. 181188. riemer, k., and tavakoli, a. 2013. "the role of groups as local context in large enterprise social networks: a case study of yammer at deloitte australia," the university of sydney, sydney. schlagwein, d., and bjorn-andersen, n. 2014. "organizational learning with crowdsourcing: the revelatory case of lego," journal of the association for information systems (15:11). schneckenberg, d. 2009. "web 2.0 and the empowerment of the knowledge worker," journal of knowledge management (13:6), pp. 509-520. schwade, f., and schubert, p. 2017. "social collaboration analytics for enterprise collaboration systems: providing business intelligence on collaboration activities," 50th hawaii international conference on system sciences, hawaii, pp. 401-410. song, w.-m., di matteo, t., and aste, t. 2012. "hierarchical information clustering by means of topologically embedded graphs," plos one (7:3), p. e31929. stieglitz, s., dang-xuan, l., bruns, a., and neuberger, c. 2014. "socialmedia analytics: an interdisciplinary approach and its implications for information systems," business & information systems engineering:2), pp. 89-96. stieglitz, s., mirbabaie, m., ross, b., and neuberger, c. 2018. "social media analytics – challenges in topic discovery, data collection, and data preparation," international journal of information management (39), pp. 156-168. strogatz, s. h. 2001. "exploring complex networks," nature (410:6825), p. 268. tavakoli, a., schlagwein, d., and schoder, d. 2015. "open strategy: consolidated definition and processual conceptualization," in: international conference on information systems (icis). fort worth, usa. treem, j. w., and leonardi, p. m. 2012. "social media use in organizations: exploring the affordances of visibility, editability, persistence, and association," communication yearbook (36), pp. 143-189. wasserman, s., and faust, k. 1994. social network analysis: methods and applications. cambridge: cambridge university press. australasian journal of information systems riemer, lee, kjaer & haeffner 2020, vol 24, selected papers from acis 2018 enterprise social network archetypes in analytics 18 yakovenko, v. m., and rosser jr, j. b. 2009. "colloquium: statistical mechanics of money, wealth, and income," reviews of modern physics (81:4), p. 1703. zhang, j., qu, y., cody, j., and wu, y. 2010. "a case study of microblogging in the enterprise: use, value, and related issues," 28th annual sigchi conference on human factors in computing systems, atlanta. zhao, d., and rosson, m. b. 2009. "how and why people twitter: the role that microblogging plays in informal communication at work," proceedings of the acm 2009 international conference on supporting group work: acm, pp. 243-252. australasian journal of information systems riemer, lee, kjaer & haeffner 2020, vol 24, selected papers from acis 2018 enterprise social network archetypes in analytics 19 appendix algorithm for the gini coefficient 1. count number of contributions for each active user of the group(likes, posts, replies, mentions), then sort them from low to high. 2. calculate lorentz curve: y-axis: proportion of total contributions that are made by the bottom x% of the users (see figure) 3. calculate size of area between red and blue line of figure 4. standardize by multiplying by 2 5. get value between 0 (if all users contributed equally) and 1 (if only one user contributed) responses to client survey respondent q1: what are your thoughts on the groups presented? do they make sense? are they useful? why/why not? q2: if you think there is a particular type of group missing, please tell us about it. 1 i thought the presentation was amazing and i really got a better understanding of swoop and just how many businesses use it! no 2 yes they make sense n/a 3 yes they resonate but there is a cross over typebetween project and community of practice. based off networks working with broader bandwidth than one project or topic. similar several stronger contributors but lots of conversation to purpose example culture and strategy networks across teams maybe to define the prior mix between cop and project 4 yes i think you need a work social info group. there is an employee discount group at westpac that is both broadcast and also questions on discounts. 5 mostly useful. don’t include non-business groups. otherwise they cover most of the groups fairly well. non-work groups. 6 yes, they cover the groups we have seen in our organisation not really 7 they make sense, but i think there might be some groups that do not fit into these definitions we have some common interest groups that publish memes, etc 8 yes n/a 9 yes. certainly resonates with our network not that i can think of 10 useful informal groups external benchmarks 11 yes all are applicable. we have a mix of them all! na 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 share of users 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 s h a re o f co n tr ib u tio n s lorentz curve (example) line of equality australasian journal of information systems riemer, lee, kjaer & haeffner 2020, vol 24, selected papers from acis 2018 enterprise social network archetypes in analytics 20 respondent q1: what are your thoughts on the groups presented? do they make sense? are they useful? why/why not? q2: if you think there is a particular type of group missing, please tell us about it. 12 yes, they are the most commonly useful groups in a yammer network. corporate resource. our organization has some very interactive corporate resource groups where users can comment on current initiates and ask questions. health and wellness is an example. 13 yes am witnessing similar structures the short term team related to large engagement events like hackathons 14 seems sensible... perhaps there is another category of persistent team e.g. call centre team...as opposed to project team, which has a defined beginning and end. copyright: © 2020 riemer, lee, kjaer & haeffner. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v24i0.2355 http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems prakash & das 2024, vol 28, research article (why) do we trust ai? 1 (why) do we trust ai?: a case of ai-based health chatbots ashish viswanath prakash indian institute of management tiruchirappalli, tamil nadu, india ashish@iimtrichy.ac.in saini das indian institute of technology kharagpur, india abstract automated chatbots powered by artificial intelligence (ai) can act as a ubiquitous point of contact, improving access to healthcare and empowering users to make effective decisions. however, despite the potential benefits, emerging literature suggests that apprehensions linked to the distinctive features of ai technology and the specific context of use (healthcare) could undermine consumer trust and hinder widespread adoption. although the role of trust is considered pivotal to the acceptance of healthcare technologies, a dearth of research exists that focuses on the contextual factors that drive trust in such ai-based chatbots for selfdiagnosis (aicsd). accordingly, a contextual model based on the trust-in-technology framework was developed to understand the determinants of consumers’ trust in aicsd and its behavioral consequences. it was validated using a free simulation experiment study in india (n = 202). perceived anthropomorphism, perceived information quality, perceived explainability, disposition to trust technology, and perceived service quality influence consumers’ trust in aicsd. in turn, trust, privacy risk, health risk, and gender determine the intention to use. the research contributes by developing and validating a context-specific model for explaining trust in aicsd that could aid developers and marketers in enhancing consumers’ trust in and adoption of aicsd. keywords: artificial intelligence, health chatbot, trust in technology, explainability, contextualization, free simulation experiment. 1 introduction the popularity of autonomous chatbots/conversational agents (cas) has grown substantially in recent years, owing to recent advancements in artificial intelligence (ai) technology and the growing digitalization of industries (araujo, 2018; chi et al., 2021). an ai-based health chatbot is an autonomous system that can converse intelligently about healthcare-related issues in either audio or textual format (laumer et al., 2019; prakash & das, 2020). these chatbots are posited to address the most pressing problems of healthcare services, such as misinformation, information overload, and timely access to quality healthcare (denecke et al., 2019; wang & siau, 2018). an exciting innovation in this area is the ai-based chatbots for self-diagnosis (aicsd) (laumer et al., 2019). these user-facing intelligent applications engage with individuals in real-time, collect information about the condition from the user, make personalized diagnostic inferences from the data, and offer recommendations. ada health, microsoft healthbot, nhs chatbot, babylon health, etc., are popular aicsds offering directto-consumer diagnosis/triage services (siwicki, 2018). the global healthcare chatbot market size is growing at a cagr of 21.56% and is expected to be us$ 967.7 million by 2027 (pr newswire, 2021). australasian journal of information systems prakash & das 2024, vol 28, research article (why) do we trust ai? 2 aicsds are seen as credible alternatives to traditional search engines and can potentially improve access and alleviate the overburdened healthcare resources (wang & siau, 2018). however, the benefits promised come with significant risks to the safety and life of patients as there are doubts regarding the efficacy and accuracy of such chatbots (bickmore et al., 2018; laranjo et al., 2018). further, handling personal health information raises a plethora of issues relating to data privacy, security, consent, and ownership (powell, 2019). moreover, regardless of how effective and reliable they are, most of these bots, by virtue of their inscrutable algorithms, provide little hints as to how they arrive at their decisions, raising concerns about transparency, accountability, and responsibility – which are fundamental to building trust in a system (gille et al., 2020; powell, 2019; rai, 2019). further, it is observed that the perceived humanness of these intelligent agents elicits paradoxical responses from the users (ho & macdorman, 2017). this may further exacerbate the consumers’ trust in these applications. thus, the adoption of this promising healthcare technology faces acute challenges owing to its unique characteristics and the sensitivity of the context (o’connor et al. 2021). prior research has argued that trust in specific technology is central to understanding individual technology use behavior (mcknight et al., 2011). trust in aicsd is the belief that it has the essential qualities to provide competent, reliable, and truthful symptom-checking and triage services. further, studies on ai-enabled systems in general (bach et al., 2022; hengstler et al., 2016) and those specific to ai-based health chatbots (laranjo et al., 2018; nundy et al., 2019) indicate that lack of trust in the system could be one of the major impediments to the adoption. hence, it is valuable for practitioners to know the comprehensive set of interconnected factors that predict the trust in aicsd to debottleneck its adoption. however, academic research into this specific area of patient/consumer trust in ai applications in healthcare (including chatbots) has been sparse (asan & choudhury, 2021; laumer et al., 2019; o’connor et al., 2021; prakash & das, 2020). research on the specific issue of what constitutes consumers’ trust in aicsd and the conditions for achieving it is yet to be explored (o’connor et al., 2021; prakash & das, 2020). moreover, aicsd, due to their unique characteristics, raises an array of new concerns, such as patient safety, lack of transparency, and trust manipulation (through deceptive use of anthropomorphic features) (bach et al., 2022; hengstler et al., 2016; miner et al., 2019; prakash & das, 2020; rai, 2019; scorici et al., 2022). consequently, existing trust models that work in the contexts of non-ai systems/generic information technologies (it) may not be relevant to ai-based systems, given these unique characteristics (lockey et al., 2021). additionally, the context adds another layer of complexity as the ai chatbot usage in a non-health setting (e.g., retail, banking, etc.) cannot be compared to the unique healthcare context in which aicsd is used (o’connor et al., 2021). therefore, we must rethink and appropriately contextualize the existing trust models, which were developed for generic technologies and non-health settings, before using them to understand the complex nature of trust in ai-based technology in healthcare services (chatbot for selfdiagnosis). such contextualization may generate rich theoretical and practical insights (johns, 2006; hong et al., 2014). we, therefore, aim to develop a contextualized research model to understand the determinants of consumers’ trust in aicsd and its behavioral consequences. in line with this purpose, we propose the following research questions: (1) what factors determine consumers' trust in aicsd? (2) how does consumers’ trust in aicsd impact the behavioral intentions to use it? to this end, a contextualized model was formulated by integrating relevant contextual factors australasian journal of information systems prakash & das 2024, vol 28, research article (why) do we trust ai? 3 into the trust in technology framework (mcknight et al., 2011). the model was tested empirically using a free simulation experiment (fromkin & streufert, 1976) in india. the findings of this study indicate that perceived anthropomorphism, perceived information quality, perceived explainability, disposition to trust technology, and perceived service quality predict trusting beliefs in aicsd and trusting beliefs, risk beliefs (related to privacy and health), in turn, determine intention to use. results also reveal the interplay between trust and risk perceptions wherein trust in aicsd contributes to assuaging the risk perceptions, thus indirectly helping to improve the intention to use. the study contributes to the theory and practice by developing and validating a contextual research model that captures the complexity of the novel technology and the setting in which it is deployed (healthcare). the insights from the study will aid designers and developers in creating trustworthy and purpose-fit aicsd, which could lead to a greater return on investment for organizations. 2 literature review 2.1 prior research on ai-based health chatbots chatbots are software programs that can converse with humans via text or voice via interactive interfaces (mctear et al., 2016). technology has improved considerably in the more than six decades since the invention of the first chatbot, eliza, from being able to recognize a few keywords to having real-time voice conversations with people. much of this progress can be attributed to the development of advanced machine learning (ml) algorithms and computational linguistics (shah et al., 2016). new-generation chatbots are being widely deployed across various industries to automate interaction with customers (følstad & brandtzæg, 2017). in the domain of healthcare services, ai-based chatbots are poised to play a significant role in supporting/complementing human service providers. there is an increasing body of research evidence favoring the effectiveness of the use of chatbots for supporting behavioral changes, mental health therapy, support for the elderly, etc. (laranjo et al., 2018). ai-powered chatbots that can check symptoms and triage patients (wang & siau, 2018) with their dynamic learning capability and natural language processing (nlp), represent a huge leap from the earlier generation of symptom checkers, which simply used programmed decision trees to match symptoms with diseases (razzaki et al., 2018; wang & siau, 2018). despite the immense value, the literature on aicsd is scant (laumer et al., 2019; o’connor et al., 2021; prakash & das, 2020). previous research on aicsd has focused on issues such as design and development (minutolo et al., 2018), estimation of efficacy (razzaki et al., 2018), and patient risk (bickmore et al., 2018); research on consumer/patient perspectives on the use is scant. the table below (table 1) summarizes the available literature on the use of aicsd. study objective theory methodology important findings razzaki et al. (2018) to validate the accuracy and safety of aicsd. nil experimental study aicsd was able to detect the medical condition presented by a clinical scenario with accuracy similar to human doctors. bickmore et al. (2018) to ascertain the extent and nature of the harm that could be caused by using cas for medical information. nil scenario-based experiment identified failure modes of cas in the scenarios tested. found that subjects were led to take actions that could have caused harm (12.4%) or death (6.9%). australasian journal of information systems prakash & das 2024, vol 28, research article (why) do we trust ai? 4 wang & siau (2018) to formulate a theory on trust in health chatbots. keeney’s valuefocused thinking conceptual reviewed factors affecting trustbuilding in health chatbots. proposed a study to formulate a theory on trust in health chatbots. laumer et al. (2019) to develop a conceptual model that explains consumer adoption of aicsd. utaut2 qualitative – semi-structured interviews (n=35) developed a research model that explains the adoption of cas for disease diagnosis by extending the utaut2 model with factors such as privacy risk, trust in provider and system, compatibility, experience in ediagnosis, and access to the health systems. nadarzynski et al. (2019) to explore the participant’s willingness to engage with an ai-based health chatbot. tfa mixed-method: interviews (n=29), survey (n=216), binary regression. analysis of the interviews revealed user concerns about cyber-security, the accuracy of the bot, and the inability of aibased chatbots to empathize. perceived it skills, dislike for talking to computers, perceived utility, attitude, and perceived trustworthiness were correlated with acceptability. prakash & das (2020) to explain consumer's trust in aicsd. ttm, risk theory, srt quantitative – survey (n= 107), sem social presence, usefulness, safety risk, and propensity to trust predicted trust in aicsd. the effect of ease of use, privacy risk, and third-party endorsement was insignificant. trusting beliefs determined trusting intention. mesbah & pumplun (2020) to investigate the factors that influence seniors' adoption of health chatbots. utaut2 qualitative – semi-structured interviews (n=21) proposed an extended utaut2 model with additional factors such as technology anxiety, privacy risk, trust, resistance to change, etc., to explain the acceptance of health chatbots by seniors. fan et al. (2021) to understand the usage of health chatbots in the actual world, as well as the difficulties and limitations of their usage. nil case study – data-driven approach in analyzing the system log 47,684 consultation sessions over six months users consulted on a wide variety of issues, especially those that entail privacy and social stigma. a significant percentage of users left their consultation in the middle. identified user concerns such as insufficient actionable information and inaccurate diagnostic suggestions. seitz et al. (2021) to understand the difference between trust and distrust toward diagnosis, chatbots ttm qualitative – interviews, brainstorm sessions, thinkaloud, and participatory observation (n=8) findings reveal that distrust in chatbots arises affectively, whereas trust arises cognitively. australasian journal of information systems prakash & das 2024, vol 28, research article (why) do we trust ai? 5 o'connor et al. (2021) to develop a theoretical framework that explains patients’ trusting intentions towards robots in healthcare ttm conceptual proposes a conceptual framework based on trust in technology model for investigating trusting intentions towards intelligent agents in healthcare. table 1. prior research on the use of aicsd note. utaut2, unified theory of acceptance and use of technology; ttm, trust in technology model; srt, social response theory; tfa, theoretical framework of acceptability of healthcare interventions; sem, structural equations modeling. the current literature on consumer/patient perspectives majorly focuses on acceptance/ adoption (laumer et al., 2019; mesbah & pumplun, 2020; nadarzynski et al., 2019) and the actual use of aicsd (fan et al., 2021). it is observed that the issue of trust in aicsd (o’connor et al., 2021; prakash & das, 2020; seitz et al., 2021; wang & siau, 2018) is an emerging area of interest. in terms of the methodologies used, most of these studies are conceptual or qualitative in nature (laumer et al., 2019; mesbah & pumplun, 2020; o’connor et al., 2021; seitz et al., 2021; wang & siau, 2018) with the exceptions of prakash and das (2020) (quantitative), nadarzynski et al. (2019) (mixed-methods) and fan et al. (2021) (data-driven approach). with respect to the theoretical foundations of the studies, technology acceptance theories (utaut2 and tfa) and trust in technology model (ttm) are most frequently used. we make a few important observations from our analysis of the available literature. first, despite the agreement that customer trust is critical for the successful adoption of aicsd (laumer et al., 2019; mesbah & pumplun, 2020; nadarzynski et al., 2019), research on this topic is not adequately represented in the literature. the first attempt in this direction was by wang and siau (2018), which proposed developing a trust framework but provided no empirical findings. the only empirical study that attempted to study the determinants of trust in aicsd was by prakash and das (2020). although this study provides some preliminary insights into the consumer’s trust in aicsd, it just re-applies well-known theories in a new context but fails to consider the unique context-specific ai-specific factors that might influence trust. another recent study (seitz et al., 2021), which explored the difference between trust and distrust towards diagnosis chatbots using a qualitative approach, found that distrust has affective roots, whereas trust arises cognitively. a more recent study on trust in healthcare robots (o’connor et al. 2021) points out that healthcare possesses a unique context compared with its counterparts (e.g., manufacturing, retail) and argues for the need to explore contextual factors influencing patients’ trusting intentions towards robots in healthcare. second, we argue that prior research on similar technologies like healthcare it (song & zahedi, 2007; xie et al., 2020) may not adequately explain the trust in and use of the novel ai-based chatbots for self-diagnosis. this is primarily due to its unique characteristics autonomous nature, anthropomorphic features, self-learning ability, superhuman accuracy levels, inherent bias, and the non-transparent nature of the algorithms (hengstler et al., 2016; o’connor et al., 2021; rai, 2019). further, ai-based chatbots are an exception among software tools due to their position as highly responsive interaction partners. they exhibit specific social traits as compared to previous generations of information systems (is) since they simulate human intelligence and are capable of independent decision-making (glikson & woolley, 2020). they australasian journal of information systems prakash & das 2024, vol 28, research article (why) do we trust ai? 6 are also anthropomorphized deliberately to make the interaction more natural (feine et al., 2019). third, these unique attributes of ai-based systems raise a plethora of new concerns, such as patient safety (prakash & das, 2020; miner et al., 2019), lack of transparency (rai, 2019), and concerns of trust manipulation through deceptive use of anthropomorphic features (bach et al., 2022; scorici et al., 2022). the existing models of trust in technology (mcknight et al., 2011) or healthcare it (song & zahedi, 2007; xie et al., 2020) or recent ones on ai-based chatbots/personal assistant (zierau et al., 2020) does not adequately capture the intricacies in the context of ai-based chatbot applied in a self-diagnosis context (o’connor et al. 2021). hence, we strongly believe that the existing models on trust are ineffective in capturing the phenomenon of trust in aicsd in its full essence. therefore, we must rethink and contextualize existing theoretical models on trust in technology, which were initially developed for non-ai-based technologies, before we can use them to understand the trust formation in aicsd and its behavioral consequences. such contextualization can lead to rich theoretical and practical insights (johns, 2006; hong et al., 2014). 2.2 theoretical background from a thorough review of the literature on trust, we believe the framework of trust in technology (ttm) (mcknight et al. (2011) and lankton et al. (2015) could be a useful model in explaining the complex phenomena of trust in aicsd. trust generally represents “the willingness of a party to be vulnerable to the actions of another party based on the expectation that the other party will perform a particular action important to the trustor, irrespective of the ability to monitor or control the party” (mayer et al., 1995, p.712). prior research has established that trust is central to understanding human behavior in diverse domains ranging from workgroup dynamics to commercial relations (mcknight et al., 2011). trust was studied originally in the context of interpersonal relationships (mayer et al., 1995); later, it was extended to explain the human relationship with nonhuman entities (mcknight et al., 2011). accordingly, trust (or, more accurately, trusting beliefs) in technology is defined as “the beliefs that a specific technology has the attributes necessary to perform as expected in a given situation in which negative consequences are possible” (mcknight et al., 2011, p.7). the original model operationalizes the trust in technology constructs consisting of three sets of concepts: (a) disposition to trust general technology, (b) institution-based trust in technology, and (c) trust in a specific technology. disposition to trust captures person-specific characteristics that can influence trust formation in a specific technology (mcknight et al., 2011). institution-based trust focuses on the belief that success is likely due to the presence of supportive situations and structures tied to a specific context or a class of trustees (e.g., structures and guarantees, such as third-party certifications/seals and privacy assurances). unlike disposition to trust, institution-based trust beliefs are highly situation-specific. furthermore, trust in technology shapes both users’ attitudes and behavior toward the target technology (lankton et al., 2015). it is an important psychological step that enables the users to rule out the probability of unintended consequences of using the technology and increase the intention towards using it (gefen et al., 2003a). additionally, to adequately capture the specificity of the context, we did a thorough review of the literature on trust in ai-based technologies (bach et al., 2022; lockey, 2021) and closely related applications such as social robots (naneva et al., 2020), intelligent conversational agents (zierau et al., 2020) to understand the relevant factors that could impact the user’s trust. australasian journal of information systems prakash & das 2024, vol 28, research article (why) do we trust ai? 7 as per the social response theory (srt), humans are predisposed to see computers as “social actors” unconsciously, even when they are aware that they lack feelings, motives, or intentions (nass & moon, 2000). this is particularly true when technology exhibits anthropomorphic (human-like) characteristics such as verbal skills or physical appearances (troshani et al., 2020). it is widely agreed that anthropomorphism is one of the fundamental characteristics that separate ai systems from non-ai systems (troshani et al., 2020). anthropomorphism is defined as “the attribution of human-like characteristics, behaviors, and emotions to nonhuman agents” (adam et al., 2020). anthropomorphic design characteristics (social/emotional cues) of an agent trigger a sense of social presence (“awareness of the other person in the interaction” (short et al., 1976, p. 65)) leading to a sense of emotional closeness and/or social connectedness with the agent, this ultimately increases trust (adam et al., 2020; qiu & benbasat, 2009). consequently, humans tend to apply social rules to anthropomorphically designed systems during an interaction (nass & moon, 2000). however, it is also observed that even though human likeness improves user affinity when it increases beyond a certain point, it can trigger a sense of eeriness (ho & macdorman, 2017). moreover, anthropomorphic design cues can also create undue trust in the agents and, hence, could be used to manipulate the users into trusting untrustworthy systems (culley & madhavan, 2013). hence, consumers’ perception of anthropomorphism in the aicsd could play a substantial role in the formation of trust. similarly, another prominent concern related to the use of ai-based technologies in aiding decision-making is the non-transparent black-box nature of the algorithms (rai, 2020). according to the literature, the "black box" problem is one of the key barriers to promoting users' trust and acceptance of ai systems. decision-makers have difficulties comprehending how ai systems generate specific results (adadi & berrada, 2018), especially in healthcare settings (london, 2019). building trust is hence considered necessary to deal with complexity and ambiguity since people cannot fully comprehend the inner workings of non-transparent ai systems (rai, 2019). transparency into the inner processes of ai systems and the explainability of why a certain outcome was generated is essential to building trust (rai, 2019). hence, the perceived explainability of the system is a critical factor to be considered while modeling trust in aicsd (rai, 2019). further, as a recommender of health information, the perceived quality of both information, the system, and the overall service provided could influence trust formation. the is success model by delone and mclean (2003) linked these quality factors to system use, user satisfaction, and, ultimately, to is success. empirical studies that followed revealed that these quality factors were the fundamental drivers of system use, user satisfaction, and trust formation process in various systems, specifically the e-health systems. e.g. song and zahedi (2007) argued that for health infomediaries to become customers’ favorite information source, it is critical to foster trust in the quality of information they provide. the infomediary's interactivity and ease of use (key dimensions of system quality) were crucial to fostering loyalty and trust (song & zahedi, 2007). moreover, a recent qualitative study investigating the requirements for trust for ai systems identified system quality and service quality as important prerequisites for building trust (bedué & fritzsche, 2022). accordingly, we believe the perceived is quality (delone & mclean, 2003) could drive trusting beliefs in the aicsd. further, trust implies the presence of uncertainties/risk factors in the usage context (mcknight et al., 2011). the users typically make a leap of faith in spite of the presence of these australasian journal of information systems prakash & das 2024, vol 28, research article (why) do we trust ai? 8 uncertainties due to their confidence in the attributes of the specific technology (gefen et al., 2003b). from the literature, perceived risk generally denotes the probability of a loss as well as the perception or belief of unfavorable outcomes (cunningham, 1967). it is usually measured as a consolidation of risk beliefs (e.g., financial, performance, physical, psychological, social, etc.) (pavlou, 2003). perceived risk is critical in establishing interpersonal, social, and economic relationships (chiles & mcmackin, 1996; song & zahedi, 2007). according to existing research, a negative relationship often exists between perceived risk and “intention to transact” (pelaez et al., 2019). in the context of aicsd, two specific risk beliefs are pertinent: perceived privacy risk and health risk. in this case, the privacy risk perception stems from the possibility of losing privacy while using the bot for diagnosis, which typically requires disclosing private personal information (laumer et al., 2019). similarly, health risk refers to the user’s perceptions of negative health consequences if the information provided by the chatbot is erroneous. according to the risk calculus perspective, users engage in a mental calculus of risk against benefits while attempting to depend on (trust) or use new technology. from the literature, it becomes clear that in order to explain the intricacies of the phenomenon under consideration, i.e., trust in aicsd, the contextual factors that are linked to the phenomenon have to be weaved into the fabric of the generic theory of trust in technology which is ineffective in its own to capture the phenomenon in its entirety (prakash & das, 2020). given that studying the phenomenon of trust in aicsd and its consequences continues to be underexplored in the literature (prakash & das, 2020; o’connor et al., 2021), and without identifying the comprehensive set of interconnected factors that drive trust in such technologies, the adoption by users is likely to lag behind the rapid pace of technological advancement. hence, we took a contextualized approach to modeling trust in aicsd. 2.3 contextualization for trust in aicsd context is defined as “situational opportunities and constraints that affect the occurrence and meaning of organizational behavior as well as functional relationship between the variables” (johns, 2006, p. 386). according to whetton (2009), context is a set of factors around a phenomenon that exerts a direct or indirect impact. in information systems, the research context refers to the characteristics and usage context of the technology artifact (orlikowski & iacono, 2001). prior research has observed that although embracing contextualization may necessitate researchers to sacrifice parsimony and generalizability (hong et al., 2014), its impact on research outcomes can be powerful (johns, 2006). it is observed that contextualization helps develop a comprehensive understanding of the interaction between technology, people, and context, without which the findings of the study might be incomplete (johns, 2006). furthermore, contextualization can make the research appealing to practitioners as the results are more relevant in solving practical problems (breward et al., 2017; johns, 2006). hong et al. (2014) provide some useful guidelines for context-specific theorizing in is research. the paper outlines two major approaches for incorporating context into theory development. the first approach (single-context theory contextualization) typically starts with identifying some well-established general theories that are relevant to the domain of interest. then, it incorporates contextual factors as antecedents of the core constructs, moderators of the relationship, or decomposing the core constructs into contextual factors (hong et al., 2014). the identification of the context-specific factors can be based on past research on relevant australasian journal of information systems prakash & das 2024, vol 28, research article (why) do we trust ai? 9 technologies and/or an in-depth analysis of the technology under investigation using qualitative methods (hong et al., 2014). the second approach (cross-context theory replication) aims to replicate theoretical models in different contexts and consolidate findings into a context-contingent theory using meta-analysis. in this study, we use the first approach (hong et al., 2014), where the well-established trust in technology theory is taken as the general framework and refined by incorporating relevant contextual factors as antecedents to the core theory constructs. despite the growing interest in the use of ai-based tools for self-diagnosis, our literature review reveals that empirical studies concerning consumer trust in aicsd are limited both in terms of number and its examination of the antecedents that may influence the trust in aicsd (o’connor et al., 2021). as discussed in section 2.1, a study by prakash and das (2020) is one of the first attempts to explain the drivers of trust in aicsd. the study followed seitz et al. (2021), who explored the difference between trust and distrust towards aicsd without giving a comprehensive insight into the determinants. a more recent conceptual study (o’connor et al. 2021) argued for a more contextualized model that incorporates contextual factors (such as individual characteristics/personality traits, explanation competency, health-related beliefs, etc.) to explain the complex phenomenon of trust in robots in healthcare. while these articles offer some insights into the intricate understanding of consumer trust in aicsd, they do not explicitly integrate the relevant contextual factors that could impact trust in aicsd. to address this gap, we followed guidelines proposed by hong et al. (2014) and chose the ttm (mcknight et al., 2011) as the initial generic theoretical framework for explaining the trust in aicsd and its behavioral consequences. subsequently, we combed through the relevant studies in the field to identify the relevant contextual factors as the antecedents of the contextualized core constructs. the approach of identifying contextual factors by examining extant literature is supported by hong et al. (2014). we identified four sets of factors, namely perceived is quality, perceived anthropomorphism, risk beliefs, and individual characteristics, as the key contextual factors that could influence the core constructs in the general model through a thorough literature review on aicsd and related technologies. we followed the guidelines by hong et al. (2014) to decompose the higher-level risk beliefs into two specific risk perceptions, namely, health risk and privacy risk, which are relevant to aicsd usage. further, following the guidelines of hong et al. (2014), we also examine the interdependence between the salient characteristics of technology (e.g., trust signs), characteristics of users (age, gender, prior experience), users’ trusting beliefs, risk perceptions stemming from the sensitivity of usage context and behavioral outcomes (intention to use). the proposed theoretical model is explained in the following section. 3 hypotheses development the proposed theoretical framework (figure 1) illustrates the hypothesized relationships between the constructs. as indicated earlier, trusting beliefs in a specific technology (aicsd) is the main dependent variable in the model, which refers to the user's perception that the specific technology (aicsd) possesses the necessary attributes to function as planned in a setting where adverse consequences are likely (mcknight et al., 2011). intention to use aicsd was included as a behavioral outcome of trusting beliefs formed after the initial use. intention to use also captures the essence of the concept of trusting intentions, defined as the intention to engage in trust-related behaviors (mcknight et al., 2002; li et al., 2006, 2008). in the australasian journal of information systems prakash & das 2024, vol 28, research article (why) do we trust ai? 10 subsequent sections, the proposed hypotheses are rationalized with appropriate theory and supporting evidence from the related literature. figure 1. theoretical framework 3.1 trusting beliefs and behavioral intentions to use trusting beliefs (or trust) in technology refers to the user's perception that the specific technology possesses the necessary attributes to function as planned in a setting where adverse consequences are likely (mcknight et al., 2011). regardless of whether an objective technology characteristic exists, users' views about performance could vary based on their experience or the context in which it is used (mcknight et al., 2011). trusting beliefs are typically assessed using human-like trust beliefs or systems like trust beliefs depending on the degree of technology humanness (lankton et al., 2015). an ai-based chatbot qualifies as a human-like technology due to its anthropomorphic features. hence, human-like trusting beliefs could be used to measure trusting beliefs (lankton et al., 2015). competence is the notion that a person/entity possesses the abilities, competencies, and traits necessary to have an influence in the domain of interest. benevolence is the presumption that a person/entity will want to do good for the trustor for reasons other than self-centered benefits. finally, the belief that a person follows an appropriate set of standards is referred to as integrity (lankton et al., 2015; mcknight et al., 2002). the higher the perceptions along the competence dimension about the technological artifact, the higher will be the perception that it can perform as intended. similar higher benevolence will reduce the likelihood of opportunistic behavior from the part of the technology artifact (mcknight et al., 2002). finally, higher integrity implies that the aicsd sticks to the ethical guidelines. together, these would create confidence in using the system and hence promote usage intentions. we also know from the literature that trusting beliefs in technology drive behavioral intentions (lankton et al., 2015; li et al., 2008; mcknight et al., 2002, 2011). people who believe that the technology has the characteristics that make it trustworthy are more inclined to trust it and show intentions to rely on or use it (mcknight et al., 2011; lankton et al., 2015). hence, we suggest the following hypothesis, australasian journal of information systems prakash & das 2024, vol 28, research article (why) do we trust ai? 11 h1: trust (trusting beliefs) will positively influence the intention to use aicsd. 3.2 disposition to trust technology and trust disposition to trust refers to a tendency to trust other people (meyer et al., 1995; rotter, 1971). when applied to the technology context, disposition to trust is the extent to which the person displays a consistent tendency to be willing to depend on technology in general across a broad spectrum of situations and technologies (mcknight et al., 2011). according to the literature on trust, it is neither technology-specific nor situation-specific, unlike trusting beliefs and intentions, which are object-specific but cross-situational (mcknight & chervany, 2001; mcknight et al., 2011). this construct primarily derives from trait psychology, and it states that actions/behaviors are molded by specific childhood-derived characteristics that become more or less stable through time. (mcknight & chervany 2001). people may grow up with a disposition or develop because of their life experiences (mcknight & chervany 2001). dispositional trust will affect trusting beliefs (in specific technology) when the situation and technology are unfamiliar. thus, if the individual has a higher disposition to trust technology (dt) in general, in the case of novel technology in an unfamiliar situation, he or she is likely to rely on this dispositional propensity. as a result, he/she is likely to form favorable opinions and beliefs about a novel technology like aicsd. moreover, dt has been identified as a positive driver of trust (trusting beliefs) in various technology use settings (chi et al., 2021; mcknight et al., 2011; lankton et al., 2015). accordingly, we propose, h2: user’s disposition to trust technology is positively related to their trust (trusting beliefs) in aicsd. 3.3 trust signs and trust trust signs/cues are generally referred to as characteristics of an online service that users evaluate when determining the trustworthiness of the online service (hoffman et al., 2014; wang & benbasat, 2008). in the case of health infomediaries, these informational cues include self-regulating policies such as privacy and security declarations, as well as third-party seals that ensure the provider adheres to fundamental ethical standards while providing information on its website (song & zahedi, 2007). it could also include users' feedback/reviews about the services offered. hence, we define trust signs in this context as “users' perceptions about privacy assurance, third-party seals/certificates, and user reviews/ratings on the aicsd.” the effectiveness of trust cues/signs is based on “signaling theory” (bacharach & gambetta, 2001). the trust cue/signs activate previously engraved cognitive and affective associations in the mind, allowing for rapid information processing and decision-making (hoffmann et al., 2014; wells et al., 2011). there are several reasons why these trust signs could bolster trust. for example, endorsements from previous users showcase the previous use of the service and signal a social norm regarding the general acceptability to the users (pennington et al., 2003; rice, 2012). similarly, third-party seals/certificates could signal adherence to the ethical standards and best practices, which the user interprets as a decreased likelihood of opportunistic behavior on the part of the provider (hoffmann et al., 2014). thus, trust signs are elements designed to convince consumers that there is no risk in using the system and reaffirm their perceptions about the integrity of the system (pennington et al., 2003; song & zahedi, 2007). these cues could strengthen consumers' trusting beliefs in the bot's integrity, benevolence, and competence (hoffman et al., 2014; song & zahedi, 2007). it has been demonstrated earlier that trust signs/cues directly impact trusting beliefs in the case of the website used for online services (hoffmann et al., 2014; mcknight & chervany, 2001). hence, based on these arguments, we propose, australasian journal of information systems prakash & das 2024, vol 28, research article (why) do we trust ai? 12 h3: trust signs will positively influence trust (trusting beliefs) in aicsd. 3.4 perceived explainability and trust the deep learning approaches used by ai systems are typically inscrutable (rai, 2020). thus, the users of ai systems typically have little understanding of why a particular decision is being recommended by the system (zierau et al., 2020). this inscrutability may exacerbate the issue of trust, particularly in settings where the implications are significant (rai, 2020). machines are beneficial to the extent their actions can be expected to achieve the user’s objectives (russell, 2019). suppose no information is provided on why a particular recommendation is being suggested; then the user at that point has no logical reason to believe that the course of action suggested by the chatbot could be relevant and beneficial for him/her (the consequences may not be immediately apparent). the lack of visibility and uncertainty thereof is likely to undermine the user’s beliefs about the competence of the aicsd. the inscrutability and the resultant ambiguity about the functionality may also lead to the system being rejected by the users (rai, 2020; zierau et al., 2020). literature on trust in ai systems argues that over and above the information about the accuracy and performance of the ai system, providing an explanation about the system’s behavior is likely to make the recommendation more valuable/actionable and can thus boost the user’s trust in the ai system (rai, 2020). however, although users will be interested in knowing “why” a particular diagnosis is recommended, they are not likely to be interested in knowing “how” the complex ai algorithms arrived at the recommendation (rai, 2020). high degrees of transparency about how ai models work could be counterproductive as it can entail high attention costs, create information overload, and annoy end-users (rai, 2019). thus, a simple “why” explanation instead of a complex “how” is likely to suffice (rai, 2020). moreover, the explanation-driven trust stream of research in chatbots argues for the significance of explanation in the formation of trust in ai-powered chatbots (zierau et al., 2020). additionally, recent research on personalized ai recommendation systems has identified and validated the role of explainability in determining users’ trust (trusting beliefs) in the system (shin, 2020). hence, we propose, h4: perceived explainability will positively influence the users’ trust (trusting beliefs) in aicsd. 3.5 perceived information quality and trust the most significant aspect for users of health information is the quality of information (song & zahedi, 2007). perceived information quality in this context can be defined as the user’s cognitive beliefs about the usefulness, adequacy/completeness, accuracy, relevance, currency, and understandability of the information provided by the aicsd. it has already been established as a crucial factor in the trust-building process in online interactions (nicolaou & mcknight, 2006; song & zahedi, 2007). since perceived information quality involves the user’s perception of positive information attributes such as accuracy and completeness, it can impact the trustee's trusting belief–integrity (nicolaou & mcknight, 2006). the users may interpret the accuracy and completeness of information as a sign that the health chatbot is committed to providing comprehensive and accurate information which enhances the user’s beliefs about the integrity of the chatbot in question (nicolaou & mcknight, 2006; song & zahedi, 2007). similarly, accurate, understandable, and reliable information could imply that the source of information is competent (nicolaou & mcknight, 2006; song & zahedi, 2007). as a result, perceived information quality would have a positive relationship with the trusting belief– australasian journal of information systems prakash & das 2024, vol 28, research article (why) do we trust ai? 13 competence. finally, higher perceived information quality also implies that the information is relevant, useful, and reliable to the end-user; it suggests that the chatbot is concerned enough to offer the user helpful information (song & zahedi, 2007). therefore, perceived information quality should also influence the benevolence dimension of trusting beliefs. thus, all three dimensions of trust may be positively influenced by the perceived information quality of the aicsd. based on these arguments, we suggest, h5: perceived information quality is positively related to their trust (trusting beliefs) in aicsd. 3.6 perceived system quality and trust perceived system quality represents the features of an online system that is preferred in terms of usability, availability, responsiveness, and interactivity (delone & mclean, 2003). it reflects the user’s perceptions of the system's technical ability to provide easy and quick access to information (delone & mclean, 2003). usability or ease of use is a key construct that impacts consumer trust in the context of e-commerce (gefen et al., 2003a). suppose the users perceive difficulty in accessing the needed information. they are likely to conclude that the health chatbot and its designers lack the competence to create a user-friendly design (song & zahedi, 2007). similarly, a lack of responsiveness and interactivity could cause the users to conclude that the health chatbot lacks the benevolence to address their queries (song & zahedi, 2007). similarly, the feature that a health chatbot is always up and available signals the integrity and reliability of the system. in other words, the component dimensions of perceived system quality directly and positively influence the dimensions of trusting beliefs (i.e., benevolence, integrity/reliability, and competence). hence, the following hypothesis is suggested, h6: perceived system quality will positively influence their trust (trusting beliefs) in aicsd. 3.7 perceived service quality and trust perceived service quality is defined as the consumer's perceptions of “an entity’s overall excellence or superiority” (dagger et al., 2007, p. 124). in the current study context, we define perceived service quality as the user’s perception of the excellence of services provided by the aicsd. customers/users typically use five aspects of service, “tangibles,” “reliability,” “responsiveness,” “assurance,” and “empathy,” while evaluating the service quality (jiang et al., 2002; parasuraman et al., 1988). regarding its outcomes, service quality perceptions influence attitude and behavioral intention (cronin et al., 2000). in this case, if the users perceive the chatbot to be highly reliable, they are likely to interpret that the chatbot is high on the integrity aspect, i.e., they consistently deliver what was promised. similarly, assurance (the feeling of safety in transactions) adds to reliability, reinforcing the integrity dimension of trusting beliefs. empathy and care shown by the chatbot, along with the promptness of the response provided (responsiveness), could signal that the chatbot is kind, thoughtful, and considerate of the requests made by the user. the agent's courteous, caring, and responsive behavior will inspire confidence in customers, especially in a high-involvement professional service context (westinger, 1998). finally, having a visually attractive user interface could signal significant investments on the provider's part, which could positively impact the user’s perception of the entity's functionality (eisingerich & bell, 2008). thus, perceived service quality dimensions could positively influence the trusting beliefs in the aicsd. additionally, prior research has discovered that in the context of professional, high-credibility services (e.g., e-healthcare), consumers' uncertainty may be reduced by providing consistent quality services, which will eventually lead to trustworthy relationships between consumers australasian journal of information systems prakash & das 2024, vol 28, research article (why) do we trust ai? 14 and providers (akter et al., 2013; eisingerich & bell, 2008). overall, customers will be more trusting of an entity that routinely meets or exceeds their technical or core performance expectations (eisingerich & bell, 2008). furthermore, the link between perceived service quality and consumer trust has been confirmed by subsequent research (akter et al., 2013; chiou & droge, 2006; eisingerich & bell, 2008). based on these arguments, we propose, h7: perceived service quality is positively related to their trust (trusting beliefs) in aicsd. 3.8 perceived anthropomorphism and trust anthropomorphism is an inductive inference mechanism whereby individuals attribute distinct human characteristics to nonhuman entities, especially the ability for critical thinking (agency) and conscious feeling (epley et al., 2007). prior research has argued that the more technology appears to have human-like cognitive abilities, the more individuals will trust it to fulfill its intended purpose competently (epley et al., 2006; waytz et al., 2014). this prediction is based on the commonly understood connection between an individual’s perceptions of the mental states of others and competent action performed by them (waytz et al., 2014). when people see an agent perform a competent action, they tend to believe that there is an intelligent mind inside that is performing the action with awareness and foresight. attributing a humanlike mind to a nonhuman entity would make the agent appear more capable of managing its own actions and, thus, more able to execute its planned functions adeptly (waytz et al., 2014). thus, as the agent becomes more similar to a human, users are more prone to engage in the correspondence bias, attributing human motivations, reasoning powers, and capacities to this non-human machine (culley & madhavan, 2013). therefore, as the agent (aicsd) displays human-like cognitive abilities, users start attributing a human-like intelligent mind to it (perceived anthropomorphism) and consequently would feel more confident in the agent’s abilities (moussawi et al., 2020) to perform the medical diagnosis effectively (trust). thus, high perceived anthropomorphism would lead to high trust in aicsd. additionally, recent research in autonomous cars (waytz et al., 2014) and intelligent personal agents (hu et al., 2021; moussawi et al., 2020) have demonstrated a positive link between perceived anthropomorphism and users’ trust in technology. on this premise, we suggest the following hypothesis, h8: perceived anthropomorphism will positively influence trust (trusting beliefs) in aicsd. 3.9 trusting beliefs and risk beliefs in general, perceived risk refers to the likelihood of a loss as well as the perception or belief of unfavorable consequences (cunningham, 1967). in the specific context of aicsd use, the two specific risk beliefs that can emerge are related to loss of privacy (perceived privacy risk) and related to the health and safety of the person who follows the chatbot recommendations. the research agrees that whether it is trust in people or trust in technology, both involve risk (mcknight et al., 2011). when a person trusts a cloud-based service, such as dropbox, to safely store data, he or she exposes themselves to the risk and uncertainty associated with transferring data over the internet and storing sensitive data on a server. similarly, when an individual trusts an aicsd and decides to use it for self-diagnosis, he or she exposes themselves to the risk of their sensitive private data on their health and medical condition getting stolen and/or misused for target advertising (laumer et al., 2019; prakash & das, 2020). additionally, it exposes them to a risk of potential misdiagnosis (wang & siau, 2018) and the adverse health consequence resulting from it. in this way, risk, uncertainty, and lack of total australasian journal of information systems prakash & das 2024, vol 28, research article (why) do we trust ai? 15 user control are the inherent contextual conditions present in all trusting scenarios (mcknight et al., 2011). so when an individual trusts another individual or technology artifact, it is because they have confidence in the trustee to perform as intended under the conditions of risk (mcknight et al., 2011). that is, despite the risk, the person takes a leap of faith based on their assessment of the attributes of the technology (gefen et al., 2003b; holmes, 1991; nicolaou & mcknight, 2006). while research remains divided on whether trusting beliefs predict perceived risk or vice versa (koller 1988, pavlou & gefen 2004), the majority of evidence shows that trust impacts risk perceptions (gefen et al., 2003b; mcknight et al., 2017; nicolaou & mcknight, 2006). placing trust as a predictor of perceived risk is consistent with psychological descriptions of how trusting—as a leap of faith—provides a sense of security even when the outcomes are uncertain (gefen et al., 2003b; holmes, 1991; nicolaou & mcknight, 2006). accordingly, we propose the following hypotheses, h9a: trust (trusting beliefs) in aicsd will negatively influence perceived privacy risk. h9b: trust (trusting beliefs) in aicsd will negatively influence perceived health risk. 3.10 impact of trust signs on risk beliefs the trust signs include self-regulating policies such as statements about privacy security, third-party seals that guarantee that the health infomediary follows basic ethical codes (in recommending information), and reviews that reflect users' feedback. tan and theon (2003) noted that trustworthy cues are control mechanisms for reassuring online users. further, according to actor-network theory, trust signs are the inscription of the need to increase users' trust (song & zahedi, 2007). trust signs are part of the input for assessing the risk connected with the trust context as well as the integrity of the health infomediary in the chain of trust formation (song & zahedi, 2007). this point of view is strengthened by "signaling theory," which divides trust-related features into observable and non-observable categories (bacharach & gambetta, 2001). information quality could be viewed as an observable property in the context of a health information provider. in contrast, trust signs are non-observable properties that are meant to reassure web users that there is no risk involved in using the health infomediary and reinforce their beliefs in its integrity (pennington et al., 2003). thus, trust signs are subtle measures intended to alleviate concerns associated with obtaining and using information given by aicsd. trust cues signal to users that aicsd follows certain ethical guidelines in recommending information and adheres to values and standards of conduct. therefore, users are likely to believe that using aicsd is less likely to cause harm to them. an earlier study conducted in the context of health infomediaries has confirmed the role of trust cues in toning down the risk perceptions of the user (song & zahedi, 2007). accordingly, we believe user perceptions about trust signs will reduce privacy and health risk perceptions associated with the use of aicsd. hence, we propose the following hypotheses, h10a: trust signs will negatively influence perceived privacy risk. h10b: trust signs will negatively influence perceived health risk. australasian journal of information systems prakash & das 2024, vol 28, research article (why) do we trust ai? 16 3.11 risk beliefs and behavioral intentions to use 3.11.1 perceived privacy risk and behavioral intentions to use dinev and hart (2006) defined perceived privacy risk as "the degree to which people believe that there is a potential loss involved with the disclosure of personal information." in this case, it refers to the perceived risk associated with disclosing sensitive private information on their health and medical condition while using the services provided by the chatbot. the uncertainty lies in the possibility of private information being sold or used for targeted advertising. given the risk, consumers will weigh risk against benefits while considering using the services, and this "privacy calculus" will shape their attitude and behavior towards using the service (dinev & hart, 2006). so, if the consumers perceived that the benefits derived from using the diagnostic services of aicsd are outweighed by the risk associated with the loss of valuable personal information, they would refrain from using it. furthermore, in various similar settings, empirical evidence from prior research has revealed a negative association between perceived privacy risk and propensity to use technology. for example, wearables (adebesin & mwalugha, 2020; li et al., 2016), nutrition recommender systems (berezowska et al., 2015; wendel et al., 2013), health apps (zhang et al., 2019; zhao et al., 2018). based on the evidence mentioned above, we suggest the following hypothesis, h11a: perceived privacy risk will negatively influence intention to use aicsd. 3.11.2 perceived health risk and behavioral intentions to use relying on the internet for health advice and self-diagnosis poses severe risks to the lives of patients (robertson et al., 2014). using fully autonomous chatbots for health advice can potentially aggravate these risks (prakash & das, 2020). following the advice of chatbots which has unpredictable performance and that too without any further review by health professionals, poses a severe health risk to the patient (wang & siau, 2018). many risk scenarios are plausible; for example, a misdiagnosis by the bot could result in 1) selfmedication by patients that could cause adverse health consequences or 2) unnecessary anxiety among the patients about a disease that they do not have (bickmore et al., 2018). there are also chances that patients may delay seeking professional care following the advice of the chatbot, which could have dire health consequences (bickmore et al., 2018). according to the risk calculus perspective, consumers are highly likely to weigh these risks against benefits while making technology use decisions (cocosila et al., 2007; cocosila & turel, 2016). if the perceived risk of adverse health consequences resulting from the use is high and exceeds the benefits of using aicsd, users’ willingness to depend on the aicsd will be low, and hence, they may refrain from using it. thus, we posit the following hypothesis, h11b: perceived health risk will negatively influence intention to use aicsd. 4 research methodology 4.1 study design, procedure, and participants we used a “free simulation experiment” (fse) (fromkin & streufert, 1976; söllner et al., 2016) to test our theoretical model. unlike the traditional lab experiment, which relies on treatments to change one or more predictor variables, fse exposes subjects to a range of real-world conditions, such as performing a specific set of activities within a predetermined time frame (fromkin & streufert, 1976). the technique ensures that the subject not only completes the assigned activities but also organically explores the system to generate relevant impressions australasian journal of information systems prakash & das 2024, vol 28, research article (why) do we trust ai? 17 before answering the associated questions (söllner et al., 2016). furthermore, unlike in a field environment, this form of experiment allows us to control for various aspects, such as ruling out effects produced by different mobile devices or familiarity with an existing system (which would be difficult for assessing initial trust). a significant number of participants (see appendix 1) were regular students enrolled in mba courses at two business schools in india. apart from students, a small group of non-student participants (n = 25) were selected based on convenience sampling. regarding demographics, the participants are tech-savvy and likely to seek health information online. a large share of the participants (85.64 percent) declared that they have previously accessed the internet to obtain health information. it was also confirmed that none of them had previously used an aicsd (although 78.71% had previously used automated ai-based chatbots for other uses, such as e-commerce). the experiment was conducted in different batches via online meetings, and all the participants had an internet-connected computer. we selected a popular web-based aicsd, namely, your.md (recently rebranded as healthily) (ćirković, 2020; your.md, 2021) for the experiment (see figure 2). london-based your.md (healthily) is a chatbot that provides customers with personalized and trustworthy healthcare assistance and information (your.md, 2021). it is a ce-accredited medical device that is registered with the “medicines and healthcare products regulatory agency” (mhra) as a class i medical device (your.md, 2021). it is available on multiple platforms, including the website and smartphone apps for android and ios. it currently has a user rating of 4.1/5 from 14,055 users on the google play store and a rating of 4.4/5 from 907 users on the app store. its self-assessment feature allows users to ask free text questions using its chatbot. it then asks a series of questions to compile an "assessment report," which contains a list of likely conditions based on reported symptoms (with details such as reasons for suggesting the particular condition, symptoms, treatment methods etc.), the seriousness of likely conditions (e.g., red – emergency condition, orange – urgent condition, blue – routine condition, and green – no need to see a health professional) and recommendation on next steps to be taken (ćirković, 2020; your.md, 2021). the participants received a 10-minute briefing on the health chatbot about its functionality, features, and usage. subsequently, they were assigned two predetermined activities that addressed the core functionality of the chatbot under evaluation. activity 1: participants had to log on to the chatbot website and spend 10 minutes exploring the website and reading reviews on the play store. activity 2: participants were required to look for symptoms of (1) an illness they experienced recently and (2) a recent illness a friend/relative experienced (15 mins). after this, participants were asked to complete an online questionnaire based on their experience. informed consent was obtained from all the participants, and participation in the experiment was entirely voluntary. in all, 254 people took part in the experiment, and 202 valid responses were received. the demographic profile of the participants is given in appendix 1. australasian journal of information systems prakash & das 2024, vol 28, research article (why) do we trust ai? 18 figure 2. the user interface of the your.md chatbot (source: your.md (2021) australasian journal of information systems prakash & das 2024, vol 28, research article (why) do we trust ai? 19 4.2 survey instrument we collected the data using a structured questionnaire. all the items/indicators in the questionnaire were adapted versions of preexisting scales. except for the constructs perceived anthropomorphism and perceived health risk, all indicators were measured on a seven-point likert scale (1 = “strongly disagree” to 7 = “strongly agree”). for measuring the construct perceived anthropomorphism, we used a popular five-point semantic differential scale (bartneck et al., 2009; sheehan et al., 2020). a seven-point likert scale ranging from 1 = “extremely low” to 7 = “extremely high” was used to measure perceived health risk. the measures and their sources are given in appendix 2. in addition to the main constructs, we used certain control variables to control for the possible variation in the outcome variable due to variations in the respondent’s characteristics. the control variables are age, gender, and prior experience with automated chatbots (general) (see appendix 2). 5 results 5.1 structural equations modeling analysis we employed the partial least square structural equation modeling (pls-sem) method to test the propositions. pls-sem is a “prediction-oriented” method, making it an appropriate choice for the current investigation (hair et al., 2016). furthermore, it performs effectively with small data sets and even in the case of non-normal data (hair et al., 2016). hence, this method was found to be appropriate for this study. 5.1.1 measurement model assessment following the rules set by hair et al. (2019), the validity and reliability of the empirical model were checked. first, the indicator loadings and composite reliability (cr) of constructs were used to assess the scale reliability of the constructs in the framework. as per the guidelines, indicator loadings must be greater than 0.708 to meet the acceptable item reliability (hair et al., 2019). all the items, apart from sq1 (0.553), had an indicator loading greater than the cutoff value. the item sq1 was hence removed from further analysis (hair et al., 2019). the cr values of the constructs were examined to determine the internal consistency reliability of the constructs. as shown in appendix 3, all the cr values were greater than the cut-off value of 0.7 (hair et al., 2019), thus confirming the internal consistency reliability. after this, the construct validity was assessed by evaluating the convergent and discriminant validity of the constructs. convergent validity was proved when all of the extracted average variance (ave) was larger than the cut-off of 0.5 (hair et al., 2019). we used htmt criteria to assess the discriminant validity (hair et al., 2019). all relevant htmt values (see appendix 4) were observed to be less than the 0.85 criterion, confirming discriminant validity (hair et al., 2019). thus, based on the above results, we can infer that our constructs demonstrated adequate internal consistency, indicator reliability, and convergent and discriminant validity. 5.1.2 assessing potential common method bias in research employing a cross-sectional survey design, common method bias (cmb) might be an issue (podsakoff et al., 2003). we used some ex-ante measures recommended by podsakoff et al. (2012) to reduce the cmb. first, the measures of the predicted and predictor variables were proximally separated (using sections in the online survey form). second, a panel consisting of a professor from the is domain and two senior doctoral students reviewed the scale items for ambiguous terms, complexity, and other discrepancies. third, different scale australasian journal of information systems prakash & das 2024, vol 28, research article (why) do we trust ai? 20 types (likert scale, semantic differential scale) were used to measure constructs to eliminate common scale properties. we used two distinct ways to conduct post hoc statistical analysis to determine the degree of cmb. first, harman’s single-factor test (podsakoff et al., 2003) revealed that no single factor explained most of the variance. the largest single factor that emerged from the test accounted for 37.832% of the variance, which is less than the cut-off value of 50% (podsakoff et al., 2003). second, we applied an advanced method known as the "marker variable" method (lindell & whitney 2001) by including a conceptually irrelevant marker variable in the research model (lindell &whitney 2001). fashion consciousness (malhotra et al., 2006), a popular marker variable theoretically unrelated to the constructs, was chosen for this study. the shared variance of the marker variable with other constructs in the model was found to be very low. hence, it was concluded that no substantial cmb exists in the data (johnson et al., 2011). 5.1.3 structural model assessment the latent constructs' variance inflation factor (vif) values were used to examine the model for multi-collinearity issues (hair et al., 2019). the vif values of all constructs were observed to be lower than the cut-off level of 3.3, indicating that there were no multi-collinearity issues in the data (hair et al., 2019). the vif values are shown in appendix 5. the validity and significance of the path coefficients in the empirical model were assessed using the pls bootstrapping method with 5,000 subsamples (hair et al., 2016). table 2 and figure 3 summarize the findings of path analysis. first, the influence of the control variables on the dependent variable intention to use (iu) was examined. the results suggest that none of the control variables except gender (β = -0.176, p < 0.05) had a statistically significant influence on the outcome variable iu (see table 2). all proposed hypotheses, except h3, h6, h10a, and h10b, were found significant at p < 0.05. the factors, namely perceived anthropomorphism, perceived information quality, perceived explainability, disposition to trust technology, and perceived service quality (in the order of the magnitude of their path coefficients), were found as statistically significant predictors of trusting beliefs supporting the hypotheses h8, h5, h4, h2, and h7 respectively. however, the data did not support the effects of factors, namely perceived system quality (h6) and trust signs (h3), on trusting beliefs. further, the impact (negative) of trusting beliefs on perceived health risk (h9b) and perceived privacy risk (h9a) were significant. the impact of trusting beliefs on perceived health risk (β = -0.697, p < 0.05) was higher than that on perceived privacy risk (β = -0.455, p < 0.05). however, the impact of trust signs on perceived privacy risk (h10a) and perceived health risk (h10b) turned out to be statistically insignificant. similarly, trusting beliefs, perceived privacy risk and perceived health risk were statistically significant predictors of intention to use aicsd, supporting hypotheses h1, h11a, and h11b. hypothesis path β t statistics p-value inference h1 tb → iu 0.578 8.510 0.000 supported h2 dt → tb 0.153 3.553 0.000 supported h3 ts → tb 0.050 1.070 0.285 not supported h4 ex → tb 0.226 3.523 0.000 supported h5 iq → tb 0.236 3.476 0.001 supported h6 sy → tb -0.091 1.384 0.166 not supported h7 sq → tb 0.150 1.996 0.046 supported australasian journal of information systems prakash & das 2024, vol 28, research article (why) do we trust ai? 21 h8 pa → tb 0.319 6.521 0.000 supported h9a tb → pr -0.455 6.479 0.000 supported h9b tb → hr -0.697 14.154 0.000 supported h10a ts → pr 0.106 1.278 0.201 not supported h10b ts → hr 0.069 1.048 0.295 not supported h11a pr → iu -0.194 3.503 0.000 supported h11b hr → iu -0.168 2.583 0.010 supported control variables path β t statistics p-value inference age age → iu -0.060 1.064 0.287 not significant gender gender → iu -0.176 2.005 0.045 significant experience exp → iu 0.007 0.080 0.937 not significant table 2. path analysis note. β, path coefficient; t, two-tailed t-test values; p-value, the significance level. figure 3. structural model further, the r2 values of the endogenous constructs were checked to ascertain the explanatory power of the model. the r2 values are reported in figure 3. the r2 values of the constructs reveal the percentage of variance explained by its predictors in the model (hair et al., 2019). for example, the r2 values of the outcome variable intention to use (r2 = 0.667) indicate that approximately 66.7% of its variance is explained by the model. similarly, our model explains 69.2% of the variance of the key construct trusting beliefs. furthermore, the “predictive sample reuse technique” (stone-geisser's q2) was used to examine the predictive relevance of the model (hair et al., 2016). it displays how successfully empirically collected data can be recreated using the model and pls parameters (hair et al., 2019). the q2 values were obtained through a “blindfolding procedure” by keeping the omission distance at 7 (hair et al. 2019). all of the endogenous constructs had q2 values larger australasian journal of information systems prakash & das 2024, vol 28, research article (why) do we trust ai? 22 than zero, indicating substantial predictive relevance for the empirical model (hair et al., 2016). 6 discussion the purpose of this research was to develop a contextualized research model to understand the determinants of consumers’ trust in aicsd and its behavioral consequences. we extended the ttm with relevant contextual factors as per the guidelines proposed by hong et al. (2014). the results supported all the hypotheses except h3, h6, and h10a. the discussions in this section synthesize the results of our study with the existing literature by rationalizing the similarities and differences with the literature. perceived anthropomorphism has the most prominent (positive) impact among the antecedents of trusting beliefs. this finding lends credence to the theorized link between assessments of others' mental capacities and assessments of competence, trust, and accountability (waytz et al., 2014). it also provides evidence for the proposition made by troshani et al. (2020) regarding anthropomorphism's role in forming trust in ai-based systems. the prominence of anthropomorphism among the other determinants also points to the argument that anthropomorphism could lead to undue trust in hci agents (culley & madhavan, 2013). that is, the feelings and perceptions about an anthropomorphic agent may be used in the construction of a mental model of the system, which may result in incorrect calibrations of trust based on an emotional connection with the anthropomorphic agent rather than actual system performance (culley & madhavan, 2013). prior research has also argued that incorporating human characteristics may have other unintended consequences; for example, using apologies to boost perceptions of a machine's humanness may create unreasonable expectations of changed behavior. if these expectations are not met, the agent may be perceived as deceptive or lacking in integrity, which can be extremely damaging to trust (de visser et al., 2016). further, past research has reported concerns that overanthropomorphism may lead to an overestimation of the ai's capabilities, thus placing the stakeholder at risk (culley & madhavan, 2013), as well as leading to a host of ethical and psychological issues, such as manipulation (salles et al., 2020). consequently, anthropomorphism may not always lead to higher trust (chui et al., 2019; moussawi, 2021), although our study and some other studies in different contexts (verberne et al., 2015; waytz et al., 2014) report a positive impact. the second most prominent determinant of trusting beliefs is perceived information quality. this finding is very much in line with the propositions made by the earlier studies in e-health services (sillence et al., 2007; song & zahedi, 2007). our results thus support and validate the argument that users may regard a health infomediary's high-quality information as a proxy for its goal alignment and appropriate actions as the online users' agent (song & zahedi, 2007). here, such a sense of the agent’s goal alignment could result in higher trust in the agent on the user's part. as is the case with the literature on interpersonal trust, where people trust a speaker who provides truthful or credible information (giffin 1967), our study argues that users are likely to trust an aicsd that provides current, accurate, and reliable information. perceived explainability is the third most significant predictor of trusting beliefs. we observed that higher perceived explainability leads to higher trust. our research adds to the specific stream of explanation-driven trust in chatbots (zierau et al., 2020) and to the broader ongoing investigations on whether ai explanations impact people's trust in ai systems. while some australasian journal of information systems prakash & das 2024, vol 28, research article (why) do we trust ai? 23 studies have found that user interfaces that offer explanations are effective at increasing users' trust in ai systems (mcguinness et al., 2006; pu & chen, 2006), other studies have found contradictory results—providing explanations may not increase satisfaction or even erode users' trust in a system (cramer et al., 2008; kizilcec, 2016; zang et al., 2021). among the very few studies reported in the context of patient-facing ai systems, the positive impact of explainability on the trusting beliefs observed by our research is in stark contrast, for example, to the result reported by zang et al. (2021). as expected, the user-specific factor, i.e., disposition to trust technology, registered a significant positive impact as in the original ttm (lankton et al., 2015; mcknight et al., 2011). users' disposition to trust could be significant in the early usage stages, as mcknight et al. (2011) observed. finally, the positive effect of perceived service quality was also supported, like in the case of m-health services (akter et al., 2013). consumers gradually build trust by evaluating many explicit and implicit indicators about the service. among those signs, perceived service quality represents evaluations of direct experience (chiou & droge, 2006). if it is perceived favorably, adverse selection and moral hazard problems will be addressed, and customers will have greater confidence in the system; this, in turn, will strengthen their trust in the system. nevertheless, we note that the role of perceived service quality in determining trust has rarely been tested in the context of ai-delivered services. however, the insignificance of the influence of perceived system quality and trust signs on trusting beliefs is noteworthy. the finding related to perceived system quality deviates from the earlier research on health infomediaries (song & zahedi, 2007). the possible reason could be that the dimensions of perceived system quality, such as usability, access, and availability of the system, did not appear critical to the tech-savvy participants in this study. the users possibly considered these aspects a hygiene factor rather than critical antecedents to trusting beliefs. similarly, trust signs, i.e., cues about assurance of performance and privacy, thirdparty endorsements/certifications, and customer reviews, do not seem to influence trusting beliefs as in earlier research on health infomediaries (song & zahedi, 2007). the likely explanation could be that these trust signals may aid in developing an initial trust before the consumer uses the product/service. however, once they use it, trust (trusting beliefs) is shaped by their actual experience of using it rather than these cues. an earlier study (mcknight et al., 2004), which observed similar results in the context of e-commerce, remarked that because of the uncertainties surrounding the web, third-party seals are insufficient to nurture trust; rather, 'seeing (or interacting) is believing.' similar conclusions could be made in this case as well. further, the dampening effect of trust signs on privacy or health risk perceptions was not observed as envisaged. there is a growing notion among internet users that trying to protect privacy is a pointless endeavor (xie et al., 2019), given the stories about data breaches/misuse in cyber settings. adding to this, though the companies often display privacy seals and declare privacy policies, many of them secretly and deliberately violate these policies by sharing personal identifiers with third parties without declaring it (brandtzaeg et al., 2019; okoyomon et al., 2019; yu et al., 2016). this growing fatalistic view of privacy (xie et al., 2019), coupled with secret violations of the privacy policies, could be a reason that trust cues/signs were unable to produce a significant negative impact on privacy risk perceptions. likewise, the trust signs also did not assuage the perception of health risks. thus, having trust cues such as a third-party seal of approval, a significant amount of positive customer ratings, and privacy australasian journal of information systems prakash & das 2024, vol 28, research article (why) do we trust ai? 24 assurance was not seen to help alleviate the fear of negative health consequences of using aicsd. this result supports the observations by a previous study on health infomediaries that trust signs were insufficient on their own to reduce the users' risk perceptions (song & zahedi, 2007). however, even though the trust cues/signs had no influence on the user’s perception of trust or on their risk perceptions, our study makes an important observation that trust (trusting beliefs) in the aicsd formed after the initial use will reduce both the privacy risk perceptions and health risk perceptions. this supports the prevailing arguments and evidence in the literature on how trust reduces uncertainty and provides a sense of assurance when outcomes are unclear (gefen et al., 2003b; jarvenpaa et al., 2000; nicolaou and mcknight, 2006; pavlou & gefen, 2004). when these results are read together, it implies that after the initial experience, the trust is dependent heavily on a person's individual experience with the technology, and cues such as a third-party seal of approval, positive customer ratings, and privacy assurance will not matter. according to the classification in signaling theory (cook, 2001), the trust cues/signs will fall under the category of non-observable properties. these non-observable properties, even though effective in forming initial trust (similar to trust in unknown parties from the interpersonal trust literature), will not be relevant after the individual has had the initial experience of using the technology (mcknight et al., 2011). the knowledge-based trust shaped by the user experience will determine behavioral consequences (siau & wang, 2018). here, the trust in aicsd is formed from an individual’s perceptions about anthropomorphism, information quality, explainability, and service quality, which are derived based on his/her use experience and his/her predisposition to trust technology in general. with respect to the antecedents of behavioral intention to use, trusting beliefs in aicsd were found to be the strongest predictor (positive) supporting the original hypothesis in the ttm (lankton et al., 2015; mcknight et al., 2011). similarly, the risk perception perceived privacy risk shows a significant effect on the intention to use, supporting the extant literature on various e-health services (berezowska et al., 2015; wendel et al., 2013; zhang et al., 2019; zhao et al., 2018). likewise, health risk perception was also identified as a significant barrier, validating a similar claim made by (prakash & das, 2020). finally, among the control variables, only gender (β = -0.176, p < 0.05) was significantly associated with intentions to use. the effect of age and prior experience of using automated chatbots (general) turned out to be insignificant. as per the findings, females are more likely to show a higher intention to use aicsd compared to males. the strong influence of gender on the intention to use ai-based health technology is noteworthy. several studies in the united states have previously revealed that females are more likely than males to seek online health information and use health apps (atkinson et al., 2009; escoffery, 2018; haluza & wernhart, 2019). similar findings, indicating a female majority in health app use and online searching for health-related material, have been reported in other countries, including china, saudi arabia, and germany (alghamdi & moussa, 2012; baumann et al., 2017; zhang et al., 2014). with respect to the explanatory power of the model, it was found to be superior to the original base/generic model ttm (mcknight et al., 2011; lankton et al., 2015), where the values of variables trusting beliefs and trusting intentions were 50% and 50%, respectively. a likely reason for our model's superior explanatory power could be incorporating appropriate australasian journal of information systems prakash & das 2024, vol 28, research article (why) do we trust ai? 25 contextual factors into the base model (ttm). thus, our model offers good explanatory power in explaining the trust (trusting beliefs) in aicsd and its consequences. 6.1 theoretical contributions from a theoretical perspective, this study makes a few significant contributions to the literature on trust in ai-based technologies for self-diagnosis in healthcare settings. examining contextual factors and formulating context-specific theories is crucial for advancing research in is (hong et al. 2014; orlikowski & iacono 2001). the existing literature on trust in aicsd for healthcare is deficient in this aspect. the contextualization approach we applied by integrating relevant factors to the trust in technology framework contributes to a more profound comprehension of the specific factors that might impact the consumers’ trust in this emerging ai-based technology. by empirically validating our model, we demonstrated that such integration can significantly improve the generic trust in the technology model’s explanatory power. this is a valuable contribution to the literature on trust in technologies (mcknight et al., 2011; lankton et al., 2015), especially emerging ai-based technologies. second, the study contributes to the emerging literature on explanation-driven trust in ai technologies (shin, 2021; zierau et al., 2020) by demonstrating the mechanism and impact of perceived explainability on the trust formation process in aicsd. our study provides evidence for establishing perceived explainability as a significant predictor of trusting beliefs. because the impact of this factor on trust is rarely validated (shin, 2020, 2021) in chatbots/personal assistants (zierau et al., 2020) and ai systems in general (rai, 2019), especially in healthcare technologies (o’connor et al., 2021; zhang et al., 2021) and ambiguity regarding the magnitude and direction of the relationship still exists in the literature (cramer et al., 2008; kizilcec, 2016; mcguinness et al., 2006; pu & chen, 2006; rai, 2019; zang et al., 2021), our study makes a valuable contribution by showing that such explanations might assist users in interpreting opaque ai recommendations and forming a higher trust towards the ai system. additionally, in the context of patient-facing ai systems, the positive effect of explainability on trust observed by our research is noteworthy as it furthers the ongoing debate on the direction of the impact of explainability on trust (zang et al., 2021). third, our study contributes to the literature on interaction-driven trust in ai-based chatbots/agents (de visser et al., 2016; troshani et al., 2020) by demonstrating the impact of perceived anthropomorphism on trust. it demonstrates that despite being a highly utilitarian context, users care about the humanness of the chatbot more than the other utilitarian factors. thus, we add to the argument by culley and madhavan (2013) that undue trust could be formed due to an overestimation of the system's capabilities as the user gets carried away by the anthropomorphic features of the chatbot. it also points to the emerging concept of ‘humanwashing,’ which is the deceptive use of anthropomorphic features to mislead organizational stakeholders and the general public about the machines' true capabilities (scorici et al., 2022; seele & schultz, 2022). fourth, we validate the role of risk perceptions (privacy and health risk) as the key barriers to aicsd usage intentions. additionally, our research confirms the critical role of trust (trusting beliefs) in alleviating the risk perceptions about the use in the post-usage context. while research is still equivocal on whether trusting beliefs predict perceived risk or vice versa, our findings add to the line of argument put forward by gefen et al. (2003b) and holmes (1991) and the supporting literature that followed (jarvenpaa et al., 2000; lou, 2002; nicolaou and australasian journal of information systems prakash & das 2024, vol 28, research article (why) do we trust ai? 26 mcknight, 2006; pavlou 2003; pavlou & gefen, 2004) that trust reduces uncertainty and provides a sense of assurance when outcomes are unclear. additionally, the non-significant impact of trust signs and perceived system quality on trusting beliefs presents a deviation from the prior literature (hoffmann et al., 2014; song & zahedi, 2007). investigating why the relationships do not hold well in specific contexts could lead to the refinement of theory and a better understanding of causality (christensen & raynor, 2003). this might represent an opportunity to test a similar hypothesis in a different cultural or demographic context to understand the variability and underlying causes. 6.2 implications to practice from a practitioner’s perspective, our context-specific model, which focuses on the specific d2c ai applications (chatbots) used for self-diagnosis in a healthcare setting, provides specific and actionable insights for enhancing the consumers’ trust in and acceptance of these novel health technologies. first, the intention to use is determined by trusting beliefs and risk beliefs (privacy and health risk). enhancing the trust in the technology (ability, integrity, benevolence, and reliability) is hence key to promoting the usage. our study identifies the factors, namely perceived anthropomorphism, perceived information quality, perceived explainability, disposition to trust technology, and perceived service quality, as the significant predictors of trust (trusting beliefs), which offers several actionable implications. since perceived anthropomorphism is the most prominent determinant of building trust, designers should consider including human-like features as a conscious design choice. they should improve upon the anthropomorphic design elements of the chatbot that would make it appear more natural, human-like, and conscious. a few ways to achieve this are through visual embodiment (e.g., avatar), improving conversational ability (contingency and interactivity of the chatbot) (go & sundar, 2019), or empathic cues of the agent (zierau et al., 2020). however, the decision to infuse the chatbot with more human-like attributes should be carefully and consciously considered to ensure that any impacts that emerge as a result are consistent with the intended design objectives. because there is a possibility of consumers overtrusting the chatbot due to its human-like features (culley & madhavan, 2013) as well as invoking an eerie reaction happening due to the ‘uncanny valley effect’ (ho & macdorman, 2017) when the anthropomorphism exceeds a certain threshold. another actionable determinant of trusting beliefs is perceived information quality. this implies that in order to improve trust in aicsd, the service providers should continuously monitor the accuracy and currency of the medical/health-related information provided to the user. they must also monitor if the information shared is useful and relevant to their condition/query and ensure it is complete and understandable. similarly, service quality perceptions can be improved by using a visually attractive user interface, providing reliable services, ensuring the user's safety, and improving the bot's responsiveness and empathy in conversations. further, the perceived explainability, i.e., the reason why the bot suggests a particular diagnosis and course of action, is an important determinant of trust (trusting beliefs). this implies that users are keen on receiving an explanation justifying the suggestion of the bot, and a satisfactory explanation can open up the otherwise opaque process/algorithm involved in the generation of recommendations and instill trust in the bot. however, what constitutes a good explanation and how the characteristics of the explanation affect trust is yet to be australasian journal of information systems prakash & das 2024, vol 28, research article (why) do we trust ai? 27 determined (rai, 2019). finally, consumers could be segmented according to their disposition to trust technology. providers could put efforts into convincing the consumers (with a moderate or low propensity) about the trustworthiness of chatbots by clarifying their doubts and addressing their concerns through personalized marketing communication/customer service. further, reducing the risk belief is critical to improving the usage, as per our study. accordingly, user perception of privacy risk could be assuaged by safeguarding user privacy by implementing privacy-preserving settings, privacy seals, and certifications issued by reputable third-party organizations such as truste/verisign and communicating the same to the public. however, this could become very challenging as new research indicates that online users are becoming increasingly fatalistic about privacy protection (xie et al., 2019) due to deliberate/secret violations of privacy policies by the service providers (brandtzaeg et al., 2019; okoyomon et al., 2019; yu et al., 2016). in this study itself, the impact of privacy-related trust signs was found to be ineffective in reducing privacy risk perceptions. strict regulatory oversight by government agencies could help in rebuilding consumer trust in privacy protection measures/settings. similarly, for managing health risks, endorsements from doctors representing professional medical bodies, such as the indian medical association, etc., could be added to their marketing communications. consumers could also be given the option to consult with a human medical practitioner in case of doubts regarding the chatbot recommendations. even though, according to our results, the trust cues are ineffective at reducing the risk perceptions, our results indicate that these risk perceptions can be assuaged by building trust in the aicsd. given the evidence that this ‘trust in aicsd’ depends on the user’s perception of attributes of the technology, i.e., anthropomorphism, information quality, explainability, and service quality, it allows enhancing/manipulating some of these attributes to create an enhanced sense of security (even a false perception), which would reduce the risk perception and thereby promote the adoption/use. for example, as previously indicated, perceived anthropomorphism can be enhanced by manipulating design characteristics such as voice, text, or physical embodiment or improvements in conversational ability, speech recognition, or adaptive behavior (go & sundar, 2019; moussawi, 2021), which could build trust in aicsd and, in turn, will help reduce the risk perceptions. 6.3 limitations & future research direction withstanding its findings, this work has some limitations, some of which may provide avenues for further research. first, by selecting india as the research setting, our findings may primarily apply to comparable developing nations. we acknowledge the influence of culture in the formation of trust and acceptance of emerging technology and suggest that crosscultural research might help to confirm our findings. second, participants of this study were predominantly post-graduate students (although there was a small group of non-student respondents), which could restrict the generalizability of the findings. however, we contend that our samples sufficiently represent the population since our participants are representative of the target consumers for health apps in general. nevertheless, substantiating our findings with a larger representative sample would add credibility to our conclusions. lastly, our research was cross-sectional in nature, and it did not examine actual behavior; instead, it focused on “use intentions.” a longitudinal approach is advised for further empirical australasian journal of information systems prakash & das 2024, vol 28, research article (why) do we trust ai? 28 evaluation of the framework in similar scenarios with other contextual factors. it would also be worthwhile to investigate the link between stated intention and actual usage. declaration the authors report there are no competing interests to declare. the first author was partially supported by the university grants commission, government of india [grant number f.156(dec. 2015)/2016(net)]. references adadi, a., & berrada, m. (2018). peeking inside the black-box: a survey on explainable artificial intelligence (xai). ieee access, 6, 52138-52160. adam, m., wessel, m., & benlian, a. (2021). ai-based chatbots in customer service and their effects on user compliance. electronic markets, 31(2), 427–445. https://doi.org/10.1007/s12525-020-00414-7 adebesin, f., & mwalugha, r. (2020). the mediating role of organizational reputation and trust in the intention to use wearable health devices: cross-country study. jmir mhealth uhealth, 8(6), e16721. https://doi.org/10.2196/16721 akter, s., ray, p., & ambra, j. d. (2013). continuance of mhealth services at the bottom of the pyramid : the roles of service quality and trust. electronic markets, 23(1), 29–47. https://doi.org/10.1007/s12525-012-0091-5 alghamdi, k. m., & moussa, n. a. (2012). internet use by the public to search for healthrelated information. international journal of medical informatics, 81(6), 363-373. araujo, t. (2018). living up to the chatbot hype: the influence of anthropomorphic design cues and communicative agency framing on conversational agent and company perceptions. computers in human behavior, 85, 183–189. https://doi.org/10.1016/j.chb.2018.03.051 asan, o., & choudhury, a. (2021). research trends in artificial intelligence applications in human factors health care: mapping review. jmir human factors, 8(2), e28236. atkinson, n., saperstein, s., & pleis, j. (2009). using the internet for health-related activities: findings from a national probability sample. journal of medical internet research, 11(1), e1035. bach, t. a., khan, a., hallock, h., beltrão, g., & sousa, s. (2022). a systematic literature review of user trust in ai-enabled systems: an hci perspective. international journal of human–computer interaction, 1-16. bacharach, m., & gambetta, d. (2001). trust in signs. in k. cook (ed.), trust in society (pp. 148– 184). new york, ny, usa:: russell sage foundation. bartneck, c., kulić, d., croft, e., & zoghbi, s. (2009). measurement instruments for the anthropomorphism, animacy, likeability, perceived intelligence, and perceived safety of robots. international journal of social robotics, 1(1), 71–81. https://doi.org/10.1007/s12369-008-0001-3 baumann, e., czerwinski, f., & reifegerste, d. (2017). gender-specific determinants and patterns of online health information seeking: results from a representative german health survey. journal of medical internet research, 19(4), e92. https://doi.org/10.2196/16721 https://doi.org/10.1007/s12525-012-0091-5 https://doi.org/10.1016/j.chb.2018.03.051 https://doi.org/10.1007/s12369-008-0001-3 australasian journal of information systems prakash & das 2024, vol 28, research article (why) do we trust ai? 29 bedué, p., & fritzsche, a. (2022). can we trust ai? an empirical investigation of trust requirements and guide to successful ai adoption. journal of enterprise information management, 35(2), 530-549. benbasat, i., & wang, w. (2005). trust in and adoption of online recommendation agents. journal of the association for information systems, 6(3), 4. berezowska, a., fischer, a. r. h., ronteltap, a., van der lans, i. a., & van trijp, h. c. m. (2015). consumer adoption of personalised nutrition services from the perspective of a risk–benefit trade-off. genes & nutrition, 10(6), 42. https://doi.org/10.1007/s12263-0150478-y bickmore, t. w., trinh, h., olafsson, s., o’leary, t. k., asadi, r., rickles, n. m., & cruz, r. (2018). patient and consumer safety risks when using conversational assistants for medical information: an observational study of siri, alexa, and google assistant. journal of medical internet research, 20(9), e11510. https://doi.org/10.2196/11510 brandtzaeg, p. b., pultier, a., & moen, g. m. (2019). losing control to data-hungry apps: a mixed-methods approach to mobile app privacy. social science computer review, 37(4), 466-488. https://doi.org/10.1177/0894439318777706 breward, m., hassanein, k., & head, m. (2017). understanding consumers’ attitudes toward controversial information technologies: a contextualization approach. information systems research, 28(4), 760-774. chi, o. h., jia, s., li, y., & gursoy, d. (2021). developing a formative scale to measure consumers’ trust toward interaction with artificially intelligent (ai) social robots in service delivery. computers in human behavior, 118, 106700. https://doi.org/10.1016/j.chb.2021.106700 chiles, t. h., & mcmackin, j. f. (1996). integrating variable risk preferences, trust, and transaction cost economics. academy of management review, 21(1), 73–99. https://doi.org/10.5465/amr.1996.9602161566 chiou, j.-s., & droge, c. (2006). service quality, trust, specific asset investment, and expertise: direct and indirect effects in a satisfaction-loyalty framework. journal of the academy of marketing science, 34(4), 613. https://doi.org/10.1177/0092070306286934 christensen, c. m., & raynor, m. e. (2003). why hard-nosed executives should care about management theory. harvard business review, 81(9), 66-75. chui, k. t., liu, r. w., lytras, m. d., & zhao, m. (2019). big data and iot solution for patient behaviour monitoring. behaviour & information technology, 38(9), 940-949. ćirković, a. (2020). evaluation of four artificial intelligence–assisted self-diagnosis apps on three diagnoses: two-year follow-up study. journal of medical internet research, 22(12), e18097. https://doi.org/10.2196/18097 cocosila, m., & turel, o. (2016). a dual-risk model of user adoption of mobile-based smoking cessation support services. behaviour & information technology, 35(7), 526-535. cocosila, m., turel, o., archer, n., & yuan, y. (2007). perceived health risks of 3g cell phones: do users care?. communications of the acm, 50(6), 89-92. cook, k. (ed.). (2001). trust in society. new york, ny, usa: russell sage foundation. https://doi.org/10.1007/s12263-015-0478-y https://doi.org/10.1007/s12263-015-0478-y https://doi.org/10.2196/11510 https://doi.org/10.1177/0894439318777706 https://doi.org/10.1016/j.chb.2021.106700 https://doi.org/10.5465/amr.1996.9602161566 https://doi.org/10.1177/0092070306286934 https://doi.org/10.2196/18097 australasian journal of information systems prakash & das 2024, vol 28, research article (why) do we trust ai? 30 cramer, h., evers, v., ramlal, s., van someren, m., rutledge, l., stash, n., ... & wielinga, b. (2008). the effects of transparency on trust in and acceptance of a content-based art recommender. user modeling and user-adapted adapted interaction, 18, 455-496. cronin, j. j., brady, m. k., & hult, g. t. m. (2000). assessing the effects of quality, value, and customer satisfaction on consumer behavioral intentions in service environments. journal of retailing, 76(2), 193–218. https://doi.org/10.1016/s0022-4359(00)00028-2 culley, k. e., & madhavan, p. (2013). a note of caution regarding anthropomorphism in hci agents. computers in human behavior, 29(3), 577–579. https://doi.org/10.1016/j.chb.2012.11.023 cunningham, s.m. (1967). the major dimensions of perceived risk. in d. f. cox (ed.), risk taking and information handling in consumer behavior (pp. 82–108). harvard university press. dagger, t. s., sweeney, j. c., & johnson, l. w. (2007). a hierarchical model of health service quality: scale development and investigation of an integrated model. journal of service research, 10(2), 123–142. https://doi.org/10.1177/1094670507309594 de visser, e. j., monfort, s. s., mckendrick, r., smith, m. a. b., mcknight, p. e., krueger, f., & parasuraman, r. (2016). almost human: anthropomorphism increases trust resilience in cognitive agents. journal of experimental psychology: applied, 22(3), 331. delone, w. h., & mclean, e. r. (2003). the delone and mclean model of information systems success: a ten-year update. journal of management information systems, 19(4), 9–30. https://doi.org/10.1080/07421222.2003.11045748 denecke, k., gabarron, e., grainger, r., konstantinidis, s. t., lau, a., rivera-romero, o., ... & merolli, m. (2019). artificial intelligence for participatory health: applications, impact, and future implications. yearbook of medical informatics, 28(01), 165-173. dinev, t., & hart, p. (2006). an extended privacy calculus model for e-commerce transactions. information systems research, 17(1), 61–80. https://doi.org/10.1287/isre.1060.0080 eisingerich, a. b., & bell, s. j. (2008). perceived service quality and customer trust: does enhancing customers’ service knowledge matter? journal of service research, 10(3), 256– 268. https://doi.org/10.1177/1094670507310769 epley, n., caruso, e. m., & bazerman, m. h. (2006). when perspective taking increases taking: reactive egoism in social interaction. journal of personality and social psychology, 91(5), 872. epley, n., waytz, a., & cacioppo, j. t. (2007). on seeing human: a three-factor theory of anthropomorphism. psychological review, 114(4), 864. escoffery, c. (2018). gender similarities and differences for e-health behaviors among us adults. telemedicine and e-health, 24(5), 335-343. fan, x., chao, d., zhang, z., wang, d., li, x., & tian, f. (2021). utilization of self-diagnosis health chatbots in real-world settings: case study. journal of medical internet research, 23(1), e19928. https://doi.org/10.2196/19928 følstad, a., & brandtzæg, p. b. (2017). chatbots and the new world of hci. interactions, 24(4), 38-42. https://doi.org/10.1016/s0022-4359(00)00028-2 https://doi.org/https:/doi.org/10.1016/j.chb.2012.11.023 https://doi.org/10.1177/1094670507309594 https://doi.org/10.1080/07421222.2003.11045748 https://doi.org/10.1287/isre.1060.0080 https://doi.org/10.1177/1094670507310769 https://doi.org/10.2196/19928 australasian journal of information systems prakash & das 2024, vol 28, research article (why) do we trust ai? 31 fromkin, h. l., & streufert, s. (1976). laboratory experimentation. in m. d. dunnette (ed.), handbook of industrial and organizational psychology (pp. 415-465). rand mcnally. gambetta, d., & bacharach, m. (2001). trust in signs. trust and society, new york, ny,usa: russell sage foundation, 148-184. gefen, d., karahanna, e., & straub, d. w. (2003a). trust and tam in online shopping: an integrated model. mis quarterly, 27(1), 51–90. https://doi.org/10.2307/30036519 gefen, d., srinivasan rao, v., & tractinsky, n. (2003). the conceptualization of trust, risk and their electronic commerce: the need for clarifications. in r. h. sprague (ed.), proceedings of the 36th annual hawaii international conference on system sciences (hicss 2003), article 1174442. institute of electrical and electronics engineers (ieee). https://doi.org/10.1109/hicss.2003.1174442 gille, f., jobin, a., & ienca, m. (2020). what we talk about when we talk about trust: theory of trust for ai in healthcare. intelligence-based medicine, 1–2, 100001. https://doi.org/10.1016/j.ibmed.2020.100001 go, e., & sundar, s. s. (2019). humanizing chatbots: the effects of visual, identity and conversational cues on humanness perceptions. computers in human behavior, 97, 304316. hair jr, j. f., hult, g. t. m., ringle, c., & sarstedt, m. (2016). a primer on partial least squares structural equation modeling (pls-sem). sage publications. hair, j. f., risher, j. j., sarstedt, m., & ringle, c. m. (2019). when to use and how to report the results of pls-sem. european business review, 31(1), 2–24. https://doi.org/10.1108/ebr11-2018-0203 haluza, d., & wernhart, a. (2019). does gender matter? exploring perceptions regarding health technologies among employees and students at a medical university. international journal of medical informatics, 130, 103948. hengstler, m., enkel, e., & duelli, s. (2016). applied artificial intelligence and trust—the case of autonomous vehicles and medical assistance devices. technological forecasting and social change, 105, 105-120. ho, c.-c., & macdorman, k. f. (2017). measuring the uncanny valley effect. international journal of social robotics, 9(1), 129–139. hoffmann, c. p., lutz, c., & meckel, m. (2014). digital natives or digital immigrants? the impact of user characteristics on online trust. journal of management information systems, 31(3), 138–171. https://doi.org/10.1080/07421222.2014.995538 holmes, j. g. (1991). trust and the appraisal process in close relationships. in w. h. jones & d. perlman (eds.), advances in personal relationships: a research annual, vol. 2, pp. 57–104). london, uk: jessica kingsley publishers. hong, w., chan, f. k., thong, j. y., chasalow, l. c., & dhillon, g. (2014). a framework and guidelines for context-specific theorizing in information systems research. information systems research, 25(1), 111-136. https://doi.org/10.2307/30036519 https://doi.org/10.1016/j.ibmed.2020.100001 https://doi.org/10.1108/ebr-11-2018-0203 https://doi.org/10.1108/ebr-11-2018-0203 https://doi.org/10.1080/07421222.2014.995538 australasian journal of information systems prakash & das 2024, vol 28, research article (why) do we trust ai? 32 hu, p., lu, y., & gong, y. (2021). dual humanness and trust in conversational ai: a personcentered approach. computers in human behavior, 119, 106727. https://doi.org/10.1016/j.chb.2021.106727 jiang, j. j., klein, g., & carr, c. l. (2002). measuring information system service quality: servqual from the other side. mis quarterly, 26(2), 145–166. https://doi.org/10.2307/4132324 johns, g. (2006). the essential impact of context on organizational behavior. academy of management review, 31(2), 386-408. johnson, v. l., kiser, a., washington, r., & torres, r. (2018). limitations to the rapid adoption of m-payment services: understanding the impact of privacy risk on m-payment services. computers in human behavior, 79, 111–122. kizilcec, r. f. (2016). how much information? effects of transparency on trust in an algorithmic interface. in proceedings of the 2016 chi conference on human factors in computing systems (pp. 2390–2395). association for computing machinery (acm). https://doi.org/10.1145/2858036.2858402 koller, m. (1988). risk as a determinant of trust. basic and applied social psychology, 9(4), 265276. lankton, n. k., mcknight, d. h., & tripp, j. (2015). technology, humanness, and trust: rethinking trust in technology. journal of the association for information systems, 16(10), 880. laranjo, l., dunn, a. g., tong, h. l., kocaballi, a. b., chen, j., bashir, r., surian, d., gallego, b., magrabi, f., lau, a. y. s., & coiera, e. (2018). conversational agents in healthcare: a systematic review. journal of the american medical informatics association, 25(9), 1248– 1258. https://doi.org/10.1093/jamia/ocy072 laumer, s., maier, c., & gubler, f. (2019). chatbot acceptance in healthcare: explaining user adoption of conversational agents for disease diagnosis. proceedings of the 27th european conference on information systems (ecis), 27(1), 88. https://aisel.aisnet.org/ecis2019_rp/88 li, h., wu, j., gao, y., & shi, y. (2016). examining individuals’ adoption of healthcare wearable devices: an empirical study from privacy calculus perspective. international journal of medical informatics, 88, 8–17. https://doi.org/10.1016/j.ijmedinf.2015.12.010 li, x., hess, t. j., & valacich, j. s. (2006). using attitude and social influence to develop an extended trust model for information systems. acm sigmis database: the database for advances in information systems, 37(2-3), 108-124. li, x., hess, t. j., & valacich, j. s. (2008). why do we trust new technology? a study of initial trust formation with organizational information systems. the journal of strategic information systems, 17(1), 39–71. https://doi.org/10.1016/j.jsis.2008.01.001 lindell, m. k., & whitney, d. j. (2001). accounting for common method variance in crosssectional research designs. the journal of applied psychology, 86(1), 114–121. https://doi.org/10.1037/0021-9010.86.1.114 london, a. j. (2019). artificial intelligence and black-box medical decisions: accuracy versus explainability. hastings center report, 49(1), 15-21. https://doi.org/10.1016/j.chb.2021.106727 https://doi.org/10.2307/4132324 https://doi.org/10.1093/jamia/ocy072 https://aisel.aisnet.org/ecis2019_rp/88 https://doi.org/10.1016/j.ijmedinf.2015.12.010 https://doi.org/10.1016/j.jsis.2008.01.001 https://doi.org/10.1037/0021-9010.86.1.114 australasian journal of information systems prakash & das 2024, vol 28, research article (why) do we trust ai? 33 malhotra, n. k., kim, s. s., & patil, a. (2006). common method variance in is research: a comparison of alternative approaches and a reanalysis of past research. management science, 52(12), 1865-1883. mayer, r. c., davis, j. h., & schoorman, f. d. (1995). an integrative model of organizational trust. academy of management review, 20(3), 709-734. https://doi.org/10.5465/amr.1995.9508080335 mcguinness, d. l., ding, l., glass, a., chang, c., zeng, h., & furtado, v. (2006, november). explanation interfaces for the semantic web: issues and models. in proceedings of the 3rd international semantic web user interaction workshop (swui'06). https://hdl.handle.net/20.500.13015/4736 mcknight, d. h., & chervany, n. l. (2001). what trust means in e-commerce customer relationships: an interdisciplinary conceptual typology. international journal of electronic commerce, 6(2), 35–59. https://doi.org/10.1080/10864415.2001.11044235 mcknight, d. h., carter, m., thatcher, j. b., & clay, p. f. (2011). trust in a specific technology: an investigation of its components and measures. acm transactions on management information systems, 2(2), 12:1-12:25. https://doi.org/10.1145/1985347.1985353 mcknight, d. h., choudhury, v., & kacmar, c. (2002). developing and validating trust measures for e-commerce: an integrative typology. information systems research, 13(3), 334-359. mcknight, d. h., kacmar, c. j., & choudhury, v. (2004). shifting factors and the ineffectiveness of third party assurance seals: a two-stage model of initial trust in a web business. electronic markets, 14(3), 252-266. mcknight, d. h., lankton, n. k., nicolaou, a., & price, j. (2017). distinguishing the effects of b2b information quality, system quality, and service outcome quality on trust and distrust. the journal of strategic information systems, 26(2), 118-141. mctear, m., callejas, z., & griol, d. (2016). conversational interfaces: devices, wearables, virtual agents, and robots. in m. mctear, z. callejas, & d. griol (eds.), the conversational interface: talking to smart devices (pp. 283–308). cham: switzerland: springer international publishing. https://doi.org/10.1007/978-3-319-32967-3_13 mesbah, n., & pumplun, l. (2020). “hello, i’m here to help you” – medical care where it is needed most: seniors’ acceptance of health chatbots. proceedings of the european conference on information systems. 28(1), 209. https://aisel.aisnet.org/ecis2020_rp/209 miner, a. s., shah, n., bullock, k. d., arnow, b. a., bailenson, j., & hancock, j. (2019). key considerations for incorporating conversational ai in psychotherapy. frontiers in psychiatry, 10. https://doi.org/10.3389/fpsyt.2019.00746 minutolo, a., esposito, m., & de pietro, g. (2017). a conversational chatbot based on kowledge-graphs for factoid medical questions. in new trends in intelligent software methodologies, tools and techniques (pp. 139-152). amsterdam, the netherlands: ios press. moussawi, s., koufaris, m., & benbunan-fich, r. (2021). how perceptions of intelligence and anthropomorphism affect adoption of personal intelligent agents. electronic markets, 31(2), 343-364. https://doi.org/10.1007/s12525-020-00411-w https://doi.org/10.5465/amr.1995.9508080335 https://doi.org/10.1080/10864415.2001.11044235 https://doi.org/10.1145/1985347.1985353 https://doi.org/10.1007/978-3-319-32967-3_13 https://aisel.aisnet.org/ecis2020_rp/209 https://doi.org/10.1007/s12525-020-00411-w australasian journal of information systems prakash & das 2024, vol 28, research article (why) do we trust ai? 34 nadarzynski, t., miles, o., cowie, a., & ridge, d. (2019). acceptability of artificial intelligence (ai)-led chatbot services in healthcare: a mixed-methods study. digital health, 5, 2055207619871808. naneva, s., sarda gou, m., webb, t. l., & prescott, t. j. (2020). a systematic review of attitudes, anxiety, acceptance, and trust towards social robots. international journal of social robotics, 12(6), 1179-1201. nass, c., & moon, y. (2000). machines and mindlessness: social responses to computers. journal of social issues, 56(1), 81-103. nicolaou, a. i., & mcknight, d. h. (2006). perceived information quality in data exchanges: effects on risk, trust, and intention to use. information systems research, 17(4), 332–351. nowak, k. l., & rauh, c. (2005). the influence of the avatar on online perceptions of anthropomorphism, androgyny, credibility, homophily, and attraction. journal of computer-mediated communication, 11(1), 153-178. nundy, s., montgomery, t., & wachter, r. m. (2019). promoting trust between patients and physicians in the era of artificial intelligence. jama, 322(6), 497-498. o'connor, y., kupper, m., & heavin, c. (2021). trusting intentions towards robots in healthcare: a theoretical framework. proceedings of the 54th hawaii international conference on system sciences, 54(1), 586. https://doi.org/10.24251/hicss.2021.071 okoyomon, e., samarin, n., wijesekera, p., elazari bar on, a., vallina-rodriguez, n., reyes, i., ... & egelman, s. (2019). on the ridiculousness of notice and consent: contradictions in app privacy policies. in workshop on technology and consumer protection (conpro 2019), in conjunction with the 39th ieee symposium on security and privacy (pp. 1-7). http://hdl.handle.net/20.500.12761/690 orlikowski, w. j., & iacono, c. s. (2001). research commentary: desperately seeking the “it” in it research—a call to theorizing the it artifact. information systems research, 12(2), 121-134. parasuraman, a., zeithaml, v. a., & berry, l. l. (1988). servqual: a multiple-item scale for measuring consumer perceptions of service quality. journal of retailing, 64(1), 12–40. pavlou, p. a. (2003). consumer acceptance of electronic commerce: integrating trust and risk with the technology acceptance model. international journal of electronic commerce, 7(3), 101-134. pavlou, p. a., & gefen, d. (2004). building effective online marketplaces with institution-based trust. information systems research, 15(1), 37-59. pelaez, a., chen, c.-w., & chen, y. x. (2019). effects of perceived risk on intention to purchase: a meta-analysis. journal of computer information systems, 59(1), 73–84. https://doi.org/10.1080/08874417.2017.1300514 pennington, r., wilcox, h. d., & grover, v. (2003). the role of system trust in business-toconsumer transactions. journal of management information systems, 20(3), 197–226. https://doi.org/10.1080/07421222.2003.11045777 https://doi.org/10.24251/hicss.2021.071 http://hdl.handle.net/20.500.12761/690 https://doi.org/10.1080/08874417.2017.1300514 https://doi.org/10.1080/07421222.2003.11045777 australasian journal of information systems prakash & das 2024, vol 28, research article (why) do we trust ai? 35 podsakoff, p. m., mackenzie, s. b., & podsakoff, n. p. (2012). sources of method bias in social science research and recommendations on how to control it. annual review of psychology, 63(1), 539–569. https://doi.org/10.1146/annurev-psych-120710-100452 podsakoff, p. m., mackenzie, s. b., lee, j. y., & podsakoff, n. p. (2003). common method biases in behavioral research: a critical review of the literature and recommended remedies. journal of applied psychology, 88(5), 879. https://doi.org/10.1037/00219010.88.5.879. powell, j. (2019). trust me, i’m a chatbot: how artificial intelligence in health care fails the turing test. journal of medical internet research, 21(10), e16222. https://doi.org/10.2196/16222 pr newswire. (2021, march 5). healthcare chatbot market to reach us$ 967.7 million by 2027, globally. pr newswire. https://www.prnewswire.com/in/news-releases/healthcarechatbot-market-to-reach-us-967-7-million-by-2027-globally-cagr-21-56-univdatosmarket-insights-840043251.html prakash, a. v., & das, s. (2020). would you trust a bot for healthcare advice? an empirical investigation. proceedings of the pacific asia conference on information systems, 24(1), 62. https://aisel.aisnet.org/pacis2020/62 pu, p., & chen, l. (2006). trust building with explanation interfaces. in proceedings of the 11th international conference on intelligent user interfaces (pp. 93–100). association for computing machinery (acm). https://doi.org/10.1145/1111449.1111475 qiu, l., & benbasat, i. (2009). evaluating anthropomorphic product recommendation agents: a social relationship perspective to designing information systems. journal of management information systems, 25(4), 145–182. https://doi.org/10.2753/mis07421222250405 rai, a. (2020). explainable ai: from black box to glass box. journal of the academy of marketing science, 48(1), 137–141. https://doi.org/10.1007/s11747-019-00710-5 razzaki, s., baker, a., perov, y., middleton, k., baxter, j., mullarkey, d., sangar, d., taliercio, m., butt, m., majeed, a., dorosario, a., mahoney, m., & johri, s. (2018). a comparative study of artificial intelligence and human doctors for the purpose of triage and diagnosis. 1–15. http://arxiv.org/abs/1806.10698 rice, s. c. (2012). reputation and uncertainty in online markets: an experimental study. information systems research, 23(2), 436-452. robertson, n., polonsky, m., & mcquilken, l. (2014). are my symptoms serious dr google? a resource-based typology of value co-destruction in online self-diagnosis. australasian marketing journal (amj), 22(3), 246–256. https://doi.org/10.1016/j.ausmj.2014.08.009 russell, s. (2019). human compatible: artificial intelligence and the problem of control. publisher london, uk: penguin uk. salles, a., evers, k., & farisco, m. (2020). anthropomorphism in ai. ajob neuroscience, 11(2), 88-95. https://doi.org/10.1146/annurev-psych-120710-100452 https://doi.org/10.1037/0021-9010.88.5.879 https://doi.org/10.1037/0021-9010.88.5.879 https://doi.org/10.2196/16222 https://www.prnewswire.com/in/news-releases/healthcare-chatbot-market-to-reach-us-967-7-million-by-2027-globally-cagr-21-56-univdatos-market-insights-840043251.html https://www.prnewswire.com/in/news-releases/healthcare-chatbot-market-to-reach-us-967-7-million-by-2027-globally-cagr-21-56-univdatos-market-insights-840043251.html https://www.prnewswire.com/in/news-releases/healthcare-chatbot-market-to-reach-us-967-7-million-by-2027-globally-cagr-21-56-univdatos-market-insights-840043251.html https://aisel.aisnet.org/pacis2020/62 https://doi.org/10.2753/mis0742-1222250405 https://doi.org/10.2753/mis0742-1222250405 https://doi.org/10.1007/s11747-019-00710-5 http://arxiv.org/abs/1806.10698 https://doi.org/https:/doi.org/10.1016/j.ausmj.2014.08.009 australasian journal of information systems prakash & das 2024, vol 28, research article (why) do we trust ai? 36 sbaffi, l., & rowley, j. (2017). trust and credibility in web-based health information: a review and agenda for future research. journal of medical internet research, 19(6), e218– e218. https://doi.org/10.2196/jmir.7579 scorici, g., schultz, m. d., & seele, p. (2024). anthropomorphization and beyond: conceptualizing humanwashing of ai-enabled machines. ai & society, 39(2), 789–795. https://doi.org/10.1007/s00146-022-01492-1 seele, p., & schultz, m. d. (2022). from greenwashing to machinewashing: a model and future directions derived from reasoning by analogy. journal of business ethics, 178(4), 10631089. shah, h., warwick, k., vallverdú, j., & wu, d. (2016). can machines talk? comparison of eliza with modern dialogue systems. computers in human behavior, 58, 278–295. https://doi.org/10.1016/j.chb.2016.01.004 sheehan, b., jin, h. s., & gottlieb, u. (2020). customer service chatbots: anthropomorphism and adoption. journal of business research, 115, 14–24. https://doi.org/10.1016/j.jbusres.2020.04.030 shin, d. (2020). user perceptions of algorithmic decisions in the personalized ai system: perceptual evaluation of fairness, accountability, transparency, and explainability. journal of broadcasting & electronic media, 64(4), 541–565. https://doi.org/10.1080/08838151.2020.1843357 shin, d. (2021). the effects of explainability and causability on perception, trust, and acceptance: implications for explainable ai. international journal of human-computer studies, 146, 102551. https://doi.org/10.1016/j.ijhcs.2020.102551 sillence, e., briggs, p., harris, p., & fishwick, l. (2007). health websites that people can trust– the case of hypertension. interacting with computers, 19(1), 32–42. siwicki, b. (2018, february 1). special report: ai voice assistants have officially arrived in healthcare. healthcare it news. http://www.healthcareitnews.com/news/special-reportai-voice-assistants-have-officially-arrived-healthcare. accessed october 22, 2021. söllner, m., hoffmann, a., & leimeister, j. m. (2016). why different trust relationships matter for information systems users. european journal of information systems, 25(3), 274–287. https://doi.org/10.1057/ejis.2015.17 song, j., & zahedi, f. (2007). trust in health infomediaries. decision support systems, 43(2), 390– 407. https://doi.org/10.1016/j.dss.2006.11.011 tan, y. h., & thoen, w. (2003). electronic contract drafting based on risk and trust assessment. international journal of electronic commerce, 7(4), 55-71. troshani, i., rao hill, s., sherman, c., & arthur, d. (2020). do we trust in ai? role of anthropomorphism and intelligence. journal of computer information systems, 1–11. https://doi.org/10.1080/08874417.2020.1788473 verberne, f. m., ham, j., & midden, c. j. (2015). trusting a virtual driver that looks, acts, and thinks like you. human factors, 57(5), 895-909. https://doi.org/10.2196/jmir.7579 https://doi.org/https:/doi.org/10.1016/j.chb.2016.01.004 https://doi.org/10.1016/j.jbusres.2020.04.030 https://doi.org/10.1080/08838151.2020.1843357 https://doi.org/10.1016/j.ijhcs.2020.102551 http://www.healthcareitnews.com/news/special-report-ai-voice-assistants-have-officially-arrived-healthcare http://www.healthcareitnews.com/news/special-report-ai-voice-assistants-have-officially-arrived-healthcare https://doi.org/10.1057/ejis.2015.17 https://doi.org/10.1016/j.dss.2006.11.011 https://doi.org/10.1080/08874417.2020.1788473 australasian journal of information systems prakash & das 2024, vol 28, research article (why) do we trust ai? 37 wang, w., & siau, k. l. (2018). living with artificial intelligence: developing a theory on trust in health chatbots-research in progress. proceedings of the sighci 2018, 4. https://aisel.aisnet.org/sighci2018/4 waytz, a., heafner, j., & epley, n. (2014). the mind in the machine: anthropomorphism increases trust in an autonomous vehicle. journal of experimental social psychology, 52, 113–117. https://doi.org/10.1016/j.jesp.2014.01.005 wells, j. d., valacich, j. s., & hess, t. j. (2011). what signal are you sending? how website quality influences perceptions of product quality and purchase intentions. mis quarterly, 35(2), 373–396. https://doi.org/10.2307/23044048 wendel, s., dellaert, b. g., ronteltap, a., & van trijp, h. c. (2013). consumers’ intention to use health recommendation systems to receive personalized nutrition advice. bmc health services research, 13(1), 126. whetten, d. a. (2009). an examination of the interface between context and theory applied to the study of chinese organizations. management and organization review, 5(1), 29-56. xie, h., prybutok, g., peng, x., & prybutok, v. (2020). determinants of trust in health information technology: an empirical investigation in the context of an online clinic appointment system. international journal of human–computer interaction, 36(12), 10951109. xie, w., fowler-dawson, a., & tvauri, a. (2019). revealing the relationship between rational fatalism and the online privacy paradox. behaviour & information technology, 38(7), 742759. https://doi.org/10.1080/0144929x.2018.1552717 your.md. (2021). healthily: health questions, answered. https://www.livehealthily.com/. assessed november 9, 2021. yu, l., luo, x., liu, x., & zhang, t. (2016). can we trust the privacy policies of android apps?. proceedings of the ieee/ifip international conference on dependable systems and networks, 46(1), 538-549. https://doi.org/10.1109/dsn.2016.55 zahedi, f. & song, j. (2008). dynamics of trust revision: using health infomediaries. journal of management information systems, 24(4), 225–248. https://doi.org/10.2753/mis07421222240409 zhang, x., guo, x., lai, k. h., guo, f., & li, c. (2014). understanding gender differences in mhealth adoption: a modified theory of reasoned action model. telemedicine and ehealth, 20(1), 39-46. zhang, y., liu, c., luo, s., xie, y., liu, f., li, x., & zhou, z. (2019). factors influencing patients’ intentions to use diabetes management apps based on an extended unified theory of acceptance and use of technology model: web-based survey. journal of medical internet research, 21(8), e15023. zhang, z., genc, y., wang, d., ahsen, m. e., & fan, x. (2021). effect of ai explanations on human perceptions of patient-facing ai-powered healthcare systems. journal of medical systems, 45(6), 64. https://doi.org/10.1007/s10916-021-01743-6 https://aisel.aisnet.org/sighci2018/4 https://doi.org/10.1016/j.jesp.2014.01.005 https://doi.org/10.1080/0144929x.2018.1552717 https://www.livehealthily.com/ https://doi.org/10.1109/dsn.2016.55 https://doi.org/10.2753/mis0742-1222240409 https://doi.org/10.2753/mis0742-1222240409 https://doi.org/10.1007/s10916-021-01743-6 australasian journal of information systems prakash & das 2024, vol 28, research article (why) do we trust ai? 38 zhao, y., ni, q., & zhou, r. (2018). what factors influence the mobile health service adoption? a meta-analysis and the moderating role of age. international journal of information management, 43, 342–350. https://doi.org/10.1016/j.ijinfomgt.2017.08.006 zhou, t. (2011). the impact of privacy concern on user adoption of location-based services. industrial management & data systems, 111(2), 212–226. https://doi.org/10.1108/02635571111115146 zierau, n., engel, c., söllner, m., & leimeister, j. m. (2020). trust in smart personal assistants: a systematic literature review and development of a research agenda. proceedings of the international conference on wirtschaftsinformatik (wi 2020), 15(1), 1-15. https://doi.org/10.30844/wi_2020_a7-zierau appendix appendix 1. respondent demographics item category frequency percentage gender female 65 32.18 male 137 67.82 age 21-25 123 60.89 26-30 73 36.14 31-35 2 0.99 51-55 2 0.99 56-61 2 0.99 educational qualification postgraduate 22 10.89 graduate 180 89.11 employment status student 177 87.62 employed 25 12.38 prior experience of using automated ai-based chatbots (general) no 43 21.29 yes 159 78.71 appendix 2. measurement items construct item code item source intention to use (iu) iu1 i intend to use this health chatbot for making health decisions when a health issue arises in the future adapted from venkatesh et al. (2003) iu2 i predict i would use this health chatbot for making health decisions when a health issue arises in the future iu3 i plan to use this health chatbot for making health decisions when a health issue arises in the future perceived privacy risk pr1 in general, it would be risky to give personal health information to this health chatbot. (xu et al., 2011) pr2 there would be a high potential for privacy loss associated with giving personal health information to this health chatbot. https://doi.org/10.1016/j.ijinfomgt.2017.08.006 https://doi.org/10.1108/02635571111115146 https://doi.org/10.30844/wi_2020_a7-zierau australasian journal of information systems prakash & das 2024, vol 28, research article (why) do we trust ai? 39 (pr) pr3 personal health information could be inappropriately used by this health chatbot. perceived health risk (hr) hr1 how risky do you feel it would be to make a decision based on the health information provided by this chatbot? adapted from mun et al. (2013) hr2 how risky do you feel it would be to accept and apply the provided health information to your life? hr3 how risky do you feel it would be to accept and apply the provided health information to the lives of others important to you? perceived information quality (iq) iq1 this health chatbot provides useful health information to the user (mun et al., 2013; song & zahedi, 2007) iq2 this health chatbot provides sufficient health information regarding the symptoms of the questioner. iq3 this health chatbot provides accurate health information. iq4 this health chatbot provides health information that the questioner is seeking for. iq5 the diagnosis of the consultation is based on the newest (up to date) health information. iq6 the information provided by the health chatbot is easy to comprehend perceived system quality (sy) sy1 this health chatbot is responsive to your request (aladwani & palvia, 2002; mckinney et al., 2002; pantano et al., 2017; song & zahedi, 2007) sy2 this health chatbot is easy to use sy3 this health chatbot is always up and available sy4 this health chatbot allows me to interact with it to receive tailored information about my health issue perceived service quality (sq) sq1 this health chatbot has a modern-looking/visually attractive interface* (ashfaq et al., 2020; roca et al., 2006) sq2 this health chatbot provides dependable services. sq3 i feel safe in my transactions with this health chatbot sq4 this health chatbot gives me a prompt response. sq5 this health chatbot has the users' best interests at heart. perceived anthropomorphism (pa) pa1 please rate your impression of the health chatbot on these scales: fake (unnatural) 1 2 3 4 5 natural (bartneck et al., 2009; sheehan et al., 2020) pa2 machine-like 1 2 3 4 5 human-like pa3 artificial 1 2 3 4 5 lifelike pa4 unconscious 1 2 3 4 5 conscious pa5 communicates inelegantly 1 2 3 4 5 communicates elegantly perceived explainability (ex) ex1 this health chatbot clearly explained the reason why it arrived at a particular diagnosis. developed from (rai, 2020) ex2 this health chatbot clearly explains the reason for suggesting a particular diagnosis. ex3 this health chatbot provided a clear explanation as to why it is recommending a particular course of action. trust signs (ts) ts1 the services provided by this health chatbot was tested by an independent organization and given a seal/certificate of approval (hoffmann et al., 2014) ts2 this health chatbot received a significant amount of positive ratings from other users. ts3 this health chatbot has a third-party privacy assurance seal (e.g., eprivacy) that guarantees privacy protection. australasian journal of information systems prakash & das 2024, vol 28, research article (why) do we trust ai? 40 ts4 this health chatbot has a strong privacy policy to protect my sensitive information (beldad et al., 2010; song & zahedi, 2007) trusting beliefs (tb) tb1 i believe that this health chatbot would act in my best interest. adapted from mcknight et al.(2002) tb2 i believe this health chatbot is truthful in its dealings with me. tb3 this health chatbot is competent and effective in identifying health conditions from symptoms (diagnosis) and recommending an appropriate course of action. tb4 i believe this health chatbot is very reliable for symptom checking & triage disposition to trust technology dt1 my typical approach is to trust new technologies until they prove to me that i shouldn’t trust them. (lankton et al., 2015) dt2 i usually trust a technology until it gives me a reason not to trust it. dt3 i generally give a technology the benefit of the doubt when i first use it. marker variable: fashion consciousness mv1 when i must choose between the two, i usually dress for fashion, not for comfort. (malhotra et al., 2006) mv2 an important part of my life and activities is dressing smartly. mv3 a person should try to dress in style. note. * item removed from the analysis due to poor loadings (<0.708). appendix 3. reliability and validity statistics construct item loading disposition to trust technology (dt) cr = 0.946; ave = 0.853; µ = 4.850; σ = 1.484 dt1 0.910 dt2 0.933 dt3 0.928 perceived explainability (ex) cr = 0.960; ave = 0.890; µ = 4.686; σ = 1.139 ex1 0.935 ex2 0.950 ex3 0.946 perceived health risk (hr) cr = 0.963; ave = 0.897; µ = 4.687; σ = 1.430 hr1 0.934 hr2 0.963 hr3 0.943 perceived information quality (iq) cr = 0.938; ave = 0.717; µ = 4.872; σ = 1.001 iq1 0.833 iq2 0.879 iq3 0.898 iq4 0.868 iq5 0.857 australasian journal of information systems prakash & das 2024, vol 28, research article (why) do we trust ai? 41 iq6 0.737 perceived anthropomorphism (pa) cr = 0.873; ave = 0.579; µ = 3.373; σ = 0.626 pa1 0.768 pa2 0.795 pa3 0.736 pa4 0.788 pa5 0.716 perceived privacy risk (pr) cr = 0.945; ave = 0.852; µ = 4.252; σ = 1.487 pr1 0.916 pr2 0.946 pr3 0.908 perceived service quality (sq) cr = 0.896; ave = 0.683; µ = 4.979; σ = 0.969 sq2 0.788 sq3 0.849 sq4 0.777 sq5 0.888 perceived system quality (sy) cr = 0.924; ave = 0.754; µ = 5.673; σ = 0.945 sy1 0.839 sy2 0.917 sy3 0.883 sy4 0.831 trusting beliefs (tb) cr = 0.967; ave = 0.854; µ = 4.472; σ = 1.249 tb1 0.911 tb2 0.916 tb3 0.933 tb4 0.925 tb5 0.936 trust signs (ts) cr = 0.860; ave = 0.607; µ = 4.395; σ = 0.921 ts1 0.744 ts2 0.791 ts3 0.740 ts4 0.837 intention to use (iu) cr = 0.984; ave = 0.953; µ = 4.554; σ = 1.689 iu1 0.974 iu2 0.973 iu3 0.983 note. cr, composite reliability; ave, average variance extracted; µ, mean; σ, standard deviation australasian journal of information systems prakash & das 2024, vol 28, research article (why) do we trust ai? 42 appendix 4. discriminant validity dt ex hr iq pa pr sq sy tb iu dt ex 0.401 hr 0.349 0.432 iq 0.433 0.749 0.508 pa 0.252 0.647 0.550 0.685 pr 0.397 0.334 0.544 0.345 0.367 sq 0.458 0.774 0.409 0.806 0.685 0.390 sy 0.379 0.681 0.227 0.730 0.580 0.332 0.801 tb 0.468 0.739 0.701 0.759 0.759 0.433 0.750 0.583 iu 0.463 0.673 0.685 0.739 0.699 0.545 0.639 0.547 0.796 ts 0.227 0.556 0.274 0.574 0.447 0.173 0.580 0.497 0.511 0.370 note. as per the htmt criteria, all the htmt values should be less than 0.85 to confirm discriminant validity (hair et al., 2019). appendix 5. variance inflation factor (vif) values hr pr tb iu dt 1.256 ex 2.506 hr 2.077 iq 2.837 pa 1.789 pr 1.445 sq 2.87 sy 2.178 tb 1.261 1.261 1.841 ts 1.261 1.261 1.415 australasian journal of information systems prakash & das 2024, vol 28, research article (why) do we trust ai? 43 copyright: © 2024 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v28.4235 http://creativecommons.org/licenses/by-nc/3.0/au/ https://doi.org/10.3127/ajis.v28.4235 1 introduction 2 literature review 2.1 prior research on ai-based health chatbots 2.2 theoretical background 2.3 contextualization for trust in aicsd 3 hypotheses development 3.1 trusting beliefs and behavioral intentions to use 3.2 disposition to trust technology and trust 3.3 trust signs and trust 3.4 perceived explainability and trust 3.5 perceived information quality and trust 3.6 perceived system quality and trust 3.7 perceived service quality and trust 3.8 perceived anthropomorphism and trust 3.9 trusting beliefs and risk beliefs 3.10 impact of trust signs on risk beliefs 3.11 risk beliefs and behavioral intentions to use 3.11.1 perceived privacy risk and behavioral intentions to use 3.11.2 perceived health risk and behavioral intentions to use 4 research methodology 4.1 study design, procedure, and participants 4.2 survey instrument 5 results 5.1 structural equations modeling analysis 5.1.1 measurement model assessment 5.1.2 assessing potential common method bias 5.1.3 structural model assessment 6 discussion 6.1 theoretical contributions 6.2 implications to practice 6.3 limitations & future research direction appendix 1. respondent demographics appendix 2. measurement items appendix 3. reliability and validity statistics paper title (paper title style) australasian journal of information systems parker, agustiono, carr & saundage 2019, vol 23, research article recruiting farmers into is design 1 conceptualising the relationship types and attributes that enable stakeholders to recruit sme farmer users into is design craig m. parker deakin university craig.parker@deakin.edu.au wahyudi agustiono university of trunojoyo madura rodney carr deakin university dilal saundage deakin university abstract it can be difficult for organisations which develop an information system (is) for use by many small and medium enterprises (smes) to recruit sme personnel during is design. the paper addresses this problem by exploring the nature of relationships that organisational stakeholders can use to recruit sme personnel during is design, which has received little attention in the literature. we present an interpretive, revelatory case study of the insights from managers and field officers who recruited sme farmers during the design of an interorganisational is. we identified three relationship types, based on an existing framework in the literature derived from stakeholder theory: between managers of organisations; between managers and field officers; and between field officers and farmers. we extend this framework by incorporating relationship attributes based on social capital concepts: ties, shared cognition, structure, homophily and bridging capital. we found that the complex interplay of the three relationship types, and the degree of strength of the social capital attributes of these relationships, help explain how farmers were recruited into, or were discouraged from participating in, a lengthy is design process. keywords: small and medium enterprises, farmers, information systems design, social capital, stakeholder theory, user recruitment 1 introduction most businesses in many economies are small and medium enterprises or smes (cragg et al. 2011), which in our australian context are defined as businesses with fewer than 200 staff (abs 2015). governments and other types of organisations develop information systems (is) to support smes. for example, governments may develop decision support systems (dss) to help farmers optimise land use (miah et al. 2014), or develop websites which save smes time to access services and achieve compliance (holgersson 2014). organisations develop dss for use by smes, including farmers, in developing countries in africa (doerflinger & dearden 2013; oduor et al. 2018) and south east asia (noor & nordin 2012). these is can fail due to a lack of sme adoption (eastwood et al. 2017; heeks 2008), which emphasises that it is important for the is academic and practitioner communities to address this issue. australasian journal of information systems parker, agustiono, carr & saundage 2019, vol 23, research article recruiting farmers into is design 2 a factor supporting is adoption is sme involvement during is design (holgersson 2014), because it helps ensure an is tool meets sme needs (heeks 2008; lynch & gregor 2004). the challenge, however, is that most sme owners are time poor and focus on their business (cragg et al. 2011; harwood 2011). in australia, most smes are small firms with 19 or fewer staff (abs 2015), and are independent decision-makers (parker et al. 2015). farmers in developing countries can distrust is designers (heeks 2008). this means is designers can find it hard to recruit smes into an often long design process. making matters worse, smes are heterogeneous (parker et al. 2015), so that is designers must recruit a diverse range of smes to increase the chances that disparate smes will adopt the is (holgersson 2014). the problem we address in this paper is how organisations can capitalise on people who have existing, or can develop new, relationships with sme personnel to recruit them into is design. a literature review by parker et al. (2015) shows that research on smes and is tends to focus on is adoption and use, rather than is design. we show in section 2 that the few studies reporting on sme participation in is design offer little insight into how sme personnel are recruited and by whom. the limited work on recruiting sme personnel in is design suggests that recruitment is not trivial and that three relationship types help with recruitment, but offers little insight into these relationships: 1) recruiter relationships involve recruiters from organisations who use existing, or develop new, relationships with sme personnel to recruit the personnel; 2) enlister relationships are existing/new relationships between managers and recruiters, which enable recruiters to be enlisted into an is design project; and 3) manager relationships are existing/new relationships among managers of organisations that lead to new organisations joining an is design project, where these new organisations may employ or have access to potential recruiters. we explore each relationship type via a revelatory and interpretive case study of how a government-led team recruited sme farmers when designing an inter-organisational is that we call the farm planning tool or fpt (pseudonym). we examined relationships among managers of organisations, some of whom had relationships with and enlisted field officers, which led to these field officers, who had relationships with farmers, to recruit farmers. we identified attributes of the relationship types that explained how managers involved new managers, how managers enlisted field officers, and how field officers recruited farmers. relationship attributes are characteristics of a relationship, including the duration, and are traits of each member in the relationship that affects the relationship, such as the degree of similarity in knowledge/experience between the field officers and farmers. in this paper we thus address the research question: what attributes of relationships among managers, field officers and farmers help explain how farmers were recruited during fpt design? the paper, in addressing this question, makes the following contributions. first, we fill a knowledge gap by presenting empirical evidence of how the three relationship types are interconnected, and enable managers and recruiters to recruit sme personnel into is design. the few existing studies in this area only provide superficial insight. second, we show that social capital concepts can help theorise the attributes of the three relationship types, and help explain how managers and recruiters can recruit farmers into is design. third, existing studies relating to smes and is typically use social capital concepts to study is adoption/use by smes, or is design in an intra-organisational context. this paper instead provides empirical evidence that these concepts can also be used to theorise about recruiting sme personnel in is design, which is prior to adoption/use, in an inter-organisational context. fourth, we extend boonstra’s australasian journal of information systems parker, agustiono, carr & saundage 2019, vol 23, research article recruiting farmers into is design 3 (2009) is stakeholder management framework to address the neglected issue of how to recruit users during the is design phases of an is project by adding the three relationship types and their associated attributes based on social capital concepts. a practical contribution of the paper is to highlight the importance of the three relationship types and their associated attributes when recruiting users into is design. is design teams can use this insight to formulate strategies that can enable the emergence of desirable relationship types and attributes, and/or identify existing relationships with attributes most likely to recruit sme personnel. this is valuable because literature evidence confirms (e.g. lynch & gregor 2004) that sme involvement in is design can increase is adoption. the paper is structured as follows. section 2 synthesises the limited literature which is relevant to the paper’s research question. section 3 summarises theoretical concepts relating to the three relationship types and their associated attributes. section 4 summarises the revelatory case study and how the data was collected and analysed to address our research question. section 5 uses concepts from section 3 and presents evidence relating to the three relationship types, and associated attributes, that led to farmers being recruited. section 6 discusses the paper’s contributions. section 7 concludes with the limitations, practical contributions and future research agenda arising from this study. 2 literature review in this section we review empirical work that reports on designing an is tool intended for use by many smes. specifically, we review studies about existing/new relationships among stakeholders that led to sme personnel being recruited into is design. we review studies in developed countries (e.g. australia, usa and china) reporting on, for instance, the design of web portals for use by rural/regional smes (e.g. fisher & craig 2005) and decision support systems (dss) for farmers (e.g. breuer et al. 2009; cabrera et al. 2008; huang et al. 2010; jakku & thorburn 2010; miah et al. 2014). we also review studies in developing countries, including kenya (gichamba & lukandu 2012; oduor et al. 2018), ethiopia (zewge et al. 2014; 2015), tanzania (churi et al. 2013), south africa (doerflinger & dearden 2013; merz et al. 2007), tunisia (ndou et al. 2011), malaysia (noor & nordin 2012), and bangladesh (islam & grönlund 2007). studies in developing countries often report on the design of dss for use by farmers, including mobile phone systems. we focus on studies where sme personnel were recruited to participate actively in, for instance, in-depth interviews, workshops and focus groups, because this can increase the success of is adoption/use in developed countries (holgersson 2014; lynch & gregor 2004) and developing countries (heeks 2008). active methods imply that stakeholders need existing or new relationships with other stakeholders, and with sme personnel, to recruit sme personnel, and that the relationships will have attributes to support/hinder recruitment such as common norms and trust. the literature suggests there are three broad relationship types among stakeholders, shown in figure 1 and explained next, that enable the recruiting of sme personnel in one or more is design stages. we review the literature that implies the importance of each type of relationship, and that certain attributes might enable one party in the relationship to recruit the other party. a relationship attribute, in the context of this paper, is a trait that can be compared between each party in a relationship, including degree of similarity in knowledge/ experience, or a characteristic of the relationship itself including duration. the attributes affect if or how one party can recruit the other via an existing or new relationship. we will give examples of the attributes that are evident in the literature in the sections that follow. australasian journal of information systems parker, agustiono, carr & saundage 2019, vol 23, research article recruiting farmers into is design 4 figure 1: relationship types enabling sme recruitment into is design stages 2.1 recruiter relationships: recruiters and sme personnel recruiter relationships are where recruiters develop new, or use existing, relationships with sme personnel to recruit smes into is design. this paper does not explore sme personnel who enlist other sme personnel, which has been found in the literature (e.g. cerf et al. 2012; zewge et al. 2014). instead, we focus on recruiters, who are not sme personnel, and what relationships recruiters have or develop with sme personnel, because this is the first step so that, later on, any recruited sme personnel can enlist other sme personnel. it was not clear in many studies who the recruiters were that recruited smes (breuer et al. 2009; cerf et al. 2012; churi et al. 2013; gichamba & lukandu 2012; islam & grönlund 2007; merz et al. 2007; miah et al. 2014; parker & sinclair 2001; zewge et al. 2014; 2015). fisher and craig (2005), salmivalli et al. (2008) and tatnall and burgess (2009) reported on failed is projects and found that smes had little involvement in is design, and did not state how representatives were recruited to steering committees nor whether the representatives were smes. researchers coordinated some participatory methods (cabrera et al. 2008; huang et al. 2010; ndou et al. 2011; nolan et al. 2007; noor & nordin 2012), but did not state if researchers such as doctoral students or employed research assistants, or other stakeholders such as staff of third-party organisations, were the ones who recruited sme personnel. dans and freire (2002) said two researchers had existing relationships with fishermen and fisher communities prior to designing an is for use by fisheries and restaurants in spain, but did not state if the researchers or other stakeholders were the ones who recruited sme personnel into the is design project. only a few studies offer any insight into who were the recruiters, and what relationship attributes enabled recruiters to recruit sme personnel. one recruiter type is researchers. oduor et al. (2018) and jakku and thorburn (2010) stated that they, as researchers, had prior relationships with farmers during an earlier project that helped them to recruit the same farmers into a new is design project. breuer et al. (2008), by contrast, stated they were given farmer contact details by other stakeholders, implying they had no relationships with farmers prior to recruitment. this implies that two recruiter relationship attributes were: 1) whether the relationship between recruiters and sme personnel existed prior to the is design project, and 2) the duration or length of any pre-existing relationship. the limited studies to-date means it is unclear what impact these attributes have on recruitment. these studies offer some limited insight into other possible relationship attributes. for example, breuer et al. (2008, p. 389) said that recruiters were “… graduate students from a number of backgrounds including anthropology, agronomy, agricultural extension, natural resource management, forestry, geography, and community development”. breuer et al. did not state why this background was important, but it implies that such background is a relationship attribute that helped students recruit farmers because they would have common knowledge. oduor et al. (2018) also did not explain the relationships that the researchers had with the secretariat (who were farmers) of the irrigation scheme in a kenyan community. oduor et al. said the research australasian journal of information systems parker, agustiono, carr & saundage 2019, vol 23, research article recruiting farmers into is design 5 team included kenyans and that the “… first author had lived in kenya for a combined period of ~25 years... [and undertaken] research in user practices around water management and use in remote parts of arid and semi-arid northern kenya” (p. 4). oduor et al. did not explain why this background was important, but it implies that researchers’ local knowledge/experience was needed to recruit farmers. oduor et al. emphasised the need for is scholars to interact “… with farmers to understand internal politics, local dynamics, and historical changes…” (p. 8) to ensure is designs help farmers. breuer et al. and oduor et al. thus imply that an important relationship attribute is common knowledge/experience between recruiters and sme personnel, because it helps ensure that recruiters understand the farmers’ context. these studies suggest that if there is more common knowledge/experience, farmers are more likely to see the recruiters as credible and thus agree to participate in is design. oduor et al. also explained that the research team developed and maintained relationships with village elders, who can be farmers and can in turn espouse the team’s credibility to other farmers. this suggests that a related relationship attribute may include ongoing trust with village elders, that must be developed over time, so that elders will in turn facilitate the recruiting of other farmers in their local community. this is consistent with is studies in developing countries which do not explicitly report on designing is for farmers. for example, heeks (2008) and winschiers-theophilus et al. (2012) emphasise that western is scholars must know local community cultures, because our knowledge/experiences, including technical know-how and no experience of poverty, reduces our credibility/trust with, and thus ability to recruit, farmers, etc. further, heeks and winschiers-theophilus et al. suggest that recruiting farmers would be essential because community members including farmers, rather than researchers, should drive is design. recruiters can include other types of stakeholders, not just researchers and their students. for example, doerflinger and dearden (2013) outline a methodology for conducting is design projects for communities in developing countries. the methodology involves buying agents from local organisations, and volunteers from foreign aid organisations, working as recruiters, because they have pre-existing and trusted relationships with sme personnel. this is consistent with the relationship attribute apparent between researchers and smes that we noted earlier. doerflinger and dearden’s methodology does not, however, explain other relationship attributes that aid recruitment. for example, they do not explain the type of foreign aid work that local volunteers engage in, whether the volunteers are westerners or from the same country, nor how long the relationships have existed between volunteers and sme personnel. these details are needed to understand the extent to which shared experience/ knowledge is needed by recruiters to convince sme personnel to participate. in summary, our review suggests that recruiters can be researchers, students and volunteers from not-for-profit organisations, and that existing/new relationships between recruiters and sme personnel are needed to recruit sme personnel into is design interviews, workshops, etc. the review emphasises, however, that there is limited insight into the relationship attributes which might explain the success or otherwise of the recruitment. 2.2 enlister relationships: between managers and recruiters enlister relationships are where managers develop new or use existing relationships with recruiters. managers use enlister relationships to identify, and enlist, recruiters who can then in turn recruit sme personnel via recruiter relationships. there is some evidence in the literature that enlister relationships are important but, similarly to recruiter relationships, the literature offers limited insight into the attributes of enlister relationships. australasian journal of information systems parker, agustiono, carr & saundage 2019, vol 23, research article recruiting farmers into is design 6 the majority of studies cited in section 2.1 do not provide evidence of enlister relationships because they do not clarify or provide information about how smes are recruited. instead, the importance of enlister relationships, and attributes, is evident in the literature that explicitly indicates that recruiters were researchers (breuer et al. 2008; jakku & thorburn 2010; oduor et al. 2018). for example, in section 2.1 we said the breuer et al. (2008) study involved graduate students being recruiters. this means the researchers must have enlisted students (i.e. enlister relationship), and students enacted the recruiter relationships with sme personnel. breuer et al. did not clarify what teacher-student, or enlister, relationship attributes were needed, including if researchers made student involvement mandatory. we argue that such details may help explain the conditions that are needed to facilitate enlister relationships successfully. similarly, oduor et al. (2018) implied that enlister relationships were important because the authors, from ibm research africa, acknowledged that other work colleagues from ibm research africa were responsible for “… organizing the study logistics and facilitating continuous association with the participants” (p. 8). this suggests the authors had a managerial role in the project and enlisted colleagues, while the authors’ colleagues were the recruiters. oduor et al. imply a relationship attribute was the collegiality at ibm research africa that enabled oduor et al., as managers, to enlist colleagues (i.e. enlister relationship), who then recruited farmers (i.e. recruiter relationship). the methodology that doerflinger and dearden (2013) outline for conducting is design projects for communities in developing countries also implies enlister relationships are important. for example, managers in foreign aid organisations, and perhaps other managerial stakeholders, would likely be the ones who enlisted volunteers who, as explained in section 2.1, then became recruiters. doerflinger and dearden do not, however, clarify who enlisted the volunteers, nor the relationship attributes that the managers had with volunteers (i.e. enlister relationship) that leads to volunteers enacting recruiter relationships. in summary, these examples suggest that understanding enlister relationships, and their attributes, may be important to help researchers and organisations determine how to enact enlister and recruiter relationships so that sme personnel can be recruited successfully. our review highlights, however, that enlister relationships, and their attributes, have been largely overlooked in existing research into how to recruit sme personnel into is design. 2.3 manager relationships: among organisational managers manger relationships involve managers, or representatives, of organisations developing new or using existing relationships with managers/representatives of other organisations. these relationships involve negotiations and coordination that facilitate the enlister and recruiter relationships, and can include bringing in organisations with access to potential recruiters. most studies provided little or no insight into manager relationships. some do not report on the relationships between researchers and managers of other organisations that facilitated enlister and recruiter relationships (cerf et al. 2012; churi et al. 2013; gichamba & lukandu 2012; huang et al. 2010; jakku & thorburn 2010; merz et al. 2007; miah et al. 2014; ndou et al. 2011; noor & nordin 2012). some studies stated the organisations involved, but not the roles each organisation had regarding direct or indirect sme recruitment (dans & freire 2002; islam & grönlund 2007; merz et al. 2007; oduor et al. 2018; parker & sinclair 2001). there was evidence in the literature that manager relationships and associated attributes are important, but some studies did not clarify how manager relationships led to enlister and/or australasian journal of information systems parker, agustiono, carr & saundage 2019, vol 23, research article recruiting farmers into is design 7 recruiter relationships. for instance, some studies stated that organisational managers or representatives had existing, or developed new, relationships to form steering committees or consortia (breuer et al. 2008; breuer et al. 2009; fisher & craig 2005; salmivalli et al. 2008; tatnall & burgess 2009). of these studies, only fisher and craig (2005) and salmivalli et al. (2008) outline the evolving political issues that affected the relationships among committee/ consortia members, and the effect on is design, but they did not state if committee/consortia members had or developed relationships with recruiters (i.e. enlister relationships). this implied that manager relationship attributes include structure (e.g. committees), formality (e.g. contracts, funding), and power among managers of organisations. nolan et al. (2007) stated how researchers developed relationships with managers of sme business networks by discussing the business case for a new is to be used by smes, implying common interest was an important attribute in manager relationships. cabrera et al. (2008) stated the researchers contacted government agencies to get the contact details of farmers, which implied that researchers had or developed relationships with agency managers. zewge et al. (2015) said the researchers developed “official” relationships with managers in government agencies (i.e. manager relationships) as part of the research design. nolan et al., cabrera et al. and zewge et al. offered no further, clear details about recruiting sme personnel that enabled us to determine how manager relationships led to enlister and recruiter relationships. doerflinger and dearden (2013) state how manager relationships can lead to enlister and recruiter relationships. their methodology for is design in developing countries involves manager researchers developing relationships with local partner organisations that are formalised into contracts, while no contracts are used with local contacts including foreign aid organisations. this implies a manager relationship attribute is varying degrees of formality, which would then facilitate the enlister and recruiter relationships that are explained in the preceding sections. nonetheless, they provide no further details of manager relationships to determine other attributes that enable these relationships to succeed. in summary, these examples suggest that understanding manager relationships, and their attributes, may be important to help other is scholars determine how to enact manager relationships, which then facilitate enlister and recruiter relationships. our review suggests, however, that manager relationships, and their attributes, have been largely overlooked in prior research examining how to recruit sme personnel to participate actively in is design. overall, we have shown there is little research that analyses in detail how the three relationship types, and associated attributes, can lead to recruiting sme personnel into an often lengthy is design process. this is essential because empirical evidence in the literature shows that sme involvement can increase is adoption by smes (e.g. breuer et al. 2009; doerflinger & dearden 2013; lynch & gregor 2004). the next section examines the theoretical concepts that had potential to explain these relationship types and attributes. 3 theoretical background the literature cited in section 2 does not use theories or concepts relevant to this paper’s research question on the relationships among managers, recruiters and sme personnel, nor on the attributes that explain how the relationships lead to sme personnel being recruited into is design. this is not surprising because we argued that most studies about designing is for use by many smes do not examine these relationships, and associated attributes. australasian journal of information systems parker, agustiono, carr & saundage 2019, vol 23, research article recruiting farmers into is design 8 we found boonstra’s (2009) is stakeholder management framework, derived from stakeholder theory, to be a useful overarching framework for our study for three reasons. first, this framework identifies various stakeholder types involved in is design projects, including project owners, project managers, developers and users. is scholars often use stakeholder theory as an analytical framework to identify, analyse, categorise and prioritise stakeholder types in an is project, including relations and tensions among stakeholder types (pouloudi et al. 2016). in our study we focused on stakeholders who were managers in the manager and enlister relationships, and recruiters in the enlister and recruiter relationships. the second reason boonstra’s framework was useful is because it categorises all stages of an is project, where the two most relevant stages in this paper are the concept and development stages. the concept stage includes all activities prior to any programming that leads to a concept design, including eliciting user needs. the development stage involves using the concept design to create multiple prototype versions based on stakeholder evaluations. this latter stage ends when project leaders feel, in the case of prototyping, it specifies an is enough to develop a fully-functional is for deployment during an implementation stage. in our study we examined how the three relationship types, and associated attributes, can lead to many sme personnel being recruited into the concept and development stages. many of the studies cited in section 2 emphasise that sme personnel need to be involved in both stages to increase the chances that the resulting is tool will be adopted by smes. third, boonstra’s framework specifies a continuum of stakeholder involvement ranging from passive, or being informed about the is or consulted for opinions, to active involvement that increases in degree from influence on, or collaboration with other stakeholders, or to being responsible for one or more aspects of the is. in our study we focused on how the three relationship types led to sme personnel being recruited in order to influence an is design. addressing this paper’s aim required theorising about the relationship attributes that help explain how the three relationship types contribute to sme personnel being recruited into an is design project. we needed another theory to complement boonstra’s work, because the framework did not offer sufficient concepts to analyse the stakeholder relationships. social capital had the potential to complement boonstra’s framework because it posits that people can gain value or achieve their objectives via face-to-face and online social networks (parker & castleman 2009). the objective for us was recruiting smes via the relationship types. social capital supports analysing a social system to identify actors who have relationships, which is a process consistent with our goal of examining relationships. table 1 summarises the interrelated social capital concepts. actor network theory (tatnall & burgess 2009) and stakeholder theory (pouloudi et al. 2016) posit that stakeholder relations can be based on power, but this attribute is part of the strength of ties in social capital. the concepts from social capital appeared to suggest attributes of relationships that might facilitate sme personnel being recruited into is design. our review in section 2 provided some initial support for our conclusion. for example, structure had potential to theorise if recruiters have prior relationships with sme personnel (i.e. recruiter relationships), managers have a working relationship with recruiters (i.e. enlister relationships), and managers have existing or need to develop new relationships with managers of other organisations (i.e. manager relationships). ties had the potential to theorise the duration of prior relationships, the trust that can develop over time in the three relationship types and, in the case of manager relationships, the degree of formality such as the use of contracts. homophily had the potential australasian journal of information systems parker, agustiono, carr & saundage 2019, vol 23, research article recruiting farmers into is design 9 to theorise the degree of common knowledge/experience among those in all three relationship types. shared cognition had the potential to make sense of why the three relationship types can be difficult to develop, including when is researchers have different worldviews to farmers in developing countries. despite the potential of these social capital concepts, the limited research into the three relationship types meant further in-depth research was needed to examine if these attributes explained how relationships among stakeholders helped recruit sme personnel into is design. attribute concept/attribute description ties stronger ties nahapiet and ghoshal (1998) call it ‘relational dimension’) involve actors having interpersonal, direct relations characterised, for instance, by reciprocity, long duration, frequency, trust, shared obligations and power. for instance, families have very strong ties. weaker ties involve low intimacy, trust, duration, etc, and absent ties involve no direct actor relations (granovetter 1973). structure the extent/number, or patterns, of connections in a social network which enable an actor to be reached by, and to reach other, actors. structural holes can occur where many actors in the social network are disconnected from, or have no/weak ties to, other actors in the same network (nahapiet & ghoshal 1998). shared cognition the degree to which actors in a social network share, for instance, common language, codes, narratives, interests and values (nahapiet & ghoshal 1998). homophilyheterophily a continuum from people being homophilous (e.g. common behaviour, practices, status, demographics, etc) and forming stronger ties at higher rates, relative to the other extreme: very dissimilar or heterophilous (mcpherson et al. 2001). bridging capital the degree to which resources can be brought into existing relationships from external social networks (putnam 2000). this is affected by structural holes when actors in a social network are disconnected from, or have no/weak ties to, actors in external networks (nahapiet & ghoshal 1998). table 1: social capital concepts (attributes of the relationship types) 4 research approach we needed a research approach for examining the design of an is intended for use by smes so that we could explore in detail the relationships that enabled managers and recruiters to recruit smes. relationships, and associated attributes, are subjective and depend on sociohistorical contexts, which implied a qualitative and interpretive approach was best (walsham 1995). yin (2014) explains that a single case is suitable when the study is revelatory in nature, which applied here due to the limited research into how the three relationship types, and associated attributes, lead to sme personnel being recruited into is design. we had access to a revelatory case study where sme farmers were recruited into the concept and development stages of an inter-organisational, web-based is called the farm planning tool (fpt, pseudonym). field officers from a government agency and a private organisation had worked closely with farmers prior to the project, and recruited farmers into the fpt design. the next sections introduce the case study, and the data collection and analysis approach, which helped us address the question: what attributes of relationships among managers, field officers and farmers help explain how farmers were recruited during fpt design? 4.1 overview of the revelatory case study the case study that addressed the research question related to the design, from 2005 to 2007, of the fpt for farmers. the fpt was an inter-organisational web-based is with aerial, salinity australasian journal of information systems parker, agustiono, carr & saundage 2019, vol 23, research article recruiting farmers into is design 10 and other maps that farmers could use to monitor and manage farm activity and reduce environmental impact. the organisations involved during the project were: • the department of environment (abbreviated to deptenv) was a government agency in an australian state or province, which protected public land such as parklands and forests from ecological damage, including from activities by adjacent private farms. the deptenv provided funding for the fpt design project. • four state catchment authorities (sca1, sca2, sca3, sca4), which were australian state government agencies each charged with protecting public waterways such as rivers, including fertiliser run-off from farms, in different regions in the state. the scas ran the project. • the department of land (deptland), which was a government agency charged with improving farming practices to achieve higher productivity, including initiatives relating reducing ecological damage by farmers if there was a link to productivity. • spatialco, which ran design workshops and developed the fpt prototype. • dairyco, which was a cooperative of independent dairy farmers, and became a new member of the project team in later stages of the fpt design. prior to the start of the fpt project, government restructures meant deptenv and deptland and their field officers could no longer help farmers with reducing environmental impact. they therefore were not formally involved in the fpt design project, but continued to be involved indirectly. deptenv, as noted above, provided funding for most fpt design stages. the government agencies aimed to offer farmers with help, such as developing software like the fpt, to reduce their environmental impact by better managing farming activity such as fertiliser use, nutrient loss and effluent management. this was critical because there were over 35,000 farms in the state at the time, all potentially causing ecological damage. the agencies employed their own field officers, or extension officers, to work with farmers. the impetus for designing the web-based fpt was that an earlier project, where farmers used paper worksheets to plan reduced environmental impacts of farms, did not meet the needs of farmers, nor the needs of agencies wanting to aggregate information to report against government targets. the fpt project team knew it was critical to involve farmers actively during the design because, at the time, web-based spatial tools (e.g. google maps), were new and unknown to most farmers. the team selected farmers interested in reducing their environmental impact, and more likely to participate in months of fpt design cycles. in addition to developing an is to help farmers manage farm activities, the project team needed the fpt to be designed so that agencies could access information entered by farmers, and to aggregate that information to monitor and report on progress toward government targets on reducing environmental impacts. the fpt was thus an inter-organisational is involving bidirectional information sharing between farmers and agencies. also, the field officers of each agency needed the fpt to be designed so they could use it to encourage farmers to implement farming practices which aligned with government targets. table 2 summarises the three types of intended users of the fpt and their basic needs. australasian journal of information systems parker, agustiono, carr & saundage 2019, vol 23, research article recruiting farmers into is design 11 user types information inputs information outputs government managers specify targets to reduce environmental impact, conditions of natural resources, provide recent spatial maps for use by farmers. aggregate farm data for reports and planning at all state and catchment/regional levels. field officers review and approve farmer action plans. entry of farm details on behalf of farmers if required. aggregate farm data, develop summary reports on activities undertaken (e.g. farms visited, outcomes reviewed, activities for follow up), create maps to assist with planning/activities. farmers input farm data including paddock details (e.g. layout), details on initiatives to reduce environmental impact (e.g. action plans, targets, proposed activities, support requests, locations with issues). produce farm paddock map, report on activities (e.g. fertiliser inputs, progress on targets), view benchmark against other farms, measurements (e.g. fencing). table 2: intended fpt users and information inputs/outputs the project involved over 50 farmers during various design stages, but with challenges. for instance, most farms in australia are small, defined as land area of 500 hectares or less (abs 2012), rather than defined by staff numbers, and most farmers in the fpt project had farms under 1000 hectares. the exact farm sizes for participants are unknown. farms have typical sme traits, including farmers being time poor, having decision-making power (abs 2012), and farms being heterogeneous in terms of is expertise, crops, livestock (e.g. dairy, beef and mixed crop farms), sizes ranging from a few to 1000 hectares, hobby farms or commercial farms, and farmer gender/age (abs 2012). the fpt team succeeded in recruiting and involving diverse farmers in is design over many months, as explained further in section 4.2. the fpt team claimed the project was successful on three grounds. first, 1,350 farmers from eight sca regions had fpt accounts by august 2009. of these farmers, 832 used the fpt to record basic property details and boundaries, and 696 created paddock boundaries. this was despite the fpt still being a prototype because it was not released, was promoted mainly via word-of-mouth, and was only a proof-of-concept. second, the fpt team reported that, overall, farmers found the fpt usable and useful based on evaluations involving interviews and surveys of farmers who were involved during the design. third, the proof-of-concept achieved by the fpt project was used as input into a new project to develop standards, which thirdparty is vendors can use to exchange spatial and farming data with agency maps. 4.2 data collection and analysis the data collection and analysis used to answer the research question in this paper was part of a larger study conducted from june 2012 to december 2013. we used the framework by boonstra (2009), summarised in section 3, to identify managers and recruiters who recruited farmers into the concept and development stages of fpt design. a hermeneutic process was used to collect and analyse interviewee recollections (klein & myers 1999). the managers and recruiters we interviewed were identified using snowballing by asking each interviewee who else to contact, because it was not known prior to data collection who recruited farmers and how. heckathorn (2011) states that snowballing is a suitable method to study social network construction. this aligned with our aim of studying the recollections of managers and recruiters about whether and how they recruited farmers. for example, using this method it was determined that: australasian journal of information systems parker, agustiono, carr & saundage 2019, vol 23, research article recruiting farmers into is design 12 • field officers were the main recruiters who recruited farmers during the concept and development stages of the fpt project; • spatialco was the is vendor during both stages, and individual spatialco staff ran workshops with farmers, but they were not involved in recruiting farmers; • deptland managers initially helped during the project, but later encouraged its field officers to recruit farmers to use an alternative tool to the fpt prototype; and • dairyco managers joined the fpt project during the development stage. dairyco field officers, or recruiters, were the ones who recruited dairy farmers during an evaluation of the fpt prototype. we continued the snowballing approach until we identified all organisational managers and recruiters who were available to participate in the study. table 3 lists the interviewees, their main roles during fpt design (and explained further in section 5), and the insights they provided. the underlined text in table 3 identifies recruiters who recruited farmers directly during one or more stages of the fpt design. these interviewees gave access to over 200 documents, including evaluation reports and design documents. interviewees fpt design roles and insights provided project-managersca1 represented one sca (i.e. sca1) on the steering committee; managed the fpt design project; directly recruited some farmers; and had first-hand experience of the project history, stakeholder roles, and how farmers were recruited during the fpt design. spatialco-manager spatialco was commissioned to develop the fpt prototype; ran all design workshops with farmers; developed fpt specification documents; improved the fpt using farmer feedback; and had first-hand experience of the project history, and how farmers were recruited during the fpt design. deptlandresearcher represented a government agency on the steering committee; participated in all fpt design stages; and had a key role in designing the fpt prototype evaluation activities involving sme farmers. deptland-manager provided background details on the fpt project and issues during the design stages which impacted on recruiting farmers. deptland-fellow provided technical background of the fpt and the design stages. field-officer-sca2 provided first-hand recollections of how she and other sca field officers recruited farmers; provided training and support for farmers during the fpt design; and created spreadsheet tools for farmers which complemented the fpt prototype. deptland-fieldofficer1 and deptland-fieldofficer2 supported farmers with farm planning and using software; and provided first-hand recollections of tensions affecting involving farmers during fpt design. deptlandfield-officer1 was initially a project officer and provided fpt training for farmers during the fpt design until tensions emerged. dairyco-manager represented dairyco in the fpt design consortium and steering committee, and gave details of how farmers were recruited during fpt design. dairyco-fieldofficer1 and dairyco-fieldofficer2 provided first-hand recollections of how dairyco field officers recruited dairy farmers; provided fpt training to farmers; and supported farmers with various dairy-related farm management activities (e.g. compliance). table 3: interviewees, fpt design roles and insights provided the recollections of managers and recruiters were elicited using semi-structured interviews to give interviewees the flexibility to provide recollections in their own words (kvale 2007) about recruiting farmers during fpt design. the interviews were audio-taped and transcribed. questions started broadly, including asking about their role during fpt design and how the australasian journal of information systems parker, agustiono, carr & saundage 2019, vol 23, research article recruiting farmers into is design 13 project started, and then asked the interviewees to describe the design steps involving smes. this helped align the steps with boonstra’s two design stages. the questions then focused on when and how the managers and recruiters influenced how farmers were recruited, and if they got other recruiters to invite farmers. we asked managers about the challenges they had with enlisting recruiters, and field officers about the challenges with recruiting farmers. we asked the interviewees how they handled these challenges. table 4 summarises the fpt design steps during the concept and development stages in which farmers were involved, the duration of farmer involvement during the steps, the main organisations involved, and the field officers who recruited farmers. the table shows that many farmers invested a lot of time by taking part in workshops and evaluations over months. table 4 implies that participation numbers decreased as time commitments increased, which emphasised the important role of recruiter relationships, between field officers and farmers, in supporting farmer participation. stage pre-is design both stages concept concept development development step pre-fpt project (paper-based) fpt reference group formed fpt concept design workshops fpt mockup workshops fpt prototype evaluation 1 fpt prototype evaluation 2 funding entities federal govt gave funding. deptenv gave funding. deptenv gave funding. deptenv gave funding. deptenv gave funding. dairyco used its govt. funding. main agencies/ companies involved deptenv, sca1, sca2, sca5*, fa*, deptland sca1, sca2, sca3, spatialco sca1, sca2, sca3, spatialco sca1, sca2, sca3, spatialco sca1, sca2, sca3, sca4, spatialco dairyco, spatialco key issues/ activities farmers used paper booklets/ worksheets to manage environmental impact. farmers found this complex & timeconsuming. agencies found farmer data was not useful for monitoring targets. the fpt was intended to solve the issues. deptenv & deptland left because state govt. restructure meant they could not help farmers with environmental impact. scas took over project. reference group farmers gave input so high-level concept design of the fpt would cater for farmer needs. high-level design was presented to farmers during local workshops arranged by sca field officers. spatialco ran the workshops to show concept design, & farmers gave input. this input was used to create mockup screens of fpt design. farmers took part in workshops arranged by sca field officers. spatialco showed the mock-up to show how it might work, & sought farmer input to help. this input was used to create a detailed fpt prototype specification. deptland tried to stop farmer involvement by encouraging use instead of their own desktop mapping is. sca field officers trained/ supported farmers to use the fpt. farmers joined workshops, interviews & surveys to evaluate fpt. no deptenv funding; no sca field officer time to support more farmers. deptland put dairyco in contact with fpt team. dairyco funded changes they needed. evaluation same as first but longer period & field officer support from dairyco. recruiters 3 sca field officers kept 104 farmers involved. projectmanagersca1 kept 3 farmers involved. sca field officers (farmer data not available). sca field officers (farmer numbers not available). 5 sca field officers kept 46 farmers involved. dairyco field officers kept 18 farmers involved. australasian journal of information systems parker, agustiono, carr & saundage 2019, vol 23, research article recruiting farmers into is design 14 stage pre-is design both stages concept concept development development sme diversity many livestock/ crop & size farms not diverse. many livestock/crop farms; farm sizes; owner ages, gender & is skills; hobby vs commercial farms. dairy farmers. sme involvement duration over 12 months (sca1 – 38, sca2 – 34, sca5* – 32). entire fpt design project. one workshop (farmer data not available). same farmers as prior workshop (farmer data not available). 4 months (sca1 – 10, sca2 – 14, sca3 – 6, sca4 – 16). 18 months. 18 dairy farmers in sca4 region. table 4: summary of fpt design stages and the involvement of farmers wolcott's (1994) description, analysis and interpretation approach was applied to transform the qualitative data into a credible case narrative. analysis and interpretation involved hermeneutic cycles based on the klein and myers (1999) set of principles of interpretive field research. each cycle involved (re-)reading documents and interview transcripts, and coding (sub-)nodes on the chronological processes of the project. a case narrative, or description, was then written, which satisfied the principle of ‘contextualisation’ (klein & myers 1999), by including the socio-historical context of the fpt design. the writing process led to further cycles of analysis and interpretation. the principle of ‘multiple interpretations’ (klein & myers 1999) was satisfied by using hermeneutic cycles of triangulation to mitigate issues with interviewees recalling project details after 5-6 years (patton 2002). this involved comparing their recollections with one another and with fpt project documents. the social capital concepts in section 3, which we used in this paper to make sense of the findings, were identified retrospectively during the hermeneutic cycles of data analysis. this is consistent with other is research, including bødker et al. (2014) and schlichter and rose (2013). the emergent themes about the three relationship types, and the concepts to make sense of the relationship attributes, thus co-evolved with the data analysis. each relationship type became a theme in our analysis, and the relationship attributes were sub-themes. 5 findings figure 2 shows how the three relationship types from section 2, each in a different colour, evolved from the pre-fpt design stage, to the concept and development stages of the fpt design. figure 2 summarises how recruiters were sca and dairyco field officers, and projectmanager-sca1 from the project management group (i.e. recruiter relationships). deptland field officers also had relationships with farmers, but these field officers impeded farmer recruitment during the development stage. figure 2 shows that it was sca, dairyco and deptland representatives or managers who enlisted field officers (i.e. enlister relationships) to facilitate recruiter relationships. finally, figure 2 shows that manager relationships among the scas, deptenv, deptland, spatialco and dairyco evolved throughout the fpt design project. manager relationships thus contextualised how the enlister and recruiter relationships evolved, and the effect on farmer recruitment. in the following sections we explain the findings for each relationship type in more detail. we use the table 1 attributes, based on social capital concepts, to explain how the three relationship types influenced farmer recruitment in the fpt design project. table 5 summarises the attribute levels which emerged from our hermeneutic analysis of the findings. australasian journal of information systems parker, agustiono, carr & saundage 2019, vol 23, research article recruiting farmers into is design 15 we state ‘extrapolated’ when the description of a level was not observed in the findings, but are extrapolated based on other levels which were evident in the findings. the levels varied during is design, and among instances of each relationship type, so that each relationship type can appear in multiple cells in table 5. we use italics in each section below to show how the social capital concepts apply, as used in other is studies (e.g. lyytinen & damsgaard 2011; rodon et al. 2008). figure 2: evolving relationship types during fpt design to recruit farmers levels ties shared cognition homophily structure bridging capital weak no power/trust. short duration/ infrequent. temporary shared obligation/reciprocity prior to is design (rr, er, mr). some general shared language, narratives and values. can align to the is design (rr, mr). some similar practices/ behaviour due to collaborative roles. can align to the is design (rr). few direct links prior to is design (mr). few external resources to support the relationship type (extrapolated). medium medium frequency/ duration. maybe shared obligation/ reciprocity/trust development prior to is design (rr, er, mr). some specific shared language, narratives and values. can align to the is design (rr, mr). many similar practices/ behaviour due to collaborative roles. can align to the is design (rr, er). some direct links prior to is design (extrapolated). some external resources to support the relationship type (rr, mr). high trust, long duration/ high frequency. maybe some shared obligation/ reciprocity/power prior to is design (rr, er). many specific shared language, narratives and values. can align to the is design (rr, er). very similar roles/practices/ behaviour in different contexts. can align to the is design (mr). many direct links prior to is design (rr, er, mr). many external resources to support the relationship type (mr). table 5: attribute levels emerging from the data/evident in the relationship types australasian journal of information systems parker, agustiono, carr & saundage 2019, vol 23, research article recruiting farmers into is design 16 5.1 manager relationships: manager relationships with other managers table 4 shows that the involvement by the scas, spatialco, deptenv, deptland and dairyco evolved during paper-based pre-fpt project, and concept and development stages of the fpt project. table 6 summarises the manager relationship attributes, which shows how relations among managers and representatives of these organisations evolved. we explain each of the three key evolutionary stages in table 6, and their associated attributes, separately next. attribute paper-based pre-fpt project stage concept stage and fpt evaluation step 1 (development stage) fpt evaluation step 2 (development stage) structure high with sca, deptenv, deptland managers due to all attending meetings. weak with sca, deptenv, deptland after restructure. high with sca, spatialco managers (attended meetings). high with sca, spatialco, dairyco managers due to all attending meetings. ties weak weak with sca, deptenv, deptland managers. medium with sca, spatialco manager. weak shared cognition medium weak with sca, deptenv, deptland managers. medium with sca, spatialco managers. medium homophily high high high bridging capital medium medium high table 6: evolution of manager relationships and associated attributes deptenv, deptland and each sca had managers/representatives with direct relationships at consortium meetings (high structure) for the paper-based pre-fpt project. this project evaluated if paper-based worksheets helped farmers make changes to monitor/reduce their negative environmental impact. the managers had medium shared cognition because they shared policy language/narratives about the specific goal of reducing environmental impact by farms, but their agencies focused on different aspects (see section 4.1). the scas focused on public waterways and adjacent farms. deptenv focused on public land and adjacent farms. deptland helped farmers identify and implement productivity innovations, including those which reduced environmental impact. these managers had weak ties due to short meetings during projects, and due to negotiations aligning interests rather than shared obligations or power. these managers had high homophily due to having equivalent positions in their agencies and behaviours, including managing targets, but in different contexts. they had medium bridging capital because, via collaboration and links to the federal government, they collectively secured federal funding for the pre-fpt project. further evidence of medium shared cognition was that the managers reached the collective decision that the fpt design project should overcome the issues faced by farmers, and by staff in their agencies, who used the worksheets (see section 4.1). government restructuring at the start of the project meant deptenv and deptland could no longer help farmers reduce environmental impact. further support for weak ties was that deptenv and deptland managers stopped collaborating after the restructure. this resulted in no meetings (weak structure), and weak shared cognition due to having only the same general policy language/ narrative. spatialco was contracted to run the fpt project. there was medium bridging capital among managers because spatialco offered is-based spatial technology expertise, and deptenv provided project ownership and funding. spatialco and sca managers had high australasian journal of information systems parker, agustiono, carr & saundage 2019, vol 23, research article recruiting farmers into is design 17 structure because all had relationships in meetings. they had medium shared cognition due to common language/narratives about is-based spatial technology, and to spatialco’s work on similar past projects. spatialco and sca managers had medium weak ties due to the scas’ contractual power over many months. homophily was not affected due to all managers having similar management roles and practices, but in different contexts. the high structure and medium shared cognition among the sca and deptland managers during the pre-fpt project led to dairyco joining the second fpt evaluation during the development stage. dairyco and deptland managers had weak ties during a project to help dairy farmers reduce costs and increase productivity, which led to a spreadsheet-based model to reduce fertiliser use. the managers had medium shared cognition due to similar narratives about farm productivity, but dairyco manager narratives included farmers reducing environmental impact. deptland managers suggested that dairyco managers should contact sca managers to ask if the model could be embedded in the fpt. dairyco and sca managers had medium shared cognition due to similar narratives on reducing farmers’ environmental impact, but dairyco managers were driven by productivity and global market issues. the managers had medium homophily due to similar management level positions and behaviours in different contexts. dairyco and sca managers did not have relationships before the fpt project (i.e. no structure and ties). deptland managers contributed high bridging capital, which sca managers needed so they could run fpt evaluation 2 during the development stage, in the form of bringing in dairyco and its resources such as field officers and funding: we invested a little bit of money to try and improve [the fpt], or give it extra functions that we thought were going to be useful, and we then provided, sort of, on-the-ground support [i.e. field officers] through pilot programmes to work with our [farmers] [dairycomanager/interview] in summary, the findings suggest that manager relationships can bring in managers from new organisations that are needed for is design (i.e. new high structure and bridging capital), even with weak ties, if there was medium shared cognition and homophily among the original managers during pre-is and/or is design stages. 5.2 enlister relationships: manager relationships with recruiters we found that sca managers had difficulty enlisting sca field officers to act as recruiters. managers had enlister relationships with employed field officers (high structure), but field officer involvement was voluntarily due to limited funding to pay for this work on top of their normal roles. this meant sca managers had no power over field officers, and that field officers could leave the project, even after starting. a factor contributing to optional involvement was high sca field officer turnover (i.e. weak ties). sca managers relied on high ties and shared cognition to enlist field officers. their high ties had developed over many projects due to shared obligations to reduce farmer impact on waterways. high shared cognition occurred because the field officers’ interests aligned with sca manager policy targets. they had medium homophily due to linked practices/behaviours, where managers worked toward policy targets and field officers supported this through their fieldwork. similar enlister relationship attributes applied to deptland managers and field officers. for example, deptland managers and field officers could have weak to medium ties depending on staff turnover. there were high ties with field officers because both focused on productivity, australasian journal of information systems parker, agustiono, carr & saundage 2019, vol 23, research article recruiting farmers into is design 18 rather than environmental impact, in their employment. there was high shared obligation due to common language, narratives and interests, and medium homophily due to their linked work practices centred around improving farm productivity. similar levels for each attribute applied to dairyco managers and field officers along the same lines. in summary, the findings suggest that enlister relationships, among managers and recruiters, have high structure. managers can enlist recruiters into is design, if participation is voluntary, so long as ties and shared cognition are high, and there is medium homophily. enlister relationships can then lead to recruiter relationships, which are discussed next. 5.3 recruiter relationships: recruiter relationships with sme personnel the next sections explain that recruiters in recruiter relationships can include people in managerial roles in the fpt project (section 5.3.1), not just field officers (section 5.3.2). 5.3.1 project-manager-sca1 relationships with three farmers the findings suggested the relationship between project-manager-sca1 and three farmers, who committed to be on the fpt reference group, were instances of recruiter relationships between a recruiter and sme personnel. the three farmers were proactively exploring ways to improve farming practices, and were perceived as leading farmers: so we had three farmers [in the reference group] who were leading farmers, who previously had been involved in things like precision agriculture, farm mapping. [project-managersca1/interview] project-manager-sca1 had medium shared cognition with the three farmers due to shared language about technology and shared values on using technology to reduce environmental impact, but no shared farming language because project-manager-sca1 was not a farmer. project-manager-sca1 had direct collaborative relations with various stakeholders working on environmental impact and mapping technology. he had high structure with the farmers, because over many years he had worked in related projects, and met and worked with the farmers via his wide social network. project-manager-sca1 had weak homophily with these farmers because, although they had collaborated previously, his agency role and behaviour was quite different to the farmers who ran rural properties. he had medium ties with these farmers due to years of collaboration on projects, through which they developed trust and reciprocity. reciprocity was highlighted by project-manager-sca1 enticing the farmers with medium bridging capital, made possible by his network. for example, project-manager-sca1 organised study trips to universities and farming businesses, paid for the farmers to attend agronomy conferences, and invited the farmers as speakers to share and discuss their experiences and views at conferences. we were not privy to the farmers’ views, but our interviews with other stakeholders confirmed that project-manager-sca1 was considered a leader in the broader network. the farmers were part of this network, so it is plausible the farmers might also have seen him as a leader regarding environmental farming technology. in summary, the findings imply managers can be recruiters. the findings also suggest that recruiter relationships can overcome weak homophily and lead to recruiting sme personnel into reference groups for all is design stages, so long as managers have medium shared cognition, ties and bridging capital, and high structure with these sme personnel. australasian journal of information systems parker, agustiono, carr & saundage 2019, vol 23, research article recruiting farmers into is design 19 5.3.2 field officer relationships with sme farmers table 7 summarises the attributes between sca, deptland and dairyco field officers, and farmers or sme personnel, in recruiter relationships. the table shows that, outside the fpt project, the sca and deptland field officers had high structure with farmers via farmer associations, and that ties and shared cognition ranged from weak to medium. field officers offered medium bridging capital, which farmers valued to varying degrees depending on how they favoured environmental improvements versus productivity goals. field officers offered expertise about farm initiatives and projects to attract government funding. sca field officers (fos) deptland fos dairyco fos attributes of relationship type 1 structure high: fos had direct links to all farmers in environment associations in their region; less outside the associations. high: fos had direct links only to farmers (in associations) focused on productivity/innovation. high; fos had direct links to all farmers supplying to the dairyco cooperative. ties weak to medium based on years in the role, affected by high fo turnover. fpt/fos facilitated farmer control/ access to spatial maps, and project/funding access linked to sca needs (reciprocity). weak to medium based on role duration, affected by high fo turnover. fos: impeded farmer access to spatial maps; facilitated productivity projects/ funding (reciprocity). high; employed to help farmers comply with food regulations, reduce impact on environment (optional), boost yields (duration, reciprocity, trust, shared obligation). shared cognition weak to medium based on if fos grew up on a farm. farm diversity and farmer views of fo academic degrees affected shared cognition. fos targeted farmers reducing their environmental impact. weak to medium based on if fos grew up on a farm. shared cognition was hard due to farm diversity. farmers focused on productivity and innovation were targeted. medium due to greater farmer homogeneity (i.e. dairy only). not all farmers aimed to reduce their environmental impact, which fos tried to encourage. homophily weak; fos helped with projects, but behaviour/ work differed to farmers. weak; fos helped with projects, but behaviour/ work differed to farmers. medium; fos helped with many practices, but did no farming work. bridging capital medium; fos aided access to sca funds, expertise on reducing ecological impact. medium; fos aided access to deptland funds, productivity expertise. medium; fos aided access to dairyco funds, dairyrelated expertise. fpt design stages nature of influence recruited farmers during: • the pre-fpt design stage • the concept stage • the fpt evaluation 1 of the development stage. discouraged farmers from being recruited during: • the fpt evaluation 1 of the development stage. recruited farmers during: • the fpt evaluation 2 of the development stage. table 7: field officer relationships with farmers table 7 implies that nuanced challenges affected the degree to which each type of field officer was able to recruit farmers into the fpt project. this included that deptland field officers impeded the ability of sca field officers to recruit some farmers, especially during fpt evaluation 1 of the development stage, after deptland left the consortium (see table 6 and section 5.1). these nuances and tensions are elaborated on next. the first nuance for sca and deptland field officers, noted in section 4.1, was dealing with the diversity of farm size, farm type (e.g. livestock/crop, hobby versus commercial), and farmer is skills that ranged from basic to proficient. the diversity meant it was difficult for a single australasian journal of information systems parker, agustiono, carr & saundage 2019, vol 23, research article recruiting farmers into is design 20 field officer to have medium or higher shared cognition and homophily with every farmer. this constrained the ability of sca field officers to satisfy the goal of involving many diverse farmers in the fpt development stage (evaluation 1). dairyco field officers had fewer issues because they were employed (high ties) specifically to support an homogeneous type of farm with a common language and similar business goals (medium shared cognition). a second nuance was that our analysis of secondary sources such as government reports and biographies implied that field officers varied on farming background such as being on farms as children, and on the university education that field officers typically needed to have such as agricultural, environmental and natural sciences. this has implications for field officer relationships with farmers, which would then vary depending on the value each farmer placed on the traits of the particular field officers with whom they might engage. this issue potentially weakens shared cognition and strength of ties, and thus possibly the field officers’ ability to convince farmers to participate during the fpt design stages. since we did not interview farmers, we cannot verify if farmers considered this when deciding to join the fpt project. nonetheless, the fact the sca field officers were able to recruit farmers meant these farmers, at the very least, likely respected the field officers’ background/experience. a tension between sca and deptland field officers during fpt evaluation 1 (development stage) emerged due to four interrelated nuances and complications: • section 4.1 noted that deptenv, deptland and the scas had their own, often different, field officers working with farmers to implement changes aligned with each government agency’s portfolio. a single farmer could thus be approached and supported by a field officer from one or more of these agencies. • the farmers varied in how they prioritised reducing environmental impact with other concerns including productivity and their work practices. this created tensions regarding which field officer type each farmer would consider credible. • fpt evaluation 1 required four months of involvement (see table 4), which made it more challenging for sca field officers to convince farmers to join the fpt project. • the manager relationship issue of deptland leaving the project meant deptland field officers used another software mapping tool when visiting farmers, and did not give farmers access and sent printed maps to farmers, instead of promoting the fpt. project-manager-sca1 believed the fpt threatened deptland field officer power, and thus the field officers used their mapping tool and withheld fpt details to maintain this power: so we were very interested in the power relationship between, if you are the [deptland field] officer and i’m the farmer, and you have all the information and i get to see it, but i don't get to... i am not able to interact with it after you go, then the power relationship is unequal. and we were keen to provide more power to the farmers so they could access the information themselves… but some [deptland field officers] found that they had to give up some of their power through this [fpt project], so they were less enthusiastic about it. [project-managersca1/interview] deptland field officers were thus able to discourage farmers from joining fpt evaluation 1, especially if they had greater shared cognition, ties and/or bridging capital compared to what australasian journal of information systems parker, agustiono, carr & saundage 2019, vol 23, research article recruiting farmers into is design 21 farmers had with sca field officers. for instance, if farmers felt productivity outweighed reducing ecological impact, or deptland field officers convinced them, then sca field officers would have had difficulty enticing such farmers to take part in fpt evaluation 1. sca field officers thus relied on relations with farmers from local environment associations (high structure) that they established over time before and during the fpt project (medium ties), and farmers who valued sca funds and expertise (medium bridging capital). these enduring relationships were more likely to give farmers confidence that sca field officers would support farmers during the four month fpt evaluation 1, despite weak homophily. dairyco field officers had fewer issues recruiting farmers during fpt evaluation 2, which lasted eighteen months (see table 4), for two reasons adding to the above points. first, dairy farmers were less likely to be approached by other field officers, because dairyco provided dedicated field officers who covered all dairy work (medium homophily). second, dairyco field officers’ success in recruiting farmers to the evaluation depended on the farmer’s degree of interest in environmental issues, and on their business goals (high ties). the field officers focused on using the fpt to create farm maps if farmers were not interested in the fpt’s environmental management capabilities. sca field officers, by contrast, supported farmers of different types (e.g. crops) and targeted environmental issues as required by their agency. in summary, the findings suggest that recruiter relationships, among recruiters and sme personnel, may lead to sme personnel being recruited into is design if there is medium shared cognition, ties and bridging capital, and high structure. however, this depends on whether other recruiter relationships, with the same or better relationship attributes, are able to impede the recruitment of the same sme personnel, because these recruiters recruit sme personnel into alternative projects, for instance, to maintain their power over smes. 6 discussion we argued that limited research has examined the relationships among stakeholders that can lead to sme personnel being recruited into is design. this research is needed for two reasons. first, sme involvement in is design is an important contributor to later is adoption by smes in developed (lynch & gregor 2004) and developing countries (doerflinger & dearden 2013). second, recruiting sme personnel is not trivial because they can be time poor (cragg et al. 2011) and, in developing countries, may not trust recruiters (oduor et al. 2018). the limited research summarised in section 2 implied there are three relationship types. recruiter relationships are between recruiters and sme personnel, where recruiters use existing/new relationships to recruit sme personnel. enlister relationships involve organisational managers using existing/new relationships to enlist recruiters, who will enact recruiter relationships. manager relationships involve managers from organisations using existing/new relationships with managers of other organisations to facilitate enlister and recruiter relationships. the limited research meant it was unclear what attributes of the three relationship types can explain how sme personnel are recruited into is design, and what theoretical concepts underpin the attributes. we presented the findings of an in-depth revelatory case study about the design of an inter-organisational, web-based spatial is called the farm planning tool (fpt). the fpt collected information from diverse sme farmers about the ecological impact of their farms, which was used by state catchment authorities (scas) and a dairy organisation (dairyco). sca and diaryco managers collaborated, via manager relationships, and enlisted their field officers, via enlister relationships, who in turn recruited sme farmers australasian journal of information systems parker, agustiono, carr & saundage 2019, vol 23, research article recruiting farmers into is design 22 via recruiter relationships. our hermeneutic approach to data analysis enabled us to address the question: what attributes of relationships among managers, field officers and farmers help explain how farmers were recruited during fpt design? in addressing this research question, the paper makes the following contributions. first, boonstra’s (2009) framework conceptualises is stakeholders using stakeholder theory, is project stages, and stakeholder participation as degrees of involvement including having an ‘influence’ on is design. we focused on manager and field officer stakeholders who recruited farmers, or users, into the concept and development stages of an is project. these stakeholders wanted farmers to ‘influence’ the fpt design. this paper extends boonstra’s framework by providing evidence about the importance of adding our three relationship types to the framework to facilitate examination of how stakeholders can recruit farmers, as users, during the concept and development stages of an is project. we also extended the framework by finding empirical evidence that the social capital concepts, which were summarised in table 1 and table 5, can help theorise the attributes of the three relationship types. we shows that the relationship types and social capital concepts can help explain how farmers can be recruited, because we were able to use these to analyse the strength (weak, medium and high) and combination of the concepts to explain the findings. second, we used social capital in the context of the relationship types that lead sme farmers to be recruited into is design, which makes a contribution to our discipline for two reasons: 1. social capital concepts are used mainly to explore sme perspectives of knowledge sharing and value creation among smes using existing is (e.g. mason et al. 2008a; mason et al. 2008b). this paper’s contribution is to show these concepts can explain manager and recruiter perspectives (who are not sme personnel) about recruiting farmers when designing a new is, not an existing is. 2. is design studies use social capital concepts to explore design teams interactions (e.g. chang et al. 2011; chou & he 2011; kotlarsky & oshri 2005; paavola & hallikainen 2016; robert et al. 2008), and snowball techniques to recruit users in large-scale intra-organisational projects (e.g. lim & finkelstein 2012; van limburg et al. 2015). that is, these studies explore relationships among stakeholders in a single organisation. even studies involving smes have used social capital concepts to examine relationships among consultants and sme owner-managers when designing a website for a single sme (e.g. burgess et al. 2012). a contribution of this paper is showing these concepts can also make sense of relationships between is project members and users during the design of an inter-organisational is. third, this study highlights the interrelatedness of the three relationship types when it comes to recruiting sme personnel into is design, and the need to examine them together in detail to understand the complexity. for example, the findings showed that the tensions in manager relationships among different organisations, including the scas and deptland, led deptland field officers to promote an alternative is to farmers via recruiter relationships, and to discourage farmers from being recruited into the fpt design project. our insight into the interrelatedness of the relationship types makes a contribution because the few similar studies in this area only report on or clarify aspects of manager relationships (e.g. fisher & craig 2005; salmivalli et al. 2008; zewge et al. 2015), or enlister and recruiter relationships (e.g. oduor et al. 2018). the studies offering some insight into all three relationship types (e.g. breuer et al. 2008; doerflinger & dearden 2013) are consistent with our findings, because the superficial australasian journal of information systems parker, agustiono, carr & saundage 2019, vol 23, research article recruiting farmers into is design 23 insights from these studies imply all three relationship types are relevant. but our study extends this work by exploring the attributes and complex interrelatedness of the three relationship types in detail. specifically, our theorising makes sense of how the relationship types evolve and are interconnected. fourth, our study confirms the findings of the few studies which offer insights into the relationship attributes that can enable the recruitment of sme personnel into is design. we extend this prior work by positing that social capital concepts can underpin these attributes. with recruiter relationships, for example, we found that field officer and farmer relationships prior to the fpt project aided recruitment, which is consistent with oduor et al. (2018) and jakku and thorburn (2010). such prior relationships can be theorised using the social capital concepts of: structure, including the extent/number of connections among recruiters and sme personnel; and ties including the duration of a relationship and the trust that develops over time. our finding that field officer relationships with farmers were supported by bridging capital, including helping farmers access grants, was similar to oduor et al. (2018). oduor et al. report that goodwill was created with an ethiopian community of farmers when the researchers left water sensor technology that was used in a past project with the community, which helped recruit farmers during a later is design project. the finding in our study, that greater homophily among field officers and farmers can help explain recruitment, is consistent with breuer et al. (2008) and oduor et al. (2018) who imply that common knowledge/experience among researchers and farmers helped to recruit farmers. finally, our finding that shared cognition, or common language/values, helped field officers recruit farmers is consistent with oduor et al. (2018), who implied that researchers’ common language and knowledge of local ethiopian norms helped the researchers recruit farmers. overall, these examples suggest our approach to theorising relationship attributes may apply to other contexts, including recruiting farmers into is design in canada (breuer et al. 2008) and in developing countries such as ethiopia (oduor et al. 2018). these examples suggest that the three relationship types, and their associated attributes, have potential to extend the is design methodology developed by doerflinger and dearden (2013) for use in developing countries. the methodology highlights the importance of negotiations and contracts, and building strong relationships between researchers, a change manager and managers of local partners (i.e. manager relationships). this is needed before local farmers and other stakeholders will participate in is design. our findings, together with oduor et al. (2018), suggest that social capital concepts may be a useful extension of this methodology. for example, the concepts posit that strong ties and higher degrees of homophily need to be developed between, for instance, the change manager and local partner managers. these concepts may also explain why the doerflinger and dearden methodology emphasises the critical role played by local partner and local contact organisations when it comes to recruiting sme personnel in developing countries. for example, managers of local contact organisations would have access to a network of volunteers (structure) with established voluntary aid collaborations (ties) that enable the managers to enlist volunteers (i.e. enlister relationships). the managers would need to convince volunteers that the is design project is consistent with the values of the volunteers (shared cognition). doerflinger and dearden imply that the relationships among volunteers and local african communities (i.e. recruiter relationships) enable volunteers to recruit and execute is design activities. the social capital concepts may help theorise why. volunteers with pre-existing relationships with sme personnel (structure) from prior volunteer work (strong ties), have common norms (shared cognition) and access to australasian journal of information systems parker, agustiono, carr & saundage 2019, vol 23, research article recruiting farmers into is design 24 aid (bridging capital), which enables volunteers to recruit local community members, including sme personnel. 7 conclusion this next section concludes the paper by summarising the practical contributions of this study, the limitations of this study, and the future research agenda. 7.1 practical implications this is a detailed empirical study, and practitioners may benefit from our extensions to boonstra’s (2009) is stakeholder management framework, and to doerflinger and dearden’s (2013) is design methodology for developing countries. practical implications of these extensions imply that is design teams need to facilitate the effective interplay of the three relationship types, and pursue favourable relationship attributes, to increase the likelihood that sme personnel will be recruited into the concept and development stages of is design. our empirical study highlights that this is not trivial, because other stakeholders may use the three relationship types, and associated attributes, to undermine the recruitment of sme personnel. this means that is design teams must identify the manager, enlister and recruiter relationships which may be threats, and which may mean sme personnel are targeted by recruiters who promote a competing project. stakeholders must devise strategies that exploit the three relationship types to counteract such threats. our findings indicate that developing new relationships, or capitalising on existing relationships, with higher degrees of structure or access to sme personnel, and ties, shared cognition and homophily between recruiters and sme personnel, may help with recruiting sme personnel during is design projects. 7.2 limitations the study reported in this paper has limitations including: 1) being a single case study which, we emphasise, is not intended to demonstrate best practice in how relationship types and associated attributes can help is design teams recruit sme personnel ; 2) identifying what relational types and attributes were used and how, but not why they were used; 3) not reporting on sme personnel experiences; and 4) examining manager and recruiter recollections of a historical is design project rather than studying a project as it unfolded. nonetheless, the paper is among the first in-depth studies of: the three relationship types and attributes that explain how sme personnel were recruited into an is design project; and the theoretical concepts which scholars can use to frame much needed research in this area. 7.3 future research agenda we anticipate our proposed extensions to boonstra’s is stakeholder management framework and doerflinger and dearden’s is design methodology for developing countries can underpin a future research agenda by is scholars. for example, case studies and action research can examine is design projects in developed and developing countries where sme personnel involvement is desired, and apply the three relationship types and associated attributes based on social capital to explore why sme personnel were or were not recruited. future research can determine if the three levels we have used for each relationship attribute in table 5, and the descriptions of each, are appropriate for other is design contexts. scholars can conduct action research studies to identify and examine effective strategies, informed by our extensions to the above framework and methodology, that can improve sme personnel recruitment during is design. finally, scholars could integrate our work with other theories to explore australasian journal of information systems parker, agustiono, carr & saundage 2019, vol 23, research article recruiting farmers into is design 25 nuances further. for example, tatnall and burgess (2009) imply that actor network theory could theorise the negotiations in manager relationships. activity theory could extend our understanding of homophily, ties and shared cognition in all three relationship types, because this theory focuses on the sources of tension in relationships among actors involved in day-today practices (karanasios 2018). references abs (2012). australian social trends, dec 2012: australian farming and farmers. catalogue no. 4102.0, australian bureau of statistics, canberra. retrieved from http://www.abs.gov.au/ausstats/abs@.nsf/lookup/4102.0main+features10dec+2012 abs (2015). counts of australian businesses, including entries and exits, jun 2010 to jun 2014. catalogue no. 8165.0, australian bureau of statistics, canberra. retrieved from http://www.abs.gov.au/ausstats/abs@.nsf/mf/8165.0 bødker, m., gimpel, g., & hedman, j. (2014). time-out/time-in: the dynamics of everyday experiential computing devices. information systems journal, 24(2), 143-166. https://doi.org/10.1111/isj.12002 boonstra, a. (2009). identifying and managing stakeholders in enterprise information system projects. lntemational journal of enterprise lnfomation systems, 5(4), 1-16. https://doi.org/10.4018/jeis.2009090201 breuer, n.e., cabrera, v.e., ingram, k.t., broad, k., & hildebrand, p.e. (2008). agclimate: a case study in participatory decision support system development. climatic change, 87, 385-403. https://doi.org/10.1007/s10584-007-9323-7 breuer, n.e., fraisse, c.w., & hildebrand, p.e. (2009). molding the pipeline into a loop: the participatory process of developing agroclimate, a decision support system for climate risk reduction in agriculture. journal of service climatology, 3(1), 1-12. burgess, s., bingley, s., & hunter, m.g. (2012). involvement in small community based organisations' websites. journal of information, information technology, and organizations, 6, 63-80. https://doi.org/10.28945/1696 cabrera, v.e., breuer, n.e., & hildebrand, p.e. (2008). participatory modeling in dairy farm systems: a method for building consensual environmental sustainability using seasonal climate forecasts. climatic change, 89(3-4), 395-409. https://doi.org/10.1007/s10584-0079371-z cerf, m., jeuffroy, m.-h., prost, l., & meynard, j.-m. (2012). participatory design of agricultural decision support tools: taking account of the use situations. agronomy for sustainable development, 32(4), 899-910. https://doi.org/10.1007/s13593-012-0091-z chang, k.-c., wong, j.-h., li, y., lin, y.-c., & chen, h.-g. (2011). external social capital and information systems development team flexibility. information and software technology, 53, 592-600. https://doi.org/10.1016/j.infsof.2011.01.007 chou, s.-w., & he, m.-y. (2011). the factors that affect the performance of open source software development – the perspective of social capital and expertise integration. information systems journal, 21(2), 195-219. https://doi.org/10.1111/j.13652575.2009.00347.x australasian journal of information systems parker, agustiono, carr & saundage 2019, vol 23, research article recruiting farmers into is design 26 churi, a.j., mlozi, m.r.s., mahoo, h., tumbo, s.d., & casmir, r. (2013). a decision support system for enhancing crop productivity of smallholder farmers in semi-arid agriculture. international journal of information and communication technology research, 3(8), 238-248. cragg, p.b., caldeira, m.m., & ward, j.m. (2011). organizational information systems competences in small and medium-sized enterprises. information & management, 48(8), 353-363. https://doi.org/10.1016/j.im.2011.08.003 dans, e., & freire, j. (2002). it as an agent of social change: lonxanet and the case of the galician artisanal fisheries. 23rd international conference on information systems, 15-18 december, barcelona, spain. doerflinger, j., & dearden, a. (2013). evolving a software development methodology for commercial ictd projects. information technologies & international development, 9(3), 4360. eastwood, c.r., dela rue, b.t., & gray, d.i. (2017). using a ‘network of practice’ approach to match grazing decision-support system design with farmer practice. animal production science, 57(7), 1536-1542. https://doi.org/10.1071/an16465 fisher, j., & craig, a. (2005). developing business community portals for smes – issues of design, development and sustainability. electronic markets, 15(2), 136-145. https://doi.org/10.1080/10196780500083811 gichamba, a., & lukandu, i.a. (2012). a model for designing m-agriculture applications for dairy farming. the african journal of information systems, 4(4), 120-136. granovetter, m.s. (1973). the strength of weak ties. american journal of sociology, 78(6), 13601380. https://doi.org/10.1086/225469 harwood, s.a. (2011). the domestication of online technologies by smaller businesses and the 'busy day'. information and organization, 21(2), 84-106. https://doi.org/10.1016/j.infoandorg.2011.03.002 heckathorn, d.d. (2011). comment: snowball versus respondent-driven sampling. sociological methodology, 41(1), 355-366. https://doi.org/10.1111/j.1467-9531.2011.01244.x heeks, r. (2008). ict4d 2.0: the next phase of applying ict for international development. computer, 41(6), 26-33. https://doi.org/10.1109/mc.2008.192 holgersson, j. (2014). user participation in public e-service development: guidelines for including external users. phd thesis, university of skovde. huang, g.h., sun, w., nie, x.-h., qin, x.-s., & zhang, x.-d. (2010). development of a decisionsupport system for rural eco-environmental management in yongxin county, jiangxi province, china. environmental modelling & software, 25(1), 24-42. https://doi.org/10.1016/j.envsoft.2009.07.010 islam, m.s., & grönlund, å. (2007). agriculture market information e-service in bangladesh: a stakeholder-oriented case analysis. in m.a. wimmer, h.j. scholl and å. grönlund (eds.), electronic government (vol. 10.1007/978-3-540-74444-3_15, pp. 167-178). berlin, heidelberg: springer. https://doi.org/10.1007/978-3-540-74444-3_15 australasian journal of information systems parker, agustiono, carr & saundage 2019, vol 23, research article recruiting farmers into is design 27 jakku, e., & thorburn, p.j. (2010). a conceptual framework for guiding the participatory development of agricultural decision support systems. agricultural systems, 103(9), 675682. https://doi.org/10.1016/j.agsy.2010.08.007 karanasios, s.s. (2018). toward a unified view of technology and activity: the contribution of activity theory to information systems research. information technology & people, 31(1), 134-155. https://doi.org/10.1108/itp-04-2016-0074 klein, h.k., & myers, m.d. (1999). a set of principles for conducting and evaluating interpretive field studies in information systems. mis quarterly, 23(1), 67-93. kotlarsky, j., & oshri, i. (2005). social ties, knowledge sharing and successful collaboration in globally distributed system development projects. european journal of information systems, 14, 37-48. https://doi.org/10.1057/palgrave.ejis.3000520 kvale, s. (2007). doing interviews. london: sage publications. lim, s.l., & finkelstein, a. (2012). stakerare: using social networks and collaborative filtering for large-scale requirements elicitation. ieee transactions on software engineering, 38(3), 707-735. https://doi.org/10.1109/tse.2011.36 lynch, t., & gregor, s. (2004). user participation in decision support systems development: influencing system outcomes. european journal of information systems, 13(4), 286-301. https://doi.org/10.1057/palgrave.ejis.3000512 lyytinen, k., & damsgaard, j. (2011). inter-organizational information systems adoption – a configuration analysis approach. european journal of information systems, 20(5), 496-509. https://doi.org/10.1057/ejis.2010.71 mason, c., castleman, t., & parker, c.m. (2008a). communities of enterprise: developing regional smes in the knowledge economy. journal of enterprise information management, 21(6), 571-584. https://doi.org/10.1108/17410390810911186 mason, c., castleman, t., & parker, c.m. (2008b). socio-technical factors influencing channel use for knowledge-sharing in regional sme networks. international journal of knowledge management studies, 2(3), 303-319. https://doi.org/10.1504/ijenm.2008.018794 mcpherson, m., smith-lovin, l., & cook, j .m. (2001) . birds of a feather: homophily in social networks. annual review of sociology , 27, 415-444. https://doi.org/10.1146/annurev.soc.27.1.415 merz, c., de louw, r., & ullrich, n. (2007). collaborative working environments for enterprise incubation – the sekhukhune rural living lab. 2nd ist-africa conference & exhibition, 9-11 may, maputo, mozambique. miah, s.j., kerr, d., & von hellens, l. (2014). a collective artefact design of decision support systems: design science research perspective. information technology & people, 27(3), 259279. https://doi.org/10.1108/itp-04-2012-0041 nahapiet, j . , & ghoshal, s. (1998). social capital, intellectual capital, and the organizational advantage. academy of management review , 23(2), 242-266. https://doi.org/10.5465/amr.1998.533225 ndou, v., del vecchio, p., & schina, l. (2011). designing digital marketplaces for competitiveness of smes in developing countries. in m. obaidat and j. filipe (eds.), eaustralasian journal of information systems parker, agustiono, carr & saundage 2019, vol 23, research article recruiting farmers into is design 28 business and telecommunications (vol. 10.1007/978-3-642-20077-9_6, pp. 82-93). milan, italy: springer berlin heidelberg. https://doi.org/10.1007/978-3-642-20077-9_6 nolan, t., brizland, r., & macaulay, l.a. (2007). individual trust and development of online business communities. information technology & people, 20(1), 53-71. https://doi.org/10.1108/09593840710730554 noor, n.l.m., & nordin, a. (2012). requirements elicitation for the technology conception of a community information system for the indigenous microenterprise: a contextual multianalysis approach on business and community requirements of batik making. electronic journal of information systems evaluation, 15(1), 102-115. oduor, e., waweru, p., lenchner, j., & neustaedter, c. (2018). practices and technology needs of a network of farmers in tharaka nithi, kenya. 35th acm conference on human factors in computing systems, 21-26 october, montreal, canada. https://doi.org/10.1145/3173574.3173613 paavola, r., & hallikainen, p. (2016). antecedents for successful collaboration in requirements engineering. vine journal of information and knowledge management systems, 46(3), 353370. https://doi.org/10.1108/vjikms-03-2016-0013 parker, c., & sinclair, m. (2001). user-centred design does make a difference. the case of decision support systems in crop production. behaviour & information technology, 20(6), 449-460. https://doi.org/10.1080/01449290110089570 parker, c.m., burgess, s., & al-qirim, n.a.y. (2015). a review of studies on information systems and smes in high ranked is journals (2000-2014). australasian journal of information systems, 19, s153-s173. https://doi.org/10.3127/ajis.v19i0.1219 parker, c.m., & castleman, t. (2009). small firm ebusiness adoption: a critical analysis of theory. journal of enterprise information management, 22(1/2), 167-182. https://doi.org/10.1108/17410390910932812 patton, m.q. (2002). qualitative research and evaluation methods (3rd ed.). thousand oaks, california: sage publications. pouloudi, n., currie, w.l., & whitley, e.a. (2016). entangled stakeholder roles and perceptions in health information systems: a longitudinal study of the u.k. nhs n3 network. journal of the association for information systems, 17(2), 107-161. putnam, r.d. (2000). bowling alone: the collapse and revival of american community. new york: simon & schuster. robert, l.p., dennis, a.r., jr., & ahuja, m.k. (2008). social capital and knowledge integration in digitally enabled teams. information systems research, 19(3), 314-334. https://doi.org/10.1287/isre.1080.0177 rodon, j., pastor, j.a., sese, f., & christiaanse, e. (2008). unravelling the dynamics of iois implementation: an actor-network study of an iois in the seaport of barcelona. journal of information technology, 23(2), 97-108. https://doi.org/10.1057/palgrave.jit.2000131 salmivalli, l.a.t., salmela, h., & kestila, t. (2008). building inter-organizational cooperative network for it collaboration. 41st annual hawaii international conference on system sciences, 7-10 january, waikoloa village, hawaii. https://doi.org/10.1109/hicss.2008.81 australasian journal of information systems parker, agustiono, carr & saundage 2019, vol 23, research article recruiting farmers into is design 29 schlichter, b.r., & rose, j. (2013). trust dynamics in a large system implementation: six theoretical propositions. european journal of information systems, 22(4), 455-474. https://doi.org/10.1057/ejis.2012.24 tatnall, a., & burgess, s. (2009). portals then and now: development and use of portals in australia and bangladesh. international journal of web portals, 1(4), 21-33. https://doi.org/10.4018/jwp.2009071302 van limburg, m., wentzel, j., sanderman, r., & van gemert-pijnen, l. (2015). business modeling to implement an ehealth portal for infection control: a reflection on co-creation with stakeholders. jmir research protocols, 4(3). https://doi.org/10.2196/resprot.4519 walsham, g. (1995). interpretive case studies in is research: nature and method. european journal of information systems, 4(2), 74-81. https://doi.org/10.1057/ejis.1995.9 winschiers-theophilus, h., bidwell, n.j., & blake, e. (2012). community consensus: design beyond participation. design issues, 28(3), 89-100. https://doi.org/10.1162/desi_a_00164 wolcott, h.f. (1994). transforming qualitative data: description, analysis, and interpretation. thousand oaks, california: sage publications. yin, r.k. (2014). case study research: design and methods (5th ed.). thousand oaks, california: sage publications. zewge, a., dittrich, y., & bekele, r. (2014). providing market information for ethiopian farmers: extending participatory design. 13th participation for development conference, 610 october, windhoek, namibia. zewge, a., dittrich, y., & bekele, r. (2015). adapting participatory design to design information system with rural ethiopian community. 12th africon conference in africa , 14-17 september, addis ababa, ethiopia. https://doi.org/10.1109/afrcon.2015.7331974 copyright: © 2019 parker, agustiono, carr & saundage. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v23i0.1943 http://creativecommons.org/licenses/by-nc/3.0/au/ http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems parker, agustiono, carr & saundage 2019, vol 23, research article recruiting farmers into is design 30 1 introduction 2 literature review 2.1 recruiter relationships: recruiters and sme personnel 2.2 enlister relationships: between managers and recruiters 2.3 manager relationships: among organisational managers 3 theoretical background 4 research approach 4.1 overview of the revelatory case study 4.2 data collection and analysis 5 findings 5.1 manager relationships: manager relationships with other managers 5.2 enlister relationships: manager relationships with recruiters 5.3 recruiter relationships: recruiter relationships with sme personnel 5.3.1 project-manager-sca1 relationships with three farmers 5.3.2 field officer relationships with sme farmers 6 discussion 7 conclusion 7.1 practical implications 7.2 limitations 7.3 future research agenda paper title (paper title style) australasian journal of information systems senarathna, yeoh, warren & salzman 2016, vol 20, research article security & privacy for sme cloud adoption security and privacy concerns for australian smes cloud adoption: empirical study of metropolitan vs regional smes ishan senarathna department of information systems and business analytics deakin business school william yeoh department of information systems and business analytics deakin business school william.yeoh@deakin.edu.au matthew warren department of information systems and business analytics deakin business school scott salzman department of information systems and business analytics deakin business school abstract new national infrastructure initiatives such as the national broadband network (nbn) allow small and medium-sized enterprises (smes) in australia to have greater access to costeffective cloud computing. however, the ability of cloud computing to store data remotely and share services in a dynamic environment brings with it security and privacy concerns. evaluating these concerns is critical to address the cloud computing underutilisation issue and leverage the benefits of costly nbn investment. this paper examines the influence of privacy and security factors on cloud adoption by australian smes in metropolitan and regional area. data were collected from 150 australian smes (specifically, 79 metropolitan smes and 71 regional smes) and structural equation modelling was used for the analysis. the findings reveal that privacy and security factors do not significantly influence the decision-making of australian smes in the adoption of cloud computing. moreover, the results indicate that cloud computing adoption is not influenced by the geographical location (i.e., metropolitan or regional location) of the smes. the findings extend the current understanding of cloud computing adoption by australian smes. the results will be useful to smes, cloud service providers and policy makers devising cloud security and privacy policies. keywords: cloud adoption, smes, security, privacy, structural equation modelling 1 introduction the national broadband network (nbn) enables australians to access affordable, high-speed internet and telephone services (nbnco, 2015a) and, therefore, increases the proliferation of cloud computing across australia. in particular, cloud computing has become increasingly important for australian smalland medium-sized enterprises (smes), as cloud computing affords smes access to the information and communication technology (ict) applications that, previously, only large companies could afford (nbnco, 2015b). many cloud service providers, such as amazon web services and salesforce.com, are offering smes cost-effective solutions to lower their operating costs and improve their productivity (marston et al., 2011). gartner defined cloud computing as “a style of computing in which scalable and elastic itenabled capabilities are delivered as a service using internet technologies” (plummer et al., 2009). besides shifting a specific business function to an outsourcer (lee & kim, 1999), cloud computing constitutes a more radical move and involves the automation of a business process or a subset of activities. while having similar benefits as ict outsourcing, cloud computing is 1 australasian journal of information systems senarathna, yeoh, warren & salzman 2016, vol 20, research article security & privacy for sme cloud adoption more flexible because it is location independent, and this enables customers to access and use the services wherever they have access to the internet. also, cloud computing is device independent, which means the service is able to function on a wide variety of devices. as marston et al. (2011) explained, cloud computing provides almost direct access to shared computing resources, thus smes can launch new operations quickly with little or no upfront capital investment, enabling a faster time to market. in fact, forrester research estimated that the global cloud computing market will grow from $732 million in 2011 to $3.2 billion in 2020 (acma, 2014). international trade administration predicted that businesses will spend over $191 billion on cloud services by 2020, compared to $72 billion in 2014 (ita, 2015). the international data corporation (idc) forecast a 2017 cloud market worth $107 billion, over twice as much as its 2013 estimate of $47.4 billion (idc, 2015). cloud computing is a new business model, which is particularly valuable for smes with limited budgets for ict investment (mudge, 2010). cloud computing is commercially viable for many smes due to its flexibility and pay-as-you-go cost structure (sultan, 2011). however, within the sme sector, and despite the potential benefits, the adoption rate of cloud computing is still relatively low in australia compared with other countries in the asian region (acca, 2012). according to the australian communications and media authority, less than half of the smes in australia are currently using cloud services, due in particular to concerns over privacy and security factors (ericson, 2015; nbnco, 2015b). the most recent global survey on cloud computing conducted by dresner advisory services revealed that security concerns are a primary barrier to adoption of cloud computing (ericson, 2015). currently, cloud computing poses an inherent challenge to privacy because cloud computing services usually present data in an unencrypted form on a machine owned and operated by a different organisation than the data owner. the research literature indicates that the main inhibiting factor for cloud computing adoption is the fear of sending organisational data to a third party (rittinghouse & ransome, 2009; shimba, 2010). further, previous findings suggest that cloud computing adoption is more than just technology adoption (tehrani & shirazi, 2014). it includes a number of important changes relating to cross-border data transfer, storing data with a third party, remotely accessing resources and utilising applications through the internet. in other words, there are a number of privacy and security issues associated with cloud computing, including lack of user control, potential unauthorised secondary usage, data proliferation, cross-border data flow, multi-tenancy, and lack of standardisation (pearson & benameur, 2010). because cloud computing providers employ multiple data centres at different geographical locations around the world to optimally serve consumers’ needs, jurisdictional issues can cause regulatory and legislative complexities. in most cloud service scenarios, consumers have no idea where their data are stored, due to the dynamic nature of the cloud (pearson & benameur, 2010). legal and regulatory issues arise that require careful consideration, because the physical location of data centres determines the set of laws that govern the management of data (sahandi, alkhalil & justice, 2012). it is also interesting to note that it adoption mainly refers to in-house it infrastructure, whereas cloud computing adoption involves accessing resources outside the organisation via the internet. yet the security and privacy issues related to cloud computing adoption among australian smes have not been the subject of previous empirical research. therefore, this research aims to investigate the security and privacy concerns of australian smes adopting internet-based cloud computing, with a focus on metropolitan and regional sme organisations. section two of the paper reviews the literature and proposes the research hypotheses. section three outlines the quantitative research methodology and measurement validity. section four discusses the empirical findings, research contributions and limitations, and provides suggestions for further studies. section five concludes the study. 2 australasian journal of information systems senarathna, yeoh, warren & salzman 2016, vol 20, research article security & privacy for sme cloud adoption 2 literature review and hypotheses 2.1 cloud computing cloud computing provides different types of services delivered under different deployment models and uses a pay-as-you-go method. three service models are extensively used by the cloud computing community to categorise cloud computing services (ahuja & rolli, 2011; dillon et al., 2010; george & shyam, 2010), such as software as a service (saas), platform as a service (paas), and infrastructure as a service (iaas) on demand and pay-as-you-go. saas in cloud computing eliminates the need to install and run an application on the client’s computer (marston et al., 2011). in addition, it is not necessary for smes to worry about software licensing or upgrading, because it is performed by the cloud service providers. smes need to focus on service level agreement or contract only. according to sullivan (2010), there are various types of services that come under saas, namely, customer relationship management (crm), video conferencing, it service management, accounting, web analytics, and web content management. similarly, application design, development, testing, deployment and hosting are services provided by paas. the development and deployment of applications without the cost and complexity of buying and managing the underlying hardware and software layers is facilitated by paas (marston et al., 2011). further, sullivan (2010) explained that iaas provides services such as server space, networking equipment, memory, storage space and computing capabilities. services provided by cloud computing can be categorised according to the level of service and mode of provision. deployment models are recorded based on these service characteristics. four cloud computing deployment models have been defined in the cloud computing community (i.e., public cloud, private cloud, hybrid cloud and community cloud) (chang et al., 2013; gangwar et al., 2015; tehrani & shirazi, 2014; zhao et al., 2014). a public cloud computing service model is available from a third-party service provider via the internet. it is a cost-effective way to deploy it solutions and provides many benefits, such as being elastic and service-based, as, most of the time, resources are not utilised to maximum capacity, which wastes money. the scalability of cloud computing significantly reduces the time resources are idle and allows smes to use only the amount of computing resources they need. they can instantly scale resources up or down when demand increases or decreases. public cloud is the most commonly used model and is especially suitable for smes because it provides nearimmediate access to hardware resources, with no upfront capital investments for users, leading to a faster time to market for many businesses. this treats it as an operational expense, rather than a capital expense (‘opex’ as opposed to a ‘capex’ model) (marston et al., 2011). private cloud computing provides greater control over the cloud computing infrastructure and can be managed within the organisation. therefore, it is often suitable for large organisations, as they are using larger installations (marston et al., 2011). hybrid cloud computing is a combination of public and private cloud computing models that addresses the limitations of each (zhang et al., 2010). the community cloud computing infrastructure is controlled and shared by a group of organisations and supports a specific community with shared concerns (e.g., mission, security requirements, policy and compliance considerations) (sasikala, 2011). 2.2 cloud computing adoption in smes the australian bureau of statistics defined smes as businesses with less than 200 employees, with medium-sized businesses employing 20–199 staff, and small businesses employing fewer than 20 staff (diisr, 2011). australian smes play a substantial role in economic development; they represent 96% of all australian business and 69% of the total workforce, and account for 33% of gross domestic product (gdp) (acma, 2014). senarathna et al. (2015) suggested that, in many instances, using cloud computing provides the first opportunity for smes to try new software approaches in a cost-effective manner. often smes are unable to afford their own dedicated it, but they have an it budget sufficient to buy the bandwidth and pay according to their need and usage (monika et al., 2010). in a cloud computing environment, smes can reduce their capital expenditure for it infrastructure and, instead, utilise and pay for the resources and services provided by cloud computing (rittinghouse & ransome, 2009). a 3 australasian journal of information systems senarathna, yeoh, warren & salzman 2016, vol 20, research article security & privacy for sme cloud adoption kpmg report on australian lessons and experiences showed that using cloud computing allowed australian organisations to adopt innovative it technologies quickly, without paying upfront for capital investment (mccabe & hancook, 2009). according to lawrence et al. (2010), all direct and indirect go-to-market models in cloud computing are able to cater to the needs of smes, but they are not necessarily suitable for large enterprises because of their organisational scale and complexity. it has been found that the current charging pattern and other aspects of cloud computing make it more suitable for smes than for larger organisations (misra & mondal, 2010). further, the public cloud service provides a more valuable service to smes, as they require many of the same business services provided to large organisations (e.g., software such as crm, erp, hrm and payroll, and hardware such as server and storage), even though they may only have a pc and an internet connection (handler et al., 2012). cloud adoption in australia is growing rapidly, with key global players such as amazon web services extending their offerings in the region with new data centres, while a host of local and regional cloud providers are also expanding (miller, 2014). telstra (2011) stated that its entire network is robust and sufficiently extensive to provide reliable cloud services to customers in metropolitan and regional areas. moreover, the nbn was introduced as an investment in infrastructure to provide high capacity data communications across the nation. the nbn planned to reach 93% of the australian homes, schools and businesses that could be accessed by the nbn through optic fibre, providing peak speeds of up to 25mbps. the nbn consists of the provision of broadband services over a mix of three technologies: optic fibre, fixed wireless, and next-generation satellite (matthew, 2014). the nbn could enable smes in australia to further embrace cloud computing services. minifie (2014) argued that businesses need reliable high bandwidth, low-latency uploads and downloads, and sufficient international backbone bandwidth. minifie (2014) also argued that bottlenecks in suburbia and regional and remote areas should be addressed to enable rapid cloud adoption. in addition, the findings of sultan (2011) and bharadwaj and lal (2012) revealed that cloud computing is likely to be a more attractive option for most smes because of flexible cost structure and scalability. further, cloud services can be used easily by smes due to relative advantage, flexibility and scalability features (salleh et al., 2012). according to gartner, the public cloud services market (the most popular services needed by smes) would reach us$250 billion by 2017, with continuous rapid growth from 2011 to 2017 (anderson et al., 2013). therefore, there is a burgeoning need to conduct focussed research on the issue. 2.3 hypotheses development to ensure a comprehensive coverage, the constructs used in this study are explored through a myriad of theoretical, practitioner and government underpinnings. the academic, government and practitioner literatures for each construct was reviewed and is listed separately in table 1. based on the literature, cloud security and privacy factors are discussed in detail. constructs academic government practitioner cloud security gangwar et al., 2015; li et al., 2015; oliveira et al., 2014; ross & blumenstein, 2015; safari et al., 2015; stieninger & nedbal, 2014; tang and liu, 2015; tarmidi et al., 2014; tehrani & shirazi, 2014; zhao et al., 2014 anthony, 2012; dbcde, 2013; dfd, 2011b, 2011c, 2011d; doc, 2014; imo, 2012; imo, 2013; itiic, 2011; jansen & grance, 2011; swi, 2012 acca, 2014; ben, 2014; dave, 2012; handler et al., 2012; herhalt & cochrane, 2012; hutley, 2012; mark, 2011; minifie, 2014; pearson, 2012; ren et al., 2012; telstra, 2011 cloud privacy fairchild, 2014; fakieh et al., 2014; gangwar et al., 2015; kauffman et al., 2014; mahmood et anthony, 2012; dbcde, 2013; dfd, 2011b, 2011c, 2011d; doc, 2014; imo, acca, 2014; ben, 2014; dave, 2012; herhalt & cochrane, 2012; hutley, 2012; 4 australasian journal of information systems senarathna, yeoh, warren & salzman 2016, vol 20, research article security & privacy for sme cloud adoption al., 2014; ross & blumenstein, 2015; safari et al., 2015; stieninger & nedbal, 2014; tang & liu, 2015; tehrani & shirazi, 2014; vanessa, 2014 2012; imo, 2013; itiic, 2011; jansen & grance, 2011 ko et al., 2011; lemt, 2011; mark, 2011; minifie, 2014; pearson, 2012; ren et al., 2012; telstra, 2011 table 1: constructs used to examine cloud computing adoption 2.4 cloud privacy privacy is an important issue in technological innovations, particularly when it has an online interaction. in the cloud environment, privacy reflects a consumer’s concerns about information being stored in the cloud and accessed by other individuals anywhere in the world (vanessa, 2014). in other words, input data for cloud services are uploaded by a user to the cloud, which the user does not own or control. as abadi (2009, pp. 2–3) indicated, “computer power is elastic, but only if the workload is parallelisable … data is stored at an un-trusted host … data is replicated, often across large geographic distances”, which are some of the cloud characteristics that make the cloud a risk. featherman and pavlou (2003) recognised privacy as a key factor hindering the uptake of some ict technologies, due to the open nature of the internet. privacy risk is typically a major concern of businesses considering cloud computing (aziz, 2010; hailu, 2012). according to a survey carried out among chief information officers (cios) in europe, approximately 70% of cios were prevented from launching cloud computing solutions due to their own concerns over privacy and security fears (wijesiri, 2010). in particular, lack of transparency creates legal issues that are affected by the cloud’s physical location, which creates difficulties in determining jurisdiction. because of this key issue, the australian federal government is concerned about the locations of outsourced personal data storage and strongly prefer for cloud services to be located only within australia’s borders (hutley, 2012). it is notable that the european union (eu) has privacy regulations that prohibit the transmission of some types of personal data outside the eu (sultan, 2010). in a survey conducted by tang and liu (2015), the respondents indicated that data privacy issues were their top concern in considering whether to adopt cloud computing. according to gupta, seetharaman and raj (2013), privacy is the top concern of 50% of organisations in considering whether to adopt cloud computing. a study by safari, safari and hasanzadeh (2015) likewise showed that privacy is a significant determinant in the adoption of cloud computing. xu and gupta (2009) included privacy concerns as a way of evaluating the potential adoption of location-based services. chen, paxon and katz (2010) expressed concern that privacy has emerged as one of the most significant disadvantages of joining a cloud computing group. according to alkhater, wills and walters (2014), privacy is the main concern for organisations thinking about cloud computing, because they cannot fully control the information stored on cloud-based servers. tancock, pearson and charlesworth (2013) also argued that privacy issues form the major hurdle to large-scale acceptance of cloud computing. trust in privacy has also been shown to be a primary determinant of it innovation acceptance and diffusion in organisations (li, hess & valacich, 2008; schoorman, mayer & davis, 2007). transferring personal data to a third party without privacy policies in place creates huge risks in terms of data loss, data theft, data damage and data misuse. even with policies in place, risks remain (ko et al., 2011) but the associated controls that support the policies, help minimise the risk. the victorian state government identified key legal issues with cloud computing, including information privacy related to cloud services and their relationship with the australian privacy principles (anthony, 2012). the australian government’s department of finance and deregulation published a best practice guide to help agencies navigate typical legal issues in cloud computing agreements, with the intention of emphasising privacy concerns (dfd, 2011b). further, the australian federal information management office provided a best practice guide (‘privacy and cloud computing for australian government 5 australasian journal of information systems senarathna, yeoh, warren & salzman 2016, vol 20, research article security & privacy for sme cloud adoption agencies’) to better understand how to comply with privacy laws and regulations when choosing cloud-based services (imo, 2013). along these lines, this study hypothesises that: h1: cloud privacy influences the decision-making of australian smes considering whether to adopt cloud computing. 2.5 cloud security information security refers to the “preservation of confidentiality, integrity and availability of information; in addition, other properties such as authenticity, accountability, nonrepudiation and reliability can also be involved” (pearson, 2012, p. 14). the foundations of information security are based upon the confidentiality, integrity, availability, accountability, assurance and resilience of information (friedman & west, 2010). in recent studies, information security concerns have been cited as the most significant barrier to cloud adoption (armbrust et al., 2010; kshetri, 2013; xiao & xiao, 2013). security issues in cloud computing were generally discussed by many researchers (jamil & zaki, 2011; kshetri, 2013; zissis & lekkas, 2012) and with specific reference to smes (adam, 2014; alshamaila, 2013; gupta et al., 2013). further, several studies have demonstrated that security is one of the major challenges keeping end-users away from cloud computing (gens et al., 2009; shaikh & haider, 2011). in particular, major issues pertaining to data security in the cloud computing environment include data location and data transmission, data availability, data security (mahmood, 2011), malicious insiders, outside attacks and service disruptions (behl, 2011). other major challenges include the delegation of confidentiality, and the availability and integrity of data provided to a third party. the security of cloud computing is complicated because of the multi-tenancy of the virtualised resources (opala, 2012). cloud users may think that cloud computing simplifies security issues for users by outsourcing the responsibility to another party who is presumed to be highly skilled in such matters (anthes, 2010). industry practitioners have reported that security was a critical concern in the initial stages of cloud computing adoption (chakraborty et al., 2010; mccabe & hancock, 2009). bhayal (2011) stated that cloud security is the most important concern among cloud clients, as the data owner does not know where the data are stored and data hosts cannot be considered completely reliable. according to benlian and hess (2011), research regarding the adoption of cloud computing has considered security as a barrier and feuerlicht, burkon and sebesta (2011) identified security as a potential risk and challenge in cloud computing adoption. the security issues of cloud computing related to third parties’ involvement also pose challenges (subashini & kavitha, 2011). according to katzan (2010), cloud computing security is not just about authenticity, authorisation and accountability; it is more concerned with data protection, disaster recovery and business continuity. the architecture of cloud computing also leads to many new security issues, such as data leakage, virtualisation vulnerability and hypervisor vulnerability (gonzalez et al., 2012). garg and stiller (2014) suggested that cloud security risks also include contractual loopholes, confidentiality, information security and service outages. a survey of cios and it executives by the idc rated security as their main cloud computing concern, and almost 75% of respondents were worried about security (sultan, 2010). dillon and vossen (2014) also found that security issues were the most limiting factor for cloud adoption. security is one of the concerns about cloud computing that continues to delay adoption (jamwal, sambyal & sambyal, 2011). sarwar and khan (2013) found that security is the biggest issue in cloud computing as, while utilising storage service in a remote location, the consumers are generally unaware of what happens to their data. according to forrester research, sahandi et al. (2012) reported that security concerns are the most commonly cited reason why enterprises are not interested in saas cloud. therefore, this study hypothesises that: h2: cloud security influences the decision-making of australian smes considering the adoption of cloud computing. 6 australasian journal of information systems senarathna, yeoh, warren & salzman 2016, vol 20, research article security & privacy for sme cloud adoption 2.6 metropolitan vs regional the location of an organisation is an important consideration in innovation adoption (carcary et al., 2014; drew, 2003; gengatharen & standing, 2005; minifie, 2014). robert and mira (2010) argued that smes in regional areas are particularly constrained by great distances. vanessa (2014) suggested that it is useful to examine different geographic regions to further distinguish the differences between urban and rural areas in facilitating cloud computing adoption behaviour. alshamaila et al. (2013) pointed out that the nature of cloud computing transcends boundaries and regional ict infrastructure is not considered a major obstacle for the adoption process. cheng and kam (2008) observed that cloud computing allows for computing resources to be distributed across regional locations for seamless access. further, trend micro’s annual security survey found that significant regional differences, such as regional differences in cloud security, are less likely to adopt cloud computing (dave, 2012). therefore, the geographical location of the smes was considered and included as the control variable of this study. thus, this study also hypothesises that: h3: the decision-making process to adopt cloud computing is different between australian smes in metropolitan and regional areas. based on the aforementioned hypotheses, figure 1 presents the research model of this study. in this model, latent variables, that is, cloud privacy (cp) (hypothesis 1) and cloud security (cs) (hypothesis 1), are the independent variables that could influence the decision of australian smes considering whether to adopt cloud computing. these latent variables are measured using three items each, while sme location (regional vs metropolitan) represents the control variable. cloud adoption is the dependent variable and measured by four items. the detail of the items is available at: https://goo.gl/rd76uf figure 1. research model 3 methodology a positivist paradigm quantitative research method was applied in this study. in this study, a structured questionnaire was used to collect the quantitative data from australian smes in metropolitan and regional areas. data were collected from the sme it managers or decisionmakers. the australian communications and media authority reported that 94% of smes are connected to some form of internet service (acma, 2014). hence, an online survey was used for data collection. the questionnaire was divided into two parts. the first part of the survey 7 australasian journal of information systems senarathna, yeoh, warren & salzman 2016, vol 20, research article security & privacy for sme cloud adoption captured the demographic details of the responding organisations, and the second part of the survey captured perceptions of the security and privacy of cloud computing. for each construct, three to four items were formulated capturing the perception and adoption of cloud computing by smes. all of the reflective indicators of a construct were measured on a 7-point likert scale, using scales ranged from (1) ‘strongly disagree’ to (7) ‘strongly agree’. as recommended by bryman and bell (2015), ethical approval to undertake research is an important step, hence ethics approval was obtained from the faculty human ethics advisory group (available at: https://goo.gl/btildt). this was performed to protect the respondents from any adverse consequences that could result from conducting this research. 3.1 measurement reliability and validity the researchers used structural equation modelling (sem) (ibm spss amos) to perform the data analysis. first, the reliability of the measurement was assessed based on the composite reliability and cronbach’s alpha coefficient. as shown in table 2, the minimum composite reliability and cronbach’s alpha values for all constructs in this study were above the recommended threshold value of 0.7 (hair et al., 2010; malhotra, 2010), suggesting good reliability. construct items loading ave cronbach’s alpha composite reliability cloud adoption adopt1 0.994 0.989 0.993 0.871 adopt2 0.982 adopt3 0.994 adopt4 0.987 cloud security cs1 0.774 0.834 0.703 0.815 cs2 0.777 cs3 0.834 cloud privacy cp1 0.730 0.847 0.750 0.889 cp2 0.882 cp3 0.820 table 2: measurement reliability and validity results second, to assess convergent validity, the average variance extracted (ave) for each construct must meet the minimum recommended value of 0.50 (fornell & larcker, 1981). the ave for the constructs of this study ranged from 0.834 to 0.989, hence adequately demonstrating convergent validity. third, confirmatory factor analysis was conducted to assess the factor loadings of each construct. as depicted in table 3, the results indicate that items load higher on their constructs and higher than cross-loadings. overall, the results have good reliability, convergent validity and discriminant validity, and thus are appropriate for testing the research model. moreover, to reduce the likelihood of common methods bias, items were randomised within the instrument to limit the ability of participants to detect underlying construct patterns (straub et al., 1989). because both the dependent variable and independent variables were measured using the same instrument, harman’s one-factor test (podsakoff et al., 2003) was also performed, ie., all items were entered into an unrotated exploratory factor analysis to determine whether a single factor emerges or a single factor accounts for the majority of the variance. in this test, three factors emerged clearly; therefore, common methods bias is not an issue in our data set. construct items adopt cs cp cloud adoption adopt1 0.994 0.013 0.006 adopt2 0.982 0.027 0.010 adopt3 0.994 0.010 0.004 adopt4 0.987 0.013 0.006 8 australasian journal of information systems senarathna, yeoh, warren & salzman 2016, vol 20, research article security & privacy for sme cloud adoption construct items adopt cs cp cloud security cs1 −0.088 0.774 −0.182 cs2 −0.017 0.777 0.044 cs3 0.157 0.834 −0.012 cloud privacy cp1 −0.123 −0.094 0.730 cp2 0.083 0.010 0.882 cp3 0.065 −0.043 0.820 table 3: efa results 4 results and discussion table 4 summarises the demographic characteristics of the sme respondent organisations. not surprisingly, those states with larger populations provided higher response rates. survey participant organisations (n=150 smes) state/territory vic 29 19.3% nsw 30 20.0% qld 41 27.3% wa 19 12.7% sa 17 11.3% tas 11 7.3% nt 3 2.0% n=150 100% table 4: demographic characteristics of respondent organisations the locations of the organisations were categorised based on regional australian postcodes (tcp, 2015). table 5 shows that 52.7% of the respondents in the sample were from metropolitan areas, and 47.3% were from regional areas. frequency percent valid percent cumulative percent valid metropolitan 79 52.7 52.7 52.7 regional 71 47.3 47.3 100.0 total 150 100.0 100.0 table 5: frequencies of respondent location the descriptive analysis of all the multi-item factors in the research model used to examine the hypotheses are presented as follows. 9 australasian journal of information systems senarathna, yeoh, warren & salzman 2016, vol 20, research article security & privacy for sme cloud adoption 4.1 adopt (intention to adopt) table 6 shows the frequency distribution of the items used to measure the perception of smes to adopt cloud computing. the means of the items of adopt ranged from 𝑋𝑋 = 5.43 (stdev. 1.759) to 𝑋𝑋 = 5.57 (stdev. 1.569). the highest mean was for item adopt2: ‘our organisation feels that organisations’ needs can be met by cloud computing’ (𝑋𝑋 = 5.57; stdev. 1.569). the lowest mean was for item adopt4: ‘our organisation will adopt cloud computing within the next 12 months’ (𝑋𝑋 = 5.43; stdev. 1.759). the overall mean of adopt was 𝑋𝑋 = 5.51, which indicates a high degree of agreement by smes to adopt cloud computing. the majority of the respondents agreed or strongly agreed that they are planning or intending to adopt cloud computing in the near future. 10 australasian journal of information systems senarathna, yeoh, warren & salzman 2016, vol 20, research article security & privacy for sme cloud adoption item sme responses mean std. dev. strongly disagree disagree somewhat disagree neither agree nor disagree somewhat agree agree strongly agree adopt1 8 7 3 17 5 69 41 5.50 1.670 adopt2 7 4 4 18 7 69 41 5.57 1.569 adopt3 7 6 5 16 6 69 41 5.53 1.625 adopt4 10 7 4 17 2 69 41 5.43 1.759 overall mean 5.51 table 6: frequency distribution of the adopt 4.2 cloud security table 7 depicts the frequency distribution of the items of cs. the means of the items of cs ranged from 𝑋𝑋 = 4.29 (stdev. 0.824) to 𝑋𝑋 = 4.92 (stdev. 0.799). the highest mean was for item cs1: ‘our organisation is not concerned about the security of the cloud computing’ (𝑋𝑋 = 4.92; stdev. 0.799). the lowest mean was for item cs2: ‘our organisation considers that cloud computing is more secure than traditional computing’ (𝑋𝑋 = 4.29; stdev. 0.824). the overall mean of cs was 𝑋𝑋 = 4.60, which indicates a fair degree of agreement of smes’ perceptions of cs to adopt cloud computing. this factor was not correlated with adopt, with a correlation coefficient of only 0.006. item sme responses mean std. dev. strongly disagree disagree somewhat disagree neither agree nor disagree somewhat agree agree strongly agree cs1 0 0 5 36 78 28 3 4.92 0.799 cs2 0 0 20 80 37 12 1 4.29 0.824 cs3 0 1 16 60 49 16 8 4.58 1.018 overall mean 4.60 table 7: frequency distribution of the cs 4.3 cloud privacy table 8 presents the frequency distribution of the items of cp. the means of the items of cp ranged from 𝑋𝑋 = 3.19 (stdev. 1.161) to 𝑋𝑋 = 3.57 (stdev. 1.200). the highest mean was for item cp3: ‘our organisation feels cloud computing can be trusted’ (𝑋𝑋 = 3.57; stdev. 1.200). the lowest mean was for item cp1: ‘our organisation prefers to store data in the cloud data centre located within australia’ (𝑋𝑋 = 3.19; stdev. 1.161). the overall mean of cp was 𝑋𝑋 = 3.34, which indicates a fair agreement of smes’ perception of cp to adopt cloud computing. this factor was not correlated with adopt, with a correlation coefficient of only 0.044. in short, the results show that, although cloud security and privacy factors are considered somewhat important by the smes, these factors do not significantly influence the smes’ decision to adopt cloud computing. 11 australasian journal of information systems senarathna, yeoh, warren & salzman 2016, vol 20, research article security & privacy for sme cloud adoption item smes responses mean std. dev. strongly disagree disagree somewhat disagree neither agree nor disagree somewhat agree agree strongly agree cp1 8 37 47 40 13 5 0 3.19 1.161 cp2 11 29 42 52 12 4 0 3.25 1.158 cp3 9 18 36 61 17 9 0 3.57 1.200 overall mean 3.34 table 8: frequency distribution of the cp next, the hypotheses proposed in the study were tested using sem. the model fit was evaluated with comparative fit index (cfi), root mean square error of approximation (rmsea) and relative chi-square (cmin/df). the results indicated a good model fit, as the cfi was 0.985, the rmsea was 0.073, and the cmin/df was 1.803 (i.e., less than 3). as shown in figure 2, the analysis revealed that cloud security and privacy factors do not have a significant influence in relation to cloud computing adoption by smes. figure 2. results of amos structural model analysis hypothesis 1, which proposes that higher levels of privacy in the cloud computing environment, as perceived by smes, may motivate them to adopt cloud computing services, was not supported by the analysis (path coefficient = 0.10, p>0.5). this finding is not consistence with previous research by chen and chang (2013), indicating that privacy influence the decision of technological innovation adoption. the second hypothesis (h2), which proposes that higher levels of security in the cloud computing environment may motivate smes to adopt cloud computing services, was not supported (path coefficient = 0.11, p>0.5). a possible explanation for this may be that the smes were not fully aware of the security and privacy issues related to internet-based cloud computing provision. it is also possible that smes, which are usually on a tight budget, are more concerned about costeffectiveness rather than security and privacy issues. as can be observed in table 9, the anova analysis indicated that the sme location difference between metropolitan and regional areas was not statistically significant (f (1, 148) 1.601 and p>0.05). in other words, this result revealed that variances of cloud computing adoption among the locations of the smes do not differ. therefore, h3 is not supported. this finding is in contrast with the assertions of previous studies (e.g., carcary et al., 2014; minifie, 2014). for 12 australasian journal of information systems senarathna, yeoh, warren & salzman 2016, vol 20, research article security & privacy for sme cloud adoption instance, minifie (2014) argued that regional location was an inhibiter for cloud computing, and recommended that the bottlenecks in the suburbs and regional areas should be addressed. carcary et al. (2014) suggested location as a potential barrier to cloud computing adoption. however, the present finding indicates that the perceptions of australian smes considering cloud adoption is not different between metropolitan and regional areas. a possible explanation for this is that internet connection is easily accessible in regional areas, offering smes convenient access to cloud computing services. in fact, according to the australian communications and media authority, 94% of smes in australia, including those in regional areas, were connected to the internet, with 87% of these utilising a broadband service (acma, 2014). hence, the location of the sme does not exert a significant influence on the cloud adoption. sum of squares df mean square f sig. between groups within groups total 1.601 140.631 142.232 1 148 149 1.601 .950 1.685 .196 table 9: sme geographical location and cloud adoption anova 4.4 contributions this paper contributes to cloud adoption research. the study bridges the research gap and provides insightful evidence on these issues, especially within the context of australian smes. it is one of the first scholarly, empirical studies of australian smes, with 150 observations across metropolitan and regional areas. contrary to the existing literature (tang & liu, 2015; vanessa, 2014; dillon & vossen, 2014), the survey findings indicate that company location, security and privacy factors do not play the most significant role in the decision-making process of smes considering whether to adopt cloud computing. a possible reason for this might be that they are not fully aware of cloud computing and its nature. also a fair explanation is that smes are typically less concerned about security and privacy threats, in part because they do not have dedicated it staff and the associated knowledge. thus, this study extends our current understanding of cloud computing adoption by australian smes and contributes to the literature of cloud computing and organisational innovation adoption. additionally, a methodological contribution of the study is the synthesis of the existing literature to define forms of measurement and operationalisation. the research constructs were developed through a comprehensive review of theoretical, practitioner and government underpinnings. as the field of cloud computing is still relatively new, the research constructs provide foundations for further studies on the adoption of specific cloud services, such as iaas, paas and saas. this study has practical contributions. the new understandings (i.e., that security and privacy factors do not possess significant influences on smes) enable smes, cloud service providers, it practitioners and policy makers to concentrate on other critical factors that have more impact on cloud computing adoption, such as broadband affordability and speed. as smes are generally not capable of spending significant amounts of investment on ict, it is essential for cloud service providers, professional bodies and the government to devise strategies for the widespread adoption of cloud computing for this sme category. it is also essential to reduce the feeling of uncertainty regarding cloud computing adoption by smes through a wider coverage of nbn, especially for those in regional areas. 4.5 limitations and future research as with any research work, this research has some limitations that also offer opportunities for future studies. the first limitation is that this research focuses only on smes within australia, 13 australasian journal of information systems senarathna, yeoh, warren & salzman 2016, vol 20, research article security & privacy for sme cloud adoption excluding smes from other countries. therefore, the findings of this study are limited and should be considered carefully in other regions, as they may not be applicable to smes from different parts of the world. hence, this study would have benefited from the inclusion of perspectives of smes from other countries. second, this study is limited in terms of its qualitative data. further qualitative research, such as case studies and longitudinal studies, are required to gain a solid understanding of this phenomenon. third, cloud computing adoption by smes is still in its initial stage, and further detailed research incorporating other important aspects in this area is required. future studies could focus on variables such as cultural factors and the differences in legal systems in the use of technology throughout different countries. 5 conclusion this paper has empirically examined how security and privacy concerns for cloud computing are influencing the adoption by smes in australia. through an online survey method, this study concludes that security and privacy factors are not significantly influential for australian smes in cloud adoption decisions. further, the findings reveal that cloud computing adoption is not influenced by the geographical location of the smes (i.e., metropolitan vs regional). this research contributes to the emerging literature of cloud computing adoption. references abadi, d. j. (2009) "data management in the cloud: limitations and opportunities", ieee data engineering bulletin, 32(1): 3-12. acca (2012) cloud readiness index 2012, asia cloud computing association. acma (2014) communications report 2013-2014, the australian communications and media authority, 1-130. adam, i.o. (2015) 'small and medium enterprises (smes) in the cloud in developing countries: a synthesis of the literature and future research directions', journal of management and sustainability, 5(1): 115-139. ahuja, s. p. & rolli, a. c. (2011) "survey of the state-of-the-art of cloud computing", international journal of cloud applications and computing, 1(4): 34-43. alkhater, n., wills, g. & walters, r. (2014) 'factors influencing an organisation's intention to adopt cloud computing in saudi arabia', ieee 6th international conference on cloud computing technology and science (cloudcom), 15-18 dec 2014, 1040-1044. alshamaila, y., papagiannidis, s. & li, f. (2013) cloud computing adoption by smes in the north east of england: a multi-perspective framework. journal of enterprise information management, 26(3): 250-275. anthes, g. (2010) "security in the cloud", communications of the acm, 53(11): 16-18. anthony, b. (2012) "forecast: cloudy but fine?", privacy risks and potential benefits in the cloud, victoria privacy commissioner. armbrust, m., fox, a., griffith, r., joseph, a.d., katz, r., konwinski, a., lee, g., patterson, d., rabkin, a., stoica, i.o.n. & zaharia, m. (2010) 'a view of cloud computing', communications of the acm, 53(4): 50-8. aziz, n. (2010) french higher education in the cloud, grenoble, france: grenoble ecole de management. behl, a. (2011) emerging security challenges in cloud computing: an insight to cloud security challenges and their mitigation. world congress on information and communication technologies (wict), 217-222. 14 australasian journal of information systems senarathna, yeoh, warren & salzman 2016, vol 20, research article security & privacy for sme cloud adoption benlian, a., koufaris, m. & hess, t. (2011) service quality in software-as-a-service: developing the saas-qual measure and examining its role in usage continuance. journal of management information systems, 28(3): 85-126. bharadwaj, s.s. & lal, p. (2012) "exploring the impact of cloud computing adoption on organizational flexibility: a client perspective”, international conference on cloud computing technologies, applications and management (iccctam), 121-131. bhayal, s. (2011) a study of security in cloud computing, available from proquest dissertations and theses. (umi no: 1504430) bryman, a. & bell, e. (2015). business research methods, oxford university press. carr, n.g. (2005) "the end of corporate computing", mit sloan management review 46(3): 67-73. carcary, m., doherty, e. & conway, g. (2014) the adoption of cloud computing by irish smes–an exploratory study. electronic journal information systems evaluation, 17(1):3-14. chakraborty, r., ramireddy, s., raghu, t.s. & rao, h.r. (2010) "the information assurance practices of cloud computing vendors", it professional 12(4): 29-37. chang, v., walters, r.j. & wills, g. (2013) "the development that leads to the cloud computing business framework", international journal of information management, 33(3): 524-538. chen, k.y. & chang, m.l. (2013) “user acceptance of ‘near field communication’ mobile phone service: an investigation based on the unified theory of acceptance and use of technology model”, the services industries journal, 33(6): 609-623. cheng, s.k. & kam, b.h. (2008) a conceptual framework for analysing risk in supply networks. journal of enterprise information management, 21(4): 345-360. chin, w.w., marcolin, b.l. & newsted, p.r. (2003) a partial least squares latent variable modeling approach for measuring interaction effects: results from a monte carlo simulation study and an electronic-mail emotion/adoption study. information systems research, 14(2): 189-217. dave, a. (2012) the state of cloud computing security in asia, trend micro, 1-5. dbcde (2013) the national cloud computing strategy, department of broadband, communication and the digital economy, 1-38. dfd (2011) "negotiating the cloud – legal issues in cloud computing agreements", department of finance and deregulation, 1-18. diisr (2011) key statistics: australian small business, australian government, 1-56. dillon, t., wu, c. & chang, e. (2010) "cloud computing: issues and challenges”, 27-33. doc (2014) cloud computing regulatory stock take, australian government, 1-91. drew, s. (2003) strategic uses of e-commerce by smes in the east of england. european management journal, 21(1): 79-88. ericson, j. (2015) cloud computing and bi 2015: three takeaways, available at: http://www.dresneradvisory.com/2015_cloud_computing_and_bi [accessed 20 april 2015]. fakieh, b., blount, y. & busch, p. (2014) success in the digital economy: cloud computing, smes and the impact to national productivity. paper presented at the 25th australasian conference on information systems, auckland, new zealand. fairchild, a.m. (2014) 'patterns of trust: role of certification for sme cloud adoption', in continued rise of the cloud, springer, 145-58. 15 australasian journal of information systems senarathna, yeoh, warren & salzman 2016, vol 20, research article security & privacy for sme cloud adoption featherman, m. & pavlou, p. (2003) 'predicting e-services adoption: a perceived risks facets perspective', international journal of human-computer studies, 59(1): 451-74. feuerlicht, g., burkon, l. & sebesta, m. (2011) cloud computing adoption: what are the issues, systémová integrace. fornell, c. & larcker, d.f. (1981) evaluating structural equation models with unobservable variables and measurement error. journal of marketing research, 18(1): 39-50. friedman, a.a. & west, d.m. (2010) privacy and security in cloud computing, issues in technology innovation, center for technology innovation at brookings, (3). gangwar, h., date, h., ramaswamy, r., irani, z. & irani, z. (2015) understanding determinants of cloud computing adoption using an integrated tam-toe model. journal of enterprise information management, 28(1): 107-130. garg, r. & stiller, b. (2014) 'design and evaluation of an impact analysis methodology for the adoption of cloud-based services (iamcis)', in network and service management (cnsm), 2014, 10th international conference, 260-263. gengatharen, d. e. & standing, c. (2005) a framework to assess the factors affecting success or failure of the implementation of government-supported regional e-marketplaces for smes. european journal of information systems, 14(4): 417-433. gens, f., mahowald, r., villars, r.l., bradshaw, d. & morris, c. (2009) cloud computing 2010: an idc update, available at: http://www.slideshare.net/jorfigor/cloudcomputing-2010-an-idc-update [accessed 13 august 2012]. george, f. & shyam, g. (2010) "impact of cloud computing: beyond a technology trend", systems integration, 262-269. gonzalez, n., miers, c., redígolo, f., simplicio, m., carvalho, t., näslund, m. & pourzandi, m. (2012) 'a quantitative analysis of current security concerns and solutions for cloud computing', journal of cloud computing, 1(1): 1-18. grover, v., cheon, m.j. & teng, j.t. (1996) 'the effect of service quality and partnership on the outsourcing of information systems functions', journal of management information systems, 12(4): 89-116. gupta, p., seetharaman, a. & raj, j.r. (2013) 'the usage and adoption of cloud computing by small and medium businesses', international journal of information management, 33(5): 861-74. hailu, a. (2012) 'factors influencing cloud-computing technology adoption in developing countries', phd thesis. hair, j. f., black, w.c., babin, b.j. & anderson, r.e. (2010) multivariate data analysis, (7 ed.) englewood cliffs: prentice hall. handler, d.p., barbier, j. & schottmiller (2012) smb public cloud adoption: opening a hidden market, cisco internet business solutions group. herhalt, j. & cochrane, k. (2012) exploring the cloud a global study of governments’ adoption of cloud, klynveld peat marwick goerdeler (kpmg), 1-46. hutley, n. (2012) modelling the economic impact of cloud computing, kpmg, 1-52. idc (2015) worldwide cloud it infrastructure market growth expected to accelerate to 21% in 2015, driven by public cloud datacenter expansion, according to idc, international data corporation, available at: https://www.idc.com/getdoc.jsp?containerid=pr us25576415 [accessed 13 december 2015]. imo (2013) privacy and cloud computing for australian government agencies, version 1.1', 1-10. 16 australasian journal of information systems senarathna, yeoh, warren & salzman 2016, vol 20, research article security & privacy for sme cloud adoption itiic (2011) cloud computing: opportunities and challenges, department of innovation industry, science and reasearch, 1-31. ita (2015) cloud computingtop markets report, international trade administration, 1-38. jamil, d. & zaki, h. (2011) 'security issues in cloud computing and countermeasures', international journal of engineering science and technology (ijest), 3(4): 2672-6. jamwal, d., sambyal, a. & sambyal, g.s. (2011) "cloud computing: its security & privacy aspects", international journal of latest trends in computing, 2(1): 25-28. jansen, w. & grance, t. (2011) guidelines on security and privacy in public cloud computing, us department of commerce. katzan, h. (2010) 'on the privacy of cloud computing', international journal of management & information systems, 14(2): 1-12. kauffman, r. j., ma, d. & yu, m. (2014) a metrics suite for firm-level cloud computing adoption readiness economics of grids, clouds, systems, and services, springer,19-35. ko, r. k., jagadpramana, p., mowbray, m., pearson, s., kirchberg, m., liang, q. & lee, b. s. (2011) trustcloud: a framework for accountability and trust in cloud computing, in ieee world congress on services (services), 584-588. kshetri, n. (2013) 'privacy and security issues in cloud computing: the role of institutions and institutional evolution', telecommunications policy, 37(4): 372-86. lawrence, m.w.l., brad, d.c.c., chris, c. & denna, m. (2010) cloud computing business models for the channel, a compti a cloud/saas community resource, 1-12. lee, j.n. & kim, y.g. (1999) "effect of partnership quality on is outsourcing success: conceptual framework and empirical validation." journal of management information systems, 15(4): 29-60. lemt (2011) the potential for cloud computing services in australia a lateral economics report to macquarie telecom, 1-45. li, m., zhao, d. & yu, y. (2015) 'toe drivers for cloud transformation: direct or trustmediated?', asia pacific journal of marketing and logistics, 27(2): 226-48. li, x., hess, t.j. & valacich, j.s. (2008) 'why do we trust new technology? a study of initial trust formation with organizational information systems', journal of strategic information systems, 17(1): 39-71. mahmood, z. (2011) data location and security issues in cloud computing. international conference on emerging intelligent data and web technologies (eidwt), 49-54. malhotra, n.k. (2010) marketing research: an applied orientation, pearson upper saddle river, nj. mark, d.r. (2011) cloud computing privacy concerns on our doorstep, communications of the acm, 54(1): 36-38. marston, s., li, z., bandyopadhyay, s., zhang, j. & ghalsasi, a. (2011) "cloud computing-the business perspective", decision support systems, 51(1): 176-189. matthew, l.j. (2014) national broadband network (nbn) 2014, available at: http://www.aph.gov.au/about_parliament/parliamentary_departments/parliamentar y_library/pubs/rp/budgetreview201314/nbn [accessed 2 december 2015]. mccabe, b. & hancook, i. (2009) cloud computing: australian lessons and experiences, kpmg, 1-20. miller, m. (2014) cloud adoption in australia, available at: http://www.abc.net.au/technology/articles/2014/10/24/4113810.htm [accessed 5 january 2015]. 17 australasian journal of information systems senarathna, yeoh, warren & salzman 2016, vol 20, research article security & privacy for sme cloud adoption minifie, j. (2014) the silver lining: cloud computing and small and medium enterprises, sydney, australia. misra, s.c. & mondal, a. (2010) "identification of a company’s suitability for the adoption of cloud computing and modelling its corresponding return on investment", mathematical and computer modelling, 53(3): 504-521. monika, s., ashwani, m., haresh, j., anand, k., madhvendra, m. & vijayshri, t. (2010) "scope of cloud computing for smes in india", journal of computing, 2(5): 144-149. mudge, j.c. (2010) cloud computing opportunities and challenges for australia, atse, melbourne, victoria, 1-34. nbnco (2015a) the cloud – what’s all the fuss about, available at: http://www.nbnco.com.au/blog/the-cloud-whats-all-the-fuss-about.html [accessed 10 november 2015]. nbnco (2015b) the cloud: what can it do for aussie businesses?, available at: http://www.nbnco.com.au/blog/the-cloud-what-an-it-do-for-aussie-businesses.html [accessed 16 november 2015]. oliveira, t., thomas, m. & espadanal, m. (2014) "assessing the determinants of cloud computing adoption: an analysis of the manufacturing and services sectors", information & management, 51(5): 497-510. opala, o.j. (2012) an analysis of security, cost-effectiveness, and it compliance factors influencing cloud adotion by it managers, available from proquest dissertations and theses. pearson, s. (2009) "taking account of privacy when designing cloud computing services", workshop on software engineering challenges of cloud computing (cloud'09), 4452. pearson, s. (2012) privacy, security and trust in cloud computing, 1-57. pearson, s. & benameur, a. (2010) privacy, security and trust issues arising from cloud computing. annual international conference cloud computing technology and science (cloudcom), 693–702. plummer, d.c., smith, d.m., bittman, t.j., cearley, d.w., cappuccio, d.j., scott, d., kumar, r. & robertson, b. (2009) "five refining attributes of public and private cloud computing", gartner research. podsakoff, p.m., mackenzie, s.b., lee, j.y. & podsakoff, n.p. (2003) common method biases in behavioral research: a critical review of the literature and recommended remedies. journal of applied psychology, 88(5), 879-903. ren, k., wang, c. & wang, q. (2012) security challenges for the public cloud. ieee internet computing, 16(1): 69-73. rittinghouse, j.w. & ransome, j.f. (2009) cloud computing: implementation, management, and security, crc press: new york, london. robert, c.m. & mira, k. (2010) perception of barriers to e-commerce adoption in smes in a developed and developing country: a comparison between australia and indonesia, journal of electronic commerce in organizations, 8(1): 61-82. ross, p.k. & blumenstein, m. (2015) 'cloud computing as a facilitator of sme entrepreneurship', technology analysis & strategic management, 27(1): 87-101. safari, f., safari, n. & hasanzadeh, a. (2015) 'the adoption of software-as-a-service (saas): ranking the determinants', journal of enterprise information management, 28(3): 400-422. 18 australasian journal of information systems senarathna, yeoh, warren & salzman 2016, vol 20, research article security & privacy for sme cloud adoption sahandi, r., alkhalil, a. & justice, o.m. (2012) "smes’ perception of cloud computing: potential and security", collaborative networks in the internet of services, springer, 186-195. salleh, s.m., teoh, s.y. & chan, c. (2012) "cloud enterprise systems: a review of literature and its adoption", pasis 2012 proceedings, 1-11. sarwar, a. & khan, m.n. (2013) 'a review of trust aspects in cloud computing security', international journal of cloud computing and services science (ij-closer), 2(2): 116-22. sasikala, p. (2011) "cloud computing in higher education", international journal of cloud applications and computing, 1(2): 1-13. schoorman, f.d., mayer, r.c. & davis, j.h. (2007) 'an integrative model of organizational trust: past, present, and future', academy of management review, 32(2): 344-54. sen, j. (2013) security and privacy issues in cloud computing. architectures and protocols for secure information technology infrastructures, 1-45. senarathna, i., warren, m., yeoh, w. & salzman, s. (2015) 'a conceptual model for cloud computing adoption by smes in australia', in delivery and adoption of cloud computing services in contemporary organizations, igi-global, 100-128. shaikh, f.b. & haider, s. (2011) 'security threats in cloud computing', in internet technology and secured transactions (icitst), 2011 international conference, 214-219. shimba, f. (2010) cloud computing: strategies for cloud computing adoption. dissertation. stieninger, m. & nedbal, d. (2014) 'diffusion and acceptance of cloud computing in smes: towards a valence model of relevant factors', in 47th hawaii international conference on system sciences (hicss 2014), 3307-3316. straub, d.w. (1989) 'validating instruments in mis research', mis quarterly, 13(2): 147-69. subashini, s. & kavitha, v. (2011) 'a survey on security issues in service delivery models of cloud computing', journal of network and computer applications, 34(1): 1-11. sullivan, d. (2010) the definitive guide to cloud computing, realtime publishers, ibm. sultan, n. (2010) "cloud computing for education: a new dawn?", international journal of information management, 30(2): 109-116. sultan, n. a. (2011) "reaching for the “cloud”: how smes can manage", international journal of information management, 31(3): 272-278. swi (2012) the power of cloud computing for small businesses. software world, 43(3): 1112. tancock, d., pearson, s. & charlesworth, a. (2013) 'a privacy impact assessment tool for cloud computing', in privacy and security for cloud computing, springer, 73-123. tang, c. & liu, j. (2015) 'selecting a trusted cloud service provider for your saas program', computers & security, 50(1): 60-73. tarmidi, m., rasid, s.z.a., alrazi, b. & roni, r.a. (2014) 'cloud computing awareness and adoption among accounting practitioners in malaysia', procedia-social and sciences, 164(1): 569-74. tcp (2015) regional australia postcode list, available at: http://www.travellers.com.au/default.aspx?tabid=505 [accessed 25 december 2015]. tehrani, s. r. & shirazi, f. (2014) factors influencing the adoption of cloud computing by small and medium size enterprises (smes) human interface and the management of information, information and knowledge in applications and services, springer, 631642. 19 australasian journal of information systems senarathna, yeoh, warren & salzman 2016, vol 20, research article security & privacy for sme cloud adoption telstra (2011) moving to cloud: key considerations for government. telstra report, 1-21. vanessa, r. (2014) "a us-china comparative study of cloud computing adoption behavior: the role of consumer innovativeness, performance expectations and social influence", journal of entrepreneurship in emerging economies, 6(1): 53-71. wijesiri, s. (2010) cloud computing a new wave in it, daily news. xiao, z. & xiao, y. (2013) 'security and privacy in cloud computing', communications surveys & tutorials, ieee, 15(2): 843-59. xu, h. & gupta, s. (2009) 'the effects of privacy concerns and personal innovativeness on potential and experienced customers' adoption of location-based services', electronic markets, 19(2): 137-49. zissis, d. & lekkas, d. (2012) 'addressing cloud computing security issues', future generation computer systems, 28(3): 583-92. zhang, q., cheng, l. & boutaba, r. (2010) "cloud computing: state-of-the-art and research challenges", journal of internet services and applications 1(1): 7-18. zhao, f., scheruhn, h.j. & rosing, m. (2014) the impact of culture differences on cloud computing adoption human-computer interaction. applications and services, springer, 776-785. copyright: © 2016 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. 20 http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 literature review and hypotheses 2.1 cloud computing 2.2 cloud computing adoption in smes 2.3 hypotheses development 2.4 cloud privacy 2.5 cloud security 2.6 metropolitan vs regional 3 methodology 3.1 measurement reliability and validity 4 results and discussion 4.1 adopt (intention to adopt) 4.2 cloud security 4.3 cloud privacy 4.4 contributions 4.5 limitations and future research 5 conclusion paper title (paper title style) australasian journal of information systems faber, hernandez-mendez, rehm & matthes 2020, vol 24, selected papers from acis 2018 collaborative modelling & visualization of business ecosystems 1 collaborative modelling and visualization of business ecosystems: insights from two action design research case studies anne faber technical university of munich, department of informatics garching bei muenchen, germany anne.faber@tum.de adrian hernandez-mendez technical university of munich, department of informatics garching bei muenchen, germany sven-volker rehm university of strasbourg, em strasbourg business school, humanis (ea 7308) strasbourg, france florian matthes technical university of munich, department of informatics garching bei muenchen, germany abstract business ecosystems are increasingly gaining relevance in research and practice. because ecosystems progressively change, enterprises are required to analyse their ecosystem, in order to identify and respond to such changes. for gaining a comprehensive picture of the ecosystem, various enterprise stakeholders need to be involved in the analysis process. we use an action design research approach to implement a collaborative process for modelling and visualizing business ecosystems in two case studies. we look at the challenges of the collaborative process and study how a model-driven approach addresses these challenges. we validate and discuss the modelling process along six steps; definition of the business ecosystem focus, model instantiation, data collection, provision of tailored visualizations, model adaption, and using visualizations ‘to tell a story’. in a cross-case analysis, we draw conclusions with respect to process implementation and the role of visualizations. keywords: business ecosystem, collaborative modelling, ecosystem visualization, case study, action design research 1 introduction1 undoubtedly, firms increasingly recognize the relevance of their complex business environment in which they develop, produce and distribute their services and products; this environment is often referred to as business ecosystem (kapoor & argarwal, 2017; parker, van alstyne, & jiang, 2017; sako, 2018). the growing relevance of business ecosystems 1 this paper is an extension of the conference paper entitled “visualizing business ecosystems: applying a collaborative modelling process in two case studies” by the same authors, published in the conference proceedings of the 29th australasian conference on information systems, track: digital business innovation, sydney, australia, 3-5 december 2018, where it has been granted the acs best paper prize third place. australasian journal of information systems faber, hernandez-mendez, rehm & matthes 2020, vol 24, selected papers from acis 2018 collaborative modelling & visualization of business ecosystems 2 substantiates through the perceived shift of the competitive environment from single companies and their supply chains towards ecosystems competing against each other (bosch, 2016; parker, van alstyne, & jiang, 2017). addressing the associated challenges and opportunities has thus become a reality for most companies (peltoniemi & vuori, 2004). research on business ecosystems has looked at their networked structure and strategic relevance (moore, 1993; 1996; adner, 2016), emphasized their dynamics and evolution (adner & kapoor, 2016; peltoniemi & vuori, 2004), and discussed aspects such as ‘healthiness’ or the role of information technologies (kim, lee, & han, 2010; sako, 2018). extant research has emphasized that as of today, the economic success of an enterprise significantly depends on its capability to manage and evolve in relation to its business ecosystem (parker, van alstyne, & jiang, 2017; huber, kude, & dibbern, 2017). in this regard, the role of an enterprise within its ecosystem can range from a keystone to a niche player (iansiti & levien, 2004). despite acknowledging the overall significance of business ecosystems for a single firm, research has remained comparably silent about what exactly a firm can do to analyse their business ecosystem in order to identify and respond to changes within their ecosystem (basole, huhtamäki, still, & russell, 2016), adapt own business activities accordingly and to “learn what makes the environment tick” (porter, 1979). one way to support decision makers in their ecosystem-related tasks are visualizations of business ecosystems (basole, huhtamäki, still, & russell, 2016; huhtamaki & rubens, 2016; evans & basole, 2016). visualizing information about the firm environment and relationships can help to spot anomalies, identify keystone and niche players, or recognize change patterns and trends prevalent in the ecosystem (vartak, huang, siddiqui, madden, & parameswaran, 2017). however, deriving value from information about ecosystems has proven challenging due to several factors including the nature, availability and volume of required data, as well as data collection and analysis processes (basole, russell, huhtamäki, rubens, still, & park, 2015; iyer & basole, 2016; rehm, faber, & goel, 2017). ecosystem data is large and heterogeneous, ranging from technology-related data about applied standards and platforms to use, to market information and legal regulations (basole, russell, huhtamäki, rubens, still, & park, 2015; faber, rehm, hernandez-mendez, & matthes, 2018a). when focusing on business aspects of a firm’s business ecosystem, information about business partners, competitors, interesting startups and their strategies, partnerships and offered solutions, and cooperative initiatives become relevant and need to be integrated into a coherent business ecosystem model (faber, hernandez-mendez, rehm, & matthes, 2018a). relevant data can originate from various sources, such as existing databases, newspaper articles or blogs addressing recent developments within the ecosystem, but also company and institutional web presences and publications. hence, the issues associated to data collection—particularly when taking into account the evolutionary nature of business ecosystems—are not yet resolved (iyer & basole, 2016; hao, zhu, & zhong, 2015). this state of play poses particular challenges for generating and analysing visualizations for business development (rehm, faber, & goel, 2017). in addition, from a use perspective, various stakeholders need to be included in the process of ecosystem visualization and analysis who represent the interests of a firm’s strategy, business units or functions, and who bring in diverse expertise and perspectives (rehm, faber, & goel, 2017). we are building on previous research in the domain of collaborative modelling that provides an approach for integrating stakeholder groups, including them into cooperative, shared model australasian journal of information systems faber, hernandez-mendez, rehm & matthes 2020, vol 24, selected papers from acis 2018 collaborative modelling & visualization of business ecosystems 3 creation and evaluation (dollmann, houy, fettke, & loos, 2011). we aim at providing a collaborative approach to model and visualize business ecosystems from a company-internal perspective. particularly, we address the following research question: what are the challenges of collaborative approaches for modelling and visualizing business ecosystems, and how does a model-driven approach address these challenges? our contributions involve the description of (1) a process to initiate the modelling of business ecosystems, (2) important roles during the modelling process, and (3) lessons learned from two action design research case studies we conducted with two companies of different business ecosystems. in section 2 we introduce related work regarding the business ecosystem concept, visualization and collaborative modelling of business ecosystems. in section 3, we describe the research method; section 4 outlines the case studies. in section 5 we discuss our case study insights and summarize lessons learned. finally, in section 6, we provide a short conclusion. 2 related work 2.1 the concept of business ecosystem the term business ecosystem was coined by james f. moore in the mid-1990s, in analogy to the metaphor of a biological ecosystem (moore, 1993). since its introduction that defined it as a collection of interacting companies (moore, 1993), the concept has been widely studied and used in management science (guittard, schenk, & burger-helmchen, 2015). the initial definition was enriched by including various roles of firms as “suppliers, distributors, outsourcing firms, makers of related products or services, technology providers, and a host of other organizations” (iansiti & levien, 2004). thus, the concept of business ecosystem extends the idea of the classic supply chain, consisting of suppliers and customers, by including further entities of a firm’s business environment. we define business ecosystems holistically, as the environment of an enterprise including current and potential future business partners and relationships, involving entities such as customers, suppliers, competitors, regulatory institutions, and innovative start-ups. ecosystems usually exhibit high dynamics as continuously firms enter and leave the ecosystem (see peltoniemi & vuori, 2004 for a comprehensive discussion). therefore, business ecosystems constantly evolve, exhibiting a dynamic structure (peltoniemi & vuori, 2004), with not only enterprises but also human actors entering and leaving the ecosystem, which “are interconnected through a complex, global network of relationships” (basole, russell, huhtamäki, rubens, still, & park, 2015), and which all potentially affect business success and failure of other firms active within the ecosystem. the role of a single enterprise within its ecosystem can range from a keystone to a niche player, with varying level of influence on the overall health of the ecosystem (iansiti & levien, 2004). the business ecosystem concept is distinct from extant concepts such as clusters or networks as it focuses on “value-creating activity (…) rather than an industrial sector” (sako, 2018). in this respect, sustainability, self-governance and evolution have been defined as three metacharacteristics of business ecosystems (sako, 2018). current research considers frameworks to grasp the scope of ecosystem complexity (iyer & basole, 2016), visualizations to understand emerging structures and patterns (iyer & basole, 2016), or policy contexts such as smart city australasian journal of information systems faber, hernandez-mendez, rehm & matthes 2020, vol 24, selected papers from acis 2018 collaborative modelling & visualization of business ecosystems 4 (visnjic, neely, cennamo, & visnjic, 2016; rehm, faber, & goel, 2017). in addition, various types of ecosystems have drawn researchers’ attention as for instance, the internet of things (iot) business ecosystems, addressing business ecosystem design methods (uchihira, ishimatsu, & inoue, 2016). 2.2 business ecosystem visualization visualizations of business ecosystems enable ecosystem stakeholders to take better-informed decisions (basole, huhtamäki, still, & russell, 2016; huhtamaki & rubens, 2016; evans & basole, 2016). research addressing ecosystem visualizations has used data sets collected from commercial databases on business and economic data or from social or business media (basole, russell, huhtamäki, rubens, still, & park, 2015). similar to the framework we used in our studies, park, bellamy, & basole (2016) present a visual analytic system for analysis of a supply chain management ecosystem. the authors identify three salient design requirements: (1) to support multiple views in an integrated interface, (2) to enable interactive investigation of supply networks, and (3) to provide data-driven analytic capabilities. in their scenario, system users are enabled to interactively explore the ecosystem model using multiple views, all integrated in one user interface. in addition, data-driven analytics are provided. the system introduces five network layouts, which are force-directed, circular or chord diagram, tree map, matrix, and substrate-based layout. thereby, all visualizations provide interactive features, such as clicking, dragging, hovering, and filtering. these authors’ further work comprehensively adds to research in the area of modelling, visualizing and analysing business ecosystems (basole, russell, huhtamäki, rubens, still, & park, 2015; park, bellamy, & basole, 2016; park & basole, 2016; basole, 2009a; basole, 2009b). 2.3 collaborative modelling of business ecosystems collaborative modelling originated in the 1970s and has since increasingly gained relevance, mirroring the increased need for collaboration amongst experts (renger, kolfschoten, & de vreede, 2008). collaborative modelling has been applied to various research fields, such as business process modelling (dollmann, houy, fettke, & loos, 2011), enterprise architecture modelling (roth, hauder, & matthes, 2013), or group decision support system modelling (liu & zhang, 2010). various roles have been identified as being essential for collaborative modelling, whereby these roles can be allocated to different persons—or several roles can be assigned to the same person (richardson & andersen, 1995; renger, kolfschoten, & de vreede, 2008). generic roles comprise: (a) facilitator, monitoring the group process and stimulating the model building effort; (b) modeller, focusing on the model outcome; (c) process coach, observing the process and the dynamics of the participants; (d) recorder, documenting the modelling process; and (e) gatekeeper, responsible for the process and major decision maker (richardson & andersen, 1995). australasian journal of information systems faber, hernandez-mendez, rehm & matthes 2020, vol 24, selected papers from acis 2018 collaborative modelling & visualization of business ecosystems 5 however, extant literature does not suggest collaborative processes that specifically address business ecosystem modelling and the instantiation of such models. 3 method 3.1 action design research approach our research intends to provide insights into challenges of collaborative approaches for modelling and visualizing business ecosystems, and into how model-driven approaches address these challenges. as we have the aim to inform both research and practice we position our work as action design research (adr) (avison, lau, myers, & nielsen, 1999; baskerville & myers, 2004; hevner, march, park, & ram, 2004; sein, henfridsson, purao, rossi, & lindgren, 2011). due to the lack of existing research—or reports from practice—about such collaborative approaches, and in order to comprehensively understand related issues and challenges, we focus our work on instantiating such a process (sherer, 2014). we do this on basis of a collaborative work information system, which we refer to as business ecosystem explorer (beex). adr allows generating prescriptive design knowledge while implementing and evaluating design artefacts in distinct organizational settings (sein, henfridsson, purao, rossi, & lindgren, 2011). in our study, the collaborative modelling and visualization processes represent a methodical design artefact, while the employed collaborative work system’s visualization capabilities— which have been adapted during our research—represent a technological design artefact. our report focuses in particular on aspects of developing, testing and re-configuring the collaborative work system’s visualization capabilities as well as the modelling method in direct interaction with the end-users in two case studies. hence, regarding the adr research cycle, our case studies can be positioned in a ’beta’ cycle of the ‘building, intervention and evaluation’ stage of adr in its organization-dominant form that includes operational system use (sein, henfridsson, purao, rossi, & lindgren, 2011). 3.2 case study design we have carried out two adr case studies with two different organizations (hevner, march, park, & ram, 2004; sein, henfridsson, purao, rossi, & lindgren, 2011; yin, 2014). in preparing our research, we identified two companies that had not previously engaged with business ecosystem modelling but that demonstrated interest in a project to analyse their ecosystem. one organization is an automotive company, headquartered in europe with approximately 120,000 employees. the other is a publishing company, headquartered in europe with approximately 16,000 employees. both organizations had a high interest in modelling and visualizing business ecosystems of their specific focus, one related to smart cities and the other to the service and person landscape within their business area. for both, the ecosystem of focus was not yet modelled or visualized. we conducted several workshops with both organizations between december 2017 and august 2018. in advance, we discussed the study schedule and discussed our earlier work. during the project, we scheduled workshops and meetings on an ad-hoc basis. 3.3 data collection and analysis we collected data with help of documentation, by examining archival records, and through direct observation (benbasat, goldstein, & mead, 1987). all involved stakeholders had access to the provided collaborative work system beex, which they used to model their business australasian journal of information systems faber, hernandez-mendez, rehm & matthes 2020, vol 24, selected papers from acis 2018 collaborative modelling & visualization of business ecosystems 6 ecosystem of focus after an introduction in a primarily self-directed manner. we had access to all aspects of the modelling process and related documentation during the entire study period, which contributed to a shared understanding of the addressed business ecosystems. as part of both case studies, we conducted several workshops with a high degree of interaction, which provided us with a fertile ground for direct observations. the workshops focused particularly on executing the collaborative modelling procedure to create ecosystem data and view models (see section 4), which represent the distinct outcomes of the process. after each workshop session, we documented our direct observations and eventually adjusted the models in the collaborative work system. in addition, each workshop was documented in form of a report that was validated by company representatives. in both case studies, final workshops with a larger group of stakeholders (including executive managers who represented ‘gatekeepers’) were conducted in which both, the status quo of the created model, and the resulting visualization were discussed to gain insights on the business ecosystems. 3.4 methodical design artefact: modelling and visualization process for both studies, we followed six principal steps to guide the modelling and visualization process of the business ecosystem, as illustrated in figure 1. figure 1. modelling and visualization process (adapted from basole et al., 2016) in the first step, the business ecosystem focus is defined. examples for a business ecosystem focus are ecosystems established all around a technology platform, an ecosystem of a specific market exploiting specific digital technologies (as e.g., in sako, 2018) or ecosystems that emerge around one focal firm. in the second step, model instantiation, initial models are created for two types of models, data and view model. within the data model instantiation, the relevant entities of the ecosystem are defined, including attributes describing them. in addition, relation types between identified entities are preliminarily set. for the view model, the type of visualization including the specifications for this visualization are established. to ensure tailored visualizations in a later phase of the process, we have recognized that several stakeholders need to be involved into collecting the requirements for both types of models already in this early model instantiation phase. in the next phase, an iterative process consisting of three steps follows: the data collection step gathers data about the ecosystem according to the specified data model; tailored visualizations need to be provided according to the defined view model; and feedback collected from involved stakeholders might lead to adaptions of the models. ideally, this three-step process australasian journal of information systems faber, hernandez-mendez, rehm & matthes 2020, vol 24, selected papers from acis 2018 collaborative modelling & visualization of business ecosystems 7 is conducted iteratively until the collected data and the tailored visualization fulfil the stakeholders’ requirements and needs. in our case studies, the iteration was conducted a limited number of times to the extent that satisfactory levels of informative value and validity of the models were achieved. in the final process step, the created visualizations are used to extract knowledge about the ecosystem in form of a “story told”, which contributes to a better understanding of the ecosystem in focus. in both studies we deployed the collaborative work system beex to instantiate business ecosystem models. that is why for both studies prior to the actual modelling workshops, a dedicated workshop to explain the system was conducted, explaining the concept of data and view model and the system’s capacity to adapt in run-time. within these workshops, the aims for both studies were set; firstly, realising an instance of the business ecosystem model and secondly, learning to deploy the system for adapting the ecosystem model by following the described process. preliminary to starting the process, each stakeholder received a dedicated user identity. this allowed us to track model changes and who had carried them out; thus making the adaptations transparent. it also allowed us to identify particular contributions of the different stakeholders involved in addition to the workshops’ observations. we consequently identified roles participants assumed in group model building across the different process steps as well as their influence on the modelling results; we will further detail these roles during our analysis. 3.5 technological design artefact: business ecosystem explorer (beex) 3.5.1 the hybrid wiki approach to collaborative work in order to cope with the requirements originating from the dynamics of business ecosystems, we deploy an agile framework for modelling ecosystems. this framework rests on the hybrid wiki approach (reschenhofer, bhat, hernandez-mendez, & matthes, 2016) and has been implemented as an integrated, adaptive, collaborative work system that serves as application development platform and contains features for data management as well as collaboration and decision support. this system supports adaptation of both the model and its instances at runtime as carried out by stakeholders and ecosystem experts, i.e., users without programming knowledge or skills. it allows to track all model changes back to the user responsible and the time of change. to create the business ecosystem model within this system, we use the hybrid wiki metamodel (reschenhofer, bhat, hernandez-mendez, & matthes, 2016) as visualized in figure 2. figure 2. hybrid wiki meta-model (adapted from reschenhofer (2017)) australasian journal of information systems faber, hernandez-mendez, rehm & matthes 2020, vol 24, selected papers from acis 2018 collaborative modelling & visualization of business ecosystems 8 the hybrid wiki metamodel contains the following model building blocks: workspace, entity, entitytype, attribute, and attributedefinition. these concepts structure the model inside a workspace and capture its current snapshot in a data-driven, i.e., bottom-up, process. an entity contains a collection of attributes, and the attributes are stored as a key-value pair. the attributes have a name and can store multiple values of different types, for example, strings or references to other entities. the user can create an attribute at run-time to capture structured information about an entity. an entitytype allows users to refer to a collection of similar entities, e.g., organizations, persons. the entitytype consists of multiple attributedefinitions, which in turn contain multiple validators such as multiplicity validator, string value validator, and link value validator. additionally, an attribute and its values can be associated with validators for maintaining integrity constraints. 3.5.2 business ecosystem explorer model the agile framework relies on two types of models, (a) ecosystem data model, and (b) ecosystem view model, each with respective features for instantiation and adaptation. both model types are encoded using the hybrid wiki metamodel. the data model contains the entitytypes of relevance for the business ecosystem in focus. the view model is encoded as one entitytype called visualizations. each visualization has two elements: the first element is the link between the data model and the visualizations. the second element is the specification of the visualization using a declarative language. the used system provides the feature of adapting the models at runtime. in case of changes in the data model, such as adding a category of entities, changing or deleting existing categories, the visualizations are updated at runtime. 3.5.3 business ecosystem explorer views the agile framework currently consists of six views: a landing page, a list of all entities, detail view with company information, a relation view, a visualization overview, and several visualizations. for all views, a menu bar at the top of the web page provides links to the other views available. we used this framework in our case studies to discuss the principal idea of business ecosystem modelling and visualization. at a later stage of the study, stakeholders of both enterprises used the framework to create their own company workspace and self-selected suitable visualizations. 4 case studies: collaborative business ecosystem modelling and visualization 4.1 study 1: innovative mobility services business ecosystem the first study of an automotive company took place over five months, from december 2017 to april 2018. within the entire study time, overall, five representatives were involved; two of these five were active in the modelling workshops; the three others were involved in major decisions. the study comprised of 11 workshops, each lasting between 60 and 120 minutes. all workshops took place on enterprise premises using laptops, a whiteboard, and pen and paper. australasian journal of information systems faber, hernandez-mendez, rehm & matthes 2020, vol 24, selected papers from acis 2018 collaborative modelling & visualization of business ecosystems 9 4.1.1 course of the study the definition of the business ecosystem focus was set prior to the first modelling workshop. purpose of this study was to model and visualize a business ecosystem of innovative mobility services. the firm stakeholders were interested in a better understanding about which cities are providing innovative mobility services within their landscape of mobility offers, and which service might be relevant for a city currently allocating less mobility services overall. also of interest was a better understanding of which service provider offers which mobility services and how well each service provider is interconnected with which city. prior to the study, in order to inform ourselves about the study context, the research team had analysed the business ecosystem of focus in a rudimentary way. here, information about the business ecosystem had been collected with help of stakeholders from several organizations and documented in an unstructured form. this information however had not been processed further; a tracking about who had included which information and who had accessed it in a later stage was not conducted, and no visualization had been created). within the first three modelling sessions (december and january), the initial data model was created. as beex provides the feature of model adaptation, the data model was continuously updated in the following. in the subsequent two months, two company representatives documented the data collected within the company in the system, and enriched it with additional data. also, company external data sources such as newspaper articles, news feeds and free-of-charge online databases were used. the data collection process was conducted manually by enterprise representatives and not supported by the research team. during this phase, four modelling workshops were conducted. within these workshops, the involved researchers answered question about the usage of the system. in addition, inconsistencies of the model were addressed and resolved. in three workshops in february and march, the view model was created, and the initial visualization type defined, a force layout view. this decision was taken after two months of business ecosystem data collection and modelling. in a dedicated workshop, the building blocks (marks, scales, and signals) of each visualization were discussed and aligned. after having prepared this aligned view model, the research team created the business ecosystem visualizations accordingly. the tailored views were presented and discussed in the following workshops end of march. collecting the feedback, the visualizations were adapted and further discussed in two workshops in april. the final workshop took place at the end of april. besides the gatekeeper, two modellers, two researchers and two additional enterprise stakeholders participated. the results of the previous modelling workshops were presented: besides the two force layout views (figure 3), a list view of all entities, a detailed view for each entity and one additional force layout view were presented and discussed. the feedback of the gatekeeper and additional modeller regarding the view model were incorporated immediately. 4.1.2 data and view model the hybrid wiki metamodel was used to set up the data and view model. after the first data model creation workshop, the data model consisted of six entitytypes and eight attributedefinitions. in the following session, the data model grew to finally eight entitytypes and 26 attributedefinitions, a greater increase on the attribute level australasian journal of information systems faber, hernandez-mendez, rehm & matthes 2020, vol 24, selected papers from acis 2018 collaborative modelling & visualization of business ecosystems 10 than on the entity level. as entities of the ecosystem, the modellers created cities, organizations (e.g., services providers such as automotive oems), mobility services and mobility related projects. mobility services for instance comprised car sharing, bike sharing, ride sharing and others. the view model consisted of the three entities describing the three force layouts. the three visualizations as depicted figure 3 are described in the following. figure 3. the three visualizations created in the first study. due to anonymization requirements, the entity names and categories are blurred. australasian journal of information systems faber, hernandez-mendez, rehm & matthes 2020, vol 24, selected papers from acis 2018 collaborative modelling & visualization of business ecosystems 11 4.1.3 created business ecosystem visualizations the first visualization, city2organization, shows the network of cities and organizations and addresses the first two points of interest for the business ecosystem modelling, i.e., which cities are providing innovative mobility services within their mobility service landscape, and which service might be interesting to offer for a city allocating fewer mobility services currently. the visualization was specified to display cities as hexagons and organizations as circles. for the second visualization, project2organization, the network of projects and organizations is shown; the shape of nodes is identical for projects and organizations. in the third force layout, organization2organization, organizations and their relations are displayed. in this visualization, the relations are color-coded as well, as this was a request by one company representative. a corresponding legend was not implemented in this session. for all three force layouts, the organization category is displayed using color-coded nodes as explained in the legends on the right sides of each visualization. all three visualizations provide interactive features. clicking on a specific node directs the user to a view providing detail information about the respective entity. 4.1.4 roles of the collaborative process we validated four roles in this study: facilitator, modeller, recorder and gatekeeper. the gatekeeper was the most prominent participant in the beginning of the study. she decided on the business ecosystem focus and assigned stakeholders to participate in the modelling sessions. within the second process step, one company representative established herself as the facilitator. she paid constant attention to the group process. she also took the lead in organizing all following modelling sessions. the facilitator was also the leading contributor in implementing existing data, enriching it with external data sources and thereby acting as a modeller especially between workshops. during this final workshop, she again took the lead explaining the results and the outcome. the researchers present in the workshops fulfilled both the modeller and recorder role by documenting each workshop in form of a written report. 4.2 study 2: publishing company business ecosystem the second study was conducted in cooperation with a publishing company. the study took place over six months, from february to august 2018. overall, six enterprise representatives participated in the study in seven workshops, each lasting between 90 and 120 minutes. all seven workshops took place on the enterprise premises using laptops, a whiteboard, and pen and paper. 4.2.1 course of the study within the first workshop in february, the focus of the business ecosystem was set: modelling of a publishing business ecosystem, located in germany, with regard to key persons, publishing landscape and the services offered and consumed. besides identifying the ecosystem focus, previous activities and relevant additional stakeholder to involve were discussed. information about the ecosystem in focus had not yet been collected or analysed and the ecosystem not yet modelled. a group of potentially relevant stakeholders to contribute to the modelling process were identified. australasian journal of information systems faber, hernandez-mendez, rehm & matthes 2020, vol 24, selected papers from acis 2018 collaborative modelling & visualization of business ecosystems 12 in the next two workshops, the ecosystem data model was discussed and defined, whereby the second workshop focused on enriching the model through the implementation of attribute definitions. during a workshop in april, the ecosystem view model including the building blocks was initially discussed and determined. the study participants chose two force layout visualizations: one for the service landscape and the other for the key person network. at the beginning of may, the framework including two visualizations was provided by the researchers using the data collected in the collaborative work system. both, company internal data sources but also news articles were used to collect relevant business ecosystem information. the data was entered mostly during the workshop sessions. in the following workshops, in which the prototype was demonstrated each time, the business ecosystem model was iteratively adapted. therefore, five additional company representatives were included in two workshops in may to discuss the utility of the prototype and to provide feedback. two more workshops took place in july for adapting the business ecosystem model further together with the study participants. the final workshop was held in august. besides the core team, three additional company representatives participated who had already participated occasionally earlier. the study results and potential next steps were discussed. 4.2.2 data and view model within the study, both data and view model were created. after the first data model creation workshop, three entitytypes with seven attributedefinitions were defined. at the end of the study, the final data model consisted of nine entitytypes. these entitytypes included 24 attributedefinitions. the data model evolved during the modelling process, especially on the attribute level and during the initial phase of the project. for the data model, ecosystem entities such as key persons, publisher, and publishing groups were identified, including relation types such as the role of the person within the ecosystem and type of service. after the tool was provided, the data model on the entity level stayed the same. the ecosystem view model consisted of the two entities covering the visualizations displayed in figure 4. after the view model was aligned upon, only one specification changed: the relation type berichtswesen (german for ‘reporting system’) was adapted as it is a directed relation. this relation type is not displayed as a straight line connecting two ecosystem entities but through an arrow. both visualizations are force layouts targeting to tell different stories about the company’s business ecosystem. 4.2.3 created business ecosystem visualizations in the first visualization (left hand in figure 4), key individuals of the publishing area in germany are visualized with their professional relations to publishers. the links are colorcoded according to the roles occupied by identified key stakeholders of the ecosystem, which are explained in the legend in the top left corner. in addition, the structure of the relations provides insight if the role is executed currently, has been performed in the past, or will be established in the future. the legend of the structure is located in the top right corner. both legends provide filter options to select a specific role of interest, like ceo, or status of employment, like current. when clicking on any entity, the user is directed to the collaborative work system and the according page in the workspace. australasian journal of information systems faber, hernandez-mendez, rehm & matthes 2020, vol 24, selected papers from acis 2018 collaborative modelling & visualization of business ecosystems 13 figure 4. two visualizations created in the second study. due to anonymization requirements, the entity names and categories are blurred. the second visualization displays publishing-related it services provided within the industry. dark grey nodes display service providers and lighter nodes publishers. the services provided are listed in the legend on the right. the visualization offers the interactive feature of showing the color-coded relations when hovering over any node. the node size of the publishing companies is set accordingly to their last year’s business turnover as explained in the legend in the top right corner, whereas the size of the service providers is fixed. again, both the turnover and the provided service legend provide filter options to select only publishers with a specific turnover or display only chosen services. when clicking on any node, the user is directed to the dedicated page in the system. 4.2.4 roles of the collaborative process similar to the first study, we could validate four roles in this study: facilitator, modeller, recorder and gatekeeper. within this study, the composition of the modelling groups changed within the study period. all participants who actively modelled participated in the pre-study workshop and thus were familiar with the beex system and the study aim. for all workshops, the group consisted of at least three participants. already in the first workshop, one of the australasian journal of information systems faber, hernandez-mendez, rehm & matthes 2020, vol 24, selected papers from acis 2018 collaborative modelling & visualization of business ecosystems 14 company’s representatives positioned herself as the facilitator. she kept this role over the entire study period and participated in all workshops. the facilitator also actively participated in the modelling as such but acted as decision maker and discussion leader. one example is the decision to use a force layout. the gatekeeper only participated twice in workshops in may and thus was identified at a late state of the process. even though she had not been involved in previous workshops, her opinion and feedback to the provided and discussed visualizations were high prioritized by the facilitator. the researcher present in the workshops filled the role of the recorder and documented each workshop in form of a written report. 5 cross-case analysis and discussion even though in both studies different topics and ecosystem models have been instantiated, we can draw some cross-case conclusions. following patton (2001) who defines lessons learned as the knowledge which is derived from the screening of a situation and which can be applied in similar situations in the future, we will present our finding and lessons learned of both studies. 5.1 availability of information about the business ecosystem while the challenge of ecosystem data collection is not new (basole et al., 2015), in both studies, the collection of information about the business ecosystem turned out to be more challenging than expected by the study participants. especially in the first study, this constituted a reason for why the force layout visualization of organizations and their involvement in projects was rather neglected and the focus oriented towards the other two visualizations. a possible measure to improve ecosystem data collection could be using internet data sources as an initial starting point for ecosystem modelling. we found that providing visualizations can point ecosystem stakeholders towards ecosystem information that is currently insufficient or missing, and can guide them towards novel information sources. 5.2 evolution of the data model in both case studies, we noticed that the data model evolved rather on the attributetype level than on the entitytype level. that means, the entities visualized remained nearly untouched whereby for all entities the attributes changed during the modelling process. in our opinion, this is due to the user being accustomed to the provided visualization and is thus less willing to change it. a change of the attribute, which might also include adding or deleting an attribute, does not involve changes of the force layout view. furthermore, as the attributes can be defined as mandatory or not, the user is free to add attributes for specific entities that are not available for others. 5.3 expertise of modelling participants comparing both studies, we noticed that the knowledge workshop participants bring into the project, has a great influence on the achieved results. the modellers involved in the first study were less experienced with modelling activities as such. one consequence was that more time was spent on the data and view model creation and thus resulted in receiving the implemented interactive visualizations at a later point within the process. to address this, we recommend an additional workshop prior to the study period focussing on how to create a model. our pre-study workshop solely focused on explaining the existing framework and the data and view model in use, but did not exercise the active participation of study participants. australasian journal of information systems faber, hernandez-mendez, rehm & matthes 2020, vol 24, selected papers from acis 2018 collaborative modelling & visualization of business ecosystems 15 5.4 maintaining motivation during the modelling process especially in the first study, we noticed a decrease in motivation during the data and view model creation phase. as described above, the study participants—including the facilitator— were unfamiliar with modelling activities. thus, reaching the fourth process step took longer than expected by the company representatives. this holds especially true, as the predefined outcome of the study was an instance of the beex system including tailored interactive visualizations. 5.5 importance to provide visualizations the aforementioned findings lead us to conclude that the visualizations play a crucial role in the perceived success of the modelling process. we noticed within both studies that as soon as the interactive visualizations were provided with the additional feature of adapting changes to the data model and the data collected in run-time, contributed heavily to motivate the participants to collect and implement data. 5.6 story telling as for both studies the focus was on the instantiation of the ecosystem model, the provided visualizations can be considered as early visualizations of the ecosystems. within the studies, this process step was rather used to present the results of the study to the gatekeeper. 5.7 collaborative modelling in both studies, several stakeholders were included in the modelling process. during the workshops, all participants contributed as modellers. between the workshops, the overall contribution decreased, and only key team members—those we identified as facilitators— continued collecting and implementing data. changes of the model were only implemented during the workshops, which might be due to the still rather unfamiliar framework in use. we recognized a lively discussion during the workshops, implying the relevance of managing the business ecosystem model in focus. overall, we identified four distinct roles within both studies: the facilitator, the modeller, the recorder and the gatekeeper. as we conducted this process of instantiating a business ecosystem model first time, we assert that a process coach was missing. 6 conclusion we have conducted two action design research case studies focused on collaborative approaches for modelling and visualizing business ecosystems. we report on the insights gained with regard to challenges that occur in the process, and discuss how model-driven approaches help addressing these challenges. the generic modelling and visualization process we have instantiated (figure 1) and the deployed collaborative work system, beex, support several stakeholders and roles to collaboratively contribute to the instantiation of a business ecosystem model. the process consists of six steps, namely, defining the business ecosystem in focus, instantiating both data and view model, collecting relevant data, providing tailored visualizations, obtaining feedback, which is fed into the iteration process of the two preceding steps, and finally, storytelling. we report in detail about the progress of our studies, the involvement of participants and conclude with findings and lessons learned. australasian journal of information systems faber, hernandez-mendez, rehm & matthes 2020, vol 24, selected papers from acis 2018 collaborative modelling & visualization of business ecosystems 16 a noticeable limitation of the presented work is the manual implementation of data, which is a time-consuming and tedious work. as visualizations are data-driven, business ecosystem visualizations in general heavily rely on the availability and quality of data. complementing the manual data collection with (semi-) automated data extraction processes, such as provided by natural language processing, could contribute to the enhanced availability of data and thus to richer visualizations, leading to better options for extracting knowledge. regarding data quality, approaches towards data governance in the business ecosystem context in general as well as in our studies are missing; we envision the facilitator being a good starting point for addressing such a task. in addition, with respect to establish ecosystem modelling as a continuous task in an enterprise’s strategy processes, we envision the iterative process steps to be conducted repeatedly, which would lead to data enrichment and potential inclusion of additional stakeholders within the company. as a major challenge for ecosystem modelling, we recognize the right balance between earlyon provision of tailored visualizations and following the process procedure. as our focus was on establishing a shared language within the modelling process, we provided visualizations only after aligning data and view model, which lead to a motivational decrease during the study. nevertheless, we offer the presented process as a reference to organizations interested in modelling their business ecosystem. we hope our insights inform fellow researchers about the challenges associated with collaborative approaches for modelling and visualizing business ecosystems. acknowledgements the research described in this article rests on results of the project grant beex+ 01is17049 sponsored by the german federal ministry of education and research (bmbf). references adner, r. (2016). ecosystem as structure. journal of management, 43(1), 39–58. https://doi.org/10.1177/0149206316678451 adner, r., & kapoor, r. (2016). innovation ecosystems and the pace of substitution: re‐ examining technology s‐curves. strategic management journal, 37(4), 625–648. https://doi.org/10.1002/smj.2363 avison, d., lau, f., myers, m., & nielsen, p. a. (1999). action research. communications of the acm, 42(1), 94–97 baskerville, r., & myers, m. d. (2004). special issue on action research in information systems: making is research relevant to practice: foreword. mis quarterly, 28(3), 329. https://doi.org/10.2307/25148642 basole, r. c. (2009a). structural analysis and visualization of ecosystems : a study of mobile device platforms. proceedings of the fifteenth americas conference on information systems, san francisco, california, august 6th-9th 2009 1, 1–10 basole, r. c. (2009b). visualization of interfirm relations in a converging mobile ecosystem. journal of information technology, 24(2), 144–159. http://doi.org/10.1057/jit.2008.34 basole, r. c., huhtamäki, j., still, k., & russell, m. g. (2016). visual decision support for business ecosystem analysis. expert systems with applications, 65, 271–282. http://doi.org/10.1016/j.eswa.2016.08.041 australasian journal of information systems faber, hernandez-mendez, rehm & matthes 2020, vol 24, selected papers from acis 2018 collaborative modelling & visualization of business ecosystems 17 basole, r. c., russell, m. g., huhtamäki, j., rubens, n., still, k., & park, h. (2015). understanding business ecosystem dynamics: a data-driven approach. acm transactions on management information systems, 6(2), 1–32. http://doi.org/10.1145/2724730 benbasat, i., goldstein, d. k., & mead, m. (1987). the case research strategy in studies of information systems case research. mis quarterly, 11(3), 369–386. http://doi.org/10.2307/248684 bosch, j. (2016). speed, data, and ecosystems: the future of software engineering. ieee software, 33(1), 82–88. http://doi.org/10.1109/ms.2016.14 dollmann, t., houy, c., fettke, p., & loos, p. (2011). collaborative business process modeling with comomod: a toolkit for model integration in distributed cooperation environments. in proceedings of the 2011 20th ieee international workshops on enabling technologies: infrastructure for collaborative enterprises, wetice 2011 (pp. 217–222). ieee. http://doi.org/10.1109/wetice.2011.36 evans, p. c., & basole, r. c. (2016). revealing the api ecosystem and enterprise strategy via visual analytics. communications of the acm, 59(2), 26–28. http://doi.org/10.1145/2856447 faber, a., hernandez-mendez, a., rehm, s.-v., & matthes, f. (2018a). an agile framework for modeling smart city business ecosystems. in proceedings of the 20th international conference on enterprise information systems (vol. 2, pp. 39–50). scitepress science and technology publications. http://doi.org/10.5220/0006696400390050 faber, a.; hernandez-mendez, a.; rehm, s.-v. & matthes, f. (2018b). visualizing business ecosystems: applying a collaborative modelling process in two case studies. proceedings of the 29th australasian conference on information systems, track: digital business innovation, sydney, australia, 3-5 december 2018 faber, a., rehm, s.-v., hernandez-mendez, a., & matthes, f. (2018). modeling and visualizing smart city mobility business ecosystems: insights from a case study. information, 9(11), 270. https://doi.org/10.3390/info9110270 guittard, c., schenk, e., & burger-helmchen, t. (2015). crowdsourcing and the evolution of a business ecosystem. in f. j. garrigos-simon, i. gil-pechuàn, & s. estelles-miguel (eds.), advances in crowdsourcing (pp. 49--62). springer international publishing. http://doi.org/10.1007/978-3-319-18341-1_4 hao, j., zhu, j., & zhong, r. (2015). the rise of big data on urban studies and planning practices in china: review and open research issues. journal of urban management, 4(2), 92–124. http://doi.org/10.1016/j.jum.2015.11.002 hevner, a. r., march, s. t., park, j., & ram, s. (2004). design science in information systems research. mis quarterly, 28(1), 75–105. retrieved from http://www.jstor.org/stable/25148625 huber, t. l., kude, t., & dibbern, j. (2017). governance practices in platform ecosystems: navigating tensions between cocreated value and governance costs. information systems research, 28(3), 563–584. https://doi.org/10.1287/isre.2017.0701 australasian journal of information systems faber, hernandez-mendez, rehm & matthes 2020, vol 24, selected papers from acis 2018 collaborative modelling & visualization of business ecosystems 18 huhtamaki, j., & rubens, n. (2016). exploring innovation ecosystems as networks: four european cases. proceedings of the annual hawaii international conference on system sciences, 2016–march, 4505–4514. http://doi.org/10.1109/hicss.2016.560 iansiti, m., & levien, r. (2004). strategy as ecology. harvard business review, 82(3). http://doi.org/10.1108/eb025570 iyer, b. r., & basole, r. c. (2016). visualization to understand ecosystems. communications of the acm, 59(11), 27–30. http://doi.org/10.1145/3000610 kapoor, r., & agarwal, s. (2017). sustaining superior performance in business ecosystems: evidence from application software developers in the ios and android smartphone ecosystems. organization science, 28(3), 531–551. https://doi.org/10.1287/orsc.2017.1122 kim, h., lee, j.-n., & han, j. (2010). the role of it in business ecosystems. communications of the acm, 53(5), 151–156 liu, f., & zhang, c. (2010). role-based collaborative model of group decision support system. in 2010 seventh international conference on fuzzy systems and knowledge discovery (pp.1039–1043). ieee. http://doi.org/10.1109/fskd.2010.5569581 moore, j. f. (1993). predators and prey: a new ecology of competition. harvard business review, 71(3), 75–86 moore, j. f. (1996). the death of competition: leadership and strategy in the age of business ecosystems. new york: harperbusiness park, h., & basole, r. c. (2016). bicentric diagrams: design and applications of a graph-based relational set visualization technique. decision support systems, 84, 64–77. http://doi.org/10.1016/j.dss.2016.02.001 park, h., bellamy, m. a., & basole, r. c. (2016). visual analytics for supply network management: system design and evaluation. decision support systems, 91, 89–102. http://doi.org/10.1016/j.dss.2016.08.003 parker, g., van alstyne, m., & jiang, x. (2017). platform ecosystems: how developers invert the firm. mis quarterly, 41(1), 255-a4 patton, m. q. (2001). evaluation, knowledge management, best practices, and high quality lessons learned. the american journal of evaluation, 22(3), 329–336. http://doi.org/http://dx.doi.org/10.1016/s1098-2140(01)00147-3 peltoniemi, m., & vuori, e. (2004). business ecosystem as the new approach to complex adaptive business environments. proceedings of ebusiness research forum, 267–281 porter, m. (1979). how competitive forces shape strategy. harvard business review, (march), 102–117 rehm, s.-v., faber, a., & goel, l. (2017). visualizing platform hubs of smart city mobility business ecosystems. thirty eighth international conference on information systems, 1-10. https://aisel.aisnet.org/icis2017/digitalplatforms/presentations/14/ renger, m., kolfschoten, g. l., & de vreede, g. j. (2008). challenges in collaborative modeling: a literature review. in lecture notes in business information processing (10th ed., vol. 10 lnbip, pp. 61–77). http://doi.org/10.1007/978-3-540-68644-6_5 australasian journal of information systems faber, hernandez-mendez, rehm & matthes 2020, vol 24, selected papers from acis 2018 collaborative modelling & visualization of business ecosystems 19 reschenhofer, t., bhat, m., hernandez-mendez, a., & matthes, f. (2016). lessons learned in aligning data and model evolution in collaborative information systems. in proceedings of the 38th international conference on software engineering companion icse ’16 (pp. 132– 141). new york, new york, usa: acm press. http://doi.org/10.1145/2889160.2889235 reschenhofer, t. (2017). empowering end-users to collaboratively analyze evolving complex linked data. doctoral dissertation, technical university of munich, munich, germany. richardson, g. p., & andersen, d. f. (1995). teamwork in group model building. system dynamics review, 11(2), 113–137. http://doi.org/10.1002/sdr.4260110203 roth, s., hauder, m., & matthes, f. (2013). collaborative evolution of enterprise architecture models. ceur workshop proceedings, 1079, 1–12 sako, m. (2018). business ecosystems: how do they matter for innovation? communications of the acm, 61(4), 20–22. http://doi.org/10.1145/3185780 sein, m. k., henfridsson, o., purao, s., rossi, m., & lindgren, r. (2011). action design research. mis quarterly, 35(1), 37–56 sherer, s. a. (2014). advocating for action design research on it value creation in healthcare. journal of the association for information systems, 15(12), 860–878 uchihira, n., ishimatsu, h., & inoue, k. (2016). iot service business ecosystem design in a global, competitive, and collaborative environment. in 2016 portland international conference on management of engineering and technology (picmet) (pp. 1195–1201). ieee. http://doi.org/10.1109/picmet.2016.7806694 vartak, m., huang, s., siddiqui, t., madden, s., & parameswaran, a. (2017). towards visualization recommendation systems. acm sigmod record, 45(4), 34–39. http://doi.org/10.1145/3092931.3092937 visnjic, i., neely, a., cennamo, c., & visnjic, n. (2016). governing the city. california management review, 59(1), 109–140. http://doi.org/10.1177/0008125616683955 yin, r. k. (2014). case study research: design and methods. hematology (5th ed., vol. 86). sage publication copyright: © 2020 faber, hernandez-mendez, rehm & matthes. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v24i0.2229 http://doi.org/10.1145/2889160.2889235 http://creativecommons.org/licenses/by-nc/3.0/au/ http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems faber, hernandez-mendez, rehm & matthes 2020, vol 24, selected papers from acis 2018 collaborative modelling & visualization of business ecosystems 20 paper title (paper title style) australasian journal of information systems thorpe et al. 2021, vol 25, research on indigenous use of designing archival information systems information and communication technologies 1 designing archival information systems through partnerships with indigenous communities: developing the mukurtu hubs and spokes model in australia kirsten thorpe university of technology sydney, australia kirsten.thorpe@uts.edu.au kimberly christen washington state university, usa kachristen@wsu.edu lauren booker university of technology sydney, australia lauren.booker@uts.edu.au monica galassi university of technology sydney, australia monica.galassi@uts.edu.au abstract indigenous peoples in australia have been heavily documented in colonial archives and collections. the past two decades have seen significant materials from galleries, libraries, archives and museums (glam) institutions being returned to indigenous communities in australia through physical or digital repatriation of materials. the digital return of materials requires both appropriate systems for returning both the digital collections, metadata and contextual information that relates to them, and agreements, policies, and procedures for meaningfully engaging with indigenous communities throughout the process. importantly, the information returned needs to be accessible, readable, and usable in local community contexts based on understanding local community needs. this paper discusses priorities around engaging with indigenous peoples to reshape and build archival information systems and access points that support community requirements for digital return and management of cultural heritage materials in local settings. the paper discusses future priorities for designing archival information systems to support indigenous sovereignty, including data stewardship and preservation approaches. these concerns are discussed and raised as part of the research and development of the global mukurtu content management system (cms) project, including within the new south wales (nsw) australian mukurtu hub. keywords: archival information systems, indigenous sovereignty, indigenous digital return, indigenous archives 1 introduction indigenous peoples in australian have been heavily documented in colonial archives and collections. the past two decades have seen significant materials from galleries, libraries, archives and museums (glam) institutions being returned to indigenous communities in australia through physical repatriation or digital return of materials. the digital return of materials requires both appropriate systems for returning the digital collections, its metadata and contextual information, and strategies to develop agreements, policies, and procedures australasian journal of information systems thorpe et al. 2021, vol 25, research on indigenous use of designing archival information systems information and communication technologies 2 for meaningfully and respectfully engaging with indigenous communities throughout the process. importantly, the information returned needs to be accessible, readable, and usable in local community contexts based on understanding local community needs. a number of major australian and international glam institutions work collaboratively with indigenous communities and local organisations to expand conversations about digital return and engagement. often, from an institutional perspective dialogue is spearheaded by the development of exhibitions or other projects that aiming to open up access to collections. however, communities may have very different sets of priorities and timelines for collections related to them. as a result, when the information systems and projects that are designed to digitally return collection materials are formed without community involvement, they often do not meet community requirements for proper access, use, and circulation of cultural heritage materials. this paper discusses priorities around engaging with indigenous peoples to reshape and build information systems and access points that support community requirements for digital return and management of cultural heritage materials in local settings. finally, the paper discusses future priorities for designing archival information systems to support indigenous sovereignty, including approaches for data stewardship and preservation. these concerns are discussed and raised as part of the research and development of the global mukurtu cms project, including within the new south wales (nsw) australian mukurtu hub. 2 background the paper weaves together research and practice from the contributing authors around the theme of indigenous peoples1 sovereignty and its implications for archival information systems2. the paper identifies mandates for action to support indigenous priorities with designing archival information systems for the curation, dissemination, and management of indigenous archives across government, institutional, and community contexts. this article extends a panel presented on ‘collaborative design through partnerships with indigenous communities: the mukurtu cms hubs and spokes model’ with papers delivered by the authors at the international council on archives conference (ica) held in adelaide, australia in october 2019. the broader conference theme was related to the need for archives to design human-centered approaches to ensure benefits to diverse communities (australian society of archivists, 2019). the conference closed with the first indigenous summit of the newly formed ica expert group on indigenous matters which aimed to identify key issues indigenous peoples face in relation to archives and examine a proactive international agenda to re-design archives to support decolonisation led by indigenous peoples. the summit also included the launch of the adelaide-tandaya declaration, a statement that calls on the progressive action of archives internationally to support indigenous people’s archive and 1 this paper uses the term indigenous throughout this paper to refer to first nations people internationally. we use indigenous peoples in australia when we discuss specific references to aboriginal and torres strait islander peoples in australia. we acknowledge the diversity of indigenous peoples internationally and acknowledge the variances of terminology across regions. we also use capital for ‘i’ for indigenous as a sign of respect. 2 we use the term ‘archival information systems’ to refer to both the description and classification of archival materials, and the management of born digital or digitised cultural heritage materials and their associated metadata. australasian journal of information systems thorpe et al. 2021, vol 25, research on indigenous use of designing archival information systems information and communication technologies 3 information needs (ica, 2019). this article brings together the four presentations at the ica conference to refocus a discussion about how to better support the design of archival information systems with respect for indigenous self-determination and sovereignty through partnerships with indigenous communities, institutions and diverse groups of stakeholders. each contributing author discusses themes related to transforming archival information systems, policies and approaches to support indigenous people’s goals, particularly systems that support indigenous cultural safety, governance, participation and self-determination. these themes are discussed against the backdrop of the mukurtu cms platform and project, which operates internationally to support indigenous communities as they manage their cultural heritage materials through informed protocols.3 our aim is to highlight how a number of indigenous peoples and communities are working together with glam institutions to create significant change in collections repatriation and return to improve access and management of digital cultural heritage materials. the research connected with the mukurtu cms project and the nsw australian hub seeks to transform engagement and relationships between indigenous peoples and the archives in support of indigenous sovereignty in australia. 3 introducing the authors we begin our article by introducing our standpoints to give background to our work, experiences and motivations as contributing authors. we also provide some context to our relationships in and out of institutional and research contexts in australia and the united states. kimberly christen i am a professor in and the director of the digital technology and culture program in the college of arts and sciences and the director of the center for digital scholarship and curation at washington state university. i am founder of mukurtu cms, a free and open source community digital access platform designed to meet the needs of indigenous communities. i am also the director of the sustainable heritage network, and the co-director of the local contexts initiative. these projects are aimed at providing educational resources and practical applications for stewarding digital cultural heritage and supporting indigenous communities in the management of intellectual property. as part of this paper, i provide background on the development of the mukurtu cms and discuss the principles of engagement, repatriation, and collaboration that guide our ongoing work focused on grassroots priorities identified by indigenous peoples. lauren booker (garigal clan, north-west sydney) i am a research fellow at the jumbunna institute for indigenous education and research at the university of technology sydney (uts). my research involves facilitating projects with aboriginal communities for the digitisation of recorded and documented cultural material and 3 mukurtu cms is a free and open-source digital access platform designed with and by indigenous communities globally. communities use the platform to provide culturally relevant access to a broad range of collections materials. see: www.mukurtu.org, and christen (2019), "the songline is alive in mukurtu": return, reuse, and respect https://scholarspace.manoa.hawaii.edu/handle/10125/24882 http://www.mukurtu.org/ https://scholarspace.manoa.hawaii.edu/handle/10125/24882 australasian journal of information systems thorpe et al. 2021, vol 25, research on indigenous use of designing archival information systems information and communication technologies 4 the organisation of appropriate digital archives. i advocate for post custodial archival practice and indigenous self-determination in the glam sector, particularly in relation to indigenous data sovereignty, the ‘right to know’ and ‘right of reply’ in records. my current doctoral research focuses on institutional ethics and the transparency of collection management and administrative processes. as part of this paper, i will discuss the significance of the ‘right to know’ (deloria, 1978), introduce my doctoral research on community-led collection care for specific ancestral remains (hair samples) that are still held in collecting institutions. kirsten thorpe (worimi, port stephens nsw) i am a researcher and professional archivist who has been involved in the library and archive sector over the past two decades leading projects around developing protocols, policies, and services for indigenous peoples in australia (thorpe, 2013). my research interests relate to indigenous self-determination in libraries and archives, with a particular focus on the return of historical collections to indigenous peoples and communities (thorpe, 2017). i advocate for a transformation of library and archival practice to centre indigenous priorities and voices in regard to the management of data, records, and collections. as part of my contribution to this paper, i discuss key concepts of cultural safety related to libraries and archives in australia. monica galassi i am an italian researcher who has worked in australia since 2010 on research interests relating to promoting human rights and equity of access, particularly through digital archives. my doctoral studies focus on how historical records held in italy and in the vatican can support self-determination and sovereignty of aboriginal peoples and communities and deepen the understanding of transnational colonial histories. as part of this paper, i will reflect on lessons i have learned working with communities in institutional settings. i will discuss how i approached this engagement with the aim to shift practice to facilitate cross-cultural and crossinstitutional partnerships. i will discuss how i drew on these principles when negotiating practices of digital connection and digital return of indigenous collections. 4 indigenous self-determination, sovereignty and archives in australia as a direct result of the advocacy for self-determination driven by aboriginal and torres strait islander peoples, discussions concerning sovereignty are now more visible in australian public discourse (behrendt, 2002; moreton-robinson, 2007). however, the topic has not been discussed adequately in the australian library and archival sector or within the literature in the field, despite long-term advocacy by indigenous peoples in australia to demand greater control over the management of their cultural collections held in libraries and archives (fourmile, 1989; ormond-parker & sloggett, 2012). as a consequence, mainstream library and archival approaches, including the development of information systems, are limited in their ability to support indigenous peoples’ self-determination and sovereignty. the mid-nineties saw a significant shift in glam institutions responding to indigenous peoples demands for greater access to cultural collections. in the australian library and archive sector, this period also saw the development of protocols that sought to provide clear processes and guidelines for managing culturally sensitive or offensive collections (garwoodhoung & blackburn, 2014; nakata et al., 2005). the protocols for libraries, archives and information services published in 1995 provide sector-wide guidance for working with australasian journal of information systems thorpe et al. 2021, vol 25, research on indigenous use of designing archival information systems information and communication technologies 5 indigenous peoples to redress issues relating to the failures to manage information in ways that protect and promote a diversity of cultural needs (atsilirn, n.d). although the protocols do not specifically mention self-determination and sovereignty, they offer opportunities to reimagine different sets of requirements and relationships that may be needed to access and manage indigenous cultural heritage collections held in glam institutions. however, these needs have not been addressed at a systemic level. often in the australian context it is only in discrete projects, for example, through exhibition or curatorial engagement, that indigenous people have a direct impact on how indigenous voices are represented in collections. while government reports and enquiries such as the royal commission into aboriginal deaths in custody (royal commission into aboriginal deaths in custody, 1991) and the bringing them home report, from the national inquiry into the separation of aboriginal and torres strait islander children from their families (australian human rights commission, 1997), provided opportunities for archival access and exposure of indigenous stories, they failed to disrupt the archival information systems that supported the access and management of these significant records. within this paper’s context, we draw on indigenous led conceptualisations of sovereignty discussed across social, political and academic domains. former aboriginal and torres strait islander social justice commissioner william jonas has described the many ways that aboriginal and torres strait islander peoples continue to assert sovereignty through, for example, ongoing assertions of indigenous cultural protocols and systems of law and governance (jonas, 2002). jonas stresses the importance of indigenous sovereignty being determined by indigenous peoples, noting that sovereignty co-exists, rather than is determined in relation to state sovereignty. in keeping with a bottom-up approach described by jonas (2002), moreton-robinson (2007) dismisses the need to describe a “quintessential definition of indigenous sovereignty” to instead consider sovereignty as being ‘embodied”, “ontological (or being)” and “epistemological (our way of knowing)” (moreton-robinson, 2007, p.2). for the purposes of this paper, we draw on a wide view of indigenous sovereignty and acknowledge the many manifestations of sovereignty both formal and informal that exist in diverse settings internationally. in the context of archival information systems and indigenous peoples in australia, we must acknowledge the ongoing impact of colonisation in relation to the support for indigenous selfdetermination and sovereignty. libraries and archives, by their very nature, are colonial constructs, their systems of collecting sought to define and categorise knowledge, peoples and culture based on eurocentric frameworks and theories (cunningham, 2005, p.32). along with museums and galleries, these collecting practices also supported a system of ‘othering’ and subjugating indigenous peoples knowledges held in their institutional collections. artist jonathan jones (wiradjuri/gamilaroi) has noted that “many of these ‘hobbyist researchers and adventurers’ in australia amassed impressive collections, often trading objects on the evergrowing market as exotic souvenirs and scientific specimens of a dying race or exhibiting them in international exhibitions” (jones, 2018, p. 17). similarly, australian colonial archives were developed through patriarchal collecting processes informed by a view of ‘the other’ so that indigenous peoples and communities have been documented primarily as subjects of the record through a western gaze. we consider these collecting efforts to be pillars of the australian colonial project described by indigenous scholar irene watson as: australasian journal of information systems thorpe et al. 2021, vol 25, research on indigenous use of designing archival information systems information and communication technologies 6 the colonial project has embodied a centuries-long, ongoing campaign to annihilate, define, subordinate and exclude the ‘native’, and an arsenal of tools has been applied to these ends. (watson, 2016, p.30) many australian library and archival collections tacitly or overtly deny the diversity of indigenous nations and communities across the continent and surrounding seas, and instead describe people through a homogenous and objectifying “pan-aboriginal” lenses (russell, 2001). as nathan ‘mudyi’ sentance (wiradjuri) has argued, many collections silence and decentre first nations peoples by privileging narratives related to the collector rather than the people who are documented. sentance (2017) describes how as part of his work in libraries, he had a role in describing collections of missionary papers to discuss their usefulness for aboriginal family history research. he shows how invariably, in opening up these discussions, he spoke more about the missionaries themselves, which was hard to avoid as the collections were named after them (sentence, 2017, n.p). in this example, the role of historical collections in the erasure and silencing of indigenous peoples is ongoing and ever present. indeed, as paper co-author christen further explains, “archives were established as places where official records became anchors for nations in the making as they documented the accepted demise of their first peoples. as a result, the archival imagination is both a process of political work and ideological maneuvering.” (christen 2015, p.115) this legacy of collecting and maintaining collections developed by legislation, racist ideology and the colonial archival imaginary profoundly impacts how indigenous peoples can access and use the archive. archival information systems support the structures and process surrounding the description and classification of archival collections, and the processes associated with access and distribution of digital or digitised content. both professional practice and communities’ feedback have largely demonstrated that many of these standards and approaches do not accommodate indigenous ontologies and epistemologies. as such, these systems, unless challenged and transformed, deny a view of indigenous sovereignty in archives. it is therefore vital that we acknowledge the ongoing impacts of colonisation in opening up archives for the digital return of collections back to their communities. we cannot transform these spaces in line with indigenous sovereignty without a fundamental rethinking of these colonial practices. as indigenous author and scholar tony birch (2007) asserts: before sovereignty can become even a viable concept for discussion amongst the wider nonindigenous community, it needs to redeem and take responsibility for its colonial debts. (birch, 2007, p.106) in summary, we argue that library and archival institutions, and by extension librarians and archivists, play a critical role in the movement to acknowledge these colonial debts. a first step is to acknowledge that the colonial archive in australia holds materials that are culturally unsafe4 for indigenous people. these collections document the trauma and conflicts of australian colonial histories, including evidence of settler relationships and warfare, the forced dispossession of aboriginal people from their lands onto settlements, reserves and missions, and other atrocities such as colonial massacres and the forced removal of children as 4 as further described by thorpe in this paper, we draw on the term cultural safety as defined by williams (1999) as “an environment which is safe for people; there is no assault, challenge or denial of their identity,” conversely a culturally unsafe environment is one where people feel their cultural values and beliefs are silenced and not respected. australasian journal of information systems thorpe et al. 2021, vol 25, research on indigenous use of designing archival information systems information and communication technologies 7 part of the stolen generations (evans et al., 2019, p.131). without appropriate intervention, these archives become tools to whitewash australian history and settler histories further. enabling a transformation of archival and information systems that enable digital return and enhancing of contextual information is one important way of supporting indigenous selfdetermination and sovereignty in the archives. 5 the challenge of digital return of archives to indigenous communities internationally there is growing recognition that the policies, practices and technologies utilised in the information and archive sector perpetuate dominant power structures (brilmyer & caswell, 2017; duff et al., 2013). within this context, critical archival studies seek to address areas where social justice interests intersect with the sector. for example, punzalan & caswell (2016) describe four areas requiring attention, including “inclusion of underrepresented and marginalized sectors of society; reinterpretation and expansion of archival concepts; development of community archives; rethinking archival education and training; and efforts to document human rights violations.” (p.3). similarly, evans et al. (2017, p. 9-10) describe the need for critical approaches to archives and recordkeeping through a “critical archival methodology [which] is concerned with ideas about decolonizing and pluralizing the archive, and is influenced by postmodernism, postcolonialism, feminism, gender studies, sexuality studies, cultural studies, indigenous studies and ways of knowing, and the “archival turn” — the positioning of the archive beyond its traditional administrative and academic constituency.” a critical view of archiving is key to aspire to structural and long-lasting changes, as it supports the transformation of the hegemonic structures and systems supporting those communities who harness less power in society. in a human rights and indigenous context, international frameworks such as the united nations declaration on the rights of indigenous people (2007) and the universal declaration of human rights (1948) should inform archival and recordkeeping practices (gooda, 2012). by engaging with international human rights mandates, the archive sector can work with accountability and respect of international standards such as the united nations declaration on the rights of indigenous people (undrip): article 43 the rights recognized herein constitute the minimum standards for the survival, dignity and well-being of the indigenous peoples of the world. undrip (2007) undrip was ratified by australia in 2009, after initially voting against in 2007, and has gone on to be a cited document for sector specific and nationally recognised roadmaps, declarations and policies across the australian glam sector, i.e.; first peoples: a roadmap for enhancing indigenous engagement in museums and galleries 2018, tandanya adelaide declaration 2019, aiatsis code of ethics for aboriginal and torres strait islander research 2020. as undrip is not legally binding, to have its meaning and intention fully realised it requires active assertion and implementation. the right to repatriation, restitution and return of ancestral remains, ceremonial items and all indigenous cultural and intellectual property collected and held away from communities is addressed in undrip article 11 and 12. both articles refer to the australasian journal of information systems thorpe et al. 2021, vol 25, research on indigenous use of designing archival information systems information and communication technologies 8 imperative for this to be done via ‘effective mechanisms developed in conjunction with indigenous people concerned’ (undrip, 2007). the potential for action and the delivery on these undrip articles rests on several overlapping areas of progressive action across professional, educational and institutional contexts. critical archival studies provide opportunities for a fundamental reassessment of how the sector positions and puts into practice goals of indigenous self-determination and sovereignty. one key area of focus uniting these principles is the development of archive and information systems that are designed to support the digital return of archives to indigenous peoples and communities. however, this work cannot be done in isolation, and it must be led at a grass-roots level by indigenous peoples to ensure that indigenous ways of knowing, being and doing are embedded in the process, recognising indigenous worldviews, knowledges and realities. therefore, the library and archive sector cannot lead it, but it can play a key role in supporting those needs and aspirations. one of the major challenges in relation to digital return of collections to communities is the fixity of institutional power and the prevalence of systems that are not fit for managing indigenous knowledges. there is also a general lack of awareness of important concepts such as indigenous self-determination or awareness of community governance structures for institutions to engage with that give recognition to principles where indigenous peoples have a foundation of control of their future destiny – whether as an individual or as a community (behrendt and vivian, 2010, p.2). the calls to action for decolonising research and the academy that have been expressed internationally by leading indigenous scholars such as smith (2013) and tuck & yang (2012) are not yet realised in the archival, library and information sectors. this transformation requires that library and archival institutions relinquish power over collections policies and structures and implement a range of practices provided by indigenous peoples and communities. as smith (2013) asserts, “decolonization, once viewed as the formal process of handing over the instruments of government, is now recognized as a long-term process involving the bureaucratic, cultural, linguistic and psychological divesting of colonial power.” (smith, 2013, p. 101). while scholarship broadly relating to digital return of archives has had a focus on language documentation and ethnomusicology (toner, 2004; barwick et al., 2020) and on the deepening of information related to cultural revitalisation and creative practice (thorpe & galassi, 2014; grieves and kelada, 2017), there has been limited research and dialogue on questions of selfdetermination and sovereignty with archives and information systems. this paper contributes to filling that gap while also continuing reflection on the failure of the design of archive and information systems that incorporate requirements for indigenous self-determination and sovereignty. this paper also contributes to the dialogue of the indigenous data sovereignty movement, albeit with a focus on reclaiming indigenous data rights (walter & suina, 2019, p.237) in the context of archival information systems. 6 transforming archival and information systems in collaboration with indigenous peoples and communities: the mukurtu project in australia in the next section of the paper, we will introduce background on the mukurtu cms project, based at washington state university (wsu) in the us, and discuss the development of the australasian journal of information systems thorpe et al. 2021, vol 25, research on indigenous use of designing archival information systems information and communication technologies 9 nsw australian mukurtu hub, a collaboration between wsu, the uts and the state library of nsw (slnsw). within this section, each author shares reflections from their papers presented at the ica congress, including outlining principles and mandates for action that have inspired research and collaboration with communities to design information systems that recognise indigenous self-determination and sovereignty. 6.1 ‘mukurtu—a safe keeping place’ (kimberly christen) mukurtu (mook-oo-too) is a grassroots project aiming to empower communities to manage, share, narrate and exchange their digital heritage in culturally relevant and ethically-minded ways (mukurtu (a), n.d). based at the center for digital scholarship and curation (cdsc) at wsu in the usa, the project leads the development of the mukurtu cms, and more recently, support for a hubs and spokes model, which aims to support the continued development and use of the software (christen, merrill and wynne, 2017). the main priority of the mukurtu project is to build a platform that fosters relationships of respect and trust in relation to indigenous digital cultural heritage. the mukurtu project began in australia in 2002 as part of a collaboration with the warumungu aboriginal community in central australia. during a visit to the national archives in darwin, the warumungu community members expressed both tension and relief when viewing the images and documents held in the archives. the tension centered on the violation of cultural protocols observed by warumungu people in distributing, circulating, and reproducing their cultural materials. materials that were only meant to be viewed by certain families or kingroups were publicly viewable. images of relatives who were no longer living were found within the archive and the online catalogue. following the visit, we worked together to identify the specific sets of needs for a platform to care for those materials. a system whose functionality respected the dynamic social and cultural systems, relationships, and cultural protocols for sharing, circulating and creating knowledge within the warumungu community. this meant focusing on familial and kin-based systems of relation and circulation and allowing for different parameters for access, viewing, listening and annotating content. the mukurtu wumpurrarni-kari archive was launched in 2007. mukurtu translates directly to “dilly bag” in the warumungu language however, the warumungu elder, michael jampin jones recast the term as a “safe keeping place.” like the dilly bag, the mukurtu platform is meant to protect and preserve cultural materials while also circulating and sharing them in appropriate ways (christen, 2012, p. 2283). as michael jampin jones described: mukurtu is that dilly bag. the way that warumungu people use it, they used to have the old dilly bags. in the early days, old people kept their stuff in that bag and no one else was allowed to open it up or even to look at it. now we have this archive, it’s good. in every archive that you go to there’s a lot of stuff that’s in a safe place, in south australia, queensland. but this archive will be different, it will be here, it will be the safe place for men like me. i can’t see women’s stories or even if she is my daughter or niece i’m not allowed to see, so it won’t open up for me, because we have different passwords. that is very important to make it safe. (mukurtu (b), n.d). after the mukurtu wumpurrarni-kari archive launch, it was clear that indigenous communities across the globe shared similar sets of archival, cultural heritage, and content management needs. from its beginnings, the mukurtu project and cms have been developed to assist the management of indigenous cultural collections aligned with indigenous ways of australasian journal of information systems thorpe et al. 2021, vol 25, research on indigenous use of designing archival information systems information and communication technologies 10 knowing and doing. the project provides an opportunity for redressing the gap in approaches in systems that currently serve the library and archive management, including online discovery, access and preservation. mukurtu cms is primarily an access platform, addressing one component of the larger digital stewardship ecosystem, through access, preservation and collections management sustainability. directly related to access is digital return. from the original version to the software’s newest release, a main component of the platform has been to allow communities to reframe, renarrate, and share collections materials from national, regional and local collecting institutions. mukurtu’s core features and the system’s functionality provide pathways for sharing, enriching and circulating materials following local cultural protocols. alongside providing a flexible and customisable platform, a key core value of the mukurtu cms project are the educational and community engagement components. mukurtu support has always been a core part of the project. community workshops, seminars and one-on-one training ensure that communities can use the platform sustainably. as the system matured and moved to a browser-based framework, the mukurtu team was aware that both online and offline support would be crucial. the mukurtu hubs and spokes project grew out of simultaneous conversations with local communities using mukurtu cms and national repositories discussing ways to work with indigenous communities to facilitate content sharing. there are currently five mukurtu hubs, four in the us and one in australia. the hubs provide regional support to indigenous communities, “the spokes” in their areas. simultaneously, communities provide direct feedback on their needs with the platform. in this way, development, support and training are cyclical. the continued development of mukurtu cms through the hubs and spokes model expands the mukurtu cms support structure and enhances its features through direct feedback from a vast community of users. working in tandem between hubs and spokes, the project builds sustainability at the community and platform level — indeed, the two are inseparable. 6.2 indigenous self-determination and disrupting centralised systems of archival control (lauren booker) for collecting institutions to consider engaging meaningfully in supporting indigenous sovereignty over collections held, an institution must, as birch phrased, ‘redeem and take responsibility for its colonial debts’ (birch, 2007, p. 106). ̂ in the work that i have engaged in across the glam sector, i consider a central challenge to indigenous sovereignty to be the deeply embedded structures of institutional control over collections and how that affects indigenous people's self-determination regarding knowledges, heritages and identities embedded in collection materials and records. institutional control of collections includes determining how materials, recordings and records held are cared for, described, curated and shared. i suggest that radical transparency of these systems of control and the ongoing connection to glam institution’s colonial foundations are the first steps to engaging in meaningful accountability and supporting indigenous sovereignty. karen martin uses the phrase ‘terra nullius research’ to describe research that positions indigenous peoples as the subject of discussion while eschewing indigenous agency and voice (martin, 2003, p. 203). the notion of ‘terra nullius’ uninhabited land was a colonising legal definition enabling the possession of land through the disregard of indigenous peoples as sovereign peoples. aileen moreton-robinson argues that the legal fiction of terra nullius is so pervasive that it continues to dictate the relationship between aboriginal and torres strait australasian journal of information systems thorpe et al. 2021, vol 25, research on indigenous use of designing archival information systems information and communication technologies 11 islander peoples and the australian state today and continues to inform understandings of belonging or not-belonging (moreton-robinson, 2015). the australian colonial history, which saw the dispossession of indigenous nations from their lands, simultaneously comes with the relentless colonial project to undermine self-determination of indigenous identity and culture as connected to land and water. the collecting institution sector (libraries, archives, museums) in australia has its foundations in supporting and proliferating what karen martin defines as “terra nullius research’ (martin, 2003, p. 203) and has played a substantial role in controlling, constructing and deconstructing aboriginal identities (russell, 2001). systems of control in the glam sector that i have experienced directly impact aboriginal and torres strait islander focused projects, include data governance, institutional time constraints on projects, and digital systems that flatten and assume indigenous knowledges into archival paradigms that push cultural materials deeper into digital spaces for research findability rather than community usability. i have seen these systems of control push reciprocal relationship building and indigenous data sovereignty to the back of the list time and time again. conversations that do not centre indigenous methodologies in the realm of data and technology often work to decentre indigenous peoples and indigenous knowledges, even if there is the intention to facilitate and support communities. i want to highlight the importance of the ‘right to know’ as developed by native american scholar vine deloria jr (standing rock, sioux) in the 1978 paper titled ‘‘the right to know’’, prepared for the white house pre-conference on indian library and information services on or near reservations. i became aware of deloria’s clear imperative of the “right to know” through the papers of jennifer o’neal (the confederated tribes of grand ronde) and allison boucher krebs (anishinabe); both papers consider “deloria’s right to know to do list” as highly influential in strengthening tribal repositories and the broader archive decolonisation movement in north american (krebs, 2012; o’neal, 2015). in my own work, and the work of my colleagues, i likewise consider the “right to know” central to the work of indigenous archives, archiving and archivists. the transparency needed in the australian glam sector to centre indigenous self-determination and disrupt centralised systems of archival control closely follows that of deloria’s “right to know,” that indigenous peoples; need to know; to know the past, to know the traditional alternatives advocated by their ancestors, to know the specific experiences of their communities… (deloria, 1978). a key difference in context that should be recognised is that deloria situates the right to know as necessary under treaty obligations (deloria, 1978). recognising that in australia, we are without the history of a treaty but not without a dedicated conversation on a treaty there is still a settlercolonial relationship between the australian state and aboriginal and torres strait islander peoples that requires accountability of the state (atsic & aiatsis, 2003). deloria points out that to move forward on upholding obligations to support indigenous people’s “right to know” in the archive sector there is but left the straightforward “will to act” (deloria, 1978). in the australian archives context the mandate for centering of aboriginal and torres strait islander people’s needs, imperatives and self-determination, and institutional accountability is increasingly being pushed forward with the creation of glam sector policies, guidelines and protocols. the most recent being the tandanya/adelaide statement (ica, 2019), which shows a sector that is poised with intention it is now the will to act that needs to follow through. australasian journal of information systems thorpe et al. 2021, vol 25, research on indigenous use of designing archival information systems information and communication technologies 12 in my phd studies, i am building upon aboriginal and torres strait islander led conversation to engage in deep listening and communication, focused on ideas and imperatives of community-led collection care for ancestral remains in the form of hair samples that are still held in collecting institutions. the research questions i am considering centre on building reciprocal relationships with an indigenous-led participant group to examine issues of transparency and control of ancestral remains and related documentation, and imagine together what a radical reorientation of indigenous self-determination for these collections and stories could and should be. the research aims to interrupt centralised systems of archival control and terra nullius research by asking aboriginal and torres strait islander experts what is a proper way forward and centring the right to know and the right of reply in the collecting institution sector. the nsw australian mukurtu hub, and connections with the us hubs and spokes projects, enable international discussions on addressing historical systems of colonial and assimilatory archival control. mukurtu functions in support of working collaboratively on collections that require community response to negotiate access protocols, indigenous cultural and intellectual property (icip) rights, handling protocols, and the right of reply. mukurtu approaches enable discussion on those higher mandate questions of how can we engage with indigenous self-determination in the archive, post-custodial archives, institutional transparency and reciprocal relationship building. it supports a reconsideration of notions of time: including the time it takes to build relationships before building digital projects, and that consent and consultations are not one-time events they are conversations built upon dynamic relationships. the hubs and spokes model supports a reconsideration of how we in the glam sector share information and skills and how we can continue to ask and re-ask questions that are not focused on software. rather they are focused on relationships. 6.3 indigenous cultural safety and collaborative design (kirsten thorpe) for many indigenous people in australia, glam institutions are places to either love or loathe (andrews, 2017, p.91). they are institutions that have supported the colonisation of australia through their collecting practices, and as such, they have assisted the creation of a colonial narrative that portrays indigenous peoples as dangerous, barbaric and nameless peoples (behrendt, 2016, p.5). although australian libraries and archives can play a significant role in supporting indigenous peoples aspirations, they remain sites of great distrust (mckemmish, faulkhead & russell, 2011) and trauma (russell, 2005, p.164). narungga researcher natalie harkin describes her encounter with the colonial archive as being witness to “extraordinary acts of surveillance” and the trauma of accessing and engaging with family records held in government archives as a witnessing of “histories of silencing and forgetting.” (harkin, 2019, p.15). colonial archives, by their very nature, can be highly traumatic for indigenous people to access. they are often bureaucratic and challenging to understand because of the context in which they were created. many records held in collecting institutions contain derogatory and offensive language; they are also incomplete and biased and sometimes contain information that has been fabricated or inaccurate (garwood-houng, 2005, p.144). these records and their narratives were created in violent, intrusive and racist contexts, with collections being developed as an outcome of administrative processes associated with the control of indigenous people and communities. this includes, for example, records created as a result of the removal of indigenous children as part of the stolen generations. australasian journal of information systems thorpe et al. 2021, vol 25, research on indigenous use of designing archival information systems information and communication technologies 13 traditional archival and information systems do not accommodate indigenous peoples counter-narratives, including oral memories, histories and records. instead, they support a single view of history, written by those who held positions of power (faulkhead, 2008, p.88). these historical archives support dominant narratives constructed within the broader australian colonial project. they privilege the view of the records creator rather than the person or peoples who are the subject of the records. providing access to these historical archives in digital environments, particularly through online library and archive catalogues and systems, is hugely problematic, and the lack of context and existence of indigenous voice and perspective further perpetuates existing biases. as i have argued elsewhere (thorpe, 2019), caswell’s (2014) discussion on “symbolic annihilation” helps us to understand the impacts of how marginalised people are silenced in the archive, and so, are not able to see themselves represented within history. this silencing of indigenous perspectives has a major effect, for example, on the teaching of history in australia and the inclusion of indigenous perspectives in school and university curricula. my research interests relate to how indigenous people in australia can be culturally safe when accessing libraries and archives and how more broadly, indigenous people can build selfdetermination to support local priorities for information and knowledge management. my doctoral studies focus on investigating indigenous archiving and cultural safety, including questioning the role of decolonisation in australian libraries and archives. the research investigates how libraries and archives can negatively impact indigenous peoples wellbeing through a failure to produce archive and information systems that can best meet communities’ cultural and information needs. i utilise williams’s (1999) concept of ‘cultural safety’ to investigate how libraries and archives can either make aboriginal and torres strait islander peoples feel safe or unsafe. williams (p.213) defines cultural safety as: [a]n environment that is safe for people: where there is no assault, challenge or denial of their identity, of who they are and what they need. it is about shared respect, shared meaning, shared knowledge and experience of learning, living and working together with dignity and truly listening. i argue that current archival information systems, methods and approaches do not acknowledge the ongoing impacts of colonisation and intergenerational trauma on indigenous peoples lives, so they do not meet the requirements of being culturally safe. as i reflect on my own experiences of developing policy, protocols and being involved in practice-based research around indigenous engagement with archives (thorpe, 2001; thorpe, 2005; thorpe, 2014), i would consider relationships and consent to be vital elements for building culturally safe archival information systems. i am interested in developing methods and approaches to facilitate the digital return of archives to local communities based on principles of respect and trust. that means that digital return needs to be considered in its fullest sense, including the handing over of decisions around the overall management of archives spanning access, description, distribution and use of materials and facilitating a “right of reply” to the records. relationships are vital to building archival information systems for digital return, and the wider grass-roots mukurtu community of practice works to support these principles of engagement with archives. i have experienced working in culturally unsafe libraries and archives. i became interested in how archival information systems were failing to support indigenous people’s information australasian journal of information systems thorpe et al. 2021, vol 25, research on indigenous use of designing archival information systems information and communication technologies 14 needs while contributing to reparations projects at the nsw state archives. in the mid-2000s indigenous people in nsw become more aware of the state archives collections, including records created by the former nsw aborigines protection and welfare boards. the nsw aboriginal trust fund repayment scheme (atfrs) and the in living memory exhibition, challenged normative approaches to archival management5 . they also identified multiple and complex wicked problems concerning access and use of these historical records, problems that require a rethinking of approaches to support indigenous people’s agency and selfdetermination. the mukurtu cms and wider hubs and spokes model aim to engender respect for indigenous perspectives and experiences to build dialogue around the different ways people see, know, and operate in the world. being engaged with other indigenous peoples in the mukurtu project also activates a research and community agenda for change in the sector. drawing on the words of the former australian aboriginal and torres strait islander social justice commissioner, mick gooda, who called for indigenous people in australia to move from being “passive and powerless subjects, to active participatory agents” (p.97), mckemmish et al. (2012, p.108) note that: digital technologies, participatory models and postmodern concepts like co-creatorship and records as social entities have the potential to support the ‘decolonisation’ of the archive, and the exercise of cultural rights in archives as human rights, repositioning indigenous peoples who have hitherto been the ‘captives of the archives’ (fourmile 1989), the subjects of records, and the objects of the archival gaze as active participatory agents in archival and recordkeeping program and system design and implementation (p.108). the research associated with the nsw australian mukurtu hub enables investigation into indigenous self-determination and sovereignty and the challenges that need to be unshackled with traditional archiving approaches. we need to invest time to discuss communities’ requirements to design archival information systems that do not perpetuate trauma and the silencing of indigenous people’s voices. culturally safe archives and libraries require new investigations into indigenous peoples’ user needs, and this design can only be achieved if archival institutions work in partnership with communities. 6.4 collaborative design through partnerships with indigenous communities: the mukurtu cms hubs and spokes model (monica galassi) in my professional journey i have been interested in how partnerships can be built to co-design digital systems and tools to support access of indigenous archival collections held in collecting institutions, and the mechanisms that are required to support the local management of cultural heritage collections. i came to appreciate the mukurtu cms as it has been adopted by a number of different communities in australia and internationally as a tool for this collaboration and 5 the records of the nsw aborigines protection and welfare boards, 1883 – 1969 document the policies of the boards which had a major impact on the lives of aboriginal people in nsw, including for example in relation to the removal of children and establishment of aboriginal reserves. the atfrs was established by the nsw government in late 2004 to investigate the repayment of monies that were held in trust by the nsw aborigines protection and welfare boards and never repaid. see: thorpe, k. (2019). ethics, indigenous cultural safety and the archives. archifacts, 2018, no 2, 33 – 47. retrieved on 29 may 2020, https://www.researchgate.net/publication/339014668_ethics_indigenous_cultural_safety_and_the_archives. australasian journal of information systems thorpe et al. 2021, vol 25, research on indigenous use of designing archival information systems information and communication technologies 15 community engagement. my experiences have been that mukurtu has been considered a system suitable for indigenous communities because of the distinct features of the cms to manage indigenous knowledges, and the overall guiding principles of the project that prioritise partnership with communities. the support network built around mukurtu has a democratic ethos; it is a platform that is free, open-source and encourages interconnections with other systems. this long-term view of sustainability was also at the core of establishing the nsw australian mukurtu hub – a key interest of mine in connecting with the project to support communities in building and managing their local collections. as has been argued in this paper previously, the information systems that support access to historical archives relating to indigenous peoples in australia often do not meet their primary users’ needs. in many instances, they lack the ongoing relationships needed to conceptualise and co-design new ways of managing and curating digital archives springing from communities’ needs and aspirations. from an institutional point of view, providing access to historical documentation can be complex, particularly in archives where the scale and depth of collections are diverse (for example, where the collection materials relate to communities across australia). providing transparent and culturally safe access to historical collections for many cultural and collecting institutions is still limited to isolated projects due to funding constraints and tight timeframes. likewise, some of the opportunities which arise when trying to support digital keeping places at a local level tend to slow down quickly when negotiations start to center on authority, ownership, and ongoing support of partnership models which are not aligned with institutional collecting strategies. often, the goodwill of opening up archives and information is there, but making it an ongoing process embedded at the core of institutional practices is more challenging to achieve. my experience has been that the process of embedding a cms like mukurtu at the core of cultural institutions often highlights tensions that are far from being just linked to technology or funding availability. some of the questions that arise during this process concern the why (‘why do we need a ‘special’ system dedicated to aboriginal collections?/why can we use a system built/used by other multicultural communities who want to access the collections?). the how (‘how are we going to manage the ownership of the information, and how do we manage the storage and preservation of the information which will be provided?’). the what (what does it mean to support aboriginal sovereignty? what is going to happen to the future of collecting institutions if we don’t own these collections anymore?). in most cases, support for local collections (both in terms of sharing skills through training and provide assistance with tools and physical and digital preservation) is currently ad hoc and supported differently by the various archival and collecting institutions. that is, often it is the institution that decides when and how to provide support according to its own opinion on the interest of the local collections, perpetuating power over community interests. this results on lack of long-term sustainability, in particular through not supporting community employment. importantly, current approaches also largely lack a focus on community driven local translation of meanings and requirements. these are just some of the reasons why new models of engagement are required to support the recognition of aboriginal self-determination and sovereignties. the development of the nsw australian mukurtu hub supports this model of capacity building and training communities in curriculum related to digital stewardship. the hubs and spokes model works australasian journal of information systems thorpe et al. 2021, vol 25, research on indigenous use of designing archival information systems information and communication technologies 16 together to share information and resources and discuss challenges surrounding digital curation and the management of protocols for collections. the project also enables meaningful conversations around systems development, keeping people in dialogue about design, functions and features that might be suitable for their community’s needs. again, the focus is not on the technology, but on the relationships. when conversations started around building the nsw australian mukurtu hub, i worked as part of the indigenous engagement team at the state library of nsw. there, my role was responsible for developing of the first platform for the digital return of the library’s archives to communities, using the cms mukurtu. called gather, the site aims to return copies of historical documents, manuscripts and photographs digitally, collaborate to identify people, places and stories in historical photographs, and add local knowledge and perspectives to this historical material (state library of nsw, 2020). since then, i have been actively involved in the establishment and development of the hub, which was launched in december 2018. the nsw australian mukurtu hub is now a participating hub in the mukurtu hubs & spokes model to connect australian communities to the mukurtu project in the us. it functions as a central point of connection with the mukurtu project team at wsu and with local communities and other interested users across nsw and beyond. the nsw australian mukurtu hub aims to support regional centers in providing guidance and training to communities who wish to gain more insights into how to manage their local collections digitally. a key focus is on keeping collections on country while also gathering feedback on how local communities want to use and further develop the list of requirements related to the mukurtu cms. a commitment of the australian hub is to develop tools and guidelines in partnership with communities to support processes relating to the return and maintenance of indigenous collections. we also aim to drive a national agenda on research community needs related to indigenous archives and digital stewardship for the responsible dissemination and access of collections (nsw australian mukurtu hub, 2019). i have learned many valuable lessons from my experiences of working in institutions to support the digital return of collections. both professional and structural changes need to occur in cultural institutions to fully embrace aboriginal sovereignty on information and data held across collections. adopting a digital system for indigenous collections, and supporting communities to manage their own, is not just a matter of adopting the right technology or to have access to funding. it is a matter of strong leadership and willingness to make internal changes, such as building cultural competence among staff. as has been previously argued, to build collaborative partnerships with aboriginal communities around the design of archival and information systems that support sovereign community views, we must first acknowledge the deep harm that these past colonial practices have involved. hence, to do this, we need to be culturally competent to understand the impact that these histories had and still have on people. a meaningful and respectful user experience when accessing aboriginal collections starts asking people what they want and requires staff working in this space to have cultural competence to understand the complexities that will arise. this deep listening is fundamental in understanding diverse standpoints and worldviews, and supporting direct calls for action, both from an organisational and personal point of view. non-indigenous allies and accomplices, working to make changes in the overall power structures embedded in archives, can play a vital role in supporting aboriginal sovereignty in this space. hence, i strongly believe that the labour to progress these agendas should not rest solely on the australasian journal of information systems thorpe et al. 2021, vol 25, research on indigenous use of designing archival information systems information and communication technologies 17 leadership of indigenous people working in the cultural sector. it requires broad support from all levels of leadership to transform the way we do things to support shifts and changes, starting from our own professional area. it is all about learning from each other and establishing long-lasting relationships with communities, taking responsibilities in letting communities drive us through their needs, aspirations and priorities. 7 conclusion this paper brings together discussions on the area of indigenous self-determination, sovereignty and archival information systems. it looks at how mainstream archival information systems do not support the appropriate management and digital return of indigenous collections held in archives, and connects these actions to supporting indigenous self-determination and sovereignty. it is an ongoing issue that libraries, archives and museums often ignore aboriginal sovereignties and disregard existing community governance and language structures to maintain an institutional status-quo. this is particularly apparent in the information systems used by libraries, archives and museums that lack appropriate community governance and respect for indigenous peoples cultural safety. indigenous systems of governance and knowledge access needs are complex, nuanced and vary widely from nation to nation internationally, and this needs to be reflected in the information and knowledge systems used by institutions that work with communities and their collections. furthermore, to engage with building systems and services that more appropriately serve indigenous communities, there must be a focus on relationship building and sustaining. the research and projects associated with mukurtu project, including the mukurtu cms and the hubs and spokes model, provide an example of information systems and collection management projects that centre on building relationships of respect and trust in relation to indigenous digital cultural heritage. the reciprocal relationship model engaged with by the mukurtu project is an example of how collecting institutions should be reconsidering the services and information systems they provide to communities. the development of statements such as the recent ica adelaide-tandaya declaration provides an international call to action to centre indigenous sovereignty and self-determination in the archives sector, however, what remains is, as vine deloria jr observed, the will to act. acknowledgments we would like to acknowledge the traditional owners of the land where we all write from in australia and the us and all the aboriginal, first nations, native american and indigenous peoples who, before us, lead the way in fighting for indigenous self-determination and sovereignty. as authors, we also acknowledge that the order of authorship reflects our collaboration on the panel at the ica congress. references aboriginal and torres strait islander commission (atsic) & australian institute of aboriginal and torres strait islander studies (aiatsis). (2003). treaty: let's get it right! canberra, act: aboriginal studies press. aboriginal and torres strait islander library, information and resource network (atsilirn). (2012). the aboriginal and torres strait islander protocols for libraries, archives and information services. retrieved 5 may 2020, from https://atsilirn.aiatsis.gov.au/docs/protocolbrochure2012.pdf australasian journal of information systems thorpe et al. 2021, vol 25, research on indigenous use of designing archival information systems information and communication technologies 18 andrews, j. (2017). indigenous perspectives on museum collections. artlink, 37(2), pp. 88-91. australian human rights commission. (1997). australian human rights commission (1997) bringing them home: report of the national inquiry into the separation of aboriginal and torres strait islander children from their families. retrieved 31 march 2021, from https://humanrights.gov.au/our-work/bringing-them-home-report-1997 australian society of archivists. (2019). 2019 program, designing the archive. https://www.archivists.org.au/conference/program-and-abstracts/abstracts barwick, l., green, j., & vaarzon-morel, p. (2020). archival returns: central australia and beyond. sydney, nsw, sydney university press. behrendt, l. (2002). self-determination and indigenous policy: the rights framework and practical outcomes. journal of indigenous policy, 1, pp. 43-58. http://www.austlii.edu.au/au/journals/jlindigp/2002/7.pdf behrendt, l. (2016). finding eliza: power and colonial storytelling. university of queensland press. behrendt, l., vivian, a. (2010). indigenous self-determination and the charter of human rights and responsibilities. a framework for discussion, victorian equal opportunity & human rights commission, state of victoria. https://www.deadlystory.com /icms_docs/275955_indigenous-self-determination charter-of-human-rights.pdf birch, t. (2007). ‘the invisible fire': sovereignty, history and responsibility. in moretonrobinson, a. (ed). sovereign subjects: indigenous sovereignty matters. (pp.105-117) allen & unwin. brilmyer, g., & caswell, m. (2017). easy to print: identifying and dismantling white supremacy in archives. escholarship, university of california. retrieved august 21, 2021, from https://escholarship.org/uc/item/0zn923k8 christen, k. (2011) opening archives: respectful repatriation. the american archivist: spring/summer 2011, 74(1), pp. 185-210. christen, k. (2012). does information really want to be free? indigenous knowledge systems and the question of openness. international journal of communication, 6, pp. 2870– 2893. christen, k. (2015). sovereignty, repatriation, and the archival imagination: indigenous curation and display practices. collections, 11(2), pp. 115-138. https://doi.org/10.1177/155019061501100204 christen, k., merrill, a. and wynne, m. (may/june 2017). a community of relations: mukurtu hubs and spokes. d-lib magazine, 23(5/6). doi https://doi.org/10.1045/may2017-author cunningham, a. (2005). archival institutions. in mckemmish, s., piggott, m., reed, b., & upward, f. archives: recordkeeping in society. (pp. 21-50). chandos publishing. deloria v, united states and white house pre-conference on indian library and information services on or near reservations. (1978) the right to know: a paper. office of library and information services, u.s. department of the interior, washington. australasian journal of information systems thorpe et al. 2021, vol 25, research on indigenous use of designing archival information systems information and communication technologies 19 duff, w. m., flinn, a., suurtamm, k. e., & wallace, d. a. (2013). social justice impact of archives: a preliminary investigation. archival science, 13(4), pp. 317-348. https://doi.org/10.1007/s10502-012-9198-x evans, j., mckemmish, s., & rolan, g. (2017). critical approaches to archiving and recordkeeping in the continuum. journal of critical library and information studies, 1(2). https://doi.org/10.24242/jclis.v1i2.35 evans, j., faulkhead, s., thorpe, k., adams, k., booker, l., & timbery, n. m. (2020). indigenous archiving and wellbeing: surviving, thriving, reconciling. in j. a. bastian, & a. flinn (eds.), community archives, community spaces: heritage, memory and identity (pp. 129-164). facet publishing. faulkhead, s. (2008). narrative creation and koorie victoria. centre for australian indigenous studies, faculty of arts. melbourne, monash university. phd fourmile, h. (1989). who owns the past? aborigines as captives of the archives. aboriginal history, pp. 1-8. garwood-houng, a. (2005). protocols: meeting the challenges of indigenous information needs. australian academic & research libraries, 36(2), pp. 143-151. https://doi.org/10.1080/00048623.2005.10721254 garwood-houng, a., & blackburn, f. (2014). the atsilirn protocols: a twenty-first century guide to appropriate library services for and about aboriginal and torres strait islander peoples. the australian library journal, 63(1), pp. 4-15. https://doi.org/10.1080/ 00049670.2014.890018 general assembly of the united nations. (2015). united nations declaration on the rights of indigenous peoples. http://declaration.humanrights.gov.au/ gooda, m. (2012). the practical power of human rights: how international human rights standards can inform archival and record keeping practices. archival science, 12(2), pp. 141-150. grieves, g., & kelada. o. (2017) bleeding the archive, transforming the mythscape, in jorgensen, d., & mclean, i. (eds.). (2017). indigenous archives: the making and unmaking of aboriginal art. uwa publishing. harkin, n. (2019) ‘colonial archive’, archival poetics 1, vagabond press. international council on archives (ica). (2019). ica/naa indigenous matters summit, see us, hear us, walk with us: challenging and decolonising the archive, retrieved august 21, 2021, from https://www.ica.org/en/icanaa-indigenous-matters-summit jonas, w. (2002). recognising aboriginal sovereignty implications for the treaty process, speech at the atsic national treaty conference. retrieved august 21, 2021, from https://humanrights.gov.au/about/news/speeches/recognising-aboriginal-sovereignty implications-treaty-process-2002 jones, j. (2018). murruwaygu: following in the footsteps of our ancestors (doctoral dissertation). university of technology sydney, sydney, australia. http://hdl.handle.net/10453/133196 australasian journal of information systems thorpe et al. 2021, vol 25, research on indigenous use of designing archival information systems information and communication technologies 20 krebs, a. (2012). native america’s twenty-first-century right to know. (report). archival science: international journal on recorded information, 12(2), pp 173-190. local contexts. (n.d.). traditional knowledge (tk) labels. https://localcontexts.org/labels/ traditional-knowledge-labels/ martin, k. (2003). ways of knowing, ways of being, ways of doing: a theoretical framework and methods for indigenous research and indigenist research, journal of australian studies, 76, pp. 203-14. mckemmish, s., faulkhead, s., & russell, l. (2011). distrust in the archive: reconciling records. archival science, 11(3-4), pp. 211-239. https://doi.org/10.1007/s10502-011-9153-2 moreton-robinson, a. (2007). sovereign subjects: indigenous sovereignty matters. allen & unwin. moreton-robinson, a. (2015). the white possessive: property, power, and indigenous sovereignty. minneapolis: university of minnesota press. mukurtu (a). (n.d). mukurtu: about. https://mukurtu.org/about/ mukurtu (b), n.d). mukurtu wumpurrarni-kari archive. https://mukurtu.org/project/ mukurtu-wumpurrarni-kari-archive/ nakata, m., byrne, a., nakata, v., & gardiner, g. (2005). indigenous knowledge, the library and information service sector, and protocols. australian academic & research libraries, 36(2), pp. 7-21. https://doi.org/10.1080/00048623.2005.10721244 nsw australian mukurtu hub. (2019). about. https://mukurtu-australiansw.libraries.wsu.edu/nsw-australian-mukurtu-hub o'neal, r. (2015). "the right to know": decolonizing native american archives. journal of western archives, 6(1), article 2. https://doi.org/10.26077/fc99-b022 ormond-parker, l., & sloggett, r. (2012). local archives and community collecting in the digital age. archival science, 12(2), pp. 191-212. 10.1007/s10502-011-9154-1 punzalan, r. l., & caswell, m. (2016). critical directions for archival approaches to social justice. the library quarterly, 86(1), pp. 25-42. https://doi.org/10.1086/684145 royal commission into aboriginal deaths in custody. (1991). royal commission into aboriginal deaths in custody—national report. retrieved 31 march 2021, from http://www.austlii.edu.au/au/other/indiglres/rciadic/ russell, l. (2001). savage imaginings: historical and contemporary constructions of australian aboriginalities. melbourne: australian scholarly publishing. sentance, n. (2017). maker unknown and the decentring first nations people [blog post], archival decolonist. retrieved from https://archivaldecolonist.com/2017/07/21/makerunknown-and-the-decentring-first-nations-people/ state library of new south wales. (2020). gather: connecting aboriginal communities with collections and stories from the state library of nsw. retrieved 20 april 2020, from https://gather.sl.nsw.gov.au/ smith, l. t. (2013). decolonizing methodologies: research and indigenous peoples. london, uk, zed books ltd. australasian journal of information systems thorpe et al. 2021, vol 25, research on indigenous use of designing archival information systems information and communication technologies 21 thorpe, k. (2001). indigenous records: how far have we come in bringing the history back home?. archives and manuscripts, 29 (2), p. 10. thorpe, k. (2005). indigenous knowledge and archives. australian academic & research libraries, 36(2), pp. 179-184. thorpe, k. (2013). protocols for libraries and archives in australia: incorporating indigenous perspectives in the information field. proceedings of ifla wlic conference 2013 singapore future libraries: infinite possibilities in session 125 education and training with library services to multicultural populations and indigenous matters special interest group. retrieved from http://library.ifla.org/99/1/125-author-en.pdf thorpe, k. (2014). indigenous records: connecting, critiquing and diversifying collections. archives and manuscripts, 42(2), pp. 211-214. https://doi.org/10.1080/01576895. 2014.911692 thorpe, k. (2017). aboriginal community archives. in a. gilliland, s. mckemmish, & a. lau (eds.), research in the archival multiverse (pp. 900-934). monash university publishing. thorpe, k., & galassi m. (2014). rediscovering indigenous languages: the role and impact of libraries and archives in cultural revitalisation. australian academic & research libraries, 45(2), pp. 81-100. https://doi.org/10.1080/00048623.2014.910858 thorpe, k. (2019). ethics, indigenous cultural safety and the archives. archifacts, 2018, no 2, 33 – 47. retrieved on 29 may 2020, from https://www.researchgate.net/publication/ 339014668_ethics_indigenous_cultural_safety_and_the_archives. toner, p. (2004). history, memory and music: the repatriation of digital audio to yolngu communities, or memory as metadata. proceedings of the open conference systems, sydney, australia: university of sydney, faculty of arts. http://hdl.handle.net/2123/1518 tuck, e., & yang, k. w. (2012). decolonization is not a metaphor. decolonization: indigeneity, education & society, 1(1). united nations (1948). the universal declaration of human rights. retrieved from https://www.un.org/en/universal-declaration-human-rights/ un general assembly (2007). united nations declaration on the rights of indigenous peoples. retrieved from https://www.un.org/development/desa/ indigenouspeoples/declaration-on-the-rights-of-indigenous-peoples.html walter, m., & suina, m. (2019). indigenous data, indigenous methodologies and indigenous data sovereignty. international journal of social research methodology, 22(3), pp. 233243. https://doi.org/10.1080/13645579.2018.1531228 watson, i. (2016). first nations and the colonial project. inter gentes the mcgill journal of international law and legal pluralism, 1(1), pp. 30-39. williams, r. (1999). cultural safety—what does it mean for our work practice?. australian and new zealand journal of public health, 23(2), pp. 213-214. australasian journal of information systems thorpe et al. 2021, vol 25, research on indigenous use of designing archival information systems information and communication technologies 22 copyright: © 2021 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v25i0.2917 http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 background 3 introducing the authors 4 indigenous self-determination, sovereignty and archives in australia 5 the challenge of digital return of archives to indigenous communities 6 transforming archival and information systems in collaboration with indigenous peoples and communities: the mukurtu project in australia 6.1 ‘mukurtu—a safe keeping place’ (kimberly christen) 6.2 indigenous self-determination and disrupting centralised systems of archival control (lauren booker) 6.3 indigenous cultural safety and collaborative design (kirsten thorpe) 6.4 collaborative design through partnerships with indigenous communities: the mukurtu cms hubs and spokes model (monica galassi) 7 conclusion paper title (paper title style) australasian journal of information systems behl et al. 2021, vol 25, research on user involvement gamifying the gig 1 gamifying the gig: transitioning the dark side to bright side of online engagement abhishek behl o.p. jindal global university, india abhishekbehl27@gmail.com pratima sheorey scm hrd, symbiosis international (deemed) university, pune, india kokil jain amity international business school, noida, india meena chavan macquarie university sydney australia isha jajodia birla institute of management technology (bim tech), india zuopeng (justin) zhang university of north florida, usa abstract gig work has transformed the work culture, globally; its spread and popularity has attracted excellent talent, most of it online. while the gig sector has opened new avenues to showcase talent, lack of consistent engagement of gig workers have resulted in significant dropouts. the study addresses a key research gap by investigating digital platform gig work dropouts through the moderating impact of gamified interventions on online platforms. we have based our arguments and derived hypotheses on the basis of the social exchange theory and the selfdetermination theory. a total of 367 responses were collected from white-collar gig workers who had completed tasks on one or more gig platforms in the past two years. we test our hypotheses using partial least square structural equation modelling (pls-sem). results confirm that gamifying the online platform would enhance job satisfaction and productivity of gig employees, thereby reducing their chances of quitting gig work. it is further observed that in the case of gig workers, high-performance work systems have a non-significant effect on the intentions to quit. the results contribute to the redesigning of online gig platforms with a layer of gamified artifacts to increase gig workers' retention. keywords: gamification, gig work, intention to quit, dark side, engagement 1 introduction the changing nature of employment and a progressive shift in the working ecosystem that is fundamentally powered by the autonomous nature of the workforce has given rise to the “gig economy”. characterized by a free market system, conventional businesses in the “gig economy” engage independent freelancers on a short-term contract basis to perform tasks that they have expertise in. the term “gig” has been adopted from the context of performing artists who perform specific to an occasion or location referred to as gig work (longley, 2020). the gig economy workers are paid as per tasks, unlike the typical salary-based pay-out in hierarchical organizations. this progressively evolving work structure signals a shift from the australasian journal of information systems behl et al. 2021, vol 25, research on user involvement gamifying the gig 2 traditional career paths that made people work for firms as full-time employees to a more selfdriven alternative that empowers them to wisely choose gig work (kurin, 2016). with technologies disrupting standard working and employment practices, restructuring of business models and individuals aspiring for new and non-conventional career paths (cooper, 2014; davis, 2016) gig work appears to be an emerging and preferred pattern of work. it is reported that 36% of all u.s. workers were gig workers during 2018 and, 29% of all workers in the u.s. have an alternative work arrangement as their primary job. these figures indicate that about 57 million americans had one or more gig jobs (mcfeely & pendell, 2018). these prevailing trends have sought the attention of organizational researchers and practitioners, who are only beginning to explore the experiences of gig workers in coping with the complexities of this new employment structure (caza, moss, & vough, 2018; petriglieri et al., 2018) and examining their motivation to engage in gig work (jabagi et al., 2019). despite the growing popularity and demand for gig work among consumers, workers as well as employers, the gig economy sector is discussed as highly unregulated that poses a plethora of challenges (kurin, 2016; kuhn & galloway, 2019; pichault & mckeown, 201). firms appear to be in a state of flux when they start involving gig in their routine work or a selection of occasional work. from the workers’ perspective as well, there are significant concerns, as gig workers are largely contractual workers, who are not covered by the benefits given to full-time employees like health and retirement benefits, medical leaves, periodic increase in pay and job security. additionally, there is an irregular and modest flow of income, high taxes, constant stress of looking for new gigs, etc. recent studies have pointed out that this sector suffers from frequent dropouts by the workers, irregular patterns of work and lack of motivation that is often not contemplated by new entrants (anwar & graham, 2020; burtch et al., 2018). many gig workers hold multiple jobs, like simultaneously or sequentially driving their cars for uber, ola, lyft, bolt, along with leasing out rooms in their homes through airbnb. they also work on “digital earning platforms,” like ebay, marketplace, and etsy, by selling their used items and participating in online handyman services, like taskrabbit. while this provides them independence, flexibility of remote work and enables them to create a diverse career portfolio (caza et al., 2018) it also inhibits their loyalty to a single organization or occupation (ashford et al., 2018). clearly, inhabiting different roles, juggling between different organizations and tasks is expected to have significant implications in terms of the worker’s motivation. ravenelle (2019) argue that despite digital platforms, marketing gig work as a form of entrepreneurship, gig-workers consider themselves as part-time employees and not entrepreneurs, which can affect their satisfaction and commitment towards the platform and the task. as an outcome of these issues, gig employers confront substantial challenges as they often report a mismatch between supply and demand due to a lack of available gig workforce (healy et al., 2017). this may also lead to unhappiness of customers and loss of business. all these aspects together point towards the dark side of the gig economy which gives rise to some essential questions regarding the factors driving the motivation and retention of gig workers. abraham et al., (2018) have argued that, even though the sector has large potential, its future largely depends on how we can control the irregularities in the operations of the gig work and keep the gig workers motivated and encourage their long term commitment. existing theories have long been used to explore employee motivation in traditional career structure and work settings. but this new form of employment has emphasized the need for australasian journal of information systems behl et al. 2021, vol 25, research on user involvement gamifying the gig 3 scholars and practitioners to understand what accounts for individual motivation as they delve through constantly changing conditions in a gig economy (ashford et al., 2018). it is not merely engagement with the online platform, but the continuous, uninterrupted nature of the engagement that is required to keep the business profitable (behl, 2020). this unique context, where gig workers operate independently without any direct supervisors, necessitates the understanding of the process by which organizations can motivate gig-workers and support their well-being. firms have practiced multiple ways like offering bonuses to gig workers during times of high demand, giving protection to the gig workers by offering life or workrelated insurance (stewart & stanford, 2017) and, engaging them using lucrative schemes and offers (prabowo et al., 2019). while such measures are mostly temporary (jabagi et al., 2019), essential structural difference between conventional organizations and work in the gig economy calls for embedding the existing workflow with something that can help control the intrinsic and extrinsic motivation of the gig workers (faisal et al., 2019). against this background, this study applies the concept of gamification in work-related tasks to explore and assess its role in controlling the gig workers’ intentions to quit from the digital earning platforms. the growing research stream of work on gamification has extensively discussed its motivational and social benefits. due to its potential to modify behavior, theorists and practitioners have begun to acknowledge the importance of gamified elements in influencing individual’s motivation to encourage repeated or continuous behavior. in particular, application of gamification has been found helpful where long-term commitment is desirable (koivisto & hamari, 2019). the authors argue that the use of game elements in work (faisal et al., 2019) like badges, leader boards, points, avatars, etc. can encourage motivation through competition (sailer et al., 2017). it can also make the work more enjoyable which in turn can increase the gig workers' satisfaction and thereby their productivity. while the applications of work-based gamification are numerous, its role in engaging the gig workers is not adequately understood (keith et al., 2019; faisal et al., 2019; poon, 2019). further, friedman (2014) emphasized that in situations where there are uncertainties, and there is no overarching theory that explains the phenomenon perfectly, it is essential to draw insights from multiple theoretical lenses. this study, therefore, integrates the social exchange theory (set) and self-determination theory (sdt) to provide a comprehensive understanding of gig workers motivation, to stick to a digital earning platform. work and functioning of gig tasks and the worker's decision to stay or quit, are governed mainly by social exchange theory (set) which claims that the exchange between employees and firms will continue to happen if the cost-benefit analysis is in favor of the employee. insights from the self-determination theory (sdt) are drawn to help explain, how extrinsic and intrinsic motivation will help the gig workers stay more satisfied and highly productive, which will inhibit their chances to quit the gig ecosystem. overall, using the theoretical lens of set and sdt, the study aims to examine how can gamification help in controlling the “intentions to quit” of the gig workforce engaged on online platforms. this study contributes to the existing body of knowledge in the following three ways. firstly, it expands the current theoretical and practical understanding of gig workers operating on online platforms by layering the current knowledge of employee engagement with a mixed theoretical approach using set and sdt. this, in turn, helps explaining the factors that can reduce the chances of gig workers to quit the gig platform. it is established that the use of gamification as a motivational tool, can help prevent the gig workers from fatigue and reduced australasian journal of information systems behl et al. 2021, vol 25, research on user involvement gamifying the gig 4 motivation that is caused by the unconventional nature of gig work. second, the study offers a novel perspective on the mitigating the impact of the dark side of the gig economy by virtue of proper system design. finally, the study contributes to the existing and growing body of knowledge on gig workforce, which is scarcely discussed in both academic and practical worlds. additionally, the study also raises implications regarding the existing engagement practices followed by gig employers to run their business without compromising the quality of work. the paper is divided into seven sections. section two provides an in-depth review of the literature on gig economy and gamification. this section also discusses the theoretical foundations of set and sdt in the context of gamifying the gig work. section three discusses the research hypotheses and theoretical model. the next two sections discuss the research design and the subsequent results. following which the theoretical and practical implications are discussed in section six which sheds light on the contributions of the study and offers a critical debate between the results of the current study and the previous ones. in the end, section 7 lays out the conclusion, limitations of the study, and provides constructive recommendation for future researchers. 2 literature review and theoretical underpinning 2.1 gig economy the gig sector has gradually gained attention and importance amid the changing nature of business. gillispie (2017) defined the digital contractual workforce as “gig employees,” thereby offering what we know today as the “new economy.” while the gig workforce has been growing, there remains ambiguity in its operational definition (kilhoffer et al., 2017). while there exists no formal definition of its breakup of the sector, studies have classified it into two sectors based on the kind of work done by gig workers: blue collared and white collared (kilhoffer et al., 2017; gillispie, 2017). a white collared gig worker is a person who performs professional, desk, managerial, or administrative work. hasija et al. (2020) in their recent hbr article discusses the rough boundaries of skilled white gig workers by pointing to works that involve consultation and management. the workers belonging to this category are highly skilled and possess high knowledge to perform specific tasks. thus, they can be treated as specialists available in the freelancing space. most of the published literature discusses three critical dimensions of gig work: high degree of autonomy, payment against completion of task/assignment, and short-term relationship between employee and employer. further bifurcations of gig workers are based on their engagement in the online-offline model, whitecollar or blue-collar jobs, individual or crowd work (kässi and lehdonvirta, 2018; rockman et al., 2018). the growth of internet led services that involve gig workers has primarily shifted the gig workforce to online platforms. multiple platforms engage gig workers with gig employers. wood et al. (2019) and meijerink and keegan (2019) observed all three players in the gig economy the employer, the employee, and the platform. petriglieri et al. (2018) reported that gig work was usually seen as extra work performed, especially in the case of blue-collar gig workers; they used this as an opportunity to develop supplementary income sources. it was found that with white-collar online gig workers, there are issues related to higher autonomy leading to lower engagement and a mismatch between job characteristics and job deliverables. while the growth in the number of people joining the gig workforce is significant, the number of dropouts is equally substantial (gandhi et al., 2018; australasian journal of information systems behl et al. 2021, vol 25, research on user involvement gamifying the gig 5 faisal et al., 2019). jabagi et al. (2019) indicated that a lack of commitment and an unregulated work environment often translates into lower quality of deliverables and quality of work. manyika et al. (2016) noted that in the context of white-collar jobs, the rising trend in gig work enrolment is mostly driven by choice and not by necessity; however, it is exactly the opposite for blue-collar gig work. contrarily, studies have also discussed various reasons pertinent to the dark side of gig work, and a lack of motivation is one of the listed reasons (pichault& mckeown, 2019). it is imperative to understand that while motivation drives people to join the gig workforce, its absence also leads to gig workers opting out thereof (poon, 2019; friedman, 2014). therefore, it is vital to identify a theoretical lens that will put light on the switching motivational behaviour of gig workers or any mechanism that may will help explain how gig workers can remain in the gig economy. so far it had been identified that the work profile of gig workers and the tasks they undertake are vital criteria that decide the outcomes of their intentions to continue in gid roles (healy et al., 2017; stewart & stanford, 2017). knowing that attrition rate and switching rate among gig workplaces is high, it is important to understand theoretical and practical ways to engage the gig workforce. the next section discusses, both theoretically and practically, the role of gamification in motivation and engagement of people. 2.2 gamification and its application in gig sector gamification is defined as an “application of game-design elements and game principles in non-game context” (robson et al., 2015, p.411). similarly, werbach (2014, p267), defined gamification as “the process of making activities more game-like”, in order to bring academic and practitioner perspectives closer together. non-game context applications include marketing, teaching, learning, human resource management, health and fitness, etc. cardador et al. (2017) discussed the theory of work gamification; they broadly proposed using game artifacts or game features in the working of an organization/employee to enhance productivity and promote desired work behaviour. most common game artifacts used by firms are points, badges, leader boards, levels and character sheets (kapp, 2012; dale, 2014). the phenomenon of gamifying work in a company or for an individual is mostly captured by employee's motivation. often, firms use gamified plans to engage employees. the most common plans include awarding titles to employees like “employee of the month,” “best manager,” etc. the use of such practices by firms is also categorized under enterprise gamification (ruhi, 2015). some prominent examples are: the salesforce initiative, hunt for chickens; coca cola’s initiative, “my coke rewards”; nike’s initiative, “cheers”; “my marriot hotel” for recruitment in marriot hotels; “escape room” activity designed by phoenix software; etc. such applications have helped firms to deal with most of the issues listed in the octalysis framework – meaning, accomplishment, ownership, scarcity, avoidance, unpredictability, social influence and empowerment. when considering the application of gamification in the context of human resource management, it is reported that firms use game mechanics at almost every step of human resource management, from recruitment drives to exit interviews (cardador et al., 2017; swacha & muszyńska, 2016). it has applications both within and outside the organization. studies report that gamification has helped firms operate better in situations of uncertainty, and with people not on the payrolls of the firm. the role of gamification assumes greater importance where the firm deals with freelance employees, free agents, casual workers, parttime employees, clock hour based workers, or gig workers (lehdonvirta, 2018; carr et al., 2017) australasian journal of information systems behl et al. 2021, vol 25, research on user involvement gamifying the gig 6 because gamification has the ability to engage the attention of gig workers who are switchers. schwellnus et al. (2019) reported challenges related to loyalty, quality of work, and misuse of autonomous power. the application of game elements in the gig economy has been very recent. the primary reasons for lower engagement as reported by petriglieri et al. (2019) include low enrolment rate, delay in completion of task, high dropout rates, lack of motivation to be part of the gig workforce, seasonal behavior of gig workers, etc. to address most of these issues, gig employers have started practicing game-based work engagements (huang et al., 2020). some of the prominent ones are as follows: lyft, a ride-hailing app, uses an algorithmically generated “challenge” for every driver to complete a predefined number of rides in a stipulated time to unlock a reward. mostly, this cash-based reward is in the form of a bonus. occasionally, they fix such tasks once every week to increase the value of the reward. a similar practice is also followed by their competitor, uber (berger et al., 2019). the task assigned by uber aims to ensure availability of drivers at peak hours at locations where there is relatively high demand. drivers are also offered extra perks to drive late hours to meet customers' requirements. both these firms celebrate the performance of drivers by asking customers to rate them and award them badges (scheiber, 2017; morschheuser et al., 2019). drivers with a higher rating are then promoted to a cadre that offers them a better chance of getting the subsequent customer. amazon offers delivery boys extra perks to deliver packages on promised dates (wood et al., 2019; çeker and özdaml, 2017). delivery rating for the delivery person is further linked to other factors like polite behavior, punctuality, damage-free deliveries, timely pickup for return, etc. firms also engage and reward teachers, trainers, and tutors for offering services to students. coursehero and chegg grants online tutors’ badges (badges denote bonus) for helping students in solving problems and finishing homework/class assignments. tutors are also placed on a leader board that accounts for the number of questions correctly answered by a gig tutor and the speed at which the answer was given. similar practices are followed by other gig firms. applied gamified principles have helped gig employers gain better control over gig workers (heeks, 2017; gleim et al., 2019). firms have reported a better engagement rate and greater motivation in their workforce (brown, 2009). moreover, reward driven gamified practices enable a balance of autonomy, freedom, and work control without impacting quality of deliverables and loss of critical resources like time and intellectual capital. gamification acts like a cue that helps in improving their experiences and motivates them to be more participative. this in turn helps firms to use gamification as a tool to combat the dark side of engagement. similarly, kumar (2013) asserted that game features like points, leader boards, badges, and avatars made employees more engaged in work and increased their productivity by boosting intrinsic and extrinsic motivation. to foster the discussion further theoretically, we now present a detail debate on the theoretical aspects of engagement. the use of theoretical lens will help in understanding the phenomenon and interlinkages between the core constructs that drive engagement and help us understand how game artifacts can trigger a positive behaviour. 2.3 social exchange theory (set) social exchange theory is based on the psychological and sociological foundation that aims to study social behavior during interactions between two parties (emerson, 1976). the theory australasian journal of information systems behl et al. 2021, vol 25, research on user involvement gamifying the gig 7 proposes that social behavior is a result of the exchange process that minimizes cost and derives the maximum benefit (blau, 1964). homans (1961; 1967) argued that individuals carried out cost-benefit analysis after weighing the risk involved in social trade. as the risk quotient grows, people tend to quit the relationship. cost versus benefit analysis mostly considers inputs such as time, money, or effort, and returns are often proxied by rewards that are either tangible or intangible. as the theory is grounded in psychological and sociological literature, its application is largely seen in human resource management (cook et al., 2013), labor relations, and service/product-based relationships between firm and customer (cropanzano & mitchell, 2005). recent studies have also discussed its application in gig work (biron & boon, 2013). social exchange is based on the pillars of self-interest and interdependence. interdependence is explained mostly by individualism as a phenomenon in the exchange process, while self-interest is often studied as a combined effect of psychological and economic needs (cropanzano et al., 2017). the application and use of social exchange theory were also proposed through twelve theoretical propositions by nye (1978). in the context of the nature of gig work, some of the theoretical propositions applicable are: a) costs and other rewards being equal, individuals choose statuses and relationships that provide the most autonomy; b) immediate outcomes being equal, they choose those alternatives that promise better longterm outcomes. thus, the theory is mostly linked to performance derived from a given set of efforts in a preferred autonomous setting. however, in the absence of rewards and benefits or in a scenario that involves high risk with lower returns, there is a stronger tendency to exit the social exchange relationship (liu et al., 2020; stanford, 2017). therefore, set principally helps in explaining the act of continuance or discontinuance by gig workers. in other words, in a social exchange relationship between gig workers and employers, gig workers’ intentions to quit are driven mainly by factors involving a higher degree of risk or work fatigue (stanford, 2017; fisher & cassady, 2019). thus, workers who see a bleak future in monetary or social rewards are likely to quit. while the social exchange theory explains how and under what circumstances it works, there is a need to examine the antecedents of gig workers' motivation through a different lens. moreover, it is essential to look at the intrinsic and extrinsic motivation of gig workers to reduce their intensity to quit (brawley, 2017; wienner et al., 2020). therefore, we explored theories of organizational behavior and found social determination theory most suitable for our current study. 2.4 self determination theory (sdt) self determination theory represents a broad framework that explains the motivation of an individual (deci & ryan, 2012). it is based mainly on the constitutional argument that an individual’s motivation has two facets intrinsic and extrinsic (deci & ryan, 2002; gagné & deci, 2005). intrinsic and extrinsic motivation are based on cognitive and social development that stems from individuals themselves. the theory also focuses on how cultural and social factors (wu et al., 2016) promote or demote an individual’s ability to take initiative, their wellbeing (cognitive and psychological), and the quality of their performance. the theory is based on the three pillars of relatedness, competence, and autonomy (deci & ryan, 2008) that fosters a sense of motivation leading to greater engagement, better performance, more persistence, and enhanced creativity. deci and ryan (2008) discuss three key elements of the theory. first, individuals are powerful beings who can master their inner self mostly by australasian journal of information systems behl et al. 2021, vol 25, research on user involvement gamifying the gig 8 keeping a check on their emotions. second, individuals have an inherent tendency to grow by contributing functionally to their work. third, optimal actions and developments do not happen automatically with individuals and usually require an external push. thus, the growth in motivation is triggered primarily by an external drive that this study proposes to establish using gamification. the use of gamification justifies the increase in motivation (intrinsic and extrinsic), engagement, performance and satisfaction, all of which are leading indicators of increase in productivity (shi &cristea, 2016; sailer et al., 2017; landers et al., 2015; van roy & zaman, 2019). the three components of the theory (autonomy, competence, and relatedness) also fall in line with the nature of the work of a gig employee. recent studies have pointed out that intrinsic motivation, which is often a result of one’s choice to opt for a gig task on a platform, results in gig workers scoring high on autonomy (lehdonvirta, 2018; poon, 2019). zuckerman et al. (1978) asserted that when given a choice, an individual selected options that they loved, which drove their intrinsic motivation. this explains the connection between intrinsic motivation and gig work. competence is another critical piece explaining motivation and is often explained by feedback (morschheuser et al., 2019). with greater positive feedback, motivation tends to be higher; creating a competitive environment also improves productivity. game elements like points and leader boards resemble factors required to enhance competence; these lead to gig employee satisfaction and enhance productivity (sepehr & head, 2013; weber et al., 2018). the last aspect that explains motivation better is relatedness. it is often used interchangeably with belongingness. it is acknowledged that choice of work and a high degree of association or belongingness are a few of the prime reasons people opt for gig work. gig work, often discussed as an extension of freelance work, offers (by its very nature) the freedom to choose the kind of work (blanchard et al., 2019; poon, 2019). online platforms also allow gig workers to post their expertise and experience to attract gig employers. moreover, the application of game artifacts in work-related tasks strongly explains attachment-motivation relations between gig employer-employees (hamari et al., 2014; huotari & hamari, 2017). we looked at the strengths and weaknesses of both the theories and found that set helped in explaining the trade-off between the efforts and resources that went into performing the task, and the rewards expected by a gig employee considering the risks involved in the transaction. while it is documented in the literature that with higher risks in job roles, there lies a higher chance for the employees to quit, it becomes interesting to understand the same behaviour in the context of gig workers. unlike full time jobs which are largely dominated by the company policies and fixed job roles, gig work has weak regulatory frameworks which causes a drop in the risk levels as well. moreover, as gig workers are in commanding position to decide to work on any work assignment or not, it becomes important to understand their mental position and decision making abilities to evaluate risk levels and stay motivated at their job. in the lack of external regulations, the power of intrinsic motivation holds larger significance which can be initiated by gamified elements. with this backdrop, it becomes important to study the motivation of gig workers from the lens of sdt and set. the next section discusses each of the proposed hypotheses. 3 hypotheses development we explore the association between organizational-level variables and their effect on individual performance in the context of an online engagement platform. huselid (1995) postulated a positive association between hrm practices adopted by firms and employee australasian journal of information systems behl et al. 2021, vol 25, research on user involvement gamifying the gig 9 performance. while the seminal study listed multiple variables that could measure employee performance, job satisfaction is one of the key drivers that have been studied often in literature (gannon et al., 2015; chang et al., 2011). locke (1976) described job satisfaction as a “positive emotional state arising from an individual’s subjective experience of their job” (p.1300). recent literature asserts that job satisfaction is quite often the driver of an employee’s stint with an organization, therefore, it becomes important to explore specific factors that could help in enhancing job satisfaction (zhang et al., 2013; gracis-chas et al., 2016). high power work systems (hpws) is reported to be such a key antecedent that has shown a positive relationship with job satisfaction in the recently published meta-analytical review (jiang et al., 2012). it is reported that employees’ intrinsic and extrinsic motivation largely drives them to perform better in high-performance situations related to jobs categorized under the hpws category (heffernan & dundon, 2016). most of the white-collar jobs listed on the gig platform require a niche skillset, and most of such jobs are classified as hpws. a gig employee needs to display a far better performance than a normal employee to be considered by the platform to carry out other gig tasks matching their skillset (fabi et al., 2015). thus, while employees are largely hooked because of job satisfaction achieved by working in an hpws engagement, it is worthwhile to examine if a similar relationship exists in an online gig work setting. therefore, we hypothesize: h1: high power work system (hpws) has a positive impact on job satisfaction (js) of whitecollar gig workers. job satisfaction is one of the key metrics to measure employee motivation. traditional human resource literature has largely used it for organizations and employees. the association of js with intentions to quit (itq) is grounded deeply in literature as there are dozens of studies that have discussed the relationship. job satisfaction is a positive driver of success for any organization while intentions to quit depict negative behavior (fabi et al., 2015; frenkel et al., 2013; behravesh et al., 2019). thus, there exists an inverse relationship between the two. the relationship is also substantiated by meta-analysis literature (gracia-chas, 2016). however, when it comes to gig employees whose association with the firm is not the same as that of a permanent employee, the link between job satisfaction and intentions to quit is not easily understood. it is noteworthy that gig employees have the option to switch multiple platforms and decide whether they want to take up more assignments on a particular online gig platform; thus, by virtue of having more options, gig employees are more empowered than permanent employees (prabowo et al., 2019; cardador et al., 2017). the autonomous nature of the work performed by gig workers also makes it easy to decide whether to work on an online platform. studies have reported that it is the online gig platforms and their engagement strategies that can keep the gig workforce engaged (gandhi et al., 2018; meijerink and keegan, 2019). holt and kennedy (2019) opined that white-collar gig workers were offered specific tasks that involved higher intellectual skills. given this higher level of required performance, it is worthwhile to understand if the relationship between job satisfaction and their intentions for gig workers. thus, we hypothesize: h2: job satisfaction (js) has a negative impact on intentions to quit (itq) of white-collar gig workers. aryee et al. (2016; 2012) demonstrated that service oriented hpws had a positive impact on employees’ service performance. most similar studies also factor in an important construct employee training organized by the firm (michaelis et al., 2015). garmendia et al., (2019) australasian journal of information systems behl et al. 2021, vol 25, research on user involvement gamifying the gig 10 mentioned that employees who possessed higher order skillsets or had a proven record of better service-related knowledge showed superior performance at the workplace. literature also supports the practice of paying higher compensation and retaining employees performing hpws tasks and whose service performance is consistently better than the average (wagner et al., 2012; guest, 2017). this can also be supported using the amo model which discusses three key components of serviceability knowledge, ability, and skills which translate into better performance (van rhee & dul, 2018). a white-collar gig employee also possesses these elements in different proportions and exhibits them according to the nature of the work. thus, the performance of a gig worker who is involved in hpws tasks is largely accounted for by the firm using different metrics. we draw insights from the amo model which postulates that service motivation enables job performance in the context of gig workers (guest, 2017; ornek et al., 2020). as part of the onboarding mechanism, gig workers have to perform tasks that are shorter duration that are often classified under hpws based on the level of skills and expertise required to perform those tasks. with such short durations, high skill work environments, and lesser gig workforce autonomy for gig employers), gig worker performance could be uncertain. it is therefore important to test the relationship between hpws and job performance in the context of the online gig workforce. we therefore postulate: h3: high-performance work system (hpws) has a positive impact on performance (pm) of white-collar gig workers. human resource studies have discussed how and why employee burnout triggers behavior and intentions to quit the job. this relationship is often explained using conservation of resources theory (hobfoll et al., 2000) which calls “to maintain the employees' well-being, by constructing and sustaining resources in such a way that employees use the available resources to guard themselves against the negative consequences of stressful experiences.” treuren (2019) based their study on this theoretical framework and explained the positive impact of fatigue and burnout on employees' intentions to leave the organization. much of existing literature also offers similar results, but mainly examines organizations and their employees. considering the work culture and engagements that an online gig platform offers (lertxundi & landeta, 2011; wu et al., 2015), burnout is perceived differently. it is of value to examine high performing gig workers’ intentions to stay (or quit). gig stems from the concept of freelance or independent work, and the idea of burnout is mostly not applicable here (jyoti & rana, 2019) due to the presence of a high degree of flexibility and autonomy for a gig worker in opting for a job. there is a greater chance of gig workers performing better if the work environment is selected by them. but the same autonomy and flexibility make the gig workspace an easy pathway to enter as well as exit (jyoti & rana, 2019; gracia-chas et al., 2014). with an increase in gig workstations and gig opportunities, gig employers need to retain high performing gig employees to run their business (cafferkey & dundon, 2015). similarly, a gig platform that encourages and empanels high performing gig workers will face lesser attrition rates; we test this conjecture and propose: h4: performance (pm) of white-collar gig workers has a negative impact on intentions to quit (itq). we referred to the seminal definition by tett and meyer (1993) conceptualizing “intentions to quit” in our study. while it is not considered a behavioural outcome, itq is still treated as a key factor contributing to a firm's performance matrix. backed by cognitive withdrawals, the “intentions to leave” construct is often ranked very high when assessing the performance of a australasian journal of information systems behl et al. 2021, vol 25, research on user involvement gamifying the gig 11 firm (fugate et al., 2012). kundu and gahlawat (2016) postulated that in countries where there is greater competition due to the availability of a similarly skilled workforce, intention to quit is an important marker in terms of defining competitive advantage. companies with happy employees and workforce witness lesser intentions to quit. we explored extant literature for enablers and barriers to itq, and out of multiple prominent factors, hpws was recorded as a critical barrier (kehoe and wright, 2013; boon et al., 2011). a study by garcía-chas et al. (2014) claimed that high-performance work practices like rewards and employee participation, result-oriented appraisal, job security, extensive training, and rigorous staffing had a negative impact on itq. thus, an employee rated highly on the parameters of hpws would be less likely to leave the organization. while there are other factors like performance, procedural justice, intrinsic motivation, etc. that control the overall nature of itq (gkorezis et al., 2018), the underlying relationship does not change. it is reported that hr systems and staffing rules adopted in gig employment are different. however, the nature of gig employment makes it difficult for the firm to retain the gig worker. it is reported that gig workers continue to work with gig employers because of their needs which may be temporary or permanent (behravesh et al., 2019). other reasons to continue could include nature of job, work culture of the organization, and competition in the market. in recent years, firms have incorporated hpws to manage gig employees. there are limited studies examining whether hpws would have a negative effect on intentions to quit. thus, drawing from what is known from published literature, we hypothesize: h5: high-performance work system (hpws) has a negative impact on intentions to quit (itq) of white-collar gig workers. cardador et al. (2017) discussed the importance of work gamification in conceptualizing how game mechanics could help improve performance of employees (de marcos et al., 2014; hamari, 2013). performance management of employees is primarily driven by their level of intrinsic motivation. the hook offered by game mechanics helps generate interest among workers in the short term (von roy & zaman, 2019). game elements like points, badges and leader boards enhance employee performance by offering an informative and effective mechanism to be connected with their task (cardador et al., 2017). gamification can be applied to both offer feedback and intrinsically motivate employees. feedback is provided as encouragement as well as suggestions for improvement. most gamification-based studies have discussed how game elements positively influence intrinsic motivation, which boosts productivity (haivas et al., 2013; mekler et al., 2017; xi & hamari, 2019). gamified applications work better in situations of unpredictability and scarcity, both of which are elements of the octalysis framework (chou, 2019). while studies have primarily claimed that gig workers possess a high degree of freedom, the dark side of gig employer-gig employee engagement highlights issues related to discipline and regularity in work regimes. thus, with freedom, there comes a degree of risk in this engagement that needs to be managed using appropriate engagement mechanisms (shatz, 2015). the gamified platform would help them gain higher job satisfaction. it is important to examine competitiveness in situations where there is no interaction between participants, though similar jobs are being performed (morschheuser et al., 2019; kumar, 2013). firms like ola, uber, zomato, etc. use game mechanics driven engagement approaches to acquire and retain drivers and delivery agents. one of the main shortcomings of whitecollar gig work is the nature of work and its specific demands. unlike blue-collar jobs with australasian journal of information systems behl et al. 2021, vol 25, research on user involvement gamifying the gig 12 firms like uber, white-collar gig work demands a different category of craft, which at times, is rare (holt & kennedy, 2019). also, gig workers may or may not at times agree to the compensation offered. therefore, firms need tangible reward-based mechanisms to retain such workers. competitive strategies need to be used to maintain a strategic advantage; gamification is one such strategy. we, therefore, base our arguments on both the perspectives and propose to explore the relationship between hpws and performance, and hpws and job satisfaction of gig workers. another prime reason for examining the effect of gamification on job satisfaction and performance is that both are antecedents that have a negative relationship with “intentions to quit” (redondo et al., 2019; masood et al., 2020). therefore, while assessing the link between performance and job satisfaction of gig workers, we observe the role of gamification as moderator. the study proposes: h6a: gamification (gm) has a positive moderating effect on the relationship between highperformance work systems (hpws) and job satisfaction (js). h6b: gamification (gm) has a positive moderating effect on the relationship between highperformance work systems (hpws) and performance (pm). the study offers new dimensions to explain motivation, performance, and job satisfaction of gig workers in the context of gamified online platforms engagements and reduce attrition. the proposed theoretical model with hypotheses is shown in figure 1. figure 1. theoretical model 4 research design we collected primary data to test the proposed research hypotheses. primary data was collected using a structured questionnaire pre-tested from 12 academicians each having more than five years of experience and have published their work in a related area of study. seven human resource professionals working with firms in the online gig workspace was invited to pre-test the questionnaire. suggestions received from both groups were used to modify the questionnaire in terms of length, positioning of questions, and wording. the final questionnaire was then used for data collection. data were collected from gig workers working australasian journal of information systems behl et al. 2021, vol 25, research on user involvement gamifying the gig 13 with online platforms operational in primarily asian countries. we now discuss the operationalization of constructs as it is a critical step in designing a psychometrically sound survey instrument. 4.1 research instrument and operationalization of the constructs we designed the initial questionnaire on the basis of existing scales for each of the constructs. scale adoption was discussed with renowned academics who had published empirical research using one or more of the constructs used in the study. the panel of academics was contacted over a group video call and based on the unanimous decision made by the panel, the research instrument was finalized. we also approached research scholars and faculty members working in a multi-disciplinary yet related field of work to ask for feedback on the questionnaire. on the basis of our discussions with all knowledge sources, we revised the questionnaire in terms of flow of questions, selection of items, and choice of language. this helped in gaining clarity and removing ambiguity, thus improving validity and maintaining parsimony in the process of iterative feedback. each of the constructs was drawn from existing literature and modified to the context of the present study. the process of adaptation and finalization of items for each construct was further discussed and approved by a panel of experts. all the constructs used in the study are reflective. hpws was measured using the scale adapted from sun et al. (2007). this scale uses eight subscales to assess the eight core hr practices. for employment security,” “clear job description,” “extensive training,” “participation”, “results-oriented appraisal,” and “incentive reward” subscales, the original number of items was kept. we dropped “selective staffing” and “internal mobility,” and for subscales, the number was reduced to three items per subscale. consequently, the resulting scale comprised 18 items. job satisfaction was measured with a three-item scale adapted by lee and bruvold (2003) from quinn and staines (1979) assumed to generate an overall measure of job satisfaction. job performance was measured using two sub scales measuring role behavior and task performance. the scale for role behavior was adapted from williams and anderson (1991) while that for task performance from bott et al. (2003). we further referred to ang et al. (2013) to adapt the scale for “intention to quit” which had 3 items. lastly, for gamification, we adapted the scale from behl and dutta (2020) with 9 items. 4.2 data collection we searched the archival history of websites using waybackmachine (https://archive.org/web/) to list down the tasks listed in the past two years (january 2018december 2019). the rationale for using an archival data is to understand the overview of tasks created on different crowdsourcing websites and the degree and skills of engagement required by gig workers. archival data also helps in understanding the changing needs and requirements posted on the portals across different geographies. however, studies have also reported that data validation is key for archival data to improve the robustness of the results (lee et al., 2018). we then contacted website owners seeking permission to send a questionnaire to the participants in these tasks. the preliminary search gave us a list of 87 websites that hosted gig work and after scrutinizing them on the basis of rate of completion of tasks, we eliminated the ones whose success rate was less than 30% (ye and kankanhalli, 2017). we reached a total of 34 websites (refer to appendix a) that had hosted “gig work” in the given period. a total of 1567 respondents were contacted out of which 374 respondents filled the survey. we sent repeated reminders to respondents to fill the questionnaire. the respondents confirmed that they had contributed to listed tasks on the given website, which australasian journal of information systems behl et al. 2021, vol 25, research on user involvement gamifying the gig 14 was validated by task creators and website managers. the three-level validation helped us to reduce self-reporting bias in our data. respondents participated in one or more tasks hosted on one or more websites. the tasks ranged from higher degree of creativity (including designing, photography, writing, music, etc.) to lower degree of creativity (data entry, data collection, making cold calls, etc.). most of the respondents were relatively young (average age 23.4 years) and from tier 1 cities. we ensured that in our final dataset, tasks creators and task solvers belonged to different nationalities. this helped us control the data and reduce the chances of bias during the data collection process. as gig workers were paid in their local currency, domicile effect was controlled during data collection. we further tested non-response bias by following guidelines by armstrong and overton (1977). we performed a t-test to compare the performance attributes (like average time spent on gig work) of respondents and non-respondents. test results reported no significant difference between the two categories (p>0.05). the profile of the respondents is tabulated in table 1. the next section discusses the tools used to test hypotheses, key results of the analysis performed, and the rationale behind the procedure followed at each step. demographic variables classification no. of respondents number of gig work done in the past 24 months 0 to 5 156 5 to 10 87 10 to 15 46 15 to 20 31 more than 20 54 average time spent per task less than 6 hours 165 6 to 12 hours 97 12 to 18 hours 63 18 to 24 hours 40 more than 24 hours 9 average money earned per task less than $ 250 201 $250$750 126 $750$1250 41 more than $ 1250 6 gender male 188 female 185 transgender 1 table 1. demographic profile of respondents 5 data analysis and results 5.1 common method bias (cmb) we collected data using a single source (survey-based questionnaire) which might result in common method bias (lindell and whitney, 2001). ketokivi and schroeder (2004) explained australasian journal of information systems behl et al. 2021, vol 25, research on user involvement gamifying the gig 15 that in order to resolve the issue of common method bias in survey-based research, one needs to use multiple informants for each of the observational units. some of the key reasons reported by podsakoff et al. (2003) for common method bias include social desirability and consistency motif. therefore, we followed the guidelines of guide and ketokivi (2015) to minimize the chances of committing common method bias. as the first step, we requested the respondents to refer to the details of the deliverables shared with them before accepting the terms and conditions to perform the gig work. based on the data and information, the respondents were asked to answer the questions. we also performed harman’s single factor test and found that maximum covariance explained was 36.18% which was lesser than the threshold (50%), thereby confirming that the data did not suffer from common method bias. referring again to lindell and whitney’s (2001) guidelines, we performed a marker variable test by introducing a marker variable that was theoretically unrelated to the dependent variable. we found a low correlation between the categories of marker variable (different tiers of cities of residence of respondents) and independent and dependent variables. as the last step, we referred to model wide collinearity assessment proposed by kock (2015). this test acts like a litmus test that helps measure the extent to which the measurement instrument can establish generalizability of results. variance inflation factors (vif’s) for each of the constructs reported values less than the threshold value of 3.5. the battery of tests confirmed that the data did not suffer from cmb. 5.2 validation of measurement instrument we used partial least square (pls) structured equation modelling (sem) as compared to covariance-based sem. henseler et al. (2015) point out a key advantage of using pls-sem estimates are more generic and less affected by model misspecification in some subparts of the model (dubey et al., 2019). it is also known that pls-sem is better suited for exploratory models and theory development and has a higher degree of relaxations related to the normality of data (chin, 1998; gefen & straub, 2005). it is known that pls-sem is used when the objective is predicting target constructs and the structural model has purely formative constructs. also, hair et al. (2011) and hair et al. (2017) confirmed that it is advisable to use pls-sem when the research is exploratory or an extension of existing theoretical structure unlike in covariance based sem (cb-sem) , that works on the principle of theory testing. the present study fits into the framework of theory integration and exploratory in nature, thereby making pls-sem a better choice to be used in this case. to test the hypotheses, we used warppls 6.0 software. we also drew inspiration and guidance from peng and lai (2012) who argued that pls works on prediction orientation and helps in assessing the predictability of the exogenous variable. we examine how “intentions to quit” can be explained by a set of antecedents, thus, it is vital to calculate each antecedent’s explanatory power. moreover, in the context of the gig economy, the rate of dropouts and underlying intentions are not explained by any established theory. this makes pls based modeling more appropriate and suitable in the given context (henseler et al., 2015). the proposed model is complex, thus, to calculate the estimations for each of the predictor variables, we used a two-step approach proposed by peng and lai (2012). the first step evaluates the reliability and validity of the measurement model, followed by analysis of the structural model in the next step. we now discuss both the steps in detail. australasian journal of information systems behl et al. 2021, vol 25, research on user involvement gamifying the gig 16 5.3 assessment of measurement model table 2. loading of indicator variables, scr and ave construct items factor loading variance error scale composite reliability (scr) average variance explained (ave) hpws hpws1 0.81 0.66 0.34 0.87 0.61 hpws2 0.73 0.53 0.47 hpws3 0.85 0.72 0.28 hpws4 0.72 0.52 0.48 hpws5 0.88 0.77 0.23 hpws6 0.71 0.50 0.50 hpws7 0.80 0.64 0.36 hpws8 0.68 0.46 0.54 hpws9 0.77 0.59 0.41 hpws10 0.70 0.49 0.51 hpws11 0.78 0.61 0.39 hpws12 0.84 0.71 0.39 hpws13 0.74 0.55 0.45 hpws14 0.69 0.48 0.52 hpws15 0.88 0.77 0.23 hpws16 0.82 0.67 0.33 hpws17 0.75 0.56 0.44 hpws18 0.89 0.79 0.21 js js1 0.77 0.59 0.41 0.81 0.68 js2 0.81 0.66 0.34 js3 0.89 0.79 0.21 pm pm1 0.75 0.56 0.44 0.82 0.62 pm2 0.79 0.62 0.38 pm3 0.87 0.76 0.24 pm4 0.74 0.55 0.45 pm5 0.71 0.50 0.50 pm6 0.78 0.61 0.39 pm7 0.84 0.71 0.29 pm8 0.82 0.67 0.33 pm9 0.77 0.59 0.41 itq itq1 0.88 0.77 0.23 0.78 0.65 itq2 0.81 0.66 0.34 itq3 0.72 0.52 0.48 gm gm1 0.70 0.49 0.52 0.83 0.62 gm2 0.84 0.71 0.29 gm3 0.74 0.55 0.45 gm4 0.76 0.58 0.42 gm5 0.81 0.66 0.34 gm6 0.84 0.71 0.29 gm7 0.85 0.72 0.28 gm8 0.72 0.52 0.48 gm9 0.79 0.62 0.38 australasian journal of information systems behl et al. 2021, vol 25, research on user involvement gamifying the gig 17 we checked for individual convergent validity and reliability along with factor loadings. results reveal that cr values are above the recommended threshold value of 0.7 (hair and hult, 2017) which confirms construct reliability. to test convergence validity, we tested the item loadings and found them to be more than 0.7 (hair and hult, 2017) and average variance extracted to be more than 0.50 (fornell and larcker, 1981). as a follow up step to test for discriminant validity, we calculated heterotrait-monotrait (htmt) ratio of correlations with fixed cut offs and inferential tests. following the guidelines of henseler et al. (2015), we observed that htmt ratios are found to be greater than 0.85 and are significantly different from 1 (franke and sarstedt, 2019) thereby confirming discriminant validity in our data. as suggested by hair et al. (2018), we also referred to the upper bound of the 95% of the ci of htmt and results (<0.85) confirmed disticnct nature of the constructs. table 2 reports that measurement item loadings obtained from a confirmatory factor analysis of their respective constructs were above the suggested minimum threshold of 0.50 (hair et al., 2010), indicating that the items had acceptable convergent validity. next, we tested for multicollinearity in our data by performing a full collinearity test that confirmed that all the values of vif for each of the constructs were greater than 5 (hair et al., 2014). we applied bera and jarque's (1981) test to check for multivariate normality and found that two constructs were not normally distributed. however, as this is not a necessary condition for applying pls-sem, we continued using pls-sem for further data analysis. hpws js pm itq gm hpws 0.72 js 0.55 0.88 pm 0.68 0.43 0.79 itq -0.11 -0.05 -0.03 0.82 gm 0.32 0.57 0.54 0.38 0.91 note: hpws: high performance work systems; js: job satisfaction; pm: performance; itq: intention to quit; gm: gamification; square root of average variance extracted (ave) highlighted in bold across the diagonal. table 3. correlation among constructs 5.4 test for endogeneity the last step performed before testing the hypotheses was testing for endogeneity for the exogenous variable in the model. for example, in our theoretical model, high-performance work systems (hpws) are considered exogenous with respect to other constructs in the model. thus, endogeneity is not a concern in our data. further, we applied durbin-wuhausman to our dataset (davidson & mackinnon, 1993). the test involved regressing a high-performance work system on all moderating and control variables. we observed that the residual of all parameter estimates was non-significant which indicates that hpws is exogenous in our model. the results fall in line with the initial conceptualization and operationalization of the construct. 5.5 structural model assessment we used bootstrapping resampling method with 150 resamples to test the structural model. we then tested for the structural model’s quality by calculating and reporting the model fit australasian journal of information systems behl et al. 2021, vol 25, research on user involvement gamifying the gig 18 indices shown in table 4. the table reveals that all the indices were statistically significant and exceeded the respective thresholds as indicated in literature. we can, therefore, conclude the adequacy of our structural model. index value sig value (acceptable range wherever applicable) average path coefficient (apc) 0.311 p < 0.01(rosenthal and rosnow, 1991) average r square (ars) 0.399 p < 0.01(rosenthal and rosnow, 1991) average adjusted r square (aars) 0.376 p < 0.01(rosenthal and rosnow, 1991) average block vif (avif) 2.672 acceptable if ≤ 5; ideally ≤ 3.3 (kock, 2015) tenenhaus goodness of fit (gof) 0.489 large ≥ 0.36; medium ≥ 0.25; small ≥ 0.1(tenenhaus et al., 2005) table 4. model fit indices we also performed a multi group analysis (mga) using measurement invariance of composite models(micom) approach (henseler et al., 2016). we followed a three-step approach wherein we calculated configural invariance to begin with, followed up with compositional variance and lastly evaluating equality of composite means and variances between groups. henseler’s mga technique uses p-value difference between path coefficients that are either higher than 0.95 or lower than 0.05 thereby confirming that the groups are statistically different. we performed the steps for all the hypotheses with groups like gender, frequency of gig work, time invested, and money earned. the results reveal that there is no significant different between men and women (we did not test it for transgender), between four levels of frequency of engagement of gig workers, however there was a significant difference found between groups for time invested on an average to perform tasks and the amount earned. the probable reasons for the same could be dependent on the type and complexity of tasks which results in a gig worker investing more time and earning more money. brawley (2017) also confirmed that the reason for this could also be dependent on the platform and its operational requirements as well. 5.6 results of hypotheses testing figure 2 presents the results of the proposed model estimates, e.g., the standardized path coefficients associated with each proposed hypothesis, the significance of the path coefficients, and the variance explained (r²) by the independent variables. we also summarize the results in table 5 which shows the results of the tested hypotheses. all proposed hypotheses were supported except the relationship between hpws and itq. results show that the independent variable in the model i.e., “intentions to quit” explains 73.2 % variance, while job satisfaction and productivity explain 43.4% and 35.8% variance, respectively. results confirm a significantly positive impact of hpws on the job performance of gig workers (h1) (β = 0.42, p <0.001), and significant positive impact of hpws on the productivity of gig workers (h3) (β = 0.56, p <0.00). the last dimension that explains the phenomenon of dropping out of the online platforms is hpws, wherein we found counter-intuitive results(h5). referring to the results (β = 0.014), we confirm that the functions under hpws are either functionally or administratively different, and do not negatively impact gig workers’ motivations. reflecting upon the underlying cost-benefit analysis framework drawn using social exchange theory, the phenomenon of disengaging with the online platform has mixed results. we then tested if gamification could help improve upon the two prominent antecedents job satisfaction and australasian journal of information systems behl et al. 2021, vol 25, research on user involvement gamifying the gig 19 productivity of gig workers. as discussed in literature, game artifacts induce intrinsic and extrinsic motivation among people. its importance and relevance become greater in the context of an unregulated and autonomous environment. both hypotheses h6a and h6b – were supported and a significantly positive moderating relationship was found between hpws and jp, as well as hpws and pm. results also help in explaining the worker’s motivation from the lens of sdt. a highly motivated and performance-driven worker is less likely to quit, and results support the proposition. figure 2. final sem model hypothesis effect of effect on β p-value result h1 hpws js 0.42 *** supported h2 js itq -0.39 *** supported h3 hpws pm 0.56 *** supported h4 pm itq -0.28 *** supported h5 hpws itq 0.014 * not supported h6a hpws x gm js 0.55 *** supported h6b hpws x gm pm 0.68 *** supported note: hpws: high-performance work systems; js: job satisfaction; pm: performance; itq: intention to quit; gm: gamification *** denotes p< 0.01; * denotes p> 0.1 table 5. structural estimates we examined the model's exploratory power by calculating the explained variance of the endogenous constructs. this method is in line with the objectives of pls that aim to maximize the variance explained by the endogenous variable. r² values obtained were 0.732 for itq, 0.434 for js, and 0.358 for pm which are moderately strong (chin, 1998). next, we calculated r² =0.434 r² =0.732 r² =0.358 high performance work system (hpws) job satisfaction (js) intentions to quit (itq) performance (pm) β= 0.42 β = 0.39 β =0.56 β = 0.28 β = 0.014 gamification (gm) β =0.55 β =0.68 australasian journal of information systems behl et al. 2021, vol 25, research on user involvement gamifying the gig 20 the effect size of each predictor using cohen f² (cohen, 1998). f² is equal to the increase in r² relative to the proportion of variance that remains unexplained in the endogenous latent variable (dubey et al., 2019). following the guidelines laid by cohen (1998), we found that f² for all the relationships was higher than 0.35, which concludes that f² is considerably large. as the last step, we calculated the model’s capability to predict for which we calculated stonegeisser’s q² for all the endogenous variables. peng and lai (2012) stated that the model should consider having acceptable predictive relevance if the values are greater than zero. we found q² for js, pm, and itq and found their values to be 0.657, 0.592, and 0.603, respectively. the results of hypothesis testing can be seen in table 5. 6 discussion of results the study offers exciting results in the context of the gig sector. the study provides strong insights into how to engage gig workers and control turnover in the sector. we drive our discussions on the basis of social exchange theory to explain how firms and employees work in tandem, weighing opportunities and rewards against threats and risks involved in tasks (firth et al., 2004; zito et al., 2018). studies have examined the application of set in various industries and used it to determine reasons behind employees staying or leaving the organization. moreover, unlike the much-discussed employer-employee relationship with respect to social exchange theory, there lies a gap in the context of gig employee-employer relationship that is fundamentally different from the former. while reflecting upon the nuances and functions of the gig work arrangement, it is important to add another supporting theoretical lens to understand its functions better. we draw insights from sdt which helps in explaining the positive motivation that a gig worker experiences by doing gig work, which in turn accelerates work efficiency, boosts motivation and reduces dropout rate (wagner et al., 2012; frenkel et al., 2013). we discuss the theoretical and managerial contributions of the study in this section. 6.1 theoretical implications the study uses supporting arguments drawn from set and sdt in the context of understanding the motivations and intentions of gig workers, which is a unique contribution of the study. results, therefore, broadly support underlying arguments drawn from social exchange theory. our research confirms that gig workers who are highly satisfied and productive are less likely to quit the platform. h2 and h4 showed negative relationship between job performance and intentions to quit, and productivity and intentions to quit, respectively. considering the coefficient of h2 and h4, we can conclude that while there exists a lower chance of gig workers exiting the online platform, it is essential to seek a way to further reduce the chances of exiting the system. first, the study supports that hpws has a significant positive impact on job satisfaction and productivity of gig workers (h1). although similar arguments have been made in literature (gracia-chas et al., 2016; fabi et al., 2015), the contextual application in the gig space adds a new dimension. the autonomous and partially regulated work environment further supports the idea that gig employees face lesser burnouts and mental fatigue. the online engagement of the blue-collar gig workforce primarily driven by a higher level of creativity and mental ability gives the workforce the independence to choose work assignments complementing their competence (lertxundi & landeta, 2011; gkorezis et al., 2018). additionally, freedom in work adds to their productivity. second, against the backdrop of the existing negative relationship between hpws and intentions to quit (fabi et al., 2015), we posed a counter-intuitive argument (h5). results confirm that there australasian journal of information systems behl et al. 2021, vol 25, research on user involvement gamifying the gig 21 is no significant relationship between dependent and independent variables. heffernan and dundon (2016) also presented a similar non-significant relationship. instead of the work and the working attributes of a gig worker, the study adds a new dimension. in the context of gig platforms, interaction between employees and employers is minor as compared to that in fulltime employment. thus, high-performance work systems that involve selection, training, performance appraisal, and compensation do not apply to a large extent in the gig economy (jabagi et al., 2019). for example, the selection process is like an agreement based on resource valuations discussed between employer and employee. thus, the relationship between the hpws and intention to quit or intentions to switch is grey in this case (stewart & stanford, 2017). the third theoretical contribution is the role of gamification as moderator which motivates gig employees to a higher degree (h6a and h6b). this phenomenon is elucidated mainly by the principles of sdt that help explain intrinsic and extrinsic motivations of the gig worker. as gig workers do not operate in a regulated and monitored physical and administrative environment, the degree of autonomy at times has a counterproductive effect, resulting in late delivery or substandard quality (mekler et al., 2017; brawley, 2017). gamification shows a positive and significant impact on the relationship between hpws and job satisfaction, as well as between hpws and productivity (h6a and h6b). just as firms use multiple activities under hpws to ensure high employee efficiency, the same is the case of a gig worker. the use of game elements in critical hr functions (like compensations, bonuses, training and development, etc.) helps in reinforcing engagement with the online platform and the work involved. results also support the conceptual model proposed by xi and hamari (2019) on gamified work engagements. 6.2 practical implications the study makes some significant contributions to existing approaches and ways of engaging gig workers on online platforms. the unregulated character of the gig sector makes it explicitly hard for both organizations and employees to have a sustainable relationship with each other. the organizations are confronted with the challenges of building capabilities to adapt to this new work idea. employees find it correspondingly complex to manage challenges of lack of job structure and financial anxiety, leading to decreased motivation. hence, managing employees in alternative work structures such as the gig economy requires new strategies. the results established that introducing game-based engagements would boost the motivation of gig employees, resulting in higher productivity and job satisfaction. these insights can be applied for both white and blue-collar gig workers (huang et al., 2020). this becomes more critical when, in the future, we will witness a growth of gig work. furthermore, in uncertain and challenging times like the one seen during the current global pandemic of covid 19, gig work is expected to gain more relevance and hence the existing hrm practices are likely to gain more attention and expected to evolve specific to the dynamic requirements of new work structures. in proposing the use of gamified rewards by gig platforms in the form of lucrative deals/offers, the current study offers practitioners with a tangible, innovative strategy that can motivate gig employees and reduce attrition. our results would also appeal to firms in the process of moving some of their regular processes to gig platforms. the study will help them with a requisite design for gig employee engagement to achieve the sought after goals (kumar, 2013; burtch et al., 2018). scouting for and retaining productive and talented white-collar workers could be bolstered by game mechanics. australasian journal of information systems behl et al. 2021, vol 25, research on user involvement gamifying the gig 22 organizations can link point-driven leader boards, trophies, and accolades to job-related performance metrics, which will further drive commitment and help classify and retain productive and talented white-collar workers. the classification through this retention mechanism will help firms understand the expectations and working styles of workers leading to more effective task allotment. use of gamification can yield expected results if designed, keeping in mind the participant types and motivations that drive the players. organizations can analyse and aim to align specific gamification mechanics like rewards and challenges with a particular employee type for desired outcomes. the playfulness aspect associated with gamification can help organisations build a positive work environment for gig workers, thus reducing work-related stress leading to greater productivity. our results will also aid gig workers by helping them resolve administrative issues and cope with work stress and getting rewards over and above their compensation levels; findings could also provide inputs to strengthen brand loyalty (hsu & chen, 2018). a loyal gig employee could lead to an equally minded workforce, and gamification could then be used in the referral scheme of the website. 6.4 future scope of work the study considers only white-collar gig workers enrolled with online platforms, while the population of blue-collar gig workers is large, a comparable study needs to be conducted to explore the differences. while the study did explore online engagement and why workers quit , the study did not test for all scenarios that could make workers quit. the study can be further extended to understand how gig work is managed across different work disciplines and cultures. moreover, studies could use simulations or experiment-based approaches to determine which game artifacts lead to a higher engagement rate. following the guidelines of morschheuset et al. (2019) and feng et al. (2018), we propose to design websites where different game artifacts can be used to determine the reaction and response of gig workers with similar tasks. additionally, the tasks can also be changed in terms of its rigor, time and monetary rewards to answer three key questions: when, how and which game artifacts would be useful for controlling/reducing the chances of quitting. a mixed-method approach to understanding the reasons for gig workers quitting needs to be conducted, followed by analysis of drivers using longitudinal data. the study calls for an examination of circumstances under which gig workers would be more likely to continue, and that of those under which gig workers would be more likely to quit. similarly, studies could examine ways to improve gig worker retention by exploring this area using a grounded theory approach and a case-based approach as suggested by jabagi et al. (2019). future studies can be developed using ketokivi and choi(2014) guidelines to understand the phenomenon of the functioning of game elements in hpws. there appears to be a need for an appropriate engagement plan that could be customized and flexible so that they can be revised as and when required in different situations. an extension of this study could involve further exploration in the fields of psychology, human resources and law with respect to gig work challenges. 7 conclusion the study explores a significant phenomenon within the emerging sector of gig economy– intentions of workers to quit gig work on online platforms. the study formulates hypotheses using social exchange theory and self-determination theory and proposed that the dark side australasian journal of information systems behl et al. 2021, vol 25, research on user involvement gamifying the gig 23 of gig work can be resolved by working on ways to reduce the rate of dropouts of gig workers using gamification. results confirm that high-performance work systems help gig workers achieve a higher degree of productivity and satisfaction . while these attributes are essential, the sector still witnesses significant dropouts. results suggest that by introducing gamified online platforms, we can improve work productivity, job satisfaction, and engagement of gig workers; this, in turn, could improve gig workforce retention. the nature and complexity of work performed by white collared gig workers makes their availability low on such platforms and therefore gig platforms often empanel them in order to draw more benefits from their services. the study offers mechanisms that would help online platforms retain the gig workforce. references abraham, k. g., haltiwanger, j. c., sandusky, k., & spletzer, j. r. (2018). measuring the gig economy: current knowledge and open issues (no. w24950). national bureau of economic research. cambridge, ma. ang, s. h., bartram, t., mcneil, n., leggat, s. g., & stanton, p. (2013). the effects of highperformance work systems on hospital employees' work attitudes and intention to leave: a multi-level and occupational group analysis. the international journal of human resource management, 24(16), 3086-3114. anwar, m. a., & graham, m. (2020). between a rock and a hard place: freedom, flexibility, precarity and vulnerability in the gig economy in africa. competition & change. doi: https://doi.org/10.1177/1024529420914473 armstrong, j. s., & overton, t. s. (1977). estimating nonresponse bias in mail surveys. journal of marketing research, 14(3), 396-402. aryee, s., walumbwa, f. o., seidu, e. y., & otaye, l. e. (2012). impact of high-performance work systems on individual-and branch-level performance: test of a multilevel model of intermediate linkages. journal of applied psychology, 97(2), 287-300. aryee, s., walumbwa, f. o., seidu, e. y., & otaye, l. e. (2016). developing and leveraging human capital resource to promote service quality: testing a theory of performance. journal of management, 42(2), 480-499. behl, a. (2020). antecedents to firm performance and competitiveness using the lens of big data analytics: a cross-cultural study. management decision. behl, a., & dutta, p. (2020). engaging donors on crowdfunding platform in disaster relief operations (dro) using gamification: a civic voluntary model (cvm) approach. international journal of information management, 54, 102140. doi: https://doi.org/10.1016/j.ijinfomgt.2020.102140 behravesh, e., tanova, c., & abubakar, a. m. (2020). do high-performance work systems always help to retain employees or is there a dark side? the service industries journal, 40(11-12), 825-845. bera, a. k., &jarque, c. m. (1981). efficient tests for normality, homoscedasticity and serial independence of regression residuals: monte carlo evidence. economics letters, 7(4), 313318. australasian journal of information systems behl et al. 2021, vol 25, research on user involvement gamifying the gig 24 berger, t., frey, c. b., levin, g., &danda, s. r. (2019). uber happy? work and well-being in the ‘gig economy’. economic policy, 34(99), 429-477. biron, m., & boon, c. (2013). performance and turnover intentions: a social exchange perspective. journal of managerial psychology, 28(5), 511-531. blanchard, c., baker, a., perreault, d., mask, l., & tremblay, m. (2019). the importance of keeping employees satisfied. journal of health organization and management, 34(1), 23-39. blau, p.m. (1964), social exchange theory. new york: john wiley & sons. boon, c., den hartog, d. n., boselie, p., & paauwe, j. (2011). the relationship between perceptions of hr practices and employee outcomes: examining the role of person– organisation and person–job fit. the international journal of human resource management, 22(01), 138-162. bott, j. p., svyantek, d. j., goodman, s. a., & bernal, d. s. (2003). expanding the performance domain: who says nice guys finish last? international journal of organizational analysis, 11(2), 137-152. brawley, a. m. (2017). the big, gig picture: we can't assume the same constructs matter. industrial and organizational psychology, 10(4), 687-696. brown, t. (2009). the gig economy. the daily beast. accessed on 12 march, 2020 at https://www.thedailybeast.com/articles/2009/01/12/the-gigeconomy.html. burtch, g., carnahan, s., & greenwood, b. n. (2018). can you gig it? an empirical examination of the gig economy and entrepreneurial activity. management science, 64(12), 5497-5520. cafferkey, k., &dundon, t. (2015). explaining the black box: hpws and organisational climate. personnel review, 44(5), 666-688. cardador, m. t., northcraft, g. b., &whicker, j. (2017). a theory of work gamification: something old, something new, something borrowed, something cool?. human resource management review, 27(2), 353-365. carr, c. t., hall, r. d., mason, a. j., & varney, e. j. (2017). cueing employability in the gig economy: effects of task-relevant and task-irrelevant information on fiverr. management communication quarterly, 31(3), 409-428. çeker, e., & özdaml, f. (2017). what" gamification" is and what it's not. european journal of contemporary education, 6(2), 221-228. chang, p. c., & chen, s. j. (2011). crossing the level of employee's performance: hpws, affective commitment, human capital, and employee job performance in professional service organizations. the international journal of human resource management, 22(04), 883-901. chin, w. w. (1998). the partial least squares approach to structural equation modeling. modern methods for business research, 295(2), 295-336. chou, y. k. (2019). actionable gamification: beyond points, badges, and leaderboards. packt publishing ltd. cohen, j. e. (1988). statistical power analysis for the behavioral sciences. hillsdale, nj: lawrence erlbaum associates, inc. australasian journal of information systems behl et al. 2021, vol 25, research on user involvement gamifying the gig 25 connelly, c. e., fieseler, c., černe, m., giessner, s. r., & wong, s. i. (2020). working in the digitized economy: hrm theory & practice. human resource management review, 100762. cook, k. s., cheshire, c., rice, e. r., & nakagawa, s. (2013). social exchange theory. in handbook of social psychology. springer, dordrecht. cropanzano, r., & mitchell, m. s. (2005). social exchange theory: an interdisciplinary review. journal of management, 31(6), 874-900. cropanzano, r., anthony, e. l., daniels, s. r., & hall, a. v. (2017). social exchange theory: a critical review with theoretical remedies. academy of management annals, 11(1), 479-516. dale, s. (2014). gamification: making work fun or making fun of work? business information review, 31(2), 82-90. davidson, r., & mackinnon, j. g. (1993). estimation and inference in econometrics. oup catalogue, london, uk. deci, e. l., & ryan, r. m. (2002). overview of self-determination theory: an organismic dialectical perspective. handbook of self-determination research, 3-33. deci, e. l., & ryan, r. m. (2008). self-determination theory: a macrotheory of human motivation, development, and health. canadian psychology/psychologie canadienne, 49(3), 182. deterding, s. (2019). gamification in management: between choice architecture and humanistic design. journal of management inquiry, 28(2), 131-136. dubey, r., gunasekaran, a., childe, s. j., bryde, d. j., giannakis, m., foropon, c., ... & hazen, b. t. (2020). big data analytics and artificial intelligence pathway to operational performance under the effects of entrepreneurial orientation and environmental dynamism: a study of manufacturing organisations. international journal of production economics, 107599. doi: https://doi.org/10.1016/j.ijpe.2019.107599 gillespie, p. (2017), “intuit: gig economy is 34% of us workforce”, available at: http://money.cnn.com/2017/05/24/news/economy/gig-economy-intuit/index.html (accessed may 10, 2020). emerson, r. m. (1976). social exchange theory. annual review of sociology, 2(1), 335-362. fabi, b., lacoursière, r., & raymond, l. (2015). impact of high-performance work systems on job satisfaction, organizational commitment, and intention to quit in canadian organizations. international journal of manpower, 36(5), 772-790. faisal, a. l. f., sucahyo, y. g., ruldeviyani, y., & gandhi, a. (2019, july). discovering indonesian digital workers in online gig economy platforms. in 2019 international conference on information and communications technology (icoiact), 554-559. ieee. feng, y., ye, h. j., yu, y., yang, c., & cui, t. (2018). gamification artifacts and crowdsourcing participation: examining the mediating role of intrinsic motivations. computers in human behavior, 81, 124-136. firth, l., mellor, d. j., moore, k. a., &loquet, c. (2004). how can managers reduce employee intention to quit?. journal of managerial psychology, 19(2), 170-187. http://money.cnn.com/2017/05/24/news/economy/gig-economy-intuit/index.html australasian journal of information systems behl et al. 2021, vol 25, research on user involvement gamifying the gig 26 fisher, s. l., & cassady, e. a. (2019). use of relational ehrm tools in gig worker platforms. in hrm 4.0 for human-centered organizations. emerald publishing limited, uk. fornell, c., & larcker, d. f. (1981). evaluating structural equation models with unobservable variables and measurement error. journal of marketing research, 18(1), 39-50. frenkel, s., sanders, k., & bednall, t. (2013). employee perceptions of management relations as influences on job satisfaction and quit intentions. asia pacific journal of management, 30(1), 7-29. friedman, g. (2014). workers without employers: shadow corporations and the rise of the gig economy. review of keynesian economics, 2(2), 171-188. fugate, m., prussia, g. e., & kinicki, a. j. (2012). managing employee withdrawal during organizational change: the role of threat appraisal. journal of management, 38(3), 890-914. gagné, m., & deci, e. l. (2005). self‐determination theory and work motivation. journal of organizational behavior, 26(4), 331-362. gannon, j. m., roper, a., & doherty, l. (2015). strategic human resource management: insights from the international hotel industry. international journal of hospitality management, 47, 65-75. gandini, a. (2019). labour process theory and the gig economy. human relations, 72(6), 10391056. gandhi, a., hidayanto, a. n., sucahyo, y. g., & ruldeviyani, y. (2018, october). exploring people’s intention to become platform-based gig workers: an empirical qualitative study. in 2018 international conference on information technology systems and innovation (icitsi), 266-271, ieee. garcía-chas, r., neira-fontela, e., & castro-casal, c. (2014). high-performance work system and intention to leave: a mediation model. the international journal of human resource management, 25(3), 367-389. garcía-chas, r., neira-fontela, e., & varela-neira, c. (2016). high-performance work systems and job satisfaction: a multilevel model. journal of managerial psychology, 31(2), 451-466. garmendia, a., elorza, u., & aritzeta, a. (2019, july). hpws, job satisfaction and productivity: a longitudinal study of a spanish retail company. in academy of management proceedings, 2019(1), 10673-10675. gefen, d., & straub, d. (2005). a practical guide to factorial validity using pls-graph: tutorial and annotated example. communications of the association for information systems, 16(1), 91-109. gkorezis, p., georgiou, l., & theodorou, m. (2018). high-performance work practices and nurses’ intention to leave: the mediating role of organizational cynicism and the moderating role of human resource management-related educational background. the international journal of human resource management, 29(3), 465-484. gleim, m. r., johnson, c. m., & lawson, s. j. (2019). sharers and sellers: a multi-group examination of gig economy workers' perceptions. journal of business research, 98, 142152. australasian journal of information systems behl et al. 2021, vol 25, research on user involvement gamifying the gig 27 guest, d. e. (2017). human resource management and employee well‐being: towards a new analytic framework. human resource management journal, 27(1), 22-38. guide jr, v. d. r., & ketokivi, m. (2015). notes from the editors: redefining some methodological criteria for the journal. journal of operations management, 37(1), v-viii. hair jr, j. f., sarstedt, m., hopkins, l., & kuppelwieser, v. g. (2014). partial least squares structural equation modeling (pls-sem). european business review, 26(2), 06-121. haivas, s., hofmans, j., & pepermans, r. (2013). volunteer engagement and intention to quit from a self‐determination theory perspective. journal of applied social psychology, 43(9), 1869-1880. hamari, j., koivisto, j., & sarsa, h. (2014, january). does gamification work?--a literature review of empirical studies on gamification. in 2014 47th hawaii international conference on system sciences (pp. 3025-3034). ieee. hasija, s., padmanabhan, v., & rampal, p(2020, june). will the pandemic push knowledge work into the gig economy? harvard business review. accessed on 8th october 2020 at https://hbr.org/2020/06/will-the-pandemic-push-knowledge-work-into-the-gigeconomy healy, j., nicholson, d., & pekarek, a. (2017). should we take the gig economy seriously?. labour & industry: a journal of the social and economic relations of work, 27(3), 232-248. heeks, r. (2017). decent work and the digital gig economy: a developing country perspective on employment impacts and standards in online outsourcing, crowdwork, etc. development informatics working paper, (71). available at: http://hummedia.manchester.ac.uk/institutes/gdi/ publications/workingpapers/di/ di_wp71.pdf (accessed 20 march 2020). heffernan, m., & dundon, t. (2016). cross‐level effects of high‐performance work systems (hpws) and employee well‐being: the mediating effect of organisational justice. human resource management journal, 26(2), 211-231. henseler, j., ringle, c. m., & sarstedt, m. (2015). a new criterion for assessing discriminant validity in variance-based structural equation modeling. journal of the academy of marketing science, 43(1), 115-135. hobfoll, s. e., shirom, a., & golembiewski, r. (2000). conservation of resources theory. appears in handbook of organizational behavior, rt golembiewski (ed.), marcel dekker, new york, 57-80. holt, t. j., & kennedy, j. p. (2019). technology's influence on white‐collar offending, reporting, and investigation. the handbook of white‐collar crime, 449-468. homans, c. g. (1967). fundamental social processes. in sociology. ed. n. j. smdser. new york: wiley homans, g. c. (1961). social behavior: its elementary forms (revised ed.). harcourt brace jovanovich. australasian journal of information systems behl et al. 2021, vol 25, research on user involvement gamifying the gig 28 hsu, c. l., & chen, m. c. (2018). how gamification marketing activities motivate desirable consumer behaviors: focusing on the role of brand love. computers in human behavior, 88, 121-133. huang, n., burtch, g., hong, y., &pavlou, p. a. (2020). unemployment and worker participation in the gig economy: evidence from an online labor market. information systems research. doi: https://doi.org/10.1287/isre.2019.0896 huotari, k., & hamari, j. (2017). a definition for gamification: anchoring gamification in the service marketing literature. electronic markets, 27(1), 21-31. huselid, m. a. (1995). the impact of human resource management practices on turnover, productivity, and corporate financial performance. academy of management journal, 38(3), 635-672. jabagi, n., croteau, a. m., audebrand, l. k., & marsan, j. (2019). gig-workers’ motivation: thinking beyond carrots and sticks. journal of managerial psychology, 34(4), 192-213. jiang, k., lepak, d. p., hu, j., & baer, j. c. (2012). how does human resource management influence organizational outcomes? a meta-analytic investigation of mediating mechanisms. academy of management journal, 55(6), 1264-1294. jyoti, j., & rani, a. (2019). role of burnout and mentoring between high performance work system and intention to leave: moderated mediation model. journal of business research, 98, 166-176. kapp, k. m. (2012). the gamification of learning and instruction: game-based methods and strategies for training and education. john wiley & sons. kässi, o., & lehdonvirta, v. (2018). online labour index: measuring the online gig economy for policy and research. technological forecasting and social change, 137, 241-248. kehoe, r. r., & wright, p. m. (2013). the impact of high-performance human resource practices on employees’ attitudes and behaviors. journal of management, 39(2), 366-391. keith, m. g., harms, p., & tay, l. (2019). mechanical turk and the gig economy: exploring differences between gig workers. journal of managerial psychology, 34(4), 286-306. ketokivi, m., & choi, t. (2014). renaissance of case research as a scientific method. journal of operations management, 32(5), 232-240. ketokivi, m., & schroeder, r. (2004). manufacturing practices, strategic fit and performance. international journal of operations & production management, 24(2), 171-191. khan, a., boroomand, f., webster, j., & minocher, x. (2020). from elements to structures: an agenda for organisational gamification. european journal of information systems, 1-20. kilhoffer, z., lenaerts, k., & beblavý, m. (2017), “the platform economy and industrial relations applying the old framework to the new reality”, research report no. 2017/12, european centre for political studies (ceps), brussels. kock, n. (2015). common method bias in pls-sem: a full collinearity assessment approach. international journal of e-collaboration (ijec), 11(4), 1-10. koivisto, j., & hamari, j. (2014). demographic differences in perceived benefits from gamification. computers in human behavior, 35, 179-188. australasian journal of information systems behl et al. 2021, vol 25, research on user involvement gamifying the gig 29 kuhn, k. m., & galloway, t. l. (2019). expanding perspectives on gig work and gig workers. journal of managerial psychology, 34(4), 186-191. kumar, j. (2013, july). gamification at work: designing engaging business software. in international conference of design, user experience, and usability. springer, berlin, heidelberg. kundu, s. c., & gahlawat, n. (2016). high performance work systems and employees’ intention to leave. management research review, 39(12), 1587-1615. kurin, j. (2016). a third way for applying us labor laws to the online gig economy: using the franchise business model to regulate gig workers. journal of business & technology law, 12(2), 193. landers, r. n., bauer, k. n., callan, r. c., & armstrong, m. b. (2015). psychological theory and the gamification of learning. in gamification in education and business (pp. 165186). springer, cham. lee, c. h., & bruvold, n. t. (2003). creating value for employees: investment in employee development. the international journal of human resource management, 14(6), 981-1000. lee, h. c. b., ba, s., li, x., & stallaert, j. (2018). salience bias in crowdsourcing contests. information systems research, 29(2), 401-418. lehdonvirta, v. (2018). flexibility in the gig economy: managing time on three online piecework platforms. new technology, work and employment, 33(1), 13-29. lertxundi, a., & landeta, j. (2011). the moderating effect of cultural context in the relation between hpws and performance: an exploratory study in spanish multinational companies. the international journal of human resource management, 22(18), 3949-3967. lindell, m. k., & whitney, d. j. (2001). accounting for common method variance in crosssectional research designs. journal of applied psychology, 86(1), 114. liu, w., he, c., jiang, y., ji, r., &zhai, x. (2020). effect of gig workers’ psychological contract fulfillment on their task performance in a sharing economy—a perspective from the mediation of organizational identification and the moderation of length of service. international journal of environmental research and public health, 17(7), 2208. locke, e. a. (1976). the nature and causes of job satisfaction. handbook of industrial and organizational psychology. rand mcnally, chicago, il longley, r. (2020). gig economy: definition and pros and cons. thoughtco, accessed on feb. 11, 2020, available at thoughtco.com/gig-economy-4588490. mcfeely, s., & pendell, r. (2018). what workplace leaders can learn from the real gig economy. gallup workplace. manyika, j., lund, s., bughin, j., robinson, k., mischke, j., & mahajan, d. (2016). independent work: choice, necessity, and the gig economy. mckinsey global institute, 2016, 1-16. masood, a., feng, y., rasheed, m. i., ali, a., & gong, m. (2020). smartphone-based social networking sites and intention to quit self-regulatory perspective. behaviour & information technology, 1-17. doi: https://doi.org/10.1080/0144929x.2020.1740787 australasian journal of information systems behl et al. 2021, vol 25, research on user involvement gamifying the gig 30 meijerink, j., & keegan, a. (2019). conceptualizing human resource management in the gig economy. journal of managerial psychology, 34(4), 214-23. mekler, e. d., brühlmann, f., tuch, a. n., & opwis, k. (2017). towards understanding the effects of individual gamification elements on intrinsic motivation and performance. computers in human behavior, 71, 525-534. michaelis, b., wagner, j. d., & schweizer, l. (2015). knowledge as a key in the relationship between high-performance work systems and workforce productivity. journal of business research, 68(5), 1035-1044. morschheuser, b., hamari, j., & maedche, a. (2019). cooperation or competition–when do people contribute more? a field experiment on gamification of crowdsourcing. international journal of human-computer studies, 127, 7-24. nye, f. i. (1978). is choice and exchange theory the key? journal of marriage and family, 40(2), 219-233. ornek, o. k., weinmann, t., waibel, j., & radon, k. (2020). precarious employment and migrant workers’ mental health: a protocol for a systematic review of observational studies. systematic reviews, 9(1), 1-6. peng, d. x., & lai, f. (2012). using partial least squares in operations management research: a practical guideline and summary of past research. journal of operations management, 30(6), 467-480. petriglieri, g., ashford, s. j., & wrzesniewski, a. (2019). agony and ecstasy in the gig economy: cultivating holding environments for precarious and personalized work identities. administrative science quarterly, 64(1), 124-170. petriglieri, g., ashford, s., & wrzesniewski, a. (2018). thriving in the gig economy. hbr’s 10 must, 109. pichault, f., & mckeown, t. (2019). autonomy at work in the gig economy: analysing work status, work content and working conditions of independent professionals. new technology, work and employment, 34(1), 59-72. poon, t. s. c. (2019). independent workers: growth trends, categories, and employee relations implications in the emerging gig economy. employee responsibilities and rights journal, 31(1), 63-69. prabowo, r., sucahyo, y. g., gandhi, a., &ruldeviyani, y. (2019, october). does gamification motivate gig workers? a critical issue in ride-sharing industries. in 2019 international conference on advanced computer science and information systems (icacsis) (pp. 343-348). ieee. quinn, r., & staines, g. (1979). the 1977 quality of employment survey: descriptive statistics, with comparison data from the 1969-70 and the 1972-73 surveys, survey research center, institute for social research, university of michigan press, ann arbor, mi. redondo, r., sparrow, p., & hernández-lechuga, g. (2019). the effect of protean careers on talent retention: examining the relationship between protean career orientation, organizational commitment, job satisfaction and intention to quit for talented australasian journal of information systems behl et al. 2021, vol 25, research on user involvement gamifying the gig 31 workers. the international journal of human resource management, 1-24. doi: https://doi.org/10.1080/09585192.2019.1579247 robson, k., plangger, k., kietzmann, j. h., mccarthy, i., & pitt, l. (2015). is it all a game? understanding the principles of gamification. business horizons, 58(4), 411-420. rockmann, k., jacob, m. r., & george, e. (2018). the drive to drive: understanding gig work by understanding the gig worker. academy of management global proceedings. accessed on 6th october 2020 at https://journals.aom.org/doi/abs/10.5465/amgblproc.surrey. 2018.0088.abs rosenthal, r., & rosnow, r. l. (2008). essentials of behavioral research: methods and data analysis. mcgraw-hill, boston. ruhi, u. (2015). level up your strategy: towards a descriptive framework for meaningful enterprise gamification. technology innovation management review, 5(8), 5-16. seaborn, k., & fels, d. i. (2015). gamification in theory and action: a survey. international journal of human-computer studies, 74, 14-31. sailer, m., hense, j. u., mayr, s. k., & mandl, h. (2017). how gamification motivates: an experimental study of the effects of specific game design elements on psychological need satisfaction. computers in human behavior, 69, 371-380. scheiber, n. (2017). how uber uses psychological tricks to push its drivers’ buttons. the new york times, 2. available at: https://www.nytimes.com/interactive/2017/04/02/ technology/100000005019770.app.html?nytapp=iphone&_r=1 (accessed 2 april 2020). schwellnus, c., geva, a., pak, m., & veiel, r. (2019). gig economy platforms: boon or bane?.oecd economics department working papers, no. 1550, oecd publishing, paris, https://doi.org/10.1787/fdb0570b-en. sepehr, s., & head, m. (2013, october). competition as an element of gamification for learning: an exploratory longitudinal investigation. in proceedings of the first international conference on gameful design, research, and applications, 2-9. shatz, i. (2015). using gamification and gaming in order to promote risk taking in the language learning process. in proceedings of the 13th annual meital national conference. haifa, israel: technion, 227-32. shi, l., & cristea, a. i. (2016, june). motivational gamification strategies rooted in selfdetermination theory for social adaptive e-learning. in international conference on intelligent tutoring systems (pp. 294-300). springer, cham. sun, l. y., aryee, s., & law, k. s. (2007). high-performance human resource practices, citizenship behavior, and organizational performance: a relational perspective. academy of management journal, 50(3), 558-577. stanford, j. (2017). the resurgence of gig work: historical and theoretical perspectives. the economic and labour relations review, 28(3), 382-401. stewart, a., & stanford, j. (2017). regulating work in the gig economy: what are the options? the economic and labour relations review, 28(3), 420-437. australasian journal of information systems behl et al. 2021, vol 25, research on user involvement gamifying the gig 32 swacha, j., & muszyńska, k. (2016, november). design patterns for gamification of work. in proceedings of the fourth international conference on technological ecosystems for enhancing multiculturality, 763-769. tenenhaus, m., vinzi, v. e., chatelin, y. m., & lauro, c. (2005). pls path modeling. computational statistics & data analysis, 48(1), 159-205. tett, r. p., & meyer, j. p. (1993). job satisfaction, organizational commitment, turnover intention, and turnover: path analyses based on meta‐analytic findings. personnel psychology, 46(2), 259-293. tirapani, a. n., & willmott, h. c. (2020). theorizing radical conflict: employment relations in the gig economy. in academy of management proceedings (vol. 2020, no. 1, p. 18975). briarcliff manor, ny 10510: academy of management. treuren, g. j. (2019). employee embeddedness as a moderator of the relationship between work and family conflict and leaving intention. the international journal of human resource management, 30(17), 2504-2524. van rhee, h., & dul, j. (2018, july). filling the black box of hr: unraveling the amo model and elevating it to the organizational level. in academy of management proceedings, 2018, (1), 13840-45. van roy, r., & zaman, b. (2019). unravelling the ambivalent motivational power of gamification: a basic psychological needs perspective. international journal of humancomputer studies, 127, 38-50. wagner, j. d., michaelis, b., & schweizer, l. (2012, july). knowledge as a key factor in the relationship between hpws and workforce productivity. in academy of management proceedings (vol. 2012, no. 1, p. 11647). briarcliff manor, ny 10510: academy of management. weber, j., azad, m., riggs, w., & cherry, c. r. (2018). the convergence of smartphone apps, gamification and competition to increase cycling. transportation research part f: traffic psychology and behaviour, 56, 333-343. werbach, k. (2014). (re) defining gamification: a process approach. in persuasive technology (pp. 266–272). springer international publishers wiener, m., cram, w. a., & benlian, a. (2020). technology-mediated control legitimacy in the gig economy: conceptualization and nomological network. in information systems outsourcing (pp. 387-410). springer, cham. williams, l. j., & anderson, s. e. (1991). job satisfaction and organizational commitment as predictors of organizational citizenship and in-role behaviors. journal of management, 17(3), 601-617. wood, a. j., graham, m., lehdonvirta, v., & hjorth, i. (2019). networked but commodified: the (dis) embeddedness of digital labour in the gig economy. sociology, 53(5), 931-950. wu, n., hoque, k., bacon, n., & bou llusar, j. c. (2015). high‐performance work systems and workplace performance in small, medium‐sized and large firms. human resource management journal, 25(4), 408-423. australasian journal of information systems behl et al. 2021, vol 25, research on user involvement gamifying the gig 33 wu, y., li, c., & khoo, s. (2016). predicting future volunteering intentions through a selfdetermination theory perspective. voluntas: international journal of voluntary and nonprofit organizations, 27(3), 1266-1279. xi, n., &hamari, j. (2019). does gamification satisfy needs? a study on the relationship between gamification features and intrinsic need satisfaction. international journal of information management, 46, 210-221. ye, h. j., & kankanhalli, a. (2017). solvers’ participation in crowdsourcing platforms: examining the impacts of trust, and benefit and cost factors. the journal of strategic information systems, 26(2), 101-117 zhang, m., zhu, c. j., dowling, p. j., & bartram, t. (2013). exploring the effects of highperformance work systems (hpws) on the work-related well-being of chinese hospital employees. the international journal of human resource management, 24(16), 3196-3212. zito, m., emanuel, f., molino, m., cortese, c. g., ghislieri, c., & colombo, l. (2018). turnover intentions in a call center: the role of emotional dissonance, job resources, and job satisfaction. plos one, 13(2), e0192126. zuckerman, m., porac, j., lathin, d., & deci, e. l. (1978). on the importance of selfdetermination for intrinsically motivated behavior. personality and social psychology bulletin, 4(3), 443-446. appendix a list of crowdsourcing platforms 1. kickstarter 18. hackerone 2. gofundme 19. band of hands 3. indiegogo 20. gigwalk 4. patreon 21. thumbtack 5. rockethub 22. guru 6. ulule 23. flexjobs 7. designhill 24. freelancer 8. utest 25. bellhops 9. mechanical turk 26. handy 10. trendwatching 27. hellotech 11. quri 28. hubstaff talent 12. catalant 29. moonlighting 13. turning art 30. peopleperhour 14. wonolo 31. snagajob 15. law trades 32. spare5 16. upcounsel 33. ifreelance 17. toptal 34. aquent australasian journal of information systems behl et al. 2021, vol 25, research on user involvement gamifying the gig 34 copyright: © 2021 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v25i0.2979 http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems poulsen 2018, vol 22, post publication review threats to autonomy from emerging icts 1 dainow, b. o. (2017). threats to autonomy from emerging icts. australasian journal of information systems, 21, 1-16. doi:10.3127/ajis.v21i0.1438 review dainow’s publication addresses perhaps the most important current issue: the threat that emerging icts have on human values which are not uniformly agreed upon and which fail to be correctly regulated by professional design ethics. the literature is rich with value considerations, in engineering (gordijn & akkermans, 2003; manders-huits, 2010), philosophy (nissenbaum & walker, 1998; j weckert, 2007), ict (burmeister, 2010, 2012, 2016; friedman, hendry, & borning, 2017; ridgley, 2009; teipel et al., 2016), as well as robotics/ai (sharkey, 2014; a van wynsberghe, 2015), yet dainow’s focus brings a refreshing new perspective. the systematic approach, by way of using the etica project, is an excellent frame. not only is it appropriate, using one of the best sources of an information on the ethical concerns of emerging icts, it creates a pleasing flow for the examination. the literature on autonomy was well sourced and the versions well defined. dainow linked those definitions through the similarity they share, “self-governance” (2017), to icts and the threat that each has on autonomy. although a great deal has been written about codes of professional practice (al-saggaf, burmeister, & schwartz, 2017; bowern, burmeister, gotterbarn, & weckert, 2006; burmeister, 2013, 2017; burmeister & weckert, 2003; capurro & britz, 2010; gotterbarn, 2009; pye & warren, 2006; john weckert & lucas, 2009), dainow’s differing versions on autonomy, and the examination, should be used to inform professional codes of ethics and conduct. this is an area where more work needs to be done. noting the potential deceptiveness that ai may play in affective computing and ambient intelligence is a much needed warning for a very real concern. perhaps ai could have also been discussed in other contexts where the ai isn’t the enabler but the entire system, such as a companion carebot avatar like the responsive interactive advocate (rita) (garner, powell, & carr, 2016). rita lacks robotic components and is purely an ai system. rita is not an enabler like a carebot with robotic components such as my spoon (aimee van wynsberghe, 2013). some ai systems, like companion carebots, emulate not only emotions (dainow, 2017) but consciousness and a subjective experience while giving users the impression that they truly possess those qualities that they emulate. that being said, the threats that ai systems create regarding autonomy are the same as those from affective computing but just from the perspective of the ai system itself and not just as an enabler of another ict. the examples to describe the technologies are many. such as referencing medicine delivery technology in bioelectronics (dainow, 2017). but, more specific cases of particular emerging icts that threaten autonomy would make for a more informative paper. for example, the u2apple incident where 500 million apple iphone and ipod users had a music album automatically downloaded onto their device (booth, 2014). the automatic download, albeit free, made personal choices for users that threatened their autonomy in a small way. this example could have concluded the human-machine symbiosis section rather than a general reference to enforced software updates (dainow, 2017). granted, it might be the case that since the paper focuses on emerging icts, then specific cases are obscure or limited. it is made clear that personalisation of emerging icts secures or safeguards autonomy. this point is reinforced by almost all of the closing sentences on each of the emerging ict sections. each of those closing remarks describe that a failure to allow personalisation threatens all australasian journal of information systems poulsen 2018, vol 22, post publication review threats to autonomy from emerging icts 2 forms of autonomy. the solution dainow presents (personalisation capabilities) is the proper conclusion to the threat. adam poulsen computer scientist adamrosspoulsen@hotmail.com brandt dainow’s response adam polson’s comments are appreciated. his suggestion for specific examples, and those he provides, are insightful and raise an interesting question of methodology. i chose to use the etica findings because they constitute a robustly developed taxonomy of technologies. obviously a discussion of how etica identified these technologies was beyond the scope of a 6,000 word paper. in fact, etica did not start with a taxonomy, but by examining specific ict developments. etica analysed literally hundreds of ict systems, from both commerce and academic research, in multiple countries, initially using bibliometric analysis of thousands of documents (veikko, kanerva, and kouri 2009), then surveys and expert panels (heersmink et al. 2010). from this data they developed the set of fourteen technology groups (ikonen et al. 2010). my conversations with etica project members indicate that there was often “passionate” debate about where some systems sat in the taxonomy, though (obviously) you won’t see those debates in the final publications. so, from a methodological perspective, selecting specific, current, examples of each technology group risks raising a distracting disagreement about whether that example really does belong to that group. etica chose to tackle this issue with the development of scenarios describing someone’s interaction with technologies of each group. the emphasis in these scenarios was not so much the features and operation of the system as the impact of that style of technology much more accessible for politicians, but much less satisfying for readers of the ajis. the underlying issue for ict ethics in this regard is that, in fact, few technologies impact a person in isolation. while it is possible to discuss them individually, one ict’s impact on the person’s life can rarely be separated from the impact of other ict’s supporting that system, or being delivered at the same time. many deployed ict’s depend on others for their delivery, and the operation and design of those systems can deeply affect the way in which the delivered ict interacts with the person (dainow 2017). likewise, the ethical impact of many ict’s depends not on the system, but the business model under which it is delivered (dainow 2015), as adam’s reference to the apple incident shows. from a pedagogical perspective, this raises issues for teaching ethics in computer science. we must teach students how to construct individual systems, because that’s how people must work. however, to have an ethical understanding which can inform their coding, they need to be aware of the context in which their systems will be deployed; both in terms of other interacting technologies and in terms of the business model under which it will be rolled out. at this stage, ethical education within university computer science departments is rare and lacks established methodologies. the task is thus finding a way in which we can make accessible to students the complexity of philosophical concepts and the close interaction between business and technology, as well as how every line of code has the potential for ethical impact. many pedagogical approaches are being developed, but it seems to me the most likely to succeed is to cultivate concrete principles similar to formal standards and protocols, but which embed ethical conduct. for example, langheinrich’s principles for privacy-aware ubiquitous systems (langheinrich 2001) offer simple rules which make systems both more ethical. australasian journal of information systems poulsen 2018, vol 22, post publication review threats to autonomy from emerging icts 3 moreover, the range of versions of autonomy conceals an even greater complexity. irrespective of how it is understood, autonomy involves allowing each person to determine how they live for themselves. even if one is restricted to a single version of autonomy, one is still confronted by a vast range of lifestyles. unless a system is trivial, one-size-fits-all is almost always going to restrict someone’s autonomy somehow. references al-saggaf, y., burmeister, o. k., & schwartz, m. (2017). qualifications and ethics education: the views of ict professionals. australasian journal of information systems, 21. booth, r. (2014). u2’s bono issues apology for automatic apple itunes album download. the guardian. retrieved from https://www.theguardian.com/music/2014/oct/15/u2-bonoissues-apology-for-apple-itunes-album-download bowern, m., burmeister, o. k., gotterbarn, d., & weckert, j. (2006). ict integrity: bringing the acs code of ethics up to date. australasian journal of information systems, 13(2), 168-181. doi:10.3127/ajis.v13i2.50 burmeister, o. k. (2010). websites for seniors: cognitive accessibility. international journal of emerging technologies and society, 8(2), 99-113. burmeister, o. k. (2012). what seniors value about online community. journal of community informatics, 8(1). burmeister, o. k. (2013). achieving the goal of a global computing code of ethics through an international-localisation hybrid. ethical space: the international journal of communication ethics, 10(4), 25-32. burmeister, o. k. (2016). the development of assistive dementia technology that accounts for the values of those affected by its use. ethics and information technology, 18(3), 185-198. doi:10.1007/s10676-016-9404-2 burmeister, o. k. (2017). professional ethics in the information age. journal of information, communication & ethics in society, 15(2). burmeister, o. k., & weckert, j. (2003). applying the new software engineering code of ethics to usability engineering: a study of 4 cases. journal of information, communication & ethics in society, 3(3), 119-132. capurro, r., & britz, j. b. (2010). in search of a code of global information ethics: the road travelled and new horizons. ethical space: the international journal of communication ethics, 7(2/3), 28-36. dainow, b. (2017a). threats to autonomy from emerging icts. australasian journal of information systems, 21, 1-16. doi:10.3127/ajis.v21i0.1438 dainow, b. (2017b). ‘smart city transcendent understanding the smart city by transcending ontology’. orbit 1. https://www.orbit-rri.org/volume-one/smart-citytranscendent/. dainow, b. (2015). ‘key dialectics in cloud services’. computers & society ethicomp special issue: 52–59. https://doi.org/10.1145/2874239.2874247. australasian journal of information systems poulsen 2018, vol 22, post publication review threats to autonomy from emerging icts 4 friedman, b., hendry, d. g., & borning, a. (2017). a survey of value sensitive design methods. foundations and trends® in human–computer interaction, 11(2), 63-125. doi:10.1561/1100000015 garner, t. a., powell, w. a., & carr, v. (2016). virtual carers for the elderly: a case study review of ethical responsibilities. digital health, 2, 1-14. doi:10.1177/2055207616681173 gordijn, j., & akkermans, h. (2003). value based requirements engineering: exploring innovative e-commerce ideas. requirements engineering journal, 8(2), 114-134. gotterbarn, d. (2009). ict governance and what to do about the toothless tiger(s): professional organizations and codes of ethics. australasian journal of information systems, 16(1). doi:10.3127/ajis.v16i1.601 heersmink, r., timmermans, j., van den hoven, j., & wakunuma, k. (2010). ‘d.2.3 normative issues workshop report’. etica project. ikonen, k., kanerva, m., kouri, p., stahl, b., and wakunuma, k. (2010). ‘d.1.2. emerging technologies report’. etica project. langheinrich, m., (2001). ‘privacy by design principles of privacy-aware ubiquitous systems’. in proceedings of the third international conference on ubiquitous computing, edited by gregory abowd, barry brumitt, and steven shafer, 273–91. lncs. atlanta, usa: springer-verlag. manders-huits, n. (2010). what values in design? a critical analysis of value-sensitive design. science and engineering ethics, 17(2), 271–287. nissenbaum, h., & walker, d. (1998). will computers dehumanize education? a grounded approach to values at risk. technology in society, 20(3), 237-273. pye, g., & warren, m. (2006). striking a balance between ethics and ict governance. australasian journal of information systems, 13(2). doi:10.3127/ajis.v13i2.53 ridgley, c. m. (2009). the decision disconnect. australasian journal of information systems, 16(1). doi:10.3127/ajis.v16i1.602 sharkey, a. j. c. (2014). robots and human dignity: the effects of robot care on the dignity of older people. ethics and information technology, 16(1), 53-75. doi:10.1007/s10676-014-93385 teipel, s., babiloni, c., hoey, j., kaye, j., kirste, t., & burmeister, o. k. (2016). information and communication technology solutions for outdoor navigation in dementia. alzheimer's & dementia: the journal of the alzheimer's association, 12(6), 695-707. doi:10.1016/j.jalz.2015.11.003 van wynsberghe, a. (2013). designing robots for care: care centered value-sensitive design. science and engineering ethics, 19(2), 407-433. doi:10.1007/s11948-011-9343-6 van wynsberghe, a. (2015). healthcare robots: ethics, design and implementation: ashgate publishing, ltd. veikko, ikonen, minni kanerva, and panu kouri. 2009. ‘d.1.1 heuristics & methodology report final’. d.1.1. etica project. australasian journal of information systems poulsen 2018, vol 22, post publication review threats to autonomy from emerging icts 5 weckert, j. (2007). social values and nanotechnology. presentation. cappe, charles sturt university. canberra, australia. weckert, j., & lucas, r. (2009). the governance of information and communication technology. australasian journal of information systems, 16(1). doi:10.3127/ajis.v16i1.599 copyright: © 2018 poulsen. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems kumar & thakurta 2024, vol 28, research article dt: enterprise transformation perspective received 9 dec 2022, accepted 30 april 2024 after 3 rounds of review with karlheinz kautz as accepting editor. 1 digital transformation: an enterprise transformation perspective exploring the underlying constituents of digital transformation from the lens of enterprise transformation theory rahul kumar indian institute of management (iim), calcutta, india rahulk@iimcal.ac.in rahul thakurta xavier institute of management (xim) bhubaneswar, odisha, india abstract digital transformation (dt) has emerged as one of the most talked-about phenomena of the decade. the rush of things around dt also exposes its challenges towards effectively integrating digital technologies into the scheme of things. the proliferating literature on dt offers a fragmented understanding and is unclear about the constituents and configuration of the phenomena. the above concerns primarily arise from insufficient theoretical grounding and deficiencies in the extant conceptualizations. to address these concerns, the article posits an over-arching research question to examine the phenomena while theoretically uncovering its foundational elements. accordingly, the study resorts to the enterprise transformation framework to explicate the transformative aspects based on a two-phase analysis. the first phase adopts a text-mining approach for uncovering the latent themes underlying the dt scholarship, followed by a qualitative approach involving content analysis. we finally propose a theoretically motivated conceptualization of the tenets of dt. we specifically investigate the phenomenon's scope, ways, means, and ends. the proposed framework is further validated following a multi-case analysis. our conceptualization grounds and establishes the significance of foundational elements having a theoretical basis for identifying dt. our examination offers an implementation guide for the practice, which we delineate subsequently. keywords: conceptual framework, digital transformation, enterprise transformation, theory development, topic modelling. 1 introduction the recent decade has witnessed the growing attention of organizations toward digital technologies. resultantly, digital transformation (dt) is the focus of board room discussions, with its planning and implementation being a key concern (chanias et al., 2019). the crucial challenge is knowing how to exploit digital technologies and integrate them into and around the organization (hess et al., 2016). since digital technologies have the capability to transform and reinvent businesses radically, their failure rates are also striking (wade & shan, 2020). the constituents of carrying out a successful transformation based on digital technology are unknown, and so is the case with the configurations of a digital strategy (cetindamar et al., 2021; omrani et al., 2022). mailto:rahulk@iimcal.ac.in australasian journal of information systems kumar & thakurta 2024, vol 28, research article dt: enterprise transformation perspective 2 pertinent to organizations, these challenges arise from the need to understand the notions underlying dt. to most, dt is synonymous with information technology (it)-enabled organizational transformation (wessel et al., 2020), and to others, it is analogous to strategic renewal using digital technologies (g. c. kane, 2017c; sebastian et al., 2017; vial, 2019). to the rest, dt is all about having a digital strategy in place to drive digital innovations (baiyere et al., 2020; hanelt et al., 2021) in tandem with the traditional viewpoints of managing technological changes to foster innovations (faulkner & runde, 2009, 2019). even the academia lacks a comprehensive and unified understanding of dt as the notion of dt shares its genesis with other fields, particularly the information systems and the strategy literature (baiyere et al., 2020; canhoto et al., 2021; teubner & stockhinger, 2020; vial, 2019; wessel et al., 2020). few scholarly attempts highlighting the prominence of digital technologies as it artifacts having the potential to transform an organization, overlooking the strategic direction of the change initiatives (faulkner & runde, 2009, 2013, 2019). such a mixed and unclear understanding of the field, coupled with misconceptions in the scholarship, is majorly due to a lack of transdisciplinary theoretical perspectives and deficient conceptualizations (vial, 2019; wessel et al., 2020; wimelius et al., 2021). previous works have made a few attempts to illuminate the nuances of dt and add clarity to this scholarship; however, their efforts are limited to offering a taxonomy of the field (matt et al., 2015; mergel et al., 2019; vial, 2019; westerman et al., 2014). even the frameworks and models offered intend to classify the contributions rather than conceptualize the tenets of dt (hanelt et al., 2021, nadkarni and prügl, 2021; warner and wäger, 2017). mere concentration on the phases and levels of dt (teubner & stockhinger, 2020) or focus on its driving force the digital technologies and advanced it artifacts (faulkner & runde, 2013, 2019; verhoef et al., 2021), and do not cater to the fundamental questions around such initiatives. this study, therefore, has two objectives: first, to explore and understand dt using a theoretical lens, and second, to advance the theory by conceptualizing the foundational elements of dt. in pursuit of these objectives, we formulate the following research question to guide our exploration: rq: what are the integral elements of a dt endeavour? the realm of dt is warranted with answers about the purpose, nature, extent, and constituents to ensure successful planning and execution of the transformation initiative. therefore, to guide the exploration, this paper resorts to theorizing the foundational elements of dt. a multitude of perspectives have been used to analyse dt; however, a comprehensive and congruent conceptualization is wanting (lindgreen et al., 2021; philip & mckeown, 2004; piazza & castelluci, 2014). the existing theoretical lens either earmarks the organizational level attributes as foundational elements or focuses on the technological constructs prevalent in the it strategy scholarship (nwaiwu, 2018). in this study, we resort to the enterprise transformation (et) framework, grounded in a transdisciplinary lens of transformation that incorporates both systems and management viewpoints (rouse, 2005a; rouse & baba, 2006). this combined viewpoint helps in unveiling the complex configurations of the dt phenomenon and explaining it (rouse, 2005a; rouse & baba, 2006). the relevance of the framework stems from the explication of the scope, means, and ends of a transformative phenomenon which is in tandem with the focus of our inquiry. grounded on the et framework, we synthesize the extant literature in two phases, involving text mining a large corpus of selected articles in phase-1, followed by an in-depth qualitative analysis of a sample of shortlisted articles in phase-2. the findings offer supporting evidence based on which we australasian journal of information systems kumar & thakurta 2024, vol 28, research article dt: enterprise transformation perspective 3 divide the realm of dt into four conceptual elements justifying the scope, ways, means, and ends. these foundational elements (i.e., scope, ways, means, and ends) configure our proposed dt conceptual framework as a transformational phenomenon while extending the et framework. the exploration of value creation paths (i.e., “ways”): value co-creation and value re-creation offer a strategic outlook to substantiate the strategic standpoint of transformational endeavours leveraging digital technologies. moreover, the role of big data analytics remains overarching for any dt initiative, while the importance of digital governance can never be ignored in advanced digital ecosystems. in continuation, we also validate our findings by drawing relevance from two cases from the practice. overall, this study offers a clear distinction from the past works and provides a multi-fold contribution to both academia and practice as listed below: 1. the concept of dt is grounded on a transdisciplinary theoretical lens of et, revealing the dts purpose, nature, extent, constituents, and end results. 2. the study extends the et framework by proposing a conceptual framework with added dimensions, validated through case studies. 3. the framework components justify the purpose and extent of any planned dt, while the ways help clearly articulate the strategic intent of such endeavours. 4. the study incrementally reveals the pivotal role of bda (big data analytics) and digital governance as cornerstones for dt. 5. the study provides insights into organizing for dt, aiding practitioners in anticipating and planning for possible challenges. the rest of this paper is arranged as follows. section 2 presents an overview of the literature around the main concepts supporting this research toward justifying the work. section 3 documents our research approach relying on theory synthesis and extension. in section 4, we present our research findings contributing to the dt framework and elaborate on its validation. section 5 includes a discussion of the results. the final section wraps up the content by offering a summary, delineating the contributions, acknowledging the limitations, and showcasing opportunities for future research. to enhance readability, some additional details are relegated to the appendices. 2 related studies 2.1 transformation transformation characterizes “a complete change in the appearance or character of something or someone, especially so that thing or person is improved” (cambridge university press, 2013). over a considerable period, the academic literature focusing on enterprises has conceptualized transformation as “a process that engenders a qualitatively different organization” (besson & rowe, 2012, p. 103). enterprise transformation is motivated by value deficiencies that lead to considerably revised or new work processes depending on management's capabilities, constraints, and preferences (rouse & baba, 2006). the aforementioned characteristics apply to the enterprise as a whole and the social networks of management in particular (rouse, 2006). enterprise transformation is about change, not repetitive but fundamental, that significantly changes how an enterprise australasian journal of information systems kumar & thakurta 2024, vol 28, research article dt: enterprise transformation perspective 4 engages in a relationship with its primary stakeholders, such as employees, customers, suppliers, and investors (rouse, 2005b). the transformation has been viewed as a fundamental change impacting the organization in various ways (hess et al., 2016; kotter, 1995). for example, a transformation may entail new value propositions for products and services and new ways to deliver and manage these offerings (rouse, 2005a). additionally, transformation frequently occurs in a discontinuous or even sudden manner. it differs from changes that are continuous, and that result in improvements that are realized slowly and steadily. transformation can result in significant changes occurring sporadically, leading to gains to for the enterprise (purchase et al., 2011; rouse, 2005b). 2.2 digital (technologies) digital refers to electronic technology that produces, stores, and analyses data in terms of two states, viz., positive and non-positive. various forms of digital technologies have become pervasive, representing “... a combination of information, computing, communication, and connectivity technologies” (bharadwaj et al., 2013, p. 2). the implicit understanding of these varied digital technologies in the information systems literature is in the form of technologies incorporating bitstrings (digital objects) (e.g., faulkner & runde, 2019). the representation takes a simplistic view of digital technologies and fails to justify its uniqueness and diversity, as noted by several (faulkner & runde, 2019; saeed & sidorova, 2023). there may be uncertainties at the onset about how digital technology may be employed, resulting in ambiguity regarding its boundaries (hund et al., 2021). in order words, individuals and groups may relate to digital technologies differently, assigning them varied meanings, thereby shaping its boundary. these arguments present a perspective of digital technology as possessing technical components (i.e., material like servers or computers that have a physical mode of being, and nonmaterial, such as operating systems or software that do not have a physical mode of being) and social components where users assign meanings and thus shape the boundary of digital artifacts (faulkner & runde, 2019). this viewpoint is in tandem with characterizing digital technologies as resources where apart from the material and nonmaterial properties, considerable attention is warranted to information technology-related competencies in the form of managerial and technical knowledge, skills, and processes (faulkner & runde, 2019). digital technologies have brought significant shifts in organizations (gerster, 2017). the shift leverages the social component of digital technology and goes beyond “digitizing resources” towards transforming processes, services, communication means, or even the overall business operations models (haffke et al., 2017). the dt we are witnessing presently is propelled by digital technologies to fundamentally change how companies operate and bring value to customers (henriette et al., 2015; reis et al., 2018). digital technologies are now considered “forces” which would change markets completely (lucas jr et al., 2013; vial, 2019), such as platforms (tan et al., 2015; tiwana et al., 2010), analytics (günther et al., 2017; gust et al., 2017), cloud computing (clohessy et al., 2017; du et al., 2016). hence it is also imperative for organizations to react to these changes efficiently (li et al., 2016; liu et al., 2013; yeow et al., 2018). digital technologies have a transformative effect in three ways: (1) by fundamentally redefining “business capabilities,” revolutionizing both internal and external standard operating procedures and business relationships; (2) by involving “strategic acquisitions” to gain new business capabilities or establish a new market area; and (3) by demonstrating the australasian journal of information systems kumar & thakurta 2024, vol 28, research article dt: enterprise transformation perspective 5 use of it to significantly alter how tasks are performed in organizations (dehning et al., 2003). the impact of digital technologies on decision-making, customer behaviour, competitive landscape, and market dynamics is already recognized (bharadwaj et al., 2013; markus & loebbecke, 2013; vial, 2019). 2.3 digital transformation 2.3.1 conceptualization and positioning drawing relevance from the previous discussion, dt can now be viewed as the transformation triggered by digital technologies. dt highlights the effects of digital technologies on organizational attributes such as the structure, procedures, flow of information, and organizational capacity to accept and adapt to the digital technologies (cui & pan, 2015; zhu, 2004). the majority of theorizing around dt is driven by digital technologies and how they ensure improved business outcomes (li et al., 2018; vial, 2019). the various efforts have conceptualized dt in diverse ways. the extant literature presents dt as the orchestration of digital technologies toward automating firm-level processes and activities, alternation of essential business processes driven by technologies ( singh et al., 2017; demirkan et al., 2016; nwankpa & datta, 2017), it impact on organizational contexts, and business outcomes (fitzgerald et al., 2014; hanelt et al., 2015; liere-netheler et al., 2018). some other definitions relate dt to aims toward digital maturity and modernization supported by digital technologies and processes (ivančić et al., 2019; mergel et al., 2019), and improvement of the process triggered by technologies (vial, 2019). as it turns out, the vast literature on dt has conceptualized the phenomenon in varied and confusing ways (e.g., benlian & haffke, 2016; berghaus & back, 2016; hess et al., 2016; horlacher et al., 2016). indeed, the lack of consistency around dt conceptualization has been identified as a concern (morakanyane et al., 2017; riasanow et al., 2019; vial, 2019; wessel et al., 2020). apart from the conceptualization dilemma, the connection to dt as the most pervasive phenomenon affecting business operations, society, and individuals also suffers a positioning dilemma (cetindamar et al., 2021; omrani et al., 2022). this is evident from the calls to the academic community to distinguish it from other organizational phenomena (e.g., information systems/information technology-enabled organizational transformation) and related explorations (riasanow et al., 2019; wessel et al., 2020). the diverse connotations, use of unclear terminology, blurred positioning, and shared conceptions, among other challenges, are deterrents to the clarity of dt (hanelt et al., 2021b). the vast but diverse literature on dt thereby portrays a lacking of consistent understanding of the phenomenon (warner & wäger, 2019; wessel et al., 2020). simply assuming that dt is unique and different without a theoretical basis would hinder the conceptual foundations of the field. 2.3.2 synthesis of contributions in line with the increasing excitement around dt, several authors also attempted to synthesize prevalent research efforts. the list of review papers on dt is ever-increasing. table 1 summarizes some of the notable efforts by delineating each review article’s objective and supporting information, with the articles listed chronologically. we also reflect on the current study in the last row to differentiate it from the extant attempts. the articles listed in table 1 collectively aim to enhance our understanding of digital transformation (dt), focusing on various aspects of the phenomenon. some articles concentrate on conceptualizing dt, outlining its characteristics, drivers, impact, as well as australasian journal of information systems kumar & thakurta 2024, vol 28, research article dt: enterprise transformation perspective 6 opportunities and challenges (e.g., morakanyane et al. 2017, reis et al., 2018, riasanow et al., 2019, vial 2019). others delve into the organizational and ecosystem aspects, exploring the supporting resources and capabilities within the dt context (e.g., henriette et al., 2015, leão & da silva, 2021, nischak et al., 2017). furthermore, certain review articles aim to provide models and frameworks that align with their specific objectives around dt (e.g., hanelt et al., 2021, vial, 2019). collectively, these reviews contribute to a nuanced understanding of dt, presenting diverse perspectives and frameworks and highlighting the multifaceted impacts on organizations. few other research efforts around dt have focussed on developing models and frameworks aligned with the research objectives. for example, mergel et al. (2019) focus on exploring dt in the public sector. following a series of interviews with experts from the public sector, the work contributes to defining dt and offering a conceptual framework segregating the reasons, objects, processes, and results of dt. warner and wäger (2017) explore the concept and mechanism of dt based on evidence collected from many sources over time. the findings indicated inconsistencies regarding organization leaders’ understanding of the digital transformation phenomena. the authors offer a process model to clarify the dt mechanism comprising nine micro-foundations to support the dynamic capabilities required for dt. wessel et al. (2021) investigate some of the fundamental tenets based on which dt can be differentiated from the phenomena of it-enabled organizational transformation. the authors offer a process model of transformation for comparing the two phenomena. the differentiation aspects from their research emerged around value proposition (i.e., redefining in the case of dt versus supporting in the case of it-enabled organizational transformation) and organizational identity (creating in the case of dt versus enhancing in the case of it-enabled organizational transformation). these frameworks and models portray some commonalities. these adopt an organizational-level perspective on dt and delineate some of the key considerations (e.g., technological considerations, mechanisms, and outcomes) in their portrayal of dt. most tend to place technology at the center stage, overlooking other relevant organizational characteristics and governance mechanisms. despite the need for more clarity about the notions of dt and multiple framing of its structure, a recurring subject in the present discussion is how modern businesses are impacted by and must adapt to the growth of digital technology. moreover, organizations also undergo internal transformations to alter the way they structure their processes (hanelt et al., 2015; morakanyane et al., 2017), develop ways to address inertia (kohli & johnson, 2011; röcker et al., 2017), and realize value (dremel, wulf, et al., 2017; günther et al., 2017; huang et al., 2017). the phenomenon is naturally linked with the canvas of transformation, propelled through technologies, and contributing to fundamental organizational changes in various ways (kotter, 1995; matt et al., 2015). in summary, the current understanding of the fundamentals of dt lacks clarity, and the contributions in the literature indicate an absence of theoretical grounding. in other words, the concept of dt seems to be obscured within the myriad of explorations dedicated to it. australasian journal of information systems kumar & thakurta 2024, vol 28, research article dt: enterprise transformation perspective 7 article objective sample information henriette et al. (2015) the review identifies the specific digital capabilities needed for the transformation while also earmarking how a dt journey impacts business models, operational processes, and user experience (primarily customers). the study reports a systematic review of 202 scopuslisted articles until 2015. the search strategy encapsulates business models, operational processes, and user experience as the study context. morakanyane et al. (2017) the review attempts a reconciliation of the dt literature to offer an inclusive understanding. it focuses on components of dt, such as characteristics, drivers, and the impact of dt. the impact is motivated by earlier attempts to reemphasize the elements of business models, operational process, user experience, employees, and culture. the study reports a systematic review of 53 articles published in conferences and journals in or after 2010. nischak et al. (2017) the article concentrates on unveiling the importance and nuances of value ecosystems in the parlance of dt. the authors further posit the interactions of platforms and related ecosystems with traditional is and other cross-disciplinary perspectives. the study is based on 42 selective articles focusing on ecosystems in a dt endeavor and featuring in databases: journal storage (jstor), business source complete, and association for information systems electronic library (aisel). reis et al. (2018) the review attempts to uncover the opportunities and challenges enveloping dt while informing scholars about digital business strategy. the study reports a systematic survey of 206 peerreviewed articles on process and operations management from the isi (institute for scientific information – web of science) database. riasanow et al. (2019) the review attempts to clarify the notions of dt by attending to the disagreements and inconsistencies enveloping the field. the authors also document the diverse viewpoints of the heterogeneous schools of thought. the study is based on 175 articles published in select is and management journals and is conferences. specific to dt publications, the search year was specified as 2015 onwards. dang & vartiainen (2019) the review reports an in-depth study of articles on digital strategy to discuss strategists’ various aspects and distinguish it from traditional it/is strategy. the discussion pivots around the environment, approaches, stakeholders, challenges, and capabilities. the study considers 42 articles from the basket of eight senior is journals (now updated)1 and proceedings of the international conference on information systems (icis). vial (2019) the study offers a detailed conceptualization and an inclusive definition of the dt phenomena. the author proposes a dt framework representing the phases of the dt process at the organizational level. this offers four compartments to understand dt in terms of the affected entity and the extent of transformation. the author also deliberates on the required technologies and other means to drive transformation while highlighting potential impacts. the study reports a systematic review of 282 articles published in databases: ais library, business source complete, and sciencedirect. 1 https://aisnet.org/page/seniorscholarlistofpremierjournals australasian journal of information systems kumar & thakurta 2024, vol 28, research article dt: enterprise transformation perspective 8 teubner & stockhinger (2020) the review focuses on resynthesizing it/is strategy at the current juncture dominated by emerging technologies. accordingly, the authors delve into digitalization and related concepts such as digital ecosystems, digital innovation, and dt. they posit the role and nuances of is/it strategy in the digital age. the study relies on a focused review of 141 articles between 2008-2018, featuring in databases: ais library, business source complete, and sciencedirect. nadkarni & prügl (2021) the article focuses on developing thematic maps and organizing contributions from a technological viewpoint and the actors/entities related to the transformation initiative. the authors further supplement their analysis by integrating cross-disciplinary perspectives involving digital disruptions and entrepreneurship. the study reports a systematic review of 58 articles published between 2001 and 2019 within five electronic databases (business source complete, scopus, sciencedirect, social sciences citation index (ssci), and google scholar). leão & da silva (2021) the study underscores the evaluation of the impact of dt on firms' competitive advantages. it illuminates the diverse nature of these impacts, encompassing both positive and undesirable/negative outcomes. the study draws upon a sample of 20 articles from the ebsco database, shortlisted based on a specific focus on dt and its impact on competitive advantage. trenerry et al. (2021) the review focuses on employee-related aspects of dt initiatives, categorizing them into individual factors (attitudinal), group-level factors (collaborative), and organization-level factors (cultural), delineating their roles in the dt journey. the study reviews dt articles published in confluence with psychology, organizational behaviour, and management science in the sciencedirect database after 2000. hanelt, et al. (2021) the review presents thematic patterns related to organizational designs and digital business ecosystems in the transformative phenomenon. the authors present a multi-dimensional framework integrating contextual conditions, mechanisms, and dt outcomes toward clarifying the phenomena' boundary conditions. through this, the authors emphasize the impact of technology, the extent of adaptation, systemic shifts, and holistic co-evolution in the context of a dt endeavor. the study is based on 279 articles on management science listed in business source complete between 2000 and 2018, with particular attention to the publications featuring in the financial times (ft) 50 list of journals. our study the current study investigates the foundational elements of dt. the study adopts a transformational perspective, using the lens of the et framework to examine the phenomena. the authors introduce a comprehensive dt framework consisting of four integral components that represent the scope, ways, means, and ends of dt. the addition of the "ways" dimension contributes to the novelty, extending the et framework by delineating the path of value-creation opportunities. the study considers a total of 4314 articles published since 2010 and featuring in the scopus database. this is subjected to a two-stage synthesis involving quantitative analysis of the entire corpus and followed by qualitative analysis of a subset involving 182 articles. table 1. selected review papers on digital transformation australasian journal of information systems kumar & thakurta 2024, vol 28, research article dt: enterprise transformation perspective 9 3 digital transformationa conceptual frontier 3.1 theory synthesis the role of theory in information systems remains pivotal (bichler et al., 2016; markus, 2014; orlikowski & iacono, 2001; weber, 2003). more than just organizing and summarizing the notions, it helps impose a tangible directive on the phenomenon (maanen & bacharach, 2015). in this realm, we resort to the enterprise transformation (et) framework (rouse, 2005a;) as the theoretical foundation to capture and accumulate knowledge about dt as a phenomenon from an abstract perspective of an enterprise. the et framework presents the basis for understanding the nature of transformation (rouse, 2005a). the idea is to comprehensibly understand and identify the phenomena' nature, purpose, and purview, while uncovering the structural constituents, thereby adhering to our research question. earlier attempts have tried to offer a conceptual description of dt; however, most lack a theoretical grounding (mendling et al., 2020; rowe, 2018; venable, 2006; verhoef et al., 2021). as noted above, these include conceptualizing dt in various ways, for example, offering frameworks and process models to explain dt or its facets (e.g., hanelt, et al., 2021; mergel et al., 2019; vial, 2019; warner & wäger, 2019; wessel et al., 2020) in this parlance, a few studies have attempted to theorize dt, and the lenses used have offered polarized viewpoints (lindgreen et al., 2021; nwaiwu, 2018). business strategy has dominated the discussion around organizational capabilities, with technology playing a sub-ordinate role (canhoto et al., 2021; li et al., 2012; philip & mckeown, 2004). alternately, technology has been recommended as the engine for transformation while surpassing the role of firm-specific attributes (kumar, 2021; rowe, 2018; wimelius et al., 2021). this study, therefore, plugs this significant gap by resorting to the et framework thereby offering a transdisciplinary perspective integrating both the systems and the management domain (rouse, 2005a;). figure 1 below presents the (enterprise) transformation framework borrowed from rouse (2005a), which depicts the foundational elements of enterprise transformation. figure 1 classifies a transformation initiative in three dimensions (i.e., scope, means, and ends). following the et framework, the scope is indeed extensive, encompassing activity, function, organization, and the broader enterprise. at the most granular activity level, transformation can impact individual activities or tasks within an organization. this might involve automating repetitive tasks, streamlining data entry, or enhancing communication through appropriate tools. the goal is to make these activities more efficient, accurate, and costeffective. moving up the scale, at the functional level, an enterprise transformation can revamp entire functions within an organization. human resources, marketing, finance, and supply chain management are areas where significant changes are possible using appropriate technologies. enterprise transformation often extends across the entire organization. it involves aligning strategies, processes, and culture toward achieving broader goals. this might entail restructuring teams, fostering a culture of innovation, and redesigning workflows for greater efficiency. finally, beyond individual organizations, et can affect entire industries or ecosystems. it involves collaboration and integration among multiple organizations, leveraging technological advancements. australasian journal of information systems kumar & thakurta 2024, vol 28, research article dt: enterprise transformation perspective 10 figure 1. (enterprise) transformation framework, borrowed from rouse (2005a). the approach or means adopted for transformation pursuits relates to both the goals pursued alongside the nature and competencies. following the et framework, this is categorized into skills, processes, technology, and strategy. fostering a skilled workforce is paramount, necessitating the acquisition of new competencies like digital literacy and data analysis. upskilling and reskilling initiatives guarantee that employees can effectively utilize these skills for transformative purposes. fundamental to et is the reassessment and redesign of existing business processes. this can involve streamlining workflows, eliminating inefficiencies, and embracing agile methodologies, ultimately boosting operational efficiency. in addition, innovative technologies such as cloud computing, artificial intelligence, data analytics, and internet of things (iot) solutions are central to et. adopting the right technology stack empowers organizations to gather insights, automate tasks, and drive innovation in products and services. a clear transformation strategy guides the entire process. strategic planning ensures alignment between business goals and technology investments, fostering innovation and bolstering competitiveness. the entire et exercise is carried out towards reaching the end, i.e., satisfying the intended goals and realizing the planned impacts of such initiatives. as depicted in the et framework, the realized impacts on the organization can vary from mere cost savings to improved perceptions. cost reduction may involve streamlining processes, reducing overhead, minimizing waste, and maximizing resource allocation toward maintaining or improving overall performance and competitiveness. likewise, as an outcome of transformation, improved perceptions can result from enhanced customer experiences, increased trust, and brand equity. for an enterprise-level change that is more strategic, the end goals can be unique offerings or entering australasian journal of information systems kumar & thakurta 2024, vol 28, research article dt: enterprise transformation perspective 11 new markets. enterprise transformation can lead to developing new and innovative offerings that better align with market demands and customer needs. it may involve entering new markets, expanding into different geographic regions, or targeting previously untapped customer segments. the outcome is often an increased market share, a broader customer base, and diversified revenue streams. resorting to the et framework allows for a conceptualization of the dt initiative. we already recognize that the foundation of the dt initiative is in anticipation of how digital technologies contribute to value realization by impacting the work processes (henriette et al., 2015; reis et al., 2018). digital transformation in enterprises can be a fallout of revamping opportunities arising out of anticipated value deficiencies in terms of loss of opportunities (markets), threats or (risks) of failure, or a crisis in the firm (e.g., stressed financial health) (baiyere et al., 2020; hanelt, firk, et al., 2021; lanamäki et al., 2020). accordingly, the extent of transformation can be vast, spanning from a focus on the functional aspects to societal considerations (ganju et al., 2016; leong et al., 2016; pagani, 2013). consequent to identifying the risks and the opportunities, the focus is shifted to the core elements of the work processes of the enterprise towards realizing the dt. therefore, we focus on the means for the transformation, which largely depends on the organization, its processes, resources, and competencies (dremel, et al., 2017; tabrizi et al., 2019; vial, 2019). from an organizational perspective, the structure of the firm and its networks have a crucial role in supporting dt (nadkarni & prügl, 2021; verhoef et al., 2021). moreover, at a finer level, the redistribution or allocation of firm-wide assets and resources influences on the extent of success derived. the unique characteristics underlying managerial decision-making will also be decisive at an individual level (li et al., 2012; zangiacomi et al., 2020). furthermore, technology's role in data and information management, digital policies, and regulations has been observed to play a significant role in supporting dt initiatives (kumar, 2021; wimelius et al., 2021). the culmination of dt manifests in multifaceted impacts that redefine organizational paradigms. this can include streamlined business processes arising from the re-evaluation and re-design of workflows, coupled with the integration of automation (dremel, et al., 2017). embracing innovative technologies like cloud computing and artificial intelligence facilitates heightened insights, task automation, and product/service innovation (hanelt, et al., 2021). a clear digital strategy aligns business goals with technology investments, fostering innovation and bolstering competitiveness, thereby positioning the organization for sustained success in the digital landscape (kauffman et al., 2010). 3.2 theory adaptation and extension here, we focus on conceptualizing the foundational elements of dt following the theoretical foundation adopted. in order to offer a holistic and integrative framework, we remain in tandem with the previous works offering similar contributions (elliot, 2011). a comprehensive and scientific review involving two phases was undertaken to uncover and explain the foundational elements documented within the scholarship. by comprehensive and scientific review, we mean a logical article search strategy, retrieving an overarching corpus of articles and analysing and interpreting them in two phases in sequence: phase-1 using statistical means (hasan et al., 2017; sharma et al., 2021), and phase-2 involving qualitative content analysis (krippendorff, 2018). we describe these below. figure 2 presents our overall research design. australasian journal of information systems kumar & thakurta 2024, vol 28, research article dt: enterprise transformation perspective 12 figure 2. overall research design review methodology: in order to proceed with our inquiry, we adopt an explanatory design. the explanatory design allows researchers to explain patterns that may be observed to arrive at a general understanding of the phenomena (creswell & creswell, 2017). we, accordingly, resort to a mixed method design in two distinct phases – a quantitative phase (phase 1) followed by a qualitative phase (phase 2). (creswell & clark, 2017) argue that used by themselves, quantitative data are limited in explaining the nuances of a phenomenon given the complexities that may be involved. thereby, supporting qualitative evidence is required to explain or elaborate quantitative results (creswell & clark, 2017; creswell & creswell, 2017). the combination of the quantitative and qualitative phases in a sequence should provide a more grounded explanation and insights. phase 1: quantitative proceed with our exploration in the following manner: we initiate the work by reviewing the extant journal publications focusing on dt within the scopus database since 2010, given the recognition and growing importance of the phenomena since the last decade. in order to perform a comprehensive literature review, we adopt a guided strategy. foremost, the choice of keywords was zeroed on based on the following query: australasian journal of information systems kumar & thakurta 2024, vol 28, research article dt: enterprise transformation perspective 13 “title-abs-key ( "digital transformation" ) or title-abs-key ( "digital strategy" ) or title-abs-key ( "digital innovation" ) or title-abskey ( "digitalization" ) or title-abs-key ( "digital disruption" ) and ( limitto ( srctype , "j" ) ) and ( limit-to ( subjarea , "busi" ) or limitto ( subjarea , "deci" ) or limit-to ( subjarea , "econ" ) ) and ( limitto ( doctype , "ar" ) ) and ( limit-to ( language , "english" ) ).” figure 3: search strategy the above keywords were linked through a ‘boolean logic’ and searched in “metadata fields: title, keywords, and abstract of the articles in the scopus database.” scopus, one of the most widely used databases, provides superior global content coverage and gives access to relevant research published in diverse domains, including outlets focusing on management, information systems, and related disciplines (qaiser et al., 2017). the articles were restricted to those published in journal outlets and communicated in english. the above listed search descriptors resulted in a corpus of 4314 articles (documents) for further analysis. since the central element of this study was to synthesize and extend the conceptual frontier of dt, the primary objective focussed on unleashing the sub-structures of the scholarship. this objective was catered to using natural language processing and supporting statistical techniques. the use of this technique makes it possible to extract the underlying structure of the documents. specifically, the topic modelling technique assumes that documents are a distribution of a group of words, while themes are a distribution of a group of documents (blei et al., 2003; pang & lee, 2008). the underlying substructures were unveiled using advanced probabilistic models, precisely structural topic modelling. principally, structural topic modelling is used to extract the latent dimensions underlying the corpus of articles by assuming distributions corresponding to each dimension (blei, 2012; roberts et al., 2019). since a detailed explanation of structural topic modelling is out of the purview of the current article, we refer readers to appendix a for a detailed explanation. nevertheless, in summary, one may visualize that each document arises from ‘k’ latent distributions (dimensions) and is oriented to these dimensions with different magnitudes. since there are no clear guidelines concerning the optimal number of latent themes, we carried out sensitivity (ranging between 2-50 topics) for our exploration. we referred to standard metrics: exclusivity and semantic coherence to arrive at the optimal choice (blei, 2012; sanchez-franco et al., 2021). semantic coherence highlights the terms which frequently co-occur within a dimension; exclusivity echoes their unique correspondence to a particular dimension (sharma et al., 2021). the resultant analysis led to a sub-optimal value of eight for ‘k,’ indicating the eight latent themes (i.e., topics) based on the above two metrics and corresponding results (appendix b). foremost, we concentrated on the most probable or frequent word underlying each latent theme (bastani et al., 2019). to capture some incremental insights, we also referred to metrics such as “lift” and “frex,” suggested in the major “statistical text analysis” (kuhn & johnson, 2013; roberts et al., 2019; sharma et al., 2021). lift highlights the more frequently appeared and common words within a particular theme and not elsewhere (in the remaining themes). essentially, lift reflects on rare words within each theme (sharma et al., 2021). similarly, we resorted to another measure, namely, frequency–exclusivity or frex, which identifies both australasian journal of information systems kumar & thakurta 2024, vol 28, research article dt: enterprise transformation perspective 14 frequency and exclusivity of words for a particular topic. note that frex scores were alternatively referred to identify the most frequently appeared words to describe the associated scope or topic best, thus assisting us in ratifying most of the foundational elements of dt. the detailed word clouds corresponding to all eight themes were referred to for further ratification. due to brevity, we are reporting the word clouds in appendix c. phase 2: qualitative: in phase 2, which followed our initial quantitative analysis in phase 1, we engaged in qualitative inquiry to uncover the eight latent themes and provide a comprehensive interpretation of our findings. this qualitative phase is essential in mixed research designs that commence with quantitative research, as it serves to add meaning and context to phase 1 results. following the preliminary analysis, we noticed substantial overlap among the themes. consequently, a thorough examination was necessary to grasp the essence of each observed theme. thereby, the objective of this phase was to develop explanations of all the latent themes from the earlier analysis (i.e., phase 1) towards arriving at a coherent understanding of these in the context of digital transformation. to proceed with this phase, we first identified the articles for content analysis. content analysis offers a systematic approach to describing and classifying text material (krippendorff, 2018). we adopted a systematic approach to synthesize all the explored latent themes thoroughly. structural topic modelling (phase 1) presented a flexibility for the textual observations, articles in our case, to be aligned to each topic with different magnitudes (i.e., scores) of orientation (roberts et al., 2019; sharma et al., 2021). this contrasts with singlemembership models or deterministic techniques such as clustering (ding et al., 2020; fresneda et al., 2021). based on the descending ordering of scores of article orientation corresponding to each topic, we filtered the articles most aligned toward each topic. this exercise allowed us to retrieve the top 30 articles with orientation scores ranging from 0.65 -0.90 (on a scale of 1). this criterion indicated that the selected article's contribution (s) are dominated (more than twothirds) by the corresponding latent theme having minimal overlap with other dimensions. the identification process led to a basket of 240 articles spread across eight topics of interest that emerged from the preceding analysis. these articles were further screened for appropriateness. the screening procedure included a manual review of the abstract, introduction, and conclusion (if required) to determine if the article in question is attending to an aspect related to the dt phenomena. this was done as articles can resort to keywords like digital transformation or digitalization given the setting or the context, while the focus of the article may be some other phenomena, e.g., marketing or consumer behavior (chu et al., 2019; stewart et al., 2019; victor et al., 2019). this process led to a shortlisting of 182 articles for a detailed analysis. through the review of the shortlisted articles, we primarily aimed at having a nuanced understanding of the themes. this started with a bottom-up approach where we first ingested the evidence from each article within the corresponding latent theme. later, we synthesized these evidence to come up with broad perspectives underlying each theme. this also empowered our analysis to organically interpret the tenets of each latent theme. for this purpose, the spreadsheet application was used, and supporting evidence emerging from the articles were outlined. corresponding to each latent theme, the topic words (primarily the top words; please refer to table 2) emerging from phase 1 results were treated as initial codes. these codes served as the foundation for relating textual evidence from the assigned articles australasian journal of information systems kumar & thakurta 2024, vol 28, research article dt: enterprise transformation perspective 15 to their corresponding latent themes. the evidence from the selected articles was synthesized accordingly at the latent theme level, contributing to explaining the latent theme. the authors of this paper carried out the task independently, with the first author dealing with articles belonging to the first, fifth, sixth, and seventh latent themes and the second author dealing with the rest (second, third, fourth, and eighth). this distribution between the authors was purely based on their areas of interest and expertise. furthermore, to remain consistent, the analysis done by one was reviewed by the other, and observations were discussed and settled. 4 findings and demonstration 4.1 review results table 2 presents the results from our two-phase analysis. in the table, the proposed conceptualized foundational elements are mentioned in the first column. the second column, “topic identifier,” presents the themes underlying the dt scholarship. here we also include a short descriptor to highlight what the theme is all about. the third column includes, for each theme, a sub-set of the top words characterizing the theme. our phase 2 analysis helped us understand the broad perspectives of each theme. the articles selected within the corresponding theme(s) reflected nuances of the theme, which helped us interpret it. resultantly, in the fourth column, we present an interpretation of each corresponding theme. the final column, “rank,” highlights the themes' relevance and importance in the dt context. in the paragraphs below, we elaborate on the table 2 findings. given that our findings are grounded on the evidence from the dt literature following our two-phase exploration, we include relevant citations to support our explanations. the first foundational element (i.e., “scope”) includes topic 5 (table 2). the recognition of value deficiencies, either as untapped opportunities or threats and uncertainties eroding the value creation potential of firms, earmarks the purpose of any transformation. our detailed analysis of selected papers helped us outline two broad triggers for attending to value considerations: external and internal. by external, we mean the potential opportunities and growth prospects outside the boundary, while internal points toward the deficiencies in the existing practices/processes or offerings (chanias et al., 2019). therefore, the primary objectives for any transformational initiative involve exploring untapped opportunities through introducing new products, entering new markets, or both. competitive forces and industry dynamics can introduce new opportunities in the business environment, prompting companies to refresh and revitalize themselves (koch & windsperger, 2017). consequently, to stay relevant, firms adjust their digital strategies, by entering new markets or revitalizing current practices (tan et al., 2020; yeow et al., 2018). another crucial objective is addressing crises and threats associated with existing practices (vial, 2019). this involves reshaping value propositions and reconfiguring systems to enhance overall value creation (guo et al., 2021; zutshi et al., 2021). the above deliberations echo the fundamental considerations driving the dt endeavour by showcasing the opportunities around dt. these opportunities determine the extent of dt (berghaus & back, 2016) indicating whether the transformation is enacted at a process level, within a specific function, or across the entire organization. traditionally, it strategy has been useful as a functional-level strategy. however, with the emergence of digital technologies, the controls have percolated from a unit to the entire landscape of firms (tabrizi et al., 2019). australasian journal of information systems kumar & thakurta 2024, vol 28, research article dt: enterprise transformation perspective 16 instances are documented in the extant literature where leveraging on the digital infrastructure, industrial boundaries are crossed, contributing to sectoral changes (kane, 2016, 2017). simultaneously, it contributes to driving societal impact (benzerga, 2016; leong et al., 2016). the preceding discussion presents the broad landscape of a dt endeavour by showcasing the opportunities contributing to the purview of the phenomenon. these observations delineate the purpose of any transformational journey, defining the “scope”, i.e., the need and level of transformation. we now shift our attention towards the “ways” of value creation, which fundamentally highlight the configurations inherent in any dt endeavour. topic 6 hints at the role of digital technologies in facilitating product innovation and customer-centricity; however, the guiding principle and roadmap are always set by the generic business strategy (lipsmeier et al., 2020). with the proliferation and percolation of digital technologies, scholars have recently advocated the concert and synchrony of formulating a business strategy coupled with digital technologies (tabrizi et al., 2019; tanriverdi & lim, 2018; vial, 2019). this has interchangeably become popular as a digital strategy leading to innovative solutions, products, and new markets for significant revenue generation (catlin et al., 2018; westerman, 2017). in this parlance, independent units and entities collaborate to create value, which is far more than their standalone value-creating capabilities (chang et al., 2014). historically, conventional it artifacts played a role in distributed value creation within a network. however, they fell short in co-creating heightened value, a capability now achievable through utilizing a wide range of digital technologies (nischak et al., 2017). for example, platforms and innovative ecosystems have allowed crossing industrial boundaries to create new markets and prospects. thus, this constituent theme delineates a crucial roadmap for realizing value by capitalizing on the potential of value co-creation with the capacity to reshape industrial structure (constantinides et al., 2018; kauffman et al., 2010). topic 7 focuses on incremental innovation, through product creation, packaging, or combining product and service (together), for increased benefits (pagani, 2013). the associated articles present various aspects connected to streamlining and automating manufacturing processes. emphasis on digitalizing supply chain and logistics management to enable efficient and sustainable operations is visible in these conversations (boyes et al., 2018; dalenogare et al., 2018). the rapid pace of technological advancement has fostered increased productivity and scaled efficiency for better customer engagement and value creation (kumar, 2021). digital technologies play a key role in anticipating and addressing risks related to uncertain demand, rapid market change, and heightened customer experiences (pagani, 2013; vial, 2019). australasian journal of information systems kumar & thakurta 2024, vol 28, research article dt: enterprise transformation perspective 17 foundational elements topic identifier (select) topic words interpretation rank scope topic 5 (need) value deficiencies (level) function; chain; firm; industry; sector; society [purpos(e), growth, market, improv(e), global, risk, problem, process] at one end, the words characterizing the theme reflect the opportunities contributed by identified value deficiencies. at the other end, the words reflect the span of transformation. focusing on the words describing these opportunities, such need can be external, such as triggered by anticipating potential growth opportunities, or internal, by sensing concerns and risks in the ongoing practices (chanias et al., 2019). thereby, the constituent words refer to value deficiencies in terms of risk, process, needs for improvement, and scope for growth in new markets with global presence. the above aspects, referring to the opportunities, determine the purview of the dt. the internal deficiencies result in functional or process-level transformation, or heightened growth may foster a firm or an enterprise-level transformation (riasanow et al., 2019). since digital technologies enable change, they have the potential to cross industrial boundaries and revolutionize the sector leaving societal marks. we observe corresponding words such as sector, industry, and society dominating the discussion within this theme. 2 ways topic 6 value co-creation: innovative solutions (both product and service) entering new markets, developing new channels tapping growth opportunities. [innov(ation), transf(ormation), ecosystem, platform, (value), proposition, creation] the topic words here earmark an essential roadmap for value creation. the thrust is on the collaboration and networking of various entities, which can together co-create more value than their individual valuecreation capabilities (kauffman et al., 2010). the transformational element spans across all the entities in the network that are digitally connected (nischak et al., 2017). platforms and innovative ecosystems have the potential to create more value for each contributor while passing the maximum benefits to the endpoint, which is the customers (constantinides et al., 2018). 1 topic 7 value re-creation: fallout of losing business to competition., risks of failures, threats from the disruption fostering internal deficiencies [manufactur(ing), supply chain, function, procure(ment), manag(ing), energy, sustain(ing)] the topic words allude to another avenue for value creation, primarily through incremental innovation achieved by either crafting distinctive content or repackaging content and services in a redefined manner (leischnig et al., 2017). ultimately, this approach leads to the redefinition of practices or offerings, fostering further value generation. for example, as the keywords refer, these can be around supply chain restructuring, procurement, manufacturing. the discussion and keywords advocate for the transformation of fragmented networks of strategic partnerships. these endeavors aim to address risks stemming from uncertain demand and rapid market fluctuations (pagani, 2013). 5 means topic 4 organizational capabilities and attributes [organiz(ation), firm, culture, manageri(al), resource(e), structur(e), conduct, readi(ness)] the topic words here relate to the organization or its attributes (e.g., structure, culture). the evidence resonates with the element classified under means. the aspect has a clear implication on the resource (re)-allocation and managerial abilities. the literature documents that any transformational endeavor requires a delicate balance of capabilities and redistribution of resources (tabrizi et al., 2019; vial, 2019).furthermore, the organizational attributes such as the leadership, composition structure, and culture, as the terms allude to remain pivotal for bringing in any change (dremel, et al., 2017). 3 australasian journal of information systems kumar & thakurta 2024, vol 28, research article dt: enterprise transformation perspective 18 topic 3 digital talent [train, technolog(y), workplace, academ(ic), compet(ency), learn, adapt, challeng(es)] the topic words here demonstrate the importance of the individual abilities that are instrumental in supporting dt endeavours. the words relate to competency development, training, learning, and adaptability requirements of the workforce, focusing on the supporting technologies that are considered essential for embracing the changes (kane, 2019). the entire ecosystem needs to train and retain practices and resource people with the desired skill sets to address people-specific dt challenges (kane, 2017). the importance of academic partnerships is also explicit in conversations supporting the skill requirements. 4 topic 2 digital governance [govern(ance), infrastructure, process, group, relat(ion)] the topic words broadly relate to digital governance as a foundational aspect underlying dt endeavours. the keywords refer to descriptors around digital governance. the words refer to infrastructure readiness which acts as the precursor to a digital establishment. in addition, the supporting processes and the interlinkages with the supporting groups characterize the governance structure, supporting the transformational realm. the nature, extent, and outcome of transformation is a result of how various entities exercise their ownership, authority, control and contribution in the complex network of digital interconnectedness (constantinides et al., 2018; kauffman et al., 2010). 6 topic 1 big data analytics (bda) [(big) data, network, connect, analytics, cloud, mobil(e)] the terms characterizing the theme resonate with the nuances of bda. the topic words refer to various digital technologies, for example, mobile, analytics (e.g., big data), cloud, and networking, which are imperative for the landscape of dt. the observed keywords pertain to technologies assisted by artificial intelligence linked to big data analytics (gupta & george, 2016; wamba et al., 2017). the dt literature advocates the role of bda as a critical success factor in shaping digital transformation, particularly by leveraging vast amounts of data to supplement evidence-based decision-making (dremel, et al., 2017). 8 ends topic 8 impact (costs, perceptions, offerings, markets) [perceive(e), satisfact(ion), experi(ence), impact, media, brand, consum(er)] this refers to the various consequences arising from the transformation. value (re)-creation can be realized through improved consumer satisfaction, stakeholders’ perception, and increased brand equity. as organizations navigate dt, they not only seek to optimize costs through streamlined processes but also influence stakeholder perceptions, fostering an enhanced and contemporary organizational image (kane, 2016; svahn et al., 2017). this transformative journey extends to offerings, where dt catalyzes the creation of innovative products and services (constantinides et al., 2018). these changes reverberate through the market landscape, enabling organizations to competitively position themselves by realizing cost efficiencies, increased revenue, and growth in market share (leischnig et al., 2017). indeed, dt initiatives hold the potential not only to benefit individual firms but also to transform entire industries by addressing mass concerns and yielding sectoral outcomes (agarwal et al., 2010; leong et al., 2016). 7 table 2. digital transformation conceptualization australasian journal of information systems kumar & thakurta 2024, vol 28, research article dt: enterprise transformation perspective 19 thereby, this particular avenue for value creation emphasizes the incremental or recreation of value. ultimately delivering the enhanced benefits to the end consumers (leischnig et al., 2017). the third foundational element (i.e., “means”) presents an understanding of the support mechanisms for a dt endeavour and includes topics 1, 2, 3, and 4 (table 2). articles representing topic 1 embody big data analytics (bda) that has immensely benefitted business tasks and processes. over the years, bda has evolved as an essential aid (gupta & george, 2016; wamba et al., 2017) rather than just a computational problem-solving resource (kumar, 2021). therefore, the observance of bda as a constituent of dt was not surprising. evidence from the constituent articles posits multiple viewpoints relating to bda. first, the focus is more on architectural considerations, such as integration of different sources for gathering data from multiple touch points (e.g., social, mobile, cloud) and in varying formats (zimmermann et al., 2015). the sources could be internal and external, and their proper storage and maintenance are instrumental. in this context, robust big data architectures are required to handle high-speed and voluminous data flow (correani et al., 2020). another set of scholars has considered analysis as another vital component, underscoring the importance of deriving insights and incremental knowledge from robust bda architectures (koch et al., 2021; verma & bhattacharyya, 2017). the widespread use of computational intelligence, such as data mining tools, machine learning, natural language processing, etc., for prediction and providing solutions to a multitude of problems has established the potential of bda to drive transformational initiatives (sivarajah et al., 2017; verma & bhattacharyya, 2017). bda, in a nutshell, focuses on the business value underlying vast amounts of data, which fosters experience-based decision-making to understand profound customer experiences, anticipate uncertain demand, and prepare for sudden market change (dremel, et al., 2017). concerning topic 2, the articles describing the theme present various issues related to governance around digital transformation. the discussion broadly concentrates on the digital means (e.g., digital infrastructure), identification of groups (e.g., experts), designing the processes, and the governance structure (dunleavy & margetts, 2015; kellogg, 2021; nauhaus et al., 2021). all these, in combination, reflect the essence of digital governance, broadly understood as a framework for establishing accountability, roles, and decision-making authority for an organization's digital presence (fitzgerald et al., 2014). summarily, digital governance is a constitution that keeps businesses and people aligned with the evolution of digital technologies (charalabidis & lachana, 2020; dunleavy & margetts, 2015). digital governance serves as a clear guide for various entities and actors within the complex network, enabling them to navigate their roles, exercise authority, uphold rights, and align incentives for the purpose of value creation (constantinides et al., 2018; gasser & almeida, 2022). topic 3 concentrates on digital talent. talent encompasses the skills, experience, knowledge, talents, intelligence, and competencies the employees need to resolve various situations and problems the organization faces (ugboego et al., 2022). talent is critical, the absence of the right skills is a challenge for organizations (papagiannidis et al., 2020). the articles on the theme focus on the skill shortage, the requirements for reskilling and upskilling of talents, skill requirements, and the necessary support ecosystem (e.g., academic partnerships, curriculum, technology-mediated learning environment) to showcase the importance of digital talent and its relevance in the context of dt (bennett & mcwhorter, 2021; herczeg, 2021; klus & müller, 2021; manesh et al., 2021). australasian journal of information systems kumar & thakurta 2024, vol 28, research article dt: enterprise transformation perspective 20 the discussion in the articles representing topic 4 pivots around the organization. these articles aim to identify organizational-level attributes and capabilities that may facilitate dt. we can note deliberations around various capabilities (e.g., it capability, digital platform capability, network capability) that are necessary for dt (gasser & almeida, 2022; sebastian et al., 2017; tabrizi et al., 2019). a section of reviewed articles also discusses the role of leadership and organizational structure (matsunaga, 2021). cultural aspects (organizational eagerness for dt, employee mindset toward change) have also garnered attention in these communications (halpern et al., 2021; ritala et al., 2021). past studies highlight the instrumental role of leadership, structure, culture, hierarchy, size and age in any transformational journey (faulkner & runde, 2013; riasanow et al., 2019; tabrizi et al., 2019). in the context of dt, particular emphasis is placed on the redistribution of existing resources and capabilities (dremel, et al., 2017; riasanow et al., 2017). the fourth and final foundational element (i.e., “ends”) includes topic 8 (table 2). the articles within this theme focus on the impact of dt, addressing the potential effects. dt contributes significantly to organizational cost efficiencies (mufraini et al., 2020; penco et al., 2021). by leveraging digital technologies, organizations streamline processes, optimize resource allocation, and enhance operational efficiency, ultimately leading to cost benefits (kumar, 2021). simultaneously, dt strategically shapes stakeholder perceptions, emphasizing improved customer engagement, relationship-building, and enriched experiences (choi et al., 2020; hirata, 2019; rodgers et al., 2021). this external impact aligns with the transformative goal of creating favourable impressions and positioning organizations as customer-centric entities. the evolution of offerings is a central theme, where dt stimulates the design of innovative products and services, creating distinct value propositions aligned with market demands (constantinides et al., 2018; kauffman et al., 2010). furthermore, dt's transformative influence extends to entering new markets and reshaping entire sectors, exemplified in healthcare, financial services, and education (felgenhauer et al., 2017; henderson et al., 2015; kane, 2016, 2017; park et al., 2019). the transformative impact is observable in improved health outcomes, enhanced financial access, quality education, and the potential to reconfigure geographical landscapes (ganju et al., 2016; leong et al., 2016). thus, the effects of dt are profound and extend beyond economic value delivery, influencing consumers, fostering profitability, and generating incentives for all market players. supported by the preceding explanations and following our two-phase analysis, we offer our framework of dt (figure 3). the framework includes four components that represent the foundational elements (table 2) of a dt endeavour, binding the constituent themes of our conceptual framework. the proposed framework, grounded on the et framework, addresses our research question regarding the integral elements of a dt initiative, encompassing aspects such as purpose and extent (scope), the path for value creation (ways), constituents (means), contributing to the impact (ends). our framework integrates these essential components to describe the diverse facets that characterize dt. our dt framework is theoretically grounded and comprehensive, offering a canvas to describe dt initiatives in practice. to demonstrate the same, we resort to a case study-based inductive approach. the subsequent section elaborates on the case and its findings to validate the sanctity of the proposed dt framework. 4.2 framework relevance to showcase the relevance of our framework, we consider two cases (i.e., general electric and siemens ag) demonstrating divergent experiences with dt. case studies are ideal for australasian journal of information systems kumar & thakurta 2024, vol 28, research article dt: enterprise transformation perspective 21 examining dt in action because they enable exploration of a timely phenomenon “within its actual setting,” mainly when the distinctions between phenomenon and setting are unclear or vague (yin, 2018). a multi-case approach was taken because the knowledge obtained from replicated case studies enables comparisons, which contributes to external validity (yin, 2018). as our focus is not on external variations, we chose these cases since they represent companies belonging to the same domain and have different approaches to dt. general electric (ge) and siemens represent highly diversified conglomerates, with various divisions operating in the same domain. these two companies have historically built their reputations on producing cutting-edge machinery like jet engines, locomotives, or industrial automation systems. both are pitched as one other’s rival and, as per reports, have 70% of the businesses2. we base our cases on published sources3,4 supported by the company, and media reports available online. we have included the overview of these cases in appendix d. we use the terms dt and digitalization interchangeably in our case description and analysis. digitalization refers to the diverse socio-technical phenomena and processes that entail adopting and using digital technologies in larger organizational or societal contexts (legner et al., 2017). the term has been often used to imply dt (haffke et al., 2017). for analyzing the cases, we draw on our dt conceptual framework components. table 3 lists how our proposed framework components are manifested in the descriptions of these two cases, thereby demonstrating their relevance. scope ways means ends figure 4. proposed digital transformation conceptual framework 2 https://www.economist.com/business/2016/12/03/siemens-and-general-electric-gear-up-for-the-internet-ofthings 3 austin, r.d., & pelow, g. (2019). dt at ge: what went wrong? w19499. ivey publishing (publication date: september 13, 2019) 4 collis, d.j., & junker, t. (2017). digitalization at siemens. hbs no. 717-428. harvard business school publishing (publication date: february 26, 2017) need level value cocreation value recreation organizational capabilities & attributes digital governance big data analytics digital talent impact australasian journal of information systems kumar & thakurta 2024, vol 28, research article dt: enterprise transformation perspective 22 general electric siemens ag scope • absence of value assessment of the proposed platform • lack of consideration of existing deficiencies • inadequate consideration of issues around dt scoping ambitions • clear delineation of dt goals • identification of value enhancement opportunities through dt • emphasizing dt potentials across all businesses ways roadblocks: • absence of strategic considerations either at the business unit or at the firm level (e.g., external opportunities) • disparate preferences of ge digital and other business units thrust areas: • focus on initiatives to strengthen different business units’ contributions for co-creating value • focus on strategic considerations with a focus on customer centricity means • dt planned as a centralized initiative in a decentralized structure • lack of relationship capital in ge digital • leadership displaying over-confidence • leadership failure in visualizing the dt approach • software getting used more as a process enablement internally • culture hampered transparency • dearth of pertinent talent • setting up of divisional organizational structure tailored to customer groups • shifting of dt ownership to divisions • establishing a committee to oversee dt • visible leadership commitment • software platform, enabling advanced analytics towards leveraging the potentials of internet of things • establishing a digital culture • talent acquisition strategies focusing on desired competencies through necessary alliances ends • focus on short-term gains • inadequate customer perception • inadequate appreciation in market of industrial internet’s potential • realization of cost benefits and growth opportunities • consolidation of the company’s image • tapping growth opportunities through a customer value-focused approach table 3. evidence from the cases general electric: a dt initiative needs to have a clear and consistent focus from the onset. this should involve identifying value deficiencies and opportunities having the potential for value creation. in the absence of a similar focus, for large conglomerates like ge, a dt can quickly become overwhelming. ge rolled out its software platform predix intending to develop one common platform for all its business units. the platform was created to assist in networking, managing, and analysing enormous volumes of data gathered from industrial devices to assist with the automation of processes. ge did succeed in digitizing and automating several of its business processes. however, harnessing potential opportunities where operations are reinvented, flaws are addressed, and a new, digital-first business model is supported, remained out of reach. clearly, ge lacked proper planning regarding dt. as per reports, the effort was driven solely by monetary considerations and not from the pain points in the existing offerings. the scope of ge’s digital ambitions was very apparent when the company set up ge digital as its own business within the industrial conglomerate to propel its dt initiatives. ge digital aimed at centralizing all of the it operations of the company. even though ge digital was meant to substantiate the firm’s data analytics resource(s) and present general electric as a more technology-oriented enterprise, the objectives of the other business units (e.g., ge power australasian journal of information systems kumar & thakurta 2024, vol 28, research article dt: enterprise transformation perspective 23 (wind turbines), ge aviation (jet engines), ge transportation (railroads), and others) were strikingly different. despite the grand aims, the predix software mainly served as a platform to promote internal initiatives. the various business units, led by the respective ceos (i.e., chief executive officers), had different it requirements to be supported. hence the efforts were less about business transformation and appeared more as it-enabled process improvement. more importantly, the external opportunities did not get the required attention since most of the revenue was realized from existing markets and channels. despite significant expenditures and best-in-class people, ge struggled with digital efforts owing to the imbalance between the business demands and capabilities. ge had a decentralized governance structure, while the digitalization initiative was planned as a centralized, top-down initiative of ge digital. this implied that a decentralized structure within ge needed to operate in a new coordinated manner. there were, thus, both mismatches and deficiencies at multiple levels. the ceo in charge of dt displayed more overconfidence around the planned initiatives. this spawned a culture in which the employees believed that the ceo did not want to hear discouraging reports; thereby, the actual problems around the transformation initiative got suppressed. additionally, ge created a massive organization that was not yet necessary rather than tasking a small team to create the best product and then allowing the operation to grow with the product's evolution. the leadership hurried through the transitional effort for which the company was unprepared. adding to the concern, ge digital lacked relationship capital with the rest of ge. roughly ge digital had only 2% of employees who had worked in other business units of ge, which acted as a bottleneck while rolling out the predix platform in other units. there needed to be more internal resources and capabilities to support the transformation initiative. ge needed to acquire new technical and analytical capabilities. it also required strategic alignment and proper planning for the dt initiative. all these requirements proved a burden for the company in the face of a hurried transition effort. customer perceptions regarding ge’s digital initiative were not very bright. the majority of ge’s stakeholders had trouble understanding its proposition. ge’s market offerings suffered a hit in the absence of organizational readiness of the customer organizations. a vast majority of ge’s consumer base did not appreciate the potential of the industrial internet and hence failed to realize the value of the offerings. simultaneously, ge digital placed more importance on profit and loss accounts and made quarterly commitments on performance. this constrained the organization to short-term objectives that failed to contribute any significant value to the company's mission. siemens ag: from the onset, siemens made clear the core objectives driving the dt initiative. vision 2020 had clearly identified goals such as a cost reduction of €1 billion and creating sustainable value with a targeted return on capital employed of 15-20%. further goals were set for a twenty-percentage improvement in the company’s “net promoter score” and a seventy-five percent “approval rating” in terms of “leadership and diversity” in an engagement survey to increase the company’s brand image. driven by these objectives, the management at siemens was early to recognize the opportunities digitalization can bring to the company. for example, focusing on the service component of all its businesses, and with access to data from all machines, maintenance can be predictive (rather than scheduled), thereby contributing to machine uptime improvement and the resultant gains. australasian journal of information systems kumar & thakurta 2024, vol 28, research article dt: enterprise transformation perspective 24 the digitalization at siemens was a decentralized initiative. the digitalization initiative focussed on encouraging all its businesses to innovate critical digital services and to support the development of a shared platform. the key to this initiative was to motivate all business units to collaborate and incorporate new technologies for combined value creation and crafting competitive advantage in each business. the different businesses of siemens leveraged the industrial cloud-based data platform mindsphere5, allowing these units to develop specific components that addressed their problems. hence, the internal initiatives concentrated on how the different businesses with siemens took advantage of the digital offerings to their benefit. the external opportunities were also explored around value creation, combining its knowledge about the customer needs in various industries, such as railroads. the digitalization initiative at siemens was supported at various levels. from the organizational perspective, siemens followed a new structure, where the organization was split into nine divisions, each serving distinct customer groups. to streamline the digitalization efforts, a committee was set up, which consisted of the chief technology officer as the official leader of the initiative and the different divisional ceos. the committee met regularly and evaluated the progress various divisions made in their digitalization initiatives. this shifted the ownership to all divisions responsible for driving their digitalization efforts. the leadership commitment to the digitalization effort at siemens was noticeable. the ceo summits were launched and planned bi-yearly to raise awareness among its employees regarding how these developments would impact the various businesses. the annual strategy review was undertaken to keep the business units accountable for the digital plans. this ensured that the corporate strategy focused on the priorities and contributed to the divisions' core strategic direction. culturally, the shift was meant to transform from being a “family” that provided “lifetime employment” to becoming a "sports team" with a shared identity, where members could be replaced if they did not deliver. the interactions among the digitallyoriented top executives also helped to establish a digital culture within siemens. siemens also planned its resource acquisition strategies as it sought to incorporate digitalization. the company's biggest challenge was finding people with analytical skill sets and business acumen. the company also developed partnerships with several universities to attract the necessary talents in the required realm. siemens's initiative focused on customer relationships and concentrated on creating value for the customers. one of the top priorities of the digitalization initiative was to improve efficiency. siemens also realized cost benefits through its industrial data analytics initiative, catering to the entire industry. these developments enabled siemens to effectively expand into and capitalize on new growth opportunities and eliminate those areas that were no longer carrying their weight. in this manner, siemens regained its position as a top industry player and significantly outperformed its competitors in agility, resilience, and profitability. the above elaboration of the two implementation examples in siemens and ge helps us realize the importance of the various elements of the dt conceptual framework. focusing on the scope, 5 https://plm.sw.siemens.com/en-us/insights-hub/start/ australasian journal of information systems kumar & thakurta 2024, vol 28, research article dt: enterprise transformation perspective 25 the opportunities for dt stem from the very purpose driving such initiatives. the above delineation portrays a lack of value considerations at ge, while the initiative at siemens attended to the opportunities and remained grounded on the purpose driving the initiative. digital transformation at ge was primarily confined to business units; however, the planning had shortcomings. at the same time, siemens’ initiative was more balanced in terms of both unit-level focus and firm level considerations. the pathways to ge's digital ambition encountered hurdles, marked by the absence of a strategic emphasis and visible incongruency between ge digital and the business units. these impediments significantly constrained ge's approach to dt. in contrast, siemens crafted a roadmap for dt that prioritized initiatives to strengthen various business units' contributions, coupled with an emphasis on strategic considerations centred around customer experience and expectations. leveraging these factors, siemens adeptly drove their dt efforts. the end results showed more promise for siemens as the case records. 5 discussion to the curious mind, the familiarity with the rich and diverse literature might lead them to ponder what exactly dt is. as selected commentaries have pointed out, assuming that dt is distinct while lacking a conceptual foundation puts us at risk of recreating the wheel that is devoid of a transparent and unique implementation proposal (nadkarni & prügl, 2021; wessel et al., 2020). accordingly, this article establishes two objectives for examining the foundational elements of dt: first, to establish a theoretical perspective and second, to recognize the foundational elements. driven by these objectives and following a two-phase literature synthesis, our study culminates into a dt conceptual framework. further validation is achieved through multiple case analyses to add clarity and offer a reference basis to understand, discuss, or even compare the phenomenon. the proposed dt conceptual framework extends the et framework on which it has been grounded. our analysis posits “ways” as a new dimension relevant to understanding the dt phenomenon. ways represent the choices or the initiatives that may enable an enterprise to direct its digital transformation efforts. broadly, this resonates with the strategic initiatives around a dt endeavour. as our findings portray, dt offers a distinct roadmap on value cocreation and re-creation; nevertheless, this viewpoint has never been emphasized in the extant literature. value co-creation centres around innovative solutions that tap into growth and potential opportunities. through the integration of innovative technologies, dt fosters ecosystems where various entities collaboratively contribute to and benefit from value cocreation. platforms and interconnected systems play a pivotal role in this, transcending industrial boundaries and facilitating the convergence of diverse stakeholders. value recreation, on the other hand, emphasizes incremental innovation for enhanced benefits. this may involve a comprehensive transformation of functions and processes, resulting in the redefinition of practices or offerings to foster further value generation. these emphases on value co-creation and re-creation also allow us to position dt uniquely. the core of dt resides in generating value through specific pathways, establishing it as a strategic initiative. this sets dt apart from other initiatives, primarily focusing on creating value in isolation. based on the synthesis of the extant literature, our framework offers scope to discuss its relevance to the extant knowledge base. as previously mentioned (section 2), we note several process models and frameworks in the literature around dt (e.g., hanelt, et al., 2021; mergel et al., 2019; nadkarni & prügl, 2021; vial, 2019; warner & wäger, 2019), each shaped by its australasian journal of information systems kumar & thakurta 2024, vol 28, research article dt: enterprise transformation perspective 26 respective objectives and examination lens. for example, the dt framework offered by vial (2019) emphasizes technology as the "means" while remaining silent on other organizational attributes such as governance. furthermore, specific strategies are indicated but lack considerations of pathways that generate value. our results are also consistent with some seminal contributions examining the notion of dt. for example, vial (2019) also outlines the broad components for understanding dt as a phenomenon. our framework structure also resembles commonality with the dt definitional components (mergel et al., 2019) and includes the necessary themes that have already received importance in the extant attempts around dt nature and conceptualization (riasanow et al., 2019; warner & wäger, 2019; wessel et al., 2020). our distinction from these presentations lies in our explicit consideration of value generation mechanisms, specifically focusing on value co-creation and re-creation, as emphasized earlier. this differentiation is achieved by adopting a transformational perspective grounded in the et framework. in addition, the structural aspects of the framework also warrant a mention. first, we compare the proposed framework with the et framework (figure 1) in terms of the common dimensions: scope, means, and ends. focusing on the scope dimension, the dt framework includes the primary elements as the need (symbolizing opportunities for dt) and the level (indicating the purview of dt). “need” emerges as a new entry in the context of the dt framework, signifying opportunities determining the level of dt. contrasting the “scope” dimension of the et framework, the purview of dt is much broader, extending beyond organizational boundaries. focusing on “means”, this dimension echoes the importance of the current digital shifts. the underlying elements: digital talent, big data analytics, and digital governance make these preferences explicit while relating to skills, technology, and processes characterizing the et framework. the remaining element of the proposed dt framework, i.e., organizational capabilities and attributes, demonstrate organizational preparedness for driving dt. this also serves as the basis for formulating the strategy, representing the remaining element within the means dimension of the et framework. finally, our findings concerning the “ends” are identical for both the frameworks. the topic words co-existed in our case, alluding to costs, perceptions, offerings, and markets. consequently, we have designated this element as impact, encompassing these effects of dt. we also focus on the importance (as rank) of the themes from table 1 (column 5) results. the strategic relevance of dt is already recognized by the corporates, and the perceptions are similar within academia. based on our results, we observe that topic 6 (value co-creation) emerges as the most prevalent (i.e., rank 1). this further reinforces that dt is about developing collaboration and networks as part of strategic initiatives by leveraging digital technologies (tabrizi et al., 2019; vial, 2019; wessel et al., 2020). next, we observe that topic 5 (need, level), signifying the scope, occupies the second seat of prominence (i.e., rank 2) in the dt literature. the theme delineates the span of dt by highlighting opportunities, prompting firms to innovate or adapt in challenging times (kumar, 2021). this is attainable by leveraging the opportunities for value co-creation, further corroborating the importance of the two themes (i.e., topic 5 & 6) in unison. topic 4, signifying organizational capabilities and attributes, follows the third rank. previous literature and the underlying theories have conceptualized dt as another synonym for strategic change or renewal, thereby highlighting the intangible aspects as support mechanisms for dt. the theme points to management preferences, which are critical to a dt australasian journal of information systems kumar & thakurta 2024, vol 28, research article dt: enterprise transformation perspective 27 endeavour, such as redistribution of resources, leadership roles, etc. as corroborated in the extant literature (gasser & almeida, 2022; sebastian et al., 2017; tabrizi et al., 2019). in addition, there is a need for a delicate balance of structure, culture, and people (imran et al., 2021; sia et al., 2016) to drive the dt endeavour, which also gets emphasized through the keywords characterizing the theme. along with the leadership approval, the execution of a dt endeavour rests on the associated workforce's mindset and skill (boneva, 2018; ivančić et al., 2019). topic 3 (digital talent) thereby emerges as the next (i.e., rank 4) in order. given that dt involves fundamental changes, openness to training, learning, and flexibility to adapt are pivotal for implementing the changes within an enterprise (ossiannilsson, 2018; trenerry et al., 2021). the importance of employee-specific skills in managerial and non-managerial roles is already established, and digital talent has started gaining attention around the dt phenomena (kim et al., 2020). the prominence of topic 1 (big data analytics), representing specific digital technological aids, is observed to be the least. while surprising, this result can be explained by the hygiene and motivational factors relevant to stakeholders driving the transformation. as per the “twofactor motivation theory,” also commonly called “herzberg’s motivation-hygiene theory” (herzberg, 2018), the hygiene factors correspond to elements that do not contribute to satisfaction or motivation concerning the tasks at hand, whereas the motivating factors lead to satisfaction. digital technologies serve as the vehicle for the transformation and hence might have been viewed, from the transformational context, as the bare minimum needs (i.e., hygiene). our study thus underscores bda as an indispensable landscape for successful dt endeavors (kane, 2017; tabrizi et al., 2019; westerman et al., 2014). 6 conclusion 6.1 summary the importance of dt in the present context is absolute. given the proliferation of its significance, the quest for clarity on the phenomenon is indispensable. nevertheless, the popularity of any field is coupled with the challenge of falling apart (er, 1988; kumar & thakurta, 2019). thus, owing to a shared understanding of dt in the extant literature, it is essential to clarify its foundational elements. accordingly, the article adopts a transformational perspective, using the lens of the et framework to examine dt based on our research question. our examination resorts to a two-phase synthesis of the extant literature around dt, involving a quantitative text-mining approach in the first phase, followed by qualitative content analysis in the second phase. results based on the combined inquiry allow us to propose the dt conceptual framework to explain dt and satisfy the research objectives. 6.2 contributions and limitations the work makes contributions to both theory and practice. first, the primary theoretical contribution is the theory-grounded framework that we offer in this piece, which is expected to serve as a notable contribution to the body of knowledge around dt. the framework comprising four foundational elements assists in reducing the intricacies of the phenomenon by offering a tangible directive. second, our framework differs from other dt frameworks by explicitly considering the (enterprise) transdisciplinary and transformational viewpoint, which is missing in the extant contributions. third, the proposed framework extends the et framework on which it is grounded by incorporating a new dimension (i.e., ways). this australasian journal of information systems kumar & thakurta 2024, vol 28, research article dt: enterprise transformation perspective 28 addition reinforces the strategic nature of the phenomenon, emphasizing the significance of specific choices and initiatives within an enterprise's dt journey. the added dimension enhances the framework's ability to capture and guide strategic efforts in the complex landscape of dt initiatives. by offering this conceptualization, we are further able to offer a narrative to supplement the phenomena from a governance point of view. indeed, as the preceding discussions have revealed, several of the extant notions of dt have failed to clarify the importance of roles, responsibility, authority, and ownership of various entities involved in the transformational endeavour. we earmark digital governance as a crucial aspect that will regulate and govern collaborative practices in the digitally advanced ecosystem. through clarifying the nature of dt, our study assists managers in anticipating the complexities enveloping a transformation journey. irrespective of the transformational objectives, the goal has been mostly to leverage digital technology in redefining their value propositions and identities. this further has implications on how to organize for transformation. our study findings suggest that strategic considerations are supreme, and transformation endeavours must also be value-driven. thereby managers need to have clarity on the strategic motive and value propositions as part of planning for dt. a key consideration would be defining the role of bda in the transformation process since we observe from our findings that they are perceived as hygiene factors in the dt landscape. consequently, recognizing and leveraging these crucial resources aligned with strategic imperatives is the central element in pursuing dt. our proposed conceptual framework can serve as a dt implementation guide for the practice. before planning and investing, managers should carefully evaluate the need for any such change. moreover, the exploration of needs would further help ascertain the dt's purview. the findings indicate that the constituents of a successful dt are the firm’s ability to organize and manage its intangible assets, particularly its structure, culture, networks, and other managerial attributes. one may interpret that increased utilization of organizational resources towards planning a digital governance protocol while simultaneously leveraging bda will be among the critical success factors for a dt initiative. like all research works, this study is also not free from limitations. the framework proposed here is not of a universal kind but presents a transformative perspective of dt. this viewpoint is expected to guide its subsequent usage in academia and practice. there are also methodological limitations resorting to how we have made choices while employing the text mining approach (e.g., focusing only on the journal publication) while synthesizing the dt framework. we also admit a few limitations around our qualitative examination where the existence of selection bias (in terms of shortlisting articles for review) and confirmation bias (in terms of the tendency of adherence to phase 1 results) cannot be ruled out. we acknowledge that ongoing developments in the field of dt may introduce emerging phenomena not captured in our results. however, these aspects can still be mapped with the framework elements due to their generic presentation. our study is also silent on the interrelationships of the various themes unveiled within the dt scholarship. therefore, investigating the interrelationships among these different themes requires further exploration, particularly the orchestration of all four constituents enabling dt. 6.3 future research apart from attending to some of the possibilities identified in the content and addressing the limitations documented above, the study motivates scholars of related disciplines to pursue australasian journal of information systems kumar & thakurta 2024, vol 28, research article dt: enterprise transformation perspective 29 research on the specific foundational elements: scope, ways, means, and ends describing a dt endeavour. there is also enough room to extend our offering of the dt framework. future inquiries can focus on developing testable hypotheses around our framework by envisaging the links within the foundational elements towards the ends (i.e., impact) and thereby quantifying its relevance to academia and practice. continuing in this direction, it may also be possible to propose an explanatory theory on dt following theory development guidelines (gregor, 2006; swanson et al., 2013). other explorations can extend our contribution by offering propositions around various linkages between some of the integral constituents: organizational, human, and bda leading to planned goals and examining these. furthermore, the sanctity of digital governance would also help formulate and align a digital strategy focusing on value co-creation and re-creation. having digital governance in place would involve several important constituents adhering to a digital strategy. in this realm, further inquiries are warranted to uncover the planning or emerging nature of a digital strategy. future studies can explore the nature of formulating the digital strategy for existing vis-à-vis new firms. references anna singh, thomas hess, singh, a., & hess, t. (2017). how chief digital officers promote the digital transformation of their companies. mis quarterly executive, 16(1), 1–17. http://misqe.org/ojs2/index.php/misqe/article/viewfile/685/454 baiyere, a., salmela, h., & tapanainen, t. (2020). digital transformation and the new logics of business process management. european journal of information systems, 29(3), 238–259. https://doi.org/10.1080/0960085x.2020.1718007 bastani, k., namavari, h., & shaffer, j. (2019). latent dirichlet allocation (lda) for topic modeling of the cfpb consumer complaints. expert systems with applications, 127, 256– 271. https://doi.org/10.1016/j.eswa.2019.03.001 benlian, a., & haffke, i. (2016). does mutuality matter? examining the bilateral nature and effects of ceo–cio mutual understanding. the journal of strategic information systems, 25(2), 104–126.[doi available] bennett, e. e., & mcwhorter, r. r. (2021). virtual hrd’s role in crisis and the post covid-19 professional lifeworld: accelerating skills for digital transformation. advances in developing human resources, 23(1), 5–25. https://doi.org/10.1177/1523422320973288 berghaus, s., & back, a. (2016). stages in digital business transformation: results of an empirical maturity study. mediterranean conference on information systems (mcis), 1–17. besson, p., & rowe, f. (2012). strategizing information systems-enabled organizational transformation: a transdisciplinary review and new directions. the journal of strategic information systems, 21(2), 103–124. https://doi.org/https://doi.org/https://doi.org/10.1016/j.jsis.2012.05.001 bharadwaj, a., el sawy, o. a., pavlou, p. a., & venkatraman, n. (2013). digital business strategy: toward a next generation of insights. mis quarterly, 37(2), 14-001. bholat, d., hansen, s., santos, p., & schonhardt-bailey, c. (2015). text mining for central banks. available at ssrn 2624811. https://doi.org/10.1016/j.eswa.2019.03.001 https://doi.org/10.1177/1523422320973288 https://doi.org/https:/doi.org/https:/doi.org/10.1016/j.jsis.2012.05.001 australasian journal of information systems kumar & thakurta 2024, vol 28, research article dt: enterprise transformation perspective 30 bichler, m., frank, u., avison, d., malaurent, j., fettke, p., hovorka, d., krämer, j., schnurr, d., müller, b., suhl, l., & others. (2016). theories in business and information systems engineering. business \& information systems engineering, 58(4), 291–319. blei, d. m. (2012). probabilistic topic models. communications of the acm, 55(4), 77–84. https://doi.org/10.1145/2133806.2133826 blei, d. m., ng, a. y., & jordan, m. i. (2003). latent dirichlet allocation. journal of machine learning research, 3, 993–1022. https://doi.org/10.1016/b978-0-12-411519-4.00006-9 boneva, m. (2018). challenges related to the digital transformation of business companies. in innovation management, entrepreneurship and sustainability (imes 2018) (p. 101–114). boyes, h., hallaq, b., cunningham, j., & watson, t. (2018). the industrial internet of things (iiot): an analysis framework. computers in industry, 101(april), 1–12. https://doi.org/10.1016/j.compind.2018.04.015 cambridge university press. (2013). cambridge advanced learner’s dictionary. https://dictionary.cambridge.org/dictionary/english/transformation canhoto, a. i., quinton, s., pera, r., molinillo, s., & simkin, l. (2021). digital strategy aligning in smes: a dynamic capabilities perspective. journal of strategic information systems, 30(3). https://doi.org/10.1016/j.jsis.2021.101682 catlin, t., laberge, l., & varney, s. (2018). digital strategy: the four fights you have to win. mckinsey quarterly, 2018(4), 78–89. cetindamar, d., abedin, b., & shirahada, k. (2021). the role of employees in digital transformation: a preliminary study on how employees&#x2019; digital literacy impacts use of digital technologies. ieee transactions on engineering management, 1–12. https://doi.org/10.1109/tem.2021.3087724 chanias, s., myers, m. d., & hess, t. (2019). digital transformation strategy making in predigital organizations: the case of a financial services provider. the journal of strategic information systems, 28(1), 17–33. https://doi.org/10.1016/j.jsis.2018.11.003 charalabidis, y., & lachana, z. (2020). towards a science base for digital governance. the 21st annual international conference on digital government research, 383–389. https://doi.org/10.1145/3396956.3400062 choi, y., mehraliyev, f., & kim, s. (sam). (2020). role of virtual avatars in digitalized hotel service. international journal of contemporary hospitality management, 32(3), 977–997. https://doi.org/10.1108/ijchm-03-2019-0265 chu, s.c., kamal, s., & kim, y. (2019). re-examining of consumers’ responses toward social media advertising and purchase intention toward luxury products from 2013 to 2018: a retrospective commentary. journal of global fashion marketing, 10(1), 81–92. https://doi.org/10.1080/20932685.2018.1550008 clohessy, t., acton, t., & morgan, l. (2017). the impact of cloud-based digital transformation on it service providers: evidence from focus groups. international journal of cloud applications and computing (ijcac), 7(4), 1–19. https://doi.org/https://doi.org/10.4018/ijcac.2017100101 https://doi.org/10.1016/b978-0-12-411519-4.00006-9 https://doi.org/10.1016/j.compind.2018.04.015 https://dictionary.cambridge.org/dictionary/english/transformation https://doi.org/10.1016/j.jsis.2021.101682 https://doi.org/10.1109/tem.2021.3087724 https://doi.org/10.1016/j.jsis.2018.11.003 https://doi.org/10.1145/3396956.3400062 https://doi.org/10.1108/ijchm-03-2019-0265 https://doi.org/10.1080/20932685.2018.1550008 https://doi.org/https:/doi.org/10.4018/ijcac.2017100101 australasian journal of information systems kumar & thakurta 2024, vol 28, research article dt: enterprise transformation perspective 31 correani, a., de massis, a., frattini, f., petruzzelli, a. m., & natalicchio, a. (2020). implementing a digital strategy: learning from the experience of three digital transformation projects. california management review, 62(4), 37–56. https://doi.org/10.1177/0008125620934864 creswell, j. w., & clark, v. l. p. (2017). designing and conducting mixed methods research. sage publications. creswell, j. w., & creswell, j. d. (2017). research design: qualitative, quantitative, and mixed methods approaches. sage publications. cui, m., & pan, s. l. (2015). developing focal capabilities for e-commerce adoption: a resource orchestration perspective. information & management, 52(2), 200–209. dalenogare, l. s., benitez, g. b., ayala, n. f., & frank, a. g. (2018). the expected contribution of industry 4.0 technologies for industrial performance. international journal of production economics, 204(july), 383–394. https://doi.org/10.1016/j.ijpe.2018.08.019 dang, d., & vartiainen, t. (2019). digital strategy patterns in information systems research. proceedings of the 23rd pacific asia conference on information systems: secure ict platform for the 4th industrial revolution, pacis 2019, 1–18. dehning, b., richardson, v. j., & zmud, r. w. (2003). the value relevance of announcements of transformational information technology investments. mis quarterly, 27(4), 637–656. demirkan, h., spohrer, j. c., & welser, j. j. (2016). digital innovation and strategic transformation. it professional, 18(6), 14–18. ding, k., chong, w., yap, k., & imm, s. (2020). employing structural topic modelling to explore perceived service quality attributes in airbnb accommodation. international journal of hospitality management, 91(september), 102676. https://doi.org/10.1016/j.ijhm.2020.102676 dremel, c., wulf, j., herterich, m. m., waizmann, j.-c., & brenner, w. (2017). how audi ag established big data analytics in its digital transformation. mis quarterly executive, 16(2), 81–100. du, w. d., pan, s. l., & huang, j. (2016). how a latecomer company used it to redeploy slack resources. mis quarterly executive, 15(3), 195–213. dunleavy, p., & margetts, h. (2015). design principles for essentially digital governance. in annual meeting of the american political science association. elliot, s. (2011). transdisciplinary perspectives on environmental sustainability: a resource base and framework for it-enabled business transformation. mis quarterly: management information systems, 35(1), 197–236. https://doi.org/10.2307/23043495 er, m. c. (1988). decision support systems: a summary, problems, and future trends. decision support systems, 4, 355–363. faulkner, p., & runde, j. (2009). on the identity of technological objects and user innovations in function. academy of management review, 34(3), 442–462. https://doi.org/10.5465/amr.2009.40632318 faulkner, p., & runde, j. (2013). technological objects, social positions, and the transformational model of social activity. mis quarterly, 37(3), 803–818. https://doi.org/10.1177/0008125620934864 https://doi.org/10.1016/j.ijpe.2018.08.019 https://doi.org/10.2307/23043495 https://doi.org/10.5465/amr.2009.40632318 australasian journal of information systems kumar & thakurta 2024, vol 28, research article dt: enterprise transformation perspective 32 faulkner, p., & runde, j. (2019). theorizing the digital object. mis quarterly: management information systems, 43(4), 1278–1302. https://doi.org/10.25300/misq/2019/13136 fitzgerald, m., kruschwitz, n., bonnet, d., & welch, m. (2014). embracing digital technology: a new strategic imperative. mit sloan management review, 55(2), 1–16. fresneda, j. e., burnham, t. a., hill, c. h., fresneda, j. e., burnham, t. a., & hill, c. h. (2021). structural topic modelling segmentation: a segmentation method combining latent content and customer context method combining latent content and customer context. journal of marketing management, 37(7–8), 792–812. https://doi.org/10.1080/0267257x.2021.1880464 gasser, u., & almeida, v. (2022). futures of digital governance. commun. acm, 65(3), 30–32. https://doi.org/10.1145/3477502 gerster, d. (2017). digital transformation and it: current state of research. pacific asia conference on information systems, 1–12. http://aisel.aisnet.org/pacis2017%0ahttp://aisel.aisnet.org/pacis2017/133 gregor, s. (2006). the nature of theory in information systems. mis quarterly, 30(3), 611–642. https://doi.org/10.2307/25148742 günther, w. a., rezazade mehrizi, m. h., huysman, m., feldberg, f., mehrizi, m. h. r., huysman, m., & feldberg, f. (2017). debating big data: a literature review on realizing value from big data. the journal of strategic information systems, 26(3), 191–209. https://doi.org/https://doi.org/10.1016/j.jsis.2017.07.003 guo, h., yang, j., & han, j. (2021). the fit between value proposition innovation and technological innovation in the digital environment: implications for the performance of startups. ieee transactions on engineering management, 68(3), 797–809. https://doi.org/10.1109/tem.2019.2918931 gupta, m., & george, j. f. (2016). toward the development of a big data analytics capability. information and management, 53(8), 1049–1064. https://doi.org/10.1016/j.im.2016.07.004 gust, g., neumann, d., flath, c. m., brandt, t., & ströhle, p. (2017). how a traditional company seeded new analytics capabilities. mis quarterly executive, 16(3), 215–230. haffke, i., kalgovas, b., & benlian, a. (2017). options for transforming the it function using bimodal it. mis quarterly executive, 16(2), 101–117. halpern, n., mwesiumo, d., suau-sanchez, p., budd, t., & bråthen, s. (2021). ready for digital transformation? the effect of organisational readiness, innovation, airport size and ownership on digital change at airports. journal of air transport management, 90, 101949. https://doi.org/https://doi.org/10.1016/j.jairtraman.2020.101949 hanelt, a., firk, s., hildebrandt, b., & kolbe, l. m. (2021a). digital m&a, digital innovation, and firm performance: an empirical investigation. european journal of information systems, 30(1), 3–26. https://doi.org/10.1080/0960085x.2020.1747365 hanelt, a., bohnsack, r., marz, d., & antunes marante, c. (2021b). a systematic review of the literature on digital transformation: insights and implications for strategy and organizational change. journal of management studies, 58(5), 1159–1197. https://doi.org/https://doi.org/10.1111/joms.12639 https://doi.org/10.25300/misq/2019/13136 https://doi.org/10.1080/0267257x.2021.1880464 https://doi.org/10.1145/3477502 http://aisel.aisnet.org/pacis2017%0ahttp:/aisel.aisnet.org/pacis2017/133 https://doi.org/10.2307/25148742 https://doi.org/https:/doi.org/10.1016/j.jsis.2017.07.003 https://doi.org/10.1109/tem.2019.2918931 https://doi.org/10.1016/j.im.2016.07.004 https://doi.org/https:/doi.org/10.1016/j.jairtraman.2020.101949 https://doi.org/10.1080/0960085x.2020.1747365 https://doi.org/https:/doi.org/10.1111/joms.12639 australasian journal of information systems kumar & thakurta 2024, vol 28, research article dt: enterprise transformation perspective 33 hanelt, a., piccinini, e., gregory, r. w., hildebrandt, b., & kolbe, l. m. (2015). digital transformation of primarily physical industries-exploring the impact of digital trends on business models of automobile manufacturers. international conference on wirtschaftsinformatik, 1313–1327. hansen, r., & sia, s. k. (2015). hummel’s digital transformation toward omnichannel retailing: key lessons learned. mis quarterly executive, 14(2), 51–66. hedman, j., sarker, s., & veit, d. (2016). digitization in business models and entrepreneurship. information systems journal, 26(5), 419–420. https://doi.org/https://doi.org/10.1111/isj.12119 hendry, s., & madeley, a. (2010). text mining and the information content of bank of canada communications. available at ssrn 1722829. henriette, e., feki, m., & boughzala, i. (2015). the shape of digital transformation: a systematic literature review. mcis 2015 proceedings, 431–443. herczeg, m. (2021). post-constructivist digital learning through ambient learning spaces. i-com, 20(3), 263–277. https://doi.org/doi:10.1515/icom-2021-0031 herzberg, f. i. (2018). happiness and unhappiness: a brief autobiography. in management laureates (p. 1–38). routledge. hess, t., matt, c., benlian, a., & wiesböck, f. (2016). options for formulating a digital transformation strategy. mis quarterly executive, 15(2), 123–139. hirata, e. (2019). service characteristics and customer satisfaction in the container liner shipping industry. the asian journal of shipping and logistics, 35(1), 24–29. https://doi.org/https://doi.org/10.1016/j.ajsl.2019.03.004 horlacher, a., klarner, p., & hess, t. (2016). crossing boundaries: organization design parameters surrounding cdos and their digital transformation activities. 22nd americas conference on information systems. huang, j., henfridsson, o., liu, m. j., & newell, s. (2017). growing on steroids. mis quarterly, 41(1), 301–314. https://www.jstor.org/stable/26629649 imran, f., shahzad, k., butt, a., & kantola, j. (2021). digital transformation of industrial organizations: toward an integrated framework. journal of change management, 21(4), 451–479. ivančić, l., vukšić, v. b., & spremić, m. (2019). mastering the digital transformation process: business practices and lessons learned. technology innovation management review, 9(2), 36–50. kane, g. c. (2017). ‘digital transformation’ is a misnomer. mit sloan management review, 1– 5. http://sloanreview.mit.edu/article/digital-transformation-is-a-misnomer/ kellogg, k. c. (2021). local adaptation without work intensification: experimentalist governance of digital technology for mutually beneficial role reconfiguration in organizations. organization science, 33(2), 571–599. https://doi.org/10.1287/orsc.2021.1445 kim, d. j., bose, i., mukhopadhyay, a., & shukla, s. k. (2020). bright information and communication technologies in the 21st century. information systems frontiers. https://doi.org/https:/doi.org/10.1111/isj.12119 https://doi.org/doi:10.1515/icom-2021-0031 https://doi.org/https:/doi.org/10.1016/j.ajsl.2019.03.004 https://www.jstor.org/stable/26629649 http://sloanreview.mit.edu/article/digital-transformation-is-a-misnomer/ https://doi.org/10.1287/orsc.2021.1445 australasian journal of information systems kumar & thakurta 2024, vol 28, research article dt: enterprise transformation perspective 34 klus, m. f., & müller, j. (2021). the digital leader: what one needs to master today’s organisational challenges. journal of business economics, 91(8), 1189–1223. https://doi.org/10.1007/s11573-021-01040-1 koch, h., chipidza, w., & kayworth, t. r. (2021). realizing value from shadow analytics: a case study. journal of strategic information systems, 30(2), 101668. https://doi.org/10.1016/j.jsis.2021.101668 koch, t., & windsperger, j. (2017). seeing through the network: competitive advantage in the digital economy. journal of organization design, 6(1), 6. https://doi.org/10.1186/s41469017-0016-z kohli, r., & johnson, s. (2011). digital transformation in latecomer industries: cio and ceo leadership lessons from encana oil & gas (usa) inc. mis quarterly executive, 10(4), 141–156. kotter, j. p. (1995). leading change: why transformation efforts fail. harvard business review, march-apri, 1–12. krippendorff, k. (2018). content analysis: an introduction to its methodology. sage publications. kuhn, m., & johnson, k. (2013). applied predictive modeling. in applied predictive modeling (vol. 26). springer. https://doi.org/10.1007/978-1-4614-6849-3 kumar, r. (2021). managing business in the digital era – the use of it and analytics for process transformation. journal of decision systems, 30(4), 410–413. https://doi.org/10.1080/12460125.2021.1925397 kumar, r., & thakurta, r. (2019). towards re-conceptualizing dss: a text mining based approach. in proceedings of the 27th european conference on information systems (ecis), stockholm & uppsala, sweden, june 8-14, 2019. kumar, r., & thakurta, r. (2021). exfoliating decision support system: a synthesis of themes using text mining. information systems and e-business management. https://doi.org/10.1007/s10257-020-00490-4 legner, c., eymann, t., hess, t., matt, c., böhmann, t., drews, p., mädche, a., urbach, n., & ahlemann, f. (2017). digitalization: opportunity and challenge for the business and information systems engineering community. business & information systems engineering, 59(4), 301–308. https://doi.org/https://doi.org/10.1007/s12599-017-0484-2 li, l., su, f., zhang, w., & mao, j. (2018). digital transformation by sme entrepreneurs: a capability perspective. information systems journal, 28(6), 1129–1157. https://doi.org/https://doi.org/10.1111/isj.12153 li, w., liu, k., belitski, m., ghobadian, a., & o’regan, n. (2016). e-leadership through strategic alignment: an empirical study of small-and medium-sized enterprises in the digital age. journal of information technology, 31(2), 185–206. https://doi.org/https://doi.org/10.1057/jit.2016.10 li, y., hou, m., & liu, h. (2012). towards a theoretical framework of strategic decision , supporting capability and information sharing under the context of internet of things. information technology management, 13, 205–216. https://doi.org/10.1007/s10799-012-01211 https://doi.org/10.1007/s11573-021-01040-1 https://doi.org/10.1186/s41469-017-0016-z https://doi.org/10.1186/s41469-017-0016-z https://doi.org/10.1007/978-1-4614-6849-3 https://doi.org/10.1080/12460125.2021.1925397 https://doi.org/10.1007/s10257-020-00490-4 https://doi.org/https:/doi.org/10.1007/s12599-017-0484-2 https://doi.org/https:/doi.org/10.1111/isj.12153 https://doi.org/https:/doi.org/10.1057/jit.2016.10 https://doi.org/10.1007/s10799-012-0121-1 https://doi.org/10.1007/s10799-012-0121-1 australasian journal of information systems kumar & thakurta 2024, vol 28, research article dt: enterprise transformation perspective 35 liere-netheler, k., vogelsang, k., & packmohr, s. (2018). drivers of digital transformation in manufacturing. hawaii international conference on system sciences, 3926–3935. lindgreen, a., benedetto, an. di, brodie, r. j., & jakkola, e. (2021). how to develop great conceptual frameworks for business-to-business marketing. industrial marketing management, 94, 2–10. https://doi.org/10.1016/j.indmarman.2020.04.005 lipsmeier, a., kühn, a., joppen, r., & dumitrescu, r. (2020). process for the development of a digital strategy. procedia cirp, 88, 173–178. https://doi.org/10.1016/j.procir.2020.05.031 liu, h., ke, w., wei, k. k., & hua, z. (2013). the impact of it capabilities on firm performance: the mediating roles of absorptive capacity and supply chain agility. decision support systems, 54(3), 1452–1462. https://doi.org/10.1016/j.dss.2012.12.016 lucas jr, h., agarwal, r., clemons, e. k., el sawy, o. a., & weber, b. (2013). impactful research on transformational information technology: an opportunity to inform new audiences. mis quarterly, 37(2), 371–382. maanen, v., & bacharach, a. (2015). editor's comments : why theory ? academy of management review, 4015(1), 1–5. manesh, m. f., pellegrini, m. m., marzi, g., & dabic, m. (2021). knowledge management in the fourth industrial revolution: mapping the literature and scoping future avenues. ieee transactions on engineering management, 68(1), 289–300. https://doi.org/10.1109/tem.2019.2963489 markus, m. l. (2014). maybe not the king, but an invaluable subordinate: a commentary on avison and malaurent’s advocacy of ‘theory light’is research. journal of information technology, 29(4), 341–345. markus, m. l., & loebbecke, c. (2013). commoditized digital processes and business community platforms: new opportunities and challenges for digital business strategies. mis quarterly, 37(2), 649–653. matsunaga, m. (2021). testing the theory of communication and uncertainty management in the context of digital transformation with transformational leadership as a moderator. international journal of business communication, 23294884211023970. https://doi.org/10.1177/23294884211023966 matt, c., hess, t., & benlian, a. (2015). digital transformation strategies. business & information systems engineering, 57(5), 339–343. https://doi.org/10.1007/s12599-015-0401-5 mendling, j., pentland, b. t., & recker, j. (2020). building a complementary agenda for business process management and digital innovation. european journal of information systems, 29(3), 208–219. https://doi.org/10.1080/0960085x.2020.1755207 mergel, i., edelmann, n., & haug, n. (2019). defining digital transformation: results from expert interviews. government information quarterly, 36(4), 1–16. https://doi.org/10.1016/j.giq.2019.06.002 morakanyane, r., grace, a. a., & o’reilly, p. (2017). conceptualizing digital transformation in business organizations: a systematic review of literature. 30th bled econference, 427– 443. https://doi.org/10.1016/j.indmarman.2020.04.005 https://doi.org/10.1016/j.procir.2020.05.031 https://doi.org/10.1016/j.dss.2012.12.016 https://doi.org/10.1177/23294884211023966 https://doi.org/10.1007/s12599-015-0401-5 https://doi.org/10.1080/0960085x.2020.1755207 https://doi.org/10.1016/j.giq.2019.06.002 australasian journal of information systems kumar & thakurta 2024, vol 28, research article dt: enterprise transformation perspective 36 mufraini, m. a., saharuddin, d., prabowo, m. a., & wicaksono, a. t. s. (2020). sharia insight factors: does it matter to shift metropolitan decision behavior towards islamic bank? management science letters, 10(14), 3395–3404. https://doi.org/10.5267/j.msl.2020.5.039 nadkarni, s., & prügl, r. (2021). digital transformation: a review, synthesis and opportunities for future research. management review quarterly, 71(2), 233–341. nag, r., hambrick, d. c., & chen, m. j. (2007). what is strategic management, really? inductive derivation of a consensus definition of the field. strategic management journal, 28(9), 935955. nauhaus, s., luger, j., & raisch, s. (2021). strategic decision making in the digital age: expert sentiment and corporate capital allocation. journal of management studies, 58(7), 1933– 1961. https://doi.org/https://doi.org/10.1111/joms.12742 nwaiwu, f. (2018). review and comparison of conceptual frameworks on digital business transformation. journal of competitiveness, 10(3), 86–100. https://doi.org/doi.org/10.7441/joc.2018.03.06 nwankpa, j. k., & datta, p. (2017). balancing exploration and exploitation of it resources: the influence of digital business intensity on perceived organizational performance. european journal of information systems, 26(5), 469–488. omrani, n., rejeb, n., maalaoui, a., dabić, m., & kraus, s. (2022). drivers of digital transformation in smes. ieee transactions on engineering management, 1–14. https://doi.org/10.1109/tem.2022.3215727 orlikowski, w. j., & iacono, c. s. (2001). research commentary: desperately seeking the “it” in it research—a call to theorizing the it artifact. information systems research, 12(2), 121–134. ossiannilsson, e. (2018). visionary leadership for digital transformation: in a time when learners take ownership of their learning. asian journal of distance education, 13(1), 128– 148. pang, b., & lee, l. (2008). presentation: opinion mining and sentiment analysis. foundations and trend in information retrieval, 2(2), 1–135. https://doi.org/10.2200/s00416ed1v01y201204hlt016 papagiannidis, s., harris, j., & morton, d. (2020). who led the digital transformation of your company? a reflection of it related challenges during the pandemic. international journal of information management, 55, 102166. https://doi.org/https://doi.org/10.1016/j.ijinfomgt.2020.102166 penco, l., serravalle, f., profumo, g., & viassone, m. (2021). mobile augmented reality as an internationalization tool in the “made in italy” food and beverage industry. journal of management and governance, 25(4), 1179–1209. https://doi.org/10.1007/s10997-020-09526w philip, g., & mckeown, i. (2004). business transformation and organizational culture : the role of competency , is and tqm. european management journal, 22(6), 624–636. https://doi.org/10.1016/j.emj.2004.09.026 https://doi.org/10.5267/j.msl.2020.5.039 https://doi.org/https:/doi.org/10.1111/joms.12742 https://doi.org/10.1109/tem.2022.3215727 https://doi.org/10.2200/s00416ed1v01y201204hlt016 https://doi.org/https:/doi.org/10.1016/j.ijinfomgt.2020.102166 https://doi.org/10.1007/s10997-020-09526-w https://doi.org/10.1007/s10997-020-09526-w australasian journal of information systems kumar & thakurta 2024, vol 28, research article dt: enterprise transformation perspective 37 piazza, a., & castelluci, f. (2014). status in organization and management theory. journal of management, 40(1), 287–316. https://doi.org/10.1177/0149206313498904 purchase, v., parry, g., valerdi, r., nightingale, d., & mills, j. (2011). enterprise transformation: why are we interested, what is it, and what are the challenges? journal of enterprise transformation, 1(1), 14–33. qaiser, f. h., ahmed, k., sykora, m., choudhary, a., simpson, m., hasan, q. f., karim, a., martin, s., alok, c., mike, s. (2017). decision support systems for sustainable logistics: a review and bibliometric analysis. industrial management and data systems, 117(7), 1376– 1388. https://doi.org/10.1108/imds-09-2016-0410 reis, j., amorim, m., melão, n., & matos, p. (2018). digital transformation: a literature review and guidelines for future research. world conference on information systems and technologies, 411–421. riasanow, t., setzke, d. s., böhm, m., & krcmar, h. (2019). clarifying the notion of digital transformation: a transdisciplinary review of literature. journal of competences, strategy and management, 10(1), 5–31. ritala, p., baiyere, a., hughes, m., & kraus, s. (2021). digital strategy implementation: the role of individual entrepreneurial orientation and relational capital. technological forecasting and social change, 171, 120961. https://doi.org/https://doi.org/10.1016/j.techfore.2021.120961 roberts, m. e., stewart, b. m., & tingley, d. (2019). stm: an r package for structural topic models. journal of statistical software, 91(li), 1–40. https://doi.org/https://doi.org/10.18637/jss.v091.i02 röcker, j., mocker, m., & novales, a. (2017). digitized products: challenges and practices from the creative industries. twenty-third americas conference on information systems, boston, 2017: proceedings, 1–10. rodgers, w., yeung, f., odindo, c., & degbey, w. y. (2021). artificial intelligence-driven music biometrics influencing customers’ retail buying behavior. journal of business research, 126, 401–414. https://doi.org/https://doi.org/10.1016/j.jbusres.2020.12.039 ross, j. w., sebastian, i. m., & beath, c. m. (2017). how to develop a great digital strategy. mit sloan management review, 58(2), 7–9. https://doi.org/10.7551/mitpress/11633.003.0004 rouse, w. b. (2005a). a theory of enterprise transformation. systems engineering, 8(4), 279–295. rouse, w. b. (2005b). enterprises as systems: essential challenges and approaches to transformation. systems engineering, 8(2), 138–150. https://doi.org/10.1002/sys.20029 rouse, w. b. (2006). enterprise transformation: understanding and enabling fundamental change. john wiley & sons. rouse, w. b., & baba, m. l. (2006). enterprise transformation. communications of the acm, 49(7), 66–72. https://doi.org/10.1145/1139922.1139951 rowe, f. (2018). being critical is good, but better with philosophy! from digital transformation and values to the future of is research. european journal of information systems, 27(3), 380– 393. https://doi.org/10.1080/0960085x.2018.1471789 https://doi.org/10.1177/0149206313498904 https://doi.org/10.1108/imds-09-2016-0410 https://doi.org/https:/doi.org/10.1016/j.techfore.2021.120961 https://doi.org/https:/doi.org/10.18637/jss.v091.i02 https://doi.org/https:/doi.org/10.1016/j.jbusres.2020.12.039 https://doi.org/10.7551/mitpress/11633.003.0004 https://doi.org/10.1002/sys.20029 https://doi.org/10.1145/1139922.1139951 https://doi.org/10.1080/0960085x.2018.1471789 australasian journal of information systems kumar & thakurta 2024, vol 28, research article dt: enterprise transformation perspective 38 sanchez-franco, m. j., arenas-m, f. j., & alonso-dos-santos, m. (2021). using structural topic modelling to predict users ’ sentiment towards intelligent personal agents . an application for amazon ’ s echo and google home. journal of retailing and consumer services, 63(june). https://doi.org/10.1016/j.jretconser.2021.102658 schneckenberg, d., benitez, j., klos, c., velamuri, v. k., & spieth, p. (2021). value creation and appropriation of software vendors: a digital innovation model for cloud computing. information \& management, 58(4), 103463. sebastian, i. m., ross, j. w., beath, c., mocker, m., moloney, k. g., fonstad, n. o., ross, j. w., fonstad, n. o., beath, c., & mocker, m. (2017). how big old companies navigate digital transformation. mis quarterly executive, 16(3), 197–213. https://doi.org/10.4324/9780429286797-6 sharma, a., rana, n. p., & nunkoo, r. (2021). fifty years of information management research: a conceptual structure analysis using structural topic modeling. international journal of information management, 58(july 2020), 102316. https://doi.org/10.1016/j.ijinfomgt.2021.102316 sia, s. k., soh, c., & weill, p. (2016). how dbs bank pursued a digital business strategy. mis quarterly executive, 15(2), 105–121. sivarajah, u., kamal, m. m., irani, z., & weerakkody, v. (2017). critical analysis of big data challenges and analytical methods. journal of business research, 70, 263–286. https://doi.org/10.1016/j.jbusres.2016.08.001 stewart, k., kammer-kerwick, m., auchter, a., koh, h. e., dunn, m. e., & cunningham, i. (2019). examining digital video advertising (dva) effectiveness. european journal of marketing, 53(11), 2451–2479. https://doi.org/10.1108/ejm-11-2016-0619 swanson, r. a., chermack, t. j., & swanson, r. a., chermack, t. j. (2013). theory building in applied disciplines. berrett-koehler publishers. tabrizi, b., lam, e., girard, k., & irvin, v. (2019). digital transformation is not about technology. harvard business review, 13(march), 1–6. https://hbr.org/2019/03/digitaltransformation-is-not-about-technology tan, b., pan, s. l., lu, x., & huang, l. (2015). the role of is capabilities in the development of multi-sided platforms: the digital ecosystem strategy of alibaba.com. journal of the association for information systems, 16(4), 2. https://doi.org/https://doi.org/10.17705/1jais.00393 tan, f., ondrus, j., tan, b., & oh, j. (2020). digital transformation of business ecosystems: evidence from the korean pop industry. information systems journal, 30(5), 866–898. https://doi.org/10.1111/isj.12285 teubner, r. a., & stockhinger, j. (2020). literature review: understanding information systems strategy in the digital age. journal of strategic information systems, 29(4), 101642. https://doi.org/10.1016/j.jsis.2020.101642 tirunillai, s., & tellis, g. j. (2014). mining marketing meaning from online chatter: strategic brand analysis of big data using latent dirichlet allocation. journal of marketing research, 51(4), 463-479. https://doi.org/10.1016/j.jretconser.2021.102658 https://doi.org/10.1016/j.jbusres.2016.08.001 https://hbr.org/2019/03/digital-transformation-is-not-about-technology https://hbr.org/2019/03/digital-transformation-is-not-about-technology https://doi.org/https:/doi.org/10.17705/1jais.00393 https://doi.org/10.1111/isj.12285 https://doi.org/10.1016/j.jsis.2020.101642 australasian journal of information systems kumar & thakurta 2024, vol 28, research article dt: enterprise transformation perspective 39 tiwana, a., konsynski, b., & bush, a. a. (2010). research commentary—platform evolution: coevolution of platform architecture, governance, and environmental dynamics. information systems research, 21(4), 675–687. https://doi.org/https://doi.org/10.1287/isre.1100.0323 trenerry, b., chng, s., wang, y., suhaila, z. s., lim, s. s., lu, h. y., & oh, p. h. (2021). preparing workplaces for digital transformation: an integrative review and framework of multi-level factors. frontiers in psychology, 12, 620766. ugboego, c. a., edeh, f. o., amarachi, o., & charles, o. (2022). talent management and resilience post-covid-19. september. venable, j. r. (2006). the role of theory and theorising in design science research. verhoef, p. c., broekhuizen, t., bart, y., bhattacharya, a., dong, j. q., fabian, n., haenlein, m., qi dong, j., fabian, n., & haenlein, m. (2021). digital transformation: a multidisciplinary reflection and research agenda. journal of business research, 122(january 2021), 889–901. https://doi.org/https://doi.org/10.1016/j.jbusres.2019.09.022 verma, s., & bhattacharyya, s. s. (2017). perceived strategic value-based adoption of big data analytics in emerging economy: a qualitative approach for indian firms. journal of enterprise information management, 30(3), 354–382. https://doi.org/10.1108/jeim-10-20150099 vial, g. (2019). understanding digital transformation: a review and a research agenda. the journal of strategic information systems, 28(2), 118–144. https://doi.org/10.1016/j.jsis.2019.01.003 victor, v., thoppan, j. j., fekete-farkas, m., & grabara, j. (2019). pricing strategies in the era of digitalisation and the perceived shift in consumer behaviour of youth in poland. journal of international studies, 12(3). wade, m., & shan, j. (2020). covid-19 has accelerated digital transformation, but may have made it harder not easier. mis quarterly executive, 19(3), 213–220. https://doi.org/10.17705/2msqe.00034 wamba, s. f., gunasekaran, a., akter, s., ren, s. j. fan, dubey, r., & childe, s. j. (2017). big data analytics and firm performance: effects of dynamic capabilities. journal of business research, 70, 356–365. https://doi.org/10.1016/j.jbusres.2016.08.009 warner, k. s. r., & wäger, m. (2019). building dynamic capabilities for digital transformation: an ongoing process of strategic renewal. long range planning, 52(3), 326–349. weber, r. (2003). theoretically speaking1. mis quarterly, 27(3), iii. wessel, l., baiyere, a., ologeanu-taddei, r., cha, j., & jensen, t. b. (2020). unpacking the difference between digital transformation and it-enabled organizational transformation. journal of the association of information systems, 22(1 (may 2020)), 102–129. https://doi.org/10.17705/1jais.00655 westerman, g. (2017). your company doesn’ t need a digital strategy better strategy. mit sloan management review, 1–6. westerman, g., bonnet, d., & mcafee, a. (2014). the nine elements of digital transformation opinion & analysis. mit sloan management review, 55(3), 1–6. https://doi.org/https:/doi.org/10.1287/isre.1100.0323 https://doi.org/https:/doi.org/10.1016/j.jbusres.2019.09.022 https://doi.org/10.1108/jeim-10-2015-0099 https://doi.org/10.1108/jeim-10-2015-0099 https://doi.org/10.1016/j.jsis.2019.01.003 https://doi.org/10.17705/2msqe.00034 https://doi.org/10.1016/j.jbusres.2016.08.009 https://doi.org/10.17705/1jais.00655 australasian journal of information systems kumar & thakurta 2024, vol 28, research article dt: enterprise transformation perspective 40 wimelius, h., mathiassen, l., holmström, j., & keil, m. (2021). a paradoxical perspective on technology renewal in digital transformation. information systems journal, 31(1), 198–225. https://doi.org/10.1111/isj.12307 yeow, a., soh, c., & hansen, r. (2018). aligning with new digital strategy: a dynamic capabilities approach. the journal of strategic information systems, 27(1), 43–58. https://doi.org/https://doi.org/10.1016/j.jsis.2017.09.001 yin, r. k. (2018). case study research and applications (6th editio). sage. zangiacomi, a., pessot, e., fornasiero, r., bertetti, m., & sacco, m. (2020). moving towards digitalization: a multiple case study in manufacturing. production planning & control, 31(2–3), 143–157. https://doi.org/doi.org/10.1080/09537287.2019.1631468 zhu, k. (2004). the complementarity of information technology infrastructure and ecommerce capability: a resource-based assessment of their business value. journal of management information systems, 21(1), 167–202. zimmermann, a., schmidt, r., sandkuhl, k., wißotzki, m., jugel, d., & möhring, m. (2015). digital enterprise architecture-transformation for the internet of things. proceedings of the 2015 ieee 19th international enterprise distributed object computing conference workshops and demonstrations, edocw 2015, october, 130–138. https://doi.org/10.1109/edocw.2015.16 zutshi, a., grilo, a., & nodehi, t. (2021). the value proposition of blockchain technologies and its impact on digital platforms. computers & industrial engineering, 155, 107187. https://doi.org/https://doi.org/10.1016/j.cie.2021.107187 copyright: © 2024 kumar and thakurta. this is an open-access article distributed under the terms of the creative commons attribution-non-commercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v28i0.4259 https://doi.org/10.1111/isj.12307 https://doi.org/https:/doi.org/10.1016/j.jsis.2017.09.001 https://doi.org/doi.org/10.1080/09537287.2019.1631468 https://doi.org/10.1109/edocw.2015.16 https://doi.org/https:/doi.org/10.1016/j.cie.2021.107187 http://creativecommons.org/licenses/by-nc/3.0/au/ https://doi.org/10.3127/ajis.v28i0.4259 paper title (paper title style) australasian journal of information systems prokofieva & miah 2019, vol 23, research note blockchain in healthcare 1 blockchain in healthcare maria prokofieva vu business school victoria university melbourne, australia shah j. miah vu business school victoria university melbourne, australia shah.miah@vu.edu.au abstract blockchain is treated as a ledger system that manages data and their transactions using timestamped blocks through cryptography and works in a decentralised manner over the computing network. although blockchain is originally used as a backbone for the cryptocurrency, bitcoin, its capabilities and applications have yet to be extended far beyond cryptocurrencies. in this paper, through conducting a latest systematic literature review aiming to produce new source of evidence, we identify potential applications of the blockchain technologies in healthcare. the comprehensive review looks at the professional and academic open-sourced journals published between 2008 to 2019 to recognise the potential of blockchain based approaches in the purpose of healthcare information disseminations, as well as to segregate issues for the implementation and development of blockchain applications. we identify several major application domains that present research opportunities and challenges for the future advancements and directions for the benefits of is researchers and professionals. keywords: blockchain; is solution design; healthcare; smart contracts 1 introduction blockchain is considered as a ledger system that assists in managing and storing data in timestamped blocks that works mainly decentralised manner over any computing networks and linking using cryptography. introduced in the fintech area, the blockchain capabilities have a potential of wider realisations. however, application venues and effective solution design are still underdeveloped. blockchain promises fundamental changes across different industries, including fintech, government, health and supply chain (casino, dasaklis, & patsakis, 2019). its potential benefits include reduction in costs and complexity of transactions between parties, enhanced security, improved transparency and regulation. healthcare is characterised as a traditional industry that is considerably rigid to measure due to the facts of change and resistant to innovative practices. issues in healthcare (e.g. privacy, quality of care, information security) have been drawing attention in recent years worldwide. blockchain technologies have been increasingly recognised as a tool to address existing information distribution issues. it may improve immediate healthcare practices for example in improving health service delivery and quality of care support. a recent report of market research future (mrfr) shows that blockchain technology in healthcare is expected to generate over 42 million in value and reach a compound annual growth rate of 71.8% by 2023. such strong growth is driven by inherited blockchain characteristics of decentralised ledge australasian journal of information systems prokofieva & miah 2019, vol 23, research note blockchain in healthcare 2 technology with greater transparency, improved security and privacy, increased traceability, boosted efficiency and reduced costs. considered as a major disruptor in the industry, blockchain is defined in a context-sensitive manner. for instance, the blockchain is defined as an increasing collection of records, usually referred to as blocks, that are linked together using cryptography in such a way that does not allow modifications (deloitte, 2018) and promotes transparency and security. this way, the technology can assist to improve health service delivery and quality of care support through elimination of costs and privacy concerns, enhancing coverage and quality, while enabling user provisions of healthcare (gordon & catalini, 2018). based on peer-to-peer networks, one of the benefits of using blockchain is, it updates with realtime, leaving no space for intermediaries and associated costs (zhang, schmidt, white, & lenz, 2018). being resistant to modifications, blockchain offers a transparent environment where health professionals as well as patients are able to access records seemingly and without added costs (zhang, white, schmidt, lenz, & rosenbloom, 2018). it also increases security of the system by reducing the chance of lost records and errors (benchoufi, 2017). various innovative is solution have been proposed to utilise blockchain oriented applications in healthcare. such still-growing solutions are mainly designed for healthcare professionals and patients to make decisions regarding treatments, care and administration arrangements for service support. illustrative examples can be as follows: • nebula genomics (https://nebula.org): a company uses blockchain technology to enhance genomic data protection, enable buyers to efficiently acquire genomic data and address the challenges of genomic big data. • secure health chain (https://secure.health): a company offers a blockchain healthcare emr solution, which maintains personal digital medical records with help of blockchain. • doc.ai (https://doc.ai): a company uses natural language processing, computer vision and blockchain to generate insights from medical data. at the same time, numerous surveys highlight that understanding of the blockchain in general and its applications in healthcare is still at their emergent stages. despite a growing hype around the blockchain technology, a recent deloitte (2018) survey shows that understanding of the technology varies due to its facts of what purposes are served. although with little knowledge (e.g. almost 40% of us senior executives had little knowledge about blockchain), 55% of us senior executives are planning to invest more than $1m usd in the blockchain technology over the next year. these data suggest that while senior management is optimistic regarding the blockchain prospects, there is an emergent call for knowledge growing to build understanding on applications and provisions of such technology, in particular for is industry practitioners and researchers. this reinforces the needs to develop a landscape study that illustrates potential benefits of the technology and how it can address several traditional problems in terms of wasted resources, unmet expectations and slow adoption of emerging technologies. the focus of the paper is to landscape the newly emerged area in is domain to develop a theoretical understanding to illustrate key components of the blockchain technologies and promote is research in this area. australasian journal of information systems prokofieva & miah 2019, vol 23, research note blockchain in healthcare 3 for the is research the study pioneers blockchain research by presenting a consistent and systematic review of current research and developments in health blockchain. triangulating academic research, practical implementations and user experience develop a unique perspective in blockchain applications in health to equip researchers and professionals with a knowledge and skills required for further knowledge advancements. the paper contributes to the research in blockchain by presenting a structured overview of existing academic research in healthcare blockchain and identifying directions of future developments. contribution of the paper in healthcare research is embodied by highlighting new critical areas of health information systems research that employs blockchain opportunities. the paper is structured as follows. the next section provides the background information on the blockchain and its applications across various industries. section 3 overviews the methodology and provides the summary of the research process. section 4 presents the finding of the research and section 5 concludes the paper by discussing the results and summarising limitations and outlining the directions for future research. 2 background 2.1 blockchain: what is it? as a recent survey shows deloitte (2018), while applications of the blockchain technology are rapidly growing, understanding of the actual technology and its potentials is far from widespread. this creates an obstacle for further developments and implementation of blockchain which has a potential to introduce a dramatic change in the society, including improvements in the healthcare information exchange. this technological solution works mainly in a decentralised manner to immutably store digital data, so that generated data or meta-data can be securely shared across networks and users. blockchain includes a list of records that are linked using cryptography (zhang, white, et al., 2018). this design makes modification of the data impossible: once recorded in a block, the data cannot be altered without alteration of all subsequent blocks, requiring a consensus of the network majority. as a decentralised ledger system, blockchain allows independent agents to collaborate within the ecosystem ensuring transparency and time-stamped recording of the information to bring improvement in processing speed, revenue generation, security, while decreasing risk and costs (deloitte, 2018). unlike traditional ways to store information on one central point, the blockchain uses a peer-to-peer (p2) network involves multiple copies of the same data that are stored in different locations and different devices. a peer allows a portion of computing resources (e.g. disk storage, network bandwidth, processing powers), to be used by other participants without the need for central coordination by servers. such nodes can take different roles within the network while ensuring preservation, coordination and security of business data exchanges (van rijmenam & ryan, 2018). the security of the blockchain is implemented using cryptographic keys, a distributed network and a network servicing protocol by recording information in a block. once information (e.g. a transaction request) is validated, meta-data is recorded in a block and cannot be disputed, removed or altered without the knowledge and permission of those who created the record as well as the network. when a block is added to the chain of other blocks it remains unchanged. one of the applications of blockchain is smart contracts, also referred to as self-executing contracts, blockchain contracts, or digital contracts (griggs et al., 2018; zhang, white, et al., australasian journal of information systems prokofieva & miah 2019, vol 23, research note blockchain in healthcare 4 2018). smart contract is a software program that self-execute complex instructions, thus reduces the costs of contracting, monitoring and enforcing contracts and making payments. smart contracts open up ways to develop rich applications that can act as autonomous agents. this arrangement eliminates agency and coordinating costs and can potentially lead to any distributed enterprises such as in hospital systems. while the initial growth was happening within the financial industry, implementations are under way across a wider range of fields, including marketing (kumar, 2018), supply chain (zheng, xie, dai, chen, & wang, 2017). for example, in marketing, especially martech, blockchain implementations based on smart contracts are used to ensure unified promotion actions and timely launch of promotional campaigns. this approach has a potential to simplify the industry landscape and eliminate cost-adding middleman parties as well as reduce the risk of fraudulent traffic reports and multiple ad requests (sklaroff, 2017). in supply chain popular applications of blockchain include, but not limited to smart contracts and global searchable databases of transactions. a real-world example of the application of such technology includes the collaboration between wave and barclays, which are two start-ups working in the area of blockchain (kelly, 2016). in 2016 they completed their first smart contract for a trade finance transaction between an irish agri-food cooperative and seychelles trading company. further application of the technology can be seen with walmart which introduced a blockchain-based system for some of its suppliers in 2018 (corkery & popper, 2018). the use of blockchain in supplier-customer relationship allows to ensure that the product is delivered timely and in good condition without involvement of the intermediaries, thus cutting the cost, reducing the waste and promoting sustainability (corkery & popper, 2018). 2.2 blockchain in health the characteristics of the blockchain, which are its decentralised nature, openness and permissionless, may offer a unique solution for healthcare. wider applicability of the technology paves its way into different aspects of healthcare, including wearables and progress of medical research. healthcare sector has growing demands for blockchain developments and a recent survey by deloitte shows that the traditional industry is actively explores new avenues for the use of the blockchain to address its critical needs. (deloitte, 2018). immutability of the blockchain is the vital option for healthcare data. it can secure health records, the results of clinical trials and ensure regulatory compliances. employment of smart contracts demonstrate how blockchain can be used to support real-time patient monitoring and medical interventions (griggs et al., 2018). such systems ensure security of records while providing access for patients and medical professionals in a health insurance portability and accountability act (hipaa) compliant manner. further application of blockchain relates to supply chain in pharma and developing measures against counterfeit drugs. while the development of new drugs incur substantial costs related to trials to evaluate safety and efficacy of the drug, the use of smart contracts allow to facilitate the procedure of the informed consent as well as improve identify management and data quality (razak, 2018). providing access to patients for managing their own identify also allow integration of the informed consent procedure while ensuring privacy of individual heath data. australasian journal of information systems prokofieva & miah 2019, vol 23, research note blockchain in healthcare 5 3 systematic literature methodology to understand the potential of the blockchain in healthcare, assess its application and uncover implementation issues, the study has conducted a systematic literature review of academic and industry publications between 2008 and 2019. considering the innovative nature of the blockchain and longer time frames for reviews and publications of academic papers, the main focus was given to the analysis of publications through open-sourced google scholar. the criteria for inclusion of the papers in the review were defined as being published between 2008 and 2019, published in premier academic journals and conference proceedings1 that were in english. this ensured credibility of the sources. we also reviewed industry publications to ensure the rapidly changing nature of the blockchain is validly reflected in the selected academic publications. the time frame for the analysis was selected from the first reference to the blockchain appeared in 2008 and continued until the paper online publication in 2019. it is due to the fact that most of the whole research have been appeared in recent years going beyond the widespread conversations or online blogs. the preliminary results showed that very limited substantial publications were available before 2016, but further analysis included all selected papers. we started the review by collected papers through two sources, including academic databases and google scholar. overall, we perform the search in 14 journal databases related to information systems and healthcare and included published papers as well as articles in press (till march 2019). appendix a illustrates the names of the 14 electronic databases. through the database search, we have reviewed over 25 premier healthcare, information systems and business journals (provided in appendix b). the selected journals are ranked as q1 in the scimago ranking system2, which was our initial basis of inclusion in the review process (see appendix b for the list of the selected journals). to automate data collection through databases, we developed a script in r programming language that accessed the journals metadata using application programming interface (apis) of the relevant databases. the extract from the script is available in appendix c3. the list of search terms included “blockchain”, “block chain”, “distributed ledger”, “smart contracts” to satisfy prisma conditions (moher, liberati, tetzlaff, altman, & group, 2009)4. the prisma (i.e. preferred reporting items for systematic reviews and meta-analyses) framework specifies an evidence-based minimum set of items for reporting in systematic reviews and meta-analyses and has been widely utilised in the academic studies, including (kruse, goswamy, raval, & marawi, 2016). the benefits of using prisma for the analysis allowed to employ guidelines to review clearly formulated questions and use systematic and explicit methods to locate, select, and critically evaluate relevant publications to address the research questions identified earlier. the downloaded meta-data was further examined independently by two independent researchers to analyse its relevance to the study. the articles not in english were excluded. the 1 the full list of reviewed sample papers (136) is available at the online repository: https://github.com/mariaprokofieva/digit 2 scimago (https://www.scimagojr.com/journalrank.php) is a scopus driven ranking system for classifying the academic journals into various rankings on the basis of citations and h-index. 3 we also provide a link to the full script available at the github repository, https://github.com/mariaprokofieva/digit 4 prisma checklist includes 27 items related to the content of a systematic review. the full checklist is available at http://www.prisma-statement.org/documents/prisma%202009%20checklist.pdf australasian journal of information systems prokofieva & miah 2019, vol 23, research note blockchain in healthcare 6 selection method is presented in figure 1 panel a. the results were analysed to produce the final list of papers that included 136 articles. further analysis of the identified papers was performed using content analysis technique which is a research method for exploring content from human interaction process, verbal and written document with a purpose of analysing data (creswell & poth, 2018). it is a powerful method that allows researchers to analyse documents as important sources of information to identify patterns of content quantitatively as well as qualitatively analyse meanings of content to identify new phenomena. content analysis has been widely used in is research (al-debei & avison, 2017; miah, gammack & hasan, 2017). we employed both qualitative and quantitative approaches to content analysis. further review of the selected papers was done using an input-process-output perspective (ipo). this approach has been widely used in prior research (chan & ngai, 2011) and is based on analysing inputs and outputs and understanding the underlying processes, i.e. antecedents, processes and consequences of blockchain application in healthcare. panel a. sample papers selection process panel b. input-process-output figure 1. literature review process inputs antecedents of blockchain applications in healthcare. problems that existing solutions are unable to resolve, process technological solution that employs blockchain to address existing deficiencies and needs in healthcare. platforms, systems, interfaces, frameworks. output impact of the blockchain solution in the healthcare application. practical implementation, industry solution, theoretical framework that translates inputs through process to achievable results. australasian journal of information systems prokofieva & miah 2019, vol 23, research note blockchain in healthcare 7 the developed ipo framework is presented in figure 1 panel b. findings the analysis of the findings showed that 136 papers identified as final samples are originating from different areas of healthcare-related fields. generally, publications come from three major topic areas, including technology, life sciences and biomedicine and social sciences. figure 2 shows distribution of articles across these three areas, including the breakdown of particular research fields. the majority of the publications are cross-disciplinary research, and the separation between the research areas is relatively blurred. the identified three research areas suggest three aspects of blockchain life cycle in healthcare, from idea generation and experimentation, to commercialization and to diffusion and implementation. these stages coincide with stages of innovation process (desouza et al., 2009) as well as key elements in diffusion of innovations (greenhalg, robert, bate, macfarlane, & kyriakidou, 2005). while, inclusion of life science and bio-medicine area is a natural extension of the healthcare application of blockchain, it and is provide critical components in blockchain life cycle as they provide the technological foundation for implementation, diffusion and business use of the blockchain in healthcare. the social science area signifies the commercialisation and adoption of the blockchain in organizations and society (desouza et al., 2009). it also reflects the adoption aspect of the blockchain as affected by social characteristics of its application and diffusion throughout the social system (valente, 1996). the timeline of publications (figure 2 panel b) suggests that the majority of publications are from the last 2 years while the volume is growing. the results demonstrate the network effect which becomes visible as the increase in the number of users improve the value of a product or service. this is also supported by the recent gartner report (gartner, 2018) that suggests that blockchain is entering the early majority adoption stage as the technology is spreading out throughout the society. however, the use of blockchain in healthcare is at the initial stage and mainstream adoption is likely to be reached only in 10 years. driven by the cryptocurrencies development which current usage reached more than 20 million users worldwide (coinbase, 2019), the use of blockchain in other areas is also growing. the accelerated development is generally slowed by costs of private ledgers deployment and positive returns from it (gartner, 2018). this is in part due to undeveloped application market and complexity. companies use self-created cryptocurrencies, i.e. initial coin offerings (ico) that has a value of company shares. ico provides blockchain developers with an option to crowdsource development funds and accelerate the development of the blockchain. ico tokens are generally associated with access to platforms or service provided by developers, while do not give ownerships or decision-making rights. the review of blockchain projects available at icobench5 shows 5,533 published projects out of which 54.67% projects related to the development of blockchain platforms, followed by cryptocurrency (39.85%) and business services projects (22.48%), while healthcare projects take only 4.97% (icobench, 2019). the healthcare blockchain market in 2017-2018 was assessed at usd $34.47 million and is expected to grow at compound annual growth rate of 70.45% to generate revenue of usd $1,415.59 million by 2024 (zion market research, 2018). 5 icobench.com is free initial coin offering (ico) rating platform supported by the blockchain community which provides insights to the investors. australasian journal of information systems prokofieva & miah 2019, vol 23, research note blockchain in healthcare 8 the majority of publications are coming from us and china (see figure 2 panel c). while the original blockchain interest was stimulated through developments in the us, the latest breakthrough of china deserves a special interest. china was reported to have 263 blockchain projects as on november 2018 under development which is accounted for more than 25 percent of projects worldwide (china daily, 2019). these projects include well established chinese companies, such as baidu, alibaba and tencent, while each of these companies show interest in healthcare applications and establishing their strong presence there. china is also reported to be a leader in blockchain patents with more than half of all patents applications in 2017 (china daily, 2019). the development of blockchain has a strong support from the chinese government through the development of the blockchain development centre and publication of a list of objectives to encourage the introduction and standardization of blockchain technology in 2018. while the chinese government has introduced various bans on cryptocurrency trading and use, the blockchain technology is promoted at various levels. for examples, the china’s 13th five-year plan is shown to link domestic and international resources, support investments in blockchain projects and participation in blockchain related events at the international level. panel a. distribution of identified papers across research areas panel b. distribution of publications across years technology 41% life sciences and biomedicine 36% social sciences 23% technology area includes such subfields as computer science, engineering, telecommunication, information science, automation control systems. life sciences & biomedicine area includes such subfields as healthcare sciences services, medical informatics, computational biology, legal medicine, medical ethics, biochemistry, biotechnology, general internal medicine. social sciences area includes such subfields as business economics, communication, behavioral sciences, psychology, sociology, law. definition of the areas and subfields australasian journal of information systems prokofieva & miah 2019, vol 23, research note blockchain in healthcare 9 panel c. distribution of publications across countries/regions figure 2. analysis of the results the analysis of the papers identified the several major topics that present research opportunities and challenges for current and future advancements. application of ipo framework allowed to organise the findings to focus on business problems, rather than the technological solution. the results of the analysis are presented in table 1. 0 5 10 15 20 25 30 35 40 45 usa china uk canada india south korea japan other australasian journal of information systems prokofieva & miah 2019, vol 23, research note blockchain in healthcare 10 input business problem process solution output icobench projects examples of academic papers low transparency of contracts and high intermediary costs use of smart contracts decentralised consensus improves the healthcare system transparency while removing an intermediary party that inflates the cost of healthcare. further benefits include a significant reduction of inefficiency and waste in the healthcare ecosystem(zhang, schmidt, et al., 2018). the use of encryption, transparency and decentralised ledger removes miscommunication and mistrust and promote a transition to valuebased care (griggs et al., 2018) resulting in negotiations of complex bundle claims that are tied to value, but not on opportunistic fee-for-service model, while making claiming adjudication process seamless. insure network: a decentralised network that uses smart contracts and artificial intelligence to ensure low cost, transparency and integrity of data when dealing with insurance claims. aimedis: a medical ecosystem that is based on smart contracts to connect patients, doctors, and hospitals. users are given full control over their medical information and can share it using right management system. the system also integrates a videochat and allows communication with other patients and doctors, provision of online prescriptions and online appointments, as well as inclusion of fitness trackers and wearables. hcx pay: an insurance platform that provides payment and data solutions using smartcontracts and hcxi tokens. dagher, mohler, milojkovic, & marella, 2018 griggs et al., 2018 macrinici, cartofeanu, & gao, 2018 stanciu, 2017 difficulties in supervision of drug intake and distribution healthcare supply chain processes the use of the distribute ledger can improve supervision of drug intake and distribution (tseng et al., 2018), promote regulatory compliance or managing healthcare supplies (raghavendra, 2019), such as in compliance with hipaa. the use of blockchain is shown to address a vital problem in healthcare supply chain which is lack of standardisation of business processes. it can promote the missing cooperation between business partners is necessary in areas such as product identification and barcoding, master data synchronisation, ordering, delivery and invoicing farmatrust: a platform to facilitate supply chain within the pharmaceutical industry. the platform aims to eliminate counterfeit drugs and increase efficiencies, while ensuring customer access. baikalika: a solution to ensure protection against counterfeit products, and water safety and quality control. the solution uses blockchain to confirm the place of origin and a transparent chain of supply to the end-user. jayaraman, alhammadi, & simsekler, 2018 mackey et al., 2019 radanović & likić, 2018 tseng et al., 2018 australasian journal of information systems prokofieva & miah 2019, vol 23, research note blockchain in healthcare 11 processes as well as logistics processes. with rising costs and standards of living, healthcare is experiencing increasing political pressure to control costs and improve efficiency, making the need for optimisation even more urgent. absence of unified access to patient data data storage and exchange a unified decentralised database can store individual patient details, including claims, medical history, transactions, overdue payments, etc. this database facilitates the transfer or scheduling of appointments with medical staff which can be activated using smart contracts. the use of smart contracts can ensure medical staff availability and timely payments (yue, wang, jin, li, & jiang, 2016). further opportunities are present by extended the stored data by the massive data accumulated through healthcare hardware and wearables (liang, zhao, shetty, liu, & li, 2017). such devices, including life support machine, activity trackers etc., monitor and record physical development, while a unified distributed access to this information can ensure effective decision making by medical staff, patience satisfaction and privacy. blockchain offers a strong opportunity to streamline such data networks and secure security and privacy of information that they produce and retain. nwpsolution: a platform that integrates smart devices and databases to provide access to healthcare. the platform uses artificial intelligence, data security and blockchain to minimise the healthcare costs. medicohealth: a platform that allows unified access to patient data as well as provide patients with access to physicians or specialists of their choice to receive a prompt, fordable and reliable consultation about their condition alphacon: a platform to distribute healthcare data and provide personalized healthcare solutions using big data and analytics. cura network: a platform that facilitates managing and sharing healthcare data by providing a secure and instant access to records seamlessly across institutions. efanov & roschin, 2018 gordon & catalini, 2018 yue et al., 2016 zhang, white, et al., 2018 australasian journal of information systems prokofieva & miah 2019, vol 23, research note blockchain in healthcare 12 data integrity issues through concerns of confidentiality and protection integration and encryption of digital assets blockchain provides opportunities to integrate and encrypt digital assets, including medical records, or processing claims on a ledger (yue et al., 2016). the ledger can ensure patient confidentiality and protection of relevant data as well as ensure regulatory compliance, for example hipaa. private blockchain can be used to store and access research output conducted in compliance with hipaa laws in a confidential and secure manner, which can ensure access via extra authorization by concerned parties. medicohealth: apart from creating a doctorpatient environment, the platform ensures fully anonymous and protected communication with leading physicians. physician details, including license validity information are stored in an immutable decentralised database. patient data is anonymous and accessed only by selected physicians for a limited amount of time. payments are fully tokenised and anonymous. the system operates on tokens that are used to compensate service provider. anjum, sporny, & sill, 2017 esposito, de santis, tortora, chang, & choo, 2018 yue et al., 2016 lack of centralised approach to financing opportunities to promote and disseminate high quality medical research facilitation and promotion of medical research another strong application of the blockchain technology is facilitation of clinical studies and disease prevention (ekblaw, azaria, halamka, lippman, & vieira, 2016). with competition of funding for medical research, centralised approach to financing opportunities promotes high quality research. a suggested approach may include using a research-based blockchain ledger where researcher create their own tokens, or participate in the ledger to fund their projects. the ledger allows donors to interact and choose areas of interest, access real-time research data and as well as allow for crowdfundinglike donation system based on micropayments. the trustless nature of the ledger and transparency minimize the chance of academic fraud and of duplication of work. the same embleema: a network that connects patients directly with medical research providers, allows patients to manage their data and provide incentives to share their data through the use of tokens. the network introduce an alternative way on how clinical research data is currently collected by pharmaceutical companies through contrat research organisation. the data can be used to assess the efficacy of new investigational drugs and speed up the process of drug approvals through data exchange between patients, life sciences and regulators. ojomoko et al., 2017 shae & tsai, 2017 till, peters, afshar, & meara, 2017 https://lisk.io/academy/blockchain-business/cryptocurrencies https://lisk.io/academy/blockchain-basics/benefits-of-blockchain/why-is-blockchain-trustless australasian journal of information systems prokofieva & miah 2019, vol 23, research note blockchain in healthcare 13 ledger further allows distribution of digital research information and its transparency. table 1. input-process-output framework of blockchain health australasian journal of information systems prokofieva & miah 2019, vol 23, research note blockchain in healthcare 14 the analysis of the literature allowed to identify the following issue in current healthcare blockchain solutions: • privacy issues: increase in the user base of blockchain-related implementations leads to the increase in the information required to be verified and distributed across the network. with multiple validators that are required, such distribution creates a potential privacy issue. • coordination issues: inflated flow of information in the decentralised blockchain bring the issue of coordination among agents on the network and requires technical proficiency. • time issues: the flow of information poses a question of its organisation and speed of processing. • human factor issues: adoption and development of blockchain-based system requires sufficient familiarity with the system and benefits/risks it introduces. 4 discussion and conclusion the blockchain have become an important technological movement for is application design. it is therefore of paramount task to landscape knowledge for developing understanding so that design practices can be improved. in healthcare, there are critical challenges for information exchange and disseminations, service providers and patients are required to have combined secured data exchange technologies to make informed clinical decisions. in this paper, we uncovered insights of blockchain based technologies and their potential applications for healthcare sectors. as it was discussed earlier, digitalization of the records allows to create further opportunities for analysing medical trends and evaluation of the quality of care, relevant technologies are still emerging and blockchain can offer various benefits for the support service. while records are digitised, they are isolated in locally centralised data storages which present a strong impediment to further developments. implementation of blockchain improves the connectivity while security and privacy and reduces the costs are also ensured for the parties. in this paper we introduced an automated method for collecting published relevant literature through the use of a text mining algorithm which provided expected outcome successfully for conducting the review study. we provided the r script of the searching method to access journal’s meta-data using the database api (application programming interface) in the appendix c. the literature review initiated to develop strong profile of blockchain in the following areas of healthcare application where provisions can be viewed in terms of smart contracts, data storage and exchange, physician credentialing and peer-topeer data exchange. these would be considered as further aspects for healthcare solutions design and their innovations in meeting contextual or organisational demands for information disseminations. in addition, it is anticipated that blockchain based medical records can improve the accuracy of diagnoses, a more information treatment choice and a more cost-effective solution by using a database with medical records as individual patience information. concluding the paper, it is necessary to highlight rapid development of blockchain implementations in healthcare, which is slowed down by the issues identified earlier. our systematic literature review revealed opportunities as well as challenges that blockchain offers australasian journal of information systems prokofieva & miah 2019, vol 23, research note blockchain in healthcare 15 to the healthcare industry. data privacy and security are the primary concern, identified in more than 50% of the paper. further challenges relate to data structure and coordination. the findings of the paper identify directions for future research that will promote further developments and innovation in health information systems design (eg public healthcare decision support (miah et al, 2017) or other similar problem domain such as agricultural decision support system (eg knowledge-based decision support platform (miah, 2008)) in which blockchain can offer authenticated data dissemination methods. acknowledgement the authors would like to thank weng marc lim and two independent reviewers for providing constructive feedback to improve the overall quality of the paper in the three revision cycles. their feedback was valuable and helpful to reshape the paper. references al-debei, m. m., & avison, d. (2017). developing a unified framework of the business model concept developing a unified framework of the business model concept. european journal of information systems, 9344. https://doi.org/10.1057/ejis.2010.21 anjum, a., sporny, m., & sill, a. (2017). blockchain standards for compliance and trust. ieee cloud computing, 4(4), 84–90. https://doi.org/10.1109/mcc.2017.3791019 arnott, d., & pervan, g. (2012). design science in decision support systems research: an assessment using the hevner, march, park, and ram guidelines. journal of the association for information systems, 13(11), 923–949. benchoufi, m. (2017). blockchain technology for improving clinical research quality, 1–5. https://doi.org/10.1186/s13063-017-2035-z casino, f., dasaklis, t. k., & patsakis, c. (2019). a systematic literature review of blockchainbased applications : current status , classification and open issues. telematics and informatics, 36(may 2018), 55–81. https://doi.org/10.1016/j.tele.2018.11.006 chan, y. y. y., & ngai, e. w. t. (2011). conceptualising electronic word of mouth activity: an input-process-output perspective. marketing intelligence and planning, 29(5), 488–516. https://doi.org/10.1108/02634501111153692 china daily. (2019). nation leads world in blockchain projects. retrieved may 21, 2019, from http://www.china.org.cn/business/2019-04/02/content_74636929_2.htm coinbase. (2019). about coinbase. retrieved may 30, 2019, from https://coinbase.com/about corkery, m., & popper, n. (2018, september 24). from farm to blockchain: walmart tracks its lettuce. the new york times. retrieved from https://www.nytimes.com/2018/09/24/business/walmart-blockchain-lettuce.html creswell, j. w., & poth, c. n. (2018). qualitative inquiry and research design : choosing among five approaches. dagher, g. g., mohler, j., milojkovic, m., & marella, p. b. (2018). ancile: privacy-preserving framework for access control and interoperability of electronic health records using blockchain technology. sustainable cities and society, 39, 283–297. https://doi.org/10.1016/j.scs.2018.02.014 australasian journal of information systems prokofieva & miah 2019, vol 23, research note blockchain in healthcare 16 deloitte. (2018). breaking blockchain opendeloitte’s 2018 global blockchain survey. deloitte insights, 48. https://doi.org/10.1002/ejoc.201200111 desouza, k. c., dombrowski, c., awazu, y., baloh, p., papagari, s., jha, s., & kim, j. y. (2009). crafting organizational innovation processes. innovation: management, policy and practice, 11(6), 33. efanov, d., & roschin, p. (2018). the all-pervasiveness of the blockchain technology. procedia computer science, 123, 116–121. https://doi.org/10.1016/j.procs.2018.01.019 ekblaw, a., azaria, a., halamka, j. d., lippman, a., & vieira, t. (2016). a case study for blockchain in healthcare: &quot;medrec&quot; prototype for electronic health records and medical research data. in 2nd international conference on open & big data 2016. retrieved from https://pdfs.semanticscholar.org/56e6/5b469cad2f3ebd560b3a10e7346780f4ab0a.pdf esposito, c., de santis, a., tortora, g., chang, h., & choo, k. k. r. (2018). blockchain: a panacea for healthcare cloud-based data security and privacy? ieee cloud computing, 5(1), 31–37. https://doi.org/10.1109/mcc.2018.011791712 gordon, w. j., & catalini, c. (2018). blockchain technology for healthcare : facilitating the transition to patient-driven interoperability. computational and structural biotechnology journal, 16, 224–230. https://doi.org/10.1016/j.csbj.2018.06.003 greenhalg, t., robert, g., bate, p., macfarlane, f., & kyriakidou, o. (2005). diffusion of innovations in health service organisations. (t. greenhalgh, g. robert, p. bate, f. macfarlane, & o. kyriakidou, eds.). oxford, uk: blackwell publishing ltd. https://doi.org/10.1002/9780470987407 griggs, k. n., ossipova, o., kohlios, c. p., baccarini, a. n., howson, e. a., & hayajneh, t. (2018). healthcare blockchain system using smart contracts for secure automated remote patient monitoring. journal of medical systems, 42(7), 1–7. https://doi.org/10.1007/s10916018-0982-x icobench. (2019). icos and ratings. retrieved may 29, 2019, from https://icobench.com/icos jayaraman, r., alhammadi, f., & simsekler, m. c. e. (2018). managing product recalls in healthcare supply chain. in 2018 ieee international conference on industrial engineering and engineering management (ieem) (pp. 293–297). ieee. https://doi.org/10.1109/ieem.2018.8607403 kelly, j. (2016, september 8). barclays says conducts first blockchain-based trade-finance deal. reuters. kruse, c. s., goswamy, r., raval, y., & marawi, s. (2016). challenges and opportunities of big data in health care: a systematic review. jmir medical informatics, 4(4), e38. https://doi.org/10.2196/medinform.5359 kumar, v. (2018). transformative marketing: the next 20 years. journal of marketing, 82(4), 1– 12. https://doi.org/10.1509/jm.82.41 liang, x., zhao, j., shetty, s., liu, j., & li, d. (2017). integrating blockchain for data sharing and collaboration in mobile healthcare applications. in 2017 ieee 28th annual australasian journal of information systems prokofieva & miah 2019, vol 23, research note blockchain in healthcare 17 international symposium on personal, indoor, and mobile radio communications (pimrc) (pp. 1–5). ieee. https://doi.org/10.1109/pimrc.2017.8292361 mackey, t. k., kuo, t.-t., gummadi, b., clauson, k. a., church, g., grishin, d., … palombini, m. (2019). ‘fit-for-purpose?’ – challenges and opportunities for applications of blockchain technology in the future of healthcare. bmc medicine, 17(1), 68. https://doi.org/10.1186/s12916-019-1296-7 macrinici, d., cartofeanu, c., & gao, s. (2018). smart contract applications within blockchain technology: a systematic mapping study. telematics and informatics, 35(8), 2337–2354. https://doi.org/10.1016/j.tele.2018.10.004 miah, s.j., gammack, g., & hasan, n. (2017). extending the framework for mobile health information systems research: a content analysis, information systems, 69, 1–24 miah, s.j. (2018). an ontology based design environment for rural decision support, unpublished phd thesis, griffith business school, griffith university, brisbane, australia miah, s.j., hasan, r., hasan, n., & gammack, j. (2017). a decision support framework for public healthcare: an approach to follow-up support service, in proceedings of the 28th australasian conference on information systems,03 dec 2017, hobart, australia moher, d., liberati, a., tetzlaff, j., altman, d. g., & group, t. p. (2009). preferred reporting items for systematic reviews and meta-analyses: the prisma statement. plos medicine, 6(7), 4419–4424. https://doi.org/10.1371/journal.pmed.1000097 ojomoko, l., romantsov, k., yanovich, y., prikhodko, p., ostrovski, a., izumchenko, e., … botezatu, a. (2017). converging blockchain and next-generation artificial intelligence technologies to decentralize and accelerate biomedical research and healthcare. oncotarget, 9(5), 5665–5690. https://doi.org/10.18632/oncotarget.22345 radanović, i., & likić, r. (2018). opportunities for use of blockchain technology in medicine. applied health economics and health policy, 16(5), 583–590. https://doi.org/10.1007/s40258018-0412-8 raghavendra, m. (2019). can blockchain technologies help tackle the opioid epidemic: a narrative review. pain medicine. https://doi.org/10.1093/pm/pny315 razak, o. (2018, december 6). revolutionizing pharma — one blockchain use case at a time. medium. retrieved from https://medium.com/linum-labs/revolutionizing-pharma-oneblockchain-use-case-at-a-time-e7922c8aa9a3 shae, z., & tsai, j. j. p. (2017). on the design of a blockchain platform for clinical trial and precision medicine. in 2017 ieee 37th international conference on distributed computing systems (icdcs) (pp. 1972–1980). ieee. https://doi.org/10.1109/icdcs.2017.61 sklaroff , j . m. (2017). smart contracts and the cost of inflexibility. university of pennsylvania law review , 166(1), 263–305. retrieved from https://heinonline.org/hol/page?handle=hein. journals/pnlr166&id=267 &div=8&collection=journals stanciu, a. (2017). blockchain based distributed control system for edge computing. in 2017 21st international conference on control systems and computer science (cscs) (pp. 667–671). ieee. https://doi.org/10.1109/cscs.2017.102 australasian journal of information systems prokofieva & miah 2019, vol 23, research note blockchain in healthcare 18 till, b. m., peters, a. w., afshar, s., & meara, j. g. (2017). from blockchain technology to global health equity: can cryptocurrencies finance universal health coverage. bmj global health, 2(4), 1–6. https://doi.org/10.1136/bmjgh-2017-000570 tseng, j.-h., liao, y.-c., chong, b., liao, s., tseng, j.-h., liao, y.-c., … liao, s. (2018). governance on the drug supply chain via gcoin blockchain. international journal of environmental research and public health, 15(6), 1055. https://doi.org/10.3390/ijerph15061055 valente, t. w. (1996). models of the diffusion of innovations. network models of the diffusion of innovation, 2(2), 163–164. van rijmenam, m., & ryan, p. (2018). blockchain : transforming your business and our world. routledge. yue, x., wang, h., jin, d., li, m., & jiang, w. (2016). healthcare data gateways: found healthcare intelligence on blockchain with novel privacy risk control. journal of medical systems, 40(10), 218. https://doi.org/10.1007/s10916-016-0574-6 zhang, p., schmidt, d. c., white, j., & lenz, g. (2018). blockchain technology use cases in healthcare. in blockchain technology: platforms, tools and use cases (vol. 111, pp. 1–42). zhang, p., white, j., schmidt, d. c., lenz, g., & rosenbloom, s. t. (2018). fhirchain: applying blockchain to securely and scalably share clinical data. computational and structural biotechnology journal, 16, 267–278. https://doi.org/10.1016/j.csbj.2018.07.004 zheng, z., xie, s., dai, h., chen, x., & wang, h. (2017). an overview of blockchain technology: architecture, consensus, and future trends. in proceedings 2017 ieee 6th international congress on big data, bigdata congress 2017 (pp. 557–564). https://doi.org/10.1109/bigdatacongress.2017.85 australasian journal of information systems prokofieva & miah 2019, vol 23, research note blockchain in healthcare 19 appendix a. journal databases related to healthcare, information systems and business disciplines • academic search elite • acm digital library • ais electronic library (aisel) • biomed central • bmj journals collection • business source complete • directory of open access journals (doaj) • emerald • jstor • medline • pubmed • sage journals • sciencedirect • scopus australasian journal of information systems prokofieva & miah 2019, vol 23, research note blockchain in healthcare 20 appendix b. example of premier healthcare, information systems and business journals used in literature search • acm transactions on information systems • asia pacific journal of management • australasian journal of information systems • briefings in bioinformatics • computational and structural biotechnology journal • decision sciences • decision support systems • european journal of information systems • implementation science • information and management • information and organization • information systems research • international journal of medical informatics • journal of biomedical informatics • journal of intelligent and fuzzy systems • journal of management • journal of medical internet research • journal of the american medical informatics association: jamia • journal of the association of information systems • journal of the medical library association: jmla • medical reference services quarterly • mis quarterly: management information systems • nursing education perspectives • telematics and informatics • trials australasian journal of information systems prokofieva & miah 2019, vol 23, research note blockchain in healthcare 21 appendix c. example of the r script to access journal’s meta-data using the database api #the full script is available for public use at the github repository https://github.com/mariaprokofieva/digit #fulltext is a package to help r users to get published literature from the web in preferred formats and across different publishers #the package allows to search for articles, gets abstracts and full text articles, extract text from articles, etc. #fulltext currently supports search for plos, crossref, entrez, arxiv, and bmc, biorxiv, europepmc, and scopus. #a typical workflow with the package is to search for articles with ft_search(), download articles with ft_get(), collect the text into an object with ft_collect(), get particular sections with ft_chunks(), or use other text-mining packages to analyse the text further #install the latest version of fulltext library from cran #using devtools library install.packages("devtools") devtools::install_github("ropensci/fulltext") #load fulltext library library(fulltext) # searching crossref (res2 <ft_search(query= ‘blockchain’, from='crossref')) res2$crossref #searching bmc (res <ft_search(query='blockchain', from='bmc')) res$bmc australasian journal of information systems prokofieva & miah 2019, vol 23, research note blockchain in healthcare 22 #searching europe pmc (res <ft_search(query=' blockchain', from='europmc')) res$europmc #searching scopus (res <ft_search(query = 'blockchain', from = 'scopus', limit = 100, scopusopts = list(key = sys.getenv('elsevier_scopus_key')))) res$scopus #the list is long, applying pagination (res <ft_search(query = 'blockchain', from = 'scopus', scopusopts = list(key = sys.getenv('elsevier_scopus_key')), limit = 5)) copyright: © 2019 prokofieva & miah. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 background 2.1 blockchain: what is it? 2.2 blockchain in health 3 systematic literature methodology findings dagher, mohler, milojkovic, & marella, 2018 insure network: a decentralised network that uses smart contracts and artificial intelligence to ensure low cost, transparency and integrity of data when dealing with insurance claims. griggs et al., 2018 macrinici, cartofeanu, & gao, 2018 aimedis: a medical ecosystem that is based on smart contracts to connect patients, doctors, and hospitals. users are given full control over their medical information and can share it using right management system. the system also integrates a videochat and allows communication with other patients and doctors, provision of online prescriptions and online appointments, as well as inclusion of fitness trackers and wearables. stanciu, 2017 hcx pay: an insurance platform that baikalika: a solution to ensure protection against counterfeit products, and water safety and quality control. the solution uses blockchain to confirm the place of origin and a transparent chain of supply to the end-user. efanov & roschin, 2018 nwpsolution: a platform that integrates smart devices and databases to provide access to healthcare. the platform uses artificial intelligence, data security and blockchain to minimise the healthcare costs. gordon & catalini, 2018 yue et al., 2016 medicohealth: a platform that allows unified access to patient data as well as provide patients with access to physicians or specialists of their choice to receive a prompt, fordable and reliable consultation about their condition alphacon: a platform to distribute healthcare data and provide personalized healthcare solutions using big data and analytics. cura network: a platform that facilitates managing and sharing healthcare data by providing a secure and instant access to records seamlessly across institutions. anjum, sporny, & sill, 2017 medicohealth: apart from creating a doctor-patient environment, the platform ensures fully anonymous and protected communication with leading physicians. physician details, including license validity information are stored in an immutable decentralised database. patient data is anonymous and accessed only by selected physicians for a limited amount of time. payments are fully tokenised and anonymous. the system operates on tokens that are used to compensate service provider. esposito, de santis, tortora, chang, & choo, 2018 ojomoko et al., 2017 embleema: a network that connects patients directly with medical research providers, allows patients to manage their data and provide incentives to share their data through the use of tokens. the network introduce an alternative way on how clinical research data is currently collected by pharmaceutical companies through contrat research organisation. the data can be used to assess the efficacy of new investigational drugs and speed up the process of drug approvals through data exchange between patients, life sciences and regulators. shae & tsai, 2017 4 discussion and conclusion paper title (paper title style) australasian journal of information systems warren, wahlstrom, wigan & burmeister 2020, vol 24, selected papers from the 2019 aice conference preface 1 preface: ethics in the cyber age and exploring emerging themes and relationships between ethics, governance and emerging technologies matthew warren, rmit university, australia. kirsten wahlstrom, university of south australia, australia. marcus wigan, oxford synaptics, australia. oliver burmeister, charles sturt university, australia. oburmeister@csu.edu.au this is special section comprises nine selected papers from the 8th australasian institute of computer ethics (aice) conference, held in melbourne, 19-20th august 2019. of the many papers presented the authors of these papers were invited to extend their arguments from the short conference paper to a full-length journal article. they all then underwent at least one round of review through the ajis system, before being accepted for this special section. the aice 2019 program committee were invited to be the joint guest editors of this special section. the aice 2019 conference followed on from the highly successful aice conference series, with the next one to be held at the end of this year. the conference theme was ‘ethics in the cyber age and exploring emerging themes and relationships between ethics, governance and emerging technologies’. there were two keynote speakers at the conference: katina michael, arizona state university, usa presented on ‘professional ethics and technology in the cyber age’, and greg adamson, university of melbourne, australia presented on ‘the role of standards in professional ethics in the ieee’. the nine articles in this special section address ethics in the cyber age from interesting perspectives. a historical review by wigan sets the scene for the whole special section. the topic of ‘harms’ is explored by kaluarachchi et al. in relation to cyberbullying, by warren in regard to fake news, and by wang in terms of gender. wang approach uses three lenses, the physical, institutional, and the psychological. ethical issues are explored in detail by robinson in terms of virtual influencers, by wildenaur in terms of shared responsibilities, and by wahlstrom et al. with a return to privacy, a topic frequently discussed at aice events. in this instance it is from the perspective of blockchains. fernando and scholl examine value tensions, seeking to reframe the value of data beyond market-based online social norms. finally, poulsen et al return to another common topic of aice events, namely codes of ethical conduct. the 9th aice conference will be held at the university of south australia in adelaide on november 26-27th 2020. the conference theme will be computer ethics in the new normal: australasian journal of information systems warren, wahlstrom, wigan & burmeister 2020, vol 24, selected papers from the 2019 aice conference preface 2 examining the challenges of online life. the call for papers is forthcoming and for those unable to travel, online video presentations will be available. about the editors where one of more of the guest editors were authors to a paper, they were excluded from all editorial decisions about those papers. the four guest editors were professor matthew warren (conference chair), dr kirsten wahlstrom, professor emeritus marcus wigan and professor oliver burmeister. professor warren has published eight times in ajis over the past 17 years (senarathna, et al, 2018, senarathna, et al, 2016, pye and warren 2006, pierce, et al 2006, leitch and warren 2003, warren and hutchinson 2003, hutchinson and warren 2002a, hutchinson and warren 2002b, hutchinson and warren 2001). he has served as a chair of information systems, then was a professor of cyber security at deakin university, before moving to his current role at rmit as a professor of cyber security. in her research, dr wahlstrom considers the ways in which emerging technologies disrupt social constructs, most recently with a focus on privacy and brain-computer interfaces (wahlstrom & fairweather, 2013; wahlstrom, fairweather, & ashman, 2016, 2017; wahlstrom, fairweather, ashman, & istance, 2013). she serves as the vice chair of the ethics committee of the australian computer society and she organises the 2020 aice conference, to be held at the university of south australia. although retired, prof wigan remains research active (clarke & wigan, 2018; wigan, 2012, 2014a, 2014b, 2017; wigan & clarke, 2013). he is both a current and past member of acs ethics committee and task force. most of burmeister’s research in ethics has to do with moral and social value considerations in codes of conduct (burmeister, 2017) and health information systems (bennett et al., 2017; burmeister, 2016; burmeister, islam, dayhew, & crichton, 2015; burmeister & kreps, 2018; burmeister et al., 2019; burmeister, weckert, & williamson, 2011; islam, miah, kamal, & burmeister, 2019; poulsen & burmeister, 2019; sayers, cleary, hunt, & burmeister, 2017; teipel et al., 2016). references bennett, e., berry, k., emeto, t. i., burmeister, o. k., young, j., & shields, l. (2017). attitudes to lesbian, gay, bisexual and transgender parents seeking health care for their children in two early parenting services in australia. journal of clinical nursing, 26(7-8), 1021– 1030. doi:10.1111/jocn.13595 burmeister, o. k. (2016). the development of assistive dementia technology that accounts for the values of those affected by its use. ethics and information technology, 18(3), 185-198. doi:10.1007/s10676-016-9404-2 burmeister, o. k. (2017). professional ethics in the information age. journal of information, communication and ethics in society, 15(4), 348-356. doi:10.1108/jices-11-2016-0045 burmeister, o. k., islam, m. z., dayhew, m., & crichton, m. (2015). enhancing client welfare through better communication of private mental health data between rural service providers. australasian journal of information systems, 19, 1-14. doi:10.3127/ajis.v19i0.1206 burmeister, o. k., & kreps, d. (2018). power influences upon technology design for age-related cognitive decline using the vsd framework. ethics and information technology, 20(3). burmeister, o. k., ritchie, d., devitt, a., chia, e., dresser, g., & roberts, r. (2019). the impact of telehealth technology on user perception of wellbeing and social functioning, and the australasian journal of information systems warren, wahlstrom, wigan & burmeister 2020, vol 24, selected papers from the 2019 aice conference preface 3 implications for service providers. australasian journal of information systems, 23. doi:10.3127/ajis.v23i0.1501 burmeister, o. k., weckert, j., & williamson, k. (2011). seniors extend understanding of what constitutes universal values. journal of information, communication & ethics in society, 9(4), 238-252. doi:10.1108/14779961111191048 clarke, r. a., & wigan, m. r. (2018). the information infrastructures of 1985 and of 2018: the sociotechnical context of computer law & security. computer law & security review: the international journal of technology law and practice, 34(4), 677-700. islam, m. r., miah, s. j., kamal, a. r. m., & burmeister, o. k. (2019). a design construct developing approaches to measure mental health conditions. australasian journal of information systems, 23. https://doi.org/10.3127/ajis.v23i0.1829 pierce, j., jones, a. and warren, m.j. (2006). penetration testing professional ethics: a conceptual model and taxonomy, australasian journal of information systems, vol 13, no 2. https://doi.org/10.3127/ajis.v13i2.52 poulsen, a., & burmeister, o. k. (2019). overcoming carer shortages with care robots: dynamic value trade-offs in run-time. australasian journal of information systems, 23. doi:10.3127/ajis.v23i0.1688 pye, g. and warren, m.j. (2006). striking a balance between ethics and ict governance, australasian journal of information systems, vol 13, no 2. https://doi.org/10.3127/ajis.v13i2.53 sayers, j. m., cleary, m., hunt, g. e., & burmeister, o. k. (2017). service and infrastructure needs to support recovery programmes for indigenous community mental health consumers. international journal of mental health nursing, 26, 142–150. doi:10.1111/inm.12287 senarathna, i, wilkin c, warren, m, yeoh, s & salzman s (2018). factors that influence adoption of cloud computing: an empirical study of australian smes, australian journal of information systems, vol 22. https://doi.org/10.3127/ajis.v22i0.1603 senarathna, i, warren, mj, yeoh, w & salzman, s (2016) security and privacy concerns for australian smes cloud adoption: empirical study of metropolitan vs regional smes, australasian journal of information systems, vol 20. https://doi.org/10.3127/ajis.v20i0.1193 teipel, s., babiloni, c., hoey, j., kaye, j., kirste, t., & burmeister, o. k. (2016). information and communication technology solutions for outdoor navigation in dementia. alzheimer's & dementia: the journal of the alzheimer's association, 12(6), 695-707. doi:10.1016/j.jalz.2015.11.003 wahlstrom, k., & fairweather, n. b. (2013). privacy, the theory of communicative action and technology. paper presented at the ethicomp 2013, kolding, denmark. wahlstrom, k., fairweather, n. b., & ashman, h. (2016). privacy and brain-computer interfaces: identifying potential privacy disruptions. sigcas comput. soc., 46(1), 41-53. retrieved from http://dx.doi.org/10.1145/2908216.2908223 wahlstrom, k., fairweather, n. b., & ashman, h. (2017). brain-computer interfaces and privacy: method and interim findings. paper presented at the ethicomp 2017, turin, italy. australasian journal of information systems warren, wahlstrom, wigan & burmeister 2020, vol 24, selected papers from the 2019 aice conference preface 4 wahlstrom, k., fairweather, n. b., ashman, h., & istance, h. (2013). brain-computer interfaces and privacy: clarifying the risks. proceedings of the seventh australian institute of computer ethics conference. warren, m.j. and hutchinson, w. (2003). australian hackers and ethics, australian journal of information systems, vol 10, no 2. https://doi.org/10.3127/ajis.v10i2.163 warren, m.j. and hutchinson, w. (2001). information warfare and ethics, australian journal of information systems, vol 8, no 2. https://doi.org/10.3127/ajis.v8i2.242 wigan, m. r. (2012). contestability, democracy, and trust in the anti-terror age. ieee technology and society magazine, 31(1 ), 26-32. doi:10.1109/mts.2012.2185730 wigan, m. r., & clarke, r. a. (2013). big data’s big unintended consequences. ieee computer, 46(6), 46-63. doi:http://dx.doi.org/10.1109/mc.2013.195 wigan, m. r. (2014a). smart meter technology and user issues. ieee technology and society magazine, 33(1), 49-53. doi:http://dx.doi.org/10.1109/mts.2014.2301856 wigan, m. r. (2014b). uberveillance and faith-based organizations: a renewed moral imperative. in k. michael & m. g. michael (eds.), uberveillance and the social implications of microchip implants: emerging technologies (pp. 408-416). hershey pa: igi global. wigan, m. r. (2017) ethics and brain implants in the military [commentary], ieee technology and society magazine, vol. 36, no. 1, pp. 65-68, march, doi: 10.1109/mts.2017.2654292. copyright: © 2020 warren, wahlstrom, wigan & burmeister. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v24i0.2889 http://creativecommons.org/licenses/by-nc/3.0/au/ http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems techakriengkrai, techatassanasoontorn &tan 2021, vol 25, research article post-adoptive change of enterprise system implementations 1 examining post-adoptive change of enterprise system implementations: a socio-technical perspective wallayaporn techakriengkrai new zealand parliament wally.techakriengkrai@parliament.govt.nz angsana a. techatassanasoontorn auckland university of technology, new zealand felix b. tan auckland university of technology, new zealand abstract the implementation of a new enterprise system is a major change event for end-users. since organisations need to modify their processes and structures to align with the enterprise system, users must learn and understand the new system as well as engage with it in their work practices. past research has largely focused on the initial organisational adoption of an enterprise system. however, there has been little research concerning the change process in the post-adoption stage. this research addresses this gap by drawing on the punctuated sociotechnical information system change (psic) model to explain the change through critical events, gaps between socio-technical components, responses to gaps, and outcomes. the research question is: how do socio-technical changes unfold in an enterprise system implementation? the study employs a qualitative interpretive case study method. the results reveal that changes in the structure of work following enterprise system implementation affect organisational performance, the social system, and individual work practices. this research contributes to a better understanding of technical and social changes and their impacts in the post-adoption of enterprise system implementation. the findings may assist organisations in providing appropriate resources and support for successful enterprise system implementation. keywords: enterprise system implementation, socio-technical perspective, post-adoption, qualitative interpretive case study 1 introduction enterprise systems (es) such as enterprise resource planning (erp), customer relationship management (crm), and supply chain management (scm) are generally perceived by users to be complex systems (wognum et al., 2004). from an organisation’s perspective, new enterprise system implementation helps to improve efficiency, effectiveness, and employees’ job performance in organisations (davis, 1993; dong, 2012). however, the introduction of a new enterprise system tends to require substantial changes in work processes, work practices, relationships between different professional workgroups, roles, skills, and capacities of the system users (schilhavy & iyer, 2007). in a recent survey by merkle group inc. (2018), it was reported that crm implementation has had a 63% fail rate. this failure can potentially be attributed to a possible technology misfit and organisations’ emphasis on the system itself without the necessary efforts to integrate culture, processes, people, and technology to appropriately manage organisational change (finnegan & currie, 2010; finnegan & willcocks, australasian journal of information systems techakriengkrai, techatassanasoontorn &tan 2021, vol 25, research article post-adoptive change of enterprise system implementations 2 2007; schwartz, 2002). lack of appropriate technology use is another important failure factor of enterprise system implementations (zablah et al., 2004). previous research on enterprise systems has largely focused on the initial adoption of enterprise systems and organisational performance (awasthi & sangle, 2012; chuchuen & chanvarasuth, 2011; hung et al., 2010; ko et al., 2008; williams et al., 2017). in contrast, there has been a lack of in-depth analysis that explains changes as a result of es implementation. therefore, this study focuses on socio-technical changes associated with the disruptions involved in a new es implementation in an organisation. the main research question guiding this study is: how do socio-technical changes unfold in an enterprise system implementation? this study focuses on the post-adoption of an es, specifically crm. in the information system (is) implementation model, pre-adoption refers to the stage of is implementation that leads to an adoption decision, while post-adoption refers to the stage of is implementation after an adoption decision (karahanna et al., 1999). the pre-adoption stage explains how individuals develop expectations toward the is (veiga et al., 2014). the stage in which users begin to use an is to support their work after the system has been installed and made accessible to them is regarded as post-adoption (jasperson et al., 2005). this research applies the punctuated socio-technical information system change (psic) model (lyytinen & newman, 2008) to explain the change process through critical events, gaps between socio-technical components, as well as responses and outcomes. a socio-technical model enables an in-depth analysis of dynamic and multi-level changes at the macro-level (e.g., organisational structure and policy), the micro-level (individuals’ work practices), and the social level (e.g., the relationship among individuals) by paying attention to the interactions among socio-technical components. this study adopts a qualitative interpretive case study method using the embedded multiple-case design (yin, 2003). three concurrent data analysis activities proposed by miles and huberman (1994) were conducted to theorise the change process in three organisations. in the next section, the study’s theoretical foundation is discussed with reference to the literature on es implementation and organisational change. the socio-technical perspective and the psic model are also reviewed. 2 theoretical foundation 2.1 enterprise system implementation and organisational change enterprise systems constitute “a suite of integrated software modules and a common database used to help manage a company’s human resources, financials, and the service and/or manufacturing processes” (sawyer & southwick, 2002, p. 264). since enterprise systems are embedded with prescribed business logic, organisations are often required to change their business processes and restructure their organisations to obtain the desirable benefits (elbanna, 2007; ignatiadis & nandhakumar, 2007; seethamraju & sundar, 2013). effective es implementation depends on not only technical, but also several organisational factors such as organisational politics, cultures, environments, structures, routines, and business processes (schwartz, 2002; zablah et al., 2004). in addition, social integration, including acceptance from work colleagues and support from management, is crucial to support changes during es implementation (aburub, 2015; elbanna, 2007; seethamraju & sundar, 2013). es implementation, in some cases, may unfold in unpredictable ways with unintended consequences (bloomrosen et al., 2011; ignatiadis & nandhakumar, 2007; lyytinen et al., 2009; australasian journal of information systems techakriengkrai, techatassanasoontorn &tan 2021, vol 25, research article post-adoptive change of enterprise system implementations 3 newell et al., 2002; rollins, 2012; strong & volkoff, 2010; williams & hardy, 2005). for example, lyytinen et al. (2009) found that the enterprise system institutionalisation process emerges through unanticipated events (e.g., change of project manager, increased competition to attract skilful it employees, and customer complaints) that significantly affect the outcome. the complexities of es implementation can lead to several unintended consequences such as workarounds that prevent organisations from gaining better control over their data and operations (ignatiadis & nandhakumar, 2007), misfits of es processes that lead to reduced efficiency or effectiveness (strong & volkoff, 2010), changes in communication practices, and impact on social relationships (newell et al., 2002; rollins, 2012). institutional theory, institutional logics, and socio-technical theory have been widely used to investigate information-technology (it)-enabled organisational change in general and enterprise system-enabled organisational change in particular (lyytinen et al., 2009; yoo et al., 2007; soh & sia, 2004). it is important to note that both institutional theory and institutional logics are macro-level (i.e., organisational level) theories and concepts that may not be applicable in the study of micro-level changes at an individual level (i.e., work practices) (khalil, 1995). it is important to recognise that es implementation changes and outcomes involve both the technical and social aspects (aburub, 2015; elbanna, 2007; sawyer & southwick, 2002). next, we discuss how the approach that views changes as a socio-technical phenomenon can offer insights to unravel complexities as well as unpredictable and unintended consequences from es implementations. 2.2 a socio-technical perspective is change can be characterised as technical and social change (lyytinen & newman, 2008). leavitt’s socio-technical (s-t) system model views an organisational system as a system with four socio-technical components. these components include actors (e.g., project participants and stakeholders) and their characteristics and attributes; tasks which refer to what and how work is accomplished; structure which represents institutionalised rules and arrangements; and technology including hardware, software, and tools (leavitt, 1964). the four components are interdependent; they affect each other and the change of one component leads to subsequent changes in other components, which helps us better understand the is change process. previous is research has adopted a socio-technical perspective (kim et al., 2012; luna-reyes et al., 2005; lyytinen & newman, 2008; lyytinen et al., 2009; mcleod & doolin, 2012) to develop an understanding of changes involved in the development and implementation of new is by highlighting the modification of organisational structures and work processes through the interactions of the four socio-technical components. however, most studies have used a socio-technical system model to study the change process either in individuals or organisations. to obtain richer insights on changes induced by is implementation, we believe there is a need to explain both the macro-level and the micro-level changes to individuals who constitute an important part of the change process. the multi-level changes allow us to better understand the interaction among individuals, the crm system, organisations, and the relationship among them to unveil changes in individual adaptation as well as explain changes in an organisation, such as the introduction of new policy related to system use. therefore, this study draws on lyytinen and newman’s (2008) psic model that was proposed as “a sensitising device to explain complex is changes” (p. 590). the psic model can explain is changes as a multi-level and punctuated socio-technical change (critical events, gaps, as well australasian journal of information systems techakriengkrai, techatassanasoontorn &tan 2021, vol 25, research article post-adoptive change of enterprise system implementations 4 as responses and outcomes). organisational change is a construction of a sequence of incremental adaptations and punctuations (gersick 1991; lyytinen & newman, 2008; newman & robey, 1992). building on the premise that is change is a socio-technical change, the psic model characterises socio-technical components as the content and engine of change. in other words, the socio-technical components and their interrelationships are regarded “as the general ‘lexicon’ for describing generative mechanisms and outcomes associated with the is change” (lyytinen & newman, 2008, p. 594). from a system perspective, a socio-technical system is in a stable state, where the four components are mutually aligned, and its performance is maintained. an introduction of a new information system such as an es creates system instability and introduces a gap among socio-technical components (technology, actors, task, structure). the psic model offers key concepts to explain the process of is change and observed outcomes. critical events, gaps, and responses1 are underlying concepts to explain how a change outcome emerges. a critical event is defined as an intended or unintended event that leads to the generation of gaps between socio-technical components. a gap reflects a structural misalignment in a system. it is defined as “any contingency in the system which, if left unattended, will reduce the system’s performance and threaten its viability” (lyytinen & newman, 2008, p. 595). two possible responses to a gap are incremental adaptation and punctuation. an incremental adaptation response exhibits a gradual or stepwise adaptation of one or several components of a system (lyytinen & newman, 2008). a punctuated response exhibits deep structure transformations which fundamentally change the nature of the sociotechnical system. punctuations reconfigure socio-technical elements and their interactions, and result in sudden or pronounced change(s) (gersick, 1991; lyytinen & newman, 2008). four types of outcomes from responses are (1) failed intervention: the system maintains its current misalignment; (2) successful intervention: the system experiences an incremental change to resolve the misalignment; (3) punctuation: the system experiences a radical change to resolve the misalignment; and (4) crisis: the system suffers from new misalignments or further problems. following the argument of lyytinen and newman (2008) that “events have varying impacts on the system state, we must also find out what sort of impact the particular event … had on the system(s)” (p. 599) and lyytinen et al.’s (2009) application of the psic model, we do not consider punctuation as one of the outcomes in our analysis because it has already been considered in the analysis of response as either incremental or punctuated. in this study, the psic model and its constitutive process components, namely critical events, gaps, as well as responses and outcomes, will be used to guide the analysis of socio-technical changes as a result of crm implementation. we pay attention to responses and outcomes of gaps in terms of individual work practices, individual work performance, as well as organisational performance in the post-adoption stage of crm implementation. in the pastadoption stage, is is available for use in an organisation, users are trained both in the new work processes and the is, and they are encouraged to commit to continued is use. 1 lyytinen and newman (2008) define an intervention/event as “[a] planned measure taken towards one or more socio-technical elements, or a system as a whole … to mitigate against or remove an observed gap” (p. 613). to avoid potential confusion with critical events, we refer to these types of events as responses in this study. australasian journal of information systems techakriengkrai, techatassanasoontorn &tan 2021, vol 25, research article post-adoptive change of enterprise system implementations 5 3 research methodology this research adopted the qualitative interpretive case study method with an embedded multiple-case design to examine organisational change in es implementation. a multiple-case design is more appropriate than a single case approach because we are not interested in testing the boundaries of well-formed theory or examining an extreme or a unique case. the benefits of adopting a multiple-case design in this study are well aligned with the research aim to identify organisational changes in different types of organisations. the theoretical sampling strategy was adopted to select case studies (patton, 1990). the enterprise systems of interest to this study are the customer relationship management (crm) systems. research into crm has largely focused on individual factors that shape initial adoption (vella et al., 2012; wu & wu, 2005; zablah et al., 2004) as well as the implementation process and outcomes at the organisational level (awasthi & sangle, 2012; chuchuen & chanvarasuth, 2011; dong, 2012; hung et al., 2010; ko et al., 2008; sebjan et al., 2014). few studies have examined post-adoptive changes at both an individual level and an organisational level. for instance, crm research has paid little attention to individual work routines that may impact organisational change (jones et al., 2002; winter & nelson, 1982). table 1 presents the background information of the three organisations. cases rc hp bi industry types innovative office automation solutions hospital insurance no. of employees approximately 2,000 approximately 1,300 approximately 1,400 no. of participants 6 users 4 it support 4 management 6 users 1 it support 4 management 8 users 3 it support 7 management crm software package used salesforce crm seibel crm microsoft dynamics crm crm module used sales automation marketing automation, and customer service sales automation, marketing automation, and customer service departments implementing crm system sales and marketing department local marketing, overseas marketing, and contact centre motor-claims department, non-motor claims department, and five business units (agent business unit, broker business unit, financial business unit, commercial lines business unit, and personal lines business unit) table 1. background information of the organisations there were three criteria for choosing participating organisations. firstly, organisations needed to have implemented crm systems within the past three years and were currently using the systems. secondly, organisations needed to be medium or large-sized businesses since organisations of these sizes are typical crm users (srivihok & batanov, 2005). the third criterion was that the selected organisations needed to be using the main crm modules, such as customer service, sales automation, and marketing automation modules. three australasian journal of information systems techakriengkrai, techatassanasoontorn &tan 2021, vol 25, research article post-adoptive change of enterprise system implementations 6 organisations were selected as research sites: a hospital, an insurance company, and an innovative office automation solutions provider in thailand. having organisations from different industries allowed us to extend the findings to develop a rich understanding of postadoptive changes (harris & sutton, 1986). 3.1 data collection semi-structured interviews were the main data collection technique employed in this study. participants were selected according to the likelihood that they could offer theoretical insight into changes associated with crm implementation. all of the interview sessions were one-onone to obtain insightful information from participants’ real experiences and perceptions. each interview contained questions about participants’ work experience and educational background, their experiences related to information technology, their perception toward crm systems, their crm use, and ensuing changes. there were three different sets of interview questions for three groups of participants – end-users, it support staff, and management. the main reason to interview different groups of participants was to obtain rich information from the perspectives of the organisation, users, and it staff. according to alvesson (1995), data from both managerial and technical experts can help eliminate the possible managerial bias that can occur when only managers are included in research studies. we also used supporting documents such as it system infrastructure documents, implementation plans, and user manuals to enrich our understanding and, in some cases, to verify participants’ quotes. participants were selected based on their experience of using the crm system in their daily work routines and their ability to give us rich information relating to changes that occurred after crm implementation. the total number of participants in this study was 43 (14 for rc, 11 for hp, and 18 for bi). 3.2 data analysis we adopted three concurrent data analysis processes (data reduction, data display, and conclusion drawing and verification) proposed by miles and huberman (1994) to analyse interview data. within-case and cross-case analyses were conducted to investigate sociotechnical change in a specific organisation and to compare the organisation with other organisations to identify similarities and differences. in this study, the data reduction commenced with our choice of a socio-technical perspective to examine the change process. during the interview process, data reduction was achieved by making short notes. during the data analysis, the coding process began by labelling data with low-level codes. next, we grouped conceptually related low-level codes into interpretive codes – interpretive codes are used to identify categories that coherently conceptualise the change process. in the last stage, the pattern codes were developed. the next step involved linking critical events to gaps as well as responses and outcomes to theorise socio-technical changes. the critical events, gaps, as well as responses and outcomes were derived from either low-level codes, interpretive codes, or pattern codes. to explain how socio-technical changes unravel, we traced the process from a critical event to gaps, as well as responses and outcomes. 4 findings in this section, socio-technical changes from crm implementation are discussed for rc, hp, and bi. australasian journal of information systems techakriengkrai, techatassanasoontorn &tan 2021, vol 25, research article post-adoptive change of enterprise system implementations 7 4.1 rc the introduction of the crm system can transform and readjust all components of the existing socio-technical system. the previous structure of work at rc focused on a decentralised structure by using basic office applications to manage customer data without the requirement to share them among sales professionals. the new crm system ultimately changed the organisational structure, work processes, work systems, and individual work practices. the changes in the socio-technical components included actors (sales professionals and newly appointed crm officers), tasks (input customer data and systematic task management), structure (centralised customer repository, integrated work processes, and new organisational rules), and technology (crm and ris system), which are explained below. the ris system was used as an in-house system to maintain customer information and create invoices. the critical events, gaps between socio-technical components, as well as responses and outcomes are discussed to explain how the observed changes were unveiled (see table 2). actor: rc established a new crm team who were newly hired for the sole purpose of supporting crm implementation. after the crm implementation, sales professionals and crm officers were required to work closely together. all customer accounts created by sales professionals were required to be approved and verified by the crm team. the sales supervisor explained that, “sales professionals must send a request to the crm team to verify new accounts. it takes time to wait for approval from the crm team. previously, we were able to input customers’ data without the need for verification.” (rc14) task: the introduction of crm added a new task for sales professionals to input customer data into the crm system. in particular, they were required to switch from using systems of their choice (e.g., excel and google calendar) to using two organisational sanctioned systems (ris and crm systems) to complete their work tasks. as a result, their work processes now included using crm as a centralised data repository to input customer data, produce sales forecasts, and create sales activities. indeed, the task of inputting customer data into the crm system had a significant impact on sales professionals’ work practices. one participant stated that, “there is an additional work task. users must input customers’ data in the crm system before generating orders in the ris system.” (rc07) in addition, the interviews suggested that data were not well prepared because of the lack of appropriate data cleansing during migration (critical event). as a result, some data were not migrated into the new crm system and some data were duplicated (gap). the critical event of a poor data migration strategy (including duplicated data and customer data loss which occurred during data migration) led to the generation of a gap in the technology-task components. to address this gap, the management introduced an additional task of manually inputting customer data into the system as well as requiring the sales professionals to work with the crm team to remove duplicated accounts (incremental adaptation). although the data problem was resolved, the sales professionals were left with the feeling that their professional identity had been challenged because they were asked to perform what they considered non-sales related tasks. a sales professional explained that, “during the period of data migration from the previous system to the crm system, some data were lost. i need to put my own customers' accounts manually into the crm system. i have hundreds of accounts to enter into the crm system. on top of that, all accounts need to be approved by the crm team. my task is to sell products not to perform administration tasks.” (rc05) therefore, this intervention can be simultaneously considered as a successful intervention to alleviate the gap between technology and task australasian journal of information systems techakriengkrai, techatassanasoontorn &tan 2021, vol 25, research article post-adoptive change of enterprise system implementations 8 components and a failed intervention because it introduces a new misalignment between actor and task. structure: following the introduction of crm, information about which sales professionals worked with which customers became transparent and visible among all sales professionals. the increased transparency of sales and customer data contributed to the increasing conflict among the sales staff. some sales professionals believed that others were stealing their customer accounts without informing them. to address this issue, the management set up new rules to resolve conflicts that occurred during the change process. the new rules were related to sales authority and the owners of accounts with several criteria to eliminate the conflicts among the sales staff. for example, the organisation set up a strict sales boundary. a boundary can be a geographical area or business types of customer organisations in which each sales team is able to sell products. in this case, the sales professionals were not allowed to sell products outside of their assigned boundaries. the crm team was granted the authority to resolve conflicts related to sales boundaries. in particular, the crm officers were empowered to decide whether a sales professional met the criteria to be the owner of a specific customer account. technology: the crm system was integrated with the existing systems, including the ris system. sales professionals could use the crm system to view customer information anytime, anywhere, without the need to carry printed copies of customer profiles as they did before the crm implementation. both sales professionals and the crm team used the crm system to generate reports. table 2. summary of the critical events, gaps, responses and outcomes in the rc organisation critical events gaps between components response to gaps outcomes poor data migration strategy identifying problems with data quality including duplicated data and customer data loss occurred during the data migration stage. technology-task data was not well prepared because of the lack of appropriate data cleansing and migration. as a result, some data was not migrated into the new crm system and some data was duplicated. incremental adaptation the management introduced an additional task of manually inputting customer data into the system, as well as requiring the sales professionals to work with the crm team to remove duplicated accounts. due to this intervention, the data problem or the gap between technology and task components was resolved. however, this response introduced a new gap between actor and task. increased transparency after its implementation, the crm system became a “transparency tool” that revealed sales activities among sales professionals. technology-actor the critical event of increased transparency negatively affected the relationships among sales professionals. some sales professionals thought that others were stealing their customer accounts without informing them. punctuation increased transparency led to conflict among users. to address the conflicts, management set up new rules to prevent cross-selling across districts. the intervention not only failed but also added a new misalignment between structure and actor manifesting in the distrust among sales professionals. australasian journal of information systems techakriengkrai, techatassanasoontorn &tan 2021, vol 25, research article post-adoptive change of enterprise system implementations 9 after its implementation, the crm system became a “transparency tool” that stopped ambiguity in sales activities among sales professionals. information was shared within the sales department and among sales teams with a different level of authorisation to view the data. in particular, the critical event of increasing transparency of customer data led to the generation of a gap in technology-actor components, which negatively affected the relationships among sales professionals (gap). one participant explained that, “there was a conflict among the sales staff. users were able to search for customers’ accounts in the crm system. it showed details of account owners. previously, this kind of data was not shared with others.” (rc07) as a response, management set up new rules to prevent cross-selling across districts (punctuation), which led to distrust and unpleasant relationships among sales professionals. a sales professional explained that, “in the beginning, the crm team got confused regarding how to manage conflicts in the sales department. management then set up new rules to solve those conflicts. nowadays, the crm team applies these rules with several criteria that consider account owners, business type and sales’ district boundaries to all sales professionals.” (rc12) in this case, the intervention not only failed but also added a new misalignment between structure and actor, manifesting in distrust among sales professionals. further examination of responses and outcomes from gaps showed that two critical events in the rc organisation led to the generation of different gaps in the socio-technical components. these gaps occurred between technology-task, and technology-actor components. as a result, these gaps and responses led to the generation of unintended consequences including additional tasks, loss of benefits, and conflict among users. some issues were resolved, while others remained at the time of data collection. 4.2 hp hp initially implemented crm as a mandatory system in three departments: local marketing, overseas marketing, and the contact centre departments. marketing officers used the crm system to analyse customer data, create marketing campaigns, and segment customer groups. the main objective was to increase customer loyalty and the customer retention rate. the email correspondent team under the overseas marketing department used the crm system to reply to e-mails from foreign customers using a pre-defined template. the system generated a serial number (sr number) to track and store the email correspondence. the crm system completely replaced the previous email systems (lotus notes and microsoft outlook). the contact centre department used the crm system to record and view customers’ data, such as names and telephone numbers. the crm system was integrated with the cisco ip phone to automatically recognise customers’ names. for hp, the new crm system ultimately changed the organisational structure, work processes, work systems, and individual practices. the changes in the socio-technical components included actors (staff and management committee team), tasks (customer-centric focused, systematically reply to customers’ e-mails), structure (data integration, additional work process, and systematic workflow), and technology (crm and hospital systems), which are explained below. the critical events, gaps between socio-technical components, as well as outcomes are discussed to explain how the changes were unveiled (see table 3). it is important to note that hp did not introduce appropriate interventions to address the emerging gaps from critical events, which meant the misalignments between system components persisted. actor: the organisation set up a new management committee to brainstorm and generate the implementation plan. this team also provided support for all activities related to the crm australasian journal of information systems techakriengkrai, techatassanasoontorn &tan 2021, vol 25, research article post-adoptive change of enterprise system implementations 10 system and maintained the crm system. a local marketing staff member explained that, “the organisation set up a management committee to support the crm system. this committee was comprised of top management, middle management, and team leaders who had expertise with the crm system. this team attempted to support crm users if they made a request, such as implementing more features or modules, customising the system to suit users’ work practices, and so on.” (hp06) task: in the local marketing and contact centre departments, a customer-centric focus was adopted by focusing on providing personalised services to customers in order to increase customer retention and acquisition. the local marketing staff focused on increasing revenues, developing new products, creating campaigns, marketing activities, and maintaining relationships with customers to enhance customer satisfaction. however, the crm system affected individuals’ work performance due to an increase in abandoned calls in the contact centre department. the new requirement for the contact centre staff to input customers’ information into the crm system could be considered as a critical event which led to the development of a gap between task-structure components. this is because there was inadequate staff in the contact centre to manage the workload (gap). previously, contact centre staff used only the trakcare system. after crm was introduced, they were required to use both systems. they needed to input data into the trakcare system as well as the crm system. it took a minute to one and a half minutes to complete data entries in both systems. to cope with the increased data entry demand, some users did not complete all data entries. during the interview period, there was no further information on whether this problem had been resolved or not. the head of the contact centre department explained that, “the crm system affects kpis, increases talk time, and increases abandoned calls. therefore, we avoid some tasks such as inputting customer data into the system.” (hp10) structure: data integration was introduced in the local marketing department. the crm system became an additional work system used to analyse customer information and create systematic campaigns. work processes were changed to incorporate crm as a centralised database. for example, contact centre staff needed to first access the crm system prior to other systems. work routines were changed by adding tasks that users needed to perform on the crm system. for example, front office staff needed to check customer preferences in the crm system to serve customers’ needs. the overseas marketing department used the crm system to respond to customers’ inquiries by using a pre-defined response template and generating a serial number (sr number) to manage email correspondence. previously, they used microsoft outlook to reply to e-mails and manually tracked records of incoming e-mails. in this case, the new communication tool that the overseas marketing department were required to use is a critical event. the team had previously used the outlook application, which was now superseded by the crm system, leading to the development of a gap between structure-task components. users had to spend additional time to create sr numbers and input data (gap). as a result, this event had a negative effect on kpis in terms of decreasing numbers of response e-mails to customers. in other words, the new crm system completely changed the email correspondence team's work practices. one of the overseas marketing staff explained that, “prior to the crm system, we sent approximate 40 response e-mails to customers a day. after crm replaced microsoft outlook, it increased our workload because additional work steps were needed in the crm system such as creating e-mail templates and generating serial numbers. the number of e-mail replies was now halved.” (hp04). australasian journal of information systems techakriengkrai, techatassanasoontorn &tan 2021, vol 25, research article post-adoptive change of enterprise system implementations 11 the contact centre department used the crm system as the primary system to identify and recognise customers when they call the hospital. the introduction of the crm system created additional tasks in terms of inputting and updating customer information. the head of the contact centre department described that, “previously, i used only the trakcare system. after crm was introduced, i must use both systems. i must input data into the trakcare system and then, i must also input data into the crm system as well. it affects my work performance. it takes a minute to one and a half minutes to complete all these.” (hp10) technology: in the local marketing department, the crm business intelligence tool was used to gather, store, analyse, and access customer data. users were able to access customers’ database without the need to ask for data from the it department. in the overseas marketing department, the crm system completely replaced the previous systems used to support customer enquiry (lotus notes and microsoft outlook). in the contact centre department, the staff admitted that it was more convenient to use the crm system to find information rather than using several systems as before. critical events gaps between components response to gaps outcomes requirement to input customer information contact centre officers were required to input customer data into the crm system. task-structure the existing number of contact centre staff was inadequate to support the task at the initial stage of post-adoption. n/a some users minimally used some features to finish work on time. since the organisation did not attempt to mitigate the problem, the gap between task and structure persisted. replacement of new communication tool the e-mail correspondence team had previously used the outlook application, which was now superseded by the crm system. structure-task the existing workflow was affected, and the work tasks were added. after crm implementation, users had to spend additional time to create sr numbers and input data. n/a this event led to a negative effect on kpis in terms of decreasing the numbers of response e-mails to customers. since the organisation did not attempt to mitigate the problem, the gap between structure and task persisted. slow connection to crm server the location of the crm system server slowed down access to the database. furthermore, the crm server and the hospital information system (his) server were not located in the same place. technology-task it led to a slow crm server response. in addition, it took longer to synchronise data between the his system and the crm system. the data was updated daily, not in real-time. n/a this event affected work performance in terms of increased time needed to finish tasks. since the organisation did not attempt to mitigate the problem, the gap between technology and task persisted. table 3. summary of the critical events, gaps, responses and outcomes in the hp organisation however, users experienced some problems in their use of the crm system. the crm system server was physically located far away from the users. the physical distance of the crm system server in particular could be considered as a critical event, which led to the generation of a gap between technology-task components and the slow crm server response (gap). this australasian journal of information systems techakriengkrai, techatassanasoontorn &tan 2021, vol 25, research article post-adoptive change of enterprise system implementations 12 event led to negative impacts on work performance in terms of increased time needed to finish tasks. the executive office manager explained that, “the crm system server is located in bangkok. we need to access the server via a wan link. access via the crm system involves delay for a while. in addition, the hospital’s in-house system is located here. therefore, the data from the his system and the crm system are not real-time data. the data is updated daily. it affects our work performance because of speed. we must respond to customers’ e-mails as soon as possible. using the crm system instead of microsoft outlook, sometimes leaves us hanging around. it wastes our time.” (hp04) further examination of the outcomes of gaps showed that three critical events in the hp organisation led to the generation of different gaps between task-structure and technologytask components. as a result, these gaps led to the generation of unintended consequences including increased abandoned calls, a negative effect on kpis, and a negative effect on work performance. 4.3 bi similar to rc and hp, bi introduced the crm system as a mandatory system. bi implemented the crm system in two departments – the motor claims department and the non-motor claims department – as well as five business units (agent business unit, broker business unit, financial business unit, commercial lines business unit, and personal lines business unit). the structure of the bi organisation was complicated in that it had both departments and business units. therefore, the analysis of organisational change was conducted on a similar module used in different departments or business units. there were primarily two different work practices – that of the contact centre and that of the account executive (ae) and underwriter staff. the bi organisation had contact centres in three business units: the motor claims contact centre, the non-motor claims contact centre, and the personal line business unit. all three contact centre units used the phone call and case management module. the ae and underwriter staff used the phone call and case management module, the visit plan module, and the renewal module. for bi, the new crm system ultimately changed the organisational structure, work processes, work systems, and individual practices. the changes in the sociotechnical components included actors (staff and it support team), tasks (systematic tasks, record customer information), structure (customer-centric data-focused, centralise customers’ information, additional work process), and technology (crm and in-house systems), which are explained below. the critical events, gaps between socio-technical components, as well as responses and outcomes are discussed to explain how the changes were unveiled (see table 4). actor: users were able to download reports from the crm system without the need to request them from the it department as before. the work system in the contact centre department was, therefore, more efficient and more systematic. similarly, the ae and underwriter staff were able to obtain renewal data from the crm system rather than asking for information from the it department. a user explained that, “the crm system assists in displaying all insurance policies that are going to expire.” (bi03) however, there were issues in terms of crm usage. inadequate training and the complexity of the crm system could be considered as a critical event which led to the development of a gap between actor-task components. users did not know how to use some features, which had a negative effect on work performance because they struggled to use crm features to complete work tasks. a user explained that, “the work process in the crm system was complicated. australasian journal of information systems techakriengkrai, techatassanasoontorn &tan 2021, vol 25, research article post-adoptive change of enterprise system implementations 13 additionally, training was inadequate. furthermore, it support staff were inadequate as well. at the initial stage of crm usage, users faced issues such as not being fluent in using the system. as a result, the work processes were stopped and it was not possible to move to the next processes.” (bi03) the organisation initiated training sessions and one-on-one support from the it department, which can be considered as an incremental adaptation response to successfully alleviate this gap. task: after the crm system was introduced, customer data was recorded into the crm system in the contact centre department. contact centre officers used case management in the crm system to assign tasks to responsible persons. every transaction was recorded into the system. the ae and underwriter staff used the crm system to replace the paper-based visit plans, to systematically assign case management, and to replace the manual work of renewing insurance policies. another participant explained the process involved in completing her work tasks: “my task involves sending a renewal notice to the individual customer, not passing through a broker. i use the renewal report in the crm system as a control sheet to remind me. normally, the list of insurance policies that are going to expire shortly is transferred to the crm system. i use this list to send an e-mail to customers. i can also check the status in the system.” (bi03) however, a management staff member mentioned that the crm system had affected users’ work performance regarding the amount of time needed to record customer information. in particular, the new requirement to input customer information into the system could be considered as a critical event, which generated a gap between task-structure components. a number of staff were unable to complete their tasks at the initial stage of post-adoption, which led to an increase in abandoned calls and a negative impact on work performance. the it department attempted to streamline the user interaction with crm and substantially reduce the working time (incremental adaptation). due to this intervention, both parties were happy with the solution and the gap was successfully addressed. the team leader of the motor claims department explained that, “at the initial post-adoption stage of the crm system, it had a huge effect on our work performance. it increased call time by one minute per call. if i received 60 calls, i would waste an hour per day. i raised the issue with the it department that it was wasting our human resources and we needed to finish calls earlier. the it staff came up with the solution of finishing the task within 30 seconds by removing unnecessary steps.” (bi18) another critical event was that the different implementation periods of the phone call and case management modules across business units generated a gap between task-actor components. some users were unaware that they had to use the phone call and case management modules in the crm system, which led to conflict among users. once users were made aware of the need to use the new crm system (incremental adaptation), then the problem was resolved. a user explained that, “conflict occurred among users across business units. it was a communication conflict. for example, the conflict occurred because of using the phone call and case management module. some users were unaware that they had to use case management to send/receive cases from other business units. once the crm system had been completely implemented to all business units, then there were no conflicts.” (bi10) structure: in the contact centre department, users could use the crm system to view information related to a customer’s account, such as the list of all insurance policies belonging to that customer. the organisation changed from a product-centric data focus to a customercentric data focus to align with their business strategy. additionally, the information in the crm system was integrated with the bki application. before crm was introduced, bi had australasian journal of information systems techakriengkrai, techatassanasoontorn &tan 2021, vol 25, research article post-adoptive change of enterprise system implementations 14 two existing systems in use called bki app and e-surveyor app. bki app was mainly used for generating insurance policies and maintaining customer information. users were able to click the icon in the crm system to view further information about each insurance policy in the bki application. therefore, customer information became centralised, and the workflow was substantially changed after crm implementation. however, there was a critical event regarding the incomplete integration of crm and the existing systems during the initial stage of implementation. this generated a gap between structure-task components. the provided structure of work did not support users to complete their tasks, which led to time-wasting in terms of staff double keying information to complete their work tasks. a management staff member explained that “at the initial stage of crm adoption, information in the crm system did not integrate with information in the e-surveyor application. when i keyed information into the crm system, the information was not displayed in the e-surveyor application. we informed it support staff that we would not do double work.” (bi15) to address this gap, the it team was instructed to fully integrate the two systems in the later stage of implementation (incremental adaptation), which successfully removed the misalignment between structure and task. technology: in the contact centre, the crm system helped to identify customers through its integration with the phone system and other in-house systems. for the ae and underwriter staff, tools to support their work tasks were changed after the introduction of the crm system. the crm system was integrated with the existing in-house systems (mainly the bki application). as a result, ae and underwriter staff were required to use the crm system to systematically record customer data and inquiries. they used the crm system to replace paper-based visit plans and the renewal of insurance policies. a management staff member confirmed that, “we have additional work. we shifted from manual work to work on the system. we need to record everything into the crm system.” (bi12) however, the crm system affected individuals’ work practices because of the limitations of customisation and limited user experience with off-the-shelf software. in particular, the critical event of the workflow adjustment limitations generated a gap between technology-structure components. this affected the structure of work because the crm system did not fit well with the existing workflow. this event led to a negative impact on work performance because users had to spend more time to learn and adapt to the new system. a user from the agent business unit explained that, “the crm system has several features. i feel that it is suitable for a marketing company or suitable for a business unit which takes care of retail customers. in my business unit, we take care of sales agents. it is unsuitable for the nature of our work.” (bi02) the it team attempted to customise the crm system to suit the existing workflow (incremental adaptation). due to this intervention, the problem was successfully resolved. a user explained that, “the crm system is difficult to use and unsuitable for users. it staff attempted to customise it to suit each business unit. from my point of view as a user, each business unit has different work processes.” (bi02) also, this critical event generated a gap between technology-actor components. most users were not familiar with the off-the-shelf software (gap), which had a negative effect on users’ work routines. one of the participants explained that, “the wording in the crm system and the wording that we use every day are totally opposite – for example, the words ‘lead’ and ‘customer account’. we are not familiar with sales and marketing vocabulary. on top of that, the wording in the crm system is in english.” (bi08) the it team provided extra support to users to help them australasian journal of information systems techakriengkrai, techatassanasoontorn &tan 2021, vol 25, research article post-adoptive change of enterprise system implementations 15 adapt to the new system, including one-on-one and on the job training (incremental adaptation). due to this intervention, the problem was successfully resolved. further examination of outcomes from gaps showed that five critical events in the bi organisation led to the generation of different gaps between technology-structure, technologyactor, structure-task, and task-actor components. as a result, these gaps led to the generation of unintended consequences including effect on work performance, additional work, conflict among users, struggles with workflow, and increased abandoned calls. critical events gaps between components response to gaps outcomes customisation limitations and mismatch of the terminology in the crm system customisation of the crm system had limitations in terms of workflow adjustment and too many unnecessary features. furthermore, users were unfamiliar with the terminology in the crm system. technology-structure the crm system did not fit well with the existing workflow. technology-actor the terminology in the crm system affected users’ work routines and work practices. most users were not familiar with the off-the-shelf software. incremental adaptation the it team attempted to customise the crm system to suit the existing workflow, as well as supported users to adapt to new system. due to this intervention, the effect on work performance in terms of spending more time to learn and adapt to the new system was successfully resolved. incomplete integration there was incomplete integration of the two systems at the initial stage of implementation. users had to input data twice into both systems. structure-task the provided structure of work did not support users to complete their tasks. incremental adaptation this event led to additional work in terms of double keying information to complete the work processes. to address this gap, the it team resolved the problem by fully integrating the two systems. due to this intervention, the it team was instructed to fully integrate the two systems in the later stage of implementation, which successfully removed the misalignment between structure and task. different implementation periods different implementation periods of the phone call and case management modules in different business units. task-actor some users were unaware that they had to use the phone call and case management modules in the crm system. each department started using the crm system at a different time. incremental adaptation this event happened during the initial stage of postadoption. it led to conflict among users. once users were made aware of the need to use the new crm system, the problem was resolved. due to this intervention, once users were made aware of the need to use the new crm system, then the conflicts among users were resolved. inadequate training and the complexity of the crm system users were not fluent at using the crm system because of inadequate actor-task users did not know how to use some features. incremental adaptation this event happened during the initial stage of postadoption. it led to struggles with workflow because staff were unable due to intervention, the problem was successfully resolved. australasian journal of information systems techakriengkrai, techatassanasoontorn &tan 2021, vol 25, research article post-adoptive change of enterprise system implementations 16 critical events gaps between components response to gaps outcomes training and the complexity of the crm system. to move to the next processes to accomplish work tasks on time. to address this misalignment, the it support offered additional training sessions and one-on-one support. requirement to input customer information users were required to input customer data into the crm system. task-structure a number of staff were unable to complete their tasks at the initial stage of post-adoption because of the workload, human resource limitations, time limitations, and system interruption. incremental adaptation this event led to increased abandoned calls and had a negative effect on work performance. the it department came up with a solution to reduce work processes in the crm system. due to this intervention, users and the it team were happy with the solution. table 4. summary of the critical events, gaps, responses and outcomes in the bi organisation 4.4 cross-case analysis this section discusses the critical events and gaps that reveal misalignment among the four socio-technical components as well as responses, if any, and outcomes (see figure 1). taken together, the generation of gaps had significant impacts on individuals in terms of (i) the relationship among users and (ii) work practices, as well as organisations in terms of (iii) organisational performance. 4.4.1 relationship among users the generation of gaps between technology-actor and task-actor components in the rc and bi cases led to unintended behavioural change in actors due to benefit losses and conflicts among users. in particular, the rollout and nature of crm use affected the socio-technical systems in different ways. in the bi case, each department started using the crm system at different times. the different implementation periods led to a misunderstanding among users. the conflict occurred because some users were unaware that they had to use the case management feature in the crm system to send/receive cases to/from other business units. once users were made aware of the need to use the new crm system (incremental adaptation), then the problem was resolved. in the rc case, after the implementation, the crm system became a transparency tool because sales professionals’ sales activities were no longer private information to each sales staff. as a result, the increased transparency of sales and customer information negatively affected the relationships among sales professionals. this led the system into a new deep structure whereby sales professionals could only sell products within their sales boundary due to the visibility of sales data in the crm system. 4.4.2 work practices users from all three organisations were required to change their work practices to adapt to new crm system implementations. however, the gaps between the socio-technical components produced several negative impacts on work practices and work performance australasian journal of information systems techakriengkrai, techatassanasoontorn &tan 2021, vol 25, research article post-adoptive change of enterprise system implementations 17 albeit temporarily in some cases. in the bi case, the gap of actor-task components due to inadequate training and the complexity of the crm system led to delay in accomplishing work tasks. the organisation initiated training sessions and one-on-one support from the it department, which can be considered as an incremental adaptation response to successfully alleviate this gap. in the rc case, the gap between technology-task components from a poor data migration strategy resulted in the addition of an unnecessary task for users to manually input customer data into the system to remove duplicated accounts (incremental adaptation). although the data problem was resolved, the sales professionals were left with the feeling that their professional identity had been challenged because they were asked to perform what they considered non-sales related tasks. in the hp case, the gap between technology-task components as a result of a slow connection to the crm server led to more time required to finish tasks. since the organisation did not attempt to mitigate the problem, the gap between technology and task persisted. in addition, in the bi case, the gap between technology-structure and technology-actor components, based mainly on the customisation of the crm system and the different terminologies used in the crm system, led to individuals temporarily spending more time learning and adapting to the new system. the it team attempted to customise the crm system to suit the existing workflow (incremental adaptation). due to this intervention, the problem was successfully resolved. lastly, the gap between task-structure components in the hp and bi cases led to increased abandoned calls in both cases. the team in the hp case had previously used the outlook application, which was now superseded by the crm system, leading to the development of a gap between structure and task components and a negative effect on work performance in terms of the number of response e-mails to customers (kpis). since the organisation did not attempt to mitigate the problem, the gap between structure and task persisted. in the bi case, the task-structure gap was the result of the incomplete integration of crm systems with the existing system at the initial stage of implementation, leading to time wasting because staff had to enter the same information into both systems to complete their work tasks. to address this gap, the it team was instructed to fully integrate the two systems in the later stage of implementation (incremental adaptation), which successfully removed the misalignment between structure and task. 4.4.3 organisational performance in terms of consequences on organisational performance, all three organisations changed the structure of work, reshaped their work processes and work systems in order to integrate the new es with their existing systems, and set up appropriate organisational policies to support the use of the crm system. however, these changes had negative effects on organisational performance. in the bi case, the gap between actor-task components due to inadequate training and the complexity of the crm system temporarily affected organisational performance negatively at the initial stage of crm implementation because users were struggling with some processes and unable to move to the next processes to accomplish work tasks. similarly, the gap between technology-task and technology-actor components in the three cases led to increased time needed to finish tasks. as a result, the overall organisational performance was affected, in some cases temporarily and others permanently. in the rc case, the data migration issue led to the creation of a gap between sales professionals’ tasks and australasian journal of information systems techakriengkrai, techatassanasoontorn &tan 2021, vol 25, research article post-adoptive change of enterprise system implementations 18 crm technology. sales professionals had to manually input data into the system, which is considered as an incremental adaptation. figure 12: the critical events, gaps, responses, and outcomes from the three organisations 2 note that st, ta, te, ac are the abbreviations for structure, task, technology and actor, respectively. the notation of (i) stands for incremental adaptation and (p) stands for punctuation. australasian journal of information systems techakriengkrai, techatassanasoontorn &tan 2021, vol 25, research article post-adoptive change of enterprise system implementations 19 this gap also impacted on organisational performance in terms of meeting the sales targets. in the hp case, the synchronisation of data between the crm system and the his system did not occur in real time, which subsequently delayed the work process and negatively impacted organisational performance. since the organisation did not attempt to mitigate the problem, the problem remained. in the bi case, the critical event of customisation limitations, including workflow adjustment limitations and users’ unfamiliarity with the terminology in the crm system, had a negative but temporary impact on the structure of work and users’ work routines because the crm system did not fit well with the existing workflow. lastly, the gap between task-structure components in the hp and bi cases led to increased abandoned calls and negatively affected users’ kpis. since users had to spend time to input customer information, abandoned calls from customers increased. this permanently and negatively impacted organisational performance in the hp case as the organisation did not attempt to mitigate the problem. a similar problem temporarily impacted organisational performance in the bi case. both the users and the it team were happy with the solution. in conclusion, the findings across the three cases show that crm implementations affected organisational performance, as well as individual work practices and relationships among users. 5 discussion and conclusion 5.1 discussion this research applied a psic model as a framework to explain the change process through critical events, gaps between socio-technical components (actor, task, structure, and technology), as well as responses and outcomes in the post-adoption of es implementation. the results revealed that the new es became a centralised database, changed the way customer data was viewed, changed the pattern of communication among users, became a control mechanism between management and users, and changed their structure of work. this led to the generation of gaps in socio-technical components and produced consequences, some of which were unintended. the findings deepen our understanding of the change process by tracing critical events that generate gaps in the socio-technical components, responses and outcomes manifested in changes in organisational performance, the social system, and individual practices. lyytinen and newman (2008) argue that outcomes ranging from a failed intervention to a crisis that adds new misalignments into a system are likely to be generated from an event or a response. our results complement those of lyytinen and newman (2008), in that there were two types of responses – incremental adaptation and punctuation – in the three organisations. these responses to address gaps led to the generation of outcomes. in terms of the change outcomes, the findings revealed that crm system implementation can lead to conflict among users with a negative effect on work performance. the analysis across two cases (rc and bi) revealed that the misalignment between technology-actor and task-actor components led to unintended conflict among users. in addition, the findings across the three cases revealed that the introduction of the crm system led to a mismatch between the crm system and work practices in some departments. the misalignment between task-actor, tasktechnology, task-structure, technology-structure, and technology-actor components had a negative effect on work performance. when crm was introduced into these organisations, it was perceived as an additional work system for employees in some departments across the australasian journal of information systems techakriengkrai, techatassanasoontorn &tan 2021, vol 25, research article post-adoptive change of enterprise system implementations 20 three cases. for example, users were required to log in to the crm system to input customer information or record customers’ requests or inquiries. users were required to use the crm system first before working with other systems to complete their tasks. the findings also explained that all three organisations changed their structure of work and set up appropriate organisational policies to support the use of the crm system after its implementation. these changes led to negative effects on organisational performance, which is far from the ideal objectives that these organisations aimed for. our results complement those by bloomrosen et al. (2011), mackay and chia (2013), and strong and volkoff (2010), who argue that workflow adjustment as a result of crm system implementations can impact work performance and effectiveness. our results also confirm earlier studies regarding conflict among users. for example, in studies by newell et al. (2002) and rollins (2012), unfavourable changes in communication patterns and practices and negative impacts on social relationships were found to be unintended consequences associated with complexities of software use. in terms of effect on work performance, our results complement studies by ignatiadis and nandhakumar (2007) and williams and hardy (2005), which argue that unintended consequences in work performance are a result of changes in work practices associated with new system implementations. this study extends the extant literature on enterprise systems and organisational change by explaining the sequence of change associated with es implementations. prior studies have paid little attention to all the impacts of changes on organisational structures and policies, work processes, individual practices, and relationships among employees. some is studies have independently shed light on the area of individual practices, work practices, adaptation behaviours, and job outcomes relating to is implementation (bala & venkatesh, 2015; kim et al., 2012; lyytinen & newman, 2008; lyytinen et al., 2009; mcleod & doolin, 2012). for instance, bala and venkatesh’s (2015) study focused on individual adaptation to it and job outcomes. some is studies have applied the notion of institutional logics to study organisational change associated with new information systems (lyytinen et al., 2009; yoo et al., 2007; soh & sia, 2004). for instance, soh and sia (2004) suggest that organisations can address misalignment by package customisation or organisation adaptation, which include the redesign of organisational processes and structures. similarly, this study found that system or workflow customisation to suit users’ work practices can reduce misalignment between users (actor) and new es implementation (technology). despite the above, prior studies have not offered indepth explanations of how es implementation affects changes at the organisational and individual levels. this study addresses this gap by providing a theoretical understanding of how es implementation affects organisational structure and individual work practices in the post-adoption of crm implementation. the findings can help organisations to be aware of unintended consequences associated with enterprise implementation that may occur after crm implementation. although several studies have investigated crm adoption and implementation at an organisational level (awasthi & sangle, 2012; chuchuen & chanvarasuth, 2011; hung et al., 2010; ko et al., 2008), they have focused on post-adoption changes in terms of crm use and organisational performance (sebjan et al., 2014). klein and kozlowski (2000) suggest that there is a lack of multi-level studies that can provide a rich understanding of phenomena that unfold across levels in an organisation. this study adds new knowledge to the current enterprise australasian journal of information systems techakriengkrai, techatassanasoontorn &tan 2021, vol 25, research article post-adoptive change of enterprise system implementations 21 system research in the context of the crm system by developing an understanding of both individual work practices and organisational change in the post-adoption stage of es implementation. for instance, the findings showed that in bi, the critical event of the new requirement to input customer information into the system generated a gap between taskstructure components. a number of staff were unable to complete their tasks at the initial stage of post-adoption (gap), which led to an increase in abandoned calls and a negative impact on work performance (outcome). the result showed that changes at the individual and organisational levels were interrelated and need to be considered in theorising change following is implementation. 5.2 conclusion 5.2.1 theoretical and practical contributions in terms of the theoretical contribution, this study used a socio-technical perspective as a lens to study the socio-technical changes related to es implementation. the findings revealed that all four interacting socio-technical components experienced changes after the introduction of an es in an organisation. additionally, the sequence of change proposed by lyytinen and newman’s (2008) study enabled this study to identify gaps or misalignments that occurred during the es implementation process. the mutually changing components could be examined, while identifying critical events and gaps. the critical events, which occurred during the change process, led to the generation of gaps between components. these gaps led to responses in some organisations and change outcomes that occurred in the post-adoption stage of crm implementation. this study provides a richer vocabulary for a process of change by revealing consequences relating to the generation of gaps between the socio-technical components. in terms of practical contributions, the findings offer strong evidence that organisations should take a proactive approach by preparing appropriate resources and support for successful es implementation. in particular, organisations should be aware of unanticipated events and unintended outcomes that may occur during the change process. since these events, as well as responses and outcomes, cannot be predicted in advance, they may require spontaneous interventions as they arise to increase user acceptance and to realise the benefits from es implementation. 5.2.2 limitations of the study the study also has limitations that should be acknowledged. firstly, at the time of the interview sessions, participants had been going through the change process relating to the crm system for at least one year or more. it is not possible to perfectly capture every moment of the change process because the change process is both complex and long. the retrospective nature of this research may affect the way participants recalled their entire change process. to minimise the retrospective nature of the study, we asked additional questions to allow participants to recall circumstances in the past and to elaborate and confirm their answers (huber & power, 1985). secondly, research findings generated in qualitative research are difficult to generalise because the findings are strongly impacted by individuals’ perspectives and values (lewis & ritchie, 2003). to enhance the generalisability of the findings of this study, multiple cases and cross-case analysis were employed to assess similarities and differences in results across three cases and to strengthen the findings. australasian journal of information systems techakriengkrai, techatassanasoontorn &tan 2021, vol 25, research article post-adoptive change of enterprise system implementations 22 5.2.3 future research an aspect that needs further examination relates to the outcomes of es usage. even though the results of this study represent the outcomes of using an es, the study did not investigate how an es shapes organisational performance. there is a need for further research on how organisations can improve their performance associated with es implementation. this study focused on different organisational contexts from three different types of business to reveal similarities and differences regarding organisational change after crm implementation, rather than focusing on national and organisational cultural contexts. future research could expand this study to consider the influence of culture on organisational change. this study was conducted in thailand, and it is suggested that the study be replicated in different countries in order to investigate other cultures. future research could expand this study to examine the cross-country context in order to deepen the understanding of different cultural backgrounds. finally, this research studied es implementation in the context of the crm system. future research could use this study and apply it to other es contexts to examine the change process in the post-adoption stage. references aburub, f. (2015). impact of erp systems usage on organizational agility. information technology & people, 28(3), 570-588. doi:10.1108/itp-06-2014-0124 alvesson, m. (1995). management of knowledge-intensive companies. new york, nj: walter de gruyter awasthi, p., & sangle, s. p. (2012). adoption of crm technology in multichannel environment: a review (2006-2010). business process management journal, 18(3), 445-471. doi:10.1108/14637151211232641 bala, h., & venkatesh, v. (2015). adaptation to information technology: a holistic nomological network from implementation to job outcomes. management science, 62(1), 156-179. doi:10.1287/mnsc.2014.2111 bloomrosen, m., starren, j., lorenzi, n. m., ash, j. s., patel, v. l., & shortliffe, e. h. (2011). anticipating and addressing the unintended consequences of health it and policy: a report from the amia 2009 health policy meeting. journal of the american medical informatics association, 18(1), 82-90. doi:10.1136/jamia.2010.007567 chuchuen, c., & chanvarasuth, p. (2011). the adoption factors of e-crm in service sector of thai smes. presented at the international conference on networking and information technology, wuhan, china davis, f. d. (1993). user acceptance of information technology: system characteristics, user perceptions and behavioral impacts. international journal of man-machine studies, 38(3), 475-487. doi:10.1006/imms.1993.1022 dong, s. (2012). decision execution mechanisms of it governance: the crm case. international journal of information management, 32(2), 147-157. doi:10.1016/j.ijinfomgt.2011.09.003 elbanna, a. r. (2007). implementing an integrated system in a socially dis-integrated enterprise. information technology & people, 20(2), 121-139. doi:10.1108/09593840710758040 australasian journal of information systems techakriengkrai, techatassanasoontorn &tan 2021, vol 25, research article post-adoptive change of enterprise system implementations 23 finnegan, d., & currie, w. l. (2010). a multi-layered approach to crm implementation: an integration perspective. european management journal, 28(2), 153-167. doi:10.1016/j.emj.2009.04.010 finnegan, d., & willcocks, l. (2007). implementing crm: from technology to knowledge. west sussex, uk: john wiley & sons gersick, c., (1991). revolutionary change theories: a multilevel exploration of the punctuated equilibrium paradigm. academy of management review, 16(1), 10–36 harris, s. g., & sutton, r. i. (1986). functions of parting ceremonies in dying organizations. academy of management journal, 29(1), 5-30. doi:10.2307/255857 huber, g. p., & power, d. j. (1985). retrospective reports of strategic-level managers: guidelines for increasing their accuracy. strategic management journal, 6(2), 171-180. doi:10.1002/smj.4250060206 hung, s. y., hung, w. h., tsai, c. a., & jiang, s. c. (2010). critical factors of hospital adoption on crm system: organizational and information system perspectives. decision support systems, 48(4), 592-603. doi: 10.1016/j.dss.2009.11.009 ignatiadis, i., & nandhakumar, j. (2007). the impact of enterprise systems on organizational resilience. journal of information technology, 22(1), 36-43. doi: 10.1057/palgrave.jit.2000087 jasperson, j. s., carter, p. e., & zmud, r. w. (2005). a comprehensive conceptualization of post-adoptive behaviors associated with information technology enabled work systems. mis quarterly, 29(3), 525-557. retrieved from http://dl.acm.org jones, e., sundaram, s., & chin, w. (2002). factors leading to sales force automation use: a longitudinal analysis. journal of personal selling & sales management, 22(3), 145-156. retrieved from www.tandfonline.com karahanna, e., straub, d. w., & chervany, n. l. (1999). information technology adoption across time: a cross-sectional comparison of pre-adoption and post-adoption beliefs. mis quarterly, 23(2), 183-213. doi:10.2307/249751 khalil, e. l. (1995). institutional theory of the firm? extension and limitation. review of political economy, 7(1), 43-51. doi:10.1080/09538259500000003 kim, m., sharman, r., cook-cottone, c. p., rao, h. r., & upadhyaya, s. j. (2012). assessing roles of people, technology and structure in emergency management systems: a public sector perspective. behaviour and information technology, 31(12), 1147-1160. doi:10.1080/0144929x.2010.510209 klein, k. j., & kozlowski, s. w. (2000). multilevel theory, research, and methods in organizations: foundations, extensions, and new directions. san fancisco, ca: josseybass ko, e., kim, s. h., kim, m., & woo, j. y. (2008). organizational characteristics and the crm adoption process. journal of business research, 61(1), 65-74. doi: 10.1016/j.jbusres.2006.05.011 leavitt, h. j. (1964). applied organization change in industry: structural, technical and human approaches. new perspectives in organizational research, 55-71 australasian journal of information systems techakriengkrai, techatassanasoontorn &tan 2021, vol 25, research article post-adoptive change of enterprise system implementations 24 lewis, j., & ritchie, j. (2003). generalising from qualitative research. in qualitative research practice: a guide for social science students and researchers (pp. 263-286). london, uk: sage publications luna-reyes, l. f., zhang, j., gil-garcía, j. r., & cresswell, a. m. (2005). information systems development as emergent socio-technical change: a practice approach. european journal of information systems, 14(1), 93-105. doi: 10.1057/palgrave.ejis.3000524 lyytinen, k., & newman, m. (2008). explaining information systems change: a punctuated socio-technical change model. european journal of information systems, 17(6), 589-613. doi:10.1057/ejis.2008.50 lyytinen, k., newman, m., & al-muharfi, a.-r. a. (2009). institutionalizing enterprise resource planning in the saudi steel industry: a punctuated socio-technical analysis. journal of information technology, 24(4), 286-304. doi:10.1057/jit.2009.14 mackay, r. b., & chia, r. (2013). choice, chance, and unintended consequences in strategic change: a process understanding of the rise and fall of northco automotive. academy of management journal, 56(1), 208-230. doi:10.5465/amj.2010.0734 mcleod, l., & doolin, b. (2012). information systems development as situated socio-technical change: a process approach. european journal of information systems, 21(2), 176-191. doi:10.1057/ejis.2011.43 merkle executive survey, 2018: 7 must-know crm adoption statistics and what they mean for your crm. retrieved from https://www.business2community.com/customerexperience/7-must-know-crm-adoption-statistics-mean-crm-02023236 miles, m. b., & huberman, a. m. (1994). qualitative data analysis: an expanded sourcebook. london, uk: sage newell, s., tansley, c., & huang, j. (2002). social capital and knowledge creation in an erp project team: unintended incapacitating consequences of social capital. in 3rd european conference on organisational knowledge, learning and capabilities. amcis 2001 proceedings. 217 newman, m., & robey, d. 1992. a social process model of user-analyst relationships, mis quarterly, 16(2), 249–266 patton, m. q. (1990). qualitative evaluation and research methods (2nd ed.). newbury park, ca: sage publications rollins, g. (2012). unintended consequences: identifying and mitigating unanticipated issues in ehr use. journal of ahima, 83(1), 28-3. sawyer, s., & southwick, r. (2002). temporal issues in information and communication technology-enabled organizational change: evidence from an enterprise systems implementation. the information society, 18(4), 263-280. doi:10.1080/01972240290075110 schilhavy, r., & iyer, l. (2007). exploring the information effects of identity and information transparency in reputation systems. presented at the digit workshops, montreal, canada australasian journal of information systems techakriengkrai, techatassanasoontorn &tan 2021, vol 25, research article post-adoptive change of enterprise system implementations 25 schwartz, s. j. (2002). in search of mechanisms of change in identity development: integrating the constructivist and discovery perspectives on identity. international journal of theory and research, 2(4), 317-339. doi:10.1207/s1532706xid0204_03 sebjan, u., bobek, s., & tominc, p. (2014). organizational factors influencing effective use of crm solutions. procedia technology, 16, 459-470. doi:10.1016/j.protcy.2014.10.113 seethamraju, r., & sundar, d. k. (2013). influence of erp systems on business process agility. iimb management review, 25(3), 137-149. doi:10.1016/j.iimb.2013.05.001 soh, c., & sia, s. k. (2004). an institutional perspective on sources of erp package– organisation misalignments. the journal of strategic information systems, 13(4), 375397. doi: 10.1016/j.jsis.2004.11.001 srivihok, a., & batanov, d. (2005). analysis of the readiness of thai sme for applying crm. presented at the fourth international conference on ebusiness, bangkok, thailand. retrieved from www.ijcim.th.org strong, d. m., & volkoff, o. (2010). understanding organization-enterprise system fit: a path to theorizing the information technology artifact. mis quarterly, 34(4), 731-756. doi:10.2307/25750703 veiga, j. f., keupp, m. m., floyd, s. w., & kellermanns, f. w. (2014). the longitudinal impact of enterprise system users’ pre-adoption expectations and organizational support on post-adoption proficient usage. european journal of information systems, 23(6), 691707. doi:10.1057/ejis.2013.15 vella, j., caruana, a., & pitt f., l. (2012). the effect of behavioural activation and inhibition on crm adoption. international journal of bank marketing, 30(1), 43-59. doi:10.1108/02652321211195695 williams, s. p., & hardy, c. (2005). public eprocurement as socio-technical change. strategic change, 14(5), 273-281. doi:10.1002/jsc.728 williams, p., ashill, n., & naumann, e. (2017). toward a contingency theory of crm adoption. journal of strategic marketing, 25(5-6), 454-474. doi: 10.1080/0965254x.2016.1149211 winter, s. g., & nelson, r. r. (1982). an evolutionary theory of economic change. cambridge, ma: harvard university press wognum, p., krabbendam, j., buhl, h., ma, x., & kenett, r. (2004). improving enterprise system support: a case-based approach. advanced engineering informatics, 18(4), 241253. doi:10.1016/j.aei.2005.01.007 wu, i. l., & wu, k. w. (2005). a hybrid technology acceptance approach for exploring e-crm adoption in organizations. behaviour & information technology, 24(4), 303-316. doi:10.1080/0144929042000320027 yin, r. k. (2003). case study research: design and methods (3rd ed.). thousand oaks, ca: sage publications yoo, y., lyytinen, k., & berente, n. (2007). an institutional analysis of pluralistic responses to enterprise system implementations. presented at the international conference on information systems, montreal, canada australasian journal of information systems techakriengkrai, techatassanasoontorn &tan 2021, vol 25, research article post-adoptive change of enterprise system implementations 26 zablah, a. r., bellenger, d. n., & johnston, w. j. (2004). customer relationship management implementation gaps. journal of personal selling and sales management, 24(4), 279-295. retrieved from www.tandfonline.com copyright: © 2021 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v25i0.2391 http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 theoretical foundation 2.1 enterprise system implementation and organisational change 2.2 a socio-technical perspective 3 research methodology 3.1 data collection 3.2 data analysis 4 findings 4.1 rc 4.2 hp 4.3 bi 4.4 cross-case analysis 4.4.1 relationship among users 4.4.2 work practices 4.4.3 organisational performance 5 discussion and conclusion 5.1 discussion 5.2 conclusion 5.2.1 theoretical and practical contributions 5.2.2 limitations of the study 5.2.3 future research paper title (paper title style) australasian journal of information systems roy, singh & shabnam 2021, vol 25, research on user involvement modelling customer engagement behaviour 1 modelling customer engagement behaviour in smart retailing sanjit k. roy univeristy of western australia, perth, australia sanjit.roy@uwa.edu.au gaganpreet singh op jindal global university, india saadia shabnam curtin university, perth, australia abstract smart retail technologies are transforming the way companies engage with customers in retailing context. however, the real onset is unknown of which smart retail technology (srt) characteristics and customer responses pave the way for customer engagement behaviour. this research investigates customer engagement behaviour in smart retailing by integrating meta-utaut and srt characteristics. by analysing quantitative survey data using pls path modelling, customer engagement manifestations are measured. results show complex relationships between srt characteristics (novelty, effectiveness, and interaction quality) and meta-utaut variables (performance expectancy, effort expectancy, social influence, and facilitating conditions) in influencing customers’ attitude, behavioural intention, and engagement behaviour towards smart retailing. retailers can use the findings to influence customers’ attitudes towards smart retail technologies, encouraging them to display customer engagement behaviour. keywords: customer engagement behaviour, smart retailing, smart services, meta-utaut, social influence 1 introduction the massive infusion of new in-store retail technologies is making customer engagement increasingly challenging (grewal et al., 2020a). in-store retail technologies are most commonly employed to transform retail stores' engagement with their customers (ferracuti et al., 2019; grewal et al., 2018). smart retail technologies (hereafter, srt) refers to an interactive and connected retail system that involves the effective management of different customer touchpoints to improve customer experience and enhance performance across these touchpoints (roy et al., 2017). for instance, self-service technologies, interactive displays, touch screens, virtual shelves, in-store mobile apps, self-check-out functionality (amazon go), and informative touchpoints influence customer engagement in retail stores (fan et al., 2020, adapa et al., 2020). with global retail sales projected to reach approximately 26.7 trillion u.s. dollars by 2022, up from 23.6 trillion u.s. dollars in 2018 (liam o'connell, 2020), retailers are in a constant search to explore srt with notable characteristics that may significantly influence consumer behaviour during the in-store interaction (grewal et al., 2020a). prior studies in the information systems (is) and marketing literature have investigated the impact of several srt characteristics on customers’ perceived value (adapa et al., 2020; karjaluoto et al., 2019; shaw and sergueeva, 2019), shopping experience (roy et al., 2017), intention to use srt (roy et al., 2018a), value co-creation (knote et al., 2020), and retail engagement (fan et al., 2020). however, australasian journal of information systems roy, singh & shabnam 2021, vol 25, research on user involvement modelling customer engagement behaviour 2 there is limited research investigating the role srt characteristics play in driving customer engagement behaviour (hereafter, ceb) (wang, 2020). ceb has received increased academic and practitioner attention because of its essential contribution to firm performance in terms of profitability and sustainable competitive advantage (hollebeek et al., 2016). given its importance, the marketing science institute also highlights customer engagement drivers as a research priority for 2018–2020 (msi, 2018). extant research in is literature focuses on the customer-smart object circumplex (novak and hoffman, 2019, knote et al., 2020), technology characteristics as an object of customer engagement (pöyry et al., 2020), and customers’ cognitive and emotional engagement as the drivers of ceb towards srt (fan et al., 2020). however, limited research attention has gone into empirically examining the impact of technology attributes and contextual factors on ceb towards srt. drawing from the meta-unified theory of acceptance and use of technology (meta-utaut) in the is literature, this study considers performance expectancy and effort expectancy as technology attributes (ta) and facilitating conditions and social influence as contextual factors (cf) (dwivedi et al., 2019; karjaluoto et al, 2019; shaw and sergueeva, 2019). in addition, service convenience is recommended as an essential driver of ceb (roy et al., 2020b). we argue that the ta (i.e., performance expectancy and effort expectancy) would offer customers a considerable gain and convenience, thereby, capable of influencing ceb towards srt. similarly, an individual's engagement toward srt may also be influenced by important people in their social network, i.e., family, friends, colleagues, and their opinion (baishya and samalia, 2020; venkatesh et al., 2012). specifically, it is assumed that people can perform a specific behaviour if they believe that one or more of the crucial referents think they should (peng et al., 2017). hence, we argue that social influence and facilitating conditions will influence ceb towards srt. based on the propositions of dwivedi et al. (2019; 2020), this study examines the role of customers’ attitude and behavioural intention towards srt in influencing ceb. most importantly this study extends the meta-utaut by examining the influence of srt characteristics such as novelty, effectiveness, and interaction quality on ta and cf to form customers' attitudes towards srt (adapa et al., 2020; roy et al., 2020a). based on the preceding discussion, this study has threefold objectives: 1. to examine the role of attitude and behavioural intention in influencing customers’ engagement behaviour towards srt. 2. to examine the impact of ta (i.e., performance expectancy, effort expectancy), and cf (i.e., social influence, facilitating conditions) on customers’ attitude towards srt. 3. to examine the influence of srt characteristics (novelty, effectiveness, interaction quality) on ta and cf. given that traditional retailing is now experiencing severe challenges from online retailers (chiu et al., 2011), in-store srt can be introduced as an effective strategy to offer a unique instore customer experience (grewal et al., 2020a; kim et al., 2017) and thus reduce the threat of customer churn. however, with srt implementation still in the growth phase, several in-store retailers are still anxious about its ability to influence ceb towards srt. we chose the meta-utaut (dwivedi et al., 2019) over the original unified theory of acceptance and use of technology (utaut) as the overarching theoretical framework because of its strengths. first, the original utaut model was short of attributes on 'individuals' who engaged with technology and were significant in describing their australasian journal of information systems roy, singh & shabnam 2021, vol 25, research on user involvement modelling customer engagement behaviour 3 dispositions towards the use of underlying technology. dwivedi et al.'s (2019) revised model (meta-utaut) incorporated and tested the influence of attitude as a mediating construct using meta analytic structural equation modelling with data collected from 162 prior utaut based studies. the results revealed that attitude partially mediated the effects of all four utaut exogenous variables on behavioural intention. in addition, the inclusion of attitude increased the explanatory power on the model significantly to 45 %, from 38 % without attitude (dwivedi et al., 2019). second, the current study did not test the influence of any moderating variable. this is consistent with the recommendations of meta-utaut which does not advocate a mandatory role of any moderating variable as moderators may not be unanimously applicable in different contexts (dwivedi et al., 2020). hence, meta-utaut is considered to be a meaningful alternative for the utaut model in understanding acceptance, usage, and behaviour towards technology (dwivedi et al., 2020). while previous studies have investigated the role of customer engagement in driving ceb towards srt (fan et al., 2020; baldus et al., 2015; vivek et al., 2012), this study contributes to the body of knowledge by investigating the determinants of ceb towards srt. the current research deviates and contributes to the is literature by examining the direct and mediating effect of customers’ attitudes in influencing ceb towards srt. also, this study validates the role of utaut variables (performance expectancy, effort expectancy, social influence, and facilitating conditions) in influencing customers’ attitude towards srt. the study contributes to the existing literature by investigating the antecedents capable of influencing customers’ attitude towards srt (roy et al., 2018a; vahdat et al., 2020). besides, the results also contribute to the ongoing debate on srt characteristics that impacts customer behaviour (adapa et al., 2020). overall, this study responds to the need for further research on investigating customers’ engagement and its associated behaviour with technologies including srt (grewal et al., 2020a; wang 2020; adapa et al., 2020). the rest of the paper is organised as follows. section two provides a literature review and theoretical background and explains the research model proposed in this study. next, the research methods, operationalisation of constructs, data collection, and sampling are discussed. finally, we present the findings of the study, followed by the theoretical and managerial implications and discuss the limitations and future research directions. 2 literature review 2.1 smart retailing smart retailing refers to "an interactive and connected retail system which supports the seamless management of different customer touchpoints to personalise the customer experience across different touchpoints and optimise performance over these touchpoints" (roy et al., 2017 p.3). smart retailing differs from other traditional technologies, including eretailing based on several characteristics and attributes (kabadyi et al., 2019). while the former incorporates interactions among the customers; smart objects, products (brands), retailer, and retail channel (touchpoints), the latter technologies are limited to retail channels (touchpoints) and a dyadic interaction (between firm and customers) (foroudi et al., 2018; roy et al., 2017). also, the traditional services as human-centred processes involve value creation through the interaction of two or more actors (individuals, organisations, or public authorities). however, in the context of smart services, the focus shifts towards the value creation at the time of human-technology interaction (knote et al., 2020). moreover, smart retailing offers a greater australasian journal of information systems roy, singh & shabnam 2021, vol 25, research on user involvement modelling customer engagement behaviour 4 sense of flexibility relative to the traditional retailing. also, the ability of smart retailing to connect both the physical and digital worlds enables real-time interaction with the customers (gregory, 2016). smart retailing involves the transformation of both front-and-back ends of the traditional instore retail spaces through mobile applications and contactless, omnichannel, ubiquitous technology (vazquez et al., 2017). the transformation at the front end involves the incorporation of the internet of things, intelligent application shelves, and intelligent hardware (roy et al., 2018a), thereby reimagining the customers’ in-store experience. the transformation at the backend involves the digital platform to ensure connectivity based on “network logic” to support omnichannel marketing (wu and cheng, 2018). hence, srt holistically improves the perception, communication, and computing power of offline retailing (fan et al., 2020). for instance, nike has launched touchpads to pay for products, wall screens to encourage customers to engage with the catalogue, qr codes to memorise a product, and smartphones to test the nike mobile app (attié et al., 2020). the ability of srt to provide perceived flexibility in comparison to traditional retailing (roy et al., 2017) is expectedly transforming the customer behaviour throughout the decision process stages, i.e., search, purchase, consumption, and after-sales (vrontis et al., 2017). the application of srt demands organisations to acquire new competencies to monitor, control, optimise the operations more effectively. such competencies require marketers to regularly examine srt characteristics that may assist in the development of an enabling environment where the smart retailers and the customers interact with each other. therefore, investigating the influence of srt characteristics such as novelty, effectiveness, and interaction quality on ta and cf in shaping customers’ attitudes towards srt becomes imperative. 2.2 customer engagement behaviour (ceb) the concept of customer engagement has recently received substantial attention both in academia and industry (precourt, 2016; hollebeek et al., 2019). ceb is viewed as an accumulation of the techniques through which customers engage in the company’s activities apart from just restricted to purchase (brodie et al., 2011; jaakkola and alexander, 2014). from the customers’ perspective, ceb is defined as “customers behavioural manifestations towards the brand or firm, beyond purchase, resulting from motivational drivers” (van doorn et al., 2010). this concept also supports the s-d logic informed definition of ceb (hollebeek et al., 2019) where authors argued that consumers deploy their competencies, interact to co-create, and phenomenologically determine the service value. ceb assumes customers as an exogenous entity whose behaviour in a smart retail setting may be deemed as being persuaded by both the in-store retail environments and their action and intention driving responses towards smart retailing (vivek et al., 2012). for instance, organisations offer significant investments in providing engagement platforms to customers through the online brand community (baldus et al., 2015), and through other interactive technologies. similarly, customers invest their resources (money, time, and effort) as a part of their effort towards ceb (van doorn et al., 2010). consistent with previous research (ángeles oviedogarcía et al., 2014; islam and rahman, 2016; jaakkola and alexander, 2014; van doorn et al., 2010), this study considers ceb as one-dimensional construct because it captures the behavioural manifestation of consumers interacting with smart technologies in retail stores. this conceptualisation clarifies the context-dependent nature of customers' expressions, such australasian journal of information systems roy, singh & shabnam 2021, vol 25, research on user involvement modelling customer engagement behaviour 5 as their engagement which tend to develop over time under conducive smart service context and favourable smart service experience (brodie et al., 2011). the preceding discussion indicates that there is a need to address the impact of srt characteristics and consumers affective responses to increase the likelihood of ceb in smart retailing context (adapa et al., 2020; grewal et al., 2020a; wang 2020). the following section presents the research model and specifies the relationships between variables in the model. 3 the research model the research model proposed (see figure 1) in this study adopts the (meta-utaut) theory (dwivedi et al., 2019) as an overarching framework and provides a holistic framework for determining ceb towards srt. prior research in the relevant domain has examined customer attitudes and behaviours towards srt by integrating several theories, such as the technology acceptance model (davis 1989) technology readiness perspective, and organisational theory (roy et al., 2018a) etc. the current study uses the meta-utaut model for its superior predictive validity compared to other technology usage models (tamilmani et al., 2020; dwivedi et al., 2020). we extended the meta-utaut to incorporate srt characteristics, namely, srt novelty, srt effectiveness, and srt interaction quality. the proposed research model examines the impact of srt characteristics (e.g., novelty, effectiveness, and interaction quality) on customers affective responses measured through the utaut factors (e.g., ta (i.e., performance expectancy and effort expectancy), and cf (social influence and facilitating conditions). further, the study models the impact of affective responses on customers' attitudes toward and intention for ceb towards srt. figure 1. research model australasian journal of information systems roy, singh & shabnam 2021, vol 25, research on user involvement modelling customer engagement behaviour 6 4 research hypotheses 4.1 relating srt characteristics and utaut variables 4.1.1 srt novelty, technology attributes, and contextual factors the novelty of srt refers to the degree to which customers perceive srt as exclusive for its innovativeness to contribute to accomplish their shopping goals more enjoyably (adapa et al., 2020) and satisfy their desire for newness (leroi-werelds, 2019). customers’ perception of novelty is subjective; hence, can manifest in different ways and must, therefore, be carefully examined (blythe, 1999). prior literature suggests that novelty characteristics of any service offered cause either positive or negative affective reactions (wells et al., 2010). these affective responses evolve because customers perceive novelty as either something they have never experienced before or as previously experienced attributes that are presented in unprecedented combinations (wells et al., 2010). the ability of srt to offer customers with enhanced convenience, experience, value, and engagement, over traditional retailing triggers several positive affective responses among customers (roy et al., 2017, fan et al., 2020). also, the increased interactivity in srt allows the customers to shop with ease (roy et al., 2017). these positive outcomes are possible because the perceived benefit of shopping through srt outweighs the costs associated with maintaining the status quo. thus, we advance the following hypotheses: h1 (a-b) srt novelty positively influences technology attributes (performance expectancy and effort expectancy). h1 (c-d) srt novelty positively influences contextual factors (social influence and facilitating conditions). 4.1.2 srt effectiveness, technology attributes, and contextual factors srt effectiveness refers to the extent to which customers perceive srt’s contribution to their shopping goals and eventually improves their overall shopping experience (roy et al., 2020a). customers perceive an increase in effectiveness when they can perform a transaction with increasing speed, control, and convenience (collier et al., 2015; de keyser et al., 2020) when shopping via srt. the ability of srt to leverage improved experience enhances customers’ shopping satisfaction and ultimately increases their shopping effectiveness (roy et al., 2017). in addition to that, smart customer experience also reduces customers’ perceived risk towards srt, thereby, also increasing their shopping effectiveness (roy et al., 2017). the ability of srt to allow customers to accomplish their shopping tasks with efficiency and ease will impact customers’ affective behaviour (overby and lee, 2004). the srt effectiveness is likely to influence customers’ perceptions of the resources and support available by the smart retailer to perform a behaviour (brown and venkatesh, 2005; collier et al., 2015; de keyser et al. 2020). therefore, we advance the following hypotheses: h2 (a-b) srt effectiveness positively influences technology attributes (performance expectancy and effort expectancy). h2 (c-d) srt effectiveness positively influences contextual factors (social influence and facilitating conditions). australasian journal of information systems roy, singh & shabnam 2021, vol 25, research on user involvement modelling customer engagement behaviour 7 4.1.3 srt interaction quality, technology attributes, and contextual factors srt interaction quality is customers’ perception toward any specific event of interaction with the srt, which takes into account the environmental elements in the store. the quality of interaction thus fosters individuals' engagement levels throughout a specific event of interaction and generate certain affective responses (bejerholm and eklund, 2007). in smart retailing, the interactions between customers and srt occurs in several ways. these include interactions among the customers, smart objects, products (brands), retailers, and retail channels (touchpoints) (roy et al., 2017; knote et al., 2020). when customers interact with technology including srt, their overall interaction quality is contingent on factors such as information content quality (park et al., 2008), system quality, and service quality (shin, 2015) or a combination thereof. the information quality is determined by both the informativeness and entertainment ability of the technology (negash et al., 2003). the former deals with information accuracy, relevance, timeliness, convenience, and completeness. the latter discusses the ability of the technological interface to be entertaining, enjoyable, pleasing, fun, and exciting for customers (negash et al., 2003). the system quality is assessed by the interactivity and the accessibility of the technology. interactivity refers to the degree to which customers can participate in altering the form and content of a media-based environment in real-time. access refers to the ease of using and accessing the interface while extracting the information (negash et al., 2003). the quality interactions and information-sharing mechanisms may subsequently tender social influence on one another (bapna and umyarov, 2015). also, the quality system may serve as platforms for facilitating interactions with and among customers (viswanathan et al., 2017; grewal et al., 2020b). based on the preceding discussion, we propose the following hypotheses: h3 (a-b) srt interaction quality positively influences technology attributes (performance expectancy and effort expectancy). h3 (c-d) srt interaction quality positively influences contextual factors (social influence and facilitating conditions). 4.2 relating utaut variables (technology attributes and contextual variables) and attitude towards srt 4.2.1 performance expectancy and attitude towards srt performance expectancy is defined as the degree to which an individual believes that srt will assist them in achieving gains (venkatesh et al., 2003). we draw upon the equity theory (adams, 1963) to examine customer attitude towards and their behavioural intention towards srt. according to this theory, customers would engage with new technology, including srt depending on their gain/effort ratio. customers will view the technology with more optimism and develop a positive attitude if their engagement would accrue higher gains than their efforts (evanschitzky et al., 2015). on the contrary, customers will develop negative attitudes when the benefits of new technology are observed by retailers at shoppers' expense (inman and nikolova, 2017; grewal and roggeveen, 2020). therefore, we propose the following hypothesis: h4 (a) performance expectancy positively influences attitude towards srt. 4.2.2 effort expectancy and attitude towards srt effort expectancy is defined as “the degree of ease associated with customers’ use of technology” (venkatesh et al., 2012, p. 159). customers’ desire to adopt new technology is not australasian journal of information systems roy, singh & shabnam 2021, vol 25, research on user involvement modelling customer engagement behaviour 8 only envisaged by the associated positive values, but also by the ease and free efforts with which individuals can operate this, plays an essential role (davis, 1989). due to the specific nature of srt, customers need to acquaint themselves with a certain level of knowledge and skill. hence, effort expectancy could play a vital role in determining customers’ attitudes towards srt. customers may perceive difficulty in using srt since it entails a higher level of customer involvement compared to their limited involvement in the full-service option. for instance, a struggle to handle smart retailing mobile application may act as one of the vital dimensions of complexity (inman and nikolova, 2017; grewal et al., 2020b). also, personality characteristics, such as inertia, or the desire for human interaction (white et al., 2012) are likely to contribute to customers’ perceived difficulty to use new technologies including srt (meuter et al., 2000). moreover, customers may perceive the information overload as complex (iyengar and lepper, 2000). while some customers may perceive the extra effort as only marginal, others may feel it burdensome and thus form an unfavourable attitude towards srt. based on the preceding discussion, we advance the following hypothesis: h4 (b) effort expectancy positively influences attitude towards srt. 4.2.3 social influence and attitude towards srt according to the utaut model, social influence is characterised as “the degree to which an individual perceives that important others believe he or she should use the new system” (venkatesh et al., 2003 p 451). this explanation incorporates two dimensions, namely subjective norm and descriptive norm. the former defines an individual's perception of other expectations (compliance), and the latter refers to the perception of attitudes possessed by significant others' behaviours (wang and lin, 2011). besides, the literature proposes that the demonstrability of several people’s actions is significant to persuade a single individual’s behaviour (elek et al., 2006). this implies an individual is likely to believe certain behaviour as sensible when they observe that many others are following it. previous research suggests that, when the customer selects to either accept or resist new technology, including srt, they also consider the effects of this selection on their relationship with others. hence, customers incline to conform to the majority decision (qin et al., 2011). in this context, conformity may be assumed as personal acceptance leading to private attitudinal change (book et al., 2018). normative conformity might also be invoked through social norms, which sees individuals decides to act while considering the expected or acceptable behaviour in social interactions (knoll et al., 2015). individual compliance with expectations or social norms, thus, encourages them to execute an action if they perceive of gaining greater social influence in return (talukder and quazi, 2011). it is argued that public opinion plays a vital role in impacting individuals' attitude towards a given technology (kang, 2014). likewise, with increasing technology diffusion, probable adopters are more likely to observe the technology in use, which may encourage them to believe it is useful (graf-vlachy et al., 2018). therefore, it is expected that customers' attitudes towards srt may be affected when they perceive many others using it, especially in their social groups (carter and yeo, 2016; roggeveen et al., 2020). thus, we propose the following hypothesis: h4 (c) social influence positively influences attitude towards srt. 4.2.4 facilitating conditions and attitude towards srt facilitating conditions are defined as “the degree to which an individual believes that an organisational and technical infrastructure exists to support the use of the system” (venkatesh et al., 2003 p 453). in other words, facilitating conditions can be viewed as customer perception australasian journal of information systems roy, singh & shabnam 2021, vol 25, research on user involvement modelling customer engagement behaviour 9 about the availability of resources that enhance their use of technology (hossain et al., 2017). dwivedi et al. (2019) theorised that facilitating conditions has a significant influence on customers’ attitude. this implies that facilitating conditions such as assistance centres and training programs for using the technology may act as vital enablers in influencing customers' attitudes (dwivedi et al., 2019; sabharwal et al., 2006; grewal et al., 2020a). in the context of srt, facilitating conditions may include the presence of intelligent systems, human-machine interactions, self-service, and product content quality (fan et al., 2020; roggeveen et al., 2020). these facilitating conditions impact customers' smart experience quality and eventually influence their attitude towards srt. based on the proceeding discussion and empirical evidence, we hypothesise the following: h4 (d) facilitating conditions positively influences attitude towards srt. 4.3 influence of attitude towards srt and ceb attitude refers to an individual’s positive or negative feelings about performing the target behaviour (davis 1989; taylor and todd 1995). customers attitude plays a key role in determining their acceptance and use of technology, including srt. dwivedi et al. (2019) hypothesised behavioural intention to mediate the effect of attitude on usage behaviour. however, their meta-analysis results revealed that attitude has a direct impact on customer usage behaviour as well. this signifies that attitude may significantly influence behaviour even though the customer may not intend to use the technology. existing literature indicates, the influence of behavioural intention on usage behaviour may not be particularly strong or predictable (pardamean and susanto 2012; weerakkody et al., 2013), which further emphasises the significance of the direct effect between attitude and usage behaviour. therefore, we argue that customers with a positive attitude towards srt will be influenced to help organisations and customers directly and indirectly beyond the purchase process (grewal and roggeveen, 2020). based on the proceeding discussion and theoretical conceptualisation by dwivedi et al. (2019), we hypothesise the following: h5 attitude towards srt positively influences ceb towards srt. 4.4 the mediating role of attitude towards srt the four exogenous constructs in the meta-utaut (i.e., technology attributes and contextual factors) represent customer perceptions regarding the technology and the context. though there exists significant evidence that these four constructs explain a significant proportion of variance in customer adoption and usage behaviours, utaut excluded several key individual characteristics that describe customers’ disposition in influencing their behaviours. customers’ attitude is one of the several such individual aspects (chong 2013; venkatesh et al., 2011). realising the importance, dwivedi et al., (2019) recommended the inclusion of customer attitude in examining customer behaviour towards technology. the inclusion of attitude in models of technology acceptance is consistent with the theory of reasoned action and the theory of planned behaviour (ajzen, 1991). we position attitude as a mediator between performance expectancy, effort expectancy, social influence, facilitating conditions, and ceb towards srt. this is because an individual’s attitude may be shaped by perceived usefulness (vahdat et al., 2020); perceived ease of use, and superior functionality (roy et al., 2018a). in other words, technology capabilities may influence customers’ attitudes towards srt. also, facilitating conditions such as technology readiness (roy et al., 2018a) and social influence (vahdat et al., 2020; roggeveen et al., 2020), which are contextual factors may also influence australasian journal of information systems roy, singh & shabnam 2021, vol 25, research on user involvement modelling customer engagement behaviour 10 customers’ attitude towards srt. therefore, it is expected that customers' attitudes towards srt will influence them to involve in engagement behaviour towards srt. based on the above discussion and empirical evidence, we propose the following hypothesis: h6: attitude towards srt mediates the relationship between technology attributes (e.g., performance expectancy and effort expectancy) and contextual factors (e.g., social influence and facilitating conditions) and ceb. 4.5 the mediating role of behavioural intention towards srt in addition to attitude, researchers also identified the influence of the behavioural intention on actual behaviour (bagozzi and yi, 1989; bhattacharjee and sanford, 2009). we also argue that only customers’ positive attitude may not be sufficient to encourage customers to voluntarily contribute their resources (time, money, and effort) to help organisations and other customers through ceb (jaakkola and alexander, 2014; wang, 2020). customers’ willingness and effort, thus the intention to perform the underlying behaviour is imperative (ajzen, 2011). as the theory of planned behaviour (tpb) and theory of reasoned action (tra) argues that stronger the intentions, the higher is the probability of performing the underlying behaviour (fishbein and ajzen, 1975; ajzen, 1991). hence, it can be argued that individual behaviour is driven by their behavioural intention, which is, in turn, a function of an individual's attitude towards the behaviour (ajzen, 2011). based on the preceding discussion we propose the following hypothesis: h7 behavioural intention towards srt mediates the relationship between attitude towards srt and ceb. 4.6 control variables according to the prior literature, certain demographic variables can influence key constructs examined in this study. hence, to allow for better delineation of the relationship proposed in our model and to provide a more rigorous test of the theoretical linkages, we included demographic variables, such as age group, and gender as control variables. this helps to account for respondent heterogeneity and avoid any confounding influences on the results. controlling for these variables will enhance the generalisability of the study findings (balaji and roy, 2017). 5 methodology 5.1 research context retailing is one of the major sectors in the australian economy comprising of various subsectors including consumer goods and supermarket retailing. it contributes approximately 9 % to the australian gdp per annum (roy et al., 2019). the supermarkets and grocery market together make up $103 billion approximately in australia. out of this, woolworths and coles accounted for over 65% of industry revenue in 2018 (ibis world, 2018). in the changing retail landscape as a result of technology infusion, it is expected that traditional retailers will need to adapt their marketing strategies and business models to stay competitive (magner, 2016). various industry reports suggest that the australian retail industry is undergoing a major transformation with the infusion of new technologies in service delivery (australian retailers association, 2020). australasian journal of information systems roy, singh & shabnam 2021, vol 25, research on user involvement modelling customer engagement behaviour 11 smart retailing in australia is the research context for this study to examine ceb towards srt. the ability of the internet of things (iot) to institute transformational changes and hypercompetition encourages traditional retailers to periodically incorporate new technological innovations for growth and efficiency (caro and sadr, 2019; roy et al., 2019). smart retailing (e.g., smart shopping carts, shopping assistants, near field communications, smart payment systems, and ai-based technologies) is one such common business transformation practice adopted by the retailers. smart retailing incorporates multiple interactions (roy et al., 2017). these interactions influence customer outcome variables, including word-of-mouth, purchase loyalty (fan et al., 2020; roy et al., 2017). therefore, smart retailing becomes one of the ideal contexts to examine ceb. also, due to the general lack of studies focused on ceb towards srt, the effects of technology attributes and contextual factors have been overlooked. therefore, it is imperative to study what influences the ceb explicitly towards srt. 5.2 sample and data collection the web-based survey data was collected in australia using qualtrics. potential respondents were invited to participate through a web-link coupled with a survey questionnaire along with small incentives. data was collected in the second quarter of 2019. the services of a market research company were availed to collect data. the consumer panel members were directed to respond to the survey on srt in the retailing context. the market research agency used a quota-based approach to ensure that the respondents represented the population of interest in australia as closely as possible. upon agreeing to participate, respondents were requested to complete the questionnaire. the questionnaire contained measurement items of srt novelty, srt effectiveness, srt interaction quality, performance expectancy, effort expectancy, social influence, facilitating conditions, attitude towards srt, behavioural intention towards srt, ceb and question on demographics. the respondents were duly informed to consider their most recent experience with srt (i.e., within the last six months) as a reference stimulus while responding to the questionnaire. industry reports and prior literature was referred to develop a shared understanding of customer facing srt (gregory, 2016, willems et al., 2017). smart checkouts, personal shopping assistance, point-of-sale smart displays, nfc systems, and augmented reality were some of the commonly reported srts in the questionnaire. a total of 293 valid responses were obtained with a response rate of 45%. soper’s (2014) sample size calculator was used to calculate the minimum sample size requirement. the minimum sample recommended for structural equation modelling with ten latent variables, 40 observed variables, p-level, 0.05, and anticipated size effect, 0.03 is 100 (soper's (2014). the study met the sample size criterion to model the structure. the t-tests did not reveal significant differences in the study constructs for the early and late response groups, eliminating the concern of non-response bias in this study. the sample had: (1) gender: 45% of respondents were male and 55% were female; (2) age group: 56% of respondents were in 21–35 age group, representing a comparatively younger group of customers; and (3) education level: 54% of respondents had bachelor’s degree. 5.3 measurement instrument the constructs in this study were measured using validated items from the prior literature (see appendix a). the three srt characteristics (srt novelty, srt effectiveness, and srt interaction quality) were measured using items proposed by o’brien and crains, (2015) collier et al., (2015), and brady and cronin, (2001) respectively. the affective responses representing the dimensions of utaut (performance expectancy, effort expectancy, facilitating conditions, australasian journal of information systems roy, singh & shabnam 2021, vol 25, research on user involvement modelling customer engagement behaviour 12 and social influence) were measured using items suggested by zhou et al., (2010), and kaushik and rahman, (2015). attitude towards srt consisted of three 7-point semantic differential items adapted from weijters et al., (2007). the ceb was measured using items suggested by yi et al., (2013) and balaji (2014). responses to all the questions were measured using a 7point likert scale anchored at "strongly disagree (1)" to "strongly agree (7)". the draft questionnaire was evaluated, and pilot tested with fifty students to check for any ambiguous sentence wording or expressions. the feedback received from the pilot study was included in the questionnaire, and the final questionnaire was then administered to actual retail shoppers in australia by a market research firm. 6 data analysis partial least squares path modelling (pls-pm) using smartpls 3.0 (ringle et al., 2015) was selected to analyse the data. the reasons to choose pls-pm include: pls-pm can effectively manage non-normal data, unlike covariance-based structural equation modelling (lowry and gaskin, 2014). pls-pm can proficiently handle complex predictive models by maximising the variance explained in the ultimate dependent variable (sarstedt et al., 2020). the use of plspm was suitable for this study because of its exploratory nature, its complex and predictionoriented model, the likelihood that the items would not be normally distributed, and its ability to consider the total variance of measurement items while testing the model (hair et al., 2012; hair et al., 2017). in this study, the objective is to predict relationships between the constructs of interest rather than confirming a theory. also, the prediction orientation of pls-pm makes it an ideal technique for determining the factors influencing ceb. following hair et al., (2017), the model was analysed and interpreted in two stages: examining the validity and reliability of the measurement model followed by empirical testing of the proposed hypotheses. 6.1 common method bias (cmb) the collection of data from the single source allowed cmb to possibly affect the relationships between the constructs (podsakoff et al., 2003). in this study, the cmb was controlled using both procedural and statistical methods. several procedural remedies were adopted in the study design phase. these include ensured the anonymity of the participants, informed participants that there are no correct or incorrect answers, requested them to respond to the questionnaire as honestly as possible, used different scale formats (likert scale and semantic differential scale), and improvised the wording of measurement item during the pilot study (podsakoff et al., 2003). statistically, harman’s single-factor test was performed, and it was found that no single factor exceeds the threshold of more than 50% of the total variance (podsakoff et al., 2003). moreover, an examination of the model constructs’ correlation matrix showed 0.601 being the highest correlation value, thus remaining well below the critical threshold limit of 0.90 (pavlou et al., 2006). the marker variable approach recommended by lindell and whitney (2001) and tehseen et al. (2017) was also used. we used respondents’ mobile phone usage intensity as an unrelated construct. mobile phone usage intensity refers to the extent to which mobile phones are integrated into the daily routines of the respondents. we adapted the items from valenzuela et al. (2009) to measure the market variable, i.e., mobile phone usage intensity. the inclusion of the marker variable did not significantly increase the variance of the dependent variable. results indicate that the average correlation between latent variables in the proposed research model and the marker variable is 0.035, and the average significance was 0.53, which is well above the threshold of 0.05 and makes it necessary to australasian journal of information systems roy, singh & shabnam 2021, vol 25, research on user involvement modelling customer engagement behaviour 13 consider the correlations as being significant. therefore, common method bias is not a major issue in this study. 6.2 measurement properties table 1 shows the measurement model and the psychometric properties of the scale. the constructs' reliability, convergent validity, and discriminant validity were examined. the results highlighted that the standardised factor loadings are well above the recommended minimum threshold levels of 0.50 (hair et al., 2006). the reliability of the variables as indicated by cronbach's alpha and composite reliability is greater than the recommended threshold levels of .70 and .80, respectively (hair et al., 2006). results highlight that the measurement model has adequate internal consistency. also, the average variance extracted for each construct surpassed the threshold levels of 0.50 (hair et al., 2006). this value supported the convergent validity of the measurement model. the discriminant validity of the measurement model was examined using two methods, namely, fornell and larcker (1981) method and the htmt-ratio approach. the former method proposes the discriminant validity of the construct to be satisfactory if the square root of the average variance extracted of each construct was greater than the correlations it shared with other constructs (fornell and larcker, 1981). it was found satisfactory in the measurement model. with htmt ratio values less than 0.90 (henseler et al., 2015), the model also indicated satisfactory discriminant validity. we also computed the variance inflation factor (vif) to check for the multicollinearity among the constructs in the proposed research model. results show that the full vif value was 2.56. hence, there is no concern around multicollinearity (petter et al., 2007). since the full vif value was less than the cut-off threshold value of 3.3, multicollinearity is not an issue in this study (kock, 2015). 6.3 structural model results following the measurement model, the structural model was evaluated. we employed the rsquared coefficient to assess the fit of the structural model. the explanatory power of the model was assessed using the measure of explained variance. the r2 value of performance expectancy (17%), effort expectancy (15%), social influence (12%), and facilitating conditions (14%) indicated medium effect size (~.09, cohen, 1988). the r2 value of attitude (40%) and ceb (46 %) indicated large effect sizes (~.25, cohen, 1988). considering the exploratory nature of the study, we contend that r2 values highlight a good representation of the data to the measurement and structural model (hair et al., 2012). the stone-geisser q2 values for all the endogenous constructs in this study are positive, highlighting the predictive validity of the research model (chin and newsted, 1999; hair et al., 2012). further, the goodness-of-fit (gof) is defined “as the geometric mean of the average commonality, and the average r2” (tenenhaus et al., 2005, p. 173) for the complete structural model was 0.41 which confirms that the model performs well. table 2 presents the path coefficients along with t-statistics. the pls structural equation modelling results support several of the proposed hypotheses. the hypothesis h1-h3 predicted the impact of srt characteristics on affective responses measured by the utaut variables. h1 (a-d), which suggested there was a positive relationship between srt novelty and utaut variables was partially supported. australasian journal of information systems roy, singh & shabnam 2021, vol 25, research on user involvement modelling customer engagement behaviour 14 constructs and items measurement properties loadings t statistics cronbach's alpha composite reliability average variance extracted (ave) performance expectancy (pex) 0.925 0.947 0.816 pex1 0.884 47.353 pex2 0.92 56.984 pex3 0.926 76.814 pex4 0.882 50.772 effort expectancy (eex) 0.888 0.923 0.752 eex1 0.884 44.772 eex2 0.896 51.37 eex3 0.905 57.704 eex4 0.776 25.471 social influence (sif) 0.892 0.916 0.609 sif1 0.783 23.617 sif2 0.808 24.37 sif3 0.729 21.152 sif4 0.793 26.201 sif5 0.852 34.501 sif6 0.834 29.363 sif7 0.646 12.968 facilitating conditions (fcd) 0.858 0.903 0.7 fcd1 0.849 35.336 fcd2 0.861 34.24 fcd3 0.824 26.399 fcd4 0.811 24.884 srt novelty (novel) 0.921 0.943 0.804 novel1 0.848 25.564 novel2 0.913 73.385 novel3 0.916 66.338 novel4 0.909 56.802 srt effectiveness (sef) 0.815 0.872 0.63 sef1 0.739 15.429 sef2 0.822 17.171 sef3 0.84 21.404 sef4 0.771 15.437 srt interaction quality (inq) 0.834 0.923 0.858 inq1 0.927 62.455 inq2 0.925 49.6 attitude towards srt (att) 0.939 0.961 0.891 att1 0.942 85.473 att2 0.947 110.216 att3 0.942 94.324 behavioural intentions towards srt (bin) 0.937 0.955 0.841 bin1 0.92 81.925 bin2 0.916 78.302 bin3 0.907 63.397 bin4 0.924 87.8 customer engagement behaviour (ceb) 0.85 0.9 0.701 ceb1 0.888 57.783 ceb2 0.928 60.312 ceb3 0.934 91.221 ceb4 0.531 7.474 table 1. measurement model and psychometric properties of the scale australasian journal of information systems roy, singh & shabnam 2021, vol 25, research on user involvement modelling customer engagement behaviour 15 specifically, srt novelty was found to positively influence performance expectancy, thereby supporting h1a (β= 0.121, p< 0.05), however, h1b (β= 0.009, p =0.888), h1c (β= 0.291, p = 0.077), h1d (β= 0.038, p =0.51) are not supported as srt novelty did not impact effort expectancy, social influence and facilitating conditions. h2 (a-d), which anticipated a positive relationship between srt effectiveness and utaut variables, was partially supported. specifically, support was found for the suggested positive relationship between srt effectiveness and performance expectancy, effort expectancy and facilitating conditions, thereby supporting h2a (β= 0.122, p< 0.01), h2b (β= 0.328, p< 0.01), h2d (β= 0.342, p< 0.01), however, h2c (β= 0.059, p= 0.316) is rejected as srt effectiveness did not support social influence. h3 (a-d), which suggested a positive relationship between srt interaction quality and utaut variables was partially supported. precisely, support was found for the suggested positive relationship between srt interaction quality and performance expectancy, effort expectancy and facilitating conditions, thereby supporting h3a (β= 0.279, p< 0.01), h3b (β= 0.183, p< 0.01), h3d (β= 0.151, p< 0.05) however, h3c (β= 0.125, p= 0.07) is rejected as srt interaction quality did not support social influence. performance expectancy (β= 0.385, p< 0.01), social influence (β= 0.165, p< 0.01), and facilitating conditions (β= 0.122, p< 0.01), are shown to positively affect online customers' attitude towards srt, so h4a, h4c, and h4d are supported. however, h4b is not supported since effort expectancy (β= 0.138, p=0.112) did not impact customers' attitudes towards srt. the direct effect of attitude towards srt on ceb is significant and positive (β= 0.254, p< 0.001), which provides support to hypothesis h5. 6.4 mediation analysis in this study, h6 and h7 proposed that attitude and behavioural intention plays key mediating roles in the proposed research model. we investigated the mediating effects by testing the indirect effects and bias-corrected bootstrap confidence intervals (hair et al., 2017; nitzl et al., 2016), and results are shown in table 2. results also show that the indirect effects between performance expectancy and ceb (β= 0.226, p< 0.001) and between social influence and ceb (β= 0.077, p< 0.01) are significant the bias-corrected bootstrap confidence intervals are different from zero (as shown in table 2). the direct effect of performance expectancy on ceb is significant. hence, attitude is a complementary mediator between performance expectancy and ceb. the direct effect of social influence on ceb is not significant. thus, attitude is an indirect only mediator between social influence and ceb (zhao et al., 2010). the indirect effects of effort expectancy and ceb and facilitating conditions are not significant. hence, results show partial support for hypothesis h6. results also show that the indirect effect of attitude towards srt on ceb is significant (β= 0.125, p< 0.01). the bias-corrected bootstrap confidence interval of behavioural intention as a mediator between attitude and ceb is different from zero (0.049 to 0.209). the direct effect of attitude on ceb is significant (β= 0.254, p< 0.001). thus, we conclude that behavioural intention acts as a complementary mediator (zhao et al., 2010). therefore hypothesis h7 is supported in this study. 7 discussion and implications the findings of this study offer several key insights associated with examining factors influencing ceb towards srt. using pls-pm, this research focuses on examining the role of srt characteristics in shaping customers' affective response towards srt, which eventually impacts customers' attitude, intention, and ceb towards srt. more specifically, this study shows how srt novelty, srt effectiveness, and srt interaction quality impacts customers' perceptions about ta (performance expectancy, effort expectancy), and, cf (social influence australasian journal of information systems roy, singh & shabnam 2021, vol 25, research on user involvement modelling customer engagement behaviour 16 and facilitating conditions). findings also show how ta and cf shape customers’ attitude towards srt and ultimately impacts ceb towards srt. finally, we investigate the mediating role of behavioural intention in the relationship between attitudeand ceb towards srt. hypothesis relationships path coefficients t value pvalue inference control relationships age →ceb_srt 0.068 1.649 0.106 not significant gender →ceb_srt 0.044 1.035 0.313 not significant model relationships h1 (a) srt_novelty→ performance_expectancy 0.121 2.023 < .05* significant h1 (b) srt_novelty → effort_expectancy 0.009 0.141 0.888 not significant h1 (c) srt_novelty → social_influence 0.291 1.766 0.077 not significant h1 (d) srt_novelty→ facilitating_conditions 0.038 0.659 0.51 not significant h2 (a) srt_effectiveness→ performance_expectancy 0.227 4.012 < .01** significant h2 (b) srt_effectiveness→ effort_expectancy 0.328 5.716 < .01** significant h2 (c) srt_effectiveness→ social_influence 0.059 1.003 0.316 not significant h2 (d) srt_effectiveness→ facilitating_conditions 0.342 5.942 < .01** significant h3 (a) srt_interaction_quality→ performance_expectancy 0.279 3.8 < .01** significant h3 (b) srt_interaction_quality→ effort_expectancy 0.183 2.537 < .05* significant h3 (c) srt_interaction_quality→ social_influence 0.125 1.766 0.077 not significant h3 (d) srt_interaction_quality→ facilitating_conditions 0.151 2.161 < .05* significant h4 (a) performance_expectancy→ attitude_srt 0.385 4.856 < .01** significant h4 (b) effort_expectancy → attitude_srt 0.138 1.591 0.112 not significant h4 (c) social_influence → attitude_srt 0.165 3.147 < .01** significant h4 (d) facilitating_conditions → attitude_srt 0.122 2.28 < .05* significant h5 attitude_srt → ceb 0.254 < 0.001*** significant table 2. structural model results mediation relationships indirect effect t values p values lcl (2.5%) ucl (97.5%) inference h6 performance_expectancy→ attitude_srt → ceb 0.226 3.746 0 0.115 0.35 significant effort_expectancy → attitude_srt →ceb 0.089 1.622 0.097 -0.017 0.192 not significant social_influence→ attitude_srt→ ceb 0.077 1.981 0.04 0.002 0.152 significant facilitating_conditions → attitude_srt → ceb 0.15 0 0.05 -0.001 0.184 not significant h7 attitude_srt→ behavioural intention→ ceb 0.125 3.059 0.002 0.049 0.209 significant australasian journal of information systems roy, singh & shabnam 2021, vol 25, research on user involvement modelling customer engagement behaviour 17 7.1 theoretical contributions the findings of this study adds to the contemporary marketing and is literature. specifically, findings contribute to the ongoing debate about evolving customer engagement and associated behaviours due to the rapid infusion of futuristic in-store retail technologies (grewal et al., 2020). prior research has explored srt (huang and liao, 2015; müller-seitz et al., 2009) and provided understandings regarding the factors that influence customer adoption of these srt (adapa et al., 2020; roy et al., 2018a). however, to our knowledge, there is no thorough examination of factors driving ceb towards srt (fan et al., 2020). this study is one of the first to examine whether and how technology attributes and contextual factors derived from meta-utaut (dwivedi et al., 2019) influence customers' attitudes towards srt and eventually impacts ceb towards srt. results contribute to the literature in several ways. first, the study empirically validates the theoretical conceptualisation of meta-utaut (dwivedi et al., 2019) in the is literature. in doing so, researchers propose that (a) attitude directly impacts usage behaviour (b) attitude mediates the effects of ta (performance expectancy and effort expectancy), and cf (facilitating conditions and social influence) on user behaviour. the pls-pm results validate that customers' attitudes towards srt directly influenced ceb towards srt and also mediated the effects of ta, cf on ceb. by doing so, the study deviates from the widely prevalent theoretical conceptualisation in the literature that does not include the role of customer attitude in shaping usage behaviour and instead relies on directly monitoring the impact of ta, cf on behavioural intention and engagement behaviour (alalwan et al., 2017; kapser and abdelrahman, 2020). secondly, previous studies have examined the role of customer engagement (cognitive and emotional) in establishing ceb towards srt (fan et al., 2020). since, attitude plays a central role in the usage behaviour of technologically innovative products (dwivedi et al., 2019, 2020), the present study incorporated and investigated the role of customers’ attitude and behavioural intention in displaying ceb towards srt. this contribution is a welcome addition to the literature given the lack of research into ceb towards srt (wang, 2020). third, the results advance earlier research on factors influencing customers’ attitude towards srt. the findings of this study extend the antecedents of customers attitude towards srt by incorporating the role of ta and cf in influencing customers’ attitudes towards srt. this study answers the call for more research into including the contextual factors in technologyenabled retail environments (patil et al., 2020; hollebeek et al., 2019). results reveal effort expectancy to be a non-significant predictor of customers' attitude towards srt. this result can be interpreted with an argument that effort expectancy may not act as a strong predictor if facilitating conditions are included in the model (venkatesh et al., 2003). these findings extend our knowledge concerning the role of additional antecedents influencing customers' attitudes towards srt. finally, regarding srt characteristics, results support the role of srt novelty, srt effectiveness, and srt interaction quality in influencing customers' performance expectancy. the results also support the role of srt effectiveness and srt interaction quality in influencing customers' perception of effort expectancy and facilitating conditions. these findings corroborate and enhance previous research findings on the role of srt characteristics in enhancing perceived value (adapa et al., 2020). overall, by examining the impact of srt characteristics on customers’ affective responses and its subsequent impact in driving australasian journal of information systems roy, singh & shabnam 2021, vol 25, research on user involvement modelling customer engagement behaviour 18 customers' attitude, behavioural intention, and ceb extend our understanding of factors that influence ceb towards srt. 7.2 managerial implications our findings show that customers' attitude towards srt significantly influenced ceb towards srt, which implies smart retailers may find it beneficial to shape customers’ attitudes towards srt for eventually encouraging them to display ceb. our results reveal that customers’ attitude towards srt is shaped by one of the technology attributes (i.e., performance expectancy) (weerakkody et al., 2017). this result signifies customers put significant importance to the fact of whether technology in question offers a significant gain in their shopping task. therefore, retailers should concentrate on selecting those srt that can lower shopping hassle, and discomfort, and also can efficiently meet customer needs. possible ways to accomplish these objectives may include ensuring the increased coordination with the developers/testers of srt to synchronise the performance of srt with customer expectations (roy et al., 2018a). in addition to that, retailers may execute strategies to communicate srt capabilities to potential customers through product brochures, live demonstrations, and success stories (e.g., martin and herrero 2012; zuiderwijk et al., 2015). the results also signify that customers’ attitudes towards srt are also shaped by the contextual factors (i.e., facilitating conditions and social influence). interpreting further reveals customers attribute high importance to facilitating conditions such as intelligent systems linked with srt and as well as to the experiences of other individuals in using the technology. hence, retailers should consider offering high-grade infrastructural facilities such as virtual shelves, interactive screens, and in-store mobile apps (fan et al., 2020). also, assistance/training centres in the premises may act as one of the important drivers to encourage customers' attitudes towards srt (sabherwal et al., 2006; chiu et al., 2012). smart retailers may proactively engage in creating social influence with a capacity of exerting influence on customers. this engagement may include arranging forums/panels for exchanging best use practices or instituting advocates who are inspired to use srt which eventually can generate positive word-of-mouth, and handle negative feedback (pynoo et al., 2011). our results reveal that the technology characteristics influence customers' perception of technology attributes (performance expectancy and effort expectancy) and contextual factors (social influence and facilitating conditions). in the context of srt, performance expectancy is influenced by all three srt characteristics examined in the study (i.e. novelty, effectiveness, and interaction quality). also, effort expectancy and facilitating conditions are influenced by effectiveness and interaction quality. hence, retailers should consider adopting those srt that have a novel stimulus, quality interaction, and are effective. 7.3 limitations and future research directions 7.3.1 limitations despite its vital contributions to literature, the study has several limitations. first, while several proposed hypotheses are supported, the study reveals the low explanatory power of the structural model. the low explanatory power may be attributed to the wide variety of srt examined in the research model. secondly, the present study used only the meta-utaut (dwivedi et al., 2019) as an overarching framework to understand ceb towards smart retailing. third, the present study used a cross-sectional design. the present study examines the determinants of ceb towards srt by adopting a unidimensional view of ceb. australasian journal of information systems roy, singh & shabnam 2021, vol 25, research on user involvement modelling customer engagement behaviour 19 furthermore, the present study controlled for the effects of customer demographics in understanding ceb towards smart retailing. 7.3.2 future research directions this research sets the direction for researchers to explore the scope to accommodate other significant psychological constructs, i.e. motivation. future studies may use alternate behavioural theories such as behavioural reasoning theory (brt) (westaby, 2005; claudy et al., 2015) since brt based models are known for high prediction powers in terms of the variance explained in the dependent variables (sahu et al., 2020). as srt may differ in terms of characteristics (wünderlich et al., 2015), future research should, therefore, consider the research model for specific srt such as smart displays, smart shopping carts, or nfc-based systems and evaluate how differently it influences behavioural outcomes. future studies can adopt a longitudinal design for a more insightful understanding of the ceb (bijmolt et al., 2010). the present study examines the determinants of ceb towards srt by adopting a unidimensional view of ceb. the multidimensional conceptualisation of ceb (roy et al., 2018b; jaakkola and alexander, 2014) can add to the existing insights. since the age group and gender may influence ceb towards technology, future research could examine their role in ceb for srt. 8 conclusion the analysis of customer attitude and behaviour towards srt considering the srt characteristics and the given context of customer affective responses captures the backdrop of ceb toward srt. the results highlight the mediating role of attitude and behavioural intention towards srt to enhance ceb. the noteworthy findings emerging from this research demonstrates that ceb towards srt can be achieved via carefully considering srt characteristics such as novelty, effectiveness and interaction quality. given the importance of engaging customers, for managers of smart retailing, this study specifies the significance of services characteristics to influence customers’ affective responses positively. references adapa, s., fazal-e-hasan, s. m., makam, s. b., azeem, m. m., & mortimer, g. (2020). examining the antecedents and consequences of perceived shopping value through smart retail technology. journal of retailing and consumer services, 52, 101901. adams, j. s. (1963). wage inequities, productivity and work quality. industrial relations: a journal of economy and society, 3(1), 9-16. ajzen, i. (1991). the theory of planned behaviour. organisational behaviour and human decision processes, 50(2), 179–211. ajzen, i. (2011) the theory of planned behaviour: reactions and reflections, psychology & health, 26:9, 1113-1127. alalwan, a. a., dwivedi, y. k., & rana, n. p. (2017). factors influencing adoption of mobile banking by jordanian bank customers: extending utaut2 with trust. international journal of information management, 37(3), 99-110. ángeles oviedo-garcía, m., muñoz-expósito, m., castellanos-verdugo, m. and sanchomejías, m. (2014), "metric proposal for customer engagement in facebook", journal of research in interactive marketing, 8(4), 327-344. australasian journal of information systems roy, singh & shabnam 2021, vol 25, research on user involvement modelling customer engagement behaviour 20 attié, e., meyer-waarden, l., & bachié, e. (2020). consumer acceptance and resistance factors toward smart retail stores. in chkoniya, v., madsen, a. o., & bukhrashvili, p. (ed.), anthropological approaches to understanding consumption patterns and consumer behaviour (pp. 119-136). igi global. http://doi:10.4018/978-1-7998-31150.ch007 australian retailers association (2020), retail technology trends shaping customer experience in 2020, https://blog.retail.org.au/newsandinsights/retail-technology-trends-shapingcustomer-experience-in-2020 bagozzi, r. p., & yi, y. (1989). the degree of intention formation as a moderator of the attitudebehaviour relationship. social psychology quarterly, 266-279. baishya, k., & samalia, h. v. (2020). extending unified theory of acceptance and use of technology with perceived monetary value for smartphone adoption at the bottom of the pyramid. international journal of information management, 51(10); 20-36. bapna, r., & umyarov, a. (2015). do your online friends make you pay? a randomised field experiment on peer influence in online social networks. management science, 61(8),1902-1920. balaji, m. s. (2014). managing customer citizenship behaviour: a relationship perspective. journal of strategic marketing, 22(3), 222-239. balaji, m. s., & roy, s. k. (2017). value co-creation with internet of things technology in the retail industry. journal of marketing management, 33(1-2), 7-31. baldus, b. j., voorhees, c., & calantone, r. (2015). online brand community engagement: scale development and validation. journal of business research, 68(5), 978-985. bejerholm, u., & eklund, m. (2007). occupational engagement in persons with schizophrenia: relationships to self-related variables, psychopathology, and quality of life. american journal of occupational therapy, 61(1), 21-32. bhattacherjee, a., & sanford, c. (2009). the intention–behaviour gap in technology usage: the moderating role of attitude strength. behaviour & information technology, 28(4), 389401. bijmolt, t. h., leeflang, p. s., block, f., eisenbeiss, m., hardie, b. g., lemmens, a., & saffert, p. (2010). analytics for customer engagement. journal of service research, 13(3), 341356. blythe, j. (1999), innovativeness and newness in high‐tech consumer durables, journal of product & brand management, 8(5), 415-429. book, l. a., tanford, s., montgomery, r., & love, c. (2018). online traveler reviews as social influence: price is no longer king. journal of hospitality & tourism research, 42(3), 445475. brown, s. a., & venkatesh, v. (2005). model of adoption of technology in households: a baseline model test and extension incorporating household life cycle. mis quarterly, 29(3), 399-426. brady, m. k., & cronin jr, j. j. (2001). some new thoughts on conceptualising perceived service quality: a hierarchical approach. journal of marketing, 65(3), 34-49. australasian journal of information systems roy, singh & shabnam 2021, vol 25, research on user involvement modelling customer engagement behaviour 21 brodie, r. j., hollebeek, l. d., jurić, b., & ilić, a. (2011). customer engagement: conceptual domain, fundamental propositions, and implications for research. journal of service research, 14(3), 252-271. carter, s., & yeo, a. c. m. (2016). mobile apps usage by malaysian business undergraduates and postgraduates. internet research, 26 (3), 733-757. caro, f., & sadr, r. (2019). the internet of things (iot) in retail: bridging supply and demand. business horizons, 62(1), 47–54. chin, w. w., & newsted, p. r. (1999). structural equation modeling analysis with small samples using partial least squares. statistical strategies for small sample research, 1(1), 307-341. chiu, h. c., hsieh, y. c., roan, j., tseng, k. j., & hsieh, j. k. (2011). the challenge for multichannel services: cross-channel free-riding behaviour. electronic commerce research and applications, 10(2), 268-277. chiu, y. t. h., lee, w. i., liu, c. c., & liu, l. y. (2012). internet lottery commerce: an integrated view of online sport lottery adoption. journal of internet commerce, 11(1), 68–80. chong, a. y. l. (2013). predicting m-commerce adoption determinants: a neural network approach. expert systems with applications, 40 (2), 523–530. claudy, m. c., garcia, r., & o’driscoll, a. (2015). consumer resistance to innovation—a behavioural reasoning perspective. journal of the academy of marketing science, 43(4), 528-544. collier, j. e., moore, r. s., horky, a., & moore, m. l. (2015). why the little things matter: exploring situational influences on customers' self-service technology decisions. journal of business research, 68(3), 703-710. cohen, s. (1988). psychosocial models of the role of social support in the etiology of physical disease. health psychology, 7(3), 269. davis, f. d. (1989). perceived usefulness, perceived ease of use, and user acceptance of information technology. mis quarterly, 13(3), 318–339. de keyser, a., verleye, k., lemon, k. n., keiningham, t. l., & klaus, p. (2020). moving the customer experience field forward: introducing the touchpoints, context, qualities (tcq) nomenclature. journal of service research, doi.org/10.1177/1094670520928390 dwivedi, y. k., rana, n. p., tamilmani, k., & raman, r. (2020). a meta-analysis based modified unified theory of acceptance and use of technology (meta-utaut): a review of emerging literature. current opinion in psychology, 36, 13-18. dwivedi, y. k., rana, n. p., jeyaraj, a., clement, m., & williams, m. d. (2019). re-examining the unified theory of acceptance and use of technology (utaut): towards a revised theoretical model. information systems frontiers, 21(3), 719-734. elek, e., miller-day, m., & hecht, m. l. (2006). influences of personal, injunctive, and descriptive norms on early adolescent substance use. journal of drug issues, 36(1), 147172. australasian journal of information systems roy, singh & shabnam 2021, vol 25, research on user involvement modelling customer engagement behaviour 22 evanschitzky, h., iyer, g. r., pillai, k. g., kenning, p., & schütte, r. (2015). consumer trial, continuous use, and economic benefits of a retail service innovation: the case of the personal shopping assistant. journal of product innovation management, 32(3), 459-475. fan, x., ning, n., & deng, n. (2020). the impact of the quality of intelligent experience on smart retail engagement. marketing intelligence & planning, (forthcoming). ferracuti, n., norscini, c., frontoni, e., gabellini, p., paolanti, m., & placidi, v. (2019). a business application of rtls technology in intelligent retail environment: defining the shopper's preferred path and its segmentation. journal of retailing and consumer services, 47, 184-194. fishbein, m., & ajzen, i. (1975). belief, attitude, intention, and behaviour: an introduction to theory and research. reading: addison-wesley publication company. fornell, c., & larcker, d. f. (1981). evaluating structural equation models with unobservable variables and measurement error. journal of marketing research, 18(1), 39-50. foroudi, p., gupta, s., sivarajah, u., & broderick, a. (2018). investigating the effects of smart technology on customer dynamics and customer experience. computers in human behaviour, 80, 271-282. graf-vlachy, l., buhtz, k., & könig, a. (2018). social influence in technology adoption: taking stock and moving forward. management review quarterly, 68(1), 37-76. gregory, j., (2016). the internet of things: revolutionising the retail industry. accenture strategy. retrieved from 〈https://www.accenture.com/gb-en/insight-internet grewal, d., & roggeveen, a. l. (2020). understanding retail experiences and customer journey management. journal of retailing, 96(1), 3-8. grewal, d., noble, s. m., roggeveen, a. l., & nordfalt, j. (2020a). the future of in-store technology. journal of the academy of marketing science, 48(1), 96-113. grewal, d., hulland, j., kopalle, p.k. (2020b) the future of technology and marketing: a multidisciplinary perspective. journal of the academy of marketing science. 48, 1–8 (https://doi.org/10.1007/s11747-019-00711-4 grewal, d., ahlbom, c. p., beitelspacher, l., noble, s. m., & nordfält, j. (2018). in-store mobile phone use and customer shopping behaviour: evidence from the field. journal of marketing, 82 (4), 102-126. hair jr, j. f., sarstedt, m., ringle, c. m., & gudergan, s. p. (2017). advanced issues in partial least squares structural equation modeling. sage publications. hair, j. f., black, w. c., babin, b. j., anderson, r. e., & tatham, r. l. (2006). multivariate data analysis (vol. 6). hair, j. f., sarstedt, m., ringle, c. m., & mena, j. a. (2012). an assessment of the use of partial least squares structural equation modeling in marketing research. journal of the academy of marketing science, 40(3), 414-433. henseler, j., ringle, c. m., & sarstedt, m. (2015). a new criterion for assessing discriminant validity in variance-based structural equation modeling. journal of the academy of marketing science, 43(1), 115-135. australasian journal of information systems roy, singh & shabnam 2021, vol 25, research on user involvement modelling customer engagement behaviour 23 hollebeek, l. d., conduit, j., & brodie, r. j. (2016). strategic drivers anticipated and unanticipated outcomes of customer engagement, journal of marketing management, 32 (5-6), 393-398. hollebeek, l.d., sprott, d.e., andreassen, t.w., costley, c., klaus, p., kuppelwieser, v., karahasanovic, a., taguchi, t., ul islam, j. and rather, r.a. (2019). customer engagement in evolving technological environments: synopsis and guiding propositions. european journal of marketing, 53 (9), 2018-2023. hossain, m. a., hasan, m. i., chan, c., & ahmed, j. u. (2017). predicting user acceptance and continuance behaviour towards location-based services: the moderating effect of facilitating conditions on behavioural intention and actual use. australasian journal of information systems, 21, 1-22. huang, t. l., & liao, s. (2015). a model of acceptance of augmented-reality interactive technology: the moderating role of cognitive innovativeness. electronic commerce research, 15(2), 269-295. ibis world (2018), retail trade in australia, https://www.ibisworld.com/au/retail-sector/ inman, j. j., & nikolova, h. (2017). shopper-facing retail technology: a retailer adoption decision framework incorporating shopper attitudes and privacy concerns. journal of retailing, 93(1), 7-28. islam, j.u. and rahman, z. (2016), "the transpiring journey of customer engagement research in marketing: a systematic review of the past decade", management decision, 54 (8), 2008-2034. iyengar, s. s., & lepper, m. r. (2000). when is choice demotivating: can one desire too much of a good thing?. journal of personality and social psychology, 79(6), 995-1006. jaakkola, e., & alexander, m. (2014). the role of customer engagement behaviour in value cocreation: a service system perspective. journal of service research, 17(3), 247-261. kabadayi, s., ali, f., choi, h., joosten, h. and lu, c. (2019), "smart service experience in hospitality and tourism services: a conceptualisation and future research agenda", journal of service management, 30(3), 326-348. kang, s. (2014). factors influencing intention of mobile application use. international journal of mobile communications, 12(4), 360-379. kapser, s., & abdelrahman, m. (2020). acceptance of autonomous delivery vehicles for lastmile delivery in germany–extending utaut2 with risk perceptions. transportation research part c: emerging technologies, 111, 210-225. karjaluoto, h., shaikh, a. a., saarijärvi, h., & saraniemi, s. (2019). how perceived value drives the use of mobile financial services apps. international journal of information management, 47, 252-261. kaushik, a. k., & rahman, z. (2015). an alternative model of self-service retail technology adoption. journal of services marketing, 29 (5), 406-420. kock, n. (2015). common method bias in pls-sem: a full collinearity assessment approach. international journal of e-collaboration (ijec), 11(4), 1-10. australasian journal of information systems roy, singh & shabnam 2021, vol 25, research on user involvement modelling customer engagement behaviour 24 kim, h. y., lee, j. y., mun, j. m., & johnson, k. k. (2017). consumer adoption of smart in-store technology: assessing the predictive value of attitude versus beliefs in the technology acceptance model. international journal of fashion design, technology and education, 10(1), 26-36. knoll, l. j., magis-weinberg, l., speekenbrink, m., & blakemore, s. j. (2015). social influence on risk perception during adolescence. psychological science, 26(5), 583-592. knote, r., janson, a., söllner, m., & leimeister, j. m. (2020). value co-creation in smart services: a functional affordances perspective on smart personal assistants. journal of the association for information systems. (forthcoming) liam o'connell (2020), global retail sales 2018-2022, statista, https://www.statista.com/statistics/443522/global-retail-sales/ leroi-werelds, s. (2019), an update on customer value: state of the art, revised typology, and research agenda, journal of service management, 30( 5), 650-680. lindell, m. k., & whitney, d. j. (2001). accounting for common method variance in crosssectional research designs. journal of applied psychology, 86(1), 114. lowry, p. b., & gaskin, j. (2014). partial least squares (pls) structural equation modeling (sem) for building and testing behavioural causal theory: when to choose it and how to use it. ieee transactions on professional communication, 57(2), 123-146. magner l (2016) fast fashion in australia. ibis world, melbourne marketing science institute. (2018). 2018-2020 research priorities. martin, h. s., & herrero, a. (2012). influence of the user’s psychological factors on the online purchase intention in rural tourism: integrating innovativeness to the utaut framework. tourism management, 33(2), 341–350. meuter, m. l., ostrom, a. l., roundtree, r. i., & bitner, m. j. (2000). self-service technologies: understanding customer satisfaction with technology-based service encounters. journal of marketing, 64(3), 50-64. müller-seitz, g., dautzenberg, k., creusen, u., & stromereder, c. (2009). customer acceptance of rfid technology: evidence from the german electronic retail sector. journal of retailing and consumer services, 16(1), 31-39. negash, s., ryan, t., & igbaria, m. (2003). quality and effectiveness in web-based customer support systems. information & management, 40(8), 757-768. nitzl, c., roldan, j.l. and cepeda, g. (2016), "mediation analysis in partial least squares path modeling: helping researchers discuss more sophisticated models", industrial management & data systems, 116( 9), 1849-1864. novak, t. p., & hoffman, d. l. (2019). relationship journeys in the internet of things: a new framework for understanding interactions between consumers and smart objects. journal of the academy of marketing science, 47(2), 216-237. o’brien, h., & cairns, p. (2015). an empirical evaluation of the user engagement scale (ues) in online news environments. information processing & management, 51(4), 413-427. australasian journal of information systems roy, singh & shabnam 2021, vol 25, research on user involvement modelling customer engagement behaviour 25 overby, j. w., & lee, e. j. (2006). the effects of utilitarian and hedonic online shopping value on consumer preference and intentions. journal of business research, 59(10-11), 11601166. park, y. j., heo, p. s., & rim, m. h. (2008). measurement of a customer satisfaction index for improvement of mobile rfid services in korea. etri journal, 30(5), 634-643. pardamean, b., & susanto, m. (2012). assessing user acceptance toward blog technology using the utaut model. international journal of mathematics and computers in simulation, 1(6), 203–216. patil, p., tamilmani, k., rana, n. p., & raghavan, v. (2020). understanding consumer adoption of mobile payment in india: extending meta-utaut model with personal innovativeness, anxiety, trust, and grievance redressal. international journal of information management, 54, 102-144. pavlou, p. a., liang, h., & xue, y. (2006). understanding and mitigating uncertainty in online environments: a principal-agent perspective. mis quarterly 31(1), 105-136. peng, s., yang, a., cao, l., yu, s., & xie, d. (2017). social influence modeling using information theory in mobile social networks. information sciences, 379, 146-159. petter, s., straub, d., & rai, a. (2007). specifying formative constructs in information systems research. mis quarterly, 31 (4), 623-656. podsakoff, p. m., mackenzie, s. b., lee, j. y., & podsakoff, n. p. (2003). common method biases in behavioural research: a critical review of the literature and recommended remedies. journal of applied psychology, 88(5), 879. precourt, g. (2016). how is consumer engagement reshaping marketing? journal of advertising research, 56(1), 2–3. pöyry, e., parvinen, p., mattila, o., & holopainen, j. (2020). engaged, but with what? objects of engagement in technology-aided b2b customer interactions. journal of marketing management, 36(3-4), 334-360. pynoo, b., devolder, p., tondeur, j., van braak, j., duyck, w., & duyck, p. (2011). predicting secondary school teachers’ acceptance and use of a digital learning environment: a cross-sectional study. computers in human behaviour, 27(1), 568–575. qin, l., kim, y., hsu, j., & tan, x. (2011). the effects of social influence on user acceptance of online social networks. international journal of human-computer interaction, 27(9), 885-899. ringle, c. m., wende, s., & becker, j. m. (2015). smartpls 3. boenningstedt: smartpls gmbh. http://www. smartpls.com. roggeveen, a. l., grewal, d., & schweiger, e. b. (2020). the dast framework for retail atmospherics: the impact of in-and out-of-store retail journey touchpoints on the customer experience. journal of retailing, 96(1), 128-137. roy, s. k., balaji, m. s., quazi, a., & quaddus, m. (2018a). predictors of customer acceptance of and resistance to smart technologies in the retail sector. journal of retailing and consumer services, 42, 147-160. australasian journal of information systems roy, singh & shabnam 2021, vol 25, research on user involvement modelling customer engagement behaviour 26 roy, s. k., balaji, m. s., soutar, g., lassar, w. m., & roy, r. (2018b). customer engagement behaviour in individualistic and collectivistic markets. journal of business research, 86, 281-290. roy, s. k., balaji, m. s., sadeque, s., nguyen, b., & melewar, t. c. (2017). constituents and consequences of smart customer experience in retailing. technological forecasting and social change, 124, 257-270. roy, s. k., balaji, m. s., & nguyen, b. (2020a). consumer-computer interaction and in-store smart technology (ist) in the retail industry: the role of motivation, opportunity, and ability. journal of marketing management, 36(3-4), 299-333. roy, s. k., shekhar, v., quazi, a., & quaddus, m. (2020b). consumer engagement behaviours: do service convenience and organisational characteristics matter?. journal of service theory and practice, 30(2), 195-232. roy, s. k., singh, g., hope, m., nguyen, b., & harrigan, p. (2019). the rise of smart consumers: role of smart servicescape and smart consumer experience co-creation. journal of marketing management, 35(15-16), 1480-1513. sabherwal, r., jeyaraj, a., & chowa, c. (2006). information system success: individual and organisational determinants. management science, 52(12), 1849–1864. sahu, a. k., padhy, r. k., & dhir, a. (2020). envisioning the future of behavioural decisionmaking: a systematic literature review of behavioural reasoning theory. australasian marketing journal (amj )in press. sarstedt, m., ringle, c. m., cheah, j. h., ting, h., moisescu, o. i., & radomir, l. (2020). structural model robustness checks in pls-sem. tourism economics, 26 (4), 531-554 shaw, n., & sergueeva, k. (2019). the non-monetary benefits of mobile commerce: extending utaut2 with perceived value. international journal of information management, 45, 44-55. shin, d. h. (2015). quality of experience: beyond the user experience of smart services. total quality management & business excellence, 26(7-8), 919-932. soper, d.s., 2014. a-priori sample size calculator for structural equation models〈 http://www.danielsoper.com/statcalc talukder, m., & quazi, a. (2011). the impact of social influence on individuals' adoption of innovation. journal of organizational computing and electronic commerce, 21(2), 111135. tamilmani, k., rana, n. p., & dwivedi, y. k. (2020). consumer acceptance and use of information technology: a meta-analytic evaluation of utaut2. information systems frontiers, 1-19. taylor, s., & todd, p. a. (1995). assessing it usage: the role of prior experience. mis quarterly, 19(2), 561–570. tehseen, s., ramayah, t., & sajilan, s. (2017). testing and controlling for common method variance: a review of available methods. journal of management sciences, 4(2), 142-168. tenenhaus, m., vinzi, v. e., chatelin, y. m., & lauro, c. (2005). pls path modeling. computational statistics & data analysis, 48, 159–205. australasian journal of information systems roy, singh & shabnam 2021, vol 25, research on user involvement modelling customer engagement behaviour 27 vahdat, a., alizadeh, a., quach, s., & hamelin, n. (2020). would you like to shop via mobile app technology? the technology acceptance model, social factors and purchase intention. australasian marketing journal (amj), in press. valenzuela, s., park, n., & kee, k. f. (2009). is there social capital in a social network site?: facebook use and college students' life satisfaction, trust, and participation. journal of computer-mediated communication, 14(4), 875-901. vazquez, d., dennis, c., & zhang, y. (2017). understanding the effect of smart retail brand– consumer communications via mobile instant messaging (mim)–an empirical study in the chinese context. computers in human behaviour, 77, 425-436. van doorn, j., lemon, k. n., mittal, v., nass, s., pick, d., pirner, p., & verhoef, p. c. (2010). customer engagement behaviour: theoretical foundations and research directions. journal of service research, 13(3), 253-266. venkatesh, v., morris, m. g., davis, g. b., & davis, f. d. (2003). user acceptance of information technology: toward a unified view. mis quarterly, 27(3), 425-478. venkatesh, v., thong, j. y., & xu, x. (2012). consumer acceptance and use of information technology: extending the unified theory of acceptance and use of technology. mis quarterly, 36(1), 157-178. venkatesh, v., thong, j. y., chan, f. k., hu, p. j. h., & brown, s. a. (2011). extending the twostage information systems continuance model: incorporating utaut predictors and the role of context. information systems journal, 21(6), 527–555. viswanathan, v., sese, f. j., & krafft, m. (2017). social influence in the adoption of a b2b loyalty program: the role of elite status members. international journal of research in marketing, 34(4), 901-918. vivek, s. d., beatty, s. e., & morgan, r. m. (2012). customer engagement: exploring customer relationships beyond purchase. journal of marketing theory and practice, 20(2), 122-146. vrontis, d., thrassou, a., & amirkhanpour, m. (2017). b2c smart retailing: a consumerfocused value-based analysis of interactions and synergies. technological forecasting and social change, 124, 271-282. wang, s. m., & lin, j. c. c. (2011). the effect of social influence on bloggers' usage intention. online information review. 35 (1), 50-65. wang, r. j. h. (2020). branded mobile application adoption and customer engagement behaviour. computers in human behaviour. https://doi.org/10.1016/j.chb.2020.106245 wells, j. d., campbell, d. e., valacich, j. s., & featherman, m. (2010). the effect of perceived novelty on the adoption of information technology innovations: a risk/reward perspective. decision sciences, 41(4), 813-843. weerakkody, v., el-haddadeh, r., al-sobhi, f., shareef, m. a., & dwivedi, y. k. (2013). examining the influence of intermediaries in facilitating e-government adoption: an empirical investigation. international journal of information management, 33(5), 716– 725 australasian journal of information systems roy, singh & shabnam 2021, vol 25, research on user involvement modelling customer engagement behaviour 28 weerakkody, v., irani, z., kapoor, k., sivarajah, u., & dwivedi, y. k. (2017). open data and its usability: an empirical view from the citizen’s perspective. information systems frontiers, 19(2), 285-300. weijters, b., rangarajan, d., falk, t., & schillewaert, n. (2007). determinants and outcomes of customers' use of self-service technology in a retail setting. journal of service research, 10(1), 3-21. westaby, j. d. (2005). behavioural reasoning theory: identifying new linkages underlying intentions and behaviour. organisational behaviour and human decision processes, 98(2), 97-120. white, a., breazeale, m., & collier, j. e. (2012). the effects of perceived fairness on customer responses to retailer sst push policies. journal of retailing, 88(2), 250-261. willems, k., smolders, a., brengman, m., luyten, k., & schöning, j. (2017). the path-topurchase is paved with digital opportunities: an inventory of shopper-oriented retail technologies. technological forecasting and social change, 124, 228-242. wu, h. c., & cheng, c. c. (2018). what drives experiential loyalty toward smart restaurants? the case study of kfc in beijing. journal of hospitality marketing & management, 27(2), 151-177. wünderlich, n. v., heinonen, k., ostrom, a. l., patricio, l., sousa, r., voss, c., & lemmink, j. g. (2015). "futurising" smart service: implications for service researchers and managers. journal of services marketing, 29 (6-7), 442-447 yi, y., gong, t., lee, h., 2013. the impact of other customers on customer citizenship behaviour. psychology & marketing. 30 (4), 341–356. zhou, t., lu, y., & wang, b. (2010). integrating ttf and utaut to explain mobile banking user adoption. computers in human behaviour, 26(4), 760-767. zhao, x., lynch jr, j. g., & chen, q. (2010). reconsidering baron and kenny: myths and truths about mediation analysis. journal of consumer research, 37(2), 197-206. zuiderwijk, a., janssen, m., & dwivedi, y. k. (2015). acceptance and use predictors of open data technologies: drawing upon the unified theory of acceptance and use of technology. government information quarterly, 32(4), 429–440. appendix a: study instruments for measuring srt novelty 1. [srt] stimulates my curiosity 2. [srt] arouses my imagination 3. [srt] made me feel interested in it 4. [srt] incited my inquisitiveness effectiveness 1. [srt] allows me to have a more efficient shopping experience 2. [srt] improves the productivity of my shopping experience 3. i find using a [srt] can help me to have a smoother shopping experience australasian journal of information systems roy, singh & shabnam 2021, vol 25, research on user involvement modelling customer engagement behaviour 29 4. using [srt] can improve my ability to easily complete a shopping trip. interaction quality 1. overall, i’d say the quality of my interaction with the [srt] is excellent 2. i would say that the quality of my interaction with [srt] is high performance expectancy 1. i gain time using [srt] 2. [srt] optimises my shopping. 3. [srt] allows me to make my shopping quicker. 4. shopping will be more efficient using [srt] effort expectancy 1. learning to use [srt] is easy. 2. it’s easy to use [srt]. 3. it’s easy to use the [srt] skilfully. 4. i do not have any doubts about what i’m doing when i’m using the [srt] social influence 1. my friends and peers expect me to use [srt] in my shopping 2. people who influence my behaviour think that i should use [srt] 3. i find [srt] trendy. 4. the use of [srt] gives me professional status. 5. my social environment expects me to use [srt] in my shopping 6. people who are important to me think that i should adopt [srt] 7. most people i know using [srt] frequently facilitating conditions 1. i have all the necessary resources to use [srt] during shopping. 2. i have the know-how to use [srt] 3. if i have any doubts about how to use the [srt], i can get help from the retail store. 4. if i have any doubts about how to use the [srt], i do have a store employees and manager to help me attitude how would you describe your feeling towards using smart retail technology in the retail store? 1. unfavourable – favourable 2. i dislike it – i like it 3. bad – good customer engagement behaviour 1. i would like to introduce [srt] to others 2. i would speak favourably about the [srt] to others 3. i would say positive things about the [srt] to others. 4. i would assist other customers with the [srt] if they need my help australasian journal of information systems roy, singh & shabnam 2021, vol 25, research on user involvement modelling customer engagement behaviour 30 copyright: © 2021 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v25i0.2967 http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems goode 2020, vol 24, selected papers from the is foundations conference isf preface 1 preface to the special section on the 9th information systems foundations workshop sigi goode associate professor (information systems) australian national university sigi.goode@anu.edu.au it's difficult to believe that it's been 20 years since the first information systems foundations workshop. the workshop series was begun by kit dampney in 1999 at macquarie university. thereafter, the workshop has been held biennially at the australian national university. a number of people have come and gone in that time. however, it's heartening to note that the workshops have maintained their collaborative, collegiate and thoughtful approach to new work, new ideas and new scholars. in this, the 9th workshop, the focus was on the theoretical foundations of information systems at the social intersection. in particular, papers had to clearly address some theoretical body or bodies of knowledge in the information systems or a related domain, and the implications for practice of the theory or theories under consideration. we sought papers that are carefully argued, stimulating, speculative and controversial. we received papers on a variety of topics, from which a number were selected for presentation at the workshop. from these papers, three feature in this special section. in "explanations as discourse: towards ethical big data analytics services", sadaf afrashteh, ida someh and michael davern use the lens of discourse ethics to examine the use of explanations arising from big data analytics. in "the challenges involved in establishing a research technique", roger clarke identifies conceptual and methodological challenges and processes involved in the development of new research techniques. in "making is relevant in a connected world: revisiting the intellectual structures framework", henry linger and helen hasan revisit the intellectual structures framework against the backdrop of an ad-hoc, diverse is field. each of these works witnessed lively discussion at the workshop. the papers presented in this special section were accepted following a double-blind peerreview process. the workshop relies each year on the time and effort of a number of anonymous reviewers it is these anonymous reviewers who truly embody the vital spirit of the workshop. i am also grateful to the other program chairs, prof. john campbell and prof. kai riemer for their support, and to prof. shirley gregor for delivering the keynote address. finally, as many know, we lost our dear friend and colleague, prof. john campbell, not long after the isf workshop. john was a guiding light and a longstanding contributor to the isf workshop series over many years. we miss john terribly. copyright: © 2020 goode. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v24i0.2731 http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems goode 2020, vol 24, selected papers from the is foundations conference isf preface 2 paper title (paper title style) australasian journal of information systems slapnicar, vidmar & tsen 2025, vol 29, research article process theory of supplier cyber risk assessment received 07 jul 2024, accepted 02 mar  2025 after 3 rounds of review with humza naseer as accepting senior editor, m. davern & s. black as editors in chief 1 process theory of supplier cyber risk assessment sergeja slapnicar school of business university of queensland australia tim vidmar school of business university of queensland australia elinor tsen school of business university of queensland australia email: e.tsen@uq.edu.au abstract managing cyber risk in the supply chain represents one of the most significant challenges in cyber risk management. the paper studies how organizations assess supplier cyber risk. we used a mixed-method approach. we conducted 33 semi-structured interviews with cybersecurity experts from various organizations closely involved in supplier cyber risk assessments, as well as consultants. we complemented our qualitative findings by surveying 53 security experts about their supplier cyber risk assessment. based on the qualitative findings, we formulate a process theory of supplier cyber risk assessment. this theory explains how organizations assess supplier cyber risk and which contextual factors affect the maturity of cyber risk assessment and monitoring. the quantitative analysis supports the qualitative findings and suggests that the process can effectively identify risky suppliers. the paper sheds light on challenges and strategies associated with supply chain cyber risk assessment. the practical implications of our findings offer actionable insights for organizations seeking to enhance their cyber supply chain risk management. keywords: third-party cyber risk, supplier cyber risk, cyber supply-chain risk management, risk assessment, assurance. 1 introduction cyberattacks targeting suppliers1 have become increasingly common, and several prominent cases have garnered significant attention in recent years. a common belief is that major threats come from information and communication technology (ict) suppliers, such as the attack on solarwinds or the notpetya attack, in which cyber criminals gained access to major global corporations and government agencies, causing extensive operational disruptions and global 1 for clarity, we use the word supplier relating to various synonyms: supplier, which is an organization that enters into an agreement with the acquirer for the supply of a product or service, vendor, a commercial supplier of software or hardware, and a third-party provider who are service providers, system integrators, vendors, telecommunications, and infrastructure support that are external to an organization, as defined by the us national institute of standards and technology (nist): https://csrc.nist.gov/glossary/term/ https://csrc.nist.gov/glossary/term/ australasian journal of information systems slapnicar, vidmar & tsen 2025, vol 29, research article process theory of supplier cyber risk assessment 2 damage of tens of billions of dollars (alkhadra et al., 2021; crosignani et al., 2023; wolff, 2021). however, a severe cyberattack on an organization can also be launched through a micro provider, such as a heating, ventilation and air-conditioning vendor in the attack on target (simon and omar, 2020). in the analysis of 1,397 cyber incidents from 2000 to 2020, benaroch (2021) found that suppliers caused 18% of these incidents. the attacks on the supply chain (sc) were growing proportionally with other cyber incidents but compromised a larger volume of confidential data. an organization can suffer a cyber incident through a direct attack on a supplier; a supplier being a conduit for a cyberattack; when a supplier stores another partner’s data, i.e., as a partner-custodian; and when a supplier attacks the focal organization using privileged information (deane et al. 2023). cyber incidents in sc are not caused only by cyberattacks. noteworthy are also nonintentional risks committed by suppliers’ employees or faulty supplier hardware or software that can have a crucial effect on the focal organization’s provision of services (ghadge et al., 2019; urciuli et al., 2013; 2014). a case in point is crowdstrike’s failure, which affected 8.5 million microsoft users and caused a global outage (li, 2024). this incident highlights the increasing complexity of managing cyber risks in a sc, especially with global technology sourcing, system ownership, diverse legal jurisdictions, and heavy reliance on third parties for vital functions (boyes, 2015). these risks underscore the importance of the emerging discipline known as cybersecurity supply chain risk management (c-scrm) (boyson, 2014; khan and estay, 2015). recognizing which supplier is more likely to face disruptions because of a cyber incident is critical in effectively managing the impact of sc disruptions (boyson, 2014). however, according to the 2022 ponemon institute survey of over 1,100 security experts, approximately two-thirds of participants are uncertain about having a comprehensive inventory of suppliers with which they share information (the ponemon institute, 2022). only about one-third of respondents stated their organizations evaluate suppliers' security practices before establishing business relationships involving sensitive or confidential information sharing. while the dominant cyber security frameworks and standards such as nist, control objectives for information and related technologies (cobit 2019), the international organization for standardization and the international electrotechnical commission (iso), and banking prudential regulation address processes of managing cyber risks in the sc, they are high-level, require significant expertise, and are costly (for their overview, see supplement a). complying with standards is not fit for purpose for small and medium-sized organizations (smes) that do not have sufficient resources (ghadge et al., 2019) and for whom is also being assessed by such standards by multiple customers too costly. it is also unclear how supplier cyber risk assessment can be integrated into organizational-level cyber risk assessment, as guided by iso/iec 27005 and nist sp 800-30. most prior research addresses a diverse array of topics, predominantly analyzing technical solutions, standards and protocols (e.g., akinrolabu et al. 2019; caldwell 2015; hao & cai 2011), supplier cyber risk characteristics, or the financial impact of a cyber incident in a sc (benaroch 2021; crossignani et al. 2023). studies analyzing organizational approaches to cscrm are scarce due to data sensitivity and because frequently, even focal organizations do not have a good understanding of cyber risks beyond their direct (tier 1) suppliers (colicchia et al. 2019; gani et al. 2023; lewis et al. 2014; pandey et al. 2019). we identified only six such australasian journal of information systems slapnicar, vidmar & tsen 2025, vol 29, research article process theory of supplier cyber risk assessment 3 studies, and they highlight a significant lack of proper c-scrm (e.g., boyens et al. 2020; colicchia et al. 2019; gaudenzi & siciliano 2018; tran et al. 2016). up to this point, existing research has not focused on the supplier cyber risk assessment process, nor has it examined how contextual factors influence it. it is not well understood what process organizations employ to assess a supplier cyber risk, how contextual factors impact this process and how effective it is in identifying a risky supplier. our primary goal is to develop a process theory of supplier cyber risk assessment. we adopted the straussian grounded theory approach (corbin & strauss 1990), broadly based on two process theories – life-time and teleological theory. the former is concerned with the sequential steps that form a process leading to the attainment of organizational outcomes (mohr 1982; markus & robey 1988), and the latter with the goals of the process and process adaptability in response to resource constraints and environmental factors (van de ven & poole 1995; baskerville 2005). our methods of investigation are mixed. to formulate a process theory of cyber risk assessment, we conducted semi-structured interviews with 33 security experts in charge of cscrm in various countries. as qualitative findings cannot answer whether the process is effective, we complemented the interview findings with a survey of 53 security experts to establish whether the controls the process entails can identify riskier suppliers. our study makes two original contributions to the c-scrm literature: first, we define the process theory of supplier cyber risk assessment, which explains how organizations assess supplier cyber risk and which contextual factors affect the maturity of cyber risk assessment. the interview findings provide in-depth insights into how these practices are carried out in organizations, and the survey findings suggest that the assessment process can effectively identify riskier suppliers, especially by identifying their insufficient technological controls, the lack of supplier management support to cyber security risk management, the lack of supplier’s transparency and reputation, and by considering threat intelligence’s red flags. second, the practical implications of our findings offer actionable insights for organizations seeking to enhance their cyber supply chain risk management. 2 theoretical background 2.1 process theory to develop a grounded theory of a supplier cyber risk assessment process, we draw on process theories (langley 1999), which are at the core of theories of explanation (gregor 2006). process theories are frameworks that explain how organizational activities unfold over time, emphasizing the sequences of actions and interactions among actors. for our research questions, the combination of two process theories proved particularly relevant: a process theory that theorists van de ven and poole (1995) call life-cycle process theory and teleological process theory. life-cycle process theory suggests that some organizational phenomena cannot be explained by a determinate cause-and-effect relationship between variables but as the outcome of a sequence of necessary action steps (mohr 1982; markus & robey 1988). instead of pinpointing single predictors of outcomes, it explores the progression of activities over time and how complex interdependencies shape the final results. this perspective is particularly relevant in fields like information systems (is), organizational behavior, and management. cyber risk management is among the phenomena that cannot be explained with variance australasian journal of information systems slapnicar, vidmar & tsen 2025, vol 29, research article process theory of supplier cyber risk assessment 4 theories – that is, “more x leads to more y”, because risks are highly unpredictable and transient. rather, the logic in management of cyber risk is “if not x, then not y”, implying that the outcomes depend upon a complete chain of process activities (baskerville 2005). the second theory is teleological process theory (van de ven & poole 1995), which stresses that processes are designed as a trajectory to an organization’s intended goal. an organization develops a repetitive sequence of goal formulation, implementation, evaluation and modification of goals based on what it has learnt (van de ven & poole 1995, p. 516). however, while the theory stresses the purpose, it also recognizes that an organization is constrained by its resources and environment (contextual factors), which lead to its adaptability (bekmeierfeuerhahn 2009). in the context of cyber risk management, for instance, if the highest quality of controls cannot be achieved, it is desirable to have even poor quality controls in place to prevent some damage (baskerville 2005). the development of a cyber risk management system is cyclical and agile. in explaining organizational phenomena, these two theories have been combined: corporate strategic planning and the process of drug registration are examples that highlight the importance of the goal and adaptability and of following the sequential steps (chakravarthy & lorange 1991; nutt 2002). 2.2 supplier cyber risk factors one of the major questions in supplier risk assessment is which factors predict the likelihood and the consequences (impact) of a cyber incident related to a supplier. keskin et al. (2021) and dean et al. (2023) classify supplier cyber risk factors as (i) supplier cyber risk posture characteristics and (ii) the relationship between the supplier and the focal organization. ad i) supplier cyber risk posture characteristics that define its security posture are technology solutions (do amaral & gondim 2021; yeo et al. 2014); certifications (bartol 2014; sindhuja & kunnathur 2015; wolden et al. 2015); it governance, e.g., the presence of ciso, cio and/or ict manager roles and policies, and the alignment of the sc risk management with cyber risk management (liu et al. 2020; vanajakumari et al. 2021); industry affiliation (akinrolabu et al. 2019; hao & cai 2011; keskin et al. 2021), and an organization’s targeting factor which depends on the type of data it possesses (e.g., technological inventions or recipes). supplier characteristics also comprise a human factor (employee negligence, misinformation from leadership, inadequate training, and inappropriate access, sharing authentication credentials) (adams & makramalla 2015; boyson 2014; boyens 2015; collichia et al. 2019; kweon et al. 2021; tender, 2015). angst et al. (2017), giunipero and eltantawy (2004), and lewis et al. (2014) allude to organization size and financial resources, pointing out increased attacks targeting smes, often considered the weakest links in information security management. other important factors are geographical location (iovan and iovan 2016), foreign ownership (topping et al. 2021) and the complexity of a supplier’s sc (linton et al. 2014). ad ii) among factors related to the relationship between the supplier and the focal organization, keskin et al. (2021) suggested the nature of the it integration in the connectivity between the two firms. the risk is significantly greater if a supplier is an ict provider or its business is itbased, with most activities conducted online (benthall 2017). the second factor by keskin et al. (2021) is the type and magnitude of information sharing, which has been found critical for sc partners' preparedness to invest in network security (bandyopadhyay et al. 2010). in addition, highly specific services/products increase the criticality of the supplier to the focal organization (bode & wagner 2015; crosignani et al. 2023). australasian journal of information systems slapnicar, vidmar & tsen 2025, vol 29, research article process theory of supplier cyber risk assessment 5 while this research is highly instructive, much of it is normative (boyson 2014; davis 2015; lewis et al. 2014), focuses on technical solutions, or examines a single supplier risk factor. none of these studies have analyzed how these factors are collectively assessed in the supplier cyber risk assessment process or identified which factors are most critical for identifying risky suppliers. 2.3 organizational approaches to supplier cyber risk assessment in our literature review, we identified only six qualitative studies that investigated the organizational practices of c-scrm. we review them in this section. boyson (2014) studied two organizations and found that they assessed suppliers and formalized risk agreements but lacked centralized governance and integrated communication across functions. tran et al. (2016) studied supplier post-onboarding monitoring and noted that organizations rely on personal relationships rather than formal monitoring. gaudenzi and siciliano (2018) observed minimal cyber risk management of suppliers, focusing more on protecting clients' information. they did not observe practices of stipulating security requirements in the contractual agreements with key suppliers, considering disaster recovery and business continuity plans, or backing up sensitive data. however, they noted higher investments in cscrm and better risk management in finance and high-tech sectors. miscommunication and misaligned awareness between it and supply chain managers were also identified (siciliano & gaudenzi 2018). colicchia et al. (2019) found that while organizations required partners to comply with security and privacy policies, c-scrm initiatives were mainly reactive, focusing on response and recovery rather than proactive adaptation. requirements related to data management, it security tools, and operational resilience were enforced, but efforts rarely extended beyond tier 1 partners or included audits. business continuity and disaster recovery plans were adopted reactively. boyens et al. (2020) reported that organizations periodically surveyed suppliers to understand their security posture using tribal knowledge, self-assessment questionnaires, early risk assessments, government watch lists, supplier criticality scores, and nist csf assessments. suppliers were found to implement technical controls inconsistently, which focal organizations compensated for by training of supplier personnel and requirements of implementing controls in the future. they rarely monitored suppliers after onboarding. prior findings suggest that organizations employ deficient and unsystematic processes. while organizations may understand how their direct suppliers or customers interact with their information systems and manage their data, they frequently lack visibility into how these immediate partners source, transmit, and exchange data, services, and components with other companies upstream and downstream in the sc. while the reviewed studies are informative, none specifically focused on the process of supplier risk assessment and monitoring nor analyzed the contextual factors affecting its maturity. additionally, these studies investigated a rather small number of organizations, ranging from two to eleven, which calls for additional data to provide solid evidence about the state of play in c-scrm. this sets the stage for the investigation that is undertaken by the present study. 3 methods and data given the exploratory stage of this research area, we relied on the concurrent embedded mixed method design (creswell 2009). our primary method was qualitative – semi-structured australasian journal of information systems slapnicar, vidmar & tsen 2025, vol 29, research article process theory of supplier cyber risk assessment 6 interviews which explores the processes experienced by individuals in charge of supplier cyber risk assessment. it allowed us to gain an in-depth and holistic understanding of the research questions and conceptualize organizations' processes to assess a supplier's cyber risk. the secondary method – a quantitative analysis provided support to the primary method and enabled a statistical analysis of the question of how effective the process of supplier cyber risk assessment is in the identification of risky suppliers. concurrent design relates to the simultaneous data collection with interviews and a survey. the reason for the concurrent data collection is the significant challenge of recruiting interviewees in relevant positions willing to discuss such a sensitive topic. this process requires considerable time and effort. the data collection for the interviews lasted over a year, from march 2023 to june 2024. a survey was carried out simultaneously to ensure that the data related to the same time period and did not become outdated in this fast-developing discipline. we embed the findings of quantitative research within the findings of qualitative research (creswell 2009). 3.1 qualitative method 3.1.1 sample we conducted interviews with 33 cybersecurity experts, encompassing cybersecurity professionals, chief information officers (cios), chief information security officers (cisos), it managers, vendor managers, and consultants from various organizations, industries and countries (table 1). we recruited them via our professional network: isaca brisbane chapter and isaca slovenia chapter, women in cyber security (australia), and auscert (australia). among our 33 interviewees, 23 come from 18 for-profit and government organizations, and 10 are consultants providing supplier risk assessments to various clients. each interview lasted from 45 minutes to one hour. interviews were audio-recorded and subsequently transcribed using the zoom platform. in cases where consent for recording was not given, detailed notes were taken. the study has obtained ethical clearance from the authors’ university. all recorded videos were deleted after transcription and coding to safeguard the identity of participants and the confidentiality of the information. the interview data yielded invaluable insights into the multifaceted realm of cyber security risk assessment during the supplier selection process and after onboarding. participant # role experience in security (years) organization size (employees) industry country p1 principal analyst 25 15 consulting australia p2 it consultant 5 less than 10 consulting slovenia p3 ciso 25 300 employment & training australia p4 executive vice president >30 400 cloud security singapore p5 ciso 25 over 7,000 transportation (critical infrastructure) australia p6 discipline head of business resilience 15 over 7,000 transportation (critical infrastructure) australia p7 security sales specialist 4 over 1,000 consulting australia australasian journal of information systems slapnicar, vidmar & tsen 2025, vol 29, research article process theory of supplier cyber risk assessment 7 p8 cyber security consultant in governance, risk and compliance 2 over 1,000 consulting australia p9 cyber security consultant and founder 18 self-employed consulting australia p10 chief technology officer >20 50 software development new zealand p11 security analysts, specialist in 3rd party cs risk 2 over 2,500 consulting australia p12 security director 9 over 1,300 banking slovenia p13 chief it internal auditor 8 over 1,300 banking germany p14 cyber security architect 7 over 7,000 tertiary education australia p15 consultant, developer 7 less than 10 consulting australia p16 cio 7 over 200 banking slovenia p17 cyber security consultant and founder 20 self-employed + subcontractors cyber security and assurance company slovenia p18 cyber security consultant 15 over 200 vendor assessment australia p19 ciso 8 over 1,000 insurance slovenia p20 ciso of a subsidiary 19 over 3,000 banking austria p21 cyber security manager 20 over 2,000 municipality australia p22 ciso 20 over 5,000 insurance central europe p23 analyst of operational risk 6 over 5,000 insurance central europe p24 manager in support of it systems 14 over 5,000 insurance central europe p25 chief it officer 25 over 7,000 university hospital central europe p26 ciso 20 over 3,000 energy trading central europe p27 lawyer 5 over 3,000 energy trading central europe p28 ciso 23 over 700 banking slovenia p29 chief technology auditor 20 over 13,000 insurance australia p30 vendor risk manager 14 over 3,000 tertiary education australia p31 cyber security consultant 31 self-employed + sub-contractors cyber security and assurance consulting australia p32 head of it architecture and cyber security 15 approx. 60 primary and secondary education servicing over 700 schools australia p33 information security manager 15 about 500 financial, legal and it consulting australia australasian journal of information systems slapnicar, vidmar & tsen 2025, vol 29, research article process theory of supplier cyber risk assessment 8 n=33 consultants = 10 experts/managers in organisations = 23 organizations= 18 table 1: list of interview participants 3.1.2 data analysis the aim to develop a process theory of cyber risk assessment warranted the employment of the straussian variant of grounded theory (corbin & strauss 1990; glaser & strauss 2017), which focuses on the interplay between data and process theory, with a more structured coding process. the key features of this approach are the detailed procedure and three steps of coding (open, axial, and selective), as well as the emphasis on context and integration with the existing literature. adopting the approach of corbin and strauss (1990), the interview data was initially analyzed using open coding. in addition to the codes emerging from the data, we sought to relate to concepts from the theoretical background to guide the coding process. these concepts included supplier risk factors from prior c-scrm research, and concepts from the leading international cybersecurity frameworks (supplement a). data was uploaded to nvivo. two authors coded the interviews. the first author conducted all the interviews and coded them using the open coding method, while the second author played a supporting role in open coding and was intensively engaged in axial coding, in which we qualitatively and substantively compared the emerging codes, discussed and elaborated them, and organized related codes into second order codes and aggregate dimensions based on similarities, differences, and relationships between the codes (gioia et al., 2013). this method enabled us to systematically analyze qualitative data and identify key elements of the process. finally, we employed a selective coding process in which we unified all dimensions and categories in the process of supplier cyber risk assessment with the sequence of its elements and the contextual factors impacting the process (corbin & strauss 1990; mohr 1982; markus & robey 1988; baskerville 2005), presented in figure 1. we did not explicitly code against process theory. however, it broadly guided the search for the process’s sequential steps, contextual factors and the identification of the risk-based principle as the form of adaptability to constrained resources. we constructed a data structure of 126 first-order codes that are grouped into 38 second-order concepts, which are further grouped into 24 aggregated dimensions on four levels to conceptualize the process of supplier cyber risk assessment. table 2 presents the structure of the data (for the sake of brevity, we present only examples of first order codes). when constructing the process, we elevated some second order codes to higher dimensions based on the position they present in the process. level 1 dimensions level 2 dimensions level 3 dimensions level 4 dimensions second order codes first order codes selection process risk identification identification of suppliers shadow it open-source platforms lack of central procurement thresholds for onboarding a supplier risk analysis criticality of a supplier criticality to focal organization' s process concentration risk number of suppliers of the same service specificity of services/products provision of critical services australasian journal of information systems slapnicar, vidmar & tsen 2025, vol 29, research article process theory of supplier cyber risk assessment 9 annual spending for a supplier percentage of revenues spent data sensitivity the supplier hosts type of information (such as personal vs non-personal) personal, sensitive, nonpersonal jurisdiction in which data is stored foreign or domestic jurisdiction it integration nature of services/products or technology provided software-as-a-service supplier cyber risk assessment risk assessment methods general due diligence legal, financial, operational due diligence cyber security due diligence scanning of outwardlooking technology questionnaires self-made incorporating financial regulation templates by the government/sector log reviews supplier cyber risk posture characteristic s technology data protection (data storage management, encryption, cryptography frameworks) processes presence of the information security role people employee security awareness program regulation and industry standards data protection laws and regulations past incidents occurrence of a past incident complexity of supplier's supply chain multi-layered and complex supply chain foreign operations/ ownership/location foreign operations targeting factor government supply chain business characteristics financial health reputation customer's references assurance requirement of reports and certifications pen test reports no validation of information trust in suppliers' security employees evaluation determining supplier risk level acceptance of a supplier scoring (expert judgment, formulaic determination) score: low risk provisional acceptance scoring (expert judgment, formulaic determination) score: high to medium risk rejection of a supplier scoring (expert judgment, formulaic determination) score: high risk level 1 dimensions level 2 dimensions level 3 dimensions level 4 dimensions second order codes first order codes monitoring after onboarding risk mitigation uplifting a supplier's security posture collaboration with a supplier to improve their cyber posture discussions of necessary upgrades master agreements with timelines investments in security training of supplier personnel cost-benefit principle australasian journal of information systems slapnicar, vidmar & tsen 2025, vol 29, research article process theory of supplier cyber risk assessment 10 monitoring monitoring methods compliance with contractual requirements compliance with contractual requirements checked regularly face-to-face meetings with a supplier on-site visits obligation to report incidents and changes performance reviews requirement of reports penetration tests continuous monitoring with intrusive methods vulnerability scanning shared monitoring by a consortium of organizations shared on-site audit of the three largest cloud providers monitoring after termination of the contract checking for data being deleted frequency of monitoring depends on criticality contextual factors   regulation/industry critical infrastructure industry size large organizations vs smes it governance top leadership support efficiency (costs vs resources) costs note: next to the first-order codes emerging from the interviews, we also related to the concepts from the theoretical background (supplier risk factors from prior c-scrm research, and concepts from the leading international cybersecurity frameworks). the examples of codes from the c-scrm literature are the second order codes related to criticality of a supplier (e.g., specificity of products/services, annual spending for the supplier) and codes related to supplier cyber security posture (e.g., technology, people, processes, targeting factor). the codes related to leading international cybersecurity frameworks are level 2 dimensions (risk identification, analysis, evaluation, mitigation, monitoring). table 2: data structure 3.2 quantitative method 3.2.1 sample as our secondary method, we conducted a survey (the questionnaire is provided in supplement b). we recruited respondents with the assistance of isaca slovenia and isaca brisbane chapter australia, who posted invitations to participate in our research to their members. moreover, we promoted the survey to the participants of the auscert conference on gold coast, australia, in may 2023. we also invited individuals through our professional networks. we ended up with 71 completed questionnaires but excluded 18 because it was uncertain if they were involved in supplier cyber risk management (4 were academics and 14 were managers without specifying the area), resulting in 53 completed questionnaires. we collected responses from may 2023 till october 2023 via qualtrics. the respondents have, on average, 23 years of experience and 11 in cyber security; their median age is in the 35-44 years category, and the average sales revenues are in the 2-10 million eur category. they come from various countries (australia, china, and europe) and industries. the most frequent industries are finance and insurance (10 respondents) and it and other information services (7). australasian journal of information systems slapnicar, vidmar & tsen 2025, vol 29, research article process theory of supplier cyber risk assessment 11 3.2.2 data analysis the survey data was complementary to our interview data as the qualitative data does not allow us to analyze whether the process of supplier cyber risk assessment is effective, that is, whether it identifies risky suppliers. the small number of respondents allowed us to perform only descriptive analyses of association, such as analysis of variance and 2 tests. we analysed this question by comparing the mean differences in scores on controls between suppliers that suffered an incident vs those who did not. furthermore, we also evaluated the support of our qualitative findings (with anova and correlations). 4 process theory of supplier cyber risk assessment based on our findings, we developed a process theory of supplier cyber risk assessment presented in figure 1. the theory explains: 1. how organizations assess a supplier cyber risk, which supplier cyber risk posture characteristics are being assessed and how they are synthesized in an overall assessment of a supplier cyber risk level, 2. which contextual factors affect the maturity of cyber risk assessment and monitoring, and 3. whether the process is effective – does it identify risky suppliers? figure 1: process theory of supplier cyber risk assessment and monitoring. high lo w medium supplier risk level criticality of supplier criticality to focal organization‘s processes data sensitivity the supplier hosts it integration risk assessment methods supplier‘s cyber risk posture characteristics assurance supplier cyber risk assessment select but mitigate risk select a supplier avoid a supplier uplift supplier‘s security posture monitoring methods and frequency after onboarding selection process c o n te xt u al f ac to rs : re g u la ti o n , s iz e, i t g o v er n an ce , c o st -b en ef it risk analysis risk mitigation monitoring risk identification identify suppliers for cyber risk assessment evaluation likelihood impact supplier cyber risk posture technology processes people regulation and industry standards past incidents complexity of supplier's supply chain foreign operations/ ownership business characteristics targeting factor reputation australasian journal of information systems slapnicar, vidmar & tsen 2025, vol 29, research article process theory of supplier cyber risk assessment 12 we analyzed the first two questions using interview data (sections 4.1.-4.3.) and the last question using survey data (section 4.4.). the process of supplier cyber risk assessment consists of two broad phases: cyber risk assessment (i) in the selection process and (ii) after the onboarding of a supplier. ad i) supplier cyber risk assessment in the selection process starts with comprehensively identifying suppliers, followed by establishing a supplier’s criticality, which then determines the scope and the depth of supplier risk assessment. the scope is related to the number of assessment methods and the number of assessed suppliers' risk posture characteristics. the depth is related to the assurance of information. this is then followed by determining the supplier risk level that leads to the decision of whether a supplier will be onboarded. ad ii) cyber risk assessment after onboarding comprises uplifting a supplier’s risk posture if a medium risk supplier was onboarded and risk monitoring methods after onboarding. we then consider how this process fits to a general cyber risk assessment guided by iso 27005 and nist sp 800-30 and their phases of risk identification, risk analysis, evaluation, risk mitigation, and monitoring of cyber risks. in the following sections, we describe these phases in detail. 4.1 cyber risk assessment in the selection process 4.1.1 identification of suppliers one of the most challenging steps in the process of supplier cyber risk assessment is a comprehensive identification of suppliers. one challenge is the presence of shadow it, which in the past has involved employees installing applications without the awareness of the central procurement or the it department. now, shadow it has moved to cloud computing, where employees take data out of corporate applications and upload it onto free websites, like generative ai applications or cloud computing services. “i'm aware of one organisation putting the data of 2 million australians into some cheap cloud service, where some executive used a credit card doing some data analytics for $50 a month” (p31). mature organizations address this challenge by communicating the risk of exposing data to their employees or by blocking such actions technically. the lack of central procurement is another problem in the identification of suppliers. some organizations have policies in which only suppliers above certain thresholds are subject to due diligence, while those below may be onboarded by business units. risk assessors could find out about all suppliers by obtaining information about credit card payments from the accounting department. a third problem is open-source software components, which are not even billed. “one of the root causes developers are downloading components from other frameworks to add to their software is culture. i mean, most developers feel like they're under massive delivery time pressures2. so, it's better to borrow another component from somewhere else” (p31). requiring a software bill of materials, as mandated by the us federal government for software provided to the federal government (biden’s executive order on improving the nation’s cybersecurity, eo 14028, 2021), would contribute to the identification of all software components. however, we have not observed this practice in the organizations we analyzed. 2 more about the impact of time pressure on cybersecurity behaviours in chowdhury et al. (2019). australasian journal of information systems slapnicar, vidmar & tsen 2025, vol 29, research article process theory of supplier cyber risk assessment 13 4.1.2 criticality once suppliers are identified, organizations assess their criticality by considering the following three factors: (i) the criticality of a supplier to the focal organization’s business processes, defined by the concentration risk (number of suppliers of the same service that an organization relies on), specificity of services/products, and annual spending for a certain supplier; (ii) the data sensitivity the supplier hosts or has access to (type of data, management of data in its entire life-cycle, jurisdiction in which the data is stored); and (iii) it integration, which is associated with the nature of services, product and/or technology provided by a supplier, such as software-as-a-service (saas), infrastructure-as-a-service (iaas), or platformas-a-service (paas). in the eu banking sector, prudential regulation (the european banking authority) mandates a clear distinction between regular and critically important suppliers, with an assessment process that matches the level of criticality. consequently, banks follow the most rigorous processes of supplier criticality classification, formally linking it to business impact analysis. organisations generally rate suppliers using qualitative scores for criticality, such as critical, high, medium, and low. however, not all organizations have a defined process and key risk indicators in place and determine supplier criticality by a rule of thumb. organizations tend not to spend resources on suppliers of low criticality; our interviewees suggest that in some organizations, they are entirely left out of cyber risk assessment. the size of a prospective supplier is also considered − if it is small and does not have many resources and dedicated security staff, it won’t be able to complete an in-depth risk assessment process. 4.1.3 risk assessment methods our findings suggest that organizations primarily assess suppliers via three methods undertaken in different sequences by different organizations. not all organizations rely on all methods and the extent to which each method is applied varies based on the supplier’s criticality. general due diligence checks a supplier’s financial, operational and strategic risks, such as legal, geopolitical, environmental, social, and governance risks, the quality of service, and the supply chain complexity. this information is sought from a supplier or is based on its annual reports and public information. it is carried out by various organizational units (e.g., strategic procurement, risk management, legal or compliance department). "if an organization is financially non-viable, or non-viable in any other respect, it does not even get to be assessed for cyber risks but is rejected immediately” (p26). internal auditors might also be involved at this stage; however, mostly to ensure that in the focal organisation “all the right risk people or legal and compliance people have been involved and done their sign-offs … we review risk artefacts before they sign the contract. from our point of view, it doesn't make sense to review things after the arrangement has been done because then you can't change the contract.” (p29). a second method is cyber security due diligence based on public information about cyber risks. organizations double-check in the registers whether suppliers have security certifications. they rely on threat intelligence, providing information about past incidents and the activity on the dark web related to a prospective supplier, and scan the supplier’s outward-looking technology. some interviewed organizations (belonging to critical infrastructure) have the capability of a cloud access security broker (casb) or rely on a thirdparty provider: “we use public information prior to onboarding and after onboarding. the simplest procedure for that is qualys scanning of their vulnerabilities based on public australasian journal of information systems slapnicar, vidmar & tsen 2025, vol 29, research article process theory of supplier cyber risk assessment 14 information. we analyze a partner's cyber posture viruses, vulnerabilities, protocols that are public, everything that is possible to get publicly about the partner” (p20). the third method is to collect information via questionnaires, which cover a wide range of questions (from roughly 8 to 150) about supplier characteristics. questionnaires are subdivided into the assessment of the common core of controls, relevant for all suppliers, and supplements that depend on the service provided or supplier criticality. “for software companies, our questionnaire comprises 119 questions, for others only 8.” (p11). they are either self-made, based on guidelines, open-source standards, and regulation, imposed by a parent company; or provided, for example, by a government (e.g., australian signals directorate). while there might be a considerable variation in suppliers’ maturity of certain controls, the variation is not captured in most organisations as the checklists provide only yes or no answers. only the most mature organizations rely on a maturity scale for each question (among our interviewees, those were the banks). some organizations integrate questionnaires into a vendor management platform or rely on a specialized third-party provider. these providers possess the expertise and capability to perform the assessment more effectively than if an organization performs it in-house. the advantage is that third-party providers collect large amounts of information about organizations via online questionnaires, which they supplement with public information and information on past incidents, based on which they develop quantitative scores or even prediction models of a cyber incident. the second advantage is that they can also provide basic risk information for the suppliers beyond tier 1 (suppliers of suppliers). 4.1.4 supplier cyber security posture characteristics in assessing the supplier’s cyber security posture, our interviewees suggested that technology is a key characteristic of a supplier’s cyber security posture. technology questions cover all main controls related to protection, detection, response and recovery (the most important controls relate to data management, authentication and encryption methods, and identity and access management). however, the technological requirements and their checking widely differed among organizations and across suppliers: while some stated that technological requirements could not be too high, as they could then not find any supplier in a country, others were frustrated because they were unable to check the libraries within the software. most organizations impose mandatory requirements that would be a deal-breaker if a prospective supplier does not have them (e.g., a single sign-on). the importance of processes and people has also been stressed: “i'm less interested in the presence of controls than in the presence of an operating security function that selects controls that work.” (p21). regarding processes, focal organizations are looking for the presence of the information security role, whether suppliers adhere to any risk management framework, have privacy policies and documented processes, such as incident response and business continuity plans and how they would notify their partner about an incident or changes. p12 stressed that “most incidents in vendors are not related to malicious activity; it’s a failure of technology that occurs from an error. then continuity becomes very important.” regarding people, organizations look at suppliers’ attitudes towards protecting information, security awareness programs for their employees, and whether they perform vetting of employees involved in cyber security management. interviewees raised a concern that most companies fail to check their employees and provide security awareness programs only for australasian journal of information systems slapnicar, vidmar & tsen 2025, vol 29, research article process theory of supplier cyber risk assessment 15 their it staff. in some cases, focal organizations check suppliers’ employees’ and senior management’s background to verify their security qualifications and trustworthiness. regulatory requirements and industry standards are other critical factors determining supplier security posture. “many of our key suppliers are also captured by the critical infrastructure legislation. that creates a great deal of certainty. they need to produce a very detailed critical infrastructure management plan.” (p6). if a supplier is subject to privacy laws and the payment card industry standards and is from the critical industry, the regulation provides a sense of security to a focal organization. our respondents also acquire information about past incidents but have contrasting opinions about whether past incidents indicate future incidents or the lesser likelihood of another incident because a supplier has learnt the lesson (unless it had multiple breaches). the complexity of a supplier’s supply chain is another cyber risk posture characteristic, challenging to assess and manage. only one of our interviewees mentioned having engaged in the investigation beyond tier 1 suppliers: “i found that behind that third party provider, there are 12 other companies who manage the data of this company across new zealand, the us, australia, and the netherlands. it is not a supply chain; it is a multi-layer business network; one company provides infrastructure, other companies provide solutions, third companies provide hardware, and fourth companies try to manage it.” (p10). other interviewees indicated that while they are aware of their suppliers' subcontractors, they do not assess their cyber security. only if it comes to an incident, meaning that a subcontractor breaches the focal organization’s security requirements, the latter might request an on-site visit of the subcontractor. however, some vendor risk management platforms provide the basic risk scores of suppliers’ subcontractors, limited to the risks related to their outward-looking technology. foreign ownership/operations/influence are also a risk as countries have varying laws and regulations regarding data privacy, surveillance, and information control. some governments may impose strict regulations requiring companies to provide access to their data or compel them to store it within the country's borders. the supplier’s targeting factor depends primarily on its industry (e.g., critical infrastructure, financial, ict) and whether it is a government sc. the business characteristics of suppliers, such as financial health, size, age, and market share, were also suggested as important as if they are financially vulnerable and unstable, they can become vulnerable to a cyberattack. lastly, organizations assess the supplier’s reputation and customers’ and partners’ references. 4.1.5 assurance whether organizations require any assurance concerning the information provided depends on the supplier’s criticality. “it depends on the perceived risk how many assurance activities make sense or are economically viable. if there's no personal information involved, well, i'll treat that differently than when half a million personal information records are at risk. i'm gonna have stronger assurance built into the contract for the second one and might not need any for the first one.” (p21). only for critical suppliers, the interviewees require audit reports, pen tests, and/or cyber security certifications. however, not much detail might be disclosed to prospective customers: “in pen test reports, they will only give you the cover page which addresses any risks that were identified and how they were mitigated” (p30). also, the iso 27001 certification is a one pager that provides the focal organization only with proof that a prospective supplier meets the standard without any details. only the soc 2 report by the australasian journal of information systems slapnicar, vidmar & tsen 2025, vol 29, research article process theory of supplier cyber risk assessment 16 american institute of certified public accountants offers a detailed and comprehensive assessment of the effectiveness of controls. however, reports older than a year are already considered outdated. some organizations even have to verify that audits have been conducted by certified auditors. however, in most cases, organizations do not rely on validation, especially for smes and noncritical suppliers. they count on the supplier’s transparency and willingness to share information and evaluate the information’s credibility via partners’ references. most respondents stressed the importance of trust. however, several respondents shared that they caught a supplier lying about their cybersecurity certification: “i actually searched the iso 27001 database of certificates to see if i could find this certificate (of a supplier), and i couldn't. there is a lot of trust in that process, and there have been times when i've wanted to clarify their response to something, and they had said, oh, that was a mistake” (p11). 4.1.6 determining supplier cyber risk level the diverse information from the extensive questionnaire and a due diligence report is rather difficult to synthesize in an overall supplier’s risk level. organizations juxtapose the information acquired from the supplier with that from public sources and threat intelligence for consistency. ideally and in line with iso 27005, organizations would first evaluate a supplier’s criticality to adopt the proper scope and depth of the assessment process. criticality is associated with the impact of a cyber incident related to a supplier. in the second step, the organization would evaluate the likelihood of a supplier having a cyber incident based on its assessed risk posture. however, in practice, most organizations do not separately assess the impact and the likelihood but come up with an overall supplier’s risk level, which is determined as a subjective summary of numerous pieces of information, often qualitative and defined on a high, medium, and low scale. “how critical a certain supplier is a matter of the management subjective judgment in relation to which controls are missing and what is the relationship with a particular supplier” (p19). just one organization stated that it derives the risk level in a formulaic manner (taking a simple non-weighted average of the assessed characteristics). organizations that use vendor management platforms receive quantitative risk scores, yet there is limited explainability about how the scores are calculated. based on the assessed cyber risk level, the assessors (security experts) suggest to decisionmakers (executive managers) whether to accept a particular supplier. if the risk level is low, then a supplier is offered a contract specifying detailed cyber security requirements by the focal organizations. however, if the assessment indicates a medium risk level, most organizations do not outright reject a supplier but use the information as a snapshot of the supplier’s (missing) controls. they engage with a supplier to discuss further how to uplift its security posture. if the cyber risk level score indicates high risk, the assessors suggest rejecting the supplier to the management. however, they might not always be listened to. 4.2 monitoring after onboarding 4.2.1 uplifting a supplier’s security posture often, when a supplier is important to an organization on business grounds but does not have all the security requirements in place, focal organizations engage in uplifting a supplier’s cyber security posture. also, focal organizations have more tolerance if suppliers are not involved in critical processes. they tend to onboard such a supplier but impose requirements australasian journal of information systems slapnicar, vidmar & tsen 2025, vol 29, research article process theory of supplier cyber risk assessment 17 to implement missing controls in the master agreement with a specific timeline and a roadmap. they also provide support such as joint vulnerability assessment. “if someone is poorly assessed, it affects the overall risk assessment. if they are below our risk acceptance, they are contacted and looked at to see if there are any minor deficiencies, given a deadline for correction, and move on with the process” (p20). assisting suppliers in safeguarding the focal organization’s information can be mutually beneficial, as the expenses associated with remediation after a breach and the potential lack of resilience would likely surpass the costs of aiding a supplier in addressing any shortcomings. the focal organization and suppliers might also find it advantageous to collaborate on investments in staff training (davis, 2015; vanajakumari et al., 2021). 4.2.2 monitoring methods and frequency organizations should continuously or periodically monitor their suppliers’ security posture to overcome the relatively static nature of risk assessment and ensure suppliers remain in line with their security requirements (davis, 2015). monitoring can be periodic (annual, biannual or triannual), at the renewal of the contract, or based on a trigger, depending on the supplier’s criticality and the method of monitoring. our findings suggest a significant variation in the maturity of monitoring. for critical suppliers, organizations periodically check compliance with the contract, conduct face-to-face meetings, or require audit reports or pen tests. for non-critical suppliers, performance review and security requirements are mostly checked only at the renewal of the contract. the largest organisations are only just starting to invoke the right to audit. only the most mature organizations monitor critical suppliers continuously by integrating threat intelligence with the focal organization’s security information and event management system (siem). intrusive methods require direct contact with the supplier’s network and include network vulnerability scanning, log reviews, and prediction of cyber threats by machine learning methods (keskin et al., 2021; al-ansari & asubait, 2022). “so, we have threat management for our organization, we use the platform [name of the platform], and we wanna use the same kind of technology to keep an eye out for alerts when a hack happens in one of our vendors” (p33). the costs and the level of expertise associated with conducting these assessments are considerable. interestingly, nobody (but german banks) monitors the leading cloud providers because of the power imbalance between cloud providers such as microsoft, aws, and google vs. focal organizations. however, the chief it auditor of a german bank explained that the bank, in a consortium with other banks, conducts on-site audits of these systemically important cloud providers as this is required by german prudential regulation. “we are not competitors, and you also gain more power in the team. it includes the biggest german banks, the biggest insurance companies, and also, i think, the biggest italian banks.” (p13). other organizations mostly rely on public information and certifications about the largest cloud providers3. we came across some organizations that have a solid risk assessment process before onboarding but do not monitor a supplier after onboarding. “we're typically relying a lot more on the assessment than the actual contractual obligations at the moment. no follow-up of vendors later. we are developing a new policy here for vendors having protected data.” (p14). in contrast, other 3 the eu dora regulation stipulates that such providers will be monitored by the financial regulator itself because of their systemic importance. australasian journal of information systems slapnicar, vidmar & tsen 2025, vol 29, research article process theory of supplier cyber risk assessment 18 organizations have integrated their existing suppliers on their vulnerability scanning platform and conduct continuous monitoring but do not have a mature risk assessment process for suppliers before onboarding. the greatest challenge is ensuring that a supplier deletes the data as agreed and monitoring the deletion after the contract is terminated, especially in jurisdictions without regulations as strict as gdpr: “we are an organization highly dependent on third party providers. the first thing is really about getting breached by third parties, and the second risk, in my view, is about the fact that they keep our data for longer than we have agreed to beforehand. it's difficult to keep that overview unless you actually go to their organization and do an internal test whether they still have our data” (p33). 4.3 contextual factors the best way to understand how the process of supplier risk assessment is applied in organizations is by considering its maturity. boyson (2014) defines the maturity of c-scrm as emergent, that is, where no systematic risk assessment activities and no risk monitoring or digital forensics and reporting capacity take place, diligent, when organizations engage in selected risk assessment activities across the enterprise but have a limited capacity of risk monitoring, and proficient for extensive supply chain-wide risk assessment activities involving suppliers and customers and extensive capacity of monitoring. while we do not intend to classify our 18 organizations into specific maturity levels based on interviews, we have observed significant variation in the maturity of supplier cyber risk assessments along these levels – that is, which steps of the assessment they apply and how thorough they are. the variation is influenced by several contextual factors characteristics of the focal organization. one of our participating organizations stands out, and that is the german bank. the two contextual factors that differentiate the german bank from all other participating organizations are a combination of size and regulation, specifically, german prudential banking regulation. “we are doing on-site audits for all material partners. we do not rely on reports. i read the report before i start to understand the partner, but i also take my own sample testing, and i want to have a look at the whole documentation by myself and normally on-site.” (p13). both factors have implications for an organization’s risk culture and appetite (gale et al., 2022; monev, 2021), which in turn affect financial resources dedicated to c-scrm and the attraction of talent. larger organizations have a larger and more complex sc and are more exposed to cyber risk from the sc. correspondingly, they have a broader scope of assessment and assess more than just critical suppliers. in the financial industry, classification of suppliers, risk assessment and monitoring are subject to prudential oversight (e.g., eba guidelines on outsourcing arrangements, 2019). the eba guidelines, for example, require a manager managing outsourced arrangements to be in a senior management position, which significantly elevates the importance of this area and allocated resources. furthermore, the eu prudential requirements for the financial industry are considerably increasing (dora, 20234), and the affected participants stressed that they are already upscaling their capabilities to become compliant. 4 dora requires more validation checks, and stipulates the lead overseer over the critical ict service providers being one of the three eu financial authorities. australasian journal of information systems slapnicar, vidmar & tsen 2025, vol 29, research article process theory of supplier cyber risk assessment 19 however, in the case of the german bank, the crucial determinant of its proficiency is the pooling of resources within the industry. given the complexity of sc, the nature of various risks from suppliers (not just cyber but also, for example, geopolitical and esg risks) and the sheer size of cloud providers, pooling resources seems to be the most effective way to share not only costs of expensive risk assessment but also expertise and information. “we audit google, microsoft and aws every year in a team from europe with financial institutions and insurance companies, because they are under the same regulator, and we are visiting the big three american companies together. it's normally a lot of fun. it's 20 auditors.” (p13) regulation (e.g., security of critical infrastructure 2018 in australia, the network and information system directive 2022 in the eu) has also resulted in a typically higher maturity of the critical infrastructure, such as it, transport, energy, and healthcare, compared to other non-critical industries. while security is a top priority in the provision of critical services, in other industries, some interviewees acknowledged that cyber security is not the decisive factor in the selection of a supplier, but the price of a service or a product, even if such a cheaper supplier is more likely to pose a higher cyber risk. this is especially problematic in the public sector, where even if an organization belongs to critical infrastructure (in this study, a public hospital), it is subject to public tender legislation, which emphasizes the price and quality of service/product above other supplier selection criteria. while each contextual factor has an independent effect on the process, their interaction may boost (size and regulation) or downplay (critical infrastructure subject to public procurement legislation) the maturity of the process. regulated and large organizations dedicate more resources to it governance, characterized by strong top leadership support, clearly defined responsibilities, high hierarchical position of it and security roles, and experienced and knowledgeable security staff in charge of c-scrm. their agency leads to the continuous development of risk assessment and monitoring methods. one such good example of cyber security experts’ agency is a national approach to safer technologies in schools and pooling of resources in primary and secondary education in australia, where hundreds of state and territory schools rely on a common third-party risk management program completed by prospective vendors once and for all. this minimizes the duplication of work on both sides. merely participating in this program and completing the questionnaires provides security enhancement opportunities for both schools and vendors. for schools, it includes agreeing on a common set of privacy criteria and managing resources efficiently. vendors want to ‘look good’ as it opens up the market (p32). in many organizations, security roles have only recently been set up, and the supplier cyber risk process has yet to be fully established. the demand for experts exceeds the supply. organizations are also affected by significant fluctuations of experts, which can, on the one hand, have a positive effect via the mimicry of best practices but also a negative impact because of less-than-perfect transfer of knowledge. lastly, efficiency (the cost-benefit analysis) is the ultimate determinant of the maturity of the assessment process. our findings indicate that while the security experts are familiar with best practices and cyber security standards, the challenge is to comply with them due to constrained resources. a massive retail organization may have 10,000 suppliers subject to assessment and monitoring, requiring mammoth resources. allocation of resources to this area is proportionate to the perception of cyber risk exposure and the benefits of managing cyber risk. if cyber incidents from the sc have never been experienced and are perceived as australasian journal of information systems slapnicar, vidmar & tsen 2025, vol 29, research article process theory of supplier cyber risk assessment 20 highly uncertain and out of control, the benefits of allocating resources may also seem somewhat uncertain. in multi-billion dollar organizations, there are usually not more than one to three persons involved in assessments. although our interviewees demonstrated awareness of risks and how to manage them, they also expressed frustration with the extensive scope and risks’ uncontrollability. 4.4 effectiveness of the assessment process in identifying risky suppliers in this section, we present the survey findings. before presenting the findings about the effectiveness of the process in identifying risky suppliers, we triangulate the interview findings with those from the survey. we tested whether supplier criticality dictates the scope of the assessment process with the correlation analysis between criticality and the number of controls assessing supplier characteristics related to technology, people, processes and general information (see table 3). we find that if a supplier is critical to the focal organization’s operations, a significantly higher number of supplier controls is assessed related to technology and processes (but not to people and general information). however, we did not find support that suppliers with access to sensitive data are more scrutinized, partly perhaps because of the correlation between the involvement of a supplier in critical processes and their access to sensitive information. data sensitivity a supplier has access to criticality of processes a supplier is involved in data sensitivity a supplier has access to 1 .391** criticality of processes a supplier is involved in .391** 1 number of supplier’s controls assessed (total) 0.177 .360** n of supplier’s technology controls assessed 0.143 .364** n of supplier’s processes controls assessed 0.206 .326* n of supplier’s people controls assessed 0.099 0.251 n of supplier’s general information categories assessed -0.124 0.047 note: ** p < 0.01 (2-tailed), * p < 0.05 level (2-tailed). table 3: correlations between supplier’s criticality and the scope of risk assessment (the number of assessed controls) regarding the contextual factors, we do not find the size of the focal organization to be associated with the assessment scope. concerning it governance, if an organization has a cyber expert who assesses supplier cyber risk, then this is associated with a broader scope of assessment of supplier’s processes. however, we do not find the scope to differ for other areas. interestingly, we find that those organizations that use some sort of vendor management software assess fewer controls for technology and people but not for processes and general information. it might be that these platforms monitor other controls than those assessed in our questionnaire (they focus on outward-looking websites). among industries, organizations from the financial and ict industries were found to assess a significantly larger number of controls in all areas except for processes (table 4). the size of our sample and the nature of the variables did not allow us to assess a causality model and interactions among the contextual variables; however, the results, by and large, support the interview findings, except for size. australasian journal of information systems slapnicar, vidmar & tsen 2025, vol 29, research article process theory of supplier cyber risk assessment 21 note: we used analysis of variance (anova) to analyze whether the fact that an organization employs a specific expert for assessments of supplier cyber risk, uses vendor management software, and is from the financial or ict sector, affects the number of controls assessed (the scope of the assessment). the number of controls in the brackets for each area indicates the maximum number of controls considered in this study for that specific area, and bolded means with asterisks indicate which areas significantly differ between the two groups. ** p < 0.01 (2-tailed), * p < 0.05 level (2-tailed). table 4: anova analysis of the scope of assessment cyber expert who assesses supplier cyber risk n mean std. dev se mean vendor managem ent software n mean std. dev se mean fin/ ict sector n mean std. dev se mean supplier's characteristics total (n of controls =29) yes 37 25.5 5.00 0.82 yes 31 24.3** 6.13 1.10 yes 17 27.1** 3.02 0.73 no 13 23.4 5.95 1.65 no 21 26.6** 2.93 0.64 no 36 24.1** 5.71 0.95 supplier's technology yes 37 11.3 3.64 0.60 yes 31 10.5** 4.49 0.81 yes 17 12.4** 1.97 0.48 (n of controls =13) no 13 10.2 4.26 1.18 no 21 12.2** 1.70 0.37 no 36 10.5** 4.20 0.70 supplier's processes yes 37 2.9** 1.40 0.23 yes 31 2.6** 0.56 0.10 yes 17 3.2 1.29 0.31 (n of controls=4) no 13 2** 1.68 0.47 no 21 2.8** 0.40 0.09 no 36 2.5 1.56 0.26 supplier's people yes 37 2.7 0.52 0.09 yes 31 2.6 1.59 0.29 yes 17 2.8* 0.39 0.10 (n of controls=3) no 13 2.6 0.51 0.14 no 21 2.9 1.39 0.30 no 36 2.6* 0.55 0.09 supplier's general information yes 37 8.6 0.76 0.12 yes 31 8.6 0.56 0.10 yes 17 8.8* 0.44 0.11 (n of controls=9) no 13 8.6 0.65 0.18 no 21 8.7 0.80 0.17 no 36 8.5* 0.85 0.14 australasian journal of information systems slapnicar, vidmar & tsen 2025, vol 29, research article process theory of supplier cyber risk assessment 22 to analyze whether the risk assessment process can effectively identify risky suppliers we compared the assessed quality of controls between the suppliers that suffered a cyber incident and those that didn’t. the analysis is presented in tables 5 and 6. on eight out of 13 technology controls, suppliers without an incident scored significantly better than suppliers with an incident. this supports the interview findings that the assessment of technology is the most important area for understanding supplier cyber security posture and that weaknesses can be identified in a thorough assessment process. suppliers who haven’t experienced an incident were also better assessed on management attitudes towards protecting information (people), transparency and reputation (general information). in contrast to the interview findings, process controls were not significantly different. the descriptive statistics indicate that processes were perceived as well-established also for suppliers that had an incident. the survey also showed threat intelligence is paramount, as red flags were provided significantly more often for suppliers that suffered an incident than for those who did not (24% vs 3.6%). another noteworthy finding is that in our more detailed analysis (for brevity, not tabulated), those suppliers that the focal organization classified as less critical reported a cyber incident more frequently than those that were classified as more critical, but there was no difference in the number of incidents between suppliers that had access to sensitive data and those that do not. this suggests that contrary to current practices, organizations might need to pay more attention to suppliers that are classified as less critical, as they might be a weak point of access. overall, the findings from the survey suggest that the process of supplier cyber risk assessment can be effective in identifying risky suppliers. supplier controls incident n mean std. deviation std. error mean technology (1 min 4 max) commitment to secure-by-design practices yes 21 2.48 .981 .214 no 24 2.88 1.116 .228 use of secure coding practices yes 21 2.29* 1.007 .220 no 24 2.71* 1.042 .213 delivery of secure-by-default products and services yes 20 2.30 1.031 .231 no 24 2.54 1.215 .248 commitment to maintaining the security of their products and services yes 22 2.50 1.012 .216 no 25 2.76 1.052 .210 vulnerability disclosure policy yes 18 2.28 1.179 .278 no 25 2.72 1.173 .235 protected third-party access yes 18 2.22* 1.114 .263 no 25 2.76* 1.091 .218 use of encryption and cryptography yes 22 2.18** 1.053 .224 no 25 3.08** .954 .191 updated and patched software and servers yes 21 2.33** 1.017 .222 no 25 3.04** 1.060 .212 strong password systems supported with mfa yes 20 2.35* 1.226 .274 no 26 2.85* 1.084 .213 secure configuration of tools and firewalls yes 21 2.24** 1.091 .238 no 26 2.88** .993 .195 detection systems yes 21 2.29* 1.056 .230 no 26 2.81* 1.167 .229 physical security controls yes 22 2.41* 1.221 .260 no 25 2.88* 1.092 .218 people (1 min 5 max) australasian journal of information systems slapnicar, vidmar & tsen 2025, vol 29, research article process theory of supplier cyber risk assessment 23 how thoroughly does your chosen supplier perform employee background checks for those involved in cyber risk management yes 24 3.13 .900 .184 no 28 3.07 1.152 .218 how would you rate your chosen supplier's management attitudes towards the protection of information yes 25 2.56** 1.325 .265 no 28 3.18** 1.188 .225 general information (1 min 5 max) how complex is your supplier's supply chain? yes 25 3.00 1.041 .208 no 28 2.79 1.134 .214 how likely is your selected supplier subject to foreign ownership, operations or control? yes 25 2.72 1.242 .248 no 28 2.75 1.266 .239 how attractive is your chosen supplier as a target for cyber incidents? yes 25 3.60 1.000 .200 no 28 3.57 1.103 .208 what is your chosen supplier's financial health? yes 25 3.96 .978 .196 no 28 4.25 .967 .183 what is your chosen supplier's market share? yes 25 3.00 .816 .163 no 28 3.04 .922 .174 what is your chosen supplier's age? (1 min – 4 max) yes 23 2.87* .344 .072 no 26 2.65* .562 .110 what is your chosen supplier's size? (1 min – 4 max) yes 22 2.64* .727 .155 no 27 2.33* .832 .160 how would you assess your chosen supplier customers' references and reputation? yes 24 3.46* 1.062 .217 no 27 3.85* .949 .183 how would you assess the quality of information and transparency of your chosen supplier with regard to its cyber security risk management? yes 25 2.56** 1.083 .217 no 28 3.46** .838 .158 what kind of assurance does your chosen supplier provide with regard to its cyber security (e.g. penetration tests, assurance conducted by a third party, audit reports)? yes 18 3.33 1.283 .302 no 23 3.78 1.242 .259 note: variance analysis of supplier’s controls based on the notification that a supplier had an incident in the past 12 months (those suppliers are in the incident yes group, and those without an incident are in the incident no group). * p < 0.1, ** p < 0.05 (two-tailed) table 5: variance analysis of supplier’s controls based on received notification that a supplier had an incident in the past 12 months supplier controls (yes/no questions) incident frequency chi-square technology vulnerability and penetration testing performed yes 52.4% no 60.0% 0.241 processes documented incident response plan yes 75.0% no 94.1% 2.343 documented business continuity plan yes 82.4% no 83.3% 0.006 information security role in place yes 88.2% no 85.0% 0.082 security policies, frameworks, and documents yes 100.0% no 86.4% 2.511 people australasian journal of information systems slapnicar, vidmar & tsen 2025, vol 29, research article process theory of supplier cyber risk assessment 24 security awareness program for employees yes 93.8% no 85.7% 0.608 general red flags by threat intelligence yes 24%** no 3.6%** 4.8 note: we used the cross-tabulation analysis and chi-square estimator to assess whether the frequency of controls in suppliers without an incident significantly differs (incident no group) from those in suppliers who suffered an incident (incident yes group). ** p < 0.05 (two-tailed) table 6: cross-tabulation analysis of supplier’s characteristics based on received notification that a supplier had an incident in the past 12 months 5 discussion 5.1 theoretical contributions cyber security management studies are generally based on variance theories that suggest a determinate relationship between cause and effect; that is, more controls invariably lead to smaller cyber risks whereby the risks can be measured (baskerville, 2005). however, given the high unpredictability and uncontrollability of supplier cyber risks and challenges of its measurement, in formulating the process theory of supplier cyber risk, we rely on a different logical structure – that of life-cycle process theories, which propose that the outcome (in our case, the mitigation of supplier cyber risk) is contingent on a sequence of steps rather than on an expected relationship between the controls and the risk (mohr, 1982; markus & robey, 1988, van de ven & poole, 1995; baskerville, 2005). hence, if some steps in the process are missed, a risky supplier will not be identified. the process theories, however, do not suggest that the outcome is guaranteed: it might happen under certain conditions but not necessarily (markus & robey, 1988; p. 591). we identify the elements (steps) of the process of supplier cyber risk assessment and their sequential logic and study how one phase influences the next (figure 1). our quantitative findings support that the appropriate elements of the process were derived during the qualitative analysis and that assessing these elements would likely help an organization identify a risky supplier. in addition to identifying the sequential steps of the process, the teleological process theory adds the perspective that resource limitations and organizational environment constrain organizations in what can be accomplished (van de ven & poole, 1995). we considered this perspective when analyzing how contextual factors affect the maturity of the process. adapting the process’s scope and depth to supplier criticality demonstrates that organizations optimize their resources by following a risk-based principle, which focuses on identifying and mitigating the most pertinent risks first (lin & saebeler, 2019; pollmeier et al. 2022). for example, organizations with massive numbers of suppliers can only focus on critical suppliers, leaving many non-critical ones unassessed and non-monitored. “you can have hundreds of questionnaires, and you do not have resources to analyze those questionnaires; they are in excel or word documents. the first party reads it at a point in time, and then it gets it stored, but then it is never looked at again.” (p18). the adaptability of the process (to constraint resources) is a fundamental principle of teleological process theory (van de ven & poole, 1995; bekmeier-feuerhahn, 2009). we find that organizations that are more exposed to cyber risk and are larger have more resources and expand their assessment beyond just critical suppliers. they automate the process after a supplier is onboarded, invest more resources in ensuring that supplier australasian journal of information systems slapnicar, vidmar & tsen 2025, vol 29, research article process theory of supplier cyber risk assessment 25 information remains accurate and that a supplier maintains its security posture even after onboarding. compared to the findings of prior research (e.g., gaudenzi & siciliano, 2018; colicchia et al., 2019; boyens et al., 2020) who found that the majority of c-scrm initiatives are primarily associated with the it domain (technology) and that organizations are prioritizing protection measures over cyber risk assessment, we found that in the recent years, this process has rapidly progressed. still, most interviewees concur that their supplier risk assessment and monitoring process is a ‘work in progress’. while prior studies interpret the assessment of the suppliers’ it domain (technology) as early stages of maturity, we observe that most organizations have expanded beyond merely assessing this area. however, we also find that technology remains the most critical characteristic of the supplier's risk posture. a significant share of organizations is only at the low maturity of the described process – many have just started to assess the cyber risk of suppliers in the selection process but are struggling to catch up with monitoring hundreds or thousands of existing suppliers. some organizations rely on reduced assessment due to resource constraints, while in others, the assessment is lax as the assessment outcome is not decisive for the decision to onboard a supplier due to the lack of management support, which considers other factors more important. in sum, organizations repetitively review their achievable goals and strive to advance and modify their assessment processes in accordance with teleological process theory (van de ven & poole, 1995). by proposing a process theory of supplier cyber risk assessment, this study makes a theoretical contribution to the existing research on c-scrm by utilizing a perspective that hasn't been researched in previous studies that focused predominantly on normative guidance, technical solutions, or described practices in selected case studies. the fundamental theoretical contribution is the integration of life-cycle and teleological process theories in the context of c-scrm. dictated by constrained resources, this integration is achieved by focusing on suppliers' criticality and applying sequential steps in a risk-based manner. 5.2 practical contributions from our findings, significant practical implications arise. first, the most important concern is the lack of resources dedicated to supplier cyber risk assessment. pooling resources with other organizations using the same suppliers for cyber risk assessments and other operational and strategic risk assessments in the sc (e.g., esg) would be one way to optimize the available resources. another solution could be using vendor management platforms (keskin et al., 2021) and automation tools for monitoring onboarded suppliers. however, vendor management platforms should not operate as a black box – the scores need to be understood and aligned with other cyber risks related assessments if they are to affect business decisions. third, in most of the interviewed organizations, despite reliance on various methods and tools, translating a supplier’s information into risk score is a subjective expert judgment. the objectivity of assessments could improve if organizations started building a database of suppliers’ characteristics that would enable benchmarking and validation of the assessment methods by analyzing ex-post whether suppliers assessed with a higher/lower risk score were actually more/less risky. fourth, the likelihood of a supplier suffering a cyber incident and the likelihood that this incident would affect the focal organization are not perfectly correlated. the latter depends on australasian journal of information systems slapnicar, vidmar & tsen 2025, vol 29, research article process theory of supplier cyber risk assessment 26 the failure of other controls in the focal organization and what processes a supplier is involved in. this calls for better integration of supplier cyber risk assessment with the overall cyber risk assessment process in the focal organization in the light of iso/iec 27005 or nist special publication 800-30 (monev, 2021). our model in figure 1 illustrates how different steps in the supplier cyber risk assessment process can be integrated with the steps laid out in iso/iec 27005 or nist sp 800-30 for organizational cyber risk assessment – risk assessment (identification, analysis, evaluation), risk mitigation and monitoring. fifth, currently, international cybersecurity certifications are not fit for purpose for smes, and the assessments are tailored towards the ict technology providers. to get visibility across the entire sc, as suggested by one of our interviewees (p15), organizations could consider making suppliers accountable for their cyber security by getting self-attestation and acquiring a multitiered cybersecurity certification. instead of each organization assessing the risk, providing smes with the standards they can attest to would shift the accountability for cyber security to suppliers. currently, self-attestation services are only just developing. if such optimizations do not occur, a threat exists that the regulated industries subject to stringent assessment requirements will not be able to engage small and micro suppliers, who are unable to adhere to ever-increasing requirements. such an approach would provide better protection to a focal organization which could adopt a policy that all suppliers in its sc need some sort of certification, be it iso for the critical suppliers or anything else at some minimum level (p15: “black, brown and white belts, in the parlance of karate”). sixth, outsourcing cyber risk assessment would be another move towards optimizing resources and increasing the effectiveness of assessments. 5.3 limitations our findings are subject to some limitations: despite investing a great effort in attracting participants to the survey, we ended up with only 53 usable observations – some being excluded due to non-suitability. this has severely limited our analyses and the generalizability of the results. because of the niche expertise needed to participate in the study and the young phenomenon under investigation, the participants who were prepared to participate might have come from more mature organizations, and this relates to both survey and interview participants who were mostly from large, regulated, and resourceful organizations or consultants. the findings can thus not be generalized: while our participants already acknowledged the early stage of the development of the process, the findings might, in fact, show a better picture than it is on average. however, despite the limitations in data collection, we followed rigorous research methods to analyze the findings. 6 conclusion c-scrm and supplier cyber risk assessment offer ample opportunities for future research. qualitative and quantitative analyses could further examine the process and its determinants for the optimal development of cyber risk assessment given the context of an organization. a couple of ideas for future research are to validate the process framework and its contextual variables on a large sample, to further explore the role of processes and people characteristics, how they could be defined and measured to better differentiate between risky and less risky suppliers; to assess a predictive model of a supplier cyber risk; or to study the role and importance of supplier cyber risk assessment in the overall supplier assessment. australasian journal of information systems slapnicar, vidmar & tsen 2025, vol 29, research article process theory of supplier cyber risk assessment 27 references adams, m., & makramalla, m. (2015). cybersecurity skills training: an attacker-centric gamified approach. technology innovation management review, 5(5), 5–14. doi.org/10.22215/timreview/861 akinrolabu, o., nurse, j., martin, a., & new, s. (2019). cyber risk assessment in cloud provider environments: current models and future needs. computers & security, 87. doi.org/10.1016/j.cose.2019.101600 al-ansari, a. o., & alsubait, t. m. (2022). predicting cyber threats using machine learning for improving cyber supply chain security. in 2022 national computing colleges conference (nccc) (pp. 123–130). ieee. doi.org/10.1109/nccc57165.2022.10067692 alkhadra, r., abuzaid, j., alshammari, m., & mohammad, n. (2021). solarwinds hack: indepth analysis and countermeasures. in 2021 international conference on computing, networking and communications (icccnt). ieee. doi.org/10.1109/icccnt51525.2021.9579611 angst, c., block, e., d’arcy, j., & kelley, k. (2017). when do it security investments matter? accounting for the influence of institutional factors in the context of healthcare data breaches. mis quarterly, 41(3), 893–916. doi.org/10.25300/misq/2017/41.3.10 bandyopadhyay, t., jacob, v., & raghunathan, s. (2010). information security in networked supply chains: impact of network vulnerability and supply chain integration on incentives to invest. information technology & management, 11(1), 7–23. doi.org/10.1007/s10799-010-0066-1 bartol, n. (2014). cyber supply chain security practices dna: filling in the puzzle using a diverse set of disciplines. technovation, 34(7), 354–361. doi.org/10.1016/j.technovation.2014.01.005 baskerville, r. (2005). information warfare: a comparative framework for business information security. journal of information system security, 1(1), 23–50. benaroch, m. (2021). third-party induced cyber incidents—much ado about nothing? journal of cybersecurity, 7(1). doi.org/10.1093/cybsec/tyab020 benthall, s. (2017). assessing software supply chain risk using public data. in 2017 ieee 28th annual software technology conference (stc) (pp. 1–5). ieee. doi.org/10.1109/stc.2017.8234461 bekmeier-feuerhahn, s. (2009). mechanisms of teleological change. management revue, 20(2), 126–137. bode, c., & wagner, s. m. (2015). structural drivers of upstream supply chain complexity and the frequency of supply chain disruptions. journal of operations management, 36, 215–228. doi.org/10.1016/j.jom.2014.12.004 boyens, j., paulsen, c., bartol, n., winkler, k., & gimbi, j. (2021). key practices in cyber supply chain risk management: observations from industry. national institute of standards and technology. doi.org/10.6028/nist.ir.8276 boyens, j., paulsen, c., bartol, n., winkler, k., & gimbi, j. (2020). case studies in cyber supply chain risk management: summary of findings and recommendations. national institute of australasian journal of information systems slapnicar, vidmar & tsen 2025, vol 29, research article process theory of supplier cyber risk assessment 28 standards and technology. https://nvlpubs.nist.gov/nistpubs/specialpublications/nist.sp.800-161r1.pdf boyens, j., smith, a., bartol, n., winkler, k., holbrook, a., & fallon, m. (2021). cybersecurity supply chain risk management practices for systems and organizations. national institute of standards and technology. doi.org/10.6028/nist.cswp.02042020-1 boyes, h. (2015). cybersecurity and cyber-resilient supply chains. technology innovation management review, 5(4), 28–34. doi.org/10.22215/timreview/888 boyson, s. (2014). cyber supply chain risk management: revolutionizing the strategic control of critical it systems. technovation, 34(7), 342–353. doi.org/10.1016/j.technovation.2014.02.001 boyson, s., corsi, t., & paraskevas, j. (2022). defending digital supply chains: evidence from a decade-long research program. technovation, 118. doi.org/10.1016/j.technovation.2021.102380 boyson, s., corsi, t., & rossman, h. (2009). building a cyber supply chain assurance reference model. science applications international corporation (saic). boyson, s., corsi, t., rossman, h., & dorin, m. (2011). assessing scrm capabilities and perspectives of the it vendor community: toward a cyber supply chain code of practice. university of maryland robert h. smith school of business and national institute of standards and technology. caldwell, t. (2015). securing small businesses – the weakest link in a supply chain? computer fraud & security, 2015(9), 5–10. doi.org/10.1016/s1361-3723(15)30083-x chakravarthy, b. s./lorange, p. (1991): managing the strategy process. englewood cliffs. chowdhury, n. h., adam, m. t., & skinner, g. (2019). the impact of time pressure on cybersecurity behaviour: a systematic literature review. behaviour & information technology, 38(12), 1290–1308. doi.org/10.1080/0144929x.2019.1583769 colicchia, c., creazza, a., & menachof, d. a. (2018). managing cyber and information risks in supply chains: insights from an exploratory analysis. supply chain management: an international journal, 24(2), 215–240. doi.org/10.1108/scm-09-2017-0289 corbin, j. m., & strauss, a. (1990). grounded theory research: procedures, canons, and evaluative criteria. qualitative sociology, 13(1), 3–21. doi.org/10.1007/bf00988593 creazza, a., colicchia, c., spiezia, s., & dallari, f. (2021). who cares? supply chain managers’ perceptions regarding cyber supply chain risk management in the digital transformation era. supply chain management: an international journal, 27(1), 30–53. doi.org/10.1108/scm-02-2020-0073 crosignani, m., macchiavelli, m., & silva, a. (2023). pirates without borders: the propagation of cyberattacks through firms’ supply chains. journal of financial economics, 147(2), 432– 448. doi.org/10.1016/j.jfineco.2022.12.002 davis, a. (2015). building cyber-resilience into supply chains. technology innovation management review, 5(4), 19–27. australasian journal of information systems slapnicar, vidmar & tsen 2025, vol 29, research article process theory of supplier cyber risk assessment 29 deane, j., baker, w., & rees, l. (2023). cybersecurity in supply chains: quantifying risk. journal of computer information systems, 63(3), 507–521. doi.org/10.1080/08874417.2022.2081882 do amaral, t. m. s., & gondim, j. j. c. (2021, november). integrating zero trust in the cyber supply chain security. in 2021 workshop on communication networks and power systems (wcnps) (pp. 1-6). ieee. european banking authority (eba). (2019). guidelines on outsourcing arrangements. retrieved october 3, 2023, from https://www.eba.europa.eu/sites/default/documents/files/documents/10180/2551996/38 c80601-f5d7-4855-8ba3-702423665479/eba%20revised%20guidelines%20on%20 outsourcing%20arrangements.pdf gale, m., bongiovanni, i., & slapničar, s. (2022). governing cybersecurity from the boardroom: challenges, drivers, and ways ahead. computers & security, 121, 102840. doi.org/10.1016/j.cose.2022.102840 gani, a., fernando, y., lan, s., lim, m., & tseng, m. (2023). interplay between cyber supply chain risk management practices and cybersecurity performance. industrial management & data systems, 123(3), 843–861. doi.org/10.1108/imds-05-2022-0313 gaudenzi, b., & siciliano, g. (2017). just do it: managing it and cyber risks to protect the value creation. journal of promotion management, 23(3), 372–385. doi.org/10.1080/10496491.2017.1294875 gaudenzi, b., & siciliano, g. (2018). managing it and cyber risks in supply chains. in y. khojasteh (ed.), supply chain risk management: advanced tools, models, and developments (pp. 85–96). springer. doi.org/10.1007/978-981-10-4106-8_5 ghadge, a., weiß, m., caldwell, n. d., & wilding, r. (2019). managing cyber risk in supply chains: a review and research agenda. supply chain management: an international journal, 25(2), 223–240. doi.org/10.1108/scm-10-2018-0357 gioia, d. a., corley, k. g., & hamilton, a. l. (2013). seeking qualitative rigor in inductive research: notes on the gioia methodology. organizational research methods, 16(1), 15–31. doi.org/10.1177/1094428112452151 giunipero, l., & eltantawy, r. (2004). securing the upstream supply chain: a risk management approach. international journal of physical distribution & logistics management, 34, 698– 713. doi.org/10.1108/09600030410567478 glaser, b., & strauss, a. (2017). discovery of grounded theory: strategies for qualitative research. routledge. gregor, s. (2006). the nature of theory in information systems. mis quarterly, 30(3), 611–642. doi.org/10.2307/25148742 hao, j., & cai, w. (2011). trusted block as a service: towards sensitive applications on the cloud. in 2011 ieee 10th international conference on trust, security and privacy in computing and communications (pp. 73–82). ieee. doi.org/10.1109/trustcom.2011.13 healthcare & public health sector coordinating councils. (2019, october). healthcare industry cybersecurity supply chain risk management guide. private public partnership. retrieved from https://healthsectorcouncil.org/hic-scrim-v2/ australasian journal of information systems slapnicar, vidmar & tsen 2025, vol 29, research article process theory of supplier cyber risk assessment 30 international organization for standardization & international electrotechnical commission. (2021). iso/iec 27036-1: cybersecurity — supplier relationships part 1: overview and concepts. international organization for standardization & international electrotechnical commission. (2022). iso/iec 27036-2: cybersecurity — supplier relationships part 2: requirements. international organization for standardization & international electrotechnical commission. (2023). iso/iec 27036-3: cybersecurity — supplier relationships part 3: guidelines for information and communication technology supply chain security. iovan, ş., & iovan, a. a. (2016). cloud computing security. fiability & durability/fiabilitate si durabilitate, (1), 1(suppl.1), 206-212. isaca. (2018). control objectives for information and related technologies cobit 2019. retrieved from https://netmarket.oss.aliyuncs.com/df5c71cb-f91a-4bf8-85a6-991e1c2c0a3e.pdf khan o, estay s. d. (2015). supply chain cyber-resilience: creating an agenda for future research. technology innovation management review. 5,6–12. doi.org/10.22215/timreview/885 keskin, o. f., caramancion, k. m., tatar, i., raza, o., & tatar, u. (2021). cyber third-party risk management: a comparison of non-intrusive risk scoring reports. electronics, 10(10), 1168. doi.org/10.3390/electronics10101168 kweon, e., lee, h., chai, s., & yoo, k. (2021). the utility of information security training and education on cybersecurity incidents: an empirical evidence. information systems frontiers, 23, 1–13. doi.org/10.1007/s10796-019-09977-z langley, a. (1999). strategies for theorizing from process data. academy of management review, 24(4), 691–710. doi.org/10.5465/amr.1999.2553248 lewis, r., louvieris, p., abbott, p., clewley, n., & jones, k. (2014). cybersecurity information sharing: a framework for information security management in uk sme supply chains. li, f. (2024, july 24). microsoft-crowdstrike outage: how a single software update was able to cause it chaos across the globe. the conversation. retrieved from https://theconversation.com/microsoft-crowdstrike-outage-how-a-single-softwareupdate-was-able-to-cause-it-chaos-across-the-globe-235165 lin, w. c., and saebeler, d. (2019). risk-based v. compliance-based utility cybersecurity a false dichotomy? energy law journal, 40(2), 243–282. linton, j. d., boyson, s., & aje, j. (2014). the challenge of cyber supply chain security to research and practice – an introduction. technovation, 34(7), 339–341. doi.org/10.1016/j.technovation.2014.05.001 liu, c. w., huang, p., & lucas, h. (2020). centralized it decision making and cybersecurity breaches: evidence from u.s. higher education institutions. journal of management information systems, 37, 758–787. doi.org/10.1080/07421222.2020.1790190 markus, m. l., & robey, d. (1988). information technology and organizational change: causal structure in theory and research. management science, 34(5), 583–598. australasian journal of information systems slapnicar, vidmar & tsen 2025, vol 29, research article process theory of supplier cyber risk assessment 31 miller, a. r., & tucker, c. e. (2011). encryption and the loss of patient data. journal of policy analysis and management, 30(3), 534–556. doi.org/10.1002/pam.20590 mohr, l. b. (1982). explaining organizational behavior. san francisco, ca: jossey-bass. monev, v. (2021). the ‘self-assessment’ method within a mature third-party risk management process in the context of information security. 2021 ieee xx international scientific and technical conference (infotech). doi.org/10.1109/infotech52438.2021.9548373 national institute of standards and technology. (2020). sp-800-53r5: security and privacy controls for information systems and organizations. retrieved from https://nvlpubs.nist.gov/nistpubs/specialpublications/nist.sp.800-53r5.pdf nutt, p. c. (2002). why decisions fail: avoiding the blunders and traps that lead to debacles. san francisco, ca: berrett-koehler publishers. pandey, s., singh, r., gunasekaran, a., & kaushik, a. (2020). cybersecurity risks in globalized supply chains: conceptual framework. journal of global operations and strategic sourcing, 13(1), 103–128. doi.org/10.1108/jgoss-05-2019-0042 pollmeier, s., bongiovanni, i., & slapničar, s. (2023). designing a financial quantification model for cyber risk: a case study in a bank. safety science, 159, 106022. doi.org/10.1016/j.ssci.2022.106022 ponemon institute. (2022). the 2022 data risk in the third-party ecosystem study. retrieved from https://ponemonsullivanreport.com/2022/10/the-2022-data-risk-in-the-third-partyecosystem-study/ siciliano, g., & gaudenzi, b. (2018). the role of supply chain resilience on it and cyberdisruptions. in lamboglia, r., cardoni, a., dameri, r., & mancini, d. (eds.), reshaping accounting and management control systems (pp. 57–69). doi.org/10.1007/978-3-319-626369_4 simon, j., & omar, a. (2020). cybersecurity investments in the supply chain: coordination and a strategic attacker. european journal of operational research, 282(1), 161–171. doi.org/10.1016/j.ejor.2019.09.017 sindhuja, p. n., & kunnathur, a. s. (2015). information security in supply chains: a management control perspective. information & computer security, 23(5), 476-496. doi.org/10.1108/ics-07-2014-0050 slapničar, s., vuko, t., čular, m., & drašček, m. (2022). effectiveness of cybersecurity audit. international journal of accounting information systems, 44, 100548. doi.org/10.1016/j.accinf.2021.100548 slapničar, s., axelsen, m., bongiovanni, i., & stockdale, d. (2023). a pathway model to five lines of accountability in cybersecurity governance. international journal of accounting information systems, 51, 100642. doi.org/10.1016/j.accinf.2023.100642 tender, i. (2023). top 5 security threats from 3rd parties. network world. retrieved from https://www.networkworld.com/article/2991914/top-5-security-threats-from-3rdparties.html australasian journal of information systems slapnicar, vidmar & tsen 2025, vol 29, research article process theory of supplier cyber risk assessment 32 the european union digital operational resilience act (dora), articles 28–30. (2023). retrieved from https://www.digital-operational-resilience act.com/dora_articles_(proposal).html the ponemon institute. (2022). the 2022 data risk in the third-party ecosystem study. retrieved from https://ponemonsullivanreport.com/2022/10/the-2022-data-risk-in-the-third-partyecosystem-study/ topping, c., dwyer, a., michalec, o., craggs, b., & rashid, a. (2021). beware suppliers bearing gifts: analyzing coverage of supply chain cybersecurity in critical national infrastructure sectorial and cross-sectorial frameworks. computers & security, 108, 102324. doi.org/10.1016/j.cose.2021.102324 urciuoli, l., männistö, t., hintsa, j., & khan, t. (2013). supply chain cybersecurity – potential threats. information & security: an international journal, 29, 51–68. doi.org/10.11610/isij.2904 urciuoli, l., mohanty, s., hintsa, j., & boekesteijn, e. (2014). the resilience of energy supply chains: a multiple case study approach on oil and gas supply chains to europe. supply chain management: an international journal, 19. doi.org/10.1108/scm-09-2012-0307 vanajakumari, m., mittal, s., stoker, g., clark, u., & miller, k. (2021). towards a leader-driven supply chain cybersecurity framework. computers & security, 14, 42–52. van de ven, a. h., & poole, m. s. (1995). explaining development and change in organizations. academy of management review, 20(3), 510–540. doi.org/10.5465/amr.1995.9508080329 wolden, m., valverde, r., & talla, m. (2015). the effectiveness of cobit 5 information security framework for reducing cyberattacks on supply chain management systems. ifac-papersonline, 48(3), 1846–1852. doi.org/10.1016/j.ifacol.2015.06.355 wolf, j. (2021). how the notpetya attack is reshaping cyber insurance. brookings. retrieved from https://www.brookings.edu/articles/how-the-notpetya-attack-is-reshaping-cyberinsurance/ yeo, m., rolland, e., ulmer, j., & patterson, r. (2014). risk mitigation decisions for it security. acm transactions on management information systems (tmis), 5. doi.org/10.1145/2576757 acknowledgement we thank our reviewers, editor, and all our interviewees and survey participants for their time. we thank ryan ko for providing support in terms of contacts and funding, the organizers of the auscert conference, and, in particular, gary gaskell for promoting our study among the isaca brisbane members and at the auscert conference 2023 and 2024. we also thank uroš žust for promoting our study to the isaca slovenia participants and longxiao zhang to distribute our survey to participants in china. this work is a part of the research project entitled “a methodology and framework to assist organizations in managing cyber security requirements of their suppliers” that was supported by the university of queensland’s 2022 cyber security transdisciplinary research seed funding, grant number: 2022-r1 ko. australasian journal of information systems slapnicar, vidmar & tsen 2025, vol 29, research article process theory of supplier cyber risk assessment 33 copyright © 2025 slapnicar, vidmar & tsen. this an open-access article licensed under a creative commons attribution-non-commercial 4.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. https://doi.org/10.3127/ajis.v29.5323 https://creativecommons.org/licenses/by-nc/4.0/deed.en https://doi.org/10.3127/ajis.v29.5323 paper title (paper title style) australasian journal of information systems wigan 2020, vol 24, selected papers from the 2019 aice conference rethinking it professional ethics 1 rethinking it professional ethics: classical and current contexts marcus wigan oxford systematics mwigan@mwigan.com abstract professional computer ethics has widened its scope over the last 20 years as a direct result of the massive growth in computer mediated services by government and industry, and concerns over how data and interaction processes are recorded. these shifts are explored in conjunction with the parallel decline in community trust of government. the growing importance of a broader view and action framework for professional computer societies is delineated. keywords: ethics; trust; disintermediation 1 introduction there has been a steady shift from business-oriented computing, developed or deployed in a reasonably closed environment, to a far more open environment where a complex network of developers, users, and applications are involved. it has long been recognised that it professional ethics needed to be reviewed and respond to changes (bowern et al 2006). in the 20c environment it was reasonably clear who was responsible for the specification, development, and deployment of an application, and the target users were comparatively homogeneous. the general term ‘business computing’ communicated a flavour of automating existing tasks, and for a known and well-defined clientele, often within the same organisation. the leading edge of ai systems were then simple machine learning rule-based deductive processes, delivered as rule-based systems, rather than as adaptive systems with increasingly less-transparent deep-learning neural networks. the 21c growth of the internet not only expanded the scale and penetration of computer systems, but also stretched the links between an algorithm designer and the choices made for the final deployment to breaking point. by way of example, consider the implementation of underdeveloped simple rule-based systems such as centrelink’s robodebt program, with its flawed embedded reconciliation between ato data and centrelink conventions and automated action systems that assumed a debt existed (in 20% of cases it did not exist). 2 who is responsible for these continuing user interface issues for so many people? the managers of the deployment phase would be the first port of call, but there is no formal discipline-specific ethics for management for which they are required to sign up as a condition of professional practice. the ‘ethics statements’ for individual businesses tend to be used as risk avoidance assertions with generic value statements, but they could still potentially be used for disciplinary action when a violation affects the organisation as a whole. the ethical issues that might be raised are complex and underpinned by the very different priorities and perspectives of a for-profit operation. the priorities and interpretations in the public service have a different environment, and the complexities of political direction and individual australasian journal of information systems wigan 2020, vol 24, selected papers from the 2019 aice conference rethinking it professional ethics 2 execution are perhaps clearer than they are in a for-profit business, where survival and financial accountability are the over-arching and underpinning goals. failures in the for-profit environment are clearer than in the public service, as the goals in public service are inextricably bound up with the directions set by ministers. in a westminster system of government, a mistake by an employee is the final responsibility of the minister, and (used to be) grounds for a ministerial resignation. the length of the chain between algorithm creator and encoder and final policy-based deployment has now become too long to ensure values and ethical connections between those at the start of the chain (or indeed those eventually operating the resulting systems once developed), to the managerial and policy objectives of day to day delivery. they are however not new (wigan 1987), simply more extensive and pervasive. the classical approach to professional it ethics has been conditioned by an underlying and rarely questioned risk avoidance philosophy. the goal is to protect organisations rather than individuals, and to lay strictures on individuals’ actions and decisions (wigan, 1999). this approach was evident in the 1999 acs ethics task force survey of it professionals (two pages of which are shown as an appendix). 3 where are we now? the current environment, 20 years later, has shifted a long way. large scale it implementations are no longer restricted largely to corporate bodies but are distributed widely in both the public and private sectors. there has also been a pronounced shift from ‘data processing’ to decision making in such systems. perhaps the highest profile system has been the automated debt recovery system implemented by centrelink. this has been widely and pejoratively termed ‘robodebt’, due to the unintended negative impact and severe stresses caused by the algorithms embedded in it and the way in which the system had been designed to operate. the types of issues raised by such systems go well beyond the scope of the ethical issues summarised in the 1999 survey and raise questions about the end uses of the systems that it professionals are asked to create and operate. there is now a real need to expand on the scope and overage of professional ethics for the individuals within all computing fields, as the complexities of responsibilities at an organisational level have become increasingly blurred. it has become so universal that failures either in design and implementation of computing applications have become difficult to disentangle for the managerial utilisation and management of the operational tools that atsource computing professionals have been instructed to create. these questions are now of broad importance. multiple inquiries and reports attracting wide community interest on the usage and applications of machine learning (ml) (aka “artificial intelligence”) appear around the world. the responsibility for systems deployment remains with government or management. however, it professionals are now in the unique position of being able to see the possible outcomes of design and implementation decisions, and to express concern at the potential methods of deployment once completed. the generation of very large amounts of data, much of which is potentially reidentifiable, is also an issue or rising universality and importance to the community at large (wigan and clarke, 2013). australasian journal of information systems wigan 2020, vol 24, selected papers from the 2019 aice conference rethinking it professional ethics 3 the new generations of systems are designed to minimise personal contact from end users or targets, and thereby improve efficiency, but at the same time they make the expression of concerns a great deal harder. disintermediation of end user and designer is virtually complete. responsibility falls back on it professionals to raise concerns at the design stage. this occurs when and if a problematic issue is identified by the it professional that cannot be addressed internally and can only be addressed by his/her acting as a whistle-blower. in such cases extending professional computer ethics would open an additional channel for resolution without recourse to public whistleblowing. while such a move is clearly beneficial to all parties, it is only recently that such virtue-ethical approaches have become seriously considered by the professional ethics committees of computer societies, but this attention is now being given seriously. other approaches, deontological or consequential, are of course possible, but the focus of this desirable change in direction is on individual ethics and their extended enablement. ethics now plays a greater role in it , but covers too wide an area of actions and perspectives to be useful without greater focus (moss 2020) 4 governance issues it is equally clear that governance at the intersection of it, computing, and society is underdeveloped and that this is a major omission in current professional ethical frameworks1. the rapid rates of change make professional ethics an active rather than largely passive and rarely reviewed area of professional activity. this will only become more vital as computing as a whole becomes endemic and embedded in both technical machinery and business activity. the ability of it2 professionals of all kinds to create computing capacities that can be misapplied and abused is growing swiftly, and the current formal ethical frameworks, designed as they were primarily for reputational and professional protection, are simply no longer sufficient. risk assessment is no longer from the viewpoint of organisations but is moving into the hands or at least the scope of the professionals themselves. one of the signals that the public is sending is their dissatisfaction with constrained design and interaction capacities of automated interactive voice recognition ivr systems and websites with embedded rule structures that they are forced to use (mygov and the related multiple government websites -such as myagedcareare typical examples). anecdotally, insufficiently considered design and delivery is now much more widespread. the issues discussed so far might be structured as follows: • current ethical frameworks, • extensions of professional it ethics • governance changes 1 e.g. the governance and identity workstream in the current ieee standards association industry connections program: digital inclusion trust and agency (dita) initiative https://standards.ieee.org/industry-connections/diita/index.html) which has a governance and identity workstream convenor marcus wigan 2 the abbreviations it (information technology) and ict (information and communications technologies) are used interchangeably here as the contexts vary australasian journal of information systems wigan 2020, vol 24, selected papers from the 2019 aice conference rethinking it professional ethics 4 current ethical frameworks are beginning to shift. the recent whistle-blower protections limited though they are to the private sector3 , are a good start. the introduction of some form of formal protection of whistle-blowers is long overdue, but the omission of the public sector (and the effective limitation of the scope of these new powers within the many contractors while including regulated bodies) still omits huge areas of public concern. all of this is occurring when itc is rapidly becoming the means of primary delivery. outsourcing attenuates the governance chain only further. plausible deniability via this displacement of government responsibilities to the private sector adds a complex and contentious layer of political activities and responses, already obscured by the overreach of surveillance legislation. this interaction is complicating whistleblowing as the stripping of almost all previous encryption-protectable communications has already reduced the cover for the normal healthy role of journalists. as communications media, and indeed most fresh government services, are now almost exclusively ict-mediated, this adds a further layer of complication to current ethical behaviours, and retrospective identification of both content and identification of those with whom communication occurs is now a reality. the exemption of politicians and public servants from whistleblowing therefore now has a very ict resonance, and vastly increases level of personal risk. this is the current ethical environment. it is not one that encourages or supports ethical behaviours in or around the ict sector. the new territories are now data linkages, biometrics, and adaptive decision making. in each of these areas substantial impacts on people can be and are generated without any knowledge of their subjects, and the ethical concerns of those implementing them are real. 5 at which points are ethical decisions now needed? it is not clear who is responsible for ethical decision making, and at which points on the decision-chain they are situated. some candidates are: • privacy by design [ict] • authorisations for access and release [ict and management] • secret mass data flows from applications [all too common* , and in ict purview] • transparent and independent auditing of ml and algorithmic embeddings [ict] • security theatre. abuse of ‘national security’ to overwhelm all privacy and collateral impacts, objections and accountability [politicalbut ict is heavily involved] in each area ict ethics has a significant role to play. in the last area, the virtue ethics of individuals in ict arguably has a major role to play (wigan 2015) as it addresses the kind of person one is (stanford encyclopedia of philosophy, 2016). alternatives previously proposed for it include a form of consequentialism (moor, j. 1999). at first sight deontological ethics might be the most pertinent, as it draws from a stance that there are rules that can be applied to determine what we ought to do. taking a virtue ethics approach emphasises the ethical 3 treasury laws amendment (enhancing whistle-blower protections) act 2019. no.10, 2019. an act to amend the law in relation to whistleblowing, and for related purposes accessed on 16 august 2019 at https://www.legislation.gov.au/details/c2019a00010 australasian journal of information systems wigan 2020, vol 24, selected papers from the 2019 aice conference rethinking it professional ethics 5 values that resonate with an individual in his/her personal capacity and what kind of person they wish to be. this phrasing readily brings to mind the motivations often expressed by whistle-blowers as distinct from compliance with ethics statements of formal stated ethics procedures and risk assessments. why does all of this matter? it matters because the success of all such systems depends on trust. without trust, institutional and political credibility falls away, and distanced uses of data and automated decision reduce it further. trust is a function of individuals. supporting mechanisms to assist ict professionals to make ethical decisions, actions, and representations are now needed as they are at the heart of many ict developments and deployments. this has moved ict whistle-blowers into what has become a key public interest domain (snowden and boyle are simply two examples). 6 trust in government as a factor many of the most difficult areas are within government where ict is becoming almost the sole channel between citizens and government instrumentalities. but, do they now have the necessary trust? and will they have it in the future? the work of stoker et. al. at the university of canberra suggests that this is doubtful, and active steps will be needed to be taken on many sides in order to make it so. figure 1: after stoker et al 2018, 2019 one of the necessary steps has emerged from the most extensive inquiry into ai ethics to date, by the uk house of lords, which created a remarkable body of evidence and consideration. however, the most significant outcome was the response of the uk government, who australasian journal of information systems wigan 2020, vol 24, selected papers from the 2019 aice conference rethinking it professional ethics 6 concentrated on creating governance organisation and procedures in order to enable desirable outcomes (uk government, 2018. this was a response at a higher level than ‘simple’4 data protection and privacy, and it is a model that could and should be built upon. it is one of the planks on which trust in ict intermediated companies, governmental and corporations will increasingly come to depend. it is hard to see how public trust in either government – already under major threat – and the ict sector can be improved without such steps. the need to do so, or at least to address the visible decline, is strong (stoker, et al, 2018): a drop in democratic satisfaction in australia from 78% to 41% is a serious warning. the results are stark: “trust in key institutions and social leaders is eroding. by 2025, if nothing is done and current trends continue, fewer than 10 per cent of australians will trust their politicians and political institutions” (stoker et al, 2019). in the recent global pandemic, where ict has become critical in the infections tracing function, fukayama has asserted that the crucial determinant in performance [of the actions taken such as ict tracing] will not be the type of regime, but the state’s capacity and, above all, trust in government. (fukayama, f. 2020) the disintermediation role of ict, and the consequent need for trust, has suddenly become visibly critical at a global level. 7 the emergent role of professional computer societies there is however a complementary route that remains open in the absence of meaningful action by government, and that is to enhance the professional ethics activities and consequently more engaged member support within professional ict societies. this will need to involve both the australian computer society (acs) and the ieee (institute of electrical and electronic engineers) as the pervasiveness of internet of things systems, including automated vehicles, is growing. it has long been the case that the acs leadership has quietly intervened to resolve ethical conflicts for members, but the major shifts in society just outlined demand a move from a passive code of ethics to an active one. the moves to an ict mediated society are accelerating, with cash declining rapidly and automated systems for customer-facing services growing swiftly. the private sector faces increasing issues in personal data handling, release, and deidentification. while there is as yet no equivalent to the eu gdpr (pwc, 2017), its advent has precipitated moves towards better privacy and data handling regimes in australia. there are formal moves to address some aspects of data ethics – the uk asserts that “you must not proceed with your project if your data use is not proportionate to the user need” but few provide the active support for whistle-blower channels to address the long gaps between the it professional and the impact on the community at delivery. government data is a valuable commodity, but reliance on anonymisation is a key element yet this is often poorly recognised, and when an expert demonstrated how easily it could be deanonymized government moved to make such actions illegal and pressed for the party 4 as the impacts and interpretation into action of the eu gdpr has made very clear, any action in this area can never be ‘simple’, however admirable a first cut the gpdr might be and it is. australasian journal of information systems wigan 2020, vol 24, selected papers from the 2019 aice conference rethinking it professional ethics 7 concerned to leave her academic post. this example both illustrates that it professionals now have a real need for professional ethical support from their societies, and that virtue ethics is a valuable perspective on it ethics. the most recent developments in mass surveillance and social network data collection (required to trace coronavirus contacts) have catalysed many in the expert community to contest and press for transparency (and indeed technical improvements) in this large-scale government initiative to help to secure the trust required for its success. such trust is hard to create under the synoptic declines that were discussed earlier, and increasingly relies on individual motivations both within and without government to help the necessary trust to develop. such examples are unlikely to become less frequent as ict has come to disintermediate almost all large-scale interactions in the community, certainly with government at all levels but increasingly with commerce as well. the empowerment of it professionals through their professional organisations has become a key social contribution, and a further reason for professional societies to move quickly to encompass such constructive ethical action. acknowledgements the comments of the referees are acknowledged as contributing significantly to this paper. references bowern m., burmeister, o., gotterbarn, d. and weckert, j. (2006) ict integrity: bringing the acs code of ethics up to date. australasian journal of information systems 13 (3) 169-181. fukuyama, f. (2020) the thing that determines a country’s resistance to the coronavirus. the atlantic. march accessed on 29-3-20 at https://www.theatlantic.com/ideas/archive/2020/03/thing-determines-how-wellcountries-respond-coronavirus/609025/ haughey , m. (2013) the uncertainty of ethics in it. pp 62-70 in professionalism in the information and communication technology industry in weckert, j. and lucas, r. [eds] practical ethics and public policy monograph 3 series . anu epress [available at no charge from https://press.anu.edu.au/publications/series/practical-ethics-publicpolicy/professionalism-information-and-communication#pdf moor, j. (1999) just consequentialism and computing. ethics and information technology 1(1), 65-69. moss, e. and metcalf, j. (2020) too big a word. https://points.datasociety.net/too-big-a-word13e66e62a5bf (discussion covered in the forthcoming data and society report “ethics owners: a new model of organizational responsibility in data-driven technology companies”) . price waterhouse coopers (2017) gpdr fast facts. accessed at https://www.pwc.com.au/assurance/assets/gdpr-fast-facts18.pdf on 4-9-19 stanford encyclopedia of philosophy (2016) deontological ethics. accessed on 29-3-20 at https://plato.stanford.edu/entries/ethics-deontological/ australasian journal of information systems wigan 2020, vol 24, selected papers from the 2019 aice conference rethinking it professional ethics 8 stoker, g., evans, m. and halupka, m. (2018) trust and democracy in australia. democratic decline and renewal. report no 1. university of canberra (52pp). december accessed 16-8-19 at https://www.democracy2025.gov.au/documents/democracy2025-report1.pdf stoker, g., evans, m. and halupka, m. (2019) bridging the trust divide: lessons from international experience. report no 2. university of canberra. (13pp) accessed 16-8-19 at https://www.governanceinstitute.edu.au/magma/media/upload/publication/406_whodo-you-trust.pdf uk department for digital culture media and sport (2018) guidance: data ethics. accessed on 28-3-20 at https://www.gov.uk/government/publications/data-ethicsframework/data-ethics-framework weckert, j. (1999) survey of it professional’s ethical attitudes. pp1-2 of 4. school of information studies, charles sturt university wigan, m.r. (1987). legal and ethical issues in expert systems used in planning. environment and planning b (14) 305-321. wigan. m.r. and clarke, r.a. (2013) big data’s big unintended consequences. ieee computer 46(6),46-63. wigan, m.r. (2015) big data – can virtue ethics play a role? retrieved from https://works.bepress.com/mwigan/29/download 1-4-2020 uk government response to the house of lords ai select committee report on ai in the uk: ready willing and able. (2018) apgs cm9695 [ccs0618948288 978-1-5286-0607-3] 42pp weckert, j. (1999) survey of it professional’s ethical attitudes. pp1-2 of 4. school of information studies, charles sturt university copyright: © 2020 rethinking it professional ethics. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v24i0.2851 http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems miah, blake & kerr 2020, vol 24, research article meta-design knowledge for clinical decision support systems 1 meta-design knowledge for clinical decision support systems shah miah vu business school victoria university melbourne shah.miah@vu.edu.au jacqueline blake university of the sunshine coast queensland don kerr university of the sunshine coast queensland abstract knowledge gained from a decision support systems (dss) design should ideally be reusable by dss designers and researchers. the majority of existing dss research has mainly focused on empirical problem solving rather than on developing principles that could inform solution approaches for other user contexts. design science research (dsr) has contributed to effective development of various innovative dss artefacts and associated knowledge development, but there has been limited progress on new knowledge development from a practical problem context, going beyond product and process descriptions. for dss applications such as clinical decision support systems (cdss) design and development, relevant reusable prescriptive knowledge is of significance not only to understand mutability but also to extend application of theory across domains. in this paper, we develop new design knowledge abstracted from the approach taken in a representative case of innovative cdss development, specified as an architecture and six design principles. the cdss design artefact was initially designed for a specific clinical need is shown to be flexible for meeting demands of knowledge production both for diagnosis and treatment. it is argued that the proposed general strategy is applicable to designing cdss artefacts in similar problem domains representing an important contribution of design knowledge both in dss and dsr fields. keywords: dss, clinical dss, is theory, design science research, public healthcare 1 introduction the aim of this paper is to describe meta-design knowledge1 that is generated from an existing cdss project. over the past few decades research has been grown rapidly in cdss design for addressing many clinical and non-clinical decision-making issues. the majority of the research 1 in is research, the meta-design refers to a type of artefact targeting to meet the requirements of particular problem sets (walls, widmeyer, & el sawy, 1992). koehne, redmiles, and fischer (2011) described meta-design as a theoretical framework supporting for mapping individual users’ participations in system context. in this paper, new knowledge as a meta-design framework as a prescriptive knowledge (gregor and hevner, 2013) that we propose is to offer six design principles and a general design architecture to meet the requirements of particular problem sets for cdss design. australasian journal of information systems miah, blake & kerr 2020, vol 24, research article meta-design knowledge for clinical decision support systems 2 is aimed at innovative technology and approach design for the improvement of clinical or and non-clinical processes. for example, litvin et al. (2012) described a cdss design for requirements of enhancing primary-care practices in public health, while chang et al. (2016) proposed a cdss for improving clinical outcomes with adherence to evidence-based clinical guidelines. such cdss research only focuses on problem solving paradigm rather than highlighting the requirement of developing new knowledge that may be applicable to other problem contexts. while fischer and eden et al. (2004) wrote on meta-design knowledge and its reusability in is research, such ideas are rarely developed in the context of dss development and particularly so with direct relevance to cdss designers and practitioners. analysing the design trends of cdss literature we found that very few attempts have been made to develop new reusable theoretical knowledge from the design practices of existing cdss development projects. o’sullivan et al. (2014) discussed requirements of enhancing cdss evaluation theory (such as for achieving greater usability), chang et al. (2016) provided design guidelines that are based on the cognitive fit theory, and marcos et al. (2013) described theories of a sustainable platform of cdss for smooth integration of design components, for example, interoperability. this work used existing theories to encourage developing new design knowledge that could offer processes for better guiding cdss designers and researchers. it is imperative to explore the requirements for revealing knowledge that could be potentially reusable for designing and implementing cdss. therefore, reusable design knowledge can provide practical guidelines for designers and is researchers for similar cdss design. this has implications not only for improving practices for developing technical components but also to obtain appropriate guidelines to map the behavioural and organisational settings for developing innovative cdss artefacts. there has been growing attention to design science research (dsr) within the is community since nunamaker & chen (1990) first introduced this as an effective design method for contemporary is design. since then many authors have elaborated different issues of dsr to locate the it artefact in a theoretical context while retaining real-world relevance (orlikowski & iacono, 2001). significant attention has been paid to extend dsr in many directions and one of the directions is artefact development (hevner, march, park, & ram, 2004; march & smith, 1995; peffers, 2008), theoretical development (baskerville et al. 2018; gregor and hevner, 2013) and contextual development (mckay et al. 2012; carlsson, 2007). vom brocke et al. (2020) indicated that it is important to generate prescriptive knowledge2 from dsr studies, which is seen as a contribution to the is body of knowledge in terms of both theory and practices of solving real-work issues. identifying limitations for accumulation and evaluation of design knowledge in is, the authors (vom brocke et al. 2020) provided a framework to derive this prescriptive knowledge from dsr studies for addressing further practically relevant is design issues. there are other calls for is researchers to pay more attention to the extent of artefact design process and the different dimensions of artefact design (iivari, 2015). in a relatively recent design science research (dsr) special issue in the european journal of information systems, senior is researchers reinforced the importance of generating new design knowledge from dsr artefacts (baskerville, kaul & storey, 2018b). de leoz and petter (2018) 2 gregor and hevner (2013) stated that prescriptive knowledge can be design knowledge that describes principles of form and function, methods, constructs and justificatory understanding that may be used to develop an artefact. australasian journal of information systems miah, blake & kerr 2020, vol 24, research article meta-design knowledge for clinical decision support systems 3 eventually indicated “we encourage is design science researchers to think beyond the creation and evaluation of the technical components of the it artefact and also to consider the social impacts of the it artefact.” (page. 156). this provides reinforcement for researchers to develop meta design knowledge from our practical cdss design issues. the dsr methodology has grown in importance for dss solution design (arnott & pervan, 2014a) in general, although in designing specialised dss such as cdss solution artefacts was less in evidence. the increased application of dsr is positive in as much as it produces useful guidelines for developing innovative solution artefacts to address both human-oriented and organisational unsolved problems (hevner et al., 2004). in this paper, we extend the design understanding around process development to offer better guidance for designing specific artefacts. while addressing the criteria of gregor and hevner (2013) regarding generality to a class of problems and the meta-requirements that specify the goal class to which the theory applies. for developing relevant design knowledge we operationalise both the dsr theoretical statements by gregor and hevner (2013) and by baskerville et al. (2015) for positioning understanding that was inductively generated from a specific design case. adopting hevner’s et al. framework of dsr we describe meta-design3 knowledge for the problem class of diagnostic and treatment recommending cdss, using an existing sleep-disorder cdss project in which an innovative practitioner-specific cdss artefact was developed (reference removed). we identified generic and essential design problems and based on abstracted requirements we developed a theoretical understanding and model through designing and evaluating the solution artefact for the particular decision support context. this paper is structured as follows. the next section describes background literature and the problem case that we initially identified for designing the cdss solution within an interactive practitioner context. the subsequent section defines the methodological details and the section after that provides the details of meta-design knowledge that we developed and modelled from the cdss design case. the final section discusses the contribution, validation and applicability of the emergent theoretical understanding for similar cdss designs along with future research directions. 2 background clinical decision support systems (cdss) is one of the well-recognised sub-areas of information systems research. cdss is “any computer program designed to help health professionals [and patients living in their communities] make clinical decisions and [nonclinical decisions]’’ (marcos et al., 2013, p. 676). it artefacts are inevitably embedded in their organisational settings (orlikowski & iacono, 2001) and such specialised artefacts do not only assist practitioners by providing patient-specific recommendations but can also be a system that uses clinical data to support decision making in patient care (marcos et al., 2013). while hevner et al. (2004) suggested that practices of designing it artefacts must be based on 3 markus et al. (2013) designed six principles of design process: 1) design for customer engagement by seeking out native users, 2) design for knowledge translation through radical iteration, 3) design for offline action, 4) design for integrating expert knowledge with local knowledge sharing, 5) design for implicit guidance and 6) design to componentise everything including knowledge bases. australasian journal of information systems miah, blake & kerr 2020, vol 24, research article meta-design knowledge for clinical decision support systems 4 appropriate design theory or existing supportive knowledge, methodological studies for design process and guidelines are rare in the cdss research field. 2.1 relevant dsr knowledge design theories can be a type of prescriptive knowledge (simon, 1969) and many theories have been offered in recent years to assist developers or is researchers. walls et al. (1992) defined the term “is design theory” as an integrated prescription that comprises “a set of user requirements, a set of system features (or principles for selecting system features), and a set of principles deemed effective for guiding the process of development”. further to this markus et al. (2002) suggested that the role of is design theories is to assist developers with guidelines to control, manage and operate any specific design process by focusing their attention and restricting their options. knowledge contributions from effective dsr studies is also central to make contributions to the problem or domain of relevant literature (hevner et al., 2004). gregor and hevner (2013) noted that any knowledge contributions in the form of a design theory should be treated as a type of artefact. beyond the other artefact types defined by march and smith (1995) such as constructs, models, methods, and instantiations, gregor and hevner (2013) outlined that a dsr research project can produce design knowledge in terms of artefact design at three different levels. level 1 in the form of products and processes, whereas more abstract understanding can be treated as knowledge contributions at level 2 (where contributions are design principles, technological rules etc). level 3 represents a much broader abstraction of knowledge as “well-developed design theory”. while a level 1 contribution represents artefact design as a situated implementation, the higher levels concern more abstract theorizing including design principles or architectures that may have further application in new artefact design studies. 2.2 dsr for dss research one of the promises of the dsr method is that it offers greater professional relevance in is design that promote design practices for innovation. march and smith (1995) draw a distinction clearly between natural sciences and design-science research: “whereas natural science tries to understand reality, design science attempts to create things that serve human purposes” (p. 253). arnott (2014b) described a problem for a dss developers; namely how to conceptualize aspects of the decision task that need improvement during iterations of the evolutionary development process. many of the early dss studies involved designing and implementing innovative it-based systems through classic development methodologies (arnott & pervan, 2012). the dsr view shows promise beyond the classical approaches of it system development, and using dsr guidelines provides a basis for considering how to improve both the quality and impact of dss (arnott & pervan, 2012). this approach could contribute to bridging the gap between academic dss design research and professional worlds of practice. as already noted, much of dss design has been limited to specific techniques developed without looking at the practitioner’s contextual demands, and in many cases relevance issues have been ignored in favour of academic or technical priorities. 2.3 cdss research one of the most important purposes of cdss is to reduce medical errors: recognised as a major issue in healthcare and medical domains. agharezaei et al. (2013) suggested that cdss generally improves the quality of healthcare services and meeting patient’s medical demands. health domains are information intensive in which various demands apply for professional australasian journal of information systems miah, blake & kerr 2020, vol 24, research article meta-design knowledge for clinical decision support systems 5 consultation and support, both with everyday condition changes and emerging medical knowledge. in the past, cdss research has been designed for the purposes of diagnosis, treatment, and medical follow-up. zolhavarieha and parry (2017) developed a candidate matrix from their developed cdss framework called knowledge quality assessment (kqa) for discovering and evaluating the clinical knowledge for cdss design. chen et al. (2018) investigated an association of inhaled corticosteroids and fracture in order to develop a cdss. in the research chen et al. (2018) particularly addressed the issue of improving predictive performance of the cdss. yu (2015) improved the utility of the cdss by focusing on the importance of the relationship of knowledge bases and the cdss system as key drivers of rapid learning in clinical care service. these authors seldom mentioned the requirements of generating new understanding that may add value for contextual requirements of particular cdss problem. cdsss show evidence of robust use for improving health professionals’ performance however, many studies questioned its design acceptance and effectiveness due to its contextsensitiveness for example in terms of lack of user uptake (miah, gammack & mckay, 2019; miah, kerr, gammack, & cowan, 2008; miah, 2009) and a lack of options that offer flexible workflows (wright & sittig, 2008). information system use is an essential construct that indicates human behaviour in it utilization and successful is adoption in organizations (sun & teng, 2012). an influential paper by orlikowksi and iacono (2001), which restored to is a focus on the it artefact, suggested five meta-categories covering its various conceptions within is (namely: tool view, proxy view, ensemble view, computational view and nominal view). of these the “ensemble view” concerns “the dynamic interactions between people and technology” (orlikowski & iacono, 2001, p. 126), which describes the enmeshing of technologies of use. orlikowski and iacono (2001) promoted a strong socio-technical tradition for conducting is design research in that all it artefacts are inevitably embedded in a physical setting. orlikowski and iacono’s study suggested that comprehensive practices of the use of it artefacts should be integrated into relevant theory. also, the ensemble artefact should deal with outlining a combined process of problem-solving to align with both problem spaces of human – a working model of physicians and patients for their decision support. if this need can be addressed, bothphysicians’ performance can be increased and patients’ care-support for their self-management can be enriched. at the same time, cdss alignment within specific domain helps overcome the issue of information arrangement and representation for ease of use. on the other hand, cdss alignment with task representation helps deal with the issue of incorporating clinical practice guidelines (cpgs) (fox et al., 2010) and integrating with clinical workflows (kesselheim, cresswell, phansalkar, & et al., 2011). 3 method the study reported in this paper builds upon our previous work (blake, kerr & gammack, 2016) which was focused on reporting a specific case of cdss design and evaluation. the aim of design knowledge development from the specific cdss design case was based upon an artefact called a knowledge based cdss (kb-cdss), shown in figure 1. the artefact prototype received initial data on patients from physicians. the physician uses sources such as referral or/and previous medical history and test results. the cdss prototype offers decision support information to physicians for patient care, using user centred design principles (miah, 2004). the artefact addressed issues of developing a major source of information that could be used to diagnose sleep disorders; namely the patient’s sleep diary, by developing an online sleep australasian journal of information systems miah, blake & kerr 2020, vol 24, research article meta-design knowledge for clinical decision support systems 6 diary. going beyond what the artefact did, with the artefact design and evaluation, we attempt to build new design understanding, so that cdss researchers and cdss design professionals obtain guidelines. login page starting with diary page step 1 in diary subsequent step in diary final step in diary graph output figure 1: developed cdss artefact de leoz and petter (2018) analysed an existing dsr project examining the meta-requirements such as design objectives and how they were accomplished through the effective artefact design. the insights as new understandings are communicated as theory in terms of capturing principles of design function for identifying social impact of artefact. following a similar pathway in a different problem context cdss design, our objective is to further develop design knowledge as a theory artefact (gregor & hevner, 2013) based on dsr phases of problem relevance, artefact design, artefact evaluation and communication. figure 2 illustrates the methodology we used. australasian journal of information systems miah, blake & kerr 2020, vol 24, research article meta-design knowledge for clinical decision support systems 7 figure 2: overall methodology for the design science research the methodology may be conveniently partitioned into four sets of activities: problem relevance, artefact design, artefact evaluation and communication. the following table 1 includes the details of each activities. we followed hevner et al. (2004) for conducting new knowledge generation from the artefact design study. the apsara pattern retrieval approach described in purao & storey (2008) has parallels with case-based reasoning (jonassen & hernandez-serrano, 2002), and while their focus was on technology acceptance of reused designs, their work was less appropriate to our context. their approach was considered more suited to operational artefact design rather than generating new theory based on the characteristics of problems, design activities, user’s involvement and their viewpoints on design and evaluation. likewise, as goldkuhl and sjöströmb (2015) note, march and smith (1995), hevner et al (2004) and peffers et al (2008) approaches downplay theory as a dsr outcome. hevner et al’s. (2004) seven guidelines however, do provide general considerations for defining a dsr problem space, specifying a design-based solution artefact, implementing the design solution, evaluating the design artefact and communicating study details and results, and as such provide for theory as an output of dsr. we grouped these guidelines for convenience into four phases to fit our purpose of new theory development study: regarding problem relevance we utilised guideline 2; for the artefact design phase, we utilised guideline 1, 5 and 6, for the artefact evaluation phases, we used guideline 3 and for communication phase, guidelines 4 and 7 were used. table 1 includes the details of the four phases. artefact design problem relevance artefact evaluation communication problem definition and solution requirements solution strategies solution method solution outlining principles and general architecture as meta-design evaluation method criteria user groups design findings target audience australasian journal of information systems miah, blake & kerr 2020, vol 24, research article meta-design knowledge for clinical decision support systems 8 activities in phases dsr guidelines descriptions problem relevance guideline 2 sleep problems ... are common and have adverse health, social and economic costs (adams et al. 2017). extant cdss solutions are inadequate. general practice healthcare provision is a specific problem of global relevance. this is the essence of the research gap addressed by the dsr study. artefact design guidelines 1,5 & 6 guideline 1: design as an artefact. the study has produced a cdss artefact designed to support healthcare decisions for patients in the domain of sleep disorders. guideline 5: research rigor. established methods and proven techniques were used. as the aim was differential diagnosis in a space of disorders this made clear the basis for specific diagnosis in a structured and defensible manner. rule based decision support systems were also chosen as it was possible to simplify each decision point. guideline 6: design as a search process. validated healthcare knowledge was employed to ground the design artefact in this study. the design process was iterative in order to cope with much of the uncertainty inherent in the problem space (e.g. patients concerns relevant to general practices are separated from the primary medical and healthcare requirements) and to allow progressive and incremental solution development at a level so it can be presented for evaluation and componential specification. artefact evaluation guideline 3 guideline 3: design evaluation. to demonstrate artefact utility, both focus group and field studies have been conducted with representative stakeholders to capture opinion on the prototype’s use. evolutionary prototyping development in consultation with representative doctors ensured ongoing evaluation and relevance, following established design science guidelines. communication guideline 4 & 7 guideline 4: research contributions. the experimental and focus group outcomes and analysis have shown clear benefits to the target populations. guideline 7: communication of research. this study presents detail relevant to academic, management and industry professionals, and has been verbally presented to such in workshops and presentations during its development and evaluation through focus groups. table 1: the adopted research methodology that was modelled on hevner et al. framework the design case concerned providing decision support to physicians and patients for treating sleeping disorders. before abstracting to more generic context, we outline the phases, beginning with the first phase, problem relevance, where we explore and characterise the specific problem domain space. in the australian healthcare sector, sleep disorders are a significant and growing problem for the physical and psychological well-being of individual patients. sleep disorders commonly affect people around middle-age so as the population ages there are an increased number of australasian journal of information systems miah, blake & kerr 2020, vol 24, research article meta-design knowledge for clinical decision support systems 9 people with sleep disorders with risk factors in adults being obesity, alcohol, smoking, nasal congestion and menopause (young, peppard, & gottlieb, 2002). physicians, who deal with sleep disorders, and their administrative support staff and facilities, are under constant pressure to find more effective methods to deal with the increasing backlog of patients. the main symptom of a sleep disorder is excessive daytime sleepiness and there are a number of lifestyle and physical consequences associated with this. many of these patients face significant wait times before being able to attend a consultation and receive treatment to relieve symptoms. a large percentage of the australian population is affected by some form of sleep disorder, and the direct and indirect costs of sleep disorders to the australian society in 2016-170 was 66.3 billion dollars (deloitte access economics, 2017). in the second, (artefact design) phase, to operate an effective process for knowledge acquisition, the lead researcher formed a team with two sleep physicians and a psychologist. these three individuals were experts in the field of sleep disorder diagnosis and a participative approach was taken to help gain insights into their shared knowledge and experience and to develop the domain specific knowledge-base for the dss. the importance of gathering a detailed patient history during the consultation was highlighted in initial interviews with physicians. the inclusion of patient history as an input was therefore part of the system scope that the physicians expected and considered best practice during a consultation4. gaining these stakeholders’ approval of the output from the cdss and for the data gathering instrument for the patient history was therefore essential. access to the sleep investigation clinics’ patient records allowed testing of the instrument and also gave the medical team a vested interest in the success of the project (cornwall & jewkes, 1995). the two sleep specialists provided expertise on the diagnostic criteria and process, while the psychologist was specialized in psychological sleep disorders such as insomnia, the psychology impacts of sleep disorders and compliance to sleep disorder treatment by patients. the researcher acted as a team leader to set agendas, produce questionnaire drafts, schedule and keep records of the meetings and keep the team focused on the required outcomes. from that an initial cdss was planned which would automate patient history data gathering by using an online patient history questionnaire, which would act as a data-acquisition tool. this information would be stored in a database with the additional benefit of forming an evidence base of patient histories. our initial design of dss was driven by a series of three one-hour interviews with the director of the sleep investigation unit in a brisbane, australia hospital (one of the experts in the aforementioned medical team). the purpose of these interviews was to understand the environment within which the cdss will be operating and discuss and record the protocol of the sleep disorder diagnostic process. an understanding of the context of the artefact was important in order to ensure that the it artefact (cdss) instantiation does not cause unseen 4 the cdss we designed for recording patient histories and symptoms and mapping these against international sleep disorder classification categories. this was associated with a separate sleep history tool for recording sleep patterns, usually for 14 nights. each day the patient notes details of the night's sleep, along with intakes of caffeine and alcohol, to build up a picture of sleep habits (known as sleep hygiene). poor sleep hygiene, or insufficient time made available for sleep, may be a cause of excessive daytime sleepiness, the main symptom of a sleep disorder. the tool is used by health professionals, such as physicians and sleep specialists, as a primary diagnostic tool for identifying insomnia (blake, kerr, & gammack, 2016). australasian journal of information systems miah, blake & kerr 2020, vol 24, research article meta-design knowledge for clinical decision support systems 10 side effects such as reporting misleading patient history. the cdss also needs to meet the requirements of the end user (march and smith 1995). in this instance the environment is medical and therefore a strong requirement is patient privacy, provision for multiple sleep disorders in one patient and awareness of co-existing conditions impacting on sleep disorders. in third phase, for evaluating the cdss artefact for sleep physicians, real time data needed to be obtained. we used ten randomly chosen sleep patients filled in 14 days of sleep diary data and completed the patient history questionnaire. these data were used to input in our cdss application, and the produced output was presented to the physicians. these outputs are: 1. sleep diary graph; 2. sleep diary statistics; and 3. physician report that was mainly generated from the diary graph and the statistics. the cdss artefact holds an option for sending reports via email to the physicians. this is equivalent to the second stage in the consultation process, in which physician categorises patients (analysing the report) into the three levels of appointment urgency. the physicians (participants) who were involved in the development of the cdss artefact (e.g. web application) were asked to comment on the results. participants considered the difference that having this information would make when they were consulting a patient. one physician made comments focusing on the operational aspects of the reports that is, the output of the cdss rather than on how the application might affect the consultation process. the other physician focused on how the use of the application would affect a consultation, so that both points of view were discussed. for the purpose of checking acceptance of the technology, evaluation focused firstly on the ease-of-use aspects, particularly addressing readability and layout, and secondly on the usefulness of the reports themselves for supporting professional decisions. the comments made by the physicians were grouped under these themes (blake, kerr, & gammack, 2016). various presentations and stakeholder focus groups affirmed the utility and ease of use within normal workflows. this, with the academic articles reporting the work, constituted the communication aspect of the dsr guidelines. 4 design artefact as theory to address artefact design requirements, iivari (2015) proposed two strategies. the first strategy (strategy 1) is evidenced when dsr researchers first construct an it meta-artefact as a general solution concept, that can then be instantiated in multiple, specific solution contexts. the alternative strategy (strategy 2) is evident when researchers attempt to address a userspecific problem by building an it artefact for that specific problem context, and then distil from that artefact knowledge that can be generalised into a general solution concept. the case study on which this paper is based represent an example of a strategy 2 approach being adopted. this enabled the learning and initial design principles and the meta-artefact design to be instantiated, and thus tested in practice. in this section, we utilize this work to theorise and extract firstly, the relevant problem class to which our meta-artefact applies, then the design principles and architecture underpinning the meta-artefact, to present a practitionercentric design environment, a general solution concept suited to specific configurations applicable to problems that are representative of the problem class. design theory can include the other forms of design knowledge: constructs, models, methods, and instantiations that convey knowledge. markus et al. (2002) suggested that design theory formalises knowledge in dsr and provides prescriptions for design and action: it says how to australasian journal of information systems miah, blake & kerr 2020, vol 24, research article meta-design knowledge for clinical decision support systems 11 do something. as such design theory constitutes prescriptive knowledge as opposed to descriptive knowledge, which encompasses the other types of theory identified in gregor and jones (2007). this view was reinforced by gregor and hevner (2013) who characterise “design theory as an abstract, coherent body of prescriptive knowledge that describes the principles of form and function, methods, and justificatory theory that are used to develop an artefact or accomplish some end” (p. a3). 4.1 a design theory for cdss our proposed design theory is comprised of a generic solution architecture for cdss and six design principles. find figure 3 and table 2 below for them. the essential functions of cdss are to first to support a correct diagnosis, and secondly to relate this to a suitable treatment. this implies relevant information should be input, and reasoning with it be justifiable in professional terms. abstractly, this applies to all diagnostic domains from medicine to car mechanics, and indeed this activity has been formalised in a general “algorithm” for identifying component causes causing discrepancy from a normal system state. the reasoning involved follows established logic models and is applied from first principles, rather than heuristic rules derived from professional experience. medicine is not an exact science however and equally recognises that clinical judgment is essential, with diagnostic reasoning “the most critical of a doctor’s skills” (croskerry, 2009, p. 1022). these two, complementary, universal theories of diagnosis are supported in our theory and model as detailed further below. both patients and physicians are sources of the relevant information, both locally specific and general, and through the physician’s extra resource access, access to further and emerging scientific knowledge. the knowledge involved must however meet an objective standard: described in the true and justified terms of the scientific domains involved. thus, the architecture provides for patient and physician interfaces, and grounds the knowledge base in the accepted international classification. in this case the domain terms concern the sleep specialisation, but the architecture naturally supports other clinical knowledge bases. an extra, administrative interface is provided in the architecture since stakeholders other than the two principals may require read access for management or technical monitoring, or to support epidemiological or other studies. between the interface and the terminology classification layer are the tools for supporting the general functionality of diagnosis and treatment recommendations. the classification layer provides a defined ontology for the knowledge categories, and as mentioned, can be replaced by other formal ontologies in different cdss. the knowledge base itself in the layer above reflects the expert knowledge and rules involved, expressed in standard terms, and can be developed using standard knowledge acquisition techniques. we describe below our approach in developing the knowledge base for the sleep case, but this component is also generic to different specific designs and knowledge acquisition approaches. the other middle layer is the logic applied to the facts of the individual case and the domain knowledge in the system. again, various reasoning models can be used here, whether rule base, statistical, pattern recognition or various logical analyses, and the generic architecture is agnostic as to which is preferred. in the top layer, the domain specific tools reside, in this case the record of relevant patient information, and the reports produced by or required by the users. these are the general type of input and output tools used in the reasoning module, the specific tools in the diagram include the patient’s sleep diary information and a specific, standard form of australasian journal of information systems miah, blake & kerr 2020, vol 24, research article meta-design knowledge for clinical decision support systems 12 report used widely in the field. this architecture constitutes the first part of the design theory. before considering the general design principles we illustrate the theory as instantiated in the sleep case. figure 3: generic architecture of our proposed cdss system logging into the sleep tools application presents either a physician or patient landing page dependent on the login details used. the patient has an option to complete a standard questionnaire over multiple sessions and once completed send the report as a pdf document to the clinic. they also have an option to complete a 14 day sleep diary, to form a baseline of their sleep patterns. after completion of each day’s sleep diary, the patient sees a graphical representation of their sleep patterns which builds day by day. this chart is designed to give the patient an easy-to-assimilate view of their sleep patterns and their morning moods to allow for functional analysis, for instance to assess any need to allow more time for sleep or whether they are getting more sleep than they were aware of. the user evaluation demonstrated that the sleep tools application helped the patient to understand their sleep patterns. they do not receive the report drawn from the questionnaire directly as the physicians and sleep psychologist felt that this was technical information which needed to be assessed by a specialist. equally the physician, can log in, select a patient from their clinic and retrieve reports from the patient. the report provides a representation of the patient history gained through the questionnaire. the information is presented such that the physician can quickly scan for the necessary information to diagnose the sleep disorder indicated by patient’s responses. this efficiently removes a lot of information gathering from the consultation and allows more time to talk about the patient’s specific disorder. statistics and patterns useful for diagnosis are calculated within the sleep diary tool, again saving time in a face-to-face consultation. as well as being a springboard for the patient to discover information, the questionnaire also gives the patients a common basis for dialog. the biomedical information is in a format which fits the physician’s disorder model, and can be accessed beforehand to allow more consultation time focus on listening to the patient’s narrative (patel, arocha, & kushniruk, 2002). australasian journal of information systems miah, blake & kerr 2020, vol 24, research article meta-design knowledge for clinical decision support systems 13 filling the patient history questionnaire online, away from the clinic, also allows time for patients to reflect on the questions and use standard information seeking behaviours such as discussing the questions with family and peers (tuckett, 1987). after that, a registered administrative officer may obtain the reports for using in the patient history file. the cdss artefact generated reports in pdf format and exportable in xml if required. various report options were designed to support an acceptable workflow integration and efficiency. the process of a medical diagnosis is logically described by seising (2006, p. 238) as follows: “the doctor notes the patient’s signs and symptoms, combines these with the patient’s medical history, physical examination and laboratory findings and then diagnoses the disease”. however the author also argued that “it is very difficult to define sharp borders between various symptoms in the set of all symptoms and between various diseases in the set of diseases” (seising, 2006, p. 240), reinforcing the point that medical science is not exact and judgment as well as logics are required. as no two clinical presentations are identical the physician uses a set of heuristics and their own intuition to diagnose a health disorder (breslin, mullan, & montori, 2008; seising, 2006). to justify a diagnosis, the physician might construct a logical argument to decide about the diagnosis. toulmin (1958) contended that a diagnosis may be treated as an assertion, i.e. a statement which is intended to be taken seriously and if challenged can be defended with the foundation for the assertion. the author then describes how an argument is built using the details upon which the assertion was built. this argument will be presented in a series of stages or steps, however these steps do not necessarily reflect the process originally used to arrive at the assertion but rather reflects the best argument that can be put forward in its support (toulmin, 1958, p. 17). the need to provide an explanation of the process of decision making is paramount in cdss, since one of the critical factors for physicians to accept a clinical decision support system is openness and transparency of how the decision was derived. an explanation of the pathway to a decision by an application also leads to increased satisfaction and increased trust in the outcome by users. the requirement for justified diagnosis and explanation functionality drove the adoption of small rule-based decision support systems at every decision point in the argument toward diagnosis as described variously e.g. by stranieri et al. (1999). in this case we used marriot et al. (2011) hi-trees structure which is suited to formal argument mapping and decision support in hierarchical domains generally. as the aim was differential diagnosis in a space of disorders this made clear the basis for specific diagnosis in a structured and defensible manner. rule based decision support systems were also chosen as it was possible to simplify each decision point to a yes/ no response. each recognised sleep disorder, as identified in the international classification of sleep disorders (icsd), followed by their individual criteria made twentyeight natural decision points. the simplicity of the multiple decision support systems ensures that the system is flexible and easy to change in the event of additions to or changes in the diagnostic criteria. the architecture allows for other models to be used here, for instance based on likelihood ratios or bayesian models. the diagnostic environment around sleep disorders recognises them as chronic conditions which require lifestyle changes in order to manage and mitigate disorder impacts. this means that the function of the consultation along with diagnosis involves education and knowledge sharing with the patient to enable shared decision making for condition management. the method and design used aimed to benefit both the patient in this regard, allowing more consultation time for discussing treatment than previously, and in a consistently used australasian journal of information systems miah, blake & kerr 2020, vol 24, research article meta-design knowledge for clinical decision support systems 14 common vocabulary. other later life onset chronic conditions requiring lifestyle changes and shared decision making to which this cdss model could be applied include diabetes, arthritis and obesity management. in addition to the architecture, the second part of our design theory specifies general principles that we believe contributed to the success of the cdss case in our project. venable et al. (2016) suggested that prescriptive design knowledge is about “means-end relationships” between problem and solution spaces (vom brocke et al. 2020; venable, 2006). dsr literature indicated that design knowledge can be represented in a form of designed artefacts (hevner et al., 2004; gregor and hevner, 2013), design principles (markus et al. 2002), or design theories (gregor and hevner, 2013; gregor and jones, 2007). markus et al.’s (2002) study proposed a design knowledge in terms of design principles for a class of problems encompassing different processes, user requirements, and knowledge requirements. principles description principle 1: the cdss should provide a direct benefit to all stakeholders to allow functional decision making an ideal design will be aimed to serve each of its primary stakeholders: (here the patient, physician and health service record administration), implying design of interface options relevant to their roles and identified needs. this supports both acceptance and the cdss’s intended functionality and so should be explicitly considered in the development approach. principle 2: the cdss should use an established vocabulary and shared ontology the terms used in explanation and records must have consistency to ensure effective communication between patient and physician, and compatibility with central health records. grounding the knowledge base in an established ontology or otherwise agreed vocabulary will help achieve this. principle 3: the cdss should be conceptualised and developed collaboratively the nature of medical diagnosis is such that experts in the field need to provide the relevant knowledge but also the nuances of judgment around the information available. in addition, the stakeholders are best placed to identify the type of support they need, and ensure that development proceeds appropriately towards this principle 4: the cdss should fit within the users’ workflows system acceptance is more likely when it operates within the normal flow of work rather than as an overhead, and especially if workflow efficiencies are gained. our design approach paid attention to this aspect: in general cdss design should aim to fit user stakeholders’ workflows and actual decision support requirements principle 5: the cdss must be practice-sensitive and provide good quality information for all users the practical situation of use, and the user concerns must be fully understood and prioritised for deliverables. in general, communicating relevant information efficiently and providing justified analysis and reports in stakeholder terms will be needed principle 6: the cdss must be scientifically trustworthy acceptance by physicians will not occur if the science and reasoning is not transparent. designs must accommodate this, and an explanation facility will also aid patient and other user acceptance. the rigour in the development methods of design science also provides for trust in the development. table 2: design principles for cdss australasian journal of information systems miah, blake & kerr 2020, vol 24, research article meta-design knowledge for clinical decision support systems 15 the artefact design and evaluation through the four activities (defined in table 1) was used to gather reflections. this process is a valid way as it was indicated by de leoz and petter (2018). if is researchers seek to develop a new design understanding based on the resulting of a particular artefact design, the work could be a value creation as it includes assistive method or knowledge in terms of new design theory (de leoz and petter, 2018). the findings have been abstracted and are listed in table 2. the descriptions in table 2 characterise the principles applied in our design which we believe should also apply in cdss developments more generally. clearly any system developed should meet criteria both of rigour and relevance, but acceptance has also been an issue for cdss and dss generally. we believe acceptance, as well as relevance, is enhanced through collaborative and participative development approaches. likewise, a system designed to integrate with workflow, making it more usefully productive from all stakeholder viewpoints is more likely to gain acceptance. 5 discussion the aim of the study was to outline design theory for cdss researchers and designers that addresses both clinical and non-clinical design issues. our aim was to construct meta design knowledge to inform and improve cdss design practices for a particular class of problems that were addressed through our proposed kb-cdss artefact. the technical specification of the kb-cdss artefact was reported in blake et al (2016), following dsr guidelines. sleeping disorders were chosen as a problem domain in this study because effective diagnosis of sleep disorders takes time and has issues of handling massive amount of data, so efficient interaction is important not just for physician’s workflow, but also for patient wait times that have direct impact on the overall cost of treatment and patient wellbeing. our design study aims to address common aforementioned issues of cdss design (such as poor user uptake, ease of use and flexibility in supporting physicians and patients) with less cognitive effort and load. de leoz and petter (2018) recommended the design principles for is researchers the opportunity to consider “social qualities embedded into artefacts beyond merely evaluating the utility of an artefact” (p.166). following the approach, developing a sensible understanding of cdss technical attributes and how the sleeping disorder issues were addressed was the central task of our study reported in this paper. we combined a new understanding that focuses both on problems context of use and the technical aspects of artefacts. to justify the knowledge production out of the design study, researchers must talk about the theoretical basis of their artefact design or design science concepts that they propose. baskerville et al. (2015) described four different modes of reasoning of knowledge production with respect to different knowledge goals and scope. the design knowledge we produced through the important is theories in dsr within the cdss context of design will be contributing to both design researchers and professional healthcare system designers to improve their practice. this paper also provides a case demonstration for the significance of the latest theoretical improvement (e.g. for enhancing effect of technological development such as “internet of things” (de vass, shee and miah, 2018)) to increase clarity on why dsr should contribute to new knowledge production and also pay attention for generalized theory (kuechler & vaishnavi, 2012). thus, it is important to clarify design issues concerned to improve solution design theory and understanding via appropriate artefact development. australasian journal of information systems miah, blake & kerr 2020, vol 24, research article meta-design knowledge for clinical decision support systems 16 two key components of the theoretical understanding are outlined in the study: a) the general solution design or concept of artefact that is designed by following the theoretical articulations form initially design artefact for practitioners’ specific need but remains flexible for meeting both demands of knowledge production and support to transform patient’s data into useful knowledge. the knowledge acts to provide a staged approach towards a full sleeping diagnosis but also offers various evidence-based treatment options for the benefit of both patients and physicians. b) design principles generated from the proposed design understanding and solution design concept that will be applicable to any to other similar problem domain for designing similar artefact design. knowledge that is directly relevant to artefact design is one of the outputs of design science research (dsr) and has an important application in improving dss design research (miah & gammack, 2014; miah & genemo, 2016; genemo, miah & mcandrew, 2015; miah, 2009). although, several studies have been introduced for methods of constructing and evaluating artefacts (hevner et al., 2004; iivari, 2015; orlikowski & iacono, 2001; miah 2008; peffers, 2008); design theory (gregor & hevner, 2013; iivari, 2015) , a very few attempts are made on contributing to the development of artefact design knowledge such as the key activities, ground rules and processes that may construct the artefact itself. design combined within the artefact must be seen to result from the design that is defined as an “artificial objects having desired properties” (simon, 1969, p. 4). for present purposes we refer to the “ensemble artefact” that is positioned in the action design research approach (sein, henfridsson, purao, rossi, & lindgren, 2011). we also recognise the numerous guidelines that offer good analysis, coherent arguments and constructive suggestions but which have perhaps been hampered by a standardised conceptual coherence. our intention here is not to add competing guidelines, but to identify theoretical components relevant to design that explicitly follows an ensemble view, and which can become operationalized in specific design contexts. 6 conclusions the study attempted to generalise new design knowledge from a case study and has multiple implications to cdss designers and dsr researchers. the researcher developed design principles as guidelines could be an opportunity for other researchers to expand their view of the problem. impacting not only their consideration of the technology aspects, but also equally weighting the problem context of use for an improved design knowledge. the proposed understanding would drive generating a more complete view of the problem and cdss solution design space. further study is required for evaluating the proposed architecture and design principles. investigating new similar decision support problems that would ensure decision makers value and consider behavioural matters within an organizational context. future evaluation could be a longitudinal study that may examine patterns of use and perceptions of value or utility over time, in a clinical/organisational setting. this would increase confidence that ongoing relevance could be an effective outcome of this research. references adams, r. j., appleton, s. l., taylor, a. w., gill, t. k., lang, c., mcevoy, r. d., & antic, n. a. (2017). sleep health of australian adults in 2016: results of the 2016 sleep health foundation national survey, sleep health, 3(1), 35-42 agharezaei, z., tofighi sh, s. h., nemati, a., aagharezaei, l., & bahaadinbeigi, k. (2013). surveying kerman’s afazalipour hospital clinical and educational staff`s points of view australasian journal of information systems miah, blake & kerr 2020, vol 24, research article meta-design knowledge for clinical decision support systems 17 about the clinical decision support system designed for reducing the possibility of pulmonary embolism and deep vein thrombosis. hospital journal, 12(2), 29-38. arnott, d., & pervan, g. (2012). design science in decision support systems research: an assessment using the hevner, march, park, and ram guidelines. journal of the association for information systems, 13(11), article 1. arnott, d., & pervan, g. (2014a). a critical analysis of decision support systems research revisited: the rise of design science. j inf technol, 29(4), 269-293. doi:10.1057/jit.2014.16 arnott, d., & pervan, g. (2014b). a critical analysis of decision support systems research revisited: the rise of design science. journal of information technology, 29(4), 269-293. doi:10.1057/jit.2014.16 baskerville, r. l., kaul, m., & storey, v. c. (2015). genres of inquiry in design-science research: justification and evaluation of knowledge production. misq., 39(3), 541-564. doi:10.25300/misq/2015/39.3.02 baskerville, r. l., baiyere, a., gregor, s., & rossi, m. (2018a). design science research contributions: finding a balance between artifact and theory, journal of the association for information systems, 19(5):358-376 baskerville, r.l., kaul, m., & storey, v.c. (2018b) aesthetics in design science research, european journal of information systems, 27(2), 140-153 blake, j., kerr, d., & gammack, j. (2016). streamlining patient consultations for sleep disorders with a knowledge-based cdss, information systems, 56, 109-119 breslin, m., mullan, r. j., & montori, v. m. (2008). the design of a decision aid about diabetes medications for use during the consultation with patients with type 2 diabetes. patient education and counseling, 73(3), 465-472. chang, t.-m., kao, h.-y., wu, j.-h., & su, y.-f. (2016). improving physicians' performance with a stroke cdss: a cognitive fit design approach. computers in human behavior, 54, 577-586. doi:https://doi.org/10.1016/j.chb.2015.07.054 chen, yf., lin, c.s., wang, k.a., rahman, l.o.a., lee, d.j., chung, w.s., & lin, h.h. (2018). design of a clinical decision support system for fracture prediction using imbalanced dataset, hindawi journal of healthcare engineering, id 9621640, 13. cornwall, a., & jewkes, r. (1995). what is participatory research? social science & medicine, 41 (12), 1667-1676. croskerry, p. g. (2009). a universal model of diagnostic reasoning. academic medicine : journal of the association of american medical colleges, 84(8), 1022-1028. deloitte access economics. (2017). asleep on the job: costs of inadequate sleep in australia. retrieved from australia: url: https://www.sleephealthfoundation.org.au/files/asleep_on_the_job/asleep_on_the_job _shf_report-web_small.pdf de leoz, g. & petter, s. (2018). considering the social impacts of artefacts in information systems design science research, european journal of information systems, 27(2), 154-170 de vass, t., shee, h., & miah, s.j. (2018). the effect of “internet of things” on supply chain integration and performance: an organisational capability perspective, australasian australasian journal of information systems miah, blake & kerr 2020, vol 24, research article meta-design knowledge for clinical decision support systems 18 journal of information systems, 22, url: https://journal.acs.org.au/index.php/ajis/article/view/1734 fischer, g., giaccardi, e., ye, y., sutcliffe, a. g., & mehandjiev, n. (2004). meta-design: a manifesto for end-user development. communications acm, 47(9), 33-37. fox, j., glasspool, d., patkar, v., austin, m., black, l., south, m., & vincent, c. (2010). delivering clinical decision support services: there is nothing as practical as a good theory. journal of biomedical informatics, 43(5), 831-843. doi:https://doi.org/10.1016/j.jbi.2010.06.002 geremo, h., miah, s.j., & mcandrew, a. (2015). a design science research methodology for developing a computer-aided assessment approach using method marking concept, education and information technologies, 21, 1769–1784 goldkuhl, g. & sjöström, j. (2015). closing the practice loop: practice design research. ais sigprag pre-icis workshop 2015 on “practice-based design and innovation of digital artifacts”, url: https://www.semanticscholar.org/paper/closing-the-practiceloop%3a-practice-design-research-goldkuhlasj%c3%b6str%c3%b6mb/b4a10451f242a1736bfbb63c8069001667f065e6 gregor, s., & hevner, a. r. (2013). positioning and presenting design science research for maximum impact. mis quarterly, 37(2), 337-355. gregor, s., & jones, d. (2007). the anatomy of a design theory, journal of the association for information systems, 8(5), 312–335. hevner, a. r., march, s. t., park, j., & ram, s. (2004). design science in information systems research. mis quarterly, 28(1), 75-105. iivari, j. (2015). distinguishing and contrasting two strategies for design science research. european journal of information systems, 24(1), 107-115. doi:10.1057/ejis.2013.35 jonassen, d. h., & hernandez-serrano, j. (2002). case-based reasoning and instructional design: using stories to support problem solving. educational technology research and development, 50(2), 65-77. doi:10.1007/bf02504994 kesselheim, a. s., cresswell, k., phansalkar, s., & et al. (2011). clinical decision support systems could be modified to reduce 'alert fatigue' while still minimizing the risk of litigation. health aff (millwood), 30, 2310-2317. kuechler, w., & vaishnavi, v. (2012). a framework for theory development in design science research: multiple perspectives (vol. 13). koehne b., redmiles d., & fischer g. (2011). extending the metadesign theory: engaging participants as active contributors in virtual worlds. in m. f. costabile, y. dittrich, g. fischer, a. piccinno (eds.) end-user development: is-eud 2011. lecture notes in computer science, vol 6654. berlin: springer. litvin, c. b., ornstein, s.m., wessell, a.m., nemeth, l.s. & nietert, p.j. . (2012). adoption of a clinical decision support system to promote judicious use of antibiotics for acute respiratory infections in primary care. international journal of medical informatics, 81, 521526. doi:10.1016/j.ijmedinf.2012.03.002 australasian journal of information systems miah, blake & kerr 2020, vol 24, research article meta-design knowledge for clinical decision support systems 19 march, s. t., & smith, g. f. (1995). design and natural science research on information technology. decision support systems, 15(4), 251-266. marcos, m., maldonado, j. a., martínez-salvador, b., boscá, d., & robles, m. (2013). interoperability of clinical decision-support systems and electronic health records using archetypes: a case study in clinical trial eligibility. journal of biomedical informatics, 46(4), 676-689. doi:https://doi.org/10.1016/j.jbi.2013.05.004 markus, m. l., majchrzak, a., & gasser, l. (2002). a design theory for systems that support emergent knowledge processes. mis quarterly, 26(3), 179-212. marriott, k., sbarski, p., gelder, t. v., prager, d., & bulka, a. (2011). hi-trees and their layout. ieee transactions on visualization and computer graphics, 17(3), 290-304. doi:10.1109/tvcg.2010.45 miah, s.j., kerr, d., gammack, j., & cowan, t. (2008). a generic design environment for the rural industry knowledge acquisition, knowledge-based systems, 21(8), 892-899 miah, s.j. & gammack, j.g. (2014). ensemble artifact design for context sensitive decision support, australasian journal of information systems, 18 (2), 5-20 miah, s.j. & genemo, h. (2016). a design science research methodology for expert systems development, australasian journal of information systems, 20, 1-29 miah, s.j., gammack, j.g., & mckay, j. (2019). a metadesign theory for tailorable decision support, journal of the association for information systems, 20(5), 570-603 miah, s.j. (2009). end user as application developer for decision support. in proceedings of the americas conference on information systems (amcis), p. 142. url: http://aisel.aisnet.org/amcis2009/142 miah, s.j. (2008). an ontology based design environment for rural decision support, unpublished phd thesis, griffith business school, griffith university, brisbane, australia. miah, s.j. (2004). accessibility improvement of multicultural educational web interface by using the user centred design (ucd) approach, in proceedings of the informing science & information technology education joint conference. rockhampton, australia nunamaker, j. f., & chen, m. (1990, 2-5 jan. 1990). systems development in information systems research. paper presented at the twenty-third annual hawaii international conference on system sciences. o’sullivan, d., doyle, j., michalowski, w., wilk, s., thomas, r., & farion, k. (2014). expanding usability analysis with intrinsic motivation concepts to learn about cdss adoption: a case study. health policy and technology, 3(2), 113-125. doi:https://doi.org/10.1016/j.hlpt.2014.02.001 orlikowski, w. j., & iacono, s. (2001). {research commentary: desperately seeking the "it" in it research a call to theorizing the it artifact}. information systems research, 12, 121134. patel, v. l., arocha, j. f., & kushniruk, a. w. (2002). patients' and physicians' understanding of health and biomedical concepts: relationship to the design of emr systems. journal of biomedical informatics, 35(1), 8-16. doi:https://doi.org/10.1016/s1532-0464(02)00002-3 australasian journal of information systems miah, blake & kerr 2020, vol 24, research article meta-design knowledge for clinical decision support systems 20 peffers, k., tuunanen, t., rothenberger, m., & chatterjee, s. (2008). a design science research methodology for information systems research. journal of management information systems, 24(3), 45-77. purao, s., & storey, v. c. (2008). evaluating the adoption potential of design science efforts: the case of apsara. decision support systems, 44(2), 369-381. doi:https://doi.org/10.1016/j.dss.2007.04.007 sein, m., henfridsson, o., purao, s., rossi, m., & lindgren, r. (2011). action design research, 35. seising, r. (2006). from vagueness in medical thought to the foundations of fuzzy reasoning in medical diagnosis. arificial intelligence in medicine, 38(3), 237-256. simon, h. a. (1969). the sciences of the artificial. cambridge, massachusetts: m.i.t. press. stranieri, a., zeleznikow, j., gawler, m., & lewis, b. (1999). a hybrid rule – neural approach for the automation of legal reasoning in the discretionary domain of family law in australia artificial intelligence and law, 7(2-3). sun, j., & teng, j. t. c. (2012). information systems use: construct conceptualization and scale development. computers in human behavior, 28(5), 1564-1574. doi:https://doi.org/10.1016/j.chb.2012.03.016 toulmin, s. e. (1958). the uses of argument. london: cambridge university press. tuckett, d. (1987). review of meetings between experts: an approach to sharing ideas in medical consultations (vol. 5). venable, j. r. (2006). the role of theory and theorising in design science research. proceedings of the 1st conference on design science research in information systems and technology, claremont, ca, usa, 1–18. venable, j. r., pries-heje, j., & baskerville, r. l. (2016). feds: a framework for evaluation in design science research. european journal information systems, 25(1), 77–89. vom brocke, j., winter, r., hevner, a., & maedche, a. (2020). accumulation and evolution of design knowledge in design science research – a journey through time and space, journals of the association for information systems (jais), (in press) walls, j., widmeyer, g., & el sawy, o. (1992). building an information system design theory for vigilant eis. information systems research, 3(1), 36-59. wright, a., & sittig, d. f. (2008). a four-phase model of the evolution of clinical decision support architectures. international journal of medical informatics, 77(10), 641-649. doi:https://doi.org/10.1016/j.ijmedinf.2008.01.004 young, t., peppard, p. e., & gottlieb, d. j. (2002). epidemiology of obstructive sleep apnea: a population health perspective. american journal respiratory critical care medicine, 165(9), 1217-1239. yu, pp. (2015). knowledge bases, clinical decision support systems, and rapid learning in oncology. j oncol pract. 11(2):e206-11. zolhavarieha, s. & parrya, d. (2017). kqa: a knowledge quality assessment model for clinical decision support systems, medinfo 2017: precision healthcare through australasian journal of information systems miah, blake & kerr 2020, vol 24, research article meta-design knowledge for clinical decision support systems 21 informatics, a.v. gundlapalli et al. (eds.), international medical informatics association (imia) and ios press, p. 983-986 copyright: © 2020 miah, blake & kerr. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v24i0.2049 http://creativecommons.org/licenses/by-nc/3.0/au/ microsoft word 1840-5368-1-le.docx australasian journal of information systems koh & kwok 2018, vol 22, post publication review an investigation into failure of internet firms 1 ram, j., & zhang, z. (2018). an investigation into failure of internet firms: towards development of a conceptual model. australasian journal of information systems, 22. doi: 10.3127/ajis.v22i0.1622 review the paper published by ram and zhang (2018) makes a comprehensive attempt at addressing the factors that contribute to foreign internet business failures operating in china. the issue presented is current and interesting in view of the exponential growth in internet use in china and the remarkable business opportunities presented. doing business in china is extremely competitive not only for foreign firms (zhang, 2015), even large local internet firms are not spared (lee, 2017). however, as highlighted by the authors, what is intriguing is that global titans such as google, uber, and ebay have succumbed to local rivals. existing studies tend to examine the institutional constraints (e.g. legal and regulatory) faced by foreign firms in china through a political lens (stevens, xie & peng, 2016; yeo, 2016). yet, few have drawn insights by combining the established socio-technical theory and the comparative theory such as ram and zhang (2018). the authors fill the research gap through propositions and the development of a conceptual model, and as such fulfills the call by li (in press) in a similar study. therefore, ram and zhang’s (2018) paper has the potential for contribution to this area by closing the research gap with the provision of internally driven and externally imposed sources. the authors aim to “examine and identify some of the possible sources or factors driving failure of internet firms in general, by learning from the experience of china and build fundamental knowledge on the challenges of operating in a foreign internet market” (p.2, para 5). the paper gives several examples in support of each factor of failure to construct the propositions for a conceptual model. it is a different approach considering that existing studies have relied on a multiple case study approach to illustrate the performance of foreign internet firms (e.g. zeng & glaister, 2016; nummela, saarenketo & loane, 2016). while it is possible that failures of technology giants in china often get reported in the media, successes of foreign internet firms could be less well reported. e.g. mei.com (“alibaba group announces strategic investment in mei.com”, 2015) and coursera (soo, 2015). additionally, existing studies allude to regulatory barriers in favor of local firms (e.g. jiang, 2014), however, such situation is not often the case. for example, among local chinese firms, tencent has more political capital than alibaba (wang et al., 2016). a balanced view on such cases could be explored in the paper, as well as a discussion on successful practices. as such, the applicability of these practices can be enhanced. internet firms tend to introduce new business models and digital disruption. yet the adoption is also contingent on market maturity and consumer readiness (weihuan, arner & buckley, 2015). hence, several local firms owe much of their success to late-mover advantage instead (wu, ma & shi, 2010). moreover, with china rapidly transitioning to an innovation-driven economy, local firms succeed over their foreign rivals by being more innovative and adaptive (wei, xie & zhang, 2017, see also tse, 2015). therefore, better market orientation, organizational learning, and ambidexterity that enable local firm innovativeness and adaptiveness would contribute alternatively to the model development (hurley & hult, 1998; steiber, 2018; lee et al., 2015). for instance, alipay and wechat pay are unparalleled technological applications utilizing mobile payment, unique only in china (ma, 2016). australasian journal of information systems koh & kwok 2018, vol 22, post publication review an investigation into failure of internet firms 2 finally, the paper is a commendable effort in the attempt to develop a model on understanding foreign internet firm failures in china, considering broad factors that could contribute to such issue. the proposed model provides an important basis for future studies on doing business in china in the new economy. further investigation of the factors and development of the model are merited and would certainly enrich literature and encourage discussion in this area. sharon g m koh monash university malaysia koh.geokmay@monash.edu andrei o j kwok monash university malaysia response from ram and zhang thank you for the review and constructive feedback. we sincerely appreciate the reviewer for encouraging comments. referring to the review where it is mentioned that: “while it is possible that failures of technology giants in china often get reported in the media, successes of foreign internet firms could be less well reported”, we agree that economical reporting from media could be one of possible explanations where failures of foreign firm are more likely to be reported. as a matter of contributing to the debate on the example of mei.com’s success because they received strategic investments from alibaba group (business wire, 2015), we think that volume of investments received is not necessarily a measure of success. in reality, some foreign giants (e.g., uber) received even stronger financial supports, yet still failed (hook, 2016). also, it will be worthwhile to point out here that mei.com received their share of investments in 2015 when the chinese government was encouraging entrepreneurship and investments in “new business” via a national scheme sponsored by the central government (zhang, 2015). it resulted in an “investment boom” in which venture capitals, angel investors, large or even small companies invested in several early-stage firms, especially internet firms. in addition to investing in mei.com, alibaba also invested in dozens of other firms in that year, which strengthens the argument that mei.com’s success may not necessarily be attributed to the strategic investment. it should also be noted that mei.com didn’t receive any more funds afterwards. secondly, while both mei.com (business wire, 2015) and coursera (soo, 2015) may have a large customer base, but considering overall market share their success may not be that significant. mei.com has 2 – 3 thousand of orders per day and 3 million registered users (sun, 2015), which is unparalleled to alibaba, who has more than 30 million orders per day (wieczner, 2015). similarly, coursera with 1 million users (soo, 2015), takes up only approximately 1% market share in china’s online education market and less than 2% in higher education (cnnic, 2016; iresearch, 2018). in comparison, new oriental who partnered with tencent had 8.8 million online users in 2014 and nearly 20 million currently (iresearch, 2018; new oriental education & technology group, n.d.). that is in addition to their larger off-line customers base with huge future potential. nevertheless, the two examples quoted in the review have shown the potential of foreign internet firms in china. interestingly, both firms have partnered with local giants, mei.com with alibaba (business wire, 2015; sun, 2015), and coursera with netease (soo, 2015). it also australasian journal of information systems koh & kwok 2018, vol 22, post publication review an investigation into failure of internet firms 3 strengthens the viewpoints in the published paper that partnership with local firms is crucial for foreign firms’ success in china (ram and zhang, 2018). furthermore, the dominating market share of new oriental over coursera might be contributing to validate the significance of the factor “competitors’ strategic leverage” proposed in the paper, owing to local firm’s stronger network and lager customer base (ram and zhang, 2018). jiwat ram shandong university jiwat.ram@gmail.com zeyang zhang xi’an jiaotong – liverpool university references alibaba group announces strategic investment in mei.com. (2015, july 7). retrieved from https://www.businesswire.com/news/home/20150707006648/en/alibaba-group-announcesstrategic-investment-mei.com business wire. (2015). ‘alibaba group announces strategic investment in mei.com’. retrieved from: https://www.businesswire.com/news/home/20150707006648/en/alibaba-groupannounces-strategic-investment-mei.com cnnic. (2016). ‘statistical report on internet development in china (january 2016)’. retrieved from: http://cnnic.com.cn/idr/reportdownloads/201604/p020160419390562421055.pdf eusme center. (2014). ‘online education market in china’. retrieved from: http://www.eusmecentre.org.cn/report/online-education-market-china hook, l. (2016). ‘china battle drives uber to $1.3bn loss’. retrieved from: https://www.ft.com/content/9cf3f2c2-6ada-11e6-ae5b-a7cc5dd5a28c hurley, r. f., & hult, g. t. m. (1998). innovation, market orientation, and organizational learning: an integration and empirical examination. journal of marketing, 42-54. iresearch. (2018). ‘china's online education sector data in 2017’. retrieved from: http://www.iresearchchina.com/content/details7_40642.html jiang, m. (2014). the business and politics of search engines: a comparative study of baidu and google’s search results of internet events in china. new media & society, 16(2), 212-233. lee, e. (2017, january 6). here's what you can learn from china's biggest startup failures of 2016. retrieved from https://technode.com/2017/01/06/heres-what-you-can-learn-from-chinasbiggest-startup-failures-of-2016/ lee, o. k., sambamurthy, v., lim, k. h., & wei, k. k. (2015). how does it ambidexterity impact organizational agility? information systems research, 26(2), 398-417. doi: 10.1287/isre.2015.0577 li, feng. (in press). why have all western internet firms (wifs) failed in china? a phenomenon-based research. academy of management discoveries. doi:10.5465/amd.2017.0102. ma, w. (2016). launched in china. china's mobile economy: opportunities in the largest and fastest information consumption boom. john wiley & sons, 279-311. australasian journal of information systems koh & kwok 2018, vol 22, post publication review an investigation into failure of internet firms 4 new oriental education & technology group. (n.d.) retrieved from: http://www.neworiental.org/english/who/201507/8213540.html nummela, n., saarenketo, s., & loane, s. (2016). the dynamics of failure in international new ventures: a case study of finnish and irish software companies. international small business journal, 34(1), 51-69. ram, j. and zhang, z. (2018). ‘an investigation into failure of internet firms: towards development of a conceptual model’. australasian journal of information systems, 22. soo, z. (2015). ‘china is no 2 market for online course provider coursera with over 1 million users’. retrieved from: http://www.scmp.com/tech/innovation/article/1823395/china-courserassecond-largest-market-over-1-million-users steiber, a. (2018). china versus silicon valley: comparison and implications. in management in the digital age (pp. 93-113). springer, cham. stevens, c. e., xie, e., & peng, m. w. (2016). toward a legitimacy-based view of political risk: the case of google and yahoo in china. strategic management journal, 37(5), 945-963. sun, s. (2015). ‘阿里逾亿美元投资魅力惠 ’ . retrieved from: http://www.mei.com/hrbeforepage/mtbdxq?spm=a2o1p.8150593.2023578.9.1bc25218y7yjh e&page=news20150708_big tse, e. (2015). china's disruptors: how alibaba, xiaomi, tencent, and other companies are changing the rules of business. penguin. wang, d., ma, g., song, x., & liu, y. (2016). political connection and business transformation in family firms: evidence from china. journal of family business strategy, 7(2), 117-130. wei, s. j., xie, z., & zhang, x. (2017). from" made in china" to" innovated in china": necessity, prospect, and challenges. journal of economic perspectives, 31(1), 49-70. weihuan, z., arner, d. w., & buckley, r. p. (2015). regulation of digital financial services in china: last mover advantage. tsinghua china law review, 8, 25. wieczner, j. (2015). ‘alibaba: here’s why our mind-blowing numbers are real’. retrieved from: http://fortune.com/2015/09/23/alibaba-says-numbers-real-not-fake/ wu, x., ma, r., & shi, y. (2010). how do latecomer firms capture value from disruptive technologies? a secondary business-model innovation perspective. ieee transactions on engineering management, 57(1), 51-62. yeo, s. (2016). geopolitics of search: google versus china?. media, culture & society, 38(4), 591605. zeng, j., & glaister, k. w. (2016). competitive dynamics between multinational enterprises and local internet platform companies in the virtual market in china. british journal of management, 27(3), 479-496. zhang, l. (2015). ‘china's startup boom: 7 new firms every minute’. retrieved from: http://www.china.org.cn/business/2015-06/09/content_35775291.htm australasian journal of information systems koh & kwok 2018, vol 22, post publication review an investigation into failure of internet firms 5 copyright: © 2018 koh & kwok. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. paper title (paper title style) australasian journal of information systems ojokoh, akinsulire & isinkaye 2019, vol 23, research article academic staff performance evaluation system based on rough sets theory 1 an automated implementation of academic staff performance evaluation system based on rough sets theory bolanle ojokoh department of computer science federal university of technology akure nigeria victor akinsulire department of computer science federal university of technology akure nigeria folasade isinkaye department of computer science ekiti state university ado ekiti nigeria sadeisinkaye@gmail.com abstract the essence of evaluating employees’ performance in any tertiary institution is to realize the goals of the institution by measuring the contribution of each employee. effective human resource evaluation is paramount to the development of any organization. an automated method is needed to remove the limitations and facilitate the duties of human resource management. in this paper, rough set theory, a mathematical technique that deals with vagueness and uncertainty of imperfect data analysis is adopted for the evaluation of academic staff profile for promotion, grants and other academic purposes. the entire appraisal process of academic staff was translated into a web-based application where every user can fill, edit, update, and submit the annual performance evaluation report form. the indiscernible property of rough set approach is a unique factor in assessing every academic staff under the department and faculty/school by the head of department and dean respectively. with this, the system generates an information table handling all the necessary conditions for promoting academic staff and the corresponding decisions taken. a model for rating publications was proposed to reduce the sentiments involved in manual rating. reports were generated as output of each evaluation procedure. one hundred (100) dataset of academic staff of the federal university of technology, akure, nigeria was used in the experiment to evaluate the performance of the system. the results of the system obtained score were compared with the institution standard and it was found that the system scores were above standard, the average precision of the system shows 60% effectiveness which showed that the proposed system is efficient for academic performance evaluation process. keywords: academic staff, human resource management, performance evaluation, rough sets australasian journal of information systems ojokoh, akinsulire & isinkaye 2019, vol 23, research article academic staff performance evaluation system based on rough sets theory 2 1 introduction human resource management (hrm) in organizations is intended to maximize employees’ performance in the service of an employer's strategic objectives (bratton and gold, 2017). it is mainly concerned with the management of people within organizations by focusing on policies and systems. hrm is also charged with integrating human resource planning with overall organizational planning (brewster & hegewisch, 2017), which includes career development, training and compensation. others are performance evaluation, promotion, transfer and employee discipline (collings & woods, 2018). in order to accomplish the organization’s goals and support the organization strategies, human resources objectives and strategies must be developed. evaluation of staff is done manually by the hrm in tertiary institutions and this manual evaluation process is laden with inconsistent reports and its procedure lack standards. evaluation of employees of organizations (safarzadeh et al. 2014, gbadebo & adebayo, 2017) is a subtask of the human resource management, as such evaluation of the academic staff’s activities is conducted by this management and it is based on curriculum vitae assessment. the guidelines of this assessment are always aligned with the mission and goals of the institution in order to contribute towards its success. academic staff members’ evaluation has potentially dual impact on the performance of the institution and on individual staff members’ activity orientation and performance, given the effect on their career and progression in life (neogi et al. 2011, ranjan et al. 2015). human resource evaluation is an optimization problem and a multiple criteria decision making process. it is therefore usually impossible to deal with a decision in terms of a single criterion (liu & wei, 2000), bearing in mind that not every evaluation technique (widayati, 2013) is appropriate for a specific problem domain. a suitable technique must be applied to a particular situation to achieve the institution’s objectives. the evaluation techniques are divided into two categories: the objective and the subjective methods. the objective technique evaluates decisions mostly based on numerical data, thus conclusion can be fairly supported without any ambiguity. the subjective decisions need to be objectified by means of multi-criteria evaluation techniques (liu & wei, 2000). according to samuel et al (2014), computing tools help to organize, store and retrieve appropriate knowledge needed by hrms to deal with difficult managerial problems and provide appropriate decision support platforms for them. in this paper, the major criteria of evaluation are aggregated by a mathematical relation and the result of the aggregated value is used with other conditions for promotion via rough set theory (cekik .& telceken, 2018) which is implemented with hypertext pre-processor (php) for effective automated method that reduces sentiment in publication scoring while evaluating academic staff profiles. 2 related works several works have been carried out on the performance evaluation of employees in institutions. some of the techniques of evaluation used are discussed as follows. islam & bin mohd rasad, (2006) used analytic hierarchy process (ahp) to simplify complex evaluation problems into structural hierarchies. the major evaluation criteria were first identified, and consecutively subdivided into hierarchical levels. the weaknesses of the system are, it requires expert choice software to compute the weights of the criteria and subcriteria and it also requires substantial amount of time to obtain the overall performance scores. australasian journal of information systems ojokoh, akinsulire & isinkaye 2019, vol 23, research article academic staff performance evaluation system based on rough sets theory 3 neogi et al. (2011) proposed a system that was based on the articulation of cascading fuzzy inference system and centre of gravity (cog) defuzzification method. the fuzzy inference system (fis) module contains five fiss sub-modules in cascade named fuzzy communication block. the major weakness of the work is that an expert can easily modify the system’s inputs, including the set of fuzzy rules fullér et al. (2012) applied different fuzzy ordering techniques with fuzzy number to order the candidates, in order to help manager decide whether to employ, retain or reallocate employees according to the level of their performance. the operation of the system is limited to hiring and reallocation of employee, other functions of human resources management were not considered. okoye et al. (2013) applied neural networks to resolve some observed manual weaknesses, particularly in view of neural networks’ ability to learn and adapt, to resolve conflicts by collaboration, propagation and aggregation. numerical methods capable of analysing dependence structure of a multivariate population were used. human resource management metrics were used as the data modelling tool for evaluating performance of academic staff based on three main factors, namely: research, teaching and service. however, the flaw of the work is that the system developed used neural network model which cannot handle linguistic and vague information. bhosale & kamath (2013) developed a fuzzy inference system for different modules. the three modules include: fuzzy module for teaching, learning and evaluation related activities; fuzzy module for co-curricular, extension, professional development activities, and fuzzy module for research, publications and academic contributions. the system used the fuzzification method which comprises of the process of transforming crisp values into membership grades. the membership function is used to associate a grade to different linguistic terms. the weakness exhibited by the system is that it requires human experts to discover rules about data relationship. samuel et al (2014) made use of fuzzy logic based on the concept of fuzzy set theory, fuzzy ifthen rules, and fuzzy reasoning to model an expert system for human resource performance appraisal. however, the dataset used in the system were limited to the staff under consideration for promotion to senior lecturers alone, other staff were not considered and multiple users cannot access the system at the same time. macwan & sajja (2013) used fuzzy evaluation techniques to facilitate the performance appraisal process, the fuzzy logic allows reviewers to express themselves linguistically and draw definite conclusions from vague, ambiguous or imprecise information. the evaluation parameters used were not all equally important to organization levels. jamsandekar & mudholkar (2013) applied fuzzy inference technique in place of traditional approach to classify student scores according to the level of their performance, fuzzification of the input data was done by creating fuzzy inference system (fis) subject wise. the model however requires human expert to discover rule about data relationships as student’s performance evaluation needs intelligent adaptive and tutoring internet based system. banerjee & ghosh (2013) utilized fuzzy analytic hierarchy process with promethee-2 and topsis to recruit the best faculty in order to provide quality education to their students. the limitation of the work is that it focused on one particular faculty. australasian journal of information systems ojokoh, akinsulire & isinkaye 2019, vol 23, research article academic staff performance evaluation system based on rough sets theory 4 avazpour et al. (2013) applied fuzzy hybrid multiple criteria decision making approach with combination of different assessors' opinions on fuzzy ahp method for determining the weights of criteria. the weakness of the work is that it cannot be applied to academic staff evaluation huang (2014) employed high-order confirmatory factor analysis (cfa) method to evaluate the performance appraisal system of procurators by fuzzy comprehensive evaluation method based on the 360 degrees. the weakness of the system is that it does not constitute significant role in segmenting the appraisal indicators in order to enhance the operability of the indicator within system. nápoles et al. (2014) used a decision-making model called rough cognitive networks which combines the capability of rough set theory for handling inconsistent patterns, with the modelling and simulation features of fuzzy cognitive maps. the system is however exclusively designed for addressing decision-making problems concerning public transportation issues. the overall observation of former works shows that the system that are based on fuzzy logic and other existing techniques are prone to different problems restriction in the function of the system as well as slow response time which results in performance degradation of the entire evaluation process. therefore, there is a need for a fast and robust system that can support the entire evaluation process of any tertiary institution dealing with all the flaws of conventional techniques. 3 system design the architecture of the proposed system has been designed in such a way as to allow easy interaction between the system and its users. the system is also designed in such a way that it can support the entire evaluation process of any tertiary institution by dealing with all the shortcomings of the conventional approach. the general architecture of the proposed system is described in fig. 1. figure 1: architecture of the proposed system australasian journal of information systems ojokoh, akinsulire & isinkaye 2019, vol 23, research article academic staff performance evaluation system based on rough sets theory 5 3.1 the process of the proposed system the administrator is responsible for creating accounts for all categories of users in the system and he also has full control of how the system works. there are five major users of the system. the: administrator, lecturer, hod, dean and the vice-chancellor. the users use the system interface to access the system to check the authenticity of the user whether he/she is an authorized academic staff or not. when the authenticity of user has been confirmed, then the users’ profiles become active. the user profile stores the details of the five categories of users on the system. a lecturer accesses the system with his username and password, fills the electronic annual performance evaluation form and submits it for assessment. he checks the submitted form after evaluator1 which is the hod might have commented, he acknowledges the comment of the hod and comments where necessary and re-submits for recommendation and approval. it is only the lecturer that can edit his/her profile, other users can only pass comments or approve recommendations based on the information provided in the form submitted. the hod accesses the system through the user interface with his username and password. he also fills his assessment information. the hod has access to all the information filled by all the lecturers in his department. he can view their all personal information. he then uses the information filled by the lecturers to prepare his assessment. submitted data will be accessed by evaluator1 (the hod) for assessment in the relevant areas. the dean accesses the system via the user interface with his username and password. the deans have access to all the departments in his school/faculty as well as the information filled by the lecturers. the deans also have access to the lecturers assessment carried out by each head of a department. he then recommends the lecturer for any of the following: confirmation of appointment to retiring age, termination of appointment, promotion or annual increment and also the justification for such recommendations. finally, the vice chancellor accesses the system through the user interface with his username and password. he has access to all the faculties in the university and also the information filled by all lecturers, the assessment carried out by the head of the department and the recommendations of the dean. he then approves the recommendations on behalf of the a&pc. 3.2 quantitative evaluator the system auto-generates the quantitative scores for each academic staff based on the approved rules governing employee promotion of an institution. the quantitative evaluator aggregates all the obtained values of the score and stores it in the appropriate section of the electronic annual performance evaluation form in the database using the mathematical relation below: 𝐒𝐒k = ∑ 𝑋𝑋𝑖𝑖 𝑛𝑛 𝑖𝑖=1 1 where sk represents a given criteria considered when evaluating the performance of an academic staff and k = 1, 2, . . . , m. where m represents the number of criteria for evaluation. for every sk, there are set of sub-criteria, xi, where i = 1, 2. . . n where n is the number of subcriteria present in sk. the sum of all sk is the quantitative score of all the criteria which is used by the second part of the model, described in the following sub-section 3.2.1 the rough set model the rough set theory (rst) (zhang et al. 2016, gigović et al. 2017, pamučar et al.2017) was used to compute all the necessary conditions for promotion such as quantitative score (sk), number australasian journal of information systems ojokoh, akinsulire & isinkaye 2019, vol 23, research article academic staff performance evaluation system based on rough sets theory 6 of journal published, number of referred proceedings, number of journal published offshore, phd status and so on that were used in the proposed academic staff profile evaluation system. ʈ = (u, ω, vq, fq) q ɛ ω 2 where u is a finite set of staff to be evaluated, u = {x1, x2, …..xn}, where xi represents each of the academic staff. ω is a finite set of attributes (features), which represents conditions for promotion and decision taken, the attributes in ω are further classified into disjoint set, condition attribute a and decision attributes d. ω = a∪d, for each q ɛ ω 3 where a represents conditions for promotion when evaluating an academic staff. d represents decision that should be taken when evaluating an academic staff. ∪ in equation (3) represents union of the two disjoint sets of attributes. vq is a set of attribute values for q, for example the actual value for quantitative score of an academic staff. fq: uvq 4 fq represents an information function that assigns a particular value from domain of attributes to objects (academic staff) such that: fq (xi) ɛ vq ∀ xi ɛ u and q ɛ ω 5 (rq) indiscernibility relations with respect to a given q ɛ ω, the functions partitions the universe into a set of pair-wise disjoint subsets of u. rq = {x: x ɛ u ˄ f(x, q) = f (xo, q) ∀ xo ɛ u} 6 let a subset of the set of attributes, p ⊆ a. two samples x and y in u are indiscernible with respect to p if and only if f(x, q) = f(y, q) ∀ q ɛ p. the indiscernibility relation for all p ⊆ a is written as ind(p).u/ind(p) which is used to denote the partition of u given ind(p) as follows u/ind(p) = ⊗ {q ɛ p: u/ind (p) ({q})} 7 where a ⊗ b = {x ∩ y: ∀ q ɛ a, ∀ y ɛ b, x ∩ y ≠ {}} 8 ⊗ represents boolean function (∪, ∩) used to compute elementary set of p. the indiscernibility relation is adopted to handle academic staff having similar records of attributes for evaluation purpose. for example two or more staff in senior lecturers’ cadre and in same department must meet the same condition for promotion to readers’ cadre and so on. the rst use decision rules which are defined to be statements of the form “if c then d”, where the condition c is a set of elementary conditions connected by “and” and the decision d is a set of possible outcomes connected by “or”. the decision gives us the overall performance of the academic staff. from equation (3), a is a set of condition attributes compulsory for promotion of an academic staff with the corresponding range of values and labels and it is presented in table 1. australasian journal of information systems ojokoh, akinsulire & isinkaye 2019, vol 23, research article academic staff performance evaluation system based on rough sets theory 7 label attributes range of values a1 quantitative score for each academic staff 1 100 a2 number of journal papers published 0 500 a3 number of referred proceedings 0 -500 a4 number of journal papers published offshore 0 500 a5 phd status of an academic staff 0 or 1 a6 current level of an academic staff 1 – 6 (ga, al, lii, li, sl, rd) d = a7 class of decision 1 – 6 ( al, lii, li, sl, rd, prof) table 1: conditions and decision attributes of an academic staff note: each condition consists of seven features along with a label that denotes its decision class; with an instance of one of six possible classes: assistant lecturer, lecturer ii, lecturer i, senior lecturer, reader or professor. d is a set of decisions to be taken after carefully considering the needed attributes, a for promotion. nq represents not qualified. d1 represents promotion to assistant lecturer (al), d2 represents promotion to lecturer ii (lii), d3 represents promotion to lecturer i (li), d4 represents promotion to senior lecturer (sl), d5 represents promotion to reader (rd), and d6 represents promotion to professor (prof). y1 represents staff on graduate assistant cadre, y2 represents staff on assistant lecturer cadre, y3 represents staff on lecturer ii cadre, y4 represents staff on lecturer i cadre, y5 represents staff on senior lecturers’ cadre and y6 represents staff on readers’ cadre which is presented in table a1 (appendix a1) with conditional values of attributes. 3.2.2 classification of criteria the criteria identified in this research work for the evaluation of academic include: qualification (q), research and publication (r), teaching/professional duties (t/pd), length of service since last promotion or appointment (ls), and contribution to the university and nation (c) with assigned scores shown in the tables 2-5. criteria 1 (s1) attribute scores (15) qualification b.tech or b.sc 6 m.tech or m.sc 7 m.phil 8 p.hd 10 total qualification = highest qualification score + (no of professional qualification /100 *5) table 2: qualification score research and publication score = ∑ (j/2) + g + h 9 j = u + m + (% c) *% c 10 j = u + m + (% c * 0.3) * % c 11 where g represents number of m.tech/phd supervision (ongoing)*0.5, h represents the number of research in progress *0.5 and j represents the total sum of journals or proceedings scores which is represented in the algorithm below, where u represents score of authors’ position, m represents off/onshore score, c represents percentage contribution when it is a australasian journal of information systems ojokoh, akinsulire & isinkaye 2019, vol 23, research article academic staff performance evaluation system based on rough sets theory 8 journal and when it is a proceeding, percentage contribution is multiplied by 0.3 to scale down the score of proceeding to maximum of two. however, the system auto generates scores for the other criteria according to the scores shown in tables 2, 3, 4 and 5. criteria 3 (s3) attribute scores teaching/professional duties professor reader senior lecturer lecturer i lecturer ii assistant lecturer graduate assistant 23 21 19 18 17 16 5 table 3: teaching and professional duty score criteria 4 (s4) attribute range of scores length of service current year – year of last promotion xi < 2 2 ≤ xi < 5 xi ≥ 5 table 4: length of service score criteria 5 (s5) attribute scores contribution to the university and the nation professor substantive hod /reader/ dean ag hod /exam officer/pg rep/school rep others 4 3 2 1 table 5: score on contribution to the university and the nation 3.2.3 conditions necessary for promotion of an academic staff there are quite a number of conditions that are necessary for promotion of academic staff from one level to another level in terms of the minimum numbers of papers required, where the paper is to be published and also the minimum quantitative scores required in moving an academic staff from one level to another which the system considered. the system checks if the academic staff satisfies the necessary conditions for promotion with the conditional statements of the rough set theory using the relation below to compute the elementary set. this relation is a single module used by rough set to determine the status of each academic staff using the system. sample of the algorithm for the conditions are indicated as follows: algorithm for conditions of academic staff promotion if (quantitative score ≥ “22”) and (number of paper ≥ “0”) and (referred proceedings ≥ “0”) and (number of offshore publication ≥ “0”) and (phd status =”1”) and (employee cadre = “2”) then employee is qualified for promotion to lecturer ii australasian journal of information systems ojokoh, akinsulire & isinkaye 2019, vol 23, research article academic staff performance evaluation system based on rough sets theory 9 else if (quantitative score ≥ “22”) and (number of paper ≥ “1”) and (referred proceedings ≥ “1”) and (number of offshore publication ≥ “0”) and (phd status =”0”) and (employee cadre = “2”) then employee is qualified for promotion to lecturer ii else employee not qualified for lecturer ii end if ….. ….. end the system allows the evaluators to select the appropriate attributes and values that are necessary for awards, grants, and other academic purposes based on the performance of the academic staff either by quantitative score, impact on journal publication and so on. 4 system implementation the essence of the implementation phase is to translate the system design into source code. each component of the design is implemented as a program module that works together to achieve the aim of the research. all program codes were implemented using html (dreamweaver) with css, php, javascript and mysql. the html and the css tags were employed to structure the outlook of the web pages in the system, while java script codes were introduced to validate the data entered on the designed web pages. php has its object oriented features combining those of java and c# languages. it runs faster than java with higher speed. it is extremely easy to use in developing complex web applications, in a considerable period of time. php is also a scripting language with a large amount of users worldwide (kelvin et al. 2012). the local host environment characterized by windows apache, php, mysql (wamp) server was adopted in the work. windows operating system, apache http server, mysql relational database management system (rdbms), and php programming language. (wamp) is a model of web service, which is an open source application that allows users to run web based applications on their local machine just the same way the applications behave in an internet environment. 4.1 system implementation results some of the screenshots of the system implementation for the evaluation of academic staff are presented. fig. 2 shows the log in page. the login page is the first page that is displayed to the user of the system. this page is necessary to control and manage the various types of users in the system australasian journal of information systems ojokoh, akinsulire & isinkaye 2019, vol 23, research article academic staff performance evaluation system based on rough sets theory 10 figure 2. login page the publication list page is shown in fig. 3. it is where the system accepts all the various publications of an academic staff ranging from thesis, articles that have appeared in learned journals, papers already accepted for publication and published conference proceedings. figure. 3. publication list page fig. 4 shows the profile of the head of department, showing the details of lecturers whose profile could be evaluated and recommended for promotion or incremental as the case may be. australasian journal of information systems ojokoh, akinsulire & isinkaye 2019, vol 23, research article academic staff performance evaluation system based on rough sets theory 11 figure. 4. lecturers under a particular hod (evaluator 1) the assessment of a lecturer by the dean and remarks given is shown in fig. 5. figure 5. the recommendation of the dean on a lecturer the page where the administrator login to the system to access it at the backend is shown in fig. 6. the administrator has the ability to update the database from time to time in order to get the new academic staff added to the evaluation database. australasian journal of information systems ojokoh, akinsulire & isinkaye 2019, vol 23, research article academic staff performance evaluation system based on rough sets theory 12 figure 6. administrator login page 4.2 experimental evaluation of the system the experimental evaluation of the system was done with data collected from manual annual performance evaluation report. in the evaluation of the system, the dataset of an academic staff of different cadre was used and the quantitative score given by the system, current cadre, number of journals published, phd status, users (u) and number of publications offshore were considered. an academic staff that satisfies all the necessary conditions for promotion is automatically recommended by the system for promotion into the next cadre while those who are not qualified for promotion are recommended for step increment by the system. it is observed that some users of the system score above the futa standard score but they were not promoted due to other important conditions for promotion that they did not satisfy. 4.2.1 evaluation metrics the evaluation of the system was conducted using standard deviation metrics as a system of statistical procedure for the comparative analysis of the performance of user in any institution and also using equation 13, (akinyokun, 2002). 𝜎𝜎 = �∑ (𝑋𝑋− 𝜇𝜇)2 𝑛𝑛 13 x represents each user in the department, 𝛍𝛍 is the mean of the values, n represents number of values. the standard score d i.k.j of a human resource at the department is defined by d i.k.j = m i.k + (d i.k.j – a i.k) / s i.k 14 where mi.k represents the minimum score required by a user in the kth cadre of ith department. d i.k.j represents the obtained score of the jth user in the kth cadre of ith department. a i.k represents the mean of the obtained scores of all the staff in the kth cadre of ith department australasian journal of information systems ojokoh, akinsulire & isinkaye 2019, vol 23, research article academic staff performance evaluation system based on rough sets theory 13 s i.k represents the standard deviation of the raw scores of all the staff in the kth cadre of ith department. using equation 13, all users that belong to the same cadre were grouped according to their various departments: department a, department b, department c, and department d. their mean and standard deviation are presented in table 6. department a department b department c department d mean score 45.07 48.66 42.16 43.13 standard deviation 8.61 3.68 2.75 6.28 table 6: mean and standard deviation of obtained scores standard deviation computes how much an individual measurement should be expected to deviate from the mean on average. the smaller the standard deviation, the greater the consistency, predictability and quality of the system. it was discovered from fig. 7 that department c has the smallest standard deviation showing that the proposed system works closely with the standard of the institution. figure 7. standard deviation of the proposed system when equation 14 was applied, it shows that the average score of human resource standard score per department is relatively the same as the futa standard score using actual value or approximate value except users who are not yet due for promotion to next cadre or users whose other conditions for promotion were not met. however, table a2 (appendix a2) and fig. 8 show that the scores obtained by academic staff that used the system were above the two standards. 0 10 20 30 40 50 60 department a department b department c department d mean score standard deviation australasian journal of information systems ojokoh, akinsulire & isinkaye 2019, vol 23, research article academic staff performance evaluation system based on rough sets theory 14 figure 8. obtained score versus hr standard score per department/futa standard score 4.2.2 proposed system performance prediction the predictions of proposed system and their possibility scores for ten academic staff manually graded among the users were compared with the conventional scores manually giving. table a3 (appendix a3) and fig. 9 depict the value of the system performance status that shows whether the performance of each academic staff is accurately predicted or not when compared with the manual score. figure 9. system obtained score versus manual scores 4.2.3 precision evaluation for the proposed system precision is a measure that helps to evaluate the quality of an unordered set of retrieved items. average precision (ap) measures the average of un-interpolated precision values at every rank where relevant items are found (bestgen, 2015, zhai et al.2015). let uk represent the status of the prediction for the kth performance outcome such that uk = � 1(if the kth outcome is accurate 0(if the kth outcome is inaccurate the average precision of the proposed system is computed as follows: 0 10 20 30 40 50 60 u 3 u 4 u 8 u 9 u 11 u 12 u 13 u 16 u 22 u 30 u 31 u 49 u 50 u 51 u 62 u 68 u 69 u 83 u 84 u 90 system obtained scores standard futa score for next cadre hr standard score per department di.k.j 0 10 20 30 40 50 60 70 80 90 u1 u2 u3 u4 u5 u6 u7 u8 u9 u10 system obtained score manual score australasian journal of information systems ojokoh, akinsulire & isinkaye 2019, vol 23, research article academic staff performance evaluation system based on rough sets theory 15 𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑖𝑖𝑃𝑃𝑖𝑖𝑃𝑃𝑛𝑛 = 1 𝑛𝑛 ∑ 𝑈𝑈𝑘𝑘 𝑛𝑛 𝑘𝑘=1 𝑓𝑓𝑃𝑃𝑃𝑃 𝑎𝑎𝑎𝑎𝑎𝑎 𝑈𝑈𝑘𝑘 = 1 15 𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑖𝑖𝑃𝑃𝑖𝑖𝑃𝑃𝑛𝑛 = 𝐶𝐶𝑃𝑃𝐶𝐶𝑛𝑛𝐶𝐶 𝑃𝑃𝑓𝑓 𝐴𝐴𝑃𝑃𝑃𝑃𝐶𝐶𝑃𝑃𝑎𝑎𝐶𝐶𝑃𝑃 𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑖𝑖𝑃𝑃𝐶𝐶𝑖𝑖𝑃𝑃𝑛𝑛𝑃𝑃 𝑇𝑇𝑃𝑃𝐶𝐶𝑎𝑎𝑎𝑎 𝑛𝑛𝐶𝐶𝑛𝑛𝑛𝑛𝑃𝑃𝑃𝑃 𝑃𝑃𝑓𝑓 𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑖𝑖𝑃𝑃𝐶𝐶𝑖𝑖𝑃𝑃𝑛𝑛𝑃𝑃 = 6 10 = 0.6 while the average imprecision of the proposed system is computed as follows 𝐼𝐼𝑛𝑛𝐼𝐼𝑃𝑃𝑃𝑃𝑃𝑃𝑖𝑖𝑃𝑃𝑖𝑖𝑃𝑃𝑛𝑛 = 𝐶𝐶𝑃𝑃𝐶𝐶𝑛𝑛𝐶𝐶 𝑃𝑃𝑓𝑓 𝐼𝐼𝑛𝑛𝑎𝑎𝑃𝑃𝑃𝑃𝐶𝐶𝑃𝑃𝑎𝑎𝐶𝐶𝑃𝑃 𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑖𝑖𝑃𝑃𝐶𝐶𝑖𝑖𝑃𝑃𝑛𝑛𝑃𝑃 𝑇𝑇𝑃𝑃𝐶𝐶𝑎𝑎𝑎𝑎 𝑁𝑁𝐶𝐶𝑛𝑛𝑛𝑛𝑃𝑃𝑃𝑃 𝑃𝑃𝑓𝑓 𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑖𝑖𝑃𝑃𝐶𝐶𝑖𝑖𝑃𝑃𝑛𝑛𝑃𝑃 = 4 10 = 0.4 also, from the result of the precision measure, the proposed system has 0.6 probability of predicting accurately the score of an academic staff and 0.4 chances of predicting wrongly the score of an academic staff. 4.3 comparison of result with related works precision method was used by (samuel and omisoore, 2014) for the same cadre of academic staff; they got 0.78 accurate predictions and 0.22 inaccurate prediction of an academic staff appraisal. table 7 shows a clear difference between the proposed system and (samuel et al, 2014) system according to the categories of comparison between the two systems, the proposed system shows a better performance than samuel et al (2014) system owing to the fact that the system used both comparative analysis as well as precision evaluation metrics. categories of comparison samuel et al system proposed system accurate prediction (%) 0.78 0.6 category of academic staff one cadre all cadre number of users per time one user simultaneous/multiple users number of system used one system many systems system requirements limited requirement more hardware & software requirements dataset used 50 100 table 7: comparison of proposed system with related work 5 conclusions this work presents a mathematical relation and rough set model to effectively evaluate academic staff of an institution for promotion. the system allows users to interact online and presents a faster system void of bias and sentiments for annual performance evaluation. rough set theory was adopted to evaluate the criteria for promotion vis-à-vis qualifications, research and publications, teaching and professional duties, length of service and contribution to the university and nation. this work particularly proposes a model that could be generally acceptable for evaluating publications to eliminate the stress it takes manually, considering publication venue, number of authors and contribution of each. experiments were carried out to evaluate the performance of the proposed system. the system obtained score was compared with the institution standard and it was found that the system scores were above standard; the average precision of the system shows 60% effectiveness using precision measurement. australasian journal of information systems ojokoh, akinsulire & isinkaye 2019, vol 23, research article academic staff performance evaluation system based on rough sets theory 16 references akinyokun, o. c. (2002), “neuro-fuzzy expert system for evaluation of human resource performance”, first bank of nigeria plc endowment fund lecture, the federal university of technology akure, nigeria. avazpour, r. ebrahim,i e. fathi, m. r. (2013), “a 360 degree feedback model for performance appraisal based on fuzzy ahp and topsis”, international journal of economy, management and social sciences, 2(11), 969-976. banerjee, r. ghosh, d. n. (2013), “faculty recruitment in engineering organization through fuzzy multi-criteria group decision making methods”, international journal of u-and eservice, science and technology. 6(4): 139-154. bestgen, y. (2015), “exact expected average precision of the random baseline for system evaluation”, the prague bulletin of mathematical linguistics. 103(1): 131-138. bhosale, g. a. kamath, r. s. (2013), fuzzy inference system for teaching staff performance appraisal”, international journal of computer and information technology, 2(3): 381-385. bratton, j., & gold, j. (2017), “human resource management: theory and practice” palgrave. brewster, c., & hegewisch, a. (eds.). (2017), “ policy and practice in european human resource management: the price waterhouse cranfield survey”, taylor & francis. cekik, r.& telceken, s.(2018), “a new classification method based on rough sets theory”, soft computing. 22(6): 1881-1889. collings, d. g., wood, g. t., & szamosi, l. t. (2018), “human resource management: a critical approach”, in human resource management (pp. 1-23). routledge. fullér, r., canós-darós, l., & daros, m. j. c. (2012). transparent fuzzy logic based methods for some human resources problems. asepuma. asociación española de profesores universitarios de matematicas aplicadas a la economia y la empresa, in rect@ (vol. 13, no. 1, pp. 27-41). gbadebo, a. o., & adebayo, a. o. (2017), “an agent-based performance evaluation system”, international journal of computer (ijc), 27(1): 28-38. gigović, l., pamučar, d., bajić, z., drobnjak, s.(2017), “ application of gis-interval rough ahp methodology for flood hazard mapping in urban areas”, water. 9(6):360. huang, y. m. (2014), “research on appraisal system of procurator performance by using highorder cfa model”, mathematical problems in engineering. islam, r., & bin mohd rasad, s. (2006), “employee performance evaluation by the ahp: a case study”, asia pacific management review, 11(3): 163-176. jamsandekar, s. s. mudholkar, r r. (2013), “performance evaluation by fuzzy inference technique”, international journal of soft computing and engineering (ijsce). .3(2): 22312307. kelvin, y., simon, m., kelly, s., diana, m., fred, b., alex, w. (2012), “php and mysql. novice to ninja”, 5th ed., copyright © 2012 site point pty ltd. australasian journal of information systems ojokoh, akinsulire & isinkaye 2019, vol 23, research article academic staff performance evaluation system based on rough sets theory 17 liu, p. h., & wei, c. c. (2000), “a group decision making technique for appraising the performance of organizations”, international journal of the computer, the internet and management, 8(2): 39-49. macwan, n., sajja, d.p.s. (2013), “performance appraisal using fuzzy evaluation methodology”, international journal of engineering and innovative technology, 3(3): 324329. nápoles, g. grau, i. vanhoof, k. bello, r (2014), “hybrid model based on rough sets theory and fuzzy cognitive maps for decision-making”, in international conference on rough sets and intelligent systems paradigms, 169-178. neogi, a., mondal, a. c., & mandal, s. k. (2011), “a cascaded fuzzy inference system for university non-teaching staff performance appraisal”, journal of information processing systems, 7(4): 595-612. okoye, u. m. n. adewole, a. p. folorunso, o. & ezike, j. o. j. (2013), “neural network model for performance evaluation of academic staff of tertiary institutions”, international journal of applied information systems. 5(1): 1-9. pamučar, d. gigović, l. bajić, z. and janošević, m. (2017), “location selection for wind farms using gis multi-criteria hybrid model: an approach based on fuzzy and rough numbers”, sustainability. 9(8): 1315. ranjan, r., chatterjee, p., & chakraborty, s. (2015), “evaluating performance of engineering departments in an indian university using dematel and compromise ranking method”,. opsearch, 52(2): 307-328. samuel, o. w., omisore, m. o., & atajeromavwo, e. j. (2014), “online fuzzy based decision support system for human resource performance appraisal. measurement”, 55: 452-461. safarzadeh, kashani f. h. sabaghi nadooshan, m. e. (2014), “the effects of biorhythm cycles on the employees job performance”, international journal of basic sciences & applied research. 3: 115-128. widayati, q. (2013). employee performance evaluation using the ahp with expert choice software (case study: pt. kereta api indonesia). pp. 444-450. zhai, c. cohen, w. w. lafferty, j. (2015), “beyond independent relevance: methods and evaluation metrics for subtopic retrieval”, in acm sigir forum, june 2015, 49(1): 2-9. zhang, q. xie. q. wang, g. (2016), “a survey on rough set theory and its applications”, caai transactions on intelligence technology, 1(4): 323-333. australasian journal of information systems ojokoh, akinsulire & isinkaye 2019, vol 23, research article academic staff performance evaluation system based on rough sets theory 18 appendix a1 criteria / condition attributes decision staff & cadre (y) a1 a2 a3 a4 a5 a6 a7 a8 = d y1 7 – 21 < 7 ≥0 ≥0 ≥0 ≥0 ≥0 ≥0 ≥0 ≥0 1 1 ≥0 ≥0 d1 nq y2 22 – 27 22 – 27 < 22 22 – 27 22 – 27 22 – 27 ≥0 ≥1 ≥0 0 0 ≥1 ≥0 ≥1 ≥0 0 ≥1 0 ≥0 ≥0 ≥0 ≥0 ≥0 0 1 ≥0 ≥0 0 0 0 2 2 2 2 2 2 ≥0 ≥0 ≥0 ≥0 ≥0 ≥0 d2 d2 nq nq nq nq y3 28 40 28 – 40 28 – 40 28 – 40 < 28 28 40 28 40 28 40 ≥2 ≥3 ≥4 ≥3 ≥0 3 2 3 ≥2 ≥0 ≥0 ≥2 ≥0 0 1 1 ≥0 ≥0 ≥0 ≥0 ≥0 0 0 ≥0 1 1 ≥0 ≥0 ≥0 0 ≥0 0 3 3 3 3 3 3 3 3 ≥0 ≥0 ≥0 ≥0 ≥0 ≥0 ≥0 ≥0 d3 d3 d3 d3 nq nq nq nq y4 41 – 51 41 – 51 < 41 41 – 51 41 – 51 41 51 ≥8 ≥6 ≥6 ≥6 ≥8 5 ≥0 ≥4 ≥0 ≥0 ≥0 ≥0 ≥2 ≥2 ≥0 ≥0 ≥0 ≥0 1 1 1 0 0 0 4 4 4 4 4 4 ≥1 ≥1 ≥1 ≥1 ≥1 ≥1 d4 d4 nq nq nq nq y5 52 – 60 52 – 60 < 52 52 60 ≥13 ≥16 ≥13 12 ≥0 ≥4 ≥0 ≥4 ≥4 ≥4 ≥4 ≥4 1 1 1 1 5 5 5 5 ≥2 ≥2 ≥2 ≥2 d5 d5 nq nq y6 61 – 100 61 – 100 < 61 61 – 100 ≥20 ≥18 ≥18 17 ≥0 ≥4 ≥4 ≥0 ≥6 ≥6 ≥6 ≥6 1 1 1 1 6 6 6 6 ≥3 ≥3 ≥3 ≥3 d6 d6 nq nq table a1: rough set information system for evaluation of an academic staff australasian journal of information systems ojokoh, akinsulire & isinkaye 2019, vol 23, research article academic staff performance evaluation system based on rough sets theory 19 appendix a2 users (u) system obtained scores standard futa score for next cadre hr standard score per department di.k.j u3 42.53 41 41.29 u4 47.50 41 41.28 u8 39.21 41 40.32 u9 45.84 41 41.09 u11 50.25 41 41.60 u12 42.35 41 39.29 u13 50.33 41 41.45 u16 46.81 41 40.49 u22 52.43 41 42.02 u30 51.39 41 41.74 u31 42.29 41 41.05 u49 42.00 41 40.94 u50 39.41 41 40.00 u51 47.17 41 42.82 u62 39.93 41 40.19 u68 42.74 41 40.94 u69 55.32 41 42.94 u83 38.30 41 40.23 u84 40.30 41 40.55 u90 39.00 41 40.34 table a2: obtained score versus hr standard score per department australasian journal of information systems ojokoh, akinsulire & isinkaye 2019, vol 23, research article academic staff performance evaluation system based on rough sets theory 20 appendix a3 users system obtained score manual score difference in score system performance status u1 73.74 74.48 0.74 1 u2 53.76 71.46 17.7 0 u3 42.53 52 9.47 0 u4 47.50 54.15 6.65 0 u5 85.05 82 3.05 1 u6 26 29 3 1 u7 61.26 63.18 1.92 1 u8 39.21 38 1.21 1 u9 45.84 40.5 5.34 0 u10 31.36 29 2.36 1 table a3: system performance prediction of an academic staff note: a value of one (1) represents accurate prediction when the difference between the system obtained score and the manual score is plus or minus 3, while zero (0) represents inaccurate prediction when the difference between the system obtained score and manual score is not within plus or minus 3. copyright: © 2019 ojokoh, akinsulire & isinkaye. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: http://dx.doi.org/10.3127/ajis.v23i0.2033 http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 related works 3 system design 3.1 the process of the proposed system 3.2 quantitative evaluator 3.2.1 the rough set model 3.2.2 classification of criteria 3.2.3 conditions necessary for promotion of an academic staff 4 system implementation 4.1 system implementation results 4.2 experimental evaluation of the system 4.2.1 evaluation metrics 4.2.2 proposed system performance prediction 4.2.3 precision evaluation for the proposed system 4.3 comparison of result with related works 5 conclusions paper title (paper title style) australasian journal of information systems poulsen, fosch-villaronga & burmeister 2020, vol 24, selected papers from the 2019 aice conference cybersecurity codes and lgbtiq+ elderly robots 1 cybersecurity, value sensing robots for lgbtiq+ elderly, and the need for revised codes of conduct adam poulsen charles sturt university australia apoulsen@csu.edu.au eduard fosch-villaronga leiden law school leiden university the netherlands oliver k burmeister charles sturt university australia abstract until now, each profession has developed their professional codes of conduct independently. however, the use of robots and artificial intelligence is blurring professional delineations: aged care nurses work with lifting robots, tablet computers, and intelligent diagnostic systems, and health information system designers work with clinical teams. while robots assist the medical staff in extending the professional service they provide, it is not clear how professions adhere and adapt to the new reality. in this article, we reflect on how the insertion of robots may shape codes of conduct, in particular with regards to cybersecurity. we do so by focusing on the use of social robots for helping lgbtiq+ elderly cope with loneliness and depression. using robots in such a delicate domain of application changes how care is delivered, as now alongside the caregiver, there is a cyber-physical health information system that can learn from experience and act autonomously. our contribution stresses the importance of including cybersecurity considerations in codes of conduct for both robot developers and caregivers as it is the human and not the machine which is responsible for ensuring the system’s security and the user’s safety. keywords: aged care, ai, ethics, lgbtiq+, healthcare robots, responsibility, value sensitive design. 1 introduction the use of robots and artificial intelligence (ai) is blurring professional delineations: aged care nurses work with lifting robots, tablet computers, and intelligent diagnostic systems, and health information system designers work with clinical teams. while robots assist the medical staff in extending the professional service they provide and robotics professionals are having closer interactions with the healthcare environment, it is not clear how professions adhere and adapt to the new reality. professional codes of conduct developed in isolation fail to recognise the interplay of systems, stakeholders, and values. since technologies are not value-neutral, they shape values and values shape them back and, in turn, these change the way we conceive the world, a cyber-attack to these technologies could have disastrous consequences for the user and alter the effectiveness of the task performed by the robot. australasian journal of information systems poulsen, fosch-villaronga & burmeister 2020, vol 24, selected papers from the 2019 aice conference cybersecurity codes and lgbtiq+ elderly robots 2 security vulnerabilities in robots raise significant concerns for manufacturers and programmers, but especially for those who interact with them in a sensitive domain of application such as healthcare. if an attacker can compromise the controlling of the machines or have effects on the production chain (quarta et al., 2017), any malfunctioning or any cybersecurity attack to a healthcare robot may affect the health, wellbeing, and safety of people, something that agencies like the food and drug administration (fda) in the u.s. identify as an unresolved, primary concern (fda, 2019). in an aged care setting where robots interact in close, direct contact with the elderly, any malfunctioning can result in a disastrous outcome. ai in medicine may improve, for instance, diagnoses made by humans (razzaki et al., 2018), but they may have an over-focus on data and disregard the context, dismissing the value of ambiguity in observed phenomena. missing these aspects could challenge, in turn, the correctness of the decision that might affect the wellbeing of the person. since cybercrime operates at the speed of light and traditional law enforcement efforts operate at a much lower rate, questions about prevention and remedies, including distribution of responsibility in highly autonomous environments (johnson, 2015), abound when we use and develop technology that may have a direct impact on a person’s wellbeing (datteri, 2013; fosch-villaronga, 2019). the delicacy of the domain of application, and the potential negative consequences these technologies could have demands for a multi-layered governance strategy that might take various forms, including guidelines, policies, standards, or codes of conduct. in this article, we argue that including cybersecurity considerations in codes of conduct for both robot developers and caregivers is imperative and of vital importance, as it is the human and not the machine which is responsible for ensuring the system's security and the user's safety (pasquale, 2017). still, although much work has gone into attempts to create universal codes of conduct (al-saggaf, burmeister, & schwartz, 2017; burmeister, 2013, 2017; capurro & britz, 2010), more work is needed to understand how those principles affect the work of designers and developers of ai-driven healthcare robots. health information systems (is) are not value-neutral and can have positive and negative impacts on the values within and arising from the ecosystem where these technologies are implemented (legassick and harding, 2017; friedman & hendry, 2019). to identify the implications for cybersecurity in codes of conduct for healthcare ai and robots, we examine data from an lgbtiq+ elderly and care robot study. using value sensitive design (vsd), we demonstrate how cyberattacks affect the diversity of values and value interpretations. we stress that current codes of conduct need to be revised to account for the diversity of values. our contribution seeks to show how cybersecurity vulnerabilities of healthcare robot technologies affect users' values, and highlight the critical role and responsibility designers and developers have to ensure the user's overall safety, not only the system's security. furthermore, this work shows how codes of conduct should be revised given the impacts of healthcare robot vulnerabilities on users’ values in the cyber and physical world. 2 literature review a recent literature review sheds light on the contributions made to health is by robotic care (miah, shen, lamp, kerr, & gammack, 2019). this multidisciplinary research involves numerous forms of information processing to improve data acquisition, retrieving, analysing, processing, displaying, and using the information in health decision making. their literature australasian journal of information systems poulsen, fosch-villaronga & burmeister 2020, vol 24, selected papers from the 2019 aice conference cybersecurity codes and lgbtiq+ elderly robots 3 analysis showed that, concerning healthcare, workforce challenges (as seen particularly in aged care) are increasingly leading to innovative technological solutions, including robotic and ai-driven care. robotics have increased productivity and resource efficiency in the industrial and retail sectors, and now there is an emerging interest in realizing a comparable transformation in other areas, including healthcare (cresswell, cunningham-burley, & sheikh, 2018). however, a healthcare system delivered by or with the help of robots is not straightforward and raises many questions. these questions range from how safe robotized care is (pino, boulay, jouen, & rigaud, 2015; fosch-villaronga, 2019), how these robots enhance or diminish the dignity of users (sharkey, 2014; zardiashvili & fosch-villaronga, 2020), as to how the work of practitioners change in light of these new robotic systems (melkas, hennala, pekkarinen, & kyrki, 2020). indeed, inserting robots in healthcare affects the whole ecosystem, which includes primary (direct robot users, clinicians, caregivers), secondary (robot makers, environmental service workers, health administrators), and tertiary stakeholders (policymakers, insurers, and advocacy groups). in this contribution, we focus on how robot makers' codes of conduct should be revised in light of the impacts healthcare robots’ vulnerabilities have on users’ values. 2.1 cyberattacks and vulnerabilities of healthcare robots cybersecurity addresses protection from external, unintended penetration, or malicious disruption, and aims to safeguard the confidentiality, integrity, and availability of is (thomas, burmeister, & low, 2019). cyberattacks on robotic and ai-driven technologies allow the materialization of attacks that go beyond the cyber world, and this deserves special attention in healthcare settings because “vulnerabilities could allow unauthorized users to remotely access, control, and issue commands to compromised devices, potentially leading to patient harm” (fda, 2019). the remote hacking into a robot may be used to confuse or even attack a patient, steal the identity of a doctor, or to induce undesirable behaviours from the patient (clark, doran, & andel, 2017). a malicious virus delivered using social engineering could manipulate the output from a diagnosis decision support tool. moreover, the use of backdoors in outdated operating systems of robots or medical devices might allow the stealing of sensitive information about the patient (coronado and wong, 2014). patient-centred healthcare culture may sometimes undermine the importance of security, although iss dominate the healthcare system. password sharing amongst healthcare workers is an example of this culture, challenging the security of the systems and the privacy of users (martin, martin, hankin, darzi, & kinross, 2017). since any system connected to the internet is subject to cyberattacks, however, the continuous use of cyber-physical systems in the healthcare sector demands robust cybersecurity mechanisms that can ultimately ensure patients’ safety. moreover, the growing interconnectivity and integration of healthcare technologies open multiple points of entry for cyberattacks, providing attackers with remote access to various interconnected systems from one access point, allowing attacks to go often unnoticed. cyberattacks on interconnected systems (with a denial of service attack, for instance) can harm a healthcare facility by disrupting the operation of networked medical devices and the integrity of information (coronado and wong, 2014). the possibility to extend home care via care robots further exacerbates this panorama. users usually fail to recognize that the robot is not the only relevant unit in the ecosystem, but that other information flows happen in the background (fosch-villaronga, felzmann, ramosaustralasian journal of information systems poulsen, fosch-villaronga & burmeister 2020, vol 24, selected papers from the 2019 aice conference cybersecurity codes and lgbtiq+ elderly robots 4 montero, & mahler, 2018; fosch-villaronga and millard, 2019). coupled with the strong industry push for the development of trustworthy robots and ai systems (hleg ai, 2019; floridi, 2019), the little knowledge on the overall functioning of the robot calls for more than just a precautionary approach when it comes to cybersecurity. 2.2 caring for lgbtiq+ elderly culture shapes a person’s value system; one of the most significant influences on an individual’s values is each person’s cultural background (drayton & weston, 2015; horton, tschudin, & forget, 2007; huang, teo, sánchez-prieto, garcía-peñalvo, & olmos-migueláñez, 2019; liu, volcic, & gallois, 2014; sunny, patrick, & rob, 2019). tenenbaum (2011) describes the lgbtiq+ community as a culture with unique values, concerns, needs, and critical and experiential interests in healthcare. many lgbtiq+ persons explicitly seek out services that are lgbtiq-friendly and healthcare professionals who are sensitive to the needs and values of their community (jann, edmiston, & ehrenfeld, 2015). the use of social robots could help members of this community cope with loneliness and depression in the same way they have benefited the broader ageing population (birks, bodak, barlas, harwood, & pether, 2016; carter-templeton, frazier, wu, & h. wyatt, 2018; khosla et al., 2012; moyle, jones, & sung, 2020). however, system vulnerabilities may lead to just the opposite. a man-in-the-middle attack or an sql injection that captures health data could reveal lgbtiq+ elders undisclosed gender identity or sexual orientation, which is considered sensitive data in most privacy regulations. wardriving could expose and abuse weak or open wi-fi wireless networks in aged care facilities and access health records of lgbtiq+ elders. hijacking a companion robot could lead unintended users to vocalize offensive, anti-lgbtiq+ language, which could seriously endanger the trust the user put into the system. a distributed denial-of-service attack on a networked telepresence robot set up to provide a social network for lgbtiq+ elders experiencing loneliness could seriously increase social isolation. 2.3 codes of conduct for a robotized healthcare despite codes of professional conduct that ensure workers respect the welfare of and sensitivity to lgbtiq+ persons, experiences of professional care have not lived up to these (bennett et al., 2017). this has led to this group being “significantly more likely to delay or avoid necessary medical care compared with heterosexuals” (29% versus 17%, respectively) (khalili, leung, & diamant, 2015). whether the inclusion of robots would change this panorama is a question that remains unanswered. considering the complexity and the potential implications of robot technologies, the european parliament proposed in 2017 a code of conduct for robotics engineers (european parliament, 2017). the ethical code was an all-embracing sector framework directed towards realizing the development of robot technologies in compliance with european law. the framework included the principles of bioethics, mainly beneficence, non-maleficence, autonomy, and justice, and also the need to respect the dignity, privacy, and safety of humans (european parliament, 2017). the parliament stressed the importance of considering humans and not robots as responsible agents to comply with fundamental rights, work with precaution and inclusiveness, maximize benefit, and minimize harm. this human-centred responsibility is shared among scholars (bryson, diamantis, & grant, 2017; floridi et al., 2018) and also among the ai principles charts and governance documents around the world (fjeld et al., 2019). australasian journal of information systems poulsen, fosch-villaronga & burmeister 2020, vol 24, selected papers from the 2019 aice conference cybersecurity codes and lgbtiq+ elderly robots 5 fjeld et al. (2019) conclude that the majority of ethical and rights-based approaches in the governance of ai focus on the protection of human rights, promotion of human values, professional responsibility, human control of technology, fairness and non-discrimination, transparency and explainability, safety and security, accountability, and privacy. however, while the ai principle of professional responsibility appears to be most prominent in google (google, 2019), and to some degree in other organizations like tesla, iti, university of montreal, ieee, future of life institute, global network initiative, smart dubai, and the european high-level expert group on ai, the principle it does not appear in the ai principles of telefonica, microsoft ai principles, the sage ethics of code, the european ethical charter on the use of ai in judicial systems, seeking ground rules for ai, the principles to promote feat ai in the financial sector, ai in the uk, ai for europe, ai at the service of citizens (italy), white paper on ai standardization (china), preparing for the future of ai (us nstc), and the think20 future of work and education for the digital age (field et al., 2019). a lack of professional responsibility coupled with a continuous focus on trustworthy ai (hleg ai, 2019) risks diminishing scrutiny of professionals and undermines certain societal obligations of ai producers, shifting accountability further towards ai systems and away from professionals (jobin, ienca, & vayena, 2019). global governance charts focus more on the principle of non-maleficence rather than beneficence, appearing that “issuers of guidelines are preoccupied with the moral obligation to prevent harm” instead of promoting those values (jobin, ienca, & vayena, 2019). in this sense, the importance of having professional codes of conduct that ground the work of robot makers in respect of fundamental rights becomes paramount. 3 methods the present constructivist study is a part of a larger project which aims to improve lgbtiq+ aged care through the use of care robots. the project suggests the use of value sensing robots to make person-centred, value-driven decisions in situ, as limited by a framework guaranteeing duty of care, professional ethics, and law. thirty-two people were interviewed about aged care, social robots, and loneliness. we followed a purposive sample that included healthcare and lgbtiq+ service provider professionals, robot designers, and seventeen australian lgbtiq+ elders (65+) (twelve gay men, two lesbian women, one gay gender-fluid person, one lesbian non-binary person, and one bisexual non-binary person). only the interviews of the seventeen australian lgbtiq+ elders are analysed here. interview length was determined by each participant, the shortest was thirty minutes and the longest was one hour and forty minutes. participants were located in rural and metropolitan new south wales and victoria, as well as metropolitan south australia and queensland. through semi-structured interviews, participants were questioned about the lgbtiq+ experience of aging, aged care, social isolation, and loneliness, as well as the older lgbtiq+ community’s values. the interviews were transcribed and analysed using thematic analysis with qsr nvivo 12, a software package for managing qualitative data. the lead investigator analysed all interviews. a selection of the interviews was also analysed independently by another investigator for inter-rater reliability, with the two investigators discussing discrepancies in analysis, to ensure the reliability of interpretation of the results. ethics australasian journal of information systems poulsen, fosch-villaronga & burmeister 2020, vol 24, selected papers from the 2019 aice conference cybersecurity codes and lgbtiq+ elderly robots 6 approval from the university and from participating lgbtiq+ communities, from which participants were recruited for this and the more extensive study, was obtained. the part of the study presented here focuses on the impressions of users concerning social robots and loneliness, with a particular focus on cybersecurity aspects. the findings in the next section have implications for codes of conduct for ai-driven and robotic healthcare systems. 4 findings four themes emerged from the analysis of the interviews. they were privacy, safe systems, internet connectivity, and determinants of security for lgbtiq+ elders, see table 1 below. exemplary quotations addressing cybersecurity considerations are explored below. themes categories privacy dignity choice historical and contemporary discrimination safe systems safety safe physical locations emergency help with robots internet connectivity robots disconnected from the internet online scams trust determinants of security for lgbtiq+ elders appreciating difference lgbtiq+ connectivity & community not tolerance, acceptance lgbtiq-friendly service provision table 1. themes and the categories of the interviews about aged care, social robots, and loneliness 4.1 theme 1: privacy several participants expressed concerns for choice and dignity regarding privacy: can you turn it off, like sometimes you just feel like not talking to anybody and just go i’m not interested. (67, gay gender-fluid person) where having a monitor and somebody observing you and you can talk to and they can move the thing around the house to make sure … it takes away a certain amount of privacy … probably your dignity, because you could be there terrified to do anything, because who’s watching me? (68, gay man) other privacy concerns directly related to being lgbtiq+ and historical and contemporary discrimination also emerged. that's what shut me off about the equal marriage plebiscite. i just felt, i thought it was so rude to ask straight people to say whether they thought it was okay for me and my friends to get married. it had nothing to do with them. (65-70, gay man) you watched the way you walked, you watched the way you spoke, you didn’t want anyone to realize your secret. and my family, i didn’t come out to my family until i was 42 and i was australasian journal of information systems poulsen, fosch-villaronga & burmeister 2020, vol 24, selected papers from the 2019 aice conference cybersecurity codes and lgbtiq+ elderly robots 7 dying, i only came out to them because everyone thought i was about to die so we had to tell mom and dad. so, it was just, the secrecy was just terrible. (66, gay man) many participants expressed fears of discrimination and highly prioritized privacy. there are existing processes in healthcare, such as an open-door policy that allows healthcare professionals to enter a person’s private room with little warning, perpetuating a system in which privacy is lacking. the introduction of healthcare robots may also remove choice and lack historical awareness of the lgbtiq+ experience, creating a fear of discrimination and heightening the need for privacy. 4.2 theme 2: safe systems common concerns among participants for safety and safe physical locations are seen in the following exemplary quotations. we value highly how to keep ourselves safe. you know, i guess that's a no brainer. because i know people who have died. i know people who have been seriously brutalized by police. you know, that's not good. but we value security if we can find it. (65, lesbian nonbinary) it's just sometimes i said to myself, “go back into the closet, [name], hide. that will keep you safe. let people think whatever they want to think.” (65, lesbian nonbinary) emergency help with robots was another category identified. multiple participants noted the practical advantages of having a robot which could aid in calling for emergency help. being the manager of the village, you have a key to everybody’s unit – well, master key to everybody’s unit so to get in in emergency situations. we got in and the neighbour came with me and i said, “just stay with me but behind me.” we found the lady on the kitchen floor, she had been there for nearly 12 hours … so, one of the things i would like to see included in your robot thing is if that particular person does have a fall, the robot can set off the alarm … that would be a very good thing to have, particularly people who are single and live alone. (68, gay man) 4.3 theme 3: internet connectivity internet connectivity was a contentious issue. some participants noted the usefulness of having a system which is connected to the internet to help connect them to others. however, as highlighted here, some participants were troubled by the idea of having robots in their home and connected to the internet for fear of hacking and having information stolen or having interactions with the robot influenced by a hacker. have to be through somebody like some sort of maybe aged care organization that can come and install them, you know what i mean? ... like the internet, some people can, you know, you hack into those and people can do some terrible things. (70, gay man) if you get the artificial intelligence to work not link to the net … it's having the conversation and the internet not knowing where it's going to end up, or where its security is. if you can write is so that its security is something that we feel comfortable with or i feel comfortable with, then i will engage with it. if i'm worried that that conversation is going to be polluted some way into, then i would be very wary. (66, lesbian woman) one participant had been the victim of an online scam in the past. australasian journal of information systems poulsen, fosch-villaronga & burmeister 2020, vol 24, selected papers from the 2019 aice conference cybersecurity codes and lgbtiq+ elderly robots 8 i was scammed, would you believe it, not long after i got the laptop when i was searching for work and i was taken like hook line and sinker. fortunately, i got the money that i put up front. i really was taken for a ride … you wouldn't want anything like that happening. (70, gay man) trust was also identified as a category in the internet connectivity theme. that is, trust in other persons online. having a conversation with a robot is fine … what i would like is a chat machine that you can trust as opposed to the internet where you don't… i wouldn't trust chatting on the internet. (66, lesbian woman) trust also came up in relation to lgbtiq+ elders with no family or partner, such as the potential of having to trust someone with handling their finances. this concern might be applied to the cybersecurity of healthcare systems, i.e. having to place trust in a virtual or robotic assistant helping with online banking. look at the lgbt elders, for instance, like myself, that's not a relationship, that has family that are on the other side of sydney. yes, i am alone. i have concerns about my life in the future, about if i got short term memory issues, who is going to do my banking that i can trust? who's going to make sure i've got food in the house, all those sorts of things? (65-70, gay man) 4.4 theme 4: determinants of security for lgbtiq+ elders this theme captures the narrative about the older lgbtiq+ community’s determinants of a feeling of security. understanding these determinants of security creates knowledge about what lgbtiq+ elders need from robotic and ai systems regarding cybersecurity. appreciating difference refers to how several participants in this study voiced the notion of inclusivity. people to be able to appreciate that you are different but you’re still a human being. (67, gay gender-fluid person) i think we have to be inclusive in our caring for the elderly. and also, from whether it’s gay or whether it’s from ethnic backgrounds, you have to be inclusive as well. (68, gay man) we value inclusion … people sort of go back into their own, you know, the gay holes, the lesbian holes, the trans holes … the community is ghettoized and difficult to get to. and there's a reason for that, and that is because the community, as in the various sections of the community have to put up walls when we're not coming together as a group like that, to protect ourselves. (65, lesbian nonbinary) lgbtiq+ connectivity and community were sought out by some lgbtiq+ elders in this study, but not all. some participants wanted to be exclusively surrounded by the lgbtiq+ community for a sense of security and connection. whereas others for community and connectivity reasons, not for security, valued diverse friendships and connections outside of the lgbtiq+ community, and the remaining few participants did not want to be connected to the lgbtiq+ community. those who did seek out lgbtiq+ connectivity and community for a sense of security are highlighted here. i have run across quite a few gays in [place] – as far as the gay men’s social group are concerned there are quite a lot of older guys. one of the oldest one in our group is 82 but he still lives at home on his own and he’s very active … we do rely on each other. and one of the reasons i’m australasian journal of information systems poulsen, fosch-villaronga & burmeister 2020, vol 24, selected papers from the 2019 aice conference cybersecurity codes and lgbtiq+ elderly robots 9 part of that group is that if anybody gets really sick that we can all support each other and help if we need to. (68, gay man) i used to do a lot of work in the gay community. and i still have a lot of friends my age, who are very community-minded and happy to usually answer questions. (65-70, gay man) the notion of family is very, very strong within the community, the value of sticking together and being together because we need each other for protection … i'd say one of the most powerful values that the community has is a sense of community, that we are a community of people. and you know, i mean, i don't hesitate when i'm around my community, i relax, my guard goes down. and i think we saw it during the same sex marriage thing. another powerful value that we have is that our lives are about love and caring … sure we can get a bit of antipathy happening across the gay/lesbian divide. and also of course, obviously trans, a lot of trans people get a very bad time within the community. so, yeah, there are, because trans are the most reviled and hated by the straight world and also misunderstood within the community. (65, lesbian nonbinary) tolerance does not instil a sense of security, whereas acceptance does. acceptance is a very key value, not tolerance, acceptance. because of course, we all deal with our own families and sometimes it's a big deal for people to come out to the straight world. (65, lesbian nonbinary) the last category identified in the determinants of security for lgbtiq+ elders theme, was lgbtiq-friendly service provision. my home may become quite a dangerous place for me if services that i access are antagonistic. so, it’s a big, big issue, really big … being already a marginalized group of people, as we get older, we need more advocacy. (65, lesbian nonbinary) i've been invited to speak to a couple of age care or age-related conferences on lgbti aging and the issues of practitioners you have to be aware of because the queer aging population has had a very different life course from those who are young and serving them now in healthcare centres or hospitals or doctor's surgeries or whatever. (72, gay man) 5 discussion the findings have implications for cybersecurity in codes of conduct. healthcare professionals and designers of health is need to be aware of the historical queer experience and bring that awareness into care delivery (shields & burmeister, 2018). their codes of conduct need to prioritize privacy and safe physical locations in which unwanted surveillance does not occur, nor the theft of personal information. lgbtiq+ elders arguably feel even more vulnerable than their heterosexual peers about hackers uncovering and revealing a person’s gender identity or sexual orientation. this feeling and potential cybersecurity threat might be increased by the introduction of healthcare robots in one’s home. values in motion design (vmd) seeks to create ai where value sensing robots make explicit value-driven decisions to govern actions; these decisions are shaped to the values of the user and user community in situ when it is safe to do so as governed by a framework guaranteeing duty of care, professional ethics, and law which is embedded into the design of the robot (poulsen & burmeister, 2019). care robot designers creating value sensing robots ought to develop a basic care robot framework based on values found in applied ethics, such as australasian journal of information systems poulsen, fosch-villaronga & burmeister 2020, vol 24, selected papers from the 2019 aice conference cybersecurity codes and lgbtiq+ elderly robots 10 professional ethics and law, thereafter they should investigate individual and communitybased values that the care robot can then shape to the individual during run-time to provide person-centred care. the guiding principle of vmd is the ability to distinguish between intrinsic (pre-programmed) and instrumental care (in-situ) values. this distinction is grounded in applied ethics (e.g., intrinsic values emerging from professional ethics and codes, healthcare law, robot design standards, and duty of care) and descriptive ethics (e.g., instrumental values emerging from determinative in practice, person-centred, culturally competent care), respectively. furthermore, care robot designers should make intrinsic and preliminary instrumental value decisions, and value sensing robots are to make effective value decisions in relationship with the user by adapting and building off of those initial instrumental value decisions within the limits set out by those intrinsic ones. consider the following scenario in table 2: scenario for cybersecurity implications for lgbtiq+ users an lgbtiq+ elder uses a telepresence care robot to video call healthcare professionals and friends from their home. the hacker performs a man-in-the-middle attack to intercept video data and learns of their closeted sexual orientation. the hacker then attempts to make a video call to the user with the intention of blackmailing the lgbtiq+ elder with that sensitive information. table 2. scenario for cybersecurity implications for lgbtiq+ users the implications of a cyberattack can be disastrous. healthcare professionals' code of conduct ensured confidentiality. however, inserting a robot challenges how cyber-physical systems ensure confidentiality. these concerns ought to be accounted for in codes of conduct, both in robot developers' codes of conduct and in caregivers as well. in principle, value sensing robots could attempt to make use of instrumental values to achieve intrinsic ones better. a value sensing robot could shape the way it achieves cybersecurity by taking into account the values of the individual user and the user community, in situ (poulsen, burmeister, & kreps, 2018; poulsen, skaines, mclaren, & burmeister, 2020). considering the scenario in table 2, a value sensing robot might be able to help because, contrary to other robots that do not account for values in situ, the value sensing robot has ascertained that the user, like many lgbtiq+ elders, is concerned with trust regarding internet-connected robot systems through its previous interactions with the lgbtiq+ elder user. accordingly, the robot accounts for the user's trust and allows only trusted contacts to initiate a call with the user. the robot would check incoming calls using facial recognition and would be able to adapt its behaviour if a non-trusted contact is calling. the hacker's incoming call would be prevented because third parties cannot initiate contact with the lgbtiq+ elder. by adapting its behaviour and accounting for the instrumental value of trust in this way, the value sensing robot better accomplishes cybersecurity in the scenario and meets the requirement of adapting to the changing needs of users. adherence to a particular design methodology, such as value sensitive design (friedman, kahn, & borning, 2006; friedman & hendry, 2019; poulsen & burmeister, 2019) or user-centred design, could help address cybersecurity and user safety considerations (denning et al., 2014; tanev, tzolov, & apiafi, 2015; henschke & ford, 2017). although there is no concrete binding law that establishes a safeguard baseline for healthcare robots to be respected by those who design these technologies, unfortunately (fosch-villaronga, 2017; poulsen, burmeister, & tien, 2018), professional codes are an excellent mechanism to actuate juridical principles. indeed, australasian journal of information systems poulsen, fosch-villaronga & burmeister 2020, vol 24, selected papers from the 2019 aice conference cybersecurity codes and lgbtiq+ elderly robots 11 although existing laws can better inform healthcare technology design decisions when following these methods, inserting these methodologies in professional codes of conduct could be more efficient in making the work of developers account for user’s values. an example is the united kingdom department of health & social care (2019) released a code for datadriven health and care technology. the idea behind it was to ‘enable the development and adoption of safe, ethical, and effective data-driven health and care technologies.’ its principle nine promoted the integration of security and data protection into the design of the technology and released a toolkit to ease its implementation. the code of conduct of the uk seems a mere compliance guideline, nonetheless with art. 25 of the general data protection regulation, which focuses on privacy by design and by default (gdpr, 2018). however, these are not the only values involved in care practices, at least not for lgbtiq+ elders. the literature suggests that lgbtiq+ elders prioritize values of acceptance, privacy, and personhood (tenenbaum, 2011); inclusive language and normalization of disclosing gender identity or sexual orientation (huygen, 2006); autonomy and empowerment (waite, 2015); and non-judgemental care (latham and barrett, 2015). lgbtiq+ elders also define values uniquely (waling & roffee, 2017). not accounting for all these values would disregard the lgbtiq+ community values and perpetuate discrimination against the queer community (poulsen, fosch-villaronga, & søraa, 2020; gomes, antonialli, & dias-oliva, 2019). the gdpr is also a corpus that mostly misses the cyber-physical nature of robots and embodied ai (fosch-villaronga and millard, 2019). ai and robotic technologies are not mere data-driven technologies and do not only challenge data protection. in this respect, a code of conduct for professionals working on ai and robotics should take into consideration how the embodiment of such technologies plays a role in the overall interplay between user interaction and the protection of fundamental rights. moreover, it should take into account the whole ecosystem surrounding these technologies, which includes the manufacturer of the physical robot, the operating system, firmware, software, mobile/remote control applications; the vendor of internet, cloud services, and networks; and the professionals working with it, including the hospital or the direct caregivers and the care-receivers. 6 conclusion more research is needed in areas such as the governance of cybersecurity for ai-driven, and robotic healthcare is. these systems have a dual cyber-physical nature, the capacity to learn from experience and act autonomously. these capabilities demand the careful attention of those working on the design and development of such systems with vulnerable populations. in our work, we identified four categoric concerns implicated in the cybersecurity of ai-driven and robotic healthcare is for the older lgbtiq+ community: 1) privacy, 2) safety, 3) internet connectivity, and 4) determinants of security for lgbtiq+ elders. cybersecurity in the design, development, and use of ai-driven robot healthcare technologies often goes ungoverned, although it may affect users’ values. codes of conduct need to reflect such end-user concerns so that robot-human interaction design meets their requirements. currently, such codes reflect the individual responsibilities of is professionals (burmeister, 2017; burmeister, thomas, & poulsen, 2018), but they need to be extended to include the evolution of their products, beyond the initial implementation stages, such as for dynamically changing online platforms, and for value sensing robots which adapt care to the changing australasian journal of information systems poulsen, fosch-villaronga & burmeister 2020, vol 24, selected papers from the 2019 aice conference cybersecurity codes and lgbtiq+ elderly robots 12 needs of their users (poulsen & burmeister, 2019). that adaptation ability also needs to be the subject of codes of conduct and, ultimately, the law. it is imperative that robot developers understand the magnitude and scale of cybersecurity implications for users in particular and society at large and that caregivers understand that including robots for care delivery is not straightforward (fosch-villaronga, 2019). our study indicates the importance of including cybersecurity considerations in codes of conduct as it is the human and not the machine which is responsible for ensuring the system’s security. acknowledgment part of this project was funded by the leading fellows marie curie cofund fellowship, a project that has received funding from the european union’s horizon 2020 research and innovation programme under the marie skłodowska-curie grant agreement no. 707404. references al-saggaf, y., burmeister, o. k., & schwartz, m. (2017). qualifications and ethics education: the views of ict professionals. australasian journal of information systems, 21 bennett, e., berry, k., emeto, t. i., burmeister, o. k., young, j., & shields, l. (2017). attitudes to lesbian, gay, bisexual and transgender parents seeking health care for their children in two early parenting services in australia. journal of clinical nursing, 26(7-8), 1021– 1030. doi:10.1111/jocn.13595 birks, m., bodak, m., barlas, j., harwood, j., & pether, m. (2016). robotic seals as therapeutic tools in an aged care facility: a qualitative study. journal of ageing research, 2016. doi:10.1155/2016/8569602 bryson, j. j., diamantis, m. e., & grant, t. d. (2017). of, for, and by the people: the legal lacuna of synthetic persons. artificial intelligence and law, 25(3), 273-291 burmeister, o. k., thomas, g., & poulsen, a. (2018). professional ethics involving cyber security and autonomous robots. in s. g. tzafestas (ed.), information, communication, and automation technology ethics in the knowledge society age. nova science publishers burmeister, o. k. (2017). professional ethics in the information age. journal of information, communication and ethics in society, 15(4), 348-356. doi:10.1108/jices-11-2016-0045 burmeister, o. k. (2013). achieving the goal of a global computing code of ethics through an international-localisation hybrid. ethical space: the international journal of communication ethics, 10(4), 25-32 capurro, r., & britz, j. b. (2010). in search of a code of global information ethics: the road travelled and new horizons. ethical space: the international journal of communication ethics, 7(2/3), 28-36 carter-templeton, h., frazier, r. m., wu, l., & h. wyatt, t. (2018). robotics in nursing: a bibliometric analysis. journal of nursing scholarship, 50(6), 582-589. doi:10.1111/jnu.12399 clark, g. w., doran, m. v., & andel, t. r. (2017). cybersecurity issues in robotics. paper presented at the 2017 ieee conference on cognitive and computational aspects of situation management (cogsima) australasian journal of information systems poulsen, fosch-villaronga & burmeister 2020, vol 24, selected papers from the 2019 aice conference cybersecurity codes and lgbtiq+ elderly robots 13 coronado, a. j., & wong, t. l. (2014). healthcare cybersecurity risk management: keys to an effective plan. biomedical instrumentation & technology, 48(s1), 26-30 cresswell, k., cunningham-burley, s., & sheikh, a. (2018). health care robotics: qualitative exploration of key challenges and future directions. journal of medical internet research, 20(7), e10410 datteri, e. (2013). predicting the long-term effects of human-robot interaction: a reflection on responsibility in medical robotics. science and engineering ethics, 19(1), 139-160 denning, t., kramer, d. b., friedman, b., reynolds, m. r., gill, b., & kohno, t. (2014). cps: beyond usability: applying value sensitive design based methods to investigate domain characteristics for security for implantable cardiac devices. proceedings of the 30th annual computer security applications conference. new orleans, louisiana, usa. doi:10.1145/2664243.2664289 european parliament (2017) resolution on civil law rules on robotics. retrieved from http://www.europarl.europa.eu/doceo/document/ta-8-2017-0051_en.html fjeld, j., hilligoss, h., achten, n., levy daniel, m., feldman, j., kagay, s. (2019) principled artificial intelligence. retrieved from https://cyber.harvard.edu/publication/2020/principled-ai floridi, l. (2019). establishing the rules for building trustworthy ai. nature machine intelligence, 1(6), 261-262 floridi, l., cowls, j., beltrametti, m., chatila, r., chazerand, p., dignum, v., ... & schafer, b. (2018). ai4people—an ethical framework for a good ai society: opportunities, risks, principles, and recommendations. minds and machines, 28(4), 689-707 food and drug administration, fda (2019) cybersecurity. retrieved from https://www.fda.gov/medical-devices/digital-health/cybersecurity fosch villaronga, e. (2017). towards a legal and ethical framework for personal care robots. analysis of person carrier, physical assistant and mobile servant robots. (doctoral dissertation, university of bologna, bologna, italy) fosch-villaronga, e. (2019). robots, healthcare, and the law: regulating automation in personal care. routledge fosch-villaronga, e., felzmann, h., ramos-montero, m., & mahler, t. (2018). cloud services for robotic nurses? assessing legal and ethical issues in the use of cloud services for healthcare robots. proceedings of 2018 ieee/rsj international conference on intelligent robots and systems (iros) (290-296) fosch-villaronga, e. and millard, c. (2019). cloud robotics law and regulation: challenges in the governance of complex and dynamic cyber-physical ecosystems. robotics and autonomous systems 119, 77-91. retrieved from https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3305353 friedman, b., & hendry, d. g. (2019). value sensitive design: shaping technology with moral imagination: mit press http://www.europarl.europa.eu/doceo/document/ta-8-2017-0051_en.html http://www.europarl.europa.eu/doceo/document/ta-8-2017-0051_en.html australasian journal of information systems poulsen, fosch-villaronga & burmeister 2020, vol 24, selected papers from the 2019 aice conference cybersecurity codes and lgbtiq+ elderly robots 14 friedman, b., kahn, p. h. j., & borning, a. (2006). value sensitive design and information systems. in p. zhang & d. galletta (eds.), human-computer interaction and management information systems: foundations (pp. 348-372). new york: m. e. sharpe general data protection regulation, gdpr (2018). article 25 eu general data protection regulation. retrieved from https://www.privacy-regulation.eu/en/article-25-dataprotection-by-design-and-by-default-gdpr.htm gomes, a., antonialli, d. & dias-oliva, t. (2019) drag queens and artificial intelligence. should computers decide what is toxic on the internet? internet lab blog. retrieved from http://www.internetlab.org.br/en/freedom-of-expression/drag-queens-and-artificialintelligence-should-computers-decide-what-is-toxic-on-the-internet/ henschke, a., & ford, s. b. (2017). cybersecurity, trustworthiness and resilient systems: guiding values for policy. journal of cyber policy, 2(1), 82-95. doi:10.1080/23738871.2016.1243721 high-level expert group on artificial intelligence, hleg (2019). ethics guidelines for trustworthy ai. retrieved from https://ec.europa.eu/futurium/en/ai-allianceconsultation huygen, c. (2006). understanding the needs of lesbian, gay, bisexual, and transgender people living with mental illness. medgenmed: medscape general medicine, 8(2), 29-29. retrieved from https://www.ncbi.nlm.nih.gov/pmc/articles/pmc1785208/ jobin, a., ienca, m., & vayena, e. (2019). the global landscape of ai ethics guidelines. nature machine intelligence, 1(9), 389-399 johnson, d. g. (2015). technology with no human responsibility?. journal of business ethics, 127(4), 707-715 khosla, r., chu, m.-t., kachouie, r., yamada, k., yoshihiro, f., & yamaguchi, t. (2012). interactive multimodal social robot for improving quality of care of elderly in australian nursing homes. proceedings of the 20th acm international conference on multimedia. doi:10.1145/2393347.2396411 latham, j., & barrett, c. (2015). appropriate bodies and other damn lies: intersex ageing and aged care. australasian journal on ageing, 34(s2), 19-20. doi:10.1111/ajag.12275 legassick, s., & harding, v. (2017). why we launched deepmind ethics & society. retrieved from https://deepmind.com/blog/announcements/why-we-launched-deepmind-ethicssociety martin, g., martin, p., hankin, c., darzi, a., & kinross, j. (2017). cybersecurity and healthcare: how safe are we?. bmj, 358, j3179, 1-4 melkas, h., hennala, l., pekkarinen, s., & kyrki, v. (2020). impacts of robot implementation on care personnel and clients in elderly-care institutions. international journal of medical informatics, 134, 1-6. doi:10.1016/j.ijmedinf.2019.104041 miah, s., shen, j., lamp, j. w., kerr, d., & gammack, j. (2019). emerging insights of health informatics research: a literature analysis for outlining new themes. australasian journal of information systems, 23. doi:10.3127/ajis.v23i0.2137 australasian journal of information systems poulsen, fosch-villaronga & burmeister 2020, vol 24, selected papers from the 2019 aice conference cybersecurity codes and lgbtiq+ elderly robots 15 moyle, w., jones, c., & sung, b. (2020). telepresence robots: encouraging interactive communication between family carers and people with dementia. australasian journal on ageing, 39(1), 127-133. doi:10.1111/ajag.12713 pasquale, f. (2017). toward a fourth law of robotics: preserving attribution, responsibility, and explainability in an algorithmic society. ohio state law journal, 78. retrieved from https://ssrn.com/abstract=3002546 pino, m., boulay, m., jouen, f., & rigaud, a. s. (2015). “are we ready for robots that care for us?” attitudes and opinions of older adults toward socially assistive robots. frontiers in aging neuroscience, 7, 141, doi:10.3389/fnagi.2015.00141 poulsen, a., skaines, i., mclaren, s., & burmeister, o. (2020). value sensing robots: the older lgbtiq+ community. proceedings of the 18th international conference on the ethical and social impacts of ict (ethicomp) poulsen, a., & burmeister, o. k. (2019). overcoming carer shortages with care robots: dynamic value trade-offs in run-time. australasian journal of information systems, 23. doi:10.3127/ajis.v23i0.1688 poulsen, a., burmeister, o. k., & kreps, d. (2018). the ethics of inherent trust in care robots for the elderly. in d. kreps, c. ess, l. leenen, & k. kimppa (eds.), this changes everything – ict and climate change: what can we do? (pp. 314-328). doi:10.1007/978-3319-99605-9_24 poulsen, a., burmeister, o. k., & tien, d. (2018). a new design approach and framework for elderly care robots. proceedings of the australasian conference on information systems. retrieved from http://www.acis2018.org/wp-content/uploads/2018/11/acis2018_paper_162.pdf poulsen, a., fosch-villaronga, e., & søraa, r. a. (2020). queering machines. nature machine intelligence, 2, 152. doi:10.1038/s42256-020-0157-6 quarta, d., pogliani, m., polino, m., maggi, f., zanchettin, a. m., & zanero, s. (2017). an experimental security analysis of an industrial robot controller. proceedings of 2017 ieee symposium on security and privacy (sp) (pp. 268-286) razzaki, s., baker, a., perov, y., middleton, k., baxter, j., mullarkey, d., ... & dorosario, a. (2018). a comparative study of artificial intelligence and human doctors for the purpose of triage and diagnosis. arxiv preprint arxiv:1806.10698 sharkey, a. (2014). robots and human dignity: a consideration of the effects of robot care on the dignity of older people. ethics and information technology, 16(1), 63-75 shields, l., & burmeister, o. (2018). education needed to enhance inclusive, nondiscriminatory nursing practice towards lesbian, gay and bisexual parents. evidence based nursing, 21(2), 47. doi:10.1136/eb-2017-102853 tanev, g., tzolov, p., & apiafi, r. (2015). a value blueprint approach to cybersecurity in networked medical devices. technology innovation management review, 5(6). retrieved from http://timreview.ca/article/903   australasian journal of information systems poulsen, fosch-villaronga & burmeister 2020, vol 24, selected papers from the 2019 aice conference cybersecurity codes and lgbtiq+ elderly robots 16 tenenbaum, e. m. (2011). sexual expression and intimacy between nursing home residents with dementia: balancing the current interests and prior values of heterosexual and lgbt residents. temple political & civil rights law review, 21, 459 thomas, g., burmeister, o., & low, g. (2019). the importance of ethical conduct by penetration testers in the age of breach disclosure laws. australasian journal of information systems, 23. doi:10.3127/ajis.v23i0.1867 united kingdom department of health & social care. (2019). code of conduct for data-driven health and care technology. retrieved from https://www.gov.uk/government/publications/code-of-conduct-for-data-driven-healthand-care-technology/initial-code-of-conduct-for-data-driven-health-and-caretechnology waite, h. (2015). old lesbians: gendered histories and persistent challenges. australasian journal on ageing, 34(s2), 8-13. doi:10.1111/ajag.12272 waling, a., & roffee, j. a. (2017). knowing, performing and holding queerness: lgbtiq+ student experiences in australian tertiary education. sex education, 17(3), 302-318. doi:10.1080/14681811.2017.1294535 zardiashvili, l., & fosch-villaronga, e. (2020). “oh, dignity too?” said the robot: human dignity as the basis for the governance of robotics. minds and machines, 1-23 copyright: © 2020 poulsen, fosch-villaronga & burmeister. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v24i0.2789 http://creativecommons.org/licenses/by-nc/3.0/au/ http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems wee, scheepers & tian 2022, vol 26, research article understanding the processes 1 understanding the processes of how small and medium enterprises derive value from business intelligence and analytics mak wee swinburne university of technology, melbourne australia makwee7@gmail.com helana scheepers swinburne university of technology, melbourne australia xuemei tian swinburne university of technology, melbourne australia abstract this paper provides an in-depth study of how small and medium enterprises (smes) use business intelligence and analytics (bi&a) to derive business value and why so many smes fail to do so. a qualitative research approach based on semi-structured interviews with five smes in australia was applied with the goal is to understand the process in which smes adopt bi&a to derive business value. this involved exploring how owners and managers lead their employees in using data and analytical processes to derive insights to make business decisions. the findings suggest that smes which adopt bi&a use a short and simple six-step iterative bi&a process to derive insights for business process application. in addition to the short process, a longer three phase process has been identified which progresses smes from solving operational issues to strategic challenges. the resulting short and long bi&a implementation process framework provides a progressive pathway for sme owners and managers to initiate and lead bi&a transformation in their smes to derive greater business value. the process model considers dimensions of data, analysis, business process change, social influence, level of information use and financial returns. keywords: business intelligence and analytics, small and medium enterprises, process model. 1 introduction in today’s economic climate, business intelligence and analytics (bi&a) are crucial in strategising, planning, executing and monitoring business operations to improve efficiency and performance. bi&a is an overarching term for decision support systems used to collect, analyse and disseminate organisational data to improve decision-making (fink et al., 2017). the implementation of bi&a has been simplified over time due to advances in digital platforms and bi&a cloud computing (mikalef et al., 2020), providing more accessibility to small and medium enterprises (smes). however, sme research shows a low adoption rate of bi&a (agostino et al., 2013; llave, 2019). mailto:makwee7@gmail.com australasian journal of information systems wee, scheepers & tian 2022, vol 26, research article understanding the processes 2 research on bi&a focuses primarily on large organisations, despite the significant contribution that smes make to the world economy and the potential benefits that smes can gain from bi&a. llave (2019) conducted a systematic review of research on bi&a in smes published between 2000 and 2018, which included topics on bi&a benefits, cloud bi&a, mobile bi&a, bi&a solutions and implementation. she found that while several frameworks and models were proposed to guide smes on successful bi&a implementation, there is no clear indication of success. she further highlights the need to consider the unique characteristics of smes and not rely on models developed for large organisations. mikalef et al. (2020) also indicate that further research is necessary to understand the processes large organisations undertake to adopt bi&a. a small number of frameworks have been devised to explain how large organisations gain value from bi&a (fink et al., 2017; seddon et al., 2017). for example, business analytics success model (basm) is a process model that describes the 'analyse-insight-decision-action' process of large organisations to derive value from bi&a (seddon et al., 2017, pg. 237). process models are event-driven explanations of events and patterns that describe change over time (payne et al., 2017; van de ven, 2007). however, none of these frameworks have been developed for smes and they do not consider their unique characteristics. basm is used as a starting point for this research to understand the process that smes undertake to derive value from bi&a. the research question investigated in this paper is: what processes do smes use to derive value from bi&a? three main findings are derived from qualitative case studies in five melbourne, australiabased smes. the first is that while all the smes had the basic building blocks for bi&a processes, many smes are still traditionally bound by heuristics and networking and are unwilling to venture into the use of bi&a. the second is that bi&a applications are not simply a matter of transforming resources into capabilities. smes need to consider the goal of the information use (organisational, managerial or strategic), the data required to achieve this goal, how to apply insights and how to inculcate cultural change in the organisation. the third major finding is that there are distinct measurable changes as the sme progresses in adopting bi&a. these findings extend the basm model by providing greater details on how bi&a adoption takes place in smes. the paper is structured as follows. the first section reviews the current surrounding contemporary understanding of smes' adoption of information technology, then introduces and adapts the basm for smes. the second section describes the research method used in this study, while the third section focuses on the analysis of the data and concludes with the findings and recommendations. 2 literature review 2.1 smes' adoption of information systems (is) smes account for 99% of all organisations in oecd countries and employ on average 70% of workers in oecd countries (oecd, 2017). this research adopts the australian bureau of statistics' definition that smes have less than 200 employees (statistics, 2021). however, smes are not homogenous in size and are instead represented by micro, small and medium businesses, as shown in table 1. australasian journal of information systems wee, scheepers & tian 2022, vol 26, research article understanding the processes 3 due to the importance of smes to the economy, many researchers have studied how smes adopt various types of information technology such as the internet, customer relationship management (crm) and enterprise resource planning (erp), just to name a few. in a review of high-ranking journal studies on is and smes, parker et al. (2015) noted that smes are heterogeneous, citing that they vary in size, industry and independence versus acting as a subsidiary. micro small medium large headcount 14 5 19 20200 200 + total operating business @30 june 2020 609,276 214,454 47,649 4,160 875,539 % of total 69.59% 24.49% 5.44% 0.48% 100.00% source (statistics, 2021) table 1. count of australian businesses by size as of 30 jun 2020 due to the importance of smes to the economy, many researchers have studied how smes adopt various types of information technology such as the internet, customer relationship management (crm) and enterprise resource planning (erp), just to name a few. in a review of high-ranking journal studies on is and smes, parker et al. (2015) noted that smes are heterogeneous, citing that they vary in size, industry and independence versus acting as a subsidiary. despite these differences, researchers have found common characteristics amongst smes that will affect the introduction of new is. zach et al. (2014) summarised the characteristics of smes into three high-level groups: organisational, environmental and knowledge of is. the organisational characteristics are further divided into the availability of resources (both human and financial), management, organisation structure, culture, processes and procedures (zach et al., 2014). the environmental factors are further refined into uncertainty, markets and customers (zach et al., 2014). finally, the knowledge of is subdivided into is knowledge, technical expertise and complexity of the is (zach et al., 2014). focusing on a specific information system such as erp, zach et al. (2014) found that sme owner-managers significantly influenced almost all aspects across the erp life cycle, such as the system selection and customisation as well as the implementation team's work. this finding further emphasises the central role of the owner-manager of an sme in the adoption of is. the lack of resources also plays an important role in adopting is. seethamraju (2014) found that the total cost of ownership and investment costs in adopting new technology were key factors in the adoption decision. jaouen and nakara (2015) proposed that smes use bricolage as a form of strategic financial creativity, "making do by applying combinations of resources already at hand" (baker & nelson, 2005, pg. 333). they identified two forms of bricolage: ' necessity bricolage' to keep their business small and avoid recruitment and the second, 'strategic bricolage' aiming at success, performance and sometimes growth. power and gruner (2017) highlight the lack of research into the role of decision-making style in smes and broadly divide the literature on is decision-making strategies into two streams. the first stream highlights a positive relationship between performance and a deliberate decision-making style related to is. the second stream of research identifies a positive relationship between an emergent decision-making style to is and performance. power and gruner (2017) indicate no consensus in the literature on the best decision-making strategy for smes. in the context of inter-organisational systems, decision-making is often ongoing and australasian journal of information systems wee, scheepers & tian 2022, vol 26, research article understanding the processes 4 contextual and involves more intuition than rational, careful analysis and omniscient projection of future states (power & gruner, 2017). 2.2 theoretical model relating to bi&a implementation early research has indicated that the use of bi&a technology has improved organisation performance (mikalef et al., 2019). but recent research suggests that the adoption process is a complex endeavour and it is necessary to have a more in-depth understanding of how organisations need to change to adopt bi&a effectively (vidgen et al., 2017). mikalef et al. (2020) argue that business analytics (ba) research assumes that ba resources are easily orchestrated and mobilised effectively by organisations and that the desired outcomes follow naturally. as a result, they highlight that more research is needed to understand the structures, processes and activities necessary to incorporate bi&a resources into organisational capabilities. as such, we firstly investigate the current understanding of the adoption of bi&a by smes. llave (2019) conducted a comprehensive review of the literature on bi&a implementation by smes and found 25 publications related to this area. however, only four publications focus on frameworks, architecture and models, whilst other papers identify critical success factors, antecedents and prototypes. table 2 below summarises the four papers that focus on frameworks and models for bi&a adoption by smes (llave, 2019). reference contribution empirical testing 1 (raymond, 2003) developed a conceptual framework focussing on the competitiveness of smes that could orient bi&a activities & projects. conceptual study 2 (sadok & lesca, 2009) develop a bi tacit knowledge model based on interviews with 20 french smes leaders. model-based on interviews with 20 smes 3 (mahmoud et al., 2012) developed a semantic-enabled enterprise service-oriented architecture (sesoa), an enterprise solution linking businesses to external systems based on web services and service-oriented architecture. conceptual study 4 (guarda et al., 2013) developed a framework that consists of 4 phases, planning, technology, intelligence and dissemination. conceptual study table 2. proposed framework and models for bi&a implementation in smes (llave, 2019) each of the four frameworks was evaluated for comprehensiveness and appropriateness for guiding smes in the adoption of bi&a: • raymond (2003) proposed a conceptual and operational bi&a framework for detecting trends and understanding strategic issues that emerge from the global knowledge economy. however, this framework focuses mainly on smes' competitive advantage from scanning the environment. • sadok and lesca (2009) acknowledged organisational and social aspects of bi&a implementation by developing a bi&a model based on tacit knowledge for environmental intelligence gathering. however, the model focuses on strategic decision-making without addressing operational decision-making. australasian journal of information systems wee, scheepers & tian 2022, vol 26, research article understanding the processes 5 • mahmoud et al. (2012) and guarda et al. (2013) have proposed either an architecture, framework, or bi&a tools for sme usage. these models focus on specific technical aspects of bi&a implementation but do not consider any social or organisational changes. all four models focus on some aspects of bi&a adoption, detection and scanning (raymond, 2003); tacit and social factors (sadok & lesca, 2009); and technical factors (guarda et al., 2013; mahmoud et al., 2012). yet, with the exception of sadok and lesca (2009), few of these models have been empirically tested. due to the lack of suitable frameworks for sme bi&a implementation processes, the focus of the literature review moves to generic bi&a process implementation models for large organisations. for example, seddon et al. (2017) developed the business analytics success model (basm), fink et al. (2017) combine the resource-based view with organisational learning, and trieu (2017) adapts the soh and markus (1995) model of investments to assets to impacts. the basm model is chosen for this paper as it takes an 'analyse-insight-decisionaction' process, which is simple and can be related to smes. the basm process model (see figure 1 below) integrates literature on problem-solving (newell & simon, 1972; pretz et al., 2003), insight (bowden et al., 2005; sternberg & davidson, 1995), the resource-based view of the firm (barney, 1991), the dynamic capabilities view of the firm (teece et al., 1997), the viable system model (beer, 1984) and the is literature on business analytics and business intelligence, in particular the work of davenport et al. (2010). compared to the four models developed for smes, the basm is more holistic, showing components of people, technology and resources together with process flows in how value/benefits are created. these four components should be orchestrated to deliver the potential benefits bi&a can provide to organisations (mikalef et al., 2020). figure 1. basm process model (seddon et al., 2017) as shown in figure 1, the objects in the dotted boxes reflect the three main parts of the model (constructs from figure 1 are used in the discussion below and placed in quotation marks): • the dotted box in the top left describes the 'use of analytical resources' and the potential 'insight(s)' gained from its use. australasian journal of information systems wee, scheepers & tian 2022, vol 26, research article understanding the processes 6 • the bottom left dotted box (called 'analytical resources') relates to the analytic resources, consisting of people with an analytical mindset and the technology, data, processes and governance of bi&a. • the right dotted box relates to the use of bi&a for operational purposes (called 'intendedly value-creating actions that use the organisation's existing resources') and the creation of new organisational capabilities, typically for strategic purposes (called 'intendedly value-creating actions that change the organisation's resources'). the model further highlights three paths to value creation: • path 1 commences with insights leading to decisions that will create value with existing resources, focusing on the operational impact of bi&a. for example, reports and dashboards produce insights that will aid the organisation to create value. • path 2 can occur, in some cases, instead of path 1. the insights will change organisational resources to create new capabilities rather than create value based on operational data described in path 1. this path highlights the strategic impact bi&a can have on an organisation. • path 3 is the organisational learning that takes place while using bi&a, leading to changes in the analytical resources of the organisation. for example, the insights may trigger better data quality or more powerful analytical tools or skilled people. the basm model is adapted for the unique characteristics of smes (see figure 2 below). the reasons for the changes are explained thereafter. figure 2. adapted basm process model for smes (adapted from seddon et al., 2017) (numbers in the diagram are used in the explanation below and in figure 3) in an sme setting where decision-making is centralised, the 'use of analytical resources' (figure 2, no. 1) is executed mainly by sme owners/managers. this view is supported by research on adopting information technology in smes, where the owner/manager plays a pivotal role (renken & heeks, 2019). furthermore, the continuation of adoption and use of australasian journal of information systems wee, scheepers & tian 2022, vol 26, research article understanding the processes 7 technology relies on the owner/manager's perceived value of the technology (power and gruner, 2017). this central role of the owner/manager is reflected in the changing from 'senior manager' (figure 1) to 'owner/manager' in 'organisational benefits from bi&a use' (figure 2 no. 5). lastly, the 'analytical resources' (figure 2, no. 4, 6, 7) are expected to be significantly smaller than large organisations due to constraints in resources (zach et al., 2014) such as systems, tools and people. basm depicts a flow of organisational components relating to bi&a implementation and use. however, one key aspect that is not evident in the process model is how bi&a culture can be developed in an organisation. seddon et al. (2017) identify analytical leadership, enterprisewide analytics orientation and evidence-based decision-making as key factors for the success of bi&a in a large organisation. these factors are captured in a bi&a culture (figure 2, no. 4) that acts as a catalyst in creating value by providing actionable insights to decision-makers (skyrius et al., 2016) in which senior management (or owner/managers in the case of smes) plays a crucial role (davenport et al., 2010). however, in the context of smes, decision-making is not necessarily evidence-based (hauser et al., 2020) and changing to a bi&a culture will require significant organisational change. also, smes have limited financial resources and will not necessarily have the ability to upgrade analytical resources. based on the literature review and considering the unique characteristics of smes, the basm model can be simplified for bi&a value creation for smes as illustrated in figure 2. therefore, figure 2 will form the basis for investigating smes' process to adopt bi&a. the key changes made to the basm model are as follows: • the sme owner/manager is the primary decision-maker. therefore, the use of the analytical resources is focused on the owner/manager (see 'use of analytical resources' (figure 2, no. 1) and the 'decision(s)' process. the owner/manager should perceive the benefits of implementing bi&a (see figure 2, no. 5). • the 'analytical resources' also change to accommodate the characteristics of smes. most smes do not use sophisticated information technology or bi&a resources so, 'enabling technology' (figure 2, no. 6) is simplified. the 'analytical people' (figure 2, no. 7) reflect the smaller headcount in smes and highlight the importance of the owner/manager. the most significant change is the bi&a culture (figure 2, no. 4) to reflect the necessary changes to business processes and the sme owner/manager support. • the three paths (figure 2, no. 2, 3, 8) related to bi&a use will remain. 3 research methodology this study employs the case study method to determine the value smes derive from bi&a. case studies are an appropriate research method to study a phenomenon in context (benbasat et al., 1987). according to llave (2019, pg. 34), "case studies represent a key method of obtaining a more in-depth understanding of the phenomena of bi&a adoption and use by smes." this research takes a cross-sectional view rather than an industry view. however, it is envisaged that cross-industry sme leaders participating in in-depth interviews would be more forthcoming in using data to outmanoeuvre their competitors if they understand that they are the only smes being studied in the same industry. australasian journal of information systems wee, scheepers & tian 2022, vol 26, research article understanding the processes 8 a convenience sampling method (patton, 2002) was used to select participating smes. the following criteria were used to determine whether a sme will be included as a case organisation in the study: is the organisation classified as a small or medium sme and does the organisation have some form of bi&a implementation (to ensure a combination of organisations with mature and less mature bi&a implementation). organisations from a mix of industries were included. the data collection comprised semi-structured interviews and observations of the bi&a reports and systems within three medium and two small enterprises, as shown in table 3. eisenhardt (1989) indicated that 4 – 7 case studies are appropriate to develop theories. sme nature of business headcount perceived bi&a maturity sme type respondent no. of interviews 1 charity 22 high medium ceo 3 2 recruiter 50 low medium director 1 3 engineering 50 high medium erp manager 1 4 retail pharmacy 13 low small partner 1 5 wholesaler 15 medium small ceo 1 table 3. details of the five sme case studies a brief narrative of each sme is given by category below, where the nature of the business, size, types of is and the sme participants who engaged in this research are described. medium smes sme1 is a charity with 22 staff in melbourne. the leading executives were the ceo, marketing manager, merchandising manager, personal secretary and finance manager. they also operated a gift or merchandising department to sell gifts for fundraising. they had both accounting and customer relationship management (crm) systems. they generally relied on vendors for it equipment and they retained an it contractor/employee to operate their structured query language (sql) server and a host of management reports. the participant in our research was the ceo who joined the charity six years ago. sme2 is a recruitment company that recruited blue-collar workers for the warehousing industry. although the company was categorised as a medium sme with an average of between 50-70 employees, many employees were workers contracted to companies as hired labour. the management structure was minimal and they had a computerised accounting system. the workers were contracted daily and they manually tracked the labour hours to be invoiced using a paper-based approach. the respondent in our study was a recruitment director responsible for recruiting staff and potential candidates for their corporate customers. sme3 is a local australian engineering company that designed and manufactured transport and storage products for the mining and petroleum industries. the company had a headcount of 50 australian staff. they had an enterprise resource planning (erp) and a crm system. the respondent for our study was a manager in charge of the erp system. australasian journal of information systems wee, scheepers & tian 2022, vol 26, research article understanding the processes 9 small smes sme4 is a retail pharmacy serving a local community and operated independently of any franchise arrangement. they were categorised as small smes with approximately 13 employees. they operated several different systems: a point-of-sale (pos) system, an ordering system and a pharmaceutical dispensing system. the person interviewed was the managing proprietor of the pharmacy. sme5 is a coffee wholesaler encompassing three lines of business. they operated three cafes, a coffee bean wholesaling business and a coffee academy offering training courses for coffee lovers and baristas. they used an accounting system as well as a system for managing labour hours. in addition, they used a cloud-based pos system to record all sales and shelf products. this sme was categorised as small with 15 full-time employees. the interview participant was the ceo of the company. all five sme case studies are based in metropolitan melbourne and initial interviews were face-to-face. an interview guide was developed to source data in two parts. part 1 investigated the smes' current 'analytical resources' (see figure 2, no. 4, 6, 7). the questions in part 1 focused on the factual information of the company, including size, nature of business and types of computer systems employed. part 1 also explored the availability of data in their systems and the impact on business processes. this was followed by investigating the case organisation's decision-making (the 'use of analytical resources' (figure 2, no. 1) and 'decision(s)' constructs in figure 2). most interviewees were not familiar with the specifics of bi&a. hence, the thrust of the interviews focused on the management and usage of data to derive insights to gain business value. during the interviews, concrete examples of operational, tactical and strategic decisions (see the 'value creation' construct in figure 2 and figure 2, no. 5) were sought. part 2 of the interview related to their bi&a journey to determine if they had evolved their systems (figure 2, no. 6), processes (figure 2, no. 2, 3, 8) and culture (figure 2, no. 4) over time. these topics allowed participants to elaborate not only on the process of bi&a implementation but also on the intertwining factors associated with the process. it also allowed interviewees to relate their stories/experiences from their viewpoint without influencing them unnecessarily (walsham, 2006). the interviews ranged from 45-60 minutes, depending on the extent of bi&a implementation of the sme. most of the findings were audio-recorded. but there are instances where the interviewer relied on notetaking; for example, when a ceo logged into a system during the interview to show the type of reports that they used to track performance. in addition, exhibits such as yearly work plans and charts, which functional managers in the sme used to track key performance indicators, were collected. the interviews were conducted between august 2019 and february 2020. data were analysed using an interpretative research approach (klein & myers, 1999; walsham, 1995). figure 3 illustrates the analytical process by providing examples of the coding categories that emerged in each of the four steps. australasian journal of information systems wee, scheepers & tian 2022, vol 26, research article understanding the processes 10 figure 3. data analysis of codes (process adapted from niemi and pekkola (2019) categories from the literature (figure 2 no. 1-8) are linked to categories a-e, which in turn is connected to table 4 columns a-e) the interview themes were first searched as initial coding categories derived from the literature review based on the basm constructs in figure 2. these initial coding categories are numbered 1 – 8 in figures 2 and 3 (see figure 3, steps 1 and 2). the initial constructs in figure 2 (as defined by seddon et al. 2017) were simplified and contextualised to the second-order constructs a – e (see figure 3, step 3). through subsequent iterative re-analyses, constructs and interrelationships were coded into the second-order constructs a – e (see figure 3 a-e). paths 1, 2 and 3 were related to construct b ('bi&a built into business processes') and played a key role in identifying how smes orchestrated their resources to create value. the emergent interrelationships were then mapped into appropriate pairs and generalised as interrelationships between the associated constructs leading to the identification of three bi&a phases (step 4). one of the interesting findings was that very few of the interviewees knew what bi&a was even after describing the term. the sme organisations included in the study varied from smes that implemented simple bi&a tools to those that did not consider using any bi&a tools. 4 findings five main constructs were identified through the iterative coding (figure 3) of the interviews and observations from the five sme owners and managers. the five constructs map the interrelationships of bi&a use in smes. these constructs are the decision-making process (figure 3, a, table 4 column a), bi&a built into business processes (figure 3, b, table 4 column b), types of bi&a culture (figure 3, c, table 4 column c), level of information use (figure 3, d, table 4 column d) and financial investments (figure 3, e, table 4 column e). the value of 'bi&a built into business processes' (figure 3, b, table 4 column b) is a summarised integration of the use of bi&a (no function, some functions and organisationwide). however, the first order construct of path 1, path 2 and path 3 (figure 2, nr 2, 3, 8) australasian journal of information systems wee, scheepers & tian 2022, vol 26, research article understanding the processes 11 provided a means to describe how smes orchestrate the resources and thus create value for the organisation. each of the paths has a specific value that it creates. in path 1, the bi&a decision provides value through the existing operational resources of the sme. path 2 improves the organisational resources, which provide strategic value to the sme. finally, path 3 leads to organisational learning that enhances the analytical resources of the organisation. the path construct is used in the descriptions of all the constructs below. subsequent mapping of the first level attributes (five constructs) through cross-case analysis identified three distinct implementation stages. finally, the five smes are categorised based on the three identified stages (figure 3, bi&a phases, table 4 column f). the implementation phases are further elaborated on in the discussion section. each of the constructs is now described in more detail. sme enabling technologies [a] decisionmaking process [b] bi&a built into business process [c] types of bi&a culture [d] level of information use [e] financial investment [f] bi&a phase 1 crm*, accounting, sql server# rational organisation wide bi&a culture strategic $$$$ organisa -tion wide 2 accounting, resumes heuristic no individualism operational $ novice 3 erp**, customer support rational some functions influencing management $$ crossfunctional 4 pos***, accounting mixed no individualism operational $ novice 5 pos***, labour, accounting rational some functions influencing management $$ crossfunctional table 4. details of five smes from interviews and observations *crm = customer relationship management, **erp = enterprise resource management, ***pos = point of sales, #sql server = database, $ = low investment amount, $$ = medium investment, $$$$ = significant high investment (elaborated in section 4.6) 4.1 enabling technologies all of the smes have invested in is which comprises the "enabling technology" construct of basm (see figure 2). these is included accounting, pos, crm and erp systems. accounting systems were necessary for all smes, as all companies in australia are required by statutory laws to file their annual accounts and tax statements (ato, 2020). some smes have additional is related to the nature of their business. for example, the pharmacy (sme4) and wholesaler (sme5) have a pos system and sme4 have a pharmaceutical dispensing system. sme1, a charity, has a crm system but did not use a pos system as they sell products and services online. sme3 has an erp and a separate system for customer support. sme2 is a recruitment agency with an accounting system and access to job portals. all these systems are transactional processing systems (tps) that facilitate business processes and, in most cases, offer bi&a capabilities. australasian journal of information systems wee, scheepers & tian 2022, vol 26, research article understanding the processes 12 sme1 is the only organisation that extensively invested in dedicated bi&a technologies (see column e in table 4) above those provided through tps. for example, sme1 uses an sql database and microsoft excel® to develop many reports for management decision-making. although the other smes also have access to information in their respective tps with its related bi&a functionality and basic bi&a tools such as excel®, very few have the full suite of bi&a technologies to integrate the data between their tps and other systems for a bigger business picture. it was surprising that sme3 has a separate crm when it already has an erp system. in most cases, distinct standalone software can be customised to meet the functional requirement of the sme than a generic software package. but customised systems lose their ability to integrate the data to form a holistic view. hence, for an integrated view of two or more functions, additional database systems and bi&a tools such as sql, excel, or power bi are required to join data for broader and deeper insights. 4.2 decision-making process the first construct (table 4, column a) is about how decisions were made in the sme. although all smes have access to tps data and therefore have the necessary data, not all smes invest significantly in bi&a technologies. this decision to invest in further bi&a technologies can be directly related to the decision-making style of the owner/manager and whether they believe that data should be the basis for decision-making. zach et al. (2014) made a similar finding regarding the adoption of erp. scott and bruce (1995) identify four decision-making styles in practice. these are heuristics/intuition, rational decision-making, networking (deferring to others for decisionmaking) and an avoidance decision-making style. only rational decision-making can lead to advanced analysis skills using bi&a tools and data (davenport et al., 2010; seddon et al., 2017). using scott and bruce (1995) as a guide, three main categories of decision-making styles were identified from the interview data (see column a in table 4): (1) heuristics and working memory, (2) rational decision-making using data and, in some cases, simple bi&a tools such as microsoft excel® for sorting of data and simple calculations and (3) a mix of (1) and (2). an example of using heuristics instead of bi&a methods is the director of sme2, a recruitment sme. while she has access to data from job websites (equivalent to tps data), she prefers heuristics and networking. this type of decision-making style does not match a typical bi&a mindset that will lead to adopting and implementing bi&a tools. she describes it as follows: "seek.com [an australian job application system] is a barrel (resume database), everyone who wants a job will put their resume in that barrel… i do not want to take from the same barrel... for local positions, i will use word of mouth. my mobile is very useful for me in using my network and contacts. it is bound to get better results, although it requires more effort… the people who come for interviews will tell me the good things they want me to hear… i will paint the worst picture of the job and will look at their reaction, not interested in what they say. based on that, i have a feeling [heuristics] of whether they will suit the position or not." gigerenzer and gaissmaier (2011, pg. 454) define heuristics as "a strategy that ignores part of the information, to make decisions more quickly, frugally and/or accurately than more complex methods." the aspect of personal networking provides smes with more access to resources, capabilities, complementary skills and knowledge that are not available internally (gronum et al., 2012). networking allows smes to increase information, commercialised australasian journal of information systems wee, scheepers & tian 2022, vol 26, research article understanding the processes 13 knowledge and skills (olsen et al., 2012; teirlinck & spithoven, 2013). sme4 (the retail pharmacist) tends to use a mix of heuristics and networking. the retail pharmacist commented that he gets a lot of information from sales representatives on the benefits of the pharmaceutical products they are promoting and, given their sales interest, he relied on his networks to provide an unbiased view of the effectiveness of the available products. hence although tps data is available, the pharmacist relies on his network and does not use bi&a tools. amongst other smes, there are owners/managers who use simple bi&a tools on the data in the tps. for example, the coffee bean wholesaler (sme5) owns a few cafes employing 15 fulltime staff. the owner-manager described the use of data as follows: "in terms of [an] accounting system, we use [name of a pos software]. on the point of sales, we use a pos system ... so, basically, [name of a pos software] will record all the data, we will be able to assess every single shelf item we have at home, on our mobile phone… we make a lot of decisions based on the data. for instance, like, if we look at the data, if items are not selling well, they will be off the menu next season. we do everything based on data… we only believe in data because data never lies." sme 5 uses pos data to track items that sell, stock, monitor customer demand and profitability. sme1, 3 and 5 use rational analysis as their main decision-making method. this section reveals that smes use a range of decision-making styles, with smes 1, 3 and 5 leaning more towards rational decision-making related to bi&a tools. whereas sme 2 uses heuristics and networking and sme 4 uses both styles depending on the type of decisions. we found that the expertise gained from experience in using data for making decisions for operational purposes (path 1, figure 2, no. 2) and strategic purposes (path 2, figure 2, no. 3) increased the knowledge of owner/manager and other analytical people (path 3, figure 2, no. 8). the owner/manager perceive value from the use of data, which then lead them to invest in improving the quality of analytical resources. hence, we speculate that there should be a feedback loop from 'organisational benefit' (figure 2, no. 4) to analytical resources as the experience and expertise of analytical people (figure 2, no. 7) are developed. 4.3 bi&a built into business processes this second construct (table 4, column b) focuses on decisions being applied to business processes, which relates to path 1, path 2 and path 3 of basm. throughout this analysis, the path construct is used to show how resources are orchestrated. therefore, the focus in this section is on a summarised singular integration of bi&a. the integration of bi&a into business processes ranges from simple to complex. there are three levels of bi&a integration into business processes (see column b of table 4): no bi&a application in business processes, bi&a adapted processes in some functions (this can be in marketing or financial functions) and lastly, organisation-wide use of bi&a, which include most major business functions, such as marketing and finance, operations, customers or supply chain. the first category, no bi&a built into business processes, is related to a heuristics decisionmaking style of the owner/manager, where decisions are made in an ad-hoc manner and not based on data. for example, in sme2, the recruitment director relies on heuristics and personal networks for decision-making. decisions are made spontaneously and not incorporated into a australasian journal of information systems wee, scheepers & tian 2022, vol 26, research article understanding the processes 14 standard business process. it reflects a personal and individual management style dependent on personal intuition and contacts. the second category is where smes use bi&a insights in some functional processes (sme3 and sme5) which is classified in 'some functions' in table 4 column b. for example, sme5 uses the data in pos to determine what products to introduce or discontinue under the sales function or how much stock to hold under the supply chain function. these functions that have been bi&a enabled are accessing some reporting capability in the pos or extracting data into a spreadsheet. smes at this stage are starting to realise the benefits of employing bi&a in their business processes and are slowly extending bi&a to other functions where it is feasible. the last category uses bi&a in processes organisation-wide with overarching performance monitoring over many business processes. for example, the ceo of sme1, the charity, explained: "internally, we asked many questions, for example, what are early indicators of problems? i brought the moving average metric from the stock market and tweaked it to the correct corresponding period moving average. customer complaints are also a very important source of information for us to improve. we also debated and agreed on the 'importance' hierarchy of information and business rules. quite earlier on, i created an organisation wide kpi [key performance indicators] dashboard and over time, each manager/department had its own kpi to watch out for. every campaign was tracked for the degree of success as any new initiative." as the quote indicates, the ceo of sme1 developed an overarching dashboard that displayed key performance indicator (kpi) data to measure organisation-wide business processes. according to the ceo, this was achieved through many iterations of small bi&a projects over five years. these improvements are examples of path 2, where a more holistic view was used to develop new strategic directions. this was made possible through changes and modifications to the analytical resources of the organisation. in summary, we can relate the three levels of bi&a integration in smes to the three paths in basm. the first category signifies that no bi&a adoption has occurred where the sme is still functioning with heuristics and networking, resulting in no basm paths. when bi&a is introduced and integrated into some functional business process, path 1 is initiated where the organisation's current resources are actioned. the iterative success of these integrations will result in greater knowledge and skills in transforming data into insights (path 2), leading to actions that will change the organisation's resources. this improved knowledge and skills will initiate path 3 in building and improving the organisation's analytics resources (analytical people and bi&a culture). we have also found that the expertise gained from experience in paths 1 and 2 enhances and increases knowledge by analytical people (path 3) for better decision-making. hence, there should be a feedback loop from 'organisational benefit' (figure 2, no. 4) to 'analytical resources' as bi&a experience and expertise of analytical people (figure 2, no. 7) are developed. at the same time, the bi&a culture (figure 2, no. 4) can be further developed. 4.4 types of bi&a culture the original basm did not include the construct of bi&a culture. in our adapted basm, we have included bi&a culture as part of 'analytical resources' (figure 2, no. 4). the previous section indicated an interrelationship between the decision-making process (table 4, a) and the inclusion of bi&a in business processes (table 4, b). similar interrelationships can be australasian journal of information systems wee, scheepers & tian 2022, vol 26, research article understanding the processes 15 identified between the decision-making process (table 4, b) and organisational culture (table 4, c). cultural growth is a gradual process and the coding suggests three different levels of culture (table 4, c): individualism, influencing and bi&a culture. we code individualism (defined as: decision making by an individual) in the interview data as the owner/manager making decisions using mainly heuristics. decision-making in smes is typically centralised and the information is also localised to the owner/manager (zach et al., 2014). an owner/manager can be the catalyst for change or the main obstruction. a clear example of an sme owner/manager operating in such a manner is the owner of a retail pharmacy (sme4). below, he explains his decision-making style and culture: "i also do the book-keeping side of the business. keeping an eye on when i do the monthly bas statement [taxation-related process]. just seeing figures in the bas reports. i don't micro analyse things. from a business sense, mainly ordering stock, maintaining stock level, decisions to introduce new stock to our shelves, also services as well." influencing is a cultural aspect where the sme owner/manager is personally convinced of the benefits of bi&a and has practised bi&a techniques to a sufficient level to demonstrate and influence other employees in the organisation. this type of behaviour increases the analytical resources (figure 2, no. 4, 6 and 7) related to path 3. both sme3 and sme5 exhibit such characteristics by introducing controls where employees enter the correct information into the tps. this influencing can also be associated with path 2, where activities change the organisation's resources to achieve more reliable information. bi&a culture is the highest level of 'culture' and is defined as a shared understanding and desire to make better business decisions through data usage, bi&a principles and tools. this is best illustrated by the ceo of sme1when he described the 5-year journey of implementing bi&a by training, motivating and prodding his employees. this journey is an example of both path 2 and 3 in the basm model (figure 2). the following quote from the ceo of sme1 indicates how the organisation moved from individualism to a bi&a culture. "when i first joined as a ceo 5 years ago, department managers used their intuition and it was not backed by information. i had to inculcate them with a culture of philanthropy and fact-based decisionmaking… other than information issues, obstacles were power struggles and differing opinions and attitudes, basically people. these result in different ways of measuring results. eventually, when i started standardising business rules and measures as well as changing some business processes, there was some employee turnover." the quote from the ceo of sme1 highlights some employee opposition to bi&a. these employees subsequently left the organisation. however, the ceo set an example by reorganising the departments and placing the is directly under his control. he also personally coached the team and jointly debated the kpis to where managers took responsibility for certain kpis. he also made a conscious decision to use microsoft excel® as a simple bi&a tool to achieve simplicity and accessibility to all employees. davenport (2010) views culture as an accumulation of attributes and behaviour of individuals over time and is a combination of pushing back on inappropriate behaviours whilst encouraging correctness which, in this case, is making decisions based on data. although the pushback resulted in three management staff departing simultaneously, it paved the way for the ceo to hire new people who were receptive to the ceo's analytical decision-making style. the five-year journey, as narrated by the ceo of sme1, highlights the gradual cultural change australasian journal of information systems wee, scheepers & tian 2022, vol 26, research article understanding the processes 16 in implementing bi&a as an innovation. the ceo began by trying to understand the data of the organisation and proceeded to work with functional managers, ultimately reaching a point where the entire organisation accepted kpis and was on board with using data analysis for decision-making. though bi&a culture is intangible, it plays an important part in all three paths of the basm. the original basm describes the process being executed iteratively in different parts of the organisation. what is not depicted in the basm model is the growth of the bi&a culture initiated by the sme owner/manager spreading to other employees resulting in new/modified business process changes brought about by bi&a insights. the findings suggest that bi&a culture begins with sme owner/manager and grows with iterative execution of the three paths. 4.5 level of information use the level of information use is not explicitly apparent in the basm model. information use is primarily generated within path 1, focusing on operational level information use. as information use increases, path 2 is activated to improve organisation resources. as subsequent iterations increase the use of information, path 3 is triggered to change the organisation's analytical resources, leading to the strategic use of information. these identified levels of information use are not new. decision support literature has identified three levels of is: transactional processing systems for operational information needs, management is for management needs and executive support systems for strategic needs (olsen et al., 2012; teirlinck & spithoven, 2013; weill & broadbent, 1998). the 'level of information use' codes how information is used in the smes using these three levels (see table 4, d). table 4 shows the list of five smes with two of the businesses (sme2 and sme4) using information for operational purposes, two (sme3 and sme5) for management information purposes and one (sme1) for strategic objectives. an example of operational use is the retail pharmacy (sme4), where the owner-manager relies on the pos system to check stock levels and product sales when restocking. this use of information relates to path 1, where the organisation's resources (in the pos) are used to optimise another organisational resource of stock for sale. management goals are a higher-level goal than operational goals as it considers many variables, usually from different but related sources of information. an example of using information for a management goal is sme5 which incorporates information from different sources such as their pos system, accounting system and labour management systems to manage their wholesale and retail functions. therefore, while path 1 remains the same, paths 2 and 3 become more critical in achieving a higher goal and more organisational and analytical resources (path 3) will be required. the use of information for strategic planning relates to longer-term challenges regarding competitive forces and the external environment. for example, the charity ceo (sme1) introduced industry metrics to benchmark their organisation against comparable charities. regarding risks from the external environment, the ceo identified lower interest rates as an external strategic issue, as it deters people from donating. moreover, sme1 has invested in a bi&a system that firstly stores internal data, collects external data and generates reports through a sql server. the ability for sme1 to generate reports and insights on its performance and gather external data provides it with extensive information to plan strategically. australasian journal of information systems wee, scheepers & tian 2022, vol 26, research article understanding the processes 17 developing an information system that combines many aspects of internal and external information can be seen as the highest state of paths 2 and 3, where superior analytical and organisation resources are required. 4.6 financial investment the term ‘investment’ suggests that additional financial resources are required, which would relate to improvements to the enabling technology (path 3). therefore, financial investment (table 4, e) is a key construct in this study as bi&a enabling technologies are expensive, as are additional resources that lead to a higher total cost of ownership. according to dholakiya (2016), many smes are hesitant to implement bi&a due to concerns about cost and effort. the cost of existing tps is not included in this evaluation, as their primary purpose is to facilitate the daily operational business processes. the financial investment for each sme is reflected in column e of table 4. the three levels of financial investment are not explicit in basm because it eventuates in multiple iterations of the three paths in an sme. for example, as the sme owner/manager experiments with a bi&a tool for the first time with an individualistic culture (table 4, column c), a low financial investment ($) is required for additional software licenses. this experimentation is in line with the concept of 'necessity bricolage' in which smes make do with whatever they have to achieve new goals (jaouen & nakara, 2015). a medium investment ($$) is needed when the sme embarks on the path 2 or 3 of bi&a enabling technologies, requiring more hardware and software (table 4, column c). ultimately, the complexity of gathering, cleaning and joining disparate functional data at the organisational level requires significant high investment ($$$$). the value of insight increases with the need to disseminate the insight to appropriate employees with legitimate access. with the above overview, examples from the case organisations are discussed below. sme4 had the lowest level of bi&a investment, where the retail pharmacist worked individually to gauge customer demand and restock the retail pharmacy. there was hardly any additional investment required beyond the existing pos systems where sales records and trends were recorded. with respect to the basm, path 1 is carried out with very minimal forays into paths 2 and 3 as there was little incentive to invest in better bi&a tools. the mid-level financial investment is when the sme owner/manager works across functions to solve management-level problems. in the case of sme5, the coffee wholesaler invested in a shared server to store data from various systems (pos and an online labour vouching system) to gain and share further insights. this is an example where paths 1, 2 and 3 were initiated to increase experience, expertise, confidence and achieve net financial gains given the investment size. jaouen and nakara (2015) refer to this as 'strategic bricolage' which aims to boost success, performance and sometimes growth. the next example with the highest level of investment is sme1, which was achieved over five years in multiple phases. the ceo was able to justify and build the bi&a infrastructure over multiple financial years. these include enhancing the reporting module of crm and upgrading it systems to develop performance dashboards. the ceo shared that he could define performance metrics and justify to the charity board the it resources to achieve those measures. the above findings support the iterative and gradual investment strategy put forth by llave et al. (2018), where initial success stories and organisational learning improve future bi&a investment decisions. table 4, column e shows that financial investment costs and returns are australasian journal of information systems wee, scheepers & tian 2022, vol 26, research article understanding the processes 18 incremental. for example, as smes move from operational to management and then strategic levels of information use, they have to invest more in the technology. while basm does not consider the factor of financial investment, such matters are considered by owners/managers of smes. a lack of financial resources is an enduring constraint for smes investing in is (zach et al., 2014). the sme owner/manager is the same person possessing both the cost/benefit information and making the executive decision to invest, which is reflected in the basm model for smes. what is of significance is that iterative executions of prior bi&a projects will increase knowledge of both likely cost and benefit (an example of path 3). this also reinforces the suggestion that there should be an 'enabling loop' from organisational benefits to 'analytical people'. 5 discussion 5.1 three stage bi&a implementation model in this section, we discuss three bi&a implementation stages derived from the identified constructs (table 4, columns a-e) and how they relate to the three paths of basm. while this is not a longitudinal study, more mature bi&a practitioners (e.g. sme1) have narrated the journey they have taken and the challenges they overcome along the way. in addition, the less mature practitioners offer a snapshot of where they are in their bi&a evolution journey. table 5 below shows three stages of bi&a implementation in smes in conjunction with the three basm paths. each of these stages (novice, cross-functional and organisation-wide) is now explained in more detail. dimensions novice cross-functional organisation wide a decision-making process heuristics to simple bi&a simple bi&a reports and dashboards basm path 1 path 1,2 & 3 path 1,2 & 3 b bi&a built into business process na some functions organisation wide basm path 1 path 1,2 & 3 path 1,2 & 3 c types of bi&a culture individualistic influencing bi&a culture basm path 1 path 1,2 & 3 path 1,2 & 3 d level of information use operational goals management goals strategic goals basm path 1 path 1,2 & 3 path 1,2 & 3 e financial investment $ $$ $$$$ basm path 1 path 1,2 & 3 path 1,2 & 3 table 5. relationship between three-stage bi&a implementation model and basm 3 paths from a basm perspective, path 1 is where a novice (sme owner/manager) will start engaging in bi&a through many areas identified in the constructs. as organisational benefits are achieved with data from a single function, the sme owner/manager moves on to a crossfunctional stage to apply bi&a to a combination of data from different functional areas. higher management goals are sought at this juncture and employees need to be persuaded to use bi&a. paths 2 & 3 are likely to be triggered, where additional organisational resources will australasian journal of information systems wee, scheepers & tian 2022, vol 26, research article understanding the processes 19 be needed to cater to more users. insights gained should be applied to other functional areas and investment needs to be increased to enable the pooling and sharing of information. all three paths are triggered where the organisational resources are improved along with bi&a knowledge and skills. similarly, all three paths are used in the third organisation-wide stage, but with a strategic goal for information use and organisation-wide integration of processes. novice two smes (sme2 and sme4) are in the novice phase of bi&a implementation. the owners/managers use a heuristic decision-making style that can include some simplistic bi&a analysis. these owners/managers work individually to solve operational goals and they do not need to justify their decisions to others in the organisation. in addition, both owners/managers (of sme2 and sme4) showed a preference for personal networks. this finding is supported by gudfinnsson et al. (2019), who found limited motivation for smes to invest in bi&a efforts. the owners/managers do not perceive value from the use of bi&a, although they have the tps that generates data for bi&a. cross-functional two organisations (sme3 and sme5) are classified in the cross-functional phase of bi&a implementation. in this phase, the main characteristic of the organisations is that the owner/manager uses a rational decision-making style and relies on data for making decisions. the owner/manager drives the changes at the organisational level, such as integrating their bi&a insights into some functional business processes (table 4, column b), influencing bi&a culture (table 4, column c) and using bi&a to solve problems at the management level (table 4, column d). in this category, the sme owners/managers have put the organisation on the path of implementing bi&a. organization wide only sme1 is at the organisation-wide stage in bi&a implementation. this organisation integrates bi&a applications into organisation-wide processes (table 4, column b) and addresses challenges and concerns at a strategic level (table 4, column d). the sme ownermanager has also instilled a bi&a culture (table 4, column c) throughout the organisation via numerous discussions with the management team, designing and using an overarching performance dashboard to monitor organisation-wide business processes (table 4, column c). the organisation can then introduce new campaigns in the future to achieve strategic customer acquisition goals (table 4, column e) and increase donor lifespans. this was accomplished by a gradual process of building a bi&a culture through motivating, training and incentivising bi&a targets and performance metrics. bi&a culture encompasses data orientation (kiron & shockley, 2011), analytical decision-making (popovič et al., 2012) and information infrastructure (presthus, 2014). a bi&a culture promotes an inclusive environment where information processes are understood and supported. the impetus for leveraging bi&a applications comes from the management and users rather than the availability of technology (skyrius et al., 2016). progression from one stage to another one advantage that smes have is their smaller and flatter organisation structure and simpler business processes (zach et al., 2014). the owners/managers take the championing role by acting on an individual basis and then translating into a gradual cultural organisational australasian journal of information systems wee, scheepers & tian 2022, vol 26, research article understanding the processes 20 change. the direct outcome of this transition is that bi&a insights are applied on an individual, function and organisation-wide basis. the lack of resources in smes is clearly expressed by a lack of a central data warehouse or database amongst less mature bi&a implementers. buying these technologies is expensive and is only viable if the organisation has instilled an organisation-wide bi&a culture to appreciate such an investment. hence, amongst the five dimensions, cultural change factors and extent of bi&a insights in business processes signal the key transition phases in bi&a implementation in smes. the narratives from the ceo of the mature sme demonstrate a progression in skills through influencing, guiding and coordinating employees in using analytical tools and integrating insights into the business processes. transitioning an sme from a novice to an organisation-wide stage is a lengthy process, as indicated by sme1 (5 years). some managers had strong opposition to this new bi&a culture and left the organisation as a result of internal member disputes. gartner (2008) indicated that obstacles are more significant outside technology areas, such as sponsorship, politics and data quality rather than in deploying bi&a infrastructure, tools and applications. the findings related to the sme basm process model (see figure 2) indicate that the key to any change on an organisational level is 'decision(s) made by the owner/manager'. this decision initiates path 3 to create the 'analytical resources' of 'enabling technologies (figure 2, no. 6), 'analytical people' (figure 2, no. 7) and most importantly, a 'bi&a culture' (figure 2, no. 4). most smes can initiate path 1 as they have the necessary operational data in their is. when owners/managers perceive a value at an operational level, they can action path 3 to improve or instigate the 'analytical resources' of the sme. path 2 is mainly related to the use of bi&a for strategic purposes (seddon et al., 2017). only one sme (sme1) was able to use bi&a for strategic purposes. the stage of the cross-functional value indicates that an additional path can be added to the basm model as these organisations derived managerial value from the use of bi&a which can result in changes in organisational resources. 5.2 assessment of the adapted basm from the findings of the five sme case studies, the adapted basm can only serve as a conceptual model viewed at a macro level. theoretically, it offers a simplistic view of how analytical insights can translate organisational resources to analytical capabilities resulting in organisational benefits. therefore, we propose that basm be revised to include a feedback loop from the organisational benefits to the analytical resource (see figure 4 below). success from iterative bi&a projects will increase bi&a technology, processes, goal setting and investment in bi&a knowledge and skills. we believe the proposed revisions to basm improves the original in three areas: • clearer identification of the ideal bi&a champion in an sme organisation should be the owner/manager. • stronger importance placed on bi&a culture that the sme owner/manager must cultivate. • presence of a feedback loop from 'organisational benefits' to 'analytical resources'. every bi&a project, whether successful or not, will bring best practices or lessons learned for improvement. australasian journal of information systems wee, scheepers & tian 2022, vol 26, research article understanding the processes 21 figure 4. revised basm for sme based on study findings however, as a high-level conceptual model, the revised basm lacked depth and did not address the practical challenges smes face regarding goal prioritisation, skills and resources. therefore, we propose that the three-stage implementation model can augment the macrolevel conceptual basm, identifying smes' issues and challenges regarding goals, skills, process improvements and bi&a tool investments. 6 conclusion the paper describes how smes adopt bi&a and proposes an improved basm as a theoretical model for exploration. llave (2019) identifies a lack of research into the use of bi&a by smes and calls for more research in this area. this paper investigates and adapts the basm model based on the unique characteristics of smes and data collected from 5 smes using bi&a. the findings reveal three stages smes go through in adopting bi&a. the analysis further highlights the importance of the owner/manager as the champion of bi&a and the importance of developing a bi&a culture. owners/managers of five smes from various industries were interviewed to ascertain how they use bi&a to derive business value. we found that the basm model can be adapted for smes as a high-level conceptual process model, but the adapted basm model lacks sufficient details on the changes that smes need to make to adopt and use bi&a. three stages can be identified: novice, cross-functional and organisation-wide. therefore, we propose the three-stage implementation model to extend and augment the revised basm to better understand sme owners/managers' concerns during adoption. this study contributes to both theory and practice. in theory, this paper adapts the basm process model for sme bi&a adoption, considering the unique characteristics of smes. furthermore, it adds the element of bi&a culture and a feedback loop that enhances the quality and proficiency of analytical resources. australasian journal of information systems wee, scheepers & tian 2022, vol 26, research article understanding the processes 22 in practice, the outcomes of this study will benefit smes that are considering using bi&a to derive business value in smes. the findings provide a clear and coherent road map that allows sme owners/managers to optimally use bi&a to realise benefits for their businesses. the three stages also indicate practical and incremental steps to achieve operational, management and ultimately strategic goals from bi&a. smes embarking on this journey can plan their implementation and have a greater understanding of the three stages of bi&a value creation. case study research suffers from limitations, especially with a small number of cases. hence, there may be concerns about generalisability. however, stake (1978, pg. 6) posits that such studies can produce a "naturalistic generalization" in which the results of this study may be applied to smes adopting bi&a in decision making and business processes across different levels of the business to achieve specific goals. further research is needed to determine if the findings can be generalised more broadly. the sme basm process model constructs and subconstructs can be quantitatively validated to test and measure their strengths and correlations. references agostino, a., solberg søilen, k., & gerritsen, b. (2013). cloud solution in business intelligence for smes – vendor and customer perspectives. journal of intelligence studies in business, 3(3), 5-28. doi:10.37380/jisib.v3i3.72 ato. (2020). business activity statements (bas). retrieved from https://www.ato.gov.au/business/business-activity-statements-(bas)/ baker, t., & nelson, r. e. (2005). creating something from nothing: resource construction through entrepreneurial bricolage. administrative science quarterly, 50(3), 329-366. barney, j. (1991). firm resources and sustained competitive advantage. journal of management, 17(1), 99-120. doi:10.1177/014920639101700108 beer, s. (1984). the viable system model: its provenance, development, methodology and pathology. journal of the operational research society, 35(1), 7-25. benbasat, i., goldstein, d. k., & mead, m. (1987). the case research strategy in studies of information systems. mis quarterly, 11(3), 369-386. bowden, e. m., jung-beeman, m., fleck, j., & kounios, j. (2005). new approaches to demystifying insight. trends in cognitive sciences, 9(7), 322-328. davenport, t., harris, j., & morison, r. (2010). analytics at work: smarter decisions, better results. boston, ma: harvard business press. dholakiya, p. (2016). the 4 biggest business-intelligence challenges facing entrepreneurs. retrieved from https://www.entrepreneur.com/article/280432 eisenhardt, k. m. (1989). building theories from case study research. academy of management review, 14(4), 532-550. fink, l., yogev, n., & even, a. (2017). business intelligence and organizational learning: an empirical investigation of value creation processes. information & management, 54(1), 3856. gartner. (2008). gartner reveals nine fatal flaws in business intelligence implementations. press release. retrieved from https://www.gartner.com/newsroom/id/774912 https://www.ato.gov.au/business/business-activity-statements-(bas)/ https://www.entrepreneur.com/article/280432 https://www.gartner.com/newsroom/id/774912 australasian journal of information systems wee, scheepers & tian 2022, vol 26, research article understanding the processes 23 gigerenzer, g., & gaissmaier, w. (2011). heuristic decision making. annual review of psychology, 62, 451-482. gronum, s., verreynne, m. l., & kastelle, t. (2012). the role of networks in small and mediumsized enterprise innovation and firm performance. journal of small business management, 50(2), 257-282. guarda, t., santos, m., pinto, f., augusto, m., & silva, c. (2013). business intelligence as a competitive advantage for smes. international journal of trade, economics and finance, 4(4), 187-190. gudfinnsson, k., rose, j., & aggestam, l. (2019). tackling lack of motivation in aspirational analytics companies: sme examples from the manufacturing industry. international journal of business intelligence research (ijbir), 10(1), 1-18. hauser, a., eggers, f., & güldenberg, s. (2020). strategic decision-making in smes: effectuation, causation, and the absence of strategy. small business economics, 54(3), 775790. doi:http://dx.doi.org/10.1007/s11187-019-00152-x jaouen, a., & nakara, w. a. (2015). ‘bricolage’ in the implementation and the use of is by micro-firms: an empirical study. in (pp. 449-458). cham: cham: springer international publishing. kiron, d., & shockley, r. (2011). creating business value with analytics. mit sloan management review, 53(1), 51-57. klein, h. k., & myers, m. d. (1999). a set of principles for conducting and evaluating interpretive field studies in information systems. mis quarterly, 23(1), 67-94. llave, m. r. (2019). a review of business intelligence and analytics in small and mediumsized enterprises. international journal of business intelligence research (ijbir), 10(1), 1941. llave, m. r., hustad, e., & olsen, d. h. (2018). creating value from business intelligence and analytics in smes: insights from experts. twenty-fourth americas conference on information systems, new orlean, 1-10. mahmoud, t., marx gómez, j., rezgui, a., peters, d., & solsbach, a. (2012). enhanced bi systems with on-demand data based on semantic-enabled enterprise soa. ecis 2012 proceedings. mikalef, p., boura, m., lekakos, g., & krogstie, j. (2019). big data analytics capabilities and innovation: the mediating role of dynamic capabilities and moderating effect of the environment. british journal of management, 30(2), 272-298. mikalef, p., pappas, i. o., krogstie, j., & pavlou, p. a. (2020). big data and business analytics: a research agenda for realizing business value. information & management, 57(1), 103237. doi:10.1016/j.im.2019.103237 newell, a., & simon, h. a. (1972). human problem solving (vol. 104): prentice-hall englewood cliffs, nj. niemi, e., & pekkola, s. (2019). the benefits of enterprise architecture in organizational transformation. business & information systems engineering, 62(6), 1-13. http://dx.doi.org/10.1007/s11187-019-00152-x australasian journal of information systems wee, scheepers & tian 2022, vol 26, research article understanding the processes 24 oecd. (2017). enhancing the contributions of smes in a globalised and digital economy meeting of the oecd council at ministerial level retrieved from https://www.oecd.org/mcm/documents/c-min-2017-8-en.pdf olsen, n. v., elvekrok, i., & nilsen, e. r. (2012). drivers of food smes network success: 101 tales from norway. trends in food science & technology, 26(2), 120-128. parker, c. m., burgess, s., & al-qirim, n. (2015). a review of studies on information systems and smes in high ranked is journals (2000-2014). ajis. australasian journal of information systems, 19. doi:10.3127/ajis.v19i0.1219 patton, m. (2002). qualitative research and evaluation methods. thousand oaks. sage publications. payne, g. t., pearson, a. w., & carr, j. c. (2017). process and variance modeling: linking research questions to methods in family business research. in (vol. 30, pp. 11-18): sage publications sage ca: los angeles, ca. popovič, a., hackney, r., coelho, p. s., & jaklič, j. (2012). towards business intelligence systems success: effects of maturity and culture on analytical decision making. decision support systems, 54(1), 729-739. doi:10.1016/j.dss.2012.08.017 power, d., & gruner, r. l. (2017). variable use of standards-based ios enabling technologies in australian smes: an examination of deliberate and emergent decision making processes. european journal of information systems, 26(2), 164-184. doi:10.1057/s41303-0170034-5 presthus, w. (2014). breakfast at tiffany s: the study of a successful business intelligence solution as an information infrastructure. paper presented at the twenty second european conference on information systems (ecis), tel aviv. pretz, j. e., naples, a. j., & sternberg, r. j. (2003). recognizing, defining, and representing problems. the psychology of problem solving, 30(3), 3-30. raymond, l. (2003). globalization, the knowledge economy, and competitiveness: a business intelligence framework for the development smes. journal of american academy of business, cambridge, 3(1-2), 260-260. renken, j., & heeks, r. (2019). champions of is innovations. communications of the association for information systems, 44(1), 811-851. sadok, m., & lesca, h. (2009). a business intelligence model for smes based on tacit knowledge. communications of the ibima, 7(20), 1-10. scott, s. g., & bruce, r. a. (1995). decision-making style: the development and assessment of a new measure. educational and psychological measurement, 55(5), 818-831. seddon, p. b., constantinidis, d., tamm, t., & dod, h. (2017). how does business analytics contribute to business value? information systems journal, 27(3), 237-269. seethamraju, r. (2014). adoption of software as a service (saas) enterprise resource planning (erp) systems in small and medium sized enterprises (smes). information systems frontiers, 17(3), 475-492. doi:10.1007/s10796-014-9506-5 https://www.oecd.org/mcm/documents/c-min-2017-8-en.pdf australasian journal of information systems wee, scheepers & tian 2022, vol 26, research article understanding the processes 25 skyrius, r., katin, i., kazimianec, m., nemitko, s., rumšas, g., & žilinskas, r. (2016). factors driving business intelligence culture. issues in informing science and information technology, 13, 171-186. soh, c., & markus, m. l. (1995). how it creates business value: a process theory synthesis. paper presented at the icis 1995 proceedings. stake, r. e. (1978). the case study method in social inquiry. educational researcher, 7(2), 5-8. statistics, a. b. o. (2021, 16 feb 2021). counts of australian businesses, including entries and exits. retrieved from https://www.abs.gov.au/statistics/economy/businessindicators/counts-australian-businesses-including-entries-and-exits/latestrelease#states-and-territories sternberg, r. j., & davidson, j. e. (1995). the nature of insight. cambridge, ma: the mit press. teece, d. j., pisano, g., & shuen, a. (1997). dynamic capabilities and strategic management. strategic management journal, 18(7), 509-533. teirlinck, p., & spithoven, a. (2013). formal r&d management and strategic decision making in small firms in knowledge-intensive business services. r&d management, 43(1), 37-51. trieu, v.-h. (2017). getting value from business intelligence systems: a review and research agenda. decision support systems, 93, 111-124. van de ven, a. h. (2007). engaged scholarship: a guide for organizational and social research: oxford university press on demand. vidgen, r., shaw, s., & grant, d. b. (2017). management challenges in creating value from business analytics. european journal of operational research, 261(2), 626-639. walsham, g. (1995). the emergence of interpretivism in is research. information systems research, 6(4), 376-394. walsham, g. (2006). doing interpretive research. european journal of information systems, 15(3), 320-330. weill, p., & broadbent, m. (1998). leveraging the new infrastructure: how market leaders capitalize on information technology: harvard business press. zach, o., munkvold, b. e., & olsen, d. h. (2014). erp system implementation in smes: exploring the influences of the sme context. enterprise information systems, 8(2), 309-335. doi:10.1080/17517575.2012.702358 copyright: © 2022 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. https://www.abs.gov.au/statistics/economy/business-indicators/counts-australian-businesses-including-entries-and-exits/latest-release#states-and-territories https://www.abs.gov.au/statistics/economy/business-indicators/counts-australian-businesses-including-entries-and-exits/latest-release#states-and-territories https://www.abs.gov.au/statistics/economy/business-indicators/counts-australian-businesses-including-entries-and-exits/latest-release#states-and-territories http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems wee, scheepers & tian 2022, vol 26, research article understanding the processes 26 doi: https://doi.org/10.3127/ajis.v26i0.2969 https://doi.org/10.3127/ajis.v26i0.2969 1 introduction 2 literature review 2.1 smes' adoption of information systems (is) 2.2 theoretical model relating to bi&a implementation 3 research methodology 4 findings 4.1 enabling technologies 4.2 decision-making process 4.3 bi&a built into business processes 4.4 types of bi&a culture 4.5 level of information use 4.6 financial investment 5 discussion 5.1 three stage bi&a implementation model 5.2 assessment of the adapted basm 6 conclusion microsoft word 1678-5409-1-le.docx australasian journal of information systems morgan, cheong & bedingfield 2018, vol 22, research on recent advances in social media power to the people 1 “power to the people!”: social media discourse on regional energy issues in australia kerri morgan monash university & deakin university kerrimorgan@deakin.edu.au marc cheong monash university susan bedingfield monash university abstract social media provides people from all socio-economic sectors with the opportunity to voice their opinions. platforms such as twitter provide the means to share one’s opinion with little effort and cost. but do these media empower everyday people to make their voice heard? in this research, we introduce a novel approach for investigating the voice of different twitter groups on social media platforms, by combining text clustering and an analysis of cliques in the resulting network. we focus on a case study using twitter interactions with respect to energy issues, in particular the closure of coal-fired power stations such as hazelwood. implications from this study will benefit stakeholders from governments to industry to the ‘common man’, in understanding how discourse on social media reflects public consumer sentiment. keywords: social media, twitter, graph theory, data mining, pattern recognition. 1 introduction recently, there has been an increasing global focus on green energies (byrnes, brown, foster, & wagner, 2013; furlan & mortarino, 2017; twidell & weir, 2015). some consequences of this are an increasing number of new green technologies (bilgili, ozbek, sahin, & kahraman, 2015; kolokotsa, 2017; sen & bhattacharyya, 2014), the increasing likelihood of energy brownouts and blackouts (puddy, 2015), and the closure of traditional coal fired power plants (davies, 2014; evans & phelan, 2016; johnstone & hielscher, 2017). while in the past, wide-reaching discussion of issues of national or global importance has been in the domain of the press, industry leaders and politicians, the rise of social media has provided the opportunity for other stakeholders, in particular the vox populi of the ‘common man’, to voice their opinions on similar issues (shamma, kennedy, & churchill, 2009; dudleynicholson, 2016). in this paper, we consider the power of the voice of the ‘common man’ in social media. we present the results of our pilot study using the discussion on twitter about the closing of hazelwood power station, an event that affects a number of social issues such as employability and thus could be expected to trigger comments from a wide range of people. clustered behaviour is important in this context as groups of people often function as cohorts on social media. for this reason, we cluster the tweet contents in order to identify possible cohorts of users with common opinions. we then analyse the interactions between these australasian journal of information systems morgan, cheong & bedingfield 2018, vol 22, research on recent advances in social media power to the people 2 clusters using a new approach utilising clique analysis on the clustered tweet contents. the metadata for the twitter users in each cluster is illustrated using tableau. 2 background 2.1 case study: the hazelwood power station the discussion surrounding the closure of the hazelwood power station in the la trobe valley, australia, has provoked a variety of issues. these include the use of green energies, the cost of energy, the sufficiency (or insufficiency) of existing energy sources, and the uncertainty of employment for the hazelwood employees, and other related social issues (anderson, 2017). in this paper, we examine the interactions between different twitter stakeholders in the australian context, specifically around hazelwood, by studying the following key themes: interaction patterns, the impact of the views of the ‘common man’ and the behaviour of distinct groups of twitter users. 2.2 twitter dating back to initial experimental studies in 2007-2009, twitter data has been used extensively across academic disciplines to study human phenomena – manifest in discourse patterns and group behaviours in social media – in areas ranging from election campaigns to natural disasters to celebrity worship (cheong, 2013). we define several key terms as follows. tweet refers to a message sent on the twitter microblogging service, which is at most 140 characters long. we use the term tweet and message interchangeably. a tweeter is a colloquialism for a twitter user. both terms user and tweeter will be used interchangeably. a #hashtag is a keyword preceded by a hash (#) symbol that allows users to tag their tweets as belonging to a certain topic. a @user is a kind of message that is directed at, or replies to, another user, akin to an email reply. finally, a retweet, or an rt, indicates messages that are re-broadcasted by another user, similar to the concept of forwarding an email message. a full literature review of the general properties of twitter can be found in (cheong, 2013). twitter has an application programming interface (api) which allows researchers to download a proportion of tweets in an automated fashion. in addition to the tweets themselves, additional metadata about the tweet (e.g. timestamp, device used) as well as the users associated with the tweet (twitter user name, profile picture, etc.) can be collected to enrich data analysis. 2.3 clustering and pattern recognition in this study, we identify distinct cohorts of users based on the textual content of their tweets. clustering is a common method of segmenting structured data into similar subgroups, and is often used in social media research focusing on groups (e.g. zhang, et al., 2007 and jansen, et al., 2011). determining which data objects are similar is based on a predefined metric relevant to the particular data set. an overview of clustering algorithms is available in (xu & wunsch, 2005) and (abbas, 2008). specific text clustering algorithms can be applied to textual data such as a tweet. a useful survey of the currently used text algorithms can be found in (aggarwal & zhai, 2012). australasian journal of information systems morgan, cheong & bedingfield 2018, vol 22, research on recent advances in social media power to the people 3 2.4 representing a social network as a graph a graph is a set of vertices (or nodes) that model objects and edges, connecting pairs of vertices, that model relationships between the objects. graphs are used to model a wide range of networks. examples include environmental management (aittokallio & schwikowski, 2006; bunn, urban, & keitt, 2000), biological networks (mason & verwoerd, 2007), social networks (freeman, 1977) and electricity networks (holmgren, 2006). in this paper, we use standard definitions for graph concepts: see for example (bang-jensen & gutin, 2008; diestel, 2005). a clique is a subset of vertices in which every vertex is connected to every other vertex in the set by an edge. a (connected) component in a graph is a maximum set of vertices such that every pair of vertices in the set are connected by a path of edges. graphs are invaluable tools in modelling social networks, dating as far back as the 1930s (moreno, 1934). an example of a graph representation of a real-world social network is illustrated in figure 1 (bearman, moody, & stovel, 2004). figure 1. graph visualisation of a real-world social network. this is meant to represent romantic relationships between students in a high school in the usa. figure reproduced from bearman, et al (2004). the use of graphs in social network modelling and representation is even more relevant in the analysis of online social networks (osns) due to the richness of information that osns provide. in this paper, we use standard definitions and concepts found in social network literature; see for example (wasserman & faust, 1994). we designed our methodology around the research themes presented in section 3. based on the structure of the twitter social network and its general characteristics, four research questions are proposed. australasian journal of information systems morgan, cheong & bedingfield 2018, vol 22, research on recent advances in social media power to the people 4 3 research questions rq1. do the communication and engagement patterns reflect those of a typical social network? a typical online social network, such as twitter, exhibits ‘scale-free’ behaviour (barabási and albert, 1999), in contrast to localised real-world social networks, which are characterised as ‘small world’ (cheong, 2013; green, 2014). online social network users can form connections with people at any geographic location (green, 2014). an important characteristic of a scalefree network is that popular people “acquire hundreds, thousands, or even millions of [online] ‘friends’”, leading to a hierarchical distribution (green, 2014). in rq1, we examine the layout of communication and engagement between twitter users (and indirectly, messages) discussing the hazelwood power station issue. in particular, we examine the chain of @user replies to illustrate the flow of information between users. is the discourse between users about hazelwood typical of the scale-free behaviours exhibited on twitter and osns in general? rq2. does clustering effectively identify user groups? as described in section 4.2, the tweets in our sample are grouped into distinct clusters so that tweets with similar content are grouped together. each tweet cluster has a group of users associated with it. for convenience, we say the user ‘belongs to’ the cluster. a given user may belong to more than one tweet cluster depending on the range of topics that they tweet about. the number of clusters a given user belongs to ranges between 1 and 30 with the majority, approximately 67%, of users belonging to exactly one cluster and approximately 15% of users belonging to exactly two clusters. the remaining 18% minority of users belong to multiple tweet clusters implying that these tweeters used twitter to communicate about multiple issues during the sampling period. we then analyse the users belonging to each tweet cluster to determine whether there are specific user characteristics associated with each cluster’s tweet group. rq3. using graph-theoretic approaches, can we identify emergent communication patterns between clusters of twitter users? we examine the communication flow between the groups of users associated with each cluster. two groups of users associated with two different clusters are connected if there exists a directed path connecting the two clusters in the model of the network. the minimum distance of the path gives an indication of how closely the two groups of users are connected. the identification of groups of users that only communicate with other users within their own cluster may indicate that these users are essentially powerless in the network with respect to the tweet contents related to that specific cluster. rq4. does the ‘common man’ have a ‘powerful’ voice on social media? more explicitly: do australian twitter users ‘amplify’ their own voice, the voice of politicians, the media, and opinion-leaders (retweets)? a central theme in this research is to determine whether there is a group, or groups, of users that can be identified with ‘the common man’ and the effectiveness of their voice in communicating with other users in the network. australasian journal of information systems morgan, cheong & bedingfield 2018, vol 22, research on recent advances in social media power to the people 5 4 methodology and results 4.1 twitter data collection 4.1.1 collection method twitter metadata was collected via the twitter api using a php data collection framework. (borra & rieder, 2014) the twitter network was polled based on the following search queries. @greens the official twitter username of the australian greens, an environmentallyconscious political party that campaigned for hazelwood’s closure1 ‘hazelwood power’ a wildcard matching references to the hazelwood power station ‘snowy hydro’ a wildcard matching references to the snowy mountains hydro-electric scheme, in order to capture related discourse2. data was collected from 23 march 2017 to 14 june 2017, and includes the date of the official closure of hazelwood (29 march 20173), as well as during the resulting post-closure aftermath. due to the limitations of the twitter api on downloads, approximately 10% of all tweets containing the three search queries were collected (cheong, 2013; borra & rieder, 2014). the tweets in our data collection thus represent 10% of all available tweets, selected at random by the twitter api. 4.1.2 properties of the twitter corpus the corpus consists of 43,081 tweets with attached metadata records, originating from 9,075 unique twitter users. the sample size of tweets and messages are commensurate with – if not better – than traditional studies on social networks' messaging patterns based on "observable communications" (quintaine & kleinbaum, 2011; bernard, killworth, & sailer, 1980). the entire corpus is imported into the tableau bi suite to visualise the distribution of twitter messages over time (figure 2). all times are normalised to utc (universal coordinated time), to avoid issues with daylight savings time ending in victoria partway through the data collection window. figure 2. volume of twitter messages per day (normalised to utc), as visualised in tableau. 1 an example of their press releases: https://greens.org.au/magazine/vic/wins-hazelwood-transitionlong-way-go 2 see http://www.australia.gov.au/about-australia/australian-story/snowy-mountains-scheme 3 press release: http://www.ipplc.com.au/media/newsitem/end-of-generation-at-hazelwood australasian journal of information systems morgan, cheong & bedingfield 2018, vol 22, research on recent advances in social media power to the people 6 the first spike in the data coincides with reactions to prime minister malcolm turnbull’s plan to “underwrite the rail line for… [the] proposed massive adani central queensland thermalcoal mine”, a controversial issue as the project “threatens their livelihoods” (hull, 2017). the second spike concerns the australian greens supporting the “turnbull government… [in] overhauling school funding” (knott, 2017). the distribution of the dataset can be divided into three basic types of tweets: 1. @user messages there are 10,100 messages with the prefix “@user”, to denote addressivity of a particular tweet to another twitter user. 2. retweet or rt messages there are 25,940 messages with the prefix “rt” to denote retweeting of another’s message, either paraphrased or verbatim. 3. “ether” messages the remaining 7,041 messages are messages that are not addressed to, or targeted at, anyone in particular i.e. posting a generic ‘status update’ for public consumption. we term these non-directed messages “ether messages”, as they are akin to broadcasting a message to the ether. these groups will be revisited in section 4.2 when we apply clustering techniques. 4.1.3 addressing rq1 rq1. do the communication and engagement patterns reflect those of a typical social network? in addressing rq1, we represent the users in our corpus, as well as the connections between them, in terms of a social network graph. we must first define what a connection is. social networks such as twitter can contain different kinds of connections between users (cormode, krishnamurthy, & willinger, 2010). these include following (x follows updates posted by y), retweeting (x is forwarding/retweeting content made by y), and replying (x sends @user messages to y). to answer rq1, we intend to examine the layout of communication and engagement between twitter users discussing the hazelwood power station within the timeframe of data collection. hence, we define a connection between users as replies, or @user messages, between users4. having defined connection, we now extract all messages from our corpus that only match the @user behaviour described above. we determine if our corpus contains scale-free behaviour (cheong, 2013; green, 2014). a scalefree online social network is typified by a power-law distribution of connections between users. briefly, a power-law is defined as a relationship between two variables “in which a relative change in [the independent variable] ... gives rise to a proportional relative change in the [dependent variable] ... independent of [their]... initial size” (bar-yam, 2011). in this case study, our null hypothesis is that the frequency distribution of messages received fits a power law distribution. to test this, we generated a frequency distribution of @user message recipients in microsoft excel, as illustrated in figure 3. 4 ‘following’ is more longitudinal in nature and reflects long-term links or similarities between users, while studying ‘rt’ connections are normally used for studying the ‘amplification’ of an influencer’s message (e.g. politicians, celebrities, newsmakers). therefore, ‘following’ and ‘rt’ links are not suitable for answering rq1. australasian journal of information systems morgan, cheong & bedingfield 2018, vol 22, research on recent advances in social media power to the people 7 figure 3. frequency distribution of @user messages received by a user in our corpus. a power-law distribution trendline illustrated by the dashed line, was generated for the frequency distribution. a pearson r-squared coefficient of best-fit of 0.9638 was obtained which indicates a close fit between the empirical distribution and the estimated power-law model. in our investigation, we also visualise the distribution of such @user connections between users. we employ the pajek network package (batagelj & mrvar, 2004) to preprocess the raw graph data and gephi (bastian, heymann, & jacomy, 2009) to generate the final visualisation with emphasis on hubs. figure 4 visualises the network, with directed edges (arcs) representing the presence of connections between users (vertices). figure 4a: initial pajek network visualisation of all @user connections (grey lines) amongst target users (black dots) found in the corpus. australasian journal of information systems morgan, cheong & bedingfield 2018, vol 22, research on recent advances in social media power to the people 8 figure 4b: the same network, stylised in gephi to emphasise popular ‘hub users’ (filled nodes) -circle diameter and colour intensity are proportional to the number of messages (edges) received by a given user. from figure 4a, a giant connected component is obtained from the graph visualisation. however, to illustrate the scale-free nature of our network with better clarity, we stylised the graph using gephi to emphasise the presence of large hubs: a node (user) has size and colour proportional to the number of edges (@user messages) received. figure 4b illustrates the stylised version, where a few ‘hubs’, i.e. popular users with many @user messages received, can be clearly seen. our results above answer rq1 in the affirmative communication patterns relating to our chosen topic, reflected in @user messages between users, does reflect those exhibited in a typical social network. our investigations also show from that analysis at a macro level, i.e. figure 4, it is hard to visualise finer-grained detail about constituent users, considering the sheer number of connections. this leads us to conclude that it is useful to organise the users in distinct cohorts. ergo, we cluster the users by their unique texts, which will be the focus of rq2 below. 4.2 clustering of tweets for this research, distinct cohorts of tweets were identified by clustering the text contained in the tweets using the k-means algorithm. the content of each tweet was prepared for clustering by implementing established text pre-processing techniques including tokenising, stemming, and filtering tokens and stop words. the processing also included generating two word phrases, or n-grams where n=2. the data was then converted into a term frequency–inverse australasian journal of information systems morgan, cheong & bedingfield 2018, vol 22, research on recent advances in social media power to the people 9 document frequency (tf-idf) matrix (aggarwal & zhai, 2012) which is a structured form of the data used in the clustering process. the matrix was pruned so that the frequency of the tokens lay between 3 and 30 percent. the output of the clustering process was then used to create a social network graph. the structure of this graph was used to analyse discussions based on identified themes in the tweets. this process is described in section 4.3. 4.2.1 data analysis the processed tweets were clustered and a collection of 40 clusters (cohorts of tweeters) was identified. the clusters identified distinct groups of users based on the content of their tweets. we note that the k-means algorithm requires a specified value for k. in this case a value of k=40 was used as it provided a collection of distinct groups of tweeters that could be further analysed. a directed graph was then created to represent the network of interactions between these groups of tweeters. after clustering, we also identified key features of users in those clusters including the tweeter’s name and description (from the bio-data provided upon sign up to twitter). 4.2.2 question rq2. does clustering effectively identify user groups? the clustering is used to identify tweets with similar content. in section 5.1 we show that individual clusters of interest can be associated with distinct groups of users. 4.3 network analysis in this section, we identify features in the twitter network by modelling it as a graph. in section 5, the relationship between the graph and the underlying twitter network will be discussed. we analysed the network using the following graph properties: the largest/smallest clique to identify groups of users and individual users, source and sink vertices, that is, vertices with no in-going or no outgoing edges respectively, and the shortest directed path or trail between pairs of vertices to indicate the possible extent of individual cluster’s influence. two types of graphs were used to model the twitter network. the first was a directed graph that was used to model communication patterns between clusters, and the second was a graph of cliques that was used to model the interdependence of groups of clusters. 4.3.1 directed graph in order to study the interactions between different clusters, a directed graph representing communications between clusters was formed as follows. the vertex set represented the 40 clusters identified using clustering techniques described in section 4.2. we now extrapolate rq1’s definition of connections to clusters. for every pair of clusters ci and cj where an individual in cluster i tweeted an individual in cluster j, the graph has a corresponding directed edge ij. tweets between individuals in the same cluster are represented by loops, that is, edges connecting a vertex (i.e. cluster) to itself in the directed graph. only tweets between clusters -i.e. between agglomerations of users -were included in the edges of the graph. tweets in any of the three categories below were ignored: 1. @user messages (type 1 in section 4.1.2) addressed to users not found in any cluster (e.g. to a user whose metadata is not captured during data acquisition due to sampling limitations); australasian journal of information systems morgan, cheong & bedingfield 2018, vol 22, research on recent advances in social media power to the people 10 2. retweets (type 2 in section 4.1.2) whose purpose is not for inter-tweeter discourse, but merely ‘passing the message’ akin to an email ‘forward’; and 3. “ether” tweets (type 3 in section 4.1.2) that are not directed to anyone in particular. 4.3.2 results there were 6353 tweets between clusters. the maximum number of tweets between any pair of clusters was 927 tweets from individuals in cluster 0 to individuals in cluster 0 (a loop in the graph). the maximum number of tweets between any pair of distinct clusters was 252 tweets from cluster 11 to cluster 0. the resulting directed graph had 423 unique edges. the directed graph contained two components: the first component was a single isolated vertex corresponding to cluster 38, and a second component containing all the other clusters. individuals in cluster 38 only communicate to individuals in their own cluster: there is no communication with (and hence influence on, or from) the outside world. cluster 38 will be discussed in section 5.1. it would be expected that individuals in the same cluster would communicate with one another. this corresponds to a loop incident to the corresponding vertex in the directed graph. all vertices (and hence the cluster they represent) had loops except for the vertices corresponding to clusters 22, 26, 27, 35, 36 and 39. in each of these clusters, there is no communication between members of the same cluster. a discussion of the reasons why no internal tweeting occurs between users in each of these cases is discussed in section 5.2. in directed graphs, it is interesting to note vertices which are sources, that is, have only outgoing edges (and possibly loops) and sinks, that is, vertices that only have in-going edges (and possibly loops). the isolated vertex corresponding to cluster 38 is trivially both a source and sink. the only edge incident to this vertex is a loop. there are no other source vertices in the graph. hence, the underlying twitter network has no identifiable non-trivial source clusters, that is, no group of users who only tweet out to other clusters while not receiving inward tweets. there are two sink vertices in the graph. these vertices correspond to clusters 27 and 35. both these vertices have no loops. a discussion of the groups corresponding to these “sink” clusters is given in section 5.3. an analysis of the shortest distance between pairs of vertices in the graph is given in table 1. each row in the table gives the minimum distance between the vertex indexed by that row and all other vertices. rows with no green entries correlate with clusters that are closely connected to all clusters in the network. there are many vertices that are directly connected to each other via an edge. these are given a distance 1. in each of these cases the individual tweets directly to another individual. the longest shortest path is from cluster 39 to cluster 35: a directed path of length 29. australasian journal of information systems morgan, cheong & bedingfield 2018, vol 22, research on recent advances in social media power to the people 11 table 1. shortest distances between vertices (representing clusters) in directed graph. when no directed path connects two vertices, the cell is left blank. 4.3.3 graph of cliques a clique is a set of vertices where every pair of vertices in the set is connected by a bi-directed edge. we created a second graph which we refer to as a graph of cliques, by identifying the maximal cliques in the directed graph and representing each maximal clique by a vertex in this graph. two vertices are connected by an edge in the graph of cliques if they contain some clusters in common. the graph of cliques is undirected. we ignored loops as every vertex trivially contains clusters in common with itself. 4.3.4 results the graph of cliques had 44 vertices representing the 44 maximal cliques (excluding the 16 maximal cliques that were trivial, that is, a single vertex). the maximum possible number of edges in an undirected graph with 44 vertices that has no loops is ( ) = 946� �� . in the graph of cliques formed from this twitter network, all but 167 of these edges were present almost 83% of all possible edges. australasian journal of information systems morgan, cheong & bedingfield 2018, vol 22, research on recent advances in social media power to the people 12 seven of the cliques were connected by edges to all other cliques. the sets of clusters belonging to each of the seven cliques is given in table 2. cluster 14 is common to all of these sets of clusters. clusters 21 and 31 are each in all but two of the cliques in table 2 (albeit in different sets). we discuss characteristics of these clusters in section 5.4. {3, 7, 14, 19, 20, 25, 31} {3, 7, 13, 14, 20, 25, 31} {7, 14, 19, 21, 31} {14, 16, 21, 22} {14, 16, 21, 31} {14, 17} {14, 21, 29, 31} table 2. cliques (sets of clusters) adjacent to all other cliques in graph of cliques 4.3.5 addressing rq2 rq2. using graph-theoretic approaches, can we identify emergent communication patterns between clusters of twitter users? in this section, we identified groups of twitter users that were isolated from the rest of the network. we also identified groups that did not communicate outside their cluster but did communicate within their cluster (corresponding to sink vertices in the network). an analysis of the directed graph summarised in table 1 gives some idea of the minimum length of tweets that would be required for one cluster to communicate with another using existing communication pathways. the graph of cliques enabled us to identify a cluster of users that are central to our twitter network. in section 5, we examine the clusters identified and the characteristics and properties of these users in order to explain the nuances in communication patterns of these groups. 5 analysis of interesting clusters in order to profile the users related to clusters that we identified as interesting, we used the original metadata in our main corpus. for example, some tweets can be clearly identified as coming from a media source, some from a politician, and some appear to be from a member of the public. using the tableau bi suite, we filter out certain clusters of interest and observe the mostfrequent tweeters to those clusters, based on volume of tweets found in the corpus. we utilise metadata attached to these users to infer who they are. this can be achieved by studying their self-bio description, their full name and nickname, and twitter username, as gathered from our dataset. if more information is needed (e.g. due to the absence of meaningful metadata, or the vagueness of provided information), we explore that particular user’s twitter feed (including profile, posts, images, and links) on the twitter website itself. australasian journal of information systems morgan, cheong & bedingfield 2018, vol 22, research on recent advances in social media power to the people 13 in addition, if they are public figures, we can search their twitter profile and mentions of their username via the twitter search facility and google to look up more information about that public figure. 5.1 isolated clusters the isolated cluster, cluster 38 (figure 5) contains users who, based on their self-description, self-identify as active contributors to the community based on their occupation. from the most active users (by volume of tweets) in cluster 38, we find that they are in the health, legal, and educational professions. figure 5. tableau visualisation of the most active users in cluster 38 (the isolated cluster), including their user description on twitter as well as tweet volume. names of the users are coded to ensure anonymity. these active users exhibit traits of liberalism or progressiveness based on their self-description. a recent market research survey on australian twitter users indicate that australian twitter users tend to be progressive or liberal, inferred based on their political leanings (roy morgan research, 2016). our results suggest that this cluster has no influence on other clusters in our network. 5.2 clusters with no internal communication the vertices representing clusters 22, 26, 27, 35, 36 and 39 had no incident loops, that is, there were no tweets from each individual cluster to itself. users in these clusters are tweeting only to, and receiving tweets from, users in other clusters. the most frequent tweets in these clusters were from the nsw teachers federation (see figure 6). other active users in these clusters are also education-focused. australasian journal of information systems morgan, cheong & bedingfield 2018, vol 22, research on recent advances in social media power to the people 14 figure 6. tableau visualisation of the most active users in clusters 22, 26, 27, 35, 36, and 39 (clusters with no internal communication), including their user description on twitter as well as tweet volume. names of the users are coded to ensure anonymity. other personally-identifiable information in descriptions is redacted. this possibly indicates that this cluster identifies group(s) who are using twitter as a forum to discuss education rhetoric with a broader community. from these ‘no-loop’ clusters, we agglomerated the users found within these clusters and studied the descriptions of the top users in the agglomeration. the topmost users (by frequency) self-describe their vocations as teachers and educators. by analysing their description, these ‘no-loop’ cluster users tend to be progressive and self-describe as feminists and proud union members. however, based on our graph cluster analysis, their communication patterns are different compared to cluster 38 (section 5.1) -cluster 38’s communications are all internal, but this group of clusters are only external. in other words, cluster 38 keeps information for users within the cluster, while clusters 22, 26, 27, 35, 36 and 39 are for dispersing information to users outside their current cluster. these clusters with no internal communication can be said to exhibit ‘call-to-action’ behaviour – examples would be tweets calling for support for a particular party’s initiative or to promote awareness of a particular policy. 5.3 sink clusters vertices representing clusters 27 and 35 are sinks with no loops in the directed graph. a loopless sink vertex in the directed graph represents a cluster where all tweets are coming to users in the clusters, that is, there are no outgoing tweets and no tweets between individuals in the cluster. thus, these clusters identify users that are receivers only. clusters 27 and 35 are a subset of the clusters discussed in section 5.2. australasian journal of information systems morgan, cheong & bedingfield 2018, vol 22, research on recent advances in social media power to the people 15 in figure 7, we see the demographic breakdown is similar to figure 6 even when we restricted to these clusters. by agglomerating clusters 27 and 35 (being a subset of the ‘no-loop’ clusters), we also notice similar characteristics to the superset in section 5.2, where users are mainly teachers who are progressive. however, the differences between the frequency distributions in figures 6 and 7 exist only after the top 20th user (i.e. the user coded as “6dff8c” in figure 7). figure 7. tableau visualisation of the most active users in clusters 27 and 35, including their user description on twitter as well as tweet volume. names of the users are de-identified to ensure anonymity. other personally-identifiable information in descriptions is redacted. 5.4 important clusters in the graph of cliques the graph of cliques contained seven maximal cliques that were connected by edges to all other maximal cliques in the graph (see section 4.3.4). cluster 14 was in each of these seven cliques. this universal cluster could represent the most engaged users in our network. cluster 14 contains 288 users, with each user only contributing 6 tweets or less, which is far less than those found in the previous clusters discussed. it is difficult to conclusively describe features common to most of the top users in this cluster, as the descriptions contain an eclectic mix of terms and descriptors, tableau highlighted an interesting anomaly the top users (seen in figure 8) frequently change/edit their self-written descriptions (bio) on twitter, as illustrated by users with more than one “from user description” metadata item. it appears that our methods have identified an engaged, freethinking and genuine user group. australasian journal of information systems morgan, cheong & bedingfield 2018, vol 22, research on recent advances in social media power to the people 16 figure 8. tableau visualisation of the most active users in cluster 14, including their user description (bio) on twitter as well as tweet volume. names of the users are coded to ensure anonymity. cluster 21 and 31 were also identified as interesting in section 4.3.4. cluster 21 exhibits similar traits to cluster 14, such as an eclectic mix of user descriptions and frequent editing of their descriptions on twitter. however, an outlier of interest in cluster 21 is the user coded as “cd9123”, who tweeted a disproportionate number of tweets (150) nearly 5 times as many as the second most active tweeter in this cluster. all other tweeters tweeted a maximum of 20 times. australasian journal of information systems morgan, cheong & bedingfield 2018, vol 22, research on recent advances in social media power to the people 17 figure 9. tableau visualisation of the most active users in cluster 21, including their user description on twitter as well as tweet volume. names of the users are de-identified to ensure anonymity. other personally-identifiable information in descriptions are redacted. finally, we analyse the trends of users in cluster 31. again, similar to individual analyses of clusters 14 and 21, an eclectic mix of user descriptions and frequent changing of user description on twitter is found in the top user list. however, the distribution of tweets is similar to cluster 14, as the maximum number of tweets per user detected in this cluster is 13, while all others have 6 or less tweets. australasian journal of information systems morgan, cheong & bedingfield 2018, vol 22, research on recent advances in social media power to the people 18 figure 10. tableau visualisation of the most active users in cluster 31, including their user description on twitter as well as tweet volume. names of the users are de-identified to ensure anonymity. clusters 21 and 31 are in cliques that are almost as connected as those containing cluster 14. the similar behaviour of the nodes in the graph of cliques (see section 4.3.4) is mirrored in the eclectic behaviour of the users within the corresponding clusters. 6 conclusions and further work we have demonstrated that a methodology using a combination of clustering and graph theoretical techniques can provide significant insight into twitter user interactions in relation to topical issues. this methodology can be applied to any specific topics or combination of topics trending on social media. our analysis of cliques based on clustered features of tweet content, by definition, is unique compared to social network research utilising user ties alone (quintaine & kleinbaum, 2011; kwak, lee, park & moon, 2010). using this methodology, we have established that, in the context of our particular case study, cohorts of the general public that we could identify as being the ‘common man’, have very limited presence in the wider discourse. the formation of cliques indicate that the identified cohorts exhibit strong homophily – in other words, such "…networks are homogeneous with regard to many sociodemographic, behavioural, and intrapersonal characteristics" (mcpherson, 2001). homophily in this regard is also found in other online network settings (quintaine & kleinbaum, 2011). in this study, the strong homophily in certain groups (e.g. political and environmental) appears to have had the greatest influence over the network structure. we conclude that the ‘common man’ does not have a voice in this particular network. australasian journal of information systems morgan, cheong & bedingfield 2018, vol 22, research on recent advances in social media power to the people 19 there are some exceptions to this. for example, users identified as teachers mainly belonged to one of two clusters. the tweeters in one cluster mainly communicated to users within their own cluster regarding interest internal to their group. conversely, the tweeters in the second cluster predominantly tweeted to external users regarding topics of more widespread interest. 6.1 further work. our case study has only considered a single twitter network observed over a window in time. applying this approach to multiple contexts including other media and/or other issues of discourse would provide further information. possibly the ‘common man’ has a voice when the domain of discourse is not of interest to large cohorts of users. another direction of future research is to investigate the long-term influence of groups of users in the hazelwood discourse. an alternative approach to the analysis conducted in this research, would be to cluster based on the descriptions of the users in the sample. however, the descriptions may not be accurate or consistent as many users prefer anonymity. acknowledgements the authors would like to extend their gratitude to the anonymous referees, whose helpful and insightful comments have contributed to this paper. ethics approval: power to the people: a case study of the influence of twitter on energy issues. ethics approval: 8715 (granted by monash university, 23rd march 2017). references abbas, o.a. (2008) comparisons between data clustering algorithms. the international arab journal of information technology, 5(3), 320-325. aggarwal, c. c., & zhai, c. (2012). a survey of text clustering algorithms. in mining text data (pp. 77–128). aittokallio, t., & schwikowski, b. (2006). graph-based methods for analysing networks in cell biology. briefings in bioinformatics, 7(3), 243–255. anderson, s. (2017, march 30). everything you need to know about the hazelwood shutdown. retrieved september 1, 2017, from http://www.abc.net.au/news/2017-03-30/hazelwoodpower-plant-shutdown-explained/8379756 bang-jensen, j., & gutin, g. z. (2008). digraphs: theory, algorithms and applications. springer. barabási, a.-l. and r. albert, science 286, 509 (1999). bar-yam, y. (2011). concepts: power law | necsi. retrieved october 4, 2017, from http://www.necsi.edu/guide/concepts/powerlaw.html bastian, m., heymann, s., & jacomy, m. (2009). gephi: an open source software for exploring and manipulating networks. in international aaai conference on weblogs and social media. retrieved from http://www.aaai.org/ocs/index.php/icwsm/09/paper/view/154 batagelj, v., & mrvar, a. (2004). pajek — analysis and visualization of large networks. in mathematics and visualization (pp. 77–103). bearman, p. s., moody, j., & stovel, k. (2004). chains of affection: the structure of adolescent romantic and sexual networks. the american journal of sociology, 110(1), 44–91. australasian journal of information systems morgan, cheong & bedingfield 2018, vol 22, research on recent advances in social media power to the people 20 bernard, h., killworth, p. d. & sailer, l. (1980). informant accuracy in social network data iv: a comparison of clique-level structure in behavioral and cognitive network data. social networks, elsevier bv, 2, 191-218. bilgili, m., ozbek, a., sahin, b., & kahraman, a. (2015). an overview of renewable electric power capacity and progress in new technologies in the world. renewable and sustainable energy reviews, 49, 323–334. borra, e., & rieder, b. (2014). programmed method: developing a toolset for capturing and analyzing tweets. aslib journal of information management, 66(3), 262–278. bunn, a. g., urban, d. l., & keitt, t. h. (2000). landscape connectivity: a conservation application of graph theory. journal of environmental management, 59(4), 265–278. byrnes, l., brown, c., foster, j., & wagner, l. d. (2013). australian renewable energy policy: barriers and challenges. renewable energy, 60, 711–721. cheong, m. (2013). inferring social behavior and interaction on twitter by combining metadata about users & messages (phd). https://doi.org/10.4225/03/58b5009d3726a cormode, g., krishnamurthy, b., & willinger, w. (2010). a manifesto for modeling and measurement in social media. first monday, 15(9). https://doi.org/10.5210/fm.v15i9.3072 davies, p. a. (2014). green crime and victimization: tensions between social and environmental justice. theoretical criminology, 18(3), 300–316. diestel, r. (2005). graph theory. springer science & business media. dudley-nicholson, j. (2016, may 27). twitter to help decide australian federal election with most uncommitted. retrieved september 27, 2017, from http://www.news.com.au/technology/twitter-may-help-decide-who-rules-australiawith-millions-of-votes-up-for-grabs-in-the-online-election/newsstory/d2304b039651b86fe203f8dab83837bb evans, g., & phelan, l. (2016). transition to a post-carbon society: linking environmental justice and just transition discourses. energy policy, 99, 329–339. freeman, l. c. (1977). a set of measures of centrality based on betweenness. sociometry, 40(1), 35. furlan, c., & mortarino, c. (2017). forecasting the impact of renewable energies in competition with non-renewable sources. renewable and sustainable energy reviews. https://doi.org/10.1016/j.rser.2017.05.284 green, d. g. (2014). of ants and men. berlin, heidelberg: springer berlin heidelberg. holmgren, a. j. (2006). using graph models to analyze the vulnerability of electric power networks. risk analysis: an official publication of the society for risk analysis, 26(4), 955– 969. hull, c. (2017, april 27). turnbull’s foolish gamble to pander to big fossil over adani coal mine. retrieved september 1, 2017, from https://www.brisbanetimes.com.au/opinion/malcolm-turnbulls-foolish-gamble-topander-to-big-fossil-over-adani-coal-mine-20170427-gvtusc.html australasian journal of information systems morgan, cheong & bedingfield 2018, vol 22, research on recent advances in social media power to the people 21 jansen, b.j., sobel, k and cook, g. (2011). classifying ecommerce information sharing behaviour by youths on social networking sites. journal of information sciences, 37, 120136. johnstone, p., & hielscher, s. (2017). phasing out coal, sustaining coal communities? living with technological decline in sustainability pathways. the extractive industries and society. https://doi.org/10.1016/j.exis.2017.06.002 knott, m. (2017, may 12). “we have to face reality”: greens prepared to strike school funding deal with the coalition. retrieved september 1, 2017, from http://www.smh.com.au/federal-politics/political-news/we-have-to-face-reality-greensprepared-to-strike-school-funding-deal-with-the-coalition-20170512-gw3m31.html kolokotsa, d. (2017). smart cooling systems for the urban environment. using renewable technologies to face the urban climate change. solar energy. https://doi.org/10.1016/j.solener.2016.12.004 kwak, h., lee, c., park, h., & moon, s. (2010). what is twitter, a social network or a news media? proceedings of the 19th international conference on world wide web (www '10). http://dx.doi.org/10.1145/1772690.1772751 mason, o., & verwoerd, m. (2007). graph theory and networks in biology. iet systems biology, 1(2), 89–119. mcpherson, m., smith-lovin, l., & cook, j. m. (2001). birds of a feather: homophily in social networks. annual review of sociology, annual reviews, 27, 415-444. moreno, j. l. (1934). who shall survive?: a new approach to the problem of human interrelations. nervous and mental disease publishing co. puddy, r. (2015, december 21). south australia faces crisis over power availability, prices. retrieved september 1, 2017, from http://www.theaustralian.com.au/business/miningenergy/south-australia-faces-crisis-over-power-availability-prices/newsstory/16855143aae5e9eebcccfee2ea043673 quintane, e. & kleinbaum., a. (2011). matter over mind? e-mail data and the measurement of social networks. connections, 31, 22-46. roy morgan research. (2016, june 21). twitter: a bird in the hand for 5.4m australians. retrieved october 5, 2017, from http://www.roymorgan.com/findings/6859-twitteractive-users-australia-march-2016-201606210913 sen, r., & bhattacharyya, s. c. (2014). off-grid electricity generation with renewable energy technologies in india: an application of homer. renewable energy, 62, 388–398. shamma, d. a., kennedy, l., & churchill, e. f. (2009). tweet the debates. proceedings of the first sigmm workshop on social media wsm ’09. https://doi.org/10.1145/1631144.1631148 twidell, j., & weir, t. (2015). renewable energy resources. routledge. wasserman, s., & faust, k. (1994). social network analysis: methods and applications. cambridge university press. xu, r., & wunsch, d., 2nd. (2005). survey of clustering algorithms. ieee transactions on neural networks / a publication of the ieee neural networks council, 16(3), 645–678. australasian journal of information systems morgan, cheong & bedingfield 2018, vol 22, research on recent advances in social media power to the people 22 zhang, s., wang, r., and zhang, x. identification of overlapping community structure in complex networks using fuzzy c-means clustering. (2007). physica a: statistical mechanics and its applications, 374(1): 483-490. copyright: © 2018 morgan, cheong & bedingfield. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. paper title (paper title style) australasian journal of information systems le et al. 2023, vol 27, selected papers from acis 2021 factors affecting prosocial sharing 1 factors affecting prosocial sharing of health-related information on social media during a health crisis: a dual exchanging-protecting model long hoang le the school of business, rmit vietnam, ho chi minh city, vietnam long.le18@rmit.edu.vn ai-phuong hoang the school of business, rmit vietnam, ho chi minh city, vietnam hiep cong pham the school of business, rmit vietnam, ho chi minh city, vietnam abstract during a health crisis, prosocial sharing of health-related information (hri) on social media can help deliver early warnings about new diseases, raise social awareness, exchange support, and spread health policies. current literature has mainly focused on the factors of general sharing of hri under normal conditions but neglected those motivations under the health crisis context. this study aims to investigate factors that influence prosocial sharing of hri on social media during a health crisis. to obtain the objective, this study proposes a dual exchanging-protecting model derived from the fear appeal model and social exchange theory. a partial least squares analysis, carried out on surveyed data of 326 participants, suggested that online users performed two steps of threat appraisal and coping appraisal when they share hri on social media. specifically, both health and information risks were found to have impacts on prosocial sharing via motivational factors. additionally, the motivations of prosocial sharing include both protecting factors (i.e., sharing efficacy, response efficacy) and an exchanging factor (i.e., reciprocity expectation). our findings offer several theoretical implications and practical contributions for health communicators. keywords: prosocial sharing, health information, health problems, information quality problems, health crisis. 1 introduction during a health crisis, health-related (e.g., physical illness and mental health issues) and information-related problems (e.g., information overload, information uncertainty, and misinformation) have posed a serious concern to public and governmental agencies alike. for example, popular mental and psychological issues during covid-19 have been found, including fear, anxiety, and unsafety feelings under uncertain conditions (usher et al., 2020). psychological distress can emerge from misleading and overloaded information (torales et al., 2020), social distancing, unemployment, caregiving stress, and impacts of death and illness (sritharan & sritharan, 2020). the director-general of the world health organisation, tedros adhanom ghebreyesus recently emphasised that: “we are not just fighting an epidemic; we’re fighting an infodemic” (world health organisation, 2020). sharing health-related information (hri) on social media can either reduce or worsen the problems. on the one hand, sharing messages about a health promotion campaign on social media can help to enhance people's awareness about the disease, address the problems of low australasian journal of information systems le et al. 2023, vol 27, selected papers from acis 2021 factors affecting prosocial sharing 2 health literacy, or connect people with similar health concerns (berry et al., 2017; kye et al., 2019). recognising the advantage of social media in health promotion and advertising, over half of the public health departments in the united states adopted social media for their campaigns (thackeray et al., 2012). similarly, in china, healthcare knowledge is encouraged to be shared via social networks to enhance the health literacy of online users (zhao et al., 2020). another worthy note is that social media can change the way hri and healthcare advice is shared. traditionally, sharing healthcare advice has a one-way direction from health professionals to patients. the availability of social media helps online users to exchange hri and obtain healthcare advice not only from healthcare providers but also from other peer users – which enables “the future of mental healthcare”, so-called peer-to-peer healthcare (naslund et al., 2016; zhao et al., 2021). on the other hand, online sharing of hri can cause several negative problems for online users. specifically, sharing misinformation about health on social media might cause unfavourable effects on both individuals and society. for example, sharing inaccurate health treatments can lead to wrong decisions of parents and even to children's deaths, and financial losses for society (chen et al., 2018). social media can translate a health crisis into a severe social crisis when spreading extensive information publicly and rapidly across communities (kilgo et al., 2018). in a worsening situation, social media and online interactions can also fuel such information-sharing problems by transmitting misinformation and negative emotions (e.g., chen et al., 2018). this study focuses on prosocial sharing of hri on social media during a health crisis with the belief that this behaviour can help to reduce both health and information risks aroused from the crisis. as stated above, sharing misinformation, or even sharing information “without much thought” (marin, 2021) might even worsen a health crisis. meanwhile, promoting online prosocial sharing of hri, which refers to a voluntary activity that aims to help others rather than oneself and without reward anticipation (leng et al., 2020), can provide more qualified information about the disease and health of the community. a review of the literature on online sharing of hri revealed several knowledge gaps. first, current literature has not differentiated domains of sharing behaviour, for example, sharing without much thought (marin, 2021) from prosocial sharing (dunfield, 2014). while sharing without much thought might stimulate the problems of misinformation and information overload (marin, 2021), prosocial sharing can reduce the likelihood of sharing misinformation since online users need to consider it carefully before performing the sharing action. specifically, sharing hri on social media can be considered a coping strategy for psychological stress during a health crisis, such as during covid-19 (le et al., 2022). lacking behavioural differentiation might reduce the understanding of contributing factors to each behaviour. second, during a health crisis, both health-related problems and information quality problems become a serious concern, which typically causes fear among communities (usher et al., 2020). under such fear of pressure, many people are likely to share extensive information on social media. nevertheless, few studies have considered fear-aroused factors when examining the motivations for sharing hri. finally, there is a lack of a theoretical framework to explain hri-sharing behaviour during a health crisis. the popular theories used in the literature on hri sharing (e.g., social exchange theory, social support theory) and online prosocial sharing (e.g., altruism, cognitive development, affective-driven motivations) are not sufficient to justify sharing behaviour during critical events. the motivations and mechanisms underlying prosocial behaviour have australasian journal of information systems le et al. 2023, vol 27, selected papers from acis 2021 factors affecting prosocial sharing 3 been emphasised to be difficult to investigate (dunfield, 2014; eisenberg, 2014) because of its various dimensions, contextual dependence, and diverse theoretical justifications. consequently, heretofore, “although the motivation for performing prosocial actions is critical, it is often unknown” (eisenberg, 2014). this research aims to identify motivating factors of prosocial hri sharing on social media in a health crisis. specifically, this study sets to answer two research questions: (i) what factors motivate online users’ prosocial sharing of hri on social media during a health crisis?, and (ii) how do healthand information-related risks, and motivational factors affect prosocial sharing of hri on social media during a health crisis? the next section reviews the relevant literature about problems aroused by a health crisis and online prosocial sharing. after that, the research model and hypotheses are postulated. the method section presents the measures, sampling, and analysis techniques. in the subsequent section, data analysis and findings are reported. finally, we discuss the results and conclude the study. 2 literature review 2.1 healthand information-related problems during a health crisis a crisis is defined as “an unusual event of overwhelmingly negative significance, that carries a high level of risk, harm, and opportunity for further loss” (seeger et al., 2003, p. 4). gaspar et al. (2016, p. 180) provided a more specific definition of a crisis, which is the situation when “potentially stressful events associated with the emergence of health threats (e.g., epidemics, biological and chemical contamination of food), terrorist attacks, natural disasters (e.g., hurricanes, floods), industrial accidents (e.g., nuclear) or even events related with macroeconomic changes” take place. generally, different types of crises share several similar characteristics, including the low probability of occurrence, severe negative damages and loss, informational uncertainty, and community negative emotions. a health crisis, a specific type of social crisis, has several distinctions. health crises can be derived from natural sources such as a disease (kilgo et al., 2018), and/or social-related sources such as cultural and political factors (campos et al., 2006). health crises typically put high pressure on the healthcare system and stimulate fear in the community. in the social media decade, a health crisis can be translated into a severe crisis when it is spread publicly and rapidly (kilgo et al., 2018). public communication under a health crisis, accordingly, might deal with both healthand information-related problems. literature on health crises indicates that a health crisis can cause serious harm to both physical and mental health (bonanno et al., 2010). for example, shanbehzadeh et al. (2021) indicate that covid-19 patients have commonly suffered physical health problems of fatigue, pain, arthralgia, physical capacity diminishing, and negative impacts on daily activities. besides the physical issues, psychological issues were discovered, such as fear, anxiety, and unsafety feelings (usher et al., 2020). especially, psychological distress can be emerged from the disease relevance such as impacts of death and illness and caregiving stress (sritharan & sritharan, 2020), and misinformation (torales et al., 2020). in another example, during the ebola outbreak in 2014, many americans were reported to have psychological distress and perceived a high risk of the african pandemic to their health in the future (thompson et al., 2017). in chinese rural areas, the spread of several pandemic diseases such as hiv/aids and avian flu caused major issues in the mortality and death rates for over a decade (dummer & cook, 2007). australasian journal of information systems le et al. 2023, vol 27, selected papers from acis 2021 factors affecting prosocial sharing 4 besides the health threats, information problems also soar during a health crisis. in the early stage of a crisis, the lack of relevant information might cause information uncertainty, whereas, in the later stage of the crisis, people typically deal with information overload and poor information quality. information overload refers to a situation when the amount of information is large and surpasses the cognitive capacity of individuals (bawden & robinson, 2009). information quality refers to “the degree to which individuals believe that the health information obtained from the online environment is high quality” (liang et al., 2017, p. 436). the characteristics of information quality include accuracy, completeness, currency, and transparency (liang et al., 2017). in general, the quality of hri received from online sources was found to be poor quality, i.e., inadequate, incomplete, and source ambiguity (sudau et al., 2014). 2.2 social media usage during a health crisis social media usage in health crisis communication has recently received increased attention. a study by the oecd indicated that: “social media can enhance risk and crisis communication in several ways: (1) they are collaborative and participatory and thus can improve situation awareness, (2) they are decentralized, thus, information can circulate quickly, and (3) they are geographically traceable and thus allow for the monitoring of a crisis” (oecd, 2015, p. 78). oecd also classified five types of social media used in health communication during a health crisis, including (1) social network sites, (2) content-sharing platforms, (3) collaborating knowledge-sharing platforms, (4) blogs and micro-blogs, and (5) crisis management platforms (oecd, 2015, p. 78). these platforms can be used differently in managing crisis communication. for example, social network sites are used for sharing information across site communities, whereas, crisis management platforms are employed to map and update emergency responses. however, the actual use of social media in dealing with health communication problems remains unclear. theoretically, online discussions and social media engagement can be among several effective measures that mitigate the impacts of mental and psychological issues (sritharan & sritharan, 2020; torales et al., 2020). however, social media and online information sharing can turn a health crisis into a severe social crisis since the crisis can spread quickly without proper verification across communities (kilgo et al., 2018). albarracin and jung (2021) pointed out that studies on information dissemination during covid-19 need to be specific because different online contents can change the attitude and behaviour of online users accordingly. 2.3 prosocial sharing of health-related information on social media staub (1978, p. 89) referred to prosocial as “behaviour that benefits the self as well as others, such as cooperation, and positive acts that are part of a person's attempts to initiate a relationship with another person or are aspects of an ongoing relationship” and “demands some form of self-sacrifice from the actor”. another relevant term, altruism, was used for a more restrictive situation when people carry out activities with the apparent goal of benefiting others more than themselves (underwood & moore, 1982, p. 27). other studies also recognised prosocial behaviour as voluntary behaviour intended to benefit others (eisenberg, 2014; eisenberg et al., 2006), while altruism is considered a subtype of prosocial behaviour (eisenberg, 1982; eisenberg, 2014). australasian journal of information systems le et al. 2023, vol 27, selected papers from acis 2021 factors affecting prosocial sharing 5 online prosocial sharing of hri is an extension of face-to-face prosocial behaviour (wright & li, 2011). it is part of the three main domains of online prosocial behaviour: online donation, online sharing, and online comfort (leng et al., 2020; sproull, 2011). leng et al. (2020) pointed out that online prosocial behaviour differs from face-to-face prosocial behaviour in terms of lower cost, anonymity, and less social pressure. prosocial hri sharing might require further efforts of information verification before sharing. information verification, or validating information quality before sharing (flanagin & metzger, 2000) becomes crucial when people share information to help the community. even though understanding the underlying mechanisms of prosocial behaviour can offer profound implications (donald et al., 2019), we have yet to understand the factors motivating prosocial action. there are several possible reasons leading to this under-researched situation. the extensive types of prosocial sharing and diverse contexts can be one of the reasons that the motivations of prosocial sharing are difficult to investigate (eisenberg, 2014; eisenberg & spinrad, 2014). for example, spontaneously prosocial behaviours (e.g., prosocial sharing) are costly, whereas response prosocial behaviours such as cheering others are at low cost. prior studies suggest that prosocial behaviour is motivated by altruism rather than personal selfreward (eisenberg et al., 2006); however, in many situations, prosocial behaviour can be nonaltruism motivated but following norms, and/or because of expectation of self-reputation (donald et al., 2019). the development of prosocial behaviour can be an outcome of cognitive development or socialisation, affective motivations (e.g., empathy), and moral reasoning (eisenberg, 1982). additionally, the evaluation of motivations for prosocial behaviour is difficult because the development of prosocial behaviour can be influenced by either individual traits or universal aspects (dunfield, 2014; eisenberg & spinrad, 2014). we thus discover several research gaps through our literature review. first, as stated above, although the motivations for performing prosocial behaviour are crucial, there is still a lack of understanding of those motivations (eisenberg & spinrad, 2014). the various types of prosocial behaviour and contextual dependence on the behaviour have challenged researchers to put their effort into this topic. second, given healthand information-related risks emerge during health crises, how they can affect online users’ sharing behaviour is still an uncovered question. finally, online prosocial sharing of hri during a health crisis requires knowledge in several research areas such as prosocial behaviour, online information sharing, and health crisis communication. the use of a single theory (e.g., social exchange theory, altruism theory) is not sufficient to explain the mechanisms underlying the behaviour. to fill these gaps, this study develops a model by integrating social exchange theory (set) and the fear appeal model (fam) to justify the antecedent factors of prosocial sharing of hri on social media during a health crisis. the following sections present the our theoretical foundation of our research. 2.4 social exchange theory set has been used widely to understand the behaviour of information and knowledge exchange on virtual platforms (yan et al., 2016). originally, set stated that human behaviour can be considered as the outcome of a cost/benefit calculation, or maximising the benefits and minimising the costs (emerson, 1976). under the assumption of rationality, set can be applied in various contexts, from the market base to the social base (blau, 1964). additionally, the straightforward approach to the reciprocity process, e.g. provide help and expect to receive back, can be used widely to develop conceptual frameworks (cropanzano et al., 2017; australasian journal of information systems le et al. 2023, vol 27, selected papers from acis 2021 factors affecting prosocial sharing 6 cropanzano & mitchell, 2005). for example, extensive past studies adopted set in the research on online knowledge and information sharing in a normal routine (jin et al., 2010; lin & chang, 2018; yan & tan, 2014; yan et al., 2016). however, given its broad conceptual framework, set showed limitations when being applied to justify individuals’ behaviour during a critical situation (cook et al., 2013; cropanzano et al., 2017). first, as a broad framework, set did not specify the factors that affect the behaviour, and additionally, the subsequent explanations of behaviourd offered by set might be too general and imprecise (cropanzano et al., 2017). second, the rationality assumption of set suggested that people would make “conscious choices based upon self-interest deliberation before taking action” (cook et al., 2013, page 340). however, in the context of human interaction, the assumption might not be valid as people might not always perform prior calculations of returns before acting (cook et al., 2013). rationality would not hold when examining prosocial behaviour, which is a voluntary action motivated by the benefit for others. finally, given the great increase in information demand during a health crisis (dreisiebner et al., 2020), set might be not sufficient to explain the sharing behaviour in such a critical event. in other words, set in its original was not developed to explain sharing behaviour in crises. maslow's basic need theory specifies four levels of basic needs, including physiological, safety, love and belongingness, esteem, and self-actualisation (maslow, 1954). during a health crisis, the need for physiological, safety, and love and belonging become more salient than the top two higher levels of need (i.e., esteem and self-actualisation) (ryan et al., 2020; yuen et al., 2021). in this situation, prosocial sharing behaviour might be motivated by protecting factors and community benefit expectations. set is limited in addressing what motivational factors are incorporated to explain prosocial sharing during a health crisis and how they influence an individual’s behaviour. we thus included maslow’s complementary theory of basic needs in our application of set as the foundation for exploring the impacts of fear-aroused motivations on prosocial sharing. 2.5 the fear appeal model fear appeal models describes the mechanism and changes in people’s behaviour under a fear stimulus (witte, 1992). johnston and warkentin (2010) recently developed a fear appeal model (fam) in the information security and management context. mainly based on protection motivation theory, their fam suggests a sophisticated relationships among the cognitive processes of the users of information technologies, in particular in information management and security contexts. specifically, while protection motivation theory did not specify how threat appraisal and coping appraisal jointly influence the change in human behaviour (witte, 1992), the fam suggests a sequential impact amounts process, as “only if a threat is perceived to be relevant and potentially harmful will an appraisal of efficacy occur.” (johnston & warkentin, 2010, p. 552). in other words, the change in online and information technologies users’ behaviour is influenced by two sequential processes, threat appraisal and coping appraisal. threat appraisal includes perceived threat severity and perceived threat vulnerability (johnston & warkentin, 2010; rogers, 1983). witte (1992, p. 332) defined perceived severity as “an individual’s beliefs about the seriousness of the threat”, whereas perceived vulnerability, or perceived susceptibility, is “an individual’s beliefs about his or her chances of experiencing the threat”. the fam indicates that both threat severity and threat vulnerability might influence behavioural change but in distant relationships rather than direct relationships australasian journal of information systems le et al. 2023, vol 27, selected papers from acis 2021 factors affecting prosocial sharing 7 (johnston & warkentin, 2010). coping appraisal, or evaluation of the ability of adaptive response (i.e., sharing behaviour), includes the cognitive calculus between response efficacy and self-efficacy (floyd et al., 2000; rogers, 1983). response efficacy refers to “the belief that the adaptive response will work, that taking the protective action will be effective in protecting the self or others” (floyd et al., 2000, p. 411). self-efficacy refers to the belief of an individual to be capable or not performing the response (bandura, 1977; rogers, 1983). the fam suggests that response efficacy and self-efficacy have positive effects on adaptive behaviour, whereas response costs (i.e., information verification cost) negatively affect adaptive behaviour (rogers, 1983). bandura (1977) emphasised that adaptive behaviour can be enabled once people perceive its coping effectiveness and their ability to implement it. therefore, sharing efficacy and self-efficacy possibly have direct impacts on sharing activity. the next section presents the research model and hypotheses. 3 research model and hypotheses 3.1 the threat appraisal process: impacts of health risk and information quality risk online users might perceive threats in a crisis from two main sources: health risk (i.e., from a disease) and information risk. a health crisis typically has a low probability of occurrence but can cause severe outcomes and overload in the healthcare system, people's lives, and psychological stress. within the fear aroused from a health crisis, people can perceive health risk and information quality risk. this study, therefore, conceptualises two main perceived threats including perceived health risk and perceived information quality risk, which can jointly cause problems for the community. during a crisis, people perceive a higher risk for their and others’ health, especially in terms of severity and vulnerability. perceived severity relates to the perception of the degree of damage and loss caused by the health crisis, whereas perceived vulnerability involves the high probability of being infected. compared to a regular disease (e.g., seasonal flu), a health crisis is typically accompanied by a new, serious, and infectious disease, which causes perceived severity and perceived vulnerability to people’s health. information quality risk is another threat to people during a health crisis. information quality risk might include multiple dimensions, for example, (information) shortage, inaccuracy, incompleteness, irrelevance, and unreliability (nicolaou & mcknight, 2006). these problems can be rooted in newly discovered diseases (e.g., new variants, new symptoms, and new treatment methods) or spreading unverified information. numerous past studies investigated the occurrence of information quality problems during past health crises, for example, misinformation, information uncertainty, and information overload during the global zika outbreak (bode & vraga, 2018; kilgo et al., 2018), and the covid-19 pandemic (apuke & omar, 2021; cuan-baltazar et al., 2020; laato et al., 2020). user perception of information quality risk was found to have significant impacts on users’ mental health, attitudes toward health campaigns, and information-sharing behaviour. specifically, laato et al. (2020) investigated the positive effects of this perception on cyberchondria and unverified information-sharing behaviours. additionally, soroya et al. (2021) discovered that information overload and excessive information can cause information anxiety, and consequent information avoidance activities. featherstone and zhang (2020) discovered that perceived australasian journal of information systems le et al. 2023, vol 27, selected papers from acis 2021 factors affecting prosocial sharing 8 information quality risk can facilitate users’ negative emotions and attitudes towards vaccination. whatever types of problems they have, the more severe threats people perceive, the more they carry out responses (i.e., sharing of information that they perceive to be of good quality) to protect themselves and other people. when online users perceive higher risks from health and information, they will more likely expect that their information sharing will help to reduce the risks. the following hypotheses are put forward: h1a. online users who perceive higher health risk will perceive higher response efficacy. h2a. online users who perceive higher information quality risk will perceive higher response efficacy. regarding the impacts of risk perceptions on sharing self-efficacy, different types of risk might have different effects. the emergency of a health crisis can force people to update disease information and acquire knowledge to protect themselves and others (laato et al., 2020). social media provides an interactive environment, which not only helps online users to update new information but also to share personal experiences and health conditions (oecd, 2015). during a health crisis, when other face-to-face communication types might be limited, online users tend to use social media more frequently to connect with others, which also helps them to improve their social media sharing capability. however, the use of social media for information sharing might lead to information overload and quality issues because of a large volume of unstructured information pushed on social media in a short time. consequently, information quality risk might have a different impact on self-efficacy. the model of information behaviour by wilson (1999) suggested that the information quality risk can influence online users’ self-efficacy, or their conviction to successfully share information to help others. accordingly, the next two hypotheses, postulate: h1b. online users who perceive higher health risk to others will perceive higher sharing selfefficacy. h2b. online users who perceive higher information quality risk to others will perceive lower sharing self-efficacy. reciprocity, grounded in set, refers to “actions that are contingent on rewarding reactions from others and that cease when these expected reactions are not forthcoming” (blau, 1964, p. 6). another worthy argument is that when online users perceive high health risk and information quality risk, expectations of reciprocity will increase. yang, (2016) \ found that under high-risk conditions such as a health crisis, systematic processing becomes more salient compared to the heuristic system. consequently, when people perceive higher risks, they are willing to provide prosocial activities to help other people (yang, 2016). the risks under a health crisis are significantly greater than those during the normal routine, which means, online users might perceive a higher threat from health risk and information quality risk to both them and others. when they consider online sharing of hri over social media as a coping method to reduce the risks, they are likely to interact more frequently on social media to exchange information. in other words, online users share quality information with a higher expectation of receiving quality information in the future as reciprocity. these arguments can be hypothesised as follows: australasian journal of information systems le et al. 2023, vol 27, selected papers from acis 2021 factors affecting prosocial sharing 9 h1c. online users who perceive higher health risk to others will perceive higher reciprocity expectations. h2c. online users who perceive higher information quality risk to others will perceive higher reciprocity expectations. 3.2 exchanging motivations for online prosocial sharing online users typically assess benefits (i.e., reciprocity) and costs (i.e., information verification effort and operational costs) before the actual sharing activity (yan et al., 2016). sproull (2011) indicated that people are willing to help others once they receive support in the past, or anticipate support in the future. set explains that people share information online with an expectation of mutual reciprocity (chiu et al., 2006). reciprocity was identified as a driver of sharing information and knowledge online (singh et al., 2018; yan & tan, 2014). during the uncertain situation of a health crisis, people share hri over social media to inform and help others. moreover, they typically also expect to receive reciprocal information from others. therefore, it is hypothesised that the expected reciprocity will have a positive effect on sharing behaviour. h3. perceived reciprocity positively influences the likelihood of prosocial sharing hri on social media during a health crisis. the costs of prosocial sharing of hri can include operational cost (i.e., cost of the sharing process) and cognitive cost (i.e., verification cost) (yan et al., 2016). as stated above, online prosocial sharing can be done at low operational costs (leng et al., 2020). since social media performs explicit functions for users to share information conveniently (le et al., 2022), the sharing execution might be less costly. however, information verification effort, including time and energy spent on checking information accuracy (fallis, 2004), might be costly during a health crisis when online users have to process overwhelming information from extensive sources. the verification cost can differ across situational conditions and users’ capabilities (fallis, 2004). information verification is complex and uncertain, i.e., being at risk of accepting false information even after checking it (lazer et al., 2018). however, whether the verification is successful or not, the effort of information verification can imply the responsibility of the information senders, accompanied by their prosocial sharing goals. set indicates that online users analyse the benefits and costs before taking the action, or specifically, if the verification cost becomes higher, or even impossible in some situations, the likelihood of information sharing will decrease. hypothesis 4 is posited as follows: h4. perceived information verification costs negatively influence the likelihood of prosocial sharing hri on social media during a health crisis. 3.3 protecting motivations for online prosocial sharing since there are extensive ways to verify online information (flanagin & metzger, 2000), the verification cost highly depends on the verifying skill of online users. literature on information verification behaviour indicates that information verification behaviour is highly associated with personal information literacy (jones-jang et al., 2021), and perceived selfefficacy (khan & idris, 2019). in line with these past studies, this research argues that online users who perceive higher sharing self-efficacy are likely to have higher skills in checking information quality, and therefore, take less time and energy in the verification process. the hypothesis relates to the negative relationship between self-efficacy and verification cost is presented as follows: australasian journal of information systems le et al. 2023, vol 27, selected papers from acis 2021 factors affecting prosocial sharing 10 h5. online users who perceive higher sharing self-efficacy will have a lower cost of information verification. in this study, prosocial sharing of hri over social media is considered as the adaptive response to the fear aroused by a health crisis. the fam suggests a two-stage process (i.e., threat appraisal and coping appraisal) to explain why people implement an adaptive response when facing fear (johnston & warkentin, 2010; rogers, 1983; witte, 1992). in the second phase of coping appraisal, when people think that their behaviour can help to resolve the problem, they are willing to act. in this study, after the first stage of health risk and information quality risk evaluations, online users carry out the second stage of examining their online sharing action. in this stage, online users might make evaluations in three aspects: (i) to assess the effectiveness of their sharing action (i.e., prosocial sharing efficacy); (ii) to assess their ability to share information on social media (i.e., sharing self-efficacy); and (iii) to assess the costs of sharing (i.e., mainly involving the information verification cost). the possible negative impact of information verification costs has been rationalised and proposed in the above hypothesis h4. this section focuses on the role of prosocial sharing efficacy and sharing self-efficacy on prosocial sharing behaviour. following the explanation of the fam, when online users expect that their sharing action can help to reduce the risks aroused by a health crisis, they are likely to share hri on social media. regarding sharing self-efficacy, online users also consider their ability to share information, which significantly relies on four sources: mastery experiences, vicarious experience, persuasiveness, and emotion (bandura, 1977). in general, a social media platform can provide extensive functions for information sharing, for example, facebook has the ‘share’, ‘like’, ‘tag’, ‘hashtag’, and ‘copy/paste’ functions, which are appropriate for different community ranges. additionally, social media has been considered a public environment (barak et al., 2008), where people can share hri across platforms to different targets and even repeatedly look at their sharing experience (e.g., facebook’s ‘memories‘ function), indicating mastery experiences. online users can observe hri sharing actions from others on social media, indicating vicarious experiences. regarding persuasiveness and emotional arousal, social media users are encouraged by the platforms and others to share information. for example, facebook users who share hri on social media can receive comments and/or emoticon reactions such as ‘like’, ‘love’, and ‘care’. online users might consider these sources of prosocial sharing self-efficacy and their capability to share before taking action. accordingly, this study proposes that online users who perceive higher prosocial sharing self-efficacy are willing to perform a sharing action. in addition to the viewpoint of the fam, the moral behaviour manner can be considered since prosocial sharing is highly associated with altruism and morality (eisenberg, 2014; eisenberg et al., 2006; krebs, 1982). a study by marin (2021, p. 7) stated three conditions of moral responsibility: “a causal connection between the agent’s actions and an outcome, the agent’s knowledge of the consequences, and the agent’s freedom to act”. accordingly, a prosocial sharing action of hri can be implemented as a moral responsibility when online users perceive: (i) sharing effectiveness; (ii) their knowledge and value contribution by an information-verifying process; and (iii) sharing cost (i.e., mainly involving information verification cost). before sharing hri over social media, people evaluate the efficacy of the sharing and their capability of sharing and compare them with the costs of sharing. as argued above, sharing qualified information over social media can be one of the best measures to australasian journal of information systems le et al. 2023, vol 27, selected papers from acis 2021 factors affecting prosocial sharing 11 reduce information uncertainty and alleviate health illiteracy; therefore, if people are confident about the effectiveness of sharing information and their ability to perform sharing, the likelihood of sharing behaviour will increase. therefore, the related hypotheses are presented as follows: h6. online users who perceive higher response efficacy will have a higher likelihood of prosocial sharing of hri on social media during a health crisis. h7. online users who perceive higher sharing self-efficacy will have a higher likelihood of prosocial sharing of hri on social media during a health crisis. figure 1 presents the conceptual framework for this research. figure 1. dual-motivation model of sharing hri on social media during a health crisis 4 method this study follows the positivist paradigm and applies deductive reasoning. the conceptual framework and hypotheses are deducted from two theoretical bases (set and the fam). the hypothesis testing was carried out via an online survey instrument and quantitative data analysis. 4.1 measures we first reviewed the construct definitions and their measurements in the literature. to warrant the validity of the scale, the question items were adapted from previous studies, i.e., prosocial sharing from wright and li (2011) (e.g., “share good information related to health”, “share hri to help”, “share hri to cheer someone up”, “share hri to let someone know i care about them”), reciprocity expectation from singh et al. (2018), perceived severity, perceived vulnerability, response efficacy, sharing self-efficacy, and response costs from menard et al. (2017) and johnston and warkentin (2010), perceived information quality risk (i.e., measured in five dimensions of timeliness, accuracy, relevance, completeness, and reliability) from nicolaou and mcknight (2006). the measurements were adapted by adding the social media context and the context of a health crisis. a pilot survey was carried out to examine the face and content validity. most of the constructs were measured reflectively, whereas perceived australasian journal of information systems le et al. 2023, vol 27, selected papers from acis 2021 factors affecting prosocial sharing 12 health risk was formative with two components of perceived severity (i.e., magnitude) and perceived vulnerability (i.e., likelihood). all items were measured on the likert scale. specifically, for the sharing behaviour items, the likelihood scale was adopted (1 = “extremely unlikely” to 7 = “extremely likely”), whereas, for the motivation items, the agreement scale was adopted (1 = “strongly disagree” to 5 = “strongly agree”). concepts measurement items prosocial sharing of hri adapted from wright and li (2011): 1. during health crises, i share good information related to health on social media. 2. during health crises, i share health-related information on social media to help others. 3. during health crises, i share health-related information on social media to cheer someone up. 4. during health crises, i share health-related information on social media to let someone know i care about them. perceived health risk perceived severity (adapted from menard et al. (2017)): 1. it would be severe if i and people around me were infected. 2. it would be serious if i and people around me were infected. 3. it would be significant if i and people around me were infected. perceived vulnerability (menard et al. 2017; johnston and warkentin 2010; hodgkins and orbell 1998): 1. i and people around me are at risk of becoming infected with the pandemic disease. 2. it is likely that i and people around me will become infected with the pandemic disease. 3. it is possible that i and people around me will become infected with the pandemic disease. 4. my chances of being infected with the pandemic disease in the future are very low. (* reversed) perceived information quality risk adapted from nicolaou and mcknight (2006): 1. the information exchanged during health crises is currently not enough to meet my needs. (timeliness) 2. there are accuracy problems in the information exchanged during health crises (accuracy) 3. the information exchanged during health crises is not as what i need. (relevance) 4. the information exchanged during health crises is not complete. (completeness) 5. the information exchanged during health crises can hardly be relied upon. (reliability) prosocial sharing efficacy adapted from johnston and warkentin (2010) and hodgkins and orbell (1998): 1. sharing health-related information through social media works for protection. 2. sharing health-related information through social media is effective for protection. 3. when sharing health-related information through social media, people are more likely to be protected. 4. if i share health-related information through social media, i would ensure the early detection of any abnormalities. sharing selfefficacy adapted from johnston and warkentin (2010): 1. sharing health-related information through social media is easy. 2. sharing health-related information through social media is convenient. 3. i am able to share health-related information through social media without much effort. information verification cost adapted from menard et al. (2017): 1. verifying health-related information is time-consuming for me. 2. verifying health-related information is burdensome for me. 3. verifying health-related information is financially costly for me. 4. verifying health-related information would require too much from me. 5. verifying health-related information is not worth it. reciprocity expectation adapted from singh et al. (2018), bock et al. (2005): 1. sharing health-related information through social media would make me visible to the world. 2. sharing health-related information through social media would help me get acquainted with new people. 3. sharing health-related information through social media will help me meet my future needs. 4. if i share health-related information through social media, i can get back when i need it. table 1. construct measurements australasian journal of information systems le et al. 2023, vol 27, selected papers from acis 2021 factors affecting prosocial sharing 13 4.2 sample selection the online questionnaire was composed by qualtrics and delivered to respondents via amazon mechanical turk (m-turk), which has been previously validated as a platform where academics can employ respondents for their rigorous research (steelman et al., 2014). several criteria were settled: (i) the survey was to be carried out in english-native (as in officially recognised) speaking countries (i.e., u.s., u.k., australia, singapore, and hong kong), (ii) respondents must hold social media accounts, (iii) respondents must perceive the covid-19 occurrence in their place during the survey time. the sample size had to be at least 300 respondents to warrant the “good quality” for both factor analysis and structural equation modelling (hair et al., 2016). the respondents were paid a one-dollar compensation fee for their time. several filtering questions were also set to determine whether the respondents understood the context of a health crisis. in the pilot survey, 38 respondents were included to help refine the questionnaire. after the pilot survey, the content of several questions has been revised to be clearer and more specific. in the main survey, a total of 346 responses were received after two weeks, in the preriod from july 18th, 2021, to august 2nd, 2021. among them, 20 respondents failed the reversed questions and were eliminated from the data analysis. the final sample included 326 respondents. the data descriptions are presented in table 2. demographic variables frequency (n = 326) percentage (%) gender 1-male 175 53.7 2-female 151 46.3 age 18-25 40 12.3 26-33 103 31.6 33-40 59 18.1 over 40 124 38.0 education not graduated yet 48 14.7 bachelor’s degree 202 62.0 post-graduate 76 23.3 frequency of social media usage (hours/day) less than 2 hours 100 30.7 2 less than 4 hours 136 41.7 4 less than 6 hours 71 21.8 6 hours or more 19 5.8 table 2. data descriptions 4.3 data analysis technique for the measurement model, confirmatory factor analysis was adopted to assess the construct’s reliability and validity. for the path analysis, two types of structural equation models were considered, including covariance-based structural equation modelling (cb-sem) and partial least squares structural equation modelling (pls-sem) (hair et al., 2019). this study adopted the pls-sem because of three main reasons. first, while cb-sem focuses on theory testing and theory confirmation, pls-sem is appropriate for predictive purposes (i.e., maximising the r2 values) (hair et al., 2019). the goal of this study is to adopt two theories to develop a conceptual framework including risk perceptions and motivational factors that predict prosocial sharing behaviour. second, pls-sem can easily incorporate reflective and australasian journal of information systems le et al. 2023, vol 27, selected papers from acis 2021 factors affecting prosocial sharing 14 formative construct measurement models, whereas cb-sem requires construct specification modifications to handle formative constructs (hair et al., 2019). this study includes both reflective and formative constructs. specifically, perceived health risk is a formative construct, which includes two dimensions of health severity (i.e., the magnitude of risks) and health vulnerability (i.e., the likelihood of risks). finally, when cb-sem requires the normality assumption, the data used in pls-sem can be non-normally distributed (hair et al., 2019). finally, this study focuses on the behaviour of online users during critical events such as emergencies and crises, therefore, data might hardly satisfy the normality assumption. 5 research findings 5.1 measurement model the measurement model was evaluated via construct reliability and validity. the findings from table 3 showed that all cronbach’s alpha and composite reliability were larger than 0.7, and factor loadings were larger than 0.708, indicating good reliability and convergent validity (hair et al., 2019). to assess discriminant validity, we used fornell and larcker’s criterion (see table 4) and the heterotrait-monotrait ratio criterion (see table 5) (hair et al., 2019). all the square roots of aves were larger than the correlation than other inter-construct correlations. additionally, the htmt value is below 0.90 (except for the higher-order formative construct), and discriminant validity has been established (henseler et al., 2015). therefore the discriminant validity of the measurement was acceptable. no. of items cronbach’s alpha composite reliability ave prosocial sharing 4 0.932 0.952 0.831 reciprocity expectation 4 0.839 0.892 0.674 info. verification cost 4 0.874 0.900 0.646 response efficacy 4 0.849 0.899 0.689 sharing self-efficacy 3 0.731 0.848 0.650 disease severity 3 0.772 0.868 0.687 disease vulnerability 3 0.699 0.833 0.624 perceived info. qual. risk 3 0.775 0.863 0.678 table 3. construct reliability (1) (2) (3) (4) (5) (6) (7) (8) disease severity (1) 0.829 disease vulnerability (2) 0.484 0.790 info. verification cost (3) 0.031 0.248 0.804 perceived info. qual. risk (4) -0.017 0.146 0.566 0.823 prosocial sharing (5) 0.282 0.297 0.276 0.198 0.911 reciprocity expectation (6) 0.241 0.301 0.462 0.363 0.639 0.821 response efficacy (7) 0.231 0.305 0.279 0.124 0.674 0.647 0.830 sharing self-efficacy (8) 0.311 0.269 -0.154 -0.131 0.269 0.170 0.310 0.806 table 4. construct correlation note. bold numbers on the diagonal are the square root of the ave australasian journal of information systems le et al. 2023, vol 27, selected papers from acis 2021 factors affecting prosocial sharing 15 (1) (2) (3) (4) (5) (6) (7) (8) prosocial sharing (1) reciprocity expectation (2) 0.720 info. verification cost (3) 0.287 0.517 sharing self-efficacy (4) 0.321 0.225 0.173 response efficacy (5) 0.757 0.764 0.300 0.390 perceived info. qual. risk (6) 0.212 0.423 0.685 0.170 0.143 perceived health risk (7) 0.394 0.390 0.209 0.445 0.381 0.149 severity (8) 0.332 0.300 0.067 0.415 0.284 0.046 1.113* vulnerability (9) 0.372 0.400 0.318 0.376 0.400 0.229 1.150* 0.656 table 5. htmt for discriminant assessment note: * severity and vulnerability were the sub-construct of perceived health risk 5.2 structural model the structural model was evaluated via instructions of hair et al. (2019), the structural analysis included collinearity examination, r-squared value assessment for explanatory power, fsquared for the effect size evaluation, q-squared for predictive accuracy, and path coefficients for hypothesis testing. all the variance inflation factors (vif) were less than 5, representing no multicollinearity issue. the r-squared value equals 0.56, indicating that 56% of the variance of prosocial sharing can be explained by the research model. the blindfolding procedure from smartpls 3.0 performed that the q2 value of prosocial sharing was 0.460, indicating a medium predictive relevance of the pls-path model (hair et al., 2019). hypotheses f2 beta t-value conclusions h1a perceived health risk → response efficacy 0.102small 0.303*** 4.801 supported h1b perceived health risk → sharing self-efficacy 0.140small 0.349*** 5.360 supported h1c perceived health risk → reciprocity 0.107small 0.290*** 4.603 supported h2a perceived info. qual. risk → response efficacy 0.011 0.101* 2.012 supported h2b perceived info. qual. risk → sharing selfefficacy 0.028small -0.156** 3.058 supported h2c perceived info. qual. risk → reciprocity 0.147small 0.341*** 6.986 supported h3 reciprocity → prosocial sharing 0.113small 0.319*** 3.917 supported h4 info. verification cost → prosocial sharing 0.000 -0.016ns 0.345 not supported h5 sharing self-efficacy → info. verification cost 0.026small -0.158** 2.583 supported h6 sharing self-efficacy → prosocial sharing 0.019 0.102* 2.067 supported h7 response efficacy → prosocial sharing 0.195medium 0.405*** 5.415 supported control age → prosocial sharing 0.001 0.026ns 0.662 not sig. control education → prosocial sharing 0.044small 0.142** 3.039 *** control gender → prosocial sharing 0.004 0.044 ns 1.203 not sig. control social media freq. → prosocial sharing 0.018 0.099* 2.208 ** table 6. structural model results significance level: ***p < 0.001, **p < 0.01, *p < 0.05; effect size f2: small (>0.02), medium (>0.15), large (>0.35) table 6 provides the structural findings, including f2 analysis and path coefficient results. most of the hypotheses, except h2, are supported. specifically, both perceived health risk and perceived information quality risk significantly influence motivational factors, which include both social exchange motivation (i.e., reciprocity) and protection motivations (i.e., self-efficacy australasian journal of information systems le et al. 2023, vol 27, selected papers from acis 2021 factors affecting prosocial sharing 16 and response efficacy). additionally, these motivational factors were found to have significant impacts on prosocial sharing. regarding the control variables, education and social media frequency were found to have significant impacts on prosocial sharing of hri, whereas other demographic variables such as age and gender did not influence the prosocial sharing behaviour. figure 2 summarises the sturctural equation findings. figure 2. structural equation findings 6 discussion this study aimed to identify factors affecting the prosocial sharing of hri during a health crisis by integrating the set and the fam. while past studies have focused on either health risk (e.g., kar et al., 2021; sritharan & sritharan, 2020) or information risk (e.g., tully et al., 2020), our findings highlight that both healthand information-related risks have positive impacts on prosocial sharing via motivational factors. the measurement findings indicate a higher health risk perception across online users in both formative dimensions (meanseverity = 4.035, medianseverity = 4, meanvulnerability = 3.678, medianvulnerability = 4, as measured by the 5-point scale). our findings are in correspondence with past studies that also investigated increased human perceptions of crisis health risk, for example, the covid-19 risk perception score ranged from 4.78 to 5.45 (over 7) across surveyed participants in ten countries in dryhurst et al.'s (2020) study. additionally, the factor analysis revealed that only three dimensions of timeliness, relevance, and reliability were reliable and valid to reflect perceived information quality risk. in other words, the survey participants evaluated that much information in a health crisis was typically outdated, irrelevant, and not reliable. regarding the motivational factors, during a health crisis, we found that online users behave both rationally and cognitively. specifically, online users were likely to share information on social media with the expectation of getting reciprocal information and support from others. the sharing behaviour was also affected by fear-aroused factors, which means online users appeared to evaluate the crisis risks, followed by appraising the effectiveness of online australasian journal of information systems le et al. 2023, vol 27, selected papers from acis 2021 factors affecting prosocial sharing 17 prosocial sharing activities before taking the sharing action. the insignificance of the impact of information verification cost on sharing behaviour can be explained in two ways. first, the information verification might have been low-cost, so online users did not consider it. second, online users might not care to verify online information before consuming it. this would be in line with past studies that found that online users shared online information on social media without verifying effort during a health crisis (see e.g. laato et al., 2020). 6.1 theoretical contributions this study makes several theoretical contributions. the first contribution is the development of a conceptual framework to explain factors influencing online prosocial sharing of hri during a health crisis. the research model was developed from two theoretical bases, set and the fam. the literature review indicated that although theories of prosocial sharing of hri on social media theories are commonly applicable and effective in justifying the online sharing of hri during the normal routine (le et al., 2022), they are not sufficient to justify this behaviour during a health crisis. information behaviour during a health crisis becomes greatly different compared to it under the normal routine (dreisiebner et al., 2020; soroya et al., 2021); consequently, additional factors need to be considered when explaining the higher frequency of sharing behaviour. this study advances the literature by incorporating the fear-aroused factor of the fam into a research model (li, 2021) based on set. in this way, this study also extends the fam by incorporating social exchange factors and applying the integrated model to explain prosocial communication during a health crisis. the research model furthermore offers contributions to the literature on health crisis communication. in particular, the findings provide theoretical implications along with the two types of risk perceptions (i.e., health risk and information quality risk), two types of motivational factors (social exchange factors and coping appraisal factors), and the process of prosocial sharing of hri. the findings of this study will also respond to the call for research on an investigation of the psychological influences that affect users’ information behaviour on social media during a health crisis (mheidly & fares, 2020). another theoretical contribution of significance is the differentiation between online sharing domains. literature on sharing hri on social media did not distinguish prosocial sharing from other sharing domains and, consequently, did not differentiate the perceptions and motivations behind hri sharing. the extant literature focuses on examining factors of sharing misinformation (alvarez-galvez et al., 2021; chen et al., 2018; khan & idris, 2019; wang et al., 2019), and sharing information “without much thought” (marin, 2021), and scholars have put little effort into identifying the antecedents of online prosocial sharing. the findings of this research can pave the way for further research in the future since each type of sharing domain accompanies different motivations and derives different consequences. with this differentiation, this study contributes to the literature on prosocial behaviour. it also addresses a research call by eisenberg (2014) by discovering motivations and underlying mechanisms of a dimension of prosocial behaviour (i.e., online prosocial sharing) in a specific context (i.e., a health crisis). specifically, this study proposes that under the negative impacts of a health crisis, people perform prosocial sharing actions with dual motivations, including information exchange and health protection. 6.2 practical implications the practical implications are also profound. first, this study confirms the common use of social media in sharing hri during a health crisis. given the wide spreadability of social media australasian journal of information systems le et al. 2023, vol 27, selected papers from acis 2021 factors affecting prosocial sharing 18 information, social media platforms enhance their use in sharing hri during a health crisis. specifically, the theoretical model validated in this study suggests that both healthand information risks can urge online users to share more information on social media. additionally, the main motivations for sharing health information on social media include expectations of returning information and community protection. from these findings, social media companies can put further efforts into the feature of sharing hri during health crises. for example, current pro-community features “crisis response” and “emotional health” on facebook can be suggested directly to online users during health crises. furthermore, facebook can expand these features with further functions such as providing questions about information quality and sharing objectives, which might help to reduce the efforts of sharing non-validated information. second, although information problems during a health crisis have been emphasised recently by the world health organisation (world health organisation, 2020), this study is among the early studies providing scientific evidence that both health and information problems can jointly affect the evaluations and behaviours of online users. the findings of this research suggest that policymakers should not only focus on health problems but also include instruments to update and clarify information during their campaign to fight against a health crisis. finally, the identification of motivational factors can provide suggestions for health communicators to promote online users’ prosocial sharing of hri on social media during a health crisis, which helps to enhance disease awareness and reduce health risks. regarding information problems during a health crisis, information problems might vary throughout a health crisis, for example, information shortage in the early stage, but information overload in the later stage. understanding the sharing motivation can help to provide appropriate incentive/restrictive policies to harmonise the information volume across the crisis stages, and reduce the problems of misinformation and information overload (oecd, 2015). for example, to promote a health message (e.g., wearing masks during the covid-19, drinking more water), health communicators can spread the message by emphasising the expected outcomes (i.e., sharing for receiving further information as exchange, sharing for protecting the community) and the efficacy of sharing (e.g., “sharing this message, a little activity but huge community support”). 6.3 limitations and future research this study has several limitations. first, using the survey instrument, this study only captured the impacts of health and information risks and motivational factors on sharing behaviour at one particular point in time. since health crises typically develop over a time period and users’ behaviour changes continually during a health crisis, future process research should track the sharing behaviour of online users for a longer time to better understand both the users’ motivations and the changes. second, online behaviour might differ across personal traits and cultural values (e.g., le & duong, 2020). future research should extend this argument by investigating the differences in online sharing engagement of online users from different cultural (including language) backgrounds. finally, this study’s findings indicated the significant impacts of education and social media use frequency on prosocial sharing. future research should explore further these impacts, especially the roles of health literacy and social media literacy (jones-jang et al., 2021). australasian journal of information systems le et al. 2023, vol 27, selected papers from acis 2021 factors affecting prosocial sharing 19 7 conclusion social media can assist people in effective communication during a health crisis. however, the use of social media in sharing information might inflame the crisis problems when spreading misinformation, rumours, and fake news. this research argued that only prosocial sharing, or responsible sharing for the community’s benefit, can help reduce the crisis’ problems, especially in terms of health risk and information quality risk. this research introduced a theoretical framework for understanding the process underlying prosocial sharing of hri on social media during a health crisis. specifically, the dual exchanging-protecting model, developed from integrating set and the fam fear appeal, investigated the relationships between perceived risks (i.e., perceived health risk and perceived information quality risk), motivational factors and online users’ prosocial sharing of hri on social media. to validate the model, this research employed confirmatory factor analysis and pls-sem on the surveyed data of 326 participants. the findings revealed that 56.7% of prosocial sharing can be explained by protecting motivations (i.e., prosocial sharing efficacy and sharing self-efficacy), exchanging motivations (i.e., reciprocity expectation), and risk perceptions. the model developed and validated in this research serves as an open framework for future research to explore further factors affecting the online prosocial sharing of hri. the findings of this study provide scientific evidence and advice for health communicators and social media companies to increase the effective use of social media in health campaigns. acknowledgement the rmit university’s business and law college human ethics advisory network (bl chean) granted ethics approval for this research to be conducted on 7th december 2020. references albarracin, d., & jung, h. (2021). a research agenda for the post-covid-19 world: theory and research in social psychology. asian journal of social psychology, 24(1), 10. doi:10.1111/ajsp.12469 alvarez-galvez, j., suarez-lledo, v., & rojas-garcia, a. (2021). determinants of infodemics during disease outbreaks: a systematic review. frontiers in public health, 9(236). doi:10.3389/fpubh.2021.603603 apuke, o. d., & omar, b. (2021). fake news and covid-19: modelling the predictors of fake news sharing among social media users. telematics and informatics, 56, 101475. bandura, a. (1977). self-efficacy: toward a unifying theory of behavioral change. psychological review, 84(2), 191-215. doi:10.1037/0033-295x.84.2.191 barak, a., boniel-nissim, m., & suler, j. (2008). fostering empowerment in online support groups. computers in human behavior, 24(5), 1867-1883. bawden, d., & robinson, l. (2009). the dark side of information: overload, anxiety and other paradoxes and pathologies. journal of information science, 35(2), 180-191. doi:10.1177/0165551508095781 berry, n., lobban, f., belousov, m., emsley, r., nenadic, g., & bucci, s. (2017). # whywetweetmh: understanding why people use twitter to discuss mental health problems. journal of medical internet research, 19(4), e107. doi:10.2196/jmir.6173 australasian journal of information systems le et al. 2023, vol 27, selected papers from acis 2021 factors affecting prosocial sharing 20 blau, p. m. (1964). exchange and power in social life. new brunswick, nj, usa: transaction publishers. bode, l., & vraga, e. k. (2018). see something, say something: correction of global health misinformation on social media. health communication, 33(9), 1131-1140. bonanno, g. a., brewin, c. r., kaniasty, k., & greca, a. m. l. (2010). weighing the costs of disaster: consequences, risks, and resilience in individuals, families, and communities. psychological science in the public interest, 11(1), 1-49. doi:10.1177/1529100610387086 campos, p., saguy, a., ernsberger, p., oliver, e., & gaesser, g. (2006). the epidemiology of overweight and obesity: public health crisis or moral panic? international journal of epidemiology, 35(1), 55-60. doi:10.1093/ije/dyi254 chen, l., wang, x., & peng, t.-q. (2018). nature and diffusion of gynecologic cancer–related misinformation on social media: analysis of tweets. journal of medical internet research, 20(10), e11515. doi:10.2196/11515 chiu, c.-m., hsu, m.-h., & wang, e. t. g. (2006). understanding knowledge sharing in virtual communities: an integration of social capital and social cognitive theories. decision support systems, 42(3), 1872-1888. doi:10.1016/j.dss.2006.04.001 cook, k. s., cheshire, c., rice, e. r., & nakagawa, s. (2013). social exchange theory. in j. delamater, & a. ward (eds), handbook of social psychology (pp. 61-88). dordrecht, the netherlands: springer publisher. https://doi.org/10.1007/978-94-007-6772-0_3 cropanzano, r., anthony, e. l., daniels, s. r., & hall, a. v. (2017). social exchange theory: a critical review with theoretical remedies. academy of management annals, 11(1), 479516. doi:10.5465/annals.2015.0099 cropanzano, r., & mitchell, m. s. (2005). social exchange theory: an interdisciplinary review. journal of management, 31(6), 874-900. doi:10.1177/0149206305279602 cuan-baltazar, j. y., muñoz-perez, m. j., robledo-vega, c., pérez-zepeda, m. f., & soto-vega, e. (2020). misinformation of covid-19 on the internet: infodemiology study. jmir public health and surveillance, 6(2), e18444. donald, j. n., sahdra, b. k., van zanden, b., duineveld, j. j., atkins, p. w., marshall, s. l., & ciarrochi, j. (2019). does your mindfulness benefit others? a systematic review and meta-analysis of the link between mindfulness and prosocial behaviour. british journal of psychology, 110(1), 101-125. doi:10.1111/bjop.12338 dreisiebner, s., märz, s., & mandl, t. (2020). information behavior during the covid-19 crisis in german-speaking countries. arxiv preprint arxiv:2007.13833. doi:10.1108/jd-12-20200217 dryhurst, s., schneider, c. r., kerr, j., freeman, a. l., recchia, g., van der bles, a. m., . . . van der linden, s. (2020). risk perceptions of covid-19 around the world. journal of risk research, 23(7-8), 994-1006. dummer, t. j., & cook, i. g. (2007). exploring china's rural health crisis: processes and policy implications. health policy, 83(1), 1-16. doi:10.1016/j.healthpol.2006.12.002 australasian journal of information systems le et al. 2023, vol 27, selected papers from acis 2021 factors affecting prosocial sharing 21 dunfield, k. a. (2014). a construct divided: prosocial behavior as helping, sharing, and comforting subtypes. frontiers in psychology, 5, 958. doi:10.3389/fpsyg.2014.00958 eisenberg, n. (1982). the development of prosocial behavior.(first edition. new york, ny usa: academic press. eisenberg, n. (2014). altruistic emotion, cognition, and behavior (ple: emotion). london, uk: psychology press. eisenberg, n., fabes, r., & sprinrad, t. (2006). prosocial behavior. in n. eisenberg, d. william, & r. m. lerner (eds.), handbook of child psychology, social, emotional, and personality development (vol. 3, pp. 646-718). hoboken, nj, usa: john wiley & sons. eisenberg, n., & spinrad, t. l. (2014). multidimensionality of prosocial behavior: rethinking the conceptualization and development of prosocial behavior. in l. m. padilla-walker & g. carlo (eds.), prosocial development: a multidimensional approach (pp. 17-39). new york, ny, usa: oxford university press. emerson, r. m. (1976). social exchange theory. annual review of sociology, 2(1), 335-362. doi:10.1146/annurev.so.02.080176.002003 fallis, d. (2004). on verifying the accuracy of information: philosophical perspectives. library trends, 52(3), 463-487. featherstone, j. d., & zhang, j. (2020). feeling angry: the effects of vaccine misinformation and refutational messages on negative emotions and vaccination attitude. journal of health communication, 25(9), 692-702. flanagin, a. j., & metzger, m. j. (2000). perceptions of internet information credibility. journalism & mass communication quarterly, 77(3), 515-540. doi:10.1177/107769900007700304 floyd, d. l., prentice-dunn, s., & rogers, r. w. (2000). a meta-analysis of research on protection motivation theory. journal of applied social psychology, 30(2), 407-429. doi:10.1111/j.1559-1816.2000.tb02323.x gaspar, r., pedro, c., panagiotopoulos, p., & seibt, b. (2016). beyond positive or negative: qualitative sentiment analysis of social media reactions to unexpected stressful events. computers in human behavior, 56, 179-191. doi:10.1016/j.chb.2015.11.040 hair, j. f., risher, j. j., sarstedt, m., & ringle, c. m. (2019). when to use and how to report the results of pls-sem. european business review. doi:10.1108/ebr-11-2018-0203 hair, j. j. f., hult, g. t. m., ringle, c., & sarstedt, m. (2016). a primer on partial least squares structural equation modeling (pls-sem). thousand oaks, ca, usa: sage publications. henseler, j., ringle, c. m., & sarstedt, m. (2015). a new criterion for assessing discriminant validity in variance-based structural equation modeling. journal of the academy of marketing science, 43(1), 115-135. doi:10.1007/s11747-014-0403-8 jin, b., park, j. y., & kim, h.-s. j. b. (2010). what makes online community members commit? a social exchange perspective. behaviour & information technology, 29(6), 587-599. doi:10.1080/0144929x.2010.497563 johnston, a. c., & warkentin, m. (2010). fear appeals and information security behaviors: an empirical study. mis quarterly, 34(2), 549-566. doi:10.2307/25750691 australasian journal of information systems le et al. 2023, vol 27, selected papers from acis 2021 factors affecting prosocial sharing 22 jones-jang, s. m., mortensen, t., & liu, j. (2021). does media literacy help identification of fake news? information literacy helps, but other literacies don’t. american behavioral scientist, 65(2), 371-388. doi:10.1177/0002764219869406 kar, n., kar, b., & kar, s. (2021). stress and coping during covid-19 pandemic: result of an online survey. psychiatry research, 295, 113598. doi:10.1016/j.psychres.2020.113598 khan, m. l., & idris, i. k. (2019). recognise misinformation and verify before sharing: a reasoned action and information literacy perspective. behaviour & information technology, 38(12), 1194-1212. doi:10.1080/0144929x.2019.1578828 kilgo, d. k., yoo, j., & johnson, t. j. (2018). spreading ebola panic: newspaper and social media coverage of the 2014 ebola health crisis. health communication, 34(8), 811-817. doi:10.1080/10410236.2018.1437524 krebs, d. (1982). altruismm — a rational approach. in n. eisenberg (ed.), the development of prosocial behavior (pp. 53-76). new york, ny, usa: academic press. kye, s., shim, m., kim, y., & park, k. (2019). sharing health information online in south korea: motives, topics, and antecedents. health promotion international, 34(2), 182-192. doi:10.1093/heapro/dax074 laato, s., islam, a. n., islam, m. n., & whelan, e. (2020). what drives unverified information sharing and cyberchondria during the covid-19 pandemic? european journal of information systems, 29(3), 288-305. doi:10.1080/0960085x.2020.1770632 lazer, d. m., baum, m. a., benkler, y., berinsky, a. j., greenhill, k. m., menczer, f., . . . rothschild, d. (2018). the science of fake news. science, 359(6380), 1094-1096. doi:10.1126/science.aao2998 le, l. h., & duong, g. h. (2020). engagement in the online brand community: impacts of cultural traits. journal of international consumer marketing, 32(2), 146-158. doi:10.1080/08961530.2019.1649992 le, l. h., hoang, p. a., & pham, h. c. (2022). sharing health information across online platforms: a systematic review. health communication, 38(8), 1550-1562. doi:10.1080/10410236.2021.2019920 leng, j., guo, q., ma, b., zhang, s., & sun, p. (2020). bridging personality and online prosocial behavior: the roles of empathy, moral identity, and social self-efficacy. frontiers in psychology, 11. doi:10.3389/fpsyg.2020.575053 li, r. (2021). fear of covid-19: what causes fear and how individuals cope with it. health communication, 1-10. doi:10.1080/10410236.2021.1901423 liang, h., xue, y., & zhang, z. (2017). understanding online health information use: the case of people with physical disabilities. journal of the association for information systems, 18(6), 433. doi:10.17705/1jais.00461 lin, h.-c., & chang, c.-m. (2018). what motivates health information exchange in social media? the roles of the social cognitive theory and perceived interactivity. information & management, 55(6), 771-780. doi:10.1016/j.im.2018.03.006 australasian journal of information systems le et al. 2023, vol 27, selected papers from acis 2021 factors affecting prosocial sharing 23 marin, l. (2021). sharing (mis) information on social networking sites. an exploration of the norms for distributing content authored by others. ethics and information technology, 1-10. doi:10.1007/s10676-021-09578-y maslow, a. h. (1954). the instinctoid nature of basic needs. journal of personality, 22, 326-347. doi:10.1111/j.1467-6494.1954.tb01136.x menard, p., bott, g. j., & crossler, r. e. (2017). user motivations in protecting information security: protection motivation theory versus self-determination theory. journal of management information systems, 34(4), 1203-1230. doi:10.1080/07421222.2017.1394083 mheidly, n., & fares, j. (2020). leveraging media and health communication strategies to overcome the covid-19 infodemic. journal of public health policy, 41, 410-420. doi:10.1057/s41271-020-00247-w naslund, j., aschbrenner, k., marsch, l., & bartels, s. (2016). the future of mental health care: peer-to-peer support and social media. epidemiology and psychiatric sciences, 25(2), 113-122. doi:10.1017/s2045796015001067 nicolaou, a. i., & mcknight, d. h. (2006). perceived information quality in data exchanges: effects on risk, trust, and intention to use. information systems research, 17(4), 332-351. doi:10.1287/isre.1060.0103 oecd. (2015). the changing face of strategic crisis management. retrieved from paris: https://www.oecd-ilibrary.org/content/publication/9789264249127-en rogers, r. w. (1983). cognitive and psychological processes in fear appeals and attitude change: a revised theory of protection motivation. new york, ny, usa: guilford press. ryan, b. j., coppola, d., canyon, d. v., brickhouse, m., & swienton, r. (2020). covid-19 community stabilization and sustainability framework: an integration of the maslow hierarchy of needs and social determinants of health. disaster medicine and public health preparedness, 14(5), 623-629. doi:10.1017/dmp.2020.109 seeger, m. w., sellnow, t. l., & ulmer, r. r. (2003). communication and organizational crisis. westport,. ct, usa: greenwood publishing group. shanbehzadeh, s., tavahomi, m., zanjari, n., ebrahimi-takamjani, i., & amiri-arimi, s. (2021). physical and mental health complications post-covid-19: scoping review. journal of psychosomatic research, 110525. doi:10.1016/j.jpsychores.2021.110525 singh, j. b., chandwani, r., & kumar, m. (2018). factors affecting web 2.0 adoption: exploring the knowledge sharing and knowledge seeking aspects in health care professionals. journal of knowledge management, 22(1), 21-43. doi:10.1108/jkm-08-20160320 soroya, s. h., farooq, a., mahmood, k., isoaho, j., & zara, s.-e. (2021). from information seeking to information avoidance: understanding the health information behavior during a global health crisis. information processing & management, 58(2), 102440. doi:10.1016/j.ipm.2020.102440 sproull, l. (2011). prosocial behavior on the net. daedalus, 140-153. doi:10.1162/daed_a_00120 australasian journal of information systems le et al. 2023, vol 27, selected papers from acis 2021 factors affecting prosocial sharing 24 sritharan, j., & sritharan, a. (2020). emerging mental health issues from the novel coronavirus (covid-19) pandemic. journal of health and medical sciences, 3(2), 157-162. doi:10.4103/psychiatry.indianjpsychiatry_372_20 staub, e. (1978). predicting prosocial behavior: a model for specifying the nature of personality-situation interaction. in l. a. pervin, & m. lewis (eds), perspectives in interactional psychology (pp. 87-110). boston, ma, usa: springer publisher. https://doi.org/10.1007/978-1-4613-3997-7_5 steelman, z. r., hammer, b. i., & limayem, m. (2014). data collection in the digital age: innovative alternatives to student samples. mis quarterly, 38(2), 355-378. doi:10.25300/misq/2014/38.2.02 sudau, f., friede, t., grabowski, j., koschack, j., makedonski, p., & himmel, w. (2014). sources of information and behavioral patterns in online health forums: observational study. journal of medical internet research, 16(1), e10. doi:10.2196/jmir.2875 thackeray, r., neiger, b. l., smith, a. k., & van wagenen, s. b. (2012). adoption and use of social media among public health departments. bmc public health, 12(1), 1-6. doi:10.1186/1471-2458-12-242 thompson, r. r., garfin, d. r., holman, e. a., & silver, r. c. (2017). distress, worry, and functioning following a global health crisis: a national study of americans’ responses to ebola. clinical psychological science, 5(3), 513-521. doi:10.1177/2167702617692030 torales, j., o’higgins, m., castaldelli-maia, j. m., & ventriglio, a. (2020). the outbreak of covid-19 coronavirus and its impact on global mental health. international journal of social psychiatry, 66(4), 317-320. doi:10.1177/0020764020915212 tully, m., bode, l., & vraga, e. k. (2020). mobilizing users: does exposure to misinformation and its correction affect users’ responses to a health misinformation post? social media+ society, 6(4), 2056305120978377. doi:10.1177/2056305120978377 underwood, b., & moore, b. (1982). perspective-taking and altruism. psychological bulletin, 91(1), 143-173. doi:10.1037/0033-2909.91.1.143 usher, k., durkin, j., & bhullar, n. (2020). the covid-19 pandemic and mental health impacts. international journal of mental health nursing, 29(3), 315. doi:10.1111/inm.12726 wang, y., mckee, m., torbica, a., & stuckler, d. (2019). systematic literature review on the spread of health-related misinformation on social media. social science & medicine, 240, 112552. doi:10.1016/j.socscimed.2019.112552 wilson, t. d. (1999). models in information behaviour research. journal of documentation, 55(3), 249-270. doi:10.1108/eum0000000007145 witte, k. (1992). putting the fear back into fear appeals: the extended parallel process model. communications monographs, 59(4), 329-349. doi:10.1080/03637759209376276 world health organisation (2020, 15 february 2020). re: munich security conference. message posted to https://www.who.int/director-general/speeches/detail/munichsecurity-conference australasian journal of information systems le et al. 2023, vol 27, selected papers from acis 2021 factors affecting prosocial sharing 25 wright, m. f., & li, y. (2011). the associations between young adults’ face-to-face prosocial behaviors and their online prosocial behaviors. computers in human behavior, 27(5), 1959-1962. doi:10.1016/j.chb.2011.04.019 yan, l., & tan, y. (2014). feeling blue? go online: an empirical study of social support among patients. information systems research, 25(4), 690-709. doi:10.1287/isre.2014.0538 yan, z., wang, t., chen, y., & zhang, h. (2016). knowledge sharing in online health communities: a social exchange theory perspective. information & management, 53(5), 643-653. yang, z. j. (2016). altruism during ebola: risk perception, issue salience, cultural cognition, and information processing. risk analysis, 36(6), 1079-1089. doi:doi.org/10.1111/risa.12526 yuen, k. f., leong, j. z. e., wong, y. d., & wang, x. (2021). panic buying during covid-19: survival psychology and needs perspectives in deprived environments. international journal of disaster risk reduction, 62, 102421. doi:10.1016/j.ijdrr.2021.102421 zhao, h., fu, s., & chen, x. (2020). promoting users’ intention to share online health articles on social media: the role of confirmation bias. information processing & management, 57(6), 102354. doi:10.1016/j.ipm.2020.102354 zhao, y., da, j., & yan, j. (2021). detecting health misinformation in online health communities: incorporating behavioral features into machine learning based approaches. information processing & management, 58(1), 102390. doi:10.1016/j.ipm.2020.102390 copyright: © 2023 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v27i0.4349 http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 literature review 2.1 healthand information-related problems during a health crisis 2.2 social media usage during a health crisis 2.3 prosocial sharing of health-related information on social media 2.4 social exchange theory 2.5 the fear appeal model 3 research model and hypotheses 3.1 the threat appraisal process: impacts of health risk and information quality risk 3.2 exchanging motivations for online prosocial sharing 3.3 protecting motivations for online prosocial sharing 4 method 4.1 measures 4.2 sample selection 4.3 data analysis technique 5 research findings 5.1 measurement model 5.2 structural model 6 discussion 6.1 theoretical contributions 6.2 practical implications 6.3 limitations and future research 7 conclusion paper title (paper title style) australasian journal of information systems alam, sun & campbell 2020, vol 24, research article motivation in glam crowdsourcing 1 helping yourself or others? motivation dynamics for highperforming volunteers in glam crowdsourcing sultana lubna alam deakin business school deakin university australia lubna.alam@deakin.edu.au ruonan sun research school of management australian national university john campbell australian national university abstract while most crowdsourcing (cs) cases in the literature focus on commercial organisations, little is known about volunteers’ motivation of initial and continued participation in not-for-profit cs projects and importantly, about how the motivations may change over time. it is vital to understand motivation and motivational dynamics in a not-for-profit context because a fundamental challenge for not-for-profit cs initiations is to recruit and keep volunteers motivated without any formal contract or financial incentives. to tackle this challenge, we explore high performing volunteers’ initial motivation for joining and sustaining with a glam (galleries, libraries, archives and museums) cs project. we situated our interpretive exploration in a case study of the australian newspapers cs project initiated by the national library of australia. based on the case study, we found that high-performing volunteers were motivated by a combination of personal, collective, and external factors classified into intrinsic, extrinsic, and internalised extrinsic motivations. further, we found that these motivations changed over time. specifically, many volunteers presented substantial personal (i.e., personal interest and fun) and community-centric motivations (i.e. altruism and non-profit cause) when they initially joined the project, whereas external motivations (i.e., recognition and rewards) had a greater impact on long-term participation. our findings offer implications for cs system design (e.g., user profiles, tagging and commenting), incentive structure (e.g., reputationbased ranking, leader boards), and relational mechanisms (e.g., open communication channels) to stimulate sustainable contributions for not-for-profit cs initiatives. keywords: motivation, crowdsourcing, not-for-profit, motivation dynamics, glam 1 introduction crowdsourcing (cs) refers to the act of harnessing diverse potential of external contributors by issuing open calls for accomplishing particular tasks via online platforms (adams and ramos 2010; howe 2006). gartner predicted that 75 per cent of high-performing enterprises would use cs in some forms by 2018 (grewal-carr et al. 2016). not-for-profit organizations, such as government agencies and glam (galleries, libraries, archives, and museums), increasingly employ cs to better leverage collective knowledge and experience of external crowd to establish connections, understand requirements, and coordinate actions. this paper focuses on not-for-profit cs because its fundamental challenge is to recruit and keep australasian journal of information systems alam, sun & campbell 2020, vol 24, research article motivation in glam crowdsourcing 2 volunteers motivated to contribute without any monetary incentives (alam and campbell 2017). furthermore, not-for-profit organizations do not have a reasonably agreed upon goal of profit maximization and, thereby making success measurement more challenging (chandler 1962). although cs has drawn increasing attention from the literature, most studies focused on cs activities initiated by commercial organisations (afuah and tucci 2012; brabham 2010; grewal-carr and bates 2016; kaufmann et al. 2011; rodell 2013; schlagwein and bjørnandersen 2014). not-for-profit cs initiations are scant in the is literature. the growing cs use by not-for-profit organisations is facilitating new forms of digital volunteerism where no formal contract or monetary compensation are provided (benkler and nissenbaum 2006; cobb et al. 2014; dutton 2010; starbird and palen 2011). cs for social goals (e.g., supporting public libraries and people with disabilities) has recently received increasing attention from scholars (alam and campbell 2017; kobayashi et al. 2015). in particular, cs use by glam is boosted by the emergence of digital workers and the rising importance of information and cultural production (oomen and aroyo 2011; owens 2013; ridge 2013). in a not-for-profit context, cs can help extend and redefine the long-standing traditions of volunteerism, such as citizen involvement in the creation and enhancement of public and cultural goods (holley 2010; karen and cyndi 2011; oomen and aroyo 2011). the tasks carried out by cs volunteers typically include correction, contextualization, collection, classification, co-curation, and crowdfunding (oomen and aroyo 2011). the literature has identified volunteers’ motivation as a critical factor for cs success. however, limited research considered motivation dynamics in cs settings (crowston and fagnot 2018; taylor and joshi 2019), especially in a not-for-profit glam context. glam cs initiative is different in terms of ownership, task autonomy, and the types of tasks outsourced. specifically, volunteers contribute towards a common cultural goal where the data, knowledge, or other products created by volunteers are overseen and managed by a glam institution, as the custodian or caretaker of these new cultural assets. many glam institutions found it difficult to attract and retain volunteers on their cs platforms. our study seeks to address this issue by exploring volunteers’ motivations of a successful library cs with the aim of gaining an understanding of the motivation dynamics from initial to continued participation. while contextual factors have a significant bearing on the efficacy of different motivators in cs projects, there remains the need for a more rigorous theoretical orientation to understand cs phenomena in the glam sector. in so doing, we also respond to the call for information systems researchers to extend their focus to non-traditional commercial organisations (majchrzak and malhotra 2013). we investigate why and how high-performing volunteers joined and continued to participate in a large-scale, not-for-profit glam cs project, the australian newspaper digitization program (andp), initiated by the national library of australia (nla)); now known as trove (https://trove.nla.gov.au/). we employed the notions of intrinsic and extrinsic motivations to gain insights into (1) volunteers’ motivation for joining the andp trove, (2) their ongoing motivations to continue contributing to andp, and (3) work practices and strategies for transitioning novice volunteers into leading high-performance contributors over time. greater understanding of these motivations and their temporal aspects will lead to better cs designs to facilitate broader and sustained collaboration between glam and volunteers. australasian journal of information systems alam, sun & campbell 2020, vol 24, research article motivation in glam crowdsourcing 3 the rest of the paper proceeds as follows. we first survey the state of knowledge about volunteer motivation to identify gaps in motivation research. next, we report on our case study research method followed by a comprehensive case analysis. based on the case analysis, we present a typology of the motivation dynamics, leading to a discussion on the importance of understanding the temporal aspects in volunteer motivation in not-for-profit cs initiations. we conclude with implications, limitations, and opportunities for future research. 2 conceptual background 2.1 cs volunteer motivation the extant research has built on the dichotomy of intrinsic and extrinsic motivations based on the self-determination theory (sdt) to examine motivational dynamics in cs settings (deci et al. 1989; ryan and deci 1985; 2000). the sdt focuses on human intensions from which inferences could be drawn about whether specific events tend to be autonomous or controlled (deci et al. 1989, p. 580). autonomy means ‘endorsing one’s action at the highest level of reflection’ (gagné and deci 2005, p. 334). intrinsic motivation exemplifies autonomy and emphasises inherent sources of satisfaction (e.g., acting just for fun) when people engage an activity (ryan and deci 2000). on the contrary, control involves ‘acting with a sense of pressure, a sense of having to engage in the action’ (gagné and deci 2005, p. 334). for example, extrinsic motivation (e.g., acting for money) has been experimented as an instrument for achieving a certain desired outcome.(ryan and deci 2000). in their systematic review of open source software development literature1 , von krogh and colleagues (2012) identified three classes of motivations based on the sdt: (1) intrinsic motivations such as ideology, altruism, kinship, and fun, (2) extrinsic motivations such as career and pay, and (3) internalised extrinsic motivations that have extrinsic qualities but are internalised in ways that become self-regulating. hemetsberger (2004) viewed motivation as ‘self-interest’ and ‘others-orientation’ (as cited in (von krogh et al. 2012). self-interest was further segmented into task and product-related motivation (corresponding to intrinsic motivation); others-orientation, including long-term utilitarian goals and social significance (corresponding to extrinsic motivation) was separated into internalised group goals and values, and socio-emotional relationships. kobayashi et al. (2015) added similar dimensions of personal vs. social aspects to intrinsic and extrinsic motivations by studying a system that supported people with disabilities. table 1 summarises relevant literature delineating motivations for participating in cs projects and uses kobayashi et al.’s personal and social categories to demarcate internally and externally sourced motivation. most cs projects in table 1 were in the not-for-profit context (e.g. wikipedia, open source community, user generated content, online community, crowdfunding). only kaufmann et al. (2011) (amazon mechanical turk) and brabham (2010) (threadless) examined contexts where monetary incentives were provided. 1 we argue that motivations of open source software development can indicate insights into motivations of mot-for-profit cs initiations because extensive research on motivating crowds to contribute to cs initiatives has been in the context of open source software development (afuah and tucci 2012). australasian journal of information systems alam, sun & campbell 2020, vol 24, research article motivation in glam crowdsourcing 4 motivation category sub-category term from paper(s) papers intrinsicpersonal affective fun, enjoyment, curiosity, task autonomy, task identity, learning, ownuse, pastime, rituals and habit, crowston and fagnot 2018, ye and kankanhalli 2017, mcinnis et al. (2016), kobayashi et al. (2015), von krogh et al. (2012), nov (2007), brabham (2010), rafaeli and ariel (2008) personally (self-oriented) motivation interest, personal growth/self-fulfilment, personal taste/preference crowston and fagnot 2018, rafaeli and ariel (2008), kaufmann et al. (2011), kobayashi et al. (2015) intrinsicsocial communitybased altruism, kinship, external self-concept (meets expectations of group), reciprocity, ideology crowston and fagnot 2018, mcinnis et al., 2016 , kobayashi et al. (2015), gerber and hui (2013), von krogh et al. (2012), malone et al. (2010), rafaeli and ariel (2008), yang and lai (2010) extrinsicpersonal human capital advancement internal self-concept, human capital advancement, signalling, career, learning deng et al. (2016, yang and lai (2010), gerber and hui (2013), von krogh et al. (2012), brabham (2012), rafaeli and ariel (2008 payment pay (monetary) deng et al 2016, kaufmann et al 2011, gerber and hui (2013) reward recognition, reputation, self-expression crowston and fagnot (2018), deng et al (2016), kobayashi et al. (2015), von krogh et al. (2012), malone et al. (2010), brabham (2012) extrinsicsocial social contact indirect feedback, diversion, social integration crowston and fagnot (2018), kobayashi et al. (2015), rotman et al. (2012), kaufmann et al. (2011), malone et al. (2010) chance to socialize, chance for collaboration, advocacy crowston and fagnot (2018), kobayashi et al. (2015), rotman et al (2012), malone et al. (2010), brabham (2012) table 1. summary of the literature on motivation for participating in cs projects the intrinsic-personal motivation including fun and enjoyment are frequently found positively related to cs participation (crowston and fagnot 2018; von krogh et al. 2012; nov et al. 2007; malone et al. 2010; brabham 2012; rafaeli and ariel 2008; ye and kankanhalli 2017). for example, nov et al. (2007) and rafaeli and ariel (2008) examined motivations of contributors to wikipedia and identified that fun was one of the top motivators for participation. similarly, crowston and fagnot (2018) found curiosity and fun as major motivators for user-generated content contributors further, even in a commercial context, kaufmann et al. (2011) developed a comprehensive model of worker’s motivations for paid cs initiatives (e.g. amazon mechanical turk) and found that ‘task autonomy’ was one of the two most prominent factors for motivation. in terms of intrinsic-social motivation, gerber and hui (2013), van krogh et al. (2012), and rafeli and ariel (2008) identified altruism. geiger and hui (2013) studied motivations and deterrents for creators and supporters to participate in crowdfunding projects. yang and lai (2010) identified external self-concept motivation in their study of wikipedia contributors that fits well with the category of intrinsic-social motivation. external self-concept motivation is the primary motivation for individuals to adopt an activity that is congruent with the expectations of a reference group. crowston and fagnot (2018) found social motives important for useraustralasian journal of information systems alam, sun & campbell 2020, vol 24, research article motivation in glam crowdsourcing 5 generated content. von krogh et al. (2012) also found reciprocity as an important motivator for non-profit contexts. kobayashi et al. (2015) classified motivations in extrinsic-personal category as human capital advancement, signalling, payment, and reward. human capital advancement is related to the notion of internal self-concept (yang & lai, 2010). internal self-concept motivation refers to the force that drives individuals to pursue an activity that meets their inherent standards. yang and lai (2010) found that internal self-concept was strongly correlated with knowledgesharing behaviour through an online survey. human capital advancement is also equivalent with learning (brabham, 2012; gerber and hui 2013; rafaeli & ariel 2008; van krogh et al., 2012) as well as own use for personal purposes (von krogh et al., 2012). others are motivated by signalling (kobayashi et al., 2015), such as reputation (gerber and hui 2013; von krogh et al., 2012; malone et al., 2010; rafaeli & ariel 2008) and self-expression (brabham 2012). brabham (2012) synthesized ryan and deci’s (2000) model with knoke and wright-isak's (1982) classifications to examine a transit-planning project. the only reward offered to the winners of that competition was acknowledgement in a press release and a formal announcement on the competition web site. brabham (2012) found that a mixture of intrinsic and extrinsic motivators including the opportunity for career advancement, motivated cs participation behaviour. this finding is also consistent with the findings by geiger and hui (2013) in their study of crowdfunding. further, extrinsic-personal motivations are enabled by more traditional incentives (whyte 1955), such as payment and reward (deng et al 2016; geber and hui 2013; kaufmann et al., 2011; rafaeli & ariel 2008; von krogh et al. (2012).for example, . glory or recognition is at hand if volunteers are motivated by the desire to be recognised by peers for their contributions (malone et al. 2010). the final category, extrinsic-social motivations include social integration (rafaeli and ariel 2008), chance to socialize (malone et al. 2010), chance for collaboration (brabham 2012), indirect feedback (rotman et al. 2012; kaufmann et al. 2011), and advocacy (rotman et al. 2012). 2.2 motivation dynamics in this paper, we define motivation dynamics as changes in motivation over time; specifically from initial participation to continued participation. although the literature has shown rich insights into motivation of participating in cs projects, we recognize that few studies considered motivation dynamics. in psychology, researchers have found that people’s motivation of doing something is often different and changes over time (turner and patrick 2008). cs projects are inherently complex activities that spread over long periods and span multiple tasks. in these projects, motivations may not only change over time, but could also be particularly salient at critical junctions of activity and decision making (rotman et al. 2014; rotman et al. 2012; turner and patrick 2008). however, in prior research, motivation for contribution has been considered ‘a single, static and individual phenomenon’ (crowston and fagnot 2018, p.89). in a recent study on wikimedia editor survey, crowston and fagnot (2018) found that different stages of contribution have distinct motives. using the theory of helping behaviour as a framework and integrating social movement theory, they proposed a stage theory that distinguishes separate motivations for initial, sustained, and meta contributions for participation in user-generated content. they found initial contributions were motivated largely by curiosity; sustained contributions largely by intrinsic interest; and metaaustralasian journal of information systems alam, sun & campbell 2020, vol 24, research article motivation in glam crowdsourcing 6 contributions increasingly by social motives. rotman et al. (2012, 2014) studied the dynamic aspects of motivation in collaborative citizen science projects and found that the temporality affected three decision points in time: (1) volunteers’ initial decision to participate in a project, (2) active participation, and (3) the ensuing decision to continue once the initial task was completed. they found that both scientists and volunteers presented egoism as the primary motivation for initial engagement. however, beyond initial engagement, altruism, and collectivism played important roles in the decision to continue participation. this scant research confirms our proposition that it is a mistake to assume that the motives for sustained volunteers are just more of whatever got them to initially participate. thus, motivation dynamics have potentially important theoretical and design implications that this study will explore. in sum, the literature on cs motivation has not reflected a cumulative body of systematic work on the changing nature of motivational dynamics over time. this indicates a need for further work on theorising the temporal aspects with a rich in-depth exploration, in particular without any formal contract and financial incentive. responding to this gap in literature, this study addresses the research question: how does motivation changes from initial participation to continued participation in a not-for-profit setting? answering this question is important because they are relevant to tackling some of cs’s fundamental challenges, such as how to recruit and keep volunteers motivated to continue participation (doan et al. 2011). the current paper builds on existing work of the authors for non-profit cs for glam sector. in alam and campbell (2016), the authors studied temporal aspects of organisational motivation to cs. in alam and campbell (2017), the focus was on changes in participant motivation across different types of contributions. they proposed that there were two forms of contribution: data shaping and knowledge shaping. they theorised how the focus and locus of motivation changes across these two types of contributions at crowd and community level over time. continuing the work on the role of temporality in participant motivation for notfor-profit glam cs (e.g. alam and campbell 2012), in this current paper, we focus on how motivation changes from initial participation to continued participation based on the intrinsic and extrinsic dichotomy. we examine how motivation to cs participation changes over time by examining the participation journeys of high performing volunteers in a case study. 3 research method we adopted an interpretive research approach to explore cs motivation dynamics with a unique cs case. an interpretive approach is suitable because it is particularly strong in uncovering novel insights for accessing intricate details, thought processes and emotions (klein and myers 1999; orlikowski and baroudi 1991). the andp and the trove is an appropriate site for this study for three reasons. first, the andp is a large-scale glam cs initiative and as such publicly available data ensures the richness of our data sources. notably the nla is the third largest library in the world. second, trove was launched in 2008, which provided us longitudinal, first-hand observations of participants’ contributions and motivations. third, since andp (i.e. trove) is an on-going project, it allows us to explore motivation dynamics on a temporal base. 3.1 the andp case we explore the motivations of high performing correctors in the andp. the andp aimed to provide an online full-text searchable digitised newspaper delivery system of out of copyright australasian journal of information systems alam, sun & campbell 2020, vol 24, research article motivation in glam crowdsourcing 7 australian newspapers from 1803 to 1954 (https://trove.nla.gov.au/). however, numerous errors were recorded during the optical character recognition (ocr) process that greatly limited the searchability of entries. unfortunately, the nla did not have the resources to rectify the errors thus cs was a potential cost-effective solution. to address such issue, the andp released the trove platform, which provides a split screen view of the scanned image of the newspaper and the ocr transcription. the design of the page is suited for corrections, as the original document is immediately juxtaposed with the transcription, so errors are more apparent. corrections can also be made in the form of comments linked to an article, giving users an opportunity to record their own narrative or interpretation of historical events. users can view the history of corrections, including both the corrected and the original ocr text. text corrections are saved to a database and are subsequently added to the search results. however, corrected text does not overwrite the original text contained in the article. both the corrected text and the original text are indexed and searchable. as at september 2019, 294,889 registered volunteers have corrected and enhanced more than 325 million lines, added seven million tags, left 248,212 comments, and created 36,287 merges/splits on trove. on an average, 8,000 registered volunteers actively contribute to the correction each month. trove users who corrected text were more likely to be family historians, retired, and long-term (more than one year) frequent trove users (ayres 2013). sixty-five percent of users are aged 50 or over; 34 per cent are aged 60 or over; only 17 per cent of users were aged under 40 (ayres 2013). similar to other commons-based peer production systems like wikipedia, hagon (2013)’s research on trove user engagement found that the top 100 users have undertaken 43 percent of all corrections (i.e., 41 million lines of text), and the top 1,000 users have undertaken 81% of the corrections. hence, we examined the motivations of the high performing text correctors and were in the aged 50 or over 3.2 data collection and analysis overall, 20 in-depth interviews were carried out over 2012-2013, which included six leading text correctors and two representatives of group text correctors (see table 2, note mr and mrs hoy corrected using the same trove account). interviews lasted between one to two hours and were undertaken by telephone or skype. volunteers were selected using purposeful sampling and snowball approach (patton 2000). as we focused on high-performing volunteers, a majority (five out of six) of the interviewees were listed as the top 10 text correctors in the trove hall of fame. they were approached through the trove forum platform, sending invitations by personal messages using their user profile. the interviews were both semistructured and relatively informal. they usually centred on involvement with the australian newspapers site – motivations for participating, text correction practices, perspectives and sentiments towards cs, risks and concerns, support received, and cs platform. the interviewees to illustrate their perspectives and experiences often shared anecdotes and specific narratives. data collection ended at the point of redundancy (lincoln and guba 1985) ‘when efforts to get additional members cannot be justified in terms of the additional outlay of energy and resources’ (p.233). the secondary interviews (twelve) and trove user profiles were used to triangulate the text corrector’s recount of their journey. to achieve triangulation, we collected data from three sources of evidence: andp documentation, trove observation, and interviews (eisenhardt 1989). we analysed andp press releases, nla reports, trove user surveys and extant literature on the case study (e.g. in particular publications by holley 2009, 2010 who was the australasian journal of information systems alam, sun & campbell 2020, vol 24, research article motivation in glam crowdsourcing 8 project manager of andp). further, first author registered as a trove user to observe how the cs platform was used to gain representative experience. all interviews were recorded and transcribed. nvivo was used for textual content analysis. we used data gathered from documentation and trove operation to corroborate, validate, and complement interview data. number age status brief narrative of their journey 1 60-70 retired. principal technical officer interested in family history research. top 5 corrector. 1.2+ m lines of text correction, 840 comments (feb 2014). joined in august 2008. 2 40-50 stay at home mother interested in family/local history and murders. top 10 corrector. volunteers at bendigo historical society 1 day per week. joined in august 2008. 3 60-70 retired. used to own a small business in it mostly interested in family research. top 5 corrector. corrected 2.3+ m lines of text correction, 7 comments. joined in june 2009. 4 50-60 retired. systems analyst programmer interested in family history. top 5 corrector. full-time volunteer. 1.7+ m lines of text correction, 460 comments. joined in july 2009. 5 50-60 retired. worked in banking interested in family/local history and shipping. top 5 corrector. married to mr hoy, shares the same trove account. 1.1+m lines of text correction, 36 comments (feb 2014). joined in august 2008. 6 60-70 retired worked in banking interested in book chapters in newspaper, shipping. top 5 corrector. married to mrs hoy, shares the same trove account. 1.1+m lines of text correction, 36 comments. joined in august 2008. 7 70+ retired, author and historian interested in various things, related to her work. top 100 corrector. active trove forum volunteer. 8 60-70 manager of professional standards and student protection interested in light railways. active member of light railway research association (lrrsa) and editor of their magazine. only tags relevant articles with lrrsa. 9 30-40 works parttime. phd student interested in weather tables. he is the communications and project officer for the climate research group in university of melbourne’s arc linkage grant with nla. the project collects weather tables from trove newspapers. also, through their citizen science project, additional people get involved with trove. table 2.summary of volunteer demographics and text correction practices this study followed a thematic data analysis technique on the basis of iteratively exploring themes from gathered data as proposed by saldaña (2015). first-cut coding was carried out using an ‘open-coding’ approach to identify common themes (saldana 2015). second cut analysis delved into further relevant concepts from the existing literature (i.e., intrinsic and extrinsic motivations) to gain a better understanding and explanations and lastly further concepts inductively developed during the analysis from on-going research into relevant theories (i.e. internalised extrinsic motivations). final analysis broken these motivations into initial and continued participation. one author conducted the initial round of coding in the original languages of the transcribed data. as we were particularly interested in cs motivations, we paid special attention that the coding included information such as why the participants joined the project, why the participants continued contributing to the project, and why the participants withdrew from australasian journal of information systems alam, sun & campbell 2020, vol 24, research article motivation in glam crowdsourcing 9 the project (if applicable). the other author went through the coding and verified the codes until reaching a consensus. both authors then organised the initial codes into potential categories and added motivational dynamics for initial and continued participation. as table 3 shows, sample excerpts from the data for the emergent themes are presented as examples of saturation. text-corrector motivation intrinsic motivation (category) personal (interest) oriented motivation (sub-category) theoretical concepts: personal interest (e.g. research goals), trust , challenge, learning new knowledge, competition, topic of interest (australian history), addiction, obligation, supportive environment interview comment: “consequently my reason for using the site is foremost for family history/genealogy reasons, but i also find the site invaluable as an aid in my research undertakings”. data from documentation review: “when a library appears to give up some control over its content it embodies in its public a great deal of trust. the trust is honoured and grows as users become actively involved with the content and feel a sense of responsibility working for the institution/service/common good. data from trove forum: my motives are far from altruistic. i'm addicted to tatting and the prospect of finding new patterns interests me, especially when they're by an australian author whose book sits on my shelf. i live in the los angeles, california area timeline: initial participation community-based motivation (sub-category) theoretical concepts: altruism, collectivism (e.g. genealogy), principalism (or action significance by external values) interview comment: “the rockhampton group actually arranged that as a meeting; as a chance to get together and have a chat and have a coffee as well as correct the text”. timeline: continued participation enjoyment-based motivation (sub-category) theoretical concepts: enjoyable/fun/pleasure/recreation, simplicity, task autonomy, pastime interview comment: “just the enjoyment is my motivation and gain”. “if i'm not doing anything else, i do this”. timeline: initial and continued participation extrinsic motivation (category) social motivation/mechanisms (non-monetary rewards) (sub-category) theoretical concepts: acknowledgement, attribution and reward, desire for recognition (ranking table and hall of fame), rewards (australian day awards), indirect feedback, advocacy interview comment: wow, the national library not so much i could get invited too but the national library really cares about this and acknowledges it. i think that was a really good thing to do. data from documentation review: the library invited the top five text correctors to canberra on australia day in 2010 to meet andp staff in recognition of their contributions to the project. timeline: continued participation table 3. sample excerpts from the data for the emergent themes and coding analysis 4 findings in this section, we present our case analysis with rich illustrative quotations from interviews supported by documentary data. to highlight motivation dynamics, we detail whether the motivations instigated initial participation or continued participation. australasian journal of information systems alam, sun & campbell 2020, vol 24, research article motivation in glam crowdsourcing 10 4.1 cs volunteers’ motivation: from initial to continued participation intrinsic motivations intrinsic motivation emphasises inherent sources of satisfaction rather than the separable consequences of the act (e.g., acting just for fun) (ryan and deci 2000). volunteers reported various intrinsic motivations when initially joined trove. personally (self)-oriented motivations self-interest (own-use value): the volunteers interviewed reported that their initial involvement originated mostly from their own personal interest or goals. this is similar to open-sourced based peer production systems where developers were also found to be motivated by personal interests (bonabeau 2009; von krogh et al. 2013). the majority were people interested in family history research or genealogy, but some volunteers were also interested in researching historical events such as famous crimes and public transport history. a senior trove support officer explained: actually, if you break down the top 10 text correctors, it's probably only 50 per cent genealogists. the others are sort of what we call recreational researchers. there's one who's fascinated with the history of murders and goes through and looks for those. generally most of the top the heavy text correctors or the enthusiasts have started at least as enthusiasts of something, but it's not necessarily family history if that makes sense. trove volunteers reported a range of shortand long-term goals for initial and continued participation. for example, one volunteer was an author and used the facility to research topics relevant to her ongoing needs as a writer, while another volunteer only corrected articles with references to his family history. addiction: some volunteers reported that they were ‘hooked in’ and ‘addicted’ to text correction, hinting addiction being a motivator for continued participation. the contributions by the top 10 text correctors consistently reflected a workload similar to that of a full-time job (40-60 hours per week). the top corrector at the time of interview admitted when asked about her reasons for continuation: ‘i suppose initially it was to fill in some of the family history gaps in my family and i can say it is very addictive once you get really involved’. learning: volunteers appreciated that they could learn new things and gain insights from the collection as they worked (“you learn stuff all the time”), hence indicating that learning contributed to both initial and continued participation. affective motivations we observed two types of affective motivations observed: hedonic-based motivations (e.g., fun, pastime, and interesting) and task-based motivations (e.g., simplicity and autonomy). fun: many volunteers participated in text correction because it was fun and enjoyable. for example, one volunteer believed that‘[j]ust the enjoyment is my motivation and gain’. they reported that they enjoyed the task and the interactions through tagging, commenting, and forming, which keeps them motivated over time. pastime: many of the high performing volunteers were in their 50s and some of them were transitioning to retirement or already retired. they used the site to pass time and to keep active. a volunteer who had long retired said, ‘if i'm not doing anything else, i do this’. hence, pastime was found to be a motivator for both initial and continued participation. australasian journal of information systems alam, sun & campbell 2020, vol 24, research article motivation in glam crowdsourcing 11 passion for topic: people were generally interested in topics such as history, science, animals, or particular hobbies and this was identified as motivator for their short and long-term participation (ridge 2013). newspapers were regarded as historical artefacts that people can relate to and connect with on a personal level. as one volunteer explained, ‘i think people just like history, especially the sort of history that’s presented in newspapers. it’s something that they’re very familiar with; it’s not threatening like getting a history book off the shelf’. simplicity: text correction was an easy task that does not need training. such simplicity in operation allowed an easy-start for volunteers to join trove. further, many volunteers had previous experience of managing large volumes of data either at work or through their involvement with other genealogy projects (e.g., ryerson index andancestry.com). as one volunteer explained, ‘just personally, for example, a lot of old documents i have transcribed for my own personal use’. task autonomy: another important motivator was the high-level of autonomy in task selection, which was attractive proposition for initial participation. volunteers were free to act according to self-articulated goals and principles. there were no restrictions on the time taken to complete a task. there were no strict guidelines on how to do text correction and multiple options were available for carrying out the task. task autonomy thus offered the volunteers freedom when working on trove. as one volunteer commented, ‘i do whatever tickles my fancy on the day’. another said, ‘you're not told how to do it. you're not told what you should do and what you shouldn't do. do it at your leisure’. community-based motivations community-based motivations are characterised by an individual’s perception of the task being important, meaningful, valuable, or worthwhile for a community: ‘a common core is the disposition to benefit others’ (benkler and nissenbaum 2006, p.407). we observed four related themes from the case. altruism: many volunteers saw text correction as an altruistic patriotic task that would help preserve australian history and the national library. volunteers believed that they were helping to provide an accurate record of australian history. the search facility was being improved through text correction and, in turn, having a wider impact on the australian community. for example, a volunteer expressed that ‘it’s being part of something bigger than them, being able to contribute something that has lasting value; being able to make a difference, being able to improve things for other people’. altruism thus motivates both initial and continued participation. kinship/collectivism: volunteers also reported that they corrected text to increase the welfare of specific groups to which they belonged (kinship/collectivism) which contributed to sustained participation over time. for example, some text correctors were also members of genealogy sites (e.g. rootsweb) and others volunteered for local libraries and genealogy societies (e.g., bendigo historical society). volunteers formed many interest groups within and outside the trove platform. for example, one such group were a rockhampton based group who focused their correction efforts on their local newspaper. another group were the light railway research society whose members tagged relevant newspaper articles to assist light railway researchers. we also observed another form of community that had no formal group structure but consisted of a loose network of volunteers. this loose affiliation led to collaboration independent of the nla, which resulted in a text correction guideline. the need australasian journal of information systems alam, sun & campbell 2020, vol 24, research article motivation in glam crowdsourcing 12 for a guideline stemmed from ongoing user practices and operational level text correction issues that arose due to the absence of formal rules. the top contributors collaborated through emails and the trove forum to create and disseminate their own guideline with little assistance from nla. ownership: registered users comprise around 85 percent of corrections on trove (hagon 2013). the high correlation between user registration and text correction indicated that volunteers felt they ‘belonged’ to the service, and that they had forged a long-term relationship with the library and the collection (ayres 2013). as the trove support officer explained: well i think by allowing users to register and create a profile, we have given them the opportunity to take ownership of the work that they're doing ... and would view themselves as individual stakeholders in the project, not just as part of the project. non-profit cause: volunteers were much more likely to help not-for-profit organisations than commercial companies because they did not want to feel that their work can be commercially exploited as they believe they are contributing to public good. based on norms and values of volunteerism, trove volunteers were empathetic to the not-for-profit heritage context of the library that motivated their initial contribution. as the project manager explained: some of the success factors why they would help us as opposed to anyone else, is because we are not-for-profit, we don't have as much money as we like, and they see us as a worthy cause. people are much more likely to help libraries and archives than they are a whole lot of other organisations. libraries per se, people are much more likely to help the national library, because they perceive us to be more trustworthy and more honest and more reliable. internalised extrinsic motivations internalised extrinsic motivations are by definition extrinsic but were internalised by volunteers., such motivations were perceived as self-regulating behaviour rather than externally imposed (ryan and deci 2000; von krogh et al. 2013). we identified four types of such motivations based on the case. trust: nla’s experience indicated that the greater the level of freedom and trust given to volunteers resulted in more being contributed, greater feelings of loyalty, and higher level of accuracy. rather than assuming everything would go wrong and spending valuable resources putting systems in place to control the potential for vandalism (e.g., moderation), nla assumed that users would do their best and monitor and help each other (holley 2009). this approach succeeded and led to positive trusting relationship. the project manager wrote in a related nla report (holley 2010): when a library appears to give up some control over its content it embodies in its public a great deal of trust. the trust is honoured and grows as users become actively involved with the content and feel a sense of responsibility working for the institution/service/common good. obligation/reciprocity: volunteers felt a sense of gratitude to the library for providing the resource and in return felt morally obligated to assist the library in improving the searchability of the resource, as on volunteer said ‘i felt an obligation to repay the assistance that i have received from the existence of the site. to sing for my supper, so to speak’. challenge: for some volunteers, the more content provided, the more they felt motivated to contribute. large amounts of new content were added as the program progressed which further motivated some volunteers to do more correction. australasian journal of information systems alam, sun & campbell 2020, vol 24, research article motivation in glam crowdsourcing 13 competition (internal self-concept): two forms of competitive activity were observed within the case study. one common competitive activity involved volunteers setting their own productivity targets (e.g., how much text correction to be done per month) – an internal selfconcept motivation that drives individuals to pursue an activity that meets their inherent standards (yang and lai 2010). in this instance, the volunteers were interested to being able to check what and how much material they had corrected over time (monthly, weekly, etc.). these personal goals and aspirations increased their self-esteem. however, evidence of rivalry among the top volunteers were evidenced over time as they competed to hold onto or improve their position on the ranking tables. this competitive attitude was observed between two top performers. extrinsic motivations in our case, volunteers were mostly intrinsically motivated, and the library offered no monetary incentives for carrying out text correction. instead, the library deployed nonmonetary incentive mechanisms. essentially two types of extrinsic motivators were found through data analysis: (1) reputation-based rewards and (2) social mechanisms. the social (or status) motivations implemented by the library thus had a ‘crowding in’ effect (frey and jegen 2001) over time as illustrated in the examples below. reputation-based rewards initially, the nla did not offer acknowledgement or reward to volunteers, but later instigated a number of simple and cost-free rewards and acknowledgements as suggested by the volunteers. acknowledgement of their achievements was important to the volunteer community. such acknowledgement took many forms: attribution was done by identifying text correctors (or their account handles) on the articles they amended. volunteers could immediately see what they had corrected. importantly, it was possible for others to retrieve a newspaper article and see the list of people who contributed to the corrections in that article. it was also possible to search by a ‘user profile’ (or account handle) and see what corrections that person had done, their rankings, comments, and tags. the acknowledgements, attributions, and rewards recognised volunteers’ contribution and became a key motivator for continued contribution. recognition via leader boards (e.g., ranking tables and the hall of fame) were extrinsically rewarding to volunteers. leader boards were one of the few ways to obtain positive feedback for their effort, which enabled volunteers to track their progress and monitor others over time. originally, the ranking table only showed the top 10 correctors. gradually, due to requests from other volunteers, a text correctors hall of fame was added listing anyone who had corrected more than 1,500 lines of corrections in a month, as well as including a user’s overall ranking on their user profile. these features added some competitiveness and continuity to motivate future contributions. several staff and volunteers also mentioned other forms of recognition during interviews. for example, the library invited the top five text correctors to canberra on australia day 2010 to meet andp staff as acknowledgement of their contributions to the project. the lead it architect, kent, reflected: that was a fantastic idea. it got a lot of news in the media and made those people feel good and other text correctors think: wow, the national library really cares about this and acknowledges it. australasian journal of information systems alam, sun & campbell 2020, vol 24, research article motivation in glam crowdsourcing 14 social mechanisms indirect feedback: feedback mechanisms were initially expected to create an open communication channel between the nla and volunteers, but these mechanisms also became prominent motivational factors. indirect feedback motivated volunteers by increasing trust and self-competition. indirect feedback took many forms within australian newspapers site, such as impromptu updates about the newspaper correction and its progress on the actual website, regular email acknowledgement outstanding text correction work in general, and public postings on the forum and in nla newsletters by the project team recognising the efforts of the text corrector volunteers. the project manager and senior nla staff also recognised the work of the volunteers in conferences and other publications and through this, a lot of interest both nationally and internationally was generated among other libraries and in the media. text corrector profiles were highlighted in invited seminars and workshops around the country. the publicity peaked when the text correction activity became reported by popular media, and many personal stories of text correction work were lauded in local newspapers. thus, feedback motivated both initial and continued participation. advocacy: advocacy can be seen as a collectivist motivation (unlike education, which emphasises the personal gains volunteers may receive from their participation in cs projects) that unfolded during continued participation. there was evidence of this form of motivation at work within the adnp but to a lesser extent than reported in citizen science cs projects (see rotman et al. 2012). some volunteers saw the newly found knowledge (e.g., family history resource) as an educational benefit from which they not only gained personally, but also a knowledge base that they could later bring to their local and distributed communities and social networks. summary drawing from the case analysis, we propose a motivation typology for the not-for-profit cs settings as presented in table 4. the typology has three high level categories: (1) intrinsic motivation, (2) internalised extrinsic motivation, and (3) extrinsic motivation. intrinsic motivation includes the personally (self)-oriented motivations mostly driven by self-interest (or own-use value) and affective motivations that are related to task characteristics and enjoyment, as well as community-based motivations that identify with the welfare and principles of the group or collective. the extrinsic motivations include non-monetary social mechanisms and reputation-based motivations. the data also shows internalized extrinsic motivations that are extrinsic in nature but perceived as intrinsic due to their self-regulatory nature. the proposed motivation typology extends previous studies in the not-for-profit cs contexts (cf. table 1 and table 4). for example, volunteers are dominantly motivated by personalintrinsic motivations, particularly in this case for their interests in family history research and genealogy. indirect feedback and reputation-based rewards were also found to be useful. two motivators were found to be prominent in contrast to previous studies: passion for topic and non-profit cause. passion for historical artefacts such as old newspapers were found to play an important role in engaging volunteers. additionally, not-for-profit cause was found to be a major player in their decision to continue to correct text, as it was a national resource. these two motivators were unique to the glam cs contexts. australasian journal of information systems alam, sun & campbell 2020, vol 24, research article motivation in glam crowdsourcing 15 motivation type category constructs intrinsic motivations personal personally (self)oriented motivation own-use (self-interest), learning, addiction affective motivation fun, pastime, passion for topic, simplicity, task autonomy social community-based motivation altruism, kinship/collectivism) non-profit cause, ownership internalised extrinsic motivations personal self-esteem challenge, competition (internal self-concept), social pro-social motivation trust, obligation extrinsic motivations personal reputation-based recognition & reputation-based reward, attribution social social mechanisms indirect feedback, advocacy table 4. motivation typology of motivation of voluntary cs 4.2 motivational dynamics from beginner to leading contributions in addition to previous analysis, to further explore motivational dynamics, we observed how the motivations and work practices of contributors changed over time from initial to sustained participation as they moved from being a novice volunteer to high-performing volunteer (ranked top 1000 in the hall of fame). high-performing volunteers reported they were initially motivated by self-serving research interests (e.g., genealogy, australian history, and famous crimes). for example, many volunteers first encountered the digitised resource while researching a specific topic. while others were referred to the site from genealogy forums and blogs: i was alerted to the site through an email posted on a jewish email site that i subscribe to. i straight away began to look for any family notices that i could add to my family tree, and i think one of the first articles i corrected was the death notice of my great great grandfather in 1922. i have been interested in family history for about 35 years, and the newspaper site was just another avenue to explore to garner more details. volunteers described the transition from initial involvement to active involvement as broadening of focus from their initial research interest to other sometimes much wider areas of interest: my personal interests are in family history, but i don't confine myself to correcting articles or family notices, as we call them. i often go off on tangents and i have corrected many things. text correction also becomes a routine task in conjunction with other daily chores: any correction done during the day may be done on and off through normal household interruptions. in the move from active volunteer to high-performing, motivations broadened to include altruistic goals of improving a public good that is of historical significance to australians, and collective goals of servicing specific needs of special interest groups (e.g. genealogy societies, railway researchers). the project manager of andp reflected: “the sense of making history, being a part of history and recording history is very important to most australians. so australian genealogists are perhaps even more community spirited than other nations and perhaps this is why the text correction australasian journal of information systems alam, sun & campbell 2020, vol 24, research article motivation in glam crowdsourcing 16 has been so successful.” a top text corrector who was a genealogist confirmed, “i also voluntarily help a lot of people with their family history research”. further, volunteers reported a greater awareness of advanced features of the cs platform as they become more active on the site. individuals could establish an identity profile within the site through their user account, which allowed them to track their own contributions (e.g., text corrections, tags, and comments), progress and comparative ranking against others (e.g., monthly leader boards and the hall of fame). they used the additional features of tagging and commenting as necessary to support their work as one volunteer explained: i tag most of my work, where it seems appropriate, and when considering a new tag, i check to see how any existing tag on the topic is entered. i enter comments only rarely, usually when there is some serious fault either in the original article, or in the computerisation, and occasionally if there is a gross factual error. regular contributors also assumed that others would adopt the role of moderator: i am able to ask when i feel i need help. if i am accidentally doing the wrong thing, i hope that somebody will notice and tell me. if they do not, well, at least the text will be better when i have finished working on it than it was when i started. as volunteers moved from occasional contribution to frequent and routine text correction, some assumed additional administrative and leadership roles. their scope of activities extended beyond mere serendipitous editing. for example, leading contributors also assumed the role of moderation and maintained a strong sense of preserving standards. they used user profiles to monitor and track accuracy and quality of correction as if it was their responsibility to maintain the integrity of the corrected articles. they actively voiced their concerns over incorrect editing (e.g. how to correct em-dashes). for example: it upsets me to see someone changing data which is fixed historically and not correcting it properly. i found, in the last week, another user has been doing corrections and using ampersands, changing line breaks, capitalising words, adding his own words to births, deaths and marriages. i've done the corrections that he's been doing and i'm recorrecting them. a small number of leading correctors reported that they felt compelled to take further responsibility as their tenure of involvement extended beyond one or two years. two of the leading correctors interviewed had volunteered to draft text correction and tagging guidelines for the library. they also offered words of advice through participation in the trove forum and through invited presentations: if people want help, the couple of particular librarians involved in local studies and family history have my name and contact number and so they will give people my phone number if they need assistance and take it from there. 4.3 typology of dynamic changes in motivation over time we have shown insights into how leading contributors began their journey and over time became heavily active. specifically, their participation transformed from novices to leading contributors. as their participation become routine and frequent, volunteers reported that they adopted new goals and leadership roles even though there was no formal role structure within the site. over time, their motivations changed and broadened as the extrinsic motivations were introduced through reputation-based mechanisms (e.g., recognition and attribution). australasian journal of information systems alam, sun & campbell 2020, vol 24, research article motivation in glam crowdsourcing 17 high-performing volunteers reported high intrinsic motivation but lower extrinsic motivation at the initial stage. most text correctors presented a range of personal and community-centric motivations as the most substantial motivation for their initial engagement. external motivations such as recognition and rewards played a secondary role, but these factors had a greater impact on long-term engagement in the project (see table 5). for example, when text correctors reassessed their ongoing participation in text correction, both intrinsic and extrinsic motivations combined to play a vital role in their decision to continue to support the andp. establishing the “right balance” between intrinsic and extrinsic motivations of top text correctors facilitated their sustained participation over time. for non-profit glam context, internalised extrinsic motivators (e.g. trust and reciprocity) can play a vital role. motivation category initial participation continued participation intrinsic motivations personally (self)oriented motivation own-use (self-interest), topic of interest learning, addiction affective motivation fun, pastime, passion for topic, simplicity, task autonomy fun, pastime, passion for topic, task autonomy community-based motivation altruism, non-profit cause altruism, kinship (collectivism), ownership internalised extrinsic motivations self-esteem challenge challenge, competition (internal self-concept) pro-social motivation trust obligation/reciprocity extrinsic motivations reputation-based attribution recognition & reputation-based reward social mechanisms indirect feedback advocacy table 5. a typology of motivational dynamics for voluntary cs 5 discussion and conclusion in this section, we first summarised the key contributions of our study, followed by their implications for researchers and practitioners. we then illustrate limitations of this study and propose opportunities for further research. 5.1 contributions the paper makes two principal contributions. first, drawing on the trove case, the paper offers a comprehensive understanding of the changing nature of volunteer motivation in a not-for-profit cs context. the majority of the volunteers presented a range of personal (i.e., personal interest, and fun) and community-centric motivations (i.e., altruism and non-profit cause) as the most substantial motivation for their initial engagement. external motivations (i.e., recognition and rewards) played a secondary role but had a greater impact on long-term engagement in the project. thus, we contribute to the understudied debate on volunteer motivation changes for different stages of contributions and respond to the call for research on motives for contribution to consider both the stage of participation and separate motives for those stages for contribution (crowston and fagnot 2018). second, the empirically grounded theorisation of the typology and detailed motivational, technical, and relational aspects of motivation have implications for how to design and configure volunteer interactions and incentive designs over time to be successful. australasian journal of information systems alam, sun & campbell 2020, vol 24, research article motivation in glam crowdsourcing 18 5.2 implications for research the main contribution from this study was the temporality in the proposed volunteer motivation typology for not-for-profit cs projects. the case provided insights into the work practices, tool use, and management strategies behind volunteer engagement and transition from initial involvement to ongoing active contribution. we responded to the call by crowston and fagnot (2018) to consider different stage of participation and to separate motives for different stages of participation and contribution. the study found that a complex framework of personal, collective, and external factors motivated the volunteers. volunteers were highly intrinsically motivated, but community and external factors including non-monetary rewards played a vital role in their continued involvement. volunteers largely began participating because they were interested in a particular subject area and had some use for the material that they found (e.g., their own family history), and they found correcting to be fun. they could also justify the time spent because they felt that they were contributing to social good. some cs support mechanisms were principally useful in encouraging the leading volunteers by acknowledging their valuable work. the study also found that volunteer’s motivations were varied and often temporal, which suggests that motive alignment is also dynamic and may require adjustment over time to encourage ongoing participation. as we come to better understand the changing motivations of this new breed of digital volunteers of cultural data, we can better incorporate these motivations into the design of future not-for-profit cs collaboration sites. 5.3 implications for practice the volunteer motivations and dynamics have implications for future design and practice in the following areas (see table 6). design implications motivation involvement design mechanics design appropriate incentive mechanisms extrinsic motivation both initial and sustained involvement design non-monetary reputationbased incentive mechanisms (e.g. leader boards, user profiles) design appropriate organisational relational mechanisms both intrinsic and extrinsic motivation both initial and sustained involvement design for open communication and treat users as partners (e.g. forum, contact us form) design to build community intrinsic motivation sustained involvement design to support smaller communities within the larger crowdsourcing site (i.e. subgroups) with common interest (e.g. forum/forum threads) design to support different forms of contributions intrinsic initial and sustained involvement design features like tagging, commenting, lists, merge and splits to allow submission of multiple forms of contributions design for social support extrinsic initial and sustained participation design should incorporate new social networking tools (e.g. facebook, twitter, blog) table 6. practice and design implications of volunteer motivation design appropriate incentive mechanisms the findings from this study demonstrate emphatically the importance of considering appropriate motivation elements in cs platform design. cs platforms and support systems australasian journal of information systems alam, sun & campbell 2020, vol 24, research article motivation in glam crowdsourcing 19 should align with volunteer motivations in order to attract, incentivise, sustain, and guide the crowd to complete required tasks. although the nla was initially unable or unwilling to offer acknowledgement or rewards to volunteers (e.g. during the design stage), they later instigated a number of simple and cost-free rewards and acknowledgements (e.g., hall of fame, user profiles, website acknowledgement, and leader boards) which became effective extrinsic motivators for active and sustained participation. design appropriate organisational relational mechanisms the nla engaged volunteers through both formal and informal channels of communication and progressively gained a better understanding of their motivations and requirements. organisational relational mechanisms (e.g., beta launch, community moderation, and feedback) were also used to reinforce intrinsic and extrinsic motivations. through iterative and progressive version releases, the nla incorporated design elements suggested by volunteers thus creating motivational affordances that aligned volunteer motivations with that of the trove cs project. while moderation and punitive measures can reduce malicious behaviour, they can also have negative effects on participation. hence, the nla chose to put its trust in volunteers and community moderation by making the platform more accessible and not requiring registration. design to build a community wikipedians view their participation on the site as membership in a community where volunteers may adopt new goals and roles, and use different tools such as email, forum (bryant et al. 2005). in the trove case, a sense of community was present through participation in the online forum. there was also evidence of smaller groups forming informally within the site (e.g. lrrsa, climate change group, and rockhampton trovers). although the forum supported creation of sub-groups for respective purposes, these groups were rather loosely coupled. hence, it is useful to facilitate the formation of sub-groups within the wider volunteer cohort supported by appropriate social mechanisms (e.g., promotion of sub-groups) and design features (e.g. ability to send invitation to join sub-groups, a list of active sub-groups). designing to support different forms of contributions yates et al. (2010) and majchrzak and malhotra (2013) in their recent work on virtual cops and organisational wiki distinguish between personal knowledge contribution (i.e., adding content to wiki) and knowledge shaping (i.e., rewriting, integrating and restructuring wiki content). they use the term ‘shaping’ as an activity that changes a knowledge asset without adding domain knowledge. in a similar disposition, glam cs volunteers contributed to not only text correction, but also contributed additional shaping data regarding the digitised newspaper through features such as tagging and commenting. the library provided a forum to facilitate further meaningful conversations around the newspaper data that motivated volunteer engagement. hence while designing cs technology for volunteerism, the designed features should enable and support not just the cs task (e.g., text correction/text transcription/text translation), but also knowledge shaping activities with design features such as tagging, commenting, forums, and lists (zheng et al. 2011). different forms of contributions can be promoted by highlighting the uses of tags, comments and lists on the cs platform along with statistics (see figure 5 on how the library advertises the knowledge shaping activities on its home page). the forum can again provide further illustrations of how different forms of contributions are being sought within the cs system. australasian journal of information systems alam, sun & campbell 2020, vol 24, research article motivation in glam crowdsourcing 20 design for social support cs platforms can provide extended ways of communicating and connecting with the cs service through social networking sites for volunteer engagement and awareness. for example, the trove team decided in 2011 to include a blog (within the trove forum) and twitter account (https://twitter.com/troveaustralia) to promote their collections. the blog was a way to display their collections to the public and to assist with finding new things. links to these social networking sites were visible from the trove home page. 5.4 research limitations and future research the study is exploratory in several respects and has other limitations that warrant comment. first, the research was conducted as a qualitative interpretive single case study. despite the richness and comprehensives, generalisability has long been a shortcoming for single case studies. thus, we encourage future studies to further examine and generalise our findings. for example, gerber and hui (2013) investigated the motivations for both creators and supporters of crowdfunding. future studies should be conducted of cs work in more diverse settings in order to further enrich understanding about volunteer motivation in cs projects (e.g. for-profit company sponsored cs projects, cross-project variables and impacts). in addition, the nationality of volunteers may yield different results (e.g., kobayashi et al. 2015). empirical studies from a greater variety of cultural settings may serve to further explore, validate or highlight new issues and complexities. having distinguished the variances in motives in different stages of cs participation, future research should examine empirically how volunteers move from one stage to the other (crowston and fagnot 2018). based on our findings, we saw that novice contributors are motivated to move to sustained participation through active involvement with the project. future research should examine relational, technical, and organisational factors that explain the dynamics described above (e.g., arazy et al. (2017) examines roles in wikipedia). another significant stream of potential research is the conduct of longitudinal studies to investigate dynamic changes in motivation for collaborative glam cs initiatives to understand better the antecedents of participation and subsequent behavioural influences on sustained involvement. while several important motivational factors have been identified in this study, additional research is required to assess their interdependence, as not all constructs may be monocausal. participation in online communities tends to manifest as a long-tail distribution. that is, a comparatively small, active minority produce most of the content while the majority of community members produce very little (budhathoki and haythornthwaite 2013; halfaker et al. 2013). a similar usage and text correction pattern were observed in the andp project (cf. ayres 2013, hagon 2013). however, studies of contributors to wikipedia and similar online platforms (e.g., amazon mechanical turk) show that top performers behave markedly different to others (halfaker et al. 2013). this study did not investigate all possible forms of participation, but instead concentrated on the leading volunteer group who carried out the majority the text enhancements. future research should consider the motivations of lessproductive contributors in the glam cs context. the relationship between intrinsic and extrinsic motivation at depth and ‘crowding-out’ effects on intrinsic motivations requires further detailed examination (frey and jegen 2001). also needed is an understanding of how different combinations of intrinsic and extrinsic australasian journal of information systems alam, sun & campbell 2020, vol 24, research article motivation in glam crowdsourcing 21 motivations can best work with different problem types (e.g., contest vs. problem solving), and how these motivators affect composition of the crowd (pedersen et al. 2013). vale john campbell john campbell was a professor of business information systems at the australian national university. a major theme throughout his research work is how users interact through information systems in the social world and, in particular, the ways in which organizational decision-making and community interaction are enacted through collaborative technologies. john engaged in a wide range of research topics including it governance and business alignment, user security practices, evaluating it investments, technology and work nexus, and virtual organizations, communities and teams. unfortunately, john passed away on 14th january 2019. john was the first author’s phd supervisor and the second author’s academic mentor. this paper extends the first author’s thesis work. we would like to take this opportunity to acknowledge john’s contribution to the national and international information systems community. references adams, c., and ramos, i. 2010. "the past, present and future of social networking and outsourcing: impact on theory and practice," uk academy for information systems conference proceedings 2010: information systems: past, present and looking to the future. afuah, a., and tucci, c. l. 2012. "crowdsourcing as a solution to distant search," academy of management review (37:3), pp. 355-375. alam, s. l., and campbell, j. 2017. "temporal motivations of volunteers to participate in cultural crowdsourcing work," information systems research (28:4), pp. 744-759. arazy, o., liifshitz-assaf, h., nov, o., daxenberger, j., balestra, m., and cheshire, c. 2017. "on the how and why of emergent role behaviors in wikipedia," proceedings of the 2017 acm conference on computer supported cooperative work and social computing: acm, pp. 2039-2051. ayres, m ( 2013). singing for their supper: trove, australian newspapers, and the crowd. paper presented at the ifla wlic 2013, singapore, 1-9 benkler, y., and nissenbaum, h. 2006. "commons‐based peer production and virtue," journal of political philosophy (14:4), pp. 394-419. brabham, d. c. 2010. "moving the crowd at threadless: motivations for participation in a crowdsourcing application," information, communication & society (13:8), pp. 1122-1145. brabham, d. c. 2012. "motivations for participation in a crowdsourcing application to improve public engagement in transit planning," journal of applied communication research (40:3), pp. 307-328. budhathoki, n. r., and haythornthwaite, c. 2013. "motivation for open collaboration: crowd and community models and the case of openstreetmap," american behavioral scientist (57:5), pp. 548-575. chandler, a. d. 1962. "strategy and structure: chapters in the history of american industrial enterprises," cambridge. hlass.: mit press (14), p. 16. australasian journal of information systems alam, sun & campbell 2020, vol 24, research article motivation in glam crowdsourcing 22 cobb, c., mccarthy, t., perkins, a., bharadwaj, a., comis, j., do, b., and starbird, k. 2014. "designing for the deluge: understanding & supporting the distributed, collaborative work of crisis volunteers," proceedings of the 17th acm conference on computer supported cooperative work & social computing: acm, pp. 888-899. crowston, k., and fagnot, i. 2018. "stages of motivation for contributing user-generated content: a theory and empirical test," international journal of human-computer studies (109), pp. 89-101. deci, e. l. 1971. “effects of externally mediated rewards on intrinsic motivation,” journal of personality and social psychology (18:1), pp. 105-115. deci, e. l., connell, j. p., and ryan, r. m. 1989. “self-determination in a work organization,” journal of applied psychology (74:4), pp. 580-590. deng, x., joshi, k., and galliers, r. d. 2016. "the duality of empowerment and marginalization in microtask crowdsourcing: giving voice to the less powerful through value sensitive design," mis quarterly (40:2), pp. 279-302. doan, a., ramakrishnan, r., and halevy, a. y. 2011. "crowdsourcing systems on the worldwide web," communications of the acm (54:4), pp. 86-96. dutton, w. h. 2010. "networking distributed public expertise: strategies for citizen sourcing advice to government," one of a series of occasional papers in science and technology policy, science and technology policy institute, institute for defense analyses, pennsylvania avenue, washington dc). eisenhardt, k. m. 1989. "building theories from case study research," academy of management review (14:4), pp. 532-550. frey, b. s., and jegen, r. 2001. "motivation crowding theory," journal of economic surveys (15:5), pp. 589-611. gagné, m., and deci, d. 2005. “self-determination theory and work motivation,” journal of organizational behavior (26:4), pp. 331-362. gerber, e. m., and hui, j. 2013. "crowdfunding: motivations and deterrents for participation," acm transactions on computer-human interaction (tochi) (20:6), p. 34. grewal-carr, v., and bates, c. 2016. "the three billion: enterprise crowdsourcing and the growing fragmentation of work," deloitte, retrieved from https://www2.deloitte.com/content/dam/deloitte/de/documents/innovation/us‐cons‐ enterprise‐crowdsourcing‐and‐growing‐fragmentation‐of‐work (20:3). hagon, p. 2013. "trove crowdsourcing behaviour." retrieved 21 november 2019, from https://www.nla.gov.au/our-publications/staff-papers/trove-crowdsourcing-behaviour halfaker, a., keyes, o., and taraborelli, d. 2013. "making peripheral participation legitimate: reader engagement experiments in wikipedia," proceedings of the 2013 conference on computer supported cooperative work: acm, pp. 849-860. holley, r. 2009. many hands make light work: public collaborative ocr text correction in australian historic newspapers. national library of australia. holley, r. 2010. "crowdsourcing: how and why should libraries do it?," d-lib magazine (16:3/4 ma). australasian journal of information systems alam, sun & campbell 2020, vol 24, research article motivation in glam crowdsourcing 23 howe, j. 2006. "the rise of crowdsourcing," wired magazine (14:6), pp. 1-4. karen, s.-y., and cyndi, s. 2011. "social metadata for libraries, archives and museums part 1: site reviews." retrieved 21 november, 2019, from http://www.oclc.org/research/publications/library/2011/2011-02.pdf kaufmann, n., schulze, t., and veit, d. 2011. "more than fun and money. worker motivation in crowdsourcing-a study on mechanical turk," amcis: detroit, michigan, usa, pp. 111. klein, h. k., and myers, m. d. 1999. "a set of principles for conducting and evaluating interpretive field studies in information systems," mis quarterly (23:1), pp. 67-94. kobayashi, m., arita, s., itoko, t., saito, s., and takagi, h. 2015. "motivating multigenerational crowd workers in social-purpose work," proceedings of the 18th acm conference on computer supported cooperative work & social computing: acm, pp. 18131824. lincoln, y. s., and guba, e. g. 1985. "naturalistic inquiry," the blackwell encyclopedia of sociology). majchrzak, a., and malhotra, a. 2013. "towards an information systems perspective and research agenda on crowdsourcing for innovation," the journal of strategic information systems (22:4), pp. 257-268. malone, t. w., laubacher, r., and dellarocas, c. 2010. "the collective intelligence genome," mit sloan management review (51:3), p. 21. mcinnis, b. j., murnane, e. l., epstein, d., cosley, d., and leshed, g. 2016. "one and done: factors affecting one-time contributors to ad-hoc online communities," proceedings of the 19th acm conference on computer-supported cooperative work & social computing: acm, pp. 609-623. nov, o. 2007. "what motivates wikipedians?," communications of the acm (50:11), pp. 60-64. oomen, j., and aroyo, l. 2011. "crowdsourcing in the cultural heritage domain: opportunities and challenges," proceedings of the 5th international conference on communities and technologies: acm, pp. 138-149. orlikowski, w. j., and baroudi, j. j. 1991. "studying information technology in organizations: research approaches and assumptions," information systems research (2:1), pp. 1-28. owens, t. 2013. "digital cultural heritage and the crowd," curator: the museum journal (56:1), pp. 121-130. pedersen, j., kocsis, d., tripathi, a., tarrell, a., weerakoon, a., tahmasbi, n., xiong, j., deng, w., oh, o., and de vreede, g.-j. 2013. "conceptual foundations of crowdsourcing: a review of is research," 2013 46th hawaii international conference on system sciences: ieee, pp. 579-588. rafaeli, s., and ariel, y. 2008. "online motivational factors: incentives for participation and contribution in wikipedia," psychological aspects of cyberspace: theory, research, applications (2:08), pp. 243-267. ridge, m. 2013. "from tagging to theorizing: deepening engagement with cultural heritage through crowdsourcing," curator: the museum journal (56:4), pp. 435-450. australasian journal of information systems alam, sun & campbell 2020, vol 24, research article motivation in glam crowdsourcing 24 rodell, j. b. 2013. "finding meaning through volunteering: why do employees volunteer and what does it mean for their jobs?," academy of management journal (56:5), pp. 1274-1294. rotman, d., hammock, j., preece, j. j., boston, c. l., hansen, d. l., bowser, a., and he, y. 2014. "does motivation in citizen science change with time and culture?," proceedings of the companion publication of the 17th acm conference on computer supported cooperative work & social computing: acm, pp. 229-232. rotman, d., preece, j., hammock, j., procita, k., hansen, d., parr, c., lewis, d., and jacobs, d. 2012. "dynamic changes in motivation in collaborative citizen-science projects," proceedings of the acm 2012 conference on computer supported cooperative work: acm, pp. 217-226. ryan, r. m., and deci, e. l. 2000. "intrinsic and extrinsic motivations: classic definitions and new directions," contemporary educational psychology (25:1), pp. 54-67. saldaña, j. 2015. the coding manual for qualitative researchers. sage. schlagwein, d., and bjørn-andersen, n. 2014. "organizational learning with crowdsourcing: the revelatory case of lego," journal of the association for information systems (15:11), pp. 754-778. starbird, k., and palen, l. 2011. "voluntweeters: self-organizing by digital volunteers in times of crisis," proceedings of the sigchi conference on human factors in computing systems: acm, pp. 1071-1080. taylor, j., and joshi, k. d. 2019. "joining the crowd: the career anchors of information technology workers participating in crowdsourcing," information systems journal (29:3), pp. 641-673. turner, j. c., and patrick, h. 2008. "how does motivation develop and why does it change? reframing motivation research," educational psychologist (43:3), pp. 119-131. von krogh, g., haefliger, s., spaeth, s., and wallin, m. w. 2012. "carrots and rainbows: motivation and social practice in open source software development," mis quarterly (36:2), pp. 649-676. yang, h.-l., and lai, c.-y. 2010. "motivations of wikipedia content contributors," computers in human behavior (26:6), pp. 1377-1383. yates, d., wagner, c., and majchrzak, a. 2010. "factors affecting shapers of organizational wikis," journal of the american society for information science and technology (61:3), pp. 543554. zheng, h., li, d., and hou, w. 2011. "task design, motivation, and participation in crowdsourcing contests," international journal of electronic commerce (15:4), pp. 57-88. copyright: © 2020 alam, sun & campbell. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems alam, sun & campbell 2020, vol 24, research article motivation in glam crowdsourcing 25 doi: https://doi.org/10.3127/ajis.v24i0.2599 paper title (paper title style) australasian journal of information systems talwar et al. 2021, vol 25, research on evaluation a study of mobile payment innovations of mobile applications 1 why have consumers opposed, postponed, and rejected innovations during a pandemic? a study of mobile payment innovations shalini talwar kj somaiya institute of management studies and research, mumbai, india manish talwar alkesh dinesh mody institute for financial and management studies, mumbai, india puneet kaur department of psychosocial science, university of bergen, bergen, norway optentia research focus area, north-west university, vanderbijlpark, south africa gurmeet singh the university of the south pacific, fiji amandeep dhir school of business & law, university of agder, kristiansand, norway norwegian school of hotel management, university of stavanger, stavanger, norway optentia research focus area, north-west university, vanderbijlpark, south africa amandeep.dhir@uia.no abstract the highly infectious nature of the covid-19 virus has made the use of contactless payment methods a health exigency. yet, consumers are resisting using mobile payments (m-payments) during the pandemic, a confounding behavior that needs to be better understood. the present study explicates this behavior by examining consumer resistance to m-payments during the covid-19 pandemic. in addition, it provides more granular findings by measuring three levels of resistance/non-adoption, namely, postponement, opposition, and rejection. in this way, the study adds depth to the literature, which has largely examined resistance at an aggregate level to yield generic findings. toward this end, the study draws upon the innovation resistance theory (irt) to propose that usage, value, risk, tradition, and image barriers influence the three levels of resistance/non-adoption differently. an artificial neural network analysis (ann) of the data collected from 406 non-users of m-payments confirmed that the influence of the five barriers varies for the three levels of resistance/non-adoption. the results further suggest that the usage barrier is the most significant contributor to opposition and rejection intentions toward m-payments, whereas the image barrier is the most influential for postponement intentions. this study thus makes a useful contribution to theory and practice. keywords: consumer resistance, functional and psychological barriers, innovation resistance theory, non-adoption, mobile wallets 1 introduction covid-19 has transcended geographical boundaries with a global pandemic that has an exponentially rising number of confirmed cases and deaths (undp, 2020; who, 2020a). australasian journal of information systems talwar et al. 2021, vol 25, research on evaluation a study of mobile payment innovations of mobile applications 2 countries have responded to the pandemic by introducing various control measures, including complete domestic shutdown or lockdown of activity, social distancing norms, travel restrictions, and provisions of quarantine for those who have been exposed to the virus (fang, nie, & penny, 2020). the transmission routes of covid-19 span not only the spread through infected individuals but also through contaminated surfaces (who, 2020b). therefore, the risk of infection through physical currency, such as banknotes and coins, cannot be ignored, given that the virus can survive on solid surfaces for several days (doremalen et al., 2020). in fact, research during pre-covid-19 times has revealed that human influenza viruses deposited on currency notes could survive and remain infectious for extended periods (thomas et al., 2008). other studies have also confirmed that currency notes can be contaminated with a variety of pathogenic bacteria that transmit infections to the individuals who handle them (abd alfadil, mohamed, ali, & el nima, 2018). given the extremely infectious nature of covid-19 virus (xu et al., 2020), it is pertinent to expect that individuals may worry about its transmission through currency notes, thereby resorting to the use of contactless payment transactions, e.g., mobile wallets (m-wallets). thus, it can be argued that the pandemic presents an opportunity to divert individuals' habit of using cash towards digital modes of transactions instead, an agenda that governments have been pursuing for quite some time (jakhiya, bishnoi, & purohit, 2020). based on the above discussion, we can say that a silver lining in the gloom of the covid-19 pandemic is that, globally, there is a hope that the use of digital payments, which include all non-mobile and mobile payments made through the internet, will exponentially increase as people avoid the use of currency notes for fear of the virus transmission. initial media reports about the increased usage of digital payment modes seem to confirm this expectation. for instance, according to a report in the initial weeks of the pandemic (i.e., march 2020 onward), 42% of individuals in india used different modes of digital payment solutions (both nonmobile and mobile) more than during the pre-pandemic period (et government, 2020). however, a more recent report published in july 2020 suggested that 53% of individuals in india have reported no change in their use of digital payment methods during the postpandemic lockdown period, with only 9% reporting the exclusive use of them (jaganmohan, 2020). these reports have highlighted that the use of digital payment methods may not have increased as much as anticipated during the pandemic. a more worrying thought is whether the reported increase in the usage of digital payment solutions is only temporary and, if so, whether individuals will return to their pre-pandemic transaction behavior, i.e., revert to using currency notes for transactions, after the pandemic ends. this is an important manifestation that service providers facilitating various digital modes of payments, as well as governments and policymakers, want to understand. a comprehensive review of the extant literature has revealed that even before the onset of the covid-19 pandemic, the diffusion of digital payment systems, particularly m-payments, was a global concern in both advanced and emerging economies, despite the ease of interface they offer through smartphones. for example, even in an advanced economy such as japan, 73% consumers had used currency notes for daily transactions in 2019 (fujiki, 2020). in fact, with the exception of china, where the use of m-payments reached 86% in 2019 (yin, gong, guo, & wu, 2019), other countries have continued to see a lower utilization rate. the low diffusion or non-adoption of m-payment usage has attracted the attention of researchers, who have noted the existence of certain barriers that cause individuals to resist the use of m-payments (e.g., australasian journal of information systems talwar et al. 2021, vol 25, research on evaluation a study of mobile payment innovations of mobile applications 3 kaur, dhir, singh, sahu, & almotairi, 2020a; leong, hew, ooi, & wei, 2020; laukkanen, 2016; gupta & arora, 2017), despite the benefits they offer. in this regard, a recent study also examined the existence of barriers toward the use of m-payments for booking travel during the covid-19 pandemic (khanra, dhir, kaur, & joseph, 2021). however, the previous literature providing a resistance perspective is limited, with most of the past studies on mpayments offering adoption-related findings (e.g., chatterjee & bolar, 2019; kaur, dhir, bodhi, singh, & almotairi, 2020b; talwar, dhir, khalil, mohan, & islam, 2020c; gao & waechter, 2017; silic & ruf, 2018). indeed, consumer resistance to digital innovations, in general, continues to be less examined than consumer adoption behavior toward such innovations (talwar et al., 2020a). underscoring the importance of the resistance perspective, ram and sheth (1989) proposed the framework of innovation resistance theory (irt), which explained the resistant behavior of consumers through five barriers: namely, usage, value, risk, tradition, and image. irt uses these barriers to explain the functional and psychological factors that cause consumers to resist a given innovation. extending the resistance debate further, scholars have equated resistance with non-adoption, and proposed its three levels, namely, postponement, opposition, and rejection (e.g., kleijnen, lee, & wetzels, 2009; szmigin & foxall, 1998). such disaggregation of resistance offers deeper insights into the process of consumer decision-making when resisting the adoption of innovations. despite the seminal work to support an understanding of consumers' resistance/non-adoption behavior toward the products and services offered to them, consumer resistance continues to be an under-explored phenomenon at the aggregate level. even fewer studies have examined the antecedents of resistance or non-adoption by disaggregating them into the three levels of postponement, opposition, and rejection, despite scholars' initial interest in examining resistance to mobile and internet banking (elbadrawy & abdel aziz, 2011; laukkanen, 2016; laukkanen, sinkkonen, & laukkanen, 2008). recent studies have also noted the lack of empirical evidence to explicate how different barriers affect the three levels of resistance behaviors (chen, lu, gong, & tang, 2019; talwar et al., 2020a). such a lack of findings, especially in the context of m-payment non-adoption, represents a research gap that needs to be addressed, particularly as scholars have found that the enablers of adoption are not very useful in explaining resistance or non-adoption behavior (claudy, garcia, & o'driscoll, 2015; talwar, dhir, kaur, & mäntymäki, 2020b). we argue that the ongoing pandemic offers an opportunity to examine the resistant consumer behavior toward digital payment methods, in general, and m-payments, in particular, despite the contingent need to use them as an infection avoidance approach. specifically, we posit that the academic research on consumer resistance to m-payments, especially in the context of a pandemic, needs to be deepened in terms of (a) further examination of barriers toward the use of m-payments for transactions, (b) the varying levels of consumer resistance to m-payments, i.e., postponement, opposition, and rejection behaviors, and (c) the resistant behavior of consumers toward m-payments despite the safety benefits they offer in the current pandemic period. responding to this need, we utilized the theoretical framework of irt to examine consumer resistance to m-payments during the pandemic through the following research questions: australasian journal of information systems talwar et al. 2021, vol 25, research on evaluation a study of mobile payment innovations of mobile applications 4 rq1: do usage, value, risk, tradition, and image barriers toward the use of mobile payments (m-payments) during the pandemic impact varying levels of resistance/non-adoption, i.e., postponement, opposition, and rejection? rq2: is there a difference between the relative influence of usage, value, risk, tradition, and image barriers toward the use of m-payments during the pandemic on the varying levels of resistance/non-adoption, i.e., postponement, opposition, and rejection? to test the developed research model, we collected data from 406 non-users of m-wallets (e.g., a form of m-payments) during the present pandemic. we then analyzed the collected data using the artificial neural network (ann) method to uncover the influential barriers and yield robust results, taking both the linear and non-linear relationships of the proposed antecedents with the identified outcome variables. the novelty of the study comes from the following: (a) it is the first empirical study to examine postponement, opposition, and rejection toward m-wallets, which are a very versatile form of m-payments that have remained under-explored from the resistance perspective; (b) it is one of the limited studies to use ann, rather than structural equation modeling, as the methodological approach to analyze consumer resistance to m-payments; and (c) the study has been conducted during the ongoing covid-19 pandemic, thereby providing insights into the payment behavior of consumers during a global health crisis. this area has remained under-researched so far. the rest of the study is structured as follows: we discuss the theoretical background and research model in sections two and three, present the data and methods in section four, followed by the results in section five, the discussion in section six, and the conclusion, implications, limitations, and future research areas in section seven. 2 theoretical background 2.1 innovation resistance theory (irt) innovation resistance as a concept is relatively new compared with the adoption perspective. ram (1987) was the first to discuss the idea that resistance was not the opposite of adoption. other scholars have supported this claim, arguing that not only is innovation resistance an independent attitude, but that resistance can also co-exist with adoption (e.g., oh, park, & min, 2019). irt (ram & sheth, 1989) presents a formal attempt at providing a theoretical framework to capture the sources of resistance that may cause consumers to avoid adopting an innovation. the theory postulates that consumers resist innovations that present potential alternation in the status quo or are misaligned with their beliefs or values. irt proposes two broad classes of barriers, namely, functional, and psychological, as measures of consumer resistance toward innovations. these barriers translate into resistance, thereby implying that the consumers do not accept the innovation. this non-acceptance is, in essence, non-adoption, as discussed in the next section. the three classic functional barriers correspond with usage, value, and risk. usage barriers are related to using the innovation and the adjustments that the consumers need to make to do so based on the complexity or lack of compatibility of the innovation; value barriers, meanwhile, are related to the benefit an innovation gives over its substitutes, as seen from a performanceto-price perspective; and, finally, risk barriers represent the perceived threats, such as australasian journal of information systems talwar et al. 2021, vol 25, research on evaluation a study of mobile payment innovations of mobile applications 5 breaches of privacy and financial risk, that are associated with adopting the innovation (laukkanen, sinkkonen, kivijärvi, & laukkanen, 2007; talwar et al., 2020a). the two classic irt psychological barriers correspond with tradition and image. tradition barriers are related to the consumers' perception of how the offered innovation deviates from the norms of society, thereby requiring them to act against well-entrenched traditions; image barriers, meanwhile, are related to the extent to which the innovation has an unfavorable image in the eyes of the consumer (claudy et al., 2015). although subsequent to the introduction of irt, other barriers have been proposed and examined, such as the vulnerability barrier (e.g., talwar et al., 2020b; talke & heidenreich, 2014). however, we consider only the five generic irt barriers in our study since these appreciably capture the breadth of consumers' resistance to innovation. irt provides a suitable theoretical lens for our study for the following reasons: (a) we propose to examine different levels of resistance in non-users of m-payments, and irt barriers are welldocumented in the research as antecedents of consumer resistance in the context of mobile banking and other related financial services (e.g., kaur et al., 2020a; leong et al. 2020; laukkanen, 2016); (b) irt allows us to understand resistance to innovation as a combination of both attitude and behavior (e.g. bagozzi & lee, 1999), which helps us better explicate varying levels of resistance; (c) scholars have proposed irt and its extended versions as a useful basis for explaining pre-adoption behavior in the area of finance ( e.g., seth, talwar, bhatia, saxena, & dhir, 2020); and (d) irt has been invoked by scholars to examine the nonadoption of digital innovation, yielding robust results and offering deeper insights into the slow or no diffusion of innovations in the digital domain (e.g., soh et al., 2020). 2.2 non-adoption versus resistance toward innovations over time, researchers realized that the enablers of innovation adoption were not very effective in explaining the inhibiting factors that could lead to non-adoption and resistance (e.g., kleijnen et al., 2009; szmigin & foxall, 1998). non-adoption of innovation, as discussed above, is actually the non-acceptance of innovation, which manifests as consumer resistance. there are two streams of thought in the existing literature. the first considers non-adoption and resistance as synonyms and describes them accordingly (e.g., claudy, o'driscoll, garcia, & mullen, 2010). the second stream considers resistance to be a subset of non-adoption (e.g., patsiotis, hughes, & webber, 2013). in the following discussion, we outline the contours of both streams to present a complete picture of the thought process that has evolved over the past few decades in the context of resistance and non-adoption. 2.2.1 non-adoption and resistance as identical concepts one of the earliest mentions of non-adoption of innovation was in 1976 by mittelstaedt, grossbart, curtis, and devere (1976). they identified three key factors as the causes of this non-adoption. first is symbolical rejection, wherein consumers use the information available to decide that the product is not suitable for them; second is that although consumers accept the innovation symbolically, other factors make them unwilling to adopt the innovation; and the third is that although consumers symbolically adopt the innovation and are also willing to accept it, they still postpone the adoption. thus, non-adoption may manifest as one of the three responses of consumers, namely, postponement, opposition, and rejection. australasian journal of information systems talwar et al. 2021, vol 25, research on evaluation a study of mobile payment innovations of mobile applications 6 later, a shift in the focus from the non-adoption of innovation to the resistance toward it occurred (szmigin & foxall 1998), wherein scholars used postponement, opposition, and rejection as three levels of resistance to explain the resistant behaviors of consumers. for instance, claudy et al. (2010) discussed resistance as a consumer response spanning its weak form (postponement) to strong form (rejection), arguing that the factors causing resistance were varied and possibly related to social context, the features of the innovation, and the complex interaction of retail consumers with one another. from this perspective, non-adoption and consumer resistance may be seen as similar concepts. the three terms are described in detail below. postponement is a milder form of non-adoption or resistance that refers to a delay in the decision to adopt by actively deciding to defer adoption for the time being (cornescu & adam, 2013; kuisma, laukkanen, & hiltunen, 2007). it represents a situation in which consumers prefer to wait despite finding the innovation acceptable (gatignon & robertson, 1989). furthermore, postponement is largely attributable to situational factors (poon, 2008). interestingly, nabih, bloem, and poiesz (1997) argued that postponement offers an escape route for consumers when faced with the dilemma of adopting the innovation or not. a similar view was posited by gurtner (2014), who argued that postponement is a temporary manifestation that is associated with a passive denial to immediately accept or reject the innovation. opposition represents an active attack or protest directed toward an innovation (kuisma et al., 2007; kleijnen et al., 2009). scholars have argued that while opposition is a form of rejection, it is not an outright one since consumers may be willing to evaluate the innovation further or collect more information about it before actually rejecting it (cornescu & adam, 2013; hosseini et al., 2016). presenting a more aggressive picture of opposition, gurtner (2014) defined it as overt behavior with an objective to prevent others from accepting an innovation. notably, opposition is a much stronger response than postponement and is attributable to factors, such as habits, situations, a perceived disadvantage, or cognitive style (kleijnen et al., 2009; poon, 2008). rejection is the strongest form of non-adoption. it is usually seen in the case of innovations that consumers feel have nothing worthwhile to offer, causing a straightforward refusal to accept them (patsiotis et al., 2013; talwar et al., 2020a). thus, rejection implies the outright, direct, and active non-adoption of an innovative offering (cornescu & adam, 2013; kleijnen et al. 2009; lian & yen, 2013). the possible reasons behind rejection could be the complexity, riskiness, or lack of value offered by the innovation (szmigin & foxall, 1998), or else the consumer's perception that it is not appropriate for them, based on the existing information (szmigin & foxall, 1998). rejection may also occur due to inertia on the part of consumers (ganiere, chern, hahn, & chiang, 2004), their conservatism (hirschheim & newman, 1988), or their tendency to adhere to the status quo (woodside & biemans, 2005). furthermore, rejection can be passive or active, with passive rejection being a situation where the innovation is not even considered, and active rejection implying a situation where it is considered but ultimately rejected (pousttchi & schurig, 2004). adding one more perspective, kuisma et al. (2007) argued that opposition could transform into rejection or adoption over time. similarly, cornescu and adam (2013) posited that postponement could also turn into adoption or rejection eventually. australasian journal of information systems talwar et al. 2021, vol 25, research on evaluation a study of mobile payment innovations of mobile applications 7 2.2.2 resistance as a subset of non-adoption presenting a different view from the one discussed above, chen et al. (2019) distinguished between non-adoption and resistance. in this context, non-adopters may be seen as consumers who do not adopt an innovation offered in the market (dickerson & gentry, 1983), and resistors may be seen as postponers who may be delaying the decision until a more suitable time (laukkanen et al., 2008). in addition, resistors could also be opponents with a negative attitude toward the offered innovation (kleijnen et al., 2009) or rejecters who have made a decision not to adopt the innovation (laukkanen et al., 2008). patsiotis et al. (2013) also endorsed the view that non-adoption is different from resistance, arguing that while the nonadoption of an innovation may be attributable to a high degree of resistance, there are other factors that may lead to this outcome. such factors could include a lack of need, low awareness, or inertia (suzuki &williams, 1998; patsiotis et al., 2013). in fact, resistance may be best looked at as a subset of non-adoption (patsiotis et al., 2013). 2.2.3 mapping non-adoption and resistance to the present context the preceding discussion provides the basis for considering non-adoption and resistance as identical concepts, which manifest toward m-payments in three levels, namely postponement, opposition, and rejection. in consonance with kleijnen et al. (2009), we interpret postponement as the decision of the consumers to wait to adopt any innovation, opposition as the aggressive objection of it, and rejection as the outright refusal to accept it based on their cognitive evaluation of it. drawing on the study by mohtar and abbas (2015), we further consider postponement, opposition, and rejection as three forms of active resistance. to better explicate the three levels, we use the five classic irt barriers as antecedents of these manifestations toward m-payment usage during the pandemic. figure 1. non-adoption versus adoption (naa) framework since the research on the influence of barriers on the three levels of resistance/non-adoption is still in its nascent stages, we propose a transition framework, titled the non-adoption versus adoption (naa) framework, presented in figure 1, to provide conceptual clarity about the dynamics of the barriers and the levels of resistance/non-adoption and adoption. the australasian journal of information systems talwar et al. 2021, vol 25, research on evaluation a study of mobile payment innovations of mobile applications 8 framework represents the essence of the preceding discussion and posits two outcomes. outcome a indicates the non-acceptance of innovation in the absence of any interventions to engage with potential resistors. it implies that the five irt barriers may lead to three levels of resistance, which are actually manifestations of non-adoption (e.g., mittelstaedt et al., 1976; szmigin & foxall, 1998; claudy et al., 2010). however, there is also a possibility that nonadoption, particularly postponement and opposition, may transform into adoption, as argued by past studies (i.e., kuisma et al., 2007; cornescu & adam, 2013). this possibility is captured in outcome b, wherein we posit that the interventions introduced by service providers can cause the postponers, opposers, and rejectors to change their minds and adopt the innovation. 3 research model as illustrated in figure 2, we examine the relative influence of irt barriers (i.e., usage, value, risk, tradition, and image) on three levels of resistance/non-adoption (i.e., postponement, opposition, and rejection) in the context of m-payment use during the pandemic. the review of the extended literature revealed that scholars have organized similar investigations in different contexts, such as online shopping (lian & yen, 2013), mobile health applications (gurtner, 2014), mobile banking (laukkanen, 2016), brand mobile apps (chen et al., 2019) and convergence products (park & koh, 2017). figure 2. research model chen et al. (2019) revealed differences in the effect of three irt barriers, i.e., usage, image, and value, on the three levels of resistance toward brand mobile apps. specifically, usage and image barriers significantly affected rejection, while value barriers significantly affected postponement and opposition. park and koh (2017) similarly identified differences in the antecedents of rejection and postponement, finding that rejection was associated with the australasian journal of information systems talwar et al. 2021, vol 25, research on evaluation a study of mobile payment innovations of mobile applications 9 expectation of higher quality coupled with lower price, while postponement was associated with the expectation of lower price. the study also found that the perceived pace of technology change indirectly affected rejection and postponement. in another digital context, laukkanen et al. (2008) examined the effect of the five irt barriers on the three non-adopter groups in the context of their usage intentions toward internet banking: postponers, opponents, and rejectors. the outcome of their empirical analysis confirmed significant statistical differences among the three non-adopter groups in terms of the irt barriers. specifically, these groups differed in the degree of the usage, value, tradition, and image barriers they had toward internet banking. furthermore, the study revealed that the differences were highest for usage barriers followed by those for image, value, and tradition. moreover, the three groups did not differ in terms of risk barrier as the risk perception for internet banking was high across all three categories. elbadrawy and aziz (2011) similarly classified the non-adopters of mobile banking into postponers, opponents, and rejectors. their study examined the effect of the five irt barriers on these non-adopters, revealing that the three groups differed significantly with regard to usage, value, and image barriers. in comparison, no statistically significant differences were found between the groups regarding the risk and tradition barriers. consolidating these findings, laukkanen (2016) examined adopters versus non-adopters in the context of mobile banking and internet banking. the results showed that the value barrier was the most statistically significant barrier toward the adoption of both banking methods. moreover, the image barrier was found to slow down the adoption of mobile banking, while the tradition barrier led to the rejection of internet banking. a limited number of studies have examined the three levels of resistance or non-adoption in other digital contexts. for instance, gurtner (2014) examined the usage, value, risk, and tradition barriers as antecedents of rejection and opposition toward mobile health applications. the study measured the risk barrier through physical, performance, and social risk and found differences in the effect of these barriers on rejection and opposition. whereas rejection was significantly predicted by usage, value, and performance risk barriers, opposition was predicted by physical and performance risk instead. similarly, lian and yen (2013) used irt to examine resistance to online shopping in the context of a cosmetic company. the empirical findings indicated that value and tradition barriers toward usage intentions were the major cause of resistance. the results also revealed that the non-adopter groups differed in terms of usage and image barriers, such that rejectors were found to have the highest barriers, followed by opponents and postponers. in sum, significant differences in usage, value, tradition, and image barriers were found between adopters and non-adopters in the context of online purchase of experience goods. the above discussion confirms that examining non-adoption by breaking it down into three levels is quite informative and can help theorists and managers develop more effective strategies to counter resistance in the form of irt barriers. given that non-adoption can impede the diffusion of innovations (talwar et al., 2020a), we contend that the academic research related to the three levels of resistance/non-adoption is limited not only in terms of the depth of findings but also in terms of the variety of digital innovations examined. due to this, we expect that examining the three levels of resistance/non-adoption in the context of mpayments during the pandemic will further enrich the accumulated knowledge along with providing useful insights for researchers and service providers in the area. thus, we propose australasian journal of information systems talwar et al. 2021, vol 25, research on evaluation a study of mobile payment innovations of mobile applications 10 to examine the effect of five classic irt barriers on the three levels of resistance/non-adoption to uncover their differential influence. 4 data and methods the online data collection method is quite popular in the consumer behavior literature (e.g., sharma et al., 2020a; sharma, singh, & pratt, 2020b). as such, we collected the data for the study through an online survey conducted in india during august 2020. the survey link was shared on different social media platforms, including facebook and whatsapp. we followed a robust process to prepare the survey questionnaire. first, informal interviews were conducted with seven participants representing the target user group of non-users of mpayments and three experts (two professors and one market researcher). the purpose of these interviews was to evaluate the content and face validity of the preliminary questionnaire, which had been prepared by adapting pre-validated, multi-item scales, to assess the relevance, semantics, and clarity of each item included. profile frequency % gender female 161 39.66 male 245 60.34 age group 25 years and below 74 18.23 26-35 years 113 27.83 36-45 years 105 25.86 46-55 years 95 23.40 56 and above 19 4.68 monthly income (inr) 25000 or less 16 3.94 2500150000 116 28.57 50001-75000 172 42.36 75001-100000 57 14.04 more than 100000 45 11.08 educational background pursuing/completed graduate degree 169 41.63 pursuing/completed a post-graduate degree 208 51.23 pursuing/completed doctorate (phd or equivalent) 29 7.14 household size single 54 13.30 married but no children 93 22.91 married and one child 221 54.43 married and more than one child 38 9.36 table 1. socio-demographic profile of respondents we used the input from the interviews to modify the items and prepare the final survey instrument. all items were measured on a 5-point scale varying from '1' = strongly disagree australasian journal of information systems talwar et al. 2021, vol 25, research on evaluation a study of mobile payment innovations of mobile applications 11 and '5' = strongly agree. all five barriers were measured through a 5-item scale, with the exception of the value barrier, which was measured through a 4-item scale adapted from kaur et al. (2020a), laukkanen (2016), and talwar et al. (2020b). the three levels of resistance/nonadoption were measured through a 5-item scale for postponement and opposition and a 4item scale for rejection, which was adapted from chen et al. (2019) and park and koh (2017). the first page of the online survey clarified the study objectives, the preservation of anonymity, and the absence of an incentive for responding to the survey questions. we recruited respondents using three screening criteria: (a) they should be non-users of mwallets (representing m-payments) before and during the covid-19 pandemic, (b) they should possess a smartphone and have experience using mobile apps, and (c) they should be aware of different m-payment methods. india was selected as the geography of interest since scholars have noted that despite the recent push by the indian government to promote mpayment use (jakhiya, bishnoi, & purohit, 2020), the usage of such methods, particularly mwallets, has not increased much (talwar, talwar, najmul islam, & dhir, 2020e). for instance, only 6% of indians use m-wallets compared with 36% of users in china (bepari, 2019). resistance toward m-payments has continued to manifest even during the pandemic (khanra et al., 2021), despite the health benefit of using contactless payment methods. given that india has a large number of smartphone users and significantly improved internet connectivity, the reasons for non-adoption of m-payments go beyond infrastructural issues. there is thus a need to understand this phenomenon better to help the regulators and service providers develop and implement strategies to lower such resistance. we received 450 responses, and the final dataset of 406 was taken forward for further analysis after removing 44 invalid responses. table 1 presents the socio-demographic profile of the respondents in terms of their age, gender, monthly income, educational background, and household size. we analyzed the data using the ann method since it was the most suitable approach, given the sample size and multivariate characteristics of the data collected. ann is a novel approach that yields robust results, yet it is less utilized as compared to covariance-based structural equation modeling (cb-sem), which is more commonly used (talwar, dhir, kaur, zafar, & alrasheedy, 2019; kumar, dhir, talwar, chakraborty, & kaur, 2021). 5 results 5.1 data characteristics we assessed the four multivariate assumptions of normality, homoscedasticity, linearity, and multicollinearity to better understand the nature of the data under study. first, we conducted the kolmogorov-smirnov (k-s) test to check the distribution of the data, per the recommendation of scholars (e.g., simard & l'ecuyer, 2011). the results revealed that the data followed a non-normal distribution. since cb-sem requires the data to be normally distributed, ann was more suited for analysis in the present study. although ann has no homoscedasticity or heteroscedasticity restrictions, we also examined the data for homoscedasticity to better understand its nature. of the three dependent variables, the data for rejection appeared to be homoscedastic, while it was heteroscedastic for postponement and opposition. this was visually confirmed by the scatter plot of the residuals and three dependent variables (figures 3-5). australasian journal of information systems talwar et al. 2021, vol 25, research on evaluation a study of mobile payment innovations of mobile applications 12 in consonance with recent studies (talwar, talwar, kaur, & dhir, 2020f; hew, leong, tan, ooi, & lee, 2019), we conducted an anova test and found that all three dependent variables (postponement, opposition, and rejection) had a non-linear relationship with the usage, value, and image barriers. opposition and rejection had a non-linear relationship with the risk barrier as well. furthermore, all three dependent variables had a linear relationship with the tradition barrier. although ann is suitable for both linear and non-linear relationships, the existence of nonlinear relationships further supported the use of ann in the present study. finally, we examined the data for the issue of multicollinearity, as suggested by recent studies (talwar et al., 2019; talwar et al., 2020e). for all three dependent variables, the values of tolerance and variance inflation factor (vif) were within the recommended limits, with tolerance being more than 0.1 and vif being less than 5 (hair, black, babin, & anderson, 2016). figure 3. test of homoscedasticity/heteroscedasticity (postponement) figure 4. test of homoscedasticity/heteroscedasticity (opposition) australasian journal of information systems talwar et al. 2021, vol 25, research on evaluation a study of mobile payment innovations of mobile applications 13 figure 5. test of homoscedasticity/heteroscedasticity (rejection) 5.2 common method variance (cmv) since the data was collected using a single survey instrument, it might have the issue of common method variance (cmv), although we followed all procedural remedies at the instrument development and data collection stages. therefore, we used harman's single factor test to rule out the possibility of cmv (podsakoff, mackenzie, lee, & podsakoff, 2003) and found that a single factor explained only 41.06% of the total variance explained, which is below the required cut-off of 50%. 5.3 validity and reliability of the survey instrument to evaluate the validity and reliability of the survey instrument, we conducted a confirmatory factor analysis. first, cronbach's alpha and the composite reliability (cr) values for all constructs were more than 0.7, confirming the reliability of the survey instrument. next, the average variance extracted (ave) value exceeded 0.5 for all constructs, thereby demonstrating their convergent validity. finally, since the square roots of aves were higher than the corresponding correlations, we concluded that the instrument had discriminant validity as well. the inter-construct correlations were also less than 0.8, as recommended. thus, all of the validity and reliability measures that we examined conformed to the recommended cut-offs values indicated by prior studies (hair et al., 2016; kumar et al., 2021; talwar, dhir, kaur, & mäntymäki, 2020d). table 2 presents the specific values of the validity and reliability measures. 5.4 predictive accuracy of the ann models we generated the alternate ann models using five input neurons, two hidden neurons, and three output neurons. in consonance with prior studies (e.g., talwar et al., 2020f; hew et al., 2019; leong et al., 2020), we confirmed the predictive accuracy of the ann models by assessing the root mean square errors (rmse). the models had a high accuracy of prediction in the case of postponement, with the mean rmse of training data equal to 0.2004 and that of the validation data being equal to 0.2149. the predictive accuracy of the models was also high in the case of opposition and rejection, with the mean rmse of training data equal to 0.1748 and 0.1970, respectively, and that of the validation data equal to 0.1794 and 0.2097. australasian journal of information systems talwar et al. 2021, vol 25, research on evaluation a study of mobile payment innovations of mobile applications 14 α ave cr ub vb rb tb ib post opp rej ub 0.92 0.72 0.93 0.85 vb 0.85 0.60 0.86 0.75 0.77 rb 0.90 0.71 0.92 0.59 0.57 0.84 tb 0.82 0.57 0.85 0.51 0.55 0.38 0.75 ib 0.81 0.50 0.82 0.65 0.67 0.57 0.53 0.71 post 0.92 0.68 0.92 0.52 0.57 0.41 0.41 0.60 0.82 opp 0.89 0.62 0.90 0.70 0.69 0.56 0.52 0.66 0.64 0.79 rej 0.76 0.58 0.80 0.62 0.54 0.39 0.45 0.45 0.40 0.57 0.76 table 2. validity and reliability statistics α = cronbach's alpha; cr = composite reliability; ave = average variance extracted; ub = usage barrier; vb = value barrier; rb= risk barrier; tb = tradition barrier; ib = image barrier; post = postponement; opp = opposition; rej = rejection bold values in diagonal are the square root of ave, and the off-diagonal values are correlations (significant at p < 0.05) 5.5 sensitivity analysis finally, we conducted a sensitivity analysis to determine the relative influence of the five barriers on the three levels of resistance/non-adoption. toward this end, we calculated their normalized importance by expressing each value as a percentage of the maximum relative importance. as presented in the heat map in figure 6, the influence of the five barriers on the three levels of resistance/non-adoption differed. figure 6. heat map of the relative influence of the irt barriers on resistance/non-adoption 6 discussion of findings our study revealed that all five barriers influenced postponement, opposition, and rejection intentions toward m-payments during the covid-19 pandemic. fig. 7 presents the values, direction, and linearity/non-linearity of the relationships uncovered by our analyses. the finding related to the differences in how the classic irt barriers influence the three levels of resistance/non-adoption is in consonance with prior studies in different contexts (elbadrawy australasian journal of information systems talwar et al. 2021, vol 25, research on evaluation a study of mobile payment innovations of mobile applications 15 & aziz, 2011; gurtner, 2014; laukkanen et al., 2008; laukkanen, 2016; lian & yen, 2013). the differences in the effect of each individual barrier on the three levels of resistance are discussed below. figure 7. results of sensitivity analysis usage barrier and resistance/non-adoption: based on the values of normalized relative importance, the influence of the usage barrier was highest on opposition and rejection, followed by postponement (figures 6 and 7). moreover, the relationship was non-linear and positive (based on the correlation value in table 2) for all three dependent variables. this implies that consumers develop a strong feeling of opposition and even rejection toward such services due to issues, including difficulty in handling the apps, slow interface, the tedious process of using them to send or receive payments, and the complicated information and data entry requirements. such issues would also cause some individuals to postpone using mpayments while they seek more information. however, the influence of the usage barrier on postponement intentions is very small, indicating that issues with handling and interface are unlikely to cause consumers to postpone using such methods for transactions. value barrier and resistance/non-adoption: the influence of the value barrier on opposition and postponement intentions was almost same, as seen from the values of normalized relative importance in figures 6 and 7. moreover, the relationship was positive and non-linear. this finding implies that if consumers feel that m-payment methods do not offer any additional advantages for sending and receiving payments compared with other methods, such as cash, card, and internet banking, they are likely to postpone or even oppose the use of these methods. similarly, limits on the amount of money that can be transferred in a day, no enhancement in the ability to control financial matters, and limited promotional benefits, such australasian journal of information systems talwar et al. 2021, vol 25, research on evaluation a study of mobile payment innovations of mobile applications 16 as a discount or cashback, influence the intentions to postpone or oppose the use of these methods for sending and receiving payments during a pandemic. finally, the influence of the value barrier on rejection intentions was quite small, indicating that the lack of value offered is unlikely to lead consumers to completely reject m-payment methods for sending and receiving payments during the pandemic. this relationship was also positive and non-linear (table 2 and figure 7). risk barrier and resistance/non-adoption: surprisingly, the risk barrier had the lowest relative influence of all three levels of resistance/non-adoption, as seen from the value of its normalized relative importance (figure 6). moreover, the relationship was positive and linear in the case of postponement and positive and non-linear in the case of opposition and rejection (table 2 and figure 7). these results imply that the fear of transferring money to the wrong contact, paying extra amounts, being debited multiple times, and failed transactions have a relatively low influence in causing consumers to postpone, oppose, or reject the use of these methods. consequently, the issues related to the safety of making a mobile-based payment may not cause consumers to delay using them, actively criticize them, or try to prevent others from using them. tradition barrier and resistance/non-adoption: based on the value of normalized relative importance, as presented in the heat map in fig. 6, the tradition barrier had a maximum influence on opposition intentions, followed by some influence on rejection intentions and comparatively low influence on postponement intentions. moreover, the relationship was positive and linear for all three levels of resistance/non-adoption (table 2 and figure 7), suggesting that if consumers are more comfortable with cash transactions for sending and receiving payments and prefer physically going to banks to make money transfers, they are also likely to actively attack or protest the use of m-payments and discourage others from using them. moreover, such consumers would oppose or even reject the use of m-payment methods on the grounds that they are out of sync with the well-entrenched ways of sending and receiving payments. image barrier and resistance/non-adoption: finally, the image barrier was the most important barrier for postponement intentions, followed by opposition intentions and rejection intentions, as indicated by the normalized relative importance values presented in figure 6. furthermore, the relationship was positive and non-linear for all three levels of resistance/nonadoption (table 2 and figure 7), which indicates that consumers who feel that using mpayments for sending and receiving payments is too difficult and complicated are likely to delay the use of these methods. similarly, if the consumers feel that m-payments have frequent instances of transaction failures and require a lot of work, such as linking a bank account and mobile number, they may delay the use of these methods. image barrier had a similar effect on opposition intentions, indicating that non-users not only think that m-payments are not useful, but they also tell their social circle not to use these methods for sending and receiving payments. in comparison, the influence of the image barrier on rejection was quite low. differences among three levels of resistance/non-adoption: as anticipated, the relative influence of these five barriers on the three levels of resistance/non-adoption varied. specifically, the image barrier was the most influential barrier in the case of postponement, while the usage barrier was the most influential barrier for opposition and rejection. the other barriers influencing postponement, in decreasing order of relative importance, were value, usage, risk, and australasian journal of information systems talwar et al. 2021, vol 25, research on evaluation a study of mobile payment innovations of mobile applications 17 tradition. in the case of opposition, the other influential barriers were, from most to least important, the image, tradition, value, and risk barriers. finally, in the case of rejection, the other influential barriers, again in decreasing order of importance, were tradition, value, image, and risk barriers. 7 conclusion, implications, limitations, and future research areas the adoption/non-adoption of mobile-based payments during the pandemic has varied nuances that need to be understood by the academic researchers as well as the service providers. since the covid-19 pandemic is a temporary aberration that is expected to end with the coming of a vaccine or the success of governments in controlling its further spread, an investigation of the depth and breadth of mobile based-payment non-adoption/adoption behavior during the pandemic is important for generating consumer behavior insights from a long-term perspective. the present study thus attempted to examine the effect of barriers toward these payment methods on varying levels of resistance. the study objectives were formally stated through two research questions. rq1 proposed to examine the influence of usage, value, risk, tradition, and image barriers toward the use of m-payments during the pandemic to assess the impact of varying levels of resistance/non-adoption, i.e., postponement, opposition, and rejection. rq2 sought to confirm if there were differences between the relative influence of usage, value, risk, tradition, and image barriers toward the use of m-payments during the pandemic on the varying levels of resistance/non-adoption, i.e., postponement, opposition, and rejection. we proposed the conceptual model to respond to these research questions on the basis of irt (ram & sheth, 1989) and analyzed data collected from 406 smartphone users who had not yet adopted m-payment methods using ann. the results of the study indicated that all five barriers, namely, usage, value, risk, tradition, and image, impacted the varying levels of resistance/non-adoption, i.e., postponement, opposition, and rejection, to using m-payments during the pandemic. evaluating the alternative ann models further confirmed that the barriers differed in their relative influence on consumers' postponement, opposition, and rejection of such services. specifically, the usage barrier was the most important barrier for opposition and rejection intentions, whereas the image barrier was the most influential barrier for postponement intentions. the other influential barriers impacting postponement were value, usage, risk, and tradition. in the case of opposition, the other influential barriers were image, tradition, value, and risk. finally, in the case of rejection, the other influential barriers were tradition, value, image, and risk barriers. the study has several interesting implications for theory and practice. 7.1 theoretical implications the study offers four key theoretical implications. first, it contributes to an improved understanding of resistance and non-adoption. there is a conflict in the existing literature regarding whether the two are the same or different (cornescu & adam, 2013; chen et al., 2019; patsiotis et al., 2013). this study presents the opposing view and attempts to find the sources of similarity and differences in these two conceptualizations. by doing so, the study sets the groundwork for a more nuanced debate centered on resistance versus non-adoption. australasian journal of information systems talwar et al. 2021, vol 25, research on evaluation a study of mobile payment innovations of mobile applications 18 second, the study provides empirical evidence that enhances the knowledge of consumer behaviour toward m-payments in three ways: (a) by bringing forth the fact that resistant behaviors need as much focus as adoption behaviors to explain consumer response to these payment methods; (b) by shifting focus to a less developed but more realistic idea that consumer resistance should not be looked at in an aggregate manner alone but by dividing it into varying levels of resistance, as argued by recent studies in varying contexts (e.g., chen et al., 2019; park & koh, 2017); and (c) by underscoring the fact that classic irt (ram & sheth, 1989) barriers (usage, value, risk, tradition, and image) are useful in explaining the varying levels of resistance/non-adoption of m-payments and that, furthermore, the relative influence of each barrier on postponement, opposition, and rejection is different. third, the study adds a new perspective to the growing irt literature by extending it to examine consumer behavior in an unprecedented health crisis like the covid-19 pandemic. since the governmental pandemic control measures, such as complete domestic shutdown and social distancing (fang et al., 2020), have severely impacted the psychological processes of consumers, utilizing irt to examine consumer resistance in this context can stimulate the examination of consumers' general resistant behavior during the pandemic. finally, this is the first empirical study to apply an artificial neural network (ann) approach to evaluate the influence of irt barriers on varying levels of resistance/non-adoption. ann is a sophisticated data analysis approach that is not restricted by the various multivariate assumptions of normality, multicollinearity, homoscedasticity, and linearity. in fact, it allows the examination of both linear and non-linear relationships among the independent and dependent variables (talwar et al., 2020f). moreover, it presents a clear view of the relative influence of the independent variables on the dependent variables. this helps generate more robust and insightful inferences. thus, our study contributes novelly to the methodological advancement of research on consumer resistance to m-payments. 7.2 practical implications the study offers four main implications for practice. first, it advances the idea that approaching the non-adoption of m-payments in an aggregate manner might not be helpful to service providers since their strategies are likely to be more generic. an understanding of the varying levels of consumer resistance in terms of postponement, opposition, and rejection is extremely important because it can assist service providers in developing counterstrategies and timely interventions. the finding that the influence of the five classic irt barriers varies for different levels of resistance indicates that managers should try to understand the extent of consumer resistance and use tailor-made interventions for each class of resistors, as supported by scholars (e.g., talwar et al., 2020a). for example, a 'call to action' type of communication or repeated follow-ups may act as a pull strategy for postponers, but not for those who actively oppose innovations. second, the findings of our study show that opponents and rejectors are most affected by usage barriers. due to this, we suggest that m-payment service providers focus specifically on having a robust architecture in place for simple and fast processing of transactions as well as ensuring that their communication with potential users, both personally and through mass media platforms, emphasizes (a) ease of handling, (b) convenience, (c) a fast interface, and (d) the limited data entry requirements of their payment interface. this is in consonance with prior australasian journal of information systems talwar et al. 2021, vol 25, research on evaluation a study of mobile payment innovations of mobile applications 19 findings on the importance of communication to overcome consumer resistance (e.g., mani & chouk, 2017; 2018). third, image barriers are the highest among the postponers of m-payment usage, which implies that managers should identify the specific reasons why individuals perceive that using m-payments is too difficult and has frequent instances of transaction failures. this realization is important because postponers are a resistor group who are not only temporarily delaying adoption (kuisma et al., 2007) but are also equally likely to reject the use of m-payments with time (cornescu & adam, 2013). to counter this issue, service providers may develop advertisements with a storyline that underscores how m-payments benefit both society and individuals, how they make life easy, and how vulnerable populations, such as women, the elderly, and those suffering from medical issues, can have an easier and safer payment transaction interface. the backdrop of the covid-19 pandemic can be used to further strengthen the narrative that contactless payment methods are a natural progression to protect oneself from infection and, furthermore, are a simple way to complete financial transactions. if properly executed, such efforts to reduce postponement can be rewarding for the service providers since scholars have argued that with time and information, postponement can transform into adoption (kuisma et al., 2007). finally, our findings reveal that opposition is highly influenced by usage barriers as well as image (91.63%) and tradition barriers (75.83%). the existence of these barriers is thus likely to exacerbate opposition intentions and cause difficulties for service providers. this is particularly concerning because opposers not only aggressively challenge the innovation but also try to prevent others from accepting it (claudy et al., 2015; gurtner, 2014). however, opposers can still potentially adopt the innovation with time (kuisma et al., 2007). managers should thus look beyond lowering the usage and image barriers alone to identify the reasons why individuals perceive m-payments to be against their norms or well-entrenched practices; in turn, this knowledge will guide them in developing strategies to present m-payment methods as a healthier option for transactions, given the risk of transmission associated with currency notes (abd alfadil et al., 2018). 7.3 limitations and future research areas despite its novel approach and contributions, the study presents certain limitations. first, the data was collected from a single geography using a self-report survey instrument. thus, the study is likely to suffer from socially desirable responses, and the findings may lack generalizability. however, the study sets the foundation for future researchers to examine the varying levels of resistance among consumers and the influence of irt barriers on these manifestations. researchers can thus expand our model to different contexts by identifying product-specific barriers, as done by some recent studies (e.g., talwar et al., 2020b). second, our study used ann for data analysis, which is a robust method for uncovering the relative influence of antecedents on outcome variables, as posited by recent studies (e.g., hew et al., 2019; leong et al., 2020; talwar et al., 2020f). however, ann does not support hypothesis testing. due to this, the findings of our study offer an interesting but limited perspective. future studies can apply the more commonly used cb-sem to test the related hypotheses and offer a more granular understanding of the dynamics. third, from a conceptual perspective, our model is limited to testing the influence of each barrier separately on the postponement, opposition, and rejection of m-wallets. however, past australasian journal of information systems talwar et al. 2021, vol 25, research on evaluation a study of mobile payment innovations of mobile applications 20 studies have posited postponement and opposition as intermediate stages that may transform into rejection or adoption (cornescu & adam, 2013; kuisma et al., 2007). as such, we suggest that scholars should develop models that capture this transition and consider various mediating and moderating influences that may affect the underlying associations. furthermore, researchers can examine our model in the post-pandemic period and compare the differences in the relative importance of barriers. if any differences are uncovered, it might be possible to identify specific barriers that are heightened in the presence of external stressors, such as a pandemic. references abbas, m., nawaz, m .s., ahmad, j., & ashraf, m. (2017). the effect of innovation and consumer-related factors on consumer resistance to innovation. cogent business & management, 4(1), 1312058. abd alfadil, n. a., mohamed, m. s., ali, m. m., & el nima, e. a. i. (2018). characterization of pathogenic bacteria isolated from sudanese banknotes and determination of their resistance profile. international journal of microbiology, 2018, 1-7. bagozzi, r. p., & lee, k. h. (1999). consumer resistance to, and acceptance of, innovations. advances in consumer research, 26(12), 218-225. bepari, s. (2019). the number of mobile wallet users grows by 140 million per year. retrieved october 23, 2020 from: https://www.fintechnews.org/the-number-of-mobile-walletusers-grows-by-140-million-per-year/ chatterjee, d., & bolar, k. (2019). determinants of mobile wallet intentions to use: the mental cost perspective. international journal of human-computer interaction, 35(10), 859-869. chen, q., lu, y., gong, y. (yale), & tang, q. (2019.). why do users resist service organization's brand mobile apps? the force of barriers versus cross-channel synergy. international journal of information management, 47, 274-282. claudy, m. c., garcia, r., & o'driscoll, a. (2015). consumer resistance to innovation—a behavioral reasoning perspective. journal of the academy of marketing science, 43(4), 528544. claudy, m., o'driscoll, a., garcia, r., & mullen, m. (2010). consumer resistance to green innovations: developing a new scale and an underlying framework. in 35th macromarketing conference, wyoming, usa. cornescu, v., & adam, c. r. (2013). the consumer resistance behavior towards innovation. procedia economics and finance, 6, 457–465. doremalen, n. v., morris, d. h., holbrook, m. g., gamble, a., williamson, b. n., tamin, a., harcourt, j. l., thornburg, n. j., gerber, s. i., lloyd-smith, j. o., wit, e. d., & munster, v. j. (2020). aerosol and surface stability of sars-cov-2 as compared with sars-cov1. the new england journal of medicine, 382(16), 1-4. dickerson, m. s., & gentry, j. w. (1983). characteristics of adopters and non-adopters of home computers. the journal of consumer research, 10(2), 225-235. etgovernment. (2020). digital payments swell as 42% indians make multiple online payments during covid-19 lockdown. retrieved october 23, 2020, from australasian journal of information systems talwar et al. 2021, vol 25, research on evaluation a study of mobile payment innovations of mobile applications 21 https://government.economictimes.indiatimes.com/news/digital-payments/digitalpayments-swell-as-42-indians-make-multiple-online-payments-during-covid-19lockdown/75172943. elbadrawy, r., & aziz, r. a. (2011). resistance to mobile banking adoption in egypt: a cultural perspective. international journal of managing information technology, 3(4), 9-21. fang, y., nie, y., & penny, m. (2020). transmission dynamics of the covid-19 outbreak and effectiveness of government interventions: a data-driven analysis. journal of medical virology, 92(6), 645–659. fujiki, h. (2020). the use of noncash payment methods for regular payments and the household demand for cash: evidence from japan. the japanese economic review, 71, 719-765. gao, l., &waechter, k. a. (2017). examining the role of initial trust in user adoption of mobile payment services: an empirical investigation. information systems frontiers, 19(3), 525548. ganiere, p., chern, w., hahn, d., & chiang, f. s. (2004). consumer attitudes towards genetically modified foods in emerging markets: the impact of labeling in taiwan. the international food and agribusiness management review, 7(3), 1–20. gatignon, h., & robertson, t. s. (1989). technology diffusion: an empirical test of competitive effects. journal of marketing, 53(9), 35-49. gupta, a., & arora, n. (2017). understanding determinants and barriers of mobile shopping adoption using behavioral reasoning theory. journal of retailing and consumer services, 36, 1–7. gurtner, s. (2014). modelling consumer resistance to mobile health applications. ecis 2014 proceedings – 22nd european conference on information systems, tel aviv, israel. hair, j. f., black, w. c., babin, b. j., & anderson, r. e. (2016). multivariate data analysis.7th edition. pearson: india. hew, j. j., leong, l. y., tan, g. w. h., ooi, k. b., & lee, v. h. (2019). the age of mobile social commerce: an artificial neural network analysis on its resistances. technological forecasting and social change, 144, 311-324. hirschheim, r., & newman, m. (1988). information systems and user resistance: theory and practice. the computer journal, 31(5), 398-408. hosseini, m. h., delaviz, m., derakhshide, h., & delaviz, m. (2016). factors affecting consumer resistance to innovation in mobile phone industry. international journal of asian social science, 6(9), 497-509. jaganmohan, m. (2020). covid-19 impact on use of digital payments in india 2020. retrieved october 23, 2020, from https://www.statista.com/statistics/1111082/india-coronavirusimpact-on-digital-payment-usage/. jakhiya, m., bishnoi, m.m., & purohit, h. (2020). emergence and growth of mobile money in modern india: a study on the effect of mobile money. 2020 advances in science and https://government/ australasian journal of information systems talwar et al. 2021, vol 25, research on evaluation a study of mobile payment innovations of mobile applications 22 engineering technology international conferences (aset), 1-10, doi: 10.1109/aset48392.2020.9118375. kaur, p., dhir, a., singh, n., sahu, g., & almotairi, m. (2020a). an innovation resistance theory perspective on mobile payment solutions. journal of retailing and consumer services, 55, 102059. kaur, p., dhir, a., bodhi, r., singh, t., & almotairi, m. (2020b). why do people use and recommend m-wallets? journal of retailing and consumer services, 56, 102091. khanra, s., dhir, a., kaur, p., & joseph, r. p. (2021). factors influencing the adoption postponement of mobile payment services in the hospitality sector during a pandemic. journal of hospitality and tourism management, 46, 26–39. doi:10.1016/j.jhtm.2020.11.004 kleijnen, m., lee, n., & wetzels, m. (2009). an exploration of consumer resistance to innovation and its antecedents. journal of economic psychology, 20(3), 344-357. kuisma, t., laukkanen, t., & hiltunen, m. (2007). mapping the reasons for resistance to internet banking: a means-end approach. international journal of information management, 27(2), 75-85. kumar, s., dhir, a., talwar, s., chakraborty, d., & kaur, p. (2021). what drives brand love for natural products? the moderating role of household size. journal of retailing and consumer services, 58, 102329. doi:10.1016/j.jretconser.2020.102329 laukkanen, t. (2016). consumer adoption versus rejection decisions in seemingly similar service innovations: the case of the internet and mobile banking. journal of business research, 69(7), 2432-2439. laukkanen, p., sinkkonen, s., & laukkanen, t. (2008). consumer resistance to internet banking: postponers, opponents and rejectors. international journal of bank marketing, 26(6), 440-455. laukkanen, t., sinkkonen, s., kivijärvi, m., & laukkanen, p. (2007). innovation resistance among mature consumers. journal of consumer marketing, 24(7), 419-427. leong, l. y., hew, t. s., ooi, k. b., & wei, j. (2020). predicting mobile wallet resistance: a twostaged structural equation modeling-artificial neural network approach. international journal of information management, 51, 102047. lian, j. w., & yen, d. c. (2013). to buy or not to buy experience goods online: perspective of innovation adoption barriers. computers in human behavior, 29(3), 665-672. mani, z., & chouk, i. (2017). drivers of consumers' resistance to smart products. journal of marketing management, 33(1-2), 76-97. mani, z., & chouk, i. (2018). consumer resistance to innovation in services: challenges and barriers in the internet of things ea. journal of product innovation management, 35(5), 780-807. mittelstaedt, r. a., grossbart, s. l., curtis, w. w., & devere, s. p. (1976). optimal stimulation level and the adoption decision process. journal of consumer research, 3(2), 84-94. australasian journal of information systems talwar et al. 2021, vol 25, research on evaluation a study of mobile payment innovations of mobile applications 23 mohtar, s., & abbas, m. (2015). consumer resistance to innovation due to perceived risk: relationship between perceived risk and consumer resistance to innovation. journal of technology and operations management, 10(1), 1-13. nabih, m. i., bloem, j. g., & poiesz, t. b. c. (1997). conceptual issues in the study of innovation adoption behavior. advances in consumer research. 24, 190-196. oh, y. j., park, h. s., & min, y. (2019). understanding location-based service application connectedness: model development and cross-validation. computers in human behavior, 94, 82-91. park, k., & koh, j. (2017). exploring the relationship between perceived pace of technology change and adoption resistance to convergence products. computers in human behavior, 69, 142-150. patsiotis, a. g., hughes, t., & webber, d. j. (2013). an examination of consumers' resistance to computer-based technologies. journal of services marketing, 27(4), 294-311. podsakoff, p. m., mackenzie, s. b., lee, j. y., & podsakoff, n. p. (2003). common method biases in behavioral research: a critical review of the literature and recommended remedies. journal of applied psychology, 88(5), 879-903. poon, w. c. (2008). users' adoption of e-banking services: the malaysian perspective. journal of business and industrial marketing, 23(1), 59-69. pousttchi k., & schurig m. (2004). assessment of today's mobile banking applications from the view of customer requirements. proceedings of the 37th hawaii international conference on system sciences, big island, hi, usa. ram, s. (1987). a model of innovation resistance. advances in consumer research, 14, 208-212. ram, s., & sheth, j. n. (1989). consumer resistance to innovations: the marketing problem and its solutions. journal of consumer marketing, 6(2), 5-14. seth, h., talwar, s., bhatia, a., saxena, a., & dhir, a. (2020). consumer resistance and inertia of retail investors: development of the resistance adoption inertia continuance (raic) framework. journal of retailing and consumer services, 55, 102071. sharma, s., singh, g., sharma, r., jones, p., kraus, s., & dwivedi, y. k. (2020a). digital health innovation: exploring adoption of covid-19 digital contact tracing apps. ieee transactions on engineering management, 1–17. sharma, s., singh, g., & pratt, s. (2020b). does consumers' intention to purchase travel online differ across generations? empirical evidence from australia. australasian journal of information systems, 24, 1-31. silic, m., & ruf, c. (2018). the effects of the elaboration likelihood model on initial trust formation in financial advisory services. international journal of bank marketing, 36(3), 572-590. simard, r., & l'ecuyer, p. (2011). computing the two-sided kolmogorov-smirnov distribution. journal of statistical software, 39(11), 1-18. soh, p. y., heng, h. b., selvachandran, g., anh, l. q., chau, h. t. m., son, l. h., abdel-baset, m., manogaran, g., varatharajan, r. (2020). perception, acceptance and willingness of australasian journal of information systems talwar et al. 2021, vol 25, research on evaluation a study of mobile payment innovations of mobile applications 24 older adults in malaysia towards online shopping: a study using the utaut and irt models. journal of ambient intelligence and humanized computing. https://doi.org/10.1007/s12652-020-01718-4. suzuki, y., & williams, l. r. (1998). analysis of edi resistance behavior. transportation journal, 37(4), 36-44. szmigin, i., & foxall, g. (1998). three forms of innovation resistance: the case of retail payment methods. technovation, 18(6-7), 459-468. talke, k., & heidenreich, s. (2014). how to overcome pro-change bias: incorporating passive and active innovation resistance in innovation decision models. journal of product innovation management, 31(5), 894-907. talwar, s., dhir, a., kaur, p., & mäntymäki, m. (2020b). barriers toward purchasing from online travel agencies. international journal of hospitality management. 89, 102593. talwar, s., dhir, a., kaur, p., & mäntymäki, m. (2020d). why do people purchase from online travel agencies (otas)? a consumption values perspective. international journal of hospitality management, 88, 102534. talwar, s., dhir, a., kaur, p., zafar, n., & alrasheedy, m. (2019). why do people share fake news? associations between the dark side of social media use and fake news sharing behavior. journal of retailing and consumer services, 51, 72–82. talwar, s., dhir, a., khalil, a., mohan, g., islam, a. k. m. n. (2020c). point of adoption and beyond. initial trust and mobile-payment continuation intention. journal of retailing and consumer services, 55, 102086. talwar, m., talwar, s., kaur, p., islam, a. k. m. n., & dhir, a. (2020e). positive and negative word of mouth (wom) are not necessarily opposites: a reappraisal using the dual factor theory. journal of retailing and consumer services, 102396. doi:10.1016/j.jretconser.2020.102396 talwar, s., talwar, m., kaur, p., & dhir, a. (2020a). consumers' resistance to digital innovations: a systematic review and framework development. australasian marketing journal. https://doi.org/10.1016/j.ausmj.2020.06.014. talwar, m., talwar, s., kaur, p., & dhir, a. (2020f). has financial attitude impacted the trading activity of retail investors during the covid-19 pandemic? journal of retailing and consumer services, 58, 102341. thomas, y., vogel, g., wunderli, w., suter, p., witschi, m., koch, d., tapparel, c., kaiser, l., (2008). survival of influenza virus on banknotes. applied and environmental microbiology, 74(10), 3002-3007. undp. (2020). covid-19 pandemic. retrieved october 23, 2020, from https://www.undp.org/content/undp/en/home/coronavirus.html. who. (2020a). who coronavirus disease (covid-19) dashboard. retrieved october 23, 2020, from https://covid19.who.int/. https://covid/ australasian journal of information systems talwar et al. 2021, vol 25, research on evaluation a study of mobile payment innovations of mobile applications 25 who. (2020b). coronavirus disease (covid-19): how is it transmitted? retrieved october 23, 2020, from https://www.who.int/news-room/q-a-detail/coronavirus-disease-covid-19how-is-it-transmitted woodside, a.g., & biemans, w. g. (2005). modeling innovation, manufacturing, diffusion, and adoption/rejection processes. journal of business and industrial marketing, 20(7), 380393. xu, x., chen, p., wang, j., feng, j., zhou, h., li, x., zhong, w., & hao, p. (2020). evolution of the novel coronavirus from the ongoing wuhan outbreak and modeling of its spike protein for the risk of human transmission. science china life sciences, 63(3), 457-460. yin, z., gong, x., guo, p., & wu, t. (2019). what drives entrepreneurship in digital economy? evidence from china. economic modelling, 82, 66-73 appendix: measurement items usage barrier 1. using a mobile wallet for sending and receiving payments was difficult for me 2. using a mobile wallet for sending and receiving payments was inconvenient for me 3. mobile wallets often hang or work slowly when used for sending and receiving payments 4. the steps to use mobile wallets for sending and receiving payments are not clear to me. 5. entering data (name, age, etc.) in a mobile wallet for sending and receiving payments is a complicated process. value barrier 1. a mobile wallet does not offer any advantage for sending and receiving payments compared with other payment methods (e.g., cash, card, internet banking) 2. using a mobile wallet for sending and receiving payments does not increase my ability to control my financial matters by myself. 3. mobile wallets only allow a limited amount of money to be transferred in a day when used for sending and receiving payments 4. mobile wallets do not offer lucrative promotional benefits (discount, cashback, etc.) when used for sending and receiving payments. risk barrier 1. i fear that money may be transferred to the wrong contact when using a mobile wallet for sending and receiving payments. 2. i fear that i may send the wrong amount (e.g., with an extra zero) when using a mobile wallet for sending payments. 3. i fear that money may be debited multiple times from my bank account when using a mobile wallet for sending payments. 4. an incoming phone call can fail the in-process transaction on a mobile wallet even after the money is debited for sending payments. 5. if the network signal is low, mobile wallets do not process the transaction for sending and receiving payments. tradition barrier 1. i am more comfortable with cash transactions. 2. mobile wallet transactions for sending and receiving payments seem complicated to me. https://www/ australasian journal of information systems talwar et al. 2021, vol 25, research on evaluation a study of mobile payment innovations of mobile applications 26 3. i like going to bank branches on a weekday for money transfers. 4. chatting with the teller in the bank boosts my confidence in money transfers. 5. i feel that mobile wallets are not in sync with the well-entrenched ways of sending and image barrier 1. using a mobile wallet for sending and receiving payments requires a lot of work (linking a bank account, mobile number, aadhar card, etc.) 2. using a mobile wallet for sending and receiving payments causes frequent transaction failures. 3. using a mobile wallet for sending and receiving payments is suitable only for the youth. 4. using a mobile wallet for sending and receiving payments is too complicated to be useful. 5. i have an image that using mobile wallets for sending and receiving payments is a difficult process. postponement 1. i think that sending and receiving payments using a mobile wallet is easy, but i do not use it. 2. i think a mobile wallet is useful, but i am not using a mobile wallet for sending and receiving payments now. 3. i am not sure whether using a mobile wallet for sending and receiving payments is the right decision now. 4. i shall observe if people face any problem with using a mobile wallet before using it for sending and receiving payments myself. 5. i do not generally jump into using an innovation. opposition 1. i do not think that using a mobile wallet for sending and receiving payments is a good idea. 2. i do not think that a mobile wallet is useful for sending and receiving payments. 3. i tell my friends not to use mobile wallets for sending and receiving payments. 4. i tell my friends why it is not a good idea to use a mobile wallet for sending and receiving payments. 5. if a friend suggests that a mobile wallet is useful for sending and receiving payments, i shall oppose him/her. rejection 1. i think that avoiding mobile wallets for sending and receiving payments is the right decision. 2. i think that users do not get any financial benefit from sending and receiving payments using a mobile wallet. 3. i do not recommend to my friends using mobile wallets for sending and receiving payments. 4. if a friend suggests that a mobile wallet is useless for sending and receiving payments, i shall support him/her. australasian journal of information systems talwar et al. 2021, vol 25, research on evaluation a study of mobile payment innovations of mobile applications 27 copyright: © 2021 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v25i0.3201 http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 theoretical background 2.1 innovation resistance theory (irt) 2.2 non-adoption versus resistance toward innovations 2.2.1 non-adoption and resistance as identical concepts 2.2.2 resistance as a subset of non-adoption 2.2.3 mapping non-adoption and resistance to the present context 3 research model 4 data and methods 5 results 5.1 data characteristics 5.2 common method variance (cmv) 5.3 validity and reliability of the survey instrument 5.4 predictive accuracy of the ann models 5.5 sensitivity analysis 6 discussion of findings 7 conclusion, implications, limitations, and future research areas 7.1 theoretical implications 7.2 practical implications 7.3 limitations and future research areas paper title (paper title style) australasian journal of information systems binos, bruno & adamopoulos 2021, vol 25, selected papers from acis 2019 an intelligent framework in dynamic demand environments 1 intelligent agent based framework to augment warehouse management systems for dynamic demand environments tania binos rmit university melbourne, australia binos.tania@rmit.edu.au vince bruno rmit university melbourne, australia arthur adamopoulos rmit university melbourne, australia abstract warehouses are being impacted by increasing e-commerce and omni-channel commerce. the design of current wmss (warehouse management systems) may not be suitable to this mode of operation. the golden rule of material handling is smooth product flow, but there are dayto-day operational issues that occur in the warehouse that can impact this and order fulfilment, resulting in disruptions. standard operational process is paramount to warehouse operational control but may preclude a dynamic response to real-time operational constraints. the growth of iot (internet of things) sensor and data analytics technology provide new opportunities for designing warehouse management systems that detect and reorganise around real-time constraints to mitigate the impact of day-to-day warehouse operational issues. this paper presents the design and development stage of a design science methodology of an intelligent agent framework for basic warehouse management systems. this framework is distributed, is structured around operational constraints and includes the human operator at operational and decision support levels. an agent based simulation was built to demonstrate the viability of the framework. keywords warehouse management systems, distributed intelligence, software agents, decision support. 1 introduction increasingly, the future of warehouses is considered to be completely automated and humanless. not all warehouses can feasibly be automated, however. the expense of automation can preclude this solution in many smaller warehouses. products that are odd shapes and bulky can also be difficult to handle robotically. this study proposes that advances in iot and ai can bridge the gap between existing largely manual warehouses and full automation. changes in product make up and e-commerce orders present new issues for warehouses and distribution centres. the impact of expanding e-commerce and omni-channel business models (michel, 2016) create a new way of working that is characterised by increasingly changing customer demands, higher product variety, smaller order size, the expectation of reliably shorter response times (lu, giannikas, mcfarlane, & hyde, 2014) and irregular order arrival (leung et al., 2018). storing a wider variety of products in a warehouse in smaller quantities australasian journal of information systems binos, bruno & adamopoulos 2021, vol 25, selected papers from acis 2019 an intelligent framework in dynamic demand environments 2 can mean that the way the warehouse and its systems work is different (davarzani & norrman, 2015). in fact, only 10% of distribution centres exclusively handle pallets now, while 66% handle a mixture of pallets, cases, split cases, and pieces (michel, 2018a). many warehouse management systems (wms) are not designed for small orders picked in piece units. in the 2018 warehouse/distribution center (dc) equipment survey(michel, 2018b), 29% of respondents indicated that the most common process for e-commerce fulfilment in their operation was “buy online and ship to customer from dc” although there was growth in “buy online and ship to customer from vendor”. this suggests that the dc might be struggling with the processing of e-commerce orders. this pressure may see the default of members of the supply chain and this can alter the behaviour and the exposure to risk of other members of the supply chain (gibilaro & mattarocci, 2019). warehouse design including operational processes are predominantly based on current and projected future demand (de koster, le-duc, & roodbergen, 2007) and changes to design after construction are costly (gu, goetschalckx, & mcginnis, 2007). the warehouse control systems and software that operate them are costly to implement and change and require considerable training and support (min, 2006). warehouses are designed to be well-defined process-driven product flow machines (bartholdi & hackman, 2008). their aim is to achieve maximum throughput with minimum investment and operational costs (bartholdi & hackman, 2008; rouwenhorst et al., 2000; van den berg & zijm, 1999). however, their inability to flexibly respond to dynamic changes in demand leads to common issues such as aisle congestion, mistimed replenishments of the pick face, short-picks and double handling (gong & de koster 2011, bartholdi & hackman 2008; gu, goetschalckx & mcginnis 2009). dynamic responses are needed when the fixed constraints of the environment have been impacted. in dynamic environments, a small number of constraints can lead to operational issues that have a large impact on the warehouse throughput as a whole. these can impact warehouse performance if not detected and corrected in a timely manner. these include: • the operator is unable to complete order selection or replenishment of a selection location. potential causes: • congestion (for selector or forklift operator) (m. zhang, batta, & nagi, 2009) • inventory system inaccuracies (inventory record does not match reality). many factors can cause inventory inaccuracies including visual complexity (barratt, kull, & sodero, 2018). • inaccurate product and packaging dimension data that can misrepresent the actual height and weight of a storage unit. sku weights and dimensions can be hard to maintain due to supplier changes(michel, 2016) • slow replenishments • a replenishment that is triggered too early or too late has flow-on effects to picking (richards, 2014; rushton, croucher, & baker, 2014) such as overfull location (too early) or short-pick (too late). short picks require extra order selection trips and delay order completion. • slow receiving/checking processes. • a failure to prioritise stock that will be required sooner based on order demand (richards, 2014) australasian journal of information systems binos, bruno & adamopoulos 2021, vol 25, selected papers from acis 2019 an intelligent framework in dynamic demand environments 3 • picking from incorrect location, incorrect stock in location, over-picking, underpicking • identification of this issue (occurring even with bar-coding, rf (radio frequency) scanning and voice picking technology) can highlight incorrect case or inner case barcoding or operator error. picking inaccuracies that are not discovered prior to shipping can result in returns, repeat deliveries and loss of goodwill (garcía, chang, abarca, & oh, 2007). pick error rates were reported at between 0.02% and 0.05% by companies participating in a qualitative analysis of work schedule deviances (glock, grosse, elbert, & franzke, 2017). • staging dock congestion • badly managed staging areas can lead to congestion and scattered staging of picked orders causing issues when loading trucks for delivery. picked goods for dispatch should not be ready too early as this causes congestion and should also not be late as this holds up the loading process and the despatch door (walker, 2018) as with most errors that occur with flow-on effects in a system, prevention or detection and correction in a real-time or near real-time manner is desirable. this is especially important in the order fulfilment space, where truck scheduling and delivery time windows need to be met and returns are costly or become losses (especially with high value stock). current warehouse management systems (wms) are generally top-down, centralised systems and their decision support functions are the same. the disadvantages of central control include a single point of failure, inconsistent speed of response and a high dependency in the structure (haneyah, schutten, schuur, & zijm, 2013). most of these software packages deal with producing tasks to completely pick and despatch a batch of customer orders in a given time period. inventory management and interfaces to transport and billing (erp) systems are managed as part of order fulfilment. the dynamic operational control is mainly done by humans via summary reports and screens that require human intervention and that are highly customised (haneyah et al., 2013). this study sought to find a better way to design warehouse management systems, utilizing intelligent agents. it sought to answer the following research question: how can distributed intelligent agents augment warehouse operations to mitigate the impact of dynamic day-to-day issues? the purpose of this paper is to propose a distributed model for designing warehouse management systems that detects and reorganises around real-time constraints to mitigate the impact of day to day warehouse operational issues. the model is depicted in figure 1. australasian journal of information systems binos, bruno & adamopoulos 2021, vol 25, selected papers from acis 2019 an intelligent framework in dynamic demand environments 4 figure 1 conceptual model of proposed wms design framework the elements of this model include: • distributed intelligence via cooperative agents • real time feedback/detection mechanisms • incorporate real-time constraints in the framework • embed the human operator as an actor in the framework with consideration of: • physical, mental and social real-time constraints • the information required at different levels for decision support • adaptive learning based on trends for decision support • allow for ai algorithms to be plugged in where applicable and appropriate based on the temporal environment. 2 literature review the challenges faced in decision support for complex systems such as warehouses are: infeasibility (all the information is not available in one place), impracticality (it would not be practical to come up with a centrally determined optimal solution), inadvisability (even if you had such a solution it would not be advisable) (marik & mcfarlane, 2005). distributed intelligence (di) as a bottom-up approach to solving these issues has been proposed to enhance flexibility and agility (garcía et al., 2007). di is a system of elements that have a degree of autonomy of operation, can reason solely or jointly and can interpret the state of the australasian journal of information systems binos, bruno & adamopoulos 2021, vol 25, selected papers from acis 2019 an intelligent framework in dynamic demand environments 5 environment and the intentions of other elements (mcfarlane, parlikad, neely, & thorne, 2012). distributed intelligence can be implemented using an intelligent agent framework (shukla & frank chen, 1996). decision making within distributed intelligence can be determined by the data visibility at each point or node of a system. technologies such as rfid, internet of things (iot), gps and other tracking systems are making this easier (ding, 2013; karagiannaki, papakiriakopoulos, & bardaki, 2011; liu et al., 2013). one can imagine a warehouse environment in which information from sensors is used to form a picture of the environment and make a decision in real-time. estanjini, lin, li, guo, and paschalidis (2011) developed such a system to determine the next task to be allocated to forklift drivers in a grocery warehouse. the system used a sensor network, an information collection system, a localisation algorithm and dynamic programming to determine the next task to allocate to an available forklift. a dynamic programming technique – actor-critic algorithms was used to determine the optimal average cost in simulation and this value was used in the implementation in a case study. dynamic programming is used in multi-dimensional problem spaces such as this one where finding an exact optimum solution is almost impossible. one approach using product intelligence (giannikas, lu, mcfarlane, & hyde, 2013) uses a product and a shelf agent to select an appropriate storage location. this introduces the concept of non-human agency in the warehouse giving agency to products and shelves and is an important aspect of actor-network theory (latour, 2005). this “intelligent product” concept has been extended into concepts such as “communicating objects”(trab et al., 2017) and “smart warehouses” (ding, 2013). a new model using multi-agent systems (mas) and iot is proposed by reaidy, gunasekaran, and spalanzani (2015), responding to the growing norm of uncertainty in supply chains. this model uses a bottom-up approach in which simple functions with local objectives are distributed. ambient intelligence is achieved through sensors, control science (expert systems) and telecommunications. agent communication strategies may be cooperation, competition, coo-petition or comp-eration (reaidy et al., 2015). trab et al. (2017) use the “communicating object” concept and iot to deal with the safe handling of hazardous products in a warehouse. all actors in the warehouse are modelled including human operators. the proposed hybrid intelligent system (central and distributed elements) has three levels of interaction o2o (object to object), o2h (object to human) and o2e (object to environment). liu et al. (2013)) discusses full electronic and gps tracking of all goods coming in and out of the warehouse using rfid, gprs (gps) and zigbee wsn for tracking within the warehouse. a model for the safe handling of hazardous goods in warehouses is also proposed using rfid and iot to track materials and notify of incorrect placement and to support storage decisions (l. zhang, alharbe, & atkins, 2016). an artificial neural network (ann) is used for environment detection. what emerges from the above research is a model where information about the environment is pieced together from sensors and made sense of through algorithms and ai. multi-agent systems (mas) have been studied in warehouse management systems (garcía et al., 2007; kim, graves, & heragu, 2002; rubrico, ota, higashi, & tamura, 2006). a mas system was proposed to control stock levels using information from inputs and outputs of a bedding company warehouse captured via rfid (garcía et al., 2007). the stock level control was done australasian journal of information systems binos, bruno & adamopoulos 2021, vol 25, selected papers from acis 2019 an intelligent framework in dynamic demand environments 6 at the erp level while the enhanced rfid/ims system was used to improve warehouse inventory management and picking accuracy. this was a bottom up approach feeding decision support systems. multi-agent systems that communicate have been used to optimise the assignment of a pick route to an available picker based on proximity to the first location to be picked (gharbi, zgaya, & hammadi, 2013). the location of the pickers is determined via rfid. in one agent-based model presented for the movement of products from the main warehouse to storage in the production warehouse for use on the production line four agents are used: production, storage, forklift and main warehouse (maka, cupek, & wierzchanowski, 2011). the products are moved between the storage and production warehouses by forklift. as well as communication, there is some complex logic in scheduling which products should be moved, in what priority and in what sequence they should be loaded and unloaded from the forklift. the authors make an interesting point about the value of following global rules in storage decisions compared to the intelligence gained from analysing historical data such as product trends. instead of the rule that all products must be stored in the closest empty location for instance, it is more efficient for products that are usually retrieved together for the production line to be stored together. however, this information is dynamic and based on recent historical data. of the indirect measures of warehouse performance, labour is especially important since any inefficiency on the part of a human operator can impact shipping and delivery time. the importance of the human element (from operators to supervisors and management) as the actor whose “service failure or inefficient performance directly increases customer-order cycle time and negatively impacts the level of service as perceived by the customers” (staudt, alpan, di mascolo, & rodriguez, 2015) cannot be overlooked. a different way of thinking of work environments is that of a cooperative partnership rather than a differentiation between the “computer system” and the “human” as entities that enhance the fulfilment of goals. the desired precepts of a system in which human and nonhuman agents interact in a “team” are that: 1) agents agree to collaborate to achieve a joint state and goal, 2) agents are predictable, able to be directed, observant and transparent in intentions and behaviour and 3) agents coordinate, negotiate and communicate (klien, woods, bradshaw, hoffman, & feltovich, 2004). the design of decision support systems (dss) that can emulate human decision making would best be based on an understanding of human decision making but this is a complex psychological construct (arnott & pervan, 2005). firstly, humans are restricted by the availability and limits of cognitive resources, by biases and by time (einhorn & hogarth, 1981). nowadays, vast amounts of data can be stored and analysed. information overload can stunt human decision making. dss systems can assist humans most in the areas where they are most deficient: computational speed and planning (scenario generation) and it has been suggested that a human-machine collaboration is the best way to breach the gap between human limitations and computer understanding (pohl, 1999). as warehouse operations become more complex, decision support tools that augment the ability of people to make appropriate and timely decisions has many advantages (miller, 2018). real-time data capture supported by iot, data mining advances and non-human agency converge on the concept of a digital twin first developed in glaessgen and stargel (2012). the digital twin is proposed to have a complete and individual representation of its non-digital counterpart. it uses available information from sensors and via data mining of other sources australasian journal of information systems binos, bruno & adamopoulos 2021, vol 25, selected papers from acis 2019 an intelligent framework in dynamic demand environments 7 and mirrors the operating physical twin in order to predict the probability of future failure points based on hypothetical simulated scenarios. a digital twin consists of three parts: physical product, virtual product, and connected data that tie the physical and virtual product space (tao et al., 2018). the virtual product is a realtime reflection of the physical product and is being updated in real-time to maintain this mirroring. connected data consists of various real-time and historical data that can be analysed and mined in a converged state (tao et al., 2018). the connected data helps form a complete view of the state of an element and enhances decision-making. in the model presented in this paper, new technologies and algorithms are used to augment the natural decision-making and learning capacity of people to create more flexible and productive work environments. 3 methodology design science research methodology (dsrm) was used for this study because of its iterative approach to developing and improving an it artefact. an artefact is considered “any designed object in which a research contribution is embedded in the design” (peffers, tuunanen, rothenberger, & chatterjee, 2007). hevner, march, park, and ram (2004) describe design science research as “inherently, a problem-solving process”. the seven guidelines for effective design science research established by hevner et al. (2004) included designing a viable artifact, problem relevance, evaluation, contribution, research rigour, design as a search process and the communication of that research. in this research, the artefact is a distributed wms model tested via simulation. the six dsrm activities as defined by peffers et al. (2007) were used as follows: 1. identify problem and motivate. the problem addresses the real-time operational issues that cause bottlenecks and impede product flow in warehouses. as well as a literature review, semi-structured exploratory interviews were undertaken with warehouse professionals to confirm operational issues in warehouses. the results of these interviews are the subject of another paper. 2. define objectives of a solution. following the thematic analysis of the exploratory interviews, the data was examined to determine the link to the problem domain and the measures that would be used to evaluate the artefact that would be developed. the determined objective was to develop a model for designing a wms that makes use of distributed intelligence. this model provides a way to reorganise around real-time constraints before they become bottlenecks and augment human decision-making and operational control. 3. design and development. this involved the requirements analysis, the design of a conceptual model, agent model, agent communication protocol, state diagrams, simulation development and the development of evaluation scenarios and measures. this step is the main focus of this paper. 4. demonstration. the developed model was tested in the simulation using generated data (locations, products, inventory, orders etc.) against standard wms algorithms. australasian journal of information systems binos, bruno & adamopoulos 2021, vol 25, selected papers from acis 2019 an intelligent framework in dynamic demand environments 8 5. evaluation. this activity involves determining how well the artefact matches the problem solution defined in activity 2. the simulation environment that was designed and constructed was used to evaluate the feasibility of the wms design framework developed. 6. communication. this paper is communicating the wms design framework developed and evaluated in this study. we hope to make all phases and results of this study available through publication. three distinct design iterations are described below. these comprised the design and development activity of the dsrm methodology and were iteratively modified following demonstration/testing activities. 1. development of the conceptual model (depicted in figure 1). this involved defining the industry problem and reviewing the literature. the conceptual model was developed via a synthesis of research literature and industry experience. 2. design of the multi-agent system. this involved defining and specifying the types of agents required, their specifications, goals and behaviours. this is elaborated in section 4. 3. simulation. this involved developing a warehouse simulation environment and coding the behaviour of the agents, using jadex. the simulation was used to validate the design of the conceptual model. real-world scenarios and constraints were used to test and evaluate the model. this is elaborated in section 5. 4 conceptual design overview the design development of a distributed framework for a warehouse management system involved: requirements modelling and agent modelling. requirements analysis was used to determine the required agents and their interactions. agent modelling determined the agent states, goals, plans and modes of interaction. 4.1 requirements analysis anemona (a multi-agent methodology for holonic manufacturing systems) (botti & giret, 2008) was used to develop a basic requirements analysis of the agent and agent responsibilities for a warehouse. anemona was chosen because of the structured and iterative approach to analysing and defining agents from the organisational diagram to abstract agents, goals, tasks and interactions into agents. figure 2 shows a version of the abstract agents representing various warehouse roles. australasian journal of information systems binos, bruno & adamopoulos 2021, vol 25, selected papers from acis 2019 an intelligent framework in dynamic demand environments 9 figure 2 abstract agents and the organisational diagram 4.2 the agent model three types of agents were modelled: entity agents that were connected to a physical entity (location, inventory pallet, human, forklift, shipping pallet), mentor agents (forklift operator and selector operator mentors) and service agents. entity agents could be associated with sensors and other direct feedback mechanisms. mentor agents were associated with a human operator and were in control of communicating with other agents to obtain available tasks and communicating task instructions to the human operator. service agents were either providers of a function or information or fulfilled a role (generally a manager role). the design of the agents involved goals and plans that were either triggered by a request from another agent via a communication protocol or a change in the state of the agent due to a change in the environment. as such, the design of the agents involved state diagrams and an inter-agent communication protocol. each agent had a defined communication interface through which they receive a request for information or action. each communication protocol involved the exchange (incoming and outgoing) of a message (request or response) and an object of data. figure 3 shows some of the agent interactions in a mostly non-hierarchical configuration. figure 3 agent interactions australasian journal of information systems binos, bruno & adamopoulos 2021, vol 25, selected papers from acis 2019 an intelligent framework in dynamic demand environments 10 4.2.1 state diagrams the agent goals were either triggered by a change of agent state or by a request from another agent. a change in state indicated a change in the environment and hence a change in the actions required from the agent. in the example of a receiving pallet in figure 4, a change of state to “arrived” triggers a goal to unload the pallet from the truck, creating the task required to unload it. this task is prioritised by the work priorities agent and allocated to a forklift operator through the task service agent. once the pallet has been unloaded from the truck its state changes to “unloaded” triggering a goal to find a suitable warehouse location for it. the goals that are initiated by a change of state in an agent are predictable. entity agents maintain a state (status) and a position within the warehouse so that this can be maintained when the simulation is restarted. figure 4 state diagram for a pallet received into the warehouse the entity agents such as the receiving pallet agent above were paired with inanimate objects to which agency is not usually ascribed. all entity agents had a real-time vision of other entity agents in their shared environment. within the simulation this could be set to the entire environment or to a more realistic proximal distance. granting agency to inanimate objects means that they are not just represented as a number within a wms that are acted on but have the ability to determine their required action based on the current state. another example of this is the location or point (staging, door, equipment etc.) entity agent (figure 5). when the receiving pallet agent requests a storage location for itself, the location manager service may use any suitable algorithm which may change based on the current state of the environment. when a suitable location for put-away is found, the location agent must agree to allocate. this is a point at which the location agent, having more accurate information about its own state and the state of its surroundings may refuse the allocation thus preventing an operational exception from occurring when the forklift driver arrives at that location with the receiving pallet. entity agents are a form of digital twin albeit a not very complicated one and without analytical capabilities. australasian journal of information systems binos, bruno & adamopoulos 2021, vol 25, selected papers from acis 2019 an intelligent framework in dynamic demand environments 11 figure 5 state diagram for a location spot entity agent the mentor agents that are associated to a human agent are more complex. the mentor agent mirrors the state of their human operator counterpart who is constantly moving within their environment, unlike the physical objects paired with an entity agent. the mentor agents (forklift operator mentor and picker operator mentor) have the ability to re-arrange the order of the task allocation based on environment information such as congestion, to release a task to be completed by another operator or to signal a review of similar tasks once it is determined that the current task cannot be completed. for example, when the picker operator mentor agent (figure 6) detects a short pick at a primary selection location, the primary inventory pallet will request a replenishment pallet. if there are none available and none expected into the warehouse, then the picking manager service agent is signalled and it will either move or invalidate subsequent pick tasks from the same location preventing unnecessary work that cannot be completed. figure 6 picker operator mentor agent state diagram australasian journal of information systems binos, bruno & adamopoulos 2021, vol 25, selected papers from acis 2019 an intelligent framework in dynamic demand environments 12 4.2.2 communication protocol each agent has a defined communication interface through which they receive a request for information or action. each communication protocol involved the exchange (incoming and outgoing) of a message (request or response) and an object of data. figure 7 communication of forklift operator mentor the communication between agents does not follow a strict hierarchical configuration unless a request was required to be directed to a manager service agent instead of an entity agent. for instance, a picker mentor agent could not request another picker mentor agent to alter the path of its mission directly. this request had to be made to a picker manager service. figure 7 shows an excerpt of the communication initiated from the forklift mentor agent and other agents when the forklift operator signs on for their shift. the communication between the mentor and the human operator is a two-way communication whereby a request is issued by the mentor and accepted or completed by the operator. in a real-life implementation this communication would be via a voice headset or via an rf screen or hand-held device. 4.3 service agents – provision for plug-ins service agents have the greatest scope for including rules surmised from the data mining of historical data. since they receive real-time requests, instead of following one rule or algorithm for responding to a request, such as a request for a forklift work task or a selection mission, it may adjust its rules based on the state of the environment. the state of the environment can include time of the day, current number of operators, the number of available equipment all of which equate to the capacity of the warehouse, versus the demand on the warehouse (orders). for instance, the work priorities service agent receives requests to review the priorities of work tasks from other agents based on some operational event. these requests are received into a queue. when the warehouse capacity is under pressure, the priorities for task allocation become more important so the queue (figure 8 and figure 9) itself can be re-ordered based on australasian journal of information systems binos, bruno & adamopoulos 2021, vol 25, selected papers from acis 2019 an intelligent framework in dynamic demand environments 13 the number of requests for a particular product, the number of orders requiring that product and the expected departure time of the orders with those same products for instance. figure 8 work priorities service agent multiple requests for priority reviews for the same product beyond a threshold (from machine learning) can signal a potential issue to a human operator and assist in the decision-making process for resolving the issue with potential solution scenarios. this form of dss involves human-machine collaboration. it is not necessarily always up to the human decision-maker to identify potential issues with time-consuming reports but it may be more effective to allow a human decision-maker to decide the best course of action based on scenarios generated by the dss and by critical evaluation of the situation. figure 9 work priorities service agent actioning requests another example of this is the inventory pallet agent determining its replenishment requirements. it will use time estimates from the picker manager service for the “time until empty” and from the inventory service manager for “time until replenishment can be australasian journal of information systems binos, bruno & adamopoulos 2021, vol 25, selected papers from acis 2019 an intelligent framework in dynamic demand environments 14 completed”. both these estimates can be based on the current environment instead of stored averages so that the replenishment is created and allocated in time (figure 10). figure 10 primary selection inventory pallet reviewing replenishment requirements 5 simulation development this research examined various tools for the simulation development (bordini, dastani, dix, & seghrouchni, 2009). the tool had to be open-source and programmable in a language that was already familiar to the researchers. the agent programming tool had to support bdi (belief, desire, intentions) agents where a goal could be achieved through different plans that were deployed based on the real-time environment. jadex (pokahr, braubach, & lamersdorf, 2005) was chosen and used to develop the agent simulation with associated database tables using mysql (widenius, axmark, & arno, 2002). jadex agents are based on the traditional bdi agent model and its active component features facilitate the interactions between agents. the open-source multi-agent platform tools considered for the development of this simulation were jason (bordini, hübner, & wooldridge, 2007), jade (bellifemine, caire, & greenwood, 2007) and jadex. jason is an interpreter for an extended version of the logic-based language of agentspeak. jade did not have bdi agent infrastructure, and this was provided by jadex. although all three were trialled, jadex was chosen for its perceived ease of use and the availability of resources for supporting the development of the simulation. 5.1 simulation scenarios three common warehouse scenarios were selected to demonstrate the utility of the agent model in simulation. these are: scenario 1 – generating a replenishment. this scenario is based on the precept that inventory received into the warehouse should be checked and stored in a sequence that prioritises stock that will be required sooner based on order demand. this is often a manual process in warehouse systems that are increasingly reliant on “just in time” inventory. the visibility of incoming goods required for order fulfilment in real-time, discerning and updating the australasian journal of information systems binos, bruno & adamopoulos 2021, vol 25, selected papers from acis 2019 an intelligent framework in dynamic demand environments 15 priorities of the receipt of these goods in order to avoid an exception in picking is demonstrated in this scenario. scenario 2 – short pick in selection. order selection is still the most costly warehouse task performed and travel time in order selection is considered a major and “unproductive” component. extant research focuses on optimising pick paths to minimise travel time. this scenario demonstrates minimising wasted travel time by avoiding short-picks in selection when an exception is detected which means that the intended selection tasks will not be successful. scenario 3 – congestion in selection. congestion occurs in real-time and hence is difficult to predict and is largely left up to the selectors to manage using the available options at their disposal – generally “skip” the pick or “wait”. congestion is also generally not tracked for analysis. this scenario demonstrates minimising unproductive time in selection due to congestion and the ability to track congestion and the success of congestion avoidance scenarios. 5.1.1 simulation scenario 1 generating a replenishment a replenishment is the refill of a primary selection location with product from another location. it involves identifying that there is a shortage in the primary selection location and triggering the move required to fill it in enough time to prevent shortages during order selection. the scenario is as follows: • a customer order for 4 of product 108081 arrives into the system. the selection location for product 108081 is “01-01-1-1” and only has a quantity of 3. a pallet of 40 cases for product 108081 is on a receiving truck that has arrived with 10 other pallets but has not been unloaded yet. table 1 shows the sequence of events in the distributed model in simulation. description screenshot order manager agent: • receives a request to release customer orders for picking and initiates plan to release order. • detects that there is only a quantity of 3 in active (in a warehouse location) inventory. • creates a selection mission for 3 and backorders 1. • requests work priorities manager review priorities of “receiving” for this product. • work priorities manager agent: • increases the priority of all unload work on the receiving appointment truck. the pallet with 108081 has a higher priority. • task manager agent: • changes the destination of the pallet with product 108081 to the priority lane. other pallets go to the staging lane • forklift driver is allocated the unload task and completes it. australasian journal of information systems binos, bruno & adamopoulos 2021, vol 25, selected papers from acis 2019 an intelligent framework in dynamic demand environments 16 description screenshot inventory pallet agent: • generates a put-away task when the 108081 pallet is placed in the priority lane. this putaway task is of high priority because the unload task was high priority. • forklift driver is allocated the put-away task and completes it. • with augmented reality wearable glasses connected to the mentor agent, put-away pickup could be allocated to a specific operator and directed with colour coding. • completing a high priority put-away task triggers a request by the forklift mentor agent on the primary selection location to review its replenishment requirements. inventory pallet agent: • requests the pick manager for an estimate of time before location is empty. • requests the inventory manager for an estimate of time to complete the “best” replenishment. • requests task service to generate a replenishment. • requests work priorities manager review priorities of “replenishment” for this product. the priority manager can do this in cooperation with the resource agent. • selector visits location and picks 3. • forklift driver is allocated the replenishment task and completes it. • order manager releases the backorder quantity. table 1 distributed model for replenishment scenario 5.1.2 simulation scenario 2 – short pick in selection there has been a promotion for end of model product that has caused a spike in demand for that product. the warehouse expected to have enough in stock to meet the projected demand but an error in master data (the number of products on a supplier pallet) has meant that the warehouse has overestimated stock levels. the scenario is as follows: • four customer orders for 4 each of product 108081 arrive into the system. the system indicates that the selection location for product 108081 is “01-01-1-1” and has 30 but actually only has a quantity of 10. there are no available pallets in reserve for this product. table 2 shows the sequence of events in the distributed model in simulation. description order manager agent: • receives a request to release customer orders for picking and initiates plan to release orders. • erroneously detects that there is a quantity of 30 in active (in a warehouse location) inventory. • creates 5 selection missions for this product along with other products ordered by the customer. australasian journal of information systems binos, bruno & adamopoulos 2021, vol 25, selected papers from acis 2019 an intelligent framework in dynamic demand environments 17 description picking manager service: • receives a request from the mission service to make selection missions available. • makes all 5 selection missions available for pickers. mission service agent: • allocates missions to picker mentor agent for associated picker. pick operator agent/picker mentor agent • first 2 selection missions complete successfully. inventory pallet agent: • requests replenishment but there are no available reserve pallets. • informs picking manager of anticipated short pick issue. pick operator agent/picker mentor agent • picker operator travels to selection location, picks remaining 2. two are not picked (shorted). inventory pallet agent: • updates balance in location to zero. • requests replenishment but there are no available reserve pallets. • informs picking manager of actual short pick issue. picking manager • modifies available but as yet unallocated selection missions to remove this pick instruction. selection mission cancelled if no other pick instructions. • requests order manager update orders and backorder the items not picked. • notifies warehouse manager agent of issue. warehouse manager service agent • sends issue to inventory controller mentor agent. inventory controller mentor agent • displays issue and potential scenarios to inventory controller human agent table 2 distributed scenario for short pick in selection australasian journal of information systems binos, bruno & adamopoulos 2021, vol 25, selected papers from acis 2019 an intelligent framework in dynamic demand environments 18 5.1.3 simulation scenario 3 – congestion in selection there has been a promotion for end of model product that has caused a spike in demand for that product. the warehouse does have enough in stock to meet the projected demand. the scenario is as follows: • forty customer orders for 1 each of product 108081 arrive into the system. the system indicates that the selection location for product 108081 is “01-01-1-1” and has 50. table 3 shows the sequence of events in the distributed model in simulation. description order manager agent: • receives a request to release customer orders for picking and initiates plan to release orders. • creates 40 selection missions for this product along with other products ordered by the customer. picking manager service: • receives a request from the mission service to make selection missions available. • makes 40 selection missions available for pickers. mission service agent: • allocates missions to picker mentor agent for associated picker. pick operator agent/picker mentor agent • requests status from passage agent (sensor in the aisle for the next selection location to be picked) passage agent • indicates that it is at capacity. picker mentor agent: • alters the pick path of the selection mission for a less congested path or next aisle to become uncongested or other rule. • picker operator is informed of change in pick sequence. table 3 distributed scenario for congestion in selection 5.2 simulation evaluation these scenarios as described above with the agent model interaction were reviewed by an expert from the field with 25 years of experience in warehousing and distribution. all three scenarios were evaluated as realistic. feedback on scenario one related to directing the australasian journal of information systems binos, bruno & adamopoulos 2021, vol 25, selected papers from acis 2019 an intelligent framework in dynamic demand environments 19 putaway of priority pallets directly to the selection location from priority staging instead of via reserve inventory, and has been incorporated into a future test scenario. 6 discussion warehouses and warehouse management systems are complex and dynamic systems traditionally controlled by strict adherence to process and sequence. the problem with this is that when an unexpected exception occurs, the process cannot be followed and something else has to be done in its place and this causes delays and flow-on effects. adaptive mechatronic systems with societies of autonomous adaptive agents can exhibit self-organising behaviours that can overcome these small bottlenecks in operational processing. as in the scenario in section 5, the pallet yet to be unloaded can be prioritised and the task destination changed dynamically by the real-time detection of a deficit in selection. adaptation can occur by recording predicted estimates of task completion against actual task completion outcome and incorporating the error (difference) into future estimates. adaptation can also occur in storage by connecting patterns of inventory inaccuracies with product characteristics such as similarity contributing to operator error. agents can communicate a real-time status. this means that dynamic tasks can be generated closer to the time that they will be actioned. as a result, more appropriate locations will be available for inventory to be put away, the resources to complete the tasks will be available (potentially) and time and productivity is not spent actioning tasks that are not ready to be completed and that may potentially not be able to be completed. agents have goals and goals can persist. agents continue to execute plans until the goal is fulfilled so an exception does not need to be handled manually all the time. for instance, a replenishment that failed to be physically completed will trigger a replacement task (until the selection location is refilled). a pallet that failed to be physically put away in its designated location will be taken to the resolution zone and if appropriate will self-generate another put away task for itself. tracking the reason for task failure (exceptions) in real-time allows more information to be given to the inventory manager to resolve issues that could not be selfresolved, aiding the decision-making process. human intervention, where needed, can correct the system data to the point where the agents can continue their normal processing. the purpose of the mentor agent is to provide a real-time ability to react to real-time changes in the environment. human operators are generally just following instructions and encountering a real-time constraint issue such as congestion. the mentor agent, communicating agent-agent, agent-human or human-agent, can provide the advantage of a smarter view of the warehouse environment for the operator. with sensors detecting changes in the warehouse environment, different algorithms can be deployed to determine selector travel path or to re-evaluate the sequence of available tasks. in a working warehouse environment this simulation could be run with actual data and could be used to enact scenarios which have caused an operational exception that was pre-emptively avoided to determine how the operational exception could happen again and become a failure point that is not managed to be avoided (increasing demand, increasing congestion, decreasing staff and equipment). alternatively, scenarios that were a failure point can be used in the simulation to determine corrective action or pre-emptive detection that can be used to avoid the failure. australasian journal of information systems binos, bruno & adamopoulos 2021, vol 25, selected papers from acis 2019 an intelligent framework in dynamic demand environments 20 the increase in e-commerce and omni-channel commerce has been putting a strain on warehouses and wms systems. many warehouse management systems (wms) are not designed for this new way of working. in addition, warehouses by their design run efficiently when predicted demand is adequately met by warehouse capacity (goods, people and resources) and when operational procedures are clear and stringently followed. however, there are a number of small operational exceptions that can impede this smooth product flow and manifest as disruptions if not detected and mitigated early. past warehouse research literature has largely focused on optimisation strategies (pick paths, product storage and human ergonomic aspects). more recently research has begun to focus on realising a living or “smart” warehouse by giving inanimate objects properties and agency in order to respond faster to real-time events. it seems that this is a unique point in time, with emerging technologies such as data mining and analysis, sensor and iot interconnectivity to create a wms framework that is not just based on maintaining “standard” operations and their optimisation but also includes a basis of the real-time constraints that cause exceptions and lead to disruptions. estanjini et al. (2011) uses sensors and dynamic programming to calculate the most costeffective way to allocate the next forklift task which is smart optimisation. the intelligent product agent in giannikas et al. (2013) seeks the best storage location each time it is received into the warehouse and this is based on a number of factors including “turnover rate, demand, the relationships with other products, the layout of the warehouse etc.” this could also include minimisation of warehouse real-time exceptions. giannikas et al. (2013) describes “unexpected events” such as “arrivals of new orders”. in a warehouse environment there can also be many other unexpected events. the wms agent in trab et al. (2017) is based on real-time safety compatibility constraints of goods and people in a hazardous goods environment and is focused on minimising risk. trab et al. (2017) describes an “unexpected event” as something that increases risk to “products, humans and the environment” such as “the presence of an empty product owing to a leak, and a human user without safety protection equipment”. in everyday warehouse operations, unexpected events pose a risk to efficiency and productivity. while the type of unexpected events may be known, the timeline of their occurrence and which operational exception will occur as a result is not known. for this reason, a wms framework that can detect their occurrence and has built-in plans that can be deployed when they occur is needed both to mitigate flow-on risk and to provide real assistance to decision makers. current literature focuses on recording exceptions (weissbach, radmanu, & grabowski, 2009) and expediting individual exception resolution (mcdonald jr et al., 2018) but not on detecting an exception that is about to occur in order to avoid it, detecting exceptions in real-time, minimising the flow-on effect when an exception has occurred and augmenting the decisionmaking capacity of humans tasked with resolving these exceptions. this study proposes that incorporating these exceptions specifically into the wms framework will make warehouses more able to respond to dynamic changes in the environment. 7 conclusion warehouses are real-time environments that involve a work schedule, available resources, and time and space constraints. increasingly, the outlook to solving this problem is to fully automate, however this may not be feasible for small to medium size warehouses or australasian journal of information systems binos, bruno & adamopoulos 2021, vol 25, selected papers from acis 2019 an intelligent framework in dynamic demand environments 21 warehouse with specialist requirements. this research aims at innovation by augmenting people rather than replacing them. this paper presents a framework for designing warehouse management systems that uses distributed intelligence to react to real-time events and constraints, in order to mitigate the impact of small operational issues on product flow and order fulfilment. a simulation based on intelligent agents was designed and constructed in order to validate the viability of the framework. the contribution of a new framework for wms design is three-fold. firstly, there are advantages to be gained in detecting warehouse exceptions in real-time before they escalate into disruptions. although adherence to warehouse processes dictate smooth product flow, unexpected events can have a big impact on productivity and efficiency. secondly, academic research provides new algorithms and processes for optimisation and efficiency of warehouse processes, but these cannot be easily incorporated into current warehouse management systems. the design of service agents in this model can make adding new features such as the above more feasible in an environment where they can be used when applicable based on real-time environmental constraints and where their effectiveness can be evaluated in real-time and historically through data-mining. thirdly, decision support that supports the cognitive, memory and actual time constraints of the warehouse decision makers by providing packaged information about an exception and potential resolution scenarios facilitate the resolution of warehouse exceptions faster where a human decision-maker is required. this paper contributes to industry by proposing a new way of designing warehouse management systems that augments and enhances the operations of manual warehouses instead of replacing them with automation. with advances in iot and sensor technologies this approach may be more suitable to the challenges posed by demand being driven by the changing consumer behaviour. future research iterations of this project may take several paths. various ai methods and algorithms can be tested as plug-ins within the intelligent agents, in the simulation, to improve operational functionality and the ability of the model to forecast and resolve problems. the metrics that will be used for evaluation purposes include stock out rate, orders shipped on time, shipping accuracy, queuing time and throughput. these metrics are based on the direct indicators of performance as defined by (staudt et al., 2015). the number of operational exceptions that occur will also be measured for comparison. references arnott, d., & pervan, g. (2005). a critical analysis of decision support systems research. journal of information technology, 20(2), 67-87. barratt, m., kull, t. j., & sodero, a. c. (2018). inventory record inaccuracy dynamics and the role of employees within multi-channel distribution center inventory systems. journal of operations management, 63, 6-24. bartholdi, j. j., & hackman, s. t. (2008). warehouse & distribution science: release 0.89: supply chain and logistics institute. bellifemine, f. l., caire, g., & greenwood, d. (2007). developing multi-agent systems with jade (vol. 7): john wiley & sons. australasian journal of information systems binos, bruno & adamopoulos 2021, vol 25, selected papers from acis 2019 an intelligent framework in dynamic demand environments 22 bordini, r. h., dastani, m., dix, j., & seghrouchni, a. e. f. (2009). multi-agent programming: springer. bordini, r. h., hübner, j. f., & wooldridge, m. (2007). programming multi-agent systems in agentspeak using jason (vol. 8): john wiley & sons. botti, v., & giret, a. (2008). anemona: a mulit-agent methodology for holonic manufacturing systems. london: springer london, london. davarzani, h., & norrman, a. (2015). toward a relevant agenda for warehousing research: literature review and practitioners’ input. logistics research, 8(1), 1. de koster, r., le-duc, t., & roodbergen, k. j. (2007). design and control of warehouse order picking: a literature review. european journal of operational research, 182(2), 481-501. ding, w. (2013). study of smart warehouse management system based on the iot. in intelligence computation and evolutionary computation (pp. 203-207): springer. einhorn, h. j., & hogarth, r. m. (1981). behavioral decision theory: processes of judgement and choice. annual review of psychology, 32(1), 53-88. estanjini, r. m., lin, y., li, k., guo, d., & paschalidis, i. c. (2011). optimizing warehouse forklift dispatching using a sensor network and stochastic learning. ieee transactions on industrial informatics, 7(3), 476-486. garcía, a., chang, y., abarca, a., & oh, c. (2007). rfid enhanced mas for warehouse management. international journal of logistics research and applications, 10(2), 97-107. doi:10.1080/13675560701427379 gharbi, s., zgaya, h., & hammadi, s. (2013). optimization of order picker path based on agent communication in warehouse logistics. ifac proceedings volumes, 46(24), 7-14. doi:10.3182/20130911-3-br-3021.00024 giannikas, v., lu, w., mcfarlane, d., & hyde, j. (2013). product intelligence in warehouse management: a case study. paper presented at the international conference on industrial applications of holonic and multi-agent systems. gibilaro, l., & mattarocci, g. (2019). the impact of corporate distress along the supply chain: evidences from united states. supply chain management: an international journal. glaessgen, e., & stargel, d. (2012). the digital twin paradigm for future nasa and us air force vehicles. paper presented at the 53rd aiaa/asme/asce/ahs/asc structures, structural dynamics and materials conference 20th aiaa/asme/ahs adaptive structures conference 14th aiaa. glock, c. h., grosse, e. h., elbert, r. m., & franzke, t. (2017). maverick picking: the impact of modifications in work schedules on manual order picking processes. international journal of production research, 55(21), 6344-6360. gu, j., goetschalckx, m., & mcginnis, l. f. (2007). research on warehouse operation: a comprehensive review. european journal of operational research, 177(1), 1-21. haneyah, s., schutten, j. m., schuur, p., & zijm, w. h. (2013). generic planning and control of automated material handling systems: practical requirements versus existing theory. computers in industry, 64(3), 177-190. australasian journal of information systems binos, bruno & adamopoulos 2021, vol 25, selected papers from acis 2019 an intelligent framework in dynamic demand environments 23 hevner, a. r., march, s. t., park, j., & ram, s. (2004). design science in information systems research. mis quarterly, 28(1), 75-105. karagiannaki, a., papakiriakopoulos, d., & bardaki, c. (2011). warehouse contextual factors affecting the impact of rfid. industrial management & data systems, 111(5), 714-734. kim, b.-i., graves, r., & heragu, s. (2002). intelligent agent modeling of an industrial warehousing problem. iie transactions, 34(7), 601-612. doi:10.1080/07408170208928897 klien, g., woods, d. d., bradshaw, j. m., hoffman, r. r., & feltovich, p. j. (2004). ten challenges for making automation a" team player" in joint human-agent activity. ieee intelligent systems, 19(6), 91-95. latour, b. (2005). reassembling the social an introduction to actor-network-theory. oxford, new york: oxford university press. leung, k., choy, k., siu, p. k., ho, g., lam, h., & lee, c. k. (2018). a b2c e-commerce intelligent system for re-engineering the e-order fulfilment process. expert systems with applications, 91, 386-401. liu, x. j., liu, q., xu, w. j., yang, l. w., fan, l., & chen, b. (2013). design and implementation of intelligent warehousing management system using internet of things. applied mechanics and materials, 432-432, 609-614. doi:10.4028/www.scientific.net/amm.432.609 lu, w., giannikas, v., mcfarlane, d., & hyde, j. (2014). the role of distributed intelligence in warehouse management systems. in service orientation in holonic and multi-agent manufacturing and robotics (pp. 63-77): springer. maka, a., cupek, r., & wierzchanowski, m. (2011). agent-based modeling for warehouse logistics systems. paper presented at the 2011 uksim 13th international conference on modelling and simulation. marik, v., & mcfarlane, d. (2005). industrial adoption of agent-based technologies. ieee intelligent systems, 20(1), 27-35. mcdonald jr, r. d., agriel, f. l., levi, e. s., patel, v. n., nanjanath, m., madan, u., & glick, d. d. (2018). utilizing automated aerial vehicles for transporting priority pick items. in: google patents. mcfarlane, d., parlikad, a., neely, a., & thorne, a. (2012). a framework for distributed intelligent automation systems developments. ifac proceedings volumes, 45(6), 758-763. doi:10.3182/20120523-3-ro-2023.00325 michel, r. (2016). ready to confront complexity. logistics management (2002), 55(11), 648s,49s,50s,52s,54s,55s. michel, r. (2018a). 2018 warehouse/dc operations survey: labor crunch driving automation: the combined forces of a strong economy, e-commerce growth and a tight labor market are making it more important for distribution center operations to find ways to make their existing infrastructure and people more productive. software and automation continue to prove to be a vital part of the solution. modern materials handling, 73(11), 62. michel, r. (2018b). automation & robotics lead robust outlook. logistics management (2002), 57(3), 40-47. australasian journal of information systems binos, bruno & adamopoulos 2021, vol 25, selected papers from acis 2019 an intelligent framework in dynamic demand environments 24 miller, s. (2018). ai: augmentation, more so than automation. asian management insights, 5(1), 1-20. min, h. (2006). the applications of warehouse management systems: an exploratory study. international journal of logistics: research and applications, 9(2), 111-126. peffers, k., tuunanen, t., rothenberger, m. a., & chatterjee, s. (2007). a design science research methodology for information systems research. journal of management information systems, 24(3), 45-77. pohl, j. g. (1999). collaborative decision-support and the human-machine relationship. collaborative agent design (cad) research center, 18. pokahr, a., braubach, l., & lamersdorf, w. (2005). jadex: a bdi reasoning engine. in multiagent programming (pp. 149-174): springer. reaidy, p. j., gunasekaran, a., & spalanzani, a. (2015). bottom-up approach based on internet of things for order fulfillment in a collaborative warehousing environment. international journal of production economics, 159, 29-40. richards, g. (2014). warehouse management : a complete guide to improving efficiency and minimizing costs in the modern warehouse (2nd edition). london: london, gbr: kogan page. rouwenhorst, b., reuter, b., stockrahm, v., van houtum, g.-j., mantel, r., & zijm, w. h. (2000). warehouse design and control: framework and literature review. european journal of operational research, 122(3), 515-533. rubrico, j. i. u., ota, j., higashi, t., & tamura, h. (2006). scheduling multiple agents for picking products in a warehouse. paper presented at the robotics and automation, 2006. icra 2006. proceedings 2006 ieee international conference on. rushton, a., croucher, p., & baker, p. (2014). the handbook of logistics and distribution management: understanding the supply chain: kogan page publishers. shukla, c., & frank chen, f. (1996). the state of the art in intelligent real-time fms control: a comprehensive survey. journal of intelligent manufacturing, 7(6), 441-455. doi:10.1007/bf00122834 staudt, f. h., alpan, g., di mascolo, m., & rodriguez, c. m. t. (2015). warehouse performance measurement: a literature review. international journal of production research, 53(18), 5524-5544. doi:10.1080/00207543.2015.1030466 tao, f., cheng, j., qi, q., zhang, m., zhang, h., & sui, f. (2018). digital twin-driven product design, manufacturing and service with big data. the international journal of advanced manufacturing technology, 94(9-12), 3563-3576. trab, s., bajic, e., zouinkhi, a., thomas, a., abdelkrim, m. n., chekir, h., & ltaief, r. h. (2017). a communicating object’s approach for smart logistics and safety issues in warehouses. concurrent engineering, 25(1), 53-67. van den berg, j. p., & zijm, w. h. (1999). models for warehouse management: classification and examples. international journal of production economics, 59(1-3), 519-528. walker, m. (2018). spotlight on the 7 key warehouse processes. mhd supply chain solutions, 48(1), 20-22. australasian journal of information systems binos, bruno & adamopoulos 2021, vol 25, selected papers from acis 2019 an intelligent framework in dynamic demand environments 25 weissbach, s., radmanu, o., & grabowski, s. (2009). handling exceptional situations in a warehouse management. in: google patents. widenius, m., axmark, d., & arno, k. (2002). mysql reference manual: documentation from the source: " o'reilly media, inc.". zhang, l., alharbe, n., & atkins, a. s. (2016). an iot application for inventory management with a self-adaptive decision model. paper presented at the internet of things (ithings) and ieee green computing and communications (greencom) and ieee cyber, physical and social computing (cpscom) and ieee smart data (smartdata), 2016 ieee international conference on. zhang, m., batta, r., & nagi, r. (2009). modeling of workflow congestion and optimization of flow routing in a manufacturing/warehouse facility. management science, 55(2), 267-280. copyright: © 2021 binos, bruno & adamopoulos. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v25i0.2845 http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems chatterjee 2021, vol 25, research on user involvement impact of addiction of online platforms on quality of life 1 impact of addiction of online platforms on quality of life: age and gender as moderators sheshadri chatterjee postdoctoral researcher indian institute of technology kharagpur, kharagpur, west bengal india sheshadri.academic@gmail.com abstract the excessive usage of online platforms is inviting several unwanted problems in the society. the excessive use of online platforms is adversely interfering in many social activities. this uncontrolled and excessive use of online platforms is causing addiction to the users. this is unexpectedly impeding the normal social flow of life culminating an adverse effect on the individuals’ quality of life. studies reveal that age and gender have influence towards addiction. in this background, the purpose of this study is to identify the factors impacting addiction of online platforms. from studies of several addiction theories, some hypotheses have been formulated and a conceptual model has been developed. these have been validated by partial least square – structural equation modeling (pls-sem) analysis with the help of survey involving 320 usable respondents. the study highlights that loneliness, perceived enjoyment, depression, perceived ease of use and perceived usefulness act as vital predictors of addiction of online platforms that impacts quality of life. the moderating factors age and gender are found to have effective impacts on the influence of predictors on the addiction of online platforms. the article is ended mentioning the limitations of this study incorporating the scopes for the future researchers to nurture the untouched points. keywords: addiction, age, gender, online platforms, quality of life 1 introduction studies reveal that more than 32% world population use online platforms for various transactions (e marketer, 2016). various banking and shopping activities can be completed by e-commerce sites. even, social networking sites can help to understand the consumers’ behaviour (lee et al., 2014; chatterjee et al., 2020b). the household consumers are expected to use different kinds of online platforms for shopping of grocery items as they become more tech-savvy (sarin et al., 2020; grover & kar, 2020). thus, use of different online platforms is providing many advantages to the users (whiteside et al., 2018). this would develop the habit of the users to use online platforms more that might indulge the individuals to eventually become victims of addiction in using online platforms (augner & hacker, 2012; choliz, 2012). this addiction is apprehended to affect quality of life (choi & seo, 2015; mard & cui, 2017; soral et al., 2020). several studies highlighted that age and gender of individuals might affect the extent of addiction of individuals (hu et al., 2019). the addiction habit affects the individuals’ social activities (boreham, 2016; arpaci et al., 2018). addiction of online platforms includes addiction of social media, smartphones, online shopping, online banking and so on (disen & deshpande, 2018; wang, 2019). excessive use of different online platforms causes online addiction. it has negative impacts towards several areas of human life affecting the australasian journal of information systems chatterjee 2021, vol 25, research on user involvement impact of addiction of online platforms on quality of life 2 quality of life (park & lee, 2012; xue et al., 2018). use of online platforms initially invites pleasure, interest, and excitement to the users. however, this motivation induces the users to use the system frequently causing addiction. this causes harm to the society affecting the users’ quality of life (jiang, 2014; carlson et al., 2015; ram & zhang, 2018). easiness of use of a technology helps a user to frequently use that technology (dolan et al., 2016; carlson et al., 2017; grover et al., 2018, 2019). stress causes depression and the depressed persons keep themselves engaged in frequently using a technology to be relieved from the depression. this has been seen in other studies in smartphone usage where the users were addicted (kumar et al., 2020). some people exhibit avoidant attitude, and this feeling is associated with the sense of loneliness (wei et al., 2005; lee et al., 2019). feelings of usefulness of a system lead an individual to use the system frequently. this causes addiction to that technology (mc kinley et al., 2016; wang et al., 2018). the perceived enjoyment is considered as an important predictor of use-behaviour of a system as is found from the other studies (van der heijden, 2003; 2004; baccarella et al., 2018). it has been already stated that age and gender of individuals are perceived to impact addiction to any technology. but, how, to what extent and in which way, these two factors influence addiction of online platform have remained under studied. in this background the objectives of this studies are as follows. 1. to identify the factors impacting addiction of online platforms by the users. 2. to examine the moderating effects of age and gender for the addiction of online platform. 3. to understand how the addiction of online platforms impacts quality of life of the users. the rest of the article is structured as follows. discussions have been made on the theoretical backgrounds. thereafter, hypothesis have been formulated and a conceptual model has been developed. next, hypotheses have been tested statistically and the results have been discussed. then discussions have been made on theoretical contributions and practical implications followed by mentioning of limitations of this study with future scope of research. 2 theoretical background and development of conceptual model 2.1 theoretical background addiction is considered as a special behavioural manifestation of an individual. this manifestation leads the individual to have lost control that causes harmful consequences (shelton et al., 2014). this is considered as a psychiatric motivational disorder (collier, 1993). to identify antecedents of addiction, some relevant theories are considered convenient. the challenges and objectives of different addiction theories are many folds. it is important to identify how addiction occurs in the society and what are its remedies (kim et al., 2017). studies reveal that all the theories do not match to exhaustively interpret addiction to any technological use. but the adult attachment theory (fraley & shaver, 2000) provides that addiction in any technology is impacted by the human centric traits like loneliness and depression (kim et al., 2017; csibi et al., 2019). from the study of adult attachment theory, it is evident that younger individuals use online platforms as an attachment object to be relieved from depression and loneliness (jeon & jang, 2014; fox & moreland, 2015). from the study of the tension reduction theory (cappell & greelay, 1987), it is transpired that depression can supplement an individual for impairing control that would lead to inimical consequences australasian journal of information systems chatterjee 2021, vol 25, research on user involvement impact of addiction of online platforms on quality of life 3 (collier, 1993). the tension reduction theory subscribes that addiction to any specific technological use emerges from the feelings of loneliness and depression (hassanbeigi et al., 2013; lee et al., 2019). in this study, we have also lent the idea of technology acceptance model (tam) (davis et al., 1989) wherefrom two core constructs, perceived usefulness and perceived ease of use have been used. this model is a parsimonious model and has been considered as a foundation of adoption in technology acceptance (dolan et al., 2012). tam finds its appropriate applications when a technology is adopted by a user for the first time (yi et al., 2009). besides, the two core exogeneous constructs of tam, perceived usefulness and perceived ease of use include other important beliefs (park, 2009). it is further to mention here that tam has been extended subsequently taking into account another factor perceived enjoyment while another study was conducted for investigation of the use of websites by the users (van der heijden, 2003; chung & tan, 2004). with this conception, we have used these beliefs which are loneliness, depression, perceived enjoyment, perceived usefulness, and perceived ease of use as exogeneous variables. in addition to these exogeneous variables, addiction is also found to have been affected by age and gender of the users (korpinen & paakkonen, 2011; borcham, 2016). 2.2 development of hypotheses and conceptual model from the conception of literature review, different theories, and models, we have identified the salient independent factors that impact addiction of online platforms. now, we will discuss each of the constructs and would take an attempt to formulate some hypotheses that would help us to develop the conceptual model. 2.2.1 loneliness (lon) humans possess a special behavioural characteristic by which humans exhibit an avoidantattachment style. it is perceived to have been closely associated with the feelings of loneliness to impede inter-personal relationship (yi et al., 2012; chatterjee, 2020a). the feelings of being alone or being separated from others bring in the sense of loneliness (tomaka et al., 2006). these individuals like to live alone and even when they are in a community, they feel that they are still alone. use of modern technologies by the individuals helps them to be relieved from the strain of loneliness (wei et al., 2005; jeong et al., 2016). this idea of getting relief from the feelings of loneliness through frequent use of a particular technology leads the individuals to eventually be addicted to that technology (yang & lin, 2014; kim & kim, 2016; chatterjee et al., 2019). this concept has also been supported already in other studies (park & lee, 2012). all these inputs lead to formulate the below-mentioned hypothesis. h1: loneliness (lon) of the individuals has a significant impact towards the addiction of online platforms (aop). 2.2.2 perceived enjoyment (pen) studies of online games helped to infer that hedonic values are related with the concept of enjoyment and it influences user behaviour (yang & lin, 2014). perceived enjoyment is interpreted as the degree to which the concerned activity surrounding the use of a technology is “perceived to be enjoyable in its own right, apart from any performance consequences that may be anticipated” (van der hijden, 2003). we have evidence that perceived enjoyment acts as an effective predictor to ascertain usage behaviour of humans which has been observed in studies conducted to analyse use-behaviour of individuals using social networking services (snss) (han & windsor, 2011). the study helps us to conceptualize that individuals’ feelings australasian journal of information systems chatterjee 2021, vol 25, research on user involvement impact of addiction of online platforms on quality of life 4 of enjoyment or playfulness emerge from the interactions of the users with the environmentrelated factors (lee & tsai, 2010). the feelings of enjoyment emerge from the use of a technology like playing of online games or videos or use of social media and so on which are not related with task-oriented issues (shin, 2010; wei & lu, 2014). people use online platforms for intrinsic as well as extrinsic reasons. intrinsic motivation is related with entertainment and pleasure issues and users of different ages are found to use online platforms even by playing games and so on. this tendency of having entertainment, enjoyment, playfulness and pleasure propels the users to use frequent and uncontrolled use of online platforms instrumental for addiction (lee & tsi, 2010; sarin et al., 2020; grover & kar, 2020). these discussions lead to propose the following hypothesis. h2: perceived enjoyment (pen) of the users has a significant impact on addiction of online platforms (aop). 2.2.3 depression (dep) depression of individuals would bring in addiction under various contexts (yeo et al., 2014; gong et al., 2020). it appears from the different studies that individuals are found to be aligned for being involved using various types of online platforms. this will provide the individuals to be relieved from depression (kim et al., 2007; wang et al., 2019). study was conducted on the korean students about depression. it revealed that if the students suffer from any sort of depression, they are inclined to use different online platforms. this will help them to be relieved from that depression (jeon & jang, 2014). depression is considered as a special mental status of individuals (yeo et al., 2014; chen et al., 2017; lee et al., 2019). the individuals perceive that if they frequently use social media, smartphone, and other online platforms, they might get relief from depression. it leads them to be addicted to that technology (kim & kim, 2016). thus, this mental state of humans finds its solution to remove this strain-oriented trait through the frequent use of any technology (yi et al., 2012). this will help the humans to come out from depression. this frequent use of the technology induces the users to be addicted to that technology (kim et al., 2007). these inputs help us to formulate the following hypothesis. h3: depression (dep) of individuals has a significant impact on addiction of online platform (aop). 2.2.4 perceived ease of use (peu) and perceived usefulness (pu) in consonance with the concept of technology acceptance model (davis et al., 1989), perceived usefulness is defined as “the degree to which an individual believes that using the services will contribute to reaching a particular objective”. it is believed that increased usefulness of a system is associated with the use of the system frequently inviting addiction to it (taylor & todd, 1995). thus, whenever the users perceive that by using online platforms, they will be able to successfully perform their jobs, they will start using the online platform frequently that would lead to addiction to that (bisen & deshpande, 2018; chatterjee et al., 2019). conception of usefulness of a system induces an individual to use that system frequently. it leads to addiction as has been observed in the study of internet addiction (sulaiman et al., 2019). hence, it can be inferred that perceived usefulness might act as an effective predictor of addiction of online platforms. if an individual feels easy and fruitful to use a system, the individual exhibits the tendency to use that system (lee et al., 2007; mard & cui, 2017). this sense and perception of easiness to use a system impacts one’s cognition and it also influences one’s behaviour and the individual does not hesitate to frequently use the system resulting in addiction to that system (kwon et al., 2013). ease of use of a system may be considered an influential predictor for addiction (jiang, 2014). in another study, it has appeared that ease of use has impacted an australasian journal of information systems chatterjee 2021, vol 25, research on user involvement impact of addiction of online platforms on quality of life 5 individual in addiction to internet (arpaci et al., 2018). hence, it can safely be inferred that ease of use may be one of the reasons for addiction with a system. with all these inputs, the following hypotheses are formulated. h4: perceived ease of use (peu) has a significant impact on addiction of online platforms (aop). h5: perceived usefulness (pu) has a significant impact on addiction of online platforms (aop). 2.2.5 addiction of online platforms (aop) and quality of life (ql) addiction to any technology use is construed as an unhealthy behaviour of the individuals. it has been observed that smartphone addiction impacts adversely on the health of the users (hong et al., 2012). here, smartphone acts as a medium to online platform. such type of use of online platforms in an uncontrolled way severely affects the individuals. even, the professionals’ daily works are likely to be hampered if they are always engaged to use online platforms for other purposes (park & lee, 2012; he et al., 2019). this has an adverse effect on the quality of life of the professionals also. frequent usage of the online platforms keeps the users always engaged in the online activities. the users cannot control them to be dissuaded from being engaged in using the online platforms. this will lead to deterioration of the quality of life of the users (costanza et al., 2008; whiteside et al., 2018). here quality of life is associated with personal life and professional life of the users. thus, addiction of online platforms by the users negatively affects their quality of life. these inputs help us to develop the following hypothesis. h6: addiction of online platforms (aop) negatively affects the quality of life (ql). 2.2.6 age and gender as moderators in this study, we have been able to hypothesize five hypotheses affecting addiction of online platforms covering h1, h2, h3, h4, and h5. studies reveal that age factor has a dominating effect to impact on these five linkages affecting addiction of online platforms (cameron, 1969). it has been observed in different studies that age factor of individuals is instrumental to control the individual-centric traits and technological issues that impact the addiction of online platforms (nysveen et al., 2005; jeon & jang, 2014). in the study of cameron (1969), the effect of age has been divided in different age group marking as young-adult having age of 18-35 years, mid-aged adult having age of 36-55 years and older adult having age more than 55 years. in our study, we have categorized the users of online platforms into two categories, young-adult (18-35 years) and mid-aged adult (36-55 years) as number of online users in these two categories are relatively high (csibi et al., 2019; sulaiman et al., 2019). details will be discussed, how individuals of these two categories of age-group affect the different linkages covering hypotheses h1 to h5. these discussions propose to formulate the following hypotheses. h7: age of individuals impacts on the loneliness-addiction of online platforms linkage covering h1. h8: age of individuals impacts on the perceived enjoyment-addiction of online platforms linkage covering h2. h9: age of individuals impacts on the depression-addiction of online platforms linkage covering h3. h10: age of individuals impacts on the ease of use-addiction of online platforms linkage covering h4. h11: age of individuals impacts on the usefulness-addiction of online platforms linkage covering h5. australasian journal of information systems chatterjee 2021, vol 25, research on user involvement impact of addiction of online platforms on quality of life 6 again, this study perceives that like age factor of individuals, the gender factor has also impact on the linkages between the different predictors of addiction with addiction of online platforms as is evident from other studies (van deursen et al., 2015; yang et al., 2018). another study has also revealed that human-centric traits like loneliness, perceived enjoyment, depression affect the women and men with varied extent under identical conditions (ameen et al., 2018). hence, the author has perceived that gender may also be considered as a moderator to moderate the linkages covering h1 to h5. this has led to hypothesize as follows: h12: gender acts as a moderator to impact the loneliness-addiction of online platforms linkage covering h1. h13: gender acts as a moderator to impact the perceived enjoyment-addiction of online platforms linkage covering h2. h14: gender acts as a moderator to impact the depression-addiction of online platforms linkage covering h3. h15: gender acts as a moderator to impact the ease of use-addiction of online platforms linkage covering h4. h16: gender acts as a moderator to impact the usefulness-addiction of online platforms linkage covering h5. after formulation of these hypotheses, the conceptual model is developed and is shown in figure1. figure 1: the conceptual model 3 research methodology the hypotheses and the conceptual model are required to be statistically tested. the conceptual model shows that the number of independent variables is more than the number of dependent variables. hence, it is cogent to apply partial least square (pls) – structural equation modeling (sem) technique (abdi, 2010). the pls-sem approach involves survey to obtain feedbacks from the usable respondents against some structured set of questions (questionnaire) in the form of statements (measurement instruments). australasian journal of information systems chatterjee 2021, vol 25, research on user involvement impact of addiction of online platforms on quality of life 7 3.1 measurement instruments the concepts of the constructs have helped us to prepare the set of questions (questionnaire) which could ensure content validity. the process of preparing the questionnaire was undertaken through some step-by-step correctional approaches. it includes thematic literature analysis, in depth interpretation of the constructs, qualitative studies, pre-test, opinion of some experts specialized in the domain of this study and pilot test (carpenter, 2018). in this way, 33 questions in the form of statements were prepared. the details of the measurement instruments along with the sources are provided in a tabular form in appendix 1. 3.2 data collection strategy to collect the names and email ids of the prospective online users, we attended some conferences held in different parts of india. in those conferences, discussions took place on the subject that covers the domain of this study. in these conferences, we could target some resource persons. they helped us by supplying lists of different potential online users with varied age, qualification and so on. their supplied lists contained 906 potential respondents. we took their consent for providing us the feedbacks against those 33 questions. they were requested to provide their responses within three months (october 2019 to december 2019). we sent them two reminders in the meantime with a request to provide their responses in time. we received 391 responses within the stipulated period. the response rate is 43.1%. it was observed that out of 391 responses, 71 responses were incomplete. these were perceived not to yield any effective result through analysis. we did not consider those 71 responses. we started our statistical analysis with 320 usable responses against 33 questions. these are well within the permissible range (deb & david, 2014). the demographic information of the respondents is shown in table 1. particulars characteristics number percentage (%) gender male 224 70 female 96 30 age young adult (18-35 years) 192 60 mid-aged adult (36-55 years) 128 40 education undergraduate 48 15 graduate 70 22 postgraduate 144 45 researcher/phd 58 18 table 1: demographic information of the respondents 4 data analysis and results 4.1 data analysis for validity and reliability for identifying the convergent validity of the 33 research instruments, loading factor (lf) of each item has been measured. for ascertaining reliability, consistency, validity of the constructs, we have estimated average variance extracted (ave), cronbach’s alpha (α) and composite reliability (cr) of each construct respectively (fornell & larcker, 1981). to examine multicollinearity defects, variance inflation factor (vif) of each construct has been estimated. on computation, it appears that all the parameters are within the acceptable range. this confirms that the instruments are reliable, and the constructs are reliable, consistent, and australasian journal of information systems chatterjee 2021, vol 25, research on user involvement impact of addiction of online platforms on quality of life 8 valid. the constructs do not suffer from the defects of multicollinearity. the entire results of measurement property are shown in table 2. moderator / construct mean sd lf t-value ave alpha (α) cr vif number of items age 4.71 1.46 1.00 1.00 1.00 gender 3.81 1.87 1.00 1.00 1.00 lon .84 .88 .91 4.8 5 lon1 3.2 1.2 .85 26.41 lon2 4.1 1.4 .99 25.27 lon3 4.6 1.9 .97 39.11 lon4 3.8 1.8 .88 27.01 lon5 2.9 1.1 .85 19.48 pen .86 .94 .96 3.7 5 pen1 2.7 1.4 .84 21.21 pen2 2.9 1.3 .98 18.78 pen3 3.8 1.9 .90 12.11 pen4 3.9 1.9 .96 26.12 pen5 4.8 1.5 .95 29.52 dep .81 .91 .98 3.3 5 dep1 4.8 1.6 .87 27.48 dep2 4.8 1.6 .88 22.12 dep3 4.1 1.9 .90 18.99 dep4 3.1 1.1 .84 29.37 dep5 4.5 1.1 .99 21.00 peu .84 .86 .97 4.9 5 peu1 3.2 1.6 .90 26.81 peu2 3.5 1.4 .87 27.48 peu3 4.1 1.7 .88 29.01 peu4 3.8 1.7 .93 37.35 peu5 4.9 1.1 .99 26.81 pu .87 .90 .92 3.7 5 pu1 3.6 1.8 .88 28.21 pu2 3.2 1.7 .89 19.34 pu3 3.0 1.2 .99 23.11 pu4 2.8 1.4 .95 26.42 pu5 3.7 1.5 .84 29.03 aop .82 .89 .93 3.9 4 aop1 2.9 1.3 .95 26.41 aop2 4.9 1.4 .90 19.34 aop3 4.7 1.7 .88 28.95 aop4 3.1 1.9 .89 21.06 ql .84 .92 .94 4.9 4 ql1 3.7 1.8 .99 24.81 ql2 4.8 1.8 .97 27.21 ql3 3.9 1.6 .85 21.11 ql4 4.4 1.4 .84 19.39 table 2: measurement properties 4.2 discriminant validity test it is always expected that each item should strongly interpret its own construct and would weakly explain the other constructs. to examine this, discriminant validity test is needed to australasian journal of information systems chatterjee 2021, vol 25, research on user involvement impact of addiction of online platforms on quality of life 9 be conducted (fornell & larcker, 1981). for this, correlation coefficients for each construct with other constructs have been computed. it is found that they all are less than the corresponding square root of ave of each construct. this confirms discriminant validity. the entire results are shown in table 3. moderator/ construct age gender lon pen dep peu pu aop ql ave age single item 1.00 gender .24 single item 1.00 lon .32 .27** .92 .84 pen .26* .36 .27 .93 .86 dep -.29 .38 .29* .28* .90 .81 peu .34 -.22 .41** .27* .32 .92 .84 pu .24 .32* .43 .24* .37* .31 .93 .87 aop .32** -.44 .36 -.36** .41 .36* -.33 .91 .82 ql -.26 .41 -.21 .41 -.43 .47 .37* .39 .92 .84 table 3: discriminant validity test 4.3 path analysis using structural equation modeling (sem) to ascertain how the latent variables are related, sem is conducted. it also confirms if the model is in order or not. for this, root mean square error (rmse), ratio of chi square and degree of freedom, comparative fit index (cfi), normed fit index (nfi), and tucker–lewis index (tli) have been measured. their estimated values are 0.03, 2.041, 0.97, 0.99 and 0.98 respectively which all are within the permissible range as the rmse has maximum value 0.07 (steiger, 2007), chi square and degree of freedom ratio has the maximum value 3 (kline, 2005), cfi has the minimum value 0.93 (hair et al., 2006), nfi has the lowest permissible value 0.95 (schumacker & lomex, 2004) and tli has the minimum value 0.95 (sharma et al., 2005). hence, the model after validation is in order. through this, we have been able to find out the coefficient of determinants (r2), p-values and path coefficients. the entire results are shown in table 4. measure r2 values hypotheses path coefficients p-values remarks effect on aop 0.52 by lon h1 0.32 *(p<0.05) supported by pen h2 0.46 *(p<0.05) supported by dep h3 0.39 **(p<0.01) supported by peu h4 0.46 *(p<0.05) supported by pu h5 0.52 *(p<0.05) supported effect on ql 0.71 by aop h6 -0.49 ***(p<0.001) supported table 4: r2 values, p-values, path coefficients and remarks the conceptual model after validation has been shown in figure 2. australasian journal of information systems chatterjee 2021, vol 25, research on user involvement impact of addiction of online platforms on quality of life 10 figure 2: validated model by pls-sem analysis, we have found out the path weights of all the direct linkages. also, we have been able to estimate the path weights of the two moderators affecting the five linkages covering h1, h2, h3, h4 and h5. the entire results are shown in table 5. path path weight p-value hypothesis remarks lon→aop .32 *(p<0.05) h1 supported pen→aop .46 *(p<0.05) h2 supported dep→aop .39 **(p<0.01) h3 supported peu→aop .46 *(p<0.05) h4 supported pu→aop .52 *(p<0.05) h5 supported aop→ql -.49 ***(p<0.001) h6 supported (lon→aop) × age .21 *(p<0.05) h7 supported (pen→aop) × age .32 *(p<0.05) h8 supported (dep→aop) × age .34 **(p<0.01) h9 supported (peu→aop) × age .46 **(p<0.01) h10 supported (pu→aop) × age .27 ***(p<0.001) h11 supported (lon→aop) × gender .22 **(p<0.01) h16 supported (pen→aop) × gender .36 **(p<0.01) h15 supported (dep→aop) × gender .47 *(p<0.05) h14 supported (peu→aop) × gender .39 *(p<0.05) h13 supported (pu→aop) × gender .52 **(p<0.01) h12 supported table 5: path weights with consideration of moderators 4.4 moderator analysis to examine the moderating effects of the moderators (age and gender) on the five linkages covering h1, h2, h3, h4 and h5, a multi group analysis (mga) has been conducted. for this, help of smart-pls with the use of accelerated bias corelated bootstrapping with consideration of 6000 resamples has been taken for computing the path coefficient differences and p-value australasian journal of information systems chatterjee 2021, vol 25, research on user involvement impact of addiction of online platforms on quality of life 11 differences relating to two selected categories for each moderator (for age, two categories are considered, young adults covering range of age as 18-35 years and mid-aged adults covering range of age 36-55 years, for gender obviously two categories are considered, male and female). the effects of the moderators are perceived to be significant if the difference of pvalues for each moderator concerning to two categories as mentioned above, becomes either smaller than 0.05 or greater than 0.95 (hair et al., 2016). the results highlight that both the moderators have significant effects on each of the five linkages that cover h1, h2, h3, h4 and h5. the results are shown in table 6. path moderator diff. of path coefficients diff. of p-values remarks lon →age →aop age 0.116 0.002 significant pen → age →aop age 0.121 0.004 significant dep → age →aop age 0.132 0.042 significant peu → age →aop age 0.115 0.032 significant pu → age →aop age 0.124 0.014 significant lon → gender →aop gender 0.152 0.004 significant pen → gender →aop gender 0.141 0.007 significant dep → gender →aop gender 0.116 0.016 significant peu → gender →aop gender 0.143 0.032 significant pu → gender →aop gender 0.139 0.024 significant table 6: multi group analysis (mga) for testing moderator effects 4.5 common method bias (cmb) since the study stands on the foundation of self-reported data, it is essential to see if these data are suffering from the defects of cmb. however, for reducing the possibility of occurrence of cmb, at the time of survey, the respondents were assured of confidentiality as well as anonymity. to alleviate this, a post-hoc harman’s single factor test has been conducted. on verification, it appears that the first factor emerged for 41.3% of the variance. it is less than the cut-off (highest) value 50% as recommended by podsakoff et al. (2003). thus, the results highlight that there is no cmb in this study. as such, there is no chance of distortion of the predicted results of this study. 4.6 results for data analysis through studies of literature and by conceptualizing the concerned theories, it has been possible to hypothesize six hypotheses. on statistical verification through pls-sem analysis, all these hypotheses have been supported. the exogeneous independent variables lon, pen, dep, peu and pu impact on the aop significantly that in turn impacts ql, the goal of this study. of course, the impact of aop on ql appears to be strong but negative in nature as the concerned path coefficient is -.49 with level of significance p<0.001(***). among the effects of the exogeneous variables on aop, the effect of impact on aop by pu is the maximum as the concerned path coefficient is 0.52 with level of significance p<0.05(*). in this study, we have separately considered the effects of two moderators age and gender on the five linkages which are lon→aop (h1), pen→ aop (h2), dep→aop (h3), peu→aop (h4) and pu→ aop (h5). so far as impacts of the moderator age on these five linkages are concerned, the impact of the moderator age on the linkage peu→aop (h4) is the highest as the concerned path coefficient is 0.46 with the level of significance p<0.01(**). so far as effects of the moderator age on the other four linkages are concerned, it appears that the effects are all significant as australasian journal of information systems chatterjee 2021, vol 25, research on user involvement impact of addiction of online platforms on quality of life 12 the concerned path-coefficients are 0.21 (p<0.05) for the linkage lon→aop (h1), 0.32 (p<0.05) for the linkage pen→aop (h2), 0.39 (p<0.01) for the linkage dep→aop (h3) and 0.27 (p<0.001) for the linkage pu→aop (h5). so far as effects of the other moderator, gender are concerned, it appears that this moderator has the highest effect on the linkage dep→aop (h3) as the concerned path coefficient is the highest and it is 0.47 (p<0.05). so far as coefficients of determinant are concerned, it appears that lon, pen, dep, peu and pu can simultaneously explain aop to the extent of 52% since the concerned r2=0.52. moreover, the aop can explain the ql (goal of this study) to the tune of 71% as the concerned r2=0.71. the explanative power of the model is 71% which is considerably high. 5 discussion and key findings feelings to be alone or to be separated is instrumental with the concept and feeling of forlornness. it arises because of having inadequate societal relationship. this concept gives rise to the feelings of loneliness. the study hypothesized that the loneliness impacts addiction of online platforms and it has been supported by statistical validation. this concept has been supported by earlier studies (tomaka et al., 2006; yang & lin, 2014). this study hypothesized that perceived enjoyment positively impacts addiction of online platforms which has been validated through statistical analysis. this concept received sanction from earlier studies wherein it has been envisaged that enjoyment is conceptualized as an extent to use a technology which is thought to be enjoyable regardless of any performance consequence that may be anticipated (van der heijden, 2003). depression, being a special mental status of individuals, has an effective impact on addiction of online platforms as is clear from the validated hypothesis h3. this concept has been supplemented by earlier studies (jeon & jang, 2014) where the study analysed the conduct of some korean students. it highlighted that if the students suffer from depression, they are forced to be intimately involved in a frequent use of technology. this they do to be relieved from depression. this eventually leads to addiction in that technology. perceived ease of use and perceived usefulness both impact on addiction of online platforms as has been transpired from this study (h4 and h5). these two constructs are the core constructs of technology acceptance model (davis et al., 1989) and many studies subscribed that these two beliefs have a considerable impact on use of a technology and such use of technology eventually gets the users addicted to that technology (taylor & todd, 1995; lee et al., 2007; arpaci et al., 2018; sulaiman et al., 2019). studies reveal that addiction of online platforms negatively impacts quality of life. this has too received support from earlier studies (kingpadung & phusavat, 2010). now we shall discuss how age and gender affect (as moderators) the addiction of online platforms through the five exogeneous predictors lon, pen, dep, peu and pu impacting aop. this covers the linkages concerning to h1, h2, h3, h4 and h5. first, the effects of age of the users on these five linkages will be analysed. in this analysis, we shall categorize the two age groups of the online users, one is young adult (18-35 years), and another is med-aged adult (36-55 years). this idea has been lent from cameron (1969) as already mentioned earlier. the effects will be discussed graphically. we shall analyse the effects of the two age groups of the users in the five linkages arising out of the predictors-addiction linkages covering h1, h2, h3, h4 and h5. the presentation has been projected here through five separate graphs, collectively marked as figure 3. australasian journal of information systems chatterjee 2021, vol 25, research on user involvement impact of addiction of online platforms on quality of life 13 figure 3: effects of addiction by the moderator, age we shall discuss the effects of age through two categories which are young-adult and midaged adult on the different linkages lon→aop, pen→aop, dep→aop, pue→aop and pu→aop in a brief way. this has been shown through five graphical representations. in each graph, the continuous line represents young-adult character whereas dotted line represents mid-aged character. with increase of loneliness, the increase of addiction affinity is more for young adult compared to the mid-aged adult as the gradient of the continuous line is greater compared to that of the dotted line. young adults are more prone to addiction with the increase australasian journal of information systems chatterjee 2021, vol 25, research on user involvement impact of addiction of online platforms on quality of life 14 of feelings of loneliness. same is the situation when effects of young-adult users and mid-aged adult users are studied relating to effects of perceived enjoyment (pen) on the addiction of online platform (aop). here, it appears that with increase of feelings of perceived enjoyment, the rate of young-aged users’ affinity for addiction to online platforms increases more compared to the case of mid-aged adults since the dotted line representing mid-aged adult character bears less gradient compared to the continuous line representing the case of young adults. but effects of age in two categories as moderators on the dep→aop (h3) linkage show a reverse result. here, sense of depression is seen to have affected much on the mid-aged adults. that is why the dotted line representing mid-aged adults character bears greater inclination (higher gradient) compared to the continuous line representing the case of young adults. the mid-aged adults are found more sensitive to depression compared to young adult. similarly, the effects of age on the peu→aop (h4) linkage show that with increase of ease of use, the mid-aged adults become more addicted compared to young adults. easiness affects in such a way as with feelings of easiness, the young adults do not exhibit much addictive affinity like mid-aged adults. same is the case when age acts as a moderator on the pu→aop (h5) linkage. dotted line representing the nature of mid-aged adults bears greater gradient showing that with increase of pu, the rate of increase of aop is more for mid-aged adults in comparison to young adults. we shall now discuss the effects of gender as a moderator in the five linkages covering h1, h2, h3, h4 and h5. here, obviously two categories of gender have been considered, male and female. these are shown in different graphs collectively marked as figure 4. australasian journal of information systems chatterjee 2021, vol 25, research on user involvement impact of addiction of online platforms on quality of life 15 figure 4: effects of addiction by the moderator, gender the continuous lines in all the graphs represent the effects of male whereas the dotted lines in all the graphs represent the effects of female in this context. so far as effects of loneliness in addiction of online platforms are concerned, it appears that with increase of feelings of loneliness, the rate of addiction is increased for female more compared to male as the gradient of the dotted line is greater than the gradient of the continuous line. the situation is reversed while considering the effects of gender moderator on the pen→aop (h2) linkage. here, with increase of perceived enjoyment (pen), the rate of growth of addiction for male is more compared to female as the gradient of the continuous line is greater than the gradient of the dotted line. interestingly, in case of effects of gender moderator on dep→aop (h3) and pu→aop (h5) linkages, the situation appears to be almost identical. the rate of increase of addiction tendency with increase of feelings of depression and perceived usefulness remains the same for male and female and that is why these two graphs represent almost two parallel lines. again, with increase of peu, the rate of increase of aop appears to be more in case of male compared to female as the gradient of the continuous line is greater than the gradient of the dotted line. 5.1 theoretical contributions this research study could yield an enriched and comprehensive theoretical understanding by bringing connection between loneliness, perceived enjoyment, depression, perceived ease of use and perceived usefulness with addiction of online platforms. this theoretical model has been able to provide effective inputs as to how the addiction of online platforms adversely affects the quality of life of individuals and how the age and gender factors moderate the impacts of different predictors on the addiction of online platforms. the theoretical model has been developed lending inputs from some addiction theories. there are many theories, but, from those theories, adult attachment theory, tension reduction theory and technology acceptance model have been selected to identify successfully the factors affecting the addiction of online platforms. this is a specific theoretical contribution of this study. moreover, to identify the predictors of addiction of online platforms, we did not consider many factors, but selected some better suited specific factors with the help of the already mentioned theories. by selecting such antecedents, it was possible to achieve 71% explanative power of the model. this is also considered as one of the theoretical contributions of this study. by selecting two core constructs (perceived ease of use and perceived usefulness) of acceptance model, it has been possible to include many vital beliefs since perceived ease of use includes self-efficacy, simplicity and compatibility (yi et al., 2009) whereas perceived usefulness australasian journal of information systems chatterjee 2021, vol 25, research on user involvement impact of addiction of online platforms on quality of life 16 includes some other vital determinants like performance, trust, effectiveness, risk perception and productivity (turner et al., 2010). hence, by inclusion of these two beliefs, it has been possible to consider at least eight vital factors. this is also considered as one of the important theoretical contributions of this study. this study also included one exogeneous factor perceived enjoyment. this belief has been included in the extended technology acceptance model as is transpired from the other studies (van der heijden, 2003). consideration of this belief (perceived enjoyment) is claimed to have strengthened the performance of the model. this may also be construed to be another theoretical contribution of this study. moreover, consideration of two moderators, age and gender towards their impacts on the predictorsaddiction linkages could strengthen the model by achieving its explanative power as high as 71%. this is also claimed to be another theoretical contribution of this study. 5.2 practical implications this study has highlighted that excessive usage of online platform by the users brings addiction to it. it affects the quality of life of the users adversely. addictive users cannot control their behaviour towards repeated usage of a technology (hong et al., 2012). hence, the policymakers are needed to take appropriate actions towards developing the sense of the addicted users of online platforms. this would help to control their usage habits. it will help them not to be eventually addicted in the online frequent usage activities. the policy makers also have the responsibilities to structure appropriate policies that would restrict the users to use online platforms frequently. this may be done by limiting their usage for a certain specific period per day. this could restrict the habit of the online users not to use online platforms frequently. it would help the users not to be addicted to it. it has been observed in other studies that for restricting addiction to smartphones, which is considered as one of the online platforms, an app has been developed (ko et al., 2015). the authority should take initiatives to provide a devise or otherwise in this line to restrict the users to use excessive other online platforms that could cause addiction detrimental to the quality of life. in brief, the authority is needed to think how the self-controlling skills of the users towards using online platforms can be developed. to achieve this, the authorities needed to take appropriate steps to frame implementable and consistent regulations for restricting the frequent usage of the online platforms. this would help the users not to be addicted in using online platforms. it eventually deteriorates the quality of life. addiction is a menace to the society. awareness among the users of online platforms is required to be grown regarding the dark sides of addiction of the online platforms. for this, the authority is needed to conduct awareness programs frequently with regular interval. this awareness program is considered vital. addiction is related with such a behavioural trait of an individual that the individual can hardly control themselves even knowing its inimical consequences. hence, development of awareness among the users of online platforms is perceived to be an effective treatment of this ailment. 5.3 limitations and future scope all the research studies have some limitations. likewise, this study has also some limitations. in the validation stage through pls-sem analysis, survey was conducted considering the respondents from metropolitan and semi-urban areas. no respondents were considered from rural areas. but presently, even in all developing countries, online usage has become ubiquitous and it has spread in almost all areas regardless of their nature. non consideration of respondents from rural areas does not project the result in a generic form. in the survey, we australasian journal of information systems chatterjee 2021, vol 25, research on user involvement impact of addiction of online platforms on quality of life 17 used the feedbacks of 320 respondents. this is inadequate to represent the whole society. inputs of people from all categories possessing varied socio-economic status ought to have been considered. the model provided could achieve 71% explanative power. ideally, to make it near to 100%, we could have considered inclusion of other boundary conditions in this model which has not been done in our study. thus, it is seen that there are many limitations of this study. it is expected that the future researchers would take up these uncovered issues to give this study a full proof shape. references abdi, h. (2010). partial least squares regression and projection on latent structure regression (pls regression). wires computational statistics, 2(1), 97-106. https://doi.org/10.1002/wics.51. arpaci, i., kesici, ş. & baloğlu, m. (2018). individualism and internet addiction: the mediating role of psychological needs. internet research, 28(2), 293-314. https://doi.org/10.1108/ intr-11-2016-0353. aslanbay, y., aslanbay, m. & çobanoğlu, e. (2009). internet addiction among turkish young consumers. young consumers, 10(1), 60-70. https://doi.org/10.1108/17473610910940792. augner, c., & hacker, g. w. (2012). associations between problematic mobile phone use and psychological parameters in young adults. international journal of public health, 57(2), 437-441. https://doi.org/10.1007/s00038-011-0234-z. baccarella, christian & wagner, timm & kietzmann, jan & mccarthy, ian. (2018). social media? it's serious! understanding the dark side of social media. european management journal, 36. 431-438. https://doi.org/10.1016/j.emj.2018.07.002. bisen, s. & deshpande, y. (2018). understanding internet addiction: a comprehensive review. mental health review journal, 23(3), 165-184. https://doi.org/10.1108/mhrj-07-2017-0023. boreham, m. (2016). transforming addiction: gender, trauma, transdisciplinary. drugs and alcohol today, 16(3), 228-229. https://doi.org/10.1108/dat-07-2015-0037. cameron, p. (1969). age parameters of young adult, middle-aged, old, and aged. journal of gerontology, 24(2), 201–202. https://doi.org/10.1093/geronj/24.2.201. cappell, h., & greeley, j. (1987). alcohol and tension reduction: an update on research and theory. in h. blane & k. leonard (eds.). psychological theories of drinking and alcoholism, 1554. new york: guilford press. carlson, b. l., farrelly, t., frazer, r., & borthwick, f. (2015). mediating tragedy: facebook, aboriginal peoples and suicide. australasian journal of information systems, 19. https://doi.org/10.3127/ajis.v19i0.1174. carlson, b. l., jones, l. v., harris, m., quezada, n., & frazer, r. (2017). trauma, shared recognition and indigenous resistance on social media. australasian journal of information systems, 21. https://doi.org/10.3127/ ajis.v21i0.1570. carpenter, s. (2018). ten steps in scale development and reporting: a guide for researchers. communication methods and measures, 12(1), 25-44. https://doi.org/10.1080/19312458. 2017.1396583. australasian journal of information systems chatterjee 2021, vol 25, research on user involvement impact of addiction of online platforms on quality of life 18 chatterjee, s. (2020a). internet of things and social platforms: an empirical analysis from indian consumer behavioural perspective. behaviour & information technology, 39(2), 133149, https://doi.org/10.1080/0144929x.2019.1587001. chatterjee, s., chaudhuri, r., vrontis, d., thrassou, a., ghosh, s. & chaudhuri, s. (2020b). social customer relationship management factors and business benefits. international journal of organizational analysis, in press. https://doi.org/10.1108/ijoa-11-2019-1933. chatterjee, s., ghosh, s., chaudhuri, r. & nguyen, b. (2019). are crm systems ready for ai integration? a conceptual framework of organizational readiness for effective ai-crm integration, the bottom line, 32(2) 144-157. https://doi.org/10.1108/bl-02-2019-0069. chen, b., liu, f., ding, s., ying, x., wang, l., & wen, y. (2017). gender differences in factors associated with smartphone addiction: a cross-sectional study among medical college students. bmc psychiatry. ncbi. 17(1), 341-347. https://doi.org/10.1186/s12888-017-1503z. choi, y.y. & seo, y.s. (2015). the relationship between insecure adult attachment and smartphone addiction: the mediation effect of impulsiveness moderated by social support. korean journal psychological counselling, 27(3), 49-72. choliz, m. (2012). mobile-phone addiction in adolescence: the test of mobile phone dependence (tmd). progress in health science, 2(1), 33-44. chung, j. & tan, f.b. (2004). antecedents of perceived playfulness: an exploratory study on user acceptance of general information-searching websites. information & management, 41(7), 869–881. https://doi.org/10.1016/j.im.2003.08.016. collier, m.r. (1993). a mathematical model of habituation and addiction. international journal of the addictions, 28, 175–185. https://doi.org/10.3109/10826089309039622. costanza, r., fisher, b., ali, s., beer, c. & bond, l. (2008). an integrative approach to quality of life measurement, research, and policy. sapiens, 1(1), 17-21. https://doi.org/10.5194/ sapiens-1-11-2008. csibi, s., griffiths, m.d., & demetrovics, z. et al. (2019). analysis of problematic smartphone use across different age groups within the ‘components model of addiction’. international journal of mental health addiction. earlysite. https://doi.org/10.1007 › s11469019-00095-0. davis, f.d., bagozzi, r., & warshaw, p. (1989). user acceptance of computer technology: a comparison of two theoretical models. management science, 35(8), 982–1003. https://doi.org/10.1287/mnsc.35.8.982. deb, m. & david, e.l. (2014). an empirical examination of customer’s adoption of mbanking in india. marketing intelligence & planning, 32 (4), 475-494. https://doi.org/10.1108/mip-07-2013-0119. dolan, r., conduit, j., fahy, j., & goodman, s. (2016). social media engagement behaviour: a uses and gratifications perspective. journal of strategic marketing, 24(3/4), 261-277. https://doi.org/10.1080/0965254x.2015.1095222. australasian journal of information systems chatterjee 2021, vol 25, research on user involvement impact of addiction of online platforms on quality of life 19 dolan, r., goodman, s. & habel, c. (2012). engaging friends and influencing people: consumer perceptions of brand communication strategies on facebook. in: proceedings of australian & new zealand marketing academy conference. adelaide, australia. e marketer, (2016). nearly one-third of the world will use social networks regularly this year (online). available at: https://www.foliomag.com/report-finds-nearly-one-thirdworlds-population-will-use-social-networks-2016/ (accessed on 22 march 2020) fornell, c., & larcker, d. f. (1981). evaluating structural equation models with unobservable variables and measurement error. journal of marketing research, 18(1), 39–50. https://doi.org/10.2307/3151312. fox, j., & moreland, j. j. (2015). the dark side of social networking sites: an exploration of the relational and psychological stressors associated with facebook use and affordances. computers in human behavior, 45, 168–176. https://doi.org/10.1016/j.chb.2014.11.083. fraley, r.c. & shaver, p.r. (2000). adult romantic attachment: theoretical developments, emerging controversies, and unanswered questions. review of general psychology. 4(2), 132–154. https://doi.org/10.1037/1089-2680.4.2.132. gong, m., xu, m., luqman, a., yu, l. & masood, a. (2020). understanding the role of individual differences in mobile sns addiction. kybernetes, ahead-of-print. https://doi.org/10.1108/k-05-2019-0367. grover, p. & kar, a.k. (2020). user engagement for mobile payment service providers – introducing the social media engagement model. journal of retailing and consumer services, 53(3), in press. https://doi.org/10.1016/j.jretconser.2018.12.002. grover, p., kar, a.k. & davies, g. (2018). “technology enabled health” – insights from twitter analytics with a socio-technical perspective. international journal of information management, 43(12), 85-97. https://doi.org/10.1016/j.ijinfomgt.2018.07.003. grover, p., kar, a.k., & ilavarasan, p.v. (2019). impact of corporate social responsibility on reputation—insights from tweets on sustainable development goals by ceos. international journal of information management, 48, 39-52. https://doi.org/10.1016/j.ijinfomgt. 2019.01.009. hair, j. f., jr., hult, g. t. m., ringle, c., & sarstedt, m. (2016). a primer on partial least squares structural equation modeling (pls-sem) (2nd ed.). thousand oaks: sage publications. hair, j., black, w., babin, b., anderson, r. & tatham, r. (2006). multivariate data analysis. 6th edition pearson: prentice hall, new jersey, usa. han, b., & windsor, j. (2011). user’s willingness to pay on social network sites. journal of computer information systems, 51(4), 31-40. https://doi.org/10.1080/08874417.2011. 11645499. hassanbeigi, a., askari, j., hassanbeigi, d. & pourmovahed, z. (2013). the relationship between stress and addiction. 3rd world conference on psychology, counselling and guidance (wcpcg-2012). procedia social and behavioral sciences. 84, 1333 – 1340. he, h., liu, y. & zhou, z. (2019). how does weibo keep users hooked? a weibo addictive behavior study based on netnography. journal of contemporary marketing science, 2(2), 176-195. https://doi.org/10.1108/jcmars-01-2019-0013. australasian journal of information systems chatterjee 2021, vol 25, research on user involvement impact of addiction of online platforms on quality of life 20 hong, f. y., chiu, s. i., & huang, d. h. (2012). a model of the relationship between psychological characteristics, mobile phone addiction and use of mobile phones by taiwanese university female students. computers in human behavior, 28(6), 2152-2159. https://doi.org/10.1016/ j.chb.2012.06.020. jeon h.s. & jang s.o. (2014). a study on the influence of depression and stress on smartphone addiction among university students: focusing on moderating effect of gender. korean journal youth studies. 21(8), 103-29. jeong, s. h., kim, h., yum, j. y., & hwang, y. (2016). what type of content are smartphone users addicted to? sns vs. games. computers in human behavior, 54, 10-17. https://doi.org/10.1016/j.chb.2015.07.035. jiang, q. (2014). internet addiction among young people in china: internet connectedness, online gaming, and academic performance decrement. internet research, 24(1), 2-20. https://doi.org/10.1108/intr-01-2013-0004. kim, e., kim, e.j., & cho, c.i. (2017). structural equation model of smartphone addiction based on adult attachment theory: mediating effects of loneliness and depression. asian nursing research. 11(2), 92-97. pubmed id: 28688505. kim, h.j., & kim, e.h. (2016). the relationship between anxious attachment and smartphone addiction: the mediating effects of loneliness and college adjustment. korean journal of counselling. 17(2), 157-161. kim, y., & srivastava, j. (2007). impact of social influence in e-commerce decision making. proceedings of the ninth international conference on electronic commerce. minneapolis, mn, 293–302. kingpadung, k., & phusavat, k. (2010). examinations and testing the impacts of quality of work life on productivity. technical report, faculty of engineering, kasetsart university, bangkok. kline, r. b. (2005), principles and practice of structural equation modeling. new york, guilford. ko, m., yang, s., lee, j., heizmann, c., jeong, j., & lee, u. (2015). nugu: a group-based intervention app for improving self-regulation of limiting smartphone use. in proceedings of the 18th acm conference on computer supported cooperative work & social computing. acm. 1235-1245. https://doi.org/10.1145/2675133.2675244. korpinen, l. & pääkkönen, r. (2011). working-aged population's mental symptoms and the use of the internet. mental illness, 3(1), 25-28. https://doi.org/10.4081/mi.2011.e7. kumar, r., sachin, a., & kumar, r. (2020). the impact of service delivery system process and moderating effect of perceived value in internet banking adoption. australasian journal of information systems, 24, 1-22. https://doi.org/10.3127/ajis.v24i0.1923. kwon, m., lee, j. y., won, w. y., park, j. w., min, j. a., hahn, c., gu, x., choi, j. h. & kim, d. j. (2013). development and validation of a smartphone addiction scale (sas). plos one, 8(2). https://doi.org/10.1371/journal.pone.0056936. lee, j., ko, d. & lee, h. (2019). loneliness, regulatory focus, inter-personal competence, and online game addiction: a moderated mediation model. internet research, 29(2), 381-394. https://doi.org/10.1108/intr-01-2018-0020. australasian journal of information systems chatterjee 2021, vol 25, research on user involvement impact of addiction of online platforms on quality of life 21 lee, m.-c. & tsai, t.-r. (2010). what drives people to continue to play online games? an extension of technology model and theory of planned behavior. international journal of human-computer interaction, 26(6), 601-620. https://doi.org/10.1080/10447311003781318. lee, y. k., chang, c. t., lin, y., & cheng, z. h. (2014). the dark side of smartphone usage: psychological traits, compulsive behavior and technostress. computers in human behavior, 31, 373-383. https://doi.org/10.1016/j.chb.2013.10.047. lee, y., lee, z., & kim, y. (2007). understanding personal web usage in organizations. journal of organizational computing and electronic commerce, 17(1), 75-99. https://doi.org/ 10.1080/10919390701291067. mckinley, a. c., ritchie, d., & burmeister, o. k. (2016). a post publication review of “tragedy: facebook, aboriginal peoples and suicide” australasian journal of information systems, 20. https://doi.org/10.3127/ajis.v20i0.1324. mrad, m. & cui, c. (2017). brand addiction: conceptualization and scale development. european journal of marketing, 51(11/12), 1938-1960. https://doi.org/10.1108/ejm-10-20160571. nysveen, h., pedersen, p., & thorbjornsen, h. (2005). explaining intention to use mobile chat services: moderating effects of gender. journal of consumer marketing, 22(5), 247–256. https://doi.org/10.1108/07363760510611671. park, n., & lee, h. (2012). social implications of smartphone use: korean college students’ smartphone use and psychological well-being. cyberpsychology, behavior, and social networking, 15(9), 491-497. https://doi.org/10.1089/cyber.2011.0580. podsakoff, p. m., mackenzie, s. b., lee, j. y., & podsakoff, n. p. (2003). common method biases in behavioral research: a critical review of the literature and recommended remedies. journal of applied psychology, 88(5), 879. https://doi.org/10.1037/00219010.88.5.879. ram, j., & zhang, z. (2018). an investigation into failure of internet firms: towards development of a conceptual model. australasian journal of information systems, 22. https://doi.org/ 10.3127/ajis.v22i0.1622. sarin, p., kar, a.k., kewat, k. & ilavarasan, p.v. (2020). factors affecting future of work: insights from social media analytics. procedia computer science, 167, 1880-1888. https://doi.org/10.1016/j.procs.2020.03.207. schumacker, r. e., & lomax, r. g. (2004). a beginner's guide to structural equation modeling (2nd ed.). lawrence erlbaum associates publishers. sharma, s., mukherjee, s., kumar, a. & dillon, w. r. (2005). a simulation study to investigate the use of cutoff values for assessing model fit in covariance structure models. journal of business research, 58(7), 935-943. https://doi.org/10.1016/j.jbusres.2003.10.007. shelton, ashleigh & skalski, paul. (2014). blinded by the light: illuminating the dark side of social network use through content analysis. computers in human behavior, 33. 339-348. https://doi.org/10.1016/ j.chb.2013.08.017. australasian journal of information systems chatterjee 2021, vol 25, research on user involvement impact of addiction of online platforms on quality of life 22 shin, d.h. (2010). the dynamic user activities in massive multiplayer online role-playing games. international journal of human-computer interaction, 26(4), 317-344. https://doi.org/ 10.1080/10447310903575473. soral, p., arayankalam, j. & pandey, j. (2020). the impact of ambivalent perception of bureaucratic structure on cyberloafing. australasian journal of information systems, 24, https://doi.org/10.3127/ajis.v24i0.2087 steiger, j. h. (2007). understanding the limitations of global fit assessment in structural equation modeling. journal of personality and individual differences, 42(5), 893-898. https://doi.org/10.1016/j.paid.2006.09.017. sulaiman, a., shin, k. & rofaie, n. (2019). personality traits and internet addiction among selected financial institution employees. international journal of ethics and systems, 35(2), 260-271. https://doi.org/10.1108/ijoes-12-2017-0220. taylor, s. & todd, p.a. (1995). understanding information technology usage: a test of competing models. information systems research, 6(2), 144-176. https://doi.org/10.1287/mnsc.35.8.982 tomaka, j., thompson, s., & palacios, r. (2006). the relation of social isolation, loneliness, and social support to disease outcomes among the elderly. journal of aging and health, 18, 359-384. https://doi.org/10.1177/0898264305280993. turner, e.l., malo, m.e., pisclevich, m.g., dash, m.d., davies, g.f., arnason, t.g. & harkness, t.a. (2010). the saccharomyces cerevisiae anaphase-promoting complex interacts with multiple histone-modifying enzymes to regulate cell cycle progression. eukaryote cell, 9(10), 1418-1431. https://doi.org/10.1128/ec.00097-10. van der heijden, h. (2003). factors influencing the usage of websites: the case of a generic portal in the netherlands. information & management, 40(6), 541-549. https://doi.org/ 10.1016/s0378-7206(02)00079-4. van der heijden, h. (2004). user acceptance of hedonic information systems. mis quarterly, 28(4), 695–704. https://doi.org/10.2307/25148660. van deursen, a. j., bolle, c. l., hegner, s. m., & kommers, p. a. (2015). modeling habitual and addictive smartphone behavior: the role of smartphone usage types, emotional intelligence, social stress, self-regulation, age, and gender. computers in human behavior, 45, 411–420. https://doi.org/10.1016/j.chb.2014.12.039. wang, x. (2019). developing a mobile sns addiction scale utilizing factor analysis and the rasch model. online information review, 43(7), 1284-1301. https://doi.org/10.1108/oir-102018-0300. wang, y., alahakoon, d., & de silva, d. (2018). an extended cognitive situation model for capturing subjective dynamics of events from social media. australasian journal of information systems, 22. https://doi.org/10.3127/ajis.v22i0.1701. wei, m., vogel, d.l., & ku, t., zakalik, r.a. (2005). adult attachment, affect regulation, negative mood, and interpersonal problems: the mediating roles of emotional reactivity and emotional cutoff. journal of consumer psychology, 52(1),14-24. https://doi.org/10.1037/ 0022-0167.52.1.14. australasian journal of information systems chatterjee 2021, vol 25, research on user involvement impact of addiction of online platforms on quality of life 23 wei, p.-s. & lu, h.-p. (2014). why do people play mobile social games? an examination of network externalities and of uses and gratifications. internet research, 24(3), 313-331. https://doi.org/10.1108/intr-04-2013-0082. whiteside, n., aleti, t., pallant, j., & zeleznikow, j. (2018). helpful or harmful? exploring the impact of social media usage on intimate relationships. australasian journal of information systems, 22. https://doi.org/10.3127/ajis.v22i0.1653. xue, y., dong, y., luo, m., mo, d., dong, w., zhang, z. & liang, h. (2018). investigating the impact of mobile sns addiction on individual’s self-rated health. internet research, 28(2), 278-292. https://doi.org/10.1108/intr-05-2017-0198. yang, h.-l. & lin, c.-l. (2014). why do people stick to facebook web site? a value theorybased view. information technology & people, 27(1), 21-37. https://doi.org/10.1108/itp-112012-0130. yang, shang-yu & lin, chung-ying & huang, yueh-chu & chang, jer-hao. (2018). gender differences in the association of smartphone use with the vitality and mental health of adolescent students. journal of american college health. 66. 1-19. https://doi.org/10.1080/ 07448481.2018.1454930. yeo, j.y., kang, s.y., & kim, d.h. (2014). the impact of attachment anxiety on smartphone addiction with a mediating effect of impulsivity and loneliness: focused on the group of sns and game. korean journal of youth counselling. 22(1), 47-69. https://doi.org/10.35151/kyci.2014.22.1.003. yi, c., liao, p., huang, c. & hwang, i. (2009). acceptance of mobile learning: a re-specification and validation of information system success. international journal of educational and pedagogical sciences, 3(5), 475-479. yi, j.y., & choi, h.c. (2012). hope as a mediator between adult attachment and depression in university students. korean journal of counseling. 3(2), 819-38. https://doi.org/10.15703/ kjc.13.2.201204.819. australasian journal of information systems chatterjee 2021, vol 25, research on user involvement impact of addiction of online platforms on quality of life 24 appendix 1: measurement instruments constructs sources research instruments loneliness (lon) tomaka et al., 2006; wei et al., 2005; yi et al., 2012; yang & lin, 2014; chatterjee, 2020 lon1: i consider online platforms as the only option for me to have a conversation with my friends, colleagues, and relatives. lon2: most of the time in a day, i remain completely isolated. lon3: i do not have any companionships to talk. lon4: i like to be alone with different online platforms. lon5: i feel uncomfortable to meet a person physically. perceived enjoyment (pen) ven der heijden, 2003; lee & tsai, 2010; shin, 2010; wei & lu, 2014; sarin et al., 2020 pen1: i feel good to connect with my friends online instead of physically. pen2: i enjoy online shopping very much. pen3: i like to spend maximum amount of time using social networking sites. pen4: i feel good by chatting with my friends and colleagues via online platforms. pen5: i am fond of playing online video games. depression (dep) kim et al., 2007; yi et al., 2012; yeo et al., 2014; wang, 2019; gong et al., 2020 dep1: i do not find any pleasure to do things. dep2: most of the time i feel depressed for want of physical contacts with others. dep3: i find it very difficult to concentrate on things like reading books, playing games etc. dep4: i often feel tired to do things. dep5: i do not find any interest of experimenting new things. perceived ease of use (peu) davis et al., 1989; taylor & todd, 1995; bisen & deshpande, 2018; sulaiman et al., 2019; chatterjee et al., 2020b peu1: use of online platforms is easier to learn and apply. peu2: it is easier to use different online applications like online shopping, online games, social networking etc. peu3: all the online platforms are flexible to use. peu4: people can be engaged with different online activities as they have smartphones. peu5: i do not need any particular training to use different online platforms like social networking, online shopping, playing online video games etc. perceived usefulness (pu) davis et al., 1989; lee et al., 2007; kwon et al., 2013; jiang et al., 2014; arpaci et al., 2018 pu1: i shall be more efficient if i use different online platforms for different purposes. pu2: most of the online platforms are inexpensive to use. pu3: i find use of different online platforms beneficial for me. pu4: i can get all the information by using online platforms on a real-time basis. pu5: i can buy things sitting at my home using shopping websites. addiction of online platforms (aop) hong et al., 2012; park & lee, 2012; he et al., 2019 aop1: i use different online platforms for more than 12 hours a day. aop2: i find it quite difficult to physically communicate with people. aop3: because of my frequent online activities, i do not get enough sleep. aop4: i cannot remain offline for more than 30 minutes at a stretch. quality of life (ql) costanza et al., 2008; kingpadug & ql1: my productivity has gone down because of overuse of different online platforms. ql2: i spend less time talking with my family members. australasian journal of information systems chatterjee 2021, vol 25, research on user involvement impact of addiction of online platforms on quality of life 25 phusavat, 2010; whiteside et al., 2018 ql3: i cannot remember easy things. ql4: i am always busy with online activities and thus do not get time to exercise. copyright: © 2021 chatterjee. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v25i0.2761 http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems mukerji & roy 2019, vol 23, research article platform interactions and evolution of ola 1 platform interactions and evolution of ola’s organizational field maitrayee mukerji indian institute of management kashipur india maitrayee_mukerji@yahoo.com partha sarathi roy school of management & entrepreneurship shiv nadar university, greater noida india abstract most digital enterprises begin as technology start-ups offering niche services, and then evolve into digital platforms providing a range of services. against the backdrop of almost nonexistent logistic and supply chain infrastructure, weak regulatory framework, low penetration of smartphones and internet, low consumer confidence and trust on online transactions, digital entrepreneurs in developing countries literally have to build the platform ecosystem along with their organization. in this paper, we use institutional theory as a lens to trace the emergence and evolution of digital platforms in india. we posit that digital platforms are emerging organizational fields embodying new network logics with algorithms and technology-mediated interactions playing a central role in creating value. in particular, we take the case of ola, an online cab aggregator from india to illustrate that digital platforms as organizational field evolve through three phases of (a) innovation introduction, (b) mobilization and (c) structuration. in the first phase, the organizational field characteristics are consequences of activities undertaken to raise cognitive legitimacy and respond to competitive forces. in the second phase, a mix of coercive and mimetic forces shape the institutional field resulting in increase in socio-political legitimacies. in the third phase, the process of structuration sets in as isomorphic forces become more normative. the theoretical framework developed in this paper contributes to extant information system research by putting forth that the characteristics of emerging organizational field are shaped by the interplay between the responses of the focal organization to isomorphic forces, and the actions taken by it to establish cognitive and socio-political legitimacies. empirically the study is on digital platforms located in a large developing country with very high potential for online transactions but relatively low penetration of internet and smartphones. conceptualizing organizational field as digital platforms enables making sense of the changing business landscape, and examine issues of sustainability and legitimacy. keywords platforms, institutional theory, organizational field, ola, india 1 introduction in recent years, india has seen the emergence of a large number of digital platforms. such technology-based digital platforms function as two-sided or multi-sided marketplaces enabling service providers and users to connect to each other. examples include global giants australasian journal of information systems mukerji & roy 2019, vol 23, research article platform interactions and evolution of ola 2 like uber, airbnb, and amazon along with their indian competitors like ola, oyo, and flipkart among others. almost all digital enterprises began as technology start-ups offering niche services, and since then have evolved into digital platforms providing a range of services. in this paper we use institutional theory as a lens to argue that digital platforms in india constitute an emerging organizational field, shaped by network logics and interactions between various constituents. the primary motivation for undertaking this research was to make sense of changing business landscape in india following the emergence of digital platforms about fifteen years back. these platforms emerged in a context of relatively low internet penetration but with very high potential for online transactions enabled by rapid diffusion of smart phones. against the backdrop of almost non-existent logistic and supply chain infrastructure, weak regulatory framework, lack of trust and consumer confidence on online transactions, digital entrepreneurs literally had to build the business ecosystem along with their organization. however, accompanying the enthusiasm about the potential, there is also scepticism regarding legitimacy and sustainability of such digital ventures. for example, digital enterprises constantly appear to be in need of investments, but simultaneously they give high discounts to customers. further, high valuations of certain digital enterprises are accompanied by sudden closures of apparently successful ones. conceptualizing digital platforms as emerging organizational field enables us to understand their characteristics and seemingly contradictory practices. organizational field as a construct enables us to comprehend frequent occurrences of conflicts between incumbents and entrants, or between platform orchestrators and service providers. further, digital platforms are organizational manifestation of network logics, bringing together producers and consumers in high-value exchanges (alstyne, parker, & choudhary, 2016). unlike traditional organizations, the organizational structure of a digital platform is open by design and its organizational boundaries are porous. interactions between various entities are shaped by the network logics and play a key role in creating value for the platform ecosystem. since norms and rules governing these platform interactions are still evolving, it’s an opportune time to examine them as organizational field. in this research we seek to contribute to is research by examining the dynamics of the formation of a new and emerging organizational field. we put forth that digital platforms in india embody a shift towards network logics where interactions between various constituents play a central role in creating value. repeated interactions between different constituents help in developing common understandings and practices. at the same time, these emerging institutions shape the ongoing patterns of interaction from which they are produced (lawrence et al., 2002). specifically, the theoretical framework developed in this paper posits the relationship between institutional forces, legitimization strategies of focal organization and the characteristics of emerging organizational field. empirically, the study is located in the context of a large developing country with very high potential for e-commerce but relatively low penetration of smartphones and internet. in particular, the study examines the institutional work undertaken by an indian mobility platform ola, that revolutionized the way indians book taxis and travel. acting as an institutional entrepreneur, ola played an important role in creating the mobility platform ecosystem. following the introduction, the paper is divided into eight sections. the second section briefly delineates the logics of digital platforms, followed by a review of institutional theory in is australasian journal of information systems mukerji & roy 2019, vol 23, research article platform interactions and evolution of ola 3 research in section three. the fourth section puts forth the theoretical framework followed by discussion on methods and data. the evolution of ola’s organizational field is traced in section six followed by a discussion on digital platforms as emerging organizational field in india. the last section puts forth the conclusion and scope for future work. 2 the network logics of digital platforms institutional logics refers to macro-level beliefs that shape cognition and influence decisionmaking processes in organizational fields (mcpherson & sauder, 2015). logics is converted to action by actors who use it as a guide for their micro-level day to day activities and interactions (ibid). logics shape goals and expectations, define legitimate activities, and are often embodied in organizational strategies, structures, and practices (dacin, jerry, & scott, 2002). all types of digital platforms provide a way to connect various participants in a market and are based on “network logics”. as the number of participants grow, value of the platform increases for all. this phenomenon is referred to as the “network effect” and is central to all platform strategies (alstyne, parker, & choudhary, 2016). firms strive to achieve higher “volume” of participation than competitors in order to achieve higher average value per transaction. further, as the network size increases, better the matches between supply and demand. the concomitant data generated is also richer and can be used again for making even better matches. a virtuous feedback loop is created as greater scale generates more value, which attracts more participants. network effects is what distinguishes platforms from pipeline business that have a linear supply/ value chain or from traditional non-it enabled platforms like newspapers connecting subscribers and advertisers or, shopping malls linking consumers and merchants etc. (alstyne et al., 2016). the key difference arises from use of information technologies to enhance capabilities to capture, analyse and exchange huge amounts of data and thus enabling the network logics in practice. information technology is used for building apps, scaling-up businesses, aggregating demand, and monitoring interactions. in traditional pipeline businesses, value is created by controlling inimitable resources and key assets, and increasing efficiencies of linear chain of value activities from sourcing to servicing. in contrast, the chief assets and source of value creation in platforms are its participants with their resources, information and interactions (alstyne et al., 2016). digital platforms orchestrate resources and set rules determining the degree of access and control given to consumers, producers, providers and even competitors. interactions or exchange of value between producers and customers are key performance metrics. the unit of exchange can be as small as a “like” or a picture upload, with almost no monetary transactions (ibid). increasing network size, boosting and monitoring core interactions are critical for digital platforms. positive interactions and their spillover effects help platforms rapidly increase volume of interactions. while negative interactions arising from congestion, non-availability or nonfulfillment of demand, “noise”, misbehaviour or low-quality services often diminish network effects, sometime leading to defection by participants. 3 institutional theory in is research broadly institutions are defined as rules and norms that guide or constraint actions. they are also considered as social patterns, evolved and reproduced through system of rewards and sanctions (jepperson, 1991). institutional theory was first put forth to explain why australasian journal of information systems mukerji & roy 2019, vol 23, research article platform interactions and evolution of ola 4 organizations tend to resemble each other or become isomorphic (dimaggio & powell, 1983). it provides a powerful theoretical lens to understand evolution of organizational fields. an organizational field is defined as a set of institutions and network of organizations that, in aggregate constitute a recognized area of organizational life (dimaggio & powell, 1983; maguire et al., 2004). relationships and interactions between different constituents of the organizational field are shaped by the institutional logics. repeated interactions help in developing common understandings and practices in any organizational field. and at the same time, these emerging institutions guide the ongoing patterns of interaction (lawrence et al., 2002). following aldrich & fiol (1994), new organizational fields emerge when constituent organizations respond to institutional forces and undertake activities to gain socio-political or cognitive legitimacies. the characteristics of an emerging organizational field are generally described in terms of (a) the extent of interactions among various constituents of the field, (b) the development of clearly defined structures of domination and coalition, (c) the amount of field-relevant information, and (d) degree of mutual awareness among members of the organizational field (dimaggio & powell, 1983). within information system research, institutional theory has been mooted to examine and understand “how institutions influence the design, use, and consequences of technologies, either within or across organizations” (orlikowski & barley, 2001). empirical studies have used this perspective to investigate emergence of standards and open source platforms (bala & venkatesh, 2007; garud, jain, & kumaraswamy, 2016; ojha & rao, 2014), diffusion and adoption of technologies to handle y2k (cannon & woszczynski, 2002), organizational intranets (damsgaard & lyytinen, 1998), electronic data interchange (damsgaard & lyytinen, 1998; teo, wei, & benbasat, 2003), enterprise resource planning (chatterjee, d., grewal, r. and sambamurthy, 2002), supply chains (kauppi, 2013) and health care (currie & guah, 2007). the primary focus of most empirical studies has been at the organizational level (weerakkody, dwivedi, & irani, 2009) with very few looking at multi-level, multi-stakeholder relationships at the field level (currie, 2009). the organizational context includes government departments and agencies (currie & guah, 2007), manufacturing firms (kauppi, 2013), oil company (avgerou, 2000), and banks (tan and fichman, 2002) among others. further, most studies consider both external environment and institutional forces as given and constraining, with little investigation into how actions and responses of any focal organization shape the organization field. organizational field as a theoretical construct has been studied only for software-based platform like linux (ojha & rao, 2014) or java (garud et al., 2016). in such studies, the organizational boundaries are clearly demarcated, making it easy to classify interactions, information flow and structures. this paper adds to existing is research by seeking to use institutional theory as a lens to examine (a) how have digital platforms evolved as organizational field in india? and (b) in what ways institutional forces and legitimization strategies of the digital platform shape the characteristics of the organizational field? the study seeks to contribute to extant is research by examining dynamics of the formation of a new and emerging organizational field through empirical investigation of the translation of new institutional logics into micro-level responses and activities (yoo, henfridsson, & lyytinen, 2019). the theoretical framework is an important addition to is literature, linking for the first time, multiple concepts of institutional theory to trace the emergence of a new organizational field. it posits that the evolution of an organizational field from innovation to structuration can be ascertained by examining its field-level characteristics. at different points in time, these characteristics are outcome of the interplay between responses of the focal australasian journal of information systems mukerji & roy 2019, vol 23, research article platform interactions and evolution of ola 5 organization to institutional pressures and the activities undertaken by them to increase cognitive and socio-political legitimacies. the use of institutional theory to study digital innovations and digital platforms have been mooted by many recent research reviews and this study thus contributes towards addressing this gap in is research (hinings, gegenhuber, & greenwood, 2018; mair & reischauer, 2017). the study specifically looks at the 2-sided marketplaces, where value is co-created based on set of interactions (reuver, sørensen, & basole, 2017). this study on formation of a new organizational field is located in the context of a developing country. operationalization of the network logic and making explicit the categorization of various competitive and institutional forces is a valuable empirical contribution for both research and practice. it gives an explanation for seemingly irrational practices like high valuation, and deep discounts of digital platforms and facilitates understanding of issues related to regulation, sustainability, and legitimacy. 3.1 theoretical framework we define institutions as relatively widely diffused practices, technologies, or rules that are preferred in a particular context over others (lawrence, hardy, & phillips, 2002). organization field as a construct is operationalized as the totality of relevant actors and also as a functionally specific arena (machado-da-silva, guarido filho, & rossoni, 2006). thus, organizational field consists of a set of interdependent organizations sharing systems of common meanings and interacting more frequently among themselves than with actors outside the field. further, these organizations are not necessarily similar to each other, but operate in a functionally specific area with certain technical and institutional characteristics (ibid). thus, in an organizational field, apart from the focal organization there are suppliers, resource providers, product or service consumers, and regulatory agencies among others. to examine the evolution of an emerging organizational field, we adopt the framework proposed by purdy & gray (2009). the framework outlines a three-stage process by which any innovation gets effected as an organizational field. it integrates the macro-micro perspective, putting forth that institutionalization is result of repeated and recursive interactions between individual and organizational actions with the field level dynamics. in other words, the diffusion and process of institutionalization is due to both structural characteristics of the field and actions of institutional entrepreneurs who theorize new fields and launch new organizations (maguire, hardy, & lawrence, 2004). in this framework, the first stage is characterized by introduction of innovation where new logics are initiated. in the next phase of mobilization, the innovation picks momentum and various pioneering organizations/ actors, each with their own version(s) of the innovation compete to gain adherents for their logics (purdy & gray, 2009). the last stage is called structuration where new logics get translated to concrete practices. these practices gain the status of being standardized, taken-for-granted and well accepted as legitimate conducts. figures 1 and 2 together describe the theoretical framework of the study. at any particular time (see figure 1), the characteristics of a given organizational field is described in terms of (i) the extent of interaction among constituent organizations, which serves as indication of the level of economic activities between them (ii) presence of sharply defined inter-organizational patterns of coalition and structures of domination (iii) information sharing between constituent organizations and, (iv) mutual awareness, common purpose or identity among constituent organizations (dimaggio & powell, 1983). as an organizational field moves towards maturity, the value of these four dimensions increase (anand & peterson, 2000; maguire et al., 2004; purdy & gray, 2009). once an innovation is introduced in a context, the organizational field is likely to experience lots of inconsistency australasian journal of information systems mukerji & roy 2019, vol 23, research article platform interactions and evolution of ola 6 and contestations from stakeholders (greenwood, suddaby, & hinnings, 2002). during the stage of mobilization, institutional norms and practices get established through repeated interactions and knowledge sharing among the constituents. in a positive reinforcement, these institutions further shape the ongoing patterns of interaction from which they are produced (maguire et al., 2004). the process of structuration leads towards mature organizational field where constituent members follow a shared logic and adhere to collective norms, rules and practices. as an innovation gets diffused, there is a strong tendency among similar type of constituent organizations to resemble each other or become isomorphic (dimaggio & powell, 1983). extant literature provides multiple explanations for this phenomenon. organizations rely on external environment for resources and need to compete for the same. similarity in organizational forms helps in reducing uncertainty in decision-making and operations (mizruchi & fein, 1999). further, isomorphism emerges not only because of competition or an objective requirement of efficiency but also as result of organizations' quests to gain legitimacy within their larger environments (aldrich & fiol, 1994). dimaggio & powell (1983) distinguishes between two types of isomorphism. competitive isomorphism results in similarity due to market competition for resources and customers. in contrast, institutional isomorphism takes place as result of three different types of institutional forces coercive, mimetic and normative (ibid). coercive force refers to formal and informal pressures exerted on a focal organization by other stakeholders upon which it is dependent for resources or acceptance as legitimate form (dimaggio & powell, 1983). it implies that organizations' actions can be constrained by the actions of other, more powerful units (mizruchi & fein, 1999). uncertainty in external environment is often the source of mimetic pressure. in situations where, clear directions or course of action are ambiguous, organizations often imitate practices of peers who are perceived to be more successful or legitimate. diffusion of such practices can happen intentionally by imitating innovations, by consultants or trade associations, or indirectly thru employee transfer or attrition (dimaggio & powell, 1983). the third source of institutional pressure is generally normative and results from professionalization. professionalization involves collective quest towards defining work practices and norms (ibid). interactions between members who have received similar training or belong to same professional and trade associations enable diffusion of ideas and norms. scott (2001) puts forth that organizational responses to institutional pressures does not necessarily emanate from a single organization, but from multiple constituents of an organizational field. the articulated response might be a concerted one which has “the potential to shape the nature of the demands and even to redefine the rules and logics operating within the field” (ibid). legitimacy is defined as generalized perceptions of being valid, appropriate, reasonable or acceptable. while industry related factors like market size, incentives, revenue and profits are important for survival of a nascent industry, extant research indicates that legitimacy often plays a role in growth and acceptance of the industry (aldrich & fiol, 1994; suchman, 1995). new organizational fields emerge when entrepreneurs succeed in building credibility and can mobilize support and resources for starting a venture. identifying opportunities, raising funds, collecting resources, recruiting and training employees are some challenges faced by all entrepreneurs. all of these activities require cooperation and strategic interaction of others (aldrich & fiol, 1994). for our theoretical framework, we refer to two types of legitimacies put forth by aldrich & fiol (1994). the first one refers to cognitive legitimacy based on awareness and knowledge among key constituents regarding a venture, and its related activities, norms australasian journal of information systems mukerji & roy 2019, vol 23, research article platform interactions and evolution of ola 7 and practices and activities. the highest form of cognitive legitimation is achieved when a new product, process, or service becomes so familiar that it is taken for granted (ibid). cognitive legitimation is gauged by the willingness of new entrants to mimic an existing organizational form and also by number of knowledgeable users of associated products or service. the second type of legitimacy is based on value attributed to particular activity by cultural or political norms. socio-political legitimation refers to the process by which key stakeholders like the general public, key opinion leaders, or government officials accept a venture as appropriate within existing norms and laws. one can measure socio-political legitimation by assessing public acceptance of a venture (ibid). fig1: the three stages of formation of an organization field (of) figure 2: process of institutionalization at any given point of time australasian journal of information systems mukerji & roy 2019, vol 23, research article platform interactions and evolution of ola 8 4 methods and data considering the relative newness of the phenomenon, we use exploratory case study research as research design. such naturalistic mode of inquiry to examine organizational fields have been followed by (garud et al., 2002; greenwood & suddaby, 2006). in particular, by examining the chronology of key field configuring events, we trace the evolution of ola’s organizational field, often referred to as “uber’s rival”. following dunn & jones (2010), we use secondary sources of information like newspaper articles, and market intelligence reports to gather field level happenings. the number of articles in each month gives an indication about the level of activities in ola’s organizational field. we extracted this data for the time period between april 2011 and september 2018, from crunchbase (www.crunchbase.com), which has date-wise repository of all key events and news articles related to ola. after removing duplicates and articles that referred to same events, we had 1016 articles on ola and its field. for analysis, we followed process similar to (lawrence et al., 2002). all articles were (a) read and summarized, (b) the summaries were categorized and coded, corresponding to key constructs of institutional theory and (c) examined for pattern in the relationships between different conceptual categories. while summarizing each news article, we identified the constituents of the field, their relationship with others, the isomorphic forces, if any and the organizational responses. the reasons for particular intervention, and their possible impact on the four dimensions of the emerging organizational field was identified or in some cases inferred from the text. the processes of categorizing, and coding iterated between theory and empirical data to minimize subjective interpretation. see appendix 1 for chronological categorization of key events of the organizational field. in our theoretical framework, we posited a three-stage evolution of an organizational field. in each phase, the field characteristics are outcome of interplay between the responses of the focal organization to various institutional forces and its actions to increase cognitive and sociopolitical legitimacies. for our analysis, we considered activities that seek to generate awareness among various constituents as those for gaining cognitive legitimacy. while activities like partnering for national events, negotiating for resources, liaising for legal and regulatory issues as those undertaken to gain socio-political legitimacy. although analytically distinct, it is not always easy to distinguish the three types of institutional forces in empirical settings (dimaggio & powell, 1983). thus, in this study, we use the following criteria to categorize institutional forces and the response of the focal organization: • all activities that are undertaken by the focal organization to strengthen implementation of logic and increase market power are considered as response to competitive forces. examples include investment in technologies, mobilization of funds, acquisition of customers, or enrolment of service providers. • responses to mimetic pressures include those that are used for increasing or maintaining interactions by creating value by similar offerings or services as others, and adopting practices that have been proved to be successful in similar settings. • coercive pressures are exerted on the focal organization by constituents of organizational field that have access to and control of resources or power, and can constrain a focal organization’s work directly or indirectly. • normative pressure as those related to establishment or diffusion of work norms and practices. australasian journal of information systems mukerji & roy 2019, vol 23, research article platform interactions and evolution of ola 9 5 the evolution of ola’s organizational field ola (www.olacabs.com) is a popular mobile app for cab bookings in india. established in december 2010 by two engineering graduates, it is headquartered in bangalore. along with flipkart, india’s first e-commerce firm, ola is often credited for making indians familiar with online transactions. starting with the modest mission to ‘provide a cab to every mumbaikar’ the organization now positions itself as a platform to bring together cab drivers and customers using technology to make transportation hassle-free for everyone. ola’s organizational field is diverse and includes (i) customers, (ii) service providers consisting primarily of cab aggregators, and drivers, (iii) competitors and rivals including uber, radio cab service provides, traditional taxis and auto-rickshaws (iv) government and regulatory bodies, (v) media, (vi) investors and venture capitalists, (vii) partners including big and small technology developers and providers, car manufacturers and other emerging platforms (see figure 3). while ola orchestrates the relationships and interactions in the field, taxi drivers form the main interface with the customers. a taxi driver is not an employee of ola, and can become a customer and vice-versa. further, the field overlaps with other organizational fields of both traditional and other technology-based platforms. pure technology-based platforms like those of google, amazon etc. provide the necessary base for building the platform architecture and services. other digital platforms from different verticals like e-commerce, foodtech and fintech also are very much part of the organizational field. among the traditional sectors, car manufacturers like nissan and mahindra & mahindra, and financial institutions are some important constituents of the platform. online news aggregators, and social media help in generating cognitive awareness about start-ups and their external environment. figure 4 shows the quarter-wise distribution of articles. the frequency of articles indicates the intensity of field level activities at different time period periods. we divide the emergence and evolution of ola’s organizational field into three parts, broadly corresponding to (a) introduction of innovation, (b) mobilization, and (c) onset of structuration. considering ola as a focal organization, key events shaping the characteristics of the emerging organizational field are (a) entry of uber as a competitor in the indian market (august 2013), (b) delhi rape case (december 2014), (c) uber’s exit from china and concentrated focus on india (august 2016), (d) protests and strikes by incumbents, partners and other service providers (february 2017 – may 2017 ). australasian journal of information systems mukerji & roy 2019, vol 23, research article platform interactions and evolution of ola 10 figure 3: ola’s organizational field figure 4: distribution of articles on ola in the crunchbase repository 5.1 stage 1: innovation introduction (start – june 2014) since beginning, ola positioned itself as a technology firm rather than a taxi service provider. it was registered as a ride sharing app that provided direct interface between taxis drivers and commuters using state-of-the-art mobile technologies. information technology formed the core of ola’s value proposition and offerings. it operationalized the network logic for the digital platform. data was used for analysing demand, location and route planning, sending real time traffic alerts and enabling customer interactions including bookings, payments and e-invoicing. further, ola sought to provide better trip experience to customers by using technology to check behaviour of taxi drivers, who were not employees of the organization. while technology was available to translate network logics into an innovative product, building a sustainable and scalable digital platform was a challenge. as a pioneering ridesharing start-up, ola faced the uphill task of creating awareness about the business proposition of new technology product, building trust and convincing stakeholders to adopt their online offerings. unlike existing incumbents like taxi aggregators and radio-cab service providers, ola did not own any fleet of its own. it worked with local taxi providers and other 1 2 1 4 2 1 4 3 10 22 22 31 57 82 114 186 135 99 94 89 2 12 11 22 9 1 0 50 100 150 200 q tr 2 q tr 2 q tr 4 q tr 2 q tr 3 q tr 4 q tr 1 q tr 2 q tr 3 q tr 4 q tr 1 q tr 2 q tr 3 q tr 4 q tr 1 q tr 2 q tr 3 q tr 4 q tr 1 q tr 2 q tr 3 q tr 4 q tr 1 q tr 2 q tr 3 q tr 4 2011 2012 2013 2014 2015 2016 2017 2018 australasian journal of information systems mukerji & roy 2019, vol 23, research article platform interactions and evolution of ola 11 individuals to provide transportation services. since networks generate value only when adequate number of both service providers and customers transact with each other, it was imperative for ola to on-board as many taxi drivers and customers as quickly as possible. thus, in the first phase, much of organizational activities were driven by the need to create the ecosystem, build cognitive legitimacy among stakeholders and compete with uber. building the ecosystem required addressing issues of low penetration of credit cards and smartphones, inadequate logistics and supporting infrastructure, and low consumer trust and low levels of disposable income. this apart from convincing price conscious customer to shift preference from cash to cards and using apps instead of telephone to book taxis. peer start-ups like flipkart, justdial and redbus had demonstrated that using available, and accepted methods of transactions helped in building trust and subsequent adoption of new technologies. hence, during the phase of innovation introduction, ola facilitated cab booking using the web and via call centres, and accepted payments in cash. in this phase, ola had two main competitors. the first one was an indian start-up called taxifor-sure, operating in the economy segment and having significant recall value among people. the second one was uber, a leading us based ride-sharing firm with significantly high market valuation. uber had pioneered and successfully scaled up an app-only business model that focused on providing only premium services and accepting payments using credit cards. uber launched its operations in india in august 2013, and unlike other places, it was willing to be flexible and fine tune its model for indian business environment. ola competed with both for on-boarding customers and drivers onto its platform. it launched ola mini for price conscious customers, and simultaneous launched premium service for those willing to pay. familiarity with the indian context enabled ola to expand rapidly by launching new services and in new cities. efforts to increase cognition among potential customers included reaching out to them during strikes by existing incumbents with special discounts and schemes. in one of its very early interventions, ola on-boarded the regular city taxis on its network. using the ola app, users could book the regular taxi in the vicinity and paid according to the taxi fare meter. doing so, allowed ola not only to minimize the demand – supply gap in availability of taxis but familiarized both users and taxi drivers about the app and its future possibilities. thus, proposition 1: during the first stage of evolution of organizational field through innovation introduction, digital platforms focus on building cognitive legitimacy and responding to competitive pressures. 5.2 stage 2: mobilization (july 2014 to june 2017) the mobilization phase between july 2014 and june 2017 was intense in terms of events and activities, defining much of the characteristics of the emerging organizational field. concomitant with rapid expansion in terms of number of cities, taxis and customers, this phase was marked by much environmental uncertainty due to number of field-defining events. these included the delhi uber rape case, acquisition of taxiforsure by ola, protests by traditional city cab aggregators and drivers against ola/uber and later protests by ola drivers for better work norms. the period was also defined by fierce competition between uber and ola to retain market dominance, especially after uber withdrew from china and started focusing fully on india. australasian journal of information systems mukerji & roy 2019, vol 23, research article platform interactions and evolution of ola 12 in december 2014, following an unfortunate incident of a woman getting raped by an uber driver, ola cabs along with others were banned in delhi. it was the first-time regulatory authorities realized the difference between traditional or radio taxi service and app-based service. since there were no specific regulation for digital ventures, ola and uber were asked to get taxi licenses. to do so required operating like a traditional taxi company by maintaining a fleet of cars and setting up a 24/7 call centre instead of just relying on technology. ola and uber resisted the change in business model, taking the risk to continue operations till new guidelines were notified. however, one field level impact of such regulatory uncertainty was unavailability of funds for certain period of time. it was during this period ola made its move to acquire its indian competitor taxi-for-sure for $200 million. by this, it sought to consolidate its position vis-à-vis its competitor. the acquisition enabled ola to add more services, more drivers and customers belonging to a particular segment on its platform. more importantly, it increased its market valuation, making it easier to raise funds subsequently. however, the episode bought to the fore, the regulatory vacuum regarding digital platforms in india and also marked the beginning of increasing run-in with various state governments regarding legal compliance of payment methods and different kinds of services, surge pricing, and women safety among others. the next big confrontation with governments and regulatory authorities was with regards to surge pricing. the government of delhi in their efforts to reduce air pollution experimented with an odd-even policy, where on alternate days cars with either odd or even registration numbers were allowed on the road. for app-based taxi aggregators, it was a business opportunity and algorithm determined taxi fares surged upwards, drawing criticism from customers and ire of state government. it was also a manifestation of conflict between old logics and new one governed by algorithms. initially, ola and uber both maintained that surge pricing were reflecting demand-supply conditions, it had to subsequently suspend surge pricing following pressure from both customers and state governments. in subsequent months, other state governments also intervened from time to time to regulate surge pricing. as response, both ola and uber introduced subscriptionbased packages that in lieu of monthly payments provided regular customers immunity from surge prices. it also helped to maintain repeat customers and increase their transactions with the platform. the conflict between traditional and network logics was also manifested, first during protests by regular taxis against ola and uber and later as strikes by their own driver partners. the livelihoods of traditional taxis were deeply impacted low fares charged by ola. they demanded government intervention to bring about some parity. the second set of protests were by cab aggregators and drivers formally associated with either ola or uber. in order to increase the number of drivers on the network, ola offered them cheap car loans and incentives. however, once scale was achieved, ola sought profitability and started rationalizing its operating expenses. this led to tougher performance norms for drivers that impacted earnings and investments. drivers were not able to pay their emis for car loans taken. within a short span of time, such strikes happened in many parts of the country. around 25% of enrolled drivers supposedly abandoned the platform. while the protests were called off with negotiations, relaxation in performance norms and commission fee, it led to incorporation of new practices. the strikes bought to fore risks associated with asset-light companies. the digital platforms were fully dependent on driver partners for continuation of customer service. subsequently, both ola and uber set up car-leasing business to control and ensure a supply of cars at all times. the protests by drivers can also be considered as a sign of australasian journal of information systems mukerji & roy 2019, vol 23, research article platform interactions and evolution of ola 13 increasing field maturity. it led to rationalization of expectations. the drivers who were subsequently enrolled as platform members were aware of lower incentives, and willing to work for longer hours and lower earnings. one key development was that drivers were allowed to be partners to both the competing platforms. during this particular time period, ola and uber sought to increase interactions with existing and new customers by introducing similar type of services like motorbike taxis, autorickshaws, carpooling, and long-hour rentals among others. both the platforms also mimicked each other by investing in other platform verticals like fintech, foodtech and home delivery of groceries. a particular characteristic of the emerging organizational field was the increasing role and dominance of global investor firms. not only they invested heavily in select digital platforms, they also intervened directly so as to ensure their returns on investments. through their investments, digital platforms in india including ola became part of a global fight for dominance between chinese and american investment groups and firms. a particular intervention was orchestrating the formation of a global alliance of ride-sharing companies to compete against uber. among the stated objectives of the alliance was to enable sharing of information and tactics, and incorporation of features like cross-booking or in-app integration to each other’s services. while confrontations and negotiations with regulatory agencies and investment firms led to gains in socio-political legitimacy, ola also engaged in many public programs for the same. it partnered with state government to training youths as entrepreneurs, incentivizing drivers to convert to green fuel, committed investment’s for electric vehicles, ferried passengers for free during mishaps and disasters. thus, in the second phase of mobilizations, all the three institutional forces appeared in various forms and intensities. coercive and normative pressures were exerted by partner drivers, regulatory agencies and venture capitalists for regulatory guidelines, and rationalization of work practices. further, both ola and uber mimicked each other’s strategies to increase platform interactions. proposition 2: during the second stage of development of organizational field via mobilization of the new institutional logic, digital platforms encounter all three types of institutional pressures – mimetic, coercive and normative from other field constituents and take actions to build socio-political legitimacy at the intra and inter industry level. 5.3 stage 3: onset of structuration (july 2017) sudden decrease in the reporting of field level activities after july 2017 indicate start of the structuration process of ola’s organizational field. the ride-sharing domain in india is dominated by ola and uber, mirroring similar duopolistic structures in other countries. in september 2018, both the competitors had a common investor, and speculations were rife regarding their possible merger in india. at organizational field level, another round of evolutionary iteration seems to suggest onset of isomorphism. digital platforms were investing in or acquiring business in verticals other than their core. ola, with its acquisition of food tech called foodpanda mimicked the trend. further, as the number of digital platforms increased, so did the coercive power of government and other regulatory bodies as issues related to data privacy, physical safety, surge pricing and tariff issues. regulatory issues also manifest with regards to better work norms and working conditions of drivers and delivery boys. isomorphism is also induced via coercive pressures from venture capitalist / investors. global investor firms hold control over key financial resources, but also help in diffusion of innovation and best practices from other parts australasian journal of information systems mukerji & roy 2019, vol 23, research article platform interactions and evolution of ola 14 of the world. they hugely influence valuations of digital enterprises and often force mergers of erstwhile rivals. proposition 3: during the third stage of development of organizational field through structuration, normative and coercive pressures from regulatory and investors will lead to organizational isomorphism. 6 digital platforms as emerging organizational field in the previous section, we documented the evolution of ola’s organizational field using the three-stage process of institutionalization as put forth by purdy and gray (2009). in the second phase, efforts were directed towards addressing regulatory issues and building socio-political legitimacies. institutional forces were mostly coercive and mimetic. the third phase indicates the start of the structuration process and it would continue till the time issues related to regulation and work norms are resolved. the emerging nature of the organizational field can be ascertained by examining the four dimensions characterizing the same. in the context of digital platforms, one cannot draw a one to one relationship between the actions of the focal organization or its response to institutional forces and the four dimensions. further, technology enabled interactions and information sharing form the core of institutional logic. with overlapping organizational boundaries, the four dimensions are not mutually independent of each other. a particular response by the focal organization to any of the institutional or competitive pressure brings about change in all the four dimensions. to illustrate, in order to generate value based on network logic, it is imperative to increase number of interactions among members of the organizational field. this increase in interaction is made possible by technology aided data collection and processing leading to increase in information and its sharing among constituents. as more information is generated by different constituents and shared with others, there is greater mutual awareness and possibilities of formation of dominant structures and coalitions. however, the evolution of ola’s organizational field brings forth some discernible patterns in its four characteristics. 6.1 increase in interactions in the first phase of organizational field development, the focal organization undertook activities to operationalize the network logics using technology, raise cognitive legitimacy about the innovation among various constituents of the organizational field and to respond to competitive pressures by mobilizing resources and undertaking market expansion. it was imperative to bring on board as many drivers and as many customers as possible. ola did so by introducing new services and taking them to new cities, and investing in technology or technology firms that help in improving the network performance. once adequate number of drivers and customers were on board, the next task was to increase the number of their interactions via schemes, incentives, discounts etc. during the second stage of mobilization, it can be observed that ola and its main competitor mimicked each other to bring more people on the platform and increase their platform interactions. thus, it can be said that: proposition 4: during the process of evolution of digital platforms as organizational field, actions and responses of the focal organization to competitive and mimetic pressures lead to increase in interaction among various constituents. australasian journal of information systems mukerji & roy 2019, vol 23, research article platform interactions and evolution of ola 15 6.2 information sharing ola was conceptualized as a direct interface between traditional taxis and their customers to address certain on-ground uncertainties. the ola app could book a taxi, provide instant booking confirmation, send details of the cab and its drivers, and track its movement. its interface gave the cab driver details about location of the customer and expected time to reach. however, each phase of field evolution bought its own set of information related issues. for ola, any customer’s experience depended on its taxi drivers, thus information to monitor performance and customer feedback was built in. issues related to women safety made it necessary to check driver’s profile and track routes. customer profiles, and usage pattern also became important to design appropriate strategies. focal organizations also have to keep track of information and speculations with regards to valuation, possible mergers and acquisitions. valuations often reflect the beliefs and expectations of market participants, especially the investors. it also indicates the willingness of any party to pay for a particular enterprise and in the process enhance the portfolio’s valuation of current investors. higher valuations also enable the founders to keep attracting capital for maintaining their leadership. thus, while technology formed the core of addressing information related requirements and challenges, ola also had to undertake activities to gain trust, build reliability, and reputation in order to be accepted as legitimate business models. hence, proposition 5: during the process of evolution of digital platforms as organizational field, activities undertaken by the focal organization to increase cognitive and socio-political legitimacies lead to increase in information sharing among various constituents. 6.3 mutual awareness the activities undertaken by the focal organization to increase cognitive and socio-political legitimacy of the digital venture also leads to an increase in mutual awareness between various constituents. the increase in mutual awareness can be seen in the willingness of focal organizations and other digital platforms to make their apis open, allow in-app access to the services of partners and other platforms, and more partnerships with technology firms and others. increasing mutual awareness within the ecosystem can also be seen in technology products using services provided by ola, or mimicking them. for e.g. cab aggregator apps, location-based services etc. thus, proposition 6: during the process of evolution of digital platforms as organizational field, activities undertaken by the focal organization to increase cognitive and socio-political legitimacies lead to increase in mutual awareness among various constituents. 6.4 dominant structures and coalition during the phase of innovation introduction, no specific structures of domination or patterns of coalition could be observed. however, in subsequent three years, multiple coalitions, both formal and informal emerged with identifiable patterns of interaction such as domination, subordination, conflict and cooperation (dimaggio & powell, 1983). the triggers for formation of groups and coalitions were to deal with regulatory and normative issues related to work norms. thus, formal association of ola and uber taxi driver, for collective bargaining and setting of working norms. similarly, protests from associations of traditional taxi drivers against ola and uber, along with concerns of surge pricing were reasons for various state governments to either put forth advisories or issue warnings to digital platforms. perhaps, one australasian journal of information systems mukerji & roy 2019, vol 23, research article platform interactions and evolution of ola 16 of the prominent group emerging is the global alliance of ridesharing platforms formed to compete with uber worldwide. proposition 7: during the process of evolution of digital platforms as organizational field, activities undertaken by the focal organization to increase cognitive and socio-political legitimacies lead to formation of dominant structures and coalitions 7 conclusions technological advancements and innovations often create new organizational fields. we have seen such institutionalization of new practices into standards across domains of computer software, music ratings, microfinance and professional services. in this paper, we attempt to trace the emergence of digital platforms as organizational fields based on network logic. digital platforms are new organizational forms, considered to be much different from traditional it-enabled business. the key differentiator among the two is the central role played by big data technologies and algorithms to enable what is popularly referred to as network logics. digital platforms are often considered to be disruptive for traditional businesses. however, adopting an institutional perspective enables us to comprehend that occasional protests by various stakeholders are part of the process of institutionalization where a new logic is seeking legitimacy and adoption. similarly, the supposedly irrational market valuations of digital firms, acquisitions and sale of ventures are all imperative to translate the logics into business. future extension of the study will incorporate case studies on other leading indian digital platforms to refine the theoretical framework. references aldrich, h. e., & fiol, c. m. (1994). fools rush in: the institutional context of creation. academy of management review, 19(4), 645–670. https://doi.org/10.2307/258740 alstyne, m. w. van, parker, g. g., & choudhary, s. p. (2016). pipelines, platforms , and the new rules of strategy. harvard business review, (april 2016). anand, n., & peterson, r. a. (2000). when market information constitutes fields: sensemaking of markets in the commercial music industry. organization science, 11(3), 270–284. https://doi.org/10.1287/orsc.11.3.270.12502 avgerou, c. (2000). it and organizational change: an institutionalist approach. information technology and people, 13(4), 234–262. bala, h., & venkatesh, v. (2007). assimilation of interorganizational business process standards. information systems research , 18(3) , 340–362. https://doi.org/10.1287/isre.1070.0134 cannon, a. r., & woszczynski, a. b. (2002). crises and revolutions in information technology: lessons learned from y2k. industrial management & data systems, 102(6), 318–324. https://doi.org/10.1108/02635570210432019 chatterjee, d., grewal, r. and sambamurthy, v. (2002). shaping up for e-commerce: institutional enablers of the organizational assimilation of web technologies. mis quarterly, 26(2). australasian journal of information systems mukerji & roy 2019, vol 23, research article platform interactions and evolution of ola 17 currie, w. (2009). contextualising the it artefact: towards a wider research agenda for is using institutional theory. information technology and people, 22(1), 63–77. https://doi.org/10.1108/09593840910937508 currie, w. l., & guah, m. w. (2007). conflicting institutional logics: a national programme for it in the organisational field of healthcare. journal of information technology, 22(3), 235–247. https://doi.org/10.1057/palgrave.jit.2000102 dacin, m. t., jerry, g., & scott, w. r. (2002). institutional theory and institutional change: introduction to the special research forum. academy of management journal, 45(1), 45–57. damsgaard, j., & lyytinen, k. (1998). contours of diffusion of electronic data interchange in finland. the journal of strategic information systems, 7(4), 275–297. https://doi.org/10.1016/s0963-8687(98)00032-8 dimaggio, p. j., & powell, w. w. (1983). the iron cage revisited: institutional isomorphism and collective rationality in organizational fields. american sociological review, 48(2), 147–160. garud, r., jain, s., & kumaraswamy, a. (2016). institutional entrepreneurship in the sponsorship of common technological standards: the case of sun microsystems and java. the academy of management journal, 45(1), 196–214. https://doi.org/10.2307/3069292 greenwood, r., suddaby, r., & hinnings, c. (2002). theorizing change: the role of professional associations in the transformation of institutionalized fields. academy of management journal, 45(1), 58–80. hinings, b., gegenhuber, t., & greenwood, r. (2018). digital innovation and transformation: an institutional perspective. information and organization, 28(1), 52–61. https://doi.org/10.1016/j.infoandorg.2018.02.004 kauppi, k. (2013). extending the use of institutional theory in operations and supply chain management research: review and research suggestions. international journal of operations and production management, 33(10), 1318–1345. https://doi.org/10.1108/ijopm10-2011-0364 maguire, s., hardy, c., & lawrence, t. b. (2004). institutional entrepreneurship in emerging fields: hiv / aids treatment advocacy. academy of management journal, 47(5), 657–679. https://doi.org/145.107.119.108 mair, j., & reischauer, g. (2017). capturing the dynamics of the sharing economy: institutional research on the plural forms and practices of sharing economy organizations. technological forecasting & social change, 125(may), 11–20. https://doi.org/10.1016/j.techfore.2017.05.023 mcpherson, c. m., & sauder, m. (2015). logics in action: managing institutional complexity in a drug court. administrative science quarterly, 58(2), 165–196. https://doi.org/10.1177/0001839213486447 ojha, a. k., & rao, r. a. (2014). the emergence of an organizational field: the case of open source software. vikalpa, 39(2), 127–143. https://doi.org/10.1177/0256090920140212 purdy, j. m., & gray, b. (2009). conflicting logics, mechanisms of diffusion and multilevel dynamics in emerging institutional fields. academy of management journal, 52(2). australasian journal of information systems mukerji & roy 2019, vol 23, research article platform interactions and evolution of ola 18 suchman, m. c. (1995). managing legitimacy: strategic and institutional approaches. academy of management review, 20(3), 571–610. https://doi.org/10.5465/amr.1995.9508080331 teo, h. h., wei, k. k., & benbasat, i. (2003). predicting intention to adopt interorganizational linkages: an institutional perspective. mis quarterly, 27(1), 19–49. australasian journal of information systems mukerji & roy 2019, vol 23, research article platform interactions and evolution of ola 19 appendix 1: chronological list of events, isomorphic pressures, actions and responses stage 1: innovation introduction month / year event comments field configuring event dec 2010 ola established as a firm innovation introduction aug 2013 uber plans entry in india field configuring event funding apr 2011 $330, 000 resource apr 2012 $5000,000 resource nov 2013 $20, 000, 000 resource jul 2014 $40, 000, 000 resource oct 2014 $210, 000, 000 resource partnerships nov 2012 makemytrip customers / increasing interaction and cognition jun 2014 appiterate technology/ increasing interaction legitimacy may 2014 olacabs offer 50% discount in delhi as autos go on strike cognitive legitimacy oct 2014 ola builds bridges with traditional taxis socio-political legitimacy competition jan 2014 launches luxury fleet in mumbai competitive (new market) feb 2014 launches ola mini in bangalore competitive (new market) mar 2014 launches ola mini in delhi competitive (new market) aug 2014 launch in ahmedabad competitive (new market)/ mimetic (tfs) sep 2014 launches ola prime in six cities competitive (new market) / mimetic (uber) stage 2: mobilization month / year event comments field configuring events dec 2014 delhi rape case field configuring event dec 2015 / apr 2016 odd-even field event dec 2016 uber gets funding from ola investor field event / source of competitive and coercive pressure aug 2016 uber exits from china field configuring event funding apr 2015 $315 million resource apr 2015 $400 million resource jul 2015 ratan tata invests in ola funding / cognitive legitimacy sep 2015 didi invests in ola funding / mutual awareness / partnership for taking on competition nov 2015 $500m outcome of sociopolitical legitimacy feb 2017 $330 million resource apr 2017 $250 million resource australasian journal of information systems mukerji & roy 2019, vol 23, research article platform interactions and evolution of ola 20 may 2017 6, 700,000,000 resource may 2017 reports loss of $360 million loss partnerships / acquisitions / investments feb 2015 ola acquires taxiforsure for $200 million competitive (increase customers and cabs) jul 2015 tie with myntra increase with existing customers / mutual awareness aug 2015 oneplus 2 increase with existing customers / mutual awareness nov 2015 nissan partnerships for car sale dec 2015 partnership with lyft mutual awareness / taking on competition oct 2015 ola buys majority stakes in zipcash investments in technology / ecosystem nov 2015 acquires geotagg investments in technology jan 2016 partnership with hike microapp investments in technology jan 2016 launches enterprise transport solution technology / increase in information among partners mar 2016 google maps integrate ola bookings technology / network logics / increase in information and interaction mar 2016 acquires qarth, a mobile payments startup technology / new service integration / boosting network logics mar 2016 bajaj alliance insurance partnership / mutual awareness/ normative work practice apr 2016 funds tork motors to develop electric motorcycles investments in technology / ecosystem jun 2016 launces b2b app to track fleet and evaluate drivers technology / increase in information among partners aug 2016 flipkart promotion of lenovo smartphone / mutual awareness sep 2016 integrates siri in its app technology sep 2016 mahindra and mahindra partnership for cars oct 2016 bmw partnership for luxury fleet nov 2016 apple music ola play for connected car experience / competition/ interaction / mar 7 ola invest $7.5 in leasing subsidiary investment for a structural change apr 2017 integrates app with upi technology coercive pressures dec 2014 delhi government bans app-based taxies coercive (regulatory) mar 2015 ola / uber ordered to stop operating in delhi till they obtain license coercive (regulatory) may 2015 central govt issues advisory on taxi-apps normative june 2015 license application for ola rejected in delhi – cannot operate legally, later invalidated by court coercive / socio-political legitimacy june 2015 taxis and autos strike against ola in mumbai field event / coalition against aug 2015 delhi govt proposes “city taxi scheme” normative (regulatory) sep 2015 ride sharing companies team up to take on uber competitive pressure from uber/ resulting in a coalition oct 2015 proposed homogenous rules on ride hailing services normative (regulatory) oct 2015 hc allows ola to run diesel taxis coercive/ socio-political legitimacy oct 2015 regulation of surge prices coercive dec 2015 ola shuttle services suspended; luxury diesel cars ban in delhi coercive jan 2016 delhi govt complains to high court about overcharging coercive jan 2016 karnataka government mulls stricter provisions for taxi aggregators normative australasian journal of information systems mukerji & roy 2019, vol 23, research article platform interactions and evolution of ola 21 jan 2016 ola shuttle service termed illegal by government in karnataka coercive mar 2016 ola bike taxis termed illegal by karnataka govt coercive mar 2016 ola asked to provide info about taxis to be phased out normative / regulative apr 2016 karnataka govt cracks down on surge pricing coercive apr 2016 diesel cabs banned in delhi ncr coercive may 2016 rbi issues regulation for compliance for mobile payments normative may 2016 bangalore airport may allow to wait in premises normative / outcome of cognition and mutual awarenrss may 2016 geospatial bill normative jun 2016 ola licensed under karnataka on-demand transportation technology aggregator rules normative jan 2017 ola and uber ridesharing declared illegal by karnataka government normative competitive and mimetic pressure mar 2015 introduces food ordering service competitive (increase customers, service) apr 2015 uber launches autorickshaw service, allows cash payment mimetic / follows ola’s practice jun 2015 introduces the concept of cashback mimetic (followed other e-commerce sites) (maintain existing customers and increase interaction) jun 2015 launches app for local grocery deliveries mimetic jul 2015 uber sues ola for $7.5 million competitive aug 2015 ola select launched – exclusive benefit club mimetic (retain existing customer, increase interaction) aug 2015 opens ola money to other platforms mimetic (in response to similar service by ola) sep 2015 introduces shuttle service competitive / mimetic` sept 2015 opens up its api – integration with inmobi & others competitive / information sharing, mutual awareness oct 2015 launches olapool mimetic dec 2015 launches auto rickshaw in chandigarh competitive jan 2016 launched ola corporate, followed by uber within a week mimetic / new service for customers who are aware of ola feb 2016 expands ride sharing to pune and hyderabad competitive mar 2016 launch motobike taxis mimetic / follows uber mar 2016 launches ola micro apr 2016 ubergo reduces price in response to ola micro mimetic apr 2016 ola autos launched in 12 more cities competitive apr 2016 ola “micro” to 75 cities competitive mar 2016 auto richshaws available on ola app competitive (reach a new segment) / mimetic (follows uber) mar 2016 shuts down ola café, ola store consolidation mar 2016 uber and others accuse ola of unlawful practices competition / implications for legitimacy may 2016 introduces “ola lux” mimetic nov 2016 use ola money to pay utility bills / adds credit and debit card payments increase interaction jan 2017 launches “share express” to share rides with customers only on your route competitive normative pressure jun 2015 ola offers 2bhk flats as incentive to drivers normative (work practice) australasian journal of information systems mukerji & roy 2019, vol 23, research article platform interactions and evolution of ola 22 aug 2015 incentives to drivers to go green normative sep 2015 free medical insurance to drivers normative dec 2015 personal loans to driver partners normative (work practice) jan 2016 ola drivers protest claiming non-payment normative (rationalization) -> coalition / dominant group feb 2016 ola and uber drivers remove branding normative (work practice) / dominant group apr 2016 ola and uber drivers can simultaneously operate on both platforms normative work practice / mutual awareness apr 2016 cuts down on joining bonus normative may 2016 senior executives quitting / funding plunges normative may 2016 ola drivers earnings fall by 70% lower incentives and stiff competition normative jul / aug 2016 funding falters, cost cutting measures, dip in hirings, firing and layoff normative oct 2016 raise fare, reduces driver incentives normative (rationalization of work norms) jan 2017 protest by ola and uber drivers in bangalore and telengana normative / dominant coalition feb 2017 / apr 2017 / may ola / uber drivers strike in delhi and bangalore, guwahati better pay may 2017 sbi suspend bank loan for ola and uber taxis actions for gaining legitimacy jun 2015 free rides for blood donors cognitive legitimacy dec 2015 launches ride sharing in delhi to address odd even socio-political legitimacy / uber followed – mimetic dec 2015 chennai rains socio-political legitimacy jan 2016 users in delhi seek out ola/uber during odd/even experiment cognitive legitimacy gained jan 2016 mumbai kaali-peeli field their own 9211 app others wanting to adopt practice jan 2016 pune lococab as a market place for small time operators/ hyderabad based strat-up wants to aggregate cabs, food and mobile recharge app others wanting to adopt practice feb 2016 ola signs pact with up government to create 50k entrepreneurs socio-political legitimacy mar 2016 invests 200 million to cng adoption normative (regulatory + work practice) + socio-political legitimacy apr 2016 provides free rides to and from kolkata flyover collapse site socio-political legitimacy apr 2016 prime minister makes first booking of ola e-rickshaw competitive / socio-political legitimacy apr 2016 temporaliy removes surge pricing socio-political legitimacy apr 2016 / may 2016 ixigo cabs allow booking cabs without internet connection , delhi govt announces “app-based” premium bus service, cabto, taxivaxi, mindyourfleet, instago others copy or building solutions on top of ola jun 2016 signs mou with haryana govt to invest 350 crore and build 10, 000 entrepreneurs in 5 years socio-political legitimacy oct 2016 ola ambulance campaign with hospitals cognitive legitimacy jan 2017 partners with apollo hospital for road safety cognitive legitimacy stage 3: onset of structuration month/year event comments funding aug 2018 $225000000 resource sept $50m resource normative pressure jan 2018 ola drivers protest in chennai normative partnerships / acquisitions / investments australasian journal of information systems mukerji & roy 2019, vol 23, research article platform interactions and evolution of ola 23 dec 2017 ola acquires food panda for $200 million increasing interaction jan 2018 plans to expand to australia competition apr 2018 invests in vogo a two-wheeler rental startup technology apr 2018 acquires ridlr technology may 2018 setup a holding company structure to own ola cabs, foodpanda and electric vehicle unit structire jun 2018 phonepay partnership for autopay copyright: © 2019 mukerji & roy. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://journal.acs.org.au/index.php/ajis/article/view/2113 http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 the network logics of digital platforms 3 institutional theory in is research 3.1 theoretical framework 4 methods and data 5 the evolution of ola’s organizational field 5.1 stage 1: innovation introduction (start – june 2014) 5.2 stage 2: mobilization (july 2014 to june 2017) 5.3 stage 3: onset of structuration (july 2017) 6 digital platforms as emerging organizational field 6.1 increase in interactions 6.2 information sharing 6.3 mutual awareness 6.4 dominant structures and coalition 7 conclusions paper title (paper title style) australasian journal of information systems lamp 2017, vol 21 editorial 1 editorial john lamp ajis editor-in-chief deakin business school deakin university the past year, 2016, has been one of recognition and emerging maturity. ajis is now listed in cabell’s international directory, and indexed by german and korean language search engines, as well as other english language indices, such as thomson reuters’ emerging sources citation index (esci). there are current negotiations for addition to china’s ckni chinese language service. a full list is available on the ajis website. ajis has also been awarded the directory of open access journals seal for best open access publishing practice. thirteen general research articles were published in 2016, plus special sections on  design research for decision support systems development  research on rural and remote ict  selected papers from the information systems foundations conference (isf) ten post-publication reviews were published. these reviews were well received and in one instance that i have been told about, resulted in a new joint research project being established with the existing team plus the reviewer. this is an excellent development by any measure. the most significant news is that the ajis h-index has increased from 30 to 32 over 2016. this reflects a greater rate of citation of ajis articles. hopefully this is being driven by our increasing visibility and quality of authorship. our readership continues to grow (figure 1) with most now discovering our material through google scholar or cabell’s international directory. figure 1: ajis readership our greater visibility is resulting in an increasing number of papers being submitted. in 2016, 154 papers were submitted. it is worth reflecting on what that means in an organisational sense. the most significant issue is that this creates a demand for 154 section editors and 463 reviewers. that is over 600 positions which must be filled over the year. 0 2000 4000 6000 8000 10000 12000 australasia northern america southeast asia northern europe southern asia western europe western asia eastern asia eastern africa southern europe location of readers 2016 2015 australasian journal of information systems lamp 2017, vol 21 editorial 2 this results in a major exercise in diplomacy on my part when seeking section editors, and on their part when seeking reviewers. it goes without saying that ajis owes a huge debt to those colleagues who volunteer, especially those taking on the editorship of special sections. i would encourage new phd students in particular to become reviewers as that is, in my view, the best way to get an idea of standards of papers being submitted, and the degree of development they go through before publishing. this can be a great confidence builder. the five most popular ajis articles in 2016 were: baskarada, s., & koronios, a. (2013). data, information, knowledge, wisdom (dikw): a semiotic theoretical and empirical exploration of the hierarchy and its quality dimension. gartlan, j., & shanks, g. (2007). the alignment of business and information technology strategy in australia. kendall, j., & kendall, k. (2012). storytelling as a qualitative method for is research: heralding the heroic and echoing the mythic. lauer, t., & tanniru, m. (2001). knowledge management audit a methodology and case study. mathrani, s., & viehland, d. (2009). business benefits from enterprise systems implementation in small and medium-sized enterprises. the ajis advisory board has been most helpful as always and i must acknowledge them. australasian association for information systems deborah bunker, university of sydney don kerr, university of the sunshine coast australian council of professors and heads of information systems caroline chan, rmit university glenn stewart, queensland university of technology professors and heads of information systems, new zealand annette mills, university of canterbury felix tan, excelsia college to all those who have given their time to ajis in 2016, many thanks, you know we would not exist without your valuable input. another year lies ahead. copyright: © 2017 lamp. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems mishra & shah 2023, vol 27, research article prioritising sustainability factors 1 prioritising sustainability factors for public-private partnership (ppp)-based mature telecentres using the ‘akshaya’ project as a case gaurav mishra development management institute, patna, india gmishra@dmi.ac.in reena shah shreyath university, ahmedabad, india abstract telecentres are considered an essential information and communication technology (ict) platform to deliver e-government services and play a vital role in reducing the digital divide, thereby enhancing access to e-government services. this study aimed to identify the factors that financially sustain a ppp-based mature telecentre and explore the hierarchy among these factors. an in-depth review of existing literature was conducted to identify sustainability factors. these factors were subjected to further validation through a qualitative field study. then, interpretive structural modelling (ism) was used to understand the interrelationships and identify the hierarchy between the identified factors. finally, the decision-making trial and evaluation laboratory (dematel) was used to validate the developed hierarchical model. the results indicated that in order of their hierarchy, ‘centre resources’, ‘affordability’, ‘quality of services’, ‘quality of operators’, and ‘convenience’ were the influential factors that sustained a mature telecentre. the critical role of village-level entrepreneurs and the importance of integration between public and private entities at each stage of the hierarchy were emphasised. this paper presents theoretical and practical implications for stakeholders to adopt appropriate measures to sustain telecentres. keywords: ict, telecentre adoption, ism, e-governance. 1 introduction technological innovation and using it as a tool for human development have been the focus of discussion among academics, practitioners, and policymakers (rhodes, 2004). information and communication technology (ict) can play an important role in development. egovernment is a term that refers to the application of ict for conducting governance functions through interactions between government and its citizens for efficient delivery of services (iyer & rao, 2017; joshi, 2018). as an ict application, telecentres are crucial for governance reform required for the development of communities (madon, 2005). telecentres have become an important ict platform for e-government service delivery (kiran, 2014). telecentres aim to support development among the underprivileged population (furuholt & sæbø, 2018). telecentres serve as single facilitation points for delivering government and business services for rural people. moreover, they act as multipurpose community centres serving as communication hubs that provide various services. they have a complex, interrelated structure based on the involvement of multiple stakeholders, such as government, citizens, private sector, development organisations, other public organisations, and even australasian journal of information systems mishra & shah 2023, vol 27, research article prioritising sustainability factors 2 village-level entrepreneurs (vles) at the micro-level. the vles invest, maintain, and operate the telecentres (mishra, 2018) and are intermediaries between technology and citizens. mobile government (m-government) is being tractioned because of mobile penetration in urban and rural areas. in volume terms in 2018, india was the second-largest smartphone market, with a growth rate of nearly 15% (india cellular and electronics association, 2020). several factors, such as reduced smartphone prices, affordable mobile data, increase in rural users, rise in vernacular accessibility, and government impetus, including internet banking for payment of bills and managing the aadhar card (unique identification card issued by the indian government) by using the mobile application (aledhari, aneja, bashir, bennett, bielby, hicks, johnson, mandal, mantri, nighot & wack, 2018; motiani, 2019), may increase the use of certain m-government services. moreover, telecentres may become extinct in india because of m-government (sharma & ray, 2019). however, many innovative mobile-based applications in government domains remain in their early stages (jauhari & maheshwari, 2014). m-government should not be viewed as a replacement for e-government but is expected to be used to explore avenues for e-government integration (hellström, 2008). telecentres remain the optimum platform for delivering egovernment services in countries, such as india, where factors, including limited internet access, technology skills, and literacy rate, majorly hinder the progress of m-government. the government of india data state that in the financial year 2019–2020, the operational telecentres (also referred to as common service centres (cscs)) increased to 360,873 against 345,246 operational telecentres at the close of the financial year 2018–191. 2 public–private partnership (ppp) public–private partnership has received considerable interest as cooperative institutional mechanisms between the public and private sectors in areas such as public services and infrastructure. the ppp approach provides an opportunity to overcome the problems of complex policies and public services because of joint development and risk sharing between partners (wang, xiong, wu & zhu, 2018). in addition, given the opportunities of innovation, efficiency, and finance (bayliss & van waeyenberge, 2018), both the developed and developing economies use ppp as an alternative solution in project design and implementation. in the literature, ppp has not been defined consistently by practitioners and researchers (cuttaree & mandri-perrott, 2011). world bank defines ppp as a ‘contractual agreement between a private party and a government agency, for providing a public asset or service, in which the private party bears significant risk and management responsibility’2. in a wider view, organisation for economic co-operation and development (oecd) (2012), defines ppp as ‘long-term contractual arrangements between the government and a private partner whereby the latter delivers and funds public services using a capital asset, sharing the associated risks’, while emphasising the role of private partners. similarly, garvin and bosso 1 csc annual report, 2019–20, available on https://csc.gov.in/assets/events-report/annual-report-201920.pdf, accessed on 20/12/2021 2 accessed from https://ppp.worldbank.org/public-private-partnership/overview/what-are-publicprivate-partnerships, accessed on 20/04/2021 https://csc.gov.in/assets/events-report/annual-report-2019-20.pdf https://csc.gov.in/assets/events-report/annual-report-2019-20.pdf https://ppp.worldbank.org/public-private-partnership/overview/what-are-public-private-partnerships https://ppp.worldbank.org/public-private-partnership/overview/what-are-public-private-partnerships australasian journal of information systems mishra & shah 2023, vol 27, research article prioritising sustainability factors 3 (2008) highlight the management, operations, and financial risk aspects of the private party in this long-term agreement primarily set up for mutual benefits. despite these differences, the following common elements are present in all the definitions (hodge & greve, 2017): a formal partnership contract; objective to meet government’s economic or social requirements; substantial risks and management responsibilities of the private entity; remuneration to private entities based on output; and varied forms of undertakings and rewards. the form of ppps changes with the nature and duration of the project, purpose, and desired outcomes, and the level of investment involved. to simplify the resulting structures, cruz and marques (2013) classify them into two parameters: (a) the stage of the partnership, that is, design, build, and maintenance and (b) ownership of asset, that is, public ownership, lease, rent, and divestiture. for instance, in some popular ppp forms, such as build-own-operate (boo), the private entity finances, designs, develops, and maintains the public project and retains the ownership. on the other side, the government (public entity) may not have a longterm service purchase agreement, for example, power purchase agreement in a power distribution project. in contrast to boo, in design-build-operate (dbo), the private entity designs, builds, and operates the public facility but the government retains the ownership. further, in case of the boo-transfer (boot) form, the operation of the project is undertaken by the private entity for a specified period of time, after which it is transferred to the government at a previously agreed price or even market price as stated in the contract. 3 ppp in telecentre-based e-government projects to drive their growth agenda through telecentre-based e-government projects, several countries face constraints such as insufficient funds, lack of technical expertise, and limited execution capabilities. the ppp approach provides an opportunity to address such constraints in e-government and other project implementation. for example, india is a prominent global ppp leader (world bank, 2018) and has used the ppp alternative in various sectors such as infrastructure, energy, transport, water, sanitation, telecommunication, and e-government (laksmanan, 2008). research supports the association between ppp and e-government to propel socioeconomic development (mkude & wimmer, 2015). according to aundhe and narasimhan (2016), ppp helps the government leverage the private enterprise’s operational efficiency to achieve their development goals. in e-government, ppp is defined as a legally enforceable contract between a private sector entity and government body that requires the private partner to provide a desired electronic public service, for which the private sector is required to invest some of its resources and become responsible for some risks of service delivery (emilio, 2015). risk in e-government ppp projects is associated with changing technology and service requirements that must be upgraded constantly (aundhe & narasimhan, 2016). in ppp-based telecentres, the private sector runs the telecentre with the revenue generated from the services provided to citizens, the government, and businesses. the government departments must prepare themselves in backend processes and appropriate systems and with a mindset to deal effectively with the private sector. telecentres that operate on the ppp model provide services and other necessary support (e.g. rent-free offices). hence, telecentres mainly follow the boo type of ppp. australasian journal of information systems mishra & shah 2023, vol 27, research article prioritising sustainability factors 4 another pertinent aspect of these telecentres is that stakeholders directly affect the implementation of a project and can influence the system at various stages of project development. telecentre projects have multi-stakeholder participation and cooperation. de (2005) classifies these stakeholders into supply and demand sides. the supply-side stakeholders are primarily involved in funding, designing, implementing, and maintaining the system, whereas the demand-side stakeholders consume the system's services and help provide revenue for sustaining the system. for instance, the government is not a single entity because it reaches and affects departments, groups, organisations, and citizens (sealy, 2003). in the ppp model, the telecentre ecosystem is likely to increase people’s participation, reduce government interference, provide service at the doorstep, improve transparency, increase employment opportunities, and nurture entrepreneurs. however, at the ground level, the practitioners face several challenges and difficulties while operating these ppps (basílio, 2017). according to heeks (2003), most telecentre-based e-government projects fail either totally or partially. one of the primary reasons for this failure is the gap between the ‘design’ and ‘reality’ of such interventions. 4 telecentre sustainability the telecentre literature discusses mainly five types of sustainability—financial, social, technological, policy (institutional), and environmental. according to madon (2010), financial sustainability refers to the telecentre’s ability to generate adequate revenue from the local community for activities it offers. social sustainability is related to how compatible the telecentre and its services are with the socio-political context of its location (masiero, 2011). technological sustainability involves the ability of telecentres to upgrade computer hardware and software continuously so that services are not affected by degradation/obsoleteness of the equipment or network (kumar & best, 2006). policy or institutional sustainability reflects the telecentre’s acceptance and institutionalisation by political actors (faroqi, siddiquee, & ullah, 2019). environmental sustainability entails plans for electronic equipment disposal or reuse when they reach the end of their life (kumar & best, 2006). attwood, diga, braathen, & may (2013) indicate that the sustainability of telecentres depends on countless interconnected factors involving multiple stakeholders. telecentres require time to evolve and mature institutionally and functionally. ppp-based telecentres usually require three years for financial sustainability or independence (faroqi & siddiquee, 2017). after the start-up phase, telecentres have to sustain themselves financially while operating in the free market (masiero, 2011). according to hudson (2001), in the long term, because of the realisation of socioeconomic effects and opportunity cost of alternative delivery modes, telecentres may become financially sustainable. in this study, the financial sustainability of telecentres is specifically focused. we are of the view that financial sustainability is relevant for a ppp-based telecentre supported by an external public entity. in ppp-based telecentres, because vles invest, operate, and maintain telecentres, revenue from telecentres is crucial in supporting vle livelihood. the vles would like to receive high returns from their investment. hence, financial sustainability of telecentres is crucial not only for their operation and maintenance but also for vles’ livelihood. telecentres are similar to enterprises, and therefore, financial sustainability is an immediate concern for telecentres (shadrach & sharma, 2011). australasian journal of information systems mishra & shah 2023, vol 27, research article prioritising sustainability factors 5 5 focus of the study this study focused on understanding the factors and their interrelationships that financially sustain matured ppp-based telecentres. previous telecentre literature has mainly explored aspects related to service delivery (furuholt & sæbø, 2018; weerakkody, el-haddadeh, sivarajah, omar, & molnar, 2019), social impacts (faroqi & siddiquee, 2017; rashid, 2017), usage (kapondera & namusanya, 2017; verkijika & de wet, 2018), and adoption (alawadhi, 2019; carter, weerakkody, phillips, & dwivedi, 2016; jaganathan, ahmad, ishak, nafi, & uthamaputhran, 2018). however, literature on what sustains a matured telecentre in the longterm is scarce. the authors believe that a mature telecentre may provide better insights into sustainability factors rather than one with a shorter existence. mature telecentres have been in the community for some time and have reached operational sustainability and stable management (fuchs, 1998; lo, songan, cheuk, atang, & yeo, 2013). the maturity stage of telecentres affects their effectiveness; hence, research on the factors that sustain matured telecentres is recommended (palaco, park, kim, & rho, 2019). in addition, understanding how the factors that sustain telecentres interact or affect each other given the complexity of their systems is crucial (nasim, 2011). the identified factors must be ranked so that the stakeholders in the ppp allocate their limited resources to the most critical factors and draw necessary strategies to improve e-government service delivery through telecentres (gupta, bhaskar, & singh, 2017). lal & haleem (2009) recommended using graphics and words to portray the complexity of interrelationships among factors in a simplified way. the direct and indirect relationships between the factors picture the situation far more precisely than when considered in isolation (janssen, rana, slade, & dwivedi, 2018). thus, with the view to understand the complex interactions of sustainability factors and represent the hierarchy and interactions among the factors simplified, interpretive structural modelling (ism) is employed. ism is a method used to transform multifaceted issues into a clear and structured model showing well-defined interrelationships among various elements (sage, 1977; warfield, 1974). further, the decision-making trial and evaluation laboratory (dematel) is used to validate ism. a combination of ism and dematel is used to provide a complete and highly comprehensible analysis of sustainability factors associated with ppp-based telecentres. ism and dematel are similar because both methods attempt to explain complex nodes among factors. therefore, integrating them to obtain a clear view of variable arrangements is easy (zhou, zhang, & li, 2006). warfield first proposed ism in 1974, and since then, it has been instrumental in providing deep insights into diverse subjects to practitioners and academicians. in ism, ‘i’ stands for interpretative because an expert’s judgment and practical experience are used to define the relationships among all variables, ‘s’ denotes the structure, which is developed on the basis of the extracted interrelationship among variables, and ‘m’ signifies graphical presentation of the overall model depicting the hierarchical relationship among variables. since its inception, this methodology has been employed to develop fields such as education programmes (hawthorne & sage, 1975), cross-cultural communication (jedlicka & mayer, 1980), energy conservation in the cement industry (saxena & vrat, 1992), vendor selection (mandal & deshmukh, 1994), and waste management (h. d. sharma & gupta, 1995). furthermore, information-technology-driven studies, including information security management (muktesh chander, jain, & shankar, 2013), e-commerce (valmohammadi & australasian journal of information systems mishra & shah 2023, vol 27, research article prioritising sustainability factors 6 dashti, 2016), and risk considerations in software engineering projects (samantra, datta, mahapatra, & debata, 2016), have witnessed the rising endorsement of ism. dematel is used to validate the model developed through ism. this comprehensive method originated in battelle geneva institute in the 1970s and provides direct and indirect causal relationships between variables in complex systems (gabus & fontela, 1972). the dematel method can be used to validate and reinforce the ism model. e-government services through telecentres may be an emerging field where ism can convert complex systems with multiple stakeholders into well-defined models. few researchers have used ism and dematel in e-government services (lal & haleem, 2009; liu, wang, xie, & zhan, 2020) but not explicitly for telecentre sustainability or a combination of both models. this study, therefore, attempts to develop a mature telecentre model by examining and analysing the factors and interrelationships that financially sustain ppp-based mature telecentres. overall, the objectives of the present study are: 1. to identify the factors that financially sustain a ppp-based mature telecentres 2. to identify the hierarchy among sustainability factors using ism and validate them using dematel 6 methodology in this study, first, related literature is reviewed, and based on this review, telecentre sustainability factors were identified. then, a field study was conducted to verify the identified factors in the field setting using the case of the ‘akshaya’ project. ism was then used to develop a model illustrating the hierarchy and interrelationships among the identified factors. further, the model was validated using dematel. finally, the data were analysed using appropriate tools. the following sections describe these steps in detail. 6.1 literature review we focused on the literature mainly on e-government, factors influencing telecentre use and sustainability, the role of ppp in ict-based projects, adoption of a telecentre, types of sustainability, and telecentre lifecycle (stages of telecentre growth) because they are related to our study objectives. the methodology proposed by webster & watson (2002) to enlist factors that influence telecentre sustainability was employed. a literature review is required to ascertain and evaluate the functional know-how to determine the beneficial or ineffective measures for an identified phenomenon. (knopf, 2006). we identified the articles using the isi web of science (wos) library databases. in addition, for inclusivity, as thapa and sæbø (2014) suggested, we searched relevant journals and a broader generic literature in e-government, telecentre movement, telecentre sustainability, and ppp, which are essential for this research, focussing mainly on high-rated journals. table 1 reflects the literature review process. based on the selection criteria of most relevant articles, 75 articles were identified for the study. according to thapa and sæbø (2014), literature reviews are conducted based on a specific selection strategy, and the process runs a high risk of excluding potentially relevant literature from sources that are not included. however, considering these limitations, we assume that the identified articles provided a relevant list of relevant factors of telecentre use. australasian journal of information systems mishra & shah 2023, vol 27, research article prioritising sustainability factors 7 particulars details library used for search e-government journals, ict4d journals, isi wos search terms telecentres for good governance, telecentre sustainability, ict4d, ppp in telecentre initiative, ppp in project sustainability, ppp in e-government, telecentre adoption, telecentre project life cycle, mature telecentres, sustainable telecentres total search results 187 selected for review 75 language english subject areas computer science, information systems, supply chain management, public policy, social science, ict and interdisciplinary areas criteria for inclusion most cited papers, relevance to the research, most cited authors, known authors in the field, published after 2000, excluded articles based on technical/computer concepts table 1. literature review process 6.2 field verification of sustainability factors the factors derived from the literature review were verified in a contextual setting. the ‘akshaya’ telecentre project was used as a case for this purpose since it is a mature telecentre project with over ten years of existence. the ‘akshaya’ project, implemented in kerala, india, aims to bridge the digital divide through ict access and is designed to link government and citizens. ‘akshaya’ telecentres are set up within a maximum of two kilometres for any household and network leveraging entrepreneurship. these centres or access points are visualised as the principal route to the project’s success. according to the project website, currently, there are approximately 2,650 ‘akshaya’ telecentres. various government services, including aadhar (unique id) enrolment, utility bill payment, ration card applications, motor vehicle license payment, commercial tax filing, and university fee payment, are provided through these centres. at least two telecentres are available in each panchayat (a local rural administrative unit). 6.2.1 selection of telecentres and respondents two districts were selected in kerala, namely mallapuram and kollam. these were selected on the basis of project maturity. in mallapuram and kollam, the project has been active since 2001 and 2005, respectively. maturity was an essential criterion for selection because this study relates to the factors that sustain such telecentres. in each district, two centres were selected on the basis of the usage pattern. one centre is used comparatively more than the other (based on the number of transactions in the project records). nendumgolam (high use) and thenmala (less use) centres were selected in kollam, and kishreri (high use) and velliyampuram (less use) centres were selected in mallapuram. table 2 presents the number of respondents (users) with telecentres. for the study, we randomly selected the users. every fourth person who visited the telecentre was interviewed. each interview took 30–45 min. however, convenient sampling was adopted for non-users because it helped us interact with them as vles knew them. the interview protocol is available in appendix 1. districts in kerala telecentre location respondents-users respondents-non-users kollam nendumgolam 34 30 thenmala 28 31 mallapuram velliyampuram 36 33 kishreri 37 25 table 2. number of respondents for each telecentre australasian journal of information systems mishra & shah 2023, vol 27, research article prioritising sustainability factors 8 during the investigation in the telecentres, data saturation was attained after 25–30 respondents in each of the four telecentres. the number of respondents was based on the generally used qualitative technique of data saturation, that is, nothing new becomes evident beyond a particular stage of interviewing (fusch & ness, 2015). we also interacted with approximately 20 vles and two senior project officials at the state level to comprehensively understand the context and two officials at the district level. 6.3 model development using interpretive structural modelling multiple sources and techniques for data collection and analysis such as detailed literature reviews, in-depth interviews, expert’s judgement, and brainstorming sessions are recommended to enhance the reliability and validity of ism (warfield, 1974). in the present study, all these techniques were used; the factors identified from the literature review and verified through the field study were presented to experts. figure 1. ism process used in the study initially, three academics with more than ten years of research experience and three experts who closely worked with telecentres were provided with the factors, definitions, and explanations. subsequently, the final list of factors with their description was presented to the experts to establish contextual relationships. in the second brainstorming session, the group australasian journal of information systems mishra & shah 2023, vol 27, research article prioritising sustainability factors 9 reached a consensus for the final contextual relationship, which was ultimately used for developing the ism model. the steps in the ism process are illustrated in figure 1. 6.4 ism model validation using dematel this study used dematel to identify the factors that drive sustainability. for dematel analysis, a five-step process was used and analysed in detail; the steps are (a) create a direct relationship matrix with the experts’ support, (b) normalise the direct relationship matrix, (c) construct the total relationship matrix, (d) determine the causal parameter, and (e) determine the threshold value. a. direct relationship matrix experts solicited a pair-wise comparison between factors at five levels from 0 to 4: 0—no influence; 1—very low influence; 2—medium influence; 3—high influence, and 4—very high influence. xij represents that factor i influences factor j. in this analysis, the diagonal influence was considered zero. b. normalised direct relation matrix the normalised matrix was calculated by the following equation: s = 1 𝑀𝑀𝑀𝑀𝑀𝑀1≤𝑖𝑖≤𝑛𝑛 ∑𝑀𝑀𝑖𝑖𝑎𝑎, where s i.j = 1,…,n c. total relation matrix (trm). the trm includes all direct and indirect relations among factors and is obtained as follows: t = n (i − n) − 1 where i = identity matrix n=normalised direct relation matrix d. determine the causal parameters in the trm, ri represents the summation of all the direct and indirect relations of enabler i, while cj represents that of the columns of enabler j. ri + cj signifies the total effect of i on other factors, and ri − cj indicates the net effect of i on the system. the causes or effect relationships were decided through ri − cj. a positive result indicates a net cause, and a negative result represents a net effect. e. determining the threshold value a threshold value is the average entry of the trm. 6.5 analysis of data both qualitative and quantitative data were collected at different stages of the study. for quantitative data analysis, we used ms excel mainly in building ism and dematel models. we used nvivo software for qualitative data analysis, mainly to identify and contextualise sustainability factors in the field. the corpus of transcripts obtained from in-depth interviews was content analysed, which helped in the systemic coding and categorising of the large pool of texts (hsieh & shannon, 2005). an iterative process was then used to familiarise the content by reading, re-reading, and understanding the nature of the content to explore factors. broad themes of the factors were initially identified in the literature. based on this, the software highlighted transcript content and linked to these themes. a new node was created for coding if any new theme was reflected in the identified text. the thematic grouping passed australasian journal of information systems mishra & shah 2023, vol 27, research article prioritising sustainability factors 10 through several rounds of a recursive process, in which researchers consistently refined and reviewed codes by constantly comparing the codes and themes that emerged. the transcript text was coded with an axial code ‘faith on telecentre for transaction services’, which is part of the parent theme ‘trust’, as reflected in table 3. responses axial code overarching theme “the centre is run as a private business. i do not have faith in the services of the centres. i would prefer government offices for utility bill payments. for other services, i may use the centre.” [kp16] faith on telecentre for transaction services trust “i pay electricity bill here as i do not have time to go to the government offices. there is a long queue in a government office and therefore much time is wasted. i have to spend around 2 hours to pay the bills. when i come to the akshaya centre, i do not waste time.” [kp9] the benefit of using servicestime and money saved • time saving • convenience table 3. coding process of transcripts 7 results the results are discussed in four parts, namely literature review, field verification of factors, ism, and dematel. 7.1 literature review: factors influencing telecentre’s financial sustainability this section focuses on the relevant literature related to factors that may sustain telecentres financially operating under a ppp model. based on the literature review, we identify the following factors that affect the sustainability of telecentres: a. cost of services: the service cost is crucial for sustainability (verdegem & verleye, 2009). according to attwood et al. (2013), the financial costs of using a telecentre are a major demand-side factor influencing the use of telecentre. telecentres' reduced service costs may influence its use and enhance social inclusion (gilbert, balestrini, & littleboy, 2004; mayanja, 2006). b. money and time-saving: telecentres assist users to save money because they reduce trips to cities to seek government services (mbangala & samzugi, 2014). if the telecentres are available in a community, it helps citizens save money with the availability of the services at a nearer location (hudson, 2001; kapondera & namusanya, 2017). perceptions about inconvenient locations might result in the telecentres not being used (gollakota & pick, 2020). telecentres provide efficiency in terms of time (hudson, 2001; kumar & best, 2006). citizens save time because they do not have to travel considerably far to use government services (chander mahesh, rathod, & balaraju, 2014; mbangala & samzugi, 2014). c. reliability: here, reliability is related to the availability of services whenever citizens need them. for e-government, reliable services hold importance for citizens (verdegem & verleye, 2009). according to elliman (2006), citizens are unlikely to use e-government services if there is a lack of reliable information and data about services. telecentres may be a reliable point of contact for citizens to access information and download and send applications (naik, chitre, bhalla, & rajan, 2016). australasian journal of information systems mishra & shah 2023, vol 27, research article prioritising sustainability factors 11 d. skillsets of operator: a vle or an operator runs the operation of a telecentre. according to attwood et al. (2013), a telecentre manager (operator) is found to be an essential driver of telecentre development. poor entrepreneurship among operators contributes to non-functional telecentres (faroqi, 2019). according to oestmann & dymond (2001), telecentre resources may remain underutilised because of operators' lack of ability to utilise them. the skillset of operators remains an important consideration for the smooth functioning of telecentres (hudson, 2001). e. trust: trust of citizens in the services through telecentres is vital for its sustained use (alawadhi, 2019; carter et al., 2016; schaupp, carter, & mcbride, 2010). trust among different stakeholders is crucial for a sustainable telecentre (foko, 2018). trust in the telecentres may be one of the linkages between financial viability and broader goals in terms of social development (masiero, 2011). f. satisfaction: verdegem and verleye (2009) reported that satisfaction is a factor that affects people’s intention to use telecentres. unless users are satisfied with the services, they may not continue using telecentres and maintain the status quo (gollakota & pick, 2020). haddad & oliveira (2019) showed that satisfaction in using telecentres encourages empowerment, autonomy, and solidarity among users. g. social influence: here, social influence refers to someone’s perception that he/she should use services of a telecentre that people in their social circle may suggest (brown, dennis, & venkatesh, 2010). social influence may affect telecentre use where people are mostly illiterate and usually pass information verbally (omogor, 2013). the literature confirms the importance of social influence in using e-government services through telecentres (almajali, maqableh, & moh’d taisir masa, 2015; kapondera & hart, 2016). h. media influence: in rural areas, propagation through media plays a significant role in creating awareness about innovations (roman, 2003). media may be a means to make people aware of telecentres and their services. media provides new information that may influence people to accept new norms (arias, 2019). this factor focuses on the role of media in influencing telecentre use. here, media relates to both printed and digital media. i. digital divide: access to a telecentre may be affected by a person’s gender, caste, and education (hill, 2016; naik et al., 2016; ndiku & mwai, 2016). here, the digital divide is related to citizens’ perceptions of equality/inequality in services through telecentres. citizens may feel deprived of the telecentre services due to their gender, caste, or education. j. centre infrastructure: a combination of resources in telecentres are necessary for financial and operational sustainability (faroqi & siddiquee, 2017). these include vital ict infrastructure, including the internet, computers, printers, and photostat machine (faroqi et al., 2019). k. complexity and compatibility: complexity refers to the extent to which people perceive technology as challenging to understand and use (kante, oboko, & chepken, 2019). lack of proper knowledge about how the technologies work is a vital barrier in telecentre adoption. contrary to this, the dimension of compatibility suggests the degree to which the information given in telecentres matches with the australasian journal of information systems mishra & shah 2023, vol 27, research article prioritising sustainability factors 12 information generated via traditional sources and past experiences (gollakota, pick, & sathyapriya, 2012; rogers, 2004). consequently, the use of telecentres increases when there is better compatibility with the users’ prevailing economic, social, and value systems. l. relative advantage: relative advantage refers to how people perceive any product or service as better than the previous idea, product, or program it replaces (venkatesh, morris, davis, & davis, 2003). what constitutes a relative advantage depends upon the needs and perception of the actual users. in the case of egovernment, the greater the perceived relative benefit of the user, because of the use of computers to access current information in comparison to previous conventional information, the more rapid will be the rate of adoption of telecentres (samuel, doctor, christian, & baradi, 2020). m. observability: observability relates to the degree to which results are tangible and visible to others (rogers, 2004). high demonstrability reduces ambiguity and helps the telecentre user give concrete feedback to other community members interested in knowing about the user’s experience. thus, easier observability of results increases the likelihood of telecentres’ adoption (luo & chea, 2018). n. trialability: trialability is the extent to which the user can try a product or service on a limited basis before committing to adopt it (rogers, 2004). it is an essential factor, specifically in the case of services where a user needs to experience to perceive its value. the trialability of the telecentres would reduce the users’ uncertainty and experimentation without any cost attached is likely to increase the chances of its adoption (chigona & licker, 2008). 7.2 field verification of factors on the basis of literature review and thematic analysis, ten factors were finalised— affordability (includes direct cost and indirect savings), awareness (overall information about availability of services), convenience (denotes ease of access and use), time saving (relatively less time consumed in accessing the services), quality of operators (reflects the competency and entrepreneurial ability of the operator), quality of services (general effectiveness and efficiency of the services), social influence (recommendation by friends or relatives), media influence (impact of mass media communication about the centre), centre resources (availability of operational resources for regular functioning), satisfaction (overall satisfaction with the services), and trust (the faith placed in the operator). the factors digital divide, complexity, and compatibility, relative advantage, observability, and trialability, derived from the literature, were dropped from subsequent analysis because these factors were not validated in the field study. detailed description of the factors with their implied meanings are shown in appendix 2. 7.3 ism results the discussed steps of ism were followed, and the resulting model is described below: 7.3.1 structural self-interaction matrix (ssim) the ism methodology recommends the use of the following four symbols (warfield, 1974): • v: parameter i will lead to parameter j (one direction); australasian journal of information systems mishra & shah 2023, vol 27, research article prioritising sustainability factors 13 • a: parameter j will lead to parameter i (one direction); • x: parameters i and j will lead to each other (both directions); • o: parameters i and j are unrelated. the developed ssim is depicted in table 4 with a pair-wise comparison between the eleven telecentre factors affecting sustainability. code factors 11 10 9 8 7 6 5 4 3 2 1 1 affordability 0 v a 0 0 v v 0 a 0 x 2 awareness about centre and services 0 0 0 a a 0 a 0 a x 3 convenience v v x 0 0 v 0 v x 4 time saving v v a 0 0 v a x 5 quality of operator v v a 0 0 v x 6 quality of services v v a 0 v x 7 social influence x a a a x 8 media influence v 0 0 x 9 centre resources v v x 10 satisfaction v x 11 trust x table 4. structural self interaction matrix note: ‘i’ represents the row parameter,s while ‘j’ represents the column parameters 7.3.2 reachability matrix and transitivity check the ssim was converted into a binary relationship matrix called the initial reachability matrix (irm) by replacing v, a, x, and o in the range from 1 to 0 based on the following rules: if the (i, j) entry in the ssim is v, then the (i, j) entry in the irm becomes 1 and (j, i) entry becomes 0. if the (i, j) entry in the ssim is a, then the (i, j) entry in the irm becomes 0, and the (j, i) entry becomes 1. if the (i, j) entry in the ssim is x, then the (i, j) entry in the irm becomes 1, and the (j, i) entry also becomes 1. if the (i, j) entry in the ssim is o, then the (i, j) entry in the irm becomes 0, and the (j, i) entry also becomes 0. in addition, the relationship between nature and its strength was considered to improve the analysis further. the strength of the relationship is the measure of i’s impact on j’s probability of occurrence. the relationship strength was measured as 0.25 (low strength), 0.50 (medium strength), 0.75 (high strength), and 1 (highest strength). the values were assigned after the third round of discussions with the experts (appendix 3). the relationships with low or medium strength were omitted. those with high strength of occurrence, i.e. relationships ≥ 0.75, were retained and converted to 1 and were finally superimposed in the irm. australasian journal of information systems mishra & shah 2023, vol 27, research article prioritising sustainability factors 14 7.3.3 final reachability matrix the final reachability matrix was obtained by checking and removing the transitivity if any present in the irm. transitivity check is the process of searching 0 in the initial matrix, that is, if x leads to y and y leads to z, it implies that x leads to z, and finally replacing 0 with 1* (appendix 4). 7.3.4 level partitioning in this stage, the final reachability was degenerated into different hierarchical ordering to create the structural model (warfield, 1978). the final reachability matrix grouped factors into different levels based on the interrelationships between factors. the reachability set comprised the factor itself and other factors in the row that may assist in achieving it. as depicted in the factor column, an antecedent set comprised the factor itself and other factors that help attain it. an intersection set comprised the factors present in the reachability and the antecedent set. the factor(s) with the same intersection and reachability was assigned the top-level position in the ism hierarchy. once the factor(s) has achieved the top level, it is discarded from all sets. repeated iterations were conducted for each factor until the hierarchy's bottom-most levels were achieved (appendix 5). appendix 6 presents the levels of all factors at the end of the eight iterations. the level partition helped construct the digraph, based on which the final ism model was constructed. after transitivity removal, the digraph was transformed into ism (figure 2). figure 2. ism model australasian journal of information systems mishra & shah 2023, vol 27, research article prioritising sustainability factors 15 the relative importance of factors is from bottom to top in the hierarchy. for example, ‘centre resources’ are more important than ‘affordability’. similarly, the model's relatively most minor factor is ‘awareness’. a horizontal bi-directional arrow represents the factors at the same level. for example, ‘quality of service’ and the ‘quality of operator’ are at the same level and have bidirectional interaction. 7.4 dematel results the cause-and-effect relationship obtained on the basis of the dematel steps enlisted in section 6.4 is exhibited in table 5. the threshold value for this study was 0.0562. influential factors are factors that influence other factors. based on ri − cj values, a causal diagram depicted in figure 3 classifies factors into influential or influenced factors. seven factors were grouped as influential and four as influenced. figure 3. causal diagram based on ri − cj values sr no factors ri ci ri + ci ri − ci cause/effect 1 affordability 0.3957 0.2154 0.6111 0.1802 cause 2 awareness 0.0000 0.6904 0.6904 -0.6904 effect 3 convenience 1.0521 0.1254 1.1775 0.9267 cause 4 time saving 0.5210 0.5303 1.0514 -0.0093 effect 5 quality of operator 1.0043 0.4055 1.4098 0.5989 cause 6 quality of services 0.8951 0.7504 1.6455 0.1447 cause 7 social influence 0.3051 1.1194 1.4244 -0.8143 effect 8 media influence 0.4095 0.0000 0.4095 0.4095 cause 9 centre resources 1.6100 0.0450 1.6550 1.5649 cause 10 satisfaction 0.4022 1.2429 1.6451 -0.8407 effect 11 trust 0.2088 1.0800 1.6790 -0.8712 effect table 5. cause and effect influence australasian journal of information systems mishra & shah 2023, vol 27, research article prioritising sustainability factors 16 the influential factors identified using dematel are ‘centre resources’, ‘affordability’, ‘quality of services’, ‘quality of operator’, and ‘convenience’. these factors influence other factors ‘time saving’, ‘satisfaction’, ‘social influence’, ‘trust’, and ‘awareness’. ‘media influence’ is identified as an autonomous factor, i.e. it is not influenced and does not influence other factors. hence, media influence plays an insignificant role in the sustainability of ppp-based telecentres. 8 discussion the study's first objective was to identify factors that sustain mature ppp-based telecentres financially. the literature review identified fourteen factors influencing the financial sustainability of telecentres. we conducted field verification using 'akshaya' as a case to ascertain whether the identified factors hold in the context, i.e. a ppp-based mature telecentre. the field verification helped us identify eleven key factors and define them in the contextual setting. the field verification revealed a new factor, 'awareness', which is about citizens being aware that the telecentre services are that of government and that they are run by private entities that provide government-related services. 'awareness' as a direct linkage between individual users, government, and private entities has not been highlighted as an independent factor in previous studies. contrary to this, the factors complexity, compatibility, relative advantage, observability, trialability, and digital divide often cited in the literature did not resonate in the field study. for the study's second objective, we developed the ism model to identify the factors' hierarchy and interrelationships. for ism, the interrelationships were enriched through anecdotal evidence in the field and brainstorming with experts. finally, we used dematel to identify the most influential factors and relate them with the findings of the ism model. the hierarchy of the factors derived from ism (figure 2) illustrates that 'centre resources' is the foundational factor for sustaining a ppp-based mature telecentre. any other factor does not influence this factor because it is present at the model's base. 'centre resources' include internet connection, continuous electricity, seating facility, operational hardware (including computer system, printers, photocopying machine, and camera) and software, and availability of services to provide e-government services. these resources are the vle's responsibility, and thus in a ppp model, the private entity is the prime driver for the telecentre's financial sustainability. the next important factor is the 'affordability' of services. here, 'affordability' is not limited to the cost of services, as reported in the literature. the data from the field also shows that it includes the expenses saved in travel, food, and daily wages while using a telecentre. hence, the factor reflects the vle's role in keeping the cost of services appropriate to attract a high number of users. in addition, the telecentre is to be operated from a strategic location to help citizens save expenses they would otherwise incur while using government offices. at this point, it is worth noting that 'centre resources' influence 'affordability'. with time, the cost of services become less (chen & gant, 2001). hence, the services are less costly for mature telecentres because the vles can provide more services by using the same resources. moreover, the public entity, i.e. the state government, encourages citizens to use services by paying a part of the service cost. for example, the telecentres supported the e-literacy programme during the initial years. the government telecentres envisaged providing basic computer literacy to at least one family member. the beneficiary had to pay only 12.5% of the total fees, and the state government paid the remaining fees. in addition, vles made australasian journal of information systems mishra & shah 2023, vol 27, research article prioritising sustainability factors 17 additional efforts to include services on their own, such as linkages with banks for bank account-related services and mobile phone companies to provide services related to mobile phone connection. in addition, some telecentres were located in the rent-free campuses of panchayats. hence, the public entity may provide the necessary support to the vles through subsidies towards operational costs such as the internet, rent-free space, and others to make services affordable in the long run. at the next level are the factors 'quality of services' and 'quality of operator'. when services become affordable in the long run due to an increase in the number of services and optimum utilisation of resources, the quality of services and operator influence telecentres' sustainability. quality of services for customers is how quickly and proactively services are delivered, issues resolved, and questions responded to. the quality of service would improve when both public and private entities learn from each other about e-government applications and their capability (chen & gant, 2001). the same was evident in the case of 'akshaya', where vles, operators, and the state government work together during workshops towards improved quality of services. the government trains vles and operators on e-government applications and keeps them educated on the latest services, processes, and computer hardware and software requirements. such capacity-enhancing activities helps in maintaining the necessary quality of services and the quality of operators. thus, 'quality of services' and 'quality of operators' make telecentres more convenient to use than government offices because 'convenience' comes at the next level in ism. this finding is in line with gollakota et al. (2012) that capable management of telecentres improves the convenience of services and stimulates improved access. therefore, the chances of financial sustainability may increase due to enough users willing to pay for the services over a long period (furuholt & sæbø, 2018). 'convenience' leads to 'time saving'. owing to various aspects of convenience, there is time saving, citizens can access services locally with flexible timings, and service delivery is faster and hassle free. at the next level of the ism model, 'satisfaction' reflects the overall contentment with the ease of access, quality of service, and service delivery. we observe that the overall satisfaction with the telecentre is propagated through word of mouth, resulting in 'social influence', i.e. people tend to use the services because their social contacts have used them or vles have influenced them to use the telecentres. people are also influenced by television advertisements and printed materials circulated by vles and the government. people influenced by media pass the information to their other social contacts. 'social influence' leads to the overall trust in a telecentre and its services and vice-versa. 'awareness' is the last level in the ism model. if citizens trust services, they are likely to tell other people to use those services. a personal account is vital in creating social influence about the telecentre and its services. hence, vles should put efforts at the local level to promote services, which would strengthen trust and generate confidence in people to use telecentre services. lastly, the ism model was reinforced by using the dematel analysis. in the dematel findings, five factors, namely 'centre resources', 'affordability', 'convenience', 'quality of operator', and 'quality of services', were established as influential factors that indicated strong driving power in telecentre sustainability. meanwhile, factors such as 'satisfaction', 'trust', australasian journal of information systems mishra & shah 2023, vol 27, research article prioritising sustainability factors 18 'awareness', 'social influence', and 'time-saving' are seen as influenced factors because they strongly rely on the influential factors to drive them. finally, 'media influence' is an autonomous factor that is relatively disconnected from the other factors, and it illustrates that 'social influence' as a factor influences mature telecentre sustainability more than 'media influence'. the following table 6 shows the factors obtained through ism and dematel. the factors marked ‘driving factors’ are essential for telecentres’ financial sustainability. ism dematel centre resources-level 8 (at the base of the model) driving factor affordability-level 7 driving factor quality of services and quality of operator-level 6 driving factors conveniencelevel 6 driving factor time saving-level 4 dependent factor satisfaction and media influence-level 3 satisfaction-dependent factor media influenceautonomous factor trust and social influence-level 2 dependent factor awareness-level 1 (at the top of the model) dependent factor table 6. comparison of relationships obtained using ism and dematel 9 study implications the research study has a novel contribution to the literature on telecentre sustainability, focusing on ppp-based e-government projects. one of the theoretical implications is that some factors established to be valid in literature have not been relevant in mature ppp-based telecentres in our study. second, a new factor, ‘awareness’, is not only about being aware of telecentre and its services but it also emphasises the separate roles of public and private entities. the study provides valuable insight into the complementarity roles of both public and private entities for financially sustaining telecentres. the study may be a benchmark for less mature telecentres. the study has methodological implications too. we employ two methods, ism and dematel, for this study, a combination which past researchers have not used much to solve complex problems involving ppp-based telecentre sustainability. this research has implications that are notable for practitioners too. first, ranking sustainability factors will help decision-makers formulate strategies to use their limited resources for improved telecentre sustainability. the study reveals that the initial focus of the telecentre entrepreneurs should be on centre resources. next, both the public and private entities should work together to make the services affordable to the citizens. affordability enhances telecentre use. in addition, affordability of services comes as the centres mature. ‘quality of services’ and ‘quality of operators’ must be maintained by ppp stakeholders. there needs to be constant engagement between the public and private entities to learn from each other and enhance service quality and the quality of operators. ‘convenience’ in terms of proactive service delivery, location, and telecentre operating time is crucial at the next level. the aforementioned five factors are critical because they are influencing factors. being influential, they drive other factors in the hierarchy: time saving, satisfaction, trust, social influence, and awareness. if the stakeholders utilise their limited resources for the five factors, ppp-based telecentres may have financial sustainability. australasian journal of information systems mishra & shah 2023, vol 27, research article prioritising sustainability factors 19 10 limitations and scope for future research a significant constraint of this study is the lack of focus on m-government services. in the view of emerging m-government initiatives, in future, a study can be undertaken to check how mgovernment services are flourishing in specific areas, which are the factors that influence them, and how telecentres complement m-government services and vice versa. further, study could be conducted on the effect of literacy on the adoption of telecentre services. to the best of our knowledge, because ism was applied in telecentre-related research for the first time, future studies based on this model in a different context or a new ism with innovative factors are likely to provide more clarity about its generic applicability. furthermore, future studies can statistically substantiate this model. for instance, structured equation modelling may be used to verify the already developed theoretical model. other approaches such as the analytical network process and fuzzy ism could be used for model validation. this study could also be used as a benchmark for assessing less-mature telecentres. further, the two applied methods (ism and dematel) can be used and reviewed by the experts to justify their use in future benchmarking studies. 11 conclusion telecentre-related studies have highlighted the complexity in the system on account of multiple stakeholders with diverse objectives and interdependencies in a ppp-based context. this study addresses the call for a robust and integrated framework by providing a simple yet comprehensive understanding of the telecentre model and processes by studying actual users and operators rather than the literature. the study included a large group of factors derived from the literature review. few contextual factors were added based on the results of the field study. the ism model representing the hierarchy of sustainability factors for mature ppp-based telecentres was developed based on the identified factors. the proposed model is likely to have universal applicability with contextual differences. on account of this, the interrelationships between factors in the derived framework may vary with different socioeconomic and cultural contexts. the study found that the factors ‘centre resources’, ‘affordability’, ‘quality of services’, ‘quality of operators’, and ‘convenience’ are the most important for mature telecentre sustainability because they are at the first five levels in the ism model and are influential factors in dematel. for a successful ppp-based mature centre, the private entity (vles) role is crucial for better centre resources. for affordability, both the public and private entities should work together. the private entity has to utilise centre resources to provide affordable services to citizens optimally. the public entity (the government) has to provide more e-government services for financial sustainability. for ‘quality of services’, ‘quality of operator’, and ‘convenience’, both private and public entities must create an environment to learn from each other. a public entity should conduct capacity-building exercises for vles and operators to keep them abreast of current e-government services and skills for using them. in addition, the vles should strive for efficient and proactive service delivery. thus, in ppp-based telecentres, both the public and private entities play a crucial role at each stage in the hierarchy of the ism model. australasian journal of information systems mishra & shah 2023, vol 27, research article prioritising sustainability factors 20 references alawadhi, s. (2019). a proposed model of trust factors for e-government adoption and civic engagement. proceedings of the 52nd hawaii international conference on system sciences. hawaii, usa: grand wailea, 3161 – 3170. https://hdl.handle.net/10125/59752. aledhari, m., aneja, n., bashir, a. k., bennett, r., bielby, j., hicks, h. a., … nighot, r. (2018). options and challenges in providing universal access. retrieved from internet initiative ieee: https://internetinitiative. ieee. org/images/files/resources/white_papers/options-and-challenges-inproviding-universal-access-v180418.pdf. almajali, d. a., maqableh, m., & moh’d taisir masa, r. (2015). assessing the digital divide status of the jordanian telecentre. international journal of communications, network and system sciences, 8(11), 428. https://doi.org/10.4236/ijcns.2015.811039 arias, e. (2019). how does media influence social norms? experimental evidence on the role of common knowledge. political science research and methods, 7(3), 561–578. https://doi.org/10.1017/psrm.2018.1 attwood, h., diga, k., braathen, e., & may, j. (2013). telecentre functionality in south africa: re-enabling the community ict access environment. the journal of community informatics, 9(4). http://ci-journal.net/index.php/ciej/article/view/970/1060. aundhe, m. das and narasimhan, r. (2016), public private partnership (ppp) outcomes in egovernment – a social capital explanation. international journal of public sector management, 29(7), 638-658. https://doi.org/10.1108/ijpsm-09-2015-0160. basílio, m. (2017). the degree of private participation in ppps: evidence from developing and emerging economies. in leitão, j., de morais sarmento, e. and aleluia, j. (eds.) the emerald handbook of public–private partnerships in developing and emerging economies (pp. 81-111). emerald publishing limited, bingley. https://doi.org/10.1108/978-1-78714-493420171003. bayliss, k., & van waeyenberge, e. (2018). unpacking the public private partnership revival. the journal of development studies, 54(4), 577-593. https://doi.org/10.1080/00220388.2017.1303671 brown, s. a., dennis, a. r., & venkatesh, v. (2010). predicting collaboration technology use: integrating technology adoption and collaboration research. journal of management information systems, 27(2), 9–54. https://doi.org/10.2753/mis0742-1222270201 carter, l., weerakkody, v., phillips, b., & dwivedi, y. k. (2016). citizen adoption of egovernment services: exploring citizen perceptions of online services in the united states and united kingdom. information systems management, 33(2), 124–140. https://doi.org/10.1080/10580530.2016.1155948. chander, mahesh, rathod, p., & balaraju, b. l. (2014). rural telecentres as innovation brokers in livestock innovation system in india: a review. indian research journal of extension education, 14(3), 14-23. australasian journal of information systems mishra & shah 2023, vol 27, research article prioritising sustainability factors 21 chander, muktesh, jain, s. k., & shankar, r. (2013). modeling of information security management parameters in indian organizations using ism and micmac approach. journal of modelling in management, 8 (2), 171-189. https://doi.org/10.1108/jm2-10-20110054. chen, y.-c., & gant, j. (2001). transforming e-government services: the use of application service providers in us local governments. amcis 2001 proceedings, 308. https://aisel.aisnet.org/amcis2001/308 chigona, w., & licker, p. (2008). using diffusion of innovations framework to explain communal computing facilities adoption among the urban poor. information technologies & international development, 4(3), 57-73. cruz, c. o., & marques, r. c. (2013). infrastructure public-private partnerships. springer berlin, heidelberg. https://doi.org/10.1007/978-3-642-36910-0. cuttaree, v., & mandri-perrott, x. c. (2011). public-private partnerships in europe and central asia: designing crisis-resilient strategies and bankable projects. world bank publications. de’, r. (2005). e-government systems in developing countries: stakeholders and conflict. in proceedings of international conference on electronic government, (26-37), springer, berlin, heidelberg. elliman, t. (2006). generating citizen trust in e-government using a trust verification agent: a research note. european and mediterranean conference on information systems (emcis). july 6-7, 2006, costa blanca, alicante, spain. http://bura.brunel.ac.uk/handle/2438/8273. emilio, b. i. (2015). public-private partnerships in e-government guide. ukraine: public private partnership development program-p3dp. faroqi, m. g. (2019). the role of telecentre in developing entrepreneurship: a case study on union digital centres in bangladesh. proceedings of the 12th international conference on theory and practice of electronic governance, april 2019, 446–456. https://doi.org/10.1145/3326365.3326424 faroqi, m. g., & siddiquee, n. a. (2017). impacts of telecentre on users: the experience of the union digital centre in bangladesh. proceedings of the 10th international conference on theory and practice of electronic governance, march 2017, 149–156. https://doi.org/10.1145/3047273.3047346 faroqi, m. g., siddiquee, n. a., & ullah, s. (2019). sustainability of telecentres in developing countries: lessons from union digital centre in bangladesh. telematics and informatics, 37, 113–127. https://doi.org/10.1016/j.tele.2018.05.006. foko, t. (2018). developing a framework for sustainable information and communication technology platforms for resource scarce rural communities. the journal of community informatics, 14(2–3). https://doi.org/10.15353/joci.v14i2-3.3415. fuchs, r. p. (1998). if you have a lemon, make lemonade: a guide to the start-up of the african multipurpose community telecentre pilot projects. https://idl-bncidrc.dspacedirect.org/bitstream/handle/10625/14740/107738.pdf?sequence=6 australasian journal of information systems mishra & shah 2023, vol 27, research article prioritising sustainability factors 22 furuholt, b., & sæbø, ø. (2018). the role telecentres play in providing e-government services in rural areas: a longitudinal study of internet access and e-government services in tanzania. the electronic journal of information systems in developing countries, 84(1), e12006. https://doi.org/10.1002/isd2.12006. fusch, p. i., & ness, l. r. (2015). are we there yet? data saturation in qualitative research. the qualitative report, 20(9), 1408. gabus, a., & fontela, e. (1972). world problems, an invitation to further thought within the framework of dematel (pp. 1–8). battelle geneva research center, geneva, switzerland, gilbert, d., balestrini, p., & littleboy, d. (2004). barriers and benefits in the adoption of egovernment. international journal of public sector management, 17(4), 286–301. https://doi.org/10.1108/09513550410539794 gollakota, k., & pick, j. b. (2020). a resistance to change perspective on nonusers of telecenters. the electronic journal of information systems in developing countries, 86(3), e12127. https://doi.org/10.1002/isd2.12127 gollakota, k., pick, j. b., & sathyapriya, p. (2012). using technology to alleviate poverty: use and acceptance of telecenters in rural india. information technology for development, 18(3), 185–208. https://doi.org/10.1080/02681102.2011.643195 gupta, k.p., bhaskar, p. and singh, s. (2017). prioritization of factors influencing employee adoption of e-government using the analytic hierarchy process. journal of systems and information technology, 19 (1/2), 116-137. https://doi.org/10.1108/jsit-04-2017-0028. haddad, s. r., & oliveira, a. (2019). telecenters in brazil: the need for survival despite of the evolution of cell phones. 7799–7808. 10.21125/inted.2019 hawthorne, r. w., & sage, a. p. (1975). on applications of interpretive structural modeling to higher education program planning. socio-economic planning sciences, 9(1), 31–43. https://doi.org/10.1016/0038-0121(75)90039-7 heeks, r. (2003). most egovernment-for-development projects fail: how can risks be reduced? (igovernment working paper no. 14). https://ssrn.com/abstract=3540052. hellström, j. (2008). mobile phones for good governance–challenges and way forward. stockholm university/upgraid. https://www.w3.org/2008/10/mw4d_ws/papers/hellstrom_gov.pdf. hill, w. c. (2016). identifying socio-cultural determinants to access: implications for e-governance in the water & sanitation sector (master's thesis, university of cape town). https://open.uct.ac.za/bitstream/handle/11427/20947/thesis_com_2016_hill_wesley_clift on.pdf?sequence=1&isallowed=y hodge, g. a., & greve, c. (2017). on public–private partnership performance: a contemporary review. public works management & policy, 22(1), 55-78. https://doi.org/10.1177/1087724x16657830. hsieh, h.-f., & shannon, s. e. (2005). three approaches to qualitative content analysis. qualitative health research, 15(9), 1277–1288. https://doi.org/10.1177/1049732305276687. australasian journal of information systems mishra & shah 2023, vol 27, research article prioritising sustainability factors 23 hudson, h. (2001). telecentre evaluation: issues and strategies. in c. latchem and d. walker, (eds.), telecentres: case studies and key issues (pp. 169-181). commonwealth of learning, vancouver, canada. india cellular and electronics association. (2020). contribution of smart phones to digital governance in india. retrieved from https://icea.org.in/blog/wpcontent/uploads/2022/06/icea-digital-governance-in-india-report-2020.pdf iyer, l., & rao, s. (2017). e-governance implementation in the telecentres of karnataka, india: costs, governance & benefits. asian journal of research in social sciences and humanities, 7(5), 377–385. jaganathan, m., ahmad, s., ishak, k. a., nafi, s. n. m., & uthamaputhran, l. (2018). determinants for ict adoption and problems: evidence from rural based small and medium enterprises in malaysia. international journal of entrepreneurship, 22(4), 1-13. janssen, m., rana, n. p., slade, e. l., & dwivedi, y. k. (2018). trustworthiness of digital government services: deriving a comprehensive theory through interpretive structural modelling. public management review, 20(5), 647–671. https://doi.org/10.1080/14719037.2017.1305689. jauhari, s., & maheshwari, d. (2014). m governance: challenges and prospects. international journal of innovative research and development, 3(12). jedlicka a, mayer r (1980). interpretive structural modeling cross cultural uses. ieee transactions on systems, man, and cybernetics, 10(1), 49–51. joshi, d. b. (2018). e-governance in india: the transformation for sustainable development. professional panorama: an international journal of management & technology, vol. v (i), janjune, 20-36. accessed from https://www.researchgate.net/profile/parag-shukla/publication/ 344664746_reviewing_the_dynamics_of_corporate_social_responsibility_interventio ns_by_women_empowerment/links/5f87e181299bf1b53e28e2e0/reviewing-thedynamics-of-corporate-social-responsibility-interventions-by-womenempowerment.pdf#page=27, retrieved on 24/10/2022. kante, m., oboko, r., & chepken, c. (2019). an ict model for increased adoption of farm input information in developing countries: a case in sikasso, mali. information processing in agriculture, 6(1), 26–46. https://doi.org/10.1016/j.inpa.2018.09.002 kapondera, s. k., & hart, g. (2016). the use of multipurpose community telecentres and their services in malawi: the case of lupaso community telecentre. south african journal of libraries and information science, 82(1), 13–25. kapondera, s. k., & namusanya, d. m. (2017). uses, benefits and challenges of using rural community telecentres as tools for development: the case of vikwa community telecentre in kasungu, malawi. journal of development and communication studies, 5(1), 1–21. https://doi.org/10.4314/jdcs.v5i1.1 kiran, g. r. (2014). multipurpose nature of telecentres: the case of e-governance service delivery in akshaya telecentres project (doctoral dissertation). the london school of economics and political science (lse). http://etheses.lse.ac.uk/935/1/kiran_multipupose-nature-oftelecentres.pdf australasian journal of information systems mishra & shah 2023, vol 27, research article prioritising sustainability factors 24 knopf, j. w. (2006). doing a literature review. ps: political science and politics, 39(1), 127–132. https://doi.org/10.1017/s1049096506060264 kumar, r., & best, m. l. (2006). impact and sustainability of e-government services in developing countries: lessons learned from tamil nadu, india. the information society, 22(1), 1–12. https://doi.org/10.1080/01972240500388149 laksmanan, l. (2008). public-private partnership in indian infrastructure development: issues and options. reserve bank of india occasional paper, 29(1), 37-76. lal, r., & haleem, a. (2009). a structural modelling for e-governance service delivery in rural india. international journal of electronic governance, 2(1), 3–21. https://doi.org/10.1504/ijeg.2009.024962 liu, l., wang, t., xie, l., & zhan, x. (2020). influencing factors analysis on land-lost farmers’ happiness based on the rough dematel method. discrete dynamics in nature and society, 2020. https://doi.org/10.1155/2020/6439476. lo, m. c., songan, p., cheuk, s., atang, a., & yeo, a. w. (2013). communities’ attitudes towards telecentre and its impact on rural tourism. international journal of innovation, management and technology, 4(6), 534-530. luo, m. m., & chea, s. (2018). internet village motoman project in rural cambodia: bridging the digital divide. information technology & people, 31(1), 2-20. https://doi.org/10.1108/itp-07-2016-0157 madon, s. (2005). governance lessons from the experience of telecentres in kerala. european journal of information systems, 14(4), 401–416. https://doi.org/10.1057/palgrave.ejis.3000576 madon, s. (2010). telecentres and development: a social space approach. information systems and innovation group london school of economics and political science working paper series, 44(0), 0–27. mandal, a., & deshmukh, s. g. (1994). vendor selection using interpretive structural modelling (ism). international journal of operations & production management, 6, 52-59. https://doi.org/10.1108/01443579410062086 masiero, s. (2011). financial vs social sustainability of telecentres: mutual exclusion or mutual reinforcement? the electronic journal of information systems in developing countries, 45(1), 1–23. https://doi.org/10.1002/j.1681-4835.2011.tb00319.x. mayanja, m. (2006). rethinking telecentre sustainability approaches. the journal of community informatics, 2(3). https://doi.org/10.15353/joci.v2i3.2073 mbangala, b., & samzugi, a. (2014). the role of telecentres in tanzania’s rural development. a case study of sengerema district council, mwanza region. library philosophy and practice (e-journal). 1224. http://digitalcommons.unl.edu/libphilprac/1224 mishra, g. (2018). understanding the role of rural entrepreneurs in telecentre sustainability: a comparative study of the akshaya and eseva projects in india. technology innovation management review, 8(1), 16-23. mkude, c.g., wimmer, m.a. (2015). e-government systems design and implementation in developed and developing countries: results from a qualitative analysis. in australasian journal of information systems mishra & shah 2023, vol 27, research article prioritising sustainability factors 25 international conference on electronic government. egov 2015. lecture notes in computer science, vol 9248. springer, cham. https://doi.org/10.1007/978-3-319-22479-4_4. motiani, p. (2019, december 31). install maadhaar app in your phone and never carry a physical aadhaar card. https://economictimes.indiatimes.com/wealth/save/install-maadhaar-appin-your-phone-and-never-carry-a-physical-aadhaar-card/articleshow/73042617.cms naik, g., chitre, c., bhalla, m., & rajan, j. (2016). can technology overcome social disadvantage of school children’s learning outcomes? evidence from a large-scale experiment in india. evidence from a large-scale experiment in india (may 5, 2016). iim bangalore research paper, (512). nasim, s. (2011). total interpretive structural modeling of continuity and change forces in egovernment. journal of enterprise transformation, 1(2), 147–168. https://doi.org/10.1080/19488289.2011.579229. ndiku, d. m. t., & mwai, n. w. (2016). equalizing utilization of information among rural schools in kenya through satellite technology. scholars journal of arts, humanities and social sciences, 3(9b), 1476-1484. oestmann, s., & dymond, a. c. (2001). telecentres—experiences, lessons and trends. telecentres: case studies and key issues, 1, 1-15. oecd. (2012). recommendation of the council on principles for public governance of public-private partnerships. paris: organization for economic cooperation & development (oecd). omogor, m. (2013). channels of information acquisition and dissemination among rural dwellers. international journal of library and information science, 5(10), 306–312. https://doi.org/10.5897/ijlis11.036. osei-kyei, r., & chan, a. p. c. (2015). review of studies on the critical success factors for public–private partnership (ppp) projects from 1990 to 2013. international journal of project management, 33(6), 1335–1346. https://doi.org/10.1016/j.ijproman.2015.02.008 palaco, i., park, m. j., kim, s. k., & rho, j. j. (2019). public–private partnerships for egovernment in developing countries: an early-stage assessment framework. evaluation and program planning, 72, 205–218. https://doi.org/10.1016/j.evalprogplan.2018.10.015 rashid, a. t. (2017). inclusive capitalism and development: case studies of telecenters fostering inclusion through icts in bangladesh. information technologies & international development, 13, 14. rhodes, j. (2004). south african, rural ict implementation: a critical retrospective application of latour’s due process model. australasian journal of information systems, 11(2). https://doi.org/10.3127/ajis.v11i2.115 rogers, e. m. (2004). a prospective and retrospective look at the diffusion model. journal of health communication, 9(s1), 13–19. https://doi.org/10.1080/10810730490271449 roman, r. (2003). diffusion of innovations as a theoretical framework for telecenters. information technologies & international development, 1(2), 53. shadrach, b., & sharma, s. (2011). telecentre sustainability. in proceedings of international conference on misnomers, challenges, and opportunities, 3, 1-6. https://www.academia.edu/1046862/telecentre_sustainability. australasian journal of information systems mishra & shah 2023, vol 27, research article prioritising sustainability factors 26 sage, a.p. (1977). interpretive structural modelling: methodology for large scale systems (pp. 91164). mcgraw-hill, new york, ny. samantra, c., datta, s., mahapatra, s. s., & debata, b. r. (2016). interpretive structural modelling of critical risk factors in software engineering project. benchmarking: an international journal, 23(1), 2-24. https://doi.org/10.1108/bij-07-2013-0071. samuel, m., doctor, g., christian, p., & baradi, m. (2020). drivers and barriers to e-government adoption in indian cities. journal of urban management, 9(4), 408–417. https://doi.org/10.1016/j.jum.2020.05.002 saxena, j. p., & vrat, p. (1992). scenario building: a critical study of energy conservation in the indian cement industry. technological forecasting and social change, 41(2), 121–146. https://doi.org/10.1016/0040-1625(92)90059-3. schaupp, l. c., carter, l., & mcbride, m. e. (2010). e-file adoption: a study of us taxpayers’ intentions. computers in human behavior, 26(4), 636-644. https://doi.org/10.1016/j.chb.2009.12.017 sealy, w. u. (2003). empowering development through e-governance: creating smart communities in small island states. the international information & library review, 35(2– 4), 335–358. https://doi.org/10.1016/s1057-2317(03)00020-1 sharma, h. d., & gupta, a. d. (1995). the objectives of waste management in india: a futures inquiry. technological forecasting and social change, 48(3), 285–309. https://doi.org/10.1016/0040-1625(94)00066-6 sharma, r., & ray, s. (2019). exploring the impact of mobile technology adoption on financial sustainability of telecentres: the case of india. proceedings of the 12th international conference on theory and practice of electronic governance, 427–434. https://doi.org/10.1145/3326365.3326421. thapa, d., & sæbø, ø. (2014). exploring the link between ict and development in the context of developing countries: a literature review. the electronic journal of information systems in developing countries, 64(1), 1–15. https://doi.org/10.1002/j.1681-4835.2014.tb00454.x valmohammadi, c., & dashti, s. (2016). using interpretive structural modeling and fuzzy analytical process to identify and prioritize the interactive barriers of e-commerce implementation. information & management, 53(2), 157–168. https://doi.org/10.1016/j.im.2015.09.006 venkatesh, v., morris, m. g., davis, g. b., & davis, f. d. (2003). user acceptance of information technology: toward a unified view. mis quarterly, 425–478. https://doi.org/10.2307/30036540 verdegem, p., & verleye, g. (2009). user-centered e-government in practice: a comprehensive model for measuring user satisfaction. government information quarterly, 26(3), 487–497. https://doi.org/10.1016/j.giq.2009.03.005 verkijika, s. f., & de wet, l. (2018). a usability assessment of e-government websites in subsaharan africa. international journal of information management, 39, 20–29. https://doi.org/10.1016/j.ijinfomgt.2017.11.003 australasian journal of information systems mishra & shah 2023, vol 27, research article prioritising sustainability factors 27 wang, h., xiong, w., wu, g., & zhu, d. (2018). public–private partnership in public administration discipline: a literature review. public management review, 20(2), 293-316. https://doi.org/10.1080/14719037.2017.1313445. warfield, j. n. (1974). toward interpretation of complex structural models. ieee transactions on systems, man, and cybernetics, (5), 405–417. https://doi.org/10.1109/tsmc.1974.4309336 warfield, j. n. (1978). societal systems planning, policy and complexity. cybernetics and system, 8(1), 113–115. https://doi.org/10.1080/01969727808927573 webster, j., & watson, r. t. (2002). analyzing the past to prepare for the future: writing a literature review. mis quarterly, 26 (2), xiii–xxiii. weerakkody, v., el-haddadeh, r., sivarajah, u., omar, a., & molnar, a. (2019). a case analysis of e-government service delivery through a service chain dimension. international journal of information management, 47, 233–238. https://doi.org/10.1016/j.ijinfomgt.2018.11.001 world bank. (2018). procuring infrastructure public-private partnerships report 2018: assessing government capability to prepare, procure, and manage ppps. retrieved from https://bpp.worldbank.org/economy/ind?survey=ppp zhou, d., zhang, l., & li, h. (2006). a study of the system’s hierarchical structure through integration of dematel and ism. in 2006 international conference on machine learning and cybernetics, 1449–1453. ieee. 10.1109/icmlc.2006.258757. appendices appendix 1. interview protocol interview guide for the users q1. gender: ………………. age: ………………. education……………………….profession………………………….religion……. caste………………………. income (rs/month)………………… disability (if any) ……………distance from akshaya centre (km)………………… q2. have you heard of e-government services (like e-payment, e-filing, etc.) at akshaya centre: y/n q3. whether you go to akshaya service centre y/n q4: how often do you normally visit akshaya centre: daily/twice a week/weekly/monthly/quarterly /other…………. q5: how much time do you spend there: 0-20 min/20-40 min/40-1 hr/more than 1 hr. q6: why do you go to the akshaya service centre (you can choose more than one) a. to get information about government b. to get information on government schemes c. avail education service provided by the government d. to get information on health issues australasian journal of information systems mishra & shah 2023, vol 27, research article prioritising sustainability factors 28 e. to get information on agriculture f. getting contact details of government officials g. birth/death certificate h. for land registration i. to complain against government officials/delay in work etc. j. pay tax k. pay utility bills l. other……………………………………… q7: which of the following you have in your home (you can choose more than one): a. television b. radio c. video player d. landline e. mobile phone f. computer g. internet q8. how did you hear about akshaya services? q9. are you aware of akshaya service centres, and can you receive government service from these centres? if yes, how did you come to know about it? q10. if you are not aware of the e-government service centre, will you use them if you know about services available and why? q11. what is the purpose for which you use the akshaya service centres? q12. will you continue to use the services, and if not, why? q13. what reasons make you use akshaya service centres instead of a traditional government department? q14. do you use the services due to your need? has someone asked you to use the services? q15. do you think that the services are available to all the sections of the society in your locality? q16. are there any problems you have come across while using these centres concerning: a. services available b. time taken c. mediator’s response d. accessibility e. others……………………………………………………. q17. with the introduction of government service at akshaya service centre, what change have you observed in the way you: a. communicate b. gather information c. organise meetings d. use for welfare e. education f. health australasian journal of information systems mishra & shah 2023, vol 27, research article prioritising sustainability factors 29 g. agriculture h. others ………………………………………… q18. what is your opinion on akshaya service centres, i.e. do you feel they are useful or not? should the government not invest in these projects and focus on other development problems? q19. do you think e-government services have given you the opportunity for development, and if yes do you use them and reasons for using or not using them: • skills and training: personal development, livelihood generation etc. or any other skill enhancement programmes. • education: has akshaya service centres helped in increasing access to education • health and safety programmes: have you accessed health and safety-related programmes. • basic amenities: has akshaya service centres fulfilled the need for basic amenities like food, housing, land etc. • participation in decision making • participation in project implementation • participation in policy-making at the local level • representation • making government accountable • making your voice heard by higher authority • employment: has akshaya service centres given any employment opportunities, information related to employment • wages: has akshaya service centres helped in increasing wages, information on better-earning opportunities • credit: has the e-government given access to the bank for credit needs. • market: has e-government given access to better market interview guide for nonusers q1. gender: ………………. age: ………………. education……………………….profession………………………….religion……. caste………………………. income (rs/month)………………… disability (if any) ……………distance from akshaya centre (km)………………… q2. have you heard of e-government services (like e-payment, e-filing, etc.) at akshaya centre? if yes, what do you know about it? q3: where do you go and pay utility bills? q4: do you have trust in the centres? q5: how much far is your home from the nearest centres? where are the government departments for paying bills or getting information? q6: what are the reasons for the non-use of the centre if you are aware of akshaya centres? other questions were asked based on the responses given. interview guide for entrepreneurs q1. gender: ………………. age: ………………. education……………………….religion……………………………. australasian journal of information systems mishra & shah 2023, vol 27, research article prioritising sustainability factors 30 caste………………………. income………………… q2. name of the centre……………………………….place…………………………. q3. date of opening…………………………………. q4. why did you open the akshaya centre? q5. which services does your centre offer to the people (select the appropriate) • e-literacy • e-payment • ignou courses • antegramam • malayalam computing • e-krishi • others…………………………………. (please mention) q6. does your husband/father/mother/wife/ help in running the centre? q7. are you satisfied with the way your centre is working and delivering services? please mention reasons. q8. approximately how many households does your centre provide services to? q9. how have you created awareness about the services at your centre to the people? how do you create trust in people to use akshaya services? q10. are you satisfied by how the government is extending support to you for services? give reasons. q11. what role does government play in running of akshaya centre? do you think the success of the akshaya centre depends on the government? q12. how much trust do you have in government for the support of services? q13. what are the main reasons you were selected to open the akshaya centre? which things made you stand out from other applicants (maybe your qualification, age, experience, infrastructure, political contact etc.)? q14. do you run another business (like computer centre etc.) along with akshaya centre? if yes, then who takes care of your second business? which business gives you more income? q15. is your centre providing sufficient income to run the centre? have you recovered the amount you invested? please give the amount you invested and the average monthly income through e-centre? q16. please tell problems you are facing concerning services, infrastructure, running etc. of your centre? q17. what are the things you need or expect from the government? q18. can you tell your concern/problems to government authorities about your centre and services? how do they receive your requisition, complaint etc.? q19. how many people visit your centre on a day? how are the citizens reacting towards your e-centre? australasian journal of information systems mishra & shah 2023, vol 27, research article prioritising sustainability factors 31 q20. are you able to provide all the services which a citizen asks you? if no, which services you are not able to provide? q21. which services are used maximum by the people, and why do you think they are used the most? q22. can you specify which category of people is using most of the services? (like high income, poor people, women, children, disabled, caste, religion etc.). why some people are using, and others are not using? q23. what changes has akshaya’s e-payment, e-literacy, e-filing services brought in the life of people? q22. do you think akshaya centre is benefitting people? give reasons, examples? q24. do you want to tell anything else about your experience with centre usage? interview guide for project officials 1. what is your view on e-government services? how do you perceive them making an impact in society? 2. what are the policies government implementing to make these services reach the most disadvantaged sections of society? 3. what specific problems did you see in using these services in villages? 4. what is your opinion on using other delivery channels (like mobile, call centre etc.) for e-government services? what opportunities or issues do you see in using them? 5. do you think e-government has served the purpose? 6. how government plans the service to be delivered? 7. in your opinion, has e-government tackled the issue of social inequality? the services concerning empowerment are handled in which way. how much support does the government give to these services, i.e. are people given a chance to participate in decision making, do their complaints get cleared in optimum time etc.? appendix 2. factors used for the study after field verification factors example quotes of respondents total references to the overarching theme remarks implied meaning in the contextual setting affordability “in cities, the cost of doing medical transcription course is rs. 35,000, but at this centre, it is only rs 17,000 plus some tax, but it is far less than what i have to pay in the city.” [kt16] 97 variable retained from literature and validated in the field. cost of services, money saved in travel and food, no loss of daily wage awareness “i am aware of the akshaya project because, in the beginning, mammooty promoted the project, but at present, i do 39 this is a new variable observed linking services of telecentres to government through efforts of entrepreneurs, australasian journal of information systems mishra & shah 2023, vol 27, research article prioritising sustainability factors 32 not know whether the project is running or not.” [kn22] “the government supports akshaya centres as the government has advertised about it and the services like tax-filing in newspapers” [nd19] in the field study. advertisement by the government convenience “as a woman, it is difficult for me to go to the government departments to pay the bills. there are different types of government departments and have many counters. it is confusing; hence i find it difficult to go there. when the akshaya centre was available, i started paying here. it is effortless to use; i do not feel any fear in using the centre.” [kt17] “it is quick and simple to get work done at the akshaya centres. if i have any doubt, i can ask them. in government offices, they do not care about our requirements. they only do their duty like a robot.” [kn8] 73 the reliability factor from the literature review is subsumed in this factor as convenienc e is due to the reliability and simplicity of services. simple to use telecentre, ease of access due to location and working hours, availability of necessary services time-saving “the government office takes more time in delivering services, and they do not give satisfactory services. i have to stand in a queue, but it takes only 5 minutes at the akshaya centre, and they provide a good service. because of this, i have an interest to come here again.” [kk24] 80 variable retained from literature, validated in the field. time-saving while accessing services near home time saved while accessing services quality of operator “the operator is not very friendly. therefore, i do not prefer to use this centre. i look first at the behaviour of a person; if it is good, then i continue a relationship with him.” [kn4]. “the government officers go out anywhere without informing anyone about their absence. we cannot ask them any questions. they are not friendly; ‘akshaya’ centre is different and very friendly.” [kn21] 66 variable retained from literature and validated in the field. competency of operator/vle to use ict, the entrepreneurial ability of operator, i.e. how they create awareness, receive and serve customer, and manage and maintain required resources to run centre, promote customer-oriented services, unbiased nature of the operator quality of services “the government office takes more time to deliver services, and they do not give satisfactory services. i have to stand in a queue, but it takes only 5 minutes at the akshaya centre, and they provide a good service. because of this, i have an interest to come here again.” [kk24] 73 in literature, not much is available on this about the use of telecentre. however, this variable overall effectiveness and efficiency with which services are delivered through telecentres. australasian journal of information systems mishra & shah 2023, vol 27, research article prioritising sustainability factors 33 emerged prominentl y in the field. social influence “my friend told me that he pays electricity bill in akshaya centre. i enquired more about it and then started using.” [kk9] “i read about the akshaya project in the newspaper. however, when mr gopal (the vle) visited us and told us about the services, i started using the centre.” [kn7] 84 retained from literature and validated in the field. visit by entrepreneurs, the influence of friends or relatives in using services media influence “the government advertised about the akshaya centres in the newspapers and television. since then, i have been aware of the centre and its services.” [kk26] 45 retained from literature and validated in the field. people are influenced by media (newspaper, tv advertisement, radio broadcast, print materials by vles) in using the services. centre resources “my place akshaya centre does not have a good internet connection. i am not able to download government job applications. much time is wasted because of slow speed.” [kt18] 93 retained from literature and validated in the field. availability of quality internet connection, apt number of staff services, technical equipment, sitting space (for a minimum of 5-7 people), and necessary software satisfaction “i am happy and satisfied with the services and the centre as there is more interaction with the centre staff. i come for interaction in the centre. people do not answer a query in a satisfied manner.” [kk21] 89 retained from literature and validated in the field. overall satisfaction with access, customer service, services delivery trust “i use the government office to pay bills. the akshaya centre does not put any seal, but they put a seal in the government office. i have been paying in the government office for ages, and i do not want to change the way now. akshaya centre cannot tell me how to fill an application form, but the government officials can.” [kk19] “when i go and pay in akshaya centre, then they will collect all bills and pay in the government office. therefore, it will be late. it is better to pay in the government office directly.” [amr146] 47 variable retained from literature and validated in the field. trust in using services, perceived risk associated in using the services and trust in the intermediary, i.e. vle or operator, giving the services. australasian journal of information systems mishra & shah 2023, vol 27, research article prioritising sustainability factors 34 digital divide (10), complexity and compatibilit y (12), relative advantage (16), observabilit y (8) and trialability (10) the factors were not evident in the field as the number of references to these codes was below the specified limit of 30 references to the overarching theme. appendix 3. initial reachability matrix appendix 4. final reachability matrix note 1* denotes transitivity relationship appendix 5: level partitioning iteration 1 factors reachability set antecedent set intersection set level 1 1, 5, 6, 7, 10, 11 1,9 1 2 2 2, 3, 6, 7,8, 9, 10, 11 2 i factors 1 2 3 4 5 6 7 8 9 10 11 1 1 0 0 0 0 0.75 0 0 0 0.75 0 2 0 1 0 0 0 0 0 0 0 0 0 3 0.5 0.75 1 1 0 0.50 0 0 0.25 1 0.50 4 0 0 0 1 0 0.50 0 0 0 1 0.75 5 0 0.25 0 1 1 1 0 0 0 1 1 6 0 0 0 0 1 1 75 0 0 1 1 7 0 1 0 0 0 0 1 0 0 0 0.75 8 0 1 0 0 0 0 0.50 1 0 0 0.75 9 0.75 0 0.75 0.75 0.75 1 0.25 0 1 1 1 10 0 0 0 0 0 0 1 0 0 1 1 11 0 0 0 0 0 0 1 0 0 0 1 factors 1 2 3 4 5 6 7 8 9 10 11 driving power 1 1 0 0 0 1* 1 1* 0 0 1 1* 6 2 0 1 0 0 0 0 0 0 0 0 0 1 3 0 1 1 1 0 0 1* 0 0 1 1* 6 4 0 0 0 1 0 0 1* 0 0 1 1 4 5 0 0 0 1 1 1 1* 0 0 1 1 6 6 0 1* 0 1* 1 1 1 0 0 1 1 7 7 0 1 0 0 0 0 1 0 0 0 1 3 8 0 1 0 0 0 0 1* 1 0 0 1 4 9 1 1* 1 1 1 1 1* 0 1 1 1 10 10 0 1* 0 0 0 0 1 0 0 1 1 4 11 0 1* 0 0 0 0 1 0 0 0 1 3 dependence power 2 8 2 5 4 4 10 1 1 7 10 australasian journal of information systems mishra & shah 2023, vol 27, research article prioritising sustainability factors 35 3 2, 3, 4, 7, 10, 11 3, 9 3 4 4, 7, 10, 11 3, 4, 5, 6, 9 4 5 4, 5, 6, 7, 10, 11 1, 5, 6, 9 5, 6 6 2, 4, 5, 6, 7, 10, 11 1, 5, 6, 9 5, 6 7 2, 7, 11 1, 4, 5, 6, 7, 8, 9, 10, 11 7, 11 8 2, 7, 8, 11 8 8 9 1, 2, 3, 4, 5, 6, 7, 9, 10, 11 9 9 10 2, 7, 10, 11 1, 3, 4, 5, 6, 9, 10 10 11 2, 7, 11 1, 3, 4, 5, 6, 7, 8, 9, 10, 11 7, 11 iteration 2 factors reachability set antecedent set intersection set level 1 1, 5, 6, 7, 10, 11 1,9 1 3 2, 3, 4, 7, 10, 11 3, 9 3 4 4, 7, 10, 11 3, 4, 5, 6, 9 4 5 4, 5, 6, 7, 10, 11 1, 5, 6, 9 5, 6 6 4, 5, 6, 7, 10, 11 1, 5, 6, 9 5, 6 7 7, 11 1, 4, 5, 6, 7, 8, 9, 10, 11 7, 11 ii 8 7, 8, 11 8 8 9 1, 3, 4, 5, 6, 7, 9, 10, 11 9 9 10 7, 10, 11 1, 3, 4, 5, 6, 9, 10 10 11 7, 11 1, 3, 4, 5, 6, 7, 8, 9, 10, 11 7, 11 ii iteration 3 factors reachability set antecedent set intersection set level 1 1, 5, 6, 7, 10, 11 1, 9 1 3 3, 4, 10 3, 9 3 4 4, 10 3, 4, 5, 6, 9 4 5 4, 5, 6, 10 1, 5, 6, 9 5, 6 6 4, 5, 6, 10 1, 5, 6, 9 5, 6 8 8 8 8 iii 9 1, 3, 4, 5, 6, 9, 10 9 9 10 10 1, 3, 4, 5, 6, 9, 10 10 iii iteration 4 factors reachability set antecedent set intersection set level 1 1, 5, 6, 7, 10, 11 1, 9 1 3 3, 4 3, 9 3 4 4 3, 4, 5, 6, 9 4 iv 5 4, 5, 6 1, 5, 6, 9 5, 6 6 4, 5, 6 1, 5, 6, 9 5, 6 9 1, 3, 4, 5, 6, 9 9 9 iteration 5 factors reachability set antecedent set intersection set level 1 1, 5, 6, 7, 10, 11 1,9 1 3 3 3, 9 3 v 5 5, 6 1, 5, 6, 9 5, 6 6 5, 6 1, 5, 6, 9 5, 6 9 1, 3, 5, 6, 9 9 9 iteration 6 australasian journal of information systems mishra & shah 2023, vol 27, research article prioritising sustainability factors 36 factors reachability set antecedent set intersection set level 1 1, 5, 6, 7, 10, 11 1, 9 1 5 5, 6 1, 5, 6, 9 5, 6 vi 6 5, 6 1, 5, 6, 9 5, 6 vi 9 1, 5, 6, 9 9 9 iteration 7 factors reachability set antecedent set intersection set level 1 1, 5, 6, 7, 10, 11 1, 9 1 vii 9 1, 5 9 9 iteration 8 factors reachability set antecedent set intersection set level 1 1, 5, 6, 7, 10, 11 1, 9 1 viii appendix 6. factors and levels sr. no. factor name factor code levels 1 awareness 2 i 2 social influence 7 ii 3 trust 11 ii 4 media influence 8 iii 5 satisfaction 10 iii 6 time saving 4 iv 7 convenience 3 v 8 quality of operator 5 vi 9 quality of services 6 vi 10 affordability 1 vii 11 centre resources 9 viii copyright: © 2023 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v27i0.3135 http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 public–private partnership (ppp) 3 ppp in telecentre-based e-government projects 4 telecentre sustainability 5 focus of the study 6 methodology 6.1 literature review 6.2 field verification of sustainability factors 6.2.1 selection of telecentres and respondents 6.3 model development using interpretive structural modelling 6.4 ism model validation using dematel 6.5 analysis of data 7 results 7.1 literature review: factors influencing telecentre’s financial sustainability 7.2 field verification of factors 7.3 ism results 7.3.1 structural self-interaction matrix (ssim) 7.3.2 reachability matrix and transitivity check 7.3.3 final reachability matrix 7.3.4 level partitioning 7.4 dematel results 8 discussion 9 study implications 10 limitations and scope for future research 11 conclusion paper title (paper title style) australasian journal of information systems ackerley et al. 2022, vol 26, research article errors, irregularities, and misdirection 1 errors, irregularities, and misdirection: cue utilisation and cognitive reflection in the diagnosis of phishing emails mitchell ackerley charles sturt university, bathurst, australia ben w. morrison macquarie university, sydney, australia ben.morrison@mq.edu.au kate ingrey charles sturt university, bathurst, australia mark w. wiggins macquarie university, sydney, australia piers bayl-smith macquarie university, sydney, australia natalie m. v. morrison western sydney university, blacktown, australia abstract the study aimed to examine the role of, and potential interplay between, cue utilisation and cognitive reflection in email users’ ability to accurately (and efficiently) differentiate between phishing and genuine emails. 145 participants completed the cognitive reflection test (crt), a phishing diagnostic task, and the expert intensive skill evaluation (expertise 2.0) battery, which provided a gauge of users’ cue utilisation in the domain. the results revealed an interaction between users’ cognitive utilisation and cue reflection, whereby users low in both facets performed significantly worse in diagnosing phishing emails than all other groups. further, those participants with both higher cue utilisation and cognitive reflection took significantly longer to make their diagnosis. it is concluded that a high level of cognitive reflection was able to compensate for a lower level of cue utilisation, and vice versa. participants reported using seven types of cue during diagnosis, however, there was no significant relationship between the types of cues used and users’ level of cue utilisation. taken together, the findings have implications to the design of user-level interventions in relation to the identification of vulnerable users, as well as the need to consider training approaches that extend beyond the use of simple cue inventories. keywords: cyber security, phishing, cue utilisation, cognitive reflection, expertise. 1 introduction within a technologically advancing world, organisations have become increasingly reliant on information technology (it) to facilitate business services. however, this reliance is met with certain risks to an organisation’s cyber security. the australian government suggests that cyber-crime results in losses of one billion dollars per year for australian organisations (department of prime minister and cabinet, 2019). the most common means of accessing australasian journal of information systems ackerley et al. 2022, vol 26, research article errors, irregularities, and misdirection 2 personal information by cyber criminals in australia is via phishing emails (acsc, 2021; moore, 2021). phishing emails are socially engineered messages that are designed to appear legitimate communications in order to lure email users to ‘bite’ or engage with the email (mayhorn & nyeste, 2012). engaging with these emails (e.g., clicking on links) invariably leads to breaches of sensitive information, malware infections, and in some cases the forceful request of direct payment (abawajy, 2012; fette et al., 2007). consequently, understanding the factors that underlie users’ misdiagnosis of phishing emails as trustworthy is a critical line of enquiry. while system design is undoubtedly vital to countering phishing scams, user-level differences play a significant role (downs et al.; fette et al., 2007; jones et al., 2015). email users’ skilled use of visual cues (i.e., features present within an email that signal a problem) have been implicated in their ability to detect phishing emails (bayl-smith, et al., 2020; nasser et al., 2020a, 2020b). for example, irregularities such as an unfamiliar email address or a typographical error will regularly cue the email user to be suspicious of the author’s intentions. in more sophisticated phishing attempts, scammers will tailor content to revolve around a message that prompts the user to act urgently. the message will often include an element of social proof (i.e., everyone is said to be doing the act, or have already done it). further, the author will frequently mimic an authority figure, such as the email user’s boss. indeed, hassandoust et al. (2020) recently found that email users are more vulnerable to phishing attacks when the messages they receive are specific to their context, thereby targeting their individual vulnerabilities. these phish authors aim to conceal their intentions by bypassing users’ analytical or rational modes of thinking (often referred to as system 2 in dual process information processing models; stanovich & west, 2000), and instead target users’ heuristic system of mental shortcuts (i.e. system 1 reasoning). system 1 is automatically favoured upon perception of information, as processing is associated with a miserly use of cognitive resources. this system is very efficient, and extremely useful, leveraging a powerful associative network in memory. however, it is also associated with an increased risk of bias-related error (tversky & kahneman, 1974;1975), which may lead to the misapplication of cues, particularly in conditions characterised by uncertainty and time pressure (kahneman & tversky, 1972). for instance, emotive messages may trigger an affect heuristic, which often lead us to make decisions based on how we feel, rather than think. in this sense, just as a magician uses their knowledge of the nature and limitations of human perception to draw the observer’s attention away from key details of their tricks (e.g., distracting them with movement in one hand, while doing something with the other), the phish author can engineer their email to exploit the cognitive biases of email users. however, such an approach assumes that email users will be largely unable to utilise diagnostic cues, either as a result of misdirection tactics, or a lack of user knowledge and/or skill. cue utilisation has shown to differ as a function of domain expertise, with more experienced people leveraging their store of mental cues to yield more accurate and reliable responses to complex tasks (brams et al., 2019; shanteau, 1992). in addition to expertise, the acquisition and use of cue-based associations may be related to an individual’s typical mode of cognitive reflection, with more intuitive users being more prone to simple heuristic reasoning, compared to more rational users (i.e., system 2 users) who may naturally devote more attentional resources to inspecting emails for suspicious features (harrison et al., 2016; luo et al., 2013; vishwanath et al., 2016; vishwanath et al., 2011); the latter group being better positioned to australasian journal of information systems ackerley et al. 2022, vol 26, research article errors, irregularities, and misdirection 3 ultimately develop skilled intuition within a particular context, which will manifest in the skilful recognition of diagnostic cues (kahneman & klein, 2009). the current study aimed to examine the role of, and potential interplay between, cue utilisation and cognitive reflection in email users’ ability to accurately (and efficiently) differentiate between phishing and genuine emails. in doing so, this work directly extends upon existing phishing victimology frameworks that have established the role of cue utilisation (nasser et al., 2020a, 2020b; bayl-smith et al., 2020) and factors relating to dual process information processing (butavicius et al., 2016; frauenstein & flowerday, 2020; harrison et al., 2016; jones et al., 2019; luo et al., 2013; valecha et al., 2015; vishwanath et al., 2011; vishwanath et al., 2016; vishwanath et al., 2018; yan & gozu, 2012; zhang et al., 2012) in determining users’ phishing susceptibility. critically, clarifying the interplay between users’ perceptual-cognitive skills and their existing preferences concerning information processing have implications to the future design of cyber security interventions such as education and training programs, and more broadly, to our understanding of the acquisition of skilled intuition and expertise in similar domains of judgement and decision-making. 2 cue utilisation relatively greater levels of skilled cue utilisation have been associated with effective decisionmaking in a range of work domains (wiggins, 2021) such as medicine (galanter & patel, 2005), lie detection (morrison et al., 2020; walczyket al., 2009), disaster recovery (gacasan et al., 2016), aviation (wiggins, et al., 2014), drone piloting (shekhet al., 2018), rugby league (johnston & morrison, 2016), forensic investigation (morrison et al., 2018; morrison & morrison, 2015) and electricity power control (wiggins et al., 2014). this is partly because, in complex work settings, the frugal use of a limited number of valid cues will invariably reduce the demands placed on individuals’ working memory during decision-making (brams et al., 2019). consequently, a cue-based strategy enables users to manage multiple tasks simultaneously (klein et al., 1986), which is particularly relevant when considering a phishing diagnosis context wherein engagement with emails is often considered a secondary or ‘background’ task. one model that explores the relationship between cue utilisation and decision-making in a phishing email context is the instance-based learning theory (iblt; gonzalez, 2013). the iblt extends on anderson’s (1996) adaptive control of thought-rational (act-r) architecture, proposing that dynamic decisions are made based on generalisations of previous experiences. in line with the act-r model, these instances reside as chunks within the users’ declarative memory and are activated by cues (e.g., poor grammar) within the email. once cued, the chunk with the highest activation level in memory is retrieved. this is similar to klein et al.’s (1986) recognition-primed decision (rpd) model, which posits that experienced decision makers will use cues to draw on a repertoire of stored patterns that accumulate as a result of domain-specific experiences. in matching these patterns, the operator will nonconsciously generate and prioritise a series of plausible responses, which they can test via a process of mental simulation (a notion consistent with degroot’s concept of ‘progressive deepening’; 1978). thus, cues are thought to be central to both intuitive (system 1) and deliberate (system 2) aspects of the rpd model. upon first attending to email content, it is expected that users form an impression of validity based on the unconscious activation of cuebased associations (the make-up and quality of this network is largely contingent on their domain knowledge). if this impression exceeds a ‘suspicion threshold’, the user will engage a australasian journal of information systems ackerley et al. 2022, vol 26, research article errors, irregularities, and misdirection 4 more deliberate review of the email’s features and content in forming a final judgement of its validity. shonman et al. (2018) investigated the application of iblt on users’ suspicion thresholds, maximum cues processed, and strength of prior cue associations (i.e., chunks) in a phishing diagnosis task. results showed that users who exhibited higher levels of prior activation in memory chunks more accurately identified phishing emails. moreover, users who identified up to four deceptive cues, and therefore reached a suspicion threshold, showed greatest accuracy (shonman et al., 2018). this suggested that users’ capacity for cue utilisation directly shaped their susceptibility to being deceived by phishing emails. in a similar study of cue utilisation and phishing diagnosis, nasser et al. (2020a) employed a cyber security edition of expertise 2.0 (wiggins, 2016; wiggins, et al., 2014), which is a software-based assessment battery designed to objectively assess users’ recognition of cues. the program comprised five tasks, asking individuals to identify diagnostic features (feature identification task; fit), recognise key features (feature recognition task; frt), associate features and/or events in memory (feature association task; fat), discriminate relevant from less relevant features (feature discrimination task; fdt), and prioritise information during problem solving (feature prioritisation task; fpt). participants who showed greater levels of phishing cue utilisation in the program were found to perform better in a separate performance measure of phishing email diagnosis (nasser et al., 2020a). despite differentiating levels of performance based on phishing cue utilisation, the authors found no differences in the types of cues used across ‘high’ and ‘low’ cue users. however, this may be attributed to the fact that participants were asked to select the cues that they used from a list, rather than recall them from memory, which may have led to a rationalisation effect. a further limitation of the work was that it did not address the impacts of users’ individual differences, many of which have been implicated previously in phishing susceptibility. in particular, email users’ cognitive reflection tendencies have been a focal point of recent studies relating to phishing email detection (butavicius et al, 2016; frauenstein & flowerday, 2020; jones et al, 2015; jones et al., 2019; yan & gozu, 2012). critically, such differences may moderate the cue utilisation-performance relationship. 3 cognitive reflection dual-process models (see evans, 2008; stanovich & west, 2000) of information processing have been widely adopted by cyber security researchers as a theoretical framework for understanding phishing victimology (bayl-smith et al., 2020; frauenstein & flowerday, 2020; harrison et al., 2016; luo et al., 2013; valecha et al., 2015; vishwanath et al.,2016; vishwanath et al., 2018; zhang et al., 2012). in such models skilled cue utilisation is presumed to occur as part of the associative mechanisms of system 1, which is characterised by rapid, automatic, unconscious, and relatively effortless responses. however, relatively unskilled cue utilisation and the use of simple ‘rules of thumb’ are also theorised to reside in system 1; the difference between the two types of user behaviour will largely be a function of one’s accumulated expertise in a domain (i.e., whether they possess the right cue-based associations for the job). in contrast, mental simulation, and ‘deepening’ processes (kahneman & klein, 2009) are better aligned to system 2, which is characterised by slow, systematic, deliberate, and relatively effortful progressing. australasian journal of information systems ackerley et al. 2022, vol 26, research article errors, irregularities, and misdirection 5 the elaboration likelihood model (elm) also presents a dual-cognitive model (petty & cacioppo, 1986). the model suggests that individuals using the central information processing pathway activate two sub-processes: attention (i.e., mental focus) and elaboration, described as the process forming connections between the present experience and past knowledge (harrison et al., 2016). according to proponents of this model, a proposed benefit of the system 2 pathway is that individuals engage deeply with contextual features such as those found in phishing emails. here, upon receipt of an email, activation of the system 2 pathway may prompt a user to continue their information search and carefully examine the email for suspicious features (harrison, et al., 2016; luo et al., 2013; vishwanath et al., 2016; vishwanath, et al., 2011). a body of research has implicated cognitive reflection as a significant activator of the systematic thinking pathway (isler et al., 2020; kahneman & klein, 2009; pennycook & rand, 2019). cognitive reflection is defined as an information processing technique and is measured by behavioural outcomes characterised along a spectrum of rapid, impulsive decision-making or slower, rational, and stepwise approaches (frederick, 2005). using the cognitive reflection test (crt; frederick, 2005), butavicius et al. (2016) tested the relationship between participants’ pre-existing cognitive reflection tendencies and their ability to detect phishing email threats. their results revealed that participants who were less cognitively impulsive (i.e., scored higher on the crt) were more likely to judge fraudulent emails as dangerous. here, the authors highlighted the importance of heuristic use, suggesting that participants engaging in less reflective and more intuitive processes (i.e., greater system 1 and less system 2 thinking) may be more prone to increased errors in judgement when assessing phishing emails (butavicius et al., 2016). jones et al. (2019) revealed a similar relationship between participants’ crt results in predicting user susceptibility to phishing emails. in line with these findings, yan and gozu (2012) investigated differences in intuitive or rational decision-making strategies of 171 email users. participants were instructed to give either a rapid (i.e., intuitive) response or assess the email carefully (i.e., rationally) before submitting their response. the results revealed that those in the rational decision-making group correctly identified more emails as suspicious than those who provided an intuitive response (yan & gozu, 2012). comparable effects on performance have been found when implementing a time pressure, which was presumed to induce intuitive responses (jones et al., 2019). similarly, bayl-smith et al., (2020) found that those participants who deliberated longer per email, seemingly demonstrating a greater degree of cognitive reflection, demonstrated an increased ability to correctly identify phishing features. 4 study aims while the heuristic-systematic framework accounts for an unskilled perspective of phishing susceptibility akin to kahneman and tversky’s heuristics and biases paradigm (tversky & kahneman, 1974; 1975), it fails to account for instances of skilled cue utilisation, which are typically studied within a naturalistic decision-making framework (klein, 2008). indeed, with the accumulation of operational experience, we posit that email users will possess numerous cue-based associations that reside in long-term memory, which may be triggered upon reviewing an email. here, if a suspicion threshold is exceeded (presumably based on activation strength; see anderson, 1996), users will engage a more deliberate (and conscious) process of feature scrutiny and verification; presumably via system 2 reasoning. further, we contend that reaching suspicion threshold may be moderated by a multitude of individual differences, for australasian journal of information systems ackerley et al. 2022, vol 26, research article errors, irregularities, and misdirection 6 instance, a user’s natural inclination of engaging a process of cognitive reflection appears highly relevant. therefore, we propose that email users who demonstrate higher cue utilisation in a phishing context and/or a general tendency to engage in cognitive reflection, are likely to be best positioned to detect phishing threats. conversely, we would expect those users with relatively low cue utilisation and cognitive reflection to be most vulnerable to phishing scams. the study aimed to examine the role of, and potential interplay between, cue utilisation and cognitive reflection in email users’ ability to accurately (and efficiently) differentiate between phishing and genuine emails. as in bayl-smith et al. (2020), and nasser et al. (2020a; 2020b), expertise 2.0 (wiggins et al., 2014) was used to gauge participants’ level of phishing cueutilisation. consistent with their findings we expected that those participants with higher levels of cue utilisation would demonstrate superior performance on a phishing diagnosis task compared to those with lower levels of cue utilisation, but that, based on the theorised relationship between greater cognitive reflection and the development of skilled cue utilisation (kahneman & klein, 2009), this relationship would also be moderated by users’ cognitive reflection tendencies. specifically, we hypothesised (h1) that those participants found to be high in both cue utilisation and cognitive reflection would perform significantly better (in relation to phishing diagnosis) than all other participant groups. conversely, we hypothesised (h2) that those participants found to be low in both cue utilisation and cognitive reflection would perform significantly worse than all other participant groups. as higher scores on the cognitive reflection test (crt) have been associated with more ‘patient’ decision-making, presumed to be more informed by careful deliberation (frederick, 2005), it was expected that such deeper processing will manifest in significantly longer response latencies for formulating diagnoses about email suspiciousness. consequently, we hypothesised (h3) that those participants found to possess relatively lower cognitive reflection would demonstrate significantly shorter response latencies when diagnosing an email, compared to those with high cognitive reflection. further, based on the theorised role of cognitive reflection tendencies in the development of skilled cue utilisation (kahneman & klein, 2009), we expected that cue utilisation may moderate the relationship between cognitive reflection and efficiency, in that those individuals with both higher cognitive reflection and cue utilisation are most likely to resemble expert operators in this domain and demonstrate both accurate and efficient visual diagnosis (bram et al., 2019). in this sense, we expect those with high cognitive reflection tendencies to be efficient in domains in which they have accumulated a repertoire of domain-specific associations. therefore, we hypothesised (h4) that those participants who demonstrated a high degree of cognitive reflection will take significantly less time to diagnose when they also possess a high level of cue utilisation, compared to a low level of cue utilisation. finally, despite previous work differentiating levels of performance based on phishing cue utilisation (nasser et al., 2020b), and others eliciting a set of ‘correct’ phishing cues (parsons et al., 2016), differences in the types of cues used across high and low cue users are yet to be captured. this has largely been attributed to limitations in previous designs (e.g., asking participants to recognise the features they use from a pre-defined list; nasser et al., 2020b). as such, a second aim of the study was to, using an open-response format, elicit the cues used by email users when deciding whether an email is genuine or not, and test whether the types of cues reportedly used were associated with the degree of cue utilisation demonstrated by australasian journal of information systems ackerley et al. 2022, vol 26, research article errors, irregularities, and misdirection 7 participants (i.e., whether high and low cue utilisers were found to use different cues when making their decisions). 5 method 5.1 participants a convenience sample of 145 (79 female, 66 male) participants comprised first-year undergraduate psychology students enrolled at charles sturt university (csu; australia) and members of the public. female ages ranged from 18 to 76 years (mage = 38, sdage = 14), and male ages ranged from 18 to 80 years (mage = 34, sdage = 14). 5.2 materials 5.2.1 cognitive reflection the cognitive reflection test (crt; frederick, 2005) was used to measure participants’ cognitive reflection tendencies. the crt is a three-item behavioural-based questionnaire that measures participants’ tendency for intuitive or rational judgment. it is designed to be consistent with stanovich and west’s (2000) conception of a system 1 and system 2 cognitive framework. when completing the measure, participants were asked to respond to three items in open text. a total score of three was obtainable, with each correct answer equalling one point, consistent with frederick’s (2005) scoring method. a higher score on the crt indicated higher levels of cognitive reflection. 5.2.2 phishing diagnosis performance the phishing diagnosis task was hosted and completed on qualtrics (qualtrics, 2019). the task was developed in-house by the research team and measured participants’ accuracy in differentiating between genuine and phishing emails. the task comprised images of five legitimate and five phishing emails, which were presented to participants one at a time, in a random order. legitimate emails were compiled from emails the research team had received, with identifiable information removed. phishing emails were confirmed phishing attempts extracted from the research team’s spam folders over a 6-month period. upon presentation of each email image, participants were asked to click one of two on-screen labels to designate whether they thought the email was “trustworthy” or “suspicious”. 5.2.3 cue utilisation a cyber-security edition of expertise 2.0 was used to evaluate behaviour that was indicative of cue utilisation in the phishing domain. the predictive validity for expertise has been assessed in several domains such as drone search and rescue tasks (shekh et al., 2018), audiology (watkinson et al., 2018) aviation (wiggins, 2014), and most recently in cyber security (nasser et al., 2020b). 5.2.3.1 feature identification task (fit) the fit was used to assess participants’ ability to identify key visual features from 15 individual emails (presented individually) by using their cursor to click on a box appearing on the screen labelled “trustworthy” or clicking on suspicious features of the email itself. the fit measures a participant’s ability to recognise and recall diagnostic cues from the environment (loveday et al., 2013). experts are primed to use diagnostic cues and should reflect lower response latencies (wickens et al., 2013). participants with higher cue utilisation australasian journal of information systems ackerley et al. 2022, vol 26, research article errors, irregularities, and misdirection 8 show lower mean response times across scenarios when identifying key features (e.g., broken images) that were suspicious. 5.2.3.2 feature recognition task (frt) the frt measured participants’ ability to accurately recognise diagnostic features of phishing emails (wiggins et al., 2018). the frt generates a score rather than measuring response latencies. participants were shown 20 emails for a short period of time (1 second) and were then instructed to classify each one as “trustworthy”, “untrustworthy”, or “impossible to tell”. research suggests that experts possess a larger and more refined library of cues in memory, which allows for more rapid responses when identifying prominent cues (loveday et al., 2013; wiggins et al., 2018). 5.2.3.3 feature association task (fat) the fat measured participants’ capacity to recognise associations between specific diagnostic features of a phishing email. evidence has shown that associated concepts are activated within a shorter time frame due to pre-existing associations residing within memory (morrison et al., 2013). two words were presented at the same time, for a short period, with participants having to indicate the relatedness of the words. a total of 15 pairs were shown, with participants indicating relatedness by using a 7-point likert-type scale (1 = extremely unrelated to 7 = extremely related). the variance of responses across individual participants were calculated, with higher levels of cue utilisation found to be associated with greater levels of variance in responses (morrison et al., 2013). 5.2.3.4 feature discrimination task (fdt). the fdt is based on the cochran-weiss-shanteau index (shanteau, et al., 2002), which is based on the premise that expertise can be measured by a ratio of discrimination divided by consistency in responses (weiss & shanteau, 2003). from this formula, the fdt only utilises the discrimination component of this ratio, with experts predicted to display greater variance in responses (loveday, et al., 2013). the fdt measured participants’ ability to discriminate between relevant and less relevant features of a suspected phishing email. participants were asked to read two scenarios about a phishing email they encountered with an associated picture of the email. participants were then given several choices (e.g., pay the duty as requested) and were instructed to choose one. once a decision had been selected, the participants were asked to rate, using a 10-point likert-type scale (1 = not important at all to 10 = extremely important) the extent to which the 10 features of the scenario had impacted their decision. greater levels of variance between responses tend to be associated with greater discrimination, and higher levels of cue utilisation (loveday, et al., 2013; weiss & shanteau, 2003). 5.2.3.5 feature prioritisation task (fpt) the fpt measured participants’ capacity to prioritise cue-based information. participants were given 30 seconds to click on separate menus related to different features of the email (e.g., time the email was sent) across two different scenarios. clicking on separate menus revealed information about the email. at the end of the 30 seconds, participants were asked to formulate a decision about how they would respond to the email, similar to the fdt. the fpt examines the ratio of sequential transitions to total transitions and is based on the finding that superior information acquisition strategies involve discrimination between features based on their australasian journal of information systems ackerley et al. 2022, vol 26, research article errors, irregularities, and misdirection 9 utility, while relatively unskilled acquisition, typically seen in novices, tends to be largely sequential in nature (wiggins et al., 2018). 5.3 cue type survey after completing the 10 email diagnosis task questions, participants were asked one open-text question: “what feature(s) of the emails helped you make your decision?”. 5.4 procedure upon commencing the study, participants were asked to generate a unique six-digit identification code. participants then completed the crt (frederick, 2005), the phishing diagnostic task, and the cue type survey on the qualtrics platform (qualtrics, 2019). upon completion, participants were re-directed to the expertise 2.0 platform, where they would again enter their six-digit code, which was used to link data between the qualtrics and expertise 2.0 platforms. after clicking through, participants recorded their demographic information and completed the five tasks. 6 results 6.1 data reduction participants’ responses on the phishing diagnosis task were scored in relation to their precision in diagnosing a suspicious email among both suspicious (5) and trustworthy (5) emails. this score was calculated by summing the number of correct diagnosis divided by the sum of correct and incorrect diagnoses. participants’ diagnosis efficiency (i.e., time taken to diagnose an email as trustworthy or suspicious) was recorded in seconds (s) and an average response time was calculated for the ten emails. participants’ responses on the crt were scored as either correct or incorrect and tallied to yield a total score out of 3. to identify two relatively distinct groups of participants based on their cognitive reflection tendencies, participants who scored either ‘0’ (low cognitive reflection; n = 57) or ‘3’ (high cognitive reflection; n = 32) were retained (n = 89). all other participants were excluded from analyses involving cognitive reflection. in line with the conventional method for analysing expertise 2.0 data (bayl-smith, et al., 2020; brouwers et al., 2016; sturman et al., 2019), each of the five tasks of expertise 2.0 were standardised into z-scores (see table 1) prior to conducting a k-means cluster analysis. scores on the fat, fdt, and fpt yielded statistically significant mean differences across a two-group model intended to reflect relatively high and low levels of cue utilisation. participants displaying positive standardised mean for the fat and fdt, and a negative standardised mean for the fpt, as observed in the current findings, are generally considered above average in relation to cue utilisation performance (brouwers et al., 2016; loveday et al., 2013; loveday et al., 2014). scores on the fit and frt failed to reveal a statistically significant difference between the two groups and were excluded from further analysis. overall, thirty-nine participants comprised a group whose behaviour was consistent with a relatively ‘high’ degree of cue utilisation, while 50 participants comprised a ‘low’ cue utilisation group. see table 1 for standardised means from the expertise 2.0 tasks by derived cluster centroids for the three retained tasks. australasian journal of information systems ackerley et al. 2022, vol 26, research article errors, irregularities, and misdirection 10 expertise 2.0 task high cue utilisation cluster (n = 39) low cue utilisation cluster (n = 50) feature association task (variance/time) .483 -.416 feature detection task (variance) .888 -.671 feature prioritisation task (ratio) -.244 .170 table 1. standardised means from expertise 2.0 tasks by derived cluster centroids (high and low cue utilisation) for the three retained tasks note: the f test differences between clusters were statistically significant (p < .05). 6.2 cue utilisation, cognitive reflection, and phishing diagnostic performance 6.2.1 diagnosis precision participants’ phishing diagnosis precision was examined using a 2 x 2 factorial betweengroups analysis of variance (anova). results revealed a statistically significant interaction between cognitive reflection and cue utilisation for precision scores, f (1,85) = 5.06, p = .027, partial η2 = .056. a main effect for cognitive reflection was also found, f (1,85) = 4.57, p = .035, partial η2 = .051, but not for cue utilisation, f (1,85) = 1.04, p = .311, partial η2 = .012. descriptive statistics are provided in table 2, and the interaction is shown in figure 1. high cognitive reflection low cognitive reflection total cue utilisation m sd m sd m sd high .79 .11 .80 .12 .79 .11 low .83 .11 .70 .15 .75 .15 total .81 .11 .74 .14 .77 .14 table 2. mean precision for cue utilisation by cognitive reflection to further investigate the interaction, four simple effects tests were performed for cue utilisation and cognitive reflection groupings. a bonferroni adjusted alpha of .0125 was used to control the familywise error rate at .05 (field, 2013). the simple effect for cue utilisation for those in the low cognitive reflection group was statistically significant, f (1,85) = 7.43, p = .008, partial η2 = .075, with those in the high cue utilisation group (m = .796, se = .03) demonstrating greater precision than those in the low cue utilisation group (m = .703, se = .02). the simple effect for those in the high cognitive reflection group was not statistically significant, f (1,85) = .589, p = .445, partial η2 = .007, with those in the high cue utilisation group (m = .793, se = .03) scoring no differently to those in the low cue utilisation group (m = .828, se = .03). the simple effect for cognitive reflection for those in the low cue utilisation group was statistically significant, f (1,85) = 10.988, p = .001, partial η2 = .114, with those in the high cognitive reflection group (m = .828, se = .03) demonstrating greater precision than those in the low cognitive reflection group (m = .703, se = .02). the simple effect for those in the high cue utilisation group was not statistically significant, f (1,85) = .005, p = .941, partial η2 = .000, with those in the high cognitive reflection group (m = .793, se = .03) scoring no differently to those in the low cognitive reflection group (m = .796, se = .03). in other words, those with either high levels of cue utilisation or cognitive reflection (or both) performed to a similar level. this finding does not provide support for the clear advantages predicted among the high cue utilisation and cognitive reflection group (h1), as it appears that a high level of cue utilisation is able to compensate for a low level of cognitive reflection, and australasian journal of information systems ackerley et al. 2022, vol 26, research article errors, irregularities, and misdirection 11 vice versa. however, participants with both low levels of cue utilisation and cognitive reflection were found to perform significantly worse on the phishing diagnosis task than all other groups, supporting h2. figure 1. interaction between cue utilisation and cognitive reflection for diagnosis precision. error bars represent standard errors (±1 se) in other words, those with either high levels of cue utilisation or cognitive reflection (or both) performed to a similar level. this finding does not provide support for the clear advantages predicted among the high cue utilisation and cognitive reflection group (h1), as it appears that a high level of cue utilisation is able to compensate for a low level of cognitive reflection, and vice versa. however, participants with both low levels of cue utilisation and cognitive reflection were found to perform significantly worse on the phishing decision task than all other groups, supporting h2. 6.2.2 diagnosis efficiency participants’ phishing diagnosis efficiency was examined using a 2 x 2 factorial betweengroups analysis of variance (anova). results revealed a statistically significant interaction between cognitive reflection and cue utilisation for average diagnostic time, f (1,85) = 6.61, p = .012, partial η2 = .072. a main effect for cognitive reflection was also found, f (1,85) = 6.49, p = .013, partial η2 = .071, but not for cue utilisation, f (1,85) = 2.85, p = .095, partial η2 = .032. descriptive statistics are provided in table 3, and the interaction is shown in figure 2. high cognitive reflection low cognitive reflection total cue utilisation m sd m sd m sd high 17.54 5.55 12.14 4.98 14.08 5.75 low 13.27 4.16 13.55 5.52 13.45 5.03 total 15.14 5.20 12.93 5.29 13.72 5.34 table 3. mean diagnosis time for cue utilisation by cognitive reflection australasian journal of information systems ackerley et al. 2022, vol 26, research article errors, irregularities, and misdirection 12 figure 2. interaction between cue utilisation and cognitive reflection for average diagnosis time. error bars represent standard errors (±1 se) to further investigate the interaction, four simple effects tests were performed for cue utilisation and cognitive reflection groupings. a bonferroni adjusted alpha of .0125 was used to control the familywise error rate at .05 (field, 2013). the simple effect for those in the low cognitive reflection group was not statistically significant, f (1,85) = .542, p = .464, partial η2 = .006, with those in the high cue utilisation group (m = 11.95, se = .88) taking a similar amount of time to those in the low cue utilisation group (m = 12.82, se = .78). the simple effect for cue utilisation for those in the high cognitive reflection group was statistically significant, f (1,85) = 7.08, p = .009, partial η2 = .077, with those in the high cue utilisation group (m = 16.98, se = 1.18) demonstrating longer average response latencies than those in the low cue utilisation group (m = 12.97, se = 1.04). the simple effect for those in the low cue utilisation group was not statistically significant, f (1,85) = .000, p = .986, partial η2 = .000, with those in the high cognitive reflection group (m = 12.79, se = 1.04) taking a similar amount of time to those in the low cognitive reflection group (m = 12.82, se = .78). the simple effect for cognitive reflection for those in the high cue utilisation group was statistically significant, f (1,85) = 11.65, p = .001, partial η2 = .121, with those in the high cognitive reflection group (m = 16.98, se = 1.18) demonstrating longer average response latencies than those in the low cognitive reflection group (m = 11.95, se = .88). while participants’ cognitive reflection did appear to impact their efficiency in the predicted direction (h3), cue utilisation moderated this relationship, revealing that those with high levels of cue utilisation took significantly longer than those with low cue utilisation, as well as all other participant groups. this finding does not provide support for advantages predicted australasian journal of information systems ackerley et al. 2022, vol 26, research article errors, irregularities, and misdirection 13 among the high cue utilisation group in reducing the time taken to diagnose emails (h4) and suggests that this group were in fact the least efficient. 6.3 types of cues used by participants open-text responses for each participant from the phishing diagnosis task were reviewed by two raters who established thematic labels that represented different cue types (bengtsson, 2016; johnston & morrison, 2016). in line with expectations that participants would use similar cues based on previous literature (parsons et al., 2016; williams et al., 2018), seven main cue types were identified from the 145 text responses (see table 4 for seven cue types). cue type example features look of the email spelling, formatting, punctuation company brand and information if it was a recognised brand, contact information email sender whether the address matched the sender, or the address was surprising relevance of email content similar transactions in the past, or offers too good to be true urgent prompt for action link requesting prompt payment, personal information, asking to reset passwords perceived authenticity of embedded urls whether the link(s) appeared authentic or suspicious, the rationale for clicking the links look of email images image quality of email images, graphic professionalism, quality of images fit with the brand table 4. identified cue types and example features 6.3.1 cue type frequencies and inter-rater reliability in preparation for subsequent tests of association relating to cue type, two raters independently assigned all reported features to one of the established cue types. cohen’s κ was used to assess the level of agreement between the two raters’ judgement whilst controlling for chance agreement, revealing excellent agreement, κ = 0.93 (fleiss et al., 2003). all agreed assignments (287) were retained for the subsequent analyses (gwet, 2001). figure 3. frequency of type of cue used by cue utilisation group (high and low) 0 5 10 15 20 25 30 email text company brand email sender relevance prompt action authenticity url email images fr eq ue nc y cue types high cue utilisation low cue utilisation australasian journal of information systems ackerley et al. 2022, vol 26, research article errors, irregularities, and misdirection 14 6.3.2 cue type and cue utilisation a chi-square test of independence was used to investigate the frequencies of cues used between higher (n = 39) and lower cue utilisation groups (n = 50). results revealed no statistically significant association between type of cue used and cue utilisation, χ2(6) = 3.78, p = .707. this meant that the proportion of each cue type observed in the higher and lower cue utilisation groups was not significantly different. these results can be seen in figure 3. 7 discussion the study aimed to examine the role of, and potential interplay between, cue utilisation and cognitive reflection in email users’ ability to accurately (and efficiently) differentiate between phishing and genuine emails. a second aim of the study involved using an open-response format to elicit the cues used by email users when deciding whether an email is genuine or not, and test whether differences in the types of cues used existed as a function of differences in cue utilisation (i.e., whether high and low cue utilisers were found to use different cues when making their decisions). 7.1 key findings 7.1.1 cue utilisation and cognitive reflection in diagnosing phishing emails participants high in both cue utilisation and cognitive reflection were found to perform at a similar level to those with high cue utilisation but low cognitive reflection, as well as those with low cue utilisation but high cognitive reflection (h1). however, participants with both low levels of cue utilisation and cognitive reflection performed significantly worse in diagnosing phishing emails than all other groups (h2). contrary to expectations, higher levels of cue utilisation did not translate to increased accuracy on the phishing diagnosis task (h1). while these results may indicate that the relationship between cue utilisation and task performance proposed here may not translate in a cyber security context, we speculate that the findings may be attributed to the poverty of cues within a phishing email compared to other ‘cue-rich’ domains (e.g., aviation, driving, criminal investigation). indeed, the limited range of visual markers may restrict the development of perceptual-cognitive skills, limiting the performance advantage gained from valid cue-based associations (brams et al., 2019; french & nevett, 1993; harré et al. 2012). therefore, cue utilisation should be considered alongside other potent predictors of phishing vulnerability, in this case individual differences relating to cognitive reflection. these results imply that the cue-performance relationship, at least in this context, appears to be somewhat contingent on individual differences relating to cognitive reflection tendencies. indeed, taken together with the finding that a high degree of cue utilisation was associated with an increase in the time taken to diagnose emails among high cognitive reflectors (h3 and h4), it may be posited that the most crucial element in predicting performance is an individual’s willingness to ‘engage’ in the diagnostic process, whether by attending to the cues that arouse suspicion, or simply by employing an analytical approach to assessment. while there is precedent for reduced efficiency among higher cue users in other domains (brouwers et al., 2016), we can also speculate that in the current context, expertise 2.0 may not have discerned genuine differences in expertise that are attributed to varying degrees of skilled intuition. when considering the phishing context, we note kahneman and klein’s australasian journal of information systems ackerley et al. 2022, vol 26, research article errors, irregularities, and misdirection 15 (2009) conception of the development of expertise and the pre-requisite of extended exposure to cue-based associations in a ‘high validity’ environment. indeed, we speculate that the causal relationship between suspicious indicators and fraudulent emails may be less stable than in other contexts where skilled intuition is likely to develop and thrive. relatedly, users’ opportunities for feedback in this context are presumably limited, with most emails being valid and trustworthy communications. as such, in this context, expertise 2.0 may be less sensitive to domain-specific cue utilisation differences, and more of an indication of general engagement with diagnostic tasks. overall, this notion of enhanced engagement remains consistent with previous work that has adopted a heuristic-systematic information processing framework in explaining phishing victimisation (bayl-smith et al., 2020; frauenstein & flowerday, 2020; harrison et al., 2016; luo et al., 2013; valecha et al., 2015; vishwanath et al., 2018; zhang et al., 2012). indeed, a vital factor in reducing phishing susceptibility appears to be an individual’s inclination to diligently examine email features when considering its authenticity (harrison et al., 2016; luo et al., 2013; vishwanath et al., 2016; vishwanath et al., 2011). the missing piece of the puzzle may be an accurate record of users’ operational experience in diagnosing phishing emails, which is expected to be both difficult to gauge and limited in variation. for instance, identifying a group possessing extensive deliberate practice in phishing diagnosis is extremely unlikely. nevertheless, future studies should consider strategies to identify genuine experts in this field to enable an investigation of operational experience as an additional moderator in the cue utilisation, cognitive reflection, and phishing performance relationship. in terms of practical implications for organisations, identifying individuals who are less likely to engage in an assessment of an email’s validity, either via the crt or expertise 2.0, may be useful in the implementation of targeted cyber-security training and education interventions. importantly, such programs would need to be intensive and ongoing, with some findings showing the transient nature of behavioural changes among emails users who habitually rely on heuristics (canova et al., 2014; vishwanath, 2015). further, workplaces may benefit from work design interventions that recognise the importance of workers’ conscious attention during email processing (e.g., dedicated email times without concurrent tasks). 7.1.2 cue types and cue utilisation in line with expectations, participants reported using cue types largely consistent with previous literature (parsons et al., 2016; williams et al., 2018). parsons et al. (2016) identified cues such as consistent logos (e.g., company brand), links, personalisation, and spelling cues. the current study found cues relating to inconsistencies within the text of the email (e.g., spelling, formatting, and punctuation), perceived authenticity of embedded urls (e.g., if the links appeared authentic) and company brand. interestingly, an “urgent prompt for action” cue was identified by participants, which was conceptually similar to the “urgency” cue/influence technique found in williams et al. (2018). the current study explored cue types further, by analysing if they differed significantly among different levels of cue user (i.e., high and low). however, cue type did not significantly differ between different levels of cue utilisation, suggesting that participants were not using significantly different cues during decision making. this finding is noteworthy, as previous research has advocated for the use of critical cue inventories as the basis for training interventions (morrison et al., 2013; wiggins & o’hare, 2003). similarly, many cyber-experts have promoted a rule-based approach to phishing education (basnet et al., 2012; moghimi & australasian journal of information systems ackerley et al. 2022, vol 26, research article errors, irregularities, and misdirection 16 varjani, 2016). critical cue inventories and rule-based approaches place an emphasis on identifying cues as the primary factor of good decision-making. however, the disparity in the current study’s findings suggests that simply having an awareness for said cues does not appear to determine performance on its own. instead, it appears that knowledge may be a prerequisite alongside a willingness to engage in a process of assessment. thus, while presumably useful, the effectiveness of interventions centred on the learning of critical cue inventories may be largely moderated by other attributes of individual learners. it must be noted that the choice to survey respondents on the cues used across all emails makes it difficult to identify specific predictors of performance. it was decided that surveying of cue usage after each email, although ideal, would be too onerous on participants considering the time commitment associated with participation in the experiment. future research could consider more ‘online’ measures of cue utilisation, such as the use of eye-tracking. 7.2 limitations the primary limitation of the research approach was the artificial nature of the phishing diagnostic performance measure, which may have yielded several experimental artefacts. for instance, the fact that participants were advised that the study involved detecting phishing emails prior to participation may have elicited expectation effects not present during realworld operation (see levine’s truth-default theory; 2014). at worst, such a process may inadvertently prime some participants to engage more analytical, system 2 processes during decision-making, which may confound the true effects of users’ cognitive reflection tendencies. work is underway to investigate the impact of differences in the ratio of trustworthy to suspicious emails in controlled phishing experiments. it is possible that participants’ will anticipate an inflated number of phishing emails in experimental settings, which would potentially impact their precision in detecting phishing threats (see canfield et al., 2016). we also intend to test the generalisability of our results beyond controlled designs via the use of naturalistic study techniques, such as the use of simulated phishing emails delivered to participants’ actual inboxes at infrequent times. 8 conclusion the current study presents several novel findings. first, results revealed that participants with relatively low cue utilisation and cognitive reflection were found to be significantly more vulnerable to deceptive phishing emails, and that performance could be greatly improved with an elevation in either quality. next, a number of phishing email cues were identified by participants, however, the types of cues utilised did not differentiate between high and low cue users. these findings extend on a raft of others purporting the significance of cue utilisation in diagnostic performance, this time in a cyber security context, but also uncover an important moderating effect based on individual differences in email users’ cognitive reflection tendencies, which has not been established previously. taken together, the findings have implications to the design of user-level interventions in relation to the identification of vulnerable users, as well as the need to consider training approaches that extend beyond the use of simple cue inventories. references anderson, j. r. (1996). act: a simple theory of complex cognition. american psychologist, 51(4), 355-365. https://doi:10.1037/0003-066x.51.4.355 https://doi:10.1037/0003-066x.51.4.355 australasian journal of information systems ackerley et al. 2022, vol 26, research article errors, irregularities, and misdirection 17 basnet r.b., sung a.h., & liu q. (2012) feature selection for improved phishing detection. in: jiang h., ding w., ali m., & wu x. (eds) advanced research in applied artificial intelligence. iea/aie 2012. lecture notes in computer science, vol 7345. springer, berlin, heidelberg. https://doi.org/10.1007/978-3-642-310 bayl-smith, p., sturman, d., & wiggins, m. (2020). cue utilization, phishing feature and phishing email detection. in m. bernhard, a. bracciali, l. j. camp, s. matsuo, a. maurushat, p. b. rønne, & m. sala (eds.), financial cryptography and data security: fc 2020 international workshops, asiausec, codefi, voting, and wtsc, revised selected papers (pp. 56-70). (lecture notes in computer science (including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics); vol. 12063). springer. https://doi.org/10.1007/978-3-030-54455-3_5 bengtsson, m. (2016). how to plan and perform a qualitative study using content analysis. nursing plus open, 2, 8-14. https/doi:10.1016/j.npls.2016.01.001 brams, s., ziv, g., levin, o., spitz, j., wagemans, j., williams, a. m., & helsen, w. f. (2019). the relationship between gaze behavior, expertise, and performance: a systematic review. psychological bulletin, 145(10), 980–1027. https://doi.org/10.1037/bul0000207 brouwers, s., wiggins, m., & griffin, b. (2018). operators who readily acquire patterns and cues, risk being miscued in routinized settings. journal of experimental psychology: applied, 24(2), 261-274. https://doi:10.1037/xap0000151 butavicius, m., parsons, k., pattison m., & mccormac, a. (2016). breaching the human firewall: social engineering inpphishing and spear-phishing emails. arxiv, abs/1606.00887. canfield, c. i., fischhoff, b., & davis, a. (2016). quantifying phishing susceptibility for detection and behavior decisions. human factors: the journal of the human factors and ergonomics society, 58(8), 1158–1172. https://doi.org/10.1177/0018720816665025 degroot, a. d. (1978). thought and choice in chess. the hague: mouton downs, j. s., holbrook, m. b., & cranor, l. f. (2006). decision strategies and susceptibility to phishing. in proceedings of the second symposium on usable privacy and security (pp. 79-90). acm. https://doi:10.1145/1143120.1143131 fette, i., sadeh, n., & tomasic, a. (2007). learning to detect phishing emails. in www ‘07 proceedings of the 16th international conference on world wide web (pp. 649-656). new york: acm. https://doi:10.21236/ada456046 field, a. (2013). discovering statistics using ibm spss statistics. london, england: sage. fleiss, j. l., levin, b., & paik, m. c. (2013). statistical methods for rates and proportions. hoboken: john wiley & sons. frauenstein, e. d., & flowerday, s. (2020). susceptibility to phishing on social network sites: a personality information processing model. computers & security, 94, 101862. https://doi.org/10.1016/j.cose.2020.101862 frederick, s. (2005). cognitive reflection and decision making. journal of economic perspectives, 19(4), 25-42. https://doi.org/10.1257/089533005775196732 french, k. e., & nevett, m. e. (1993). the development of expertise in youth sport. in j. l. starkes & f. allard (eds.), advances in psychology (pp. 255-270). north-holland. https://doi.org/https://doi.org/10.1016/s0166-4115(08)61475-2 gacasan, e. m. p., wiggins, m. w., & searle, b. j. (2016). the role of cues in expert project manager sensemaking. construction management and economics, 34(7-8), 492-507. https://doi:10.1080/01446193.2016.1177190 https://doi/ https://doi:10.1037/xap0000151 https://doi.org/10.1177/0018720816665025 https://doi:10.1145/1143120.1143131 https://doi:10.21236/ada456046 https://doi.org/10.1016/j.cose.2020.101862 https://doi.org/10.1257/089533005775196732 https://doi.org/https:/doi.org/10.1016/s0166-4115(08)61475-2 https://doi:10.1080/01446193.2016.1177190 australasian journal of information systems ackerley et al. 2022, vol 26, research article errors, irregularities, and misdirection 18 galanter, c. a., & patel, v. l. (2005) medical decision making: a selective review for child psychiatrists and psychologists. journal of child psychology and psychiatry, 46(7), 675-689. https://doi:10.1111/j.1469-7610.2005.01452.x gonzalez, c. (2013). the boundaries of instance-based learning theory for explaining decisions from experience. in v. s. c. pammi & n. srinivasan (eds.), progress in brain research (vol. 202, pp. 73-98). oxford, uk: elsevier. gwet, k. (2001). handbook of inter-rater reliability. gaithersburg: stataxis publishing company. harrison, b., svetieva, e., & vishwanath, a. (2016). individual processing of phishing emails. online information review, 40(2), 265–281. https://doi.org/10.1108/oir-04-2015-0106 harré, m., bossomaier, t., & snyder, a. (2012). the perceptual cues that reshape expert reasoning. scientific reports, 2(1), 502–502. https://doi.org/10.1038/srep00502 hassandoust, f., singh, h., & williams, j. (2020). the role of contextualization in individuals’ vulnerability to phishing attempts. australasian journal of information systems, 24. doi: https://doi.org/10.3127/ajis.v24i0.2693 johnston, d., & morrison, b. w. (2016). the application of naturalistic decision-making techniques to explore cue use in rugby league playmakers. journal of cognitive engineering and decision making, 10(4), 391-410. https://doi:10.1177/1555343416662181 jones, h.s., towse, j. n., & race, n. (2015) susceptibility to email fraud: a review of psychological perspectives, data-collection methods, and ethical considerations. international journal of cyber behaviour, psychology and learning, 5(3). 13-29. https://doi:10.4018/ijcbpl.2015070102 jones, h.s., towse, j. n., race, n., & harrison, t. (2019). email fraud: the search for psychological predictors of susceptibility. plos one 14(1), e0209684. https://doi:10.1371/journal.pone.0209684 kahneman, d., & klein, g. (2009). conditions for intuitive expertise: a failure to disagree. american psychologist, 64(6), 515-526. https://doi:10.1037/a0016755 klein, g. a. (2008). naturalistic decision making. human factors, 50(3), 456–460. https://doi.org/10.1518/001872008x288385 klein, g. a., calderwood, r., & clinton-cirocco, a. (1986). rapid decision making on the fire ground. proceedings of the human factors society annual meeting, 30(6), 576-580. kobus, d. a., proctor, s., & bank, t. e. (2000). decision-making in a dynamic environment: the effects of experience and information uncertainty. technical report 1832. san diego, ca: spawar systems center. levine, t. r. (2014). truth-default theory (tdt). journal of language and social psychology, 33(4), 378–392. https://doi:10.1177/0261927x14535916 loveday, t., wiggins, m., festa, m., schell d., & twigg, d. (2013). pattern recognition as an indicator of diagnostic expertise. in c. p. latorre & f. a. sanchez (eds.), pattern recognition – applications and methods (pp. 1-11). berlin: springer. luo, x., zhang, w., burd, s., & seazzu, a. (2013). investigating phishing victimization with the heuristic-systematic model: a theoretical framework and an exploration. computer security, 38, 28–38. https://doi.org/10.1016/j.cose.2012.12.003 moghimi, m., & varjani, a. y. (2016). new rule-based phishing detection method. expert systems with applications, 53, 231–242. https://doi:10.1016/j.eswa.2016.01.028 morrison, b. w., johnston, d., naylor, m., morrison, n. m. v., & forrest, d. (2020). “you can’t hide your lyin’ eyes”: investigating the relationship between associative learning, cue https://doi:10.1111/j.1469-7610.2005.01452.x https://doi.org/10.1108/oir-04-2015-0106 https://doi/ https://doi/ https://doi:10.1177/1555343416662181 https://doi:10.4018/ijcbpl.2015070102 https://doi:10.1371/journal.pone.0209684 https://doi:10.1037/a0016755 https://doi.org/10.1518/001872008x288385 https://doi:10.1177/0261927x14535916 https://doi.org/10.1016/j.cose.2012.12.003 https://doi:10.1016/j.eswa.2016.01.028 australasian journal of information systems ackerley et al. 2022, vol 26, research article errors, irregularities, and misdirection 19 awareness, and decision performance in detecting lies. journal of cognitive engineering and decision making, 14(2), 99-111. https://doi.org/10.1177/1555343420918084 morrison, b., & morrison, n. (2015). diagnostic cues in major crime investigation. in m. w. wiggins, & t. loveday (eds.), diagnostic expertise in organizational environments (pp. 9198). ashgate publishing morrison, b. m., wiggins, m. w., bond n. w., & tyler, m. d. (2013). measuring relative cue strength as a means of validating an inventory of expert offender profiling cues. journal of cognitive engineering and decision making, 7(2), 211-226. https://doi:1177/ 1555343412459192 morrison, b. w., wiggins , m. w., & morrison, n. v. (2018). utility of expert cue exposure as a mechanism to improve decision-making performance among novice criminal investigators. journal of cognitive engineering and decision making, 12(2), 99-111. https://doi:10.1177/1555343417746570 nasser, g., morrison, b. w., bayl-smith, p., taib, r., gayed, m., & wiggins, m. w. (2020a). the effects of cue utilization and cognitive load in the detection of phishing emails. in asiausec’20: proceedings of the workshop on usable security (pp. 1-10). malaysia: springer. nasser, g., morrison, b. w., bayl-smith, p., taib, r., gayed, m., & wiggins, m. w. (2020b). the role of cue utilization and cognitive load in the recognition of phishing emails. frontiers in big data, 3, 546860. https://doi.org/10.3389/fdata.2020.546860 parsons, k., butavicius, m., pattinson, m., mccormac, a., calic, d., & jerram, c. (2016). do users focus on the correct cues to differentiate between phishing and genuine emails? arxiv preprint arxiv:1605.04717. qualtrics (version 2019). [web-based software]. provo, ut: qualtrics. available from http://www.qualtrics.com. shanteau, j., weiss, d. j., thomas, r. p., & pounds, j. c. (2002). performance-based assessment of expertise: how to decide if someone is an expert or not. european journal of operational research, 136(2), 253-263. https://doi:10.1016/s0377-2217(01)00113-8 shekh, s., auton, j. c., & wiggins, m. w. (2018). the effects of cue utilization and target-related information on target detection during a simulated drone search and rescue task. proceedings of the human factor and ergonomics society annual meeting, 62(1), 227-231. https://doi:10.1177/1541931218621053 shonman, m., li, x., zhang, h., & dahbura, a. (2018). simulating phishing email processing with instance-based learning and cognitive chunk activation. in s. wang, v. yamamoto, j. su, y. yang, e. jones, l iasemidis & t. mitchell (eds.), lecture notes in computer science: vol 11309. brain informatics (pp. 468-478). cham: springer. https://doi:10.1007/978-3-03005587-5_44 stanovich, k. e., & west, r. f. (2000). individual differences in reasoning: implications for the rationality debate? behavioral and brain sciences, 23(5), 645-665. https://doi:10.1017/ s0140525x00003435 kahneman, d., & tversky, a. (1972). subjective probability: a judgment of representativeness. cognitive psychology, 3(3), 430-454. https://doi.org/10.1016/0010-0285(72)90016-3. tversky, a., & kahneman, d. (1974). heuristics and biases: judgement under uncertainty. science, 185(1974), 1124-1130. https://doi:10.1126/science.185.4157.1124 tversky a., & kahneman d. (1975) judgment under uncertainty: heuristics and biases. in: wendt d., vlek c. (eds) utility, probability, and human decision making. theory and decision library (an international series in the philosophy and methodology of the https://doi.org/10.1177/1555343420918084 https://doi:10.1177/1555343417746570 https://doi.org/10.3389/fdata.2020.546860 http://www/ https://doi:10.1016/s0377-2217(01)00113-8 https://doi:10.1177/1541931218621053 https://doi:10.1007/978-3-030-05587-5_44 https://doi:10.1007/978-3-030-05587-5_44 https://doi/ https://doi:10.1126/science.185.4157.1124 australasian journal of information systems ackerley et al. 2022, vol 26, research article errors, irregularities, and misdirection 20 social and behavioral sciences), vol 11. springer, dordrecht. https://doi.org/10.1007/97894-010-1834-0_8 vishwanath, a. (2015). habitual facebook use and its impact on getting deceived on social media. journal of computer-mediated communication, 20(1), 83-98 vishwanath, a., harrison, b., & ng, y.j. (2016). suspicion, cognition, and automaticity model of phishing susceptibility. commun. res. 1–21. https://doi.org/10.1177/0093650215627483 vishwanath, a., harrison, b., & ng, y. j. (2018). suspicion, cognition, and automaticity model of phishing susceptibility. communication research, 45(8), 1146–1166. https://doi.org/ 10.1177/0093650215627483 vishwanath, a., herath, t., chen, r., wang, j., & rao, h.r. (2011), “why do people get phished? testing individual differences in phishing vulnerability within an integrated information processing model”. decision support systems, 51(3), 576-586. walczyk, j. j., mahoney, k. t., doverspike, d., & griffith-ross, d. a. (2009). cognitive lie detection: response time and consistency of answers as cues to deception. journal of business and psychology, 24, 33-49. https://doi:10.1007/s10869-009-9090-8 watkinson, j., bristow, g., auton, j., mcmahon, c. m., & wiggins, m. w. (2018). postgraduate training in audiology improves clinicians’ audiology-related cue utilisation. international journal of audiology, 57(9), 681-687. https://doi:10.1080/14992027.2018.1476782 weiss, d. j., & shanteau, j. (2003). empirical assessment of expertise. human factors: the journal of the human factors and ergonomics society, 45(1), 104-116. https://doi:10.1518/hfes.45.1. 104.27233 wickens, c. d., hollands, j. g., banbury, s., & parasuraman, r. (2013). engineering psychology and human performance. new york: psychology press. https://doi:10.4324/9781315665177 wiggins, m. w. (2021). a behaviour-based approach to the assessment of cue utilisation: implications for situation assessment and performance. theoretical issues in ergonomics science, 22(1), 46-62. https://doi.org/10.1080/1463922x.2020.1758828 wiggins, m. w. (2016). expertise and cognitive skills development for ab-initio pilots. in r. a. telfer & p. j. moore (eds.), aviation training: learners, instruction and organization (pp 5466). abington, oxon: routledge. wiggins, m. w., brouwers, s., davies, j., & loveday, t. (2014). trait-based cue utilization and initial skill acquisition: implications for models of the progression to expertise. frontiers in psychology, 5, 541. https://doi:10.3389/fpsyg.2014.00541 wiggins, m. w., crane, m., & loveday, t. (2018). cue utilization, perceptions, and experience in the interpretation of weather radar returns. proceedings of the human factors and ergonomics society annual meeting, 62(1), 721-725. https://doi:10.1177/1541931218621164 wiggins, m, w., griffins, b., & brouwers, s. (2019). the potential role of context-related exposure in explaining differences in water safety cue utilization. human factors: the journal of the human factors and ergonomics society, 61(5), 825-838. https://doi:10.1177/ 0018720818814299 wiggins, m., loveday, t., & lyons, l. (2014). cues and cue-based processing: implications for system safety. procedia engineering, 84, 55-61. https://doi:10.1016/j.proeng.2014.10.409 wiggins, m. w., whincup, e., & auton, j. c. (2018). cue utilisation reduces effort but increases arousal during a process control task. applied ergonomics, 69, 120-127. https://doi:10.1016/ j.apergo.2018.01.012 williams, e. j., hinds, j., & joinson, a. n. (2018). exploring susceptibility to phishing in the workplace. international journal of human-computer studies, 120, 1-13. https://doi:10.1016/ j.ijhcs.2018.06.004 https://doi.org/10.1007/978-94-010-1834-0_8 https://doi.org/10.1007/978-94-010-1834-0_8 https://doi.org/10.1177/0093650215627483 https://doi:10.1007/s10869-009-9090-8 https://doi:10.1080/14992027.2018.1476782 https://doi:10.4324/9781315665177 https://doi:10.3389/fpsyg.2014.00541 https://doi:10.1177/1541931218621164 https://doi:10.1016/j.proeng.2014.10.409 australasian journal of information systems ackerley et al. 2022, vol 26, research article errors, irregularities, and misdirection 21 yan, z., & gozu, h. y. (2012). online decision-making in receiving spam emails among college students. international journal of cyber behavior, psychology and learning, 2(1), 1-12. https://doi:10.4018/ijcbpl.2012010101 copyright: © 2022 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v26i0.3615 https://doi:10.4018/ijcbpl.2012010101 http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 cue utilisation 3 cognitive reflection 4 study aims 5 method 5.1 participants 5.2 materials 5.2.1 cognitive reflection 5.2.2 phishing diagnosis performance 5.2.3 cue utilisation 5.2.3.1 feature identification task (fit) 5.2.3.2 feature recognition task (frt) 5.2.3.3 feature association task (fat) 5.2.3.4 feature discrimination task (fdt). the fdt is based on the cochran-weiss-shanteau index (shanteau, et al., 2002), which is based on the premise that expertise can be measured by a ratio of discrimination divided by consistency in responses (weiss & shanteau, 2003). from this formula, t... 5.2.3.5 feature prioritisation task (fpt) 5.3 cue type survey 5.4 procedure 6 results 6.1 data reduction 6.2 cue utilisation, cognitive reflection, and phishing diagnostic performance 6.2.1 diagnosis precision 6.2.2 diagnosis efficiency 6.3 types of cues used by participants 6.3.1 cue type frequencies and inter-rater reliability 6.3.2 cue type and cue utilisation 7 discussion 7.1 key findings 7.1.1 cue utilisation and cognitive reflection in diagnosing phishing emails 7.1.2 cue types and cue utilisation 7.2 limitations 8 conclusion paper title (paper title style) australasian journal of information systems davern & black 2024, vol 28, editorial editorial 1 editors’ comments ajis thinking differently in service of the community michael davern faculty of business and economics the university of melbourne parkville, vic australia email: mjdavern@unimelb.edu.au stuart black faculty of business and economics the university of melbourne parkville, vic australia email: stuart.black@unimelb.edu.au lend me your ears and i’ll sing you a song (the beatles, “with a little help from my friends”, 1967) 1 introduction in deference to karlheinz kautz, our immediate predecessor as editor-in-chief, we thought that we would use a music reference to start our ajis journey. we thank karl for his work over the past few years to build ajis to its current position, and we are glad to have karl continue with ajis: continuing his role as commissioning editor for the submissions initially received prior to the eic transition, formally supporting us through his role as “immediate past eic” on the ajis editorial advisory board (clarke, 2024), and continuing to provide high quality ad-hoc advice. 2 with a little help from my friends ajis is first and foremost a journal for the australasian is community. we have a mandate to strengthen, nurture and develop the community of scholars in the region, but we need the help of you, our community, to do so. we need your submissions; we need your willingness to be a reviewer or senior editor when requested. we are pleased to have an expanded editorial office with two editors-in-chief (michael davern & stuart black), a chief technology officer (john james) and a production editor (jacqui blake). we have a small but growing group of leading scholars and emerging leaders forming our troop of new senior editors (angsana techatassanasoontorn, arif perdana, humza naseer, ida someh, petros chamakiotis, and wasana bandara) who will maintain ongoing appointments with the journal rather than the previous approach of situational appointments. we thank scholars who have helped in the past in editorial roles (and continue to work through the current pipeline) and look forward to working with our new senior editors. as we build our se team, we are purposefully seeking diversity in our se appointments (in topic, method, geography, institution and gender). to better serve our community we are making some changes to ajis processes and systems. with the support of aais, we are moving to have ajis operate under the aais domain (aaisnet.org). we have instigated a rapid initial review process. when a paper passes the desk australasian journal of information systems davern & black 2024, vol 28, editorial editorial 2 reject filter, the eic’s, in consultation with the ses when appropriate, provide a brief review to identify any major issues and return the paper to the authors to address and resubmit prior to assigning an se and reviewers. our goal, which we are currently meeting, is to have this rapid review back to the authors within 10 days. as a result, the papers received by ses and reviewers are already of a higher quality. by reducing the workload for ses and reviewers in this way, we can tighten expected turnaround times for ses and reviewers and ultimately expedite papers faster through the review process. to maintain accountability to our community, we will regularly report on review process metrics via the journal website in due course. 3 think differently the research marketplace has changed dramatically since the inception of ajis and so too has the role of ajis evolved. our mantra for the papers ajis seeks to publish is that they must be ‘interesting but sufficiently rigorous’. a paper in ajis should cause readers to think differently. it should provide insight to practice as well as the academy. it should be sufficiently rigorous to ensure the credibility of the findings. to this end, we encourage potential ajis authors to reverse the typical thinking on the socalled ‘research-practice gap’ (makin, 2021; shapiro et al., 2007) and explore practitioner / policymaker / individual problems that would benefit from formal research and/or explore real-world phenomena that is not well supported by existing literature. we encourage researchers to strongly consider problematization research approach (alvesson & sandberg, 2011), or alternatively to identify what the literature gets wrong (kane, 2022, p. 1074). topically and methodologically, we embrace the diversity that makes information systems such a fascinating discipline. we view the discipline as the intersection of three broad overlapping areas: a social agent (e.g., individuals, organisation), a domain context (e.g. application area or use-case setting) and an information technology artefact (software, hardware, data). while different papers may lean more in one direction than another, the research question and findings need to be relevant to the intersection. is research isn’t simply solving a domain problem or exploring the behaviour of individuals or organisations or it itself rather it combines all three. in short, we expect papers to “try to sing in tune” with information systems as a discipline, not computer science or a business or social science discipline. ajis will continue its policy of declining descriptive literature reviews and derivative adoption studies that lack new insight or challenge existing thinking. gap-spotting style submissions with limited impact (sandberg & alvesson, 2011), regardless of how well executed will likely get desk rejected. instead, we want to invest the journal’s editorial capacity to nurture articles that provide a high ‘return on reader’s time investment’. editorial capacity will be deployed to encourage and develop scholars to thrive in the more engaged academy of the future. ajis must be visible and relevant to all three stakeholder groups (the research community, practitioners and policymakers). as editors, we will seek to ensure that all accepted articles clearly articulate the implications of the research for both fellow researchers as well as practitioners / managers and policy-makers, as appropriate. in pursuing this “think differently” approach, we are prepared to take a greater risk with papers that have the potential to generate interest, insight, and broader impact beyond citation metrics and the like. we seek forward-looking and provocation-style submissions (burtonaustralasian journal of information systems davern & black 2024, vol 28, editorial editorial 3 jones et al., 2021), and already have begun work on pumping the pipeline for such submissions from leading scholars within our community. we are also open for authors to reach out with proposals for ‘registered report’ style papers where topics are particularly risky or unique to the regional community. the existing track on ‘research on indigenous use of information and communication technologies’ comes to mind here (see gregor et al., 2023 as an example), at least with respect to the indigenous peoples of our region. we are also interested in surfacing and supporting submissions from the australasian association of information systems catchment outside of australia and new zealand. 4 it's your community be part of it! while the above are our thoughts on our current plans, we stress that ajis is a community asset, and we are only the stewards thereof. if you have thoughts or suggestions on what else the journal might do to provide benefits to the aais community, please get in touch via ajis.eic@aaisnet.org. we look forward to reading your interesting and thought-provoking submissions and engaging with you through the editorial process to grow ajis as a real community asset. references alvesson, m., & sandberg, j. (2011). generating research questions through problemitization. academy of management review, 36(2), 247–271. burton-jones, a., butler, b. s., scott, s. v., & xin xu, s. (2021). examining assumptions: provocations on the nature, impact, and implications of is theory. mis quarterly, 45(1), 453–498. https://doi.org/10.25300/misq/2021/15434.1 clarke, r. (2024, may 20). aais executive vote of thanks to the outgoing ajis editor-inchief, prof. karlheinz kautz. aais. https://aaisnet.org/news/aais-executive-vote-ofthanks-to-the-outgoing-ajis-editor-in-chief-prof-karlheinz-kautz/ gregor, s., gunawardena, m., imran, a., okai-ugbaje, s., page jeffery, c., & wilson, r. (2023). indigenous knowledge-sharing interventions in australia and the use of information and communication technology: a scoping review. australasian journal of information systems, 27. https://doi.org/10.3127/ajis.v27i0.4217 kane, g. c. (2022). how to write an a paper. journal of the association for information systems, 23(5), 1071–1077. https://doi.org/10.17705/1jais.00765 makin, s. (2021). the research-practice gap as a pragmatic knowledge boundary. information and organization, 31(2), 100334. https://doi.org/10.1016/j.infoandorg.2020.100334 sandberg, j., & alvesson, m. (2011). ways of constructing research questions: gap-spotting or problematization? organization, 18(1), 23–44. https://doi.org/10.1177/1350508410372151 shapiro, d. l., kirkman, b. l., & courtney, h. g. (2007). perceived causes and solutions of the translation problem in management research. academy of management journal, 50(2), 249–266. https://doi.org/10.5465/amj.2007.24634433 copyright copyright © 2024 michael davern and stuart black. this is an open-access article licensed under a creative commons attribution-non-commercial 4.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. https://creativecommons.org/licenses/by-nc/4.0/deed.en australasian journal of information systems davern & black 2024, vol 28, editorial editorial 4 doi: https://doi.org/10.3127/ajis.v28.5427 https://doi.org/10.3127/ajis.v28.5427 paper title (paper title style) australasian journal of information systems taj, kautz & bruno 2021, vol 25, selected papers from acis 2019 it-enabled ot as a coevolution process 1 the coevolution of routines and it systems in it-enabled organizational transformation as an instance of digital transformation: a social constructivist perspective faqir taj school of accounting, information systems and supply chains college of business, rmit university karlheinz kautz school of accounting, information systems and supply chains college of business, rmit university karlheinz.kautz@rmit.edu.au vince bruno school of accounting, information systems and supply chains college of business, rmit university abstract this paper proposes a conceptual framework to study the phenomenon of it-enabled organizational transformation (it-enabled ot) as a coevolution process of organizational routines and a new it system. the framework’s objective is to understand it-enabled ot in a holistic and integrated manner by investigating how actors perceive, interpret, appropriate, and enact, the new it system in their work routines as well as how they align the new system and these routines with the social order and structures of the organization. it allows the examination of the reciprocal interactions between different aspects of organizational routines and a new it system to enhance the understanding of how change unfolds in an organization during the implementation, the adoption, use, and adaptation of a new it system. we illustrate the scope, the analytic and conceptual strength of the framework with a number of examples from the literature and, lastly, discuss its ontological positioning. the paper concludes with a call for further research to empirically validate and refine the proposed framework. keywords: it-enabled ot, organizational routines, coevolution, appropriation, enactment 1 introduction digital transformation (dt) has emerged as an important phenomenon in is research and practice. in a recent comprehensive literature review vial (2019) defines dt as “a process that aims to improve an entity by triggering significant changes to its properties through combinations of information, computing, communication, and connectivity technologies” where the concept entity refers to either an organization, an industry or society (vial, 2019, p. 121). both agarwal et al. (2010) and vial (2019) discuss, that at a high level, dt encompasses the profound changes taking place in society and industry through the use of digital technologies. additional grounding in the extant literature (e.g. hess et al., 2016) vial (2019) states that at the organizational level dt refers to the reaction of contemporary firms to find ways to innovate with these technologies by devising and implementing strategies to drive better operational performance and to remain competitive in a digital world. in this context he refers to previous research and the concept of it-enabled organizational transformation. while australasian journal of information systems taj, kautz & bruno 2021, vol 25, selected papers from acis 2019 it-enabled ot as a coevolution process 2 wessel et al. (2020) posit that dt leverages digital technology by (re)defining an organization’s value proposition and involves a new organizational identity whereas it-enabled ot leverages digital technology in supporting the existing value proposition and enhances an existing organizational identity, the authors also state that the two might overlap and that dt could comprise it-enabled ot. we therefore consider it-enabled ot as an instance of dt because it has a particular focus on organizational change as a necessary step to unlock the transformative potential of digital technologies in an organization (vial, 2019). the concept of it-enabled ot encompasses the substantial changes that occur to the structure and work practices of an organization due to the implementation comprising the adoption, use, and adaptation of a new it system (weick, 1993; orlikowski, 1996; leonardi & barley, 2010; besson & rowe, 2012). orlikowski (1996) presented a study of a customer support system where the structure and organizing practices of a software company were radically changed over a period of two years following the introduction of a new it system. for instance, the nature of work in the organization changed from unstructured to more structured, patterns of interactions changed from reactive to proactive, and coordination mechanisms changed from functional to cross-functional. orlikowski (1996) argues that the introduction of the new customer support system in the organization triggered the process of organizational transformation but that the transformative change actually happened through situated actions and improvisations as the organizational actors were enacting the new it system into their work routines (see also vaast & walsham, 2005; leonardi & barley, 2010). both academics and practitioners have been struggling for a long time to understand how the implementation, and in particular the use of a new it system transform organizations (leonardi & barley, 2010; besson & rowe, 2012). is scholars who study it-enabled ot from constructivist perspectives, generally hold that organizational change emerges from an ongoing stream of social actions in which actors respond to an it system’s affordances and constraints (leonardi & barley, 2010). leonardi and barley (2010) have approached it-enabled ot from five coherent social constructivist perspectives: perception, interpretation, appropriation, enactment, and alignment to explain the social construction of a newly implemented it system. they argue that the five constructivist perspectives build a robust approach to investigate how actors and an it system interact with each other in creating a new social reality in the organization. however, only a few studies have combined different constructivist perspectives to study the phenomenon of it-enabled ot in a holistic and integrated manner (orlikowski, 1996; leonardi, 2007; leonardi & barley, 2010; besson & rowe, 2012). leonardi and barley (2010) found only two studies, in which the enactment perspective on situated actions and improvisations is linked with the alignment perspective (orlikowski, 1996; leonardi, 2007). in addition, most studies are tilted towards studying the evolution of social processes and ignore the evolution of the it system in the implementation process (leonardi & barley, 2010). the evolution of the it system in it-enabled ot however requires due consideration (leonardi, 2009) as also highlighted by orlikowski and iacono (2001) when desperately seeking the “it” in is research. in order to address these issues, this paper proposes a conceptual framework that depicts it-enabled ot as a coevolution process of organizational routines and a new it system. the conceptual framework considers an organization as a collection of routines (winter, 2003; felin et al., 2012; deken et al., 2016), where an organizational routine is a repetitive, australasian journal of information systems taj, kautz & bruno 2021, vol 25, selected papers from acis 2019 it-enabled ot as a coevolution process 3 recognizable pattern of interdependent actions, carried out by multiple actors (feldman & pentland, 2003). it also considers an it system as a collection of affordances and constraints (leonardi, 2011). a technology affordance refers to an action potential, that is, what actors can do with an it system in their work routines whereas a constraint addresses the way an it system may be holding back the actors in achieving their goals (leonardi, 2011; seidel, recker & brocke, 2013; verhulst & rutkowski, 2017). the purpose of defining it-enabled ot as a coevolution process is to investigate how the new it system shapes organizational routines, as well as how the evolution of routines shapes the features and functions of the new it system (leonardi, 2009). unlocking the coevolution process of organizational routines and a new it system, allows us to study it-enabled ot comprehensively, by combining the five social constructivist perspectives in a systematic manner. the conceptual framework provides a new approach to study the phenomenon of it-enabled ot holistically, by enabling research to investigate how actors perceive, interpret, appropriate, and enact the new it system in their work routines as well as how they align the new system and these routines with the social order and the structures of the organization. this enhances our understanding of how change unfolds in an organization during the implementation and use of a new it system. the next section describes our approach for reviewing the existing is literature on it-enabled ot. we then present, in the third section, the theoretical background of it-enabled ot particularly highlighting the five social constructivist perspectives on it system implementation and the concept of organizational routines. the fourth section then conceptualizes it-enabled ot as a coevolution process of organizational routines and a new it system. the fifth section discusses and justifies our ontological position with regard to the proposed framework to study it-enabled ot. finally, the sixth section summarizes our work and concludes with a discussion of limitations and a call for future research to empirically validate and refine the proposed framework. 2 a hermeneutic approach for conducting the literature review we used a hermeneutic approach to review the existing is literature and to understand how the concept of it-enabled ot evolved over the last decade as a basis for the development of a framework for it-enabled ot. the hermeneutic approach is an iterative approach for conducting a literature review, which includes the following phases: searching, reading, mapping and classifying, critical assessment, and argument development (boell & cecezkecmanovic, 2014). in the first hermeneutic cycle, we identified four paradigms of it-enabled ot have been identified: a) planned change models b) technological imperative c) punctuated equilibrium (pe) models d) social constructivist perspectives (orlikowski, 1996; leonardi & barley, 2010). in the second hermeneutic cycle, we dismissed the first 3 paradigms because they do not consider emergent change, which is relevant in today organizations (orlikowski, 1996). the third hermeneutic cycle then focused on the social constructivist perspectives: perception, interpretation, appropriation, enactment, and alignment as distinguished by leonardi and barley (2010). these authors also identified two major issues in the application of constructivist perspectives to investigate the phenomenon of it-enabled ot. first, different constructivist perspectives are associated with the construction of different social processes, and hence operate at different levels of analysis (leonardi & barley, 2010). second, extant studies based on social constructivist perspectives are not considering the evolution of the new it system itself in the implementation process. to overcome these issues, the fourth hermeneutic cycle focused on the coevolution of organizational routines and a new it system australasian journal of information systems taj, kautz & bruno 2021, vol 25, selected papers from acis 2019 it-enabled ot as a coevolution process 4 (pentland & feldman, 2005; d’adderio, 2008; d'adderio, 2011), which resulted in the development of a new conceptual framework to study the phenomenon of it-enabled ot in a systematic manner. 3 theoretical background 3.1 four theoretical perspectives on it-enabled ot the theoretical perspectives on it-enabled ot can be classified into four paradigms (orlikowski, 1996; leonardi & barley, 2010): a) planned change models b) technological imperative c) punctuated equilibrium (pe) models d) social constructivist perspectives. ot paradigms attributes planned change models technological imperative punctuated equilibrium models social constructivist perspectives philosophical stance voluntarism determinism social constructivism change enablers actors it actors actors’ interaction & it change type small or big/ short or long episodic & radical continuous & cumulative change process sequential & time bound sequential & time bound non-linear & ongoing notion of emergence no no no yes table 1. the paradigms of organizational transformation the planned change model’s paradigm presumes that people deliberately initiate and implement changes to improve the performance of the organization and make it fit with the environment (dunphy & stace, 1988). planned change models are criticized for their high reliance on the rationality of managers directing the change and treating it as a separate event from the ongoing processes of organizing (orlikowski, 1996). the technological imperative paradigm considers an it system as the main driver of organizational change, so that the adoption of a new it system creates predictable changes in the structure and work practices of an organization (huber, 1990). the technological imperative paradigm is mainly criticized for ignoring the role of actors to explore, learn and adjust to different organizational situations. the punctuated equilibrium models paradigm posits that organizations are evolving through relatively long periods of stability in their basic patterns of activity, which are punctuated by relatively short periods of revolutionary change (gersick, 1991, p. 12; romanelli & tushman, 1994). the punctuated equilibrium models paradigm is mainly criticized for its assumptions of moving the organization to some sort of stable state or equilibrium state, whereas contemporary organizations are characterized by frequent change, learning, flexibility and self-organizing (orlikowski, 1996). the planned change models, technological imperative and punctuated equilibrium models paradigms do not consider the phenomenon of emergent change (orlikowski, 1996; leonardi & barley, 2010). emergent change is relevant because unprecedented environmental, technological and organizational developments facilitate patterns of organizing which cannot be prescribed by prior plans or intentions. the social constructivist perspectives paradigm uncovers the notion of emergent change during the implementation and use of a new it system. scholars who study it-enabled ot from constructivist perspectives generally hold that organizational change emerges from an australasian journal of information systems taj, kautz & bruno 2021, vol 25, selected papers from acis 2019 it-enabled ot as a coevolution process 5 ongoing stream of social actions in which actors respond to an it system’s affordances and constraints (leonardi & barley, 2010). the introduction of a new it system triggers the process of organizational transformation, but the actual change happens over a period of time through situated actions and ongoing improvisations (orlikowski, 1996). the focus of this paradigm is that change may not always be as planned, inevitable, or discontinuous as we expect; rather it is realized through ongoing variations and adjustments when the actors try to accommodate a new it system into their work routines (hutchins, 1991; orlikowski, 1996; leonardi & barley, 2010). table 1 provides a summary of these 4 paradigms. 3.2 five social constructivist perspectives on it-enabled ot leonardi and barley (2010) have approached it-enabled ot from five coherent social constructivist perspectives: perception, interpretation, appropriation, enactment, and alignment to explain the social construction of a newly implemented it system. they put forward that the perspectives focus on different phases of the implementation of a new it system. they also contend that these perspectives build a comprehensive approach to study how an organization and an it system interact with each other in creating a new social reality in the organization. the perception perspective focuses on adoption and seeks to explain why actors come to share similar perceptions about the usefulness of a new it system. the perception perspective also investigates how shared perceptions lead to the acceptance or rejection of the new it system (leonardi & barley, 2010). the social construction of the new it system occurs through the convergence of attitudes, values, and beliefs among the actors of a new it system (vishwanath, 2006). the social construction process ceases once actors have made an initial decision to adopt the new it system (rice & aydin, 1991; leonardi & barley, 2010). the decision to continue further use of the new it system depends on users’ evaluation of the it system’s performance. perception interpretation appropriation enactment alignment implementation phase of new it system adoption use use use adaptation social construction convergence of attitudes, beliefs, and values transference of knowledge & modification of schemas emergence of patterns of deviations & conformity situated improvisations & evolution of work practices reconfiguration of roles & relationships table 2. the social constructivist perspectives of organizational transformation (adapted from leonardi & barley, 2010) the interpretation perspective asks how people draw on familiar frames to make sense of and interpret the new it system in use in unfamiliar situations (orlikowski & gash, 1994; leonardi & barley, 2010). according to orlikowski and gash (1994), frames are built up repertoires of tacit knowledge that an actor uses to impart meaning in an ambiguous situation. frames implicitly guide actors to make sense of uncertain situations and then act accordingly. the social construction involves the transfer of knowledge and the modification of previously held frames to cope with new situations during the implementation and use of a new it system (hsiao, wu, & hou, 2008). australasian journal of information systems taj, kautz & bruno 2021, vol 25, selected papers from acis 2019 it-enabled ot as a coevolution process 6 the appropriation perspective investigates whether people use the new it system as it was designed and intended by the designers (leonardi & barley, 2010). according to poole and desanctis (1990), the design of an it system has two aspects: a) spirit, the general goals, and attitudes that the it system aims to promote; b) structural features, the technical features and functions of the it system. the designers inscribe affordances and constraints into an it system to encourage certain patterns of actions and behaviours (d'adderio, 2011). actors appropriate the features and functions of the new it system consistent or inconsistent with the designers’ intentions (leonardi & barley, 2010). the appropriation perspective is silent and gives rare explanations of the situated accounts why actors deviate and use an it system in an unanticipated manner (leonardi & barley, 2010), which is the focus of the enactment perspective. the enactment perspective studies how actors use a new it system in their work but focuses on the evolution of work practices rather than cognitions or norms (leonardi & barley, 2010). weick (1979) introduced the concept of enactment into organization studies as a way of emphasizing the idea that organizing is an activity and that organizational actors wittingly and unwittingly craft organizations as they try to make sense of and respond to their environments. the enactment perspective narrates how the institutional embedding of an it system into the social system takes place (berente et al., 2008). the enactment perspective aims at identifying micro-level processes, which translate the working logic of the new it system into the work practices and routines of the implementing organization (orlikowski, 1996; leonardi & barley, 2010). the enactment perspective ignores the implications of macro-level structures with respect to the use of the new it system, which is the focus of the alignment perspective (leonardi & barley, 2010). the alignment perspective focuses on the adaptation of an it system, in which the social order of an organization and the newly implemented it system reconfigure and adjust to each other (leonard-barton, 1988; sabherwal, hirschheim, & goles, 2001). the alignment perspective investigates how the social order of an organization shapes the use of a new it system and how the use of a new it system alters an existing social order (leonardi & barley, 2010). the central theme of the alignment perspective is to examine how roles and relationships change during the implementation of a new it system (barley, 1990). the social constructivist perspectives are summarized in table 2. as stated above, is scholars have been facing two major challenges when applying social constructivist perspectives to study the phenomenon of it-enabled ot. first, different constructivist perspectives are associated with the construction of different social processes, and hence operate on different levels of analysis (leonardi & barley, 2010). the perception perspective focuses on the convergence of attitudes, beliefs, and values; the interpretation perspective focuses on the transfer of knowledge; the appropriation perspective focuses on the patterns of deviation and conformity; the enactment perspective focuses on the evolution of work practices; the alignment perspective focuses on the reconfiguration of roles and relationships (leonardi & barley, 2010). the investigation of it-enabled ot demands however to examine the construction and evolution of all social processes simultaneously (leonardi & barley, 2010; besson & rowe, 2012) as only then it-enabled ot can be understood in a holistic and integrated manner. so far only a few studies have combined some of the different social constructivist perspectives to investigate the phenomenon of it-enabled ot in australasian journal of information systems taj, kautz & bruno 2021, vol 25, selected papers from acis 2019 it-enabled ot as a coevolution process 7 a comprehensive manner (orlikowski, 1996; leonardi, 2007; leonardi & barley, 2010; besson & rowe, 2012). second, extant studies based on social constructivist perspectives are skewed towards studying social processes during the implementation, and in particular the use of a new it system (leonardi & barley, 2010). it is however also important to explore how the new it system itself evolves during the implementation process (leonardi, 2009). these issues can be resolved to a great extent if we conceptualize it-enabled ot as a coevolution process of organizational routines and a new it system (pentland & feldman, 2005; d’adderio, 2008; d'adderio, 2011). this will enable research to holistically study the five social constructivist perspectives in a single organizational setting. 3.3 the concept of organizational routines organizations accomplish much of their work through executing established routines (feldman, 2000), where an organizational routine is a repetitive, recognizable pattern of interdependent actions, carried out by multiple actors (feldman & pentland, 2003). organizational scholars investigate routines from two different perspectives, the capability, and the practice perspective. the capability perspective focuses on routines as whole entities whereas the practice perspective focuses on the parts of routines (parmigiani & howardgrenville, 2011). the capability perspective is mainly concerned with the purpose and impact of routines on organizational performance (dosi, faillo, & marengo, 2008) whereas the practice perspective focuses on how organizational routines operate and how they are (re)produced or changed as people enact them (parmigiani & howard-grenville, 2011). the practice perspective is grounded in practice theory, which describes how everyday practices are accomplished, reinforced, or changed (feldman & orlikowski, 2011). practices are the recurrent, materially bounded and situated actions engaged in by actors of the organization (orlikowski, 2002; vaast & walsham, 2005). bourdieu (1990) defines a practice as shared actions performed by a group of actors. at the most micro-level, a gesture, and a speech act, are both practices (collins, 1981). they are microsocial actions through which work is carried out. at the meso-level a practice consists of multiple microsocial actions that together constitute a socially recognizable practice (orlikowski, 2000). negotiation and interaction among actors represent practices which occur at a meso-level (barley, 1986). practices can be conceptualized as the aggregate of meso-level actions into macrosocial categories such as an organizational routine (feldman & pentland, 2003). for example, the hiring routine can be seen as a combination of meso-level actions such as screening applicants, conducting interview, and negotiating contracts. the emphasis of the practice perspective is to uncover the internal working of routines (feldman & pentland, 2003) to open the black box of an organizational routine and to reconceptualise it as being made of two interacting parts its ostensive aspects and its performative aspects. the ostensive aspect represents the ideal or schematic form of the routine. it is the routine in principle (feldman 2003; feldman & pentland, 2003). the performative aspect captures specific actions, by specific people, at specific times and places (feldman & pentland, 2003). it is the routine in practice. initially, when a routine is put into practice the actors use the ostensive aspect to guide their actions, refer to the pattern of interactions and evaluate the performance outcomes (feldman & pentland, 2003). when the routine does not produce the outcome as expected actors bring about variations in the performance of the routine through adjustments, accommodations, and improvisations australasian journal of information systems taj, kautz & bruno 2021, vol 25, selected papers from acis 2019 it-enabled ot as a coevolution process 8 (feldman & orlikowski, 2011). this causes the actors to modify and recreate the ostensive aspects of the routine. hence, there are recursive relationships between the two aspects; both are mutually constitutive and essential for the development, evolution, and adaptation of organizational routines (feldman & pentland, 2003; pentland & feldman, 2005). organizational routines act as a key mechanism to capture change in the organization by investigating how change is occurring to the underlying practices in an organizational routine (nelson & winter, 1982). variation and selective retention of patterns of actions bring change to the performance of an organizational routine. pentland and feldman (2005) conceptualize an organizational routine as a generative system with internal structure that can produce a wide variety of performances depending on the contextual situations. examining the internal structure of a routine involves considering the interactions between ostensive aspects of a routine, performative aspects of a routine, and the artifacts that play a fundamental role in influencing both the aspects of a routine (d’adderio, 2008). uncovering the interactions between ostensive aspects, performative aspects and artifacts provides a sound foundation to understand change in an organization (d’adderio, 2011). it systems as artifacts are key in the production and reproduction of routines; they directly influence the rate and direction of change in the organizational routines and therefore their evolution and adaptation (d’adderio, 2011). 4 it-enabled ot as a coevolution process 4.1 a conceptual framework for it-enabled ot as a coevolution process it systems constitute an important category of technological artifacts and play an important role to accomplish work in contemporary organizations (d’adderio, 2011). they enable as well as constrain the performance of organizational routines (d’adderio, 2008) and enrolled as artifacts in the composition of an organizational routine, play a fundamental role in its creation and evolution (pentland & feldman, 2008). figure 1, adapted from pentland and feldman (2008), depicts how artifacts, of which it systems are an instance, are represented by and influence both the ostensive and performative aspects of an organizational routine. as stated above the ostensive aspects of an organizational routine as partly inscribed in an it system enable and constrain the performative aspects of that routine which in turn create and recreate its ostensive aspects. according to orlikowski (2002), it systems as technological artifacts embody affordances and constraints to structure work, extend interactions, increase visibility of information and actions, enable, and constrain the ability of actors, regulate access to resources, and facilitate knowledge sharing and accumulation. affordances and constraints are inscribed into an it system in the form of features and functions to encourage and impose certain patterns of actions and behaviours in the organization (grint & woolgar, 2013). the notion of inscription indicates that an actor’s assumptions, intentions and working logic are embedded in the it system (latour, 1992) which directly influence the organizational routines in which they are involved. this means that an organizational routine is not simply actor-embodied but instead distributed across actors and it systems (hutchins, 1995). hence, an it system also acts as a model or a partial representation of organizational routines, containing selective and codified configurations of the ostensive aspects of routines as their artifactual representation (d'adderio, 2011). australasian journal of information systems taj, kautz & bruno 2021, vol 25, selected papers from acis 2019 it-enabled ot as a coevolution process 9 figure 1. the relation of an it system and an organizational routine (adapted from pentland & feldman, 2008) the concept of artifactual representation enables researchers to investigate change in the organization by unlocking the coevolution process of organizational routines and the newly implemented it system. d'adderio (2011) argues that uncovering the reciprocal interactions between both aspects of organizational routines and an it system is key to understand how routines and an it system coevolve with each other in it-enabled ot. the interactions between ostensive aspects of routines and an it system are critical to be investigated to understand how working logic and abstract patterns of actions are inscribed into an it system (pentland & feldman, 2005; d’adderio, 2008) and how the it system shapes the two aspects. this enhances the understanding of how change is occurring to the it system in the coevolution process (goh et al., 2011). studying the interactions between the performative aspects of routines and an it system is critical to understand how actors use the it system when performing organizational routines and how an it system shapes the performances of such routines (pentland & feldman, 2005; d’adderio, 2008). this enhances the understanding of how change is occurring to the organizational routines during the implementation, adoption, use, and adaptation of a new it system. figure 2 depicts the coevolution process of organizational routines and a newly implemented it system, adapted from d’adderio (2011) and lehrig et al. (2015). it is a refinement of pentland and feldman’s (2008) model shown in figure 1. the coevolution process is triggered as the actors start using a new it system. when actors do not get the desired performance from the use of the new it system, they either change the organizational routines or the new it system. the perception of the system’s affordances triggers actors to alter their work routines while the perception of constraints leads the actors to request the modification of the new it system (leonardi, 2011). however, as resources and australasian journal of information systems taj, kautz & bruno 2021, vol 25, selected papers from acis 2019 it-enabled ot as a coevolution process 10 efforts are required to change the it system while in operation (d’adderio, 2008) most of the time actors try to accommodate the new it system in their work routines through interpretation, appropriation, and enactment (leonardi, 2011). through interpretation, the actors draw on past technological frames to understand and use the new it system in ambiguous situations (orlikowski & gash, 1994). in appropriation, the actors use the new it system in a different way and choose its features for the purposes other than what were the designers’ intentions (watson et al., 1988). in enactment, the new it system is locally adopted through situated, planned as well as unplanned actions (orlikowski, 1996). the appropriation and enactment of a new it system not only affect the individual routines but also shift and reconfigure the roles and relationships in between interconnected routines and the new it system (leonardi & barley, 2010; goh et al., 2011). these shifts of roles and relationships between routines, and between routines and the new it system impact the social order and structures in an organization and actors align both routines and the it system with the possibly changed social order and the structures of the organization. figure 2. coevolution of routines and an it system (adapted from d’adderio, 2011; lehrig et al., 2015) figure 3 below is a further extension of figure 2 and incorporates the five social constructivist perspectives. it depicts our conceptual framework and emphasizes their role for understanding it-enabled ot as a coevolution process of organizational routines and a new it system. australasian journal of information systems taj, kautz & bruno 2021, vol 25, selected papers from acis 2019 it-enabled ot as a coevolution process 11 figure 3. a framework of it-enabled organizational transformation the framework understands an organization as a collection of routines, where a routine is a repetitive, recognizable pattern of actions, carried out by multiple actors. an organizational routine is composed of two interacting parts its ostensive aspects and its performative aspects. the ostensive aspects of routines are abstract patterns of actions, which are partially also inscribed into a new it system. the performative aspects are the specific actions, taken by the specific actors to accomplish work with the use of a new it system. the it system as a technical artifact influences and mediates both the ostensive and performative aspects of organizational routines. the it system embodies affordances and constraints in the form of features and functions to encourage and impose certain patterns of actions in the organization. the it system as a collection of affordances provides new possibilities of actions and enables the actors through its use to shape the performance of their work routines. the actors accommodate the features and functions of a new it system by altering and adjusting the australasian journal of information systems taj, kautz & bruno 2021, vol 25, selected papers from acis 2019 it-enabled ot as a coevolution process 12 actions in their work routines. problems and issues emerge during the use of a new it system, which (re)shape the understanding of the actors about their work routines. if the new it system does not produce the expected outcome actors bring about variations in the performance of the routines through adjustments, accommodations, and improvisations. this enhances the understanding of the actors and causes them to (re)create the ostensive aspects of their work routines. the use of a new it system not only enables new possibilities of actions but also constrains the actors to carry out certain actions. to overcome the constraints of a new it system the actors have two options: either to change the work routines or to change the functions and features of the newly implemented it system. in case of the first option, the actors manoeuvre the use of a new it system and use it in an unintended manner whereas in the 2nd case new features and functions are requested to be inscribed in the it system. the framework further elaborates the reciprocal interactions between different aspects of organizational routines and a new it system from the five social constructivist perspectives. to understand the early phase of it system implementation, the framework focuses on how the actors perceive the features and functions of a new it system in the work routines. uncovering the perception of a new it system is key to understand why some features and functions are adopted while others are rejected. the perception of a new it system influences the adoption decisions, which have implications for the interpretation, appropriation, and enactment of a new it system. the framework explores how actors make sense of and interpret the new it system in their work routines. in ambiguous situations or in the absence of adequate training the actors interpret a new it system in their work routines in terms of familiar it systems that they have worked with in the past. uncovering the interpretation of an it system is important because it affects the ways through which a new it system is to be appropriated and enacted by the actors. the framework also comprises an analysis of the appropriation of a new it system and particularly focuses on how and why actors deviate from the original design of a new it system. the choice of using a new it system either in intended or unintended manner causes the actors to follow different actions paths and influence the evolution of organizational routines. along with appropriation, the framework also uncovers the enactment of a new it system to understand how actors actually embed the new it system in their work routines through planned as well as unplanned actions. as unexpected and emergent events appear during the enactment process then how actors cope with such situations and how they adjust the actions in the work routines is part of this perspective. new work practices emerge during the appropriation and enactment of a new it system, which challenge the existing roles and relationships of the routines and actors in the social order and the structures of the organization. this moves the focus of the framework to the adaptation phase to investigate how roles and relationships (re)align with the use of a new it system. if roles and relationships are altered and social order and structures are changed this perspective studies how actors then carry out further appropriation and enactment cycles to align the performance of the work routines with the new social order and new organizational structures. if roles and relationships remain unchanged the actors might use the new it system as is. the relationship between enactment and alignment is critical to be investigated to understand how the structure of the implementing organization is either enabling or constraining the coevolution of organizational routines and the new it system. australasian journal of information systems taj, kautz & bruno 2021, vol 25, selected papers from acis 2019 it-enabled ot as a coevolution process 13 our proposed conceptual framework offers a mechanism to explore the complex interactions between the different aspects of organizational routines and a new it system in a systematic manner. the framework allows to understand it-enabled ot as a coevolution process of organizational routines and a new it system in a holistic and integrated manner. 4.2 understanding it-enabled ot as a coevolution process through the proposed framework in the previous section we developed and explained our framework largely based on theoretical considerations. in the following we illustrate its scope as well as its conceptual and analytic strength with selected empirical examples from the research literature on it-enabled ot. these examples mostly focus on individual perspectives and aspects of the ot process, sometimes covering and combining several of them, but none is based on a comprehensive framework such as ours which further underpins the need for a holistic approach to studying and achieving a deeper understanding of the phenomenon. when a new it system is introduced in an organization, actors neither fully employ the newly implemented it system in their work routines nor do they totally by-pass the new it system. boudreau and robey (2005) found in a study of erp system implementation in a us government organization that actors initially were not willing to break their old routines and avoided the new it system as much as possible. later, after the organizational routines and the new it system had passed through a process of coevolution the erp system was adopted. goh et al. (2011) presented an account of the coevolution of organizational routines and an it system in their study of a computerized documentation system (cds). the implementation of the cds replaced paper-based patient charts/notes with digital notes in the ‘rounding routine’ in a healthcare organization. the cds enabled physicians to view the patient documentation from anywhere and to give advice remotely rather than physically having to go to the patient’s bed to check the treatment chart. the implementation of cds shaped the performative aspects of the ‘rounding routine’ in the healthcare organization. the digital notes functionality of the cds provided new possibilities for actions but also constrained the actors to carry out certain actions. for instance, no field was available in the cds to record the communication between physicians which had been part of their work practices and the existing social order and structure in the organization. this issue shaped and (re-)created the understanding of this ostensive aspect of the ‘rounding routine’ as it was reported to the technical team and a new feature was inscribed in the cds accordingly. with the ostensive aspects of the routine partially inscribed in the it systems the cds also enabled the physicians to get updated information about a patient by simply clicking on various sections of an etreatment chart as part of the ‘rounding routine’. the new pattern of actions was perceived as a significant improvement over the paper-based reporting in terms of efficiency and decision making. however, the physicians proposed further enhancements to get a consolidated view of all patient information in one place by using a single click. the need of the aggregated patient report (re)shaped the actors’ understanding of the ostensive aspects of the ‘rounding routine’, and hence they requested and received a new feature to be inscribed in the cds which subsequently further shaped the performative aspects of this routine. this example shows that unlocking the coevolution process of organizational routines and an it system is key to understand it-enabled ot by investigating how actors perceive, interpret, appropriate, and enact the new it system in their work routines, as well as how they align the australasian journal of information systems taj, kautz & bruno 2021, vol 25, selected papers from acis 2019 it-enabled ot as a coevolution process 14 new system and these routines with the social order and structure of the organization (pentland & feldman, 2008; leonardi & barley, 2010; d’adderio, 2011). as actors are not able to achieve their expected results from the use of a new it system, they have two options: either to change their work routines or to request and get a change of the functions and features of the newly implemented it system. we explained above that the perception of the new it system as a collection of affordances triggers actors to alter their work routines to make fuller use of the new it system (leonardi, 2011) and that the perception of the new it system as a bundle of constraints leads the actors to ask for changes to be made to the technical features and functions of the new it system (leonardi, 2011). goh et al. (2011) argue that the healthcare staff in their case study perceived the new it system to afford their work further by providing new possibilities of actions. this perception caused the healthcare staff to adjust their work routines to accommodate the features and functions of the new it system. on the other hand, the physicians perceived that the new it system constrained their work routines and asked for the development of new features and functions to be included in the it system. the perception of affordances and constraints leads actors to either change organizational routines or the it system, but research suggests that often organizational routines are changed (leonardi, 2011) underlining the importance of the perception perspective. as stated earlier this is due to the inflexibility of an it system to be changed during operation and because resources and efforts are required to modify, test, and deploy changes to a new it system (d’adderio, 2008). during the use of an it system organizational actors employ three interrelated techniques, namely interpretation, appropriation, and enactment (leonardi & barley, 2010) which (might) lead to both modifications of the organizational routines and of the it system. through interpretation, actors make sense of the new it system by drawing on technological frames imported from other domains (hsiao, wu, & hou, 2008). the actors attempt to situate their thoughts about actions in terms of familiar it systems that they had worked with in the past. for instance, orlikowski and gash (1994) studied the implementation of lotus notes in a consulting firm. the consultants interpreted the new it system in light of the existing applications, which they were using individually. as a consequence, they utilized the new it system for accomplishing individual tasks like e.g., sending email in contrast to the intended purpose of the system to facilitate and enhance the collaboration among them. this example shows how the interpretation of a new it system and of its interplay with the ostensive and performative aspects of the organizational routines affects the ways in which a new system, in this case contrary to its original purpose, is appropriated and enacted by the actors (leonardi & barley, 2010). in appropriation, the actors chose whether to use the new it system as designed or to deviate from the it system by using its features and functions for purposes other than those the designers intended (watson, desanctis, & poole, 1988). leonardi (2007) studied a group of technicians for the first five months using a new it system, an information technology service management (itsm) tool, in a large us government organization. the case study revealed that the technicians carried out three rounds of technology appropriations to accommodate the new it system into the ostensive and performative aspects of their work routines. in the first round of appropriation, the technicians used minimum features of the new it system to record the initiation and resolution of the service tickets. in the second round, the it system was not just used for recording tickets but also for assigning tickets to the experienced technicians australasian journal of information systems taj, kautz & bruno 2021, vol 25, selected papers from acis 2019 it-enabled ot as a coevolution process 15 across different departments. in the third round, the technicians realized the need to write problem resolution outlines before closing a ticket. the technicians noticed that the it system was lacking such documentation functionality which was an important part of their organizational work routines. this was overcome by utilizing an unused field in the data entry form, where technicians added a short paragraph about the steps, they had taken to solve a user problem. this example shows that the appropriation did not require any change to the it system but was accomplished through the use of the new it system in a different way to accommodate for the routines which in that process (re)created and shaped the ostensive and performative aspects of their routines. this leads us to importance of the enactment perspective. in enactment, the actors adjust the patterns of actions of the existing routines to accommodate a new it system (orlikowski, 1996; pentland & feldman, 2008). the working logic of the new it system is locally adopted through situated planned actions and improvisations (orlikowski, 1996; lyytinen et al., 2009). orlikowski (1996) showed in her case study of a customer support system that organizational transformation occurred as a result of planned as well as unplanned enactment of a new it system for support staff. she argues that staff enacted the new it system by adding new actions to their work routines to fully accommodate the newly implemented it system. for instance, the support staff included new actions in their work routines to electronically record customer complaints in the it system while they were on the phone with the customers. she also noted that emergent situations appeared during the enactment of planned changes. for instance, when a data entry screen was incompatible with the way the customers reported complaints, the actors further adjusted their work routines to enact a changed reporting process. they wrote some information on paper while simultaneously talking to the customers and recording their complaints in the on-line system demonstrating the mutual impact of routines and it system on each other. the unanticipated events, breakdowns, and contingencies also provided opportunities for learning, which enabled the actors to improvise their work routines. during the interpretation, appropriation and enactment of a new it system, actors go through a process of trial-and-error learning (rerup & feldman, 2011). in the above case when unpredicted problems emerged, solutions were devised to fix them accordingly. in this respect the three phases are inherently interrelated. when actors are unable to resolve problems, themselves they approach experienced colleagues, who may quickly resolve such emerging issues based on their knowledge as they may have encountered similar kinds of technical problems in comparable settings (aime et al., 2010). experienced actors are instrumental in spreading relevant knowledge in communities of it users through informal means such as sharing tips and ad hoc documentations (boudreau & robey, 2005). this type of learning is called situated learning; it helps the user community to smoothly enact the new it system (lave & wenger, 1991; brown & duguid, 1991). boudreau and robey (2005) in their before mentioned study of the implementation of an erp system without explicitly mentioning the three perspectives found that actors initially chose to avoid using the new it system. the authors report that later on, through situated learning, actors increased their understanding about the new it system and eventually progressed toward the appropriation and enactment of the new it system. situated learning as an important facet in the interplay of interpretation, appropriation and enactment is not only helpful for the evolution of organizational routines as demonstrated by australasian journal of information systems taj, kautz & bruno 2021, vol 25, selected papers from acis 2019 it-enabled ot as a coevolution process 16 boudreau and robey (2005) but also contributes to the evolution of the new it system as the study by goh et al. (2011) showed and thus plays an important role in the coevolution of organizational routines and a newly implemented it system. during the design of a new it system affordances and constraints are inscribed into the system but are only instantiated when the actors appropriate and enact the features and functions of the new it system (d’adderio, 2011). problems, issues, breakdowns, exceptions, and unanticipated events occur when the actors start accommodating the new it system in the existing work routines (orlikowski, 1996). these problems not only provide opportunities for actors to reflect on their routines and adjust them according to the new it system but also reflect on the affordances and constraints of the new it system (d’adderio, 2011). the actors provide valuable feedback and suggestions to further improve the features and functions of the new it system reinforcing the importance of both taking appropriation and enactment into account when studying the coevolution of organizational routines and a newly implemented it system. new patterns of actions emerge in the organizational routines during the appropriation and enactment of a new it system (leonardi & barley, 2010). the ensuing change challenges the existing roles and relationships in the organization (barley, 1990; schultze & orlikowski, 1994). at this point, the implementation and use of a new it system demand the reconfiguration of roles and relationships to align the new patterns of actions with the structure and social order of the organization (leonardi & barley, 2010). this reinforces the innate relationship between use and adaptation, in other words the relation between the appropriation, the enactment and the alignment perspectives. for instance, barley (1990) studied technicians in two radiology departments that had just acquired their first computerized tomography (ct) scanner. he showed that the new technology flattened the organizational structure by redefining the roles and relationships of both technicians and radiologists. if the new patterns of actions do not demand the restructuring of roles and relationships, then the use of a new it system is unlikely to trigger any significant change in the organization (leonardi & barley, 2010). in this case, existing work practices are re-enforced, and the actors may use the new it system in ways that maintain the status quo. for instance, edmondson et al. (2001) studied the implementation of a new it system at sixteen hospitals and found that change did not happen in those hospitals where work practices were replicated and the new it system supported and reinforced the traditional roles. roles and relationships evolve during the adaptation phase, in which the social order and structures of an organization and the newly implemented it system reconfigure and adjust to each other (leonard-barton, 1988; sabherwal, hirschheim, & goles, 2001). orlikowski’s (1996) study of the organizational transformation of a software company provides an example of the relationship and impact of the enactment and the mutual alignment of it systems, organizational routines, and organizational structures in it-enabled ot. likewise, leonardi (2007) in the above cited study of the implementation of an itsm showed how roles and relationships changed after the technicians had gone through three cycles of appropriation and enactment. in the first cycle, the functionality of the itsm was constructed as an incident tracking tool. in this phase, the technicians sought help from their seniors in their respective departments according to hierarchical positions. the advice seeking mechanism was exactly the same as before the it system implementation. in the second cycle, the it system was appropriated and enacted as a project assignment tool. the it system was not just used for australasian journal of information systems taj, kautz & bruno 2021, vol 25, selected papers from acis 2019 it-enabled ot as a coevolution process 17 recording a ticket but also to assign tickets to other technicians by locating them in the it system. the actors thus sought help from many people across different departments. in the third cycle, the actors realized that the itsm lacked details about the solutions of the problems, and hence the it system was reconstructed as a documentation tool. the actors started to approach those technicians, who explicitly documented their expertise in the system, and hence advice was sought not only from senior technical experts but often also from junior people across the organization. the coevolution process of organizational routines and a new it system is a complex and emergent process, and organizations often fail to manage it in an effective manner (goh et al., 2011). our proposed conceptual framework appreciates the coevolution process in a systematic manner by explaining how actors perceive, interpret, appropriate, and enact the new it system in their work routines in a situated learning process and how they align it with the social order and structures of the organization. it also explains how roles and relationships evolve during the implementation of a new it system. it allows us to capture the interplay of social and technical dynamics that causes change in the organization. our framework as depicted in figure 3 provides a new approach to investigate the phenomenon of it-enabled ot as an instance of dt in a holistic and integrated manner and to study how change unfolds in an organization as a coevolution process of organizational routines and it systems during the implementation, with its phases adoption, use, and adaptation, of a new it system. 5 the ontological positioning of the proposed framework our framework is firmly grounded in a social constructivist perspective on it-enabled ot. beyond leonardi and barley’s (2010) distinction of five social constructivist perspectives focussing on the transformative practices of perception, interpretation, appropriation, enactment, and alignment, social constructivist research can have various ontological roots (putnam, 2015). we therefore complete the presentation of our proposed framework with a clarification and discussion of its ontological positioning. as a reaction to technological determinism which emphasized the technical over the social by framing technology as the key driver of organizational change (noble, 1978; huber, 1990) social constructivism brought the social to the fore (fulk, 1993; zuboff, 1988). to refute technological determinism, social constructivists’ studies downplayed the role of the technology in the social construction process (leonardi & barley, 2010). at the most, scholars mention that an it system activates social processes, which then construct the social phenomenon without further elaborating the technical dynamics in the process of organizing. orlikowski and scott (2008) have criticized constructivist studies and advocate that researchers should adopt sociomateriality for the study of technology use in the organizations. both orlikowski and scott (2008) and leonardi (2012) put forward that sociomateriality refers to that materiality, in our case the material features and functions of an it system, gets meaning and has effects when enmeshed in a social phenomenon. orlikowski and scott (2008) go further and argue that materiality is not an intermittent aspect of organizational life but rather an integral part of organizing. they consider that both the social and the material are ontologically indistinct and do not exist independently, which means that there is no social that is not also material, and no material that is not also social (orlikowski, 2007). in line with orlikowski and scott (2008) we recognize that materiality is integral to organizing. our proposed framework is thus aligned with their stance, but we conceptualize the social australasian journal of information systems taj, kautz & bruno 2021, vol 25, selected papers from acis 2019 it-enabled ot as a coevolution process 18 and the material as ontologically distinct. our conceptual framework is based on the five social constructivist perspectives identified by leonardi and barley (2010), which inherently bring the social to the foreground. we are considering an it system as a collection of affordances and constraints and focus on its evolution in the implementation process (leonardi, 2011). in contrast to technological determinism, which favours the material over the social or conventional social constructivism that favours the social over the material, we aim at investigating the integration of both the social and the material as suggested by misa (1994) and leonardi and barley (2008) to resolve the inadequacies of technological determinism as well as the shortcomings of applying the individual perspectives of social constructivism. our conceptual framework emphasizes both the social and the material aspects of an it system implementation and theorizes it-enabled ot as a coevolution of organizational routines and a new it system. the aim of defining it-enabled ot as a coevolution process is to investigate how the it system shapes routines, as well as how routines shape the new it system (leonardi, 2009). in contrast to orlikowski and scott’s (2008) approach to sociomateriality, our conceptual framework an approach that jones (2014) would call weak sociomateriality considers that both the social and material are interdependent but ontologically distinct for several reasons. according to putnam (2015) there are five different approaches to the analysis of the relation between the social and the material that have social constructivist roots. based on a comparison of an approach’s ontological position, its epistemological stance, its focal point of entry of the analysis, its understanding nature of the relationship, and of its way of managing the dialectic in the relationship, she distinguishes a) performativity b) foucauldian analysis c) imbrication d) plenum of agencies, and e) mangle of practice as approaches to study sociomateriality. the performative approach to sociomateriality developed by orlikowski and scott (2008) considers the social and material as inseparable and indistinct, while the others consider the material and social as ontologically distinct but interdependent and intertwined. our conceptual framework is closely related to the mangle of practice (pickering, 1995; putnam, 2015) which posits that the social and the material are distinct, but emerge together, their relationship is co-constituted through what pickering (1995) calls a mangle and managed through a continual connection between the two. epistemologically, pickering (1995) gives primacy to the association between the two but does not ascribe to a relationality that assumes that the social and the material are inherently inseparable (orlikowski & scott, 2008). this resonates well with our approach to studying coevolution of organizational routines and an it system during its implementation – its adoption, use, and adaptation in pickering’s (1995) words the association of routines and it system. when an it system is implemented it disturbs the existing sociomaterial fabric of the implementing organization at the time when the new it system is moved into a live environment (leonardi & barley, 2010). according to goh et al. (2011) and to berente et al. (2016) studying it system implementation in general and it-enabled ot in particular requires to empirically unravel both the social and the technical and to look back in time to examine how the social has influenced the material features of an it system and its effects, and how the material has shaped the social. this provides the possibility to research the activities and practices that an it system lets actors perform in particular situations, to investigate what it does and does not let them perform, and to study the workarounds that actors have adopted to deal with in such situations (orlikowski, 1996). australasian journal of information systems taj, kautz & bruno 2021, vol 25, selected papers from acis 2019 it-enabled ot as a coevolution process 19 leonardi and barley (2010) argue that such data provide an opportunity to identify how the material and social have recursively influenced each other over time, and how the existing sociomaterial fabric of the organization has been developed from the start of the implementation to the actual status at the time of investigation (leonardi & barley, 2010). while orlikowski and scott (2008) conceptually emphasize the entanglement, and according to putnam (2015) the indistinctness, of the social and the material which leaves us with the challenge to analytically distinguish between them and to impose ontological boundaries for analytical purposes (introna, 2013), we follow leonardi and barley (2010) as well as goh et al. (2011) and berente et al. (2016). these authors advise to empirically unravel the social and the material in order to capture the social and technical dynamics distinctly and clearly to develop an understanding of how change unfolds in an organization during the implementation, in particular during the use of a new it system, and in our research how organizational routines and a new it system coevolve. as stated above, the purpose of our conceptual framework is to study the coevolution of organizational routines, its underlying practices, and a newly implemented it system at a meso-level (feldman & pentland, 2003). we are not proposing the framework to study the work practices at micro-level. positioning our framework to study at a meso-level has led us to combine the five social constructivist perspectives for understanding the phenomenon of it-enabled ot in a comprehensive manner. adopting orlikowski and scott’s (2008) approach to sociomateriality would require a focus on the practices at micro-level to analytically disentangle the relative contributions of the material and the social to understand the phenomenon of it-enabled ot. any research combining and studying it-enabled ot from all five social constructivist perspectives on that level of detail appears to be very challenging in practice. finally, we use the concept of organizational routine, as presented by pentland & feldman (2005, 2008), as a theoretical lens to connect and integrate the five social constructivist perspectives. the conceptualization of these authors treats an it system as a distinct artifact, which influences and shapes both the ostensive and performative aspects of an organizational routine (pentland & feldman, 2005, 2008; parmigiani & howard-grenville, 2011). such a treatment is not directly compatible with orlikowski and scott’s (2008) approach to sociomateriality. our immediate aim is not to extend pentland and feldman’s (2005, 2008) work on routines, but to use it for the analysis of the coevolution of organizational routines and it systems, thus our ontological positioning in the social constructivist perspective is justified. in doing so we follow putnam’s (2015) argument for preserving the dialectical relationship between the two by holding the tensions between them in continual interplay and focussing on how the social and the material are constituted in a dialectical relationship to each other and think about the two phenomena as she puts it as “ … empirically distinct, but mutually implicated; that is even though they may exist as an invisible whole, researchers need to examine them separately and dialectically to avoid privileging one pole over the other.” (putnam, 2015, pp. 706-707) 6 conclusion we are proposing a conceptual framework to theorize it-enabled ot as a coevolution process of organizational routines and a new it system. the conceptual framework provides a new australasian journal of information systems taj, kautz & bruno 2021, vol 25, selected papers from acis 2019 it-enabled ot as a coevolution process 20 approach to study the phenomenon of it-enabled ot in a holistic and integrated manner. it combines five constructivist perspectives on the social construction of an it system by explaining how actors perceive, interpret, appropriate, and enact a new it system in their work routines as well as how they align the new system and these routines with the social order and structures of the organization. it enables a holistic investigation of how the it system shapes organizational routines and how the evolution of routines shapes the features and functions of the new it system at the same time. such an investigation enhances the understanding of how change unfolds in an organization during the implementation, and its adoption, use, and adaptation phases of a new it system. the framework supports research that aims at exploring the reciprocal interactions between different aspects of organizational routines and a new it system, to understand how change is occurring to the work practices and to the structures of an organization. unravelling the interactions between performative aspects of routines and an it system, as well as unravelling the interactions between the ostensive aspects of routines and an it system deepens the understanding of how change is occurring to the routines in the organization and how change is occurring to the it system through revision of its features and functions. the conceptualization of it-enabled ot as a coevolution process of organizational routines and a new it system is a theoretical contribution to the existing is literature. here we have illustrated the scope and strength of the proposed framework through applying it to examples from research, which studied it-enabled ot. we suggest that future research empirically validates and further refines the proposed conceptual framework through indepth case studies. given that the conceptual framework is based on five social constructivist perspectives, we propose a pluralist methodological approach for the data collection and analysis (mingers, 2001). the pluralist approach as a theory building methodology leverages the power of a multi-perspectival inquiry to develop new theory from data (mingers et al., 2013; müller et al., 2017). as mingers (2001, p. 243) argues “the real world is ontologically stratified and differentiated, consisting of a plurality of structures that generate the events that occur (and do not occur). different paradigms each focus attention on different aspects of the situation, and so multimethod research is necessary to deal effectively with the full richness of the real world”. future case studies applying the proposed conceptual framework should e.g. jointly ask the following questions a) how does the perception of a new it system during its adoption influence its interpretation, appropriation, and enactment in an organization during its use? b) how does the interpretation of a new it system affect the ways in which the system is appropriated and enacted in the organizational routines? c) how are appropriation and enactment of a new it system related and how does this relationship influence the coevolution of organizational routines and an it system? d) how are appropriation and enactment related to the alignment of a new it system and how does this relationship influence the coevolution of organizational routines and an it system? e) how does the structure of the implementing organization enable or constrain the coevolution of organizational routines and an it system? the findings from such case studies will also be helpful for practitioners to understand and manage the coevolution process of organizational routines and a newly implemented it system in an effective manner. our conceptual framework is not without limitations. it supports the investigation of the phenomenon of it-enabled ot during adoption, use, and adaptation of a new it system first, australasian journal of information systems taj, kautz & bruno 2021, vol 25, selected papers from acis 2019 it-enabled ot as a coevolution process 21 after an implementation and deployment decision has been made. furthermore, it does not take into account, actions and events which occurred and decisions which were made during the design and development of the it system which may have implications for the it system to be adopted, used, and adapted in one way or another. it is also imperative to take into account the way a particular it system was chosen, why one it system was preferred over another, and the way an it system was designed, developed, and deployed in a certain manner. all these are important considerations which influence the social construction of a newly implemented it system, and hence the coevolution of organizational routines and an it system. future studies on it-enabled ot as a coevolution process should thus undertake research that extends the proposed framework and not only focuses on implementation and the dynamics of adoption, use, and adaptation, but adequately considers design and development dynamics to further increase the understanding of how change unfolds during it-enabled organizational transformation. references agarwal, r., gao, g., desroches, c., & jha, a. k. (2010). research commentary—the digital transformation of healthcare: current status and the road ahead. information systems research, 21(4), 796-809. aime, f., johnson, s., ridge, j. w., & hill, a. d. (2010). the routine may be stable but the advantage is not: competitive implications of key employee mobility. strategic management journal, 31(1), 75-87. barley, s. r. (1986). technology as an occasion for structuring: evidence from observations of ct scanners and the social order of radiology departments. administrative science quarterly, 78-108. barley, s. r. (1990). the alignment of technology and structure through roles and networks. administrative science quarterly, 61-103. berente, n., yoo, y., & lyytinen, k. (2008). alignment or drift? loose coupling over time in nasa's erp implementation. icis 2008 proceedings, 180. berente, n., lyytinen, k., yoo, y., & king, j. l. (2016). routines as shock absorbers during organizational transformation: integration, control, and nasa’s enterprise information system. organization science, 27(3), 551-572. besson, p., & rowe, f. (2012). strategizing information systems-enabled organizational transformation: a transdisciplinary review and new directions. the journal of strategic information systems, 21(2), 103-124. boell, s. k., & cecez-kecmanovic, d. (2014). a hermeneutic approach for conducting literature reviews and literature searches. communications of the association for information systems, 34(1), 12. bourdieu, p. (1990). the logic of practice. stanford university press. boudreau, m. c., & robey, d. (2005). enacting integrated information technology: a human agency perspective. organization science, 16(1), 3-18. brown, j. s., & duguid, p. (1991). organizational learning and communities-of-practice: toward a unified view of working, learning, and innovation. organization science, 2(1), 40-57. australasian journal of information systems taj, kautz & bruno 2021, vol 25, selected papers from acis 2019 it-enabled ot as a coevolution process 22 collins, r. (1981). on the microfoundations of macrosociology. american journal of sociology, 86(5), 984-1014. d’adderio, l. (2008). the performativity of routines: theorising the influence of artefacts and distributed agencies on routines dynamics. research policy, 37(5), 769-789. d’adderio, l. (2011). artifacts at the centre of routines: performing the material turn in routines theory. journal of institutional economics, 7(2), 197-230. deken, f., carlile, p. r., berends, h., & lauche, k. (2016). generating novelty through interdependent routines: a process model of routine work. organization science, 27(3), 659-677. dosi, g., faillo, m., & marengo, l. (2008). organizational capabilities, patterns of knowledge accumulation and governance structures in business firms: an introduction. organizational studies, 28(8-9), 1165-1185. dunphy, d. c., & stace, d. a. (1988). transformational and coercive strategies for planned organizational change: beyond the od model. organization studies, 9(3), 317-334. edmondson, a. c., bohmer, r. m., & pisano, g. p. (2001). disrupted routines: team learning and new technology implementation in hospitals. administrative science quarterly, 46(4), 685-716. feldman, m. s. (2000). organizational routines as a source of continuous change. organization science, 11(6), 611-629. feldman, m. s. (2003). a performative perspective on stability and change in organizational routines. industrial and corporate change, 12(4), 727-752. feldman, m. s., & pentland, b. t. (2003). reconceptualizing organizational routines as a source of flexibility and change. administrative science quarterly, 48(1), 94-118. feldman, m. s., & orlikowski, w. j. (2011). theorizing practice and practicing theory. organization science, 22(5), 1240-1253. felin, t., foss, n. j., heimeriks, k. h., & madsen, t. l. (2012). microfoundations of routines and capabilities: individuals, processes, and structure. journal of management studies, 49(8), 1351-1374. fulk, j. (1993). social construction of communication technology. academy of management journal, 36(5), 921–951. gersick, c. j. (1991). revolutionary change theories: a multilevel exploration of the punctuated equilibrium paradigm. academy of management review, 16(1), 10-36. goh, j. m., gao, g., & agarwal, r. (2011). evolving work routines: adaptive routinization of information technology in healthcare. information systems research, 22(3), 565-585. grint, k., & woolgar, s. (2013). the machine at work: technology, work and organization. john wiley & sons. hess, t., matt, c., benlian, a., & wiesböck, f. (2016). options for formulating a digital transformation strategy. mis quarterly executive, 15(2). hsiao, r. l., wu, s. h., & hou, s. t. (2008). sensitive cabbies: ongoing sense-making within technology structuring. information and organization, 18(4), 251-279. australasian journal of information systems taj, kautz & bruno 2021, vol 25, selected papers from acis 2019 it-enabled ot as a coevolution process 23 huber, g. p. (1990). a theory of the effects of advanced information technologies on organizational design, intelligence, and decision making. academy of management review, 15(1), 47-71. hutchins, e. (1991). organizing work by adaptation. organization science, 2(1), 14-39. hutchins, e. (1995). cognition in the wild. ma: mit press. introna, l.d. (2013). epilogue: performativity and the becoming of sociomaterial assemblages, in f.-x. de vaujany and n. mitev (eds.), materiality and space, basingstoke: palgrave macmillan, pp. 330-342. jones, m. (2014). a matter of life and death: exploring conceptualizations of sociomateriality in the context of critical care. mis quarterly, 38(3), 895-925. latour, b. (1992). where are the missing masses? the sociology of a few mundane artifacts, in w. e. bijker and j. law (eds.), shaping technology/building society: studies in sociotechnical change, cambridge, ma: mit press, pp. 225–258. lave, j., & wenger, e. (1991). situated learning: legitimate peripheral participation. cambridge: cambridge university press. lehrig, t., krancher, o., & dibbern, j. (2015). the evolution of routines under flexible information technology. ecis proceedings, research in progress, 14. leonard-barton, d. (1988). implementation and mutual adaptation of technology and organization. research policy, 17 (5), 603-631. leonardi, p. m. (2007). activating the informational capabilities of information technology for organizational change. organization science, 18(5), 813-831. leonardi, p. m. (2009). crossing the implementation line: the mutual constitution of technology and organizing across development and use activities. communication theory, 19(3), 278-310. leonardi, p. m., & barley, s. r. (2008). materiality and change: challenges to building better theory about technology and organizing. information and organization, 18(3), 159-176. leonardi, p. m., & barley, s. r. (2010). what’s under construction here? social action, materiality, and power in constructivist studies of technology and organizing. academy of management annals, 4(1), 1-51. leonardi, p. m. (2011). when flexible routines meet flexible technologies: affordance, constraint, and the imbrication of human and material agencies. mis quarterly, 35(1), 147-167. leonardi, p. m. (2012). materiality, sociomateriality, and socio-technical systems: what do these terms mean? how are they different? do we need them, in p.m. leonardi, b a. nardi and j. kallinikos (eds.), materiality and organizing: social interaction in a technological world, oxford: oxford university press, pp. 25-48. lyytinen, k., newman, m., & al-muharfi, a. r. a. (2009). institutionalizing enterprise resource planning in the saudi steel industry: a punctuated socio-technical analysis. journal of information technology, 24(4), 286-304. australasian journal of information systems taj, kautz & bruno 2021, vol 25, selected papers from acis 2019 it-enabled ot as a coevolution process 24 mingers, j. (2001). combining is research methods: towards a pluralist methodology. information systems research, 12(3), 240-259. mingers, j., mutch, a., & willcocks, l. (2013). critical realism in information systems research. mis quarterly, 37(3), 795-802. misa, t.j. (1994). retrieving sociotechnical change from technological determinism, in m.r. smith and l. marx (eds.), does technology drive history? the dilemma of technological determinism , cambridge, ma: mit press, pp. 115-141. müller, s. d., mathiassen, l., saunders, c., & kræmmergaard, p. (2017). political manoeuvring during business process transformation: a pluralist approach. journal of the association for information systems, 18(3), 3. müller, s. d., mathiassen, l., & saunders, c. (2020). pluralist theory building: a methodology for generalizing from data to theory. journal of the association for information systems, 21(1), 9. nelson, r. r., & winter, s. (1982). an evolutionary theory of economic change. cambridge, ma: belknap press/harvard university press. noble, d. f. (1978). social choice in machine design: the case of automatically controlled machine tools, and a challenge for labor. politics & society, 8(3-4), 313-347. orlikowski, w. j., & gash, d. c. (1994). technological frames: making sense of information technology in organizations. acm transactions on information systems (tois), 12(2), 174207. orlikowski, w. j. (1996). improvising organizational transformation over time: a situated change perspective. information systems research, 7(1), 63-92. orlikowski, w. j. (2000). using technology and constituting structures: a practice lens for studying technology in organizations. organization science, 11(4), 404-428. orlikowski, w. j., & iacono, c. s. (2001). research commentary: desperately seeking the “it” in it research—a call to theorizing the it artifact. information systems research, 12(2), 121-134. orlikowski, w. j. (2002). knowing in practice: enacting a collective capability in distributed organizing. organization science, 13(3), 249-273. orlikowski, w. j. (2007). sociomaterial practices: exploring technology at work. organization studies, 28(9), 1435-1448. orlikowski, w. j., & scott, s. v. (2008). sociomateriality: challenging the separation of technology, work and organization. the academy of management annals, 2(1), 433-474. parmigiani, a., & howard-grenville, j. (2011). routines revisited: exploring the capabilities and practice perspectives. academy of management annals, 5(1), 413-453. pentland, b. t., & feldman, m. s. (2005). organizational routines as a unit of analysis. industrial and corporate change, 14(5), 793-815. pentland, b. t., & feldman, m. s. (2008). designing routines: on the folly of designing artifacts, while hoping for patterns of action. information and organization, 18(4), 235-250. australasian journal of information systems taj, kautz & bruno 2021, vol 25, selected papers from acis 2019 it-enabled ot as a coevolution process 25 pickering, a. (1995). the mangle of practice: time, agency, and science. chicago: university of chicago press. poole, m. s., & desanctis, g. (1990). understanding the use of group decision support systems: the theory of adaptive structuration, in j. fulk and c. steinfield (eds.), organizations and communication technology, sage publications, pp. 173-193. putnam, l. l. (2015). unpacking the dialectic: alternative views on the discourse–materiality relationship. journal of management studies, 52(5), 706-716. rerup, c., & feldman, m. s. (2011). routines as a source of change in organizational schemata: the role of trial-and-error learning. academy of management journal, 54(3), 577-610. rice, r. e., & aydin, c. (1991). attitudes toward new organizational technology: network proximity as a mechanism for social information processing. administrative science quarterly, 219-244. romanelli, e., & tushman, m.l. (1994). organizational transformation as punctuated equilibrium: an empirical test. academy of management journal, 37(5), 1141-1166. sabherwal, r., hirschheim, r., & goles, t. (2001). the dynamics of alignment: insights from a punctuated equilibrium model. organization science, 12(2), 179-197. schultze, u., & orlikowski, w. j. (2004). a practice perspective on technology-mediated network relations: the use of internet-based self-serve technologies. information systems research, 15(1), 87-106. seidel, s., recker, j., & vom brocke, j. (2013). sensemaking and sustainable practicing: functional affordances of information systems in green transformations. mis quarterly, 27(4), 1275-1299. vaast, e., & walsham, g. (2005). representations and actions: the transformation of work practices with it use. information and organization, 15(1), 65-89. verhulst, m., & rutkowski, a. f. (2017). catch me if you can: technological constraints/affordances and mindfulness during collaborative police emergency response. in proceedings of the 50th hawaii international conference on system sciences. vial, g. (2019). understanding digital transformation: a review and a research agenda. the journal of strategic information systems, 28(2), 118-144. vishwanath, a. (2006). the effect of the number of opinion seekers and leaders on technology attitudes and choices. human communication research, 32(3), 322-350. watson, r. t., desanctis, g., & poole, m. s. (1988). using a gdss to facilitate group consensus: some intended and unintended consequences. mis quarterly, 12(3), 463-478. weick, k.e. (1979). the social psychology of organizing (2nd ed.). new york: mcgraw-hill. weick, k. e. (1993). organizational redesign as improvisation, in g.p. huber and w.h. glick, (eds.), organizational change and redesign: ideas and insights for improving performance, , 379. oxford: oxford university press, pp. 346379. wessel, l., baiyere, a., ologeanu-taddei, r., cha, j., & jensen, t. (2020). unpacking the difference between digital transformation and it-enabled organizational transformation. journal of association of information systems, 22(1), 6. australasian journal of information systems taj, kautz & bruno 2021, vol 25, selected papers from acis 2019 it-enabled ot as a coevolution process 26 winter, s. g. (2003). understanding dynamic capabilities. strategic management journal, 24(10), 991-995. zuboff, s. (1988). the future of work and power. in the age of the smart machine. new york: basic books. copyright: © 2021 taj, kautz & bruno. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v25i0.2855 http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems pereira, lapão, bianchi & amaral 2020, vol 24, research article business process redesign in emergency departments 1 improving emergency department through business process redesign: an empirical study ruben pereira instituto universitário de lisboa (iscte-iul) portugal ruben.filipe.pereira@iscte-iul.pt luís velez lapão instituto de higiene e medicina tropical, universidade nova de lisboa portugal isaias scalabrin bianchi federal university of santa catarina brazil daniel amaral instituto universitário de lisboa (iscte-iul) portugal abstract the emergency departments (ed) of hospitals are poised for continuous improvement due to increasing demand. the ed are places where the efficiency of the services provided can save lives and therefore it should be seen as an area of maximum interest for process optimisation. this research aims at verifying if the application of business process management (bpm) heuristics positively impacts the length of stay (los), without deteriorating the quality of the service provided. the methodology that underlies this research is a case study carried out in the ed of a public hospital. the data was gathered from individual interviews, focus groups, direct observation and document analysis. the bpm life cycle stages were followed. in addition, a simulation tool was used, and heuristics were chosen based on devil’s quadrangle theory. three scenarios of the ed were considered. additionally, three heuristics and any combination of them were also considered. results show that heuristics positively impact the time variable without affecting the quality of the service, resulting in value gains for the patient. in terms of time consumption, the average los in the process was reduced by 22.5%, 15.9%, and 20.9% for each of the considered scenarios, while the maximum los was reduced by 29.2%, 36.2% % and 37.4%. implications from these results were analysed. the novelty of this research is supported by the absence of studies applying bpm heuristics to ed. this research is a step forward to ally bpm heuristics and ed processes. keywords emergency departments, business process management, redesign heuristics, process redesign and simulation, quality, time. 1 introduction health care systems around the world are increasingly struggling to implement changes in their process aiming to reduce costs, improve the operational efficiency and deliver better services for the patients (feibert, andersen, & jacobsen, 2019; feibert & jacobsen, 2019; hassan, 2017). health services have two major management challenges: the use of management evidence and management innovation (hassan, 2017; lapão, 2016; pestana, australasian journal of information systems pereira, lapão, bianchi & amaral 2020, vol 24, research article business process redesign in emergency departments 2 pereira, & moro, 2020). this research focuses on the first management challenge mentioned by lapão (2016), hassan(2017), garmann-johnsen and eikebrokk (2017), the use of management evidence – more precisely, on the optimisation of healthcare work processes. the healthcare industry often presents a poor design, high fragmentation and stunning inefficiency (dickman, tong, & davis, 2012). individual achievements are no longer sustainable to reach the excellence of an overall functional healthcare system. for instance, emergency departments (ed) are facing many problems today (de ramón fernández, ruiz fernández, & sabuco garcía, 2019; feibert et al., 2019; hassan, 2017; hoot & aronsky, 2008; leva, sulis, & vinai, 2017), like: p1shortage of resources that often leads to work overload; p2long waiting times for patients make the process of ed even longer; p3excessive use of ed, often not fulfilling the criteria for emergency. these problems can lead to well-known situations of inefficiency, medical risk and financial losses. in addition, in the portuguese healthcare system, the ed works as a buffer to respond to the most serious cases effectively and promptly. thus, it should have the required conditions to cope with the critical care demand. these problems remain a case of general concern to all. it is recognized as a valueadded topic to investigate. grounded on the critical importance of ed aforementioned, “redesign heuristics” play a crucial role when improvement needs come up (dumas, la rosa, mendling, & reijers, 2018). therefore, this research aims at investigating the application of business process management (bpm) and redesign heuristics to improve ed services. to pursue our goal, a case study (cs) was performed in an ed of a portuguese hospital located in the lisbon region. this is a critical issue for portugal since citizens’ average life expectancy increased from 63 years in 1961 to 81 years in 2011, and births are decreasing significantly, in 1960 the birth rate was 16.2% and in 2011 it was only 7.9% (pordata, 2018). in addition, at the end of 2014/early 2015, a major flu epidemic emerged, creating chaos in the portuguese ed services. long waiting times for patients to be attended due to lack of beds to accommodate patients was the most relevant factor. additionally, portugal is one of the countries with higher rate of ed visits per one hundred thousand inhabitants. therefore, portugal is a suitable place to explore new ways to improve ed. moreover, the hospital under study defined wasted time as the critical point to improve assuming that service quality would not decrease. bearing this in mind and taking into account both problems and research objectives, the research question (rq) this research intends to answer is: which business process redesign heuristics can help optimise time response in an ed without deteriorating the quality of the service provided? 2 theoretical background 2.1 business process management and care process management seen as a set of activities that together may accomplish a business objective (davenport, 1993; ko, zhai, & rodriguez-zas, 2009), business processes are a key instrument to keep enterprises organized (van der aalst, ter hofstede, & weske, 2003). bpm has a settled arrangement of standards wüllenweber et al.(2008), techniques and instruments that consolidate learning from information technology, management sciences and industrial engineering with the purpose of improving business processes (dumas et al., 2018; van der aalst, 2013; van der aalst et al., 2003). in accordance with vom brocke et al. (2014), bpm focuses on improving corporate performance by managing business processes. in this australasian journal of information systems pereira, lapão, bianchi & amaral 2020, vol 24, research article business process redesign in emergency departments 3 case, it will focus on the optimisation of the ed, improving the overall services provided by a healthcare institution. although, it is regular to allude to care process management (cpm) in the healthcare sector while discussing bpm. cpm is the use of business process automation and optimisation techniques in clinical care processes in the healthcare environment. it additionally utilizes the best practices for bpm to enhance clinical results without changing health processes or dislodging the job of health stakeholders (dickman et al., 2012). cpm offers a guide that helps the healthcare sector to change delivery processes. without changing what human services specialists do best, it tends to the inalienable wasteful aspects of the care delivery process. numerous parts of care delivery can be enhanced utilizing bpm to automatize the coordination of care delivery and to empower suppliers to team up more adequately (dickman et al., 2012). 2.2 redesign heuristics heuristics are strategies derived from previous experiences with similar problems. these strategies rely on using readily accessible, though loosely applicable, information to control problem-solving in human beings, machines, and abstract issues (ippoliti, 2015; şahin, duran, yakıcı, & şahin, 2018; testa, cinque, coronato, de pietro, & augusto, 2015). in bpm, redesign heuristics exist as a method that builds upon an extensive set of redesign options (reijers & liman mansar, 2005). during the redesign process organizations’ goals compliance must be assured and therefore it is advisable to use the devil’s quadrangle framework (dumas et al., 2018). organizations’ goals and strategy determine which dimension should be maximized and what consequences these decisions have on the other dimensions. heuristics in business process redesign (bpr) have been proposed in the last years and are synthesized by dumas et al. (2018). these heuristics can be divided in six categories: customers, products, business process (operation view & behavioural view), organisation (structure & population), information and finally, technology. due to space limitations the authors do not list all the heuristics. although a redesign heuristic is not a model of a business process, it represents a set of ideas that help one think about the business process in the context of process redesign. in this case, redesign heuristics are applied in order to measure the impact on the process and then evaluate if that impact is positive or negative. 2.3 previous work following the concept-centric approach (webster & watson, 2002), this section lists the studies applying bpr in healthcare using bpm heuristics. several scientific databases were consulted, namely, ieee explore, google scholar and abi/inform collection. furthermore, the following keywords were used in the search: “process optimisation”, “business process management”, “process redesign” and “redesign heuristics”. overall, some researchers have investigated how to improve healthcare services including ed. most of the studies are not focused on ed but this is not completely new in the body of knowledge. some studies present improvements of healthcare services resulting from the implementation of healthcare reforms using bpm as a tool. for example, in italy, a hospital used a bpm system to improve service quality and efficiency. as a result, it was able to identify areas for australasian journal of information systems pereira, lapão, bianchi & amaral 2020, vol 24, research article business process redesign in emergency departments 4 improvement such as reducing the number of operating sessions, preparation of the operating rooms for each operation, availability of specific surgical instruments and so on. however, the list of studies found highlights that the application of bpm theory in ed is far less when compared with other areas and techniques in healthcare (leggat, bartram, stanton, bamber, & sohal, 2015). lean for example is commonly applied in healthcare literature and some studies exist in ed (holden, 2011). therefore, further research in this area is required. after an in-depth inquiry on the topic, seven studies were found (table 1). this research focused on redesign of organisational (healthcare) processes, supported by heuristics and having the bpm tool as an ally in order to obtain the best results. despite being identified as applying heuristics in an ed, studies 1,2,6 and 7 were not developed as officially adopting heuristics from the lens of bpm. only after a deep analysis of the studies, the authors were able to understand that a heuristic was in fact being applied to improve the ed. # author & year field of study redesign heuristics applied 1 (kelly, bryant, cox, & jolley, 2007) ed customer teams; 2 (leva et al., 2017) ed extra resources 3 (buzacott, 1996) healthcare contact reduction; empower; task elimination; task composition; parallelism; case manager; control addition; 4 (jansen-vullers & reijers, 2005) healthcare case manager; empower; extra resources; parallelism. 5 (leu & huang, 2011) healthcare task automation; integration; order types; task elimination; centralization 6 (kumar & shim, 2007) ed resequencing 7 (baumlin et al., 2010) ed integral technology table 1. redesign heuristics application in the healthcare sector 3 methods the research methodology adopted in this investigation was the case study (cs). the cs approach may be used to observe phenomenon in real events (yin, 2009). according to zucker (2009) and bromley (1990) it is a systematic inquiry into an event or a set of related events which aims to describe and explain the phenomenon of interest. the cs method is particularly appropriate for these types of studies and well-suited to capture knowledge and develop theories (benbasat, goldstein, & mead, 1987). consequently, since this research focuses on contemporary events within the real-life context (an ed of a real healthcare institution), the cs method was selected by its ability to look at the world around us and also its importance in the it area (pereira, almeida, & mira da silva, 2013). moreover, a cs is suitable to answer research questions of the type “how” or “why”, which is also aligned with the rq listed in section 1. additionally, to support the use of the cs method and as advised by some authors (yin, 2009; zucker, 2009), complementary methods will be used, such as direct observation, document analysis, interviews and focus groups with stakeholders. despite adopting cs methodology (yin, 2009), since this study also follows bpm lifecycle phases, the authors decided to organize section 4 based on bpm phases instead of yin’s steps. nevertheless, yin’s steps as well as used techniques and methods are easily australasian journal of information systems pereira, lapão, bianchi & amaral 2020, vol 24, research article business process redesign in emergency departments 5 identifiable throughout the following sections. following gary (gary, 2016) theory, our cs is a single and local cs. since we intended to explore the applicability of certain heuristics in an ed one may argue that this cs follows the testing theory approach. this study adopted mixed methods (dumas et al., 2018; yin, 2009) detailed in table 2. the considered stakeholders were: nursing manager, nursing supervisor, medical manager and an administrator. these stakeholders are able to represent the most regular concerns of all the physicians, nurses and top management. firstly, a meeting with all stakeholders was held to define the scope of the research and limits of the process to be studied. additionally, individual interviews were used to collected different inputs from different stakeholders. along the investigation, when needed, observation and document analysis were also performed. at the end, a focus group with all the stakeholders involved was formed to reach consensus. moreover, along the cs, bizagi software (bizagi, 2018) was used to model and simulate the ed process. it should be noted that process mining was excluded based on the limitations of the is used that will be further detailed throughout the document. 4 business process redesign of the emergency department 4.1 process identification, discovery and modelling in these phases of the bpm lifecycle, the authors followed dumas et al. (dumas et al., 2018) presenting a description of the most relevant processes. the unit of analysis is the ed of a portuguese public hospital (located in the lisbon region). it plays a pivotal role in responding to urgent cases given its geographic location. the hospital has three emergency departments: obstetric, paediatric and general. in this research, only the general ed was considered. dumas et al. (2018) yin (2013) execution interview-based discovery individual interview there will be as many rounds of interviews as needed. the first ones for process modelling, the second ones for process analysis and the last ones for process redesign. each round of interviews will contemplate all the interviewees, and, in a specific situation, it could be needed to do more than 1 interview with the same professional. evidence-based discovery direct observation method of getting more insights and specifications of the process. documentation analysis it will be done as a form of validation of the information collected and as a comparison between what is being done and what is written. workshop-based discovery focus group meetings with the professionals interviewed in each round in order to cross and discuss information collected in previous phases. table 2. yin and dumas perspectives of data collection. source: own source only part of the ed process is considered in this research (see figure 1). the scope of the process under study, begins with the entry of the patient in the ed and ends with the reevaluation of the patients. the ed is organized in three observation areas: emergency services of greens and blues (esgb), ambulatory and clinical observation. after the triage, the patient receives a priority (colour), which dictates the waiting room where the patient should wait. there is another designated area for red priority patients called the revival room. however, australasian journal of information systems pereira, lapão, bianchi & amaral 2020, vol 24, research article business process redesign in emergency departments 6 both revival room and esgb are outside the scope of this research since they have less patients. there is a high variety of human resources (hr) working in the ed. this research focuses on the hr directly related with the patient flow at the ed (table 3). type of resource quantity doctors (general medicine) 11 doctors–specialist* 3 nurses–t 2 nurses–cap 8 desk supporters 2 technicians 3 * the specialist (many specialities) is not 100% allocated to the ed, but he/she provides her/his services every time they are required. table 3. resources available per shift nurses-t operate in the triage while nurses-cap operate in the clinical analysis and procedures. activities like triage and re-triage are done by the same resources as medical observation and medical re-evaluation. table 4 lists the main activities, rooms available for each activity and the type and quantity of resources needed are introduced. activity rooms type/quantity per room admission 2 1 of 2 desk supporters triage/re-triage 2 1 of 2 nurses-t medical observation/medical re-evaluation 10 1 of 11 doctors evaluated for specialist 3 1 of 3 doctors-specialist medical exams 3 1 of 3 technicians clinical analysis and procedures 2 2 of 8 nurses-cap table 4. resources needed per activity five priority categories exist to decide the maximum advisable waiting time for each patient (table 5). additionally, according to the ed administrator, there are also satisfactory times to perform the medical observation activity (table 5). since red priority is seen as the most urgent situation, there is no specified time. priority waiting time (min.) observation time (min.) red 0 no time defined orange 10 45 yellow 60 30 green 120 15 blue 240 10 table 5. satisfactory waiting times and medical observation time. source: provided by the ed under study 4.2 process as-is at the end of this phase the final as-is model was obtained (figure 1). australasian journal of information systems pereira, lapão, bianchi & amaral 2020, vol 24, research article business process redesign in emergency departments 7 4.2.1 inputs for process simulation several sources were used. the authors had access to a dataset provided by the hospital. the data collection occurred between january 1st, 2015 and december 31st, 2017 (a total of 1096 days). in total there were 305,636 records, with each record representing the interaction of a patient with the ed, containing a total of 15 attributes per record (figure 2). whenever a patient visits the ed, a sequence of activities is followed the ed flow. this ed flow has five main activities. it starts with the patient admission, followed by the triage where the patient will be assessed and classified with a colour according to the manchester triage protocol. after the triage, the patient waits to be observed by a doctor (observation). then, after getting observed and getting treatment from a clinician, the patient is discharged (discharge), and later on formally allowed to leave the hospital (administrative discharge). figure 1. process as-is of the ed figure 2. original dataset attributes list admssion timestamp (t1) triage timestamp (t2) episode id nurse id discharge destination observation timestamp (t3) discharge timestamp (t4) patient id doctor id readmission flag administrative discharge timestamp (t5) disease code (code_icd) ed sub deparment discharge status triage color (mtprotocol) (tc) australasian journal of information systems pereira, lapão, bianchi & amaral 2020, vol 24, research article business process redesign in emergency departments 8 as previously stated, this research does not address the entire process. the first three activities are the ones that will be further investigated. the data provided by the is was not totally feasible. unfortunately, the ed is did not contemplate all the possible process exceptions (paths) and some information could be manually inserted after the events. in some cases, this resulted in incongruent timestamps. therefore, the authors decided to cross-examine the data from the dataset with information gathered from interviews and direct observation. the period used to simulate the models was previously defined with the stakeholders. the time of day chosen was from 9 a.m. to 12 a.m. (the daily period with a higher flow of patients to the ed) for a total of 15 hours. furthermore, three distinct scenarios were considered. each scenario has a different ratio of patient per hour. for a proper simulation, all the required information was collected (table 6). required information description resources available per shift table 3. type of resource needed for each activity and rooms available for each activity each activity needs a physical space and hr (table 4). work time per activity (minimum, maximum and average) the working times were gathered both from the is of the ed and individual interviews with stakeholders. regarding data extracted from the is of the ed, unfortunately, it was inaccurate and incomplete. thus, the times presented in table 7 are as close as possible based on both sources. percentage to be defined in the gateways this information is useful to decide the weight of each process path in the simulator (table 8). patient’s flow – three scenarios (s1, s2, and s3) all days in the ed are not equal. there are some tendencies depending on the day of the week. according to the medical manager, mondays are the most complicated days due to the high flow of patients (corroborated by the dataset we had access to). to fulfil the variety of each day during a work week, three scenarios were created based on the information collected from the is but also considering the information provided by the medical manager (table 9). from the data gathered from the is of the ed, it was possible to verify that, on average, 85% of the patients go to the ed between 9 a.m. and 12 p.m. statistical distributions were chosen triangular distribution was used to simulate the work time of all activities. it is used when the information about the real time of the activity is scarce. some activities have a required waiting time. this time is set on average time basis only table 6. required information for simulation activity required waiting time (mins) min. (mins) avg. (mins) max. (mins) admission 1 3 7 triage 12 8 14 re-triage 3 6 14 medical observation 24 33 61 re-evaluation 15 28 54 observation for specialist 10 22 25 47 medical exams 5 20 30 43 clinical analysis and procedures 5 15 26 40 table 7. working times australasian journal of information systems pereira, lapão, bianchi & amaral 2020, vol 24, research article business process redesign in emergency departments 9 activity required waiting time (mins) min. (mins) critical patient?(excluded from the process) yes 2% no 98% priority?(excluded from the process) green and blue priority 33% others 67% need re-triage? yes 15% no 85% what was defined by the doctor? discharge 8% another opinion 5% exams or cap 87% what was defined by the specialist? discharge 15% exams or cap 85% exams or cap? medical exams 40% cap 60% table 8. gateways percentages scenario patient ratio/working hour total patients/day working time total patients/working time (85% of total) as-is s1 13 patients 230 15h (9-24) 196 as-is s2 16 patients 280 15h (9-24) 238 as-is s3 19 patients 350 15h (9-24) 298 table 9. patients entry rate (three scenarios) 4.2.2 results of process simulation (process as-is) in the process simulation, what-if analysis was performed based on three scenarios (table 9) with the same resources for each. the difference between the three scenarios is the patient ratio. ed_s3 is the scenario with a higher rate of patients per hour. in 15 hours of the working day under study, 280 patients entered the ed_s3. the rate of patients in the ed_s3 is 20.2% higher than ed_s2 and 48.9% higher than ed_s1. the results of the simulation only refer to the scope of the process under study, from the moment the patient enters in the ed, to the moment the patient is re-evaluated. the average time that a single patient is within the process is 2 hours and 9 minutes in the ed_s3, taking at the maximum 6 hours and 37 minutes. comparing the maximum and the average time between the scenarios, one can sustain that the higher the rate of patients per hour, the higher the maximum and average time that patients stay in ed. the maximum and average time that a patient stays within the ed_s3 are 15.6% and 20.9% higher than in ed_s1, and 11.5% and 11.8% higher than in ed_s2. name scenario type instances completed instances started max. time (hrs) avg. time (hrs) ed_s1 as-is s1 process 161 188 5h43m 1h47m ed_s2 as-is s2 process 202 233 5h56m 1h55m ed_s3 as-is s3 process 212 280 6h37m 2h09m table 10. scenarios of process as-is considering the results present in table 10, we decided to select only one scenario to compare in the process redesign phase. the chosen scenario is the ed_s3 since it presents a higher australasian journal of information systems pereira, lapão, bianchi & amaral 2020, vol 24, research article business process redesign in emergency departments 10 margin of improvements and represents more problems for the ed. however, the results for the other two scenarios are detailed ahead in the document. table 11 introduces the hr utilization in the chosen process (ed_s3). doctors and both types of nurses are the most used. on the other hand, the doctor–specialist has the lowest utilization since he/she is not fully allocated to the ed and only works in the ed when called for specific cases. resource utilization doctor 91.95% nurse-t 87.36% nurse-cap 81.18% technician 79.54% desk supporter 77.92% doctor-specialist 10.08% table 11. utilization of resources 4.3 process analysis at this stage, the issues associated with the process as-is are identified, documented and, where possible, quantified using performance measures (dumas et al., 2018). in this way, it will be possible to understand the position of the process regarding the objectives established by the organisation. the aim is to prioritize the problems given their impact on ed and effort required to be solved. to elicit the problems of the ed, a round of interviews with all the stakeholders involved was performed followed by a focus group session aiming to: identify consensual problems (table 12), merge similar problems and rename problems if needed (table 13). all the interviewees had more than 20 years of experience in healthcare industry. the nursing manager is male, and the remaining stakeholders are females. table 12 presents which stakeholder pointed each problem (and respective percentage overall) and identifies the ones stated as consensual at the end of the focus group. no problem categories were predefined. the interviewees were free to identify any problem. additionally, some evidence was taken from the simulated model of the as-is. the final list of problems is the input to identify the possible improvements, which will, therefore, be aligned with the redesign heuristics. 4.4 process redesign the table 12 presents the wide list of problems. bpr seeks to create new ways of organising tasks, people and redesigning information systems so that processes are improved in critical and contemporary performance measures such as cost, quality, service and speed (hammer & champy, 1993; sherwood-smith, 1994). australasian journal of information systems pereira, lapão, bianchi & amaral 2020, vol 24, research article business process redesign in emergency departments 11 problem stakeholders total % fg admin mm nm ns work overload • • 2 50 • information system used is obsolete • • 2 50 • information system is not fully used • • • 3 75 • lack of information flow from external services and ed • • 2 50 lack of physical space in the ed • • • • 4 100 • inefficient disposition of areas • • 2 50 • the medical team is inexperienced • • 2 50 • many patients go to the ed with no emergent situations • • • 3 75 • the unnatural high flow of patients • • • • 4 100 • inexperienced nurses in triage • 1 25 non-specialized doctors • • 2 50 • lack of basic health knowledge from the population • • 2 50 • response after exams results is prolonged • • 2 50 • lack of security in the infrastructure • • 2 50 • low organisational culture • 1 25 call for medical observation and patient's picking in the is may not match • • 2 50 admin:administrator; mm:medical manager; nm:nursing manager; ns:nursing supervisor; fg:consensus reached during focus group. table 12. wide list of problems moreover, bpr supports organisational restructuring, focusing on business objectives. for this, business processes are analysed, non-essential or redundant tasks are eliminated, and it is used to redesign and simplify organisational processes. the organizations redesigning the processes have several advantages such as decreasing the time required to execute it, reducing its cost, improving the quality of the output, and improving the process's ability to react to variation. main problems problems (table 12) p1 work overload work overload p2 the medical team is inexperienced the medical team is inexperienced non-specialized doctors p3 information system is not adjusted to the needs of ed information system used is obsolete information system is not fully used p4 the patients wait too much time for reevaluation response after exams results is prolonged p5 the physical space of ed is fully used lack of physical space in the ed inefficient disposition of areas p6 the population is not well informed about basic health knowledge many patients go to the ed with no emergent situations lack of basic health knowledge from the population p7 the unnatural high flow of patients the unnatural high flow of patients p8 lack of security in the infrastructure lack of security in the infrastructure table 13. main list of problems in this study, the authors followed the recommendations suggested by dumas et al. (2018) (redesign heuristics) to redesign the ed process. the data to support the process redesign was gathered through another round of interviews, followed by a focus group session and then australasian journal of information systems pereira, lapão, bianchi & amaral 2020, vol 24, research article business process redesign in emergency departments 12 enhanced with the aforementioned results of process simulation. the improvements mentioned by each stakeholder during the interviews are listed in table 14. improvements type (p/m) stakeholders total % admin mm nm ns i1 increase the type of resources needed when needed p • • 2 50 i2 training sessions for the medical team p • • 2 50 i3 contract more expert doctors than non-experts p • • 2 50 i4 improve the is to fulfil all the needs of the ed and automatically provided notifications to the key-users p • • 2 50 i5 make use of more functionalities of the is p • • • 3 75 i6 create a notification that alert doctors when results of exams arrive at is p • • • 3 75 i7 increase the physical space of the ed m • • • • 4 100 i8 predict in advance the need for increasing the number of doctors and nurses m • • 2 50 i9 marketing campaigns to inform patients about when to go to ed m • 1 25 i10 create synergies with the near local healthcare centres m • • 2 50 i11 increase security staff of the ed m • • • 75 p/m:processual/managerial; admin:administrator; mm:medical manager; nm:nursing manager; ns:nursing supervisor; m:managerial; p:processual table 14. results of interviews (process redesign) the redesign heuristics was chosen considering both the hospital objectives and the processual improvements. in addition, they were approved by stakeholders during the focus group. the final list of problems, improvements and heuristics can be seen in table 15 and all the simulated scenarios in table 16. problem improvement redesign heuristic p1 work overload i1 increase the type of resources needed when needed h1 extra resources p2 the medical team is inexperienced i2 training session for the medical team h2 specialist i3 invest in more expert doctors that will make work faster and keeping the degree of quality, adjusting the ratio of specialist/nonspecialist p3 information system is not adjusted to the needs of ed i4 improve the is to fulfil all the needs of the ed and automatically provided notifications to the key-users h3 integral technology i5 make use of more functionalities of the is p4 the patients wait too much time for reevaluation i6 create a notification that alert doctors when results of exams arrive at is table 15. alignment of redesign heuristics with problems and improvements australasian journal of information systems pereira, lapão, bianchi & amaral 2020, vol 24, research article business process redesign in emergency departments 13 redesign heuristic devil’s quadrangle h1 h2 h3 c1 c2 c3 c4 time quality extra resources . • • • • specialist + • • • • integral technology + • • • • h1,h2,h3=heuristic 1,2,3; c1,c2,c3,c4=combination of heuristics table 16. combinations of redesign heuristics 4.4.1 process simulation (h1) – “extra resources” to simulate the “extra resources” heuristics the authors increased the number of most used resources by one (table 17) and results are listed in table 18. resources utilization was measured based on information gathered from interviews and on the simulation of the as-is. according to the medical manager, there is a shortage of doctors to respond to the high flow of patients, and the nursing manager stated the same regarding nurses. results of the simulation are presented in table 17 and table 18 details the comparison of those results. it should be noted that the overall maximum and average times of the process decreased 7.3% (6h37m to 6h08min (-29min)) and 7.0% (2h09m to 2h00 (-9min)) respectively. process as-is process h1 doctors=11 doctors=12(+1) nurses–cap=8 nurses–cap=9(+1) nurses–t=2 nurses–t=3(+1) table 17. extra resources resources utilization as-is utilization h1 variation doctor 91.95% 90.39% -1.56% nurse–t 87.36% 79.41% -7.95% nurse–cap 81.18% 68.92% -12.26% technician 79.54% 83.79% 4.25% desk supporter 77.92% 77.92% 0.00% doctor–specialist 10.08% 11.28% 1.20% table 18. comparison of resources utilization (h1) 4.4.2 process simulation (h2) – “specialist” specialist resources can build up routines more quickly and may have more profound knowledge in an area than generalists. as a result, they work more quickly and deliver higher quality. on the other hand, the availability of generalists adds more flexibility to the business process and can lead to better utilization of resources. in this case, the balance should tend towards the specialist. thus, two options come up: contract more expert doctors than nonexperts or provide specific training to non-experts in order to specialize them. for simulation, we assumed that the specialisation of doctors could decrease the time of the activities “medical observation” and “re-evaluation” by 10%, as presented in table 19. as can be seen in table 20, the number of patients attended increased by 10 in the overall process. regarding the whole process, the maximum time increased by 5 minutes (+1.3%) while the average time decreased by 11 minutes (-8.5%). moreover, table 21 indicates a australasian journal of information systems pereira, lapão, bianchi & amaral 2020, vol 24, research article business process redesign in emergency departments 14 reduction of only 10% of the working time of the “medical observation” activity, a 57.6% decrease of the maximum time and a 31.4% decrease of the average time of such activity. on the other hand, the “re-evaluation” activity has a similar impact regarding times. scenario activity required waiting time (mins) min. (mins) avg. (mins) máx. (mins) as-is medical observation 24 33 61 h2 21.6 29.7 54.9 as-is re-evaluation 15 28 54 h2 13.5 25.2 48.6 table 19. scenario h2 scenario instances completed instances started max. time (h) avg. time (h) process as-is 212 280 6h37m 2h09m h2–specialist 222 280 6h42m 1h58m table 20. h2 simulation name scenario instances completed instances started min. time(m) max. time(m) avg. time(m) medical observation process as-is 146 154 29m 2h12m 1h10m h2-specialist 158 167 25m 1h16m 48m variation +12 +13 -4m -56m -22m re-evaluation process as-is 101 104 20m 1h59m 1h08m h2-specialist 111 113 15m 1h07m 42m variation +10 +9 -5m -52m -26m table 21. h2 simulation activities detail 4.4.3 process simulation (h3) – “integral technology” the application of this redesign heuristic arises to answer three improvements identified to mitigate specific elicited problems (table 15). the use of “integral technology” redesign heuristic changes the way results of exams are treated. previously, no one knew when the exam results came to the is. the doctors had to sporadically go into the patient's record to find out if the exams results were already available there. with the application of this heuristic, it is intended that an alert is created in the is that notifies the doctor that the results of the exams of a given patient are already available for consultation. thus, the estimated time between the moment the results of the exams arrive at the is and the moment the doctor observes the results is reduced by 75%, from 40 minutes to 10 minutes. looking at table 22, the number of patients attended increased by 3. additionally, the maximum and average times of the whole process decreased by 33 minutes (8.3%) and 5 minutes (3.9%), respectively. there is no significant variance between the activities. even though positive, the variance of the average time of the whole process is minimal. australasian journal of information systems pereira, lapão, bianchi & amaral 2020, vol 24, research article business process redesign in emergency departments 15 5 results overall, the three chosen redesign heuristics decreased both maximum and average process time durations, except for h2 that increased the maximum time by 1.3%. curiously, h2 is the redesign heuristic with the best impact on average time, applied individually. thus, one can sustain that these three redesign heuristics, applied individually, provided a positive impact on process durations. the results of all combinations are presented in table 23. regarding average times of the whole process, it is possible to see that c4 is the combination of redesign heuristics that better improves the process as-is. c4 increased the number of patients attended by 28 while the maximum and average times decreased by 29.2% and 22.5% respectively. as can be seen in table 23, the effect on time is improved when redesign heuristics are combined. name scenario ic is min. max. avg. #0 ed (whole process) as-is 212 280 6h37m 2h09m #0 ed (whole process) h3 215 280 6h04m 2h04m #1 make admission on ed as-is 266 268 4m 15m 6m #1 make admission on ed h3 266 268 4m 15m 6m #2 initiate triage as-is 263 266 8m 31m 13m #2 initiate triage h3 263 266 8m 29m 14m #3 perform re-triage as-is 15 15 4m 18m 9m #3 perform re-triage h3 15 15 4m 18m 8m #4 do medical observation as-is 146 154 29m 2h12m 1h10m #4 do medical observation h3 146 153 31m 2h23m 1h14m #5 evaluated for specialist as-is 6 7 36m 48m 42m #5 evaluated for specialist h3 6 7 36m 48m 42m #6 do clinical analysis and procedures as-is 70 73 20m 1h21m 44m #6 do clinical analysis and procedures h3 70 73 20m 1h14m 44m #7 do medical exams as-is 58 61 27m 1h32m 56m #7 do medical exams h3 58 61 27m 1h35m 53m #8 make medical re-evaluation as-is 101 104 20m 1h59m 1h08m #8 make medical re-evaluation h3 104 108 23m 2h15m 1h12m ic='instances completed'; is='instances started' table 22. h3 simulation scenario s3 ic is overall times of the process variation on overall times of process regarding as-is max. time (h) avg. time (h) max. time avg. time as-is 212 280 6h37m 2h9m h1 221 280 6h8m 2h0m -7.3% -7.0% h2 222 280 6h42m 1h58m +1.3% -8.5% h3 215 280 6h4m 2h4m -8.3% -3.9% c1 235 280 5h42m 1h52m -13.9% -13.2% c2 229 280 5h19m 1h53m -19.6% -12.4% c3 229 280 4h55m 1h50m -25.7% -14.7% c4 240 280 4h41m 1h40m -29.2% -22.5% ic='instances completed'; is='instances started. table 23. final results of all combinations for the main scenario (s3) australasian journal of information systems pereira, lapão, bianchi & amaral 2020, vol 24, research article business process redesign in emergency departments 16 in table 24 and table 25 the results of the simulation of the same combinations of redesign heuristics mentioned in table 15 are presented for the other two scenarios mentioned in table 10. the results of these two scenarios are similar to those presented in table 24. regarding average time durations, the application of c4 provided a better impact on the main process studied (scenario s3 in table 10) by improving the average total time of the process by 22.5%. scenario s1 ic is overall times of the process variation on overall times of process regarding as-is max. time(h) avg. time(h) max. time avg. time as-is 161 188 5h43m 1h47m h1 162 188 5h31m 1h47m -3.5% 0,0% h2 164 188 5h29m 1h44m -4.1% -2.8% h3 167 188 3h46m 1h36m -34.1% -10.3% c1 165 188 5h24m 1h44m -5.5% -2.8% c2 167 188 3h46m 1h35m -34.1% -11.2% c3 169 188 3h39m 1h31m -36.2% -15.0% c4 169 188 3h39m 1h30m -36.2% -15.9% ic='instances completed'; is='instances started' table 24. final results of all combinations for scenario s1 on the other hand, regarding maximum time durations, the same combination of heuristics, c4, provided a better impact on scenario s2 (table 25) by improving the maximum total time of the process by 37.4%. in terms of patients attended, considering combination c4 as the reference, on scenario s3, 85.7% of patients that entered in the ed were successfully attended. alternatively, on scenarios s1 and s2, the percentage of patients attended was 89.9% and 90.6%. scenario s2 ic is overall times of the process variation on overall times of process regarding as-is max. time(h) avg. time(h) max. time avg. time as-is 202 233 5h56m 1h55m h1 203 233 5h16m 1h48m -11.2% -6.1% h2 206 233 5h38m 1h48m -5.1% -6.1% h3 206 233 4h10m 1h40m -29.8% -13.0% c1 206 233 5h15m 1h44m -11.5% -9.6% c2 209 233 4h1m 1h36m -32.3% -16.5% c3 209 233 3h56m 1h32m -33.7% -20.0% c4 211 233 3h43m 1h31m -37.4% -20.9% ic='instances completed'; is='instances started' table 25. final results of all combinations for scenario s2 given the simulation data, one can see that the best combination on all scenarios is the c4, which is the combination of the three redesign heuristics. thus, one can claim that regarding australasian journal of information systems pereira, lapão, bianchi & amaral 2020, vol 24, research article business process redesign in emergency departments 17 the problems identified and the improvements proposed, the c4 can be considered as the process to-be. 6 conclusion three scenarios were simulated in this research taking in consideration the ed context of a public hospital. each considering a higher rate of patients per hour than the previous one. s1 has a lower rate and s3 the highest rate. each of the three heuristics were tested as well as all the possible combinations of them (table 16). conclusions vary regarding the scenario and the combination of heuristics tested. main conclusions are: • h1 has a major impact on the maximum los of s2 and on the average los of s3. overall, h1 seems to have better results with higher demand of the ed. • similarly, h2 also has a major impact on the maximum los of s2 and on the average los of s3. however, it appears to negatively impact the maximum los of s3. • h3 has a considerably positive impact on both the maximum los and average los in s1, s2 and s3 (but in this one considerably less than the others). s2 seems to be the scenario that benefits the most from the implementation of this heuristic. • when analysed alone, h3 seems to be the most beneficial heuristic to implement. • as well as all the heuristics alone, c1 gives more benefits with higher demand and less benefits with lower demand. therefore, s1 is the worst scenario and s3 the most beneficial. • on the contrary, c2 and c3 work better with lower demand and benefits decrease as demand increases. • c4 inverts again and seems to be better for higher demand. • h3, which is present in c2, c3 and c4, seems to be the most beneficial heuristic for ed. it is curious given the nature of the heuristic (integral technology). this demonstrates the importance of technology in ed. • nevertheless, the combination of heuristics points to be more effective instead of implementing heuristics alone. • bpmn proved to be a very useful tool for modelling and simulating the ed process under study. on process redesign, the redesign heuristics were properly chosen based on the objectives of the research and the processual improvements and problems pinpointed. the results of the application of each redesign heuristic were analysed and compared with results of process simulation of the process as-is. the combination of redesign heuristics that better improves the process, in terms of time (and without deteriorating the quality), was selected to be implemented and the process to-be was defined. the analysed process proved to be optimised in terms of time duration after the application of the chosen redesign heuristics. the quality, at least, remained the same. on the other hand, the process of data gathering was not easy. the data gathered was not as lean as desired and australasian journal of information systems pereira, lapão, bianchi & amaral 2020, vol 24, research article business process redesign in emergency departments 18 it was not possible to get all the information that we were looking for. the data obtained was incomplete. this investigation also has some limitations. first one may bear in mind that cost was not one of the devil’s quadrangle vectors chosen by the stakeholders and the simulated heuristics are aligned with the chosen vectors. regarding h1 and h2, increasing the resources may increase costs with salaries and require more physical space. another limitation is the counterpart that some heuristics would require to be implemented. finding more medical staff would not be a problem. indeed, portugal has been hiring medical staff from other countries to fulfil internal demand with less investment. while cost may not truly be a problem in this scenario, the physical space is one of the problems pointed by the stakeholders (p5 in table 15) that can actually be a barrier for the implementation of these heuristics, as many hospitals are aged and do not easily cope with modern demands. this research represents an important tool to assist any ed with a similar context of the one explored. professional may use as a decision support tool while academics may see it as a former deep approach of bpm theory in healthcare industry and use it as motivation for further research. in this way, the findings of the present research are only applicable to the specific ed process studied in the context of a specific public hospital. but further investigation on this topic should be incentivised to generalise and complement the findings as far as possible, as well as to other ed components. the ed presents many areas that could benefit from improving heuristics, and less dependable of human resources: the improvement of triage processes with a larger focus on prioritization; to improve quick response to well-known emergency situations, like stroke or sepsis; and to improve operating room usage and quality standards, by addressing operating processes to allow for more efficient use of resources and reducing error. another interesting possibility is to compare between ed. additionally, we propose a deepened research on an ed taking into consideration the four constraints of devil’s quadrangle. the paper’s findings, though not being generalised so far, may still offer astute insights to practitioners or researchers who wish to consider more information regarding the importance of improvement of ed through the application of business process redesign heuristics. references baumlin, k. m., shapiro, j. s., weiner, c., gottlieb, b., chawla, n., & richardson, l. d. (2010). clinical information system and process redesign improves emergency department efficiency. the joint commission journal on quality and patient safety, 36(4), 179-ap1. https://doi.org/https://doi.org/10.1016/s1553-7250(10)36030-2 benbasat, i., goldstein, d. k., & mead, m. (1987). the case research strategy in studies of information systems. mis quarterly, 11(3), 369. https://doi.org/10.2307/248684 bizagi. (2018). bizagi. retrieved from www.bizagi.com bromley, p. d. b. (1990). academic contributions to psychological counselling. 1. a philosophy of science for the study of individual cases. counselling psychology quarterly, 3(3), 299– 307. https://doi.org/10.1080/09515079008254261 buzacott, j. a. (1996). commonalities in reengineered business processes: models and issues. management science, 42(5), 768–782. https://doi.org/10.1287/mnsc.42.5.768 australasian journal of information systems pereira, lapão, bianchi & amaral 2020, vol 24, research article business process redesign in emergency departments 19 davenport, t. h. (1993). process innovation: reengineering work through information technology. harvard business school press. retrieved from https://books.google.com.br/books?id=9qahngeacaaj de ramón fernández, a., ruiz fernández, d., & sabuco garcía, y. (2019). business process management for optimizing clinical processes: a systematic literature review. health informatics journal. https://doi.org/10.1177/1460458219877092 dickman, d. k., tong, a., & davis, g. w. (2012). snapin is critical for presynaptic homeostatic plasticity. journal of neuroscience, 32(25), 8716–8724. https://doi.org/10.1523/jneurosci.5465-11.2012 dumas, m., la rosa, m., mendling, j., & reijers, h. a. (2018). fundamentals of business process management. berlin, heidelberg: springer berlin heidelberg. https://doi.org/10.1007/9783-662-56509-4 feibert, d. c., andersen, b., & jacobsen, p. (2019). benchmarking healthcare logistics processes – a comparative case study of danish and us hospitals. total quality management & business excellence, 30(1–2), 108–134. https://doi.org/10.1080/14783363.2017.1299570 feibert, d. c., & jacobsen, p. (2019). factors impacting technology adoption in hospital bed logistics. the international journal of logistics management, 30(1), 195–230. https://doi.org/10.1108/ijlm-02-2017-0043 garmann-johnsen, n. f., & eikebrokk, t. r. (2017). dynamic capabilities in e-health innovation: implications for policies. health policy and technology, 6(3), 292–301. https://doi.org/10.1016/j.hlpt.2017.02.003 gary, t. (2016). how to do your case study (2nd ed.). singapore: sage publications. hammer, m., & champy, j. (1993). reengineering the corporation: a manifesto for business revolution (vol. 10). https://doi.org/10.1002/qre.4680100613 hassan, m. m. d. (2017). an application of business process management to health care facilities. the health care manager, 36(2), 147–163. https://doi.org/10.1097/hcm.0000000000000149 holden, r. j. (2011). lean thinking in emergency departments: a critical review. annals of emergency medicine, 57(3), 265–278. https://doi.org/https://doi.org/10.1016/j.annemergmed.2010.08.001 hoot, n. r., & aronsky, d. (2008). systematic review of emergency department crowding: causes, effects, and solutions. annals of emergency medicine, 52(2), 126-136.e1. https://doi.org/10.1016/j.annemergmed.2008.03.014 ippoliti, e. (2015). heuristic reasoning. (e. ippoliti, ed.). springer international publishing. https://doi.org/10.1007/978-3-319-09159-4 jansen-vullers, m., & reijers, h. (2005). business process redesign in healthcare: towards a structured approach. infor: information systems and operational research, 43(4), 321– 339. https://doi.org/10.1080/03155986.2005.11732733 kelly, a.-m., bryant, m., cox, l., & jolley, d. (2007). improving emergency department efficiency by patient streaming to outcomes-based teams. australian health review : a australasian journal of information systems pereira, lapão, bianchi & amaral 2020, vol 24, research article business process redesign in emergency departments 20 publication of the australian hospital association, 31(1), 16–21. retrieved from http://www.ncbi.nlm.nih.gov/pubmed/17266484 ko, y., zhai, c., & rodriguez-zas, s. (2009). inference of gene pathways using mixture bayesian networks. bmc systems biology, 3(1), 54. https://doi.org/10.1186/1752-0509-3-54 kumar, a., & shim, s. j. (2007). eliminating emergency department wait by bpr implementation. in 2007 ieee international conference on industrial engineering and engineering management (pp. 1679–1683). ieee. https://doi.org/10.1109/ieem.2007.4419478 lapão, l. v. (2016). lean in the health management: an opportunity to improve focus on the patient, respect for professionals and quality in the health services. acta médica portuguesa, 29(4), 237. https://doi.org/10.20344/amp.6615 leggat, s. g., bartram, t., stanton, p., bamber, g. j., & sohal, a. s. (2015). have process redesign methods, such as lean, been successful in changing care delivery in hospitals? a systematic review. public money & management, 35(2), 161–168. https://doi.org/10.1080/09540962.2015.1007714 leu, j.-d., & huang, y.-t. (2011). an application of business process method to the clinical efficiency of hospital. journal of medical systems, 35(3), 409–421. https://doi.org/10.1007/s10916-009-9376-4 leva, a. di, sulis, e., & vinai, m. (2017). business process analysis and simulation: the contact center of a public health and social information office. intelligent information management, 09(05), 189–205. https://doi.org/10.4236/iim.2017.95010 pereira, r., almeida, r., & mira da silva, m. (2013). how to generalize an information technology case study. in 8th design science research in information systems and technology (desrist) (vol. 7939 lncs). https://doi.org/10.1007/978-3-642-38827-9_11 pestana, m., pereira, r., & moro, s. (2020). improving health care management in hospitals through a productivity dashboard. journal of medical systems, 44(4), 87. https://doi.org/10.1007/s10916-020-01546-1 pordata. (2018). pordata esperança de vida à nascença: total e por sexo. retrieved september 17, 2018, from https://www.pordata.pt/europa/esperança+de+vida+à+nascença+total+e+por+sexo1260 reijers, h. a., & liman mansar, s. (2005). best practices in business process redesign: an overview and qualitative evaluation of successful redesign heuristics. omega, 33(4), 283– 306. https://doi.org/10.1016/j.omega.2004.04.012 şahin, h., duran, s., yakıcı, e., & şahin, m. (2018). patient classification considering the risk of restenosis after coronary stent placement. journal of heuristics. https://doi.org/10.1007/s10732-018-9394-9 sherwood-smith, m. (1994). people centred process re-engineering: an evaluation prespective to office systems to re-design. in proceedings of the ifip tc8 open conference on business process re-engineering: information systems opportunities and challenges (pp. 535–544). new york, ny, usa: elsevier science inc. retrieved from http://dl.acm.org/citation.cfm?id=646303.686970 australasian journal of information systems pereira, lapão, bianchi & amaral 2020, vol 24, research article business process redesign in emergency departments 21 testa, a., cinque, m., coronato, a., de pietro, g., & augusto, j. c. (2015). heuristic strategies for assessing wireless sensor network resiliency: an event-based formal approach. journal of heuristics, 21(2), 145–175. https://doi.org/10.1007/s10732-014-9258-x van der aalst, w. m. p. (2013). business process management: a comprehensive survey. isrn software engineering, 2013, 1–37. https://doi.org/10.1155/2013/507984 van der aalst, w. m. p., ter hofstede, a. h. m., & weske, m. (2003). business process management: a survey (pp. 1–12). https://doi.org/10.1007/3-540-44895-0_1 vom brocke, j., schmiedel, t., recker, j., trkman, p., mertens, w., & viaene, s. (2014). ten principles of good business process management. business process management journal, 20(4), 530–548. https://doi.org/10.1108/bpmj-06-2013-0074 webster, j., & watson, r. t. (2002). analyzing the past to prepare for the future: writing a literature review. mis quarterly, 26(2), xiii–xxiii. https://doi.org/10.1.1.104.6570 wüllenweber, k., beimborn, d., weitzel, t., & könig, w. (2008). the impact of process standardization on business process outsourcing success. information systems frontiers, 10(2), 211–224. https://doi.org/10.1007/s10796-008-9063-x yin, r. k. (2009). case study research and applications: design and methods (5th ed.). thousand oaks, ca, usa: sage publications. yin, r. k. (2013). case study research: design and methods. sage publications. zucker, d. (2009). how to do case study research. teaching research methods in the social sciences. retrieved from https://scholarworks.umass.edu/nursing_faculty_pubs/2 copyright: © 2020 pereira, lapão, bianchi & amaral. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v24i0.2679 http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems sharma, singh & pratt 2020, vol 24, research article online travel purchases across generations 1 does consumers' intention to purchase travel online differ across generations? empirical evidence from australia shavneet sharma university of the south pacific fiji shavneet.sharma@usp.ac.fj gurmeet singh graduate school of business university of the south pacific fiji stephen pratt school of tourism & hospitality management university of the south pacific fiji abstract this paper examines the differences in consumers' intention to purchase travel online across millennials and baby boomers. the unified theory of acceptance and use of technology 2 (utaut2) is extended by including attitude, compatibility, innovativeness, perceived trust, and perceived risk variables. data is collected using online questionnaires from millennials (n=322) and baby boomers (n=318) in australia. overall, the empirical results revealed that factors affecting australian consumers' intention to purchase travel online differ across generations. this study contributes to the literature by extending and testing the comprehensive research model to understand consumers' online travel purchase behaviour better. keywords: generational cohorts; technology acceptance; online purchase; travel; millennials; baby boomers; utaut2; australia 1 introduction the past decade has seen a drastic increase in internet connectivity and usage, which has reshaped tourist purchase behaviour (buhalis & law, 2008). global users of the internet have increased from 16 million is 1995 to approximately 4 billion users by 2018 (paridahdaud & fung, 2018). despite this, there are gaps or differences between information and communication technology (ict) usage between user groups. this 'digital divide' is influenced by socioeconomic variables such as gender, level of education, generation, and age (friemel, 2016). while some scholars argue that the digital divide between the younger and older cohorts is decreasing (gilleard et al., 2015; nishijima et al., 2017), especially in the case of developed countries, the literature suggests that consumers' online behaviour may still differ between these cohorts (confente & vigolo, 2018). age has been a determining factor in consumers' intentions and use of online shopping with younger consumers found to be more likely to shop online (khare et al., 2012). literature shows that market segmentation is more efficient through generational cohorts than by age (schewe et al., 2000). this is because cohort australasian journal of information systems sharma, singh & pratt 2020, vol 24, research article online travel purchases across generations 2 segmentation provides both the insights into factors motivating consumers, which stem from common beliefs and values shared by the generation from coming-of-age during a particular period of history (kumar & sadarangani, 2018). this study investigates consumers' intention to purchase travel online across generational cohorts. the study compares millennials (also referred to as generation y) and baby boomers. these two cohorts were selected because these generations respectively represent both 'digital natives' and 'digital immigrants' (coombes, 2009). digital natives are more comfortable with using and learning technology as they have always interacted with this medium when compared to digital immigrants who only learned to use this technology as adults (sait et al., 2004). these two groups also yield considerable purchasing power (beauchamp & barnes, 2015). as such, marketers see vast potential when it comes to capturing sales from these two important consumer groups. the contribution of this paper is threefold. first, the utaut2 technology acceptance model will be applied to compare millennials and baby boomers' intention to purchase travel products online. this model has primarily been used to examine applications, devices, and website acceptance in the tourism industry (gupta & dogra, 2017; gupta et al., 2018). the utaut2 model has not been previously used in the tourism context to shed light on generational differences between millennials and baby boomers to purchase travel online. the intergenerational examination herein also answers the call by lim (2018b) and mazaheri et al. (2020) for greater research in this area in order to provide a cross-comparison update to concepts and theories in information systems and technology (e.g. utaut) that were conceived when digital technologies in applied settings were only emerging. second, this study extends the utaut2 model with the inclusion of additional factors important in predicting online travel behaviour. literature shows that there is little empirical work that simultaneously captures the positive (success) and negative (resistance) factors that lead consumers to adopt ict (alalwan et al., 2018; hult et al., 2019; lee, 2009; martins et al., 2014; zhuang et al., 2018). the academic fraternity has always given less attention to behaviour deterrents (rehman et al., 2020) and focus primarily being on the drivers (farivar et al., 2017). therefore, given the importance of risk in e-commerce (pelaez et al., 2019; samhan, 2018), perceived risk (pr) has been added to the utaut2 model. also, dwivedi et al. (2019) found that the addition of attitude to the original utaut model would increase its predictive power. however, since then, little research has been carried out with the addition of this construct to the utaut model. this study is one of the first to incorporate attitude to the utaut2 model. additionally, compatibility, innovativeness, and perceived trust are also added to the utaut2 model as these factors generally differ across individuals. the addition of constructs to extend technology acceptance models such as utaut is in line with the recommendations by lim (2018a) as they exemplify novel and meaningful extensions that encapsulate the emerging realities of user interactions with technology in underexplored settings (e.g. tourism). third, in the tourism literature, to explore the similarities and differences amongst tourism behaviour across generations, the concept of cohorts has been used in segmentation studies (gardiner et al., 2015; kim et al., 2018). also, most generational studies have focused on travel behaviour of a specific generation, such as generation x or generation y (chen & chou, 2019; xu & pratt, 2018), but few have compared online behaviour between generations (e.g., beldona et al., 2009; gardiner et al., 2015). despite this, very little research has been carried out that australasian journal of information systems sharma, singh & pratt 2020, vol 24, research article online travel purchases across generations 3 differentiates generational cohorts by online shopping that focuses on cohort perceptions, decision making, and behaviour (herrando et al., 2019; lissitsa & kol, 2019; shulga et al., 2018). despite there been a huge growth in studies relating to tourism (athiyaman, 1995), research is still limited when it comes to the behaviour and impact of millennials in the tourism industry (ramsay et al., 2017). the results of this study will help scholars and practitioners better understand the generational differences in accepting and using online travel websites. 2 literature review, conceptual model and hypotheses 2.1 generational cohorts a generational cohort can be defined by the years of birth, extending 20–25 years in duration, or as long as it generally takes one birth group to be born, age, and have children of their own (meredith & schewe, 1994; strauss & howe, 1991). these cohorts share similar ideas, attitudes, beliefs, and values based on being born during the same time period and living through common experiences, with macro-level economic, political, and social events that occurred during their coming of age years (strauss & howe, 1991). generational cohort theory is useful in identifying propensities and patterns across generational groups as a socio-cultural theoretical framework (pendergast, 2010). in tourism studies, several researchers have examined generational cohort consumer purchase behaviour (bakewell & mitchell, 2003) and tourism behaviour (pennington-gray et al., 2003). despite tourism research identifying characteristics that are similar and different for each generation, there is a call for a deeper understanding of each generational cohort and its impact on the industry (bowen & chen mccain, 2015). a fair amount of evidence grounded in social science has been provided that demonstrates the generational approach to understanding groups and society. specifically, consumer segmentation by behaviours, attitude, and preferences can best be understood through generational analysis (parment, 2013). tourism research has widely recognized the importance of profiling consumers by cognitive processes, psychological characteristics, and behavioural patterns (lee et al., 2004). various variables, factors, and descriptors have been used to profile consumer segments in the tourism industry: holiday and decision-making behaviour (bicikova, 2014), electronic word-of-mouth (zhang et al., 2017), and purchase behaviour (beldona et al., 2009). within this line of research, this study seeks to understand consumer segments' intention to purchase travel online. 2.2 unified theory of acceptance and use of technology (utaut) venkatesh et al. (2012) proposed an extension to the original utaut model which included three additional constructs: hedonic motivation, price value and habit to the existing four original contracts of performance expectancy (pe), effort expectancy (ee), social influence (si) and facilitating conditions (fc). when it comes to consumer technology adoption, the utaut2 model has a better predictive power (venkatesh et al., 2012) and has contributed significantly to the literature on consumers' adoption of technology (tamilmani et al., 2019). this model has been successfully applied in various contexts, including mobile payments (morosan & defranco, 2016) and tourist adoption of mapping applications (gupta & dogra, 2017). 2.3 performance expectancy extant research confirms that consumers are more likely to use a technology that brings them more favourable outcomes and is more useful to the users (compeau & higgins, 1995). it has australasian journal of information systems sharma, singh & pratt 2020, vol 24, research article online travel purchases across generations 4 been found that performance expectancy has a strong positive association with behavioural intention (escobar-rodríguez & carvajal-trujillo, 2014; smith et al., 2013), mobile-based communication technologies (marriott & williams, 2018), social media applications (huang et al., 2018; lim et al., 2019) and mobile wallets (madan & yadav, 2016; slade et al., 2014). thus, it is hypothesized that: h1a: performance expectancy positively influences consumers' attitude to purchase travel online. h1b: performance expectancy positively influences consumers' intention to purchase travel online. 2.4 effort expectancy individuals prefer to use technology that provides maximum benefits and is easy to understand (davis, 1989). the ease and comfort in relation to the use of a system are referred to as effort expectancy (venkatesh et al., 2003). several studies have shown that effort expectancy is positively related to behavioural intention (awwad & al-majali, 2015; khalilzadeh et al., 2017; wang et al., 2017). therefore, it is hypothesized that: h2a: effort expectancy positively influences consumers' attitude to purchase travel online. h2b: effort expectancy positively influences consumers' intention to purchase travel online. 2.5 social influence when an individual perceives that a given behaviour shall be accepted by their peer group, the likelihood of them engaging in the given behaviour increases (ajzen, 1991). the extent to which an individual believes that others expect them to use a new system is referred to as social influence (venkatesh et al., 2003). studies carried out have found that social influence has a significant influence on behavioural intention (hsu & lin, 2016; kim & park, 2011; ozturk et al., 2016; wu & chen, 2015). davis (1989) believed that subjective norm had no influence on technology acceptance, while shin (2009) has hypothesized that social influence has a positive effect on the intention to utilize mobile payments, although the effect of social influence on intention was not supported. nakagawa and gouvêa (2010) have found that social influence is not relevant to the intention to adopt internet shopping. nevertheless, venkatesh et al. (2003) believed that social influence affected consumer behaviour. besides, chen et al. (2007) have found that subjective norms toward using electronic toll collection positively increase the intention to use. thus, the following hypothesis is proposed: h3a: social influence positively influences consumers' attitude to purchase travel online. h3b: social influence positively influences consumers' intention to purchase travel online. 2.6 facilitating conditions in this study, facilitating conditions reflects the effect of required knowledge and necessary resources (such as internet connectivity) to engage in travel purchases online. the relationship between facilitating condition and behavioural intention has been confirmed in various contexts, including nfc mobile payments (khalilzadeh et al., 2017), biometric technology adoption (akinnuwesi et al., 2016), mhealth (hoque & sorwar, 2017) and internet banking (tarhini et al., 2016). therefore, this research hypothesizes that: h4a: facilitating condition positively influences consumers' attitude to purchase travel online. australasian journal of information systems sharma, singh & pratt 2020, vol 24, research article online travel purchases across generations 5 h4b: facilitating condition positively influences consumers' intention to purchase travel online. 2.7 hedonic motivation 'hedonic motivation' is defined as the fun or pleasure derived from using technology, and it is an important determinant of consumers' technology acceptance and use (tamilmani et al., 2019). the inclusion of this factor into the utaut2 model shifts the focus from extrinsic motivation as a dominant paradigm of technology adoption research to intrinsic motivation. applications of the utaut2 has found hedonic motivation as a strong predictor in mobile apps (hew et al., 2015), elearning intention (filippou et al., 2018), online purchase (escobarrodríguez & carvajal-trujillo, 2014) and nfc technology adoption. therefore, the following is hypothesized: h5a: hedonic motivation positively influences consumers' attitude to purchase travel online. h5b: hedonic motivation positively influences consumers' intention to purchase travel online. 2.8 price-saving various industries, including tourism, have offered price-saving (ps) incentives to consumers for engaging in online purchases. consumers look for higher perceived benefits in comparison to the monetary sacrifice, which makes price value a significant predictor of behavioural intention to use technology. consumers who engage in online shopping also consider costsaving a significant factor (jensen, 2012). thus, price-saving orientation has been observed as a significant predictor of purchase intention (pi) for online bookings (escobar-rodríguez & carvajal-trujillo, 2014) and airline e-commerce websites (escobar-rodríguez & carvajaltrujillo, 2013). therefore, it is hypothesized that: h6a: price saving positively influences consumers' attitude to purchase travel online. h6b: price saving positively influences consumers' intention to purchase travel online. 2.9 habit habit is defined as the extent to which an individual tends to perform behaviours automatically because of learning (limayem et al., 2007). habit reveals the outcome of past experiences or behaviour (venkatesh et al., 2012). research has shown that habit is a significant predictor of behavioural intention (escobar-rodríguez & carvajal-trujillo, 2014; gupta & dogra, 2017; herrero & san martín, 2017; hsiao et al., 2016). based on these findings, it is hypothesized that: h7a: habit positively influences consumers' attitude to purchase travel online. h7b: habit positively influences consumers' intention to purchase travel online. 2.10 perceived compatibility the construct of perceived compatibility is defined as "the degree to which an innovation is perceived as being consistent with existing values, past experiences, and needs of potential adopters" (rogers, 1995, p. 15). the following definition of compatibility will be employed for this study as it fits online shopping context; it is the extent to which consumers believe that purchasing travel online matches/fits their needs, shopping preference, and lifestyle (vijayasarathy, 2004). according to bellman et al. (1999), those individuals that spend a considerable about of time in their personal life and job with internet and related technologies australasian journal of information systems sharma, singh & pratt 2020, vol 24, research article online travel purchases across generations 6 such as e-email have a higher likelihood to purchase online. the review of the literature shows that individuals have found a significant and positive impact of perceived compatibility on customers' attitudes to purchase online (amaro & duarte, 2015; gillenson & sherrell, 2002; vijayasarathy, 2004). also, christou and kassianidis (2002), (li & buhalis, 2006) and amaro and duarte (2015) found that perceived compatibility positively influenced customers' intention to purchase travel online. therefore, the following is hypothesized: h8: perceived compatibility positively influences consumers' intention to purchase travel online. 2.11 customer innovativeness innovativeness has been defined by rogers and shoemaker (1971) as "the degree to which an individual is relatively early in adopting new ideas than other members of his social system." according to the approach proposed by bass (1969) and midgley and dowling (1978), criticizing this temporal perspective as it ignores the communication and social processes that characterize the diffusion of innovation, and they affirm that this concept reflects the "degree to which the individual is receptive to new ideas and makes innovative decisions independently of the communicated experience of others." the concept of innovativeness has been predominantly shown in electronic commerce literature as a domain-specific variable (goldsmith, 2000, 2002; park & jun, 2002; varma citrin et al., 2000). particularly in the domain of information technology, agarwal and prasad (1998) proposed the concept of perceived innovativeness, which is defined as the willingness of an individual to try out any new information technology. following this, different authors have provided empirical evidence to support the effect of a user's innovativeness in the acquisition of products using the internet (donthu & garcia, 1999; goldsmith, 2000; varma citrin et al., 2000), frequency of acquisition (goldsmith, 2000; goldsmith & lafferty, 2001) and online purchase intention in the future (aldás-manzano et al., 2009; crespo & del bosque, 2008; goldsmith, 2002; goldsmith & lafferty, 2001; lu et al., 2011). other research carried out by san martín and herrero (2012) found that users' innovativeness had a positive influence on online purchase intention in rural tourism. therefore, the following hypothesis can be developed: h9: innovativeness positively influences consumers' intention to purchase travel online. 2.12 perceived risk due to the separation of the buyer and the seller, the concept of perceived risk becomes more common for online shopping (al-gahtani, 2011). perceived risk represents a detractor to the adoption process (gupta et al., 2018). a study by athiyaman (2002) found that consumers avoid purchasing airline tickets online because of security concerns. several studies support the relationship between perceived risk and behavioural intention (liébana-cabanillas et al., 2017; lu et al., 2011; yang et al., 2012) while others reject the relationship (kapoor et al., 2015; tan et al., 2014). therefore, this research will be of theoretical use. thus, the following is proposed: h10: perceived risk negatively influences consumers' intention to purchase travel online. 2.13 2.13 perceived trust studies carried out previously have confirmed the positive link between perceived trust and intention to purchase online in the context of e-commerce (chiu et al., 2010; gefen et al., 2003; australasian journal of information systems sharma, singh & pratt 2020, vol 24, research article online travel purchases across generations 7 kim et al., 2012). in the study conducted by wen (2010), a positive relationship was found between perceived consumer trust and consumers' intention to purchase travel online. however, kamarulzaman (2007) did not find a direct effect of trust on consumers' adoption of online travel shopping. in the tourism field, this relationship between trust on online travel shopping adoption has been supported by bigné et al. (2010); escobar-rodríguez and carvajaltrujillo (2014); kim et al. (2011); amaro and duarte (2015); ponte et al. (2015) and agag and el-masry (2017). therefore, the following hypothesis is proposed: h11: perceived trust positively influences consumers' intention to purchase travel online. 2.14 attitude prior literature has acknowledged the role of attitude in explaining individuals' acceptance of technology (bobbitt & dabholkar, 2001; dwivedi et al., 2017; kim et al., 2009; taylor & todd, 1995; yang & fang, 2004). the inclusion of attitude as a construct in technology acceptance models is consistent with tra (ajzen & fishbein, 1980; fishbein & ajzen, 1975), tpb (ajzen, 1991), and dtpb (taylor & todd, 1995). tam is considered a special case of the tra with an attitude comprising of two beliefs. according to the tra model, attitude plays a mediating role in the relationship between the types of beliefs and intentions (taylor & todd, 1995). tam postulates that technology is considered more useful when it is easier to use, and this will lead to a more positive attitude and intention for an individual to use the technology (davis et al., 1989; taylor & todd, 1995). the relationship between attitude and behavioural intention, as represented in tam, suggests that all other things being equal, individuals with a positive attitude towards a behaviour will form the intention to perform the behaviour. this relationship is key to the tra and other related models that researchers have presented (e.g., (triandis, 1977), (bagozzi, 1981) and (davis et al., 1989) while developing the tam extension (tame) model, (jackson et al., 1997). therefore, it is hypothesized that: h12: attitude positively influences consumers' intention to purchase travel online. 2.15 purchase behaviour purchase behaviour is "the manifest, observable response in a given situation concerning a given target" (ajzen, 1991). bi is considered an immediate consequence and indicates an individuals' readiness to perform a specific action. in the information technology acceptance studies, various studies have provided evidence of the relationship between bi and pb (e.g., (baabdullah et al., 2019; gupta et al., 2018; sharma et al., 2020; venkatesh et al., 2012). therefore, it is hypothesized that: h13: bi positively influences consumers' purchase behaviour to purchase travel online. australasian journal of information systems sharma, singh & pratt 2020, vol 24, research article online travel purchases across generations 8 figure 1: conceptual framework 3 methodology 3.1 survey instrument a questionnaire was designed to capture data to address the research hypothesis. the first part of the questionnaire captured demographic variables such as gender, age, education level, and income. the age variable was used to segment the sample into the respective generations. a respondent was deemed a millennial if they were between the ages of 19-37 and a baby boomer if they were between the ages of 55-73 (suh et al., 2017). other demographic variables, such as gender, were used to assess the representativeness of the sample. the second part of the questionnaire captured variables used in the utaut2 model and was adopted from previous literature. these variables were all asked on a 5-point likert scale were '1' represented 'strongly disagree’, and '5' represents 'strongly agree.' prior to carrying out the main survey, the questionnaire was piloted with 30 consumers consisting of 15 millennials and 15 baby boomers. this resulted in some minor improvements being made to the item wording to enhance the readability and flow of the questions in the survey. 3.2 data collection and sample an online survey was conducted in australia between march-april 2019. a professional data collection firm was contracted to collect data for this research using an online survey. a similar approach has been used in other studies (nam et al., 2018; njite & schaffer, 2017). as this study aims to explore factors affects consumers’ intention to purchase travel online, online data australasian journal of information systems sharma, singh & pratt 2020, vol 24, research article online travel purchases across generations 9 collection is considered relevant. this is because respondents are expected to be comfortable with the use of computers and internet to purchase travel online. the data collection firm collected data through a consumer panel for the final survey. data firms maintain extensive data sets of different categories of respondents, which they use as their sample frame. to participate, respondents were required to meet the following screening criteria: 1) be an australian citizen, 2) to be more than 18 years of age. in marketing literature, an online data collection method is quite extensively used (arnett et al., 2018; dwivedi et al., 2016; mcbride et al., 2020). the main survey received 1000 responses. generational cohorts were measured using the birth year of respondents (tsaur & yen, 2018). a part of the quality control procedure of the data collection firm, the demographic characteristics such as age gender, location were verified by the firm. the respondents were then grouped into three categories: baby boomers, generation xers, and millennials. as this study aims to assess behavioural differences between baby boomers and millennials, all respondents that did not belong to these two categories were screened out. a total of 562 responses remained. of this, 560 were completely filled out (millennials n= 341; baby boomers n=309). the demographic profiles of each cohort are presented in table 1. the sample for millennials and baby boomers is weighted towards males as well as educated people who might already be interested in purchasing travel online (see table 1 for details). however, we consider the profile of millennials and baby boomers as representative of those taking online surveys (duffy et al., 2005). 3.3 data analysis procedure this study used spss 25 for descriptive statistics and amos 24 for structural equation modelling (sem). the measurement model was evaluated for goodness of fit (goodness of fit index), reliability, validity, and measurement invariance. validity was assessed using convergent validity (ave), construct validity (gfi, cfi (comparative fit index), rmsea (root mean square error approximation), chi-square/degree of freedom), and discriminant validity. to assess the convergent validity, j. hair et al. (2006) recommended using factor loadings, composite reliability, and average variance extracted (ave). factor loadings ≥ 0.5, and preferably ≥ 0.70, show high convergent validity. discriminant validity was assessed using latent constructs correlations matrices, where the square roots of the aves along the diagonals were indicated. discriminant validity is realized when the diagonal elements (square roots of aves) exceed the off-diagonal elements (correlations between constructs) in the same row and column (fornell & larcker, 1981b). the criteria to assess these statistics was p ≥ 0.05, the 2 / df ≤2.00, [rmsea] ≤ 0.05, cfi ≥ 0.90, gfi ≥ 0.90, adjusted goodness of fit [agfi] ≥ 0.90, and root mean square residual [rmr]. the structural model was assessed by studying the r² determination coefficients, regression estimates, and statistical significance. the r² value exemplifies an amount of prognostic power and shows the extent of divergence, justified by its antecedent variables in the model. chin (1998) considered r² values of 0.67, 0.33, and 0.19 as significant, reasonable, and weak, respectively. australasian journal of information systems sharma, singh & pratt 2020, vol 24, research article online travel purchases across generations 10 3.4 demographic profile millennials baby boomers n 341 309 gender female 127 183 male 214 124 rather not say 2 education primary school education 2 1 secondary school 10 70 diploma/certificate 65 105 bachelors education 95 87 postgraduate education 105 45 rather not say 64 1 income i do not earn a fixed income 34 31 under $15,000 16 12 $15,000-$29,999 46 45 $30,000-$44,999 40 37 $45,000-$59,999 49 50 $60,000-$74,999 44 31 $75,000-$89,999 28 19 $90,000 + 48 36 rather not say 36 48 note: aggregate percentage can be slightly more or less than 100 because of rounding off. table 1: profile of respondents 4 results to operationalize the conceptual framework in figure 1 and test the hypotheses proposed in the literature review section, we undertake a confirmatory factor analysis (cfa) using a structural equation model as used in previous studies of this kind (palau-saumell et al., 2016; song et al., 2017). 4.1 descriptive statistics of items and constructs the descriptive statistics of the constructs used in the proposed research model are shown in table 2. on average, the means for most of the items were above 3.23 (out of 5) for the overall sample (n=650), above 3.51 for the millennials' sample (nml= 341) and above 3.16 for the baby boomers' sample (nbb= 309). these results show that the majority of the respondents generally express positive answers to the variables used in the research model. table 2 also shows that the standard deviations ranged from 0.861 to 1.211 for the overall sample, from 0.731 to 1.228 for the millennials' sample and from 0.916 to 1.194 for the baby boomers' sample. this indicates a narrow spread around the mean. australasian journal of information systems sharma, singh & pratt 2020, vol 24, research article online travel purchases across generations 11 single group cfa overall sample millennials sample baby boomers sample χ2 3618.082 3333.921 3112.882 df 1133 1133 1133 χ2 /df 3.193 2.943 2.747 rmsea 0.043 0.0381 0.0451 rfi 0.947 0.915 0.923 ifi 0.934 0.933 0.924 nfi 0.893 0.896 0.904 tli 0.929 0.927 0.916 cfi 0.933 0.911 0.902 table 2: the results of confirmatory factor analyses sl cr smc sl cr smc sl cr smc pe pe1 0.9 28.92 0.82 0.89 17.13 0.78 0.92 0.84 23.7 pe2 0.86 26.38 0.74 0.81 15.21 0.66 0.9 0.81 22.61 pe3 0.89 28.26 0.8 0.87 16.68 0.75 0.91 0.82 23.09 pe4 0.84 0.7 0.79 0.62 0.87 0.76 ee ee1 0.88 27.42 0.78 0.87 19.11 0.75 0.9 0.82 20.16 ee2 0.87 26.9 0.76 0.82 17.4 0.67 0.91 0.84 20.57 ee3 0.91 28.78 0.82 0.89 20 0.79 0.91 0.83 20.37 ee4 0.84 0.7 0.85 0.73 0.83 0.69 si si1 0.89 34.28 0.8 0.85 20.81 0.72 0.93 0.87 28.1 si2 0.93 37.59 0.86 0.91 24.51 0.83 0.94 0.88 28.66 si3 0.92 0.84 0.92 0.84 0.92 0.84 fc fc1 0.76 21.74 0.58 0.73 14.11 0.54 0.79 0.62 16.61 fc2 0.87 26.61 0.75 0.86 17.9 0.74 0.87 0.76 19.51 fc3 0.85 0.73 0.85 0.72 0.86 0.73 hm hm1 0.91 30.11 0.82 0.85 17.86 0.73 0.95 0.9 25.43 hm2 0.92 30.87 0.85 0.87 18.37 0.75 0.95 0.91 25.67 hm3 0.86 0.74 0.85 0.72 0.87 0.76 ps ps1 0.85 26.84 0.72 0.78 14.79 0.61 0.9 0.82 25.78 ps2 0.75 21.82 0.56 0.73 13.6 0.54 0.75 0.57 17.28 ps3 0.89 0.78 0.83 0.69 0.94 0.88 hb hb1 0.69 14.81 0.47 0.74 12.03 0.55 0.63 0.39 8.989 hb2 0.89 18.3 0.8 0.9 14.49 0.8 0.89 0.79 11.33 hb3 0.91 18.54 0.83 0.9 14.53 0.81 0.93 0.86 11.53 hb4 0.66 0.43 0.71 0.51 0.59 0.35 cp cp1 0.85 0.73 0.82 0.68 0.87 0.76 cp2 0.91 29.84 0.83 0.88 18.24 0.77 0.93 0.86 23.93 cp3 0.9 29.09 0.81 0.86 17.5 0.73 0.93 0.87 24.26 in in1 0.8 27.52 0.65 0.75 16.65 0.57 0.85 0.72 22.17 in2 0.84 30.47 0.71 0.79 18.37 0.63 0.89 0.79 25.05 in3 0.91 36.79 0.83 0.89 23.25 0.79 0.93 0.86 28.4 in4 0.93 0.87 0.93 0.87 0.93 0.86 pr pr1 0.84 0.71 0.82 0.67 0.86 0.75 pr2 0.94 31.18 0.87 0.91 19.43 0.83 0.95 0.91 24.9 table 3: confirmatory factor analysis results for refined measurement items australasian journal of information systems sharma, singh & pratt 2020, vol 24, research article online travel purchases across generations 12 4.2 measurement model assessment and invariance testing a preliminary single-group cfa was first conducted using the full-sample data (n =650), followed by two separate single-group cfa for each generation. as observed in table 3, the fit indices for all the single-group cfas were within limits recommended by j. f. hair et al. (2006), suggesting that the model fits well separately for the two generations. before conducting the multi-group comparison, it is imperative first to establish that the measures perform adequately in both sub-samples. the findings in the evaluation of the measurement models in the aggregate sample also apply to the generation-specific sub-samples. indicator reliability was verified in both sub-samples. all loadings of the reflective measurement models were significant at the 0.01 level and above the recommended 0.7 threshold (table 4) (nunnally & bernstein, 1994). the estimated indices for composite reliability and cronbach's alpha demonstrate the reliability of the constructs in each sub-sample. the respective ave values were all above the minimum requirement of 0.50 (bagozzi & yi, 1988), confirming convergent validity. discriminant validity was also assured as an inspection of the indicators' cross-loadings. this showed that none of the indicators load higher on an opposing construct. an application of the fornell and larcker (1981a) criterion further confirmed discriminant validity (tables 4, 5, and 6). having established that the measures are valid for both millennials and baby boomers, the next step was the simultaneous estimation of the unconstrained model in the two samples to verify configural invariance. the results show that the configural model fits the data adequately (χ2 = 1769.74; df=696; p < 0.01; χ2/df=2.54; rmsea=0.044; cfi=0.9108; nfi= 0.896; tli=0.925). to test for metrical invariance, we imposed equality constraints on all the factor loadings in the two groups. despite the metric model fitting well (χ2=1660.45; df=799; p<0.01; χ2/df=2.08; rmsea=0.051; cfi=0.928; nfi= 0.91; tli=0.931), a comparison of the χ2 of the configural and the constrained models suggest significant differences (δ χ2=107.85; δ df =106; p < 0.01). australasian journal of information systems sharma, singh & pratt 2020, vol 24, research article online travel purchases across generations 13 α cr ave msv maxr (h) ci at pe ee si fc hm ps hb pr pt cp pi pb ci 0.926 0.93 0.76 0.32 0.94 0.874 at 0.946 0.95 0.78 0.72 0.946 0.409*** 0.882 pe 0.927 0.93 0.76 0.72 0.931 0.405*** 0.846*** 0.873 ee 0.928 0.93 0.77 0.76 0.931 0.361*** 0.663*** 0.762*** 0.875 si 0.937 0.94 0.83 0.47 0.939 0.518*** 0.615*** 0.641*** 0.471*** 0.913 fc 0.865 0.87 0.69 0.76 0.875 0.326*** 0.713*** 0.801*** 0.872*** 0.488*** 0.828 hm 0.922 0.92 0.8 0.6 0.928 0.483*** 0.758*** 0.771*** 0.641*** 0.688*** 0.650*** 0.895 ps 0.861 0.87 0.69 0.72 0.882 0.337*** 0.791*** 0.848*** 0.696*** 0.587*** 0.759*** 0.727*** 0.829 hb 0.877 0.87 0.63 0.36 0.914 0.564*** 0.425*** 0.363*** 0.260*** 0.597*** 0.163*** 0.556*** 0.318*** 0.796 pr 0.935 0.94 0.83 0.15 0.952 0.153*** -0.247*** -0.296*** -0.283*** 0.067 -0.391*** -0.105* -0.314*** 0.258*** 0.912 pt 0.899 0.9 0.69 0.52 0.903 0.316*** 0.661*** 0.676*** 0.570*** 0.511*** 0.627*** 0.682*** 0.715*** 0.331*** -0.196*** 0.831 cp 0.916 0.92 0.79 0.71 0.921 0.403*** 0.694*** 0.747*** 0.635*** 0.589*** 0.690*** 0.678*** 0.728*** 0.465*** -0.283*** 0.720*** 0.887 pi 0.939 0.94 0.75 0.71 0.942 0.335*** 0.727*** 0.784*** 0.663*** 0.559*** 0.730*** 0.659*** 0.766*** 0.343*** -0.321*** 0.697*** 0.843*** 0.869 pb 0.804 0.82 0.6 0.35 0.844 0.465*** 0.528*** 0.518*** 0.396*** 0.592*** 0.401*** 0.507*** 0.443*** 0.533*** 0.112* 0.440*** 0.535*** 0.552*** 0.77 table 4: discriminant validity analysis from confirmatory factor analysis overall sample α cr ave msv maxr (h) ci at pe ee si fc hm ps hb pr pt cp pi pb ci 0.904 0.91 0.71 0.41 0.93 0.844 at 0.925 0.93 0.72 0.68 0.929 0.525*** 0.846 pe 0.902 0.9 0.7 0.72 0.911 0.475*** 0.827*** 0.838 ee 0.916 0.92 0.74 0.72 0.921 0.325*** 0.703*** 0.808*** 0.858 si 0.921 0.92 0.8 0.54 0.927 0.642*** 0.621*** 0.635*** 0.498*** 0.893 fc 0.852 0.86 0.67 0.72 0.868 0.357*** 0.709*** 0.801*** 0.849*** 0.465*** 0.816 hm 0.890 0.89 0.73 0.59 0.891 0.545*** 0.771*** 0.759*** 0.671*** 0.733*** 0.625*** 0.855 ps 0.819 0.82 0.61 0.72 0.83 0.445*** 0.804*** 0.850*** 0.739*** 0.622*** 0.750*** 0.750*** 0.781 hb 0.889 0.89 0.67 0.43 0.913 0.564*** 0.493*** 0.449*** 0.284*** 0.620*** 0.150* 0.657*** 0.383*** 0.816 pr 0.923 0.93 0.81 0.16 0.949 0.232*** -0.076 -0.067 -0.169** 0.232*** -0.273*** 0.079 -0.160* 0.400*** 0.898 pt 0.884 0.89 0.66 0.6 0.891 0.402*** 0.682*** 0.692*** 0.601*** 0.599*** 0.612*** 0.716*** 0.772*** 0.395*** -0.051 0.812 cp 0.887 0.89 0.73 0.78 0.892 0.512*** 0.734*** 0.813*** 0.720*** 0.638*** 0.698*** 0.723*** 0.797*** 0.544*** -0.08 0.760*** 0.853 pi 0.919 0.92 0.7 0.78 0.921 0.364*** 0.705*** 0.792*** 0.677*** 0.564*** 0.689*** 0.641*** 0.774*** 0.403*** -0.133* 0.745*** 0.882*** 0.834 pb 0.811 0.82 0.61 0.48 0.851 0.512*** 0.522*** 0.579*** 0.377*** 0.691*** 0.368*** 0.587*** 0.463*** 0.622*** 0.249*** 0.521*** 0.594*** 0.580*** 0.78 table 5: discriminant validity analysis from confirmatory factor analysis ml sample australasian journal of information systems sharma, singh & pratt 2020, vol 24, research article online travel purchases across generations 14 α cr ave msv maxr (h) ci at pe ee si fc hm ps hb pr pt cp pi pb ci 0.858 0.941 0.81 0.31 0.949 0.898 at 0.960 0.960 0.83 0.74 0.962 0.308*** 0.91 pe 0.944 0.945 0.81 0.74 0.945 0.343*** 0.860*** 0.899 ee 0.937 0.937 0.79 0.78 0.943 0.371*** 0.632*** 0.723*** 0.89 si 0.949 0.946 0.86 0.42 0.951 0.395*** 0.607*** 0.640*** 0.439*** 0.929 fc 0.874 0.876 0.7 0.78 0.882 0.285*** 0.713*** 0.793*** 0.885*** 0.502*** 0.839 hm 0.945 0.948 0.86 0.61 0.956 0.423*** 0.754*** 0.778*** 0.613*** 0.651*** 0.661*** 0.925 ps 0.894 0.898 0.75 0.72 0.928 0.241*** 0.770*** 0.846*** 0.666*** 0.548*** 0.763*** 0.703*** 0.868 hb 0.862 0.847 0.6 0.32 0.919 0.554*** 0.357*** 0.272*** 0.214** 0.563*** 0.152* 0.471*** 0.247*** 0.774 pr 0.946 0.947 0.86 0.26 0.96 0.056 -0.412*** -0.500*** -0.405*** -0.114† -0.510*** -0.278*** -0.465*** 0.094 0.926 pt 0.913 0.913 0.73 0.48 0.916 0.232*** 0.638*** 0.666*** 0.543*** 0.418*** 0.637*** 0.655*** 0.672*** 0.246*** -0.359*** 0.852 cp 0.936 0.938 0.83 0.66 0.942 0.310*** 0.664*** 0.701*** 0.574*** 0.546*** 0.678*** 0.647*** 0.682*** 0.386*** -0.465*** 0.693*** 0.912 pi 0.951 0.950 0.79 0.66 0.962 0.286*** 0.727*** 0.770*** 0.642*** 0.537*** 0.748*** 0.661*** 0.758*** 0.259*** -0.495*** 0.657*** 0.809*** 0.89 pb 0.767 0.784 0.56 0.28 0.837 0.378*** 0.526*** 0.464*** 0.393*** 0.463*** 0.424*** 0.427*** 0.427*** 0.392*** -0.081 0.327*** 0.486*** 0.513*** 0.8 table 6: discriminant validity analysis from confirmatory factor analysis bb sample australasian journal of information systems sharma, singh & pratt 2020, vol 24, research article online travel purchases across generations 15 4.3 structural relationships after the measurement model was successfully evaluated for goodness of fit, reliability, validity, and measurement invariance, a structural equation model using the maximum likelihood method of estimation was used to test the hypotheses. first, for the millennials, the model was adequate as the chi-square value was statistically significant (χ2=5224.321; df=1202; p<0.01; χ2/df=4.346; rmsea=0.0943; cfi=0.926; nfi= 0.901; tli=0.919). of the 20 estimate path coefficients, 14 were statistically significant at the .05, .01, or .001 level. the significant relationships were found on the paths for pe to at (β=0. 0.664, p < 0.001), fc to at (β=0. 187, p < 0.001), hm to at (β=0. 278, p < 0.001), ps to at (β=0.361, p < 0.001), hb to at (β=0.142, p < 0.05), pe to pi (β=0. 346, p < 0.001), si to pi (β=0.131, p < 0.05), hm to pi (β=0.121, p < 0.05), hb to pi (β=0.086, p < 0.05), at to pi (β=0.121, p < 0.05), cp to pi (β=0.659, p < 0.001), in to pi (β=0.122, p < 0.01), pt to pi (β=0.283, p < 0.001), and pi to pb (β=0.561, p < 0.001). however, h2a, h2b, h3a, h4b, h6b and h10 were not supported for millennials. second, for the baby boomers, the model was adequate as the chi-square value was statistically significant (χ2 = 5006.065; df=1202; p < 0.01; χ2/df= 4.165; rmsea= 0.0953; cfi=0.921; nfi= 0.879; tli=0.906). of the 20 estimate path coefficients, 12 were statistically significant at the .05, .01, or .001 level. the significant relationships were found on the paths for pe to at (β=0.382, p < 0.001), fc to at (β=0.121, p < 0.001), hm to at (β=0.238, p < 0.001), ps to at (β=0.221, p < 0.001), hb to at (β=0. 0798, p < 0.05), pe to pi (β=0.115, p < 0.05), si to pi (β=0.076, p < 0.05), fc to pi (β=0.184, p < 0.001), ps to pi (β=0.231, p < 0.001), cp to pi (β=0.608, p < 0.001), pr to pi (β=-0.088, p < 0.05), and pi to pb (β=0.394, p < 0.001). however, h2a, h2b, h3a, h5b, h7b, h8, h11, and h12 were not supported for baby boomers. to better understand the moderating effects of the generational cohort, we conducted further analyses to identify the structural relationships that were non-invariant. the significance and magnitude of path coefficients in the inner model were also compared to ascertain whether the strength and directionality of the structural relationships differ across the two samples. table 8 presents the generational-specific mga results for the millennials and baby boomers. the fit indices suggest that the structural model is adequate in the millennial and baby boomer samples (χ2 = 5142.52; df =1129; p < 0.01; χ2 /df=4.555; rmsea=0.057; cfi=0.916; nfi= 0.891; tli=0.917). australasian journal of information systems sharma, singh & pratt 2020, vol 24, research article online travel purchases across generations 16 path name ml beta bb beta difference in betas p-value for difference interpretation pe → at. 0.664*** 0.382*** 0.282 0.019 the positive relationship between pe and at is stronger for ml. ee → at. 0.0718 -0.0645 0.1363 0.251 there is no difference si → at. 0.0796 0.049 0.0306 0.85 there is no difference fc → at. 0.187*** 0.121** 0.066 0.858 there is no difference. hm → at. 0.278*** 0.238*** 0.04 0.975 there is no difference. ps → at. 0.361*** 0.221*** 0.14 0.418 there is no difference. hb → at. 0.142** 0.0798† 0.0622 0.964 there is no difference. pe → pi. 0.346*** 0.115† 0.321 0.067 the positive relationship between pe and pi is stronger for ml. ee → pi. -0.017 0.008 -0.025 0.874 there is no difference si → pi. 0.131** 0.076† 0.055 0.61 there is no difference. fc → pi. 0.034 0.184*** -0.15 0.155 the positive relationship between fc and pi is only significant for bb. hm → pi. 0.121** 0.022 0.099 0.169 the positive relationship between hm and pi is only significant for ml. ps → pi. 0.0226 0.231*** -0.2084 0.07 the positive relationship between ps and pi is stronger for bb. hb → pi. 0.086† 0.038 0.048 0.132 the positive relationship between hb and pi is only significant for ml. at → pi. 0.121† 0.0622 0.0588 0.581 the positive relationship between at and pi is only significant for ml. cp → pi. 0.659*** 0.608*** 0.051 0.544 there is no difference. in → pi. 0.122** 0.051 0.071 0.021 the positive relationship between ci and pi is stronger for ml. pr → pi. -0.061 -0.088 0.027 0.549 the negative relationship between pi and pr is only significant for bb. pt → pi. 0.283*** 0.041 0.232 0.023 the positive relationship between pi and pt is stronger for ml. pi → pb. 0.561*** 0.394*** 0.167 0.017 the positive relationship between pb and pi is stronger for ml. table 7: structural model relationships australasian journal of information systems sharma, singh & pratt 2020, vol 24, research article online travel purchases across generations 17 5 discussion to evaluate the predictive power of the endogenous constructs, the explained variance (r²) of the endogenous constructs was examined (chin, 2010). this explains the amount of variance in the construct that is explained by the model. using the full set of data, the r² value of the original utaut2 model is 33 percent. with the addition of attitude to the model, the r² value increases to 38 percent. furthermore, adding compatibility, innovativeness, perceived risk, and perceived trust increases the r² value to 64 percent. the three categories that can be used to classify the r² values of endogenous latent variables are: weak (19 percent), moderate (33 percent), and substantial (67 percent) (chin et al., 2008). this increase in r² highlights the importance of constructs in predicting consumers' online purchase intention. the study conducted by escobar-rodríguez and carvajal-trujillo (2014) derived a model with a r² value of 57.9%. the study conducted by san martín and herrero (2012) had an r² value of 40 percent, while the study by agag and el-masry (2017) had an r² value of 59%. 5.1 theoretical contributions this study has provided further insights into online travel purchase behaviour by extending the utaut2 model. the inclusion of attitude, compatibility, innovativeness, perceived risk, and perceived trust was able to increase the predictive power of the model. this allows for empirical evidence pertaining to new relationships to be tested, which provides greater insights into consumer behaviour. this also reiterates the importance of applying the technology acceptance model to consumer context to understand behaviour. this study highlights the importance of generational cohort theory in online travel purchase intention. it demonstrates how factors vary between millennials and baby boomers. much of the studies carried out previously have looked at a single generation (chen & chou, 2019; ladhari et al., 2019). this study is the first to apply the utaut2 model with the generational cohort theory to understanding online travel purchase behaviour. 5.2 practical implications at a time when internet use and online travel shopping are more prevalent, it is important to understand the factors that impact consumers' intention to purchase travel online, and how this outcome varies across generational cohorts. it is particularly important in the tourism industry to understand the different consumer segments to develop strategies to increase online purchases. empirical evidence from this study shows the influence of performance expectancy on behavioural intention is stronger for millennials. this can be attributed to millennial consumers having relatively higher technological innovativeness compared to baby boomers (church & iyer, 2017). despite baby boomers being open-minded and active, along with a keen sense of interest towards new technology (kumar & lim, 2008), they are unable to perceive the usefulness of new technology in the absence of ease of use (hur et al., 2017). therefore, online travel providers need to give greater emphasis on efficient content development and design so that the consumers find information on the website accurate, reliable, and useful (gupta et al., 2018). this will enhance perceived utility, which will help increase adoption. fc was also found as a variable that positively influences bb online travel purchase behaviour. this highlights the importance of resources and support services to be available to more mature customers when engaging in online purchases. as such, businesses need to ensure that australasian journal of information systems sharma, singh & pratt 2020, vol 24, research article online travel purchases across generations 18 their websites are informative about the support services available to customers should they encounter problems when purchasing online. this can include a list of frequently asked questions or online chat services to resolve customer confusion or doubts. the reason this relationship was only found significant for bb could be that they are less used to the internet compared to ml. looking at the results of the relationship between hm and pi, according to literature, only one study by baptista and oliveira (2017) found age as a significant moderator. two other studies by ramantoko et al. (2016) and ramírez-correa et al. (2014) did not find a significant relationship. ml is more likely to feel positive emotions while browsing when the travel website has a high quality of security and is well-designed (éthier et al., 2006). to increase hedonic shopping experiences on e-commerce websites, online retailers can incorporate exclusivity (e.g., deals and items that are web-exclusive) and concepts of game mechanics (i.e., gamification) (insley & nunan, 2014). to encourage the adoption of online travel shopping, web designers need to improve the hedonic features of the websites. the importance of price-saving has been highlighted in this study. as such, online travel providers need to provide customers cost-saving deals, best offers, and promotions as incentives to purchase travel online (ladhari et al., 2019). for instance, customers that purchase travel online will receive an additional discount. based on the empirical results of this study, this factor is only critical to bb. this may mean if price saving is a benefit of engaging in online buying, bb would be more likely to engage in this behaviour. results from this study have shown that habit is a significant factor impacting consumers' intention to purchase travel online in the ml sample. it is therefore suggested that businesses in australia should formulate strategies of marketing communication that create the habit of consumers' online travel purchase intention that will lead to an increase in purchase intention, thus, generating more online purchase behaviour. the findings from this study imply that in order for businesses to increase sales via online travel websites, businesses need to advertise using a combination of traditional media and the internet about how their websites can be used on different occasions and contexts to purchase travel. innovativeness is a personality trait that manifests in the ways that tourists that are more innovative are attracted more by new services that are an improvement to the existing options. this implies that marketing managers need to monitor levels of perceived value and benefits for ml constantly so that improved and innovative service options can be made available to consumers (kindström & kowalkowski, 2014; nicolau & santa-maría, 2013; sandvik et al., 2014). the results of this study support the crucial role that perceived risk plays on consumers' intention to purchase travel online. thus, it is important that online travel providers come up with strategies to reduce consumers' perceived risk to increase the adoption of online purchases. while results demonstrate that perceived risk negatively impact online shopping both age groups, the relationship is stronger for baby boomers. research carried out by chakraborty et al. (2016) and pascual-miguel et al. (2015) found similar results. the practical implication of this is that online travel providers need to convince their mature consumers about the security measures in place that help mitigate the risk. providing consumer information about their rights and protection, communicating high-security standards, having security approval symbols (e.g., verisign), offering money-back guarantees as well as clear australasian journal of information systems sharma, singh & pratt 2020, vol 24, research article online travel purchases across generations 19 information privacy policies can be some of the risk-reducing strategies employed (amaro & duarte, 2015). website designers and service providers can increase their visibility and trust by making the process and outcomes associated with purchasing travel online more easily accessible to consumers (e.g., with the use of personas and first-person-view videos that consumers are able to associate themselves with) with an explanation of the not so apparent support services (i.e., backend) will help enhance the observability. showcasing the positive outcomes to demonstrate the outcome of using online travel websites can also help businesses in increasing the adoption rate, for example, by highlighting testimonials and positive reviews about purchasing travel online. this will also help increase consumers' trust perception of online travel websites and increase adoption. this study has evidenced the importance of trust in consumer decision making to purchase online. therefore, travel providers should offer refunds, assure customers about their policies to ensure that their private and confidential information is not shared, provide formal guarantees of products and services, inform consumers about the availability of the product or service at the time of purchase and also allows comments and feedback to be posted by consumers. explaining the reliability measures, not including hidden costs, using the latest encryption technology, not disclosing credit card details, explaining how the information collected from consumers will be used, and having a well-designed website can help increase trust in online travel shopping (teubner & klein, 1998). 5.3 limitation and future research just like any other study, there are limitations to this study that provide good directions for future research. the results from this study may be difficult to generalize to other national markets because these are based on the unique experiences of consumers in australia. future research could better explore how online travel purchase behaviour varies across generations in different countries. another limitation of this research is in relation to the definition of an online travel purchase. the definition used in this research can be considered broad as it includes the purchase of all types of tourism products and services such as hotel reservations, holiday packages, cruises, and airline tickets. therefore, future research can examine the motivations to purchase travel products online while considering distinct product and service categories. studies (e.g., (bogdanovych et al., 2006; kamarulzaman, 2007) have found that consumers are less likely to purchase complex travel products online as they prefer to use travel agents to book complex travel. as such, the results of this study may differ if applied to products of high and low complexity. for comparative analysis, it would be interesting to replicate this study in different countries with different generational cohorts. it is important to highlight that the effect the extended utaut2 model together is likely to vary from one context to another. specifically, the effect of facilitating condition and social influence may vary in situations where users' risk perception regarding online purchase is higher (e.g., in countries with lower levels of legal security and technology). to add on, effort expectancy and innovativeness of users can be more significant in cases where more involvement is demanded from consumers, and the purchasing process is more complex (limayem et al., 2000). other recommendations for future research can be to adopt a different methodological approach that distinguishes between "individuals that have purchased online" and "those that have not" to study in greater detail australasian journal of information systems sharma, singh & pratt 2020, vol 24, research article online travel purchases across generations 20 psychological factors influencing consumers' online purchase intention. also, when making multimodel comparisons, future research must remain cognizant of hypothesis testing errors emerging from type one and type two errors and the ways to mitigate those errors (see lim and mandrinos (2020)). 6 conclusion this study was carried out with the aim of understanding consumers' intention to purchase travel online by extending the utaut2 model. attitude, compatibility, innovativeness, perceived trust, and perceived risk were added to the utaut2 model to increase its predictive power. empirical results from the study showed that with the addition of these factors to the utaut2 model, its predictive power was significantly increased. additionally, the analysis of results across millennials and baby boomers showed that factors affecting consumers' decision to travel online vary across the two generations. this study contributes theoretically and practically to better understanding of motivators and barriers of online purchase behaviour. references agag, g. m., & el-masry, a. a. (2017). why do consumers trust online travel websites? drivers and outcomes of consumer trust toward online travel websites. journal of travel research, 56(3), 347-369. agarwal, r., & prasad, j. (1998). a conceptual and operational definition of personal innovativeness in the domain of information technology. information systems research, 9(2), 204-215. ajzen, i. (1991). the theory of planned behaviour. organizational behaviour and human decision processes, 50, 179-211. de young, 50(2), 509-526. ajzen, i., & fishbein, m. (1980). understanding attitudes and predicting social behaviour. prentice hall. akinnuwesi, b. a., uzoka, f.-m. e., okwundu, o. s., & fashoto, g. (2016). exploring biometric technology adoption in a developing country context using the modified utaut. international journal of business information systems, 23(4), 482-521. al-gahtani, s. s. (2011). modeling the electronic transactions acceptance using an extended technology acceptance model. applied computing and informatics, 9(1), 47-77. alalwan, a. a., dwivedi, y. k., rana, n. p., & algharabat, r. (2018). examining factors influencing jordanian customers’ intentions and adoption of internet banking: extending utaut2 with risk. journal of retailing and consumer services, 40, 125-138. doi:10.1016/j.jretconser.2017.08.026 aldás-manzano, j., lassala-navarré, c., ruiz-mafé, c., & sanz-blas, s. (2009). the role of consumer innovativeness and perceived risk in online banking usage. international journal of bank marketing, 27(1), 53-75. amaro, s., & duarte, p. (2015). an integrative model of consumers' intentions to purchase travel online. tourism management, 46, 64-79. arnett, d. b., sandvik, i. l., & sandvik, k. (2018). two paths to organizational effectiveness– product advantage and life-cycle flexibility. journal of business research, 84, 285-292. australasian journal of information systems sharma, singh & pratt 2020, vol 24, research article online travel purchases across generations 21 athiyaman, a. (1995). the interface of tourism and strategy research: an analysis. tourism management, 16(6), 447-453. athiyaman, a. (2002). internet users' intention to purchase air travel online: an empirical investigation. marketing intelligence & planning, 20(4), 234-242. awwad, m. s., & al-majali, s. m. (2015). electronic library services acceptance and use: an empirical validation of unified theory of acceptance and use of technology. the electronic library, 33(6), 1100-1120. baabdullah, a. m., rana, n. p., alalwan, a. a., islam, r., patil, p., & dwivedi, y. k. (2019). consumer adoption of self-service technologies in the context of the jordanian banking industry: examining the moderating role of channel types. information systems management, 36(4), 286-305. bagozzi, r. p. (1981). attitudes, intentions, and behavior: a test of some key hypotheses. journal of personality and social psychology, 41(4), 607. bagozzi, r. p., & yi, y. (1988). on the evaluation of structural equation models. journal of the academy of marketing science, 16(1), 74-94. doi:10.1007/bf02723327 bakewell, c., & mitchell, v.-w. (2003). generation y female consumer decision-making styles. international journal of retail & distribution management, 31(2), 95-106. baptista, g., & oliveira, t. (2017). why so serious? gamification impact in the acceptance of mobile banking services. internet research, 27(1), 118-139. beauchamp, m. b., & barnes, d. c. (2015). delighting baby boomers and millennials: factors that matter most. journal of marketing theory and practice, 23(3), 338-350. beldona, s., nusair, k., & demicco, f. (2009). online travel purchase behavior of generational cohorts: a longitudinal study. journal of hospitality marketing & management, 18(4), 406420. bellman, s., lohse, g. l., & johnson, e. j. (1999). predictors of online buying behavior. communications of the acm, 42(12), 32-38. bicikova, k. (2014). understanding student travel behavior: a segmentation analysis of british university students. journal of travel & tourism marketing, 31(7), 854-867. bigné, e., sanz, s., ruiz, c., & aldás, j. (2010). why some internet users don’t buy air tickets online: springer. bobbitt, l. m., & dabholkar, p. a. (2001). integrating attitudinal theories to understand and predict use of technology-based self-service: the internet as an illustration. international journal of service industry management, 12(5), 423-450. bogdanovych, a., berger, h., simoff, s., & sierra, c. (2006). travel agents vs. online booking: tackling the shortcomings of nowadays online tourism portals. information and communication technologies in tourism, 418-428. bowen, j. t., & chen mccain, s.-l. (2015). transitioning loyalty programs: a commentary on “the relationship between customer loyalty and customer satisfaction”. international journal of contemporary hospitality management, 27(3), 415-430. australasian journal of information systems sharma, singh & pratt 2020, vol 24, research article online travel purchases across generations 22 buhalis, d., & law, r. (2008). progress in information technology and tourism management: 20 years on and 10 years after the internet—the state of etourism research. tourism management, 29(4), 609-623. chakraborty, r., lee, j., bagchi-sen, s., upadhyaya, s., & rao, h. r. (2016). online shopping intention in the context of data breach in online retail stores: an examination of older and younger adults. decision support systems, 83, 47-56. chen, c.-d., fan, y.-w., & farn, c.-k. (2007). predicting electronic toll collection service adoption: an integration of the technology acceptance model and the theory of planned behavior. transportation research part c: emerging technologies, 15(5), 300-311. chen, c.-f., & chou, s.-h. (2019). antecedents and consequences of perceived coolness for generation y in the context of creative tourism-a case study of the pier 2 art center in taiwan. tourism management, 72, 121-129. chin, w. w. (1998). the partial least squares approach to structural equation modeling. modern methods for business research, 295(2), 295-336. chin, w. w. (2010). how to write up and report pls analyses in the handbook of partial least squares (655-690): springer. chin, w. w., peterson, r. a., & brown, s. p. (2008). structural equation modeling in marketing: some practical reminders. journal of marketing theory and practice, 16(4), 287-298. chiu, c.-m., huang, h.-y., & yen, c.-h. (2010). antecedents of trust in online auctions. electronic commerce research and applications, 9(2), 148-159. christou, e., & kassianidis, p. (2002). consumer's perceptions and adoption of online buying for travel products. journal of travel & tourism marketing, 12(4), 93-107. church, e. m., & iyer, l. (2017). “when is short, sweet?” selection uncertainty and online review presentations. journal of computer information systems, 57(2), 179-189. compeau, d. r., & higgins, c. a. (1995). computer self-efficacy: development of a measure and initial test. mis quarterly, 19(2), 189-211. confente, i., & vigolo, v. (2018). online travel behaviour across cohorts: the impact of social influences and attitude on hotel booking intention. international journal of tourism research, 20(5), 660-670. coombes, b. (2009). generation y: are they really digital natives or more like digital refugees. synergy, 7(1), 31-40. crespo, á. h., & del bosque, i. r. (2008). the effect of innovativeness on the adoption of b2c e-commerce: a model based on the theory of planned behaviour. computers in human behavior, 24(6), 2830-2847. davis, f. d. (1989). perceived usefulness, perceived ease of use, and user acceptance of information technology. mis quarterly, 13(3), 319-340. davis, f. d., bagozzi, r. p., & warshaw, p. r. (1989). user acceptance of computer technology: a comparison of two theoretical models. management science, 35(8), 982-1003. donthu, n., & garcia, a. (1999). the internet shopper. journal of advertising research, 39(3), 5252. australasian journal of information systems sharma, singh & pratt 2020, vol 24, research article online travel purchases across generations 23 duffy, b., smith, k., terhanian, g., & bremer, j. (2005). comparing data from online and faceto-face surveys. international journal of market research, 47(6), 615-639. dwivedi, a., wilkie, d., johnson, l., & weerawardena, j. (2016). establishing measures and drivers of consumer brand engagement behaviours. journal of brand management, 23(5), 41-69. dwivedi, y. k., rana, n. p., jeyaraj, a., clement, m., & williams, m. d. (2017). re-examining the unified theory of acceptance and use of technology (utaut): towards a revised theoretical model. information systems frontiers, 1-16. dwivedi, y. k., rana, n. p., jeyaraj, a., clement, m., & williams, m. d. (2019). re-examining the unified theory of acceptance and use of technology (utaut): towards a revised theoretical model. information systems frontiers, 21(3), 719-734. escobar-rodríguez, t., & carvajal-trujillo, e. (2013). online drivers of consumer purchase of website airline tickets. journal of air transport management, 32, 58-64. escobar-rodríguez, t., & carvajal-trujillo, e. (2014). online purchasing tickets for low cost carriers: an application of the unified theory of acceptance and use of technology (utaut) model. tourism management, 43, 70-88. éthier, j., hadaya, p., talbot, j., & cadieux, j. (2006). b2c web site quality and emotions during online shopping episodes: an empirical study. information & management, 43(5), 627-639. farivar, s., turel, o., & yuan, y. (2017). a trust-risk perspective on social commerce use: an examination of the biasing role of habit. internet research, 27(3), 586-607. filippou, j., cheong, c., & cheong, f. (2018). a model to investigate preference for use of gamification in a learning activity. australasian journal of information systems, 22. doi:10.3127/ajis.v22i0.1397 fishbein, m., & ajzen, i. (1975). belief, attitude, intention, and behaviour: an introduction to theory and research. addison-wesley pub. co. (reading, mass). fornell, c., & larcker, d. f. (1981a). evaluating structural equation models with unobservable variables and measurement error. journal of marketing research, 18(1), 39-50. fornell, c., & larcker, d. f. (1981b). structural equation models with unobservable variables and measurement error: algebra and statistics. journal of marketing research, 382-388. friemel, t. n. (2016). the digital divide has grown old: determinants of a digital divide among seniors. new media & society, 18(2), 313-331. gardiner, s., grace, d., & king, c. (2015). is the australian domestic holiday a thing of the past? understanding baby boomer, generation x and generation y perceptions and attitude to domestic and international holidays. journal of vacation marketing, 21(4), 336350. gefen, d., karahanna, e., & straub, d. w. (2003). trust and tam in online shopping: an integrated model. mis quarterly, 27(1), 51-90. gilleard, c., jones, i., & higgs, p. (2015). connectivity in later life: the declining age divide in mobile cell phone ownership. sociological research online, 20(2), 1-13. australasian journal of information systems sharma, singh & pratt 2020, vol 24, research article online travel purchases across generations 24 gillenson, m. l., & sherrell, d. l. (2002). enticing online consumers: an extended technology acceptance perspective. information & management, 39(8), 705-719. goldsmith, r. e. (2000). how innovativeness differentiates online buyers. quarterly journal of electronic commerce, 1(4), 323-333. goldsmith, r. e. (2002). explaining and predicting consumer intention to purchase over the internet: an exploratory study. journal of marketing theory and practice, 10(2), 22-28. goldsmith, r. e., & lafferty, b. a. (2001). innovative online buyers. proceedings of the society for marketing advances, 275-278. gupta, a., & dogra, n. (2017). tourist adoption of mapping apps: a utaut2 perspective of smart travellers. tourism and hospitality management, 23(2), 145-161. gupta, a., dogra, n., & george, b. (2018). what determines tourist adoption of smartphone apps? an analysis based on the utaut-2 framework. journal of hospitality and tourism technology, 9(1), 50-64. hair, j., black, w., babin, b., anderson, r., & tatham, r. (2006). multivariate data analysis. new jersey: pearson educational: inc. herrando, c., jimenez-martinez, j., & martin-de hoyos, m. (2019). tell me your age and i tell you what you trust: the moderating effect of generations. internet research, 29(4), 799817. herrero, á., & san martín, h. (2017). explaining the adoption of social networks sites for sharing user-generated content: a revision of the utaut2. computers in human behavior, 71, 209-217. hew, j.-j., lee, v.-h., ooi, k.-b., & wei, j. (2015). what catalyses mobile apps usage intention: an empirical analysis. industrial management & data systems, 115(7), 1269-1291. hoque, r., & sorwar, g. (2017). understanding factors influencing the adoption of mhealth by the elderly: an extension of the utaut model. international journal of medical informatics, 101, 75-84. hsiao, c.-h., chang, j.-j., & tang, k.-y. (2016). exploring the influential factors in continuance usage of mobile social apps: satisfaction, habit, and customer value perspectives. telematics and informatics, 33(2), 342-355. hsu, c.-l., & lin, j. c.-c. (2016). effect of perceived value and social influences on mobile app stickiness and in-app purchase intention. technological forecasting and social change, 108, 42-53. huang, s.-y., yang, m. m., & chen, c.-h. (2018). when do motivational factors lead to negative user experience on social networking applications? australasian journal of information systems, 22. doi:10.3127/ajis.v22i0.1533 hult, g. t. m., sharma, p. n., morgeson iii, f. v., & zhang, y. (2019). antecedents and consequences of customer satisfaction: do they differ across online and offline purchases? journal of retailing, 95(1), 10-23. hur, h. j., lee, h. k., & choo, h. j. (2017). understanding usage intention in innovative mobile app service: comparison between millennial and mature consumers. computers in human behavior, 73, 353-361. australasian journal of information systems sharma, singh & pratt 2020, vol 24, research article online travel purchases across generations 25 insley, v., & nunan, d. (2014). gamification and the online retail experience. international journal of retail & distribution management, 42(5), 340-351. jackson, c. m., chow, s., & leitch, r. a. (1997). toward an understanding of the behavioral intention to use an information system. decision sciences, 28(2), 357-389. jensen, j. m. (2012). shopping orientation and online travel shopping: the role of travel experience. international journal of tourism research, 14(1), 56-70. kamarulzaman, y. (2007). adoption of travel e-shopping in the uk. international journal of retail & distribution management, 35(9), 703-719. kapoor, k. k., dwivedi, y. k., & williams, m. d. (2015). examining the role of three sets of innovation attributes for determining adoption of the interbank mobile payment service. information systems frontiers, 17(5), 1039-1056. khalilzadeh, j., ozturk, a. b., & bilgihan, a. (2017). security-related factors in extended utaut model for nfc based mobile payment in the restaurant industry. computers in human behavior, 70, 460-474. khare, a., khare, a., & singh, s. (2012). attracting shoppers to shop online—challenges and opportunities for the indian retail sector. journal of internet commerce, 11(2), 161-185. kim, h.-w., xu, y., & gupta, s. (2012). which is more important in internet shopping, perceived price or trust? electronic commerce research and applications, 11(3), 241-252. kim, m.-j., chung, n., & lee, c.-k. (2011). the effect of perceived trust on electronic commerce: shopping online for tourism products and services in south korea. tourism management, 32(2), 256-265. kim, s.-h., & park, h. j. (2011). effects of social influence on consumers' voluntary adoption of innovations prompted by others. journal of business research, 64(11), 1190-1194. kim, s., choe, j. y., & lee, s. (2018). how are food value video clips effective in promoting food tourism? generation y versus non–generation y. journal of travel & tourism marketing, 35(3), 377-393. kim, y. j., chun, j. u., & song, j. (2009). investigating the role of attitude in technology acceptance from an attitude strength perspective. international journal of information management, 29(1), 67-77. kindström, d., & kowalkowski, c. (2014). service innovation in product-centric firms: a multidimensional business model perspective. journal of business & industrial marketing, 29(2), 96-111. kumar, a., & lim, h. (2008). age differences in mobile service perceptions: comparison of generation y and baby boomers. journal of services marketing, 22(7), 568-577. kumar, s., & sadarangani, p. (2018). an empirical study on shopping motivation among generation y indian. global business review, 0972150918807085. ladhari, r., gonthier, j., & lajante, m. (2019). generation y and online fashion shopping: orientations and profiles. journal of retailing and consumer services, 48, 113-121. lee, c.-k., lee, y.-k., & wicks, b. e. (2004). segmentation of festival motivation by nationality and satisfaction. tourism management, 25(1), 61-70. australasian journal of information systems sharma, singh & pratt 2020, vol 24, research article online travel purchases across generations 26 lee, m.-c. (2009). factors influencing the adoption of internet banking: an integration of tam and tpb with perceived risk and perceived benefit. electronic commerce research and applications, 8(3), 130-141. doi:10.1177/0972150918807085 li, l., & buhalis, d. (2006). e-commerce in china: the case of travel. international journal of information management, 26(2), 153-166. liébana-cabanillas, f., marinković, v., & kalinić, z. (2017). a sem-neural network approach for predicting antecedents of m-commerce acceptance. international journal of information management, 37(2), 14-24. lim, w. m. (2018a). dialectic antidotes to critics of the technology acceptance model: conceptual, methodological, and replication treatments for behavioural modelling in technology-mediated environments. australasian journal of information systems, 22. doi:10.3127/ajis.v22i0.1651 lim, w. m. (2018b). revisiting concepts and theories in information systems and technology. australasian journal of information systems, 22. doi:10.3127/ajis.v22i0.1733 lim, w. m., lim, a. l., & phang, c. s. c. (2019). toward a conceptual framework for social media adoption by non-urban communities for non-profit activities: insights from an integration of grand theories of technology acceptance. australasian journal of information systems, 23. doi: 10.3127/ajis.v23i0.1835 lim, w. m., & mandrinos, s. (2020). incidental findings on multimodel comparison, hypothesis testing error, and its solution. australasian journal of information systems, 24. doi.org/10.3127/ajis.v24i0.2555 limayem, m., hirt, s. g., & cheung, c. m. (2007). how habit limits the predictive power of intention: the case of information systems continuance. mis quarterly, 705-737. limayem, m., khalifa, m., & frini, a. (2000). what makes consumers buy from internet? a longitudinal study of online shopping. ieee transactions on systems, man, and cybernetics-part a: systems and humans, 30(4), 421-432. lissitsa, s., & kol, o. (2019). four generational cohorts and hedonic m-shopping: association between personality traits and purchase intention. electronic commerce research, 1-26. lu, y., cao, y., wang, b., & yang, s. (2011). a study on factors that affect users’ behavioral intention to transfer usage from the offline to the online channel. computers in human behavior, 27(1), 355-364. madan, k., & yadav, r. (2016). behavioural intention to adopt mobile wallet: a developing country perspective. journal of indian business research, 8(3), 227-244. marriott, h. r., & williams, m. d. (2018). exploring consumers perceived risk and trust for mobile shopping: a theoretical framework and empirical study. journal of retailing and consumer services, 42, 133-146. martins, c., oliveira, t., & popovič, a. (2014). understanding the internet banking adoption: a unified theory of acceptance and use of technology and perceived risk application. international journal of information management, 34(1), 1-13. australasian journal of information systems sharma, singh & pratt 2020, vol 24, research article online travel purchases across generations 27 mazaheri, e., lagzian, m., & hemmat, z. (2020). research directions in information systems field, current status and future trends: a literature analysis of ais basket of top journals. australasian journal of information systems, 24. doi:10.3127/ajis.v24i0.2045 mcbride, m., carter, l., & phillips, b. (2020). integrating the theory of planned behavior and behavioral attitudes to explore texting among young drivers in the us. international journal of information management, 50, 365-374. meredith, g., & schewe, c. (1994). the power of cohorts. american demographics, 16(12), 22-28. morosan, c., & defranco, a. (2016). it's about time: revisiting utaut2 to examine consumers’ intentions to use nfc mobile payments in hotels. international journal of hospitality management, 53, 17-29. nakagawa, s. s. y., & gouvêa, m. a. (2010). identification of influent factors on products purchase through internet. paper presented at the poms 21st annual conference, vancouver, canada. nam, k., baker, j., ahmad, n., & goo, j. (2018). dissatisfaction, disconfirmation, and distrust: an empirical examination of value co-destruction through negative electronic word-ofmouth (ewom). information systems frontiers, 22, 113-130. nicolau, j. l., & santa-maría, m. j. (2013). the effect of innovation on hotel market value. international journal of hospitality management, 32, 71-79. nishijima, m., ivanauskas, t. m., & sarti, f. m. (2017). evolution and determinants of digital divide in brazil (2005–2013). telecommunications policy, 41(1), 12-24. njite, d., & schaffer, j. (2017). revisiting attributes: how important is green in the consumer selection of hotel rooms? international journal of hospitality & tourism administration, 18(2), 219-244. nunnally, j. c., & bernstein, i. (1994). psychometric theory (mcgraw-hill series in psychology) (vol. 3): mcgraw-hill new york. ozturk, a. b., nusair, k., okumus, f., & hua, n. (2016). the role of utilitarian and hedonic values on users’ continued usage intention in a mobile hotel booking environment. international journal of hospitality management, 57, 106-115. palau-saumell, r., forgas-coll, s., amaya-molinar, c. m., & sánchez-garcía, j. (2016). examining how country image influences destination image in a behavioral intentions model: the cases of lloret de mar (spain) and cancun (mexico). journal of travel & tourism marketing, 33(7), 949-965. paridahdaud, s. n. b. a., & fung, k. i. (2018). the influence of consumer perceived risks on online consumer’s repurchase behavior in malaysia online shopping: the mediating role of consumer attitude. asian journal of science and technology, 9(12), 9143-9150. park, c., & jun, j. (2002). a cross-cultural comparison of online buying intention: effects of internet usage, perceived risks, and innovativeness. paper presented at the the ausweb02: eighth australian world wide web conference. parment, a. (2013). generation y vs. baby boomers: shopping behavior, buyer involvement and implications for retailing. journal of retailing and consumer services, 20(2), 189-199. australasian journal of information systems sharma, singh & pratt 2020, vol 24, research article online travel purchases across generations 28 pascual-miguel, f. j., agudo-peregrina, á. f., & chaparro-peláez, j. (2015). influences of gender and product type on online purchasing. journal of business research, 68(7), 15501556. pelaez, a., chen, c.-w., & chen, y. x. (2019). effects of perceived risk on intention to purchase: a meta-analysis. journal of computer information systems, 59(1), 73-84. pendergast, d. (2010). getting to know the y generation. tourism and generation y, 1, 1-15. pennington-gray, l., fridgen, j. d., & stynes, d. (2003). cohort segmentation: an application to tourism. leisure sciences, 25(4), 341-361. ponte, e. b., carvajal-trujillo, e., & escobar-rodríguez, t. (2015). influence of trust and perceived value on the intention to purchase travel online: integrating the effects of assurance on trust antecedents. tourism management, 47, 286-302. ramantoko, g., putra, g., ariyanti, m., & sianturi, n. v. (2016). early adoption characteristic of consumers' behavioral intention to use home digital services in indonesia. paper presented at the 3rd international seminar and conference on learning organization. ramírez-correa, p., rondán-cataluña, f. j., & arenas-gaitán, j. (2014). an empirical analysis of mobile internet acceptance in chile. information research, 19(3), 19-13. ramsay, g., dodds, r., furtado, d., mykhayletska, y., kirichenko, a., & majedian, m. (2017). the barriers to millennials visiting rouge urban national park. sustainability, 9(6), 904. rehman, z. u., baharun, r., & salleh, n. z. m. (2020). antecedents, consequences, and reducers of perceived risk in social media: a systematic literature review and directions for further research. psychology & marketing, 37(1), 74-86. rogers, e. (1995). diffusion of innovations (4th eds.) acm the free press (sept. 2001). new york, 15-23. rogers, e. m., & shoemaker, f. f. (1971). communication of innovations; a cross-cultural approach. new york: the free press sait, s., al-tawil, k., & hussain, s. (2004). e-commerce in saudi arabia: adoption and perspectives. australasian journal of information systems, 12(1). doi: 10.3127/ajis.v12i1.105 samhan, b. (2018). revisiting technology resistance: current insights and future directions. australasian journal of information systems, 22. doi: 10.3127/ajis.v22i0.1655 san martín, h., & herrero, á. (2012). influence of the user’s psychological factors on the online purchase intention in rural tourism: integrating innovativeness to the utaut framework. tourism management, 33(2), 341-350. sandvik, i. l., duhan, d. f., & sandvik, k. (2014). innovativeness and profitability: an empirical investigation in the norwegian hotel industry. cornell hospitality quarterly, 55(2), 165-185. schewe, c. d., meredith, g. e., & noble, s. m. (2000). defining moments: segmenting by cohorts. marketing management, 9(3), 48-53. sharma, r., singh, g., & sharma, s. (2020). modelling internet banking adoption in fiji: a developing country perspective. international journal of information management, 53, 102116. australasian journal of information systems sharma, singh & pratt 2020, vol 24, research article online travel purchases across generations 29 shin, d.-h. (2009). towards an understanding of the consumer acceptance of mobile wallet. computers in human behavior, 25(6), 1343-1354. shulga, l. v., busser, j. a., & kim, h. (2018). generational profiles in value co-creation interactions. journal of hospitality marketing & management, 27(2), 196-217. slade, e. l., williams, m. d., & dwivedi, y. k. (2014). devising a research model to examine adoption of mobile payments: an extension of utaut2. the marketing review, 14(3), 310-335. smith, r., deitz, g., royne, m. b., hansen, j. d., grünhagen, m., & witte, c. (2013). crosscultural examination of online shopping behavior: a comparison of norway, germany, and the united states. journal of business research, 66(3), 328-335. song, h., lee, c.-k., reisinger, y., & xu, h.-l. (2017). the role of visa exemption in chinese tourists’ decision-making: a model of goal-directed behavior. journal of travel & tourism marketing, 34(5), 666-679. strauss, w., & howe, n. (1991). generations: the history of america’s future, 1584 to 2069. new york, ny: william morrow and company: inc. suh, e., alhaery, m., abarbanel, b., & mckenna, a. (2017). examining millennials’ online gambling behavior: a comparison of generational differences. journal of hospitality and tourism technology, 8(3), 314-336. tamilmani, k., rana, n. p., prakasam, n., & dwivedi, y. k. (2019). the battle of brain vs. heart: a literature review and meta-analysis of “hedonic motivation” use in utaut2. international journal of information management, 46, 222-235. tan, g. w.-h., ooi, k.-b., chong, s.-c., & hew, t.-s. (2014). nfc mobile credit card: the next frontier of mobile payment? telematics and informatics, 31(2), 292-307. tarhini, a., el-masri, m., ali, m., & serrano, a. (2016). extending the utaut model to understand the customers’ acceptance and use of internet banking in lebanon: a structural equation modeling approach. information technology & people, 29(4), 830-849. taylor, s., & todd, p. a. (1995). understanding information technology usage: a test of competing models. information systems research, 6(2), 144-176. teubner, r., & klein, s. (1998). planning and designing web-based electronic commerce: a case study in the insurance industry. australasian journal of information systems, 6(1). https://doi.org/10.3127/ajis.v6i1.331 triandis, h. (1977). interpersonal behavior (p. 329). monterey, ca: brooks: cole publishing company. tsaur, s.-h., & yen, c.-h. (2018). work–leisure conflict and its consequences: do generational differences matter? tourism management, 69, 121-131. varma citrin, a., sprott, d. e., silverman, s. n., & stem jr, d. e. (2000). adoption of internet shopping: the role of consumer innovativeness. industrial management & data systems, 100(7), 294-300. venkatesh, v., morris, m. g., davis, g. b., & davis, f. d. (2003). user acceptance of information technology: toward a unified view. mis quarterly, 27(3), 425-478. australasian journal of information systems sharma, singh & pratt 2020, vol 24, research article online travel purchases across generations 30 venkatesh, v., thong, j. y., & xu, x. (2012). consumer acceptance and use of information technology: extending the unified theory of acceptance and use of technology. mis quarterly, 36(1), 157-178. vijayasarathy, l. r. (2004). predicting consumer intentions to use on-line shopping: the case for an augmented technology acceptance model. information & management, 41(6), 747762. wang, x., weeger, a., & gewald, h. (2017). factors driving employee participation in corporate byod programs: a cross-national comparison from the perspective of future employees. australasian journal of information systems, 21. https://doi.org/10.3127/ajis.v21i0.1488 wen, i. (2010). online travelers' decision makings: a new equation model to evaluate impacts of website, search intention, and trust. information technology & tourism, 12(2), 153-173. wu, c.-h., & chen, s.-c. (2015). understanding the relationships of critical factors to facebook educational usage intention. internet research, 25(2), 262-278. xu, x., & pratt, s. (2018). social media influencers as endorsers to promote travel destinations: an application of self-congruence theory to the chinese generation y. journal of travel & tourism marketing, 35(7), 958-972. doi:10.1080/10548408.2018.1468851 yang, s., lu, y., gupta, s., cao, y., & zhang, r. (2012). mobile payment services adoption across time: an empirical study of the effects of behavioral beliefs, social influences, and personal traits. computers in human behavior, 28(1), 129-142. yang, z., & fang, x. (2004). online service quality dimensions and their relationships with satisfaction: a content analysis of customer reviews of securities brokerage services. international journal of service industry management, 15(3), 302-326. zhang, t., abound omran, b., & cobanoglu, c. (2017). generation y’s positive and negative ewom: use of social media and mobile technology. international journal of contemporary hospitality management, 29(2), 732-761. zhuang, h., leszczyc, p. t. p., & lin, y. (2018). why is price dispersion higher online than offline? the impact of retailer type and shopping risk on price dispersion. journal of retailing. 94(2), 136-153. copyright: © 2020 sharma, singh & pratt. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v24i0.2751 http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems sharma, singh & pratt 2020, vol 24, research article online travel purchases across generations 31 paper title (paper title style) australasian journal of information systems kautz 2024, vol 28, editorial farewell editorial 2023/2024 1 editor’s comment i bid farewell: some more reflections on 40 months with the journal, on ajis volume 27, and a little on volume 28 karlheinz kautz rmit university, melbourne, australia karlheinz.kautz@rmit.edu.au so long, farewell, auf wiedersehen, goodbye adieu, goodbye, auf wiederseh’n! i leave and heave a sigh and say goodbye die zeit mit dir war schön (the sound of music, 1959)1 (i had a nice time with you) (rammstein, 2022) this is the third and my farewell editorial as editor-in-chief (eic) of the ajis. three plus years, 40 months have passed since i took over the eic role after an appointment by the aais search panel. these were in many respects exciting, educational, but at times also exhausting months. this was my second assignment as an eic in what the australian business deans council (abdc) ranks as an a journal, but a very different experience. during my tenure at the scandinavian journal of is many years ago, i was a member of an editorial collective of four editors/co-editors each of whom took the eic role in the last year of their tenure. at the ajis the role encompassed a singular leadership role rangimg from academic and scholarly accompaniment to typesetting, production, and publication where i was supported by 'section editors' (ses) a nomenclature imposed by our review management system whom i appointed on a need basis out of a large pool of national and international scholars who served as senior/associate editors for individual submissions to the research paper or topical sections or for entire special (issue) sections which fell into their area of competence and for which they recruited qualified reviewers. while my role and the carrying of so much responsibility was very rewarding, it was also challenging and often lonely. during the years i had the support of at the time of writing in mid may 2024 240 ses from all ais regions, many from my personal academic network covering a multitude of research competencies, reseach methods, and topics. some of them served in each of the 3 plus years and many did so several times per year. i am grateful to each and everyone who accepted my invitations and requests, took the role, assigned competent reviewers out of their networks, and supported our is community. thus, i was helped by, one could say, a large, though at first mostly anonymous and unrecognized group. i therefore decided to at least publish a list of those who served as ses at the end of each of my editorials, and the present one will be no exception. however, i also contacted many, many more colleagues, who due to different reasons were not able or willing to help. hence, i had to deal with a continuous flow of rejection in this regard and i struggled sometimes as also uncountable reminders had to be sent to committed ses and reviewers to ask them to respond and keep their promised time lines. 1 none of the song titles of the song lyrics cited here is listed in the reference list; the titles are set in italics. mailto:karlheinz.kautz@rmit.edu.au australasian journal of information systems kautz 2024, vol 28, editorial farewell editorial 2023/2024 2 thus, together with the aais leadership, to distribute the immediate load on more shoulders, we brought some changes on the way, and expanded the editorial team. as such, last year we established the role of a chief technical officer, currently held by john james, from the university of wollongong, who takes care of the technical aspects of our instance of the ojs/pkp (open journal systems/public knowledge project check them out, they do important work at https://pkp.sfu.ca/software/ojs/) review management system and most recently, inaugurated the role of production editor and manager, which jaqui blake, from the university of the sunshine coast, accepted, to look after the production and publishing aspects of accepted submissions. the help of those two role bearers has already now been invaluable. i am also sure that my successors, stuart black and michael davern, both from the university of melbourne having a pair of co-editors is another innovation and will decrease the pressure on a single responsible will implement further new structures and novelties, in particular when it comes to further professionalising and to further sharing and involving the community in the task of developing a high quality journal for our region and beyond. they might perhaps establish a stable editorial board with academics from near and far in senior and/or associate editor roles and/or as members of an advisory board and/or review board, all ideas and ‘things’ i had on my list when i started in the role, but never got to do due to the daily operations of the journal. in those 40 months i handled about 940 new submissions, and a pipeline of another 60 or so, thus a total of a little over 1000 submissions, a steep increase given that the journal up to that point had handled about 1600 manuscripts in its previous 24 volumes. i will thus support my successors’ potential new approaches and, to share some of my experience, therefore accepted to serve on the editorial team in an advisory role for some more time. during my time i tried to re-establish an understanding of the is disicpline based on lee’s (2001) and sarker, chatterjee, xiao, and elbanna’s ( 2019) work on the sociotechnical roots of the discipline and its axis of cohesion. i thus referred many submitted manuscripts to journals covering other social science and business disciplines expecially to marketing and economy/finance journals, as well as to technology, engineering, and computer science outlets, in particular lately where the journal received much technical work to what under a resurrected floating signifier (chandler, 2010; lévi-strauss, 1987) and as a hegemonic ambiguous big concept (alvesson and blom, 2020) is called ‘artificial intelligence’ and even more computer science work under the misleading label of ‘machine learning’. trying to understand more about this specific misnomer as margaryan (2024) calls it, for me seems to me a noble and classic task for our discipline. margaryan (2024) provides some compelling thoughts in this respect and others have in this context contributed to an understanding and the role of data. i enjoyed re-reading jones’s (2019) thoughts on ‘what we talk about when we talk about (big) data’, as well as lebovitz, levina, and lifshitz-assal’s (2021) deliberations on the question whether ‘artificial intelligence’ ground truth is really true, and further more hirschheim’s (2021) commment on data mining fool’s gold. i can also recommend readings on the making of data commodities (aaltonen, alaimo, and kallinikos, 2021), and managing by data (alaimo and kallinikos, 2021), about the backrooms of data science (parmiggiani, østerlie, and almkov, 2022), on the open prison of the big data revolution (ngwenyama, rowe, klein, and zinner henriksen, 2023), and the performative production of trace data in knowledge work (altonen and stelmaszak, 2023). in the words of alaimo and kallinikos (2024) ‘data rules’ and plays its role australasian journal of information systems kautz 2024, vol 28, editorial farewell editorial 2023/2024 3 in technofeudalism (varoufakis, 2023) and in the rise of digital management, from industrial mobilization to platform capitalism (de vaujany, 2024), the first and the third title will be available in full later this year, so there is more to read. in this context ruster’s work on dignity centred artificial intelligence (ruster, oliva-altamirano, and daniell, 2022; ruster, 2023) is important as well. together with her, mike seymour, and sandra peter (seymour, peter, kautz, and ruster, 2023) i had the opportunity to explore the boundaries of our discipline in the field of artificial intelligence which as haigh (2023) had put is so elgently in his historical reflections is the conjoined twin of, and originated together with, and in, the computer science discipline. we hope to share our panel arguments soon with you in writing. beyond formulating my understanding of the is discipline and research based on lee’s (2001) and sarker, chatterjee, xiao, and elbanna’s ( 2019) work i have at this place also supported majchrzak, markus, and wareham (2016) and sørensen (2016) who advocate is research that transcends individual organisations and organisational boundaries and emphasise concepts such as emergence, networks, and ecosystems. backed up by avison and colleagues (avison and wood-harper, 1990; avison and fitzgerald (2006; avison, 2018) i also dwelled into the foundational concepts of information and system and the contested concept of ‘is use’ (burton-jones, stein, and mishra, 2020) and contrasted it with the concept of appropriation (riemer and johnston, 2012, 2017). this led to another development as during my period the ses and i implemented some stricter views on traditional adoption studies of new technologies in particular regions and sections of society or industry sectors in light of our pursuit for more innovative studies which attempt to break with conventional is research. we therefore directed most submissions of this kind to other outlets – i explained this development in more detail in both previous editorials (kautz, 2021, 2023). we also established some sterner positions on mere descriptive, summative and gap-spotting literature reviews, and, adding to the scope related reasons cited above, desktop rejected many of these submission (on the topic of desktop rejections, see also again dwivedi, hughes, cheung, conboy, duan, dubey, ... , and viglia, 2022). instead we promoted alternative, reflexive engagements with the existing literature that are productively grounded in a focussed coverage of literature (alvesson and sandberg, 2023; caroll et al., 2023) along the lines of a hermeneutic approach (boell and cecez-kecmanovic, 2014), alvesson and sandberg’s (2020) problematizing literature review approach, and rivard’s (2024) description of the process of conceptual leaping in the conduct of literature reviews, all as foundation for interesting, box-breaking (alvesson and sandberg, 2013, 2014) and, in the words of rivard (2024) above, conceptual leaping research. in this context i encourage(d) research based on alvesson and sandberg’s (2011) problematization research approach and now also recommend engaging with mueller and hovorka’s calls to get involved with the inquiry into digital futures (mueller and hovorka, 2023, 2024). these authors put forward speculation and imagination as fundamental to scientific research (hovorka and mueller, 2024) and inspired, among others, some of my collaborator and myself to a study on the role of serendipity in digital futures (hylving, koutsikouri, and kautz, 2024). with these reflections on the past and possible futures of the journal i like to spent the last part of this editorial comment to briefly introduce volume 27 and in an unusal manner for a ‘yearly’ editorial also the first two issues of volume 28. https://orcid.org/0000-0002-4267-7680 australasian journal of information systems kautz 2024, vol 28, editorial farewell editorial 2023/2024 4 last year we received 278 submissions, we published 24 manuscripts, 17 regular research articles, 1 in our proudly featured section on indigenous use of information and communication technologies, and 6 selected, and developed, papers from the australasian conference on information systems in 2021 and in 2022. we again received submissions and published manuscripts from all three ais regions covering sociotechnical grounds such as information systems development and project management, is/it ethics, e-government, ehealth, technostress, online game addiction, digitally mediated collaboration, it consumerisation, social media, hatespeech, cyperloafing, data-driven organisations, big data, is and sustainability, with a technology acceptance and adoption study, in this instance, of blockchain included. volume 28 is shaping up to become another volume well worth reading. in the so far 8 published manuscripts we are covering is/it use, continue to dwell into social media as well as sociotechnical perspectives on sustainability in the context of mining, return to digital transformation, and start seeing publications on the before problematized topics of ‘artificial intelligence’ and ‘machine learning’, and we have a healthy pipeline of more to come. thus if you have not yet done so, return to volume 27, and start engaging with volume 28 as well. we have also again been running some basic analytics regarding the resolution of the dois (clicks on dois referring to articles in the journal) of the now 858 articles that are registered as published in the review management system of the journal since its inception. the interest in the journal has steadily increased, in 2023 we count, as in 2022, around 90000 resolutions, on average 7500 resolutions per months. in 2023 we had about 135000 visits to our website, with 25000 direct downloads, and the top 15 articles have been been downloaded in a range from 1000 to 3650 downloads. my objective has from the very beginning of my tenure been to increase the visibility, popularity, and quality of the journal. but given my known ambivalent attitude towards competitive university and journal rankings and my shared concerns regarding their accompanying symptoms such as humble bragging (lenardic, seales, and levander, 2022) and a recognizable harsh review culture in our discipline (kautz, 2024), i ultimately leave it to others to assess the current standing of the journal and whether i have progressed to achieve this goal. now, that’s it for me, others will continue the (good) work, and ‘i say thank you for the music, for giving it to me’ (abba, 1977) and end with some more poetry and song lyrics: it's closing time and i lift my glass to the awful truth which you can't reveal to the ears of youth ah, we're drinking and we're dancing and it's partner found, it's partner lost and it's hell to pay when the fiddler stops it's closing time (l. cohen, 1992) australasian journal of information systems kautz 2024, vol 28, editorial farewell editorial 2023/2024 5 references alaimo, c., kallinikos, j. (2021). managing by data: algorithmic categories and organizing. organization studies, 42(9), 1385-1407. doi:10.1177/01708406209344062 alaimo, c., kallinikos, j. (2024), data rules reinventing the market economy. cambridge, ma, usa: mit press. aaltonen, a., alaimo, c., and kallinikos, j. (2021). the making of data commodities: data analytics as embedded process. journal of management information systems, 38(2), 401429. doi: 10.1080/07421222.2021.191298 altonen, a. and stelmaszak, m. (2023). the performative production of trace data in knowledge work. information systems research, articles in advance, 1-15. https://doi.org.101287/isre.2019.0357 alvesson, m., and sandberg, j. (2011). generating research questions through problematization. academy of management review, 36(2), 247-271. alvesson, m., and sandberg, j. (2013). has management studies lost its way? ideas for more imaginative and innovative research. journal of management studies, 50, 128-152. alvesson, m., and sandberg, j. (2014). habitat and habitus: boxed-in versus box-breaking research. organization studies, 35(7), 967-987. alvesson, m., and sandberg, j. (2020). the problematizing review: a counterpoint to elsbach and van knippenberg’s argument for integrative reviews. journal of management studies, 57(6), 1290-1304. doi:10.1111/joms.12582 alvesson, m., and blom, m. (2022). the hegemonic ambiguity of big concepts in organization studies. human relations, 75(1), 58-86. https://doi.org/10.1177/0018726720986847 alvesson, m., and sandberg, j. (2023). the art of phenomena construction: a framework for coming up with research phenomena beyond ‘the usual suspects. journal of management studies. doi:10.1111/joms.12969 avison, d. (2018). reminiscences and reflections. in fitzgerald, g., and malaurent, j., (eds.), information systems, development approaches and qualitative research: a tribute to david avison, 19-25. cergy, france: essec publishing. avison, d., and wood-harper, t. (1990). multiview an exploration in information systems development. oxford, uk: blackwell scientific publications. avison, d., and fitzgerald, g. (2006). information systems development – methodologies, techniques & tools (4th edition). maidenhead, uk: mcgraw hill education. boell, s. k., and cecez-kecmanovic, d. (2014). a hermeneutic approach for conducting literature reviews and literature searches. communications of the association for information systems, 34(1), 10. https://doi.org/10.17705/1cais.03412 burton-jones, a., stein, m.-k., and mishra, a. (2020). is use. misq curations. available as updated version to the original 2017 release at: https://www.misqresearchcurations.org/blog/2017/12/1/is-use carroll, n., conboy, k., hassan, n. r., hess, t., junglas, i., and morgan, l. (2023). problematizing assumptions on digital transformation research in the information australasian journal of information systems kautz 2024, vol 28, editorial farewell editorial 2023/2024 6 systems field. communications of the association for information systems, 53, 508531. https://doi.org/10.17705/1cais.05322 chandler, d. (1994). semiotics for beginners, www document. http://www.aber.ac.uk/media/documents/s4b/sem02a.html (visited march 23, 2024) de vaujany, f.-x. (2024). the rise of digital management: from industrial mobilization to platform capitalism. london, uk: routledge. dwivedi, y. k., hughes, l., cheung, c. m., conboy, k., duan, y., dubey, r., ..., and viglia, g. (2022). how to develop a quality research article and avoid a journal desk rejection. international journal of information management, 62, 102426. https://www.sciencedirect.com/science/article/pii/s0268401221001195 haigh, t. (2023). historical reflections conjoined twins: artificical intellignece and the invention of computer science. communications of the acm, 66(6), 33-37. doi: 10.1145/3593007 hirschheim, r. (2021). the attack on understanding: how big data and theory have led us astry: a comment on gary smith’s data mining fool’s gold. journal of information technology, 36(2), 176-183. doi:10.1177/-268396220967677 hovorka, d., and mueller, b. (204). speculation: forms and functions. in proceedings of the 57th hawaii international conference on system sciences. waikiki, oahu, hi, usa, january 3-6, 2024. https://scholarspace.manoa.hawaii.edu/items/9f1e0a22-630e-4a8d-bf68-7accff2d4db2 hylving, l., koutsikouri, d., and kautz, k. (2024). on serendipity in digital futures. in proceedings of the 57th hawaii international conference on system sciences. waikiki, oahu, hi, usa, january 3-6, 2024. https://scholarspace.manoa.hawaii.edu/items/0ad59ac6-71b5-4795-b140-955dcab67c8c jones, m. (2019). what we talk about when we talk about (big) data. the journal of strategic information systems, 28(1), 3-16. https://doi.org/10.1016/j.jsis.2018.10.005 kautz, k. (2021). editorial for the australasian journal of information systems 2021: volume 25. australasian journal of information systems, 2. https://doi.org/10.3127/ajis.v25i0.3853 kautz, k. (2023). editor’s comment : brief reflections on the journal, the concepts of systems, technologies, use, cybernetics and the ajis volume 26. australasian journal of information systems, 26. https://doi.org/10.3127/ajis.v26i0.4319 kautz, k. (2024). the harsh review culture in our discipline – a symptom for a bigger issue?!? a provocation. in proceedings of the 32nd european conference on information systems. paphos, cyprus, june 13-19, 2024 (forthcoming). lebovitz, s., levina, n., and lifshitz-assaf, h. (2021). is ai ground truth really true? the dangers of training and evaluation ai tools based on experts' know-what. mis quarterly, 45(3b), 1501-1525. https://doi.10.25300/misq/2021/16564 lee, a. (2001). editor’s comments. mis quarterly, 25(1), pp. iii-vii. lenardic, a., seales, j., and levander, a. (2022), the rise of the academic science humble brag. matter of opinion (cell press), 5 (3), 766-767. https://doi.org/1016/j.matt.2022.02.03 australasian journal of information systems kautz 2024, vol 28, editorial farewell editorial 2023/2024 7 lévi-strauss, c. (1987). introduction to the work of marcel mauss. london, uk: routledge. https://doi.org/10.4324/9781315005102 majchrzak, a., markus, m.l., and wareham, j. (2016). designing for digital transformation: lessons for information systems research form the study of ict and societal challenges. mis quarterly, 40(2), 267-277. https://doi.org/10.25300/misq/2016/ 40:2.03 margaryan, a. (2024). why ‘machine learning’ is a misnomer. available at ssrn: https://ssrn.com/abstract=4784181 or http://dx.doi.org/10.2139/ssrn.4784181 (accessed april 20, 2024) mueller, b. and hovorka, d. (2024). the challenge of “engaging with futures” in information systems research. in proceedings of the 56th hawaii international conference on system sciences. kaanapali beach, maui, hi, usa, january 3-6, 2023. https://scholarspace.manoa.hawaii.edu/items/0412c04b-be26-4653-b871-8c85633c3618 mueller, b. and hovorka, d. (2024). apprehending future informing systems: what worlds are we creating. in proceedings of the 57th hawaii international conference on system sciences. waikiki, oahu, hi, usa, january 3-6, 2024. https://scholarspace.manoa.hawaii.edu/items/b68ae791-3a84-4912-81a7-00a38cbd1710 ngwenyama, o., rowe, f., klein, s., and zinner henriksen, h. (2023). the open prison of the big data revolution: false consciousness, faustian bargains, and digital entrapment. information systems research, articles in advance. https://doi.org/101287/isre.2020.0588 parmiggiani, e., østerlie, t. and almkov, p. g. (2022). in the backrooms of data science. journal of the association for information systems (jais), 23(1), 139-164. doi: 10.17705/1jais.00718 riemer, k., and johnston, r. b. (2012). place-making: a phenomenological theory of technology appropriation. in proceedings of the 33rd international conference on information systems. orlando, fl, usa, december 16-19, 2012. https://aisel.aisnet.org/icis2012/proceedings/socialimpacts/5 riemer, k., and johnston, r. b. (2017). clarifying ontological inseparability with heidegger's analysis of equipment. mis quarterly, 41(4), 1059-1081. https://doi.org/10.25300/misq/2017/41.4.03 rivard, s. (2024). unpacking the process of conceptual leaping in the conduct of literature reviews. the journal of strategic information systems, 33(1), 101822. https://doi.org/10.1016/j.jsis.2024.101822. ruster, l., oliva-altamirano, p., and daniell, k. a (2022). centring dignity in algorithm development: testing a dignity lens, proceedings of the 34th australian conference on human-computer interaction, canberra, australia, november 2022. https://doi.org/10.1145/3572921.3572938 ruster, l. (2023). dignity-centred reflective practice for responsible action in technology startups: a preliminary approach. in proceedings of the 34th australasian conference on information systems (acis 2023). wellington, nz, december 5-8, 2023. https://aisel.aisnet.org/acis2023/25 https://doi.org/10.1145/3572921 https://doi.org/10.1145/3572921 australasian journal of information systems kautz 2024, vol 28, editorial farewell editorial 2023/2024 8 sarker, s., chatterjee, s., xiao, x., and elbanna, a. ( 2019). sociotechnical axis of cohesion for the is discipline: its historical legacy and its continued relevance. mis quarterly, 43(3), 685-719. https://doi.org/10.25300/misq/2019/13747 seymour, m., peter, s., kautz, k., and ruster, l. (2023). ai boundaries for is research. panel at the 34th australasian conference on information systems (acis 2023), wellington, nz, december 5 -8, 2023. sørensen, c. (2016). the curse of the smart machine? digitalisation and the children of the mainframe. scandinavian journal of information systems, 28(2), article 3. https:// aisel.aisnet.org/sjis/vol28/iss2/3 varoufakis, y. (2023). technofeudalism: what killed capitalism. london, uk: penguin books. ajis section editors in 2023 saeed akhlaghpour riitta hekkala jens poepppelbuss alireza amrollahi ai-phuong hoang alex richardson allen au mohammad hossain kai riemer emilia belluci dirk hovorka matti rossi magnus bergquist lena hylving paul scifleet yee ling boo jesper kjeldskov mike seymour deborah bunker robert krimmer mikko sipponen raffaele ciriello sherah kurnia stefan stieglitz rodney clarke weng marc lim juliana sutanto calvin chan meng lai henry linger lars svensson duy dang-pham martin matzner amir talaei-khoei kevin desouza sean maynard angsana a. techatassanasoontorn mike dinger julian marx janet toland bill doolin craig mcdonald holly tootell yogesh dwivedi shah miah matthias trier rebekah eden milad mirbabaie van-hau trieu amany elbana ose oshodin ofir turel mahdi fahmideh neeraj pandey nils urbach walter fernandez niki panteli john venable sigi goode konrad peszynski dongming xu ferry hassandoust australasian journal of information systems kautz 2024, vol 28, editorial farewell editorial 2023/2024 9 ajis section editors in 2024 (upto may 15) marc adam arto lanamäki christoph rosenkranz lubna alam henry linger matti rossi rohan bennett libo liu paul scifleet magnus bergquist julian marx mike seymour sebastian boell craig mcdonald maryam shabazi yee ling boo annette mills stefan stieglitz rodney j. clarke illias pappas amir talaei peter dell konrad peszynski angsana a. techatassanasoontorn bill doolin jan recker van-hau trieu ferry hassandoust kai riemer nils urbach pham cong hiep michael rosemann martin wiener copyright: © 2024 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v28.5237 http://creativecommons.org/licenses/by-nc/3.0/au/ i bid farewell: some more reflections on 40 months with the journal, on ajis volume 27, and a little on volume 28 paper title (paper title style) australasian journal of information systems wanniarachchi et al. 2023, vol 27, research article hate speech pattern in social media 1 hate speech patterns in social media: a methodological framework and fat stigma investigation incorporating sentiment analysis, topic modelling and discourse analysis vajisha udayangi wanniarachchi school of mathematical and computational sciences, massey university, auckland, new zealand v.u.wanniarachchi@massey.ac.nz chris scogings school of mathematical and computational sciences, massey university, auckland, new zealand teo susnjak school of mathematical and computational sciences, massey university, auckland, new zealand anuradha mathrani school of mathematical and computational sciences, massey university, auckland, new zealand abstract social media offers users an online platform to freely express themselves; however, when users post opinionated and offensive comments that target certain individuals or communities, this could instigate animosity towards them. widespread condemnation of obesity (fatness) has led to much fat stigmatizing content being posted online. a methodological framework that uses a novel mixed-method approach for unearthing hate speech patterns from large textbased corpora gathered from social media is proposed. we explain the use of computermediated quantitative methods comprising natural language processing techniques such as sentiment analysis, emotion analysis and topic modelling, along with qualitative discourse analysis. next, we have applied the framework to a corpus of texts on gendered and weightbased data that have been extracted from twitter and reddit. this assisted in the detection of different emotions being expressed, the composition of word frequency patterns and the broader fat-based themes underpinning the hateful content posted online. the framework has provided a synthesis of quantitative and qualitative methods that draw on social science and data mining techniques to build real-world knowledge in hate speech detection. current information systems research is limited in its use of mixed analytic approaches for studying hate speech in social media. our study therefore contributes to future research by establishing a roadmap for conducting mixed-method analyses for better comprehension and understanding of hate speech patterns. keywords: social media, hate speech, sentiment analysis, topic modelling, discourse analysis, fat stigma. australasian journal of information systems wanniarachchi et al. 2023, vol 27, research article hate speech pattern in social media 2 1 introduction social media has become an integral part of everyday lives enabling users to freely voice their opinions anytime over online platforms. the number of social media users is seen to be growing at a rate of 13% on a year-on-year basis, with current over 4.20 billion users, which represents 53% of the world’s total population (kemp, 2021). online platforms have changed the way we interact with each other, and the way we acquire news (gallacher, heerdink, & hewstone, 2021), enabling users to freely express themselves using emotive language when arguing over contested topics. unfortunately, users may target certain community groups in these emotional exchanges which could promote negativity or discrimination against them (vigna et al., 2017). communities can be aligned to multiple identities including that of race, religion, gender or sexual orientation, which can in turn lead to multiple levels of discrimination (mossie & wang, 2020). behavioural researchers are therefore keen to make sense of how public opinions are framed around certain topics or what prejudicial views are expressed in online conversations. offensive behaviours are often motivated by the anonymity facilitated by social media platforms as people are known to be more comfortable in expressing provocative views when they know they are anonymous (erjavec & kovačič, 2012). while anonymity has a liberating impact on users, it also reduces their hesitation when expressing contempt toward those posts that they do not agree with (tucey, 2010). anonymity can lead to ‘deindividuation and disinhibition’ when users let go of their emotions without fear of being judged for their aggressive posts while discussing controversial topics (de brún et al., 2014, p. 74). it liberates users from being held accountable for their hateful messages (delgado & stefancic, 2014) as they openly reveal their prejudices (mondal, silva, & benevenuto, 2017). hate speech can be thus advanced and cause further divisions based on religion, colour, race and body type amongst others. users often experience more insults in the online space compared to physical spaces, as the internet is not designed to safeguard users from online abuse (zhong, 2020). therefore, social media platforms can promote stigmatization if users feel safe in posting negative content online. it is indeed challenging to mitigate the impact of hateful commenting without having a proper understanding of the types and targets of hate speech, as comments contain overlapping targets (salminen et al., 2018). hateful comments may target various communities based on their sexual orientation, ethnic origin, religious faith, gender identity or body weight or any combination of these (albadi, kurdi, & mishra, 2018; matamoros-fernández & farkas, 2021; ștefăniță & buf, 2021; wanniarachchi et al., 2019). the offensive content posted online is therefore very subjective with no clear definition of the target (i.e., the person or community being addressed in the hate speech); hence, human interpretations of societal events must complement the data-centric computational approaches for analysing its hateful (kocoń et al., 2021). among the many forms of hate speech, weight-based teasing or fat shaming or anti-fat talk reinforces the ‘cultural preoccupation of obesity as individual responsibility’ to further perpetuate weight discrimination (brewis, sturtzsreetharan, & wutich, 2018, p. 1). as such, body weight is perceived with a judgemental stance, with obese individuals having to experience fat stigma on an ongoing basis. the past three decades have witnessed a startling increase in worldwide obesity rates (who, 2022); moreover, obesity has been linked to metabolic disorders, cardiovascular diseases and other chronic illnesses (rawal et al., 2020; szepietowska et al., 2016). alongside health risks, other perceptions associated with ‘ideal’ body shapes and sizes too have evolved. body image australasian journal of information systems wanniarachchi et al. 2023, vol 27, research article hate speech pattern in social media 3 is described as cognitions, perceptions, and attitudes towards one’s appearance, where an individual may measure their bodily features with some overall level of attractiveness (izydorczyk et al., 2021). when such forms of ‘ideal’ body image perceptions become prevalent in mass media and social media, they can reinforce thinness as a form of ideal body size, leading to more expectations across body shapes in society (maftei & merlici, 2022; mccomb & mills, 2022; veldhuis, konijn, & seidell, 2014). as such those people who do not fit within the ideal body classification face much discrimination. the social devaluation and denigration by blaming and shaming individuals based on their weight gain lead to fat stigmatization (tomiyama, 2014). this could be in many forms, such as derogatory humour, sarcastic comments or unflattering visuals amongst others. consequently, these individuals experience feelings related to body dissatisfaction and anxiety, which can further result in eating disorders, depression or poor self-esteem (albadi et al., 2018; larson, 2021; murakami & latner, 2015). a theorised pathway linking gender with body image by fredrickson and roberts (1997) is of the view that women tend to take an observer’s view as the primary view of their bodies. this perception can lead women to continuous self-monitor their bodies which could increase self-shaming behaviours. fat studies have revealed higher rates of internalized stigma and risks of weight-based discrimination toward women in education and employment (himmelstein, puhl, & quinn, 2017; puhl, andreyeva, & brownell, 2008), and that, obese female children/adolescents encountered more teasing and social marginalization (almenara & ježek, 2015; tang‐péronard & heitmann, 2008). hate speech expressions over social media, specifically hate content targeting excessive weight (or fatness) have been much studied (i.e., brooker et al., 2018; chou, prestin, & kunath, 2014; holmberg et al., 2018; hussin, frazier, & thompson, 2011; jeon et al., 2018; kent et al., 2016; lydecker et al., 2016); however, very few of these have used natural language processing (nlp) techniques (wanniarachchi et al., 2022). prior studies are found to be limited to manual analysis techniques that make use of small text-based datasets acquired from social media, but recent advances in machine learning allow for large datasets to be effectively analysed computationally. for instance, sentiment analysis enables text mining of large amounts of user opinions (in some given context) to reveal overall attitudes and the emotional content of these expressions (liu, 2012). it can facilitate the detection of hate content implanted in open-ended discussions by extracting the negative sentiments associated with sensitive topics. further, topic modelling, a machine learning technique, can be used to draw out common themes from the unstructured textual data within a given linguistic corpus (wallach, 2006). in this manner, a systematic assessment of topics, sentiments and emotions embedded in social media conversations can be undertaken to reveal hate speech patterns (lipizzi, iandoli, & marquez, 2015). additionally, discourse analysis can augment these computer-mediated quantitative approaches with a qualitative perspective. discourse analysis is a linguistic methodology that identifies social ideology on various topics through language analysis. it has been used to identify societal stances on topics such as racism, feminism or sexism (chiril et al., 2020; durrheim et al., 2018; thompson, rickett, & day, 2018). therefore, a mixed-method analytical approach comprising machine learning techniques (e.g., sentiment analysis, emotion analysis, topic modelling) and discourse analysis, can help overcome current limitations in detecting hate speech (fat stigma) patterns (chou et al., 2014; yeruva, junaid, & lee, 2019) to offer richer and more meaningful conversational insights. australasian journal of information systems wanniarachchi et al. 2023, vol 27, research article hate speech pattern in social media 4 2 research objectives prior studies have asserted that researchers must expand computational (or quantitative) capabilities with societal (or qualitative) interpretations when investigating any mediafacilitated public discussions (brooker et al., 2018; jeon et al., 2018; lydecker et al., 2016) (brooker et al., 2018; jeon et al., 2018; lydecker et al., 2016). that is, while computational approaches detect linguistic patterns within the data, qualitative approaches render meaning to the patterns that have been produced (ophir, walter, & marchant, 2020). the computational patterns that are extracted from empirical data need descriptive explanations to glean insight and convincingly represent its meaning for informing theory and practice (miranda et al., 2022). following these calls from communication researchers, our study proposes a combination of analytical approaches for examining online discourses that are framed around hate speech (containing fat stigmatizing content). two objectives are laid out in this investigation. first, we outline a novel mixed-methods framework that provides a synthesis of quantitative and qualitative text analysis approaches for detecting hate speech patterns from a large linguistic corpus. multiple methodological lines of inquiry can provide more purposeful reflexivity in the framing of narratives within their given context to generate new insights (ophir et al., 2020). to that end, our framework establishes a roadmap for applying multiple approaches like sentiment analysis, topic modelling and discourse analysis to detect hate patterns in social media discussions. second, we demonstrate how this framework informs the data analysis and interpretation of results by using large unstructured textual data extracts from two popular social media platforms. the context of our investigation is a practical hate speech example that specifically relates to discrimination of fat individuals. different data-driven analysis techniques are applied to detect underlying fat stigmatizing patterns embedded in social media conversations. current information systems research is much limited in their use of mixed analytic approaches to interpret patterns within a given corpus of texts. information systems researchers need to consider emergent computational tools in conjunction with traditional qualitative (and quantitative) approaches to construct empirical patterns across large populations and inform theory (miranda et al., 2022). this study therefore contributes to methodological design by combining multiple analytic frames that together enable meaningful interpretation of large amounts of textual data for identifying linguistic patterns within a data corpus. 3 literature review hate speech comprises public statements made in discriminatory language intended to denigrate specific individuals or community groups (delgado & stefancic, 1991). this section elaborates on prior literature to highlight methodologies that have been used to detect hate speech and fat stigma in social media. we specifically focus on the application of data-driven analysis techniques. our findings indicate sentiment analysis and topic modelling are key techniques currently used for hate speech detection from unstructured textual social media data extracts, followed by emotion analysis and discourse analysis. next, we delve into how these detection techniques have been applied for identifying fat stigma patterns. different literature streams have helped lay out a methodological framework (in section 4) for conducting a robust analysis that is substantiated with a logical chain of evidence when combining nlp techniques with other pattern matching methods (e.g., word frequencies, discourse analysis). australasian journal of information systems wanniarachchi et al. 2023, vol 27, research article hate speech pattern in social media 5 3.1 detecting hate content in social media many studies on hate speech detection in social media have been inspired by sentiment analysis and topic modelling methods. while some studies have used sentiment analysis, topic modelling, discourse analysis or emotion analysis, few have suggested a combination of these. emotional profiles of social media comments were analysed by martins et al. (2018) to classify hate speech based on different emotions and their intensity. their study highlights that emotion analysis can aid in improving the accuracy of hate speech detection. emotion analysis, along with sentiment analysis was also used by rodríguez, argueta, and chen (2019) to cluster relevant facebook pages and filter negative posts/comments that promote hate speech. their study indicates the importance of studying different aspects of hate speech by identifying sentiments and emotions of hate speech and integrating k-means clustering to determine the most frequently discussed topics. sentiment analysis was also employed by irani et al. (2020) to study social media data in the indonesian language for the study of radicalism in indonesia. they have used word embeddings (word2vec) to detect the top-most radical words and combined restricted boltzmann machine and back-propagation (rbmbp) network to enhance the classification of hate speech content which leads to radicalism. al-utbi (2019) underlines the usage of critical discourse analysis (cda) to determine hate speech in social media. the author studied facebook posts targeting islam and muslims; they found the language of the posts and the semiotic details to indicate the existence of expressed hatred. when identifying commonly discussed topics in hate speeches, topic modelling has been used to reveal rich information about user preferences, emotions and associated behaviours. topic modelling and deep learning were used by alshalan et al. (2020) to study hate speech tweets during the covid-19 pandemic. using convolutional neural network (cnn) and nonnegative matrix factorization (nmf), they detected the most hateful tweets and identified the main themes. overall, the number of non-hate tweets were found to be higher than hate tweets. unsupervised topic modelling too has applied to hate speech against immigrants in spain with the appearance of far-right party vox by calderón, de la vega, and herrero (2020). their study discovered underlying themes associated with immigrants in spain that were similar to the discourse of vox. latent dirichlet allocation (lda), an unsupervised topic modelling technique, was used by mathew et al. (2020) for performing temporal analysis on hate speeches made on posts in gab.com (i.e., a social media site with a lax moderation policy compared to mainstream social media sites like twitter and facebook). using data from different time points, the authors explored various topics of interest and found that hateful users tend to speak about blacks, muslims, jews and politics, while the non-hateful users generally speak about politics, technology, sports and free speech matters. furthermore, some researchers have considered both sentiment analysis and topic modelling methods for detecting hate speech and analysing the associated patterns. deephate, proposed by cao, lee, and hoang (2020), is a novel deep learning model which combines word embeddings, sentiments and topical information to detect hate speech in online social platforms. authors evaluated their model using three publicly available datasets and demonstrated that deephate ‘outperformed the state-of-the-art baselines’ (p. 18) and concluded that a combination of multiple text analysis methods results in effective hate speech detection. aspect-based sentiment analysis (absa) was used by pronoza et al. (2021) for instance-based hate speech detection in russian social media. they further observed that the models could benefit from a combination of linguistic and sentiment features with bert preaustralasian journal of information systems wanniarachchi et al. 2023, vol 27, research article hate speech pattern in social media 6 training and an additional dense layer. shibly, sharma, and naleer (2021) used sentiment analysis to validate the sentiment values of a hate speech dataset and combined it with topic classification to identify the most common topics within the dataset. the study observed that the most common hate speech types were related to race, ethnicity, sex and religion. therefore, the proliferation of computational approaches to detect bias or discriminatory messages from unstructured and unlabelled social media data is growing fast. 3.2 detecting obesity content in social media methods associated with fat stigma studies on social media discourses were systematically reviewed by wanniarachchi et al. (2022). the review ascertained that textual data analysis in most studies was done mainly using manual qualitative coding approaches, although, few studies have used sentiment analysis, topic modelling and emotion analysis too. sentiment analysis and word co-occurrence analysis has been used by wanniarachchi et al. (2019) to reveal gender-based fat stigmatization content in twitter and youtube. the findings from this study observed that female body objectification is more commonly displayed in fat stigma content than male body objectification. sentiments expressed toward the fat acceptance (fa) movement and links between the movement and user characteristics have been studied by bograd, chen, and kavuluru (2022) using sentiment analysis. their model indicates extreme anti-fa discourse as most fa tweets displayed ‘opposing’ views. kent et al. (2016) examined the overall sentiments towards the topics of cancer and obesity using sentiment analysis and employed manual coding to uncover discussion themes that connect cancer and obesity. sentiments indicated the “dominance of negative framing around obesity” (p. 457). lazarus et al. (2021) examined stigma associated with non-alcoholic fatty liver disease (nafld), non-alcoholic steatohepatitis (nash) and obesity using twitter data. the study analysed sentiments, geographic distribution and hashtag patterns of obesity-related content and detected much negativity in the majority of tweets on obesity. chou et al. (2014) also employed sentiment analysis and qualitative analysis to study social media discussions related to obesity. the study confirmed the stigmatization of fat people and revealed patterns of such discussions, such as themes and sentiments. the combination of multiple text analysis methods was also tested by yeruva et al. (2019) to examine the relationship between obesity and healthy eating. sentiment analysis, cooccurrence analysis and topic modelling aided the authors in understanding the social media influences on behaviour and decision-making for healthy eating and obesity prevention. while recent studies conducted for detecting hate speech and fat stigma in social media have used sentiment analysis and/or topic modelling to provide evidence that such techniques can capture hate speech in social media (i.e., bograd et al., 2022; chou et al., 2014; kent et al., 2016; yeruva et al., 2019), these are limited in their application. the literature review reveals very few studies have analysed hate speech discourses surrounding fat stigma that can show meaningful semantic patterns. uncovering behavioural aspects such as what perceptions revolve around discussions on exercise/diet plans or unhealthy lifestyles will help create a better understanding of the underlying stigmatizing content. discourse analysis can enable social scientists to recognize emergent stigma patterns present in the given social context. moreover, prior studies have mostly considered twitter as their single data source (wanniarachchi et al., 2022); but, with the text-limit constraints imposed on tweets, users have to adapt their intended message within the specified data bounds. we propose that by combining data from multiple social media platforms, we can inspect richer content. reddit is australasian journal of information systems wanniarachchi et al. 2023, vol 27, research article hate speech pattern in social media 7 one such contemporary social media platform that facilitates candid naturalistic expressions from its users to open up many methodological possibilities for conducting thematic analysis (boettcher, 2021). reddit and twitter platforms can together assist in unravelling the hidden patterns in hate speech, specifically for fat stigma. the use of novel text mining and machine learning methods along with in-depth text analysis techniques such as discourse analysis can aid in the discovery of hate speech patterns. however, we find a gap in the provision of guidelines or a frame of reference that encompasses multiple methods for facilitating detection and enables researchers to comprehend the stigma patterns embedded in social big data. the next section builds on these shortcomings wherein a methodological framework is proposed. steps on data collection, data pre-processing and data analysis for classifying different user perspectives are outlined for enabling researchers in deducing patterns that project a more holistic picture. 4 methodological framework having an overarching methodology lends support to the researcher by providing them an understanding of the data context and establishing proper checks for collecting empirical data that align with the study’s purpose. it further ensures that appropriate analytical approaches are subscribed for examining and interpreting the data. figure 1. proposed methodological framework australasian journal of information systems wanniarachchi et al. 2023, vol 27, research article hate speech pattern in social media 8 moreover, a graphical illustration aids in understanding concurrent and sequential steps as researchers move back and forth with their data analysis. having a high-level framework that identifies broad activities needed within each methodological step provides operational flow and precludes potential mistakes of missing out some tasks within each step. the proposed methodological framework, outlined in figure 1, provides a detailed representation of what tasks are required at each of the identified steps, namely data collection, data pre-processing, data analysis and pattern identification. this study aims to uncover underlying patterns within textual content extracted from social media to identify fat stigmatizing patterns. moreover, our proposed framework can be applied to other forms of hate speech as well. it follows nlp workflow for data collection and data pre-processing and subsequently combines sentiment analysis, emotional analysis, topic modelling, word frequencies and discourse analysis to assess fat stigma patterns in different social media platforms. the outputs of sentiment analysis can aid in filtering the discourse, for instance, only negative sentiments and the associated negative content can be used as inputs for conducting topic modelling to identify fat stigmatizing themes. apart from discovering these topics/themes, topic modelling can highlight the most dominant posts/comments within each topic, which can then be used to carry out discourse analysis. this framework is based on the research directions implied by recent literature and is further validated by the authors’ experiences in analysing fat stigma content in social media. the following subsections describe the activities pertaining to each step. 4.1 data collection data collection starts with a prior understanding of the context, that includes the selection of relevant keywords and determining which data sources need to be explored. therefore, relevant literature must be examined to identify pertinent keywords that reflect the chosen context and serve as a starting point for collecting data. for example, this study focuses on uncovering fat stigma, accordingly, keywords have been determined from obesity-related literature. keywords comprising ‘fat’, ‘overweight’ and ‘obesity’ were thus identified (i.e., chou et al., 2014; so et al., 2016) and these informed the text mining process. moreover, based on the direction of the research path, the keywords can be built upon further. that is, if a study seeks to investigate gender aspects in fat stigma discussions, other keywords associated with gender are added to the keyword list (i.e., ‘fat+girl’, ‘obese+man’, etc.). next, relevant social media data sources need to be determined. empirical data that is contextdependent on the selected keywords are subsequently scraped for further analysis. fat studies have made significant use of twitter in comparison to other social media platforms (wanniarachchi et al., 2022); however, other platforms, such as reddit (which appears “more as a discussion forum, and [its] comments have a wider range of document lengths than twitter data” (curiskis et al., 2020)) too are gathering momentum. twitter and reddit platforms served as our primary data sources. further, researchers must scope their study to the type and format of digital data that must be used in the knowledge discovery process. this study’s context refers to the discovery of linguistic patterns concerning fat-stigma; hence, corpora of texts extracted from obesity-related discussions informed our investigation. the publicly available data scraping tools or apis (reddit, 2020; twitter, 2019) assisted in collecting textual data for inclusion in the corpus. since a corpus of texts is characterised by words that are independent of any non-text format (i.e., image, video, etc.), conducting australasian journal of information systems wanniarachchi et al. 2023, vol 27, research article hate speech pattern in social media 9 sentiment analysis on the observed texts is rather straightforward (curiskis et al., 2020; wanniarachchi et al., 2022). a major issue often encountered while collecting social media data is of noise or the acquisition of irrelevant data. for instance, one example of irrelevant data relates to acquiring text content posted by a user whose username matches the keywords used for scraping data. if a keyword combination is ‘fat + girl’, then comments posted by a user named ‘fatgirl123’ would be scraped even though they are not relevant to the topic under consideration. or, in another case, while the given keyword exists in the scraped comment, the context of its use is different. that is, the keyword ‘fat’ could be used in different scenarios, such as, the comment has a line containing words like ‘big fat liar’. such cases can be partially solved by using a combination of keywords relevant to the study's context instead of just one keyword; although when the length of the textual content is large, this method may not be workable. in our case, we found that a combination of keywords provided us with more relevant data from twitter compared to reddit, as twitter enforces a text-limit on each post. 4.2 data pre-processing pre-processing steps assist in presenting the data in a more structured format for conducting effective and accurate analysis. pre-processing techniques, including removal of duplicates, noise and stop words and word lemmatization, are described next. 4.2.1 duplicates removal when scraping data from social media, one of the key issues is data duplication. on twitter, most of the duplicates occur with re-tweets. although re-tweets can be eliminated at data collection stage for twitter, the duplicated posts on reddit must be explicitly removed. duplicated posts occur on reddit because users often post the same content on multiple subreddits. the ‘remove duplicates’ feature in microsoft excel can aid in resolving this (microsoft corporation, 2016). 4.2.2 stop words and noise removal creating insights from raw textual data is complicated; moreover, unnecessary data can cause distractions to produce results that are inconsistent in the subsequent downstream tasks. therefore, removing stop words and noisy data are important pre-processing tasks. common stop words include prepositions (of, to, for) and conjunctions (and, or also) make a language functional rather than carry information (boban, doko, & gotovac, 2020). additionally, other superfluous characters such as symbols (@, !, *, etc.), emojis (:d, :’(, etc.) and urls can add complexity, that makes the overall analysis process harder to execute. hence, all such noisy characters need to be removed. 4.2.3 word lemmatization stemming and lemmatization are used to convert a word to its basic form. stemming refers to a crude heuristic process that truncates words in the hope of achieving their primitive meaning. this often includes the removal of derivational affixes. lemmatization uses “vocabulary and morphological analysis for removing the inflectional endings and returning the word to its base or dictionary form, also known as the lemma” (manning, raghavan, & schütze, 2008, p. 32). however, with stemming, the resulting dataset may contain some words with no meaning. lemmatization does not result in meaningless words since token words are presented in their base form by using their origins, irrespective of whether the text is used as australasian journal of information systems wanniarachchi et al. 2023, vol 27, research article hate speech pattern in social media 10 a verb or as a noun (refer to table 1). therefore, lemmatization plays an important role in text mining applications. token word stemmed word lemmatized word troubled troubl trouble saw s see table 1. stemming and lemmatization 4.3 data analysis the researcher needs to conduct a meaningful analysis of the pre-processed data for establishing generalizations. the data analysis provides overall sentiments and emotions of the hate speech data gathered and cleansed in the previous two steps. sentiment analysis can reveal the users’ attitudes (which may be positive, negative or neutral) as expressed in the selected hate speech context, while emotion analysis indicates feelings based on the choice of words (e.g., anger, surprise, trust, etc.). data analysis, in this study’s context, is deciphering of the fat stigma content expressed in social media discussions in a cumulative and holistic manner. first, it is important to have a proper grounding of the overall themes around the discussions on fatness. sentiment analysis and emotion analysis can assist in building themes that are bottom-up and data-driven. the nrc dictionary developed by mohammad and turney (2013) can capture in-depth sentiments associated with a linguistic corpus and identify a variety of emotions. therefore, all of the scraped data (comprising all acquired posts/comments extracted from twitter and reddit) are organized into one document for the purpose of executing sentiment analysis. in this manner, we can obtain gross positive and negative sentiment scores from the document with no researcher bias in play. this form of analysis is also beneficial for comparing two sets of data. next, emotion analysis can be applied to reveal the scores of the eight primary emotions, namely anger, anticipation, disgust, fear, joy, sadness, surprise and trust, for this linguistic corpus objectively. further, with pyplutchik (semeraro, vilella, & ruffo, 2021), we can interpret how these emotions are related and highlight which emotions are more prominently expressed. subsequently, sentiment analysis using stanford’s corenlp (manning et al., 2014), can be performed on specific posts/comments of the corpus to determine their sentiment scores and labels. sentiment score or valence score is determined by assigning a sentiment value (ranging from -1 for negative to +1 for positive and 0 for neutral) to each word and then by aggregating all sentiment values of the specific post/comment. therefore, posts/comments with more negative words indicate a higher negative valence score, and posts/comments with more positive words will indicate a higher positive valence. if there are equal amounts of negative and positive words in a text, the valence score would be '0' and be considered a neutral text. based on these sentiment scores, each post/comment is labelled as a positive, negative or neutral sentiment. next, by considering those posts and comments that indicate negative sentiment label, we can filter the linguistic corpus to obtain the hateful content. this can then be used as the input for the next stage, that is, to identify hate speech patterns (or fat stigma patterns in this study’s context). 4.4 pattern identification the purpose of pattern identification is to deduce those prominent features that have a shared societal context (e.g., word frequencies, topics, themes) from the big dataset. therefore, the researcher must first identify their context and construct meaning based on that context. in australasian journal of information systems wanniarachchi et al. 2023, vol 27, research article hate speech pattern in social media 11 this study, our context relates to gaining insights on fat stigma. the content that displayed negative sentiment in the sentence-based sentiment analysis has been used as the input for identifying common topics/themes and for determining those words that are most frequently associated with fat stigmatizing discussions on social media. topic modelling based on latent dirichlet allocation (lda) (blei, ng, & jordan, 2003) can inform on the probability distribution of topics spread over documents, based on their word distribution. gensim lda implementation (řehůřek & sojka, 2011) has been used to execute topic modelling on filtered fat stigma content. the topic models thus generated were evaluated by first calculating the coherence score. coherence score is the value of the relative distance between words within a topic. there are two major coherence score types, c_v (which typically range between 0 and 1) and umass (that range between -14 and 14). this study used the c_v scores as it provides the strongest correlation with human ratings and is considered a very reliable topic coherence evaluation measure (röder, both, & hinneburg, 2015). in order to determine the optimal number of topics within the male and female corpus, we generated multiple models, each with a different number of topics ranging from 2 topics through to 10 topics. we then used the model with the highest coherence score as the most optimal for determining the number of topics for each corpus. once the optimal number of topics was determined for the two datasets, the models were further tuned and optimized the models using alpha and beta hyperparameters. apart from identifying the discussion topics of the dataset, the most dominant topic of each post/comment and the most dominant post/comment for each topic helped determine patterns from our analysis. the most dominant post/comment on each topic has next been used to perform discourse analysis and comprehensively examine the negative social media content. selecting the dominant content via topic modelling eliminates the subjectivity associated with excerpt selection for discourse analysis and facilitates the examination of the most representative posts/comments in the corpus. discourse analysis can then be carried out manually by closely examining the text and exploring how social media users use language in relation to the fat stigma context. concurrently, word frequencies of the data corpus too can be used to identify words that are mostly associated with fat stigma. 5 fat stigma investigation this section presents a social media investigation that elaborates how nlp and qualitative methods have together provided multi-level insights from the textual data extracts. we specifically demonstrate the empirical analysis of fat stigma content that was extracted from social media discourses around female and male genders. we believe this empirical evidence will showcase how our proposed methodological framework (shown in figure 1) can be applied to practical real-world scenarios. 5.1 data collection and data pre-processing the analysis has been carried out to understand how males and females are positioned in obesity-related discussions. therefore, keyword combinations that represented fatness and gender were selected to scrape data (refer to table 2). as this analysis is focused on textual data only, the publicly available text-based content from twitter and reddit was scraped (via their apis) from 15th october 2021 to 15th january 2021. the data thus collected served as our data corpus. australasian journal of information systems wanniarachchi et al. 2023, vol 27, research article hate speech pattern in social media 12 obesity-related keywords fat, overweight, obese gender-related keywords female, girl, woman, women, male, boy, man, men keyword combinations used fat + (female, girl, woman, women) overweight + (female, girl, woman, women) obese + (female, girl, woman, women) fat + (male, boy, man, men) overweight + (male, boy, man, men) obese + (male, boy, man, men) table 2. keyword combinations a closer examination of the number of posts/comments scraped for different keyword combinations were not found to be equally distributed; rather, a previous study found a larger number of posts/comments for the keyword “fat” compared to “overweight” and “obese” (wanniarachchi et al., 2019). due to this difference, we selected an equal number of posts/comments representing each keyword combination from each social media platform through random sampling. data comprising 60000 comments/posts representing every keyword combination across both social media platforms were finally selected. in maintaining equal data distribution, we formed dataset samples consisting of 30000 comments/posts targeting males and an equal number targeting females. further, each of these datasets (having 30000 comments/posts) in turn comprised 10000 comments/posts representing each keyword combination. from the 10000 comments/posts that originated from twitter and reddit, 5000 posts/comments were from twitter and the other 5000 posts/comments from reddit. the datasets were cleansed next by removing stop words, symbols and emojis. later, lemmatization was also applied to further cleanse the dataset for subsequent analysis. 5.2 data analysis: overall text analysis of fatness this section discusses the three data analysis approaches that were used to unearth sentiment values, conduct emotion analysis and generate sentiment labels for text-blocks representing posts/comments. the following subsections expand on each of these approaches. 5.2.1 sentiment analysis the sentiment value for each selected comment/post has next been calculated using the nrc dictionary. the calculated sentiment values associated with females and males were then compared to identify patterns. the graphical representation of the distribution of overall sentiment (valence) scores associated with male and female genders are presented in figure 2. the sentiment values for males and females show considerable overlap with no significant difference in the distributions across negative and positive sentiments. however, we find that social media content targeting males indicates slightly more negative sentiments than positive sentiments. we therefore delved into gaining further insight into the emotions embedded within these sentiments. the following section expands on the emotion analysis results. australasian journal of information systems wanniarachchi et al. 2023, vol 27, research article hate speech pattern in social media 13 figure 2. sentiment value comparison 5.2.2 emotion analysis the eight primary emotions – joy, trust, anticipation, sadness, anger, fear, surprise and disgust – from plutchik’s wheel were analysed for the whole linguistic corpus. the 30000 comments/posts across both males and females showed the highest degrees of emotional expressions being in the form of disgust and sadness, with males receiving slightly more of the former and females more of the latter. the high-intensity values of the disgust category tended more towards loathing, while the sadness category tended towards grief. our overall analysis reveals that females receive higher expressions of emotions generally in fat-based discussions on social media compared to males (see table 3). plutchik wheel of emotions – male plutchik wheel of emotions female table 3. plutchik's wheel of emotions the male and female emotions displayed in plutchik’s wheels represent huge amounts of social media big data spread across the eight segments. while data amounts used during analysis can be huge, researchers are advised to scale their data to a coarser level with a smaller subset of data and consider such visuals that make their presentation more readable to their australasian journal of information systems wanniarachchi et al. 2023, vol 27, research article hate speech pattern in social media 14 audience (e.g., use bar charts for ranking and comparisons) (kosara, 2016). therefore, in figure 3 we have highlighted the differences in the expressed emotions between males and females using percent differences. this figure depicts the percentage of overall differences between males and females that each emotion describes. positive values describe emotions where the expression score was higher for females than males, and vice versa for negative values. figure 3. percentage of all differences between males and females explained by each emotion the figure indicates that some 25% of the total differences between emotions expressed between males and females can be explained by joy, where the females received a greater share. indeed, the graph highlights that females received greater expressions of emotions across all categories, except for the disgust emotion which was more noticeable for males. 5.2.3 sentiment labelling to extract sentiment polarity, sentence-level sentiment analysis has been carried out next. table 4 illustrates sentiment polarity labels (negative, neutral and positive) of three sentences which have been pre-processed (i.e., cleansed of stop words and other noisy characters). after obtaining the labels for each text-block, only the content which indicated negative sentiment labels was filtered and used as the input for pattern identification stage. cleaned text valence sentiment fat woman think theyre hot s**t please know die u -1 negative hit home skinny friend complain weight look like model 0 neutral also im girl will will literally see beautiful plus size girl tv 1 positive table 4. sample data for sentiment labelling 5.3 pattern identification of fat stigma discussions the underlying themes or patterns related to fat stigma have next been deduced with topic modelling approaches, word frequencies and discourse analysis to gain deeper insight on each topic. the following subsections elaborate on each of these methods. 5.3.1 topic modelling topic models were generated by using gensim lda implementation for male and female datasets. the process outlined in section 4.4 was used to determine the optimal number of australasian journal of information systems wanniarachchi et al. 2023, vol 27, research article hate speech pattern in social media 15 topics for each dataset. the analysis yielded 5 topics for female dataset and 4 for the male dataset. following the alpha and beta hyperparameter tuning for both models, table 5 presents a before-and-after example of the degree to which the model coherence scores can be improved as a result of the model tuning process. optimal number of topics coherence score of initial model alpha beta coherence score of final model male dataset 4 0.36 0.61 0.01 0.66 female dataset 5 0.36 0.91 0.01 0.65 table 5. model parameters and coherence scores after discovering the keywords generated in each topic, the authors reviewed these keywords to determine a suitable label for the topic. the set of keywords pertaining to a topic generated for each dataset along with topics’ labels are displayed in table 6. the derogatory and offensive words included in the generated topics are hidden by using the ‘*’ character. fat stigma content targeting males keywords of generated topic topic label male, black, woman, good, fat, girl, man, want, feel, think opposite-sex video, mom, time, d**k, never, eat, play, boy, call, watch lifestyle f**k, say, gay, little, a*s, guy, thing, come, drink, still sexual get, big, go, force, make, know, t*t, mind, also, way exercise fat stigma content targeting females keywords of generated topic topic label sex, woman, good, want, man, start, day, bad, new, sure relationships get, go, make, say, take, see, guy, thing, never, leave lifestyle f**k, big, girl, hot, p***y, c**k, a*s, mom, sexy, give sexual even, know, look, fat, think, body, little, tell, also, still body female, black, naked, time, love, people, old, mind, blonde, really appearance table 6. generated topics and topic labels 5.3.2 word frequencies the most prominent words of the corpus have been identified for both males and females in figures 4 and 5. we noted more derogatory words were observed in the male dataset compared to the female dataset, although some of these derogatory words referred to obese/fat males in female terms (e.g., mom, woman, girl). the use of feminine words to insult obese/fat males could indicate body shaming attitudes towards females, or, in other words, this could also indicate that voicing body objectifying content targeting females is considered acceptable. australasian journal of information systems wanniarachchi et al. 2023, vol 27, research article hate speech pattern in social media 16 male figure 4. word frequencies male australasian journal of information systems wanniarachchi et al. 2023, vol 27, research article hate speech pattern in social media 17 female figure 5. word frequencies female 5.3.3 discourse analysis discourse analysis has been conducted by closely examining the most dominant comment/post for each of the identified topics. as a result, some more key features relating to fat stigma discussions were identified. the fat stigma content targeting males indicates several features. for instance, some users shared scientific studies that they come across on social media; “people with metabolically healthy obesity were at a substantially higher risk of diabetes, heart attack and stroke, heart failure, respiratory diseases, and all-cause mortality compared with people who were not obese and with a healthy metabolic profile”. such content cannot be directly linked with fat stigma as the main purpose of sharing such information would be to educate readers. however, there is a possibility that a fat person may feel victimized based on such claims. some other content revealed that the social media users consider fatness as ‘unmasculine’; “they are either smart, nerdy, short, fat, poor. essentially unmasculine by society”. such statements further indicate that some users consider their australasian journal of information systems wanniarachchi et al. 2023, vol 27, research article hate speech pattern in social media 18 perspective as society’s perspective. individuals, who consider themselves obese, on reading such comments can adopt such statements and believe that this is how society sees them. obesity is found to be most commonly linked with diet and eating habits by social media users; “ate fast food everyday (led to me becoming extremely obese for my age)”. some even provided tips on losing weight; “i’ve been the obese 13 year old that wanted to lose weight and i asked everyone what to do about it. my parents didn’t know because they had bad eating habits themselves…. the only one who told me the secret recipe was a friend. “well i just don’t eat more than once a day, i only eat boiled veggies and i don’t drink much water because it bloats you”. and that was my diet for the next three years (13-16) and i lost all the weight reaching 50kgs/110lbs”. such statements connect body weight with lifestyle choices and mostly try to motivate people to adopt ‘healthy’ lifestyles. some users exhibited fear of rejection because of their fat bodies while at the same time indicating fat people are unattractive; “to my horror i find myself being obese, bald, and unattractive. this shocked me, if i ever find my girlfriend, is this how i will look to her? how am i going to explain myself? is she going to just dump me? and now, how am i going to reach her?”. these statements are directly linked with body objectification as the connection of body weight with attractiveness is established here. it further highlights that body objectification occurs in males as well as in females. when targeting females on fat stigma, few social media users linked fatness with appearance; “but i can confidently say she is not the sperm whale you make her sound like. from my general search, she is what some folks would call ‘thick’ which is not technically considered fat, as she does not have a belly sticking out” and sometimes with health and well-being; “your girlfriend is overweight, but she is not obese. anytime you are overweight, there is some possible significant health risk, and it can be great that you care about her well-being”. these comments further highlight the connection of body weight with lifestyle choices. the dataset also revealed that users share scientific and statistical studies relating to obesity to educate the readers; “5.8 percent of healthy-weight females with no sexual assault history attempted suicide. the percentage rose to 27.1 percent for healthy-weight girls with a history of sexual assault. weight influenced the suicide rate among women: 8.2 percent of overweight girls with no sexual assault history attempt suicide”. however, compared to males, fat stigma content targeting females exposed a higher amount of sexual content. in most of these comments/posts, the words ‘fat’ and ‘obese’ were used to describe a person or a female body part, sexually. these contents were mainly posted either to share users’ sexual preferences or to describe visual content. 6 discussion the study has proposed a methodological framework to examine hate speech patterns underpinning discussions in social media. although sentiment analysis and topic modelling methods have been integrated into prior hate speech (or fat stigma) research, the application of such integration to identify patterns of hate speech is poorly discussed. further, very few studies have incorporated natural language processing and machine learning techniques with discourse analysis for uncovering societal aspects that are associated with hate speech. our methodological framework overcomes these limitations to show how sentiment analysis, topic modelling and discourse analysis can be applied to a given dataset. section 6.1 presents deliberations on issues faced when integrating computer-mediated quantitative analytic tools with qualitative discourse analysis to provide a synthesis of multiple approaches for conducting data-driven analysis on large linguistic corpora. next, in section 6.2, we discuss the framework’s application in a practical context with a fat stigma investigation (using data australasian journal of information systems wanniarachchi et al. 2023, vol 27, research article hate speech pattern in social media 19 extracted from twitter and reddit). our analysis of obesity-related content with the selected methods has revealed the linguistic and thematic patterns associated with fat stigma content that is posted on social media conversations. 6.1 application of the framework the paper has proposed a methodological framework that shows the conceptual relevancy of different steps undertaken to detect hate speech patterns in social media. the framework details a four-step process that reveals specific activities within each step explains the conduct of each activity and provides insights on overcoming shortcomings with existing techniques, to report on data-driven findings. the first step of the proposed methodological framework is to identify the most relevant keywords from the published literature that align with the study’s context. it may be noted, however, that the usage of jargon language has evolved; hence, the nature of online commenting specifically on social media too has changed (eisenstein et al., 2014). consequently, the linguistic awareness around words expressed in everyday conversations over social media platforms needs proper examination for currency and relevancy. moreover, in collecting relevant social media data by conducting keyword searches, we are very likely to acquire the same comment/post multiple times. the twitter api assisted in the elimination of retweets (or duplicate tweets) while scraping twitter data. however, the duplicate comments/posts from reddit need to be explicitly removed in the data preprocessing step. we advise that to avoid duplicates from reddit, the data acquisition process considers data from a single subreddit since duplicates mainly occur due to posting the same content on multiple subreddits. next, we have highlighted several methods for noise removal. by removing stop words, symbols, emojis and urls, much of the noise in the text can be eliminated. however, in modern-day texting and social media postings, emojis or emoticons play a considerable role in conveying emotions. for example, “i had a rough day lll” has more emotions than “i had a rough day”. emojis or emoticons can enhance digital communication by adding non-verbal cues (alshenqeeti, 2016). another prominent characteristic of digital text communication is amplifications or abbreviated words, commonly known as sms language, textspeak, or texting language (stieglitz & dang-xuan, 2013). for example, the canonical form of the words ‘gooood’ and ‘gud’ is ‘good’. normalizing social media text is important to achieve deeper text analysis, specifically on social media data. satapathy et al. (2017) observed that the accuracy of sentiment classification has increased by ~4% when using such normalization. therefore, it is clearly important to enhance available text mining techniques to cope with emojis and textspeak for more accurate text analysis. after pre-processing the collected data, sentiment analysis and emotion analysis were undertaken. patterns of sentiments and emotions that were scattered throughout the dataset were revealed, which provided knowledge of the most prominent emotions and sentiments related to a specific subject context. further, sentiment analysis also facilitated in the filtering of the negative content, which led the analysis onward, since our purpose is to understand hate speech patterns that are exposed from negative sentiments. next, the framework guided in in revealing patterns including hidden topics, frequent words and discourse patterns related to the identified negative content of the selected context. these patterns were inferred with the use of sentiment analysis and topic modelling on the corpus of texts and then interpreted qualitatively using discourse analysis (as described in section 4). in this manner, additional analyses can be incorporated to help in disclosing dominant features of hate speech expressions on social media. our framework highlights the use of discourse analysis as an extension of computer-mediated tools and techniques to gain deeper insight into the hate australasian journal of information systems wanniarachchi et al. 2023, vol 27, research article hate speech pattern in social media 20 content. discourse analysis can be combined with topic modelling by connecting filtered comments/posts with the strength/size of the topics to thematise topic categories (shirazi, 2013; törnberg & törnberg, 2016). the knowledge categories thus discovered can provide more societal background information on the specified subject to enable better interpretation (e.g., hate content relating to an incident that has occurred). when seeking hate speech patterns, we need to focus on who the targets of offensive expressions are, or which content or words cause the most impact, or are intense discussions occurring because of some event. the manual exploration in carrying out discourse analysis cannot be overlooked; hence having relevant data extracts are important. therefore, technological solutions, such as sentiment analysis can aid in filtering out relevant posts/comments for attaining more accurate discourse analysis that can be used for larger datasets. the dataset acquired from keyword search needs to be further contextualized in the context of the study objective. hence, to observe hate speech patterns, we must first remove the content that exhibits positive sentiments. the positive sentiments will otherwise act as noise in the detection of hate speech patterns. the content that indicates negative sentiments is carried forward for understanding the hate content. finally, discourse analysis helps to provide a richer understanding of the subjective negative content, rather than labelling all content as offensive content. our framework provides the theoretical grounding for combining sentiment analysis with topic modelling, and subsequently conducting discourse analysis for identifying discussion themes/topics (e.g., fat hate speech) in social media discussions. the following section discusses the result of the data-driven analysis methods (outlined in our proposed framework) for the study of fat-stigma patterns. 6.2 emergent fat-stigma themes we have demonstrated the application of the methodological framework to investigate gender-based aspects related to fat-stigma discussions conducted over social media. datasets, extracted from twitter and reddit, representing male and female genders across fat-related discussions were examined. the study has disclosed noteworthy themes regarding the positioning of males and females from a fat stigma context. sentiment analysis tools showed a mix of positive and negative sentiments for females and males. emotion analysis has revealed the higher intensity of emotions being projected towards females than towards males. we found higher expressions relating to disgust (loathing) targeting males and of sadness (grief) targeting women. the emotion analysis results tuned the sentiment analysis outputs and highlighted that although males received a slightly higher number of positive sentiments, females tend to get more varied emotions compared to males. such findings further justify the decision to use multiple text analysis methods to analyse hate speech. fat-based themes targeting females consisted of diet plans, exercise routines and healthrelated information, with most of these comments/posts having some degree of sexual content. fat-related words were often used to describe female body parts in highly sexed manners. further, these words were paired with other derogatory words that spread more negativity toward females. moreover, males were often belittled by words that aligned with the female gender (e.g., lady, girl). this finding aligns with the objectification theory established by fredrickson and roberts (1997) which takes account of how female bodies are sexually evaluated based on “their bodies or body parts” (p. 176). such body objectification can cause women to adopt an outsider view, where they equate their self-worth with their physical australasian journal of information systems wanniarachchi et al. 2023, vol 27, research article hate speech pattern in social media 21 appearance and start self-objectifying and self-monitoring their body shapes. objectification theory has been used as a theoretical framework for studying the experiential consequences of being a female in a culture which sexually objectifies the female body. (tylka & hill, 2004). the topics that emerged from this study provide rich insights into how weight prejudices are reflected in fat-based discussions. we found common topics from both male and female datasets. these include general categories including body, appearance, time, exercise, and sex. a closer examination of these categories further indicates the extent of sexual objectification. the sexual topics targeting males comprised words that have a strong association with the female gender (e.g., ‘sex,video,girl,…’, ‘sl*t,big,wife,…’). while these findings were observed in a male context, the use of objectified female body parts (i.e., ‘fat a*s’, ‘fat b**t’) itself indicates further stigmatization of women. these fat stigma patterns illustrate the extent of female body objectification taking place and further demonstrate the application of our methodological framework for advancing the knowledge base and its alignment with current social theories. 7 conclusions this paper has presented a methodological framework, which has subsequently been applied to real-world conversations held over social media platforms. specifically, we have studied hate speech patterns related to fat stigma in a gendered context to reveal behaviours. the proposed step-by-step framework provides researchers with methodological grounding on conducting natural language processing with social media data extracts. our framework determines that the context of the problem domain needs to be articulated explicitly so that data-driven insights can be revealed. a key part of using automated analytics is that relevant data is collected and cleansed properly, following which the analysis is context-based to create real-world insights. creating insights is not straightforward, hence we propose guidelines on the use of mixed approaches that include different data-driven as well as human-cantered qualitative analysis methods. current information systems research faces methodological challenges in their use of multiple computational methods for studying large data corpus "to discover different patterns in their data, thereby highlighting different aspects of the phenomenon” which can then be further corroborated with traditional analytical methods (miranda et al., 2022, p. xi). our framework has established a roadmap for conducting mixedmethod analyses for a better comprehension and understanding of conversational patterns (specifically in the context of negatively expressed content). the framework takes hate speech detection a step forward by combining existing nlp techniques to investigate discussion patterns of fat stigma (or hate speech) in a social media context. it bridges the gap between technology and social sciences to enable a deeper understanding of the severity of the socio-psychological impact that fat stigma can cause on fat people. though this study targets fat stigma, the suggested framework can be applied to various categories of hate speech to identify discussion patterns. outputs of such pattern identification can assist in developing machine learning techniques for mitigating hate speech content on social media platforms or other public forums. one of the limitations of this research is that the proposed tools relating to each method of the framework are solely based on literature and researchers’ experience. future studies could compare multiple tools relating to each method and obtain the tools that indicate the most effective results. the proposed framework is focused on text-based hate speech analysis, while hate speech can be observed in many forms, including visual and audio. in future, the research could expand this framework to incorporate methods of visual and audio analysis to overcome australasian journal of information systems wanniarachchi et al. 2023, vol 27, research article hate speech pattern in social media 22 this limitation. also, the application of the framework has been tested on twitter and reddit. however, various types of text-based online media such as forums and weblogs are currently available. therefore, in future, studies could be conducted to identify patterns of hate speech (fat stigma) on these online media platforms. this paper has demonstrated the development of analytical social patterns from real-world conversations by using computational and replicable design methods; therefore, it has implications for conducting more forms of interdisciplinary research. we encourage future research studies to use our proposed methodological framework for investigating societal issues, such as examining online comments posted on social media sites to detect if people are being negatively targeted based on their body image, ethnic background, gender, religion or disabilities amongst similar others. references al-utbi, m. i. k. (2019). a critical discourse analysis of hate speech. journal of the college of languages (jcl) mağallaẗ kulliyyaẗ al-luġāt (39), 19-40. https://doi.org/10.36586/jcl.2.2019.0.39.0019 albadi, n., kurdi, m., & mishra, s. (2018). are they our brothers? analysis and detection of religious hate speech in the arabic twittersphere. paper presented at the 2018 ieee/acm international conference on advances in social networks analysis and mining (asonam).barcelona, spain. 28 – 31 august 2018. https://dl.acm.org/doi/10.5555/3382225.3382239 almenara, c. a., & ježek, s. (2015). the source and impact of appearance teasing: an examination by sex and weight status among early adolescents from the czech republic. journal of school health, 85(3), 163-170. doi: https://doi.org/10.1111/josh.12236 alshalan, r., al-khalifa, h., alsaeed, d., al-baity, h., & alshalan, s. (2020). detection of hate speech in covid-19–related tweets in the arab region: deep learning and topic modeling approach. journal of medical internet research, 22(12), e22609. https://doi.org/10.2196/22609 alshenqeeti, h. (2016). are emojis creating a new or old visual language for new generations? a socio-semiotic study. advances in language and literary studies, 7(6). 56-69. https://doi.org/10.31219/osf.io/4hdgs blei, d. m., ng, a. y., & jordan, m. i. (2003). latent dirichlet allocation. the journal of machine learning research, 3, 993-1022. boban, i., doko, a., & gotovac, s. (2020). sentence retrieval using stemming and lemmatization with different length of the queries. advances in science, technology and engineering systems, 5(3), 349-354. https://doi.org/10.25046/aj050345 boettcher, n. (2021). studies of depression and anxiety using reddit as a data source: scoping review. journal of medical internet research mental health, 8(11), e29487. https://doi.org/10.2196/29487 bograd, s., chen, b., & kavuluru, r. (2022). tracking sentiments toward fat acceptance over a decade on twitter. health informatics journal, 28(1), 1-16. https://doi.org/10.1177/14604582211065702 australasian journal of information systems wanniarachchi et al. 2023, vol 27, research article hate speech pattern in social media 23 brewis, a., sturtzsreetharan, c., & wutich, a. (2018). obesity stigma as a globalizing health challenge. globalization and health, 14(1), 1-6. https://doi.org/10.1186/s12992-018-0337-x brooker, p., barnett, j., vines, j., lawson, s., feltwell, t., & long, k. (2018). doing stigma: online commenting around weight-related news media. new media & society, 20(9), 3201-3222. https://doi.org/10.1177/1461444817744790 calderón, c. a., de la vega, g., & herrero, d. b. (2020). topic modeling and characterization of hate speech against immigrants on twitter around the emergence of a far-right party in spain. social sciences, 9(11), 188. https://doi.org/10.3390/socsci9110188 cao, r., lee, r. k.-w., & hoang, t.-a. (2020). deephate: hate speech detection via multi-faceted text representations. paper presented at the 12th acm conference on web science. southampton, united kingdom, 6 – 10 july 2020. https://dl.acm.org/doi/10.1145/3394231.3397890 chiril, p., moriceau, v., benamara, f., mari, a., origgi, g., & coulomb-gully, m. (2020). he said “who’s gonna take care of your children when you are at acl?”: reported sexist acts are not sexist. paper presented at the proceedings of the 58th annual meeting of the association for computational linguistics. 4055-4066. online. association for computational linguistics. https://aclanthology.org/2020.acl-main.373.pdf chou, w. y. s., prestin, a., & kunath, s. (2014). obesity in social media: a mixed methods analysis. translational behavioral medicine, 4(3), 314-323. https://doi.org/10.1007/s13142014-0256-1 curiskis, s. a., drake, b., osborn, t. r., & kennedy, p. j. (2020). an evaluation of document clustering and topic modelling in two online social networks: twitter and reddit. information processing & management, 57(2), 102034. https://doi.org/10.1016/j.ipm.2019.04.002 de brún, a., mccarthy, m., mckenzie, k., & mcgloin, a. (2014). weight stigma and narrative resistance evident in online discussions of obesity. appetite, 72, 73-81. https://doi.org/10.1016/j.appet.2013.09.022 delgado, r., & stefancic, j. (1991). images of the outsider in american law and culture: can free expression remedy systemic social ills. cornell law review, 77, 1258. https://doi.org/10.4324/9780429037627-2 delgado, r., & stefancic, j. (2014). hate speech in cyberspace. wake forest law review, 49, 319. https://scholarship.law.ua.edu/fac_articles/560 durrheim, k., okuyan, m., twali, m. s., garcía-sánchez, e., pereira, a., portice, j. s., gur, t., weiner-blotmer, o. & keil, t. f. (2018). how racism discourse can mobilize right-wing populism: the construction of identity and alliance in reactions to ukip's brexit “breaking point” campaign. journal of community & applied social psychology, 28(6), 385405. https://doi.org/10.1002/casp.2347 eisenstein, j., o'connor, b., smith, n. a., & xing, e. p. (2014). diffusion of lexical change in social media. plos one, 9(11), e113114. https://doi.org/10.1371/journal.pone.0113114 erjavec, k., & kovačič, m. p. (2012). “you don't understand, this is a new war!” analysis of hate speech in news web sites' comments. mass communication and society, 15(6), 899920. https://doi.org/10.1080/15205436.2011.619679 https://aclanthology.org/2020.acl-main.373.pdf https://doi.org/10.1007/s13142-014-0256-1 https://doi.org/10.1007/s13142-014-0256-1 https://doi.org/10.1016/j.appet.2013.09.022 https://doi.org/10.4324/9780429037627-2 https://scholarship.law.ua.edu/fac_articles/560 https://doi.org/10.1002/casp.2347 https://doi.org/10.1371/journal.pone.0113114 https://doi.org/10.1080/15205436.2011.619679 australasian journal of information systems wanniarachchi et al. 2023, vol 27, research article hate speech pattern in social media 24 fredrickson, b. l., & roberts, t. a. (1997). objectification theory: toward understanding women's lived experiences and mental health risks. psychology of women quarterly, 21(2), 173-206. https://doi.org/10.1111/j.1471-6402.1997.tb00108.x gallacher, j. d., heerdink, m. w., & hewstone, m. (2021). online engagement between opposing political protest groups via social media is linked to physical violence of offline encounters. social media+ society, 7(1), https://doi.org/10.1177/2056305120984445 himmelstein, m. s., puhl, r. m., & quinn, d. m. (2017). intersectionality: an understudied framework for addressing weight stigma. american journal of preventive medicine, 53(4), 421-431. https://doi.org/10.1016/j.amepre.2017.04.003 holmberg, c., berg, c., hillman, t., lissner, l., & chaplin, j. e. (2018). self-presentation in digital media among adolescent patients with obesity: striving for integrity, riskreduction, and social recognition. digital health, 4. https://doi.org/10.1177/2055207618807603 hussin, m., frazier, s., & thompson, j. k. (2011). fat stigmatization on youtube: a content analysis. body image, 8(1), 90-92. https://doi.org/10.1016/j.bodyim.2010.10.003 irani, a., hendry, manongga, d. h. f., & chen, r.-c. (2020). mining public opinion on radicalism in social media via sentiment analysis. international journal of innovative computing, information and control, 16(5). 1787-1800. https://doi.org/10.24507/ijicic.16.05.1787 izydorczyk, b., walenista, w., kamionka, a., lizińczyk, s., & ptak, m. (2021). connections between perceived social support and the body image in the group of women with diastasis recti abdominis. frontiers in psychology, 12. 3182. https://doi.org/10.3389/fpsyg.2021.707775 jeon, y. a., hale, b., knackmuhs, e., & mackert, m. (2018). weight stigma goes viral on the internet: systematic assessment of youtube comments attacking overweight men and women. interactive journal of medical research, 7(1), e6. https://doi.org/10.2196/ijmr.9182 kemp, s. (2021). digital 2021: global overview report. retrieved from https://datareportal.com/reports/digital-2021-global-overview-report. accessed 30 august 2022. kent, e. e., prestin, a., gaysynsky, a., galica, k., rinker, r., graff, k., & chou, w.-y. s. (2016). “obesity is the new major cause of cancer”: connections between obesity and cancer on facebook and twitter. journal of cancer education, 31(3), 453-459. https://doi.org/10.1007/s13187-015-0824-1 kocoń, j., figas, a., gruza, m., puchalska, d., kajdanowicz, t., & kazienko, p. (2021). offensive, aggressive, and hate speech analysis: from data-centric to human-centered approach. information processing & management, 58(5), 102643. https://doi.org/10.1016/j.ipm.2021.102643 kosara, r. (2016). presentation-oriented visualization techniques. ieee computer graphics and applications, 36(1), 80-85. https://doi.org/10.1109/mcg.2016.2 larson, s. r. (2021). the rhetoricity of fat stigma: mental disability, pain, and anorexia nervosa. rhetoric society quarterly, 51(5), 392-406. https://doi.org/10.1080/02773945.2021.1972131 https://doi.org/10.1111/j.1471-6402.1997.tb00108.x https://doi.org/10.1177/2056305120984445 https://doi.org/10.1016/j.amepre.2017.04.003 https://doi.org/10.1177/2055207618807603 https://doi.org/10.1016/j.bodyim.2010.10.003 https://doi.org/10.3389/fpsyg.2021.707775 https://doi.org/10.2196/ijmr.9182 https://datareportal.com/reports/digital-2021-global-overview-report https://doi.org/10.1007/s13187-015-0824-1 https://doi.org/10.1016/j.ipm.2021.102643 https://doi.org/10.1109/mcg.2016.2 australasian journal of information systems wanniarachchi et al. 2023, vol 27, research article hate speech pattern in social media 25 lazarus, j. v., kakalou, c., palayew, a., karamanidou, c., maramis, c., natsiavas, p., picchio, c.a., villota-rivas, m. zelber-sagi, s. & carrieri, p. (2021). a twitter discourse analysis of negative feelings and stigma related to nafld, nash, and obesity. liver international, 41(10), 2295-2307. https://doi.org/10.1111/liv.14969 lipizzi, c., iandoli, l., & marquez, j. e. r. (2015). extracting and evaluating conversational patterns in social media: a socio-semantic analysis of customers’ reactions to the launch of new products using twitter streams. international journal of information management, 35(4), 490-503. https://doi.org/10.1016/j.ijinfomgt.2015.04.001 liu, b. (2012). sentiment analysis and opinion mining (vol. 5). morgan & claypool publishing, springer chan. https://doi.org/10.1007/978-3-031-02145-9 lydecker, j. a., cotter, e. w., palmberg, a. a., simpson, c., kwitowski, m., white, k., & mazzeo, s. e. (2016). does this tweet make me look fat? a content analysis of weight stigma on twitter. eating and weight disorders-studies on anorexia, bulimia and obesity, 21(2), 229-235. https://doi.org/10.1007/s40519-016-0272-x maftei, a., & merlici, i.-a. (2022). am i thin enough? social media use and the ideal body stereotype: the mediating role of perceived socio-cultural pressure and the moderating role of cognitive fusion. current psychology, 1-14. https://doi.org/10.1007/s12144-02202938-x manning, c. d., raghavan, p., & schütze, h. (2008). an introduction to information retrieval. cambridge, england: cambridge university press. manning, c. d., surdeanu, m., bauer, j., finkel, j. r., bethard, s., & mcclosky, d. (2014). the stanford corenlp natural language processing toolkit. paper presented at the proceedings of 52nd annual meeting of the association for computational linguistics: system demonstrations. 55–60, baltimore, maryland. association for computational linguistics. https://aclanthology.org/p14-5010 martins, r., gomes, m., almeida, j. j., novais, p., & henriques, p. (2018). hate speech classification in social media using emotional analysis. paper presented at the 2018 7th brazilian conference on intelligent systems (bracis). 22-25 october 2018, sao paulo, brazil. 61-66. https://doi.org/10.1109/bracis.2018.00019 matamoros-fernández, a., & farkas, j. (2021). racism, hate speech, and social media: a systematic review and critique. television & new media, 22(2), 205-224. https://doi.org/10.1177/1527476420982230 mathew, b., illendula, a., saha, p., sarkar, s., goyal, p., & mukherjee, a. (2020). hate begets hate: a temporal study of hate speech. proceedings of the acm on human-computer interaction, 4(cscw2), 92. 1-24. https://doi.org/10.1145/3415163 mccomb, s. e., & mills, j. s. (2022). the effect of physical appearance perfectionism and social comparison to thin-, slim-thick-, and fit-ideal instagram imagery on young women’s body image. body image, 40, 165-175. https://doi.org/10.1016/j.bodyim.2021.12.003 microsoft corporation. (2016). microsoft excel (version 2209) [computer software]. retrieved from https://www.microsoft.com/en-nz/microsoft-365/excel. accessed 01 january 2022. https://doi.org/10.1111/liv.14969 https://doi.org/10.1016/j.ijinfomgt.2015.04.001 https://doi.org/10.1007/978-3-031-02145-9 https://aclanthology.org/p14-5010 https://doi.org/10.1109/bracis.2018.00019 https://doi.org/10.1177/1527476420982230 https://doi.org/10.1145/3415163 https://doi.org/10.1016/j.bodyim.2021.12.003 australasian journal of information systems wanniarachchi et al. 2023, vol 27, research article hate speech pattern in social media 26 miranda, s., berente, n., seidel, s., safadi, h., & burton-jones, a. (2022). editor's comments: computationally intensive theory construction: a primer for authors and reviewers. management information systems quarterly, 46(2), iii-xviii. mohammad, s. m., & turney, p. d. (2013). nrc emotion lexicon. national research council canada, technical report (nparc number: 21270984), 243p. https://doi.org/10.4224/21270984. accessed 31 may 2022. mondal, m., silva, l. a., & benevenuto, f. (2017). a measurement study of hate speech in social media. paper presented at the proceedings of the 28th acm conference on hypertext and social media. 4-7 july 1017, prague, czech republic. 85-94. https://doi.org/10.1145/3078714.3078723 mossie, z., & wang, j.-h. (2020). vulnerable community identification using hate speech detection on social media. information processing & management, 57(3), 102087. https://doi.org/10.1016/j.ipm.2019.102087 murakami, j. m., & latner, j. d. (2015). weight acceptance versus body dissatisfaction: effects on stigma, perceived self-esteem, and perceived psychopathology. eating behaviors, 19, 163-167. https://doi.org/10.1016/j.eatbeh.2015.09.010 ophir, y., walter, d., & marchant, e. r. (2020). a collaborative way of knowing: bridging computational communication research and grounded theory ethnography. journal of communication, 70(3), 447-472. https://doi.org/10.1093/joc/jqaa013 pronoza, e., panicheva, p., koltsova, o., & rosso, p. (2021). detecting ethnicity-targeted hate speech in russian social media texts. information processing & management, 58(6), 102674. https://doi.org/10.1016/j.ipm.2021.102674 puhl, r. m., andreyeva, t., & brownell, k. d. (2008). perceptions of weight discrimination: prevalence and comparison to race and gender discrimination in america. international journal of obesity, 32(6), 992. https://doi.org/10.1038/ijo.2008.22 rawal, k., patel, t. p., purohit, k. m., israni, k., kataria, v., bhatt, h., & gupta, s. (2020). influence of obese phenotype on metabolic profile, inflammatory mediators and stemness of hadsc in adipose tissue. clinical nutrition, 39(12), 3829-3835. https://doi.org/10.1016/j.clnu.2020.02.032 reddit. (2020). reddit api documentation. retrieved from https://www.reddit.com/dev/api/. accessed 01 jan 2021. řehůřek, r., & sojka, p. (2011). gensim—statistical semantics in python. poster presented at euroscipy 2011, paris, 25.–28. 8. 2011. https://initiative.eudml.org/news/presentationeuroscipy-2011-paris. accessed 01 jan 2022. röder, m., both, a., & hinneburg, a. (2015). exploring the space of topic coherence measures. paper presented at the proceedings of the eighth acm international conference on web search and data mining, february 2 -6, 2015, shanghai, china. 399-408. https://doi.org/10.1145/2684822.2685324 rodríguez, a., argueta, c., & chen, y.-l. (2019). automatic detection of hate speech on facebook using sentiment and emotion analysis. paper presented at the 2019 international conference on artificial intelligence in information and communication (icaiic). 11-13 february 2019, okinawa, japan. https://ieeexplore.ieee.org/document/8669073 https://doi.org/10.4224/21270984 https://doi.org/10.1016/j.ipm.2019.102087 https://doi.org/10.1016/j.eatbeh.2015.09.010 https://doi.org/10.1093/joc/jqaa013 https://doi.org/10.1016/j.ipm.2021.102674 https://doi.org/10.1038/ijo.2008.22 https://doi.org/10.1016/j.clnu.2020.02.032 https://www.reddit.com/dev/api/ https://initiative.eudml.org/news/presentation-euroscipy-2011-paris https://initiative.eudml.org/news/presentation-euroscipy-2011-paris https://ieeexplore.ieee.org/document/8669073 australasian journal of information systems wanniarachchi et al. 2023, vol 27, research article hate speech pattern in social media 27 salminen, j., almerekhi, h., milenković, m., jung, s.-g., an, j., kwak, h., & jansen, b. j. (2018). anatomy of online hate: developing a taxonomy and machine learning models for identifying and classifying hate in online news media. paper presented at the twelfth international aaai conference on web and social media. 330-339. 25 -28 june 2018. palo alto, united states. https://pennstate.pure.elsevier.com/en/publications/anatomy-of-online-hatedeveloping-a-taxonomy-and-machine-learning satapathy, r., guerreiro, c., chaturvedi, i., & cambria, e. (2017). phonetic-based microtext normalization for twitter sentiment analysis. paper presented at the 2017 ieee international conference on data mining workshops (icdmw). 18-21 november 2017. new orleans, la, usa https://doi.org/10.1109/icdmw.2017.59 semeraro, a., vilella, s., & ruffo, g. (2021). pyplutchik: visualising and comparing emotionannotated corpora. plos one, 16(9), e0256503. https://doi.org/10.1371/journal.pone.0256503 shibly, f., sharma, u., & naleer, h. (2021). classifying and measuring hate speech in twitter using topic classifier of sentiment analysis. paper presented at the international conference on innovative computing and communications. advances in intelligent systems and computing, vol 1165. springer, singapore. 671-678. https://doi.org/10.1007/978-981-155113-0_54 shirazi, f. (2013). social media and the social movements in the middle east and north africa: a critical discourse analysis. information technology & people. 26(1). 28-49. https://doi.org/10.1108/09593841311307123 so, j., prestin, a., lee, l., wang, y., yen, j., & chou, w.-y. s. (2016). what do people like to “share” about obesity? a content analysis of frequent retweets about obesity on twitter. health communication, 31(2), 193-206. https://doi.org/10.1080/10410236.2014.940675 ștefăniță, o., & buf, d.-m. (2021). hate speech in social media and its effects on the lgbt community: a review of the current research. romanian journal of communication and public relations, 23(1), 47-55. https://doi.org/10.21018/rjcpr.2021.1.322 stieglitz, s., & dang-xuan, l. (2013). emotions and information diffusion in social media— sentiment of microblogs and sharing behavior. journal of management information systems, 29(4), 217-248. https://doi.org/10.2753/mis0742-1222290408 szepietowska, b., polonsky, b., sherazi, s., biton, y., kutyifa, v., mcnitt, s., . . . zareba, w. (2016). effect of obesity on the effectiveness of cardiac resynchronization to reduce the risk of first and recurrent ventricular tachyarrhythmia events. cardiovascular diabetology, 15(1), 1-8. https://doi.org/10.1186/s12933-016-0401-x tang-péronard, j., & heitmann, b. (2008). stigmatization of obese children and adolescents, the importance of gender. obesity reviews, 9(6), 522-534. https://doi.org/10.1111/j.1467789x.2008.00509.x thompson, l., rickett, b., & day, k. (2018). feminist relational discourse analysis: putting the personal in the political in feminist research. qualitative research in psychology, 15(1), 93115. https://doi.org/10.1080/14780887.2017.1393586 https://pennstate.pure.elsevier.com/en/publications/anatomy-of-online-hate-developing-a-taxonomy-and-machine-learning https://pennstate.pure.elsevier.com/en/publications/anatomy-of-online-hate-developing-a-taxonomy-and-machine-learning https://doi.org/10.1109/icdmw.2017.59 https://doi.org/10.1371/journal.pone.0256503 https://doi.org/10.1108/09593841311307123 https://doi.org/10.1080/10410236.2014.940675 https://doi.org/10.21018/rjcpr.2021.1.322 https://doi.org/10.2753/mis0742-1222290408 https://doi.org/10.1186/s12933-016-0401-x https://doi.org/10.1111/j.1467-789x.2008.00509.x https://doi.org/10.1111/j.1467-789x.2008.00509.x https://doi.org/10.1080/14780887.2017.1393586 australasian journal of information systems wanniarachchi et al. 2023, vol 27, research article hate speech pattern in social media 28 tomiyama, a. j. (2014). weight stigma is stressful. a review of evidence for the cyclic obesity/weight-based stigma model. appetite, 82, 8-15. https://doi.org/10.1016/j.appet.2014.06.108 törnberg, a., & törnberg, p. (2016). muslims in social media discourse: combining topic modeling and critical discourse analysis. discourse, context & media, 13, 132-142. https://doi.org/10.1016/j.dcm.2016.04.003 tucey, c. b. (2010). online vs. face-to-face deliberation on the global warming and stem cell issues. paper presented at the western political science association 2010 annual meeting paper. https://ssrn.com/abstract=1580573 twitter. (2019). twitter api. retrieved from https://developer.twitter.com/. accessed 01 jan 2021. tylka, t. l., & hill, m. s. (2004). objectification theory as it relates to disordered eating among college women. sex roles, 51(11-12), 719-730. https://doi.org/10.1007/s11199-004-0721-2 veldhuis, j., konijn, e. a., & seidell, j. c. (2014). negotiated media effects. peer feedback modifies effects of media’s thin-body ideal on adolescent girls. appetite, 73, 172-182. https://doi.org/10.1016/j.appet.2013.10.023 vigna, f. d., cimino, a., dell’orletta, f., petrocchi, m., & tesconi, m. (2017). hate me, hate me not: hate speech detection on facebook. paper presented at the proceedings of the first italian conference on cybersecurity (itasec17). venice, italy. 8695. https://ceur-ws.org/vol1816/paper-09.pdf wallach, h. m. (2006). topic modeling: beyond bag-of-words. paper presented at the proceedings of the 23rd international conference on machine learning. pittsburgh, pennsylvania. 2529 july 2006. 977-984. https://doi.org/10.1145/1143844.1143967 wanniarachchi, v. u., mathrani, a., susnjak, t., & scogings, c. (2019). gendered objectification of weight stigma in social media: a mixed method analysis. paper presented at the australasian conference on information systems, perth, western australia. 9-11 december 2019 https://acis2019.io/pdfs/acis2019_paperfin_077.pdf wanniarachchi, v. u., mathrani, a., susnjak, t., & scogings, c. (2022). methodological aspects in study of fat stigma in social media contexts: a systematic literature review. applied sciences, 12(10), 5045. https://doi.org/10.3390/app12105045 who. (2022). data and statistics. retrieved from https://www.euro.who.int/en/healthtopics/noncommunicable-diseases/obesity/data-and-statistics. accessed 01 jan 2022. yeruva, v. k., junaid, s., & lee, y. (2019). contextual word embeddings and topic modeling in healthy dieting and obesity. journal of healthcare informatics research, 3(2), 159-183. https://doi.org/10.1007/s41666-019-00052-5 zhong, b. (2020). social consequences of internet civilization. computers in human behavior, 107, 106308. https://doi.org/10.1016/j.chb.2020.106308 https://doi.org/10.1016/j.dcm.2016.04.003 https://ssrn.com/abstract=1580573 https://doi.org/10.1007/s11199-004-0721-2 https://doi.org/10.1016/j.appet.2013.10.023 https://ceur-ws.org/vol-1816/paper-09.pdf https://ceur-ws.org/vol-1816/paper-09.pdf https://doi.org/10.1145/1143844.1143967 https://acis2019.io/pdfs/acis2019_paperfin_077.pdf https://doi.org/10.3390/app12105045 https://doi.org/10.1007/s41666-019-00052-5 https://doi.org/10.1016/j.chb.2020.106308 australasian journal of information systems wanniarachchi et al. 2023, vol 27, research article hate speech pattern in social media 29 copyright: © 2023 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v27i0.3929 http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 research objectives 3 literature review 3.1 detecting hate content in social media 3.2 detecting obesity content in social media 4 methodological framework 4.1 data collection 4.2 data pre-processing 4.2.1 duplicates removal 4.2.2 stop words and noise removal 4.2.3 word lemmatization 4.3 data analysis 4.4 pattern identification 5 fat stigma investigation 5.1 data collection and data pre-processing 5.2 data analysis: overall text analysis of fatness 5.2.1 sentiment analysis 5.2.2 emotion analysis 5.2.3 sentiment labelling 5.3 pattern identification of fat stigma discussions 5.3.1 topic modelling 5.3.2 word frequencies 5.3.3 discourse analysis 6 discussion 6.1 application of the framework 6.2 emergent fat-stigma themes 7 conclusions paper title (paper title style) australasian journal of information systems linger & hasan 2020, vol 24, selected papers from the is foundations conference making is relevant in a connected world 1 making is relevant in a connected world: revisiting the intellectual structures framework henry linger centre for organisational and social informatics (cosi) faculty of information technology monash university helen hasan australian health services research institute (ahsri), university of wollongong hasan@uow.edu.au abstract the exponential growth of the internet since the mid-1990s has greatly expanded the capacity of people everywhere to interconnect and engage through digital technologies. as a complex adaptive system of systems, the internet has extended the range and complexity of phenomena of interest to information systems (is) scholars. this is both an exciting opportunity and a challenge which we explore in this paper by revisiting the intellectual structures framework (hirshheim et al. 1996) which attempted to make sense of the fragmented adhocracy of is, before the expansion and penetration of the internet. we suggest that the is adhocracy, with its multi-disciplinary and systems-oriented nature, gives is researchers the requisite variety to contend with the increasingly diverse digital ecologies of is-enabled human activities that have emerged in the ensuing two decades. based on relevant research over these two decades we present a revised framework that (1) reflects the complexities of contemporary is phenomena and (2) can act as an instrument for analysing such phenomena across a spectrum of human activities. we justify the form and content of the revised intellectual structures framework, providing examples of its application in is research using appropriate research methods and techniques. we argue that our revisions to the original framework provides individuals, organisations, and societies with a conceptual lens that is necessary to better address the challenges and opportunities posed by the complexities of contemporary digital ecologies. keywords: intellectual structures framework; relevance of is; complexity; is adhocracy 1 introduction “[the internet is] the first thing that humanity has built that humanity doesn't understand, the largest experiment in anarchy that we have ever had.” eric schmidt, former ceo of google internet usage statistics1 show a major tipping point in the mid-1990s that heralded the internet era2. over the ensuing two decades, the expansion of the world-wide web (the web) has resulted in an extremely complex global environment of hyper-connections and mutual dependencies (phister 2010). we have seen the rapid evolution of the static web 1.0 into the interactive web 2.0 (o'reilly 2007) and then to the internet of things and the explosion of 1 https://en.wikipedia.org/wiki/global_internet_usage 2 http://www.internetsociety.org/ australasian journal of information systems linger & hasan 2020, vol 24, selected papers from the is foundations conference making is relevant in a connected world 2 social media platforms (yoo 2010; laghari and niazi 2016). the combination of diverse digital technologies and the internet has created ecologies of sociotechnical networks which have significant impact on human agency and have enabled a revolution in the range of activities in organisations (huang and sun 2016) and in society (castells 2015; rosenberg 2013). moreover, digital ecologies have profound implications for information systems (is) research and practice (e.g. yoo et al 2010; baskerville and priess-heje 2002) and, we believe, an urgent need to explore how to make sense of information systems that have emerged within these diverse digital ecologies (ulieru 2010). we propose that the is body of knowledge, accumulated over several decades, be leveraged to build evolutionary theories and methods that can support investigations of emergent dynamic phenomena from a multilevel perspective (mckelvy et al. 2015). a relevant dictionary definition of the word ecology is “the set of relationships existing between any complex system and its surroundings or environment.”3 our use of the term digital ecology is based on this definition and follows the recommendation of kleineberg and boguná (2015) who apply the term when taking an ecological perspective on the extremely complex interactions between multiple online social networks and their global environments in web 2.0. the emergence of complex digital ecologies has prompted the need to re-examine the nature of the is discipline itself (lee et al 2015; walsham 2012; winter and butler 2011). this debate is driven by the need to accommodate the new phenomena emerging from the complexity inherent in these digital ecologies. our position in this debate is that is is a “fragmented adhocracy” because of its diversity, plurality, multi-disciplinarity and systems-orientation (benbasat and weber 1996; benbasat and zmud 2003). this argument was originally proposed by banville and landry (1989) and theorised by hirshheim et al (1996) through social action theories. hirshheim et al.’s (1996) contribution is a theoretically grounded framework, the original intellectual structures framework (figure 1), which identified the variety of intellectual positions of is. the framework identifies the domains that is impacts and the orientation of changes in those domains. however, the original framework did not anticipate, and does not explicitly account for, the phenomena emerging from the complex sociotechnical digital ecologies, and in particular the wicked problems and non-equilibrium dynamic that characterises our digital worlds. although the framework is not without its critics (see for example walsham 1996; mathiassen 1996) we believe this framework has the theoretical rigour to be expanded to address the challenges posed for contemporary is by digital ecologies. 3 http://www.dictionary.com/browse/ecology australasian journal of information systems linger & hasan 2020, vol 24, selected papers from the is foundations conference making is relevant in a connected world 3 domains orientations control sense-making argumentation instrumental strategic communicative discursive technology information technology systems language formalized symbol manipulation systems manipulative communication systems systems for consensual communication systems for rational argumentation organisation mechanistic social systems (ritualized tasks) political systems (decision making hierarchy and control) cultural and social systems (negotiated meanings and practices) systems for institutional democracy (checks and balances) figure 1. is object system classes in the original intellectual structures framework (adapted from hirschheim et al. 1996). note the three empty cells in the technology domain in this paper we examine how the complexity of digital ecologies has expanded the traditional focus of is and how this has impacted on the nature of the is discipline. our objective is to articulate an is construct that has the capacity and requisite variety to accommodate diverse digital ecologies and directly address the inherent complexity of these ecologies. in pursuit of this objective we reinterpret and extend (hirshheim et al.’s (1996) original intellectual structures framework. the paper draws on evidence for the increasing complexity of the is environment and an historical assessment of the intellectual structures of is. we argue for the extension of the technology domain (see figure 1) to address developments over the last decades but particularly the impact of social media and the web. we also argue that the language and organisation domains of the framework need to be understood in terms of the dominant role of knowledge and the complex internal and external environments. our interpretation of the domains of change, in the context of digital ecologies, results in six domains. we then show how each of these six domains has a range of different is applications across the spectrum from ordered to complex phenomena. we demonstrate the implication and utility of the revised framework providing exemplar is projects that can best be explained by integrating aspects from several domains related to the phenomenon. we conclude with a discussion on the implications of the new framework for is theory and research methodology. 2 background as is widely understood, the internet is a global technical structure of interconnected computer networks using standardized communication protocols. the technical complexity of the internet as an it artefact was recognised in the 1990s (thompson et al 1997) and further established in the 2000s (faratin et al. 2008). our interest in the internet is not as an it artefact but as the infrastructure underpinning a network (i.e., the web) of is artefacts, which include technology artefacts, information artefacts and social artefacts (lee et al 2015). indeed, “the web might be considered the world’s largest information system providing a convenient framework for publishing and finding information on the internet” (stair and reynolds 2013 p 26). australasian journal of information systems linger & hasan 2020, vol 24, selected papers from the is foundations conference making is relevant in a connected world 4 to support our claim of increasing complexity of is in the internet era, we use four elements that together indicate that a phenomenon is complex (hasan et al 2016). these are listed in the first column of table 1 with well-known exemplar characteristics related to the web in column 2. the network of networks that is the web provides a dynamic backbone for the complex interconnected world of systems ranging from micro-businesses, civil society organisations to multinational companies and interwoven with the many-to-many social connections of individuals everywhere. this digital environment exhibits traits of a complex adaptive system (phister 2010). the internet has changed space-time in ways that impact the affordances of information and communication technology (ict), the information landscape and the boundaries that distinguish personal, business and social enterprise (see e.g. schalow et al. 2013). these changes coincided with a focus on knowledge, as a driver of change through innovation (boland and tenkasi 1995; nonaka 1994), and a growing understanding that complexity defines phenomena that are of interest to is in the form of wicked problems (newell et al 2000; snowden 2002). complexity indicator defining web characteristics multiple interacting elements: (adaptive systems with interdependent elements and feedback loops) by definition the web is an evolving network of interdependent networks with millions of interconnected nodes distributed control (with challenges of synthesizing the agendas of multiple stakeholders) tim berners-lee, founded and directs the world wide consortium (w3c) for the technical development of the web. he also founded the web foundation to ensure that the web serves humanity without overall control by governments or other agencies. unpredictable dynamics and emergence: with the unpredictable effects of small changes in initial conditions. the world of computes has defied predictions such as the famous 1943 quote of ibm chairman thomas watson that “… there is a world market for maybe five computers.” the web is an emergent entity as are its components and applications for business and social use. these in turn have generated new e-business models and social movements. contradictory requirements which prohibit the setting of precise goals and measurable outcomes in the planning process the web has highlighted the tensions between its benefits and threats; between its inherent free and open nature and its inherent dangers of security and identity fraud. it has blurred the boundaries between the private and the public and redefine informational roles such as user, stakeholder or creator (wagner et al. 2014). table 1 assessing the web as complex against four indicators (adapted from hasan et al 2016) we suggest that most is research deals with complicated systems that, while remarkable and may have many interesting facets, can be understood with methodical reductionist examination. however, we recognise that in the digital ecologies of the current is landscape, there are problems of concern to is scholars that are unpredictable, unstable and emergent. many of these are complex socio-technical systems “comprised of populations of interacting entities where the overall system behaviour is not predefined but rather emerges through the interactions of its entities” (kim and kaplan 2006 p37). while there are still issues worth investigating that are complicated but not complex, there is growing interest by is researchers australasian journal of information systems linger & hasan 2020, vol 24, selected papers from the is foundations conference making is relevant in a connected world 5 in accommodating complexity, where a non-reductionist approach may be adopted. this interest in complexity is evident in an emerging body of literature on is and is development as complex adaptive systems (benbya and kelvey 2006, grover 2012, kautz 2012, vigden and wang 2006). the web has effectively expanded the purview of is to address the broad and diverse constituents of society that includes individual, collective (social and/or organisational) and societal levels, and most importantly the interactions between these different levels (cecezkecmanovic and jerram 2002). the web has driven the development of a range of mobile, ubiquitous digital technologies whose success is grounded in access to (infinite) information, without concern for place or time (vodanovich et al 2010). such developments have also influenced the design and form of organisations and their underlying organising principles, highlighting their need to be flexible, adaptable and innovative in order to meet the constant and rapid changes in their environments, both internal and external (besson and rowe 2012). concurrently, the internet, and the technologies it has spawned, creates affordances that have driven the development of what is collectively termed social media that empowers individuals and collectives, both outside and inside organisational boundaries. for example, citizens now have the ability to rapidly mobilise and advocate against existing political power structures (magro 2012) in ways that were not anticipated when the internet was conceived (rosenberg 2013). 3 revisiting the intellectual structures of is among the various debates about the nature of the is and is development (isd) in the mid1990s, the intellectual structures framework (hirschheim et al 1996) depicted in figure 1, provided a theoretical basis for, and insight into, the increased complexity and diversity to come. rather than impose a unifying conceptual structure over the field such as sidorova et al. (2008), these authors developed a ‘federated’ is framework, based on social action theory. the framework is expressed as a matrix with one axis being domains of change, the areas that are changed by is, and the other axis being the orientations that signify the purpose of the change brought about by is. there are three fundamental orientations: control, sense-making (communicative) and argumentation (discursive); but a distinction is made between control over objects (instrumental) and humans (strategic). this distinction is important as instrumental control treats people as physical objects while the latter treats them as intelligent agents. the domains of change are technology (as artefacts), language (all forms of communication) and organisation. the cells of the framework represent object systems that, in an abstract way, show the diversity of is in terms of their content and purpose as it was manifest at the time. critiques of the framework at the time questioned (1) whether the categorizations in the framework were representative of systems in the real world, or intellectual constructs that can usefully be compared to the real world (walsham 1996), and (2) the limitations of the framework’s focus on institution-based notion of isd (matheessen 1996). a more recent examination of the intellectual core of is does not address the notion of is as a fragmented adhocracy, but rather argues for a unified view of the discipline (sidorova et al. 2008) that establishes the relationship of the it artefact to activities at the individual, group, organisation and market. but this view makes no attempt to either explicitly or implicitly address the diversity and complexity that characterises digital ecologies. we suggest that shifting the emphasis from an it artefact to the is artefact (lee et al. 2015), as mentioned previously, australasian journal of information systems linger & hasan 2020, vol 24, selected papers from the is foundations conference making is relevant in a connected world 6 highlights the need to address complexity. we believe this shift in emphasis is facilitated by the intellectual structures framework. the typology of the intellectual structures framework is reminiscent of the modification of giddens’ original model of structuration theory, developed by orlikowski (1992) amongst others, to have a profound influence on is thinking (jones and karsten 2008). in the is adaptation of structuration theory, technology-in-practice is both the medium and the outcome of human agency in mediation with facilities (such as technical artefacts), norms (organisations, communities and their variable cultures), and interpretive schemes (such as language/communications and all that is mediated by it). more recently, is research has explored the entanglement of the social and material that compliments and extend structuration theory (cecez-kecmanovic et al. 2014; scott and orlikowski 2014). this interest in sociomateriality affords an explicitly theoretical perspective that encompasses complexity, connectivity and meaning that is congruent with digital ecologies. however, we chose the intellectual structures framework because it reinforces the wellaccepted and broad principles of structuration and social action as its theoretical base. this theoretical grounding provides a robust basis to amend these principles to meet the imperatives of the diversity and complexity of the digital age, particularly the unprecedented changes that have occurred following the global spread of the internet and the transforming advances of the web. 3.1 reinterpreting the technology domain: tools to exploit the digital ecology we have interpreted the technology domain as tools rather than technology (kane and fichman 2009) because “… the emergence of the internet has made the digital tools necessary for innovation more affordable to a broad spectrum of previously excluded economic and innovative activity”. (yoo et al. 2010 p 726). in the intellectual structures framework (figure 1), the technology domain is only populated by object classes in the control orientation reflecting the organisational technologies of the time. historically, is discourse has been dominated by a perspective of technology that was restricted to organisational boundaries and where enterprise systems addressed the efficiencies to operations (orlikowski 1992, sidorova et al 2008). enterprise systems have become almost ubiquitous and are now part of the basic organisational infrastructure that includes not only technical elements but also organisational data, information, operating procedures and support (panetto and cecil 2013). this infrastructure provides a stable foundation on which new is applications are design and built through traditional isd. in contrast, the global infrastructure afforded by the internet establishes a digital ecology on which a diverse set of is and ict tools are continually emerging (e.g. social media) and are used to support an increasingly diverse range of human activities. as well as these significant changes in technology, is also needs to accommodate significant shifts in the business environments, which is itself driven by changing technology. the digital ecology affords radical new methods of communication, interaction, dissemination of knowledge and organising principles that were not possible previously (hirschheim and klein 2012). as a consequence, is is now concerned with the evolution and appropriation of these tools, creating new activities that the tools afford and most importantly the promulgation of the tools from the individual, to groups, to organisations through to society (klein and hirschheim 2008). the significance of this development is that is is not constrained by organisational boundaries. australasian journal of information systems linger & hasan 2020, vol 24, selected papers from the is foundations conference making is relevant in a connected world 7 digital ecology raises a different set of issues, removing constraints on tasks or activities while supporting experiential learning, collaboration and mediating competition (hirschheim and klein 2012). the challenge now is how to exploit these tools and creatively leverage the affordances offered by each new generation of digital technologies while at the same time to understand the emergent impacts that digital tools, methods and applications will have on the way work is done and how we live in society. this has led to a plethora of new research streams within is from new technical affordances enabling globalization, ubiquitous computing, crowd-sourcing, mobile applications, the cloud, social media and more. new settings for is outside organisations in communities and social networks are blurring the traditional organisational boundaries (walsham 2012; march and niederman 2012). however, this increased scope for is has intensified the efforts of some scholars to unify is, to define a bounded ‘intellectual core’ of the discipline and to capture new problems within existing frames in order to deal with complexity (grover 2012). however, there is no general consensus on this endeavour (sidorova et al. 2008). this highlights the paradox reflected in the question: “why is discussion of crisis in the is field so prevalent when the significance and usage of is has grown dramatically?” (westrup 2012 p 106). our reinterpretation of the intellectual structure framework deals with this paradox. as argued above, the digital age has brought generations of digital technologies to organisational, community and societal settings. business, government and community organisations, other institutions, as well as individuals and civil society in general, continue to exploit and explore the digital ecology. we propose that changed realities and radical new technological affordances support object classes to populate all orientation of the technology domain of the intellectual structures framework as shown in figure 2. domains orientations control sense-making argumentation instrumental strategic communicative discursive digital tools it systems, technology infrastructures information infrastructure to support: • decision making, • knowledge management, • business intelligence activities and communities in social media advocacy forums, e-democracy, ethical intelligence (ai, machine learning and algorithms) figure 2. object classes for all orientations of the technology domain in a contemporary reinterpretation of the intellectual structures framework technologies. such as intranets, km systems, big data, analytics and data driven decision making’ provide object classes for strategic decision-makers (clarke 2016). the object classes proposed in the strategic control orientation of figure 2 give organisational actors direct access to diverse data and analytic tools extend their strategic control capability within the organisational boundary. more recently, applications associated with web 2.0 and social media are impacting the technology domain in the sense-making orientation. emergent phenomena developed in social settings, such as crowd sourcing (feller et al 2012) and wiki knowledge repositories australasian journal of information systems linger & hasan 2020, vol 24, selected papers from the is foundations conference making is relevant in a connected world 8 (pfaff and hasan 2012), are crossing the boundary into corporations further blurring the organisational boundaries of the technology domain. these social technologies are instrumental in the dynamic formation, evolution and transformation of social networks and are the basis of communities within, between and across organisations, institutions and civil society. similarly, concepts such as the wisdom of (and in) the crowd (surowiecki 2004) both within institutions, and between institutions and their constituencies, may be the basis of object classes in the argumentation orientation of the technology domain. more recently we have seen an emergence into the mainstream of artificial intelligence (ai) agents and machine learning. such technologies are often predicated on the ability to link and analyse multiple and diverse data sets to provide a coherent perspective on very complex societal phenomena (e.g. strelioff et al. 2013). a particularly pertinent aspect of such “intelligence” is the current debates around human centred and ethical ai (russell et al. 2015, d'aquin 2018, baum 2017). to reflect the shift of emphasis from the technical artefact to the digital ecology, we have renamed the domain as digital tools. an important consequence of the expanded interpretation of the technology domain as digital tools is that we need to consider the implications of the changes to the technology domain for the language and organisation domains, as well as exploring its broader societal impact. 3.2 language domain: incorporating knowledge and change in the original intellectual structures framework of figure 1 we see object classes of the language domain that reflect the ability of computer-based systems to not only crunch numbers and manage structured databases in the instrumental control orientation but also to manipulate free text, support communication, which at the time was predominantly email, and even allow some rational argumentation. as noted above, the 1990s saw the emergence of a focus on knowledge within organisational and is studies inspired by the seminal work of grant (1996. 2002), nonaka (1994) and others. the openness and expanding reach of the web as well as the new interfaces, functionality and affordances of digital technologies enabled anyone connected to the internet to access, generate and exchange content with the global community. many researchers in is felt that the term ‘information’ was no longer adequate to describe all that is was about and began to talk of systems which dealt with ‘knowledge’. this expanded purview of is challenged the existing understanding of the language domain. in the field of knowledge management (km), knowledge is viewed both as a thing (created, embodied and embedded) and as a flow (emergent and fluid) (orlikowski 2002, blackler 1995, hasan and crawford 2003, 2007). we recognise this distinction in the intellectual structure framework by transforming the language domain into two domains. as shown in figure 3, the information domain is concerned not only with information but extends to knowledge creation and deployment as well as the more recent emphasis on “data”. information in this sense can be construed as a ‘thing’ that can be mobilised for building (organisational) capability. such capability is necessary to make sense of complex digital ecologies and to act appropriately in response to this situational awareness. the other domain is concerned with interaction to emphasise the changing and fluid connections between individuals, entities and organisational constructs. such connections are manifested in the knowledge and information flows that drive change through enhanced communication and learning (senge 1990; 2014). the shift of emphasis from information to australasian journal of information systems linger & hasan 2020, vol 24, selected papers from the is foundations conference making is relevant in a connected world 9 knowledge also recognises the value of human and social capital (wasko and faraj 2005) and the role of learning, particularly to explore new ideas and builds the creative capacity for renewal through social learning and shared situation awareness (warne et al 2003; strati 2007). this is consistent with march’s (1991) approach to organisational learning as a synthesis of exploitation and exploration. in this sense, interaction is seen as ‘flow’, both as content that moves between actors and as an emergent and ever-changing link in a network of the ecosystem. digital ecologies provide the ability to engage across a far broader spectrum of activities, with more diverse actors, and span previously established boundaries, particularly the now porous organisational boundaries. moreover, the interaction domain broadens the purview of is to civil society with scholars such as winter and butler (2010) advocating the responsibility of is researcher to engage with other disciplines to tackle the grand challenges that face the global society. domains orientations control sense-making argumentation instrumental strategic communicative discursive interaction formalized symbol manipulation systems information / records management wikis, co-creation of knowledge e-democracy information communication systems /email intranets blogs, chat consensual communication systems for rational argumentation figure 3. object classes for orientations of information and interaction domains in an expanded reinterpretation of the language domain of the intellectual structures framework. 3.3 organisation domain: the impact of complexity in the original intellectual structures framework of figure 1 the object classes of the organisation domain summarised across the orientations as social, political cultural and democratic systems. in the internet era, ways of organising have diversified, business models have changed, new organising principles have emerged, and whole new industries have been created. organisations that were once understandable as systems are now seen as complex adaptive systems (grover 2012; kim and kaplan 2006). the increasing porous and fuzzy organisational boundaries, the organisation domain needs to be interpreted as a verb rather than a noun; a domain that is concerned with organising rather than an organisation. this is a significant shift of emphasis as the term ‘organisation’ has generally been used in is to indicated a physical, legal entity concerned with the production of goods and services. digital ecologies have created affordances for diverse organisational forms and constructs. the increasing complexity of the process of organising (crawford et al 2009) has led us to unpack the single organisation domain of the intellectual structures framework into three domains namely structure, action, and relationships as shown in figure 4. organisational studies have traditionally been concerned with how organisations structure themselves, how they manage relationships in their coordinating mechanisms and how the division of labour relates to the way work is carried out (see e.g. mintzberg 1979; jones 2010). the growing complexity of the organisational context implies that these concerns are australasian journal of information systems linger & hasan 2020, vol 24, selected papers from the is foundations conference making is relevant in a connected world 10 interdependent. never-the-less we suggest that their longstanding position of importance in the literature makes them ideal candidates for unpacking the organisation domain of the intellectual structures framework. the structure domain emphasises the diversity and flexibility of the organising construct. traditionally, organisational systems, such as enterprise resource planning (erp), tend to support bureaucratic forms of organising, locking in existing structures and limiting flexibility and ability to change (mohr 1971), especially when shifting from local to global operations (crawford et al 2009). digital ecology has enabled distributed, virtual and temporary organisational forms and expanded the range of viable structures from micro enterprises to multi-national networks of partnerships. but most importantly, the digital ecology has facilitated flexible organisational forms that enable dynamic transformation of structure to adapt to changes in context, whether this is regulatory, operational or socio-cultural. significantly, digital ecologies also facilitate the co-existence of formal and informal (ali 2011) structures that allow organisations to become ambidextrous (agostini et al. 2017, o’reilly & tushman 2013). domains orientations control sense-making argumentation instrumental strategic communicative discursive structure bureaucracy matrix networked, selfdirected self-organised project-based actions mechanistic social systems (ritualized tasks) service dominant logic and new business models virtual teams ubiquitous sociotechnical systems relationships command and control decision making hierarchy and control cultural and social systems (negotiated meanings and practices) systems for institutional democracy figure 4: object classes for orientations of new structure, relationships and action domains in an expanded reinterpretation of the organisation domain of the intellectual structures framework the relationships domain highlights the significance of connections between actors to support diverse organising. the clear orientation of this domain is communicative. the emphasis in this domain is to support changing interactions between organisational actors by adopting flexible and scalable coordinating mechanisms (adler et al 2011). such mechanisms require a shared purpose, and an ethic of cooperation where collaboration is both valued and rewarded. one organizing principle that this domain encompasses is that organisations can cooperate and work together in parts of their business where they did not believe they had competitive advantage, while remaining fiercely competitive in other areas (fisher 1992) as demonstrated in the extensive alliances in the airline industry. another illustration of the emphasis on the relationships domain is relational contract theory (macneil 2000, 1980). this theory defines the behaviour of the actors involved in a transaction which is based on mutual trust, is enacted through collaboration (campbell, mulcahy & wheeler, 2017) and expressed in norms that define the social relations that underpin the interactions. a very practical manifestation of such contracts is performance based contracts australasian journal of information systems linger & hasan 2020, vol 24, selected papers from the is foundations conference making is relevant in a connected world 11 (pbc), especially for long term sustainment of defence equipment (mouzas 2016, pbc coe, 2014). underpinning the shift of emphasis of this domain is that the technological determinism is being undermined by innovative appropriation of digital ecologies, particularly by civil society and especially by marginalized groups and developing societies (see e.g., steyn, 2007). organisations now need to be alert to such innovations and be open to the possibilities represented by such innovations (hanseth and lyytinen 2010). the action domain deals with the organisation of work and the nature of work done, spanning the continuum from ritual tasks to independent, self-directed intellective tasks (weick and roberts 1993; cornelissen and kafouros 2008). digital ecologies afford new ways of working and thinking, transforming work by supporting the continuum of routine prescribed procedures to flexible activities. traditionally, the focus of (organisational) work was on continuous improvement and optimization with the objective of making organisations (the noun) more efficient and effective (vom brocke et al. 2011). digital ecologies afford actions encompassing a range of tasks and activities encapsulated in the concept of knowledge work (iivari and linger 2000). knowledge work integrates doing, the pragmatic task, with thinking, the cognitive task, and talking, the communicative task. these facets together influence how an instance of a work task will be performed. the knowledge work perspective is supported in activity theory where activity is the holistic unit of analysis of work and activity is the relationship between the subject (actor(s) engaged in the doing) and the object of work that encapsulates the purpose and motives of doing (leontiev 1981). an activity both mediates, and is mediated by, the physical, virtual and psychological tools used (hasan et al 2010). digital ecologies provide the means to design, construct and execute activities that are flexible to reflect both the changing context and learning from performing the activity. 4 illustrating the domains to illustrate our rethinking of the intellectual structures framework, we have selected a representative is paper that highlights an aspect of the domains. subsequently we present two examples of investigations of phenomena that provide a more holistic view of the application of our revised framework. both examples concern situations where the increasing complexity of digitally-enabled sociotechnical systems affect traditional institutions: one is the military and the other local government. what is evident from these illustrations is that the interpretations of the framework have also changed. hirschheim et al (1996) presented their framework as a matrix of domains of change on one axis and the orientation of that change on the other axis. the cells of the resulting matrix were populated with object systems that represented archetypal is. their discussion of the fragmented adhocracy that the matrix represented implied that these were discrete systems. their argument was that is as a discipline was represented as the sum total of all the object systems. the basis for our rethinking of the framework was the inherent complexity of the phenomena being addressed and the emergence of diverse digital ecologies. together, these two factors are the basis for innovative systems, and the research that underlies such developments, that do not adhere to boundaries of the original framework. digital ecologies raise affordances that australasian journal of information systems linger & hasan 2020, vol 24, selected papers from the is foundations conference making is relevant in a connected world 12 pluralise the impact of systems from individual, to group, to collectives (organisations) to societal levels (upwood 2005). and these impacts are experienced simultaneously at all levels. but more importantly, the consequences of the impact at each level also influences the development of the is artefact in other layers. it worth noting again the notion that the is artefact includes technology artefacts, information artefacts and social artefacts (lee et al 2015). our illustrations of the revised framework demonstrate that affordances of digital ecologies allow complex phenomena to be addressed in ways that span domains and orientations. our contention remains that is is a fragmented adhocracy. however, the is artefact is itself more complex and spans space-time. australasian journal of information systems linger & hasan 2020, vol 24, selected papers from the is foundations conference making is relevant in a connected world 13 4.1 domain examples from the literature domains of change comment digital tools (whitmore et al. 2015). the internet of things (iot) is a paradigm where everyday objects can be equipped with identifying, sensing, networking and processing capabilities that will allow them to communicate with one another and with other devices and services over the internet to accomplish some objective. ultimately, iot devices will be ubiquitous, context-aware and will enable ambient intelligence. this article reports on the current state of research on the internet of things by examining the literature, identifying current trends and describing challenges that threaten iot diffusion as a hybrid technical evolution. the paper demonstrates the pervasive nature of the digital ecology and its potential impact on all orientations not just the control orientation as in the original framework information (pfaff and hasan 2012) despite their reputation as an evolving shared knowledge repository, wikis are often treated with suspicion in organisations for management, social and legal reasons. this paper reports a field study that was undertaken of a corporate wiki developed to capture, and make available, organisational knowledge for a large manufacturing company as an initiative of their knowledge management program. a q methodology research approach was selected to uncover employees’ subjective attitudes to the wiki so that the firm could more fully exploit the potential of the wiki as a ubiquitous tool for tacit knowledge management. tensions were noted when employees struggled with the lack of structure in the wiki and their responsibility to created structure as well as content in a more democratic organisational culture. the significance of this paper is that it highlights this domain is contested with the impact of change dependent on the perspective adopted. it also points to the paradigm shift required of all actors particularly in the communicative and discursive orientations. interaction (melville 2010) this study confronts the contribution of is to countering the rapidly environmental degradation due to human activity. it takes the view that business enterprises are a dominant form of social organisation that contribute to the worsening, as well as the enhancement, of the natural environment. the paper describes the critical role that is can play in shaping beliefs about the environment, in enabling and transforming sustainable practices in organisations, and in improving environmental and economic performance. an augmented belief-action-outcome framework and associated research agenda provide the basis for a new discourse on is for environmental sustainability. this explicitly links social and organisational contexts regarding belief formation where individuals are conflicted between organisational values (eg short term profit motive) and personal values shaped by societal concerns (eg going green to save the planet). the paper emphasises the prominent role of the communicative and discursive orientations that are afforded by the digital ecology. structure (sharma, et al. 2002). the open source software (oss) model is a fundamentally new and revolutionary way to develop software. despite the success of oss, for-profit organisations are having difficulty building a business model around the open source paradigm. this research explores how organisations can foster an environment similar to oss to manage their software development efforts to reap its numerous advantages. drawing on organisational theory, the research develops a framework that guides the creation and management of a hybrid-oss community structures within an organisation. the paper highlights the socio-technical imperatives that underpin diverse organising principles afforded by the digital ecology australasian journal of information systems linger & hasan 2020, vol 24, selected papers from the is foundations conference making is relevant in a connected world 14 domains of change comment relationships (levy et al. 2003) co-opetition entails sharing knowledge that may be a key source of competitive advantage. yet, the knowledge gained by cooperation may also be used for competition. this paper develops a game–theoretic framework for analysing inter-organisational knowledge sharing under co-opetition and proposes guidelines for the management of explicit knowledge predicated on coordination and control theory that remain untested. the research investigates these issues in the context of small and medium-sized enterprises. the paper points to the changing landscape, in an organisational context, of the engagement between actors that are possible in the digital ecology action (vidgen and wang 2009 this paper develops a framework for the organisation of agile software development that identifies enablers and inhibitors of agility and the emergent capabilities of agile teams. the work is grounded in complex adaptive systems and draws on three principles of coevolving systems: match co-evolutionary change rate, maximize self-organizing, and synchronize exploitation and exploration. these principles are used to study the processes of two software development teams, one using extreme programming and the other using a more traditional, waterfall-based development cycle. from these cases a framework for the organisation of agile software development is developed. time pacing, self-management with discipline and routinization of exploration are among the agile enablers found in the cases studies while event pacing, centralized management, and lack of resources allocated to exploration are found to be inhibitors to agility. emergent capabilities of agile teams that are identified from the research include coevolution of business value, sustainable working with rhythm, sharing and team learning, and collective mindfulness. the paper makes two important points; the need for a complexity perspective to understand the potential of digital ecologies and the need to do things differently in such a context in order to exploit the affordances of digital ecologies. table 2: is papers that illustrate each domain within digital ecologies australasian journal of information systems linger & hasan 2020, vol 24, selected papers from the is foundations conference making is relevant in a connected world 15 4.2 research example 1: team capability in the military this project began with the development of go*team, an innovative online, team-gaming system based on the strategy board game go4. the project expanded over four years into an action research program to investigate the use of go*team to develop the human capability for a network-centric configuration of collaborative teams in the complex military environment known as the “fog of war”5. the objectives for the project focused on team dynamics, situational awareness, information sharing, communication and trust, and timely and appropriate decision making under uncertainty. players of go*team belong to teams whose members can be anywhere on a computer network. under researcher control, the protocols of play can be set to suit the circumstances of play. the most exciting capability of the go*team software is that it restricts what each player sees on their computer screen so no single team members can see the status of the whole board. online forms of communication between team members during play are integral to the game as it enables team members to share status information as well as plan subsequent moves. recordings of a game, together with audio recordings of team communications, provide data for the subsequent analysis and research. domain of change comments digital tools experiential learning tool simulating a network-centric configuration. go*team was designed to address the instrumental and strategic control orientation as well as the communicative orientation information information flows shifted from command and control hierarchy to more peer-topeer information sharing for increased group situational awareness. the game included a variety of online communication that addressed the instrumental orientation and as this was networked, it also included strategic and communicative orientation. the game allowed team members to reason about the game which addressed the discursive orientation interaction, capability for a more innovative, flexible, agile organisation through networks of self-directed teams while adhering to game rules and rules of engagement. the design of the system and its operations (game playing) addressed all orientations structures more self-organising networked configuration within the context of hierarchical structure. the game was explicitly designed to explore the network-centric paradigm. in this it explicitly addressed the communicative and discursive orientation. relationships encouraging informal collaboration in an intensive competitive game context. while the intent of the game was to encourage and develop a collaborative approach, game playing also showed that more traditional hierarchical approaches were also observed. this shows that all orientations are potentially addressed. actions game playing demonstrated the ability of team members to undertake ad hoc problem solving and independent decision making. this required players to understand and adapt to new ways of “doing things”. this has a strategic orientation but explicitly addresses the communicative and discursive orientation. table 3: an overview of the go*team games system game playing demonstrated how well the go*team game embeds players in an environment that reflects the situation of network-centric teams where the transfer of information is needed 4 https://www.usgo.org/brief-history-go 5 https://www.yourdictionary.com/fog-of-war australasian journal of information systems linger & hasan 2020, vol 24, selected papers from the is foundations conference making is relevant in a connected world 16 to enable shared situational awareness and sense-making leading to relevant decision-making and action. the research was designed to play games with a variety of participants and situations to investigate these phenomena. this expanded the initial focus on the tools domain to the domains of structure, information, actions, relationships and eventually interraction as shown in table 3. 4.3 research example 2: government-community engagement the motivation for this action research project was new state government legislation required local municipal councils to expand their community engagement (ce) programs so that a wider range of citizens could participate in government processes. we were approached by a council’s ce team to investigate how the council could use ict to move beyond the current ce practice of traditional town-hall meetings. the research investigated the tensions that arose between a bureaucratic organisation and a more open online system to engage citizens. the investigation began in 2011 with an intervention involving a workshop where the ce team would play the role of citizens in simulated online discussion on topics related to community needs and aspirations for the future. the workshop became a pilot for the use of such technologies for online ce. based on feedback from the workshop, the council implemented a proprietary online discussion forum which was evaluated over the subsequent seven years to ascertain how successful the council had been in their use of ict for ce and what impact, if any, has this had on the council. domain of change comments digital tools web-based fill in form to a moderated discussion forum to social media. the council relied on its existing it infrastructure but addressed the strategic orientation by adapting it for online communication with citizens. over the period of the project the ce approach addressed the communicative orientation but not the discursive as most communication was still pushed from the council to citizens. information government to citizen information flow changing as citizens had the means to converse directly with council. the ce project opened a variety of communications channels that addressed the both control orientations and at points also the communicative channel. however as throughout the project the council still set the agenda, the discursive orientation was not addressed. interaction broader citizen participation in democracy. although the intent of the council’s ce initiatives were ostensibly to implement e-democracy, our analysis suggested that the discursive orientation was not addressed. structures institutionalised bureaucracy. the ce project was grounded in the existing council structures and did not move beyond the instrument orientation relationships tensions between government bureaucrats and the variety of citizens in different circumstances in the community according to different local issues. the tension was based on the council retaining control over agendas and issues and the decision-making process. the emphasis on the control orientation meant that the affordances of online ce were only superficially enacted. actions corporate style management and community activism. while online ce suggests potentially new and empowering action, the reality of the project was that it was ritualised in terms of compliance with legislation and control over decision was maintained by the council. thus, the dominate orientation was control although it did open some aspects of the communicative orientation. table 4: an overview of online government-community engagement australasian journal of information systems linger & hasan 2020, vol 24, selected papers from the is foundations conference making is relevant in a connected world 17 this form of council led e-democracy represents a departure from the council management’s “comfort zone” and created tensions. however, we do not argue that the reluctance to adopt e-democracy is simply a matter resistance to change. the dialectic challenge facing government is the need to deliver mandated efficiencies while also providing opportunities for civil society to be involved in decision making and co-production of services. tensions between the openness of ict driven ce and the everyday working of council were evident in our findings. what was also apparent is that the real issue confronting council was resolving the tension between their formal operational imperatives and the demands for informal citizen participation. in our study, a synthesis to resolve this contradiction has not been found. 5 concluding remarks we propose that the revised intellectual structures framework can be used as an analytical instrument to make sense of a complex phenomenon in a complex ecology without simplifying it or breaking it into separate components. we believe that it is often important to take this holistic approach. indeed our conclusions, decisions and actions may be wrong if we don’t. this paper began by suggesting that the mid-1990s was a pivotal time in the impact of digital technologies on society just as the original intellectual structures framework (hirschheim et al 1996) was published. the subsequent evolution of digital ecologies made us re-evaluate this framework, extending its domains and incorporating more complex orientations we chose the intellectual structures framework as our starting point because it reinforces the well-accepted and broad principles of structuration and social action as its theoretical base. as stated earlier, this theoretical grounding provides a robust basis for the amendments we have made to the original framework to meet the imperatives of the diversity and complexity of the digital age, following the global spread of the internet and the transforming advances of the web. significant is papers have been used to illustrate the application of each domain of the revised intellectual structures framework. two examples of our own investigations of phenomena in complex ecologies have provided a holistic view of the application of our revised framework. we propose the resulting revised intellectual structures framework as a research tool that • is holistic and appropriate for the study of complex ecologies • incorporates and expands the original framework into six domains based on the changed digital ecology in which is now exists • captures the level of complexity within each domain of change • is able to underpin non-reductionist is research into real-world issues • recognises that the complexity of phenomena under study usually spans more than one cell of the matrix of the framework and any development in one cell needs to be conducted in the context of other cells. • enables research which is reductionist to posit their results in a wider context. • has implications for scope, methodology, theory and epistemology in is the revised intellectual structures framework supports the current trends for organisations to be ambidextrous (o'reilly and tushman, 2013; raisch et al 2009) accommodating formal structures and informal networks. this elevates decentralized decision-making, self-directed australasian journal of information systems linger & hasan 2020, vol 24, selected papers from the is foundations conference making is relevant in a connected world 18 and informal groups in network-centric arrangements where actors are empowered by giving them the authority and skills to self-organize. these trends allow innovative organising forms that exploit (and explore) the digital ecologies. these trends are not simply an appropriation of the informal into the formal structures but simply acknowledging the existence of the informal, ceding legitimate authority to those forms and working on the mechanisms for dynamic co-existence to meet the emergent needs of each situation. the revised intellectual structure framework has implications for is research in respect of the scope, methodology and in particular the ability to address complex phenomena. simplifying complicated problems can improve our understanding and lead to viable solutions. however, simplifying a complex problem can often make it wrong in the sense that solutions to the simplified problem do not work for the original complex problem (gray 2009). in attempting to be scientific, is scholars have avoided or limited the scope of their investigation of complex phenomena despite calls to conduct research that really matters (desouza et al. 2006); addresses wicked problems (kazlauskas and hasan, 2009); engage with grand challenges, (eymann et al 2015; winter and butler 2011; vom brocker et al. 2015) and especially issues associated with the fifteen global challenges identified by the united nations millennium project (un 2015; lee 2015). articulating and investigating the role of is in large-scale, complex problems can involve the adoption of difficult strategies and choice of controversial research methods and approaches (winter and butler 2011). the revised intellectual structure framework is an attempt to provide an instrument for facing these challenges in the following ways. • it broadens the scope of acceptable problems to be investigated by is researchers into realms of complex and wicked problems. when a phenomenon of interest has a large number of interacting variables, the acceptable empirical approach is to reduce the number of variables investigated and to bound the topic to make it manageable according to scientific, mode 1, approaches (gibbons et al 1994). some is researchers are moving outside these constraints, for example those who are beginning to look at is development as complex adaptive systems (e.g. kautz 2012, benbya and mckelvey, 2006). • it guides projects that address many interacting aspects of wicked problems. such holistic projects could potentially include phenomena covering all six domains of the framework including dynamic and diverse perspective of the relationship in these domains. • where research is focused on a single perspective of one domain of the framework, its bounded context can be acknowledged, but the implications of the research for domains and orientations not addressed can be discussed. in this way the research is focussed but does not lose a holistic perspective. • methodological consequences of research that addresses complex phenomena includes innovative research approaches to cross boundaries and incorporate elements of complexity alongside more traditionally scientific methods. • some is scholars are already exploring theories and methods suitable for the study of complex phenomena (see for example ketter et al. 2016 schmitz et sl. 2016). the revised framework provides a context to situate such approaches within the fragmented adhocracy of is australasian journal of information systems linger & hasan 2020, vol 24, selected papers from the is foundations conference making is relevant in a connected world 19 the contribution of this paper is an extended framework that makes sense of the fragmented adhocracy of is in the context of complexity and diverse digital ecologies. its authenticity comes from its grounding in the original insightful intellectual structures framework (hirschheim et al’s 1996) but takes into account developments in ict, the internet and digital ecologies. these developments underpin our reinterpretation of the domains of change in order to explicitly broaden the scope of is and enable us to populate the cells of the matrix of the framework with object systems that are aligned with the affordances of diverse digital ecologies. moreover, the revised framework also reinterprets is not as a discreet artefact that populates a particular domain and orientation (a cell in the matrix) but as a socio-technical artefact with many facets that span the matrix in a variety of ways. we present revised intellectual structures framework as the basis for a theory of explanation which “… provides an explanation of how, why, and when things happened, relying on varying views of causality and methods for argumentation” (gregor 206 p 619). what is more, the revised intellectual structures framework is an instrument for making sense of problems that are messy, complex, chaotic and wicked. it does not simplify complex situations, as we have already noted the folly of this approach, but it plays a vital role in making is more relevant in diverse social, institutional, organisational and digital ecologies. references adler p., heckscher c., and prusak l 2011 “building collaborative enterprise”, harvard business review 89(7-8) pp 94-101 agostini, l., nosella, a., & filippini, r. (2017). ambidextrous organisation and knowledge exploration and exploitation: the mediating role of internal networking. international journal of business innovation and research, 14(1), 122-138. ali i. 2011 “coexistence or operational necessity: the role of formally structured organization and informal networks during deployments, in collective c2 in multinational civilmilitary operations, proceeding of 16th iccrts, banville, c., and landry, m., 1989. “can the field of mis be disciplined?” comm. of the acm, 32 1 pp 48-60. baskerville, r. and pries-heje, j., 2002. information systems development@ internet speed: a new paradigm in the making! ecis 2002 proceedings, p.68. baum, s. d. (2017). social choice ethics in artificial intelligence. ai & society, 1-12. benbasat, i. and r. weber 1996 “research commentary: rethinking “diversity” in information systems research”, information systems research, 7, pp 389-399. benbasat, i., and zmud, r. w. 2003. “the identity crisis within the is discipline: defining and communicating the discipline's core properties”. mis quarterly, 27 pp 183-194. benbya, h. and mckelvey, b. 2006. ”toward a complexity theory of information systems development”. information technology & people (19:1), pp 12-34. blackler, f., 1995 “knowledge, knowledge work and organizations: an overview and interpretation”, organization studies, 16(6), pp 1021-1046 boland, r., and tenkasi, r., 1995. “perspective making and perspective taking in communities of knowing”, organization science, 6/4: pp 350-372. australasian journal of information systems linger & hasan 2020, vol 24, selected papers from the is foundations conference making is relevant in a connected world 20 campbell, d., mulcahy, l. and wheeler, s. eds., 2017. changing concepts of contract: essays in honour of ian macneil. springer. castells, m. 2015. networks of outrage and hope: social movements in the internet age. john wiley & sons. cecez-kecmanovic, d., galliers, r.d., henfridsson, o., newell, s. and vidgen, r., 2014. the sociomateriality of information systems: current status, future directions. misq, 38(3), pp.809-830. cecez-kecmanovic d. jerram c. a 2002 “sensemaking view of knowledge in organizations” proceedings of ecis2002, gdansk. clarke, r. 2016. “big data, big risks”. information systems journal, 26(1), pp 77-90. crawford, k. hasan, h. warne, and l. linger, h. 2009 “from traditional knowledge management in hierarchical organizations to a network centric paradigm for a changing world”, emergence: complexity and organization, (11:1) cornelissen, j.p., and kafouros, m., 2008. “the emergent organization: primary and complex metaphors in theorizing about organizations”, organization studies, 29 7: pp 957-978. d'aquin, m., troullinou, p., o'connor, n. e., cullen, a., faller, g., & holden, l. (2018, december). towards an ethics by design methodology for ai research projects. in proceedings of the 2018 aaai/acm conference on ai, ethics, and society (pp. 54-59). acm. desouza, k., el sawy, o., galliers, r., loebbecke, c. and watson r. 2006 “beyond rigor and relevance towards responsibility and reverberation: information systems research that really matters”, communications of the association for information systems, 17(1), #16. eymann, t., legner, c., prenzel, m., krcmar, h., müller, g., and liggesmeyer, p. 2015. “addressing grand challenges”. business & information systems engineering, 57(6), pp 409-416. faratin, p., clark, d. d., bauer, s., lehr, w., gilmore, p. w., and berger, a. 2008. “the growing complexity of internet interconnection”, communications & strategies, (72), pp 51-. feller j., finnegan p., hayes j., o’reilly p. 2012 ‘orchestrating’ sustainable crowdsourcing: a characterisation of solver brokerages, journal of strategic information systems 21 216–232 fisher, l. m., 1992. “preaching love thy competitor”, new york times, march 29, 1992. gibbons m. limoges c., nowotny h., schwartzman s. scott p. and trow m (1994). the new production of knowledge: the dynamics of science and research in contemporary societies. london: sage grant, r. m., 1996. toward a knowledge-based theory of the firm. strategic management journal, 17(s2), 109-122. grant, r.m., 2002. “the knowledge-based view of the firm”, in: the strategic management of intellectual capital and organizational knowledge, c.w. choo and n. bontis, eds., oxford university press, new york, ny, pp. pp 133-48. gray, d. 2009. “complicated vs. complex”, http://communicationnation.blogspot.com/2009/11/complicated-vs-complex.html australasian journal of information systems linger & hasan 2020, vol 24, selected papers from the is foundations conference making is relevant in a connected world 21 gregor, s. 2006, “the nature of theory in information systems,” mis quarterly 30:3 pp 612642. grover v. 2012 “the information systems field: making a case for maturity and contribution”, journal of the ais, 13:4 pp 254-272 hanseth o and lyytinen k. 2010 “design theory for dynamic complexity in information infrastructures: the case of building internet”. journal of information technology 25(1), pp 1-19. hasan, h., smith, s., and finnegan, p. 2016 “an activity theoretic analysis of the mediating role of information systems in tackling climate change adaptation”. info systems j, doi: 10.1111/isj.12104. hasan h. and crawford k. 2003 “codifying or enabling: the challenge of knowledge management systems”, journal of the operations research society. 54, pp 184-193. hasan h. and crawford k. 2007 “knowledge mobilisation in communities through sociotechnical system”, journal of km research and practice 5/4 pp 237-248. hasan, h, kazlauskas, a. and crawford, k 2010 “blending complexity and activity frameworks for a broader and deeper understanding of is”, proceeding of icis, st louis, december. hirschheim, r klein, h 2012 “a glorious and not-so-short history of the information systems field”, journal of the ais, 13:4 pp 188-235. hirschheim, r, klein, h.k. and lyytinen, k. 1996. “exploring the intellectual structures of information systems development: a social action theoretical analysis”. accounting management and information technologies, 6, pp 1-64 huang, t. t., and sun, b. q. 2016. “the impact of the internet on global industry: new evidence of internet measurement”. research in international business and finance, 37, pp 93-112. iivari, j., and linger, h. 2000. “the characteristics of knowledge work: a theoretical perspective”. proceedings of the americas conference on information systems, amcis2000, august, association for information systems, long beach, ca. jones g. 2010 organizational theory, design, and change, pearson international edition. jones m, and karsten h 2008 “giddens's structuration theory and information systems research”, mis quarterly 32:1 pp 127-157. kane, g. c., and. fichman r. g. 2009 "the shoemaker's children: using wikis for information systems teaching, research, and publication." mis quarterly 33:1 pp 1-17. kautz, k. 2012. “information systems development projects as complex adaptive systems”, proceedings of the australasian conference on information systems, geelong. kazlauskas, a., and hasan, h.m. 2009. “web 2.0 solutions to wicked climate change problems”. australasian journal of information systems, 19(2), pp 23-36. ketter, w., peters m., collins j. and gupta a. 2016 “competitive benchmarking: an is research approach to address wicked problems with big data and analytics” mis quarterly, 40(4), pp.1057-1080. australasian journal of information systems linger & hasan 2020, vol 24, selected papers from the is foundations conference making is relevant in a connected world 22 kim r. m., and kaplan s. m., 2006, "interpreting socio-technical co-evolution: applying complex adaptive systems to is engagement", information technology & people, (19: 1) pp. 35 – 54 klein, h., and hirschheim, r. 2008. “the structure of the is discipline reconsidered”, information and organization, 184 pp 280-302 kleineberg, k. k., and boguná, m. 2015. digital ecology: coexistence and domination among interacting networks. scientific reports, 5, 10268 downloaded june 25, 2018 from https://www.nature.com/articles/srep10268 laghari, s., and niazi, m. a. 2016. “modeling the internet of things, self-organizing and other complex adaptive communication networks: a cognitive agent-based computing approach”. plos one, 11(1), e0146760. lee, j.k. 2015. preface in vom brocke j. stein a., hofmann, s. tumbas s. eds “grand societal challenges in information systems research; ideas from the ercis” virtual seminar series, springer briefs in information systems. lee, a., thomas, m., and baskerville r. 2015 “going back to basics in design science: from the information technology artifact to the information systems artefact”, info systems j 25, pp 5 –21. leontiev, a.n., 1981. problems of the development of mind, moscow: progress press. levy, m., loebbecke, c. and powell, p. 2003, "smes, co-opetition and knowledge sharing: the role of information systems", european journal of information systems, (12: 1), pp. 3-17. linger, h. and warne, l. 2001 “making the invisible visible: modelling social learning in a knowledge management context”, special issue on knowledge management of the australian journal of information systems, 8, pp 56-66. macneil, i.r., 2000. relational contract theory: challenges and queries. nw. ul rev., 94, p.877908. macneil, i.r., 1980.the new social contract: an inquiry into modern contractual relations (rosenthal lectures; 1979). new haven: yale university press. magro, m. 2012 “a review of social media use in e-government” adm. sci. 2(2), pp 148-161 march, j. g. 1991. “exploration and exploitation in organizational learning”. organization science, (21), pp 71-87. march s. and niederman f. 2012 “the future of the information systems discipline: a response to walsham”, journal of information technology 27, pp 96–99 mathiassen, l. 1996 “information systems development: reflections on a discipline”, accounting management and information technologies, 6(1) pp 127-132. mckelvy b. tanriverdi h. and yoo y. 2015 “complexity and information systems research in the emerging digital world”, misq special issue, call for papers melville, n. p. 2010. “information systems innovation for environmental sustainability”. mis quarterly, 34(1), pp 1-21. mintzberg, h., 1979 the structuring of organizations: a synthesis of the research, university of illinois at urbana-champaign's academy for entrepreneurial leadership historical australasian journal of information systems linger & hasan 2020, vol 24, selected papers from the is foundations conference making is relevant in a connected world 23 research reference in entrepreneurship. available at ssrn: http://ssrn.com/abstract=1496182 mohr, l. 197, organizational technology and organizational structure administrative science quarterly, 16/4 pp. 444-459 mouzas, s., 2016. performance based contracting in long-term supply relationships. industrial marketing management, 59, 50-62. newell, s. swan j. a. and galliers r. d 2000 “a knowledge-focused perspective on the diffusion and adoption of complex information technologies: the bpr example”, info systems journal 10 pp 239-259 nonaka, i. 1994. “a dynamic theory of organizational knowledge creation”, organization science 5/1, pp 14-37. o'reilly iii, c. a., and tushman, m. l., 2013. organizational ambidexterity: past, present, and future. academy of management perspectives, 27(4), 324-338. o'reilly, t., 2007. “what is web 2.0: design patterns and business models for the next generation of software”. communications & strategies, 1 pp 17-37. orlikowski, w.j., 1992. “the duality of technology: rethinking the concept of technology in organizations” organization science, 1 1: pp 398-427. orlikowski, w.j. 2002. “knowing in practice: enacting a collective capability in distributed organizing.” organization science, 13:3: pp 249-273. panetto, h., and cecil, j. 2013. “information systems for enterprise integration, interoperability and networking: theory and applications”. enterprise information systems, 7(1), pp 1-6. pbc coe, 2014. designing successful performance based contracts, dmo pbc coe discussion paper. pfaff, c. and hasan, h. 2012, “wiki-based knowledge management systems for more democratic organizations”, journal of computing information systems 52/2 pp 73-82. phister jr, p. w. 2010. cyberspace: the ultimate complex adaptive system. air force research lab, rome ny. raisch, s., birkinshaw, j., probst, g., and tushman, m.l. 2009. “organizational ambidexterity: balancing exploitation and exploration for sustained performance”. organization science, 20(40), pp 685-695 rosenberg, r. s. 2013. the social impact of computers. elsevier. russell, s., hauert, s., altman, r., & veloso, m. (2015). ethics of artificial intelligence. nature, 521(7553), 415-416. scott, s.v. and orlikowski, w.j., 2014. entanglements in practice: performing anonymity through social media. mis quarterly, 38(3), pp.873-893. schalow, p. r., winkler, t. j., repschlaeger, j., and zarnekow, r. 2013. “the blurring boundaries of work-related and personal media use: a grounded theory study on the employee's perspective” proceedings of ecis pp. 212. australasian journal of information systems linger & hasan 2020, vol 24, selected papers from the is foundations conference making is relevant in a connected world 24 senge, p., 1990. the fifth discipline: the art and practice of the learning organization. new york: currency/doubleday. senge, p. m. 2014. the dance of change: the challenges to sustaining momentum in a learning organization. crown business. sidorova a. evangelopoulos n. valacich j. ramakrishnan t. 2008. “uncovering the intellectual core of the information systems discipline”, mis quarterly, 32(3) pp 467482. schmitz, k., teng j. and webb k. 2016 “capturing the complexity of malleable it use: adaptive structuration theory for individuals” mis quarterly, 40(3), pp.663-686. sharma, s., sugumaran, v., and rajagopalan, b. 2002. “a framework for creating hybrid-open source software communities”. information systems journal, 12(1), pp 7-25. snowden, d. 2002 “complex acts of knowing: paradox and descriptive” self-awareness. journal of knowledge management, 6(2). steyn, j., 2007. “community memory and ict in a developing economy”. in stillman, l., and johanson, g. (eds.) constructing and sharing memory: community informatics, identity and empowerment, cambridge uk. cambridge scholars publishing. stair, r., and reynolds, g. 2013. principles of information systems. cengage learning strati a. 2007 “sensible knowledge and practice-based learning” management learning pp 38: 61 strelioff, c.c., vijaykrishna, d., riley, s., guan, y., peiris, j.m. and lloyd-smith, j.o., 2013. inferring patterns of influenza transmission in swine from multiple streams of surveillance data. proc. r. soc. b, 280(1762), p.20130872. surowiecki, j. 2004 the wisdom of crowds: why the many are smarter than the few and how collective wisdom shapes business, economies, societies and nations. new york: doubleday. thompson, k., miller, g. j., and wilder, r. 1997. “wide-area internet traffic patterns and characteristics”. network, ieee, 11(6), pp 10-23. ulieru, m., 2010. special issue on digital ecologies. ieee trans. systems, man, and cybernetics, part a, 40(6), pp.1341-1343. un 2000. millennium development goals 2000-2015, http://www.un.org/millenniumgoals/ upward, f. (2005). the records continuum. in s. mckemmish, m. piggot, b. reed, & f. upward (eds.), archives: recordkeeping in society (pp. 197-222). wagga wagga, new south wales: centre for information studies, charles stuart university. vidgen, r., and wang, x. 2009. “coevolving systems and the organization of agile software development”. information systems research, 20(3), pp 29 pp 355-376. vodanovich, s., sundaram, d. and myers, m., 2010. “research commentary-digital natives and ubiquitous information systems”. information systems research, 21(4), pp.711-723. vom brocke, j., becker j., braccini a., butleris r., hofreiter b., kapočius k., de marco m., schmidt g., seidel s., simons a., skopal t., stein a., stieglitz s., suomi r., vossen g., robert winter r., and wrycza s. 2011 "current and future issues in bpm research: a australasian journal of information systems linger & hasan 2020, vol 24, selected papers from the is foundations conference making is relevant in a connected world 25 european perspective from the ercis meeting 2010" communications of the association for information systems 28.1. vom brocke, j., stein, a., hofmann, s., and tumbas s. 2015. grand societal challenges in information systems research and education ideas from the ercis virtual seminar series, springer briefs in information systems. springe wasko, m. m., and faraj, s. 2005. “why should i share? examining social capital and knowledge contribution in electronic networks of practice”. mis quarterly, 27 pp 35-57. wagner d., vollmar g., and wagner h., 2014 "the impact of information technology on knowledge creation: an affordance approach to social media", journal of enterprise information management, (27: 1), pp.31 – 44 walsham, g. 1996. “exploring the intellectual structures of information systems development: a short critique”. accounting, management and information technologies, 6(1), pp 133-138. walsham, g. 2012 are we making a better world with icts? reflections on a future agenda for the is field, journal of information technology 27: pp 87-93; warne, l., ali, i., pascoe c. 2003 social learning and knowledge management – a journey through the australian defence organization: the final report of the enterprise social learning architectures task, dsto report, dsto-rr-0257, defence systems analysis division, information sciences laboratory, department of defence westrup c 2012 “in need of narratives of is”, journal of information technology 27, pp 106–107 weick, k.e., and roberts, k.h., 1993. “collective mind in organizations: heedful interrelating on flight decks”. administrative science quarterly, 38: 357–382. whitmore, a., agarwal, a., and da xu, l. 2015. “the internet of things—a survey of topics and trends”. information systems frontiers, 17(2), pp 261-274. winter, s., and butler, b. 2011 “creating bigger problems: grand challenges as boundary objects and the legitimacy of the information systems field”, journal of information technology 26, pp 99–108 yoo, y. 2010. “computing in everyday life: a call for research on experiential computing”. mis quarterly, 34 pp 213-231. yoo, y., henfridsson, o. and lyytinen, k., 2010. “research commentary-the new organizing logic of digital innovation: an agenda for information systems research”. information systems research, 21(4), pp.724-735. copyright: © 2020 linger & hasan. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v24i0.2509 http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems linger & hasan 2020, vol 24, selected papers from the is foundations conference making is relevant in a connected world 26 1 introduction 2 background 3 revisiting the intellectual structures of is 3.1 reinterpreting the technology domain: tools to exploit the digital ecology 3.2 language domain: incorporating knowledge and change 3.3 organisation domain: the impact of complexity 4 illustrating the domains 4.1 domain examples from the literature 4.2 research example 1: team capability in the military 4.3 research example 2: government-community engagement 5 concluding remarks paper title (paper title style) australasian journal of information systems wildenauer 2020, vol 24, selected papers from the 2019 aice conference the shared responsibility model 1 the shared responsibility model: levers of influence and loci of control to aid regulation of ethical behaviour in technology platform companies michael wildenauer la trobe business school la trobe university melbourne, australia m.wildenauer@latrobe.edu.au abstract this exploratory paper provides social context for platform corporations and examples of ethical transgressions by them and then canvasses the role of various organizational actors in controlling the ethical behaviour of ‘platforms', which may be seen to be more than usually problematic in this regard. from this survey, the conclusion is drawn that there may be no single actor that offers sufficient leverage to change organizational ethical behaviour. the paper then suggests the shared responsibility model as a possible conceptual framework for a better understanding of the issue of ethical control and recommends practical interventions that may assist in realizing ethical behaviour by platforms that more closely aligns with societal expectations. the paper offers a caution about the side-effects of interventions to improve ethical behaviour, before concluding by pointing out implications of these findings for state-actor regulators and avenues for future research. keywords: ethics; technology platform; corporation; directors; shared responsibility model; regulation; ethical behaviour; locus of ethical control; ethical influence 1 introduction and context business is not seen by the public to behave in an ethical manner, and while technology is still the most trusted industry sector, that public trust is falling globally, and is already low in certain spheres such as trust in social media in the provision of general news and information (edelman, 2020). academic, government and industry commentary (e.g. hubbard, 2019; ingram, 2014; king, 2018), traditional media (e.g. lownsbrough, 2017), and social media (e.g. elvery, 2018) are replete with tales of ethical wobbliness, misdeeds, and even malfeasance by large platform corporations such as facebook, google, uber, and amazon, and accordingly the public feels that these companies cannot always be trusted to do the right thing and should be subject to greater regulation (smith, 2018). governments too, have started paying closer attention to the operations of platform companies in their jurisdictions. governments at national or supra-national scales now seek to intervene to restrict or reduce the power of platform companies, whether through anti-trust legislation, greater use of algorithmic governance techniques, and through resource [sic] to citizens’ data sovereignty (barns, 2020a, p. 198). the term regulation in the title of this paper and in its body is used in the wider sense of 'adjusting to a desired level or in accordance with principles and norms' (oxford english dictionary, n.d.), and includes but is not limited to the activities of state and private regulators australasian journal of information systems wildenauer 2020, vol 24, selected papers from the 2019 aice conference the shared responsibility model 2 where those norms may include rules, regulations and legislation. the need for regulation drives the need to identify levers of ethical influence in platform corporations and the locus of control upon which they may act. the concept of digital constitutionalism has emerged recently, which in its broadest sense seeks to establish norms and institutions that regulate and balance the powers of both state and private actors in the digital realm and protect the rights of users (celeste, 2018a, 2018b), and is an overarching concept explaining the advent of various forms of "normative counteractions" to changes to society caused by the digital (celeste, 2018a, p. 19). the related concept of digital governance has been the subject of much contemporary research in recent times (e.g. chen, pereira, & patel, 2020; flyverbom, deibert, & matten, 2019; mukhopadhyay & bouwman, 2019), with regulation and ethics being closely associated concepts required for good governance. just as regulation shapes governance of the digital sphere through compliance, ethics shapes governance through the process of ethical evaluation (floridi, 2018). both ethics and regulation can form the basis for governance goals (metcalf, moss, & boyd, 2019). there is also an obvious overlap between the domains of regulation and ethics, where levers of ethical influence applied to the correct locus of ethical control can be acted upon by hard (formal powers) or soft (principles-based or nudging) regulation to achieve the desired change in ethical behaviour. it is the identification of the ethical locus of control and available levers of influence upon them within a platform with which this paper concerns itself. this paper is of an exploratory nature, applying relevant practitioner knowledge and various theories in an inductive manner (i.e. starting from observations of the researcher) in an attempt to provide an explanation for the phenomenon of control over ethical behaviour in platforms. this is not an atypical approach for exploratory studies where well-developed theories may not exist, and results in this case in a model rather than fully developed theory (ngulube, mathipa, & gumbo, 2015). of necessity, this paper restricts itself to discussions of platform corporations founded or largely based in western liberal democracies that operate under the rule of law, and which have shareholders, directors, and a hierarchical management structure. this constraint excludes several very large platforms based in, for example, china such as tiktok, baidu, wechat and alibaba. many platforms are transnational in nature and operate across multiple jurisdictions, but typically take the corporate form available in their country of origin. accordingly, the platforms under consideration in this paper are geographically widespread, have a large user-base, and are most often of us origin and include google, facebook, airbnb, uber and their subsidiaries such as youtube and whatsapp. these platform corporations are also those considered by supporting literature and media articles; any exceptions have been noted in the text. this paper starts by considering the concept of platform corporations and the ways in which they may differ from other corporate entities, and examples of ethical misbehaviour by platforms are then provided. the question of exactly which system of ethics platforms should conform to is canvassed, and the problem of identifying the ethical locus of control and accompanying levers of influence is then introduced. once the problem has been defined, a number of ethical actors that may be identified as a locus of control are submitted for consideration in the main body of the paper. australasian journal of information systems wildenauer 2020, vol 24, selected papers from the 2019 aice conference the shared responsibility model 3 the shared responsibility model of ethical control is offered as a more realistic explanation for where and in which persons (legal or natural) ethical control is to be found in platforms, and therefore target levers of influence against which interventions may be conducted. a list of possible interventions is presented along with a list of those interventions suggested for regulators, boards, ceos, middle managers, individuals, and professional associations. the paper concludes with a cautionary example of unintended consequences that may follow from interventions to change ethical behaviour when these are targeted toward a single ethical actor in isolation, and with suggestions for further research. 1.1 platform companies minimally, platform companies or corporations (platforms) "help to make resources and participants more accessible to each other on an as-needed basis" (hagel, 2015, p. iv); in simpler terms, they are ecosystems. to achieve these aims, they need: a governance structure, including a set of protocols that determines who can participate, what roles they might play, how they might interact, and how disputes get resolved. an additional set of protocols or standards […] typically designed to facilitate connection, coordination, and collaboration (hagel, 2015, p. iv). an alternative definition is that "online platforms can be defined as socio-technical architectures that enable and steer interaction and communication between users through the collection, processing, and circulation of user data" (helberger, pierson, & poell, 2018). neither of these definitions capture the fact that platforms are not only a technology and the business that controls it, but enable creation (by third parties as well as themselves), distribution and consumption of content, software or services while claiming status as a neutral intermediary, rewarded but not liable for content (gillespie, 2010). neither do these definitions allude to platforms’ earlier conception as "de facto public spheres, spaces that encouraged discourse between free citizens" (barns, 2020b, p. 39), a fact now seemingly overlooked by the platforms themselves. significantly, while platforms may substitute for media, transport, or retail businesses, they define themselves as technology companies (barns, 2020b). in this way platforms can be seen to behave similarly to utilities, but without the attendant burden of regulation with which telecommunications providers, tv broadcasters, accommodation providers, taxi companies, or employers of casual labour must contend. regulation of platforms is also made difficult by their nature and global influence (gorwa, 2019a), with dominant players often collecting data for profit from individuals who provide this in return for some benefit (so, voluntarily) on a transnational basis, but typically with most of their operations and data storage in the us. online platforms have typically been secretive with respect to what is actually collected and the way in which their algorithms work. (helberger et al., 2018). platforms are complex systems of many moving parts, and in contrast to other corporations, social media platforms such as facebook and youtube have particularly problematic boundaries between what the platform itself may be responsible for (with its self-defined status as neutral intermediary), and what individual and corporate users who are contributors, users who are consumers, and other third parties such as advertisers and data-collectors have responsibility for. where those parties are all resident in different legal jurisdictions, governance and regulation can prove problematic (see for example digital watch observatory (n.d.) for examples of european cases regarding the activities of google, facebook and australasian journal of information systems wildenauer 2020, vol 24, selected papers from the 2019 aice conference the shared responsibility model 4 netflix). this division of responsibility is reflected in similar difficulties in determining boundaries and responsibilities for ethical behaviour (helberger et al., 2018). the platform technologies that are increasingly used to run our daily lives are in general controlled by large organizations and these large organizations all take the corporate form, are thus hierarchical in nature, and are usually governed by boards of directors elected by shareholders. it is important to note the fact that these platforms are companies, so subject to governance and regulatory forces when discussing the way in which their behaviours may be influenced (gorwa, 2019b). this paper proceeds by examining the general corporate model, with particularities of platforms noted where germane, to examine possible loci of ethical control in the organization and levers of influence that may be brought to bear on them. a range of interventions is then presented along with a caution that changing the (perceived) ethical behaviour of platforms is not without its own perils. 2 ethical transgressions condemnation of the less savoury activities and practices of platform corporations, and of the influence that the activities of these businesses may have on the lives of billions of people are most often characterised as corporate ethical transgressions. recent transgressions particular to platforms (as opposed to transgressions common to corporate entities) may be classified into three main types as follows. firstly, platforms have transgressed in terms of attempting to influence and shape laws in their favour and to the detriment of the public, and further, subverting democratic institutions. this has taken the form of behind the scenes lobbying of lawmakers and regulators, including threats to withhold investments in the country unless it passed platform-friendly laws (bloomberg, 2019; cadwalladr & campbell, 2019) and by accepting and assisting in the delivery of advertising by foreign actors to attempt to influence electoral outcomes (cadwalladr & graham-harrison, 2018). secondly, they have transgressed in the erosion of the right to privacy (acquisti, brandimarte, & loewenstein, 2015), and in norming this erosion (tsay-vogel, shanahan, & signorielli, 2018). there has been large scale collection of data without informed consent (dance, laforgia, & confessore, 2018; houser & voss, 2018), the sale of personal and sensitive data (mcgrath, 2019), and the surveillance of individuals and entire groups using tools such as online tracking (acar, van alsenoy, piessens, diaz, & preneel, 2015), geolocation (hayes, snow, & altuwayjiri, 2018), and facial recognition (murgia, 2019). thirdly, they have transgressed in terms of their manipulation of individuals and groups. this they have done through knowingly disseminating fake news; by conducting experiments on their user-base to manipulate emotions (flick, 2016); by designing-in mechanisms for the amplification of outrage, leading to riots and communal violence (farooq, 2018); and by allowing the dissemination of some forms of hate speech, while censoring others. 3 with which ethics should platforms then comply? compliance with the law is expected of platforms, as it is of any company. the transnational and online nature of these organizations means that there are multiple jurisdictions for them to contend with. in some jurisdictions where the platform has a physical presence, this is more australasian journal of information systems wildenauer 2020, vol 24, selected papers from the 2019 aice conference the shared responsibility model 5 visible, while in others where there is only a user-base this could prove more difficult to enforce. as advertisers, the platform and the users may all reside in different jurisdictions, an appeal to the law to restrain behaviour may prove to be complex. in addition, laws in some jurisdictions may not align with basic human rights, so imprecations to 'obey the law' may (rightly) be arguable on the grounds of human rights and ethics. it would seem that an approach based on ethical behaviour that imposes more stringent obligations on platform corporations is to be preferred for simplicity (the platform behaves in an ethical manner everywhere) and to avoid 'jurisdiction shopping' (seeking to be bound by only the most favourable legislation and regulation). the question of exactly which system of ethics should apply, often associated with reductionist ethical relativism, can be used as a tactic to stall or resist progress through misdirection. it is anyway a misleading question as ideological purity or perfection in ethical compliance is not as important as platform corporations making the attempt at better ethical behaviour. similarly, the detailed discussions engendered by ethical dilemmas and thought experiments have value, but companies would do well to leave aside efforts at seeking assurance of the correct course of action (if such a thing even exists) and instead pursue actions suggested by prima facie principles in order to follow a more ethical path. companies (and individuals) should for example: a) not harm others without good reason b) be honest c) keep promises d) be just e) work to repair any harms that they have caused f) strive to improve themselves (meyers, 2018). such a compact set of ethical principles may also fulfil an important additional role, as it has been suggested that given the plethora of ethical frameworks from which technology companies may now choose, a set of common criteria to evaluate these is essential (wagner, 2018). a further guiding principle for both regulators and civil society is that ethical frameworks should not be seen as a substitute for or alternative to, the upholding of fundamental human rights. rights uphold human dignity, and ought not to be understood as mere ethical imperatives (wagner, 2018), or worse still, presented as ethical aspirations for platform corporations. 4 levers and loci of ethical control and responsibility given the scale and impact of these ethical missteps, it would seem essential to understand who exactly should be held to account, where the locus of ethical control may reside, and which levers are available to attempt to promote better behaviour. it has been posited that online platforms are able to develop values beyond an aggregate of individual value sets within the company (johnson, 2017), and moore has suggested that, using mcintyre's conception of virtue, a virtuous organization is one that has a good purpose and slightly privileges the pursuit of excellence over success (moore, 2015). a more recent australasian journal of information systems wildenauer 2020, vol 24, selected papers from the 2019 aice conference the shared responsibility model 6 paper has however, suggested that corporations cannot be ethical agents, that is bear moral responsibility, as this concept does not even make sense (hühn, 2018). if this claim were true, it would suggest that any collective of persons is similarly incapable of ethical agency. many would argue that whatever the technical truth of such a claim, in practical terms people expect societies, states, governments and corporations to behave in an ethically acceptable manner. a way out of this philosophical dead-end can be found in the work of glauner (2017) who concurs that corporations are non-ethical entities (as they are non-psychological in nature), but offers the suggestion that corporate culture takes the place of conscience, providing guidance for corporate actions. so at least in the practical sense, the claim of absence for corporate ethical ability appears not be true, as it would mean not only could corporations not be expected to be just or fair, but also concepts of justice or fairness would not apply to them as subjects of other’s behaviour (because there would be no need to behave in an ethical manner toward a mere legal construct). if it is not true however, where does the locus of ethical control reside in a corporation? does the corporation as a legal person also have ethical responsibility, or should the owners, directors or executives be held accountable? in other words, it is important to identify where exactly the locus of ethical control in a corporation lies. it is also important to identify levers of ethical influence (barns, 2020a) in order to bring pressure to bear on the organization and thus intervene for improvement. the remainder of this paper seeks to explore these issues further. 5 ethical actors the set of ethical actors to take into account includes those external actors who have the ability to apply force or pressure to the platform or individuals within it via levers of influence, such as regulators, markets, customers, professional associations and wider society. this is in addition to those who are internal to the platform and in which the locus of ethical control may reside, that is, those who are involved in directing, managing, or working in the platform company. regulators are the external actors who would seem to have the most obvious power to impose changes in ethical behavioural standards on platforms, but the reality is more complex. the technology industry has been used to a self-regulatory or laissez-faire environment and actively resists via lobbying attempts at changing these arrangements where this is unfavourable to it (e.g. cadwalladr & campbell, 2019; molla, 2019). the somewhat fragmented regulatory landscape in jurisdictions such as australia, the us and europe, with their multiple regulators and sources of legislation, may not be conducive to simple solutions and may indeed promote efforts to seek favourable regulatory environments by platforms. one researcher in the area has claimed that regulators may actually prefer to avoid regulating, and instead turn to the softer option of ethical frameworks to help shape the self-regulatory projects of platforms (wagner, 2018). in australia there has recently been some interest displayed by regulators in platforms and in using existing powers e.g. under privacy laws (oaic, 2020) and consumer and competition laws (accc, 2019). in the us, the ftc seems to have existing consumer protection powers to regulate platforms but may not be keeping up with changes required for the era of digital business (mcsweeny, 2017). european regulators have so far shown the greatest willingness to take on large technology platforms with the general data protection regulation (gdpr) australasian journal of information systems wildenauer 2020, vol 24, selected papers from the 2019 aice conference the shared responsibility model 7 and formal inquiries, and the eu appears to have been the first jurisdiction to introduce legislation specifically aimed at online platforms (settembri, 2019). markets also have the power to punish ethically transgressing companies, but the magnitude of any punishment depends on how much information on the transgression is released to the public and whether the instance is taken as a sign of deeper problems, and on whether the company takes responsibility or instead assigns blame to specific individuals (carberry, engelen, & van essen, 2018). consumers of products and services can indicate their displeasure at transgressions although research has shown they may keep dealing with firms even when they are aware of unethical behaviour, especially when that service or product is important to them (haberstroh, orth, hoffmann, & brunk, 2017). this is illustrated by the continued flourishing of facebook after the cambridge analytica scandal. for platform users in particular, the switching costs engendered by network effects can make it undesirable or impossible for individuals to apply pressure to the company by seeking alternatives in response to privacy issues or ethical misbehaviour (bamberger & lobel, 2017). the subset of ethical actors within the platform company that would be likely loci of ethical control are considered below. 5.1 the platform corporation in response to pushes from civil society to more closely regulate the behaviour of platforms, platform companies often create, promote and lobby for, competing self-regulatory initiatives in order to escape formal regulation (gorwa, 2019a). as the company seeks to avoid regulation by state actors, it turn to codes of ethics, overseen by industry bodies or often by the company itself, or chosen from existing codes or frameworks, a practice known as ethics shopping (wagner, 2018). one recent research article has gone so far as to claim that ethics is "arguably the hottest product in silicon valley's hype cycle today" (metcalf et al., 2019, p. 449). as the dominant discourse in technology is that all problems can be solved through innovative solutions, it is not uncommon for silicon valley technology companies to address ethical issues by appointing an 'owner' of ethics across the business, who must however still work within the constraints of the business and its worldview (metcalf et al., 2019). indeed, ethics in platform companies would seem to be in danger of being entirely colonised by management. there is a risk of ethics becoming subsumed by bureaucratic processes such as compliance (metcalf et al., 2019) or perhaps worse, construed as an ethics-washing activity rather than an engagement with wider questions of purpose, integrity and values (wagner, 2018). the provision of ethical frameworks and checklists to, for example, individual engineers or 'ethics owners' allows the company to credibly assign blame for ethical problems to individuals when they arise rather than to the organization. technology culture is individualistic, with credit or blame accruing to the most 'deserving', so this blame-shifting is likely to be rarely challenged (metcalf et al., 2019). 5.2 the board of directors as ethical proxy shareholdings are often widely dispersed, and individual shareholders have little influence on the way a large corporation is run. large institutional investors are able to exert more influence, but they themselves are corporations, so assigning ethical control to them is unhelpful. however, while they are typically hands-off, shareholders do elect boards to make decisions on their behalf. australasian journal of information systems wildenauer 2020, vol 24, selected papers from the 2019 aice conference the shared responsibility model 8 the board of directors or equivalent is the ultimate governance organ of a corporation in the system of governance found in the us, uk, australia and most european countries. they are considered to be the guiding mind of a corporation and have been recognized as responsible for corporate culture, including ethical culture, e.g. by the hayne royal commission into the financial services industry in australia (atkins & charlton, 2019). corporate governance guidance and principles that mention the ethical obligations of boards and directors are now not uncommon (e.g. asx corporate governance council, 2019; frc, 2016), with the asx stock exchange, for example, requiring that australian listed corporations "instil a culture of acting lawfully, ethically and responsibly" (asx corporate governance council, 2019, principle 3). the digital era has increased the scope of the ethical responsibility of all directors, most visibly the protection of the privacy and data privacy of customers, and cybersecurity of the organization. this is particularly so for board directors of high-technology data-centred businesses such as platforms (windsor, 2020). it is however unclear whether that locus of ethical responsibility translates to the locus of ethical control. boards should, as a matter of course, be considering the ethical dimensions of their strategic decisions to fund new acquisitions, projects, in hiring etc. the question must be asked however, as to whether it is even possible to change ethical behaviour sufficiently for the better if the business model is based on such unethical precepts or foundations as the collection and monetisation of user data without real informed consent. in any case, many of the large platform corporations are us domiciled, silicon valley based companies (although not exclusively so) and may thus be dominated by executives who may prevail over boards due to large founder shareholdings. the possibility that it is in the founder and/or ceo group in which the locus of control resides then suggests itself. for example, facebook and twitter are both dominated by founders with large shareholdings, who must therefore bear some responsibility for setting the ethical ‘tone’, if not the day-to-day expression of ethical values in the company’s operations. 5.3 the ceo and senior executives senior executives and founders have a large say in how the company goes about its business, and if they have been with the company from inception, as is the case with many platform companies, have probably played the largest part in setting the culture (ethical or otherwise) of the organization. as we have seen, company ethical culture can substitute for conscience when evaluating corporate behaviour (glauner, 2017). the personal and professional ethics of senior executives, in particular that of ceos and cfos, has been found to have to have a large effect on corporate conduct (griffin, kruger, & maturana, 2017). griffin et al.'s (2017) research further noted that ethics tend not to be primarily situational, with a divide between private and corporate behaviour. these findings suggest at this level, executive ethics has a larger influence on corporate behaviour than does corporate culture on executive behaviour, so perhaps the locus of ethical control is then to be found in the senior executive team. as is the case with shareholders and boards, social and market pressure can be brought to bear on senior executives and in common with boards, regulatory and legislative pressure can also influence senior executive behaviour. while the board and senior executives make decisions on actions and policies that create ethical culture and guide ethical behaviour, and may even include a chief ethics officer as australasian journal of information systems wildenauer 2020, vol 24, selected papers from the 2019 aice conference the shared responsibility model 9 'ethics owner', it is usually middle management that decides how policies are actually implemented in the organization. 5.4 middle management middle management is often held to be the group actually in charge in an organization as they can block change to operating process and resist policy changes they deem to not be in their best interests (guth & macmillan, 1986), and because of numerical superiority and closeness to both the operations of the organization and to the employees providing a service or creating a product. due to length of time in the workforce, any ethical training may have been many years in the past in an undergraduate course for this cohort. in the teaching experience of the author, middle managers are well represented in mba courses, especially those courses with experience or management pre-requisites, so ethics subjects at this level could be influential if delivered and constructed correctly. see below in the ' individual contributor' section for a further discussion on ethics training. while not as focussed as an individual contributor, middle managers can however have a somewhat narrow view of the company and its stakeholders. this, and their comparatively large numbers, means that it is less likely that agreement on what exactly an ethical response should be in a given situation would be reached, which in return could result in suboptimal ethical decision-making for the wider picture. 5.5 individual contributors the final group to consider are individual employees and their personal and professional ethics. interestingly, there exists in some platform organizations a very particular type of individual who may be an executive (as chief ethics officer for example) or an individual contributor, who is considered the 'ethics owner' and carries responsibility for ethics across the organization in terms of integrating ethics into its activities (metcalf et al., 2019). the role of the 'ethics owner' is also to take external perspectives and critiques of the company's ethical stances and translate and transform them into ideas that make sense within the company's own discourse and culture. since the resulting decisions and actions are viewed through the prism of existing logic, there is an obvious danger that these may fail to respond adequately to the original critiques or fully understand external perspectives (metcalf et al., 2019). leaving aside this special case, assigning ethical control at the individual level can be problematic as there are many individuals who are unable to easily recognize that a particular ethical principal is at play in any given situation, and this lack of awareness has been shown to be a large factor in making unethical or ethics-free decisions (kim & loewenstein, 2020). even when ethics training has been provided, usually only the making of ethical determinations in response to stimuli to do so, what kim and loewenstein term prompted ethical awareness, has been assessed. what is required is spontaneous awareness of relevant ethical characteristics in decision-making settings when not prompted to do so and an understanding of any relevant ethical principles that may apply to the situation (kim & loewenstein, 2020). individual contributors have the opportunity and the obligation to use their own moral compass to guide their actions, with professional ethics as an aid to ethical decision-making. codes of ethics are a feature of nearly all professional associations, and often heavily promoted by them. for ict workers, guidance is provided by acm (gotterbarn et al., 2018) and ieee (ieee, n.d.) in the us, and in australia, the acs has codes of ethics and conduct (acs, 2014a, australasian journal of information systems wildenauer 2020, vol 24, selected papers from the 2019 aice conference the shared responsibility model 10 2014b), both currently under review by its ethics committee. in addition to being affected by guidance from professional bodies, and in common with middle managers, individuals may be able to be nudged toward ethically aware behaviour by the provision of training and education in ethics by for example, their companies or in mba courses (martinov-bennie & mladenovic, 2015), as long as these are carefully constructed to evince the desired behaviours (kim & loewenstein, 2020). ethics training has however, been found to be contingent, in that its timing and the existing company culture seem to have an impact on efficacy (kowaleski, sutherland, & vetter, 2019). kowaleski et al. (2019) found that in a financial services setting at least, ethics training has a lesser effect on those already engaging in ethical misconduct and on those operating in organizations with an ethically dubious culture. so, ethics training at undergraduate level would seem to be important in order to intervene before the start of a career, but it is insufficient; an ethically sound organizational culture is also required. individuals also need to continue the development of ethical sensitivities throughout their career, and not just rely on initial training (felton & sims, 2005). individual contributors are thus responsible for their own ethical development, and for their contribution to the company's ethical culture. the individual contributor can make a choice in every situation, and ultimately withdraw their labour if required. indeed, the departure of those employees with more ethical training from a firm has been found to be a predictor of future misconduct in financial services (kowaleski et al., 2019). while these choices are often constrained by the realities of organizational life such as career damage and potential loss of income, for those creating technology for silicon valley's platform companies such power differentials may not be as relevant due to the current abundance of employment for technical employees. most importantly, whether or not corporation can be moral agents, individuals are. while individuals may not move facebook's or uber's ethical stance easily, they can help ensure that the next large platform corporation may be better behaved. 6 the shared responsibility model as platforms are now involved in activities previously thought to be in the public domain, such as journalism, education, and even civic engagement, it follows that they are important in the fulfilment of public values and achievement of government policy objectives such as freedom of expression, equality, and diversity (helberger et al., 2018). it is the opinion of helberger et al. (2018) that platform-based activities cannot be used to fulfil public values or policy objectives unless responsibility for success is shared between the platform and its users and state institutions. analogously, for the public value of good ethical behaviour on the part of platforms, the locus of control in platform corporations is thus to be found distributed across the domains of shareholders, boards, executive teams, middle management, and individual contributors in addition to the platform corporation itself, with responsibility shared between all of those actors plus regulators, users, public and private institutions, and wider society. given inequalities of power and resources, there can be no expectation of sharing responsibility equally, but where an actor has the ability (capacity, power and resources) to contribute to change for the better, there is a general societal expectation that they ought to do so (helberger et al., 2018). australasian journal of information systems wildenauer 2020, vol 24, selected papers from the 2019 aice conference the shared responsibility model 11 according to johnson (2017), the sharing of ethical responsibilities and the boundaries between roles responsible within a platform corporation are at best nebulous. in any case, given that there exists no one locus of control, no one lever of influence would seem sufficient to ensure improvement in ethical behaviour. at the minimum, controlling ethical behaviour in these corporations may require regulators to regulate, shareholders and boards to set policies for ethical behaviour in response to public expectations and regulatory requirements, for middle management to guide, and for individual employees to take responsibility for limiting transgressions by making ethical choices. simultaneous or at least aligned interventions acting on multiple levers would seem necessary to exert sufficient pressure on platforms to improve ethical behaviour, even though some of these interventions may seem at first glance to be mutually exclusive or paradoxical. common in organizational life, such apparently paradoxical tensions arise from "perceptions of opposing and interwoven elements" (sundaramurthy & lewis, 2003, p. 397) that have been falsely dichotomised, hiding "the simultaneity of conflicting truths" (lewis, 2000, p. 761). in platform companies there needs to be more leadership from the ceo and board promoting and enforcing ethical behaviour (push), and more freedom and support for individuals to take ethical responsibility and make ethical decisions while trying to get the company to follow their lead. regulators need to lean harder and apply hard regulation, even while companies are behaving better in response to soft power. platforms need to both promote free speech and other social goods, and prevent harm arising from them. the locus of ethical control / levers of influence figure 1 levers of corporate ethical behaviour australasian journal of information systems wildenauer 2020, vol 24, selected papers from the 2019 aice conference the shared responsibility model 12 platforms both govern and are governed (gorwa, 2019b), and it is conceivable that both could be in the form of co-governance, in which governance is shared between the platform and other actors. the question asked by gorwa (2019a, p. 42), "why not try and make platform governance more participatory and democratic rather than purely technocratic?", suggested just such a model in order to make platform operations as transparent as possible and governance more representative of the diverse user base of platforms. as governance overlaps with ethics in the corporate context, a shared model of governance could support a shared model of ethical responsibility. at its most radical, co-governance could go past the merely participatory, and take the form of transnational 'deliberative assemblies' (carson & elstub, 2019) that would deliberate deeply on ethical matters and suggest courses of action for platforms and regulators. these assemblies would be a form of mini-publics (escobar & elstub, 2017), possibly chosen by sortition, to provide the representative input of wider society. 7 suggested interventions interventions involve the application of some type of pressure or obligation on the loci of ethical control via levers of influence. not all interventions are feasible for reasons of jurisdiction, public will, financial implications, or regulatory environment etc. for example, the more radical possibilities suggested by the shared responsibility model would require transnational (or at least, supra national) governance due to the transnational form of the platforms themselves. several less contentious interventions are recommended below, with a more complete list to be found in table 1 following. 1. boards and execs need to reward ethical behaviour and not reward unethical behaviour (kpis, bonuses, etc.) and include clawback provisions in executive and director remuneration agreements. 2. boards should require annual ethics compliance attestations from the ceo, sponsor the rollout of general ethics training across the organization and include completion numbers and level of positive responses to ethical climate employee surveys in ceo kpis. 3. ceos should replicate clawback provisions for any employees receiving short or long-term incentive payments 4. ceos should replicate ethics training outcomes and level of positive responses to ethical climate employee surveys for all executive and manager kpis 5. professional associations both technical (e.g. the acs in australia) and directororiented such as the aicd in australia, the iod in the uk and the nacd in the us should mandate an ethical training component of cpd hours that is based on evidence. 6. regulators in the home jurisdiction(s) of platform corporations should increase regulatory oversight of platforms. this can be soft regulation expressed as 'governance principles' or listing rules in addition to hard regulation, and includes such items as requiring annual ethics compliance attestations from boards and/or ceos, reviews of consumer complaints with concomitant legal action, mandating remuneration clawback provisions for executives and directors where there has been ethical misbehaviour. australasian journal of information systems wildenauer 2020, vol 24, selected papers from the 2019 aice conference the shared responsibility model 13 7. business and engineering/it tertiary institutions should include and improve ethics training for all students at both undergraduate and postgraduate levels, paying particular attention to undergraduate technology, analytics and marketing courses and to mba courses for aspiring senior managers. as can be appreciated, not all interventions are available to state and private regulators outside of the primary jurisdictions of the platforms, and some interventions would only partially address available levers of influence. professional associations and tertiary institutions outside of the us would likely only likely be able to influence local employees of facebook or google for example. this presses the case for a more cooperative transnational approach not only by regulators but professional associations and universities to achieve a common objective. locus levers possible interventions shareholders social forces, market forces civil society campaigns* investor campaigns company social forces, market forces, governance norms, regulatory compliance civil society campaigns* compliance with listing rules & corp gov principles compliance with securities regulator rules threat of increased regulation* increased regulation* co-regulation replaces self-regulation board of directors social forces, market forces, governance norms, regulatory compliance, professional ethics, personal ethics compliance with listing rules & corp gov principles compliance with securities regulator rules mandate director professional assoc. ethics training for members require annual ethics attestation to regulator* require annual ethics attestation to board from ceo include sti/lti clawback provisions for unethical behaviour in ceo contracts include rollout of ethics training and level of +ve responses to ethical climate questions in employee surveys in exec kpis ceo/executive governance norms, regulatory compliance, professional ethics, personal ethics include sti/lti clawback provisions for unethical behaviour in exec contracts include ethics training in mba/leadership courses include rollout of ethics training and level of +ve responses to ethical climate questions in employee surveys in exec kpis middle management ethics training, professional ethics, personal ethics include ethics training in mba courses* require professional associations to mandate ethics continuing training as part of cpd* require completion of ethics training and level of +ve responses to ethical climate questions in employee surveys in kpis individual contributors ethics training, professional ethics, personal ethics include ethics training in ug technology courses and in mba courses* regulator to require professional associations to mandate ethics continuing training as part of cpd* *also possible outside of primary location(s) of platform corporations table 1 loci, levers & intervention australasian journal of information systems wildenauer 2020, vol 24, selected papers from the 2019 aice conference the shared responsibility model 14 8 unintended consequences applying pressure via levers (seeking to exert ethical influence) may however result in unintended consequences, particularly where this is applied to a locus of ethical control in isolation or to multiple loci in an uncoordinated manner. public calls for more ethical behaviour by facebook, twitter and others in moderating content may require society to surrender something in return, such as submitting to de facto private government; i.e. platforms governing a population by means of deciding what is possible and visible and permitted, while their own decisions are impenetrably opaque and without recourse (suzor, 2019). more tangibly and more disturbingly, increased content moderation by facebook in particular has also resulted in the employees hired to moderate it being exposed to truly awful text and images, including sexual exploitation of children and murder, causing permanent mental and emotional damage (newton, 2019). many of these employees are in lower-pay countries such as the philippines (franks, 2018), and this shifting of the emotional and mental burden offshore would seem to be a very serious ethical transgression. it is not only an instrumental use of people with less power; it allows this while appearing to act more ethically at home. the ethical posture of the platform corporation itself is thus at odds with decisions made by its executives and overseen by managers within the company. a cautionary note should be sounded whenever organizations previously found to have operated in an ethically dubious fashion announce sudden and major improvements. 9 limitations and future research one obvious limitation of this paper is that the loci of ethical control and levers of influence upon them, as well as the resultant recommendations for intervention have not been considered for platforms that have their origins outside of western liberal democracies. while this means that several large platforms such as wechat and baidu were not considered, information about their operations and any ethical transgressions are not in any case as readily available to researchers, or as widely reported by media. given the degree of state-actor involvement, and the fact of limited uptake in wider geographies (albeit with large user bases), it is in any case possible that entirely different conclusions may be reached. limiting the scope of the platform corporations considered also sidesteps arguments of ethical relativity or universality, which are beyond the scope of this paper, but may be fruitfully explored elsewhere. there is a deal of opportunity for future work following this paper, a considerable portion of which may be evaluative or empirical in nature. one possible project is the development of an instrument to serve as an acceptable set of common ethical criteria for the purposes of evaluating the quality of ethical commitments made by platform corporations. several evaluative research projects also suggest themselves in response to the list of recommended interventions in this paper. these include evaluations of interventions on ethical behaviour in a corporate setting such as effect on ethical awareness after the completion of prompted versus spontaneous-awareness oriented ethics training in mba courses and undergraduate technology courses. a similar project is the evaluation of ethical awareness after mandated ethics training in cpd for directors or members of technical professional associations such as the acs, engineers australia, ieee, acm etc. australasian journal of information systems wildenauer 2020, vol 24, selected papers from the 2019 aice conference the shared responsibility model 15 there are a number of other directions that future research could profitably explore. the first of these is the question of the acceptability of a shared model of ethical responsibility to boards, executives, regulators and civil society. without some acceptance, any project to change behaviours will likely prove unsuccessful. in the australian context, there are also questions to be asked around the readiness, power, and ability of boards to take ethical control given the shared responsibilities articulated in this paper, despite it apparently being expected of them by institutional actors such as the hayne royal commission (atkins & charlton, 2019), the aicd (aicd, n.d.) and the asx (asx corporate governance council, 2019). corollary questions exist around board perceptions of their power, means, and desire to do so, and around boards’ understanding of the issues. finally, there seem to be interesting questions around innovation worthy of further exploration. for example, whether an innovative approach using regulatory technology (regtech) could create an environment for better ethical outcomes at board and executive level. the framing of this research question could of course itself be a fallback into the hegemonic technological logic that all social problems are solvable by means of innovative technology (metcalf et al., 2019), an exploration of which tendency even in the domain of digital or technology ethics, is also worthy of further study. a research project of a more abstract nature could also investigate whether the ‘move fast and break things’ ethics favoured by many innovative technology companies can be reconciled with ethically responsible technology such as that promoted by beard and longstaff (2018) or against standards developed by the ieee (koene, smith, egawa, mandalh, & hatada, 2018) and other bodies, and if so, how this might be achieved. 10 conclusion there have been calls from civil society, academia, and governments for transnational platform corporations to be better held to account for their ethical transgressions. as has been demonstrated, attempts to change the less desirable ethical behaviours of platforms cannot necessarily rely on applying leverage to a single locus of ethical control, as multiple loci exist, and multiple loci may need to be engaged to effect genuine change. the shared model of responsibility seems to provide a more pragmatic view of the possibilities for action than relying on any one of companies, boards, executives or workers. the understanding of the requirement for multiple simultaneous actions against multiple loci of ethical control may free each of regulators, boards, executives, and civil society from the perceived obligation to enact complete strategies and result in greater transnational cooperation between groups. possible interventions are many, and should be prosecuted widely, but not every intervention is required from every actor, and unintended consequences are possible. the shared model has implications for state-actor regulators outside of the primary jurisdiction(s) of the platform corporation, as their reach (and hence ability to make meaningful change for the better) is limited without the cooperation of agencies in other jurisdictions. one final implication of the shared model of responsibility is that available hard regulation (e.g. privacy and consumer protection statutes) is to be preferred to soft regulation such as nudges and attempts at ethical influence in non-domicile jurisdictions (i.e. those in which the company does business, but has no or little infrastructure or organizational footprint), as these australasian journal of information systems wildenauer 2020, vol 24, selected papers from the 2019 aice conference the shared responsibility model 16 latter will likely be unsuccessful given the range of loci of ethical control, many which exist only in other jurisdictions for transnational platform corporations. references acar, g., van alsenoy, b., piessens, f., diaz, c., & preneel, b. (2015). facebook tracking through social plug-ins. belgian privacy commission, ver, 1, 2. accc. (2019). digital platforms inquiry final report. retrieved from https://www.accc.gov.au/publications/digital-platforms-inquiry-final-report acquisti, a., brandimarte, l., & loewenstein, g. (2015). privacy and human behavior in the age of information. science, 347(6221), 509-514. acs. (2014a). acs code of conduct . retrieved from https://www.acs.org.au/content/dam/acs/rules -and-regulations/code-ofprofessional-conduct_v2.1.pdf acs. (2014b). acs code of ethics. retrieved from https://www.acs.org.au/content/dam/acs/acsdocuments/code-of-ethics.pdf aicd. (n.d.). role of the board. director tools governance relations. retrieved from https://aicd.companydirectors.com.au/-/media/cd2/resources/directorresources/director-tools/pdf/05446-3-11-mem-director-gr-role-of-board_a4-v3.ashx asx corporate governance council. (2019). corporate governance principles and recommendations. retrieved from https://www.asx.com.au/documents/regulation/cgcprinciples-and-recommendations-fourth-edn.pdf atkins, s., & charlton, p. (2019). chartered secretary: the banking royal commission final report: culture and governance implications. governance directions, 71(2), 65. bamberger, k. a., & lobel, o. (2017). platform market power. berkeley technology law journal, 32(3), 1051-1092. doi:10.15779/z38n00zt38 barns, s. (2020a). concluding reflections: an ethics of public value in an era of platform scale. in platform urbanism: negotiating platform ecosystems in connected cities (pp. 193-201). singapore: springer singapore. barns, s. (2020b). when the web became platform. in platform urbanism: negotiating platform ecosystems in connected cities (pp. 35-52). singapore: springer singapore. beard, m., & longstaff, s. (2018). ethical principles for technology. retrieved from sydney: https://ethics.org.au/ethical-by-design/ bloomberg. (2019). google spent $21 million lobbying an increasingly tech-nervous washington in 2018. retrieved from http://fortune.com/2019/01/22/google-lobbying-washington-21million-dollars-2018-privacy-tariffs-trade-immigration-sundar-pichai/ cadwalladr, c., & campbell, d. (2019). revealed: facebook’s global lobbying against data privacy laws the guardian retrieved from https://www.theguardian.com/technology/2019/mar/02/facebook-global-lobbyingcampaign-against-data-privacy-laws-investment cadwalladr, c., & graham-harrison, e. (2018). the cambridge analytica files. the guardian, 21, 6-7. australasian journal of information systems wildenauer 2020, vol 24, selected papers from the 2019 aice conference the shared responsibility model 17 carberry, e. j., engelen, p.-j., & van essen, m. (2018). which firms get punished for unethical behavior? explaining variation in stock market reactions to corporate misconduct. business ethics quarterly, 28(2), 119-151. carson, l., & elstub, s. (2019). comparing participatory and deliberative democracy. retrieved from online: https://www.newdemocracy.com.au/research-and-development-notes/ celeste, e. (2018a). digital constitutionalism: mapping the constitutional response to digital technology's challenges. hiig discussion paper series, 2018(02). doi:https://dx.doi.org/10.2139/ssrn.3219905 celeste, e. (2018b). what is digital constitutionalism? retrieved from https://www.hiig.de/en/what-is-digital-constitutionalism/ chen, y., pereira, i., & patel, p. (2020). decentralized governance of digital platforms. journal of management, forthcoming. dance, g. j., laforgia, m., & confessore, n. (2018). as facebook raised a privacy wall, it carved an opening for tech giants. the new york times. digital watch observatory. (n.d.). jurisdiction. issues. retrieved from https://dig.watch/issues/jurisdiction edelman. (2020). edelman trust barometer global report 2020. retrieved from https://www.edelman.com/trustbarometer elvery, s. [@drzax]. (2018, 25/06/2018). wow, this is really bad. medical appointment booking app healthengine sharing clients' personal information with lawyers [tweet]. retrieved from https://twitter.com/drzax/status/1011003833428762624 escobar, o., & elstub, s. (2017). forms of mini-publics: an introduction to deliberative innovations in democratic practice. new democracy. research and development note, 4. farooq, g. (2018). politics of fake news: how whatsapp became a potent propaganda tool in india. media watch, 9(1), 106-117. doi:10.15655/mw/2018/v9i1/49279 felton, e. l., & sims, r. r. (2005). teaching business ethics: targeted outputs. journal of business ethics, 60(4), 377-391. flick, c. (2016). informed consent and the facebook emotional manipulation study. research ethics, 12(1), 14-28. floridi, l. (2018). soft ethics and the governance of the digital. philosophy & technology, 31(1), 1-8. flyverbom, m., deibert, r., & matten, d. (2019). the governance of digital technology, big data, and the internet: new roles and responsibilities for business. business & society, 58(1), 3-19. franks, m. a. (2018). justice beyond dispute. harvard law review, 131(5), 1374-1397. retrieved from http://wallaby.vu.edu.au:2048/login?url=https://search.ebscohost.com/login.aspx?direct =true&db=bth&an=128431900&site=ehost-live australasian journal of information systems wildenauer 2020, vol 24, selected papers from the 2019 aice conference the shared responsibility model 18 frc. (2016). the uk corporate governance code . retrieved from https://www.frc.org.uk/getattachment/ca7e94c4 -b9a9-49e2-a824ad76a322873c/uk-corporate-governance-code-april-2016.pdf gillespie, t. (2010). the politics of ‘platforms’. new media & society, 12(3), 347-364. glauner, f. (2017). ethics, values and corporate cultures: a wittgensteinian approach in understanding corporate action. in n. capaldi, s. o. idowu, & r. schmidpeter (eds.), dimensional corporate governance: an inclusive approach (pp. 49-59). cham: springer international publishing. gorwa, r. (2019a). regulating them softly. models for platform governance. retrieved from https://www.cigionline.org/sites/default/files/documents/platform-govweb_version.pdf#page=41 gorwa, r. (2019b). what is platform governance? information, communication & society, 22(6), 854-871. gotterbarn, d. w., brinkman, b., flick, c., kirkpatrick, m. s., miller, k., vazansky, k., & wolf, m. j. (2018). acm code of ethics and professional conduct. retrieved from https://www.acm.org/code-of-ethics griffin, j. m., kruger, s., & maturana, g. (2017). do personal ethics influence corporate ethics? [available at ssrn 2745062]. pnas, 116(33), 16268-16273. doi:https://dx.doi.org/10.2139/ssrn.2745062 guth, w. d., & macmillan, i. c. (1986). strategy implementation versus middle management self‐interest. strategic management journal, 7(4), 313-327. haberstroh, k., orth, u. r., hoffmann, s., & brunk, b. (2017). consumer response to unethical corporate behavior: a re-examination and extension of the moral decoupling model. journal of business ethics, 140(1), 161-173. hagel, j. (2015). the power of platforms. retrieved from https://www2.deloitte.com/content/dam/deloitte/za/documents/strategy/za_the_pow er_of_platforms.pdf hayes, d. r., snow, c., & altuwayjiri, s. (2018). a dynamic and static analysis of the uber mobile application from a privacy perspective. journal of information systems applied research. helberger, n., pierson, j., & poell, t. (2018). governing online platforms: from contested to cooperative responsibility. the information society, 34(1), 1-14. doi:10.1080/01972243.2017.1391913 houser, k. a., & voss, w. g. (2018). gdpr: the end of google and facebook or a new paradigm in data privacy. rich. jl & tech., 25, 1. hubbard, s. (2019). the case for why big tech is violating antitrust laws. cnn business perspectives. retrieved from https://edition.cnn.com/2019/01/02/perspectives/big-techfacebook-google-amazon-microsoft-antitrust/index.html hühn, m. p. (2018). csr-the cuckoo’s egg in the business ethics nest. humanistic management journal, 3(2), 279-298. australasian journal of information systems wildenauer 2020, vol 24, selected papers from the 2019 aice conference the shared responsibility model 19 ieee. (n.d.). ieee code of ethics. retrieved from https://www.ieee.org/about/corporate/governance/p7-8.html ingram, m. (2014). giants behaving badly: google, facebook and amazon show us the downside of monopolies and black-box algorithms. retrieved from https://gigaom.com/2014/05/23/giants-behaving-badly-google-facebook-and-amazonshow-us-the-downside-of-monopolies-and-black-box-algorithms/ johnson, b. g. (2017). speech, harm, and the duties of digital intermediaries: conceptualizing platform ethics. journal of media ethics, 32(1), 16-27. kim, j., & loewenstein, j. (2020). analogical encoding fosters ethical decision making because improved knowledge of ethical principles increases moral awareness. journal of business ethics, 1-18. king, s. (2018). have 'big tech' platforms got too much power? online retrieved from https://www.pc.gov.au/news-media/speeches/big-tech-power koene, a., smith, a. l., egawa, t., mandalh, s., & hatada, y. (2018). ieee p70xx, establishing standards for ethical technology. proceedings of kdd, excel london uk, august, 2018 (kdd'18). kowaleski, z. t., sutherland, a., & vetter, f. (2019). can ethics be taught? evidence from securities exams and investment adviser misconduct. available from ssrn elibrary. doi:http://dx.doi.org/10.2139/ssrn.3457588 lewis, m. w. (2000). exploring paradox: toward a more comprehensive guide. academy of management review, 25(4), 760-776. lownsbrough, h. (2017). uber’s practices are morally unacceptable – but a boycott won’t help. the guardian retrieved from https://www.theguardian.com/commentisfree/2017/jul/13/uber-practices-companylondon-consumer-sumofus martinov-bennie, n., & mladenovic, r. (2015). investigation of the impact of an ethical framework and an integrated ethics education on accounting students’ ethical sensitivity and judgment. journal of business ethics, 127(1), 189-203. mcgrath, p. (2019). healthengine, medical booking app, facing multi-million-dollar fines for selling patient data. retrieved from https://www.abc.net.au/news/2019-08-08/healthenginefacing-massive-fine-after-abc-investigation/11394564 mcsweeny, t. (2017). ftc 2.0: keeping pace with online platforms. berkeley technology law journal, 32(3), 1027-1050. doi:10.15779/z38736m23n metcalf, j., moss, e., & boyd, d. (2019). owning ethics: corporate logics, silicon valley, and the institutionalization of ethics. social research: an international quarterly, 86(2), 449476. meyers, c. (2018). the professional ethics toolkit: john wiley & sons. molla, r. (2019). google, amazon, and facebook all spent record amounts last year lobbying the us government. recode. retrieved from https://www.vox.com/2019/1/23/18194328/google-amazon-facebook-lobby-record australasian journal of information systems wildenauer 2020, vol 24, selected papers from the 2019 aice conference the shared responsibility model 20 moore, g. (2015). corporate character, corporate virtues. business ethics: a european review, 24, s99-s114. mukhopadhyay, s., & bouwman, h. (2019). orchestration and governance in digital platform ecosystems: a literature review and trends. digital policy, regulation and governance. murgia, m. (2019). microsoft quietly deletes largest public face recognition data set. retrieved from https://www.ft.com/content/7d3e0d6a-87a0-11e9-a028-86cea8523dc2 newton, c. (2019). the trauma floor: the secret lives of facebook moderators in america. retrieved from https://www.theverge.com/2019/2/25/18229714/cognizant-facebook-contentmoderator-interviews-trauma-working-conditions-arizona ngulube, p., mathipa, e., & gumbo, m. t. (2015). theoretical and conceptual frameworks in the social and management sciences. in e. mathipa & m. t. gumbo (eds.), addressing research challenges: making headway in developing researchers. (pp. 43-66). oaic. (2020). commissioner launches federal court action against facebook [press release]. retrieved from https://www.oaic.gov.au/updates/news-and-media/commissionerlaunches-federal-court-action-against-facebook oxford english dictionary. (n.d.). "regulate, v.": oxford university press. settembri, p. (2019). 'if i can make it there, i'll make it anywhere'. online platforms meet the european union. online platforms meet the european union (june 2019). robert schuman centre for advanced studies research paper no. rscas, 35. smith, a. (2018). public attitudes toward technology companies. retrieved from online: https://www.pewinternet.org/2018/06/28/public-attitudes-toward-technologycompanies/ sundaramurthy, c., & lewis, m. (2003). control and collaboration: paradoxes of governance. academy of management review, 28(3), 397-415. suzor, n. p. (2019). lawless: the secret rules that govern our digital lives: cambridge university press. tsay-vogel, m., shanahan, j., & signorielli, n. (2018). social media cultivating perceptions of privacy: a 5-year analysis of privacy attitudes and self-disclosure behaviors among facebook users. new media & society, 20(1), 141-161. wagner, b. (2018). ethics as an escape from regulation: from ethics-washing to ethicsshopping. in e. bayamlioğlu, i. baraliuc, l. janssens, & m. hildebrandt (eds.), being profiled: cogitas ergo sum. 10 years of ‘profiling the european citizen ‘ (pp. 84-90). amsterdam: amsterdam university press, 2018. windsor, d. (2020). ethical values and responsibilities of directors in the digital era. in challenges and opportunities of corporate governance transformation in the digital era (pp. 91-116). hershey, pa, usa: igi global. abbreviations aicd australian institute of company directors ceo chief executive officer australasian journal of information systems wildenauer 2020, vol 24, selected papers from the 2019 aice conference the shared responsibility model 21 cpd continuing professional development iod institute of directors (uk) kpi key performance indicators lti long term incentives nacd national association of company directors (us) sti short term incentives copyright: © 2020 wildenauer. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v24i0.2797 http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems schoch 2023, vol 27, research article technostress and remote working performance 1 the interplay of challenge-hindrance-appraisal and selfefficacy: technostress and remote working performance manfred schoch university of augsburg & fraunhofer fit, germany manfred.schoch@fim-rc.de abstract measures to contain the covid-19 pandemic have caused many employees to work from home; a novel situation in which individuals used information systems (is) more intensively to stay in touch with co-workers. this novel is use situation affected individuals differently and resulted in both positive and negative outcomes. recent calls for research advocate for clarification regarding the conceptualisation of appraisal, which explains different individual responses to objectively equal environments. in particular, challenge-hindrance-research does not differentiate between primary and secondary appraisal. therefore, it remains unclear how individual capability beliefs, such as self-efficacy, affect challenge and hindrance is use appraisal. we conduct an empirical study with 1,553 german employees to investigate these relationships and the positive and negative outcomes during the covid-19 pandemic. we find that challenge and hindrance is use appraisal, and remote working self-efficacy are interconnected, yet different constructs. we find that self-efficacy is related to challenge is use appraisal, rather than hindrance is use appraisal. further, challenge is use appraisal is a driver for performance in a remote working environment. we conclude that there are stressful aspects of is use that are not influenced by an individual’s belief in their abilities. our study emphasises the importance of remote working self-efficacy and is use appraisal to mitigate techno-distress and increase performance during remote work. keywords: is use, cognitive appraisal, self-efficacy, remote work, challenge-hindrance. 1 introduction to contain the covid-19 pandemic, many organisations have advised their employees to work from home. studies surveying the german workforce indicate that more than 25% worked from home during the height of the first wave of the pandemic in march 2020 (möhring et al., 2020). this number was likely higher for knowledge-intense industries. to maintain communication and collaboration between employees in this physically distanced work environment, many organisations and employees reverted to digital communication and collaboration tools, such as microsoft teams or zoom. as a result, sales and usage time of such tools grew exponentially (spataro, 2020). the physical distancing measures came with many potential psychological stress and strain sources, such as reduced social contacts and increased family demands from a lack of childcare options. the use of digital technologies was both a blessing and a curse for many in this time. while it enabled individuals to stay in touch with co-workers, family, and friends, it also confronted many with new it issues. such issues include erecting and maintaining remote working infrastructure, using new technologies, or using existing technologies for new purposes. such novel circumstances are a potential source of stress for some (ellsworth & scherer, 2003). early scientific contributions have investigated the effect of the physical distancing measures, and the increased is use on psychological health (e.g. vaziri, casper, australasian journal of information systems schoch 2023, vol 27, research article technostress and remote working performance 2 wayne, & matthews, 2020). yet, individual differences in the perception of such stressful situations and ways to mitigate the adverse consequences are important avenues for further research. covid-19 came at a time when researchers have begun to recognise that technostress research has been primarily concerned with the negative side of stress (tarafdar, cooper, & stich, 2019). few contributions have already investigated the positive side of technostress. these studies have advanced our knowledge of technostress through models that differentiate between challenge and hindrance stressors (benlian, 2020; califf, sarker, & sarker, 2020). this is congruent with stress research from the realm of occupational psychology. many studies have similarly differentiated between challenge and hindrance stressors (cavanaugh, boswell, roehling, & boudreau, 2000). yet, organisational psychologists have recently suggested that analyses that recognise individual differences in appraisal of such situations may be fruitful. this emphasises diversity and the study of different individual reactions to stress. underlying is an individual assessment that explains different individual responses in objectively equal environments (krohne, 2001). in the context of is use, appraisal may include the evaluation of is as a challenging or motivating factor on the one hand or a threat and disturbing factor on the other (tarafdar et al., 2019). a recent call for research has proposed that low technology self-efficacy could be a driver of threat appraisals in the context of is use (tarafdar et al., 2019). congruently, seminal work from psychology has found that self-efficacy and appraisal are different phenomena that affect each other (jerusalem & schwarzer, 2010). while self-efficacy is a characteristic of the individual that builds on prior personal accomplishments and experiences, appraisal may vary between situations and within situations over time. this is because appraisal is a cognition that may change continuously as an individual interacts with the environment (jerusalem & schwarzer, 2010). congruently, is research has identified perceived control over it as an important factor in stress (tams, ahuja, thatcher, & grover, 2020). it control is considered an element of secondary appraisal (beaudry & pinsonneault, 2005). yet, much empirical research on the bright and dark side of is use has built on challenge-hindrance-research (e.g., califf et al., 2020; maier et al., 2021). the associated conceptualisation “does not differentiate the primary and secondary appraisal process” (lepine, zhang, crawford, & rich, 2016, p. 1052). in this study, we investigate the relationship between remote working self-efficacy and challenge/hindrance is use appraisal, the impact that individual is use appraisal has on remote working during the covid-19 pandemic, which came with increased use of digital communication technology. we further conclude how further research on is can profit from these findings. thus, the paper at hand investigates the following research question: what relationship do individual challenge/hindrance appraisal and self-efficacy have with technodistress and performance in times of remote work? the theoretical implications of this work are threefold: first, we advance the current knowledge regarding the relationship between self-efficacy and is use appraisal. we show that self-efficacy affects challenge is use appraisal rather than hindrance is use appraisal. this suggests that hindrance is use appraisal is not related to the individuals’ resources and, thus, has a different root that warrants further research. second, we show a positive relationship between the two antecedents of low remote working self-efficacy and hindrance is use australasian journal of information systems schoch 2023, vol 27, research article technostress and remote working performance 3 appraisal with technostress experienced during remote work brought about by the covid-19 pandemic. third, we portray remote working self-efficacy and challenge is use appraisal as important antecedents of performance during remote work. 2 theoretical background stress due to digital technologies has a long history. it was first described as a failure of employees to adapt to modern office technology. a more recent definition of technostress is: “stress that users experience as a result of their use of is in the organisational context” (tarafdar, pullins, & ragu-nathan, 2015, p. 103). further, technostress is “a process that involves a transaction between the individual and the environment” (tarafdar et al., 2019, p. 8). as such, technostress is primarily a dark side phenomenon focused on technology characteristics its users consider a threat (tarafdar et al., 2019). recently, research on technostress has shifted its focus to a view on technostress that accounts for its positive and negative sides. in doing so, it has (explicitly and implicitly) recognised the role of appraisal, which accounts for different individual responses in objectively equal environments (krohne, 2001). for example, studies that have considered appraisal found that technology-driven challenge stressors lead to challenge appraisal of certain is events and thus may result in positive outcomes (e.g., benlian, 2020). in a study concerning healthcare it, califf et al. (2020) have categorised positive characteristics of is use (usefulness) and aspects that facilitate is use (technology support and facilitating conditions) as challenge stressors – thus, these situations were predominantly appraised as challenging between subjects. congruent with their operationalisation, the established technostress-creators have been categorised as hindrance stressors by the study. figure 1. conceptual model, based on tarafdar et al. (2019), maier et al. (2021) recent conceptual work on technostress has differentiated between techno-distress and techno-eustress. techno-distress “embodies the negative stress that individuals face in their use of is” (tarafdar et al., 2019, p. 20). it thus involves the individual appraisal of is use as negative – hindering, threatening, or damaging – and is associated with negative outcomes (tarafdar et al., 2019). as pointed out, the operationalisation of technostress-creators already involves parts of this techno-distress process, as they have an inherent threat appraisal. australasian journal of information systems schoch 2023, vol 27, research article technostress and remote working performance 4 techno-eustress refers to the positive side of stress and involves challenge is use appraisals and positive outcomes (tarafdar et al., 2019). the authors further suggest that there may be a relationship between hindrance appraisal with positive outcomes under some circumstances (tarafdar et al., 2019). on this basis, we depict a conceptual model of this process in figure 1 (see above). there are other conceptualisations, for example, focusing on addiction (hu, park, day, & barber, 2021), but this work on remote work during covid-19 focuses on organisational is use. 2.1 sources of technostress according to theory, the root of technostress is is use (ayyagari, grover, & purvis, 2011). thus, previous work on technostress has included is use variables directly into their models. for example, ayyagari et al. (2011) include is use as a control variable, stich, tarafdar, stacey, and cooper (2019) investigate email use as a driver of stress, and maier, laumer, eckhardt, and weitzel (2015) find an effect of social network usage on stress. similarly, events that happen during is use, such as technology-induced interruptions, have been assessed as potential sources of stress (galluch, grover, & thatcher, 2015). research has identified conditions that create stress and summarised them as, for example, invasion, overload, complexity, uncertainty and insecurity (ragu-nathan, tarafdar, ragunathan, & tu, 2008). their conceptualisation has an inherent threat or hindrance appraisal and thus measures the negative side of stress (tarafdar et al., 2019). the corresponding items measure a misfit between individual resources and situational conditions (ayyagari et al., 2011). for example, techno-overload occurs when is forces individuals to work faster and longer than they want. a given number of emails is considered too many when it exceeds the level an individual feels confident dealing with (stich et al., 2019). the exact number of emails necessary to create the perception of techno-overload is highly individual and related to individual factors, such as skills, preferences, or self-efficacy. traditional technostress-creators (as conceptualised by ragu-nathan et al. (2008)) thus represent a condition after the individual appraisal (c.f., tarafdar et al., 2019). 2.2 individual factors individual factors that moderate the relationship between is use and different appraisals have been investigated regarding technostress. tarafdar et al. (2019) summarise existent research and find that such individual factors include technology self-efficacy, technology competence, or personality traits (e.g., neuroticism, agreeableness, and extraversion). the influence of personality traits on the relationship between technostress and other job-related outcomes has been studied by srivastava, chandra, and shirish (2015). other studies found age-related user characteristics, such as computer experience and self-efficacy, to be influencing factors for stress and task performance due to technology-mediated interruptions (tams, thatcher, & grover, 2018). these factors may help users experience higher degrees of control over interruptions, which helps mitigate adverse effects on well-being. congruent with seminal work on stress (lazarus & folkman, 1984), a study on cybersecurity-related threats finds that user self-efficacy is an antecedent for perceived avoidability – a perception that a threatening situation can be dealt with (liang & xue, 2009). 2.3 appraisal and cognitive mediating processes this touches upon the vital question in technostress research of how different individuals experience is use differently and what individual factors drive the relationship (tarafdar et australasian journal of information systems schoch 2023, vol 27, research article technostress and remote working performance 5 al., 2019). it has been recognised that not all users are similarly affected by the use of is. what stresses one individual might easily be handled by another. this view emphasises the role of the individual and allows for differences in how situations are perceived. the cognitive process that explains different individual responses in objectively equal situations is called appraisal (krohne, 2001). such appraisals can differ between individuals regarding the same stimulus (smith & kirby, 2011). relevant appraisal conditions in a work context are considered to be challenge appraisal and hindrance appraisal and have been used in the context of is use (lepine et al., 2016; maier et al., 2021). lazarus and launier (1978) describe challenges as situations that provide opportunities to overcome hardship and for growth. this is congruent with lepine et al. (2016), who consider challenging work conditions to promote personal growth and enable the fulfilment of work tasks, while hindering work conditions thwart them. other is studies have introduced a more nuanced appraisal model that incorporates the concept of primary and secondary appraisal (beaudry & pinsonneault, 2005, 2010). this differentiation is congruent with seminal work on stress (e.g., lazarus & folkman, 1984). in primary appraisal, a user determines whether a situation provides an opportunity or is considered a threat. this depends on the expected consequences of the situation (beaudry & pinsonneault, 2005). secondary appraisal involves the user’s perception of control over an itrelated situation. it is worth noting that general individual beliefs about capabilities may affect appraisal but are conceptually different and robust across situations (lazarus & folkman, 1984). control can, for example, be perceived through being able to schedule work independently, autonomy in the methods chosen to complete work assignments, or the ability to determine what is to be done (tams et al., 2020). a distinction can be drawn between internal control and external control (bhattacherjee, davis, connolly, & hikmet, 2018). internal control is the user’s ability to control their own behavior. individual factors and capability beliefs may influence the perception of internal control. external control is the user’s perception of control over the environment, including access to organisational resources (bhattacherjee et al., 2018). it could be argued that the constructs and items developed by lepine et al. (2016) to measure challenge and hindrance appraisal capture mainly primary appraisal. the authors build on challenge-hindrance-stressor research, which “does not differentiate the primary and secondary appraisal process” (lepine et al., 2016, p. 1052). yet, they also state that individual characteristics, such as cognitive ability, may influence their operationalisation of appraisal and that future work should investigate such individual differences. a link between such individual differences and secondary appraisal seems intuitive. 2.4 coping and adaptation the research stream that considers primary and secondary appraisal operationalisation has also drawn connections to user adaptation (bhattacherjee et al., 2018; stein, newell, wagner, & galliers, 2015). it has developed notably separately from the technostress stream yet shares many underlying theories and concepts. adaptation is congruent with the notion of coping, which is a mediating process of stress (lazarus & folkman, 1984). is research has made several contributions regarding appraisal, individual factors and how they affect the stress process. studies on coping with technostress have found that control over it and positive reappraisal are important factors for successfully dealing with techno-distress (pirkkalainen, salo, tarafdar, & makkonen, 2019). the perception of control has also been australasian journal of information systems schoch 2023, vol 27, research article technostress and remote working performance 6 associated with problem-focused coping strategies that may help overcome issues with is (e.g., salo, makkonen, & hekkala, 2020). fine granular views of the coping process have further suggested that this is associated with the availability of resources (ortiz de guinea, 2016). 2.5 outcomes of the technostress process occupational psychologists adapted the appraisal concept to the work context through the challenge-hindrance-framework (cavanaugh et al., 2000). the appraisal process is implicit in this conceptualisation (benlian, 2020). this means that most individuals generally appraise challenge and hindrance stressors (stimuli) accordingly (lepine et al., 2016). yet, individuallevel differences in appraisal cannot be accounted for by the challenge-hindrance-stressor framework. several meta-studies on the challenge-hindrance-framework have been published since and have underscored its relevance to research and practice (mazzola & disselhorst, 2019; podsakoff, lepine, & lepine, 2007). according to the studies, challenging situations are generally associated with positive outcomes, such as performance, and hindering situations with negative outcomes, such as psychological strain. we summarise the current findings and the existing research gap as follows. many studies have theoretically established and empirically investigated the relationship between is use and technostress. research on technostress has investigated several individual factors that influence the relationship between is use and the negative side of technostress. this primarily involves the relationship with various aspects of control. yet, research has acknowledged that there are conceptual issues and overlaps in technostress research that require clarification. particularly the individual factors that influence is use appraisals and their relationship with known technostress-creators that have an inherently negative connotation have seen little attention. in particular, this regards the operationalisation of appraisal following challengehindrance-research, which is widely used to investigate the bright and dark side of technostress. investigating the influence of individual differences on such is use appraisal provides an avenue to advance theoretical knowledge on technostress. in this work, we aim to address these issues. the covid-19 pandemic has brought many individuals into a novel is use situation they may not have chosen themselves and did not envision before the pandemic. this has led to a novel use situation that provides excellent opportunities for research regarding the perception of technostress and its outcomes. 3 hypothesis development we propose a research model based on hypotheses derived from the literature. following the conceptual model from left to right, the research model of this paper comprises is use for remote work during covid-19, individual remote working self-efficacy, the role of is use appraisal, and their influence on techno-distress1 and performance. a graphical representation of the research model is shown in figure 2. in the following, we derive the corresponding hypotheses in detail. 1 as hu et al. (2021) point out, it may be problematic to use the word technostress or techno-distress to refer to an outcome and that technostrain may be a better term. yet, we stay within known terminology in is research (e.g., shu et al., 2011) and use techno-distress to refer to the underlying state that users experience as a result of the techno-distress process. australasian journal of information systems schoch 2023, vol 27, research article technostress and remote working performance 7 3.1 the influence of remote working self-efficacy techno-distress has been characterised as a misfit between situations related to is use and an individual’s resources (ayyagari et al., 2011). it is important to note that this is a construct that captures is use appraised by the individual user as threatening or damaging (tarafdar et al., 2019). the conceptualisation emphasises the individuals’ ability to deal with the demands imposed by is use. thus, and congruent with tarafdar et al. (2019), individual factors play an important role in technostress. previous literature has primarily included personal resources, such as general is problem-solving competencies described as digital literacy (tarafdar et al., 2019) or technology competence (tarafdar et al., 2015) in work on the perception of technodistress. yet, the resources required for remote work have been separately studied in previous works (e.g. wang & haggerty, 2011). such resources are broader and include providing adequate information by the employer and ways to receive help regarding remote work. to our knowledge, no studies regarding techno-distress have yet included such context-specific measures. we suggest that remote working self-efficacy affects techno-distress in times of remote work situations such as the ones experienced during the covid-19 pandemic. further, the employer can contribute to this sense of self-efficacy by providing adequate support and information. this is because individuals who are self-efficacious with it will know how to operate is in a healthy manner and can prevent or circumvent techno-distress by themselves or with their organisation's help. for example, users can deactivate notifications of their work communication tools to reduce techno-invasion. such support is particularly important when close in-person contact with co-workers, and thus social support, is unavailable. thus, we bring forward the following hypothesis: h1: remote working self-efficacy has a negative effect on techno-distress. self-efficacy is a central construct in behavioural research and has been identified as a major driver of performance in occupational psychology and management science. this is because individuals with high self-efficacy, compared to those with low self-efficacy, may be more persistent in problem-solving even if they initially experience hindrances and setbacks (tims, b. bakker, & derks, 2014). analyses in the workplace related to computer hardware and software have empirically confirmed this proposition (harrison, rainer, hochwarter, & thompson, 1997). thus, we transfer this concept to the context of remote work. for example, suppose individuals with high remote working self-efficacy encounter a technical issue during videoconferences. in that case, they may work persistently to find a workaround or fix the problem, which increases their effectiveness and efficiency in completing the meeting. in turn, this may increase performance. thus, we hypothesise: h2: remote working self-efficacy has a positive effect on performance. in this paper, we extend this existing view on the role of self-efficacy to the context of individual appraisal of is use. smith and kirby (2011) refer to lazarus and folkman (1984) and point out that challenge appraisals are more likely when the individual has control over a situation. in other words, the person perceives that it “has the potential to change the circumstances to bring them more in line with his or her desires” (smith & kirby, 2011, p. 8). this suggests that individual resources, such as remote working self-efficacy, are important antecedents of is use appraisal. apart from this theoretical plane, empirical research has shown that there is a connection between self-efficacy and challenge appraisal. yet, the australasian journal of information systems schoch 2023, vol 27, research article technostress and remote working performance 8 constructs are not identical. for example, jerusalem and schwarzer (2010) show that individuals with low self-efficacy show different appraisal patterns over time than individuals with high self-efficacy. they find that individuals with high self-efficacy maintain higher levels of challenge appraisal throughout a task. this exemplifies that while self-efficacy is a characteristic of the individual, appraisal can vary from situation to situation and within situations. similar indications have been found in technostress literature, where salo, pirkkalainen, makkonen, and hekkala (2018) suggest that confidence to overcome smartphone failures is essential to positive views on stress. it thus seems intuitive that high self-efficacy influences challenging is use appraisal. jerusalem and schwarzer (2010) find that the opposite is also true: low self-efficacy is associated with increased threat and hindrance appraisals over time. this is congruent with the conceptual work of tarafdar et al. (2019), who specifically propose that low self-efficacy may be associated with increased threat appraisal (which resembles hindrance appraisal in the work context, as pointed out). we thus conclude that there are several indications for the role of remote working self-efficacy in determining a challenge or hindrance is use appraisal in times of remote work and hypothesise: h3a: remote working self-efficacy has a positive effect on challenge is use appraisal. h3b: remote working self-efficacy has a negative effect on hindrance is use appraisal. 3.2 the influence of challenge and hindrance appraisal several technostress studies have already incorporated challenge and hindrance situations into their models. califf et al. (2020) categorised technostress-creators and technostressinhibitors as either challenging or hindering in a mixed methods study in the health care sector. in their research, technostress-creators, such as unreliability, complexity, uncertainty, insecurity, and overload were categorised as hindering. similarly, benlian (2020) developed technology-driven challenge and hindrance stressors and found them to confer with challenge and hindrance appraisal. congruent with these previous results of is literature, we thus propose that hindrance is use appraisal will be positively associated with techno-distress (califf et al., 2020). this is also in line with tarafdar et al. (2019), who state that known technostress creators have an inherently negative connotation. it thus captures the “technology environment as threatening and the outcomes [as] adverse consequences” (tarafdar et al., 2019, p. 12). we thus hypothesise: h4: hindrance is use appraisal has a positive effect on techno-distress. contrarily, hindrance stressors and hindrance appraisal may hamper performance. this is because such situations provide no opportunity for personal growth or gains but rather thwart them (cavanaugh et al., 2000). thus, occupational psychology research has found negative relationships between hindrance appraisal and task performance (lepine et al., 2016). recent meta-studies have confirmed this relationship in the realm of occupational psychology (mazzola & disselhorst, 2019). previous work on technostress has suggested a connection between techno-distress and performance (tarafdar et al., 2015). it is important to note that techno-distress implies a threat or hindrance appraisal (tarafdar et al., 2019). other studies have pointed out that the relationship between hindrance appraisal and performance has not been fully understood yet and that different empirical results exist (lepine et al., 2016). we conclude from theoretical conceptualisation and empirical results that hindrance appraisal is causal for effects on performance. thus, we hypothesise: h5: hindrance is use appraisal has a negative effect on performance. australasian journal of information systems schoch 2023, vol 27, research article technostress and remote working performance 9 in contrast, challenge stress provides opportunities for growth and personal gains (lepine et al., 2016). this is because challenge stress may generally be associated with higher motivation and the ability to overcome hurdles (mazzola & disselhorst, 2019). recent work on the positive side of technostress has found that characteristics of is which make them more useful can be appraised as challenging (califf et al., 2020). similarly, benlian (2020) emphasises the role of is for learning and mastering skills in his characterisation of technology-driven challenge stressors. in addition, recent work suggests that challenge is use appraisal leads to innovative use behaviour (maier et al., 2021). in turn, advanced and innovative use behaviour has been associated with increased performance (burton-jones & straub, 2006). thus, theory and empirical findings imply that challenge is use appraisal may be associated with an increase in performance. hence, we hypothesise: h6: challenge is use appraisal has a positive effect on performance. 3.3 control variables this model's dependent variables may be influenced by other factors, too. thus, we include is-related variables and variables related to job stress in the model that have been shown to influence the outcomes. first, higher is use has been shown to influence technostress. technostress has been theorised as a consequence of is use (ayyagari et al., 2011). thus, various variables relating to is use have been included both as explanatory variables (e.g., maier, laumer, weinert, & weitzel, 2015; stich et al., 2019) and control variables (e.g., ayyagari et al., 2011) in previous studies. second, a higher workload may increase both technostress (ayyagari et al., 2011; stich et al., 2019) and performance (e.g., lepine, podsakoff, & lepine, 2005). figure 2. research model australasian journal of information systems schoch 2023, vol 27, research article technostress and remote working performance 10 4 quantitative empirical analysis 4.1 survey design and procedures to test the model empirically, we design an online survey. the survey collects data concerning is use and its consequences during covid-19. it is part of a larger study and contains questions not reported in this work. we acquired participants via an external research panel focusing on the german workforce (dynata, formerly research now ssi). the external panel provider was primarily chosen because they allow efficient access to a representative sample of german full-time and part-time employees. further, using the panel provider allowed us quick access to employees working from home, which would have been difficult during the first wave of covid-19. after internal testing of the data collection instrument, we conducted a small pretest using the panel provider. the responses were assessed, the feedback was implemented, and the data was collected anonymously. respondents were asked to answer the questionnaire honestly and to give consent to their participation. for their participation, respondents were paid a small incentive. the survey was administered in may 2020 during the initial covid-19 lockdown in germany. data quality was ensured by evaluating open questions and excluding questionnaires that were completed unrealistically fast. for example, we excluded participants who gave nonsensical answers regarding their profession in a text box. further, extreme outliers were identified and deleted after a manual assessment of the response time distribution. as a result, we collected 1,553 valid responses. we consider this sample largely representative of the german workforce. we used existing evaluated item scales for our questionnaire, which focused on individual resources regarding the digital workplace, is use, appraisal, technostress, and performance. we use the wang and haggerty (2011) scale for measuring remote working self-efficacy. recent literature acknowledges that there are few adequate measures for techno-distress2 (hu et al., 2021). we thus use the items from ragu-nathan et al. (2008) that measure “stressors appraised by the individual as damaging” (tarafdar et al., 2019, p. 9) which “create technostress in the organisation “(ragu-nathan et al., 2008, p. 421) as lower order constructs (loc). these items measure several aspects of the stress process, such as outcomes (“i spend less time with my family due to this technology”) and coping actions ("i do not share my knowledge with my co-workers for fear of being replaced”). operationalization of stress concepts, such as stimuli, coping, and outcome, have often been confounded in stress research (edwards & cooper, 1988). thus, by combining them into a reflective higher-order construct (hoc) we aim to capture the underlying construct of techno-distress. such a hoc should be viewed as an outcome of the appraisal process rather than an antecedent variable. for reflective measurement models, the underlying construct is assumed to cause changes in the indicators (e.g., jarvis, mackenzie, & podsakoff, 2003). thus, in the paper at hand, the reflective hoc is assumed to cause changes in the locs, which is consistent with ragunathan et al. (2008). this approach further helps ensure a parsimonious model (polites, roberts, & thatcher, 2012). for appraisal, we ask participants to report their appraisal of is use in general as either challenging or hindering. congruent with benlian (2020), we use the scales of lepine et al. (2016) adapted to the context of is. when answering these questions, individuals were asked to think about their overall digital technology use at work. the same 2 this important early contribution to technostress uses the term to refer to the negative side of technostress – techno-distress. australasian journal of information systems schoch 2023, vol 27, research article technostress and remote working performance 11 panel had participated in a previous questionnaire before covid-19, revealing that the digital technologies used are rather heterogeneous. for example, digital technologies consist of various devices and peripherals, communication technology, and specialised software. many individuals likely experienced changes in their work it during covid-19. changes may include newly introduced communication and collaboration systems, increased use (see controls), and increasingly digitised processes, such as digital signatures. regarding performance, we use the scale of frone, yardley, and markel (1997). for the control variable of workload, we used a copsoq iii subscale (burr et al., 2019). all of these measurements are reflective and measured on five-point likert scales. regarding the control variable of is use, we adapted a scale by venkatesh, thong, and xu (2012) to reflect a relative change in use during covid-19 for email, instant messaging, audio and video communication. this construct is formative and measured on a three-point likert scale. the appendix provides an overview of the items. 1,553 participants completed our survey, of which 41.9% are female and 58.1% male. regarding age, 1.5% were below 25, 15.1% were 25-34, 27.4% were 35-44, 31.2% were 45-54, 24.4% were 55-64, and below 0,5% were 65 and older. all respondents work and live in germany with 9.4% reporting a migration background. industries and professions are widely distributed (see table 1). most respondents work full-time (74%). while it is highly likely that all individuals were confronted with increased digital communication during the time of the data collection, roughly 51.9% report to spend substantial work time outside the office. industries public administration, safety, and defense 11.5% manufacturing/production of goods 11.2% health and social work 9.3% wholesale and retail trade 8.2% information and communication 7.9% banks/financial and insurance providers 7.7% other business and personal economic services 5.9% professional, scientific, and technical services 5.0% others (under 5%) 33.3% professions computer, information and communication technology occupations 13.0% professions in law and administration 10.4% professions in financial services, accounting, and tax consulting 9.1% professions in business management and organisation 7.5% purchasing, sales and trade occupations 6.9% medical health professions 5.5% others (under 5%) 35.0% working hours full-time (<30h/week) 74.0% part-time (>30h/week) 26.0% primary place of work office 48.9% home office 33.1% mixed 14.4% other location 3.6% table 1. demographics of the sample australasian journal of information systems schoch 2023, vol 27, research article technostress and remote working performance 12 4.2 results we assess the model through structural equation modeling (pls-sem) using smartpls 3.2. we start with the evaluation of the measurement model before assessing the structural model and testing our hypotheses. 4.2.1 evaluation of the measurement model regarding the reflective measurement model, we tested the internal consistency reliability using composite reliability (cr) and cronbach’s alpha (alpha). all scales are above 0.7 and below 0.95, which can be regarded as satisfactory. for convergent validity, we examine outer loadings and average variance extracted (ave). outer loadings are satisfactory because they all exceed the common threshold of 0.708 (hair, hult, ringle, & & sarstedt, 2017). ave is above 0.5 in all cases. this indicates convergent validity. for discriminant validity, we examine each indicator’s cross-loadings with all other constructs and find that they are indeed lower than the indicator’s outer loadings. further, we evaluate the heterotrait-monotrait (htmt) ratios. these are consistently below the threshold of 0.90 (henseler, ringle, & sarstedt, 2015) for all first-order constructs with a maximum of 0.73 (techno-invasion and techno-insecurity). thus, discriminant validity is supported. table 2 shows the respective values as well as the means and standard deviations (sd) of the reflective constructs. further, we check for common method variance (cmv). we use a post hoc correlational marker test to do so (lindell & whitney, 2001; richardson, simmering, & sturman, 2009). we determine the two smallest shared variances in bivariate correlations among substantive exogenous latent variables. we then correct for it by partialling out the shared variance. our results show that no bivariate correlation became insignificant as a result. thus, we conclude that cmv is no major concern in this study. number of indicators mean sd outer loadings alpha cr ave hindrance is use appraisal 3 2.602 1.182 0.909-0.934 0.910 0.944 0.848 challenge is use appraisal 3 3.270 1.015 0.864-0.901 0.864 0.917 0.786 remote working self-efficacy 4 3.582 0.996 0.853-0.918 0.919 0.943 0.805 performance 4 3.485 1.036 0.854-0.884 0.893 0.925 0.756 techno-overload (loc) 4 2.464 1.221 0.823-0.905 0.899 0.930 0.769 techno-invasion (loc) 3 2.161 1.217 0.809-0.900 0.833 0.900 0.751 techno-complexity (loc) 5 2.177 1.159 0.826-0.901 0.918 0.938 0.753 techno-uncertainty (loc) 4 2.510 1.173 0.860-0.890 0.894 0.926 0.758 techno-insecurity (loc) 5 2.058 1.151 0.812-0.881 0.900 0.926 0.714 techno-distress (hoc) 21 2.264 1.180 0.740-0.876 0.907 0.907 0.661 table 2. descriptive statistics reflective constructs, outer loadings, internal consistency, and average variance extracted australasian journal of information systems schoch 2023, vol 27, research article technostress and remote working performance 13 4.2.2 evaluation of the structural model and hypotheses testing collinearity is also not a major issue in the structural model since all inner variance inflation factors are lower than 5 (maximum of 1.146). figure 3 presents the path estimates for the model, including their significance level. r² values are depicted in the constructs. figure 3. model results regarding h1, we find that remote working self-efficacy is associated with technostress with a small effect size (f²=0.050). further, the data shows that remote working self-efficacy is a driver of performance in times of work from home with a small effect size (f²=0.079) supporting h2. contrary to h3a, we find that it is not significantly related to hindrance is use appraisal, indicating that there may be different reasons for hindrance appraisal. yet, we find a significantly positive relationship between remote working self-efficacy and challenge is use appraisal and a large effect size (f²=0.470). this supports h3b. regarding the relationship between hindrance is use appraisal and techno-distress, we find that it is positively associated and that the effect size is small to medium (f²=0.133). this supports h4. further, we find the relationship between hindrance is use appraisal and performance to be statistically significant. yet, the effect size is marginal (f²=0.005). therefore, and considering the large sample size of this study, we consider h5 not supported. regarding challenge is use appraisal, we find that it is indeed associated with higher performance with a small effect size (f²=0.026). this is in support of h6. regarding controls, workload (β=0.095; p<0.001; f²=0.011) and increased is use during covid-19 (β=0.071; p=0.023; f²=0.005) are related to performance. also, both workload (β=0.295; p<0.001; f²=0.123) and increased is use during covid-19 (β=0.226; p<0.001; f²=0.70) are positively related to techno-distress. table 3 summarises the empirical findings. australasian journal of information systems schoch 2023, vol 27, research article technostress and remote working performance 14 theoretical hypotheses empirical results h1 neg. remote working self-efficacy → techno-distress supported h2 pos. remote working self-efficacy → performance + supported h3a neg. remote working self-efficacy → hindrance is use appraisal n.s. not supported h3b pos. remote working self-efficacy → challenge is use appraisal +++ supported h4 pos. hindrance is use appraisal → techno-distress + supported h5 neg. hindrance is use appraisal → performance o not supported h6 pos. challenge is use appraisal → performance + supported table 1. overview of hypotheses and empirical results note: n.s. indicates a non-significant effect. for significant effects: o indicates a marginal effect (f² <0.02), +/a small (f² ≥0.02), ++/-a medium (f² ≥0.15), and +++/--a large (f² >0.35) effect size. 5 discussion 5.1 theoretical implications the theoretical implications of this work are threefold. they comprise insights on the relationship between self-efficacy and is use appraisal, insights regarding the individual factors influencing technostress in times of remote work, and insights into how these factors influence performance. our theoretical contributions are summarised in table 4. first, we follow a call for research by tarafdar et al. (2019) to investigate the relationship between individual factors and is use appraisal. the challenge-hindrance conceptualisation of is use appraisal is often used in the technostress literature (e.g., califf et al., 2020; maier et al., 2021). as such, it does not differentiate between primary and secondary appraisal (lepine et al., 2016) and thus deviates from other operationalisations of is use appraisal, which often includes it control (secondary appraisal). internal control is affected by individual beliefs about capabilities (e.g., tams et al., 2018). how this conceptualisation of is use appraisal is affected by individual factors, such as self-efficacy, is important to better understand empirical results of studies using it. we address our research question in the context of remote work during covid-19 and with a focus on remote working self-efficacy. we find that hindrance is use appraisal is not related to the individuals’ remote working self-efficacy. tarafdar et al. (2019) propose low self-efficacy as a factor that may affect appraisal, which indicates that is is a threatening and disturbing factor. the authors further point out that both hindrance and threat situations are associated with distress. therefore, it is an interesting finding of our study that hindrance appraisal is not related to self-efficacy. this contradicts our hypothesis and previous conceptual work on technostress (tarafdar et al., 2019). we conclude that hindrance is use appraisal has a different root that warrants further research. the implications of this finding may be that hindering is use might be associated with factors that the individual cannot control, regardless of individual self-efficacy. thus, the origins of such stressors could lie in either the technology or the work itself. this may be congruent with the conceptualisation of the technostress trifecta of tarafdar et al. (2019) who also consider factors related to the design of is. organisations may be able to address such issues without the involvement of their employees. if researchers and practitioners identify such sources of techno-distress, they may be able to reduce technostress through organisational measures. australasian journal of information systems schoch 2023, vol 27, research article technostress and remote working performance 15 second, and regarding performance, we find that remote working self-efficacy in general increases performance for remote work during covid-19, which is congruent with previous work on is use (e.g., compeau et al., 1999). the influence of hindrance is use appraisal, however, is only marginal, which puts the real-world impact and thus its practical relevance of the relationship in doubt (mohajeri, mesgari, & lee, 2020). yet, we show that a challenge is use appraisal further contributes to increased performance. this indicates that the two subprocesses of technostress that can be assessed using the appraisal items from challengehindrance-research may be more separate than previously thought. the fact that both remote working self-efficacy and challenge is use appraisal have positive effects on performance has theoretical implications. as we pointed out, appraisal and selfefficacy are related yet different. self-efficacy is an individual characteristic that serves as a resource factor for appraisal (jerusalem & schwarzer, 2010). thus, while self-efficacy is rather stable and tends to translate to other situations (bandura, 1977), appraisal may vary between situations and within situations over time depending on outcome expectations. this is because appraisal is a cognition that may change continuously as an individual interacts with the environment (jerusalem & schwarzer, 2010). of course, a cross-sectional survey cannot capture this time effect. yet, it shows that the two constructs are different and may be affected differently. third, we shed light on the relationship between is use during remote work brought about by covid-19 and techno-distress. in this work, we propose that the novel situation of communication and collaboration technology use during covid-19 is a source of technodistress and we control for this increased use in our study. we further show that remote working self-efficacy is a way to mitigate techno-distress in times of remote work. this is congruent with previous work on technostress (e.g., shu, tu, & wang, 2011) and previous work on the overlap between social cognitive theory and is use (compeau, higgins, & huff, 1999). we advance the current knowledge on technostress research regarding challenge is use appraisal by identifying remote working self-efficacy as a major antecedent in the particular context of remote work. previous work has given little recognition to its possible role as an antecedent for challenge is use appraisals. yet, previous research has stated that controllability of the situation (gibbons, 2010) and a high chance of coping may be associated with a positive side of stress and thus challenge is use appraisal (salo et al., 2018). self-efficacy is, in turn, an assessment of the own abilities built on past performances and experiences. in that regard, it also captures confidence in controlling a situation to some degree. a more detailed view of how self-efficacy works is provided by jerusalem and schwarzer (2010). in their research they assess the relationship between temporal patterns of appraisal and self-efficacy. their results suggest that individuals with low self-efficacy may well have challenge appraisals of a situation at first. yet, over time the negative experiences of failure results in frustration and a decreasing perception of challenge. thus, self-efficacy heavily affects challenge appraisal. in summation, self-efficacy is a construct that has been used in many studies on technostress and it may seem trivial to revisit the construct. yet, our empirical findings show that the relationships may be more complex and not as clear as might be assumed. we conclude that the relationship between self-efficacy and appraisal is worth revisiting. our empirical results show that researchers may overstate the effect of self-efficacy or challenge appraisal when not measuring the respective other construct. future studies may provide additional detail on the australasian journal of information systems schoch 2023, vol 27, research article technostress and remote working performance 16 relationship by following the stress process over time to analyse the temporal interplay (jerusalem & schwarzer, 2010). also, previous work has shown that self-efficacy may be shaped more by previous outcomes, such as performance, than it shapes future outcomes (harrison et al., 1997). thus, the construct and associated empirical results may provide misleading insights. this further emphasises the necessity to revisit self-efficacy and appraisal with future research. relationship between self-efficacy and appraisal • remote working self-efficacy does not affect hindrance is use appraisal, indicating that hindrance may be unrelated to the individual’s believe in his or her abilities • remote working self-efficacy has a strong effect on challenge is use appraisal and is an important antecedent individual factors and appraisal influencing techno-distress • confirming recent studies, hindrance is use appraisal increases techno-distress • further, remote working self-efficacy reduces techno-distress in times of remote work individual factors and appraisal influencing performance • both remote working self-efficacy and challenge is use appraisal increase the perception of individual performance • yet, the significant negative relationship of hindrance is use appraisal and performance has no substantial effect size table 4. overview of theoretical contributions note: n.s. indicates a non-significant effect. for significant effects: o indicates a marginal effect (f² <0.02), +/a small (f² ≥0.02), ++/-a medium (f² ≥0.15), and +++/--a large (f² >0.35) effect size. in summation, self-efficacy is a construct that has been used in many studies on technostress and it may seem trivial to revisit the construct. yet, our empirical findings show that the relationships may be more complex and not as clear as might be assumed. we conclude that the relationship between self-efficacy and appraisal is worth revisiting. our empirical results show that researchers may overstate the effect of self-efficacy or challenge appraisal when not measuring the respective other construct. future studies may provide additional detail on the relationship by following the stress process over time to analyse the temporal interplay (jerusalem & schwarzer, 2010). also, previous work has shown that self-efficacy may be shaped more by previous outcomes, such as performance, than it shapes future outcomes (harrison et al., 1997). thus, the construct and associated empirical results may provide misleading insights. this further emphasises the necessity to revisit self-efficacy and appraisal with future research. australasian journal of information systems schoch 2023, vol 27, research article technostress and remote working performance 17 5.2 managerial implications our findings have several managerial implications. we find that increased use of is for communication during covid-19 has adverse consequences on employees in the form of technostress. this may be driven by the novel situation that employees are facing. yet, our research suggests several measures that can be taken to mitigate such technostress. we find that hindrance is use appraisal increases technostress and that hindrance is use appraisal is not associated with the individuals’ self-efficacy. this may indicate that there are sources of technostress in the is use of employees that cannot be mitigated through individual knowledge but are inherent in either the work or the technology. this indicates that organisations can indeed take actions on these levels to reduce technostress of their employees. this could involve, for example, providing adequate technology to fulfil the communication needs of the individuals. to the best of our knowledge, such demands have been scarcely investigated. yet, a recent study has pointed to technology incompatibility as a potential source of demands for employees (vaziri et al., 2020). thus, organisations and their it departments should consider providing adequate and useful tools to mitigate technostress – particularly in the times of physical distancing. further, we find that the remote working self-efficacy of individuals strongly influences the perception of technostress during remote work. we find that it not only influences the relationship between is use and technostress, but also strongly influences challenge is use appraisal, which is associated with increased usefulness and performance. we thus conclude that it is paramount for organisations to provide an environment where employees can increase their digital literacy in general and remote working self-efficacy in particular. in a way, this is also good news, as it is easier to improve systematically than cognitive appraisal, which is said to be highly individual (lazarus & folkman, 1984). yet, there are other avenues to affect appraisal, such as cognitive reappraisal or mindfulness (garland, gaylord, & park, 2009) 6 limitations and future research this work has several limitations that leave avenues for future research. for example, our operationalisation of appraisal focuses on the general use of is. while this is congruent with previous research on technostress (e.g. benlian, 2020), research in psychology has suggested that appraisal can change from situation to situation within individuals and has thus suggested different ways of measurement (searle & auton, 2015). other studies have included frequent appraisal measurements within a single stressful situation over time (jerusalem & schwarzer, 2010). yet, the detailed measurement appraisal in individual situations requires complex data collection. it has been pointed out that it has been omitted for obvious reasons of practicality in many studies (jerusalem & schwarzer, 2010). nonetheless, we acknowledge this as a shortcoming of our study and encourage future work to look into more detailed analyses. further, the conceptual model derived from the literature offers many avenues for investigation. in our research model, we operationalise it using different constructs. this includes the appraisal concept by lepine et al. (2016), which is discussed extensively in this paper. it also involves the chosen outcome variables, which are conceptually relatively far apart. for example, a narrower operationalisation of performance, such as it-enabled australasian journal of information systems schoch 2023, vol 27, research article technostress and remote working performance 18 performance, could have been used. other outcome variables related to the bright and dark side of is use, such as affect or well-being, could also be included in subsequent studies. in addition, the relationship between self-efficacy, appraisal, and outcomes may be affected by previous outcomes more than it is a determinant of future outcomes (sitzmann & yeo, 2013). this is an intriguing proposition that has not been investigated in relation to technostress to the best of our knowledge. such analyses require data beyond cross-sectional surveys and may consider both appraisal and self-efficacy and their relationship. further, other factors may affect the measurement of the relationship between these variables. for example, previous work has shown problems with overconfidence and overestimation of selfassessed performance. this may be associated with the dunning-kruger-effect (kruger & dunning, 1999). such issues could be considered in future work. regarding the covid-19 pandemic, we acknowledge that more stressors exist that may lie outside of the realm of is use and technostress, such as childcare, job insecurity, and a lack of social contact. is use may have had positive effects during covid-19, for example, staying in touch with co-workers and continuing working from home. 7 conclusion due to the physical distancing measures to counteract covid-19, digital communication tools and their use have changed how we work, and remote work has increased dramatically. this work investigates the positive and negative consequences of is use in times of covid-19 and how they differ between individuals. we follow a call for research inquiries into the factors that influence individual appraisal of is use situations and thus its positive and negative sides (tarafdar et al., 2019). this is particularly true for the operationalisation of appraisal in challenge-hindrance-research that does not differentiate between primary and secondary appraisal. we find that hindrance is use appraisal is associated with higher technostress. yet, hindrance is use appraisal is not associated with remote working self-efficacy, which suggests that some sources of technostress cannot easily be changed by individuals. rather, they might be rooted in the provided technologies or the circumstances of digital work. such factors may be captured in a hindrance is use appraisal. nonetheless, we find that high levels of remote working self-efficacy are associated with lower levels of technostress, emphasising the role of specific competencies in mitigating stress during remote work. further, we find that remote working self-efficacy is also positively related to challenge is use appraisal, which enables growth and gains and thus leads to higher performance. as a theoretical contribution, we shed light on the relationship between is use and technostress and show that remote working selfefficacy is an important antecedent of is use appraisal. for practitioners, we emphasise the role of both the provision of adequate technology for remote work and the role of remote working self-efficacy of their employees to reduce technostress and increase performance in remote work situations. further research may go into more detail on the appraisal process and differentiate between different stressors and situations. acknowledgements the author extends his gratitude to christian regal, julia lanzl, and henner gimpel for the fruitful conversations on the paper’s subject and their collaboration regarding data collection. australasian journal of information systems schoch 2023, vol 27, research article technostress and remote working performance 19 funding this paper presents and discusses results of the bavarian research association on healthy use of digital technologies and media (fordigithealth), funded by the bavarian ministry of science and arts. references ayyagari, r., grover, v., & purvis, r. (2011). technostress: technological antecedents and implications. management information systems quarterly, 35(4), 831–858. https://doi.org/10.2307/41409963 bandura, a. (1977). self-efficacy: toward a unifying theory of behavioral change. psychological review, 84(2), 191–215. https://doi.org/10.1037/0033-295x.84.2.191 beaudry, & pinsonneault (2005). understanding user responses to information technology: a coping model of user adaptation. management information systems quarterly, 29(3), 493. https://doi.org/10.2307/25148693 beaudry, & pinsonneault (2010). the other side of acceptance: studying the direct and indirect effects of emotions on information technology use. management information systems quarterly, 34(4), 689. https://doi.org/10.2307/25750701 benlian, a. (2020). a daily field investigation of technology-driven stress spillovers from work to home. management information systems quarterly, 44(3). https://doi.org/10.25300/misq/2020/14911 bhattacherjee, a., davis, c. j., connolly, a. j., & hikmet, n. (2018). user response to mandatory it use: a coping theory perspective. european journal of information systems, 27(4), 395–414. https://doi.org/10.1057/s41303-017-0047-0 burr, h., berthelsen, h., moncada, s., nübling, m., dupret, e., demiral, y., . . . pohrt, a. (2019). the third version of the copenhagen psychosocial questionnaire. safety and health at work, 10(4), 482–503. https://doi.org/10.1016/j.shaw.2019.10.002 burton-jones, a., & straub, d. w. (2006). reconceptualizing system usage: an approach and empirical test. information systems research, 17(3), 228–246. https://doi.org/10.1287/isre.1060.0096 califf, c., sarker, s. [saonee], & sarker, s. [suprateek] (2020). the bright and dark sides of technostress: a mixed-methods study involving healthcare it. management information systems quarterly, 44(2). https://doi.org/10.25300/misq/2020/14818 cavanaugh, m. a., boswell, w. r., roehling, m. v., & boudreau, j. w. (2000). an empirical examination of self-reported work stress among u.s. managers. journal of applied psychology, 85(1), 65–74. https://doi.org/10.1037/0021-9010.85.1.65 compeau, d., higgins, c. a., & huff, s. (1999). social cognitive theory and individual reactions to computing technology: a longitudinal study. management information systems quarterly, 23(2), 145. https://doi.org/10.2307/249749 edwards, j. r., & cooper, c. l. [c. l.] (1988). research in stress, coping, and health: theoretical and methodological issues. psychological medicine, 18(1), 15–20. https://doi.org/10.1017/s0033291700001847 australasian journal of information systems schoch 2023, vol 27, research article technostress and remote working performance 20 ellsworth, p. c., & scherer, k. r. (2003). appraisal processes in emotion. in series in affective science. handbook of affective sciences (pp. 572–595). new york, ny, us: oxford university press. frone, m. r., yardley, j. k., & markel, k. s. (1997). developing and testing an integrative model of the work–family interface. journal of vocational behavior, 50(2), 145–167. https://doi.org/10.1006/jvbe.1996.1577 galluch, p. s., grover, v., & thatcher, j. b. [jason bennett] (2015). interrupting the workplace: examining stressors in an information technology context. journal of the association for information systems, 16(1), 1–47. https://doi.org/10.17705/1jais.00387 garland, e., gaylord, s., & park, j. (2009). the role of mindfulness in positive reappraisal. explore (new york, n.y.), 5(1), 37–44. https://doi.org/10.1016/j.explore.2008.10.001 gibbons, c. (2010). stress, coping and burn-out in nursing students. international journal of nursing studies, 47(10), 1299. https://doi.org/10.1016/j.ijnurstu.2010.02.015 hair, j. f., hult, g. t. m., ringle, c. m., & & sarstedt, m. (2017). a primer on partial least squares structural equation modeling (pls-sem) (2nd edition). thousand oaks, ca: sage publications. harrison, a. w., rainer, r. k., hochwarter, w. a., & thompson, k. r. (1997). testing the selfefficacy-performance linkage of social-cognitive theory. the journal of social psychology, 137(1), 79–87. https://doi.org/10.1080/00224549709595415 henseler, j., ringle, c. m., & sarstedt, m. (2015). a new criterion for assessing discriminant validity in variance-based structural equation modeling. journal of the academy of marketing science, 43(1), 115–135. https://doi.org/10.1007/s11747-014-0403-8 hu, x., park, y., day, a., & barber, l. k. (2021). time to disentangle the information and communication technology (ict) constructs: developing a taxonomy around ict use for occupational health research. occupational health science, 1–29. https://doi.org/10.1007/s41542-021-00085-6 jarvis, c. b., mackenzie, s. b., & podsakoff, p. m. (2003). a critical review of construct indicators and measurement model misspecification in marketing and consumer research. journal of consumer research, 30(2), 199–218. https://doi.org/10.1086/376806 jerusalem, m., & schwarzer, r. (2010). self-efficacy as a resource factor in stress appraisal processes. in r. schwarzer (ed.), self-efficacy: thought control of action (pp. 195–213). washington, dc: hemisphere publishing. krohne, h. w. (2001). stress and coping theories. in n. j. e. smelser & p. b. e. baltes (eds.), international encyclopedia of the social & behavioral sciences (pp. 15163–15170). amsterdam, new york: elsevier. https://doi.org/10.1016/b0-08-043076-7/03817-1 kruger, j., & dunning, d. (1999). unskilled and unaware of it: how difficulties in recognizing one's own incompetence lead to inflated self-assessments. journal of personality and social psychology, 77(6), 1121–1134. https://doi.org/10.1037/0022-3514.77.6.1121 lazarus, r. s., & folkman, s. (1984). stress, appraisal, and coping. new york: springer. australasian journal of information systems schoch 2023, vol 27, research article technostress and remote working performance 21 lazarus, r. s., & launier, r. (1978). stress-related transactions between person and environment. in perspectives in interactional psychology (vol. 2, pp. 287–327). springer, boston, ma. https://doi.org/10.1007/978-1-4613-3997-7_12 lepine, j. a., podsakoff, n. p., & lepine, m. a. (2005). a meta-analytic test of the challenge stressor–hindrance stressor framework: an explanation for inconsistent relationships among stressors and performance. academy of management journal, 48(5), 764–775. https://doi.org/10.5465/amj.2005.18803921 lepine, m. a., zhang, y., crawford, e. r., & rich, b. l. (2016). turning their pain to gain: charismatic leader influence on follower stress appraisal and job performance. academy of management journal, 59(3), 1036–1059. https://doi.org/10.5465/amj.2013.0778 liang, & xue (2009). avoidance of information technology threats: a theoretical perspective. management information systems quarterly, 33(1), 71. https://doi.org/10.2307/20650279 lindell, m. k., & whitney, d. j. (2001). accounting for common method variance in crosssectional research designs. the journal of applied psychology, 86(1), 114–121. https://doi.org/10.1037//0021-9010.86.1.114 maier, c., laumer, s., eckhardt, a., & weitzel, t. (2015). giving too much social support: social overload on social networking sites. european journal of information systems, 24(5), 447–464. https://doi.org/10.1057/ejis.2014.3 maier, c., laumer, s., tarafdar, m., mattke, j., reis, l., & weitzel, t. (2021). challenge and hindrance is use stressors and appraisals: explaining contrarian associations in postacceptance is use behavior. journal of the association for information systems, 1–60. maier, c., laumer, s., weinert, c., & weitzel, t. (2015). the effects of technostress and switching stress on discontinued use of social networking services: a study of facebook use. information systems journal, 25(3), 275–308. https://doi.org/10.1111/isj.12068 mazzola, j. j., & disselhorst, r. (2019). should we be “challenging” employees? a critical review and meta-analysis of the challenge-hindrance model of stress. journal of organizational behavior, 40(8), 949–961. https://doi.org/10.1002/job.2412 mohajeri, k., mesgari, m., & lee, a. (2020). when statistical significance is not enough: investigating relevance, practical significance, and statistical significance. management information systems quarterly, 44(2), 525–559. retrieved from https://aisel.aisnet.org/misq/vol44/iss2/4 möhring, k., naumann, e., reifenscheid, m., weiland, a., blom, a. g., wenz, a., . . . cornesse, c. (2020). die mannheimer corona-studie: schwerpunktbericht zur erwerbstätigkeit in deutschland 20.3.-15.4.2020. retrieved from retrieved from: https://www.uni-mannheim.de/media/einrichtungen/gip/corona_studie/2020-0416_schwerpunktbericht_erwerbstaetigkeit.pdf ortiz de guinea, a. (2016). a pragmatic multi-method investigation of discrepant technological events: coping, attributions, and ‘accidental’ learning. information & management, 53(6), 787–802. https://doi.org/10.1016/j.im.2016.03.003 australasian journal of information systems schoch 2023, vol 27, research article technostress and remote working performance 22 pirkkalainen, h., salo, m., tarafdar, m., & makkonen, m. (2019). deliberate or instinctive? proactive and reactive coping for technostress. journal of management information systems, 36(4), 1179–1212. https://doi.org/10.1080/07421222.2019.1661092 podsakoff, n. p., lepine, j. a., & lepine, m. a. (2007). differential challenge stressor hindrance stressor relationships with job attitudes, turnover intentions, turnover, and withdrawal behavior: a meta-analysis. journal of applied psychology, 92(2), 438–454. https://doi.org/10.1037/0021-9010.92.2.438 polites, g. l., roberts, n., & thatcher, j. (2012). conceptualizing models using multidimensional constructs: a review and guidelines for their use. european journal of information systems, 21(1), 22–48. https://doi.org/10.1057/ejis.2011.10 ragu-nathan, t. s., tarafdar, m., ragu-nathan, b. s., & tu, q. (2008). the consequences of technostress for end users in organizations: conceptual development and empirical validation. information systems research, 19(4), 417–433. https://doi.org/10.1287/isre.1070.0165 richardson, h. a., simmering, m. j., & sturman, m. c. (2009). a tale of three perspectives: examining post hoc statistical techniques for detection and correction of common method variance. organizational research methods, 12(4), 762–800. https://doi.org/10.1177/1094428109332834 salo, m., makkonen, m., & hekkala, r. (2020). the interplay of it users’ coping strategies: uncovering momentary emotional load, routes, and sequences. management information systems quarterly, 44(3), 1143–1175. https://doi.org/10.25300/misq/2020/15610 salo, m., pirkkalainen, h., makkonen, m., & hekkala, r. (2018). distress, eustress, or no stress? explaining smartphone users™ different technostress responses. proceedings of the 39th international conference on information systems (icis). retrieved from https://aisel.aisnet.org/icis2018/behavior/presentations/13 searle, b. j., & auton, j. c. (2015). the merits of measuring challenge and hindrance appraisals. anxiety, stress, and coping, 28(2), 121–143. https://doi.org/10.1080/10615806.2014.931378 shu, q., tu, q., & wang, k. (2011). the impact of computer self-efficacy and technology dependence on computer-related technostress: a social cognitive theory perspective. international journal of human–computer interaction, 27(10), 923–939. https://doi.org/10.1080/10447318.2011.555313 sitzmann, t., & yeo, g. (2013). a meta-analytic investigation of the within-person selfefficacy domain: is self-efficacy a product of past performance or a driver of future performance? personnel psychology, 66(3), 531–568. https://doi.org/10.1111/peps.12035 smith, c. a., & kirby, l. d. (2011). the role of appraisal and emotion in coping and adaptation.: in: the handbook of stress science. new york ny: springer. spataro, j. (2020). remote work trend report: meetings. retrieved from retrieved from: https://www.microsoft.com/en-us/microsoft-365/blog/2020/04/09/remote-work-trendreport-meetings/ australasian journal of information systems schoch 2023, vol 27, research article technostress and remote working performance 23 srivastava, s. c., chandra, s., & shirish, a. (2015). technostress creators and job outcomes: theorising the moderating influence of personality traits. information systems journal, 25(4), 355–401. https://doi.org/10.1111/isj.12067 stein, m.-k., newell, s., wagner, e., & galliers, r. (2015). coping with information technology: mixed emotions, vacillation, and nonconforming use patterns. management information systems quarterly. https://doi.org/10.25300/misq/2015/39.2.05 stich, j.-f., tarafdar, m., stacey, p., & cooper, c. (2019). appraisal of email use as a source of workplace stress: a person-environment fit approach. journal of the association for information systems, 20(2), 132–160. https://doi.org/10.17705/1jais.00531 tams, s., ahuja, m., thatcher, j., & grover, v. (2020). worker stress in the age of mobile technology: the combined effects of perceived interruption overload and worker control. the journal of strategic information systems, 29(1), 101595. https://doi.org/10.1016/j.jsis.2020.101595 tams, s., thatcher, j. b. [jason b.], & grover, v. (2018). concentration, competence, confidence, and capture: an experimental study of age, interruption-based technostress, and task performance. journal of the association for information systems, 19, 857–908. https://doi.org/10.17705/1jais.00511 tarafdar, m., cooper, c. l., & stich, j.-f. (2019). the technostress trifecta techno eustress, techno distress and design: theoretical directions and an agenda for research. information systems journal, 29(1), 6–42. https://doi.org/10.1111/isj.12169 tarafdar, m., pullins, e. b., & ragu-nathan, t. s. (2015). technostress: negative effect on performance and possible mitigations. information systems journal, 25(2), 103–132. https://doi.org/10.1111/isj.12042 tims, m., b. bakker, a., & derks, d. (2014). daily job crafting and the self-efficacy – performance relationship. journal of managerial psychology, 29(5), 490–507. https://doi.org/10.1108/jmp-05-2012-0148 vaziri, h., casper, w. j., wayne, j. h., & matthews, r. a. (2020). changes to the work-family interface during the covid-19 pandemic: examining predictors and implications using latent transition analysis. journal of applied psychology, 105(10), 1073–1087. https://doi.org/10.1037/apl0000819 venkatesh, v., thong, j. y., & xu, x. (2012). consumer acceptance and use of information technology: extending the unified theory of acceptance and use of technology. management information systems quarterly, 36(1), 157–178. https://doi.org/10.2307/41410412 wang, y., & haggerty, n. (2011). individual virtual competence and its influence on work outcomes. journal of management information systems, 27(4), 299–334. https://doi.org/10.2753/mis0742-1222270410 australasian journal of information systems schoch 2023, vol 27, research article technostress and remote working performance 24 appendix – measurement items increased use of digital communication tools during covid-19 (based on: venkatesh et al., 2012) three-point likert scale: substantially less than before, remained the same, substantially more than before iuc01 how frequently are you using email for business purposes compared to before the corona pandemic? iuc02 how frequently are you using instant messaging (e.g., via ms teams, slack, whatsapp) for business purposes compared to before the corona pandemic? iuc03 how frequently are you using audio calls (e.g., via telephone, ms teams, skype) for business purposes compared before the corona pandemic? iuc04 how frequently are you using video calls (e.g., via ms teams, skype, zoom) for business purposes compared to before the corona pandemic? remote working self-efficacy (source: wang and haggerty 2011) rse01 i have confidence that i can complete my virtual work because i can access appropriate support staff readily. rse02 i have confidence that i can complete my virtual work because i can access information needed to perform my job. rse03 i have confidence that i can complete my virtual work because i can set objectives that align with the organisation’s goals. rse04 i have confidence that i can complete my virtual work because i can prioritise tasks to use my time effectively. challenge is use appraisal (source: lepine et al., 2016) ca01 using digital technologies to fulfill the demands of my job helps me improve my personal growth and well-being. ca02 i feel the demands of my job relating to the use of digital technology as a challenge to achieve personal goals and accomplishment. ca03 in general, i feel that the use of digital technology promotes my personal accomplishment. hindrance is use appraisal (source: lepine et al., 2016) ha01 using digital technologies to fulfill the demands of my job thwarts my personal growth and wellbeing. ha02 i feel the demands of my job relating to the use of digital technology constrain my achievement of personal goals and development. ha03 in general, i feel that the use of digital technology hinders my personal accomplishment. performance (source: frone et al., 1997) pf01 i am viewed by my supervisor as an exceptional performer. pf02 i am viewed as an exceptional performer in this organisation. pf03 i have a reputation in this organisation for doing my work very well. pf04 my colleagues think my work is outstanding. workload (source: copsoq iii / burr et al., 2019) wl01 do you have to work very fast? wl02 do you work at a high pace throughout the day? wl03 is it necessary to keep working at a high pace? techno-distress: techno-overload (source: ragu-nathan et al., 2008) to01 i am forced by this technology to do more work than i can handle. to02 i am forced by this technology to work with very tight time schedules. australasian journal of information systems schoch 2023, vol 27, research article technostress and remote working performance 25 to03 i am forced to change my work habits to adapt to new technologies. to04 i have a higher workload because of increased technology complexity. techno-distress: techno-invasion (source: ragu-nathan et al., 2008) ti01 i have to be in touch with my work even during my vacation due to this technology. ti02 i have to sacrifice my vacation and weekend time to keep current on new technologies. ti03 i feel my personal life is being invaded by this technology. techno-distress: techno-complexity (source: ragu-nathan et al., 2008) tc01 i do not know enough about this technology to handle my job satisfactorily. tc02 i need a long time to understand and use new technologies. tc03 i do not find enough time to study and upgrade my technology skills. tc04 i find new recruits to this organisation know more about computer technology than i do. tc05 i often find it too complex for me to understand and use new technologies. techno-distress: techno-insecurity (source: ragu-nathan et al., 2008) ts01 i feel constant threat to my job security due to new technologies. ts02 i have to constantly update my skills to avoid being replaced. ts03 i am threatened by coworkers with newer technology skills. ts04 i feel there is less sharing of knowledge among coworkers for fear of being replaced. techno-distress: techno-uncertainty (source: ragu-nathan et al., 2008) tu1 there are always new developments in the technologies we use in our organisation tu2 there are constant changes in computer software in our organisation. tu3 there are constant changes in computer hardware in our organisation. tu4 there are frequent upgrades in computer networks in our organisation. note: items measured on a five-point likert scale unless stated otherwise copyright: © 2023 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v27i0.3653 http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 theoretical background 2.1 sources of technostress 2.2 individual factors 2.3 appraisal and cognitive mediating processes 2.4 coping and adaptation 2.5 outcomes of the technostress process 3 hypothesis development 3.1 the influence of remote working self-efficacy 3.2 the influence of challenge and hindrance appraisal 3.3 control variables 4 quantitative empirical analysis 4.1 survey design and procedures 4.2 results 4.2.1 evaluation of the measurement model 4.2.2 evaluation of the structural model and hypotheses testing 5 discussion 5.1 theoretical implications 5.2 managerial implications 6 limitations and future research 7 conclusion paper title (paper title style) australasian journal of information systems richter, mohammadhossein, richter & trier 2025, vol 29, research article augmented reality and the metaverse received 04 sep 2024, accepted 19 may 2025 after 3 rounds of review with arif perdana as accepting senioreditor, m. davern & s. black as editors in chief 1 augmented reality and the metaverse – speculating about the future shahper richter business school university of auckland email: shahper.richter@auckland.ac.nz nastaran mohammadhossein school of information management victoria university of wellington alexander richter school of information management victoria university of wellington matthias trier chair of information systems, social computing paderborn university abstract imagine a world where the boundaries between physical and digital realities dissolve, creating immersive experiences that transform how we interact. this potential lies at the heart of the metaverse, and a confluence of technologies such as ar, vr, ai and blockchain will be the key to unlocking it. we employ a novel combination of horizon scanning and narrative development to explore the transformative role of ar within the metaverse. this approach reveals potential technology-driven futures, highlighting emerging trends and disruptions not readily apparent through traditional forecasting methods. our narratives of the future offer surprising glimpses into potential sociotechnical futures, informing a research agenda for both industry and academia. keywords: metaverse, future thinking, augmented reality, sociotechnical lens 1 introduction the metaverse is a continuously evolving virtual realm where users can interact and engage in diverse personal and social activities in an immersive environment at the crossroads of technological innovation and societal transformation (dolata & schwabe, 2023; richter & richter, 2023). it is a sociotechnical phenomenon, amalgamating various technical components such blockchain, virtual reality (vr), and artificial intelligence (ai). in this paper, we use augmented reality (ar) as a primary example to explore the disruptive potential of emerging technologies within the metaverse. ar technology's increasing integration into the metaverse is poised to redefine user interactions, acting synergistically with other transformative technologies to enhance functionalities and security (fu et al., 2022). however, our focus is not limited to this single technology. even if the all-encompassing vision of the metaverse may currently be less salient in the media and perhaps even becomes replaced with newer industry tags such as spatial reality, the many practical applications of ar remain one component in this continuing socio-technological change (anderson & rainie, 2022). while existing research explores the technical capabilities of ar and other technologies australasian journal of information systems richter, mohammadhossein, richter & trier 2025, vol 29, research article augmented reality and the metaverse 2 within the metaverse (chen et al., 2024; geroimenko, 2024), a significant gap exists in understanding how these technologies will shape future user experiences, particularly from a holistic, sociotechnical perspective. most studies explore the technologies in isolation or use current implementation data. we are using future thinking techniques to understand what these technologies may do. this is where our work fills in the current gaps in understanding. therefore, we examine the implications of the metaverse for organisations and society. our study highlights how ar and other technologies realise their full potential within this immersive digital realm. we use a forward-looking approach to understand the interplay of these disruptive technologies. horizon scanning allows us to identify early signals and emerging trends that may shape the trajectory of technology integration within the metaverse (cuhls, 2020). constructing future narratives (amer et al., 2013) helps us better understand the implications of those various developments and explore how different social, technological, and environmental factors may interact to shape user experiences and societal outcomes. as the metaverse continues to evolve, integrating different technologies will present unprecedented opportunities and potential challenges. early identification and understanding of these dynamics enable stakeholders to develop strategies that leverage technology benefits while mitigating risks. by offering insights that inform decision-making processes, we can ensure the integration of various technologies in the metaverse fosters innovation and responsible development. with our work, we aim to contribute to a more structured and actionable understanding of what the metaverse can become by focusing on ar technologies and their potential applications. by utilizing future-looking narratives, we help stakeholders, whether developers, policymakers, or users, identify challenges, opportunities, and transformative potentials. from a methodological standpoint, we demonstrate the systematic integration of horizon scanning and envisioning narratives as a futures-thinking research approach for is. in the next sections, we position the metaverse, and the use of ar in it, as a sociotechnical phenomenon (section 2), illustrates our methods (section 3), presents a narrative analysis (section 4), the results of this analysis (section 5), a discussion (section 6), research agenda (section 7) and conclusion (section 8). 2 background 2.1 the metaverse the concept of the metaverse, a virtual realm that coexists with the physical world, has acquired significant attention from technology commentators and academics (richter & richter, 2023). the idea of the metaverse was originally introduced in neal stephenson's 1992 novel "snow crash," where it is depicted as a virtual reality (vr) space that uses the internet and augmented reality (ar) through avatars and software agents (joshua, 2017). although there is no consensus on a definitive definition, damar (2021) succinctly defines it as "the layer between you and reality", acknowledging that the metaverse integrates physical, augmented, and virtual reality, and allows users to express themselves using digital identities. with the right infrastructure, it provides a converged experience for users in both the virtual and real worlds, which can be used for a variety of purposes, including games and social interactions (hollensen et al., 2022). ar, as an integral component, adds layers of realism to the metaverse, enriching user experiences through immersive interactions (wright et al., 2008). laeeq (2022) provides an illustrative example of the use of ar: “the metaverse is intended to use australasian journal of information systems richter, mohammadhossein, richter & trier 2025, vol 29, research article augmented reality and the metaverse 3 augmented reality at some point in the future. this is where you'll be able to see "holograms" of the metaverse against the background of the real world with ar glasses.”. while this beneficial interplay has been pointed out at such aggregate levels, there is so far no detailed picture emerging of how ar can precisely contribute to the proliferation of the metaverse. answering this question requires to understand the value of ar from a socio-technical perspective, which we are outlining in the next section. 2.2 the value of ar augmented reality (ar) traces its origins back to the 1950s, but its widespread consumer applications have emerged in the 21st century, primarily driven by the proliferation of mobile devices (ahmad & junaini, 2020; yuen et al., 2011). ar technology presents layered information on various portable screen devices, including stationary devices like ar mirrors, mobile devices such as smartphones, or wearable devices like ar glasses (flavián et al., 2019). according to magerkurth et al. (2005) “augmented reality is a variation on virtual reality that draws virtual objects into a real-world environment.” nonetheless, it is not virtual reality since it uses virtual images within a live, in-world environment. unlike virtual reality, which creates entirely immersive digital environments, ar overlays virtual images onto the physical world, offering users contextual information as they navigate real-world spaces (dwivedi et al., 2022). ar applications are increasingly used in a wide range of fields today, and they can be applied practically in almost every aspect of daily life (dargan et al., 2023). several researchers have recognized that ar applications provide new opportunities in different fields due to their allin-one nature, including images, animations, or videos (ahn et al., 2019; baroroh et al., 2020; bottani & vignali, 2019). for instance, games used to be played outside, but now they are played indoors using wearable headgear and other display technologies (buruk et al., 2024). ar can improve efficiency and accuracy in manufacturing by providing workers with realtime information and guidance (porter & heppelmann, 2017; tao et al., 2019). for example, ar can overlay instructions on machinery, allowing workers to quickly and easily understand how to operate it, and to make repairs and adjustments (eswaran et al., 2023). additionally, ar can create virtual mock-ups of products and simulate assembly processes, allowing engineers and designers to identify potential problems early in the design process (eswaran et al., 2023). in the domain of cultural heritage and tourism, ar enhances visitor experiences by providing interactive interpretations and re-engineering tourism encounters at historical sites and museums (yovcheva et al., 2014). in places like the acropolis in athens, greece, or the pyramids in cairo, egypt, this would provide unprecedented interpretation opportunities (dwivedi et al., 2022). as a result, brands and destinations can engage with visitors in realtime on location, revolutionizing their travel experience (buhalis & amaranggana, 2015; see et al., 2016). ar's impact extends into education, where it creates interactive and immersive learning experiences, enabling students to visualize complex concepts and explore virtual environments (akcayir & akcayir, 2017). the curiscope virtuali-tee, for instance, uses augmented reality to let students view the inside of the human body as if they were in an anatomy lab (gohd, 2017). this can make education more engaging and effective and can also help to improve the retention of information (majeed & ali, 2020). australasian journal of information systems richter, mohammadhossein, richter & trier 2025, vol 29, research article augmented reality and the metaverse 4 moreover, ar technology has made significant strides in the medical field, offering comprehensive solutions for medical education, surgical planning, and patient management (bagwe et al., 2021). surgeons can utilize ar to simulate complex procedures, receive real-time guidance during operations, and train medical students in hands-on environments (hampiholi, 2023). in retail, ar enhances the shopping experience by offering dynamic 3d animations and realistic interfaces, allowing customers to interact with products virtually before making purchases (scholz & smith, 2016). retailers leverage ar to enable virtual try-ons for clothing and visualize furniture in real-world settings, providing customers with enhanced confidence and convenience in their purchasing decisions (bonetti et al., 2018; pachoulakis & kapetanakis, 2012). in the metaverse, ar brings transformative possibilities but also unique challenges. its integration requires advanced technology, which can be costly and complex (rabbi & ullah, 2013). user acceptance is crucial (sumadio & rambli, 2010), demanding intuitive and engaging experiences. privacy and ethical concerns (blodgett-ford & supponen, 2018; neely, 2019) surrounding data usage must be addressed to ensure responsible deployment. therefore, while ar offers immense potential in the metaverse, success relies on overcoming challenges and navigating ethical considerations. a socio-technical approach can guide stakeholders in designing inclusive, innovative ar solutions aligned with societal values. 2.3 a sociotechnical perspective on ar in the metaverse after having introduced the technological foundations and value of ar and the metaverse we need a suitable theoretical lens to emphasize how users in their social contexts make use of the technology to realize its value. the sociotechnical lens helps to understand user engagement by considering the social context in which ar technology is used (sarker et al., 2019). as technologies like ar directly alter our perception of the real world by overlaying digital information, they reshape how humans interact with their surroundings. this is particularly salient in the realm of ar, which serves as a bridge between the digital and physical realms. the very essence of ar and the metaverse is the mutual appropriation of social structures and technical functionalities, where both elements are "constitutively entangled" (orlikowski, 2007). in the context of the metaverse, the sociotechnical view can be used to understand how the design and functionality of ar technology impact user behaviour. for example, the level of interactivity and immersion provided by ar technology can influence user engagement and motivation to use it (shin, 2019; yim et al., 2017). additionally, it helps us to understand how the technical design of ar technology can impact the potential for deviant behaviours (li et al., 2022), such as how anonymity and lack of physical cues in the metaverse can contribute to deviant behaviour (dwivedi et al., 2022), necessitating continuous design and redesign based on human engagement. this iterative process, as emphasized by kallinikos et al. (2013), ensures that technology evolves in tandem with societal needs and values. this evolution is underpinned by the philosophy of human-centered design, which underscores the importance of focusing on individuals and groups when crafting technologies. by doing so, we ensure that emergent technologies are not only functional but also user-friendly and aligned with human values. australasian journal of information systems richter, mohammadhossein, richter & trier 2025, vol 29, research article augmented reality and the metaverse 5 recognizing and understanding these systemic interactions in a larger system that encompasses human actors, societal norms, and other technologies (sarker, chatterjee, et al., 2013), is crucial for the successful integration and widespread adoption of new technologies like ar. this interconnectedness means that sociotechnical systems are dynamic entities. as society evolves, so too will the technologies it adopts and the manners in which they are utilized. for example, the sociotechnical view can be used to understand how social norms and expectations in the metaverse shape user behaviour, such as how the perceived anonymity of the metaverse can lead to deviant behaviours. additionally, the sociotechnical helps to understand how the social context in which ar technology is used in the metaverse can impact user engagement (scholz & smith, 2016), such as how the presence of a supportive community can increase user engagement and motivation to use ar technology in the metaverse. investigating the metaverse from a sociotechnical viewpoint poses the challenge that many practices have not yet materialized or matured. our methodological approach offers a detailed and holistic structure that allows us to imagine a wide range of potential futures. specifically, it facilitates the exploration of diverse narratives in which technology and societal factors might develop along various paths. by deeply understanding the intertwined nature of technology and society, those at the forefront of innovation and policymaking are better equipped to make informed decisions. their choices can then reflect a balance between technological advancements and societal expectations. in the context of emerging technologies like ar, this means ensuring that their design, development, and implementation align seamlessly with societal values, meeting the broader needs and aspirations of the community. 3 methods 3.1 horizon scanning given the exploratory nature of the metaverse and the rapidly evolving capabilities of ar technologies, it is essential to adopt a methodology that can navigate uncertainty and identify emerging trends. horizon scanning is ideally suited for this purpose, as it allows researchers to identify potential future narratives, opportunities, and challenges in a given context (flick et al., 2020). web-based horizon scanning systematically gathers information from various sources, including academic research, industry reports, white papers, and the social media discourse, embracing a wide array of insights from both empirical and speculative perspectives (nemorin et al., 2023; washida & yahata, 2021; welz et al., 2021). given the nascent and rapidly developing nature of the metaverse and ar technologies, the use of horizon scanning allows for a structured approach to gather data from diverse sources and develop narratives of the future. the identification of new issues through horizon scanning is especially valuable in such an unexplored context. similar approaches have been effectively applied to anticipate future developments in areas such as the future of work and artificial intelligence. for instance, jetha et al. (2021) utilized horizon scanning to examine the impact of evolving work conditions on vulnerable workers, while nemorin et al. (2023) employed the method to explore discourse on ai in education and development. horizon scanning at the beginning of a forward‐looking activity is applied for identifying “things to come,” often new science and technology (conway, 2021). horizon scanning involves a systematic examination of potential opportunities, threats, and trends across a wide range of domains, while narrative planning constructs multiple plausible narratives of the australasian journal of information systems richter, mohammadhossein, richter & trier 2025, vol 29, research article augmented reality and the metaverse 6 future. the use of narrative development allows a more in depth and rigorous approach to the identification and impact of these trends, by enabling a detailed analysis of potential consequences. by combining these approaches, the study aims to inform decision-making processes and enhance organizational adaptability in an ever-evolving global landscape. for this study, we operationalised horizon scanning by establishing a structured framework that detailed the specific types of sources to be reviewed, the criteria for information relevance, and the methods for data extraction and analysis. they included academic journals at the forefront of exploring the metaverse, business magazines, reports from business forums and white papers, posts, podcasts, and videos by company evangelists and academics on social media like linkedin and youtube videos, and technology blogs and newspaper publications. each source was scrutinised using a set of predetermined criteria to ensure consistency and rigor in the data collection process. as a result, we identified, analysed, categorized, and synthesized more than 200 sources of information on ar in the metaverse. based on this analysis, we created future narratives based on these trends. horizon scanning is useful for identifying emerging issues, but synthesizing the information to understand key driving forces is even more valuable (cuhls, 2020; rowe et al., 2017). 3.2 narratives of and for the future narrative development in futures studies is a powerful approach that combines the crafting of compelling stories with the insights gained from horizon scanning to explore potential future outcomes. this process transforms abstract data and emerging trends into coherent narratives, making complex future possibilities more tangible and relatable for stakeholders (burnam-fink, 2015). as schoemaker (1995) argues, detailed and realistic narratives can direct attention to latent trends, wildcards, and weak signals that might otherwise be overlooked. this combination of narrative development and horizon scanning offers several key benefits, including enhanced visualization and engagement, improved decision support, and strategic foresight enhancement (ringland, 2010). narratives serve as a crucial bridge between scenarios and action, transforming passive anticipations into active engagement by providing a structured framework for decisionmaking. by enabling stakeholders to "visit" potential futures through these rich narratives, organizations can extend their understanding and actively shape the future (schwarz, 2015). this approach allows decision-makers to more effectively envision and engage with and comprehend possible futures, fostering a deeper understanding of potential challenges and opportunities (inayatullah, 2008). the integration of narrative development with horizon scanning creates comprehensive decision support products, bridging the gap between raw data and actionable insights (iden et al., 2017). furthermore, narratives derived from horizon scanning results serve as powerful tools for communicating complex future narratives to diverse audiences, facilitating more effective strategic conversations and collaborative decision-making processes (curry & schultz, 2009). 3.3 5 step process washida and yahata (2021) found that horizon scanning-based narratives provided significantly higher predictive value than narratives prepared using conventional methods. in our approach, each identified trend from the horizon scanning was evaluated for its potential impact and likelihood, enhancing the predictive value of the narratives developed. using web-based horizon scanning when creating narratives is especially useful as it can provide australasian journal of information systems richter, mohammadhossein, richter & trier 2025, vol 29, research article augmented reality and the metaverse 7 evidence for the narratives and ensure they align with present circumstances and strategy, rather than just presenting an opportunity for speculative thinking about a future disconnected from current relevance (rowe et al., 2017; schoemaker et al., 2013) we followed a five-step process (figure 1), starting with defining our search criteria and keywords, moving to automated and manual scanning of diverse forms of data, combining our findings into a comprehensive database and finally utilising this data to generate exemplary narratives (cuhls, 2020; washida & yahata, 2021). this approach was informed by hermeneutic principles (boell & cecez-kecmanovic, 2014) and designed to foster a comprehensive and nuanced understanding of ar in the metaverse. the steps taken ensured that the methodology was not only described but also clearly linked to how it was concretely applied in the research effort, emphasizing the practical execution and the rigorous analysis conducted. figure 1: horizon scanning and narrative development – a 5 step process (authors’ own creation) step 1: defining search criteria and terms defining our search criteria and selecting relevant keywords was crucial for setting a clear direction for the following scanning processes. it was guided by the hermeneutic principle of establishing interpretive criteria based on key concepts "augmented reality," "metaverse," and "applications of these technologies." the search was conducted across academic databases, including ieee xplore, acm digital library, and web of science, industry reports from market analysis companies (e.g., gartner, forrester), white papers from research australasian journal of information systems richter, mohammadhossein, richter & trier 2025, vol 29, research article augmented reality and the metaverse 8 organizations, and public social media platforms (e.g. reddit, x) . we employed search strings such as ("augmented reality" or "ar") and ("metaverse" or "virtual world") and ("user experience" or "interaction" or "engagement") and ("societal impact" or "ethical issues" or "future trends"). this deliberate and thoughtful preparation ensured a targeted yet comprehensive exploration of the field. step 2: automated and manual scanning of diverse data sources our horizon scanning process employed both automated tools and manual efforts. automated methods sifted through academic databases, industry reports, white papers, and social media platforms, identifying content that matched our predefined criteria and keywords. simultaneously, we undertook manual reviews to synthesize the gathered information, applying expert judgment to interpret complex data and evaluate sources critically. the inclusion criteria for our automated and manual scans stipulated that sources be published in english between 2020 and 2025 to remain current, focus on the application of ar within the metaverse, address societal impact or ethical issues, discuss future trends, or describe technological innovation. sources that focused solely on virtual reality, or did not address future impacts were excluded. the combination of automated and manual methods allowed us to leverage the strengths of both approaches, optimizing our data collection process. this step embodied the hermeneutic practice of engaging with diverse knowledge sources, facilitating a dynamic fusion of horizons where our interpretations of "relevance" and "credibility" continuously evolved (gadamer, 2013). step 3: combining findings into a comprehensive database the integration of our findings into a comprehensive database was done by organizing information from over 200 sources into categories such as technological innovations and societal implications. this step was instrumental in identifying significant trends, patterns, and uncertainties, enabling us to develop categories that would later inform our narrative generation. the methodological rigor of this phase is underscored by our selection of 45 exemplary references, included as endnotes in the analysis section, step 4: developing categories for narrative generation we proceeded to develop categories that served as inputs for generating exemplary narratives. this analytical phase allowed us to synthesize the vast array of data collected, drawing upon key drivers of change and uncertainties to craft speculative yet grounded projections of the future of ar in the metaverse from a sociotechnical perspective. this step exemplified the hermeneutic principle of iterative engagement and reflection, as our understanding of the data and its implications for ar in the metaverse continued to refine and deepen. step 5: generating narratives finally, we utilized the synthesized data to generate narratives, illustrating potential futures of ar in the metaverse. these narratives provided a conceptual framework for exploring the complexities and potential developments within ar and the metaverse, embodying the hermeneutic approach of achieving deeper comprehension through the integration of diverse perspectives. the generation of these is described in more detail in the next section. 4 categories for narrative generation in the categories for narrative generation, we focus on customers as a particular user role. their activities are typically categorized into a customer journey model with three stages. by utilizing the customer journey model—encompassing pre-purchase, purchase, and postaustralasian journal of information systems richter, mohammadhossein, richter & trier 2025, vol 29, research article augmented reality and the metaverse 9 purchase phases—we were able to align ar technological capabilities with specific customer interactions and expectations at each stage. this alignment provided a structured approach to synthesizing the vast array of data collected, drawing upon key drivers of change and uncertainties to craft speculative yet grounded projections of the future of ar in the metaverse from a sociotechnical perspective. a customer journey is a sequence of steps that a customer goes through to access or use a company's offerings (følstad & kvale, 2018). it puts the customers' perspective at the center of all design considerations. analyzing customer journeys can help to understand key points in the customer experience and how they impact the customer's overall experience (rawson et al., 2013). briefly, these stages differ regarding the goals and activities companies pursue within each step. the pre-purchase stage includes customer interactions before the purchase, such as identifying needs, discovering a product or brand, searching for information, and evaluating alternatives. customers may be motivated by utilitarian or hedonic factors (mikalef et al., 2012). they may rely on extrinsic cues, such as brand credibility and social sources to reduce information uncertainty (ho et al., 2015). the purchase stage involves customer interactions with a platform during the actual purchase event, including processes related to choice, ordering, and payment (lemon & verhoef, 2016). customers may experience choice overload or be dissatisfied with their decision, potentially leading to abandoned or postponed searches or purchases (wang & shukla, 2013). the postpurchase stage encompasses customer interactions with a platform after the purchase, including usage, consumption behavior, post-purchase engagement, and word-of-mouth (wom) through reviews, among other things (lemon & verhoef, 2016). for each phase we distinguished between the driver (the technological changes) and the uncertainty (or what users are weary of) and together used these to derive the hypothetical narrative (cf. figure 2). figure 2: drivers and uncertainties in each phase as ar and metaverse technologies continue to evolve, advancements in hardware design and manufacturing have significantly reduced costs, making immersive experiences more accessible to a broader range of users (nair et al., 2024). the increasing availability of lightweight ar glasses and mobile-based applications has expanded participation beyond early adopters, allowing everyday consumers to engage with ar-enhanced experiences in personal and professional contexts (pavlik & bridges, 2013; singh et al., 2025). this democratization of technology ensures that the benefits of ar, such as personalized shopping, australasian journal of information systems richter, mohammadhossein, richter & trier 2025, vol 29, research article augmented reality and the metaverse 10 secure transactions, and enhanced post-purchase experiences, are no longer limited to premium markets but can support inclusive and diverse user groups. 4.1 pre-purchase phase – predictive personalization driver: recent advancements in surveillance technologies (flysight, 2024) and ar combined with facial recognition (lun & iqbal, 2024) demonstrate significant progress in the field. for instance, kleinings (2023) outlines various applications of ai in business, while cropley et al (2023) discusses the creative potential of algorithmic design. furthermore, the integration of autonomous supply chain planning (hendricksen, 2023; sharifmousavi et al., 2024) and the capability of robots to interpret human emotions (stock-homburg, 2022) highlight the practical applications of these technologies. the improvements in computational experience through ai-enabled task offloading (rawlley et al., 2025) and the development of intelligent visual-iot systems (el ghati et al., 2024) further underscore the importance of these advancements. finally, the future potential of ar and ai is evident in the innovative approaches discussed by devagiri et al (2022). uncertainty: recent discussions on the metaverse have highlighted various privacy concerns and technological advancements. for instance, kaur et al (2024) explores how privacy is taken into account in ar/vr experiences, while al falahi et al (2024) warns of serious privacy risks for users within the metaverse. okhifun (2022) demonstrates how ai can mitigate workplace stress, complementing parlar (2023) insights on data privacy and security in the metaverse. moreover, richter & richter (2024) touch on the use of self-aware agents to analyze public selfconsciousness in complex narratives like the iterated prisoner's dilemma. crum & coglianese (2024) highlights ai's potential to understand and interpret body language, which is further supported by ergen (2021) examination of ai applications in event management and marketing. goher (2024) outlines the benefits of ai-driven scheduling, emphasizing the efficiency gains in various sectors. additionally, xu et al,(2022) discuss broader security and privacy concerns in the metaverse, underscoring the importance of comprehensive protective measures in these digital spaces. narrative development: in the pre-purchase phase, predictive personalization uses ar to enhance user engagement by adapting product visualizations based on real-time emotional and physiological feedback (v. kumar et al., 2024; valmorisco et al., 2024). this narrative was developed by considering how customers explore options and make decisions (koo et al., 2021). the integration of ar with ai analytics caters to personalized shopping experiences, a critical factor as users seek more tailored interactions in digital spaces (agriyanto, 2024; mohamed et al., 2024; venkata et al., 2024). 4.2 purchase phase – blockchain-powered ar transactions driver: in the purchase phase, the integration of blockchain technology for secure, transparent transactions is a significant driver. this approach ensures that transactions within the metaverse are both reliable and verifiable (ghosh et al., 2024; pragha et al., 2024; rafique & qadir, 2024). additionally, the use of blockchain in augmented reality platforms further enhances the security and transparency of digital transactions (logeswaran et al., 2024). another key driver is the ability of blockchain to create a secure and efficient virtual world. this capability is crucial for enabling a seamless purchase experience in the metaverse (al khaldy et al., 2024; aldweesh et al., 2023). the combination of these technologies supports the australasian journal of information systems richter, mohammadhossein, richter & trier 2025, vol 29, research article augmented reality and the metaverse 11 creation of a trustworthy environment for users, thereby boosting their confidence in making digital-only transactions (ng, 2023). uncertainty: scalability of blockchain solutions and user trust in digital-only transactions. a significant uncertainty is the scalability of blockchain solutions and user trust in digital-only transactions. blockchain and its derived technologies are shaping the future of digital businesses, particularly in decentralized finance and the metaverse (far et al., 2023). however, the integration of these technologies into mainstream commerce poses questions about their scalability and reliability (ghosh et al., 2024). another uncertainty is the impact of artificial intelligence on event experiences and user interactions within the metaverse. the application of ai in enhancing event experiences through narrative techniques highlights both opportunities and challenges in user adaptation (neuhofer et al., 2021). the comprehensive review of blockchain technology and virtual asset accounting in the metaverse also underscores the complexities involved in establishing a trustworthy and efficient virtual economy (uddin et al., 2024). furthermore, the integration of blockchain within the industrial metaverse introduces additional layers of uncertainty regarding the seamless operation and user trust in these systems (mourtzis, 2024). addressing these uncertainties is crucial for fostering user confidence and ensuring the smooth adoption of these advanced technologies. narrative development: the purchase phase narrative leverages ar and blockchain to create a seamless and secure transaction environment. this development was motivated by the need to enhance user confidence during online purchases (patel, 2024). by embedding blockchain within ar interfaces, we address both the technical capability of ar to provide immersive interactions and the social requirement for trust and security in financial transactions (logeswaran et al., 2024). the immersive experience of augmented reality, such as trying a virtual meal, exemplifies the potential of ar to engage users in innovative ways (cowan et al., 2024). the integration of ai within organizational frameworks also supports the collaborative environment necessary for effective business operations (chowdhury et al., 2022). understanding blockchain trends and their application to business practices further ensures that these technologies are effectively implemented (gad et al., 2022). additionally, comprehensive guides on the metaverse provide essential insights into the broader context of these technological advancements (fadhel et al., 2024). 4.3 post-purchase phase – enhanced product integration driver: evolution of ar capabilities to interact with physical environments. the evolution of ar capabilities to interact with physical environments is a key driver. this interaction is vital for helping users integrate and utilize their purchases effectively (ebrahimabad et al., 2024). the potential of ar in enhancing user experiences through immersive technology is significant (baxter & hainey, 2024). simulation optimization techniques, such as kriging, contribute to improving the efficiency and effectiveness of ar applications in real-world narratives (wei et al., 2023). additionally, the long-term vision of the metaverse and its impact on future interactions highlight the importance of continuous development in ar and ai technologies (rainie & anderson, 2024). the role of robots, ai, and service automation in postpurchase narratives is also crucial for enhancing customer support and interaction (smith et al., 2023). advanced applications of vr in medical fields, such as neurosurgery, demonstrate the broad potential of immersive technologies (chang & wu, 2024). furthermore, the australasian journal of information systems richter, mohammadhossein, richter & trier 2025, vol 29, research article augmented reality and the metaverse 12 exploration of 3d virtual worlds and the metaverse indicates the ongoing advancements and future possibilities in this domain (uddin et al., 2024; zalan & barbesino, 2023). uncertainty: user adaptability to augmented spaces in personal environments presents a significant uncertainty. acceptance of the metaverse, as explored through laboratory experiments on augmented and virtual reality shopping, indicates varying levels of user acceptance and adaptation (xi et al., 2024). the complexity of metaverse digital environments, including the techniques, technologies, and applications, further contributes to this uncertainty. the potential opportunities within the metaverse are vast but realizing them depends on user adaptability and technological integration (allam et al., 2022). marketing within the metaverse also faces uncertainties related to how these environments will shape future consumer research and practice (dwivedi et al., 2022). the integration of virtual assistants in event planning and the overall impact of the metaverse on human resources highlight the broader implications of these uncertainties (fadhel et al., 2024). narrative development: in the post-purchase phase, ar assists users in integrating and utilizing their purchases through interactive, context-aware guides and setups. this narrative was crafted considering the need for ongoing engagement and support after a purchase, ensuring that ar adds value beyond the initial transaction by enhancing the usability and enjoyment of products (flavián et al., 2024). ar has been shown to significantly impact the customer journey by providing an immersive experience that extends beyond the point of sale (romano et al., 2021). the role of ar in marketing is increasingly recognized for its ability to create interactive and engaging customer experiences (rauschnabel et al., 2022; tom dieck & han, 2022). for instance, ai for events demonstrates how ar can be incorporated to enhance participant engagement and experience (sung et al., 2021). developing a theoretical framework for augmented reality marketing helps to understand its potential and applications in retail environments (h. kumar et al., 2024; rehman et al., 2024). the ability of ar to transform how consumers interact with their purchases is further highlighted by applications in virtual dining and exploration, providing users with a rich, immersive experience (chiu et al., 2021). the exploration of virtual worlds through ar also contributes to ai learning in new ways, enhancing the overall user experience (verhulst et al., 2021). moreover, transforming customer experience using ar/vr technologies demonstrates the potential for these tools to add significant value post-purchase (patel, 2024). partnering with ai, as seen in digital productivity assistants, showcases how ar can support ongoing user engagement and productivity (cranefield et al., 2023). additionally, marketing research on mobile apps emphasizes the evolving nature of ar technologies and their impact on future marketing strategies (rauschnabel et al., 2022; stocchi et al., 2022). 4.4 reflection the analytical phase allowed us to apply the hermeneutic principle of iterative engagement and reflection. our understanding of the data and its implications for ar in the metaverse continued to refine and deepen throughout the process. by linking these narratives to specific phases of the customer journey, we ensure that each narrative addresses distinct aspects of user interaction, reflecting both current technological trends and potential future developments. this approach not only aids in envisioning how ar might evolve within the metaverse but also grounds these developments in practical, user-centered applications that are critical for widespread adoption and satisfaction. australasian journal of information systems richter, mohammadhossein, richter & trier 2025, vol 29, research article augmented reality and the metaverse 13 in this section, we present narratives aiming to offer a comprehensive understanding of the potential new approaches to integrating ar into the metaverse derived from rigorous analysis of 45 exemplary references collected through the horizon scanning approach. employing an expanded sociotechnical lens (sarker et al., 2019; sarker, xiao, et al., 2013), not only captures the dynamic interplay between ar technology and its users but also situates it within broader societal contexts, offering a holistic view of ar's potential in the metaverse. in order to derive a more realistic narrative we orient our discussion of ar use around a specified user persona. personas serve as representations of various stakeholder roles, assisting in guiding the information design process (reeder & turner, 2011). they outline the characteristics of individuals who will engage with an information system. as a valuable reference tool, personas aid designers in their work. when combined with narrative-based design, they become a powerful method for enhancing the quality of information designs (jansen et al., 2022). user persona: sarah is a 22-year-old environmental science student at metaverse university (mu). as digital native sarah is comfortable with vr and ar technology. she values flexibility and the ability to control her learning experience and prefers virtual environments where she can network with other professionals in her field. 5 narratives of the future of ar in the metaverse 5.1 pre-purchase phase: predictive personalization with ar sarah has recently joined a new virtual living community within the metaverse, designed to mimic a bustling modern city. as she navigates her virtual apartment through her avatar, she decides it needs a new sofa to enhance the space's aesthetic and functionality. sarah puts on her ar-enabled visor in her physical space, which seamlessly connects with her virtual environment in the metaverse. as she explores different sofas in a virtual furniture store, the ar system integrated with advanced ai syncs with her neural interface. this interface reads her subconscious signals and micro-expressions to understand her preferences. in her realworld space, she sees virtual sofas overlaid in her physical room, adjusted for scale and style according to her virtual apartment's dimensions. when she pauses on sleek, modern designs, the virtual system dynamically adjusts the options, highlighting similar styles that appear in her virtual living room. having recently upgraded to a more affordable ar visor, sarah appreciates how accessible the technology has become, allowing her to engage in immersive experiences without premium equipment. she recalls how, until recently, such experiences were limited to highend devices, but now lightweight ar glasses and mobile applications have made virtual shopping widely available, allowing her to personalize her space with ease. noticing her focus on a particular plush, navy-blue sofa, the system in the metaverse enhances the texture and comfort level to appear softer, responding to sarah’s neural feedback indicating a preference for comfort. she can see and feel the adjustments in real-time through haptic feedback gloves, providing a tactile experience as if touching the sofa. thrilled by this deep level of interaction, sarah confirms her choice. the sofa not only fits perfectly in her virtual space but also enhances her sense of presence within the digital world, making her virtual home feel more inviting and personalized. australasian journal of information systems richter, mohammadhossein, richter & trier 2025, vol 29, research article augmented reality and the metaverse 14 5.2 purchase phase: seamless and secure transactions satisfied with her virtual sofa selection in the metaverse, sarah is ready to finalize her purchase. within the digital environment, she uses a gesture to bring up the purchase interface. instantly, a 3d holographic contract projects right in front of her avatar, appearing in the virtual living room she has been decorating. the contract details—including terms, warranties, and the simulated delivery timeline—are displayed in a user-friendly, interactive format. she navigates the contract using voice commands within the metaverse, confirming that the virtual delivery aligns with her preferences in the digital world. to proceed with the purchase, sarah authorizes the transaction via a retina scan performed by her ar visor in the real world, which seamlessly integrates with her digital actions. this duallayer authentication, leveraging blockchain technology, ensures that the agreement is not only secure but also intrinsically bound to her identity across both realms, significantly reducing the risk of fraud. this method provides a unique blend of real-world security measures with metaverse transactions, enhancing her confidence in the system's transparency and reliability. the blockchain technology records her purchase within the metaverse, providing an immutable ledger that guarantees all terms are honored as agreed, both virtually and in any linked real-world interactions. this transaction showcases how ar and blockchain can create a fluid, secure bridge between digital and physical experiences, ensuring that virtual goods and services in the metaverse carry tangible credibility and legal weight. 5.3 post-purchase phase: integration into daily life after the virtual delivery of her new sofa in the metaverse, sarah is eager to explore its capabilities. she dons her advanced ar glasses, which bridge her physical living space with her virtual environment. the ar system scans both her real apartment and her virtual space, projecting multiple furniture layouts that optimize physical and virtual factors like lighting, space utilization, and even digital feng shui principles. sarah views these layouts overlaid in her actual living room and in her virtual environment simultaneously, allowing her to make informed decisions that enhance both her real and virtual living spaces. with the sofa perfectly positioned in both dimensions, sarah delves into its interactive features. through her ar glasses, she controls various aspects of both her physical home and her virtual environment directly from her sofa. a virtual control panel materializes around the sofa in the metaverse, enabling her to adjust lighting, play music, and manage smart home devices across both realms with intuitive gestures and voice commands. for movie nights, the ar system synchronizes settings between her physical and virtual worlds, dimming lights and adjusting surround sound to create an immersive atmosphere that reflects the genre of the movie being watched. in the next section we step through how each narrative or narratives elucidate the interaction or fit between social aspirations and technical capabilities, the reciprocal influences between these domains, and the sociometrical amalgamation that these technologies foster in educational, corporate, and entertainment settings. these narratives serve not only as demonstrations of current possibilities but also as projections of ar's evolving role in shaping future social and technical landscapes. 6 discussion the narratives assist in imagining possibilities of ar's evolving role in shaping the future of the metaverse. utilising horizon scanning, we synthesize recent technological developments australasian journal of information systems richter, mohammadhossein, richter & trier 2025, vol 29, research article augmented reality and the metaverse 15 to envision the potential futures that are designed as artifacts from the future. against viewing the future of is as a linear progression from current states. instead, our research contributes to the dialogue on how to foster and manage emerging forms of ar in the metaverse through a "doing futures" approach hovorka & peter (2021). our narratives unfold in a structured manner, progressively revealing complex dialogues about the implications of these technologies. this staged presentation aims to illuminate aspects of future interactions that might remain hidden without such a detailed exploration. reflecting on our use of horizon scanning and narrative creation, we recognize the tension between the need for rigour and the inherent creativity of speculative futures in is research. by maintaining transparency in our methods, we balance these demands, proposing that such futures can form a new, vital domain within academia. these narratives inform our understanding of the sociotechnical dynamics that ar will navigate and mould within the metaverse and, more broadly, in the real world. in the context of ar within the metaverse, ensuring a harmonious fit between social objectives and technical capabilities is crucial. the predictive personalization of ar during the pre-purchase phase represents an advanced integration where technology adapts based on the user’s emotional and physiological responses. this narrative mirrors the notion explored by morris and venkatesh (2010) where technology adjusts based on the user's implicit feedback, enhancing satisfaction and engagement. this adaptive technology fits within the organizational context of enhancing consumer experience, reducing the misfit that strong and volkoff (2010) caution against in domains where organizational objectives and technical systems might otherwise clash. this enhanced virtual experience may also translate to increased sales and customer loyalty in physical stores. furthermore, the data gathered in these personalized pre-purchase narratives can be used to develop more accurate customer profiles, which can be leveraged to improve the entire retail experience in both virtual and real-world stores. the reciprocal nature of interactions between social structures and technological artifacts is evident in the purchase phase narrative involving blockchain-powered ar smart contracts. here, the technology not only supports but enhances the purchasing process by integrating secure, transparent contract management directly within the user interface. this reflects the structurational model of technology (jones & karsten, 2008; orlikowski, 1992; poole & desanctis, 2004) where the technology reshapes the traditional purchasing routines (a social structure) and, conversely, the existing norms of purchasing influence how the technology is used and perceived. moreover, the adoption of blockchain in ar for smart contracts could streamline real estate and other large item purchases by offering a more transparent and secure alternative to paper-based contracts and reducing the need for third party intermediaries. the post-purchase phase, where the ar system integrates the sofa into different aspects of sarah’s daily activities, epitomizes the sociomaterial entanglement of the social and the technical. this narrative shows how ar does not simply exist alongside social activities but becomes an intrinsic part of daily routines, influencing and enhancing personal and recreational activities. the technology and social practices evolve in tandem, demonstrating the "constitutive entanglement" described by orlikowski (2007), where social and technical dimensions are not merely interacting but are fundamentally interwoven. this could lead to greater consumer engagement with their products, reduce waste through enhanced information access about product care, and influence the development of new products by offering insights into user behaviour. australasian journal of information systems richter, mohammadhossein, richter & trier 2025, vol 29, research article augmented reality and the metaverse 16 6.1 inscription of social considerations within the technological artifact throughout the narratives, social considerations such as user experience, ergonomic health, and personalized engagement are deeply inscribed within the technological artifacts. these considerations reflect the embedding of "kernel theories" from social sciences into the design of ar technologies, as suggested by walls et al. (1992). for example, the predictive personalization uses psychological insights to enhance user satisfaction, and the ergonomic suggestions in the post-purchase phase draw upon health sciences to improve physical wellbeing. this design philosophy ensures that the ar technology is not only functional but also attuned to human needs and values, illustrating how technological designs can encapsulate broader social objectives. this approach not only enhances virtual experiences but also sets a standard for more human-centered design in all technological development, and demonstrates how insights into human psychology can improve the real-world experience of a product. ar can play a significant role in the metaverse by enhancing the user's experience and immersion in virtual environments. ar technology can be used to overlay virtual objects and information on the real world, allowing users to interact with digital content in a more natural and intuitive way (billinghurst & kato, 2002; bower et al., 2014; kesim & ozarslan, 2012). for example, the added interactivity of augmented books makes them a strong advantage over print-only books. the book can be rotated or tilted to see the virtual content from different viewpoints, or the pages can be flipped to view different ar scenes (billinghurst & duenser, 2012). our predictive personalization narrative during the pre-purchase phase, where ar adapts product designs in real-time based on user reactions, illustrates the advanced potential of ar to dynamically align with user preferences, enhancing decision-making in shopping experiences. this also has implications for physical retail stores, as businesses may utilize data from these ar interactions to design better store layouts and product displays. the predictive personalization also offers an opportunity for retail to be more ethical. by reducing product returns (and therefore waste) companies that utilise this tech are demonstrating a commitment to sustainability. this exemplifies an excellent fit between technical capabilities and social objectives, where ar technology adeptly meets user needs in a personalized manner (interaction or fit between the social and the technical). additionally, ar can be used to create more realistic (wu et al., 2013) and dynamic virtual environments (salinas & gonzález-mendívil, 2017), making the metaverse more immersive and believable. furthermore, immersive virtual environments offer real-world benefits including better training for doctors and specialists, in a safe and remote environment. in a realistic and fun environment, immersive simulations provide medical specialists and students with a great opportunity to enhance their skills and gain new experiences. moreover, it provides a safe and effective way to train remotely. the blockchain-powered ar smart contracts in the purchase phase represent a reciprocal interaction between social and technical systems. this technology not only secures transactions but reshapes the commercial landscape by embedding transparency and trust directly into the transactional processes, illustrating how ar technology can adapt to enhance and secure commercial practices (reciprocal interactions between the social and the technical). the implementation of such a system could simplify complex real-world purchases and transactions, including real estate and other large assets, by reducing the need for third party intermediaries. in the post-purchase phase, ar assists users in integrating and setting up their products. this narrative, where ar provides real-time ergonomic guidance, australasian journal of information systems richter, mohammadhossein, richter & trier 2025, vol 29, research article augmented reality and the metaverse 17 showcases the sociomaterial intertwining of ar with daily living, changing how users interact with products. by providing enhanced information about the maintenance and use of products, this type of system also extends the life of products and reduces waste. therefore, the use of ar technology has implications for sustainability and resource management. the technology is fundamentally embedded within user activities, altering the traditional setup processes and enhancing user comfort and product usability. ar can enhance the social aspect of the metaverse by allowing users to interact with each other in more realistic and immersive ways (hennig-thurau et al., 2023). this has implications for the real world, including new opportunities for remote collaboration, the creation of more personalized and social e-learning environments, and new methods for communicating across cultural barriers. one way ar can enhance social interactions in the metaverse is by allowing users to see and interact with each other's avatars in a more lifelike way (hennig-thurau et al., 2023; kim et al., 2023; phakamach et al., 2022). recent advancements in ar technology have significantly improved the integration and functionality of avatars in ar experiences, enabling the overlay of photorealistic 3d avatars onto the user's view of the real world, creating more lifelike interactions that closely resemble face-to-face encounters (buhalis et al., 2023). these avatars can be animated in real-time to mimic the user's movements and expressions, enhancing the sense of presence and immersion. the integration of artificial intelligence (ai) with ar avatars has further elevated their capabilities, enabling them to understand and respond to user interactions naturally, adapt their personalities based on context, and tailor communication styles to individual user preferences (liu & siau, 2023; ramya & ramamoorthy, 2025). this tech can also be used in the real world for remote mental health support, job interviewing and improving remote customer service by making remote interactions feel more real. each detailed narrative underscores how ar can be thoughtfully designed to align with, enhance, and transform human interactions within various settings, providing practical examples that illustrate the profound impact of ar when integrated within a socio-technical framework. these applications highlight how ar technologies are designed with a keen adherence to enhancing user engagement and interaction, promoting a rich, integrated user experience that reflects the constitutive entanglement of social and technical elements. furthermore, it also has the potential to positively impact real-world interactions by providing access and inclusivity to marginalised communities by allowing people to experience virtual realities and social settings that might otherwise be inaccessible. 6.2 theoretical and practical implications our study provides a new perspective on the value of future-oriented ar within the metaverse and offers rich insights into the potential role of ar. it equips industry and academia with crucial understanding and practical strategies to help shape the metaverse. it contributes to the information systems (is) literature by addressing a significant gap in understanding how ar technologies are shaping and being shaped by social dynamics within the emerging metaverse. specifically, limited work combines future-oriented techniques to anticipate and explore the complex socio-technical interactions in the metaverse with a focus on ar. methodologically, our application of horizon scanning and narrative development is novel within the is field and provides a structured approach for anticipating future technological and societal developments related to the metaverse. from a theoretical perspective, the developed narratives highlight how the metaverse is a coevolving sociotechnical system, aligning with structuration theory, where technology australasian journal of information systems richter, mohammadhossein, richter & trier 2025, vol 29, research article augmented reality and the metaverse 18 reshapes social structures and existing norms influence technology use. the narratives showcase a shift in design thinking to embrace open-mindedness and "constitutive entanglement," where technology is interwoven with social practices rather than merely interacting with them. in this context, the concept of adaptive technology, exemplified by prepurchase personalization, can enhance user satisfaction while reducing the misfit between organizational objectives and technical systems. from a practical and ethical perspective, we acknowledge that while these narratives demonstrate the immense potential of ar technology in the metaverse, the price component of accessing this technology cannot be ignored. despite its immense value and the potential for social mobility and economic engagement that it may bring, the current price points of the technology may create a digital divide, making this technology inaccessible to the masses. therefore, it is crucial that as the technology is developed, considerations are given to how wider communities may access these technologies, so this opportunity is not only accessible to the privileged. from a managerial perspective, this study underscores the necessity of a holistic, sociotechnical approach when designing ar in the metaverse. organizations should carefully consider the social context and broader societal impacts alongside technical functionalities. for instance, integrating blockchain-powered smart contracts and personalized user profiles reveals practical pathways for improving retail and other customer experiences. 7 research agenda the future outlook of this study aims to advance the understanding of user behaviour and engagement with ar technology within the evolving landscape of the metaverse. this research agenda emphasises exploring specific cognitive states, such as cognitive load and flow, and emotional states, such as stress and anxiety, impacted by ar, providing a foundation for designing user-centric, sustainable metaverse environments. by employing narrative planning through a sociotechnical lens, the study underscores the imperative to architect the metaverse in ways that harmonize individuals' and organisations' needs and aspirations. this approach calls for a paradigm shift from traditional design models that prioritize optimization towards more adaptive, emergent strategies that accommodate the fluidity and unpredictability of user experiences in the metaverse. recognizing the intricate interplay between ar technology and human well-being, the proposed research agenda focuses on balancing technological advancements with the diverse dimensions of human experience, thereby promoting a more holistic development of the metaverse. as ar continues to blur the boundaries between virtual and physical realities, fostering highly immersive and interactive environments, it introduces unprecedented complexity and potential for user engagement. in light of the metaverse's expansive role in facilitating social interactions, economic transactions, and digital lifestyles, developing a nuanced understanding of user adoption and engagement patterns with ar technology is crucial. this understanding is key to realizing the metaverse's potential as a space supporting technological innovation and human flourishing, ensuring its growth aligns with broader societal values and needs. australasian journal of information systems richter, mohammadhossein, richter & trier 2025, vol 29, research article augmented reality and the metaverse 19 how can adaptive ar experiences in the metaverse be designed to enhance cognitive and emotional well-being, specifically by minimizing cognitive load and maximizing flow states, and what frameworks are needed to balance immersive engagement with potential psychological risks? this research could include both controlled experiments and longitudinal studies to evaluate the efficacy of various adaptive ar strategies. what new forms of digital economies and social contracts will emerge as ar technology in the metaverse enables unprecedented modes of participation, creation, and exchange, and how can these developments be guided to benefit broader society? specifically, how will value be created and distributed within these new economies, and what ethical considerations must be addressed? this could include ethnographic research, analysis of digital transactions, and comparative studies across different platforms. in what ways will the evolution of ar interfaces within the metaverse transform users’ sense of reality, identity, and presence, specifically by examining the interplay between avatar identity and the dynamic construction of presence. and how can these transformations be aligned with the principles of digital wellness and human flourishing? this could be done by using validated scales of presence and longitudinal data, and ethnographic research, including user interviews and observations, what technological, ethical, and regulatory challenges will arise from the convergence of ar and ai in creating fully immersive, adaptive metaverse experiences, particularly regarding algorithmic bias and data privacy and how can cross-disciplinary collaborations address these challenges proactively? how can ar-enhanced learning environments within the metaverse redefine education and professional training especially by personalizing learning experiences and ensuring accessibility, and what future skills will be required to navigate and thrive in these immersive digital landscapes? this research could incorporate studies on skill transfer efficacy using performance based assessments, and a comparison with traditional learning environments. 8 conclusion in the context of ar and the metaverse, ar, as a technological artifact, is not just a standalone tool; it is deeply embedded within social contexts, be it psychological, cultural, or economic (silva et al., 2022). when users like sarah step into the metaverse, they don't just interact with a piece of technology; they engage in a dynamic ecosystem where technology and societal norms coalesce. the beauty of the sociotechnical perspective lies in its balanced view. it does not prioritize the technical over the social or vice versa. instead, it acknowledges that outcomes, especially in complex environments like the metaverse, emerge from the symbiotic relationship between the two. this is evident in how ar applications in the metaverse can drive instrumental outcomes, such as enhancing retail experiences, while also promoting humanistic values like well-being, equality, and freedom (mohammed et al., 2024; singla et al., 2024). the metaverse's instrumental and humanistic objectives cannot be defined by conventional information systems (seidel et al., 2022). unlike traditional information systems that represent a real-world phenomenon or augment a physical device, the metaverse is a confluence and integration of immersive social experiences that transcend the physical limitations of the real world (ryskeldiev et al., 2018), and it may play a vital role in various social activities and economic transactions. furthermore, the incorporation of ar technology within the metaverse adds a new layer of complexity to the system, as it seamlessly blends the virtual and physical worlds (buhalis et al., 2022), thereby creating an even more immersive and interactive australasian journal of information systems richter, mohammadhossein, richter & trier 2025, vol 29, research article augmented reality and the metaverse 20 experience for users (chenna, 2023; maccallum & parsons, 2019). the metaverse's design must consider the open-mindedness, emergence, and experiences that transcend established boundaries, which departs from the traditional approach of designing for optimization and convergence to a specific outcome (dionisio et al., 2013). future research can explore these findings, emphasizing the dynamic connection between technology, social practices, and economic realities. 9 references agriyanto, r. (2024). al implication and data-driven decision making for organizational. human resource strategies in the era of artificial intelligence, 259. ahmad, n., & junaini, s. (2020). augmented reality for learning mathematics: a systematic literature review. international journal of emerging technologies in learning (ijet), 15(16), 106-122. doi.org/10.3991/ijet.v15i16.14961%0d ahn, j., han, s., & al-hussein, m. (2019). 2d drawing visualization framework for applying projection-based augmented reality in a panelized construction manufacturing facility: proof of concept. journal of computing in civil engineering, 33, 04019032. doi.org/10.1061/(asce)cp.1943-5487.0000843 akcayir, m., & akcayir, g. (2017). advantages and challenges associated with augmented reality crossmark for education: a systematic review of the literature. educational research review, 20, 1-11. doi.org/10.1016/j.edurev.2016.11.002 al falahi, a., alnuaimi, h., alqaydi, m., al shateri, n., al ameri, s., qbea'h, m., & alrabaee, s. (2024). from challenges to future directions: a metaverse analysis. 2024 2nd international conference on intelligent metaverse technologies & applications (imeta), al khaldy, m. a., ishtaiwi, a., al-qerem, a., aldweesh, a., almomani, a., & alkasassbeh, m. (2024). cryptography in business intelligence and data analytics. in innovations in modern cryptography (pp. 352-375). igi global. doi.org/10.4018/979-8-3693-53301.ch015 aldweesh, a., alauthman, m., al khaldy, m., ishtaiwi, a., al-qerem, a., almoman, a., & gupta, b. b. (2023). the meta-fusion: a cloud-integrated study on blockchain technology enabling secure and efficient virtual worlds. international journal of cloud applications and computing (ijcac), 13(1), 1-24. allam, z., sharifi, a., bibri, s. e., jones, d. s., & krogstie, j. (2022). the metaverse as a virtual form of smart cities: opportunities and challenges for environmental, economic, and social sustainability in urban futures. smart cities, 5(3), 771-801. amer, m., daim, t. u., & jetter, a. (2013). a review of scenario planning. futures, 46, 23-40. anderson, j., & rainie, l. (2022). the metaverse in 2040. pew research centre, 30, 4. bagwe, s., singh, k., kashyap, a., arora, s., & maini, l. (2021). evolution of augmented reality applications in orthopaedics: a systematic review. journal of arthroscopy and joint surgery, 8(1), 84-90. doi.org/10.1016/j.jajs.2021.02.006 ball, m. (2022). the metaverse: and how it will revolutionize everything. liveright publishing. australasian journal of information systems richter, mohammadhossein, richter & trier 2025, vol 29, research article augmented reality and the metaverse 21 baroroh, d. k., chu, c.-h., & wang, l. (2020). systematic literature review on augmented reality in smart manufacturing: collaboration between human and computational intelligence. journal of manufacturing systems. doi.org/10.1016/j.jmsy.2020.10.017 baxter, g., & hainey, t. (2024). using immersive technologies to enhance the student learning experience. interactive technology and smart education, 21(3), 403-425, doi.org/10.1108/itse-05-2023-0078. billinghurst, m., & duenser, a. (2012). augmented reality in the classroom. computer (long beach, calif.), 45(7), 56-63. doi.org/10.1109/mc.2012.111 billinghurst, m., & kato, h. (2002). collaborative augmented reality. communications of the acm, 45(7), 64-70. blodgett-ford, s., & supponen, m. (2018). data privacy legal issues in virtual and augmented reality advertising. in research handbook on the law of virtual and augmented reality (pp. 471-512). edward elgar publishing. boell, s. k., & cecez-kecmanovic, d. (2014). a hermeneutic approach for conducting literature reviews and literature searches. communications of the association for information systems, 34(1), 12. doi.org/10.17705/1cais.03412 bonetti, f., warnaby, g., & quinn, l. (2018). augmented reality and virtual reality in physical and online retailing: a review, synthesis and research agenda. augmented reality and virtual reality, 119-132. bottani, e., & vignali, g. (2019). augmented reality technology in the manufacturing industry: a review of the last decade. iise transactions, 51(3), 284-310. doi.org/10.1080/24725854.2018.1493244 bower, m., howe, c., mccredie, n., robinson, a., & grover, d. (2014). augmented reality in education–cases, places and potentials. educational media international, 51(1), 1-15. buhalis, d., & amaranggana, a. (2015). smart tourism destinations enhancing tourism experience through personalisation of services. in information and communication technologies in tourism 2015 (pp. 377-389). springer. buhalis, d., leung, d., & lin, m. (2023). metaverse as a disruptive technology revolutionising tourism management and marketing. tourism management, 97, 104724. buhalis, d., lin, m. s., & leung, d. (2022). metaverse as a driver for customer experience and value co-creation: implications for hospitality and tourism management and marketing. international journal of contemporary hospitality management(ahead-ofprint). burnam-fink, m. (2015). creating narrative scenarios: science fiction prototyping at emerge. futures, 70, 48-55. buruk, o., dagan, e., isbister, k., segura, e. m., & tanenbaum, t. j. (2024). playful wearables: understanding the design space of wearables for games and related experiences. mit press. chang, y.-z., & wu, c.-t. (2024). application of extended reality in pediatric neurosurgery: a comprehensive review. biomedical journal, 100822. doi.org/10.1016/j.bj.2024.100822 australasian journal of information systems richter, mohammadhossein, richter & trier 2025, vol 29, research article augmented reality and the metaverse 22 chatterjee, s., sarker, s., lee, m. j., xiao, x., & elbanna, a. (2021). a possible conceptualization of the information systems (is) artifact: a general systems theory perspective 1. information systems journal, 31(4), 550-578. doi.org/10.1111/isj.12320 chen, c., zhang, k. z., chu, z., & lee, m. (2024). augmented reality in the metaverse market: the role of multimodal sensory interaction. internet research, 34(1), 9-38. chenna, s. (2023). augmented reality and ai: enhancing human-computer interaction in the metaverse. available at ssrn 4324629. chiasson, m. w., & davidson, e. (2005). taking industry seriously in information systems research. mis quarterly, 591-605. doi.org/10.2307/25148701 chiu, c. l., ho, h.-c., yu, t., liu, y., & mo, y. (2021). exploring information technology success of augmented reality retail applications in retail food chain. journal of retailing and consumer services, 61, 102561. chowdhury, s., budhwar, p., dey, p. k., joel-edgar, s., & abadie, a. (2022). ai-employee collaboration and business performance: integrating knowledge-based view, sociotechnical systems and organisational socialisation framework. journal of business research, 144, 31-49. doi.org/10.1016/j.jbusres.2022.01.069 conway, m. (2021). the idea of a university: enabling or constraining possible university futures. unpublished doctoral thesis]. swinburne university of technology. cowan, k., plangger, k., & javornik, a. (2024). insights for advertisers on immersive technologies: the future of ads using vr, ar, mr and the metaverse. journal of advertising research, 64(3), 249-254. cranefield, j., winikoff, m., chiu, y.-t., li, y., doyle, c., & richter, a. (2023). partnering with ai: the case of digital productivity assistants. journal of the royal society of new zealand, 53(1), 95-118. cropley, d. h., theurer, c., mathijssen, s., & marrone, r. l. (2023). fit-for-purpose creativity assessment: using machine learning to score a figural creativity test. crum, c. r., & coglianese, c. (2024). taking training seriously: human guidance and management-based regulation of artificial intelligence. 2024 international joint conference on neural networks (ijcnn), cuhls, k. e. (2020). horizon scanning in foresight–why horizon scanning is only a part of the game. futures & foresight science, 2(1), e23. curry, a., & schultz, w. (2009). roads less travelled: different methods, different futures. journal of futures studies, 13(4), 35-60. damar, m. (2021). metaverse shape of your life for future: a bibliometric snapshot. journal of metaverse, 1(1), 1-8. dargan, s., bansal, s., kumar, m., mittal, a., & kumar, k. (2023). augmented reality: a comprehensive review. archives of computational methods in engineering, 30(2), 10571080. de leoz, g., & petter, s. (2018). considering the social impacts of artefacts in information systems design science research. european journal of information systems, 27(2), 154170. doi.org/10.1080/0960085x.2018.1445462 australasian journal of information systems richter, mohammadhossein, richter & trier 2025, vol 29, research article augmented reality and the metaverse 23 devagiri, j. s., paheding, s., niyaz, q., yang, x., & smith, s. (2022). augmented reality and artificial intelligence in industry: trends, tools, and future challenges. expert systems with applications, 207, 118002. dionisio, j. d. n., iii, w. g. b., & gilbert, r. (2013). 3d virtual worlds and the metaverse: current status and future possibilities. acm computing surveys (csur), 45(3), 1-38. doi.org/10.1145/2480741.2480751 dolata, m., & schwabe, g. (2023). what is the metaverse and who seeks to define it? mapping the site of social construction. journal of information technology, 38(3), 239266. doi.org/10.1177/02683962231159927 dwivedi, y. k., hughes, l., baabdullah, a. m., ribeiro-navarrete, s., giannakis, m., aldebei, m. m., dennehy, d., metri, b., buhalis, d., & cheung, c. m. (2022). metaverse beyond the hype: multidisciplinary perspectives on emerging challenges, opportunities, and agenda for research, practice and policy. international journal of information management, 66, 102542. doi.org/10.1016/j.ijinfomgt.2022.102542 ebrahimabad, f. z., yazdani, h., hakim, a., & asarian, m. (2024). augmented reality versus web-based shopping: how does ar improve user experience and online purchase intention. telematics and informatics reports, 15, 100152. el ghati, o., alaoui-fdili, o., chahbouni, o., alioua, n., & bouarifi, w. (2024). artificial intelligence-powered visual internet of things in smart cities: a comprehensive review. sustainable computing: informatics and systems, 101004. doi.org/10.1016/j.suscom.2024.101004 ergen, f. d. (2021). artificial intelligence applications for event management and marketing. in impact of icts on event management and marketing (pp. 199-215). igi global. doi.org/10.4018/978-1-7998-4954-4.ch012 eswaran, m., gulivindala, a. k., inkulu, a. k., & bahubalendruni, m. r. (2023). augmented reality-based guidance in product assembly and maintenance/repair perspective: a state of the art review on challenges and opportunities. expert systems with applications, 213, 118983. doi.org/10.1016/j.eswa.2022.118983 fadhel, m. a., duhaim, a. m., albahri, a., al-qaysi, z., aktham, m., chyad, m., abdalaziz, w., albahri, o., alamoodi, a., & alzubaidi, l. (2024). navigating the metaverse: unraveling the impact of artificial intelligence—a comprehensive review and gap analysis. artificial intelligence review, 57(10), 264. doi.org/10.1007/s10462-02410881-5 far, s. b., rad, a. i., bamakan, s. m. h., & asaar, m. r. (2023). toward metaverse of everything: opportunities, challenges, and future directions of the next generation of visual/virtual communications. journal of network and computer applications, 217, 103675. doi.org/10.1016/j.jnca.2023.103675 flavián, c., ibáñez-sánchez, s., & orús, c. (2019). the impact of virtual, augmented and mixed reality technologies on the customer experience. journal of business research, 100, 547-560. australasian journal of information systems richter, mohammadhossein, richter & trier 2025, vol 29, research article augmented reality and the metaverse 24 flavián, c., ibáñez-sánchez, s., orús, c., & barta, s. (2024). the dark side of the metaverse: the role of gamification in event virtualization. international journal of information management, 75, 102726. flick, c., zamani, e. d., stahl, b. c., & brem, a. (2020). the future of ict for health and ageing: unveiling ethical and social issues through horizon scanning foresight. technological forecasting and social change, 155, 119995. doi.org/10.1016/j.techfore.2020.119995 flysight. (2024). advanced surveillance technologies to watch out for in 2024 https://www.flysight.it/advanced-surveillance-technologies-to-watch-out-for-in-2024/ følstad, a., & kvale, k. (2018). customer journeys: a systematic literature review. journal of service theory and practice. doi.org/10.1108/jstp-11-2014-0261 fu, y., li, c., yu, f. r., luan, t. h., zhao, p., & liu, s. (2022). a survey of blockchain and intelligent networking for the metaverse. ieee internet of things journal, 10(4), 35873610. doi.org/10.1109/jiot.2022.3222521 gad, a. g., mosa, d. t., abualigah, l., & abohany, a. a. (2022). emerging trends in blockchain technology and applications: a review and outlook. journal of king saud university-computer and information sciences, 34(9), 6719-6742. doi.org/10.1016/j.jksuci.2022.03.007 gadamer, h.-g. (2013). truth and method. a&c black. geroimenko, v. (2024). augmented and virtual reality in the metaverse. springer. ghaffari, k., lagzian, m., kazemi, m., & malekzadeh, g. (2020). a comprehensive framework for internet of things development: a grounded theory study of requirements. journal of enterprise information management, 33(1), 23-50. doi.org/10.1108/jeim-02-2019-0060 ghosh, a., lavanya, hassija, v., chamola, v., & el saddik, a. (2024). a survey on decentralized metaverse using blockchain and web 3.0 technologies, applications, and more. ieee access, 12, 146915-146948. doi.org/10.1109/access.2024.3469193 gohd, c. (2017). new anatomy vr app lets you look inside your own body. https://futurism.com/new-anatomy-vr-app-lets-you-look-inside-your-own-body goher, g. n. (2024). navigating the integration of chatgpt in uae’s government sector: challenges and opportunities. digital transformation and society. doi.org/10.1108/dts03-2024-0024 hampiholi, n. (2023). real-world deployments of ar in medical training and surgery. journal of emerging technologies and innovative research, 10(10), 8. hendricksen, i. t. (2023). observational, digital readout, and calibration techniques for studying the redshifted 21cm signal of hydrogen. mcgill university (canada). hennig-thurau, t., aliman, d. n., herting, a. m., cziehso, g. p., linder, m., & kübler, r. v. (2023). social interactions in the metaverse: framework, initial evidence, and research roadmap. journal of the academy of marketing science, 51(4), 889-913. australasian journal of information systems richter, mohammadhossein, richter & trier 2025, vol 29, research article augmented reality and the metaverse 25 ho, c.-h., chiu, k.-h., chen, h., & papazafeiropoulou, a. (2015). can internet blogs be used as an effective advertising tool? the role of product blog type and brand awareness. journal of enterprise information management. doi.org/10.1108/jeim-03-2014-0021 hollensen, s., kotler, p., & opresnik, m. o. (2023). metaverse the new marketing universe. journal of business strategy, 44(3), 119-125. doi.org/10.1108/jbs-01-2022-0014 hovorka, d., & peter, s. (2021). research perspectives: from other worlds: speculative engagement through digital geographies. journal of the association for information systems, 22(6), 1736-1752. hovorka, d. s., & mueller, b. (2025). speculative foresight: a foray beyond digital transformation. information systems journal, 35(1), 140-162. doi.org/10.1111/isj.12530 inayatullah, s. (2008), "six pillars: futures thinking for transforming", foresight, vol. 10 no. 1, pp. 4-21. doi.org/10.1108/14636680810855991 jansen, b. j., jung, s.-g., nielsen, l., guan, k. w., & salminen, j. (2022). how to create personas: three persona creation methodologies with implications for practical employment. pacific asia journal of the association for information systems, 14(3), 1. jetha, a., shamaee, a., bonaccio, s., gignac, m. a., tucker, l. b., tompa, e., bültmann, u., norman, c. d., banks, c. g., & smith, p. m. (2021). fragmentation in the future of work: a horizon scan examining the impact of the changing nature of work on workers experiencing vulnerability. american journal of industrial medicine, 64(8), 649666. jones, m. r., & karsten, h. (2008). giddens's structuration theory and information systems research. mis quarterly, 127-157. doi.org/10.2307/25148831 joshua, j. (2017). information bodies: computational anxiety in neal stephenson's snow crash. interdisciplinary literary studies, 19(1), 17-47. doi.org/10.5325/intelitestud.19.1.0017 kallinikos, j., aaltonen, a., & marton, a. (2013). the ambivalent ontology of digital artifacts. mis quarterly, 357-370. doi.org/10.25300/misq/2013/37.2.02 kaur, s., rajvanshi, s., & kaur, g. (2024). privacy and security concerns with augmented reality/virtual reality: a systematic review. augmented reality and virtual reality in special education, 209-231. doi.org/10.1002/9781394167586.ch10 kesim, m., & ozarslan, y. (2012). augmented reality in education: current technologies and the potential for education. procedia-social and behavioral sciences, 47, 297-302. doi.org/10.1016/j.sbspro.2012.06.654 kim, d. y., lee, h. k., & chung, k. (2023). avatar-mediated experience in the metaverse: the impact of avatar realism on user-avatar relationship. journal of retailing and consumer services, 73, 103382. kleinings, h. (2023). how to get the most out of ai in 2023: 7 applications of artificial intelligence in business. levity. koo, c., xiang, z., gretzel, u., & sigala, m. (2021). artificial intelligence (ai) and robotics in travel, hospitality and leisure. electronic markets, 31, 473-476. doi.org/10.1007/s12525021-00494-z australasian journal of information systems richter, mohammadhossein, richter & trier 2025, vol 29, research article augmented reality and the metaverse 26 kumar, h., rauschnabel, p. a., agarwal, m. n., singh, r. k., & srivastava, r. (2022). towards a theoretical framework for augmented reality marketing: a means-end chain perspective on retailing. information & management, 61(2), 103910. doi.org/10.1016/j.im.2023.103910 kumar, v., ashraf, a. r., & nadeem, w. (2024). ai-powered marketing: what, where, and how? international journal of information management, 77, 102783. doi.org/10.1016/j.ijinfomgt.2024.102783 laeeq, k. (2022). metaverse: why, how and what. how and what. lemon, k. n., & verhoef, p. c. (2016). understanding customer experience throughout the customer journey. journal of marketing, 80(6), 69-96. doi.org/10.1509/jm.15.0420 li, y.-j., cheung, c. m., shen, x.-l., & lee, m. k. (2022). when socialization goes wrong: understanding the we-intention to participate in collective trolling in virtual communities. journal of the association for information systems, 23(3), 678-706. liu, y., & siau, k. l. (2023). human-ai interaction and ai avatars. international conference on human-computer interaction, logeswaran, k., savitha, s., suresh, p., prasanna kumar, k., gunasekar, m., rajadevi, r., dharani, m., & jayasurya, a. (2024). unifying technologies in industry 4.0: harnessing the synergy of internet of things, big data, augmented reality/virtual reality, and blockchain technologies. topics in artificial intelligence applied to industry 4.0, 127-147. lun, l. k., & iqbal, j. (2024). a smart production line management system using face recognition and augmented reality. machine intelligence in mechanical engineering, 13-27. doi.org/10.1016/b978-0-443-18644-8.00010-1 maccallum, k., & parsons, d. (2019). teacher perspectives on mobile augmented reality: the potential of metaverse for learning. world conference on mobile and contextual learning, magerkurth, c., cheok, a. d., mandryk, r. l., & nilsen, t. (2005). pervasive games: bringing computer entertainment back to the real world. computers in entertainment (cie), 3(3), 4-4. majeed, z. h., & ali, h. a. (2020). a review of augmented reality in educational applications. international journal of advanced technology and engineering exploration, 7(62), 20-27. doi.org/10.19101/ijatee.2019.650068 mikalef, p., giannakos, m. n., & pateli, a. g. (2012). exploring the business potential of social media: an utilitarian and hedonic motivation approach. bled econference, mohamed, g., zahra, f. f., najwa, z., soumaya, o., yassine, k., chakir, a., & mohamed, a. (2024). enhancing immersive virtual shopping experiences in the retail metaverse through visual analytics, cognitive artificial intelligence techniques, blockchainbased digital assets, and immersive simulations: a systematic literature review. engineering applications of artificial intelligence, 305-318. australasian journal of information systems richter, mohammadhossein, richter & trier 2025, vol 29, research article augmented reality and the metaverse 27 mohammed, s. y., aljanabi, m., & gadekallu, t. r. (2024). navigating the nexus: a systematic review of the symbiotic relationship between the metaverse and gaming. international journal of cognitive computing in engineering. morris, m. g., & venkatesh, v. (2010). job characteristics and job satisfaction: understanding the role of enterprise resource planning system implementation. mis quarterly, 143161. doi.org/10.2307/20721418 mourtzis, d. (2024). challenges and opportunities of the transition from industry 4.0 to industry 5.0. manufacturing from industry 4.0 to industry 5.0, 97-131. doi.org/10.1016/b978-0-443-13924-6.00004-1. nair, a. j., manohar, s., mittal, a., & chaudhry, r. (2024). unleashing digital frontiers: bridging realities of augmented reality, virtual reality, and the metaverse. in the metaverse dilemma: challenges and opportunities for business and society (pp. 85-112). emerald publishing limited. neely, e. l. (2019). augmented reality, augmented ethics: who has the right to augment a particular physical space? ethics and information technology, 21(1), 11-18. doi.org/10.1007/s10676-018-9484-2. nemorin, s., vlachidis, a., ayerakwa, h. m., & andriotis, p. (2023). ai hyped? a horizon scan of discourse on artificial intelligence in education (aied) and development. learning, media and technology, 48(1), 38-51. neuhofer, b., magnus, b., & celuch, k. (2021). the impact of artificial intelligence on event experiences: a scenario technique approach. electronic markets, 31(3), 601-617. doi.org/10.1007/s12525-020-00433-4 ng, y.-l. (2023). when communicative ais are cooperative actors: a prisoner’s dilemma experiment on human–communicative artificial intelligence cooperation. behaviour & information technology, 42(13), 2141-2151. doi.org/10.1080/0144929x.2022.2111273 okhifun, g. (2022). how artificial intelligence can tackle workplace stress. retrieved from corporate wellness magazine: https://www. corporatewellnessmagazine. com/article/howartificial-intelligence-can-fight-against-workplace-stress. orlikowski, w. j. (1992). the duality of technology: rethinking the concept of technology in organizations. organization science, 3(3), 398-427. doi.org/10.1287/orsc.3.3.398 orlikowski, w. j. (2007). sociomaterial practices: exploring technology at work. organization studies, 28(9), 1435-1448. doi.org/10.1177/0170840607081138 pachoulakis, i., & kapetanakis, k. (2012). augmented reality platforms for virtual fitting rooms. the international journal of multimedia & its applications, 4(4), 35. palomino, m. a., bardsley, s., bown, k., de lurio, j., ellwood, p., holland‐smith, d., huggins, b., vincenti, a., woodroof, h., & owen, r. (2012). web‐based horizon scanning: concepts and practice. foresight. palomino, m. a., bardsley, s., bown, k., de lurio, j., ellwood, p., holland‐smith, d., huggins, b., vincenti, a., woodroof, h., & owen, r. (2012). web‐based horizon scanning: concepts and practice. foresight, 14(5), 355-373. doi.org/10.1108/14636681211269851 australasian journal of information systems richter, mohammadhossein, richter & trier 2025, vol 29, research article augmented reality and the metaverse 28 parlar, t. (2023). data privacy and security in the metaverse. in metaverse: technologies, opportunities and threats (pp. 123-133). springer. patel, k. (2024). revolutionizing customer experience: integrating blockchain with ar and vr in retail. augmenting retail reality, part b: blockchain, ar, vr, and ai, 1-22. doi.org/10.1108/978-1-83608-708-320241003 pavlik, j. v., & bridges, f. (2013). the emergence of augmented reality (ar) as a storytelling medium in journalism. journalism & communication monographs, 15(1), 4-59. doi.org/10.1177/1522637912470819 phakamach, p., senarith, p., & wachirawongpaisarn, s. (2022). the metaverse in education: the future of immersive teaching & learning. rice journal of creative entrepreneurship and management, 3(2), 75-88. poole, m. s., & desanctis, g. (2004). structuration theory in information systems research: methods and controversies. in the handbook of information systems research (pp. 206249). igi global. porter, m. e., & heppelmann, j. e. (2017). why every organization needs an augmented reality strategy. hbr’s 10 must, 85. pragha, p., natarajan, t., & dhalmahapatra, k. (2025). adoption of nft transaction in metaverse platform: a trust transfer approach to leverage trust among users. digital policy, regulation and governance, 27(4), 422-445. doi.org/10.1108/dprg-08-2024-0178 rabbi, i., & ullah, s. (2013). a survey on augmented reality challenges and tracking. acta graphica: znanstveni časopis za tiskarstvo i grafičke komunikacije, 24(1-2), 29-46. rafique, w., & qadir, j. (2024). internet of everything meets the metaverse: bridging physical and virtual worlds with blockchain. computer science review, 54, 100678. doi.org/10.1016/j.cosrev.2024.100678 rainie, l., & anderson, j. (2024). a new age of enlightenment? a new threat to humanity? experts imagine the impact of artificial intelligence by 2040. ramya, r., & ramamoorthy, s. (2025). ai in communication technology and innovation of ar/vr in communication. advances in linguistics and communication studies, 128. doi.org/10.4018/979-8-3693-3579-6.ch001 rawlley, o., gupta, s., chandrakar, j., johnson, m. k., & kalra, c. (2025). artificial intelligence inspired task offloading and resource orchestration in intelligent transportation systems. cognitive computation, 17(1), 1-30. doi.org/10.1007/s12559024-10380-3 rawson, a., duncan, e., & jones, c. (2013). the truth about customer experience. harvard business review, 91(9), 90-98. reeder, b., & turner, a. m. (2011). scenario-based design: a method for connecting information system design with public health operations and emergency management. journal of biomedical informatics, 44(6), 978-988. rehman, a., behera, r. k., islam, m. s., elahi, y. a., abbasi, f. a., & imtiaz, a. (2024). drivers of metaverse adoption for enhancing marketing capabilities of retail smes. technology in society, 79, 102704. doi.org/https://doi.org/10.1016/j.techsoc.2024.102704 australasian journal of information systems richter, mohammadhossein, richter & trier 2025, vol 29, research article augmented reality and the metaverse 29 ringland, g. (2010). the role of scenarios in strategic foresight. technological forecasting and social change, 77(9), 1493-1498. doi.org/10.1016/j.techfore.2010.06.010 richter, s., & richter, a. (2023). what is novel about the metaverse? international journal of information management, 73, 102684. doi.org/10.1016/j.ijinfomgt.2023.102684 richter, s., & richter, a. (2024). human-ai collaboration in the metaverse—how to research the future of work? ecis 2024 proceedings. 4. romano, b., sands, s., & pallant, j. i. (2021). augmented reality and the customer journey: an exploratory study. australasian marketing journal, 29(4), 354-363. doi.org/10.1016/j.ausmj.2020.06.010 rowe, e., wright, g., & derbyshire, j. (2017). enhancing horizon scanning by utilizing predeveloped scenarios: analysis of current practice and specification of a process improvement to aid the identification of important ‘weak signals’. technological forecasting and social change, 125, 224-235. doi.org/10.1016/j.techfore.2017.08.001 ryskeldiev, b., ochiai, y., cohen, m., & herder, j. (2018). distributed metaverse: creating decentralized blockchain-based model for peer-to-peer sharing of virtual spaces for mixed reality applications. proceedings of the 9th augmented human international conference, doi.org/10.1145/3174910.3174952 salinas, p., & gonzález-mendívil, e. (2017). augmented reality and solids of revolution. international journal on interactive design and manufacturing (ijidem), 11(4), 829-837. sarker, s., chatterjee, s., & xiao, x. (2013). how" sociotechnical" is our is research?: an assessment and possible ways forward. proceedings of the 34th international conference on information systems. icis 2013, sarker, s., chatterjee, s., xiao, x., & elbanna, a. (2019). the sociotechnical axis of cohesion for the is discipline: its historical legacy and its continued relevance. mis quarterly, 43(3), 695-720. sarker, s., xiao, x., & beaulieu, t. (2013). guest editorial: qualitative studies in information systems: a critical review and some guiding principles. mis quarterly, 37(4), iii-xviii. schoemaker, p. j. h., day, g. s., & snyder, s. a. (2013). integrating organizational networks, weak signals, strategic radars and scenario planning. technological forecasting and social change, 80(4), 815-824. doi.org/10.1016/j.techfore.2012.10.020 scholz, j., & smith, a. n. (2016). augmented reality: designing immersive experiences that maximize consumer engagement. business horizons, 59(2), 149-161. https://doi.org/10.1016/j.bushor.2015.10.003 schwarz, j. o. (2015). the 'narrative turn' in developing foresight: assessing how cultural products can assist organisations in detecting trends. technological forecasting and social change, 90, 510-513. https://doi.org/10.1016/j.techfore.2014.02.024 see, z. s., billinghurst, m., rengganaten, v., & soo, s. (2016). medical learning murmurs simulation with mobile audible augmented reality. in siggraph asia 2016 mobile graphics and interactive applications (pp. 1-4). australasian journal of information systems richter, mohammadhossein, richter & trier 2025, vol 29, research article augmented reality and the metaverse 30 seidel, s., berente, n., nickerson, j., & yepes, g. (2022). designing the metaverse. proceedings of the annual hawaii international conference on system sciences. doi.org/10.24251/hicss.2022.811 sharifmousavi, m., kayvanfar, v., & baldacci, r. (2024). distributed artificial intelligence application in agri-food supply chains 4.0. procedia computer science, 232, 211-220. shin, d. (2019). how does immersion work in augmented reality games? a user-centric view of immersion and engagement. information, communication & society, 22(9), 1212-1229. doi.org/10.1080/1369118x.2017.1411519 silva, r. m. l., principe cruz, e., rosner, d. k., kelly, d., monroy-hernández, a., & liu, f. (2022). understanding ar activism: an interview study with creators of augmented reality experiences for social change. chi conference on human factors in computing systems, 1-15. doi.org/10.1145/3491102.3517605 singh, p., prasad, g., athwal, h. s., & singh, s. (2025). patient guidance and remote care with ai and ar. singla, b., shalender, k., & singh, n. (2024). creator's economy in metaverse platforms: empowering stakeholders through omnichannel approach: empowering stakeholders through omnichannel approach. igi global. schoemaker, p. j. (1995). scenario planning: a tool for strategic thinking. long range planning, 28(3), 117. doi.org/10.1016/0024-6301(95)91604-0 smith, l. w., rose, r. l., zablah, a. r., mccullough, h., & saljoughian, m. m. (2023). examining post-purchase consumer responses to product automation. journal of the academy of marketing science, 51(3), 530-550. doi.org/10.1007/s11747-022-00900-8 stocchi, l., pourazad, n., michaelidou, n., tanusondjaja, a., & harrigan, p. (2022). marketing research on mobile apps: past, present and future. journal of the academy of marketing science, 1-31. 50(2), 195-225. doi.org/10.1007/s11747-021-00815-w stock-homburg, r. (2022). survey of emotions in human–robot interactions: perspectives from robotic psychology on 20 years of research. international journal of social robotics, 14(2), 389-411. doi.org/10.1007/s12369-021-00778-6 strong, d. m., & volkoff, o. (2010). understanding organization—enterprise system fit: a path to theorizing the information technology artifact. mis quarterly, 731-756. doi.org/10.2307/25750703 sumadio, d. d., & rambli, d. r. a. (2010). preliminary evaluation on user acceptance of the augmented reality use for education. 2010 second international conference on computer engineering and applications, 461–465. doi.org/10.1109/iccea.2010.239 sung, e. c., bae, s., han, d.-i. d., & kwon, o. (2021). consumer engagement via interactive artificial intelligence and mixed reality. international journal of information management, 60, 102382. tao, w., lai, z.-h., leu, m. c., yin, z., & qin, r. (2019). a self-aware and active-guiding training & assistant system for worker-centered intelligent manufacturing. manufacturing letters, 21, 45-49. doi.org/10.1016/j.mfglet.2019.08.003 australasian journal of information systems richter, mohammadhossein, richter & trier 2025, vol 29, research article augmented reality and the metaverse 31 tom dieck, m. c., & han, d.-i. d. (2022). the role of immersive technology in customer experience management. journal of marketing theory and practice, 30(1), 108–119. uddin, m., obaidat, m., manickam, s., laghari, s. u. a., dandoush, a., ullah, h., & ullah, s. s. (2024). exploring the convergence of metaverse, blockchain, and ai: a comprehensive survey of enabling technologies, applications, challenges, and future directions. wiley interdisciplinary reviews: data mining and knowledge discovery, 14(6), e1556. doi.org/10.1002/widm.1556 valmorisco, s., raya, l., & sanchez, a. (2024). enabling personalized vr experiences: a framework for real-time adaptation and recommendations in vr environments. virtual reality, 28(3), 128. doi.org/10.1007/s10055-024-01020-0 venkata, m. d., karneedi, v., sri padmaja yandamuri, s., & siddi, n. p. (2024). ai-enhanced digital mirrors: empowering women's safety and shopping experiences. in wearable devices, surveillance systems, and ai for women's wellbeing (pp. 26-51). igi global. verhulst, i., woods, a., whittaker, l., bennett, j., & dalton, p. (2021). do vr and ar versions of an immersive cultural experience engender different user experiences? computers in human behavior, 125, 106951. doi.org/10.1016/j.chb.2021.106951 walls, j. g., widmeyer, g. r., & el sawy, o. a. (1992). building an information system design theory for vigilant eis. information systems research, 3(1), 36-59. doi.org/10.1287/isre.3.1.36 wang, q., & shukla, p. (2013). linking sources of consumer confusion to decision satisfaction: the role of choice goals. psychology & marketing, 30(4), 295-304. doi.org/10.1002/mar.20606 washida, y., & yahata, a. (2021). predictive value of horizon scanning for future scenarios. foresight, 23(1), 17-32. doi.org/10.1108/fs-05-2020-0047 wei, q., wu, h., shi, f., wan, y., & ning, h. (2023). a tutorial on meta-services and services computing in metaverse. ieee internet of things journal. doi.org/10.1109/jiot.2023.3346901 welz, j., riemer, a., döbel, i., dakkak, n., & von schwartzenberg, a. s. (2021). identifying future trends by podcast mining: an explorative approach for web-based horizon scanning. foresight, 23(1), 1-16. doi.org/10.1108/fs-07-2020-0069 wright, m., ekeus, h., coyne, r., stewart, j., travlou, p., & williams, r. (2008). augmented duality: overlapping a metaverse with the real world. proceedings of the 2008 international conference on advances in computer entertainment technology, wu, h.-k., lee, s. w.-y., chang, h.-y., & liang, j.-c. (2013). current status, opportunities and challenges of augmented reality in education. computers & education, 62, 41-49. xi, n., chen, j., gama, f., korkeila, h., & hamari, j. (2024). virtual experiences, real memories? a study on information recall and recognition in the metaverse. information systems frontiers, 1-25. doi.org/10.1007/s10796-024-10500-2 yim, m. y.-c., chu, s.-c., & sauer, p. l. (2017). is augmented reality technology an effective tool for e-commerce? an interactivity and vividness perspective. journal of interactive marketing, 39(1), 89-103. doi.org/10.1016/j.intmar.2017.04.001 australasian journal of information systems richter, mohammadhossein, richter & trier 2025, vol 29, research article augmented reality and the metaverse 32 yovcheva, z., buhalis, d., gatzidis, c., & van elzakker, c. p. (2014). empirical evaluation of smartphone augmented reality browsers in an urban tourism destination context. international journal of mobile human computer interaction (ijmhci), 6(2), 10-31. doi.org/10.4018/ijmhci.2014040102 yuen, s. c.-y., yaoyuneyong, g., & johnson, e. (2011). augmented reality: an overview and five directions for ar in education. journal of educational technology development and exchange (jetde), 4(1), 11. doi.org/10.18785/jetde.0401.10 zalan, t., & barbesino, p. (2023). making the metaverse real. digital business, 3(2), 100059. doi.org/10.1016/j.digbus.2023.100059 copyright © 2025 richter, s., mohammadhossein, n., richter, a. & trier, m. this is an openaccess article licensed under a creative commons attribution-non-commercial 4.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: doi.org/10.3127/ajis.v29.5429 https://creativecommons.org/licenses/by-nc/4.0/deed.en https://creativecommons.org/licenses/by-nc/4.0/deed.en paper title (paper title style) australasian journal of information systems chandrasekara, sedera & gao 2021, vol 25, research article boundary conditions of sns research 1 determining boundary conditions of social influence for social networking site research dharshani chandrasekara monash university, australia dharshani.chandrasekara@monash.edu darshana sedera southern cross university, australia caddie gao monash university, australia abstract social influence theoretical constructs have been utilized substantially by researchers in information systems (is) to examine interactions in social networking sites (snss). however, most of the theoretical constructs of social influence applied in is studies have been adopted from social influence theories developed before the advent of snss. yet, research on theory development indicates that when applying an existing theory to a novel context, the specification of boundary conditions is crucial. therefore, in this study, by examining how different social influence constructs have been utilized in past is studies, we derive five boundary conditions that can be applied to future sns research. the boundary conditions for sns research were determined by conducting a literature review with a sample of 65 research papers. deriving boundary conditions by analyzing recurring patterns in the literature allowed us to generate a foundational knowledge of the use of boundary conditions of social influence for sns research, which would aid researchers in generating accurate findings. our findings demonstrate that when applying a theory to a novel context, identification of boundary conditions of a theory would not only contribute to the rigor in research but also support the validation of its practical implications. keywords: social media, boundary conditions, theory and philosophy 1 introduction along with the advent of social networking sites (snss) like facebook and twitter, people around the world have been able to reach a new level of social interaction. according to the latest statistics, the global population using snss has reached 3.96 billion, with an annual increase of 10.5% (chaffey, 2020). therefore, the effect of social influence is also increasing through snss. for example, it is estimated that the average facebook user has 338 friends (smith, 2017), while 15% of facebook users have more than 500 friends (smith, 2014). the average twitter user has 707 followers (maccarthy, 2016). the high number of individuals interacting with and influencing each other on a daily basis via snss shows how the novel context of snss has enabled individuals to widen their social circles compared to the low number of members individuals had in their social circles before the advent of snss. the effect of such high volumes of influence means that the theoretical boundaries of even wellestablished theories of social influence must be reconsidered. australasian journal of information systems chandrasekara, sedera & gao 2021, vol 25, research article boundary conditions of sns research 2 the growing snss research has revealed that social influence generated by individuals via snss, while similar in many aspects to social influence in the pre-sns era, differs from and challenges the traditional boundaries of social influence (bagozzi & dholakia, 2002; zhou, 2011). for instance, reach in terms of the number of associates in a social circle (mangold & faulds, 2009) and geographical coverage (tufekci & wilson, 2012) easily exceed the traditional social influence boundaries. similarly, the frequency of interaction between members in a social circle (fischer & reuber, 2011) also provides potential differences between traditional social influence and social influence facilitated by snss. as such, while the fundamental premise of social influence theories (zhou, 2011; zeng et al., 2009) remain relevant, their boundary conditions must be revisited to understand how the changes arising through snss could affect a well-established theoretical foundation. boundary conditions can simply be defined as the “who, where, when” conditions of a theory that “constitute the range of the theory” (whetten, 1989, p.492). according to dubin (1969, p. 126) “an experimental situation is different from another by its boundary conditions”. when context-specific research is conducted, identifying the boundary conditions that are relevant to that context is the key to conducting the research successfully. based on this premise, information systems (is) researchers would benefit from identifying the boundary conditions that are applicable for examining social influence in the specific context of snss as opposed to social influence that takes place in the physical world, to generate more accurate findings based on the gradual shift that has taken place in human interactions from physical space to digital space (palekar, atapattu, sedera, & lokuge, 2018). therefore, we conducted this study with the objective of assessing the boundary conditions of social influence theoretical perspective, especially in light of the novelty in snss. aligning with this objective, the research question we aim to address through this study is: “what boundary conditions should be considered when engaging in social influence research in the context of snss?” to assess our assertion, we investigated a rigorously selected sample of 65 studies on social influence and snss published in top-tier journals. in those selected studies, we made insightful observations about the use of constructs of social influence, theoretical applications, and whether the studies had considered the natural expansions to the underlying foundations that had taken place as a result of snss. however, with regard to the theoretical expansion of social influence, we found little evidence that boundary conditions are established, reviewed, or revised explicitly. therefore, based on our literature review, we derived five boundary conditions necessary for the utilization of the theoretical perspective of social influence in snss. the paper proceeds in the following manner. first, it provides an overview of the theoretical foundations of social influence. then it investigates the constructs employed in social influence using the literature sample and identifies the most applicable constructs of social influence for snss. next, we show how we derived five boundary conditions of social influence for snss and explain the boundary conditions in light of the constructs employed in the social influence studies. finally, we explain how researchers and practitioners can apply boundary conditions in their sns research to increase rigor in both theory and practice. 2 theoretical foundations of social influence throughout the history of humankind, individuals have attempted to maintain relationships with one another in groups or networks, and the formation and characteristics of such groups australasian journal of information systems chandrasekara, sedera & gao 2021, vol 25, research article boundary conditions of sns research 3 have always evolved with time (mcpherson, popielarz, & drobnic, 1992). in its simplest form, a social network can be defined as “a set of people (or organizations or other social entities) connected by a set of social relationships such as friendship, co-working or information exchange” (garton, haythornthwaite, & wellman, 1997, para 1). with the advancement of technology, such social networks have been expanded to digital spheres and are visible in the form of snss (boyd & ellison, 2007; chang, liu, & chen, 2014). even before the proliferation of snss, there has been a continuous effort to understand the influence individuals have on one another in such social networks where there are frequent interactions among individuals (coleman, 1986; galaskiewicz & wasserman, 1993; kelman, 1958; latané, 1981). for the past several decades, studies have delivered various theories and frameworks to better understand social influence (e.g., bagozzi & dholakia, 2002; kelman, 1958; kuan, zhong & chau, 2014; latané, 1981). especially with the advent and mass proliferation of snss, it is evident that social influence is growing and changing (bagozzi & dholakia, 2002; zeng. et al., 2009). various disciplines including is (palekar et al., 2018), marketing (dholakia, bagozzi & pearo, 2004), organizational environment (arvidsson & holmström, 2013), political science tufekci & wilson, 2012) and sociology (golder & macy, 2014; zhang & centola, 2019), have examined social influence that occurs as a result of people’s interactions in snss. moreover, the social influence generated by individuals in snss has been discussed in various sectors, including the automobile industry (wang, susarla & sambamurthy, 2015), music (dewan, ho & ramaprasad, 2017), and the film industry (oh, roumani & nwankpa, 2017). in particular, in the special issue on recent advances in social media in the australasian journal of information systems in 2018, many of the published studies (e.g., grottke, hacker, & durst, 2018; morgan, cheong, & bedingfield, 2018; palekar & sedera, 2018; wang, alahakoon, & de silva, 2018; whiteside, aleti, pallant, & zeleznikow, 2018) have discussed the importance of social influence generated in snss on various aspects of attitudinal and behavioral change (refer to table 1). study aspect discussed in relation to social influence and snss grottke et al. (2018) how online interactions influence individuals in terms of generating social capital. palekar & sedera (2018) how sns users influence news consumption in social broadcasting networks. wang et al. (2018) how sns users influence the interpretation of media content. whiteside et al. (2018) the means through which interactions with different individuals (who may or may not belong to an individual’s physical social circle) can influence one’s intimate relationships. table 1. the importance of social influence for snss (ajis special issue) the studies mentioned above emphasize that when examining different phenomenon in the context of snss, social influence has always been an area of interest for is research. this not only proves that examining social influence in snss could allow us to understand online interactions better, but it also highlights that we should pay more attention to whether social influence theories which have been established by observing the individual interactions in the physical world could be applied in the same manner for snss. according to dubin (1969, p. 126), when conducting research, there is a need to identify the “boundary conditions” or “initial conditions” which differentiate one particular study context from another to specify a theoretical model accurately. considering that there are substantial differences in the way individuals interact in digital spaces such as snss and the physical world (bagozzi & dholakia, 2002) it is essential to identify what social influence constructs australasian journal of information systems chandrasekara, sedera & gao 2021, vol 25, research article boundary conditions of sns research 4 have been discussed in the past in general and what social influence constructs are discussed at present in relation to snss. such an understanding of the way social influence constructs have been adopted in different contexts would allow us to identify whether there are any conditions we should consider in the process of adopting social influence constructs to study various phenomena in the snss. 2.1 key theoretical constructs of social influence theoretical foundations of social influence mainly lie in the discipline of social psychology and consist of many different constructs used across several disciplines to understand how social influence can affect individuals. for instance, deutsch and gerard (1955) introduced a theory of social influence that consists of two main constructs, namely, normative social influence and informational social influence. kelman (1958) in his social influence theory, came up with three constructs of social influence, namely, compliance, identification, and internalization to indicate the processes through which people change their attitudes and behaviors based on the influence of others. introducing social impact theory, latané (1981) identified another three constructs of social influence, namely, strength, immediacy, and the number of people, which allow us to better understand social influence processes. peer pressure (kandel & lazear, 1992), also identified as peer influence, is another commonly adopted social influence construct that examines the role of peers in affecting individual decisions. further, in the paper “social influence: compliance and conformity”, cialdini and goldstein (2004) discussed two types of social influence namely, compliance and conformity and highlighted that an individual can be influenced based on three types of rewarding goals; goal of accuracy, goal of affiliation and goal of maintaining a positive self-concept. when examining social influence from a theoretical perspective, all these constructs have contributed to our understanding of the different processes of social influence which can influence individuals to form different attitudes and behaviors. most of these constructs have been adopted by various disciplines to investigate the role of social influence in creating attitudinal and behavioral changes in individuals. our investigation into how these constructs have been utilized over time in various domains has revealed the breadth of social influence and highlighted the need to identify the social influence constructs that have been widely used in is. 2.2 constructs of social influence used in is research the role of social influence in digital spheres has been an area of interest for is for a considerable period (bagozzi & dholakia, 2002; tsai & bagozzi, 2014). is researchers have employed several constructs of social influence to investigate various phenomena in the digital spheres. table 2 illustrates the key social influence constructs examined in is studies chronologically by the year in which the key study related to the social influence construct(s) was published. this illustration of social influence constructs examined in is studies indicates that even when digital spaces such as snss were selected as the study context, is studies have used social influence constructs that were developed prior to the advent of snss (e.g., deutsch & gerard, 1955; hovland, janis, & kelley, 1953; kelman, 1958; latané, 1981; nemeth, 1986). when considering the applicability of each of these constructs to is, kuan et al. (2014) utilized informational social influence and normative social influence to understand how purchase decisions in group buying sites can be affected by social influence. in this study, informational influence has been applied in situations where people make decisions based on others’ actions australasian journal of information systems chandrasekara, sedera & gao 2021, vol 25, research article boundary conditions of sns research 5 and judgments, thus treating them as sources of information, and normative influence is applied when people make decisions to comply with others, based on others’ preferences or expectations. further, kuan et al. (2014) indicated that both informational and normative social influences are contributing to conformity. kwahk & ge (2012) also recognized that snss facilitate the promotion of informational influence due to the dissemination of information and knowledge, encouraging sns users to engage in e-commerce activities more. however, whether or not such theoretical constructs can be applied as they are has not been adequately addressed in these past is studies. social influence construct(s) key theory publication related to the construct(s) examples of is studies that have examined the social influence construct(s) informational social influence and normative social influence (deutsch & gerard, 1955) (kuan, et al., 2014; k. zhang, lee, & zhao, 2010; zhao, stylianou, & zheng, 2018) compliance, identification and internalization (kelman, 1958) (bagozzi & dholakia, 2002; bagozzi, dholakia, & mookerjee, 2006; bagozzi, dholakia, & pearo, 2007; cheung, chiu, & lee, 2011; cheung & lee, 2010; datta, 2011; dholakia, bagozzi, & pearo, 2004; shen, lee, cheung, & chen, 2010; tsai & bagozzi, 2014; wang, meister, & gray, 2013; zhou, 2011) strength, immediacy and number (latané, 1981) (chan, skoumpopoulou, & yu, 2018; kwahk & ge, 2012; miller & brunner, 2008; mir & zaheer, 2012) peer pressure/peer influence (kandel & lazear, 1992) (godinho de matos, ferreira, & krackhardt, 2014; zhang, pavlou, & krishnan, 2018; zhang, susarla, & krishnan, 2014) table 2. a chronological presentation of social influence constructs note: the order of the social influence constructs is presented by the year in which the seminal theory paper related to the constructs was published. kelman’s social influence theory (1958) and latané’s social impact theory (1981) have also widely been used in is to analyze individual actions and attitude changes in relation to social influence (refer to table 2). notably, the literature analysis conducted through this study indicated these latter two theories as the most applied social influence theories for the context of snss. the three social influence constructs of kelman (1958) were introduced to the academic community in his study on “compliance, identification, and internalization: three processes of attitude change”. according to this particular study (kelman, 1958), social influence can be accepted by three different types of processes, namely, compliance, identification and internalization. herein, compliance refers to the influential capabilities of significant others or powerful people in the influencee’s life (bagozzi and dholakia, 2002). identification refers to the process whereby a person acts in a certain way to gain acceptance or a feeling of belongingness to a particular group or an individual (kelman, 1958). internalization refers to the process through which a person engages in a particular behavior because the behavior aligns with his/her value system (kelman, 1958). the foundation for social influence studies in is, particularly on snss, has been mainly laid with the support of kelman’s substantial contribution. a number of is studies exploring social influence in digital spaces have utilized kelman’s theory on social influence as the basis for their research (refer to table 2). when the utilization of social impact theory is considered, latané (1981, p. 343) defined social impact as “the great variety of changes in physiological states and subjective feelings, motives australasian journal of information systems chandrasekara, sedera & gao 2021, vol 25, research article boundary conditions of sns research 6 and emotions, cognitions and beliefs, values and behavior, that occur in an individual, human or animal, as a result of the real, implied, or imagined presence or actions of other individuals”. in order to influence an individual, he suggested three factors that should be present when an interaction takes place: strength “the salience, power, importance, or intensity of a given source to the target”, immediacy “closeness in space or time and absence of intervening barriers or filters” and number “how many people there are” (latané, 1981, p. 344). even though social impact theory was established before the advent of snss, several studies have utilized the theory explicitly in explaining the behavior of individuals in digital spaces (chan, skoumpopoulou, & yu, 2018; kwahk & ge, 2012; miller & brunner, 2008; mir & zaheer, 2012). miller and brunner (2008) even redefined the constructs to be appropriate to the online setting. for instance, when relating the construct of immediacy, which is mainly regarding the physical proximity between the source and the target of impact, miller and brunner (2008) defined immediacy in the digital environment in terms of user engagement in chatroom dialogues. concerning the construct of strength, the same study used interpersonal factors such as assertiveness and exaggeration as indicators of strength in the person having an influence online. such changes to the definitions of social influence constructs further denote the necessity of revisiting the theoretical constructs of social influence in light of snss. peer influence is another social influence construct that has been widely discussed in is. this can be defined as the process in which peers or friends play an important role when a specific individual decides to act in a certain way (godinho de matos, ferreira & krackhardt, 2014). bapna and umyarov (2015) investigated peer influence in snss in relation to online purchase decisions. aral and walker (2011) also studied the effect of peer influence by experimenting on viral marketing campaigns online. in addition to these social influence constructs that had been discussed even before the advent of snss, dewan et al. (2017) used popularity influence and proximity influence to identify how social influence is generated in snss. herein, the construct of popularity influence is explained in relation to the concept of word-of-mouth and observational learning through which consumers are influenced by their interactions with others. proximity influence is defined in the same study as influence generated by close social circles in the social network. in summary, in reviewing is studies that have examined social influence, we identified that social influence studies in is mainly employ the constructs1of compliance, identification, internalization, peer influence, informational social influence, normative social influence, strength, immediacy, and number of people as the key variables in analyzing the behaviors of people (e.g. bagozzi & dholakia, 2002; kuan et al., 2014; kwahk & ge, 2012; mir & zaheer, 1970; sedera, lokuge, atapattu, & gretzel, 2017; zhang, pavlou, & krishnan, 2018). while these studies have contributed to the extension of knowledge greatly, one crucial factor that has not been studied adequately is whether these constructs can be applied in the same way to snss considering they were developed prior to the advent of snss. in order to investigate this matter further, we conducted a literature review to analyze the use of the social influence constructs in past social influence and sns research and to examine what types of boundary conditions should be specified when social influence constructs are applied to snss. 1 the logical derivation of the most applied social influence constructs in is discussed in detail in section 3. australasian journal of information systems chandrasekara, sedera & gao 2021, vol 25, research article boundary conditions of sns research 7 3 methodology and analysis 3.1 overview of the literature sample table 3. the methodology for selecting papers relevant for the literature review the objectives of the literature analysis were to: (i) develop an understanding of the importance of revisiting social influence theory for snss; (ii) identify the social influence constructs applicable for snss, and (iii) derive boundary conditions for sns research as necessary. the literature analysis was deemed to be the most appropriate approach for the steps results/comments 1. identification of suitable publishing outlets to conduct the literature search. selecting the basket-of-08 is journals: management information systems quarterly, european journal of information systems, information systems research, information systems journal, journal of information technology, journal of strategic information systems, journal of management information systems, journal of the association for information systems 2. identification of appropriate keywords/strings for the literature search. keywords: “social influence”, “influence”, “social influence and social media”, “social influence and social networks”, “social media”, “social impact”, “social impact and social media”, “impact”, “social interaction and social media” 3. selecting the time period for the literature search. 2008 to 2018 was selected as the appropriate period for the initial set of research papers relevant to the objectives of the research. 4. carrying out the literature search using the keywords and downloading the relevant papers. inclusion criteria: (i) papers with titles and abstracts with a focus on either human interactions in digital spaces or social influence in the context of information technology or is, (ii) papers using both qualitative and qualitative methods, and literature reviews. downloaded 193 papers in total. 5. reviewing full content of the downloaded papers to identify the most relevant papers for the study. exclusion criteria: papers that did not discuss or examine the occurrence of different types of social influence. finalized 13 papers relevant to the research from the initial search. 6. engaging in backward search (reviewing the reference lists of the 13 papers) and forward search (reviewing the studies that have cited the 13 papers using google scholar database) to identify more suitable papers for the literature sample. inclusion criteria: (i) seminal studies on social influence, (ii) papers discussing either human interactions in digital spaces or social influence in the context of information technology or is. 45 papers were added to the literature sample. 7. adding plumx highly cited sns research that was not identified in the previous stages to the literature sample note: the plumx metrics “gathers and brings together appropriate research metrics for all types of scholarly research output” (plumx metrics plum analytics, n.d.) inclusion criteria: papers discussing either human interactions in snss or social influence in snss. 7 papers were added to the literature sample. total no. of papers selected for the literature sample 13+45+7 = 65 australasian journal of information systems chandrasekara, sedera & gao 2021, vol 25, research article boundary conditions of sns research 8 study as we established that even though different theoretical constructs of social influence have been widely utilized in the domain of is, (bagozzi et al., 2006; bagozzi & dholakia, 2002; eckhardt, laumer, & weitzel, 2009; kuan et al., 2014; sedera et al., 2017; tsai & bagozzi, 2014; vannoy & palvia, 2010), to date, a comprehensive literature analysis on this topic has not been conducted. in the meantime, a literature analysis would lay a strong foundation for future research employing social influence constructs to examine social interactions in snss. notably, is as a discipline has emphasized the necessity of producing literature reviews to strengthen the foundation of is in terms of both theory development and progression as a discipline (webster & watson, 2002). the methodology for selecting literature is outlined in table 3. 3.2 analysis once the literature sample of 65 studies was assembled, we distilled the constructs of measurement. we first identified papers in the sample that discussed theoretical constructs of social influence in relation to human interactions and attitudinal/behavioral change. we then narrowed down the most common constructs of social influence emphasized in the studies. in papers where constructs of social influence were not explicitly discussed, we assigned the most appropriate constructs for the study based on the discussions presented by authors regarding the type of influence taking place in their study. table 4 includes the study2 and the corresponding constructs employed, denoted in columns ‘a’ to ‘i’. the constructs denoted in table 4 include three from kelman’s social influence theory (1958); (a) compliance, (b) identification, (c) internalization, other commonly used social influence constructs in is; (d) peer influence, (e) informational influence, and (f) normative influence, and three constructs from latané’s social impact theory (1981); (g) immediacy, (h) strength, and (i) number of people. the initial mapping of the constructs was completed by one author and verified by the other two authors. a comparison of the individual classifications revealed average inter-coder reliability exceeding 80%3. then we added binary values to the table: ‘y’ if the study explicitly or implicitly discussed a particular construct and ‘n’ if the study did not discuss a particular construct at all. the table was designed following the “concept matrix augmented with units of analysis” structure recommended by webster & watson (2002, p. xvii) in the mis quarterly guest editorial titled ‘analyzing the past to prepare for the future: writing a literature review’. table 4 also allowed us to identify some common themes along which the analysis can be carried out. according to webster and watson (2002, p. xvii) “isolating concepts by units of analysis should result in a crisp review because it is easier to detect when you let a concept stray outside the scope of its domain”. hence, using table 4, we identified the most used constructs of social influence, overlaps between the social influence constructs, and the evolution of constructs. finally, once the literature sample was analyzed under these themes, we derived the boundary conditions of social influence that could be established for sns research. 2 the complete list of references for table 4 can be found in appendix a. 3 krippendorff (1980) recommends inter-coder reliability of at least 70%. australasian journal of information systems chandrasekara, sedera & gao 2021, vol 25, research article boundary conditions of sns research 9 study a b c d e f g h i (kelman, 1958) y y y n y y n y n (latané, 1981) y n n n n y y y y (nowak et al., 1990) y n n n y y y y y (latané et al., 1995) y n n n n y y y y (latané, 1996) y y n n n y y y y * (latané and bourgeois, 1996) n n n n n n y y y (venkatesh and davis, 2000) y y y n y y n y n *(bagozzi and dholakia, 2002) y y y n y y n n y (dholakia and talukdar, 2004) y n n n y y n y n *(dholakia et al., 2004) y y y n y y n n n (algesheimer et al., 2005) y y n n y y n y y (lu et al., 2005) y y y n y y n y n (gallivan et al., 2005) y n y y y y n y n (lee et al., 2006) y n y n n y n n n *(song and kim, 2006) y y n y y y n y n (li et al., 2006) y n y n n y n n n *(bagozzi et al., 2006) y y y n n y n y n *(bagozzi et al., 2007) y y y n n y y y n *(miller and brunner, 2008) y n n y n y y y y **(walther et al., 2008) n n n y n n y n n (eckhardt et al., 2009) y n n y y y n y n (yang et al., 2009) y y y n y y y y y **(mangold and faulds, 2009) n n n n y n n n n **(pempek et al., 2009) n n n y n y n n n (kulviwat et al., 2009) y y y n y y n y n **(zeng et al., 2009) n y y n n y n n n **(kaplan and haenlein, 2010) y y n n n y y n n *(shen et al., 2010) y y y y n y n y n *(posey et al., 2010) n y n n y y n n n *(huffaker, 2010) y n n n y y y y y (glass and li, 2010) y n n y n y n y y (vannoy and palvia, 2010) y n n n n y n y n **(cheung and lee, 2010) y y y n n y n y y **(zhou, 2011) y y y n n y n y n (datta, 2011) y y y n n y n y n (shen et al., 2011) y y y n n y n y n **(cheung et al., 2011) y y y n y y n y n **(kietzmann et al., 2011) n n n y n n n y y australasian journal of information systems chandrasekara, sedera & gao 2021, vol 25, research article boundary conditions of sns research 10 **(hanna et al., 2011) n n n y y n n n n **(fischer and reuber, 2011) n n n n n y n n n **(mir and zaheer, 2012) n n n n y n y y y **(kwahk and ge, 2012) y y n n y y y y y **(tufekci and wilson, 2012) n n n n y n n y n **(lipsman et al., 2012) n n n y n n y n y **(gensler et al., 2013) n n n y y n n y y (singh and phelps, 2013) y n n y y y y y n **(hildebrand et al., 2013) y y y n y y n y n (wang et al., 2013) y y y y y y y y n *(tsai and bagozzi, 2014) y y y n y y n y y *(kuan et al., 2014) y y n y y y n y y **(zhang et al., 2014) n n n y n n y n y (godinho de matos et al., 2014) n y n y n y n n y **(hu et al., 2015) n n n n n n n n n **(wang et al., 2015) n n n n n n n n n **(matook et al., 2015) y y n y n y n y n **(tussyadiah et al., 2015) n y y y y y n y n **(oh et al., 2017) n n n n y n n n n **(sedera et al., 2017) y y y y y y n y y **(thomaz et al., 2017) n n n n n n n n n **(james et al., 2017) n n n y n n n n n *(liao et al., 2017) n n n n n n n n n **(rueda et al., 2017) y y n y n y n y n **(brandt et al., 2017) n n n n n n n n n **(dewan et al., 2017) n n n y n n n n y (zhang et al., 2018) n n n y n n n n y table 4. literature synthesis note: a – compliance, b – identification, c – internalization, d – peer influence, e – informational social influence, f – normative social influence, g – immediacy, h – strength, i – number of people 3.2.1 most used constructs considering that the number of sns users had risen significantly by 2007 (kaplan & haenlein, 2010), the period before 2008 was set as the pre-sns era and 2008 onwards as the sns era for the purpose of identifying and comparing the use of social influence constructs in the literature. according to the analysis, the most discussed social influence constructs before 2008 were compliance (94%), normative social influence (94%), and strength (78%). when comparing these constructs with the most used social influence constructs after the advent of snss (2008 onwards), it was evident that normative social influence (62%) was the most discussed construct, followed by strength (57%) and compliance (49%). even though compliance remained among the top three constructs discussed, its emphasis in is studies decreased after the advent of snss. australasian journal of information systems chandrasekara, sedera & gao 2021, vol 25, research article boundary conditions of sns research 11 this finding adheres to the argument in the study by zhou (2011), which indicated that constructs such as compliance cannot be applied to snss since individuals have the ability to make voluntary choices in snss. 3.2.2 overlaps between constructs the analysis highlighted that there is a possible (and perhaps unintentional) overlap between the constructs of kelman’s social influence theory (1958) and latané’s social impact theory (1981). for instance, latané (1981, p. 344), defines strength in social impact as “the salience, power, importance, or intensity of a given source to the target”. kelman (1958) refers to compliance as a process due to which an individual would behave in a particular way in order to receive rewards or approval from a person or a group, or due to the fear of being rejected (bagozzi & lee, 2002; datta, 2011; shen et al., 2010; zhou, 2011). hence, when an individual is influenced due to compliance, it is because the influencer is either an important or a powerful person in his/her life. the literature analysis indicated compliance (49%) and strength (57%) as two of the most discussed constructs in the sns era. due to the similar nature of these constructs, when a study explicitly discusses one construct, it is possible to discuss the other construct implicitly, and that could be one reason why both constructs appear in the top three social influence constructs in is studies. further, kelman (1958) refers to identification as a process due to which a person is influenced to either establish or continue a successful relationship with another person or a group (cheung et al., 2011; wang et al., 2013). kelman (1958, p.53), adds that for these two processes to be successful, the determinants of influence, namely “the relative importance of the anticipated effect” and “the relative power of the influencing agent” should be considered. while strength in social impact research has been discussed in the perspective of the source of impact, compliance and identification have been discussed in the perspective of the target. however, power, salience, intensity, and importance can be considered as attributes that should be possessed by the influencer under compliance and identification to influence another person. therefore, such constructs of social impact should be best perceived as moderating constructs that affect the social influence processes. further, these overlaps may influence the boundary conditions discussed later on in this study. when considering the social influence constructs in terms of definitions, informational social influence showed similarity to the construct internalization (bagozzi & lee, 2002; venkatesh & davis, 2000), and peer influence showed similarity to the construct identification. it was also observed that the normative social influence construct provided a high-level observation of the overall social influence rather than providing a specific construct to measure social influence. this could be one reason why normative social influence was one of the most discussed constructs in both pre-sns era (94%) and sns era (62%) studies. thus, considering all the observed constructs of social influence in the literature sample, the constructs proposed in kelman’s social influence theory (1958): compliance, identification, and internalization were identified as the more explicit and dominating constructs of social influence. 3.2.3 the evolution of constructs the literature sample in table 4 is chronologically ordered. while it was expected that the number of studies employing social influence would increase over time (with the advent and proliferation of snss), in reality, the number of such studies is plateauing. the comparison between the periods in which the studies were published (i.e., pre-sns era and sns era) highlighted that there are no substantial differences in the application of social influence australasian journal of information systems chandrasekara, sedera & gao 2021, vol 25, research article boundary conditions of sns research 12 constructs in the two different contexts. the lack of acknowledgment of the novel context could be one of the reasons why the same social influence constructs (while there are minor differences in the order they are discussed) have been highlighted in is studies. however, when considering the least discussed constructs, it was evident that peer influence (11%), which was one of the least discussed constructs in the pre-sns era, was replaced by internalization (30%) in the sns era. the high number of members in an individual’s social circle in the sns era could be one reason why peer influence has been discussed more in the studies since 2008. most interestingly, both immediacy and number of people remained among the least discussed constructs in both the pre-sns and sns eras. the complexity in considering a variety of social influence constructs in both the pre-sns and sns eras emphasizes the necessity of investigating boundary conditions for sns by revisiting social influence theory and deriving the most appropriate constructs to examine attitudinal and behavioral changes in individuals. for instance, if the increase in interactions facilitated by the high number of people available in sns (smith, 2017) is considered, social influence constructs such as the number of people, and immediacy would become more important in the sns era compared to the pre-sns era. yet, such constructs still receive limited attention in sns studies since the social influence constructs that were considered important in the pre-sns era are being applied to the sns era without acknowledging these contextual differences. even though several new constructs, such as popularity influence and proximity influence (dewan et al., 2017) were introduced in is studies to capture the social influence generated by people in snss, the emphasis placed on these constructs is still inadequate. 3.2.4 context of the study the context of social influence is an important consideration as it can affect the study outcomes. for example, the theoretical perspective of social influence originated in the faceto-face, physical context of human interactions. however, we observed that this theory was employed by approximately 65% of studies that considered snss or virtual community as the context (denoted with an (*) for virtual community studies and (**) for sns studies in table 4). in contrast with the studies employing other social interaction media, no study in our sample explicitly observed the theoretical challenges in employing the social influence theory in snss. when considering the virtual community studies (*), bagozzi and dholakia (2002) examined how different social influence processes lead individuals to actively participate in virtual chat rooms. song and kim (2006) investigated the willingness of individuals to use virtual services based on social influence processes. while these two studies mainly focussed on how individuals would engage with a particular online service due to social influence, miller and brunner (2008) explored how different characteristics of individuals expressed through the digital sphere and their mere online presence would influence others to behave differently. when sns studies (**) are considered, zeng et al., (2009) examined how social identity and group norms affected “community users” group intentions to accept advertising in “online social networking communities” (zeng et al., 2009, p. 1). cheung and lee (2010, p.24) investigated the role of compliance, identification, and internalization in “the decision to use an online social network”. such diverse use of social influence constructs in both virtual community and sns studies indicates the necessity of rethinking the adaption of social influence constructs to the specific context of snss. in other words, due to the range of australasian journal of information systems chandrasekara, sedera & gao 2021, vol 25, research article boundary conditions of sns research 13 attitudinal and behavioural changes that take place within snss as a result of social influence, simply applying the same social influence constructs from pre sns era may not be adequate to understand how social influence operates in snss. further, while both virtual community and sns studies have highlighted the importance of adopting selected constructs of social influence, such studies have rarely addressed the necessity of further refining the theoretical perspective of social influence to apply it successfully to the context of snss. this limitation encouraged us to look into boundary conditions that could allow is researchers to streamline social influence for snss better. therefore, the next step of the analysis was to derive boundary conditions based on the study sample. we followed a logical approach to determine what boundary conditions would be suitable for snss when examining social influence. first, referring to selected seminal works on boundary conditions and theory building, we identified how boundary conditions could be utilized to better apply a theory to a particular context and yield accurate findings (bacharach, 1989; busse, kach, & wagner, 2017; dubin, 1969; weber, 2012; whetten, 1989). then, we carefully reviewed the papers in the literature sample, through several iterations, to examine the recurring notions/themes in the literature sample that have been identified as useful when applying social influence to the context of snss. for instance, if several studies in the sample identified that the ability of sns users to include or exclude people from their social circles (having a choice) should be considered when investigating social influence in snss, we looked at all the papers in the study sample, using that condition as a lens, to determine whether that condition could be applied to examine social influence in sns related research more effectively. the conditions identified in this way were included as boundary conditions of social influence for snss. we continued this exercise, on identifying the conditions considered essential for studying social influence in snss across all 65 papers and derived five boundary conditions. 4 focus on boundary conditions the literature sample demonstrates that, while we have made substantial leaps in technology by creating snss, arriving at a hyper-connected society, our theoretical foundations employed in is studies have barely changed. this notion has also been highlighted in a study on social network analysis and snss (kane, alavi, labianca, & borgatti, 2014) which claims that researchers should not merely rely on theories developed prior to the advent of snss to understand the sns related phenomena, since snss have changed the way social networks operate at present. similarly, in our study, we argue that even though social influence processes would still be affected by the influencer’s popularity, the influencer’s relationship with the influencee, and the internal values of the influencee (hovland et al., 1953) as in the pre sns era, there are certain conditions (boundary conditions) facilitated by snss that make the social influence processes different to those operating in the traditional context. in particular, only a limited number of studies in our literature sample pay attention to possible boundary conditions at least implicitly when applying a theoretical foundation that was derived based on face-to-face human influence to the context of augmented, inflated, and excessive social influence of snss. for instance, venkatesh and davis (2000) examined the concept of voluntariness as affecting compliance and highlighted that compliance would be effective only in situations where a particular action is mandatory. yet, the study is not specific to the context of snss. however, zhou (2011), when studying sns user participation, australasian journal of information systems chandrasekara, sedera & gao 2021, vol 25, research article boundary conditions of sns research 14 indicated that compliance does not affect sns user participation since participation is a voluntary decision made by the user. this factor is further discussed later in this study. while consistency in employing the constructs of social influence would lead to a cumulative tradition of research, the lack of acknowledgment of a major shift in the context face-to-face to virtual not only weakens our observations, but it also precludes us from forming a precise understanding of the nature of social influence in the new context (busse et al., 2017). hence, these gaps in the research could be addressed by revisiting boundary conditions and identifying which boundary conditions can be applied to the context of snss to analyze the actions that take place within this context. even though various disciplines have studied the nature of social influence generated in snss among individuals, much of the theoretical work underpinning these discussions has been based on the theories that were developed prior to the advent of snss. herein we specifically propose that, when considering social influence, studies must specify and observe conditions that the constructs are bounded by (dubin 1969). the boundary conditions are derived through a comparison of the conditions upon which the traditional social interactions (e.g., face-to-face) are based against the conditions upon which the digital interaction happens. these limiting values are particularly potent in the sns context. lack of boundary condition specificity in emerging research could lead to either type-i or type-ii errors. for example, you may reject the true null hypothesis (type-i) or fail to reject a false null hypothesis (type-ii), by inadvertently selecting a particular sample that may not be random. in order to better illustrate this, we derived five salient boundary conditions through the analysis of our literature sample that affect the nature of social influence in the sns era. these five factors can provide what dubin called the boundaries of the theory. dubin (1969, p.125), stated that “in order that a model may represent an empirical system, it has to have the boundaries corresponding to the empirical system. the boundaries are important to the specification of any theoretical model.” the derived five boundary conditions; (i) the choice, (ii) space and proximity, (iii) the locus of social influence, (iv) the number of people and times, (v) the diversity and the variety of social influence, would thus add meaning to the future social influence studies in the context of snss. 4.1 the choice of the social circle figure 1. the choice of the social circle 4 4 figure 1 to figure 5 have been inspired and adapted by dubin’s (1969) work on boundary conditions. australasian journal of information systems chandrasekara, sedera & gao 2021, vol 25, research article boundary conditions of sns research 15 figure 1 outlines the choice condition, which is necessary for sns studies. for instance, as illustrated in panel 1, some of your family members can be a part of your social circle in the snss, or as depicted in panel 2, all your family members can be members of your sns social circle. another condition that could be applicable for an individual in snss is illustrated in panel 3, when the individual decides to keep his/her family members away from the sns’s social circle. thus, the individual has a choice when making decisions in snss or when adding or removing a person from the social circle of a sns. bagozzi and dholakia (2002) differentiated such conditions in virtual communities from the conditions in traditional communities, stating that members of snss have a voluntary association with the membership. therein, the individual could quit the network at any given time without participation. hall and fagen (1956) described such boundary conditions as exterior boundary conditions. it is argued that when a new unit of measurement must be introduced into a theorem, the unit must be designated affirmatively, as must be the theorem (dubin 1969). hall and fagen (1956) emphasized that in cases when an exterior boundary condition changes the outcome of a proposed model, such variables should be considered as intervening variables (moderation, mediation assumed). therefore, in social influence studies, the choice one would exert in determining one’s own social network should be considered. zhang et al. (2018) investigated the direct vs. indirect peer influence in social networks, arguing that “peer influence can arise from immediate neighbors in the network and from indirect peers who share common neighbors (p. 1)”. their findings, based on an analysis of call logs, demonstrate significant differences between the direct and indirect influences. given that in snss, individuals are exposed to a large number of direct and indirect peer influences at once, the choice of an individual in snss to exclude/include people from his/her social circles affects the influence they receive. identifying the most used constructs in the literature sample, such as normative social influence, compliance, and identification, further confirmed the necessity of considering the boundary condition choice of the social circle. in snss, individuals have to comply with diverse types of rules because the individual can belong to many groups. zhou (2011) highlighted that since participation in snss is a voluntary choice a person makes; the opinions of others do not matter in decision making. in contrast, cheung and lee (2010) showed that compliance and identification determine a person’s motive for using snss. even though both cheung and lee (2010) and zhou (2011) discussed only the user’s participation in snss, these studies confirm the necessity of a boundary condition on choice for snss. therefore, for studies on social influence, the choice condition should be specified, as the social influence would only arise from those whom you have selected to be included in your sns profile. for example, when constructs like (a) compliance, (b) identification, (c) internalization or (d) peer influence are considered, the choice that the individual has consciously made must be considered. similarly, when measuring social impact through constructs like (g) immediacy and (h) strength, these constructs are be impacted by the choice condition. australasian journal of information systems chandrasekara, sedera & gao 2021, vol 25, research article boundary conditions of sns research 16 4.2 space and proximity figure 2. space and proximity figure 2 outlines an example of a truth table for the proximity condition. we argue that both ‘p’ and ‘q’ must be understood in social influence studies, where ‘p’ is the digital proximity, and ‘q’ is the physical proximity that one has to a network or its subjects. by digital proximity, we mean the proximity between sns users in terms of belonging to the same online group, or being members of the same sns page, due to which the sns users feel a form of connectedness. by physical proximity, we mean the proximity maintained due to one’s geographical location. people belonging to the same region or area would be influenced by each other more. latané and bourgeois (1996) in their works on the dynamic social impact theory5 highlighted that social influence, at least concerning memorable interactions, seems to decrease with the square of the distance. however, the same study suggests that with the influence of technology, the social space between people will be affected, and the effective distance between people reduced. digital proximity is how this aspect is changed due to the influence of the technology, where people feel closer to each other in spite of their physical distance. when applied to social influence in snss, the presence of both physical and digital proximity between an influencer and an influencee would mean social influence processes such as (c) internalization as investigated in table 4, would become more optimal. similarly, the proximity of influence must be considered in (g) immediacy to recognize the physical and digital distances of the influencer. the consideration of a boundary condition on space and proximity allows us to examine how different social influence processes would become less or more optimal based on digital and physical proximities between influencers and influencees. our literature analysis showed that one of the least used constructs in the study sample was immediacy, which indicates that researchers have underutilized constructs that could add more depth to our understanding of social influence in snss. however, there are sns and digital space focused studies that have identified the application of proximity conditions in digital interactions (miller & brunner, 2008; singh & phelps, 2013; wang et al., 2013). for instance, the study by dewan et al. (2018), implicitly discusses the proximity condition. they acknowledge the difficulties in assessing proximity in snss, stating that “identifying proximity influence using observational data is challenging due to homophily, which may influence both the formation of social ties and music consumption decisions” (p. 119). furthermore, miller & brunner (2008, p. 2977), in examining online social influence using the theory of social impact, discussed the immediacy construct in online communities as “a participant’s proximity to the chat room dialogue as evidenced by a participant’s number of 5 dynamic social impact theory consists of four constructs, namely clustering, correlation, consolidation and continuing diversity. latané and bourgeois (1996) have utilized these 4 constructs to analyze user behaviors in electronic groups and thereby to explore the operation of social impact in social space. australasian journal of information systems chandrasekara, sedera & gao 2021, vol 25, research article boundary conditions of sns research 17 contributions and their total number of contributed words”. pempek, yermolayeva, and calvert (2009) indicated that college students maintain connections on snss, usually with people who have been known to them in the physical world. these previous works highlight how physical proximity can influence sns users in their decisions and confirm the necessity of considering both digital proximity and physical proximity when engaging in social influence research in the context of snss. 4.3 the locus of social influence figure 3. the locus of social influence figure 3 denotes the locus of social influence. it recognizes that there are multiple realms of networks, not just one layer of social network around an individual. as such, social influence may arise from any of those realms. however, it is asserted that not all realms have the same level of social influence. depending on the nature of the event and the nature of the influencer or the group affiliated, the individual could be receptive to the influencer. for example, the #metoo movement against sexual harassment had active participation from 85 countries (park, 2017). furthermore, the locus of social influence would have an effect on studies employing the (c) internalization as an aspect of social influence. given the realms of social networks, internalization may be short-term or long-lasting, depending on the locus of social influence. for instance, tufekci and wilson (2012), examining a political protest in egypt, indicated how the political content generated by a few groups such as journalists, and activists, escalated to the level of a real-world protest due to the adoption and acceptance of the political ideology spread through snss by the general public. particularly as depicted in table 4, this study has implicitly identified social influence constructs such as (e) informational influence and (h) strength to elaborate how individuals coming from various realms in snss motivated other individuals to take part in this political protest. as such, exploring the boundary condition of locus of social influence in future studies would enable us to explore social influence processes more precisely, considering the nature of the influencers in changing the attitudes and behaviors of other individuals using snss. in addition, the ice bucket challenge can be considered as another good example that shows the applicability of the boundary condition of the locus of social influence. the ice bucket challenge was first initiated by celebrities who are famous worldwide (ni, chan, leung, lau, & pang, 2014). as per facebook (2014) “over 28 million people have joined the conversation about the ice bucket challenge including posting, commenting or liking a challenge post” and australasian journal of information systems chandrasekara, sedera & gao 2021, vol 25, research article boundary conditions of sns research 18 “2.4 million videos related to the ice bucket challenge have been shared on facebook”. this trend implicitly denotes the nature of social influence in snss. it also emphasizes how an initiative of celebrities could escalate into a mass sharing of videos by the sns population that would motivate individuals to act based on the influence generated in various realms. cheung, chiu, and lee (2011, p. 1340) stated that in snss, “if a user joins too many communities, it is hard for him/her to create a sense of belonging to a specific group”. in such instances, the users may be confused in determining whether or not to engage in a certain action, as he/she could be receiving contradictory messages from multiple influencers. as such, the observations we made under the analysis of the literature sample denoted that further investigation of boundary conditions such as the locus of social influence is crucial in examining the social influence in complex platforms such as snss. 4.4 the number of people and number of times figure 4. the number of people and times figure 4 demonstrates the possible effect that the number of people and the number of times has on social influence constructs. herein, we claim that the more people you are exposed to in snss, and the more times a particular message is circulated, the higher the tendency of a person to be influenced in the context of snss. for instance, facebook indicates that individuals have an average of 338 friends (smith, 2017). while the exact number may be debatable, it is evident that snss allow one to connect with more friends than pre sns era (mangold and faulds 2009). when a statement or a post is cited by a high number of people in your network, it is likely to have a strong influence. the fundamental premise of repeated message (black, 1949) is developed on the premise of the influence of the number and time premises. thus, constructs like (e) informational social influence, and (f) normative social influence will be affected by the number of people and number of times. for instance, kuan et al. (2014) in their study on online buying behaviors, highlighted that information on the number of people who have bought a product (informational social influence, and thereby the number of times and number of people) and the type of people who have liked a product (normative influence) in snss, affect the purchase decisions of consumers. as illustrated in table 4, the study of kuan et al. (2014) has investigated many constructs of social influence such as (a) compliance, (b) identification, (d) peer influence, (e) informational social influence, (f) normative social influence, (h) strength and (i) number of people, implicitly to examine the actions of consumers. herein, the application of the boundary condition of the number of people and number of times would allow us to explore the power that lies in snss to spread social influence to a larger population due to the repetition of the same message by many influencers in snss. furthermore, kwahk & ge (2012, p.1817) in studying the impact of snss on e-commerce, state that “more social media interaction ties can bring higher group pressure and lead one to conform to the group” (number of people). this aspect also denotes the australasian journal of information systems chandrasekara, sedera & gao 2021, vol 25, research article boundary conditions of sns research 19 possibility of applying the boundary condition of the number of people and number of times to sns research in examining social influence. 4.5 the diversity and the variety of social influence figure 5. the diversity and the variety of social influence figure 5 recognizes the diversity and the variety of connections maintained in snss (walther et al. 2008). as expressed in figure 5, maleness plus movies has a narrower domain than either one alone. when comparing the diversity and the variety of social influence in traditional vs. snss, kwahk and ge (2012, p. 1816) indicated that in the past, social circles had narrower diversity, but with the proliferation of snss, social influence is getting “broader and stronger” making the presence of snss a critical aspect of peoples’ lives. shen, lee, cheung, and chen (2010) explored how social influence and other related factors affect men and women differently when taking part in snss based team collaborations. particularly when investigating multiple constructs of social influence in the context of snss such as (a) compliance, (b) identification, (d) peer influence, (h) strength as depicted in table 4, diversity and the variety of social influence allows us to examine the role of diverse influencers in snss in encouraging individuals to change attitudes or behaviors. the researcher has an obligation to specify which spectrum of social influence is sought based on the diversity and the variety of social influence a study population receives in snss. in summary, the literature analysis conducted in the study indicates that applying social influence directly to snss and other digital spaces without considering these theoretical boundaries could lead to inconsistencies as well as ambiguities in the findings (busse et al., 2017). as such, the above mentioned five boundary conditions, which we derived through several iterations of the literature review, would guide future research to apply social influence theoretical constructs to snss precisely and generate valid results relevant to the specific context of snss (busse et al., 2017). 5 conclusion social influence is an important stream of research that allows us to understand how the interactions among individuals in society lead to attitude and behavioral change. the existing theoretical foundations that investigate social influence, particularly in is are under increasing pressure due to the advent and proliferation of snss. while much of the works on social influence have made substantial contributions to research, studies lack a nuanced view of the boundary conditions that are entrenched in snss. for instance, when utilizing the constructs generated by the social influence theories, previous studies have identified that not all constructs that generate social influence in the physical world can be applied in the same way to understand the social influence in snss (datta, 2011; shen, 2010; zhou, 2011). by identifying australasian journal of information systems chandrasekara, sedera & gao 2021, vol 25, research article boundary conditions of sns research 20 this limitation in applying social influence on snss, this paper derived five boundary conditions based on a literature analysis and demonstrated how recognizing such boundary conditions up-front can provide better insights (bacharach, 1989; busse et al., 2017; dubin, 1969; gonzalez-mulé & aguinis, 2018; whetten, 1989). our analysis of the 65 past studies suggests that the boundary conditions are applicable to understand social influence in the contemporary setting of snss. the overlaps between the constructs of social influence, the acknowledgment in previous studies regarding the contextual differences between the society before and after the advent of snss, and the application of the five boundary conditions to constructs of social influence suggest the validity of this broader application. through this study, we make both theoretical and practical contributions to is. even though many is studies have considered the theoretical perspective of social influence in examining individual attitude and behavioral changes, to date, no study has explicitly acknowledged the necessity of developing boundary conditions for sns research. as such, the findings of this study can be utilized to investigate social influence in snss further. the identification of the five boundary conditions means that researchers can now develop instruments and procedures to measure these conditions (busse et al. 2017; dubin, 1969; weber, 2012). therefore, in terms of knowledge generation, we believe the findings of this study will serve as a lens in the domain of is, allowing us to look at social influence generated by snss from a new perspective. furthermore, seminal works on theory development have highlighted the contribution of boundary conditions in assisting researchers to generate valid findings (busse et al., 2017; dubin, 1969) and improving the generalizability of theories (busse et al., 2017; whetten, 1989). similarly, by exploring five boundary conditions for social influence research in snss, we have identified that paying closer attention to boundary conditions when applying an existing theory to a new context would allow future studies to contribute to more rigorous research. the derived boundary conditions would also enable future studies to be more specific when using social influence and make their findings more relevant to the context of snss. further to this, practitioners will be able to develop strategies which can yield better outcomes with regard to their sns campaigns. notably, in terms of sns marketing, the boundary conditions would allow practitioners to identify the optimal conditions for maximum social influence, and thus, encourage different behaviors among consumers. moreover, we should also bear in mind that the exploration of boundary conditions has already been proven to reduce the gap between theory and practice (busse et al., 2017). as such, whenever a researcher applies theory to any novel context including snss, determining the boundary conditions of the selected theory and examining whether or not the theory fits into the new contextual dimensions would not only facilitate the generation of accurate results but also ensure that the findings have practical implications. however, one of the limitations we would like to acknowledge in this study is the formulation of boundary conditions based only on literature analysis. the five boundary conditions were determined based on literature, and the number of boundary conditions for snss social influence studies may vary depending on further empirical tests. for instance, conducting a social network analysis to examine the applicability of these boundary conditions would have been useful for the study. nevertheless, our aim at this point is to highlight the necessity of developing boundary conditions for snss social influence studies, which we have australasian journal of information systems chandrasekara, sedera & gao 2021, vol 25, research article boundary conditions of sns research 21 accomplished through this study. our study encourages future research to incorporate these boundary conditions for social influence in studies on snss so that the theoretical constructs of social influence can be applied in a novel way to relate to the context of snss. further, the derivation of boundary conditions for the theoretical perspective of social influence can be improved if a comparative analysis of social influence theories could be conducted across different disciplines. even though our findings reliant mainly on is literature, included research findings from other disciplines including psychology, sociology, marketing, management studies, political science and communication studies, it was not within the scope of our study to conduct a comparative analysis of the use of social influence theoretical constructs for snss by placing equal emphasis on studies published in all disciplines. references aral, s., & walker, d. (2011). creating social contagion through viral product design: a randomized trial of peer influence in networks. management science, 57(9), 1623-1639. arvidsson, v., & holmström, j. (2013). social media strategy: understanding social media, it strategy, and organizational responsiveness in times of crisis. cutter it journal, 26(12), 18-23. bacharach, s. b. (1989). organizational theories: some criteria for evaluation. academy of management review, 14(4), 496-515. bagozzi, r. p., & dholakia, u. m. (2002). intentional social action in virtual communities. journal of interactive marketing, 16(2), 2-21. bagozzi, r. p., dholakia, u. m., & mookerjee, a. (2006). individual and group bases of social influence in online environments. media psychology, 8(2), 95-126. bagozzi, r. p., dholakia, u. m., & pearo, l. r. k. (2007). antecedents and consequences of online social interactions. media psychology, 9(1), 77-114. bagozzi, r. p., & lee, k. h. (2002). multiple routes for social influence: the role of compliance, internalization, and social identity. social psychology quarterly, 65(3), 226-247. bapna, r., & umyarov, a. (2015). do your online friends make you pay? a randomized field experiment on peer influence in online social networks. management science, 61(8), 19021920. black, j. w. (1949). the relation between message-type and vocal rate and intensity. speech monographs, 16(2), 217-220. boyd, d. m., & ellison, n. b. (2007). social network sites: definition, history, and scholarship. journal of computer-mediated communication, 13(1), 210-230. doi:10.1111/j.10836101.2007.00393. busse, c., kach, a. p., & wagner, s. m. (2017). boundary conditions: what they are, how to explore them, why we need them, and when to consider them. organizational research methods, 20(4), 574-609. chaffey, d. (2020). global social media research summary august 2020. retrieved from https://www.smartinsights.com/social-media-marketing/social-media-strategy/newglobal-social-media-research/ australasian journal of information systems chandrasekara, sedera & gao 2021, vol 25, research article boundary conditions of sns research 22 chan, k. h., skoumpopoulou, d., & yu, q. (2018). firestorms on social media: effects of social information characteristics on customer responses. proceedings of the twenty-second pacific asia conference on information systems. yokohama, japan. https://aisel.aisnet.org. chang, i. c., liu, c. c., & chen, k. (2014). the push, pull and mooring effects in virtual migration for social networking sites. information systems journal, 24(4), 323-346. doi:10.1111/isj.12030 cheung, c. m., chiu, p.y., & lee, m. k. (2011). online social networks: why do students use facebook? computers in human behavior, 27(4), 1337-1343. cheung, c. m., & lee, m. k. (2010). a theoretical model of intentional social action in online social networks. decision support systems, 49(1), 24-30. cialdini, r. b., & goldstein, n. j. (2004). social influence: compliance and conformity. annual review of psychology, 55(1), 591-621. coleman, j. s. (1986). social theory, social research, and a theory of action. american journal of sociology, 91(6), 1309-1335. datta, p. (2011). a preliminary study of ecommerce adoption in developing countries. information systems journal, 21(1), 3-32. deutsch, m., & gerard, h. b. (1955). a study of normative and informational social influences upon individual judgment. the journal of abnormal and social psychology, 51(3), 629-636. dewan, s., ho, y.j., & ramaprasad, j. (2017). popularity or proximity: characterizing the nature of social influence in an online music community. information systems research, 28(1), 117-136. dholakia, u. m., bagozzi, r. p., & pearo, l. k. (2004). a social influence model of consumer participation in networkand small-group-based virtual communities. international journal of research in marketing, 21(3), 241-263. dubin, r. (1969). theory building. new york: free press. eckhardt, a., laumer, s., & weitzel, t. (2009). who influences whom? analyzing workplace referents’ social influence on it adoption and non-adoption. journal of information technology, 24(1), 11-24. facebook. (2014). the ice bucket challenge on facebook. retrieved from https: //newsroom.fb.com/news/2014/08/the-ice-bucket-challenge-on-facebook/ fischer, e., & reuber, a. r. (2011). social interaction via new social media: (how) can interactions on twitter affect effectual thinking and behavior? journal of business venturing, 26(1), 1-18. galaskiewicz, j., & wasserman, s. (1993). social network analysis: concepts, methodology, and directions for the 1990s. sociological methods & research, 22(1), 3-22. garton, l., haythornthwaite, c., & wellman, b. (1997). studying online social networks. journal of computer-mediated communication, 3(1), jcmc313. godinho de matos, m., ferreira, p., & krackhardt, d. (2014). peer influence in the diffusion of the iphone 3g over a large social network. mis quarterly, 38(4), 1103-1133. australasian journal of information systems chandrasekara, sedera & gao 2021, vol 25, research article boundary conditions of sns research 23 golder, s. a., & macy, m. w. (2014). digital footprints: opportunities and challenges for online social research. annual review of sociology, 40(1), 129-152. doi:10.1146/annurev-soc071913-043145 gonzalez-mulé, e., & aguinis, h. (2018). advancing theory by assessing boundary conditions with metaregression: a critical review and best-practice recommendations. journal of management, 44(6), 2246-2273. grottke, m., hacker, j. v., & durst, c. (2018). which factors determine our online social capital? an analysis based on structural equation modelling. australasian journal of information systems, 22. https://doi.org/10.3127/ajis.v22i0.1656 hall, a. d., & fagen, r. e. (1956). definition of system. general systems, 1(1), 18-28. hovland, c. i., janis, i. l., & kelley, h. h. (1953). communication and persuasion; psychological studies of opinion change. new haven: yale university press. kandel, e., & lazear, e. p. (1992). peer pressure and partnerships. journal of political economy, 100(4), 801-817. kane, g. c., alavi, m., labianca, g., & borgatti, s. p. (2014). what’s different about social media networks? a framework and research agenda. mis quarterly, 38(1), 274-304. kaplan, a. m., & haenlein, m. (2010). users of the world, unite! the challenges and opportunities of social media. business horizons, 53(1), 59-68. kelman, h. c. (1958). compliance, identification, and internalization three processes of attitude change. journal of conflict resolution, 2(1), 51-60. krippendorff, k. (1980). content analysis: an introduction to its methodology. beverly hills, ca: sage. kuan, k. k., zhong, y., & chau, p. y. (2014). informational and normative social influence in group-buying: evidence from self-reported and eeg data. journal of management information systems, 30(4), 151-178. kwahk, k.y., & ge, x. (2012). the effects of social media on e-commerce: a perspective of social impact theory. 45th hawaii international conference on system sciences: ieee, 18141823. latané, b. (1981). the psychology of social impact. american psychologist, 36(4), 343-356. latané, b., & bourgeois, m. j. (1996). experimental evidence for dynamic social impact: the emergence of subcultures in electronic groups. journal of communication, 46(4), 35-47. maccarthy, r. (2016). the average twitter user now has 707 followers. retrieved from https://kickfactory.com/blog/average-twitter-followers-updated-2016/ mangold, w. g., & faulds, d. j. (2009). social media: the new hybrid element of the promotion mix. business horizons, 52(4), 357-365. mcpherson, j. m., popielarz, p. a., & drobnic, s. (1992). social networks and organizational dynamics. american sociological review, 57(2), 153-170. miller, m. d., & brunner, c. c. (2008). social impact in technologically-mediated communication: an examination of online influence. computers in human behavior, 24(6), 2972-2991. australasian journal of information systems chandrasekara, sedera & gao 2021, vol 25, research article boundary conditions of sns research 24 mir, i., & zaheer, a. (2012). verification of social impact theory claims in social media context. journal of internet banking and commerce, 17(1), 1-15. morgan, k., cheong, m., & bedingfield, s. (2018). “power to the people!”: social media discourse on regional energy issues in australia. australasian journal of information systems, 22. https://doi.org/10.3127/ajis.v22i0.1678 nemeth, c. j. (1986). differential contributions of majority and minority influence. psychological review, 93(1), 23-32. ni, m. y., chan, b. h., leung, g. m., lau, e. h., & pang, h. (2014). transmissibility of the ice bucket challenge among globally influential celebrities: retrospective cohort study. bmj, 349, g7185. oh, c., roumani, y., nwankpa, j. k., & hu, h.f. (2017). beyond likes and tweets: consumer engagement behavior and movie box office in social media. information & management, 54(1), 25-37. palekar, s., atapattu, m. r., sedera, d., & lokuge, s. (2018). exploring spiral of silence in digital social networking spaces. paper presented at the international conference on information systems (icis 2015): exploring the information frontier. palekar, s., & sedera, d. (2018). influence of social broadcasting networks on news consumption. australasian journal of information systems, 22. https://doi.org/10.3127/ajis.v22i0.1707 park, a. (2017). #metoo reaches 85 countries with 1.7m tweets. retrieved from https://www.cbsnews.com/news/metoo-reaches-85-countries-with-1-7-million-tweets/ pempek, t. a., yermolayeva, y. a., & calvert, s. l. (2009). college students’ social networking experiences on facebook, journal of applied developmental psychology, 30(3), 227-238. plumx metrics plum analytics. (n.d.). retrieved from https://plumanalytics.com/learn/aboutmetrics/ sedera, d., lokuge, s., atapattu, m., & gretzel, u. (2017). likes—the key to my happiness: the moderating effect of social influence on travel experience. information & management, 54(6), 825-836. shen, a. x. l., lee, m. k. o., cheung, c. m. k., & chen, h. (2010). gender differences in intentional social action: we-intention to engage in social network-facilitated team collaboration. journal of information technology, 25(2), 152-169. singh, p. v., & phelps, c. (2013). networks, social influence, and the choice among competing innovations: insights from open source software licenses. information systems research, 24(3), 539-560. doi:10.1287/isre.1120.0449 smith, a. (2014). what people like and dislike about facebook. retrieved from http://www.pewresearch.org/fact-tank/2014/02/03/what-people-like-dislike-aboutfacebook/ smith, k. (2017). marketing: 47 incredible facebook statistics. retrieved from https://www.brandwatch.com/blog/47-facebook-statistics/ song, j., & kim, y. j. (2006). social influence process in the acceptance of a virtual community service. information systems frontiers, 8(3), 241-252. australasian journal of information systems chandrasekara, sedera & gao 2021, vol 25, research article boundary conditions of sns research 25 tsai, h.t., & bagozzi, r. p. (2014). contribution behavior in virtual communities: cognitive, emotional, and social influences. mis quarterly, 38(1),143-163. tufekci, z., & wilson, c. (2012). social media and the decision to participate in political protest: observations from tahrir square. journal of communication, 62(2), 363-379. vannoy, s. a., & palvia, p. (2010). the social influence model of technology adoption. communications of the acm, 53(6), 149-153. venkatesh, v., & davis, f. d. (2000). a theoretical extension of the technology acceptance model: four longitudinal field studies. management science, 46(2), 186-204. walther, j. b., van der heide, b., kim, s. y., westerman, d., & tong, s. t. (2008). the role of friends’ appearance and behavior on evaluations of individuals on facebook: are we known by the company we keep? human communication research, 34(1), 28-49. wang, y., alahakoon, d., & de silva, d. (2018). an extended cognitive situation model for capturing subjective dynamics of events from social media. australasian journal of information systems, 22. https://doi.org/10.3127/ajis.v22i0.1701 wang, y., meister, d. b., & gray, p. h. (2013). social influence and knowledge management systems use: evidence from panel data. mis quarterly, 37(1), 299-313. wang, y.y., susarla, a., & sambamurthy, v. (2015). the untold story of social media on offline sales: the impact of facebook in the us automobile industry. international conference on information systems. fort worth, texas, usa. weber, r. (2012). evaluating and developing theories in the information systems discipline. journal of the association for information systems, 13(1), 1-30. webster, j., & watson, r. t. (2002). analyzing the past to prepare for the future: writing a literature review. mis quarterly, 26(2), xiii-xxiii. whetten, d. a. (1989). what constitutes a theoretical contribution? academy of management review, 14(4), 490-495. whiteside, n., aleti, t., pallant, j., & zeleznikow, j. (2018). helpful or harmful? exploring the impact of social media usage on intimate relationships. australasian journal of information systems, 22. https://doi.org/10.3127/ajis.v22i0.1653 zeng, f., huang, l., & dou, w. (2009). social factors in user perceptions and responses to advertising in online social networking communities. journal of interactive advertising, 10(1), 1-13. zhang, j., & centola, d. (2019). social networks and health: new developments in diffusion, online and offline. annual review of sociology, 45(1), 91-109. doi:10.1146/annurev-soc073117-041421 zhang, k., lee, m. k., & zhao, s. j. (2010). understanding the informational social influence of online review platforms. proceedings of the international conference on information systems. st. louis, missouri, usa. https://aisel.aisnet.org. zhang, b., pavlou, p. a., & krishnan, r. (2018). on direct vs. indirect peer influence in large social networks. information systems research, 29(2), 292-314. australasian journal of information systems chandrasekara, sedera & gao 2021, vol 25, research article boundary conditions of sns research 26 zhang, b., susarla, a., & krishnan, r. (2014). a bayesian model to predict content creation with two-sided peer influence in content platforms. retrieved from http://dx.doi.org/10.2139/ssrn.2554689. zhao, k., stylianou, a. c., & zheng, y. (2018). sources and impacts of social influence from online anonymous user reviews. information & management, 55(1), 16-30. zhou, t. (2011). understanding online community user participation: a social influence perspective. internet research, 21(1), 67-81. appendix a – list of references for the literature synthesis table (table 4) algesheimer, r., dholakia, u. m., & herrmann, a. (2005). the social influence of brand community: evidence from european car clubs. journal of marketing, 69(3), 19-34. bagozzi, r. p., & dholakia, u. m. (2002). intentional social action in virtual communities. journal of interactive marketing, 16(2), 2-21. bagozzi, r. p., dholakia, u. m., & mookerjee, a. (2006). individual and group bases of social influence in online environments. media psychology, 8(2), 95-126. bagozzi, r. p., dholakia, u. m., & pearo, l. r. k. (2007). antecedents and consequences of online social interactions. media psychology, 9(1), 77-114. brandt, t., bendler, j., & neumann, d. (2017). social media analytics and value creation in urban smart tourism ecosystems. information & management, 54(6), 703-713. cheung, c. m., chiu, p.y., & lee, m. k. (2011). online social networks: why do students use facebook? computers in human behavior, 27(4), 1337-1343. cheung, c. m., & lee, m. k. (2010). a theoretical model of intentional social action in online social networks. decision support systems, 49(1), 24-30. datta, p. (2011). a preliminary study of ecommerce adoption in developing countries. information systems journal, 21(1), 3-32. dewan, s., ho, y. j., & ramaprasad, j. (2017). popularity or proximity: characterizing the nature of social influence in an online music community. information systems research, 28(1), 117-136. dholakia, u. m., bagozzi, r. p., & pearo, l. k. (2004). a social influence model of consumer participation in networkand small-group-based virtual communities. international journal of research in marketing, 21(3), 241-263. dholakia, u. m., & talukdar, d. (2004). how social influence affects consumption trends in emerging markets: an empirical investigation of the consumption convergence hypothesis. psychology & marketing, 21(10), 775-797. eckhardt, a., laumer, s., & weitzel, t. (2009). who influences whom? analyzing workplace referents’ social influence on it adoption and non-adoption. journal of information technology, 24(1), 11-24. fischer, e., & reuber, a. r. (2011). social interaction via new social media: (how) can interactions on twitter affect effectual thinking and behavior? journal of business venturing, 26(1), 1-18. australasian journal of information systems chandrasekara, sedera & gao 2021, vol 25, research article boundary conditions of sns research 27 gallivan, m. j., spitler, v. k., & koufaris, m. (2005). does information technology training really matter? a social information processing analysis of coworkers’ influence on it usage in the workplace. journal of management information systems, 22(1), 153-192. gensler, s., völckner, f., liu-thompkins, y., & wiertz, c. (2013). managing brands in the social media environment. journal of interactive marketing, 27(4), 242-256. glass, r., & li, s. (2010). social influence and instant messaging adoption. journal of computer information systems, 51(2), 24-30. godinho de matos, m., ferreira, p., & krackhardt, d. (2014). peer influence in the diffusion of the iphone 3g over a large social network. mis quarterly, 38(4), 1103-1134. hanna, r., rohm, a., & crittenden, v. l. (2011). we’re all connected: the power of the social media ecosystem. business horizons, 54(3), 265-273. hildebrand, c., häubl, g., herrmann, a., & landwehr, j. r. (2013). when social media can be bad for you: community feedback stifles consumer creativity and reduces satisfaction with self-designed products. information systems research, 24(1), 14-29. hu, t., kettinger, w. j., & poston, r. s. (2015). the effect of online social value on satisfaction and continued use of social media. european journal of information systems, 24(4), 391410. huffaker, d. (2010). dimensions of leadership and social influence in online communities. human communication research, 36(4), 593-617. james, t. l., wallace, l., warkentin, m., kim, b. c., & collignon, s. e. (2017). exposing others’ information on online social networks (osn): perceived shared risk, its determinants, and its influence on osn privacy control use. information & management, 54(7), 851865. kaplan, a. m., & haenlein, m. (2010). users of the world, unite! the challenges and opportunities of social media. business horizons, 53(1), 59-68. kelman, h. c. (1958). compliance, identification, and internalization three processes of attitude change. journal of conflict resolution, 2(1), 51-60. kietzmann, j. h., hermkens, k., mccarthy, i. p., & silvestre, b. s. (2011). social media? get serious! understanding the functional building blocks of social media. business horizons, 54(3), 241-251. kuan, k. k., zhong, y., & chau, p. y. (2014). informational and normative social influence in group-buying: evidence from self-reported and eeg data. journal of management information systems, 30(4), 151-178. kulviwat, s., bruner ii, g. c., & al-shuridah, o. (2009). the role of social influence on adoption of high-tech innovations: the moderating effect of public/private consumption. journal of business research, 62(7), 706-712. kwahk, k.y., & ge, x. (2012). the effects of social media on e-commerce: a perspective of social impact theory. the 45th hawaii international conference on system science (hicss), 1814-1823. latané, b. (1981). the psychology of social impact. american psychologist, 36(4), 343-356. australasian journal of information systems chandrasekara, sedera & gao 2021, vol 25, research article boundary conditions of sns research 28 latané, b. (1996). dynamic social impact: the creation of culture by communication. journal of communication, 46(4), 13-25. latané, b., & bourgeois, m. j. (1996). experimental evidence for dynamic social impact: the emergence of subcultures in electronic groups. journal of communication, 46(4), 35-47. latané, b., liu, j. h., nowak, a., bonevento, m., & zheng, l. (1995). distance matters: physical space and social impact. personality and social psychology bulletin, 21(8), 795-805. lee, y., lee, j., & lee, z. (2006). social influence on technology acceptance behavior: selfidentity theory perspective. acm sigmis database: the database for advances in information systems, 37(2-3), 60-75. li, x., hess, t. j., & valacich, j. s. (2006). using attitude and social influence to develop an extended trust model for information systems. acm sigmis database: the database for advances in information systems, 37(2-3), 108-124. liao, c., lin, h.n., luo, m. m., & chea, s. (2017). factors influencing online shoppers’ repurchase intentions: the roles of satisfaction and regret. information & management, 54(5), 651-668. lipsman, a., mudd, g., rich, m., & bruich, s. (2012). the power of “like”: how brands reach (and influence) fans through social-media marketing. journal of advertising research, 52(1), 40-52. lu, j., yao, j. e., & yu, c.s. (2005). personal innovativeness, social influences and adoption of wireless internet services via mobile technology. the journal of strategic information systems, 14(3), 245-268. mangold, w. g., & faulds, d. j. (2009). social media: the new hybrid element of the promotion mix. business horizons, 52(4), 357-365. matook, s., brown, s. a., & rolf, j. (2015). forming an intention to act on recommendations given via online social networks. european journal of information systems, 24(1), 76-92. miller, m. d., & brunner, c. c. (2008). social impact in technologically-mediated communication: an examination of online influence. computers in human behavior, 24(6), 2972-2991. mir, i., & zaheer, a. (2012). verification of social impact theory claims in social media context. journal of internet banking and commerce, 17(1), 1-15. nowak, a., szamrej, j., & latané, b. (1990). from private attitude to public opinion: a dynamic theory of social impact. psychological review, 97(3), 362-376. oh, c., roumani, y., nwankpa, j. k., & hu, h. f. (2017). beyond likes and tweets: consumer engagement behavior and movie box office in social media. information & management, 54(1), 25-37. pempek, t. a., yermolayeva, y. a., & calvert, s. l. (2009). college students’ social networking experiences on facebook. journal of applied developmental psychology, 30(3), 227-238. posey, c., lowry, p. b., roberts, t. l., & ellis, t. s. (2010). proposing the online community self-disclosure model: the case of working professionals in france and the uk who use online communities. european journal of information systems, 19(2), 181-195. australasian journal of information systems chandrasekara, sedera & gao 2021, vol 25, research article boundary conditions of sns research 29 rueda, l., benitez, j., & braojos, j. (2017). from traditional education technologies to student satisfaction in management education: a theory of the role of social media applications. information & management, 54(8), 1059-1071. sedera, d., lokuge, s., atapattu, m., & gretzel, u. (2017). likes—the key to my happiness: the moderating effect of social influence on travel experience. information & management, 54(6), 825-836. shen, a. x., cheung, c. m., lee, m. k., & chen, h. (2011). how social influence affects weintention to use instant messaging: the moderating effect of usage experience. information systems frontiers, 13(2), 157-169. shen, a. x. l., lee, m. k. o., cheung, c. m. k., & chen, h. (2010). gender differences in intentional social action: we-intention to engage in social network-facilitated team collaboration. journal of information technology, 25(2), 152-169. singh, p. v., & phelps, c. (2013). networks, social influence, and the choice among competing innovations: insights from open source software licenses. information systems research, 24(3), 539-560. song, j., & kim, y. j. (2006). social influence process in the acceptance of a virtual community service. information systems frontiers, 8(3), 241-252. thomaz, g. m., biz, a. a., bettoni, e. m., mendes-filho, l., & buhalis, d. (2017). content mining framework in social media: a fifa world cup 2014 case analysis. information & management, 54(6), 786-801. tsai, h.t., & bagozzi, r. p. (2014). contribution behavior in virtual communities: cognitive, emotional, and social influences. mis quarterly, 38(1), 143-163. tufekci, z., & wilson, c. (2012). social media and the decision to participate in political protest: observations from tahrir square. journal of communication, 62(2), 363-379. tussyadiah, i. p., kausar, d. r., & soesilo, p. k. (2015). the effect of engagement in online social network on susceptibility to influence. journal of hospitality & tourism research, 42(2), 201-223. vannoy, s. a., & palvia, p. (2010). the social influence model of technology adoption. communications of the acm, 53(6), 149-153. venkatesh, v., & davis, f. d. (2000). a theoretical extension of the technology acceptance model: four longitudinal field studies, management science, 46(2), 186-204. walther, j. b., van der heide, b., kim, s. y., westerman, d., & tong, s. t. (2008). the role of friends’ appearance and behavior on evaluations of individuals on facebook: are we known by the company we keep? human communication research, 34(1), 28-49. wang, y.y., susarla, a., & sambamurthy, v. (2015). the untold story of social media on offline sales: the impact of facebook in the us automobile industry. thirty sixth international conference on information systems. fort worth, texas, usa. wang, y., meister, d. b., & gray, p. h. (2013). social influence and knowledge management systems use: evidence from panel data. mis quarterly, 37(1), 299-313. australasian journal of information systems chandrasekara, sedera & gao 2021, vol 25, research article boundary conditions of sns research 30 yang, h.d., moon, y. j., & rowley, c. (2009). social influence on knowledge worker’s adoption of innovative information technology. journal of computer information systems, 50(1), 25-36. zeng, f., huang, l., & dou, w. (2009). social factors in user perceptions and responses to advertising in online social networking communities. journal of interactive advertising, 10(1), 1-13. zhang, b., pavlou, p. a., & krishnan, r. (2018). on direct vs. indirect peer influence in large social networks. information systems research, 29(2), 1-23. zhang, b., susarla, a., & krishnan, r. (2014). a bayesian model to predict content creation with two-sided peer influence in content platforms. retrieved from http://dx.doi.org/10.2139/ssrn.2554689. zhou, t. (2011). understanding online community user participation: a social influence perspective. internet research, 21(1), 67-81. copyright: © 2021 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v25i0.2233 http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 theoretical foundations of social influence 2.1 key theoretical constructs of social influence 2.2 constructs of social influence used in is research 3 methodology and analysis 3.1 overview of the literature sample 3.2 analysis 3.2.1 most used constructs 3.2.2 overlaps between constructs 3.2.3 the evolution of constructs 3.2.4 context of the study 4 focus on boundary conditions 4.1 the choice of the social circle 4.2 space and proximity 4.3 the locus of social influence 4.4 the number of people and number of times 4.5 the diversity and the variety of social influence 5 conclusion paper title (paper title style) australasian journal of information systems park et al. 2023, vol 27, research article impact of role overload and role conflict 1 the impact of role overload and role conflict on physicians’ technology adoption eun hee park old dominion university, va, usa epark@odu.edu ghiyoung p. im university of louisville, ky, usa jing zhang old dominion university, va, usa young hwan lee yeungnam university, republic of korea kyung hee chun konyang university, republic of korea young soon park catholic kwandong university, republic of korea abstract technology adoption is an important solution for physicians to increase work efficiency, and thus deal with role conflict among their multiple job roles. prior studies have not investigated how multiple job roles and role conflict influence physicians’ technology adoption intentions. based on role strain theory and role identity theory, we present a model of physicians’ technology adoption intentions to support their primary (clinical care) versus secondary (teaching or research) job roles. we test the model using surveys with 156 physicians at nine medical schools in korea. the results of our data analysis largely support our hypotheses. role overload in each of their job roles increases role conflict between any pair of associated roles. furthermore, role conflict between a physician’s primary and secondary role is affected more by role overload in the secondary role than by overload in the primary role. moreover, the impact of role conflict on technology adoption intentions is also influenced by the hierarchical relationship between two roles. this study contributes to technology adoption research by demonstrating how physicians’ job characteristics affect technology adoption. keywords: physicians’ technology adoption, multiple roles, role overload, role conflict. 1 introduction physicians need to undertake multiple job roles. in their role as clinicians, they take care of patients; in their role as teachers, they train medical students; and in their role as researchers, they conduct research in their areas of expertise. given their limited time and energy, managing multiple roles requires physicians to maintain extremely busy work schedules. when they are focusing on one role, physicians are less capable of meeting the expectations of their other roles (lenaghan & sengupta, 2007; o’dowd et al., 2018); this can cause role conflict among the associated roles. researchers have recognized physicians’ information technology (it) adoption as an important solution for improving their work efficiency and for reducing their job stress (turan australasian journal of information systems park et al. 2023, vol 27, research article impact of role overload and role conflict 2 & koç, 2022). research on physician it adoption has largely relied on the frameworks of the technology acceptance model (tam) (e.g., turan & koç, 2022), the theory of planned behavior (tpb) (e.g., deng, zheng, lu, zeng, & liu, 2021), and the unified theory of acceptance and use of technology (utaut) (e.g., jayaseelan, prasanth, & pichandy, 2020). while these studies have recognized the importance of economic, social, and psychological factors that influence the it adoption behavior of physicians, they have neglected an important contextual factor: the job characteristics of physicians (venkatesh, 2006). because multiple job roles are one of the key job characteristics of physicians (ma et al., 2020; steinert, o’sullivan, & irby, 2019) and technology adoption may help physicians reduce their job stress from multiple roles (turan & koç, 2022), it is important to ask the following overarching question: what is the impact of physician role characteristics on it adoption? to address this question, we ask three sub-questions. for the inquiry, we rely on role strain theory (e.g., berger & bruch, 2021; goode, 1960; sumra & schillaci, 2015) to identify the role strain of physicians with multiple roles and its impact on physicians’ it adoption intentions. role strain refers to the perceived difficulty of fulfilling role demands and obligations (sumra & schillaci, 2015). role strain theory is a useful lens because it offers two well studied subconstructs—role overload and role conflict (barnett & baruch, 1985; edwards, zarit, stephens, & townsend, 2002)—that help us understand physicians’ multiple roles and challenges. furthermore, role strain theory allows us to integrate physicians’ job context and it, given physicians’ multiple job roles (ma et al., 2020; steinert et al., 2019) and the possibility of it to reduce their job stress from multiple roles (jayaseelan et al., 2020; turan & koç, 2022). consistent with coverman (1989), this study conceptualizes role strain caused by multiple roles as a sequence of first role overload and then role conflict. the association between role overload and role conflict has been investigated mostly in the context of women’s (marks & macdermid, 1996), students’ (lenaghan & sengupta, 2007), and workers’ multiple roles (e.g., eatough, chang, miloslavic, & johnson, 2011). exploring the association in the job context of physicians is important because the findings will offer valuable insights, both into the factors that cause physicians’ role strain and into its mitigation mechanisms. this leads to the first sub-research question: what is the impact of role overload on role conflict in the job context of physicians? in addition, we investigate the relative impact of each role overload on role conflict between the associated roles. role identity theory (gruber & macmillan, 2017; owens, robinson, & smith-lovin, 2010) suggests that multiple roles are associated with multiple identities. those identities form a hierarchy, and one identity becomes prominent for a given situation (gruber & macmillan, 2017; owens et al., 2010). for their primary identity, physicians show more commitment, receive more support from their schools, and are better rewarded. therefore, when physicians face overload in their multiple roles, they would expect that some roles have more prominence over others and thus exert different impacts on role conflict. this leads to the second sub-research question: what is the relative impact of each role overload on role conflict in the job context of physicians? role strain research suggests that people engage in various strategies to deal with role strain caused by their multiple roles (marks & macdermid, 1996). previous studies suggest that it is one of the best means that physicians can use to boost their work performance (davidson & chismar, 2007; goldschmidt, 2005). we consider that these advantages of it motivate physicians to adopt technology when looking for solutions to role conflicts. this leads to the third sub-research question: what is the impact of role conflict on the it adoption intentions of physicians? australasian journal of information systems park et al. 2023, vol 27, research article impact of role overload and role conflict 3 our empirical context is physicians in medical schools (pmss) in korea. we conducted preliminary interviews with pmss and then collected questionnaires from 156 physicians at nine medical schools in korea. the results of our data analysis largely support our hypotheses. first, overload in each role increases the conflict between any pair of associated roles. second, role conflict between a physician’s primary and secondary role is affected more by role overload in the secondary role than by overload in the primary role. finally, the impact of role conflict on technology adoption intentions is influenced by the hierarchy between the two roles. this study advances technology adoption research by investigating the impacts of multiple roles on technology adoption. it reveals interesting dynamics among role overload, role conflict, and technology adoption intentions. our finding that role overload from multiple roles does not contribute equally to role conflicts offers a new perspective in explaining role strain. moreover, this study extends it adoption research by showing that the impact of role conflict on technology adoption intentions is determined by the relationship between the pair of roles in conflict. this study suggests that technology adoption research should consider a new influential factor: the hierarchy of multiple roles and the conflicts between them. 2 literature review 2.1 technology adoption research on physicians research has paid a considerable amount of attention to evaluating the impacts of various job/role characteristics on technology adoption and use (e.g., ansong & boateng, 2018; leidner, sutanto, & goutas, 2022). the investigated job contexts include work-family conflict (ahuja, 2002), role clarity (speier & venkatesh, 2002), job fit (venkatesh, morris, davis, & davis, 2003), job relevance (kim, 2008), job autonomy (seppänen, pajarre, & kuparinen, 2015), and mental load (shamsi, iakovleva, olsen, & bagozzi, 2021). despite such progress, little is known about how such job characteristics (e.g., role, overload, and role conflict) could impact physicians’ technology adoption. most research on technology adoption by physicians has employed economic, social, and psychological theories. three major theories have been used in the research: tam (technology acceptance model), utaut (unified theory of acceptance and use of technology), and tpb (theory of planned behavior) (see appendix a for the literature search approach). first, drawing on tam (e.g., f. d. davis, 1989), research investigates the effects of perceived ease of use, perceived usefulness, and attitude on physicians’ intentions to use health information technologies. for instance, hu et al. (1999) find that perceived usefulness positively influences physicians’ attitudes and intentions to use telemedicine technology. dünnebeil et al. (2012) modify tam to investigate physicians’ adoption of electronic health in an ambulatory care environment. turan and koç (2022) explore physicians’ health information technology (hit) adoption in a developing country. second, utaut (e.g., aria & archer, 2018; venkatesh et al., 2003) has been used to develop richer theoretical explanations. for example, venkatesh et al. (2011) investigate physicians’ adoption behaviors toward electronic medical record (emr) systems. they test both an original and a modified utaut, finding that the modified utaut performs better in explaining physicians’ adoption behaviors. the results indicate that performance expectancy, effort expectancy, social influence, and facilitating conditions positively affect behavioral intentions and use behavior. jayaseelan et al. (2020) investigate medical doctors’ information australasian journal of information systems park et al. 2023, vol 27, research article impact of role overload and role conflict 4 and communication technology adoption in india and find that effort expectancy and social influence positively drive behavioral intentions. finally, tpb (e.g., ajzen, 1991) provides an additional explanation for technology adoption by physicians. for instance, chau and hu (2001) investigate the acceptance of telemedicine technology. after comparing results across tam, tpb, and a decomposed tpb, they find that perceived behavioral control from tpb and perceived usefulness from tam positively influence behavioral intentions. deng et al. (2021) investigate physicians’ adoption of contrast enhanced ultrasound technology for liver cancer by integrating tpb and tam. they find that subjective norms, perceived behavioral control, and perceived usefulness positively impact their intentions and utilization behaviors. despite such progress, prior studies (e.g., chau & hu, 2001; deng et al., 2021) recognize that these major theories have limitations in explaining physicians’ technology adoption behaviors, and they urge future researchers to investigate novel and potential constructs to explain physicians’ behavioral intentions. 2.2 role overload and role conflict role strain research provides a novel perspective to develop a theoretical model of physicians’ technology adoption intentions (e.g., berger & bruch, 2021; goode, 1960; marks, 1977; sumra & schillaci, 2015). role strain takes place when physicians perceive challenges in fulfilling multiple role demands and obligations (sumra & schillaci, 2015). role strain can be understood as a sequence of first role overload and then role conflict (coverman, 1989). role overload occurs when an individual performs multiple roles without having adequate resources to fulfill them (marks, 1977). in this situation, a person’s roles demand more resources and skills than the job holder can handle (mittal & bhakar, 2018). research has largely conceptualized role overload in two ways. some studies (e.g., barnett & baruch, 1985; edwards et al., 2002; mittal & bhakar, 2018) focus on the perception of the collective, overall overload due to multiple roles. for example, mittal and bhakar (2018) investigate women’s multiple roles as employee, mother, and wife and their distress in taking on the combined three roles. other studies (e.g., marshall & barnett, 1993) have been concerned with the perception of role overload in terms of each individual role a person plays, among the multiple roles involved. this approach aligns with sieber’s (1974) suggestion that the different types of roles involved should be delineated and examined. role conflict is experienced when individuals face incompatible expectations from their multiple roles (ma et al., 2020; marks, 1977). research conceptualizes role conflict in two different ways. some studies (e.g., barnett & baruch, 1985; ma et al., 2020) are concerned with the perception of overall role conflict without distinguishing among individual roles. for example, in a study of physicians’ multiple roles, ma et al. (2020) examine the perception of the overall role conflict that occurred among multiple roles (clinical doctor, teacher, and researcher). other studies (e.g., bolino & turnley, 2005; coverman, 1989) have focused on the perception of role conflict that occurs in a pair of roles. this approach allows researchers to identify the unique impact of each pair of roles on the target outcome (coverman, 1989). for example, in a study of women’s and men’s multiple roles, coverman (1989) evaluates the impact of a pair of conflicts – work and family role conflict – on job satisfaction and marital satisfaction. australasian journal of information systems park et al. 2023, vol 27, research article impact of role overload and role conflict 5 while investigating this phenomenon, research has employed either the scarcity or the expansion approach (marks, 1977). adopting the majority scarcity approach, our study assumes that individuals inherently possess scarce and finite quantities of natural energy, and that they struggle to properly allocate such limited resources across their different roles, which can lead to psycho-physical problems (coser, 1974). individuals’ physio-psychological wellbeing can deteriorate with their increased number of roles, leading to role overload and role conflict (baruch & barnett, 1986). in contrast, the expansion approach assumes that individuals’ energy is profuse and expandable because it is generated from socio-cultural variables rather than from biological nature (baruch & barnett, 1986; marks, 1977; sumra & schillaci, 2015). table 1 displays representative studies on role overload and role conflict in workplaces (see appendix a for the literature search approach). two research gaps are identified. first, the research does not investigate the impacts of role overload and role conflict on technology adoption. next, research on the influence of physicians’ multiple roles on technology adoption in workplaces is rarely conducted. instead, some studies have investigated the predictors (e.g., power distance, satisfaction, mobile technology use, and technostress creator) of role overload and role conflict in the context of multiple roles among physicians and nurses’(belias, koustelios, sdrolias, & aspridis, 2015; pullins, tarafdar, & pham, 2020; román, rodríguez, & jaramillo, 2018). other studies have explored such outcomes as role overload and/or role conflict, including strain (ayyagari, grover, & purvis, 2011), citizen behavior (eatough et al., 2011), burnout (dasgupta, 2012; jiang, ma, jia, wu, & wu, 2022; kilroy, flood, bosak, & chênevert, 2016), occupational stress (karimi, omar, alipour, & karimi, 2014), innovative work behavior (clarke & higgs, 2020), job satisfaction (rorong, 2021), and organizational culture (bhalla, qazi, & miralam, 2019). source context predictor(s) outcome(s) major finding(s) engle & prince (2005) workers in us and canada • role strain (including role conflict) • job performance/ satisfaction • job strain has a more negative impact on job performance and satisfaction in us than canada. tarafdar et al. (2007) workers who use technology • technostress • role stress • role overload • role conflict • indirect influence of technostress on role overload and role conflict through its effect on role stress. schaufeli et al. (2009) junior doctors • workaholism • job demands • role conflict • well-being • burnout • role conflict mediating between workaholism/job demands and well-being/burnout. ayyagari et al. (2011) workers who use technology • presenteeism • work-home conflict • work overload • role ambiguity • strain • positive relationship between presenteeism and work-home conflict, work overload, and/or role ambiguity as well as between these factors and strain. eatough et al. (2011) employees in organizations • role overload • role conflict • job satisfaction • organizational citizenship behavior • negative relationship between role overload, role conflict and/or job satisfaction as well as between role conflict and citizenship behavior. dasgupta (2012) private hospital nurses • role overload • role conflict • self-efficacy • burnout • positive relationship between role overload and/or role conflict and burnout australasian journal of information systems park et al. 2023, vol 27, research article impact of role overload and role conflict 6 table 1. research on role overload and role conflict in workplaces source context predictor(s) outcome(s) major finding(s) (disengagement and exhaustion). karimi et al. (2014) nurses in iranian hospitals • role overload • role conflict • occupational stress • positive relationships between role overload and/or role conflict and occupational stress. belias et al. (2015) employees in greek banking organization • job satisfaction • autonomy • role conflict • negative relationship between role conflict and job satisfaction, and the moderating role of autonomy kilroy et al. (2016) employees in hospital • highinvolvement work practices (hiwp) • job demands • burnout • negative relationship between hiwp and job demands and/or burnout • role conflict and/or role overload mediating the relationship between hiwp and burnout rubel et al. (2017) employees in organizations • role conflict, overload, etc. • work-family conflict (wfc) • turnover intention • wfc mediating the relationship between role conflict and/or role overload and turnover intention román et al. (2018) sales professionals • mobile technology use • role stress • job satisfaction • role stress (role conflict, overload, etc.) mediating the relationship between mobile tech use and job satisfaction bhalla et al. (2019) employees in service sector • organizational role stress • organizational culture • negative relationship between organizational role stress and its culture venkatesh et al. (2019) parents on children’s internet addiction • family-work conflict • internet addiction, etc. • job outcomes (e.g., job satisfaction) • positive relationship between internet addiction and conflict, as well as negative relationship between work overload and/or stress and job satisfaction clarke & higgs (2020) employees in public sector • role overload • innovative work behavior • positive relationship between role overload and innovative work behavior pullins et al. (2020) sales professionals • technostress creator, inhibitors, etc. • role stress (role conflict, overload), etc. • positive relationship between technostress creators and role stress khalid et al. (2021) employees in organizations • work-family conflict • after-hours work related technology use • interpersonal/ organizational deviance, etc. • positive relationship between tech use and work-family conflict as well as between conflict and interpersonal/ organizational deviance rorong (2021) employees working from home • supervisor support, etc. • role stress • job satisfaction • negative relationships between supervisor support and role stress as well as between stress and job satisfaction jiang et al. (2022) mobile tech use • excessive use of mobile technology • technology– work conflict • role overload • job burnout • positive relationship between excess use of mobile technology, conflict, and role overload as well as between conflict/ role overload and job burnout australasian journal of information systems park et al. 2023, vol 27, research article impact of role overload and role conflict 7 2.3 role overload and role conflict and information technologies physicians experiencing role overload and role conflicts suffer when they cannot properly fulfill their roles and if they receive sanctions due to poor work performance. their psychophysical well-being also deteriorates with role overload and role conflict (e.g., baruch & barnett, 1986; karimi et al., 2014; kilroy et al., 2016). what solutions can they seek when they are in this kind of challenging situation? research suggests that, to cope with role conflict, people implement various solutions (hall, 1972; marks & macdermid, 1996). first, they look for outside support to take over some of their role activities. second, they seek support from role senders (someone who establishes role expectations) about the fulfillment of necessary activities (hall, 1972). third, they involve role senders to find and determine a solution to their role conflicts. fourth, they increase the efficiency of their role performance through better planning and organizing. finally, they work harder to fulfill all the demands on their various roles by devoting more time and energy. we suggest that it can be one of the best solutions that physicians can employ to increase their work efficiency and resolve role conflicts (jayaseelan et al., 2020; turan & koç, 2022). information technologies are known to minimize errors and to increase the efficiency and quality of physicians’ practices across multiple role activities, thereby relieving role conflict among those roles (mechanic, 2003). in our research context, physicians use clinical technologies on the front line, where they directly treat patients. for example, physicians use emr to trace the medical history of each patient and to share and transfer records with other physicians. educational technologies are used when physicians teach medical students. one example is simulation role play tools that offer students opportunities to conduct surgeries in a virtual environment, which is safe and less costly. finally, physicians use research technologies to conducti research. for example, molecular biology experiment tools can be used to isolate and manipulate molecular components in organisms. table 2 outlines the major tasks associated with the three roles physicians play and the technologies that support them. these qualitative data were collected via preliminary interviews and open-ended questions in our main survey. the exemplar technologies consist of those currently in use and those in consideration for adoption in the near future. multiple roles major task function of technology examples clinician • physical exam • history taking • laboratory tests • diagnosis • teaching patients • operation • medication orders • physician instructions c lin ic al te ch no lo gy data management electronic medical records (emr), cloud server, order communication systems (ocs), picture archiving and communication systems (pacs), and computerized physician order entry (cpoe) diagnosis support polymerase chain reaction (pcr), real-time quantitative pcr (rqpcr), ultrasonography, and telemedicine systems operation robot operation systems, teleoperation systems supplementary presentation & training video, live surgery, new treatment australasian journal of information systems park et al. 2023, vol 27, research article impact of role overload and role conflict 8 teacher • teaching (e.g., lecture, bedside teaching) • development of teaching skills • management of students (e.g., mentoring, coaching) • program evaluation • preparation for accreditation of medical education programs ed uc at io na l t ec hn ol og y learning management learning management systems, learning content management systems, united messaging systems assessment support question bank, computer-based test, web test, voting systems, testing apps education method u-class, e-learning, virtual classrooms, flipped learning tools, zoom knowledge presentation audio, video, digital cameras, video cameras, lcd projector, interactive whiteboard tools, powerpoint, prezi task structuring support microscope, clinical practice tools, simulation role play, simulation technology, webcams researcher • medical lab tasks, (e.g., testing, monitoring, assessing trials) • funding research project • completing irb • data collection & analysis • reviewing literature and writing research paper • participating in conferences, workshops, etc. re se ar ch te ch no lo gy research support e-irb, university research management systems, research management systems, clinical research support systems data management dbms, data warehouse, data mining tools, excel, endnote data collection & analysis medical statistic support systems, statistical analysis programs (e.g., spss) experiment support animal experiment tools, molecular biology experiment tools, genetic research tools, 3d mapping programs, clinical trial support technology table 2. pmss’ multiple role involvement and supporting technologies 2.4 multiple roles and role identities physicians in medical schools have multiple job roles (ma et al., 2020; steinert et al., 2019). o’brodovich et al. (2003) identify a typical profile of a pms: a clinician-teacher-researcher1. in this profile, physicians make a major commitment to clinical care, reducing their commitment to education and/or research (bunton et al., 2012). when one or more of these roles are overloaded and considered simultaneously, the impact on role conflict will not be identical, because the different roles occupy different levels of prominence in the role hierarchy. according to role identity theory, multiple roles lead to multiple identities (burke & stets, 2009; gruber & macmillan, 2017; owens et al., 2010). multiple identities are placed in a hierarchy, and one identity becomes more prominent in a given situation (burke & stets, 2009). this prominence hierarchy reflects “how individuals like to see themselves given their ideals, desires, or what is central or important to them” (burke & stets, 2009, p. 40). one identity’s prominence over other identities is determined by support, commitment, and reward. support can be built by the self (self-support) or received from others. commitment is made by the self, 1 o’brodovich et al. (2003) identify various physician job profiles with different commitments to different roles, including “clinician-specialist” (primary being clinical care and secondary being education and/or research), “clinician-teacher” (primary being clinical care and secondary teaching), and “clinician-scientist” (primary being research), among others. australasian journal of information systems park et al. 2023, vol 27, research article impact of role overload and role conflict 9 who invests in a role identity in order to achieve positive feelings or esteem. rewards (extrinsic and intrinsic) are given to the individual for the maintenance of the identity. for pmss, their clinician identity (i.e., role) is more prominent than their other role identities, such as teacher, researcher, and administrator (steinert et al., 2019). thus, individuals display more commitment to that role, achieve more support from their medical school and staff, and are rewarded more based on their performance as a clinician. such varying levels of role prominence (i.e., primary or secondary) have different impacts on role conflict when an individual is faced with overload from multiple job roles (burke & stets, 2009). 3 hypotheses development figure 1 presents the theoretical model. drawing on role strain theory and role identity theory (e.g., berger & bruch, 2021; goode, 1960; sumra & schillaci, 2015), this study investigates the impact of physician role characteristics on it adoption. role strain theory provides a theoretical basis for the two important constructs (role overload and conflict) and their relationships. role identity theory offers a theoretical basis that multiple roles imply multiple identities and that multiple identities are structured in a hierarchy (burke & stets, 2009). specifically, the study examines 1) the influence of role overload on role conflicts, 2) the relative impact of each role overload on role conflict, and 3) the impact of role conflict on the it adoption intentions of physicians. concerning role overload, we follow the approach of focusing on overload in each individual role, among the multiple roles involved. this approach helps to identify the unique aspects of overload that are involved in each role (sieber, 1974). as for role conflict, we focus on conflict between pairs of job roles (i.e., teacher and clinician, teacher and researcher, and clinician and researcher). these approaches regarding role overload and role conflict provide a more granular understanding of the phenomenon. this study focuses on the three types of roles (i.e., clinician, teacher, and researcher) that are known to be the pmss’ most common job roles (bunton et al., 2012; ma et al., 2020; o’brodovich et al., 2003). figure 1. research model australasian journal of information systems park et al. 2023, vol 27, research article impact of role overload and role conflict 10 3.1 physicians in medical schools in korea we conceptualized our research model and collected empirical data from medical schools in korea. this investigative context serves our research objectives well. the work of pmss in korea is split into three major roles (park et al., 2012). traditionally, clinical care has been a physician’s core role (park et al., 2012), and is better supported and rewarded, when compared with their research and teaching roles. seo et al. (2012) find that pmss in korea are satisfied with the number of patients, consultation hours, and quality of hospital information system support from their schools. research is also crucial because of the schools’ identity as research and teaching institutions and as institutions that provide clinical care (park et al., 2012). physicians are rewarded, partially based on their research productivity. however, the level of support and reward for research is not commensurate with that for clinical care. most physicians exhibit low satisfaction with the number of research projects, research funding, administrative supporting systems, and information system support (seo et al., 2012). in addition, they display lower satisfaction with the lab environments and incentive systems. conversely, teaching is gaining growing importance among medical schools (jeon & kim, 2017). medical schools in korea have recently emphasized the importance of medical education, innovative curricula, and student career development (jeon & kim, 2017). support and rewards for teaching performance have increased, but they are not commensurate with the level offered for clinical care. physicians are not particularly satisfied with the information systems’ support they receive from the schools (seo et al., 2012). 3.2 physicians’ role overload and role conflict we assert that, overall, role overload will lead to role conflict. specifically, we argue that role overload in teaching, clinical care, and research will bring about role conflict between any pair of the three roles. we put forward two reasons. first, physicians have limited resources. specifically, they have limited time to fulfill the obligations of their multiple roles (c. r. ward, 1986). when physicians spend a lot of time fulfilling one role, the resource pool available for other roles will diminish, due to role overload. hence, they will not be able to fulfill the obligations and expectations of the other roles (lenaghan & sengupta, 2007; marks, 1977), causing role conflict. next, physicians are not free from job stress that can affect performance in their different roles. when physicians face role overload, they experience anxiety and tension given high expectations from various social parties (e.g., patients, medical students, the university) involved in their job roles (martikainen, 1995). such psychological challenges may reduce their ability to meet the demands of one or more of their roles. this makes it difficult for an individual to fulfill other roles properly and hence leads to role conflict (lenaghan & sengupta, 2007). specifically, as their teaching becomes overloaded , physicians may have difficulty fulfilling their clinician role properly, given limited resources (marks, 1977). thus, role conflict between teacher and clinician is more likely to arise (coverman, 1989; ma et al., 2020). in a similar vein, as their teaching overload increases, role conflict between teacher and researcher is also likely to occur. thus: h1: role overload in teaching is positively associated with role conflict between a) teaching and clinical care, and b) teaching and research. australasian journal of information systems park et al. 2023, vol 27, research article impact of role overload and role conflict 11 we apply the same reasoning to the effect of role overload in research on role conflict. typically, pmss spend less time on research activities than on clinical care (ellinas et al., 2019; marks & macdermid, 1996; thomas et al., 2004). however, pmss hardly avoid research and associated scholarly activities (e.g., data collection, experiments, and data analysis) as their performance in the activities influences their job performance evaluation and promotion (ellinas et al., 2019). when physicians spend more time fulfilling their researcher role, the role overload in their research activities increases. this in turn means that the time and resources available to meet the expectations of their other roles dwindles (ellinas et al., 2019; c. r. ward, 1986). here, they may experience role conflict between their teacher and researcher roles as they struggle to meet the commitment and expectations of the second role (lenaghan & sengupta, 2007). similarly, role conflict between clinician and researcher may occur as their research overload increases. thus: h2: role overload in research is positively associated with role conflict between a) teacher and researcher, and b) clinician and researcher. we also argue that the same positive association from role overload in clinical care on role conflict occurs. prior studies show that pmss spend the majority of their time as clinicians and less time on teaching or research (marks & macdermid, 1996; steinert et al., 2019; thomas et al., 2004). thus, one may argue that because their clinical job role takes so much of their time, it may not cause role conflict with their teaching or research roles. although teaching and research activities are minor roles in pms role systems (marks & macdermid, 1996; o’brodovich et al., 2003; thomas et al., 2004), both roles are unavoidable and not easy-to-exit since their job performance evaluation criteria includes both their clinician performance and their teaching and research performance. further, performance assessment results are tightly tied to faculty salary increases and tenure evaluation (barzansky & kenagy, 2010; ellinas et al., 2019; svider et al., 2013). a prior study finds that the salience of a role to a person does not reduce stress, but that a person’s conception that a role is easy-to-exit reduces stress (thoits, 1992). when clinician role overload increases, the physicians role system become less balanced (marks & macdermid, 1996). although clinician is a major role and teacher or researcher is a minor role, the minor roles are those that are “difficult-to exit” from the role system (thoits, 1992). an increase in overload in the clinician role means more demands on clinicians’ activities and hence physicians face more difficulty meeting the demands of their other roles (barzansky & kenagy, 2010). thus, role conflict is inevitable (coverman, 1989; yang et al., 2021). thus: h3: role overload in clinical care is positively associated with role conflict between a) teaching and clinical care, and b) research and clinical care. in the next three hypotheses, we discuss the relative impact of role overload between two roles on role conflict. as role identity theory states, the argument is based on the discussion that individuals who have multiple roles have multiple identities in a hierarchy (burke & stets, 2009; owens et al., 2010). we argue that when there is role conflict between the primary role (clinical care) and the secondary role (teaching or research), role conflict will be higher with role overload in the secondary role than in the primary role. first, when role overload in the secondary role increases, physicians find less time for their primary role (ellinas et al., 2019). they will be less committed to clinical care and their primary identity will be challenged. such a situation may make them less competent and less well-rewarded, and they may receive less support from their schools because their core competency is in clinical care (o'brodovich, australasian journal of information systems park et al. 2023, vol 27, research article impact of role overload and role conflict 12 beyene, tallett, macgregor, & rosenblum, 2007). despite an increase in the importance of secondary roles, as demanded by medical schools (e.g., ahmady et al., 2007; o'brodovich et al., 2007), clinical care is central in obtaining career development opportunities, monetary and other long-term stable rewards, and support. second, when role overload in the primary role increases, physicians find less time for their secondary role (ellinas et al., 2019). in this situation, physicians find less role conflict because their primary identity is not jeopardized, and they are less disadvantaged with respect to support and reward from their schools. in contrast, physicians may make use of that role overload to justify actions that strengthen their primary role identity and competency in the workplace. thus: h4: role conflict between teaching and clinical care is more influenced by an overload in teaching than by an overload in clinical care. h5: role conflict between research and clinical care is more influenced by an overload in research than by an overload in clinical care. concerning role conflict between secondary roles (teaching and research), we argue that role overload in the secondary roles will have similar effects on the role conflict between them. when role overload in a secondary role increases, physicians will find less time for the other secondary role, committing less to the other secondary role because of limited resources (c. r. ward, 1986). in this situation, increasing activities in a secondary role will not greatly decrease the competency attributed to the other secondary role. the secondary roles have lower prominence in the role hierarchy (marks & macdermid, 1996; o’brodovich et al., 2003), and physicians receive less support and fewer rewards for those roles, as compared with their primary role (thun et al.,, 2018). thus, physicians will be less affected by a decrease in their activities when there is role overload in a secondary role. thus: h6: role conflict between teaching and research is similarly influenced by an overload in teaching and by an overload in research. we hypothesize that role conflict between a pair of job roles fosters physicians’ intentions to adopt it in each job role. physicians strive to lower role conflict by enhancing the efficiency of the tasks involved in multiple roles (hall, 1972; marks & macdermid, 1996). as shown in table 2, technologies can support the diagnosis and treatment of patients, as well as the teaching and research activities of physicians, and they offer enormous opportunities for physicians to enhance their work performance (e.g., deng et al., 2021; jayaseelan et al., 2020). first, physicians facing role conflict between their teaching and clinical care roles would adopt educational technology. for example, medical instructional technology helps physicians save time and energy by allowing them to easily integrate learning materials, organize online classes, facilitate discussions through online tools, and enhance learning processes (hopkins et al., 2018; mostaghimi et al., 2006). thus, by recognizing the advantages of technology, physicians who experience role conflict between teaching and clinical care look for solutions and likely adopt educational technologies. next, physicians experiencing role conflict between teaching and clinical care would adopt clinical technology. for example, physician order entry systems help reduce time and errors in the medication process and provide effective medication to patients (davidson & chismar, 2007). such clinical it allows physicians to save time by managing all of their clinical jobs automatically (e.g., shah et al., 2020; zobair et al., 2021). for example, telehealth and telemedicine systems allow physicians to remotely treat patients and monitor their progress, australasian journal of information systems park et al. 2023, vol 27, research article impact of role overload and role conflict 13 hence providing better care coordination and treatment outcomes (ahmad et al., 2021; zobair et al., 2021). in sum, the adoption of both types of technologies ease role conflict by helping physicians balance both roles. thus: h7: role conflict between teaching and clinical care is positively associated with intentions to adopt a) educational technology, and b) clinical technology. physicians experiencing role conflict between clinical care and research intend to adopt clinical and research technologies. physicians in role conflict seek to adopt clinical technologies that automate many parts of the clinical practice process, and thus, save time and effort. for instance, physicians make use of electronic prescription technology to manage patient care, to automate prescriptions and medication refills, to receive lab results, and to coordinate with specialists and hospitals outside their primary care facility (bulut et al., 2019; k. davis et al., 2009). the researcher role brings conflict to the clinician role because, given the extent of their clinical care, physicians struggle to find time to conduct research and generate research outcomes (goldschmidt, 2005). such tension leads physicians to adopt research technologies that can facilitate information gathering, record keeping, summarizing, ordering, and the sharing of data among researchers (goldschmidt, 2005; harrer et al., 2019). also, the use of research technology allows physicians to mine huge amounts of medical data for research purposes, to quickly discover the causes and epidemiology of diseases, and to easily monitor and assess the effectiveness of new clinical trials (goldschmidt, 2005). thus, the adoption of clinical and research technologies will alleviate the role conflict between clinical care and research. hence: h8: role conflict between clinical care and research is positively associated with intentions to adopt a) clinical technology, and b) research technology. similarly, physicians experiencing role conflict between teaching and research would intend to adopt educational and research technologies. educational technologies facilitate educational activities (e.g., lecture preparation and delivery and the management of assessment and exams) (hopkins et al., 2018; j. ward et al., 2001). the technologies offer advanced, convenient, and customized tools that support physicians in fulfilling their teaching obligations and expectations. for instance, e-learning technologies such as podcasts, onlineproblem based learning modules, and educational websites offer efficiency in the delivery of teaching content across physical distance and unlimited viewing (hopkins et al., 2018). research technologies promote research activities (e.g., searching and collecting data, monitoring trials, assessing tests, and sharing data with other researchers) (goldschmidt, 2005; harrer et al., 2019). for example, artificial intelligence (ai) based clinical trial support technology allows physicians to save time and effort by supporting optimized cohort composition, patient recruitment through automatic eligibility assessment and trial recommendation, and patient monitoring through record keeping (harrer et al., 2019). physicians experiencing role conflict between the roles seek to obtain the above-mentioned benefits and alleviate the role conflict. thus: h9: role conflict between teaching and research is positively associated with intentions to adopt a) educational technology, and b) research technology. australasian journal of information systems park et al. 2023, vol 27, research article impact of role overload and role conflict 14 4 research methodology 4.1 sample and data collection this study is the result of engaged scholarship, pursued over several years, that sought to understand the job context of pmss and the drivers in their adoption of it. the data collection involved two phases: (1) preliminary interviews and pilot surveys, and (2) the main survey. the preliminary interviews were conducted with five physicians who had ten to thirty years’ experience (with an average of seventeen years), who work in different specialties in two medical schools in korea (see appendix b for the preliminary interview protocol). the interviews offered insightful information about the multiple job roles of the physicians and their intentions to adopt it solutions. a preliminary list of clinical, educational, and research technologies was developed (table 2). pilot surveys were performed with three pmss to refine the survey instrument with respect to accuracy, syntax, and relevance. role identity questions (appendix d) were added to filter out responses from physicians whose prominent role was not clinical care. since the measures were adapted from existing research, we translated them into korean, and then had a professional translator translated them back into english. the back-translation process helped us pay particular attention to sensitive translation problems across cultures (beaton et al., 2000). second, we conducted the main surveys using the finalized instrument. the data collection was conducted in two phases. in the first phase, the survey was mailed to the departments of medical education at nine korean medical schools for distribution to the physicians at each school. we sent a total of 180 surveys and received 134 responses. we retained 128 of the responses for data analysis after removing those that did not indicate clinical care as the respondent’s primary role. this was necessary because clinical care role is considered to be more prominent than other roles in pmss (steinert et al., 2019). we performed the second phase of data collection after the covid pandemic. owing to covid 19 restrictions, fewer sites were available to participate. a total of 30 surveys was sent to three medical schools, resulting in 28 responses. taken together, we obtained a total of 156 responses from nine medical schools (total effective response rate = 74%). our data shows that, on average, clinical care, teaching, and research occupy 48%, 29%, and 23%, of pmss’ time. the majority of the participants were male (76%), and their average tenure was 22 years, ranging between 1 and 39 years. the sample demographics are presented in table 3. the survey instrument included open-ended questions that asked physicians to specify the information technologies that they use to accomplish their multiple roles and job functions (see appendix c). the answers were used to further refine and validate the list of clinical, educational, and research technologies created during the preliminary interviews (table 2). finally, we conducted in-depth cognitive interviews (beatty & willis, 2007) with five interviewees to address common method bias and to better understand the factors that hinder adoption intentions (appendix e). the five interviewees were physicians from two medical schools who had 15 to 30 years’ tenure in the schools and had not participated in the earlier data collection. australasian journal of information systems park et al. 2023, vol 27, research article impact of role overload and role conflict 15 demographic categories percent (n=156) gender male female 76.28 23.72 age, years 20-29 30-39 40-49 50-59 60-69 1.92 19.23 44.23 31.41 3.21 tenure, years 1-9 10-19 20-29 30-39 7.69 27.56 42.95 21.79 table 3. sample demographics 4.2 measurement and controls all the measures were based on existing measures and were modified, as necessary, to fit our physician context. below, we supply details regarding the way in which some of our key constructs were measured, in order to illustrate important aspects of our measurement strategy. the full details of the instrument are outlined in appendix d. we measured intentions to adopt technology by asking “intention to adopt” or “would like to adopt” each of the three types of technology –clinical, educational, and research (karahanna et al., 1999; teo et al., 2003). at the beginning of the questionnaire, we defined the three technologies and provided an example list of each (appendix c) so that the respondents would not confuse the classification of the technologies. to measure role overload, we asked four questions for each job role (marshall & barnett, 1993). we adopted the measurement approach of assessing the perception of role overload for each individual role (sieber, 1974). this approach helps us distinguish the different types of roles involved. role conflict was measured with three questions for each pair of job roles – teaching and clinical care, clinical care and research, and teaching and research (bolino & turnley, 2005; coverman, 1989). similar to the role overload measure, this granular approach allowed us to recognize the unique impact of each role conflict, in a pair of roles, on the target outcome (coverman, 1989). the following controls were measured: normative beliefs about adopting it, perceived usefulness (or relative advantage), age, gender, tenure, voluntariness, and hours spent in roles. prior studies have found that normative beliefs from top management, peers, and the department positively influence individuals’ behavioral intentions to adopt and use technology (karahanna et al., 1999). also, prior studies have suggested that physicians who perceive the usefulness of technology (chau & hu, 2001; hu et al., 1999) are more likely to have an intention to use it. we also controlled for age, gender, tenure (a surrogate for experience), and voluntariness that are identified as significant factors in technology adoption (venkatesh et al., 2003). hours spent in roles are also included because physicians in medical schools were more disposed to technology acceptance depending on the time spent in each of their roles (ozdemir & trott, 2009). australasian journal of information systems park et al. 2023, vol 27, research article impact of role overload and role conflict 16 5 data analysis and results in order to test our hypotheses, we performed multivariate analyses using stata for measure validation and ordinary least squares (ols) regression. before testing the hypothesized model, the psychometric properties of the measures were evaluated for reliability and for convergent and discriminant validity. 5.1 psychometric properties of measures first, cronbach’s alpha was assessed for each of the measures, and the minimum of the scale reliability coefficients was 0.90, exceeding the suggested minimum value of 0.70 (hair, black, babin, anderson, & tatham, 2010). also, the composite reliability was calculated using the standardized factor loadings, and the respected error variance for each measure. the reliability coefficients exceed the minimum threshold of 0.70 (table 4). variable 1 2 3 4 5 6 7 8 9 10 11 1. intentions to adopt t 2. intentions to adopt c 0.46 3. intentions to adopt r 0.51 0.44 4. role conflict tc 0.33 0.11 0.16 5. role conflict cr 0.28 0.36 0.36 0.40 6. role conflict tr 0.26 0.05 0.21 0.60 0.51 7. overload t 0.22 -0.02 0.08 0.55 0.19 0.46 8. overload c 0.10 0.11 0.19 0.44 0.36 0.32 0.38 9. overload r 0.07 0.09 0.22 0.25 0.48 0.41 0.31 0.40 10. gender 0.07 0.09 0.09 0.07 0.12 0.16 0.01 -0.01 -0.03 11. tenure 0.06 0.22 0.12 0.01 0.23 0.04 0.02 0.04 0.14 0.14 mean 5.48 5.73 5.43 4.34 4.58 3.93 3.84 4.81 4.08 1.24 21.77 s. d. 1.04 0.97 1.10 1.18 1.29 1.30 1.22 1.36 1.30 0.43 8.42 ave 0.70 0.71 0.64 0.66 0.73 0.74 0.76 0.81 0.76 comp. reliability 0.82 0.83 0.78 0.85 0.89 0.89 0.93 0.94 0.93 note. 1. n=156. all correlations greater than or equal to 0.19 are significant at the 0.05 level. 2. t: teaching/educational technology; c: clinical care/clinical technology; r: research/research technology. tc: teacher and clinician; tr: teacher and researcher; cr: clinician and researcher. table 4. descriptive statistics and correlations second, convergent and discriminant validity was assessed by examining the average variance extracted for each measure and factor loadings. the average variance extracted for each measure was calculated according to the procedure (fornell & larcker, 1981). the calculated statistics all surpass the minimum threshold of 0.50 (table 4). we compared the average variance extracted for each measure with the shared variance between all possible pairs of measures. the average variance extracted for each measure was higher than the squared correlation between the measure pairs. next, we ran a factor analysis with varimax rotation. the initial factor analysis showed that the control variable “perceived usefulness” was loaded into the same construct with compatibility. thus, we excluded compatibility from further analysis. the rotated pattern matrix showed that all items had high loadings on their respective constructs and that no item loaded higher on constructs that it was not aimed to measure. australasian journal of information systems park et al. 2023, vol 27, research article impact of role overload and role conflict 17 overall, the developed measurement instruments exhibited sufficiently strong psychometric properties to support valid testing of the proposed research model. table 4 presents the descriptive statistics and the correlations among the constructs for the 156 responses. 5.2 common method bias analysis we tested for common method bias using the marker variable approach developed by lindell and whitney (2001) and compared in malhotra et al. (2006). a marker variable is theoretically unrelated to at least one variable in the model and is used to adjust the correlations between the substantive variables and the dependent variable for common method variance (lindell & whitney, 2001). this approach can be administered without the identification of the marker variable, a priori. a good candidate for the marker variable is the one with the smallest correlation with the manifest variables. self-perceived energy in the workplace was selected as the marker variable, and the average correlation among the variables was 0.057 (average pvalue = 0.51). adjusting for the marker variable (malhotra et al., 2006), all correlations among the substantive variables showed no change when rounded up to two decimal places (average correlation change = 0.006). these findings indicate that common method bias was not a serious concern with our data. next, we conducted cognitive interviews (beatty & willis, 2007) to address common method bias potentially caused by item context effects (i.e., response in relation to other items in an instrument) and by measurement context effects (i.e., the same context when the measures were collected) (podsakoff et al., 2003) (findings presented in appendix e). 5.3 hypothesis testing hypothesis testing was conducted using stata. first, we examined the assumptions of normality and linearity. as for the normality of residuals, researchers suggest that normality is not necessary for obtaining unbiased estimates of the regression coefficients (cohen, west, & aiken, 2014). cohen et al. (2014) emphasize the importance of graphical evaluation to identify the degree of non-normality. after regression analyses, residuals were calculated using the predict command in stata. then, we obtained a standardized normal probability plot using the pnorm command. from the results, we did not find any indication of the violation of normality. we found slight deviations from normality at the lower or upper tails. however, those deviations seem to be minor. next, we evaluated the assumption of linearity. we plotted the residuals against the predictor variables in the regression equations. the plots did not show any strong indication of departure from linearity. we also generated augmented partial residual plots. we found few problematic areas partially attributed to influential observations. overall, they looked minor and did not indicate any violation of linearity. next, we conducted hypothesis testing. in the regression equation, age and tenure showed moderate multicollinearity with a variance inflation factor close to 5. thus, we dropped age from the equation. table 5 summarizes the results of the regression analyses for the hypotheses. australasian journal of information systems park et al. 2023, vol 27, research article impact of role overload and role conflict 18 hypotheses h1a, h3a, h4 h2b, h3b, h5 h1b, h2a, h6 h7a, h9a h7b, h8a h8b, h9b dv role conflict tc role conflict cr role conflict tr intentions to adopt t intentions to adopt c intentions to adopt r controls gender 0.195 (0.182) 0.316+ (0.207) 0.466* (0.205) 0.154 (0.153) -0.016 (0.145) 0.102 (0.162) tenure -0.002 (0.009) 0.023* (0.011) -0.004 (0.011) 0.008 (0.008) 0.010+ (0.008) 0.003 (0.008) hours spent t 0.016 (0.051) -0.017 (0.056) 0.065+ (0.040) hours spent c -0.001 (0.005) -0.002 (0.005) 0.001 (0.004) hours spent r -0.029* (0.017) -0.044** (0.016) 0.023* (0.012) norm. belief t 0.389** (0.062) norm. belief c 0.231** (0.055) norm. belief r 0.211** (0.074) per. usefulness t 0.270** (0.067) per. usefulness c 0.418** (0.068) per. usefulness r 0.467** (0.067) voluntariness t -0.066 (0.076) voluntariness c 0.015 (0.088) voluntariness r 0.149+ (0.103) main effects overload t 0.427** (0.071) 0.393** (0.077) overload c 0.245** (0.063) 0.196** (0.071) overload r 0.415** (0.077) 0.361** (0.074) role conflict tc 0.172** (0.067) -0.067 (0.057) role conflict tr -0.045 (0.062) -0.055 (0.063) role conflict cr 0.156** (0.054) 0.125* (0.067) n 156 156 156 156 156 156 adj. r2 0.351 0.293 0.323 0.426 0.403 0.413 notes. 1. unstandardized coefficients are shown with standard errors in parentheses. 2. +p < 0.10, *p < 0.05, **p <0.01 for one-tailed tests given directional hypotheses. 3. t: teaching/educational technology; c: clinical care/clinical technology; r: research/research technology. tc: teacher and clinician; tr: teacher and researcher; cr: clinician and researcher. table 5. results of hypotheses tests australasian journal of information systems park et al. 2023, vol 27, research article impact of role overload and role conflict 19 hypotheses 1 to 3 suggested that role overload would lead to role conflicts between teacher and clinician, teacher and researcher, and clinician and researcher. we tested the hypotheses after adding three control variables (gender, tenure, and hours spent). the results in table 5 indicate that all of the role overload coefficients are significant at the p < 0.05 level. hypotheses 4 to 6 suggested that role conflict will be higher when role overload affects the secondary role more than the primary role. we tested the hypotheses after adding three control variables (gender, tenure, and hours spent). for the testing of these hypotheses, we compared the regression coefficients for role overload in the two roles. the results showed marginal support of hypothesis 4 for the difference between teaching and clinical care (f = 2.57, p = .056), support of hypothesis 5 for the difference between clinical care and research (f = 3.11, p = .04), and support of hypothesis 6 for the difference between teaching and research (f = 0.07, p = .40). next, hypotheses 7 to 9 posited that intentions to adopt educational, clinical, or research technology were determined by role conflicts. these hypotheses were tested after including six control variables. the results indicated support of hypothesis 7a (role conflict between clinical care and teaching → intentions to adopt educational technology), hypothesis 8a (role conflict between clinical care and research → intentions to adopt clinical technology), and hypothesis 8b (role conflict between clinical care and research  intentions to adopt research technology) at the p < .05 level. in contrast, hypotheses 7b, 9a, and 9b were not supported. 5.4 mediation tests in addition, we conducted mediation tests using stata’s sem command for significant relationships to evaluate whether mediators (role conflicts) carried the influence of independent variables to dependent variables (mackinnon et al., 2002). we found that role conflict between teaching and clinical care fully mediated the effect of role overload in teaching and clinical care on physicians’ intentions to adopt educational technology (coef.=.078, p =.012; coef.=.044, p=.025). role conflict between clinical care and research fully mediated the effect of role overload in clinical care and research on physicians’ intentions to adopt clinical technology (coef.=.035, p=.034; coef.=.075, p=.005). finally, role conflict between clinical care and research fully mediated the effect of role overload in clinical care and research on intentions to adopt research technology (coef.=.042, p=.046; coef.=.089, p=.012). 6 discussion this study investigates it adoption intentions of physicians, given their multiple job roles and role conflicts in their workplaces. overall, the results largely support the hypotheses concerning the relationships among role overload, role conflict, and it adoption intentions. we found that physicians’ role overload in clinical care, teaching, and research significantly impacts role conflict between any pair of associated roles. we also found that role conflict between the primary (clinical care) and secondary (teaching or research) roles is influenced more by an overload in the secondary role than by overload in the primary role. role conflict between secondary roles is impacted similarly by each of the secondary roles. further, role conflict between clinical care and teaching was found to significantly impact physicians’ intentions to adopt technology that supports their teaching (secondary role), but role conflict between teaching and research does not influence intentions to adopt technology that supports either secondary role. also, we note that role conflict between clinical care and research affects intentions to adopt technology that can support both roles. australasian journal of information systems park et al. 2023, vol 27, research article impact of role overload and role conflict 20 6.1 discussion of the findings from the testing of hypotheses 1 to 3, our findings show positive relationships between physicians’ role overload and role conflicts. these findings are consistent with prior studies (lenaghan & sengupta, 2007; marks, 1977). for example, in the context of college students who have multiple roles as both workers and students, lenaghan and sengupta (2007) found that role overload is more likely to increase role conflict, due to the interference of work on school. rubel et al. (2017) report a positive relationship between employees’ role overload and work-family conflict. however, our study goes beyond existing research by offering a more granular level of understanding of role strain. first, prior studies have typically examined the impact of overall role overload caused by multiple roles (barnett & baruch, 1985; edwards et al., 2002; lenaghan & sengupta, 2007). in contrast, we evaluated the impact of role overload in each individual role (i.e., teacher, clinician, or researcher). this approach allows us to identify the unique impact of each role overload when multiple roles are involved. second, prior multiple job role studies have examined a simpler concept of role conflict, such as conflict between work and family roles (e.g., coverman, 1989) or between work and school roles (e.g., lenaghan & sengupta, 2007). in contrast, our study identifies the three job roles of physicians who work in medical schools and conceptualizes three types of role conflicts. such finegrained conceptualization allows us to investigate which types of role conflict could lead to technology adoption. following our work, hospital managers will be better able to diagnose the factors that hinder technology adoption and will be able to determine ways to both promote technology adoption and enhance physician performance. from the testing of hypotheses 4 to 6, we found a stronger effect from the secondary role on the role conflict than the primary role. the literature shows some attention to the relationships among multiple roles. research on role identity (burke & stets, 2009; owens et al., 2010) suggests that individuals with multiple roles frequently have competing role identities. research on multiple roles and their outcomes shows that college students who bear the multiple roles of worker and student experience more role conflict due to the interference of work on school (lenaghan & sengupta, 2007). our study goes beyond existing research by identifying the differential impacts of primary and secondary roles on role conflict. prior studies on multiple roles (lenaghan & sengupta, 2007; marks, 1977) suggest that the pressure from limited resources makes people feel that it is difficult to accomplish multiple roles appropriately, and this can lead to role conflict. building on these studies, our study showed that each of the roles in the role hierarchy exerts a different influence on role conflict. the primary role, toward which more commitment is placed, forms the core identity of individuals, and support and reward systems are designed to support it (burke & stets, 2009). thus, individuals perceive less conflict when they are overloaded in their primary role (burke & stets, 2009). from the testing of hypotheses 7 to 9, we found that role conflict is an important determinant of physicians’ technology adoption intentions. first, role conflict between the primary role (clinical care) and the secondary role (teaching or research) significantly impacts physicians’ intentions to adopt technology that supports their secondary role (h7a and h8b). in the context of pmss in korea, the clinical care role is the more established role. thus, the results indicate that physicians are more interested in routinizing less-established roles by introducing it, so that their primary role is not disrupted. in addition, when considering their clinical care, physicians often think that they do not need additional technologies for clinical care. next, our results showed that conflict between clinical care and research fosters australasian journal of information systems park et al. 2023, vol 27, research article impact of role overload and role conflict 21 intentions to adopt clinical technologies that support the primary role (h8a). we suggest that this result can be attributed to the close relationship between clinical care and research. that is, clinical care generates data that physicians can use in their research. their adoption of clinical technologies will strengthen their future research. we tested the mediation effect of the significant relationships found in the model. we found that in all six possible mediation pathways, the effect of role overload on intentions to adopt technology was fully mediated by role conflict between the roles. this finding confirms the value of our conceptualization of role strain into the two sub-components of role overload and role conflict (coverman, 1989; lenaghan & sengupta, 2007; rubel et al., 2017) and their significant influence on intentions to adopt technology. however, our findings also show three non-significant relationships. first, role conflict between teacher and clinician is not significant in fostering intentions to adopt clinical technologies (h7b). during the cognitive interviews, pmss offered two factors as to why pmss are not willing to adopt clinical technologies: (1) perceived self-sufficiency in expertise and (2) institutional barriers. the former is a belief that they have sufficient expertise in clinical care and thus do not need support from technology. the latter is the bureaucracy associated with technology adoption, requiring a complicated decision-making process which can involve many different units. bretschneider and wittmer (1993) similarly report that complex internal organizational decision-making procedures (i.e., red tape, complicated procurement rules) hinder and slow new technology adoption in organizations. next, role conflict between teacher and researcher was not significant in promoting intentions to adopt either educational or research technology (h9a, h9b). this finding runs counter to our argument and indicates that conflict between supporting roles does not engender, in physicians, an urgency to adopt technology. the physicians in the cognitive interviews mentioned (1) extra burden and (2) incompatibility of technology with practice. that is, some physicians do not see the necessity in learning new educational or research technologies, since they believe that they already have enough technology that they have mastered with sufficient skills and knowledge to fulfill either job role. hence, it is an extra burden to learn new technologies. further, some perceived that some technologies are not compatible with medical practices. prior studies report similar findings. for example, poon et al. (2004) suggest that a perceived negative influence on physicians’ workflow leads to resistance to technology adoption. chau and hu (2001) find that, compared to regular technology users, physicians tend to be very pragmatic and consider the compatibility of technology with practice a critical factor in influencing their technology acceptance. considering the interview results, we conclude that, despite the enormous potential to resolve role conflict, some physicians are more swayed by negative aspects associated with adopting clinical, educational, or research technologies. 6.2 implications for theory this study contributes to physicians’ it adoption research by examining a new influential factor: their multiple job roles. prior research has heavily relied on tam (dünnebeil et al., 2012; hu et al., 1999; turan & koç, 2022), tpb (chau & hu, 2001; deng et al., 2021), and utaut (jayaseelan et al., 2020; venkatesh et al., 2011) to explain the it adoption behaviors of physicians. these studies have focused on the impacts of technological factors (dünnebeil et al., 2012; turan & koç, 2022) and organizational and social factors (deng et al., 2021; jayaseelan et al., 2020; venkatesh et al., 2011) on physicians’ technology adoption. also, prior studies of physicians have focused on the impact of physicians’ cognitive and emotional factors, such as australasian journal of information systems park et al. 2023, vol 27, research article impact of role overload and role conflict 22 the perceived usefulness of technology (hu et al., 1999), ease of use (dünnebeil et al., 2012), performance expectancy, effort expectancy, social influence, and facilitating conditions (venkatesh et al., 2011). however, the job contexts of physicians have been largely neglected. our study echoes a call by chau and hu (2001) who argue that studying it adoption by physicians requires a different theoretical framework because physicians are different in “general competence, adaptability to new technologies, mental and cognitive capacity, and work arrangement and nature” from comparison groups (e.g., it professionals, technology users in a company) (chau & hu, 2001, p. 712). in general, our study suggests that it adoption research should study the effects of the prominent characteristics of the jobs of interest. our study, to our knowledge, is the first attempt to investigate the impact of role conflict on physicians’ technology adoption. also, our study extends role strain research in two ways. first, there is a lack of research regarding the impact of role strain on technology adoption. prior research on role strain investigated the predictors of role overload and conflict and their impacts on strain and job outcomes. for example, research examined the impacts of technostress and role stress on role overload and conflict in the context of workers’ technology use (tarafdar et al., 2007), the impacts of work-home conflict and work overload on strain of technology workers (ayyagari et al., 2011), the roles of child’s internet addiction and family-work conflict on parents’ job satisfaction (venkatesh et al., 2019), and the impact of excessive use of mobile technology, technology work conflict, and role overload on job burnout (jiang et al., 2022). here, our study contributes to role strain research by showing that physicians’ role overload and role conflict influence their intentions to technology adoption. second, our study provides insights to role strain theory by offering important insights regarding the relationship between role overload and role conflict. hall (1972) found that people should prioritize roles in their role systems such that role overload in a major role would reduce role conflict with minor roles. in our context, physicians spend more time in their role as clinicians and are less involved as teachers and researchers. however, their minor roles as teachers and researchers cannot be less emphasized, because they directly influence salary increases, as well as tenure and promotion. thus, contrary to hall’s (1972) findings, role overload in major and minor roles does not contribute to a decrease in role conflicts. finally, our study extends role identity research by identifying the three distinct role identities of pmss and by finding that their impacts on role conflict and technology adoption are dependent on their position in the role hierarchy. role identity theory suggests that multiple roles creates multiple identities in individuals (burke & stets, 2009; gruber & macmillan, 2017; owens et al., 2010). multiple identities are developed into a hierarchical structure, in which one identity becomes more dominant than the others (burke & stets, 2009). prior studies identified various role identities of physicians. however, these studies fall short of investigating the hierarchy among the roles and its implications. for example, reay et al. (2017) identified the four different identities of physicians (expert, decision maker, authority holder, and government representative). they found that physicians’ role identities are changed via their social interactions in institutional contexts. touati et al. (2019) reported the significant impacts of three physicians’ role identities (medical expert, care coordinator, and team member) on medical collaboration. unlike these studies, our study identifies the primary role identity (clinician) and the secondary role identities (teacher and researcher) and shows the relative impact of roles in the hierarchy on role conflict and technology adoption. australasian journal of information systems park et al. 2023, vol 27, research article impact of role overload and role conflict 23 6.3 implications for practice our study has two important implications for practitioners. first, understanding the barriers and facilitators of it adoption has been an important concern to medical professionals for years (e.g., palacholla et al., 2019). our findings offer crucial insights to administrators at medical schools on how better promote it adoption. the findings show that technology adoption needs to be promoted selectively to different groups of physicians, depending on their job roles and role conflicts. for instance, the role conflict between clinical care and research promotes the adoption of both clinical and research technologies. this implies that administrations should encourage the adoption of both types of technology by those who are active in both clinical and research roles. second, research recognizes that people have multiple roles when they are members of social groups (burke & stets, 2009). our findings indicate that multiple roles operate in a hierarchy, and that physicians place more emphasis on fulfilling their primary role of clinician. their secondary roles (teaching and research) are gaining in importance over time, since physicians’ performance (measured by salary and tenure) is tied to these roles (svider et al., 2013). however, these roles still have lower prominence in the role hierarchy, and physicians do not experience a significant increase in their core competency by excelling in these areas. therefore, even as they emphasize the primary role, medical schools should not neglect support for the secondary roles, because those roles exert a stronger influence on physicians’ role conflict than that of the primary role. also, they should recognize that role conflicts between the primary and secondary roles affect technology adoption decisions. 6.4 limitations and future research the findings should be interpreted in the light of limitations. this study has limited external validity (im & straub, 2015) in that the setting focuses on physicians in medical schools in korea. however, generalizing the theoretical framework to other medical school contexts is feasible, given the generalizable nature of the key constructs and the prominent role of clinical care among physicians (ahmady et al., 2007). also, the study can be generalized to other types of jobs that involve multiple roles. our study sets the groundwork for future research. first, one interesting factor is the role of the perceived self-sufficiency of physicians on technology adoption. when physicians have a high sense of perceived self-sufficiency regarding their knowledge and their expertise about current supporting technologies, they may be less willing to accommodate new technologies. second, research needs to explore how it adoption and usage can alleviate role strain (i.e., role overload and role conflict). finally, research needs to probe the possibility of role conflict caused by the characteristics of roles (e.g., the tasks, logic, processes, and interactions). 7 conclusion this study explores the impact of role overload and role conflict among multiple roles on it adoption. physicians in medical schools are involved in handling multiple job roles. to better serve their job roles as clinicians, teachers, and researchers, they face the decision of whether to adopt it and if so, which technologies—technologies that support clinical care, teaching, or research. the findings show that the multiple roles of physicians and role conflicts significantly influence the technology adoption intentions of pmss. australasian journal of information systems park et al. 2023, vol 27, research article impact of role overload and role conflict 24 acknowledgement this research was supported by a grant of yeungnam university (2015). references ahmad, r. w., salah, k., jayaraman, r., yaqoob, i., ellahham, s., & omar, m. (2021). the role of blockchain technology in telehealth and telemedicine. international journal of medical informatics, 148, 1-10. ahmady, s., changiz, t., masiello, i., & brommels, m. (2007). organizational role stress among medical school faculty members in iran: dealing with role conflict. bmc medical education, 7(1), 1-10. doi:https://doi.org/10.1186/1472-6920-7-14 ahuja, m. k. (2002). women in the information technology profession: a literature review, synthesis and research agenda. european journal of information systems, 11(1), 20-34. doi:https://doi.org/10.1057/palgrave.ejis.3000417 ajzen, i. (1991). the theory of planned behavior. organizational behavior and human decision processes, 50(2), 179-211. doi:https://doi.org/10.1016/0749-5978(91)90020-t ansong, e., & boateng, r. (2018). organisational adoption of telecommuting: evidence from a developing country. the electronic journal of information systems in developing countries, 84(1), 1-15. aria, r., & archer, n. (2018). using an educational video vs. in-person education to measure patient perceptions of an online self-management support system for chronic illness. computers in human behavior, 84, 162-170. doi:https://doi.org/10.1016/j.chb.2018.01.041 ayyagari, r., grover, v., & purvis, r. (2011). technostress: technological antecedents and implications. mis quarterly, 35(4), 831-858. doi:https://doi.org/10.2307/41409963 barnett, r. c., & baruch, g. k. (1985). women's involvement in multiple roles and psychological distress. journal of personality and social psychology, 49(1), 135-145. doi:https://doi.org/10.1037/0022-3514.49.1.135 baruch, g. k., & barnett, r. c. (1986). role quality, multiple role involvement, and psychological well-being in midlife women. journal of personality and social psychology, 51(3), 578-585. doi: https://doi.org/10.1037/0022-3514.51.3.578 barzansky, b., & kenagy, g. (2010). the full-time clinical faculty: what goes around, comes around. academic medicine, 85(2), 260-265. beaton, d. e., bombardier, c., guillemin, f., & ferraz, m. b. (2000). guidelines for the process of cross-cultural adaptation of self-report measures. spine, 25(24), 3186-3191. beatty, p. c., & willis, g. b. (2007). research synthesis: the practice of cognitive interviewing. public opinion quarterly, 71(2), 287-311. belias, d., koustelios, a., sdrolias, l., & aspridis, g. (2015). job satisfaction, role conflict and autonomy of employees in the greek banking organization. procedia-social and behavioral sciences, 175, 324-333. doi:https://doi.org/10.1016/j.sbspro.2015.01.1207 berger, s., & bruch, h. (2021). role strain and role accumulation across multiple teams: the moderating role of employees' polychronic orientation. journal of organizational behavior, 42(7), 835-850. https://doi.org/10.1186/1472-6920-7-14 https://doi.org/10.1057/palgrave.ejis.3000417 https://doi.org/10.1016/0749-5978(91)90020-t https://doi.org/10.1016/j.chb.2018.01.041 https://doi.org/10.2307/41409963 https://doi.org/10.1037/0022-3514.49.1.135 https://doi.org/10.1037/0022-3514.51.3.578 https://doi.org/10.1016/j.sbspro.2015.01.1207 australasian journal of information systems park et al. 2023, vol 27, research article impact of role overload and role conflict 25 bhalla, p., qazi, s., & miralam, m. s. (2019). effect of organizational role stress on organizational culture: evidence from service-sector. journal of business and retail management research, 13(4), 1-8. bolino, m. c., & turnley, w. h. (2005). the personal costs of citizenship behavior: the relationship between individual initiative and role overload, job stress, and work-family conflict. journal of applied psychology, 90(4), 740-748. doi:https://doi.org/10.1037/00219010.90.4.740 bretschneider, s., & wittmer, d. (1993). organizational adoption of microcomputer technology: the role of sector. information systems research, 4(1), 88-108. doi:https://doi.org/10.1287/isre.4.1.88 bulut, s., yıldız, a., & kaya, s. (2019). evaluation of transition to electronic prescriptions in turkey: perspective of family physicians. international journal of health policy and management, 8(1), 40. bunton, s. a., corrice, a. m., pollart, s. m., novielli, k. d., williams, v. n., morrison, l. a., . . . fox, s. (2012). predictors of workplace satisfaction for us medical school faculty in an era of change and challenge. academic medicine, 87(5), 574-581. doi:https://doi.org/10.1097/acm.0b013e31824d2b37 burke, p. j., & stets, j. e. (2009). identity theory. new york: oxford university press. callero, p. l. (1992). the meaning of self-in-role: a modified measure of role-identity. social forces, 71(2), 485-501. doi:https://doi.org/10.1093/sf/71.2.485 chau, p. y., & hu, p. j. h. (2001). information technology acceptance by individual professionals: a model comparison approach. decision sciences, 32(4), 699-719. doi:https://doi.org/10.1111/j.1540-5915.2001.tb00978.x clarke, n., & higgs, m. (2020). political skill and role overload as antecedents of innovative work behavior in the public sector. public personnel management, 49(3), 444-469. cohen, p., west, s. g., & aiken, l. s. (2014). applied multiple regression/correlation analysis for the behavioral sciences. new york: psychology press. coser, l. a. (1974). greedy institutions; patterns of undivided commitment. new york: free press. coverman, s. (1989). role overload, role conflict, and stress: addressing consequences of multiple role demands. social forces, 67(4), 965-982. doi:https://doi.org/10.1093/sf/67.4.965 dasgupta, p. (2012). effect of role ambiguity, conflict and overload in private hospitals’ nurses’ burnout and mediation through self efficacy. journal of health management, 14(4), 513534. doi:https://doi.org/10.1177/0972063412468980 davidson, e. j., & chismar, w. g. (2007). the interaction of institutionally triggered and technology-triggered social structure change: an investigation of computerized physician order entry. mis quarterly, 31(4), 739-758. doi:https://doi.org/10.2307/25148818 davis, f. d. (1989). perceived usefulness, perceived ease of use, and user acceptance of information technology. mis quarterly, 13(3), 319-340. doi:https://doi.org/10.2307/249008 https://doi.org/10.1037/0021-9010.90.4.740 https://doi.org/10.1037/0021-9010.90.4.740 https://doi.org/10.1287/isre.4.1.88 https://doi.org/10.1097/acm.0b013e31824d2b37 https://doi.org/10.1093/sf/71.2.485 https://doi.org/10.1111/j.1540-5915.2001.tb00978.x https://doi.org/10.1093/sf/67.4.965 https://doi.org/10.1177/0972063412468980 https://doi.org/10.2307/25148818 https://doi.org/10.2307/249008 australasian journal of information systems park et al. 2023, vol 27, research article impact of role overload and role conflict 26 davis, k., doty, m. m., shea, k., & stremikis, k. (2009). health information technology and physician perceptions of quality of care and satisfaction. health policy, 90(2), 239-246. doi:https://doi.org/10.1016/j.healthpol.2008.10.002 deng, q., zheng, y., lu, j., zeng, z., & liu, w. (2021). what factors predict physicians' utilization behavior of contrast-enhanced ultrasound? evidence from the integration of the theory of planned behavior and technology acceptance model using a structural equation modeling approach. bmc medical informatics and decision making, 21(1), 1-10. dünnebeil, s., sunyaev, a., blohm, i., leimeister, j. m., & krcmar, h. (2012). determinants of physicians’ technology acceptance for e-health in ambulatory care. international journal of medical informatics, 81(11), 746-760. doi:https://doi.org/10.1016/j.ijmedinf.2012.02.002 eatough, e. m., chang, c.-h., miloslavic, s. a., & johnson, r. e. (2011). relationships of role stressors with organizational citizenship behavior: a meta-analysis. journal of applied psychology, 96(3), 619-632. doi:https://doi.org/10.1037/a0021887 edwards, a. b., zarit, s. h., stephens, m. a. p., & townsend, a. (2002). employed family caregivers of cognitively impaired elderly: an examination of role strain and depressive symptoms. aging & mental health, 6(1), 55-61. doi:https://doi.org/10.1080/13607860120101149 ellinas, e. h., kaljo, k., patitucci, t. n., novalija, j., byars-winston, a., & fouad, n. a. (2019). no room to “lean in”: a qualitative study on gendered barriers to promotion and leadership. journal of women's health, 28(3), 393-402. engle, r. l., & prince, m. (2005). the impact of role strain on work and life: a cross-cultural comparison. insights to a changing world, 1, 1-13. fornell, c., & larcker, d. f. (1981). evaluating structural equation models with unobservable variables and measurement error. journal of marketing research, 18(1), 39-50. doi:https://doi.org/10.1177/002224378101800104 goldschmidt, p. g. (2005). hit and mis: implications of health information technology and medical information systems. communications of the acm, 48(10), 68-74. doi:https://doi.org/10.1145/1089107.1089141 goode, w. j. (1960). a theory of role strain. american sociological review, 25(4), 483-496. doi:https://doi.org/10.2307/2092933 gruber, m., & macmillan, i. c. (2017). entrepreneurial behavior: a reconceptualization and extension based on identity theory. strategic entrepreneurship journal, 11(3), 271-286. hair, j. f., black, w. c., babin, b. j., anderson, r. e., & tatham, r. (2010). multivariate data analysis. upper saddle river, nj: prentice hall. hall, d. t. (1972). a model of coping with role conflict: the role behavior of college educated women. administrative science quarterly, 17(4), 471-486. doi:https://doi.org/10.2307/2393827 harrer, s., shah, p., antony, b., & hu, j. (2019). artificial intelligence for clinical trial design. trends in pharmacological sciences, 40(8), 577-591. https://doi.org/10.1016/j.healthpol.2008.10.002 https://doi.org/10.1016/j.ijmedinf.2012.02.002 https://doi.org/10.1037/a0021887 https://doi.org/10.1080/13607860120101149 https://doi.org/10.1177/002224378101800104 https://doi.org/10.1145/1089107.1089141 https://doi.org/10.2307/2092933 https://doi.org/10.2307/2393827 australasian journal of information systems park et al. 2023, vol 27, research article impact of role overload and role conflict 27 ho, j. l. y., wu, a., & xu, s. x. (2011). corporate governance and returns on information technology investment: evidence from an emerging market. strategic management journal, 32(6), 595-623. https://doi.org/10.1002/smj.886 hopkins, l., hampton, b. s., abbott, j. f., buery-joyner, s. d., craig, l. b., dalrymple, j. l., . . . pradham, a. (2018). to the point: medical education, technology, and the millennial learner. american journal of obstetrics and gynecology, 218(2), 188-192. hu, p. j., chau, p. y., sheng, o. r. l., & tam, k. y. (1999). examining the technology acceptance model using physician acceptance of telemedicine technology. journal of management information systems, 16(2), 91-112. doi:https://doi.org/10.1080/07421222.1999.11518247 im, g., & straub, d. w. (2015). the critical role of external validity in advancing organizational theorizing. communications of the association for information systems, 37(44), 911-964. doi:https://doi.org/10.17705/1cais.03744 jayaseelan, r., prasanth, k., & pichandy, c. (2020). technology acceptance by medical doctors in india: an analysis with utaut model. international journal of scientific & technology research, 9(1), 3854-3857. jeon, w., & kim, a. (2017). present and future of korean medical education the role of yonsei university`s medical education. yonsei journal of medical history, 20(1), 51-68. jiang, s., ma, g., jia, j., wu, m., & wu, z. (2022). mobile ict overuse in the construction industry: effects on job burnout of project managers. journal of construction engineering and management, 148(6), 04022024. karahanna, e., straub, d. w., & chervany, n. l. (1999). information technology adoption across time: a cross-sectional comparison of pre-adoption and post-adoption beliefs. mis quarterly, 23(2), 183-213. doi:https://doi.org/10.2307/249751 karimi, r., omar, z. b., alipour, f., & karimi, z. (2014). the influence of role overload, role conflict, and role ambiguity on occupational stress among nurses in selected iranian hospitals. international journal of asian social science, 4(1), 34-40. khalid, j., weng, q. d., luqman, a., rasheed, m. i., & hina, m. (2021). after-hours workrelated technology use and individuals' deviance: the role of other-initiated versus selfinitiated interruptions. information technology & people, 35(7), 19551979. https://doi.org/10.1108/itp-03-2020-0136 kilroy, s., flood, p. c., bosak, j., & chênevert, d. (2016). perceptions of high-involvement work practices and burnout: the mediating role of job demands. human resource management journal, 26(4), 408-424. kim, s. h. (2008). moderating effects of job relevance and experience on mobile wireless technology acceptance: adoption of a smartphone by individuals. information & management, 45(6), 387-393. doi:https://doi.org/10.1016/j.im.2008.05.002 leidner, d. e., & kayworth, t. (2006). a review of culture in information systems research: toward a theory of information technology culture conflict. mis quarterly, 30(2), 357399. leidner, d. e., sutanto, j., & goutas, l. (2022). multifarious roles and conflicts on an interorganizational green is. mis quarterly, 46(1), 591-608. https://doi.org/10.1080/07421222.1999.11518247 https://doi.org/10.17705/1cais.03744 https://doi.org/10.2307/249751 https://doi.org/10.1108/itp-03-2020-0136 https://doi.org/10.1016/j.im.2008.05.002 australasian journal of information systems park et al. 2023, vol 27, research article impact of role overload and role conflict 28 lenaghan, j. a., & sengupta, k. (2007). role conflict, role balance and affect: a model of wellbeing of the working student. journal of behavioral and applied management, 9(1), 88-109. doi:https://doi.org/10.1037/t06070-000 lindell, m. k., & whitney, d. j. (2001). accounting for common method variance in crosssectional research designs. journal of applied psychology, 86(1), 114-121. ma, h., qiao, h., qu, h., wang, h., huang, y., cheng, h., . . . zhang, n. (2020). role stress, social support and occupational burnout among physicians in china: a path analysis approach. international health, 12(3), 157-163. mackinnon, d. p., lockwood, c. m., hoffman, j. m., west, s. g., & sheets, v. (2002). a comparison of methods to test mediation and other intervening variable effects. psychological methods, 7(1), 83-104. doi:https://doi.org/10.1037/1082-989x.7.1.83 malhotra, n. k., kim, s. s., & patil, a. (2006). common method variance in is research: a comparison of alternative approaches and a reanalysis of past research. management science, 52(12), 1865-1883. marks, s. r. (1977). multiple roles and role strain: some notes on human energy, time and commitment. american sociological review, 42(6), 921-936. doi:https://doi.org/10.2307/2094577 marks, s. r., & macdermid, s. m. (1996). multiple roles and the self: a theory of role balance. journal of marriage and family, 58(2), 417-432. doi:https://doi.org/10.2307/353506 marshall, n. l., & barnett, r. c. (1993). work-family strains and gains among two-earner couples. journal of community psychology, 21(1), 64-78. doi:https://doi.org/10.1002/15206629(199301)21:1<64::aid-jcop2290210108>3.0.co;2-p martikainen, p. (1995). women's employment, marriage, motherhood and mortality: a test of the multiple role and role accumulation hypotheses. social science & medicine, 40(2), 199212. doi:https://doi.org/10.1016/0277-9536(94)e0065-z mechanic, d. (2003). physician discontent challenges and opportunities. jama, 290(7), 941946. doi:https://doi.org/10.1001/jama.290.7.941 mittal, m., & bhakar, s. s. (2018). examining the impact of role overload on job stress, job satisfaction and job performance-a study among married working women in banking sector. international journal of management studies, 2(7), 1-11. moon, m. j., & bretschneiber, s. (2002). does the perception of red tape constrain it innovativeness in organizations? unexpected results from a simultaneous equation model and implications. journal of public administration research and theory, 12(2), 273292. mostaghimi, a., levison, j. h., leffert, r., ham, w., nathoo, a., halamka, j., prout, c., arky, r. a., tosteson, d. c., & quattrochi, j. (2006). the doctor’s new black bag: instructional technology and the tools of the 21st century physician. medical education online, 11, 1-3. doi:http://nrs.harvard.edu/urn-3:hul.instrepos:33785888 o'brodovich, h., beyene, j., tallett, s., macgregor, d., & rosenblum, n. d. (2007). performance of a career development and compensation program at an academic health science center. pediatrics, 119(4), e791-e797. doi:https://doi.org/10.1542/peds.2006-2207 https://doi.org/10.1037/t06070-000 https://doi.org/10.1037/1082-989x.7.1.83 https://doi.org/10.2307/2094577 https://doi.org/10.2307/353506 https://doi.org/10.1002/1520-6629(199301)21:1%3c64::aid-jcop2290210108%3e3.0.co;2-p https://doi.org/10.1002/1520-6629(199301)21:1%3c64::aid-jcop2290210108%3e3.0.co;2-p https://doi.org/10.1016/0277-9536(94)e0065-z https://doi.org/10.1001/jama.290.7.941 http://nrs.harvard.edu/urn-3:hul.instrepos:33785888 https://doi.org/10.1542/peds.2006-2207 australasian journal of information systems park et al. 2023, vol 27, research article impact of role overload and role conflict 29 o’brodovich, h., pleinys, r., laxer, r., tallett, s., rosenblum, n., & sass-kortsak, c. (2003). evaluation of a peer-reviewed career development and compensation program for physicians at an academic health science center. pediatrics, 111(1), e26-e31. doi:https://doi.org/10.1542/peds.111.1.e26 o’dowd, e., o’connor, p., lydon, s., mongan, o., connolly, f., diskin, c., . . . reidmcdermott, b. (2018). stress, coping, and psychological resilience among physicians. bmc health services research, 18(1), 1-11. owens, t. j., robinson, d. t., & smith-lovin, l. (2010). three faces of identity. annual review of sociology, 36, 477-499. doi:https://doi.org/10.1146/annurev.soc.34.040507.134725 ozdemir, s., & trott, p. (2009). exploring the adoption of a service innovation: a study of internet banking adopters and non-adopters. journal of financial services marketing, 13(4), 284-299. doi:https://doi.org/10.1057/fsm.2008.25 palacholla, r. s., fischer, n., coleman, a., agboola, s., kirley, k., felsted, j., . . . jethwani, k. (2019). provider-and patient-related barriers to and facilitators of digital health technology adoption for hypertension management: scoping review. jmir cardio, 3(1), e11951. park, y. h., im, s., seo, k. h., lee, b. i., & park, c. h. (2012). a study on the faculties' satisfaction of korean medical schools for their educational, medical and research environment improvement. retrieved from korean medical association: http://www.dbpia.co.kr/article/node02467967 podsakoff, p. m., mackenzie, s. b., lee, j.-y., & podsakoff, n. p. (2003). common method biases in behavioral research: a critical review of the literature and recommended remedies. journal of applied psychology, 88(5), 879-903. poon, e. g., blumenthal, d., jaggi, t., honour, m. m., bates, d. w., & kaushal, r. (2004). overcoming barriers to adopting and implementing computerized physician order entry systems in u.s. hospitals. health affairs, 23(4), 184-190. doi:10.1377/hlthaff.23.4.184 pullins, e., tarafdar, m., & pham, p. (2020). the dark side of sales technologies: how technostress affects sales professionals. journal of organizational effectiveness: people and performance. reay, t., goodrick, e., waldorff, s. b., & casebeer, a. (2017). getting leopards to change their spots: co-creating a new professional role identity. academy of management journal, 60(3), 1043-1070. román, s., rodríguez, r., & jaramillo, j. f. (2018). are mobile devices a blessing or a curse? effects of mobile technology use on salesperson role stress and job satisfaction. journal of business & industrial marketing. rorong, e. (2021). working from home during pandemic covid-19: effects of mobile technology use, supervisor support and job insecurity on employee’s role stress and job satisfaction. jurnal akuntansi, manajemen dan ekonomi, 23(2), 20-33. rubel, m. r. b., kee, d. m. h., & rimi, n. n. (2017). the mediating role of work–family conflict on role stressors and employee turnover intention relationship in labour-oriented organizations. global business review, 18(6), 1384-1399. https://doi.org/10.1542/peds.111.1.e26 https://doi.org/10.1146/annurev.soc.34.040507.134725 https://doi.org/10.1057/fsm.2008.25 australasian journal of information systems park et al. 2023, vol 27, research article impact of role overload and role conflict 30 schaufeli, w. b., bakker, a. b., van der heijden, f. m., & prins, j. t. (2009). workaholism, burnout and well-being among junior doctors: the mediating role of role conflict. work & stress, 23(2), 155-172. doi:https://doi.org/10.1080/02678370902834021 seo, k. h., lim, s. m., lee, b. i., park, c. h., & park, y. h. (2012). a survey on the education, medical practice, research, and fringe benefits of korean medical school faculty. journal of the korean medical association, 55(11), 1128-1141. seppänen, m., pajarre, e., & kuparinen, p. (2015). the effects of performance-monitoring technology on privacy and job autonomy. international journal of business information systems, 20(2), 139-156. shah, j. m., jacqueline, b., & don, k. (2020). meta-design knowledge for clinical decision support systems. australasian journal of information systems, 24(0). doi:https://doi.org/10.3127/ajis.v24i0.2049 shamsi, m., iakovleva, t., olsen, e., & bagozzi, r. p. (2021). employees’ work-related well-being during covid-19 pandemic: an integrated perspective of technology acceptance model and jd-r theory. international journal of environmental research and public health, 18(22), 11888. sieber, s. d. (1974). toward a theory of role accumulation. american sociological review, 39(4), 567-578. doi:https://doi.org/10.2307/2094422 speier, c., & venkatesh, v. (2002). the hidden minefields in the adoption of sales force automation technologies. journal of marketing, 66(3), 98-111. doi:https://doi.org/10.1509/jmkg.66.3.98.18510 steinert, y., o’sullivan, p. s., & irby, d. m. (2019). strengthening teachers’ professional identities through faculty development. academic medicine, 94(7), 963-968. sumra, m. k., & schillaci, m. a. (2015). stress and the multiple-role woman: taking a closer look at the “superwoman”. plos one, 10(3), 1-24. doi:https://doi.org/10.1371/journal.pone.0120952 svider, p. f., pashkova, a. a., choudhry, z., agarwal, n., kovalerchik, o., baredes, s., . . . eloy, j. a. (2013). comparison of scholarly impact among surgical specialties: an examination of 2429 academic surgeons. the laryngoscope, 123(4), 884-889. doi:https://doi.org/10.1002/lary.23951 tarafdar, m., qiang, t. u., ragu-nathan, b. s., & ragu-nathan, t. s. (2007). the impact of technostress on role stress and productivity. journal of management information systems, 24(1), 301-328. doi:https://doi.org/10.2753/mis0742-1222240109 teo, h. h., wei, k. k., & benbasat, i. (2003). predicting intention to adopt interorganizational linkages: an institutional perspective. mis quarterly, 27(1), 19-49. doi:https://doi.org/10.2307/30036518 thoits, p. a. (1992). identity structures and psychological well-being: gender and marital status comparisons. social psychology quarterly, 55(3), 236-256. thomas, p. a., diener-west, m., canto, m. i., martin, d. r., post, w. s., & streiff, m. b. (2004). results of an academic promotion and career path survey of faculty at the johns hopkins university school of medicine. academic medicine, 79(3), 258-264. https://doi.org/10.1080/02678370902834021 https://doi.org/10.3127/ajis.v24i0.2049 https://doi.org/10.2307/2094422 https://doi.org/10.1509/jmkg.66.3.98.18510 https://doi.org/10.1371/journal.pone.0120952 https://doi.org/10.1002/lary.23951 https://doi.org/10.2753/mis0742-1222240109 https://doi.org/10.2307/30036518 australasian journal of information systems park et al. 2023, vol 27, research article impact of role overload and role conflict 31 retrieved from http://journals.lww.com/academicmedicine/fulltext/2004/03000/results_of_an_acade mic_promotion_and_career_path.13.aspx thun, s., halsteinli, v., & løvseth, l. (2018). a study of unreasonable illegitimate tasks, administrative tasks, and sickness presenteeism amongst norwegian physicians: an everyday struggle? bmc health services research, 18(1), 1-9. touati, n., rodríguez, c., paquette, m.-a., maillet, l., & denis, j.-l. (2019). professional role identity: at the heart of medical collaboration across organisational boundaries. international journal of integrated care, 19(2). turan, a. h., & koç, t. (2022). health information technology adoption and acceptance of turkish physicians-a model proposal and empirical assessment. health informatics journal, 28(2), 1-23. venkatesh, v. (2006). where to go from here? thoughts on future directions for research on individual-level technology adoption with a focus on decision making. decision sciences, 37(4), 497-518. doi:https://doi.org/10.1111/j.1540-5414.2006.00136.x venkatesh, v., morris, m. g., davis, g. b., & davis, f. d. (2003). user acceptance of information technology: toward a unified view. mis quarterly, 27(3), 425-478. doi:https://doi.org/10.2307/30036540 venkatesh, v., sykes, t., chan, f. k., thong, j. y., & hu, p. j. (2019). children's internet addiction, family-to-work conflict, and job outcomes: a study of parent-child dyads. mis quarterly, 43(3), 903-927. venkatesh, v., sykes, t. a., & zhang, x. (2011). just what the doctor ordered: a revised utaut for emr system adoption and use by doctors. paper presented at the proceedings of the 44th hawaii international conference on system sciences, honolulu, hi. ward, c. r. (1986). the meaning of role strain. advances in nursing science, 8(2), 39-50. doi:https://doi.org/10.1097/00012272-198601000-00008 ward, j., gordon, j., field, m. j., & lehmann, h. p. (2001). communication and information technology in medical education. the lancet, 357(9258), 792-796. wiener, m., mähring, m., remus, u., & saunders, c. (2016). control configuration and control enactment in information systems projects: review and expanded theoretical framework. mis quarterly, 40(3), 741-774. yang, s., qiu, y., & he, j. (2021). meta-analysis of the impact of role stress on the turnover intention of medical staff in tertiary hospitals. annals of palliative medicine, 10(1), 469-478. zobair, k. m., louis, s., luke, h., kuldeep, s., & md jahirul, i. (2021). health seekers’ acceptance and adoption determinants of telemedicine in emerging economies. australasian journal of information systems, 25. doi:https://doi.org/10.3127/ajis.v25i0.3071 http://journals.lww.com/academicmedicine/fulltext/2004/03000/results_of_an_academic_promotion_and_career_path.13.aspx http://journals.lww.com/academicmedicine/fulltext/2004/03000/results_of_an_academic_promotion_and_career_path.13.aspx https://doi.org/10.1111/j.1540-5414.2006.00136.x https://doi.org/10.2307/30036540 https://doi.org/10.1097/00012272-198601000-00008 https://doi.org/10.3127/ajis.v25i0.3071 australasian journal of information systems park et al. 2023, vol 27, research article impact of role overload and role conflict 32 appendix a. literature review approach 1. technology adoption research on physicians we followed leidner and kayworth’s (2006) methodological approach for systematic literature review. first, develop criteria on which studies need to be incorporated into our analysis. second, develop a strategy to search the literature. third, plan how to document and analyze the selected studies. we followed wiener et al.’s (2016) four-step search strategy: (a) searched google scholar with keywords such as technology, adoption, use, acceptance, physicians, and medical faculty; (b) searched a database of information systems journals with keywords; (c) performed a similar keyword search in web of science, science direct, and pubmed; and (d) identified key articles that cited other technology adoption studies as extracted from google scholar. for the inclusion criteria, studies published during the period of 1990 to 2022 were targeted. studies written in the english language were retrieved. the following exclusion criteria were used: articles with less than 20 references, review articles, articles that did not employ a theory of technology adoption/use behavior as the main theory basis, and articles published before 1990. a total of 439 articles remained for the literature review in this study. table a summarizes the results. major theory used # of articles major predictors technology acceptance model (tam) 209 perceived usefulness, perceived ease of use, etc. united theory of acceptance and use of technology (utaut) 142 performance expectancy, effort expectancy, social influence, facilitating conditions, gender, age, experience, voluntariness of use, etc. theory of planned behavior (tpb) 88 subjective norms, perceived behavioral control, attitude toward behavior, etc. *note: some studies used a combination of these theories, but they were not classified separately because our objective is to identify the major theories used and the major predictors of technology adoption by physicians. table a. summary of physicians’ technology adoption research 2. role overload and role conflict research the same methodological approach mentioned above was used for the literature search. we first searched google scholar and the web of science, science direct, and pubmed databases with keywords such as technology adoption, role overload and role conflict, workplace, physicians, and medical faculty. no articles were retrieved, indicating the research gap. then, we extended our search by removing the keywords of physicians and medical faculty from the previous keyword set, resulting in research on role overload and/or role conflict in a variety of workplace contexts. for the inclusion criteria, studies written in the english languages and published between 2005 and 2022 were searched. we used the same exclusion criteria mentioned in appendix a: articles with less than 20 references, review articles, articles that did not employ a theory related to role strain/stress (which is grounded in the concepts of role overload and role conflict) as the main theory basis, and articles published before 2005. a total of 345 articles remained for literature review in the study. table 1 in the main text outlines the major studies. appendix b. preliminary interview protocol • what are the major tasks as a clinician, as educator, and as researcher? • what types of clinical technology are you and your colleagues currently using or intend to adopt in the near future? what are the functions of the technologies? australasian journal of information systems park et al. 2023, vol 27, research article impact of role overload and role conflict 33 • what types of educational technology are you and your colleagues currently using or intend to adopt in the near future? what are the functions of the technologies? • what types of research technology are you and your colleagues currently using or intend to adopt in the near future? what are the functions of the technologies? • how do multiple job roles influence technology adoption intentions? • in the multiple role involvement, what are the factors that influence medical school faculties’ technology adoption intentions? appendix c. open-ended questions in the main survey please, list the types of technologies that you are currently using or intend to adopt in the near future. • clinical technology is used for clinical purposes. physicians use this technology at the front line where they directly face patients and treat them. examples include electronic medical record systems, telemedicine systems, and robot operation systems. (this list is incomplete.) • educational technology is used for education purposes. physicians use this technology when they prepare to teach medical students in classes. the technology also supports students’ educational activities. examples include university research management systems, database management systems, and statistical analysis programs. (this list is incomplete.) • research technology is used for research purposes. physicians use this technology at the front line for the purpose of their research projects. examples include learning management systems, virtual classrooms, and voting systems. (this list is incomplete.) type existing new (adoption in consideration) clinical care teaching research appendix d. measurements construct measures resource role overload for teaching/ [clinical care]/ [research], the things i do add up to being just too much. the amount of work i am expected to do is too great. i never seem to have enough time to get everything done at work. it often seems like i have too much work for one person to do. barnett & baruch (1985); marshall & barnett (1993); bolino & turnley (2005) role conflict how often do you have to juggle different obligations [teaching and clinical care/ [clinical care and research]/ [research and teaching]) that conflict with one another and give you a pulled-apart feeling? how much do your jobs of teaching and clinician interfere with each other? how often are you stressed trying to balance your responsibilities (teaching and clinical care) that conflict with one another? barnett & baruch (1985); coverman (1989); bolino & turnley (2005) australasian journal of information systems park et al. 2023, vol 27, research article impact of role overload and role conflict 34 intentions to adopt technology i intend to adopt new educational (/[clinical] /[research]) technologies in my job within [the next six months]. i am likely to adopt new educational (/[clinical] /[research]) technologies in [a year’s time]. karahanna et al. (1999); teo et al. (2003) role identity as a physician i consider myself as a teacher/ [clinician]/ [researcher]. percentage of each among 100 % callero (1992) appendix e. analysis of cognitive interviews methods cognitive interviews of five interviewees were performed by two interviewers. the cognitive interviewers used a standardized interview protocol to ask probing follow-up questions, in order to obtain verbal information about why the physicians would not intend to adopt education/clinical/research technology, given their role conflicts (beatty & willis, 2007). the interviews confirmed that their opinions of “no intention to adopt technology” was not ascribed to item context or to measurement context effects. these confirm that common method bias played a minimal role with our data. analysis of cognitive interviews after the hypothesis testing, we conducted cognitive interviews to better understand the statistical findings. five physicians explained why they and their colleagues would not like to adopt it, providing insights into unsupported hypotheses. the interviews suggest that the challenges they face while considering technology adoption can be classified into the four types: 1) extra burden, 2) perceived self-sufficiency, 3) technology incompatibility, and 4) institutional barriers. table b summarizes the challenges and reasons for technology nonadoption intentions. type of challenge reason(s) description extra burden additional work adopting new educational technology may create additional work. time conflict in multiple job roles adopting new educational or research technology may create conflicts in time management among multiple job roles. perceived selfsufficiency reliance on one’s own expertise some physicians think that they already have enough knowledge and expertise and thus do not need to adopt new educational or clinical technology to support their work. reliance on traditional methods/tools some physicians hold the view that traditional educational tools are enough to prepare teaching and instruct students. tenure some senior faculty members think that new educational or research technology is for junior faculty members to improve their performance, rather than for senior faculty members. technology incompatibility incompatibility of technologies with practice some physicians see a gap between educational technology and actual application to practice. institutional barriers bureaucracy involved in technology adoption adopting clinical technology involves a complicated decision making process that goes through many different units, such as the department, the medical school, and the university. table b. summary of cognitive interviews australasian journal of information systems park et al. 2023, vol 27, research article impact of role overload and role conflict 35 (1) extra burden to some physicians, adopting new technology is burdensome and creates a new role conflict, rather than reduce one. thus, they foresee further role conflict, which leads to their technology non-adoption intentions. additional work. in some cases, adopting educational technology is considered to be additional work for physicians, unless the adoption is mandatory. adopting new educational technology is likely to add additional work (associated with the adoption) to physicians who experience a role conflict between teacher and researcher unless physicians have a particular interest in education and beliefs on educator’s role (interviewee: physician 4). time conflict in multiple job roles. adopting either educational or research technology may create time conflicts in the multiple job roles of physicians and may even undermine work efficiency. physicians have to teach a lot of knowledge within a limited time. so adopting and learning new educational technology is likely to be very often time-consuming and may not increase efficiency in teaching activities (physician 2). learning new research technology requires additional time, which creates conflicts in physicians’ time management efforts in multiple job roles (physician 3). (2) perceived self-sufficiency some physicians think that new technology is not necessary to resolving their role conflicts. the reasons for their perceived self-sufficiency reflect the characteristics of their role, expertise, and tenure. reliance on one’s own expertise. the level of physicians’ knowledge and expertise for the job role may negatively impact their intentions to adopt educational or clinical technology. if physicians believe they already have enough experiences and skills in handling teaching materials, they are less likely to think that they need to adopt new educational technology to support their teaching (physician 1). if physicians consider their major role as a clinician and believe that they have enough knowledge and expertise in their specialties, they will have low intentions to adopt new clinical technology. in contrast, physicians are more likely to learn and adopt educational technology, because they often think they lack knowledge and tools in handling teaching materials, thus adopting educational technology may help to improve their teaching efficiency (physician 2). reliance on traditional methods/tools. some physicians believe that existing educational tools are enough to fulfill their job as a teacher, and thus additional educational technology is unnecessary. it is often observed that physicians who have more than 20 years educational experiences view adopting new educational technology, such as web-based or online educational technology as unnecessary. they think that they have already had enough traditional teaching tools to prepare teaching and instruct students (physician 2). most frequently used traditional teaching tools include ppt and a projector in classroom teaching. physicians with support of residents also do bedside teaching for senior level students australasian journal of information systems park et al. 2023, vol 27, research article impact of role overload and role conflict 36 in clinical practice, where paper-based charts are often used to check and describe patient’s historical data. these physicians are less likely to adopt new educational technology, since the existing tools are sufficient to support their teaching needs (physician 4). tenure. physicians’ tenure may make them complacent in their position and may make them less inclined to take initiative in adopting new technologies. specifically, this perception may influence both educational and research technologies. it may be different based on the faculty’s tenure position. physicians who think that they reach a high level of expertise in their specialty areas are less likely to adopt new clinical technology. this phenomenon can be evident as the position of faculty rises from assistant professor, associate professor to full professor (physician 1). compared with assistant professors, full professors who have lots of experiences in research and teaching activities are less likely to adopt new research and educational technologies. they think that adopting new research and educational technologies is an opportunity for junior faculties to improve their performance. full professors often have a sense of stability resulted from their tenure position and lots of experiences accumulated over years (physician 4). (3) technology incompatibility physicians often use educational technologies (e.g., simulator) to understand the scientific basis of medicine. a challenge in such educational technology is whether the technology is compatible with practice. incompatibility to practice could explain a low intention to adopt educational technology. some physicians even think that newly introduced dummy technologies and multimedia educational technologies are not good enough to replace traditional instruction methodologies. there is a gap between teaching students with such educational technology and real world clinical practice. in that sense, educational technology may have a limitation to teach medical students (physician 2). simulation technology does not always create real-life situations. some situations are left out from simulators. learning and using such educational technology may lead students to underperform in medical training with incorrect diagnosis (physician 5). (4) institutional barriers bureaucracy in an organization is one procedural characteristic that is formulated based on the organizational hierarchical structure and bureaucrat culture (ho et al., 2011; moon & bretschneiber, 2002). such institutional factors may also lead to physicians’ technology nonadoption intentions. compared with adopting educational technology, adopting clinical technology is quite expensive and thus becomes too much of a hassle. it involves very complicated and difficult decision-making processes, which required approvals from every relevant party in the department, medical school, and university (physician 4). australasian journal of information systems park et al. 2023, vol 27, research article impact of role overload and role conflict 37 copyright: © 2023 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v27i0.3769 http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 literature review 2.1 technology adoption research on physicians 2.2 role overload and role conflict 2.3 role overload and role conflict and information technologies 2.4 multiple roles and role identities 3 hypotheses development 3.1 physicians in medical schools in korea 3.2 physicians’ role overload and role conflict 4 research methodology 4.1 sample and data collection 4.2 measurement and controls 5 data analysis and results 5.1 psychometric properties of measures 5.2 common method bias analysis 5.3 hypothesis testing 5.4 mediation tests 6 discussion 6.1 discussion of the findings 6.2 implications for theory 6.3 implications for practice 6.4 limitations and future research 7 conclusion appendix c. open-ended questions in the main survey methods analysis of cognitive interviews ajis vol. 6 no. 2 may 1999 applying quality costs in a software development environment i.p. hollingsworth+, w. keogh'and m.h. atkins% +department of commerce, university of birmingham b152tt e-mail: i.p.hollingswoith@bham.ac.uk 'aberdeen business school the robert gordon university viewfield road aberdeen ab15 7aw e-mail: w.keogh@rgu.ac.uk % (correspondence author) department of management studies university of aberdeen old aberdeen ab24 3qy e-mail: m.h.atkins@abdn.ac.uk abstract this paper shows how quality costs can be a measure of software quality. the relationship between quality costs and other software quality metrics is briefly explained, and software development oriented versions of the two principal quality cost models are described. finally the paper discusses the major issues involved in setting up a software quality cost programme. the concepts are based on previous research on quality costs in manufacturing, coupled with work on software metrics and the work currently being undertaken by the authors in a number of industries. key words: quality costs, software quality, quality metrics aim of the paper this paper illustrates the potential use of quality costs as a measure of software quality, and outlines how this could be done. based on ideas established in manufacturing, it presents the possibilities for quality costs to complement other measures of software quality. quality costs are not seen as better absolute measures of software quality than any other metric, nor is it die intention of the paper to establish a pecking order of software quality metrics. the main concepts discussed are based on manufacturing's 40 or so years' experience of quality costs and other measures of quality which are combined with more recent work on software quality and metrics. previous research has identified two basic models to identify quality costs and these are described and applied to a generic software development environment. the potential difficulties encountered in establishing a quality cost programme are also outlined, and ways to avoid them suggested. introduction there is an established view of software development which sees it as an engineering discipline, in that it occurs in a planned and systematic manner, and with predictable results. one outcome of this is the increasing interest in planning and controlling software development through measurement. this trend is particularly noticeable in the field of software quality management. this view that quality is measurable has been embodied in the international standards organisation's guide to quality systems for software development, iso 9000-3 (bsi, 1991), which makes reference to the use of both product and process metrics as necessary elements of any quality system intended to meet the standard. this interest in the measurement of software quality has also lead to several major research projects within the uk including esprit project ep-5425 pyramid (pyramid consortium, 1991) and the department of trade and industry's quantum project (praxis, 1992). the two main difficulties are deciding what 'software quality' actually means, and defining practical and meaningful ways by which to measure it. as iso 9000-3 states, there are currently no universally accepted measures of software quality.1 (clause 6.4 bsi, 1991b) 64 ajis vol. 6 no. 2 may 1999 introducing quality costs first it is necessary to consider the nature of quality costs. juran (1951) suggests that they could be identified by asking the question what present costs would disappear if all defects disappeared?' this seems to imply that the cost of quality is simply the cost incurred by making errors. some might interpret this to mean that reducing quality costs would simply require more resources to be expended in removing errors. in the context of software development, the logical conclusion of this approach is to prove mathematically all software, taking into consideration all environmental factors. this is largely impractical, and makes this interpretation of quality costs of limited use in the context. however, the original question requires little modification to address this problem. if it is rephrased as "what present costs would disappear if we could be assured that all defects had disappeared?', the costs of error prevention and error identification become part of the definition. in fact this definition is in line with the standard definition of quality related costs as given in bs4778 part 2 (bsi, 1991a), namely the 'cost in ensuring and assuring quality as well as loss incurred when quality is not achieved'. quality cost models two very different approaches to the modelling of quality costs have been developed over the years, and are outlined in bs6143, parts 1 (bsi, 1992) and 2 (bsi, 1990). this section of the paper describes the prevention, appraisal & failure (paf) and process models. the paf model is restricted to consideration of quality-related costs while the process model also considers the actual cost of the basic processes themselves. it is important that quality costs are distinguished from the process costs which would normally be anticipated, for example, the cost of a planned level of post-implementation support for users. the other test for a quality cost, defined by juran (1951) is that it should be an avoidable cost, based on the previously mentioned idea of being assured that no defects exist in the product. taking the example of post-implementation support, even if the product is defect-free and meets user requirements, users are still likely to need some support for the new product. similarly with maintenance work on the product, even if the software is defect-free and meets initial requirements, and thus requires no corrective maintenance, adaptive maintenance is still likely and therefore should not be regarded as a quality cost. the prevention appraisal & failure (paf) model. the paf model was developed through work done in the 1940s and 1950s (juran, 1951; feigenbaum, 1956), and is used as the basis for the british standard (bsi, 1990). the thinking behind this model can be summarised neatly by two statements from the standard: "failures, however caused, reduce profits" and "preventative quality control activities and the appraisal of quality standards cost money to operate". essentially the paf model breaks quality costs down into three main categories: prevention costs: which arise in the course of preventing or reducing the risk of nonconformities or defects in the product; appraisal costs: which are associated with the cost of checking conformance of the product to quality requirements; failure costs: which arise as a result of identified non-conformities or defects in the product. however, failure costs can be further sub-divided into internal and external failures: internal failure costs: which arise due to non-conformities or defects identified at any stage of the development process; external failure costs: which arise due to non-conformities or defects identified at user acceptance testing or after delivery to the customer/user. this gives a total of four categories of costs to consider. hollocker (1989) also suggested that prevention costs should be sub-divided into two separate categories (prevention and control). however, hollocker's prevention costs can be regarded as being beyond the scope of individual development projects. they are associated with non-routine activities such as training. on the other hand, control costs are associated with routine project 65 ajis vol. 6 no. 2 may 1999 control issues such as project and configuration management. what separates the two categories is this idea of routinisation. hollocker argues that if an activity is not regarded as a normal part of project control then its quality-related costs should be regarded as prevention costs. while the addition of this extra category (i.e. control) assists in identifying the impact activities such as training have on routine control costs, it does exacerbate the problem of categorising costs. control issues are "apparently the hardest to manage", and so are worthy of being singled out, though adding the caveat that if a control function is not provided by "department charter" it should be placed in the prevention category. however, the addition of an extra category exacerbates the problem of categorising costs. we have used the original paf categories to illustrate the model, though some organisations might find that the extra category better serves their needs. the limitations of the paf model have been outlined by porter and rayner (1992). they identified difficulties and problem areas associated the application of the model such as the recording of quality cost information in a comprehensive programme; difficulties in deciding the accuracy levels which the organisation may require; categorising and placing the costs within the working model; and, of course, the difficulties of obtaining the basic data necessary for reporting the costs which feigenbaum has addressed from as early as the 1950s (feigenbaum, 1956). further criticisms of the paf model are that it focuses on cost reduction rather than the positive impact of quality on sales revenue; that all successful applications of the model had hitherto been in large firms and, most importantly (they suggest), that it "implies an acceptable quality level as a result of the trade off between prevention and appraisal costs and failure costs." this, it is alleged, is incompatible with the continuous improvement philosophy of total quality management. another aspect to bear in mind is that failure costs should include the cost of any additional appraisal or prevention activities necessary to fix an error, such as re-testing or any extra configuration management effort required. as a result of this it is necessary to take care, when collecting data, that a single cost is only collected under one category. accordingly, care should be taken when dealing with the knock-on effects of errors which require new program versions to be developed and controlled, including their configuration management. the process model of quality costs the second approach to quality costs is to link all costs to their associated process rather than split all activities into separate cost categories as in the paf model . within each process the quality-related costs are then split into costs of conformance or non-conformance: cost of conformance the cost of achieving the required standard or specification for the product or service supplied by the organization, and cost of non-conformance the costs arising where specifications are not being met and failures occur at various points in the supply cycle. these correspond approximately to the paf categories, in that failure costs are equivalent to cost of nonconformance, and prevention and appraisal are equivalent to cost of conformance. accordingly the final quality costs measured by the two models will not differ significantly but one important difference is that in the process model normal process costs are also regarded as a cost of conformance (bsi, 1992; porter and rayner, 1992) a drawback of the process model is that it requires a well-defined, stable process. if the process is not clearly understood it is difficult to establish relevant costs. secondly, if the process is not stable for some reason, such as changes in personnel, organisational structure, or the processes themselves, then the quality cost model may require extensive modification. this close link to the development process itself can also make it difficult effectively to fit in the costs of activities covering a wide area of the organisation, such as setting up a quality management system or tqm programme. a useful discussion on models including the use of process models in quality costs which can be used as a measure to improve effectiveness as well as efficiency of quality activities is given in hwang and aspinall (1996). 66 ajis vol. 6 no. 2 may 1999 hidden quality costs quality costs can include costs which arise out of services provided beyond the bounds of the organisation, in either its suppliers or its customers. these may be reflected in the price paid for products/services and in the other costs incurred by the organisation such as rectification or downtime. these are hidden quality costs in the sense that they are not measured by the organisation. winchell (1987) has suggested that they can be divided into three parts: those that are incurred by the supplier within the organisation, and passed on to the buyer in terms of price; those that are incurred by the customer organisation in solving problems for the supplier, such as identifying faults in development tools; those that are not allocated to suppliers, but are incurred by the customer organisation as a result of potential or actual supplier problems; this would include the costs related to unexpected system behaviour caused by bought-in software, or extensive incoming acceptance-testing of software tools made necessary to assure their quality. quality costs en manufacturing the importance of quality costs was first recognised in the context of manufacturing industry and there is an established body of research undertaken and evidence available. in the uk, various studies including 'standards in specifications in the engineering industry' (warner, 1977), indicated that quality costs comprise very large proportions of manufacturing costs. it was estimated that the cost of achieving or not achieving the required quality standard of goods produced by uk establishments cost the uk approximately 10% of the £105,000 million total sales of goods in 1976, or £10 billion. these figures were reported in "a national strategy for quality", a consultative document produced by the uk government (dpcp, 1978), and originated from the 1976 census of production. quality costs were taken as the costs of defective quality together with the costs of attaining quality, and were estimated as lying in the range of 4 to 15 per cent of turnover. the importance of addressing quality costs was further emphasised in 1985 in the national economic development council paper "quality and value for money" (nedc 1985), which not only discussed the benefits of identifying and dealing with quality costs, but made a firm recommendation that "accountants and quality specialists in industry should examine the scope for developing management information systems to identify quality costs (based on bs 6143), with the objective of reducing these costs." the paper estimated that between 10 and 20 percent of sales value could be accounted for by quality related costs: taking the 10 per cent figure meant that there was a potential cost of £6 billion in manufacturing industry alone (1985 data). it has also been indicated from other, more recent, studies that quality costs can be as high as 25% of sales turnover (dale and plunkett,1992 and 1995). this compares with the figure of 30% for software quality costs in the u.s., quoted by a. smith of coopers & lybrand deloitte (smith, 1991). the background to the design and use of quality cost programmes lies firmly in manufacturing. an early example from h.g. crockett (1935) outlines how a hosiery mill reduced quality costs by improving workmanship in all operations and, not only identified key cost saving areas, but ultimately resulted in the reduction of the number of quality control inspectors from 25 to five. joseph juran outlined key areas (including elements of quality costs such as scrap and rework) dealing with management problems in the field of inspection and quality control based on his experiences as chief of inspection, control division, in the western electric company (juran, 1945). as previously mentioned, the paf model has been used in one form or another since the 1950s (juran, 1951; feigenbaum, 1956; masser, 1957). in the quality manager and quality costs' (1957), masser makes it clear that there are four major results which can be obtained from accurate quality costs: 1 a tool is provided for measurement of overall business quality performance 2 quality costs are an analysis tool which can be used to indicate where quality money is being spent 3 a tool is provided for programming the when, the where, and how of quality improvements 4 a fine budget tool can be made which enables the forecasting of realistic needs. the american society for quality control (asqc) has issued material to assist with quality costs based on the very early work. in 1967, the quality costs committee of the asqc issued 'quality costs: what and how' which outlined ways of achieving management commitment as well as establishing a programme. a key area 67 ajis vol. 6 no. 2 may 1999 which was addressed in this publication, and its subsequent revisions in 1971 and 1974, was that quality costs had to be sold to senior management: the bottom line of potential dollar savings was the key to gain management commitment. juran's 'gold in the mine1 analogy (juran, 1951) can even today bring it home in a very simple and easily understandable manner to senior executives. the aim of a quality cost improvement programme is to shift the costs from the failure category to prevention, thus leading to cost savings on overheads as well as expense items. for example, if fewer customer complaints and warranty claims are received, then there will be less administration required and subsequently lower costs. as a basis for quality improvement, a quality cost system can play a significant role. potentially, a quality cost system, can become an important tool to be used to assist the management of the organisation (campanella, 1990), for example by providing a tool for facilitating quality programmes, and quality improvement activities. however, as keogh and others (keogh, 1996) have shown, even in advanced industries, with a well educated work-force and a great deal of data available, problems of definition and categorisation can still occur in developing quality cost systems. therefore, by utilising quality costs properly, not only can cost reductions be made but the competitive position of the organisation can be strengthened, and these can be combined to improve profitability. masser's work was carried out in the general electric company, new york, and the results and outcomes are just as relevant today as they were in the 1950s. the 'guide for reducing quality costs' was published in 1977 by the asqc. the task group comprised members from different backgrounds and organisations, including general electric company, digital equipment corp., and international telephone and telegraph corporation. as well as dealing with the quality cost improvement philosophy, setting up a system was addressed and ways of measuring improvements were examined. supplier quality costs were also examined by the asqc and, in 1980, the "guide for managing supplier quality costs" was published to aid both purchasers and suppliers by suggesting methods by which supplier quality costs could be managed, with the second edition following in 1987 (winchell, 1987). crosby (1979) argues that "quality is free" and what this means to practitioners is that the costs of achieving quality are offset by the savings in quality improvements. regardless of whether the organisation is in the manufacturing, service, or public sectors, by analysing processes, involving people, and making improvements, then problems can be addressed and savings made (bsi, 1992). quality costs in software development we have already noted that evidence (smith, 1991) suggests that in the us software quality costs may reach 30 per cent of turnover. given this, possibly alarmist, figure, it is appropriate now to apply the previous analysis to the software development process. measuring software quality there are many aspects of the software development process that can be measured (fenton, 1991) including cost, size, complexity and quality. irrespective of which of these aspects of software development is being measured there are several issues related to how measurement can be applied. fenton (op. cit.) distinguishes between measuring for prediction, for example, estimating project costs during planning, and measuring for assessment, for example, establishing the actual cost of the project. fenton and ashley (1993) make a further distinction between: resource measures which relate to the resources used as inputs to the development process such as cost or effort, process measures which relate to the actual software development process itself such as the potential for trapping/avoiding errors, product measureswhich relate to the deliverables produced as output by the development process. a third aspect is the distinction between internal and external attributes of a resource/process/product. internal attributes are those which can be measured objectively in terms of the resource/process/product itself, and are generally directly measurable. external attributes are those which can only be measured in situ in terms of how the resource/process/product relates to the environment in which it is to be used. fenton (op.cit.) argues that the latter cannot be measured directly, and therefore require some form of substitute measure. these three characteristics of software metrics need to be borne in mind whatever aspect of the software development process is being considered and not just in the case of software quality. as far as the customer is concerned, it is arguable that the product measures of software quality are paramount as these immediately relate 68 ajis vol. 6 no. 2 may 1999 to the usefulness (or otherwise!) of the product or system in question. in the longer run, however, both resource and process measures of software quality will reflect on the individual supplier's efficiency and effectiveness. in terms of the characteristics of measures given above, quality costs have a well-identified position. they are primarily an assessment metric, aimed at measuring process quality. measurement is done indirectly, by measuring the cost of achieving a certain specified level of quality, rather than trying to measure the quality of the product directly, for example, the number of errors per thousand lines of code. it should be clear from this that quality costs are not suitable as product metrics. the information that a piece of software has a quality cost of £x, or that quality costs were a given proportion of its total development costs, tells nothing about the actual quality of the software product itself. accordingly, in order to establish that a piece of software has achieved a specified level of quality, it is necessary to use more well established software product metrics. quality costs would then be a measure of the effectiveness of the techniques and tools used to achieve that quality; in other words, the development process itself. another characteristic of a metric is the scope of the activity covered by it (pyramid consortium, 1991). by their very nature quality costs can be used as a global measure (or indicator) of the effectiveness of the development process, or as a phase specific metric focused on one particular area. indeed, the potential scope of a quality cost programme is so broad that it could be regarded as a form of business metric, by reflecting the effect quality has on the commercial success of the organisation as a whole, not just the development process. the paf model applying the paf model to software engineering gives examples of costs that may be included in each category (shown in table 1). these are not intended to be definitive but to act as a guide to likely costs. it may be possible to challenge some of the categorisations such as implementation costs, but this reflects the difficulty which can arise with the categorisation process. the important thing is that the categorisation should be consistent within the quality cost programme and suit the needs of the organisation. the process model table 2 identifies possible costs when applying the process model of quality costs to the software development process. this shows the generic sources of cost which can be applied to all the processes in the software development life cycle. the obvious advantage of this approach is that it is consistent with the process view of system development seen in project planning and control, making allocation of costs much easier than with the paf model. this table has been based upon a simplified top-level process model for software development using the idef process-based system modelling technology.21 21part 1 of the british standard (bsi.1992) is based on the tqm process modelling approach using a variation on the structured analysis and design technique developed by softech inc. in the early 1970s (marca and mcgowan, 1987). sadt activity modelling was subsequently revised by softech inc. and published as idefo, as part of the us air force's icam programme (colquhoun, 1993). idefo is a top-down, hierarchical process-based system modelling methodology originally designed for modelling the processes involved in aircraft manufacture. the basic notation is quite simple, with boxes showing activities (or processes) and arrows showing the different data constraints on those activities. these constraints are split into controls, inputs, outputs and mechanisms. controls represent inputs which determine what the activity is to do, for example "test plan" which would determine what testing would take place. mechanisms represent the resources which can be applied to the activity in order to perform it, i.e. how the activity is actually physically done (softech inc., 1980). 69 ajis vol. 6 no. 2 may 1999 table 1. possible quality costs for paf model of software development prevention costs cost of defining standards, e.g. coding, documentation, testing cost of initial project planning activities cost of progress monitoring and control cost of configuration management activities cost of quality-related training, both generic and project specific cost of quality improvement programmes, including data collection appraisal costs cost of design reviews cost of system modelling, including prototypes cost of test planning, execution, and documentation up to and including acceptance cost of checking delivery media cost of managing/maintaining test environments/tools/equipment cost of documentation reviews internal failure costs cost of problem identification during design and development cost of problem reporting and initiation of corrective action cost of rework of specifications/design cost of code rewrites cost of necessary rework of previous development phases, e.g. requirements analysis cost of reworking project plans cost of appraisal of rework cost of additional administration due to rework, e.g. configuration management opportunity costs, e.g. losses due to delays in implementation cost of development system down-time/performance problems external failure costs excessive implementation costs unplanned customer support costs costs of post-implementation problem identification costs of problem reporting and related change control costs of corrective maintenance, including field updates liability, warranty and concession costs intangible costs, e.g. loss of goodwill notes to table one 1 this includes any training which will have an impact on the quality of the finished software product, e.g. training in specific methodologies, technologies, or quality assurance procedures. 2 this includes the cost of the quality cost programme itself. 3 assuming that the prototype is not intended to be part of the final delivered product, but is only to be used a model with which to test requirements or design. 4 if during subsequent development stages it is found necessary to re-analyse pan of the problem, possibly as the result of errors found at acceptance testing, then this becomes a quality cost. 5 this would be the case for a software house or similar supplier of software. for inhouse it departments such delays may be regarded as an external failure, as the business as a whole (their customer) will suffer the loss of opportunity. 70 ajis vol. 6 no. 2 may 1999 table 2. possible quality costs for process model of software development for all processes: costs of conformance: cost of designing/defining the process as part of any wider initiative cost of carrying out the process without error, including all planning and checking internal to the process costs of non-conformance: cost of problem identification cost of problem reporting and initiation of corrective action cost of in-process reworkcost of rework caused by this process but carried out in another cost of appraisal of rework cost of additional administration due to rework, e.g. configuration management cost of unavailability of necessary equipment or resources e.g. system downtime. opportunity costs, e.g. losses due to knock-on effect of delays in this process quality cost programmes there are several potential applications for quality cost programmes in software development environments. the scope of application can obviously be varied to suit the individual needs of an organisation, but it is useful to think on three levels: department/company-wide programmes which encompass more than just the development process; development process programmes which encompass the whole of the software development process; phase-specific programmes which are focused on specific phases of the software development life cycle. broad scope programmes are aimed at overall organisational improvement, with the software development process as part of a wider programme. this approach is particularly relevant to organisations such as software houses and consultancies, where software development is a significant part of their business. narrower (project) scope programmes are geared more towards development process improvement, rather than organisational improvement. there are a number of ways in which quality costs can be applied at this level. the first is process improvement, using quality costs to analyse the performance of development projects over a period of time. if development is carried out to a consistent methodology, then by monitoring the level of quality costs for each part of the methodology the programme can highlight possible weaknesses in the methodology which can then be targeted for improvement. if, on the other hand, development is not carried out in a consistent way, by comparing the levels of quality costs across projects the programme can be used to identify which methods are most effective. one benefit of quality costs in this context is that by comparing quality costs as a proportion of total costs, it is possible to allow for the effects of different development technologies. so rather than trying to establish if one defect per thousand lines of cobol is the equivalent of one defect per form of a 4gl in order to make a reasonable comparison of quality, the issue would be which approach has higher relative quality-related costs. the second potential application is as a means of justifying investment in new techniques or technology, such as the use of automated testing tools or the developing of a software quality management system, by estimating or measuring the effect they have on the level of quality costs. setting up a quality cost programme there are a number of issues to consider concerning the setting up of a quality cost programme. these are outlined in bs 6143 part 2 (bsi.1990) and in feigenbaum's total quality control' (1991). further guidance is given by evans and lindsay (1993), who reproduce a summary of the twelve steps recommended by the u.s. national association of accountants (now the institute of management accountants). most guidance material is based on the paf model, as this has the longer history of application. however, many of the points are equally applicable to the use of the process model. 71 ajis vol. 6 no. 2 may 1999 the following points are based on the guidance given in the quality costing literature. these are also consistent with the guidance given for establishing other types of software metrics programmes, as set out by the pyramid project (pyramid consortium,1991), fenton (1991) and the work of ami (application of metrics in industry). however, it is most important to recognise the second dimension that a software quality costs programme introduces. in addition to the technical aspects of quality with which practitioners will be familiar, any quality costs programme introduces financial and commercial aspects with which practitioners may well be much less familiar. accordingly the principal differences between a quality costs programme and a conventional software metrics programme arise out of the wider scope of those involved; for example, accountants are likely to be included where they would not be in a conventional software metrics programme. setting objectives & scope as with any management initiative, it is important to establish the objective(s) of a quality costs programme before it starts. this helps to ensure that the programme will actually provide meaningful information to support the decisions the organisation wants to make. there is a wide variety of potential reasons for setting up a quality costs programme including identifying high cost problems; setting cost reduction targets, and measuring progress against them; justifying the implementation of a specific quality management/assurance measure; measuring the performance of projects; or some combination of these, either as a general business parameter, or as part of a specific quality improvement programme. no matter what the other requirements of an organisation may be, one objective that should always be included if the quality cost programme is to succeed is that it should provide a basis for action. the measurement of quality costs should always be used as a basis for action. in other words, any quality costs programme is likely to have the dual purposes of measurement of costs (providing information) and the identification of means of reducing those costs. after identifying which quality costs are most important, it is essential that their value is measured. this will permit cost reduction targets to be set and progress against them measured. there is currently a lack of information available here and issues are being reassessed and readdressed although pursgrove and dale (1996) offer an example which, whilst dealing with pvc coating, provides many parallels with other organisations. linked to the establishment of objectives is the need to clarify the scope of the programme in terms of what parts of the organisation's activities are to be covered. it is important that this is defined clearly in order to avoid relevant costs being omitted, or irrelevant costs being included. it is also a useful check on the objectives, which should be capable of being met within the scope; for example, if the objectives include being able to compare project teams but the scope only includes one team, the whole exercise becomes a waste of time and effort. the narrower the scope, the more likely it is that costs will simply be moved outside the scope by any 'improvement' programme, rather than actually being reduced. the wider the scope, the more difficult it is to hide costs. on the other hand, if the programme attempts too much, it actually becomes very difficult to achieve anything. justifying the implementation of a specific quality management or quality assurance measure can, in itself, lead to the identification of other areas which should be investigated. useful and meaningful results from measuring the performance of projects, products or services really only occur when the measurement parameters have been carefully chosen, methodically monitored and analysed. defining reporting procedures a number of difficulties arise in reporting quality costs and perhaps the key issue is "what" to report to the users of the information. this will depend to a large extent on the reasons for establishing the programme, as these dictate the target audience for quality cost data. there are a number of issues which must be borne in mind irrespective of how the data is used, the most significant of which include: the need for regular reports as part of overall management reporting; the need to keep quality costs in context; the way to report quality costs as part of the accounts for a project, or business. it is useful that reports detailing quality costs are presented in a similar manner to other management accounts and the findings should be supported using ratios and/or trend analysis. these reports can then be used within the 72 ajis vol. 6 no. 2 may 1999 context of overall cost reports. to facilitate trend analysis a number of comparative bases can be used to represent the changes occurring over time. examples include: labour base such as internal failure costs related to total labour or effort cost base such as total failure costs related to total development costs sales base such as total quality cost related to net sales turnover these are based on the use of quality costs over a broad corporate scope. the comparative bases when the scope is project or phase specific are more likely to be those of total project development cost or effort. the need for regular reporting also presents problems in situations where the processes are not continuous and this is likely to be the case in software development. quality cost reporting can be integrated with project control reporting within each project, but at a corporate level the issues become more difficult, as the characteristics of low quality may be different for different phases of the development life cycle. the other important decision is the reporting format. this will obviously depend on the use to which the information will be put. some common formats from manufacturing include: by paf category; as a process cost report; as a quality cost balancing tool for the qa/qm department; by management level, based on responsibility; by key non-conformances. hollocker (1989) suggests that, for software, reporting by project and by calendar period are suitable additional formats. given that a quality cost programme should be linked to improving quality, it is helpful if costs are tabulated in such a way that they provide information to the users and provide useful 'benchmarks' for the continuous improvement effort. if areas of management concern relating to business performance are to be monitored using the quality costs programme, critical areas must be identified, cost comparisons made and costs ranked (usually in order of importance). therefore, the reporting format should facilitate this. bs 6143 part 2 (bsi.1990) includes examples of report forms and quality cost/accounting ratios. collecting quality cost data cost data varies from company to company. the required data may be not be readily available, or it may not be in the required form. the first question has to be what to collect? just as with other metrics, it is important not to get bogged down in collecting too many different quality cost elements to start with. it is more important to collect data on a small number of the most significant relevant cost areas, on a regular and consistent basis. the more focused the objectives of the programme, the easier this will be. once the initial programme is established, the quality costs can be refined by adding more elements, or splitting existing cost elements to give more detail. when establishing what to collect, there are a number of steps which can be taken. step one calculates the costs directly attributable to the 'quality' function, such as the quality manager's salary. step two identities costs that are not directly the responsibility of the 'quality' function but which should be counted as part of the total qualityrelated cost of the organisation; these include the cost of testing. in terms of the paf model, these two steps cover the identification of prevention and appraisal costs. step three identifies and enters into a memorandum account the internal costs of budgeted failures', such as brooks' "plan to throw one away" (1975). step four identifies the internal costs of failures not allowed for in step three, such as all reworking. step five identifies the cost of failures after change of ownership; in the software development context these would include field fixes. in terms of the paf model, these last three steps cover failure costs, both internal and external once it has been decided what costs are to be collected, it is necessary to establish who will collect them. this will depend very much on an organisation's own preferences. it may be left to a centralised unit, possibly the accounts department, or it may become the remit of the systems or project managers, as part of their project monitoring activities. finally, it is necessary to decide how to collect the quality costs. although sources vary from company to company, most organizations have some sort of source documents which could be used to provide information. examples include payroll analysis, project progress reports, rework or rectification authorizations or reports, field failure reports and replacement and warranty costs reports. in addition, activities may already be categorised and/or costed as part of other software metrics programmes, and this information will be documented somewhere. the two most obvious mechanisms for collecting quality costs are existing software quality metrics programmes, and timesheets, which then feed into a costing or monitoring system, where they can be combined with other costs such as capital expenditure. obviously, the needs and requirements of the collection system 73 ajis vol. 6 no. 2 may 1999 will vary with the company, and with the development methods used on individual projects, but there are some general issues. taking software metrics first, there can be a number of drawbacks to using these for quality cost information. these tend not to include measures of appraisal and prevention activities, and they tend to be simple counts of, for example, errors per thousand lines of code, not of the consequences of those errors. thus, it is likely that even with an existing metrics programme, it will not provide data suitable for use as quality cost data. however, it will provide information on the numbers of errors, and thus provide a basis for following up the cost of correcting these errors. on the other hand, timesheets could be invaluable, as all that is required is some means of splitting out quality cost elements from normal process costs. this will then provide data on the most significant cost item labour. the main difficulty with timesheets is accuracy, and this would need to be addressed, depending on the level of accuracy required. it is dangerous, and unnecessary, to strive for absolute accuracy as this tends to lead to very bureaucratic time recording which is of doubtful veracity. most inaccuracy is unintentional, but if there are concerns that timesheets may be deliberately completed inaccurately to hide quality costs, these can be cross checked against results of reviews and tests on a sampling basis. for example, if a test is failed there should be some visible effort expended in fixing the problem. an important point is that identifying and monitoring appropriate quality-related costs, and the collecting and classifying of these costs should be consistent with accounting/costing practices within the organisation. the people responsible for analysing the costs should be given any necessary assistance or input from others (such as technical specialists), to assist with the cost allocation exercise. however, care is required when it comes to using the costing system which is fed by such data. generally speaking, it is a fallacy that it is easy to extract information from an existing costing system, as accounting systems are not set up to give quality cost information. if a costing system is already in place, it is likely to be concerned only with total project phase costs, with normal process costs and quality costs lumped together. therefore, it may be advisable to avoid the temptation to modify existing costing systems to extract quality cost data, as it is often easier to start afresh. conclusion this paper has shown that quality costs can provide an important second dimension to the measurement of quality in the software development process. it presents a picture of where quality costs fit into the overall area of measuring software quality, and highlights the major issues concerned with trying to put them into practice. given the relative lack of literature on quality costs in software development, especially empirical work, it is difficult to be precise about how the experience of manufacturing will translate into software development. however, given that the general trend in software development is towards an engineering view of the processes involved, and the fact that previous moves towards improving the management of software development (such as project planning and control) have come from the same area, it is likely that quality costing will become a serious issue particularly for the larger software houses in the near future. references advisory council for applied research and development (1982) command paper 8621: facing international competition. hmso, london ashley, n. (1993) measurement as a management tool in: management and measurement of software quality m. kelly (ed.) pp31-49, avebury technical, aldershot, hants. asqc (1967) quality costs what and how asqc, milwaukee, wisconsin asqc (1977) guide for reducing quality costs asqc, milwaukee, wisconsin british standards institution (1987) bs 4778: part 1. iso 8402. quality concepts and related definitions. bsi, milton keynes british standards institution (1990) bs 6143: part 2. guide to the economics of quality prevention, appraisal and failure model. bsi, milton keynes british standards institution (1991) bs 4778: part 2. quality concepts and related definitions. bsi, milton keynes british standards institution (1991) bs 5750:part 13: iso 9000-3. guide to the application of bs5750:part 1 to the development, supply and maintenance of software. bsi, milton keynes british standards institution (1992) bs 6143: part 1. guide to the economics of quality process cost model. bsi, milton keynes brooks, p.p. jr. (1975) the mythical man-month addison-wesley, reading, ma 74 ajis vol. 6 no. 2 may 1999 campanella jack, ed. (1990) principles of quality costs, second edition. asqc quality press, milwaukee, wisconsin colquhoun g l, baines r w and crossley r (1993) "a state of the art review of idefo", international journal of computer integrated manufacturing, 6(4), 252-264 cooper, robin and kaplan, robert s. (1991) the design of cost management systems. prentice-hall international, new jersey crockett, h g (1935) "quality, but just enough", factory management and maintenance, 93(6) 245-246 crosby, philip b. (1979) quality is free. mcgraw-hill, new york dale, barrie g. and plunkett, james j. (1991) quality costing chapman & hall, london. dale, barrie g. and plunkett, james j. (1992) the case for costing quality. uk department of trade and industry, london dale, barrie g. and plunkett, james j. (1995) quality costing 2nd edition, chapman & hall, london. dpcp (1978) a national strategy for quality, department of prices and consumer protection, london evans, james r. and lindsay, william m. (1993) the management and control of quality, second edition. west feigenbaum, armand v. (1956) "total quality control" harvard business review 34(6) 93-101 feigenbaum, armand v. (1991) total quality control: 3rd edition revised. mcgraw-hill, new york fenton, n.e. (1991) software metrics. a rigorous approach chapman & hall, london hollocker, charles p (1989) "finding the cost of software quality." in: quality costs: ideas & applications, volume 2. a collection of papers. campanella, j. (ed), pp 355-367, asqc quality press, milwaukee, wisconsin hwang and e.m. aspinall "quality cost models and their applications: a review." total quality management vol 7 no3 june 1996 pp267-282 johnson, thomas h. and kaplan, robert s. (1987) relevance lost: the rise and fall of management accounting. harvard business school press, boston, ma juran, j.m. (1945) management of inspection and quality control. harper & brothers publishers, new york juran, j.m. (1951) quality control handbook mcgraw-hill, new york juran, j.m. and gryna, frank m (eds) (1988) juran's quality control handbook, fourth edition. mcgrawhill, new york keogh, w. (1994) "total quality management the role of the professional in determining quality costs." managerial auditing journal 9(4) pp 23-32 w.keogh, p.brown and s.mcgoldrick "a pilot study of quality costs at sun microsystems." total quality management vol 7 no 1 jan 1996 pp29-38 marca d a and mcgowan c l (1987) sadt structured analysis and design technique mcgraw-hill, new york masser, w j (1957) "the quality manager and quality costs." industrial quality control 14(6) 5-7. morse, wayne j. (1993) "a handle on quality costs." cma magazine, feb. 21-24.nedc (1985) quality and value for money. national economic development council, london porter, leslie j. and rayner, paul (1992) "quality costing for total quality management." international journal of production economics 27 69-81. praxis study team (1992) quantum. a measurement-based framework for the assurance of software quality department of trade and industry, london pursgrove and b.g. dale "the influence of management information and quality management systems on the development of quality costing." total quality management vol 7 no4 aug 1996 pp421-432 pyramid consortium (1991) quantitative management: get a grip on software! pyramid consortium smith, a. (1991) "quality an integrated approach" in: software quality and reliability. tools and methods, ince, d. (ed.) pp 16-23 chapman & hall, london softech inc. (1980) architect's manual. icam definition method "idefo" dr-80-atpc-01 computer aided manufacturing-international inc., arlington, texas warner, sir frederick (1977) standards in specifications in the engineering industry. nedo, london winchell, william o. (ed) (1987) guide for managing supplier quality costs. asqc quality press, milwaukee, wisconsin 75 paper title (paper title style) australasian journal of information systems truong, nkhoma & pansuwong 2019, vol 23, research article integrated effectiveness framework for in-app advertising 1 an integrated effectiveness framework of mobile in-app advertising vinh truong rmit university vietnam s3694806@rmit.edu.vn mathews nkhoma rmit university vietnam wanniwat pansuwong rmit university vietnam abstract advertising in mobile apps has recently become one of the most popular advertising channels for businesses when its annual revenue has rapidly increased year over year. on this kind of advertising, the app publishers do not only play a critical role in the ad serving process but also receive a significant portion of the advertising revenue. their goal of maximizing the revenue sometimes contradict with those of the advertisers. this research conceptualises the role of publishers and proposes an integrated effectiveness framework to further improve the effectiveness of mobile in-app advertising not for one but all participants involved. in specific, this research explores the factors being controlled by publishers and evaluates their impact on the common outcome metric of mobile in-app advertising. an application of the proposed effectiveness framework might help to increase the global mobile in-app advertising revenue significantly higher by balancing the benefits of all participants. keywords: mobile advertising; mobile apps; effectiveness 1 introduction mobile display advertising communicates with its audiences via a mobile device in the forms of mobile web display, in-app display, and search (haghirian & inoue 2007). it is predicted that the revenue of these types will continue to grow in the future, especially the in-app display one. according to emarketer (2015), the global spending on mobile in-app advertising was almost 30 billion usd, three times that of mobile web advertising. that accordingly accounts for 37% of the total online advertising spending worldwide (iab 2017). apparently, mobile inapp advertising has become one of the most effective marketing channels for businesses. in the second decade of the millennium, businesses are creating and running in-app advertising campaigns to improve advertising effectiveness in terms of brand awareness, customer attitudes and purchase intention (barwise & strong 2002) and to increase online conversion, customer engagement and advocacy (brakenhoff & spruit 2017; ghose & todri 2015). businesses can either run their advertising campaigns through guaranteed contracts or more popularly through an unguaranteed real-time bidding process (choi et al. 2017). the real-time bidding (rtb) process is created and maintained by the interactive advertising bureau (iab), who defined and maintained the specifications for this ad serving process. there australasian journal of information systems truong, nkhoma & pansuwong 2019, vol 23, research article integrated effectiveness framework for in-app advertising 2 are two main parties that are involved in an ad serving process: the advertiser who provides the ads and the publisher who offers the ad spaces in their mobile applications. between the advertiser and the publisher, there are two service parties: the bidding service and the auction service (see appendix a). when a consumer visits the website, an auction takes place in milliseconds and the winner can make the impression (perlich et al. 2012). the bids are highly automated and informed by data and models – that is the reason why rtb-based advertising is sometimes called programmatic advertising. the intermediaries got paid from the advertisers per the numbers of impressions and click supplied and pay back the publishers by the numbers of impressions and clicks delivered (kumar, sa 2016). in 2017, more than 62% of the total display advertising revenues were priced on the basis of the number of clicks (iab 2017). a better ad click performance, therefore, brings the benefits not only to the advertisers but also to the publishers. with the help of ad networks like google ads, facebook audience network, and twitter mopub, the advertisers nowadays have many options to improve the effectiveness of their advertising campaigns by using contextual and personalized targeting (andrews 2017; broder et al. 2007; yan et al. 2009), needless to mention the traditional use of ad designs as controlled by the advertisers themselves. basically, in practice, there are three types of targeting related to ad characteristics, personal information and context (chen, p-t & hsieh 2011; de pelsmacker, geuens & anckaert 2002). on the theoretical side, the interactive advertising model (iam) proposed by rodgers and thorson (2000) categorized all the factors affecting the interactive advertising effectiveness into either advertiser or consumer-controlled (see appendix b). the online behavioral advertising (oba) framework recently proposed by boerman, kruikemeier and zuiderveen borgesius (2017) extended the iam to include more factors but basically are controlled by advertiser or consumer only (see appendix c). the mobile advertising effective framework (maef) by grewal, bart, spann and zubcsek (2016) is built around the advertiser goals and categorize factors affecting the outcome metric into ad elements, context, consumer, market, and firm (see appendix d), extending more factors that are controlled by ad networks. despite the seeming utility of iam, oba and maef, they all basically includes only factors related to consumers, advertisers, ad networks and built around the goals of advertisers – the demand side of an ad serving process (brakenhoff & spruit 2017; grewal, bart, spann & pal zubcsek 2016; rodgers & thorson 2000). on the unexplored supply side, the publishers still have their own control on supplying ad spaces (brakenhoff & spruit 2017; hao, guo & easley 2017) and delivering ad impressions (choi et al. 2017; ha 2008). the fact is that 20-30% of the global mobile in-app advertising spending, aka around 10 billion usd annually, is paid to publishers (aimonetti 2012). the publishers definitely have their own goal of maximizing their revenue, which sometimes contradicts with the advertiser goal (adler, gibbons & matias 2002; choi et al. 2017; korula, mirrokni & nazerzadeh 2016; kumar, subodha, jacob & sriskandarajah 2006). however, surprisingly the studies on the publisher role are limited and there are not that many options available for publishers, even though they are one of the key participants involved in that money flow (see appendix d). on one hand, educational materials related to mobile advertising are surprisingly scarce (nittala 2011; okazaki 2012). on another hand, there are continuing difficulties in determining and optimizing the advertisements’ effectiveness (iab 2017). this lack of academic attention and these practical difficulties is not surprising, given the inherent technical and organizational complexity of designing a meaningful field experiment with mobile ads and it required a close collaboration australasian journal of information systems truong, nkhoma & pansuwong 2019, vol 23, research article integrated effectiveness framework for in-app advertising 3 with practitioners who can provide greater access to relevant data, such as earned traffic via apps (grewal, bart, spann & zubcsek 2016). there is a need to determine the factors controlled by the publishers in specific and the necessity of building an integrated effectiveness framework for all participants involved in the mobile in-app advertising in general. that framework must be built around a common goal of all participants and included publishers-controlled factors which are missed out in previous frameworks. the present study will address the following research questions: • what factors are controlled by app publishers and their impacts? • what is the common goal of all participants involved in mobile in-app advertising and what is the outcome metric to measure that goal? what framework can combine all the participants’ factors in the process to enhance that outcome metric? what are the main and interactive impacts of those factors on the common outcome metric of mobile inapp advertising? the purpose of the present study was to fill the gap in mobile in-app advertising studies in relation to an integrated effectiveness framework and the publishers-controlled factors, particularly their relationship with consumer, advertiser, and network-controlled ones. empirically, the present study attempted to evaluate their main and interactive impacts on a common outcome metric first before testing the proposed integrated effectiveness framework on a larger scale. four main independent variables were studied in the present research: • ad space duration: refers to the duration of ad spaces supplied by publishers • ad space size: refers to the size of ad spaces supplied by publishers • ad space position: refers to the position of ad spaces delivered by the publishers • ad space timing: refers to the timing of ad spaces delivered by the publishers the dependent variable of the present study is the click-through rate, which was found in this study as the common metric which can measure the common goal of direct response interactively. besides the four publishers-controlled variables, the present study also examines these following non-publisher-controlled ones: • location: refers to the receiver contextual location where the ads are served and controlled by the ad networks (grewal, bart, spann & zubcsek 2016) • time: refers to the receiver contextual time when the ads are served and controlled by the ad networks (grewal, bart, spann & zubcsek 2016) • ad type: refers to the media type on which the ads are served and controlled by the advertisers (rodgers & thorson 2000) • ad medium: refers to the medium/app on which of ads are served and controlled by the consumers (boerman, kruikemeier & zuiderveen borgesius 2017) to answer the first research question, academic literature relating to mobile in-app advertising processes and factors are systematically reviewed. firstly, it reviewed ones in mobile in-app australasian journal of information systems truong, nkhoma & pansuwong 2019, vol 23, research article integrated effectiveness framework for in-app advertising 4 advertising, mobile display advertising and then web advertising in general. the factors when found will be grouped by their participants. for the first research question, the present study attempted to investigate the descriptive and explanatory relationships between publishercontrolled factors and the effectiveness of mobile in-app advertising. thus, it was found suitable to pursue the hypothetical-quantitative approach. it started by testing hypotheses deduced from theoretical issues subject to empirical scrutiny via the use of data collected from mobile applications. in specific, the identified publishers-controlled factors will be individually evaluated using the a/b testing – the most popular testing methodology on the internet (kohavi et al. 2009). to answer the second research question, academic literature relating to the mobile in-app advertising goals and outcome metrics will be systematically reviewed to determine what the common goal is and what metric to measure that common goal. any discussion of advertising effectiveness necessarily must involve the role of objectives in advertising management contexts (li, h & leckenby 2004). with reference to the previous effective frameworks, the present study will propose an integrated effectiveness framework based on that common goal. to test the framework, a fractional 28xn experimental design is used. multivariate tests via statistical packages for social sciences (spss), rstudio and excel data analysis toolpak, were employed for further data analyses. the multi-factorial experimental design is common in the statistical literature (cox & reid 2000), where each parameter (factor) can be experimented on independently and when each experimental value for a parameter overlaps with every other experiment value for all of the other parameters. the contributions of the present study can be presented from three perspectives, i.e. theoretical, empirical and practical, which are described below. theoretically, the present study proposed a newly integrated effectiveness framework which extends previous effectiveness framework to include new constructs and relationships. empirically, the present study set up a new way of designing multiple ad spaces in a single app. by doing so, multiple factors can be tested interactively and concurrently. that helps to save the time of doing multiple nested a/b tests. practically, the present study suggests newly integrated advertising strategies associated with publishers to further enhance the ad click performance of mobile inapp advertising. by which, newly integrated advertising strategies could be recommended to be applied in practice and could help increasing mobile in-app advertising revenue significantly higher by balancing the benefits of all participants involved. 2 background the present study is grounded in web advertising, mobile advertising and mobile display advertising. 2.1 web advertising there are various definitions of advertising. richards and curran (2002) suggested that advertising is defined as a communication designed to entice an audience to take some action, either instantly or in the near future. the advertisement is usually paid from an identifiable source and can be conveyed through print, tv, web and other methods. advertising on the web became mainstream in early 1994 when a banner ad was displayed on the website of at&t (briggs & hollis 1997; lohtia, donthu & hershberger 2003; robinson, wysocka & hand 2007). during the preliminary period when fewer than one-third of u.s. australasian journal of information systems truong, nkhoma & pansuwong 2019, vol 23, research article integrated effectiveness framework for in-app advertising 5 households had computers, and fewer than half of those machines had internet connections, only traditional methods of advertising were used (thorson & schumann 1999). at that time, banner ads were served as links to the online versions of newspapers, business directories or other direct services. by then, banner ads were one of the most prevalent online display ad formats (hoffman & novak 2000; mangani 2004). banner ads are considered as display ads (edizel, mantrach & bai 2017). they are a form of graphical ads embedded into a webpage and commonly used as a combination of static/animated images or text and/or video. these are designed to convey an advertising/marketing message and/or encourage users to take action. banner ad dimensions are generally defined by width and height, represented in pixels. by 1996, the banner ads on the web bring back a revenue of $267 million (internet advertising revenue report 1996-2016). since 1997, web advertisers have become more mature. they started focusing on targeting web advertisements. instead of links, they are more concerned with location and personalization (thorson & schumann 1999). to overcome the geographic limitation of web advertising at that time, the websites asked their users to register the zip codes. tailoring advertisements and purchase experiences to individual tastes, on the basis of people’s needs, interests, and preferences are called as personalization (chellappa & sin 2005). personalization is an antecedent of online service quality (wolfinbarger & gilly 2003). mokbel and levandoski (2009) assert that advertising contents should be personalized or matched to users’ preferences and profile. web display advertising could be in the forms of banners, skyscrapers, and rich media in order to achieve brand awareness and encourage click through to a target website. in one of the early studies of banner advertisement effects, briggs and hollis (1997) found that banner advertisements resulted in heightened awareness, brand perceptions and attitudinal shifts for brands. by 1998, another form of ads was created and become popular. those are interstitial ads. ads of this type will appear while users wait for screens to download. by then, the advertisers have learnt to maximize their advertising revenue by taking advantage of loading time. the year 1998 is also the year when the history of contextual advertising began. oingo, inc., a privately held company started by gilad elbaz and adam weissman, developed a proprietary search algorithm based on word meanings and built upon an underlying lexicon called wordnet. google acquired oingo in april 2003 and renamed the system adsense (karp 2008). later, yahoo! publish network, microsoft adcenter and advertising.com sponsored listings amongst others were created to offer similar services (kenny & marshall 2001). until then, the contextual advertising platforms have evolved to adapt to a richer media environment, such as video, audio and mobile networks with geographical information. these platforms allowed publishers to sell ad spaces on their web pages, video clips and mobile applications to make money. usually, such services are called an advertising network or a display network that are not necessarily run by search engines and can consist of a huge number of individual publishers and advertisers. a critical question that many website owners face at that time is how many ads to place. in the short-term, increasing the real-estate given to ads can increase revenue, but what will it do to the user experience, especially if these are non-targeted ads? the tradeoff between increased revenue and the degradation of the end-user experience is a tough one to assess, and that is exactly the question that the msn home page team at microsoft faced in 2007 (kohavi et al. australasian journal of information systems truong, nkhoma & pansuwong 2019, vol 23, research article integrated effectiveness framework for in-app advertising 6 2009). by then, web publishers have started thinking of maximizing revenue through their ad spaces while balancing it with user experience. around 2007, new platforms focusing on real-time buying and selling impressions were created, like adsdaq, adecn, doubleclick advertising exchange, adbrite, and right media exchange, that is now known as ad exchanges (graham 2011). unlike traditional ad networks, these ad exchanges aggregate multiple ad networks together to balance the demand and supply in marketplaces (muthukrishnan 2009). individual publishers and advertising networks can benefit from participating in such businesses. on one hand, publishers sell impressions to advertisers who are interested in associated user profiles and context. advertisers, on the other hand, could also get in touch with more publishers for better matching. during this time, other similar platforms emerged: the demand-side platform (dsp) and supply-side platform (ssp). these services help individual ad networks exchange their ad inventories in real time. due to that, the kind of advertising is sometimes called programmatic advertising (laudon & traver 2018). the emergence of ad exchanges is a result of the fact that there are thousands of ad networks available on the internet, which can act as a barrier for advertisers as well as publishers when getting into the online advertising business. advertisers have to create and maintain campaigns frequently for better coverage and analyse data across many platforms for a better impact. publishers have to register with and compare several ad networks carefully to achieve optimal revenue. the ad exchange came as an aggregated marketplace of multiple ad networks to help alleviate such problems. advertisers can create their campaigns and set desired to target only once and analyse the performance data stream in a single place, and publishers can register with ad exchanges and collect the optimal profit without any manual interference (meyer et al. 2018). the advertising marketplaces are becoming more mature with the emergence of dsp, ssp and the revenue comes higher (steel 2011). with the invention of search engines in the later years, noticeably google, web advertising revenue went even higher (evans 2009). with those search engines, the advertisers have more access to the users’ interest and behaviour, which in turn helped improving ad targeting and personalization (ha 2008). since then, the advertising campaigns become more interactive (park, shenoy & salvendy 2008). during this time, websites have started making use of an array of personally identifiable information (pii) about users whilst they are accessing the website, derived from cookies, flash cookies, web beacons, browser and other meta-data. this pii is used to profile users and so deliver relevant, targeted ads to them, which has been shown to be effective in the study by yan et al. (2009) who were able to provide empirical evidence of improvement using behavioural targeting. even in the absence of cookies and other tracking files, it is possible to perform user profiling based on browser data alone. a higher number of studies on the contextual factors is recorded in this time, e.g. adler, gibbons and matias (2002), menon et al. (2011), nakamura and abe (2005) and kumar, subodha, jacob and sriskandarajah (2006). as technology has advanced, practitioners have become more concerned about increasing the effectiveness of employing modern technology like contextual analysis and using geographic location (idwan et al. 2008). the popularity of web advertising continued to grow in the twenty-first century with the help of social media (such as facebook, twitter, and linkedin). by 2013, around $5.1 billion was already spent on social advertising. in web advertising, the top two forms are paid-search and display advertising. in paid search advertising, advertisers pay an advertising fee, usually based on ad views or click-throughs, australasian journal of information systems truong, nkhoma & pansuwong 2019, vol 23, research article integrated effectiveness framework for in-app advertising 7 to have their websites shown in top placement on search engine result pages. in web display advertising, advertisers reach out to a target internet audience via some form of a visual advertisement such as display banner ads (including mobile), flash-based rich media, or digital video (aksakallı 2012). the web advertising practice is basically based on traditional advertising theories (okazaki 2012). firstly, it is based on uses and gratifications theory, which were developed by katz, blumler and gurevitch (1973). the driving question of uses and gratifications is why do people use media and what do they use them for. the uses and gratifications theory discusses how users deliberately choose media that will satisfy given needs and allow one to enhance knowledge, relaxation, social interactions, diversion or escape. the users are normally active in their selection of content and messages from the mass media. in advertising, the uses and gratification theory implies that the consumer is not passive, helpless advertising fodder (hedges, ford-hutchinson & stewart-hunter 1997). instead, advertisers and consumers factors interactively affect the outcome of advertising and they should be consumer-based, not vice versa. another popular advertising theory is the theory of reasoned action developed by fishbein and ajzen (1975). the theory is used to explain the relationship between attitudes and behaviours within human action (the a-b relationship). it predicts how individuals will behave based on their pre-existing attitudes and behavioural intentions. based on the theory of reasoned action, ducoffe (1996) developed a well-defined advertising value model. it is a framework for predicting attitudes toward web advertising. the technology acceptance model was later developed by davis (1985). the model suggests that the user’ attitude to the information system will directly affect the user’s intention, while the perceived usefulness and perceived ease of use will affect the users’ attitude towards information system, including clicks on advertisements. okazaki and barwise (2011) claimed that, in the field of web advertising, technology acceptance model is the most frequently used model. in terms of revenue, by 2013, the total revenue was already at $43 billion, including both web display advertising and mobile advertising (internet advertising revenue report 1996-2016). 2.2 mobile advertising mobile advertising is related closely to web advertising but with a far greater reach. mobile advertising is referred to as “advertising or marketing messages delivered to portable devices, either via a synchronized download or wirelessly over the air” (laszlo 2009). the differences between fixed (e.g. pc, web) and portable devices create some new opportunities for marketers, but they also prevent them from easily generalizing the research findings from fixed to mobile online settings. typical online (but fixed) advertising strategies are either not applicable or at least demand modification to be effective (shankar & balasubramanian 2009). mobile advertising, therefore, is considered as a new type of advertising and requires new academic research and practical strategies. ads on mobile devices have a long history especially after short message service (sms) became popular (yunos, gao & shim 2003). in the early days, mobile advertising is usually considered as sms advertising (barwise & strong 2002; haghirian & inoue 2007). actually, the dawn of mobile advertising research begins with two seminal sms related works: barnes (2002) and barwise and strong (2002). the former is conceptual and the latter empirical (okazaki 2012). since 2007, the year that the first iphone is introduced, the number of people accessing the internet via a portable device, such as a smartphone or tablet, has surpassed the number of australasian journal of information systems truong, nkhoma & pansuwong 2019, vol 23, research article integrated effectiveness framework for in-app advertising 8 people using fixed devices, such as pcs. these rapid shifts have been echoed by advertisers, who have shifted their expenditures. the mobile advertising captured 22% of u.s. digital advertising expenses in 2013 and that number keeps increasing year by year (emarketer 2015). by 2014, 66% of facebook advertising revenue came from mobile ads, while that number on twitter is already 85% (bergen 2014). the success of mobile advertising is due to the fact that mobile devices are considered very personal, as extensions of users’ selves (shu & peck 2011). the increase in smartphone usage, in the sense that it provides marketers with an opportunity for electronic presence sense that this provides marketers with an opportunity for electronic presence (lemon & verhoef 2016; varnali & toker 2010). the popularity of iphone and android devices bring a new form of mobile advertising through mobile web and mobile apps. basically, the mobile ads in the forms of sms messages and as display ads on mobile web and apps are different. according to barnes (2002), they are two types of mobile advertising: push and pull. push advertising involves or 'pushing' advertising messages to consumers, usually via an alert or sms text message. pull advertising involves placing advertisements on browsed wireless content, promoting free content. on mobile web and apps, messages are transferred by the free will of the audience, which is considered as a pull-type mobile display advertising (park, shenoy & salvendy 2008). between the two types of mobile advertising, in theory, the research trends have been heavily skewed toward push advertising (okazaki 2012). empirical field research in mobile advertising mainly investigates the effectiveness of mobile coupons delivered via sms, which are of push-type (grewal, bart, spann & zubcsek 2016) leaving the pull-type mobile display advertising largely unexplored (korula, mirrokni & nazerzadeh 2016). 2.3 mobile display advertising although largely unattended in academic literature, the revenue of pull-type display advertising keeps increasing. the trend in online publishing shows that online news outlets increasingly rely on advertising to generate revenue, as most readers refuse to pay subscription fees to read online news (newman et al. 2016). in 2017, mobile display advertising already generated nearly 70 billion usd (statista 2018). as smartphones penetrate markets worldwide and applications on smartphones become more popular, a new type of form of mobile display advertising, called mobile in-app advertising, has become more popular (hirose, mineo & tabe 2017). smartphone users spend much more time on their apps while spending less time on mobile internet access through web browsers. the increase in popularity of smartphones has led to the growing need of developing smartphone applications (apps) that are gradually replacing the traditional use of internet services (gupta, khirbat & singh 2014). as of jun. 2016, android and iphone users were able to choose respectively between 2.2 million and 2 million mobile apps (gupta, khirbat & singh 2014). the users are ever more motivated using mobile apps and it was announced that more than 195 billion mobile apps have already been downloaded together from app store and android market (cumulative number of apps downloaded from the apple app store 2018). mobile in-app advertising has several advantages over mobile web advertising. firstly, in-app advertising is less clutter than websites advertising because smartphone users access the internet directly through apps. once users download an app and like it, they tend to use it frequently. secondly, advertisers can easily select their advertising media. because most apps have a specific purpose. thirdly, with mobile apps, advertisers can develop highly australasian journal of information systems truong, nkhoma & pansuwong 2019, vol 23, research article integrated effectiveness framework for in-app advertising 9 personalized advertising. compared to mobile website advertising, in-app advertising can be related to personal information collected with a global positioning system (gps) (hirose, mineo & tabe 2017). rosenkrans and myers (2012) study explored the effectiveness of the technology acceptance model (tam) and the uses and gratifications theory by comparing mobile banner ads to non-mobile banner ads on a local newspaper’s website. findings from their research indicated that mobile banner ads had significantly higher click-through rates than the non-mobile banner ads. another difference between mobile in-app advertising and mobile web advertising is the role of app publisher. it is observed that more apps adopt a pure ad strategy, that is, free apps with ads. that could be due to: consumer learning of the valuation of the app (niculescu & wu 2014); many developers adopt a freemium model where the app is made free to facilitate consumer referrals (lee, c, kumar & gupta 2013); some developers offer free trials to reduce consumers’ uncertainty about the functionalities of their products and leverage the network effect among the users (cheng, li & liu 2015; cheng & liu 2012) and some developers allow users to pay to remove ads in the free version (lin, m, ke & whinston 2012). in mobile in-app advertising, publishers have more control than any other type of advertising. for example, unlike traditional newspaper and television firms, which have control over their ad publishing channels, a mobile platform owner has to induce mobile app developers to publish ads in their apps in order for the ads to reach any app users. if a developer chooses to publish in-app ads, the platform owner shares a portion of the advertising revenue with the app developer based on a revenue-sharing contract (hao, guo & easley 2017). furthermore, compared to traditional advertising (e.g., newspaper), in-app advertising has an important difference that for in-app advertising, the platform has a split structure with the platform owner and the app developer jointly provision both the app and the ads (hao, guo & easley 2017). in april 2010, when apple’s ad network, iad, was initially launched, the percentage of ad revenue that iad passed to the developer was 60%. in 2012, apple decided to boost the developer’s ad-revenue-sharing percentage from 60% to 70%, reducing its own ad revenue percentage in favour of the app developer’s (aimonetti 2012). the new money flow which involves the app publishers is shown in appendix b. according to emarketer (2015), the global spending on mobile in-app advertising was almost 30 billion usd, three times that of mobile web advertising. that accordingly accounts for 37% of the total online advertising revenue worldwide (iab 2017). apparently, mobile in-app advertising has become one of the most effective marketing channels for businesses and have become a significant revenue source for publishers. 3 mobile in-app advertising the present study is related to literature in four areas of mobile in-app advertising: processes, participants, outcome metrics and factors. any discussion of advertising effectiveness necessarily must firstly involve the role of objectives in advertising management contexts. objectives also serve the function of providing a means by which results can be evaluated. and importantly, objectives force those involved to gain a deeper understanding of the processes underlying their particular problems. reasonable advertising objectives cannot be set without knowledge about how the advertising process works. (li, h & leckenby 2004). a keyword search of the most important electronic databases in digital advertising and communication science (i.e., proquest, web of science, australasian journal of information systems truong, nkhoma & pansuwong 2019, vol 23, research article integrated effectiveness framework for in-app advertising 10 abi/inform complete, taylor & francis, sciencedirect, acm, ieee, psychology database is performed. the keywords used were “in-app advertising processes”, “in-app advertising participants”, “in-app advertising outcome metrics” and “in-app advertising factors”. the search period covered all manuscripts available by the end of may 2019. after identifying a study, its references will be examined to find further studies. in addition, experts in the field are also contacted to inquire about other relevant manuscripts. studies that reported on empirical data will be included while nonempirical studies, such as legal studies are left out. in total, 192 manuscripts fit the criteria, and the earliest study was published in 2008. among these, 154 were from academic journals, 16 from conference proceedings, 18 was a book chapter, and 4 were published online. 3.1 mobile in-app advertising processes on mobile apps, the users consume the advertising content by clicking or tapping on the ads. before a mobile ad, or impression, is served in a mobile application, an ad serving process is performed. there are two ways that ads can be supplied to a mobile application: through guaranteed contracts or through an unguaranteed real-time bidding process (korula, mirrokni & nazerzadeh 2016). in the guaranteed contract setting, the publisher supplies the ad spaces and deliver the ad impressions strictly following the contract commitments. that guaranteed process normally involves only one publisher and one advertiser (yuan, wang & zhao 2013). the contracts agreed by advertisers and publishers, or between ad networks, are long term and of large volume. yuan et al. (2012) called that kind of private contracts as overthe-counter (otc). on the other hand, in the unguaranteed setting, between the advertiser and the publisher, there are two additional service parties that monitor the ad serving process: the bidding service and the auction service (balseiro & candogan 2017; choi et al. 2017). interactive advertising bureau is an organization who creates and maintains the standards and specifications for realtime bidding (rtb) process in order to improve the ad network landscape (see appendix a). if advertising opportunities are traded in unguaranteed and transparent markets, the trading units are usually small, although the total number of impressions for the advertising campaign could be huge (yuan et al. 2012). in rtb, the advertiser is not directly involved in the auction but is outsourcing the bidding process to a service on the demand side platform (dsp). businesses on the dsp compete in auctions to obtain a unique impression. the decisions made regarding the participation in the auction and its result is totally automatic and ambiguous. in rtb, the job is almost impossible to complete manually due to the high volume and speed of incoming requests for placing bids for every impression, which requires analysing the context, user profile and other data (yuan et al. 2012). therefore, automated systems are employed in rtb platforms, enabling advertisers to give precise bids very quickly with the help of machine learning algorithms. on the other side of this ad serving process, the publisher is the one who supplies the ad spaces. a supply-side platform (ssp) will carry out an auction among bidders for the impression. the dsp works as the agency of advertisers by bidding and tracking in selected ad networks, the ssp works as the agency of publishers by selling impressions and selecting optimal bids. when the bidding is completed, the publisher delivers the ad impressions to the user via an ad space. an ad exchange/ad network is normally an advertising service providing the rtb mechanism that enables advertisers to promote their products to targeted groups of users. the ad network/exchange acts as auctioneer, selling keywords to advertisers. besides that, an ad australasian journal of information systems truong, nkhoma & pansuwong 2019, vol 23, research article integrated effectiveness framework for in-app advertising 11 exchange manages contract negotiation between advertisers and content publishers that wish to sell ad spaces. usually, the publishers will employ the brokering services of an advertising platform such as google’s doubleclick, aol’s advertising.com and microsoft media network. for large publishers, normal practice is to sell only remnant inventory through an ad exchange, with the other inventory being negotiated directly with advertisers. smaller publishers normally sell all of their advertising spaces through ad exchanges (yuan et al. 2012). in both guaranteed and unguaranteed ad serving processes, the publisher is the one who supplies ad spaces and delivers ad impressions via their ad spaces. because of that, any mobile in-app advertising ad serving process can be broken down to ad space supply process and ad space delivery one. 3.1.1 ad space supply process if an app developer chooses to publish ads, he allocates space in the app for displaying ads. the allocated in-app ad space can then be filled by the platform owner, with a certain probability, upon receiving an ad request from the app (hao, guo & easley 2017). brakenhoff and spruit (2017) highlighted how publishers control the ad space supply in a standard ad serving process as shown in appendix a. the ad space supply process is triggered when a user opens an application and the ad spaces are loaded. when design their apps the publisher will reserve some spaces to display advertisements, which are called ad spaces. supplying ad spaces with different characteristics are something that the publisher can control by themselves. an ad is only considered as an impression when the design characteristics, specified by the advertiser, meet the specifications of the request from the publisher and it is only possible that a certain design is shown to a user if the design fits the available ad space (brakenhoff and spruit, 2017). ad space is a physical object, and like any physical object, the ad space can be measured by both spatial and temporal dimensions. information like height, width, and duration of ad spaces can be sent for auctioning via an ad exchange/ad network (edizel, mantrach & bai 2017). the present study argued that by controlling the ad space supply process, the publishers can actively influence and determine the effectiveness of mobile in-app advertising. 3.1.2 ad space delivery process besides supplying ad spaces for bidding, the publisher is also the one who controls the delivery of ad impressions. brakenhoff and spruit (2017) illustrated the publishers-controlled delivery process in a standard ad serving process as shown in appendix a. referring to that process after the ads have been selected by the advertiser and the ad network, the publisher will have full control on how to deliver them to the user. the publishers can control how to position the ads on their applications and how to schedule them. interactive advertising bureau recommends ad positions to be top or bottom of the screen and sometimes in the middle of page sections. they also recommend ad scheduling to be before, in between or after the primary content experience ('iab new standard ad unit portfolio’ 2017; rastogi et al. 2016). the present study argued that by controlling the ad space supply process, the publishers can actively influence and determine the effectiveness of mobile in-app advertising. 3.2 mobile in-app advertising participants in mobile in-app adverting, there are four main participants involved including advertisers, ad networks/ad exchanges, publishers and users (barwise & strong 2002). advertisers are those who design the advertisements and initiated the advertising campaigns. next, they will australasian journal of information systems truong, nkhoma & pansuwong 2019, vol 23, research article integrated effectiveness framework for in-app advertising 12 use ad networks/ad exchanges to bid and distribute the advertisements with the publishers. the publishers will then place those advertisements on their apps for impressing. in the end, the users are the consumers of the advertisers’ brands or products (lin, ttc et al. 2015). these four players have different goals in mind when involving advertising. firstly, the users want to receive advertisements with their permissions, personalized and relevant (barwise & strong 2002). in a study conducted by cho (2003), it was found that users who were interested in the site and shown advertisements for products and services related to those on the site achieved a higher click-through rate and that those more involved in a product were more likely to click through. jansen and spink (2007) reported from their experiments that people will retain their behaviour of clicking on organic or sponsored links even when these two types of result are mixed. this leads to a belief that from the user’s perspective, the only thing that matters is the relevance of ads. actually, kumar, v and gupta (2016) claimed that customers’ expectations for the relevant messaging are evolving; they seek customized communication designed to fulfil their specific needs. as long as the advertising message is relevant to either the content that is being viewed by the consumer or his or her consumption goals, the effectiveness of an advertisement is not adversely impacted by multimedia usage (angell et al. 2016). argued that users also clicked on ads that they found trusted, personalized and relevant. secondly, the advertisers when running a campaign aimed to achieve two main goals of branding and direct response (barwise & strong 2002; kumar, sa 2016; park, shenoy & salvendy 2008; zhu & wilbur 2011): • branding: brand awareness, attitude, purchase intention • direct response: engagement, conversion, advocacy on the other hand, ad networks/exchanges naturally try to find the best match for their ad inventories. the best match, overall, is not limited to the ‘relevance’ from the traditional informational retrieval research sense, but also includes the best revenue from the economic perspective (yuan et al. 2012). it is to maximize the revenue based on the probability that an ad will be clicked as well as the relevance of the ads to the users (richardson, dominowska & ragno 2007). as an ad exchange, revenue optimization is among the most important tasks in order to run a business. this is especially true for search engine companies, for example, google made 96% of its revenue from its advertising business (alphabet 2018). lastly, when the publishers involved with advertising, they are more concerned with the direct response (choi et al. 2017). that is how they could earn revenue. in order to maximize revenue from the guaranteed contracts, the publisher’s allocation and inventory management have to be efficient (feige et al. 2008; roels & fridgeirsdottir 2009). in general, the publishers can earn money based on the number of impressions and the number of clicks on their ad spaces. participant goal user personalized (impressions), relevance (clicks) advertiser branding (impressions), direct response (clicks) ad networks/exchanges best match (click-throughs) publisher revenue (impressions, clicks) table 1: participants and their goals australasian journal of information systems truong, nkhoma & pansuwong 2019, vol 23, research article integrated effectiveness framework for in-app advertising 13 table 1 shows the participants and their goals. out of the four participants, the goals of users and advertisers are most studied in the current literature (boerman, kruikemeier & zuiderveen borgesius 2017). for example, rodgers and thorson (2000) categorized all the factors affecting the interactive advertising effectiveness into either advertiser or consumercontrolled in their interactive advertising model (iam). the interactive advertising model (iam) has been widely referenced by scholars from various disciplines around the world since it is first proposed in 2000 (rodgers, ouyang & thorson 2017). the iam offers an integrated way to evaluate advertising effects from both consumer-controlled and advertiser-controlled viewpoints. according to iam, the consumer controls the internet motives, mode and information processes, while the advertiser controls the ad types, ad formats and ad features (see appendix b). recently, grewal, bart, spann and zubcsek (2016) proposed mobile advertising effectiveness framework (maef) to enhance the advertising effectiveness for advertisers using advertisers’ ad elements, ad networks’ context and users’ consumer factors, but not publishers. most of the online/interactive advertising frameworks are built around the advertiser and consumer goals but not the publisher’s one. noted on the missing role of publishers, choi et al. (2017) provided a comprehensive view of the display ad ecosystem, including both guaranteed and non-guaranteed selling channels and suggested publishers find the benefits by balancing the ad allocations between these two channels. the study highlights the publisher goal of maximizing revenues through clicks. yuan, wang and zhao (2013) showed a money flow, on which advertisers are those who pay for their advertising contents to be shown on publishers’ ad spaces. they pay that to the ad networks, like google ads, facebook audience network. those ad networks, in turn, will pay back the publishers. in april 2010 when apple’s ad network, iad, was initially launched, the percentage of ad revenue that iad passed to the developer was 60% (apple inc 2010). two years later, in 2012, apple decided to boost the developer’s ad-revenue-sharing percentage from 60% to 70%, reducing its own ad revenue percentage in favour of the app developer’s (aimonetti 2012). it means the revenue for publishers could start from 20-30% of the total mobile in-app advertising spending of 30 billion usd (emarketer 2015). mobile in-app advertising is, therefore, also for publisher purpose (yuan et al. 2012). typically, the publishers might not need to promote brand awareness as advertisers, but they share the same direct goal, aka clicks. the direct response also is used to measure the relevance of the ads to the users in the long run (kohavi et al. 2009) and to measure the best match for the ad networks/exchanges (kumar, s 2016). increasing the matching, in the interactive context like mobile in-app advertising, therefore, is where both the publisher, the advertisers, the ad networks/exchange and the user's goals could meet. the present study argued that enhancing the matching is the common goal of all participants involved in mobile in-app advertising. 3.3 mobile in-app advertising outcome metrics as in section 3.2, the key goals of advertisers in a mobile in-app advertising context are facilitating awareness, encouraging positive attitudes, increasing engagement, increasing conversion rates, encouraging repurchases, and promoting advocacy. those advertisers, who are willing to spend on ads for brand awareness, attitude, intention purposes, will aim at achieving the viewability goals, aka. impressions. on the other hand, if the advertisers have engagement, online conversion, advocacy goals in mind, they will pay for the performance of their displayed ads, aka. clicks. the effectiveness of the selected goals is measured by outcome metrics. australasian journal of information systems truong, nkhoma & pansuwong 2019, vol 23, research article integrated effectiveness framework for in-app advertising 14 some common web metrics include: (1) page impressions, (2) ad impressions, (3) clicks, (4) visits, (5) unique visitors, (6) path analysis, (7) conversion rate, (8) frequency, and (9) recency (rosenkrans, ginger 2007). among those metrics, clicks and ad impressions, i.e., the number of times an advertisement is served to a user's browser, are the top two metrics used for advertisement delivery reporting and audience measurement (bhat, bevans & sengupta 2002). according to ('interactive audience measurement and advertising campaign reporting and audit guidelines’ 2014, september), a click is a user-initiated action of clicking on an ad element, causing a redirect to another web location. a click is when users interact within a web browser or an app by using a device (i.e., mouse, finger, or a stylus). the 'interactive audience measurement and advertising campaign reporting and audit guidelines' 2014, september) states there are three kinds of clicks: click-through, in-unit click, and mouse-over (e.g., mouse rollover, or user rolls mouse-over ad). these three actions are referred to as clicks. a click-through is when a user initiates action by clicking on an ad and the click-through directs the user to another online location, such as another browser window or web site. clickthroughs are tracked and reported by an ad server, and it is imperative that bot activity is excluded to ensure accurate and reliable metrics. the cost-per-click (cpc) model or clickthrough rate (ctr) remains one of the most important media pricing metrics for the internet. click-throughs are based on the number of clicks divided by the number of ads requested or clicked on by users during a time period (bhat, bevans & sengupta 2002). according to the structurational point of view (giddens 1986), these two sets of measures are characterized by different, but complementary views regarding the role of interactive advertising. one set of measures focuses on the number of impressions. such measures might be descriptively labelled as control process measures centralizing around advertisers’ goal. this is related to branding goals. the second set of measures focuses on the effects of consumers' using interactive media. within an interactive context measures such as awareness, attitude, and product choice, among others, are not simply the result of exposure to advertising; they are also the result of choices made by the consumer, which are, in turn, guided by the consumer's goals and purposes. these outcome measures are the joint product, or interaction, of the consumer and the advertising. in that sense, advertising both shapes and is shaped by consumer preferences (pavlou & stewart 2000). accordingly, revenue per thousand impressions (rpm) and click through rate (ctr) are the two most common metrics currently being used (hagen, robertson & sadler 2006; punyatoya 2011; rosenkrans, g. 2007; rosenkrans, ginger 2007). when the number of impressions is chosen, the cost is calculated by the cost per thousand exposure (cpm) model; and the revenue is calculated by the revenue per thousand (rpm) metric (hoffman & novak 2000). the cpm model has commonly been used for many traditional media, such as television and newspapers, and has also been adopted for online advertising. even though highly adaverse users are unlikely to respond to ads, the cpm model charges advertisers according to the level of exposure, regardless of ad effectiveness. impressions, basically, do not track whether users interacted with an ad and definitely do not measure the matching of mobile ads. on the other hand, when the number of clicks is chosen, the cost and the revenue are both calculated by performance-based models. these performance-based revenue models take into account user ad aversion and charge advertisers on the basis of user actions. for example, in cost-per-click (cpc)—a highly successful and popular performance-based model—advertisers australasian journal of information systems truong, nkhoma & pansuwong 2019, vol 23, research article integrated effectiveness framework for in-app advertising 15 pay only when their ads generate clicks. recently, performance-based ad revenues surpassed the cpm revenues in 2006 and remain increasingly dominant (lin, m, ke & whinston 2012). click-through rate (ctr) is a performance-based metric. it is the ratio of a number of times an advertisement is clicked to the number of advertisement impressions (schonberg et al. 2000). this metric is one of the oldest methods of evaluating the success of online advertising campaigns and is an appropriate tool for assessing direct marketing objectives. ctr is commonly used to evaluate the effectiveness of banner ads (kumar, s 2016). clickthroughs are behavioural and are an accountable measure for online advertising. click-through metrics are easy to observe and they indicate a behavioural response; clicks indicate an immediate interest in the advertised brand (chatterjee, hoffman & novak 2003; singh, dalal & spears 2005) (chatterjee, hoffman, & novak, 2003; lawrence 2000; singh & dalal 1999). the ctr is considered a good diagnostic metric for direct responses. the ctr is also a good metric to measure the branding goal in the long run. hollis (2005) suggests that the two paradigms, i.e., branding and direct response, are not contradictory but they are in fact complementary and that the applicability of either model depends not only on the intent of the advertiser but also the mindset of the audience. it can thus be argued that the two objectives are not mutually exclusive, yet, at a conceptual level, how the advertising budget should be prioritized is usually a matter of debate. kohavi et al. (2009) argue that longterm goals should be part of the direct response. for example, when ads are plastered over a page, they hurt the user experience, so a good direct response should include a penalty term of usage of real-estate for ads that are not clicked, and/or should directly measure repeat visits and abandonment. similarly, delayed conversion metrics should already count for a lag from the time a user is exposed to something and take action. miller, s (2006) and quarto-von tivadar (2006) called them latent conversions (miller 2006; quarto-von tivadar 2006). according to kohavi et al. (2009), coming up with a good metric is hard, but what is the alternative? the key point here is to recognize this limitation, but avoid throwing the baby out with the bathwater. goals demand-side (advertisers) supply-side (publishers) best match impressions cost per impression (cpm) revenue per impression (rpm) ctr (long term) clicks cost per click (cpc) revenue per click (rpc) ctr (short term) table 2: ctr as the common metric to measure advertising goals in practice, behavioural campaigns seem to be especially popular in the online arena. it is estimated that behavioural advertising accounts for a larger amount of all ad dollars spent online. in 2017, more than 62% of the total display advertising revenues were measured by ctr and 4% were measured by both ctr and cpm/rpm (iab 2017). apparently, compared with ad impression, ad click has become the most important indicator of online advertising effectiveness. improving the outcome metric ctr or the ad click performance is to enhance the advertising effectiveness for all participants as shown in table 2. a better click-through rate, directly and indirectly, means a better ad relevance (users’ goal), a better direct response (advertisers’ goal) and higher revenue (ad networks and publishers’ goal). most advertising research assumes that the effectiveness of online ads depends on their benefits for individual consumers (čaić et al. 2015). the present study argued that enhancing the click-through rate (ctr) is to enhance the effectiveness of mobile in-app advertising for all participants. australasian journal of information systems truong, nkhoma & pansuwong 2019, vol 23, research article integrated effectiveness framework for in-app advertising 16 3.4 mobile in-app advertising factors factors that affects the mobile in-app advertising as well as enhances the click-through rate of mobile in-app advertising can be categorized into three groups (chen, p-t & hsieh 2011): ad characteristics (brand, price, informativeness, entertainment, etc.), user behaviours (age, gender, interest, preference, activity history, etc.) and context (time, location, weather, technology, etc.) or three-factor components: stimuli characteristics, personal characteristics, and advertising context (de pelsmacker, geuens & anckaert 2002). grewal, bart, spann and zubcsek (2016) grouped those factors into ad elements, consumer and context factors. those three groups are correspondingly controlled to the three participants: advertisers, consumers and ad networks. the ad elements in the maef are elements that characterize the look and feel of an ad and can be referred to as design characteristics, which are controlled by the advertisers. the component ‘consumer’ contains information about the consumer. besides the information of the current state of the consumer in the ‘customer journey‘, it also contains information about the history of the consumer and possibly demographic information, which are related to consumers (brakenhoff & spruit 2017). the context component contains factors relating to the environment, such as location, time, weather, events, and technology such as the size of the screen and the location (website or app) they come from. in addition to the demographic and contextual targeting, advertisers greatly enjoy behavioural targeting in rtb (choi et al. 2017). the contextual factors are controlled by ad networks/exchanges (broder et al. 2007). 3.4.1 advertisers-controlled factors advertisers make decisions regarding ad designs and inventory characteristics (zubcsek and sarvary 2011). advertisers control the ad characteristics or ad elements. the impacts of ad elements on advertising effectiveness have been examined in many studies. for example, goh, chu and wu (2015) emphasized the importance of mobile ad content to the user response. the concept of content marketing is not a new phenomenon (pulizzi 2012). the content characteristics that need to consider include informative, persuasive, images viewed, characters viewed, depth of search, and breadth of search. in web advertising, a study from ducoffe (1996) indicated that the advertising value is a useful measurement criterion for evaluating advertising effects. an advertisement is considered as valuable if it is important or useful. the research showed that all the relevant characteristics of an advertisement like informativeness, entertainment and irritation have effects on advertising value and hence affect the attitude toward online advertising. this is once again confirmed with a recent study by le and nguyen (2014) for the vietnamese market. they examined the effect of informativeness, credibility, entertainment, and irritation characteristics of mobile advertising for vietnamese customers. kim, ky and lee (2015) built a research model to show how the entertainment and informativeness affect user perception before affecting the user intention to perform. trivedi (2015) agreed with the findings that informativeness, credibility, entertainment have positive effects on mobile advertising. irritation, on the other hand, has a negative effect. in this study on gen y indians, permission has a negligible effect. lin and chen (2009) considered the animated online advertising and found out that the ad types and animation lengths are significantly related to advertising performance. readers evaluate advertisements more positively when they consider the advertisement as useful, fulfilling their needs for information and/or entertainment (van reijmersdal, neijens & smit 2005). when the information matches readers’ interests, readers respond more positive towards the advertisement, regardless of whether they recognize the persuasion attempt (sweetser et al. australasian journal of information systems truong, nkhoma & pansuwong 2019, vol 23, research article integrated effectiveness framework for in-app advertising 17 2016). another related study by lim, tan and jnr nwonwu (2013) revealed that mobile users better recall image banner ads than text ones and more likely to perceive large image banner ad as app content. certain aspects related to the specific format of the ad can have an impact on its effectiveness such as the type of ad (grigorovici & constantin 2004). li, y-w, yang and liang (2015) found out that both website interactivity and promotional methods can enhance consumer attitudes, but price discount is effective only when the brand perception is functional but not effective when the brand image is symbolic. appendices b, c and d listed out several ad elements specified in the interactive advertising model (rodgers & thorson 2000), online behavior model (boerman, kruikemeier & zuiderveen borgesius 2017) and mobile advertising effective framework (grewal, bart, spann & zubcsek 2016) respectively. nowadays, advertisers have many options to design their ads and directly improve the performance-based effectiveness of their mobile in-app advertising campaigns. 3.4.2 consumers-controlled factors consumers control user behaviours or personal characteristics. there are also many studies on consumer-controlled factors and their impact on mobile advertising performance. for example, luo et al. (2014) demonstrated that one-day coupons are best suitable for consumers who are near the provider, while multiple day coupons performed better for consumers who are far away from the provider. the research showed that a consumer journey consists of multiple stages, and at each stage, there should be a different advertising strategy to make the advertisement more relevant and consequently improve its effectiveness. online advertisers provide behaviourally targeted advertisements, based on a user’s behaviour on the internet over time (goldfarb & tucker 2011). personalizing banner ads based on products consumers placed in their shopping carts during the shopping visit increases click-through rates (bleier & eisenbeiss 2015a). personalized ads are approximately twice as effective as nonpersonalized versions of similar ads (aguirre, m et al. 2012). the industry claims that behavioural advertising creates more relevant and efficient ads and boost ad effects (chen, j & stallaert 2010). leading scholars argue that advertising will become more personalized and targeted and will involve more individual communication, where advertisers can iterate messages based on consumer behaviour and needs (keller 2016; kumar, v & gupta 2016; rust 2016; schultz 2016). furthermore, like mackenzie, lutz and belch (1986) have shown, mobile advertising effectiveness is also mediated by consumer attitudes. based on that pioneering work, korgaonkar, petrescu and karson (2015) studied education and ethnic background factors and showed that those factors play key roles in mobile services and mobile advertising. the different demographic has different usage, useful, satisfied and expensive attitude. that is confirmed in the study of hispanic americans by kim and lee (2015). basic demographics are age, gender, income, occupation and race (haley 1968). kim and lee (2015) proposed a hybrid quantitative and qualitative model to suggest preferred ads for four different consumer segments, including business partner, skilled enthusiast, new experience seeker, and close buddy. from the research, lifestyle and psychology tendency are the attributes need to be considered in mobile advertising effectiveness. shavitt, lowrey and haefner (1998) argued that people are actually more favourable toward advertising. males, wage consumers, and persons with less education and income, and non-whites generally report more favourable advertising attitudes than others. shavitt, lowrey and haefner (1998) have shown the importance of gender, age, education, income and ethnic factors in advertising performance. zhou et al. (2017) proved that historical advertisement behaviour of a user plays a key role to predict the user advertisement behaviour in the future. conner and armitage (1998) pointed australasian journal of information systems truong, nkhoma & pansuwong 2019, vol 23, research article integrated effectiveness framework for in-app advertising 18 out that when individuals engage in behaviours that are habitual for them, they use simplified decision rules, which results in enacting the same behaviours as in the past. previous behaviours, in turn, seem to be an important predictor of future behaviour (ouellette & wood 1998). understanding the role of consumer information, some of the social platforms enable advertisers to target ads using consumers' social profiles (bakshy et al. 2012). new internet technologies provide the possibility of automated tracking of consumers' internet behaviour. such tracking is used to create user profiles for the purpose of displaying advertisements that fit the interest of these individuals (goldfarb & tucker 2011; mcdonald & cranor 2010). personalization is a customer-oriented marketing strategy that aims to deliver the right content to the right person at the right time, to maximize business opportunities (tam & ho 2006). the types of information used included age, gender, location (aguirre, e et al. 2015), education level (tucker 2014), interests (aguirre, e et al. 2015), online shopping behaviour (bleier & eisenbeiss 2015a), and search history (doorn & hoekstra 2013). their findings suggest that the level of personalization influences consumer-related factors, such as feelings of intrusiveness (doorn & hoekstra 2013), feelings of vulnerability, the ad’s perceived usefulness, reactance, and privacy concerns (bleier & eisenbeiss 2015a). the level of personalization also influences the outcomes, such as click-through rates (aguirre, e et al. 2015). appendix b and d listed the factors controlled by consumers according to interactive advertising model (rodgers & thorson 2000) and mobile advertising effective framework (grewal, bart, spann & zubcsek 2016) respectively. nowadays, advertisers have many consumer targeting options to control and improve the performance-based effectiveness of their mobile in-app advertising campaigns. 3.4.3 ad networks-controlled factors ad networks facilitate the programmatic and real-time buying and selling of advertisements (choi et al. 2017; laudon & traver 2018). programmatic advertising delivers dynamic content to users based on their location, time and so on (kumar, v & gupta 2016). in general, “context” is referred to as the editorial medium environment of an advertisement (moorman 2003). in the words of norris and colman (1993), "the same source delivering the same message to the same audience on separate occasions might produce different effects depending on the differing programming or editorial contexts in which the message appears”. instead of displaying the same ads to everyone, different ads are shown with regard to the geography, language, device and other characteristics of visitors, to maximize the utilization of advertising opportunities. “context” refers to both the physical and social environments, which can be paraphrased as “situation”. advertisers can make use of the context of user actions and their personal information in order to tailor adverts to their demographic and taste, so as to maximise the number of click-throughs received for each ad and subsequently the revenue obtained (belk 1975). moorman (2003) suggested the context be categorized as objective or subjective. objective characteristics include contextual aspects such as genre, content and style, characteristics that can easily be recognized by every receiver and are not dependent on interpretation. subjective characteristics, in contrast, are not experienced uniformly but comprise the individual mental reactions that people experience when confronted with an editorial message (moorman 2003). context can also be divided as context characteristics related to the receiver of the advertisement and those related to the vehicle carrying the advertisement: the medium australasian journal of information systems truong, nkhoma & pansuwong 2019, vol 23, research article integrated effectiveness framework for in-app advertising 19 context. the receiver context can be described as the situational circumstances in which a person is exposed to an advertisement. this includes the person's physical environment, the social environment, the time frame, and the mental state a person is in prior to exposure to the medium content (moorman 2003). the medium context concerns with the environment of the ad provided by the vehicle carrying it, such as a television program, an issue of a magazine or an internet site (pieters & raaij 1992). another distinction can be made between an editorial context and commercial context. studies on commercial context have predominantly concentrated on the effect of the amount and nature of other commercial messages in the environment of an ad, referred to as clutter and competitive clutter (moorman 2003). it has been shown that as the number of other ads in the environment of the target ad increases the effectiveness of the target add decreases, especially when the other ads are directly competitive. on the other hand, knowledge about the effect of editorial context is more valuable for media planners since they are only provided with information about the editorial context, while information about the commercial context is not given in advance. this makes it difficult to take the effect of other advertisements into account when making media planning decisions (kent 1993, 1995). on the context factors, there are many studies about their impacts on ad click performance. for example, effendi and ali (2017) used linear regression along with some dynamically added features known as keywords to improve the click-through rate prediction for contextual advertisements by serving more suitable ads to the viewers. similarly, goh, chu and wu (2015) investigated the attributes of province location, mobile service plan (pre/post paid) and indicators of last digits towards the advertising performance goals. the research guided a link between the advertising response and the information search. other factors like physical location and time of day can also influence how each consumer evaluates and responds to mobile display ads. different physical location and time of day associated with different outcomes as shown in a study by luo et al. (2014). luo et al. (2014) found out that mobile ads that match users' logical location are more effective and andrews et al. (2015) that the performance of advertisements varies with the amount of local crowding. ghose, goldfarb and han (2013) discovered the relationship in the distances between consumers' homes and the point of sale, as part of so-called location-based advertising (molitor, reichhart & spann 2012). spatial and temporal factors were detected to be very important in traditional advertising in forms of ambient advertising (karimova 2012). in the context of social networks, li, y (2014) discovered that most of twitter tweets are posted from a small portion of twitter users and there is a positive linear correlation between the radius of the city and the commute distance. twitter users are found to be active from 10:00 am to 12:00 am at midnight with a peak at 9:00 pm. twitter users are also found to have more activities during weekends than weekdays. similarly, baker, fang and luo (2014) demonstrated that the effectiveness of advertisements varies with the time of day. for that, both business-to-business marketing practitioners and researchers emphasize that customers require a business culture change from “selling” to “helping” (holliman & rowley 2014; jefferson & tanton 2015). strong content effects were found in the study by nasco and bruner (2008), such that weather information was perceived as most important, most involving, and most likely to influence future mobile usage. all of those research responded to a frequent call to pay more attention to contextual variables in assessments of communication effectiveness. grewal, bart, spann and zubcsek (2016) summarized those context factors in the mobile advertising effectiveness framework as shown in appendix d. apparently, the advertisers have many contextual australasian journal of information systems truong, nkhoma & pansuwong 2019, vol 23, research article integrated effectiveness framework for in-app advertising 20 targeting options to control and improve the performance-based effectiveness of their mobile in-app advertising campaigns. this section presented an overview of the mobile in-app advertising processes, participants, outcome metrics and factors. it is noticeable that there are four participants involved in mobile in-app advertising, while there are only three components of factors being explored in previous studies. it is also noted the lack of an integrated effectiveness framework, which is built around a common goal of all participants. the knowledge vacuum and research gaps identified within the discussion pointed to further actions in the present study. theoretical and empirical literature in regard to the publishers-controlled factors gap and the integrated effectiveness framework gap are discussed in section 4 and 5 respectively. 4 publishers-controlled factors and interactions the previous literature basically only includes factors controlled by advertisers, users and ad networks, while there are actually four participants. the role of publishers and their controlled factors are not included in previous effectiveness frameworks and have not been fully explored in the advertising effectiveness studies. due to the lack of academic materials in the field, the present study will also bring in academic literature from other types of advertising and also literature from practice. 4.1 publishers-controlled factors as shown in section 3.1, from a publisher standpoint, any mobile in-app advertising ad serving process can be broken down to ad space supply process and ad space delivery one. by supplying the ad space with predefined and relevant characteristics, the publisher could significantly enhance the effectiveness of mobile in-app advertising. the question is what are those ad space characteristics? 'iab new standard ad unit portfolio' 2017) stated that ads/ad spaces could have two characteristics: duration and size. that means publishers can control how long they want the ads last on their apps regardless of how long those ads are designed by the advertisers. they can do that by setting the duration for their ad spaces. when the publisher supplies an ad space with a predefined duration or a predefined size, only ads with those elements are selected to be provided and displayed. however, while measurement guidelines have existed for other forms of online advertising, mobility has its own special set of challenges. for instance, on tv or on websites, there are fairly standard expectations from marketers about what constitutes a "view" of an ad. on smartphones, it's a little less certain. we're not at a point yet where we have a clear and fast definition on viewability. is it where half the ad is viewable for a couple of seconds, or does it need to be the entire ad?" (schick, 2013). furthermore, advertising on an app is different than advertising on tv and radio because ads on an app are usually placed alongside the content whereas ads on tv and radio appear instead of the content. in the fields of tv and website, there are several related studies on the effectiveness of ad duration. one example is a two-part study by danaher and mullarkey (2003). the study found that higher exposure time was correlated with increased recognition and recall. in tv, increasing the duration of a television ad slightly increases the probability the ad will be remembered in an aided recall task (patzer 1991). in web advertising, baltas (2003) found that banners with lengthy messages and multiple frames (animation) received fewer clicks. he australasian journal of information systems truong, nkhoma & pansuwong 2019, vol 23, research article integrated effectiveness framework for in-app advertising 21 reasoned that these two factors increase the complexity of an ad and hence have a negative effect on the viewer’s attitude towards and response to the banner. chandon, chtourou and fortin (2003) and lohtia, donthu and hershberger (2003) concluded that animation improved click-through rates. on the other hand, burke et al. (2005) suggested that animated banners may be more difficult to remember than static ones. goldstein, mcafee and suri (2011) found that displaying two shorter ads results in more total recollection than displaying one longer ad of twice the duration. studies about the impact of ad duration on the advertising effectiveness actually deliver mixed results in the context of web advertising. some research could not point out if recollection relates to click-through rates. in most cases, current monetization strategies do not explicitly consider time as a resource to be optimized (sun, et al, 2017). all the mentioned academic and practitioner literature has pointed out that the ad space duration is not fully studied in the past but could be a factor that can strongly impact the clickthrough rate of mobile in-app advertising. this study, therefore, hypothesized that: hypothesis 1: the publishers-controlled supply factor: ad space duration, has a significantly strong impact on the click-through rate of mobile in-app advertising similar to the ad space duration, publishers can set the size of their ad spaces ('iab new standard ad unit portfolio’ 2017). when the publisher supplies an ad space with a predefined ad size, only ads with that element are selected to be provided and displayed. by supplying the ad space with predefined and relevant characteristics, the publisher could significantly enhance the click-through rate. conventional industry wisdom has maintained that large banner advertisements will receive more viewer attention as measured by clicks, supporting past research finding (marx 1996). the success of larger advertisements to secure attention also has an impact on viewer impression of brand quality. a larger advertisement can imply a higher level of advertising cost and effort that the viewer equates with a higher level of brand credibility and success. with regard to banner advertisements, where a click leads the viewer to another location, this may have a positive impact on viewers’ impressions and site expectations, resulting in increased viewer response, i.e., clicks. baltas (2003) also found that bigger ads are more effective in attracting attention and (hence more likely to) trigger response. chandon, chtourou and fortin (2003) found positive interactions on five banner sizes, although no significant difference between the two bigger sizes. the empirical results from li, h and bukovac (1999), however, showed that the click-through rates do not proportionally increase accordingly to the size. drèze and hussherr (2003) claimed the smaller ads performed just as well as the large ones, in accordance with cho (2003) who found no significant relationship between banner size and clicking. the relationship between the banner size and ctr is conflicting. the impact of ad size on mobile apps could be different than that in web advertising as well. previous research has shown the limits of consumer capacity called the limited capacity model (craik 2002; miller, ga 1956). in the mobile context, there are limitations related to the screen size. the screen size of mobile devices could be as small as apple watch, while the smartphone screen is normally a fourth that of a pc. that limitation should be taken into consideration. however, current metrics do not support the measurement of size (schick 2013). herrewijn and poels (2018) found that the effect of ad size is neglectable partly because their measurement did not take into account the size variable. australasian journal of information systems truong, nkhoma & pansuwong 2019, vol 23, research article integrated effectiveness framework for in-app advertising 22 all the mentioned academic and practitioner literature has pointed out that the ad space size is not fully studied in the past but could be a factor that can strongly impact the click-through rate of mobile in-app advertising. this study, therefore, hypothesized that: hypothesis 2: the publishers-controlled supply factor: ad space size, has a significantly strong impact on the click-through rate of mobile in-app advertising besides supplying ad spaces for bidding, the publisher is also the one who controls the delivery of ad impressions. after the ads have been selected by the advertiser and the ad network, the publisher will have full control on how to deliver them to the user. the publishers can control how to position the ads on their applications and how to schedule them. interactive advertising bureau recommends ad positions to be top or bottom of the screen and sometimes in the middle of page sections. they also recommend ad scheduling to be before, in between or after the primary content experience ('iab new standard ad unit portfolio’ 2017). the publishers-related delivery factors are shown as being critical in the ad serving process that will then enhance the click-through rate of mobile in-app advertising. there are many studies on scheduling and positioning ads on a website, pioneering by adler, gibbons and matias (2002), nakamura and abe (2005) and kumar, subodha, jacob and sriskandarajah (2006). in websites, ads can be placed on top, at the bottom, in the middle of the screen or in relation with other contents (djamasbi, hall-phillips & yang 2013). prominent brand placements (e.g. large ads, central ads) seem to be better in capturing the player’s attention, resulting in a positive effect on brand awareness (jeong & biocca 2012; lee, m & faber 2007; schneider, systems & cornwell 2005). it was found that banner advertisements at the top of the webpage were more often viewed (josephson 2004; sundar & kalyanaraman 2004). herrewijn and poels (2018) claimed that the spatial position is actually the most important placement characteristic. agarwal, hosanagar and smith (2011) evaluated the impact of ad placement on revenues and profits generated from a sponsored search. the authors measure the impact of ad placement on both click-through and conversion rates. they find that while the click-through rate decreases with the position, and contrary to the conventional wisdom in the industry, the topmost position is not necessarily the revenue or profit-maximizing position. they help correct a significant misunderstanding among advertisers regarding the value of the top position. ansari and mela (2003) find that the higher position of links in an email campaign can lead to a higher probability of clicking. johnson, ej et al. (2004) find that consumers searched fewer than two stores during a typical search session. similarly, brynjolfsson, dick and smith (2010) find that only 9% of shopbot users select offers beyond the first page. in general, because of the cognitive costs associated with evaluating alternatives, consumers often focus on a small set of results (montgomery, hosanagar & clay 2004). those research have shown the importance of position in search advertising. does that apply to mobile app advertising as well? until we could find out the answer, in the meantime, many publishers simply display some banner ads and never consider how effective the placement of those ads is (oak 2008). the question of optimizing mobile advertising placements remains open (grewal, bart, spann & zubcsek 2016). that led to the following hypothesis: hypothesis 3: the publishers-controlled delivery factor: ad space position, has a significantly strong impact on the click-through rate of mobile in-app advertising australasian journal of information systems truong, nkhoma & pansuwong 2019, vol 23, research article integrated effectiveness framework for in-app advertising 23 ads can also be scheduled to be displayed before, after or in between sessions (chatterjee, hoffman & novak 2003; kumar, s., dawande & mookerjee 2007; sun 2017). with different delivery schemes like that, the click-through rate could be significantly different. in online search advertising, hoque and lohse (1999) found that consumers are more likely to choose advertisements near the beginning of an online directory than they are when using paper directories. weingarten and berger (2017) studied how temporal location—that is, whether events or experiences occurred in the past, present or future—affects word of mouth. bleier and eisenbeiss (2015b) highlighted the importance of the interplay of what, when and where, aka scheduling. the first seconds of exposure caused a steep increase in the memory for an ad, and further exposure time had a smaller, albeit still increasing the impact on recollection. unlike broadcast networks, mobile app publishers are able to observe the traffic to their websites and therefore can dynamically plan a strategy to deliver impressions (roels & fridgeirsdottir 2009). nakamura and abe (2005) developed an lp-based algorithm to schedule banner ads, where they presented three features that each ad was associated with; time of day that the ads were preferred to be viewed (e.g. the afternoon), page category (e.g. sports) and the number of impressions. the features were then used to determine the optimal ad time and location that maximises overall revenue, rather than relying solely on the ctr of an individual ad. their strategy showed an improvement over greedy and random methods. trope and liberman (2003) found out that the temporal distance to objects or events alters the way in which they are processed. sahni (2015) conducted a field experiment on a restaurant search website. the key result of their work is that increasing the time between exposures, up to two weeks, increases the probability of a purchasing event. do that imply a direct response also? goldstein, mcafee and suri (2015) claimed that there is no guidance to publishers as to how ads should be scheduled. (goldstein, mcafee & suri 2015). king (2017) recently called publishers to take back control of the inventory and to remind that timing is just as important as audience targeting. in line with these findings, the present study hypothesized that: hypothesis 4: the publishers-controlled delivery factor: ad space timing, has a significantly strong impact on the click-through rate of mobile in-app advertising 4.2 publishers-controlled interactions consumers control the consumer factors, advertisers control the ad elements while the ad networks control contextual factors. relatively, little effort has been apportioned toward assessing the interrelationship among these various inputs or their efficacy with respect to cost (johnson, g & lewis 2015), although there are few studies on the interactions among themselves. for example, another study by zorn et al. (2012) showed that different websites have different users. users on one social networking site, myspace.no, preferred animated ads but users of the other social networking site, ebuddy, preferred static ads. for surf sites, animated ads worked significantly better than static ads. yet, myspace.no accounted for 96% of all impressions on surf sites and english static ads worked best for the second surf site, ebuddy.no. ctrs for static ads and animated ads had an insignificant difference in search sites. the study by lin, y and lin (2006) showed the effects of ad types and users' gender on the click-through rate of web advertisement. if an online consumer is motivated to use the internet to shop, banner ads that match this motive will presumably be more persuasive than banner ads that do not according to a study by rodgers and sheldon (2002). that study together with others has shown the interaction between consumers, advertisers and ad networks-controlled factors. how do those factors interact with the publishers-related ones? australasian journal of information systems truong, nkhoma & pansuwong 2019, vol 23, research article integrated effectiveness framework for in-app advertising 24 firstly, in the maef, there are ten context factors listed out, including location, time, weather, events, economic conditions, devices, delivery mechanism availability, owned or 3rd party, another screen presence. about location, goh, chu and wu (2015) further categorized it as area, city, and country. goh, chu and wu (2015) investigated the attributes of province location, mobile service plan (pre/post paid) and indicators of last digits towards the advertising performance goals. the research guided a link between the advertising response and the information search. luo et al. (2014) found out that mobile ads that match users' logical location are more effective. as the location is confirmed as a contextual factor that has a significant strong impact on the effectiveness of online advertising in previous studies, the present study hypothesized that: hypothesis 5: the impact of location on the click-through rate of mobile in-app advertising is strongly moderated by the publishers-controlled factors similarly, the time has been considered as an important factor that could affect mobile advertising effectiveness. for example, li, y (2014) discovered that most of twitter tweets are posted from 10:00 am to 12:00 am at midnight with a peak at 9:00 pm. the twitter users are also found to have more activities during weekends than weekdays (li, y 2014). all of the research showed the different level of activities over weekdays. similarly, baker, fang and luo (2014) demonstrated that the effectiveness of advertisements varies with the time of day. different time of day associated with different outcomes as shown in a study by luo et al. (2014). not only the time of day, but the day of the week is also considered as an important factor. according to 'digital marketing trends in the education market' 2018) for their clients, the optimal days to send email to continue to be during the business week on tuesday, wednesday, and thursday for both the k-12 and higher ed markets. open rates for the k-12 market were highest for emails delivered on thursdays, while open rates for the higher ed market were highest on wednesdays. similarly, tuesday and friday are the best days when maximum india internet users open and click on the email communications sent to them ('annual state of email marketing in india’ 2015). in their report, tuesday is the day when they see maximum engagement rates for email open rates. as time is confirmed as a contextual factor that has a significant strong impact on the effectiveness of online advertising in previous studies, the present study hypothesized that: hypothesis 6: the impact of time on the click-through rate of mobile in-app advertising is strongly moderated by the publishers-controlled factors grewal, bart, spann and zubcsek (2016) identified six ad elements: ad medium, medium type, push/pull, interactive/static, promotional elements. the medium type is the channel through which the ad is served to a consumer. the medium types can be a webpage or a mobile application. the content of the webpage or application can influence the perception of an ad (grewal, bart, spann & zubcsek 2016). the medium type can also be the platform (e.g. ios and android) on which the app is running. as these mobile platforms have very different characteristics, it could be assumed that ads displaying on different platforms generate different click-through rate. given users differ in their motivation to access the web such as information seekers and entertainment seekers, website users might react differently to advertisements on news and entertainment sites (san josé-cabezudo, gutiérrez-cillán & gutiérrez-arranz 2008). similar to banner elements, the website type relates to user involvement. based on motives to use the web, web users search a website, scrutinizing core information to achieve search satisfaction or surf a website in an experiential mode, skimming australasian journal of information systems truong, nkhoma & pansuwong 2019, vol 23, research article integrated effectiveness framework for in-app advertising 25 core and peripheral information (muylle, moenaert & despontin 1999). a study by zorn et al. (2012) showed that different websites have different users. users on one social networking site, myspace.no, preferred animated ads but users of the other social networking site, ebuddy, preferred static ads. for surf sites, animated ads worked significantly better than static ads. yet, myspace.no accounted for 96% of all impressions on surf sites and english static ads worked best for the second surf site, ebuddy.no. ctrs for static ads and animated ads had an insignificant difference in search sites. advertisements placed on different apps could have different results as shown in the study of brakenhoff and spruit (2017). as app design is confirmed as a factor that has a significant strong impact on the effectiveness of online advertising in previous studies, the present study hypothesized that: hypothesis 7: the impact of ad medium on the click-through rate of mobile in-app advertising is strongly moderated by the publishers-controlled factors grewal, bart, spann and zubcsek (2016) distinguished four different ad types: static, dynamic, interactive, and ads containing a video. in the maef the creative attributes are indicated as interactive/static. the type of creative in an ad can be important regarding how inviting the ad is for interaction. according to edizel, mantrach and bai (2017), many advertisers have begun to implement animated banners to deliver a progressive and sequential image. it is well known that television is one of the most intrusive, involving media forms because of its ability to use moving images. when banners use animation, they also take on the character of television advertisements, and this may suggest that animated banner advertisements will attract more attention and hence be clicked more (wegert 2002). studies of the side-by-side performance of advertisements for different companies conducted by acnielsen suggest that animation increases click rate (briggs 2001). based on their experiment, li, h and bukovac (1999) illustrate that animation increases response times and recall of banner advertisements. ad interactivity has a significant impact on effectiveness. however, most of the ads today are still static, and there are static text and static image. lim, tan and jnr nwonwu (2013) revealed that mobile users better recall static image banner ads than static text ones and more likely to perceive large image banner ad as app content. as ad type has been confirmed as a factor that has a significant strong impact on the effectiveness of online advertising in previous studies, this study hypothesized that: hypothesis 8: the impact of ad type on the click-through rate of mobile in-app advertising is strongly moderated by the publishers-controlled factors in this section, four publishers-controlled factors were identified. their main and interactive effects will be evaluated accordingly in the present study. those constructs and relationships will be part of an integrated effective framework, which will be presented in section 5. 5 an integrated effectiveness framework modelling the factors into effectiveness framework has quite a long history, starting with the original ideas of elmo st. lewis in 1898 when he proposed a systematic way of discussing criteria of effectiveness. he did so in the personal selling context with his “attention, interest, desire and action” or aida model (barry 1987). later, lavidge and steiner (1961) postulated a “hierarchy of effects” in a stair-step fashion with attention leading to interest, to conviction, to desire and finally to action. importantly, they also categorized these elements into the three broader categories of “cognition,” “affection,” and “conation.” “conation” was then a popular term for behavior. many hierarchy-of-effects models have been proposed for australasian journal of information systems truong, nkhoma & pansuwong 2019, vol 23, research article integrated effectiveness framework for in-app advertising 26 advertising effectiveness. one of the first models are the defining advertising goals for measured advertising results (dagmar). the model assumes that advertising works in the sequence of awareness, comprehension, conviction and action (scholten 1996). the process was also thought to be largely inevitable. that is, if the first was established, others would follow in their natural order—with some help along the way, of course, by the advertiser (li, h & leckenby 2004). hierarchy of effect model basically categorizing elements into three categories: cognition, affection and behaviour. however, the internet introduces an additional dimension – alienation (subject’s estrangement from its community, society. so, the internet is an active media and moving consumer to the role of an active receiver from being a passive receiver (davidavičienė 2012). rodgers and thorson (2000) proposed the interactive advertising model in 2000. according to the authors, one of the most basic ways to think about how individuals process advertisements in an interactive environment are to distinguish between aspects of the internet that are consumer-controlled and those that are advertiser-controlled. traditionally speaking, advertisers have controlled which ads consumers see, when and how. of course, consumers always have the option of not paying attention to, becoming involved with or ignoring the ad. in the case of the internet, however, the control has switched from advertiser to consumer. in fact, a number of researchers and practitioners argue that consumers have more control over the internet than do advertisers (roehm & haugtvedt 1999). some have gone so far as to argue that interactive marketing and advertising techniques will not work unless practitioners "step into the shoes" of an approach the internet from the consumer's vantage point. rodgers and thorson (2000) went further to classify customers and advertisers controlled characteristics. most of these advertiser controlled variables include structural elements, such as ad types, formats and feature. the model also includes consumer controlled variables such as attitude toward online advertising and attitude toward websites. according to iam, the internet motives, the inner drive to carry out the internet activity, can explain why individuals use the internet. four categories of reasons were identified as the primary motives for entering cyberspace including researching, communicating, surfing, and shopping. mode, the extent of a user’s goal-directedness of internet activities, conjointly determines the level of ad processing with motive, as internet motive will influence the mode in which users use the internet. individuals are also expected to experience several stages of information processing of online ads: attend to, remember, and develop attitudes toward internet ads, as well as actions taken in response to internet ads (boerman, kruikemeier & zuiderveen borgesius 2017). the iam argued that information processing of online ads would be influenced by the presentation of the interactive ad, as well as characteristics of the stimulus environment. ad type represents the general structure of an advertisement and was classified into five main categories: product/service, psa, issue, corporate, and political. the ad format is the manner in which the online ad appears. several popular interactive ad formats were examined using the iam model: banners, interstitials (pop-ups), sponsorships, hyperlinks, and websites. the iam provides a comprehensive list of two subjective ad features, structures based on consumer responses (e.g., “attitude towards the website” and “interest”), and objective ad features (e.g., colour, size, or typeface) across print, broadcast, and the internet. the online behavioral advertising (oba) framework recently proposed by boerman, kruikemeier and zuiderveen borgesius (2017). it extended the iam to include more advertisers and consumercontrolled factors. by leaving out the ad networks, the framework actually grouped some factors to the other two participants. for example, the oba categorizes “level of australasian journal of information systems truong, nkhoma & pansuwong 2019, vol 23, research article integrated effectiveness framework for in-app advertising 27 personalization” to advertisers-controlled. rodgers, ouyang and thorson (2017) claimed that after 15 years the iam needs to be updated. maef designed by grewal, bart, spann and zubcsek (2016) extended and covered a broader picture with more factors including the contextual ones. it is a framework that maps the components involved in “the creation and targeting of an advertisement”. the components are context, consumer, ad goal, market, firm, ad elements, and outcome metrics. out of those components, the consumer components group factors controlled by consumers, the ad elements group those controlled by advertisers, the context factors controlled by ad networks/exchanges. maef emphasized the context component when the authors repeatedly called for more research on the interaction of context factors. the integrated effectiveness framework that this study proposed is built around the common goal of all participants and includes factor components that previously identified in other effective frameworks. two new components of factors have been introduced with the framework are supply and delivery. the framework is the answer to the present study to the second research question regarding what framework can combine the publisher goal and other participants’ ones. in this framework (figure 1), the common outcome metric is the clickthrough rate, which measures the short and long term goals of all participants. the framework is structured to how ads are served to start from consumer requested until completed. it also reflects the relationships between publishers and other participants in aligning of achieving the common goal. figure 1: the integrated mobile in-app advertising effectiveness framework, which includes factor components controlled by consumers, advertisers, ad networks and publishers there are four participants recorded in the framework, and their factors are included accordingly. the dotted line denoted the “private contract” when the ad network role is absent. the consumer, advertiser and ad networks-controlled factor components comprise theoretical content derived during the literature review stage and are critical to the conceptualization of the present study. the previous theoretical and empirical literature on australasian journal of information systems truong, nkhoma & pansuwong 2019, vol 23, research article integrated effectiveness framework for in-app advertising 28 mobile in-app advertising factors were reviewed extensively. the relationship between consumer, advertiser and ad network controlled factors and the click-through rate were also examined. their theoretical content was abstracted from various sources so as to enable the researcher to isolate variables important to the study. the review of literature is presented in chapter 4. the publishers-controlled supply and delivery components comprise variable deemed specific and critical to the present study. the proposed variables formed relationships that called for further empirical testing. the relationships focused on were those relating the factors to the click-through rate upon which the key critical moderating variables were contingent. clickthrough rate refers to the significance and implications of the present study derived from the empirical analyses and results. these indicate whether or not the conceptualized relationships have been confirmed by the data and could be generalized throughout the population. 6 experiments in this section, the methodology of the present study – the reasoning of choosing methods to evaluate the framework in section 5 is discussed. then, the results of the hypothesis tests will be presented. 6.1 methodology this research is concerned with the click-through rate of mobile in-app advertising. this study follows a deductive research approach to deduce conclusions from the data collected. it first develops hypotheses. then it designs a research strategy to test those hypotheses. this research collects primary data (the frequency of ad impressions and ad clicks) through indirect observations to improve the data quality (hewson, vogel & laurent 2016). the measurement is done through the ad spaces, which are designed with different characteristics based on the four publishers-controlled factors. in order to simulate the ad space characteristics according to the four factors, it needs to access the source of apps that the ad spaces reside. the research procedure starts with (1) developing apps, publishing them to app store, (2) designing ad spaces inside those apps, (3) using ads from one ad networks to load contents to those ad spaces, (4) collecting the number of ad impression and clicks on the loaded ad spaces and (5) measuring the click-through rates. out of the eight relationships of the present study, four of them are related to simple main effects and the remaining are related to interactive ones. to test the main effects of the four publishers-controlled factors individually, the research firstly uses a controlled experimental design. a controlled experiment is the most popular form of experiment in online advertising (kohavi et al. 2009). the present study follows the simplest form of controlled experiments, often referred to as an a/b test. the four main independent variables have the following values: • ad space duration: the duration of ad space is adjustable by publishers (sandberg & rollins 2013). many ad networks allow publishers to choose the duration of their ad spaces. for example, admob allows the ad space duration up 120 seconds (prochkova, singh & nurminen 2012). it is also found that by default admob uses a refresh rate of 60 seconds (qian et al. 2012). the present study argued that ad spaces shorter than 60 seconds, and ones longer than 60 seconds have different yield different ctrs. for that reason, 30 seconds and 90 seconds are selected as values of ad space duration. australasian journal of information systems truong, nkhoma & pansuwong 2019, vol 23, research article integrated effectiveness framework for in-app advertising 29 • ad space size: the size of ad spaces is selected by publishers (‘display and mobile advertising creative format guidelines’ 2015). many ad networks allow publishers to choose the size of their ad spaces in forms of banners. for example, admob allows the ad space size to be banner (320x50), large banner (320x100) or iab full-size banner (468x60). among those banner has the smallest size while large banner is the biggest. the present study chose those two sizes are two values of ad space size. • ad space position: refers to the position of ad spaces delivered by the publishers. an ad can be on the top and bottom of the screen ('iab new standard ad unit portfolio’ 2017). sometimes, they can be middle of the screen (djamasbi, hall-phillips & yang 2013). the research so far yields mixed results regarding the top and the bottom of the screen. some other research highlights the importance of native ads, which are normally displayed in the middle of the screen. for that reason, the top and the middle of the screen are selected as two values of ad space position. • ad space timing: refers to the timing of ad spaces delivered by the publishers. for example, when a user first opens the application and has not done any action yet. the ads showing at that time is considered as the beginning. when the user has performed the main activity, e.g. capture a photo, finish one level in games, finish a call, the ads showing during that time is considered as the end (hoque & lohse 1999). the present study selected beginning and end as two values of ad space timing. to test the interactive effects among the four publishers-controlled factors and with other participants-controlled ones, an eight-level factorial design was used. the eight independent variables were ad space duration, ad space size, ad space position, ad space scheduling, location, time, ad medium and ad type. the values of the four participant controlled factors are: • location: location is a contextual factor (grewal, bart, spann & zubcsek 2016). effendi and ali (2017) suggested that there are three types of location: city, area and country. furthermore, marketing reports showed that ctrs are significantly different between the region of north america, europe, australia and new zealand, east asia the region of latin american, africa, middle east and south asia. ('doubleclick for advertisers, a cross section of regions’ 2010; 'country ctr stats: spaniards like to tap’ 2012; 'the facebook ads benchmark report’ 2012). the present study selected those two regions as the two values of location, naming region 1 and region 2. • time: most of our research is focusing on the time of day. for example, li, y (2014) discovered that most of twitter tweets more active during weekends than weekends. furthermore, a report from chitika insights shows that marketers can best capitalize on elevated user ctr on saturdays and sundays when the rate at which users click and browse the web lies well above weekday levels (2013). user ctr on the weekdays is, on average, 7% – 12% below the ctrs during the weekend, depending on the day. the present study selected weekdays and weekend as the two values of the time. • ad medium: the medium type is the channel through which the ad is served to a consumer. the medium types can be a webpage or a mobile application. the content of the webpage or application can influence the perception of an ad (grewal, bart, spann & zubcsek 2016). the medium type can also be the platform (e.g. ios and android) on which the app is running. advertisements placed on different apps could australasian journal of information systems truong, nkhoma & pansuwong 2019, vol 23, research article integrated effectiveness framework for in-app advertising 30 have different results as shown in the study of brakenhoff and spruit (2017). the study has shown that the content of the application can influence the perception of an ad placed on it. for the present study, the researcher selects two apps with different designs as the two values of medium type. • ad type: interactive advertising bureau defines many different types of ads including static and dynamic advertisements ('iab new standard ad unit portfolio’ 2017). ad networks normally can support all kinds of ad types like those. for example, text advertisements and image banner advertisements are available in admob (prochkova, singh & nurminen 2012). lim, tan and jnr nwonwu (2013) revealed that mobile users better recall static image banner ads than static text ones and more likely to perceive large image banner ad as app content. for the present study, text and image ads are selected as two values of ad type. in the present study, there is one dependent variable which is the click-through rate. the value is typically measured as the ratio between the number of clicks and the number of impressions. for the cases of ad space duration and ad space size, the click-through rate will be adjusted following the time-based formula of ctr (truong 2016). accordingly, one impression of 90 seconds is not considered the same as one of 30 seconds, and one impression of large banner ads is actually considered as two smaller ones. the ctr is used to measure the effects of the four publishers-controlled factors and their interactions with factors controlled by other participants. 6.2 results as the population of mobile users is limitless and not all of them are given equal chances of downloading mobile apps, a non-probability sampling technique is needed (lavrakas 2008). accordingly, the sample size is calculated with the van belle (2011) formula: 𝑛𝑛 = 𝑧𝑧2 × 𝑝𝑝 × (1 − 𝑝𝑝) 𝑐𝑐2 where the z-score that corresponds to 95% confidence is 1.64. p is the percentage of picking a choice being calculated as ctr, where ctr is the average ctr of the whole sample. c is the confidence, being calculated as ctr(a) – ctr(b) (yacko 2012), with ctr (a), is the average ctr of the first variant and ctr (b) is the average ctr of the second variant. with the average ctr of 5%, and the confidence level of 1%, the sample size is calculated as 1,278 impressions. for over two months, there are 2,813 impressions and 118 clicks being recorded. the shapirowilk test has shown that the data is normally distributed (w = 0.96277, p = 0.7123). table 3 summarizes their results. australasian journal of information systems truong, nkhoma & pansuwong 2019, vol 23, research article integrated effectiveness framework for in-app advertising 31 variable variant number of impressions number of clicks ctr deviation z score p-value duration 30s 1597 66 4.13% 0.0050 4.49 1.00* 90s 1216 52 1.43% 0.0034 size small 1611 71 4.41% 0.0051 3.78 1.00* large 1202 47 1.96% 0.0040 position top 1451 51 3.51% 0.0048 1.70 0.96* middle 1287 62 4.82% 0.0060 timing beginning 1411 87 6.17% 0.0064 5.26 1.00* end 1402 31 2.52% 0.0039 location region 1 1381 45 3.26% 0.0048 2.73 1.00* region 2 1099 61 5.55% 0.0069 time weekdays 1707 77 4.51% 0.0050 -1.73 0.04* weekend 551 36 6.53% 0.0105 ad medium app1 2048 102 4.98% 0.0048 4.09 1.00* app2 765 16 2.09% 0.0052 ad type text 1367 92 6.73% 0.0068 6.47 0.04* image 1446 26 1.80% 0.0035 table 3: results of main effect z-tests on the four publishers controlled factors and four factors being controlled by other participants based on the p-values of the z-tests, it has shown that all the factors are strongly impacting the click-through rate. firstly, the 30 seconds ads and the smaller ads are shown to be more effective than the longer and larger ones, taking into account their duration and size. secondly, between the top and middle ads, this study observed a significant difference in term of ctr (with confidence level = 93%). middle ads could yield a higher ctr (4.82% ± 0.0063) than that from top ones (3.51% ± 0.0048). thirdly, the present study confirms that the ads showing in the beginning when the app is firstly initiated have higher ctr (6.17% ± 0.0064) than those showing later (2.52% ± 0.0039). those tests have confirmed hypothesis 1, 2, 3 and 4: ad space duration, ad space size, ad space position and ad space timing are all factors that can impact the click-through rate. table 3 also shows the z-test results of factors controlled by other participants. this step is necessary before the interaction test can be later performed. the average click-through rate in region 2 (6.19% ± 0.0077) is significantly higher than those in region 1 (4.13% ± 0.0063) as suggested in other reports ('doubleclick for advertisers, a cross section of regions’ 2010; 'country ctr stats: spaniards like to tap’ 2012; 'the facebook ads benchmark report’ 2012). the average click-through rate on weekends (6.52% ± 0.0105) is found to be significantly higher than those on weekdays (4.52% ± 0.0050). that agreed with the results from previous research (donnini 2013; li, y 2014). the average click-through rate in app1 (6.13% ± 0.0060) is significantly higher than that in app2 (2.37% ± 0.0059). app1 is designed with an editing screen is separated from the menu screen, while in app2, the ads are shown alongside the editing functions. the click-through rates are different on different apps agreed with the previous findings (e.g. brakenhoff and spruit (2017)). the z-tests of the present study lastly showed that the click-through rate of text ads (7.68% ± 0.0078) is significantly higher than that of image, animated image and rich media ads (2.19% australasian journal of information systems truong, nkhoma & pansuwong 2019, vol 23, research article integrated effectiveness framework for in-app advertising 32 ± 0.0044). that confirms that ad type is a factor which strongly impacts the click-through rate in the context of mobile in-app advertising (lim, tan & jnr nwonwu 2013). as the eight factors are all confirmed having main effects on the click-through rate. this study proceeded with the interaction tests. table 4 shows the results. hypothesis interaction z score p-value hypothesis 5 duration * location 0.21 0.583 size * location -1.09 0.137 position * location -2.82 0.002* timing * location 2.00 0.977* hypothesis 6 duration * time -1.20 0.115 size * time -2.34 0.01* position * time 2.61 0.995* timing * time 64.17 1.00* hypothesis 7 duration * ad medium 0.68 0.753 size * ad medium -0.09 0.464 position * ad medium -0.87 0.192* timing * ad medium 4.72 1.000* hypothesis 8 duration * ad type 1.92 0.972* size * ad type 8.72 1.00* position * ad type 1.35 0.912 timing * ad type 0.00 0.499 table 4: results of interaction tests based on the results in table 4, it is clearly shown that all four hypotheses are partly supported by the data. there are nine strong interactions of position * location, timing * location, size * time, position * time, timing * time, position * ad medium, timing * ad medium, duration * ad type and size * ad type among them. that further proved that the impacts of consumers, advertisers and ad networks factors on the click-through rate is strongly moderated by publishers-controlled factors. this section has summarized the statistic test results. the hypotheses of the study accordingly have been confirmed. a detailed discussion on how those results imply will be presented in section 7. 7 discussions 7.1 key findings at the confidence level of 95%, the hypotheses 1, 2, 3 and 4 are all confirmed. the publisherscontrolled factors: ad space duration, ad space size, ad space position and ad space timing all have a significantly strong impact on the click-through rate of mobile in-app advertising. the statistic tests also yield that location, time, ad medium and ad type all have strong impacts on the click-through rate as previously confirmed. based on that, the interaction tests involving the eight factors were carried out. at the confidence level of 95%, the hypotheses 5, 6, 7 and 8 are all partly confirmed, showing that the impacts of consumers, advertisers and ad networks factors on the click-through rate is strongly moderated by publishers-controlled factors. as the key research questions and suggested directions summarized above reveal, the publishers play an important role in the ad serving process, therefore could individually and interactively enhance the click-through rate. several key factors that define these relationships, australasian journal of information systems truong, nkhoma & pansuwong 2019, vol 23, research article integrated effectiveness framework for in-app advertising 33 including ad space duration, ad space size, ad space position and ad space timing are found and evaluated. accordingly, this study proposed an integrated effectiveness framework building around a common goal of all participants. 7.2 contributions mobile in-app advertising is a growing business. optimizing mobile in-app advertising by itself is a new subject. the present study is one of the first attempts at digging into this promising area. the research contributes to mobile in-app advertising literature by exploring the role of publishers and the impact of their supply and delivery factors on the click-through rate of mobile in-app advertising. the present study suggests new advertising strategies associated with publishers to further enhance the ad click performance of mobile in-app advertising. by which, newly integrated advertising strategies could be recommended to be applied in practice and could help increasing mobile in-app advertising revenue significantly higher by balancing the benefits of all participants involved. the present study also developed a new empirical method whereas multiple factors controlled by multiple participants could be tested interactively. for publishers, who have more than one app published, applying the new supply and delivery strategies could bring multiple benefits to them. for agents, who publish the apps on the publishers’ behalf, this strategy can bring even more values. ad networks can integrate new strategies associated with these factors to increase the matching and relevance of the ads to their users. furthermore, not only highly applicable to mobile in-app advertising area, the present study could extend to other types of advertising where the role of publishers has not been well studied. the present study shines a new light on online marketing where the interactive outcome metrics play a more important role than ever before. 7.3 limitations and future research the study has several limitations regarding theoretical conceptualisation. first, the study has included only a limited set of important variables within its framework, although other potentially influential variables can also be included in the study. in other words, although the present study has found several key publishers-related factors, there many more of them need to be explored as well. thus, future researches could gain from adding a larger number of potential variables within their frameworks. the present study has recruited thousands of mobile users. the data is only enough for the fractional factorial test. further research could access to a large user base. for which, a full factorial analysis can be performed and the higher level interaction tests with more factors can be validated. even though the research has shown the main effects of ad space duration, ad space size, ad space position and ad space timing, it has not found the optimal value for each. practitioners can continue the test on each factor with more variants to determine the optimal value. this research responds to a call of more research into e-business (jiang, liang & tsai 2019), especially the mobile advertising field (grewal, bart, spann & pal zubcsek 2016; okazaki 2012). the study itself calls for even more research into this promising area. australasian journal of information systems truong, nkhoma & pansuwong 2019, vol 23, research article integrated effectiveness framework for in-app advertising 34 references adler, m, gibbons, pb & matias, y 2002, 'scheduling space-sharing for internet advertising', journal of scheduling, vol. 5, no. 2, pp. 103-119. agarwal, a, hosanagar, k & smith, md 2011, 'location, location, location: an analysis of profitability of position in online advertising markets', journal of marketing research, vol. 48, no. 6, pp. 1057-1073. aguirre, e, mahr, d, grewal, d, de ruyter, k & wetzels, m 2015, 'unraveling the personalization paradox: the effect of information collection and trust-building strategies on online advertisement effectiveness', journal of retailing, vol. 91, no. 1, pp. 34-49. aguirre, m, mahr, d, de ruyter, k, wetzels, m & grewal, d 2012, 'the impact of vulnerability during covert personalization–a regulatory mode approach', paper presented to 41st emac conference, liston-iscte, aimonetti, j 2012, 'apple increases developer iad revenue to 70 percent', <https://www.cnet.com/news/apple-increases-developer-iad-revenue-to-70-percent/>. aksakallı, v 2012, 'optimizing direct response in internet display advertising', electronic commerce research and applications, vol. 11, no. 3, pp. 229-240. alphabet 2018, 'alphabet investor relations numbers', <https://abc.xyz/investor/>. andrews, m 2017, 'increasing the effectiveness of mobile advertising by using contextual information', gfk marketing intelligence review, vol. 9, no. 2, p. 37. andrews, m, luo, x, fang, z & ghose, a 2015, 'mobile ad effectiveness: hyper-contextual targeting with crowdedness', marketing science, vol. 35, no. 2, pp. 218-233. angell, r, gorton, m, sauer, j, bottomley, p & white, j 2016, 'don't distract me when i'm media multitasking: toward a theory for raising advertising recall and recognition', journal of advertising, vol. 45, no. 2, pp. 1-13. 'annual state of email marketing in india' 2015, <http://octaneresearch.in/research/indiaemail-marketing-2015-the-annual-state-of-email-marketing-2/>. ansari, a & mela, cf 2003, 'e-customization', journal of marketing research, vol. 40, no. 2, pp. 131-145. baker, bj, fang, z & luo, x 2014, 'hour-by-hour sales impact of mobile advertising', available in ssrn. bakshy, e, eckles, d, yan, r & rosenn, i 'social influence in social advertising: evidence from field experiments', valencia, spain, acm, 2229027, pp. 146-161. balseiro, sr & candogan, o 2017, 'optimal contracts for intermediaries in online advertising', operations research, vol. 65, no. 4, pp. 878-896. baltas, g 2003, 'determinants of internet advertising effectiveness: an empirical study', international journal of market research, vol. 45, no. 4, pp. 1-9. barnes, sj 2002, 'wireless digital advertising: nature and implications', international journal of advertising, vol. 21, no. 3, pp. 399-420. australasian journal of information systems truong, nkhoma & pansuwong 2019, vol 23, research article integrated effectiveness framework for in-app advertising 35 barry, te 1987, 'the development of the hierarchy of effects: an historical perspective', current issues and research in advertising, vol. 10, no. 1-2, pp. 251-295. barwise, p & strong, c 2002, 'permission-based mobile advertising', journal of interactive marketing, vol. 16, no. 1, pp. 14-24. belk, rw 1975, 'situational variables and consumer behavior', journal of consumer research, vol. 2, no. 3, pp. 157-164. bergen, m 2014, 'twitter beats revenue expectations again but user engagement slows', advertising age. bhat, s, bevans, m & sengupta, s 2002, 'measuring users' web activity to evaluate and enhance advertising effectiveness', journal of advertising, vol. 31, no. 3, pp. 97-106. bleier, a & eisenbeiss, m 2015a, 'the importance of trust for personalized online advertising', journal of retailing, vol. 91, no. 3, pp. 390-409. bleier, a & eisenbeiss, m 2015b, 'personalized online advertising effectiveness: the interplay of what, when, and where', marketing science, vol. 34, no. 5, pp. 669-688. boerman, sc, kruikemeier, s & zuiderveen borgesius, fj 2017, 'online behavioral advertising: a literature review and research agenda', journal of advertising, vol. 46, no. 3, pp. 363-376. brakenhoff, l & spruit, m 'consumer engagement characteristics in mobile advertising', pp. 206-2014. briggs, r 2001, 'measuring advertising success: the value of interactive branding', interactive advertising bureau. briggs, r & hollis, n 1997, 'advertising on the web: is there response before click-through?', journal of advertising research, vol. 37, no. 2, pp. 33-45. broder, a, fontoura, m, josifovski, v & riedel, l 2007, 'a semantic approach to contextual advertising', proceedings of the 30th annual international acm sigir conference on research and development in information retrieval, vol., pp. 559-566. brynjolfsson, e, dick, aa & smith, md 2010, 'a nearly perfect market?', qme, vol. 8, no. 1, pp. 1-33. burke, m, hornof, a, nilsen, e & gorman, n 2005, 'high-cost banner blindness: ads increase perceived workload, hinder visual search, and are forgotten', acm transactions on computer-human interaction (tochi), vol. 12, no. 4, pp. 423-445. čaić, m, mahr, d, aguirre, e, de ruyter, k & wetzels, m 2015, '“too close for comfort”: the negative effects of location-based advertising', advances in advertising research (vol. v), vol., pp. 103-111. chandon, jl, chtourou, ms & fortin, dr 2003, 'effects of configuration and exposure levels on responses to web advertisements', journal of advertising research, vol. 43, no. 2, pp. 217-229. chatterjee, p, hoffman, d & novak, t 2003, 'modeling the clickstream: implications for webbased advertising efforts', marketing science, vol. 22, no. 4, pp. 520-541. australasian journal of information systems truong, nkhoma & pansuwong 2019, vol 23, research article integrated effectiveness framework for in-app advertising 36 chellappa, rk & sin, rg 2005, 'personalization versus privacy: an empirical examination of the online consumer’s dilemma', information technology and management, vol. 6, no. 2-3, pp. 181-202. chen, j & stallaert, j 2010, 'an economic analysis of online advertising using behavioral targeting', vol. chen, p-t & hsieh, h-p 2011, 'personalized mobile advertising: its key attributes, trends, and social impact', technological forecasting & social change, vol. 79, no. 3, pp. 543-557. cheng, hk, li, s & liu, y 2015, 'optimal software free trial strategy: limited version, timelocked, or hybrid?', production and operations management, vol. 24, no. 3, pp. 504-517. cheng, hk & liu, y 2012, 'optimal software free trial strategy: the impact of network externalities and consumer uncertainty', information systems research, vol. 23, no. 2, pp. 488-504. cho, c-h 2003, 'the effectiveness of banner advertisements: involvement and click-through', journalism & mass communication quarterly, vol. 80, no. 3, pp. 623-645. choi, h, mela, c, balseiro, s & leary, a 2017, 'online display advertising markets: a literature review and future directions', in columbia business school research paper 18-1, <https://ssrn.com/abstract=3070706>. conner, m & armitage, cj 1998, 'extending the theory of planned behavior: a review and avenues for further research', journal of applied social psychology, vol. 28, no. 15, pp. 14291464. 'country ctr stats: spaniards like to tap' 2012, adduplex, <https://blog.adduplex.com/2012/02/22/country-ctr-stats-spaniards-like-to-tap/>. cox, dr & reid, n 2000, the theory of the design of experiments, chapman and hall/crc. craik, fi 2002, 'levels of processing: past, present... and future?', memory, vol. 10, no. 5-6, pp. 305-318. cumulative number of apps downloaded from the apple app store 2018, viewed <https://www.statista.com/statistics/263794/number-of-downloads-from-the-appleapp-store/>. danaher, pj & mullarkey, gw 2003, 'factors affecting online advertising recall: a study of students', j. adv. res., vol. 43, no. 3, pp. 252-267. davidavičienė, v 'effectiveness factors of online advertising', pp. 822-830. davis, fd 1985, a technology acceptance model for empirically testing new end-user information systems: theory and results, thesis, massachusetts institute of technology. de pelsmacker, p, geuens, m & anckaert, p 2002, 'media context and advertising effectiveness: the role of context appreciation and context/ad similarity', journal of advertising, vol. 31, no. 2, pp. 49-61. 'digital marketing trends in the education market' 2018, <https://mdreducation.com/reports/digital-marketing-trends-in-the-educationmarket/>. australasian journal of information systems truong, nkhoma & pansuwong 2019, vol 23, research article integrated effectiveness framework for in-app advertising 37 'display and mobile advertising creative format guidelines' 2015, <https://archive.iab.com/www.iab.net/media/file/iab_display_mobile_creative_guide lines_html5_20153.pdf>. djamasbi, s, hall-phillips, a & yang, r 2013, 'serps and ads on mobile devices: an eye tracking study for generation y', universal access in human-computer interaction. user and context diversity, vol., pp. 259-268. donnini, g 2013, 'how marketers can optimize for clicks based on time of day', marketingland, <https://marketingland.com/how-marketers-can-optimize-for-clicksbased-on-time-of-day-30312>. doorn, j & hoekstra, j 2013, 'customization of online advertising: the role of intrusiveness', a journal of research in marketing, vol. 24, no. 4, pp. 339-351. 'doubleclick for advertisers, a cross section of regions' 2010, smartinsights, <https://www.smartinsights.com/internet-advertising/internet-advertisinganalytics/display-advertising-clickthrough-rates/>. drèze, x & hussherr, fx 2003, 'internet advertising: is anybody watching?', journal of interactive marketing, vol. 17, no. 4, pp. 8-23. ducoffe, rh 1996, 'advertising value and advertising on the web', journal of advertising research, vol. 36, no. 5, p. 21. edizel, b, mantrach, a & bai, x 'deep character-level click-through rate prediction for sponsored search', pp. 305-314. effendi, mj & ali, sa 2017, 'click through rate prediction for contextual advertisement using linear regression', <https://arxiv.org/abs/1701.08744>. emarketer 2015, 'mobile will account for 72% of us digital ad spend by 2019', emarketer, <https://www.emarketer.com/article/mobile-will-account-72-of-us-digital-adspend-by-2019/1012258>. evans, ds 2009, 'the online advertising industry: economics, evolution, and privacy', journal of economic perspectives, vol. 23, no. 3, pp. 37-60. 'the facebook ads benchmark report' 2012, top growth marketting, <https://topgrowthmarketing.com/wp-content/uploads/2017/02/facebook-adsbenchmark-report.pdf>. feige, u, immorlica, n, mirrokni, v & nazerzadeh, h 'a combinatorial allocation mechanism with penalties for banner advertising', acm, pp. 169-178. fishbein, m & ajzen, i 1975, intention and behavior: an introduction to theory and research, addison-wesley, reading, ma. ghose, a, goldfarb, a & han, sp 2013, 'how is the mobile internet different? search costs and local activities', information systems research, vol. 24, no. 3, pp. 613-631. ghose, a & todri, v 2015, 'towards a digital attribution model: measuring the impact of display advertising on online consumer behavior'. giddens, a 1986, the constitution of society: outline of the theory of structuration, univ of california press. australasian journal of information systems truong, nkhoma & pansuwong 2019, vol 23, research article integrated effectiveness framework for in-app advertising 38 goh, k-y, chu, j & wu, j 2015, 'mobile advertising: an empirical study of temporal and spatial differences in search behavior and advertising response', journal of interactive marketing, vol. 30, pp. 34-45. goldfarb, a & tucker, c 2011, 'online display advertising: targeting and obtrusiveness', marketing science, vol. 30, no. 3, pp. 389-404. goldstein, dg, mcafee, rp & suri, s 2011, 'the effects of exposure time on memory of display advertisements', proceedings of the 12th acm conference on electronic commerce ec '11, vol., pp. 49-58. goldstein, dg, mcafee, rp & suri, s 2015, 'improving the effectiveness of time-based display advertising', acm transactions on economics and computation (teac), vol. 3, no. 2, pp. 1-20. graham, r 2011, 'a brief history of digital ad buying and sell', <http://www.clickz.com/clickz/column/1721924/a-brief-history-digital-ad-buyingselling >. grewal, d, bart, y, spann, m & pal zubcsek, p 2016, 'mobile advertising: a framework and research agenda', journal of interactive marketing, vol. 34. grewal, d, bart, y, spann, m & zubcsek, pp 2016, 'mobile advertising: a framework and research agenda', journal of interactive advertising, vol. 34, pp. 3-14. grigorovici, dm & constantin, cd 2004, 'experiencing interactive advertising beyond rich media: impacts of ad type and presence on brand effectiveness in 3d gaming immersive virtual environments', journal of interactive advertising, vol. 5, no. 1, pp. 22-36. gupta, r, khirbat, g & singh, s 2014, 'a novel method to calculate click through rate for sponsored search', arxiv preprint arxiv:1403.5771. ha, l 2008, 'online advertising research in advertising journals: a review', journal of current issues & research in advertising (ctc press), vol. 30, no. 1, pp. 31-48. hagen, p, robertson, t & sadler, k 2006, 'accessing data: methods for understanding mobile technology use', australasian journal of information systems, vol. 13, no. 2. haghirian, p & inoue, a 2007, 'an advanced model of consumer attitudes toward advertising on the mobile internet', international journal of mobile communications, vol. 5, no. 1, pp. 48-67. haley, ri 1968, 'benefit segmentation: a decision-oriented research tool', journal of marketing, vol. 32, no. 3, pp. 30-35. hao, l, guo, h & easley, rf 2017, 'a mobile platform's in-app advertising contract under agency pricing for app sales', production and operations management, vol. 26, no. 2, pp. 189-202. hedges, a, ford-hutchinson, s & stewart-hunter, m 1997, testing to destruction: a critical look at the uses of research in advertising, institute of practitioners in advertising. herrewijn, l & poels, k 2018, 'the effectiveness of in-game advertising : examining the influence of ad format.', advances in advertising research, vol. 9, no. power to consumers. australasian journal of information systems truong, nkhoma & pansuwong 2019, vol 23, research article integrated effectiveness framework for in-app advertising 39 hewson, c, vogel, cm & laurent, d 2016, internet research methods, second edition edn, sage publications ltd, london. hirose, m, mineo, k & tabe, k 2017, 'the influence of personal data usage on mobile apps', in advances in advertising research (vol. vii), springer, pp. 101-113. hoffman, dl & novak, tp 2000, 'advertising pricing models for the world wide web', internet publishing and beyond: the economics of digital information and intellectual property, vol. 5, p. 2. holliman, g & rowley, j 2014, 'business to business digital content marketing: marketers’ perceptions of best practice', journal of research in interactive marketing, vol. 8, no. 4, pp. 269-293. hollis, n 2005, 'ten years of learning on how online advertising builds brands', journal of advertising research, vol. 45, no. 2, pp. 255-268. hoque, ay & lohse, gl 1999, 'an information search cost perspective for designing interfaces for electronic commerce', journal of marketing research, vol. 36, no. 3, pp. 387394. iab, iab 2017, 'internet advertising revenue report', internet advertising revenue report, <www.iab.com>. 'iab new standard ad unit portfolio' 2017. idwan, s, alramouni, s, al-adhaileh, m & al-khasawneh, a 2008, 'enhancing mobile advertising via bluetooth technology', international journal of mobile communications, vol. 6, no. 5, pp. 587-597. 'interactive audience measurement and advertising campaign reporting and audit guidelines' 2014, september, <https://www.iab.com/wp-content/uploads/2015/06/adimpression-measurment-guideline-us.pdf>. internet advertising revenue report 1996-2016, <https://www.iab.com/insights/iab-internetadvertising-revenue-report-conducted-by-pricewaterhousecoopers-pwc-2/>. jansen, bj & spink, a 2007, 'sponsored search: is money a motivator for providing relevant results?', computer, vol. 40, no. 8, pp. 52-57. jefferson, s & tanton, s 2015, valuable content marketing: how to make quality content your key to success, kogan page publishers. jeong, ej & biocca, fa 2012, 'are there optimal levels of arousal to memory? effects of arousal, centrality, and familiarity on brand memory in video games', computers in human behavior, vol. 28, no. 2, pp. 285-291. jiang, j, liang, t-p & tsai, jc-a 2019, 'knowledge profile in pajais: a review of literature and future research directions', pacific asia journal of the association for information systems, vol. 11, no. 1. johnson, ej, moe, ww, fader, ps, bellman, s & lohse, gl 2004, 'on the depth and dynamics of online search behavior', management science, vol. 50, no. 3, p. 299. johnson, g & lewis, ra 2015, 'cost per incremental action: efficient pricing of advertising', vol. australasian journal of information systems truong, nkhoma & pansuwong 2019, vol 23, research article integrated effectiveness framework for in-app advertising 40 josephson, s 2004, 'eye tracking methodology and the internet', in handbook of visual communication, routledge, pp. 85-102. karimova, gz 2012, 'toward a bakhtinian typology of ambient advertising', journal of marketing communications, vol. 20, no. 4, pp. 251-269. karp, s 2008, 'google adwords: a brief history of online advertising innovation.', <https://publishing2.com/2008/05/27/google-adwords-a-brief-history-of-onlineadvertising-innovation/>. katz, e, blumler, jg & gurevitch, m 1973, 'uses and gratifications research', the public opinion quarterly, vol. 37, no. 4, pp. 509-523. keller, kl 2016, 'unlocking the power of integrated marketing communications: how integrated is your imc program?', journal of advertising, vol. 45, no. 3, pp. 286-301. kenny, d & marshall, j 2001, 'contextual marketing: the real business of the internet', <https://hbswk.hbs.edu/archive/contextual-marketing-the-real-business-of-theinternet>. kent, rj 1993, 'competitive versus noncompetitive clutter in television advertising', journal of advertising research, vol. 33, no. 2, pp. 40-47. kent, rj 1995, 'competitive clutter in network television advertising: current levels and advertiser responses', journal of advertising research, vol. 35, no. 1, pp. 49-49. kim, ky & lee, bg 2015, 'marketing insights for mobile advertising and consumer segmentation in the cloud era: a q–r hybrid methodology and practices', technological forecasting and social change, vol. 91, pp. 78-92. king, v 2017, 'publishers, it’s time to take back control of your inventory.', medium, <https://medium.com/buysellads-restoring-the-balance/publishers-its-time-to-takeback-control-of-your-inventory-869b72e38e6>. kohavi, r, longbotham, r, sommerfield, d & henne, r 2009, 'controlled experiments on the web: survey and practical guide', data mining and knowledge discovery, vol. 18, no. 1, pp. 140-181. korgaonkar, p, petrescu, m & karson, e 2015, 'hispanic-americans, mobile advertising and mobile services', journal of promotion management, vol. 21, no. 1, pp. 107-125. korula, n, mirrokni, v & nazerzadeh, h 2016, 'optimizing display advertising markets: challenges and directions', ieee internet computing, vol. 20, no. 1, pp. 28-35. kumar, s 2016, optimization issues in web and mobile advertising : past and future trends, springer international publishing : imprint: springer, cham. kumar, s, dawande, m & mookerjee, v 2007, 'optimal scheduling and placement of internet banner advertisements', ieee transactions on knowledge and data engineering, vol. 19, no. 11, pp. 1571-1584. kumar, s, jacob, vs & sriskandarajah, c 2006, 'scheduling advertisements on a web page to maximize revenue', european journal of operational research, vol. 173, no. 3, pp. 1067-1089. kumar, sa 2016, optimization issues in web and mobile advertising : past and future trends, springer international publishing : imprint: springer, cham. australasian journal of information systems truong, nkhoma & pansuwong 2019, vol 23, research article integrated effectiveness framework for in-app advertising 41 kumar, v & gupta, s 2016, 'conceptualizing the evolution and future of advertising', journal of advertising, vol. 45, no. 3, pp. 302-317. laszlo, j 2009, 'the new unwired world: an iab status report on mobile advertising.(interactive advertising bureau)(report)', journal of advertising research, vol. 49, no. 1, p. 27. laudon, kc & traver, cg 2018, e-commerce 2017. lavidge, r & steiner, g 1961, 'a model for predictive measurements of advertising effectiveness', journal of marketing, vol. 25, no. 6, p. 59. lavrakas, pj 2008, encyclopedia of survey research methods, sage publications. le, td & nguyen, b-th 2014, 'attitudes toward mobile advertising: a study of mobile web display and mobile app display advertising', asian academy of management journal, vol. 19, no. 2, pp. 87-103. lee, c, kumar, v & gupta, s 2013, 'designing freemium: a model of consumer usage, upgrade, and referral dynamics' , <http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.475.3716&rep =rep1&type=pdf>. lee, m & faber, rj 2007, 'effects of product placement in on-line games on brand memory: a perspective of the limited-capacity model of attention', journal of advertising, vol. 36, no. 4, pp. 75-90. lemon, kn & verhoef, pc 2016, 'understanding customer experience throughout the customer journey', journal of marketing, vol. 80, no. 6, pp. 69-96. li, h & bukovac, jl 1999, 'cognitive impact of banner ad characteristics: an experimental study', journalism & mass communication quarterly, vol. 76, no. 2, pp. 341-353. li, h & leckenby, jd 2004, 'internet advertising formats and effectiveness', center for interactive advertising, vol., pp. 1-31. li, y-w, yang, s-m & liang, t-p 2015, 'website interactivity and promotional framing on consumer attitudes toward online advertising: functional versus symbolic brands', pacific asia journal of the association for information systems, vol. 7, no. 2. li, y 2014, spatial and temporal patterns of geo-tagged tweets, thesis, proquest dissertations publishing. lim, ty, tan, tl & jnr nwonwu, ge usablity testing, 'mobile in-app advertising for tourism: a case study', in c stephanidis (ed.), four banner ads; 10 participants, berlin, heidelberg, springer berlin heidelberg, how user memorize mobile in-app banner ads and why user ignore or interact with the banner ad?, pp. 695-699. lin, m, ke, x & whinston, ab 2012, 'vertical differentiation and a comparison of online advertising models', journal of management information systems, vol. 29, no. 1, pp. 195236. lin, ttc, paragas, f, goh, d & bautista, jr 2015, 'developing location-based mobile advertising in singapore: a socio-technical perspective', technological forecasting & social change, vol. 103, no. c, pp. 334-349. australasian journal of information systems truong, nkhoma & pansuwong 2019, vol 23, research article integrated effectiveness framework for in-app advertising 42 lin, y & lin, k 'effects of ad sizes, positions, types, and user's gender on the click-through rate of web advertisement', lohtia, r, donthu, n & hershberger, ek 2003, 'the impact of content and design elements on banner advertising click-through rates', j. adv. res., vol. 43, no. 4, pp. 410-418. luo, x, andrews, m, fang, z & phang, cw 2014, 'mobile targeting', management science, vol. 60, no. 7, pp. 1738-1756. mackenzie, s, lutz, r & belch, g 1986, 'the role of attitude toward the ad as a mediator of advertising effectiveness: a test of competing explanations', journal of marketing research, vol. 23, no. 2, p. 130. mangani, a 2004, 'online advertising: pay-per-view versus pay-per-click', journal of revenue and pricing management, vol. 2, no. 4, pp. 295-302. marx, w 1996, 'how to make web ads more effective', advertising age's business marketing, vol. 81, no. 10, p. m1. mcdonald, a & cranor, lf 2010, 'beliefs and behaviors: internet users' understanding of behavioral advertising', in available at ssrn 1989092, tprc. menon, ak, chitrapura, k-p, garg, s, agarwal, d & kota, n 'response prediction using collaborative filtering with hierarchies and side-information', acm, pp. 141-149. meyer, ke, ding, y, li, j & zhang, h 2018, 'overcoming distrust: how state-owned enterprises adapt their foreign entries to institutional pressures abroad', in state-owned multinationals, springer, pp. 211-251. miller, ga 1956, 'the magical number seven, plus or minus two: some limits on our capacity for processing information', psychological review, vol. 63, no. 2, p. 81. miller, s 2006, 'the conversionlab.com: how to experiment your way to increased web sales using split testing and taguchi optimization', <http://www.conversionlab.com/>. mokbel, mf & levandoski, jj 'toward context and preference-aware location-based services', acm, pp. 25-32. molitor, d, reichhart, p & spann, m 2012, 'location-based advertising: measuring the impact of context-specific factors on consumers’ choice behavior', available at ssrn 2116359. montgomery, a, hosanagar, k & clay, k 2004, 'designing a better shopbot', management science, vol. 50, no. 2, pp. 189-206. moorman, m 2003, context considered: the relationship between media environments and advertising effects, thesis, universiteit van amsterdam [host]. muthukrishnan, s 'ad exchanges: research issues', berlin, heidelberg, internet and network economics, springer berlin heidelberg, pp. 1-12. muylle, s, moenaert, r & despontin, m 1999, 'a grounded theory of world wide web search behaviour', journal of marketing communications, vol. 5, no. 3, pp. 143-155. nakamura, a & abe, n 2005, 'improvements to the linear programming based scheduling of web advertisements', electronic commerce research, vol. 5, no. 1, pp. 75-98. australasian journal of information systems truong, nkhoma & pansuwong 2019, vol 23, research article integrated effectiveness framework for in-app advertising 43 nasco, sa & bruner, gc 2008, 'comparing consumer responses to advertising and nonadvertising mobile communications', psychology & marketing, vol. 25, no. 8, pp. 821-837. newman, n, fletcher, r, levy, da & nielsen, rk 2016, 'digital news report 2016', reuters institute for the study of journalism. niculescu, mf & wu, dj 2014, 'economics of free under perpetual licensing: implications for the software industry', information systems research, vol. 25, no. 1, pp. 173-199. nittala, r 2011, 'registering for incentivized mobile advertising: discriminant analysis of mobile users', international journal of mobile marketing, vol. 6, no. 1, pp. 42-53. norris, ce & colman, am 1993, 'context effects on memory for television advertisements', social behavior and personality: an international journal, vol. 21, no. 4, pp. 279-296. oak, p 2008, 'the importance of ad placement', econsultancy, <https://www.econsultancy.com/blog/2778-the-importance-of-ad-placement>. okazaki, s 2012, 'lessons learned for teaching mobile advertising', advertising theory, vol., p. 373. okazaki, s & barwise, p 2011, 'has the time finally come for the medium of the future?: research on mobile advertising', journal of advertising research, vol. 51, no. 1 50th anniversary supplement, pp. 59-71. ouellette, ja & wood, w 1998, 'habit and intention in everyday life: the multiple processes by which past behavior predicts future behavior', psychological bulletin, vol. 124, no. 1, p. 54. park, t, shenoy, r & salvendy, g 2008, 'effective advertising on mobile phones: a literature review and presentation of results from 53 case studies', behaviour & information technology, vol. 27, no. 5, pp. 355-373. patzer, gl 1991, 'multiple dimensions of performance for 30-second and 15-second commercials. (includes appendices)', journal of advertising research, vol. 31, no. 4, p. 18. pavlou, pa & stewart, dw 2000, 'measuring the effects and effectiveness of interactive advertising: a research agenda', journal of interactive advertising, vol. 1, no. 1, pp. 6177. perlich, c, dalessandro, b, hook, r, stitelman, o, raeder, t & provost, f 'bid optimizing and inventory scoring in targeted online advertising', acm, pp. 804-812. pieters, fgm & raaij, wf 1992, reclamewerking, stenfert kroese. prochkova, i, singh, v & nurminen, jk 2012, energy cost of advertisements in mobile games on the android platform. pulizzi, j 2012, 'the rise of storytelling as the new marketing', publishing research quarterly, vol. 28, no. 2, pp. 116-123. punyatoya, p 2011, 'how effective are internet banner advertisements in india', journal of marketing & communication, vol. 7, no. 1. qian, f, wang, z, gao, y, huang, j, gerber, a, mao, z, sen, s & spatscheck, o 'periodic transfers in mobile applications: network-wide origin, impact, and optimization', acm, pp. 51-60. australasian journal of information systems truong, nkhoma & pansuwong 2019, vol 23, research article integrated effectiveness framework for in-app advertising 44 quarto-vontivadar, j 2006, 'ab testing: too little, too soon', <https://persuasion.typepad.com/architect/2006/03/ab_testing_too_.html>. rastogi, v, shao, r, chen, y, pan, x, zou, s & riley, r 'are these ads safe: detecting hidden attacks through the mobile app-web interfaces', richards, ji & curran, cm 2002, 'oracles on "advertising": searching for a definition', journal of advertising, vol. 31, no. 2, p. 63. richardson, m, dominowska, e & ragno, r 'predicting clicks: estimating the click-through rate for new ads', acm, pp. 521-530. robinson, h, wysocka, a & hand, c 2007, 'internet advertising effectiveness: the effect of design on click-through rates for banner ads', international journal of advertising, vol. 26, no. 4, pp. 527-541. rodgers, s, ouyang, s & thorson, e 2017, 'revisiting the interactive advertising model (iam) after 15 years', digital advertising: theory and research, vol., pp. 1-18. rodgers, s & sheldon, k 2002, 'an improved way to characterize internet users', journal of advertising research, vol. 42, no. 5, pp. 85-94. rodgers, s & thorson, e 2000, 'the interactive advertising model: how users perceive and process online ads', journal of interactive advertising, vol. 1, no. 1, pp. 41-60. roehm, ha & haugtvedt, cp 1999, 'understanding interactivity of cyberspace advertising', advertising and the world wide web, vol., pp. 27-39. roels, g & fridgeirsdottir, k 2009, 'dynamic revenue management for online display advertising', journal of revenue and pricing management, vol. 8, no. 5, pp. 452-466. rosenkrans, g 2007, online advertising metrics. rosenkrans, g 2007, 'online advertising metrics', in handbook of research on electronic surveys and measurements, igi global, pp. 136-143. rosenkrans, g & myers, k 2012, 'mobile advertising effectiveness', international journal of mobile marketing, vol. 7, no. 3. rust, rt 2016, 'comment: is advertising a zombie?', journal of advertising, vol. 45, no. 3, pp. 346-347. sahni, n 2015, 'effect of temporal spacing between advertising exposures: evidence from online field experiments', qme, vol. 13, no. 3, pp. 203-247. san josé-cabezudo, r, gutiérrez-cillán, j & gutiérrez-arranz, am 2008, 'the moderating role of user motivation in internet access and individuals' responses to a website', internet research, vol. 18, no. 4, pp. 393-404. sandberg, ra & rollins, m 2013, the business of android apps development making and marketing apps that succeed on google play, amazon app store and more, 2nd ed. edn, apress : imprint: apress, berkeley, ca. schick, s 2013, 'the in-app advertising metrics that will matter', fiercedeveloper [newton]. australasian journal of information systems truong, nkhoma & pansuwong 2019, vol 23, research article integrated effectiveness framework for in-app advertising 45 schneider, l-p, systems, b & cornwell, tb 2005, 'cashing in on crashes via brand placement in computer games: the effects of experience and flow on memory', international journal of advertising, vol. 24, no. 3, pp. 321-343. scholten, m 1996, 'lost and found: the information-processing model of advertising effectiveness', journal of business research, vol. 37, no. 2, pp. 97-104. schonberg, e, cofino, t, hoch, r, podlaseck, m & spraragen, sl 2000, 'measuring success', communications of the acm, vol. 43, no. 8, pp. 53-53. schultz, d 2016, 'the future of advertising or whatever we're going to call it', journal of advertising, vol. 45, no. 3, pp. 276-285. shankar, v & balasubramanian, s 2009, 'mobile marketing: a synthesis and prognosis', journal of interactive marketing, vol. 23, no. 2, pp. 118-129. shavitt, s, lowrey, p & haefner, j 1998, 'public attitudes toward advertising: more favorable than you might think', journal of advertising research, vol. 38, no. 4, pp. 7-22. shu, sb & peck, j 2011, 'psychological ownership and affective reaction: emotional attachment process variables and the endowment effect', journal of consumer psychology, vol. 21, no. 4, pp. 439-452. singh, sn, dalal, n & spears, n 2005, 'understanding web home page perception', european journal of information systems, vol. 14, no. 3, pp. 288-302. statista 2018, 'worldwide mobile in-app advertising revenues in 2015, 2016 and 2020 (in billion u.s. dollars)', statista, <https://www.statista.com/statistics/220149/total-worldwidemobile-app-advertising-revenues/>. steel, e 2011, 'using credit cards to target web ads', <https://www.wsj.com/articles/sb10001424052970204002304576627030651339352>. sun 2017, 'not just a fad: optimal sequencing in mobile in-app advertising', information systems research, vol. 28, no. 3. sundar, ss & kalyanaraman, s 2004, 'arousal, memory, and impressionformation effects of animation speed in web advertising', journal of advertising, vol. 33, no. 1, pp. 7-17. sweetser, kd, ahn, sj, golan, gj & hochman, a 2016, 'native advertising as a new public relations tactic', american behavioral scientist, vol. 60, no. 12, pp. 1442-1457. tam, ky & ho, sy 2006, 'understanding the impact of web personalization on user information processing and decision outcomes', mis quarterly, vol., pp. 865-890. thorson, e & schumann, dw 1999, advertising and the world wide web, psychology press, mahwah, n.j. trivedi, jp 2015, 'mobile advertising effectiveness on gen ys attitude and purchase intentions', international journal of marketing and business communication, vol. 4, no. 2, p. 2. trope, y & liberman, n 2003, 'temporal construal', psychological review, vol. 110, no. 3, p. 403. truong, vn 'optimizing mobile advertising using ad refresh interval', ieee, pp. 1-4. australasian journal of information systems truong, nkhoma & pansuwong 2019, vol 23, research article integrated effectiveness framework for in-app advertising 46 tucker, ce 2014, 'social networks, personalized advertising, and privacy controls', journal of marketing research, vol. 51, no. 5, pp. 546-562. van belle, g 2011, statistical rules of thumb, john wiley & sons. van reijmersdal, e, neijens, p & smit, e 2005, 'readers' reactions to mixtures of advertising and editorial content in magazines', journal of current issues & research in advertising, vol. 27, no. 2, pp. 39-53. varnali, k & toker, a 2010, 'mobile marketing research: the-state-of-the-art', international journal of information management, vol. 30, no. 2, pp. 144-151. wegert, t 2002, 'pop-up ads, part 1: good? bad? ugly', retrieved july, vol. 9, p. 2004. weingarten, e & berger, j 2017, 'fired up for the future: how time shapes sharing', journal of consumer research, vol. 44, no. 2, pp. 432-447. wolfinbarger, m & gilly, mc 2003, 'etailq: dimensionalizing, measuring and predicting etail quality', journal of retailing, vol. 79, no. 3, pp. 183-198. yacko, s 2012, minimum sample size: how many users is enough? yan, j, liu, n, wang, g, zhang, w, jiang, y & chen, z 'how much can behavioral targeting help online advertising?', acm, pp. 261-270. yuan, s, abidin, az, sloan, m & wang, j 2012, 'internet advertising: an interplay among advertisers, online publishers, ad exchanges and web users', in arxiv preprint arxiv:1206.1754. yuan, s, wang, j & zhao, x 'real-time bidding for online advertising: measurement and analysis', acm, p. 3. yunos, hm, gao, jz & shim, s 2003, 'wireless advertising's challenges and opportunities', computer, vol. 36, no. 5, pp. 30-37. zhou, g, song, c, zhu, x, ma, x, yan, y, dai, x, zhu, h, jin, j, li, h & gai, k model testing, 'deep interest network for click-through rate prediction', alibaba data, acm, how does deep interest network compare with the mlp-based model?, pp. 1059-1068. zhu, y & wilbur, kc 2011, 'hybrid advertising auctions', marketing science, vol. 30, no. 2, pp. 249-273. zorn, s, olaru, d, veheim, t, zhao, s & murphy, j 2012, 'impact of animation and language on banner click-through rates', journal of electronic commerce research, vol. 13, no. 2, pp. 173-183. australasian journal of information systems truong, nkhoma & pansuwong 2019, vol 23, research article integrated effectiveness framework for in-app advertising 47 appendix a: real-time bidding process source: brakenhoff and spruit (2017) australasian journal of information systems truong, nkhoma & pansuwong 2019, vol 23, research article integrated effectiveness framework for in-app advertising 48 appendix b: interactive advertising model source rogers & thorson, 2000 australasian journal of information systems truong, nkhoma & pansuwong 2019, vol 23, research article integrated effectiveness framework for in-app advertising 49 appendix c: framework for online behavioral advertising source: boerman, kruikemeier and zuiderveen borgesius (2017) australasian journal of information systems truong, nkhoma & pansuwong 2019, vol 23, research article integrated effectiveness framework for in-app advertising 50 appendix d: mobile advertising effectiveness framework source: grewal et al. (2016) copyright: © 2019 truong, nkhoma & pansuwong. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems truong, nkhoma & pansuwong 2019, vol 23, research article integrated effectiveness framework for in-app advertising 51 1 introduction 2 background 2.1 web advertising 2.2 mobile advertising 2.3 mobile display advertising 3 mobile in-app advertising 3.1 mobile in-app advertising processes 3.1.1 ad space supply process 3.1.2 ad space delivery process 3.2 mobile in-app advertising participants 3.3 mobile in-app advertising outcome metrics 3.4 mobile in-app advertising factors 3.4.1 advertisers-controlled factors 3.4.2 consumers-controlled factors 3.4.3 ad networks-controlled factors 4 publishers-controlled factors and interactions 4.1 publishers-controlled factors 4.2 publishers-controlled interactions 5 an integrated effectiveness framework 6 experiments 6.1 methodology 6.2 results 7 discussions 7.1 key findings 7.2 contributions 7.3 limitations and future research paper title (paper title style) australasian journal of information systems davern & black 2025, vol 29, editorial editorial 1 embracing new insights michael davern faculty of business and economics the university of melbourne parkville, vic australia email: mjdavern@unimelb.edu.au stuart black faculty of business and economics the university of melbourne parkville, vic australia email: stuart.black@unimelb.edu.au 1 introduction as many of you would be aware, ajis operates in a virtual issue basis – we have a single volume for each calendar year, and we publish a group of research articles once we get critical mass (typically four articles). in a slight refinement of precedent, we would like to take the opportunity to introduce these articles to you – our readers – as well as to introduce our upcoming provocation series. we will close by providing you an overview of what we are seeing in the submission pipeline and what we are doing to maximise the benefit that ajis can provide our community. 2 the research articles in march, we released four traditional-style research articles that exemplify the discipline's ongoing exploration of technological impacts on organisations and society. these articles were published under the editorship of our immediate past editor-in-chief, karlheinz kautz. exploring practitioner engagement: "investigation of practitioners’ attitudes and intentions to use information systems research: a mixed method study of the research format" by mirkovski, doyle, and gaskin tackles the perennial research/practice gap through innovative multimedia formats to enhance practitioner engagement (mirkovski et al., 2025). ai empowerment and workforce integration: shi and ma's "ai empowerment of asian-australian migrant workers" integrates ai into the daily work lives of migrant workers, revealing complex dynamics and outcomes that affect both organisational integration and individual worker experiences (shi & ma, 2025). challenging conventional wisdom: the study on boredom in online commerce by ali, khan, and wang revisits this emotion's implications on digital engagement, presenting strategies to leverage understanding boredom for enhanced market performance (ali et al., 2025). ensuring accountability in digital governance: sharma, kar, and gupta’s work on e-government services proposes new accountability frameworks, highlighting transparency and userresponsiveness as critical design factors (sharma et al., 2025). in this release, we include an additional four research articles. australasian journal of information systems davern & black 2025, vol 29, editorial editorial 2 process theory of supplier cyber risk assessment: building upon insights generated from interviews with 33 cybersecurity experts, slapnicar, vidmar and tsen formulate a process theory of supplier cyber risk assessment (slapnicar et al., 2025). supporting organisational agility through boundary spanning and knowledge brokering in a dual operating system: a case from the software industry: rosenkranz, kautz and cooper examine how a software development company operationalises kotter’s concept of a dual operating system to support organisational agility (rosenkranz et al., 2025). an identification-based understanding of team engagement in global virtual teams: shaik, makhecha, varkkey and goud use an identity lens to classify four distinct configurations of gvt members, illustrating the complex nature of engagement dynamics in gvt settings (shaik et al., 2025). emerging leadership practices for hybrid work: the role of algorithmic management tools: alsamarra’i, john and panteli present a case study of a multinational organisation that adopted an algorithmic management-enabled employee experience management platform, finding that algorithmic management tools contribute to emerging leadership practices that reflect inclusive, humanised, and multimodal modes of leadership – all of which are underpinned by analytics-enabled leadership (alsamarra’i et al., 2025). these contributions collectively foster an enriched understanding of the transformative role that information systems play across different context. 3 provocations our vision is for ajis to be a journal that publishes thought-provoking, relevant research of sufficient rigour and contributes to the development of scholars and the discipline within the region. to help realise this vision, ajis introduces its new 'provocations' section, designed to stimulate intellectual debate and inspire future research. these shorter papers, initially solicited from senior is scholars in the region, are intended to challenge existing paradigms and encourage innovative thinking within our community. these provocations do go through a modified formal review process consistent with the different character and purpose of their contributions. we anticipate that provocations will become a small but ongoing part of the diverse portfolio ajis seeks to publish. researchers considering contributing a provocation should contact the editors-in-chief prior to drafting any provocation. our first provocation there is no ‘ai’ in ‘team’! or is there?was in our march release. in this provocation, richter and schwabe reflect on the evolving landscape of human-ai collaboration, calling for nuanced and context-aware analyses of teamwork in the presence of ai (richter & schwabe, 2025). our second provocation why is this article in english? it’s time to design a more impactful, inclusive science – is published with our june release. in this provocation, andrew burtonjones and other members of the association for information systems’ taskforce on ai translation for inclusive, impactful science explore how we might harness ai translation tools to enable a more impactful, inclusive science. in keeping with the spirit of the provocation, we note that the review process was conducted entirely in the language of our reviewers (burton-jones et al., 2025). further provocations are already advanced in the pipeline and will appear as they are ready. you may or may not agree with richter and schwabe or with burton-jones et al. – either way, australasian journal of information systems davern & black 2025, vol 29, editorial editorial 3 we hope that the work provokes the community of scholars to debate and perhaps respond with an empirical contribution to ajis to help advance the field and address such a debate. 4 editorial policy and developments and community engagement aligned with our strategic vision, ajis has undertaken several editorial and operational enhancements: • technological advances: integration of our submissions system ojs 3.x and forthcoming upgrades to ojs 3.5, equipped with sophisticated features such as the ithenticate plagiarism checker and orcid integration. • community engagement: a successful ajis paper development workshop at acis in 2024, underscores our focus on nurturing scholarly collaboration and driving research excellence. we thank the ajis senior editors who helped facilitate the workshop, and look forward to the next workshop at acis 2025, and expanding initiatives like this to benefit our broader research community. • visibility and impact: efforts to increase ajis's presence, such as leveraging social media and the ais elibrary, aim to elevate the journal's status internationally. • ranking: while the fundamental success of the journal rests on the scientific merit and practical relevance of the published papers, it is pleasing to note that ajis has attained a q1 business, management and accounting ranking in the sjr rankings by scimago. kudos to our past editor-in-chief, professor karlheinz kautz, for his work in steering this rising trajectory of ajis. 5 editorial team, policy and process matters as we seek to realise our vision for ajis, and better serve the community of scholars, we are refining and evolving our editorial policies and procedures. • editorial team: we are proud of our senior editor (se) team and are thankful for the efforts and support. in addition to our initial se group (ida asadi someh, wasana bandara, petros chamakiotis, humza naseer, arif perdana, and angsana techatassanasoontorn) we are pleased to announce three additional ses are joining our editorial team: olivera marjanovic, annette mills and alex richter. we are also thankful for the 50+ reviewers that have contributed their thoughts and perspectives to ajis submissions. • editorial policy: we are looking to expand opportunities for publication in new and different ways, such as with our provocations. in other aspects, we are clarifying policy; notably, ajis is not interested in publishing systematic literature review papers. we encourage submissions that provide novel insights, rigorous analysis, and thoughtprovoking discourse that drive both theoretical and practical advancements in our field. • editorial and review processes: we continue to work to reduce the time to decision for papers submitted to ajis. in many cases, this is seeing a first decision within several days, with our rapid eic review. unfortunately, there will inevitably be exceptions, and it is difficult for us to extract formal metrics here with the current version of ojs. australasian journal of information systems davern & black 2025, vol 29, editorial editorial 4 • what we don’t want at ajis desk rejects: sadly, too many papers ajis receives are desk rejected. the most common cause is out-of-scope submissions: papers that are not is papers, as evidenced by references that include little or no is papers, and a research question that, while relevant for another discipline (computer science, management, organisational behaviour), is not targeted to even a broad-minded is readership. we take no pleasure in these decisions, and encourage prospective authors to read our editorials, use the journal template, and ensure that their papers have clear, compelling motivations, contributions and real-world implications. • what we do want at ajis: ajis is interested in a diverse range of research papers, in terms of topics and methods. within the context of that diversity, the papers must contribute to matters of interest to the is community, building from the is literature and contributing back to it. papers can apply, tools, techniques and theory from other disciplines, but the questions addressed must be questions for the information systems community primarily, which presupposes a deeper conceptualisation of the is/it phenomena than would arise from simple application of outside theory/methods in the is context. as we noted in our first editorial, we seek papers that are “interesting and sufficiently rigorous”. ajis papers need to make thoughtful contributions in both approach and potential impact. prospective authors are encouraged to question, rather than simply apply existing theory, models and methods. similarly, they should substantively address implications of the research, including beyond the academy, as opposed to lip service to the issue of practical implications. • commitment: we value the volunteered time of our ses and reviewers, and we want to ensure that the time they allocate to ajis issues is well spent. their efforts, the support of aais and the community, allow ajis to operate as a diamond-open access journal, a status to which we are fundamentally committed, and part of what makes ajis special amongst is journals. 6 it's your community be part of it! as we noted in our 2024 editorial (davern & black, 2024) ajis is a community asset, and we are only the stewards thereof. if you have thoughts or suggestions on the above (or whatever else the journal might do to provide benefits to the aais community), please get in touch via ajis.eic@aaisnet.org. we look forward to reading your interesting and thought-provoking submissions and engaging with you through the editorial and other engagement processes to make ajis a valued community asset. references ali, r., khan, g., & wang, w. (2025). understanding boredom in online commerce: a systematic literature review. australasian journal of information systems, 29. doi.org/10.3127/ajis.v29.4793 alsamarra’i, z., john, b., and panteli, n. (2025). emerging leadership practices for hybrid work: the role of algorithmic management tools. australasian journal of information systems, 29. doi.org/10.3127/ajis.v29.4909 australasian journal of information systems davern & black 2025, vol 29, editorial editorial 5 burton-jones et al., (2025). why is this article in english? it’s time to design a more impactful, inclusive science. australasian journal of information systems, 29. doi.org/10.3127/ajis.v29.5875 davern, m., & black, s. (2024). ajis thinking differently in service of the community. australasian journal of information systems, 28. doi.org/10.3127/ajis.v28.5427 davison, r. m. (2017). editorial: why are you submitting to the isj? information systems journal, 27(5), 555–558. doi.org/10.1111/isj.12157 mirkovski, k., doyle, c., & gaskin, j. (2025). investigation of practitioners’ attitudes and intentions to use information systems research: a mixed method study of the research format. australasian journal of information systems, 29. doi.org/10.3127/ajis.v29.5047 richter, a., & schwabe, g. (2025). “there is no ‘ai’ in ‘team’! or is there?” – towards meaningful human-ai collaboration. australasian journal of information systems, 29. doi.org/10.3127/ajis.v29.5753 richter, s., mohammadhossein, n., richter, a., & trier, m. (2025). augmented reality and the metaverse – narratives of the future. australasian journal of information systems, 29. doi.org/10.3127/ajis.v29.5429 rosenkranz, c., kautz, k., and cooper, v. (2025). supporting organisational agility through boundary spanning and knowledge brokering in a dual operating system: a case from the software industry. australasian journal of information systems, 29. doi.org/10.3127/ajis.v29.5095 shaik, f., et al., (2025). an identification-based understanding of team engagement in global virtual teams. australasian journal of information systems, 29. doi.org/10.3127/ajis.v29.5293 sharma, s., kar, a. k., & gupta, m. p. (2025). building accountability in e-government services: inputs for policy. australasian journal of information systems, 29. doi.org/10.3127/ajis.v29.5175 shi, y., & ma, c. (2025). ai empowerment of asian-australian migrant workers: progress, potentials, and patterns. australasian journal of information systems, 29. doi.org/10.3127/ajis.v29.5189 slapnicar, s., vidmar, t., and tsen, e. (2025). process theory of supplier cyber risk assessment. australasian journal of information systems, 29. doi.org/10.3127/ajis.v29.5323 copyright © 2025 michael davern and stuart black. this is an open-access article licensed under a creative commons attribution-non-commercial 4.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: doi.org/10.3127/ajis.v29.6003 paper title (paper title style) australasian journal of information systems o’sullivan 2021, vol 25, research on indigenous use of special section editorial information and communication technologies 1 editorial for the special section on indigenous use of information and communication technologies: information systems and the practice of indigenous selfdetermination dominic o'sullivan charles sturt university, canberra, australia dosullivan@csu.edu.au self-determination is a social, political and cultural right that belongs equally to all peoples. the three articles in this special section show how information systems’ practice may limit this right on the one hand, or make significant contributions to its development, on the other. the ways in which information is collected and curated to diminish the stories that people wish to tell about themselves, or to create opportunities, is the articles’ common theme. they consider self-determination beyond its more usual study as a body of legal and political rights to show its deep cultural significance and to draw out information systems’ research and practice as inevitably cultural activities. a people’s capacity to tell its own stories, for its own purposes, and to choose how these stories are presented beyond its own citizenry is an expression of the right to self-determination. it has methodological and ethical implications for how research is done, by whom, for whom and for whose purposes. the special section addresses each of these questions in specific context while, inter alia, contributing to the broader scholarship of self-determination. in the first article of the special section thorpe, christen, booker, and galassi argue that museum, library and archival collections have contributed to the colonial narrative through practices that demean indigenous people and cultures in the telling of their stories, in the ways that their sacred artefacts are stored and displayed, and in the ways that they address the repatriation of these artefacts. in many cases questions of repatriation concern human body parts collected, in contravention of indigenous cultural norms, and as objects of scientific curiosity. thorpe et al.’s underlying question is, then, how do colonial archives repay the colonial debt? they argue for the ‘reshaping’ and ‘rebuilding’ of archival management systems, data governance and preservation so that indigenous priorities are central to their work. they show that collection management is not a culturally neutral process and argue that concepts of sovereignty (which may be understood as similar to the rights and capacities of selfdetermination) are underdiscussed in the context of library and archive management. they make a case for alternative collection management practices. the article draws on and develops papers that each of the authors presented to the international conference on archives in 2019. they discuss a collaboration between the waramungu nation in northern australia and the centre for digital scholarship and curation at washington state university as an example of a project that supports the indigenous nation’s expectations of why items should be collected and how and for which purposes they should be curated. australasian journal of information systems o’sullivan 2021, vol 25, research on indigenous use of special section editorial information and communication technologies 2 while thorpe et al.’s article is the only one of the three to make an explicit connection between their arguments and the united nations’ declaration on the rights of indigenous peoples, their discussion illuminates self-determination as a common theme. this thematic commonality shows how the articles may be located within a broader indigenous data sovereignty scholarship (discussed below), to show that they have a theoretical and empirical relevance beyond their immediate subjects. the declaration (un, 2007) provides that: ‘indigenous peoples have the right to practice and revitalize their cultural traditions and customs. this includes the right to maintain, protect and develop the past, present and future manifestations of their cultures, such as archaeological and historical sites, artefacts, designs, ceremonies, technologies and visual and performing arts and literature’ (article 11 (1)). it also affirms that ‘indigenous peoples have… the right to the use and control of their ceremonial objects; and the right to the repatriation of their human remains’ (article 12 (1)). in the second article of the special section, kutay considers the recording and transmission of histories and other stories of significance. the cultural aspects of indigenous relationships with technology must, therefore, be considered. she proposes knowledge sharing as a community narrative and that the employment of new technologies should consider target audiences and their worldviews. she describes and analyses projects that use digital technologies to record and present culturally significant knowledge. these include recording stories of the stolen generations and the development of an information sharing system for the sydney koori interagency network. her discussion of a project involving the distribution of broadcast materials for remote radio and television is a further example of information systems research and practice supporting self-determination by helping people to tell their stories for their own purposes, to audiences that they have chosen and through technological means consistent with these purposes. kutay explains that in each of the cases describe resources developed for the expression of culture. she argues that using ‘technology and knowledge transfer’ for ‘cultural maintenance’ is multifaceted and ‘may start with the interface design, the ontology of the knowledge system; and then extend to developing the components’. she argues that as an expression of selfdetermination this means that ‘eventually communities can be in control of the systems and provide ongoing innovations in this area’. in the third article of the special section clapham, hassan, fredericks, bessarab, kelly, harwoood, senior, longbottom, and dale note the importance of indigenous led research, arguing that research conducted by ‘outsiders’ may be problematic for its perpetuation of a ‘colonial mindset of non-indigenous australians leading to failed solutions to aboriginal problems’. in contrast, they propose a methodological focus on potential rather than problems which may include information systems providing ‘an accessible platform for aboriginal voices’. their purpose is to show how the application of ‘digital tools’ to existing indigenous research methodologies ‘can support authentic data collection’, analysis and dissemination. they develop these arguments from a case study of aboriginal led community research involving an aboriginal community controlled organisation and the illawara koori men’s support group in new south wales. the study showed the benefits of community knowledge and involvement in decision-making to improved health outcomes. they argue that the study australasian journal of information systems o’sullivan 2021, vol 25, research on indigenous use of special section editorial information and communication technologies 3 also demonstrated digital tools’ broader applicability to indigenous research methodologies and that ‘digital tools can (i) support many activities in the conduct of research in aboriginal communities; (ii) enable authentic outcomes to projects which address aboriginal issues and concerns; (iii) help in the management of aboriginal knowledge; and (iv) provide a platform for aboriginal voices’. hasan et al. locate their work alongside indigenous scholarship in other disciplines with a similar focus on contextualising and giving substantive effect to the right to selfdetermination. they cite education, health policy, social work and media studies as particular examples. more widely still, all three articles may also be understood as contributions to indigenous data sovereignty as an emerging field of inquiry. data sovereignty is both a precursor to selfdetermination and an expression of an indigenous capacity to know and interpret their own stories and to collect, record, store and curate information for their own purposes. data sovereignty responds to the idea that relying on others to frame how a people should understand itself, and be understood by the wider world, contributes to colonial stereotypes and to the setting of research and public policy agendas by other people. creating negative images of people to justify colonialism’s hierarchy of human worth is a phenomenon that data sovereignty challenges. the articles in this special section show self-determination’s alternative potential and its significance for information systems’ research and practice. each of the articles is important beyond its own context, because each shows why the information that information systems researchers and practitioners collect, curate and use may reflect people’s relative positioning in a social and political order that they may not have chosen or helped to construct. the articles provide insights into information systems’ potential to contribute to the perpetuation of a social order in which self-determination is not equally available to all peoples or, on the other hand, to contribute to the reclamation of indigenous people’s authority over information about themselves and its use. collectively, the articles show the diversity of data sovereignty’s concerns and the scope of the right and capacities of self-determination. access to knowledge is a measure of social and political inclusion and a measure of cultural independence. the articles show how and why and provide useful case studies for the consideration of indigenous data sovereignty scholarship. finally, we like to acknowledge yeslam al-saggaf, daniel cunliffe, te taka keegan, andrea heather duff, anisha fernando, lynette russel, tristan kennedy, kirsten wahlstrom, and stuart andrew yeates, who served as reviewers. reference united nations (2007). the universal declaration on the rights of indigenous peoples. available at: https://www.hrc.co.nz/files/2714/2415/7163/30-jan-2008_10-3935_un_declaration_on_rights_of_indigenous_people_english.pdf australasian journal of information systems o’sullivan 2021, vol 25, research on indigenous use of special section editorial information and communication technologies 4 copyright: © 2021 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v25i0.3661 http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems lamp 2020, vol 24, editorial farewell to all 1 farewell to all associate professor john lamp the time has come for me to move into a new phase of my life – retirement – and for ajis to move into a new phase of its development. i first came to academia at the age of 40, after a varied work life, over which we shall draw a discrete veil. in 1994, at the request of colleagues at the university of tasmania, i began a list of is journals we could publish in. very rapidly this moved on to the web, and developed into the index of information systems journals. it was because of that web expertise that i became associated with the australian council of professors and heads of information systems, even though it was another 20 years before i actually became formally eligible as an associate professor. in 1998 i moved to geelong campus of deakin university, which was about as close as i wanted to get to melbourne. the index, as well as the acphis, aais, and other websites were physically hosted on a machine in my office. when the excellence in research for australia project was launched, i created an sql database of journals, rather than use the 35,000 line excel spreadsheet that canberra had provided. following a random thought, i made the database generally available through my website. the subsequent 70,000 hits per month made it deakin’s highest profile web server, which did not go unnoticed by deakin’s it services, who migrated my systems to official deakin machines. in 2013 i became editor in chief of ajis. at the time i took up the position, ajis, like so many other journals, was coming to grips with the changes that were, and still are, impacting on academic journals around the world. on the technical side there was the issue of an online presence and what that should look like both aesthetically and operationally, on the specific journal side, of attracting, processing and delivery of articles, the impact of the revolution in copyright and intellectual property and even whether journals still had a place in academic publishing. i began a number of changes, focusing on increasing ajis’ profile, moving to a fully online journal and converting to a green open access journal. on the profile side, • successfully submitted ajis to twenty australian and international indexes and ranking schemes • linked to orcid author identification • published quarterly calls for papers on the major is mailing lists • established a twitter channel for ajis • established an ajis facebook page • increased the rigor involved with dois and added a qr code to each article, linking it to the full journal entry for that article • modernised the look of the ajis website and articles australasian journal of information systems lamp 2020, vol 24, editorial farewell to all 2 • publicised each article as it was released, to provide article by article publicity and build ajis’ profile • participated in the acis doctoral consortium to increase the students’ understanding of the academic publication process and to encourage their participation as reviewers as well as authors specific ajis improvements included • dropping six-monthly print publication in favour of online “publish on finalisation” to reduce publication delays and highlight each article as it became available • developing internal faqs for section editors and submissions for special sections • introducing post publication reviews and research notes sections • participating in a dark archive to preserve, and keep available, ajis articles, should the journal ever fold or suffer catastrophic data loss some of the outcomes from these activities include • increasing the h-index of the journal, measured over its full life, by nearly 50% from 30 to 44 • increasing scimago’s h-index which is measured over the past 10 years of data, by 100% to 15 • boosting ajis article downloads from about 700 per month in 2015 to over 5,000 per month in 2020 • boosting new user accesses from about 65 per day (2014) to 310 per day (2020) • increasing monthly direct accesses to ajis from 905 (2014) to 4,027 (2020), now nearly double the accesses by search engines. • being awarded the directory of open access best practice seal • drastically improving external citations as measured by scimago • attracting international recognition by initiating a section research on indigenous use of information and communication technologies none of this has been the work of me alone. in doing this, i have been supported by ajis’ advisory board, composed of representatives of the australasian association for information systems, the australian council of professors and heads of information systems, and the professors and heads of information systems, new zealand. it is impossible for any academic journal to exist without a talented and committed body of editors and reviewers. over the past six years i have seen the numbers of our editors and reviewers increase and attract members from outside the australasian area as our reputation grew. australasian journal of information systems lamp 2020, vol 24, editorial farewell to all 3 thank you all, the growth and development of ajis would not have been possible without you. ajis was established in 1993 by rob mcgregor and has become a major part of the australasian information systems community. since then, it has had a number of editors: 1993 2005 rob mcgregor 2006 – 2007 craig mcdonald 2008 – 2011 jon heales 2012 – 2013 kathy lynch 2014 – 2020 john lamp it has been a privilege to be counted as one of those people. from 2021, professor karl kautz will take on the role of editor in chief, and i wish him well for the future. as for me, i look forward to devoting more time to recording and broadcasting australian music, and adding to my collection of mechanical calculators. maybe i’ll even get that meccano “mini difference engine” built. copyright: © 2020 lamp. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v24i0.3301 http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems agarwal 2019, vol 23, research on the role of tech in workforce mgt supervisors’ communication style & cyberloafing 1 impact of supervisors’ perceived communication style on subordinate’s psychological capital and cyberloafing upasnaa agarwal national institute of industrial engineering upasnaaagarwal@gmail.com abstract drawing from conservation of resources theory (cor), the purpose of this paper is to examine the impact of supervisor’s perceived communication style (passive, aggressive, and assertive styles) on subordinates psycap and cyberloafing. the study also tests the mediating role of psychological capital (psycap) in the relationship between perceived communication style (cs) of supervisor and cyberloafing. in total, 680 full-time managerial employees from seven diverse firms in india were studied through questionnaire survey. standard instruments were used to assess the constructs. results revealed that perceived cs of supervisors-assertive, aggressive and passive styles have an impact on cyberloafing. psycap partially mediated the relationship between supervisors perceived assertive and aggressive perceived cs and cyberloafing. theoretical contributions and managerial implications of the study are discussed. keywords: cyberloafing, communication styles, passive, aggressive, assertive and psycap 1 introduction internet usage has changed drastically in the past decade with the introduction of personal electronic devices like the iphone, ipad, android, and others (de vass, shee, & miah, 2018). it has become an embracing aspect of today’s working life and is known to enhance organizational and personal productivity. yet technology is considered as a doubled-edged sword and has its dark side too. one of the most acknowledged internet misuses, is cyberloafing, which refers any ‘voluntary act of employees’ using their companies’ internet access during office hours to surf non-job-related web sites for personal purposes and to check (including receiving and sending) personal e-mail’ (lim, 2002, p. 677). cyberloafing has been conceptualized as form of workplace deviance behaviour, which is voluntary in nature and violates organizational norms and threatens the well-being of the organization and/or its members (lim, 2002; mercado et al., 2017). it has been found to have a significant cost to employers and has negative ramifications for employees (mercado et al., 2017). despite its crippling effects worldwide, extant literature remains largely atheoretical and provides little insights into its precursors and underlying mechanisms (akbulut, dönmez, & dursun, 2017; betts, setterstrom, pearson, & totty, 2018). an impressive body of research suggests that the quality of relationship with immediate supervisor has an impact on workplace deviance behaviours (mayer, kuenzi, greenbaum, bardes, & salvador, 2009; penney & spector, 2005). while studies provide important insights into the impact of supervisor behaviours on subordinate’s work outcomes, they are limited to examining the impact of subordinate-supervisor relationship from the lens of broad theories of leadership such as transformational, ethical and authentic leadership (liu, chang, fu, wang, & wang, 2012). scholars in recent years have urged that more than the general aspects of leaders and their styles, day-to-day social interactions between supervisors and australasian journal of information systems agarwal 2019, vol 23, research on the role of tech in workforce mgt supervisors’ communication style & cyberloafing 2 subordinates focusing on how employees are treated through the communication processes needs to be examined (fairhurst & connaughton, 2014; de vries, bakker-pieper, & oostenveld, 2010). communication between a subordinate and her/his superior is at the heart of leadership influence and is believed to have tremendous implications for the survival and growth of the organization (morrison, 2011). echoing this sentiment yukl & mahsud (2010, p.451) said ‘(since) leadership in essence is a group of behaviours that are communicative in nature, it is important to examine the influence of perceived supervisor’s communication style (cs) on subordinate behaviours’. anchored in conservation of resources theory (cor) (hobfoll, 1989), this study tests links between three different communication styles of supervisors using heffner’s (1997) classification passive, aggressive as well as assertive communication styles on cyberloafing. according to cor theory, employees are motivated to obtain, maintain, and invest appropriate resources to accomplish their tasks and accumulate resources generating “positive gain spirals” and avoid potential future depletion creating “negative loss spirals” (hobfoll, 1989). we posit that supervisors perceived communication style(s) can be viewed as organizational resource which can serve to either reduce work stress and replenish their resource pool or drain existing resources (hobfoll, 1989). the concept of psychological capital or simply psycap has been espoused by scholars of positive organizational behaviour (avey, luthans, smith, & palmer, 2010; luthans, avolio, avey, & norman, 2007). psycap is a personal resource, constituting of hope, efficacy, resilience, and optimism and an increasing body of research has as examined psycap as the explanatory mechanism in the relationship between resources and employee outcomes. however, efforts to examine the links between perceived cs of supervisor and employees psycap are missing. commensurate to the cor perspective, this paper posits that perceived cs of supervisor may add to or drain subordinate’s personal resources which in turn will influence their cyberloafing behaviours. this research makes three incremental contributions to extant literature. research on cyberloafing is primarily anecdotal and descriptive (akbulut et al., 2017). by examining the relationship between perceived cs of supervisor (assertive, aggressive and passive) and cyberloafing, the study provides a nuanced understanding of the impact of contextual and personal resources on cyberloafing. the focus on the communication dynamics between supervisors and employees addresses a persistent gap in our understanding of leader effectiveness by modelling leadership influence as a social interaction process as it happens in real time through communication. by examining psycap as an underlying mechanism in cscyberloafing relationship, the study additionally addresses the question of why employee perceptions of supervisor cs affects their behaviour. finally, this work contributes to the larger communication, psycap and cyberloafing literatures by examining how supervisor treatment relate to personal resources and cyberloafing outside of the us. recent calls for cross-national work in the human interface in the information systems and management field underscore the need for additional research that expands our understanding of phenomena beyond anglo-saxon nations. from a practical standpoint, managers can use this information to design organizational interventions aimed at managing cyberloafing, such as supervisors training, socialization procedures, and changes in organizational policies. australasian journal of information systems agarwal 2019, vol 23, research on the role of tech in workforce mgt supervisors’ communication style & cyberloafing 3 figure 1. conceptual model 2 theory the conservation of resources (cor) theory (hobfoll, 1989) is arguably one of the most influential theories explaining human stress and well-being and offers a useful theoretical foundation for understanding the impact of communication styles of supervisors on subordinate attitudes and behaviours. the central proposition of cor theory is that people strive to obtain, retain, and protect resources. resources are defined as objects, personal characteristics, conditions (e.g., supervisory support), or energies (e.g., time, physical/mental energy). two basic processes describe how resources can be lost or gained. the first process, the “resource gain” process describes that initial resources can help generate more resources that accumulate over time, ultimately creating a “gain spiral” of resources (hakanen, bakker, & jokisaari, 2011; hobfoll, 1989, 2002). in contrast, process of “resource loss” describes that stress occurs when there is loss of resources or a threat of loss of resources. when no action is taken, this can ultimately result in a “loss spiral” in which more and more resources are lost. the theory suggests that resource losses are the driver of reactions and in the phase of resources loss, the prime human motivation is directed towards the protection and conservation of resources (hobfoll, 1989; halbesleben & bowler, 2007). people seek to invest their resources to tackle stressful conditions and avoid negative situations, to prevent themselves from potential resource loss, and to gather additional resources. cor theory further suggests that initial resource loss makes employees vulnerable to further resource loss especially when negative emotional sequel occurs and when there is less availability of potential supportive resources (wheeler et al., 2010). 2.1 cyberloafing lim (2002) originally conceptualized cyberloafing as employees’ use of their organizations’ internet or, more simply, the “it way of idling on the job” (p. 678). however, since cyberloafing was introduced in the academic literature, opportunities to engage in technologicallyfacilitated loafing have expanded greatly, resulting in a proliferation of conceptualizations of the construct. recent empirical work in this area has also broadened the construct definition to include additional devices that employees can utilize to cyberloaf (e.g. tablets, smartphones), no longer requiring that technology be organizationally provided. these conceptual shifts reflect technological advances that have occurred in the past two decades. the concept of cyberloafing has been considered and researched as a form of workplace deviance (lim, 2002). organizational deviance, or counter-normative behaviours are initiated by employees which target employers (robinson & bennett, 1995) and range from assertive cs aggressive cs passive cs psycap cyberloafing australasian journal of information systems agarwal 2019, vol 23, research on the role of tech in workforce mgt supervisors’ communication style & cyberloafing 4 misdemeanours such as dishonesty (grover, 1993) and rumour-mongering (skarlicki & folger, 1997), to absenteeism (goldberg & waldman, 2000) and other more serious transgressions such as aggression at the workplace (dekker, greenberg, & barling, 1998), and employee theft (greenberg & scott, 1996). cyberloafing is categorized under the rubric of production deviance, which includes relatively minor, albeit, still organizationally harmful misbehaviours. cyberloafing has been found to have debilitating effects on employee time, efficiency, organizational commitment and effective functioning of organizations (dahling, whitaker, & levy, 2009). one study found that 30% of the companies surveyed were losing more than a day’s work per week from employee’s cyber activities (greenfield & davis, 2002). in the uk, the costs of wastage due to personal internet usage has been estimated to be £38,000 per year for an organization, and approximately £11.2 billion per year for the uk economy (bloxx, 2008). in a recent survey of over two thousand employed americans, 62 percent of employees reported spending some of their workday on social networking websites, and 10 percent indicated that they spend 30 percent or more of their workday on these sites (mercado et al., 2017). the extent of employee time losses as well as associated costs has alarmed organizations and consequently sparked a growing interest among both scholars and practitioners who seek to identify mechanisms to reduce or avoid these costly employee behaviours. however, despite the debilitating effects of cyberloafing, our understanding of its antecedents remains limited (cole, walter, bedeian, & o’boyle, 2012; greenbaum, hill, mawritz, & quade, 2017; liberman, seidman, mckenna, & buffardi, 2011). of the available studies examining the antecedents of cyberloafing, stressors, injustice perceptions toward the organization, external locus of control, sleep deprivation, and workplace norms supporting cyberloafing have been found to augment loafing (henle & blanchard, 2008; krishnan, lim, & teo, 2010; lim, 2002; blau, yang, & ward-cook, 2006). it is also found that employees who feel powerless in their work environment are more likely to engage in interactive forms of cyberloafing, including playing games. in contrast, job satisfaction and involvement and organizational justice perceptions are identified as restraining cyberloafing (liberman et al., 2011; lim, 2002). however, studies integrating the role of supervisor’s day-to-day behaviours and personal factors on cyberloafing are missing. this is astonishing in that supervisor style and individuals’ personal resources have been found to significantly predict job attitudes over a time span (staw & ross, 1985). to address the current gaps, this study concomitantly examines the role of supervisory perceived communication styles and personal factors as well as on cyberloafing. 2.2 supervisors communication style and cyberloafing communication style is defined as a cognitive process that entails micro behaviour in order to make a macro level judgement (raynes, 2001). it consists of verbal and non-verbal messages. verbal messages constitute of the choice of words, along with the tone, speech rate, volume and tonality of the voice and non-verbal cues range from body language, gestures, posture, eye contact and movements (lwehabura & matovelo, 2000). the purpose of communication is to get literal meaning across, contributing to task completion, in the process of working with and through others (kirtley & weaver iii, 1999; norton, 1983). supervisory communication support or lack of it and can determine the extent to which subordinates become motivated and committed in fostering extra-role behaviours (conger, 1991). australasian journal of information systems agarwal 2019, vol 23, research on the role of tech in workforce mgt supervisors’ communication style & cyberloafing 5 researchers over the last few decades have classified communication styles in various ways. one of the most well researched and parsimonious frameworks of cs, which is used in this study, is one proposed by heffner (1997) who grouped the communication styles into three typesaggressive, passive, and assertive. aggressiveness cs: aggressive communicators are perceived to be poor listeners often expressing their feelings and opinions in a way that violates the rights of others, manifesting competition, criticism, interruptions, and impatience for others’ perspectives, often shouting and in some cases even resorting to physical aggression. aggressive communicators are concerned with moving things in their own way and in preserving their own status and power over employees (newbold, 1997). employees working with such supervisors do not experience psychological safety and may thus be discouraged to express their emotions or share burdensome thoughts (cooper et al., 2003). as a result, aggressive supervisors are perceived less favourably with regard to liking (myers & johnson, 2003), social attractiveness, friendliness, attentiveness (kassing, pearce, & infante, 2000) and responsiveness. aggressive supervision has been linked to subordinates’ deviance, job dissatisfaction and organizational commitment (infante & gorden, 1987) assertive cs: assertive communicators believe in two-way communication and come across as honest, objective, tolerant and open with high respect for self and others (lwehabura & matovelo, 2000). they stand up for themselves without denying rights of others (delgadoromero, flores, gloria, arredondo, & castellanos, 2003). they express their thoughts and feelings clearly, they also listen attentively to others concerns and views, signalling interest and building trust (lwehabura & matovelo, 2000). this form of communication builds strong, balanced, and respectful relationships with others and motivates subordinates to go beyond call of their duty and be engaged in assigned tasks. passive cs: supervisors perceived as passive communicators usually go along with other ideas and try to avoid conflict. they are sometimes described as “people pleasers” and are usually unable to convey the full thrust of their message. they feel shy to express their opinion and tend to share limited information and ask limited questions. such supervisors emphasize solely on fulfillment of affiliation goals, and in maintaining harmony, often causing irritation, delays, and rework. drawing from the conservation of resource theory (hobfoll, 1989), this study examines the impact of three csaggressive, assertive and passive on cyberloafing. supervisors perceived to be assertive, fulfil employee’s implicit expectation to be heard and respected for his/her views enhancing subordinates' self-beliefs and confidence and helping them adapt to their work environment. we posit that interactions with such supervisors creates positive gain spiral which enhances employee’s motivation and discourages them from engaging in deviant behaviours, like cyberloafing (bushra, ahmad, & naveed, 2011). contrarily, supervisors engaging in an aggressive cs may be perceived to be socially and emotionally less supportive and approachable (penney & spector, 2005). commensurate to cor, to conserve resources or to avoid additional loss of resources, subordinates may prefer to avoid aggressive supervisors and engage in withdrawal behaviours like cyberloafing. similarly, passive supervisors who do not clearly put forward their ideas and thoughts, may fail to motivate employees to go beyond the call of their duty, which in turn demotivates them (newcombe & ashkanasy, 2002). inability to converse and lack of intellectual stimulation from australasian journal of information systems agarwal 2019, vol 23, research on the role of tech in workforce mgt supervisors’ communication style & cyberloafing 6 a supervisor may result in employees disengaging and withdrawing from actively contributing to organization (avolio & bass, 1995). therefore, the following hypotheses are proposed: hypothesis 1a. supervisor’s perceived assertive communication style will be negatively related to cyberloafing hypothesis 1b. supervisor’s perceived aggressive communication style will be positively related to cyberloafing hypothesis 1c. supervisor’s perceived passive communication style will be positively related to cyberloafing 2.3 supervisory communication style and psycap psycap is defined as an individual’s positive psychological state of development characterized by: (1) having confidence (efficacy) to take on and put in the necessary effort to succeed at challenging tasks; (2) making a positive attribution (optimism) about succeeding now and in the future; (3) persevering towards goals, and when necessary, redirecting paths to goals (hope) in order to succeed; and (4) when beset by problems and adversity, sustaining and bouncing back and even beyond (resilience) to attain success (luthans, avolio, et al., 2007). the common theoretical thread running through the four components of psycap (i.e., efficacy, optimism, hope, and resilience) is the “positive appraisal of circumstances and probability for success based on motivated effort and perseverance” (luthans, avolio, et al., 2007). traditionally, the four resources hope, optimism, efficacy, and resiliency, have been examined as theoretically independent of each other and assessed through distinct measures. however, empirical research in recent years has established the need for combination of these four factors and examining them simultaneously, rather than assessing them separately (avey, luthans, smith, et al., 2010). this study examined all the four dimensions of psycap simultaneously. although psycap is agentic and internalized, it is not devoid of social mechanisms and may vary on the basis of contextual conditions (luthans & youssef-morgan, 2017). social support is an integral mechanism for building employee personal resources. studies indicate that support from immediate leader behaviours directly affect subordinate perception of safety, appropriate behaviour and judgment and strengthen employee psychological capital and work performance (lind & tyler, 1988). on the basis of cor theory (hobfoll, 1989), we further examine the effect of three cs of supervisors on psycap and cyberloafing. assertive cs indicates support, care and concern for the subordinate and develops feeling of hope in subordinates. by providing guidelines for desired work behaviours and attitudes, subordinates working with assertive supervisor (rego & pina e cunha, 2008) strengthen employees’ efficacy beliefs. a sense of optimism is also strengthened by the presence of opportunities to be part of decisions making process (e.g., (carlson, ferguson, hunter, & whitten, 2012). positive supervisory behaviour creates a safe environment encourage others to seek help, encourage others, report problems and bounce back even after encountering failures (edmondson, 1999), building their resilience. overall, working with such supervisors has a positive effect on subordinates and builds their personal resources (walumbwa & schaubroeck, 2009). contrarily, persistent exposure to such negative supervisory behaviour in form of aggressive and passive cs of the supervisor, makes employees less optimistic about accomplishing their australasian journal of information systems agarwal 2019, vol 23, research on the role of tech in workforce mgt supervisors’ communication style & cyberloafing 7 goals (karatepe, 2014). embedded in an environment of negativity, employees often begin to appraise things around them from a negative perspective and lose hope and optimism (avey, reichard, luthans, & mhatre, 2011). overtime, such negative experiences shifts employees perceived locus of causality from internal to external sources, reducing their self-efficacy. employees working under aggressive supervisors may lead to development of psychological paranoia characterized by feelings of distrust, anxiety, and fear hypervigilance and rumination (chan & mcallister, 2014). such employees will have short resilience and may not be able to bounce back easily. employees in unsupportive work environment may not feel safe disclosing their failures and reframing such occasions as learning. finally, supervisors with passive cs are perceived not to have the skills and confidence to challenge subordinate’s ambiguity and misunderstanding (newbold, 1997). they are unable to discuss and create mutual understanding and support by directing subordinates in fulfilling objectives (lwehabura & matovelo, 2000), the latter experience stress. lack of social support and opportunities to communicate decreases subordinate’s resiliency (coutu, 2002), their confidence to cope and bounce back from failures and adversity (luthans, avolio, et al., 2007). subordinates experiencing a passive (unclear) or aggressive communication, overtime, may experience a feeling of “being overextended and depleted of one's emotional and physical resources and personal resources” (maslach & leiter, 2008), resulting in experiencing lack of personal resources. informed by these extant researches and anchored in cor theory, we examine the association between communication styles. this study proposes the following hypothesis: hypothesis 2a. supervisor’s perceived assertive communication style will be positively related to psycap. hypothesis 2b. supervisor’s perceived passive communication style will be negatively related to psycap. hypothesis 2c. supervisor’s perceived aggressive communication style will be negatively related to psycap. 2.4 psycap-cyberloafing empirical results have yielded support for strong associated of psycap with performance (luthans, avolio, et al., 2007). by nature, people with high psycap have greater goal-directed energy and achievement orientation and invest considerable resources towards the completion of work tasks. using positive cognition and motivational processes such individuals harness their behaviours to complete tasks and do not get easily distracted from work (like engaging in cyberloafing). in addition, such individuals tend to employ high selfregulatory motivation as well as emotion control strategies (e.g., self-set goals, controlling negative emotions, re-evaluating negative stimuli in work and nonwork settings) to avoid workplace distractions (yang & chang, 2008). with these characteristics, we expect those high on psycap to fulfil their obligations in the workplace and be less distracted and cross the line to engage in cyberloafing in the workplace. next, the individual components of psycap can specifically be contrasted with deviance (norman, avey, nimnicht, & graber pigeon, 2010). hope is driven towards the twin positive outcomes of goal accomplishment and finding ways to accomplish individual and organizational goals (snyder, 2002). by nature, then, cyberloafing is contrary to these goals and therefore such behaviours are not normally exhibited by those high in hope. resilience is australasian journal of information systems agarwal 2019, vol 23, research on the role of tech in workforce mgt supervisors’ communication style & cyberloafing 8 directed toward positive adaptation in the face of adversity. therefore, those high in resilience are looking for positive reactions and adaptations, not negative emotions and cynicism. hence, cyberloafing is clearly not in the category of positive adaptations and therefore are not likely by those high in resilience. self-efficacy is directed toward gathering the needed motivation and confidence toward succeeding at a given task. in opposition, cyberloafing behaviours are clearly behaviours that are aimed more toward demotivation of an individual relative to organizational goals. given that cyberloafing is by nature are geared toward a negative result, they are clearly not behaviours normally displayed by people high in optimism. taken together, we propose that individuals high in psycap have more resources to draw upon to pursue goals (hobfoll, 2002a) and their ability to self-control will play an important role in the self-regulation process and such individuals are less likely to lack focus in their job and cyberloaf. thus, we propose hypothesis 3: psycap will be negatively related with cyberloafing 2.5 mediating role of psycap psycap has emerged as an important mediating variable in the leadership literature. evidence shows that psycap can be modulated by features of the organizational environment (luthans et al. 2007). in particular, research has shown that effective leadership styles promote psycap among employees, and this in turn improves job performance (rego, sousa, marques, & e cunha, 2012). for instance, psycap has been found to mediate the relationship between perception of transformational leadership and citizenship behaviours (gooty, gavin, johnson, frazier, & snow, 2009). gooty et al. (2009) found that psycap fully mediated the relationship between transformational leadership and organizational citizenship behaviours (ocbs). two other studies (rego, sousa, marques, & e cunha, 2012) examined the mediating effects of psycap in the authentic leadership and work outcomes. recently, karatepe and talebzadeh (2016) found that servant leadership influences work engagement through psycap. psycap has also been examined as an underlying mechanism in stress-outcome relationship (liao & liu, 2015; liu et al., 2012). only recently other studies (agarwal, 2018; wu & lee, 2016) found that psychological capital mediates the relationship between abusive supervision and knowledge sharing. taken together, we posit that assertive cs of supervisors build personal resources which in turn is negatively related to cyberloafing. aggressive and passive cs of supervisors, discourage personal resources which in turn encourage deviant behaviours like cyberloafing (henle & blanchard, 2008), which functions as an ‘office toy’ that offers employees temporary respite from work demands (anandarajan & simmers, 2005). premised on cor theory, we posit that communication style of supervisors impacts their personal resourcesreplenishes or drains, which in turn influence employees cyberloafing behaviours (liu, chang, fu, wang, & wang, 2012; tuckey & neall, 2014). hypothesis 4: psycap mediates the relationship between abusive supervision and cyberloafing 3 methods 3.1 sample and procedures human resource heads of 14 organizations were approached. initially, we interacted personally with the hr managers of each company, and delivered a presentation to them, which included an explanation of the need and significance of the study with the emphasis on its managerial implications. seven organizations gave permission to conduct the survey. these australasian journal of information systems agarwal 2019, vol 23, research on the role of tech in workforce mgt supervisors’ communication style & cyberloafing 9 organizations were –a textile company, pharmaceutical company, chemical company, cement company, steel company, mining company and an it/ites company. participating organizations were provided with all the details regarding the survey process. seven hr managers agreed to send an invite to their full-time employees to volunteer for participation and interact in the process. employees who volunteered came together in the company meeting room where researchers explained the need and importance of the study plus the survey procedures and assured them confidentiality of participants. the completed questionnaires were directly mailed to the researchers (through e-mail) without the management’s participation in order to decrease the authority influence over participants. data were collected in two stagestime 1 and time 2. in time 1, respondents were asked to provide the demographic information and perceived communication style of their supervisors. after two weeks, the second stage (time-2), we conducted the second-phase survey, following the same procedures as in time-1. respondents provided their ratings on psycap and cyberloafing. every questionnaire was marked with a unique code which was recorded in a master file such that the responses received from the two phases can be matched. in time 1, we distributed questionnaires to 1200 full-time employees and received 800 completed questionnaires, representing the response rate of 66.0%. in the second stage, we received 680 completed and usable questionnaires, representing an overall response rate of 56%. the data were screened for outliers and missing data. within the sample, 76.3% were male and 23.7% aged from 28 to 65. sixty-eight percent worked for their companies for more than 3 years and 27.2% had a university degree or above. 3.2 measures unless otherwise indicated, all measures used a response scale ranging from 1, which was ‘strongly disagree’, to 5, which was ‘strongly agree’. perceived managerial communication styles. perceived managerial communication styles had three dimensions: passive, aggressive, and assertive and was measured by using instrument developed by brigham young university (2004) (appended). each dimension was assessed on eight items. all the items of three styles were positively keyed. the items of the scale were arranged randomly to assess the three styles, which supervisors is perceived to employ when interacting with their employees. employees were asked to indicate the extent to which each statement was true regarding their immediate superiors’ communication style. sample item on passive communication style include, “my supervisor lets other people take unfair advantage of him/her””. sample items on aggressive communication style include, “my supervisor often ignores another person’s rights”, and “my supervisor often monopolises conversations”. sample items on assertive communication style include, “my supervisor is able to recognize and express his good points”. psychological capital (psycap). we measured psycap using the 12-item scale adopted from luthans, avolio, et al., (2007). the scale included three items each for the four dimensions of psycap scaleself-efficacy, optimism, hope, and resilience. some sample items are (a) efficacy, ‘‘i feel confident helping to set goals in my work area.’ (b) hope, ‘‘right now i see myself as being pretty successful at work.’’ (c) resilience, ‘‘i usually take stressful things at work in stride.’’; and (d) optimism, ‘‘i always look at the bright side of things regarding my job. ’the 12 items were averaged to form a single measure of psycap and to reflect its nature as the higher-order construct of the four dimensions. all items were measured on a six-point scale. australasian journal of information systems agarwal 2019, vol 23, research on the role of tech in workforce mgt supervisors’ communication style & cyberloafing 10 cyberloafing was measured by using 11-item scales developed by lim (2002). respondents were asked to indicate how often they had engaged in cyberloafing over the past month using a scale ranging from 1 = never to 5 = constantly. examples from the 11-item measure of cyberloafing include “browsing investment related websites,” and “checking non-workrelated e-mail.” 3.3 control variables we controlled for variables that might influence cyberloafing behaviour. respondents’ gender (coded as 0 = male, 1 = female) was controlled for because men are more likely to engage in cyberloafing than women (lim & chen, 2009). age was controlled because it has been found that individuals in their late 20s to early 30s are more likely to use the internet (reed, doty, & may, 2005). also, educational level was controlled, because highly educated employees might have been more exposed to internet usage. educational level was measured as 1 = middle school graduate, 2 = high school graduate, 3 = 2-year college graduate, 4 = 4-year university graduate, and 5 = master’s degree or higher. 3.3.1 test for common method variance (cmv) since the data for this study were obtained from a single survey, common method variance (cmv) was possible. to control for the influence of cmv in our study, we followed established remedial and statistical steps. based on literature recommendations (podsakoff, mackenzie, podsakoff, et al., 2003) we used established scales only, explained the procedures to our participants and guaranteed anonymity. furthermore, we used filler items and different instructions to create a psychological separation between the sets of variables. we also conducted a series of confirmatory factor analyses and calculated various fit indices to determine how the model fitted our data (hair, 2005). moreover, data were collected with a gap of two weeks to avoid social desirability and common method bias. conventionally, the x2/df values less than 2.5 indicate a good fit (arbuckle 2006). comparative fit index (cfi), non-normed fit index (nfi), goodness of fit index (gfi) and root mean square error of approximation (rmsea) were additionally considered to assess the model fit. hu and bentler, (1999) suggested that for cfi, nfi, and gfi, values of 0.90 and above indicate good fit. srmr values below 0.08 suggest an acceptable model fit. for rmsea, values less than 0.05 generally indicate good fit, while values between 0.05 and 0.08 suggest acceptable fit (kline, 2005). the proposed five-factor (abusive supervision, psycap, lmx, turnover intentions and stress) hypothesized model, yielded an acceptable fit to the data (χ2=642, df=252, χ2/df=2.5, cfi=.93, gfi=0.96, nfi=0.95, rmsea=0.04). the fit of alternative models presented in table 4 were inferior to that of the hypothesized five-factor. further, harman’s single-factor test was utilized to investigate potential cmv among the study variables (podsakoff, mackenzie, lee, et al., 2003). the basic assumption of harman’s single-factor test is that if a substantial amount of cmv is present, one general factor will account for the majority (>50 %) of the covariance among the variables. the results of this test showed that multiple factors were extracted and the first factor accounted for only 34.8 % of the total variance, suggesting cmv could be partially mitigated. australasian journal of information systems agarwal 2019, vol 23, research on the role of tech in workforce mgt supervisors’ communication style & cyberloafing 11 αa crb m sd 1 2 3 4 5 6 7 8 9 1. age 25.3 8 7.3 2. gender .81 .32 .213 1 3. education .6 7 .213 -0.009 1 4. job tenure 8.7 4.5 5.61 0.017 -0.03 1 5. cyberloafing .81 .89 4.5 4.3 0.334 0.001 0.019 .078** 1 (.79) 6. passive comm .70 .81 4.2 5.1 0.313 -.113** -0.049 -0.013 .117** 1 (.75) 7. aggressive comm .81 .86 4.7 4.5 .565 -.066* -0.031 -0.009 .082** -.469** 1(.89) 8.assertivecom .83 .92 4.9 3.8 .236 .091** 0.014 0.038 -.245** -.384** .541** 1 (.85) 9.psycap .93 .88 5.1 3.1 -0.64 .231* .231 .432 -.346** -.230*** -.451** .560** 1(.93) a. α: cronbach alpha reliability; b. cr: composite reliability of the construct measures average variance extracted (ave) for each construct (nos. 4-8) is provided in parentheses along the diagonal; values below the diagonal are inter-construct correlations; values above the diagonal (i.e. ave) are square of correlations (nos. 4-8). ** p < .01(two-tailed); * p < .05 (two-tailed); n = 680 table 1 descriptive statistics and inter-construct correlations australasian journal of information systems agarwal 2019, vol 23, research on the role of tech in workforce mgt supervisors’ communication style & cyberloafing 12 3.3.2 analytic approach assessment of data was done using amos 20. the correlation matrix, means and standard deviation are found in table 1. cronbach’s α values for all the scales are estimated above 0.70. 3.3.3 measurement model analysis measurement model was evaluated by conducting confirmatory factor analysis (cfa) to analytically check how distinct the constructs were from one another. it was expected that the items would load on their respective constructs only. conventional fit indices were considered to assess the model fit: goodness-fit index (gfi), adjusted goodness-of-fit index (agfi), normed fit index (nfi), and root mean square error of approximation (rmsea). values between 0.8 to 0.9 for gfi, agfi, and nfi can be accepted (hu & bentler, 1999) and values beyond 0.9 indicate excellent model fit. values less than 0.05 for rmsea indicate an excellent fit for the model whereas values within 0.06 and 0.09 signify an acceptable fit. confirmatory factor analysis (cfa) was applied using amos 20 to analyse the measurement model. cfa results yielded a very good model fit (χ2= 605.4, degrees of freedom (df) = 423, gfi=0.91, agfi=0.90, nfi= 0.92, ifi= 0.98, cfi=0 .97, rmsea=0.03, confidence interval (ci)=0.05-0.07). the loading estimates ranged from 0.70 to 0.79 for aggressive communication, .71 to .88 and .70 to .92 for passive and assertive communication, respectively, .71 to .79 for psycap and; 0.70 to 0.88 for cyberloafing. when observed variables of the construct correlate with each other, as per the theoretical foundation, it represents a convergent validity. high composite reliability and significant loadings signify convergent validity (cable & derue, 2002) as shown in table 2. the results indicated that the model adaptability was satisfactory. discriminant validity was also tested to determine the extent of distinctness for each construct (fornell & larcker, 1981). table 2 shows the correlations among the constructs, which were smaller than the square roots of the ave in each dependent and independent construct. this effectively supports the discriminant validity of the model. further, the values of the maximum shared variance (msv) and average shared variance (asv) were put together with the ave values. according to hair et al. (2010) if all asv and msv values are less than their respective ave values, discriminant validity prevails. table 2 confirmed the stated condition for defining discriminant validity. because we took data from more than one organization, a breach of an assumption of observations independence may occur. therefore, we carried out one-way analysis of variance (anova), inter-reliability (rwg statistic) and intra-class correlation (icc) tests to analyse whether the systematic between-group variance in employee cyberloafing was present (chen & bliese, 2002) anova res. ult (f=0.363, ns), rwg statistic (0.33) and icc (1)(0.03) values for employee cob reported no systematic between-group variance in employee cyberloafing. australasian journal of information systems agarwal 2019, vol 23, research on the role of tech in workforce mgt supervisors’ communication style & cyberloafing 13 variable no of original items no of retained items standardised loadings ave msv asv aggressive communication 8 8 .72*** 0.89 0.78 0.72 .70*** .46*** .79*** .76*** .77*** .76*** .78*** assertive communication 8 8 .92*** 0.85 0.76 0.71 .87*** .70*** .74*** .77*** .76*** .73*** .76*** passive communication 8 8 .76*** 0.75 0.71 0.70 .70*** .73*** .82*** .88*** .81*** .71*** .71*** .77*** psycap 12 12 .76*** 0.93 0.91 0.89 .74*** .76*** .73*** .84*** .81*** .84*** .76*** .90*** .94*** .87*** .65*** cyberloafing 11 11 .76*** 0.79 0.75 0.72 .70*** .73*** .82*** .88*** .81*** .87** .75** .86** .71*** .75*** * p < .05 (two-tailed); ** p < .01(two-tailed); *** p<0.001; n = 680 table 2 scale reliability and validity australasian journal of information systems agarwal 2019, vol 23, research on the role of tech in workforce mgt supervisors’ communication style & cyberloafing 14 4 results 4.1 structural modelling to test the hypotheses, we examined a series of structural equation models. first, our baseline model is presented as the first model (model 1), which estimated all of the hypothesized relationships between the current variables. specifically, we included the direct paths from: assertive, aggressive and passive cs to psycap; psycap to cyberloafing table 3 presents the unstandardized coefficients for the structural paths estimated in the model. it can be seen from table 3 that assertive, aggressive and passive cs were related to cyberloafing (b = -.43, β = .11, p < .01), (b = .11, β = .21, p < .01), (b = .23, β = .21, p < .01), respectively thereby, supporting hypothesis 1a, 1b and 1c. assertive, aggressive but not passive cs were also significantly related to psycap (b = .27, β = .21, p < .01), (b = -.37, β = -.31, p < .01), (b = -19, β = -.11, p =ns), respectively, thereby, supporting hypothesis 2a, 2b but not 2c. psycap were positively related to cyberloafing (b = 1.27, β = .59, p < .01), supporting hypothesis 3. next, we evaluated a second model (model 2) to assess whether psycap was a partial or full mediator of the relationship between assertive and aggressive cs and cyberloafing by eliminating the direct path from cs to cyberloafing from model 1 (i.e. testing a full mediation model). this second model (model 2) had a better fit the data than model, and also yielded larger information criteria (see table 3 and 4), suggesting that the full mediation relationship between abusive supervision and cyberloafing was not sufficient to account for the total relationship between these two variables. as such, psycap was a partial mediator the relationship between cs and cyberloafing. then, we tested a non-mediating model by removing the direct path from cs to psycap from model 1. this third model (model 3) also had a poorer fit to the data than model 1 and lower information criteria as evidenced by the result presented in table 3 and 4. **passive cs was not found to be significantly related with psycap, hence not modelled. table 3 goodness-of-fit indexes thus, the results suggested support that in terms of model fit as well as regression weights, partial mediating model was the best fit, supporting partial mediation, hypothesis 4. to further assess the significance of the mediation, we applied sobel’s test for indirect effects. results show that the intervening effect of psycap for assertive and aggressive cs with cyberloafing, χ2 df χ2/df srmr gfi nfi cfi rmsea model 1: psycap mediates the relationship between assertive and aggressive communication and cyberloafing 552.6 200 2.7 .08 .72 .86 .76 .07 model 2: psycap partially mediates the relationship between assertive and aggressive communication and cyberloafing 303.2 246 1.2 .05 .82 .96 .99 .03 model 3: psycap does not mediates the relationship between assertive and aggressive communication and cyberloafing 550 201 2.7 .06 .82 .91 .83 .03 australasian journal of information systems agarwal 2019, vol 23, research on the role of tech in workforce mgt supervisors’ communication style & cyberloafing 15 respectively (standardized indirect effect = .19, sobel t = 5.80, p < .01); (standardized indirect effect = .09, sobel t = 3.20, p < .01) were significant. taken together, hypothesis 4 was supported. structural path unstandardized path coefficients m1 m2 m3 assertive cs cyberloafing -0.435** -0.493** -0.011* aggressive cs cyberloafing 0.119** 0.267** 0.210* passive cs cyberloafing .230** 0.101* 0.011* psycap cyberloafing 0.384** 0.427** assertive cs psycap 0.270** 0.345** aggressive cs psycap -0.371** -0.421** passive cs psycap -0.190 (ns) -0.111 ** p < .01(two-tailed); * p < .05 (two-tailed); n = 680 table 4 unstandardized path coefficients 5 discussion understanding (mis)use of technology is a timely, given its crippling effects on employee and organizational productivity. the purpose of the current study was to examine the impact of perceived communication style of supervisor and psycap on cyberloafing. the study also examined the mediating effect of psycap in perceived cs of supervisors and cyberloafing. the results suggest that cs of supervisors-aggressive, assertive and passive as well as subordinates psycap relate to cyberloafing, and the relationship is partially mediated by psycap for the former two styles (aggressive and assertive). this study of this study advances body of knowledge in the domain of cyberloafing, communication style as well as psycap. despite the diverse studies associated with cyberloafing, a review of the literature suggests that researchers investigating the organizational processes leading to cyberloafing have been ignored. though previous studies have provided valuable insights into role of leadership styles and employee outcomes, one major oversight of the existing research is that the impact of social interactions between supervisors and subordinates that captures the degree to which people are treated on a regular basis. this study tested a well-established and systematic framework to study employee responses to cyberloafing that has not been examined in the context of communication styles of the supervisor. this study also examines the role of psycap as an underlying mechanism in the leadership-cyberloafing relationship. this study is also one of the first one to concomitantly examine the impact of supervisor’s communication styles as well as personal resources on cyberloafing. the findings of this research also provide a communication lens through which to examine the existence of a positive and negative aspects of leader behaviour within the workplace (rook, 1998). by including the influence of psycap, the study moves the discourse one step forward with a specific focus on considering impact of leaders’ communication style on followers’ psychological reactions that translate leadership behaviours into unfavourable behavioural outcomes. australasian journal of information systems agarwal 2019, vol 23, research on the role of tech in workforce mgt supervisors’ communication style & cyberloafing 16 by examining the mediating role cs on psycap and its effects on cyberloafing, the study adds to the positive organizational behaviour literature which has gained prominence in the organizational sciences and emphasis on managing employees’ positive psychological capital (psycap; see (luthans, avolio, et al., 2007). despite emerging empirical work on psycap, luthans et al. noted that much remains to be done, especially in examining not only the effects of psycap, on a range of important work outcomes, but also its antecedents. this study delineates a more comprehensive map of the precursors, intervening mechanisms and outcomes of psycap. the study also provides replication and extension of previous work linking psycap and deviance (luthans, avolio, et al., 2007; norman et al., 2010) by showing in a separate sample that the relationship between these constructs can be replicated and that this relationship depends on social exchange quality. this study adds to psycap literature by examining all the four core dimensions simultaneously, since such efforts are limited 5.1 managerial implications from a managerial standpoint, new knowledge about predictors of cyberloafing can help practitioners design interventions for managing such deviance behaviours. whereas supervisors are often unable to directly influence an organization's policies and practices, they are typically free to determine the extent to which they treat their employees with dignity, respect, and truthfulness, offer clear explanations for their actions, and show concern for them (scott et al. 2014). thus, supervisors should be respectful in communication when interacting with employees for avoiding negative outcomes. research suggests that negativity bias, described as a condition in which the psychological impact of negative events tends to be longer lasting and have a greater influence on a person’s state of mind than do positive events (rook, 1998). the finding of this study can also be examined from cultural lens. employees in collectivist cultures attach greater importance to organization hierarchy (varma, pichler, & srinivas, 2005). people in such societies possess greater associative needs and look up to supervisors for support, protection, and guidance much more than employees in western cultures (tripathi, 1990; vidyarthi, liden, anand, erdogan, & ghosh, 2010). immense emphasis is placed on the quality of communication and dialogue from early years of rearing and socialization. an oftenrepeated discourse at indian homes and school isshabd sambhal boleyee, shabd ke haath na paanv. ek shabd aushadh kare, ek shabd kare ghaav. (tulsi das) (one should use words carefully as they don’t have arms or feet of their own. a word can serve as a balm or become a scar). this indicates the seriousness of communication as a virtue in especially in the indian context (srite, 2006). given the importance of relationship the with the supervisor, the latter’s communication style will have intense and pronounced effects on one’s self worth, personal resources as well as attitudes and behaviour. therefore, the effects of a verbally aggressive and passive communication will likely have a greater negative impact on an employee’s well-being than would a positive event. this study reiterates the importance of the two most important management secrets of employee productivity-pygmalion and galatea effect. others’ expectations (supervisors) of employees and their expectations of themselves are the key factors in how well people perform at work. known as the pygmalion effect and the galatea effect, respectively, the power of expectations cannot be overestimated. the pygmalion effect was described by livingston even earlier in the september/october 1988 harvard business review. "the way managers treat their australasian journal of information systems agarwal 2019, vol 23, research on the role of tech in workforce mgt supervisors’ communication style & cyberloafing 17 subordinates is subtly influenced by what they expect of them," livingston said in his article, pygmalion in management. livingston went on to say about the supervisor, "if he is unskilled, he leaves scars on the careers of the young men (and women), cuts deeply into their self-esteem and distorts their image of themselves as human beings. the findings of the study have implication for academicians as well as managers. academics institutes should design programs to create awareness of the impact of their style on subordinates, especially since it can have important implications on team motivation (fujimoto, härtel, & azmat, 2013; omilion-hodges & baker, 2013). interventions should be designed to train budding managers in effective communication skills. as a managerial implication, organizations can conduct training programs to make their managers aware of the impact of their style of communication on subordinate’s personal resources as well as organizational productivity. communication effectiveness among supervisors can be promoted through emotional intelligence training (moliner, cropanzano, & martínez-tur, 2017) and employee assistance programs which focus on teaching employees to manage their negative emotions and better cope with negative situations. given the importance of psycap in lowering cyberloafing, organizations should pay attention to screening for personal resources in addition to knowledge, skills, and abilities. recent studies show that overall psycap is malleable and can be developed in short training interventions with employees (luthans, avolio, et al., 2007). interventions customized to develop employee’s personal resources may help organizations to address the impact of supervisor communication. 5.2 limitations and future research in this study we integrate communication and leadership literature to examine the effect of perceived communication style of supervisor in cyberloafing. despite substantive theoretical contributions, the study is not without limitations. the cross-sectional design of the study does not allow us to determine causality among variables. this limitation means arrows suggesting association among constructs should not be interpreted as causal relationships, but as associations that suggest causal ordering that should be confirmed with longitudinal research designs. that the alternative model of behaviour demonstrated poor fit suggests alternative causal ordering is unlikely. only longitudinal research can disentangle cause and effect adequately. results may partly be influenced by cmv because all data were collected through self-reports. references akbulut, y., dönmez, o., & dursun, ö. ö. (2017). cyberloafing and social desirability bias among students and employees. computers in human behavior, 72, 87–95. anandarajan, m., & simmers, c. a. (2005). developing human capital through personal web use in the workplace: mapping employee perceptions. communications of the association for information systems, 15(1), 41. arbuckle, j. l. (2006). amos (version 7.0)[computer program]. chicago: spss. avey, j. b., luthans, f., smith, r. m., & palmer, n. f. (2010). impact of positive psychological capital on employee well-being over time. journal of occupational health psychology, 15(1), 17. australasian journal of information systems agarwal 2019, vol 23, research on the role of tech in workforce mgt supervisors’ communication style & cyberloafing 18 avey, j. b., luthans, f., & youssef, c. m. (2010). the additive value of positive psychological capital in predicting work attitudes and behaviors. journal of management, 36(2), 430–452. avey, j. b., reichard, r. j., luthans, f., & mhatre, k. h. (2011). meta-analysis of the impact of positive psychological capital on employee attitudes, behaviors, and performance. human resource development quarterly, 22(2), 127–152. avolio, b. j., & bass, b. m. (1995). individual consideration viewed at multiple levels of analysis: a multi-level framework for examining the diffusion of transformational leadership. the leadership quarterly, 6(2), 199–218. betts, t. k., setterstrom, a. j., pearson, j. m., & totty, s. (2018). explaining cyberloafing through a theoretical integration of theory of interpersonal behavior and theory of organizational justice. in business education and ethics: concepts, methodologies, tools, and applications (pp. 144–165). igi global. blau, g., yang, y., & ward-cook, k. (2006). testing a measure of cyberloafing. journal of allied health, 35(1), 9–17. brigham young university (2004), “assertiveness”, available at: http//f2mail.rediff.com/bn/ download.cgi/assertiveness.htm (accessed 21 december 2017). bloxx. (2008) . productivity, internet abuse, and how to improve one by eliminating the other. available from: http://www.bloxx.com/assets/downloads/bloxx_whitepaper_productivit y.pdf, downloaded on 2 n d january, 2018 bushra, f., ahmad, u., & naveed, a. (2011). effect of transformational leadership on employees’ job satisfaction and organizational commitment in banking sector of lahore (pakistan). international journal of business and social science, 2(18). cable, d. m., & derue, d. s. (2002). the convergent and discriminant validity of subjective fit perceptions. journal of applied psychology, 87(5), 875. carlson, d., ferguson, m., hunter, e., & whitten, d. (2012). abusive supervision and work– family conflict: the path through emotional labor and burnout. the leadership quarterly, 23(5), 849–859. chan, m. e., & mcallister, d. j. (2014). abusive supervision through the lens of employee state paranoia. academy of management review, 39(1), 44–66. chen, g., & bliese, p. d. (2002). the role of different levels of leadership in predicting self-and collective efficacy: evidence for discontinuity. journal of applied psychology, 87(3), 549. cole, m. s., walter, f., bedeian, a. g., & o’boyle, e. h. (2012). job burnout and employee engagement: a meta-analytic examination of construct proliferation. journal of management, 38(5), 1550–1581. conger, j. a. (1991). inspiring others: the language of leadership. the executive, 5(1), 31–45. cooper, d. r., schindler, p. s., & sun, j. (2003). business research methods. retrieved from http://sutlib2.sut.ac.th/sut_contents/h139963.pdf coutu, d. l. (2002). how resilience works. harvard business review, 80(5), 46–56. australasian journal of information systems agarwal 2019, vol 23, research on the role of tech in workforce mgt supervisors’ communication style & cyberloafing 19 dahling, j. j., whitaker, b. g., & levy, p. e. (2009). the development and validation of a new machiavellianism scale. journal of management, 35(2), 219–257. de vries, r. e., bakker-pieper, a., & oostenveld, w. (2010). leadership= communication? the relations of leaders’ communication styles with leadership styles, knowledge sharing and leadership outcomes. journal of business and psychology, 25(3), 367–380. dekker, i., greenberg, l., & barling, j. (1998). predicting union attitudes in student part-time workers. canadian journal of behavioural science/revue canadienne des sciences du comportement, 30(1), 49. delgado-romero, e. a., flores, l., gloria, a., arredondo, p., & castellanos, j. (2003). the majority in the minority: developmental career challenges for latino and latina psychology faculty. the majority in the minority: retaining latina/o faculty, administrators, and students in the 21st century, 257–283. de vass, t., shee, h., & miah, s. (2018). the effect of “internet of things” on supply chain integration and performance: an organisational capability perspective. australasian journal of information systems, 22 edmondson, a. (1999). psychological safety and learning behavior in work teams. administrative science quarterly, 44(2), 350–383. fairhurst, g. t., & connaughton, s. l. (2014). leadership: a communicative perspective. leadership, 10(1), 7–35. fornell, c., & larcker, d. f. (1981). evaluating structural equation models with unobservable variables and measurement error. journal of marketing research, 39–50. goldberg, c. b., & waldman, d. a. (2000). modeling employee absenteeism: testing alternative measures and mediated effects based on job satisfaction. journal of organizational behavior, 665–676. gooty, j., gavin, m., johnson, p. d., frazier, m. l., & snow, d. b. (2009). in the eyes of the beholder transformational leadership, positive psychological capital, and performance. journal of leadership & organizational studies, 15(4), 353–367. greenbaum, r. l., hill, a., mawritz, m. b., & quade, m. j. (2017). employee machiavellianism to unethical behavior: the role of abusive supervision as a trait activator. journal of management, 43(2), 585–609. greenberg, j., & scott, k. s. (1996). why do workers bite the hands that feed them? employee theft as a social exchange process. greenfield, d. n., & davis, r. a. (2002). lost in cyberspace: the web@ work. cyberpsychology & behavior, 5(4), 347–353. grover, v. (1993). an empirically derived model for the adoption of customer-based interorganizational systems. decision sciences, 24(3), 603–640. hair, j. f., anderson, r.e, tatham, r.l., & black, w.c.(2005): multivariate data analysis. upper saddle river, nj: prentice hall. hair, j. f., black, w. c., babin, b. j., & anderson, r. e. (2010). multivariate data analysis: international version. new jersey, pearson. australasian journal of information systems agarwal 2019, vol 23, research on the role of tech in workforce mgt supervisors’ communication style & cyberloafing 20 hakanen, j. j., bakker, a. b., & jokisaari, m. (2011). a 35-year follow-up study on burnout among finnish employees. journal of occupational health psychology, 16(3), 345. halbesleben, j. r., & bowler, w. m. (2007). emotional exhaustion and job performance: the mediating role of motivation. journal of applied psychology, 92(1), 93. heffner, c.l. (1997), “communication styles”, available at: www.siu.edu/offices/counsel/ talk.htm#chart (accessed 8 october 2006). henle, c. a., & blanchard, a. l. (2008). the interaction of work stressors and organizational sanctions on cyberloafing. journal of managerial issues, 383–400. hobfoll, s. e. (1989). conservation of resources: a new attempt at conceptualizing stress. american psychologist, 44(3), 513. hobfoll, s. e. (2002a). social and psychological resources and adaptation. review of general psychology, 6(4), 307. hobfoll, s. e. (2002b). social and psychological resources and adaptation. review of general psychology, 6(4), 307. hu, l., & bentler, p. m. (1999). cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. structural equation modeling: a multidisciplinary journal, 6(1), 1–55. infante, d. a., & gorden, w. i. (1987). superior and subordinate communication profiles: implications for independent-mindedness and upward effectiveness. communication studies, 38(2), 73–80. karatepe, o. m. (2014). hope, work engagement, and organizationally valued performance outcomes: an empirical study in the hotel industry. journal of hospitality marketing & management, 23(6), 678–698. karatepe, o. m., & talebzadeh, n. (2016). an empirical investigation of psychological capital among flight attendants. journal of air transport management, 55, 193–202. kassing, j. w., pearce, k. j., & infante, d. a. (2000). corporal punishment and communication in father-son dyads. communication research reports, 17(3), 237–249. kirtley, m. d., & weaver iii, j. b. (1999). exploring the impact of gender role self-perception on communication style. women’s studies in communication, 22(2), 190–209. kline, r. b. (2005). methodology in the social sciences , (2nd ed.). new york, ny krishnan, s., lim, v. k., & teo, t. s. (2010). how does personality matter? investigating the impact of big-five personality traits on cyberloafing. in icis (p. 6). liberman, b., seidman, g., mckenna, k. y., & buffardi, l. e. (2011). employee job attitudes and organizational characteristics as predictors of cyberloafing. computers in human behavior, 27(6), 2192–2199. lim, v. k. (2002). the it way of loafing on the job: cyberloafing, neutralizing and organizational justice. journal of organizational behavior, 23(5), 675–694. lim, v. k. g., & chen, d. j. q. (2009). impact of cyberloafing on affect, work depletion, facilitation and engagement. in conference paper siop (pp. 1–20). australasian journal of information systems agarwal 2019, vol 23, research on the role of tech in workforce mgt supervisors’ communication style & cyberloafing 21 lind, e. a., & tyler, t. r. (1988). the social psychology of procedural justice. springer science & business media. liu, l., chang, y., fu, j., wang, j., & wang, l. (2012). the mediating role of psychological capital on the association between occupational stress and depressive symptoms among chinese physicians: a cross-sectional study. bmc public health, 12(1), 1. luthans, f., avolio, b. j., avey, j. b., & norman, s. m. (2007). positive psychological capital: measurement and relationship with performance and satisfaction. personnel psychology, 60(3), 541–572. luthans, f., & youssef-morgan, c. m. (2017). psychological capital: an evidence-based positive approach. annual review of organizational psychology and organizational behavior, 4(1). lwehabura, m. j., & matovelo, d. s. (2000). effective library management: issues for managers and subordinates. new library world, 101(6), 263–269. maslach, c., & leiter, m. p. (2008). early predictors of job burnout and engagement. journal of applied psychology, 93(3), 498. mayer, d. m., kuenzi, m., greenbaum, r., bardes, m., & salvador, r. b. (2009). how low does ethical leadership flow? test of a trickle-down model. organizational behavior and human decision processes, 108(1), 1–13. mercado, b. k., mercado, b. k., giordano, c., giordano, c., dilchert, s., & dilchert, s. (2017). a meta-analytic investigation of cyberloafing. career development international, 22(5), 546–564. morrison, elizabeth w. “employee voice behavior: integration and directions for future research.” academy of management annals 5, no. 1 (2011): 373–412.(norton, 1983) myers, s. a., & johnson, a. d. (2003). verbal aggression and liking in interpersonal relationships. communication research reports, 20(1), 90–96. newbold, t. (1997), “what are communication skills?”, in scholes, e. (ed.), gower handbook of internal communication part iv, the item group, london, pp. 235-60. newcombe, m. j., & ashkanasy, n. m. (2002). the role of affect and affective congruence in perceptions of leaders: an experimental study. the leadership quarterly, 13(5), 601–614. norman, s. m., avey, j. b., nimnicht, j. l., & graber pigeon, n. (2010). the interactive effects of psychological capital and organizational identity on employee organizational citizenship and deviance behaviors. journal of leadership & organizational studies, 17(4), 380–391. norton, r. (1983). communicator style: theory, applications, and measures. sage publications beverly hills, ca. retrieved from http://library.wur.nl/webquery/clc/208125 penney, l. m., & spector, p. e. (2005). job stress, incivility, and counterproductive work behavior (cwb): the moderating role of negative affectivity. journal of organizational behavior, 26(7), 777–796. podsakoff, p. m., mackenzie, s. b., lee, j.-y., & podsakoff, n. p. (2003). common method biases in behavioral research: a critical review of the literature and recommended remedies. journal of applied psychology, 88(5), 879. australasian journal of information systems agarwal 2019, vol 23, research on the role of tech in workforce mgt supervisors’ communication style & cyberloafing 22 raynes, b. l. (2001). predicting difficult employees: the relationship between vocational interests, self-esteem, and problem communication styles. applied human resource management research, 6(1), 33–66. reed, k., doty, d. h., & may, d. r. (2005). the impact of aging on self-efficacy and computer skill acquisition. journal of managerial issues, 212–228. rego, a., & pina e cunha, m. (2008). workplace spirituality and organizational commitment: an empirical study. journal of organizational change management, 21(1), 53–75. rego, a., sousa, f., marques, c., & e cunha, m. p. (2012). authentic leadership promoting employees’ psychological capital and creativity. journal of business research, 65(3), 429– 437. robinson, s. l., & bennett, r. j. (1995). a typology of deviant workplace behaviors: a multidimensional scaling study. academy of management journal, 38(2), 555–572. rook, k. s. (1998). investigating the positive and negative sides of personal relationships: through a lens darkly? scott, r. a., fall, t., pasko, d., barker, a., sharp, s. j., arriola, l., boeing, h. (2014). common genetic variants highlight the role of insulin resistance and body fat distribution in type 2 diabetes, independent of obesity. diabetes, 63(12), 4378–4387. skarlicki, d. p., & folger, r. (1997). retaliation in the workplace: the roles of distributive, procedural, and interactional justice. journal of applied psychology, 82(3), 434. snyder, c. r. (2002). hope theory: rainbows in the mind. psychological inquiry, 13(4), 249–275. srite, m. (2006). culture as an explanation of technology acceptance differences: an empirical investigation of chinese and us users. australasian journal of information systems, 14(1). staw, b. m., & ross, j. (1985). stability in the midst of change: a dispositional approach to job attitudes. journal of applied psychology, 70(3), 469. tripathi, r. c. (1990). interplay of values in the functioning of indian organizations. international journal of psychology, 25(3–6), 715–734. tuckey, m. r., & neall, a. m. (2014). workplace bullying erodes job and personal resources: between-and within-person perspectives. journal of occupational health psychology, 19(4), 413. varma, a., pichler, s., & srinivas, e. s. (2005). the role of interpersonal affect in performance appraisal: evidence from two samples–the us and india. the international journal of human resource management, 16(11), 2029–2044. vidyarthi, p. r., liden, r. c., anand, s., erdogan, b., & ghosh, s. (2010). where do i stand? examining the effects of leader–member exchange social comparison on employee work behaviors. journal of applied psychology, 95(5), 849. walumbwa, f. o., & schaubroeck, j. (2009). leader personality traits and employee voice behavior: mediating roles of ethical leadership and work group psychological safety. journal of applied psychology, 94(5), 1275. australasian journal of information systems agarwal 2019, vol 23, research on the role of tech in workforce mgt supervisors’ communication style & cyberloafing 23 wheeler, a. r., halbesleben, j. r., & shanine, k. (2010). eating their cake and everyone else’s cake, too: resources as the main ingredient to workplace bullying. business horizons, 53(6), 553–560. wu, w.-l., & lee, y.-c. (2016). do employees share knowledge when encountering abusive supervision? journal of managerial psychology, 31(1), 154–168. yang, f.-h., & chang, c.-c. (2008). emotional labour, job satisfaction and organizational commitment amongst clinical nurses: a questionnaire survey. international journal of nursing studies, 45(6), 879–887. yukl, g., & mahsud, r. (2010). why flexible and adaptive leadership is essential. consulting psychology journal: practice and research, 62(2), 81. australasian journal of information systems agarwal 2019, vol 23, research on the role of tech in workforce mgt supervisors’ communication style & cyberloafing 24 appendix questionnaire please respond to the following statement, indicating your choice by agreeing or disagreeing ( 1 – “strongly disagree”, 2 – “disagree”, 3 – “neutral”, 4 – “agree”, 5 – “strongly agree”) 1 2 3 4 5 1. my manager is able to recognize and express his good points. 2. my manager often brags and makes unrealistic claims about his good points. 3. my manager feels guilty when he/she stands up for his/her rights or expresses his/her feelings. 4. my manager makes other people look or feel stupid, small, or afraid. 5. my manager allows other people to take unfair advantage of him/her. 6. my manager does not express his/her views and feelings. 7. my manager ignores another person's rights. 8. my manager is able to express negative feelings about other people without being abusive or cruel. 9. my manager frequently takes unfair advantage of other people. 10. my manager is able to receive compliments without denying them. 11. my manager has difficulty saying "no" when he/she does not want to say "yes" to someone's request. 12. my manager often makes unreasonable demands from other people. 13. my manager usually stands up for his/her rights and let other people do the same. 14. my manager often monopolizes conversations. 15. my manager has difficulty making reasonable requests of other people. 16. my manager is able to start or carry on conversations comfortably. australasian journal of information systems agarwal 2019, vol 23, research on the role of tech in workforce mgt supervisors’ communication style & cyberloafing 25 17. my manager has difficulty recognizing and expressing his/her good points. 18. my manager rarely stands up for his/her rights. 19. my manager can ask for what is rightfully his/her. 20. my manager can take criticism without becoming defensive. 21. my manager sometimes become physically or verbally abusive when he/she is angry with someone or when criticizing someone. 22. my manager can easily express positive feelings about other people and what their actions. 23. my manager cannot comfortably start or carry on conversations. 24. my manager usually feels good at first about getting his way but feels guilty later as a result of how he/she went about it. to determine your characteristic style of communication, count the number responses in each of the three categories. responses to items 3, 5, 6, 11, 15, 17, 18, and 24 indicate unassertive communication. " responses to items 2, 4, 7, 9, 12, 14, 21, and 23 are characteristic of aggressive communication. responses to items 1, 8, 10, 13, 16, 19, 20, and 22 signify assertive communication. australasian journal of information systems agarwal 2019, vol 23, research on the role of tech in workforce mgt supervisors’ communication style & cyberloafing 26 psychological capital psychological capital please indicate the extent to which you agree with the following statements “about yourself”: 1= strongly disagree -2 disagree – 3= somewhat disagree – 4=somewhat agree – 5= agree – 6= strongly agree 1 2 3 4 5 6 1. i feel confident in representing my work area in meetings with management. 2. i feel confident contributing to discussions about the company's strategy. 3. i feel confident presenting information to a group of colleagues. 4. if i should find myself in a jam at work, i could think of many ways to get out of it. 5. right now i see myself as being pretty successful at work. 6. i can think of many ways to reach my current work goals. 7. at this time, i am meeting the work goals that i have set for myself. 8. i can be “on my own” so to speak at work if i have to. 9. i usually take stressful things at work in stride. 10. i can get through difficult times at work because i've experienced difficulty before. 11. i always look on the bright side of things regarding my job. 12. i’m optimistic about what will happen to me in the future as it pertains to work. australasian journal of information systems agarwal 2019, vol 23, research on the role of tech in workforce mgt supervisors’ communication style & cyberloafing 27 cyberloafing cyberloafing please indicate how often you engaged in activities listed below during working hours (1= never to 5 = constantly) 1 2 3 4 5 1. browsing sports related web sites 2. browsing investment related web sites 3. browsing entertainment related web sites 4. browsing general news sites 5. browsing non-job related web sites 6. browsing download non-work related information 7. browsing shop online for personal goods 8. browsing adult-oriented (sexually explicit) web sites 9. checking non-wok related e-mail 10. sending non-work related e-mail 11. receiving non-work related e-mail copyright: © 2019 agarwal. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems clapham et al. 2021, vol 25, research on indigenous use of digital support for indigenous research methodologies information and communication technologies 1 digital support for indigenous research methodologies kathleen clapham university of wollongong, wollongong, australia kclapham@uow.edu.au helen hasan university of wollongong, wollongong, australia hasan@uow.edu.au bronwyn fredericks university of queensland, brisbane, australia dawn bessarab university of western australia, perth, australia peter kelly university of wollongong, wollongong, australia valerie harwood university of sydney, sydney, australia kate senior university of newcastle, newcastle, australia marlene longbottom university of wollongong, wollongong, australia elizabeth dale university of wollongong, wollongong, australia abstract research undertaken by outsiders into issues of concern to aboriginal communities frequently ignores community culture and the knowledge embedded within aboriginal communities. methodologies are adopted which perpetuate the colonialist mindset of non-indigenous australians leading to failed solutions to aboriginal problems. this paper describes an aboriginal-led community-based research project, exploring the role of aboriginal australians in caring for, and transforming, their own communities. it focuses on the roles that information systems can play when providing an accessible platform for aboriginal voices. the authors conducted an in-depth case study of one aboriginal community controlled organisation (acco), the illawarra koori men’s support group (ikmsg). the research consisted of a social network analysis (sna) of the inter-organisational links of the ikmsg; interviews and focus groups with members of the ikmsg and the co-design of their first website. the prominence of the ikmsg in the sna maps suggests that its work in the community is highly respected and that the model produced by this research can act as a guide for success in other accos. the findings have been used to develop a theoretical model of aboriginal community engagement and intervention. this model can enable authentic outcomes to projects which address aboriginal concerns and support the conduct of community-led research in aboriginal communities. keywords: indigenous research methodologies, digital support, community participation, aboriginal community controlled organisation, social network analysis australasian journal of information systems clapham et al. 2021, vol 25, research on indigenous use of digital support for indigenous research methodologies information and communication technologies 2 1 introduction research into issues of concern to aboriginal communities is frequently undertaken by ‘outsiders’ who are ignorant of the profound knowledge embedded within aboriginal communities and indifferent to local community culture (martin, 2008). the continuance of the deficit mindset, which focusses on problems rather than potential, contributes to the long history of failed solutions to ‘aboriginal problems’ (hollinsworth, 2013; fforde et al., 2013; fogarty, 2018; gorringe et al., 2011). many of the challenges faced by indigenous communities throughout the world are perpetuated by the colonialist narrative (smith 2012); this also persists throughout the research domain and shapes accepted research methodologies. indigenous research methodologies have been developed to disrupt this narrative. indigenous scholars such as wilson (1998), bessarab and ng’andu (2010), martin (2003), smith (2012) and walter and moreton-robinson (2010) demonstrate the efficacy of such methodologies when conducting research into indigenous issues and where the findings of the research underpin decision-making impacting aboriginal futures. the components of indigenous research methodologies include the co-design of research, training community researchers and involving them in community research, and the use of relational data collection methods such as ‘yarning’ (bessarab & ng’andu, 2010). these approaches result in community-led solutions to indigenous challenges that are viable and sustainable (see for example fredericks, 2016; fredericks et al., 2011; walker et al., 2013). in this paper we propose that the application of suitable digital tools within such methodologies can support authentic data collection and analysis as well as the meaningful display of findings and dissemination of results to community. we support this proposition with findings from aboriginal-led, community-based research which explored the role of aboriginal australians in caring for, and transforming, their own communities. the authors conducted an in-depth case study of one aboriginal community controlled organisation (acco) and the illawarra koori men’s support group (ikmsg), based in south eastern new south wales (nsw). the study, part of a wider caring for community project, examined the positive impact of the ikmsg on the health and wellbeing of the aboriginal men and families in the illawarra region. over the past decade, the ikmsg has provided placed-based culturally based programs that address issues of concern to aboriginal men and their families (ngarruwan ngadju, 2017a). the research was conducted by a multidisciplinary team of aboriginal and non-aboriginal researchers, together with members of the ikmsg and other community stakeholders. the results of the study, presented in a report given to the community (ngarruwan ngadju, 2017b), show, firstly, how accos can leverage the knowledge inherent in communities to build resilience and create better futures; and secondly, how real participation and inclusive decision making, provide the authority that enables change. in this paper, we report on the suitability and efficacy of the digital tools adopted for the study and their potential support for indigenous research methodology more generally. we argue that digital tools can (i) support many activities in the conduct of research in aboriginal communities; (ii) enable authentic outcomes to projects which address aboriginal issues and concerns; (iii) help in the management aboriginal knowledge; and (iv) provide a platform for aboriginal voices. australasian journal of information systems clapham et al. 2021, vol 25, research on indigenous use of digital support for indigenous research methodologies information and communication technologies 3 digital tools which consisted of a social network analysis (sna) of the inter-organisational links of the ikmsg, were utilised to support data collection throughout this research project. this occurred in parallel with qualitative interviews and focus groups with members of the ikmsg, and the co-design of their first website. the findings from the ikmsg case study were used to develop a theoretical model of aboriginal community engagement and interventions. the prominence of the ikmsg in the sna maps suggests its work is highly respected in both aboriginal and non-aboriginal local organisations. the central position of ikmsg in the sna maps provided authenticity to the model produced by this research so that it is widely accepted as a guide for designing programs in other accos undertaking community programs with aboriginal men. we suggest that certain contemporary digital technologies are eminently suited to data collection and analysis using indigenous research methodologies because these can be used to connect with, and be reflective of, the importance that indigenous peoples place on relationality (wilson, 1998). in our research, sna is one such technology that can be used to assist with the analysis and depiction of vital relationships within networks of accos and partner organisations. in the context of the partnership between local accos and the university-based researchers, we also found that online forums such as facebook, websites and blogs for knowledge transfer and for dissemination of the findings of the research, both inform community and provide an opportunity for their reflective comments, questions and suggestions. 2 background there is a substantial body of research in the disciplines of social work (bessarab et al., 2014), education (armstrong et al., 2012; sarra, 2011; vigo, 2017), health policy (fogarty et al., 2018) and media studies (mccallum et al., 2012) that illustrate the negative consequences of the deficit mindset resulting from the colonialist narrative among non-indigenous australians for the aboriginal communities they supposedly aim to improve. the past two decades has witnessed an emergence of a body of scholarship that is led by indigenous scholars from canada, the united states of america, australia and new zealand and elsewhere. this has established a distinct indigenous research paradigm (wilson, 1998; bessarab & ng’andu, 2010; denzin & lincoln, 2008; martin, 2003; smith, 2012; walter & moreton-robinson, 2010; williams et al., 2021). in her seminal work, decolonising methodologies, maori academic linda tuhiwai smith (smith 2012) identifies the negative history associated with research and harmful impacts that indigenous peoples have experienced since colonisation. australian aboriginal academic karen martin (2003) identifies four principles that underlie the practice of research using indigenous methodologies: firstly, recognising aboriginal worldviews, knowledge and realities as distinctive and vital to aboriginal existence and survival, and serves as a research framework; secondly, honouring aboriginal social mores as essential processes through which aboriginal people live, learn and situate themselves in their own lands and in the lands of other aboriginal peoples; thirdly, emphasising the social, linguistic, cultural, historical and political contexts which shape aboriginal experience, lives, positions and futures; and fourthly, privileging the voices, experiences and lives of aboriginal people and aboriginal lands. work on specific data collection methods is more recent. in qualitative methods, yarning has been developed as a decolonising methodology in the work of bessarab and others (bessarab australasian journal of information systems clapham et al. 2021, vol 25, research on indigenous use of digital support for indigenous research methodologies information and communication technologies 4 & ng’andu, 2010; lin, green, & bessarab, 2016; walker et al., 2013). decolonising ideas have also been applied within quantitative methods; aboriginal scholars have developed aboriginal statistics, using the terminology of data sovereignty to challenge the deficit narrative through the collection and analysis of numerical data (walter & suina, 2018; walter & andersen, 2013). many of these writings draw more broadly on critical social theories, for example walter and moreton-robinson (2010). indigenous standpoint theory (foley, 2003; moreton-robinson, 1999; nakata, 2007) and indigenous feminist theory (moreton-robinson, 2000; fredericks, 2009), provide new directions in the development of methodology which enables aboriginal researchers to critically re-assess the corpus of knowledge written about them; and for nonindigenous researchers, it provides a framework for understanding their role, their potential, and their limitations within the power relations that comprise the ‘cultural interface’ (mcgloin, 2009). as researchers, we are acutely aware of the power differentials that can be assumed by university institutions and the conduct of research, thus we wished to provide an example of a successful co-designed aboriginal community-led research project. our project adopted a strengths-based approach that identified the enormous skills within the community and draws on the resilience of those communities (bessarab & ng’andu, 2010; lin, green, & bessarab; 2016; hunt & smith, 2018).our study assumes that research into matters involving aboriginal peoples’ interests and concerns is best led by aboriginal researchers, adopt indigenous research methodologies, ensure aboriginal intellectual sovereignty, are grounded in indigenous realities, and include aboriginal ways of communication (rigney, 2006). 3 the digital divide and the non-indigenous led initiatives as background to the suitability of digital tools that can be used to support indigenous research methodologies we review the extant literature on aboriginal people’s use of digital technologies the majority of which address (1) digital exclusion as a consequence of the digital divide due to remoteness and/or disadvantage between indigenous communities and the broader community; (2) the prodigious use of social media for advocacy, mental health and other support systems; and (3) adoption of multimedia tools to preserve and express cultural heritage. while many indigenous people live in remote areas of the country and remain digitally excluded from the national communications infrastructure (australian bureau of statistics (abs), 2017; australian communications and media authority (acma), 2008; rennie et al., 2019) most of the indigenous population lives in urban or regional locations. however, even for communities in urban areas, disadvantage can also limit access as depicted in the latest australian digital inclusion index (https://digitalinclusionindex.org.au/). despite the belief that information and communications technology (ict) can overcome the tyranny of distance and disadvantage, the lack of affordable and reliable internet connectivity in aboriginal communities in australia remains an issue. for example, a project to provide opportunities for learning in country was compromised by lack of access to technology and inconsistent connectivity (vodic et al., 2012). most literature on digital divides presumes that a material connection to the network society is inherently beneficial, this is not necessarily so for aboriginal peoples where integration into australasian journal of information systems clapham et al. 2021, vol 25, research on indigenous use of digital support for indigenous research methodologies information and communication technologies 5 existing systems controlled by state and corporate entities is not in and of itself adequate. for example, karrippanon (2016) comments that media enabled health interventions are widely used to breach the problems of distance in remote aboriginal communities, but that little consideration is given to the meaning of the content and to allow a two-way conversation on the material that is presented. as a result, the interventions remain expert driven and top down in their approach despite the democratising potential of the technology. due to the cost and technical expertise involved, the creation of the infrastructure needed for rural, regional and remote internet access has been government driven and government funded. latukefu (2006) reports a case where ict only appears to remove centralised control from government and into the hands of marginalised communities but, in reality, does not. latukefu (2006) observes that struggles for self-determination play a more complex sociotechnical role in the drive to bring the information age to aboriginal communities in australia. for over two decades, community informatics has been concerned with the use of information and communication technology (ict) to enable access and assist aboriginal communities to overcome “digital divides” both within and between communities (stillman & linger, 2009; gurstein, 2007). while this study promoted the removal of centralized control of ict out of the hands of government and into the hands of the people (ibid) there is a dearth of research which addresses the core concern of aboriginal sovereignty and control. of the few projects that involve indigenous people, most early studies published in the 1990s and 2000s, were conducted within a non-aboriginal paradigm and led by non-aboriginal researchers. other studies show that many aboriginal australians have utilised the internet from its early days despite the economic, social, cultural and geographic factors that can affect their access (rice et al., 2014; clapham & gosden, 2001). in a canadian study, mcmahon (2014) notes how more successful outcomes can be achieved by the “first mile” approach. when the community end is seen as the last-mile, the differences between rural and urban communities are framed as ‘problems’ or ‘shortcomings’ to be addressed by simply linking unserved communities to already-existing urban systems and infrastructures. in contrast, the community as the first mile approach implies that decision-making about internet development must be grounded in, and emerge from, the specific needs of local communities. the first mile perspective supports proponents of aboriginal self-determination where development processes are grounded in a recognition of the inherent laws, institutions, and practices of sovereign aboriginal peoples (borrows, 2010). 4 digital technologies and community-led initiatives in one of the earliest examples of aboriginal led research into community informatics, turk and trees (2000) investigated how aboriginal community processes may be facilitated through the use of information systems which are developed via a highly participative methodology. they describe the successful ieramugadu cultural information system project designed to develop procedures for elicitation, analysis, storage and communication of aboriginal cultural heritage information with an aboriginal community in western australia. more recently social media has supported interaction among people who create and share information and ideas in virtual communities and networks. social media use in australia has grown rapidly among aboriginal population with increase in ownership and access to computers, smart phones, tablets and the internet (carlson, farrelly et al., 2015). a review of australasian journal of information systems clapham et al. 2021, vol 25, research on indigenous use of digital support for indigenous research methodologies information and communication technologies 6 literature on social media and aboriginal youth in australia (rice et al., 2016) provides an insightful overview of its impact on the empowerment of aboriginal voices. they report that in 2014, 42% of australians used facebook, while aboriginal and torres strait islander australians use of facebook was 60%. given the mobility of aboriginal people, facebook in particular serves as a platform for aboriginal young people who have moved to reconnect with their aboriginal identities through online groups where people can share their ideas, thoughts, events, music and photos and through connecting with friends and family. facebook acts as a modern site for kinship connectivity and continuity and may help reinforce young people’s mental health and wellbeing. an examination of these political and social connections forged via digital tools reveal that aboriginal movements are grounded in aboriginal ways of knowing and understanding the world (wilson et al., 2015). carlson, farrelly et al. (2015), for example, reports on a study that examines the ways in which aboriginal people utilise and interact on social media to assist with suicide prevention strategies. she found that aboriginal people engage with facebook to both seek and offer help in relation to preventing suicide and self-harm. she reports that many of the mainstream social risk factors for suicide do not appear to be relevant to aboriginal people, and that current suicide prevention measures, inappropriately based on western cultural understandings of and attitudes towards suicide and self-harm behaviour and prevention, are therefore ineffective. carlson’s research points to the need to understand aboriginal suicide in its cultural context, rather than uncritically imposing western notions of health and well-being. rice et al. (2016) identified several major themes about how and why aboriginal people use social media: identity; power and control; cultural compatibility; and community and family connections. the familiarity that many aboriginal people have with modern technology is reported to give them a sense of fearlessness and control when approaching the use of new platforms. their confidence in the use of social media suggests that it may also be a useful tool for data collection in research. the multimedia nature of the medium lends itself to the orally and visually focused cultures of aboriginal communities rather than western-based literacy. the platforms do not demand a high level of literacy who, with an intuitive understanding of the technology can create content. social media provides opportunities for the transmission of indigenous knowledge across boundaries, for example, knowledge about aboriginal people for non-aboriginal people enables the latter to better understand aboriginal cultures, perspectives and politics. social media can enable young and old aboriginal people to reconnect and understand each other better through collaborative efforts between the generations. kral (2010) comments that through their command of social media, young people are positioned to take a mediating role between cultural knowledge and new digital technologies. 5 digital tools for indigenous research methodologies our review of is related literature identified a community-based research methodology adopted by brereton et al. (2014) for a project investigating technology use in a remote australian aboriginal community. the approach is based on the concepts of participatory design (pd) and co-design from the field of human-computer interaction (hci) as an alternative to traditional ethnography. pd relies on partnership with participants in which participants bring essential knowledge of their own context and culture while designers bring australasian journal of information systems clapham et al. 2021, vol 25, research on indigenous use of digital support for indigenous research methodologies information and communication technologies 7 technical and design facilitation skills creating opportunities for mutual learning and development. a pd approach, also called co-design, can be adopted to facilitate digital data collection by trained community researchers on phone and tablet in the form of text, audio, image or video for studies of aboriginal communities. innovative research dissemination is evident in projects to digitally preserve and communicate aboriginal knowledge and, with digital empowerment, to “write ourselves back into history.” (bidwell & radoll, 2007). such projects have historically required partnerships between aboriginal and non-aboriginal expertise. for example, the digital history project, gugu badhun people worked with aboriginal studies researchers to help design and technologists to implement the interface to their video-history archive. as information systems become easier to use and aboriginal technical skill grows, aboriginal technological emancipation will dispense with the divisive praxis of us and them by empowering people as beings for themselves within the process of design (bidwell & radoll, 2007). through our literature review it was evident a case study approach was needed. 6 the case study context the broad aim of the study was to explore how aboriginal community controlled organisations (accos) contribute to improving the health and wellbeing of aboriginal people. to achieve this, we sought to: identify the factors that facilitate the operation of the ikmsg; provide evidence of the effectiveness of its programs, networks and partnerships; examine its impact on the health and wellbeing of the local aboriginal community; ascertain how it is sustained over time; and use indigenous research methodologies and theory to interpret this case. the ikmsg was initiated and is directed by aboriginal men (the members) living in the illawarra area with aboriginal elders at its helm and supported by the local community. they aim to provide culturally appropriate education programs and services to aboriginal men and male youth living in the illawarra, focusing on areas of concern which impacts not only the physical but also the social, emotional and spiritual well-being of the whole community. two groups of people participated in the research: aboriginal and non-aboriginal stakeholders, who were nominated by the ikmsg as being part of their organisational network; and ikmsg members. we used mixed methods research that involved conducting a survey and interviewing members of many of the organisations that partner with the ikmsg. focus groups were also conducted with ikmsg members. our discussions focused on the way in which aboriginal people experience and make meaning of their journey through the health system. the research was conducted in the illawarra region, a long coastal region located around 70 kilometres south of sydney, new south wales, australia. at the most recent census the region had approximately 8,364 aboriginal people, making up 2.8% of the total illawarra population. the aboriginal population is a ‘young’ population, with the average age of aboriginal people being 21 years of age, compared with 40 years of age in the non-aboriginal population (abs, 2017). the ongoing impacts of colonisation in australia are evident in the multiple levels of disadvantage which continue to be experienced by generations of aboriginal people across the region, in terms of living in areas of relative socio-economic disadvantage, higher levels of employment, lower levels educational attainment and higher levels of chronic and other health conditions (university of wollongong indigenous multi-disciplinary health research australasian journal of information systems clapham et al. 2021, vol 25, research on indigenous use of digital support for indigenous research methodologies information and communication technologies 8 coalition, 2015). this over-representation disadvantage must be understood in terms of the result of historical and ongoing marginalisation of aboriginal people within the region, rather than as any individual or community ‘deficit’. from this standpoint, the fact that aboriginal people are disproportionately represented in places of high disadvantage, higher unemployment, lower incomes and where children and young people typically experience poorer educational and health outcomes and poorer access to services, is an indictment on the settler state and not on aboriginal people. 7 methodology 7.1 approach and ethical stance to ensure the research conformed to these ethical principles for conducting research with australian aboriginal peoples (australian institute for aboriginal and torres strait islander studies, 2012), the research team worked with a project steering committee to develop a dissemination plan to ensure that the illawarra koori men’s support group received feedback on the meaningful results of the research. the researchers acknowledge that all aboriginal communities are unique and have their own individual set protocols. the illawarra koori men’s support group gave ongoing support to the researcher from inception to the reporting of results. the researchers acknowledged the wisdom, knowledge and experiences of aboriginal people living in the illawarra region. researchers followed protocols to ensure all stakeholders had an equal voice in the project’s governance and operations, and that that any cultural knowledge obtained under this research project would be acknowledged. the researchers also had a full understanding of the responsibilities in undertaking such an aboriginal community driven research project and provided participants with all necessary information within a safe environment. the research project intended to capture and promote the importance aboriginal organisations and aboriginal leadership sustain themselves in the face of complex and challenging social health problems and ongoing policy change. ethical approval for the research was obtained from the university human research ethics committee and the aboriginal health and medical research council of nsw (ethics number 1121/15). 7.2 methods faithful to the theoretical underpinning and aboriginal research approach and following a review of current literature on the role of aboriginal community controlled organisations, an exploratory mixed-methods case study was conducted over a 12-month period. data was collected from multiple sources. an online survey stakeholders and social network analysis of 18 organisational representatives was undertaken to investigate the connections of the ikmsg with other organisations within the region. semi-structured interviews and focus group discussions with members of the ikmsg (men enrolled in ikmsg programs) were conducted by aboriginal researchers who were sensitive to the ethical considerations described above. we also worked collaboratively with group coordinators on the co-development of website and social media. 7.3 qualitative analysis all of the qualitative data collected from the interviews and focus groups was entered into nvivo software (qsr international pty ltd, 2012). a project was set up in the nvivo computer program and interview transcripts were imported into the program along with researcher australasian journal of information systems clapham et al. 2021, vol 25, research on indigenous use of digital support for indigenous research methodologies information and communication technologies 9 notes and observations program data, documents and literature. the audio recorded interviews were transcribed professionally. interview transcripts and tapes were securely stored on password protected storage space on the researchers’ server. for the analysis of the semi-structured interviews, the transcribed interviews were initially auto coded based on the direct question asked in the interview schedule, and then analysed thematically to provide a more grounded analysis of this data. based on an initial analysis of the answers to this question a framework was developed for ongoing coding. the initial coding framework was around the following codes: emergence; current operations; community context; programs; clients; sustainability; networks; relationships; values; community control; and leadership. for the three focus groups a thematic analysis was undertaken exploring the personal experiences of members’ participation in group programs and activities across the broad areas of enquiry (getting involved; transforming men’s lives; impact on community; and collaboration and partnerships). 7.4 social network analysis sna is a method of mapping and analysing networks of organisations or individuals. the patterns of interaction are explored to explain the engagement between organisations or individuals in the network including where the concentrations of trust and power lie. in this study the ikmsg’s inter-organisation network is examined by means of 1) a visual analysis and 2) sna measures. in the visual analysis, sociograms are used to map the organisations which have relationships (links) with the organisation being studied and to show how these organisations are linked. nodes (circles or squares) represent the organisations in the network, the lines represent the links (relationships), and the arrows show the direction of relationship in the network. lines contain information about which organisations initiate and receive transactions, such as funding. for example, funding may be given by organisation a and received by organisation b. the primary measures used in the study to explore the interactions between organisations are all related to centrality. centrality tells us who and how important the organisation is in the network. the three centrality measures used to analyse the network are degree, between-ness and closeness as follows. degree centrality are connectors (hubs) in the network identified as those organisations with links to the greatest number of organisations in the network. degree centrality indicates status and trust and is calculated as the number of links/ties as a proportion of the number of organisations in the network in-degree and out-degree count the number of relationships to and from organisations, respectively (freeman, 1979). in-degree (connected-trusted) is used to identify connectors (hubs) and measure collaboration in a network (valente, 2010). a high in-degree indicates that an organisation plays an important role in the network (valente, 2010). they are the most trusted for information and partnerships (namely “funding”, “service delivery” and “culture”) (long et al., 2013). a low in-degree would indicate that an organisation is connected to few organisations in the network. australasian journal of information systems clapham et al. 2021, vol 25, research on indigenous use of digital support for indigenous research methodologies information and communication technologies 10 out-degree (connected-influential) measures the number of links (relationships) directed away from an organisation in the network (freeman, 1979). a high out-degree indicates an organisation is connected (linked) to many organisations, and a low out-degree would indicate that an organisation has few links. organisations who have unusually high out-degree are organisations that are able to exchange with many others or make many others aware of their views (valente, 2010; long et al., 2013). for this reason, they are often said to be influential diffusers of knowledge, attitudes and behaviours (hanneman & riddle, 2005). betweenness centrality powerful organisations in the network were identified as those organisations who were acting as gatekeepers or brokers of knowledge, attitudes and behaviour (long et al., 2013). betweenness measures the extent to which an organisation lies on the shortest path (geodesic distance) to organisations that are not connected (freeman, 1979). the greater the betweenness centrality the more control that organisation has in passing and sharing information. brokers can add substantive value to networks by generating new ideas and making advice and knowledge more accessible to others. broker organisations act as gatekeepers controlling the information passing into and out of their group or network (long et al., 2013). they may hoard information, but they are generally positively associated with diffusing useful information (long et al., 2013). closeness centrality (influential) closeness centrality is a measure of the average shortest distance from each organisation to each other organisations (freeman, 1979). a lower closeness centrality score indicates a more central therefore important position as a diffuser of knowledge, attitudes and behaviours in the network. a closeness centrality score is equal to one would indicate an organisation is directly connected to everyone in the network, good position to initiate the diffusion of information through the network. centrality scores indicate how quickly an organisation can interact and communicate with others in the network of knowledge, information and/or resources (borgatti, 2005). the study used an ego-centric model which looks at all relationships in a single bounded network of organisations (hawe, webster, & shiell, 2004). data collection was conducted through an online survey. ucinet 6tm and netdraw 2.138tm were used to compute and analyse the data and create the network maps. organisations (n=24)* type of organisation sector target population aboriginal organisation 7 education 6 non-aboriginal 11 government organisation 11 employment 3 aboriginal population 13 non-government organisation 4 families and community 6 higher education 2 health 5 justice 3 housing 1 table 1. characteristics of organisations represented in the study note: the number of organisations was 24; the divisions of a number of large organisations were included. australasian journal of information systems clapham et al. 2021, vol 25, research on indigenous use of digital support for indigenous research methodologies information and communication technologies 11 8 results 8.1 participants thirty-six people were identified by ikmsg as stakeholders and were subsequently approached to participate in the study. these stakeholders were based on the ikmsg’s current and past organisational relationships. a total of 21 stakeholders participated in the research by completing a survey and/or semi-structured interview (58%). of the stakeholders who did not participate, three indicated that their organisation was already represented by another survey respondent, nine did not respond to the email and phone invitations; one person was no longer employed by the organisation, and two declined to participate. table 1 describes the demographic characteristics of the 21 stakeholders who participated in the study. a total of 24 organisations or divisions of organisations (including ikmsg) were represented by the survey respondents and included in the network. as seen in table 1, these organisations provide services across a wide range of sectors within the illawarra region. 8.2 results: social network analysis in the sna network four organisations (nodes) were accos, five were non-government organisations and the remaining were government organisations. the types of connections listed were: having a memorandum of understanding (mou); funding relationship; worked together on projects; working together on meetings; and cultural connections. these relationships were either one way or reciprocal. all members of the network were engaged in projects and all but one were involved in meetings. there were twelve mous between organisations. there were 11 organisations involved in cultural activities and nine organisations engaged in the exchange of funding. the most relevant results for this study are now shown. these are in-degree centrality, which is an indication of trust, and between-ness, which is an indication of power/and what? 8.2.1 in-degree centrality degree centrality is a measure of the links an organisation has to other organisations in the network. in-degree counts the number of relationships to an organisation. from figure 1 (below) we can see three of the four accos including the ikmsg have the most connections in the network being highly trusted and respected organisations. 8.2.2 between-ness centrality between-ness centrality is measured by the number of times an organisation acts as a path between two organisations. figure 2 (below) shows between-ness centrality with the largest square, ikmsg, having the highest between-ness centrality in the network. the study found that accos were highly involved in all activities which were part of the network. the accos were also highly networked as is demonstrated by their central location in the network and by being visually displayed as larger players in the measurements of degree, between-ness and closeness centrality measures. figures 1 and 2 show the prominence of the ikmsg in the sna maps suggesting that its work is highly respected in both aboriginal and non-aboriginal local organisations giving validity to the model produced by the qualitative analysis. australasian journal of information systems clapham et al. 2021, vol 25, research on indigenous use of digital support for indigenous research methodologies information and communication technologies 12 the sna revealed that the ikmsg was central in the provision of a wide range of social, health-related and cultural activities. however, despite all the high activities within the network by accos, there was little funding directed to these organisations compared to non government organisations within the network. figure 1. network map measuring degree centrality figure 1. between-ness centrality australasian journal of information systems clapham et al. 2021, vol 25, research on indigenous use of digital support for indigenous research methodologies information and communication technologies 13 8.3 results: knowledge management and dissemination prior to our project commencing university of wollongong students worked with the ikmsg to set up a facebook page and trained one member of the group to administer the page. when used in this manner, facebook becomes a tool that instantiates aboriginal standpoint theory (nakata, 2007; foley, 2003; carlson, jones et al., 2015) and the concept of the cultural interface. rice et al (2016) have reported how aboriginal young people have embraced social media such as facebook to produce their own unregulated content with limited technical skills but an intuitive understanding of what the technology affords in allowing them to tell their own story. an early meeting between the ikmsg and the research team identified the value of having an online space for knowledge transfer between the ikmsg, the researchers and the community. it was decided that one of the researchers would work with one member of the ikmsg to create a website for the ikmsg. following the success of the facebook page, the content of the page would be created and maintained by members of the ikmsg. all agreed on different and distinct roles for the facebook page and the website. facebook would continue to post photos and news of ikmsg events. the website would describe the different projects and activities of the ikmsg to the community, which included local aboriginal and nonaboriginal audiences, and present a positive image to potential sponsors of the ikmsg projects where funding had become a real problem. in turn the researchers set up their own blog as one way to transfer knowledge on the outcomes of the research to the non-academic community. leveraging the strong partnership between the ikmsg and the research team, the ikmsg website and facebook page were co-developed and maintained throughout the project. the activity of the ikmsg facebook page consists of posts of labelled videos and images of significant activities of the group with 266 followers among the community. since the page was established 2.5 years ago 6 videos have had 415 views an average over 200 likes each. 250 photographic images have been posted with over 1000 likes. while there is little other activity, the regular postings provide a means by which the community shares knowledge of current events. (see https://www.facebook.com/ikmsg) the website has been live for almost two years and contains information about the ikmsg and its regular ongoing programs and provides useful health, social and other information to its members and the broader community. there is little need to update the page which is only done on a 6-monthly basis. on the research side a blog was set up on the university website providing updates to research projects and providing access to both community and research reports 8.4 results: the acco model in the introduction to this paper, we reported that the results of the study show (1) how accos can leverage the knowledge inherent in communities to build resilience and create better futures; and (2) how real participation and inclusive decision making, provide the authority that enables change. qualitative analysis of interviews, supported by the findings of the sna, show that the ikmsg has operated successfully for the past ten years. in the 2016-2017 reporting year a total of 2770 people attended the ikmsg and 21 local services regularly drew on its programs. over the australasian journal of information systems clapham et al. 2021, vol 25, research on indigenous use of digital support for indigenous research methodologies information and communication technologies 14 past decade the group has gone from strength to strength, and it is one of the most highly respected aboriginal organisations in the region. the group is guided by a clearly articulated set of principles and values. it has strong leadership and has developed a governance structure which places a high value on respect, the role of elders, and commitment to developing programs in response to community need. the model emanating from this research provides evidence for the multiple benefits of the strong presence of aboriginal elders as advisors, role models for young men and sources of cultural knowledge within the education system. the respect that people have for the elders has a positive impact all members of the men’s group. while a full presentation of the results of the qualitative research is published elsewhere (ngarruwan ngadju, 2017a) the following words of one participant is compelling: i think when you see the uncles out and they walk around, it shows a lot of honour as well, like honour towards like we're respectful towards our women, respectful towards our culture and, you know, it shows the wider view of what we want to present ourselves as honourable men out in the community and i think we bring a lot of honour and respect to the uncles that are here, and the young generation. (focus group participant) the research has identified ten key elements of success which underpin the ikmsg model as shown in figure 3 (below). the depiction of the dominant presence of the ikmsg on the sna map was interpreted by members of the community as giving authenticity to this model as a successful and influential acco, ikmsg is uniquely able to provide a range of unique culturally focused services for the aboriginal community in the illawarra. the success of ikmsg in this important work is directly due to its identity as an acco. these services cannot be replicated by mainstream organisations. the ikmsg is a part of the world leading efforts of accos to deliver best practice services that are underpinned by principles of self-determination. 9 discussion and conclusion this paper addresses issues of concern to local aboriginal communities. the investigation was led by indigenous researchers in partnership with community participants. indigenous participatory research methodologies were adopted and supported by suitable digital tools so that the investigators had access to the knowledge embedded within the aboriginal communities. this gives us confidence in the quality of the collected data and its analysis, leading to findings of authentic theoretical and practical value. the use of digital tools in an interdisciplinary context provided a unique lens and led to findings that were otherwise unavailable. importantly they provided a way of understanding the organisation in a way that had meaning for the organisation and the researchers. what is it about these digital tools that provides information that is unique and use for this organisation? returning to the previously referenced principles of indigenous research methodologies identified by martin (2003), this research project recognised aboriginal worldviews, knowledge and realities as distinctive and vital to aboriginal existence and survival, as a research framework in this study. the sna analysis provides an example of the roles that information systems can play to support the conduct of research in aboriginal communities. social networks are deeply australasian journal of information systems clapham et al. 2021, vol 25, research on indigenous use of digital support for indigenous research methodologies information and communication technologies 15 embedded in indigenous society and indigenous knowledge; relationships, reciprocity and respect are central to the way indigenous communities relate to each other. in many ways the sna validated what was already known; importantly though, it provided a visualisation of networks and relationships which replicates the modes of communication and reciprocal networks that are so fundamental to the way aboriginal organisations work. but the sna analysis also highlighted the large discrepancy between the longstanding networks and the relative lack of economic support for the organisation. 1. leadership leadership from a group of passionate, well respected and well-known elders and aboriginal community leaders has enabled the ‘coming together’ of men from many walks of life. 2. key personnel key personnel (board members, staff and volunteers) have the requisite skills and capacity to design, develop and implement programs. 3. strong values-based ethos a strong, values-based ethos of trust, support, confidentiality and of being role models for the younger generation, underlie the organisation. 4. holistic approach to service delivery the holistic approach to service delivery is characterised as ‘respectful’, ‘interactive’, ‘holistic’, ‘community based’, ‘healing’ and ‘inclusive’. 5. culturally safe the group provides a model for how to deliver culturally safe and meaningful programs that strongly resonate with the target audience. 6. effective responsive programs effective programs respond to areas of critical need and provide members with resources to look after their physical, emotional and social health and wellbeing. 7. a unique pedagogy a unique ‘ikmsg pedagogy’ describes the group’s capacity and skill in culturally engaging and developing healing relationships with aboriginal men. 8. highly connected the group is highly connected through its partnerships with a broad range of services and agencies within the region. 9. reconnecting men the group provides a much needed mechanism for reconnecting men with the aboriginal community. 10. elders and education aboriginal elders have a strong presence in the education system as advisors, role models for young men and sources of cultural knowledge. figure 3. the model of a successful acco emerging from the ikmsg study overall, the research revealed the unique role of the ikmsg in holistic service provision and support for aboriginal men and youth in the illawarra region. the holistic and culturally safe programs respond effectively to areas of critical need in health, education, justice and employment. the strong presence of elders in the educational programs has created a unique pedagogy with capacity and skill in culturally engaging and developing healing relationships with aboriginal men and reconnects them to community. through its strong networks and well-established partnerships with a broad range of organisations the group has a powerful australasian journal of information systems clapham et al. 2021, vol 25, research on indigenous use of digital support for indigenous research methodologies information and communication technologies 16 influence in promoting respect for elders amongst both aboriginal and non-aboriginal people within the local illawarra community. the strong consistent leadership from aboriginal elders and effective management are key factors in the success of the ikmsg over the past decade. the ikmsg model this research has developed offers a form of engagement which could benefit many other first nations communities. the elders are vitally important to the group and are recognized and highly utilised by government and community organisations for their expert cultural knowledge and community contacts. the research honoured aboriginal social mores as essential processes through which members of the ikmsg live, learn and situate themselves in their own lands and in the lands of other aboriginal peoples. it emphasised the social, linguistic, cultural, historical and political contexts which shape aboriginal experience, lives, positions and futures through the development of the ikmsg model. it strove to privilege the voices, experiences and lives of aboriginal people and aboriginal lands throughout the conduct of the research. we put this model forward as an instance of a successful acco for others to follow and is accepted as such by the community. the acco model confirms the critical importance to beneficial outcomes and sustainability that comes through aboriginal-community control of services provided within local aboriginal communities. this model is interpreted through the four aboriginal theoretical principles beginning with the recognition of the aboriginal worldviews, knowledge and realities as distinctive and vital to aboriginal existence and survival and consistently privileging the voices, experiences and lives of aboriginal people and aboriginal lands. the results of this research demonstrate the futility of ignoring the aboriginal first nation’s sovereignty and benefits of acknowledging their right to be included in decision-making that impacts aboriginal futures. empowering local aboriginal communities provides a mechanism for change. it disrupts the deficit mindset that frames aboriginal australians as undeserving and complicit in their disadvantage, to one of resilience and strength. acknowledgements we would like to acknowledge the uncle gerald brown, mr gregg potts, and mr michael willis of the illawarra koori men’s support group, for their ongoing support and collaboration through our research collaboration. we also acknowledge dr scott winch for his contribution to original research on which this paper is based. references abs australian bureau of statistics. (2017) census: aboriginal and torres strait islander population. http://www.abs.gov.au/ausstats/abs@.nsf/mediareleasesbyreleasedate/ 02d50faa9987d6b7ca25814800087e03?opendocument. retrieved 17/12/2018. australian communications and media authority. (2008). telecommunications in remote indigenous communities. canberra. armstrong, a., buckley, s., londsdale, m., milgate, g., kneebone, l.b., cook, l. & skelton, f. (2012). starting school: a strengths-based approach. towards aboriginal and torres strait islander children, australian council for educational research. canberra. australasian journal of information systems clapham et al. 2021, vol 25, research on indigenous use of digital support for indigenous research methodologies information and communication technologies 17 australian institute for aboriginal and torres strait islander studies. (2012). guidelines for ethical research in australian indigenous studies. canberra, aiatsis. bessarab, d., green, s., jones, v., stratton, k., young, s. & zubrycki, j. (2014). getting it right creating partnerships for change. integrating aboriginal and torres strait islander knowledges in social work education and practice. teaching and learning framework 2014. office for teaching and learning: australian government. bessarab, d. & ng’andu, b. (2010). yarning about yarning as a legitimate method in indigenous research. international journal of critical indigenous studies, 3(1). bidwell, n. j., & radoll, p. (2007). redisplacement by design. interactions, 14(2), 12-14. borgatti, s.p. (2005). centrality and network flow. social network, 27, 55–71. borrows, j. (2010). canada’s indigenous constitution. toronto, canada: university of toronto press. brereton, m., roe, p., schroeter, r. & lee hong, a. (2014). beyond ethnography: engagement and reciprocity as foundations for design research out here. in proceedings of the sigchi conference on human factors in computing systems (pp. 1183-1186). acm. carlson b., jones l., harris m., quezada n. & frazer r. (2015). trauma, shared recognition and indigenous resistance on social media, australasian journal of information systems, 21. carlson b., farrelly t., frazer r. & borthwick f. (2015). mediating tragedy: facebook, aboriginal peoples and suicide, australasian journal of information systems, 19, special issue on research on indigenous ict. clapham, k. & gosden d. (2001). education and indigenous health in a globalised world. aare, fremantle. denzin, n. k. & lincoln, y. s. (2008). handbook of critical and indigenous methodologies. thousand oaks, california: sage. fforde, c., bamblett, l., lovett, r., gorringe, s. & fogarty, b. (2013). discourse, deficit and identity: aboriginality, the race paradigm and the language of representation in contemporary australia. media international australia, 149, 162-173. fogarty, w., bulloch, h., mcdonnell, s., & davis, m. (2018). deficit discourse and indigenous health: how narrative framings of aboriginal and torres strait islander people are reproduced in policy. deficit discourse and indigenous health: how narrative framings of aboriginal and torres strait islander people are reproduced in policy, xii. foley, d. (2003). indigenous epistemology and indigenous standpoint theory social alternatives 22(1): 44-52. fredericks, b.l. (2009). the epistemology that maintains white race privilege, power and control of indigenous studies and indigenous peoples' participation in universities. australian critical race and whiteness studies association ejournal, 5(1), 1-12. fredericks, b., adams, k., finlay, s., fletcher, g., andy, s., briggs, lyn, briggs, l. & hall, r. (2011). engaging the practice of indigenous yarning in action research. action learning action research journal. 17 (2), 9-20. australasian journal of information systems clapham et al. 2021, vol 25, research on indigenous use of digital support for indigenous research methodologies information and communication technologies 18 fredericks, b., longbottom, m., mcphail-bell, k. & worner, f. in collaboration with the board of waminda. (2016). dead or deadly report. waminda aboriginal women’s health service. cq university, australia: rockhampton, 32 pages, isbn: 9781921047169. access at: http://acquire.cqu.edu.au:8080/vital/access/manager/repository/cqu:13773 freeman, l.c., roeder, d. & mulholland, r. r. (1979) centrality in social networks: ii. experimental results. social networks, 2(2), 119–141. gorringe, s., ross, j. and fforde, c. (2011). will the real aborigine please stand up?’: strategies for breaking the stereotypes & changing the conversation. aiatsis discussion paper 28. canberra, aiatsis. gurstein, m. (2007). what is community informatics? (and why does it matter), polimetrica, milan. hanneman, r.a., riddle, m. (2005). introduction to social network methods. university of california. hawe, p., webster, c. & shiell, a. (2004). a glossary of terms for navigating the field of social network analysis. journal of epidemiology and community health, 58(12), pp. 971-975. hollinsworth, d. (2013). decolonizing indigenous disability in australia. disability & society, 28(5), 601-615. hunt, j. & smith, d. e. (2018). building indigenous community governance in australia: preliminary research findings. canberra, act: centre for aboriginal economic policy research, research school of social sciences, college of arts & social sciences, the australian national university. university, national centre for indigenous studies. karripanon, k, (2016). a double edged spear: the social life of youth, mobile phones and social media in a remote aboriginal community, university of wollongong phd thesis. kral, i. (2010). plugged in: remote indigenous youth and digital culture, caepr working paper no 69/201, australian national university, canberra. latukefu, a. s. (2006). remote indigenous communities in australia: questions of access, information, and self-determination. in k. landzelius (ed.), native on the net: indigenous and diasporic peoples in the virtual age (pp. 43–60). london, uk, and new york, ny: routledge. lin, i., green, c. & bessarab, d. (2016). ‘yarn with me’: applying clinical yarning to improve clinician–patient communication in aboriginal health care. australian journal of primary health. doi:10.1071/py16051 long, l.c., cunningham f.c. & braithwaite j. (2013) bridges, brokers and boundary spanners in collaborative networks: a systematic review. bmc health service research, 13. https://doi.org/10.1186/1472-6963-13-158 martin, k. & mirraboopa, b. (2003). ways of knowing, being and doing: a theoretical framework and methods for indigenous and indigenist re-search. journal of australian studies, 27, 203-214. martin, k. l. (2008). please knock before you enter: aboriginal regulation of outsiders and the implications for researchers, teneriffe, queensland, post pressed. australasian journal of information systems clapham et al. 2021, vol 25, research on indigenous use of digital support for indigenous research methodologies information and communication technologies 19 mccallum, k., waller, l. & meadows, m. (2012). raising the volume: indigenous voices in news media and policy. media international australia, 142(1), 101-111. mcgloin, c. (2009). "considering the work of martin nakata's "cultural interface": a reflection on theory and practice by a non-indigenous academic." the australian journal of indigenous education, 38, 36-41. mcmahon, r. (2014). from digital divides to the first mile: indigenous peoples and the network society in canada. international journal of communication, (8), 25. martin, k. (2003). ways of knowing, being and doing: a theoretical framework and methods for indigenous and indigenist re-search. in k. mcwilliam, p. stephenson, & g. thompson (eds.), voicing dissent, new talents 21c: next generation australian studies (vol. 76, pp. 203-214). st lucia: university of queensland. moreton-robinson, a. (2000). talkin'up to the white woman: aboriginal women and feminism, univ. of queensland press. nakata, m. (2007). the cultural interface. the australian journal of indigenous education, (36), 7-14. ngarruwan ngadju (2017a). interdisciplinary indigenous health research group in collaboration with the illawarra koori men’s support group caring for community research report: valuing the significant contribution of the illawarra koori men’s support group. university of wollongong. ngarruwan ngadju (2017b). interdisciplinary indigenous health research group with the illawarra koori men’s support group. caring for community: community report. university of wollongong qsr international pty ltd. (2012). nvivo qualitative data analysis software; version 10 (version 10). rennie e., thomas j. & wilson c. (2019). aboriginal and torres strait islander people and digital inclusion: what is the evidence and where is it? communication research and practice, 5:2, 105-120. rice, e. s., haynes, e., royce, p. & thompson, s. c. (2016). social media and digital technology use among indigenous young people in australia: a literature review. international journal for equity in health, 15(1), 81. rigney, l.-i. (2006). indigenist research and aboriginal australia. in n. goduka, i & kunnie, j. (ed.), indigenous people’s wisdoms and power: affirming our knowledges through narrative (pp. 32-50). london: ashgate publishing. sarra, c. (2011). strong and smart – towards a pedagogy for emancipation: education for first peoples, new york, routledge. smith, l. t. (2012). decolonizing methodologies: research and indigenous peoples. (2nd edition) zed books ltd. (1st edition 1999) stillman, l., linger, h. (2009). community informatics and information systems: how can they be better connected? the information society 25(4), 255-264. australasian journal of information systems clapham et al. 2021, vol 25, research on indigenous use of digital support for indigenous research methodologies information and communication technologies 20 turk, a. & trees, k. (2000). facilitating community processes through culturally appropriate informatics: an australian indigenous community information systems case study. in community informatics: enabling communities with information and communications technologies (pp. 339-358). igi global. university of wollongong indigenous multi-disciplinary health research coalition (2015). illawarra aboriginal community profile: a snapshot of an urban aboriginal community, university of wollongong, wollongong. valente, t. w. (1996). social network thresholds in the diffusion of innovations. social network, 18(95), 69–89. https://doi.org/10.1016/0378-8733(95)00256-1 vigo, k. g. m. (2017). one country: different voices-meeting the challenge of introducing indigenous approaches to learning to non-indigenous students. creative work and exegesis. https://researchbank.swinburne.edu.au/file/92924766-2378-43d9-a4ae05d8bccb8865/1/kitty_vigo_thesis_1.pdf vodic, a., senior, k., dwyer, b. & szybiak, p. (2012). a remote chance? the potential for indigenous students to participate in online learning in remote communities in the northern territory, charles darwin university, darwin. walker, m., fredericks, b., mills, k. and anderson, d. (2013). ‘yarning’ as method and process for community-based health research with indigenous women. the indigenous women’s wellness study. healthcare for women international, 00, 1-11: 2013. walter, m. & andersen, c. (2013). indigenous statistics: a quantitative research methodology, new york, routledge. walter, m. & moreton-robinson, a. (2010). indigenous methodologies in social research (online chapter 22). in m. walter (ed.), social research methods (2nd edition), pp. 1-18. australia: oxford university press. walter, m. & suina, m. (2018). indigenous data, indigenous methodologies and indigenous data sovereignty. international journal of social research, doi:10.1080/ 13645579.2018.1531228. williams, h., hunter, k., griffin, b., kimble, r. & clapham, k. (2021). fire and smoke: indigenous research methodologies exploring psychosocial impact of paediatric burns on aboriginal and torres strait islander families. international journal of qualitative methods, 20, pp. 1-14. wilson a., carlson b. & sciascia, a. (2015). reterritorialising social media: indigenous people rise up, australasian journal of information systems, vol. 19, special issue on research on indigenous ict. wilson, s. (1998). research is ceremony: indigenous research methods. fernwood publishing. australasian journal of information systems clapham et al. 2021, vol 25, research on indigenous use of digital support for indigenous research methodologies information and communication technologies 21 copyright: © 2021 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v25i0.2885 http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 background 3 the digital divide and the non-indigenous led initiatives 4 digital technologies and community-led initiatives 5 digital tools for indigenous research methodologies 6 the case study context 7 methodology 7.1 approach and ethical stance 7.2 methods 7.3 qualitative analysis 7.4 social network analysis 8 results 8.1 participants 8.2 results: social network analysis 8.2.1 in-degree centrality 8.2.2 between-ness centrality 8.3 results: knowledge management and dissemination 8.4 results: the acco model 9 discussion and conclusion paper title (paper title style) australasian journal of information systems shrivastava & rathod 2019, vol 23, research article risk management in distributed agile projects 1 a goal-driven risk management approach for distributed agile development projects suprika shrivastava department of information systems and business analytics deakin business school deakin university australia s.shrivastava@deakin.edu.au urvashi rathod symbiosis centre for research and innovation symbiosis international (deemed university) lavale, mulshi taluk, pune maharashtra, india abstract software companies are now using distributed agile development (dad) in order to create high quality solutions, which aligns with the business priorities of lesser time and cost. although, dad is beneficial, there are significant risks involved in such projects. in order to minimize the adverse effect of risks on dad projects, it is imperative to understand, how the risks impact the project performance goals including ‘time’, ‘cost’ and ‘quality’. in this paper we present a goal driven approach for managing risks in dad projects. this approach of risk management will enable project managers to identify the most important risks with respect to the goal to be achieved and focus on managing those risks first. the study shows that if ‘time’ is a considered goal for a dad project, the most important risks that would need consideration are related to requirement management, architecture changes and coordination issues between stakeholders. similarly, if ‘quality’ is the primary performance goal in a dad project, it would be necessary to first deal with risks related to internal and external communication in the organization, team collaboration and requirement documentation availability. the awareness of top-ranking risk factors that impact a particular project goal will assist the projects managers to control the risks in a way that the desired project goals can be achieved. keywords: distributed agile development (dad), agile project management, goal driven risk management in distributed agile development, project constraints, time, cost, quality, goal driven risk management 1 introduction distributed software development (dsd) enables organizations for responding quickly to the market opportunities by employing virtual teams that can do round the clock development by exploiting the difference in the time zones (holmström et al., 2006). for reaping the benefits such as reduced time to market, increased flexibility and reduced cost, organizations have considered combining the distributed software development (dsd) and agile approaches into a common method, namely, distributed agile development (dad) (herbsleb, 2007). however, the contrast in the inherent properties of distributed development and agile approaches poses significant risks juxtaposed to the claimed gains of dad (balasubramaniam, lan, kannan, & peng, 2006; mattsson, azizyan & magarin., 2010). distributed development, australasian journal of information systems shrivastava & rathod 2019, vol 23, research article risk management in distributed agile projects 2 characterized by spatial distance, temporal distance and cultural difference (krishna, sahay & walshman, 2004) is antithetical to the agility principles of face to face communication, collocated teams, customer involvement, frequent software delivery and self-organized teams (dyba & dingsøyr, 2008). this contrast is manifested by the dad risk factors found to be emerging from poor communication among the stakeholders, lack of collaboration between the team members, low customer involvement, poor software development practices, differences in development standards among the teams and dependency among teams etc. (shrivastava & rathod, 2014; dorairaj & noble, 2013; balasubramanium et al., 2006). the inability to identify and control project risks lead to poor knowledge of requirements and the development of a low value system, which negatively impacts the project performance (wallace & keil, 2004). performance of a project is generally assessed in terms of the extent to which it could meet the goals such as the project cost, development schedule and the product/outcome quality (westerveld, 2003; islam, mouratidis & weippl, 2014). attainment of project goals needs appropriate ways to control the risk factors from obstructing the project activities. while research studies in the past have addressed the management of risks in traditional projects (islam et al., 2014; kontio, 2001), there is a lack of related work for dad projects. considering the aforesaid conflict between the basic premises of dsd and agile approaches, the risks emerging from their merger in dad approach are expected to affect the project goals and therefore, it is reasonable to find the most impactful risks experienced in practice. in this study, we determine the risk factors as perceived by the practitioners to impact project goals, ‘time’, ‘cost’ and ‘quality’ in dad projects. for this work, the researchers used the list of the dad risk factors from the earlier studies (balasubramaniam et al., 2006; herbsleb, 2007; mattsson et al., 2010; mudumba & lee, 2010) and asked the practitioners playing different roles in dad projects, through two-staged survey to rank their impact on the project goals of “time”, cost” and “quality”. the analysis of survey responses revealed that the risk factors have statistically significant perceived impact on the project goals of time and quality. this implies that the dad risks directly lead to the project delays and poor-quality releases, which in turn, may have an indirect effect on the project cost. the identified dad risk factors having significant impact on project goals are used as inputs to the goal driven risk management (gsrm) approach for managing risks in dad projects. in this research, gsrm, which was implemented by islam et al. (2014) for managing risks in traditional projects, has been introduced for managing dad risks effectively for attaining the desired project goals. this will enable the project teams to control specifically those risk factors that impede the achievement of the specific goal that is necessary to be attained for the success of the project. the work also reveals the areas of concern in dad that influence the project outcomes and it opens up the avenues for further research on devising the solutions and mechanisms for managing dad risks and deriving the relevant hypotheses, principles and theories for managing the complex phenomenon of dad. the next section describes the literature review, followed by the section 3 on research objectives. then we present the research methodology in section 4. section 5 presents data analysis and results followed by a comprehensive research findings and discussion in section 6. limitations and threats to validity are stated in section 7 before the work is concluded in section 8. australasian journal of information systems shrivastava & rathod 2019, vol 23, research article risk management in distributed agile projects 3 2 literature review 2.1 distributed agile development (dad) in the recent years, organizations are becoming globalized and developing software using virtual teams. this type of development where team members are located in remote sites during software development are called distributed software development (dsd) (jim´enez, piattini, & vizca´ıno, 2009). companies are applying agile practices in distributed environments for software development and are able to get benefits like reduce the time to market, less cost and improved quality (therrien, 2008; krutchen, 2011). although, this kind of development, also known as distributed agile development (dad), has advantages but also involves significant risks (mattsson et al., 2010; persson, mathiassen & aaen, 2012). the risks in dad projects occur due to opposing properties of agile and distributed development. dad projects are characterized by spatial, temporal and cultural difference between distributed teams which makes effective agile implementation challenging. 2.2 project risk and distributed agile development in the context of projects, risks can be defined as uncertain events or conditions that, if occur, have a positive or negative effect on at least one of the project objectives such as time, cost, scope or quality (project management institute, 2017). risk management involves, risk identification, risk analysis, developing risk response strategies, monitoring and controlling risks to determine if they have changed (kerzner, 2017). a risk factor can be defined as a situation that can present a serious threat to the success of a software development project (schmidt, lyytinen, keil & cule, 2001). number of risks are involved in dad projects like poor communication between team members and with the client, lack of trust among team members, unclear requirements and lack of documentation, design changes and consistency in development standards, difficulties in release and deployment management (mattsson et al., 2010; shrivastava & rathod, 2014; balasubramaniam et al., 2006; paasivaara m.s. durasiewicz, s., & lassenius, d.c. 2008). although, studies have explored these risks and have suggested appropriate methods to control the risks (hossain, babar, paik & verner, 2009; mudumba & lee, 2010; shrivastava & rathod, 2014; shrivastava & rathod, 2017) there is still a need to view the risks in light of the goals that define the success of the project. 2.3 risk and project performance studies have revealed that software development projects suffer due to the occurrence of risks as they lead to poor software performance, cost overruns, schedule delays, unmet requirements and production of systems giving less value to the business (wallace, keil and rai, 2004; barki, rivard, & talbot, 1993). success of a system can be measured through various dimensions including, system development process, system use, system quality, individual impact and organizational impacts (saarinen, 1996). hence system success can be classified into – business outcomes (impact to individuals and organizations), technical or system performance, efficiency (project operation in terms of cost, time and productivity), user satisfaction, is personal satisfaction (saarinen, 1996). another study by misra, kumar & kumar (2009) identified success factors for agile projects like ‘reduced delivery schedules’, ‘return on investment’ ‘meeting customer requirements’, ‘meet changing requirement’, ‘increased business processes’, which also can be transformed to the ‘time’, ‘cost’ and ‘quality’ factors. australasian journal of information systems shrivastava & rathod 2019, vol 23, research article risk management in distributed agile projects 4 project performance is defined as “the degree to which the software project achieved success in the perspective of process and product” (nidumolu, 1995). performance of a software can be measured in terms of key performance indicators including time, cost, scope and quality of software projects (agarwal & rathod, 2006; cullen & parker, 2015). in the project management context, ‘scope’ refers to the functionality of the product or a service that the project aims to deliver, ‘time’ refers to the time that will be used for completing the project, ‘cost’ is the expected cost to complete the project (schwalbe, 2016, pp. 7-9). these three criteria are also presented as constraints on the ‘iron triangle’ and are often competing in nature due to which they need to be balanced for the project to succeed (schwalbe, 2016, pp. 7-9). along with the basic elements of the iron triangle, the fourth key goal in project is to achieve good ‘quality’ and usually relates to customer satisfaction. quality in projects refers to ‘development of a product that has a measurable value for those who produce it and who use it’ (pressman & maxim, 2015). in this study we have focused on the project performance goals including, time, cost and quality because, agile teams adjust the functionality (scope) of the project in order to deliver faster and with better quality (coram & bohner, 2005), (lee & yong, 2010). risks occurring in software projects impact these goals, leading to failed or challenged projects (parker, parsons & isharyanto, 2015). the inability of the project team to identify and manage the important risks that related to the most critical project goal is a major reason for project failure. it is imperative for project teams to get a good understanding of risks when aiming to achieve a particular goal, and hence select the appropriate risk management strategies to achieve project success (wallace, keil & rai, 2004). 2.4 theoretical consideration the purpose of this work is to provide support to the community of practitioners and researchers for identifying and managing dad risks for the achievement of the success goals of the project. identification of high impact dad risks with respect to the project goals is based on the prior research (kontio, 2001; mattsson et al., 2010; islam et al., 2014; jiang & klein, 2000) and the practitioners’ perception. we have used goal driven risk management (gsrm) framework (islam et al., 2014) in order to study risks in dad projects while considering the performance goals that are important for project success. the gsrm framework adopts a goal and obstacle approach from kaos (knowledge acquisition in automated specification) goal modelling (lamsweerde, 2000). kaos is a methodology for software requirement engineering and the key concept underlying the kaos is the ‘goal’ (dardenne, fickas, lamsweerde, 1991). goal is an objective that has to be met by the composite system. a goal is nonoperational because it cannot be described exclusively in terms of objects and actions of the composite system. however, goals can be operationalized by being translated into constraints, which when resolved, contributes to the achievement of the goal (dardenne et al., 1991). an extension of the kaos goal modelling language has been used to formulate a goal driven software development risk management model (gsrm) (islam et al., 2014), for performing risk management in traditional software projects. these project goals of time, cost and quality can be operationalized in to the constraints such as ‘completing the project within the given time’ and ‘completing the project within the allocated budget’ and ‘achieving the expected level of quality level’. australasian journal of information systems shrivastava & rathod 2019, vol 23, research article risk management in distributed agile projects 5 from this perspective, we decided to operationalize the gsrm framework through this study for identification of the dad risk factors relevant to specific project goal and hence, subsequent management of risk factors for ensuring the success of dad project. at a high level, gsrm framework has four layers to support software risk management. every layer is composed of suitable tasks, methods and techniques to manage risks. the first layer in the model, the ‘goal layer’ focuses on the factors or criteria that contribute towards the project success. these goals define what needs to be done in order to achieve success and who will be responsible to achieve the goals (dardenne et al., 1991). in our study, the ‘goals’ refer to the project performance goals including, time, cost and quality that dad projects aim to achieve. the goals can be fine grained into sub goals, but for this initial study, we will address the gsrm framework considering only high-level performance goals. further, an ‘agent’ performs appropriate actions in a way that project goals can be achieved (dardenne et al., 1991). in the context of this study we refer to the ‘dad project team’ or a ‘project manager of the team’ may act as an ‘agent’ to control the project risks. the second layer is the ‘obstacle layer’ which refer to the main causes that reduce the ability to achieve single or multiple goals. the dad risk factors that have been found in this study to have a high impact on a particular performance goal relate to the ‘obstacles’ because they can pose a threat to the attainment of the specific project goal. the third layer is the ‘assessment layer’ that quantifies the severity of impact of the risk factors on the goals. in the context of this study, we have found the perceived impact of the risk factors on each of the considered performance goals separately. after performing analysis on the data, we have been able to identify the top 7-8 statistically significant risk factors that have high impact on each of the considered goal separately. the gsrm framework, identifies ‘risk events’ as consequence of single or multiple risk factors. islam et al., (2014) used risk events in order to reduce the number of raw risk factors being analysed. in the second phase of our study, we reduced the number of risk factors under analysis by considering the risk factors that had statistically significant ranking of their impact on project goals. hence, it was not required to further reduce the risk factors by considering the risk events in our study. we assessed the reduced number of risk factors to identify the top-ranking factors that impact each of the considered project goals. further, as per the gsrm framework, one risk factor may have an impact on multiple goals, and we have similar findings in our study as well which are discussed later in section 5 (data analysis and results). the fourth layer is the ‘treatment layer’ that focuses on the control action to be taken to reduce the impact of the risks that impact the project goals. for this study, the appropriate controls identified in an earlier work (shrivastava & rathod, 2014) for managing the considered dad risks have been included. the implementation of gsrm framework using kaos modelling for risk management in dad projects is shown in figure 1. we have used the same notation as used in the gsrm framework for goals (parallelogram) and obstacles (reverse parallelogram) as used in kaos model. obstacles (risk factors) are linked to the goals through obstruction links. the treatment layer also has an agent who bears the responsibility to implement the actions to control risks. 2.5 related studies software teams recognise that risks in projects need to be identified and controlled for project success that enables achievement of business objectives. there is an extant of literature that australasian journal of information systems shrivastava & rathod 2019, vol 23, research article risk management in distributed agile projects 6 have explored risks occurring in traditional software projects and have also suggested various approaches that can be used to control those risks (schmidt et al., 2001; barki et al., 1993; keil, cule, lyytinen & schmidt, 1998; junior & carvalho, 2013). these studies have identified risks like technological newness, lack of expertise, requirement risks, planning and controlling risk, personal and staffing risks, etc. wallace & keil (2004) have asserted the importance of risks for ensuring desired project outcome and identified and related risks in software projects to process outcome including project completion time, within budget and product outcome like scope and requirements fulfilment. further, the relationship of risks with project performance has also been explored in the works of (han & hung, 2007; na, simpson, li, singh & kim, 2007; wallace, keil & rai, 2004; jiang & klein, 2000). jian and klein (2000) suggest that there are number of performance criteria like user satisfaction, quality, cost/ benefit analysis, on time completion (time), team effectiveness that define the project success. chow & cao (2008) have explored the success factors in agile projects such as team capabilities, agile engineering methods, and agile delivery strategy, which also fall in the success category of time, cost, scope and quality of the project. this is an important area for advancing the knowledge of risk management because different risks impact various dimensions of project success differently (jian and klein, 2000). for example – lack of user experience causes less usable system, an aspect of quality. jian and klein (2000) say that there is a need to develop a linkage between the risks with various dimension of project success. this study examines the effect of risks on project effectiveness, a dimension of project performance, measured through various aspects such as project budget, schedules, quality of work and operational efficiency. different risks will have different impact on the aspects of project success and therefore, the approach for managing a risk must be selected carefully (jian and klein, 2000). research studies that have explored on risk factors in agile development impacting the project success include (bass, 2016; sheffield & lemetayer, 2013; coram & bohner, 2005). a number of research studies have also explored the risks occurring in dad projects (jalali & wohlin, 2010; hossain et al, 2009; mudumba & lee, 2010; mattsson et al., 2010; paasivaara & durasiewicz & lassenius, 2009). further work in identification of the most important risks in dad projects and the most frequently used methods to manage those risks have been done by (shrivastava & rathod, 2014; shrivastava & rathod, 2017). although, there are studies that have viewed risks from the perspective of project success dimensions or project goals (islam et al., 2004; jian and klein, 2000), there is rare evidence of similar work being done in the area of risk management in dad projects. this research contributes by developing the linkage between the risk factors in dad projects and the project performance goals including time, cost and quality. the findings of this study equips project managers managing dad projects with the most important risks to be considered to be controlled when attainment of one of the project goals is critical for the project. 3 research objectives this research study aims at finding the impact of risks occurring in distributed agile development (dad) projects on project performance goals including time, cost and quality. the risk factors used in this research have been derived from the past research in this area (shrivastava & rathod, 2014). since projects are executed under constraints, it is necessary to first identify and act on those risks that impact the performance goal that are critical for a project. hence, we aim to follow a goal driven approach to identify most important risk factors in dad projects that impact the project performance goal critical for the project success. project australasian journal of information systems shrivastava & rathod 2019, vol 23, research article risk management in distributed agile projects 7 managers can then use appropriate controls to reduce the impact of those considered risks and hence, increasing the chance of project success. in order to achieve the above-mentioned research objectives, we addressed the following research questions: 1. what are the risk factors having a high impact on the performance goal ‘time’ in dad projects? 2. what are the risk factors having a high impact on the performance goal, ‘cost’ in dad projects? 3. what are the risk factors having a high impact on the performance goal, ‘quality’ in dad projects? 4 research methodology this descriptive study aims to determine the extent to which a risk factor impacts a particular project performance goal in a dad project with respect to other risk factors (cooper & schindler, 2006). dad practitioners were provided with a list of dad risk factors found in earlier research and were asked to rank the ones, which are perceived by them to impact each of the project goals of time, cost and quality. data collection has been done using a two-part survey-based approach. survey research is a quantitative method, concerned with either relationships between variables, or presenting descriptive findings about a population (pinsonneault & kraemer, 1993). as we intended to find out the perceived impact of stated dad risks on each of the project goals from a set of practitioners involved in dad projects, we believe that survey-based approach was suitable for this study. we used purposive sampling for this research to ensure that we obtained responses based on the experience of the respondents in dad projects. questionnaire used in part i of the study was designed to find out the perceived impact of the dad risk factors on each of the considered project goals. questionnaire used in part ii of the study was based on the analysis of data collected in part i of the research and was used to find out the rank of the risk factors in terms of their impact on a particular project goal. the analysis of the data collected through both surveys will enable us to prioritize the risk factors corresponding to each project goals depending upon the amount of impact they have on that project goal. 4.1 data collection and analysis for survey part –i the questionnaire used for survey part i was pilot tested by sending to 25 experts and we obtained response from 11 experts, giving a response rate of 44%. these experts included nine practitioners and two academics who had significant experience in the field of software engineering. the revised questionnaire was designed to collect the ranking data about the perceived impact of risk on each of the project goals, namely, time, cost and quality. in survey questionnaire for survey part i, we enlisted previously identified, 44 risk factors from earlier research (shrivastava & rathod, 2014) that occur in dad projects. for each of the risk factor we obtained a rank (1/2/3) for each of the project goals namely, time, cost and quality depending on the extent to which that risk factor is perceived to impact the project goals. the respondents were required to provide a rank for the perceived impact of each risk factor on the project goal on a scale of ‘1’,’2’, ‘3’ referring to ‘high impact’, ‘moderate impact’ and ‘low impact’. we contacted 107 practitioners with dad experience, and obtained 56 responses, hence getting a response rate of 52%. from all respondents we contacted, 41 respondents did not respond (38%) and 10 respondents (9%) refused to participate. the 56 respondents to the self-administered survey included project managers (23), managing director/ principal australasian journal of information systems shrivastava & rathod 2019, vol 23, research article risk management in distributed agile projects 8 director/director (10), chief executive officers (ceo) (2), team lead (7), developers (7), quality analyst/testers (3), business analyst (1), architects (2), and product owner(1). the respondents belonged to australia, india, japan and the usa. the questionnaire was sent by email to the respondents and the reply was also received by email. we applied kendall’s test of concordance on the ranking data of the project goals for each risk factor in order to find out the level of concordance of the responses on the findings. kendall’s test also helped us to find out the statistical significance of the average ranks for the project goals for the impact of each risk factor. these average ranks were used to find out the project goals being impacted the most by a risk factor. we chose only those dad risk factors for further study, whose perceived impact on individual project goal has been found to be statistically significant and subsequently, the mean ranks were low (high impact) and the concordance were high. we used statistical significance (p = 0.01) to find out the generalisability of the findings. based on the mentioned criteria we eliminated certain risk factors from each set of the dad risk factors, which have been perceived to have low impact on a particular project goal of dad projects. we then used the selected risk factors to formulate the questionnaire for survey part ii. 4.2 data collection and analysis for survey part –ii as mentioned in the previous section, survey part ii questionnaire was formulated based on the research findings of survey part i of the study, which included separate lists of risk factors impacting ‘time’ and ‘quality’ as project goals. as none of the risk factors were perceived to have a high impact on cost, we could not enlist the factors relating directly to ‘cost’ as a project goal. we listed the set of risk factors, found to have statistically significant impact on the project time, and asked the practitioners to rank them in terms of their perceived impact on the project constraint of completion on time. the respondents were asked to rank the risk factors on a scale of 1 to 8 with rank ‘1’ was given to the risk with highest impact on time, rank ‘2’ given to the risk factor with comparatively lesser impact, ‘3’, ‘4’,’5’,’6’,’7’ ranks were given to risks as their impact goes on reducing and rank ‘8’ given to the risk factor with lowest impact on time. no two or more risk factors were given same rank. same approach was used to obtain the ranks for risk factors impacting quality (7 risk factors). these risks were to be given ranks on a scale of ‘1’ to ‘7’ with ‘1’ signifying highest impact of a risk factor on quality and ‘7’ signifying ‘lowest impact’ of the risk factor on quality. all other ranks from ‘2’ to ‘6’ were given to risk factor in the order of reducing impact on the quality. no two or more risk factors were allocated same ranks. we used qualtrics platform to make the survey available online. we contacted 82 practitioners and received response from 65 of them, hence a response rate of 79.2%. we did not get response from 15 people and 2 of them refused to participate. we applied kendall’s test of concordance (w) on the ranking data for risk factors collected through this survey. the test was applied separately for risk factors related to the time and quality respectively. we then used the average rank and the statistical significance (p-value <= 0.01) to decide about risk factors that have maximum impact on the project time, and the ones that have high impact on quality. statistical significance (p-value <= 0.01) was used to decide about the generalisability of the research findings. the coefficient of kendall’s w was used to find out the coefficient of concordance among the responses about the extent to which the risk factors impact the project outcomes (black, 2009). australasian journal of information systems shrivastava & rathod 2019, vol 23, research article risk management in distributed agile projects 9 4.3 relevance of questionnaire to research objectives to theoretical consideration the survey questionnaires were designed with the aim to analyse risks impacting dad projects while focusing on the project goals that are important for the project success. in the questionnaire for survey part i, we obtained the ranks for risk factors’ impact on all the three project goals separately, ’time’, ‘cost, and ‘quality’. this step helps us to formulate the ‘obstacle layer’ of the gsrm framework, which corresponds to risk factors impacting project goals and form connections with the ‘goal layer’ of the framework. in the questionnaire for survey part ii, we obtained the ranks for the risk factors corresponding to each of the project goal depending upon the severity of impact the risk factors on that project goal. the goal driven approach followed in the study enabled us to segregate risk factors that impact project goal, ‘time’ and also identifying the factors with comparatively higher ranks in terms of severity of impact on the project schedule. similar results for risk factors impacting ‘quality’ was also obtained. evaluation of risk factors based on the severity of impact on the project goals helped us to lay out the ‘assessment layer’ of the grsm framework which consists of prioritised risk factors depending on the extent to which they impact each of the considered project goals. the risk factors that are forming a part of the assessment layer are considered for further analysis to implement relevant risk control. this step also aligns with the formation of the ‘treatment layer’ of the gsrm framework. the risk control approaches suggested in this paper are based on an earlier study in this area (shrivastava & rathod, 2014). hence, the two-part survey questionnaires together, assist us to implement the gsrm framework for performing risk management in dad. 5 data analysis and results we provide the analysis of survey part i and survey part ii in this section. 5.1 survey part i data analysis we applied kendall’s test on the ranking of project outcomes for each risk factor that produced kendall’s coefficient of concordance (w), value of observed significance level (p-value) and the mean rank of each of the project outcomes, namely, time, cost and quality corresponding to each of the risk factors. kendall’s coefficient ranges from 0 to 1, where 0 reveals perfect disagreement and 1 reveals perfect agreement (gibbons, 1993). 5.1.1 data analysis for risk factors impacting project ‘time’ we then extracted all the risk factors with the mean ranks for their impact on project outcome, ‘time’, was less than the mean ranks for the project outcomes ‘cost’ and ‘quality. this helped us to identify the risk factors that have a high impact on the ‘time’ of the project. there were 23 risk factors that were perceived to be having a high impact on project as compared to the other two project performance goals, quality and cost as shown in table 1. the risk factors in table 1 are listed in the increasing order of mean ranks for the project goal ‘time’. the average of the mean ranks for the project outcome ‘time’ for these risk factors (table 1) was 1.75. there were 9 risk factors with the p-value of less than 0.01 and hence their ranks for impact on the project goal ‘time’ can be generalised. these risk factors also have good kendall’s coefficient of concordance (above 0.09) as compared to the other factors listed in table 1, indicating that the responses agree with each other. it can be observed that the risk factors impacting project ‘time’ are primarily related to communication and coordination among the dad teams. australasian journal of information systems shrivastava & rathod 2019, vol 23, research article risk management in distributed agile projects 10 some are related to poor management of requirements and inadequate requirement prioritization. one of the risk factors, namely, ‘no common definition of done between the distributed teams’, had the same rank for the project goal, time and quality, indicating that the amount of impact on the two project goals ‘time’ and ‘quality’ is the same. hence this factor has been listed to be impacting both project time (table 1) and quality (table 2). this aligns with the gsrm framework that states that a particular risk factor may more than one goal (islam et al. 2014). 5.1.2 data analysis for risk factors impacting ‘quality’ of the project. we then extracted all the risk factors having the mean rank for their impact on project outcome, ‘quality’ relatively less than the mean ranks for ‘time’ and ‘cost’. this gave us a list of 21 factors, considered to be having a high impact on the ‘quality’ of the project as shown in table 2. we now considered the risk factors with p-value less than 0.01 which gave us a list of 7 risk factors. the average of the mean ranks for the project outcome ‘quality’ for the selected 7 risk factors was 1.68. the kendall’s coefficient of concordance for these 8 risk factors is also greater than 0.09, which is higher than the concordance for all other risk factors related to the quality, as shown in table 2, indicating a reasonable amount of concordance among the respondents. the top 7 risk factors in table 2 are also related to the lack of communication between the team and the other stakeholders. risks related to poor software development, testing and design standards are also impacting the software quality. 5.1.3 data analysis for risk factors impacting ‘cost’ of the project there were no risk factors with highest value of mean rank for the project outcome ‘cost’ as compared that of ‘time’ and ‘quality’. hence, we could not categorise any risk factor to be having maximum impact on the project performance goal ‘cost’. since these three success criteria, namely, time, cost and quality are not mutually exclusive, the impact of risks on project time or quality will have an impact on the project cost as well (schwalbe, 2016, pp. 9). so, we anticipate the practitioners do not consider risks highly and directly impacting project cost. hence, survey part ii was focussed on understanding how risks impact ‘project time’ and ‘project quality’ in dad project and did not include ‘project cost’ for further data collection and analysis. risk factor no. risk factor mean rank time mean rank cost mean rank quality kendall’s w observed p-value r1 inadequate prioritization of requirements. (large project scope; multiple backlogs handled by multiple product owners) 1.71 1.90 2.39 0.187 0.000 r2 lower initial velocity (dad teams take time to build trust) 1.72 2.13 2.15 0.108 0.003 r3 rework caused by architectural changes (frequent requirement changes) 1.75 1.87 2.38 0.220 0.000 r4 poor coordination between different sites. (difficult to synchronize and integrate different activities) 1.75 2.27 1.98 0.125 0.001 australasian journal of information systems shrivastava & rathod 2019, vol 23, research article risk management in distributed agile projects 11 risk factor no. risk factor mean rank time mean rank cost mean rank quality kendall’s w observed p-value r5 requirement conflicts amongst multiple, geographically dispersed product owners 1.75 2.07 2.18 0.093 0.009 r6 inappropriate user story estimates with multiple vendors. 1.77 1.83 2.40 0.228 0.000 r7 dependency on third party for solution development leading to coordination problems between them. 1.75 2.10 2.15 0.094 0.006 r8 lack of communication infrastructure. (connectivity problems and slow network which hampers the scrum meetings) 1.75 2.11 2.14 0.090 0.008 r9 difficulty in system release management and deployment (difficult to synchronize different distributed teams) 1.76 2.11 2.12 0.078 0.016 r10 poor coordination between multiple vendors involved in the solution development. (especially if they are competing and have work dependencies between them) 1.79 2.21 2.00 0.122 0.003 r11 losing on time for end-to-end extensively interdependent transaction rich test cycle across distributed teams (applicable to transaction rich software application) 1.81 2.20 1.99 0.082 0.027 r12 growth in team size or development site (leads to instability and hence reduction in productivity) 1.81 2.15 2.05 0.060 0.038 r13 requirements unclear to the team 1.82 2.10 2.08 0.046 0.078 r14 poor collaboration between different sites. (understanding & appreciating the capability differences and reducing the gap by using technology) 1.85 2.27 1.88 0.103 0.004 r15 ineffective stand up meetings. (culture and language difference) 1.87 2.20 1.94 0.051 0.057 r16 higher interdependency between the teams. (system complexity and lack of communication between the distributed teams) 1.87 2.08 2.05 0.027 0.223 r17 unavailability of product owner (noncollocation or lack of time) 1.87 2.24 1.90 0.078 0.016 r18 team reorganizing in every sprint (due to changes in market requirements) 1.88 2.05 2.07 0.024 0.286 r19 unclear project objectives 1.88 1.90 2.22 0.067 0.023 r20 risk in code integration with multiple vendors. (lack of common understanding of requirements and interfaces) 1.89 2.13 1.99 0.030 0.236 r21 unsuitability of flat communication structure to large organization. (may expose the client to inexperienced team members) 1.91 2.01 2.07 0.020 0.393 australasian journal of information systems shrivastava & rathod 2019, vol 23, research article risk management in distributed agile projects 12 risk factor no. risk factor mean rank time mean rank cost mean rank quality kendall’s w observed p-value r22 no common definition of done between the distributed teams. (to be removed) 1.95 2.09 1.95 0.012 0.518 r23 underinvestment on travel by the management. 1.96 2.05 1.99 0.005 0.802 r24 difficult to execute fixed price projects 1.97 1.98 2.06 0.006 0.789 table 1: survey part 1analysis of the risk factors that impact the project outcome, ‘time’ is as above (the scale for ranking used in the survey – high impact= 1, moderate impact =2, low impact =3) risk factor no. risk factor mean rank time mean rank cost mean rank quality kendall’s w observe d p-value r25 poor collaboration between developers and quality assurance members. (especially if they belong to different distributed team) 2.05 2.35 1.59 0.271 0.000 r26 unavailability of requirement documents for testing. (testers depend on their assumptions for testing) 2.09 2.28 1.63 0.187 0.000 r27 inconsistency in design standards of distributed teams. 2.07 2.27 1.65 0.167 0.000 r28 cross functional teams insufficient for testing in large projects. (the ‘whole team concept’ may not be sufficient for complex testing) 2.02 2.32 1.66 0.213 0.000 r29 different software development practices and standards followed by multiple teams. (cultural diversity and lack of stakeholder support) 2.12 2.15 1.74 0.091 0.007 r30 lack of communication between the team and the client (product owner) 2.03 2.22 1.75 0.111 0.003 r31 uncommon language (team members or client may not share a common language) 1.96 2.25 1.78 0.125 0.001 r32 lack of documentation since agile downplay documentation (dad projects need some minimum level of documentation to compensate for the informal team interaction) 2.01 2.21 1.78 0.083 0.012 r33 technical debts in the code. (deferred work which is not directly related to the functionality but is required for the overall quality of the system) 2.06 2.15 1.79 0.056 0.049 r34 inappropriate tool selection (especially for practices like automated testing, code integration etc.) 1.99 2.20 1.81 0.079 0.014 r35 using component teams instead of feature teams (e.g. gui team or component x team) is unable to deliver end-to-end feature) 1.95 2.23 1.82 0.085 0.012 australasian journal of information systems shrivastava & rathod 2019, vol 23, research article risk management in distributed agile projects 13 risk factor no. risk factor mean rank time mean rank cost mean rank quality kendall’s w observe d p-value r36 issues with pair programming (distributed developers) 2.10 2.07 1.82 0.043 0.124 r37 lack of trust between the onshore and offshore teams. 2.04 2.13 1.83 0.041 0.119 r38 lack of communication between team members 1.94 2.23 1.84 0.078 0.014 r39 code integration across multiple sites. (lack of common understanding of interface requirements between teams) 2.04 2.14 1.85 0.055 0.058 r40 test data management involving test extraction from multiple databases is difficult. (small iterations in agile and lack of test database automation) 1.87 2.26 1.86 0.094 0.008 r41 lack of uniformity in multisite team’s capabilities (loss of productivity) 1.95 2.17 1.88 0.050 0.082 r42 unavailability of business analyst (ba). (single ba is assigned to multiple teams) 1.96 2.14 1.90 0.029 0.213 r22 no common definition of done between the distributed teams. 1.95 2.09 1.95 0.012 0.518 r43 unavailability of business analyst (ba). (single ba is assigned to multiple teams) 1.96 2.14 1.90 0.029 0.213 r44 inadequate communication about end user requirements (social context of end-user not understood) 2.01 2.03 1.96 0.003 0.850 r45 lack of trust between the client and the offshore team (it is difficult to customer expectations in short iteration) 1.97 2.07 1.96 0.008 0.672 table 2: survey part 1 analysis of the risk factors that impact the project outcome, ‘quality’ is as below (the scale for ranking used in the survey – high impact= 1, moderate impact =2, low impact =3) 5.2 survey part ii data analysis as stated before, we further eliminated risk factors from the list of factors obtained from survey part i results to create a survey questionnaire for survey part ii. from the list of 23 risk factors impacting project time in table 1, we extracted the ones with p-value less than or equal to 0.01, which gave us a list of 9 risk factors. we further considered risk factors with difference in their mean rank for time and the average of the mean ranks for the project outcome ‘time’ for all these 9 risk factors (1.75) was more than or equal to 0.1. this helped us to eliminate one more risk factor from a list of 9 factors and gave us a list of 8 risk factors. for these factors, it was observed that kendall’s w value is also reasonably good, hence there was concordance among the responses. these 8 factors were then used for designing the survey part ii questionnaire. same process was applied to reduce the list of factors impacting ‘quality’ as given in table 2. we first considered the risk factors with p-value less than or equal to 0.01 which gave us a list of 10 risk factors. we then eliminated the risk factors for which the difference in the mean rank australasian journal of information systems shrivastava & rathod 2019, vol 23, research article risk management in distributed agile projects 14 and the average rank of these 10 factors was more than 0.1. this led to a list of 7 risk factors which was included in the survey questionnaire part-ii. all the risk factors impacting ‘time’ and ‘quality’ have been bold-faced in table 1 and table 2 respectively. risk factor no. ranking of risk factors based on the perceived impact on project time mean ranks r1 inadequate prioritization of requirements. (large project scope; multiple backlogs handled by multiple product owners) 3.31 r2 rework caused by architectural changes (frequent requirement changes) 3.35 r3 requirement conflicts amongst multiple, geographically dispersed product owners 3.74 r4 poor coordination between different sites. (difficult to synchronize and integrate different activities) 4.28 r5 dependency on third party for solution development leading to coordination problems between them. 4.52 r6 inappropriate user story estimates with multiple vendors. 5.05 r7 lower initial velocity (dad teams take time to build trust) 5.59 r8 lack of communication infrastructure. (connectivity problems and slow network which hampers the scrum meetings) 6.15 table 3 survey part ii ranking of risk factors impacting project time kendall’s w: 0.181; observed p-value: 0.000 risk factor no. ranking of risk factors based on the impact on project quality mean rank r1 lack of communication between the team and the client (product owner) 3.08 r2 unavailability of requirement documents for testing. (testers depend on their assumptions for testing) 3.31 r3 poor collaboration between developers and quality assurance members. (especially if they belong to different distributed team) 3.46 r4 inconsistency in design standards of distributed teams. 3.86 r5 cross functional teams insufficient for testing in large projects. (the ‘whole team concept’ may not be sufficient for complex testing) 4.29 r6 different software development practices and standards followed by multiple teams. (cultural diversity and lack of stakeholder support) 4.51 r7 uncommon language (team members or client may not share a common language) 5.49 table 4 survey part ii ranking of risk factors impacting project quality kendall’s w: 0.150; observed p-value: 0.000 as discussed in section 4 the questionnaire for survey part ii was designed based on the results obtained from survey part i. we had a list of risk factors (8) that were perceived to be impacting ‘time’ the most and another list of 7 risk factors impacting ‘quality’ the most in dad projects. based on the extent to which the risk factors impact the project outcome, the respondents were asked to rank the risk factors. kendall’s test was then applied to the ranking data of risk factors for both the project outcomes, time and quality, separately. kendall’s test gave us the mean ranks for the risk factors, kendall’s coefficient of concordance (w) and the australasian journal of information systems shrivastava & rathod 2019, vol 23, research article risk management in distributed agile projects 15 observed p-value for both set of risk factors impacting ‘time’ and ‘quality’ respectively. the detailed results of application of kendall’s test on risk factors impacting project time is given in table 3 and the results of applying the same test on risk factors impacting project quality is given in table 4. the mean ranks obtained for risk factors for the project outcome, ‘time’ and for ‘quality’ was statistically significant (p-value <= 0.000) indicating that the findings can be generalised. the risk factors in table 3 and in table 4 are listed as per the ranks given to the risks. the risk factors on the top of the list are the ones that have comparatively higher impact on the project outcome as compared to the ones with lower in the list. 6 research findings and discussion the success of software development project is assessed in terms of meeting the scope, schedule, cost and quality constraints of the projects (chow & cao, 2008; attarzadeh & ow, 2008; agarwal & rathod, 2006). distributed agile projects are able to reach project goals suffice the customer needs, but suffer from significant risks caused by the opposing nature of distributed development and agile methods. these risks pose threat to the attainment of the project goals and in this work, we have found the risk factors that have a high impact on the project time and quality. the knowledge of risk factors impacting a specific project goal will be beneficial for the project manager and the team as it will enable them to control the risks effectively. one of our highly experienced participants stated about the significance of this study “while distributing work can introduce additional risk, i believe those risks are offset by the iterative and incremental nature of agile delivery. based on my experience, distributed teams delivering under traditional methods are at the highest level of risk of failure, followed by colocated teams delivering under traditional methods. whether co-located or distributed, agile approaches provide the best mitigations to delivery risk. as to whether i think the result would be useful, i think a 'top ten risks to be aware of' would be a useful tool to have in the kit of any pm, so yes, i think it would be useful.” although the results are statistically significant and verifiable, scrutiny with respect to the relevant theoretical framework is essential to make them sufficiently reliable for real world implementation. in this section we use the gsrm framework and kaos modelling language to formally present the perception of practitioners about the impact of risks on project goals in dad projects. 6.1 impact of risk factors on project time (schedule) of the project in the context of the gsrm framework, risk factors identified in our study act as the ‘obstacles’ to achieving the project goal ‘timely completion of the project’ (goal layer) of the model. in figure 1, risk factors, r1 to r21, r23 and r24 are shown to have maximum impact on project time and are presented in the ‘obstacle layer’. when we assessed their impact on the project time in survey part ii, we could identify risk factor r1 to r8, having a statistically significantly high impact on the project time along with reasonable agreement among the respondents for their ranks. these eight risk factors are forming a part of the ‘assessment layer’ of the gsrm model and the suggested approaches to manage the risks are a part of ‘treatment layer’ as shown in figure 1. these risks factors relate to poor requirement analysis, requirement conflicts among the stakeholders, changes in architecture of the system and lack of coordination among the team members and contribute to delays in distributed agile projects. australasian journal of information systems shrivastava & rathod 2019, vol 23, research article risk management in distributed agile projects 16 distributed agile projects usually have large scope and involves multiple backlogs where each backlog has large number of user stories. figure 1: application of gsrm framework for risk management in distributed agile development the requirements with every team might be even conflicting due to involvement of multiple product owners in different locations, further reducing the clarity about project requirements. in order to reduce the impact of ‘inappropriate prioritization of requirements’ dad team need good communication, backlog grooming and look ahead planning (cohn, 2010). further, agile teams welcome changes in the requirements, which causes changes in the system architecture (dingsøyr, moe, faegri & seim, 2018), which further causes project delays (begel & nagappan, 2007). architects in agile teams need to have communication with the development group and make architectural decisions on the right time (abrahamsson, babar & krutchen, 2010). it is evident in our research findings also that ‘poor coordination between different sites’ causes difficulty in the integration activities due to interdependencies leading to schedule delays. malone & crowstone (1994) proposed theory of coordination, which is based on the principle australasian journal of information systems shrivastava & rathod 2019, vol 23, research article risk management in distributed agile projects 17 that coordination is the management of dependencies and appropriate coordination mechanisms need to be applied (strode, hope, huff & link, 2011). in distributed development, multiple teams need to coordinate with each other, synchronize their activities and manage the time zone difference in order to deliver timely output (hossain et al., 2009). temporal coordination is specifically important and enables teams to achieve the performance goals (espinosa, slaughter, kraut, & herbsleb, 2007). distributed agile projects involves multiple vendors which helps them to avoid the single vendor contracting risk (sakthivel, 2007). high level of coordination is required between the team and the vendors for solution development, which if not considered, leads to delays in the project outcome (‘dependency on third party for solution development leading to coordination problems between them’). the involvement of external vendors and customers in dad teams may foster conflict between the self-interest and willingness for collaboration with the team members. organization design theory emphasizes that suitable organization design (mostly, hierarchical) and incentive systems (burton and obel, 1980, 1988; williamson, 1975) help to control the self-interested opportunism on individuals. in distributed teams, the design and the incentive systems are loosely designed, causing lack of control on opportunism leading to miscommunication and lack of trust. the extent of trust and shared social identity that development team must possess, needs strong social interactions and synchronization, failing which will lead to the team’s failure in meeting project objectives (olaisen and revang, 2017). poor communication and lack of common standards among the vendors in distributed teams impacts the requirement estimation, causing ‘inappropriate user story estimates with multiple vendors’. accurate estimation of development effort in each stage is required in order to determine the schedule of the project. with the standard effort estimation approaches it is not possible to determine the correct estimates as they do not account for the effort involved in communication, coordination and management of multiple teams, external vendors and customers (sakthivel, 2007). besides team coordination, ‘lack of communication infrastructure’ for communication also impacts the project time. multiple communication modes like teleconference, videoconference, email, instant messaging are being used to achieve the project goals (paasivaara, durasiewicz, & lassenius, 2008). there are other risk factors also that lead to schedule delays in distributed agile projects related to difficulty in release management and team synchronization, ineffective stand up meetings, team interdependencies, unclear project objectives and requirements. although, these risks are perceived to be impacting project time, they are not found to be statistically significant and had comparatively low value of kendall’s coefficient of concordance, and so, have not been discussed in this paper. 6.2 impact of risk factors on project cost as apparent, the respondents did not directly relate project cost to any risk factors as the project cost supposedly depends on time and quality (schwalbe, 2016). if there is a delay in the project schedule or rework needs to be done to improve the quality, it will lead to increase in the project cost (reichelt & lyneis, 1999). a study by agarwal & rathod (2006) have also mentioned that ‘project cost’ is the least important factor to be considered for evaluation of the project success. it has also been observed that agile manufacturing is associated with flexibility, quality of the product and has negative association with cost leadership. in other words, cost performance is not significantly affected by agile manufacturing and agile method is not a good choice for cost leaders (hallgren & olhager, 2009). these studies support the research findings that the practitioners in distributed agile projects have lesser concern for australasian journal of information systems shrivastava & rathod 2019, vol 23, research article risk management in distributed agile projects 18 cost than project time and quality. we can see in figure 1 that none of the risk factors in the ‘obstacle layer’ has been related to the project goal ‘cost’ situated in the goal layer in the gsrm model. hence, the discussion section focuses more on the impact the risk factors have on project time and quality. 6.3 impact of risk factors on project quality here we consider focusing on the project goal ‘quality’ in the goal layer, as in the gsrm framework and specifically risk factors that act as obstacles to achievement of the project goal of ‘quality’. as presented in table 1, the risk factors numbered as, r25-r42, r43, r44, r45, are having a high impact on the project goal ‘quality’ and have been presented in the ‘obstacle layer’ in the gsrm framework. as in table 2, the risk factors with numbers, r25-r31 were the ones that had statistically significant risks with high level of respondents’ concordance and were considered for further assessment of their impact on project quality. these factors obtained ranks such that they could be organised in the decreasing order of the amount of impact they had on quality and hence were placed in the ‘assessment layer’ in the gsrm framework. the risks are being controlled by using appropriate risk management methods which can be a part of the treatment layer. the risk control methods have been discussed in (shrivastava & rathod, 2014) and has been mentioned in this section also. the research findings of this study show that ‘lack of communication between the team and the client’ and poor collaboration among the team members significantly impacts the project quality. distantly located team members do not have the ease of approaching each other as they have when they are co-located. their communication is normally on lean electronic channels such as email, telephone, audio-video streaming, which do not have the richness of the content that is naturally embedded in face-to-face communication as propagated by media richness theory (daft and lengel, 1986). agile methods foster team collaboration which becomes difficult to achieve in distributed environments where participants have limited information about each other’s capability (cohn, 2010). in this situation the projects suffer due to ‘poor collaboration between developers and quality assurance members’ reducing the quality of the output. agile methods rely on negotiations between the team members and the client to achieve an acceptable level of quality at various stages of development (balasubramanium et al., 2006). agile practices rely more on informal interaction than explicit documentation, and this becomes challenging in distributed environment (balasubramanium et al., 2006), especially for testing purposes (‘unavailability of requirements documents for testing’), hence impacting project quality (inayat, salim, marczak, daneva, & shamshirband, 2014; heck & zaidman, 2018). in distributed agile projects, the absence of independent testers who can focus on the hard parts of the testing work like, system integration testing, investigative testing, formal usability testing may lead to poor quality outcome. hence in order to manage the risk, ‘cross functional teams insufficient for testing in large projects’, distributed teams may need to arrange for an independent team in addition to it cross functional team for complex projects. it is also observed that ‘different software development practices and standards followed by multiple teams’ impact the software quality in distributed agile projects. team members with different backgrounds, regions have different preference for development practices, tools and engineering practices which leads to rework (cusumano, 2008). distributed teams also suffer from a risk of ‘inconsistency in design standards of distributed teams’ due to the involvement of multiple teams belonging to distinct organizations. as per conway’s law, the system design australasian journal of information systems shrivastava & rathod 2019, vol 23, research article risk management in distributed agile projects 19 generated by each team in distributed project aligns to the communication structure of the corresponding organization (conway, 1968), which ultimately impacts the quality of the project. this risk can be overcome by having good coordination among the developers, a good modular design of the system along with having good documentation (herbsleb & grinter, 1999). one of the cultural differences like ‘uncommon language’ among the team members hinders effective communication which reduces the quality of the output. studies have proven that language difference leads to a need to use written asynchronous communication, which is more time consuming and difficult to interpret impacting quality negatively (sosa, eppinger, pich, mckenrick & stout, 2002; moe & smite, 2008). there were other risks such as lack of documentation, technical debts in the code and difficulty in performing pair programming due to team distribution that affects the project quality. other risk factors that need consideration are, ‘no common definition of done between distributed teams’, ‘inadequate communication about end user requirements, ‘lack of uniformity in team capabilities’, and ‘poor trust among the team members’. however, as these risk factors had statistically insignificant results (p-value greater than 0.01) and low kendall’s coefficient of concordance, they have not been discussed in detail. these factors are enlisted in table 2, but we cannot generalise the results, although their awareness would be useful for project managers. 6.4 review of the impact on project goals the research findings demonstrate that different types of risks impact different project goals in dad projects and this understanding about risks would help in implementing effective risk controls. we observed that risks in dad projects have a high impact on project schedule and quality, which are the primary goals considered by agile teams. agile teams iteratively create good quality solution in small time boxes that fulfils the customer needs (dyba & dingsøyr, 2008). surprisingly, the risks that have emerged in our findings are impacting these two most essential project goals of dad projects. poor requirements management and changes in the design/architecture are primarily responsible for delays in dad project. along with this, we observe that involvement of various stakeholders such as vendors leads to coordination efforts, which if neglected will also cause delays in project completion. the research findings show that that communication among the team and with the client significantly impacts the quality of the output in dad projects. further, following standard design approach, appropriate testing methods and software development practices are important areas to be considered for achieving good quality. to some extent cultural difference in dad projects also impacts quality and needs consideration while the project is being executed. 7 limitations and threats to validity in this study, we have been able to identify the top-ranking risk factor that causes delay in the project and the poor software quality. for these risk factors we have obtained statistically significant results (p-value < 0.01), but there are certain unavoidable limitations which need to be stated. in survey questionnaire part – i, due to large number of risk factors for ranking led to low response rate. data for part – i questionnaire was collected through email due to which it was difficult to convince some prospective respondents to complete the survey, which resulted in no response from them. there were time constraints as we had to collect responses in two phases which also led to reduced number of responses. this might have impacted the australasian journal of information systems shrivastava & rathod 2019, vol 23, research article risk management in distributed agile projects 20 generalization of the results and hence, the external validity (maxwell, 1992). we observed that as the number of risk factors were reduced in questionnaire part – ii and the data was collected using email as well as online platform, more responses could be received. the data was primarily collected from australia from the city where the principal researcher is located and from other cities within the country also. we tried to improve the external validity by collecting data from other countries including india, japan, u.k and us. for ensuring construct validity, we provided a brief explanation about each risk factor and project goal in the questionnaires, which helped in clarifying the meaning of each construct. since we obtained statistically significant results, it indicates that the understanding of the survey by the practitioners was correct. we could not get responses from certain respondents who had participated previously in first part of the study as they were unwilling to participate in the study multiple times. due to time and budget constraints we could not use external research services for further data collection. the quantitative methods (westen & rosenthal, 2003) for establishing construct validity could not be used because of the paucity of the time and the limited number of respondents inclined to participate in the study. lastly, the study of perception of practitioners has limitations associated with surveys. 8 conclusion conflicting philosophies of distributed and agile development approaches lead to the risk factors that pose threat to the attainment of the project goals of time, cost and quality. studies have shown that different risks have different impact on various project goals and project managers need to understand the relation between each risk in dad projects with the goals that the team is seeking to achieve (jiang & klein, 2000). considering that time, cost and quality are the primary high-level goals of a dad project, in this study we propose a goal-driven approach to manage risks using kaos modelling language (dardenne et al., 1991). we conducted two-part survey-based study in order to identify the risk factors impacting each of the project goals in dad projects. in first part of the study, the impact of forty-four dad risk factors from an earlier work on each project goal was ranked and the risk factors were segregated in terms of the project goals highly impacted by them. it was revealed that practitioners perceive that the dad risk factors impact project time and quality but not the cost. the risk factors, whose impact on time and quality separately was found to be statistically significant were chosen for the second part of the study, in which, the impact of risk factors on each corresponding project goal was ranked. we obtained the top ranking 7-8 risk factors impacting time and the ones impacting quality in dad projects. the research findings are modelled using a gsrm framework for risk management (islam et al., 2014). it was observed that poor requirement management and lack of coordination among the dad team members significantly impact the project schedule. further, changing requirement and design, poor communication, lack of trust affects the quality of the project outcome adversely. while, agile methods focus on communication among the team, collaboration among stakeholders and technical excellence, the inability of the dad teams to control these factors poses significant risks to projects. as suggested by our study, the awareness of top 7 risks impacting time and top 8 risks impacting quality in dad projects would be a useful kit for project teams dealing with dad projects. the research findings would be useful for the researchers also as it is advancing the body of knowledge by proposing a method for risk management with a focus on the project goals. moreover, the aspects of dad that need further australasian journal of information systems shrivastava & rathod 2019, vol 23, research article risk management in distributed agile projects 21 study for effective management of risk factors associated with the project time and software quality have been clearly unveiled by this study. both, the developer as well as research communities are benefitted with the finding of this study. references abrahamsson, p., babar, m.a., & kruchten, p. (2010). agility and architecture: can they coexist?, in ieee software, vol. 27, no. 2, pp. 16-22, doi: 10.1109/ms.2010.36 agarwal, n., & rathod, u. (2006). defining ‘success’ for software projects: an exploratory revelation. international journal of project management, 24, 358–370. https://doi.org/10.1016/j.ijproman.2005.11.009 attarzadeh, i., & ow, s.h. (2008). project management practices: the criteria for success or failure. communications of the ibima 1, 234–241 balasubramaniam, r., lan, c., kannan, m., & peng, xu. (2006). can distributed software development be agile? commun. acm 49, 10 (october 2006), 41-46. doi=http://dx.doi.org/10.1145/1164394.1164418 barki, h., rivard, s., & talbot, j. (1993). toward an assessment of software development risk. journal of management information systems, 10(2), 203–225. bass, j. m. (2016). artefacts and agile method tailoring in large-scale offshore software development programmes. information and software technology, 75, 1-16. begel, a., & nagappan, n. (2007). usage and perceptions of agile software development in an industrial context: an exploratory study first international symposium on empirical software engineering and measurement, ieee. black, k. (2009). business statistics: for contemporary decision making. john wiley and sons (asia) pte. ltd. burton, r.m., & obel, b. (1980). a computer simulation test of the m-form hypothesis. adm sci q 25(3):457–466 cohn, m. (2010). succeeding with agile: software development using scrum. pearson education in south asia. cooper, d.r., & schindler, p.s. (2006). business research methods tata mcgraw hill. coram, m. and bohner, s. (2005). the impact of agile methods on software project management. in proceedings of the 12th ieee international conference and workshops on engineering of computer-based systems (ecbs '05). ieee computer society, washington, dc, usa, 363-370. doi=http://dx.doi.org/10.1109/ecbs.2005.68 chow, t. & cao, d.b. (2008). a survey study of critical success factors in agile software projects. journal of systems and software 81.6: 961-971. conway, m.e (1968). how do committees invent? datamation, vol 14, no. 4, apr. 1968, pp. 28-31 cullen, k., & parker, d.w. (2015). improving performance in project-based management: synthesizing strategic theories. international journal of productivity and performance management. 64(5):608-624. doi:10.1108/ijppm-02-2014-0031. australasian journal of information systems shrivastava & rathod 2019, vol 23, research article risk management in distributed agile projects 22 cusumano, m.a. (2008). managing software development in globally distributed teams. communications of the acm, vol. 51, no. 2, pp. 15–17, 2008. daft, r., & lengel, r.h. (1986). organization information requirements, media richness and structural design. management science.v.32 no.5, 554-571 dardenne,a., fickas s., and lamsweerde,a.v. (1991). goal-directed concept acquisition in requirements elicitation, proc. iwssd-6 . 6th intl..worksh. on software specification and design, ieee, 1 99 1 , 1 4-2 1 . dingsøyr t., moe, n.b., faegri, t.e., & seim, e.a. (2018). exploring software development at the very large-scale: a revelatory case study and research agenda for agile method adaptation. empirical software engineering, (1), 490. https://doi.org/10.1007/s10664-0179524-2 dorairaj, s. and noble, j. (2013). agile software development with distributed teams: agility, distribution and trust in agile conference (agile), 2013, pp. 1-10. dyba, t. & dingsøyr, t. (2008). empirical studies of agile software development: a systematic review information and software technology, pp. 833-859, 2008. espinosa, j.a., slaughter,s.a., kraut,.r.e & herbsleb,j.d (2007) team knowledge and coordination in geographically distributed software development, journal of management information systems, 24:1, 135-169, doi: 10.2753/mis0742-1222240104 gibbons, j.d (1993) nonparametric statistics: an introduction (quantitative applications in the social sciences), 90, sage hallgren,h., & olhager,j. (2009) lean and agile manufacturing: external and internal drivers and performance outcome. international journal of operations & production management, vol. 29 issue: 10, pp.976-999, https://doi.org/10.1108/01443570910993456 han, w.m. & huang, s.j. (2007). an empirical analysis of risk components and performance on software projects. the journal of systems and software, 80, pp. 42-50 heck, p., & zaidman, a. (2018). a systematic literature review on quality criteria for agile requirements specifications. software quality journal, (1), 127. https://doi.org/10.1007/s11219-016-9336-4 herbsleb, j.d. (2007). global software engineering: the future of socio-technical coordination. future of software engineering (fose’07). ieee. herbsleb, j.d. & grinter, r.e. (1999). splitting the organization and integrating the code: conway's law revisited. in proceedings of the 21st international conference on software engineering (icse '99). acm, new york, ny, usa, 85-95. doi=http://dx.doi.org/10.1145/302405.302455 holmström, h., fitzgerald, b., ågerfalk, p.j & conchúir, e.o (2006). agile practices reduce distance in global software development, information systems management, 23:3, 7-18 hossain, e., babar, m.a., paik, h.y., & verner, j. (2009). risk identification and mitigation processes for using scrum in global software development: a conceptual framework. software engineering conference, 2009. apsec '09. asia-pacific , vol., no., pp.457,464, 1-3 dec. 2009 doi: 10.1109/apsec.2009.56. australasian journal of information systems shrivastava & rathod 2019, vol 23, research article risk management in distributed agile projects 23 inayat, i., salim, s. s., marczak, s., daneva, m., & shamshirband, s. (2014). a systematic literature review on agile requirements engineering practices and challenges. computers in human behavior, 51, 915–929. islam,s., mouratidis,h. & weippl,e.r (2014). an empirical study on the implementation and evaluation of a goal –driven software development risk management model. information and software technology, 56, 117-133. https://doi.org/10.1016/j.infsof.2013.06.003 jalali, s., & wohlin, c. (2010). agile practices in global software engineering – a systematic map. in proceedings of 5th ieee international conference on global software engineering. jiang, j. & klein, g. (2000). software development risks to project effectiveness, the journal of systems and software, 52, pp. 3-10 jim´enez, m., piattini, m., & vizca´ıno, a. (2009). review article: challenges and improvements in distributed software development: a systematic review, hindawi publishing corporation, advances in software engineering, volume 2009, article id 710971. junior, r.r., & carvalho, m.m (2013). understanding the impact of project risk management on project performance: an empirical study. journal of technology management & innovation, volume 8, special issue altec. khalili, h. a., & maleki, a. (2011). project risk management techniques in resource allocation, scheduling and planning. world academy of science, engineering and technology, 59. keil, m., cule, p. e., lyytinen, k., & schmidt, r. c. (1998). a framework for identifying software project risks. communications of the acm, 41(11), 76-83. kerzner, h. (2017). project management: a systems approach to planning, scheduling, and controlling. john wiley & sons, inc., new york, ny, usa. kontio, j. (2001) software engineering risk management: a method, improvement framework and empirical evaluation, ph.d. thesis, helsinki university of technology, 2001. krishna, s., sahay, s., & walsham, g. (2004). managing cross cultural issues in global software outsourcing. communications of the acm, vol. 47, no. 4, pp. 62–66. lamsweerde, v. (2000) requirements engineering in the year 00: a research perspective. in icse '00: proceedings of the 22nd international conference on software engineering, pages 5– 19, new york, ny, usa, 2000. acm press. lee, s., & yong, h.s. (2010). distributed agile: project management in a global environment. empir. softw. eng. 15, 204–217 malone, t. w. and crowston, k. (1994). the interdisciplinary study of coordination. acm computing surveys, 26(1), 87-119. mattsson, m.k., azizyan, g., & magarian, m.k. (2010). classes of distributed agile development problems. in proceedings of the 2010 agile conference (agile '10). ieee computer society, washington, dc, usa, 51-58. doi=http://dx.doi.org/10.1109/agile.2010.14 maxwell, j.a. (1992). understanding and validity in qualitative research. harvard educational review, vol. 62 (3), pp. 279-300. australasian journal of information systems shrivastava & rathod 2019, vol 23, research article risk management in distributed agile projects 24 misra, s.c., kumar, v., & kumar, u. (2009). identifying some important success factors in adopting agile software development practices. j. syst. softw. 82, 11, 1869-1890. doi=10.1016/j.jss.2009.05.052 http://dx.doi.org/10.1016/j.jss.2009.05.052 moe, n.b., & smite, d. (2008). understanding a lack of trust in global software teams: a multiple –case study. software process: improvement & practice, 13(3), 217-231. https://doi/10.1002/spsp.378 mudumba, v., & lee, o.k. (2010). a new perspective on gdsd risk management: agile risk management, 5th ieee international conference on global software engineering (icgse). na, k.-s., simpson, j. t., li, x., singh, t., & kim, k.-y. (2007). software development risk and project performance measurement: evidence in korea. the journal of systems & software, 80, 596–605. https://doi.org/10.1016/j.jss.2006.06.018 nidumolu, s. (1995).the effect of coordination and uncertainty on software project performance: residual performance risk as an intervening variable. information systems research (6:3), 1995, pp. 191-29. olaisen, j., & revang, o. (2017). working smarter and greener: collaborative knowledge sharing in virtual global project teams. international journal of information management, 37, 1441–1448. paasivaara m.s. durasiewicz, s., & lassenius, d.c. (2008). distributed agile development: using scrum in a large project. software process improvement and practice vol. 13 issue 6 pp. 527-544. parker, d. w., parsons, n., & isharyanto, f. (2015). inclusion of strategic management theories to project management. international journal of managing projects in business, 8(3), 552– 573. https://doi.org/10.1108/ijmpb-11-2014-0079 persson, j. s., mathiassen, l., & aaen, i. (2012). agile distributed software development: enacting control through media and context. information systems journal, (6), 411. pinsonneault, a., & kraemer, k. l. (1993). survey research methodology in management information systems: an assessment. journal of management information systems, 10(2), 75–105. pressman, r., & maxim, b. (2015). software engineering: a practitioner’s approach (8th ed.). new york: mcgraw-hill higher education. project management institute (2017). a guide to the project management body of knowledge (pmbok guide). newtown square, pa.: project management institute, inc. (6th edition) isbn: 9781628251845 reichelt, k., & lyneis, j. (1999). the dynamics of project performance: benchmarking the drivers of cost and schedule overrun. european management journal, 17(2), 135. saarinen, t. (1996). an expanded instrument for evaluating information system success. information and management 31, 103±118. sakthivel, s. (2007). managing risk in offshore systems development. communications of acm. april 2007. vol 50 no. 4. schmidt, r., lyytinen, k., keil, m., & cule, p. (2001). identifying software project risks: an international delphi study. journal of management information systems, spring. australasian journal of information systems shrivastava & rathod 2019, vol 23, research article risk management in distributed agile projects 25 schwalbe, k. (2016). information technology project management, cenage, eighth edition. canada sheffield, j.,& lemetayer, j. (2013). factors associated with the software development agility of successful projects. international journal of project management, 31, 459-472. https://doi.org/10.1016/j.jproman.2012.09.011 sosa, m.e., eppinger,s.d., pich,m. mckenrick,d.g., stout, s.k. (2002) factors that influence technical communication in distributed product development: an empirical study in the telecommunications industry ieee transactions on engineering management, (1), 45. https://doi.org/10.1109/17.9857 shrivastava, s.v. & rathod, u. (2017). a risk management framework for distributed agile projects, information and software technology, vol. 85, pp. 1-15, doi: 10.1016/j.infsof.2016.12.005. shrivastava, s.v., & rathod, u. (2014). categorization of risk factors for distributed agile projects. information and software technology, 58:373-87. strode, d.e., hope, b., huff, s. and link, s, (2011). coordination effectiveness in an agile software development context. pacis 2011 proceedings. 183. accessed from http://aisel.aisnet.org/pacis2011/183 therrien, e. (2008). overcoming the challenges of building a distributed agile organization. proc. agile 2008 conference, ieee computer society, toronto, canada, pp. 368-372. wallace, l., keil, m. & rai, a. (2004). how software project risk affects project performance: an investigation of the dimensions of risk and an exploratory model. decision sciences, 35: 289–321. doi:10.1111/j.00117315.2004.02059.x wallace, l., keil, m. (2004) software projects risks and their effect on outcomes. communications of the acm. 2004;47(4):68-73. doi:10.1145/975817.975819. westen, d., & rosenthal, r. (2003). quantifying construct validity: two simple measures. journal of personality and social psychology 84, no. 3: 608-618. westerveld, e. (2003). the project excellence model: linking success criteria success factors. international journal of project management, 21, 411-418. https://doi.org/10.1016/s02637863(02)00112-6 williamson, o.e., (1975). markets and hierarchies: analysis and antitrust implications (free press, new york). australasian journal of information systems shrivastava & rathod 2019, vol 23, research article risk management in distributed agile projects 26 appendix survey questionnaire (part i) for the study impact of risks on project goals in distributed agile development (dad) projects the aim of this research is to find out the impact of risks as perceived by the practitioners occurring in distribute agile development (dad) projects on project goals including time, cost and quality. we request the respondents to share their experiences openly and provide their honest response to the questions. we assure the respondents that their responses will be used for academic research purpose only and the confidentiality will be maintained with utmost care. section 1 – ranking of project goals for the enlisted risk factors guidelines for filling section 1 of the questionnaire 1. please rank the perceived impact of the listed dad risks on the project constraint in terms of high impact (1), moderate impact (2) and low impact (3). more than one constraint may experience the same impact (same rank). you can mark the impact 0 (zero) for no impact of the dad risk on a constraint. if you have not encountered a particular risk in dad projects and hence are unable to give ranks to project goals for that risk, please mark ‘na’ for that risk. 2. project goals definitions • time the time used for completing the project • cost – the cost incurred to complete the project • qualityeffective software process applied in a manner that creates a useful product that provides measurable value for those who produce it and who use it no. risk factor project goals time cost quality 1 unclear project objectives respondent’s background information date: name: e-mail: contact number (office): mobile: company name: profile in the company: total experience: experience in managing agile projects: please mark ‘x’ in front of the appropriate answer…. the number of employees in all the locations across the globe: < 50 ( ) 50-100 ( ) 101500 ( ) 501-1000 ( ) 1001 -10,000 ( ) >10,000 ( ) type of company: service company ( ) product company ( ) service and product both ( ) project size handled (dad projects) (man hours): (range min-max): team size handled or worked with (dad projects) (range min – max): the perspective with which, you will be ranking the risks in the questionnaire: service ( ) product ( ) common to service and product ( ) software development methodology used (in majority cases): xp ( ) scrum ( ) lean ( ) kanban ( ) traditional (waterfall) approach ( ) hybrid of agile and traditional approach ( ) hybrid of multiple agile methodologies ( ) any other (please specify): … australasian journal of information systems shrivastava & rathod 2019, vol 23, research article risk management in distributed agile projects 27 2 requirements unclear to the team 3 requirement conflicts amongst multiple, geographically dispersed product owners. 4 inadequate communication about end user requirements (social context of end-user not understood) time cost quality 5 inadequate prioritization of requirements. (large project scope; multiple backlogs handled by multiple product owners) 6 rework caused by architectural changes (frequent requirement changes) 7 inconsistency in design standards of distributed teams. 8 technical debts in the code. (deferred work which is not directly related to the functionality but is required for the overall quality of the system) 9 issues with pair programming (distributed developers) 10 code integration across multiple sites. (lack of common understanding of interface requirements between teams) 11 unavailability of requirement documents for testing. (testers depend on their assumptions for testing) 12 cross functional teams insufficient for testing in large projects. (the ‘whole team concept’ may not be sufficient for complex testing) 13 test data management involving test extraction from multiple databases is difficult. (small iterations in agile and lack of test database automation) 14 losing on time for end-to-end extensively interdependent transaction rich test cycle across distributed teams (applicable to transaction rich software application) 15 ineffective stand up meetings. (culture and language difference) 16 different software development practices and standards followed by multiple teams. (cultural diversity and lack of stakeholder support) 17 no common definition of done between the distributed teams. 18 difficulty in system release management and deployment (difficult to synchronize different distributed teams) 19 difficult to execute fixed price projects 20 lower initial velocity (dad teams take time to build trust) 21 using component teams instead of feature teams (e.g. gui team or component x team) is unable to deliver end-to-end feature) 22 growth in team size or development site (leads to instability and hence reduction in productivity) 23 team reorganizing in every sprint (due to changes in market requirements) 24 higher interdependency between the teams. (system complexity and lack of communication between the distributed teams) 25 lack of uniformity in multisite team’s capabilities (loss of productivity) 26 unavailability of business analyst (ba). (single ba is assigned to multiple teams) australasian journal of information systems shrivastava & rathod 2019, vol 23, research article risk management in distributed agile projects 28 time cost quality 27 lack of communication between team members 28 lack of communication between the team and the client (product owner) 29 uncommon language (team members or client may not share a common language) 30 lack of documentation since agile downplay documentation (dad projects need some minimum level of documentation to compensate for the informal team interaction) 31 unsuitability of flat communication structure to large organization. (may expose the client to inexperienced team members) 32 underinvestment on travel by the management. 33 poor coordination between different sites. (difficult to synchronize and integrate different activities) 34 poor collaboration between different sites. (understanding & appreciating the capability differences and reducing the gap by using technology) 35 poor collaboration between developers and quality assurance members. (especially if they belong to different distributed team) 36 lack of trust between the client and the offshore team (it is difficult to customer expectations in short iteration) 37 lack of trust between the onshore and offshore teams. 38 unavailability of product owner (non-collocation or lack of time) 39 poor coordination between multiple vendors involved in the solution development. (especially if they are competing and have work dependencies between them) 40 inappropriate user story estimates with multiple vendors. 41 risk in code integration with multiple vendors. (lack of common understanding of requirements and interfaces) 42 dependency on third party for solution development leading to coordination problems between them. 43 lack of communication infrastructure. (connectivity problems and slow network which hampers the scrum meetings) 44 inappropriate tool selection (especially for practices like automated testing, code integration etc.) survey questionnaire (part ii) for the study impact of risks on project goals in distributed agile development (dad) projects the aim of this part of the research study is to determine the impact of risks occurring in distributed agile development (dad) projects on project goals, namely, time and quality as per the practitioners’ perception. we request the respondents to share their experiences openly and provide their honest response to the questions. we assure the respondents that their responses will be used for academic research purpose only and the confidentiality will be maintained with utmost care. australasian journal of information systems shrivastava & rathod 2019, vol 23, research article risk management in distributed agile projects 29 section 2 (i): ranking of dad risk factors based on the impact on the time of the project please rank the dad risk factors listed below for the extent of the impact, they have on project time as per your perception, where, ‘1’ signifies high impact and ‘8’ signifies low impact. (‘time’ refers to the time used for completing the project) please rank the risk factors based on the amount of impact they have on the time (as a project goal) of the project. • the risk factor with highest impact on time can be given a rank of ‘1’ • the risk factor with comparatively lesser impact on time can be ranked as ‘2’ • following risk factors can be ranked as ‘3’ , ‘4’, ‘5’, ‘6’, ‘7’ as the impact on time keeps reducing. • the risk factor with lowest impact on time can be ranked as ‘8’ please give unique rank between 1 and 8 to each dad risk factor. ranking of risk factors based on the perceived impact on project time ranks (1-8) 1 inadequate prioritization of requirements. (large project scope; multiple backlogs handled by multiple product owners) 2 lower initial velocity (dad teams take time to build trust) 3 requirement conflicts amongst multiple, geographically dispersed product owners 4 rework caused by architectural changes (frequent requirement changes) 5 poor coordination between different sites. (difficult to synchronize and integrate different activities) 6 dependency on third party for solution development leading to coordination problems between them. 7 lack of communication infrastructure. (connectivity problems and slow network which hampers the scrum meetings) 8 inappropriate user story estimates with multiple vendors. respondent’s background information date: name: e-mail: contact number (office): mobile: company name: profile in the company: total experience: experience in managing agile projects: please mark ‘x’ in front of the appropriate answer…. the number of employees in all the locations across the globe: < 50 ( ) 50-100 ( ) 101500 ( ) 501-1000 ( ) 1001 -10,000 ( ) >10,000 ( ) type of company: service company ( ) product company ( ) service and product both ( ) project size handled (dad projects) (man hours): (range min-max): team size handled or worked with (dad projects) (range min – max): the perspective with which, you will be ranking the risks in the questionnaire: service ( ) product ( ) common to service and product ( ) software development methodology used (in majority cases): xp ( ) scrum ( ) lean ( ) kanban ( ) traditional (waterfall) approach ( ) hybrid of agile and traditional approach ( ) hybrid of multiple agile methodologies ( ) any other (please specify): … australasian journal of information systems shrivastava & rathod 2019, vol 23, research article risk management in distributed agile projects 30 section 2 (ii): ranking of dad risk factors based on their perceived impact on the quality of the project outcome please rank the dad risk factors listed below for the extent of the impact, they have on the quality of the project goal as per your perception, where, ‘1’ signifies high impact and ‘7’ signifies low impact. (quality of the project is referred toeffective software process applied in a manner that creates a useful product that provides measurable value for those who produce it and who use it) please rank the risk factors based on the amount of impact they have on the quality (as a project goal) of the project. • the risk factor with highest impact on quality can be given a rank of ‘1’ • the risk factor with comparatively lesser impact on quality can be ranked as ‘2’ • following risk factors can be ranked as ‘3’, ‘4’, ‘5’, ‘6’ as the impact on quality keeps reducing. • the risk factor with lowest impact on quality can be ranked as ‘7’ please give unique rank between 1 and 7 to each dad risk factor. copyright: © 2019 shrivastava & rathod. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. ranking of risk factors based on the perceived impact on quality of the project outcome ranks (1-7) 1 poor collaboration between developers and quality assurance members. (especially if they belong to different distributed team) 2 unavailability of requirement documents for testing. (testers depend on their assumptions for testing) 3 inconsistency in design standards of distributed teams. 4 cross functional teams insufficient for testing in large projects. (the ‘whole team concept’ may not be sufficient for complex testing) 5 different software development practices and standards followed by multiple teams. (cultural diversity and lack of stakeholder support) 6 lack of communication between the team and the client (product owner) 7 uncommon language (team members or client may not share a common language) http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems adam & alam 2024, vol 28, research article is research in aust. and nz: activity 2020 to 2022 received 18 feb 2024, accepted 3 july 2024 after 1 round of review with karlheinz kautz as accepting editor. 1 information systems research in australia and new zealand: a survey of research activity from 2020 to 2022 marc t. p. adam school of information and physical sciences university of newcastle australia email: marc.adam@newcastle.edu.au sultana lubna alam deakin business school deakin university australia abstract: over the past 60 years, the information systems (is) discipline has become an established research field in australia and new zealand. throughout its history, the discipline has experienced an ongoing formation and evolution of its unique identity, with important links to both business and computing-focused research. as a sociotechnical discipline, is research considers a wide range of phenomena that emerge when social systems interact with technical systems. yet, there is a lack of research into how recent socioeconomic and technical developments, such as accelerated digitisation in the aftermath of covid19, widescale university restructuring, and a revision of the government’s research classification schemes, have affected the is discipline. against this backdrop, the present paper seeks to establish the current state of the is discipline in australia and new zealand. based on a survey of 123 is researchers, we confirm the sociotechnical nature of the is discipline in this region, identify research focus areas, provide research output benchmarks at different academic levels, and assess the level of engagement with general and specialised conferences. our results provide important insights into the positioning of the is discipline at the nexus of business and computing-focused research. keywords: researcher demographics; research activity; conference engagement; survey. 1 introduction over the past 60 years, the information systems (is) discipline has become an established research field in australia and new zealand (gable, gregor, et al., 2008; lehmann & huff, 2007), with first academic programs dating back to the late 1960s in australia (pervan & shanks, 2008) and the early 1980s in new zealand (lehmann & huff, 2007). as a sociotechnical discipline, is scholars investigate a wide range of phenomena that emerge when social systems interact with technical systems (lee, 2001; niederman & march, 2012; sarker et al., 2019). as such, the is discipline has important links to both business and computing-focused research, but yet holds its own unique identity (kautz, 2021). hence, also the current state and ongoing evolution of the is discipline needs to be considered both in terms of its relation to developments in business and computing research. throughout its history, the is discipline in australia, new zealand, and beyond has experienced different phases along socioeconomic and technical developments that have influenced the evolution of its identity (e.g., programmer-in-training scheme, dot.com acceleration; clarke, 2008; hirschheim & klein, 2012). in september 2022, the australian and australasian journal of information systems adam & alam 2024, vol 28, research article is research in aust. and nz: activity 2020 to 2022 2 new zealand is professoriate conducted a joint workshop to discuss how the emergence of new external developments had impacted the current state of the is discipline. these included challenges associated with accelerated digitisation in the aftermath of covid19, generational and structural changes for is groups in the wake of widescale university restructuring, and the first large-scale revision of the australian and new zealand standard research classification (anzsrc) scheme in more than a decade. given that previous studies on the current state of the is discipline in australia and new zealand occurred about a decade ago (e.g., lehmann & huff, 2007; pervan & shanks, 2008; ridley, 2006; smyth et al., 2016), the groups agreed that there was a need for a survey that establishes the current state of the is discipline in this region to inform what the discipline can do in its own interest. towards this end, this article sets out to address the following research questions: rq1: what are the demographics of is researchers in australia and new zealand in terms of career stage, role, department, and research outputs? rq2: in which anzsrc fields are is researchers in australia and new zealand active in within and beyond the major business and computing areas? rq3: to what extent do is researchers in australia and new zealand engage with general and specialised is conferences? to address these research questions, we conducted an online survey of is researchers with a primary affiliation at an australian or new zealand university. the survey builds on established schemes for the classification of research fields and differentiation of academic levels. in total, 123 is researchers from 30 universities in australia and 9 universities in new zealand participated. the remainder of this article is organised as follows. in section 2, we provide a brief background on is research in australia and new zealand and its positioning as a field of research in the wider research classification landscape. in section 3, we describe the method of the survey. in section 4, we present the results of the survey in terms of is researcher demographics, activity in specific research fields, and engagement with is conferences. finally, section 5 provides a general discussion of our findings. 2 background 2.1 related work a range of previous studies have provided important insights into the history, focus, and impact of the is discipline in australia and new zealand. given the smaller population size and, hence, smaller number of universities, it is not surprising that the body of work is comparatively smaller for new zealand (e.g., gable, 2007; lehmann & huff, 2007). to the best of our knowledge, first academic analyses of the nature and structure of the is discipline in australia go back to a series of studies in the context of g. ridley (2000)’s doctoral dissertation. ridley et al. (1997, 1998) provided the first overview of the is discipline at australian universities based on an analysis of articles in australasian publication outlets. following up, ridley and keen (1998) reviewed the epistemology of empirical studies conducted by australian academics published between 1980 and 1996. the authors found a strong focus on positivist research, with less than 10% of studies employing interpretive or critical studies. in 2006, the australasian journal of information systems (ajis) published a special issue themed “information systems in australian universities.” building on a contextual framework by gable (2006), the special issue presented seven case studies on the structure, australasian journal of information systems adam & alam 2024, vol 28, research article is research in aust. and nz: activity 2020 to 2022 3 size, curriculum, and research of is discipline groups in the six australian states and one territory. further, it included a historical breakdown by clarke (2006) on the is discipline’s history in australia and its key milestones and phases along socioeconomic and technical developments. two years later, the ajis special issue was extended to an edited book (gable, gregor, et al., 2008). further, following up on the “is-in-oz” multi-case study, the journal communications of the association for information systems (cais) published a special issue on the state of the is discipline in pacific-asia in 2007, including one case study each for australia (gable, 2007) and new zealand (lehmann & huff, 2007). the edited book published in 2008 also included a longitudinal analysis by pervan and shanks (2008) on the current state of is research for the time from 2004 to 2006. following up on a preliminary study by pervan and cecez-kecmanovic (2001), the authors had distributed an annual survey to the heads of all is discipline groups in australian university (pervan & shanks, 2006). thereby, survey respondents reported information on discipline size and structure and as well as the foci, paradigms, methods, and performance of their research at the school level (i.e., one response per school). the authors found a range of 21 different is topics covering a wide range of organisational (organisational implications of is, is management/strategy), societal (societal effects, legal/ethics aspects), and technical matters (e.g. databases, computer and network applications). thereby, the authors found substantial research interest in is management and strategy, the organisational implications of is and it, is adoption/diffusion, theoretical underpinnings of is, and is education. further, they observed that research focus shifted with the emergence of new technologies (e.g., electronic commerce, mobile commerce), while interest in specific topics and technical issues such as computer and network applications, computer-supported cooperative work (cscw), and legal aspects of it were relatively less popular. complementary to surveys, several scholars engaged in bibliometric analysis, often with a particular focus on the peak is conference in the region, the australasian conference on information systems (acis). these include co-author network analysis (cheong & corbitt, 2009; dang-pham & kautz, 2018, 2017) and archival analysis of acis papers to examine longitudinal trends in themes, research methods, and submission rates (gable, smyth, et al., 2008; stead et al., 2020). further, clarke (2009) compared the suitability of the google scholar and thomson/isi metrics to assess the impact of is researchers, noting the importance of taking into account multiple measures to adequately consider the nature of is research. 2.2 classification of is research the anzscr provides a general classification for research in australia and new zealand. with its latest release in 2020, anzsrc was jointly developed by the australian and new zealand governments. 1 among other classifications, the anzsrc includes a fields of research (for) classification that is widely applied across the university sectors in both countries (e.g., for reporting and strategic planning purposes) and beyond (joung et al., 2020; vancauwenbergh & poelmans, 2019). it is widely used by government agencies, public and private funding bodies, research institutes, universities, and other entities to classify, measure, 1 the 2020 anzscr was jointly developed by the australian bureau of statistics, stats nz, the australian research council, and the new zealand ministry of business, innovation and employment. it is available online at: https://www.abs.gov.au/statistics/classifications/australian-and-new-zealand-standard-research-classificationanzsrc https://www.abs.gov.au/statistics/classifications/australian-and-new-zealand-standard-research-classification-anzsrc https://www.abs.gov.au/statistics/classifications/australian-and-new-zealand-standard-research-classification-anzsrc australasian journal of information systems adam & alam 2024, vol 28, research article is research in aust. and nz: activity 2020 to 2022 4 and analyse research and development activities. thereby, the for classification considers 23 general for divisions (2-digit codes), covering the breadth of research from agricultural sciences to psychology. these research divisions are then further broken down into 213 for groups (4-digit codes) and 1967 specific for fields (6-digit codes). the is discipline’s unique identity with its links to both business and computing-related research also becomes apparent in the for structure. specifically, the anzscr captures is research in two different for groups within separate divisions. on the one hand, is research is covered in for group 3503 (business systems in context), which belongs to the broader for division 35 (commerce, management, tourism and services). on the other hand, is research is covered in for group 4609 (information systems), which belongs to the broader for division 46 (information and computing sciences). the anzsrc also provides short definitions for the listed for groups. depending on the particular group, the definitions leave more or less room for interpretation. for the two major is research for groups, it is safe to state that the definitions are rather broad. specifically, the for 3503 definition states: “this group covers business systems in context.” the for group 4609 definition states: “this group covers systems that manage information for organisational purposes.” 2.3 the 2020 anzsrc revision the nature of is as a sociotechnical discipline and its positioning within two separate for divisions repeatedly sparks debates not only within the is discipline but also with neighbouring disciplines in the business and computing areas about the core, focus, and differentiation of is research. this includes a range of debates and consultations of academic bodies around the first major update of anzscr in more than a decade in 2020. across the sector, the 2020 revision saw a 35.7% (58.9%) increase in codes from 157 for groups (1238 for fields) in 2008 to 213 for groups (1967 for fields) in 2020. figure 1 provides an overview of how the classification of is research has changed in anzsrc from 2008 to 2020. one of the major changes in the 2020 anzsrc revision was the introduction of a dedicated for group 3503 (business systems in context). this group separated the business-oriented is research fields from other business and management aspects they were previously grouped with. another important change was the introduction of a new for group 4609 (humancentred computing) that had previously been covered largely by a field within group 080602 (computer-human interaction) in anzsrc 2008. in other words, research in humancomputer interaction is no longer classified within the is for group. given the importance of the anzsrc scheme and the large-scale changes in 2020, it is important to understand in which for fields is researchers are active in to inform the strategic positioning of the discipline. universities tend to align department structures with established disciplinary classifications such as the anzsrc (hider & coe, 2020). hence, information on which particular for groups is academics are active in provides important contextual information to articulate the identity, research culture, and value proposition of the is discipline. further, it is vital for the discipline to continuously monitor to what extent the revised for fields actually cover the research of is academics. this will aid in identifying potential gaps and necessary refinements in the scheme. australasian journal of information systems adam & alam 2024, vol 28, research article is research in aust. and nz: activity 2020 to 2022 5 figure 1. classification of is research in anzsrc 2008 and 2020 2.4 national and international representation of is researchers the two peak bodies to represent the australian and new zealand is professoriate are the australian council of professors and heads in information systems (acphis) and the professors and heads in information systems in new zealand (phis-nz). in close collaboration, acphis and phis-nz represent is academics in matters of national and international importance. in doing so, they liaise with government entities (e.g., the australian research council, arc), professional bodies (e.g., the australian computer society, acs), international organisations (e.g., the association for information systems, ais, and its australasian chapter, aais) and related academic bodies (e.g., australian business deans council, abdc; australian council of deans in ict, acdict). acphis and phis-nz regularly organise forums for the is professoriate to discuss important matters related to is research and the is curriculum. among other initiatives, this has led to the establishment of journal and conference rankings (e.g., the acphis journal list, and the acphis conference lists), consultation with the is professoriate and other academic bodies around the anzsrc revision,2 and recognition of is conferences and journals by related academic bodies (e.g., recognition of is journals in the abdc journal quality list). the present study builds on these rankings and classifications to establish the current state of is research 2 in may 2019, acdict, acphis, alia (australian library and information association), core (computing research & education association of australasia), and phis-nz coordinated a joint submission to the arc in response to the review of the anzsrc (alia, 2019). australasian journal of information systems adam & alam 2024, vol 28, research article is research in aust. and nz: activity 2020 to 2022 6 in australia and new zealand. also, the need for the present survey was identified during a joint acphis and phis-nz workshop in september 2022. 2.5 is conferences and journals the is discipline in australia and new zealand contributes globally to is research in a range of conference and journal outlets. through the regional chapter aais, the is communities in australia, new zealand, and oceania are directly connected with the global is community represented by the ais. acis, the peak is conference in the region, can already look back on more than three decades of history (dang-pham & kautz, 2017; gable, 2008; ridley, 1997).3 co-authorship network analyses emphasise the importance of acis for the is discipline in australia and new zealand (cheong & corbitt, 2009; dang-pham & kautz, 2018, 2017). ajis, the premier journal, first appeared in 1993 and is widely regarded as the top regional outlet of the is field in australasia (fisher et al., 2007; hirschheim & klein, 2012; macgregor, 1993). the inherent links of is as a sociotechnical discipline to business and computing research also become apparent in the importance of not only journal but also conference papers. in contrast to other business-oriented disciplines (e.g., management, marketing), is has a strong history of recognizing peer-reviewed conference papers, somewhat similar to other computingoriented disciplines (e.g., computer science, software engineering) (ridley, 1997; ridley et al., 1998). at the same time, there have been ongoing debates within the is discipline as to the impact of conference papers as compared to journal articles (ridley, 1997; ridley et al., 2008). notwithstanding these ongoing debates and developments over time, the is professoriate recognises the importance of both journal and conference publication avenues by means of discipline-specific rankings and recommendations. as for journal publications, the is professoriate in australia and new zealand supports and recognises academic excellence by means of the acphis journal list. first created in 2006 by a panel of senior is scholars in consultation with the wider is community (fisher et al., 2007), the acphis journal list has undergone several iterations.4 it provides important guidance for is researchers on the quality of different journal outlets (sellitto, 2007). the list is regularly harmonised with the abdc journal quality list, a general list of business research-related journals provided by the abdc. further, as for conference papers, acphis provides two complementary lists of recommended is conferences.5 created in 2014, these lists recognise the importance of conference publications for is research (bunker, 2014). while the generic list includes seven conferences that cover is research broadly (e.g., acis), the specialised list includes fourteen conferences that each focus on a specific is domain (e.g., business process management, decision support systems). however, to date there exists no analysis on is researchers’ engagement with these generic and specialised is conferences. 3 as noted by gable (2008), acis was first run under the name “first annual conference on information systems” at monash university in 1990, followed by “second annual conference on information systems and database special interest group” in 1991. in 1992, it was then renamed to “australian conference on information systems”, before its final naming of “australasian conference on information systems” in 1994. 4 the acphis journal list is available at: https://www.acphis.org/acphis-journal-list 5 the conference recommendation lists are available at: https://www.acphis.org/recommended-is-conferences https://www.acphis.org/acphis-journal-list https://www.acphis.org/recommended-is-conferences australasian journal of information systems adam & alam 2024, vol 28, research article is research in aust. and nz: activity 2020 to 2022 7 3 method the study was carried out within a descriptive, positivist framework (orlikowski & baroudi, 1991). descriptive statistical analysis was employed. 3.1 sample to recruit a wide range of is researchers with a primary affiliation at an australian or new zealand university, recruitment relied on email invitations using three invitation modes. first, the authors directly invited individuals that they identified as potential is researchers based on a web-search conducted on university websites and professional networks (linkedin, researchgate). second, invitation emails were sent to is research community mailing lists in australia and new zealand (acis 2022, is-aus, is-hods, phis-nz). third, the email invitations encouraged snowballing, that is, the survey respondents were encouraged to forward the survey invitations to other is researchers within their university. the study protocol was approved by the ethics committee at the university of newcastle, australia (h2022-0367). after clicking on the survey link, participants were provided with an information statement that detailed the goals, procedure, and investigators of the study, and were asked for consent to participate. they were also encouraged to ask any questions that they may have before proceeding to participation. only after confirming their consent, the survey started. in total, 123 individuals with a primary research affiliation at an australian or new zealand university participated in the survey (79 men, 35 women, 0 other, 9 prefer not to answer). 3.2 survey structure the survey was designed in several iterations with feedback sought from acphis and phisnz members. this included presenting and discussing a pilot survey with the acphis and phis-nz membership in late 2022. the feedback provided through pilot survey responses, general meetings attendees, acphis and phis-nz executives, and individual follow-up emails was then used to refine the survey. the revision of the survey included (1) simplifying the reporting of research activity for specific for codes, (2) increasing the mobile-friendliness of the survey, (3) adding a question around research income, and (4) including links for survey participants to look up background information on for codes and journal rankings. the final survey instrument is provided in the appendix. 3.3 data analysis in line with our three overarching research questions, our analysis builds on descriptive statistics to summarise survey responses along the focus areas of (1) researcher demographics, (2) research activity in specific fields of research, and (3) engagement with conferences. further, we used thematic analysis to extract themes from the optional free-text responses about engagement with is conferences (braun & clarke, 2006). to triangulate and validate our analysis, we adopted a ‘virtuous cycle’ of gathering feedback on our findings through presentations to different forums, in particular acphis special general meetings (june and october 2023). we sought feedback to better explain and interpret the analysis results of is research’s current state. many attendees of these meetings, through their active roles in the is discipline, have immediate and ongoing access to organisational memory, and have developed meaningful knowledge about the state of the is discipline more broadly. in this way, key informants were able to communicate the historical, social, and cultural nuances to the australasian journal of information systems adam & alam 2024, vol 28, research article is research in aust. and nz: activity 2020 to 2022 8 research findings (tremblay, 1957). importantly, this feedback served as progressive learning for the research team about is research in australia and new zealand. 4 results 4.1 is researcher demographics table 1 provides an overview of is researcher demographics with primary affiliation at a university in australia (101 responses from 30 universities) and new zealand (22 responses from 9 universities). this includes academic staff members (97, 78.9%), with adjunct/emeritus academics (9, 7.3%), and research higher degree (rhd) candidates (17, 13.8%) listed separately.6 the latter category comprises of 15 doctor of philosophy (phd) and 2 master of philosophy (mphil) candidates. most academic staff held ongoing positions (89, 92.8%), while 7 (7.2%) academics were on contract positions (mean contract length = 2.57 years [min = 1 year, max = 5 years]). role australia total [m | w | na] new zealand total [m | w | na] both total [m | w | na] professor 12 [8 | 3 | 1] 4 [2 | 0 | 2] 16 [10 | 3 | 3] associate professor 20 [9 | 7 | 4] 8 [4 | 4 | 0] 28 [13 | 11 | 4] senior lecturer 18 [17 | 1 | 0] 7 [5 | 2 | 0] 25 [22 | 3 | 0] lecturer 23 [15 | 7 | 1] 0 [0 | 0 | 0] 23 [15 | 7 | 1] associate lecturer 1 [0 | 1 | 0] 0 [0 | 0 | 0] 1 [0 | 1 | 0] postdoc/other 2 [2 | 0 | 0] 2 [2 | 0 | 0] 4 [4 | 0 | 0] adjunct/emeritus 9 [6 | 3 | 0] 0 [0 | 0 | 0] 9 [6 | 3 | 0] rhd candidate 16 [9 | 6 | 1] 1 [0 | 1 | 0] 17 [9 | 7 | 1] total 101 [66 | 28 | 7] 22 [13 | 7 | 2] 123 [79 | 35 | 9] note: two academics who identified as “assistant professor” are listed were mapped to lecturer rather than postdoc/other; m = man; w = woman; na = “prefer not to answer”. table 1. is researcher demographics table 2 provides an overview of how much full-time equivalent (fte) time of academic staff is dedicated to research, along with the overall fte of the respective role. overall, ftes dedicated to research range between 25 and 45% on average, with distinct differences across academic levels. notably, lecturers reported the highest fte dedicated to research (44.2%). role australia research [total] new zealand research [total] both research [total] professor 32.5% [100%] 45.8% [100%] 35.8% [100%] associate professor 33.5% [83.5%] 45.7% [100%] 37.0% [88.2%] senior lecturer 35.0% [83.3%] 33.1% [98.6%] 34.4% [87.6%] lecturer 44.2% [83.5%] 44.2% [83.5%] associate lecturer 26.7% [100%] 26.7% [100%] postdoc/other 72.7% [90%] 69% [100%] 70.8% [95%] note: the percentage in brackets lists academics’ average fte in the respective role category. table 2. full-time equivalent of academic staff dedicated to research 6 notwithstanding important differences between emeritus and adjunct roles, we decided to create a combined category adjunct/emeritus due to small response rates in the respective categories. in total, the dataset includes responses from 6 emeritus professors, 2 adjunct professors, and 1 adjunct postdoc. australasian journal of information systems adam & alam 2024, vol 28, research article is research in aust. and nz: activity 2020 to 2022 9 given the interdisciplinary nature of the is field as a sociotechnical discipline, there is a range of overarching areas that is researchers can be aligned with. table 3 provides an overview of which department (school, faculty, or college) academic staff reported to be aligned with within the structure of their primary research affiliation. department australia new zealand both business school / faculty / college 41 [53.9%] 13 [61.9%] 54 [55.7%] ict/is/it school / faculty / college 22 [28.9%] 5 [23.8%] 27 [27.8%] engineering and/or science school / faculty / college 11 [14.5%] 3 [14.3%] 14 [14.4%] other 2 [2.6%] 2 [2.1%] note: departments are ordered from highest total number of academics to lowest. “other” included law school and a stem academic unit. table 3. department of academic staff table 4 provides an overview of the research outputs and grant income of academic staff per annum over the three-year period from 2020 to 2022. in total, 97 staff responded to the optional question about research publications and 77 academic staff responded to the optional question around grant income. the calculations are based on the median values for the different academic roles, divided by three to arrive at annual values. please note that the median of the sum of research outputs (e.g., c1 articles) does not equal the sum of medians of individual categories. hence, each median needs to be considered individually for that particular category. these findings are at par with data from the mid-2000s, where the average reported publication count was about 2.3 per annum for each staff member (not broken down by academic seniority), of which one-third was in journals and two-thirds in conferences (pervan & shanks, 2006). role b1 book chapters c1 journal articles total [a* | a | b | c | no rank] e1 conf. papers total [is gen. | other] grant income au$ [nz$] professor 0.0 2.3 [0.8 | 1.2 | 0 | 0 | 0] 2.0 [0.8 | 0.3] au$36,667 [nz$39,472] (including 7.7% who reported $0) associate professor 0.0 2.5 [0.3 | 1.0 | 0 | 0 | 0.2] 1.7 [1.3 | 0] au$23,333 [nz$25,118] (including 33.3% who reported $0) senior lecturer 0.0 1.0 [0 | 0.3 | 0 | 0 | 0] 0.7 [0.3 | 0] au$1,548 [nz$1,667] (including 47.8% who reported $0) lecturer 0.0 0.7 [0 | 0.3 | 0 | 0 | 0] 1.3 [0.7 | 0] au$0 [nz$0] (all reported $0) associate lecturer 0.3 0.3 [0 | 0 | 0 | 0 | 0.3] 0.7 [0 | 0.7] postdoc/other 0.0 0.3 [0 | 0 | 0 | 0 | 0.2] 1.0 [0 | 0.2] au$2,500 [nz$2,691] (including 50.0% who reported $0) responses n = 96 n = 77 australasian journal of information systems adam & alam 2024, vol 28, research article is research in aust. and nz: activity 2020 to 2022 10 note: c1 outputs by abdc ranking. e1 outputs grouped into is generic as per acphis recommended list, or other. grant income refers to an academic’s individual contribution that they accumulated over the three-year period from 2020 to 2022 (not the total grant amount across all investigators), divided by three to arrive at annual values. au$ / nz$ conversion is based on the exchange rate on 2 jan 2023 (au$1 = nz$1.0765). responses refer to how many academic staff responded to the optional questions about research outputs and grant income. for associate lecturers, no median grant income could be calculated due to lack of responses. table 4. annual research publications and grant income of academic staff 4.2 research activity 4.2.1 research activity in for groups 3503 and 4609 the two main for groups for is research are the business-related group 3503 “business systems in context” and the computing-related group 4609 “information systems”. as can be seen in table 5, research activity for group 4609 was sightly higher than for group 3503. most researchers (95, 77.2%) reported research activity in both 3503 and 4609. only few academics reported no activity in either of these two for groups (6, 4.9%).7 further, the table also breaks down total frequencies and relative percentages of activity that is researchers reported for the individual fields within the two main groups. on average, researchers reported activity in 7.02 different for fields within the 3503 and 4609 groups (academic staff: 7.00 [min = 0, max = 21, median = 6], adjunct/emeritus: 5.33 [min = 2, max = 13, median = 3], rhd candidates: 8.00 [min = 0, max = 18, median = 8]). the main for fields that more than 40% of is researchers reported activity in are 350303 (business is; 66.7%), 460908 (is organisation & management; 47.2%), 350307 (technology management; 42.3%), and 350302 (business information management; 41.5%). for group and field academic staff adjunct / emeritus rhd candidate total for group 3503 & 4609 3503 business systems in context 80 [82.5%] 8 [88.9%] 12 [70.6%] 100 [81.3%] 4609 information systems 90 [92.8%] 7 [77.8%] 15 [88.2%] 112 [91.1%] both (3503 and 4609) 77 [79.4%] 6 [66.7%] 12 [70.6%] 95 [77.2%] only one (either 3503 or 4609) 16 [16.5%] 3 [33.3%] 3 [17.6%] 22 [17.9%] neither (neither 3503 or 4609) 4 [4.1%] 2 [22.2%] 0 [0.0%] 6 [4.9%] for fields within 3503 350303 business information systems (is) 63 [64.9%] 7 [77.8%] 12 [70.6%] 82 [66.7%] 350307 technology management 41 [42.3%] 2 [22.2%] 9 [52.9%] 52 [42.3%] 350302 business information management [incl. records, knowledge & intelligence] 41 [42.3%] 1 [11.1%] 9 [52.9%] 51 [41.5%] 350304 business systems in context 37 [38.1%] 4 [44.4%] 8 [47.1%] 49 [39.8%] 350399 not elsewhere classified 39 [40.2%] 3 [33.3%] 4 [23.5%] 46 [37.4%] 350301 business analytics 34 [35.1%] 2 [22.2%] 7 [41.2%] 43 [35.0%] 350306 forensic science and management 5 [5.2%] 1 [5.9%] 6 [4.9%] 350305 forensic intelligence 3 [3.1%] 3 [2.4%] for fields within 4609 460908 is organisation & management 46 [47.4%] 2 [22.2%] 10 [58.8%] 58 [47.2%] 460909 is philosophy, research methods & theory 34 [35.1%] 6 [66.7%] 9 [52.9%] 49 [39.8%] 7 two academic staff reported research activity in other computing-oriented fields (4604 “cybersecurity & privacy”; 4606 “distributed computing & systems software”; 4607 “graphics, augmented reality & games”; 4608 “humancentred computing”). one academic staff reported research activity in another business-oriented field (3507 “strategy, management & organisational behaviour”). the remaining three respondents did not report activity in any for group or field. australasian journal of information systems adam & alam 2024, vol 28, research article is research in aust. and nz: activity 2020 to 2022 11 460902 decision support & group support systems 37 [38.1%] 4 [44.4%] 7 [41.2%] 48 [39.0%] 460999 not elsewhere classified 38 [39.2%] 1 [11.1%] 5 [29.4%] 44 [35.8%] 460905 is development methodologies & practice 36 [37.1%] 2 [22.2%] 5 [29.4%] 43 [35.0%] 460906 is education 35 [36.1%] 2 [22.2%] 5 [29.4%] 42 [34.1%] 460912 knowledge & information management 33 [34.0%] 2 [22.2%] 7 [41.2%] 42 [34.1%] 460910 is user experience design & development 31 [32.0%] 3 [33.3%] 6 [35.3%] 40 [32.5%] 460907 is for sustainable development & the public good 32 [33.0%] 2 [22.2%] 5 [29.4%] 39 [31.7%] 460904 information security management 30 [30.9%] 1 [11.1%] 4 [23.5%] 35 [28.5%] 460911 inter-org, extra-org & global is 24 [24.7%] 1 [11.1%] 9 [52.9%] 34 [27.6%] 460903 information modelling, management & ontologies 20 [20.6%] 3 [33.3%] 8 [47.1%] 31 [25.2%] 460901 business process management 20 [20.6%] 6 [35.3%] 26 [21.1%] note: for groups are ordered from highest research activity to lowest. activity refers to the total count of responses across “somewhat active”, “active”, and “very active”. table 5. research activity in for groups 3503 and 4609 diving further into these results, figure 2 breaks down the level of reported research activity for academic staff. as the most-reported field we observe that 63 academic staff (64.9%) reported research activity in business information systems. four fields reported by more than 40% of academic staff: is organisation & management (46 staff, 47.4%), technology management (41 staff, 42.3%), business information management (41 staff, 42.3%), and business systems in context not elsewhere classified (39 staff, 40.2%). notably, all three of the non-generic for fields include the term “management” in their title, emphasizing the importance of management aspects for is researchers. this is followed by a range of 11 fields that each yield more than 30% of academic staff, and 2 fields mentioned by more than 20% of academic staff. interestingly, the two fields referring to research not elsewhere classified each yielded about 40%. specifically, business systems in context not elsewhere classified (350399) was reported by 39 staff (40.2%) and is not elsewhere classified (460999) was reported by 38 staff (39.2%). finally, only two fields were mentioned by less than 10% of academic staff: forensic science & management (350306) and forensic intelligence (350305). figure 2. is research activity of academic staff in for groups 3503 and 4609 australasian journal of information systems adam & alam 2024, vol 28, research article is research in aust. and nz: activity 2020 to 2022 12 for group 3503 and 4609 note: link strength between for codes indicates the extent to which they were co-mentioned by respondents. for codes 350305, 350306, and 460904 were dropped from a) due to their limited links with the other for codes. figure 3. links between for codes within 3503 and 4609 for academic staff figure 3 illustrates to what extent an academic staff member who was active in one particular for code was also active in another for code. to consider only meaningful relations, we only considered the levels of “active” and “very active”. thereby, the link strength between each pair of nodes is determined by the number of respondents that reported research activity for both of the respective nodes. links across the two groups were driven by 350303 (business is) and 460908 (is organisation & management), which are also generally the two most reported for fields overall, and the two generic fields (350399 and 460999). further, 350303 (business is) was also often co-mentioned with 460909 (is philosophy, research methods & theory) and 460905 (is development methodologies & practice). 4.2.2 research activity in other for groups table 6 provides an overview of other for groups that participants reported research activity in. this includes in particular business-related for groups (within for divisions 35 and 38) and computing-related for groups (within for division 46). in addition, some respondents also mentioned for groups outside business and computing which related to is applications australasian journal of information systems adam & alam 2024, vol 28, research article is research in aust. and nz: activity 2020 to 2022 13 in health and education. for simplicity, only those for groups are listed that were mentioned by at least three survey respondents. for group academic staff adjunct / emeritus rhd candidate total other business for groups 3507 strategy, management & organisational behaviour 19 [19.6%] 2 [22.2%] 4 [23.5%] 25 [20.3%] 3506 marketing 7 [7.2%] 7 [5.7%] 3502 banking, finance and investment 4 [4.1%] 1 [5.9%] 5 [4.1%] 3505 human resources & industrial relations 3 [3.1%] 1 [5.9%] 4 [3.3%] 3801 applied economics 3 [3.1%] 1 [11.1%] 4 [3.3%] 3501 accounting, auditing & accountability 3 [3.1%] 3 [2.4%] 3504 commercial services 3 [3.1%] 3 [2.4%] 3509 transportation, logistics & supply chains 2 [2.1%] 1 [5.9%] 3 [2.4%] 3802 econometrics 2 [2.1%] 1 [5.9%] 3 [2.4%] other computing for groups 4602 artificial intelligence 19 [19.6%] 3 [17.6%] 22 [17.9%] 4601 applied computing 15 [15.5%] 1 [11.1%] 3 [17.6%] 19 [15.4%] 4605 data management & data science 11 [11.3%] 2 [11.8%] 13 [10.6%] 4604 cybersecurity & privacy 11 [11.3%] 1 [5.9%] 12 [9.8%] 4608 human-centred computing 10 [10.3%] 2 [11.8%] 12 [9.8%] 4610 library & information studies 7 [7.2%] 2 [11.8%] 9 [7.3%] 4607 graphics, augmented reality & games 7 [7.2%] 7 [5.7%] 4699 other information & computing sciences 4 [4.1%] 2 [22.2%] 1 [5.9%] 7 [5.7%] 4606 distributed computing & systems software 4 [4.1%] 1 [5.9%] 5 [4.1%] 4603 computer vision & multimedia computation 3 [3.1%] 1 [5.9%] 4 [3.3%] other for groups 4203 health services and systems 6 [6.2%] 2 [11.8%] 8 [6.5%] 3901 curriculum and pedagogy 3 [3.1%] 3 [2.4%] 3903 education systems 2 [2.1%] 1 [5.9%] 3 [2.4%] 4206 public health 2 [2.1%] 1 [11.1%] 3 [2.4%] note: for groups mentioned by three or more survey respondents, ordered from highest research activity to lowest. activity refers to the total count of responses across “somewhat active”, “active”, and “very active”. table 6. research activity in other for groups 4.3 engagement with is conferences role australia new zealand both professor 1.33 2.00 1.50 associate professor 2.05 1.12 1.79 senior lecturer 1.78 2.71 2.04 lecturer 2.04 2.04 associate lecturer 4.00 4.00 postdoc/other 3.5 4.00 3.75 adjunct/emeritus 1.00 1.00 rhd candidate 3.12 3.00 3.12 note: importance values based on averages of individual five-point scale responses (0 = “not at all important”; 4 = “extremely important”). table 7. importance of is conference publications for own research strategy australasian journal of information systems adam & alam 2024, vol 28, research article is research in aust. and nz: activity 2020 to 2022 14 on average, researchers rated publishing at is conferences for their individual research strategy as “moderately important” (mean = 2.057; sd = 1.29). as shown in table 7, the reported importance of conference publications varied considerably across the different roles. the highest importance of conference publications was reported by associate lecturers (4.00), postdoc/other (3.75), and rhd candidates (3.12). by contrast, the lowest importance was reported by professors (1.43) and associate professors (1.64). figure 4 shows the range of responses for academic staff. it can be seen that importance of conference publications for one’s own research strategy decreases with progression in academic seniority. figure 4. importance to publish at is conferences for research strategy of academic staff table 8 list is researchers’ engagement with the seven generic and the fourteen specialised conferences as recommended on the acphis list. here, general engagement refers to submitting and/or reading papers at a particular conference. notably, the submission levels are consistently higher for the generic is conferences (lowest: isd; future submissions = 21%) as compared to the specialised conferences (highest: chi; future submissions = 15%). in addition to the conferences listed in table 8, 28 respondents also mentioned other conferences (26 academic staff, 1 adjunct/emeritus, and 1 rhd candidate). this included a total of 43 different conferences from a range of areas (e.g. accounting, mobile computing, process mining). notably, only two conferences were named more than once, that is the ifip wg 9.4 conference on freedom and social inclusion in a connected world (mentioned by five academic staff) and the medinfo conference (mentioned by two academic staff). conference name academic staff adjunct / emeritus rhd candidate total generic is conferences acis [australasian conf. on info. systems] 90% [70%/72%] 78% [67%/56%] 94% [59%/71%] 89% [68%/71%] icis [int. conf. on info. systems] 86% [58%/67%] 89% [44%/22%] 94% [41%/65%] 87% [54%/63%] pacis [pacific asia conf. on info. systems] 85% [57%/56%] 78% [44%/33%] 82% [41%/47%] 84% [54%/53%] ecis [european conf. on info. systems] 78% [56%/60%] 78% [56%/22%] 88% [29%/65%] 80% [52%/58%] australasian journal of information systems adam & alam 2024, vol 28, research article is research in aust. and nz: activity 2020 to 2022 15 hicss [hawaii int. conf. on system sciences] 81% [51%/55%] 67% [33%/11%] 82% [29%/47%] 80% [46%/50%] amcis [americas conf. on info. systems] 69% [37%/40%] 56% [22%/22%] 76% [24%/53%] 69% [34%/41%] isd [int. conf. on info. systems development] 39% [9%/22%] 22% [0%/11%] 41% [0%/24%] 38% [7%/21%] specialised is conferences desrist [design science research in info. systems & technology] 55% [8%/12%] 44% [11%/0%] 53% [6%/12%] 54% [8%/11%] chi [acm conf. on human factors in computing systems] 52% [8%/18%] 44% [0%/0%] 53% [6%/12%] 51% [7%/15%] bpm [int. conf. on business process management] 46% [9%/12%] 33% [0%/0%] 65% [12%/29%] 48% [9%/14%] ifip wg 8.3 [decision support systems] 45% [5%/8%] 56% [22%/11%] 59% [6%/12%] 48% [7%/9%] interact [int. conf. on humancomputer interaction] 51% [5%/13%] 33% [0%/0%] 41% [6%/6%] 48% [5%/11%] ifip wg 8.2 [the interaction of info. systems & the organization] 46% [5%/12%] 56% [11%/11%] 41% [6%/6%] 46% [6%/11%] ifip wg 8.1 [design & evaluation of info. systems] 46% [5%/9%] 33% [0%/0%] 41% [6%/6%] 45% [5%/8%] caise [int. conf. on advanced info. systems engineering] 43% [6%/8%] 33% [0%/0%] 47% [12%/18%] 43% [7%/9%] bled [bled econference] 43% [12%/7%] 56% [22%/0%] 35% [6%/6%] 43% [12%/7%] ifip wg 8.6 [diffusion, transfer & implementation of info. technology] 42% [5%/9%] 44% [11%/11%] 35% [6%/0%] 41% [6%/8%] ocis [aom organizational communication & info.] 42% [4%/10%] 33% [0%/0%] 41% [6%/6%] 41% [4%/9%] uist [acm symposium on user interface software & technology] 43% [2%/7%] 33% [0%/0%] 35% [6%/6%] 41% [2%/7%] isais [int. symposium on accounting info. systems] 40% [4%/9%] 33% [0%/0%] 35% [6%/6%] 39% [4%/8%] fois [int. conf. on formal ontology in info. systems] 35% [2%/5%] 33% [0%/0%] 35% [6%/6%] 35% [2%/5%] note: generic and specialised is conferences as per acphis recommended conferences list. general engagement [past submission/future submission]; ordered by number of is researcher engagement (from high to low). conf. = conference; info. = information; int. = international. table 8. engagement with is conferences finally, table 9 shows themes extracted from free-text comments about is conferences. in total, 85 respondents provided free-text comments (69 academic staff, 7 adjunct/emeritus, 9 rhd candidates). in terms of the value of publishing at conferences, participants mentioned particularly idea development, networking, benefits for rhd candidates, developing journal articles, and the rapid publishing process. however, participants also mentioned the limited recognition from universities, limited time and funding for attendance, and mixed experiences as challenges for academics to engage in is conferences. three respondents emphasised the importance of hybrid modes, that is, blending the face-to-face attendance with an online option to address challenges around time and funding commitments (tate et al., 2024). australasian journal of information systems adam & alam 2024, vol 28, research article is research in aust. and nz: activity 2020 to 2022 16 theme [count] (sentiment) exemplary quotes limited recognition [24] (–) “does not count towards research minima.” | “[my university] shies away from conference publications for career success which, for my career goals, means i focus less on conference publications.” | “my institution actively discourages any conference publication.” idea development [22] (+) “conferences present opportunities for early feedback on ideas and projects.” | “i enjoy the feedback on short papers and the ability to develop my ideas and get initial feedback. i am then able to proceed with some confidence with my ideas.” | “as a first step to develop a coherent and comprehensive, theoretically grounded position.” networking [21] (+) “i think the lack of participation in conferences has let us not being able to engage with our peers. it has made us a poorer community. however, unless i fund myself i will not be able to go.” | “attending conferences is important. publishing or presenting in conferences is not.” | “the importance of the conference is the contacts i make.” | “it also allows for meeting with fellow researchers, including the new academics coming into the field, and discussing our relative research.” | “publishing in is conferences gets us into the is community.” limited time & funding [17] (–) “for reasons of cost and convenience, i now try to only publish in journals.” | “institutional support for conference attendance is harder than ever.” | “not valued by my university, hard to find the money, hard to justify the time.” | “unless i fund myself i will not be able to go.” benefits for rhd students [15] (+) “publishing at conference is a great 1st step for students to have their research peer reviewed.” | “is conferences are important for my students (and to a lesser extent myself) to gain feedback on their research.” | “i do not submit to any conference, but my ph.d. students find it useful as a first step in their publication strategy.” | “moderately important from my viewpoint. very important if i was responding to this question on behalf of a phd student.” | “as a student, i am publishing primarily to improve my cv and strengthen my thesis.” | “conferences are good opportunity for early airing of joint work with my phd students.” journal paper pre-stage [9] (+) “getting feedback is important before journal submission.” | “conference publishing seems a prerequisite to a*journal consideration.” | “conferences are a playing field for testing ideas and the prestage of journal publishing, but are not valued by uni management.” conference scope [8] (+/–) “acis tends to be more computer science oriented.” | “other conferences, e.g. game design conferences, are more important to me.” | “many of my publications are in cross disciplines/fields, i believe, with cross fields publications, is venues are the most suitable venues to publish my articles.” | “even though i am in is, my work has a more technical orientation than most. so, i do not often target the standard ais conferences.” mixed experience [6] (–) “it's a waste of time. the feedback you get is very poor and the attendance is usually variable.” | “i have found very little interest in my research whilst attending conferences.” | “poor experience in the peer review process for icis, ecis and pacis. much better peer review experience when going direct to journal.” hybrid mode [3] (+) “for academic conferences to survive and remain viable in the future, i believe is conferences need to move a hybrid model combination of face-to-face and online, as travel and accommodation has become much more expensive now.” | “please use a hybrid model allow accepted papers to be presented online at a cheaper registration cost.” | “there is a need for strategic thinking around hybrid conferences that allow remote attendance.” rapid publishing process [3] (+) “journals take too long to be published my field moves too quickly.” | “quicker throughput. acis is too slow in the publication stage. papers appear very late in the ais library.” | “i find conferences largely useful for the quick reviewing process, but i am not interested in networking.” other [2] (+/–) “the is discipline has a strong conference publication culture, papers are fully peer-reviewed with rigorous process.” | “acis is very close to icis making one sometimes having to choose.” note: count refers to how many survey respondents mentioned the respective theme in their free-text comments. themes are ordered based on the number of times they were mentioned in the free-text comments. + denotes positive and – denotes negative discourse. table 9. themes in free-text comments about is conferences australasian journal of information systems adam & alam 2024, vol 28, research article is research in aust. and nz: activity 2020 to 2022 17 5 discussion in its more than 60 years of history, the is discipline in australia and new zealand has evolved along a range of socioeconomic and technical developments. major developments in recent years include in particular accelerated digitisation initiatives throughout the economy in the aftermath of covid19, generational and structural changes in the wake of widescale university restructuring, and the first revision of the government’s research classification schemes in more than a decade. motivated by a joint acphis and phis-nz workshop in september 2022 on these matters, the present study set out to establish the current state of the is discipline in australia and new zealand. in the following, we first provide a brief summary of findings in response to the three overarching research questions stated in the introduction. we then broaden the discussion about the implications that emerge from the present study. finally, we discuss limitations of this study along with concluding notes. 5.1 summary of findings for overarching research questions in terms of demographics (rq1), our survey shows that about half of is academics in australia and new zealand are based in business schools (55.7%), while the remainder are either within a dedicated ict/is/it school (27.8%), or an engineering and/or science school (14.4%). the proportion in business schools is slightly higher in new zealand (61.9%) than it is in australia (53.9%). thereby, the consolidated data show similar levels of is groups outside business schools in comparison to historical data from 2006–2008, which stood at approximately 41.6% (pervan & shanks, 2008). the overall finding is thus aligned with historical positioning of is within universities, that is, while there is a mix of locations for is groups, the majority reside in business schools with a focus on organisations and is professionals as the primary beneficiaries of their research (pervan & shanks, 2008). notably, academics’ time dedicated to research is higher in new zealand than in australia, particularly at the professorial level. differences in the amount and nature of academics’ administrative loads in australia and new zealand may explain this finding. interestingly, the highest research fte overall was reported by lecturers. this might be explained by the notion that some universities equip new lecturer appointments with an additional 10% fte for research, and the limited service loads at this level. finally, we provide an overview of median annual research publications and grant incomes. here, we observe a notable jump in both median grant income and median publications from the lecturer to the professorial levels. further, one can observe that while overall the median number of research publications is similar at the professorial levels, full professors have stronger track records of a*-ranked publications (0.8 vs. 0.3 per annum). as for research activity in specific anzsrc fields (rq2), our survey provides evidence for the strong interdisciplinary nature of the is discipline, both at the discipline and the individual level. specifically, about 80% of is researchers were active in both main for groups (3503, “business systems in context”; 4609, “information systems”). hence, it is not only that the is discipline as a whole actively researches into both areas, but that also most is academics do so at the individual level. this provides important evidence that these two for groups are inextricably linked. also, only six of the respondents reported no research activity in either of these two for groups. hence, as far as anzsrc is concerned, we can conclude that by and large the two for groups 3503 and 4609 cover the bulk of is research activity in australia and new zealand. further, in line with the findings of pervan and shanks (2008), there is a strong focus on organisational aspects of is. in fact, the four for fields reported by more than 40% of our survey respondents all directly relate to business and managerial aspects of is (i.e., australasian journal of information systems adam & alam 2024, vol 28, research article is research in aust. and nz: activity 2020 to 2022 18 350303 business is, 350307 technology management, 35030202 business information management, 460902 is organisation & management). in terms of research outside the two main is for groups, several interesting insights emerge. on the business side, for group 3508 (“strategy, management & organisational behaviour”) stands out uniquely as the only group reported by more than 10% of respondents. on the computing side, we observe a more diverse picture with several areas yielding more than 10% of academic staff: 4602 (artificial intelligence, ai), 4601 (applied computing), 4605 (data management & data science), 4604 (cybersecurity & privacy), and 4608 (human-centred computing). this points to a focus of is academics on challenges and opportunities emerging from technological developments. while pervan and shanks (2008) observed only limited research into technical issues (e.g., computer and network applications, groupware), we note a growing trend towards research into ai, business analytics, cybersecurity, and data science, which directly connect to recent advances in technology and digitisation in society more broadly (vial, 2019). finally, in terms of application areas, reported activity is mainly in the health and the education domain. finally, in terms of conference engagement (rq3), our findings provide a clear picture of which is conferences researchers in australia and new zealand engage with. the six generic conferences organised by ais all yielded high engagement levels (except isd). naturally, the specialised conferences receive less attention than the generic conferences. acis, the peak conference for the australasia region, received the highest engagement (89%). this provides important evidence for the strategic value proposition of acis for the australasian region and beyond, particularly against the backdrop of reduced travel budgets and the major disruption that occurred with covid-19. also, although academics raised concerns over limited recognition from universities, funding, and time constraints, it is reassuring to see conference engagement to slightly increase (more future submissions than past submissions). importantly, performance-based research funding in new zealand recognises conference publications. this may explain why importance of conference publications is slightly higher than in australia, irrespective of academic levels. 5.2 implications for research and practice our results provide important implications for research and practice. broadly speaking, our study provides contextual understanding of is researcher engagement with current anzsrc classification that may be used to inform the is discipline’s strategic positioning within university structures and may assist in re-establishing its identity, research culture, and value proposition in a turbulent post-covid research landscape. however, there are also challenges that arise from the affiliation of is within the university structure partly due to the focus of research on both business and computing research as a sociotechnical discipline. in the following, we formulate four propositions that we draw from our findings. proposition 1 champion unique identity of is as a sociotechnical discipline. overall, our study provides important evidence for the unique identity of is in australia and new zealand as a sociotechnical discipline, with essential links to both business and computing-focused research (kautz, 2021; lee, 2001; sarker et al., 2019). in this vein, the sociotechnical focus of is directly aligns with key challenges and opportunities that industry faces with ongoing technological advances. yet, the unique sociotechnical identity also creates unique challenges for the strategic positioning of the is discipline. australasian journal of information systems adam & alam 2024, vol 28, research article is research in aust. and nz: activity 2020 to 2022 19 specifically, our results show that about half of is academics are based in business schools, with the remainder either based in engineering/science schools or dedicated ict/is/it schools. naturally, universities attempt to align department structures with established disciplinary classifications such as the anzsrc (hider & coe, 2020). however, with the bulk of is research occurring in both business and computing-focused for groups, these attempts repeatedly raise the question of where to best position is within university structures. as noted by hider and coe (2020 p. 11), “interdisciplinary and transdisciplinary configurations may be aligning better with university interests”; yet it is vital for is academics to champion the unique identity of is as a sociotechnical discipline and articulate which benefits arise from such configurations for universities. recent years have seen widescale university restructuring, including relocations of is groups and individual is academics within university structures. these developments require is academics to continuously clarify the focus, research culture, and value proposition of the is field to various stakeholders. the results around research focus may help to inform discussions around positioning of is groups within university structures. proposition 2 –curate the two is-focused for groups as pillars for is research. the fact that the bulk of is research occurs in for groups of different divisions creates the challenge for the is discipline to actively curate both for groups. on the one hand, this requires the is discipline to directly liaise with the respective peak bodies linked to the neighbouring business (e.g., abdc) and computing (e.g., core) for groups. on the other hand, this requires the is discipline to continuously monitor to what extent the fields within the two for groups broadly capture is research, and where adjustments may be required. notably, our results show that the two generic “not elsewhere classified” for fields 350399 and 460999 ranked fifth and sixth in reported research activity. while our survey does not allow to discern which research specifically has been reported with these two generic for codes, their ranking within the top six warrant a debate within the is discipline as to why the other available for codes were not able to cover this research to a higher extent. at the same time, it is noteworthy that two for fields were reported by less than 5% of respondents (359306 forensic science and management, and 350305 forensic intelligence). given the increasing importance of the cybersecurity field for is research, it will be vital to monitor potential changes of activities within these fields over time. also, there may be an increasing need to also include other areas not currently covered by the is-focused for groups 3503 and 4609 (e.g., whether ai is adequately covered by existing for fields). importantly, the anzsrc definitions for for group 3503 and 4609 are rather broad, and there is no clarification provided on what specific for fields capture. this leaves the reading and use of specific for fields such as “business information systems” and “business systems in context” open for interpretation, both from within and outside the is discipline. in fact, even though the present survey specifically recruited is researchers, several participants reached out to the research team to request clarification on the meaning of specific for fields within the two major is for groups. this lack of guidance on the nature of is as a whole and the meaning of specific is research fields is concerning as it opens room for related disciplines, government agencies, and funding bodies to develop and apply their own understanding of is research. on the flipside, the lack of anzsrc definitions and clarifications for specific is for fields is an opportunity for the discipline to provide a clear positioning on the nature and importance for each of these research fields. such a clarification would not only assist in australasian journal of information systems adam & alam 2024, vol 28, research article is research in aust. and nz: activity 2020 to 2022 20 counteracting potential misinterpretations of the nature of is research but also assist in internal strategic discussions around ongoing developments of the is identity. proposition 3 – enhance recognition of is conferences to maximize engagement. our results provide evidence for the role of both peer-reviewed conference papers and journal articles as important publication avenues for is researchers. one the one hand, we see a strong emphasis on top journal outlets, as evidenced from is academics’ focus on a* and a ranked journals, particular at higher levels of academic seniority (see table 4).8 this is notably different from other technical fields such as computer science, where the peak body core decided to discontinue its journal rankings “due to limited resources” in march 2022 and instead exclusively directed efforts to its conference ranking. on the other hand, we see that unlike business-focused disciplines, peer-reviewed conference papers play an important role for is academics; yet, arguably in a different way than for computer science. specifically, our results show that 61% of is academics rated conference publications as moderately important or higher for their research strategy. however, this importance decreases with progression in academic seniority (100% as postdoc and associate lecturer level, 62.5% at (senior) lecturer level, 54.5% at (associate) professor level). to maximize the engagement potential with is conferences, it is important to understand its drivers (networking, idea development, benefits for rhd candidates) and inhibitors (limited recognition, limited funding). in terms of recognition and ability to secure funding, it is vital to provide is academics with strategic guidance and evidence of quality. notably, since 2019 core has not ranked is conferences. acphis does provide a list of recommended generic and specialised is conferences, but the list has not been updated since its first release in 2014. ten years on, our data shows that overall both the generic and also the specialised conference lists appear to be working well. except for isd, the general conferences yield submissions and readership from a wide range of is academics. naturally, submissions to specialised conferences are limited to academics with specific expertise (all below 20%), but readership remains strong (between 35% and 54%). in this sense, no conference stands out from our data as a potential candidate for removal. yet, discussions around whether isd should remain in the generic list or be moved to the specialised list may resurface. also, one may think about the inclusion of ifip wg 9.4 as the only specialised conference outside the existing list that was mentioned by more than two respondents. in any case, there is a need to update the conference recommendation lists as key assets of the is community. proposition 4 – leverage data to drive positioning of the is discipline. over the decades, there have been repeated calls to develop a shared understanding of the history and current practice of the is discipline (hirschheim & klein, 2012). an example is the comprehensive overview of the discipline’s historical roots and ongoing developments in australia in the late 2000s (gable, 2008). to support this, it has been argued that a data-driven approach “will allow the [is] discipline to present a cohesive front” (ridley, 1997 p. 69) and “access increased power, status, and resources” (ridley, 2006 p. 142). yet, despite notable exceptions (e.g., dang-pham & kautz, 2018, 2017; smyth et al., 2016; stead et al., 2020), there has been limited research over the past decade to leverage data collection and analysis to position the is discipline in australia and new zealand. towards this end, the present study 8 background information on the discontinuation of the core journal ranking is available at: https://www.core.edu.au/conference-portal/journal-rankings-history https://www.core.edu.au/conference-portal/journal-rankings-history australasian journal of information systems adam & alam 2024, vol 28, research article is research in aust. and nz: activity 2020 to 2022 21 provides an example of how the is discipline can collect empirical data to assess the current state of the is discipline and derive meaningful insights. building on this, we argue that there is a need for the is discipline in australia and new zealand to better leverage quantitative and qualitative data to inform its strategic positioning and ongoing development. as such, a data-driven approach may assist in the ongoing development of an is core body of knowledge, articulating the value proposition of is to key stakeholders (e.g., government, industry, university executives), and driving innovation in is research culture (hirschheim & klein, 2012; ridley, 2006). one way to support data-driven approaches could be through targeted initiatives that increase data accessibility and simplify interpretation. for instance, researchers could include their unique identifiers (e.g. orcid, scopus id) not only in journal articles but also in is conference publications. further, is researcher profiles (e.g., public google scholar, orcid, and/or university website profiles) could clarify a focus on is research and activity in specific for fields. 5.3 future work and limitations considering the limitations of the present research, several promising research avenues for future work emerge. discipline size. naturally, even though we reached out to the is discipline through three different recruitment channels, only a subset of is researchers responded. however, without reliable estimates of the actual size of the is discipline, it is not possible to determine a sampleto-population ratio and, hence, the generalizability of results needs to be considered with caution. gable and smyth (2007) identified a high-quality faculty directory as a strategic priority for the is discipline. yet, the current ais faculty directory appears to be incomplete.9 hence, there is a need for future research to estimate and monitor the size of the is discipline. complementary to actively managing a faculty directory, this may also include data-driven approaches that draw on researcher identifiers (e.g. orcid, scopus id), electronic libraries (e.g. aisel), and university profile pages. given that not all scholars who published in an is outlet would also consider themselves as an is researcher, such a data-driven approach would need to carefully weigh up the likelihood that a researcher is in fact part of the is discipline (e.g. relative percentages of articles in is journals; engagement with is conferences) and/or seek confirmation from researchers that they consider themselves part of the is discipline. discipline scope. it needs to be noted that our survey focused particularly on the two main for groups for is research (3503 and 4609), which are primarily concerned with organisational aspects of is. similarly, already pervan and shanks (2008) found that is researchers focus mostly on organisations and significantly less on is at the national or global levels. given that our study built on the anzsrc structure, respondents could only report research beyond the specific for fields within these groups by referring to the two generic “not elsewhere classified” for fields 350399 and 450999 and/or by additionally selecting other for groups (within the business and computing divisions, or beyond). this could have led to an under-representation of is research beyond the non-generic fields listed within 3503 and 4609. while the two generic codes ranked fifth and sixth overall, it is not possible for us to discern whether these captured research beyond the organisational context and, if so, which ones in particular. hence, there is a need for future research to discern to what extent the scope 9 as of 7 june 2024, the ais faculty directory lists 190 academics for australia and 53 academics for new zealand (https://directory.aisnet.org). as such, the directory exhibits gaps and the levels of the listed academics are not clear. https://directory.aisnet.org/ australasian journal of information systems adam & alam 2024, vol 28, research article is research in aust. and nz: activity 2020 to 2022 22 of the is discipline in australia and new zealand includes areas beyond the organisational context (applications and impacts beyond organisational boundaries, data modelling, software engineering), and whether this scope is adequately reflected in the anzsrc. at a broader level, clarke (2008) listed seven main thematic clusters in australian is research since 1965 (with cross-fertilisation among the themes): 1) technology as enabler and driver; 2) applications of technology; 3) data management; 4) organisations; 5) systems thinking; 6) business-school thinking; and 7) information management (p. 52). gable et al. (2008) engaged in qualitative coding of acis paper into 32 topics derived from barki et al. (1993) and palvia et al. (2004). stead et al. (2020) classified their coding into the broader four categories: technical, behavioural/managerial, educational (i.e., is curriculum related) and other (predominantly research method papers). future research may explore how these different categorisations can be mapped to the for fields defined in the anzsrc, and possible refinements thereof. research methods. in order not to further extend the survey length, we chose not to include questions around research methodology (e.g., qualitative, quantitative), unit of analysis and beneficiaries (e.g., organisation, industry, individuals), and types of research grants (e.g., arc, industry, internal). future studies may build on the present survey to explore potential methodological differences in how is scholars research into different for fields, and their underlying epistemology. for instance, while our research shows that about 20% of is academic staff research into for group 4602 (ai), the nature of this research is not clear (e.g. development of ai solutions, understanding the societal impacts of ai). future research may employ targeted studies (e.g., bibliometric, surveys) to provide insights into the nature of is research in different application areas (pervan & shanks, 2008; e.g., ridley & keen, 1998). these insights on the methods employed by is researchers will be important to further our understanding of the nature and scope of is research. longitudinal assessments. finally, even though we have made substantial effort to survey the current state of is research in australia and new zealand, we acknowledge that the reported snapshot may not be complete and is subject to ongoing change. as such, it is important to gain a deeper understanding of historical is research trends and characteristics through tracking research interests both at the discipline and the individual researcher level (gable, smyth, et al., 2008; ridley et al., 1997; stead et al., 2020). to capture changes in relation to is key areas of research focus, research output overviews at different academic levels, and overview of the general and specialised conferences that is researchers engage with, there is an ongoing need to regularly engage in further data collection efforts (e.g., regular surveys to investigate trends over time). to the best of our knowledge, the last survey-based longitudinal analysis of is research activity in australia dates back to pervan and shanks (2008). 6 concluding note over the past 60 years, is researcher and practitioners in australia and new zealand have repeatedly felt compelled to justify the importance and legitimacy of is as a sociotechnical discipline (smyth et al., 2016). almost a decade ago, gable et al. (2016 p. 693) “sensed a growing tension between the disciplines and institutions that seek increased allegiance from individuals in the face of increasingly demanding organisational key performance indicators and new directions.” with accelerating digitisation and emergence of new disruptive technologies such as generative ai, other disciplines increasingly consider sociotechnical phenomena that occur at the nexus of technology and society. yet again, this raises important australasian journal of information systems adam & alam 2024, vol 28, research article is research in aust. and nz: activity 2020 to 2022 23 questions for the is discipline’s positioning and value proposition. through the collection and analysis of evidence, studies like ours may serve as a possible vehicle of discipline reinforcement (gable et al., 2016; smyth et al., 2016). in this sense, we hope that the evidence presented in this paper, and the four propositions we have drawn from it, will aid in reinvigorating discussions within the is academic community on “where we are, where we want to be in the future, and how we aim to get there” (pervan & shanks, 2008, p. 304). references alia. (2019). alia submission in response to the review of the anz standard research classification. australian library and information association (alia). barki, h., rivard, s., & talbot, j. (1993). a keyword classification scheme for is research literature: an update. mis quarterly, 17(2), 209–226. https://doi.org/10.2307/249802 braun, v., & clarke, v. (2006). using thematic analysis in psychology. qualitative research in psychology, 3(2), 77–101. https://doi.org/10.1191/1478088706qp063oa bunker, d. (2014). president’s report. australian council of professors and heads of information systems (acphis), 1–5. cheong, f., & corbitt, b. (2009). a social network analysis of the co-authorship network of the australasian conference of information systems from 1990 to 2006. ecis 2009 proceedings, 1–13. clarke, r. (2006). key aspects of the history of the information systems discipline in australia. australasian journal of information systems, 14(1), 123–140. https://doi.org/10.3127/ajis.v14i1.12 clarke, r. (2008). a retrospective of the information systems discipline in australia. in g. g. gable, s. gregor, r. clarke, g. ridley, & r. w. smyth (eds.), the information systems academic discipline in australia (pp. 47–107). anu e press. clarke, r. (2009). a citation analysis of australian information systems researchers: towards a new era? australasian journal of information systems, 15(2), 23–44. https://doi.org/10.3127/ajis.v15i2.465 dang-pham, d., & kautz, k. (2018). a social network analysis of co-authorship at the australasian conference on information systems (acis) from 1990 to 2016. acphis, 1– 70. dang-pham, d., & kautz, k. (2017). exploring the acis community through the analysis of co-authorship across institutions. acis 2017 proceedings, 1–11. fisher, j., shanks, g., & lamp, j. w. (2007). a ranking list for information systems journals. australasian journal of information systems, 14(2), 5–18. https://doi.org/10.3127/ajis.v14i2.469 gable, g. g. (2006). the information systems discipline in australian universities: a contextual framework. australasian journal of information systems, 14(1), 103–122. https://doi.org/10.3127/ajis.v14i1.11 gable, g. g. (2007). the information systems academic discipline in australia. communications of the association for information systems, 21, 23–48. https://doi.org/10.17705/1cais.02102 gable, g. g. (2008). the information systems discipline in australian universities: a contextual framework. in g. g. gable, s. gregor, r. clarke, g. ridley, & r. w. smyth (eds.), the information systems academic discipline in australia (pp. 3–45). anu e press. gable, g. g., gregor, s., clarke, r., ridley, g., & smyth, r. w. (2008). the information systems academic discipline in australia. anu e press. https://doi.org/10.22459/isada.09.2008 australasian journal of information systems adam & alam 2024, vol 28, research article is research in aust. and nz: activity 2020 to 2022 24 gable, g. g., & smyth, r. w. (2007). the state of the is academic discipline in pacific asia: methodological learnings. communications of the association for information systems, 21, 166–194. https://doi.org/10.17705/1cais.02109 gable, g. g., smyth, r. w., & gable, a. (2016). the role of the doctoral consortium: an information systems signature pedagogy? communications of the association for information systems, 38, 678–711. https://doi.org/10.17705/1cais.03833 gable, g. g., smyth, r. w., & stark, k. (2008). an archival analysis of acis research papers. acis 2008 proceedings, 1–10. https://aisel.aisnet.org/acis2008/54 hider, p., & coe, m. (2020). disciplines in their organisational context: mapping australian faculty structures to the asced and anzsrc fields of education and research. australian universities’ review, 62(1), 10–17. hirschheim, r., & klein, h. k. (2012). a glorious and not-so-short history of the information systems field. journal of the association for information systems, 13(4), 188–235. https://doi.org/10.17705/1jais.00294 joung, s., yeongwha, s., & jeong, e. (2020). verifying the classification accuracy for korea’s standardized classification scheme of research f&e by using lda. management & information systems review, 39(1), 35–57. https://doi.org/10.29214/damis.2020.39.1.003 kautz, k. (2021). editorial for the australasian journal of information systems 2021: volume 25. australasian journal of information systems, 25, 1–8. lee, a. s. (2001). editor’s comments: research in information systems what we haven’t learned. mis quarterly, 25(4), v–xv. lehmann, h., & huff, s. l. (2007). the information systems academic discipline in new zealand. communications of the association for information systems, 21, 87–103. https://doi.org/10.17705/1cais.02105 macgregor, r. (1993). editorial. australasian journal of information systems, 1(1). https://doi.org/10.3127/ajis.v1i1.426 niederman, f., & march, s. t. (2012). design science and the accumulation of knowledge in the information systems discipline. acm transactions on management information systems, 3(1), 1–15. orlikowski, w. j., & baroudi, j. j. (1991). studying information technology in organizations: research approaches and assumptions. information systems research, 2(1), 1–28. https://doi.org/10.1287/isre.2.1.1 palvia, p., leary, d., mao, e., midha, v., pinjani, p., & salam, a. f. (2004). research methodologies in mis: an update. communications of the association for information systems, 14, 526–542. https://doi.org/10.17705/1cais.01424 pervan, g., & cecez-kecmanovic, d. (2001). the status of information systems research in australia: preliminary results. acis 2001 proceedings, 1–4. pervan, g., & shanks, g. (2006). the 2005 survey of information systems research in australia. australasian journal of information systems, 14(1), 273–279. https://doi.org/10.3127/ajis.v14i1.22 pervan, g., & shanks, g. (2008). a longitudinal study of information systems research in australia. in g. g. gable, s. gregor, r. clarke, g. ridley, & r. w. smyth (eds.), the information systems academic discipline in australia (pp. 295–305). anu e press. ridley, g. (1997). the role of conferences and refereed journals in australian information systems research. australasian journal of information systems, 5(1), 69–81. https://doi.org/10.3127/ajis.v5i1.351 australasian journal of information systems adam & alam 2024, vol 28, research article is research in aust. and nz: activity 2020 to 2022 25 ridley, g. (2000). information systems research in australia: development and diverseity, 19801996. university of tasmania. ridley, g. (2006). characterising information systems in australia: a theoretical framework. australasian journal of information systems, 14(1), 141–162. https://doi.org/10.3127/ajis.v14i1.13 ridley, g., gable, g. g., smyth, r. w., gregor, s., & clarke, r. (2008). the information systems academic discipline in australian universities: a meta-analysis. in g. g. gable, s. gregor, r. clarke, g. ridley, & r. w. smyth (eds.), the information systems academic discipline in australia (pp. 307–339). ridley, g., goulding, p., lowry, g., & pervan, g. (1998). the australian information systems research community: an analysis of mainstream publication outlets. australasian journal of information systems, 5(2), 69–80. https://doi.org/10.3127/ajis.v5i2.345 ridley, g., goulding, p., lowry, g., & pervan, g. (1997). australian is publications: a profile of institutions, authors and collaboration, 1990-1996. acis 1997 proceedings, 273–287. ridley, g., & keen, c. (1998). epistemologies for information systems research: a study of change. acis 1998 proceedings, 520–533. sarker, s., chatterjee, s., xiao, x., & elbanna, a. (2019). the sociotechnical axis of cohesion for the is discipline: its historical legacy and its continued relevance. mis quarterly, 43(3), 695–719. https://doi.org/10.25300/misq/2019/13747 sellitto, c. (2007). a study of publication performance in acphis identified journals. acis 2007 proceedings, 294–303. smyth, r. w., gable, g. g., & pervan, g. (2016). a swot analysis of the is academic discipline in australia. acis 2016 proceedings, 1–12. stead, c., smith, s., busch, p., & vatanasakdakul, s. (2020). thirty years of acis proceedings: an archival analysis. acis 2020 proceedings, 1–8. https://aisel.aisnet.org/acis2020/19 tate, m., zwanenburg, s., & clarke, r. j. (2024). generating stakeholder value through increasingly hybridized conferences: insights from 30 years of acis conferences. communications of the association for information systems, 54(1), 1–15. tremblay, m.-a. (1957). the key informant technique: a nonethnographic application. american anthropologist, 59(4), 688–701. vancauwenbergh, s., & poelmans, h. (2019). the creation of the flemish research discipline list, an important step forward in harmonising research information (systems). procedia computer science, 146, 265–278. https://doi.org/10.1016/j.procs.2019.01.075 vial, g. (2019). understanding digital transformation: a review and a research agenda. the journal of strategic information systems, 28(2), 118–144. https://doi.org/10.1016/j.jsis.2019.01.003 appendix a1 survey structure in line with the three main objectives of the survey, the final survey comprised three blocks. the first block of questions surveyed information systems (is) researcher demographics in terms of the researcher’s affiliation in australia or new zealand, their department within their organisation, their gender identity, their career stage relative to doctor of philosophy (phd) australasian journal of information systems adam & alam 2024, vol 28, research article is research in aust. and nz: activity 2020 to 2022 26 completion, their current role, their research publications, and their individual research grant income. the second block surveyed research activity in relation to the australian and new zealand standard research classification (anzsrc) field of research (for) code system over the three-year period of 2020 to 2022. this included research activity specifically in the is-related for group 3503 (business systems in context) and for group 4609 (information systems). for each 6-digit for code within for groups 3503 and 4609, participants were asked to report their research activity (0 = not active, 1 = somewhat active, 2 = active, 3 = very active). this was followed by optional questions regarding research activity in other business or computing for groups. finally, participants could optionally also mention research activity in any other for group not previously mentioned. the third block surveyed engagement with generic and specialised is conferences using the set of recommended conferences by the australian council of professors and heads in information systems (acphis). further, participants were able to additionally mention up to five other is conferences not mentioned in the acphis recommended conference list. in addition, participants rated the importance of publishing at is conferences for their research strategy (five-point scale; 0 = not at all important, 4 = extremely important). the survey concluded with optional free-text comments on (1) the importance of is conferences to them, (2) general comments on is research and/or conference, and (3) feedback on the survey. a2 survey instrument a2.1 introduction thank you for considering participating in this survey! the purpose of the research is to create an overview of information systems (is) research in australia and new zealand, in terms of 1) research activity in particular field of research (for) codes as defined by the 2020 australian and new zealand standard research classification (anzsrc); 2) engagement of is researchers with conferences; and 3) demographics of is researchers in australia and new zealand. we are recruiting people aged 18 years and older with a primary research affiliation at an australian or new zealand university who engage in is research. this includes academics (ongoing, contract, postdoc, emeritus, or adjunct) and research higher degree students (mphil or phd candidates). the survey takes about 10 to 20 minutes to complete (depending on how many of the optional questions you choose to answer). if you are ready, please proceed to the participant information statement. a2.2 information statement dear potential research participant, you are invited to participate in the research project noted above which is being conducted by researchers from the university of newcastle in collaboration with deakin university. the australasian journal of information systems adam & alam 2024, vol 28, research article is research in aust. and nz: activity 2020 to 2022 27 information below provides more detail about the study and how you can participate, if you choose to do so. a2.2.1 what is the research study about? the purpose of the research is to create an overview of information systems (is) research in australia and new zealand, in terms of: 1) research activity in particular field of research (for) codes as defined by the australian and new zealand standard research classification (anzsrc) 2020; 2) engagement of is researchers with conferences; and 3) demographics of is researchers in australia and new zealand. the need for this survey was identified during a joint workshop of the australian council of professors and heads of information systems (acphis) and the professors and heads of information systems new zealand (phis-nz) in september 2022. a2.2.2 who is conducting the research? this research project is being conducted by researchers from the university of newcastle’s school of information and physical sciences, in collaboration with a/prof sultana lubna alam from deakin university. a2.2.3 who can participate in the research? this research study is recruiting people aged 18 years and older who engage in is research with a primary research affiliation at an australian or new zealand university. this includes academics (ongoing, contract, postdoc, emeritus, or adjunct) and research higher degree students (mphil or phd candidates). you receive the invitation to participate because: a) your name was identified using a web search using google, a university website, or professional network (linkedin, researchgate, twitter) that indicated potential research activity in the is research field; or b) this invitation was sent to a research community mailing list that you are signed up to; c) it was forwarded to you by someone who had received an invitation and this person felt that the survey could be of interest to you. people who cannot participate include those who do not have a primary research affiliation with a university in australia or new zealand, or who do not engage in is research. a2.2.4 what does participation involve? if you agree to participate, you will be asked to complete an anonymous online survey about 1) your research activity in particular for codes, 2) your engagement with conferences, and 3) general demographics. the survey should take approximately 10 to 20 minutes to complete. the survey must be completed in a single session. please also consider forwarding the invitation that you received to participate in this study to other potential participants in your network. please do not directly forward any contact details of other potential participants to the researchers. a2.2.5 do you have to take part in this research study? no. participation in this research study is voluntary. if you do not want to take part, you do not have to. if you decide to participate and later change your mind, you are free to withdraw from the study at any time prior to submitting your completed survey. due to the anonymous australasian journal of information systems adam & alam 2024, vol 28, research article is research in aust. and nz: activity 2020 to 2022 28 nature of the survey, if you decide to withdraw from the project after submitting a completed survey, we cannot withdraw your responses. a2.2.6 what is the benefit of participating in this research study? by participating in this survey, you will have the opportunity to help the wider is research community to create an overview of the current state of is research in australia and new zealand. this will include important information about research activity in the different for codes, engagement with conferences, and general demographics of is researchers in australia and new zealand. a2.2.7 are there any risks involved in participating in this research? there are no anticipated risks associated with participating in this research. a2.2.8 how will your privacy be protected? due to the anonymous nature of the survey, the responses you provide will not be identifiable. the online survey will be hosted by questionpro online survey platform via a secure encrypted connection as outlined in their privacy policy. data will be retained securely for a minimum period of 5 years from the completion of the research project. information will be managed and stored in accordance with the university’s research data and materials management guideline or any successor guideline as well as any other applicable university of newcastle policy provisions. data will be securely destroyed in line with uon policy provisions. a copy of the data used for analysis will be held at the university of newcastle. a2.2.9 how will information collected by the research team be used? the information will be collated and analysed and may be presented in academic publications, at conferences or in articles. your participation in the research is anonymous and your specific responses will not be identifiable. if you would like a copy of the summary of the results, please record your details using the separate link which will be provided at the end of the main survey. this information is collected and stored separately from your completed survey data. individual participants will not be identifiable in any of the outputs generated from the research project, but individual anonymous responses may be quoted. non-identifiable data may be shared with other parties as part of a peer-review process to verify the robustness and integrity of the study, or to contribute to further research and public knowledge. a2.2.10 what you need to do in order to participate? read this information statement in its entirety and be sure you understand all of the information provided before you agree to participate. if there is anything you do not understand, or if you have questions, contact a/prof marc adam, marc.adam@newcastle.edu.au. if you would like to participate, please use the provided link and complete the online survey. if this document was provided in an electronic form you are encouraged to retain a copy for your reference. completion and submission of the survey will be taken as your implied consent to participate. a2.2.11 do you need more information? if you would like more information about this research project, please contact a/prof marc adam, marc.adam@newcastle.edu.au. thank you, a/prof marc adam, mailto:marc.adam@newcastle.edu.au australasian journal of information systems adam & alam 2024, vol 28, research article is research in aust. and nz: activity 2020 to 2022 29 the university of newcastle, australia concerns or complaints about this research this project has been approved by the university of newcastle’s human research ethics committee, approval no. h-2022-0367. if you have concerns about your rights as a participant in this research, or if you have a complaint about the manner in which the research is conducted, you can contact the chief investigator a/prof marc adam, marc.adam@newcastle.edu.au. if you would prefer to contact someone independent of the research project, you can forward your concerns to: human research ethics officer research and innovation services university of newcastle university drive callaghan nsw 2308, australia p: (02) 4921 6333 e: human-ethics@newcastle.edu.au i have read the information statement, am over 18 and have a primary research affiliation at an australian or new zealand university. i agree to participate in the above research project and give my consent freely.  hereby, i consent that my answers can be used for research purposes.  i do not consent. [→ terminate survey] a3 demographics dem1. country of your primary research affiliation  australia  new zealand dem2a. primary research affiliation in australia [if dem1 == australia] [pull-down list of australian universities] dem2b. primary research affiliation in australia [if dem1 == new zealand] [pull-down list of new zealand universities] dem3. department within the structure of your primary research affiliation.  business school / faculty / college  business & engineering school / faculty / college  engineering school / faculty / college  engineering & science school / faculty / college  ict/is/it school / faculty / college  science school / faculty / college  other [free-text option] dem4. gender identity  woman  man  non-binary  prefer not to answer  other [free-text option] dem5. career stage relative to phd completion. australasian journal of information systems adam & alam 2024, vol 28, research article is research in aust. and nz: activity 2020 to 2022 30 [if you have multiple phds, please respond relative to your phd completion that is most closely related to the is discipline.]  senior career researcher (more than 15 years post phd)  mid career researcher (6 to 15 years post phd)  early career researcher (0 to 5 years post phd)  currently enrolled in phd or mphil  n/a dem6. current role at your primary research affiliation [for emeritus and/or adjunct roles, please select your academic level. a subsequent question will address the status of your role (i.e., ongoing, contract, emeritus, adjunct).]  professor  associate professor  senior lecturer  lecturer  associate lecturer  postdoc  phd candidate  mphil candidate  other dem7. current status of your role at your primary research affiliation  ongoing  contract  stipend/scholarship  emeritus (/retired)  adjunct  other [free-text option] dem8. what is the length of your contract? [if dem7 = contract]  5 years  4 years  3 years  2 years  1 year  6 months  other [free-text option] dem9. what is the full-time equivalent (fte) of your role at your primary research affiliation?  100% (full time)  90%  80%  70%  60%  50%  40%  30%  20%  10%  5% australasian journal of information systems adam & alam 2024, vol 28, research article is research in aust. and nz: activity 2020 to 2022 31  other [free-text option] dem10: over the past three (3) years (2020-2022), what full-time equivalent (fte) percentage of your time has been dedicated to research. [for teaching & research academics, the fte dedicated to research is commonly 40%. for teachingintensive academics, it is often 20%. if you are in a part-time role, please adjust relative to your parttime load. for instance, if you are on an 80% role with 40% research focus, the fte dedicated to research is 32%. if you are research active at several institutions, please consider the total fte that you dedicate to research.] 2022: [pull-down menu: 0-100%] 2021: [pull-down menu: 0-100%] 2020: [pull-down menu: 0-100%] dem11: over the past three (3) years (2020-2022), how many peer-reviewed journal articles (c1) have you published in different australian business deans council (abdc) ranking categories? [abdc journal ranking is available here: https://abdc.edu.au/research/abdc-journal-quality-list] a* journal (abdc) a journal (abdc) b journal (abdc) c journal (abdc) journal not ranked by abdc number of c1 articles [numeric input] [numeric input] [numeric input] [numeric input] [numeric input] dem12: over the past three (3) years (2020-2022), how many peer-reviewed conference papers (e1) have you published? generic is conferences (acis, amcis, ecis, hicss, icis, isd, pacis) other conferences number of e1 articles [numeric input] [numeric input] dem13: over the past three (3) years (2020-2022), how many peer-reviewed book chapters (b1) have you published? [numeric input] dem14: over the past three (3) years (2020-2022), what was the sum of your individual research grant income? (i.e., the individual contribution that you accumulated over the three year period, not the total amount of the grants across all investigators) [don’t show question for mphil or phd candidates] [example: consider a $200k grant that you have been awarded for the year of 2021 (one-year grant) as part of a team of five researchers. assuming that your contribution to that team was 20%, then your individual research income from this grant would be $40k in 2021. conversely, if you were the sole investigator (no team) and the grant ran over four years (2021-2024), then your individual grant income would be $50k in 2021, and $50k in 2022 ($100k in total from this grant for the 2020-2022 period). the years of 2023 and 2024 are not considered in this survey as it only focuses on 2020-2022.] [amount in au$ or nz$ depending on whether your primary research affiliation is in australia or new zealand.] https://abdc.edu.au/research/abdc-journal-quality-list australasian journal of information systems adam & alam 2024, vol 28, research article is research in aust. and nz: activity 2020 to 2022 32 au$/nz$ [numeric input] a4 research activity research activity in is-related for codes (3503 and 4609) within the anzsrc for-code system, there are two main 4-digit codes linked to is research: • 3503 (business systems in context) • 4609 (information systems) res1: over the past three (3) years (2020-2022), what was your level of research activity for each of the 6-digit for codes within 3503. [no selection of a particular for-code will be interpreted as no research activity in the respective area.] not active somewhat active active very active 350301 (business analytics) 350302 (business information management; incl. records, knowledge and intelligence) 350303 (business information systems) 350304 (business systems in context) 350305 (forensic intelligence) 350306 (forensic science and management) 350307 (technology management) 350399 (business systems in context not elsewhere classified) res2: over the past three (3) years (2020-2022), what was your level of research activity for each of the 6-digit for codes within 4609. [no selection of a particular for-code will be interpreted as no research activity in the respective area.] not active somewhat active active very active 460901 (business process management) 460902 (decision support and group support systems) 460903 (information modelling, management and ontologies) 460904 (information security management) 460905 (information systems development methodologies and practice) 460906 (information systems education) 460907 (information systems for sustainable development and the public good) 460908 (information systems organisation and management) 460909 (information systems philosophy, research methods and theory) 460910 (information systems user experience design and development) 460911 (inter-organisational, extra-organisational and global information systems) 460912 (knowledge and information management) 460999 (information systems not elsewhere classified) australasian journal of information systems adam & alam 2024, vol 28, research article is research in aust. and nz: activity 2020 to 2022 33 optional: research activity in other for codes (outside 3503 and 4609) res3 (optional): select up to five business for codes that you have been research active in over the past three (3) years (2020-2022). for each of the selected for codes, indicate your level of research activity within that area. [if you have not been research active in any of these areas, please leave this question empty.] business for-code (6 digit) not active somewhat active active very active business forcode #1 [pull-down: all 6-digit codes under 3501, 3502, 3504–3509, 3801–3803, and 3899] business forcode #2 [pull-down: all 6-digit codes under 3501, 3502, 3504–3509, 3801–3803, and 3899] business forcode #3 [pull-down: all 6-digit codes under 3501, 3502, 3504–3509, 3801–3803, and 3899] business forcode #4 [pull-down: all 6-digit codes under 3501, 3502, 3504–3509, 3801–3803, and 3899] business forcode #5 [pull-down: all 6-digit codes under 3501, 3502, 3504–3509, 3801–3803, and 3899] res4 (optional): select up to five computing for codes that you have been research active in over the past three (3) years (2020-2022). for each of the selected for codes, indicate your level of research activity within that area. [if you have not been research active in any of these areas, please leave this question empty.] business for-code (6 digit) not active somewhat active active very active computing for-code #1 [pull-down: all 6-digit codes under 4601– 4608, 4610–4613, and 4699] computing for-code #2 [pull-down: all 6-digit codes under 4601– 4608, 4610–4613, and 4699] computing for-code #3 [pull-down: all 6-digit codes under 4601– 4608, 4610–4613, and 4699] computing for-code #4 [pull-down: all 6-digit codes under 4601– 4608, 4610–4613, and 4699] computing for-code #5 [pull-down: all 6-digit codes under 4601– 4608, 4610–4613, and 4699] res5 (optional): select up to five 4-digit for codes outside is, business, and computing that you have been research active in over the past three (3) years (2020-2022). for each of the selected for codes, indicate your level of research activity within that area. [if you have not been research active in any of these areas, please leave this question empty.] business for-code (6 digit) not active somewhat active active very active #1 [pull-down: all 4-digit codes outside 35 and 46] #2 [pull-down: all 4-digit codes outside 35 and 46] #3 [pull-down: all 4-digit codes outside 35 and 46] #4 [pull-down: all 4-digit codes outside 35 and 46] #5 [pull-down: all 4-digit codes outside 35 and 46] australasian journal of information systems adam & alam 2024, vol 28, research article is research in aust. and nz: activity 2020 to 2022 34 a5 is conferences conf1: how important is publishing at is conferences for your research strategy?  not at all important  slightly important  moderately important  very important  extremely important conf2 (optional): please elaborate on your answer to the previous question on the importance of publishing at is conferences for your research strategy. [free-text response] conf3: please indicate your level of interest and engagement with regard to the following generic is conferences. [please consider the three (3) year period from 2020 to 2022 when referring to "the past".] i do not (or hardly ever) engage with this conference. i read papers but do not submit (neither in past or in the future). submitted in the past but will not submit in the future. have not submitted in the past, but plan to submit in the future. submitted in the past and plan to submit in the future. australasian conference on information systems (acis) [tick one] [tick one] [tick one] [tick one] [tick one] americas conference on information systems (amcis) [tick one] [tick one] [tick one] [tick one] [tick one] european conference on information systems (ecis) [tick one] [tick one] [tick one] [tick one] [tick one] hawaii international conference on system sciences (hicss) [tick one] [tick one] [tick one] [tick one] [tick one] international conference on information systems development (isd) [tick one] [tick one] [tick one] [tick one] [tick one] international conference on information systems (icis) [tick one] [tick one] [tick one] [tick one] [tick one] pacific asia conference on information systems (pacis) [tick one] [tick one] [tick one] [tick one] [tick one] conf4: please indicate your level of interest and engagement with regard to the following specialised is conferences. [please consider the three (3) year period from 2020 to 2022 when referring to "the past".] i do not (or hardly ever) engage with this conference. i read papers but do not submit (neither in past or in the future). submitted in the past but will not submit in the future. have not submitted in the past, but plan to submit in the future. submitted in the past and plan to submit in the future. australasian journal of information systems adam & alam 2024, vol 28, research article is research in aust. and nz: activity 2020 to 2022 35 acm conference on human factors in computing systems (chi) [tick one] [tick one] [tick one] [tick one] [tick one] acm symposium on user interface software and technology (uist) [tick one] [tick one] [tick one] [tick one] [tick one] aom organizational communication and information (ocis) [tick one] [tick one] [tick one] [tick one] [tick one] bled econference [tick one] [tick one] [tick one] [tick one] [tick one] international conference on advanced information systems engineering (caise) [tick one] [tick one] [tick one] [tick one] [tick one] international conference on business process management (bpm) [tick one] [tick one] [tick one] [tick one] [tick one] international conference on design science research in information systems and technology (desrist) [tick one] [tick one] [tick one] [tick one] [tick one] international conference on formal ontology in information systems (fois) [tick one] [tick one] [tick one] [tick one] [tick one] international conference on human-computer interaction (interact) [tick one] [tick one] [tick one] [tick one] [tick one] international symposium on accounting information systems (isais) [tick one] [tick one] [tick one] [tick one] [tick one] ifip wg 8.1 – design and evaluation of information systems [tick one] [tick one] [tick one] [tick one] [tick one] ifip wg 8.2 – the interaction of information systems and the organization [tick one] [tick one] [tick one] [tick one] [tick one] ifip wg 8.3 – decision support systems [tick one] [tick one] [tick one] [tick one] [tick one] ifip wg 8.6 – diffusion, transfer and implementation of information technology [tick one] [tick one] [tick one] [tick one] [tick one] conf3 (optional): if there are any other is conferences that you have an interest and/or engagement with, please list up to five of those in the table below. [please consider the three (3) year period from 2020 to 2022 when referring to "the past".] i do not (or hardly ever) engage with this conference. i read papers but do not submit (neither in past submitted in the past but will not submit in the future. have not submitted in the past, but plan to submit in the future. submitted in the past and plan to submit in the future. australasian journal of information systems adam & alam 2024, vol 28, research article is research in aust. and nz: activity 2020 to 2022 36 or in the future). #1 [enter conference name] [tick one] [tick one] [tick one] [tick one] [tick one] #2 [enter conference name] [tick one] [tick one] [tick one] [tick one] [tick one] #3 [enter conference name] [tick one] [tick one] [tick one] [tick one] [tick one] #4 [enter conference name] [tick one] [tick one] [tick one] [tick one] [tick one] #5 [enter conference name] [tick one] [tick one] [tick one] [tick one] [tick one] a6 comments optional: do you have any additional comments about is research and/or conferences that you would like to share? (e.g. challenges and opportunities for the is discipline in australia and new zealand, future of conferences, etc) [multiple row answer text] optional: general suggestions and comments about the design and analysis of this survey [multiple row answer text] acknowledgements. the authors thank craig mcdonald and sherah kurnia for their feedback and suggestions on an earlier version of this manuscript. further, we thank members of is the professoriate in australia and new zealand for their feedback on preliminary results we presented at an online workshop in november 2023. copyright copyright © 2024 adam and alam this is an open-access article licensed under a creative commons attribution-non-commercial 4.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v28.5079 https://creativecommons.org/licenses/by-nc/4.0/deed.en https://creativecommons.org/licenses/by-nc/4.0/deed.en https://doi.org/10.3127/ajis.v28.5079 paper title (paper title style) australasian journal of information systems shaik et al., 2025, vol 29, research article an identification-based understanding of team engagement in gvts received 21 jun 2024, accepted 29 mar 2025 after 3 rounds of review with petros chamakiotis as accepting senior editor, m. davern & s. black as editors in chief 1 an identification-based understanding of team engagement in global virtual teams (gvts) farheen fathima shaik xavier labor relations institute jamshedpur, india upam pushpak makhecha indian institute of management tiruchirappalli tiruchirappalli, india biju varkkey indian institute of management ahmedabad ahmedabad, india sirish kumar gouda indian institute of management tiruchirappalli tiruchirappalli, india email: sirish@iimtrichy.ac.in abstract because of globalization and technological advancements, organizations have adopted virtual work arrangements, specifically global virtual teams (gvts). this study conducted a 16month ethnographic inquiry in a multinational enterprise to explore team engagement in gvts. the findings indicate that gvt members often handle multiple roles across various teams and organizations and identify with these entities separately, thus displaying different levels of identification with roles, teams, and organizations. these three identification cascades affect other members' engagement and overall team engagement. higher levels of identification with roles, teams, and organizations trigger a positive engagement contagion across the gvt, whereas a lower level of identification triggers a negative engagement contagion. we also identify four distinct configurations of gvt members, illustrating the complex nature of engagement dynamics in gvt settings. this identification-based understanding of engagement in gvts contributes to the literature on team engagement and is by highlighting the significance of understanding the sensitive dependence of gvt team engagement on members’ identification with their roles, teams, and organizations and subsequent engagement contagion. keywords: global virtual teams (gvts), employee engagement, identification, ethnography. 1 introduction work configurations in modern organizations have evolved significantly in recent years due to globalization and technological developments to accommodate remote forms of work arrangements. according to a kpmg report (2022), nearly 90% of the surveyed companies had remote working arrangements. microsoft work lab (2021) reported a 148% increase in weekly meetings, a 40.6b increase in the number of emails delivered, a 45% increase in weekly team chats per person, and a 66% increase in the number of people working on office documents from february 2020 to february 2021, signaling a more digitized and virtual work australasian journal of information systems shaik et al., 2025, vol 29, research article an identification-based understanding of team engagement in gvts 2 environment alongside siloed teams. with the increase in digitization, employees are at the inflection point of leaving their employers (approximately 41%), signaling a decreased overall engagement in remote work conditions. this study explores an identification-based understanding of engagement in remote working arrangements called global virtual teams (gvts). in gvts, team members are separated by time and space and come from diverse cultures and backgrounds using technology-mediated communication for collaboration and interaction to organize tasks (chamakiotis et al., 2013; choi, 2020; taras et al., 2019). although engagement in gvts has already been explored (e.g., maslach et al., 2001; shuck et al., 2017), we adopt a multilevel dynamic view of team engagement by highlighting the possibilities of multiple identifications in gvts as members juggle across multiple roles, teams, and organizations. this multiple identification-based understanding warrants separate attention to signify its impact on gvt engagement. working in gvts has challenges ranging from understanding speech patterns and communicating effectively in a multicultural setting to efficiently leading gvts, developing personal and professional rapport, working across various technology-mediated communication devices, and selecting appropriate channels of communication (adamovic, 2018; jimenez et al., 2017; shaik & makhecha, 2019). these challenges affect gvt members' productivity, increase costs, create stress, and reduce retention and morale (cathro, 2020; grenny & maxfield, 2017). organizations intend to mitigate the negative impacts by increasing employee engagement (gilson et al., 2015). while higher engagement levels lead to better organizational productivity (harter et al., 2002) and results (gallup, 2023; kpmg, 2022) engagement in gvts is often lower. existing literature has explored individual member engagement in gvts (bakker & demerouti, 2007; saks, 2019). panteli et al. (2019) examined how leadership behaviors affect engagement in gvts, shaik and makhecha, (2019) investigated the drivers of employee engagement, and shaik et al. (2020) explored the role of cultural intelligence in fostering engagement in gvts. the predominant focus on individual engagement in gvts overlooks team-level engagement processes emerging from social identification mechanisms (ashforth & mael, 1989; van dick et al., 2004). since engagement is contagious among team members (bakker, 2022; hatfield et al., 1993) and members in gvts play different roles across teams and organizations, understanding the role of multiple identifications (role, team, and organizational) in shaping engagement contagion is crucial. this study develops an identification-based perspective on gvt engagement and explores how different identification configurations influence engagement gain and loss cycles, thus bridging the existing gap in literature (costa et al., 2017; yun & beehr, 2024). thus, this study contributes to both theory and practice by offering actionable insights for organizations to enhance team engagement in virtual work environments (aon, 2022; gallup, 2023; kpmg, 2022). this study investigates the dynamics of team engagement in gvts by addressing the following key research question: how do multiple forms of identification—role, team, and organizational identification—affect engagement contagion within gvts? this study explores the interplay among these forms of identification and their impact on both individualand team-level engagement in virtual, multicultural, and geographically dispersed teams. by examining the identification-based dynamics of engagement, this study has two objectives: first, to understand how engagement spreads among team members through identification processes, and second, to offer actionable insights for organizations to australasian journal of information systems shaik et al., 2025, vol 29, research article an identification-based understanding of team engagement in gvts 3 foster positive engagement contagion to enhance overall team performance and cohesion in gvts. this study contributes to the theoretical and practical knowledge of virtual teamwork by proposing a novel framework; identifying engagement contagion; and illustrating how varying levels of identification can lead to positive or negative engagement cycles within globally distributed teams. the study introduces the concept of “identification-based engagement contagion,” theorizing how engagement spreads within gvts through social identification mechanisms, extending emotional contagion theory to virtual work settings (barsade, 2002; bakker & sanz vergel, 2013). by uncovering the non-linear dynamics of engagement contagion, this study contributes to the information systems (is) and hrm literature on virtual teamwork by providing empirical evidence from a 16-month ethnographic inquiry. this study highlights the interplay between technology-mediated communication and team identification. the findings offer practical insights for organizations to foster positive engagement contagion by strengthening identification processes in remote and hybrid teams to enhance performance and collaboration in globally distributed work environments (gallup, 2023; kpmg, 2022). for practitioners, understanding these identification forms enables targeted interventions to enhance role clarity and organizational identification, which is particularly valuable for members at client locations. 2 literature review 2.1 hybrid work and gvts the modern organizational landscape has been transformed with the emergence of hybrid work and gvts (choudhury et al., 2024; hopkins & bardoel, 2023). the covid-19 pandemic further expedited the adoption of remote and hybrid work models, introducing greater complexity into work arrangements (chamakiotis et al., 2021; kniffin et al., 2021). although gvts operate through geographical dispersion and technology-mediated communication (chamakiotis et al., 2013), hybrid work combines in-office and remote elements (raghuram et al., 2019). a significant challenge emerging from hybrid work is “presence disparity,” in which inequalities arise between members who can meet face-to-face and those who cannot (malhotra & majchrzak, 2014; santillan et al., 2023). this disparity can diminish team engagement and collaboration, thus necessitating targeted engagement strategies. the hybrid work model blurs traditional organizational structures, often requiring employees to navigate multiple team memberships and organizational affiliations simultaneously (choudhury et al., 2021; raghuram et al., 2019). this necessitates understanding of how different identification processes contribute to the engagement dynamics in virtual teams. more specifically, gvt members’ ability to identify with their roles, teams, and organizations can significantly impact team cohesion, knowledge-sharing, and overall engagement contagion cycles (bakker, 2022; panteli et al., 2019). understanding these identification processes helps unravel engagement contagion mechanisms, as stronger identification at one level can reinforce engagement at another, whereas weaker identification may lead to disengagement and knowledge silos (reade, 2001; ramarajan, 2014). this transition is crucial for developing our central argument that identification-based engagement contagion is a key driver of team-level engagement in gvts and offers new insights into the interconnected nature of engagement in globally distributed teams (bakker & sanz-vergel, 2013; yun & beehr, 2024). by identifying how these engagement patterns emerge in gvts, this study advances both the theoretical and practical australasian journal of information systems shaik et al., 2025, vol 29, research article an identification-based understanding of team engagement in gvts 4 perspectives on multi-level dynamic team engagement processes to improve the effectiveness of virtual teamwork in hybrid settings. 2.2 identification in gvts identification was introduced by sanford (1955) and later defined by kagan (1958) as an "acquired cognitive response within a person." kelman (1958) suggested that "identification is a self-defining response, set in a specific relationship." the identification literature has been explored at an organization, team, and role levels. the foundation lies in the social identity theory (sit) (turner & tajfel, 1979), which explains how individuals derive their selfconcepts from social group membership. this theory has been extensively applied in organizational contexts through “organizational identification” (ashforth & mael, 1989), linking higher identification levels to increased commitment and organizational citizenship behavior (riketta, 2005; mirbabaie et al., 2021). wiesenfeld et al., (1999) pioneered research on organizational identification in virtual contexts, emphasizing its role as a psychological bond that compensates for physical distance. in gvts, members exhibit nested identities and simultaneously identify with parent and client teams (connaughton & daly, 2004; lechner & tobias, 2022). the concept of multiple organizational identifications, introduced by reicher et al., (1995) and expanded by van dick et al. (2004), has become particularly relevant as gvt members navigate multiple work groups across organizational and cultural boundaries. sivunen (2006) identified four tactics to foster team identification: addressing individual needs, providing positive feedback, emphasizing common goals, and promoting team activities. fiol and o'conner (2005) highlighted the role of technology in shaping identity. although well studied in traditional settings (ashforth et al., 2008), role identification remains underexplored in virtual contexts. organizational identification and engagement are related to each other (he et al., 2014; panteli et al., 2008; sun et al., 2023). while engagement contagion has been studied in traditional settings (bakker et al., 2006; panteli, 2019; yun & beehr, 2024), its manifestation in gvts and its relationship with identification remains understudied, which we address by understanding team-level engagement dynamics in virtual contexts. this understanding is particularly significant because in the hybrid and virtual work context, employees experience identity fragmentation, loyalty conflicts, and shifting organizational commitments (reade, 2001; ramarajan, 2014). stronger identification with a team or organization enhances commitment, knowledge-sharing behaviors, and overall performance. however, little attention has been paid to how these multiple levels of identification interact within gvts and contribute to engagement contagion (wiesenfeld et al., 1999; fiol & o’connor, 2005). the interplay among role, team, and organizational identification suggests that engagement is a dynamic and socially shared process, where shifts in identification can trigger engagement gain or loss cycles (bakker, 2022; costa et al., 2017). understanding these identification mechanisms is essential to explore how engagement spreads across team members, reinforcing the idea that engagement is contagious in virtual settings (bakker & sanz-vergel, 2013; hatfield et al., 1993). by integrating identification theory with engagement contagion dynamics, we provide a novel theoretical perspective on gvt engagement along with practical implications for strategic management through identification-focused interventions. section 2.3 focuses on the challenges of sustaining engagement in distributed teams and the mechanisms through which identification-based engagement contagion occurs. australasian journal of information systems shaik et al., 2025, vol 29, research article an identification-based understanding of team engagement in gvts 5 2.3 employee engagement in gvts the importance of employee engagement for organizational success and competitive advantage is well established (chanana & sangeeta, 2021; lee et al., 2021; macey et al., 2009; sun & bunchapattanasakda, 2019). practitioners also find this concept extremely significant (gallup, 2023; kpmg, 2022; aon, 2022; wtw, 2021) as engaged employees bring their cognitive, emotional, and physical selves to their jobs, resulting in increased productivity and well-being. conversely, disengaged employees detach themselves cognitively and emotionally from their jobs (kahn, 1990). psychological meaningfulness, availability, and safety are vital for achieving higher engagement levels (kahn, 1990). work engagement, the opposite of burnout, is characterized by greater vigor, dedication, and absorption in job tasks (maslach et al., 2001). engagement is influenced by job characteristics, organizational and supervisor support, rewards and recognition, and justice (farndale et al., 2014; saks, 2019). although most studies focus on individual engagement, the antecedents and dynamics of team engagement (adamovic, 2017; shuck et al., 2017), specifically in gvts, have garnered little attention (gilson et al., 2015). the focus on team engagement is more relevant in the context of gvts, as team members' productivity is influenced by several factors such as differences in time zones, cultures, technology-based communication, and lack of face-to-face interaction (adamovic, 2017; dulebohn & hoch, 2017; shaik & makhecha, 2019). gvts tend to have lower cohesiveness than collocated teams (polzer et al., 2006), and team members may have less trust in each other (adamovic, 2017; aten & thomas, 2016; newell et al., 2007). furthermore, gvts are culturally diverse, leading to difficulties forming group identities and sharing norms and meanings (gilson et al., 2015; shaik et al., 2020). a limited number of studies on team engagement establish it as an emergent, multilevel process influenced by interactions and shared experiences (e.g. costa et al., 2012). costa et al. (2017) characterized engagement as dynamic, emphasizing the roles of team coordination, learning, and adaptation. they also introduced crossover effects between individual and team engagement and mentioned that the specific manifestation mechanisms require further investigation. we address this gap in our study to gain a more comprehensive understanding of team engagement dynamics. 2.4 theoretical underpinning this study is grounded in two key theoretical frameworks: the job demands-resources (jd-r) model and emotional contagion theory. these theories help understand engagement processes at both the individual and team levels in organizations. however, our findings extend, challenge, and refine certain assumptions within these frameworks, particularly in the context of gvts, where engagement operates under unique structural and identificationbased conditions (bakker & demerouti, 2007; hatfield et al., 1993; reade, 2001). the jd-r model (bakker & demerouti, 2007) explains employee engagement and burnout by categorizing work characteristics into job demands (which require sustained effort and may cause stress) and job resources (which help employees manage demands and foster motivation). when job resources outweigh job demands, engagement occurs, leading to greater vigor, dedication, and absorption at work. our study challenges and extends the jdr model in key ways, particularly in the context of the gvts. first, while the jd-r model focuses on tangible job resources, such as supervisor support, autonomy, and task clarity, we australasian journal of information systems shaik et al., 2025, vol 29, research article an identification-based understanding of team engagement in gvts 6 argue for the equally important role of intangible resources, such as social identification (role, team, and organization) in the context of gvts (ashforth & mael, 1989; van dick et al., 2004). employees with stronger team and organizational identification remain engaged even in the absence of conventional job resources, whereas weak identification functions as a hidden job demand, causing disengagement. second, the jd-r model primarily defines job demands in terms of workload, time pressure, or emotional strain. however, in gvts, low identification can act as a demand, increasing psychological strain and disengagement, even when traditional demands are low (fiol & o’connor, 2005; ramarajan, 2014). finally, while jd-r assumes that engagement is individually experienced, we argue that in gvts, engagement occurs through identification-based interactions, creating engagement gain-and-loss cycles at the team level. this challenges the view that job demands and resources operate only at the individual level, suggesting that engagement is inherently social and contagious in gvts (costa et al., 2017; yun & beehr, 2024). furthermore, emotional contagion theory (hatfield et al., 1993) posits that emotions and engagement spread among team members through social interactions, behavioral mimicry, and shared emotional states. traditionally, this contagion occurs through face-to-face cues such as body language, tone of voice, and facial expressions. however, in virtual environments, engagement contagion operates differently (barsade, 2002; wiesenfeld et al., 1999). thus, our study extends ect in the context of gvts in keyways. first, engagement in gvts spreads digitally and psychologically through a shared identification process. team members who strongly identify with their role, team, or organization influence others to engage more even in the absence of frequent interpersonal interactions. second, negative engagement contagion is amplified in gvts because of weak identification structures. in traditional co-located teams, disengagement can often be corrected through informal, spontaneous interactions (e.g., casual office conversations), whereas in gvts, disengagement can escalate quickly in the absence of immediate interventions (rockmann & pratt, 2015; santillan et al., 2023). third, our study highlights dual identification conflicts (e.g., employees balancing loyalty to a client versus their parent company), which disrupt engagement contagion processes (reade, 2001; raghuram et al., 2019). in traditional teams, engagement flows naturally within clearly defined team structures. however, in gvts, engagement shifts depending on the stronger identification (client versus parent organization). 3 organization context cyient, a multinational enterprise specializing in engineering, manufacturing, data analytics, and network operations, was the focus of this study. the company is headquartered in hyderabad, india, and has a global presence of 58+ locations across 21 countries. it employs more than 15,000 people. its expansive structure facilitates the examination of identification and engagement at multiple levels. gvt members manage multiple roles across teams and organizations, providing unique insights into how they identify various work aspects. this multirole context is particularly relevant given today's flexible, project-based work configurations, which extend beyond traditional single-role studies. multiple gvts enable a comparative analysis of team configurations. team diversity in composition, size, and purpose provides a nuanced understanding of how member identification manifests in gvts. the organization's global reach, involving projects where members work onsite with client organizations, offers insights into the identification challenges between parent and client companies. its involvement across multiple sectors (engineering, manufacturing, australasian journal of information systems shaik et al., 2025, vol 29, research article an identification-based understanding of team engagement in gvts 7 analytics, and network operations) provides opportunities to study gvts across diverse task backgrounds. this research encompassed teams from human resources, aerospace and defense (a&d), rail and transport (r&t), and telecommunications (tc) fields. each sector faces unique identification challenges due to the varying nature of work, stakeholders, and team demographics. the case captures the distinct dynamics between client and parent gvt members, presenting opportunities to study organizational identification conflicts and employee engagement contagion. diverse cultural and geographical contexts, with data from india, the usa, norway, and singapore, illuminate the influence of cross-cultural differences on identification processes. time-zone differences across global gvts provide insights into the temporal factors affecting member identification. this is particularly relevant in today's time of reliance on technology-mediated communication in hybrid and remote work contexts. with its global, complex, and dynamic organizational structure, cyient is an ideal case for understanding the identification process in gvts and its influence on gvt engagement. 4 research methodology due to the lack of available research on team engagement in the gvts domain, we used an ethnographic approach involving participant observation. we adopted an insider-outsider research design, as suggested by bartunek and louis (1996). the lead author was a full-time research associate at cyient who collected data for academic purposes while performing professional duties. the organization's leadership and subsequent corporate managers were aware of the lead author's research, but they did not influence its outcomes. the other authors had no contact with respondents. this allowed the research team to balance personal involvement and professional distance, which is necessary for ethnographic research (anteby, 2013) and facilitated the creation of a theory from micro-interaction analysis. we chose ethnographic approach for this research– because, first, team engagement is a complex, multifaceted, and nuanced phenomenon that involves intricate and sensitive social dynamics, cultural influences, and technology mediation. an ethnographic approach allows for deep contextual understanding and helps create a “thick description” to capture the nuanced and layered meanings and interpretations of social actions in a specific context (geertz, 1973). second, an exploratory approach is warranted, given the limited understanding and research on team engagement in gvts and multilevel identification by gvt members. ethnographic inquiry allows for the emergence of unexpected themes and insights (hammersly & atkinson, 2007). third, this study maps how team engagement in the gvt context is manifested and experienced by gvt members in their everyday lives and work. the focus of ethnographic inquiry on participant observation and field immersion allows researchers to capture lived experiences as they unfold in real time (barley, 1990) in ways that surveys or interviews alone cannot capture (van maanen, 2011). fourth, as gvts spread across diverse cultural and organizational contexts, the emphasis on understanding the phenomenon in natural settings makes it ideal to explore the contextual factors that influence team engagement (fetterman, 2019). finally, the identification processes at multiple levels and team engagement, in general, might be tacit processes often taken for granted by gvt members. the immersive approach facilitated by ethnographic inquiry allows researchers to explicate this knowledge through prolonged observation and participation (spradley, 2016). ethnographic inquiries were implemented using several strategies. australasian journal of information systems shaik et al., 2025, vol 29, research article an identification-based understanding of team engagement in gvts 8 4.1 data collection during the research period of 16 months, the lead author collected data from five teams in india and connecticut, usa. in the hr department, the lead author was part of the employee engagement team and hr team of the rail and transport business units. the role involved assisting corporate hr (chro and hr, india head) by providing academic insights into their issues of concern. they engaged in hr-related projects across three teams in india and two delivery teams (aerospace and defense business unit – a&d bu) in connecticut. confidentiality agreements prevented the lead author from fully immersing himself in the delivery teams. other authors who were not in the field provided balance to maintain an adequate distance. the association with hr teams lasted for 16 months and with delivery teams for 2 months (in india and connecticut). all teams involved members from various locations worldwide working as a single unit. table 1 presents detailed information on the five teams.data were collected through regular hands-on participation in hr teams in different formal and informal settings (meetings with consultants, colleagues, work sessions, daily assignments, and weekly updates), thereby providing access to several meetings with consultants (sturdy et al., 2009). field notes were taken during participant and nonparticipant observations. the lead author maintained a notebook throughout the observation period, resulting in 400 pages of field notes classified chronologically. these notes were shared with the other authors weekly. during the first two months, the field notes considered team members’ everyday activities, modes, ways of functioning, and general life as gvt members. however, when the lead author started socializing and performing several day-today activities, the professional challenges identified as attractive were carefully attended to along with related experiences. teams location of the researcher no. of members type of data hr team (rail & transport bu) hyderabad, manikonda 6 participant observation, notetaking conference calls (webex), informal meeting engagement team hyderabad, manikonda/madhapur hq 5 participant observation, formal meetings, note-taking conference calls (webex), informal meeting hr corporate team hyderabad, madhapur hq 19 participant observation, formal meetings, note-taking conference calls (webex), informal meeting email trails emails exchanged among gvt members while being a participant observer in the above three teams. delivery teams team 1 (telecommunication s bu) hyderabad, uppal sez 7 non-participant observer, interviews (ftf, webex, phone) team 2 (rail and transport bu) hyderabad, madhapur, hq 9 non-participant observer, interviews (ftf, webex, phone) team 3 (aerospace and defense bu) connecticut, us 10 non-participant observer, interviews (ftf, webex, phone) interviews employees from various business units 24 interviews (ftf, webex, phone) table 1: data collection australasian journal of information systems shaik et al., 2025, vol 29, research article an identification-based understanding of team engagement in gvts 9 as a full-time practitioner from aug 2019 to nov 2020, the lead author accessed multiple informal conversations with informants, which were often open-ended and unstructured, along with formal and informal interviews. most of these conversations were friendly for the most part, combined with descriptive and, at times, structural and contrasting questions (spradley, 1979). approximately 80 colleagues covering multiple levels – managers, partners, and cxos – participated in these conversations. the documents crafted by the gvt members during their day-to-day activities (emails, reports, powerpoint presentations, and word files) were also sources of first-hand data collected throughout the projects. they were anonymized and discussed with other authors. 4.2 data analysis this study employed the gioia methodology (gioia et al., 2013) to systematically analyze qualitative data and develop a theoretical model of identification-based engagement contagion in gvts. we fleshed any valuable theme by consistently summarizing the units of field notes in which we could capture a working definition that reflected our informants’ experiences (miles & huberman, 1994). this iterative, reflective process that involved multiple rounds of coding and analysis allowed us to obtain insights from the data directly, rather than fitting the data into preconceived themes. we familiarized ourselves with the data through immersion, in which all researchers read the transcripts, field notes, and related documents multiple times to comprehensively understand the content. then, we made initial notes on the potential patterns and exciting aspects of the data. the gvt members referred to two ways of being inclined or interested in an activity: (a) sudden inclination to engage in an activity they were not much interested in initially; and (b) sudden lack of inclination to engage in a particular activity they were initially very interested in, which the lead author also experienced on several occasions. we found that gvt members were aware of this phenomenon, but they did not explicitly mention it. upon investigating the contextual conditions of this phenomenon, we realized that this sudden shift in engagement affected the performance of gvt members. as data collection and analysis continued, we observed that sudden engagement shifts were not observed in gvt members who were individual contributors – that is, in those who did not perform team tasks. this led us to realize that gvt members who were part of team tasks, performed multiple roles, belonged to various teams, and were located in different organizations experienced sudden shifts in inclination and non-inclination while performing a specific task. we then investigated the attributes that contributed to this phenomenon at the role, team, and organizational levels. to gain a better insight into the complexities of this phenomenon, we shifted our focus to the literature on gvt member identification with their roles, the teams to which they belong, and their organizations. we then performed a more systematic axial and systematic coding of data (corbin & strauss, 1990). although the lead author collected data from five different teams, our aim here was to understand the common themes that emerged across all teams. the lead author interviewed 24 gvt members from other business units (telecommunications and aerospace & defense). these units were selected based on the accessibility provided by the chro. our main objective was to assess the transferability of our findings to gvts in different industries with varying work styles (schein, 1969). these interviews proved the reliability of our results and helped us identify the limitations of the gvt member identification and engagement association. australasian journal of information systems shaik et al., 2025, vol 29, research article an identification-based understanding of team engagement in gvts 10 d at a c o ll ec ti o n interviews interviews delivery team 3 east hartford, connecticut us delivery team 2 hyderabad, uppal india delivery team 1 hyderabad manikonda, india hr team (rail and transport bu) hyderabad, manikonda india hr corporate team hyderabad, hq india hr engagement team hyderabad, hq india timeline aug19 sep19 oct19 nov19 dec19 jan20 feb20 mar20 apr20 may20 jun20 jul20 aug20 sep20 oct20 nov-20 d at a r ef le ct io n initial references of engagement reflective process along with observation of team members doing team tasks performing multiple roles, belonging to various teams, and/or located in multiple organizational sites continuous reflection with fellow researchers table 2: data collection and reflection timeline the research team conducted systematic initial coding using in-vivo coding, where appropriate, to preserve the participants’ voices. for instance, the quote "i sometimes forget that they are not in cyient and are located on-site" was initially coded as "awareness of virtual team members" and "challenges of virtual team collaboration." these first-order codes were data-driven and captured the raw participant expressions. as the analysis progressed, related codes were grouped into broader themes, such as consolidating "role investment," "role enthusiasm," and "role detachment" under "role identification." the second-order coding phase further refined these themes, ensuring conceptual rigor through an iterative review. for example, "organizational attachment or association" was revised to "organizational identification" to better capture the nuances of how gvt members navigated identification between parent and client organizations. the research team triangulated the interpretations with multiple researchers independently coding the data before comparing insights and reaching a consensus to ensure reliability and validity. to maintain methodological transparency, the team documented coding decisions, theme development, and analytical memos on the audit trails. finally, these refined categories were synthesized into aggregate dimensions to form a higher-order theoretical framework explaining how engagement contagion operates within gvts. a representative coding structure is presented in figure 1, which illustrates the systematic coding process and theme development. gioia’s methodology bridged qualitative richness with theoretical abstraction, ensuring that the emergent australasian journal of information systems shaik et al., 2025, vol 29, research article an identification-based understanding of team engagement in gvts 11 framework was empirically grounded, methodologically rigorous, and theoretically robust (cariello et al., 2024; wessel et al., 2019). we recognize the importance of accurately documenting our ethnographic approach (van maanen, 2006) to ensure validity. our condensed account of events over 16 months was seen in the eyes of the gvt members. we include personal accounts referred to as "field notes," informal conversations with cyient's gvt members ("interview, hr" or "interview, a&d"), and interviews with individuals from different business units (interview, comms, and interview r&t). 5 findings our analysis found that team members' identification with their roles, teams, and organizations is crucial in employee engagement in gvts. we present our findings in a structured manner in four categories that capture the dynamics of identification-based team engagement in gvts: (a) configurations of virtual teamwork, including role, team, and organizational identification; (b) scenarios of gvt member identification; (c) the relationship among role, team, and organizational identification; and (d) identification-based engagement contagion in gvts. 5.1 forms of identification in gvts in virtual configuration, gvt members (a) simultaneously performed multiple roles within a team, (b) simultaneously had multiple memberships in different teams, and (c) were at client or company locations in different organizations. they identified themselves with their roles, teams, and organizations in which they were located. 5.1.1 role identification the extent to which gvt members personally identify their roles based on their preferences and interests is associated with their role identification. this personal identification guided their investments in that role and, as a result, their performance. conversely, less identification with their roles meant that they were stuck with the formal scripts of their jobs. the extent of role identification was evident in three ways (figure 1). a. personal investment in the role gvt members who identified more with their roles invested more time and effort, beyond what is formally required. for instance, when a manager describes his role toward his teammates as a "cheerleader" or "figurehead," he is investing more than required and expected. as mentioned, “i step in—not because it’s in my job description, but because i can’t imagine doing it any other way. when you identify so closely with what you do, you don’t just complete tasks; you create an impact that fuels the whole team’s momentum.” (interview, a&d) this finding is aligned with ashforth et al.’s (2016) findings on role identification, which reports that individuals with higher identification with their roles are more likely to engage in extra-role behaviors, thus demonstrating higher levels of commitment. b. role enthusiasm gvt members had to balance their personal goals and preferences with the external requirements for their assigned roles. it was essential to determine how well they fit in with the given roles and how well the role fit them. the more enthusiastic members were in their australasian journal of information systems shaik et al., 2025, vol 29, research article an identification-based understanding of team engagement in gvts 12 assigned roles, the more they reflected on their level of engagement. these members initiated additional activities to benefit their teams, such as organizing sessions and sharing knowledge. a manager in charge of the gvt mentioned that he organized on-site sessions for gvt members to share their experience of working on real-time engines with off-site team members. “[these] sessions keep the team engaged and invested, making them feel connected to the bigger picture beyond just the tasks at hand.” “i still remember when one of our engineers, vikram, returned from a project in canada. he walked into the session, jet-lagged but excited, and started sharing how different it was working in freezing temperatures at the airfield compared to the scorching heat of hyderabad. he pulled out a photo of himself wrapped in four layers of jackets, standing next to a snow-covered aircraft engine, and the whole team burst out laughing. but then he got serious—he spoke about how the canadian team approached problem-solving differently, how they handled unexpected failures, and what we could learn from them. by the end of the session, our off-site members weren’t just passive listeners; they were brainstorming how to implement some of those practices here. that’s when i knew—this wasn’t just about knowledge transfer. it was about making them feel a part of something bigger, something that extended beyond their desks and time zones.” (interview, a&d) c. role detachment our findings indicated lower role identification when gvt members adhered to their scripted ways of performing or reduced their investments in their roles. it also involved reduced awareness and acknowledgment of their roles and team members. with reduced role identification, gvt members invested less in their roles, and were thus less engaged. for example, an hr team member mentioned: “honestly, there are times when i forget that some of our team members aren’t based here in bengaluru but are stationed at client sites, like in frankfurt or são paulo. we send out important communication emails assuming everyone receives them, but it’s only when i visit the site that i realize they’ve missed critical updates—things as basic as policy changes or workflow adjustments… and when you don’t feel like a part of just about sending updates; it’s about making sure people actually feel included and recognized, no matter where they are.” (interview, comms) thus, in our findings, high role identification among gvt members was expressed during their role performance with higher dedication, absorption, and vigor (high engagement). factors that made members identify and engage more with their roles include personal goals and preferences, external role requirements, perceived fit between the individual and the role, and opportunities for role expansion and creativity. reduced role identification was reflected in “acting out” the role and sometimes donning masks in their roles reflecting lower dedication, absorption, and vigor. australasian journal of information systems shaik et al., 2025, vol 29, research article an identification-based understanding of team engagement in gvts 13 figure 1: representative coding structure illustrating the systematic coding process and theme developmen australasian journal of information systems shaik et al., 2025, vol 29, research article an identification-based understanding of team engagement in gvts 14 figure 2: representative coding structure illustrating the systematic coding process and theme australasian journal of information systems shaik et al., 2025, vol 29, research article an identification-based understanding of team engagement in gvts 14 5.1.2 team identification team identification concerned how much gvt members could express and identify themselves as team members, and this was driven by three aspects: a. shared goals and vision higher levels of team identification were characterized by a strong sense of working toward common goals. gvt members strongly identified with their teams by coordinating efforts across time zones and cultural boundaries through dedication and energy, which indicates their engagement with the team. for example, a senior director stated: “coordinating across time zones became second nature to us. someone had to compromise—some of us were up late at night in hyderabad, while others had to wake up early in seattle or adjust their schedules in frankfurt. for some, it was right in the middle of their workday, and they had to carve out time. but we made it work because we shared the same goal. we weren’t just a group of individuals assigned to a project; we became a team.” (interview, a&d) b. personal connections personal connections facilitated robust team identification among team members. gvt managers or leaders who put in effort to know and understand their team members personally fostered cohesion and high team-level engagement. this created a synergistic environment in which gvt members could freely express themselves. the frequency and quality of team interactions, leadership behaviors, social bonding, cultural awareness, and sensitivity facilitate team identification. as mentioned in an interview: “i still remember when one of our teammates, raj, in singapore, had a terrible flu. he casually mentioned this on a late night call, and the next day, someone from our dallas team ordered soup to be delivered. raj was so surprised—he said, ‘i have never even met you in person, and you just did that for me?’ that moment stuck me because it showed how much we cared about each other, beyond just working. these minor factors are important. the better we know each other, the more we trust and support one another, which makes a huge difference in how we work together. this turns a group of people into a real team regardless of where we are in the world.” (interviews, rail, and transportation). c. informal communication informal communication fostered collaboration and mutual understanding among virtual global teams. beyond structured meetings and formal emails, unplanned and flexible interactions helped team members navigate time-zone challenges and maintain a sense of cohesion. as highlighted in one interview: "working across hyderabad, germany, and the u.s. means someone is always adjusting their schedule. there are nights when i've taken customer calls at 1 am, and mornings when the team in india has started early just so we can all align. it's not easy, but over time, we've learned to manage. we know that if one of us is making a sacrifice today, someone else will do the same tomorrow. it's this kind of coordination that keeps things moving—no one is working in isolation, and that understanding makes all the difference. we might be sitting in different countries, but we operate as one." (interview, it services) this quote highlights how informal, adaptive communication—whether through late-night calls, early-morning check-ins, or quick messages—creates a rhythm of collaboration. the australasian journal of information systems shaik et al., 2025, vol 29, research article an identification-based understanding of team engagement in gvts 15 willingness to adjust schedules and accommodate one another fosters reciprocity and trust. these informal touchpoints ensure that work progresses smoothly, reinforcing the perception that, despite geographical separation, the team functions as a unified whole. 5.1.3 organizational identification organizational identification concerns how gvt members identify with the organization (parent and/or client) to which they belong. organizational identification was complex, especially for gvts whose team members were stationed at client sites. the degree of identification with parent and client organizations significantly impacted knowledge sharing and collaboration, and vice versa. the subthemes for organizational identification are as follows: a. dual organizational loyalty the degree of organizational identification displayed by gvt members differed based on whether they were stationed at client sites or parent organizations. individuals who spent more time at the client site identified more with the client organization. shared by a gvt member: “working at a customer site like pratt & whitney in east hartford or airbus in toulouse changes things. over time, you start seeing yourself more as part of their team than cyient’s. meetings with cyient feel like a formality, and before you know it, the connection starts fading…it wasn’t intentional, but the longer you stay, the more your identification shifts.” (interview, a&d) this can lead employees to lose their sense of identification with their parent organization. as mentioned by another member: “that’s the challenge—when employees feel isolated from their parent company, they start identifying more with the customer. if cyient doesn’t actively engage them, we don’t just risk losing them to disengagement—we risk losing them to the very clients they were sent to support.” (interview, comms) b. knowledge-sharing behaviors stronger identification was associated with knowledge-sharing and collaboration with members across client and parent organizations. for example, a gvt member stated: “what you’re touching on here is that when employees don’t have a strong connection with their parent organization, they hesitate to share knowledge. if they don’t have a good relationship with leadership at cyient, they don’t know what to expect from them. so, when they develop a unique insight from working at a client site like pratt & whitney in east hartford, they might hold back instead of sharing it with cyient. i’ve seen this firsthand—there was a team in toulouse that developed a streamlined process for handling design reviews while working on an airbus project. it saved so much time, but instead of bringing it back to cyient, they only discussed it within the airbus team. when i asked why they hadn’t shared it, someone just shrugged and said, ‘i don’t know if leadership at cyient would even care.’ that’s the problem— without trust and identification with the parent company, valuable knowledge gets stuck at the client site, and the organization as a whole misses out on critical improvements.” (interview, comms) australasian journal of information systems shaik et al., 2025, vol 29, research article an identification-based understanding of team engagement in gvts 16 c. employee workplace support the strength of organizational identification was also associated with perceived organizational support, including recognition, career development opportunities, and worklife balance. for example, one gvt member described the experience of a teammate: “she was one of the most proactive members of our team, but over time, she became quieter. at first, we thought she was just busy, but then she confided that she felt like she was ‘out of sight, out of mind’—she wasn’t sure if leadership at cyient even noticed her work anymore. she had no visibility into career growth opportunities and felt disconnected from the company’s internal initiatives. it wasn’t about the workload; it was about feeling valued. that’s where workplace support makes all the difference. if employees feel recognized and know that their contributions matter, they stay engaged. but if that support is missing, they gradually detach—not just from their role, but from cyient itself.” (interview, rail and transport) thus, identification processes in gvts are complex and multifaceted, and involve role, team, and organizational identification. this emphasizes the significance of identification as a multilevel, nuanced, and dynamic construct that emerges based on roles and membership of gvt members in multiple teams and organizations. 5.1.4 scenarios of role, team, and organizational identification in gvts by observing the role and team identification of members within the gvt, four distinct scenarios emerged based on the number of roles, teams, and organizations to which each member belonged: figure 2: scenario 1 of gvt member identification (source: authors) (ri=role identification; ti= team identification; oi=organizational identification) a. scenario 1: the solo virtuoso – single role, single team, single organization during our initial examination, we noted that a gvt member could perform various duties and assume different roles. their identification was based on the specific roles they performed. nevertheless, there may be instances in which such individuals did not express interest in being part of a team or engaging in team-oriented endeavors. the roles that they accept enable them to reduce their reliance on team members and managers. these personnel are recognized as "individual contributors" with exceptional proficiency in their fields. for example, the director of operations said: australasian journal of information systems shaik et al., 2025, vol 29, research article an identification-based understanding of team engagement in gvts 17 one employee who is not so young works in my team. he does his job and does it very efficiently. however, he does not respond to team initiatives or bonding exercises. you would not believe that even i have not met him until now. it is just like we pay him for the work he does for us. however, he is an expert in his field. it would be good to other team members to get to know him. they could learn so much from him (sic). in such cases, gvt members, although part of the team, did not identify with the team. they identify with their roles and perform highly efficiently. the critical characteristics observed in this scenario are high role identification and low team identification. b. scenario 2: the multitasking virtualist – multiple roles, single team, single organization the second scenario corresponds to when a gvt member carries out multiple roles and, thereby, multiple tasks in each role, identifies t with roles based on interests, and belongs to a single team (reporting to a single manager), and hence, a single organization. for example, a member of a gvt team fulfills dual responsibilities. specifically, this individual has served as both a talent acquisition manager and an assistant to their team manager for data analysis and error handling in organizational software. despite these dual roles, individuals report to a single manager. similarly, in connecticut location, a business development executive takes on responsibilities related to the hiring process for a role that is not particularly relevant to her position. figure 3: scenario 2 of gvt member identification (source: authors) (ri=role identification; ti= team identification; oi=organizational identification) similar to the previous individual, she reported to the same manager for both roles. the key characteristics observed in this scenario were varying levels of role identification, moderateto-high team identification, and high organizational identification. c. scenario 3: the virtual juggler – multiple roles, multiple teams, single organization in the third scenario, a gvt member had multiple reporting managers with distinct roles, unlike scenario 2. however, each member was tagged with only one reporting manager for performance evaluation and online system tagging ( only one reporting manager is accepted australasian journal of information systems shaik et al., 2025, vol 29, research article an identification-based understanding of team engagement in gvts 18 per employee). for example, an hrbp (semiconductors) was reporting directly to the hrasia head but was also involved in several l&d initiatives where he reported to the l&d head. in these circumstances, the gvt members were part of multiple teams but a singular organization. this scenario includes varying levels of role identification across roles, team identification across teams, and organizational identification. figure 4: scenario 3 of gvt member identification (source: author) (ri=role identification; ti= team identification; oi=organizational identification) d. scenario 4: the boundary spanner – multiple roles, multiple teams, multiple organizations figure 5: scenario 4 of gvt member identification (source: author) (ri=role identification; ti= team identification; oi=organizational identification) in the fourth scenario, certain gvt members were stationed at client locations and collaborated with their parent company teams. these members were accountable to their australasian journal of information systems shaik et al., 2025, vol 29, research article an identification-based understanding of team engagement in gvts 19 client manager and parent company managers for their separate duties. throughout this scenario, the gvt members aligned themselves more closely with either their parent or client organizations. this scenario includes varying levels of role identification across different roles, varying levels of team identification across various teams, and dual organizational identification (parent and client). for example, gvt members working on-site with client organizations can develop greater identification with their client organizations than with their parent organizations. the interplay among role, team, and organizational identification reveals their interconnected and mutually influential nature, with individual differences significantly affecting how members relate to these various aspects of their work environment. the different scenarios with varying levels of identification are summarized in table 3. s. no. scenario description role identification team identification organizational identification 1 the solo virtuoso single role, single team, single organization high – focused on specific role expertise low – limited engagement with team initiatives moderate – primarily identifies with the parent organization 2 the multitasking virtualist multiple roles, single team, single organization varied – high for preferred roles, lower for others moderate – engaged with a single team high – strong connection to parent organization 3 the virtual juggler multiple roles, multiple teams, single organization varied – based on interest in each role varied – different levels for each team high – strong identification with parent organization 4 the boundary spanner multiple roles, multiple teams, multiple organizations varied – based on role preferences complex – varies between client and parent teams dual – identifies with both client and parent organizations table 3: types of scenarios and characteristics of identifications 5.2 relationship among role, team, and organizational identification: identification-based engagement contagion the way team members in gvts identified themselves (with their roles, teams, and organizations) potentially influenced their own engagement and overall team engagement. members with higher team orientation manifested higher team identification, which was associated with higher organizational identification. for instance, the senior director mentioned how he felt more connected to his role and team when his manager wrote to him, recognizing his work. this recognition helped him to identify more strongly with his team and roles. “the work is challenging, but it’s the relationships i’ve built—whether with my teammates in bengaluru or those stationed in connecticut—that make me want to stay just as much as the job itself. when you know you’re valued, it strengthens your connection not just to the team but to the entire organization.” (interview, comms, india) these findings indicate that higher roles and team identification are linked to higher engagement at individual and team levels. for example, an associate director in the architecture and design unit highlighted the importance of the manager's role identification in ensuring that the team works cohesively towards common goals. she noted that team australasian journal of information systems shaik et al., 2025, vol 29, research article an identification-based understanding of team engagement in gvts 20 members' ability to work together well, despite never having met in person, is a testament to their strong team identification and engagement. “earlier in my role as a project lead for a repair engineering team, i was responsible for coordinating with repair stations in norway and singapore. i’ve never met those team members in person, yet we work seamlessly together. the reason? i strongly identified with my role as a manager, and that commitment set the tone for the entire team. i didn’t just see myself as someone assigning tasks—i saw my role as creating a sense of unity, making sure each team member felt included, valued, and motivated. i consistently acknowledged contributions, celebrated small wins, and ensured that our meetings were more than just status updates. because i was engaged and proactive in fostering collaboration, my team members also began to see themselves as part of something bigger. they weren’t just completing tasks remotely; they were part of a collective effort, aligned towards a shared goal. i’ve realized that when managers are deeply committed to their role, it directly influences how the team identifies with their work. my identification as a leader created a ripple effect—because i was engaged, they became engaged. that’s why, despite never having met in person, we still function as a tight-knit team.” (interview, a&d, connecticut) our findings provide evidence for a link between identification and engagement contagion. a sr. project manager in connecticut emphasized the role identification of team members affecting team identification and organizational identification, and thus team engagement, which then further boosted identification. “once we built that connection, something interesting happened—the people who had been working in isolation started seeing themselves not just as employees in a remote office, but as part of a larger team. their identification with their roles grew stronger, and that had a cascading effect. when they saw that their contributions mattered, they engaged more with the team. and as they became more engaged with the team, they naturally started identifying more with the organization as a whole. one of the key factors was having someone on-site who was as passionate about the business and the company as we were. that energy was contagious. once those role anchors were set in place, collaboration became effortless—whether electronically, over the phone, or during in-person meetings. the real takeaway? when team members strongly identify with their roles, they foster a team culture that strengthens engagement, creating a lasting bond with the organization.” (interview, a&d, connecticut) we also found that even though there is higher role identification, reduced team identification might affect organizational identification and eventually reduce the role identification of gvt members and their engagement. a few examples include: “i can’t speak for everyone, but personally, if i didn’t have a strong teammate and a manager who was involved, i think over time, i’d start feeling disconnected. right now, i enjoy my work, but if i didn’t have a team to collaborate with, it would be harder to stay engaged. even if i’m good at what i do, feeling isolated from the larger team would make me question my place in the company. and once that happens, it’s only a matter of time before i start thinking, ‘maybe i should find a workplace where i feel more connected, not just to my tasks but to the company itself.’ just knowing my work matters isn’t enough—without a team to support and engage with, it’s easy to lose motivation.” (interview, rail & transport, madhapur) australasian journal of information systems shaik et al., 2025, vol 29, research article an identification-based understanding of team engagement in gvts 21 when gvt members worked onsite in client organizations, they developed a stronger connection with the client organization than with their parent organization. this dynamic resulted in reduced interaction and knowledge-sharing with the parent team, ultimately decreasing their parent organizational identification and team engagement levels, leading to higher attrition rates within the organization. while commenting on the same, a business development manager said: “when someone is stationed at a client site like pratt & whitney in east hartford or airbus in toulouse for an extended period, they naturally start adopting the customer’s ways of working, priorities, and even their culture. over time, the connection to cyient weakens. i’ve seen employees who, after a few years on-site, barely engage with their cyient teams anymore. they stop attending internal meetings, don’t keep up with company initiatives, and eventually, when faced with a career decision, their first thought is, ‘why not just join the client company instead?’ that’s where we lose people—not just to disengagement but to actual attrition. this has been a long-standing issue, and unless we actively reinforce their connection to cyient, it will continue to be a challenge.” (field notes, connecticut) thus, we conclude that the engagement level of a gvt member is influenced by the role, team, and organizational identifications that interact to create a contagious effect on engagement within gvts. we label this phenomenon the “identification-based engagement contagion.” (see appendix-table 1 for representative quotes). it indicates that a gvt's collective engagement differs from the sum of its member engagements. positive contagion results in positive spirals of team-level engagement, whereas negative contagion leads to negative spirals. this dynamic contributes to our understanding of team engagement in the context of gvts. for instance, when the director of operations states that their work is delayed because of a fellow team member, or the hrbp finds it challenging to interpret the tone of a conversation, or the sr. the project manager mentions that email communication is difficult and phone calls impossible, it can be inferred that the success of a role and team depends on the relative investments of the gvt members. due to the structure of gvts, team members have limited face-to-face interactions with each other and their managers, particularly in multiple time zones. the longer gvt members are separated without communicating with their managers, the more disconnected they become from their roles and team performance. moreover, they lose their sense of identification with their roles and teams, which affects their investment in their roles and team members and ultimately leads to reduced team engagement. “the first thing that influences my engagement is how connected i feel to my team. but it doesn’t stop there—there’s also an organizational aspect that impacts the team itself. it’s not just about me personally, but about how the team dynamic affects my connection to cyient as a whole. for example, if my teammates in hyderabad and east hartford are engaged, motivated, and collaborative, that energy spreads—it makes me more invested in my work. but if the team struggles with communication gaps, unclear expectations, or delays, it creates a ripple effect. over time, even the most motivated team members start feeling disconnected, not just from the team but also from cyient. i’ve seen this happen when policies or processes don’t align well with how we work across different locations. if something like performance evaluations or internal promotions feels disconnected from our reality as a global team, it impacts how much effort we put into both our roles and our collaboration. australasian journal of information systems shaik et al., 2025, vol 29, research article an identification-based understanding of team engagement in gvts 22 engagement in gvts isn’t just about individual effort—it’s contagious. when the team is engaged, everyone is engaged. when the team is disengaged, even the most committed individuals start pulling back.” (field notes, hr, india) 6 discussion, conceptual framework and propositions through our research, we conceptualize “identification-based engagement contagion” in gvts as members often assume multiple roles across different teams and report to multiple managers across organization boundaries (glikson & erez, 2020; shaik et al., 2020; whillans et al., 2021; zakaria & yusof, 2020). for structural reasons, we highlight that in gvts, members hold multiple roles, belong to multiple teams, and work across multiple organizations, thus leading to different types of identifications. organizations increasingly rely on remote teams that are cross-functional, cross-border, and composed of members with highly developed and distinct areas of expertise to complete tasks (pearsall et al., 2010; sundstrom et al., 2000). in a virtual team configuration, where team members work across different time zones and have a 24-hour working schedule, they play multiple roles. for successful execution of each role, team members need to associate with their roles and have a shared understanding of each other's expertise by developing shared cognitive structures over time (nordback & espinosa, 2019; ilgen et al., 2005; kozlowski & ilgen, 2006). to achieve this, team members need to identify their roles and exhibit purposeful interpersonal interactions to understand the roles and capabilities of their team members (mohammed & dumville, 2001). these behaviors, termed role identification (kozlowski et al., 1999), tend to affect how teams function cohesively, thereby increasing team performance and satisfaction (austin, 2003; edwards et al., 2006; pearsall & ellis, 2006). improved team cohesion and shared capabilities across gvts motivate members to feel a sense of belonging (tyler & blader, 2003), thus creating team identification. in gvts, the members of multiple teams can have varied levels of team identification based on their preferences and shared team behaviors. organizational identification depends on an individual’s attraction to the organization regarding its systems and overall employer brand (edwards, 2005; pratt, 1998; riketta, 2005). however, when gvt members shared team behaviors with their team at the parent’s location, their organizational identification with the parent organization increased. similarly, when team identification was higher with the client team, it impacted the client’s organizational identification. therefore, we propose the following: p1a: in gvts, the members identify with their roles, teams, and organizations distinctively. p1b: in gvts, member role identification, team identification, and organizational identification influence one another. increased identification of gvt members with their roles, teams, or organizations can be considered a resource in terms of the jd-r model (bakker & demerouti, 2007) as it results in improved coordination and communication across team members, along with higher performance. reduced identification may be considered a demand because it negatively affects employee well-being, satisfaction, and overall performance. according to the jd-r, perceived resources and demands can initiate a gain or a loss cycle. a gain cycle (job resources) is initiated when employees perceive and positively influence their jobs, while a loss cycle (job demands) is initiated when employees perceive and negatively influence their australasian journal of information systems shaik et al., 2025, vol 29, research article an identification-based understanding of team engagement in gvts 23 jobs (bakker et al., 2020). our analysis indicates that employees with lower levels of role identification put less effort into that role, thereby contributing less toward team goals and having fewer team identifications. similarly, when the team is less cohesive, with reduced supervisor and peer support, members have less team identification, which affects the efforts they put into roles related to that team and negatively impacts their role identification. when members are part of multiple organizations (client and parent), their affiliation with organizational goals, brand, and loyalty affects their identification with their roles and teams (client and parent). reduced organizational identification with the parent organization affects team identification, which affects their role in the parent organization by reducing information-sharing, bonding with team members, and overall team performance. the adverse effects of reduced identification (role, team, or organization) initiate loss cycles in the gvt. while we did not explicitly capture the productivity of these teams, our participant observations guided us toward evidence of such loss cycles. similarly, we found that gain cycles were initiated when members identified highly with their roles, teams, and organizations, contributing positively to their job roles, leading to gain cycles. hence, we propose the following hypotheses: p2a: an increased role, team, or organizational identification of a gvt member leads to increased member engagement by initiating gain cycles. p2b: a decreased role, team, or organizational identification of a gvt member leads to decreased member engagement by initiating loss cycles. figure 5: role, team, and organizational identification of a gvt member engagement is often “a positive, fulfilling, work-related, shared psychological state, which emerges from shared experiences of members of a work team” (torrente et al., 2012, p. 107). we borrow from the ect to conceptualize the “identification-based-engagement-contagion” mechanism. ect states that different people can share and express their emotions, allowing them to spread from one individual to another (barsade, 2002; hatfield et al., 2009; hatfield et al., 2014). when workers identify with another person on the same team based on gender, race, nationality, or job seniority, they may take that person as an emotional referent (bakker and demerouti, 2007; bakker, 2009). similarly, when group members identify with their interacting groups, they form a shared conception of each other (bakker & sanz-vergel, 2013; henry et al., 1999). furthermore, when employees share many job-related experiences and job aspects, such as motives to work or any other common non-work characteristics, they enact a australasian journal of information systems shaik et al., 2025, vol 29, research article an identification-based understanding of team engagement in gvts 24 categorization process and exhibit a bias in favor of similar members (kooij et al., 2011; posthuma & campion, 2009; tsui & o’reilly, 1989). thus, these expressions of behavior and emotions within the team take the form of similar engagement, spreading, and sharing (bakker sanz-vergel., 2013). figure 6: identification-based engagement contagion in a gvt member engagement in gvts is affected by their identification with their roles, teams, and organizations, leading to either gain or loss cycles of member engagement. we found that employees’ increased gain cycles can have behavioral consequences for investing extra energy in jobs with higher persistence and dedication. this is transferable to the job setting and, therefore, shared with other team members, creating a positive contagion effect (salanova et al., 2005; torrente et al., 2013). similarly, contagion can be initiated by loss cycles, causing a lack of energy investment and reduced team cohesion, which leads to a negative contagion effect. this could have implications for promoting shared engagement within a team. the novelty of our study lies in the context of gvts and the understanding that engagement contagion is identification-based (role, team, or/and organization). thus, we propose: p3a. gain cycles across gvt members initiates a positive identification-based engagement contagion in a gvt. australasian journal of information systems shaik et al., 2025, vol 29, research article an identification-based understanding of team engagement in gvts 25 p3b. loss cycles across gvt members initiates a negative identification-based engagement contagion in a gvt. 7 contributions and conclusion there is extensive adoption of gvts in mnes; thus, making gvt members perform at their highest potential is essential for team and mne success. the challenges of managing a spatially, linguistically, and culturally diverse global team are well documented. however, the solutions require equal attention from the researchers. considering this, our study suggests that gvt member engagement is a positive force multiplier for creating team-level engagement through identification-based contagion. as engagement is linked to several positive outcomes at the individual, team, and organizational levels, contagion through gain cycles could be a source of team and mne success. furthermore, the gain and loss cycles theorized in the framework bring team dynamics (instead of team statics) to the foreground (cronin et al., 2011) to emphasize the need to capture global virtual team engagement as a dynamic and emergent construct (kozlowski & klein, 2000). this phenomenon was delineated because of the lead author’s deep ethnographic engagement with the focal organization. the propositions were drawn as a result of careful examination by the lead author as a member of the organization across multiple geographical locations, nonparticipant and participant observations, formal and informal conversations, meeting artifacts, interactions, and interviews. our findings contribute to the literature on is, identification, and engagement in technologymediated gvts, and offer practical implications for mnes using gvts to organize work. engagement literature scholars have theorized that individual engagement in gvts is relatively unexplored (gilson et al. 2015). given the complexities and changes in context in gvts, viewing gvt engagement through the lens of identification literature adds a new dimension to the existing literature on gvt engagement. we contribute to the literature on identification through our findings on the simultaneous multiple roles played by gvt members and their identification with each of these roles, which is a novel dimension. scholars have worked on “work identification” (lai et al., 2013), conceptualized it as "the aspects of identity and self-definition that are tied to participation in activities of work (i.e., job) or membership in work-related groups, organizations, occupations, or professions” (dutton et al., 2010, p. 266). however, we found that work identification is an overarching conceptualization of an employee's entire work as part of their job. we broke this into three identifications to highlight the nuances of gvt functioning. we felt the need for this australasian journal of information systems shaik et al., 2025, vol 29, research article an identification-based understanding of team engagement in gvts 26 figure 7: process flow model of identification-based gvt engagement contagion australasian journal of information systems shaik et al., 2025, vol 29, research article an identification-based understanding of team engagement in gvts 27 distinction for gvt members, as they perform multiple roles and belong to multiple teams and organizations. here, we also address the call by atewologun et al. (2017) for further exploration of this work, focusing on identification in various contexts. we contribute to the literature on team identification using an interpretative lens focusing on dynamics. 8 theoretical contributions this study contributes to the theoretical landscape of team engagement in gvts by challenging and refining the key assumptions of the jd-r model and ect. although these theories have been instrumental in explaining engagement processes in traditional work settings, they require modifications to account for engagement dynamics in virtual and hybrid team environments. this study provides empirical evidence that engagement in gvts is not solely a function of individual job resources or emotional contagion through direct interactions but rather an identification-driven process that shapes engagement contagion at the team level (ashforth & mael, 1989; van dick et al., 2004). the jd-r model (bakker & demerouti, 2007) posits that engagement occurs when job resources (e.g., autonomy, supervisor support, and task clarity) outweigh job demands (e.g., workload, time pressure, and emotional strain). although this framework effectively explains individual engagement dynamics, this study challenges its applicability to gvts in three ways. first, the jd-r model primarily focuses on tangible job resources such as access to support systems and clear task expectations. however, we find that social identification (role, team, and organizational) is an intangible but critical job resource for virtual teams (ashforth & mael, 1989; van dick et al., 2004). employees who strongly identify with their teams and organizations remain highly engaged even when traditional job resources are limited, whereas those with weak identification experience disengagement despite having access to structural resources. this study extends the jd-r framework by positioning identification as a psychological job resource essential for engagement in distributed work environments. for example, employees working in a fully remote environment may lack access to the same physical resources and leadership support as their on-site colleagues but remain engaged if they feel deeply connected to their team’s mission, values, and goals (ashforth & mael, 1989; van dick et al., 2004). in contrast, employees with weak identification, even if they have access to job autonomy and digital support systems, are more prone to disengagement, suggesting that psychological identification is a stabilizing factor in engagement. thus, future models of engagement should integrate identification as a core job resource that can offset some of the structural challenges associated with remote and hybrid work. second, the jd-r model assumes that job demands such as workload and time pressure are primarily structural. however, we argue that job demands can also be psychological and identity-driven in gvts. employees in client-facing roles, for example, experience dual identification conflicts (e.g., balancing loyalty between a client and their parent organization), which creates an engagement strain not accounted for in the traditional jd-r model (reade, 2001; ramarajan, 2014). for instance, employees working for a multinational consulting firm may be assigned to a client project, requiring them to balance loyalty to the parent organization and client expectations. this dual identification conflict creates an engagement strain that is not typically accounted for in the existing jd-r models. employees experiencing identity misalignment often feel disengaged, emotionally detached, and unable to fully commit to their roles, even if their workload is manageable. therefore, this study suggests that identity strain australasian journal of information systems shaik et al., 2025, vol 29, research article an identification-based understanding of team engagement in gvts 28 should be formally recognized as a job demand in engagement models, particularly for remote and hybrid teams where competing identifications are common. this study highlights identity misalignment as a novel job demand in virtual teams, which can increase disengagement, even when the workload remains stable. third, the jd-r model treats engagement as an individual-level outcome resulting from personal experiences with job demands and resources. however, our findings show that engagement is a socially contagious process in gvts, shaped by team-level identification interactions (costa et al., 2017; yun & beehr, 2024). engagement is a team-level phenomenon that spreads through shared identification mechanisms. therefore, we propose an extension of the jd-r model that incorporates identification-based engagement contagion as a key driver of team-level engagement dynamics in virtual settings. the ect (hatfield et al., 1993) suggests that emotions and engagement are spread through social interactions, nonverbal cues, and behavioral mimicry. in traditional work settings, contagion occurs through face-to-face interactions in which employees unconsciously mirror each other’s emotions. however, in virtual environments, such physical cues are absent or highly mediated through digital communication. our study redefines engagement contagion in gvts, emphasizing that shared identification, rather than emotional mimicry, is the primary mechanism for engagement spread (barsade, 2002). first, in virtual teams, engagement contagion occurs through identification alignment rather than through direct emotional exchange. in co-located teams, emotional contagion is facilitated by in-person interactions, body language, and spontaneous communication (barsade, 2002). however, in gvts, team members influence each other’s engagement levels through their shared sense of identity with the team or organization. those who identify strongly with a team are more likely to positively influence others' engagement levels, even in asynchronous work settings. for example, if a highly engaged team member regularly shares motivational messages, best practices, and proactive problem-solving approaches in a virtual setting, other team members who feel a sense of belonging to the same group are more likely to reciprocate it, even in the absence of direct emotional interactions (costa et al., 2017; yun & beehr, 2024). however, when identification weakens (e.g., due to poor leadership, unclear team goals, or frequent turnover), disengagement spreads more quickly and pervasively, as there are fewer informal social mechanisms to re-engage employees. this study extends ect by incorporating identificationbased engagement contagion as a digital mechanism in gvts. second, our findings reveal that negative engagement contagion is amplified in virtual teams due to a lack of informal social interactions. in traditional teams, disengaged employees can be re-engaged through spontaneous conversations, managerial interventions, or peer support (rockmann & pratt, 2015). however, disengagement can spread rapidly in gvts because there are fewer opportunities for real-time intervention. this study highlights the fact that identification weakens or strengthens engagement contagion in digital work environments, suggesting that interventions should focus on enhancing shared identity to counteract negative contagion effects. third, emotional contagion does not follow clear hierarchical structures in virtual teams, where employees often navigate multiple overlapping identities (e.g., project-based teams, regional teams, and organizational affiliations). unlike traditional work environments where emotional contagion typically flows downward from leadership, engagement contagion follows identification strength rather than a hierarchical structure (fiol australasian journal of information systems shaik et al., 2025, vol 29, research article an identification-based understanding of team engagement in gvts 29 & o’connor, 2005; reade, 2001). employees in highly engaged subgroups may sustain engagement even when organizational-level engagement is low and vice versa. 8.1 managerial implications a survey by the society for hrm indicates that firms use virtual teams to reduce costs and improve productivity. however, as evren essen, the manager of shrm’s survey center, stated: “when virtual teams work, their productivity can be impressive, … but getting them to function can be a challenge” (the association for talent development, 2012). the survey also indicated that building team relationships was a significant reason for the failure of virtual teams. therefore, team-level engagement in the gvts is crucial. mnes working with gvts should help team members identify their roles, teams, and organizations. role identification can be achieved by providing better role clarity, whereas team identification can be improved through productive team interactions. tangible and intangible benefits accrued by employees can help develop organizational identification. positive identification with roles, teams, and organizations fosters team engagement through a virtuous cycle, leading to positive outcomes linked to gvt engagement. organizations can make policy-level changes and have ground rules for the hr department, gvt managers, and corporate leaders to approach employees in gvts differently from their existing approaches. this enables identification among gvt members to affect team engagement. digital technologies such as artificial intelligence (ai), augmented reality (ar), and virtual reality (vr) transform the work configurations and identification processes in gvts. enhanced communication platforms such as slack, microsoft teams, and zoom facilitate the seamless integration of various features, potentially increasing roles, teams, and organizational identification through improved visibility and informal communication channels (bapista et al., 2020; rockmann & pratt, 2015). ar and vr technologies, through platforms such as spatial or altspacevr, create immersive team experiences and virtual office environments and strengthen connections among gvt members. these technologies enable role simulation and project-based work, enhancing role identification through virtually tangible experiences (choudhury et al., 2021; john et al., 2024). ai-powered tools include personal assistants and chatbots, streamlined onboarding, role clarity, and team coordination. nlp technologies facilitate cross-cultural communication, whereas sentiment-analysis tools monitor team morale. ai-enhanced analytics support informed decision-making, potentially strengthening roles and organizational identification (malhotra & majchrzak, 2014). these technological integrations can accelerate engagement and disengagement contagion, making identification processes more complex. managers must carefully consider how technological interventions affect team engagement in gvts. 8.2 limitations and future research this study has a few limitations. first, the study was conducted in a single multi-national service-focused enterprise headquartered in india with a global presence. while this study explored the nuances of team-level engagement using several data artifacts from multiple geographical sites and teams, exploring the variances in team-level engagement across organizations of different sizes and industries would be interesting. to study team engagement, we delineated four scenarios of team member identification across gvts along with their roles, teams, and organizations. future studies can explore these scenarios to understand team-level concepts, such as cohesion, commitment, and team learning, from australasian journal of information systems shaik et al., 2025, vol 29, research article an identification-based understanding of team engagement in gvts 30 several perspectives, including sense-making, organizational development, learning and development, and information sharing. it is interesting to explore how such constructs manifest in gvts under these configurations. references adamovic, m. (2018). an employee-focused human resource management perspective for the management of global virtual teams. the international journal of human resource management, 29(14), 2159–2187. doi.org/10.1080/09585192.2017.1323227 anteby, m. (2013). manufacturing morals: the values of silence in business school education. university of chicago press. doi.org/10.7208/chicago/9780226092508.001.0001 aon. (2022). global wellbeing survey. https://www.aon.com/global-wellbeing-survey ashforth, b. e., & mael, f. (1989). social identity theory and the organization. academy of management review, 14(1), 20–39. doi.org/10.5465/amr.1989.4278999 ashforth, b. e., harrison, s. h., & corley, k. g. (2008). identification in organizations: an examination of four fundamental questions. journal of management, 34(3), 325–374. doi.org/10.1177/0149206308316059 ashforth, b. e., schinoff, b. s., & rogers, k. m. (2016). “i identify with her,” “i identify with him”: unpacking the dynamics of personal identification in organizations. academy of management review, 41(1), 28–60. doi.org/10.5465/amr.2014.0033 aten, k., & thomas, g. f. (2016). crowdsourcing strategizing: communication technology affordances and the communicative constitution of organizational strategy. international journal of business communication, 53(2), 148–180. doi.org/10.1177/232948841562726 atewologun, d., kutzer, r., doldor, e., anderson, d., & sealy, r. (2017). individual‐level foci of identification at work: a systematic review of the literature. international journal of management reviews, 19(3), 273–295. doi.org/10.1111/ijmr.12148 austin, j. r. (2003). transactive memory in organizational groups: the effects of content, consensus, specialization, and accuracy on group performance. journal of applied psychology, 88(5), 866–878. doi.org/10.1037/0021-9010.88.5.866 bakker, a. b. (2009). the crossover of burnout and its relation to partner health. stress and health: journal of the international society for the investigation of stress, 25(4), 343–353. doi.org/10.1002/smi.1278 bakker, a. b. (2022). the social psychology of work engagement: state of the field. career development international, 27(1), 36–53. https://doi.org/10.1108/cdi-08-2021-0213 bakker, a. b., & demerouti, e. (2007). the job demands‐resources model: state of the art. journal of managerial psychology, 22(3), 309–328. doi.org/10.1108/02683940710733115 bakker, a. b., & sanz-vergel, a. i. (2013). weekly work engagement and flourishing: the role of hindrance and challenge job demands. journal of vocational behavior, 83(3), 397–409. doi.org/10.1016/j.jvb.2013.06.008 bakker, a. b., hetland, j., olsen, o. k., espevik, r., & de vries, j. d. (2020). job crafting and playful work design: links with performance during busy and quiet days. journal of vocational behavior, 122, article 103478. doi.org/10.1016/j.jvb.2020.103478 bakker, a. b., van der zee, k. i., lewig, k. a., & dollard, m. f. (2006). the relationship between the big five personality factors and burnout: a study among volunteer counselors. the journal of social psychology, 146(1), 31–50. doi.org/10.3200/socp.146.1.31-50 australasian journal of information systems shaik et al., 2025, vol 29, research article an identification-based understanding of team engagement in gvts 31 baptista, j., stein, m. k., klein, s., watson-manheim, m. b., & lee, j. (2020). digital work and organisational transformation: emergent digital/human work configurations in modern organisations. the journal of strategic information systems, 29(2), article 101618. doi.org/10.1016/j.jsis.2020.101618 barley, s. r. (1990). the alignment of technology and structure through roles and networks. administrative science quarterly, 35(1), 61–103. doi.org/10.2307/2393551 barsade, s. g. (2002). the ripple effect: emotional contagion and its influence on group behavior. administrative science quarterly, 47(4), 644–675. doi.org/10.2307/3094912 bartunek, j. m., & louis, m. r. (1996). insider/outsider team research. sage publications. bridwell-mitchell, e. n., & mezias, s. j. (2012). the quest for cognitive legitimacy: organizational identity crafting and internal stakeholder support. journal of change management, 12(2), 189–207. doi.org/10.1080/14697017.2011.645053 cathro, v. (2020). an odyssey of virtual global team activity in the experiential learning environment of the global enterprise experience (gee). computers in human behavior, 107, article 105760. doi.org/10.1016/j.chb.2018.10.026 chamakiotis, p., dekoninck, e. a., & panteli, n. (2013). factors influencing creativity in virtual design teams: an interplay between technology, teams and individuals. creativity and innovation management, 22(3), 265–279. doi.org/10.1111/caim.12039 chamakiotis, p., panteli, n., & davison, r. m. (2021). reimagining e-leadership for reconfigured virtual teams due to covid-19. international journal of information management, 60, article 102381. doi.org/10.1016/j.ijinfomgt.2021.102381 chanana, n., & sangeeta. (2021). employee engagement practices during covid‐19 lockdown. journal of public affairs, 21(4), article e2508. doi.org/10.1002/pa.2508 choi, h. (2020). consumption of visual cues in computer-mediated environments. journal of distribution science, 18(8), 23–33. doi.org/10.15722/jds.18.8.202008.23 choudhury, p., foroughi, c., & larson, b. (2021). work‐from‐anywhere: the productivity effects of geographic flexibility. strategic management journal, 42(4), 655–683. doi.org/10.1002/smj.3251 choudhury, p., khanna, t., makridis, c. a., & schirmann, k. (2024). is hybrid work the best of both worlds? evidence from a field experiment. review of economics and statistics, 106(1), 1–24. doi.org/10.1162/rest_a_01428 ciriello, r. f., richter, a., & mathiassen, l. (2024). emergence of creativity in is development teams: a socio-technical systems perspective. international journal of information management, 74, article 102698. doi.org/10.1016/j.ijinfomgt.2023.102698 connaughton, s. l., & daly, j. a. (2004). identification with leader: a comparison of perceptions of identification among geographically dispersed and co‐located teams. corporate communications: an international journal, 9(2), 89–103. doi.org/10.1108/13563280410534294 corbin, j. m., & strauss, a. (1990). grounded theory research: procedures, canons, and evaluative criteria. qualitative sociology, 13(1), 3–21. doi.org/10.1007/bf00988593 costa, p. l., passos, a. m., & bakker, a. b. (2012). team work engagement: considering team dynamics for engagement (business research unit [bru-iul] working paper no. 12/06). iscte–iul. costa, p. l., passos, a. m., bakker, a. b., romana, r., & ferrão, c. (2017). interactions in engaged work teams: a qualitative study. team performance management: an international journal, 23(5/6), 206–226. doi.org/10.1108/tpm-12-2016-0054 australasian journal of information systems shaik et al., 2025, vol 29, research article an identification-based understanding of team engagement in gvts 32 cronin, m. a., weingart, l. r., & todorova, g. (2011). dynamics in groups: are we there yet? academy of management annals, 5(1), 571–612. doi.org/10.5465/19416520.2011.590297 dulebohn, j. h., & hoch, j. e. (2017). virtual teams in organizations. human resource management review, 27(4), 569–574. doi.org/10.1016/j.hrmr.2016.12.004 dutton, j. e., roberts, l. m., & bednar, j. (2010). pathways for positive identity construction at work: four types of positive identity and the building of social resources. academy of management review, 35(2), 265–293. doi.org/10.5465/amr.35.2.zok265 edwards, b. d., day, e. a., arthur jr, w., & bell, s. t. (2006). relationships among team ability composition, team mental models, and team performance. journal of applied psychology, 91(3), 727–736. doi.org/10.1037/0021-9010.91.3.727 edwards, r. (2005). predicting aggression amongst mentally disordered offenders: the role of narcissism, self-concept clarity and aggressive cognitive bias (doctoral thesis, university college london). ucl discovery. discovery.ucl.ac.uk/id/eprint/1444700/ farndale, e., pai, a., sparrow, p., & scullion, h. (2014). balancing individual and organizational goals in global talent management: a mutual-benefits perspective. journal of world business, 49(2), 204–214. doi.org/10.1016/j.jwb.2013.11.004 fetterman, d. m. (2019). ethnography: step-by-step (4th ed.). sage publications. fiol, c. m., & o'connor, e. j. (2005). identification in face-to-face, hybrid, and pure virtual teams: untangling the contradictions. organization science, 16(1), 19–32. doi.org/10.1287/orsc.1040.0101 gallup. (2023). state of the global workplace: 2023 report. https://www.gallup.com/workplace/349484/state-of-the-global-workplace.aspx geertz, c. (1973). deep play: notes on the balinese cockfight. in the interpretation of cultures (pp. 412–453). basic books. gilson, l. l., maynard, m. t., jones young, n. c., vartiainen, m., & hakonen, m. (2015). virtual teams research: 10 years, 10 themes, and 10 opportunities. journal of management, 41(5), 1313–1337. doi.org/10.1177/0149206314559996 gilson, l. l., maynard, m. t., jones young, n. c., vartiainen, m., & hakonen, m. (2015). virtual teams research: 10 years, 10 themes, and 10 opportunities. journal of management, 41(5), 1313–1337. doi.org/10.1177/0149206314559946 gioia, d. a., corley, k. g., & hamilton, a. l. (2013). seeking qualitative rigor in inductive research. organizational research methods, 16(1), 15–30. doi.org/10.1177/1094428112452151 glikson, e., & erez, m. (2020). the emergence of a communication climate in global virtual teams. journal of world business, 55(6), article 101001. doi.org/10.1016/j.jwb.2019.101001 hammersley, m., & atkinson, p. (2007). chapter 10: ethics. in ethnography: principles and practice (pp. 209–229). routledge. doi.org/10.4324/9781315146027 harter, j. k., schmidt, f. l., & hayes, t. l. (2002). business-unit-level relationship between employee satisfaction, employee engagement, and business outcomes: a metaanalysis. journal of applied psychology, 87(2), 268–279. doi.org/10.1037/00219010.87.2.268 hatfield, e., bensman, l., thornton, p. d., & rapson, r. l. (2014). new perspectives on emotional contagion: a review of classic and recent research on facial mimicry and contagion. interpersona: an international journal on personal relationships, 8(2), 159–179. doi.org/10.5964/ijpr.v8i2.162 australasian journal of information systems shaik et al., 2025, vol 29, research article an identification-based understanding of team engagement in gvts 33 hatfield, e., cacioppo, j. t., & rapson, r. l. (1993). emotional contagion. current directions in psychological science, 2(3), 96–100. doi.org/10.1111/1467-8721.ep10770953 hatfield, e., rapson, r. l., & le, y. l. (2009). emotional contagion and empathy. in j. decety & w. ickes (eds.), the social neuroscience of empathy (pp. 19–30). mit press. doi.org/10.7551/mitpress/9780262012973.003.0003 he, h., zhu, w., & zheng, x. (2014). procedural justice and employee engagement: roles of organizational identification and moral identity centrality. journal of business ethics, 122, 681–695. doi.org/10.1007/s10551-013-1774-3 henry, k. b., arrow, h., & carini, b. (1999). a tripartite model of group identification: theory and measurement. small group research, 30(5), 558–581. doi.org/10.1177/104649649903000504 hopkins, j., & bardoel, a. (2023). the future is hybrid: how organisations are designing and supporting sustainable hybrid work models in post-pandemic australia. sustainability, 15(4), article 3086. doi.org/10.3390/su15043086 ilgen, d. r., hollenbeck, j. r., johnson, m., & jundt, d. (2005). teams in organizations: from input–process–output models to imoi models. annual review of psychology, 56, 517– 543. doi.org/10.1146/annurev.psych.56.091103.070250 janssen, o., & huang, x. (2008). us and me: team identification and individual differentiation as complementary drivers of team members' citizenship and creative behaviors. journal of management, 34(1), 69–88. doi.org/10.1177/0149206307309263 jimenez, a., boehe, d. m., taras, v., & caprar, d. v. (2017). working across boundaries: current and future perspectives on global virtual teams. journal of international management, 23(4), 341–349. doi.org/10.1016/j.intman.2017.05.001 john, b., alsamarra'i, z., & panteli, n. (2025). reconfiguring digital embeddedness in hybrid work: the case of employee experience management platforms. information systems journal, 1–30. doi.org/10.1111/isj.12545 kagan, j. (1958). the concept of identification. psychological review, 65(5), 296–303. doi.org/10.1037/h0041313 kahn, w. a. (1990). psychological conditions of personal engagement and disengagement at work. academy of management journal, 33(4), 692–724. doi.org/10.5465/256287 kelman, h. c. (1958). compliance, identification, and internalization: three processes of attitude change. journal of conflict resolution, 2(1), 51–60. doi.org/10.1177/002200275800200103 kozlowski, s. w. j., & klein, k. j. (2000). a multilevel approach to theory and research in organizations: contextual, temporal, and emergent processes. in k. j. klein & s. w. j. kozlowski (eds.), multilevel theory, research, and methods in organizations: foundations, extensions, and new directions (pp. 3–90). jossey-bass. doi.org/10.2307/3094811 kniffin, k. m., narayanan, j., anseel, f., antonakis, j., ashford, s. p., bakker, a. b., ... van vugt, m. (2021). covid-19 and the workplace: implications, issues, and insights for future research and action. american psychologist, 76(1), 63–77. doi.org/10.1037/amp0000716 kooij, d. t., de lange, a. h., jansen, p. g., kanfer, r., & dikkers, j. s. (2011). age and work‐ related motives: results of a meta‐analysis. journal of organizational behavior, 32(2), 197–225. doi.org/10.1002/job.665 australasian journal of information systems shaik et al., 2025, vol 29, research article an identification-based understanding of team engagement in gvts 34 kozlowski, s. w., & ilgen, d. r. (2006). enhancing the effectiveness of work groups and teams. psychological science in the public interest, 7(3), 77-124. doi.org/10.1111/j.15291006.2006.0003 kozlowski, s. w., gully, s. m., nason, e. r., & smith, e. m. (1999). developing adaptive teams: a theory of compilation and performance across levels and time. in e. d. pulakos (ed.), the changing nature of work performance: implications for staffing, personnel actions, and development (pp. 240–292). jossey-bass. kpmg. (2022). kpmg global tech report 2022. https://kpmg.com/xx/en/home/insights/2022/09/kpmg-global-tech-report-2022.html lai, j. y., chan, k. w., & lam, l. w. (2013). defining who you are not: the roles of moral dirtiness and occupational and organizational disidentification in affecting casino employee turnover intention. journal of business research, 66(9), 1659–1666. doi.org/10.1016/j.jbusres.2012.12.012 lechner, a., & mortlock, j. m. t. (2022). how to create psychological safety in virtual teams. organizational dynamics, 51(2), article 100849. doi.org/10.1016/j.orgdyn.2021.100849 lee, c. c., aravamudhan, v., roback, t., lim, h. s., & ruane, s. g. (2021). factors impacting work engagement of gen z employees: a regression analysis. journal of leadership, accountability and ethics, 18(3), 147–159. doi.org/10.33423/jlae.v18i3.4414 liao, j., o'brien, a. t., jimmieson, n. l., & restubog, s. l. d. (2015). predicting transactive memory system in multidisciplinary teams: the interplay between team and professional identities. journal of business research, 68(5), 965–977. doi.org/10.1016/j.jbusres.2014.09.024 macey, w. h., schneider, b., barbera, k. m., & young, s. a. (2009). employee engagement: tools for analysis, practice, and competitive advantage. john wiley & sons. malhotra, a., & majchrzak, a. (2014). enhancing performance of geographically distributed teams through targeted use of information and communication technologies. human relations, 67(4), 389–411. doi.org/10.1177/0018726713495284 maslach, c., schaufeli, w. b., & leiter, m. p. (2001). job burnout. annual review of psychology, 52(1), 397–422. https://doi.org/10.1146/annurev.psych.52.1.397 microsoft work lab. (2021). the next great disruption is hybrid work—are we ready? https://www.microsoft.com/en-us/worklab/work-trend-index/hybrid-work miles, m. b. (1994). qualitative data analysis: an expanded sourcebook (2nd ed.). sage publications. mirbabaie, m., stieglitz, s., brünker, f., hofeditz, l., ross, b., & frick, n. r. (2021). understanding collaboration with virtual assistants – the role of social identity and the extended self. business & information systems engineering, 63(1), 21–37. doi.org/10.1007/s12599-020-00672-x mohammed, s., & dumville, b. c. (2001). team mental models in a team knowledge framework: expanding theory and measurement across disciplinary boundaries. journal of organizational behavior, 22(2), 89–106. doi.org/10.1002/job.86 newell, s., david, g., & chand, d. (2007). an analysis of trust among globally distributed work teams in an organizational setting. knowledge and process management, 14(3), 158–168. doi.org/10.1002/kpm.284 nordbäck, e. s., & espinosa, j. a. (2019). effective coordination of shared leadership in global virtual teams. journal of management information systems, 36(1), 321–350. doi.org/10.1080/07421222.2018.1558943 australasian journal of information systems shaik et al., 2025, vol 29, research article an identification-based understanding of team engagement in gvts 35 o'leary, m. b., mortensen, m., & woolley, a. w. (2011). multiple team membership: a theoretical model of its effects on productivity and learning for individuals and teams. academy of management review, 36(3), 461–478. doi.org/10.5465/amr.2009.0275 orlikowski, w. j., & scott, s. v. (2016). digital work: a research agenda. in d. courpasson & s. v. scott (eds.), a research agenda for management and organization studies (pp. 88–95). edward elgar publishing. doi.org/10.4337/9781784717025.00014 panteli, n., & sivunen, a. (2019). “i am your fan; bookmarked!” members’ identification development in founder-led online communities. journal of the association for information systems, 20(6), 824–841. doi.org/10.17705/1.jais.00552 panteli, n., chiasson, m., & yan, l. (2008). regulated and emergent identifications: the case of a virtual organization. in n. panteli & m. chiasson (eds.), exploring virtuality within and beyond organizations: social, global and local dimensions (pp. 126–150). palgrave macmillan. doi.org/10.1057/9780230593978_6 panteli, n., yalabik, z. y., & rapti, a. (2019). fostering work engagement in geographicallydispersed and asynchronous virtual teams. information technology & people, 32(1), 2– 17. doi.org/10.1108/itp-07-2017-0232 pearsall, m. j., & ellis, a. p. (2006). the effects of critical team member assertiveness on team performance and satisfaction. journal of management, 32(4), 575–594. doi.org/10.1177/0149206306289 pearsall, m. j., ellis, a. p. j., & bell, b. s. (2010). building the infrastructure: the effects of role identification behaviors on team cognition development and performance. journal of applied psychology, 95(1), 192–200. doi.org/10.1037/a0017781 polzer, j. t., crisp, c. b., jarvenpaa, s. l., & kim, j. w. (2006). extending the faultline model to geographically dispersed teams: how colocated subgroups can impair group functioning. academy of management journal, 49(4), 679–692. doi.org/10.5465/amj.2006.22083024 posthuma, r. a., & campion, m. a. (2009). age stereotypes in the workplace: common stereotypes, moderators, and future research directions. journal of management, 35(1), 158–188. doi.org/10.1177/0149206308318617 pratt, m. g. (1998). to be or not to be?: central questions in organizational identification. in d. a. whetten & p. c. godfrey (eds.), identity in organizations: building theory through conversations (pp. 171–207). sage. doi.org/10.1002/9781405186407.wbieco026 raghuram, s., hill, n. s., gibbs, j. l., & maruping, l. m. (2019). virtual work: bridging research clusters. academy of management annals, 13(1), 308–341. doi.org/10.5465/annals.2017.0020 ramarajan, l. (2014). past, present and future research on multiple identities: toward an intrapersonal network approach. academy of management annals, 8(1), 589–659. doi.org/10.5465/19416520.2014.912379 reade, c. (2001). antecedents of organizational identification in multinational corporations: fostering psychological attachment to the local subsidiary and the global organization. international journal of human resource management, 12(8), 1269–1291. doi.org/10.1080/09585190110083794 reicher, s. d., spears, r., & postmes, t. (1995). a social identity model of deindividuation phenomena. european review of social psychology, 6(1), 161–198. doi.org/10.1080/14792779443000049 australasian journal of information systems shaik et al., 2025, vol 29, research article an identification-based understanding of team engagement in gvts 36 riketta, m. (2005). organizational identification: a meta-analysis. journal of vocational behavior, 66(2), 358–384. doi.org/10.1016/j.jvb.2004.05.005 rockmann, k. w., & pratt, m. g. (2015). contagious offsite work and the lonely office: the unintended consequences of distributed work. academy of management discoveries, 1(2), 150–164. doi.org/10.5465/amd.2014.0016 saks, a. m. (2019). antecedents and consequences of employee engagement revisited. journal of organizational effectiveness: people and performance, 6(1), 19–38. doi.org/10.1108/joepp-06-2018-0034 salanova, m., agut, s., & peiró, j. m. (2005). linking organizational resources and work engagement to employee performance and customer loyalty: the mediation of service climate. journal of applied psychology, 90(6), 1217–1227. doi.org/10.1037/00219010.90.6.1217 sanford, n. (1955). the dynamics of identification. psychological review, 62(2), 106–117. doi.org/10.1037/h0044055 santillan, e. g., santillan, e. t., doringo, j. b., pigao, k. j. f., & von francis, c. m. (2023). assessing the impact of a hybrid work model on job execution, work-life balance, and employee satisfaction in a technology company. journal of business and management studies, 5(6), 13–38. doi.org/10.32996/jbms schein, e. h. (1969). process consultation: its role in organization development. addison-wesley. shaik, f. f., & makhecha, u. p. (2019). drivers of employee engagement in global virtual teams. australasian journal of information systems, 23, article 1770. doi.org/10.3127/ajis.v23i0.1770 shaik, f. f., makhecha, u. p., & gouda, s. k. (2020). work and non-work identities in global virtual teams: role of cultural intelligence in employee engagement. international journal of manpower, 42(1), 51–78. doi.org/10.1108/ijm-03-2019-0118 shuck, b., adelson, j. l., & reio jr., t. g. (2017). the employee engagement scale: initial evidence for construct validity and implications for theory and practice. human resource management, 56(6), 953–977. doi.org/10.1002/hrm.21811 sivunen, a. (2006). strengthening identification with the team in virtual teams: the leaders' perspective. group decision and negotiation, 15(4), 345–366. doi.org/10.1007/s10726-0069046-6 spradley, j. p. (1979). interviewing an informant. in the ethnographic interview (pp. 46–68). waveland press. spradley, j. p. (2016). the ethnographic interview (2nd ed.). waveland press. sun, l., & bunchapattanasakda, c. (2019). employee engagement: a literature review. international journal of human resource studies, 9(1), 63–80. doi.org/10.5296/ijhrs.v9i1.14167 sun, r., li, j. y. q., lee, y., & tao, w. (2023). the role of symmetrical internal communication in improving employee experiences and organizational identification during covid19 pandemic-induced organizational change. international journal of business communication, 60(4), 1398–1426. doi.org/10.1177/232948842110506 sundstrom, e., mcintyre, m., halfhill, t., & richards, h. (2000). work groups: from the hawthorne studies to work teams of the 1990s and beyond. group dynamics: theory, research, and practice, 4(1), 44–55. doi.org/10.1037/1089-2699.4.1.44 taras, v., baack, d., caprar, d., dow, d., froese, f., jimenez, a., & magnusson, p. (2019). diverse effects of diversity: disaggregating effects of diversity in global virtual teams. australasian journal of information systems shaik et al., 2025, vol 29, research article an identification-based understanding of team engagement in gvts 37 journal of international management, 25(4), article 100689. doi.org/10.1016/j.intman.2019.100689 the association for talent development. (2012). team relationships: time differences are hampering virtual teams. td magazine. https://www.td.org/content/tdmagazine/team-relationships-time-differences-are-hampering-virtual-teams torrente, p., salanova, m., & llorens, s. (2013). spreading engagement: on the role of similarity in the positive contagion of team work engagement. revista de psicología del trabajo y de las organizaciones, 29(3), 153–159. doi.org/10.5093/tr2013a21 torrente, p., salanova, m., llorens, s., & schaufeli, w. b. (2012). teams make it work: how team work engagement mediates between social resources and performance in teams. psicothema, 24(1), 106–112. https://www.redalyc.org/pdf/727/72723431017.pdf tsui, a. s., & o’reilly iii, c. a. (1989). beyond simple demographic effects: the importance of relational demography in superior-subordinate dyads. academy of management journal, 32(2), 402–423. doi.org/10.5465/256368 turner, j. c., brown, r. j., & tajfel, h. (1979). social comparison and group interest in ingroup favouritism. european journal of social psychology, 9(2), 187–204. doi.org/10.1002/ejsp.2420090207 tyler, t. r., & blader, s. l. (2003). the group engagement model: procedural justice, social identity, and cooperative behavior. personality and social psychology review, 7(4), 349– 361. doi.org/10.1207/s15327957pspr0704_07 van dick, r., christ, o., stellmacher, j., wagner, u., ahlswede, o., grubba, c., ... & tissington, p. a. (2004). should i stay or should i go? explaining turnover intentions with organizational identification and job satisfaction. british journal of management, 15(4), 351–360. doi.org/10.1111/j.1467-8551.2004.00424.x van maanen, j. (2006). ethnography then and now. qualitative research in organizations and management: an international journal, 1(1), 13–21. doi.org/10.1108/17465640610666615 van maanen, j. (2011). ethnography as work: some rules of engagement. journal of management studies, 48(1), 218–234. doi.org/10.1111/j.1467-6486.2010.00980.x wessel, l., davidson, e., barquet, a. p., rothe, h., peters, o., & megges, h. (2019). configuration in smart service systems: a practice-based inquiry. information systems journal, 29(6), 1256–1292. doi.org/10.1111/isj.12268 whillans, a., perlow, l., & turek, a. (2021). experimenting during the shift to virtual team work: learnings from how teams adapted their activities during the covid-19 pandemic. information and organization, 31(1), article 100343. doi.org/10.1016/j.infoandorg.2021.100343 wiesenfeld, b. m., raghuram, s., & garud, r. (1999). communication patterns as determinants of organizational identification in a virtual organization. organization science, 10(6), 777–790. doi.org/10.1287/orsc.10.6.777 wtw. (2021). 2021 employee experience survey. https://www.wtwco.com/enin/insights/2021/07/2021-employee-experience-survey yun, m., & beehr, t. (2024). work engagement contagion from leader to follower through cognitive and affective mechanisms. leadership & organization development journal, 45(1), 153–166. doi.org/10.1108/lodj-04-2023-0200 zakaria, n., & yusof, s. a. m. (2020). crossing cultural boundaries using the internet: toward building a model of swift trust formation in global virtual teams. journal of international management, 26(1), article 100654. doi.org/10.1016/j.intman.2018.10.004 australasian journal of information systems shaik et al., 2025, vol 29, research article an identification-based understanding of team engagement in gvts 38 zhang, y., & chen, c. c. (2013). developmental leadership and organizational citizenship behavior: mediating effects of self-determination, supervisor identification, and organizational identification. the leadership quarterly, 24(4), 534–543. doi.org/10.1016/j.leaqua.2013.03.007 zhu, w., wang, g., zheng, x., liu, t., & miao, q. (2013). examining the role of personal identification with the leader in leadership effectiveness: a partial nomological network. group & organization management. advance online publication. https://doi.org/10.1177/1059601112456595 australasian journal of information systems shaik et al., 2025, vol 29, research article an identification-based understanding of team engagement in gvts 39 appendix table 1: representative quotes for the cascading effect of role, team, and organizational identification, leading to employee engagement contagion in gvts sno. 1st-order codes 2nd-order themes aggregate dimensions verbatim excerpt 1 when role aligns with company goals, commitment improves alignment of role with organization higher identification with the role cascade to a higher organizational identification "when my work aligns with what the company is trying to achieve, it’s easier to stay committed. it’s not just about completing tasks—it’s about knowing that what i do actually contributes to something bigger. when there’s a clear connection between my role and the company’s direction, i feel more invested in the work. i remember when cyient shifted some of its hr and payroll processes, and we had to adjust how we handled benefits here in hartford. at first, it felt like just another change being pushed down from leadership. but once it became clear how our work was helping streamline operations across locations, it made a difference. i wasn’t just processing numbers—i was part of something that made things better for employees across different sites. that kind of alignment makes you want to give your best, because you see the impact beyond just your desk." 2 feeling valued in role leads to organizational commitment feeling valued in role higher identification with the role cascade to a higher organizational identification "when i first started handling immigration at cyient, it was just me figuring things out. my boss was in india, and for years, we never even met in person—everything was over calls or webex. there were moments where i wondered if anyone even noticed the work i was putting in. but over time, when people started reaching out to me directly, trusting me to make decisions, and i saw how my work was actually making a difference, that’s when i really started feeling a sense of belonging. it wasn’t just about processing paperwork—it was about making sure people transitioning from india to the u.s. had a smooth experience. when the company backed the work i was doing, whether it was through autonomy or just acknowledging the effort, it reinforced that i wasn’t just another employee—i was part of something bigger. and i see this with others too—once they know their work is seen and valued, they stop feeling like they’re just doing a job and start feeling like they truly belong." 3 team bonding activities strengthen work relationships team cohesion higher identification with the role cascade to a higher team identification "sometimes, when we take time to do activities outside of our regular tasks, it really helps build stronger connections with the people we work with every day. it’s easy to get caught up in project deadlines and customer requirements, but when we step away from that, even for a short time, it makes a big difference in how we collaborate. i remember last year when our team in connecticut organized a small outing after wrapping up a major project for pratt & whitney. it wasn’t anything big—just a casual dinner—but that one evening helped us connect beyond just work. we got to know each other better, shared a few laughs, and the next time we were troubleshooting an issue together, things felt smoother. when you have that bond, you’re more comfortable reaching out, asking for help, and working australasian journal of information systems shaik et al., 2025, vol 29, research article an identification-based understanding of team engagement in gvts 40 together as a unit. it’s those little moments that help strengthen the way we work as a team." 4 positive peer influence boosts team motivation peer influence higher identification with the role cascade to a higher team identification "when you’re working in a close-knit team like we have here in georgia, the energy of the people around you really sets the tone for how you feel about your work. if you’re surrounded by people who are engaged, pushing hard to meet goals, and willing to step in when needed, it makes a huge difference. it’s not just about getting the work done— it’s about feeling like you’re all in it together. i remember when we were up against a tight deadline on a project for pratt & whitney, and it was one of those all-hands-on-deck situations. nobody was just clocking in and out—we were all invested. seeing my teammates put in the effort, troubleshoot together, and support each other even when things got stressful made me want to give my best too. and that kind of motivation spreads. when one person brings energy, it lifts the whole team. that’s why the people you work with daily matter so much—your team’s mindset shapes how you approach your own work." 5 smoother workflow when closely working with the team team-based workflow higher identification with the team cascade to a higher organizational identification "when we have regular interactions and stay in sync with each other, everything moves more smoothly. there’s less back and forth, fewer misunderstandings, and overall, things just get done more efficiently. when everyone is on the same page, the workflow naturally improves. i remember a time when we were working on a complex update for pratt & whitney’s systems. initially, there were delays because we weren’t coordinating properly—offshore and onsite teams were working in silos, and things kept getting lost in translation. but once we set up daily checkins and made sure everyone was aligned, the process became so much easier. we weren’t waiting on clarifications or redoing work because of miscommunication. when you work closely with your team, you don’t just finish tasks faster—you work smarter, and the entire process becomes more seamless." 6 better problemsolving skills within closelyknit teams problemsolving within teams higher identification with the team cascade to a higher organizational identification "i remember this one time at cyient when one of our engineers in india was stuck on a modeling issue. he’d been working on it for days, trying different approaches, but nothing seemed to click. finally, during one of our regular team calls, he mentioned it almost in passing. within minutes, someone from the u.s. team jumped in with a suggestion—something similar they’d dealt with a few months back. that tiny exchange solved the problem in less than 10 minutes, something that would’ve otherwise taken him another few days to figure out on his own. and i see this happen all the time. when people work in silos, progress slows down, but when we openly discuss roadblocks, things move a lot faster. it’s not just about making an individual’s job easier—it creates a mindset where people naturally reach out, share ideas, and think beyond their immediate tasks. and the impact goes beyond just the team. when you develop that habit of seeking input australasian journal of information systems shaik et al., 2025, vol 29, research article an identification-based understanding of team engagement in gvts 41 and working through challenges collectively, it makes collaborating with other departments and even clients much more effective. that’s when you’re not just solving problems—you’re building stronger connections across the organization." 7 clear expectations reduce ambiguity clear expectations higher identification with the team cascade to a higher role identification "when things aren’t clear from the start, it slows everything down. instead of focusing on the actual work, you spend time chasing answers, double-checking details, or fixing things that weren’t right the first time. but when expectations are laid out properly, you can just get on with it without second-guessing yourself. i remember when we were working through changes in payroll processing here in hartford. at first, there was a lot of back and forth because we weren’t sure exactly what was needed from us—the instructions were broad, and we kept running into errors. but once we had a structured process in place, with clear guidance for each step, everything became much more efficient. when you don’t have to constantly stop and ask for clarification, you can focus on doing the work right the first time." 8 high personal investment in role performance personal investment in role higher identification with the team cascade to a higher role identification "when my work has real value and i can see how it contributes to something meaningful, i naturally push myself to do more. it’s not just about meeting deadlines— it’s about knowing that my efforts matter, and that makes all the difference in how i approach my work. i remember when we were handling a complex integration project for pratt & whitney. there were late nights and unexpected challenges, but what kept me motivated was knowing that my contributions were critical to making the transition seamless. no one had to tell me to stay a little longer or double-check my work—i wanted to because i knew it would have a real impact. when you truly connect with what you do, you don’t just complete tasks—you take ownership and give it your best." 9 feeling lost due to lack of organizational guidance organizational disorientation reduced identification with the role cascade to reduced organizational identification "when you come here from india to the u.s., it’s not like you have someone constantly guiding you on what to do or how things work. the company gives a briefing, sure, but after that, it’s mostly figuring things out on your own. like, when i first joined cyient, i didn’t even meet my boss in person for seven years! everything was over calls or webex. so, when people land here, they naturally turn to whoever they know—other colleagues, friends who’ve been here longer— because otherwise, it’s easy to feel a bit lost. especially when you’re sitting at the customer site, not in a cyient office, it’s like you’re neither here nor there. the hr folks do try to check in, you know, with lunch meetings or potlucks to remind people that they still belong to cyient, but beyond that, it’s mostly on you to navigate things." australasian journal of information systems shaik et al., 2025, vol 29, research article an identification-based understanding of team engagement in gvts 42 10 more engagement with team members than the larger organization team-centered focus reduced identification with the role cascade to reduced organizational identification "day-to-day, my focus is really on the people i work with directly. that’s who i talk to, who i rely on, and who i feel most connected to. beyond that, cyient as a whole feels distant—decisions get made, policies change, but we’re not always looped in or given much context. it’s like we’re just expected to go along with whatever comes next. i remember when they rolled out changes to our payroll system. those of us handling payroll in the u.s. had to figure things out as we went because a lot of the setup was done with india’s processes in mind, not ours. it was frustrating because while my team here in hartford worked together to solve the issues, we weren’t getting much support from the larger organization. that’s when you really feel the difference— you trust and rely on your immediate group, but beyond that, it’s harder to feel like you’re truly part of something bigger." 11 unclear role expectations reduce team collaboration unclear role expectations reduced identification with the role cascade to reduced team identification "i remember a conversation with someone at cyient who was frustrated with how things were running. he told me, ‘i’m doing my part, but i don’t know if i’m doing it the right way or if it even fits with what others are doing. half the time, i don’t even know who’s responsible for what, so we just end up working in silos.’ and that’s a real problem. at the clinic, i saw the same thing. when expectations weren’t clear—when one person assumed a task was theirs while another thought the same—it created confusion. people would either duplicate work or leave gaps because they weren’t sure where their role ended and someone else’s began. and when that happens, collaboration takes a hit. instead of working together, people hesitate to ask questions, afraid they might step on someone’s toes or look like they don’t know what they’re doing. over time, this leads to disengagement. when no one is sure of their role, they start pulling back, and soon enough, the whole team dynamic starts to break down." 12 weak team association affects organizational commitment weak team bonds reduced identification with the role cascade to reduced team identification "i was speaking with an engineer from hyderabad the other day, and he told me, ‘mohan, i do my work, but beyond that, i don’t really talk to anyone. my manager is in the u.s., my colleagues are on different projects, and honestly, i wouldn’t say i feel particularly connected to the company. i could be working anywhere, and it wouldn’t make much of a difference.’ and that hit me because i’ve heard similar things before. i’ve seen this happen across different locations—hyderabad, connecticut, and even bangalore. when people don’t have strong working relationships, they start feeling like they’re just completing tasks instead of being part of something bigger. it’s not about whether they like the work—it’s about whether they feel a sense of connection to the people they work with. and when that connection is weak, their commitment fades. they don’t push themselves as much, they stop participating in discussions, and eventually, they start looking elsewhere. once that starts, it spreads—because when one person disengages, it affects the people around them, and before you know it, the whole team feels disconnected." 13 employees see better growth restricted career growth reduced identification "one of the biggest hurdles we have—especially with employees working at the customer site in places like australasian journal of information systems shaik et al., 2025, vol 29, research article an identification-based understanding of team engagement in gvts 43 opportunities with customers with the role cascades to reduced team identification connecticut or florida—is that they start viewing the customer as the place where real career growth happens. it’s not necessarily that cyient doesn’t offer opportunities, but because they’re embedded in the customer’s environment every day, they see firsthand how career paths unfold there. they notice people getting promoted, moving into new roles, and that makes it feel like the better option. meanwhile, they’re not as exposed to what’s happening within cyient, so they don’t see a clear path forward here. and this thinking doesn’t just stay with one person—it spreads. others in similar roles pick up on it, and soon, you’ve got an entire team that feels like cyient is just a stepping stone rather than a place to grow. that’s when engagement drops, and retention becomes a real challenge." 14 communication barriers due to time zone differences time zone constraints reduced identification with the role cascades to reduced team identification "sometimes, we have calls kicking off at 7 a.m. eastern time, and i’ve got someone on the team—key to the discussion— who’s out in oregon, and for her, it’s 4 a.m. that’s just how it is when working across different time zones. and then, at the same time, folks in india are joining in, and it’s late evening for them. you know, it’s not just about getting people on the call; it’s about how engaged they actually are. i mean, someone who’s been at work all day in india might just be drained by the time we’re getting started, and then someone else is rolling out of bed in the middle of the night just to be there. so, naturally, there’s this kind of disconnect—not that people don’t care, but because everyone’s running on a different clock. that makes it tough to feel like you’re all really in sync as a team." 15 minimal peer collaboration lowers role connection peer isolation reduced identification with the team cascade to reduced organizational identification "not working closely with my peers makes my job feel isolated. in a global organization like cyient, where teams are spread across the us, india, and germany, when i don’t have regular interaction with colleagues, it becomes difficult to feel connected to the team’s overall goals. i’ve experienced this when i worked on a project with colleagues in different locations, and there were days when i felt disconnected from the group's efforts because we weren’t collaborating closely. for example, when my peers and i weren’t engaging frequently, it felt like i was working alone, even though i was part of a bigger team. this lack of peer collaboration made my work feel less meaningful because i couldn’t see how my efforts fit into the broader picture. over time, i started feeling less motivated, and i noticed others feeling the same way. this isolation from my peers didn’t just impact me; it spread across the group. when collaboration is minimal, it’s harder to build connections with others, and that lack of interaction weakens my overall engagement with the company’s mission. the more isolated we feel from our peers, the harder it becomes to stay invested in our work and aligned with the organization’s objectives." 16 lack of teamwork reduces individual engagement reduced teambased engagement reduced identification with the team cascade to reduced "i was talking to someone at cyient who had been there for a while, and she told me, ‘honestly, some days, i just log in, do my work, and log out. i don’t really feel like it matters if i’m here or not because there’s no real collaboration— everyone just does their own thing.’ that really hit me because i’ve seen the same thing happen in different places. australasian journal of information systems shaik et al., 2025, vol 29, research article an identification-based understanding of team engagement in gvts 44 organizational identification at the clinic, for example, i noticed that when people worked together—sharing responsibilities, bouncing ideas off each other—they were more involved. even if the work was tough, they stayed engaged because they felt like they were part of something. but in teams where there was no real collaboration, where people just worked in their own little silos, motivation dropped. when there’s no back-andforth, no shared effort, work starts feeling mechanical. people stop going the extra mile, they stop feeling invested, and over time, they check out mentally. it’s not that they don’t want to be engaged—it’s just that when there’s no real interaction, there’s nothing holding them to the work beyond a paycheck." 17 low role clarity diminishes team participation role clarity deficiency reduced identification with the team cascade to reduced role identification "i was talking to one of the guys from the hyderabad office the other day, and he told me, ‘mohan, sometimes i’m not sure what exactly is expected from me. i get tasks, but there’s no real clarity on where my role begins and ends. so, i just do what’s assigned and leave it at that.’ and that really stuck with me because i’ve seen how that kind of uncertainty affects the way people work. i’ve noticed this in different places—whether it’s at cyient in connecticut or in bangalore. when people don’t have a clear sense of their role, they hesitate to step up. instead of actively contributing, they wait for instructions. they don’t share ideas, they don’t take initiative, and slowly, they pull back from engaging with others. and once that happens, it spreads. if one person is unsure about their responsibilities, others start feeling the same way, and soon, the whole group becomes less collaborative. it’s not that people don’t want to contribute—it’s just that without clarity, they don’t know how to." 18 weak job identification affects team belonging job identification weakness reduced identification with the team cascade to reduced role identification "when i don’t feel connected to my job, it makes me feel like i don’t quite belong within the team. this is particularly true in a global company like cyient, where teams are spread across different countries like the us, india, and germany. as the conversation shared, working in such dispersed teams often means that job roles can become less clear, which in turn makes it difficult to feel integrated within the group. for example, when roles are not well defined or when the connection to one’s tasks feels weak, as discussed in the transcript, it creates a sense of being ‘out of place.’ without a strong understanding of how my individual work contributes to the team’s goals, it becomes harder to connect with my colleagues on a personal level. this sense of misalignment leads to a decreased sense of belonging, not only with the team but also with the larger objectives of the organization, as it weakens the identification with the job itself." 19 lack of employee engagement makes retention harder employee retention barriers reduced identification with the team cascades to reduced organizational identification "one of the challenges we face—especially with folks working at the customer site, whether in connecticut or florida—is that they start seeing more career opportunities with the customer than with cyient. and i get it. when you're immersed in the customer’s environment every day, you see the promotions happening there, the new roles opening up, and it just feels like there’s a clearer path forward. on the other hand, because they’re not as plugged australasian journal of information systems shaik et al., 2025, vol 29, research article an identification-based understanding of team engagement in gvts 45 into what’s happening at cyient, they don’t always see the career tracks we offer internally. and when one person starts thinking this way, it spreads—others begin questioning their own future here, and suddenly, you have an entire team that feels like cyient is just a temporary stop. that’s when retention becomes really difficult." 20 emails are often misinterpreted, leading to communication breakdowns virtual communication gaps reduced identification with the team cascades to reduced organizational identification "i’ve seen team members hesitate to send emails because they’re worried about how their words might come across. one time, a colleague from india sent a message that was meant to be direct and to the point, but the person in new york read it as rude. it turned into this whole thing where people started second-guessing whether they should even email at all. instead of clarifying things quickly, they’d hold back, wait for a call, or just let issues linger. over time, that kind of hesitation makes it harder for people to feel connected to the larger team at cyient. when you’re constantly worrying about being misunderstood, you start pulling back from discussions, and that affects how much people really feel like they belong to what we’re doing." 21 feeling disconnected makes it harder to stay motivated workplace isolation reduced identification with the team cascades to reduced role identification "you know, when you’re working remotely or stationed at a customer site, like in florida, it’s easy to feel like you’re on your own. you do your work, attend meetings, send updates, but if you don’t see how your contributions fit into the bigger picture at cyient, motivation starts to slip. it’s not about the workload or the tasks—it’s about feeling connected to something larger than just a to-do list. i had a guy on my team who was great at what he did, but after a while, i could tell something was off. he was getting things done, but there was no enthusiasm. when i finally asked him what was wrong, he just shrugged and said, ‘i don’t know, i just feel like i’m working in a bubble.’ that stuck with me. he wasn’t struggling with the technical work—he was struggling because he felt isolated. and honestly, i get it. when most of your interactions are just task-based, and there’s no real connection to the broader team or company, it becomes harder to stay engaged. eventually, that lack of connection starts affecting not just motivation, but the way people see their work entirely." 22 isolation reduces overall job satisfaction workplace isolation reduced identification with the team cascades to reduced role identification "one of the toughest things about employees working at customer sites—whether in germany or canada—is that they start feeling cut off from the larger cyient network. when most of their daily interactions are with the customer’s team, they begin operating in a bubble. that isolation wears on them—job satisfaction dips, motivation fades, and morale declines. i’ve seen it happen. someone like priya, stationed in france, starts out excited but slowly drifts. she follows the customer’s schedule, attends their events, and after a while, cyient just feels like a name on a paycheck. same with rohan in canada—he once told me, ‘honestly, i’ve been here so long, i already feel like part of the customer’s team.’ when the customer offers a job, it’s an easy decision because, mentally, they already left cyient long ago. and that mindset spreads. when one person disengages, others notice, and before you know it, you’ve got an entire team questioning why they’re still here.” australasian journal of information systems shaik et al., 2025, vol 29, research article an identification-based understanding of team engagement in gvts 46 copyright copyright © 2025 farheen fathima shaik; upam pushpak makhecha; biju varkkey; sirish kumar gouda. this is an open-access article licensed under a creative commons attribution-noncommercial 4.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v29.5293 https://creativecommons.org/licenses/by-nc/4.0/deed.en https://creativecommons.org/licenses/by-nc/4.0/deed.en https://doi.org/10.3127/ajis.v29.5293 paper title (paper title style) australasian journal of information systems miah & vu 2020, vol 24, research note healthcare situation monitoring method 1 towards developing a healthcare situation monitoring method for smart city initiatives: a citizen safety perspective shah jahan miah vu business school, victoria university, melbourne, australia shah.miah@vu.edu.au h. quan vu department of information systems and business analytics, deakin business school, deakin university, burwood, australia abstract research in smart city development has proliferated over the past few years, focussing heavily on various supporting service sectors, such as healthcare. however, little effort has been made to design health surveillance support systems, which are also important for the advancement of public healthcare monitoring as an essential aspect of smart city initiatives. from an information system (is) design perspective, this paper introduces a social mediabased health surveillance supporting method, which can automatically extract relevant online posts for health symptom prediction and management. we describe and demonstrate an is design approach in this paper for hay-fever prediction solution concept based on twitter posts. this concept could be developed into a fully functional solution design by practitioners in this field. keywords: healthcare, smart city, social media data, healthcare monitoring, public heath surveillance 1 introduction smart city research initiatives have proliferated over the past few years in various service development sectors, since hollands (2008) introduced its core idea. many academic and nonacademic efforts have been initiated for developing research in smart city. studies that promote people-oriented needs, especially on issues related to healthy living and environment, such as public safety, transportation, environmental sustainability and health care for active living, are still in high demand (guedes et al. 2018). examples of relevant studies that focus on the mentioned aspects in the healthcare domain include forkan, khalil, tari, foufou, and bouras (2015), aziz et al. (2016), and aborokbah et al. (2018). however, current research into healthcare shows limited understanding on formulating real problems and practical conceptualizations for building solution design options and explorations in actual scenarios. developments are still required, such as for strategic decision support by smart city authorities. social media have created huge opportunities and data sources for developing virtual surveillance systems for healthcare authorities. social media data offers real-time access to bigaustralasian journal of information systems miah & vu 2020, vol 24, research note healthcare situation monitoring method 2 data that contain important details of citizens’ lifestyle and personal well-being. social media big data have been considered both as a valuable and reliable source for identifying concurrent details of human activities and their opinions, (thelwall et al. 2016). however, appropriate surveillance systems methodology for rigorous analysis of social media data is still underdeveloped, in particular for monitoring health conditions symptoms, such as hay-fever symptoms among communities. this would support the authorities ability to observe signs or indications of true predictions and prevalence. in this work, we aim to address this need, especially for designing social media-based health surveillance support for the prediction of public disease and their spreading symptoms for effective management. the current study differs from prior studies above by the use of online social media data (e.g. twitter), which has been proven a useful and reliable source of data collections in recent studies (he et al., 2015; chua et al., miah, vu, gammack & mcgrath, 2017; 2016; miah, vu and gammack, 2019). 2 methodology twitter data relating health symptoms, such as “hay fever” also known as pollen allergy, are extracted during the spring season in a specific country or region, using twitter’s application programing interface (api) (twitter, 2018). keywords, such as “hayfever”, “hay fever”, “allergic”, “allergy” can be provided as parameters to retrieve relevant tweets. the geographical area to perform the surveillance can be defined as a bounding box, whose coordinates can be specified as parameters corresponding to maximum latitude (maxla), minimum latitude (minla), maximum longitude (maxlo) and minimum longitude (minlo). the textual content of the tweets and their meta-data, such as userid, user location, date and time of tweet, geographical location of tweet, are also extracted where available. the raw tweet data are unstructured and needs to be converted into suitable format for subsequent analysis. we first start with processing textual content of the tweet following some standard steps for text mining (sharda et al., 2014). the text is first loaded into a text tokenizing algorithm, wherein the string of text is broken into words, phrases, or symbols called “tokens”. the tokens are then passed through a filter, where capital letters are normalized into lower case. tokens containing symbols, numbers or stop words (e.g. “the”, “that” and “very”) are removed because they are not considered useful for analysing the tweet meaning. the remaining tokens then go through a stemming processing to reduce words into their stem, base, or root form, such as “sneezing” to “sneez” and “flowers” to “flower”. the processed textual contents of each tweet are stored as a list of stemmed words for later analysis. next, we process location data of the tweets to identify place name. the latitude and longitude are input into google’s geocoding api (google, 2018), which return the name of the corresponding location (city and country). once the data are collected and pre-processed, we apply several exploratory analysis techniques to the data for extracting meaningful insights, including important word analysis, word relationship analysis and temporal pattern analysis. 3 case study we evaluated the effectiveness of the proposed approach in healthcare situation monitoring for australia, whose large population proportion suffers from hay-fever. we deployed a streaming function of twitter api to cover spring season in australia. a bounding box, whose australasian journal of information systems miah & vu 2020, vol 24, research note healthcare situation monitoring method 3 coordinates are maxla=-11.0442 minla=-45.3812, maxlo=153.8456 and minlo=110.6787, was defined to cover the entire geographical area of australia. all collected tweets were validated, duplicate tweets were removed, and the tweets arranged in temporal order from earliest to latest. after data pre-processing, a total of 1893 tweets relevant to hay fever from 756 users were available for analysis. the majority of the tweets were posted in the six most populated cities in australia; melbourne, sydney, perth, brisbane, adelaide and canberra. the first analysis stage is to identify popular words relevant to hay fever in the data set. we computed the frequency of each word with respect to the number of tweets, and then selected most popular ones for examination. from the management perspective, government agencies would like to monitor public health in relation to hay-fever such as condition, cause, effect on the body, treatment and affected season. we examined the remaining frequent words to identify those relevant to hay fever as shown in table 1. the most frequent relevant words are “allergic” “hay fever”, “fever” and “allergy”, which is consistent with the fact that these words were used as keywords to detect those relevant tweets. we found “asthma”, “cold”, “sick” are also frequently mentioned in the tweet data set. “sneez” and “itchy” are common symptoms in relation to hay fever. “sneez” is a root form of the word sneezing after removing the suffix “ing” through the stemming process. common causes of hay fever were identified in the tweets, including “pollen”, “flower”, “grass” and “tree”, with “pollen” being the most frequent cause (support value of 0.134). other allergy causes were also included, such as “food”, “cat”, “dog” and “peanut”. the most affected body part was “eye”, “nose”, “face”, “head” and “throat”. we found the word “antihistamine”, drugs that treat allergic rhinitis and allergies, were frequently mentioned in the tweets. “spring” and “summer” are common seasons mentioned posts about hay fever and allergy. group condition cause body part treatment season word (support) allergic (0.268) hay fever (0.241) fever (0.166) allergy (0.166) reaction (0.051) asthma (0.024) cold (0.021) sneez (0.020) sick (0.012) itchy (0.010) pollen (0.134) food (0.037) cat (0.029) flower (0.016) dog (0.015) peanut (0.013) grass (0.011) tree (0.010) eye (0.025) nose (0.018) face (0.012) head (0.010) throat (0.010) antihistamine (0.010) spring (0.057) summer (0.016) table 1: popular words relevant to hay fever subsequently, we analysed the relationships between them for understanding how the condition and causes are linked together. a graph visualization technique was applied to illustrate the connection between terms. each word is a node, and an edge represents a connection between two words if they appear in the same post. for the ease of analysis, we only draw the edge between words that appear together in at least five tweets. the strength of the connection is represented by the thickness of the edge and the number of tweets appearing together. figure 1a shows that people tend to use the word “allergic reaction” when mentioning about other type of allergy such as with “cat”, “dog”, “food allergy”. “pollen” from “flower”, “grass” and “tree” are the main cause of “hay fever” and “asthma”. common symptoms linked to hay fever are “sneez”, “itchy” and “sick. we found that “hay fever” and “cold” are also mentioned frequently in the same tweets, although they are basically different australasian journal of information systems miah & vu 2020, vol 24, research note healthcare situation monitoring method 4 type of conditions. this is probably because people sometime get confused between these two conditions due to their similar symptoms. figure 1b shows that “eye”, “head”, “throat” and “nose” are the body part affected by “hay fever”, and “antihistamine” is commonly mentioned as its treatment. for people have allergic reaction other than hay fever, their “face” is mostly affected. (a) condition and cause (b) condition, body part, treatment figure 1: association between words in tweets next, we examine the temporal patterns of the tweets in relation to the diseases. we found different patterns among tweets containing the words “allergic” and “hay fever. the peak period for tweeting about hay fever is in springtime of australia (september to december). however, tweets containing the word “allergic” were mainly posted at the beginning of the year (january to march) and at the end of the year (october to december), which indicates that the allergic reaction is caused from other sources rather than pollen. further temporal analysis of hay fever causes showed that the peak time for “pollen” is august to december, which is aligned with the time for spring and early summer. it should be noted that different people might react with different type of pollens, so detailed information on which pollen type may provide useful information for treatment. we can see that the peak time for “flower” is september and october while, the peak time for “tree” is december. tree pollens are different from ordinary flower pollens, as they are usually from tree species bearing separate male and female flowers in the same plant such as oak, sweet gum, pine and spruce. we further examined the patterns with respect to geographical locations (e.g. city) and found that. melbourne residents are more likely to tweet about their condition around 9:00 to 11:00, while sydney residents are likely to tweets in early morning 8:00 to 9:00 or in the afternoon around 16:00. more people in melbourne are having allergy caused by pollen from “grass” and “tree” than those in sydney. on the other hand, sydney residents posted tweets containing “cat” and “flower” more than those in melbourne and more residents having symptoms at their head. 4 discussions and conclusion under the smart city research paradigm, our study aimed to extend current views of the social media analytics in particular for the purpose of public health-related knowledge discovery. australasian journal of information systems miah & vu 2020, vol 24, research note healthcare situation monitoring method 5 the suitability of social media data for disease surveillance was demonstrated in a case study of hay fever monitoring in australia. the analysis, based on the characteristic features of hay fever related twitter messages, provided useful insights. this paved away for a larger scale investigation into the topic in the upcoming pollen seasons. as a result, the benefits of highquality input data as well as the visualization tool for patterns exploration considerably facilitated new knowledge discovery. the pilot method extracted valuable knowledge about this common disease from twitter data. this study appears to be the first that attempts to utilize social media analytics for hay fever surveillance in australia. the results have shown that our method is effective in capturing tweets posted by australian residents about their health condition, especially in relation to hay fever and allergy. textual analysis with graph visualization was able to reveal interesting relationships between the conditions, cause, and affected body part. temporal patterns revealed insights into the times when people tweet about different conditions and causes. comparison between cities suggests that residents at different location could have different health conditions and associated causes. we attempted to address healthcare authorities’ requirements for accurate discovery of public diseases and their spreading symptoms for a healthcare management. smart technologies such as text processing techniques can produce relevant insights that provide supporting information that may inform planning for surges in patient visits, therapeutic supplies, and public health information dissemination campaigns. these aspects are very important for policy makers or decision makers in government or public welfare organizations. it is important because the pollen allergy, commonly referred to as ‘hay fever’ induces a range of problematic symptoms that significantly interfere with sufferers’ daily activities. the limitation of this pilot work is that we only ran the data for a single year, and as the data was accessed using a free account with a quota limitation, the collected data may not generalize well. however, we did find some interesting and useful patterns from the current data set. the analysis with respect to location was at large scale of city level. detailed analysis at suburb level would be useful to provide further insight when there is more data with gps information available. a further study would aim to develop a fully functional prototype of a social media analytics solution that could other other aspects within community boundaries. a context sensitive design methodology (miah, 2004; miah and gammack, 2014), in particular for decision support or business intelligence design (ali, miah and khan, 2018; miah & ahmad, 2013; miah, 2008), would be applied for designing an innovative social media analytics artifact. references aborokbah, m.m., al-mutairi, s., kumar, a., & samuel. o.w., (2018). adaptive context aware decision computing paradigm for intensive health care delivery in smart cities—a case analysis, sustainable cities and society, 41, 919-924 ali, ms, miah, sj, & khan, s. (2018). antecedents of business intelligence implementation for addressing organizational agility in small business context, pacific asia journal of the association for information systems, 10 (1), 89-108 aziz, k., tarapiah, s., ismail, s.h., & atalla, s. (2016). smart real-time healthcare monitoring and tracking system using gsm/gps technologies, in the proceedings of the 3rd mec australasian journal of information systems miah & vu 2020, vol 24, research note healthcare situation monitoring method 6 international conference on big data and smart city, 2016, url: https://ieeexplore.ieee.org/document/7460394 chua, l. servillo, e., marcheggiani, a. & moere, v. (2016). mapping cilento: using geotagged social media data to characterize tourist flows in southern italy, tourism management, 57, 295–310. forkan, a., khalil, i., tari, z., foufou, s., & bouras, a. (2015). a context-aware approach for long-term behavioural change detection and abnormality prediction in ambient assisted living, pattern recognition, 48(3), 628-641. google. (2018). geocoding api. retrieved may 2018, url: https://developers.google.com/maps/documentation/geocoding/intro guedes, a. l. a. alvarenga, j.c., goulart, m.d.s.s., rodriguez, m.v.r., & soares, c. a. p. (2018). smart cities: the main drivers for increasing the intelligence of cities, sustainability, 10(9), 1-19 he, w., wu, h., yan, g., akula, v. & shen, j. (2014). a novel social media competitive analytics framework with sentiment benchmarks, information and management52, 801–812. hollands, r. g. (2008). will the real smart city please stand up? intelligent, progressive or entrepreneurial? city, 12(3), 303-320 miah, s. j. (2004). accessibility improvement of multicultural educational web interface by using the user centred design (ucd) approach, in the proceedings of the 2004 informing science and it education joint conference, rockhampton, qld, australia, 25-28 june 2004. miah, s. j. (2008). an ontology based design environment for rural decision support, unpublished phd, griffith business school, griffith university, australia. miah, s.j., & ahmad, r. (2013). a cloud-based dss model for driver safety and monitoring on australian roads, international journal of emerging sciences, 1 (4), 634 miah, s.j., vu, hq, gammack, j. & mcgrath, gm. (2017). a big-data analytics method for tourist behaviour analysis, information and management, 54, 771-785 miah, s.j., & gammack, j. (2014). ensemble artifact design for context sensitive decision support, australasian journal of information systems,18 (2), 5-20 miah, s.j., vu, hq, & gammack, j. (2019). a big-data analytics method for capturing visitor activities and flows: the case of an island country, information technology and management, 20 (4), 203-221 sharda, r., delen, d., & turban, e. (2014). business intelligence and analytics: systems for decision support, 10th edition. pearson education limited, essex, england. thelwall, m., goriunova, o., vis, f., faulkner, s., burns, a. et al. (2016). chatting through pictures? a classification of images tweeted in one week in the uk and usa, journal of the association for information science and technology, 67 (11), 2575-2586. twitter. (2018). the streaming apis. retrieved may 2018, url: https://developer.twitter.com/en/docs australasian journal of information systems miah & vu 2020, vol 24, research note healthcare situation monitoring method 7 copyright: © 2020 miah & vu. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v24i0.2551 http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems amrollahi 2021, vol 25, selected papers from acis 2019 eliminating filter bubbles in social networks 1 a conceptual tool to eliminate filter bubbles in social networks alireza amrollahi macquarie business school ali.amrollahi@mq.edu.au abstract reliance on social media as a source of information has lead to several challenges, including the limitation of sources to viewers’ preferences and desires, also known as filter bubbles. the formation of filter bubbles is a known risk to democracy. it can bring negative consequences like polarisation of the society, users’ tendency to extremist viewpoints and the proliferation of fake news. previous studies have focused on specific aspects and paid less attention to a holistic approach for eliminating the notion. the current study, however, aims to propose a model for an integrated tool that assists users in avoiding filter bubbles in social networks. to this end, a systematic literature review has been undertaken, and initially, 571 papers in six top-ranked scientific databases have been identified. after excluding irrelevant studies and performing an in-depth analysis of the remaining papers, a classification of research studies is proposed. this classification is then used to introduce an overall architecture for an integrated tool that synthesises all previous studies and offers new features for avoiding filter bubbles. the study explains the components and features of the proposed architecture and concludes with a list of implications for the recommended tool. keywords: filter bubble, social networks, prescriptive study, information bubble. 1 introduction according to a recent survey, the majority (36%) of adults between 18 to 29 years in the usa consider social media their primary source of news (shearer, 2018). currently in australia, six out of 10 people are using facebook and an estimated 4.7 million australians are actively using twitter (cowling, 2019), with 17% of users trusting social media as their primary source of news (the australia institute, 2019). this reliance on social media and the way internet users seek information in this modern era have highlighted several interrelated risks and challenges. these challenges include the proliferation of fake news and misinformation, which has led to several disastrous incidents. examples of such events include misguided beliefs about the covid-19 outbreak and several violent incidents (qureshi et al., 2020). the spread of misinformation in social media or reflecting parts of the reality to convey an intended message have also been recognised as a method of recruitment for jihadist and extremist groups. the australian strategic policy institute identified the filter bubble as the fundamental phenomenon these groups use over social media to attract their supporters (winter, 2016). however, using filter bubbles is not limited to isis and islamist extremism. previous studies have found that homogenous groups are more likely to become extreme in their thinking (spohr, 2017). therefore, a similar approach (deliberately or unintentionally) is used by other online extremist groups who convert genuine and factual news to emotionally charged, politically biased news (rehm, 2017). through this process, an angry or just curious member of society will turn to someone with extremist viewpoints or a member of radical australasian journal of information systems amrollahi 2021, vol 25, selected papers from acis 2019 eliminating filter bubbles in social networks 2 groups. this is the case for the white racist groups in charge of the tragic incidents involving the christchurch mass shooting in new zealand (purtill, 2019). the notion of a filter bubble refers to the impact of our preferences and desires on the content and results we view on search engines, social media and other online platforms. this concept has been central to social media and internet research since it was developed by eli pariser (2011) and has been investigated by several scholars using various terms. some of the terms used in the literature are information bubble (liao & fu, 2013), [online] echo chamber (möller et al. 2018), the personal ecosystem of information (helberger, kleinen-von königslöw, & van der noll, 2015), partial information blindness (haim, graefe, & brosius, 2018) and information cocoons (sunstein, 2007). several undesirable impacts have been mentioned in the literature for filter bubbles. in particular, there is a potential risk of narrowing the information sources for online users and “pushing users into the psychological comfort zone of self-confirmation and risking polarisation on a societal level” (courtois, slechten, & coenen, 2018, p. 2008). increasing the risk of exposure to filter bubbles can create barriers to rational and diversified dialogue that is necessary for a democratic society. therefore, the filter bubble has been recognised as a risk to a well-functioning democracy in modern society (bozdag, 2015). studies in various areas have also investigated the impact of filter bubbles on the polarisation of online debates (flaxman, goel, & rao, 2016; seargeant & tagg, 2018) and extremism (costello et al., 2016; liao & fu 2013). to avoid these negative impacts, several studies in the literature have recommended solutions to understand, avoid and decrease the negative impacts of filter bubbles. these studies are focused on various topics, including quantification of the bubble in social networks (hannak et al., 2013), developing secondary apps (wood et al., 2018) and approaches to stay anonymous to avoid filter bubbles (ridgway, 2017). however, none of the studies in the literature, offer a comprehensive and integrated tool to help users avoid filter bubbles. this study aims to focus on this shortcoming by integrating and synthesising the available solutions in the architecture for an integrated tool that can work as a viable design basis for an information system. by searching six scientific databases with related phrases, we attempted to systematically review the prescriptive literature and suggest an architecture for an integrated tool. we investigated these studies based on their aim and approach to avoid filter bubbles, use of technology and the effectiveness of the approach. the results of this research can help future research to find possible gaps in the literature and provide practitioners with a better understanding of the tools available to them for avoiding filter bubbles. section 2 of this paper provides a background to the concept of the filter bubble and posits the current study within the body of research. section 3 introduces our methodology and approach for review and analysis of the literature. in section 4, the suggested architecture is presented in section 4, and results are discussed with possible implications for research and practice in sections 5 and 6. the paper concludes by explaining the contributions to the body of research. australasian journal of information systems amrollahi 2021, vol 25, selected papers from acis 2019 eliminating filter bubbles in social networks 3 2 research background 2.1 filter bubble although the notion of limiting sources of information to one’s preferences has been largely studied in areas such as media (jamieson & cappella, 2008) and psychology (nickerson, 1998), the application of this notion to online and social media came under the spotlight after the development of the term filter bubble. in his book and subsequent talks (pariser 2011), eli pariser (2011) focused on the impact of web personalisation and the way it affects our understanding about various topics when we refer to search engines as a source of knowledge. other studies then extended the concept to social media, as they discovered a similar mechanism is used in these platforms to recommend content and connections to people (van dijck & poell, 2013). the impact of social media on unpredicted outcomes of the 2016 usa election and the uk european union membership (brexit) referendum, brought the topic of fake news and filter bubbles to the fore (allcott & gentzkow, 2017). there are two main research streams on the filter bubble. the first stream (inspired by the work of praiser) mainly focuses on the impact of recommendation systems (divyaa, tamhane, & pervin, 2018; nguyen et al., 2014; sanz-cruzado & castells, 2018). these recommendation systems consider the user’s demographic information, history and search behaviour in suggesting new content by social media and search engines, creating a filter bubble for the information the user receives. this stream of research has been increasingly challenged by the second wave of studies that focuses on the role of social media users rather than recommendation system technologies (garrett, 2017; möller et al., 2018; resnick et al., 2013). this perspective is supported by empirical research, including a study on facebook content that found only 5%–8% of the content provided to people with various political viewpoints is based on their profile (bakshy, messing, & adamic, 2015). however, more recent events such as facebook–cambridge analytica data scandal (cadwalladr and graham-harrison 2018) and the widespread usage of bots on social networks to influence political campaigns (lazer et al. 2018) indicated that the role of these platforms in creating filter bubbles could not be totally ignored. among previous studies, we found review research by bozdag and van den hoven (2015). the study considers two different perspectives about democracy (namely, a liberal view of democracy and deliberative democracy) and introduces several software designs that have been employed to combat the filter bubble. the study then suggests design criteria against filter bubbles based on the two models of democracy, and concludes that the reviewed tools ‘do not define the filter bubble explicitly’ and most of them “are performed for us politics” (bozdag & van den hoven, 2015, p. 263). except for this comparison of tools, we did not find any research that went beyond algorithmic enhancements of recommendation systems and investigated possible social concepts leading to the formation of filter bubbles and how the system can deal with these factors. 2.2 impacts of filter bubbles the negative consequences associated with filter bubbles are extensively studied in the literature. some of these impacts can be directly associated with the filter bubble; for example, a decline in user trust (nagulendra & vassileva, 2016), limiting people’s access to information (valdez, kluge, & ziefle, 2018) and social fragmentation (möller et al., 2018). this impact is australasian journal of information systems amrollahi 2021, vol 25, selected papers from acis 2019 eliminating filter bubbles in social networks 4 particularly concerning when we consider how the content on social media is not subject to “significant third-party filtering, fact-checking, or editorial judgment” (allcott & gentzkow, 2017, p. 211). another negative consequence that has been cited more specifically in the literature is the polarisation of political discussions in social media when people are trapped in a bubble that prevents them from receiving outsider information (bakshy et al., 2015; foth et al., 2016; lahoti, garimella, & gionis, 2018; quraishi, fafalios, & herder, 2018; thonet et al., 2017; yang et al., 2017). previous literature has not found a significant relationship between exposure to an opposing political view and a change in people’s political opinion (bail et al., 2018). however, there are many studies that investigated the impact of filter bubbles on a commitment to a populist cause (postill, 2018) and avoidance of cross-referencing (van den bulck & moe, 2018), which creates a risk to the diversity of opinions and well-functioning democracy as a result (bozdag & van den hoven, 2015; dylko et al., 2018). conversely, filter bubbles can indirectly impact or result in the proliferation of recent challenges in online media, including fake news (bhatt et al., 2018; seargeant & tagg, 2018), as they “amplify any content, from genuine, factual news to emotionally charged, politically biased news” (rehm, 2017, p. 218). social science studies have found that homogenous groups are more likely to become extreme in their thinking (spohr, 2017), so the formation of these groups as a result of a filter bubble can lead to extremism. finally, the negative impacts of filter bubbles have been studied in specific areas. for example, taramigkou et al. (2013) investigated filter bubbles in music platforms and how they impact platform users’ taste in music. other researchers have studied the negative impacts of filter bubbles in areas such as online retail (matt et al., 2014) and the source of information financial analysts receive (shah et al., 2016). table 1 summarises the negative consequences of filter bubbles. consequence explanation a decline in user trust social media platforms provide no transparency about the mechanisms they use to filter the content shown to the user. this lack of transparency, in the long term, can result in the changes to the user’s usage experience and a decrease in their trust (nagulendra & vassileva, 2016). limiting people’s access to information users rely on a limited number of sources for news that are not subject to professional editorial policies and are often ideologically biased (allcott & gentzkow, 2017; lahoti et al., 2018). social fragmentation filter bubbles result in a self-confirming feedback loop for users who are subject to like-minded information. in the long term, this phenomenon will create communities that become increasingly polarised and fragmented (möller et al., 2018). the proliferation of fake news lack of access to factual news (from outside the bubble) results in the spread of emotionally charged and biased news within the bubble, the credibility of which will never be checked or questioned (rehm, 2017). extremism ideological polarisation through the filter bubbles in social media will help the spread of extremist viewpoints (spohr, 2017). table 1 – negative consequences of filter bubbles despite the significant breakthroughs in the literature, previous studies are less focused on the long-term effects of filter bubbles. for example, although there are several studies on the impact of social networks on extremism (awan, 2017; o’callaghan et al., 2013; spohr, 2017), australasian journal of information systems amrollahi 2021, vol 25, selected papers from acis 2019 eliminating filter bubbles in social networks 5 no empirical studies have investigated the impact of filter bubbles and human authority on the formation of extremist groups. 3 research method the current paper aims to suggest an architecture for an integrated tool that can deal with the problem of filter bubbles in social networks. to this end, a systematic literature review has been undertaken, and previous prescriptive studies on bursting filter bubbles have been reviewed. the following steps were used to conduct a systematic literature review, as suggested by kitchenham and charters (2007): (1) identifying resources, (2) study selection, (3) data extraction, (4) data synthesis, and (5) writing up the study as a report. these steps are frequently used in information system research to create rigour and transparency for literature review studies (amrollahi, ghapanchi, & talaei-khoei, 2014; paré et al., 2016). to follow these steps, we searched six scientific databases: science direct, scopus, proquest, acm digital library, association for information systems electronic library, and springer link. table 2 shows the final set of papers in each scientific database. database first set of papers final set of papers association for information systems electronic library 99 2 pro quest 119 5 science direct 19 8 scopus 147 32 springer link 146 4 acm digital library 41 20 total 571 71 table 2 – distribution of the first/final set of papers in different databases we searched for the following terms in title, keywords and abstracts, depending on the services offered by the relevant search engines: ‘filter bubble’ or ‘information bubble’ or ‘social recommendation systems’, ‘social personalisation’, ‘news gatekeepers’, ‘information gatekeepers’, ‘personalised filtering’, and ‘online echo chamber’. we started searching the related references with the above-mentioned set of keywords. our initial search resulted in 571 papers. we then reviewed these initial pool of research studies first through reading abstracts and then the papers in full-text. in each of these stages, irrelevant and duplicated references were excluded. this review ended in the final pool of 71 papers. table 3 illustrates the process through which we arrived at the final pool of research papers. round number of papers excluded number of papers remaining the initial list of papers 571 exclusion based on the title 107 464 exclusion based on abstract 317 147 removal of duplicate papers 9 138 exclusion based on full text (final list) 67 71 table 3different stages of inclusion / exclusion and number of papers in each round australasian journal of information systems amrollahi 2021, vol 25, selected papers from acis 2019 eliminating filter bubbles in social networks 6 for the current paper, we studied all articles in our final pool of papers with a focus on prescriptive research studies. the results of further analysis on the content of those papers are presented in the following sections. while the study focuses on the prescriptive approaches suggested in each article to prevent the formation of filter bubbles, there is less focus on the approach used to evaluate the rigour of research data or data analysis. instead, we have concentrated on the robustness of the proposed approach and whether it can be applied in practice. 4 results the final set of papers formed the basis of the results described below. these final papers were studied in detail, and their recommendations on how to overcome the filter bubble phenomenon have been investigated. the results of this study helped us to distinguish two different perspectives in the reviewed studies on the filter bubble. each of these perspectives is studied in further detail to understand and categorise their recommended approach. there was a group of papers that proposed approaches to identify the filter bubble. this category includes identification of the bubble, confirming its existence, and quantification of the impact of the bubble. however, the second category of research studies is directly focused on approaches to take users out of the filter bubble or, as we call it in this paper, burst the bubble. details about each category are explained below: 4.1 alert about the bubble research studies under this category are focused on the identification and evaluation of filter bubbles. filter identification studies are mainly focused on tools that check and alert users (in different ways) if they are trapped in a filter bubble. for example, nagulendra and vassileva (2014) proposed an interactive visualisation to enable the user to see the filtering. according to the authors, the tool has four goals: awareness, understanding, control of personalised filtering, and increasing the user’s trust in the system. the tool has been implemented and tested on an independent platform. the authors have extended this tool in their later work (nagulendra & vassileva, 2016) to include both content and agent (users) in visualisation. this goal to detect the bubble has been continued in future studies using different approaches, including diffusion of topics (content) (tk, george, & thomas, 2015), network theory (agents) (thonet et al., 2017) and machine learning (lahoti et al., 2018). another stream of research on alerting filter bubble goes one step beyond bubble identification and considers evaluation and quantification of filter bubbles. for example, hannak et al. (2013) developed a methodology for measuring personalisation in web search results. the proposed methodology compares different search results on google considering the attributes of the agent (user) who performs the search. also, matakos, terzi and tsaparas (2017) developed an index to measure the tendency of opinion polarisation in network communities that can lead to the development of filter bubbles. this measure considers the opinion of agents (users) and the structure of the network. despite these studies and a few in-progress studies, no study has created a comprehensive framework for measuring the filter bubble. the proposed frameworks are mainly focused on the agent and less focused on the content. finally, the proposed frameworks are less applied in real-time on social networks to show users the existence or significance of a filter bubble. australasian journal of information systems amrollahi 2021, vol 25, selected papers from acis 2019 eliminating filter bubbles in social networks 7 4.2 burst the bubble this category of studies explains the suggested approach to disable or decrease the negative impact of recommendation systems that create a filter bubble by exploring new ideas and diverse perspectives. the first stream of research in this category is organised around bypassing or changing algorithms. for example, ridgway (2017) suggests staying anonymous while being online as a solution to avoid potential filter bubbles. conversely, bozdag and van den hoven (2015) reviewed possible design criteria proposed for bursting filter bubbles. another stream of research under this category is focused on bursting the bubble by extending users’ awareness and encouraging them to explore different ideas. in this stream of research, again, the focus is on either new content or new agents. the majority of studies in our final pool are focused on viewing new content. among them, the work of taramigkou et al. (2013) is the first study we identified that proposed a methodology to help users explore new music genres outside their zone of interest. webberley, allen and whitaker (2016) also suggested an algorithm for avoiding filter bubbles by focusing on the re-twitting behaviour of users rather than the scope of the user’s social circle. finally, several studies have suggested applications to enable users to view new content outside their preference (linder et al., 2018; wood et al., 2018). however, recent studies have shifted the focus from content to the agent. quraishi et al. (2018) proposed a graph-partitioning method that exploits social interactions to represent different viewpoints in a social network. a qualitative evaluation of the proposed method is also presented based on implementing it on a dataset retrieved from twitter. also, sanz-cruzado and castells (2018) focused on contact recommendations and, based on the concept of weak connection, proposed an index for diversity. despite the diversity of research studies in this category, a lack of focus on the real-time application of the proposed methods in social networks is observable among all the studies. 5 suggested architecture based on the outcome of our review, we propose an architecture for an integrated tool that can be implemented in social networks (regardless of the content type) to help avoid the formation of filter bubbles. based on the outcome of our literature review, we propose two primary functions for this integrated tool: (1) alerting users about a potential filter bubble, and (2) bursting the bubble. under the alert component, the integrated tool first focuses on the identification of a filter bubble. many social network users are not aware of the filter bubble they are kept due to a lack of transparency in the algorithms used by social networks (bozdag & timmermans, 2011), so the tool needs to alert users. the cause of filter bubbles shifts from the recommendation algorithms to features enabling users to put themselves in a filter bubble (amrollahi & mcbride, 2019), so it is important to also inform the users about the consequences of their actions. for example, users should have the right to see how blocking or muting one specific user may result in missing out on a network of users and their perspectives. therefore, this bubble identification feature should be designed in connection with a bubble evaluation feature that assesses the significance of the bubble and also predicts future significance after certain events. australasian journal of information systems amrollahi 2021, vol 25, selected papers from acis 2019 eliminating filter bubbles in social networks 8 potential improvements in the recommendation systems have been suggested as a possible solution for bursting the bubble. however, in the current study, we have not considered this as a feature in the proposed architecture. recommendation systems are not included in the tool for two reasons: (1) the architecture is proposed independent to the social network, type of content they provide, and the recommendation algorithm they use; and (2) the recommended integrated tool is focused on not only the recommendation algorithm but also on social network facilities that enable users to build a bubble around them. the architecture of the proposed integrated tool is illustrated in figure 1. figure 1 – the proposed architecture for an integrated tool as illustrated in figure 1, the proposed tool will focus on both agent and content. it means that to identify filter bubbles (as part of the alert component), both the connections of and content viewed by the user should be investigated. also, the significance of the bubble should be evaluated by considering both the recommended content and recommended users. finally, the awareness of users should be increased by suggesting both novel (out of bubble) content and connections. the next section discusses the implications of our findings for both research and practice. 6 discussion the results of our review indicate that studying the filter bubble phenomenon is an interdisciplinary research area. our final pool of research studies includes work from areas ranging from information systems, information technology and management to political science, sociology, law and journalism. in the analysis of the related literature, we observed a focus on the content provided in social networks in few studies in various categories, while australasian journal of information systems amrollahi 2021, vol 25, selected papers from acis 2019 eliminating filter bubbles in social networks 9 many others were focused on the users posting the content (agent) and how they impact the formation of filter bubbles. in this study, we reviewed several negative consequences of filter bubbles. table 4 revisits these negative consequences and explains how the proposed architecture can help in avoiding these consequences. consequence how it can be avoided using the proposed architecture a decline in user trust the architecture provides some level of transparency about the algorithms used by social media platforms and the way they filter the content for users. this transparency should give a sense of control to users and improve their trust in the platform. limiting people’s access to information the architecture can show the user both the content and agents that have been filtered for them. this feature will potentially enable users to refer and visit those viewpoints. social fragmentation the architecture potentially presents different viewpoints to a user by increasing their awareness about ‘outside the bubble’ content. the proliferation of fake news the architecture permits more fact-checking and presents users with more ‘outside bubble’ content, which decreases the chance for fake news to spread within the bubble. extremism the architecture can potentially show the legitimacy of the content and increase a constructive discourse about news and information, preventing the risk of biased and emotionally charged news getting over-spread. table 4 – how the proposed architecture can help users avoid the negative consequences of filter bubbles the rest of this section explains the implications of the research for practice and research. 6.1 implications for practice although the current study is based on the available academic literature, the outcome is presented in a way that can be implemented in practice. designers and managers of social media platforms can benefit from the proposed architecture by seeing various avenues for improving their service or possible pitfalls in their social network that allows the formation of filter bubbles. for example, the architecture can direct them to various algorithms being used so they can present the significance of the filter bubble to their users. implementation of tools like this will help social media platforms to improve the transparency of their mechanism and subsequently improve customers’ trust in the platform (nagulendra and vassileva 2016). the results can also benefit governments and policy-makers to regulate the operation of social networks. governments should particularly focus on the role of filter bubbles in the spread of misinformation (including health-related misinformation) and investigate various approaches to increase community awareness about the filter bubbles. furthermore, the performance of social networks in the systematic development of filter bubbles should be monitored, and regulatory actions should be taken to deal with them. the burst and alert components of the proposed architecture can benefit governments in this regard. 6.2 implications for research although the proposed architecture in this study seeks to consider the various approaches for busting the filter bubble in the literature, it has not been put into practice as a solution in any social network. therefore, we recommend future studies to consider this architecture as a basis australasian journal of information systems amrollahi 2021, vol 25, selected papers from acis 2019 eliminating filter bubbles in social networks 10 and implement or evaluate it in social networks. most of the components of the proposed tool have already been developed in the previous prescriptive studies. however, there are features that require further development in future research. future studies can also focus on different parts of the proposed tool and develop algorithms for each part. specifically, the study proposes a real-time (predictive) bubble evaluation, which has not been directly studied and developed in previous studies. various algorithms used in other areas such as semantic analysis (hutchison, daigle, & george, 2018; müller et al., 2016) and big data analytics (weerasinghe et al., 2018) can be beneficial for future research to improve this feature. moreover, the development and integration of the proposed tool with social networks can be a subject for future research. conversely, the current study does not provide evidence for the effectiveness of the proposed components of the system as a whole. instead, it refers to the isolated studies used to suggest each component and various approaches they used to study the rigour and effectiveness of their research. hence, future research should focus on the effectiveness of the proposed approach and check whether people in such filter bubbles will break free when given the opportunity. both quantitative and qualitative methods can be used for the purpose of evaluation. finally, the results of the current study can inform future studies, especially future design research. in particular, future studies can use the proposed framework as a kernel theory to expand the results in the form of design theory (gregor & jones, 2007). the available literature on areas such as sociology and psychology can benefit the design theory. for example, the initiation of filter bubbles has been studied in the sociology literature (woon, 2018). also, previous psychological studies can be used to investigate the motives and antecedents of selfconfirming behaviour that leads to the formation of filter bubbles (mullainathan & washington, 2009) or attitude polarisation that strengthens them (corner, whitmarsh, & xenias, 2012). 7 conclusion filter bubbles are problematic consequences of modern media and social networks as they create barriers to the rational and diversified dialogue that is necessary for a democratic society. filter bubbles can develop in social networks as a result of users’ confirmation bias, the structure of the network, the algorithms incorporated within the network, or a mixture of all the above (bozdag et al. 2014; lr et al. 2018). this research, however, looks at filter bubbles as not only a product of recommendation algorithms in social networks but also a social issue that considers an active role for users building a filter bubble around them. in the current paper, we proposed an architecture for an integrated tool that can be incorporated into social networks to prevent the formation of filter bubbles around users. this tool is proposed based on a systematic review of the literature and classification of the studies according to their aim. the proposed components of the integrated tool cover both liberal (through alert component) and deliberative (through awareness component) models of democracy (bozdag & van den hoven, 2015). moreover, by considering the identification and evaluation of filter bubbles and increasing users’ awareness, the proposed architecture can be used against any type of filter bubble regardless of the source creating them. australasian journal of information systems amrollahi 2021, vol 25, selected papers from acis 2019 eliminating filter bubbles in social networks 11 the results of our review show a lack of empirical studies on the effectiveness of the proposed tools. even those studies that included their empirical results did so by using test data on developed hypothetical platforms. therefore, the application of the proposed methods in social networks should be considered in future studies. this improvement in future research will lead to a better evaluation of the effectiveness of the proposed methods, which is another shortcoming we identified in the literature. based on the results of our literature review, the components of an integrated tool are proposed in the form of an architecture map. unlike what is offered in the previous literature, our proposed method considers different perspectives on filter bubbles. these perspectives include a concurrent focus on alerting users to the formation of bubbles and bursting bubbles through increasing users’ awareness. the proposed integrated tool also has a dual focus on both content and agent, which cannot be found in the previous literature. references allcott, h., & gentzkow, m. (2017). social media and fake news in the 2016 election. journal of economic perspectives 31(2), 211–236. amrollahi, a., ghapanchi, a. h., & talaei-khoei, a. (2014). three decades of research on strategic information system plan development. communications of the association for information systems 34(1), 84. amrollahi, a., & mcbride, n. (2019). how to burst the bubble in social networks? in 24th uk academy for information systems international conference. oxford, uk. tk, a., george, k., & thomas, j. p. (2015). an empirical approach to detection of topic bubbles in tweets. 2015 ieee/acm 2nd international symposium on big data computing (bdc) (pp. 31–40). new york: ieee. awan, i. (2017). cyber-extremism: isis and the power of social media. society 54(2), 138–149. bail, c. a., argyle, l. p., brown, t. w., bumpus, j. p., chen, h., hunzaker, m. f., … volfovsky, a. (2018). exposure to opposing views on social media can increase political polarisation. proceedings of the national academy of sciences 115(37), 9216–9221. bakshy, e., messing, s., & adamic, l. a. (2015). exposure to ideologically diverse news and opinion on facebook. science 348(6239), 1130–1132. bhatt, s., joglekar, s., bano, s., & sastry, n. (2018). illuminating an ecosystem of partisan websites. arxiv preprint arxiv:1803.03576. bozdag, e., & timmermans, j. (2011). values in the filter bubble ethics of personalisation algorithms in cloud computing. 1st international workshop on values in design–building bridges between re, hci and ethics. lisbon, portugal. bozdag, e., gao, q., houben, g.-j., and warnier, m. (2014). does offline political segregation affect the filter bubble? an empirical analysis of information diversity for dutch and turkish twitter users. computers in human behavior 41, 405-415. bozdag, e., & van den hoven, j. (2015). breaking the filter bubble: democracy and design. ethics and information technology 17(4), 249–265. bozdag, v. e. (2015). bursting the filter bubble: democracy, design, and ethics (doctoral thesis). delft university of technology, delft, netherlands. australasian journal of information systems amrollahi 2021, vol 25, selected papers from acis 2019 eliminating filter bubbles in social networks 12 cadwalladr, c., and graham-harrison, e. (2018). revealed: 50 million facebook profiles harvested for cambridge analytica in major data breach. the guardian 17, 22-23. corner, a., whitmarsh, l., and xenias, d. (2012). uncertainty, scepticism and attitudes towards climate change: biased assimilation and attitude polarisation. climatic change 114(3–4), 463–478. costello, m., hawdon, j., ratliff, t., & grantham, t. (2016). who views online extremism? individual attributes leading to exposure. computers in human behavior 63, 311–320. courtois, c., slechten, l., & coenen, l. (2018). challenging google search filter bubbles in social and political information: disconforming evidence from a digital methods case study. telematics and informatics 35(7), 2006–2015. cowling, d. (2019). social media statistics australia—february 2019. retrieved from https://www.socialmedianews.com.au/social-media-statistics-australia-february-2019/ divyaa, l. r., tamhane, a., & pervin, n. (2018). a clustering based social matrix factorisation technique for personalised recommender systems. paper presented at the24th americas conference on information systems, new orleans, la. dylko, i., dolgov, i., hoffman, w., eckhart, n., molina, m., & aaziz, o. (2018). impact of customizability technology on political polarisation. journal of information technology & politics 15(1), 19–33. flaxman, s., goel, s., & rao, j. m. (2016). filter bubbles, echo chambers, and online news consumption. public opinion quarterly 80(s1), 298–320. foth, m., tomitsch, m., forlano, l., haeusler, m. h., & satchell, c. (2016). citizens breaking out of filter bubbles: urban screens as civic media. proceedings of the 5th acm international symposium on pervasive displays (pp. 140–147). new york: association for computing machinery. garrett, r. k. (2017). the ‘echo chamber’ distraction: disinformation campaigns are the problem, not audience fragmentation. journal of applied research in memory and cognition 6(4), 370–376. gregor, s., & jones, d. (2007). the anatomy of a design theory. journal of the association for information systems 8(5), 312–335. haim, m., graefe, a., & brosius, h.-b. (2018). burst of the filter bubble? effects of personalisation on the diversity of google news. digital journalism 6(3), 330–343. hannak, a., sapiezynski, p., molavi kakhki, a., krishnamurthy, b., lazer, d., mislove, a., & wilson, c. (2013). measuring personalisation of web search. proceedings of the 22nd international conference on world wide web (pp. 527-538). new york: association for computing machinery. helberger, n., kleinen-von königslöw, k., & van der noll, r. (2015). regulating the new information intermediaries as gatekeepers of information diversity. info 17(6), 50–71. hutchison, p. d., daigle, r. j., & george, b. (2018). application of latent semantic analysis in ais academic research. international journal of accounting information systems 31, 83–96. jamieson, k. h., & cappella, j. n. (2008). echo chamber: rush limbaugh and the conservative media establishment. oxford: oxford university press. australasian journal of information systems amrollahi 2021, vol 25, selected papers from acis 2019 eliminating filter bubbles in social networks 13 kitchenham, b. a., & charters, s. (2007). guidelines for performing systematic literature reviews in software engineering (ebse techinical report). durham, uk: university of durham. lahoti, p., garimella, k., & gionis, a. (2018). joint non-negative matrix factorisation for learning ideological leaning on twitter. proceedings of the 11th acm international conference on web search and data mining (pp. 351–359). new york: association for computer machinery. lazer, d. m., baum, m. a., benkler, y., berinsky, a. j., greenhill, k. m., menczer, f., metzger, m. j., nyhan, b., pennycook, g., and rothschild, d. (2018). the science of fake news, science 359(6380), 1094-1096. liao, q. v., & fu, w.-t. (2013). beyond the filter bubble: interactive effects of perceived threat and topic involvement on selective exposure to information. proceedings of the sigchi conference on human factors in computing systems (pp. 2359–2368). new york: association of computer machinery. linder, r., stacy, a. m., lupfer, n., kerne, a., & ragan, e. d. (2018). pop the feed filter bubble: making reddit social media a vr cityscape. 2018 ieee conference on virtual reality and 3d user interfaces (vr) (pp. 619-620). piscatawy, nj: institute of electrical and electronics engineers. lr, d., tamhane, a., and pervin, n. (2018). a clustering based social matrix factorization technique for personalized recommender systems, proceedings of the twenty-fourth americas conference on information systems, new orleans, usa. matakos, a., terzi, e., & tsaparas, p. (2017). measuring and moderating opinion polarisation in social networks. data mining and knowledge discovery 31(5), 1480–1505. matt, c., benlian, a., hess, t., & weiß, c. (2014). escaping from the filter bubble? the effects of novelty and serendipity on users’ evaluations of online recommendations. paper presented at the 35th international conference on information systems, auckland, nz. möller, j., trilling, d., helberger, n., & van es, b. (2018). do not blame it on the algorithm: an empirical assessment of multiple recommender systems and their impact on content diversity. information, communication & society 21(7), 959–977. mullainathan, s., & washington, e. (2009). sticking with your vote: cognitive dissonance and political attitudes. american economic journal: applied economics 1(1), 86–111. müller, o., schmiedel, t., gorbacheva, e., & vom brocke, j. (2016). towards a typology of business process management professionals: identifying patterns of competences through latent semantic analysis. enterprise information systems 10(1), 50–80. nagulendra, s., & vassileva, j. (2014). understanding and controlling the filter bubble through interactive visualisation: a user study. proceedings of the 25th acm conference on hypertext and social media (pp. 107–115). new york: association for computing machinery. nagulendra, s., & vassileva, j. (2016). providing awareness, explanation and control of personalised filtering in a social networking site. information systems frontiers 18(1), 145– 158. nguyen, t. t., hui, p.-m., harper, f. m., terveen, l., & konstan, j. a. (2014). exploring the filter bubble: the effect of using recommender systems on content diversity. proceedings australasian journal of information systems amrollahi 2021, vol 25, selected papers from acis 2019 eliminating filter bubbles in social networks 14 of the 23rd international conference on world wide web (pp. 677–686). new york: association for computing machinery. nickerson, r. s. (1998). confirmation bias: a ubiquitous phenomenon in many guises. review of general psychology 2(2), 175–220. o’callaghan, d., greene, d., conway, m., carthy, j., & cunningham, p. (2013). the extreme right filter bubble. arxiv preprint arxiv:1308.6149. paré, g., tate, m., johnstone, d., & kitsiou, s. (2016). contextualising the twin concepts of systematicity and transparency in information systems literature reviews. european journal of information systems 25(6), 493–508. pariser, e. (2011). the filter bubble: what the internet is hiding from you. london: penguin. pariser, e. 2011. “ted talks: eli pariser: beware online “filter bubbles”.” online video (https://www.ted.com/talks/eli_pariser_beware_online_filter_bubbles) postill, j. 2018. populism and social media: a global perspective. media, culture & society 40(5), 754–765. purtill, j. (2019). fuelled by a toxic, alt-right echo chamber, christchurch shooter’s views were celebrated online. abc news. retrieved from https://www.abc.net.au/triplej/programs/hack/christchurch-shooters-views-werecelebrated-online/10907056 quraishi, m., fafalios, p., & herder, e. (2018). viewpoint discovery and understanding in social networks. proceedings of the 10th acm conference on web science (pp. 47–56). new york: association for computing machinery. qureshi, i., bhatt, b., gupta, s., & tiwari, a. a. (2020). causes, symptoms and consequences of social media induced polarisation (smip). information systems journal. manuscript in preparation. rehm, g. (2017). an infrastructure for empowering internet users to handle fake news and other online media phenomena. international conference of the german society for computational linguistics and language technology (pp. 216–231). new york: springer. resnick, p., garrett, r. k., kriplean, t., munson, s. a., & stroud, n. j. (2013). bursting your (filter) bubble: strategies for promoting diverse exposure. proceedings of the 2013 conference on computer supported cooperative work companion (pp. 95–100). https://doi.org/10.1145/2441955.2441981 ridgway, r. (2017). against a personalisation of the self. ephemera: theory & politics in organization 17(2), 377–397. sanz-cruzado, j., & castells, p. (2018). enhancing structural diversity in social networks by recommending weak ties. proceedings of the 12th acm conference on recommender systems (pp. 233–241). new york: association for computer machinery. seargeant, p., & tagg, c. (2018). social media and the future of open debate: a user-oriented approach to facebook’s filter bubble conundrum. discourse, context & media 27, 41–48. shah, d., koneru, p., shah, p., & parimi, r. (2016). news recommendations at scale at bloomberg media: challenges and approaches. proceedings of the 10th acm conference on recommender systems (pp. 369–369). new york: association for computer machinery. australasian journal of information systems amrollahi 2021, vol 25, selected papers from acis 2019 eliminating filter bubbles in social networks 15 shearer, e. (2018, 10 december). social media outpaces print newspapers in the u.s. as a news source. pew research center. retrieved from https://www.pewresearch.org/facttank/2018/12/10/social-media-outpaces-print-newspapers-in-the-u-s-as-a-news-source/ spohr, d. (2017). fake news and ideological polarisation: filter bubbles and selective exposure on social media. business information review 34(3), 150–160. sunstein, c. (2007). republic. com 2.0. princeton, nj: princeton university press. taramigkou, m., bothos, e., christidis, k., apostolou, d., & mentzas, g. (2013). escape the bubble: guided exploration of music preferences for serendipity and novelty. proceedings of the 7th acm conference on recommender systems (pp. 335–338). new york: association for computer machinery. the australia institute. (2019). abc still australia’s most trusted news source. retrieved from http://www.tai.org.au/content/abc-still-australia-s-most-trusted-news-source thonet, t., cabanac, g., boughanem, m., & pinel-sauvagnat, k. (2017). users are known by the company they keep: topic models for viewpoint discovery in social networks. proceedings of the 2017 acm conference on information and knowledge management (pp. 87– 96). new york: association for computer machinery. valdez, a. c., kluge, j., & ziefle, m. (2018). elitism, trust, opinion leadership and politics in social protests in germany. energy research & social science, 43, 132–143. van den bulck, h., & moe, h. (2018). public service media, universality and personalisation through algorithms: mapping strategies and exploring dilemmas. media, culture & society 40(6), 875–892. van dijck, j., & poell, t. (2013). understanding social media logic. media and communication 1(1), 2–14. webberley, w. m., allen, s. m., & whitaker, r. m. (2016). retweeting beyond expectation: inferring interestingness in twitter. computer communications 73, 229–235. weerasinghe, k., pauleen, d., scahill, s., & taskin, n. (2018). development of a theoretical framework to investigate alignment of big data in healthcare through a social representation lens. australasian journal of information systems 22. https://doi.org/10.3127/ajis.v22i0.1617 winter, c. (2016). an integrated approach to islamic state recruitment. canberra: australian strategic policy institute. wood, g., long, k., feltwell, t., rowland, s., brooker, p., mahoney, j., … lawson, s. (2018). rethinking engagement with online news through social and visual co-annotation. proceedings of the 2018 chi conference on human factors in computing systems (p. 576). new york: association for computer machinery. woon, j. (2018). primaries and candidate polarisation: behavioral theory and experimental evidence. american political science review 112(4), 826–843. yang, m., wen, x., lin, y.-r., & deng, l. (2017). quantifying content polarisation on twitter. 2017 ieee 3rd international conference on collaboration and internet computing (cic) (pp. 299–308). new york: ieee. australasian journal of information systems amrollahi 2021, vol 25, selected papers from acis 2019 eliminating filter bubbles in social networks 16 copyright: © 2021 amrollahi. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v25i0.2867 http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems bunker, mills & vogel 2021, vol 25, selected papers from acis 2019 30th acis 2019 section 1 special section of invited papers from the 30th australasian conference on information systems deborah bunker the university of sydney, australia d.bunker@sydney.edu.au annette mills university of canterbury, new zealand annette.mills@canterbury.ac.nz doug vogel harbin institute of technology, china isdoug@hit.edu.cn as program chairs of the 30th australasian conference on information systems (acis) we were privileged to work with a wonderful group of best paper authors as well as a select panel of reviewers for the australasian journal of information systems (ajis 2019) special section. acis2019 was the 30th anniversary of the australasian information systems conference and as such has symbolic significance for our community. these papers reflect the diversity of ideas and inclusiveness within our community and very high standards of academic contribution. these papers cover a range of topics which focus on: tool development to eliminate filter bubbles in social networks; understanding how peer influence impacts teenager’s adoption of is security guidelines; applying a social constructivist perspective on digital transformation; using decision support systems in the context of cyber-physical systems; and using an intelligent agent-based is framework for dynamic demand environments. for this special section, acis 2019 best papers were initially improved and extended by their authors and then double blind reviewed by our review panel members. authors were then allowed the necessary time to respond and further improve their papers. we sincerely thank our reviewer panel for their time and expertise to assist authors during what was a particularly difficult and uncertain period at the onset of the covid-19 pandemic. we are sure that you will agree that these papers represent some of the best of what the australasian information systems community has to contribute to knowledge development in information systems. deborah bunker, annette mills and doug vogel ajis special section co-editors copyright: © 2021 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems bunker, mills & vogel 2021, vol 25, selected papers from acis 2019 30th acis 2019 section 2 doi: https://doi.org/10.3127/ajis.v25i0.3449 https://doi.org/10.3127/ajis.v25i0.3449 paper title (paper title style) australasian journal of information systems hinton, wood, singh & reiners 2019, vol 23, research article enterprise gamification systems and employment legislation 1 enterprise gamification systems and employment legislation: a systematic literature review sean hinton spark new zealand seanahinton@gmail.com lincoln wood otago university harminder singh auckland university of technology torsten reiners curtin university abstract a recent innovation in employee motivation systems is the introduction of ‘gamification’, which refers to the use of game design mechanics and principles to influence behaviour to enhance staff motivation and engagement. enterprise gamification systems aggravate the differences in information availability between employers and employees, and employees who may be forced to adopt such systems may be placed under stress, worsening employment relationships in the workplace. therefore, this research examines the potential legal implications of gamified employee motivation systems. this study undertook a systematic review of enterprise gamification and then used thematic analysis coupled with a review of legislation to examine whether gamification in workplaces meets the legal obligations of employers under their ‘duty of good faith’ in the new zealand context. we find that carefully designed enterprise gamification systems should provide sufficient information and clarity for employees and support positive employment relationships. deployments of enterprise gamification systems should be carefully planned with employee consultation and feedback supporting the introduction of an enterprise gamification system. future research should look beyond the ‘good faith’ obligation and examine the relationship between gamification systems and the law on personal grievances. keywords gamification, performance management, legal, employment relationship, good faith, game mechanics 1 introduction performance monitoring of employees “the observation, examination, and/or recording of employee work-related behaviors” (stanton, 2000, p. 87) – is used to motivate employees so as to increase their work output, reduce costs and enhance customer satisfaction (alder, 1998; chalykoff and kochan, 1989; jeske & santuzzi, 2015; annakis et al., 2011). employees too are aware that some data about them can be collected and used to evaluate whether they are meeting key performance indicators (kpis) (e.g., monthly sales targets), and thus be rewarded with financial rewards, tangible services or other benefits (herzberg, 1966; kalleberg and loscocco, 1983; mottaz, 1988). setting kpis and monitoring whether employees attain these requirements could be termed a ‘passive’ approach to performance monitoring. traditionally, the transactional nature of the enterprise resource planning systems which dominate australasian journal of information systems hinton, wood, singh & reiners 2019, vol 23, research article enterprise gamification systems and employment legislation 2 workplaces has allowed operational performance data to be captured in real-time and compared to kpis (alder, 1998). such systems have thus emphasised efficiency gains. contemporary information systems have afforded new uses of data that push beyond such backwards-looking passive systems, towards more active monitoring and adjustment of circumstances in a way that encourages employee participation. the dominant form of these new employee performance monitoring systems is ‘enterprise gamification systems’. such systems use game-based principles (deterding, dixon, khaled, and nacke, 2011a) to shift employee behaviour (reiners and wood, 2015) so that the outcomes transcend efficiency improvements and may influence employee behaviour in more subtle ways compared to existing performance management systems (callan, bauer, and landers, 2015). enterprise gamification may also be more effective as it enables organisations to stimulate individuals’ intrinsic motivation as well (herzig, ameling, and schill, 2015). this shift towards the use of game design principles (sicart, 2008) in performance monitoring systems is fairly recent, and with early research still focusing on how to develop kpis connected to relevant activities (conley and donaldson, 2015). most studies on performance monitoring systems examine traditional performance monitoring systems, which are relatively transparent and passive, often logging time spent on particular tasks, or when certain activities are taken (jeske and santuzzi, 2015). in such systems, data on a limited number of performance indicators is collected and analysed, with focus given to performance against a minimum standard for relatively simple kpis (e.g., minimum calls per hour) with the ability to connect this to extrinsic motivators; e.g., financial rewards or bonuses (wickham and collins, 2004). emerging enterprise gamification systems differ from these systems in two ways. first, employees can be closely monitored, with more data collected and analysed in greater detail, and performance data can be relayed to employees instantly (e.g., through leaderboards). second, such systems may encourage employees to undertake activities that, while being playful or game-like, provide employers more oversight and control of employees’ actions. the move towards the use of gamification for employee management has therefore been a shift away from the relatively transparent monitoring associated with traditional passive systems to an arguably more ‘subversive’ and manipulative system, where the ‘play’ is merely symbolic. the game design principles on which enterprise gamification systems are based are derived from games, where participation is clearly voluntary. however, this is not the case when game-like activities are used for performance monitoring. further, gamification systems can disrupt the balance in workplace relationships, as they provide employers with overwhelmingly greater informational advantages above and beyond those conventionally available to employees. the increase in the use of systems based on game-like activities for monitoring and manipulating behaviours in the workplace, therefore, is an area that professionals must better understand, because these systems may potentially change the balance of power between employees and managers. employee motivation systems have been used in the past to influence behaviour, but the addition of game-like activities to them may, while increasing their attractiveness and ease of use, mask their potential to disturb the order in a workplace. however, even though employee motivation systems may now include gamification features, the legal requirements guiding their use are likely to be similar to what they were previously. therefore, this paper’s aim is to explore how game-like structures can make employee australasian journal of information systems hinton, wood, singh & reiners 2019, vol 23, research article enterprise gamification systems and employment legislation 3 performance management more effective and what their implications are in workplaces. to examine this issue, this study asks two research questions: rq1. given that enterprise gamification systems change the employer-employee relationship, are there any legal considerations that gamification system designers must take into account? rq2. how should gamification principles be deployed in enterprise gamification systems? the current study was designed to address these questions by undertaking a systematic review in the new zealand context. we first reviewed the literature on gamification, and next examined new zealand legal cases pertaining to employment law and the concept of ‘good faith’. these two sets of literature were thematically analysed to surface relevant concepts and relationships between the themes of ‘good faith’ in employment and gamification systems. the paper concludes by outlining considerations for designers of enterprise gamification systems and managers who use them. 2 background and related work to address the study objectives, a clearer theoretical background was required. therefore, we searched for literature that provided an explanation of call centre work and job satisfaction, in addition to work more generally on gamification. the gamification literature we searched for focused on motivation and enterprise gamification (that is, influencing employees rather than customers) who have no choice to participate. finally, we examined the role of employment law to understand the implications of these changing technologies in the workplace. 2.1 motivation motivating employees is necessary when using human resources to carry out tasks. motivation is defined as a force that energises behaviour, gives direction, and supports the tendency to persist (bartol and martin, 1998). employee motivation levels and expectations can be changed by designing systems to encourage specific behaviours. formally, motivation is usually manifested in workplaces in the form of performance monitoring systems which observe, examine, and/or record the work-related behaviours of employees (stanton, 2000, p. 87); they are championed as improving work output, reducing costs and ensuring customer satisfaction (alder, 1998; chalykoff and kochan, 1989). the nature and pervasiveness of monitoring solutions in organisations plays an important and influential role in the long-term wellbeing of an employee (holman, 2002). critics of performance monitoring, however, claim the practice is intrinsically threatening to workers as the information disparity between the employer and the employee has the potential to impact future remuneration or negatively influence social workplace relationships (alder, 1998). smith, carayon, sanders, lim, and legrande (1992) argue that the simple act of being monitored can increase an employee’s emotional labour, resulting in higher degrees of depression, boredom, fatigue and anxiety. for example, greater dissatisfaction is observed among monitored employees (aiello and kolb, 1995). conversely, in some instances, performance monitoring offers immediate feedback on employee performance and can result in positive job satisfaction (chalykoff and kochan, 1989; jeske & santuzzi, 2015). however, it is important to consider how feedback is delivered, the intensity of monitoring, and its transparency among other employees (carayon, 1994); positive influences on job satisfaction may drive not from the monitoring but from improved managerial practices (annakis, lobo, australasian journal of information systems hinton, wood, singh & reiners 2019, vol 23, research article enterprise gamification systems and employment legislation 4 & pillay, 2011). this may also be relative to past work experience; where work involved greater autonomy previously, individuals may find that constant performance monitoring can generate negative feelings and acceptance (jeske and santuzzi, 2015). feedback from performance monitoring systems also carries elevated importance to employees beyond personal growth, as meeting key performance indicators, e.g., monthly sales targets, can lead to financial rewards or benefits (herzberg, 1966; kalleberg and loscocco, 1983; mottaz, 1988). therefore, managers must comprehend which rewards motivate employees, and how individual and organisational objectives may be balanced (dunham, grube, and castañeda, 1994). it is in this context that the emerging technologies of pervasive ict systems and new sociomechanic approaches are colliding in the enterprise environment. one outcome is the rise of ‘gamification’ systems, which adapt existing work processes by adding game-like mechanics so as to modify behaviour to achieve performanceor motivation-focused outcomes. paradoxically, this use of game-like mechanics may lead to non-playful and non-enjoyable outcomes at work (callan et al., 2015). the gamification of processes encapsulates a form of electronic performance monitoring, and may bring about the “potential negative consequences of lower job satisfaction, affective commitment, lower perceived control and lower self-efficacy among employees” (jeske and santuzzi, 2015, p. 76). 2.2 the case of call centres the call centre industry has highlighted the nature of the motivational problem as a fundamental trade-off between quantity and quality (of assistance) for employees (de ruyter, wetzels, and feinberg, 2001), and the selection, implementation and use of management technology to facilitate the physical concentration and monitoring of staff (mehrotra, 1997). wallace, eagleson, and waldersee (2000) argue that call centre roles which focus on providing efficient and standardised performance in large quantities offer employees a transactional relationship with an organisation. these employees are tasked with resolving high call volumes comprised of repetitive and straightforward resolution processes. in these environments, management tends to use quantifiable indicators for performance appraisal (wallace et al., 2000), supporting high volumes of service using scripts or flowcharts. companies benefit as they can provide accurate and timely feedback on employee performance using performance monitoring technologies (grant and higgins, 1991). however, the adverse effects of these practices on employee stress and job satisfaction are often overlooked. in part, this negligence is due to the obligations of managers to pursue low-cost operations through high productivity, and a prevailing attitude that churning through burnt-out staff will increase revenue (lewig and dollard, 2003). as a result, new employees, recruited in batches, are routinely favoured over existing staff; they provide fresh, enthusiastic, motivated customer service at low cost (wallace et al., 2000). volume-focused companies often take this approach, cognisant that they may be obligated to effectively forego three to four months wages annually in lost productivity for each new employee recruited (holman, 2002). deery and kinnie (2002) argue that this mentality contributes to a growing perception that call centre jobs are ‘deadend’ and are characterised by “low status, poor pay and few career prospects” (p. 3). misalignment of organisational objectives and the motivational requirements of the individual call agent (wallace et al., 2000) can develop from the outset of the employment relationship, as companies actively seek employees with high levels of intrinsic motivation to provide australasian journal of information systems hinton, wood, singh & reiners 2019, vol 23, research article enterprise gamification systems and employment legislation 5 quality services, but go on to overwhelm them with high call quantity and limited autonomy. the dissonance created by these objectives emerges as staff struggle to resolve dynamic complaint calls when constrained by pre-scripted resolutions. bandura, freeman, and lightsey (1999) argue that fostering dissonance and relying on ‘surface acting’ (a mental state of projecting, yet not experiencing, emotions) can lower an employee’s sense of self-worth, leading to increased stress, and in extreme cases, symptoms of depression. lewig and dollard (2003) support this, noting emotional dissonance may “lead to lowered self-esteem, depression, cynicism, and alienation from work” (p. 368). actions which inhibit empowerment directly influence employee stress and diminish motivation (de ruyter et al., 2001). chiles and zorn (1995) note employment relationship empowerment consists of competence and authority. competence is belief in ability to perform tasks proficiently; authority is the degree of autonomy when completing a task. gamification research has largely overlooked these misalignments or has been limited to speculation around hypothetical employment scenarios (callan et al., 2015). 2.3 gamification for behavioural change some claim that gamification is not actually a new expression but one that simply encompasses many other parallel definitions; e.g., ‘funware’, ‘behavioural games’ and ‘surveillance entertainment’ (deterding et al., 2011a). the concepts of ‘game’ and ‘play’ are distinct; games use rules system, often encourage competition, and have determined goals or objectives; in contrast, play is often experiential, is less guided or focused, and is often fun (deterding et al., 2011a; tekinbaş and zimmerman, 2004). gamification, as an academic term, has more to do with the concepts of “gamefulness” than that of “playfulness” through the application of game elements in non-game contexts (deterding et al., 2011a). this distinction is crucial, as methods of incorporating games into work environments have been attempted under labels including ‘serious games’, ‘alternate reality games’ and ‘simulation games’ (reiners, wood, gregory, and teräs, 2015). in each, the underlying game provides a context for successful integration, something gamification does not require. ‘enterprise gamification’ is the emerging field of gamification systems deployed to manage day-to-day activities in the enterprise, often with a focus on efficiencies (kumar, 2013). gamification drives behavioural change (reiners and wood, 2015, p. vi) through ‘gamemechanics’ while improving users’ experience and increasing engagement (deterding, sicart, nacke, o'hara, and dixon, 2011b). these mechanics are the structures and methods game designers use in traditional games to induce behaviour to achieve objectives. reward systems (e.g., points, badges and leaderboards) are considered fundamental to gamified systems. data is collected on what happens and when, so that a form of ‘surveillance’ takes place. this may be legitimised by reference to a data-driven approach that improves fairness, increases efficiency, and rewards workers that achieve required outcomes; however, some employees may fear that the system fails to meet their expectations in the workplace and does not respect their personal data (i.e., the system might go ‘too far’) (ball and margulis, 2011, pp. 122-123). 2.4 user motivation through gamification much gamification research is underpinned by self-determination theory (sdt) (ryan and deci, 2000), which focuses on ‘growth tendencies’ and psychological needs. sdt posits that two motivational principles determine behaviour. first, intrinsic motivation is driven by activities that induce interest in a user through an inherent drive to complete the task or to seek australasian journal of information systems hinton, wood, singh & reiners 2019, vol 23, research article enterprise gamification systems and employment legislation 6 out and extend one’s capacities. second, extrinsic motivation is driven by external sources (e.g., rewards) which drive task completion. while many employment schemes currently use a number of extrinsic motivators, gamification processes often focus on harnessing the power of intrinsically motivated activities and deploying them in non-game contexts. cognitive evaluation theory (cet) offers three insights into how these systems perpetuate engagement. first, they offer a sense of autonomy, usually highest when participants are completing activities for their own interest; however, increasing choice and informational feedback along with non-controlling instructions can all elevate this sensation (ryan, rigby, and przybylski, 2006). second, they reward competence, a need for challenge and feelings of effectiveness (gagné and deci, 2005). third, they promote relatedness, experienced when a person feels connected with others (la guardia, ryan, couchman, and deci, 2000). human motivation is principally driven by intrinsic values and other crucial factors include mastery and purpose; in contrast, extrinsic rewards are limited relative to intrinsic motivators (pink, 2010). such attempts often build on sharing and connectedness between users and a sense of interest in the activity. 2.5 examples of enterprise gamification an enterprise gamification implementation of particular significance was undertaken in disneyland hotel in anaheim (lopez, 2011) to increase employee motivation, work output, and engagement. digital screens displayed employee’s work speeds on a leaderboard. almost immediately, the opposite effect was observed: some workers insisted that publicly displaying ‘work efficiency’ generated undesired competition among staff, with those less proficient fearing that their performance would displease management. another concern raised by the employees was their behavioural changes around bathroom breaks; to maintain a competitive edge over others on the virtual leaderboards, some staff felt it important to forgo these otherwise basic human necessities of bathroom breaks. it was blasted as an “electronic whip”. one commentator has claimed that ‘gamification’ is ‘exploitationware’. bogost (2011, para. 58) asserts that “[g]amification replaces these real, functional, two-way relationships with dysfunctional perversions of relationships. organizations ask for loyalty, but they reciprocate that loyalty with shams, counterfeit incentives that neither provide value nor require investment.” gamification principles can change dynamics in employment relationships in ways that may not only be ‘offensive’ to employees, but which may fall foul of employment law. disneyland hotel’s gamified system raises some interesting questions about the relationship between employment laws and motivational tools embedded with the intent to manipulate behaviour. gamification systems may seek to manipulate employees with less clear-cut incentives and carefully designed intrinsic motivators. just as disney’s anaheim hotel implementation highlights several unexamined realities in the context of employment relationships, we ask what exactly is the scope of an employer’s obligation to provide a reasonable gamification implementation? do enterprise gamification systems introduce new commitments to the employment agreement that parties must adhere to in order to fulfil their statutory obligations? to answer these questions, we turned to new zealand legislation and common law that is relevant to employment. australasian journal of information systems hinton, wood, singh & reiners 2019, vol 23, research article enterprise gamification systems and employment legislation 7 2.6 scope of law within the new zealand1 judicial system, significant legislative acts expressly mandate the provision of ‘good faith’. while good faith principles are practiced in other jurisdictions (e.g., australia and canada), this mostly relies upon the implied, and not express, covenant of good faith. a global examination of ‘bona fides’ would therefore be a sizeable challenge beyond the scope of this paper. it should be noted that, like new zealand, many international legal systems are based on the same foundations and principles as those found in the united kingdom’s legal structure. therefore, many issues and concerns are likely to translate when the applicable judicial systems are similar in nature. the new zealand-based obligations of ‘duty of good faith’ under section 4 of the employment relations act 2000 (era) ("employment relations act," 2000) were examined. we believe that this provides a sensible starting point for employers and employees in relation to enterprise gamification systems. 3 new zealand employment law 3.1 good faith duty the new zealand employment relations act 2000 ("employment relations act," 2000) is the principal piece of legislation relating to employment, covering all necessary legal obligations between business owners and employees who enter into an employment agreement; e.g., everything from the formation of the employment agreement, to trial provisions, and acceptable conduct from either party in the course of work duties. the goal stated in section 3 is “to build productive employment relationships through the promotion of good faith in all aspects of the employment environment and of the employment relationship” (emphasis added). thus, good faith is at the heart of the legislation and influences virtually every provision in the act. while good faith is loosely defined in legislation, it broadly means “the duty to act in good faith is wider in scope than the implied mutual obligations of trust and confidence” (s 4(1a)(a)), while also requiring parties to an employment relationship to be “active and constructive in establishing and maintaining a productive employment relationship” (s 4(1a)(a)) ("employment relations act," 2000). further refinement through common law occurred where it was stated that “good faith has more to do with notions of honesty, frankness and what lawyers call ‘bona fides’ rather than adherence to legal rules” (national distribution union inc v carter holt harvey ltd, 2001). special reference was made to section 4(1)(b) of the era, which outlines misleading and deceiving behaviour, clarifying that good faith is more “the spirit than the letter of the law”. while the concept of good faith has been largely left to the new zealand judicial system to define, assistance from similar legislations (e.g., from canada and the united states) has been relied upon (nz amalgamated engineering etc union inc v carter holt harvey ltd, 2002). 1 focusing on new zealand law only is a limitation of the study, but we feel that it is an appropriate starting place because it is the legal system that is most familiar to us. we hope other researchers will build on our findings and extend the analysis of this topic to other legal systems, from both common and civil law traditions. australasian journal of information systems hinton, wood, singh & reiners 2019, vol 23, research article enterprise gamification systems and employment legislation 8 3.2 employment relationship obligations parties subject to an employment agreement must comprehend what constitutes “active and constructive” behaviour. section 4(1a)(b) of the era mandates that parties must be considered to be “responsive and communicative”; section 4(1a)(c) declares that during the process of making decisions related to the continuation of an employment relationship, employers are obligated to provide employees with access to relevant information and an opportunity to comment on it. this obligation engrosses the “timely provision of useful information” (baguley v coutts cars ltd 2000). furthermore, an employer was demonstrated to have an obligation to provide relevant information pertaining to the specifics of a complaint about an employee’s behaviour beyond a generic disciplinary form (gaut v bp oil new zealand ltd, 2011). section 4(1b) of the era does place limitations on the information employers must provide when considering the continuation of employment; specifically, information may be withheld to comply with statutory requirements of confidentiality or where the risk exists that an employer may be unreasonably prejudiced. equally, cautionary information relating to an employee’s on-going employment must not be significantly different than what is used during the decision (gwilt v briggs and stratton new zealand ltd 2011). this was evident where one employer failed to provide relevant and sufficient information relating to redundancies (nee v tlnz auckland ltd 2006); and another employer merely told a staff member of allegations instead of providing the original material the allegations were based on (lawless v comvita new zealand ltd 2005). if there is perceived employee confusion regarding the employment termination, employers must proactively correct any misguided beliefs; e.g., where an employee was informed by a third party his job had ended and the employer failed to make a correction (new zealand cards ltd v ramsay, 2012). should an employee terminate their employment agreement as a result of an inaction against abusive behaviour, it is likely that a personal grievance raised for constructive dismissal will be successful (shaw v schering-plough animal health ltd, 2013). 3.3 balancing the power bias in employment relationships while it is clear there are responsibilities from the employer to the employee, employees also have responsibility to be ‘communicative and responsive’; e.g., an employee that failed to adequately respond to inquiries about sick leave after being recognised at a sporting event during the time off (taiapa v te runanga o turanganuia a kiwa trust, 2013). if an employee suspects trouble or potential issues they must act in a responsive and communicative fashion; e.g., a staff member accessed an employer’s computer to verify their personal suspicions about the situation and later refused to comment on the information gathered (o’hagan v waitomo adventures ltd, 2012). 3.4 misleading and deceiving another significant obligation is under section 4(1) relating to behaviour “likely to mislead or deceive”. again, while no precise definition is provided it was later defined as “real risk, a substantial risk, or something that might well happen” (r v pier 1987); alternatively, it could be held that it was “real or not remote chance or possibility regardless of whether it is less or more than 50 percent” (global sportsman ltd v mirror newspapers, 1984). 3.5 the case of disneyland hotel in the disneyland hotel scenario mentioned above, the first and most crucial issue to consider is whether the three major components of good faith (viz., being “active and constructive”, australasian journal of information systems hinton, wood, singh & reiners 2019, vol 23, research article enterprise gamification systems and employment legislation 9 “communicative and responsive” and “likely to mislead or deceive”) are being adhered to by all parties. we can deduce that a degree of communication breakdown occurred. both parties would be obligated to be ‘active and constructive’; therefore, appropriate discussion should occur and opportunities to raise concerns must be ensured. in many respects, a substantial failure to be ‘active and constructive’ may have already taken place. this could be through a lack of direct communication between the involved parties, and involvement of third-party media outlets that reported on employee concerns. a breakdown may have been exacerbated if the system was designed to hamper communication through facilitation of a one-way communication channel, or by disallowing a ‘break’ in being monitored during the work-day when staff wanted to voice concerns. second, both parties have an obligation to be ‘responsive and communicative’. while this applies to the discussion with media outlets, it also extends to other aspects of the work environment. should employees be placed in a situation where their employment is jeopardised, there is a reasonable expectation that information pertaining to their ongoing employment be conveyed in a way that is manageable and comprehensible, while providing a right of response. however, many gamification systems convey quantifiable performance metrics around ‘efficiency’ (e.g., time to complete tasks), but may neglect to convey qualitative measures (e.g., quality of outcome). here, the measurement of efficiency during a ‘review of work’ is likely to be a mixture of both subjective and objective requirements, of which each party must be able to comment on in a way that is consistent with their legal obligations to be responsive and communicative. third, both parties must refrain from behaviour likely to “mislead or deceive”. while inherently subjective, it will be viewed by judicial systems on a case-by-case basis; however, as in the global sportsman ltd v mirror instance, any misleading behaviour that holds a “real or not remote chance” of coming to fruition will likely lead to a breach of good faith. in disney anaheim’s case, parties will need to monitor the performance systems to ensure they provide accurate data and that the informational displays adequately describe reported statistics. publicly displayed performance monitoring must not be presented in a way that engenders an inaccurate impression to those monitored. this process is also important in other areas of employment law which mandate the fair treatment of all employees; e.g., statutory requirements around unjustified workplace disadvantages (department of labour, 2006). from this case, we hope that it is clear that a poorly designed gamification system could present legal difficulties for the employer. this study thus asks: what are the key gamification principles that the system designer must consider and incorporate in a way to ensure that the system does not break ‘good faith’ principles? 4 research methodology addressing the research questions requires an overview of two bodies of literature: gamification and a body of legal cases. first, a structured analysis of the issues was undertaken by identifying relevant themes from the gamification literature that may pertain to employment. second, we identified the relevant themes from important legal cases. these themes were then compared and possible areas of overlap identified. australasian journal of information systems hinton, wood, singh & reiners 2019, vol 23, research article enterprise gamification systems and employment legislation 10 4.1 sampling relevant gamification literature and legal cases were selected using a non-probabilistic, purposive sampling (kumar, 2005) approach that allowed us focus on specific characteristics. this was particularly relevant as robust research on gamification is limited. academic experts guided the selection of material and to help identify similar or relevant materials – a form of snowball or convenience sampling (hallberg, 2006). the lead author used the proquest academic database with keyword searches (details in appendix a), using terms such as gamification, motivational psychology, duty of good faith, and gamified system to find initial pieces of literature. the initial list was reviewed by the research team and additions were suggested. when initial items were identified, the lead author checked the citations for these articles to find related items, following the process outlined in kumar (2005), and this list was checked by the research team. as gamification is a relatively new topic, we included all articles identified and so the sample included both journal articles and conference papers. our inclusion criteria required that the articles focus specifically on gamification and gamified systems; we excluded articles that focused on the related but different topics of ‘serious games’, ‘alternate reality games’ and ‘simulation games’ (reiners, wood, gregory, and teräs, 2015). 4.2 data analysis procedures a thematic content analysis approach was used identify persistent patterns and themes while also uncovering previously unknown links between the subjects (birks and mills, 2011). this involved identifying recurring patterns through an inductive analytic process. as more publications were added to the sample, further thematic analysis occurred; eventually, on the addition of the last articles, no new themes were emerging and existing themes were increasingly populated; this was treated as being a point of theoretical sufficiency and indicated that the sampling phase was completed. this occurred separately for each of the two bodies of literature; viz., gamification and good faith in employment law. first, each item was read to provide an understanding of content. the items were viewed holistically and care was taken to preserve the flow of each document by avoiding truncation of key passages of text. second, texts were carefully examined with annotations placed where keywords and potential themes emerged. the preliminary themes contained mainly descriptive underlying principles from the text. this decontextualised the data from the literature to identify previously unnoticed information, aided by a line-by-line analysis rather than holistic viewing. third, the descriptive themes where collated into basic categories to illuminate other significant themes. the formation and iterative analysis allowed concepts to be re-analysed to determine how exactly each underlying theme was assigned to the specific categories; this process assists in highlighting meaning from the information and aids in defining the evolving themes. fourth, the original texts were then approached with an analysis tasked with recontextualising the data with the assistance of the emerging themes a process known as ‘axial coding’ (corbin and strauss, 1990). 5 results after initial sampling and analysis, two thematic category tables were created to represent the subsequent themes emerging from the 13 pieces of literature in each category. we are not claiming that these themes are the most important overall, or that they are relevant in all australasian journal of information systems hinton, wood, singh & reiners 2019, vol 23, research article enterprise gamification systems and employment legislation 11 situations; however, we note these as they are within the scope and interest of an exploratory study (appendix b). we first present each of the gamification and legal themes before we discuss how they interact such that a gamified employee motivation system may change the employer-employee relationship. 5.1 gamification themes personalised experience was perceived by both researchers and businesses to be an important aspect of an individualised gamification system. this may be crucial to the success of any system designed for long-term use; users are influenced by different motivational paradigms, which should be represented in any gamification implementations tasked with driving user behaviour (paharia, 2013; hamari et al., 2014). the exception is a system designed with the express intent of driving one-off task completion through the use of extrinsic rewards. increased performance feedback was closely connected to personalised experience, partially due to the requirements of motivational psychology (ryan, rigby, & przybylski, 2006) and the need to understand how well one is performing at a given task, which may influence feelings of effectiveness (gagné & deci, 2005), along with business’ desire to measure performance. this theme is significant within the context of legal implications under new zealand employment legislation. raised enterprise prominence many firms indicated they wished for more meaningful engagement with their staff to better achieve business objectives (deterding, sicart, nacke, o'hara, & dixon, 2011). of particular note was the understanding each of the cases had towards enterprise gamification with the belief that it is an inevitable reality for most companies. there is a clear relationship between the use of enterprise gamification systems and the likelihood of unlawful or detrimental gamification implementations (raftopoulos, 2014), especially with respect to obligations under new zealand employment law for good faith. cognitive dissonance is a recurrent state in which users or organisations hold conflicting views about gamification as a viable solution. primarily, this stems from the limited understanding of human behaviour and motivation drivers (ryan, rigby, & przybylski, 2006; statista, 2013). concern was expressed over the effectiveness of extrinsic rewards for task completion and the possible negative effects these could have on intrinsic motivation focused systems (pink, 2010). a gamification system designed in adherence to employment laws must ensure that any reward or action designed to direct behavioural change must abide by the obligations of ‘good faith’. unintended consequences relates to negative outcomes in initial business-oriented gamification implementations that were not intended (callan et al., 2015). these consequences are likely due to the exploratory nature of most gamification implementations as the concept is still relatively new in business environments. system implementation occurs in an everchanging landscape, which subsequently has a direct impact on a company’s ability to honour their legal employment obligations. generation disparity relates to assumptions made about the future workforce being dominated by younger, tech-savvy staff (vodanovich, sundaram, & myers, 2010). these younger generations are believed to be more comfortable with gamification systems. however systems designed with generation disparity in mind should not be prohibitive to older australasian journal of information systems hinton, wood, singh & reiners 2019, vol 23, research article enterprise gamification systems and employment legislation 12 generations or outliers. this is a best-practice design principle, rather than a legal obligation to serve all age groups equally. knowledge-focused gamification systems revealed a movement towards business objectives to increase workforce skill levels through knowledge expansion. however, these systems have a tendency to lend themselves to traditional motivational theory (e.g., self-determination theory (ryan & deci, 2000)), by providing challenges or enhancing users’ drive to do better at a given task. this does not appear relevant to legal obligations. co-produced experience provides tasks outside of the standard role and usually in more socially-inclined environments with the goal of creating higher levels of user engagement and motivation in conjunction with others (la guardia et al., 2000). this reflects the shift towards systems with peer-to-peer networks (dimension data, 2015). co-produced experiences have limited legal implications for businesses using them as a motivational device; social workforce conduct must abide by the statutory obligations of good faith if behavioural change is desired. increased stimulus was most evident in literature outlining systems that relied on methods of increased user performance by altering behaviours and perceptions of a given information system or work-related task. the notion of enhancing a user’s positive desire to complete a task does not inherently introduce any new legal obligations. the legal focus should not be the desired enhancement, but rather how this is achieved. socially-focused systems are a primary principle driving designs, often based on motivational types (bartle, 1996), in consumer-focused systems with reliance on social interaction to facilitate and enhance the intrinsic value. as enterprise social networks become more prominent and best practice guides of gamification processes are followed, socially focused systems will play far more significant role and this may create the requirement to be able to disentangle a specific employee’s performance from that of the wider group they work within. mitigating monotony with respect to relevance of legal obligations, there was little evident relationship between the requirements of mitigating monotony and adhering to legal obligations. 5.2 legal themes social and ethical responsibility highlights the necessity to ensure reasonable steps are taken to act in an honest and consistent manner at all times. this is referred to as ‘bona fides’ and symbolises the sincere approach of parties engaged in any action and is core to the concept of “good faith” and translates immediately to the employment relationship. this theme was also evident in several common law cases each of which outlined obligations of social and ethical responsibility outside of strict adherence to legal rules. interpretation is subject to bias: as with the concept of ‘bona fides’, or honesty and frankness, it is apparent that the theme of subject bias in legal proceedings is prevalent. in much of the legal literature surrounding ‘good faith’, objective definitions of what constitutes an ‘honest’ approach appear elusive. each ruling on the issue tended to deviate from or iterated on previous concepts, and in many instances, clear bias on behalf of the judiciary was present. conflicting paradigms of those overseeing rulings are likely to play a significant role in the interpretation of gamification. australasian journal of information systems hinton, wood, singh & reiners 2019, vol 23, research article enterprise gamification systems and employment legislation 13 international influence: in some scenarios, the interpretation of good faith was subject to international influences, in part due to interconnected global norms on social and ethical responsibility and the biases present in a diverse background of judicial administrations. it is also observable that several international legislative systems have a similar approach to labour laws, especially in countries with a comparable legal heritage. contextual difference relates to the legal system’s reliance on the subjective definition of qualitative values. in multiple instances, how an obligation applied in a given situation was not based on the immediate description or previous definition, but instead upon the context in which the obligation was placed at the time. of particular note were approaches towards the statutory duty to refrain from behaviour “likely to mislead or deceive”, with numerous definitions given, and all with substantial differences and legal implications depending on their given context. higher good faith obligation: this theme was less apparent than “contextual difference” and “social and ethical responsibility” in the literature, demonstrating that, even in instances where a good faith obligation has been observed by the parties, there may be in some situations an extended, or higher, expectation of what constitutes “good faith” behaviour. this is similar to the expectation for acts to be ‘active and constructive’, along with the concepts of ‘bona fides’ to ensure parties strive to the best of their ability to act in good faith. negligent action or inaction is one of the most recurrent themes in legal literature. it was central to many disputes under case law revolving around obligations of good faith, and attempts have been made at a statutory level to reduce the value of individuals disregarding their duties towards other parties, subject to their employment agreements. negligent action or inaction was most observable when parties deviated from their duty to be “active and constructive”, “communicative and responsive” and behaved in a way “likely to mislead or deceive”. 6 discussion 6.1 the relationship between ‘good faith’ and gamification systems addressing the first research question (rq1) requires an understanding of enterprise gamification systems and the nature of legislation about employment relationships; therefore, we first synthesise the two sets of themes to answer rq1. businesses must be aware of the connections between good faith and gamification systems: if unaccounted for, they could not only prohibit existing gamified processes from achieving their intended purpose, but also risk disrupting employment relationships while leaving companies vulnerable (callan et al., 2015). first, the design and implementation of gamified employee motivation systems must consider good faith, following the three key principles of being ‘active and constructive’, ‘communicative and responsive’, and refraining from actions ‘likely to mislead or deceive’. in the case of negligent action or inaction, this obligation extends to a party’s duty to proactively inform each other of any irregularities found in a gamification system. further, this would extend to the manipulation of such systems through action or inaction to elevate false impressions of work performance or availability. businesses must facilitate a personalised experience or provide increased performance feedback in a gamified system if it is actively monitored to prevent irregularities (e.g., public misrepresentations of an employee’s performance) from being acted upon as part of the predictive behavioural analysis (paharia, australasian journal of information systems hinton, wood, singh & reiners 2019, vol 23, research article enterprise gamification systems and employment legislation 14 2013). employees, too, have an obligation to ensure that systems are not intentionally manipulated in a way that would constitute misleading or deceptive behaviour. what defines ‘likely to mislead and deceive’ is inherently subjective. as such, gamification requires consideration of the social and ethical responsibilities observable in both the judicial system and each individual actor, and also their ability to act with a higher degree of good faith obligation where it is to be reasonably expected. such expectations will also stem from motivational drivers (e.g., use of self-determination theory (ryan & deci, 2000) and how the system is used to influence employee behaviours. as a result of this contextual differentiation present in any given interpretation, it should be expect that not all examples of good faith actions will be directly transferable between systems or work environments. equally, employees and employers should understand that as gamification emerges, unintended consequences are expected. ideally, businesses should communicate clearly and early with employees while gamification systems are developed and make efforts to open channels to discuss discrepancies and concerns in a proactive manner. it is important to acknowledge that some unintended consequences of new gamification systems are likely to give rise to behaviours viewed as subjectively outside of employees’ or employers’ good faith obligations (callan et al., 2015). this is due to the complex nature of human motivation and the lack of objective blanket definitions of ‘good faith’. in some instances, cognitive dissonance towards a gamified-system, -process, or -behaviour may be the catalyst for complaints under good faith; the effect may be substantially mitigated by ensuring parties are receptive in their duty to be communicative and responsive. by retroactively observing the lessons of disneyland anaheim in this light, three recommendations for the deployment of gamification implementations could be made based on these findings, addressing the second research question (rq2). first, informational displays must be carefully designed to ensure the accurate and fair representation of data appearing in public spaces about employees. this could be accomplished through the introduction of an intermediary party dedicated to reviewing and actively monitoring information appearing on public displays. feedback from employees should be used to ensure there is a comprehensive review of how the information is used and understood and any behavioural consequences can be considered during deployment. second, given the subjective nature of good faith interpretations, it would be beneficial for employers to provide clear and definitive guidelines of what constitutes acceptable behaviour and conduct with respect to a gamified system. ideally, these guidelines should be introduced in the initial employment agreement; however, instituting them into existing gamified systems is possible, providing effort is made to inform employees and they remain communicative and responsive. third, gamified systems should follow an ‘opt-in’ rather than an ‘opt-out’ policy. an opt-in policy would provide a safeguard for companies who acknowledge the inevitability of cognitive dissonance towards gamified systems by employees. in summary, given these challenges, gamification principles in enterprise gamification systems must be carefully deployed, with extensive communication with employees during development and deployment. efforts should be made to be transparent about the types of data and to seek employee feedback on the use and representation of the data. australasian journal of information systems hinton, wood, singh & reiners 2019, vol 23, research article enterprise gamification systems and employment legislation 15 6.2 implications for information system managers answering the second research question allows us to consider how gamification principles can be deployed in a gamification system, without negative consequences in the employment relationship. is managers and designers tasked with introducing enterprise gamified systems should be aware of ways to mitigate these effects by examining their obligations under ‘good faith’. this has implications for the development and design phase of the system, where extensive consultation and feedback from employees will be valuable. it also influences how the information is provided and publicly made available, in a way that is tamper-proof and is immediately clear to the users what the displays are indicating. during system design, there should be consideration of job demands and job resources (bakker et al., 2014) and how the appropriate data is displayed to employees, given these circumstances. employers and employees could best interpret the naturally subjective intentions of the statutory requirements of good faith by ensuring the is managers respond quickly to complaints about system intrusiveness, providing clear communication of problems, defining gathered/displayed performance metrics, and issuing clear ‘good faith’ codes of conduct. 6.3 limitations & future research we based our arguments on new zealand employment law and, although it is possible that our results will be generalisable to other common-law based jurisdictions (e.g., australia and canada), this conclusion rests on other researchers conducting a similar analysis of the relevant case law and statutes in those jurisdictions. while acknowledging that all jurisdictions have differences in the laws that they apply, we believe our results point to underlying issues that will be pertinent across the different countries. the literature and legal cases analysed in this paper were purposively selected to highlight key themes. in particular, legal cases were selected on their relevance to obligations under the duty of good faith; however, in many instances these obligations were part of larger complaints or disputes; e.g., personal grievances for unjustified dismissal or unjustified disadvantage. these legal concepts were drawn from the new zealand domestic legal system with minor affordances made for the influence of international rulings. as international influence emerged as a theme, this should be addressed in future research. while legal analysis emphasises the importance of analysing all relevant cases from the high courts, instead of sampling, and of discussing the specific content of the cases, that method serves a different purpose than the thematic analysis presented here. the use of gamification in workplaces is a fairly new phenomenon, so the goal of the paper was to explore one aspect of the legal implications of this practice. other aspects that we have not explored include the privacy implications of gamification; this is another area of concern for organisations since gamified employee motivation systems collect, use, and disclose information about individuals that may potentially cause them some loss or harm (shahri et al., 2014). another perspective that is worth exploring in the future is the effect of the use of gamification on the implied common law duties of mutual trust and confidence, fair dealing, and fair and reasonable treatment. further work is required to explore these issues, specifically pertaining to obligations under duty of good faith. while our exploration was limited, we have raised issues of adequate importance that a more comprehensive evaluation of case law revolving around personal australasian journal of information systems hinton, wood, singh & reiners 2019, vol 23, research article enterprise gamification systems and employment legislation 16 grievances for unjustified dismissals and unjustified dismissal, such as the occurrence of misleading or deceitful behaviour, are likely to significantly contribute to subjective understanding of “good faith” behaviour. finally, this field of study would benefit substantially from examinations of employee perspectives on obligations of good faith within the employment relationship and other relevant areas of employment law. this would enhance our understanding of user resistance to deployed systems and how users perceive the advantages and disadvantages of gamification and may enable development of implementationor practice-guides and selfhelp tools for managers. 7 conclusions unlike much prior research that focuses on gamification from the perspective of improving user engagement or organisational performance (e.g. liu, santhanam, & webster, 2017), this paper examines its fit with an organisation’s legal obligations to its employees. gamification systems can offer many advantages for business and have the potential to stimulate higher levels of motivation among employees. the use of gamified systems can also be viewed as the next iterative step in customer and employee loyalty, where greater levels of satisfaction are obtained through true intrinsic value stimulation. however, it is equally clear that an inherently subjective legal system and statutory obligations under new zealand employment law make it risky to deploy enterprise gamification systems. this paper’s contribution lies in clarifying how the requirements under the duty of good faith introduces limitations on how and when gamification can be used by organisations, which come as a direct result of employment law’s overarching goal of balancing the legal relationship between employees and employers. to best mitigate the effects of good faith-based obligations on gamified systems, businesses should be mindful and proactive in following the three key principles for duty of good faith under new zealand law. by being active and constructive, communicative and responsive and refraining from behaviour likely to mislead or deceive, they may reap the benefits of a gamified system while avoiding penalties for breaches under the employment relations act 2000. references aiello, j. r., & k. j. kolb. (1995). 'electronic performance monitoring and social context: impact on productivity and stress', journal of applied psychology, 80, 3, 339-353. alder, g. s. (1998). 'ethical issues in electronic performance monitoring: a consideration of deontological and teleological perspectives', journal of business ethics, 17, 7, 729-743. annakis, j., lobo, a., & pillay, s. (2011). ‘exploring monitoring, work environment and flexibility as predictors of job satisfaction within australian call centres’, international journal of business and management, 6, 8, 75–93. https://doi.org/10.5539/ijbm.v6n8p75 baguley v coutts cars ltd [2000] 2 ernz 409. bakker, a., demerouti, e., & sanz-vergel, a. i. (2014). ‘burnout and work engagement: the jd–r approach’, annual review of organizational psychology and organizational behavior, 1, 1, 389-411. australasian journal of information systems hinton, wood, singh & reiners 2019, vol 23, research article enterprise gamification systems and employment legislation 17 ball, k. s., & s. t. margulis. (2011). 'electronic monitoring and surveillance in call centres: a framework for investigation', new technology, work and employment, 26, 2, 113-126. doi:10.1111/j.1468-005x.2011.00263.x bandura, a., w. freeman, & r. lightsey. (1999). 'self-efficacy: the exercise of control', journal of cognitive psychotherapy, 13, 2, 158-166. bartol, k. m., & d. c. martin. (1998), management (3rd ed.). mcgraw-hill birks, m., & j. mills. (2011), grounded theory: a practical guide: sage publications ltd. bogost, i. (2011). persuasive games: exploitationware. retrieved from http://www.gamasutra.com/view/feature/134735/persuasive_games_exploitationware. php?print=1 callan, r. c., k. n. bauer, & r. n. landers. (2015). how to avoid the dark side of gamification: ten business scenarios and their unintended consequences. in t. reiners & l. c. wood (eds.), gamification in education and business (pp. 553-568). cham, germany: springer. carayon, p. (1994). 'effects of electronic performance monitoring on job design and worker stress: results of two studies', international journal of human‐computer interaction, 6, 2, 177-190. chalykoff, j., & t. a. kochan. (1989). 'computer‐aided monitoring: its influence on employee job satisfaction and turnover', personnel psychology, 42, 4, 807-834. chiles, a. m., & t. e. zorn. (1995). 'empowerment in organizations: employees’ perceptions of the influences on empowerment', journal of applied communication research, 23, 1, 125. conley, k., & c. donaldson. (2015). gamification: the measurement of benefits. in t. reiners & l. c. wood (eds.), gamification in education and business (pp. 673-688). cham, germany: springer. retrieved from http://dx.doi.org/10.1007/978-3-319-10208-5_34. doi:10.1007/978-3-319-10208-5_34 corbin, j. m., & a. strauss. (1990). 'grounded theory research: procedures, canons, and evaluative criteria', qualitative sociology, 13, 1, 3-21. de ruyter, k., m. wetzels, & r. feinberg. (2001). 'role stress in call centers: its effects on employee performance and satisfaction', journal of interactive marketing, 15, 2, 23-35. deery, s., & n. kinnie. (2002). 'call centres and beyond: a thematic evaluation', human resource management journal, 12, 4, 3-13. department of labour. (2006). unjustified disadvantage: developments in and examples from 2005. retrieved 2015-07-28, from http://www.dol.govt.nz/er/services/law/case/themes/200602-unjustified-disadvantage.pdf deterding, s., d. dixon, r. khaled, & l. nacke. (2011a), from game design elements to gamefulness: defining gamification proceedings of the 15th international academic mindtrek conference: envisioning future media environments deterding, s., m. sicart, l. nacke, k. o'hara, & d. dixon. (2011b), gamification: using gamedesign elements in non-gaming contexts chi 2011 conference on human factors in computing systems workshop on gamification australasian journal of information systems hinton, wood, singh & reiners 2019, vol 23, research article enterprise gamification systems and employment legislation 18 dimension data. (2015). dimension data’s 2015 global contact centre benchmarking report. retrieved september 17, 2017, from https://www.dimensiondata.com/global/downloadable documents/2015 global contact centre benchmarking summary report.pdf dunham, r. b., j. a. grube, & m. b. castañeda. (1994). 'organizational commitment: the utility of an integrative definition', journal of applied psychology, 79, 3, 370-380. employment relations act (2000). retrieved from http://www.legislation.govt.nz/act/public/2000/0024/latest/dlm58333.ht ml gagné, m., & e. l. deci. (2005). 'self‐determination theory and work motivation', journal of organizational behavior, 26, 4, 331-362. gaut v bp oil new zealand ltd [2011] nzempc 71; (2011) 8 nzelr 687. global sportsman ltd v mirror newspapers [1984] 6 atpr 45,339. grant, r. a., & c. a. higgins. (1991). 'computerized performance monitors: factors affecting acceptance', engineering management, ieee transactions on, 38, 4, 306-315. gwilt v briggs and stratton new zealand ltd [2011] nzempc 140; (2011) 9 nzelr 383. hallberg, l. r.-m. (2006). 'the “core category” of grounded theory: making constant comparisons', international journal of qualitative studies on health and well-being, 1, 3, 141-148. hamari, j., koivisto, j., & sarsa, h. (2014). ‘does gamification work?--a literature review of empirical studies on gamification’. paper presented at the 47th hawaii international conference on system sciences, waikoloa, hi. herzberg, f. (1966), work and the nature of man. cleveland oh: world publishing. herzig, p., m. ameling, & a. schill. (2015). workplace psychology and gamification: theory and application. in t. reiners & l. c. wood (eds.), gamification in education and business (pp. 451-471). cham, germany: springer. retrieved from http://dx.doi.org/10.1007/9783-319-10208-5_23. doi:10.1007/978-3-319-10208-5_23 holman, d. (2002). 'employee wellbeing in call centres', human resource management journal, 12, 4, 35-50. jeske, d., & a. m. santuzzi. (2015). 'monitoring what and how: psychological implications of electronic performance monitoring', new technology, work and employment, 30, 1, 62-78. doi:10.1111/ntwe.12039 kalleberg, a. l., & k. a. loscocco. (1983). 'aging, values, and rewards: explaining age differences in job satisfaction', american sociological review, 78-90. kumar, j. (2013), gamification at work: designing engaging business software. berlin: springer. kumar, r. (2005), research methodology: a step-by-step guide for beginners. los angeles ca: sage. la guardia, j. g., r. m. ryan, c. e. couchman, & e. l. deci. (2000). 'within-person variation in security of attachment: a self-determination theory perspective on attachment, need fulfillment, and well-being', journal of personality and social psychology, 79, 3, 367-384. australasian journal of information systems hinton, wood, singh & reiners 2019, vol 23, research article enterprise gamification systems and employment legislation 19 lawless v comvita new zealand ltd [2005] employment court, auckland ac 77/05. lewig, k. a., & m. f. dollard. (2003). 'emotional dissonance, emotional exhaustion and job satisfaction in call centre workers', european journal of work and organizational psychology, 12, 4, 366-392. liu, d., santhanam, r., & webster, j. (2017). ‘toward meaningful engagement: a framework for design and research of gamified information systems’, mis quarterly, 41, 4, 10111034. lopez, s. (2011). disneyland workers answer to 'electronic whip'. retrieved 07/09/2014, 2014, from http://articles.latimes.com/2011/oct/19/local/la-me-1019-lopez-disney-20111018 mehrotra, v. (1997). 'ringing up big business', orms today, 24, 18-25. mottaz, c. j. (1988). 'determinants of organizational commitment', human relations, 41, 6, 467482. national distribution union inc v carter holt harvey ltd [2001] ernz 822. nee nee v tlnz auckland ltd [2006] employment court, auckland ac 13/06. new zealand cards ltd v ramsay [2012] nzempc 51. nz amalgamated engineering etc union inc v carter holt harvey ltd [2002] ac 53/02. o’hagan v waitomo adventures ltd [2012] nzempc 161. paharia, r. (2013). loyalty 3.0: how to revolutionize customer and employee engagement with big data and gamification. new york, ny: mcgraw hill education. pink, d. h. (2010), drive: the surprising truth about what motivates us. edinburgh: canongate. r v pier [1987] 1 nzlr 66. raftopoulos, m. (2014). ‘towards gamification transparency: a conceptual framework for the development of responsible gamified enterprise systems’, journal of gaming & virtual worlds, 6, 2, 159-178. reiners, t., & l. c. wood. (2015), gamification in education and business. berlin: springer. reiners, t., l. c. wood, s. gregory, & h. teräs. (2015). gamification design elements in business education simulations. in m. khosrow-pour (ed.), encyclopedia of information science and technology (3rd ed., pp. 3048-3061). hershey, pa: information science reference. ryan, r. m., & e. l. deci. (2000). 'self-determination theory and the facilitation of intrinsic motivation, social development, and well-being', american psychologist, 55, 1, 68-78. ryan, r. m., c. s. rigby, & a. przybylski. (2006). 'the motivational pull of video games: a selfdetermination theory approach', motivation and emotion, 30, 4, 344-360. shahri, a., hosseini, m., phalp, k., taylor, j., & ali, r. (2014, november). ‘towards a code of ethics for gamification at enterprise’, in ifip working conference on the practice of enterprise modeling (pp. 235-245). springer, berlin, heidelberg. shaw v schering-plough animal health ltd [2013] nzempc 22. sicart, m. (2008). 'defining game mechanics', game studies, 8, 2, 1-14. australasian journal of information systems hinton, wood, singh & reiners 2019, vol 23, research article enterprise gamification systems and employment legislation 20 smith, m. j., p. carayon, k. j. sanders, s.-y. lim, & d. legrande. (1992). 'employee stress and health complaints in jobs with and without electronic performance monitoring', applied ergonomics, 23, 1, 17-27. stanton, j. m. (2000). 'reactions to employee performance monitoring: framework, review, and research directions', human performance, 13, 1, 85-113. taiapa v te runanga o turanganuia a kiwa trust [2013] nzempc 38; 10 nzelr 378. tekinbaş, k. s., & e. zimmerman. (2004), rules of play: game design fundamentals. cambridge, ma: mit press. vodanovich, s., sundaram, d., & myers, m. (2010). ‘research commentary— digital natives and ubiquitous information systems’, information systems research, 21, 4, 711–723. wallace, c. m., g. eagleson, & r. waldersee. (2000). 'the sacrificial hr strategy in call centers', international journal of service industry management, 11, 2, 174-184. wickham, j., & g. collins. (2004). 'the call centre: a nursery for new forms of work organisation?', the service industries journal, 24, 1, 1-18. doi:10.1080/02642060412331301102 australasian journal of information systems hinton, wood, singh & reiners 2019, vol 23, research article enterprise gamification systems and employment legislation 21 appendix a keywords used in literature search keywords relevance “gamification” provided a broad spectrum of literature on the topic of gamification “motivational psychology” illuminated basic motivational principles leading to the discovery of self determination theory “duty of good faith new zealand” highlighted relevant aspects of nz law with respect to “good faith” obligations “defining game” provided academic literature around the definition of “game” “gamified system” showed examples of current gamified systems in commercial and enterprise settings australasian journal of information systems hinton, wood, singh & reiners 2019, vol 23, research article enterprise gamification systems and employment legislation 22 appendix b – thematic categories and source documents legal themes thematic category classification criteria number of source documents this matched social and ethical responsibility instances of apparent social or ethical implications in legal interpretation 4 interpretation subject to bias instances of legal interpretation subject to paradigm bias 5 international influence instances of legal interpretation influenced by international sovereigns 2 contextual differentiation instances of legal interpretation definitions subject to given context 5 higher good faith obligation instances where legal obligation has wider scope than definition 3 negligent action or inaction instances where legal definition a result of negligence or inaction. 6 gamification themes thematic category classification criteria number of source documents this matched raised enterprise prominence instances of increased enterprise application 6 personalised experience highlighted requirement to personalise systems 9 increased performance feedback highlighted importance of increased feedback 8 mitigating monotony highlighted importance of selfmotivation 4 cognitive dissonance highlighted instances of participants holding conflicting views 4 unintended consequences highlighted unaccounted for events in gamified systems 7 generation disparity highlighted age considerations 4 knowledge focused instances of knowledge focused systems 4 co-produced experience instances of co-produced experiences 5 increased stimulus instances of stimulus focus 6 socially focused instances of socially focused system 3 australasian journal of information systems hinton, wood, singh & reiners 2019, vol 23, research article enterprise gamification systems and employment legislation 23 appendix c – literature used in thematic analysis law literature gamification literature (global sportsman ltd v mirror newspapers, 1984) ryan, r. m., & deci, e. l. (2000). self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. american psychologist, 55(1), 68-78. (r v pier 1987) aparicio, a. f., vela, f. l. g., sánchez, j. l. g., & montes, j. l. i. (2012). analysis and application of gamification. paper presented at the proceedings of the 13th international conference on interacción persona-ordenador. (baguley v coutts cars ltd 2000) singh, s. (2012) gamification: a strategic tool for organizational effectiveness. anveshak-international journal of management, 1(1), 108-113. (national distribution union inc v carter holt harvey ltd, 2001) ryan, r. m., rigby, c. s., & przybylski, a. (2006). the motivational pull of video games: a self-determination theory approach. motivation and emotion, 30(4), 344-360. (lawless v comvita new zealand ltd 2005) landers, r. n., & callan, r. c. (2011). casual social games as serious games: the psychology of gamification in undergraduate education and employee training in: m. ma, a. oikonomou, l. c. jain, serious games and edutainment applications (pp. 399-423). berlin: springer. (nee nee v tlnz auckland ltd 2006) huotari, k., & hamari, j. (2012). defining gamification: a service marketing perspective. paper presented at the proceeding of the 16th international academic mindtrek conference. (gaut v bp oil new zealand ltd, 2011) epstein, z. (2013) enterprise gamification for employee engagement. retrieved 08/07/2015 from http://enterprisegamification.com/attachments/article/137/enterprise%2 0gamification%20paper%20-%20zoe%20epstein%20%20ithaca%20college.pdf (gwilt v briggs and stratton new zealand ltd 2011) deterding, s., sicart, m., nacke, l., o'hara, k., & dixon, d. (2011). gamification: using game-design elements in nongaming contexts. chi 2011 conference on human factors in computing systems workshop on gamification (new zealand cards ltd v ramsay, 2012) greenbaum, j. (2012). enterprise gamification ready to make the collaborative dream real. informationweek online. australasian journal of information systems hinton, wood, singh & reiners 2019, vol 23, research article enterprise gamification systems and employment legislation 24 (o’hagan v waitomo adventures ltd, 2012) korolov, m. (2012). gamification of the enterprise. retrieved 28/07/2015 from http://www.networkworld.com/article/2160336/softwar e/gamification-of-the-enterprise.html (shaw v schering-plough animal health ltd, 2013) thom, j., millen, d., & dimicco, j. (2012). removing gamification from an enterprise sns. paper presented at the proceedings of the acm 2012 conference on computer supported cooperative work. (taiapa v te runanga o turanganuia a kiwa trust, 2013) zichermann, g., & cunningham, c. (2011). gamification by design: implementing game mechanics in web and mobile apps. sebastopol, ca: o'reilly media. ("employment relations act," 2000) deterding, s., dixon, d., khaled, r., & nacke, l. (2011). from game design elements to gamefulness: defining gamification. paper presented at the proceedings of the 15th international academic mindtrek conference: envisioning future media environments. copyright: © 2019 hinton, wood, singh & reiners. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems burmeister & warren 2019, vol 23, research on applied ethics (cybersecurity) preface 1 preface to research on applied ethics (cybersecurity) matthew warren deakin university matthew.warren@deakin.edu.au oliver burmeister charles sturt university this is the second special section on applied ethics for ajis. as was the case for the first special section on ethics, of the various submissions, only three have been accepted for publication. this is not an indication that little work is being done in relation to cybersecurity ethics, but rather a reflection of the difficulty of getting published in a high quality journal. a great deal of research is being done in the area of ethics as regards cybersecurity, particularly in europe as a result of the recent toughening of its privacy legislation and the implications that has for all manner of ethics and technology, from blockchain, to wearable robots and through to cybersecurity. an overview of those three articles follows, after which the guest editor backgrounds are described. the paper by miller on ‘machine learning, ethics and law’ explores ethical machine learning issues concerning data firm cambridge analytica’s illegitimate use of the data of millions of facebook users. its focus is on the three areas of profiling and predictive policing, legal adjudication, and machines’ compliance with legally enshrined moral principles. the paper by luck on ‘entrapment behind the firewall: the ethics of internal cyber-stings’ addresses in part the focus of so much of the security literature, which reveals that the majority of breaches happen by insiders. it explores the ethical issues, such as trust, of conducting internal stings involving an organisation’s own employees. the paper by thomas et al. on ‘the importance of ethical conduct by penetration testers in the age of breach disclosure laws’ looks at ethical hacking in the light of breach disclosure laws, which are designed to protect privacy, such as the notifiable data breaches scheme in australia and the general data protection regulation (gdpr) in the european union. in relation to the ethical hacking paper, which was coauthored by one of the guest editors, it should be noted that he was not involved in any of the reviewing and editorial decision making for that paper. instead those tasks were carried out by the editor-in-chief of ajis. the ajis editorial system was set-up such that prof burmeister could not see that submission, nor any processing of it. two guest editors were appointed, professor matthew warren (deakin university) and professor oliver burmeister (charles sturt university). professors warren and burmeister have collaborated in various ways over the past 20 years. they were both executive members of the australasian institute on computer ethics (aice) and have both been members of the australian computer society’s (acs) ethics committee; prof burmeister still is. they are also both involved in the cybersecurity collaborative research centre (crc), which has six research and 25 industry partners, as well as significant funding from the government to the amount of $139 million over seven years. professor matthew warren has published eight times in ajis over the past 17 years (senarathna, et al, 2018, senarathna, et al, 2016, pye and warren 2006, pierce, et al 2006, leitch and warren 2003, warren and hutchinson 2003, hutchinson and warren 2002a, hutchinson and warren 2002b, hutchinson and warren 2001). professor warren has served as a chair of information systems, but for the past few years has been a professor of cyber security at deakin university. professor burmeister has also published australasian journal of information systems burmeister & warren 2019, vol 23, research on applied ethics (cybersecurity) preface 2 multiple times in ajis (bowern, burmeister, gotterbarn, & weckert, 2006; burmeister, 2015; burmeister, islam, dayhew, & crichton, 2015; burmeister et al., 2019; islam, miah, kamal, & burmeister, 2019; poulsen & burmeister, 2019; simpson, nevile, & burmeister, 2003). most of his ict ethics work been in the areas of value sensitive design (burmeister, 2010, 2012, 2016; burmeister & kreps, 2018; burmeister, weckert, & williamson, 2011; teipel et al., 2016) and professional ethics (burmeister, 2013, 2017). references bowern, m., burmeister, o. k., gotterbarn, d., & weckert, j. (2006). ict integrity: bringing the acs code of ethics up to date. australasian journal of information systems, 13(2), 168-181. doi:http://dx.doi.org/10.3127/ajis.v13i2.50 burmeister, o. k. (2010). websites for seniors: cognitive accessibility. international journal of emerging technologies and society, 8(2), 99-113. burmeister, o. k. (2012). what seniors value about online community. journal of community informatics, 8(1). burmeister, o. k. (2013). achieving the goal of a global computing code of ethics through an international-localisation hybrid. ethical space: the international journal of communication ethics, 10(4), 25-32. burmeister, o. k. (2015). improving professional it doctorate completion rates. australasian journal of information systems, 19, 55-70. doi:10.3127/ajis.v19i0.1073 burmeister, o. k. (2016). the development of assistive dementia technology that accounts for the values of those affected by its use. ethics and information technology, 18(3), 185-198. doi:10.1007/s10676-016-9404-2 burmeister, o. k. (2017). professional ethics in the information age. journal of information, communication & ethics in society, 15(4), 348-356. burmeister, o. k., islam, m. z., dayhew, m., & crichton, m. (2015). enhancing client welfare through better communication of private mental health data between rural service providers. australasian journal of information systems, 19, 1-14. doi:10.3127/ajis.v19i0.1206 burmeister, o. k., & kreps, d. (2018). power influences upon technology design for age-related cognitive decline using the vsd framework. ethics and information technology, 20(3). burmeister, o. k., ritchie, d., devitt, a., chia, e., dresser, g., & roberts, r. (2019). the impact of telehealth technology on user perception of wellbeing and social functioning, and the implications for service providers. australasian journal of information systems, 23. doi:10.3127/ajis.v23i0.1501 burmeister, o. k., weckert, j., & williamson, k. (2011). seniors extend understanding of what constitutes universal values. journal of information, communication & ethics in society, 9(4), 238-252. doi:10.1108/14779961111191048 leitch, s. and warren, m.j. (2003). designing systems for e-commerce, australian journal of information systems, vol 10, no 2. http://dx.doi.org/10.3127/ajis.v10i2.161 hutchinson, w. and warren, m.j. (2002). information warfare: using the viable system model as a framework to attack organisations, australian journal of information systems, vol 9, no 2. http://dx.doi.org/10.3127/ajis.v9i2.193 australasian journal of information systems burmeister & warren 2019, vol 23, research on applied ethics (cybersecurity) preface 3 islam, m. r., miah, s. j., kamal, a. r. m., & burmeister, o. k. (2019). a design construct developing approaches to measure mental health conditions. australasian journal of information systems, vol 23. http://dx.doi.org/10.3127/ajis.v23i0.1829 poulsen, a., & burmeister, o. k. (2019). overcoming carer shortages with care robots: dynamic value trade-offs in run-time. australasian journal of information systems, 23. doi:10.3127/ajis.v23i0.1688. pierce, j., jones, a. and warren, m.j. (2006). penetration testing professional ethics: a conceptual model and taxonomy, australasian journal of information systems, vol 13, no 2. http://dx.doi.org/10.3127/ajis.v13i2.52 pye, g. and warren, m.j. (2006). striking a balance between ethics and ict governance, australasian journal of information systems, vol 13, no 2. http://dx.doi.org/10.3127/ajis.v13i2.53 senarathna, i, wilkin c, warren, m, yeoh, s & salzman s (2018). factors that influence adoption of cloud computing: an empirical study of australian smes, australian journal of information systems, vol 22. http://dx.doi.org/10.3127/ajis.v22i0.1603 senarathna, i, warren, mj, yeoh, w & salzman, s (2016) security and privacy concerns for australian smes cloud adoption: empirical study of metropolitan vs regional smes, australasian journal of information systems, vol 20. http://dx.doi.org/10.3127/ajis.v20i0.1193 simpson, c., nevile, l., & burmeister, o. k. (2003). doing ethics: a universal technique in an accessibility context. australasian journal of information systems, 10(2). doi:http://dx.doi.org/10.3127/ajis.v10i2.159 teipel, s., babiloni, c., hoey, j., kaye, j., kirste, t., & burmeister, o. k. (2016). information and communication technology solutions for outdoor navigation in dementia. alzheimer's & dementia: the journal of the alzheimer's association, 12(6), 695-707. doi:10.1016/j.jalz.2015.11.003 warren, m.j. and hutchinson, w. (2003). australian hackers and ethics, australian journal of information systems, vol 10, no 2. http://dx.doi.org/10.3127/ajis.v10i2.163 warren, m.j. and hutchinson, w. (2001). information warfare and ethics, australian journal of information systems, vol 8, no 2. http://dx.doi.org/10.3127/ajis.v8i2.242 copyright: © 2019 burmeister & warren. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems burmeister & warren 2019, vol 23, research on applied ethics (cybersecurity) preface 4 paper title (paper title style) australasian journal of information systems nayal & pandey 2020, vol 24, research article digital coupon redemption 1 digital coupon redemption: conceptualization, scale development and validation preeti nayal national institute of industrial engineering (nitie) india neeraj pandey national institute of industrial engineering (nitie) india npandey@nitie.ac.in abstract the purpose of this study is to develop a reliable and valid scale for measuring digital coupon redemption. the scale development process began with the grounded theory approach for item generation. this was followed by the development of a conceptual framework, item generation, initial reliability analysis, exploratory factor analysis and confirmatory factor analysis (cfa) based on primary data collected from 609 participants. for validating the scale, the cfa was repeated using data from 411 unique participants. results from two independent surveys supported the proposed scale in terms of construct, convergent and discriminant validity. survey study resulted in a 45-item digital coupon redemption scale. it also covers digital aspects such as intention to search, attitude toward internet searching, perceived risk, perceived convenience and personal innovativeness which were not covered in earlier promotion scales. this scale would assist marketers to maximize digital coupon redemption during their promotion campaigns. this is the first attempt to provide a comprehensive scale to measure digital coupon redemption in the marketing literature. keywords: scale development, digital coupon, redemption, promotion 1 introduction global estimates suggest that retail e-commerce sales will account for revenue worth usd 4,878 billion by 2021 (www.statista.com). not surprisingly, most companies today spend a considerable part of their marketing budget on online promotional tools (madlberger and matook, 2017; pandey and singh, 2012), such as digital coupons. however, low rates of coupon redemption, which lead to sub-optimal utilization of company resources, have been a concern for most marketers and organizations (mills and zamudio, 2018: pandey and maheshwari, 2017). danaher et al. (2015) reported a redemption rate as low as 1.06% in their longitudinal survey on digital coupons. according to the nch 2018 report, out of 256.5 billion digital coupons distributed in the united states (us) in 2018, only 1.715 billion coupons were redeemed (0.7%). the extant literature on digital coupons does not offer an empirically tested scale to measure or evaluate digital coupon redemption. most studies limit themselves to discussing the main variables that influence coupon redemption such as coupon proneness (chen and lu, 2011), perceived coupon value (liu et al., 2015), perceived risk (ha and im, 2014), and perceived convenience (danaher et al., 2015). honea and dahl (2005) proposed a promotion affect scale (pas) to measure consumer response to promotion; however, pas is not tailored to couponbased promotions, and therefore does not consider several coupon-specific factors, including australasian journal of information systems nayal & pandey 2020, vol 24, research article digital coupon redemption 2 coupon proneness and perceived coupon value. further, consumer behaviour has undergone tremendous changes since the introduction of smartphones, and because pas was developed in the pre-smartphone era, its applicability to digital coupons is limited. this study attempts to address the need for a more comprehensive, rigorous, and integrated analysis of coupon redemption by developing a scale for it. this proposed scale can contribute to marketing literature by delineating the construct of “intention to redeem digital coupons” and identifying and validating measures to capture intention to redeem digital coupons. addressing the call to better understand digital coupon redemption, the study presents a generic scale for coupon redemption that can be used across industries. the standardized scale can also be used by e-marketers to better understand factors that influence digital coupon redemption. the scale development process is structured according to the procedures suggested by churchill (1979); parasuraman et al. (1988); hinkin (1995) and tanwar and prasad (2017). it has three major steps: conceptualization, item generation, and scale refinement and validation (figure 1). in the following section, we conceptualize the construct of coupon redemption using inputs from grounded theory (gt). this is reinforced by the review of available literature on coupon redemption. next, we develop a pool of items with the help of both inductive and deductive approaches. third, the results from our quantitative inquiry are presented to refine and validate the scale for digital coupon redemption. finally, we discuss how consumer preference for coupons may provide new insights that can enhance the redemption of digital coupons. the first step in the scale development process is conceptualization of the construct “intention to redeem digital coupons” (parasuraman et al., 1988; netemeyer et al., 2003). to accomplish this, we first use an inductive gt approach and then a deductive process based on extant literature. this was followed by finalization of the dependent and independent variables with the inputs from experts. both inductive and deductive approach were used for item generation of identified key variables. the exploratory and confirmatory analysis were conducted to finalize the coupon redemption intention scale. 2 conceptualization of coupon redemption 2.1 grounded theory gt is appropriate to apply where the phenomena are quite recent and need more in-depth inquiry (murphy et al., 2017). most technology adoption theories do not consider variables from the post-smartphone era; therefore, their relevance and comprehensiveness in explaining digital coupon redemption are questionable. furthermore, these adoption theories mainly consist of positive variables to explain digital coupon redemption, but one cannot ignore negative variables such as perceived risk to explain digital coupon redemption in the technology era (im and ha, 2015). therefore, in this study, we conceptualize digital coupon redemption and its dimensions using gt. accordingly, 37 interviews were conducted with australasian journal of information systems nayal & pandey 2020, vol 24, research article digital coupon redemption 3 figure 1: procedure adopted for scale development assessment of various models such as one-factor, two-factor etc. (to check which is superior). deductive through literature, generation of 62 items item scanning face validity, content validity and pilot testing construct validity (n=411) • confirmatory factor analysis • assessment of convergent, discriminant validity • comparative analysis scale purification (n=609) • initial reliability analysis • exploratory factor analysis (dependent and independent efa separately) • cfa and assessment of series of model for it with discriminant and convergent validity • common method bias • comparative analysis data collection was completed in two stages (antecedents and consequences separately). 47 items were retrieved after efa item generation inductive generation of 50 items through brainstorming session with 4 experts, 5 professors and 7 scholars conceptualization of construct inductive (grounded theory) • in depth interviews of 37 consumers • nvivo 12 has been used to analyse transcripts and 3 stage coding to finalize the themes deductive (literature review) • through literature review (extended tpb and chen and lu, 2011) finalization of 21 variables for conceptual framework • brainstorming sessions with academicians and industry experts in promotion • 4 dependent variables and 17 independent variables 1 . c o n cep tu a liza tio n 2 . item g en era tio n 3 . s ca le p u rifica tio n a n d v a lid a tio n australasian journal of information systems nayal & pandey 2020, vol 24, research article digital coupon redemption 4 professionals working in diverse sectors, such as information technology (it), finance, manufacturing, and defence, to ensure data saturation. each interview lasted for 45 ± 15 minutes and was conducted face to face at a mutually convenient location. the consolidated transcripts of the interviews were analysed using qualitative analysis software (nvivo 12). a three-stage coding process (i.e., open coding, axial coding, and selective coding) was used to finalize the gt model for digital coupon redemption (corbin and strauss, 1990). intention to search was identified as a major theme, followed by coupon proneness and perceived coupon value. the following themes were introduced as antecedents: perceived economic benefits, perceived convenience, perceived usefulness, emotional value, attitude, innovativeness, subjective norms, past experience and perceived enjoyment. finally, minor themes were added as moderators: face value, customization, expiration date, perceived risk, consumption frequency, and distance. excerpts from the interview transcripts are given below: “i find searching a convenient way to get the latest and most exciting deals” [intention to search] “searching is better than subscription sometimes as you search exactly what you need and do not have fears of spams” [intention to search] “coupons are a good way to save your money and time as sometimes you choose that product to buy for which you have coupon rather than wasting time to decide” [perceived coupon value] “i always check the deal for the product which i want to purchase. it makes me feel like a smart shopper.” [coupon proneness] 2.2 literature review factors influencing digital coupon redemption have been studied in the past (fortin, 200; kang et al., 2006; chen and lu, 2011; im and ha, 2013; danaher et al., 2015). lichtenstein et al. (1990) explained coupon redemption behaviour with the help of the transaction-utility theory. as per utility theory, coupon redemption is a function of acquisition utility and transaction utility. the theory of reasoned action (tra), which has been used in two studies from traditional coupon context (bagozzi et al., 1992; shimp and kavas, 1984), posits that a consumer’s redemption intention depends on their attitude and subjective norms about coupons. clark et al. (2013) have used theory of reasoned action (tra) to explain digital coupon redemption behaviour. liu et al., (2015) have used unified theory of acceptance and use of technology (utaut) theory to explain consumer’s redemption intention for digital coupons. the tpb model of digital redemption advances the tra model by including perceived behaviour control as a third antecedent predicting coupon redemption intention. fortin (2000) explained coupon redemption behaviour using the theory of planned behaviour (tpb). ha and im (2014) studied both positive and negative factors and found that attitude positively influences users’ redemption intention while perceived risk negatively influences it. danaher et al. (2015) tested several drivers of digital coupon redemption such as time of delivery, location, and expiry date but failed to include customization, which is an important variable. kang et al. (2006) used the same the tpb model but introduced two new variables: attitude toward internet searching and past behaviour with coupons to explain digital coupon redemption. australasian journal of information systems nayal & pandey 2020, vol 24, research article digital coupon redemption 5 our framework for digital coupon redemption is based on chen and lu (2011) framework which was based on the extended tpb model (kang et al., 2006). moreover, chen and lu (2011) argued that one cannot ignore coupon proneness as a mediator to explain digital coupon redemption. we have used gt and extant literature to finalize the construct of intention to redeem digital coupons. we consulted four industry experts with an average of ten years’ experience in marketing and three scholars of marketing, who had more than 14 years of average academic experience. the conceptual framework, based on the gt and literature review, has been presented in figure 2. 3 variables of the construct the variables of the intention to redeem digital coupons have been categorized as dependent and independent (figure 2). the dependent variables consist of the intention to redeem digital coupons, the intention to search, coupon proneness, and perceived coupon value. the rest, such as antecedents and moderators, constitute the independent variables. 3.1 dependent variables consumers who access digital coupons exhibit intention to search. some consumers believe that searching is better because it allows them to find coupons whenever they need without falling prey to unwanted spam (nayal and pandey, 2019). jiménez and mendoza (2013) showed that the intention to search for coupons as needed has even helped consumers find online reviews about the products/services they wish to purchase. therefore, it can be expected that intention to search is a key variable to explain digital coupon redemption. perceived coupon value was found one of the crucial variables in the context of internet banking (kumar et al., 2020). perceived coupon value has been identified as a crucial factor explaining coupon redemption intention (pura, 2005; liu et al., 2015; clark et al., 2013). similarly, several scholars have argued that consumers’ coupon redemption intention cannot be explained without considering coupon proneness (chen and lu, 2011; swaminathan and bawa, 2005: clark et al., 2013). perceived coupon value is more related to acquisition utility that results from the utility of the product and coupon proneness is related to transaction utility that results from the hedonic value of the product (lichtenstein et al. 1990). moreover, souiden et al., (2019) also used utility theory to explain digital coupon redemption behaviour. in the context of online shopping, to et al., (2007) found that hedonic benefits and utilitarian benefits are the main determinants of intention to search or intention to purchase. therefore, it can be expected that perceived coupon value and coupon proneness are two major key antecedents for intention to search for digital coupons. 3.2 independent variables 3.2.1 antecedents studies have shown that perceived economic benefit positively influences the intention to redeem digital coupons (raghubir, 2004; dickinger and kleijnen, 2008; souiden et al., 2019), along with perceived convenience (colwell et al., 2008; khajehzadeh et al., 2015). pura, (2005) demonstrated that perceived economic benefit and perceived convenience are key variables to explain perceived value in mobile technology context. perceived enjoyment refers to the hedonic value associated with redeeming coupons (wierich and zielke, 2014; im and ha, 2015). ieva et al., (2019) found that perceived enjoyment is a crucial variable to explain intention to redeem digital coupons. because digital coupons can be accessed via laptops and australasian journal of information systems nayal & pandey 2020, vol 24, research article digital coupon redemption 6 smartphones, they can be made visually appealing with stickers, emoticons, or sounds. this visualization process makes consumer feel a certain hedonic value while redeeming digital coupons. ha and im, (2014) used rogers’s theory to explain adoption of mobile coupons. as per roger’s theory, relative advantage (usefulness) of the given technology is a key determinant to build the adoption intention. chi and kilduff, (2011) said that perceived emotional value is a hedonic aspect of perceived value that is aroused by feelings created by product or services. digital coupon redemption provides shoppers with self-satisfaction and a feeling of being smart (shimp and kavas, 1984). perceived usefulness (sweeney and soutar, 2001) and emotional value (sweeney and soutar, 2011; pura, 2005) have been reported as key dimensions of perceived coupon value. therefore, perceived economic benefit, perceived convenience, perceived enjoyment, perceived usefulness and emotional value are expected to be key dimensions to explain perceived coupon value of digital coupons. attitude and subjective norms have also been considered key constructs in technology adoption theories such as tra and tpb. subjective norms refer to the social trends and beliefs that influence consumers to adopt a particular behaviour. chen and lu (2011) showed that attitude toward digital coupons and toward internet searching, subjective norms, and past behaviour were the key antecedents of coupon proneness. highly coupon-prone consumers tend to be high on personal innovativeness (agarwal and prasad, 1998), and liu et al. (2015) found that personal innovativeness positively influences one’s intention to redeem digital coupons. some studies (garretson and burton, 2003; clark et al., 2013) have shown that coupon-prone consumers are not only sensitive to price but also seek psychological gains. therefore, it can be expected that attitude toward digital coupon and internet searching, subjective norms, past behaviour and personal innovativeness are the key dimension for coupon proneness. 3.2.2 moderators perceived risk may adversely influence the sharing of personal information online (featherman et al., 2010; dang-pham, pittayachawan, and nkhoma, 2015), especially if consumers believe that their information may be used for spamming. therefore, although a consumer perceives value in digital coupons, s/he may be sceptical to search for it as searching for coupon require personal details sometimes and this study expects perceived risk as a moderator. huang et al. (2014) showed that consumption frequency moderates repeat purchase behaviour on online platforms. in fact, when buying online as a group, a consumer’s propensity for repeat purchase doing depends on their consumption frequency, which also acts as a moderator (hsu et al., 2006). various studies found that distance to the redemption site negatively influences the chances of redemption (chiou-wei and inman, 2008; danaher et al., 2015; beeck and toporowski, 2017). travelling long distances to redeem coupons requires spending more time and resources on redemption, which reduces the attractiveness of coupon redemption (hongjai rhee and bell, 2002; chiou-wei, 2004). in this study, distance has been considered a moderator. australasian journal of information systems nayal & pandey 2020, vol 24, research article digital coupon redemption 7 figure 2: framework for intention to redeem digital coupons after a consumer accesses a coupon—through internet search—its redemption depends on the face value of the coupon. the higher the face value, the higher the chances of digital redemption (danaher et al., 2015). reichhart et al. (2013) noted that the medium of coupon delivery influences consumers’ intention to redeem digital coupons. they found a difference between redemption of coupons delivered via e-mail and mobile text messages. thus, apps, e-mail, and sms influence coupon redemption intention differently. generally, coupon redemption rates are high immediately after the launch date, decline in the middle, and spike again just before the expiration date (bowman, 1980). danaher et al. (2015) found that long expiry dates for digital coupons negatively influence coupon redemption intention (figure 2). customization helps companies to deliver the coupon as per the preference of consumers (danaher et al., 2015). for example, companies track the online browsing history of consumers and send related coupons (banerjee and yancey, 2010). it can be also done by sending the coupons to the consumers as per their preferred time of the day. once consumer search for the coupons, s/he may register in few coupon websites. s/he may receive coupons. but it is not necessary that redemption happens directly after search, coupon redemption depends on various variables which is considered moderator in current study. these include face value, expiry date, medium of coupon delivery, customization, and distance. australasian journal of information systems nayal & pandey 2020, vol 24, research article digital coupon redemption 8 4 item generation and refinement after conceptualizing the construct, we developed items to measure the intention to redeem digital coupons. item generation can follow an inductive approach when no theoretical foundation is available in the literature, and the topic needs to be investigated through sample respondents. alternatively, a deductive approach can also be used when items are well defined in the extant literature (papadas et al., 2017). this study used both an inductive and a deductive approach to generate the items. for the inductive approach (brainstorming), we selected four marketing industry experts, seven scholars and five professors from marketing. to be selected, the participants had to have used digital coupons at least once in the last six months. item generation involved multiple iterations—items were discussed, modified, deleted, and added in preparation for the next step. finally, 62 items were generated through an inductive approach, and 50 items were generated through a deductive approach. thus, at the end of this step, we had 112 items to measure digital coupon redemption. 4.1 item finalization using expert opinions to determine the content and face validity of the instrument, consisting of 112 items, we conducted an expert review. two professors, from different universities, who had published papers on scale development in reputed journals were contacted. additionally, two industry experts with more than 15 years of relevant experience were asked to assess the consistency, appropriateness, and ambiguity of the items. on the basis of the feedback from the experts and professors, the wording of some items was refined, and the final scale was subjected to a pilot test. each item was evaluated on a 7-point likert scale, with 1 indicating “strongly disagree” and 7 indicating “strongly agree.” 5 pilot testing forty-three participants pilot-tested the scale and shared their feedback on the language, logical sequence, and consistency of the items included. they did not report any difficulty in understanding or answering the questions. no inconsistency, ambiguity, or cultural specifics were observed, and thus, the adequacy of the proposed items was confirmed (patyal and koilakuntla, 2015). because of satisfactory evaluation in the pilot test phase, all the items were retained for final data collection. 6 data collection (1st stage) data were collected in two stages: on antecedent variables (63 items) in the first stage and on consequence variables (61 items) after a gap of 14 days (second stage). this helped reduce the effect of common method variance, which is associated with a single source and one-time data collection (podsakoff et al., 2003). for the first stage, we surveyed professionals from various industry sectors; and students studying in university b-schools. the professionals worked in either government or private firms in diverse industries such as banking, human resources, marketing, manufacturing, research, it, and supply chain. in total, 833 participants were approached, and completed surveys were received from 609 respondents over a period of 10 months. only respondents who had experience in redeeming digital coupons were invited to participate in the survey. all the survey items were in english as it was well understood by all the participants. australasian journal of information systems nayal & pandey 2020, vol 24, research article digital coupon redemption 9 6.1 data analysis 6.1.1 reliability analysis reliability reflects the consistency of the items on the measurement scale. cronbach’s alpha is a widely used measure of reliability, and an alpha value of 0.60 is considered an acceptable score (nunnally and bernstein, 1994). table 1 shows the results of the reliability analysis of the proposed scale. two variables with a cronbach’s alpha value less than 0.60 were omitted from the subsequent scale development process. next, we assessed the corrected item-to-total correlation and discarded items with scores less 0.40 (nunnally and bernstein, 1978). thus, by the end of the reliability analysis, we had retained 84 items: 62 items for independent variables and 22 items for dependent variables. 6.2 exploratory factor analysis (efa) efa was conducted separately for the 22 items for dependent variable items and the 62 items for independent variables (patyal and koilakuntla, 2015). before efa, data appropriateness of the 22 items was assessed through the kaiser-meyer-olkin (kmo) statistic, which was found to be 0.919. a kmo value above 0.60 is considered satisfactory for efa (kaiser and rice, 1974). the results of bartlett’s test of sphericity were also satisfactory (p<.01), implying that the correlation matrix was “meritorious.” principal component analysis with promax rotation was performed on 22 items as the factors were correlated (sieger et al., 2016). the efa yielded a 4factor model, with an eigenvalue greater than 1 for all the factors. all the communalities were above 0.40. the threshold level for deletion of items was a factor loading of <0.50 (hair et al., 2006). thus, 9 items were deleted: items with a factor loading less than 0.50 and those that cross-loaded with each other. the efa analysis identified factors that explained 59% of the variance. a second efa was conducted on 62 items of the independent variables. the kmo for these items was 0.945, and a principal component analysis with promax rotation was performed on all the 62 items. the efa resulted in a 9-factor model that explained 61% of the variance, with an eigenvalue greater than 1 for all the factors. all the communalities were above 0.40. the threshold level for item deletion was a factor loading <0.50 (hair et al., 2006). as a result, 30 items were deleted. table 1 shows the factor loading on items in terms of efa and cronbach’s alpha values. 6.3 common method bias harman’s single factor test was used to examine the potential for common method bias among the studied factors (podsakoff et al., 2003). high common method variance is indicated if a single factor accounts for more than 50% of the covariance among the dependent and independent variables (lindell and whitney, 2001). in this study, multiple factors were extracted from the principal component analysis, and the first factor accounted for 23% of the 59% variance. thus, we concluded that self-report bias of the respondents is not a concern in this study. australasian journal of information systems nayal & pandey 2020, vol 24, research article digital coupon redemption 10 constructs items factor loading cronbach alpha intention to redeem digital coupons i will probably redeem the digital coupons 0.914 0.806 i am certain to redeem the digital coupons 0.793 i will use a digital coupon if i find something i like 0.761 i like to redeem the digital coupons 0.732 i would consider searching for digital coupons in the near future 0.503 perceived coupon value when purchasing a product, i always try to maximize the quality i get for the money i spend 0.892 0.677 when i buy products, i like to be sure that i am getting my money's worthy 0.872 getting digital coupon for combo offers gives me value for money 0.619 coupon proneness digital coupons have caused me to buy products i normally would not buy 0.894 0.746 i have favourite brands, but most of the time i buy the brand i have a coupon for 0.758 i am more likely to buy brands for which i have a digital coupon 0.755 i enjoy using digital coupons, regardless of the amount i save by doing so 0.5 intention to search i search various sites whenever i want to use digital coupons for a movie 0.867 0.726 i find searching coupons on various sites is the best option to get the coupons 0.747 searching for the coupons is the good option as you do not need to register with your personal details 0.736 attitude towards coupon it is fun to use digital coupon 0.964 0.778 i always look for coupons before buying products 0.865 i react favourably to digital coupons 0.526 subjective norms using a digital coupon would make me feel accepted by my friends and family 0.859 0.837 society acceptance matters to me for using digital coupons 0.811 i would use digital coupons to get social appraisals 0.777 usage of coupons depend on what my acquaintances think about me when they come to know that i have used coupons to save money 0.744 consumption frequency i will purchase products more often if i receive digital coupon with combo offer. 0.703 0.734 i will purchase products more often if i receive buy one get one free digital coupon 0.687 i will increase my purchasing frequency if i get substantial economic benefit from coupons 0.673 i am likely to increase my purchasing frequency if i get rs.100 off digital coupon 0.514 australasian journal of information systems nayal & pandey 2020, vol 24, research article digital coupon redemption 11 constructs items factor loading cronbach alpha distance from multiplex i do not mind traveling provided i have digital coupon 0.815 0.768 if medium of travelling is economical, i can go far to redeem the digital coupons 0.767 if i own a car, distance does not bother me to redeem the digital coupon 0.712 the ambience of the shopping mall overcomes the pain of distance travelled to redeem the digital coupon 0.52 attitude towards internet searching i feel searching the internet gives me an edge in the competitive world 0.833 0.746 i feel internet searching adds value to my lifestyle 0.78 i like searching for information on products online before deciding to purchase them 0.619 i like to search internet before making any buying decision of any product 0.612 perceived risk in general, i find digital coupons forced 0.774 0.687 in general, i find digital coupons disturbing 0.704 in general, i find digital coupons intrusive 0.658 face value i find coupons with a face value of at least 10% discount worth using. 0.695 0.676 the face value should cover at least the convenience charges levied on the product 0.514 face value decides the value of digital coupons for me 0.743 personal innovativeness i like to experiment with new technologies 0.776 0.675 among my peers, i am usually the first to try out new information technologies 0.708 if i heard about a new information technology, i would look for ways to experiment with it 0.667 perceived convenience using digital coupon apps enable me to use coupons at any time 0.905 0.719 using apps for digital coupon give me convenience to find needed coupons 0.691 using digital coupons is clear and understandable 0.574 using apps for digital coupon enable me to use coupons in any place 0.565 table 1: factor loading for constructs 6.4 confirmatory factor analysis (cfa) cfa was conducted with 609 data points using amos version 23.0. the 15-item scale for four dependent factors was validated with the help of cfa. the measurement model was evaluated on the basis of factor loading, standardized residuals, goodness-of-fit indices, and modification indices (patwardhan et al., 2017). for the dependent factors, modification indices ranged from 4.428 to 14.31. two items were omitted because the factor loading was below 0.50. we deleted the items after ensuring that the deletion does not affect the integrity of the factors (nahm, et al., 2004). table 2 presents all the fit indices, which had satisfactory values. finally, the measurement model contained 13 items (4 dependent factors) that measured the intention to redeem digital coupons. australasian journal of information systems nayal & pandey 2020, vol 24, research article digital coupon redemption 12 next, we used cfa to validate the 32-item scale covering 9 independent factors. modification indices ranged between 4.2 and 26.43. no items were deleted as the factor loading of all the items was above 0.50. the fit indices also showed satisfactory values. finally, we compared the two proposed models—the 4-factor model for dependent variables and the 9-factor model for independent variables—with a series of other models. table 3 shows that the 4-factor solution is the best-fit model for dependent variables, with the lowest χ2/df and rmsea(root mean square error of approximation) and the highest cfi (comparative fit index) and gfi(goodness fit index) (sweeney and soutar, 2001). similarly, table 4 shows that the 9-factor model is the best-fit for the independent variables, with the lowest χ2/df and rmsea and the highest cfi and gfi (sweeney and soutar, 2001) (figure 3). fit indices criteria dependent cfa (fit indices) independent cfa (fit indices) χ2/ df <3 2.66 2.30 agfi >0.80 0.94 0.89 cfi >0.90 0.95 0.91 gfi >0.90 0.96 0.91 rmsea <0.05 0.05 0.05 table 2: model fit statistics χ2/df agfi cfi gfi rmsea 4-factor model 2.658 0.940 0.950 0.961 0.052 3-factor model (a) 5.403 0.87 0.86 0.908 0.085 2-factor model (b) 8.373 0.804 0.76 0.86 0.110 1-factor model (c) 12.14 0.73 0.64 0.80 0.135 (a) intention to search (is) + intention to redeem (ir), (b) bis+ir+ coupon proneness (cp), (c) all items in one factor table 3: comparative analysis for dependent factors (1st stage data collection) australasian journal of information systems nayal & pandey 2020, vol 24, research article digital coupon redemption 13 χ2/df agfi cfi gfi rmsea 9-factor model 2.304 0.885 0.906 0.907 0.046 8-factor model (a) 3.855 0.795 0.789 0.831 0.069 7-factor model (b) 4.127 0.768 0.766 0.805 0.072 6-factor model (c) 4.248 0.768 0.753 0.802 0.073 5-factor model (d) 4.764 0.742 0.711 0.778 0.079 4-factor model (e) 5.492 0.699 0.652 0.739 0.086 3-factor model (f) 5.803 0.689 0.626 0.729 0.089 2-factor model (g) 6.529 0.646 0.567 0.689 0.095 1-factor model (h) 6.89 0.633 0.538 0.677 0.098 (a) attitude toward coupon (ac)+ subjective norms(sn), (b) ac+sn+ perceived convenience (pc), (c) ac+sn+pc+ personal innovativeness (pi), (d) ac+sn+pc+pi+ attitude toward internet searching (ais), (e) ac+sn+pc+pi+ais+ perceived risk (pr), (f) ac+sn+pc+pi+ais+pr+face value (fv), (g) ac+sn+pc+pi+ais+pr+fv+distance, (h) all items in one factor table 4: comparative analysis for independent factors (1st stage data collection) figure 3: final framework after scale purification australasian journal of information systems nayal & pandey 2020, vol 24, research article digital coupon redemption 14 6.5 convergent and discriminant validity a validity analysis was conducted after developing the measurement model with dependent and independent factors. convergent validity is calculated by checking whether the maximum likelihood loading of each item is significant to its underlying factors (peter, 1981; arnold and reynolds, 2003). for all the scale items of dependent and independent factors, factor loading exceeded the minimum threshold value of 0.50 (tanwar and prasad, 2017). the dependent and independent factors had a cr value above 0.70 and an ave above 0.50, indicating good convergent validity (fornell and larcker, 1981) (table 5). discriminant validity was calculated by comparing the shared variance between factors with the ave value of individual factors (fornell and larcker, 1981). discriminant validity is acceptable if the maximum shared variance is less than ave (msv<ave), and the average shared variance is less than ave (asv<ave). factors cr ave msv asv intention to redeem digital coupons 0.88 0.66 0.40 0.25 perceived coupon value 0.85 0.66 0.40 0.25 coupon proneness 0.86 0.68 0.28 0.22 intention to search 0.83 0.62 0.24 0.17 attitude towards coupon 0.84 0.65 0.59 0.26 subjective norms 0.87 0.63 0.23 0.13 consumption frequency 0.72 0.50 0.46 0.23 distance from multiplex 0.80 0.50 0.31 0.18 attitude towards internet searching 0.80 0.51 0.37 0.18 perceived risk 0.78 0.53 0.26 0.073 face value 0.69 0.54 0.43 0.22 personal innovativeness 0.76 0.51 0.49 0.25 perceived convenience 0.77 0.54 0.53 0.26 table 5: validity statistics for 1st stage data collection 7 validation (2nd stage data collection) we replicated the cfa on an independent sample to minimize the probability of any error (maccallum et al., 1992) and validate the model. 7.1 replication of cfa professionals from different industries, such as manufacturing, defence, hospitality, it, and banking, were asked to complete a survey comprising 45 items that accounted for the 13 factors obtained after the first stage of scale purification. a total of 416 completed surveys were received, and because 5 were unusable, we had a final sample of 411 surveys. cfa was replicated in order to validate the scale on a new independent sample for dependent and independent variables separately. the results of the cfa showed a good fit (table 6). the lowest χ2/df and rmsea and the highest cfi and gfi (sweeney and soutar, 2001). australasian journal of information systems nayal & pandey 2020, vol 24, research article digital coupon redemption 15 fit indices criteria dependent cfa (fit indices) independent cfa (fit indices) χ2/df <3 2.106 2.011 agfi >0.80 0.907 0.816 cfi >0.90 0.965 0.918 gfi >0.90 0.942 0.852 rmsea <0.05 0.060 0.057 table 6: model fit statistics (2nd stage data collection) χ2/df agfi cfi gfi rmsea 4-factor model 3.143 0.933 0.963 0.957 0.058 3-factor model (a) 4.712 0.899 0.933 0.931 0.076 2-factor model (b) 10.548 0.756 0.822 0.828 0.122 1-factor model (c) 13.558 0.695 0.762 0.782 0.140 (a) intention to redeem (ir) + perceived coupon value (pcv), (b) ir+pcv+ coupon proneness (cp), (c) ir+pcv+cp+ intention to search table 7: comparative analysis for dependent factors (2nd stage data collection) χ2/df agfi cfi gfi rmsea 9-factor model 2.601 0.873 0.931 0.897 0.050 8-factor model (a) 4.646 0.758 0.839 0.800 0.076 7-factor model (b) 4.911 0.726 0.824 0.770 0.078 6-factor model (c) 5.252 0.720 0.806 0.762 0.082 5-factor model (d) 6.092 0.682 0.766 0.726 0.089 4-factor model (e) 6.994 0.629 0.722 0.678 0.097 3-factor model (f) 7.397 0.620 0.701 0.668 0.100 2-factor model (g) 8.073 0.581 0.668 0.633 0.105 1-factor model (h) 8.581 0.568 0.643 0.620 0.109 (a) attitude towards coupon (as)+ subjective norms (sn), (b) as+sn+perceived convenience (pc), (c) as+sn+pc+personal innovativeness (pi), (d) as+sn+pc+pi+attitude toward internet searching (ais), (e) as+sn+pc+pi+ais+perceived risk (pr), (f) as+sn+pc+pi+ais+pr+face value (fv), (g) as+sn+pc+pi+ais+pr+fv+distance, (h) as+sn+pc+pi+ais+pr+fv+distance+consumption frequency. table 8: comparative analysis for independent factors (2nd stage data collection) further, we compared the proposed 4-factor and 9-factor models with other models. table 7 and table 8 show that that the 4-factor model and the 9-factor model showed the best fit, with the lowest χ2/df and rmsea and the highest cfi and gfi (sweeney and soutar, 2001) (see online appendix). australasian journal of information systems nayal & pandey 2020, vol 24, research article digital coupon redemption 16 7.2 convergent and discriminant validity in order to assess the validity of the process followed to develop the scale, the purification step was repeated. factor loading for all the items exceeded 0.50(hair et al., 2006; fornell and larcker, 1981). further, the cr and ave values were above 0.70 and above 0.50, respectively, for all the factors (fornell and larcker, 1981). the discriminant validity of the factors was confirmed as the maximum shared variance (msv<ave) and the average shared variance were both less than ave (asv<ave) (table 9). factors cr ave msv asv intention to redeem 0.84 0.56 0.49 0.48 perceived coupon value 0.70 0.54 0.38 0.40 coupon proneness 0.79 0.56 0.51 0.47 intention to search 0.78 0.54 0.37 0.32 attitude towards coupon 0.85 0.65 0.44 0.32 subjective norms 0.86 0.60 0.55 0.29 consumption frequency 0.85 0.59 0.44 0.38 distance 0.83 0.62 0.55 0.35 attitude towards internet searching 0.80 0.52 0.50 0.28 perceived risk 0.79 0.54 0.33 0.11 face value 0.73 0.51 0.44 0.38 personal innovativeness 0.75 0.50 0.49 0.24 perceived convenience 0.85 0.58 0.42 0.45 table 9: validity statistics 8 discussion and conclusions this study presents the first measurement instrument for digital coupon redemption using the rigorous procedure of scale development. results from two surveys and tests for construct, convergent, and discriminant validity confirmed the robustness of the developed scale. the proposed scale consists of 45 items that measure 13 factors (intention to redeem, intention to search, perceived coupon value, coupon proneness, attitude toward coupons, subjective norms, consumption frequency, distance, attitude toward internet searching, perceived risk, face value, personal innovativeness, and perceived convenience). the developed scale lends support to the findings of previous studies on coupon redemption. for instance, chen and lu (2011) showed that coupon proneness, attitude toward coupons, attitude toward internet searching, and subjective norms are crucial factors influencing users’ intention to redeem digital coupons. danaher et al. (2015) showed that face value, distance, and perceived convenience were factors affecting redemption. liu et al. (2015) found that perceive value, coupon proneness, and personal innovativeness positively influence the intention to redeem digital coupons whereas perceived risk negatively influences it. in addition to above factors, the present study considers important new variables such as intention to search and consumption frequency, which also play a role in coupon redemption. by operationalizing the intention to redeem digital coupons, the present study makes a novel contribution to coupon literature. australasian journal of information systems nayal & pandey 2020, vol 24, research article digital coupon redemption 17 8.1 implications for theory the proposed scale captures a variety of variables relevant to the redemption of digital coupons. it contributes to marketing literature by drawing attention to coupon redemption— a construct that has the potential to increase sales, enhance brand value, and attract new customers but has not been operationalized till date. second, this study introduces a new variable—intention to search. this provides new link between: (1) intention to redeem coupons and coupon proneness and (2) intention to redeem coupons and perceived coupon value. these new theoretical linkages offer a fresh perspective on digital coupon redemption. third, in line with previous literature, the scale confirms that both hedonic benefits (coupon proneness) and utilitarian benefits (perceived coupon value) (sweeney and soutar, 2001) influence the consumer’s decision to redeem coupons (clark et al., 2013). fourth, the proposed scale demonstrates that consumers redeem digital coupons more for hedonic benefits (coupon pronenesssubjective norms, innovativeness, and attitude toward internet and coupons) than utilitarian benefits (perceived coupon valueperceived convenience). the reason could be digital coupons are accessed online and it is more fun activity for the consumers than just the money-saving activity. danaher et al. (2015) found that the redemption rate of digital coupons increases if they are delivered at the right time and place. fifth, in addition to factors that positively influence digital redemption, this is the first scale on coupons that includes two negative variables: perceived risk and distance. previous studies argued that one cannot explain digital coupon redemption by considering only the positive variables as technological advancements also come with certain costs. this study has considered perceived risk as negative variables, thereby filling the gap in the literature (im and ha, 2015). finally, the study also raises the following questions, which can be addressed by future researchers: • hedonic and utilitarian benefits are crucial for coupon redemption. are both of them equally important, or is one of them more important than other? • how can the scale be used by the services and manufacturing industries? • does consumer perception about redemption vary across different product categories? • are the factors relevant to the redemption of traditional coupons different from those for digital coupons? • which positive or negative factors play a dominant role in coupon redemption? 8.2 implications for practice our findings can help strengthen the strategies used to increase digital coupon redemption. first, the scale can be used by marketers to gain insights into the factors that influence coupon redemption. second, intention to search for digital coupon can be enhanced by making navigation of the coupon website as an enjoyable and fun activity. therefore, marketers should make the website interesting by introducing various innovative features. third, the redemption experience can be made enjoyable through innovative messages and emojis. emarketers should ensure that coupon redemption is a hassle-free experience. the marketers can also work on end to end coupon redemption journey. the marketers should explore the methods to make it seamless and convenient thus reducing the customer efforts in redemption leading to customer delight. moreover, consumer preferences such as preferred time and place (danaher et al., 2015) should be incorporated into coupon campaigns to make the redemption process more convenient. fourth, coupon redemption may increase if e-marketers invest australasian journal of information systems nayal & pandey 2020, vol 24, research article digital coupon redemption 18 efforts in making the journey of coupon redemption more exciting instead of focusing only on offering discounts (ieva et al., 2018). fifth, e-marketers can deliver coupons to consumers who purchase products more frequently as their chances of redemption may be higher. sixth, e-marketers should not waste their promotional budget on consumers who live far from the coupon redemption location as long distances are negatively correlated with redemption intention. lastly, to mitigate the perceived risk factors in an online environment, e-marketers should design websites and policies that address consumers’ privacy, security, and other concerns (pandey and gudipudi, 2019). customized coupons delivered at schedule times (mantel and kellaris, 2003) are known to reduce the fear of spam among consumers (mogilner and aaker, 2009). moreover, marketers can track consumer browsing history with analytics and send the coupons accordingly. it will also reduce the spams in the consumer’s inbox. 8.3 avenues for further research the reliable and valid scale for coupon redemption presented here is not without limitations. in fact, it serves as a good starting point for researchers and practitioners to conduct more insightful investigations into the redemption phenomenon. one of the main limitations of the data collected for scale development is that it is cross-sectional in nature, and it measures intention rather than actual behaviour. in future, researchers can collect longitudinal data from consumers who use digital coupons to refine the scale and observe how users’ redemption intention evolves with changes in technology and experience. it would also be interesting to develop a scale that is based on actual redemption behaviours of the consumers. finally, this scale is based on data obtained from consumers in an emerging market. given that user behaviour changes with context, it may be useful to replicate the scale development process with consumers in a developed country. references agarwal, r., & prasad, j. (1998). a conceptual and operational definition of personal innovativeness in the domain of information technology. information systems research, 9(2), 204-215. ajzen, i. (1985). from intentions to actions: a theory of planned behavior. in action control (pp. 11-39). springer, berlin, heidelberg. arnold, m. j., & reynolds, k. e. (2003). hedonic shopping motivations. journal of retailing, 79(2), 77-95. bagozzi, r. p., baumgartner, h., & yi, y. (1992). appraisal processes in the enactment of intentions to use coupons. psychology & marketing, 9(6), 469-486. banerjee, s., & yancey, s. (2010). enhancing mobile coupon redemption in fast food campaigns. journal of research in interactive marketing. beeck, i., & toporowski, w. (2017). when location and content matter: effects of mobile messages on intention to redeem. international journal of retail & distribution management, 45(7/8), 826-843. bowman, r. d. (1980). couponing and rebates: profit on the dotted line. lebhar-friedman books. chen, m. f., & lu, t. y. (2011). modeling e-coupon proneness as a mediator in the extended tpb model to predict consumers' usage intentions. internet research, 21(5), 508-526. australasian journal of information systems nayal & pandey 2020, vol 24, research article digital coupon redemption 19 chi, t., & kilduff, p. p. (2011). understanding consumer perceived value of casual sportswear: an empirical study. journal of retailing and consumer services, 18(5), 422-429. chiou‐wei, s. z. (2004). the determinants of direct mail coupon usage revisited: evidence from count panel data models. agribusiness: an international journal, 20(2), 189-200. chiou-wei, s. z., & inman, j. j. (2008). do shoppers like electronic coupons?: a panel data analysis. journal of retailing, 84(3), 297-307. churchill jr, g. a. (1979). a paradigm for developing better measures of marketing constructs. journal of marketing research, 16(1), 64-73. clark, r. a., zboja, j. j., & goldsmith, r. e. (2013). antecedents of coupon proneness: a key mediator of coupon redemption. journal of promotion management, 19(2), 188-210. colwell, s. r., aung, m., kanetkar, v., & holden, a. l. (2008). toward a measure of service convenience: multiple-item scale development and empirical test. journal of services marketing, 22(2), 160-169. corbin, j. m., & strauss, a. (1990). grounded theory research: procedures, canons, and evaluative criteria. qualitative sociology, 13(1), 3-21. danaher, p. j., smith, m. s., ranasinghe, k., & danaher, t. s. (2015). where, when, and how long: factors that influence the redemption of mobile phone coupons. journal of marketing research, 52(5), 710-725. dang, d., pittayachawan, s., & nkhoma, m. (2015). demystifying online personas of vietnamese young adults on facebook: a q-methodology approach. australasian journal of information systems, 19, 1-22. dickinger, a., & kleijnen, m. (2008). coupons going wireless: determinants of consumer intentions to redeem mobile coupons. journal of interactive marketing, 22(3), 23-39. featherman, m. s., miyazaki, a. d., & sprott, d. e. (2010). reducing online privacy risk to facilitate e-service adoption: the influence of perceived ease of use and corporate credibility. journal of services marketing, 24(3), 219-229. fornell, c., & larcker, d. f. (1981). evaluating structural equation models with unobservable variables and measurement error. journal of marketing research, 18(1), 39-50. fortin, d. r. (2000). clipping coupons in cyberspace: a proposed model of behavior for deal‐ prone consumers. psychology & marketing, 17(6), 515-534. garretson, j. a., & burton, s. (2003). highly coupon and sale prone consumers: benefits beyond price savings. journal of advertising research, 43(2), 162-172. ha, y., & im, h. (2014). determinants of mobile coupon service adoption: assessment of gender difference. international journal of retail & distribution management, 42(5), 441-459. hair, j. f., black, w. c., babin, b. j., anderson, r. e. & tatham, r. l. (2006): multivariate data analysis. auflage, upper saddle river, nj. hinkin, t. r. (1995). a review of scale development practices in the study of organizations. journal of management, 21(5), 967-988. honea, h., & dahl, d. w. (2005). the promotion affect scale: defining the affective dimensions of promotion. journal of business research, 58(4), 543-551. australasian journal of information systems nayal & pandey 2020, vol 24, research article digital coupon redemption 20 hsu, m. h., chang, c. m., chu, k. k., & lee, y. j. (2014). determinants of repurchase intention in online group-buying: the perspectives of delone & mclean is success model and trust. computers in human behavior, 36, 234-245. huang, h. c., chang, y. t., yeh, c. y., & liao, c. w. (2014). promote the price promotion: the effects of price promotions on customer evaluations in coffee chain stores. international journal of contemporary hospitality management, 26(7), 1065-1082. ieva, m., de canio, f., & ziliani, c. (2018). daily deal shoppers: what drives social couponing?. journal of retailing and consumer services, 40, 299-303. im, h., & ha, y. (2012). who are the users of mobile coupons? a profile of us consumers. journal of research in interactive marketing, 6(3), 215-232. im, h., & ha, y. (2013). enablers and inhibitors of permission-based marketing: a case of mobile coupons. journal of retailing and consumer services, 20(5), 495-503. im, h., & ha, y. (2015). is this mobile coupon worth my private information? consumer evaluation of acquisition and transaction utility in a mobile coupon shopping context. journal of research in interactive marketing, 9(2), 92-109. jiménez, f. r., & mendoza, n. a. (2013). too popular to ignore: the influence of online reviews on purchase intentions of search and experience products. journal of interactive marketing, 27(3), 226-235. kaiser, h. f., & rice, j. (1974). little jiffy, mark iv. educational and psychological measurement, 34(1), 111-117. kang, h., hahn, m., fortin, d. r., hyun, y. j., & eom, y. (2006). effects of perceived behavioral control on the consumer usage intention of e‐coupons. psychology & marketing, 23(10), 841-864. khajehzadeh, s., oppewal, h., & tojib, d. (2015). mobile coupons: what to offer, to whom, and where?. european journal of marketing, 49(5/6), 851-873. kumar, r., sachan, a., & kumar, r. (2020). the impact of service delivery system process and moderating effect of perceived value in internet banking adoption. australasian journal of information systems, 24. lichtenstein, d. r., netemeyer, r. g., & burton, s. (1990). distinguishing coupon proneness from value consciousness: an acquisition-transaction utility theory perspective. journal of marketing, 54(3), 54-67. lindell, m. k., & whitney, d. j. (2001). accounting for common method variance in crosssectional research designs. journal of applied psychology, 86(1), 114. liu, f., zhao, x., chau, p. y., & tang, q. (2015). roles of perceived value and individual differences in the acceptance of mobile coupon applications. internet research, 25(3), 471495. maccallum, r. c., roznowski, m., & necowitz, l. b. (1992). model modifications in covariance structure analysis: the problem of capitalization on chance. psychological bulletin, 111(3), 490. australasian journal of information systems nayal & pandey 2020, vol 24, research article digital coupon redemption 21 madlberger, m., & matook, s. (2017). theorizing e-commerce business models: on the impact of partially and fully supported transaction phases on customer satisfaction and loyalty. australasian journal of information systems, 21. mantel, s. p., & kellaris, j. j. (2003). cognitive determinants of consumers' time perceptions: the impact of resources required and available. journal of consumer research, 29(4), 531538. mills, p., & zamudio, c. (2018). scanning for discounts: examining the redemption of competing mobile coupons. journal of the academy of marketing science, 46(5), 964-982. mogilner, c., & aaker, j. (2009). “the time vs. money effect”: shifting product attitudes and decisions through personal connection. journal of consumer research, 36(2), 277-291. murphy, c., klotz, a. c., & kreiner, g. e. (2017). blue skies and black boxes: the promise (and practice) of grounded theory in human resource management research. human resource management review, 27(2), 291-305. nahm, a. y., vonderembse, m. a., & koufteros, x. a. (2004). the impact of organizational culture on time‐based manufacturing and performance. decision sciences, 35(4), 579-607. nayal, p., & pandey, n. (2020). redemption intention of coupons: a meta-analytical review and future directions. journal of promotion management, 1-24. nch report 2018, https://www.nchmarketing.com/2018-year-end-coupon-facts-at-aglance.aspx accessed on may 1, 2019 netemeyer, r. g., bearden, w. o., & sharma, s. (2003). scaling procedures: issues and applications. sage publications. nunnally, j., & bernstein, i. (1994). psychometric theory. new york, ny: mcgraw-hall. pandey, n., & gudipudi, b. (2019). understanding ‘what is privacy for millennials on facebook in india. journal of data protection & privacy, 2(3), 224-233. pandey, n., & maheshwari, v. (2017). four decades of coupon research in pricing: evolution, development, and practice. journal of revenue and pricing management, 16(4), 397-416. pandey, n., & singh, g. (2012). marketing issues in smes: cases from india. pearson education india. papadas, k. k., avlonitis, g. j., & carrigan, m. (2017). green marketing orientation: conceptualization, scale development and validation. journal of business research, 80, 236-246. parasuraman, a., zeithaml, v. a., & berry, l. l. (1988). servqual: a multiple-item scale for measuring consumer perc. journal of retailing, 64(1), 12. patwardhan, a. a., pandey, n., & dhume, s. m. (2017). integrated model for understanding indian physicians’ internet usage pattern: an empirical approach. international journal of healthcare management, 10(1), 19-33. patyal, v. s., & koilakuntla, m. (2015). infrastructure and core quality practices in indian manufacturing organizations: scale development and validation. journal of advances in management research, 12(2), 141-175. australasian journal of information systems nayal & pandey 2020, vol 24, research article digital coupon redemption 22 peter, j. p. (1981). construct validity: a review of basic issues and marketing practices. journal of marketing research, 18(2), 133-145. podsakoff, p. m., mackenzie, s. b., lee, j. y., & podsakoff, n. p. (2003). common method biases in behavioral research: a critical review of the literature and recommended remedies. journal of applied psychology, 88(5), 879. pura, m. (2005). linking perceived value and loyalty in location-based mobile services. managing service quality, 15(6), 509-538. raghubir, p. (2004). coupons in context: discounting prices or decreasing profits? journal of retailing, 80(1), 1-12. reichhart, p., pescher, c., & spann, m. (2013). a comparison of the effectiveness of e-mail coupons and mobile text message coupons for digital products. electronic markets, 23(3), 217-225. rhee, h., & bell, d. r. (2002). the inter-store mobility of supermarket shoppers. journal of retailing, 78(4), 225-237. shimp, t. a., & kavas, a. (1984). the theory of reasoned action applied to coupon usage. journal of consumer research, 11(3), 795-809. sieger, p., gruber, m., fauchart, e., & zellweger, t. (2016). measuring the social identity of entrepreneurs: scale development and international validation. journal of business venturing, 31(5), 542-572. souiden, n., chaouali, w., & baccouche, m. (2019). consumers’ attitude and adoption of location-based coupons: the case of the retail fast food sector. journal of retailing and consumer services, 47, 116-132. swaminathan, s., & bawa, k. (2005). category-specific coupon proneness: the impact of individual characteristics and category-specific variables. journal of retailing, 81(3), 205214. sweeney, j. c., & soutar, g. n. (2001). consumer perceived value: the development of a multiple item scale. journal of retailing, 77(2), 203-220. tanwar, k., & prasad, a. (2017). employer brand scale development and validation: a secondorder factor approach. personnel review, 46(2), 389-409. to, p. l., liao, c., & lin, t. h. (2007). shopping motivations on internet: a study based on utilitarian and hedonic value. technovation, 27(12), 774-787. wierich, r., & zielke, s. (2014). how retailer coupons increase attitudinal loyalty–the impact of three coupon design elements. european journal of marketing, 48(3/4), 699-721. copyright: © 2020 nayal & pandey. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v24i0.2469 http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems nayal & pandey 2020, vol 24, research article digital coupon redemption 23 paper title (paper title style) australasian journal of information systems lamp & burmeister 2016, vol 20 editorial 1 the conduct of a review john lamp ajis editor-in-chief deakin business school deakin university ajis.eic@gmail.com oliver k. burmeister school of computing and mathematics charles sturt university, bathurst the australasian journal of information systems (ajis) has been published since 1993 and is the premier journal in the information systems (is) discipline in this region of the world. that is reflected in its ranking as an “a” journal in both the 2013 australian business deans council (abdc) rankings and those of the australian council of professors and heads of information systems (acphis). it also holds the doaj seal for best open access publishing practice. clearly it is important to guard this ranking and one way of achieving that is with rigorous review of submitted articles. this is a special editorial addressing the review process. the primary purpose of this editorial is to promote a greater consistency and rigour amongst reviewers. a secondary purpose is to help contributing authors better understand the review process that their article will be subjected to. as with other journals, the first point of call after submission is that a section editor (se) reads the article and decides whether it is suitable for ajis or not. if not, a ‘desk’ rejection follows. if it is suitable, that section editor then assigns reviewers. for the purpose of this exercise dr burmeister, an ajis se, has agreed to have his ajis publications serve as examples. of the five articles he has published in ajis (bowern, burmeister, gotterbarn, & weckert, 2006; burmeister, 1995, 2015; burmeister, islam, dayhew, & crichton, 2015; simpson, nevile, & burmeister, 2003), only the two on scholarship are solo authored. so as not to embarrass co-authors, only the reviews of those two articles are detailed here; those two articles span almost the entire history of ajis. the first was a scholarship paper published over 20 years ago (burmeister, 1995), with reviewer responses having to be submitted to the editor via ‘diskette and hard copy’ (letter from the then editor, r. macgregor); back then the “a” in ajis was ‘australian’, not ‘australasian’. at that time ajis did not have its current online submission and review process, and instead manuscripts were mailed to reviewers, who in turn mailed back their recommendations (without including the original manuscript). instructions to reviewers were simple. a reviewer had to choose one of three options (accept, accept with minor modifications, reject) and had to send comments to the consulting editor (now se). for the latter the instructions were as follows: please detail on the page attached any praise or specific criticisms about the manuscript. these comments will be released to the authors. also, if appropriate, please complete the separate page of comments which may be of use to the editors. this information will not be communicated to the authors. such comments may support any criticisms that you have made above. that 1995 paper had gone to two reviewers, one recommending acceptance and the other acceptance with minor modifications. the process now is similar, but reviews are typically done by either two or three reviewers. as mentioned above, desk rejections are when an editorial decision is made that an article is not suited to the journal. the second scholarship paper (burmeister, 2015) had that experience. it had previously been submitted to a journal ranked a* in abdc, but received a desk rejection. it was then reformatted and submitted to australasian journal of information systems lamp & burmeister 2016, vol 20 editorial 2 ajis. before detailing the review process for this article, it is important to first describe the ajis process as it is now. upon submission the editor-in-chief firstly engages in a preliminary process, before assigning it to a se, a domain expert in the area most closely related to the submitted paper. the preliminary process involves ensuring the article matches the scope of the journal, the length is suitable, it adheres to ajis manuscript guidelines, and checking it for plagiarism using turnitin. the se then reads it and rejects it (desk rejection) or assigns it to at least two reviewers, to undergo blind review. the reviewers are volunteers and not paid for their time, hence a first step is to ask potential reviewers if they are willing to engage in the review (it involves sending them the title and abstract). potential reviewers have two weeks to respond. if, as frequently happens, one or more cannot commit to undertaking the review, then other potential reviewer requests are made by the se, and those people are again given two weeks to respond. once two or more have agreed to conduct the review, they are encouraged to perform their review within a six week period. at times that does not happen and in some cases the se has to find replacement or additional reviewers, which then extends the review period. upon receiving the reviewer comments, the se then makes a decision about the article and informs the author(s). that decision is made critically, not just taking reviewer recommendations, nor averaging them. one of four decisions is made: accept, revise, revise and resubmit, decline. revise is not the same as the 1995 example above of ‘accept with minor modifications’. instead, only after the revisions are received will the se make a decision to accept or not. returning to burmeister (2015), it underwent two rounds of review. the first involved three reviewers and resulted in a ‘resubmit for review’ now called a ‘revise and resubmit’ decision. the second round involved the same three reviewers and resulted in an ‘accept’ decision. however, accept does not mean accept as is, but instead, accept subject to minor revisions, which are assessed by the se. a rigorous review process is necessary to ensure the high standards of this a ranked journal are maintained. the following extract is from the first round of review, showing the assessment of one of the three reviewers. overall assessment: please provide your overall assessment of the publishability of the manuscript. please provide written feedback to the author/s and /or editors in the text boxes below: revisions required relevance to the journal: the manuscript falls within the scope of ajis and is of relevance to the journal's readership: highly relevant motivation: why this research is important from theoretical and practical viewpoints. what is the contribution: adequately motivated literature review: current state of the research in the area… the story so far… forms the basis for the research... may lead in to the theoretical section: literature review satisfactory theory: the theoretical development is logically sound and builds on a solidly established framework. e.g., development of a model or theoretical argument that leads to a series of testable propositions. the paper then sets out to prove or reject these propositions. alternatively, the paper may build theory based on observation: poor theoretical development australasian journal of information systems lamp & burmeister 2016, vol 20 editorial 3 research method: how the research was conducted, that the method is appropriate and justified: poor research method. conduct of the research: how the experiment was conducted or how the data was gathered and manipulated prior to analysis: satisfactorily conducted analysis of the findings: how the data were analysed, appropriate analysis methods and statistics used: analysis satisfactorily conducted implications drawn from the analysis: do the implications, both theory and practice follow from the results of the analysis? not over/understated.: implications are well handled conclusions and limitations: what is concluded, and how is it limited by the nature of the experimental procedures or the population from which data was gathered, or from limitations of analytical analyses. statement of contribution: conclusions and limitations satisfactorily drawn. grammar, spelling, format, and conformance with ajis standards: please detail any issues with this aspect of the manuscript in the text box below: feedback to authors: the paper reports on a research project investigating the experience of professional doctorate participants in the discipline area of information technology. the paper reports the findings and some conclusions from a survey of a broad range of stakeholders in the delivery of a doctor of information technology at charles sturt university (csu). the paper’s topic and findings are very interesting and well-suited to the readership of ajis. there are three areas that i think the author(s) might valuably ‘polish’ the work: (1) the literature review is quite good, but the paper does not provide a concise summary of the issues in the area that the literature reviews (it is only a summary statement that is missing), (2) the research method statement is too superficial, with no discussion about how the interviews were coded, analysed, or how the findings emerged from whatever analytical process took place (a more disciplined recounting of the data coding and analysis is warranted), and (3) the analytical findings are not summarised into the concise statement that the abstract implies (again, only a summary statement is required to address this issue). we can see from the above example that there is a standard form that all reviewers are expected to complete. in addition there are comments to the author(s) and optionally, comments to the se, which are not passed on to the author(s). use of this format ensures a uniform approach to the review process. most authors submitting articles to ajis and elsewhere find it difficult to cope emotionally with rejection and criticism. therefore we recommend the pip approach to reviewers; praise, improvement, praise. that is, start and end with something praise-worthy in the article. reviewers are encouraged to provide constructive feedback, even if they recommend ‘decline’. ajis considers this a hallmark of a quality publication and part of the scholarly, collegial process. that is, with appropriate feedback even rejected papers can be reworked and submitted elsewhere, for eventual publication. ideally the comments to the australasian journal of information systems lamp & burmeister 2016, vol 20 editorial 4 author(s) should include advice that can be acted upon, as is seen above, where a suggestion is made to have a summary of the findings. looking at the published version of the paper, there is a summary table at the end of the paper, which came about as a result of this recommendation from the reviewer. professor allen lee, a keynote speaker at acis this year, wrote an excellent article many years ago on the reviewing process, which is worth reading (lee, 1995). this is also available on the web at http://www.people.vcu.edu/~aslee/referee.htm professor lee mentions making the extent of your expertise clear, and that your comments are framed to be actionable. reviewers should specify their expertise in their review, eg “this reviewer’s research area is ontology, but focusing on the philosophy of information, not on the use of approaches such as ontolingua.” knowing the reviewer’s expertise allows authors to evaluate the relative strengths of the review comments. providing comments that are actionable makes a review useful, rather than an abstract opinion. for example, stating “the statistical method is inappropriate” is a valid comment, but “analysis using anova would be more appropriate” provides the same information, but in a form that allows the author(s) to not only see the issue that the reviewers have identified, but have some idea what they can do to improve the paper. figure 1: ajis article throughput 2010 to 2014 if you think of the need for good reviews of papers against the number of papers submitted to ajis some pretty dramatic issues emerge. figure 1 puts into a wider context. the profile of ajis in the world has dramatically changed over the past few years. figure 1 shows the increase in submissions, and the acceptance rate over the past five years. submissions have pretty much tripled, and the quality of papers submitted, as shown by the acceptance rate, have also improved. at the time of writing there are five newly received papers, 53 in review and four in copyediting. while that might seem pretty innocuous, the implications are that ajis is at this time using the services of about 45 section editors and over 150 reviewers. ajis prefers to call upon academics from the region, but does not hesitate to use the services of international experts, especially where sophisticated domain knowledge is required. despite casting the net wide, it is worth remembering that the index of information systems journals (lamp, 2004) lists 700 is journals at present. that requires a very large number of is researchers and academics to adequately review submitted articles. 0% 5% 10% 15% 20% 25% 0 50 100 150 200 2010 2011 2012 2013 2014 2015 a cc ep te d n u m b er o f a rt ic le s year ajis article throughput submitted percentage accepted australasian journal of information systems lamp & burmeister 2016, vol 20 editorial 5 participation in the review process is essential for the viability of delivering properly scrutinized research to the research community and to the wider community of practitioners and interested public. experienced researchers understand that necessity, and the obligation to participate in both ends of the process, however much of a strain it can be. post-graduate students should also participate as reviewers. post-graduate students benefit from participation in the review process by gaining a deeper understanding of the process, the standard of submissions before publication and, quite often, exposure to ideas in their area which they had not considered – it is definitely a two way process. post-graduate students often contribute a fresh approach to a topic, which is a benefit to the journal. section editors will ensure a balance of reviewers to deliver high quality reviews to the authors they are assisting. acting as a reviewer is essential for the continued existence of a journal in which to publish your research and has direct benefits to those reviewing. your participation is greatly appreciated. references bowern, m., burmeister, o. k., gotterbarn, d., & weckert, j. (2006). ict integrity: bringing the acs code of ethics up to date. australasian journal of information systems, 13(2), 168-181. burmeister, o. k. (1995). evaluating the factors that facilitate a deep understanding of data analysis. australasian journal of information systems, 3(1). burmeister, o. k. (2015). improving professional it doctorate completion rates. australasian journal of information systems, 19, 55-70. doi: 10.3127/ajis.v19i0.1073 burmeister, o. k., islam, m. z., dayhew, m., & crichton, m. (2015). enhancing client welfare through better communication of private mental health data between rural service providers. australasian journal of information systems, 19, 1-14. doi: http://dx.doi.org/10.3127/ajis.v19i0.1206 lamp, j. w. (2004). index of information systems journals. geelong, deakin university. [online] available: http://lamp.infosys.deakin.edu.au/journals/ lee, a. s. (1995). reviewing a manuscript for publication. journal of operations management, 13(1), 87-92. simpson, c., nevile, l., & burmeister, o. k. (2003). doing ethics: a universal technique in an accessibility context. australasian journal of information systems, 10(2). copyright: © 2016 lamp & burmeister. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems roohy gohar & indulska 2020, vol 24, research article environmental sustainability through green bpm 1 environmental sustainability through green business process management shahrzad roohy gohar uq business school university of queensland sh.roohygohar@business.uq.edu.au marta indulska uq business school university of queensland abstract the climate change phenomenon, directly or indirectly, affects industries and nations. governments and organizations have been challenged to identify their environmental impacts to address environmental sustainability issues. a promising, yet under-studied in this context, theme of information systems (is) literature that has the potential to help with identifying, quantifying and managing environmental impacts is business process management (bpm). with its focus on continual process performance improvement, the capacity of bpm to contribute to environmental sustainability (es) needs to be further explored. yet, contributions from the bpm research community and the impact of these contributions appear to be fragmented. in this paper, we present a systematic literature review to explore bpm contributions to es, with a focus on environmental performance indicators (epis) as well as relevant organizational factors related to es and bpm. in doing so, we identify and explore ‘green bpm’ contributions and suggest ways to advance bpm research in the context of es. keywords: environmental sustainability, green bpm, environmental performance indicators (epis) 1 introduction sustainability has, at its core, the principle of continual and mindful growth, supporting societies, and maintaining economic performance without compromising the natural environment (brown et al., 1987; wced, 1987). environmental sustainability (es) in particular, with its focus on sustaining the living (goodland, 1995), has become of increasing concern due to mankind’s exploitation of natural resources. recently, the united nation’s sustainable development goals (united nations, 2016) came into force with 175 world leaders agreeing to increase their efforts through research and practice to tackle climate change. as part of this initiative, organizations are required to proactively manage the use of their natural resources and continuously improve their environmental performance. such continual improvement of environmental performance is achievable through the identification and measurement of environmental performance indicators (epis). epis are metrics that organizations use to measure environmental performance and to calculate their impact on the environment (epstein & roy, 2001; ipcc, 2014; jamous & müller, 2013; jasch, 2000; young & rikhardsson, 1996). core epis include measures of water consumption, energy consumption, waste management, recycling, and co2 and ghg emissions, and are commonly used by a wide range of industry sectors (epstein & roy, 2001; ipcc, 2014; jamous & müller, 2013; jasch, 2000; young & rikhardsson, 1996). it is thus important to consider how these epis australasian journal of information systems roohy gohar & indulska 2020, vol 24, research article environmental sustainability through green bpm 2 can be managed and continually improved as organisations carry out their operations. in this context, we consider the field of business process management (bpm), which has at its core the continuous improvement of processes and their key performance indicators (kpis), and explore its contributions to es with a specific lens on epis. we refer to bpm research concerning es as ‘green bpm’. the term is used by seidel et al. (2012) to include understanding, documenting, executing and continually improving processes by focusing on their environmental impacts. broadly, green bpm supports the design and implementation of environmentally sustainable processes. in the context of is, opitz et al. (2014b) defined green bpm as all is-supported management activities that reduce the environmental impact of business processes, including design, improvement, process lifecycle and operational steps. the outlook of green bpm is promising, yet the body of research focusing on es remains lean (maciel, 2017). indeed, there is “a significant gap” between global calls to resolve climate change issues and what is discipline research offers (gholami et al., 2016), based on the number of is research studies in the last decade that focus on sustainability (chen et al., 2011; gholami et al., 2016). the same trend is observed in bpm academic literature addressing sustainability (hernández gonzález et al., 2019). it is also unclear to what extent green bpm has been studied and how it has contributed to the es body of knowledge. to explore this notion, we conduct a systematic literature review (slr) (kitchenham et al., 2009) of green bpm literature. our review aims to identify relevant contributions of bpm to epis and organizational factors relating to es (brooks et al., 2012) and to understand the extent of bpm research contribution to es. thus, our study explores how epi and es concepts have been addressed in bpm literature and suggests further research opportunities. our study also explores the extent of alignment of these contributions with the focus of industry on various epis. understanding this alignment is particularly important as it is an indicator of the relevance of green bpm research to the needs of industry (applegate & king, 1999; rosemann & vessey, 2008). the structure of the paper is as follows. section 2 provides a background to es and bpm. related studies that identify bpm contributions to es are summarized in section 3. our methodology is presented in section 4. section 5 discusses the results of our systematic literature review and contrasts these with an industry focus. finally, sections 6 and 7 propose next steps and offer final remarks. 2 background the calls to focus on the environment and to change industrial practices began with the brundtland report published by the world commission for environment and development (1987). since then, sustainability-related studies have accelerated in various research disciplines (dehghanian & mansour, 2009). while some researchers consider ecological concerns or social responsibility as sustainability (linnenluecke & griffiths, 2010), current research looks more to the relationship between the three pillars of corporate sustainability (elkington, 1998; epstein & roy, 2001; savitz, 2012). namely: (1) economic, referring to economic capital; (2) social, referring to the equitable redistribution of resources; and (3) environmental, referring to natural capital (goodland, 1995). these pillars of corporate sustainability are referred as the “triple bottom line” (elkington, 1998; savitz, 2012). since the late 1990s, sustainability studies have focused on these pillars and applied them to advancing technologies and growing consumer demand. our focus in this study is on the environmental australasian journal of information systems roohy gohar & indulska 2020, vol 24, research article environmental sustainability through green bpm 3 pillar and in the context of information systems research, specifically business process management. 2.1 environmental sustainability and information systems while information technology (it) is responsible for producing a large volume of air and water-polluting waste in its manufacture, operations and disposal, it also allows resource use to be captured, monitored and controlled in all areas of business and industry. indeed, utilizing it and is for this purpose is expected to reduce carbon emissions in businesses by up to 15 percent by 2020 (lacy et al., 2010), thus contributing positively to es. increasing pressure from environmental legislation, and from consumers, motivates organizations to reduce the environmental impact of their practices (bansal & roth, 2000; lintukangas et al., 2016; rajeev et al., 2017), motivating businesses to further invest in is solutions (lane et al., 2011) and implement environmental management systems (ems) (tinsley & pillai, 2006; whitelaw, 2004). an ems encompasses the organizational structure, technological resources, practices, processes and information resources for determining and implementing environmental policies (british standards institution, 1994). many organizations have implemented an ems to improve their compliance with environmental regulations and to reduce their environmental impact and improve their reputation (sullivan & wyndham, 2001; tinsley & pillai, 2006). indeed by 2013, businesses compliant with the iso14001 standard were spread across 170 countries (international organisation for standardization, 2013). iso14001 provides organizations with a framework to develop their environmental objectives, measure the environmental impact of their operations, and monitor improvement and compliance with environmental regulations (melnyk et al., 2003; phan & baird, 2015; wathern, 2013; whitelaw, 2004). it is not the only example of a standard that provides is solutions, integrating es into all aspects of business. numerous is-aided solutions and approaches exist to assist estimating the environmental impact of products, services and operations, such as lifecycle assessment (lca) (guinée, 2001) and environmental input-output analysis (eio). lca, which is also part of iso14000, provides a method to assess the impact of materials from their initial use to their disposal (guinée, 2001), whereas eio provides an approach to estimate the carbon footprint from epis (finnveden et al., 2009). these methods individually are not holistic and, therefore, are required to be combined to sufficiently assess the environmental impact of products and services (finnveden et al., 2009). neither, however, provides a way to measure and manage the environmental impacts of business operations. yet, organizations adopt process-oriented approaches, where a ‘process’ is a sequence of activities, events and decisions that directly or indirectly deliver value to the organization (dumas et al., 2013), to manage the performance of their operations (kohlbacher & reijers, 2013) and align their operations with organizational strategies and culture. 2.2 environmental sustainability and bpm bpm is well-placed to address es management challenges because it focuses on improving organizational efficiency (rehan et al., 2018) and operational effectiveness (hammer, 2010), by optimizing processes, technology use, reducing waste and improving performance (rosemann & vom brocke, 2010). it is a holistic management approach that facilitates managing operations, which enact an organization’s strategy to deliver value (de burgos jimenez & céspedes lorente, 2001; longoni & cagliano, 2015), and continuously improving australasian journal of information systems roohy gohar & indulska 2020, vol 24, research article environmental sustainability through green bpm 4 the economic sustainability of an organization, thus advancing the organization’s competitive advantage (hung, 2006). process visualization, measurement, analysis, improvement and monitoring are core to bpm. they drive overall organizational performance and augment economic resources (weske, 2012) and natural capital (e.g., time, cost and raw materials). such resources become key performance indicators (kpis). a collective and clear understanding of kpis and process resources facilitates the measurement and analysis of process performance in organizations (rosemann & vom brocke, 2015). operational and process management approaches, including the bpm lifecycle, are employed to measure and optimize kpis (kohlbacher & reijers, 2013; sanders jones & linderman, 2014). improvement can therefore take on different meanings based on organizational strategies and objectives (sanders jones & linderman, 2014). for example, improvement could bring reduced costs and process execution time or improved quality and productivity, or reduction of environmental impact. in the es context, epis of organizational processes can be measured through a bpm approach to reduce the environmental impact of organizational activities (parmenter, 2010; wetzstein et al., 2008). in other words, kpis can be extended to include epis, and bpm analysis techniques can be applied to identify, measure, improve and monitor the environmental performance of business processes. therefore, bpm has strong links with es due to its process-oriented approach and facilitation of measuring and analysing epis at the process level (detoro & mccabe, 1997; ghose et al., 2010). such managed and controlled use of environmental resources, together with increasing organizational capabilities, ultimately results in a competitive advantage for the organization (porter, 2008; porter & van der linde, 1995; schmidheiny, 1992). 3 related work stolze et al. (2012) reviewed 2006–2011 literature from english and non-english sources in the is and bpm disciplines. they identified and categorized the literature as ‘green it’, ‘green is’, ‘sustainable’, and ‘business process’. while the study provides high-level categories, it does not explore how bpm contributes to es. through examining 127 research papers, opitz et al. (2014a) explored the potential of measuring an organization’s ability to implement green bpm. by classifying the papers according to ‘attitude’, ‘strategy’, ‘governance’, ‘modelling’, ‘optimizing’, and ‘monitoring’, these authors proposed a green bpm readiness model motivated by the green ict readiness model (wabwoba et al., 2013). however, the study did not identify specific bpm concepts nor es concepts to which contributions are made and, thus, offers a narrower focus. subsequently, opitz et al. (2014b) categorized the literature into ‘green it/is’, ‘bpm’, ‘green bpm’ as well as categories relating to ‘reduce environmental impact’, ‘monitoring’, ‘economical’, ‘cultural change’ and ‘definition’. however, the extent of the theoretical contributions and details of bpm and es concepts covered in the study is unspecified. moreover, there is a lack of attention to epis which are necessary for measuring the environmental impact of organizations and monitoring performance improvement. maciel (2017) reviewed green bpm literature according to six bpm components: strategic alignment, governance, method, information technology, people and culture, to extend the means to address es and green initiatives. he found that green bpm research literature has concentrated on bpm lifecycle steps such as design, measurement, and improving es processes while paying little attention to strategic alignment, governance, people and culture. an initial taxonomy of sustainability in business process models was proposed by schoormann et al. (2017), while couckuyt (2017) focussed on the business process lifecycle in australasian journal of information systems roohy gohar & indulska 2020, vol 24, research article environmental sustainability through green bpm 5 the green bpm context. couckuyt and van looy (2019) conducted an slr on the contribution of bpm, operations management and is publications to es. the study identified the number of research studies from various disciplines, the range of environmental topics, the affiliations contributing to es, and the types of scholarly contributions. our study offers a different perspective. it offers a single consolidated resource of green bpm literature using the lens of bpm concepts, organizational factors and epis (dada et al., 2013) to explain current contributions. identifying, measuring and monitoring epis is crucial because organizations use them to analyse their environmental impact and monitor their performance. epis, if measured and reported correctly, influence an organization’s strategic decisions and provide transparent and meaningful information about its environmental impact. such data can be analysed over time to assess improvement and compliance of firms with environmental regulations and standards. 4 methodology we use a systematic literature review approach to carry out our study. we do so because an slr assists with the identification of contributions and pinpoints research gaps related to the phenomenon under study (kitchenham et al., 2009). slr allows researchers to extract and evaluate the available research on a particular phenomenon of interest (kitchenham et al., 2009) and serves as a foundation to the research in the field (webster & watson, 2002). to conduct our slr, we follow guidelines from bandara et al. (2015) to ensure our review provides an exhaustive and comprehensive insight into bpm contributions related to es. because the earliest es-related academic paper in is was published in 2006, while research into es in bpm prior to 2005 is non-existent, our study covers the time period of 2005 to 2019. to identify as many relevant articles as possible, we relied on google scholar rather than purely is or bpm publication outlets, after identifying keywords via an initial is and bpm data set. in addition to identifying academic papers, we identified several authoritative industry reports on es issues and challenges to allow us to contrast industry focus with academic contributions. the search strategies for both types of contributions are outlined in the following sub-section. 4.1 search strategy to identify search terms relevant for our study, we first explored es-related keywords, variations and word-stems in use specifically by the is and bpm community. we used a set of publications in prominent is journals and high-quality conferences, because this is where most is and bpm publications can be found. our data set included the association for information systems (ais) senior scholars’ basket of eight journals1, the business process management journal (bpmj) and top is and bpm conferences2. we reviewed the is and bpm literature for frequently used es-related keywords and identified ‘green’, ‘environmentally sustainable’, ‘sustainability’ and ‘sustainable’, as key terms. we 1 european journal of information systems (ejis), information systems journal (isj), information systems research (isr), journal of ais (jais), journal of information technology (jit), journal of management information systems (jmis), journal of strategic information systems (jsis), and mis quarterly. 2 pacific asia conference on information systems (pacis), european conference on information systems (ecis), international conference on information systems (icis), american conference on information systems (amcis), australasian conference on information systems (acis), and business process management conference (bpm). australasian journal of information systems roohy gohar & indulska 2020, vol 24, research article environmental sustainability through green bpm 6 also considered stemmed searches using derivations and variations of the terms ‘environmental’ and ‘sustainability’. through keeping the search terms broad and combining them with boolean operators (see table 1), we identified a pool of papers to analyse to identify more specific es-related terms used by is and bpm researchers. broad bpm/is search terms boolean operators broad es terms information systems (is) and environmentally sustainable bpm es business process reengineering (bpr) green environmental, environmentally, environment*, sustainable, sustainability, sustain* narrow bpm/is search terms narrow es search terms process carbon-footprint, ghg emission, energy, ecological, environmentally-aware, carbon-aware, energy-aware table 1 search terms and operators via a manual review, we observed that some papers did not specifically mention es in their title, yet still focused on es (e.g. ‘carbon-footprint (co2)’, energy, ‘ghg emission’ and ‘ecological’). therefore, we included these terms to ensure our later search using google scholar was inclusive. terms such as ‘environmentally-aware’, ‘carbon-aware’, ‘energyaware’ were also used to identify search terms. by identifying and choosing the most-used es terms in is and bpm publications, we derived a list of relevant search terms (see table 1). after identifying these terms, we combined stemmed terms to conduct searches using google scholar to identify publications as our primary collection of literature. we searched only articles by authors who used titles that combined our bpm and es keywords as we considered these articles to have a central focus on es and bpm. then, we iteratively performed forward and backward searches, exploring bibliographic references and authors in retrieved publications. through this process we identified 269 journal and conference papers containing green bpm, green it and green is contributions. by maintaining the same approach of first considering article titles, we created three categories of papers. forty-nine publications matching es relevant keywords and business process/bpm were considered to be within our tier 1 publications. subsequently, 122 papers with titles containing es related terms and is were ranked as tier 2, while 98 papers focused on es and it were ranked as tier 3 papers. we excluded tiers 2 and 3 to maintain our focus on es and process/bpm. table 2 presents our criteria for including and excluding papers. to ensure an exhaustive collection of bpm-focused articles, and assuming some is-focused articles may contain bpm-related concepts, we double-checked excluded papers. specifically, we conducted a full-text search for the term ‘process’ in the 122 tier 2 articles, which resulted in 20 matching articles. however, because a manual review revealed none focused on bpm, they were not included in our analysis. accordingly, using forward and backward searches, overall, we identified 49 relevant academic papers. to ensure an exhaustive collection of bpm-focused articles, and assuming some is-focused articles may contain bpm-related concepts, we double-checked excluded papers. specifically, we conducted a full-text search for the term ‘process’ in the 122 tier 2 articles, which resulted australasian journal of information systems roohy gohar & indulska 2020, vol 24, research article environmental sustainability through green bpm 7 in 20 matching articles. however, because a manual review revealed none focused on bpm, they were not included in our analysis. accordingly, using forward and backward searches, overall, we identified 49 relevant academic papers. table 2 inclusions and exclusions to identify industry es challenges, we performed a google search using the keywords: ‘environmentally sustainable’, ‘es’ and ‘environmental performance indicators (epi)’. our resulting corpus included six authoritative agendas and annual reports. these were reports from global bodies (e.g., the united nations [un]), peak industry bodies and, for a local perspective, the australian government: • the australian government sustainability plan (2010–2015) presents a plan to reduce carbon emissions and improve the sustainability of ict operations (commonwealth of australia, 2010). • a new era of sustainability: un global compact — accenture’s 2010 ceo study reports the challenges within the journey toward a sustainable economy (lacy et al., 2010). • the 2014 annual report by the united nations environment programme (2014) highlights global development and challenges to overcome climate change and other sustainability issues. • the synthesis report on climate change presents findings of three working groups that contributed to the fifth assessment report (ar5) of the intergovernmental panel on climate change (ipcc): the most thorough assessment of climate change undertaken by the ipcc (2014). • the sustainable australia report 2013 covers emerging issues and major trends for australia’s sustainability, including es (national sustainability council, 2013). • es and industry: road to a sustainable future is the largest survey of environmental practices developed from findings of the national sustainability council (2013). it was conducted by the australian industry group in conjunction with sustainability victoria (australian industry group, 2007). 4.2 coding and content analysis after compiling our corpus of 49 academic papers (see appendix), we coded it to identify the significance and frequency of concepts contained within. we looked for search terms, relevant phrases, theoretical constructs and proposed research artefacts. specifically, we looked for inclusion tier and criteria tier 1: green bpm – articles with a focus on bpm and es (49 papers) industry reports with a focus on es (6 reports) exclusion tier and criteria tier 2: green is (122 papers) tier 3: green it (98 papers) articles did not match the inclusion criteria non-english articles articles containing search terms such as bpm and ‘process’ but not relevant to our focus and inclusion criteria australasian journal of information systems roohy gohar & indulska 2020, vol 24, research article environmental sustainability through green bpm 8 relevant bpm concepts addressed, advanced and developed, as well as any of the epis and es concepts addressed, studied, tested and or implemented using process-oriented methods. to do so, we first identified a set of coding criteria, using an excel spreadsheet. we began with basic codes, resulting in spreadsheet columns headed, ‘title’, ‘year of publication’, ‘main contribution’, ‘bpm concept’, ‘research methodology’, ‘data collection method’, ‘data analysis method’, ‘assumptions’ and ‘limitations’. in our first round of analysis, the es and bpm specific coding criteria were based primarily on sustainability keywords identified from key papers in the sustainability literature and frequently mentioned concepts in bpm academic publications (australian industry group, 2007; epstein & roy, 2001; goodland, 1995; hammond et al., 1995; hoesch-klohe & ghose, 2012; jasch, 2000). we based the main themes of these papers on ‘types of sustainability (economic, social, environmental)’, and epis (e.g., ‘energy consumption’, ‘co2 footprint’, ‘ghg emissions’, ‘waste management’, ‘water consumption’ and ‘recycling’). similarly, we identified organizational factors related to es and recorded them as ‘management’, ‘strategy’ and ‘culture’. we classified studies about management of structure, practices, operations, and inter-organizational collaborations that support es as ‘management’ factors; decision-making, and internal and external policies as ‘strategy’ factors; and es organizational culture as ‘culture’ factors (dada et al., 2013; jakobi et al., 2016; sharma, 2000; wesumperuma et al., 2011). all 49 publications were repeatedly examined to identify the presence of the concepts mentioned above. moreover, we identified and included literature reviews (couckuyt & van looy, 2019; maciel, 2017; schoormann et al., 2017; stolze et al., 2012) in our literature set for completeness. while the epis and organizational factors were identified from the literature before coding the 49 articles, a set of bpm concepts (presented in section 5) emerged through consolidating the ‘bpm concept’ code and recoding all papers iteratively. we conducted a second round of analysis on our literature corpus to identify themes relevant to environmental impact assessment methods such as ‘lca method’, ‘eio analysis’, ‘hybrid assessment methods’, and further emerging concepts relevant to es. 5 results our 49-paper corpus (see appendix) consisted of 29 conference papers, 8 journal articles and 12 book chapters published between 2005 to 2019 (see figure 1). while the results indicate some preliminary interest in es from bpm researchers, there was no evident trend. however, as the publication distribution suggests, es gained more attention from bpm researchers during 2011 and 2012, due to special issues of journals and conference tracks on the topic of sustainability. in the following subsections, we explore green bpm research from different perspectives. first, we explore the main bpm concepts contributing to es and relevant organizational factors. subsequently, using the epi focus in industry reports, we summarise green bpm research from an epi perspective. in addition, we explain how green bpm research has contributed to environmental impact assessment methods, and, finally, we explore the use of theory in bpm research in the context of es. australasian journal of information systems roohy gohar & indulska 2020, vol 24, research article environmental sustainability through green bpm 9 figure 1 distribution of publications by type per year 5.1 contributions of predominant bpm concepts to es through multiple iterations of reading and coding the papers, we identified nine core bpm concepts within the green bpm literature, as follows: ‘bpm lifecycle extension’ suggests additional components to the current bpm lifecycle models (rosemann & vom brocke, 2010; van der aalst, 2004; weske, 2012) aiming to design, analyse, model and validate business processes based on es objectives (i.e., green bpm lifecycle) (jakobi et al., 2016; nowak, leymann, & schumm, 2011; opitz et al., 2014a; recker et al., 2012) ‘bpm architecture extension’ refers to models and tools proposed to improve the business process management architecture of organizations based on targeted es goals (harmon, 2010; lübbecke et al., 2016b). ‘capability maturity model extension’ refers to extended theoretical dimensions for organizations to measure their current bpm maturity level based on es as a capability (cleven et al., 2012; nowak, leymann, & schumm, 2011; nowak et al., 2011; seidel et al., 2012). ‘process performance measurement method extension’ refers to performance measurement methods developed and or adopted to capture environmental process performance based on epis (ardagna et al., 2008; hoesch-klohe & ghose, 2010; nowak, et al., 2011a; recker et al., 2012; thies et al., 2012; wesumperuma et al., 2013). ‘process modelling extension’ refers to additional modelling methods or process modelling notations to enrich process models with es information (hoesch-klohe & ghose, 2012; hoesch-klohe et al., 2010; recker et al., 2012; wesumperuma et al., 2011). ‘process reengineering’ and ‘process design’3 refer to suggested techniques and methods to assist organizations in designing and reengineering (hammer & champy, 1993) their business processes based on environmental objectives, e.g. reducing co2 or ghg emissions (ghose et al., 2010; hoesch-klohe & ghose, 2010; lübbecke et al., 2016b; nowak & leymann, 2013; nowak et al., 2011a; wesumperuma et al., 2013). 3 process design is concerned with creating processes to meet specific requirements inside the organization. process reengineering is concerned with radically changing processes to maximize value in the organization, which can involve process design. 1 1 1 8 1 2 6 3 4 3 2 2 5 1 1 1 1 1 1 1 1 2 0 2 4 6 8 10 12 14 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 book chapter conference journal australasian journal of information systems roohy gohar & indulska 2020, vol 24, research article environmental sustainability through green bpm 10 ‘process optimization’ (by re-designing, improving and adopting) refers to developed concepts based on iterative stages of monitoring, redesigning and improving business processes to move them towards es organizational objectives (ghose et al., 2010; houy et al., 2012; jakobi et al., 2016; nowak & leymann, 2013; pernici et al., 2008; wesumperuma et al., 2013; wesumperuma et al., 2011). ‘definition of green bpm extension’ includes attempts to define green bpm based on past definitions of bpm (weske et al., 2004) and proposed environmental performance aspects of processes (ghose et al., 2010; opitz et al., 2014b; seidel & recker, 2012). the top three most prominent categories of contribution from bpm to es were through process optimization, process performance measurement methods and process design (see figure 2).4 several other bpm concepts were also featured, as per figure 2. for example, twenty-four per cent of the examined papers have contributed to the bpm lifecycle and 12% to definition extension (most being conceptual studies). in the following subsections we provide a summary of these research contributions. figure 2 relevant bpm concepts 5.1.1 process optimization context-aware approaches have been suggested to optimize the use of operational resources (pernici et al., 2008). a context-aware approach increases the capability of an is to react dynamically to operating conditions and, therefore, improves the use of resources with minimum energy consumption. further approaches have been explored in the context of defining and measuring ecological characteristics of processes (nowak and leymann, 2013; wesumperuma et al., 2011) and to do so, the concept of key ecological indicators (the same as epis) was suggested by nowak et al. (2011). furthermore, the requirement for a multidimensional framework for business process optimization was identified by wesumperuma et al. (2011). the framework demonstrated how modelling, measuring, analysing and optimizing business processes supports the mitigation of ghg emissions from 4 twenty-eight papers had multiple contributions in terms of bpm concepts, therefore, we have coded the authors and their contributions through different lenses. 24% 8% 6% 35% 18% 18% 33% 53% 12% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% bpm lifecycle extension bpm architecture extension capability maturity model extension process performance measurement methods business process modelling notation… process reengineering process design process optimization green bpm definition extension australasian journal of information systems roohy gohar & indulska 2020, vol 24, research article environmental sustainability through green bpm 11 business processes. although the proposed framework was not implemented at the time of the study and therefore was not validated, the study was extended to investigate further methods to optimize ghg emissions of processes and to develop green activity-based management (abm) (wesumperuma et al., 2013). green abm merges activity-based costing (abc) and the critical path method (cpm) and proposes the use of environmental metrics such as ghg emissions in the same manner as businesses use process performance metrics such as time and cost in regard to processes. accordingly, a green abm approach provides an opportunity for organizations to include green objectives in their business performance objectives and thus adopt multi-dimensional process optimization. to further align green initiatives with business and process objectives, the concept of ‘collaborative green bpm’ was introduced, which enables stakeholder involvement in organisational sustainability initiatives (jakobi et al., 2016). similarly, to minimize the environmental impacts of processes, a theoretical roadmap for a carbon modelling framework was proposed by ghose et al. (2010). its aim is to reflect the carbon footprint of process activities, so the carbon footprint of processes becomes visible and can be easily communicated to stakeholders. furthermore, to assist in identifying and reducing the environmental impact of processes, an ecological workflow pattern was developed to optimize green business processes (lübbecke et al., 2016b). this approach provided decision analytics support for operations, control flow and data. the study was extended to explore further ecological process optimizations through compliance checking (lübbecke et al., 2017) and suggested process pattern checking (lübbecke et al., 2017; lübbecke et al., 2018) as a method to categorize processes based on design patterns. the study extended the application of a compliance checking method by identifying ecological weaknesses and ultimately optimizing business processes. 5.1.2 process performance measurement methods research extending ‘process performance measurement methods’ has generally focused on the epis of ‘co2 footprint’, ‘energy consumption’ and ‘ghg emissions’ (ardagna et al., 2008; cleven et al., 2012; hoesch-klohe & ghose, 2010; hoesch-klohe et al., 2010; lübbecke et al., 2016b; recker et al., 2012; thies et al., 2012; wesumperuma et al., 2013). the earliest theoretical framework for an active energy-aware resource management mechanism suggested the development of process-based applications that offer high performance, are energy efficient and also measure energy consumption (ardagna et al., 2008). a multidimensional quality of service (qos) measurement approach for measuring the emissions of processes was also suggested using an algebraic structure of c-semiring, aiming to measure the carbon footprint of processes (hoesch-klohe & ghose, 2010). the study suggested the abnoba framework to measure the heterogeneous environmental impacts of activities in processes. wesumperuma (2015) expanded on earlier studies (wesumperuma et al., 2013; wesumperuma et al., 2011) to develop an activity-based reporting tool that creates, calculates and includes a ghg emissions inventory from activities undertaken at the business-process level. in addition, a method with a focus on process performance measurement was proposed (cappiello et al., 2013), to improve measurement indicators such as energy consumption, co2 emissions, and process performance requirements, measured by suitable metrics. the study used a virtual machine to test the approach in regard to improving performance while maintaining energy efficiency. australasian journal of information systems roohy gohar & indulska 2020, vol 24, research article environmental sustainability through green bpm 12 5.1.3 process design and reengineering process design has been explored in several studies. a conceptual study explored the benefits of service-oriented process design (pernici et al., 2008) to achieve sustainability objectives in organisations. based on the same concept, and to improve the environmental impact in organizations, nowak et al. (2011b) identified frequently used process design patterns. the study focuses on organizational factors of es, such as management. further research related to design patterns classified green process design patterns and described the connection of the design with services, artefacts and applications (nowak et al., 2012). later, the study was extended (nowak & leymann, 2013) to assist organizational stakeholders in identifying sustainable patterns and designing environmentally aware business processes. ecological (green) business process patterns were further analysed using general and specific recurring green patterns in german public sector administration (lübbecke et al., 2016a). the study was extended with guidelines to model and design ecology-aware (green) processes (lübbecke et al., 2017). however, both studies are ongoing, and the results are based on synthetic scenarios. 5.1.4 bpm lifecycle and green bpm definition extension the need to extend the bpm lifecycle into a ‘green bpm lifecycle’ was identified by seidel et al. (2011). by reviewing es in the context of it and is literature, the study explored how it could enable organizations to make sustainability transparent in their processes. a range of enablers of, and barriers to, adopting such processes in operations were identified through this research. consequently, the study suggested that organizations require green initiatives, capabilities and motivators as a means of encouragement to include sustainability in their operations and to enable them to provide sustainable it and is solutions to their customers. to this end, the authors proposed the green bpm lifecycle. further, seidel et al. (2012) emphasized the central role bpm plays in creating environmentally sustainable practices and argued that business and it managers should provide a process-centred environment and opportunities for understanding organizational processes. the study further developed a framework for green bpm research by exploring opportunities and contributions that bpm can offer in the context of es. focusing on phases of the bpm lifecycle, the influence of green bpm was examined in the malaysian manufacturing sector (kuppusamy & gharleghi, 2015). with a specific focus on green-supplier selection, monitoring and collaboration, the study found that green-supplier monitoring and management significantly improves the greening processes in the manufacturing sector. they also defined green bpm differently to seidel et al. (2012) and opitz et al. (2014b), as a factory’s institutionalization of internal environmental management practices that includes process design such that environmental impacts are reduced. 5.1.5 process modelling notation extension a process modelling notation was suggested to reflect the relationship between resources and activities (hoesch-klohe et al., 2010) in a study that illustrated how the measurement and modelling method could be used to improve the environmental impact of business processes. similarly, several studies suggested various modelling methods that could be used to represent the environmental impact of business processes (lübbecke et al., 2017; lübbecke et al., 2018; schoormann et al., 2017). a process modelling notation to allow for the representation of the carbon footprint of a business process was also developed by recker et al. (2012). australasian journal of information systems roohy gohar & indulska 2020, vol 24, research article environmental sustainability through green bpm 13 furthermore, an emissions modelling and reporting method for co2 and ghg emissions was proposed by wesumperuma et al. (2013). 5.1.6 bpm architecture and capability maturity model extension a capability maturity model that enables organizations to define and control their corporate sustainability is one of the contributions within this category. the model, based on a bpm capability model, integrates es into the regular bpm capability of the organization and helps employees to understand the current performance and targets for future improvements in different areas, including es (seidel et al., 2012; cleven et al., 2012). capability maturity enables organizations to define, implement and monitor their sustainability efforts, based on a bpm capabilities model (de bruin & rosemann, 2007). 5.2 organizational factors in green bpm research organizational factors include management, strategy and culture, all of which are essential in achieving es (wesumperuma et al., 2011). we found 45 per cent of the analysed articles acknowledged the significance of organizational perspectives by using bpm to develop conceptual frameworks and models that involved organizational perspectives (see figure 3). contributions to these areas in the current body of green bpm literature are outlined below. 5.2.1 management the main es focus, from an organizational perspective, of our 49-paper research corpus was the overarching concept of ‘management’. we identified organizational structures, practices, operations and inter-organizational collaborations as key factors that support es. management also encompasses decision-making in internal and external policies and overlaps with ‘strategy’ factors; and ‘culture’ factors (dada et al., 2013; sharma, 2000; wesumperuma et al., 2011). figure 3 epis and relevant organizational factors 5.2.2 strategy and culture organizational strategy and culture influence an organization’s environmental performance (dada et al., 2013; epstein & roy, 2001; gonzález‐benito & gonzález‐benito, 2006; sharma, australasian journal of information systems roohy gohar & indulska 2020, vol 24, research article environmental sustainability through green bpm 14 2000). thus, to positively affect environmental performance, new values need to be implemented at all levels of the organizational structure and novel strategies need to be planned and regulated by national and international bodies (lacy et al., 2010). 5.3 epi focus in industry versus green bpm research in addition to analysing academic articles, we reviewed six reports from global institutions such as the un, large industry bodies and the australian government. table 3 shows the industry reports consistently focus on nearly all of the epis and organizational factors. the commonwealth of australia (2010) has set mandatory environmental standards for energy consumption (i.e., the iso14001 standard family) for the ict sector to which organizations must comply by resource recovery and recycling of material (e.g., digital devices). the ipcc (2014) addressed the challenges of climate change and provided a future pathway for adaptation, risk mitigation and sustainable development according to epis. indeed, industry is expected to reduce the environmental impact of its activities in all its sectors, as per the paris climate change agreement (harvey, 2015; morgan et al., 2014; rogelj et al., 2016). reports organizational factors environmental performance indicators (epi) m an ag em en t s tr at eg y c u lt u re e n er g y c o n su m p ti o n c o 2 f o o tp ri n t g h g e m is si o n s (o th er ) r ec y cl in g w as te m an ag em en t w at er c o n su m p ti o n (commonwealth of australia, 2010) ✗ (lacy et al., 2010) (united nations environment programme, 2014) ✗ (ipcc, 2014) (national sustainability council, 2013) ✗ (australian industry group, 2007) table 3 focus of industry reports by es organizational factors and epis energy consumption is a main agenda of all of our six identified industry/government reports, including the united nations environment programme (2014) and the ict sustainability plan by the commonwealth of australia (2010). an environmental management strategy is required to audit and control energy consumption. of the 49 identified research articles, 15 (i.e., 31%) contain contributions to energy consumption management. from these 15, 14 are conceptual in nature and focus on energy-aware applications to reduce energy consumption (ardagna et al., 2008), purifier-based approaches (pernici et al., 2008), a case study on networkcentric solutions (thies et al., 2012), applying an algebraic framework to multiple heterogeneous dimensions (hoesch-klohe & ghose, 2012), process improvement methods to support measuring and monitoring performance and energy efficiency, and a conceptual integration model for energy consumption from it components to business processes (reiter et al., 2014). energy consumption and waste in processes can be reduced by applying australasian journal of information systems roohy gohar & indulska 2020, vol 24, research article environmental sustainability through green bpm 15 techniques and solutions from bpm (houy et al., 2011). furthermore, energy consumption feedback systems can change organizational behaviour in relation to energy consumption (jakobi et al., 2016). co2 and ghg emissions directly influence climate change (ipcc, 2014; young & rikhardsson, 1996). from 1970 to 2010, co2 emissions from fossil fuels and industrial processes contributed 78 per cent of the total worldwide ghg accumulation (ipcc, 2014). continued emissions of high levels of co2 and ghg will increase the possibility of pervasive, severe impacts on the population and the ecosystem by increasing surface and ocean temperatures (ipcc, 2014). the united nations environment programme (2014) produced a roadmap for cutting emissions because costs of climate change adaptation will reach $300 billion per year by 2050. the concern about climate change from ghg emissions also continues to grow (lacy et al., 2010). of the 49 bpm research articles, 11 articles (22%) focus on measuring and managing co2 emissions. these contributions, which are mostly conceptual in nature, include a framework for carbon-aware process improvement (hoesch-klohe & ghose, 2012), a roadmap to optimize the carbon modelling framework (ghose et al., 2010), activity-based emissions analysis for measuring co2 in processes (including a modelling notation extension, specifically bpmn) (recker et al., 2011; recker et al., 2012), network centric solutions (thies et al., 2012), conceptual advancement of methods for measuring and monitoring process performance based on epis such as co2 (cappiello et al., 2013) and theoretical principles for capturing, measuring, modelling and reporting co2 and ghg emissions (wesumperuma et al., 2013). a process-based method was proposed by ghose et al. (2010) to measure emissions from business processes. the method uses three scopes: 1) direct emissions that occur from internal activities in an organization; 2) indirect emissions from sources external to the organization; and 3) all other indirect emissions not part of scope two which are material emissions, employee commuting emissions etc. based on this approach, there are three types of resources, atomic, shared resources and hybrid resources, which organizations need to consider while measuring the carbon footprint and ghg emissions of their activities. the paper provides a roadmap for carbon-aware bpm. the studies focusing on co2 footprint and ghg emissions are mainly conceptual. recycling, waste management and water consumption have received less focus from the bpm research community. of the 49 publications studied, four focus on waste management, with recycling and water consumption having three related articles each. hoesch-klohe and ghose (2010) discussed the potential of extended epis in the application of the abnoba algebraic framework for process optimization. the authors presented a conceptual framework for the green quality of service measures that generalizes qualitative and quantitative scales and permits the integration of multiple heterogeneous measures into a single composite scale. as a result, the framework can measure ‘water consumption’, ‘co2 emissions’ and ‘waste generation’, which are mainly quantifiable, and ‘damage to fauna and flora’, which is qualitative. combined measures such as ‘air quality’ and ‘environmental performance’ could be heterogeneously measured together. thus, while the main global and industry reports indicate the significance of these three epis in achieving es (australian and new zealand environment and conservation council state of the environment reporting task force, 2000; australian industry group, 2007; commonwealth of australia, 2010; ipcc, 2014; united nations environment programme, 2014), a substantial response from the bpm literature is yet to come. again, most of the studies are conceptual with the three epis being narrowly discussed. australasian journal of information systems roohy gohar & indulska 2020, vol 24, research article environmental sustainability through green bpm 16 table 4 summarizes the number of articles that contribute to an epi through a specific bpm concept. for each bpm concept, it also shows the contributions to relevant organizational factors, and the overall number of research papers that have focused on that particular bpm concept. bpm concepts organizational factors environmental performance indicators (epi) t o ta l n u m b e r o f p a p e rs m an a g em en t s tr at eg y c u lt u re e n e rg y co n su m p ti o n c o 2 fo o tp ri n t g h g e m is si o n s (o th e r) r ec y cl in g w as te m an ag em en t w at er co n su m p ti o n bpm lifecycle extension 5 4 2 1 3 1 1 12 bpm architecture extension 2 2 1 1 4 capability maturity model extension 3 2 1 3 process performance measurement extension 6 3 1 6 6 5 2 2 2 17 process modelling extension 3 1 1 1 4 2 1 9 business process reengineering 4 2 3 1 2 1 1 1 9 process design 6 2 1 6 5 4 2 2 2 16 process optimization 11 5 2 8 7 7 1 1 1 26 green bpm definition extension 3 2 1 6 table 4 bpm concepts contributing to es organizational factors and epis 5.4 contribution to environmental impact assessment methods several articles have contributed to assessing the environmental impact of processes using existing impact assessment techniques applied to products, services and operations such as lifecycle assessment (lca), input-output analysis (eio) or resource-based analysis. process-analysis techniques for activity-based costing were adapted by recker et al. (2011) to measure the environmental emissions of processes under the title of activity-based emissions. the authors also discussed three diverse methods to measure co2 emissions from business processes: bottom-up, which is the same as process-based analysis; top-down, which is the analysis of inputs and outputs of processes or products; and a hybrid or combined method that uses a combination of the two other methods. they calculated co2 emissions on an exploratory case of a direct invoicing process in an organization. later, recker et al. (2012) proposed an approach to modelling and recording the carbon footprints of business processes. using a business process model example and an extended activity-based costing example, the authors presented a process-based perspective of modelling and analysing co2 emissions from business processes. an extension to process modelling was suggested which allows better communication and monitors the compliance and recording of environmental information about processes. example case studies were used to demonstrate the practicality of conducting an activityand process-based measurement of environmental emissions from business processes. australasian journal of information systems roohy gohar & indulska 2020, vol 24, research article environmental sustainability through green bpm 17 with a focus on identifying the environmental impact of activities, hoesch-klohe and ghose (2012) reviewed suggested potential bpm contributions to es in organizations. the authors noted that the environmental impact of activities and processes could be identified using a variety of different methods (e.g., educated guess by experts, activity-based costing, and derivation from resource model and carbon-dioxide accumulation). they used case studies to demonstrate a trade-off analysis for environmentally aware business process design and a framework for green process improvement. regarding bottom-up impact assessment, wesumperuma et al. (2013) proposed a green activity-based management (abm) approach to measure, report and manage environmentally sustainable business processes. this study was later extended by wesumperuma (2015) to develop an activity-based reporting tool which creates, calculates and incorporates a ghg emissions inventory of activities at the business-process levels. 5.5 use of theory in green bpm research academic literature should develop, test and use theories; this applies in the is field (gregor, 2006) as well as in the bpm fields and sets academic publications apart from practitioner and consultant reports. therefore, we also reviewed the literature to identify examples of theory building. of the 49 publications we analysed, five applied theories to their studies and two theorized models using case studies, but none attempted to extend previously developed theories. ardagna et al. (2008) applied systems and control theory and queuing theory (qin & wang, 2007) to their framework for active energy-aware management of business processes. whereas, cleven et al. (2012) used socio-technical theory (bostrom & heinen, 1977) to understand the different components, systems and sub-systems in organizations that should be considered to understand, measure and manage the impact of is activities on the environment. to develop energy-aware and optimized processes, cappiello et al. (2013) used a multiple criteria decision-making theory (triantaphyllou, 2000). furthermore, to highlight the benefits to companies for adopting environmentally sustainable practices, kuppusamy and gharleghi (2015) used rbv theory (barney, 1991) to study the competitive advantage companies can gain by applying their valuable resources to sustainable practices. finally, considering the role of the consumer, jakobi et al. (2016) explained that rational choice theory (jackson, 2005) and value belief norm theory (stern et al., 1999) support feedback campaigns for change in consumer behaviour to reduce energy consumption. a case study on the adoption of environmentally sustainable practices conducted by seidel et al. (2010) recognized that it can enable sustainable operations. their study also encouraged future research to investigate the adoption of sustainable practices to develop a more generalized view and theoretical models. also based on case studies, seidel and recker (2012) developed a framework to theorize sustainable business process implementation. 6 green bpm: the next steps our analysis of global industry reports reflects an increasing understanding of organizational factors and epis related to es (see table 3). by using various bpm concepts, we propose several research directions within the bpm and es context, as outlined below. several studies focused on management and strategy within organizations have recommended extending the bpm lifecycle into a green bpm lifecycle. bpm is often referred australasian journal of information systems roohy gohar & indulska 2020, vol 24, research article environmental sustainability through green bpm 18 to as a holistic management approach in managing an organization’s performance; therefore, naturally, it should also focus on the cultural, strategic and management success factors, and complexities of continual process improvement to achieve es throughout green bpm lifecycle phases. future green bpm research should involve designing, evaluating, implementing and monitoring environmentally sustainable processes that meet the es strategies for all epis. further, while green bpm literature has generated knowledge about organizational aspects of environmentally sustainable bpm, successful adoption and sustainable solutions that include economic and social sustainability should also be explored in more depth. consequently, critical success factors for green bpm should be explored. bpm architecture studies have not considered epis specifically, yet the need exists for researchers to consider further dimensions of bpm architecture that address epis and combine processes, practices and structure of organizations into a capability model. a capability model should include environmentally sustainable practices, strategies and governance that guides the targets and assessment and improvement of green capability maturity of organizations. a green capability maturity model would enable organizations to identify, assess and improve their practices, processes and strategies with regards to their es objectives. such research into suitable capability maturity models that address specific epis is restricted even though the extent of the overlap of the capabilities related to managing specific epis is clear. more than any other bpm concept, a considerable number of studies have focussed on measuring process performance. several studies explore methods to measure and manage epis using energy consumption and co2 and ghg emissions. while some evidence exists of suitable measurement methods for other epis (e.g., recycling, water consumption and waste management), most research is conceptual in nature and lacks further evidence of how to apply measures successfully and correctly. further study is required to identify the right epis and techniques to measure the impact in organizations. to support industry to reduce climate change, we see a need for studies focused on implementing epi measurement methods, organizational planning, and changing organizational procedures as well as changing organizational culture to address the management of all epis, to provide guidance to organisations. several researchers have suggested extending process modelling notations to reflect epis such as co2 footprint and ghg emissions in process models. future research should explore whether and what es information would be useful to visualise in business process models in practice, whether they also apply to other epis and, ultimately, whether they enable better management/awareness of epis in the long run. we suggest further research in identifying the success factors of environmentally sustainable process design, reengineering and optimization. stolze et al. (2012) and thies et al. (2012) provide an initial study of success factors related to implementing environmentally sustainable processes. however, design, adoption and implementation of es in organizations incurs a cost and requires significant change (adger et al., 2005; christmann, 2000), thus it is important that studies identify and empirically validate related success factors. we particularly note a lack of studies that focus on adopting green practices and processes without compromising organizational and process competence. in general, we also see a lack of research into the cultural factors underlying bpm and es (see table 4). while benefits can be achieved from applying bpm initiatives because of the australasian journal of information systems roohy gohar & indulska 2020, vol 24, research article environmental sustainability through green bpm 19 incremental improvement approach they take and their holistic view of people and systems (pritchard & armistead, 1999; vom brocke & sinnl, 2011), adopting es requires a culture change by organizations (harris & crane, 2002). culture change may be achieved through reflecting on the environmental impact of individual behaviour in organizations; also, by democratizing information and holding discussions with individuals (degirmenci & recker, 2018). green or eco nudging has also been shown to influence individual behaviour and overall practices in organizations (hall, 2013). however, changing any practice in organizations requires continual improvement (gao & low, 2014). therefore, change in organizational practices and individual behaviour towards es requires raising public and individual awareness about the impact of climate change, together with offering practical solutions to individuals (linnenluecke & griffiths, 2010; nerlich et al., 2010). future studies could consider the effect of increasing individual es awareness, perhaps via games, apps and/or devices, to raise awareness of specific epis. for example, gamification has received recent academic attention (hamari et al., 2014; schlagenhaufer & amberg, 2015) yet its value in the context of es, by changing individuals’ awareness of es and their behaviour, is unknown. 7 conclusion in this study, we conducted a systematic review of green bpm literature to identify relevant contributions to environmental sustainability, with a specific lens on epis and relevant organizational factors. to this end, we identified, collected and analysed 49 relevant academic research articles. our analysis identified the core bpm contributions, viz., bpm lifecycle extension, bpm architecture extension, capability maturity model extension, process performance measurement extension, process modelling extension, business process reengineering, process design, process optimization, green bpm definition extension, with the main ones being process optimization, process performance measurement methods and process design. we also identified green bpm literature’s most researched epis as energy consumption, co2 emissions and ghg emission and summarised green bpm contributions through the epi lens. further, we presented green bpm contributions from the perspective of environmental impact assessment methods, and, finally, considered the use of theory underlying green bpm research. as a result of our analysis, we identified future research directions related to the core bpm concepts identified. our literature review is not without limitations. while papers were collected and identified by one researcher and checked by another, coding of the papers to bpm and es was done by one researcher. despite the coding being conducted using multiple iterations of full readings of relevant papers, having a single coder remains a shortcoming that might introduce bias in the analysis. in addition, while we strived to consider the largest set of relevant papers possible, our focus was on contributions published in english only. our search also resulted in some green supply chain management (scm) articles. while ‘process’ is a common term in scm, because none of the articles we found explicitly focused on green bpm we excluded green scm from this study, which presents a limitation in scope. references adger, w. n., arnell, n. w., & tompkins, e. l. (2005). successful adaptation to climate change across scales. global environmental change, 15(2), 77-86. australasian journal of information systems roohy gohar & indulska 2020, vol 24, research article environmental sustainability through green bpm 20 applegate, l. m., & king, j. l. (1999). rigor and relevance: careers on the line. management information systems quarterly, 23, 17-18. ardagna, d., cappiello, c., lovera, m., pernici, b., & tanelli, m. (2008). active energy-aware management of business-process based applications: springer. australian and new zealand environment and conservation council state of the environment reporting task force. (2000). core environmental indicators for reporting on the state of the environment. retrieved from www.environment.gov.au/system/files/pages/-a94a73f8-79db-4347-8a7a f03ecdc48074/files/core-indicators.pdf australian industry group. (2007). environmental sustainability and industry: road to a sustainable future findings of the national survey on environmental sustainable practices. sustainability victoria, melbourne bandara, w., furtmueller, e., gorbacheva, e., miskon, s., & beekhuyzen, j. (2015). achieving rigour in literature reviews: insights from qualitative data analysis and tool-support. communications of the association for information systems, 37(1), 8. bansal, p., & roth, k. (2000). why companies go green: a model of ecological responsiveness. academy of management journal, 43(4), 717-736. barney, j. (1991). firm resources and sustained competitive advantage. journal of management, 17(1), 99-120. blohm, i., & leimeister, j. m. (2013). gamification: design of it-based enhancing services for motivational support and behavioral change. business & information systems engineering, volume 5(4), 275-278. doi:10.1007/s12599-013-0273-5 bostrom, r. p., & heinen, j. s. (1977). mis problems and failures: a socio-technical perspective, part ii: the application of socio-technical theory. mis quarterly, 11-28. british standards institution. (1994). specification for environmental management systems. retrieved from https://shop.bsigroup.com/productdetail/?pid=000000000000322818 brooks, s., wang, x., & sarker, s. (2012). unpacking green is: a review of the existing literature and directions for the future. in j. vom brocke, s. seidel, & j. recker (eds.), green business process management (pp. 15-37): springer berlin heidelberg. brown, b., hanson, m., liverman, d., & merideth, r., jr. (1987). global sustainability: toward definition. environmental management, 11(6), 713-719. doi:10.1007/bf01867238 cappiello, c., plebani, p., & vitali, m. (2013). energy-aware process design optimization. proceeding of the 2013 international conference on cloud and green computing (cgc), (pp. 451-458). chen, a. j., watson, r. t., boudreau, m.-c., & karahanna, e. (2011). an institutional perspective on the adoption of green is & it. australasian journal of information systems, 17(1) cheong, c., cheong, f., & filippou, j. (2013). using design science research to incorporate gamification into learning activities. proceedings of the 2013 pacific asia conference on information systems, jeju island, korea (pp. 156-171) australasian journal of information systems roohy gohar & indulska 2020, vol 24, research article environmental sustainability through green bpm 21 christmann, p. (2000). effects of “best practices” of environmental management on cost advantage: the role of complementary assets. academy of management journal, 43(4), 663680. cleven, a., winter, r., & wortmann, f. (2012). managing process performance to enable corporate sustainability: a capability maturity model. in j. vom brocke, s. seidel, & j. recker (eds.), green business process management (pp. 111-129): springer berlin heidelberg. commonwealth of australia. (2010). australian government ict sustainability plan 2010-2015 (978-1921733154). retrieved from canberra, a.c.t. retrieved from https://dga.or.th/upload/download/file_bd887ff18f65ded54fe5cf7f38858446.pdf couckuyt, d. (2017). an overview of challenges and research avenues for green business process management. proceedings of the otm confederated international conferences" on the move to meaningful internet systems". (pp. 270-279). springer, cham. couckuyt, d., & van looy, a. (2019). green bpm as a business-oriented discipline: a systematic mapping study and research agenda. sustainability, 11(15), 4200. dada, a., stanoevska-slabeva, k., & gómez, j. m. (2013). organizations' environmental performance indicators: springer. de bruin, t., & rosemann, m. (2007). using the delphi technique to identify bpm capability areas. proceedings of the 18th australasian conference on information systems, the university of southern queensland, toowoomba, australia, pp. 643–653 (2007) de burgos jimenez, j., & céspedes lorente, j. j. (2001). environmental performance as an operations objective. international journal of operations & production management, 21(12), 1553-1572. degirmenci, k., & recker, j. (2018). creating environmental sensemaking through green is: an experimental study on eco-nudging paper printing behavior. proceedings of the twentyfourth americas conference on information systems, new orleans. (pp. 110) dehghanian, f., & mansour, s. (2009). designing sustainable recovery network of end-of-life products using genetic algorithm. resources, conservation and recycling, 53(10), 559-570. deterding, s., sicart, m., nacke, l., o'hara, k., & dixon, d. (2011). gamification. using gamedesign elements in non-gaming contexts. proceedings of the chi'11 extended abstracts on human factors in computing systems. pp. 2425-2428 detoro, i., & mccabe, t. (1997). how to stay flexible and elude fads. quality progress, 30(3), 55. dumas, m., la rosa, m., mendling, j., & reijers, h. a. (2013). introduction to business process management. in fundamentals of business process management (pp. 1-31). berlin, heidelberg: springer berlin heidelberg. elkington, j. (1998). partnerships from cannibals with forks: the triple bottom line of 21st‐ century business. environmental quality management, 8(1), 37-51. epstein, m. j., & roy, m.-j. (2001). sustainability in action: identifying and measuring the key performance drivers. long range planning, 34(5), 585-604. australasian journal of information systems roohy gohar & indulska 2020, vol 24, research article environmental sustainability through green bpm 22 finnveden, g., hauschild, m. z., ekvall, t., guinée, j., heijungs, r., hellweg, s., . . . suh, s. (2009). recent developments in life cycle assessment. journal of environmental management, 91(1), 1-21. doi:https://doi.org/10.1016/j.jenvman.2009.06.018 gao, s., & low, s. (2014). the toyota way. in lean construction management (pp. 49-100): springer singapore. gholami, r., watson, r. t., molla, a., hasan, h., & bjørn-andersen, n. (2016). information systems solutions for environmental sustainability: how can we do more? journal of the association for information systems, 17(8), 521. ghose, a., hoesch-klohe, k., hinsche, l., & le, l.-s. (2010). green business process management: a research agenda. australasian journal of information systems, 16(2). gonzález‐benito, j., & gonzález‐benito, ó. (2006). a review of determinant factors of environmental proactivity. business strategy and the environment, 15(2), 87-102. goodland, r. (1995). the concept of environmental sustainability. annual review of ecology and systematics, 26, 1-24. doi:10.2307/2097196 gregor, s. (2006). the nature of theory in information systems. mis quarterly, 611-642. guinée, j. (2001). handbook on life cycle assessment—operational guide to the iso standards. the international journal of life cycle assessment, 6(5), 255-255. hall, c. m. (2013). framing behavioural approaches to understanding and governing sustainable tourism consumption: beyond neoliberalism,“nudging” and “green growth”? journal of sustainable tourism, 21(7), 1091-1109. hamari, j., & koivisto, j. (2013). social motivations to use gamification: an empirical study of gamifying exercise. proceedings of the european conference on information systems (ecis). utrecht, the netherlands, june 5–8, 2013. hamari, j., koivisto, j., & sarsa, h. (2014). does gamification work?--a literature review of empirical studies on gamification. proceedings of the 47th hawaii international conference on system sciences (hicss), usa, january 6–9, 2014. hammer, m. (2010). what is business process management? in j. v. b. m. rosemann (ed.), handbook on business process management: introduction, methods and information system (vol. 1, pp. 3-16). berlin, germany: springer. hammer, m., & champy, j. (1993). reengineering the corporation: a manifesto for business revolution. business horizons, 36(5), 90-91. hammond, a., adriaanse, a., rodenburg, e., bryant, d., & woodward, r. (1995). environmental indicators: a systematic approach to measuring and reporting on environmental policy performance in the context of sustainable development. retrieved from world resources institute: http://pdf.wri.org/environmentalindicators_bw.pdf harmon, p. (2010). the scope and evolution of business process management. in handbook on business process management 1 (pp. 37-81): springer. harris, l. c., & crane, a. (2002). the greening of organizational culture: management views on the depth, degree and diffusion of change. journal of organizational change management, 15(3), 214-234. australasian journal of information systems roohy gohar & indulska 2020, vol 24, research article environmental sustainability through green bpm 23 harvey, f. (2015). paris climate change agreement: the world’s greatest diplomatic success. the guardian, 14, 15. hernández gonzález, a., calero, c., pérez parra, d., & mancebo, j. (2019). approaching green bpm characterisation. journal of software: evolution and process, 31(2), e2145. hoesch-klohe, k., & ghose, a. (2010). carbon-aware business process design in abnoba. in p. maglio, m. weske, j. yang, & m. fantinato (eds.), service-oriented computing (vol. 6470, pp. 551-556): springer berlin heidelberg. hoesch-klohe, k., & ghose, a. (2012). environmentally aware business process improvement in the enterprise context. harnessing green it: principles and practices, 265. hoesch-klohe, k., ghose, a., & lê, l.-s. (2010). towards green business process management. proceedings of the 2010 ieee international conference on services computing (pp. 386393). ieee. houy, c., reiter, m., fettke, p., & loos, p. (2011). towards green bpm–sustainability and resource efficiency through business process management. proceedings of the business process management workshops 2010, lnbip 66, hoboken, nj, usa (pp. 501-510). houy, c., reiter, m., fettke, p., loos, p., hoesch-klohe, k., & ghose, a. (2012). advancing business process technology for humanity: opportunities and challenges of green bpm for sustainable business activities. in j. vom brocke, s. seidel, & j. recker (eds.), green business process management (pp. 75-92): springer berlin heidelberg. hung, r. y.-y. (2006). business process management as competitive advantage: a review and empirical study. total quality management & business excellence, 17(1), 21-40. doi:10.1080/14783360500249836 international organisation for standardization, i. (2013). the iso survey. retrieved from https://www.iso.org/news/2014/09/ref1893.html ipcc. (2014). climate change 2014: synthesis report. contribution of working groups i, ii and iii to the fifth assessment report of the intergovernmental panel on climate change (9291691437). retrieved from https://www.ipcc.ch/pdf/assessmentreport/ar5/syr/syr_ar5_final_full.pdf jackson, t. (2005). motivating sustainable consumption. sustainable development research network, 29, 30. jakobi, t., castelli, n., nolte, a., schönau, n., & stevens, g. (2016). towards collaborative green business process management as a conceptual framework. in advances and new trends in environmental and energy informatics (pp. 275-293): springer. jamous, n., & müller, k. (2013). environmental performance indicators. in a. dada, k. stanoevska, & j. m. gómez (eds.), organizations’ environmental performance indicators (pp. 3-18): springer berlin heidelberg. jasch, c. (2000). environmental performance evaluation and indicators. journal of cleaner production, 8(1), 79-88. doi:http://dx.doi.org/10.1016/s0959-6526(99)00235-8 kankanhalli, a., taher, m., cavusoglu, h., & kim, s. h. (2012). gamification: a new paradigm for online user engagement. proceedings of the thirty third international conference on information systems (icis), orlando, fl, december 16–19, 2012, pp. 1–10. australasian journal of information systems roohy gohar & indulska 2020, vol 24, research article environmental sustainability through green bpm 24 kitchenham, b., brereton, o. p., budgen, d., turner, m., bailey, j., & linkman, s. (2009). systematic literature reviews in software engineering–a systematic literature review. information and software technology, 51(1), 7-15. kohlbacher, m., & reijers, h. a. (2013). the effects of process-oriented organizational design on firm performance. business process management journal, 19(2), 245-262. kuppusamy, m., & gharleghi, b. (2015). green business process management in manufacturing firms: examining the role of upstream and downstream suppliers international journal of applied business and economic research ijaber, 13(1), 259-271. lacy, p., cooper, t., hayward, r., & neuberger, l. (2010). a new era of sustainability: united nations global compact-accenture ceo study retrieved from https://www.comunicarseweb.com/sites/default/files/biblioteca/pdf/1321968280_334347 37-new-era-sustainabiltiy-accenture-ungc-study-2010.pdf lane, m. s., kolbe, l., & zarnekow, r. (2011). editorial for special issue of ajis on green it/is (sustainable computing). australasian journal of information systems, 17(1). doi:10.3127/ajis.v17i1.666 linnenluecke, m. k., & griffiths, a. (2010). corporate sustainability and organizational culture. journal of world business, 45(4), 357-366. doi:http://dx.doi.org/10.1016/j.jwb.2009.08.006 lintukangas, k., kähkönen, a.-k., & ritala, p. (2016). supply risks as drivers of green supply management adoption. journal of cleaner production, 112, 1901-1909. longoni, a., & cagliano, r. (2015). environmental and social sustainability priorities: their integration in operations strategies. international journal of operations & production management, 35(2), 216-245. lübbecke, p., fettke, p., & loos, p. (2016a). sustainability patterns for the improvement of itrelated business processes with regard to ecological goals. in bpm workshops (281), m. dumas and m. fantinato (eds.), cham: springer international publishing, pp. 428–439. lübbecke, p., fettke, p., & loos, p. (2016b). towards ecological workflow patterns as an instrument to optimize business processes with respect to ecological goals. proceedings of the 49th hawaii international conference on system sciences (hicss), 1049-1058. lübbecke, p., fettke, p., & loos, p. (2017). towards guidelines of modeling for ecology-aware process design. proceedings of the international conference on business process management. bpm workshops, vol. 308, springer, cham, pp. 510-519 lübbecke, p., goswami, a., & fettke, p. (2018, 11-14 july 2018). a method for ecological process optimization based on compliance checking. proceedings of the 2018 ieee 20th conference on business informatics (cbi) (vol. 1, pp. 119-128). ieee. maciel, j. c. (2017). the core capabilities of green business process management–a literature review. proceedings of the 13th internationalen tagung wirtschaftsinformatik (wi 2017), st. gallen, s. 1526-1537. melnyk, s. a., sroufe, r. p., & calantone, r. (2003). assessing the impact of environmental management systems on corporate and environmental performance. journal of operations management, 21(3), 329-351. australasian journal of information systems roohy gohar & indulska 2020, vol 24, research article environmental sustainability through green bpm 25 morgan, j., dagnet, y., & tirpak, d. (2014). elements and ideas for the 2015 paris agreement. washington, dc: agreement for climate transformation. national sustainability council. (2013). sustainable australia report. retrieved from http://base.socioeco.org/docs/sustainable-report-full.pdf nerlich, b., koteyko, n., & brown, b. (2010). theory and language of climate change communication. wiley interdisciplinary reviews: climate change, 1(1), 97-110. nowak, a., binz, t., fehling, c., kopp, o., leymann, f., & wagner, s. (2012). pattern-driven green adaptation of process-based applications and their runtime infrastructure. computing, 94(6), 463-487. nowak, a., & leymann, f. (2013). green business process patterns--part ii. proceedings of the 2013 ieee 6th international conference on service-oriented computing and applications (pp. 168-173). nowak, a., leymann, f., & mietzner, r. (2011a). towards green business process reengineering. in service-oriented computing (pp. 187-192): springer. nowak, a., leymann, f., schleicher, d., schumm, d., & wagner, s. (2011b). green business process patterns. proceedings of the 18th conference on pattern languages of programs (pp. 1-10). nowak, a., leymann, f., & schumm, d. (2011c). the differences and commonalities between green and conventional business process management. proceedings of the 2011 ieee 9th international conference on dependable, autonomic and secure computing (pp. 569576). ieee. nowak, a., leymann, f., schumm, d., & wetzstein, b. (2011). an architecture and methodology for a four-phased approach to green business process reengineering. in information and communication on technology for the fight against global warming (pp. 150164): springer. opitz, n., krüp, h., & kolbe, l. m. (2014a). environmentally sustainable business process management–developing a green bpm readiness model. proceedings of the pacific asia conference on information systems (pacis), chengdu, china pp. 12. opitz, n., krüp, h., & kolbe, l. m. (2014b). green business process management--a definition and research framework. proceedings of the 2014 47th hawaii international conference on system sciences (pp. 3808-3817). parmenter, d. (2010). key performance indicators (kpi): developing, implementing, and using winning kpis: john wiley & sons. pernici, b., ardagna, d., & cappiello, c. (2008). business process design: towards servicebased green information systems. in e-government ict professionalism and competences service science (pp. 195-203): springer. phan, t. n., & baird, k. (2015). the comprehensiveness of environmental management systems: the influence of institutional pressures and the impact on environmental performance. journal of environmental management, 160(supplement c), 45-56. doi:https://doi.org/10.1016/j.jenvman.2015.06.006. australasian journal of information systems roohy gohar & indulska 2020, vol 24, research article environmental sustainability through green bpm 26 porter, m. e. (2008). competitive advantage: creating and sustaining superior performance: simon and schuster. porter, m. e., & van der linde, c. (1995). toward a new conception of the environmentcompetitiveness relationship. the journal of economic perspectives, 9(4), 97-118. pritchard, j.-p., & armistead, c. (1999). business process management-lessons from european business. business process management journal, 5(1), 10-35. qin, w., & wang, q. (2007). modeling and control design for performance management of web servers via an lpv approach. ieee transactions on control systems technology, 15(2), 259-275. rajeev, a., pati, r. k., padhi, s. s., & govindan, k. (2017). evolution of sustainability in supply chain management: a literature review. journal of cleaner production, 162(supplement c), 299-314. doi:https://doi.org/10.1016/j.jclepro.2017.05.026 recker, j., rosemann, m., & gohar, e. r. (2011). measuring the carbon footprint of business processes. in business process management workshops, springer berlin heidelberg (2011), pp. 511-520 recker, j., rosemann, m., hjalmarsson, a., & lind, m. (2012). modeling and analyzing the carbon footprint of business processes. in j. vom brocke, s. seidel, & j. recker (eds.), green business process management (pp. 93-109): springer. rehan, s., bandara, w., erica, f., & glenn, s. (2018). getting it right! critical success factors of bpm in the public sector: a systematic literature review. australasian journal of information systems, 22(0). doi:10.3127/ajis.v22i0.1265 reiter, m., fettke, p., & loos, p. (2014). towards green business process management: concept and implementation of an artifact to reduce the energy consumption of business processes. paper presented at hawaii international conference on system sciences (hicss), 2014 rogelj, j., den elzen, m., höhne, n., fransen, t., fekete, h., winkler, h., . . . meinshausen, m. (2016). paris agreement climate proposals need a boost to keep warming well below 2 °c. nature, 534(7609), 631-639. doi:10.1038/nature18307 roohy gohar, s., & indulska, m. (2015). business process management: saving the planet? proceedings of the australasian conference on information systems (acis). adelaide, australia, 30 november–4 december 2015; pp. 1–14. rosemann, m., & vessey, i. (2008). toward improving the relevance of information systems research to practice: the role of applicability checks. mis quarterly, 1-22. rosemann, m., & vom brocke, j. (2010). the six core elements of business process management. in handbook on business process management 1 (pp. 107-122): springer. rosemann, m., & vom brocke, j. (2015). the six core elements of business process management. in j. vom brocke & m. rosemann (eds.), handbook on business process management 1 (pp. 105-122): springer. sanders jones, j. l., & linderman, k. (2014). process management, innovation and efficiency performance: the moderating effect of competitive intensity. business process management journal, 20(2), 335-358. australasian journal of information systems roohy gohar & indulska 2020, vol 24, research article environmental sustainability through green bpm 27 savitz, a. (2012). the triple bottom line: how today's best-run companies are achieving economic, social and environmental success--and how you can too: john wiley & sons. schlagenhaufer, c., & amberg, m. (2015). a descriptive literature review and classification framework for gamification in information systems. paper presented at the european conference on information systems (ecis), münster, germany. schmidheiny, s. (1992). changing course: a global business perspective on development and the environment (vol. 1): mit press. schoormann, t., behrens, d., & knackstedt, r. (2017). sustainability in business process models: a taxonomy-driven approach to synthesize knowledge and structure the field. seidel, s., & recker, j. (2012). implementing green business processes: the importance of functional affordances of information systems. paper presented at the acis 2012: location, location, location: proceedings of the 23rd australasian conference on information systems 2012. seidel, s., recker, j., & brocke, j. (2012). green business process management. in j. vom brocke, s. seidel, & j. recker (eds.), green business process management (pp. 3-13): springer berlin heidelberg. seidel, s., recker, j. c., pimmer, c., & vom brocke, j. (2010). enablers and barriers to the organizational adoption of sustainable business practices. paper presented at the proceeding of the 16th americas conference on information systems: sustainable it collaboration around the globe. seidel, s., vom brocke, j., & recker, j. c. (2011). call for action: investigating the role of business process management in green is. sprouts: working papers on information systems, 11(4). sharma, s. (2000). managerial interpretations and organizational context as predictors of corporate choice of environmental strategy. academy of management journal, 43(4), 681697. stern, p. c., dietz, t., abel, t., guagnano, g. a., & kalof, l. (1999). a value-belief-norm theory of support for social movements: the case of environmentalism. human ecology review, 81-97. stolze, c., semmler, g., & thomas, o. (2012). sustainability in business process management research–a literature review. paper presented at the amcis 2012, seattle. sullivan, r., & wyndham, h. (2001). effective environmental management. principles and case studies allen and unwin sydney, australia. thies, h., dada, a., & stanoevska-slabeva, k. (2012). the potential of a network-centric solution for sustainability in business processes. in green business process management (pp. 181-201): springer. tinsley, s., & pillai, i. (2006). environmental systems management: understanding organisational drivers and barriers. in: earthscan: london. triantaphyllou, e. (2000). multi-criteria decision making methods. in multi-criteria decision making methods: a comparative study (pp. 5-21): springer. australasian journal of information systems roohy gohar & indulska 2020, vol 24, research article environmental sustainability through green bpm 28 united nations. (2016). the sustainable development agenda. retrieved from https://www.un.org/sustainabledevelopment/development-agenda. united nations environment programme. (2014). unep 2014 annual report ( dcp/1884/na). retrieved from http://apps.unep.org/publications/pmtdocuments/unep_2014_annual_report-2015unep_annual_report_2014_production_lq.pdf.pdf van der aalst, w. m. (2004). business process management demystified: a tutorial on models, systems and standards for workflow management. in lectures on concurrency and petri nets (pp. 1-65): springer. vom brocke, j., & sinnl, t. (2011). culture in business process management: a literature review. business process management journal, 17(2), 357-378. wabwoba, m. f., wanyembi, g. w., omuterema, s., & omieno, m. k. k. (2013). green ict readiness model for developing economies: case of kenya. international journal. wathern, p. (2013). environmental impact assessment: theory and practice: routledge. world commission on environment and development. (1987). our common future: report of the world commission on the environment and development(wced) [bruntlandt report]. general assembly, united nations, forty-second session, supplement no. 25, a/42/25. also published as our common future. 1987. oxford and new york: oxford university press. webster, j., & watson, r. t. (2002). analyzing the past to prepare for the future: writing a literature review. management information systems quarterly, 26(2), 3. weske, m. (2012). business process management: concepts, languages, architectures: springer science & business media. weske, m., van der aalst, w. m., & verbeek, h. (2004). advances in business process management. data & knowledge engineering, 50(1), 1-8. wesumperuma, a., ginige, a., ginige, j., & hol, a. (2013). green activity based management (abm) for organisations. proceedings of the 24th australasian conference on information systems (acis). rmit university; 2013; pp. 1‐ 11. wesumperuma, a., ginige, j. a., ginige, a., & hol, a. (2011). a framework for multi-dimensional business process optimization for ghg emission mitigation. proceedings of the 22nd australasian conference on information systems (acis), sydney, australia; december 2011 wesumperuma, a. e. (2015). multi-dimensional business process optimisation for greenhouse gas (ghg) emission management. university of western sydney, australia. wetzstein, b., ma, z., & leymann, f. (2008). towards measuring key performance indicators of semantic business processes. proceedings of the international conference on business information systems (pp. 227-238). springer, berlin, heidelberg. whitelaw, k. (2004). iso 14001 environmental systems handbook: routledge. young, c. w., & rikhardsson, p. m. (1996). environmental performance indicators for business. eco‐management and auditing, 3(3), 113-125. australasian journal of information systems roohy gohar & indulska 2020, vol 24, research article environmental sustainability through green bpm 29 appendix (• concept is addressed) table 5 distribution of research contributions to es and bpm concepts in 49 analysed papers r ef er en ce s p ub lis hi ng y ea r m an ag em en t s tr at eg y c ul tu re e ne rg y c on su m pt io n c o 2 f oo tp ri nt g h g e m is si on (o th er ) r ec yc lin g w as te m an ag em en t w at er c on su m pt io n g re en b p m d ef in it io n e xt en si on p ro ce ss o pt im iz at io n p ro ce ss d es ig n p ro ce ss r ee ng in ee ri ng b us in es s p ro ce ss m od el lin g n ot at io n e xt en ti on p ro ce ss p er fo rm an ce m ea su re m en t m et ho ds e xt en si on c ap ab ili ty m at ur it y m od el e xt en si on p ro ce ss a rc hi te ct ur e e xt en si on b p m l if ec yc le e xt en si on a rd ag na e t a l. 20 08 20 08 l l l l p er ni ci e t a l. 20 08 20 08 l l l l g ho se e t a l. 20 09 20 09 l l l l h oe sc hk lo he a nd g ho se 2 01 0 20 10 l l l l l l l l l l l s ei de l e t a l. 20 10 20 10 l l l h oe sc hk lo he e t a l. 20 10 20 10 l l l l l l l w es um pe ru m a et a l. 20 11 20 11 l l l l n ow ak e t a l. 20 11 d 20 11 l l l l l l l s ei de l e t a l. 20 11 20 11 l l l l l n ow ak e t a l. 20 11 b 20 11 l l l r ec ke r et a l. 20 11 20 11 l l l n ow ak e t a l. 20 11 c 20 11 l l l h ou y et a l. 20 11 20 11 l l n ow ak e t a l. 20 11 a 20 11 l l l h ou y et a l. 20 12 20 12 l l l l w at so n et a l. 20 12 20 12 l l h oe sc hk lo he a nd g ho se 2 01 2 20 12 l l l l l l s ei de l e t a l. 20 12 20 12 l l s ei de l a nd r ec ke r 20 12 20 12 l l l o pi tz e t a l. 20 12 20 12 l c le ve n et a l. 20 12 20 12 l l l l r ec ke r et a l. 20 12 20 12 l l l l l n ow ak e t a l. 20 12 20 12 l l s to lz e et a l. 20 12 20 12 t hi es e t a l. 20 12 20 12 l l l l l l l l l h as an 2 01 2 20 12 l l l n ow ak e t a l. 20 13 20 13 l l l l c ap pi el lo e t a l. 20 13 20 13 l l l l l w es um pe ru m a et a l. 20 13 20 13 l l l l n ow ak a nd l ey m an n 20 13 20 13 l l l l o pi tz e t a l. 20 14 b 20 14 l o pi tz e t a l. 20 14 a 20 14 l r ei te r et a l. 20 14 20 14 l g oh ar a nd i nd ul sk a 20 15 20 15 l l l l l l l l l w es um pe ru m a et a l. 20 15 20 15 l l l k up pu sa m y an d g ha rle gh i 2 01 5 20 15 l l p . l üb be ck e et a l. 20 16 20 16 l l ja ko bi , c as te lli e t a l. 20 16 20 16 l l l l l l p . l üb be ck e et a l. 20 16 20 16 l l l l l m ac ie l, j. c . 2 01 7 20 17 l l l l l l l l l l l s ch oo rm an n et a l. 20 17 20 17 l l l c ou ck uy t 2 01 7 20 17 l l l üb be ck e et a l. 20 17 20 17 l l l l c ou ck uy t e t a l. 20 17 20 17 l d eb na th 2 01 8 20 18 l l l l l l l üb be ck e et a l. 20 18 20 18 l l l h er ná nd ez g on zá le z et a l. 20 19 20 19 l l s ta dt lä nd er e t a l. 20 19 20 19 l l c ou ck uy t a nd v an l oo y 20 19 20 19 l australasian journal of information systems roohy gohar & indulska 2020, vol 24, research article environmental sustainability through green bpm 30 copyright: © 2020 roohy gohar & indulska. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v24i0.2057 http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems zhu, nimmagadda, reiners & rudra 2020, vol 24, research article integrated search framework for the knowledge-based web ecosystem 1 an integrated search framework for leveraging the knowledge-based web ecosystem dengya zhu curtin business school curtin university shastri lakshman nimmagadda curtin business school curtin university shastri.nimmagadda@curtin.edu.au torsten reiners curtin business school curtin university amit rudra curtin business school curtin university abstract the explosion of information constrains the judgement of search terms associated with knowledge-based web ecosystem (kbwe), making the retrieval of relevant information and its knowledge management challenging. the existing information retrieval (ir) tools and their fusion in a framework need attention, in which search results can effectively be managed. in this article, we demonstrate the effective use of information retrieval services by a variety of users and agents in various kbwe scenarios. an innovative integrated search framework (isf) is proposed, which utilises crawling strategies, web search technologies and traditional database search methods. besides, isf offers comprehensive, dynamic, personalized, and organization-oriented information retrieval services, ranging from the internet, extranet, intranet, to personal desktop. in this empirical research, experiments are carried out demonstrating the improvements in the search process, as discerned in the conceptual isf. the experimental results show improved precision compared with other popular search engines. keywords: integrated search framework, information retrieval, search engine, text classification, digital ecosystem, information management, crawler 1 introduction diverse domains and contexts make systems complex, effecting retrieval of data and information more challenging. the digital ecosystem (gartner, 2017) is an open community which holds data from associated communities and at the same time produces consumable data to benefit others. more specifically, people and enterprises in a digital ecosystem share standardized digital content in various platforms, mutually beneficial to each other. moore et al. (2018) assess the digital value in a case study with more than 3000 executives as “digital ecosystems are transforming the way their organisations deliver value” (moore et al, page 5). however, data in digital ecosystems are distributive, complex, heterogeneous, and multidimensional (barrows and traverso, 2006). in other words, they have all the features of big data. for example, the volume of the data generated per hour in a digital ecosystem ranges australasian journal of information systems zhu, nimmagadda, reiners & rudra 2020, vol 24, research article integrated search framework for the knowledge-based web ecosystem 2 from megabytes to gigabytes to terabytes; tens of thousands of bytes of data are transported per seconds, demonstrating velocity of data. for a variety of data, formats vary from, for example, emails to instant messages and images to streaming data. the heterogeneity and multidimensionality of ecosystems and their data sources force the management information systems to make amendments, for creating scopes of better information retrieval methods and search functions, adaptable to a new information-access era. information retrieval needs more specific search tools and formulations in addition to the presentation of search results in a way they can be better interpreted and analysed. multiple information scenarios of web ecosystems make businesses shifting their focus to new flexible, re-configurable, and collaborative search models. consequently, they should adapt to digitalization trends, and strive to leverage their data for competitive advantage. with the vast amount of data in hand, one of the distinct steps for organizations is to facilitate the data search process by easing the complexity of digital web ecosystems. for example, commonwealth bank of australia (cba) developed an app that assists users to search properties that they are interested in with sale price history as well as similar information of other properties nearby, and then with mortgage link points to cba. after six months, consumers made more than 1.2 million property searches, and the app’s return on investment is 109% (weill and woerner, 2015). for exploring web-based digital ecosystems, relevant enterprise-level search tools/platforms are described in barrows and traverso (2006). the apache solar and lucene are typical examples of the type of search tools, which provide services at an enterprise level (mccandless et al. 2010). they offer distributed indexing/searching services with high scalability, availability and extensibility features. the customized version may include an entity extractor, thesaurus, classification, filtering and other characteristics. desktop search that focuses on retrieving local files residing on personal desktop computers, messages, emails, and browsing history, is another type of searching tool. the third type of such gadget is an intranet search engine, which crawls information from servers within an intranet to local networks. for resolving enterprise-scale problems, a search engine should support and combine tagging, categorization and navigation tools to improve the end-user experience. an enterprise metadata category – ontology-based metadata – can be built to define a metadata schema, to index a set of documents, and write a user interface for querying and displaying results. even though automatic metadata extraction is never perfect, a user interface is needed to allow amendments and re-use of the metadata. in addition, an ecosystem that is supported by kbwe should satisfy scalability, security, metadata update, view privilege and query (search-term) optimization criteria (albro, 2006). the introduction has motivated us exploring the current literature to identify the research purpose in the contexts of information retrieval from not only large size data sources but also the type of support needed to retrieve the relevant facts and ascertain their evaluable precision. in addition, we review the existing ir designs with implementable framework, as described in the following sections. 2 literature review several researchers have described the existing concepts, tools, technologies and challenges to examine the data and ir from the world wide web (www). we examine the existing literature on managing large and complex datasets with traditional hardware and software australasian journal of information systems zhu, nimmagadda, reiners & rudra 2020, vol 24, research article integrated search framework for the knowledge-based web ecosystem 3 systems and the frameworks that can process and retrieve required information with precision. the current information retrieval models and how they can replace the kbwe guided isf and leverage their evaluations in knowledge management are described. 2.1 information retrieval from big data the traditional database systems and analytic tools have weaknesses to process the big data, which is unstructured or semi-structured with a large amount of image and audio data (haneef et al. 2018). as an example, managing heterogeneity of big data, in particular, data volumes and varieties with deliverable quality and precision through ir systems are challenging. for retrieving large size images, data related to audio and videos including social media posts and web blogs, tools for interrogation of big data, data warehouse technologies, and data analytics are all supportive in the information retrieval research. mccreadie et al. (2012) developed an information retrieval mechanism to improve the textual information using map/reduce to process terabyte size data files. they demonstrated that the proposed per-posting list indexing strategy was the most efficient indexing strategy, which leveraged a combination of both local machine memory and compression techniques to attack the i/o intensive weakness of map-reduce. soille et al. (2018) proposed a petabyte-scale platform that consists of hardware and open-source software, distributed file system, and task schedulers for batch processing with containerization of user-specific applications. they proposed an interactive geo-spatial visualization and processing with a series of applications together with performance metrics. gregory et al. (2019) focused on data search and retrieval from open sources of social networks. they organized a bibliometric study with a focus on information retrieval of user profiles and analysed their behaviour through contextual knowledge, facilitating the design of data discovery systems. 2.2 information retrieval framework dean (2009) described the requirements for complex design trade-off while building and operating large-scale ir systems to manage by millions of users. the author emphasised on several user-queries with response latency, the size of various corpora-search, the latency and frequency with which documents were updated or added to the corpora, including the quality and cost of the ranking algorithms for retrieval. in addition, the author focused on google hardware and ir systems and their design challenges. yang et al. (2015) presented adaptable ir dynamic systems with a sequence of events occurring in artificial intelligence and reinforcement learning. the authors offered ir solutions in a changing environment to learn from past interactions and predict future utilities. in addition, the authors presented advances in it interfaces with personalization and ad-display demand models through which users intelligently and contextually responded to ir systems in real-time. behnert and lewandowski (2017) described an information retrieval framework for online public access catalogues stored in digital library information systems, demonstrated web search features from heterogeneous library data sources. however, keeping in view the heterogeneity different elements of library information systems posed connectivity challenges and compromised the information quality. kumar et al. (2016) have developed a framework for data centres to retrieve information, acknowledging the redundancy, security and integrity. the data centres were typically geographically located and characterised as voluminous in the cloud-computing environment. domain experts further facilitated the information retrieval framework to upgrade the tools in big data scale. yue (2011) described machine-learning techniques that could help resolve the complexity involved in information australasian journal of information systems zhu, nimmagadda, reiners & rudra 2020, vol 24, research article integrated search framework for the knowledge-based web ecosystem 4 retrieval models. further, the author asserted that the learning framework could simplify the overall development process, advocating structured prediction and interactive learning, realizing the feature-rich retrieval models. seyler et al. (2018) proposed a framework for leveraging the contextual information for which users, documents and contextualized manuscripts were considered as heterogeneous objects. graphical embedding was created to learn the objects in the same semantic vector space. jung (2007) described the heterogeneity of web information spaces, creating uncertainty among search engine users. the author proposed a mediator agent system to estimate the semantics of unknown spaces by learning the fragmented gathers and applying to crawlers. however, the integration of contextual spaces and their semantics were missing due to poorly associated tools. further, the detachment between contextualisation features and tools was an added challenge while managing complex queries and information search. hernandez et al. (2007) explored thematic views of users’ specific data motivating semantically indexed contexts and their ontology descriptions. though the approach has benefit to the semantic representation of contexts, flexibility and adaptability challenges persist. liu et al. (2010) computed the rank of web pages using real browsing behaviours of web users. the authors described the issues and challenges of the hypergraph link as incomplete and inaccurate when calculate page rank. instead, they used the real behaviours of the web users. qin et al. (2010) conducted experiments on benchmark datasets, suggesting practical algorithms for the proposed framework to optimise the ir measures. zuccon et al. (2013) used a crowdsourcing platform that captured the user interactions, searching their behaviours at low cost with more data within short period times. still, the approach compromises the quality of information and its search. simpson et al. (2014) proposed a practical multimodal solution for indexing and retrieving the images contained in the biomedical articles. text-based visual representations were used in conjunction with the solutions that significantly improved the retrieval accuracy. soldaini et al. (2016) investigated the utility of bridging the gaps between layperson and expert vocabularies for query classifications from which a supervised classifier is selected with the most appropriate synonym that best fit with the query. despite precision in terminology presentation, the classifier lacks collaboration with complex queries. koopman et al. (2016) present a graphical inference retrieval model that integrates structured knowledge with statistical information and its inference in a unified framework. the analysis suggested when and how to apply the inference for retrieval, including categorization of queries affected by inferences. though the authors concluded that the inference retrieval method was more effective compared to a general retrieval method, the inference analysis lacked relevant judgement while choosing queries. tolosa et al. (2017) lately used information retrieval systems at unique stages to speed up computations. the authors proposed a static cache and evaluate its space for query executions and differences to reveal between raw and compressed forms. karanam et al. (2017) discussed a cognitive model, building a process for information search and enabling the use of semantic spaces. the study concluded with an interpretation of high-domain knowledge in an expert-semantic space irrespective of the knowledge acquisition and lack of refinement process. information retrieval and its refinement is much-needed research using an integrated framework. previous information retrieval researchers have not dealt with a new integrated approach that can cope with complex queries and information needs. we propose a australasian journal of information systems zhu, nimmagadda, reiners & rudra 2020, vol 24, research article integrated search framework for the knowledge-based web ecosystem 5 framework to accommodate multiple constructs and models, envisioning the refinement in the search process. we articulate the framework with new artefacts that can search and deliver quality information, minimising the ambiguity of information constrained by various search engines. 3 the existing issues and challenges the information explosion, low information accuracy, improper search outcomes and their management, mismatched human-computer interaction clusters are significant issues, making the information retrieval challenging (croft et al. 2015). a dynamic and flexible information retrieval system is needed to provide services to users in various kbwe scenarios. in digital ecosystems, other challenges include inadequately integrated domains, systems and their associated data sources, which are briefly summarised in the following sections. integrated search tools: specific search tools and functions exist, for example, desktop, music, language-specific searches, and an explicit full-text database with bibliographic searching, in addition to the general-purpose web search engines, such as google, bing and yahoo. however, information seekers must install the tools on their computers, and then match the search tools/functions with their information retrieval needs. the process may involve considerable trial and error and investment in learning a variety of systems. substantial resources are used such as time, memory, disk space and processing power, in particular, accessing the high-resolution images. on the other hand, integrated search function and data ingest tools can ease the issues of separate search engines and tools by providing a more effective search for a particular domain or field, as compared with the general-purpose search engines and individual search tools. the syntax-based search is not necessarily semantic centric: when the search is syntactical, but not semantic-based, the search results may not be adequately relevant. web search engines in such cases look for factual similarities between search-terms and the web pages (arasu et al. 2001). search engines crawl websites from the internet and download web pages from various sites. content of the crawled pages are tokenized and indexed. all tokenized terms are used to create a set of vocabulary, and a term-document matrix can thus be generated. accordingly, similarities between search-terms and documents are estimated by ranked-based algorithms such as vector space or probability models (manning et al. 2009). during this process, the semantic characteristics and issues of search-terms may have been incorrectly elaborated. as an example, it is not a surprise when using “ups” as a search term to retrieve information about the “uninterruptible power supply” that may return irrelevant or ambiguous results such as the “united parcel services”. similar is the case with idiom associated with ambiguous “jaguar”. untailored search results: wherever contextualized and personalized search is not commonly considered significant, still, most search engines make an effort to return search results based on general-purpose search (croft et al. 2015). no matter what role a searcher has – a car sales representative, an environmentalist, or a computer technician – if they all search “jaguar”, they get the same search results. however, submitting the query, the sales representative might think of the “jaguar” car and not anything else. the environmentalist seeks information about the animal jaguar, whereas the it technician might think of using apple’s jaguar as an operating system. the general-purpose search tools thus need improvement to get quality and relevant search results, for example, by creating user profiles. however, croft et al. (2015) australasian journal of information systems zhu, nimmagadda, reiners & rudra 2020, vol 24, research article integrated search framework for the knowledge-based web ecosystem 6 indicate that using such models does not improve the effectiveness of ranking on its average. on the other hand, local geography-based search, and contextual information extracted from users’ interaction with search engines are promising approaches (croft et al. 2015). although some search engines provide personalised results, such as google (bunz 2009), retrieval and precision are still a concern, as discussed in croft et al. (2015). enterprise-level search personalisation: the customisation and personalisation features move conjointly at the enterprise level. personalisation ensures delivering the content and functionality that match with specific users’ needs, what they search for in pursuit of innovative and new terminologies. netflix is an example, which has established a market with the adaptability of user views and search-terms. without any clue of queries, most search engines rank results based on the general-purpose search. however, arnold (2004) cites that the enterprise search is not a web search, although it can manage the indexing of content that resides on the internet sites. enterprise search can even support the queries from special classes of authorized users, as in the extranet. integrated holistic enterprise-level search engine: enterprise software applications typically control the search functions, making adaptable to integrated search engines. more popular gmail, microsoft outlook and other email services have built-in search functions. microsoft sharepoint built-in search allows users to search sharepoint pages. nevertheless, users in an enterprise need to frequently change several portals and explore different types of data relevant to business operations. as we move forward with new emerging information needs and queries through ir innovations, including for evaluable measures, we articulate a unified and adaptable framework with different artefacts. semantic knowledge from multiple multimedia sources has significance in analysing the information needs and queries that motivated us to explore alternate ir solutions. with a sheer volume of multimedia information, indexing bears challenges of manual annotations. our research explores for an integrated framework with improved search algorithms and evaluable measures to facilitate the usability and evaluation criteria. the deficiencies existing in the dynamic kbwe that need a rule-based new algorithmic isf approach with unified articulations have motivated us to draw research objectives and leverage the kbwe. 4 research goal, motivation and significance based on the current information retrieval challenges, we outline the research questions and objectives. research questions (rq) are: 1. how do we design and develop a structured and unified information search framework? 2. what types of algorithms can achieve high precision search results? 3. how do we accomplish the information search through isf and evaluate the results? the corresponding research objectives (ro) are: 1. develop isf that can integrate a variety of search engine tools and applications. 2. improve search experience using algorithmic approaches. 3. analyse and evaluate the search results of the isf. australasian journal of information systems zhu, nimmagadda, reiners & rudra 2020, vol 24, research article integrated search framework for the knowledge-based web ecosystem 7 the goal of the research is to design and develop a framework to search for unified information and validate the given model through a series of experiments with evaluable measures. another research target is to extract structured textual data that can be shared between different search engines/tools and design the type of framework needed to explore such fused information. increased search efforts and outcome of terms searched in various contexts is the motivating factor for investigation of the new framework. activation, persistence and intensity of search terms and their findings usable by type of users motivate us to develop new search articulations. the integrated methodological approach can minimise the ambiguity involved in exploring further information and knowledge acquisition through interfaces and desktop integration. 5 research methodology as described in research objective 1, we adopt a design science approach, with the methodology motivating a rigor in data science, based on empirical evidence. as a part of design science, we take the support of the approach used in vaishnavi and kuechler (2007) and baskerville et al. (2015), construct coherent and intelligible artefacts within an integrated framework. in pursuance of research objective 2, we utilise various algorithmic approaches to build search tools. as a part of empirical research, we propose to analyse the experimental data relevant to search terms and corresponding information needs from multiple search engines and present them for their performance. figure 1: an integrated search framework (isf) search results categorization request analysis user profilequery full text db desktop search intranet search engine meta search engine search results security scrutiny information representation web crawler db in dess filtering ontology category selection part 1. information streams part 2. categorizing/clustering part 3. query expansion part 4. personalization indexed docs indexer searcher cluster solr/lucene search engines user nosql hbase australasian journal of information systems zhu, nimmagadda, reiners & rudra 2020, vol 24, research article integrated search framework for the knowledge-based web ecosystem 8 as per research objective 3, we carry out various experiments and evaluate the performance of isf in terms of precision, recall and f1 scores. in this context, we bring the concept of integrated search framework (isf), as illustrated in figure 1, put forward with kbwe representation, and articulated with various artefacts. the artefacts are detailed in the following sections. 5.1 information streams when users submit search requests, search-terms match with information from diverse sources including the internet, intranets, full-text databases, databases of digital ecosystems and personal desktop computers. search results are categorized by svms or naïve bayes based on the ontologies associated with web ecosystems. search results are also clustered by example-based algorithms such as knn to further facilitate user to disambiguate semantic features of the search-teams. the search processes are described in the following sections, with the functionality of each component, as explained in figure 1. the framework is divided into four main parts, namely, information streams and sources; query or search-term expansion; search results categorizing, clustering and filtering; and personalised search results’ representation. it is worth mentioning that isf is independent of any specific platform. it can be deployed on an intranet computer cluster with hadoop ecosystem, as we experiment on our big data lab’s hadoop cluster. it can also be deployed on a public cloud, which provides iaas that supports distributed computational and storage capability. it can also be deployed on a hybrid cloud that can provide enhanced security. we can keep sensitive data in the private cloud while the crawled public-accessible data can be stored in a public cloud. deployment of isf is transparent to end-users, and from a business perspective, a hybrid cloud model is appropriate for most application scenarios, but not limited to any specific solutions. in short, no matter which platform an organization adopted for its it infrastructure, isf can easily be deployed. search process: the research objective 2 motivates us devising new algorithms for retrieving information on the internet. a programmable script is designed as a crawler for browsing the www. currently, the crawlers emerge with large-size scripts, complexity and rapid growth of the www. with the result, the page selection, importance, recency and refresh options create many operational challenges, including limitation of network bandwidth and disk space needed to run the isf. other constraints are the rapid update of web content which can make crawled pages obsolete, and limited coverage of web content of a crawler has the potential of missing relevant data from the web (baeza-yates et al. 2005). as a consequence, in addition to regular page refresh, crawlers should focus on the effectiveness of crawling. usually, a crawler starts with an initial set of urls that are placed in a queue and are prioritised. a url is selected based on specific ordering strategies. the crawlers download web pages, extract urls from the downloaded pages, and put the new urls in the queue, expanding into (crawling) relevant websites. this process is repeated until crawlers decide to terminate. prioritising the urls in the queue and setting the stop conditions are both related to estimating, or measuring the relevance of the url content to the semantic need of digital web ecosystems. pagerank (brin and page, 1998) is used to evaluate the relevance of web pages. supposed that at a given time, one web surfer may follow a link with fixed probability δ, then the probability of selecting a page uniformly at random and jumping to the page is 1 – δ. if the user can browse australasian journal of information systems zhu, nimmagadda, reiners & rudra 2020, vol 24, research article integrated search framework for the knowledge-based web ecosystem 9 the web quickly and tirelessly, the value of pagerank γ(α) is an estimator of the probability that the surfer visits page α. 𝑟(𝛼) = 𝛿 ∙ ( ∑ 𝑟(𝛽) 𝑜(𝛽) 𝛽→𝛼 + ∑ 𝑟(𝛾) 𝑁 𝛾𝜖𝛤 ) + (1 − 𝛿) ∙ ∑ 𝑟(𝛼) 𝑁 𝛼𝜖𝛷 where β→α is the set of all pages that point to page α, and o (β) is the total number of out links from page β. γ is the set of all sink pages, which also contribute its page rank to α. further, since the choice of α is arbitrary, so we have ∑ 𝑟(𝛼) = 𝑁𝛼𝜖𝛷 , thus 𝑟(𝛼) = 𝛿 ∙ ( ∑ 𝑟(𝛽) 𝑜(𝛽) 𝛽→𝛼 + ∑ 𝑟(𝛾) 𝑁 𝛾𝜖𝛤 ) + (1 − 𝛿) figure 2: the crawling strategy for the isf as illustrated in figure 2, the proposed crawling strategy consists of the following stages. firstly, a set of seed urls, which contains all relevant websites picked up by ecosystem domain experts, is created and stored in a url link-database (linkdb). the set of urls are then injected in the url list (urls, also url frontier). the seed url list is adjustable based on users’ business requirements. for example, an environmental research institute can put only a list of websites related to their research interest into the seed url list, such as www.environmentalscience.org/sustainability and environmentalprofessionalsnetwork.com/. with the seed urls injected into the url frontier, a crawler downloads all webpages in the urls and stores the extracted content into a database (db). a search engine indexes the content after the documents are pre-processed (stop word removing, stemming, named entity extraction). at the same time, urls contained in the downloaded webpages are extracted and then put into linkdb after duplicates and irrelevant urls are removed by applying a filtering strategy, which preserves only pages that are crawled from the domains included in the seed urls. for all the downloaded webpages in the db, some of them may not be as relevant as others. therefore, a document categorisation algorithm such as support vector machines is trained by using existing documents stored in the ecosystem as a training data set. only the relevant documents moved from db to the relevant database (rdb) are indexed, where the pagerank calculates the relevance of the documents. following is the pseudo-code and a brief explanation for the crawling algorithm with pagerank value priorities as described in figure 3. 1. load in seeds urls and connect to linkdb 2. extract urls that can be crawled in next crawling round urls urls urls urls linkdb db crawler db crawler rdb crawler db rdbrdb seed ruls australasian journal of information systems zhu, nimmagadda, reiners & rudra 2020, vol 24, research article integrated search framework for the knowledge-based web ecosystem 10 3. content parse and process 4. repeat. after applying relevant filtering algorithms, the hardware storage requirement can be reduced dramatically. in our big data lab (with six servers plus two edge nodes, total 16tb hd, 416gb ram), we have crawled over 10 million news articles in an hadoop ecosystem, and the total hard drive used for the crawled pages is only about 140gb in total. therefore, isf’s filtering algorithms and configurations enable us to keep all relevant data in our private db. search aggregators: a metasearch engine is a system that provides unified access to several existing search engines (meng et al. 2000). the aggregator in isf is based on the following considerations: 1) single search engine’s processing power may not scale to the big increase and virtually unlimited amount of data; 2) it is hard or even impossible for a single search engine to index all the data on the web and keep it up to date; and 3) some ‘deep web’ sites may not allow their documents to be crawled by external websites, but allow their documents to be accessible for respective search engines. figure 3. crawler algorithm in isf the conceptual architecture of the meta-search engine is illustrated in figure 4. meta-search starts with initializing the user query and selecting a set of suitable databases (coupled with search engines) by the database selector. the document selector chooses the number of documents that are to be crawled from the component search engines. the local similarity threshold can also be used to limit the documents retrieved from the component search engines. query dispatcher connects the server with each of the selected search engines and passes the query to them. results manager integrates crawled results from search engines into a single ranked list and renders it to the user (meng et al. 2000). isf crawlor; initilising: starting: load in seed urls from a text file and put it into linkdb; initialising priorityqueue with linkdb; crawling: while(priorityqueue not empty) { read urls from priorityqueue; distribute urls to parallel crawler s waiting queue; foreach parallel crawler: { if(waiting queue not empty) { fetch content from url in the waiting queue; put fetch content into db; remove fetched url from waiting list; } } parse fetched content from db; extract urls and put them into linkdb; run pagerank algorithm on linkdb; remove low pagerank value urls from linkdb; index content into solr; stop condition satisfied ? yes: stop; no: updating priorityqueue with urls in linkdb, goto crawling; australasian journal of information systems zhu, nimmagadda, reiners & rudra 2020, vol 24, research article integrated search framework for the knowledge-based web ecosystem 11 figure 4. meta-search engine structure (meng et al. 2000) reprising databases: a relational database management system (rdbms) typically manages the existing databases in an enterprise that can be re-used by isf in the kbwe contexts. in addition, the rdbms manages the metadata and other security-sensitive data, which usually provides security and integrity management (elmasri and navathe, 2016). in isf, rdbms is one of its data sources and constructive components. user queries are submitted to rdbms as well as part of the integrated search platform for structured data searching together with other extracted relevant information from all data sources. the retrieved results from the rdbms are then presented to the categorisation/clustering module for further processing, as illustrated in figure 1. in addition, the extracted entities such as title, gender, person name, address, email address, organisation, telephone number, and mobile phone number from text documents are stored in the rdmbs. these extracted entities are used to match records of rdbms and, thus, a connection between structured data in rdbms, and unstructured/semi-structured text data can be established, and verse vice. users of isf can search for data by providing a google-like search term and can inspect data in the rdbms as well. nosql database such as hbase data can also be ingested into isf. hbase (http://hbase.apache.org) is a schema-less nosql-style data storage designed for large scale, over billions of rows and millions of columns, random read and write operations of sparse data. the data in db and rdb, as shown in figure 2, is stored in hbase in isf. solr’s java api package solrj is used to index data stored in hbase. pseudo-code is presented in figure 5, indexing hbase data into lucene/solr cloud, an opensource search service that indexes data in isf and response queries for the indexed data with blazing-fast response speed. isf utilizes solrcloud to provide high availability, scalability and fault tolerance. hadoop and solrcloud enable isf the ability to linearly scale out its computations and storage capability by merely adding more working nodes to the cluster when more data storage and computations are required. figure 5. pseudo-code used for indexing user interface database selector document selector query dispatcher result manager search engine search engine ……………… australasian journal of information systems zhu, nimmagadda, reiners & rudra 2020, vol 24, research article integrated search framework for the knowledge-based web ecosystem 12 desktop search: as indicated by barrows and traverso (2006), google, microsoft, yahoo! and other major search engine players are providing free downloaded desktop search solutions. as the storage allows hundreds of gigabytes of data and beyond, the desktop search can significantly make better user productivity. in these models, indexing or tagging can enable users to access information through dynamic integrated navigational retrieval systems. they can act as agents to return pointers or links to the desired information. intranet engine: searching for information on the intranet is rather a daunting task, which is addressed by search tools development companies, such as google and thunderstone. because the performance of intranet search engines differs dramatically, isys search software (www.isysdemo.com) suggests that features should be considered carefully. however, in isf contexts, the intranet engine selection is the user’ choice. 5.2 search results categorisation and clustering knowledge-based hierarchical ontologies are considered for categorising and interpreting a considerable amount of information and its semantic content. another method is to arrange the itemized information into different clusters according to their similarities. in isf, these two approaches are combined to leverage the advantages of both tactics, as suggested in chau and chen (2008). fine-grained ontologies: the ontologies are created articulating various artefacts in the isf to improve the search process, categorising the search results based on semantics. the search results are further filtered based on the user selection and accordingly classified under the selected category for description and presentation. in isf, the open directory project (odp, www.odp.org) is employed as an ontology to present the web knowledge structure. the semantic characteristics of each category in the odp are manifested by a category-document that includes the topic of the odp category, the description of the type and a list of submitted web-pages (composed of the title of the web-pages and a brief description of each of the submitted web-pages) under this category (zhu et al. 2018). categorising search results: text categorisation is the problem of assigning pre-defined categories to free-text documents (croft et al. 2015). in isf, search results are categorised based on the odp as a lightweight ontology. the category-documents in the odp are employed as a training data set and an example-based categorization algorithm, which utilizes a vector space model (manning 2009), is described as following. let 𝑑𝑗 = {𝑤1,𝑗, 𝑤2,𝑗, … 𝑤𝑇,𝑗} be the j-th category-document, where t is the total number of vocabularies in isf, and 𝑞 = {𝑤1,𝑞, 𝑤2,𝑞 , … 𝑤𝑇,𝑞} is the search item; wi,j is the tf-idf (term frequency-inverse document frequency) weight of i-th term in j-th document, wi,q is the tf-idf weight of i-th term in the search item. the similarity between q and dj is estimated by the cosine value of the angle θ of the two vectors: 𝑠𝑖𝑚(𝑑𝑗 , 𝑞) = cos(𝜃) = 𝑑𝑗•𝑞 |𝑑𝑗| × |𝑞| = ∑ (𝑤𝑖,𝑗 × 𝑤𝑖,𝑞)𝑇 𝑖=1 √∑ 𝑤𝑖,𝑗 2𝑇 𝑖=1 ∑ 𝑤𝑖,𝑞 2𝑇 𝑖=1 the similarities between q and the dj, j = 1, 2 … n (where n is a total number of categorydocuments in isf) are ranked/sorted in their descending order. for top k ranked categorydocuments, suppose their corresponding odp category is ∁ = {c1, c2… ck}, q is assigned to the category selected from ∁ by dominant majority voting algorithm, as shown in figure 6. australasian journal of information systems zhu, nimmagadda, reiners & rudra 2020, vol 24, research article integrated search framework for the knowledge-based web ecosystem 13 figure 6. majority voting algorithm for text classification clustering: text clustering aims at assembling documents that are related among themselves and satisfy a set of features. it can be used to expand a user query with new and related index terms (croft et al. 2015) and to facilitate users to browse the retrieved results. many clustering algorithms are available, such as the k-mean clustering algorithm and fuzzy c-means. in isf, k-mean is chosen to cluster returned search results. two essential issues of k-means are how to decide a proper k and how to select the original k cluster centres. since the search results are categorized based on the odp category, the number of the first level categories under which search results are assigned is a suitable candidate for k in isf. meanwhile, the returned search result rk that is most similar to a candidate category cj, is assigned as the centre of a new cluster, and as a consequence, we can select k <= c clusters for search results clustering where k is the number of generated clusters and c is the total number of top-level odp categories. cosine similarity (as described above) is utilized to estimate the “likelihood” among neighbouring groups. filtering: google and facebook introduced personalisation features with improved algorithms that filter information as per user requirements and motivate the filtering process. we further analyse filtering processes to show how the personalisation can be linked to filtering techniques without any bias of human and computer interaction. search result filtering is decided in isf by two factors: one is the user’s selection of an existing category of the ontology; another factor is the pre-built user profile that is to be discussed in the next section. with a user choosing several categories, only search results categorised under the categories are presented to the user, and other information is filtered out. however, even if the user does not select any pre-defined odp categories, the search results are filtered as well by default based on a pre-built user profile. search results are compared with the features in user profiles and re-ranked accordingly to ensure only results having similar features described by user profiles are presented to the users. 5.3 expanding and analysing queries user profile and personalisation: a user profile is a reference ontology in which each concept has a weight indicating the user’s interest in that concept (croft et al. 2015). an information space of the odp (pitkow et al. 2002) is used to represent user models. as suggested in dolog and nejdl (2003), the user model combines two proposed standard learner profiles, ieee personal and private information and ims learner information package (lip) to express the features of a user. the precision and recall of metasearch engines are improved through mining association rules that reflect the user’ past search behaviour. isf cuts off the odp knowledge hierarchy from the second level to obtain 573 topics and uses these topics to represent users’ australasian journal of information systems zhu, nimmagadda, reiners & rudra 2020, vol 24, research article integrated search framework for the knowledge-based web ecosystem 14 search interests that are represented by <topic, weight> tuples. the userprofile is initialised by asking users to assign a weight (integer) to an existing topic to indicate how import the topic is. users are allowed to choose any number of interesting topics. to map a user search interest into these topics, for each search result ri visited by a user, let ci is the topic, as detailed in section 2, the corresponding weight of ci in the tuple is increased. query and request analysis: query augmentation and result processing are two primary uses of user profiles. in the isf scenarios, after a user selects an odp topic, the query takes as an alternative augmentation to user profiles, which allows the user to re-submit the query q + = q ∪ {tk, k = 1, 2 … k}, where q is the query submitted by the user, tk is the term selected using )()()()( )],(),(),(),([ 2 2 kkkk ikikikik cpcptptp ctpctpctpctpn − = where n are the 573 topics in the user profile, and these topics are now represented by categorydocument. p (t, c) is the joint probability of term t and category c; c , t indicate that c or t, respectively, do not appear in that category. k is the number of terms determined by the confidence p (λ2> 10.83) < 0.001 = 99.9% that the assumption of independence of the term tk and q can be rejected (manning et al. 2009). 5.4 representing personalised search results information representation: in the isf, search results are extracted from full-text databases, intranet, internet, traditional databases and desktop searches all results are integrated into one coherent information representation. users of isf can choose which data sources are to be included during the search process, thus providing the flexibility to access data sources to satisfy their information needs. search results from security scrutiny: the component in the isf performs a search result scrutinizing security feature. a security scrutiny task concerns “who is allowed to update a piece of metadata and view a particular piece of metadata about a document or know that the document exists at all” (barrows and traverso, 2006). we apply apache sentry with solr that makes isf with the ability to fine-grained control of access of data indexed by solrcloud. 6 experimental results järvelin (2007) has discussed two versions of ir approaches, one with the lab version on the design of retrieval models with refinements in evaluable query results, and the other relevant to cognitive ir approach with object-oriented data entities, structures and their relationships. in our research, we conduct experiments from www through kbwe using a number of queries and information needs. in this section, we use processes, methods, artefacts, and algorithms to retrieve information that is accountable to new knowledge from isf and comparable with other search engines. for ascertaining the research purpose and evaluating the search process through artefacts and articulations of isf, the data are generated through experiments with evaluable measurements. a pragmatic approach is adopted with initial experiments done to establish an information retrieval method and a search process mechanism as described in zhu et al (2018). additional experiments are carried out for testing the isf and its search precision through different queries, information needs, and validation of relevant judgements as interpreted in terms of the cut-off levels p@5 and p@10. australasian journal of information systems zhu, nimmagadda, reiners & rudra 2020, vol 24, research article integrated search framework for the knowledge-based web ecosystem 15 evaluable measurements: the two widely accepted measurements of information retrieval effectiveness are precision and recall. recall measures the ability of an information retrieval system to retrieve all relevant documents, whereas precision measures the ability of an information retrieval system to extract only relevant items. for text categorisation purposes, the two measures are defined as (manning et al. 2009): foundcategoriestotal correctandfoundcategories precision correctcategoriestotal correctandfoundcategories recall = = for the evaluation process, search engines are made available for retrieved documents in a ranked list according to the degree of relevance of the documents to a given query (alonso and mizzaro, 2009). users examine the ranked list starting from the top document. the recall and precision measures vary since the users proceed with an examination of the returned set. in addition to the overall precision recall measure, a precision versus recall curve based on eleven standard recall levels is usually employed to evaluate the ranked list (croft et al. 2015). another measurement is the precision at a given cut-off level. a cut-off level is a rank that defines the retrieved set. for example, a cut-off level of 10 represents the top ten retrieved documents in the ranked list. if seven out of ten of the returned documents is relevant, the precision at cut-off level ten (p@10) is 7/10 = 0.7 = 70%. experiment description: a special search browser (ssb), which is a component of isf, has been developed to categorise web search results from yahoo! under various categories of the odp (zhu et al 2018). five search-terms with general or ambiguous meanings are selected, as shown in table 1. for each search-term, 50 search results are retrieved by utilizing the yahoo! search web service api. the returned search results are presented to human judges to perform relevance judgment. the relevant judgment results are summarized using a weighted schema. a final binary decision is made regarding whether a returned search item is relevant to the specified information need or not. based on the relevance judgment results, the standard 11 recall-precision curves are drawn for each search-term of the search results of yahoo!, and of the categorised results of ssb, as shown in figure 7a. p@5 and p@10 of the search result sets are presented in table 2. query information needs clinton the american president william j clinton ford henry ford, the founder of ford motor company health state of physical, mental, and social well-being jaguar information about the animal “jaguar” ups information about “uninterrupted power supply” table 1. search terms and information needs p@5 p@10 average yahoo 46.7 42 44.4 ssb (isf) 85 70 77.5 improvement 38.3 28 33.2 table 2. p@5 and p@10 of yahoo and ssb australasian journal of information systems zhu, nimmagadda, reiners & rudra 2020, vol 24, research article integrated search framework for the knowledge-based web ecosystem 16 query google bing lycos ms live p5 p10 p5 p10 p5 p10 p5 p10 clinton 40 40 20 10 40 40 0 0 ford 20 30 40 20 20 20 20 20 health 100 100 100 70 100 80 60 40 jaguar 20 40 20 10 20 40 20 10 ups 20 10 0 0 20 10 20 20 average 40 44 36 22 40 38 20 42 29 39 21 table 3. p@5 and p@10 of google, ms live search, bing, and lycos the data documented in tables 2 and 3 summarize the relevant judgement results and the summary of estimation. following the macro-averaging style (manning et al. 2009), while drawing the standard 11-point recall-precision curve, the precision pj at recall level j is calculated by:  = == n i jij jp n p 1 , 10...,,1,0 1 n = 5 is the number of queries in the experiment, pi,j is the precision of the i-th query at j-th recall level. the overall precision is calculated by:  = = = 10 0 1 , 11 1 j n i jip n p relevance judgment: relevance judgment is inherently subjective (mizzaro, 1997). for easing the issue of subjectivity, we organized five judges from our university to conduct relevance judgement in the research, as discussed in zhu et al (2018). the human judges know nothing about the categorised results of the isf. according to the relevant judgement results of five judges, a final binary decision was reached for each returned search item of yahoo api. the experimental results in figure 7a and table 2 of isf are based on 50 search results for each of the five search terms of yahoo!. that is, the isf has categorised yahoo’s 50 results into different odp categories. in this context, the comparison between figure 7a and table 2 is a direct comparison. because isf does not categorise the search results of the rest of four search engines, as in figure 7b, for the reason they do not have corresponding search apis as provided by yahoo!, and in such sense, the comparison may not be considered straightforward. however, the indirect comparison also reveals that without applying the proposed search strategy, the performance of the search engines is relatively unsatisfactory regarding the recall-precision curve measures and the p@5 and p@10 evaluations. australasian journal of information systems zhu, nimmagadda, reiners & rudra 2020, vol 24, research article integrated search framework for the knowledge-based web ecosystem 17 figure 7 (a): average recall-precision curves of yahoo! and isf categorized search results over the five search-terms (clinton, ford, health, jaguar, ups) (b) average recall-precision curves of yahoo!, google, ms live search, bing, lycos, and the categorised results of isf1 to further test the isf, we have added five more queries with information needs as shown in table 4. the search results for p@5 and p@10 are presented in table 5. query information needs susan dumais the researcher susan dumais world war 2 history related to world war 2 graphic design the art and practice of graphical design jokes the funniest jokes time zones time zones of the world table 4. five more search terms and information needs query google yahoo bing aol.com baidu p@5 p@10 p@5 p@10 p@5 p@10 p@5 p@10 p@5 p@10 susan sumais 100 90 80 70 80 70 80 70 60 60 world war 2 100 100 100 90 100 90 100 100 100 80 graphic design 60 50 40 60 60 80 20 60 100 100 jokes 80 70 80 80 80 80 80 80 100 90 time zones 80 80 100 100 100 100 80 80 100 100 average 84 78 80 80 84 84 72 78 92 86 81 80 84 75 89 table 5. p@5 and p@10 instances of google, ms live search, bing, yahoo, aol.com and baidu to further verify the robustness and adaptability of isf, we have added another five more queries as represented in tables 6 and 7. 1 ssb is an earlier version of isf so we kept the name of ssb unchanged in this figure. (a) (b) australasian journal of information systems zhu, nimmagadda, reiners & rudra 2020, vol 24, research article integrated search framework for the knowledge-based web ecosystem 18 query information needs sports sport news soccer league names of soccer league teams viruses description of viruses coronavirus coronavirus status environment the environment species surviving table 6. five more search terms and information needs query google yahoo bing aol.com baidu p@5 p@10 p@5 p@10 p@5 p@10 p@5 p@10 p@5 p@10 sports 80 60 100 90 80 70 100 100 80 90 soccer league 80 100 80 70 80 60 60 70 80 90 viruses 100 100 80 70 80 70 20 60 60 70 coronavirus 80 90 100 80 60 50 40 60 60 70 environment 80 70 60 50 50 40 40 60 60 70 average 84 84 84 72 70 58 52 70 68 78 84 78 64 61 73 table 7. p@5 and p@10 instances of google, ms live search, bing, yahoo, aol.com and baidu a summary overview of the experimental data: • as shown in figure 7a, the overall average precision of the 50 search results of yahoo! is 458.8 / 11 = 41.7%. • the overall average precision of the categorized results is 716.7 / 11 = 65.2% • the average improvement on the precision of the categorised results is 65.2% – 41.7% = 23.5 (%) • table 2 demonstrates that isf improves p@5 and p@10 by 38.3% and 28.0%, respectively, with an average improvement of 33.2%. • to lessen the issue of subjectivity (mizzaro 1997), the relevant judgement is further analysed by three judges. figure 8 and table 5 demonstrate the recently conducted search results from multiple search engines. • table 5 illustrates bing and baidu search engines perform better retrieval than yahoo! and google, all outperform aol, for added queries described in table 4. • table 7 shows google exhibits better results compared with other search engines. 7 discussions search results of isf, google, bing, ms live and lycos for queries in table 1: figure 7b and table 3 reveal that 1) the isf outperforms other search engines regarding averaged precision based on standard 11 recall-precision curves and p@5 and p@10 in the experiment. 2) the performance of google and lycos is nearly the same; ms live search and ms bing perform relatively weak, as both the recall-precision curve and p@5 and p@10 demonstrated. 3), lycos performs better by the measure of the recall-precision curve, but only better than ms live search and bing when evaluated by p@5 and p@10 criteria. australasian journal of information systems zhu, nimmagadda, reiners & rudra 2020, vol 24, research article integrated search framework for the knowledge-based web ecosystem 19 search results of isf, google, bing, yahoo, aol and baidu based on all 15 queries: to verify the effectiveness of isf, recall-precision curve is drawn based on the search results of all fifteen-search terms given in tables 1, 4 and 6 and, as shown in figure 8. figure 8: comparative performances between various search engines figure 8 compares search engines by recall-precision curves. queries and their precision instances are given in table 3 and table 5 are combined in the schematic view representation in figure 8 to assess the search engine performances. the isf emerges as a performing tool, with a better search than any other investigating engines considered in the study. the recallprecision curves for all search engines show steady trends and patterns, compared with isf. the isf exhibits high precision at lower recall values, but its precision decreases at higher recall values. the curves of other search engines exhibit a concave-down shape, implying that the search precision and information qualities respond to early and late recall instances. the precision is steady for recall range of 0.3 – 0.9. isf appears more sensitive to higher recall values. however, we deal with a trade-off between the information needs and queries, because of the broader scope of existing search engines and ambiguities arisen in among complex queries, information needs and interrogative relevant judgements. the google, bing, yahoo, aol.com and baidu search engines could return significant results for all p@5 and p@10 queries for the five new queries in table 4, but relatively weak for queries in table 1, except “health”, which we give a more general information need. the information need “clinton” has delivered weak results for all search engines. syllables “ups and jaguar” appear to be ambiguous for all search engines that returned comparatively insignificant results for both queries. as shown in table 3, although ms live could return relevant information for “health”, for the rest four queries, the formed clusters are in general irrelevant to the specified information needs. for example, when search results of “jaguar” are clustered, all formed clusters are about cars: jaguar cars, jaguar xf, jaguar uk, jaguar dealers, jaguar accessories, bmw, mercedes, and jaguar x type. ms live performed even worse, and we could hardly find relevant information from it. the google, yahoo, ms bing, aol.com, and baidu search engines respectively provide 49%, 50%, 62%, 54% and 59% precision levels based on 11 recall-precision curves when search results from all fifteen queries are summarized. whereas isf offers 65.2%, a better precision 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 p re ci si o n recall google yahoo bing aol baidu isf australasian journal of information systems zhu, nimmagadda, reiners & rudra 2020, vol 24, research article integrated search framework for the knowledge-based web ecosystem 20 compared with other search engines. largely, the framework validates the case studies with variables used in the search results. 8 future scope and limitations the experimental results so far demonstrate that text categorization in isf improves the precision of web search results. implementing the rest of the search framework and conducting a wide range of experiments are ongoing research. though improved precision is observed, the recall of categorized search results is lower than the search results of yahoo and other search engines. the reasons for the issue are, firstly one search result is assigned to only one category, even if the second or third ranked category has very close similarity instance with the top one. another reason is that categorization algorithm utilized in the isf is not optimal. combining text clustering and categorization is likely the next research direction, which can improve the recall of the categorized web-search results. since search engine results are dynamic, the compared results from google, bing and others are based on search requests when we conducted the experiments and can vary with time. in addition, these search engines can also allow personalized search; therefore, a further comparison of personalized results of search engines with isf can fetch enormous scope and opportunity in the ir research. 9 conclusions the information search with isf is aimed at providing effective information retrieval services leveraged by the kbwe. it is an innovative approach compared with the existing search methods. isf integrates not only traditional database search (mis) and web search engines, but also effectively bringing intranet, desktop and full-text database search tools into the framework. the personalization, ontological search results categorization, clustering and security scrutiny are added features. with the collaboration of empirical research, the experimental data demonstrate that text categorization based on the isf can improve the precision of web search results. the algorithms developed in the isf template have provided high information precision value in comparison with the latest search engines. in terms of accuracy, experience and innovation, the isf appears promising. the comparison made among search engines offers new improvements with satisfactory search results from the isf. references albro, e. n. (2006). google mini is a mighty search tool," pc world, https://www.pcworld.com/article/126139/article.html, june 21. alonso, o. & mizzaro, s. (2009). relevance criteria for e-commerce: a crowdsourcing-based experimental analysis, proceedings of the 32nd international acm sigir conference on research & development in information retrieval, 760-761, boston, ma, usa — july 19 23, 2009, https://doi.org/10.1145/1571941.1572115. arnold, stephen e. (2004). how google has changed enterprise search. in: searcher 12, s. 8-17. arasu, a. cho, j. garcia-molina, h. paepcke, a. & raghavan, s. (2001). searching the web," acm transactions on internet technology, 1 (1), 2001, 2-43. australasian journal of information systems zhu, nimmagadda, reiners & rudra 2020, vol 24, research article integrated search framework for the knowledge-based web ecosystem 21 baeza-yates, r., castillo, c., marin, m. & rodriguez, a. (2005). crawling a country better strategies than breadth-first for web page ordering. the 14th international conference on world wide web, may 10–14, 2005, chiba, japan. barrows, r. & traverso, j. (2006). search considered integral," acm queue, may 2006, 30-36. baskerville, r. l., kaul, m., & storey, v. c. (2015). genres of inquiry in design-science research: justification and evaluation of knowledge production. mis quarterly, 39 (3), 541-564. behnert, c. & lewandowski, d. (2017). "a framework for designing retrieval effectiveness studies of library information systems using human relevance assessments", journal of documentation, 73 (3), https://doi: 10.1108/jd-08-2016-0099 brin, s. & page, l. (1998). "the anatomy of a large-scale hypertextual web search engine" computer networks and isdn systems. 30 (1–7): 107–117. citeseerx 10.1.1.115.5930. https://doi.org/10.1016/s0169-7552(98)00110-x bunz, m (2009). "google extends personalised search to all users". the guardian. tue 8 dec, 2009. https://www.theguardian.com/media/pda/2009/dec/07/google-personalisedsearch. chau, m. & chen, h. (2008). “a machine learning approach to web page filtering using content and structure analysis,” decision support systems, 44 (2), 482-494. croft, w.b., metzler, d. & strohman, t. (2015). search engines – information retrieval in practice, pearson education, boston, usa. dean, j. 2009. challenges in building large-scale information retrieval systems, google, acm conference series, acm international conference on web search and data mining, wsdm 2009, https://pdfs.semanticscholar.org/fc32/72302461b74217662085a8a05a5e500dbf05.pdf dolog, p. & nejdl, w. (2003). challenges and benefits of the semantic web for user modelling, in de bra, p., davis, h., kay, j. and schraefel, m. (eds.) proc. of ah2003: workshop on adaptive hypermedia and adaptive web-based systems, budapest, hungary, eindhoven university of technology, pp. 99-111. available online at: <http://wwwis.win.tue.nl/ah2003/proceedings/um-1/>. elmasri, r., & navathe, s. (2016). fundamentals of database systems, hoboken, nj : pearson, usa, 2016. gartner, (2017). "insights from the 2017 gartner cio agenda report: seize the digital ecosystem opportunity," 2017. gregory, k. m., cousijn, h., groth, p. scharnhorst, a. & wyatt. s. (2019). understanding data search as a socio-technical practice, journal of information science. https://doi.org/10.1177/0165551519837182 haneef, i., munir, e. u., qaiser, g., hafiz gulfam, h. & ahmad, u. (2018). big data retrieval: taxonomy, techniques and feature analysis, ijcsns international journal of computer science and network security, 18 (11). australasian journal of information systems zhu, nimmagadda, reiners & rudra 2020, vol 24, research article integrated search framework for the knowledge-based web ecosystem 22 hernandez, n. mothe, j., chrisment, c., & egret, d. (2007). modeling context through domain ontologies, information retrieval journal (2007) 10:143–172, https://doi 10.1007/s10791-0069018-0 järvelin, k. (2007). an analysis of two approaches in information retrieval: from frameworks to study designs, journal of the american society for information science and technology, 58 (7), https://doi.org/10.1002/asi.20589 jung, j. j. (2007). ontological framework based on contextual mediation for collaborative information retrieval, information retrieval journal (2007) 10:85–109. https://doi 10.1007/s10791-006-9013 karanam, s., jorge-botana, g., olmos, r. & oostendorp, h. v. (2017). the role of domain knowledge in cognitive modelling of information search, information retrieval journal (2017), 20:456–479. https://doi 10.1007/s10791-017-9308-8 koopman, b., zuccon, g., bruza, p. sitbon, l. & lawley, m. (2016). information retrieval as semantic inference: a graph inference model applied to medical search, information retrieval journal (2016) 19:6–37. https://doi 10.1007/s10791-015-9268-9 kumar, s. s., mahapatra, d. p. & balabantaray, r. c. (2016). challenges for information retrieval in big data: product review context, international journal of computer applications (0975 – 8887), 136 (3), february 2016. liu, y., liu, t. y., gao, b., ma, z. & li, h. (2010). a framework to compute page importance based on user behaviours, information retrieval journal (2010) 13:22–45. https://doi 10.1007/s10791-009-9098-8 manning, c. d. raghavan, p. & schütze, h. (2009). introduction to information retrieval, cambridge: cambridge university press, new york, ny, usa, 2009. mccandless, m., hatcher, e. & gospodnetić, o. (2010). lucene in action, 2nd, greenwich: manning publications. mccreadie, r., macdonald, c. & ounis, l. (2012). mapreduce indexing strategies: studying scalability and efficiency, information processing & management, 48 (5), september 2012, 873-888. https://doi.org/10.1016/j.ipm.2010.12.003 meng, w., yu, c. & liu, k. l. (2000). building efficient and effective metasearch engines," acm computing surveys, 34 (1), 48-89. mizzaro, s. (1997). "relevance: the whole history," journal of the american society for information science 48, 810-832. moore, r., seedat, y., & chen, j. y. j. (2018). south africa: winning with digital platforms, accenture, 2018. pitkow, j. schütze, h. cass, t. cooley, r., turnbull, d. edmonds, a. adar, e. & breuel, t. (2002). personalized search: a contextual computing approach may prove a breakthrough in personalized search efficiency," communications of the acm, 45 (9), 5055. qin, t., liu, t. y. & li, h. (2010). a general approximation framework for direct optimization of information retrieval measures, information retrieval journal (2010) 13:375–397. https://doi 10.1007/s10791-009-9124-x australasian journal of information systems zhu, nimmagadda, reiners & rudra 2020, vol 24, research article integrated search framework for the knowledge-based web ecosystem 23 seyler, d., chandar, p. & davis, m. (2018). an information retrieval framework for contextual suggestion based on heterogeneous information network embeddings, sigir ’18, july 8–12, 2018, ann arbor, mi, usa c 2018 association for computing machinery. retrieved from https://doi.org/10.1145/3209978.3210103 simpson, m. s., demner-fushman, d., antani, s. k. & thoma, g. r. (2014). multimodal biomedical image indexing and retrieval using descriptive text and global feature mapping, information retrieval journal (2014) 17:229–264. https://doi 10.1007/s10791-0139235-2 soille, p., burger, a., marchi, d. d., kempeneers, p., d.rodriguez, d., syrris, v. & vasilev, v. (2018). a versatile data-intensive computing platform for information retrieval from big geospatial data, future generation computer systems, elsevier, volume 81, april 2018, pages 30-40, https://doi.org/10.1016/j.future.2017.11.007 soldaini, l., yates, a., yom-tov, e., frieder, o. & goharian, n. (2016). enhancing web search in the medical domain via query clarification, information retrieval journal 19 (1-2), 149173 (2016). https://doi 10.1007/s10791-015-9258-y tolosa, g., feuerstein, e., becchetti, l. & marchetti-spaccamela, a. (2017). performance improvements for search systems using an integrated cache of lists + intersections, information retrieval journal (2017) 20 (3):172–198. https://doi.org/10.1007/s10791-0179299-5 vaishnavi, v. k. & kuechler, w. (2007). design science research methods and patterns: innovating information and communication technology. auerbach publications, boston, ma. weill, p. & woerner, s. l. (2015). thriving in an increasingly digital ecosystem, mit sloan management review, 56 (4), 27-34. yang, h., sloan, m. and wang, j. 2015. dynamic information retrieval modeling, wsdm’15, february 2–6, 2015, shanghai, china. acm 978-1-4503-3317-7/15/02. http://dx.doi.org/10.1145/2684822.2697038. yue, y. (2011). new learning frameworks for information retrieval, (phd thesis, faculty of the graduate school of cornell university, ny, usa). retrieved from http://www.yisongyue.com/yue_thesis.pdf zhu, d., nimmagadda, s.l. & reiners, t. (2018). an integrated information retrieval framework for managing the digital web ecosystem, australasian conference of information systems (acis, 2018), uts, sydney, australia. http://www.acis2018.org/wp-content/uploads/2018/11/acis2018_paper_12.pdf zuccon, g., leelanupab, t., whiting, s., yilmaz, e., jose, j. m. & azzopardi, l. (2013). crowdsourcing interactions: using crowdsourcing for evaluating interactive information retrieval systems, information retrieval journal (2013) 16:267–305. https://doi 10.1007/s10791-012-9206-z copyright: © 2020 zhu, nimmagadda, reiners & rudra. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems zhu, nimmagadda, reiners & rudra 2020, vol 24, research article integrated search framework for the knowledge-based web ecosystem 24 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v24i0.2331 http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems batabyal & bhal 2023, vol 27, research article do hindrance stressors effect cyberloafing differently? 1 do challenge and hindrance stressors effect cyberloafing differently? attentiveness and anger as mediators and trait mindfulness as a moderator sauvik kumar batabyal international management institute new delhi, india batabyal.sauvik@gmail.com kanika tandon bhal indian institute of technology delhi, india abstract the objective of this study is to test the differential effect of challenge and hindrance stressors on cyberloafing, specifically using attentiveness and anger as mediators, respectively. it also investigates the effect of trait mindfulness as a moderator on the effect of hindrance stressors on cyberloafing through anger. we collected data from 304 full-time indian employees, working in broad-ranging industries and functions. the findings showed that challenge stressors have an indirect negative effect on cyberloafing through attentiveness. the results also indicated that anger partially mediates the positive relationship between hindrance stressors and cyberloafing. simple moderation analysis revealed that high trait mindfulness lowers the impact of hindrance stressors on anger. moreover, trait mindfulness moderated the indirect effect of hindrance stressors on cyberloafing through anger, in a way that the indirect effect became weaker in the case of high trait mindfulness. to regulate cyberloafing, we discussed the importance of optimizing challenge stressors to increase attentiveness and enhancing trait mindfulness to weaken the negative effects of hindrance stressors on anger and cyberloafing in the context of the theoretical and practical contribution of the study. keywords: cyberloafing, challenge and hindrance stressors, trait mindfulness, moderated mediation. 1 introduction cyberloafing, or the behavior of employees spending time in a broad range of online activities for personal or non-job reasons during the scheduled working hours (lim, 2002; lim & teo, 2022; tandon et al., 2022; wu et al., 2020) has become a major concern for organizations. it has been labelled as a “productivity killer” (koay et al., 2022), a deviant behavior (lim, 2002; she & li, 2022) and withdrawal and counterproductive behavior (askew et al., 2014) due to its farreaching negative consequences. when employees waste a significant amount of time out of their scheduled working duration in cyberloafing, the ill effects do not stay limited to causing quality and productivity related inefficiency only (alharthi, levy, wang, & hur, 2021; d'abate & eddy, 2007), but those also raise ethical (batabyal & bhal, 2020; block, 2001; friedman, 2000), moral (mazidi et al., 2020), legal (cheng, li, zhai, & smyth, 2014; lim & teo, 2005), financial (jandaghi et al., 2021; lim, koay, & chong, 2021; wu et al., 2022; zakrzewski, 2016) and security (jiang, tsohou, siponen, & li, 2020; koay et al., 2022) related concerns. reports suggest that 62% of the employees engage in cyberloafing regularly (ethics resource center, 2012; she & li, 2022). a recent qualitative study conducted in the indian context by batabyal and bhal (2020) showed that employees’ duration of cyberloafing range between 15 mins to 3 hours per day. this behavior can cost organizations around $85 billion yearly (zakrzewski, australasian journal of information systems batabyal & bhal 2023, vol 27, research article do hindrance stressors effect cyberloafing differently? 2 2016). it can also be a major reason for distraction (o’neill, hambley, & bercovich, 2014), which can potentially hamper the employees’ performance in the organizations (lim, koay, & chong, 2021). hence, scholars are increasingly trying to unearth various antecedents and constructs to understand possible associations with cyberloafing for the purpose of controlling this behavior. in this connection, researchers have investigated the influence of various constructs on cyberloafing like abusive supervision (agarwal & avey, 2020; lim, koay, & chong, 2021), communication style of supervisors (agarwal, 2019), organizational justice and injustice (lim, 2002; restubog et al., 2011), workplace ostracism and emotional exhaustion (koay, 2018, lim, koay, & chong, 2021), job burnout (aghaz & sheikh, 2016), coercive bureaucracy (soral, arayankalam, & pandey, 2020), anger toward organization (zhang, akhtar, zhang, & sun, 2020) and exposure to aggression at the workplace (andel et al., 2019). an overarching causal factor used to explain these, and other influence factors is under the rubric of stress. workplace stressors have been associated with cyberloafing (lavoie & pychyl 2001; ng, shao, & liu, 2016). the concept of stress comes to the picture when job demands surpasses the resources (lazarus & folkman, 1984). cyberloafing has been said to be used as a coping mechanism when an employee encounters stress at the workplace (page, 2015). however, past research has shown that not all stress is counterproductive. stress is shown to have both positive and negative outcomes depending on its level and its nature/ sources. challenge and hindrance stressors are shown to have different (and sometimes opposite) effect on work behaviors (rodell & judge, 2009; xu & wang, 2020). in this study, we use this stress framework for its impact on cyberloafing which has not been studied much (zhou et al., 2021). according to cavanaugh, boswell, roehling, and boudreau (2000) and rodell and judge (2009), certain job demands which are perceived as rewarding and beneficial because they contribute to a sense of achievement and growth are known as challenge stressors. examples of this kind of stressors include high volume of work or assignments at a certain point, significant responsibilities, time pressure and time involvement (cavanaugh et al., 2000; rodell & judge, 2009). in contrast, job demands which are perceived as obstacles, interferences and barriers to achieving one’s valued objectives, are known as hindrance stressors (cavanaugh et al., 2000; rodell & judge, 2009). examples include lack of job security, prevalence of organizational politics, stalled career prospects, existence of red tapes and unclear role expectations (cavanaugh et al., 2000; rodell & judge, 2009). in a recent study, zhou et al. (2021) found that challenge and hindrance stressors effect cyberloafing differently; the former with a negative direct relation and the later one as the opposite. their results also indicated that challenge stressors can have a positive indirect effect on cyberloafing through emotional exhaustion however resilience was not found to be significant mediator between challenge stressors and cyberloafing (zhou et al., 2021). these results show that further investigation is needed to study the process paths through which challenge-hindrance stressors may impact cyberloafing. to further this understanding, we explore the mediating role of emotions and cognition in this relationship. emotions are considered as reactions to events (weiss & cropanzano, 1996, p.18). affective events theory (weiss & cropanzano, 1996) suggests that events at the workplace can result to particular behaviors through emotional responses (rodell & judge, 2009). subsequently, these emotions can in turn lead to different behaviors (rodell & judge, 2009; xu & wang, 2020). in australasian journal of information systems batabyal & bhal 2023, vol 27, research article do hindrance stressors effect cyberloafing differently? 3 this study, our aim is to explore the mediating role of emotions between the relationship of challenge-hindrance stressors and cyberloafing. one possible classifications of emotions can be positive emotion and negative emotion (weiss & cropanzano, 1996, pp. 24-25). positive emotion is a result of need satisfaction whereas negative emotion can be caused by failure in meeting needs of an individual (xu & wang, 2020). in that context, challenge stressors can create a positive emotion and the cognitive resource of attentiveness whereas hindrance stressors can result in a negative emotion like anger (rodell & judge, 2009; xu & wang, 2020). meeting challenging job demands can induce an increased sense of attentiveness, which can facilitate building resources for the actor-agent when working toward growth and achievement, reducing the tendency to cyberloaf (zhou et al., 2021). in contrast, hindrance stressors can invoke anger and logically the actor-agent can utilize cyberloafing as an “emotion focused”, “escape-avoidance coping strategy” (henle & blanchard, 2008, p. 386). conservation of resource theory (cor) states that “people strive to retain, protect, and build resources and that what is threatening to them is the potential or actual loss of these valued resources” (hobfoll, 1989, p. 513). when an employee encounters hindrance stressors and deals with negative emotions like anger in the process, the employee can choose to protect personal resources or energies by escaping to counterproductive and withdrawal behaviors like cyberloafing (zhou et al., 2021). so, at the backdrop of challenge-hindrance stressors framework (cavanaugh et al., 2000; rodell & judge, 2009), affective events theory (aet) (weiss & cropanzano) and conservation of resource theory (hobfoll, 1989), the expectation of our study lies in the notion that challenge stressors will invoke more attentiveness and so it will have a negative indirect effect on cyberloafing where attentiveness will act as a mediator. on the contrary, hindrance stressors will generate stronger emotions like anger and so, it will positively and indirectly effect cyberloafing, through anger. further, we also explored a potential way to buffer the effect of hindrance stressors on anger, along with the possibility of reducing the impact the hindrance stressors on cyberloafing through anger. research shows that people who are mindful, exhibit lesser negative affect when they encounter various stressors (arch & craske, 2010; good et al., 2016). mindful people tend to observe events with greater sense of objectivity by curbing automaticity and keeping a distance psychologically (farb et al., 2007; good et al., 2016; hülsheger et al., 2014). mindfulness facilitates individuals with improved coping ability during chronic and stressed moments at the workplace by “decoupling habitual stimulus-response associations”, resulting to reinterpretation of the situations, faster recovery and developing resilience (good et al., 2016, p. 131). previous research has shown that mindfulness showed negative association with anger (borders, earleywine, & jajodia, 2010) and hostility (krishnakumar & robinson, 2015). hence, we propose trait mindfulness as a moderator on the effect of hindrance stressors on cyberloafing through anger. we particularly argue here that the effect of hindrance stressors on cyberloafing through anger will be weaker when the trait mindfulness of the employee is high versus low. employees with relatively high score on trait mindfulness will tend to cyberloaf less when triggered by hindrance stressors and subsequent emergence of anger. this is consistent with the prior literature where mindfulness showed negative association with anger (borders, earleywine, & jajodia, 2010) and in the context of cyberloafing, mindfulness has been found useful to reduce mind-wandering during the usage of smartphones for nonjob purposes during working hours (chen et al., 2022). so, there are two key contributions of this research. first, based on cavanaugh et al.’s (2000) framework of challenge-hindrance stressors, this research investigates the mediating effect of australasian journal of information systems batabyal & bhal 2023, vol 27, research article do hindrance stressors effect cyberloafing differently? 4 attentiveness and anger, while examining the possible opposing relationships between challenge-hindrance stressors and cyberloafing. second, we examine the moderating effect of trait mindfulness on the impact of hindrance stressors on cyberloafing through anger. here, we particularly explore the change of strength of the relationship between hindrance stressors and anger and the change in the intensity of the indirect effect of anger between hindrance stressors and cyberloafing, when moderated by trait mindfulness of employees. 2 literature review and hypotheses development 2.1 cyberloafing cyberloafing or the personal use of the ‘organizational’ internet during working hours at the workplace (lim, 2002), or utilizing personal mobile-internet devices and online services during working hours for non-job purposes (batabyal & bhal, 2020) or spending time online during working hours for non-job reasons while operating from remote locations (o’neill, hambley, & bercovich, 2014; o’neill, hambley, & chatellier, 2014) is increasingly gaining attention among the information systems, management and organizational psychology scholars (soral, arayankalam, & pandey, 2020). the emergence of this field can be tracked back to late 20th century (polito, 1997) and early 21st century (lim, 2002) when the term ‘cyberloafing’ came into the picture. subsequently, other terms like “cyberslacking” (friedman, 2000; block, 2001), “personal internet use at work” (garrett & danzinger, 2008; vitak, crouse, & larose, 2011), “personal web usage in the workplace” (anandarajan & simmers, 2004; kim & byrne, 2011; mahatanankoon, anandarajan, & igbaria, 2004) also started getting used interchangeably. over the years, with the proliferation of ict enabled personal mobile devices like smartphones, tablets etc. and easy accessibility and affordability of high-speed personal internet, cyberloafing through mobile devices also became a reality in the organizations (andel et al., 2019; askew, 2012; jamaluddin, ahmad, alias, & simun, 2015; kwak et al., 2021; sheikh, atashgah, & adibzadegan, 2015; vitak, crouse, & larose, 2011). through the pervasiveness and impact of the internet, remote working facility came to existence, so did the concept and practice of remote-cyberloafing (o’neill, hambley, & bercovich, 2014; o’neill, hambley, & chatellier, 2014). this practice became more evident when the world encountered the issue of covid-19; more and more organizations opted the ‘work from home’ culture as their complete or hybrid operational model (zhong, chen, yan, & luo, 2022). so, cyberloafing has evolved to be an umbrella term (batabyal & bhal, 2023) that refers to the abuse of the internet resources provided by the organizations and/or wasting the productive working hours by spending time online for non-work reasons using personal resources at the workplace (batabyal & bhal, 2020; lim & teo, 2022; wu et al., 2020) and/or in the distributed work contexts (tandon et al., 2022, p. 65). 2.2 workplace stressors and cyberloafing given widespread cyberloafing behavior and its consequent detrimental effects on organization, research has focussed on identifying causal factors like abusive supervision (agarwal & avey, 2020; lim, koay, & chong, 2021), perceived injustice (lim, 2002; restubog et al., 2011), job burnout (aghaz & sheikh, 2016), workplace ostracism and emotional exhaustion (koay, 2018, lim, koay, & chong, 2021). most of these observable causes are explained through perceived stress of the employees. workplace stressors comes into the picture when job demands surpass individuals’ resources (lazarus & folkman, 1984) or the coping ability with the same is considered stretched (kahn & byosserie, 1992). scholars have australasian journal of information systems batabyal & bhal 2023, vol 27, research article do hindrance stressors effect cyberloafing differently? 5 explored and empirically validated various ‘workplace stressors’ as antecedents of cyberloafing at the workplace. lavoie and pychyl (2001) threw light on the possibility that internet procrastination can be a way to get relief from stress. in the context of workplace, henle and blanchard (2008) showed cyberloafing as an emotion-focussed coping mechanism when an employee experiences higher role conflict and ambiguity in the workplace. another study examining the personal web use of trainee teachers in england showed that personal web use at work also acts as a coping mechanism to manage workplace stressors or work intensification (page, 2015). ng, shao and liu (2016) utilised the literature of avoidance coping strategy, performance theory and theory of goal orientation and proposed that stress in a job context can lead to avoidance coping mechanism, which can subsequently lead to social media use at workplace for personal reasons. based on the conservation of resources theory and general strain theory, koay, soh, & chew (2017) did a study among ict employees in malaysia and found that job stress was positively and significantly associated with cyberloafing. andel et al. (2019) examined another workplace stressor like workplace aggression in the context of cyberloafing and found that workplace aggression shares positive association with cyberloafing and the latter acts as a coping mechanism that weakens the relationship between workplace aggressions and job satisfaction and intent to turnover. in contrast, garrett and danziger (2008) found that factors related to dissatisfaction at workplace such as stress did not share a significant association with surfing web for personal reasons at work. varghese and barber (2017) found that only role conflict was the significant positive predictor of cyberloafing, but role overload and role ambiguity were not. another study which showed contrasting evidence related to job stress and cyberloafing was by zoghbi-manrique-de-lara, and sharifatashgah (2021). they unearthed that job stress was not significantly associated with cyberloafing and also did not act as a significant individual mediator between perceived crowding and cyberloafing. güğerçin’s (2020) study showed mixed findings. while exploring the influence of multi-dimensional techno-stress (“technooverload, techno-complexity and techno-invasion”) on cyberloafing, only techno-invasion was found to be a predictor of minor-cyberloafing (güğerçin, 2020). prior literature has shown that there is inconsistency related to the influence of stressors on cyberloafing or other behavioral outcomes. however, in addition to the previous discussion, stress has also been seen to have certain positive effects in the work settings like performance at team level (pearsall, ellis, & stein, 2009), performance in various roles (wallace et al., 2009), work engagement (crawford, lepine, & rich, 2010) etc. lepine, podsakoff, & lepine (2005) observed that the inconsistency in the stress-performance relationship can be explained from the perspective of good-bad stress where performance depends on the magnitude of it, giving it a quantitative mechanism (p. 766). on the other hand, the difference in results can be answered through a qualitative outlook in which one type of stressors is perceived as a way of learning whereas the other one as unnecessary obstacle (lepine, podsakoff, & lepine, 2005). so, in this study, we explore the influence of different types of stressors on cyberloafing and the different process paths present in it from the perspective of challenge-hindrance stressors framework (cavanaugh et al., 2000). 2.3 challenge-hindrance stressors and cyberloafing literature suggests that not all forms of stressors operate in a similar way. cavanaugh et al. (2000) suggested that work stress may not result to negative outcomes all the time; some stress can result to favorable outcomes. in this context, cavanaugh et al. (2000) invoked the eustressaustralasian journal of information systems batabyal & bhal 2023, vol 27, research article do hindrance stressors effect cyberloafing differently? 6 distress theory (selye, 1982, as cited in cavanaugh et al., 2000) where eustress is said to lead to the feelings of challenge, fulfilment and achievement. transactional stress theory (lazarus & folkman, 1984) states that upon encountering stressful demands and deciding on the possible strategies of coping, individuals attempt to understand the meaningfulness and seriousness of the same, along with the positive and negative outcomes it may result to, followed by appraising the resource availability and coping ability (xu & wang, 2020). as an extension of the mentioned theory, cavanaugh et al.’s (2000) challenge–hindrance stressor framework postulates that challenging demands or challenge stressors lead to favorable consequences at work and distress producing demands or hindrance stressors to the opposite. extant literature has uncovered that challenge-hindrance stressors effect various work and affective outcomes differently. for example, breevaart and bakker (2018) showed that challenge stressors share positive relationship with work engagement, but hindrance stressors show the opposite result. this is aligned with the results of tadić, bakker, and oerlemans, (2014). challenge stressors have been found to have negative association with turnover (kang & jang, 2019) and positive association with transactive memory (pearsall, ellis, & stein, 2009), daily positive affect (tadić, bakker, & oerlemans, 2014), performance in the job (zhang, lepine, buckman, & wei, 2014), well-being (chen, wang, yuan, & xu, 2021), life satisfaction (flinchbaugh, luth, & li, 2015) whereas the influence of hindrance stressors have been the opposite. in the context of cyberloafing, zhou et al. (2021) found partial support for the hypothesis of challenge stressors having a negative relationship with cyberloafing, whereas full support for the hypothesis of hindrance stressors having a positive relationship with cyberloafing. it can be compared with the findings of henle and blanchard (2008). they showed that role conflict and role ambiguity, which can be closely understood as hindrance stressors, had a positive effect on cyberloafing; whereas role overload which can be perceived as challenge stressors, had a negative effect on cyberloafing (henle & blanchard, 2008; zhou et al., 2021). however, varghese and barber (2017) did not find significant influence of role overload and role ambiguity on cyberloafing. hence, testing the relationship between challenge-hindrance stressors and cyberloafing may lead to useful insights on cyberloafing. the relationship between challenge-hindrance stressors and cyberloafing can be understood in the light of conservation of resources theory (hobfoll, 1989). from the perspective of this theory, stress occurs when resource loss is either expected or actual in a process or there is a clear sense of deficit of gaining resources in a process (hobfoll, 1989, p. 516). here, resources stand for “personal characteristics, conditions, or energies” which are utilized to achieve certain objectives (hobfoll, 1989, p. 516). as per this theory, the primary motivations of individuals are to reduce the net loss of resources when encountered with stressors and acquire more resources when not thwarted by inhibitory set of stressors (hobfoll, 1989; zhou et al., 2021; rodell & judge, 2009). in this study, we argue and hypothesize that challenge stressors can be seen as opportunities for growth and advancement which could facilitate an individual to fuel one’s own reservoir of resources (rodell & judge, 2009). in that sense, an individual would be less likely to indulge in any counterproductive behavior like cyberloafing as the focus would be more on acquiring resources by engaging with challenge stressors (rodell & judge, 2009). however, when encountered with hindrance stressors which are perceived as interference and barriers to one’s growth, the motivation would be the reduction or prevention of one’s loss of personal resources and so, the individual would be more likely australasian journal of information systems batabyal & bhal 2023, vol 27, research article do hindrance stressors effect cyberloafing differently? 7 to cyberloaf and utilize it in the form of withdrawal behavior for the purpose of preservation of existing set of resources (zhou et al., 2021). accordingly, we propose the following hypotheses: h1a: challenge stressors will negatively impact cyberloafing. h1b: hindrance stressors will positively impact cyberloafing. 2.4 attentiveness and anger as mediators besides studying the impact of two types of stressors, we wanted to study the process paths through which these stressors lead to different behaviors. cognitive and affective reactions mediate a stimulus and the behavioral response to the stimulus. using the affective events theory (weiss & cropanzano, 1996), we look at the different emotional responses to the two types of stressors as the mediator of this relationship. the central tenet of affective events theory is related to the “structure, causes and consequences of affective experiences at work” (p. 11). the two main components which are at the core of affective events theory are mood and emotions (p.17). emotions are considered to be arising as a response to events (weiss & cropanzano, 1996). individuals tend to react emotionally when they encounter stressful events or situations at the workplace or during working hours, which subsequently lead to typical attitudinal and behavioral consequences (weiss & cropanzano, 1996). research suggests that challenge stressors which are perceived as beneficiary in nature as far as the consequences are concerned, invoke positive emotions like attentiveness (rodell & judge, 2009). in contrast, hindrance stressors which are perceived as negative for blocking the path to attain one’s objectives, invoke negative emotions like anger (rodell & judge, 2009). affective events theory states that specific emotions can determine the nature of the path to certain behaviors (weiss & cropanzano, 1996, p. 23). for example, by combining transactional stress theory (lazarus & folkman, 1984) and affective events theory (weiss & cropanzano, 1996), rodell and judge (2009) found a positive indirect effect of challenge stressors on citizenship behaviors where attentiveness acted as a mediator. consistent with the mentioned framework and comparable theory base, xu and wang (2020) uncovered that challenge stressors have a negative impact and hindrance stressors have a positive impact on unethical pro-organizational behavior, mediated by attentiveness. attentiveness is manifested in alertness, concentration and determination (rodell & judge, 2009). attending to challenge stressors and overcoming the same in the process of work can be rewarding, fulfilling and can evoke positive emotions (cavanaugh et al., 2000; rodell & judge, 2009; xu & wang, 2020). an individual, who happens to face challenge stressors will be inclined to feel more alert and attentive in the process (xu & wang, 2020) in order to complete the tasks in hand (rodell & judge, 2009). this will naturally facilitate the employee to prioritize the work in hand without being distracted in mind-wandering or time-wasting behaviors, like cyberloafing. moreover, taking cues from conservation of resources theory (hobfoll, 1989), positive emotion like attentiveness at the time of attending to challenge stressors can rejuvenate the resource pool of the individual, resulting to lesser tendency for any counterproductive behavior like cyberloafing (xu & wang, 2020). hence, we propose the following hypotheses: h2a: challenge stressors will positively impact attentiveness. h2b: attentiveness will negatively impact cyberloafing. australasian journal of information systems batabyal & bhal 2023, vol 27, research article do hindrance stressors effect cyberloafing differently? 8 h2c: there will be a negative indirect relationship between challenge stressors and cyberloafing through attentiveness. affective events theory (weiss & cropanzano, 1996) suggests that negative events can invoke negative emotion like anger. anger is experienced in the form of psychological distress when an individual faces adverse situation, unexpected or undesirable in the common process of normal functioning (khansa et al., 2017; nyer, 1997). hindrance stressors hinder the process of goal attainment and personal growth at the workplace (cavanaugh et al., 2000), and they can lead to a high-intensity negative emotion like anger (xu & wang, 2020). extant literature indicates that anger is associated with counterproductive behaviors (rodell & judge, 2009), including cyberloafing (zhang et al., 2020; khansa, kuem, siponen, & kim, 2017). additionally, from the perspective of conservation of resources theory (hobfoll, 1989), individuals experiencing anger due to a response of hindrance stressors, will attempt to restore emotional resources by immersing one’s self in the world of internet for personal purposes (zhou et al., 2021). moreover, in order to alleviate negative emotions like anger, individuals can be more prone to exhibit counter-productive behaviors as exercising “evening the score” (spector & fox, 2002; rodell & judge, 2009). hence, we propose the following hypotheses: h3a: hindrance stressors will positively impact anger. h3b: anger will positively impact cyberloafing. h3c: there will be a positive indirect relationship between hindrance stressors and cyberloafing through anger. 2.5 trait mindfulness as a moderator of hindrance stressors-cyberloafing relationship however, this causal relationship between hindrance stressors and cyberloafing can be mitigated and moderated by some factors, mindfulness being one of the important ones. mindfulness is defined as “paying attention in a particular way: on purpose, in the present moment, and nonjudgmentally” (kabat-zinn, 1994, p. 4), which refers to working in a way that is quite opposite to performing in the automatic fashion (zoghbi-manrique-de-lara, viera-armas, & garcía, 2019). the origin of the mindfulness concept can be tracked back to early western and buddhist literature (rhys davids, 1881, as cited in schuman-olivier et al., 2020; sun, 2014). it has been found to be beneficial for improving the stability, control and efficiency with respect to the attention and enhancing cognitive flexibility and capacity (good et al., 2016). mindfulness has been classified as a ‘trait’ and ‘state’, where the former reflects a person’s predisposition of living a mindful life in general (baer et al., 2006) and the latter refers to as experiences through mindfulness meditation (kiken et al., 2015; lau et al., 2006). extant literature suggests that mindfulness helps in regulating stress (creswell & lindsay, 2014), improving performance at the job (dane & brummel, 2014) and shares a positive association with prosocial behaviors (donald et al., 2019). trait mindfulness has been particularly found to be associated with greater ethical behavior and lesser deviant behaviors (reb, narayanan, & ho, 2015). a study done by krishnakumar and robinson (2015) showed that trait mindfulness had a significant relation with lesser instances of counterproductive behaviors, in which reduction in hostile feelings mediated the effect. in the context of cyberloafing, mindfulness has been found useful to reduce mind-wandering during the usage of smartphones for non-job purposes during working hours (chen et al., 2022). several studies have found that mindfulness is associated with lower levels of anger (borders, earleywine, & australasian journal of information systems batabyal & bhal 2023, vol 27, research article do hindrance stressors effect cyberloafing differently? 9 jajodia, 2010) and hostility (krishnakumar & robinson, 2015) too. in this paper, we propose that given its impact both on counterproductive behavior and anger, mindfulness would impact this relationship. specifically, we hypothesize that the hindrance stressors’ indirect effect on cyberloafing through anger, will be weaker when the trait mindfulness of the employee is high versus low. employees with relatively high score on mindfulness will be relatively less inclined to spend time on cyberloafing when triggered by hindrance stressors and subsequent emergence of anger. h4a: trait mindfulness will moderate the positive relationship between hindrance stressors and anger such that the relationship will be weaker when trait mindfulness is high. h4b: trait mindfulness will moderate the indirect effect of hindrance stressors on cyberloafing through anger such that the indirect effect will be weaker when trait mindfulness is high. the research model can be found in figure 1. figure 1. research model 3 methods 3.1 data collection and demography of participants for our research work, we collected data from working employees involved in india in broadranging industries and domains, which is aligned with past research (usman et al., 2021). we collected the data from employees who were working in different business sectors, which include it, engineering, telecom, e-commerce, banking, finance, consultancy, logistics, insurance, energy, oil and gas, fmcg, pharma and healthcare, hospitality etc. extant australasian journal of information systems batabyal & bhal 2023, vol 27, research article do hindrance stressors effect cyberloafing differently? 10 literature on cyberloafing suggests that in many studies, data had been collected from school or college students as well. in our work, we have only focused on cyberloafing among working employees in various business organizations. in order to reach and invite prospective candidates, we utilized different social and professional networks (including social media platforms) (zhong et al., 2022), along with industry and alumni networks. as per the recent update available on the ministry of electronics and information technology (government of india) portal, direct employment numbers in it services have reached 4.47 million (with 36 % women participation). this sector also generates huge ancillary employment in real estate, hospitality and transportation sectors (meity, 2022). latest reports suggest that it sector is leading the hiring growth in india, followed by other sectors like banking, insurance services, financial services, retail, healthcare, automation, hospitality, pharma etc. (bhattacharyya, 2022; ibef, 2021). ministry of labour and employment’s (govt of india) report indications that around 15 lakh employees work in commercial banking systems (2022). hence, our sample represents quite fairly the white-collar job population in india. we used purposive (akbulut et al., 2016) and snowball sampling method (charoensukmongkol, 2014; hu et al., 2021) to have representation from variety of business industries, along with ensuring the fair representation of age, gender, ethnic and regional diversity of our country’s workforce. as we were investigating the employee cyberloafing in india, only those participants were requested to participate who were working in business organizations as an employee in india at the time of data collection. first, we contacted the prospective candidates through various online means for voluntary participation in our research work. the individuals who gave their permission to be a part of the research voluntarily, we shared the online questionnaire with them. the participants still had the option of denying to participate from the study even before starting to submit their responses, along with the option of withdrawing from the study at any point of time during submitting their responses. the opening page of the questionnaire mentioned the topic in focus, along with the utility of the research and assurance of the anonymity and confidentiality of the participants. only after the participants gave their consent to the details mentioned, they could move to the second page of the questionnaire where the actual submission of the responses began. for the purpose of data collection, 622 prospective working participants were contacted for the voluntarily participation in the research work. we received 304 useful responses via online mode which were finally retained for the study. around 27 % of the sample were females. the minimum and maximum age of the participants were 21 years and 51 years, in which the mean value stood at approximately 28 years, with a standard deviation of 3.86. majority of the participants possessed post-graduate qualifications in varied fields. the details on the demographic details like age, gender, educational level, total number of years of work experience, sector in current employment, present designation in the organization, place of working (working from home, working from office and both) were recorded through the questionnaire. the summary of the demography of the participants can be found in table 1. in our study we have included participants from a wide range of industries. in terms of female workforce participation in white-collar jobs, a report (bhattacharyya, 2016) suggests that it stands arounds 23%. in our sample demography, around 27% participants are females, which is neither very high or low from the present situation prevalent in india and represents the gender diversity of the country’s workforce. australasian journal of information systems batabyal & bhal 2023, vol 27, research article do hindrance stressors effect cyberloafing differently? 11 measure value frequency percentage mean sd gender male 222 73.03 female 82 26.97 qualification graduate 50 16.45 post graduate 254 83.55 age (in years) 27.96 3.86 total work exp 4.68 3.78 place of work working from home 187 61.51 working from office 41 13.49 working from both home and office 76 25.00 table 1. demographic profile of the participants (n= 304) 3.2 measures 3.2.1 cyberloafing for cyberloafing, we utilized a 13-item scale adapted from lim and teo (2005) and sheikh, atashgah, and adibzadegan (2015). for emailing activities, we dropped the item ‘receive’ as we believe it overlaps with ‘check’ and ‘send.’ we also consulted with two academic experts regarding the items and after careful moderation, we incorporated the scale. the 5-point likert scale, ranging from strongly disagree (1) to strongly agree (5) was utilized for response collection. item 11 (“during working hours, i visit adult-oriented (sexually explicit) websites”) was removed due to poor loading. sample items of the scale include “during working hours, i visit news websites for personal reasons”, “during working hours, i use instant messaging apps (e.g. whatsapp) for personal reasons.”, “during working hours, i use social media platforms (e.g. facebook, instagram, etc.) for personal reasons.”. cronbach's alpha of the cyberloafing scale was 0.87. 3.2.2 challenge stressors and hindrance stressors we adopted cavanaugh et al.’s (2000) scale with regards to challenge and hindrance stressors, in which there are 11 items in total. challenge stressors consist of 6 items and hindrance stressors consist of 5 items. respondents are asked to assess the degree of stress each of these items cause them at their respective workplaces or working hours. challenge stressors’ sample items include “the number of projects and or assignments i have”, “the volume of work that must be accomplished in the allotted time”. items related to hindrance stressors include “the lack of job security i have”, “the degree to which my career seems stalled”. cronbach alpha value of challenge stressors was 0.91 and that of hindrance stressors was 0.85. 3.2.3 attentiveness and anger attentiveness and anger were measured using the scale incorporated by rodell and judge (2009). the scale consisted of number of words or adjectives which can describe different feelings and emotions. respondents were asked to share the degree of those emotions they felt during working hours or at the workplace in recent times. 5-point likert scale (1= very slightly or not at all to 5 = extremely) was utilised for capturing the data. attentiveness consisted of 3 items (attentive, alert and determined) and anger was measured using 2 items (anger, hostility). cronbach alpha values of attentiveness and anger were 0.811 and 0.79, respectively. australasian journal of information systems batabyal & bhal 2023, vol 27, research article do hindrance stressors effect cyberloafing differently? 12 3.2.4 trait mindfulness we used brown and ryan ‘s (2003) 15-item mindful attention awareness scale to assess trait mindfulness. responses were obtained on a 6-point likert scale (1 = almost always to 6 = almost never). “i could be experiencing some emotion and not be conscious of it until sometime later”, “i find it difficult to stay focused on what’s happening in the present”, “it seems i am ‘running on automatic’ without much awareness of what i’m doing” are few sample items. the value of cronbach’s alpha of trait mindfulness was 0.93. 3.2.5 control variable we considered gender (male =1 and female = 2) as a control variable (lim, koay, & chong, 2021) as past literature indicates that gender influences cyberloafing significantly (vitak, crouse, & larose, 2011; garrett & danzinger, 2008). this is needed for the purpose of preventing potential confounding effect (henle & blanchard, 2008; koay, 2018). 4 results we followed comprehensive measures to prevent common method bias (podsakoff, mackenzie, lee, & podsakoff, 2003) related concerns. at the time of data collection, the informed consent section clearly mentioned about keeping the responses confidential and anonymous, along with the declaration of utilizing the responses for academic purposes only. this was done to prevent any potential bias while collecting the data and encourage honest and true responses from the participants (bhimavarapu, mohanty, acharya, & gupta, 2021; soral, arayankalam, & pandey, 2020). subsequently, we checked the existence of any significant common method bias by three ways. at first, harman’s single-factor test was conducted (harman, 1960; singh, 2020). we loaded all the constructs in the factor analysis as single factor for examining its variance in the unrotated solution. the general factor consisting of all the variables was set to be loaded into one factor, which accounted for the 24% of variance (n= 304), being it much less than the recommended 50 % threshold (singh, 2020; podsakoff et al., 2003). additionally, to check the existence of any multicollinearity problems in the data, we assessed the variance inflation factor (vif) of all constructs (bhimavarapu et al., 2021; chatterjee, 2021). the results showed that all the values were less than 2 and within the acceptable range (lin & lu, 2015; xu, zheng, & yang, 2023). we also checked the correlation matrix (table 3) in order to assess the presence of any high inter-correlation (r>0.90) which might indicate potential common method bias issue (bhimavarapu et al., 2021; pavlou, liang, & xue, 2007). the result showed no such instance. hence, we could conclude that common method bias was not a significant concern in our study. construct items mean s.d. factor loadings cr ave ca cyberloafing cl1 3.32 1.22 0.61 0.86 0.34 0.87 cl2 3.81 0.98 0.45 cl3 3.24 1.14 0.59 cl4 2.46 1.25 0.70 cl5 3.17 1.18 0.61 cl6 2.77 1.22 0.63 cl7 3.02 1.21 0.48 cl8 2.51 1.30 0.56 australasian journal of information systems batabyal & bhal 2023, vol 27, research article do hindrance stressors effect cyberloafing differently? 13 cl9 2.44 1.16 0.48 cl10 1.57 0.96 0.45 cl12 2.57 1.24 0.68 cl13 2.49 1.27 0.71 challenge stressors chal1 2.93 0.98 0.76 0.90 0.60 0.91 chal2 2.91 1.09 0.81 chal3 3.02 1.02 0.86 chal4 3.02 1.08 0.84 chal5 2.99 1.02 0.73 chal6 3.03 1.05 0.64 hindrance stressors hind1 2.96 1.31 0.75 0.85 0.53 0.85 hind2 2.81 1.21 0.75 hind3 2.67 1.22 0.85 hind4 2.44 1.30 0.65 hind5 2.88 1.29 0.61 mindfulness mind1 3.73 1.17 0.48 0.93 0.49 0.93 mind2 4.38 1.31 0.60 mind3 4.02 1.33 0.73 mind4 3.71 1.38 0.62 mind5 3.85 1.44 0.57 mind6 3.80 1.61 0.50 mind7 4.08 1.32 0.85 mind8 4.14 1.37 0.87 mind9 3.99 1.29 0.72 mind10 4.21 1.31 0.81 mind11 3.67 1.34 0.63 mind12 4.56 1.34 0.74 mind13 3.54 1.45 0.68 mind14 4.20 1.21 0.86 mind15 4.56 1.44 0.65 attentiveness atten1 3.90 0.83 0.86 0.82 0.61 0.81 atten2 3.89 0.84 0.80 atten3 4.01 0.86 0.66 anger ang1 2.45 1.25 0.83 0.79 0.65 0.79 ang2 2.11 1.20 0.78 table 2. items, means, standard deviations and factor loadings of each item, average variance extracted (ave), composite reliability (cr), and cronbach’s alpha of the variables we conducted cfa at the beginning for assessing the construct validity. we evaluated the model fit by including all variables in a measurement model. the sample size being 304, the study fulfilled the parameters to sample size ratio 1:5 criteria, which was recommended by prior researchers (gupta & bhal, 2020; bentler & chou, 1987). the results computed through australasian journal of information systems batabyal & bhal 2023, vol 27, research article do hindrance stressors effect cyberloafing differently? 14 amos software showed satisfactory model fit indices (cmin/df = 1.737, cfi = 0.909, gfi = 0.820, tli = 0.902, rmsea = 0.049) compared to the single factor model (cmin/df = 4.437, cfi = 0.567, gfi = 0.550, tli = 0.543, rmsea = 0.107) (hu & bentler, 1999; huo et al., 2022; xie, pinto, & zhong, 2022). to test reliability, we computed the cronbach’s alpha (ca) and composite reliability (cr) values. the summary of these values can be found in table 2. minimum recommended values of ca and cr are 0.6 and 0.7, respectively (hair et al., 2017; verma & singh, 2022). in our study, ca values ranged from 0.79 to 0.93, and the range of cr values was from 0.79 to 0.93, showing satisfactory reliability. factor loadings ranged from 0.45 to 0.87, all above required cut-off 0.4 (aunger et al., 2010; kim & koh, 2018; marikyan, papagiannidis, rana, & ranjan, 2022) and significant. to assess the convergent validity of the variables, we checked the average variance extracted (ave). in our study, the minimum and maximum values of ave were 0.34 and 0.65, respectively. the details can be found in table 2. according to fornell and larcker (1981), if average variance extracted values are lesser than 0.5 but the composite reliability values are greater than 0.6, the convergent validity of a dimension can be considered sufficient (lam, 2012) as average variance extracted is a more conservative measure. for discriminant validity, we adhered to the fornell and larcker criterion, in which the square root of ave of each variable should exceed the values of inter-correlations. table 3 shows that discriminant validity stands satisfied, and each construct was unique and explicitly independent (verma & singh, 2022). mean sd 1 2 3 4 5 6 7 1 cyberloafing 2.78 0.753 0.59 2 challenge stressors 2.986 0.858 (-) 0.089 0.78 3 hindrance stressors 2.755 1.000 0.197** 0.267** 0.73 4 attentiveness 3.936 0.719 (-) 0.371** 0.131* (-) 0.215** 0.78 5 mindfulness 4.029 0.965 (-) 0.334** (-) 0.007 (-) 0.390** 0.548** 0.70 6 anger 2.280 1.110 0.199** 0.054 0.398** (-) 0.287** (-) 0.411** 0.81 7 gender 1.270 0.445 (-) 0.062 0.107 (-) 0.020 0.099 0.045 0.06 note. **. correlation is significant at the 0.01 level (2-tailed); *. correlation is significant at the 0.05 level (2-tailed). values in bold are the square root of ave scores of the variables. table 3. assessing discriminant validity through fornell-larcker criterion 4.1 statistical analysis initially, we checked the significance of inter-correlation between the constructs. subsequently, we analyzed simple mediation through process macro in spss (hayes, 2013). we particularly incorporated process model number 4 which also facilitates utilizing 5000 bootstrap estimates that helps in constructing 95% confidence interval related to the indirect effect during the testing of mediation. we tested h4a and h4b by means of moderated mediation analysis. to assess the moderating effect of trait mindfulness, we utilised model number 7 in process macro in spss (hayes, 2013). 4.2 testing of hypotheses table 3 shows the values related to the means, standard deviations and inter-correlations of the constructs undertaken in our present research work. it shows that challenge stressors have australasian journal of information systems batabyal & bhal 2023, vol 27, research article do hindrance stressors effect cyberloafing differently? 15 a significant positive association with attentiveness (r = 0.13, p < 0.05) and attentiveness has a significant negative correlation with cyberloafing (r = -0.37, p < 0.01) however challenge stressors do not share a significant association with cyberloafing (r = -0.09, p > 0.05). hence, h2a and h2b received initial support but h1a did not. it also shows that the hindrance stressors have a significant positive correlation with cyberloafing (r = 0.197, p < 0.01), and anger (r = 0.398, p < 0.01), supporting h1b, as well as h3a. moreover, the zero-order correlation matrix reveals that anger has a significant positive relation with cyberloafing (r = 0.199, p < 0.01), lending initial support to h3b. subsequently, we used process macro (model 4) to check the significance of the mediating effect of attentiveness between challenge stressors and cyberloafing. unlike barron and kenny’s (1986) methodology of checking mediation, process macro doesn’t require that the independent variable (challenge stressors in our study) and dependent variable (cyberloafing in our case) have to be associated for the purpose of simple mediation analysis (hayes, 2013). the process macro-output (table 4) indicated that challenge stressors had a significant positive effect on attentiveness (coeff = 0.1023, se = 0.0480, t = 2.1313, p < 0.05, 95 % confidence interval = 0.0078 to 0.1967) and attentiveness had a negative effect on cyberloafing (coeff = -0.3813, se = 0.0568, t = 6.7189, p < 0.01, 95 % confidence interval = -0.4930 to -0.2697). it also showed that challenge stressors did not have a significant effect on cyberloafing (coeff = -0.0342, se = 0.0476, t = 0.7178, p > 0.05, 95 % confidence interval = -0.1279 to 0.0595). hence, we found support to h2a and h2b but not h1a. the bootstrap analysis showed (table 5) a statistically significant indirect effect as zero is not a part of the process generated 95% bootstrap confidence interval (effect = -0.0390, se = 0.0209, 95 % confidence interval = -0.0842 to -0.0019). the results also uncovered that there was a significant negative indirect effect of challenge stressors on cyberloafing through attentiveness, supporting h2c. for testing h3c, we again used process macro (model 4) to check the significance of the mediating effect of anger between hindrance stressors and cyberloafing. the results showed (table 4) that hindrance stressors had a significant positive effect on anger (coeff = 0.4436, se = 0.0585, t = 7.5831, p < 0.01, 95 % confidence interval = 0.3285 to 0.5587) and anger had a significant positive effect on cyberloafing (coeff = 0.1010, se = 0.0415, t = 2.4347, p < 0.05, 95 % confidence interval = 0.0194 to 0.1826). the bootstrap analysis showed (table 5) a statistically significant indirect effect as zero is not a part of the process generated 95% bootstrap confidence interval (effect = 0.0448, se = 0.0186, 95 % confidence interval = 0.0103 to 0.0840). it revealed that there was a positive indirect association between hindrance stressors and cyberloafing through anger. moreover, hindrance stressors had a significant direct effect on cyberloafing (effect = 0.1026, se = 0.0459, t = 2.2334, p < 0.05, 95 % confidence interval = 0.0122 to 0.1930). hence, we found support to h3a and h3b and we can conclude that anger partially mediated the association between hindrance stressors and cyberloafing, supporting h3c partially. for testing h4a and h4b, we utilized process macro model number 7 to perform moderated mediation analysis. in the context of h4a, we found that the interaction term was significant (coeff = 0.1255, t = -2.4953, p < 0.05, 95% confidence intervals = -0.2245 to -0.0265). table 6 indicates that when trait mindfulness was at a low level (-1 sd), the effect of hindrance stressors on anger was 0.4433, and the 95% confidence interval was 0.2861 to 0.6005, which did not include zero and indicated a significant effect. when trait mindfulness was at the mean level, hindrance stressors’ effect on anger was 0.3222, and the 95% confidence interval was australasian journal of information systems batabyal & bhal 2023, vol 27, research article do hindrance stressors effect cyberloafing differently? 16 0.2037 to 0.4406, which did not include zero and indicated a significant effect. when trait mindfulness was at a high level (+1 sd), the effect of hindrance stressors on anger was 0.2010, and the 95% confidence interval was 0.0541 to 0.3480, which did not include zero and indicated a significant effect. so, the higher the level of trait mindfulness, the lower the effect of hindrance stressors on anger. hence, we found support to h4a (figure 2). chal → atten → cl β se t p llci ulci chal → atten 0.1023 0.0480 2.1313 0.03 0.0078 0.1967 atten → cl -0.3813 0.0568 -6.7189 0.00 -0.4930 -0.2697 chal → cl -0.0342 0.0476 -0.7178 0.47 -0.1279 0.0595 total effect of chal → cl -0.0732 0.0506 -1.4462 0.15 -0.1727 0.0264 hind → ang → cl β se t p llci ulci hind → ang 0.4436 0.0585 7.5831 0.00 0.3285 0.5587 ang → cl 0.1010 0.0415 2.4347 0.02 0.0194 0.1826 hind → cl 0.1026 0.0459 2.2334 0.03 0.0122 0.1930 total effect of hind → cl 0.1474 0.0424 3.4733 0.00 0.0639 0.2310 note. chal= challenge stressors, hind = hindrance stressors, atten = attentiveness, ang = anger, cl = cyberloafing table 4 mediation analysis effect bootse bootllci bootulci chal → atten → cl -0.0390 0.0209 -0.0842 -0.0019 hind → ang → cl 0.0448 0.0186 0.0103 0.0840 table 5. indirect effects analysis mindfulness effect se t p llci ulci -0.9652 0.4433 0.0799 5.5495 0.0000 0.2861 0.6005 0.0000 0.3222 0.0602 5.3526 0.0000 0.2037 0.4406 0.9652 0.2010 0.0747 2.6919 0.0075 0.0541 0.3480 table 6. conditional effects of hindrance stressors at values of trait mindfulness figure 2. trait mindfulness as a moderator on the association between hindrance stressors and anger table 7 shows that zero is not a part of the moderated-mediation index, which indicates the significance of the conditional indirect effect. table 8 shows that the indirect relationship of hindrance stressors on cyberloafing through anger is significant when trait mindfulness is 0.0000 0.5000 1.0000 1.5000 2.0000 2.5000 3.0000 3.5000 low mod high a n g e r hindrance stressors low mod high australasian journal of information systems batabyal & bhal 2023, vol 27, research article do hindrance stressors effect cyberloafing differently? 17 high, moderate or low. table 7 and 8 also show that mindfulness moderates the indirect effect of hindrance stressors on cyberloafing through anger in such a way that the indirect effect becomes weak in the case of high trait mindfulness, supporting h4b. moderator: trait mindfulness index bootse bootllci bootulci hindrance stressors>anger>cyberloafing -0.0127 0.0071 -0.0293 -0.0017 table 7. index of moderated mediation mediator control indirect effect bootse bootllci bootulci anger -0.9652 0.0448 0.0187 0.0099 0.0832 anger 0.0000 0.0325 0.0135 0.0074 0.0609 anger 0.9652 0.0203 0.0106 0.0026 0.0441 note. values for trait mindfulness (moderator) are the mean and plus/minus one standard deviation (sd) from mean. table 8. conditional indirect effect analysis 5 discussion the objective of this research work was to examine how challenge and hindrance stressors effect cyberloafing differently, along with attentiveness and anger as mediators, respectively. we also examined how trait mindfulness can be beneficial for reducing the effect of hindrance stressors on cyberloafing, as mediated through anger. for this purpose, we collected data from 304 working individuals from various industries and found support to most of the hypotheses. the results show that all stressors do not impact cyberloafing similarly, specifically when mediated through various emotions. 5.1 differential influence of challenge and hindrance stressors on cyberloafing we hypothesized that job stressors which are appraised as challenge, will significantly and negatively impact cyberloafing whereas the job stressors which are perceived as hindrance, will be positively and significantly related to cyberloafing. the results have shown support to the latter, but not the former. zhou et al.’s (2021) study found no significant correlation between challenge stressors and cyberloafing at the beginning, but observed a direct negative effect of challenge stressors on cyberloafing. hence, they labelled challenge stressors as double-edged swords (zhou et al., p. 7). though the nature of hindrance stressors can seem threatening and become cause of severe energy depletion, challenge stressors can be seen as manageable but not necessarily favorable all the time (van den broeck, de cuyper, de witte, & vansteenkiste, 2010, p. 742). challenge stressors of higher degree can offset the benefits drawn from challenge stressors of optimum nature; causing problems related to the employees’ well-being and performance (edwards et al., 2014; nixon et al., 2011). it can also have a positive indirect effect on cyberloafing when emotional exhaustion acts as a mediator (zhou et al., 2021). mazzola and disselhorst (2019) reckoned the curvilinear nature of the association between challenge stressors and work-related outcomes (p. 958), which can potentially explain the non-significant direct association between challenge stressors and cyberloafing in our present work. australasian journal of information systems batabyal & bhal 2023, vol 27, research article do hindrance stressors effect cyberloafing differently? 18 5.2 mediating role of attentiveness and anger theoretically, this study extended our understanding on the differential effect of challengehindrance stressors in cyberloafing literature. it particularly investigated the role of two emotions, attentiveness and anger as mediators. we hypothesized that challenge stressors would lead to increased attentiveness, and attentiveness will negatively impact cyberloafing. we particularly highlighted the mediating effect of attentiveness between challenge stressors and cyberloafing. zhou et al. (2021) did not find resilience to be a significant mediator between challenge stressors and cyberloafing. in our paper, we observed that challenge stressors lead to attentiveness, which in turn negatively related to cyberloafing. the results also indicated that there exists a negative indirect effect of challenge stressors on cyberloafing through attentiveness. in the context of hindrance stressors, we expected that hindrance stressors will lead to anger, and anger will positively impact cyberloafing. we also specifically looked at the role of anger as a mediator between hindrance stressors and cyberloafing. the results showed hindrance stressors have a significant positive effect on anger, and anger also has a significant positive effect on cyberloafing. it also showed that anger partially mediated the relationship between hindrance stressors and cyberloafing. hence, this paper further extends the conceptual understanding of challenge-hindrance stressors framework to the domain of cyberloafing and supports the theoretical underpinning that different types of stressors do not influence certain behaviors in a similar way (cavanaugh et al., 2000). moreover, the present research also uncovered the existence of attentiveness and anger as mediators in the contextual impact of challenge and hindrance stressors on cyberloafing. utilizing affective event theory (weiss & cropanzano, 1996) and conservation of resources theory (hobfoll, 1989), the study extended the theoretical applications by showing that challenge stressors can lead to a sense of attentiveness or alertness, which can reduce the tendency to indulge in cyberloafing. this also supports our argument that employees may get motivated to acquire resources while attending to challenge stressors and staying attentive in the process for the purpose of achieving valued goals and objectives. in contrast, employees can indulge in cyberloafing to prevent loss of resources or conserve existing resources when they encounter hindrance stressors and in result, experience anger (rodell & judge, 2009; xu & wang, 2020; zhou et al., 2021). 5.3 trait mindfulness, weakening the impact of hindrance stressors on anger and cyberloafing finally, we examined the moderating role of mindfulness. we proposed that mindfulness as a non-invasive mechanism has the potential to moderate the relationship between hindrance stressors and anger and also can moderate the mediating effect of anger between hindrance stressors and cyberloafing. the index of moderated mediation showed full support to our hypotheses in this case. past literature has explored various tenets of mindfulness in the context of hindrance stressors (reina & kudesia, 2020). for example, wei, zhu, & chen (2020) uncovered the role of it mindfulness as a moderator between hindrance stressors and the usage of enterprise systems innovatively. in the healthcare sector, liu, zhao, & lu (2021) found that mindfulness of leaders reduces the nurses’ hindrance stress. in our research, we have looked into the moderating effect of trait mindfulness in two ways; first, on the relationship between hindrance stressors and anger and second, on the indirect effect of australasian journal of information systems batabyal & bhal 2023, vol 27, research article do hindrance stressors effect cyberloafing differently? 19 hindrance stressors on cyberloafing through anger. in both cases, we found that the strength of the relationships was stronger when trait mindfulness was low versus high. 5.4 practical implication in the context of organizations, this study has multiple key takeaways. first, in the domain of cyberloafing, organizations must acknowledge that all stressors may not lead to cyberloafing at the first place. understanding and differentiating the same will be the first step to leverage and control the effects of challenge and hindrance stressors. second, organizations must increasingly work on the gradual removal of hindrance stressors from the work context (xu & wang, 2020; zhou et al., 2021). this is turn can reduce the prevalent anger among the employees in organizations, which may also reduce the employees’ propensity to cyberloaf during working hours. third, organizations need to be careful while handling challenge stressors as well. companies must ensure that the challenge stressors remain within the tolerable limit in order to extract the benefits drawn from it, otherwise it can potentially harm the performance of the employees and organization (webster, beehr, & love, 2011). when the degree or severity of challenge stressors are optimized, employees will remain more engaged and attentive, resulting to lesser tendency to cyberloaf. fourth, corporate bodies must also take a closer look on the benefit of various mindfulness related dimensions. in our research, we explored the effect of employees’ trait mindfulness and how it can possibly reduce influence the inclination to cyberloaf, when impacted by hindrance stressors and anger. the results in our paper also gave support to our hypotheses. past research has shown that trait mindfulness can strengthen the relationship in terms of how internet usage policy in the organization is perceived and the prevalence of automatic behavior in the domain of cyberloafing (luo et al., 2022). organizations must consider including various mindfulness related interventions to improve the state mindfulness of the individuals who are operating in the workplace, which can simultaneously improve their trait mindfulness as well (kiken et al., 2015). employees can also proactively include various mindfulness-meditative practices to increase not only their trait mindfulness but also to improve their overall well-being. 6 limitation and future research the present research work is not immune to few limitations. first, we collected data from 304 working individuals which makes it limited to a single-source data collection. moreover, for this paper, we collected data pertaining to all constructs at a single point of time which makes it vulnerable to common-method bias. although, we took all relevant measures during data collection (soral, arayankalam, & pandey, 2020) and all statistical calculations showed no issues related to common method bias, future research can collect multi-source, time lagged data, along with other mediating emotions and constructs. second, in this study, we have majorly relied on affective events theory and conservation of resources theory, along with rodell & judge’s (2009) challenge-hindrance framework, future studies can also look into other theoretical underpinnings as well. third, this paper has only investigated the following constructs; challenge stressors, hindrance stressors, attentiveness, anger, trait mindfulness and cyberloafing. future studies can look into other constructs as well in this context to regulate cyberloafing behavior. for example, three out of seven dimensions of individual work ethic namely centrality of work, hard work and not wasting time (meriac et al., 2013) can potentially act as moderators between the challenge-hindrance stressors and cyberloafing. moreover, future research can also study the influence of contextual variables like peer cyberloafing australasian journal of information systems batabyal & bhal 2023, vol 27, research article do hindrance stressors effect cyberloafing differently? 20 (khansa et al., 2017) or employees’ daily work duration using the internet on the relationship between challenge-hindrance stressors and cyberloafing. 7 conclusion this study revealed that challenge and hindrance stressors follow differential paths to cyberloafing through different emotions. where challenge stressors take the route to cyberloafing with a negative indirect effect through attentiveness, hindrance stressors indirectly and positively effect cyberloafing through anger. the prevalence of cyberloafing is evident in the organization and it has reflected extensively in the human resource management, organizational behavior, psychology, human-computer interaction literature. it is imperative that the companies incorporate the required steps to reduce the hindrance stressors, and keep the challenge stressors within reasonable limit in order to regulate cyberloafing among its employees. moreover, to regulate the impact of hindrance stressors on cyberloafing, organizations may explore improving the employees’ trait mindfulness by incorporating various mindfulness-based interventions on a regular basis. lastly, contrary to our expectation, as challenge stressors did not show a significant negative direct relationship with cyberloafing, future research may look into other relevant attitudinal, motivational and emotional variables in this context to examine this issue. references agarwal, u. a. (2019). impact of supervisors’ perceived communication style on subordinate’s psychological capital and cyberloafing. australasian journal of information systems, 23. https://doi.org/10.3127/ajis.v23i0.1759 agarwal, u. a., & avey, j. b. (2020). abusive supervisors and employees who cyberloaf: examining the roles of psychological capital and contract breach. internet research, 30(3), 789-809. https://doi.org/10.1108/intr-05-2019-0208 aghaz, a., & sheikh, a. (2016). cyberloafing and job burnout: an investigation in the knowledge-intensive sector. computers in human behavior, 62, 51-60. https://doi.org/10.1016/j.chb.2016.03.069 alharthi, s., levy, y., wang, l., & hur, i. (2021). employees’ mobile cyberslacking and their commitment to the organization. journal of computer information systems, 61(2), 141-153. https://doi.org/10.1080/08874417.2019.1571455 anandarajan, m., & simmers, c. a. (2004). constructive and dysfunctional personal web usage in the workplace: mapping employee attitudes. in personal web usage in the workplace: a guide to effective human resources management (pp. 1-27), igi global, hershey, pe, usa. andel, s. a., kessler, s. r., pindek, s., kleinman, g., & spector, p. e. (2019). is cyberloafing more complex than we originally thought? cyberloafing as a coping response to workplace aggression exposure. computers in human behavior, 101, 124-130. https://doi.org/10.1016/j.chb.2019.07.013 arch, j. j., & craske, m. g. (2010). laboratory stressors in clinically anxious and non-anxious individuals: the moderating role of mindfulness. behaviour research and therapy, 48(6), 495-505. https://doi.org/10.1016/j.brat.2010.02.005 australasian journal of information systems batabyal & bhal 2023, vol 27, research article do hindrance stressors effect cyberloafing differently? 21 askew, k. (2012). the relationship between cyberloafing and task performance and an examination of the theory of planned behavior as a model of cyberloafing. university of south florida, tampa graduate theses and dissertations. https://digitalcommons.usf.edu/etd/3957 askew, k., buckner, j. e., taing, m. u., ilie, a., bauer, j. a., & coovert, m. d. (2014). explaining cyberloafing: the role of the theory of planned behavior. computers in human behavior, 36, 510-519. https://doi.org/10.1016/j.chb.2014.04.006 aunger, r., schmidt, w. p., ranpura, a., coombes, y., maina, p. m., matiko, c. n., & curtis, v. (2010). three kinds of psychological determinants for hand-washing behaviour in kenya. social science & medicine, 70(3), 383-391. https://doi.org/10.1016/j.socscimed.2009.10.038 baer, r. a., smith, g. t., hopkins, j., krietemeyer, j., & toney, l. (2006). using self-report assessment methods to explore facets of mindfulness. assessment, 13(1), 27-45. https://doi.org/10.1177/1073191105283504 batabyal, s. k., & bhal, k. t. (2023). push-pull factors and means-end chain framework of cyberloafing: a soft laddering study using ladderux. journal of information, communication and ethics in society, 21(1), 143-160 https://doi.org/10.1108/jices-05-20220046 batabyal, s.k., & bhal, k.t. (2020). traditional cyberloafing, mobile cyberloafing and personal mobile-internet loafing in business organizations: exploring cognitive ethical logics. journal of information, communication and ethics in society, 18(4), 631-647. https://doi.org/10.1108/jices-07-2019-0081 bentler, p. m., & chou, c. p. (1987). practical issues in structural modeling. sociological methods & research, 16(1), 78-117. https://doi.org/10.1177/0049124187016001004 bhattacharya, r. (2016). number of women in workforce stagnant, reveals survey. retrieved from https://economictimes.indiatimes.com/jobs/number-of-women-in-workforce-stagnantreveals-survey/articleshow/54594570.cms?from=mdr bhattacharya, r. (2022). white-collar job openings stay above pre-covid levels in april. retrieved from https://economictimes.indiatimes.com/jobs/white-collar-job-openings-stay-abovepre-covid-levels-in-april/articleshow/91271041.cms bhimavarapu, r., mohanty, p. k., acharya, a., & gupta, m. (2021). interactive app-based services: recovery evaluations and the mediating role of satisfaction in the relationship between customer-brand engagement and electronic word of mouth. australasian journal of information systems, 25. https://doi.org/10.3127/ajis.v25i0.3199 blau, g., yang, y., & ward-cook, k. (2006). testing a measure of cyberloafing. journal of allied health, 35(1), 9-17. block, w. (2001). cyberslacking, business ethics and managerial economics. journal of business ethics, 33(3), 225-231. borders, a., earleywine, m., & jajodia, a. (2010). could mindfulness decrease anger, hostility, and aggression by decreasing rumination?. aggressive behavior: official journal of the international society for research on aggression, 36(1), 28-44. https://doi.org/10.1002/ab.20327 australasian journal of information systems batabyal & bhal 2023, vol 27, research article do hindrance stressors effect cyberloafing differently? 22 breevaart, k., & bakker, a. b. (2018). daily job demands and employee work engagement: the role of daily transformational leadership behavior. journal of occupational health psychology, 23(3), 338. https://doi.org/10.1037/ocp0000082 brown, k. w., & ryan, r. m. (2003). the benefits of being present: mindfulness and its role in psychological well-being. journal of personality and social psychology, 84(4), 822. https://doi.org/10.1037/0022-3514.84.4.822 cavanaugh, m. a., boswell, w. r., roehling, m. v., & boudreau, j. w. (2000). an empirical examination of self-reported work stress among us managers. journal of applied psychology, 85(1), 65. https://doi.org/10.1037/0021-9010.85.1.65 chatterjee, s. (2021). impact on addiction of online platforms on quality of life. australasian journal of information systems, 25. https://doi.org/10.3127/ajis.v25i0.2761 chen, h., wang, h., yuan, m., & xu, s. (2021). daily challenge/hindrance demands and cognitive wellbeing: a multilevel moderated mediation model. frontiers in psychology, 12, 616002. https://doi.org/10.3389/fpsyg.2021.616002 cheng, l., li, w., zhai, q. and smyth, r. (2014). understanding personal use of the internet at work: an integrated model of neutralization techniques and general deterrence theory. computers in human behavior, 38, 220-228. https://doi.org/10.1016/j.chb.2014.05.043 crawford, e. r., lepine, j. a., & rich, b. l. (2010). linking job demands and resources to employee engagement and burnout: a theoretical extension and meta-analytic test. journal of applied psychology, 95(5), 834. https://doi.org/10.1037/a0019364 creswell, j. d., & lindsay, e. k. (2014). how does mindfulness training affect health? a mindfulness stress buffering account. current directions in psychological science, 23(6), 401-407. https://doi.org/10.1177/0963721414547415 dane, e., & brummel, b. j. (2014). examining workplace mindfulness and its relations to job performance and turnover intention. human relations, 67(1), 105-128. https://doi.org/10.1177/0018726713487753 davids, t. w. r. (ed.). (1881). buddhist suttas (vol. 11). clarendon press, london, uk. donald, j. n., sahdra, b. k., van zanden, b., duineveld, j. j., atkins, p. w., marshall, s. l., & ciarrochi, j. (2019). does your mindfulness benefit others? a systematic review and meta-analysis of the link between mindfulness and prosocial behaviour. british journal of psychology, 110(1), 101-125. https://doi.org/10.1111/bjop.12338 edwards, b. d., franco-watkins, a. m., cullen, k. l., howell, j. w., & acuff, r. e. jr. (2014). unifying the challenge-hindrance and sociocognitive models of stress. international journal of stress management, 21(2), 162. https://doi.org/10.1037/a0034730 farb, n. a., segal, z. v., mayberg, h., bean, j., mckeon, d., fatima, z., & anderson, a. k. (2007). attending to the present: mindfulness meditation reveals distinct neural modes of self-reference. social cognitive and affective neuroscience, 2(4), 313-322. https://doi.org/10.1093/scan/nsm030 flinchbaugh, c., luth, m. t., & li, p. (2015). a challenge or a hindrance? understanding the effects of stressors and thriving on life satisfaction. international journal of stress management, 22(4), 323. https://doi.org/10.1037/a0039136 australasian journal of information systems batabyal & bhal 2023, vol 27, research article do hindrance stressors effect cyberloafing differently? 23 fornell, c., & larcker, d. f. (1981). evaluating structural equation models with unobservable variables and measurement error. journal of marketing research, 18(1), 39-50. https://doi.org/10.1177/002224378101800104 friedman, w.h. (2000). is the answer to internet addiction internet interdiction?. proceedings of the americas conference on information systems 2000, association for information system, california, 1562-1567. http://aisel.aisnet.org/amcis2000/226 garrett, r. k., & danziger, j. n. (2008). on cyberslacking: workplace status and personal internet use at work. cyberpsychology & behavior, 11(3), 287-292. https://doi.org/10.1089/cpb.2007.0146 good, d. j., lyddy, c. j., glomb, t. m., bono, j. e., brown, k. w., duffy, m. k., ... & lazar, s. w. (2016). contemplating mindfulness at work: an integrative review. journal of management, 42(1), 114-142. https://doi.org/10.1177/0149206315617003 güğerçin, u. (2020). does techno-stress justify cyberslacking? an empirical study based on the neutralisation theory. behaviour & information technology, 39(7), 824-836. https://doi.org/10.1080/0144929x.2019.1617350 gupta, s., & bhal, k. t. (2020). leadership styles, justice and whistle-blowing intention: testing a mediation model. european business review. https://doi.org/10.1108/ebr-03-2020-0068 hair, j. f., hult, g. t. m., ringle, c. m., sarstedt, m., & thiele, k. o. (2017). mirror, mirror on the wall: a comparative evaluation of composite-based structural equation modelling methods. journal of the academy of marketing science, 45(5), 616-632. https://doi.org/10.1007/s11747-017-0517-x harman, h. h. (1960). modern factor analysis. university of chicago press, chicago, il, usa. hayes, a. f. (2017). introduction to mediation, moderation, and conditional process analysis: a regression-based approach. guilford publications, new york, ny, usa, henle, c. a., & blanchard, a. l. (2008). the interaction of work stressors and organizational sanctions on cyberloafing. journal of managerial issues, 383-400. https://www.jstor.org/stable/40604617 hobfoll, s. e. (1989). conservation of resources: a new attempt at conceptualizing stress. american psychologist, 44(3), 513. https://doi.org/10.1037/0003-066x.44.3.513 hu, l. t., & bentler, p. m. (1999). cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. structural equation modeling: a multidisciplinary journal, 6(1), 1-55. https://doi.org/10.1080/10705519909540118 hülsheger, u. r., lang, j. w., depenbrock, f., fehrmann, c., zijlstra, f. r., & alberts, h. j. (2014). the power of presence: the role of mindfulness at work for daily levels and change trajectories of psychological detachment and sleep quality. journal of applied psychology, 99(6), 1113-1128. https://doi.org/10.1037/a0037702 huo, w., zheng, g., yan, j., sun, l., & han, l. (2022). interacting with medical artificial intelligence: integrating self-responsibility attribution, human–computer trust, and personality. computers in human behavior, 132, 107253. https://doi.org/10.1016/j.chb.2022.107253 australasian journal of information systems batabyal & bhal 2023, vol 27, research article do hindrance stressors effect cyberloafing differently? 24 ibef (2021). it & bpm. retrieved from https://www.ibef.org/download/it-and-bpm-march2021.pdf jamaluddin, h., ahmad, z., alias, m., & simun, m. (2015). personal internet use: the use of personal mobile devices at the workplace. procedia-social and behavioral sciences, 172, 495502. https://doi.org/10.1016/j.sbspro.2015.01.391 jiang, h., tsohou, a., siponen, m. and li, y. (2020). examining the side effects of organizational internet monitoring on employees. internet research, 30(6), 1613-1630. https://doi.org/10.1108/intr-08-2019-0360 kabat-zinn, j. (1994). wherever you go, there you are. hyperion, new york, ny, usa kahn, r. l., & byosiere, p. (1992). stress in organizations. in m. d. dunnette & l. m. hough (eds.), handbook of industrial and organizational psychology (pp. 571–650). consulting psychologists press, palo alto, ca, usa. kang, j., & jang, j. (2019). what do employees perceive as hindrance or challenge stressors in the hotel industry? the role that hope plays. journal of human resources in hospitality & tourism, 18(3), 299-322. https://doi.org/10.1080/15332845.2019.1599783 khansa, l., kuem, j., siponen, m., & kim, s. s. (2017). to cyberloaf or not to cyberloaf: the impact of the announcement of formal organizational controls. journal of management information systems, 34(1), 141-176. https://doi.org/10.1080/07421222.2017.1297173 kiken, l. g., garland, e. l., bluth, k., palsson, o. s., & gaylord, s. a. (2015). from a state to a trait: trajectories of state mindfulness in meditation during intervention predict changes in trait mindfulness. personality and individual differences, 81, 41-46. https://doi.org/10.1016/j.paid.2014.12.044 kim, e., & koh, e. (2018). avoidant attachment and smartphone addiction in college students: the mediating effects of anxiety and self-esteem. computers in human behavior, 84, 264271. https://doi.org/10.1016/j.chb.2018.02.037 kim, s. j., & byrne, s. (2011). conceptualizing personal web usage in work contexts: a preliminary framework. computers in human behavior, 27(6), 2271-2283. https://doi.org/10.1016/j.chb.2011.07.006 koay, k. y. (2018). workplace ostracism and cyberloafing: a moderated–mediation model. internet research, 28(4), 1122-1141. https://doi.org/10.1108/intr-07-2017-0268 koay, k. y., lim, v. k., soh, p. c. h., ong, d. l. t., ho, j. s. y., & lim, p. k. (2022). abusive supervision and cyberloafing: a moderated moderation model of moral disengagement and negative reciprocity beliefs. information & management, 59(2), 103600. https://doi.org/10.1016/j.im.2022.103600 koay, k. y., soh, p. c. h., & chew, k. w. (2017). do employees’ private demands lead to cyberloafing? the mediating role of job stress. management research review, 40(9), 10251038. https://doi.org/10.1108/mrr-11-2016-0252 krishnakumar, s., & robinson, m. d. (2015). maintaining an even keel: an affect-mediated model of mindfulness and hostile work behavior. emotion, 15(5), 579. australasian journal of information systems batabyal & bhal 2023, vol 27, research article do hindrance stressors effect cyberloafing differently? 25 kwak, d. h., lee, s., ma, x., lee, j., lara, k., & brandyberry, a. (2021). announcement of formal controls as phase-shifting perceptions: their determinants and moderating role in the context of mobile loafing. internet research, 31(5), 1874-1898. https://doi.org/10.1108/intr-10-2020-0581 lam, l. w. (2012). impact of competitiveness on salespeople's commitment and performance. journal of business research, 65(9), 1328-1334. https://doi.org/10.1016/j.jbusres.2011.10.026 lau, m. a., bishop, s. r., segal, z. v., buis, t., anderson, n. d., carlson, l., ... & devins, g. (2006). the toronto mindfulness scale: development and validation. journal of clinical psychology, 62(12), 1445-1467. https://doi.org/10.1002/jclp.20326 lavoie, j. a., & pychyl, t. a. (2001). cyberslacking and the procrastination superhighway: a web-based survey of online procrastination, attitudes, and emotion. social science computer review, 19(4), 431-444. https://doi.org/10.1177/089443930101900403 lazarus, r. s., & folkman, s. (1984). stress, appraisal, and coping. springer publishing company, cham, switzerland. lepine, j. a., podsakoff, n. p., & lepine, m. a. (2005). a meta-analytic test of the challenge stressor–hindrance stressor framework: an explanation for inconsistent relationships among stressors and performance. academy of management journal, 48(5), 764-775. https://doi.org/10.5465/amj.2005.18803921 lim, p.k., koay, k.y. and chong, w.y. (2021). the effects of abusive supervision, emotional exhaustion and organizational commitment on cyberloafing: a moderated-mediation examination. internet research, 31(2), 497-518. https://doi.org/10.1108/intr-03-2020-0165 lim, v. k. g. (2002). the it way of loafing on the job: cyberloafing, neutralizing and organizational justice. journal of organizational behavior, 23(5), 675–694. https://doi.org/10.1002/job.161 lim, v. k., & teo, t. s. (2005). prevalence, perceived seriousness, justification and regulation of cyberloafing in singapore: an exploratory study. information & management, 42(8), 1081-1093. https://doi.org/10.1016/j.im.2004.12.002 lim, v. k., & teo, t. s. (2022). cyberloafing: a review and research agenda. applied psychology. https://doi.org/10.1111/apps.12452 lin, k. y., & lu, h. p. (2015). predicting mobile social network acceptance based on mobile value and social influence. internet research, 25(1), 107-130. https://doi.org/10.1108/intr01-2014-0018 liu, b., zhao, h., & lu, q. (2021). effect of leader mindfulness on hindrance stress in nurses: the social mindfulness information processing path. journal of advanced nursing, 77(11), 4414-4426. https://doi.org/10.1111/jan.14929 luo, x., xu, f., zhang, j., xiao, s., & xue, b. (2022). effects of organizational controls on employees' cyber-loafing: the moderating effects of trait mindfulness. acm sigmis database: the database for advances in information systems, 53(1), 61-79. https://doi.org/10.1145/3514097.3514102 australasian journal of information systems batabyal & bhal 2023, vol 27, research article do hindrance stressors effect cyberloafing differently? 26 mahatanankoon, p., anandarajan, m., & igbaria, m. (2004). development of a measure of personal web usage in the workplace. cyberpsychology & behavior, 7(1), 93-104. https://doi.org/10.1089/109493104322820165 marikyan, d., papagiannidis, s., rana, o. f., & ranjan, r. (2022). blockchain adoption: a study of cognitive factors underpinning decision making. computers in human behavior, 131, 107207. https://doi.org/10.1016/j.chb.2022.107207 mazzola, j. j., & disselhorst, r. (2019). should we be “challenging” employees?: a critical review and meta-analysis of the challenge-hindrance model of stress. journal of organizational behavior, 40(8), 949-961. https://doi.org/10.1002/job.2412 meity (2022). employment generation. retrieved from https://www.meity.gov.in/content/employment meriac, j. p., woehr, d. j., gorman, c. a., & thomas, a. l. (2013). development and validation of a short form for the multidimensional work ethic profile. journal of vocational behavior, 82(3), 155-164. https://doi.org/10.1016/j.jvb.2013.01.007 ministry of labour & employment (2022). employment situation in new india indicators back to pre-covid levels but government keen on improving situation further. retrieved from https://static.pib.gov.in/writereaddata/specificdocs/documents/2022/feb/doc20222241 8201.pdf ng, j. c. y., shao, i. y. t., & liu, y. (2016). this is not what i wanted: the effect of avoidance coping strategy on non-work-related social media use at the workplace. employee relations, 38(4), 466-486. https://doi.org/10.1108/er-12-2015-0216 nixon, a. e., mazzola, j. j., bauer, j., krueger, j. r., & spector, p. e. (2011). can work make you sick? a meta-analysis of the relationships between job stressors and physical symptoms. work & stress, 25(1), 1-22. https://doi.org/10.1080/02678373.2011.569175 nusrat, a., he, y., luqman, a., waheed, a., & dhir, a. (2021). enterprise social media and cyber-slacking: a kahn’s model perspective. information & management, 58(1), 103405. https://doi.org/10.1016/j.im.2020.103405 nyer, p. u. (1997). a study of the relationships between cognitive appraisals and consumption emotions. journal of the academy of marketing science, 25(4), 296-304. https://doi.org/10.1177/0092070397254002 o’neill, t. a., hambley, l. a., & bercovich, a. (2014). prediction of cyberslacking when employees are working away from the office. computers in human behavior, 34, 291-298. https://doi.org/10.1016/j.chb.2014.02.015 o’neill, t. a., hambley, l. a., & chatellier, g. s. (2014b). cyberslacking, engagement, and personality in distributed work environments. computers in human behavior, 40, 152-160. https://doi.org/10.1016/j.chb.2014.08.005 page, d. (2015). teachers’ personal web use at work. behaviour & information technology, 34(5), 443-453. https://doi.org/10.1080/0144929x.2014.928744 pavlou, p. a., liang, h., & xue, y. (2007). understanding and mitigating uncertainty in online exchange relationships: a principal-agent perspective. mis quarterly, 31(1), 105-136. https://doi.org/10.2307/25148783 australasian journal of information systems batabyal & bhal 2023, vol 27, research article do hindrance stressors effect cyberloafing differently? 27 pearsall, m. j., ellis, a. p., & stein, j. h. (2009). coping with challenge and hindrance stressors in teams: behavioral, cognitive, and affective outcomes. organizational behavior and human decision processes, 109(1), 18-28. https://doi.org/10.1016/j.obhdp.2009.02.002 podsakoff, p. m., mackenzie, s. b., lee, j. y., & podsakoff, n. p. (2003). common method biases in behavioral research: a critical review of the literature and recommended remedies. journal of applied psychology, 88(5), 879-903. polito, a. (1997). cyberloafing can be curbed. workforce, 76, 18. reb, j., narayanan, j., & ho, z. w. (2015). mindfulness at work: antecedents and consequences of employee awareness and absent-mindedness. mindfulness, 6(1), 111-122. https://doi.org/10.1007/s12671-013-0236-4 reina, c. s., & kudesia, r. s. (2020). wherever you go, there you become: how mindfulness arises in everyday situations. organizational behavior and human decision processes, 159, 78-96. https://doi.org/10.1016/j.obhdp.2019.11.008 restubog, s. l. d., garcia, p. r. j. m., toledano, l. s., amarnani, r. k., tolentino, l. r., & tang, r. l. (2011). yielding to (cyber)-temptation: exploring the buffering role of self-control in the relationship between organizational justice and cyberloafing behavior in the workplace. journal of research in personality, 45(2), 247-251. https://doi.org/10.1016/j.jrp.2011.01.006 rodell, j. b., & judge, t. a. (2009). can “good” stressors spark “bad” behaviors? the mediating role of emotions in links of challenge and hindrance stressors with citizenship and counterproductive behaviors. journal of applied psychology, 94(6), 1438. https://doi.org/10.1037/a0016752 schuman-olivier, z., trombka, m., lovas, d. a., brewer, j. a., vago, d. r., gawande, r., dunne, j. p., lazar, s. w., loucks, e. b., & fulwiler, c. (2020). mindfulness and behavior change. harvard review of psychiatry, 28(6), 371–394. https://doi.org/10.1097/hrp.0000000000000277 selye, h. (1982). history and present status of the stress concept. in l. goldberger & s. breznitz (eds.), handbook of stress (pp. 7-17). free press, new york, usa. she, z., & li, q. (2022). when too little or too much hurts: evidence for a curvilinear relationship between cyberloafing and task performance in public organizations. journal of business ethics, 1-18. https://doi.org/10.1007/s10551-022-05038-9 sheikh, a., atashgah, m. s., & adibzadegan, m. (2015). the antecedents of cyberloafing: a case study in an iranian copper industry. computers in human behavior, 51, 172-179. https://doi.org/10.1016/j.chb.2015.04.042 singh, s. (2020). an integrated model combining ecm and utaut to explain users’ postadoption behaviour towards mobile payment systems. australasian journal of information systems, 24. https://doi.org/10.3127/ajis.v24i0.2695 soral, p., arayankalam, j., & pandey, j. (2020). the impact of ambivalent perception of bureaucratic structure on cyberloafing. australasian journal of information systems, 24. https://doi.org/10.3127/ajis.v24i0.2087 australasian journal of information systems batabyal & bhal 2023, vol 27, research article do hindrance stressors effect cyberloafing differently? 28 spector, p. e., & fox, s. (2002). an emotion-centered model of voluntary work behavior: some parallels between counterproductive work behavior and organizational citizenship behavior. human resource management review, 12(2), 269-292. https://doi.org/10.1016/s1053-4822(02)00049-9 sun, j. (2014). mindfulness in context: a historical discourse analysis. contemporary buddhism, 15(2), 394-415. tadić, m., bakker, a. b., & oerlemans, w. g. (2015). challenge versus hindrance job demands and well-being: a diary study on the moderating role of job resources. journal of occupational and organizational psychology, 88(4), 702-725. https://doi.org/10.1111/joop.12094 tandon, a., kaur, p., ruparel, n., islam, j. u., & dhir, a. (2022). cyberloafing and cyberslacking in the workplace: systematic literature review of past achievements and future promises. internet research, 32(1), 55-89. https://doi.org/10.1108/intr-06-20200332 ugrin, j. c., pearson, j. m., & odom, m. d. (2008). profiling cyber-slackers in the workplace: demographic, cultural, and workplace factors. journal of internet commerce, 6(3), 75-89. https://doi.org/10.1300/j179v06n03_04 usman, m., javed, u., shoukat, a., & bashir, n. a. (2021). does meaningful work reduce cyberloafing? important roles of affective commitment and leader-member exchange. behaviour & information technology, 40(2), 206-220. https://doi.org/10.1080/0144929x.2019.1683607 van den broeck, a., de cuyper, n., de witte, h., & vansteenkiste, m. (2010). not all job demands are equal: differentiating job hindrances and job challenges in the job demands–resources model. european journal of work and organizational psychology, 19(6), 735–759. https://doi.org/10.1080/13594320903223839 van doorn, o. n. (2011). cyberloafing: a multi-dimensional construct placed in a theoretical framework, ms thesis, eindhoven university of technology, eindhoven: netherlands. retrieved from https://pure.tue.nl/ws/portalfiles/portal/47016165 varghese, l., & barber, l. k. (2017). a preliminary study exploring moderating effects of role stressors on the relationship between big five personality traits and workplace cyberloafing. cyberpsychology: journal of psychosocial research on cyberspace, 11(4). https://doi.org/10.5817/cp2017-4-4 verma, s., & singh, v. (2022). impact of artificial intelligence-enabled job characteristics and perceived substitution crisis on innovative work behavior of employees from high-tech firms. computers in human behavior, 131, 107215. https://doi.org/10.1016/j.chb.2022.107215 vitak, j., crouse, j., & larose, r. (2011). personal internet use at work: understanding cyberslacking. computers in human behavior, 27(5), 1751-1759. https://doi.org/10.1016/j.chb.2011.03.002 wallace, j. c., edwards, b. d., arnold, t., frazier, m. l., & finch, d. m. (2009). work stressors, role-based performance, and the moderating influence of organizational support. journal of applied psychology, 94(1), 254-262. doi: 10.1037/a0013090 australasian journal of information systems batabyal & bhal 2023, vol 27, research article do hindrance stressors effect cyberloafing differently? 29 webster, j. r., beehr, t. a., & love, k. (2011). extending the challenge-hindrance model of occupational stress: the role of appraisal. journal of vocational behavior, 79(2), 505-516. https://doi.org/10.1016/j.jvb.2011.02.001 wei, s., zhu, f., & chen, x. (2020). do stressors stifle or facilitate employees' innovative use of enterprise systems: the moderating role of it mindfulness. information technology & people, 34(3), 955-977. https://doi.org/10.1108/itp-09-2019-0499 weiss, h. m., & cropanzano, r. (1996). affective events theory. research in organizational behavior, 18(1), 1-74. xie, l., pinto, j., & zhong, b. (2022). building community resilience on social media to help recover from the covid-19 pandemic. computers in human behavior, 134, 107294. https://doi.org/10.1016/j.chb.2022.107294 xu, c., zheng, x., & yang, f. (2023). examining the effects of negative emotions on review helpfulness: the moderating role of product price. computers in human behavior, 139, 107501. xu, l., & wang, j. (2020). influence of challenge–hindrance stressors on unethical proorganizational behavior: mediating role of emotions. sustainability, 12(18), 7576. https://doi.org/10.3390/su12187576 zakrzewski, c., (2016). the key to getting workers to stop wasting time online. retrieved from https://www.wsj.com/articles/the-key-to-getting-workers-to-stop-wasting-time-online1457921545. zhang, j., akhtar, m.n., zhang, y. and sun, s. (2020). are overqualified employees bad apples? a dual-pathway model of cyberloafing. internet research, 30(1), 289-313. https://doi.org/10.1108/intr-10-2018-0469 zhang, y., lepine, j. a., buckman, b. r., & wei, f. (2014). it's not fair… or is it? the role of justice and leadership in explaining work stressor–job performance relationships. academy of management journal, 57(3), 675-697. https://doi.org/10.5465/amj.2011.1110 zhong, j., chen, y., yan, j., & luo, j. (2022). the mixed blessing of cyberloafing on innovation performance during the covid-19 pandemic. computers in human behavior, 126, 106982. https://doi.org/10.1016/j.chb.2021.106982 zhou, b., li, y., hai, m., wang, w., & niu, b. (2021). challenge-hindrance stressors and cyberloafing: a perspective of resource conservation versus resource acquisition. current psychology, 1-10. https://doi.org/10.1007/s12144-021-01505-0 zoghbi-manrique-de-lara, p., & sharifiatashgah, m. (2021). the relationship between perceived crowding and cyberloafing in open offices at iranian it-based companies. cognition, technology & work, 23(2), 331-342. https://doi.org/10.1007/s10111020-00627-y zoghbi-manrique-de-lara, p., viera-armas, m., & garcía, g. d. b. (2019). does supervisors’ mindfulness keep employees from engaging in cyberloafing out of compassion at work?. personnel review, 49(2), 670-687. https://doi.org/10.1108/pr-12-2017-0384 australasian journal of information systems batabyal & bhal 2023, vol 27, research article do hindrance stressors effect cyberloafing differently? 30 copyright: © 2023 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v27i0.4105 http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 literature review and hypotheses development 2.1 cyberloafing 2.2 workplace stressors and cyberloafing 2.3 challenge-hindrance stressors and cyberloafing 2.4 attentiveness and anger as mediators 2.5 trait mindfulness as a moderator of hindrance stressors-cyberloafing relationship 3 methods 3.1 data collection and demography of participants 3.2 measures 3.2.1 cyberloafing 3.2.2 challenge stressors and hindrance stressors 3.2.3 attentiveness and anger 3.2.4 trait mindfulness 3.2.5 control variable 4 results 4.1 statistical analysis 4.2 testing of hypotheses 5 discussion 5.1 differential influence of challenge and hindrance stressors on cyberloafing 5.2 mediating role of attentiveness and anger 5.3 trait mindfulness, weakening the impact of hindrance stressors on anger and cyberloafing 5.4 practical implication 6 limitation and future research 7 conclusion paper title (paper title style) australasian journal of information systems saheb & hosseinpouli mamaghani 2021, vol 25, research article exploring the digital business ecosystem 1 exploring the digital business ecosystem of the internet of things in emerging economies with a focus on the role of pseudo-private companies tahereh saheb tarbiat modares university, tehran, iran t.saheb@modares.ac.ir faranak hosseinpouli mamaghani tarbiat modares university, tehran, iran abstract the internet of things (iot) is one of the emerging drivers of the economy that transforms the business landscape and forms of collaboration and value creation. in this study, we performed a delphi-based study to explore the digital business ecosystem (dbe) of iot in emerging economies, focusing on power dynamics among actors, the processes of value co-creation, topologies, and critical success factors affecting the health of the ecosystem. we conducted two panels of a three-round delphi study from 2017 through 2019 among iranian iot experts. this study identifies two types of the digital business ecosystems surrounding iot in iran. one type comprises only pseudo-private companies that have strong interactions with government organizations and state-owned banks. the other type of digital business ecosystem surrounding iot technology comprises start-ups and private small and medium companies that have no interaction with pseudo-private companies. the interaction of start-ups and small medium enterprises (smes) is mainly with universities. the study shows that pseudo-private companies play a dominant role in the business to business (b2b) and business to government (b2g) markets while smes and start-ups have a dominant role in the b2c market and promote creativity and innovation in the business to customer (b2c) market. this study extends the literature on digital business ecosystems with a focus on the health of the dbes of iot technologies and the potential for future innovation in the digital economy. keywords: digital business ecosystem, internet of things, emerging economies, co-creation, health of business ecosystem, pseudo-private companies, niche creation 1 introduction the internet of things (iot), as a flourishing digital technology, is one of the emerging drivers of the economy in many countries, transforming the business landscape and forms of collaboration and value creation (selander et al., 2013). iot refers to the network of small and intelligent devices that are widely distributed across geographic locations, are connected through the internet, and have extended human sensing capabilities (saheb & izadi, 2019). the promising nature of iot products and services has ignited a series of collaborative organizational networks, such as digital business ecosystems (dbe). iot technologies not only connect a series of interacting objects together, but also include stakeholders who connect with iot (rong et al., 2015). in this study, we investigated the digital business ecosystem of the iot. we define the business ecosystem as a loosely connected business community composed of various actors with a shared common goal who co-evolve with each other (moore, 1993). the "digital business ecosystem," as an extension of moore's theory of business ecosystems, is also australasian journal of information systems saheb & hosseinpouli mamaghani 2021, vol 25, research article exploring the digital business ecosystem 2 defined as a collaborative network in which digital technology plays a major role in developing economic communities of loosely-coupled organizations and individuals developing cross-industry collaborations and modes of value co-creation (moore, 1993, 1996). as mazhelis et al. (2012) define it, the digital business ecosystem surrounding the iot technology, is a special type of digital business ecosystem composed of “the community of interacting companies and individuals along with their socio-economic environment, where the companies are competing and cooperating by utilizing a common set of core assets related to the interconnection of the physical world of things with the virtual world of internet” (mazhelis et al., 2012, p.5). the dbe of iot is a sociotechnical environment composed of individuals, organizations, and digital technologies using collaborative and competitive interactions to form new modes of value co-creation by different entities within the ecosystem. as a collaborative network, the dbe of iot is composed of various actors and entities in which traditional modes of value creation, collaboration and progress are disrupted to achieve a common goal (ikävalko et al., 2018; kar et al., 2018). in this study, we did not limit our research to a specific industry and business ecosystems around that industry since collaborative interactions form the collection of actors of an ecosystem. unlike traditional business settings, in an iot-enabled digital business ecosystem, the process of value co-creation is facilitated through collaborative interactions between various actors and the business community (leminen et al., 2015, 2012; rong et al., 2015; westerlund et al., 2014). in this research, we surveyed all companies that are registered as iot companies regardless of the type of their industry. we defined an iot company as a company that is involved in all five categories of hardware, middleware, software, presentation, and protocols and the use of these technologies to deliver products and services to consumers (saheb & izadi, 2019). inside the business ecosystem of iot, many entrepreneurs have been developing new iot products and services and establishing start-ups and companies around their ideas (chuang, 2018); simultaneously, many large companies have been upgrading their legacy systems with iot products and services (jónasdóttir et al., 2018) and spending billions of dollars on the development of new iot products. at the same time, governments provide policies (lee, 2019) and financial and non-financial support of iot ecosystems (herweijer et al., 2018; lee, 2019; shin, 2019). the digital business ecosystem theory has become pervasive in it strategic management literature (q. sun et al., 2018); however, there is scant scholarly research regarding the health of a digital business ecosystem. previous studies have mainly explored digital business ecosystems from the following perspectives: dbe network, dbe value co-creation, dbe collaboration, dbe knowledge management, and dbe strategy-making (senyo et al., 2019). little attention has been devoted to the health of iot enabled digital business ecosystems in emerging markets concerning the four aspects of robustness, creativity, interoperability, and productivity. this paper proposes a theoretical perspective of the various components enriching or threatening the health of an iot-enabled digital business ecosystem. the empirical case study for this paper is the dbe of the iot in iran. this research builds on existing knowledge in the fields of it strategic management literature by extending the literature on the digital business ecosystem. the central issue addressed in this study is the dbe of the iot ecosystem in iran, and our results are based on a two-panel delphi study and semi-structured interviews with iot experts. australasian journal of information systems saheb & hosseinpouli mamaghani 2021, vol 25, research article exploring the digital business ecosystem 3 the evaluation of key critical success factors within a digital business ecosystem through a qualitative study enables us to better explain the existence of varied actors, their complex interactions, and value creation processes. the current study fills the research gap on dbe health arguing that the critical success factor of a digital business ecosystem can be determined through measuring productivity, robustness, creativity, and interoperability, which provide new insights into the survival or bankruptcy of digital companies (iansiti & levien, 2004a). this study seeks to contribute to our current knowledge of it strategic management in three ways, depicted in figure 1: first, it extends our knowledge of iot-enabled digital business ecosystems. the application of digital business ecosystem theory can further the theoretical reasoning. this theory concerns the complex interactions of actors that have particular roles, heterogeneous levels of control, and distinct degrees of contribution with interaction networks of value creation and innovation; these interactions occur in emerging economies inside business ecosystems centered on digital technologies. prior research has identified different value structures and topologies in the us and europe surrounding information communication technology (ict) (corallo et al., 2017; iansiti & levien, 2002). furthermore, previous studies demonstrate how iot business ecosystems develop in europe and the us (e.g., leminen et al., 2015; westerlund et al., 2014). however, there is a paucity of scholarly attention on the dbe of iot in emerging economies, and this study aspires to fill this research gap. previous studies have explored the digital business ecosystem surrounding iot technology in developed countries (e.g., leminen et al., 2015, 2012; rong et al., 2015; westerlund et al., 2014). they have also incorporated the theory of business ecosystems, explaining the actors, roles, and value-creation processes without examining the critical success factor of an iot business ecosystem. the present work aims to explore this marginalized area of the digital business ecosystem theory by conducting an empirical investigation of the iot-enabled digital business ecosystem in an emerging economy with its unique socio-political and economic contexts. this study will unveil the differences and similarities between emerging economies and developed countries in terms of value cocreation processes, actors, roles, interactions, and power dynamics in the dbe of iot. according to the gsma, iot connections in the middle east region are expected to reach 1.1bn by 2025 (the mobile economy: middle east and north africa 2018, 2018). this report suggests that iot revenue in the mena region will increase at an average annual rate of 19% by 2025 to reach $55 billion. the iranian government has been pushing for more iot infrastructure in the future since 2017 (financial tribune, 2017). the second contribution of this study to the current knowledge of the digital business ecosystem is an analysis of four major critical success factors of a business ecosystem, i.e. robustness, productivity, interoperability, and creativity. disruptive technologies are accompanied by uncertainties surrounding digital ecosystems (rong et al., 2013) requiring the nurturing of ecosystems (moore, 1996) and healthy networks of cooperative actors (iansiti & levien, 2004b). therefore, it is worth exploring how critical success factors explain the overall health of digital businesses and the behavior of enterprises and other actors. previous studies recommend that the survival of a single enterprise interlinks with the survival of an ecosystem (iansiti & levien, 2004b). businesses are inclined toward developing networks of value-cocreation in which various actors, such as customers, partners, and producers, are integrated into mutually co-create values (hein et al., 2019). to the best of the authors' knowledge, this is the first study exploring the health of an iot enabled digital business ecosystem in an emerging economy. australasian journal of information systems saheb & hosseinpouli mamaghani 2021, vol 25, research article exploring the digital business ecosystem 4 the third contribution of this study to the current knowledge on the digital business ecosystem is that it identifies a series of factors impacting the health of iot business ecosystems in emerging economies from the perspective of experts. the delphi study will enable the prediction of future scenarios of factors influencing the health of digital business ecosystems in emerging economies. figure 1. theoretical contributions of the study the structure of the paper is as follows: section 2 outlines the theoretical background of the study, section 3 describes the research methodology used to unveil the findings of the study, reported in section 4. section 5 then concludes with a discussion of the study's contributions and limitations, and a consideration of possible future research topics. 2 theoretical foundation 2.1 digital business ecosystems a digital business ecosystem is an extension of the business ecosystem theory (moore, 1993). the dbe theory argues that digital technologies are one of the elements of a business ecosystem. senyo et al. (2019) define a dbe as a sociotechnical environment composed of three elements of individuals, organizations, and digital technologies that develop collaborative and competitive relationships to co-create value through shared digital platforms. these elements transform into a single collection to support the survival of the technology. they achieve this goal by strengthening network effects, and they value co-creation strategies and harmonizing varying sources of intellectual capacity to improve the technology (selander et al., 2013). previous studies have analyzed varying aspects of a digital business ecosystem including dbe relationships and alliances, network analysis, value co-creation, dbe governance, legal issues, trust, risk and security, knowledge development, dissemination and management, and dbe strategies, processes, and management (senyo et al., 2019). however, examining the health of extending the knowledge on iot enabled digital business ecosystems in emerging economies analyzing four major critical success factors of a business ecosystem, i.e. robustness, productivity, interoperability, and creativity. introducing a series of factors impacting the health of the digital business ecosystem surrounding the iot technology in emerging economies australasian journal of information systems saheb & hosseinpouli mamaghani 2021, vol 25, research article exploring the digital business ecosystem 5 digital business ecosystems remains unexplored. the existing dbe theory argues that business participants in a dbe collaborate with each other to promote innovation and to facilitate the process of value co-creation (l. sun et al., 2016). prior research has mainly focused on the digital business ecosystem surrounding iot technology models and the role of actors in europe and the us (e.g. leminen et al., 2015; rong et al., 2015). regarding the actors, their roles, and their interactions within the ecosystem, previous studies have demonstrated how different value structures and topologies are shaped in the us and europe around ict. in the us, the topology follows the keystone model and has a hub-centered star structure. however, in europe, it follows a flat model and has a flat mesh-like structure. in the us, the ecosystem is dominated by large hub firms, mainly private companies that hierarchically interact with numerous small firms. alternatively, in europe, the ecosystem consists of small and medium firms interacting in a non-hierarchical manner, without having a keystone or a dominator player (corallo et al., 2017; iansiti & levien, 2004b). prior research has emphasized that business ecosystem strategies require a keystone to promote creativity and the health of niche players by increasing collective productivity, robustness, and diversity (aksenova et al., 2018). research has also investigated how and why tensions among competing values inside digital business ecosystems influence ecosystem relationships (selander et al., 2010); as previous studies confirm, being a member of a digital business ecosystem generates new risks and uncertainties for firms as they delegate some control to the network (adner, 2006; iansiti & levien, 2004a). 2.2 business ecosystem health the theory of a business ecosystem has been extended by iansiti & levien (2002, 2004a) who define factors determining the health of a business ecosystem. they identify three determinants for productivity, five determinants for robustness, and two determinants for niche creation (iansiti & levien, 2002, 2004a). the determinants of productivity are total factor productivity, productivity improvements, and the delivery of innovations. the determinants of robustness are survival rates, the persistence of structure, predictability, limited obsolescence, and continuity. determinants of niche creation are variety and value creation (iansiti & levien, 2002, 2004a). to measure the robustness of an ecosystem (iansiti and levien, 2002, p.34), the theory incorporates: 1) the survival rate item to measure the number of bankrupt or start-up companies inside the ecosystem; 2) the persistence of ecosystem structure factor to measure the number of relations and agents that are subject to structural changes and external shocks; 3) the predictability item to measure the outbound links (an affiliation of a company to more than one business ecosystem); 4) the limited obsolescence factor to measure the change in the market share of technology; and 5) the continuity of the use experience item to measure the gradual changes of consumer experience of ecosystem products. to measure productivity (iansiti and levien, 2002, p. 36), the theory uses three determinants of 1) the total factor productivity variable to measure the productivity of participants in converting factors of production into value; 2) the productivity improvement factor to measure productivity over time; and 3) the delivery of innovation factor to measure whether the ecosystem effectively delivers new technologies, services, and ideas to its members. to measure niche creation (iansiti and levien, 2002, p 37), the theory uses two factors of 1) variety, to measure the variety of new services and options, and 2) value creation, to measure the overall value of new options created inside the ecosystem. in sum, based on this theory, robustness is the competency of an australasian journal of information systems saheb & hosseinpouli mamaghani 2021, vol 25, research article exploring the digital business ecosystem 6 ecosystem to survive disruptions while productivity refers to the efficiency with which an ecosystem changes inputs into outputs and values. niche creation is also defined as creating variety and novel competencies (iansiti & levien, 2002). 3 research methodology as depicted in figure 2, in this study, we conducted two panels of a three-round delphi study. the first panel was implemented from january 2017 to february 2018, and the second panel was implemented from may 2019 until august 2019. the delphi study was selected because: 1) this method assists in collecting expert comments on a specific topic to reach a collective consensus over controversial or uncertain issues (von der gracht, 2012); 2) this method has been previously incorporated in the is literature to study digital technologies; 3) since the contributors to the study are well-informed about the digital business ecosystem surrounding iot technology in iran, their ideas can be beneficial (linstone & turoff, 1975) and can address the questions of our study. to complement the results of our delphi study (keil et al., 2013), we also conducted a series of semi-structured interviews. figure 2. research steps of the study 3.1 first panel of the delphi study we collected empirical data from january 2017 to february 2018, consisting of a three-round delphi study and content analysis. we chose the delphi method with open-ended questions and semi-structured interviews. based on national statistics for 2018, around 48 companies in iran have registered as iot companies, whose major mission is to deliver iot products and services. since that number was limited, we sent interview requests to all the companies, and only 15 companies agreed to be interviewed. of this number, eight were start-ups, five were smes, and two were pseudo-private companies (ppcs). of the 15 companies included in the second round, only 12 companies agreed to participate in the study. all twelve companies attended the third round. in the first round, we conducted two-hour open-ended interviews. we inquired about the following items: 1. the description of the iot ecosystem in iran 2. the factors influencing the performance of the system 3. the role of the interviewee's company within the ecosystem 4. the actors of the ecosystem and their roles stage 1: content analysis stage 2: twohour open-ended interviews with 15 iot companies (from 2017-2018) stage 3: 45minute semistructured interviews with 21 iot experts and senior staffs (from 2017-2018) stage 4: questionnaire and semi-structured interviews with 20 iot experts (2019) australasian journal of information systems saheb & hosseinpouli mamaghani 2021, vol 25, research article exploring the digital business ecosystem 7 5. the interaction and communication of the actors within the ecosystem 6. the networks and communities of the iot ecosystem. the delphi study enabled us to naturally forecast unclear topics with no constraint on responses. we employed facilitated and interactive group communication meetings to solicit feedback from our experts including ceos, marketing managers, business development managers, senior executive managers, and ctos. after consolidating the feedback and returning it to each respondent individually, we asked the experts to revise their feedback to derive a forecast based on the groups' opinions. the participants remained anonymous, so nobody dominated the discussion. . we repeated the delphi process in three rounds as we reached the stability of responses at this round. the semi-structured interviews at the second stage lasted for 45 minutes. the questions consisted of two parts: the first part was about the company's biography and its status in the ecosystem. the second part of the questions asked about the role of various actors within the ecosystem, the value creation process, the exchange of tangible and intangible assets in the ecosystem, and interactions with other actors and value networks. we asked the experts to elaborate on the iot ecosystem summarized from the last round. in the third round, we also included the perspectives of senior staff members of the companies, and in total, 21 individuals attended the interviews. 3.2 second panel of the delphi study we conducted the second panel from may 2019 to august 2019. we interviewed around 20 experts working with iot companies in iran. at this stage, we developed a questionnaire (table 1) based on the business ecosystem theory (moore, 1996), the theory of business ecosystem health by (iansiti & levien, 2004a), and the results of the first panel in regard to the administration of the delphi questionnaire (table 1), we designed some of the questions so that the experts could rate variables based on a 5-point numerical scale and some of the questions were designed so that the experts could choose from multiple items. we shared feedback of each round with each panellist and asked them to re-answer the questions until the panellists could reach a consensus. we calculated the mean rank, kendall's w, standard deviation (sd), and frequency statistics.. in order to increase the reliability of the questionnaires, we pre-tested the questionnaire among a sample of respondents. we developed the questionnaire on the café pardazesh website in farsi and sent a link to the experts via email and linkedin. one of the major elements of increasing the quality of results of delphi studies is selecting a panel of appropriate experts (paré et al., 2013). the identities of the experts remained anonymous to avoid affecting the respondents. in 2019, the number of iot companies increased to 150 compared to 48 companies in 2018. we sent the invitations to 40 companies, and 20 companies agreed to participate in our study. the experts were it managers, marketing managers, ceos, and r&d managers with more than 3 years' experience. we then interviewed 10 iot experts to complement the results of our delphi study. the questionnaire of the second panel was composed of seven sections designed to measure the actors, roles, interactions, value exchange, and health of the ecosystem. to design the questions on the "roles," "actors," "value exchange," and "interactions," we incorporated the definitions of “keystone,” “niche,” and “dominator” as well as the business ecosystem theory introduced by (iansiti & levien, 2004b; moore, 1993). we derived questions to measure the health of the ecosystem (parts a, b, and c) from the health of the ecosystem theory by (iansiti & levien, 2002, 2004a). australasian journal of information systems saheb & hosseinpouli mamaghani 2021, vol 25, research article exploring the digital business ecosystem 8 sections questions actors and roles (part a, multi-item choice) 1which players control the ecosystem and do not produce value in the ecosystem? 2which players monitor the key activities of the ecosystem and co-create value alongside the other actors? 3which players are major innovators of the ecosystem and produce value regarding specific and niche topics? items: a) start-ups b) pseudo-private companies c) regulatory bodies d) smes; e) public companies f) consumers g) others, please specify actors and roles (part b, multi-item choice) 1what is the role of pseudo-private companies (ppcs) inside the iot ecosystem? 2what is the role of regulatory bodies inside the iot ecosystem? 3what is the role of start-ups inside the iot ecosystem? 4what is the role of smes inside the iot ecosystem? items: a) short-term, to gain benefit without creating value inside the ecosystem b) to share knowledge and physical assets c) to establish continuous interaction with other actors to support and monitor them d) to encourage innovation and creativity e) to control the market and cycle of value network interactions (ranking from very bad to very good) 1from a general perspective, how do you estimate the relationship of actors inside the ecosystem? items: a) very bad: relationships are inappropriate or ineffective. actors only take advantage of each other. b) bad: no outcomes are expected from relationships. management is weak and inefficient. based on lobbying and favoritism. c) good: relationships produce good results, and actors are stronger than they were in the past. d) very good: ongoing and lasting relationships produce good value for all shareholders. other ecosystem actors improve each other's performance. health of the ecosystem (part a) (ranking from very high to very low) how do you evaluate the health of the digital business ecosystem surrounding iot technology in iran? 1there are continuous and consistent interactions among the actors of the ecosystem. 2there is a continuous knowledge flow among the actors. 3the environment of the ecosystem is promoting innovation and creativity. 4the actors encourage each other to be efficient and effective. health of the ecosystem (part b) (ranking from strongly agree to strongly disagree) do you agree or disagree with the following statements? a. robustness of the ecosystem 1. the number of successful iot start-ups is increasing inside the iot ecosystem in iran (survival rates). australasian journal of information systems saheb & hosseinpouli mamaghani 2021, vol 25, research article exploring the digital business ecosystem 9 2. the number of relations and agents is subject to structural changes (persistence of ecosystem structure). 3. actors of the digital business ecosystem surrounding iot technology belong to other business ecosystems and offer other services and products as well (predictability of remaining in the ecosystem through outbound links). 4. changes in the market share of iot depend on the diffusion of innovation (limited obsolescence). 5. consumer experience is evolving in response to the introduction of new technologies (continuity of user experience). b. productivity 1. there are relations between the amount of labor and capital employed and the additional value created (total productivity). 2. the productivity of firms is gradually improving (productivity improvement). 3. the members of the ecosystem can deliver innovation capabilities with no restriction and barrier (delivery of innovations). a) niche creation 1) the number of new options, technological building blocks, categories, and products is varied (variety) 2) iot technologies have created an increasing value inside the ecosystem (value creation). health of the ecosystem (part c) (ranking from very important to unimportant) please rate the importance of the following factors in increasing the health of digital business ecosystem surrounding iot technology in iran. 1integrated communication 2standardization 3transparency 4trust 5knowledge sharing 6economic collaborations 7re-engineering of business and organizational culture 8comprehensive regulations and their implementation 9security of the ecosystem 10educational programs table 1. questions asked in the second panel 4 results 4.1 results of the interviews in this part of the paper, we answer the following questions, asked in the first round of the study: who are the major actors and what roles do they play inside the ecosystem? what values do they exchange? how do the actors contribute to the evolution and foundational structure of the iranian iot industry? we have organized our analysis based on the pillars of value network offered by (westerlund et al., 2014). 4.1.1 value nodes and drivers the experts of the first round identified five major actors inside the ecosystem: pseudo-private companies (ppcs), start-ups, smes, consumers, and the government. the analysis of the interviews shows that the most dominant actor in the dbe of iot in iran is ppcs, which are called qosoo-lati in farsi. qosoo-lati is a private company that is mainly managed by the players who formerly worked in the state or are currently working in the state. the consensus australasian journal of information systems saheb & hosseinpouli mamaghani 2021, vol 25, research article exploring the digital business ecosystem 10 was that most of the ppcs violate the law of commerce to limit the activities of smes and startups, and to dominate the iot market. the experts reported that these actors dominate the b2b and b2g markets and their main interaction is with the government. the experts described the ppcs as "dominators" that do not create any shared value (value driver pillar) and focus mainly on the monetary benefits of their own companies without creating any shared tangible and intangible values for the other nodes. current stage of digital business ecosystem surrounding iot technology the stage is emerging: most iot companies have registered during the last three years; most of them have not engaging and dynamic websites. there is a lack of transparency in reporting the annual activities and statistics of enterprises particularly the ppcs. actors dominator actors: pseudo-private companies niche actors: smes, start-ups keystone player: no keystone player intention of the start-ups and sems to enter the iot ecosystem major incentives: monetary reasons, and desire to improve operational and management processes, interaction with other value nodes, and decisionmaking minor incentives: desire to improve interactions with the customers and sustainability intention of the ppcs to enter the iot ecosystem major incentive: need to exploit knowledge and expertise minor incentives: desire to improve interaction with the other value nodes and with the customers, and desire to promote shared values such as sustainability interaction of start-ups and smes with the most interaction with: private financial networks the least interaction with: state networks and research institutions interaction of ppcs with the most interaction with: government and other ppcs the least interaction with: start-ups and smes the major customer of startups and smes the general public (b2c) the major customer of ppcs government organizations and industries (b2b and b2g) challenges that the start-ups and smes face lobbying by the ppcs legal restrictions lack of universal standards lack of access to external data resources consumers' lack of trust lack of a unified platform for sharing iot data weak collaboration among the iot companies with the other actors, especially the networks need for re-engineering of business models and organizational culture lack of comprehensive regulations lack of security of the ecosystem lack of prosumers as there are only consumers of iot products and services table 2. intention to enter, interaction with the other actors, and challenges that start-ups and smes face within the iot ecosystem the experts reported that the smes and start-ups are the most active networks within the ecosystem and have the most effective interactions. the experts believed that the greatest interaction of the iot smes and start-ups is with the financial entities and banks to receive financial support. they also believed that networks composed of accelerators and innovation hubs have the least impact on the ecosystem. according to the experts, the iranian digital business ecosystem surrounding iot technology is not agile and volatile and lacks a fierce competition among the australasian journal of information systems saheb & hosseinpouli mamaghani 2021, vol 25, research article exploring the digital business ecosystem 11 nodes. the experts linked the situation to the dominance of ppcs inside the iot market. the experts identified a series of legal, business, data, and security factors that have negatively impacted the health and performance of the iot business ecosystem. these factors are summarized in table 2. regarding the role of the government, most experts reached a consensus that the government's major role has been the commercialization of ideas, products, services. 4.1.2 value structure (topology) analysis of the experts' opinions shows that the topology of the digital business ecosystem surrounding iot technology in iran (table 3) is composed of two major categories of players: the dominator players, who are a few large pseudo-private companies controlling the b2g and b2b markets, and the start-ups and smes mainly controlling the b2c market. the experts identified the ppcs as the dominator actors, and smes and start-ups as niche companies dominating the b2c market. the study shows that the ecosystem lacks a keystone player. the study also shows that the public is only comprised of the buyers and consumers of iot products that do not co-create any value in the ecosystem. so there is no "prosumer" and the ecosystem lacks "co-creation" of values by the consumers. context topology major category of players us keystone model and has a hub-centered star structure a large hub firm composed of private companies that interact with a large number of small suppliers europe flat mesh-like structure small and medium firms interact in a non-hierarchical manner, without a keystone or dominator player. iran dominator model we identified two major players: the first category of players is composed of a limited number of large ppcs dominating the business ecosystem of iot, and controlling the b2g and b2b markets. these actors are called dominators. the other category of players consists of niche companies delivering iot products to the public and controlling the b2c market. they are called niche players. the ecosystem lacks keystone players and value co-creation. interaction among the silos is non-hierarchical. table 3. comparison of value structures (topologies) in various contexts 4.1.3 value exchange and extract in this section, we describe value exchanges and value extracts among the actors. regarding the tangible values that smes and start-ups deliver to the ecosystem, most of the experts mentioned the iot hardware and devices as the main value. most experts have referred to smes and start-ups as "thing providers" of iot products and devices. the most value delivered in the ecosystem is the company's annual reports, mainly delivered at major exhibitions. the annual reports are summaries of the company's activities that are not available online. this limited access hinders transparency, accountability, and the public's right to access to data. the reports do not convey expertise and knowledge and are composed mainly of statistical information, and most of the content on the companies' websites is pure information and news. the websites of iot companies do not provide digital platforms for social interaction, engagement of actors, and knowledge sharing. regarding the tangible value that the iot ppcs deliver, the experts argued that the ppcs act within their silos with no value exchange or interaction with other smes and start-ups. the major interaction of the ppcs is with the government and large corporations and organizations. the experts believed, as one expert wrote, “the ppcs just extract knowledge australasian journal of information systems saheb & hosseinpouli mamaghani 2021, vol 25, research article exploring the digital business ecosystem 12 and experience from other iot companies without delivering anything into the ecosystem." the study shows that one of the major features of the digital business ecosystem surrounding iot technology is the lack of partnerships among partners. our analysis of the interviews shows that while the ppcs have not initiated partnerships with smes or start-ups, start-ups and smes have also shown little interest in partnerships with other smes and start-ups. most experts believe that the least powerful roles within the iot ecosystem belong to the cloud and big data providers, and this is because of the lack of service-centered logic in the ecosystem: "the ecosystem lacks cloud-based platforms to share data and other services with each other. the only service that smes and start-ups can offer is mobile applications for iot devices," said one of the experts. this study shows that most iot smes deliver iot products, mainly hardware and devices with no value-added services, such as big data-based services. the experts mentioned that the lack of strong competition in the market, weak infrastructures such as the weak role of cloud services, and dominance of ppcs in the b2b and b2g markets have forced the iot smes and start-ups into the traditional business model of selling goods: “the main business models of most iot smes and start-ups are distributing or importing iot devices, and developing a mobile app as the value-added of their devices, mainly for the public to be used in smart buildings." products and services are delivered with a "fixed price" revenue stream. the experts mentioned the lack of effort inside the b2c market to deliver value-added services for iot products, and the fact that iot smes and start-ups operate product-centered businesses. most iot companies define themselves as "sellers of iot devices" or “thing providers." ironically, the experts mentioned the urgent need to re-engineer their business models and monetization methods, but they limited their roles to traditional sellers of iot devices, and considered developing an organizational vision and enterprise architecture as a "null and useless" effort to succeed in the ecosystem. the current study demonstrates the lack of interest among iot smes and start-ups in developing meaningful interactive and engaging social communities around iot products. as a result, the chances that smes and start-ups will be able to extract value have been reduced and iot products and services are not customized based on the perspectives of the consumers. for instance, one of the experts mentioned this fact by arguing that, "at this stage, we can't customize products based on the requests of our customers. we do not think communities around iot can deliver us meaningful information about the behavior and requests of the customers." the experts noted that the iot products that smes or start-ups deliver to the market "are not designed to offer too much choice for consumers." lack of proliferation of choices is due to the actors' disbelief in social media and the value of opinions expressed through social networks: "social networks are full of unsatisfied customers nagging each other about the delivery process of their devices, and not the product itself." to the experts, the iot ecosystem in iran is still in its early stages, and since funding and budget are very limited, the iot products are "a fixed set of features," and smes and start-ups cannot offer highly customized products. 4.2. results of the delphi study in this section, we describe the results of the delphi study to explore the health of the digital business ecosystem surrounding iot technology based on the perspectives of experts. australasian journal of information systems saheb & hosseinpouli mamaghani 2021, vol 25, research article exploring the digital business ecosystem 13 4.2.1 actors and roles regarding the keystone, niche, and dominator actors, table 4 shows that in round 3, the experts reached a consensus regarding ppcs as actors controlling the dbe of iot (with 85% agreement, .366 = sd and 1.85= mean) (table shows only the top-selected items). the results show that smes are keystone players (90% agreement, .308= sd and 2.90= mean), and startups are niche players (95% agreement, .224= sd, and 3.95=mean) inside the ecosystem. with regard to the roles of ppcs, regulatory bodies, start-ups, and smes, as table 4 shows, the main role of ppcs and the regulatory bodies is controlling the market (95% agreement, .224= sd; and 4.95=mean), while the role of start-ups, with a high sd, is sharing knowledge and encouraging innovation (50 % agreement, 1.026=sd, and 3.00=mean). the role of smes is also continuous interaction with other actors to support and monitor them (85% agreement, .366= sd, and 2.85= mean). questions round 1 round 2 round 3 percentage sd mean percentage sd mean percentage sd mean which players control, or monitor, or major innovators inside the ecosystem? dominators (control) 1.761 2.45 .470 1.70 .366 1.85 1ppcs 40 70 85 2regulatory bodies 35 30 15 3public organizations 15 0 0 keystones (monitor) .759 3.45 .562 2.90 .308 2.90 1smes 55 85 90 2start-ups 30 0 0 3ppcs 5 10 10 niche (innovator) .718 3.90 .410 3.80 .224 3.95 1start-ups 80 80 95 2smes 10 20 5 what is the role of ppcs, regulatory bodies, start-ups and smes? ppcs 1.773 2.75 .410 4.80 .224 4.95 1short term 45 20 5 2encouraging innovation 20 0 0 3control market 25 80 95 regulatory bodies 1.556 4.00 .410 4.80 .224 4.95 1short term 15 0 5 2encourage innovation 5 20 0 3control market 65 80 95 start-ups 1.234 2.45 1.161 2.25 1.026 3.00 1short term 25 30 0 2share knowledge 40 40 50 3encourage innovation 35 25 50 smes 1.482 2.75 .827 1.50 .366 2.85 1short term 30 20 5 2share knowledge 10 10 10 3continuous interaction 35 70 85 respondents number (n) 20 20 20 australasian journal of information systems saheb & hosseinpouli mamaghani 2021, vol 25, research article exploring the digital business ecosystem 14 table 4. results of the experts' consensus regarding the actors and their roles inside the iot business ecosystem 4.2.2 interaction of actors with regard to the interactions, as table 5 shows, the experts reached a consensus in the third round that the relationship of actors is "bad" (85% agreement, .366= sd, and 1.85= mean). meaning that no outcomes are expected from relationships, and that management is weak and inefficient and is based on lobbying and favoritism. questions round 1 round 2 round 3 percentage sd mean percentage sd mean percentage sd mean from a general perspective, how do you estimate the relationship of actors inside the ecosystem? .759 2.05 .444 1.75 .366 1.85 very bad 25 25 15 bad 45 75 85 good 30 0 0 table 5. interaction of the actors inside the ecosystem 4.2.3 health of the ecosystem we initiated our inquiry regarding the health of the ecosystem with a general question: how do you evaluate the health of the digital business ecosystem surrounding iot technology in iran? as table 6 shows, the experts reached a consensus that there is low continuous and consistent interaction among the actors (90% agreement, .308= sd; and 4.10= mean), and the level of continuous knowledge flow among the actors is low as well (90% agreement, .308= sd; and 4.10= mean) (the table shows only the top three options). the study also shows that the environment of the ecosystem in promoting innovation and creativity is at a low level (70% agreement; .470=sd; and 3.70=mean), and the level of encouragement by actors to be efficient and effective is low as well (70% agreement; .470=sd; and 3.70=mean). questions round 1 round 2 round 3 percentage sd mean percentage sd mean percentage sd mean how do you evaluate the health of digital business ecosystem surrounding iot technology in iran? continuous interaction .875 4.15 .686 3.95 .308 4.10 1. average 30 10 0 2. low 25 70 90 3. very low 45 15 10 continuous knowledge flow .826 3.95 .887 3.95 .308 4.10 1average 35 10 0 2low 35 55 90 3very low 30 25 10 encouraging atmosphere .865 3.70 .718 4.10 .470 3.70 1average 25 5 30 2low 50 65 70 3very low 15 25 0 promoting innovation 1.040 3.65 1.046 3.60 .470 3.70 1average 30 20 30 2low 30 50 70 australasian journal of information systems saheb & hosseinpouli mamaghani 2021, vol 25, research article exploring the digital business ecosystem 15 3very low 25 15 0 table 6. results of the experts' opinions regarding the overall health of the iot ecosystem we also measured the health of the ecosystem based on the determinants of robustness, productivity, and niche creation. the experts rated the statements from "strongly agree" to "strongly disagree." table 7 shows the combined percentage of "strongly agree" and "agree" and the combined percentage of "strongly disagree" and "disagree" for each factor for each round. question round 1 round 2 round 3 percentage sd mean percentage sd mean percentage sd mean robustness . 1survival rate agree: 60 disagree:30 1.234 2.55 50 45 1.210 2.90 60 40 1.050 2.95 2persistence of ecosystem structure 55 20 .883 2.60 65 35 1.040 2.65 85 15 .759 2.45 3predictability 90 10 .910 1.75 75 25 .995 2.40 90 10 .650 2.05 4limited obsolescence 70 5 .813 2.15 55 45 1.089 2.85 55 20 .988 2.65 5continuity of use experience 85 15 .912 2.10 65 35 1.192 2.50 75 10 .967 2.25 productivity 1total productivity 55 25 1.273 2.60 55 40 1.210 2.90 65 15 .882 2.40 2productivity improvement 65 20 1.276 2.55 55 45 1.099 2.95 60 40 1.071 2.90 3delivery of innovations 50 50 1.572 3.05 45 55 1.099 3.05 25 60 .998 3.45 niche creation 1variety 65 30 1.410 2.25 65 35 1.192 3.05 60 30 1.137 2.65 2value creation 50 20 1.231 2.60 55 40 1.192 2.50 40 60 .945 2.95 respondents number (n) 20 20 20 table 7. results of the experts' consensus regarding the critical success factors of iot ecosystem concerning the robustness of the ecosystem, as table 7 shows, 60% of the experts agreed that the number of successful iot start-ups is increasing inside the iot ecosystem in iran; 85% believed that the number of relations and agents is subject to structural changes; 90% argued that actors of the digital business ecosystem surrounding iot technology belong to other business ecosystems and offer other services and products as well; 55% affirmed that changes in the market share of iot depend on the diffusion of innovation; and 75% claimed that consumers experience is evolving in response to the introduction of new technologies. in total, half of the experts reached a consensus that the digital business ecosystem surrounding iot technology in iran is robust. regarding the productivity of the ecosystem, 65% of the experts agreed that there are relationships between the amount of labor and capital employed and the additional value created, while 60% of the experts claimed that the productivity of firms is gradually australasian journal of information systems saheb & hosseinpouli mamaghani 2021, vol 25, research article exploring the digital business ecosystem 16 improving. concerning the delivery of innovations, only 25% believed that the members of the ecosystem can deliver innovation capabilities with no restrictions, and 60% claimed otherwise. in total, around 50% of the experts believed that the digital business ecosystem surrounding iot technology in iran is productive. concerning niche creation, 60% believed that the number of new options, technological building blocks, categories, and products is varied, and only 40% believed that iot technologies have created increasing value within the ecosystem. in sum, around 50% of the experts reached a consensus that the digital business ecosystem surrounding iot technology has niche creation competencies. in the third step in measuring the health of iot business ecosystem, we asked the experts to rate the importance of a series of the factors impacting the health of the digital business ecosystem surrounding iot technology in iran. these items were highly referenced by the experts in the first panel. we consolidated the list and extracted 10 highlyreferenced factors. in this part, we measured the rank, kendall's w, and standard deviation (sd) of each item. as table 8 shows, in the first round, 20 iot experts filled out the items. the most important items were integrated communication among the actors, educational programs, job security, iot regulations, and standardization (mean rank >1.50). the study shows that the highest standard deviation was .489 for the economic collaboration item and .470 for the trust option. this high sd shows the lack of consensus among the experts. the final kendall's w score is .479 showing a moderate degree of consensus among the experts. items round 1 round 2 round 3 mean sd rank mean sd rank mean sd rank integrated communication 1.30 .470 7 2.05 .686 1 1.90 .308 1 educational programs 1.45 .686 5 2.05 .887 1 1.80 .410 2 job security 1.45 .686 5 1.40 .503 6 1.75 .444 3 iot regulations 1.80 .894 2 1.40 .503 6 1.75 .444 3 standardization 1.40 .598 6 1.55 .510 5 1.75 .444 3 economic collaborations 1.55 .686 4 1.70 .470 3 1.65 .489 4 trust 1.40 .598 6 1.65 .489 4 1.30 .470 5 knowledge sharing 1.70 .733 3 1.20 .410 8 1.15 .366 6 organizational culture 2.00 .918 1 1.35 .489 7 1.10 .308 7 transparency 1.45 .605 5 1.95 .510 2 1.05 .224 8 total number of respondents 20 20 20 kendall's w .115 .248 .479 table 8. results of experts' opinions regarding the factors influencing the health of digital business ecosystem surrounding the iot technology 5 discussion, implications, limitations, and possible future research in this research, we conducted two panels of a delphi study. in the first panel, we conducted interviews and in the second panel, we conducted a three-round study by distributing questionnaires. we conducted our study in two phases: during the first phase, we interviewed iot experts working in iot companies in iran. during the second phase, we conducted a threeround delphi study by distributing questionnaires among the iot experts. table 9 shows the convergence of the results from two phases. the main objective of this research is to extend australasian journal of information systems saheb & hosseinpouli mamaghani 2021, vol 25, research article exploring the digital business ecosystem 17 the literature on the digital business ecosystem by exploring the health of the digital business ecosystem surrounding iot technology, complex interactions of actors with unique roles, heterogeneous levels of control, and distinct degrees of interaction with the other actors of the ecosystem in iran. inquiries first panel ( interviews) second panel (delphi study) major actors pseudo-private companies (ppcs), start-ups, smes, consumers, and the government pseudo-private companies (ppcs), start-ups, smes, and the government roles of major actors ➢ dominator: pseudo-private companies (the b2g and b2b market) ➢ niche players: start-ups, smes dominating the b2c market ➢ keystone: not detected ➢ dominator: pseudo-private companies ➢ niche: start-ups ➢ keystone: smes nature of interactions ➢ most interaction with financial entities and banks ➢ ppcs have zero or little interaction with the smes and start-ups ➢ the most interactions occur among the ppcs, regulatory bodies and the state ➢ lack of partnerships among the actors ➢ no outcomes are expected from the relationships between the ppcs and niche players; however, the major outcome of relationships among the ppcs is financial with banks and financial institutions. ➢ management is weak and inefficient and is based on lobbying and favoritism health of the ecosystem ➢ lack of agility, volatility, and fierce competition inside the ecosystem due to the dominance of ppcs ➢ lack of development of digital business models and reliance on legacy business models ➢ lack of interest in developing mechanisms of co-creating products and services specifically with consumers general health status: ➢ lack of continuous and consistent interaction among the actors ➢ lack of continuous flow of knowledge among the actors ➢ lack of encouraging environment for innovation and creativity robustness (consensus 50% >) ➢ the increasing number of successful iot start-ups ➢ involvement of iot actors in other business ecosystem ➢ changes in the iot market share are dependent on the diffusion of innovation ➢ consumer experience is evolving in response to the introduction of new technologies productivity (consensus 60% >) ➢ there are relationships between the amount of labor and capital employed and the additional value created ➢ the productivity of firms is gradually improving ➢ the members of the ecosystem can deliver innovation capabilities with no restriction or barrier (consensus 25%) niche creation (consensus 40% >) ➢ the number of new options, technological building blocks, categories, and products is varied ➢ iot technologies have created an increasing value inside the ecosystem table 9. comparisons of the results of the first panel and second panel australasian journal of information systems saheb & hosseinpouli mamaghani 2021, vol 25, research article exploring the digital business ecosystem 18 as the study shows, there are two aspects of the digital business ecosystem around iot in iran: one ecosystem consists of only pseudo-private companies with strong interactions with government organizations and state-owned banks. the other digital business ecosystem surrounding iot technology is composed of start-ups and private small and medium companies with no interaction with pseudo-private companies. the interaction between startups and smes is mainly with the universities. most of the start-ups are located in the science and technology parks of universities, and their funding is mainly from either private investors or universities. this shows that these two digital business ecosystems have not formed one unified collaborative network or modes of value-co-creation. as a result, in the iranian context, due to the lack of collaborative and competitive interactions between the iot pseudo-private companies and the iot start-ups and smes, companies and individuals compete and cooperate in two separate digital business ecosystems and utilize their own common set of core assets. the study shows that pseudo-private companies who have greater access to expanded networks of financial, social, knowledge and political resources control and dominate the b2g and b2b markets. start-ups and smes have access to the b2c market, which owns a very small portion of the iot market in iran. the nature of interactions in digital business ecosystems of iot in iran is mainly financial and the exchange of knowledge between the actors is minimal. this has impacted the development and competitive power of iot start-ups. the paucity of knowledge exchange between actors is exacerbated by the shortage of valuable partnerships, which is one of the most effective strategies for developing the iot market. the dearth of valuable collaborations adversely affects the ability of start-ups to increase the value of their products to target markets. the pseudo-private companies’ lack of interest in establishing mechanisms for co-creating products and services with iot start-ups is another roadblock preventing the distribution of the iot value chain and increasing the centralized value chain managed and monitored by ppcs. the monopoly of pseudo-private companies on the b2g and b2b markets has also minimized agility, volatility, and fierce competition while weakening the competitiveness of start-ups. as a result, the environmental context for the growth of the digital business ecosystem surrounding iot technology is deprived of valuable partnerships and collaborations, which is a major obstacle to cultivating a digital business ecosystem. in summary, this study explores the digital business ecosystem of iot in the context of a developing country. this study can therefore be used for further research on the business ecosystem of cutting-edge technologies in other developing countries, especially in contexts in which governments play a major role. this study has its own limitations, as most pseudoprivate companies declined to participate in our study. the other drawback of this analysis is that we have not interviewed government entities or international companies. one potential direction for future research is to involve more stakeholders to provide a more detailed and informative description of the digital business ecosystem surrounding iot technology in iran. another possible line for future research is to study the business ecosystem of iot in other contexts in which ppcs play a key role in the market growth of cutting-edge technologies to recognize similarities and differences between these contexts. references adner, r. (2006). match your innovation strategy to your innovation ecosystem. harvard business review, 84(4), 98–107; 148. http://www.ncbi.nlm.nih.gov/pubmed/16579417 australasian journal of information systems saheb & hosseinpouli mamaghani 2021, vol 25, research article exploring the digital business ecosystem 19 aksenova, g., kiviniemi, a., kocaturk, t., & lejeune, a. (2018). from finnish aec knowledge ecosystem to business ecosystem: lessons learned from the national deployment of bim. construction management and economics. https://doi.org/10.1080/01446193.2018.1481985 chuang, m. y. (2018). association for information systems ais electronic library (aisel) towards iot entrepreneurship: a preliminary investigation. mwais 2018 proceedings, 36. http://aisel.aisnet.org/mwais2018/36 corallo, a., passiante, g., & prencipe, a. (2017). the digital business ecosystem. edward elgar publishing. https://dl.acm.org/citation.cfm?id=1534739 financial tribune. (2017). iran gov’t prioritizing establishment of iot infrastructure. financial tribune. https://financialtribune.com/articles/economy-sci-tech/68116/iran-gov-tprioritizing-establishment-of-iot-infrastructure hein, a., weking, j., schreieck, m., wiesche, m., böhm, m., & krcmar, h. (2019). value cocreation practices in business-to-business platform ecosystems. electronic markets. https://doi.org/10.1007/s12525-019-00337-y herweijer, c., combes, b., johnson, l., mccargow, r., bhardwaj, s., jackson, b., & ramchandani, p. (2018). enabling a sustainable fourth industrial revolution: how g20 countries can create the conditions for emerging technologies to benefit people and the planet. https://www.econstor.eu/handle/10419/177283 iansiti, m., & levien, r. (2002). keystones and dominators: framing the operational dynamics of business ecosystems. in the operational dynamics of business ecosystems. https://pdfs.semanticscholar.org/8dcc/511cf90a76811107bf6c7757d8b630505e4b.pdf iansiti, m., & levien, r. (2004a). strategy as ecology. harvard business review , march. https://hbr.org/2004/03/strategy-as-ecology iansiti, m., & levien, r. (2004b). the keystone advantage: what the new dynamics of business ecosystems mean for strategy, innovation, and sustainability. https://www.hbs.edu/faculty/pages/item.aspx?num=16920 ikävalko, h., turkama, p., & smedlund, a. (2018). value creation in the internet of things: mapping business models and ecosystem roles. technology innovation management review, 8(3), 5–15. https://doi.org/10.22215/timreview/1142 jónasdóttir, h., dhanani, k., mcrae, k., & mehnen, j. (2018). upgrading legacy equipment to industry 4.0 through a cyber-physical interface (pp. 3–10). springer, cham. https://doi.org/10.1007/978-3-319-99707-0_1 kar, s., chakravorty, b., sinha, s., & gupta, m. p. (2018). analysis of stakeholders within iot ecosystem (pp. 251–276). springer, cham. https://doi.org/10.1007/978-3-319-78378-9_15 keil, m., lee, h. k., & deng, t. (2013). understanding the most critical skills for managing it projects: a delphi study of it project managers. information & management, 50(7), 398– 414. https://doi.org/10.1016/j.im.2013.05.005 lee, g. (2019). what roles should the government play in fostering the advancement of the internet of things? telecommunications policy, 43(5), 434–444. https://doi.org/10.1016/j.telpol.2018.12.002 australasian journal of information systems saheb & hosseinpouli mamaghani 2021, vol 25, research article exploring the digital business ecosystem 20 leminen, s., rajahonka, m., mika, w., & siuruainen, r. (2015). ecosystem business models for the internet of things. internet of things finland, 10(13). https://doi.org/10.13140/rg.2.1.4292.1126 leminen, s., westerlund, m., rajahonka, m., & siuruainen, r. (2012). towards iot ecosystems and business models (pp. 15–26). springer, berlin, heidelberg. https://doi.org/10.1007/9783-642-32686-8_2 linstone, h., & turoff, m. (1975). the delphi method. https://www.researchgate.net/file.postfileloader.html?id=563b341d5cd9e375988b45bc &assetkey=as%3a292381292285964%401446720541026 mazhelis, o., luoma, e., & warma, h. (2012). defining an internet-of-things ecosystem. lecture notes in computer science (including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics), 7469 lncs, 1–14. https://doi.org/10.1007/978-3-64232686-8_1 moore, j. (1993). predators and prey: a new ecology of competition. harvard business review . https://hbr.org/1993/05/predators-and-prey-a-new-ecology-of-competition moore, j. (1996). the death of competition : leadership and strategy in the age of business ecosystems. wiley harper business. https://lib.ugent.be/catalog/rug01:000431120 pan, j.-s., chen, s.-m., & nguyen, n. t. (computer scientist). (2012). intelligent information and database systems : 4th asian conference, aciids 2012, kaohsiung, taiwan, march 19-21, 2012, proceedings. part ii. springer. paré, g., cameron, a.-f., poba-nzaou, p., & templier, m. (2013). a systematic assessment of rigor in information systems ranking-type delphi studies. information & management, 50(5), 207–217. https://doi.org/10.1016/j.im.2013.03.003 rong, k., hu, g., lin, y., shi, y., & guo, l. (2015). understanding business ecosystem using a 6c framework in internet-of-things-based sectors. international journal of production economics, 159, 41–55. https://doi.org/10.1016/j.ijpe.2014.09.003 rong, k., shi, y., & yu, j. (2013). nurturing business ecosystems to deal with industry uncertainties. industrial management & data systems, 113(3), 385–402. https://doi.org/10.1108/02635571311312677 saheb, t., & izadi, l. (2019). paradigm of iot big data analytics in the healthcare industry: a review of scientific literature and mapping of research trends. telematics and informatics. https://doi.org/10.1016/j.tele.2019.03.005 selander, l., henfridsson, o., & svahn, f. (2010). transforming ecosystem relationships in digital innovation. international conference on information systems. http://www.divaportal.org/smash/record.jsf?pid=diva2%3a390104&dswid=3642 selander, l., henfridsson, o., & svahn, f. (2013). capability search and redeem across digital ecosystems. journal of information technology, 28(3), 183–197. https://doi.org/10.1057/jit.2013.14 senyo, p. k., liu, k., & effah, j. (2019). digital business ecosystem: literature review and a framework for future research. international journal of information management, 47, 52–64. https://doi.org/10.1016/j.ijinfomgt.2019.01.002 australasian journal of information systems saheb & hosseinpouli mamaghani 2021, vol 25, research article exploring the digital business ecosystem 21 shin, d. (2019). a living lab as socio-technical ecosystem: evaluating the korean living lab of internet of things. government information quarterly, 36(2), 264–275. https://doi.org/10.1016/j.giq.2018.08.001 sun, l., tan, c., robertson, s., liu, k., cook, m., & collins, c. (2016). open digital business ecosystems: a pathway for value co-creation (pp. 85–94). https://doi.org/10.1007/978-3-31942102-5_10 sun, q., wang, c., zuo, l., & lu, f. (2018). digital empowerment in a weee collection business ecosystem: a comparative study of two typical cases in china. journal of cleaner production, 184, 414–422. https://doi.org/10.1016/j.jclepro.2018.02.114 the mobile economy: middle east and north africa 2018. (2018). www.gsmaintelligence.com von der gracht, h. a. (2012). consensus measurement in delphi studies: review and implications for future quality assurance. technological forecasting and social change, 79(8), 1525–1536. https://doi.org/10.1016/j.techfore.2012.04.013 westerlund, m., leminen, s., & rajahonka, m. (2014). designing business models for the internet of things. in technology innovation management review (vol. 4, issue 7). talent first network. https://timreview.ca/article/807 copyright: © 2021 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v25i0.2719 http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems berglund 2017, vol 21, research on indigenous ict the youtube activism of the 1491s 1 ‘go cry over someone else’s tragedy’: the youtube activism of the 1491s jeff berglund northern arizona university jeff.berglund@nau.edu abstract this article examines the activist role played by the comedy troupe, the 1491s, in social media spaces, particularly on youtube. since 2009 the 1491s have used the positive energy of comedy to galvanise others, with a particular emphasis on shaping the ways indigenous peoples are seen, how indigenous peoples see themselves, and how changes might lead indigenous people to think critically about the conditions under which they are living. in addition to examining their comedic output, this article discusses the videos which honour indigenous resilience and advocate for political causes; these efforts are also supported by their production infrastructure and disseminated via their youtube channel. strategically disseminating their videos to a global audience through youtube, the 1491s should be seen as a major force in social change, inspiring indigenous and non-indigenous audiences alike, many of whom interact and form a core fanbase despite being separated by time and space. keywords: youtube; social media; activism; indigenous; comedy; resilience 1 introduction the gravity and magnitude of colonisation make it all too simple for non-indigenous peoples sympathetic to indigenous causes to focus on dire inequities and the lack of economic, political, social and educational access to the rights and privileges of the nation states who colonised their sovereign nations. indigenous artists, authors, and filmmakers have similarly been pulled to the ‘gritty realism’ end of the representational spectrum and have fallen prey to similar limitations. some critics have called the works of documentarians, filmmakers and writers alike who only focus on dysfunction and deprivation, ‘poverty porn’ (ross, 2013). by contrast, the 1491s, an indigenous comedy troupe which broadcasts through youtube, twitter, and facebook, vigorously work against this reductive take on indigenous realities, not by ignoring truths, but by highlighting the everyday incarnations of beauty, intelligence, survival and wit manifest in the humorous scenarios they construct. bobby wilson, one of the founding members of the 1491s, summed this up succinctly during a 2013 tedx talk in manitoba: “the big thing we’re trying to fight, man, is go cry over somebody else’s tragedy, ‘cause we’re alive and thriving, we have absolutely…the thing that gets forgotten in the middle of all the other things is that we’re modern, we’re human, we’re here!” (tedx talks, 2013). this article examines the activist role played by the 1491s in social media spaces, particularly on youtube but also on supportive platforms such as facebook and twitter. while their work is humorous and deeply ironising and quite simply brings happiness to people, it also connects fans together and uses the positive energy of comedy to galvanise others, with a particular emphasis on shaping the ways indigenous peoples are seen, how indigenous peoples see themselves, and how changes might be implemented that lead indigenous people to think critically about the conditions which they may resist and change. certainly, individual members of the 1491s have their own activist engagements in social media spaces, ones that are more overtly and recognizably politically activist in orientation. most notably, dallas goldtooth (santee dakota and diné) has been extremely active on facebook, facebook live and twitter as he broadcast from standing rock, north dakota resistance to the dakota access pipeline from august 2016 through january 2017. while goodtooth’s recent efforts deserve particular consideration in a separate analysis, i contend that his and his colleagues’ work as the 1491s offers equally profound and impactful work and deserves to be understood as activist in orientation and intent. the magnitude of their reach and their subversive power rests in australasian journal of information systems berglund 2017, vol 21, research on indigenous ict the youtube activism of the 1491s 2 their ability to lift up others through razor-sharp critiques, energizing, often humorous and positive messages, directed inside as well as outside of indigenous communities. in addition to goldtooth, the 1491s are composed of bobby wilson (sisseton dakota), ryan red corn (osage), and migizi pensoneau (ponca and ojibwe). occasionally, two other regular contributors make appearances: sterlin harjo (mvskoke) and tito ybarra (red lake band of ojibwe). core members have been producing their work and uploading videos to youtube since 2009. their name posits a point of origin prior to western colonisation, as 1492 marks in the western hemisphere the onslaught of the columbiad and european empire-building and genocide throughout the americas. as of march 2017, their youtube channel has had 7,205,585 views with 39,109 subscribers (figure 1). on facebook 60,955 follow them and 19,400 follow them at @1491s on twitter. figure 1: the 1491s’ youtube channel my aim in this article is to contribute to a growing body of scholarship on indigenous social media use and to a deeper understanding about the role played by youtube in connecting indigenous and non-indigenous viewers through work by indigenous performers such at the 1491s. i first situate this analysis within scholarship on youtube and indigenous social media use. then, of the dozens and dozens of choices to draw from, i turn my attention to their most popular humorous video, one that keys in on debilitating cycles of behaviour (including internalised colonialist logic) within communities, particularly centering on men. the logo for the 1491s is an arrow turned in on itself, a symbol of the group’s willingness to turn the critique on themselves and the communities of which they are a part. i situate this analysis in the tradition of humour within native culture. finally, i look at their more typical activist-based work such as public service announcements (psas) and celebrations of cultural strength and resilience. undoubtedly, the audience base for these more serious works in which they are the producers and not the stars in front of the camera, has been built and sustained by their comedic performances. together, the great variety of their work strategically disseminated to a global audience through the platform of youtube, positions the 1491s as a major force in social change, inspiring indigenous and non-indigenous audiences alike. australasian journal of information systems berglund 2017, vol 21, research on indigenous ict the youtube activism of the 1491s 3 2 youtube as a performative social media platform for indigenous activism before 2009, sterlin harjo explains, “no one had really picked up a video camera and made youtube videos for native people. there was a space for us to do it, because no one else had done it” (cited in bullock, 2015). with the advent of technology that can be held in the palm of one’s hand, and which is more affordable than ever before, expressive artists like the 1491s who plug into social media now possess the potential to reach broad audiences quickly and efficiently. social media has transformed the typical power relationships between media technologies and media makers. mobile technologies mean that like no other time in the history of the world, technologies can be used to connect us, to help us understand one another. while the role of youtube is currently understudied in the u.s. context of native american social media use, social media, generally, has undeniably reshaped the nature of community and provided ways to bridge the time-space divide, bringing people together around common causes and vision. this was well-documented with the ascendancy of idle no more in 2012 and 2013 from one province in canada to a national and then global phenomenon. diné/navajo writer, natanya ann pulley, reflects on the powerful connections that continue to be made among native people in online spaces that plug into indigenous peoples’ cosmologies of time and space in her essay ‘indigitechs’: “i exist outside the western grid of time-space, in a space where my encounters with my native brothers and sisters strengthens our connection to the cosmos. it is the air-spirit of the first world alive and kicking” (2014, p. 99). “of course,” she continues, “i can, at any time, socialise with my native relatives and future-friends on facebook. i can also keep in touch with native artists, writers, and poets. but more than this, native people acquire a political pulse online that in the past was sequestered on the reservation—and beaten down there, as was the american indian movement at pine ridge” (2014, pp. 99-100). though surveillance of politically engaged native gatherings continues even online, status updates, the circulation of petitions, announcements of events, and shared memes and jokes join people together in ways that mainstream media and in-person events cannot. “the ‘natives be like’ joke memes are just as important as shared links to pages about the latest rally against the keystone xl pipeline” (2014, p. 100). pulley emphasises here that the “micro utterance,” perhaps something like a meme, has macro consequences as it “reverberates across hyperspace” (p. 100). new scholarship is emerging on the expansive role played by social media in indigenous communities with clear reminders that indigenous peoples are early and frequent adapters of new technologies. research by indigenous scholars bronwyn carlson, acushla dee o’carroll [sciascia], and alex wilson has examined the role played by social media in indigenous australian, māori, and first nations canadian and international indigenous contexts. previously, i have written about what indigenous theorists would describe as the ‘visual sovereignty’ of the 1491s comedic stylings on youtube, particularly in light of misrepresentation and narrow visions of native american peoples (berglund, 2016). in addition to looking at the role played by social media in revitalising communities and connections across great distances, this research also highlights the impact on activism and social justice initiatives. as carlson (2013) notes, we still have a lot to learn about aboriginal use of social media and the potential that social media may offer in terms of social and cultural interaction. but one thing is quite clear—social media is here, aboriginal people are online and are posting and interacting with one another, having conversations, debates and forming relationships. social media is a social site but as i have demonstrated, it is also a political site where aboriginal struggles and identities are being played out in the ‘new frontier’ (p. 164). 2.1 youtube in terms of scholarship on youtube, very little has yet been focused on indigenous content and subscriber and viewer comments. that said, one of the first full-length scholarly books on youtube, youtube: online video and participatory culture (burgess & green, 2009) did focus australasian journal of information systems berglund 2017, vol 21, research on indigenous ict the youtube activism of the 1491s 4 a few pages on the rapid proliferation on youtube of the then australian prime minister kevin rudd’s 2008 apology to the stolen generations and video reactions and edited responses and reinterpretations of this landmark political event. the reaction to this moment marked one of the first sustained and rapidly unfolding dialogues on youtube surrounding indigenous causes. indigenous peoples’ reactions circulated because of youtube, voices that mainstream media outlets would have ignored or marginalised, and perspectives that might have been rendered primarily in indigenous newspapers and even in limited newscasts at later dates because of production delays. in addition to scant attention paid to indigenous content on youtube, little attention has been paid to the variable ways that youtube functions as a social media format. while it does offer the space for interaction and some limited collaboration—as detailed below—its primary function for many users is sometimes outside of social media spaces as yet another disseminating conduit for entertainment, tutorials, news, and self-expression, all of which are commercialised or monetised to varying degrees. the website youtube went live in 2005 and was purchased a year later by google for u.s. $1.56 billion, leading quickly to its commercialisation and monetisation. the 1491s uploaded their first video in 2009, well after youtube had gone commercial, providing a platform for professional and amateur videos alike. fan bases elevate the status of all works, irrespective of origin, and dictate the terms of further monetisation and thus commercial viability of particular sites. some independent producers of videos have engineered their careers to earn up to $12 million a year (berg, 2015). building an audience base that increases revenue streams depends on industries’ perception of the profit-potential of the audience. a play-through gaming site or a children’s toy site is directly linked to consumer purchasing—both of these types of youtube channels and stars routinely make forbes top-ten lists of richest youtubers (berg, 2015). what is the profit-potential in indigenous sovereignty? while this might be taken as a sarcastic, rhetorical question, there’s actually an answer: educational and business-oriented services. colleges, grammerly—directed at college students in need of help with their writing— and gotomeeting, an online service to facilitate long-distance business partnerships, are all currently advertisers on the 1491s’ youtube site (although, admittedly, one wonders, if this ad placement is dictated by analytics about my own online identity). by the time readers of this published article turn to the site, there may be different advertisers. even with regular sponsorship, youtubers such as the 1491s only make income if audiences click on the ads or watch the initiating ads (ones that roll before the video content will display) in their entirety, both of which are unlikely if consumers are concerned about conserving data streaming charges (even more likely the case if there’s a mismatch between the advertisement and the consumer interests). on top of this, for their use of the platform, youtube commands a 45% cut of profits. according to comments made by billy wilson at mesa community college in 2014, the 1491s earn a modest u.s. $150-$200/month and depend on sales of t-shirts and performance fees to supplement income earned in their other ‘day jobs’ (haskie-gonzalez, 2014). it costs money doesn’t it? when we get together we do videos. we’re indian guys so we do other things…then we’re getting live shows and make money. our youtube channel pays us $150-200 a month. we sell shirts, and once a year we get together and make videos, plan. we budget, we plan things, we figure out how we’ll pay for things, etc. when some asks, ‘what do you do in “real life’? billy explains that he manages the group and its schedules. he’s also a graffiti artist and does murals. he goes on to say, dallas goldtooth works with the indigenous environmental network and does a shitload of planning for this. ryan red corn owns two businesses, a fry bread mix company, he inherited from his grandfather, and a marketing firm. migizi is a student working on his masters in creative writing. sterlin harjo makes real movies, not this bullshit like we do. (haskie-gonzalez, 2014). australasian journal of information systems berglund 2017, vol 21, research on indigenous ict the youtube activism of the 1491s 5 currently there is no analytic to track ethnic or gender identities of viewers. youtube tracks numbers: the number of viewers (from unique ip addresses; return viewers from the same computer are not tabulated); the number of likes (thumbs up) or dislikes (thumbs down), allowed only by registered users; and, the number of subscribers to the 1491s channel. registered users are allowed to leave comments on the 1491s’ videos. registered users can reply to others’ comments. each video has many more viewers than commenters, as might be expected. it’s hard to know how many viewers read comments left by other viewers; this information isn’t tracked. viewers may sort by two categories, ‘top comments’ and ‘most recent comments.’ if a viewer is interested in reading more than twenty or so comments, they may click ‘show more’ to go further back in the comment archive (comments are preserved unless the 1491s delete comments). in her in-depth study of youtube and its unique facets that alter the reception of digital film, carol vernallis (2013) writes, [youtube video’s] online distribution also encourages participation. in miniature, the video’s performers may carry more weight than they would on television—the technological magic that brings them to us feels palpable. we may also experience greater agency with viral media, because a click allows us to seek out the video’s performers, who address us directly—one more click or turn away from the monitor would break a fragile bond. as we forward the link to those in our affinity groups, our sense of connection branches outward (p. 171). to enjoy the 1491s’ work, all one needs to do is google their name or enter their name in the youtube search bar. but if that’s not enough, you may follow them on twitter, you may subscribe to their youtube channel and you may like them on facebook. these various multimedia platforms serve to notify viewers of new postings, keep them updated about other news, and, importantly, maintain a relationship across space and time. depending on one’s smartphone settings, announcements about new postings can immediately reach followers. while there are differences of opinions about the possibilities of youtube to build a community that receives and creates progressive and innovative media of value, there are some scholars who contend that the circulation and sharing of videos fosters ‘kinship networking.’ vernallis (2013) writes, youtube provides intense media experiences…sometimes a clip sent by a friend or colleague produces a moment of intimacy, as if we were in a conversation and one of us had suddenly turned the witty phrase that concretized gathering experience. the threeminute clip swells in the light of our shared feeling. the clip’s cleverness helps it merge into our paths of dialogue and mutuality (p. 149). the on-going responses and comments by viewers is a surprising feature of youtube and grounds the experiences in the real lives of people. in previous generations, fans could have written a letter, formed a fan club, bought a ticket to an event, but youtube enables direct engagement. 3 humorous/satirical engagements in engaged resistance (2011, p. 93), dean rader coins a term to articulate the activist contributions made by artists—“aesthetic activism,” which is “a manner of political and social activism that finds representation in the artistic realm”. i find it useful to think about the 1491s work as ‘aesthetic activism’ because while they’re not writing policy statements, and they’re not leading marches, they are forcefully shaping the discourse about the value and rights of indigenous peoples, and their work definitely issues responses to relevant and timely incidences and everyday occurrences in “indian country.” this is an especially useful interpretative tactic when analysing works that seem humorous, silly, and sometimes downright “lite.” an understanding of the function of humour within indigenous cultures, though, recognises that humour is a tactic used to move people, to change attitudes, to critique behaviours, and to lead to change. laguna pueblo writer and theorist, paula gunn allen australasian journal of information systems berglund 2017, vol 21, research on indigenous ict the youtube activism of the 1491s 6 considers humour “the best and sharpest weapon we’ve always had against the ravages of conquest and assimilation” (as cited in lincoln, 1993, p. 7). perhaps it is the transformative possibility of humour that led playwright hanay geiogamah (kiowa and delaware ) to say, “i see the indi’n capacity for humor as a blessing…it’s a miraculous thing that’s pulled us through so much. it’s a force that’s part of religion” (as cited in lincoln, 1993, p. 336). 3.1 ‘slapping medicine man’ one of the most popular of the 1491s’ humorous videos is “slapping medicine man” (2011a) with a simple format and direct purpose: to dismantle a culture of rationalization that displaces to others blame for one’s conditions (see figure 2). while the video risks oversimplification of the complexities of obesity, depression, gossip, unemployment, broken relationships, and so forth, the healer in the series asks a series of questions of three young men who visit him and gives them a hard dose of reality: a real question that cuts to the heart of the men’s own complicity in their hardship followed by a physical rebuke—a slap in the face. a fixed over-theshoulder-shot puts viewers in the position of the medicine man who is consulting with three young men played by tito ybarra, dallas goldtooth, and ryan red corn. when tito’s character talks about being overweight and sick, he’s asked if he drinks or parties; when he replies in the affirmative, he’s met with a “well, knock it off!” and a slap. when ryan’s character says, “i’m a student, and i don’t think i’m eligible but i’d like one of those free indian scholarships,” he’s met with a slap. and, when dallas’ character worries, “i don’t know what to do to get ‘em (my family) back,” the medicine man asks, “well, are you still drinking?” to which dallas replies, “well, yeah, from time to time!” then, without missing a beat, the medicine man delivers a sharp slap. the unexpected hard truth and the physical surprise of the slap combine to elicit laughter from viewers. the scenarios with the different characters cycle through and the medicine man continues to dispense with truths about hard work, employment and education. figure 2: the 1491s, ‘slapping medicine man’ with the greatest number of views of all of the 1491s’ videos (918,193) since it was uploaded in may 2011, ‘slapping medicine man’ has elicited a whopping 623 comments. the extreme ridiculousness of the scenario combined with physical humour and predictable, repeated australasian journal of information systems berglund 2017, vol 21, research on indigenous ict the youtube activism of the 1491s 7 patterns, has all the ingredients of classic comedy. michael redcrow’s response highlights typical audience reactions bwahahaha poor ryan, i felt so bad for him (ok, i was laughing so hard i almost fainted but i promise, i was feelin bad, too), and the slap (aho !) i literally fell out of my chair onto the floor, my wife thought i was havin some kind of attack ! i loves me some 1491's, and this one was almost the best ever! other typical responses focus on the medicine man’s logic. for example, wa-ya hilderbrand wonders, “is it kind of sad that my school beyond needs this medicine man?” another, gobbles wells (ketchuppopsicles), also wishes this hard medicine were in readier supply: “if only today’s medicine man was this serious about slapping some sense into the younger generations. nowadays you find that most are in it just for the financial opportunities to be had, much like the white man that many of them complain about...” shiidenzhoneshash comments on the way the 1491s are able to deliver these insights through their humorous scenario: “you know watching this video from time to time is good for a laugh but what a message it sends us through humor. the slap of reality, wake up and smell the coffee, so to speak snap outta it,…if you want change then change. what a jello community we have become. so because i can’t go around slapping my natives this video is some way good therapy for me . . . hahahahahhah...”. the satirical humour of ‘slapping medicine man’ targets self-pity and self-abuse by issuing a call for a commitment to action and control over one’s own life, a sovereignty of self that will build the possibility of transformation and resilience for others. ‘slapping medicine man’ shows the need for self-accountability and the sobering prescription of discipline, dedication, and hard work. 4 resistance, resilience building, and celebrations of indigenous brilliance the 1491s use the youtube platform to establish and maintain a community that can bear witness to and honour the many ways to be indigenous which empowers others to tell their stories and join the conversation. their work creates models of engagement and resistance, models of problem-solving, models of resilience-building and indigenous resurgence, testimonials of indigenous brilliance, beauty, and talent. a specific genre of serious and noncomedic activism by the 1491s is the public service announcements (psas) they routinely distribute through their youtube channel. disseminating psas online is not unusual, but their placement alongside the 1491s’ comedic playlists is unique and models an interdependent relationship, both in terms of production and praxis, or theory in action, reminding viewers of their serious commitment to indigenous peoples and communities even in their comedic engagements. their production infrastructure and celebrity affords the group a reach that extends far beyond local activist engagements; moreover, the 1491s openly share their equipment, filmmaking knowledge, youtube platform, and their existing fanbase, all to support work outside of typical commercial media infrastructures. 4.1 public service announcements (psas) one provocative 2011 psa joined a public dialogue about the epidemic of violence against indigenous women, both at the hands of indigenous and non-indigenous men alike during a period when the u.s. congress was considering the renewal of the violence against women act, originally passed in 1994. based on a poem written by randy red corn, “to the indigenous woman” was uploaded in october 2011 (figure 3). while red corn reads the poem in voiceover, viewers see images of indigenous women, loving and caring for their families and being loved by them in return—the reality that should exist but that does not match the stark reality facing the majority of indigenous women. australasian journal of information systems berglund 2017, vol 21, research on indigenous ict the youtube activism of the 1491s 8 figure 3: the 1491s, ‘to the indigenous woman’ details supplied with the video remind us of shocking, disturbing statistics: native women are murdered at 10 times the national rate; 1 out 3 native women will be raped in her lifetime, and 3 out of 5 physically assaulted. even worse, 88% of the perpetrators are non-indian and cannot be prosecuted by tribal governments. stand and take action now to restore safety and justice for native women. do something! visit www.indianlaw.org. red corn’s words give voice to feelings of regret and complicity and the need for everyone, all people, men and women, adults and children, native and non-native, to stand up and take action. to ignore this stark reality is to be complicit: to the indigenous woman
i'm sorry we have not fought harder for you i am the dysfunctional man i will borrow your forgiveness like i always have, and you will be there for me like you always are ask her, and she will tell you i stole her tongue replaced it with guilt saddled it with blame
and rode off on it like a horse . . . to doctors without clues for say nothing neighbors
 do nothing attorneys
 and quiet parents with no memories thank you.
 you make all of this possible.
 we couldn't fail these women without your help. for the woman and her baby left for dead by the police in her home, while they gave a ride to her attacker back to his house
 to the girlfriend punched in her pregnant stomach australasian journal of information systems berglund 2017, vol 21, research on indigenous ict the youtube activism of the 1491s 9 to the wife who took the beatings so her kids wouldn't have to to the daughter who found a man as abusive as her dad to the coed who will never ever go to the nine again to the restraining order as strong as the paper its made from to the shelter with not enough beds i give 1000 sweats for rape victims 1000 doctorings for husbands 1000 prayer ties for courage
 1000 meetings for silence 1000 songs for patience
 1000 fires... for enough light...
to fill a room .... to reflect off a mirror the size of the moon just so we can see ourselves for what we are. complicit. and if this recognition of complicity—the word ‘complicit’ flashes on the screen as red corn pauses--in inaction is not enough to move viewers of the video poem, red corn reminds those watching that these victims are all of our relatives. as he continues to speak, close up shots of women follow. the women stare directly into the camera, making eye contact with viewers, imploring us not to ignore their plight. red corn shifts to first-person in the closing lines, giving voice to the women themselves: i am your wife
 i am your sister
 i am your mother
 i am your daughter you are supposed to protect me you are supposed to be a warrior protect me from you,
 from him, from all of them. tell me you have daughters
 tell me you don't want this for them
 tell me you won't joke about this with your friends tell me you won't forget we talked
 tell me you will do something do something. this last line, like ‘complicit,’ flashes on the screen. the logic is clear. if you don’t ‘do something,’ you’ll be complicit and things won’t improve. viewer response to this video was intense and positive. one viewer, star kiona, writes, when i feel like crap because of statistics and due to being a native women, also feeling like i am going nowhere because of my history, and where i am from. i watch this. makes me feel like i am cared for, it inspires me to try my hardest to make it different and easier for us native people. i can almost see my future occupation when i watch this video. many thanks, 1491s. <3. lucretia lovato, another viewer comments, so moving and raw... thank you for calling out to the native men to help keep the women safe... you’re opening their eyes to what they are doing to their own women. thank you for being the voice of all those that are too scared, too beaten, or can no longer ask for justice... many thanks and bless you all for being brave... one male fan, rocky rockholt, stands by red corn’s call for all men, as well as native men, to recognise their responsibility toward indigenous women: australasian journal of information systems berglund 2017, vol 21, research on indigenous ict the youtube activism of the 1491s 10 thank you, thank you, thank you, for calling out the ‘warriors’ in our native american culture who will look away from a ‘family matter.’ my mom in an ndn woman super hero!! she raised three of us unruly kids to be solid citizens. along the way, she was raped, beaten, and treated as if she was disposable. she is not disposable!! she is the mother of a man who is hard wired to keep trying for his kids. she is a businesswoman. she doesn't even recognise her past. nor do i!!! rockholt speaks to the endurance of his mother and the way she transformed her past, but he also speaks to the intergenerational responsibility to be watchful and protective of women and children and to recognise this drive as a fulfillment of being ‘a warrior.’ the 1491s refined this message about putting an end to violence against women in later videos, as well, including a brief, thirty-second one that’s garnered 13,748 views since march 2012. titled ‘ten little indians,’ and underscored by the sing-song of the outdated and racist nursery rhyme of the same name, the simply formatted black and white video features extreme closeups of indigenous women looking into the camera. after viewers are shown a succession of ten women and are reminded that six out of ten native women will be physically assaulted, they are implored to ‘do something.’ the dissonance between the all-too-familiar song, not to mention its inappropriateness in today’s world, estranges the familiar and works to draw attention to an important cause. in 2013 the u.s. congress did reauthorise the violence against women act and then two years later, for the first time in u.s. history, the most significant update for native women was finally implemented: a provision allowing tribal courts to investigate and prosecute non-native men accused of abuse of indigenous women on federally recognised tribal reservations. in addition to these psas, the 1491s have produced a number of videos that beautifully and forcefully represent the diversity of indigenous lives. several focus on language: one issued on valentine’s day with dozens of speakers sharing a litany of words for love in different tribal languages; another nsfw video hilariously features phrases and strategies for indicating one’s attraction and romantic or sexual interest in others. in this category are also spoken word performance poems such as ‘bad indians,’ which feature multiple contributors, all native, collectively performing lines of the poem also written by ryan red corn and built around a simple yet profound argument about all of the ways that native people persist in brilliant contemporary ways. each speaker looks and sounds different; most speak english; some use their tribal language. collectively, their voices assert a contemporary indigenous presence, one that claims sovereignty and control over their own reality and a renewed and collective sense of empowerment. speakers look confidently and proudly into the camera showing respect and honour for their subject, models of strength, and endurance. they show how to turn on its head civil war general philip sheridan’s perhaps apocryphal comment, “the only good indians i ever saw were dead.” alive and thriving, the speakers in the video are very ‘bad indians,’ indeed, but bad in all the very best ways possible. they refuse to die, they refuse to simplistically embody homogenising stereotypes. similarly, the 1491s themselves are ‘bad indians’ who continue to persist, who continue to turn their humour on the colonisers and even turn it toward those within their communities who have internalised colonising ideologies. 4.2 the represent series: showcasing indigenous brilliance and resiliance the 1491s’ work revitalises native communities through a modeling of resistance to pernicious, damaging stereotypes, but they also do so, specifically, through their ‘represent’ series (which more than just nominally) embodies linda tuhiwai smith’s ‘representing’ project she lists among other indigenising research projects. and, furthermore, the represent series fulfills tuhiwai smith’s goals of ‘creating,’ ‘networking,’ and ‘sharing’ as this series is the outcome of collaborations (referred to as ‘collabs’) with people whom the 1491s meet on university campuses or within communities they visit. the 1491s offer the opportunity for others to tell their own stories by their using equipment, editing software, and the social media platform the group has built. migizi pensoneau explains the group’s motives: “we’ve put cameras in people’s hands and have asked what you would like to see. it goes in direct contrast australasian journal of information systems berglund 2017, vol 21, research on indigenous ict the youtube activism of the 1491s 11 to that sort of aaron huey national geographic thing where it’s like you know we’re just gonna showcase all the sad. look at this poor destitute indian and we’re like, you know what do you want to see when your culture is vital and thriving?...we’ve created these outlets for people to tell their own stories because it’s not something that it’s our place to, it’s someone else’s’ (tedx talk, 2013). the multiple, varied stories are all connected by the series title that is superimposed at the beginning and close of every video; a fixed camera is used in all videos, cuts are minimal to non-existent. these minimalist choices focus attention on the subjects’ talent. the represent films create the opportunity for other indigenous peoples to showcase their talents, to demonstrate their presence, intelligence, and what gerald vizenor (ojibwe) would call ‘survivance,’ ‘an active sense of presence, the continuance of native stories, not a mere reaction, or a survivable name. native survivance stories are renunciations of dominance, tragedy, and victimry. survivance means the right of succession or reversion of an estate, and in that sense, the estate of native survivancy’ (1999, preface). represent becomes a means of resisting stereotypes through the process of honoring and valuing the full humanity of native peoples. for native viewers, the represent series recognises and honors indigenous viewers’ own knowledge and experiences, provides aspirational models, and shows that transformations of age-old traditions are not merely a rejection of tribal identity or indigeneity, but a new strategy of empowerment, a means of indigenizing other spaces. a young man moonwalks like michael jackson in front of a natural history museum display; a teacher speaks ojibewemowin to teach math in an ojibwe school; a jingle dancer (without regalia) in jeans, with earbuds in, dances on the quad at dartmouth college; several young hip hop artists rap about indigenous resurgence and idle no more, surrounded by friends and family at a community centre in detroit; a young mother places her child in a cradleboard; an awardwinning recording artist sings of the riches of family while billy wilson beatboxes at her kitchen table; berkeley students who are native go about the mundane business of being college students; a nambe pueblo woman, then a student at yale university, throws candy in a ‘pueblo throw’ at young men passing by; three young women, students at fort lewis college in colorado, sing a ’49 about love and desire. uploaded in october 2012 the represent video featuring fort lewis college students sherri willeto, diné, (a sociology major/minor in native american studies); teahonna james, tlingit/athabaskan, (a humanities major); and mariana harvey, yakama, (an indigenous studies major) has received 182, 483 views, the most of all of the represent series (see figure 4). the video, as with all others in the series, is filmed with a fixed position camera, framing the women in a medium shot which fosters a sense of intimate familiarity; the camera filter casts the three women in golden light. figure 4: the 1491s, ‘represent-singing on campus-ft. lewis college’ australasian journal of information systems berglund 2017, vol 21, research on indigenous ict the youtube activism of the 1491s 12 the video opens with the three waiting to begin, smiling, and laughing. they sing a ’49, a secular popular musical form, typically sung in english about everyday subjects (with ‘vocable’ or non-linguistic, but traditional stylings such as “yah-he-he-yah”); ’49s are sung at pow-wows, at parties, when friends get together and are extemporaneous fusions of pop and traditional elements. the lyrics the three women sing are simple and repeated with each one taking turns singing the lead in a round: “i got this loving feeling/ ya-he-he-yah/ i got this loving feeling/ come a little closer/ i’ll hold onto you all night long/ hey-a-hi-hey-a-hi-yo.” their harmonious voices are striking and moving to listeners. they sing of love and desire, components of personhood and a full humanity, for too long absent from any representation of native peoples on film (albeit something that has been changing over the course of the last two decades). viewer chehalarose comments, beautiful to see my indigenous sisters out there representing our strong and resilient culture! makes me proud to be nehiyaw. and for anyone who has left negative comments, i feel sad for you because our roles as human beings, as women is not to tear each other down but rather it should be about empowering each other, supporting each other on our journeys. anyway hiy hiy for sharing 1491's! after another viewer critically questions the combination of vocable stylings in a love song about desire—“so i guess if u add a few way yah has in a country ass song its all of a sudden cultural....damn how our ways have changed smh :/” while derick gill, the previous commenter, may be smh [shaking my head], another viewer takes him to task and questions his expectations for native singers. bulldoggrrr writes, “lmao, what?...you expect them to play the flute on horseback. true, “our" ways have changed, but you are projecting about what you think a reflection of our culture is. have you ever been to a 49? i would guess not.” videos which are part of the represent series combat stereotypes and change common narratives about native peoples. often they celebrate the efforts to maintain and revitalise important tribal traditions, in other cases, they offer clear juxtapositions of unquestionable indigenous pride in the midst of supposedly non-native contexts. in this way, the videos push back against expectation—that native culture has been lost or died out, that native people are part of history, not the present, and that indigenous people lead separate existences outside the purview of mainstream culture. the videos remind non-native viewers, in particular, that indigenous people are everywhere and that their presence is vital and enriching. and this is the real activist effort of the 1491s, a project supported by their expansive national and international reach on youtube and the fanbase that responds and communicates with the performers and with one another about important political matters affecting the lives of indigenous people everywhere. as natanya ann pulley puts it so succinctly, “a native american presence online is always a political act. it is a testament to our staying power and to the people’s ability to surge onward…i no longer need to open a book or work through my mother’s memory of navajoland to reach my people (though i deeply miss its sand and growing winds). i simply have to open an app. and i am not faced with static images, but with emergent and pulsing paths of inclusion and action” (2014, p. 100). there’s no room for fixating on tragedy in this space. there’s too much to be done, and a little bit of laughter and appreciation of beauty courtesy of the 1491s will make the work easier to undertake and sustain. references berg, madeline. (2015) “the world’s top-earning youtube stars 2015”, forbes. available at http://www.forbes.com/sites/maddieberg/2015/10/14/the-worlds-highest-paidyoutube-stars-2015/#3e560a26542c. [accessed 5 march 2017]. berglund, j. (2016) “i’m just as indian standing before you with no feathers popping out of my head’: critiquing indigenous performativity in the youtube performances of the 1491s”, alternative: an international journal of indigenous peoples,12(5): 541-557. australasian journal of information systems berglund 2017, vol 21, research on indigenous ict the youtube activism of the 1491s 13 bullock, molly. (2015) “the making of the 1491s: native humor goes viral”, the tulsa voice. available at http://www.thetulsavoice.com/june-b-2015/the-making-of-the-1491s/ [accessed 5 march 2017]. burgess, jean and joshua green. (2009) youtube: online video and participatory culture. polity press, cambridge. carlson, bronwyn. (2013) “the new frontier: emergent indigenous identity and social media”, in michelle harris, martin nakata & bronwyn carlson (eds.), the politics of identity: emerging indigeneity. utsepress, sydney. haskie-gonzalez, janelle. (2014) “the 1491s bobby wilson’s visit to mcc september 24, 2014”, [video file]. available at https://www.youtube.com/watch?v=4gvaoogxffg [accessed 5 march 2017]. lincoln, kenneth. (1993) ind’n humour: bicultural plans in native america. oxford university press, london. o’carroll, acushla deanne. (2013) kanohi kit e kanohi – a thing of the past? an examination of māori use of social networking sites and the implication for māori culture and society, [doctoral dissertation], available at http://www.whariki.ac.nz/massey/fms/colleges/college%20of%20humanities%20an d%20social%20sciences/shore/reports/ocarroll%20a%20d%20%20phd%20thesis.pdf?07be5c6ebfe9aaff2aca54dd8ad295d4 [accessed 5 march 2017]. pulley, natanya ann. (2014) “indigetechs: the original time-space traveling native americans and our modern world hyper-pow-wow”, western humanities review 68 (2): 95-101. rader, dean. (2011) engaged resistance: american indian art, literature, and film from alcatraz to the nmai. university of texas press, austin. ross, gayasi. (2013) “don’t believe the hype: beyond poverty porn, pain and drunk indians”, indian country today. available at http://indiancountrytodaymedianetwork.com/2013/12/12/dont-believe-hype-beyondpoverty-porn-pain-and-drunk-indians-152673 [accessed 5 march 2017]. smith, linda tuhiwai. (1999) decolonizing methodologies: research and indigenous peoples. zed books, london. the 1491s. (2011) “bad indians”, [video file], available at https://www.youtube.com/watch?v=3fugdutdauq [accessed 5 march 2017]. the 1491s. (2011a) “slapping medicine man”, [video file]. available at https://www.youtube.com/watch?v=mvwlhmz-cee. [accessed 5 march 2017]. the 1491s. (2011b) “to the indigenous woman—a poem by the 1491s”, [video file]. available at https://www.youtube.com/watch?v=6fxp95eu8xs. [accessed 5 march 2017]. the 1491s. (2012) “ten little indians—violence against native women”, [video file]. available at https://www.youtube.com/watch?v=tjhqutcsfqu. [accessed 5 march 2017]. the 1491s. (2012a) “represent-singing on campus-ft. lewis college”, [video file]. available at https://www.youtube.com/watch?v=oblfcoxsmfw. [accessed 5 march 2017]. the 1491s. (2013) “indigenous love words project”, [video file]. available at https://www.youtube.com/watch?v=zvt4eba_ejm. [accessed 5 march 2017]. tedx talks. (2013) “1491s play with themselves, imagery, and reclamation: 1491s at tedxmanitoba 2013.” [video file]. available at https://www.youtube.com/watch?v=mqkvwrrgw_q. [accessed 5 march 2017]. australasian journal of information systems berglund 2017, vol 21, research on indigenous ict the youtube activism of the 1491s 14 vernallis, carol. (2013) unruly media: youtube, music videos, and the new digital cinema. oxford university press, new york. vizenor. gerald. (1999) manifest manners: narratives on postindian survivance. bison books, lincoln. wilson, alex. (2015) “idle no more round dance: revolution at the hemispheric institute of performance and politics montreal encuentro: a discussion with roewan crowe and alex wilson”, caribbean rasanblaj, 12(1). available at http://hemisphericinstitute.org/hemi/en/emisferica-121-caribbean-rasanblaj/crowe [accessed 5 march 2017]. copyright: © 2017 berglund. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems hassandoust, singh & williams 2020, vol 24, research article contextualisation, vulnerability and phishing 1 the role of contextualization in users’ vulnerability to phishing attempts farkhondeh hassandoust auckland university of technology new zealand ferry@aut.ac.nz harminder singh auckland university of technology new zealand jocelyn williams manukau institute of technology new zealand abstract hackers who engage in phishing manipulate their victims into revealing confidential information by exploiting their motives, habits, and cognitive biases. drawing on heuristicsystematic processing and the anchoring effect, this study examines how the contextualization of phishing messages, in the form of modifications to their framing and content, affects individuals’ susceptibility to phishing. this study also investigates if there is a discrepancy between the way individuals believe they will react to phishing attempts and their actual reactions. using two fake phishing campaigns and an online survey, we find that individuals are more susceptible to phishing attempts when the phishing messages they receive are specific to their context, thereby appealing to their psychological vulnerabilities. there is also a significant gap between how individuals believe they will react and their actual reactions to phishing attempts. keywords: word; information security, phishing, contextualization, heuristic-systematic processing 1 introduction with the spread of technology and the global reach of the internet, cybercriminal activities are occurring more frequently (nouh, nurse, webb, & goldsmith, 2019). for example, new zealand’s national computer emergency response team (cert nz) announced that the highest number of cybersecurity incidents ever were reported in the third quarter of 2019, with a 27% increase in reports of phishing and credential harvesting (certnz, 2019). phishing, a type of social engineering attack carried out by hackers who send spoof emails to trick individuals into giving up sensitive information, is a particular worry because it enables hackers to circumvent information security countermeasures. hackers who engage in phishing manipulate individuals into revealing their confidential information by exploiting their motives, habits and cognitive biases (mitnick & simon, 2003). although internet users have become more aware of them, phishing messages have also become more sophisticated, meaning that users need even more information security awareness so that they can remain secure and safe while on the internet. information security awareness programs provide users with knowledge of information security threats and australasian journal of information systems hassandoust, singh & williams 2020, vol 24, research article contextualisation, vulnerability and phishing 2 solutions to mitigate or avoid the impact of security attacks (hassandoust & techatassanasoontorn, 2020). despite the increased number of such programs, convincing individuals not to reveal their private information when they receive phishing messages and to improve their protective security practices overall remains a significant research challenge (hassandoust & techatassanasoontorn, 2020). although phishing has attracted a great deal of academic and practitioner attention, it remains a major it security problem. one reason for this is that phishing emails can trigger emotions that over-ride individuals’ training and awareness so that they comply with the disguised malicious request (goel, williams, & dincelli, 2017). these emotions include fear, curiosity, patriotism, friendship, authority, community, and belongingness. for example, phishing practitioners may scare potential victims into divulging their private credentials by implying that if they do not do so, they will lose something valuable (kim & kim, 2013). a common scenario is in the banking context: customers are sent contextualized1 phishing emails that are ostensibly from the bank they use. these phishing attacks leverage fear by suggesting that the recipient will not be able to access her/his bank account unless s/he changes their banking credentials or information by clicking on a given url. hence, it is important to consider the phishing attack context when interpreting recipients’ responses, because the context can elicit certain emotions. however, few studies have investigated the influence of contextualization in phishing or compared the impact of contextualized against non-contextualized phishing messages. while goel and his colleagues (2017) studied how the framing of a phishing message affects the susceptibility of message recipients to phishing attempts, less is understood about how this vulnerability interacts with receivers’ attributes. we build on that study in three ways. first, we expand its theoretical explanation to include the role of anchoring. second, we investigate if there is a discrepancy between how individuals believe they will react to phishing attempts and their actual reactions. third, we examine if individuals’ reactions to phishing are influenced by their prior experiences. we test these propositions by manipulating fraudulent email messages to appeal to two distinct motivations among tertiary students: receiving a free computing device or finding out their academic results. while this study investigates whether individuals respond differently to phishing messages with different types of content, it also examines whether individual responses differ by demographic attributes. understanding which demographic groups are more susceptible to information security threats will help in designing and delivering relevant education programs. for example, information security practices, as well as internet attitudes and phishing practices, are known to vary by gender (chou & sun, 2017; goel et al., 2017; wu, 2014). demographic factors also affect individuals’ perception of their privacy risk and the extent of information sharing on social sites (hajli & lin, 2016). research in the u.s. and korea suggests that information security practices vary across cultures and the position of men relative to women in the social hierarchy (hovav & d’arcy, 2012; turner & monk‐turner, 2007). for instance, in south korea, gender and age are associated with social status, meaning that older people and males retain a higher position in the social hierarchy. 1 contextualization may refer to adapting extent theories to account for relevant contextual conditions (whetten, 2009). however, in this paper, contextualization means adapting phishing instruments to fit recipients’ characteristics or background settings. australasian journal of information systems hassandoust, singh & williams 2020, vol 24, research article contextualisation, vulnerability and phishing 3 the next section describes the theoretical perspective of the heuristic-systematic model. following that, prior research on phishing, contextualization and information security awareness is discussed. our research model and hypotheses are then presented. this is followed by an explanation of the methodology, which comprised two phishing campaigns and an online questionnaire, and an analysis of the results. the paper concludes with a discussion of the findings and their implications for researchers and practitioners. the results shed light on factors that should be considered in promoting protective security practices and encouraging more prudent responses to phishing attempts. 2 theoretical background the dominant assumption in information security research is that individuals are rational actors who deliberately plan their behaviours in advance and are aware of the benefits and costs of their responses (dennis & minas, 2018). while this assumption may hold true in some information security contexts, it is less appropriate for situations where individuals react automatically, such as in phishing attacks. instead of using carefully deliberated actions, individuals’ responses to phishing emails are immediate, reactive, nonconscious and automatic (dennis & minas, 2018). the next section describes a dual process theory related to such behaviours the heuristic systematic model and explains how it can be used to understand individuals’ security behaviours as instant cognitive responses, instead of solely being the outcome of deliberate judgements. 2.1 the heuristic-systematic model chen and chaiken (1999, p. 74) argue that individuals apply a combination of heuristic (quick) and systematic (deliberate) processing models to make judgments: “heuristic processing entails the activation and application of judgmental rules and ‘heuristics’ that are presumed to be learned and stored in memory…. heuristic processing make[s] minimal cognitive demands on the perceiver” whereas “systematic processing entails a relatively analytic and comprehensive treatment of judgmentrelevant information…. given its nature, systematic processing requires both cognitive ability and capacity”. heuristic processing uses readily apparent cues embedded within a message, such as its source, format, and subject, to quickly generate an assessment of validity. on the other hand, systematic processing makes such an assessment by cautiously examining the information content in a message (luo, zhang, burd, & seazzu, 2013). when individuals use heuristic processing, they rely on judgmental and cognitive shortcuts that lead to quick decisions based on their immediate emotion/s; however, these decisions are subject to cognitive biases. on the other hand, individuals engaged in systematic processing inspect information attentively and deal with messages analytically (chaiken, 1987; s. chen & chaiken, 1999). this theory, referred to as the heuristic-systematic model (hsm), argues that since individuals tend to choose actions that require less effort, they use the heuristic processing mode more often than the systematic processing mode. information systems researchers have used the hsm to evaluate users’ reactions and behaviours in various contexts (hilligoss & rieh, 2008; wirth, böcking, karnowski, & von pape, 2007), including phishing (goel et al., 2017; luo et al., 2013). for example, luo and his colleagues (2013) adopted hsm to investigate the psychological mechanism underlying the effectiveness of phishing attacks. they argued australasian journal of information systems hassandoust, singh & williams 2020, vol 24, research article contextualisation, vulnerability and phishing 4 that when systematic processing occurs, individuals’ high-quality arguments lead to appropriate assessments of phishing messages. the question then arises as to when individuals move from one mode to the other. since individuals are essentially satisficers (simon, 1965), that is, they do not strive to make validity assessments with the highest accuracy or reliability, they stop processing when they think their assessments are good enough. the point at which they stop processing is referred to as the “sufficiency threshold” (luo et al., 2013). the “sufficiency threshold” refers to the acceptable level of confidence an individual has in her/his judgment (eagly & chaiken, 1993). individuals continue processing a message until they are confident that they have exceeded the sufficiency threshold. individuals using heuristic processing stop processing information once they reach their sufficiency threshold. if they do not reach it, they will continue their decision-making by using systematic processing until they reach their sufficiency threshold. according to hsm, individuals adjust their sufficiency threshold based on contextual factors. these include the importance of the decision, the involvement of purported authority figures (e.g., school administrators and bank officials), social pressures, cognitive resources, their own skill levels, and time pressures. the presence of any of these contextual factors stresses a need for urgent action. they also act as cues that direct individuals towards the peripheral route of persuasion, instead of the central route, so that they do not focus on the content of the information being presented to them (petty & cacioppo, 1984). this, in turn, leads individuals to lower their sufficiency threshold so that they use only heuristics processing to make decisions (chaiken, 1982). in the phishing scenario, attackers have a higher chance of convincing recipients of phishing messages to open them if they can reduce the recipients’ sufficiency threshold. in this way, recipients do not shift to systematic processing when they receive a phishing message, using heuristic processing to quickly and inaccurately judge its validity (luo et al., 2013). one way of lowering the sufficiency threshold of recipients is to contextualize the message to quickly trigger their motivational concerns in a context that seems important and inhibits them from inspecting the message. for example, registering for courses is important for college students, so receiving a message about course registration is likely to create a sense of urgency that needs quick action (goel et al., 2017). in some instances, heuristic and systematic processing modes can happen simultaneously (luo et al., 2013). these conditions are when: 1) heuristic and systematic processing cause the same decision, and confidence in that decision would be higher than either process alone (i.e., the modes reinforce each other, thereby strengthening the decision); 2) heuristic processing may create an initial conclusion that biases the nature and scope of the systematic processing mode (i.e., biased short-cut actions); and 3) systematic processing may generate conclusions that overturn or limit those of heuristic processing mode (i.e., attenuation processing). 3 literature review 3.1 phishing online social engineering attacks include phishing e-mails that spoof legitimate institutions in order to defraud users, and others that aim to commit advance-fee fraud. e-mail spoofing (phishing), the focus of our study, targets individual users as well as potentially bypassing the best technical security systems to gain access to an organization’s critical information. one of the most well-known deceptive techniques in online communications (certnz, 2019), email phishing involves credible-looking emails that are sent to individuals to fool them into australasian journal of information systems hassandoust, singh & williams 2020, vol 24, research article contextualisation, vulnerability and phishing 5 providing their sensitive personal information. specific forms include spear phishing malicious emails sent to a specific person, and whaling phishing attacks targeting senior executives (irwin, 2020; pienta, thatcher, & johnston, 2020). previous phishing studies have mostly focused on two areas: a) how individual susceptibility to phishing varies by individual attributes, such as cognitive limitations, personality traits, identity, and demographics; and b) how interventions such as training can decrease individuals’ susceptibility to phishing (goel et al., 2017). researchers have found that individuals can usually protect themselves when they are exposed to social engineering attacks online because they know that internet use has some inherent risk and they are familiar with how to manage such risks (downs, holbrook, & cranor, 2006). however, they are less able to defend themselves when exposed to sophisticated deception or contextualized messages. these findings indicate that messages limiting individuals to heuristic processing would increase their susceptibility to phishing scams (downs et al., 2006). most previous studies were based on theories which assumed that individuals consider their security intentions and behaviours deliberately. however, individuals confronted with phishing attacks are unlikely to follow that conceptualization and think carefully about the issue facing them (dennis & minas, 2018). phishing messages include at least some false content that can usually be recognized with some amount of systematic processing. to improve the success rate of phishing attacks, hackers mislead their targeted victims into making a quick but incorrect validity assessment of the message. this demonstrates the link between heuristic processing modes and the nonconscious automatic cognition found in phishing victimization (dennis & minas, 2018; luo et al., 2013). the heuristic processing mode depends on available heuristic cues such as source credibility and catchy message content. for example, most internet users are familiar with amazon, as it is a reputable company, and they may have already given amazon their personal information when they transact with it. so, if they receive a phishing email that claims to be from amazon and asks for their personal information, they would most probably provide the information willingly (downs et al., 2006). halevi and her colleagues (2015) found that 63% of employees who received an email addressed to them individually that was purportedly from their company’s it manager would click on a link embedded in that email, as they judged it to come from a reliable source. another study found that use of persuasion principles in the design of phishing emails was highly effective through increasing the susceptibility of recipients to phishing (wright, jensen, thatcher, dinger, & marett, 2014). successful phishing attacks take advantage of the human willingness to make intuitive judgments based on initial impressions of the context. although phishing messages include false information that an individual may notice with careful scrutiny, a well-designed phishing message activates motivations that push victims toward accepting the message (goel et al., 2017). however countermeasures including learning designs based on learning science principles (kumaraguru et al., 2007; sheng, holbrook, kumaraguru, cranor, & downs, 2010) and games (arachchilage & love, 2013), as well as browser add-ons and toolbars (wu, miller, & garfinkel, 2006) can be used to reduce the vulnerability of individuals to phishing. the research discussed above on individual responses to phishing and phishing countermeasures indicates a need to explore how individuals make judgments about messages they receive. studies on the behavioural aspects of phishing try to ascertain why phishing works and how it can be detected and prevented. this has been done by investigating the impact of email australasian journal of information systems hassandoust, singh & williams 2020, vol 24, research article contextualisation, vulnerability and phishing 6 recipients’ individual attributes, and the attributes of the email itself (e.g., anderson, vance, kirwan, jenkins, & eargle, 2016; arachchilage, love, & beznosov, 2016; wang, li, & rao, 2017; wright et al., 2014). for example, moody, galletta, and dunn (2017) used the delphi method to determine the effect of personality and situational factors on individuals’ susceptibility to phishing attacks. most studies in this domain use theories such as protective motivation theory, routine activity theory and technology threat avoidance theory. the default assumption here is that individuals are engaged in systematic, theoretical reflection (arachchilage & love, 2014; blythe, petrie, & clark, 2011; jansen & van schaik, 2018; mcelwee, murphy, & shelton, 2018). by using the lens of these theories, the previous studies empirically tested and tracked subjects’ actual clicking behaviour. the challenge here is that responses to security threats do not take place after systematic reflection. while these theories can provide a framework for interpreting the cognition and behaviours of phishing email recipients, they cannot fully predict and interpret users’ automatic cognition based on their reactive and immediate emotions, which are arguably a better fit with responses to phishing. therefore, theories such as hsm are needed to evaluate individuals’ immediate responses to phishing attacks. the table in appendix a summarizes the objectives, theoretical background, and findings of the literature related to this domain. 3.2 information security awareness while it has been suggested that research on individuals’ security awareness is needed, most previous studies have focused on organizational security policies to deter information security threats (d'arcy, hovav, & galletta, 2009). this is because providing effective information security awareness is the most cost-effective way to encourage users to adopt more protective (hanus & wu, 2016) and preventive strategies (x. chen, chen, & wu, 2018; dinev & hu, 2007). prior research has suggested utilizing individuals’ information security awareness to examine their intentions in relation to information security protective practices (hanus & wu, 2016). from an hsm perspective, information security awareness can increase individuals’ sufficiency threshold level, meaning that individuals would move beyond heuristic quick processing to systematic deliberate processing when making judgments about potential information security threats. in addition, activities to enhance individual alertness, such as training programs, are an effective way of decreasing heuristic-based judgments. individuals’ lack of awareness is considered to be the main predictor of their vulnerability to security threats, by, for example, engaging in risky security behaviour (haeussinger & kranz, 2017; siponen, 2000). although the significance of individuals’ phishing awareness has been mostly recognized, recent research indicates that awareness remains a substantial topic since most users lack awareness of phishing and how it occurs (goel et al., 2017; lim, ahmad, chang, & maynard, 2010). as a result, it may be inferred that an increase in individuals’ awareness of information security or phishing would increase their sufficiency level, so that they will be more likely to use systematic processing to make decisions about information security threats. this would minimize the likelihood of risky information security behaviour and enhance the efficacy of protective techniques and countermeasures in their workplaces and personal lives (haeussinger & kranz, 2017). 3.3 individual differences in susceptibility to phishing individuals’ sufficiency threshold is also affected by their personal attributes such as age, gender, and personality, which consequently have an impact on their heuristic and systematic australasian journal of information systems hassandoust, singh & williams 2020, vol 24, research article contextualisation, vulnerability and phishing 7 decision-making processes. for example, users who score highly in agreeableness and conscientiousness are more inclined to use heuristics to make quick judgments (eroglu & croxton, 2010). the experiences, current contextualization, personality, culture and demographic characteristics of individuals have a profound impact on their behaviour (dennis & minas, 2018). demographic characteristics (e.g., gender, age, education level and technical skills) also influence phishing susceptibility (flores, holm, nohlberg, & ekstedt, 2015; halevi, lewis, & memon, 2013; halevi et al., 2015; jagatic, johnson, jakobsson, & menczer, 2007; sheng et al., 2010). demographic aspects include characteristics such as gender and social identities inherited from ancestors, such as ethnicity (dennis & minas, 2018). females and males differ significantly in relation to their information security privacy concerns and the sharing of personal information on social networking sites (chou & sun, 2017; hajli & lin, 2016). gender may also be related to differences in individuals’ perceptions and use of the internet (wu, 2014). males have been found to be less concerned about unethical computer practices, such as using unlicensed programs (beycioglu, 2009), and more likely to correctly recognize phishing and legitimate websites than females (kumaraguru, sheng, acquisti, cranor, & hong, 2010). similarly, goel and his colleagues (2017) found that female students are more likely than male students to open a phishing message but not necessarily click on the malicious links. the higher susceptibility of females to phishing may be due to their lack of technical experience (sheng et al., 2010). however, pattinson et al (2015) did not find significant correlations between individuals’ demographic characteristics (age and gender) and phishing behaviours. they did find a significant positive relationship between age and security behaviour, indicating that older adults show safer behaviour. flores and his colleagues (flores et al., 2015) investigated individuals’ personal and cultural determinants of phishing. they found phishing behaviour differs across indian, swedish and american employees from nine different firms. 4 research model and hypothesis development the preceding sections described how phishing researchers have explored the impact of the contextualization of phishing emails, and individual differences on individual susceptibility to phishing. this paper uses a heuristic perspective to explain those findings, as well as explain the role of security awareness training as a possible preventive mechanism. in this section, we describe the hypotheses we will use to examine the validity of our theorizing. 4.1 hypothesis 1: trustworthiness as an anchor contextualized phishing attacks make individuals more likely to engage in heuristic processing and repress their systematic processing, leading to successful phishing attacks. previous researchers argued that contextualizing a message to quickly trigger individuals’ motivational concerns in a context specific to them would reduce their sufficiency threshold, encourage the use of heuristic processing, and avoid careful scrutiny of the message (luo et al., 2013). for example, an email from a university support centre or administration office that is considered a trustworthy source can create a sense of urgency for quick action. students who receive such messages may make insufficient adjustments and judgments based on the parameter of the reliable source initially presented. if the email sender is known to be a reliable source, heuristic processing will occur, and judgments will be made quickly. we postulate that reliable sources act as anchors that affect individual decision-making. the anchoring effect is a cognitive bias that explains why individuals’ behaviour and judgments australasian journal of information systems hassandoust, singh & williams 2020, vol 24, research article contextualisation, vulnerability and phishing 8 are impacted by the initial value, perspective or impression they receive (epley & gilovich, 2006). for example, jacowitz and kahneman (1995) found that participants gave higher estimates of the length of the mississippi river after they first considered whether it is shorter or longer than 5000 miles than when they considered whether it is shorter or longer than 200 miles. the judgement influenced by an anchor can also be based on non-numeric values and information (cohen & reed, 2006; esch, schmitt, redler, & langner, 2009). for example, iuga et al. (2016) examined whether the first three webpages that individuals saw during a phishing experiment influenced the zones that their participants inspected on the fourth webpage they saw. in a decision-making process, anchoring effects are stronger when the source of the initial value is more ambiguous, less familiar, or more trustworthy (furnham & boo, 2011; iuga, nurse, & erola, 2016). for example, a message from a trustworthy source such as a school administrator presents a strong anchor, which lowers the individual’s sufficiency threshold. therefore, individuals do not proceed to systematic, deliberate processing when making their decision; instead, they use quick heuristic adjustments from the information provided by that source. thus, in the context of phishing, drawing on hsm and the anchoring effect, we infer that messages from reliable email senders, which are perceived to be strong anchors, will lead to greater heuristic processing and judgments will be made quickly. on the other hand, when initial values or parameters are too extreme, individuals ignore the values or question their validity, leading to a smaller anchoring effect being generated (wegener, petty, detweilerbedell, & jarvis, 2001). for example, a message which tells the recipient they have won an expensive computing device may seem too good to be true. thus, its anchoring effect will be minor, and the recipient will engage in less heuristic processing. therefore, we hypothesize: h1: contextualized email messages from strong anchors increase the vulnerability of recipients to phishing compared to non-contextualized email messages. 4.2 hypothesis 2: information security awareness another factor that is likely to influence the relative extent of heuristic versus systematic processing in the context of phishing is the individual’s awareness of information security issues. these include phishing and the countermeasures that could be performed against phishing. individuals with higher levels of threat awareness can enhance their protective reactions and minimize their risky responses to information security threats (hanus & wu, 2016). knowledgeable users with high expertise are less affected and less uncertain in making relevant decisions (wilson, houston, etling, & brekke, 1996). heuristic processing as a quick procedure mostly relies on judgmental rules and cognitive shortcuts that would likely cause risky reactions. on the other hand, systematic processing, where individuals carefully scrutinize and analyse information, is likely to lead to protective reactions. being aware of phishing and countermeasures to phishing would likely increase the sufficiency threshold of individuals, leading them to bypass the quick heuristic judgment process and use systematic processing. at this stage, individuals would deliberately analyse phishing messages, which is likely to improve their protective security reactions. activities that enhance individual alertness to information security attacks, such as training programs, are an effective way to increase individuals’ knowledge, which leads to fewer quick heuristic judgments and more systematic deliberate decision-making. thus, we hypothesize: australasian journal of information systems hassandoust, singh & williams 2020, vol 24, research article contextualisation, vulnerability and phishing 9 h2a: users’ phishing awareness positively influences their perceived protective practices. h2b: users’ phishing awareness positively influences their phishing reactions. 4.3 hypothesis 3: prior experiences demographic characteristics and personality traits are associated with an individual’s prior experience with information security threats. drawing on the big five model (personality traits are assumed to represent the basic structure behind all personality traits), if a person high in agreeableness had fallen victim to cybercrime previously, it is likely that heuristics associated with the prior bad experience would dominate the person’s fundamental personality trait, when s/he is in a context that triggers cognitive heuristics based on that experience (dennis & minas, 2018). individuals tend to use heuristics that are often motivated by proximity and vividness (loewenstein, o'donoghue, & bhatia, 2015). thus, prior bad experiences that are rather vivid, such as identity theft, are expected to outweigh personality traits, such as agreeableness (dennis & minas, 2018). moreover, avoidance responses overcome appetitive responses (to reach for a presented stimulus), because humans have evolved to avoid danger (kahneman, 2011). for example, an appetitive response can be compliance with a request, whether the request is to comply with an organization’s security policies or react to a phishing message. hence, based on an avoidance heuristic, a prior bad experience would often replace a set of good experiences. if the prior experience happened recently, then the associated avoidance heuristic with it would be even stronger. therefore, we theorize the following proposition: h3: users with prior phishing experiences are less likely to fall for a phishing attack. 5 research design although phishing experiments arguably contravene informed consent requirements and involve deception, they can be conducted ethically if risks are minimized, privacy and confidentiality are protected, potential participants have an opportunity to opt in/out to the research before it begins, and subjects are debriefed after their participation ends (resnik & finn, 2018). to test the hypotheses, simulated phishing emails were sent to a sample of students in a tertiary institution. the participants in this study were undergraduate and postgraduate students enrolled at a higher education institution in new zealand. in this study, we used an education setting, as students are a frequent target of online attacks, and they fit in the age bracket of those most susceptible to phishing threats (johnston & warkentin, 2010) and are thus a suitable group for applied behavioural research on information security. all the students at the educational institution were sent an email that included a brief explanation about this information security research project, along with an information sheet and consent form. key aspects such as the opt-out option and completely voluntary nature of participation have been clearly explained to the students. those who wished to participate in the study signed the consent form and sent it back (through email or personal hand over) to the researchers. individuals who did not want to participate in the research were excluded from the study. consenting participants were sent simulated phishing emails to evaluate how they would respond to them. these were designed to test the effects of contextualization in two ways. first, we varied the content of the message so that it could be relevant specifically to tertiary students (e.g., obtaining their course results) or to anyone (e.g., winning an ipad mini). australasian journal of information systems hassandoust, singh & williams 2020, vol 24, research article contextualisation, vulnerability and phishing 10 second, we characterized the message sender as being from within the college/university (e.g., student support manager) or outside the college/university (e.g., the apple research team). an apple-related email was used as a general context because some students had used “apple educational benefits/pricing” as a case study for a course before this research project, indicating that this topic interested them at that time. we used gophish, an open-source phishing framework to run the study. for the simulated phishing email, the sender’s name appeared as ‘college admin’; however, the email address did not have a university domain, but instead was a personal gmail address. in addition, there was a typo in the sender’s email address (***addmin@gmail.com). the simulated phishing email also contained three misspellings such as ‘attendence’, a url that was redirected to another page, and the wrong domain for the sender’s email address was ‘@***.com’ instead of ‘@***.ac.nz’. upon completion of the experiment, students received a debriefing email informing them about the purpose of study, how phishing works and advice about how to avoid phishing attempts. debriefing in a phishing experiment has the potential to deliver a long-lasting positive impact, if it is structured in such a way that the subject learns how to avoid being a victim of future real phishing attacks (jakobsson & ratkiewicz, 2006). the phishing responses were coded 1/2/3/4, where 1 represents that respondents did not open the email, 2 that respondents opened the email only, 3 that respondents opened the email and clicked the link, and 4 that respondents opened, clicked the link and submitted data. once their responses to the phishing attempt had been recorded, an online survey was conducted in order to capture their perceptions on phishing messages. the survey measurement items had been validated in previous studies, as shown in appendix b (bailey, mitchell, robert, & bradley, 2008; dinev & hu, 2007; stanciu & tinca, 2016; wang, li, & rao, 2016). in the first section of the survey, participants were asked to provide their demographic information and in the second section, to indicate their awareness of and familiarity with (scale 1 to 9) security threats such as phishing. individuals’ overall prior experience of online security threats was assessed using six items, using yes/no responses for each. the ratings for each respondent on each item sum were summed to provide an overall score. in a third section, participants were asked to rate their perception of their information security practices on a five-point likert scale. for perceived protective practices, the ratings for each respondent on each scenario were averaged to provide an overall score. the survey questionnaire was refined in two stages: a pre-test and a pilot study (straub, boudreau, & gefen, 2004). the pilot study had 35 respondents and the findings indicated that there were no major difficulties in understanding the questionnaire items and instructions. the data collection process was conducted over five weeks during july-august 2018. in total, we sent the survey to 550 students and received 293 responses. after removing incomplete responses, 269 valid responses remained. 6 data analysis and findings we used ibm spss for correlation and chi-square analysis to test the proposed hypotheses. the chi-square statistic is a non-parametric technique for analysing group differences when the dependent variable is measured at a categorical level (mchugh, 2013). the demographic attributes of participants that were collected included their gender, age, level of study, and ethnicity. analysis showed that out of the 269 student respondents, 50.3% were female and australasian journal of information systems hassandoust, singh & williams 2020, vol 24, research article contextualisation, vulnerability and phishing 11 47.6% male. most respondents (66%) were aged between 24 and 32 years old; most were studying for undergraduate degrees (69.9%), and 30.1% for postgraduate qualifications. over 71% of the participants were asian international students. the findings revealed vast differences between message conditions in the rates at which the emails were opened, whether the links in the emails were clicked on, and submit data rates (table 1). for example, 71.3% (63.9% + 7.4%, as shown in table 1) of those who received the “course-related” phishing email engaged in risky behaviour2 by either clicking on the malicious link or submitting their credentials (e.g., college username and password), compared to only 5.9% for those who received the “winning ipad” phishing email. similarly, only 28.6% of the respondents in the “course-related” group showed safe behaviour through either not opening the email (9.3%) or simply opening the email without clicking on the link and then submitting confidential data (19.3%) compared to 94.2% of those in the “winning ipad” group. these results suggest that highly contextualized emails capture recipients’ attention, supporting hypothesis 1. this hypothesis was tested with a chi-square test, which revealed a significant association between the contextualization of a phishing email message and the recipients’ vulnerability to phishing (χ2 (3) = 269.00, p = .000). type of phishing email3 safe behaviors risky behaviors % did not open % open % clicked % submitted data course-related 9.3% 19.3% 7.4% 63.9% winning ipad 67% 27.2% 2.9% 3% table 1. proportion of respondents who opened the phishing email, clicked on the link and submitted data to test hypotheses 2a and 2b, we examined the pearson product-moment correlation coefficient between users’ phishing awareness, their perceived protective practices and their actual phishing reactions. table 2 below indicates a positive significant relationship (0.51**) between phishing awareness and perceived protective practices (p<0.01, one-tailed), supporting hypothesis 2a. these results suggest that greater phishing awareness is associated with a higher perception that the appropriate phishing security practices will be carried out. the results also indicate inverse relationships between users’ phishing awareness and their actual phishing reactions (-0.76***, -0.13*), which supports hypothesis 2b. this finding suggests that individuals with higher levels of phishing awareness are less likely to engage in risky reactions, such as clicking on either type of phishing email. 2 in this study, “did not open” and “open” the emails (as no further action followed) were considered as safe behaviors, while clicking on the malicious link and submitting the data were considered as risky behaviors. 3 in this study, the “did not open” group refers to those participants who did not open the email; the “open” group refers to those who opened the email but neither clicked on the link nor submitted their data”; the “clicked on the link” group refers to those who opened the email and clicked on the link but did not submit their data; and the “submitted data” refers to those participants who opened the email, clicked on the link and submitted their confidential information. australasian journal of information systems hassandoust, singh & williams 2020, vol 24, research article contextualisation, vulnerability and phishing 12 phishing awareness perceived protective practices course-related phishing reaction winning ipad phishing reaction phishing awareness 1 perceived protective practices 0.51** 1 course-related phishing reaction -0.76** -0.43** 1 winning ipad phishing reaction -0.13* -0.10* 0.14** 1 note: ** correlation is significant at the 0.01 level, * correlation is significant at the 0.05 level table 2. correlations between phishing awareness, perceived and actual practice constructs to test hypothesis 3, chi-square analysis revealed a significant association between users’ prior experience and their perceived protective practices as well as their contextualized phishing reactions. in our study, users with prior experience of security threats, such as phishing, used more protective phishing practices (χ2 (12) = 807.0, p = .000) and showed protective reactions (χ2 (68) = 147.78, p = .000) by neither clicking on the link nor submitting their credential information through simulated phishing attacks. we also examined the correlation between users’ prior experience and their perceived and actual protective practices. table 3 below indicates a positive significant relationship (0.41**) between prior experience and perceived protective practices (p<0.01). prior experience perceived protective practices course-related phishing reaction winning ipad phishing reaction prior experience 1 perceived protective practices 0.41** 1 course-related phishing reaction -0.97** -0.43** 1 winning ipad phishing reaction -0.15* -0.10* 0.14** 1 note: ** correlation is significant at the 0.01 level, * correlation is significant at the 0.05 level table 3. correlations between prior experience, perceived and actual practice constructs these results suggest that prior experiences with phishing or other security threats are associated with a stronger perception that the individual would use appropriate phishing security practices. the results also indicate an inverse relationship between users’ prior experience and their actual phishing reactions (-0.97***, -0.15*). this finding suggests that individuals with more prior experience of phishing or other security threats are less likely to australasian journal of information systems hassandoust, singh & williams 2020, vol 24, research article contextualisation, vulnerability and phishing 13 engage in risky reactions, such as clicking on either type of phishing emails. users identified prior experiences with security threats that included examples such as a cyber-attack from visiting a website, important personal information such as a social security number being stolen, or being the victim of an online scam and losing money. 6.1 post-hoc analysis we also tested users’ phishing awareness, perceived protective practices and actual phishing reactions by gender (table 4). first, we compared participants’ phishing awareness based on their gender. the result was statistically significant with p< 0.005. therefore, we conclude that phishing awareness among female students is significantly lower than male students. n mean std. deviation std. error mean t df sig. (2-tailed) gender4 female 135 3.30 2.10 .18 45.89 268 .000 male 128 3.45 2.05 .18 total sample 269 3.35 2.06 .13 26.60 268 .000 table 4. respondents’ awareness of phishing next, we examined participants’ perceptions of how they would react to phishing attempts. this was captured in five questions in the follow-up survey on a 5-point likert scale (‘1=never’, ‘2=rarely’ ‘3=sometimes’, ‘4=often’ and ‘5=always’). participants who ‘never’ or ‘rarely’ engaged in risky security behaviours were considered to use “protective practices”, while those who ‘sometimes’, ‘often’ or ‘always’ engaged in risky behaviours were considered to engage in “risky practices”. participants who engage in risky practices are at risk of being hacked, while participants who engage in protective practices are more likely to recognize phishing threats and not fall victim to phishing attempts. table 5 presents our findings based on the mean for each gender (1 to 5) and the probability of perceived protective practices. the protective practices percentages indicate the percentage of participants (both male and female) who claimed they would engage in protective practices. therefore, a higher percentage represents lower risk to the users as well as organizations that the participants engage with. the findings reveal that male participants believed they would engage in more protective practices than female participants, indicating that the latter group are more vulnerable to phishing. 4 female and male respondents made up 97.9% of the sample, with around 2.1% of respondents preferring not to reveal their gender. australasian journal of information systems hassandoust, singh & williams 2020, vol 24, research article contextualisation, vulnerability and phishing 14 context of messages mean t gender (mean) %perceived protective practice female male if you received an email containing the logo and web address of your bank or one of your credit card companies requesting that you should verify information such as your date of birth, account number, address, etc., and the email was addressed to you personally – would you click on the link and provide the requested information? 2.43 29.57 *** 2.49 2.37 79.1% if you received an email containing the logo and web address of your bank or one of your credit card companies requesting that you should verify information such as your date of birth, account number, address, etc., and the email was addressed to “dear customer” – would you click on the link and provide the requested information? 2.69 33.80 *** 2.72 2.67 78.4% would you fill out an email form asking for personal financial information if the email appeared to be from a trusted site and was addressed to you personally? 2.34 28.87 *** 2.34 2.38 75.5% if you received an email containing the logo and web address of your college/university requesting that you should verify information such as your name, student id, date of birth, address, etc., and the email was addressed to you personally –would you click on the link and provide the requested information? 2.60 32.65 *** 2.67 2.55 65.1% i click on email links or posts with touching winning messages such as winning an apple ipad. 1.92 29.42 *** 1.94 1.87 81.4% table 5. participants’ perception of their protective practices third, to examine users’ actual phishing reactions by gender, we tested the proportion of males and females who opened, clicked on the link, and submitted data for both message conditions. we found a principal effect for gender: collapsing across both message conditions, females were more likely to open the email message, click on the malicious link and submit their credential data than males. we found significant gender differences in the rates at which the ipad mini and course registration emails were opened. details are presented in table 6 with a chi-square test, which revealed a significant difference between female and male phishing reactions χ2 (9) = 289.48, p = .000). australasian journal of information systems hassandoust, singh & williams 2020, vol 24, research article contextualisation, vulnerability and phishing 15 context of phishing email mean t % did not open % open % clicked % submitted data female course related 3.30 37.70*** 3.7% 10.4% 3.7% 33.5% winning ipad 1.48 23.21*** 31.6% 14.9% 1.9% 1.9% male course related 3.18 31.65*** 5.6% 8.9% 3.7% 28.3% winning ipad 1.35 25.09*** 33.5% 12.3% 1.1% 0.7% table 6. breakdown of the phishing results based on gender regarding other individual differences, we also found a significant age difference in the frequency of clicking on the malicious link and submitting sensitive information (χ2 (18) = 68.33, p = .000). the results of the chi-square test presented a main effect of individuals’ technical skills on the frequency of engaging in risky practices, i.e. clicking on the link and submitting data (χ2 (9) = 86.90, p = .000). the chi-square test also revealed a significant main effect of individuals’ study level on the frequency of clicking on the link and submitting the confidential information (χ2 (12) = 87.50, p = .000). appendix d presents the frequency of opening, link clicks and submitting data by study level. 7 discussion in this study, we investigated the impact of the content and framing of phishing attempts on users’ vulnerability, and of users’ phishing awareness on their perceived protective practices as well as on their actual phishing reactions by analysing the difference between users’ perceived protective practices and their actual phishing reactions. in support of hypothesis 1, and in line with goel and colleagues (2017), users in the present study were more susceptible to a highly contextualized message that appeared to pertain to their course results than to another generic message about winning an ipad mini. almost three-quarters (71.3%) of our participants who opened the course-related message then clicked on the simulated phishing link, while 63.9% of them submitted their credential username and password. therefore, the contextualized message channelled participants through the first two steps of the phishing process and led them to read the message, click on the embedded link and submit the data, actions which are consistent with the heuristic-systematic model (eagly & chaiken, 1993). in the courserelated message, cues such as an important college matter and a credible sender as a reliable anchor led individuals to act without carefully considering the outcome of their actions. the highly contextualized email may have deterred the initial scrutiny of the users’ systematic processing system. existing research suggests that the most effective phishing messages function on both the heuristic and systematic processing systems (goel et al., 2017; luo et al., 2013). drawing on anchoring effect, for our student participants, the importance of checking final course results may have lowered their sufficiency threshold and pushed them toward quick action, despite some spelling mistakes and the use of an incorrect college domain name. hypothesis 2a and 2b states that students’ phishing threat awareness has a positive impact on their perceived protective practices and reactions. the results suggest that there is a significant positive relationship between students’ phishing awareness level and their protective practices as well as an inverse relationship between users’ awareness and their phishing reactions. accordingly, the need for effective security awareness, robust solutions and training australasian journal of information systems hassandoust, singh & williams 2020, vol 24, research article contextualisation, vulnerability and phishing 16 in regard to information security in order to improve users’ protective behaviours is dramatically proved (y. chen, ramamurthy, & wen, 2015; stanciu & tinca, 2016; g. white, ekin, & visinescu, 2017). thus, students must be trained in regard to this important issue. we infer that information security training is likely to result in more employable graduates who are better prepared for contemporary professional practice. users who are aware of threats are also more engaged in protective practices (hanus & wu, 2016). therefore, effective security training and education programs should take a systematic approach by making sure they address the multiple dimensions of security awareness (g. l. white, 2015). effective information security awareness and training programs should lead to improvement in graduates’ ability to exercise protective reactions. training would improve users’ awareness and increase their sufficiency threshold, which would equip them to move beyond the heuristic mode and better engage in systematic processes. awareness may lead knowledgeable users to more deliberate judgements, decisions and protective reactions. additionally, it appears that such programs should also pay more attention to educating users about the risks caused by security threats. hypothesis 3 posited that users’ prior security (i.e., phishing) experience or having been a victim of cybercrime play a significant role in users’ phishing susceptibility. based on our findings, users who had prior experience with security threats would engage more in safe practices and reactions. prior experience may likely increase users’ sufficiency threshold that they would shift from heuristic processing to more deliberate mode before any reaction. regarding other individual differences, based on our findings, younger students (e.g., between 18 to 20 years old) presented riskier reactions than older students (above 35 years old). users with expert skills (e.g., the use of programming languages to develop applications) and advanced technical skills (e.g., the ability to manage and configure applications) demonstrated a safer phishing reaction than users with basic or intermediate technical skills. in addition, our findings show undergraduate students were more likely to click on a malicious link and submit their data than postgraduate students. the results of the present study also show that students’ vulnerabilities and practices differ according to gender, in line with previous studies suggesting female users are more susceptible to phishing than male users. previous studies found that before training, female users were more likely to click on the simulated phishing links and enter their information, which was mostly due to lack of technical experience and skills (flores et al., 2015; sheng et al., 2010). therefore, this variation may be bridged by providing customized training in order to improve security awareness and protective reactions among female users. according to hsm, less expertise and higher internet anxiety would decrease the sufficiency threshold; thus, female users may make their judgements in the heuristic quick mode before reaching the systematic deliberate process. although we can make no assumptions about why these gender differences exist, any information security intervention strategy should consider them. beyond findings from goel and colleagues (2017), the findings of this study suggest that there is a large discrepancy between students’ perceived protective practices and their actual reactions to contextualized phishing messages. researchers have previously reported a conflict between users’ beliefs about information security and their information security reactions. for example, users believe that they can protect their computers from hackers, despite their unfamiliarity with different types of security threats such as phishing, and countermeasures (e.g., installing a firewall) (stanciu & tinca, 2016). in addition, our findings australasian journal of information systems hassandoust, singh & williams 2020, vol 24, research article contextualisation, vulnerability and phishing 17 revealed that students were less susceptible to the phishing message geared toward free goods (an ipad mini) than for course-related messages. this implies that the deception of offering free goods does not lower the sufficiency threshold in students or cause them to pass the heuristic processing mode to reach the systematic processing system and analyse the risks associated with phishing emails. on the other hand, students reported high confidence in their perceived protective practices which conflicted with their phishing reactions in the courserelated context. therefore, relying on technical skills and high self-confidence in detecting phishing messages would increase users’ vulnerability to phishing attacks. drawing on hsm, users’ self-confidence along with the deception of course-related messages from reliable sources would lower their sufficiency threshold that lead them to make quick uncertain judgments in the heuristic mode. 8 theoretical and practical contributions this study provides important theoretical contributions to the information security domain. it has been recommended to develop and test security models with a principal focus on the heuristic, automatic cognition worldview compared to the deliberate thoughtful process of security decision making and behaviours, to fill the gap in the current understanding of information security (dennis & minas, 2018). accordingly, the present study investigated users’ security reactions from the heuristic processing view to capture their unconscious phishing reactions. it establishes that contextual factors modify the effectiveness of phishing attempts. heuristics would likely reduce rational logic processes and increase vulnerability to fraudulent messages. most previous studies applied theories that may explain the cognition and behaviours of individuals but cannot fully predict and interpret users’ automatic cognition based on their reactive and immediate emotions. therefore, theories that focus on users’ automatic, heuristic cognitions such as hsm are needed to understand their immediate reactions against phishing attacks. the findings of this study offer support to the hsm. a contextualized phishing message from a reliable source may likely prompt individuals to act quickly without carefully considering the possible consequences of the action. contextualized social engineering threats such as emails to students related to their academic results, may lead them to overlook cues of deception that they might normally catch in non-contextualized messages. although initial suspicion causes recipients to process a message more systematically, the contextualized message from a strong, trustworthy source would likely convince them that the message is legitimate. our survey results support this assertion by showing that respondents who clicked on the embedded link were suspicious of the email but clicked on the link nonetheless. perhaps the emotion elicited by the message lowered the user’s sufficiency threshold, thus influencing their appraisal of or their response to the legitimacy of the message. this study suggests that hsm is a potentially worthwhile theoretical foundation for studying phishing and related cybercrime reactions. the implications for practice call for thoughtful changes. in order to improve security compliance behaviour, users’ experience-based heuristics can be changed. to reduce their vulnerability to phishing, the appetitive response approach should be replaced with heuristics that generate an avoidance response to the phishing messages, which can be referred to as aversion training. for example, organizations can send a series of phishing emails with links that trigger an alarm when they are clicked. after being tricked, most users would develop a strong avoidance heuristic to not click the links without deliberate evaluation. this will australasian journal of information systems hassandoust, singh & williams 2020, vol 24, research article contextualisation, vulnerability and phishing 18 improve users’ heuristics and trigger the more systematically deliberate thinking mode for clicking on email links. based on the findings of this study, in order to improve the impact of information security training, we contend that it is necessary to provide highly focused and contextualized awareness training (e.g., workshops, campaigns) targeted to different audiences. given that students take emails from school administration and lecturers seriously, such interventions may provide students with more effective ways to distinguish phishing emails from legitimate emails. there are several ways to do so such as: checking that emails are from the school domain; understanding the importance of verifying emails, especially those that require credential information; and providing students with a procedure to verify the authenticity of messages either by web or phone. students should also be educated in ways to identify the legitimacy of messages. in order to make contextualized strategies, individuals’ demographic characteristics should be taken into account to counteract user vulnerabilities4. training counteracts users’ phishing susceptibility through raising their sufficiency threshold effects to stop or limit heuristic processing, and thus it may increase the chance of systematic processing of messages. we suggest it is highly important that higher education institutions not only develop security training programs, but students’ enrolment in such training, their security awareness as well as their actual security practices (e.g., through utilizing phishing campaigns) should be monitored regularly. in order to prepare and motivate students to build up their security awareness level appropriately, instructors should integrate learning activities showing how to recognize security threats such as phishing, how to respond to and report these threats, and what is the magnitude and the cost of these threat issues. in addition, students should be educated on what to do after falling victim to a security threat. over-reliance on technical skill/knowledge for protection is common but unsafe. we consider this an important issue in a world increasingly being harmed by malicious internet practices. graduates entering employment require a high level of security awareness and the skills to function online in a defensive manner, to protect both themselves and the organizations for which they work. 9 limitations, future research and conclusion the present study was conducted in a single higher education institution in new zealand, so replicating it in wider community settings will help further validate its findings. this is especially because individuals with different cultures from different countries may behave differently regarding phishing messages (eroglu & croxton, 2010). this study focuses on general phishing attacks through emails and does not investigate users’ practices through spear phishing or phishing via online social media platforms. future research should examine the influence of contextualized messages on these other forms of phishing. we sent both types of emails to the same sample of students, so their responses to the later email (winning an ipad mini) may be influenced by their response to the first email (course-related). for a future study, separate samples that receive different phishing emails should be used. we applied only two frames for the phishing emails. however, comparisons between these two scenarios provide interesting insights into the influence of framing and contextualization. building mistakes into the phishing email suggests that these findings are only valid if phishers make mistakes, but phishing attacks are becoming increasingly sophisticated with stolen credentials and flawless english. therefore, future studies should incorporate flawless phishing messages. australasian journal of information systems hassandoust, singh & williams 2020, vol 24, research article contextualisation, vulnerability and phishing 19 hsm provides a theoretical framework for future studies in which both quantitative and qualitative data may be collected to investigate and measure the research models and hypotheses more systematically. quantitative data collected through surveys (e.g., scenariobased) and phishing experiments, would allow researchers to assess the hypotheses in a positivist way. on the other hand, qualitative data collected through field observations and interviews would allow investigators to obtain more first-hand insights into the phishing attacks, how individuals respond to and think about them, and provide the chance to validate theoretical reasoning and refine it. future studies may consider incorporating the anchoring effect in phishing experiments. the anchoring effect refers to the disproportionate impact of initially presented values on individuals when they make decisions and judgments (tversky & kahneman, 1974). in a decision-making process, anchoring effects are stronger when the source of the initial value is more ambiguous, less familiar, or more trustworthy (furnham & boo, 2011). the impact of a strong anchor such as users’ prior experience or a trustworthy source in phishing experiments needs to be explored to realize if individuals who are presented with a particular parameter or a trustworthy source would make insufficient adjustments when making their final decision. we tested the proposition that successful phishing attacks take advantage of internet users by reducing their sufficiency threshold and luring recipients to quickly and intuitively judge the validity of the message based on initial impressions of the immediate context. we also investigated the premise that susceptibility to phishing is significantly associated with the contextual setting of a phishing message. it has been investigated through an experiment with emails framed based on the proposed hypotheses to evoke user reactions. we found that phishing awareness is associated with users’ protective practices and reactions related to phishing. we also tested the assertion that there is a discrepancy between users’ perceptions of their phishing practices and their actual phishing reactions. we found a large gap between users’ perception and their actual reaction in contextualized phishing messages, while not much difference between perception and reaction exists in a non-contextualized general message. our findings reveal that the different demographics are associated with susceptibility to phishing. therefore, the study suggests the need for context-based and targeted education based on demographic features (e.g., gender). acknowledgement we acknowledge internetnz for providing financial support for this paper. references aleroud, a., & zhou, l. (2017). phishing environments, techniques, and countermeasures: a survey. computers & security, 68, 160-196. alsharnouby, m., alaca, f., & chiasson, s. (2015). why phishing still works: user strategies for combating phishing attacks. international journal of human-computer studies, 82, 6982. anderson, b. b., vance, a., kirwan, c. b., jenkins, j. l., & eargle, d. (2016). from warning to wallpaper: why the brain habituates to security warnings and what can be done about it. journal of management information systems, 33(3), 713-743. australasian journal of information systems hassandoust, singh & williams 2020, vol 24, research article contextualisation, vulnerability and phishing 20 arachchilage, n. a. g., & love, s. (2013). a game design framework for avoiding phishing attacks. computers in human behavior, 29(3), 706-714. arachchilage, n. a. g., & love, s. (2014). security awareness of computer users: a phishing threat avoidance perspective. computers in human behavior, 38, 304-312. arachchilage, n. a. g., love, s., & beznosov, k. (2016). phishing threat avoidance behaviour: an empirical investigation. computers in human behavior, 60, 185-197. bailey, j. l., mitchell, d., robert, b., & bradley, k. (2008). analysis of student vulnerabilities to phishing. amcis 2008 proceedings, 271. beycioglu, k. (2009). a cyberphilosophical issue in education: unethical computer using behavior–the case of prospective teachers. computers & education, 53(2), 201-208. blythe, m., petrie, h., & clark, j. a. (2011). f for fake: four studies on how we fall for phish symposium conducted at the meeting of the proceedings of the sigchi conference on human factors in computing systems https://dl.acm.org/doi/pdf/10.1145/1978942.1979459 certnz. (2019). quarter three report 2019. retrieved 2019, https://www.cert.govt.nz/about/quarterly-report/quarter-three-report-2019/ chaiken, s. (1982). the heuristic/systematic processing distinction in persuasion symposium conducted at the meeting of the symposium on automatic processing, society for experimental social psychology, nashville, in chaiken, s. (1987). the heuristic model of persuasion. hillsdale, nj: lawrence erlbaum. symposium conducted at the meeting of the social influence: the ontario symposium. 5 (3-39). chen, s., & chaiken, s. (1999). the heuristic-systematic model in its broader context. in s. chaiken & y. trope (eds.), dual-process theories in social and cognitive psychology (pp. 7396). new york (ny): guilford. chen, x., chen, l., & wu, d. (2018). factors that influence employees’ security policy compliance: an awareness-motivation-capability perspective. journal of computer information systems, 58(4), 312-324. chen, y., ramamurthy, k., & wen, k.-w. (2015). impacts of comprehensive information security programs on information security culture. journal of computer information systems, 55(3), 11-19. chou, h.-l., & sun, j. c. y. (2017). the moderating roles of gender and social norms on the relationship between protection motivation and risky online behavior among in-service teachers. computers & education, 112, 83-96. cohen, j. b., & reed, a. (2006). a multiple pathway anchoring and adjustment (mpaa) model of attitude generation and recruitment. journal of consumer research, 33(1), 1-15. d'arcy, j., hovav, a., & galletta, d. (2009). user awareness of security countermeasures and its impact on information systems misuse: a deterrence approach. information systems research, 20(1), 79-98. dennis, a. r., & minas, r. k. (2018). security on autopilot: why current security theories hijack our thinking and lead us astray. acm sigmis database for advances in information systems, 49(si), 15-38. australasian journal of information systems hassandoust, singh & williams 2020, vol 24, research article contextualisation, vulnerability and phishing 21 dinev, t., & hu, q. (2007). the centrality of awareness in the formation of user behavioral intention toward protective information technologies. journal of the association for information systems, 8(7), 386-408. downs, j. s., holbrook, m. b., & cranor, l. f. (2006). decision strategies and susceptibility to phishing symposium conducted at the meeting of the proceedings of the second symposium on usable privacy and security. https://dl.acm.org/doi/pdf/10.1145/1143120.1143131 eagly, a. h., & chaiken, s. (1993). the psychology of attitudes: harcourt brace jovanovich college publishers. epley, n., & gilovich, t. (2006). the anchoring-and-adjustment heuristic: why the adjustments are insufficient. psychological science, 17(4), 311-318. eroglu, c., & croxton, k. l. (2010). biases in judgmental adjustments of statistical forecasts: the role of individual differences. international journal of forecasting, 26(1), 116-133. esch, f. r., schmitt, b. h., redler, j., & langner, t. (2009). the brand anchoring effect: a judgment bias resulting from brand awareness and temporary accessibility. psychology & marketing, 26(4), 383-395. flores, w. r., holm, h., nohlberg, m., & ekstedt, m. (2015). investigating personal determinants of phishing and the effect of national culture. information & computer security, 23(2), 178-199. furnham, a., & boo, h. c. (2011). a literature review of the anchoring effect. socio-economics, 40(1), 35-42. goel, s., williams, k., & dincelli, e. (2017). got phished? internet security and human vulnerability. journal of the association for information systems, 18(1), 22-44. haeussinger, f., & kranz, j. (2017). antecedents of employees’ information security awareness –review, synthesis, and directions for future research. proceedings of the 25th european conference on information systems, guimarães, portugal. hajli, n., & lin, x. (2016). exploring the security of information sharing on social networking sites: the role of perceived control of information. journal of business ethics, 133(1), 111123. halevi, t., lewis, j., & memon, n. (2013). phishing, personality traits and facebook. arxiv preprint arxiv:1301.7643. halevi, t., memon, n., & nov, o. (2015). spear-phishing in the wild: a real-world study of personality, phishing self-efficacy and vulnerability to spear-phishing attacks. available at ssrn: https://ssrn.com/abstract=2544742 hanus, b., & wu, y. a. (2016). impact of users’ security awareness on desktop security behavior: a protection motivation theory perspective. information systems management, 33(1), 2-16. hassandoust, f., & techatassanasoontorn, a. a. (2020). understanding users' information security awareness and intentions: a full nomology of protection motivation theory. in cyber influence and cognitive threats (pp. 129-143): elsevier. australasian journal of information systems hassandoust, singh & williams 2020, vol 24, research article contextualisation, vulnerability and phishing 22 hilligoss, b., & rieh, s. y. (2008). developing a unifying framework of credibility assessment: construct, heuristics, and interaction in context. information processing & management, 44(4), 1467-1484. hovav, a., & d’arcy, j. (2012). applying an extended model of deterrence across cultures: an investigation of information systems misuse in the us and south korea. information & management, 49(2), 99-110. irwin, l. (2020). the 5 most common types of phishing attack. retrieved from https://www.itgovernance.eu/blog/en/the-5-most-common-types-of-phishing-attack iuga, c., nurse, j. r., & erola, a. (2016). baiting the hook: factors impacting susceptibility to phishing attacks. human-centric computing and information sciences, 6(1), 1-20. jacowitz, k. e., & kahneman, d. (1995). measures of anchoring in estimation tasks. personality and social psychology bulletin, 21(11), 1161-1166. jagatic, t. n., johnson, n. a., jakobsson, m., & menczer, f. (2007). social phishing. communications of the acm, 50(10), 94-100. jakobsson, m., & ratkiewicz, j. (2006). designing ethical phishing experiments: a study of (rot13) ronl query features symposium conducted at the meeting of the proceedings of the 15th international conference on world wide web. https://dl.acm.org/doi/pdf/10.1145/1135777.1135853 jansen, j., & van schaik, p. (2018). persuading end users to act cautiously online: a fear appeals study on phishing. information & computer security, 26(3), 264-276. johnston, a. c., & warkentin, m. (2010). fear appeals and information security behaviors: an empirical study. mis quarterly, 34(3), 549-566. kahneman, d. (2011). thinking, fast and slow: macmillan. kim, d., & kim, j. h. (2013). understanding persuasive elements in phishing e-mails. online information review. kumaraguru, p., rhee, y., sheng, s., hasan, s., acquisti, a., cranor, l. f., & hong, j. (2007). getting users to pay attention to anti-phishing education: evaluation of retention and transfer symposium conducted at the meeting of the proceedings of the anti-phishing working groups 2nd annual ecrime researchers summit https://dl.acm.org/doi/pdf/10.1145/1299015.1299022 kumaraguru, p., sheng, s., acquisti, a., cranor, l. f., & hong, j. (2010). teaching johnny not to fall for phish. acm transactions on internet technology, 10(2), 1-31. lim, j. s., ahmad, a., chang, s., & maynard, s. b. (2010). embedding information security culture emerging concerns and challenges symposium conducted at the meeting of the pacific asia conference on information systems https://aisel.aisnet.org/cgi/viewcontent.cgi?article=1041&context=pacis2010 loewenstein, g., o'donoghue, t., & bhatia, s. (2015). modeling the interplay between affect and deliberation. decision, 2(2), 55-81. luo, x. r., zhang, w., burd, s., & seazzu, a. (2013). investigating phishing victimization with the heuristic–systematic model: a theoretical framework and an exploration. computers & security, 38, 28-38. australasian journal of information systems hassandoust, singh & williams 2020, vol 24, research article contextualisation, vulnerability and phishing 23 mcelwee, s., murphy, g., & shelton, p. (2018). influencing outcomes and behaviors in simulated phishing exercisesieee. symposium conducted at the meeting of the southeastcon 2018. https://ieeexplore.ieee.org/document/8479109 mchugh, m. l. (2013). the chi-square test of independence. biochemia medica, 23(2), 143-149. mitnick, k. d., & simon, w. l. (2003). the art of deception: controlling the human element of security: john wiley & sons. moody, g. d., galletta, d. f., & dunn, b. k. (2017). which phish get caught? an exploratory study of individuals′ susceptibility to phishing. european journal of information systems, 26(6), 564-584. musuva, p. m., getao, k. w., & chepken, c. k. (2019). a new approach to modelling the effects of cognitive processing and threat detection on phishing susceptibility. computers in human behavior, 94, 154-175. nouh, m., nurse, j. r., webb, h., & goldsmith, m. (2019). cybercrime investigators are users too! understanding the socio-technical challenges faced by law enforcement. arxiv preprint arxiv:1902.06961. parsons, k., mccormac, a., pattinson, m., butavicius, m., & jerram, c. (2015). the design of phishing studies: challenges for researchers. computers & security, 52, 194-206. pattinson, m., butavicius, m., parsons, k., mccormac, a., & calic, d. (2015). factors that influence information security behavior: an australian web-based study. international conference on human aspects of information security, privacy, and trust. (pp. 231-241). springer, cham. petty, r. e., & cacioppo, j. t. (1984). the effects of involvement on responses to argument quantity and quality: central and peripheral routes to persuasion. journal of personality and social psychology, 46(1), 69. pienta, d., thatcher, j. b., & johnston, a. (2020). protecting a whale in a sea of phish. journal of information technology, 0268396220918594. resnik, d. b., & finn, p. r. (2018). ethics and phishing experiments. science and engineering ethics, 24(4), 1241-1252. salah el-din, r (2012) to deceive or not to deceive! ethical questions in phishing research. in hci research in sensitive contexts: ethical considerations workshop at hci, birmingham, uk. http://eprints.leedsbeckett.ac.uk/4834/ sheng, s., holbrook, m., kumaraguru, p., cranor, l. f., & downs, j. (2010). who falls for phish? a demographic analysis of phishing susceptibility and effectiveness of interventions proceedings of the sigchi conference on human factors in computing systems. https://dl.acm.org/doi/pdf/10.1145/1753326.1753383 simon, h. a. (1965). administrative behavior. a study of decision-making processes in administrative organization: macmillan. siponen, m. t. (2000). a conceptual foundation for organizational information security awareness. information management & computer security, 8(1), 31-41. soghoian, c. (2008, october). legal risks for phishing researchers. in 2008 ecrime researchers summit (pp. 1-11). ieee. https://ieeexplore.ieee.org/document/4696971 australasian journal of information systems hassandoust, singh & williams 2020, vol 24, research article contextualisation, vulnerability and phishing 24 stanciu, v., & tinca, a. (2016). students' awareness on information security between own perception and reality–an empirical study. accounting and management information systems, 15(1), 112-130. straub, d., boudreau, m. c., & gefen, d. (2004). validation guidelines for is positivist research. communications of the association for information systems, 13(1), 380-427. turner, c. g., & monk‐turner, e. (2007). gender differences in occupational status in the south korean labor market: 1988‐1998. international journal of social economics. tversky, a., & kahneman, d. (1974). judgment under uncertainty: heuristics and biases. science, 185(4157), 1124-1131. vishwanath, a., harrison, b., & ng, y. j. (2018). suspicion, cognition, and automaticity model of phishing susceptibility. communication research, 45(8), 1146-1166. wang, j., li, y., & rao, h. r. (2016). overconfidence in phishing email detection. journal of the association for information systems, 17(11), 759-783. wang, j., li, y., & rao, h. r. (2017). coping responses in phishing detection: an investigation of antecedents and consequences. information systems research, 28(2), 378-396. wegener, d. t., petty, r. e., detweiler-bedell, b. t., & jarvis, w. b. g. (2001). implications of attitude change theories for numerical anchoring: anchor plausibility and the limits of anchor effectiveness. journal of experimental social psychology, 37(1), 62-69. whetten, d. a. (2009). an examination of the interface between context and theory applied to the study of chinese organizations. management and organization review, 5(1), 29-56. white, g., ekin, t., & visinescu, l. (2017). analysis of protective behavior and security incidents for home computers. journal of computer information systems, 57(4), 353-363. white, g. l. (2015). education and prevention relationships on security incidents for home computers. journal of computer information systems, 55(3), 29-37. wilson, t. d., houston, c. e., etling, k. m., & brekke, n. (1996). a new look at anchoring effects: basic anchoring and its antecedents. journal of experimental psychology: general, 125(4), 387. wirth, w., böcking, t., karnowski, v., & von pape, t. (2007). heuristic and systematic use of search engines. journal of computer-mediated communication, 12(3), 778-800. wright, r. t., jensen, m. l., thatcher, j. b., dinger, m., & marett, k. (2014). research note— influence techniques in phishing attacks: an examination of vulnerability and resistance. information systems research, 25(2), 385-400. wu, j. y. (2014). gender differences in online reading engagement, metacognitive strategies, navigation skills and reading literacy. journal of computer assisted learning, 30(3), 252271. wu, m., miller, r. c., & garfinkel, s. l. (2006). do security toolbars actually prevent phishing attacks? proceedings of the sigchi conference on human factors in computing systems. https://dl.acm.org/doi/pdf/10.1145/1124772.1124863 australasian journal of information systems hassandoust, singh & williams 2020, vol 24, research article contextualisation, vulnerability and phishing 25 appendix a. summary of related literature on phishing author(s)/year research approach elements investigated behavioral aspect theory relevant findings musuva, getao, and chepken (2019) observations of phishing susceptibility and self-reported questionnaires attack quality, motivation to process, ability to process and knowledge, threat detection and elaboration phishing susceptibility elaboration likelihood model threat detection was found to explain why people who expend cognitive effort processing phishing communication are less likely to fall for phishing threats. jansen and van schaik (2018) online survey with fear appeal conditions perceived vulnerability and severity, fear, response efficacy, self-efficacy and response cost protective motivation protection motivation theory self-efficacy and fear were the most significant determinants of protection motivation. mcelwee et al. (2018) simulated phishing exercise outcome-based controls and behavior-based controls susceptibility to phishing agency theory behavior-based controls (e.g., targeted training) were more successful in reducing susceptibility to phishing. resnik and finn (2018) ethics and phishing experiments phishing experiments are ethical if there is an opt-out option for participants and they are debriefed afterwards ethical phishing experiments although phishing experiments consist of deception and breach informed consent requirements, the risks can be minimized if the confidentiality and the privacy of participants are protected. vishwanath, harrison, and ng (2018) two experimental studies cyber risk beliefs, deficient selfregulation, heuristic and systematic processing, email habit suspicion, cognition, and automaticity model (scam) suspicion individuals are likely to fall victim to phishing emails when aspects of the email aroused suspicion about the request. when individuals believe their cyber actions are quite risky, they tend to systematically process emails. aleroud and zhou (2017) online survey a taxonomy including attacking techniques, countermeasures, targeted environments and communication media phishing detection and prevention the taxonomy provides guidance for the design of effective techniques for phishing detection and prevention. the taxonomy also helps practitioners to evaluate and select methods, tools and features to handle phishing problems. australasian journal of information systems hassandoust, singh & williams 2020, vol 24, research article contextualisation, vulnerability and phishing 26 author(s)/year research approach elements investigated behavioral aspect theory relevant findings goel et al. (2017) online survey situational factors and contextualization of phishing emails vulnerability to phishing heuristicsystematic processing model contextualized messages that appeal to recipients’ psychological weaknesses increase their vulnerability to phishing. moody et al. (2017) phishing experiment and online survey situational and personality factors susceptibility to phishing delphi method emails sent from a known source significantly increase user susceptibility to phishing, as does a user’s risk propensity, curiosity, internet anxiety and general internet usage. wang et al. (2017) online survey experiment perceived phishing threat and detection efficacy, phishing anxiety coping responses in the process of phishing email detection. extended parallel process model perceived detection efficacy increases coping adaptiveness. coping adaptiveness positively impacts the two objective measures in the study, detection effort and detection accuracy. anderson et al. (2016) phishing experiment neurois techniques, functional magnetic resonance imaging (fmri) polymorphic warnings and conventional warnings behavior towards security messages (e.g. warning against phishing) via neurois techniques stimulusmodel comparator theory and the dualprocess theory polymorphic warnings reduced habituation compared to conventional warnings. arachchilage et al. (2016) mobile game prototype. perceived severity, susceptibility and threat, safeguard effectiveness and cost, selfefficacy and avoidance motivation avoiding phishing attacks participants' threat perception, selfefficacy, safeguard effectiveness, perceived threat severity and susceptibility elements positively influence threat avoidance behavior, whereas safeguard cost had a negative impact on it. australasian journal of information systems hassandoust, singh & williams 2020, vol 24, research article contextualisation, vulnerability and phishing 27 author(s)/year research approach elements investigated behavioral aspect theory relevant findings iuga et al. (2016) web-based study demographic characteristics of individuals, time-related factors and users’ ability to correctly detect a phishing attack phishing behavior anchoring effect psychological anchoring effect, gender and the years of computer usage have a significant positive impact on users’ ability to detect phishing attacks. popupbased phishing attacks have a higher success rate. alsharnouby, alaca, and chiasson (2015) user eye tracking study to identify the phishing websites improved browser security indicators and increased awareness of phishing phishing detection practice individuals still fall for phishing and don’t invest much time looking at security measures, although those that do are less susceptible to phishing. flores et al. (2015) scenario-based survey, phishing experiments, followup interviews added personal information about the target to an attack, individual’s trust, computer experience at work, helpfulness and gender risky phishing behavior the degree of target information in an attack increased the likelihood that an organizational employee falls victim to an actual phishing attack. parsons, mccormac, pattinson, butavicius, and jerram (2015) a role play scenario experiment category (e.g., industry or sender) of emails phishing detection practice signal detection theory informed participants were significantly better at discriminating between phishing and genuine emails than uninformed participants. arachchilage and love (2014) online questionnaire conceptual knowledge or procedural knowledge on computer users’ self-efficacy phishing threat avoidance behavior technology threat avoidance theory conceptual and procedural knowledge positively impact on computer users’ self-efficacy, which enhance their phishing threat avoidance behavior. wright et al. (2014) field experiment influence techniques (liking, reciprocity, social proof, consistency, references to authority, and scarcity) susceptibility to phishing attacks persuasion and motivation theory influence techniques such as liking, reciprocity, social proof, consistency, authority, and scarcity were significant predictors, implying fictitious experiences decrease phishing effectiveness australasian journal of information systems hassandoust, singh & williams 2020, vol 24, research article contextualisation, vulnerability and phishing 28 author(s)/year research approach elements investigated behavioral aspect theory relevant findings luo et al. (2013) qualitative explorative study psychological mechanisms such as argument quality, source credibility, genre conformity underlying the effectiveness of phishing attacks spear phishing victimization heuristic systematic model high argument quality, strong source creditability and strong genre conformity appears to lead to successful victimization. salah el-din (2012) controlled-lab studies need for deception, legal restrictions of conducting phishing studies phishing reaction proposed a roadmap to consider legal and ethical aspects prior conducting a phishing study. blythe et al. (2011) multi-method set of a few studies strategies used to identify phish phishing detection critical theory detection rates for phishing messages with logos were significantly lower than for those without. warnings about how to avoid phishing look exclusively to the user. soghoian (2008) case studies legal risks social phishing reaction the key risks are abuses of a provider’s terms of use, intellectual property copyright and rights infringement. kumaraguru et al. (2007) a user study simulated phishing emails in detection of the real phishing attacks phishing detection behavior a methodology called phishguru has been developed that was proved to be effective in training individuals regarding phishing attacks. australasian journal of information systems hassandoust, singh & williams 2020, vol 24, research article contextualisation, vulnerability and phishing 29 appendix b. measurement items of the study variable measurement items source phishing awareness (familiarity) on a scale from 1 to 9 (1 = not familiar at all; 9 = very familiar), please indicate your level of familiarity with the threat of phishing. (hanus & wu, 2016) prior experience of security threats have you ever experienced the following situation (such as desktops, phones, tablets, laptops etc.)? (yes or no you may select multiple answers) • a cyber-attack from opening a link or an attachment in a fraudulent email (called “phishing”) • a cyber-attack from visiting a website • a cyber-attack from a new icon or program (e.g., pop-up offering a free computer security scan) appeared out of nowhere • you had important personal information stolen, such as your credit card number • someone used or attempted to use your personal information without permission to obtain new credit cards or loans, run up debts, open other accounts, or commit other fraud. • someone used or attempted to get your accounts’ username and passwords such as bank account, or debit/check cards. the ratings for each respondent on each item sum were summed to provide an overall score. adapted from wang et al. (2016) and modified for phishing context australasian journal of information systems hassandoust, singh & williams 2020, vol 24, research article contextualisation, vulnerability and phishing 30 variable measurement items source perceived protective practices on a scale from 1 to 5: • if you received an email containing the logo and web address of your bank or one of your credit card companies requesting that you should verify information such as your date of birth, account number, address, etc., and the email was addressed to you personally – would you click on the link and provide the requested information? • if you received an email containing the logo and web address of your bank or one of your credit card companies requesting that you should verify information such as your date of birth, account number, address, etc., and the email was addressed to “dear customer” – would you click on the link and provide the requested information? • would you fill out an email form asking for personal financial information if the email appeared to be from a trusted site and was addressed to you personally? • if you received an email containing the logo and web address of your college/university requesting that you should verify information such as your name, student id, date of birth, address, etc., and the email was addressed to you personally –would you click on the link and provide the requested information? • i click on email links or posts with touching winning messages such as winning an apple ipad. the ratings for each respondent on each scenario were averaged to provide an overall score. (bailey et al., 2008) australasian journal of information systems hassandoust, singh & williams 2020, vol 24, research article contextualisation, vulnerability and phishing 31 appendix c: phishing messages phishing message 1: obtaining their course results sender: a personal email address, (***addmin@gmail.com) dear student, you are receiving this email in order to check the status of your attendence and assessments results. please click on the button below in order to check your attendence and assignments results. if you have any pending “disciplinary hearings” it may also influence your results. see the policy below. passing a paper students must and submit all assessments for a paper final examination note: the final examination is the final assessment a student takes for their paper and is normally worthe 60%. re-sit examinations note: this is an opportunity for a student to take the final assessment at a later date in their program (and not imediately after their finishing their paper). failed students students will be marked as having failed a paper if they have not achieved an overall paper mark of 50% after all opportunities have been given to the student. thank you, cathy exam officer [school name] t: +649 xxxxxx | f: +649 xxxxxx | adress: xxxxx e: cathy@[school domain with a typo].com (it should be ‘co.nz’ instead) phishing message 2: winning an ipad mini from: nz apple research team subject: get a free ipad mini for giving it a test drive dear student: you’ve won an ipad mini! apple (nz premium reseller) is distributing its new mini tablet to select university/institution students who are willing to help evaluate it. the tablet has the same capabilities as an ipad with a smaller screen. in return for the free tablet all we will request is for you to provide us feedback on the product every two weeks. you will be provided a template to fill out your experiences with the tablet. apple is an equal opportunity company and you were randomly selected without any cultural or racial bias. please register at the following link and make sure that you accept the terms and conditions at the end of the form. http://applle.com/resarchnzipadmini/ best wishes, apple research team, nz australasian journal of information systems hassandoust, singh & williams 2020, vol 24, research article contextualisation, vulnerability and phishing 32 appendix d. proportion of respondents who opened the phishing email, clicked on the link and submitted data by study level % did not open % opened % clicked % submitted data total number of respondents postgraduate – level 9 2.6% 6.7% 0.7% 4.5% 39 (14.5%) postgraduate – level 8 2.6% 5.9% 3% 4.1% 42 (15.6%) undergraduate – level 7 1.1% 3% 1.5% 6.7% 33 (12.3%) undergraduate – level 6 0.7% 1.1% 1.9% 30% 92 (33.7%) undergraduate – level 5 2.2% 2.6% 0.4% 18% 63 (23.2%) copyright: © 2020 hassandoust, singh & williams. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v24i0.2693 http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems gupta, kar & jebarajakirthy 2021, vol 25, research on user involvement special section on research on engaging stakeholders online 1 special section on research on engaging stakeholders online: the bright and the dark sides guest editors manish gupta department of hr, ibs hyderabad, a constituent of ifhe, deemed to be university, india manish.gupta.research@gmail.com arpan kumar kar indian institute of technology delhi, india charles jebarajakirthy department of marketing, griffith business school, griffith university, australia 1 background owing to the growing interest of people in using online media to communicate in their personal and professional life, organizations are striving hard to engage their stakeholders online much more than ever before. this has encouraged scholars to analyze what is specific to virtual means of engagement. from the little research available in the area of online engagement, it appears that along with advantages, online engagement also comes with grave consequences for an organization’s stakeholders. multi-stakeholder engagement and multistakeholder experience are gradually gaining prominence in management and is research to investigate how organizations attempt to connect both internal and external stakeholders (becker & jaakkola, 2020). such connection creates a better experience among the partners and also addresses long term sustainable relationships in networks of value co-creation where partners are mostly connected through the use of information and communication technologies as well as through service exchange in larger ecosystem (chuluun et al., 2017). recently, the focus of research has turned to the differences in experiences and the different treatment of the partners in a firm’s networked ecosystem. this has predominantly been driven by the adverse impacts of relationships, often driven by differences in the relative bargaining power of the participants within the network of value co-creation. however, there are a lot of benefits that strengthen such relationships in a network of value co-creation, where both parties engaged in co-creation of value get ample benefits and form a strong relationship (payne et al., 2008). such relationships and their outcome are documented in the existing literature, but the dynamics of such online engagement need a lot more research than has been undertaken so far. given this background, this special section explores various theoretical models that can be used to identify phenomena and factors that drive engagement in such dynamic and volatile ecosystems where stakeholders engage with each other for the co-creation of value. thus, along with the bright side, this special section puts a particular emphasis on the dark side of engaging stakeholders online, a lesser researched area. 2 the bright and the dark sides organizations are increasingly using the internet to engage their stakeholders (employees, suppliers, customers, government, and management) for outcomes such as higher australasian journal of information systems gupta, kar & jebarajakirthy 2021, vol 25, research on user involvement special section on research on engaging stakeholders online 2 productivity, growing revenue, and increased profits. different theoretical models of engagement have been explored for online platforms such as uses and gratification theory, dual processing theory, and technology adoption models (dolan et al., 2016; dolan et al., 2019; grover & kar, 2020a; grover et al., 2019; gupta et al., 2019). though research suggests that engaging stakeholders has a lot of benefits for individuals, groups, and organizations, the concerns relating to the overuse of these engagement practices especially through the internet cannot be ignored (shelton & skalski, 2014; fox & moreland, 2015; baccarella et al, 2018). this side includes the darker aspects that cover ‘too much of engagement’ such as physical and mental strain leading to imbalances in the stakeholders’ life. there are both positive and negative consequences of engaging stakeholders online and each engagement practice or intervention needs to be seen through both lenses. further, there has been a plethora of evidence surrounding disinformation and misinformation in the existing literature which impacts stakeholder engagement (aswani et al., 2019). such an exploration necessitates to investigate the context of engagement through more than one lens and from the perspectives of multiple stakeholders (friedman & miles, 2002). this enhances the chances of a more holistic theoretical understanding of this new and complex nature of engagement. to address issues of such complexity, mixed-method research designs are often suitable. a plethora of emerging research methodologies like user-generated content mining, social media analytics, artificial intelligence, and other approaches are guiding a lot of such research beyond the traditional approaches based on surveys, expert feedback, and analyses using inferential statistics (fan & gordon, 2014; rathore et al., 2017). such studies on engagement often lead to insights about stakeholder satisfaction, and mixed research methods that combine big data, unstructured data, surveys, and qualitative methods may collectively create better theoretical insights (kar & dwivedi, 2020). multi-stakeholder ecosystems (figure 1) create value for all participants. however, there is evidence that not all the stakeholders are in the focus of such value co-creation which may result in disruptions of the e in the long-run. the stakeholders not only interact with the core organization in the ecosystem, but also need to interact with other stakeholders (kar et al., 2019). given such interaction is necessary, the value created through some interaction often leads both to positive as well as negative encounters among the stakeholders. the stakeholders are often collectively connected in a value chain and more so in a more complex network of value co-creation. positive engagement during encounters strengthens the value chain and the network of value co-creation, while negative engagement may adversely affect the long term sustainability of the ecosystem. while in the short run, benefits are more visible, especially for the larger partners of the ecosystem (kar et al., 2019), in the long run, the network needs intervention if the darker sides become more prevalent than the positive encounters. encounters of engagement may also impact the social life of the customers, when the mediation of relationships move beyond technical platforms and the entire ecosystem shows the effects of the engagement. such engagement may also emerge between citizens and government when online platforms are used for capturing citizen participation in engagement models (grover & kar, 2020b). australasian journal of information systems gupta, kar & jebarajakirthy 2021, vol 25, research on user involvement special section on research on engaging stakeholders online 3 figure 1: multi-stakeholder engagement ecosystem based on this discussion, the overall objective of this special section and issue is to provide more insights about multi stakeholder engagement ecosystems. the special section thus has the following objectives. it sets out • to challenge existing theories in the online engagement literature • to explore different theoretical models of the dual effects, positive and negative, on stakeholders who are involved in online engagement • to examine how the internet itself can be a means to shift stakeholders from the dark side to the bright side of engagement • to establish the need for interventions in such engagement in online communities of multi stakeholder ecosystems. articles in the special section the following six articles have been selected for inclusion in the special section: article 1: demystifying the dark side of social networking sites through mindfulness while the authors of this article recognize the positive outcomes of using the social networking sites (sns) for communication, they argue that using these site is spoiling individuals’ real life human connections. in a sequential mixed method design, they studied a sample of 236 users and found that mindfulness impacts and counteracts exhaustion that is caused through compulsive usage and otherwise is a worrying factor for society. article 2: do policymakers use social media for policy design? a twitter analytics approach australasian journal of information systems gupta, kar & jebarajakirthy 2021, vol 25, research on user involvement special section on research on engaging stakeholders online 4 this paper takes a macro-level view and investigates ayushman bharat, a government of india scheme, to explore what role does social media, in this case, twitter, plays in the public policy process and design. the authors analyzed twitter data about the scheme under investigation and found that despite consistent issues being raised by citizens and experts about the scheme, the policy design has not significantly been changed. article 3: impact of addiction of online platforms on quality of life: age and gender as moderators using the partial least square structural equation modeling technique, the authors of this article analyzed 320 responses to find factors that affect online platform addiction, often considered the dark side of excessive online engagement. such factors in particular loneliness and depression, were found to predict addiction and in turn, the quality of life. age and gender seem to affect this relationship. one of the findings, for example, is that compared to the midaged individuals, young adults are more prone to addiction while feeling lonely. article 4: gamifying the gig: transitioning the dark side to bright side of online engagement the authors of this article link an emerging section of the workforce, gig workers with an equally emerging online engagement technique, gamification. by analyzing 367 responses of gig workers, the authors recommend that gamification does enhance job satisfaction and productivity of gig employees and enhances retention rates. article 5: the dark side of social media engagement: an analysis of user-generated content in wildlife trade online communities in this article, the authors used user-generated content (ugc) in the context of online exotic wildlife trade communities in a biodiversity hotspot, the republic of indonesia, by studying comments in 12 facebook groups. they found that factors such as motivation, opportunity, and ability affect the information processing – comment valence relationship. their findings encourage conservation practitioners to post the pro-conservation content after considering the user’s unique attitudes and knowledge to engage their target audience online. article 6: modelling customer engagement behaviour in smart retailing the authors of this study investigated the role of smart retail technologies (srt) in engaging customers online. the findings of this study reveal complex relationships and provide evidence for the influence of srt characteristics (novelty, effectiveness, and interaction quality) and meta-utaut variables (performance expectancy, effort expectancy, social influence, and facilitating conditions) on customers’ attitude, behavioural intention, and engagement behaviour towards smart retailing. 3 conclusion the articles that are included in this special section, collectively accomplish the section objectives by reviewing existing literature and investigating new primary and secondary data. the articles cover several types of online engagement such as, social, customer, and employee engagement among others. while some focus on the bright side, others have an emphasis on overcoming the dark sides of online engagement by discussing phenomena such as exhaustion, depression, compulsive usage, and addiction. we believe that together the six selected manuscripts provide an interesting reading of new insights into the bright and dark sides of online engagements. australasian journal of information systems gupta, kar & jebarajakirthy 2021, vol 25, research on user involvement special section on research on engaging stakeholders online 5 acknowledgement beyond the reviewers of the submitted articles and the ajis which provided the opportunity to publish these ideas, the guest editors wish to thank associate professor john lamp and professor karlheinz kautz for their continuous guidance and support without which this special section would not have been possible. references aswani, r., kar, a.k. & ilavarasan, p.v. (2019). experience: managing misinformation in social media – insights for policy makers from the twitter analytics. journal of data and information quality, 12(6), 1-18. baccarella, c. v., wagner, t. f., kietzmann, j. h., & mccarthy, i. p. (2018). social media? it's serious! understanding the dark side of social media. european management journal, 36(4), 431-438. becker, l., & jaakkola, e. (2020). customer experience: fundamental premises and implications for research. journal of the academy of marketing science, 48(4), 630-648. chuluun, t., prevost, a., & upadhyay, a. (2017). firm network structure and innovation. journal of corporate finance, 44, 193-214. dolan, r., conduit, j., fahy, j., & goodman, s. (2016). social media engagement behaviour: a uses and gratifications perspective. journal of strategic marketing, 24(3-4), 261-277. dolan, r., conduit, j., frethey-bentham, c., fahy, j., & goodman, s. (2019). social media engagement behavior. european journal of marketing, 53(10), 2213-2243. fan, w., & gordon, m. d. (2014). the power of social media analytics. communications of the acm, 57(6), 74-81. fox, j., & moreland, j. j. (2015). the dark side of social networking sites: an exploration of the relational and psychological stressors associated with facebook use and affordances. computers in human behavior, 45, 168-176. friedman, a. l., & miles, s. (2002). developing stakeholder theory. journal of management studies, 39(1), 1-21. grover, p., & kar, a. k. (2020a). user engagement for mobile payment service providers– introducing the social media engagement model. journal of retailing and consumer services, 53. https://doi.org/10.1016/j.jretconser.2018.12.002 grover, p., & kar, a. k. (2020b). “# government”-understanding dissemination, transparency, participation, collaboration and engagement on twitter for citizens. in international working conference on transfer and diffusion of it (pp. 365-376). springer, cham. grover, p., kar, a. k., dwivedi, y. k., & janssen, m. (2019). polarization and acculturation in us election 2016 outcomes–can twitter analytics predict changes in voting preferences. technological forecasting and social change, 145, 438-460. gupta, m., pandey, j., gaur, j., & vohra, n. (2019). research on role of technology in workforce management, australasian journal of information systems 23(1), 1-5. australasian journal of information systems gupta, kar & jebarajakirthy 2021, vol 25, research on user involvement special section on research on engaging stakeholders online 6 kar, a. k., & dwivedi, y. k. (2020). theory building with big data-driven research–moving away from the “what” towards the “why”. international journal of information management, 54, 102205. kar, a. k., ilavarasan, v., gupta, m. p., janssen, m., & kothari, r. (2019). moving beyond smart cities: digital nations for social innovation & sustainability. information systems frontiers, 21(3), 495-501. payne, a. f., storbacka, k., & frow, p. (2008). managing the co-creation of value. journal of the academy of marketing science, 36(1), 83-96. rathore, a., kar, a.k. and ilavarasan, p.v. (2017). social media analytics – literature review and directions for future research. decision analysis, 14(4), 229-249. shelton, a. k., & skalski, p. (2014). blinded by the light: illuminating the dark side of social network use through content analysis. computers in human behavior, 33, 339-348. copyright: © 2021 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v25i0.3371 http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems jung et al. 2022, vol 26, research on applied ethics ‘cambridge moralica’ 1 ‘cambridge moralica’ towards an ethical framework for social media analytics anna-katharina jung university of duisburg-essen, germany anna-katharina.jung@uni-due.de sünje clausen university of duisburg-essen, germany aline shatki franzke university of duisburg-essen, germany julian marx university of duisburg-essen, germany abstract en route to the unravelling of today’s multiplicity of societal challenges, making sense of social data has become a crucial endeavour in information systems (is) research. in this context, social media analytics (sma) has evolved to a promising field of data-driven approaches, guiding researchers in the process of collecting, analysing, and visualising social media data. however, the handling of such sensitive data requires careful ethical considerations to protect data subjects, online communities, and researchers. hitherto, the field lacks consensus on how to safeguard ethical conduct throughout the research process. to address this shortcoming, this study proposes an extended version of a sma framework by incorporating ethical reflection phases as an addition to methodical steps. following a design science approach, existing ethics guidelines and expert interviews with sma researchers and ethicists serve as the basis for redesigning the framework. it was eventually assessed through multiple rounds of evaluation in the form of focus group discussions and questionnaires with ethics board members and sma experts. the extended framework, encompassing a total of five iterative ethical reflection phases, provides simplified ethical guidance for sma researchers and facilitates the ethical self-examination of research projects involving social media data. keywords: social media analytics, research ethics, design science research. 1 introduction in the past decade, social media have become pervasive information systems (is) that penetrate many areas of our lives. in 2020, social media platforms such as instagram, facebook, twitter, or tiktok have been actively used by 3.81 billion users globally and this number is still growing (statista, 2020). users generate content, seek information, and interact on social media platforms. the resulting data presents valuable opportunities for companies, governments, and researchers to understand and interact with users (kaplan & haenlein, 2010; wiener, saunders, & marabelli, 2020; zeng, chen, lusch, & li, 2010). the process of collecting, monitoring, analysing, summarising, and visualising social media data is referred to as social media analytics (sma; zeng et al., 2010), which has evolved into a recognised research methodology in the is discipline and beyond (stieglitz, mirbabaie, ross, & neuberger, 2018). at present, it is applied in various fields, such as disaster response australasian journal of information systems jung et al. 2022, vol 26, research on applied ethics ‘cambridge moralica’ 2 (mirbabaie, bunker, stieglitz, marx, & ehnis, 2020), political communication (anstead & o’loughlin, 2015), or marketing (misirlis & vlachopoulou, 2018). while sma poses a plethora of potential benefits, it is also subject of an ethical debate. for example, the collection of personal data, often without informed consent, raises questions about the privacy rights of users, the publicness of data stemming from social media platforms, and whether agreeing to the terms and conditions of the platform is sufficient for using the data in research (abbasi, sarker, & chiang, 2016; marabelli & markus, 2017; ravn, barnwell, & barbosa neves, 2019). these and other ethical issues in the context of social media data have not been sufficiently addressed in is research (marabelli & markus, 2017). with the growing impact and utilisation of social media data, respecting ethical considerations in the sma process becomes increasingly critical. this is not only reflected in different calls for more research in this domain (marabelli & markus, 2017; taylor & pagliari, 2018) but also in recently established legal requirements such as the general data protection regulation (gdpr; european parliament and european council, 2018) and rigorous ethical assessment requirements on sides of financing boards and scientific journals. different communities such as the association of internet researchers (aoir), the institute of electrical and electronics engineers (ieee), and the association of computing machinery (acm) have published ethical guidelines, which address separate elements of sma research, for example the “internet research ethics 3.0” (acm, 2018; franzke, bechmann, zimmer, ess, & association of internet researchers, 2020; ieee, 2020). however, these fail to provide sufficient practical guidance for researchers within the specified sma research process (see e.g. ieee, 2020; taylor & pagliari, 2018) and navigating this diverse landscape of multidisciplinary patchwork remains challenging for sma researchers. ethical guidelines can be complex, too ambiguous for certain research projects, and their consideration timeconsuming, which might explain their insufficient integration in sma practices. therefore, in this paper, we explore the following research question: rq: how can ethical conduct be incorporated in the social media analytics process? to answer this question, we follow the design science research (dsr) approach (hevner, 2007; hevner, march, park, & ram, 2004) to develop an extended version of an acquainted sma framework (stieglitz, dang-xuan, bruns, & neuberger, 2014; stieglitz et al., 2018), which includes ethical reflections that iteratively attach to the process of conducting sma research. specifically, we adapt the three-cycle view of dsr. to establish our knowledge base and draft an initial version of the artefact, we turn to the state of the art of sma research and existing ethical guidelines in adjacent fields, followed by a description of the application environment. subsequently, we refine the framework based on twelve expert interviews with leading scholars in the fields of sma and digital ethics. to evaluate the applicability of the final framework, we conducted two focus group discussions with sma researchers and collected feedback from the initial round of experts, senior sma researchers as well as members of university ethics boards. our study attempts to make both theoretical and practical contributions and is thus an example of applied ethics which seeks to give guidance on how sma researchers can act normatively right (beauchamp, 2008). the common values and norms of the specific group of sma researchers forms the core interest of this study. first, we provide them with a refined framework of sma research that incorporates ethical reflections at every stage of the research process, which closes the gap between practical implementation of sma measures and australasian journal of information systems jung et al. 2022, vol 26, research on applied ethics ‘cambridge moralica’ 3 theoretical ethical considerations, which can create implications both on an individual and a societal level. as a result, our research mediates between two disjoined perspectives on sma, that is, application and ethics, and equips sma researchers with an applicable framework to facilitate ethical conduct throughout the sma process. according to stahl (2012), normative is research deals with four different areas: moral intuition, explicit morality, ethical theory, and meta-ethical reflections. based on this classification, our research is located at the explicit morality level. this means that it goes beyond individual moral intuition (kekes, 1986), which is the direct personal reaction if something is the right or wrong but instead defines the agreed moral values of sma researchers and applies them to the different steps of their research process. this step forms the basis for the formation of ethical theories and meta-ethical reflections, which not only ask what action is right, but why a certain action is right and how it can be interpreted in the context of different ethical schools of thought. the remainder of the paper is structured as follows. first, we give an insight into the methodology of the study, followed by the rigor cycle, which is based on the literature on sma in is and existing ethical guidelines. subsequently, we present the rigor cycle, which explicates the application environment of the artefact. this will be followed by the artefact description, and the explanation of our evaluation methods and results. conclusively, we present the final framework and discuss its implications, limitations, and provide suggestions for further research. 2 methodology the methodological basis of this study is grounded in the dsr approach popularised by hevner (2007). it constitutes a paradigm that looks for innovative solutions of problems through the design of artefacts (hevner et al., 2004), which can be of different nature. as depicted in figure 1, the approach includes three research cycles: the rigor, the relevance, and the design cycle. figure 1. ethical sma framework development based on the dsr cycles by hevner (2007) australasian journal of information systems jung et al. 2022, vol 26, research on applied ethics ‘cambridge moralica’ 4 the first cycle of dsr, the rigor cycle, gives an overview of the existing knowledge base, including the most relevant scientific theories and models, the state-of-the-art in the field and the description of existing artefacts and processes (hevner, 2007). the sma framework by stieglitz et al. (2014;2018) forms our knowledge base, as it thoroughly summarizes the steps of the sma research process. further, we analysed the existing literature and guidelines for ethical research focussing on those relevant to sma and big data analytics. the relevance cycle, which will be addressed in the fourth section, provides the context of the dsr project by analysing the environment (actors, organisation, and technology) and formulates the main problem and resulting needs. in addition, it provides the criteria for the evaluation of the final artefact (hevner, 2007). the third and internal cycle of dsr is the design cycle. it is driven by the input from the rigor and relevance cycle and aims at designing, building, and evaluating the artefact. overall, we turn to ‘ethics by design’ (d’aquin et al., 2018) to provide us with design principles for the artefact as it suggests accompanying the entire research lifecycle with ethical questions and the anticipation of future implications. specifically, our design cycle included four steps: 1) developing a first version of the framework based on the input from the rigor and relevance cycle, 2) the substantive evaluation of the first version with expert interviews with ethicists and sma researchers, an 3) evaluation of the second version with two focus groups of sma researchers focusing on the two acceptance criteria, and an 4) evaluation of the third version of the artefact based on feedback from researchers applying the framework to a recent sma research project. 3 rigor cycle – literature review 3.1 social media analytics in information systems research the notion of sma refers to an interdisciplinary approach to social data analysis that primarily takes a methodical standpoint. it combines a set of data-driven analysis techniques including the collection, analysis, and visualisation of social media data (stieglitz et al., 2014, 2018). incorporating the sma approach adds significant innovative strength to the discipline of is as it broadens data foundations, provides swift analysis of real-world phenomena and allows a higher-level perspective on collective online behaviour. in recent years, we could observe is literature spawning resourceful studies using sma in various fields such as disaster response (mirbabaie et al., 2020), social movements (tye, leong, tan, tan, & khoo, 2018), or political communication (anstead & o’loughlin, 2015). specifically, sma encompasses the tracking and scientific utilisation of large social media datasets. the approach envelopes a set of suitable methods to systematically analyse social media data. stieglitz et al. (2018) divided this process in the work stages of (1) discovery, (2) tracking, (3) preparation, and (4) analysis. figure 2 provides an overview of the sma framework. australasian journal of information systems jung et al. 2022, vol 26, research on applied ethics ‘cambridge moralica’ 5 figure 2. social media analytics framework, adapted from stieglitz et al. (2018). in this paper, we build on this sma framework because it provides researchers with methodical guidelines to undergo an application-driven, replicable process of deploying social media as a data source for empirical research (stieglitz et al., 2014). the framework was extended to cover challenges in event and topic detection, setting up a software architecture, and data visualisation and quality (stieglitz et al., 2018). it constitutes a seminal aid to orientation for novice and experienced researchers, which, as of this writing, is mirrored in the combined citation count of 714 between the original version and its first extension. we carefully reviewed other sma frameworks but found that they are either context-specific, e.g. enterprises (lee, 2018), or focus on certain methodical approaches, e.g. machine learning (jimenez-marquez, gonzalez-carrasco, lopez-cuadrado, & ruiz-mezcua, 2019). more recently, scholars have advanced the field of sma by testing the possibilities of data exploration and establishing social media as a viable data source for is research. however, aside from its methodical value and high degree of applicability, the sma framework by stieglitz et al. (2014; 2018), or other related literature, have not yet sufficiently addressed ethics as an integral part of the sma research process. 3.2 ethical issues of social media analytics along with the application-oriented benefits of sma, several ethical challenges emerge in the research process, for example related to the privacy of social media users and “identifying acceptable levels of intrusion” (abbasi et al., 2016, p. 10), non-consensual experiments (marabelli & markus, 2017), and accounting for potential biases (günther, rezazade mehrizi, huysman, & feldberg, 2017). recent publications in is highlight the shortage and importance of ethical considerations in big data analysis and sma and called for more research in this domain (abbasi et al., 2016; marabelli & markus, 2017; taylor & pagliari, 2018; wiener et al., 2020). thereby, data scientists are considered to play an important role as they translate data into knowledge and actionable insights, and function as “change agents” (wiener et al., 2020, p. 77). however, prevalent practical frameworks describing the application of sma (stieglitz et al., 2014, 2018) do not broach the issue of ethics. concurrently, existing ethical guidelines in this domain insufficiently capture the ethical challenges that emerge at different stages of the sma research process (taylor & pagliari, 2018). australasian journal of information systems jung et al. 2022, vol 26, research on applied ethics ‘cambridge moralica’ 6 due to its interdisciplinary and multi-method nature, sma brings together several ethical discourses and disciplines such as information systems, communication technology and computing ethics, big data ethics, and ethics of artificial intelligence. hence, if sma researchers seek ethical guidance, one option is reviewing existing ethical guidelines from these referential fields. in the following section we give an overview of ethical guidelines relevant to sma. 3.3 ethical guidelines and codes of conduct applicable to sma scholarship in the field of research ethics guidelines concerning the internet, big data and sma, ranges from various associations from within academia, such as the aoir (ess & association of internet researchers, 2002; franzke et al., 2020; markham, buchanan, & association of internet researchers, 2012), the acm (acm, 2018), the research code of conduct of the ais (ais, 2014) to guidelines of professional organisations like the us-based ieee (ieee, 2020). a high-level overview of their codes of conduct and guidelines is provided in table 1. each of these guidelines or codes of conduct covers some aspects which are relevant to sma researchers, such as data privacy. guidelines focus association of computing machinery (acm): code of ethics and professional conduct (2018) principles and responsibilities to inspire and guide ethical conduct (e.g., avoid harm, respect privacy) of all computing professionals (e.g., practitioners, instructors, students). association of internet researchers (aoir): ethical decision-making and internet research: recommendations from the aoir ethics working committee (v.1: 2002; v.2: 2012; v.3: 2019) recommendations designed to support and inform those responsible (e.g., researchers, ethicists, students, organizations, or academic societies) for making decisions about the ethics of internet research (e.g., rights and expectations of human subjects for autonomy, privacy, and informed consent). association for information systems (ais): code of research conduct (2014) requirements, recommendations, and advice for conducting research and publishing in is (e.g., give priority to public interest when designing or implementing new is). ieee: code of ethics (2020) principles of ethical and professional conduct in the context of emerging technologies (e.g., disclose limitations). table 1. overview of guidelines and codes of conduct applicable to sma. the broad emergence of ethics guidelines indicates a high demand for practical guidance which brings together theoretically derived ethical concerns but also the everyday experience of researchers and practitioners (stahl, timmermans, & mittelstadt, 2016). besides hands-on guidance, additional meta studies on the role of ethics guidelines also address the guidelines’ shortcomings (hagendorff, 2020; jobin, ienca, & vayena, 2019; mittelstadt, 2019). due to the abundance of guidelines and research on ethical questions, sma researchers face the challenge to identify those ethical questions, which are relevant for their specific research process. to design our artefact, we draw on the meta concept of “ethics by design”, which understands ethical research not as a binary decision before the start of a research project, but rather as a continuous iterative process. ethics by design, which evolved from the concept of “privacy by design”, suggests an accompaniment of the research lifecycle by asking ethical questions and anticipating its implications for society (d’aquin et al., 2018). even though an emergence of tentative research and guidelines related to the field of sma can be observed, however, guidance that focuses on the specificities of the sma research process has not been provided so far. to address this shortcoming, our paper will proceed with the specific challenges and australasian journal of information systems jung et al. 2022, vol 26, research on applied ethics ‘cambridge moralica’ 7 requirements of academic sma research, which informs the relevance cycle of our dsr approach. 4 relevance cycle – institutional challenges of ethical sma research the purpose of this section is to highlight involved actors, organizational, and technological structures, as well as problems and opportunities of our research (hevner, 2007). this helps to determine the application context of the artefact and requirements that need to be considered. the application environment of this study is academia. the potential users of the artefact, which aims to provide ethical guidance throughout the entire sma research process are researchers of different disciplines, experience levels, ranging from undergraduate students to full professors, resulting in (a) discipline disparities. sma is applied by researchers from social sciences and computer sciences such as information systems. however, each discipline approaches social media with different research questions and varying methods, which evoke distinct ethical problems. consequently, the way sma is applied and understood may significantly differ. while researchers from computational sciences are equipped with sophisticated methods for the analysis of social media data, they are often less concerned with methodological underpinnings and theoretical models, which researchers from social science employ to make sense out of social media data (stieglitz et al., 2018; tinati, phillipe, pope, carr, & halford, 2014). academia has an international orientation, despite cultural differences and contrasting trends of serving national priorities and operating in an international setting (guri-rosenblit, 2015), while maintaining its core objectives of teaching, research, and community/business partnerships (whitchurch, 2012). this (b) cultural pluralism in academia may lead to ethical pluralism, which affects the ethical choices made in the sma research process (hongladarom, 2017). the acceptance of scientific quality standards oftentimes collides with the high pressure to succeed and publish in academia, colloquially known as the (c) “publish or perish” culture (wiener et al., 2018). this tension sparks a lot of ethical questions regarding the validity of sma research and the implications of research for academia, the industry, and society. additionally, academia is witnessing an ongoing discourse about open-access culture, involving an enhancement of research data management (open data) to benefit the community and the individual researcher (stieglitz et al., 2020). although improved collaboration and knowledge sharing may entail benefits and opportunities from an ethical point of view, it also sparks issues regarding the privacy protection of users in social media datasets, which keeps sma researchers stuck in a dilemma. the (d) technological infrastructures available to sma researchers differ tremendously as they highly depend on the financial resources of each institution and on the technology access in the respective country (boyd & crawford, 2012; cruz-jesus, vicente, bacao, & oliveira, 2016). generally, however, academia can be considered a digitised area of work. the target group of this study, researchers who conduct sma, work with their computer and complete most of their work digitally. their media competence may vary depending on their area of expertise, which may result in different abilities to evaluate the ethical hurdles linked to the technological components of sma, such as machine learning algorithms supporting the data analysis (boyd & crawford, 2012; stieglitz et al., 2018). australasian journal of information systems jung et al. 2022, vol 26, research on applied ethics ‘cambridge moralica’ 8 additionally, there are (e) regulatory differences, for example between international academic partners, exemplified by authorities creating a disparity in ensuring compliance with certain ethical standards. in 2018, the gdpr a binding law in the european union converted ethical challenges, especially regarding the protection of data and research subjects, into hard law (european parliament and european council, 2018). the gdpr lately influenced the debate on research ethics and is relevant for all entities which deal with data collection, analysis, and storage in countries of the european union. from these and the arguments presented prior in the rigor cycle, we derive two acceptance criteria which should be met in the evaluation of the artefact: 1. awareness for ethics: the artefact encourages or extends ethical reflection throughout the sma research process for a diverse group of researchers 2. applicability of the artefact: the artefact is a useful tool for sma researchers based on the reviewed knowledge base (section 3: rigor cycle) and the identified application environment and acceptance criteria (section 4: relevance cycle) we derive the initial version of the artefact in the first design cycle (section 5). 5 design cycle artefact description the initial artefact was composed of the sma framework (stieglitz et al., 2014, 2018) and five additional ethical reflection stages highlighting relevant ethical considerations in each step of the sma process. these stages were derived from ethical guidelines and scholarship originating in research fields referential to sma (e.g., table 1). the first version of the artefact is depicted in figure 3. figure 3. initial artefact with adaptations to the framework highlighted. if the initial idea or research design is unethical, following ethics guidelines later does not solve the initial issue. thus, the meta reflection stage covers necessary theoretical and ethical australasian journal of information systems jung et al. 2022, vol 26, research on applied ethics ‘cambridge moralica’ 9 overtones of sma research including initial biases, avoiding harm doing good, legal considerations, power issues, transparency, cultural dimension, and ethical questions of your field which accompany and guide the entire research process. initial biases are based on the need to reflect on personal and technical biases that relate to one’s socio-demographic or cultural background or the expected results. what are my personal expectations of my research and how they influence the research design? who might be missing or silent in the proposed research design (e.g., elders who do not use social media)? (chen, zheng, & ceran, 2016). overall, researchers should strive to magnify the public good (acm, 2018; ess & association of internet researchers, 2002). thus, the subcategory avoiding harm doing good was included to encourage researchers to reflect on who benefits from their research: i, my discipline, or the society as a whole? how could possible harm be avoided and how can the researcher give something back to the researched communities (acm, 2018; ais, 2014)? ethics and law are highly interwoven; hence, the subcategory of legal considerations was added. for example, many ethical issues surrounding individual data rights are covered by the gdpr (safari, 2016) which is binding law in the european union. hence, sma researchers should be aware of the legal frameworks which apply to their research and potentially discuss these with collaborators from other judicial areas (ess & association of internet researchers, 2002). besides binding laws, sma researchers are confronted with the terms and conditions of the social media platforms. these often restrict (automatic) data collection (lomborg & bechmann, 2014) and thereby limit possible research designs and outcomes of sma (bruns, 2019; møller & bechmann, 2019). this tension is referred to as power issues and researchers should reflect on how these have shaped the research design. other power issues can occur between third-party funding institutions or between researchers and research subjects, especially when researching more vulnerable groups (leurs, 2017). the value of transparency is very prevalent in most ethics guidelines (jobin et al., 2019) and is becoming increasingly important in research (e.g., open data initiatives). thereby, a challenging question for sma researchers is how much access into social media datasets is acceptable without violating individual privacy (abbasi et al., 2016). sma researchers should consider how they can operationalise transparency in their research, for example by obtaining strong informed consent agreements. the importance of the so-called cultural dimension for ethical reflection has been stressed by the aoir ever since (franzke et al., 2020). conducting internet research, especially in social media, involves crossing cultural boundaries, norms, practices, and beliefs. individual versus relational concepts of selfhood cause the necessity for individual or more relational consents (ess, 2005). internet research ethics is “grown from the local source, meaning that the [ethical] vocabulary comes from the traditional and intellectual source of the culture in which a particular researcher is working” (hongladarom, 2017, p. 161). lastly, each field of research follows its own code of conduct. with the subcategory ethical questions of your field we aim to encourage and provide room for an open and ongoing dialogue with the ethical guidelines of one's faculty or institute. the tracking reflection stage covers informed consent, issues related to the venue of research, privacy, autonomy & anonymity, discrimination, and data minimisation. while research has shown that informed consent might be difficult to obtain in big data research before data tracking (van schie, westra, & schäfer, 2017), the question remains of how to inform subjects about their data being used for research purposes. how can research be conducted so that autonomy of australasian journal of information systems jung et al. 2022, vol 26, research on applied ethics ‘cambridge moralica’ 10 the user is protected?1 the subcategory venue is derived from the necessity of context-sensitive research (ess & association of internet researchers, 2002, p.1) and refers to the characteristics of the researched social media platform and subjects. specifically, the type and sensitivity of the communities, actions, and interactions under study but also the expectations of the individuals (e.g., regarding the publicity of their content) evoked by the platform (ess & association of internet researchers, 2002). for example, despite having signed the terms and conditions, users might not expect that data from a seemingly more private venue (e.g., a telegram channel) can be accessed by third parties. privacy, too, is a dominant norm in ethics guidelines and refers to the right to control data about oneself (mittelstadt, 2017). thereby, a privacy paradox can be observed. people share a lot of personal information online despite valuing privacy which might be due to a lack of awareness that this information is entirely public (adjerid, peer, & acquisti, 2018). moreover, classifying individuals into ad hoc groups based on (behavioural) social media data (e.g., likely a voter of a political party) to inform algorithmic decision-making might lead to violations of group privacy (mittelstadt, 2017). which steps can be taken by sma researchers to protect individual and group privacy? protecting users’ autonomy & anonymity requires researchers to reflect on whether applied social media interventions or analysis methods could be manipulative or invasive. a negative example which might have harmed research subjects is an experiment by facebook which altered the timeline to show more positive/negative content to users to investigate emotional contagion (jouhki, lauk, penttinen, sormanen, & uskali, 2016). further, ethical issues might arise if analysis methods are used to infer sensitive information or make predictions about individuals based on social media data, for example whether someone is likely to become depressed (laacke, mueller, schomerus, & salloch, 2021). social media data is rarely representative of the researched community (e.g., due to silent or missing actors) and derived insights might lead to discrimination of individuals or groups (o’neil, 2016). sma researchers should reflect on who will be represented in the tracked data and how that might influence possible results and the generalisability of their research. lastly, data minimisation is not only a principle of the gdpr but might also reduce ethical risks (franzke et al., 2020). while big data sets are crucial for some types of analyses, researchers should try to minimise collected data. for example, by using specific timeframes and keywords, or sharing and reusing data (møller & bechmann, 2019). the preparation reflection stage entails data cleaning, biases, and unintended insights. social media data can be of low quality or ‘noisy’ (stieglitz et al., 2018), for example due to colloquial language, typos, and emoticons. thus, unstructured data such as tweets often require extensive data cleaning prior to analysis. the cleaning decisions typically reflect the researcher’s value judgements of the data which can influence the subsequent analysis and results. thus, researchers should reflect on their motivation and expectations when cleaning social media data and consider its impact. for example, how does removing data affect obtained results or minority representation (franzke et al., 2020)? biases during the preparation phase refer to incomplete or inherently biased data sets provided by social media platforms and other data providers (bruns, 2019; møller & bechmann, 2019). which procedures are employed by the platform, are these transparently reported, and was the obtained data set examined for inherent biases? conducting big data driven research, such as sma, can lead to unintended insights about user (groups) which were not of primary interest to the researcher. if these 1 a helpful approach at this point might be stages of consent by porter (mckee & porter, 2009) australasian journal of information systems jung et al. 2022, vol 26, research on applied ethics ‘cambridge moralica’ 11 insights touch on sensitive areas (e.g., hate speech, cyberbullying, disinformation), researchers might have a responsibility to act for example by informing authorities to avoid harm (european commission, 2018, p. 14). the method reflection stage involves the aspects validity and algorithms. as in every research project, the validity of applied sma methods should be ensured. thus, methods should not be chosen habitually but selected based on their ability to accurately measure the aspects relevant to the research question (burton-jones, 2009). the use of algorithms, for example in the form of supervised or unsupervised machine learning approaches to identify patterns or clusters in the data set, entails several ethical hurdles (franzke et al., 2020; ieee, 2020). major concerns involve possible algorithmic biases, which are closely linked to biases in the training data (johnson, 2020) and discrimination, as well as the black box problem, which is related to the question of algorithmic explainability, accountability, and traceability (martin, 2019; mittelstadt, allo, taddeo, wachter, & floridi, 2016). do i understand the functionality of the algorithm, and can i explain the results? who is responsible for potential harm resulting from the use of this algorithm? am i aware of the limitations, such as the shortcomings of natural language processing approaches for detecting sarcasm and irony, which can influence the results of a sentiment analysis (eke, norman, liyana shuib, & nweke, 2020)? the publication reflection stage comprises the ethical considerations informed consent, sharing & storing of data, gender diversity, cultural dimensions, and future implications. as already mentioned in the tracking reflection, informed consent is essential for protecting the privacy of research subjects in sma. if informed consent cannot be obtained prior to data tracking, another approach is to ask for informed consent of those subjects whose accounts or content are analysed or mentioned in the paper prior to publication (franzke et al., 2020). studies show that users are only slightly concerned regarding the use of their social media data for academic research but they do expect to be asked for their consent (williams, burnap, & sloan, 2017). when publishing sma research, sharing & storing of data becomes relevant as sma researchers need to evaluate how to store and share data while complying with legal and ethical considerations. while research grant providers often require an open access publishing of results and data sets, and researchers increasingly see the advantages of open data, this principle collides with the terms and conditions of social media platforms like twitter and might lead to re-identification of research subjects (stieglitz et al., 2020; twitter, 2020). furthermore, sma researchers should reflect on gender diversity and cultural dimensions of their analysis and results prior to publication. the sma researcher should be aware that social media platforms often do only display a minor part of society, which not always include diverse gender and cultural backgrounds. thus, findings are often not generalizable to society and sma researchers should consider and report the limitations of their data and analysis (zook et al., 2017). lastly, the future implications refer to possible consequences of sma research. this point is closely related to the idea of avoiding harm doing good in the meta reflection stage. what effect might my analysis have on the individuals and groups involved in the study, and on society? the awareness that my research has future implications is tied to the idea of rri (stahl, 2013) and the approach of ethics by design (d’aquin et al., 2018), but is also an integral part of ethical guidelines for example reflected in the principles of the acm (acm, 2018). australasian journal of information systems jung et al. 2022, vol 26, research on applied ethics ‘cambridge moralica’ 12 6 design cycle evaluation methods 6.1 round 1 expert interviews the first version of the framework was evaluated with twelve interviews conducted with experts in digital ethics and sma research on the professorial level. the selection criteria included expertise in the use of sma or expertise in related ethical questions, and diversity in terms of cultural background and gender. table 2 provides an overview of the interviewees. id job title (gender) research discipline area of expertise country length r1 professor (f) is sma and public communication nor 1h 03m r2 researcher (f) ethics ethics and digital media ger 1h 07m r3 professor (m) media studies ethics and computational technologies nor 30m r4 professor (m) digital media studies sma on online communities aus 1h 14m r5 professor (m) is sma of social media crisis communication nor 1h 14m r6 professor (m) ethics information ethics usa 46m r7 professor (f) communication science sma of online social phenomena bra 33m r8 professor (f) communication science sma on virtual communities bra 29m r9 professor (f) computer science natural language processing and information retrieval nl 1h 08m r10 professor (m) is sma of social media crisis communication ger 1h 02m r11 professor (f) ethics media and information ethics ger 42m r12 professor (m) ethics digital ethics uk 55m table 2. overview of the interviewed experts. the semi-structured interviews took place via skype, zoom, or google hangouts between may and august 2020. all participants provided their informed consent to record and analyse their interviews. the data was analysed and stored within one research institution. the interview guidelines, which were slightly adapted for sma researchers and ethicists respectively, consisted of six question blocks: the introductory questions aimed at evaluating the expertise and focus of the interviewees. the question blocks two-five asked for the ethical challenges and implications of each phase of the sma research process. then, the experts evaluated the extended framework, which was provided one week in advance. the experts were encouraged to add ethical implications or to propose changes. lastly, the interviewees were asked for general comments and if the framework would be helpful for them and their research. the transcription was conducted with the tool maxqda following kuckartz’ transcription guidelines for computer-aided content analysis (kuckartz, 2016). the qualitative content analysis was conducted according to the rules of mayring (2014). the category system was based on deductive codes, which were derived from the literature, for example informed consent and privacy are deductive codes, one category derived from the interview input, for example, is protecting the researcher. to ensure the reliability of the coding process, one out of twelve interviews was coded separately by two coders. cohen’s κ coefficient of 0.8 confirmed the agreement between the coders to be substantial (landis & koch, 1977). australasian journal of information systems jung et al. 2022, vol 26, research on applied ethics ‘cambridge moralica’ 13 6.2 round 2 focus groups to evaluate the artefact with regards to the defined acceptance criteria of “awareness for ethics” and “applicability of the artefact”, two confirmatory focus group discussions were conducted. according to tremblay et al. (2010), focus groups can be used to refine and evaluate (it) artefacts in dsr. focus groups were chosen as the evaluation method because they allow the evaluation of the artefact in a realistic scenario and offer very rich qualitative interaction data. morgan (1988) defines four to twelve participants as suitable. to keep the level of interaction high and to prevent “social loafing”, we included four participants in each group (tremblay et al., 2010). corresponding to the application environment of the artefact, the participants were selected based on their experience with sma and their general level of research expertise. as the artefact should effectively support researchers from different experience levels, the focus groups included phd candidates and senior researchers. table 3 provides an overview of the focus groups. id job title (gender) research discipline area of expertise country fg 1 focus group 1 f1 phd student (f) computer science sma for health forums nl f2 assistant professor (f) is sma for enterprise social networks lie f3 phd student (f) is sma for crisis communication ger f4 phd student (m) is sma and ai ethics ger fg 2 focus group 2 f5 phd student (f) is sma for enterprise social networks ger f6 phd student (f) is sma and vr educational contexts ger f7 phd student (f) is sma for crisis communication ger f8 post-doctoral researcher (m) is sma for crisis communication aus table 3. overview of the focus group participants and their prescribed pseudonyms. all participants provided their informed consent to record and analyse the discussion. prior to the two confirmatory focus groups, a pilot focus group with six student assistants with experience in research and sma was conducted via google hangouts to test and refine the guidelines and the procedure. the discussion (total of 60min) was structured as follows: after welcoming the participants and summarising the contents of the discussion, all participants were asked to introduce themselves briefly to establish a basic familiarity within the group. afterwards, the following task was presented: the group was asked to imagine being a research team aiming to develop a research design to investigate the different actors who are involved in the discussion about covid-19 on social media. then, the group was invited to a virtual google jamboard where they were asked to write down a research design and the connected ethical considerations (15 minutes). as an inspiration for the different steps, the sma framework by stieglitz et. al (2014, 2018) was given. due to the challenging task of designing a research project in a team and the limited time, it was unlikely that participants would touch on all relevant ethical considerations included in the framework in the first round of discussion. therefore, as an indication of the perceived importance and completeness of ethical considerations in the developed research design, we asked each participant separately to rate their agreement to the following two statements on a 5-point likert scale: “ethical considerations played an important role in the development of australasian journal of information systems jung et al. 2022, vol 26, research on applied ethics ‘cambridge moralica’ 14 the research design” and “i am confident that we discussed all ethical considerations that are relevant for sma research” (i don’t agree (1) to i fully agree (5)). afterwards, the extended sma framework was explained by the moderator. participants were then asked to discuss if and how they would refine their research design based on the extended sma framework (15 minutes). after this discussion, another questionnaire was filled out individually by the participants, to avoid a social desirability bias. it included five questions, which aimed at evaluating the applicability of the framework. in the final 15 minutes, the participants were asked to jointly discuss the framework with a special focus on the evaluation criteria “awareness for ethics” and “applicability of the artefact”. the guidelines for the focus group discussions are included in the appendix b. we consider the first evaluation criterion to be fulfilled if the extended framework stimulates and extends the awareness of ethical considerations in the development of an sma research design. we operationalised this by 1) comparing the ethical considerations mentioned in the first discussion supported by the original sma framework (stieglitz et al., 2014) with those mentioned in the second round of discussion supported by the proposed extended framework, and by 2) analysing statements on the extended framework during the discussion which support or challenge its capacity to stimulate and extend ethical awareness. we consider the second evaluation criterion, “applicability of the artefact”, to be met if the extended framework is perceived as a useful and applicable tool by sma researchers. we operationalised this criterion by 1) analysing the comments on and interactions with the extended framework during the discussion, 2) the direct feedback provided on the extended framework at the end of the focus group discussion, and 3) the private questionnaire ratings on the five statements assessing the applicability of the framework (see appendix b). the analysis of the focus group data was conducted according to the described approach for the analysis of focus group data in is research by nili et al. (2017). as this focus group was conducted virtually, non-verbal interaction data, which requires an interpretation of movements and gestures in interaction with other participants, was not considered. instead, we focused on the content and verbal interaction data. 6.3 round 3 survey of initial experts, senior sma researchers, and ethics boards in the final evaluation, a survey was conducted among six ethics board members to collect reasons for criticizing or rejecting sma studies and consulted sources for ethical evaluation. this was done to ensure that all mentioned aspects are covered by the final artefact. we chose this approach over a detailed evaluation of the ethical issues in the artefact as it covers all phases of the sma process in a small-step manner and exceeds the scope of work of the ethics boards which only make one-time decisions before the start of a study. thus, a second survey was conducted to evaluate the framework itself involving four experts from the first evaluation round as well as seven additional sma researchers to do justice to the circular nature of dsr and to test the artefact in a realistic scenario. the surveyed sma researchers were asked to apply the extended framework to a planned or recently conducted sma study. based on the use case, they were asked to indicate for each ethical issue whether it sparked new thoughts (acceptance criterion i) and to provide general feedback in open text form. additionally, they were asked to answer eleven questions on a 5-point likert scale (“i don’t australasian journal of information systems jung et al. 2022, vol 26, research on applied ethics ‘cambridge moralica’ 15 agree” to “i fully agree”) on the applicability of the framework and the reflective questions (acceptance criterion ii). table 4 shows the expert sample for this round. id job title (gender) research discipline area of expertise country e1 professor (f) computer science natural language processing nl e2 professor (f) computer science and mathematics ethnomathematics applied statistics idn e3 professor (f) is sma and public communication nor e4 assistant professor (m) psychology self-regulation consumer research ger e5 professor (f) media studies internet regulation nor e6 post-doc and docent (f) communication science technology and organization fin ethic board members professors from the first interview round r4, r5, r8, r10 for details please see table 2 professors who were confronted with the artefact for the first time r13 professor (m) is sma in organizational contexts ger r14 professor (m) psychology sma and politics ger r15 professor (m) is sma of social media crisis communication ger r16 professor (f) communication science sma on social movements bra r17 professor (m) communication science sma and politics and journalism ger r18 professor (m) is sma in organizational contexts ger r19 assistant professor (m) is sma and communication uk table 4. overview of last evaluation round with professors and ethic board members 7 design cycle evaluation results 7.1 interview results the aim of the expert interviews was to evaluate the completeness of the first prototype of the artefact. the results of the interviews have led to several additions and changes, which will be outlined in the following. the meta reflection phase was moved prior to the ‘research domain’ in the framework to emphasise the overarching nature of these questions, regardless of the specific research project. additionally, the subcategory of avoiding harm doing good was changed to general orientation of research and orientation towards the common good. the former includes questions related to the beneficiaries of the research and the gut-feeling of the researcher about the research aim. the latter was included to account for research which might have to violate certain ethical principles to serve the common good. to emphasise that the tracking reflection should be done prior to data tracking, it was renamed to pre-tracking reflection. furthermore, data gathering was added to point to the ethical implications of alternative or complementary data sources, such as purchased datasets or existing (decontextualised) datasets, and their impact on the analysis and results. further, the interviewees highlighted the complexity of informed consent, for example in terms of cultural diversity (“this does not make a lot of sense in a highly relational culture, so you have to develop a form of a kind of collective informed consent”, r03), and feasibility due to low number of replies on social media and that “there is no other sensible way to reach these people” (r04). overall, interviewees often regarded consent more relevant for small data sets, private australasian journal of information systems jung et al. 2022, vol 26, research on applied ethics ‘cambridge moralica’ 16 accounts, case-specific analyses than for mass data, public accounts, and aggregated analyses, thus, this distinction was added. the related concern of re-identification of individuals after publication was added to the publication reflection. due to different national legislations, sharing & storing was added as researchers might have to agree on gdpr-compliant services and procedures. also, when sharing data sets, r06 highlighted that, “if you merge it [your data] with my data, i might now suddenly be able to identify your subjects and there is no fault of yours other than the fact that you shared it with me”. therefore, the aspect of data merging was added to the preparation reflection. in the preparation reflection, before analysing the data, the protection of the researcher should be considered. r03 mentioned, “we have to offer researchers more and more sophisticated ways of protecting their identity online so that they don’t become a target”. but also analysing hate speech or misinformation on social media can be emotional stress. lastly, r12 emphasised that, “you have to make value judgements on the data before you process the data, before you analyse the data. […] i think it’s important to understand that statistical is also based on value judgements […] whether you choose a particular confidence level of 5 % or 1 % or .5 % will determine what comes out as significant in the end. and that’s not a natural occurrence, that is a value judgement that is driven by something and has consequences for whatever you can say in the end.” hence, we added that researchers should reflect on the limitations of their data and analysis and make those explicit. in the method reflection, four aspects were added. first, the potential exposure of sensitive attributes or relationships of social media users, for example, when conducting social network analysis. biases were added to raise questions about unequal representation in the data and the generalisability of the results. furthermore, aggregated analyses might not harm individuals, but the derived insights could have harmful implications for groups of people. to address these ethical concerns, researchers should examine whether their results are supported by other data. in the last reflection phase, the publication reflection, gender, and cultural aspects were merged in the more inclusive promoting diversity aspect, asking researchers to discuss diversity in their studies and to reflect their own socio-economic, cultural, and personal background. lastly, giving back was added to inspire researchers to think about ways of compensating participants and sharing research benefits or insights with them. 7.2 focus group results the confirmatory focus group evaluation of the second version of the framework, which resulted from the expert interviews, targeted two evaluation criteria: “awareness for ethics” and “applicability of the artefact”. table 5 provides a summary of the supporting and opposing or challenging evidence for the two evaluation criteria. in the first survey of fg1, one reply is missing due to technical issues. in fg1, while participants perceived ethical considerations to have played a moderately important role (m = 3.34, sd = 1.15), the perceived completeness of discussed ethical considerations was very low (m = 1.34, sd = 0.58). hence, possibly, the participants could have discussed more ethical hurdles when given more time. this limits the insights for the first evaluation criterion derived from the pre-post analysis of discussed ethical considerations in fg1. in fg2, however, the perceived importance and completeness was medium to high (m = 4; sd = 0.82; m = 3.25, sd = 1.26), suggesting that the additional ethical reflection in the second round, for example related to power issues (see appendix b), can be assigned more clearly to the guidance provided by the extended framework. moreover, the interactions and statements during the australasian journal of information systems jung et al. 2022, vol 26, research on applied ethics ‘cambridge moralica’ 17 discussions, such as “and i never really thought about this that we also might have an obligation to, for example, report fake news” or “i am really thinking all the time is this everything that we should ask ourselves in this step” and the direct feedback during the focus groups imply that the extended framework stimulates further ethical reflection. support challenges a w a r e n e s s 1 extension of discussed ethical considerations (e.g., unintended insights, cultural biases, power issues; see appendix for an overview of both discussion rounds) not all keywords are self-explanatory, especially if researchers are not familiar with the referenced ethical debate (discussion not extended by those keywords) 2 stimulating consideration (and discussion) of ethical questions starting point for further ethical reflection beyond the extended framework raising awareness for the importance of ethics in sma unclear purpose: framework as a “rule book” which researchers have to follow? unclear scope: what is unique to sma ethics? a p p l i c a b i l i t y 1 structuring the discussion memory aid overwhelming representation difficult to understand 2 saving time & reducing workload with concise overview confidence in research design avoiding ethical issues to be overlooked framework as a point of reference for ethical standards in sma research & reports how to translate reflection into actions and which actions to choose additional material (reflective questions, best practices, examples) needed 3 medium to high ratings on perceived support for planning and conducting ethical sma research (m = 4,sd = 0.53)), willingness to use the framework (m = 3.88,sd = 0.83) ), and added value beyond existing guidelines (m = 3.75,sd = 0.46) ) mixed ratings on understandability of the framework (m = 3.5, sd = 0.93) table 5. summarised evaluation of the criteria “awareness for ethics” and “applicability of the artefact”. the perceived applicability of the framework can be outlined by the following statements: “there is a lot out there about ethics and also social media but what i like about the figure is that it brings it somehow to the point [...] you don't have to read 10 papers to find all the issues that might occur when doing ethical sma research” and, “it seems useful because there are a couple of things where you think: oh yeah, i might have thought about that but it would have taken me more time or... i would have thought about it at a too late stage”. further feedback on the framework addressed the terminology, the need for a contact person for ethical guidance, and the challenge of balancing ethical considerations and research interests. thus, the focus group evaluation highlighted that for the extended sma framework to effectively support ethical reflection and be useful for sma researchers, the ethical issues in the framework require additional explanatory material. to address this feedback, we compiled a manual including a short description of each issue, relevant sources, and a list of reflective questions to complement the extended sma framework (see appendix a). thus, the third australasian journal of information systems jung et al. 2022, vol 26, research on applied ethics ‘cambridge moralica’ 18 version of the artefact consists of the extended sma framework and the supplementary material. both were once again evaluated regarding the acceptance criteria. 7.3 results of the final evaluation in the final evaluation round, the sma researchers applied the artefact to a variety of sma topics and methodological approaches. for example, to studies with a focus on enterprises or the analysis of infodemics and pop-cultural phenomena with qualitative and quantitative methods. regarding ethical awareness, twelve of the 34 ethical issues each provided additional food for thought for at least six of the sma researchers. for each of the remaining 22 ethical issues, at least one of the sma researchers indicated that it stimulated ethical reflection. thereby, the ethical issue „common good“ of the meta reflection stage was selected most frequently (n = 9) and discussed in the open feedback as well. for example, r18 asked himself in how far it would be unethical to do research which has no contribution to the common good. further, r5 emphasized that the focus of one's own study (e.g., company or society) influences the individual importance of various sub-aspects, which makes every ethical evaluation an individual process. as all participants were senior researchers with extensive experience, several noted that the had already implicitly taken many points into consideration (r18, r5, r19), sometimes due to special industry standards of their application environment (r18). yet, the participants shared the positive impression that the framework would support them in planning and conducting sma studies (m = 4.64, sd = 0.5) and provides additional value to existing guidelines (m = 4.36, sd = 0.67). the easiness to understand the framework was rated slightly lower (m = 4.18, sd = 0.6). yet, all participants moderately to strongly agreed that they would apply the framework in their research (m = 4.45, sd = 0.69) and recommend it to their phds and students (m = 4.45, sd = 0.82). more divided was the opinion of the participants if the framework reduced the methodological quality of their work (m = 2.1, sd = 1.51). while nine researchers could not identify any negative impact, two were sure that there was a negative impact on their methodological quality. the support by the initial list of reflective questions was rated as high (m = 4.55, sd = 0.82). the easiness to understand the list of initial reflection questions was rated a bit higher than the overall framework (m = 4.27, sd = 0.79). similar to the overall framework, the participants saw added value of the initial reflection questions to existing guidelines (m = 4.45, sd = 0.69). all participants moderately to strongly agreed that they would apply the list of reflection questions in their research (m = 4.36, sd = 0.67) and that they would recommend using the reflective questions to their phds and students (m = 4.72, sd = 0.47). the survey of six ethics board members found that the main reasons ethics boards rejected research designs was due to privacy-related issues which were not considered sufficiently by research teams. respondents mentioned privacy as a generic term, but also profiling, reidentification, anonymity, and the legal issue of informed consent and gdpr. furthermore, the respondents mentioned neglected sensitivity of data, planned manipulation of subjects, and protection of the researcher from negative influences in the form of false news and violent scenes as points to be considered. two other meta-themes raised were the limitations of the open-science ideals in social media research due to legal boundaries and taking potential negative impacts of sma into consideration. to guide their evaluation, the respondents consulted different sources such as institutional bodies (e.g., data protection officer), or articles, books, and existing guidelines, such as legal considerations (e.g., gdpr). all agreed that additional guidance with a more specific hands-on approach is missing. australasian journal of information systems jung et al. 2022, vol 26, research on applied ethics ‘cambridge moralica’ 19 8 discussion the evaluation rounds involving two focus groups, individual application of the framework to a realistic scenario, and consultation of ethics boards supported both acceptance criteria, indicating that the artefact provides a valuable addition to the environment of sma researchers. specifically, the extended sma framework and the accompanying reflective questions cover the crucial points for ethical approval of ethics boards from different cultures and regulatory environments and thus, could help sma researchers to prepare better for such evaluations. thereby, the framework goes beyond the one-time evaluation of a study by providing specific guidance throughout the entire research process. in the evaluation with a group of culturally and thematically diverse, highly experienced sma researchers, the artefact was perceived to stimulate ethical reflection. the combination of the extended sma framework and the reflective questions were rated as highly applicable and useful for researchers with less experience in sma (students, phds) as well. nevertheless, the evaluation also highlighted certain challenges which need to be considered while applying the extended sma framework and which we will discuss in the following section. one of these challenges concerns the interpretation of the ethical issues in each phase as a ‘rule book’ that the researcher must follow and the wish for specific ethical best practices (cf. table 5). this perception points to a prevalent, yet problematic ‘tick the box’-approach to research ethics (dawson, lignou, siriwardhana, & o’mathúna, 2019; grieves, 2019). while ethical committees examine and approve research designs only prior to the start of the research project, these approvals are often based on standardised criteria, which does not do justice to the explorative character of sma. due to the unique challenges of sma research, such as quickly changing circumstances (new social media platform(s), changing terms and conditions, new features etc.) and the interdisciplinary nature of sma, providing explicit rules to follow would likely fail to capture the reality of sma research. thus, by providing descriptions of possible ethical issues and reflective questions we aim to stimulate reflection, yet not every aspect is equally relevant for every sma study. especially given that sma research is also used in time-pressing and societally highly relevant domains such as disaster response (mirbabaie et al., 2020), some ethical considerations might be neglectable in those cases (“common good”, appendix a). however, these ethical deliberations are complex and need to be evaluated on a case-by-case basis (zook et al., 2017). hence, similar to the ethics by design approach (d’aquin et al., 2018) our study shows that sma research ethics require open reflection (i.e., adaption, interpretation, dialectic) throughout the research process and iterative adjustments (i.e., tailored to the research process stages). the extended sma framework (figure 4) in combination with the reflective questions (appendix a) embody these principles. the optimal presentation of the framework is an area for future research. based on the feedback from the interviews and focus groups, we imagine an interactive website which allows users to gradually “zoom in” on the stage of the research process they are currently interested in to avoid the framework to be overwhelming. moreover, the website might incorporate ethical questions tailored to the users’ sma project. such a website could also provide a platform for researchers to discuss research designs and best practice examples. lastly, it is noted that the different reflective questions are often interrelated or even overlapping. for example, the question of informed consent is always linked to the value of transparency, which relates to the autonomy and privacy of the user. to strike balance australasian journal of information systems jung et al. 2022, vol 26, research on applied ethics ‘cambridge moralica’ 20 between providing concrete questions tailored to sma processes without oversimplifying abstract ethical concepts is an ongoing challenge. the artefact in its final form marks a first step towards providing much-needed applicable ethical guidance for sma researchers (taylor & pagliari, 2018) and shall serve as a basis to ignite further discussions and subsequent research within the field on how to ethically approach sma. it can be understood as research routed in explicit morality, setting the foundation for deeper thoughts and a step towards ethical theory (stahl, 2012). we encourage researchers to investigate additional methods and useful theory that supports the ethical reflection of sma researchers in the future, including the mapping of additional maybe institutional or other hindrances to why ethics guidelines are accepted and used. we are convinced that there will be a rich vein for further research also on the theoretical underpinnings of the design and usage of ethics guidelines in sma. figure 4. final version of the extended sma framework with five ethical reflection phases. 9 limitations and future research ethics and sma research are constantly evolving fields. thus, the presented framework can only be as good as the current knowledge base. to succeed at providing sma researchers with practical guidance, it will be necessary to regularly check and update the framework. although the framework (and supplementary material) was evaluated with the help of twelve expert interviews, two focus groups, and applied by individual experts to a realistic scenario, applying the framework in practice and over longer period will provide richer insights. simultaneously, the optimal presentation of the framework should be elaborated. by testing the artefact in diverse cultural contexts, the limitation that the authors all have a western european background can be overcome. further, it would be helpful to conduct a systematic literature review on the prevalent ethical topics in sma research to map the debate, indicate the overlaps of different concepts, and thus increase the applicability of those discourses to the research process. australasian journal of information systems jung et al. 2022, vol 26, research on applied ethics ‘cambridge moralica’ 21 10 conclusion to decide what is morally right or wrong in the research process can be a major challenge for sma researchers, as they are confronted with an abundance of theoretical ethical debates, dilemmas, ethical guidelines, and legislations. the excessive demands, a possible lack of sensitisation for ethical issues, and high time pressure in academia can lead sma researchers to squander the chance to conduct more ethical research. in this paper, we present a framework for ethical sma research which assists researchers to reflect the ethical implications of their decisions throughout the entire sma research lifecycle. the sma framework by stieglitz et al. (2014, 2018) was chosen as a basis and was extended by five ethical reflection stages: the meta–, pre-tracking–, preparation–, methodological–, and publication reflection. the reflection stages provide researchers with thought-provoking impulses to ethically reflect their research design and implementation. to better understand which questions might be relevant for their specific studies, a list of additional descriptions and reflective questions is provided. based on ‘ethics by design’ principles, the framework for ethical sma research can accompany researchers throughout the whole research lifecycle, and thus, can function as the practical guidance researchers need to conduct more ethical sma research. the improvement of individual moral decisions can lead to an improvement not only on an individual level (e.g. privacy, autonomy, protection of the researcher) but might have positive implications on the society as a whole (e.g. future implications of the research, cultural dimensions and diversity). besides our practical contribution, the study highlights the ethical concepts which are of relevance for is research and the need to spark discussions in our community to increase the positive impact of is research on both academia and society. acknowledgement this project has received funding from the european union’s horizon 2020 research and innovation programme under the marie skłodowska-curie grant agreement no 823866. references abbasi, a., sarker, s., & chiang, r. (2016). big data research in information systems: toward an inclusive research agenda. journal of the association for information systems, 17(2), i– xxxii. https://doi.org/10.17705/1jais.00423 acm. (2018). acm code of ethics and professional conduct. retrieved june 18, 2020, from https://www.acm.org/code-of-ethics adjerid, i., peer, e., & acquisti, a. (2018). beyond the privacy paradox: objective versus relative risk in privacy decision making. mis quarterly, 42(2), 465–488. https://doi.org/10.25300/misq/2018/14316 ais. (2014). code of research conduct. retrieved june 17, 2020, from https://aisnet.org/page/admbullcresearchcond?&hhsearchterms=%22code%22 anstead, n., & o’loughlin, b. (2015). social media analysis and public opinion: the 2010 uk general election. journal of computer-mediated communication, 20(2), 204–220. https://doi.org/10.1111/jcc4.12102 beauchamp, t. l. (2008). the nature of applied ethics. in r. g. frey & c. h. wellman (eds.), a companion to applied ethics (pp. 1–14). wiley. retrieved from https://books.google.de/books?id=qws8gf4f4iuc boyd, danah, & crawford, k. (2012). critical questions for big data. information, communication & society, 15(5), 662–679. https://doi.org/10.1080/1369118x.2012.678878 australasian journal of information systems jung et al. 2022, vol 26, research on applied ethics ‘cambridge moralica’ 22 bruns, a. (2019). after the ‘apicalypse’: social media platforms and their fight against critical scholarly research. information, communication & society, 22(11), 1544–1566. https://doi.org/10.1080/1369118x.2019.1637447 burton-jones, a. (2009). minimizing method bias through programmatic research. mis quarterly, 33(3), 445–471. chen, h., zheng, z. e., & ceran, y. (2016). de-biasing the reporting bias in social media analytics. production and operations management, 25(5), 849–865. https://doi.org/10.1111/poms.12509 cruz-jesus, f., vicente, m. r., bacao, f., & oliveira, t. (2016). the education-related digital divide: an analysis for the eu-28. computers in human behavior, 56, 72–82. https://doi.org/10.1016/j.chb.2015.11.027 d’aquin, m., troullinou, p., o’connor, n. e., cullen, a., faller, g., & holden, l. (2018). towards an “ethics by design” methodology for ai research projects. in proceedings of the 2018 aaai/acm conference on ai, ethics, and society (pp. 54–59). new york, ny, usa: acm. https://doi.org/10.1145/3278721.3278765 dawson, a., lignou, s., siriwardhana, c., & o’mathúna, d. p. (2019). why research ethics should add retrospective review. bmc medical ethics, 20(1), 68. https://doi.org/10.1186/s12910-019-0399-1 eke, c. i., norman, a. a., liyana shuib, & nweke, h. f. (2020). sarcasm identification in textual data: systematic review, research challenges and open directions. artificial intelligence review, 53(6), 4215–4258. https://doi.org/10.1007/s10462-019-09791-8 ess, c. (2005). “lost in translation”?: intercultural dialogues on privacy and information ethics (introduction to special issue on privacy and data privacy protection in asia). ethics and information technology, 7(1), 1–6. https://doi.org/10.1007/s10676-005-0454-0 ess, c., & association of internet researchers. (2002). ethical decision-making and internet research: recommendations from the aoir ethics working committee. retrieved september 21, 2020, from https://aoir.org/reports/ethics.pdf european commission. (2018). ethics in social science and humanities. retrieved september 21, 2020, from https://ec.europa.eu/research/participants/data/ref/h2020/other/hi/ h2020_ethics-soc-science-humanities_en.pdf european parliament and european council. (2018). regulation (eu) 2016/679 / general data protection regulation. retrieved september 18, 2020, from https://eurlex.europa.eu/legal-content/en/txt/?qid=1532348683434&uri=celex:02016r067920160504 franzke, a. s., bechmann, a., zimmer, m., ess, c., & association of internet researchers. (2020). internet research: ethical guidelines 3.0. retrieved september 21, 2020, from https://aoir.org/reports/ethics3.pdf grieves, g. (2019). just tick the box. in the meeting of aesthetics and ethics in the academy (pp. 101–109). routledge. https://doi.org/10.4324/9780429197321-8 günther, w. a., rezazade mehrizi, m. h., huysman, m., & feldberg, f. (2017). debating big data: a literature review on realizing value from big data. journal of strategic information systems, 26(3), 191–209. https://doi.org/10.1016/j.jsis.2017.07.003 guri-rosenblit, s. (2015). internationalization of higher education: navigating between contrasting trends. in a. curaj, l. matei, r. pricopie, j. salmi, & p. scott (eds.), the european higher education area: between critical reflections and future policies (pp. 13–26). cham: springer international publishing. https://doi.org/10.1007/978-3-319-20877-0_2 australasian journal of information systems jung et al. 2022, vol 26, research on applied ethics ‘cambridge moralica’ 23 hagendorff, t. (2020). the ethics of ai ethics: an evaluation of guidelines. minds and machines, 30(1), 99–120. https://doi.org/10.1007/s11023-020-09517-8 hevner, a. r. (2007). a three cycle view of design science research. scandinavian journal of information systems, 19(2), 87–92. hevner, a. r., march, s. t., park, j., & ram, s. (2004). design science in information systems research. mis quarterly, 28(1), 75-105. https://doi.org/10.2307/25148625 hongladarom, s. (2017). internet research ethics for the social age. (m. zimmer & k. kinderkurlanda, eds.), internet research ethics in a non-western context. peter lang us. https://doi.org/10.3726/b11077 ieee. (2020). ieee code of ethics. retrieved june 17, 2020, from https://www.computer.org/education/code-of-ethics jimenez-marquez, j. l., gonzalez-carrasco, i., lopez-cuadrado, j. l., & ruiz-mezcua, b. (2019). towards a big data framework for analyzing social media content. international journal of information management, 44, 1–12. https://doi.org/10.1016/ j.ijinfomgt.2018.09.003 jobin, a., ienca, m., & vayena, e. (2019). the global landscape of ai ethics guidelines. nature machine intelligence, 1(9), 389–399. https://doi.org/10.1038/s42256-019-0088-2 johnson, g. m. (2020). algorithmic bias: on the implicit biases of social technology. synthese. https://doi.org/10.1007/s11229-020-02696-y jouhki, j., lauk, e., penttinen, m., sormanen, n., & uskali, t. (2016). facebook’s emotional contagion experiment as a challenge to research ethics. media and communication, 4(4), 75–85. https://doi.org/10.17645/mac.v4i4.579 kaplan, a. m., & haenlein, m. (2010). users of the world, unite! the challenges and opportunities of social media. business horizons, 53(1), 59–68. https://doi.org/10.1016/j.bushor.2009.09.003 kekes, j. (1986). moral intuition. american philosophical quarterly, 23(1), 83–93. kuckartz, u. (2016). qualitative inhaltsanalyse: methoden, praxis, computerunterstützung. (3rd ed.). weinheim und basel: beltz juventa. laacke, s., mueller, r., schomerus, g., & salloch, s. (2021). artificial intelligence, social media and depression. a new concept of health-related digital autonomy. the american journal of bioethics, 21(7), 4–20. https://doi.org/10.1080/15265161.2020.1863515 landis, j. r., & koch, g. g. (1977). the measurement of observer agreement for categorical data. biometrics, 33(1), 159–174. https://doi.org/10.2307/2529310 lee, i. (2018). social media analytics for enterprises: typology, methods, and processes. business horizons, 61(2), 199–210. https://doi.org/10.1016/j.bushor.2017.11.002 leurs, k. (2017). feminist data studies: using digital methods for ethical, reflexive and situated socio-cultural research. feminist review, 115(1), 130–154. https://doi.org/10.1057/s41305017-0043-1 lomborg, s., & bechmann, a. (2014). using apis for data collection on social media. the information society, 30(4), 256–265. https://doi.org/10.1080/01972243.2014.915276 marabelli, m., & markus, m. l. (2017). researching big data research: ethical implications for is scholars. retrieved from https://www.hhs.gov/ohrp/regulations-and-policy/ regulations/nprm markham, a., buchanan, e., & association of internet researchers. (2012). ethical decisionmaking and internetresearch:recommendations from the aoir ethics working committee(version 2.0). retrieved september 21, 2020, from https://aoir.org/ reports/ethics2.pdf australasian journal of information systems jung et al. 2022, vol 26, research on applied ethics ‘cambridge moralica’ 24 martin, k. (2019). designing ethical algorithms. mis quarterly executive, 18(2), 129–142. https://doi.org/10.17705/2msqe.00012 mayring, p. (2014). qualitative content analysis: theoretical foundation, basic procedures and software solution. klagenfurt. retrieved from https://nbn-resolving.org/urn:nbn:de:0168-ssoar395173 mirbabaie, m., bunker, d., stieglitz, s., marx, j., & ehnis, c. (2020). social media in times of crisis: learning from hurricane harvey for the coronavirus disease 2019 pandemic response. journal of information technology, 35(3), 195–213. https://doi.org/10.1177/ 0268396220929258 misirlis, n., & vlachopoulou, m. (2018). social media metrics and analytics in marketing – s3m: a mapping literature review. international journal of information management, 38(1), 270–276. https://doi.org/10.1016/j.ijinfomgt.2017.10.005 mittelstadt, b. (2017). from individual to group privacy in big data analytics. philosophy & technology, 30(4), 475–494. https://doi.org/10.1007/s13347-017-0253-7 mittelstadt, b. (2019). principles alone cannot guarantee ethical ai. nature machine intelligence, 1(11), 501–507. https://doi.org/10.1038/s42256-019-0114-4 mittelstadt, b., allo, p., taddeo, m., wachter, s., & floridi, l. (2016). the ethics of algorithms: mapping the debate. big data & society, 3(2), 205395171667967. https://doi.org/10.1177/2053951716679679 møller, l. a., & bechmann, a. (2019). research data exchange solution. retrieved from https://pure.au.dk/portal/files/177764165/research_data_exchange_solution.pdf morgan, d. l. (1988). focus groups as qualitative research: an expanded sourcebook (vol. 2). thousand oaks: sage publications. nili, a., tate, m., & johnstone, d. (2017). a framework and approach for analysis of focus group data in information systems research. communications of the association for information systems, 40, 1–21. https://doi.org/10.17705/1cais.04001 o’neil, c. (2016). weapons of math destruction: how big data increas inequality and threatens democracy. broadway books. ravn, s., barnwell, a., & barbosa neves, b. (2019). what is “publicly available data”? exploring blurred public–private boundaries and ethical practices through a case study on instagram. journal of empirical research on human research ethics, 15(1–2), 40– 45. https://doi.org/10.1177/1556264619850736 safari, b. a. (2016). intangible privacy rights: how europe’s gdpr will set a new global standard for personal data protection. seton hall l. rev. 809, 47(809). stahl, b. c. (2012). morality, ethics, and reflection: a categorization of normative is research. journal of the association for information systems, 13(8), 636-656. stahl, b. c. (2013). responsible research and innovation: the role of privacy in an emerging framework. science and public policy, 40(6), 708–716. https://doi.org/10.1093/scipol/sct067 stahl, b. c., timmermans, j., & mittelstadt, b. (2016). the ethics of computing. acm computing surveys, 48(4), 1–38. https://doi.org/10.1145/2871196 statista. (2020). number of active social media users worldwide. retrieved september 16, 2020, from https://de.statista.com/statistik/daten/studie/739881/umfrage/monatlich-aktivesocial-media-nutzer-weltweit/ stieglitz, s., dang-xuan, l., bruns, a., & neuberger, c. (2014). social media analytics. business & information systems engineering, 6(2), 89–96. https://doi.org/10.1007/s12599-014-0315-7 australasian journal of information systems jung et al. 2022, vol 26, research on applied ethics ‘cambridge moralica’ 25 stieglitz, s., mirbabaie, m., ross, b., & neuberger, c. (2018). social media analytics – challenges in topic discovery, data collection, and data preparation. international journal of information management, 39(1), 156–168. stieglitz, s., wilms, k., mirbabaie, m., hofeditz, l., brenger, b., lópez, a., & rehwald, s. (2020). when are researchers willing to share their data? – impacts of values and uncertainty on open data in academia. plos one, 15(7), e0234172. https://doi.org/10.1371/journal.pone.0234172 taylor, j., & pagliari, c. (2018). mining social media data: how are research sponsors and researchers addressing the ethical challenges? research ethics, 14(2), 1–39. https://doi.org/10.1177/1747016117738559 tinati, r., phillipe, o., pope, c., carr, l., & halford, s. (2014). challenging social media analytics. in proceedings of the 2014 acm conference on web science websci ’14 (pp. 177– 181). new york, new york, usa: acm press. https://doi.org/10.1145/2615569.2615690 tremblay, m. c., hevner, a. r., & berndt, d. j. (2010). focus groups for artifact refinement and evaluation in design research. communications of the association for information systems, 26. https://doi.org/10.17705/1cais.02627 twitter. (2020). developer agreement and policy. retrieved september 16, 2020, from https://developer.twitter.com/en/developer-terms/agreement-and-policy tye, m., leong, c., tan, f. t. c., tan, b., & khoo, y. h. (2018). social media for empowerment in social movements: the case of malaysia’s grassroots activism. communications of the association for information systems, 42(1), 408–430. https://doi.org/10.17705/1cais.04215 van schie, g., westra, i., & schäfer, m. t. (2017). get your hands dirty: emerging data practices as challenge for research integrity. in m. t. schäfer & k. es, van (eds.), the datafied society (pp. 183–200). amsterdam university press. https://doi.org/10.5117/9789462981362 whitchurch, c. (2012). expanding the parameters of academia. higher education, 64(1), 99–117. https://doi.org/10.1007/s10734-011-9483-2 wiener, m., saunders, c., chatterjee, s., dennis, a. r., gregor, s., mähring, m., & mertens, p. (2018). information systems research: making an impact in a publish-or-perish world. communications of the association for information systems, 43(1) 466–481. https://doi.org/10.17705/1cais.04326 wiener, m., saunders, c., & marabelli, m. (2020). big-data business models: a critical literature review and multiperspective research framework. journal of information technology, 35(1), 66-91. https://doi.org/10.1177/0268396219896811 williams, m. l., burnap, p., & sloan, l. (2017). towards an ethical framework for publishing twitter data in social research: taking into account users’ views, online context and algorithmic estimation. sociology, 51(6), 1149–1168. https://doi.org/10.1177/ 0038038517708140 zeng, d., chen, h., lusch, r., & li, s. (2010). social media analytics and intelligence. ieee intelligent systems, 25(6), 13–16. https://doi.org/10.1109/mis.2010.151 zook, m., barocas, s., boyd, d., crawford, k., keller, e., gangadharan, s. p., pasquale, f. (2017). ten simple rules for responsible big data research. plos computational biology, 13(3), 1– 10. https://doi.org/10.1371/journal.pcbi.1005399 australasian journal of information systems jung et al. 2022, vol 26, research on applied ethics ‘cambridge moralica’ 26 appendix a disclaimer: the following manual serves as a complementing resource to the extended framework. it resulted from the knowledge base, the interviews, and the focus group discussions. each of the following aspects relates to an often highly complexethical debate. please bear in mind that these questions and short descriptions are therefore not exhaustive. feel free to contact us in case you find something missing. ethical aspect (added in version) description & types of issues arising in sma research sources exemplary reflection questions meta reflection general orientation of research? (version 2) description: the question and goals of the research aim at a goal which is good and desirable. ire 1.0 (2020), p. 19 acm, 2018 ais, 2019 interviews ● what is your gut feeling regarding your research? ● do you feel that something could be harmful/ wrong/ problematic about your research? common good? description: the expected results of the research are beneficial to society as a whole so that minor ethical shortcomings associated with the research are justifiable. interviews ● does your research make an outstanding contribution to society? ● does your research help to solve a societytranscending grievance? ● is the impact of the research positive enough to justify minor ethical violations? power issues? (version 2) description: sma researchers can face various power issues regarding their research, e.g. dependence on social media platforms to access data (e.g., apis, web scraping), terms and conditions, and technical possibilities. this imbalance might affect possible research directions and designs. moreover, researchers often depend on thirdparty funding, with a funder potentially impacting the research through self-imposed censorship. at last, there can be a power imbalance between the researcher and the research subjects, which might result in divergent expectations regarding the research. lomborg & bechmann (2014) bruns (2019) møller & bechmann (2019) ire 3.0 (2020) ieee (2020) ● how do possible power imbalances influence your research design? how can you mitigate these influences when developing your design? ● how does the source of funding influence your research? which options do you have to safeguard your independence as a researcher? ● what is your relationship with the research subjects? what expectations do the different parties of the research have? legal description: ethics and law are ire 1.0 (2002) ● which legal australasian journal of information systems jung et al. 2022, vol 26, research on applied ethics ‘cambridge moralica’ 27 considerations? (version 1) interwoven. some ethical considerations are (partially) covered in legal frameworks (e.g., data protection and data minimization in the gdpr), which should thus be considered when planning and conducting sma research. frameworks apply to your research? are (non-european) research partners aware of and prepared to comply with gdpr regulations? expected outcomes? (version 2) description: approaching an investigation with an open mind is as important for sma researchers as for any other scientist. they must question if certain expectations or biases on their part are influencing the research design and/or analysis. the composition of the data set and the constant reflection about the inclusion of all groups of people is crucial. interviews ire 3.0 (2020), p.9 chen, zheng, & ceran, 2016 ● what are the expected outcomes of your research? how are those expectations influenced by your socio-economic / cultural background? how do they influence your research design? ● does your data collection include all user groups you want to include in your research? ethical questions of your field? (version 1) description: each field of research, faculty, or institute follows its own code of conduct and often has an open and ongoing dialogue on ethical guidelines. sma researchers should be aware of these developments, depending on which research branch they identify with they might be aware of ethics authorities. ire 1.0 (2002) ire 2.0 (2012) ire 3.0 (2020) ieee (2020) acm, 2018 ais, 2019 ● which ethical debates or considerations from your field of research are relevant to your research? transparency? (version 1) description: ethical guidelines often call for more transparency in research (e.g. open data) but social media content can in many cases be easily linked back to individuals. thus, sma researchers need to find a balance between the protection of individual privacy of social media users and the growing need for transparency in research. ire 3.0 (2020), p. 20 jobin et al. (2019) ● how can you increase the transparency of your research? which standards could you agree on with your collaborators (for example for documenting design or analysis choices)? ● can you design your research in a way that is compatible with open data / open research / data donation? pre-tracking reflection venue? (version 1) description: the venue refers to i) the actual and the perceived publicity of content to the general ire 1.0 (2002), p. 4-5 ● where does the interaction, action, and communication under australasian journal of information systems jung et al. 2022, vol 26, research on applied ethics ‘cambridge moralica’ 28 public. the more private a venue is or the more private it is perceived by users (e.g., telegram channel vs twitter), the more responsibility sma researchers have to inform and protect research subjects. moreover, it is determined by ii) the vulnerability and type of research subjects (e.g. minors, marginalized groups). the higher the vulnerability, the higher the responsibility of sma researchers to protect research subjects. study take place? ● how is it shaped by the venue and how might that influence your research and results? ● how can you protect vulnerable groups and subjects in your research? discrimination? (version 1) description: social media users and data are often not representative of society (e.g., elders might not use social media). certain actors might communicate very actively (e.g., influencers, activists, bot accounts), while others remain silent observers. unequal representation limits the informative value and generalizability of the findings and could be harmful to individuals or groups. thus, sma researchers should plan their data tracking with representation and diversity in mind to avoid or reduce potential discrimination. ire 3.0 (2020) ieee (2020) acm (2018) ● who is represented in the data you consider for your research? who is not? ● who are the subjects of your research and how might that influence your results? ● how do you include and account for diversity (gender, culture) in your research? data gathering? (version 2) description: self-collected or purchased data may be subject to power issues or discrimination. thus, sma researchers should explore options for gathering additional data sets when planning their data tracking. interviews ● can you complement your data set for example by buying data sets, using existing data sets, or alternative methods? ● (if applicable) in which context were alternative data sets created or collected? how do they affect the outcomes of your research? informed consent? (public / private) (version 1; distinction added in version 2) description: not all social media users are aware that terms and conditions often allow third-party usage of their data. thus, sma researchers should explore ways of informing social media users about the usage of their data in research. obtaining users’ informed consent is especially relevant for small data sets, closed groups, private ire 3.0 (2020) fiesler and proferes (2018) interviews ● are you tracking public or private profiles and how do you distinguish them? ● which options do you have to obtain informed consent when tracking private profiles? australasian journal of information systems jung et al. 2022, vol 26, research on applied ethics ‘cambridge moralica’ 29 profiles, and case-specific analyses as compared to mass data, public profiles, and aggregated analyses. privacy? (version 1) description: withholding or selectively sharing information about oneself (one’s privacy) is an important norm in ethics guidelines. sma researchers should be aware that social media often blurs the lines between public and private and people might share information online without being fully aware that these are public. adjerid, peer, & acquisti (2018) mittelstadt (2017) zimmer (2010) ire 1.0 (2002) acm (2018) ● are you aware of the paradox that people share information online but overlook that these are entirely public? ● which steps can you take to protect the privacy of social media users? ● do you investigate public figures or do you focus on ordinary citizens? autonomy? (version 1) description: maintaining autonomy means that social media users can make their own decisions and behave the way they desire without getting manipulated. sma researchers can affect the autonomy of social media users by applying ai in the analysis of the data to derive certain criteria and predictions about the individuals, e.g. their likelihood to suffer from depression. manipulation of timeline or newsfeed which unconsciously steers the discourse of a group in one direction can hurt autonomy, too. laacke et al. (2021) jouhki et al. (2016) ● are you planning interventions (e.g. on social media) which could be seen as manipulative or a restriction of the users’ autonomy? ● are you planning to derive results which make predictions about individual users? data minimisation? (version 1) description: data minimisation refers to collecting the minimal amount of (private) data that is sufficient to answer a research question. ire 3.0 (2020) møller and bechmann (2019) ● even if you are a big data researcher are there ways of minimising the collection of (private) data? ● how can you refine the time frames for tracking, the selection of keywords etc. to adhere to this principle? ● how will you deal with collected data that does not contribute to answering your research question? sharing & storing (pre publication)? description: safeguarding data security can be subject to both ethical and legal frameworks. interviews ● how do you plan to store your data? is it possible to share the australasian journal of information systems jung et al. 2022, vol 26, research on applied ethics ‘cambridge moralica’ 30 (version 2) data with others? ● is your data storage and sharing compliant with legal frameworks? ● how can you organise the access management when working with international partners? ● what tools and platforms are available? preparation reflection data cleaning? (version 1) description: social media data is often of low quality or ‘noisy’ (e.g. colloquial language, typos, emoticons) and requires data cleaning prior to analysis. replacing, correcting, or removing inaccurate or incomplete records reflects the researcher’s expectations and value judgments of the data and are likely to influence subsequent analysis and results. thus, sma researchers should reflect on their motivation and expectations when cleaning social media data. stieglitz et al. (2018) ire 3.0 (2020), p. 41 ● how do you clean your data and what is your motivation behind it? ● what other research outcomes would be possible if the dataset was not cleaned? ● how do you identify outliers and what are the consequences of removing them? missing data? (version 1) description: social media data is rarely representative for the population or phenomenon under investigation. thus, sma researchers should try to check their data for representativity of (minority) groups and diversity of perspectives and opinions. missing data might influence the results and could cause harm. ire 3.0 (2020), p. 41 boyd & crawford (2012), p.669 ● did you check who is represented in your data set? ● who or what might be missing? ● how can you react to missing data? preparation biases? (version 1) description: procedures and methods used by (commercial) data providers or social media platforms for data collection might have built-in biases. sma researchers should aim to understand and reflect on the process which resulted in the obtained data set. bruns (2019) møller & bechmann (2019) ire 3.0 (2020), p. 20 ● which procedures and methods does the platform or data provider use to supply the data? could these have built-in biases? unintended insights? (version 1) description: sma research can lead to insights about groups, users or topics which were not of primary interest for the research. if these insights touch on sensitive areas or could have substantial harmful impacts, they might european commission (2018), p. 14 ● how would you react to unintended insights (e.g., hate speech, cyberbullying, sedition, harmful fake news, terrorist groups or acts, manipulation of users australasian journal of information systems jung et al. 2022, vol 26, research on applied ethics ‘cambridge moralica’ 31 require sma researchers to take some form of action, e.g. to report to authorities via bot networks)? merging data? (version 2) description: supposedly irrelevant information in a data set might lead to identification of research subjects if it is combined with information from other data sets. sma researchers should thus take special care when merging datasets for analysis. interviews zook et al. (2017) ● does the possibility of re-identification increase because you are merging different datasets? limitations? (version 2) description: the selection of analysis methods (e.g., type of algorithm) and statistical criteria (e.g., significance levels) can result in limitations. the value judgments and expectations of researchers influence which analysis is chosen and what results will be considered relevant. interviews ● how is your data set, data preparation, and choice of analysis limited? ● how do you declare or account for those limitations? protection of the researcher? (version 2) description: researchers can become a target (e.g., subject to hate speech) if they conduct sma research on certain topics or within certain communities. further, being confronted with disturbing social media content might cause psychological harm. thus, protecting the researcher for example by carefully concealing their online identity is important for some areas of sma. interviews ● does the researcher need to be protected? is the researcher confronted with disturbing material? ● does the researcher conduct research in a conflictual area which could put the researcher at risk due to the research results? methodological reflection validity? (version 1) description: this principle refers to the suitability of an analytical approach to answer a particular research question. sma methods should be selected based on fit rather than habit. for example, social network analysis can be performed on almost all major sma datasets, but results might not always be meaningful. when using machine learning, sma researchers often get a result that looks valid, but do not have enough knowledge how the result was achieved. burton-jones, 2009 ● do you select your method because it is a habit and existing competence or because it is the best fit for your research design? algorithms? (version 1) description: analysing social media data with algorithms can bear ethical hurdles. for example, eke, norman, liyana shuib, & nweke (2020) ● if machine learning is applied, how do you separate the dataset australasian journal of information systems jung et al. 2022, vol 26, research on applied ethics ‘cambridge moralica’ 32 if social media data serve as training data for machine learning, inherent biases in the data will be reproduced by the model. further, if algorithms are used in sma to make or inform decisions, researchers should be aware of the black box problem and explore options to increase transparency and explainability of the employed models and discuss who is accountable for the algorithmic decisions. when using pre-trained models or analysis services, researchers should consider the training data used for pre-training (e.g., cultural context, demographics). ieee (2020) ire 3.0 (2020) martin (2019) mittelstadt, allo, taddeo, wachter, & floridi (2016) zook et al. (2017) into training, validation, and test data? do you test on previously unseen data? ● do you use a pretrained model? if so, what do you know about the training data and the performance of the model (e.g. embedded cultural biases)? ● do you understand the functionality of the algorithm or is it a black box for you? ● is the algorithm used to make decisions? who is accountable for those? ● what implications has the “imperfection” of automated analysis of for example unstructured data such as tweets (spelling mistakes, colloquial language etc.)? ● which additional materials are you using (e.g., word dictionaries), how were these developed, and how might they influence the results? method biases? (version 2) description: sma researchers should check for and mitigate potential biases in their data analysis and in predictions resulting from the analysis (e.g. in machine learning models). these also serve as a basis for estimating the generalisability of the findings. interviews ● does the dataset allow for a generalisation of results? ● is it possible that the data could have multiple meanings? ● is there any danger of misclassifying people or groups for example based on their language and / or culture? exposure of sensitive attributes or relationships? (version 2) description: during the process of sma, sensitive attributes or relationships can be discovered and subsequently exposed (e.g. relationships between users based on social network analysis). interviews zook et al. (2017) ● in which sense does your research allow insights to sensitive attributes of people / groups? implications for description: insights on a groupinterviews ● is there a risk that your australasian journal of information systems jung et al. 2022, vol 26, research on applied ethics ‘cambridge moralica’ 33 groups? (version 2) level are often perceived as less or not problematic. however, they can have harmful consequences such as creating or manifesting certain stereotypes. research insights harm a specific group? supported by other data? (version 2) description: sma researchers should explore to what extent their findings align with existing research and whether the data or analysis could be triangulated with insights from other sources. interviews ● how do my results connect to other / existing research? can you support or complement your analysis by other data (non-social media data)? publication reflection informed consent (version 1) description: obtaining informed consent of social media users included in the study in the pretracking reflection is often not feasible. an alternative approach to address this important ethical consideration is to obtain informed consent of those users whose social media content will be presented in the study (e.g., in social network graphs or content analysis results in the publication). ire 3.0 (2020) acm (2018) williams, burnap, & sloan (2017) ● would it be an option to ask those users for informed consent, whom you want to quote directly in the study and thus overcome the problem of prior informed consent in big data analysis? sharing & storing (post publication)? (version 1) description: sma researchers often face the challenge of finding a balance between protecting individual privacy, complying with terms and conditions of social media platforms for sharing or redistributing content, and complying with open access requirements of a publisher when finalising an sma project. stieglitz et al., 2020 ● how can you react to the dilemma of required open access publishing (open data) by research grant providers and restrictions through legal frameworks and terms of conditions of social media platforms? future implications? (version 1) description: according to the idea of responsible research and innovation, sma researchers should always reflect if the outcomes of their work are desirable. this includes participating what implications their findings (e.g., the development of a new sma method) might have on society. potential harm which might result from it should be anticipated and prevented. stahl et al., 2013 ● are there any measures you can take today to avoid future harm that can result from increased technological possibilities, or a combination with other data sets? promoting diversity? description: when assessing or publishing sma results, interviews ● does your research publication account for australasian journal of information systems jung et al. 2022, vol 26, research on applied ethics ‘cambridge moralica’ 34 (version 2) researchers should avoid generalising findings to populations which were not represented in the study. instead, researchers are advised to discuss the diversity (e.g. culture, gender) of their research, transparently report limitations, and reflect on the impact of their own socioeconomic, cultural, and personal background. zook et al. (2017) boyd & crawford 2012 and promote diversity? ● are you aware of your own ‘cultural lenses’ and do you make them explicit in te publication? re identification? (version 2) description: in sma, the risk of reidentification is particularly high as for example a tweet or author name can be easily found online. avoiding re-identification is crucial to protect users’ privacy, especially if users are not aware that their data was used in research, and they did not provide explicit informed consent. thus, researchers should carefully check the risks for re-identification and explore options to anonymise their findings as much as possible. interviews williams, burnap, & sloan (2017) zimmer (2010) ● is there a possibility to re-identify individuals and groups when combined with other data? ● are you encrypting, anonymising or pseudo-anonymising your data and / or your results? ● are you aware of the possibility of fabricating tweets? giving back? (version 2) description: unlike research participants in other studies, sma subjects are usually not compensated for their contribution to research. by adding this aspect, we would like to encourage researchers to think about ways of giving back to the researched community, for example by sharing benefits or insights in an accessible way. interviews ● in which sense will your research publication give something back to the community you have researched? ● are there other ways of giving back? table 6. final framework with description, sources, and initial reflection questions. appendix b additional material on the focus groups ethical considerations round 1 ethical considerations round 2 fg 1 relevant data sources differ depending on the country secure data storage (concerns about commercial platforms such as google drive) avoiding re-identification of twitter users potential harm through clustering groups based on sensitive informed consent not possible on twitter potentially revising keywords for tracking to represent minorities better country / cultural bias when analysing tweets only in a certain language data cleaning, identifying misinformation re-identification when publishing tweets reporting identified bot accounts to twitter (unintended insights) australasian journal of information systems jung et al. 2022, vol 26, research on applied ethics ‘cambridge moralica’ 35 information (e.g., extreme political orientation) important role: m = 3.34 completeness: m = 1.34 (n = 3) sharing data and code with publication fg 2 popular data sources (userbase) depend on the country public vs private data on twitter informed consent (contacting individuals, terms of use) secure storing and sharing of data (gdpr compliant) anonymizing data being transparent about own biases & background important role: m = 4.0 completeness: m = 3.25 (n = 4) cultural dimensions (different policies in different countries / cultures) generalisability of findings being transparent about own biases & background data minimisation (sma tendency to track everything) power issues (restrictions of platform api) transparent data cleaning description unintended insights (obligation to report fake news, hate speech?) giving back (non-academic publications, contacting platform provides) researcher’s biases throughout all phases table 7. overview ethical consideration in the discussion of the research design supported by the original sma framework (round 1) compared to the discussion of the research design supported by the extended sma framework (round 2). statement mean (range) the extended framework would support me in planning and conducting more ethical sma research. 4 (3-5) the extended framework is easy for me to understand. 3.5 (2-5) the extended framework is an addition / provides added value to existing ethical guidelines. 3.75 (3-4) i would apply this framework in my research. 3.88 (2-5) applying the ethical reflections reduced the methodical quality of my research design. 1.38 (1-2) table 8. assessment of the perceived usability and applicability of the framework after discussing the research design supported by the extended framework (n = 8). copyright: © 2022 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v26i0.3121 http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 methodology 3 rigor cycle – literature review 3.1 social media analytics in information systems research 3.2 ethical issues of social media analytics 3.3 ethical guidelines and codes of conduct applicable to sma 4 relevance cycle – institutional challenges of ethical sma research 5 design cycle artefact description 6 design cycle evaluation methods 6.1 round 1 expert interviews 6.2 round 2 focus groups 6.3 round 3 survey of initial experts, senior sma researchers, and ethics boards 7 design cycle evaluation results 7.1 interview results 7.2 focus group results 7.3 results of the final evaluation 8 discussion 9 limitations and future research 10 conclusion references appendix a paper title (paper title style) australasian journal of information systems freier & schumann 2021, vol 25, selected papers from acis 2019 dss in the context of cyber-physical systems 1 decision support systems in the context of cyberphysical systems: influencing factors and challenges for the adoption in production scheduling pascal freier university of goettingen germany pfreier@uni-goettingen.de matthias schumann university of goettingen germany abstract cyber-physical systems promise a complete networking of all actors and resources involved in production and thus an improved availability of information. in this context decision support systems enable appropriate processing and presentation of the captured data. in particular, production scheduling could benefit from this, since it is responsible for the shortterm planning and control of released orders. since decision support systems and cyberphysical systems together are not yet widely used in production scheduling, the aim of this research study is to analyse the adoption of these technologies. in order to do so, we conducted a qualitative interview study with experts on production scheduling. thereby, we identified eleven influencing factors and 22 related challenges, which affect the adoption of decision support systems in production scheduling in the context of cyber-physical systems. we further discuss and assess the identified influencing factors based on the interview study. the results help to explain and improve the adoption of those systems and can serve as a starting point for their development. keywords decision support system, production scheduling, cyber-physical systems, industry 4.0, challenges. 1 introduction "if you have to reschedule often, such a [decision support] system is worth its weight in gold!" (expg) with this quotation, an expert, whom we talked to during our interview study, underlines the importance of decision support systems in production scheduling. especially production scheduling, which is responsible for scheduling, executing and monitoring released production jobs, is facing high demands on flexibility due to ever increasing customer demands, for example with regard to customization and delivery times (schuh, potente, thomas, & hauptvogel, 2014). if furthermore deviations from the initially generated production schedule or disruptions occur (e.g., due to unplanned events like a machine break down), the responsible production management needs to identify and take care of them promptly (schuh, potente, thomas, & hempel, 2014). the increasing diffusion of cyberphysical systems in companies, which, for example, record real-time data from the shop floor, simplifies monitoring the shop floor as well as the identification of those deviations on the one hand. on the other hand, as a result of a better information basis decisions on possible australasian journal of information systems freier & schumann 2021, vol 25, selected papers from acis 2019 dss in the context of cyber-physical systems 2 reactions can improve (schuh, potente, thomas, & hempel, 2014). nevertheless, the use of cyber-physical systems itself does not inherently lead to an improvement of the initial situation. on the contrary, the mass of sensor data may lead to an information overload, so that it is hard to identify the concrete problems or to make associated decisions. therefore, there is a need for decision support systems that provide the decision maker with an overview of the current situation and its problem areas as well as the effects of all possible alternative actions and reactions (cupek, ziebinski, huczala, & erdogan, 2016). however, as cyberphysical systems and decision support systems that work with real-time data are not yet in use together in most industrial companies, research analysing how the introduction of those systems in production scheduling can be promoted is necessary. especially since cyberphysical system-based decision support systems in production scheduling are not sufficiently considered in the current state of research yet, there is also a lack of theoretical knowledge in this regard (see section 2.1). therefore, the aim of our study presented in this paper is to identify factors, which influence the adoption of decision support systems in production scheduling in the context of cyber-physical systems and analyse corresponding challenges. this leads to the following research questions that we intend to answer by presenting results from a qualitative interview study among experts on production scheduling in the industrial sector: rq1: which factors influence the adoption of decision support systems in production scheduling in the context of cyber-physical systems? rq2: what challenges impede the successful adoption of decision support systems in production scheduling in the context of cyber-physical systems? in order to answer these questions, the remainder of this research paper proceeds as follows: in the next sections, we describe the basics and outline related research regarding production scheduling, decision support systems, and cyber-physical systems. in addition, the second section explains the theoretical background. thereafter, we explain the applied research design. afterwards, we present the findings of our study by describing the identified influencing factors and the corresponding challenges. following this, we first discuss and access our findings in the discussion section and then state limitations and future research directions. finally, we briefly summarize our findings in the conclusion. 2 background 2.1 basics and related research since both production scheduling and decision support systems (dss) are well-known and well researched concepts, widely accepted definitions and explanations already exist. therefore, production scheduling is part of the production planning and control (ppc) and describes the creation of a processing sequence for released orders, taking into account the underlying objectives (e.g., adherence to delivery dates or minimization of lead times; pinedo, 2009; schneeweiß, 1999). in this respect, a distinction can be made between flow shop problems and job shop problems. in the case of flow shop problems, all orders are processed in an identical machine sequence, whereby it is possible that orders may overtake each other so that the sequence of orders can be changed. with job shop problems, the jobs can pass through the processing stations in different sequences. the processing sequence for job shop problems is either fixed for each individual order, but varies between orders, or there is no predefined processing sequence for individual orders (pinedo, 2009; schneeweiß, 1999). since sequencing australasian journal of information systems freier & schumann 2021, vol 25, selected papers from acis 2019 dss in the context of cyber-physical systems 3 therefore is a mathematically complex optimization problem, heuristics based on priority rules (e.g., due date rule, shortest processing time rule, first-in-first-out) are often used to achieve the desired goals (pinedo, 2009; schneeweiß, 1999). furthermore, the fulfillment of the plan is monitored within the production scheduling and, in the event of disruptions or deviations, countermeasures are taken (schneeweiß, 1999). the countermeasures range from waiting until the cause of the problem is eliminated over repairing the plan (e.g. left or right shift) to a partial or even complete rescheduling (schneeweiß, 1999; sabuncuoglu & goren, 2009; vieira, herrmann, & lin, 2003). dss in general can be defined as “computer technology solutions that can be used to support complex decision making and problem solving” (shim et al., 2002). they focus “on supporting and improving managerial decision making” (arnott & pervan, 2005). therefore, they are primarily used to solve semi-structured or unstructured problems (e.g., decisions between alternative schedules; gorry & scott morton, 1971; sprague & carlson, 1982). to support decision makers dss are interactive systems that utilize models, methods and problemoriented data to provide and edit required information (sprague & carlson, 1982). dss help to monitor business activities and processes (e.g., by using alerts when metrics fall below predefined thresholds), analyse root causes of problems (e.g., by exploring timely and relevant information), and manage processes as well as people in order to improve and optimize decisions and the performance (eckerson, 2010). in the context of production scheduling, dss shall provide the decision makers with an overview of the current situation, alerts and information on deviations, disruptions and their effects, as well as on (re-)planning options and support them in associated decisions. cyber-physical systems (cps) offer possibilities of collecting the real-time data from production required for this application of dss. cps emerged within the concept of industry 4.0 in the last few years and consequently are not well researched yet. they form the technological basis for industry 4.0 and related concepts such as smart factories and the internet of things (iot; frontoni, loncarski, pierdicca, bernardini, & sasso, 2018). cps are embedded systems that integrate physical objects, computation, communication, and networking processes (j. lee, 2015). “cps can be illustrated as a physical device, object, equipment that is translated into cyberspace as a virtual model” (j. lee, 2015). they feature physical components like sensors and actuators to interact with their surroundings as well as networking and processing capabilities to process and communicate information. therefore, they are able to monitor and control physical processes, usually with feedback loops where physical processes affect computations and vice versa (e. a. lee, 2008). figure 1 schematically illustrates the structure of a cps. with regard to decision support in production scheduling, cps can, for example, provide a comprehensive real-time data basis on which dss can present decision makers with an overview of the current situation in production in real-time. this also enables to identify deviations from the initial production schedule promptly as well as to simulate and compare possible reactions to the current situation. australasian journal of information systems freier & schumann 2021, vol 25, selected papers from acis 2019 dss in the context of cyber-physical systems 4 figure 1: schematic structure of a cps while the topics of production scheduling and dss themselves are subjects of research for several decades, cps is a young field of research. in the area of production scheduling, research exists, which considers different disruptions as well as corresponding (re)scheduling strategies and reaction possibilities (sabuncuoglu & goren, 2009; vieira et al., 2003). however, since most of this research originated before the emergence of cps, it consequently does not take them into account. although some prior research in the area of production scheduling in the context of cps exists and already names or addresses some influencing factors and challenges of the use of dss to support production scheduling, a holistic overview is missing. in most cases, the existing contributions neglect dss and focus on the changes of production scheduling caused by cps. the authors primarily deal with the effects for the ppc systems and describe theoretical potentials as well as the technical hurdles that have to be overcome (krumeich, jacobi, werth, & loos, 2014). karner, glawar, sihn, and matyas (2019) for example use real-time data to improve production planning based on machine conditions. jiang, jin, e, and li (2018) and dafflon, moalla, and ouzrout (2018) include decision-making of systems in their publications. the former develop a decision model on the basis of which a multi-agent system can dynamically adapt the planning. dafflon et al. (2018), on the other hand, describe a comparable system that provides the decision-making basis for a self-adaptive production system. only a few authors include dss in their consideration. those who do so, primarily deal with general challenges for decision support in production planning (schuh et al., 2013), the necessity of dss in production planning (schuh, potente, thomas, & hempel, 2014) or requirements for a prototypical implementation of a dss (schuh, potente, thomas, & hauptvogel, 2014). schuh and fuß (2015) and schreiber, vernickel, richter, and reinhart (2019), who each present a dss in the context of cps focus on its implementation and neglect the influencing factors for the adoption of the developed systems. schreiber et al. (2019) further limit their work mainly to maintenance planning. although some of these studies contain cyber-physical system processor actuator(s) communicator sensor(s) environment (production) interactionperception australasian journal of information systems freier & schumann 2021, vol 25, selected papers from acis 2019 dss in the context of cyber-physical systems 5 influencing factors or challenges, they do not develop or present them in a structured manner. moreover, many of the prior contributions only refer to specific industry sectors (krumeich, werth, loos, schimmelpfennig, & jacobi, 2014) or cases (cupek et al., 2016). thus, it is not possible to make general statements. therefore, rigorous research that identifies influencing factors and corresponding challenges of the adoption of dss in production scheduling in the context of cps is missing. 2.2 theoretical background the technology-organization-environment framework (toe framework; tornatzky & fleischer, 1990) is a widely used framework that aids to explain which influencing factors affect the adoption of new technologies, for instance, by classifying them in environmental conditions, organizational characteristics and technological attributes (see figure 2; baker, 2012; doolin & al haj ali, 2008). we use the toe framework because prior studies like, for example, angeles (2013) and doolin and al haj ali (2008) show that this framework is suitable to explain the implementation and adoption of innovations in enterprises in is research. while the former analyses the use of radio-frequency identification (rfid)-based systems in the industrial sector, doolin and al haj ali (2008) deal with the adoption of mobile commerce technologies for supply chain activities. figure 2: technology-organization-environment framework (tornatzky & fleischer, 1990) 3 research design in order to identify influencing factors (rq1) and corresponding challenges (rq2) of the adoption of dss in production scheduling in the context of cps, we conducted a qualitative and exploratory interview study among experts on production scheduling in the industrial sector. we conducted an exploratory interview study design because it serves particularly to collect new findings and insights. for this purpose, we followed a three-stepped methodological approach. first, we selected potential experts from industrial enterprises based on their work experience. we selected experts, which either have experience with the practical application of production scheduling or provide corresponding software solutions. furthermore, all experts had to have some experience with the concept of cps. based on this, we contacted 60 experts in total. nine of those accepted our interview invitation, which led to an acceptance rate of 15.00 %. table 1 displays the summarized characteristics of our sample. although nine interviews represent a small sample size, the participants were specialized in our field of investigation by meeting the above-mentioned requirements. furthermore, we decided not to conduct further interviews, since the latter interviews did not reveal many new insights and thus showed signs of theoretical saturation (glaser, 1978). technology organization environment innovation australasian journal of information systems freier & schumann 2021, vol 25, selected papers from acis 2019 dss in the context of cyber-physical systems 6 expert industry position duration a automotive it planer logistic systems ~ 37 min b large electrical appliance head of production technology ~ 36 min c large electrical appliance head of process management ~ 36 min d laboratory technology production engineer ~ 37 min e laboratory technology production engineer ~ 37 min f energy and automation technology production engineer ~ 46 min g printing head of it ~ 35 min h automotive plant manager ~ 45 min i software supplier technical distribution ~ 41 min mean value ~ 39 min table 1. sample characteristics in the second step, we conducted eight of the interviews via phone and the interview with expert g face-to-face from november 2017 to april 2018. the interviews lasted between 36 and 46 minutes. in order to identify sufficient results regarding our research questions as well as to leave the interviewees enough room for own ideas, we prepared a semi-structured interview guideline that contains open questions about the use of cps and dss in production scheduling (see the full interview guideline in appendix a; myers, 2013). to allow in-depth analysis of our interviews, we recorded and transcribed all interviews. as the interviews were conducted in german, we translated relevant quotations from german into english by using constant contextual comparison (suh, kagan, & strumpf, 2009). third, we coded and analysed the transcripts by using open and selective coding as well as the structured content analysis approach (mayring, 2014). therefore, we constantly doublechecked and discussed the coding during the analysis to minimize subjective influences, assigned the codes to the core topics of our study (influencing factors and challenges) and classified them according to the dimensions of the toe framework (see section 2.2). 4 findings in this section, we present our findings regarding the influencing factors and the resulting challenges of the adoption of dss in production scheduling in the context of cps. we subdivide our results according to the three categories of the toe framework. figure 3 shows an overview of our results including the relative number of interviewees naming the respective factor. figure 3: influencing factors of the adoption of dss in production scheduling in the context of cps environmental factors organizational factors technological factors adoption of dss in production scheduling in the context of cps e1: it-security (44 %) e2: social compatibility (11 %) o1: expert knowledge (44 %) o2: employee concerns (67 %) o3: corporate culture (67 %) t1: network infrastructure (56 %) t2: system landscape (67 %) t3: data availability (67 %) o4: regional culture (22 %) o5: age structure (33 %) t4: sensor equipment (56 %) australasian journal of information systems freier & schumann 2021, vol 25, selected papers from acis 2019 dss in the context of cyber-physical systems 7 4.1 technological factors based on our study, we identified four technological factors. technological factors encompass characteristics of technologies, which are already in use as well as those, which are not yet present in enterprises (baker, 2012). the first of which is the network infrastructure (t1), which was mentioned by 56 % of the surveyed experts. in this respect, the experts first cited the need for the existence of wi-fi coverage (tc1.1). a sufficient network connection is a prerequisite for connecting sensors. although the experts described that the implementation of factory-wide network coverage is technically no longer a limit today, it does pose further challenges (see e.g., e1: it-security). furthermore, the experts describe that it is currently not yet possible to simply integrate machines or sensors into the network infrastructure according to the plug and produce principle (tc1.2). table 2 shows exemplary statements on this factor and the associated challenges. t1: network infrastructure “as long as we have a network connection […], we can do that." (expa) “in the long run interfaces need to be created, which simply minimize the effort, so that in the best case plug and play is available for the most different solutions." (expi) challenges: tc1.1: existence of area-wide wi-fi coverage tc1.2: integration of new nodes in the infrastructure table 2. technological influencing factor (1/4) in addition to the network infrastructure, 67 % of the experts also mentioned the system landscape (t2) as a critical factor. they describe the current situation in companies as a historically grown collection of information systems that communicate with each other via proprietary interfaces. according to the interviewees, joint databases or integrated data processing are rarely to be found. consequently, the provision of interfaces to the systems (tc2.1) as well as a consistent database (tc2.2) are central challenges for the adoption of a dss in production scheduling. furthermore, the experts described that the connection of further systems is associated with considerable effort, which is why the connection of the dss to the existing system landscape (tc2.3) is also a challenge. table 3 displays the challenges of the influencing factor system landscape with exemplary quotations. t2: system landscape "because i always experience in this whole discussion that a world is described, which we do not see from the practical point of view at all yet. we do still have no interfaces, still have no databases, which can do the whole evaluation. in principle, this is not yet available. and that's what makes it so difficult for us." (expc) “the interplay of certain databases or systems […] is a big challenge in industry 4.0, to let such systems talk to each other or to exchange the right data.” (expe) "second, how is the signal processed? the signal must again intervene somewhere in the next system. every company not only has an erp software package, but also a few others that are connected via interfaces. and that, of course, results in a huge effort for the installation." (exph) challenges: tc2.1: providing interfaces tc2.2: providing a consistent database tc2.3: connecting the dss to the existing system landscape table 3. technological influencing factor (2/4) furthermore, 67 % of the interviewees indicated that data availability is another technological factor, which needs to be considered. the experts referred to master data (e.g., machine or australasian journal of information systems freier & schumann 2021, vol 25, selected papers from acis 2019 dss in the context of cyber-physical systems 8 process master data) that is currently not managed, known or even available (tc3.1). this challenge poses a major problem for the development and introduction of a dss, since accurate mapping and simulation of production is not possible without the corresponding master data. another challenge is the availability of machine or sensor data and measured values (tc3.2). although the experts stated that standard industrial machines nowadays have sensors, these are either uninterpretable on their own or can only be accessed to a limited extent by the company, as the manufacturer primarily uses the data for its own evaluations. table 4 depicts exemplary quotations for data availability and the resulting challenges. t3: data availability "if i want to point out such possibilities, then i must know the complete basics, the complete master data first. i honestly couldn't say whether we know this master data at all at the moment." (expa) “if the people do not know where the data is or what data is available at all, which you would find out in most cases, i think, then the project is doomed to failure. and that is […] a big obstacle to spreading it to the masses.” (expg) "of course, every reasonable machine you buy today, no matter for what, i.e. for the industrial sector, for industrial production, will be equipped with sensors, will be equipped with control computers […]. the manufacturers of such machines are more and more interested in it because they have recognized the potential to access the data of their own machines, even if they are at the customer's site. […] but that does not mean that the data is in the company, and that's where i see the big problem.” (expg) challenges: tc3.1: availability of master data tc3.2: availability of machine/sensor data table 4. technological influencing factor (3/4) as mentioned above, the possibility of collecting real-time data depends on the sensor equipment of the machines. since the service life of machines in industrial practice can be several decades, sensor equipment is not inevitably available in industrial enterprises. this leads to the influencing factor of existing sensor equipment (t4), which 56 % of the experts mentioned. the experts addressed both technical and financial hurdles when retrofitting old machines (tc4.1). they furthermore addressed the problem that the sensors and their measured values are by no means standardized (tc4.2). this can lead to incompatibilities between new machines equipped with sensors and retrofitted machines as well as between different new machines. table 5 shows these challenges together with exemplary quotations on the factor of sensor equipment. t4: sensor equipment "in our investment strategy, we […] have old machines and new machines in combination. the fact that you can buy new machines means you have to retrofit old machines. then there is the question, how do the sensors interact? how do you get the transfer to the platform?” (expc) "especially with older machines, the retrofitting, i imagine difficult." (expf) challenges: tc4.1: ensuring that old machines can be retrofitted tc4.2: ensuring the compatibility of different sensor data table 5. technological influencing factor (4/4) 4.2 organizational factors with regard to the organizational aspects, we identified five factors relating to the structure of the respective companies and the associated organizational aspects (baker, 2012). 44 % of the interviewees named expert knowledge (o1) as the fundamental prerequisite for adopting cps australasian journal of information systems freier & schumann 2021, vol 25, selected papers from acis 2019 dss in the context of cyber-physical systems 9 in production scheduling. accordingly, the employees must have the competences and abilities to retrofit the equipment with production resources (e.g., equipping with sensors and actuators; oc1.1) and to connect the resulting cps to the it infrastructure (oc1.2). the employees of the companies themselves do not have sufficient expertise, but the external acquisition of knowledge is also problematic, since on the one hand, the providers of the corresponding resources tend to strive for the sale of new devices and on the other hand, those devices often only offer proprietary interfaces. table 6 provides an exemplary quotation for this influencing factor and its corresponding challenges. o1: expert knowledge "everyone talks about it and thinks it is all great, but by saying, “i'd like to have such a system”, all these topics start. you have to retrofit the available systems, where you stick sensors on engines for example, which then report themselves. we actually have to say, we are not set up in respect of qualification of our employees and to create the framework conditions in the it.” (expc) challenges: oc1.1: low expertise in technical upgrading and retrofitting of machines oc1.2: low expertise about the connection to the system landscape table 6. organizational influencing factor (1/5) furthermore, 67 % of the respondents mentioned employee concerns (o2) as an additional influencing factor that might limit the acceptance of the introduced system and thus also its use. on the one hand, they stated that employees could not use the dss because they are afraid not to understand it, do not want to learn it or do not trust it (oc2.1). the latter is also a possible reason for employees doing double work (oc2.2), as they still use the original problem-solving paths in addition to the new system. furthermore, employees fear that the progressing automation will make them increasingly less important for the company and that they will lose their jobs as a result of rationalization measures (oc2.3). this could, for example, influence the employees in their work or also result in rejecting the system. table 7 presents quotations on this influencing factor with the resulting challenges. o2: employee concerns "you have to know that it's always a critical point. that is very individual, very employee-related. [...] one is afraid of losing his job, the next one is afraid, he could not understand it, the next one simply does not want to learn something new again, because he retires in three years and the other one does not trust in the system and then perhaps makes duplicate work because of the new system, which he is supposed to use, instructed by the management, but he also does everything on paper, because he has no trust in the it, so there are manifold reasons." (expg) “especially when it comes to decision support, it is slowly changing into the system takes the decision away from you and at some point the worker may feel superfluous and the fear of losing the job is always present.” (expe) challenges: oc2.1: non-use of the solution oc2.2: double work for employees who do not trust the dss oc2.3: fear of losing their job table 7. organizational influencing factor (2/5) a further organizational factor we identified based on the study is the existing corporate culture (o3). in this respect, both the support of the management (oc3.1) and the departments themselves (oc3.2) are important factors for the realization of a dss in the production scheduling. with regard to the management support, it is necessary, on the one hand, that the management is open to innovative technologies and, on the other hand, that it can be australasian journal of information systems freier & schumann 2021, vol 25, selected papers from acis 2019 dss in the context of cyber-physical systems 10 convinced of the potential benefits of the technology despite any high investment costs. with regard to the department support, it is important to convince the employees of the active use of the solution and minimize their concerns (see o2). in addition, some of the experts stated that enterprises introduce new technologies in the production environment quite conservatively (oc3.3), since they only expect small benefits of the technologies, which do not outweigh the investment costs. table 8 shows exemplary quotes for the influencing factor corporate culture and the corresponding challenges. o3: corporate culture "if i can get a production control system for 8 million euro, which shows me whether my punch at the end of my line is full or i just go there and see that there are three boxes on top of it. we keep things a bit simpler. we are not that it crazy. […] that is just how we work. that's why it's hard for us to imagine things like that to be real beneficial." (expb) “the best system will not work if people do not cooperate." (expd) challenges: oc3.1: lack of management support oc3.2: lack of support from departments oc3.3: conservative attitude in industrial enterprises table 8. organizational influencing factor (3/5) two of the experts mentioned regional culture (o4) as another influencing factor. they indicated that there are factors, both national and international that can inhibit the acceptance of dss (oc4.1). on the one hand, there is a different understanding of responsibility and selfinitiative in different countries. thus, in some cases employees only follow direct instructions from the superior. this could also lead to rejecting the use of a dss, as it merely provides the basis for making a decision. furthermore, especially employees in economically underdeveloped regions seem to reject innovative solutions like dss, as they are afraid of rationalization measures (see o2 employee concerns). table 9 depicts exemplary quotations for this challenge. o4: regional culture "i notice, now that i am here in mexico, the way of working, the understanding of how one works, is quite different from what it is in germany. we have a lot of personal responsibility and think, a skilled worker has the courage to make his own decisions. in mexico, someone decides at the top and passes on the order downwards. e.g. the executive organ, the worker on the shop floor, or even a skilled worker, simply does not make decisions, either because he does not dare, or because he simply is not used to it. [...] it's a different way of working from ours and you have to take that into account when you talk about whether or not there can be such a thing.” (expa) “especially here in the region, we are relatively weak in terms of structure and of course i always have to ask the question when creating something new […] what is the goal? is the goal rationalizing in order to be even more effective […] in terms of costs?” (expg) challenges: oc4.1: regional differences in acceptance table 9. organizational influencing factor (4/5) the last organizational factor we identified in the study is the age structure (o5). 33 % of the experts cited this factor and described younger employees as more open to new technologies that would accept and use a dss faster, while older employees are less willing to accept new technologies (oc5.1). accordingly, the effort required to convince older employees to use a new system is higher than for younger employees (oc5.2). the experts stated, however, that it is of australasian journal of information systems freier & schumann 2021, vol 25, selected papers from acis 2019 dss in the context of cyber-physical systems 11 central relevance to also convince older employees of the benefits of the solutions. table 10 shows the described challenges and exemplary quotations for them. o5: age structure "i believe that in the future, younger generations will be more likely to accept such an approach, so that a system will evaluate something like this, perhaps also show connections that people do not even notice in the first place. our management is, on average, 40 years old, or even older. they made decisions for themselves all their lives, from the gut, according to their feelings and experience, and i find it difficult to say to those people that a system will do that now.” (expa) “you often start with a team of young, dynamic people. you should better not start off with those who are already at war with it.” (expg) challenges: oc5.1: lower interest in new technologies on the part of older employees oc5.2: more convincing effort required for older employees to use the system table 10. organizational influencing factor (5/5) 4.3 environmental factors the category of environmental factors includes conditions that originate outside the company like guidelines and laws or attacks initiated by externals (tornatzky & fleischer, 1990). in this category, we identified two factors and, based on these, we derived two challenges. 44 % of the interviewees regard it security (e1) as the central factor in this category. by integrating various nodes (e.g., machines, work pieces or products) into the corporate network, the number of potentially vulnerable connections rises sharply. therefore, the security measures in question for securing the network are considered essential (ec1.1). the data and information as well as their transmission need protection as well as the individual nodes in the network (e.g., machines). although the influencing factor of it security could also be categorized as a technological factor, we understand it as an external factor in accordance with the definition given above, since security threats are mostly caused by external attacks. table 11 presents the influencing factor of it security and associated quotations. e1: it security “when machines in production are connected to the internet, the issue of safety inevitably has a very high priority.” (expi) “if we get a lot of signals now and process them, how do we make our system open, but still closed enough to fend off attacks. at the moment, we cannot have machines going into our networks.” (expc) challenges: ec1.1: security measures to protect against it attacks table 11. environmental influencing factor (1/2) one of the experts sees a further environmental factor in the social compatibility (e2) of the new solutions. the increasing automation of production and its planning and control potentially offers the possibility of reducing the number of employees in production (ec2.1). when introducing a dss, enterprises must therefore consider social responsibility and rather use the dss to increase the flexibility and efficiency of the company. table 12 shows an exemplary quotation on this influencing factor. australasian journal of information systems freier & schumann 2021, vol 25, selected papers from acis 2019 dss in the context of cyber-physical systems 12 e2: social responsibility "are we technically capable of building a system that can make all the decisions? i would almost say technically yes. the question is do we want that at all? [...] we could build systems, which then decide for themselves. [...] so you have not an ethically, but a social responsibility." (expg) challenges: ec2.1: automation enables rationalization table 12. environmental influencing factor (2/2) 5 discussion the findings we presented in this paper imply that four technological, five organizational and two environmental factors influence the adoption of dss in production scheduling in the context of cps. altogether, we identified eleven influencing factors and derived 22 corresponding challenges. although in general all these factors are relevant for the adoption of a dss in production scheduling, it emerged from the discussions with the experts that a differentiated consideration of the challenges is necessary. with regard to the technological factors, the challenges are to be assessed as uncritical as considering them at an early stage can help to overcome them. the challenges associated with the network infrastructure (t1), such as the existence of comprehensive wi-fi coverage (tc1.1), are currently still existing problems in companies, but can already be solved technically, for example by setting up additional access points. in this regards, expert f gave the following assessment, “it is not rocket science to get this. of course, it depends on the size of halls, but when i think about universities, there is wi-fi in every room. we also have power and cable lines everywhere in production and adding a few lan cables with routers should not be the bottleneck of the problem.” (expf) the challenges regarding the system landscape (t2) are also described as solvable, although this is not easily possible for a single company. in this regard, progressive standardization is particularly relevant, especially with regard to interfaces, which could also contribute to mastering the tc1.2 challenge. the experts described the solutions in this area as feasible, but machine manufacturers, for example, delay the solution process. expert i stated in this context, “the problem with machine manufacturers is that many have somehow come up with their own protocols. there are often associations for industries that have joined […], but the current market is of course very difficult for a software manufacturer, because there is an incredible variability. every machine manufacturer is doing his own thing.” (expi) the influencing factor of data availability (t3) and the associated challenges are not critical for companies as well. to target the corresponding challenges, it is necessary to maintain the existing master data (tc3.1), which does not encounter any technical obstacles apart from the resulting maintenance effort. ensuring the availability of machine and sensor data (tc3.2) can be considered in combination with the influence factor sensor equipment (t4). although machines nowadays generally have the sensor equipment to record the machine data, and retrofitting existing machines is technically feasible (tc4.1), the collected data differs in terms of both quantity and quality (tc4.2), which leads to additional data maintenance or standardization efforts. expert i summarizes this as follows, “there are approaches for retrofitting old machines. they are designed to make it relatively easy to connect them to the existing system and to install them in order to extract certain data from the machine […]. nevertheless, the variety of data in the old machines is still relatively limited. you can read out the controller, but that is of course not enough to make an overall equipment effectiveness analysis.” (expi) australasian journal of information systems freier & schumann 2021, vol 25, selected papers from acis 2019 dss in the context of cyber-physical systems 13 with respect to the organizational factors, the interviewees regard the challenges derived from the influencing factors expert knowledge (o1) and age structure (o5) as uncritical, since, on the one hand, the increasing research and dissemination of cps and dss is expected to lead to an increasing number of experts. on the other hand, with regard to the age structure, they assume that in future an increasing number of younger employees with a high affinity for technology will lead to a shift in the age structure of the workforce, which will enable enterprises to meet the challenge of the age structure in mid to long term. in contrast, the factors of corporate culture (o3) and regional culture (o4) must be regarded as critical. a conservative attitude on the part of company management towards new technologies (oc3.3), for example, represents a hurdle that is difficult to overcome when introducing cps and dss into production scheduling. to overcome this challenge, possibilities must be created to quantify the benefits of the solutions. however, even then there is no guarantee that the solution will be introduced and accepted in the company. expert g summarizes this challenge as follows, “if it is not understood by the management, who in the end make the decisions, what we are introducing, what we are spending money on, then an it specialist as such will always fail.” (expg) furthermore, the experts stated that employee concerns (o2) are similarly problematic. here, too, protracted measures are necessary to reduce employees' concerns or to convince them of the solution. employees must be involved as early as possible in the planning and implementation processes (e.g., in workshops) in order to create acceptance of the dss. however, it cannot be guaranteed that employees will also use the application (oc2.1) or not do additional double work (oc2.2) when implementing such measures, as the employees differ in their concerns. with regard to the environmental factors, the experts stated that enterprises could overcome corresponding challenges if they consider them at an early stage in the planning process. in order to ensure adequate it security (e1), appropriate security measures (ec1.1) must be included as early as possible. in particular, the integration of cps and the associated networking of production resources leads to further risks for it security. security precautions must therefore be taken when designing and implementing a dss as well as for the individual production resources and their production. the social compatibility (e2) of the solution can also be achieved by making it credibly clear to the users of the dss that the system is not intended to rationalize, but rather to relieve and support the users. expert g stated the following in this regard, “create free capacities in order to support the good people with new tasks, maybe even just create a little space to look a little to the left and right. that is actually the first goal, knowing it is a small degree to a certain point where the system can be used to rationalize.” (expg) in summary, our study shows that the organizational influencing factors represent the greatest challenges for the introduction and deployment of a cps-based dss in production scheduling. the environmental and technological factors, however, are less critical or uncritical if they are taken into account at an early stage. these results imply that for the successful adoption of dss in production scheduling in the context of cps not only technical factors are decisive, but also that the companies as well as all actors and stakeholders have to be considered. figure 4 shows an overview of the results of the discussion and assessment of the influencing factors. australasian journal of information systems freier & schumann 2021, vol 25, selected papers from acis 2019 dss in the context of cyber-physical systems 14 figure 4: assessment of the identified influencing factors of the adoption of dss in production scheduling in the context of cps 6 limitations and future research directions as with any similar qualitative interview studies, we are aware that there exist several potential limitations. first, our interview study is based on a relatively small sample size. even though we were trying to reach theoretical saturation, we cannot assure that our results are complete. second, the results of an interview study are dependent on the selection of interviewees. although we carefully selected a broad variety of experts from industrial enterprises as well as software suppliers, there might be additional experts that would supply further results. related thereto, although some of our experts were working abroad (e.g. in mexico), we only interviewed german experts. thus, our results feature a limited generalizability. hence, further research should investigate whether they can confirm our results in other countries. third, as the analysis of interviews is always subjective, different researchers might come to different results interpreting our data. however, in order to minimize subjective influences, we used, for instance, structured content analyses and doublechecked our codes and results. in order to address these limitations and further investigate our findings, we are currently developing a prototypical dss to support production scheduling in the context of cps. thereby, we want, on the one hand, to verify whether the identified and presented results occur in practice and to generate further insights on, for example, design science research knowledge like functional requirements or generalizable design principles and potential effects on the other hand (freier & schumann, 2020; peffers, tuunanen, rothenberger, & chatterjee, 2007). 7 conclusion the goal of this research paper was to analyse the adoption of dss in production scheduling in the context of cps by identifying influencing factors (rq1). we further investigated which related challenges (rq2) result from those factors. therefore, we conducted an empirical interview study among nine domain experts. based on the results of this study, we identified eleven influencing factors and 22 related challenges and classified them in three context categories according to the toe framework (technological, organizational and environmental). although all factors are relevant, it emerged from discussing the results that especially organizational factors are critical, as short-term measures are not sufficient to meet them. environmental factors technological factors adoption of dss in production scheduling in the context of cps e1: it-security (44 %) e2: social responsibility (11 %) o1: expert knowledge (44 %) o2: employee concerns (67 %) o3: corporate culture (67 %) t1: network infrastructure (56 %) t2: system landscape (67 %) t3: data availability (67 %) o4: regional culture (22 %) o5: age structure (33 %) t4: sensor equipment (56 %) uncritical less critical critical legend: australasian journal of information systems freier & schumann 2021, vol 25, selected papers from acis 2019 dss in the context of cyber-physical systems 15 the results of our research can contribute to both, research and practice: our study expands the existing knowledge base by contributing to the understanding of using dss in production scheduling in the context of cps. thus, the results may help to improve the adoption of cpsbased dss in enterprises as they enable to explain and predict challenges on the one hand. on the other hand, they can serve as a starting point for further studies (e.g., regarding the overcoming of challenges) as well as for the development of dss for practical use in a cpsbased production scheduling. references angeles, r. (2013). using the technology-organization-environment framework and zuboff’s concepts for understanding environmental sustainability and rfid: two case studies. international journal of social, behavioral, educational, economic, business and industrial engineering, 7(11), 2878–2887. https://doi.org/10.5281/zenodo.1088850 arnott, d., & pervan, g. (2005). a critical analysis of decision support systems research. journal of information technology, 20(2), 67–87. https://doi.org/10.1057/palgrave.jit.2000035 freier, p., & schumann, m. (2020). design and implementation of a decision support system for production scheduling in the context of cyber-physical systems. in n. gronau, m. heine, k. poustcchi, & h. krasnova (eds.), wi2020 zentrale tracks (pp. 757–773). gito verlag. https://doi.org/10.30844/wi_2020_g5-freier baker, j. (2012). the technology–organization–environment framework. in integrated series in information systems. information systems theory (vol. 28, pp. 231–245). new york, ny. https://doi.org/10.1007/978-1-4419-6108-2_12 cupek, r., ziebinski, a., huczala, l., & erdogan, h. (2016). agent-based manufacturing execution systems for short-series production scheduling. computers in industry, 82, 245– 258. https://doi.org/10.1016/j.compind.2016.07.009 dafflon, b., moalla, n., & ouzrout, y. (2018). cyber-physical systems network to support decision making for self-adaptive production system. in 2018 12th international conference on software, knowledge, information management & applications (skima), phnom penh, cambodia. doolin, b., & al haj ali, e. (2008). adoption of mobile technology in the supply chain: an exploratory cross-case analysis. international journal of e-business research, 4(4), 1–15. https://doi.org/10.4018/jebr.2008100101 eckerson, w. w. (2010). performance dashboards: measuring, monitoring, and managing your business (second edition). finance professional collection. hoboken, nj: wiley. frontoni, e., loncarski, j., pierdicca, r., bernardini, m., & sasso, m. (2018). cyber physical systems for industry 4.0: towards real time virtual reality in smart manufacturing. in l. t. de paolis & p. bourdot (eds.), lecture notes in computer science. augmented reality, virtual reality, and computer graphics (vol. 10851, pp. 422–434). cham: springer. https://doi.org/10.1007/978-3-319-95282-6_31 glaser, b. g. (1978). theoretical sensitivity: advances in the methodology of grounded theory. mill valley, ca: sociology press. gorry, g. a., & scott morton, m. s. (1971). a framework for management information systems. sloan management review, 13(1), 55–70. australasian journal of information systems freier & schumann 2021, vol 25, selected papers from acis 2019 dss in the context of cyber-physical systems 16 jiang, z., jin, y., e, m., & li, q. (2018). distributed dynamic scheduling for cyber-physical production systems based on a multi-agent system. ieee access, 6, 1855–1869. https://doi.org/10.1109/access.2017.2780321 karner, m., glawar, r., sihn, w., & matyas, k. (2019). an industry-oriented approach for machine condition-based production scheduling. procedia cirp, 81, 938–943. https://doi.org/10.1016/j.procir.2019.03.231 krumeich, j., jacobi, s., werth, d., & loos, p. (2014). big data analytics for predictive manufacturing control a case study from process industry. in 2014 ieee international congress on big data (bigdata congress), anchorage, ak, usa. krumeich, j., werth, d., loos, p., schimmelpfennig, j., & jacobi, s. (2014). advanced planning and control of manufacturing processes in steel industry through big data analytics: case study and architecture proposal. in 2014 ieee international conference on big data (big data), washington, dc, usa. lee, e. a. (2008). cyber physical systems: design challenges. in 2008 11th ieee international symposium on object and component-oriented real-time distributed computing, orlando, fl, usa. lee, j. (2015). smart factory systems. informatik-spektrum, 38(3), 230–235. https://doi.org/10.1007/s00287-015-0891-z mayring, p. (2014). qualitative content analysis: theoretical foundation, basic procedures and software solution. klagenfurt: beltz. myers, m. d. (2013). qualitative research in business & management (2. ed.). london: sage. peffers, k., tuunanen, t., rothenberger, m. a., & chatterjee, s. (2007). a design science research methodology for information systems research. journal of management information systems, 24(3), 45–77. https://doi.org/10.2753/mis0742-1222240302 pinedo, m. l. (2009). planning and scheduling in manufacturing and services. new york, ny: springer new york. https://doi.org/10.1007/978-1-4419-0910-7 sabuncuoglu, i., & goren, s. (2009). hedging production schedules against uncertainty in manufacturing environment with a review of robustness and stability research. international journal of computer integrated manufacturing, 22(2), 138–157. https://doi.org/10.1080/09511920802209033 schneeweiß, c. (1999). einführung in die produktionswirtschaft (siebte, neubearbeitete auflage). springer-lehrbuch. berlin, heidelberg: springer. retrieved from http://dx.doi.org/10.1007/978-3-662-06876-2 https://doi.org/10.1007/978-3-662-06876-2 schreiber, m., vernickel, k., richter, c., & reinhart, g. (2019). integrated production and maintenance planning in cyber-physical production systems. procedia cirp, 79, 534–539. https://doi.org/10.1016/j.procir.2019.02.095 schuh, g., & fuß, c. (eds.) (2015). prosense: ergebnisbericht des bmbf-verbundprojektes; hochauflösende produktionssteuerung auf basis kybernetischer unterstützungssysteme und intelligenter sensorik (1. aufl.). aachen: apprimus. schuh, g., potente, t., fuchs, s., thomas, c., schmitz, s., hausberg, c., . . . brambring, f. (2013). self-optimizing decision-making in production control. in lecture notes in production australasian journal of information systems freier & schumann 2021, vol 25, selected papers from acis 2019 dss in the context of cyber-physical systems 17 engineering. robust manufacturing control (vol. 54, pp. 443–454). berlin, heidelberg: springer. https://doi.org/10.1007/978-3-642-30749-2_32 schuh, g., potente, t., thomas, c., & hauptvogel, a. (2014). steigerung der kollaborationsproduktivität durch cyber-physische systeme. in industrie 4.0 in produktion, automatisierung und logistik (pp. 277–295). wiesbaden: springer. https://doi.org/10.1007/978-3-658-04682-8_14 schuh, g., potente, t., thomas, c., & hempel, t. (2014). short-term cyber-physical production management. procedia cirp, 25, 154–160. https://doi.org/10.1016/j.procir.2014.10.024 shim, j. p., warkentin, m., courtney, j. f., power, d. j., sharda, r., & carlsson, c. (2002). past, present, and future of decision support technology. decision support systems, 33(2), 111– 126. https://doi.org/10.1016/s0167-9236(01)00139-7 sprague, r. h., & carlson, e. d. (1982). building effective decision support systems. englewood cliffs, nj: prentice-hall. suh, e. e., kagan, s., & strumpf, n. (2009). cultural competence in qualitative interview methods with asian immigrants. journal of transcultural nursing: official journal of the transcultural nursing society, 20(2), 194–201. https://doi.org/10.1177/1043659608330059 tornatzky, l. g., & fleischer, m. (1990). the processes of technological innovation. issues in organization and management series. lexington, mass.: lexington books. vieira, g. e., herrmann, j. w., & lin, e. (2003). rescheduling manufacturing systems: a framework of strategies, policies, and methods. journal of scheduling, 6(1), 39–62. https://doi.org/10.1023/a:1022235519958 australasian journal of information systems freier & schumann 2021, vol 25, selected papers from acis 2019 dss in the context of cyber-physical systems 18 appendix a: interview guideline australasian journal of information systems freier & schumann 2021, vol 25, selected papers from acis 2019 dss in the context of cyber-physical systems 19 australasian journal of information systems freier & schumann 2021, vol 25, selected papers from acis 2019 dss in the context of cyber-physical systems 20 australasian journal of information systems freier & schumann 2021, vol 25, selected papers from acis 2019 dss in the context of cyber-physical systems 21 copyright: © 2021 freier & schumann. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v25i0.2849 http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems memar, krishna, mcmeekin & tan 2020, vol 24, research article testing gamification with time restrictions 1 investigating information system testing gamification with time restrictions on testers’ performance navid memar curtin university, bentley, wa 6102 australia navid.memar@postgrad.curtin.edu.au aneesh krishna curtin university, bentley, wa 6102 australia david a. mcmeekin curtin university, bentley, wa 6102 australia tele tan curtin university, bentley, wa 6102 australia abstract this paper presents the results obtained from the evaluation of gamified software testing platform that was developed following series of focus group discussions comprising of software developers and testers. the purpose of this study is to understand the effect of gamification as an additive method that can help improve the performance of software testers. additionally, in this study, new metrics have been introduced to quantify the performance of software testers fairly and more accurately. moreover, the effect of time restriction impacting on the performance of software testers will be discussed from results of this study. findings suggest that the proposed metrics, which more accurately capture the difficulty level of the software code defects, are able to better analyse and compare the performances of software testers in the gamified testing environment. moreover, results indicated that time restriction may compromise the performance of software testers and the quality of written software test code. on the other hand, results suggest that the performance of software testers in detecting low priority bugs in the gamified software-testing platform was better compared to the other more difficult to detect bugs. keywords: gamification; information system; evaluation; fairness 1 introduction information system testing and bug detection practice is still a challenging topic in the information software community. these activities are vital for the successful delivery of software projects. information system testing helps with assessing the software quality. one of the most crucial goals of software development activities is to achieve a high quality product that meets the estimated budgeted price and be delivered based on the estimated plan (blum, australasian journal of information systems memar, krishna, mcmeekin & tan 2020, vol 24, research article testing gamification with time restrictions 2 1992). information system testing is a very old concept in the history of digital computers and software products. however, it is expected that this practice will remain in the future as one of the best tools to help with assuring the reliability of software products (briand, 2007). additionally, information system testing practice plays an important role in software engineering by consuming 40 to 50% of the entire software development efforts and this rate might be higher for products which require higher level of reliability (luo, 2001). with the introduction of fourth generation languages (4gl), there is an acceleration in the implementation process of software products, which requires higher level of maintenance and upgrade of software systems. thus, for such processes, a proper amount of information system testing activity will be required to ascertain the quality of software products after the changes are made to the product (marciniak, 1994). with the increased frequency of software releases a higher amount of information system testing will be required to ensure the quality of the software products after changes are made (luo, 2001). insufficient testing can lead to various issues such as a need to constantly upgrade applications, economic loss by uncovering software bugs in banking or flight software systems and risking the lives of people due to software failures in vehicles and many other safety related areas (fraser, 2017). effective manual or automated testing can be performed to reduce the number of software related risks and disasters. manual and automated testing approaches can be seen as complementary to each other. automated testing is capable of performing a large number of tests in a short period of time, whereas manual testing depends on the tester’s skills and knowledge for specific cases where automated testing may fail (leitner, ciupa, meyer, & howard, 2007). testing requires comprehensive understanding of the program context to provide meaningful test suits while automation can lead to unrealistic tests without clear purpose (fraser, 2017). to design a useful test suite to detect bugs, an understanding of the intended program behaviour is required. thus, automation often requires additional human effort to achieve this goal (fraser, 2017). with manual testing, testers write test suites that may best exercise the program. whereas automated testing works towards removing the tediousness of the process by the software tool, which generates test cases from the software’s specification. on the other hand, information system testing exercise can be a repetitive and mundane activity and often not a very motivating exercise for many software developers (mäntylä & smolander, 2016). gamification may be a potential solution to these issues by transferring repetitive and boring tasks into fun and motivating activities, providing engaging and competitive environment to perform information system testing practices. this method helps to turn difficult tasks to components of entertaining gameplay, increasing player’s motivation and engagement by the use of game elements and mechanics in non-game contexts and using competitive nature of humans to motivate them to compete (deterding, dixon, khaled, & nacke, 2011; garcía, pedreira, piattini, cerdeira-pena, & penabad, 2017). this method may invoke the element of engagement to the serious activities involved in software development. thus, repetitive and manual tasks such as maintenance and writing unit test code for the software developers, could become stimulating using rewards and recognition obtained from gamification (de jesus, ferrari, de paula porto, & fabbri, 2018). gamification method has been successfully applied in different domains such as character recognition (von ahn, maurer, mcmillen, abraham, & blum, 2008), information system testing (chen & kim, 2012), language translation (von ahn, 2013), education (kapp, 2012) and others. australasian journal of information systems memar, krishna, mcmeekin & tan 2020, vol 24, research article testing gamification with time restrictions 3 this paper concentrates on the evaluation of the developed gamified software-testing tool to advance software-testing experience with the purpose to provide more engaging and rewarding environment for software testers. furthermore, this paper presents a new gamified metrics to evaluate the effectiveness of software testers’ performances fairly and more accurately. finally, this paper explores the importance of time pressure on information system testing practice and to evaluate if the proposed tool may improve the software testers’ performance. 2 background 2.1 gamification and motivational factors gamification can be defined as the use of game design elements, mechanics, aesthetics, and game thinking in non-game context. the goal of gamification is to improve the level of interest in technology by improving the engagement and motivation of users to perform activities (de sousa borges, durelli, reis, & isotani, 2014; kumar, 2013; pedreira, garcía, brisaboa, & piattini, 2015; seaborn & fels, 2015). gamification has recently become a buzzword and organizations are often adopting this method to increase their employee’s engagement and motivation and to increase the level of intended users’ engagement. to design a gamified system, effectiveness, efficiency, satisfaction, gaming and gamification are the main objectives in the context of gamification (kumar, 2013). additionally, collecting, connecting, achievement, feedback, self-expression, reciprocity and blissful productivity are examples of human motivational factors that have applicability to motivate users in real and virtual world (kumar, 2013). people enjoy collecting recognitions and, in some cases,, they may compare their collections to others, which may enhance the competency of the participants. some collections may be symbolic such as social statuses and some may have monetary values. furthermore, people are often motivated when they are part of something larger than themselves. connecting to different group of people, joining social clubs and socialising are approaches people often take to have meaningful shared experiences and to make life more enjoyable. on the other hand, people enjoy achievements. achievement delivers great satisfaction to people. feedback is another factor that is essential in increasing users’ motivation. for instance, software that provides no feedback to their users is not as much enjoyable to use as one that does. self-expression is another important motivation factor for users. users improve their profiles or players spend time to customize their avatars to control how other users or players view them. reciprocity is another technique to increase the motivation of users to participate or invest time on specific tasks. for instance, by entering to a store and receiving small free sample, customers might start feeling compelled to purchase the item out of a sense of reciprocity. finally, blissful productivity is essential to keep the users motivated to continue progressing. for instance, easy task may cause boredom and when a task is too difficult, it may cause users to be anxious. therefore, it is important to design tasks appropriately to keep the users motivated. in many studies, players’ level of engagement significantly increased after introducing game elements. australasian journal of information systems memar, krishna, mcmeekin & tan 2020, vol 24, research article testing gamification with time restrictions 4 there are also other studies that suggest a positive outcome after adopting gamification method. for instance, barata et al. (barata, gama, jorge, & gonçalves, 2013) used game elements such as scoring, levels, leader boards, challenges and badges to experiment the outcome of a master’s level college course. authors assessed the impact of gamification on the learning experience by comparing the gamified course and non-gamified version of it, which was practiced the previous year together with the rate of student satisfaction compared to other available courses in the similar academic context. results suggested a significant increase in lecture attendance, activity and perusing the course reference materials. furthermore, students indicated that gamification helped in turning complicated materials into interesting, motivating and easier to learn as compared to other courses. de freitas and de freitas (de freitas & de freitas, 2013) performed an experiment at the us air force academy by using a software gamification tool. they used classroom live technique to streamlines the gamification practice for the trainer by making mutual practices simpler. the tool also provided students with motivating user interface, which provided students with rewards in exchange for their participation. the experience suggested that the gamification tool was successful, and students’ respond were positive to gamification. in (de jesus et al., 2018), the authors performed a literature review on the use of gamification in the testing context in both education and practice. authors, retrieved 540 studies in total, however, based on the selection criteria specified by them, 15 studies were finally selected for their literature study. authors, conducted the literature review for contexts which proposed or applied a technology (approach, tool, framework, method etc.) to gamify information system testing practices and education. the study indicates that the gamification goals were to increase the users awareness regarding their performance and results, boost the adoption of information system testing, motivate the development of creativity to perform testing tasks, minimizing the effort in maintenance, encouraging developers to turn testing practice into a habit, increase the level of software tester’s engagement, and many others. further, they suggested that game elements such as achievement, avatar, badges, duel, leader board, level, points, quest, social graph, team and virtual good were used in the selected studies to achieve gamification goals. 2.2 gamification, serious game and game based learning (gbl) game based learning (gbl) is mainly used in educational domains and it is based on a learning game which has a beginning and an end. moreover, serious game is another common term used to propose games with the educational intention (connolly, boyle, macarthur, hainey, & boyle, 2012). gamification and gbl are two terms that sometimes could be confusing and cause misunderstanding for many researchers and game designers. (nah, telaprolu, rallapalli, & venkata, 2013) stated that turning activities or processes into a game by the use of game design elements significant behavioural change can be induced. the intention of gamification is not to make changes into learning but rather helps with enhancing learning, engagement and positive behaviour by using game design elements (alsawaier, 2018). additionally, a serious game is not created with the intention of entertainment rather it is created with a particular intention (djaouti, alvarez, & jessel, 2011). the concept of serious game could be used in different domains such as education, advertisements, politic, etc. in gbl, learners play games in order to learn content but in contrast; gamification involves the use of game elements in an environment outside of digital games. the main difference of australasian journal of information systems memar, krishna, mcmeekin & tan 2020, vol 24, research article testing gamification with time restrictions 5 serious games and gamifications could be noticed by knowing that gamified systems are not actual games while serious games are actual games (djaouti et al., 2011). 2.3 gamification and information testing the gamification method has been successfully practiced in information systems and applications in various domains (hamari, koivisto, & sarsa, 2014; kazhamiakin et al., 2015). information system testing is often a tedious, monotonous and boring practice (alrmuny, 2014; briand, 2007; shah & harrold, 2010). moreover, information system testing is also considered time-consuming and difficult activity to perform (alrmuny, 2014). however, gamification can be used as an effective method to help remedy the high level or repetitive tasks during testing. this method may help to increase testing engagement and performance during testing practice. recently there have been many studies in the field of software engineering by injecting gamification in the serious tasks to raise the level of motivation, performance and engagement of participants in software engineering practice (pedreira et al., 2015). for instance, (arai, sakamoto, washizaki, & fukazawa, 2014) introduced gamification as a method to encourage software developers to remove warnings of bug pattern tools and the results indicate that developers were successful in removing 150% warning with the suggested method in compare to the case where participants did not practice the proposed method. moreover, (singer & schneider, 2012) used gamification method to motivate computer science students to deploy additional frequent commits to version control. in (johansson & ivarsson, 2014), the authors explored the effects of gamification on the improvement of unit testing. they used the gamification method to motivate the software testers to perform testing practices. authors conducted the experiments by including 24 individuals. the effectiveness was evaluated by considering the number of detected bugs as well as requirements covered by the tests. the results suggested a significant improvement for the gamification group in identifying faults while improving level of code coverage by tests written. further, results indicate that gamification method increased the level of motivation among the testers. the study conducted by (arnarsson & jóhannesson, 2015) used the gamification method to encourage developers to write better unit test codes. developers’ performances were evaluated after analysing both static and dynamic qualities of their written unit test codes. developers indicated that gamification encouraged them to generate better tests and the tool assisted them in learning about essential information system testing metrics and concepts. in (rojas & fraser, 2016), researchers applied gamification technique to demonstrate mutation testing and to improve testing skills. code defenders tool was introduced by researchers to assist practitioners in providing complex mutation testing concepts and to provide a more enjoyable learning experience for students. in code defenders players play the role of attackers and defenders to test a java class. the role of attackers is to provide subtle mutants of the java class, which are difficult to detect. the defenders’ goal is to create good tests that can detect and isolate the attacker’s mutants which can be used as an effective test suite for the unit under test. 2.4 time pressure effects on task performance the effects of time pressure on software development with real (agrawal & chari, 2007; nan & harter, 2009) and virtual data (austin, 2001; valett & mcgarry, 1989) at the project level have been discussed in several studies. these research studies often focused on assessing the australasian journal of information systems memar, krishna, mcmeekin & tan 2020, vol 24, research article testing gamification with time restrictions 6 project context on large tasks. in (nan & harter, 2009), the authors studied the effect of time pressure and their findings suggests a u-shape effect, which is known as yerkes-dodson law (yerkes & dodson, 1908). the results suggested that, less time pressure can improve the performance of the participants, but with excessive time pressure, the performance drops and has undesirable effects due to increasing level of errors and poor individual commitment to unrealistic expectations. it is also important to identify the effect of time pressure on small tasks. in (topi, valacich, & hoffer, 2005), authors studied the effect of time pressure in the software engineering domain in the small task context on human performance. their findings on participants’ creation database queries indicated that time pressure did not affect task performance. additionally, another study in the domain of accounting (mcdaniel, 1990), evaluated the effect of time pressure on 179 professional staff auditors. the results indicated that time restriction and pressure provided a better efficiency but affected the effectiveness of individual auditors. furthermore, in (mäntylä & itkonen, 2013), the authors studied the effect of crowd size and time restriction in information system testing and results suggested that time pressure on group of five testers using 10 hours in total resulted in detected more defects than a single non time restricted tester using 9.9 hours. 2.5 relationship between performance evaluation and job satisfaction many studies have been conducted about the role of social and situational effects on performance-rating process. performance evaluation plays an important role in human resources systems in organizations. managers and supervisors’ rating of employees’ performance leads to serious decisions that are the main influences on a many subsequent human resources actions and outcomes. indeed, the importance of performance evaluation has resulted in higher efforts to provide a more enhanced performance rating process. the close relationship between job satisfaction and job performance has been studied and addressed previously (gross & etzioni, 1985). in (bartol, durham, & poon, 2001) authors explored the effect of rating segmentation on fairness and motivation level. findings supported that rating segmentation is an important factor as it may influence employees’ motivation and work attitude. additionally, employees often have concerns about their individual performance and how they get evaluated compared to relevant peers (lyubomirsky & ross, 1997). the importance of fairness in the organizations and its impact on organisational effectiveness, forms a crucial component and it has been named ‘organizational justice research’ (sholihin & pike, 2009). existing organizational literature suggests that justice perceptions regarding the organizational procedures are strongly connected to workplace outcomes such as motivation, commitment of employees and performance (sholihin & pike, 2009). it is also suggested that performance evaluation practice is an important factor in organizational justice perception (sholihin & pike, 2009). lau et al. (lau, wong, & eggleton, 2008) studied about the relationship between job satisfaction and fairness of performance evaluation procedures. this study suggested that performance evaluation fairness impacts job satisfaction level by two separate processes, one is the fairness of outcome and the second is from the trust in superior and organisational commitment. in this paper, we also study existing metrics to evaluate the software testers’ performances and argue that how these metrics could be improved by introducing a new gamified metric to evaluate the performance of software testers more accurately. australasian journal of information systems memar, krishna, mcmeekin & tan 2020, vol 24, research article testing gamification with time restrictions 7 3 earlier work in this section, a brief discussion on the related works carried out previously is presented. it helps lay the context within which the gamified information system testing platform reported in this study has being developed and evaluated. 3.1 prototype implementation in our initial study (memar, krishna, mcmeekin, & tan, 2017), issues such as lack of interest and motivation were identified as barriers for computing graduates (and students) to consider with respect to information system testing. gamification is the strategy suggested as a potential solution to address these issues (memar, krishna, mcmeekin, & tan, 2018). the gamification method has the potential to increase software testers’ engagement. also, it can help remedy the high level of repetition and reduce the boredom level of testers while executing their testing activities. the serious game principles suggested by whyte et al. (whyte, smyth, & scherf, 2015) assisted the process of identifying the game elements that are suitable for the gamified information system testing platform. a series of focus group sessions involving software developers and testers assisted in determining the main elements to be applied in the gamified information system testing platform. the main purpose of the study (memar et al., 2017) was to determine if the design elements and game core elements can help to increase the learning and motivation of software testers. the core serious game principles are categorised as story line, goal directed learning around targeted skills, feedback and rewards, badges and levels and provision of choice. story line helps in improving the level of motivation, quality and engagement. additionally, goal directed learning helps with providing a challenging environment. furthermore, feedback and awards play a critical part in shaping behaviour in players and serious games. badges and levels help to provide a challenging, but still achievable level of difficulty for software testers. finally, provision of choice helps players to be able to have choice in some aspects of the game to have choice over some aspects of the game. figure 1 shows the vital game elements to design a gamified information system testing platform. results suggested that all game core elements are essential for gamifying the information system testing platform. among the listed categories (storyline with 33%, goals direct learning around targeted skills with 39%, increasing level of difficulty and individuation with 33%, feedback and rewards shape learning with 78% and provision of choice with 17%) participants agreed that feedback and rewards are the main factors that help motivate software testers (memar et al., 2017). additionally, figure 2 presents the essential information required for software testers identified through the focus group sessions. the majority of participants suggested that requirements and design documentation with 86% as well as knowledge with 36% are the key elements required for software testers among the other listed elements (tools with 29% and time with 7%) (memar et al., 2017). australasian journal of information systems memar, krishna, mcmeekin & tan 2020, vol 24, research article testing gamification with time restrictions 8 figure 1. design elements and motivation (memar et al., 2017) figure 2. factors required for software testers (memar et al., 2017) 3.2 preliminary prototype validation the developed prototype was subsequently validated by a group of software developers and testers and reported in (memar et al., 2017). the final product consists of all essential game elements that were identified during the earlier study. game elements, such as points, real gifts, badges (for difficulty levels), feedback, comparison and provision of choice have been included into the gamified software product in order to increase the level of software testers’ motivation and engagement. in the gamified platform, testers require certain amount of points to unlock each badge. additionally, testers’ performance was than evaluated by the review team and appropriate points assigned to each tester. moreover, the comparison helps to boost the performance of testers after reviewing other testers’ performances. for instance, testers will be able to compare their performance against other testers and get motivated to work harder to achieve better results. furthermore, provision of choice allows each tester to have control over incoming testing requests to either accept or decline the requests. finally, gifts (rewards) are presented to each tester who succeeded in collecting the appropriate points to unlock the final badge (memar et al., 2017). 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% storyline goals direct learning around targeted skills increasing level of difficulty and individuation feedback and rewards shape learning provision of choice r es p o n se r at e design elements serious games design elements 0% 20% 40% 60% 80% 100% requirement & design documentation tools time knowledge r es p o n se r at e factor categories vital information required for software testers australasian journal of information systems memar, krishna, mcmeekin & tan 2020, vol 24, research article testing gamification with time restrictions 9 results of that study indicated that the developed gamified information system testing platform and gamification may be a solution to improve testers’ performance, engagement and testing experience. participants of this study suggested that the developed platform is interesting and motivating for software testers. furthermore, results suggested that gamification can be used as a tool to increase quality of information system testing activities (memar et al., 2018). 4 methodology as discussed in the background section, there are few studies related to information system testing and gamification. critical gaps remain in the current framework that need to be addressed to ensure its wider adoption. the objectives of this study are as follow: 1. to identify the perceived effect of time restriction in a gamified information system testing environment on individual software tester performance; 2. introduce a new metric to evaluate the performance of software testers fairly and more accurately; 3. to identify the effect of gamification on detecting different levels of bugs within the software. this section explains the chosen method for the evaluation of software testers’ performance in a gamified information system testing platform. for this purpose, activities have been discussed in detail. 4.1 evaluation of final gamified testing platform the next step of the research was to first, evaluate the effect of time pressure on the software testers’ performance and to identify if time restriction is a factor to motivate the software testers to enhance their productivity. for this purpose, 25 computing undergraduate students who had experience in software development and software testing as part of their undergraduate course were recruited. some participants have participated in industry-based software development projects. the human research ethics committee of curtin university (approval number hr28/2016) approved the study. additionally, participants have all been introduced to information system testing technique using the junit framework. participants were briefed on the gamified platform and were given enough time to become familiar with the environment. in total participants were given two tasks. task 1 consisted of 6 easy, 2 medium and one hard bugs while the second tasks consisted of 2 easy, 4 medium and 2 hard bugs. participants were briefed about each task and had the opportunity to ask questions during their reading time. participants were given 15 minutes and 20 minutes to work on task 1 and 2 respectively. moreover, an additional 5 minute’s reading time at the beginning of each task was assigned to help participants be familiar with the tasks to be performed. to ensure that participants were aware of the code contents, comments about the code were added alongside the code. table 1 presents the number of bugs introduced for each task. bugs were grouped into three categories: easy, medium and hard. tasks easy medium hard 1 6 2 1 2 2 4 2 table 1. number of bugs for each task australasian journal of information systems memar, krishna, mcmeekin & tan 2020, vol 24, research article testing gamification with time restrictions 10 the following are the description and examples for each bug type: a) easy: a bug or mistake in the software that can be fixed by modifying one line of source code, or a bug whose invalid logic resides in one line of source code. example 1: var = “temp”; // instead of var = imported_variable example 2: setting the value of a field to the imported value; without validation (i.e. checking for null). b) medium: logical errors that will not function correctly on all occasions. example 1: fuel += amount; // this refuels the fuel tank by the given amount, but it does not check for invalid fuel amounts (i.e. negative). example 2: not checking array bounds. the logic will work fine until the array is full, and then the program will crash. c) hard: bugs are classified hard if they are both hard to detect, and only trigger on rare cases. example 1: in a class named ‘book’, writing a page works fine unless the page is full and the number of pages in the book is full. example 2: driving the car works fine in most cases, except that the car should not drive if there is no fuel. written questionnaires and recorded discussions were used to establish the evaluation results. in the following section, the findings from this study are explained in detail. 5 results and analysis this section presents the results and findings obtained from the prototype. in the first section of results, the focus is on the qualitative results obtained from the questionnaires during the evaluation session. the given questions are designed to identify the effect of time restriction on software testers’ performances. in the second part, the available metrics are defined and new metrics to measure the performance of software testers are introduced. furthermore, a validation of the proposed metrics together with a comparison of existing and the proposed metrics will be explained. finally, the last section of results, presents the performance evaluation results of software testers for each level of task difficulty. this comparison helps to identify the effectiveness of gamification based on the level of task difficulty. 5.1 qualitative results in this evaluation, 25 participants, attended the evaluation session and out of them 52% have both software development and software testing background, 44% have software development background and rest have information system testing knowledge. all participants have experience in unit testing technique using the junit framework. during the evaluation, students were given an introduction about the gamified platform and had the chance to test the platform and experience the gamified environment. participants were given the chance to write test codes for 2 different tasks. as discussed earlier, students were given 5 minutes of reading time for each task. additionally, testers were given 15 and 20 minutes to work on task 1 and task 2 respectively. tasks were designed in a way for testers to be able to understand the code easily and to be able to write test codes for the given tasks in the given time frame. at the end of the evaluation session, participants were given the questionnaire to australasian journal of information systems memar, krishna, mcmeekin & tan 2020, vol 24, research article testing gamification with time restrictions 11 provide feedback on the effect of time restriction on their testing experience. in the questionnaire, participants were given the following question to answer: “what is the effect of time restriction on the information system testing performance in the current gamified information system testing platform?”. results suggested that majority of participants agreed to the fact that time pressure may compromise the performance of software testers. some of the responses are listed below as ready reference: • “having a time restriction may place pressure on the tester, causing the tester to rush.” • “the time constraints make me feel under pressure and i do not think logically due to the stress.” • “the time restriction puts a level of pressure, making you feel rushed, leading to the lack of time to properly read the code and understand what is going on and leading to rushed testing.” • “time restriction in my opinion is a limiting factor when it comes to software testing. a good software tester is able to get work done in a good amount of time without having an artificial restriction imposed on them. longer timeframes may improve performance because the tester will be able to write all test cases anyways, but overall, i feel it is diminishing when the timeframe is too low.” another question was asked on what is the effect of time restriction on the software testing quality in the current gamified information system testing platform and results suggest that everyone agreed that time restriction can compromise the software testing quality. some of the responses are listed below as ready reference: • “depending on how long the restriction is, just like writing code, with writing test harnesses mistakes can be made if in a rush. and without the proper time to go through it all, the mistakes can get through and create incorrect results.” • “under time restriction, a subject would be more focused on finding more solutions to a problem rather than elaborating on them. limited time may leave a subject spending less time on an individual task to ensure they get more tasks done.” • “when having a time restriction it led to feeling rushed, meaning i didn’t have time to properly test to a high enough quality as i had to keep in mind of the time. meaning i could not cover all major test cases, meaning more bugs could slip through.” • “when there is a time restriction, there is pressure to rush and compromise quality for quantity. this leaves room to miss several edge cases and therefore not catch as many bugs.” furthermore, participants were given the following question to answer: “how likely is time pressure making testers more productive?”. the responses suggest that 36% agreed of the fact that time pressure may be a method to increase the productivity of software testers while 32% did not agree based on the fact and the rest of participants were not sure if that would make testers more productive. at the end of the questionnaire, participants were given a chance to provide additional comments or suggestions which some of the responses are listed below as ready reference: australasian journal of information systems memar, krishna, mcmeekin & tan 2020, vol 24, research article testing gamification with time restrictions 12 • “if time restriction is ever to have any positive impact, a large amount of research should be dedicated to finding how much time testers should be given for a certain task. giving too much time or too little time would decrease the performance of the tester, but if a certain limit is reached, it may improve performance. but this limit would be different per tester and per task, so it seems unlikely to be feasible.” • “it really all depends on who the testers are, as some people become more productive with time restraints and others will get stressed out and end up doing less work. personally, having a flexible time pressure helps me ensure that i try to finish it for a dead line, however if needed can extend the time to get more work done.” 5.2 quantitative results 5.2.1 metrics definition in the information system testing context, one of the important factors for software testers’ performances is to be evaluated fairly and accurately and this may result in higher job satisfaction rate in organisations (a detailed discussion has been provided in the background section). in this section new metrics will be introduced for this purpose. by using the proposed metrics, software testers’ performance could be evaluated based on the importance of identified bugs. additionally, a comparison of new metrics with existing metrics will be provided to validate the proposed metrics. a detailed discussion will be provided to explain why these metrics should be adopted for information system testing performance evaluation to provide more accurate evaluation results. finally, a detailed discussion about the accuracy and fairness of proposed metrics will be discussed. 1. effectiveness metric can be calculated with the following formula : e = vdf / vdt while the proposed gamified metric evaluates the performance of testers with the use of following formula: gamified e= (((0.2*e)/e)+((0.3*m)/m) + ((0.5*h)/h)). in the effectiveness metrics, e is effectiveness, vdf is the number of unique valid defects found, and vdt represents the total number of unique valid defects (mäntylä & itkonen, 2013). in contrast, in the gamified effectiveness metric, gamified e is the effectiveness, e represents the number of unique valid easy defects found, e is the total number of easy defects, m is the number of unique valid medium defects found, m is the total number of medium defects, h in the number of unique valid hard defects identified by software testers and h is the total number of hard bugs existing in the software. this new metric acknowledges the performance of software testers based on the level of detected bugs. for instance, each identified easy bug consists of 0.2 points, while each valid identified medium and hard bug consist of 0.3 and 0.5 points respectively. the new metric provides weight for each particular bug found by the tester depending on the importance of each bug detected. figure 3 presents the comparison of effectiveness, known as recall in (baeza-yates & ribeiro, 2011)) and gamified metrics. results suggest that the correlation coefficient between the two metrics is 0.96656 which shows a strong relation between the two metrics. 2. although each tester may identify a certain share of unique defects, they additionally may produce a set of invalid bugs (also referred as false positive (dunsmore, roper, & wood, 2003)). the share of valid unique findings among all australasian journal of information systems memar, krishna, mcmeekin & tan 2020, vol 24, research article testing gamification with time restrictions 13 findings is called validity (hartson, andre, & williges, 2001) and in the domain of information retrieval, this is commonly referred to as precision (baeza-yates & ribeiro, 2011) and this number is often not reported in in empirical software engineering (mäntylä & itkonen, 2013). validity or precision can be calculated with the following formula: v= (tp / (tp + fp)) where tp is the true positive, (when tester detects a bug and bug exists) and fp represents the false positive (invalid bug reports). in contrast, the proposed gamified validity metric evaluates the performance of testers with the use of following formula: gamified v = (((0.2*e)/(e + fp) )+((0.3*m)/(m + fp)) + ((0.5*h)/(h + fp))). figure 4 presents the comparison of validity and gamified validity (known as precision (baeza-yates & ribeiro, 2011)) metrics. results indicate that the two metrics have a high correlation coefficient with a value of 0.96364. this number supports the fact that there is a strong relation between the two metrics. 3. lastly, for decision making purposes, the combination measures of effectiveness and validity will be beneficial to evaluate the overall performance of testers more accurately. the importance of this combination measure is in determining better performance results. in the information retrieval domain, both effectiveness and validity are combined in a measure called f-score (van rijsbergen, 1986). f score can be calculated with the following formula: fs = 2 * ((validity * effectiveness) / (validity + effectiveness)) (mäntylä & itkonen, 2013) while, the proposed gamified f score can be calculated as following: gamified fs= 2 * ((gamified validity * gamified effectiveness) / (gamified validity + gamified effectiveness)). figure 5 presents the comparison of f score and gamified f score metrics. results indicate that the two metrics have a strong correlation coefficient with a value of 0.96534. figure 3. effectiveness metric vs. gamified effectiveness 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 ef fe ct iv en es s gamified effectiveness australasian journal of information systems memar, krishna, mcmeekin & tan 2020, vol 24, research article testing gamification with time restrictions 14 figure 4. validity metric vs. gamified-validity metric figure 5. f score metric vs. gamified f score metric in order to categorise the performance of software testers, gamified effectiveness metric has been selected. the performance of the participants was grouped into three categories of low (performance rate less than 0.2), medium (performance score between 0.2 and 0.5) and high (performance rate greater than 0.5). table 2 presents the performance evaluation of participants using effectiveness metric while table 3 presents the performance of testers using the gamified effectiveness metric. for this purpose, performance of all 25 participants have been evaluated. findings suggest that effectiveness metric provide similar performance result for participants that identified the same number of bugs by disregarding the importance of bugs identified by the participants while the proposed gamified effectiveness metric provides more accurate performance results by considering the type of bug detected by participants. for instance, participants who identified more important bugs get better performance results compared to those who identified non-critical bugs. for this comparison, we calculated the average performance of 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 v al id it y gamified validity 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 fs co re gamified f score australasian journal of information systems memar, krishna, mcmeekin & tan 2020, vol 24, research article testing gamification with time restrictions 15 participants using both metrics. furthermore, average performance of participants in identifying total number of bugs helped to find the relationship between the scores obtained from effectiveness metric and number of bugs detected by the participants. in contrast, average performance of participants in detecting different categories of bugs helped the researcher to identify the relationship between the performance of participants and their performance in identifying different levels of defects using the gamified effectiveness metric. moreover, gamified validity and gamified f score metrics provide more accurate evaluation result considering different level of bugs while the existing validity and f score metrics do not consider the importance of bugs identified by testers. table 4 presents the performance evaluation of participants using validity metric while table 5 shows the performance evaluating using gamified validity metric. furthermore, as discussed earlier, for decision making purposes, the combination measures of effectiveness and validity will help to evaluate the performance of participants more accurately. table 6 represents the participants’ performance evaluation using the f score metric. however, in order to provide more accurate and fairer results, gamified f score metric was used. table 7 represent the results of participants’ performance using gamified f score metric. effectiveness performance score (number) total bugs (%) low 0.1235 12.35 % medium 0.2941 29.41% high 0.6039 60.39% table 2. performance evaluation using effectiveness metric gamified effectiveness performance score (number) easy (%) medium (%) hard (%) low 0.0797 14.28% 15.47% 0.95% medium 0.2893 30.35% 28.57% 28.57% high 0.6055 66.66% 50% 64.44% table 3. performance evaluation using gamified effectiveness metric validity performance score (number) total bugs (%) invalid bugs (number) low 0.1108 12.35 % 2.5 medium 0.2727 29.41% 1.42 high 0.5692 60.3% 1 table 4. performance evaluation using validity metric gamified validity performance score (number) easy (%) medium (%) hard (%) invalid bugs low 0.0740 14.28% 15.47% 0.95% 2.5 medium 0.2764 30.35% 28.57% 28.57% 1.42 high 0.5666 66.66% 50% 64.44% 1 table 5. performance evaluation using gamified validity metric australasian journal of information systems memar, krishna, mcmeekin & tan 2020, vol 24, research article testing gamification with time restrictions 16 f score performance score (number) total bugs (%) invalid bug(number) low 0.1166 12.35 % 2.5 medium 0.2829 29.41% 1.42 high 0.5857 60.3% 1 table 6. performance evaluation using f score metric gamified f score performance score (number) easy (%) medium (%) hard (%) invalid bug(number) low 0.0767 14.28% 15.47% 0.95% 2.5 medium 0.2826 30.35% 28.57% 28.57% 1.42 high 0.5845 66.66% 50% 64.44% 1 table 7. performance evaluation using gamified f score metric 5.2.2 effectiveness of software testers’ performance based on the level of difficulty in this section, we study the effect of gamification on the performance of the participants based on the level of task difficulty. table 8 presents the performance of all participants in the given tasks. in order to calculate the performance of the participants, we use the effectiveness, validity and f score metrics respectively. it is important to note that the proposed gamified metrics could evaluate the overall performance of participants when considering all levels of bugs. however, in this case, we are evaluating the performance of testers focusing on a specific category of bugs. thus, the existing metrics will be used to evaluate the participants’ performance. in order to evaluate the performance of the participants we use the following metrics: 1. effectiveness = vdf / vdt 2. validity= (tp / (tp + fp)) 3. f score = 2 * ((validity * effectiveness) / (validity + effectiveness)) results suggest that the performance of participants in detecting easy bugs in the gamified information system testing platform was higher compared to other levels categories. the average performance of all 25 participants in performing testing activity to detect bugs categorised as easy using f score metric was 23.65% while their average performance for detecting medium and hard bugs were 18.9% and 12.8% respectively. in order to calculate these, measures that were presented in the information retrieval domain were used (baezayates & ribeiro-neto, 1999). however, these measures have been partially adopted by the usability community (hartson et al., 2001) and software engineering community (mäntylä & itkonen, 2013). australasian journal of information systems memar, krishna, mcmeekin & tan 2020, vol 24, research article testing gamification with time restrictions 17 measures e as y (t as k 1 ) e as y ( t as k 2 ) a v er ag e p er fo rm a n ce fo r e as y t as k 1 & 2 m ed iu m ( t as k 1 ) m ed iu m ( t as k 2 ) a v er ag e p er fo rm a n ce fo r m e d iu m t as k 1 & 2 h ar d ( t as k 1 ) h ar d ( t as k 2 ) a v er ag e p er fo rm a n ce fo r h ar d t as k 1 & 2 effectiveness 24.66% 24% 24.33% 8% 30% 19% 4% 22.4% 13.2% validity 23.06% 23.33 23.19% 8% 29.8% 18.9% 4% 21.06% 12.53% f score 23.7% 23.6% 23.65% 8% 29.8% 18.9% 4% 21.6% 12.8% table 8. effectiveness of software testers’ performance based on the level of difficulty 6 discussion and limitation in this paper, results were provided after evaluating the final gamified information system testing platform through the evaluation session conducted with 25 undergraduate computing students. due to higher expectation or different view in relation to evaluation of gamified information system testing by a larger group of professional software testers, the results obtained may impact the outcome of this study. additionally, in order for students to show interest in participating in the focus group session, the duration of the session had to be less than an hour period. thus, time pressure may be the main factor that affected the participants’ performances in this study and resulted in lower performance rate by the participants. moreover, in the given questionnaire, students agreed that the time pressure could be a factor to affect the quality and performance of software testers. in this study, researchers have tried to choose participants who had knowledge in both software testing (mainly unit testing) and software development. finally, researcher discovered that the proposed metric is beneficial when evaluating the performance of software testers in combination of different levels of bugs (by considering the importance of bugs identified by software testers). however, in order to evaluate the performance of software testers for a specific category of bugs, the proposed metric may not be beneficial to evaluate the performance of software testers. 7 conclusion and future work in this paper, the topics of gamification, information system testing, and effect of time restriction on performance were discussed. results suggested that majority of participants agreed to the fact that time pressure may compromise the performance of software testers impacting on the information system testing efficacy. the quantitative results suggested that the performance of software testers was affected by the time pressure introduced during the prototype evaluation. in addition, a set of new metrics were proposed to better capture the performance of software testers. it has been demonstrated that these metrics are able to fairly distribute the scores to reflect on the types of bugs being reported. further work will include evaluating the use of gamified software-testing platform, the performance of software testers working in teams versus individually. additionally, researcher plans to investigate the importance of introducing the proposed metrics for evaluating the performance of software testers in the gamified software-testing platform prior to conducting the testing activity. this then will help to identify if the new metrics could be beneficial in assisting the testers to detect australasian journal of information systems memar, krishna, mcmeekin & tan 2020, vol 24, research article testing gamification with time restrictions 18 higher priority bugs within the software. finally, researcher also plan to investigate the amount of time required for information system testing tasks. this may help to identify if this can lead to higher level of performance efficiency. references agrawal, m., & chari, k. (2007). software effort, quality, and cycle time: a study of cmm level 5 projects. ieee transactions on software engineering, 33(3). alrmuny, d. z. (2014). open problems in software test coverage. lecture notes on software engineering, 2(1), 121. alsawaier, r. s. (2018). the effect of gamification on motivation and engagement. the international journal of information and learning technology, 35(1), 56-79. arai, s., sakamoto, k., washizaki, h., & fukazawa, y. (2014). a gamified tool for motivating developers to remove warnings of bug pattern tools. paper presented at the 6th international workshop on empirical software engineering in practice (iwesep), 2014 . arnarsson, d., & jóhannesson, í. h. (2015). improving unit testing practices with the use of gamification. chalmers university of technology, gothenburg, sweden. austin, r. d. (2001). the effects of time pressure on quality in software development: an agency model. information systems research, 12(2), 195-207. baeza-yates, r., & ribeiro-neto, b. (1999). modern information retrieval (vol. 463): acm press new york. baeza-yates, r., & ribeiro, b. d. a. n. (2011). modern information retrieval: new york: acm press; harlow, england: addison-wesley. barata, g., gama, s., jorge, j., & gonçalves, d. (2013). engaging engineering students with gamification. paper presented at the 5th international conference on games and virtual worlds for serious applications (vs-games), 2013. bartol, k. m., durham, c. c., & poon, j. m. (2001). influence of performance evaluation rating segmentation on motivation and fairness perceptions. journal of applied psychology, 86(6), 1106. blum, b. i. (1992). software engineering: a holistic view: oxford university press, inc. briand, l. c. (2007). a critical analysis of empirical research in software testing. paper presented at the first international symposium on empirical software engineering and measurement (esem 2007), ieee. chen, n., & kim, s. (2012). puzzle-based automatic testing: bringing humans into the loop by solving puzzles. proceedings of the 27th ieee/acm international conference on automated software engineering (ase), 2012. connolly, t. m., boyle, e. a., macarthur, e., hainey, t., & boyle, j. m. (2012). a systematic literature review of empirical evidence on computer games and serious games. computers & education, 59(2), 661-686. de freitas, a. a., & de freitas, m. m. (2013). classroom live: a software-assisted gamification tool. computer science education, 23(2), 186-206. australasian journal of information systems memar, krishna, mcmeekin & tan 2020, vol 24, research article testing gamification with time restrictions 19 de jesus, g. m., ferrari, f. c., de paula porto, d., & fabbri, s. c. p. f. (2018). gamification in software testing: a characterization study. paper presented at the proceedings of the iii brazilian symposium on systematic and automated software testing. de sousa borges, s., durelli, v. h., reis, h. m., & isotani, s. (2014). a systematic mapping on gamification applied to education. paper presented at the proceedings of the 29th annual acm symposium on applied computing. deterding, s., dixon, d., khaled, r., & nacke, l. (2011). from game design elements to gamefulness: defining gamification. paper presented at the proceedings of the 15th international academic mindtrek conference: envisioning future media environments. djaouti, d., alvarez, j., & jessel, j.-p. (2011). classifying serious games: the g/p/s model. in handbook of research on improving learning and motivation through educational games: multidisciplinary approaches (pp. 118-136): igi global. dunsmore, a., roper, m., & wood, m. (2003). the development and evaluation of three diverse techniques for object-oriented code inspection. ieee transactions on software engineering, 29(8), 677-686. fraser, g. (2017). gamification of software testing. paper presented at the 12th international workshop on automation of software testing (ast), 2017 ieee/acm. garcía, f., pedreira, o., piattini, m., cerdeira-pena, a., & penabad, m. (2017). a framework for gamification in software engineering. journal of systems and software, 132, 21-40. gross, e., & etzioni, a. (1985). organizations in society: prentice-hall. hamari, j., koivisto, j., & sarsa, h. (2014). does gamification work?--a literature review of empirical studies on gamification. paper presented at the 2014 47th hawaii international conference on system sciences (hicss). hartson, h. r., andre, t. s., & williges, r. c. (2001). criteria for evaluating usability evaluation methods. international journal of human-computer interaction, 13(4), 373-410. johansson, m., & ivarsson, e. (2014). an experiment on the effectiveness of unit testing when introducing gamification. phd thesis, master's thesis, chalmers university of technology (june), kapp, k. m. (2012). the gamification of learning and instruction: game-based methods and strategies for training and education: john wiley & sons. kazhamiakin, r., marconi, a., perillo, m., pistore, m., valetto, g., piras, l., . . . perri, n. (2015). using gamification to incentivize sustainable urban mobility. paper presented at the ieee first international smart cities conference (isc2), 2015. kumar, j. (2013). gamification at work: designing engaging business software. paper presented at the international conference of design, user experience, and usability, berlin heidelberg, 2013. lau, c. m., wong, k. m., & eggleton, i. r. (2008). fairness of performance evaluation procedures and job satisfaction: the role of outcome‐based and non‐outcome‐based effects. accounting and business research, 38(2), 121-135. australasian journal of information systems memar, krishna, mcmeekin & tan 2020, vol 24, research article testing gamification with time restrictions 20 leitner, a., ciupa, i., meyer, b., & howard, m. (2007). reconciling manual and automated testing: the autotest experience. paper presented at the 40th annual hawaii international conference on system sciences, 2007. hicss 2007.. luo, l. (2001). software testing techniques. institute for software research international carnegie mellon university pittsburgh, pa, 15232(1-19), 19. lyubomirsky, s., & ross, l. (1997). hedonic consequences of social comparison: a contrast of happy and unhappy people. journal of personality and social psychology, 73(6), 1141. mäntylä, m. v., & itkonen, j. (2013). more testers–the effect of crowd size and time restriction in software testing. information and software technology, 55(6), 986-1003. mäntylä, m. v., & smolander, k. (2016). gamification of software testing-an mlr. paper presented at the international conference on product-focused software process improvement. marciniak, j. j. (1994). encyclopedia of software engineering (vol. 1 an): wiley-interscience. mcdaniel, l. s. (1990). the effects of time pressure and audit program structure on audit performance. journal of accounting research, 267-285. memar, n., krishna, a., mcmeekin, d. a., & tan, t. (2017). gamification of information system testing-design consideration through focus group discussion. paper presented at the 26th international conference on information systems development, sep 6, 2017, larnaca, cyprus. memar, n., krishna, a., mcmeekin, d. a., & tan, t. (2018). gamifying information system testing–qualitative validation through focus group discussion. paper presented at the 27th international conference on information systems development, aug 9, 2018, larnaca, sweden. nah, f. f.-h., telaprolu, v. r., rallapalli, s., & venkata, p. r. (2013). gamification of education using computer games. paper presented at the international conference on human interface and the management of information. nan, n., & harter, d. e. (2009). impact of budget and schedule pressure on software development cycle time and effort. ieee transactions on software engineering, 35(5), 624637. pedreira, o., garcía, f., brisaboa, n., & piattini, m. (2015). gamification in software engineering–a systematic mapping. information and software technology, 57, 157-168. rojas, j. m., & fraser, g. (2016). code defenders: a mutation testing game. paper presented at the 2016 ieee ninth international conference on software testing, verification and validation (icstw). seaborn, k., & fels, d. i. (2015). gamification in theory and action: a survey. international journal of human-computer studies, 74, 14-31. shah, h., & harrold, m. j. (2010). studying human and social aspects of testing in a service-based software company: case study. paper presented at the proceedings of the 2010 icse workshop on cooperative and human aspects of software engineering. sholihin, m., & pike, r. (2009). fairness in performance evaluation and its behavioural consequences. accounting and business research, 39(4), 397-413. australasian journal of information systems memar, krishna, mcmeekin & tan 2020, vol 24, research article testing gamification with time restrictions 21 singer, l., & schneider, k. (2012). it was a bit of a race: gamification of version control. paper presented at the 2nd international workshop on games and software engineering (gas), 2012 topi, h., valacich, j. s., & hoffer, j. a. (2005). the effects of task complexity and time availability limitations on human performance in database query tasks. international journal of human-computer studies, 62(3), 349-379. valett, j. d., & mcgarry, f. e. (1989). a summary of software measurement experiences in the software engineering laboratory. journal of systems and software, 9(2), 137-148. van rijsbergen, c. j. (1986). a non-classical logic for information retrieval. the computer journal, 29(6), 481-485. von ahn, l. (2013). duolingo: learn a language for free while helping to translate the web. paper presented at the proceedings of the 2013 international conference on intelligent user interfaces. von ahn, l., maurer, b., mcmillen, c., abraham, d., & blum, m. (2008). recaptcha: humanbased character recognition via web security measures. science, 321(5895), 1465-1468. whyte, e. m., smyth, j. m., & scherf, k. s. (2015). designing serious game interventions for individuals with autism. journal of autism and developmental disorders, 45(12), 3820-3831. yerkes, r. m., & dodson, j. d. (1908). the relation of strength of stimulus to rapidity of habit‐ formation. journal of comparative neurology and psychology, 18(5), 459-482. copyright: © 2020 memar, krishna, mcmeekin & tan. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v24i0.2179 http://creativecommons.org/licenses/by-nc/3.0/au/ http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems cooper, hayes & karanasios 2022, vol 26, research article building social resilience and inclusion in disasters 1 building social resilience and inclusion in disasters: a survey of vulnerable persons’ social media use vanessa cooper rmit university, melbourne, australia vanessa.cooper@rmit.edu.au peter hayes rmit university, melbourne, australia stan karanasios university of queensland, st. lucia, australia abstract social media (sm) is increasingly used to reach out to populations for preparedness and response to disasters. given the disproportionate impacts of disasters on vulnerable populations (e.g., older persons, persons socially/geographically isolated, persons living with disabilities, persons of low socio-economic means) in this study we focus specifically on the attitudes, needs and future plans of vulnerable persons towards accessing and sharing information via sm during extreme weather events. advancing understanding in this area is important as there is growing evidence that people who may be described as more vulnerable may have different communication needs and less access to disaster related information and technologies. we present the results of a survey of 215 vulnerable persons in victoria, australia. rather than consider vulnerable persons as a homogenous group, we examine how persons with different vulnerabilities perceive sm for accessing and sharing information in the context of disasters and report findings which challenge prevalent assumptions about vulnerable persons and sm. overall we find that vulnerable persons are not passive recipients of support during disasters but have self-awareness, a strong desire to receive information and the capacity to usefully contribute to the provision of reliable information via sm. with a view to improving outcomes for vulnerable persons in disasters we offer an agenda for future research. keywords: social media, vulnerable persons, disaster management, communication, community resilience. 1 introduction over the last 15 years social media (sm) has grown to become a significant part of many peoples’ lives and has had a profound impact on the way in which society functions and communicates. sm incorporates social networking, forums, blogs and micro-blogs, social book-marking, collaborative document creation and the sharing of audio, photographic and video files (alexander, 2014). sm represents a significant technological innovation and has transformed digital information sharing and networking (panagiotopoulos, barnett, bigdeli, & sams, 2016). organisations are increasingly relying on sm channels to inform, influence, and interact with the community. this is certainly the case when disaster events occur (gill, alam, & eustace, 2015). across the globe there are numerous examples where sm platforms have been reported to help keep people informed and safe before, during and after wildfires/bushfires (abedin & babar, 2018), floods (al-saggaf & simmons, 2015), hurricanes (kryvasheyeu et al., 2016), snowstorms (panagiotopoulos et al., 2016) and tsunamis and australasian journal of information systems cooper, hayes & karanasios 2022, vol 26, research article building social resilience and inclusion in disasters 2 earthquakes (peary, shaw, & takeuchi, 2012). yet while emergency response organisations (eros) and other actors increasingly understand the value of sm in disasters, and there is a growing body of literature in this area (e.g., al-saggaf & simmons, 2015; elbanna, bunker, levine, & sleigh, 2019; panagiotopoulos et al., 2016), to date there has been limited research undertaken to understand how sm is being used to connect to parts of the community which may be considered more vulnerable. advancing research in this area is important as there is growing evidence that people who may be described as more vulnerable may have different communication needs and less access to disaster related information and technologies (howard, agllias, bevis, & blakemore, 2017) and suffer disproportionately during and post large-scale disasters (ifrc, 2005, 2013). the term vulnerability originates from the latin word ‘vulnerare’ meaning “to wound” and generally indicates the degree of exposure to the possibility of disruption or harm. in investigating the complex concept of vulnerability, researchers have developed a plethora of models, metrics and tools to depict, measure and assess vulnerability (e.g., cutter, 1996; luna, 2014; zebardast, 2013). yet vulnerability remains an “elusive and slippery concept” (luna, 2018) which is defined and used differently across disciplines (alwang, siegel, & jorgensen, 2001). while acknowledging the complexity and range of approaches to defining and categorizing vulnerable persons (e.g. eakin & luers, 2006; wisner, 2004), and the difficulty in defining vulnerability in multi-disciplinary contexts such as disasters (enang et al., 2019), the united nations international strategy for disaster reduction defines vulnerability as, “the conditions determined by physical, social, economic and environmental factors or processes which increase the susceptibility of an individual, a community, assets or systems to the impacts of hazards” (unisdr, 2016, p. 41). we focus on older persons (aged 60+), those who are geographically and/or socially isolated (gsi) (e.g., live in a remote community and/or live alone in an urban area), people who suffer from physical limitations (pl) in terms of physical functioning, mobility, dexterity and those of low socio-economic (lse) means. this is because these people are frequently identified as amongst the most vulnerable people in the disaster literature. the elderly were overrepresented in those people who were impacted by the 2005 hurricane katrina in the usa (gibson & hayunga, 2006), 2011 earthquake and tsunami in japan (nakahara & ichikawa, 2013) and, based on figures to date, during the current global coronavirus pandemic (covid19) (onder, rezza, & brusaferro, 2020). examples where isolation has increased the impact of disasters include chicago’s 1995 heatwave where a disproportionate number of deaths occurred among people living alone (smoyer tomic, 2003). in australia, it has been identified that residing in a rural area substantially increases the risk of natural disasters and living with the ongoing threat of natural disasters can have adverse psychosocial, mental health and community impacts (morrissey & reser, 2007). people living with disabilities were more than twice as likely to have been injured during tropical cyclone pam in vanuatu (bakera et al., 2017) and using data from japan, persons living with disabilities are predicted to be four times more likely to die when a disaster strikes (ohchr, 2015). one’s socio-economic status can influence disaster awareness levels (teo, goonetilleke, ahankoob, deilami, & lawie, 2018), ability to absorb losses and resilience towards hazards and impacts through lack of access to insurance, social safety nets and entitlement programs (cutter, boruff, & shirley, 2003). disasters such as hurricane katrina in 2005 (masozera, bailey, & kerchner, 2007) and global pandemics such as covid-19 (buheji et al., 2020) illustrate how people of lse means can be more vulnerable to disasters. australasian journal of information systems cooper, hayes & karanasios 2022, vol 26, research article building social resilience and inclusion in disasters 3 to ensure sm use better supports vulnerable people during disasters, an important first step is to identify vulnerable persons’ attitudes, needs and future plans to use sm to access and share information in these critical contexts. several surveys on the use of sm in the disaster context have been undertaken (e.g., american red cross, 2012; canadian red cross, 2012; reuter, kaufhold, schmid, spielhofer, & hahne, 2019; reuter & spielhofer, 2017) but the extent these studies are generalisable to more vulnerable populations is unclear. our study addresses the following research question: “what are the attitudes, needs and future plans of vulnerable persons towards accessing and sharing information via sm during extreme weather events?” to answer this question, we report the results of an online survey administered to vulnerable persons in the australian state of victoria that explores their attitudes, needs and future plans to use sm to access and share information during extreme weather events. for the empirical component of our study, we focus on “extreme weather events” as this terminology was predicted to be more relevant to survey respondents than “emergencies” by interview and focus group participants (see section 3.1). extreme weather events are generally described by a range of attributes including the rate of occurrence (rarity), magnitude (intensity), temporal duration and timing, spatial structure and multivariate dependencies (stephenson, diaz, & murnane, 2008). in our study we are interested in extreme weather events that are consistent with the local hazard profile of victoria, australia — namely, floods, storms, fire and heatwaves. the remainder of the paper is structured as follows. in section 2 we review related research and in section 3 we describe our research method. we report our findings in section 4. in section 5 we discuss the findings against extant literature. in section 6 we present our contributions to research and practice, offer an agenda for future research and acknowledge the limitations of the study. in section 7 we conclude our paper. 2 related work in this section we examine the literature on the use of sm in disasters (section 2.1). the scant literature on sm use by vulnerable persons in disasters is also considered (section 2.2). 2.1 sm use in disasters sm is a relatively new phenomenon and consequently, there is limited research on sm use in disasters predating 2007 (alexander, 2014). nonetheless the use of sm in disasters is increasing and researchers have shown a growing interest in the area. in this section we focus on five key themes addressed in the literature. 2.1.1 how sm is used in disasters a body of research has focused on how sm is used in disasters (alexander, 2014; kim & hastak, 2018). based on a review of prior research and two case studies (a volcano eruption in iceland and a mass panic at a music festival in germany) four categories of sm use in crisis situations are outlined by reuter, marx, and pipek (2012): from organisations to the public addresses the integration of sm into classical crisis communication activities such as eros’ use of facebook to provide broadcast messages to members of the public. from the public to the public addresses the domain of self-help communities whereby sm can provide a platform for community groups that allow members of the public to ask questions, express their needs and fears, and offer assistance and support to others. from the public to organisations captures the australasian journal of information systems cooper, hayes & karanasios 2022, vol 26, research article building social resilience and inclusion in disasters 4 integration of citizen generated content and the various ways in which this can be analysed and used by organisations. from organisations to organisations addresses inter-organisational crisis management activities whereby sm can support a range of organisational actors in disasters to share information, communicate and network to improve inter-organisational awareness and informal processes. 2.1.2 community initiated sm groups in disasters literature has focused on community initiated groups and how sm group administrators disseminate information, the questions asked by group members and community dynamics and influences (bird, ling, & haynes, 2012; kulemeka, 2014). for example, bird et al. (2012) studied the emergence of community facebook groups during the 2010/2011 queensland and victorian floods. they identified through an online survey that group administrators sourced information from formal organisations (e.g., eros, local governments, bureau of meteorology, news media) and published information from citizens in near-real time. group members posted information and questions, requested and received help and advice, while travellers posted and received information on road closures and flooding. kulemeka (2014) undertook a content analysis of a community facebook group comprised of people affected by the 2014 fire at the hazelwood coal mine in victoria, australia and compared this to an internet forum on tianya following the 2008 earthquake in china (qu, wu, & wang, 2009). despite differences in culture, location and disaster events, members of both communities used sm to share or seek information, support each other, express emotion, try to make sense of events and organise action. differences were evident in the types of threads occurring, when threads were created and preferences users showed toward threads (kulemeka, 2014). 2.1.3 organisational integration of sm in disasters from an organisational perspective, research has investigated the way organisations have integrated sm into their internal operations. for instance, the red cross created the ready2help platform that matched individuals needing help with people who could offer it (schmidt, wolbers, ferguson, & boersma, 2017). further examples include crowdsourcing and mapping platforms (e.g., quakemap used in the 2015 nepalese earthquake) that guide relief efforts (wolbers, boersma, ferguson, mulder, & groenewegen, 2016). the complexity of the organisational landscape in disasters has been examined in light of the emergence of digital networks. this research suggests the convergence of the organisational boundaries and activities, as well as the convergence of physical and online spaces (hughes & tapia, 2015; schmidt et al., 2017). the need to carefully consider how sm is integrated with organisations’ traditional communication strategies and technologies has been highlighted (applebyarnold, brockdorff, fallou, & bossu, 2019; elbanna et al., 2019). as the interactions and relationships between organisations, individuals and technology move away from the bureaucratic, top-down logic, they become more complex and may result in behaviour that is difficult to predict (luna & pennock, 2018). for example, eros often need to contend with “flash volunteers” and “disaster tourists” and the tensions these phenomena cause to their traditional prevention, preparedness, response and recovery (pprr) protocols (elbanna et al., 2019). others have investigated the attitudes of emergency services personnel towards sm and their perceptions of organisations’ current and future sm use (reuter, kaufhold, spahr, spielhofer, & hahne, 2020). recognising the complexities that sm can bring, guidelines have been developed to prevent chaotic use of sm in disasters (kaufhold, gizikis, reuter, habdank, & grinko, 2019). australasian journal of information systems cooper, hayes & karanasios 2022, vol 26, research article building social resilience and inclusion in disasters 5 2.1.4 information-related issues for sm in disasters information-related issues are critical in sm and disasters as the provision of transparent information can facilitate community resilience (cohen, goldberg, lahad, & aharonsondaniel, 2017; cooper, fairbrother, elliott, walker, & ch'ng, 2020). there has been a push towards the use of sm in disasters to transmit fast and clear information and to collect information from at-risk populations. for example, following the japan tsunami, platforms such as twitter, mixi, facebook and google’s person finder served as a point where requests for assistance could be made, enabled the dissemination of information such as the location of local evacuation centres and provided a way for survivors to locate friends and family and let people know that were safe (appleby, 2013). nonetheless, disasters are characterised by several types of information challenges that sm does not necessarily resolve. for instance, there can be challenges surrounding the choice of the most appropriate channels for distributing and sharing information, the volume of information and the selection of the format and content of information (gill et al., 2015; hagar, 2013; pang, karanasios, & anwar, 2020; schmidt & galea, 2013). there are challenges surrounding perceptions of the reliability and veracity of information on sm whereby agencies and citizens are afraid that social networks will produce inaccurate information of dubious provenance (elbanna et al., 2019; goolsby, 2010; palen, vieweg, & anderson, 2011). other research, however, suggests that harmful and inaccurate rumours are not particularly enhanced by sm because with mass participation, false rumours are easily corrected by knowledgeable people (bird et al., 2012; hjorth & kim, 2011). the predominant use of sm by organisations is to broadcast generic information rather than provide targeted information (karanasios, cooper, hayes, & adrot, 2019). the lack of two-way communication has been problematic as those at risk in a disaster may lack specific advice about what actions their household should take to protect themselves and their property (poblet-balcell, cooper, & karanasios, 2018). 2.1.5 sentiment and network analysis via sm in disasters research in sm has focused on sentiment and network analysis. sm provides people with a unique platform to express their thoughts and feelings publicly during times of disasters (qiu, xu, wang, & gu, 2020). using data on twitter referring to the kashmir floods, kaur and kumar (2015) develop a model to assist authorities strategize during disaster based on public sentiments. kim and hastak (2018) leveraged data from facebook in the city of baton rouge after the 2016 louisiana flood to analyse the emergent networks after the flood. the researchers were able to explore connections and patterns based on aggregated interactions on facebook, highlight the social roles and key players in the network and analyse the content of posts (kim & hastak, 2018). with the volume of data being generated on tools such as facebook and twitter it is likely that automated approaches to analyse data will grow. 2.2 sm use by vulnerable persons in disasters post-disaster reports across the globe have highlighted the disproportionate impact and number of fatalities amongst groups defined as ‘vulnerable’ (deacon, 2018; nakahara & ichikawa, 2013). typically, academic research that addresses disaster and sm treats individuals and communities as homogenous and there is relatively little research that has examined the use of sm by vulnerable persons in disasters specifically. in this section we highlight some of the exceptions in the literature. the use of sm by people with disabilities during disasters and public emergencies was examined by morris, mueller, and jones (2014). the us-based study involved a survey of australasian journal of information systems cooper, hayes & karanasios 2022, vol 26, research article building social resilience and inclusion in disasters 6 people with hearing, vision, cognitive, mobility, dexterity, and speech limitations and found that these people are at greater risk during emergencies because they may not receive or understand emergency alert information or may not be able to take the required actions (morris et al., 2014). the study highlights that the use of sm for accessing and sharing information during emergencies is relatively low among the respondents and that differences in sm usage across people with different disability types are minimal, the exception being people who are deaf or who have speech limitations who use sm more than people with other disabilities. it is inferred from the findings that disaster communication strategies should be tailored to consider the diverse needs of people living with a variety of disabilities. other studies point to the lack of tailored information on sm for vulnerable persons in disasters. in an analysis of information posted to twitter and facebook about extreme weather events in victoria, australia, poblet-balcell et al. (2018) identified there were few examples where the posts targeted vulnerable groups such as older persons, those socially/geographically isolated, people with disabilities and refugee/recent migrant communities— instead, sm was used to raise awareness of the need to check-in with others. some studies that have focused on information behaviours of vulnerable persons during disasters, and while sm was not the primary focus, have offered some insight into vulnerable persons’ sm usage. through a survey of residents of a local council area in southeast queensland that had experienced multiple natural hazards (predominately floods and bushfires) in recent years, teo et al. (2018) studied the disaster information seeking needs and preferences of people from lse backgrounds and how this affected their level of disaster awareness. the study revealed that television (60%) was the most trusted information source for both people from lse and non-lse backgrounds whereas sm did not feature highly for respondents from either background—with facebook, twitter and other sm considered a trusted source for disaster information by 18%, 2% and 8% of respondents respectively. this suggests that people from both lse and non-lse backgrounds still prefer traditional media to acquire disaster related news. howard et al. (2017) report a series of focus groups undertaken in regional australia on the communication channels used by vulnerable people to prepare for and respond to natural disasters. in focusing on older people, people with disabilities, culturally and linguistically diverse populations, families with young children and people from low-income households they identify that use of specific communication methods was attributable to membership of specific population groups. in terms of preferences for sm, it was found families with young children relied more on mobile phone apps, sm and websites for information on natural disasters whereas older people listen to the radio or expect phone calls on their landline (howard et al., 2017). other studies examine the use of sm in disasters and during the analysis, by considering the demographic details of participants, offer insight into how people who might potentially be considered vulnerable, use sm. for example, feldman et al. (2016) analyse a survey of residents in newport beach california to identify the ways in which people prefer to receive information about flood risks. by considering a range of socio-economic characteristics they identify that older persons prefer to use traditional media (tv, radio, newspaper, printed material, websites) when compared to sm sources (e.g., facebook and twitter) and expressed little desire to use sm in the future. studies such as this which reveal the communication preferences of different groups are important as the effectiveness of risk communication has been found to be linked to factors that affect an individual’s capacity to access relevant information. australasian journal of information systems cooper, hayes & karanasios 2022, vol 26, research article building social resilience and inclusion in disasters 7 we now outline the methods we adopted to answer our research question. 3 methods we first focus on the survey design (section 3.1) and describe the characteristics of participants (section 3.2). this is followed by a description of the analysis process (section 3.3). 3.1 survey design the aim of the survey was to identify the attitudes, needs and future plans of vulnerable persons to use sm to access and share information in the context of extreme weather events. the design of the survey was informed by extant research on sm use by vulnerable persons in disasters (section 2) and a series of semi-structured interviews and two focus groups, undertaken as part of a wider study, with persons aged 60+, living with pl, of lse means as well as gsi persons. in total 47 perspectives of vulnerable persons from rural and metropolitan areas of victoria, australia were included in the individual interviews and focus groups. participants were asked about how they obtained information during disasters, the digital tools they used (including sm), the types of information sought and shared, and their attitudes towards various information providers and communication channels. the main themes identified from these interviews and focus groups related to: ict and non-ict information sources; digital technology and sm use patterns (in general and in relation to disasters); awareness of emergency services’ use of sm; trust and attitudes towards sm; and the important role of informal community networks and relationships. additional details pertaining to the interviews and focus groups, including participant recruitment, the interview protocol, data analysis procedures and findings can be found in karanasios, cooper, adrot & mercieca (2020). the findings of the interviews and focus groups confirmed the existing instrument developed by reuter and spielhofer (2017) as relevant for our study. we adapted the instrument to include additional questions relating to: qualifying and demographic information; the various technology devices used by vulnerable persons; how much time vulnerable people spend using sm (typically and during extreme weather events); information needs; and perceptions on the helpfulness of information on sm provided by different types of organisations. as noted, the importance of informal networks and relationships as a source of information and support emerged as a key theme from the interviews and focus groups. consequently, we included a set of questions to identify whether and how vulnerable persons use information from sm provided to them indirectly (e.g., via friends and family). adding these questions increased the length of our survey, so several questions from the original instrument deemed less relevant to our research question were removed to minimise participant fatigue (i.e., questions on the downloading of smartphone apps and questions relating to working or volunteering in emergency services). we also changed the wording of the questions. for example, we revised questions to focus on “extreme weather events” rather than “emergencies”. the resulting instrument employed quantitative and qualitative methods to generate descriptive statistical results and elicit reasons for participants’ answers. the instrument is provided in appendix one. the survey was administered by qualtrics1 who sent a link to the online survey to people residing in victoria who report to have used sm to search or share information during an 1 www.qualtrics.com last accessed 12th july 2021. australasian journal of information systems cooper, hayes & karanasios 2022, vol 26, research article building social resilience and inclusion in disasters 8 extreme weather event, and who also identify as belonging to different vulnerable groups (i.e., aged 60+, gsi, living with pl and/or of lse means). 3.2 characteristics of survey participants we collected 215 survey responses. the majority of participants (77.7%) reported to belong to one vulnerability group, with 15.8% reporting to belong to two vulnerable groups, 4.7% reporting to belong to three vulnerable groups and 1.9% belonging to four vulnerable groups. respondents were from different national backgrounds, namely, australian (75.4%), asian (9.3%), european (8.4%), american (2.3%) and african (0.9%), however, most were from an english-speaking background (75.0%). respondents included women (54.0%), men (45.6%) and other (0.5%) across diverse age groups but with around half (52.6%) of the respondents being 22-39 years old (figure 1). figure 1. age profile of participants the highest level of formal education completed by most participants was high school (33.5%) or a bachelor’s degree (27.4%). respondents live in a variety of household types with the most common type comprising families with children (40.5%) and 33.5% of respondents having children under the age of 18. the employment status of participants included full-time (31.6%), part-time (15.9%), casual (7.4%), homemaker (7.4%), retired (10.2%), pensioner (5.2%), selfemployed (5.1%) and unemployed (14%). 3.3 survey analysis the survey was analysed using quantitative and qualitative techniques. for the initial quantitative analysis, the survey data was extracted and initially analysed using excel. basic frequencies for each question and cross-tabulations were undertaken to identify any significant differences across types of respondents. the response format for the quantitative items in the survey generally comprised a four or five-point likert scale such as: never, seldom, sometimes, and often; or strongly disagree, disagree, neither agree/disagree, agree, and strongly agree. this type of data is ordinal and thus nonparametric methods are better suited (kraska-miller, 2013). to identify any differences based on vulnerability type (i.e., 60+, lse, pl, gsi), we used spss 25 to complete a series of kruskal-wallis h one-way analysis of variance (anova) tests (chan & walmsley, 1997). the kruskal–wallis test identifies if there are differences among three or more groups but does not identify which groups are different. to ascertain this, we used a series of post hoc paired mann-whitney u tests (also known as the wilcoxon rank sum test), to determine which groups were different from others (mann & whitney, 1947; wilcoxon, 1945). for this analysis we included the 167 responses where respondents reported only one vulnerability type to ensure the condition of independence was met. summary tables for the quantitative results are in appendix two. 6% 21.9% 30.7% 15.8% 7.4% 11.2% 7% 0% 10% 20% 30% 40% 18-21 22-29 30-39 40-49 50-59 60-69 70 or older australasian journal of information systems cooper, hayes & karanasios 2022, vol 26, research article building social resilience and inclusion in disasters 9 for the qualitative analysis, responses to open-ended questions were analysed in excel based on an inductive approach. two researchers analysed the responses to the open-ended questions and assigned participant statements to open code categories (thomas, 2006). through this process we obtained an overview of the relevant themes before focusing on the themes which help address our research question. as participant responses to the open-ended questions were typically brief, in the analysis presented here we summarise the high-level themes which emerged from each open-ended question and supplement this with participant quotes. 4 empirical results 4.1 current use of technology and views towards sm respondents were asked about their use of technology devices (figure 2). the results show that smartphones are the most frequently used device with 78.1% of respondents reporting they used smartphones often. respondents also reported that they often used laptops (65.6%) and desktop computers (51.6%). tablets were used often by 30.7% of respondents, with 29.8% reporting they used tablets sometimes. regular (non-internet enabled) mobile phones were not used by 58.1% of respondents. other technologies used included gaming consoles (5) and smart tvs (8). of the five types of technology, only smartphones were used significantly differently by respondents with differing vulnerabilities (χ2 (3, n = 167) = 11.002, p < .012). specifically, those aged 60+ were significantly less likely to use smartphones than lse or gsi respondents. figure 2. common use of different technology devices figure 3 reports respondents’ use of different sm applications. facebook is by far the most frequently used type of sm, with 70.2% reporting they used facebook often. many also used youtube often (48.8%), or at least sometimes (31.2%). some sm applications were not popular amongst respondents. many never used twitter (43.3%), blogs (41.4%), chat tools (40.9%) or instagram (39.1%). several participants identified they used ‘other’ sm applications such as reddit (5) and snapchat (7). four of the six sm applications (i.e., twitter, instagram, youtube and chat tools) were used significantly differently based on the type of vulnerability reported. respondents aged 60+ were significantly less likely to use: (i) twitter than gsi respondents, desktop computer laptop tablet regular mobile phone smartphone never 19.5 8.8 26.5 58.1 6 seldom 12.1 7.4 13 12.1 3.7 sometimes 16.7 18.1 29.8 10.7 12.1 often 51.6 65.6 30.7 19.1 78.1 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% often sometimes seldom never australasian journal of information systems cooper, hayes & karanasios 2022, vol 26, research article building social resilience and inclusion in disasters 10 (ii) instagram than lse and gsi respondents, (iii) youtube than lse and gsi respondents, and (iv) chat tools than gsi respondents. figure 3. current use of different sm applications most respondents either strongly agreed (42.3%) or agreed (34.4%) that they use sm very often in their private lives, with 61.4% regularly posting messages on sm (figure 4). most held the view that their friends also use sm to keep in touch (83.3%). the majority of respondents (86.9%) thought that it is important for emergency services to use sm with 61.4% reporting that they would not trust messages on sm, apart from those posted by official sources. around one-third (31.6%) strongly agreed or agreed that emergency services should not trust information on sm. in contrast, almost another third (27.0%) strongly disagreed or disagreed that emergency services should not trust information on sm. respondents from differing vulnerability groups responded significantly differently to four of the six items on sm use and trust towards sm. the four items concerned were: in my private life, i use sm very often; i regularly post messages on sm such as twitter, facebook or instagram; most of my friends use sm to keep in touch; emergency services should not trust information on sm. respondents aged 60+ were significantly less likely to: (i) use sm than lse, pl, and gsi respondents, (ii) post messages on sm than lse and gsi respondents, and (iii) perceive their friends use sm to stay in touch than lse and gsi respondents. the fourth item asked respondents whether emergency services should not trust sm information. this reverse scored item was recoded so that a higher participant score indicates greater trust. gsi respondents were significantly more likely to believe emergency services should trust information on sm information than respondents aged 60+ years and from lse backgrounds. awareness about existing safety services provided on twitter and facebook varied. only 29.8% of respondents reported to be very aware of facebook safety checks and 20% being very aware of twitter alerts. knowledge of these sm tools were significantly different by type of vulnerability. gsi and lse respondents were more aware of these sm tools than those aged 60+. 4.2 sm as an information source during disasters when asked which communication channels they have used to get information about an extreme weather event, sm was the second most common channel used (73%) behind tv facebook twitter instagram blogs youtube chat tools (e.g. whatsapp) never 4.7 43.3 39.1 41.4 12.1 40.9 seldom 8.4 15.3 13 15.3 7.9 13 sometimes 16.7 20 18.1 29.8 31.2 25.1 often 70.2 21.4 29.8 13.5 48.8 20.9 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% often sometimes seldom never australasian journal of information systems cooper, hayes & karanasios 2022, vol 26, research article building social resilience and inclusion in disasters 11 (87%). online news (72%) was a common source of information, followed by local radio (61%), online sites (59%), text messages (56%) and mobile apps (45%). figure 4. use and trust of sm we were interested in how much time respondents actively spend on sm per day during an extreme weather event compared to the time they actively spend on sm on a typical day. there were 118 (54.9%) respondents who reported that they actively use sm more during extreme weather events when compared to a typical day. there were 82 (38.1%) who reported that they are less active on sm during extreme weather events and 15 (7.0%) who reported to be active on sm for the same amount of time under both conditions. respondents reported using sm to find out information on a variety of topics during an extreme weather event (figure 5). most reported they had looked for information about weather conditions or warnings (85%), damage caused by an event (66%) or road or traffic conditions (65%). many reported they had also used sm to find out the location or status of friends or family (55%). further, almost half the respondents used sm to find information about how others were coping with the extreme weather event (49%), what to do to keep yourself safe (49%) and to locate eyewitness videos or photographs (47%). in my private life, i use social media very often i regularly post messages on social media such as twitter, facebook or instagram most of my friends use social media to keep in touch it is important for emergency services to use social media to keep in touch with the public during emergencies i would not trust a message posted on social media during an emergency unless it came from an official source such as the police or fire service emergency services should not trust information on social media strongly disagree 1.9 4.7 1.4 0 2.3 3.7 disagree 6.5 17.7 7 1.4 12.1 23.3 neither agree or disagree 14.9 16.3 8.4 11.6 24.2 41.4 agree 34.4 38.1 34.9 31.6 32.6 22.3 strongly agree 42.3 23.3 48.4 55.3 28.8 9.3 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% strongly agree agree neither agree or disagree disagree strongly disagree australasian journal of information systems cooper, hayes & karanasios 2022, vol 26, research article building social resilience and inclusion in disasters 12 figure 5. types of information about extreme weather events sought using sm we asked respondents how helpful they found the information they sought about an extreme weather event from a range of organisations using sm (figure 6). it was anticipated this might not only illuminate the information preferences of vulnerable persons but reveal those organisations that might usefully review the information they provide to increase its relevance to different vulnerable groups. results showed that the majority of respondents considered the information sought from eros (70.7%) and government agencies (64.1%) as either extremely helpful or very helpful. a relatively high proportion considered the information sought from local online groups (49.3%), ngos (44.1%), local community groups (38.6%) and local government (37.6%) as extremely helpful or very helpful. some reported that information from local community groups (18.1%) and ngos (11.6%) was not applicable to them. for three of the six types of organisations (i.e., eros, local community groups and local online groups) the level of perceived helpfulness was significantly different by type of vulnerability. gsi respondents rated the perceived helpfulness of information they sought via sm from different eros more highly than lse and pl respondents. a similar pattern was found for local community groups and local online groups except that gsi respondents only had more positive perceptions of the helpfulness of the information they sought via sm from these organisation than pl respondents. most respondents (88.4%) indicated that it was very likely or quite likely that they would use sm in the future to look for information as a result of an extreme weather event. in contrast, only 1.4% of respondents thought it not at all likely that they would use sm in the future to look for information about an extreme weather event, with 2.3% reporting it not very likely and 7.9% reporting to be neutral on this question. respondents aged 60+ were significantly less likely to believe they would seek information from sm in the future than respondents from lse backgrounds. there were no other significant differences between respondents’ perceptions of whether they were likely to use sm to find out information about an extreme weather event in the future based on the type of vulnerability reported. 85% 66% 65% 55% 49% 49% 47% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% weather conditions or warnings damage caused by the event road or traffic conditions the location or status of friends/family information about how others are coping with the extreme weather event what to do to keep yourself safe eyewitness photographs or videos australasian journal of information systems cooper, hayes & karanasios 2022, vol 26, research article building social resilience and inclusion in disasters 13 figure 6. helpfulness of information on extreme weather events sought via sm from different organisations when asked what would make them even more likely to use sm in the future to look for information as a result of an extreme weather event, a number of themes emerged. issues related to characteristics of information were highlighted, including the need for ‘official information’ that enabled ‘quick and live updates’ from more ‘reliable sources’, that is, information that is ‘easier to access’ and ‘more accurate’. several respondents explained that they would use sm to cross-check information if they had doubts about the usefulness of other information sources, or, if there were no other information sources available. characteristics of the event were also important. the ‘type’ of extreme weather event was identified by some respondents as likely to influence whether they would use sm in the future (e.g., size or type of natural hazard). the ‘proximity to event or area’ was important. for example, one participant stated they would be more likely to use sm, “if i had friends or family who lived in the affected area or if i lived in the affected area and wanted to let loved ones know i was safe.” safety concerns were also mentioned because “if i’m going out or travelling somewhere far. i want to be safe. i want to avoid dangerous area (sic)”. issues relating to friends and family were identified as particularly important for using sm in the future. on the one-hand respondents remarked that “if [the] event is likely to impact myself or extended family members” they could use sm to “… alert my friends and family”. on the other hand, respondents revealed that the extent to which friends or family use sm during extreme weather events will influence their own future sm use. for some respondents, certain conditions needed to be met in order for them to use sm more often as a result of an extreme weather event. for instance, “if the extreme weather event hadn’t affected wi-fi, data or phone signal”. not all respondents were able to provide specific insights into what would make them emergency response organisations (e.g. police, fire brigade) government agencies (e.g. department of human services, bureau of meteorology) nongovernment organisations (e.g. red cross, salvation army) local government (e.g. local council) local community groups (e.g. grey nomads, surf clubs) local online groups (e.g. twitter feed or facebook page of local community group) not applicable 2.3 3.7 11.6 7.9 18.1 6 not at all helpful 1.4 2.8 4.2 7.9 7 3.7 slightly helpful 3.7 5.1 12.1 15.8 14.4 12.1 somewhat helpful 21.9 24.2 27.9 30.7 21.9 28.8 very helpful 30.7 36.7 26 22.3 22.3 33.5 extremely helpful 40 27.4 18.1 15.3 16.3 15.8 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% extremely helpful very helpful somewhat helpful slightly helpful not at all helpful not applicable australasian journal of information systems cooper, hayes & karanasios 2022, vol 26, research article building social resilience and inclusion in disasters 14 more likely to use sm for gathering information in an extreme weather event providing responses such as, “nothing”, “i don’t know”, “not sure”, “n/a” as well as “i can't think of anything”. 4.3 use of information received indirectly via sm almost two-thirds of respondents (65.6%) had received information about extreme weather events via sm indirectly (e.g., had a conversation with a friend about a post they’d seen on sm about a flood). an additional 22.3% were not sure if they had received information about extreme weather events via sm indirectly. the most common type of information received indirectly was about weather conditions or warnings (75%), followed by information on damage caused by the event (55%) and road or traffic conditions (52%) (figure 7). over twothirds (69.7%) of all respondents reported that they would either be very likely or quite likely to act on information from sm provided to them indirectly by friends or relatives as a result of an extreme weather event, with only 2.8% of respondents reporting that they were not at all likely to act based on such information. there was no significant difference in the likelihood of people acting on information sourced indirectly from family or friends’ sm in the future based on the different types of vulnerability reported. figure 7. types of information about extreme weather events received indirectly via sm when asked what would make respondents even more likely to act upon information about an extreme weather event from sm indirectly through friends or relatives, a number of themes emerged. first, reliability was important, in terms of the perceived trustworthiness of the friend or family member providing them the information, as well as the perceived reliability of the original source that the friend or relative had acquired the information. some respondents emphasised the importance of being able to verify the information provided by friends or relatives by triangulating this with other information sources. for some, the nature (e.g., severity, extent of threat posed) and proximity of the event (i.e., to themselves, to friends/ relatives) would influence the likelihood of them acting on information received from sm indirectly. timeliness of the information was also a factor, with ‘constant’, ‘faster’, ‘current’, ‘regular’ updates being important. a number of respondents noted that “nothing”, “i don’t know”, “not sure”, as well as “i can't think of anything” would make them more likely to act 75% 55% 52% 40% 36% 29% 13% 0% 10% 20% 30% 40% 50% 60% 70% 80% weather conditions or warnings damage caused by the event road or traffic conditions the location or status of friends/family information about how others are coping with the extreme weather event eyewitness photographs or videos what to do to keep yourself safe australasian journal of information systems cooper, hayes & karanasios 2022, vol 26, research article building social resilience and inclusion in disasters 15 on information received from sm indirectly. others felt they would need to directly hear or see the news as they lacked confidence in information received indirectly. 4.4 sharing information using sm during emergencies almost two-thirds (62.3%) of respondents have used sm to share information as a result of an extreme weather event. of those that have shared information, the most common type of information shared was about weather conditions or warnings (75%), followed by the actions they were taking to stay safe (57%), road or traffic conditions (49%) and reassurance that they are safe (46%). sharing eyewitness videos (16%) and advice about actions others should take to stay safe (23%) were the least common types of information shared (figure 8). almost threequarters of respondents (72.5%) reported that they would be either quite likely or very likely to use sm in the future to share information regarding an extreme weather event. only 5.1% reported that they were not at all likely to do so. there was no significant difference in the likelihood of people using sm in the future to share information based on the different types of vulnerability they reported. figure 8. types of information shared about extreme weather events using sm we also asked respondents about what would make them even more likely to use sm to share information about an extreme weather event in the future. the most common theme that emerged related to inter-personal relationships such that if respondents felt family or friends were at risk, they would be more likely to share information on sm. the extent to which the respondent thought their posts would help others, and the need to be confident that the information they shared was reliable, also emerged as important. having photographic evidence to post and the ability to give ‘live updates’ were reported as influencing factors. the need for easy to access to information from other sources (e.g., emergency services’ pages) was considered to influence the likelihood that respondents would share information on sm. further, proximity of the event, to either themselves or to friends and family, would increase the likelihood that respondents would share information on sm in the future. the severity of australasian journal of information systems cooper, hayes & karanasios 2022, vol 26, research article building social resilience and inclusion in disasters 16 the event was also a factor. for some respondents, specific technical conditions would need to be met, such as “social media being more likely to show my posts within the main news feed” or if there were no other communication medium available (e.g., phone service). a number of respondents noted that “nothing”, “i don’t know”, “not sure”, “n/a” in response to what would make them more likely to share information on sm regarding an extreme weather event. others noted that “[i] used [sm] a lot when we had our last flood event, and i don't think i could have used it more” and “if i used sm more generally” they would be more likely to post information as a result of an extreme weather event. 4.5 preference for sm as an information source compared to alternative information sources we asked respondents about why some people prefer to look for information on sm over more traditional media. many respondents think that people prefer to look for information via sm sites because it is available more quickly (83.3%) and is more accessible (81.4%) (figure 9). yet while 40.0% of respondents either agreed or strongly agreed that people prefer to look for information via sm because it may be more accurate, 41.9% neither agreed nor disagreed, 15.3% disagreed, and 2.8% strongly disagreed that this was the case. similar proportions of respondents perceived that people prefer to look for information via sm because it is more reliable (38.2%), with 40.9% of respondents neither agreeing nor disagreeing, 17.7% disagreeing and 3.3% strongly disagreeing. almost half the respondents (49.3%) thought people prefer to look for information via sm because it is richer than information available via more traditional sources, with 11.2% either disagreeing or strongly disagreeing, and 39.5% neither agreeing nor disagreeing that this was the case. figure 9. perceived reasons people prefer to look for information via sm than other sources three of the five criteria for why some people prefer to look for information on sm (i.e., information: available faster, more accessible, and more reliable) were significantly different by the type of vulnerability reported. specifically, lse and gsi respondents are significantly faster available more accurate more accessible more reliable richer strongly disagree 0.5 2.8 0.5 3.3 2.8 disagree 3.3 15.3 4.2 17.7 8.4 neither agree or disagree 13.0 41.9 14.0 40.9 39.5 agree 37.7 26.5 40.9 23.3 32.6 strongly agree 45.6 13.5 40.5 14.9 16.7 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% strongly agree agree neither agree or disagree disagree strongly disagree australasian journal of information systems cooper, hayes & karanasios 2022, vol 26, research article building social resilience and inclusion in disasters 17 more likely than those aged 60+ to perceive that people prefer to use sm to look for information because the information is available faster. lse respondents are also significantly more likely than pl respondents to perceive that people prefer to use sm to look for information because the information is available faster. in terms of accessibility, lse and gsi respondents are significantly more likely than those aged 60+ and those identifying as living with pl to perceive people prefer to look for information on sm than other information sources because it is more accessible. with respect to people preferring to look for information via sm because it is more reliable than other information sources, gsi respondents rated this aspect significantly higher than those aged 60+. 4.6 expectations towards emergency services the survey included a series of questions exploring how respondents perceived eros would or should react to them requesting help or information on their sm site. while 75.4% agreed that emergency services should regularly monitor their sm sites to be able to respond promptly to such a request, 60.5% of respondents believed emergency services would not know about such a request. further, 52.5% of respondents would expect a response within an hour, yet 61.8% of respondents thought that emergency services are too busy during emergency events to monitor sm (figure 10). figure 10. expectations of emergency services’ use of sm three of the four criteria capturing participant expectations of emergency services’ use of sm (i.e., eros should regularly monitor sm, expect a response on sm within an hour, and eros are too busy to monitor sm) were significantly different by the type of vulnerability reported. gsi and lse respondents had higher expectations of eros regularly monitoring sm than those aged 60+. in terms of expecting a response within one hour there were no significant pairwise differences between the vulnerability types. for the third criteria regarding whether emergency services should regularly monitor sm emergency services will not know about request expect a response within an hour emergency services are too busy to monitor sm during an emergency strongly disagree 1.4 1.9 4.7 3.3 disagree 6.0 9.3 14.0 10.7 neither agree or disagree 17.2 28.4 28.8 24.2 agree 33.5 41.9 30.2 35.8 strongly agree 41.9 18.6 22.3 26.0 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% strongly agree agree neither agree or disagree disagree strongly disagree australasian journal of information systems cooper, hayes & karanasios 2022, vol 26, research article building social resilience and inclusion in disasters 18 eros were too busy to monitor sm, the gsi group agreed that this was the case more so than the lse respondents. over two-thirds of respondents (67.5%) agreed that the information on sm from emergency services organisations meets their needs (figure 11). many respondents (61.0%) believed their information needs during extreme weather events are very specific, with almost half (49.7%) preferring to use community groups on sm to obtain information because it is tailored towards their needs. notably, just under half of the respondents (47.4%) reported that they obtain information specific to their situation offline rather than via sm during extreme weather events. figure 11. participant information needs during extreme weather events for three of the four information needs criteria (i.e., ero sm information meets my needs, my information needs are very specific, and i obtain most my specific information offline) were significantly different by type of vulnerability. gsi respondents were significantly more likely to believe that information provided by emergency services via sm meets their needs than those aged 60+ or respondents with pl. this same pattern was seen for the second question regarding the specificity of information needs although gsi respondents also rated their information needs as significantly more specific than lse respondents. for the third question on preferences for specific information sourced offline, there were no significant pairwise differences. respondents were given the opportunity to provide additional comments regarding the use of sm before, during and after an extreme weather event. a number of themes emerged. several respondents re-emphasised the need for confidence in the information and its source. some highlighted what they perceived as limitations with sm that could be improved. for example, several respondents highlighted that in extreme weather events phone lines or information on social media from emergency services organisations meets my information needs my information needs during extreme weather events are very specific i prefer to use community groups on social media to obtain information about extreme weather events because the information is tailored to my needs i get most information related to my specific situation offline rather than through social media during extreme weather events strongly disagree 1.4 1.9 3.3 2.8 disagree 6.0 6.5 13.5 18.6 neither agree or disagree 25.1 30.7 33.5 31.2 agree 44.2 40.5 30.2 30.2 strongly agree 23.3 20.5 19.5 17.2 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% strongly agree agree neither agree or disagree disagree strongly disagree australasian journal of information systems cooper, hayes & karanasios 2022, vol 26, research article building social resilience and inclusion in disasters 19 electricity may be down and thus sm may not be available to everyone, while others considered sm inferior to other information sources such as radio, was burdened by advertisements and suffered from having “fake information”. others focused on what they considered benefits of sm, for example, “it’s always informative, and there will always be groups started to help those affected. social media is amazing in pulling people together in a crisis”. several suggested features such as alert systems being built-in to sm and the need for more tailored information. it was noted by one participant that sm applications like twitter were seen to be more professional and reliable than those like instagram, yet another emphasised their preference for sm applications that used images. a number offered suggestions around what emergency services organisations could do to improve the usefulness of sm. for example, “emergency services should have designated people working on social media” and “it is helpful for emergency services to continue to update their social media during an extreme weather event but at the moment they do not share enough information”. 5 discussion researchers and practitioners have called for research to focus on the specific information needs of vulnerable persons as well as how they use sm (ifrc, 2005, 2013; pang et al., 2020). in response to this call, we undertook a survey of vulnerable persons in the australian state of victoria and focused on understanding their attitudes, needs and future plans to use sm to access and share information during extreme weather events. 5.1 challenging prevalent assumptions about vulnerable persons and sm in disasters extant literature has traditionally focused on the ways in which vulnerable persons are more at-risk during disasters, implying a state of weakness, rather than adopting strength-based approaches which emphasise equality, autonomy and empowerment (ogie & pradhan, 2019). the reasons behind this tradition are understandable given that disasters present life or death situations and there is an urgent need to reduce their impact on those who may be less able to fend for themselves. however, recent shifts towards “shared responsibility” in emergency management, whereby the obligations for planning and mitigating against hazards are distributed among different actors (mclennan & eburn, 2015) to reduce unrealistic expectations and unsustainable dependences on emergency services personnel (lukasiewicz, dovers, & eburn, 2017), prompts questions around existing assumptions about vulnerability and vulnerable persons. in the early stages of our study the importance of informal networks and relationships as a source of information and support for vulnerable persons emerged as important (section 3.1). we therefore investigated information-related issues surrounding trust, accuracy and usefulness when people receive information from sm indirectly. as highlighted in section 4.3 almost two-thirds of vulnerable persons (65.6%) reported they had received information about extreme weather events indirectly from sm and over two thirds (69.7%) reported they’d be either very likely or quite likely to act on information provided to them indirectly by friends or family. the relatively high proportion of vulnerable persons receiving and acting on information received from sm indirectly is not unexpected given that posts on sm by emergency organisations tend to target the general public as carers—effectively suggesting they ‘check-in’ with vulnerable persons (poblet-balcell et al., 2018). poblet-balcell et al. (2018) australasian journal of information systems cooper, hayes & karanasios 2022, vol 26, research article building social resilience and inclusion in disasters 20 note that that one reason for this may be because emergency organisations are assuming that vulnerable persons will be less likely or able to use sm to access such information. while this assumption may be accurate for many vulnerable persons, our study indicates that this is certainly not always the case. vulnerable persons in our study exhibit a strong desire to use sm during disasters to receive and share information—and in some cases were even more likely to view sm favourably than samples of the general population captured in other studies. this may be because vulnerable people recognise that their level of risk in disasters is higher and consequently, they adopt sm to strengthen their personal networks and improve their access to information. this is consistent with risk homeostasis theory (wilde, 1998) which proposes that if people subjectively perceive that their level of risk is higher than acceptable, they will compensate for this and modify their behaviour to exercise greater caution. in comparing the results of our study (n = 215) to those of reuter and spielhofer (2017) (n = 839), effect sizes for the difference between two independent proportions (with 95% confidence interval (ci) upper and lower limits shown in square brackets) (cumming, 2012), reveal that vulnerable persons were significantly more likely to perceive that they: i) regularly use sm in their private lives (0.137 [0.068, 0.198] p < .001); ii) use sm in the context of extreme weather events (0.311 [0.239, 0.375] p < .001); and iii) will use sm in the future to search (0.303 [0.244, 0.353] p < .001) and share (0.245 [0.174, 0.31] p < .001) information on extreme weather events. perceptions of sm compared to traditional media in terms of speed (0.072[0.01, 0.125] p < .024), accessibility (0.274[0.208, 0.331] p < .001), richness (0.524[0.459, 0.579] p < .001), reliability (0.161[0.092, 0.233] p < .001) and accuracy (0.27[0.202, 0.34] p < .001) were more positive for vulnerable persons. our study’s vulnerable participants were also considerably more aware of twitter alerts (20% vs. 6%; 0.14[0.089, 0.201] p < .001) and facebook safety checks (29.8% vs. 3%; 0.268[0.209, 0.333] p < .001) than the sample of the general population captured by reuter and spielhofer (2017). thus, sm offers an increasingly valuable way for governments, eros, ngos and community organisations to reach out directly to vulnerable persons and for vulnerable persons to reachout directly to organisations and their communities in ways which complement the wider information ecology of vulnerable persons. in light of the positive view that vulnerable persons in this study held towards sm in the disaster context, further research into areas where their perceptions do differ from the general population is worthwhile. such insights could be used to inform tailored sm solutions that maximise outcomes for vulnerable persons. for example, if we compare the expectations vulnerable persons hold of eros in our study with the sample of the general population captured by reuter and spielhofer (2017), vulnerable persons were found to have a significantly higher expectation that emergency services should respond to a post on sm within an hour (52.5% compared to 41%; 0.116[0.041, 0.189] p < .002). this suggests that emergency services might usefully engage with vulnerable persons to better address and manage their expectations. as another example, significantly more respondents in our study (61.4%) reported that they would not trust messages on sm apart from those posted by official sources (compared to only 38% of the general population) (0.235[0.161, 0.305] p < .001). while some studies have indicated limited issues with the propagation of rumours via sm during disasters (hjorth & kim, 2011) others have reported concerns from citizens and organisations in relation to inaccurate information (elbanna et al., 2019; goolsby, 2010). future research might investigate how to improve the accuracy and reliability of information to increase australasian journal of information systems cooper, hayes & karanasios 2022, vol 26, research article building social resilience and inclusion in disasters 21 vulnerable persons’ trust in sm during disasters given they seem to have greater concerns surrounding these issues. in addition to considering how the values, needs and future plans of vulnerable persons to use sm in disasters when compared to the general population, it is valuable to consider further, how people from different vulnerable groups compare with each other to support more nuanced approaches. we explore these differences next. 5.2 comparison of perceptions of sm use in disasters between vulnerable groups in addressing the research question our findings suggest that vulnerable persons should not be treated as a homogeneous group but rather, a nuanced approach to engage vulnerable persons using sm is required. amongst the four types of vulnerable persons examined in our study, those identifying as gsi generally held more positive attitudes towards using sm to access and share information during disasters and were more likely to view their information needs as specific. the views of older persons, on the other hand, were the most divergent. most of the significant differences between older persons were with those who are of lse means and/or those who were gsi, whereby older persons typically reported lower levels of awareness and sm use than these two groups. this is not only consistent with research that has shown that older persons are less likely to use ict and sm than the general population (barnard, bradley, hodgson, & lloyd, 2013) but further highlights that this is also the case when older persons are compared to other vulnerable groups. notably, older persons in our study were less likely to believe the information on sm provided by emergency services meets their needs when compared to gsi respondents. despite the significant differences identified between older persons and other vulnerable persons in a number of areas, they were also less likely to perceive their information needs during an extreme weather event are very specific. given the projected growth in the number of older persons and increasing diffusion of ict there is a pressing need to better understand how technology, including sm, can be better leveraged to support older persons during emergencies (choudrie, pheeraphuttranghkoon, & davari, 2018). studies which consider older persons’ sm use within their broader information ecology would be of value in order to understand how their use of other tools, and their formal and informal networks, influence their sm use. the potential of sm to break down barriers and change the lives of people living with disability has been frequently highlighted, nonetheless, accessibility concerns persist (ellis & goggin, 2013). by comparing the responses of people with pls to other vulnerable groups, our study indicates that in the disaster context accessibility issues remain a concern for people living with disabilities. people with pl were significantly less likely to be positive about the accessibility of information on sm than those of lse means and gsi persons, and less likely to be convinced that information on sm was available faster than persons of lse means. people with pls were less satisfied with the information provided on sm by emergency services and less likely to rate information on sm provided by eros, community groups and local online groups as helpful compared to gsi persons. therefore, organisations in the disaster context might usefully consider how they can better customise the information they provide on sm to meet the needs of those living with pls (morris et al., 2014). in our study those of lse means were less likely to find the information on sm provided by emergency services organisations to be helpful and were less likely to perceive their information needs were specific when compared to gsi respondents. given education is used australasian journal of information systems cooper, hayes & karanasios 2022, vol 26, research article building social resilience and inclusion in disasters 22 as a common measure of socio-economic status (australian bureau of statistics, 2011), it would be valuable to identify the extent to which the information provided by emergency services is designed to address the needs of people holding different levels of education. our study also highlighted that people of lse means were less likely to think emergency services will be too busy to monitor sm during an extreme weather event when compared to gsi persons. as those of lse are often characterised by their limited access to resources (australian bureau of statistics, 2011) it would be interesting to examine how this influences their perceptions of the resources available to emergency services organisations compared to other segments of the community. 6 contributions, directions for future research and limitations in this section we summarise our contributions research and practice, offer directions for future research and acknowledge the limitations of our study. 6.1 contributions to research and practice this paper offers several contributions to information systems research. first, although there is a growing body of research on sm and disasters (e.g. bunker, 2020; elbanna et al., 2019; mirbabaie, bunker, stieglitz, marx, & ehnis, 2020; pan & zhang, 2020) and on vulnerable persons and disasters (e.g. abedin & babar, 2018; al-saggaf & simmons, 2015; kryvasheyeu et al., 2016; panagiotopoulos et al., 2016; peary et al., 2012), this is one of only a few studies to address the important topic of sm use by vulnerable persons in disasters specifically. second, this study challenges prevalent assumptions in extant literature about vulnerable persons and sm in disasters. we provide empirical evidence that indicates vulnerable persons are not passive victims and in fact adopt a shared responsibility (mclennan & eburn, 2015) towards disasters through their strong desire to use sm to receive and share information to help themselves and others. we also highlight, by comparing our results to a similar study (reuter & spielhofer, 2017), that vulnerable persons may be more likely to view sm favourably than the general population. third, this study provides new insights to the literature on the information ecology of vulnerable persons in disasters by illuminating the importance of online and offline information sources, formal and informal networks, and informationrelated issues such as trust, accuracy and usefulness for vulnerable persons using sm in disasters. this study also offers a number of contributions to practice. first, by demonstrating that vulnerable persons are not a homogeneous group, we are able to illuminate how people with different types of vulnerabilities perceive sm for accessing and sharing information in the context of disasters. the resulting insights can be used by relevant disaster organisations to review the information they provide vulnerable persons during disasters and tailor their sm strategies to better target individuals living with a variety of vulnerabilities. second, our study highlights that vulnerable persons may have different expectations of eros than the general population and organisations should usefully engage with vulnerable persons to better address and manage their expectations. 6.2 directions for future research as one of few studies to specifically examine the use of sm by vulnerable persons during disasters there are several important directions for future research that could further augment australasian journal of information systems cooper, hayes & karanasios 2022, vol 26, research article building social resilience and inclusion in disasters 23 understanding of the diverse information needs and practices of vulnerable populations during disasters. we summarise these directions in table 1. theme suggested research directions nuanced and specific focus on the information needs and practices of vulnerable persons investigation of the factors which influence the decision of vulnerable persons to use, or not to use, sm to access and share information during disasters. examination of how vulnerable persons who use sm during disasters differ to vulnerable persons who do not use sm during disasters. studies of how perceptions of sm use in disasters vary within categories of vulnerable persons— e.g., ‘young old’ (60-70), ‘old-old’ (70-80) and ‘oldest-old’ (over 80); disability types (physical, mental) etc. investigation of how different categories of vulnerability interact with each other and impact perceptions of sm use during disasters. identification of how characteristics such as gender and culture influence categories of vulnerable persons in their use of sm to access and share information in disasters. examination of how the social networks of vulnerable persons (e.g., friends, families, carers) influence their perceptions of sm use in disasters, including studies which adopt alternative units of analysis (e.g., households). studies which compare the information needs and practices of vulnerable persons to the general population in the disaster context. community initiated sm groups identification of how community-initiated sm groups can enhance the inclusion of vulnerable persons across the pprr phases of disasters. investigation of the needs of vulnerable persons in disasters by analysing relationship dynamics within community groups on sm. identification of how vulnerable persons contribute to, and benefit from, crowdsourcing initiatives in the disaster context. information related issues examination of how information on sm can better target vulnerable persons to address their needs in the context of disasters. identification of strategies to enhance sm information flow from organisations to the public; from the public to the public, from the public to organisations and between organisations to ultimately improve outcomes for vulnerable persons in disasters. examination of how the accessibility, reliability, timeliness and veracity of information on sm can be enhanced to deliver better outcomes for vulnerable persons in disasters. studies on how information received indirectly via sm influences the behaviour of vulnerable persons in disasters. organisational integration of sm investigation of how the complexities of the organisational landscape in disaster management can be managed to better leverage sm to address the specific needs of vulnerable persons. examination of how organisations can effectively integrate sm with traditional communication channels to better support the needs of vulnerable persons during disasters. identification of how vulnerable persons’ expectations of emergency services organisations’ use of sm can be managed. studies of how different types of organisations can best leverage sm to support specific groups of vulnerable persons (e.g., the elderly, those living with pls etc.) sentiment and network analysis investigation of whether sentiment and network analyses can be leveraged to design disaster management strategies that address the specific needs of vulnerable persons. identification of what sm sentiment and network analyses can tell us about the needs and role of vulnerable persons in disasters is; and identification of how such analyses can improve outcomes for vulnerable persons across the pprr phases of disaster. undertaking sentiment and network analyses to give a voice to vulnerable persons and democratise participation in disaster management. table 1. directions for future research australasian journal of information systems cooper, hayes & karanasios 2022, vol 26, research article building social resilience and inclusion in disasters 24 6.3 limitations there are several limitations to this research. most notably, this study only focuses on sm users, it does not account for individuals who do not use sm (an estimated 20.1% of the population in australia) (kemp, 2021) and may be most vulnerable during emergencies. as non-probability sampling was used it cannot be assumed that our results are representative of vulnerable persons in victoria and beyond. our study was limited to an investigation of four types of vulnerability. in comparing results between the four vulnerable groups, we did not take into account those who identify as having multiple vulnerabilities. we thus did not investigate the impact of intersectionality and how these can potentially exacerbate vulnerability. social categorisations such as race and gender can create overlapping and interdependence systems of disadvantage that impact the outcomes of vulnerable persons during emergencies and disaster. our analysis did not consider these additional social categorisations. in comparing vulnerable persons’ attitudes, needs and future plans towards using sm to access and share information in disasters to the general population (section 5.2) we utilised the study of reuter and spielhofer (2017). while both studies utilised the same survey instrument (albeit modified), they were undertaken at different times (2015 vs. 2018) and with participants from different geographical locations and cultural backgrounds (i.e., australian vs. european citizens). while arguably there may be as many cultural differences between european countries (reuter et al., 2019) as there are between australia and these countries, it is important to be cautious when interpreting the comparative results. 7 conclusion although vulnerable persons may suffer disproportionally during disasters, they are not simply passive victims and have been shown to have a strong desire to receive and provide reliable information. conceptualisations of vulnerability in the literature are moving away from victimhood and viewing vulnerable persons as passive recipients of support, towards recognising those traditionally viewed as vulnerable as having self-awareness and capacities that contribute to social protection and risk reduction (wisner, 2016). by identifying how vulnerable persons perceive sm use during extreme weather events we have highlighted how they are proactive in protecting themselves and their friends and family. our findings indicate that sm can empower vulnerable persons in disasters and calls into question assumptions about the lack of technology and sm use by vulnerable persons when compared to the general population. nonetheless, there is potential for sm to be further leveraged to deliver improved outcomes to those traditionally thought of as more vulnerable. we highlighted a potential mismatch between vulnerable persons’ expectations of emergency services in responding to requests for help on sm and the extent to which emergency services will be aware of, and be able to respond to, such requests. organisations and community groups might consider patterns of usage amongst vulnerable persons more carefully to ensure their sm strategies address their requirements. references abedin, b., & babar, a. (2018). institutional vs. non-institutional use of social media during emergency response: a case of twitter in 2014 australian bush fire. information systems frontiers, 20(4), 729-740. https://link.springer.com/article/10.1007/s10796-017-9789-4 australasian journal of information systems cooper, hayes & karanasios 2022, vol 26, research article building social resilience and inclusion in disasters 25 al-saggaf, y., & simmons, p. (2015). social media in saudi arabia: exploring its use during two natural disasters. technological forecasting and social change, 95, 3-15. https://doi.org/10.1016/j.techfore.2014.08.013 alexander, d. (2014). social media in disaster risk reduction and crisis management. science and engineering ethics, 20, 717–733. https://doi.org/10.1007/s11948-013-9502-z alwang, j., siegel, p. b., & jorgensen, s. l. (2001). vulnerability: a view from different disciplines. retrieved january 13, 2022, from https://documents.worldbank.org/en/publication/ documents-reports/documentdetail/636921468765021121/vulnerability-a-view-fromdifferentdisciplines. american red cross. (2012). social media in disasters and emergencies. retrieved july 21, 2021, fromhttp://www.a1881.g.akamai.net/7/1881/26640/v0001/redcross.download.akamai.co m/ 26640/polldata/social. appleby, l. (2013). connecting the last mile: the role of communications in the great east japan earthquake. retrieved july 21, 2021, from www.internews.org/sites/default /files/resources/internewseurope_report_japan_connecting%20the%20last%20mile%2 0japan_2013.pdf. appleby-arnold, s., brockdorff, n., fallou, l., & bossu, r. (2019). truth, trust, and civic duty: cultural factors in citizens' perceptions of mobile phone apps and social media in disasters. journal of contingencies and crisis management, 27(4), 293-305. https://doi.org/10.1111/1468-5973.12282 australian bureau of statistics. (2011). measures of socioeconomic status: information paper. retrieved july 21, 2021, from https://www.ausstats.abs.gov.au/ausstats/subscriber.nsf/ 0/367d3800605db064ca257 8b60013445c/$file/1244055001_2011.pdf. baker, s., brown, t., caleb, n., marella, m., morris, k, nasak, m. ... (2017). disability inclusion in disaster risk reduction: experiences of people with disabilities in vanuatu during and after tropical cyclone pam and recommendations for humanitarian agencies. cbm nossal institute partnership for disability inclusion development. retrieved january 22, 2022, from https://mspgh.unimelb.edu.au/_data/assets/pdf_file/0011/2567576/web-didrrreport-14112017.pdf barnard, y., bradley, m. d., hodgson, f., & lloyd, a. d. (2013). learning to use new technologies by older adults: perceived difficulties, experimentation behaviour and usability. computers in human behavior, 29(4), 1715-1724. https://doi.org/10.1016/ j.chb.2013.02.006 bird, d., ling, m., & haynes, k. (2012). flooding facebook ? the use of social media during the queensland and victorian floods. australian journal of emergency management, 27(1), 27-33. buheji, m., da costa cunha, k., beka, g., mavrić, b., de souza, y. l. d. c., da costa silva, s. s., . . . yein, t. c. (2020). the extent of covid-19 pandemic socio-economic impact on global poverty. a global integrative multidisciplinary review. american journal of economics, 10(4), 213-224. https://doi.org/10.5923/j.economics.20201004.02 bunker, d. (2020). who do you trust? the digital destruction of shared situational awareness and the covid-19 infodemic. international journal of information management, 55, 102201. https://doi.org/10.1016/j.ijinfomgt.2020.102201 canadian red cross. (2012). social media during emergencies. retrieved july 21, 2021, from http://www.redcross.ca/cmslib/general/pub_social_media_in_emergencies_survey_oc 012_en.pdf. http://www.a1881.g.akamai.net/7/1881/26640/v0001/redcross.download.akamai.com/ http://www.a1881.g.akamai.net/7/1881/26640/v0001/redcross.download.akamai.com/ http://www.redcross.ca/cmslib/general/pub_social_media_in_emergencies_survey_oc australasian journal of information systems cooper, hayes & karanasios 2022, vol 26, research article building social resilience and inclusion in disasters 26 chan, y., & walmsley, r. p. (1997). learning and understanding the kruskal-wallis oneway analysis-of-variance-by-ranks test for differences among three or more independent groups. physical therapy, 77(12), 1755-1761. https://doi.org/10.1093/ptj/77.12.1755 choudrie, j., pheeraphuttranghkoon, s., & davari, s. (2018). the digital divide and older adult population adoption, use and diffusion of mobile phones: a quantitative study. information systems frontiers, 23(3), 673-695. https://doi.org/10.1007/s10796-018-9875-2 cohen, o., goldberg, a., lahad, m., & aharonson-daniel, l. (2017). building resilience: the relationship between information provided by municipal authorities during emergency situations and community resilience. technological forecasting and social change, 121, 119125. https://doi.org/10.1016/j.techfore.2016.11.008 cooper, v., fairbrother, p., elliott, g., walker, m., & ch'ng, h.-y. (2020). shared responsibility and community engagement: community narratives of bushfire risk information in victoria, australia. journal of rural studies, 80, 259-272. https://doi.org/10.1016/ j.jrurstud.2020.09.015 cumming, g. (2012). understanding the new statistics: effect sizes, confidence intervals, and meta analysis. new york: routledge. cutter, s. l. (1996). vulnerability to environmental hazards. progress in human geography, 20(4), 529-539. https://doi.org/10.1177/030913259602000407 cutter, s. l., boruff, b. j., & shirley, w. l. (2003). social vulnerability to environmental hazards. social science quarterly, 84(2), 242-261. https://doi.org/10.1111/1540-6237.8402002 deacon, b. (2018). emergency announcements alone during disasters not reliable in saving lives. retrieved july 21, 2021, from http://www.abc.net.au/news/2018-01-06/emergency announcements-alone-during-disasters-not-reliable/9304666 eakin, h., & luers, a. l. (2006). assessing the vulnerability of social-environmental systems. annual review of environment and resources, 31(1), 365-394. https://doi.org/10.1146/ annurev.energy.30.050504.144352 elbanna, a., bunker, d., levine, l., & sleigh, a. (2019). emergency management in the changing world of social media: framing the research agenda with the stakeholders through engaged scholarship. international journal of information management, 47, 112 120. https://doi.org/10.1016/j.ijinfomgt.2019.01.011 ellis, k., & goggin, g. (2013). disability and social media. in j. hunsinger & t.m. senft (eds.), the social media handbook (pp. 126-143): new york: routledge. enang, i., murray, j., dougall, n., wooff, a., heyman, i., & aston, e. (2019). defining and assessing vulnerability within law enforcement and public health organisations: a scoping review. health & justice, 7(2). https://doi.org/10.1186/s40352-019-0083-z feldman, d., contreras, s., karlin, b., basolo, v., matthew, r., sanders, b., . . . luke, a. (2016). communicating flood risk: looking back and forward at traditional and social media outlets. international journal of disaster risk reduction, 15, 43-51. https://doi.org/10.1016/j.ijdrr.2015.12.004 gibson, m. j., & hayunga, m. (2006). we can do better: lessons learned for protecting older persons in disasters. retrieved july 21, 2021, from http://assets.aarp.org/rgcenter/il/better.pdf gill, a. q., alam, s. l., & eustace, j. (2015). social architecture: an emergency management case study. australasian journal of information systems, 19, 23-40. https://doi.org/10.3127/ ajis.v19i0.979 goolsby, r. (2010). social media as crisis platform: the future of community maps/crisis maps. acm transactions on intelligent systems and technology, 1(1), 1-11. https://doi.org/10.1145/1858948.1858955 http://www.abc.net.au/news/2018-01-06/emergencyhttp://assets.aarp.org/rgcenter/il/better.pdf australasian journal of information systems cooper, hayes & karanasios 2022, vol 26, research article building social resilience and inclusion in disasters 27 hagar, c. (2013). when disaster strikes, where do you turn? retrieved july 21, 2021, from https://www.fte.org/teachers/teacher-resources/lesson-plans/disasterslessons/lesson3when-disaster-strikes-what-can-government-do/ hjorth, l., & kim, k.-h. y. (2011). the mourning after: a case study of social media in the 3.11 earthquake disaster in japan. television & new media, 12(6), 552-559. https://doi.org/10.1177/1527476411418351 howard, a., agllias, k., bevis, m., & blakemore, t. (2017). “they’ll tell us when to evacuate”: the experiences and expectations of disaster-related communication in vulnerable groups. international journal of disaster risk reduction, 22, 139-146. https://doi.org/10.1016/j.ijdrr.2017.03.002 hughes, a. l., & tapia, a. h. (2015). social media in crisis: when professional responders meet digital volunteers. journal of homeland security and emergency management, 12(3) 679-706. https://doi.org/10.1515/jhsem-2014-0080 ifrc. (2005). world disasters report. retrieved july 21, 2021, from https://www.ifrc.org/global/ publications/disasters/wdr/69001-wdr2005-englishlr.pdf ifrc. (2013). world disasters report. retrieved july 21, 2021, from http://www.ifrc.org/pagefiles/134658/wdr%202013%20complete.pdf karanasios, s., cooper, v., hayes, p., & adrot, a. (2019, december 15-18). “an iron hand in a velvet glove’’: the embodiment of the platform logic in the emergency sector. [paper presentation]. fortieth international conference on information systems, munich, germany. https://aisel.aisnet.org/icis2019/ karanasios, s., cooper, v.,adrot, a., & mercieca, d. (2020, january 8-10). gatekeepers rather than helpless: an exploratory investigation of senior’s use of information and communication technology in critical settings. [paper presentation].53rd hawaii international conference on system sciences (hicss-53), honolulu, hawaii. https://aisel.aisnet.org/hicss-53/ kaufhold, m. a., gizikis, a., reuter, c., habdank, m., & grinko, m. (2019). avoiding chaotic use of social media before, during, and after emergencies: design and evaluation of citizens’ guidelines. journal of contingencies and crisis management, 27(3), 198-213. https://doi.org/10.1111/1468-5973.12249 kaur, h. j., & kumar, r. (2015, may 15-16). sentiment analysis from social media in crisis situations. [paper presentation]. international conference on computing, communication & automation, noida, india. https://doi.org/10.1109/ccaa.2015.7148383 kemp, s. (2021). digital 2021: australia. retrieved july 21, 2021, from https://datareportal.com/reports/digital-2021-australia kim, j., & hastak, m. (2018). social network analysis: characteristics of online social networks after a disaster. international journal of information management, 38(1), 86-96. https://doi.org/10.1016/j.ijinfomgt.2017.08.003 kraska-miller, m. (2013). nonparametric statistics for social and behavioral sciences. boca raton, florida: crc press. kryvasheyeu, y., chen, h., obradovich, n., moro, e., van hentenryck, p., fowler, j., & cebrian, m. (2016). rapid assessment of disaster damage using social media activity. science advances, 2(3). https://doi.org/10.1126/sciadv.1500779 kulemeka, o. (2014). how people affected by disaster use the internet: a study of facebook usage during the 2014 hazelwood mine fire in victoria. australasian journal of disaster and trauma studies, 18(2), 51-56. http://www.fte.org/teachers/teacher-resources/lesson-plans/disasterslessons/lessonhttp://www.ifrc.org/pagefiles/134658/wdr%202013%20complete.pdf australasian journal of information systems cooper, hayes & karanasios 2022, vol 26, research article building social resilience and inclusion in disasters 28 lukasiewicz, a., dovers, s., & eburn, m. (2017). shared responsibility: the who, what and how. environmental hazards, 16(4), 291-313. https://doi.org/10.1080/17477891.2017.1298510 luna, f. (2014). ‘vulnerability’, an interesting concept for public health: the case of older persons. public health ethics, 7(2), 180-194. https://doi.org/10.1093/phe/phu012 luna, f. (2018). identifying and evaluating layers of vulnerability–a way forward. developing world bioethics, 19(2), 86-95. https://doi.org/10.1111/dewb.12206 luna, s., & pennock, m. j. (2018). social media applications and emergency management: a literature review and research agenda. international journal of disaster risk reduction, 28, 565-577. https://doi.org/10.1016/j.ijdrr.2018.01.006 mann, h. b., & whitney, d. r. (1947). on a test of whether one of two random variables is stochastically larger than the other. the annals of mathematical statistics, 18(1), 50-60. http://www.jstor.org/stable/2236101 masozera, m., bailey, m., & kerchner, c. (2007). distribution of impacts of natural disasters across income groups: a case study of new orleans. ecological economics, 63(2-3), 299306. https://doi.org/10.1016/j.ecolecon.2006.06.013 mclennan, b., & eburn, m. (2015). exposing hidden-value trade-offs: sharing wildfire management responsibility between government and citizens. international journal of wildland fire, 24(2), 162-169. https://doi.org/10.1071/wf12201 mirbabaie, m., bunker, d., stieglitz, s., marx, j., & ehnis, c. (2020). social media in times of crisis: learning from hurricane harvey for the coronavirus disease 2019 pandemic response. journal of information technology, 35(3), 195-213. https://doi.org/10.1177/0268396220929258 morris, j. t., mueller, j. l., & jones, m. l. (2014). use of social media during public emergencies by people with disabilities. western journal of emergency medicine, 15(5), 567-574. https://doi.org/10.5811/westjem.2014.4.21274 morrissey, s. a., & reser, j. p. (2007). natural disasters, climate change and mental health considerations for rural australia. australian journal of rural health, 15(2), 120-125. https://doi.org/10.1111/j.1440-1584.2007.00865.x nakahara, s., & ichikawa, m. (2013). mortality in the 2011 tsunami in japan. journal of epidemiology, 23(1), 70-73. https://doi.org/10.2188/jea.je20120114 ogie, r. i., & pradhan, b. (2019). natural hazards and social vulnerability of place: the strength-based approach applied to wollongong, australia. international journal of disaster risk science, 10(3), 404-420. https://doi.org/10.1007/s13753-019-0224-y ohchr. (2015). thematic study on the rights of persons with disabilities under article 11 of the convention on the rights of persons with disabilities, on situations of risk and humanitarian emergencies. retrieved july 21, 2021, from https://www.refworld.org/docid/ 56c42c744.html onder, g., rezza, g., & brusaferro, s. (2020). case-fatality rate and characteristics of patients dying in relation to covid-19 in italy. jama, 323(18), 1775-1776. https://doi.org/10.1001/ jama.2020.4683 palen, l., vieweg, s., & anderson, k. m. (2011). supporting “everyday analysts” in safetyand time-critical situations. the information society, 27(1), 52-62. https://doi.org/10.1080/ 01972243.2011.534370 pan, s. l., & zhang, s. (2020). from fighting covid-19 pandemic to tackling sustainable development goals: an opportunity for responsible information systems research. international journal of information management, 55, 102196. https://doi.org/10.1016/ http://www.jstor.org/stable/2236101 australasian journal of information systems cooper, hayes & karanasios 2022, vol 26, research article building social resilience and inclusion in disasters 29 j.ijinfomgt.2020.102196 panagiotopoulos, p., barnett, j., bigdeli, a. z., & sams, s. (2016). social media in emergency management: twitter as a tool for communicating risks to the public. technological forecasting and social change, 111, 86-96. https://doi.org/10.1016/j.techfore.2016.06.010 pang, n., karanasios, s., & anwar, m. (2020). exploring the information worlds of older persons during disasters. journal of the association for information science and technology, 71(6), 619-631. https://doi.org/10.1002/asi.24294 peary, b. d., shaw, r., & takeuchi, y. (2012). utilization of social media in the east japan earthquake and tsunami and its effectiveness. journal of natural disaster science, 34(1), 318. https://doi.org/10.2328/jnds.34.3. poblet-balcell, m., cooper, v., & karanasios, s. (2018, november 5-7). look after your neighbours: social media and vulnerable groups during extreme weather events. [paper presentation]. information systems for crisis response and management (iscram) asia-pacific, wellington, new zealand. https://www.confer.co.nz/iscramasiapacific2018/conferenceproceedings-online/ qiu, j., xu, l., wang, j., & gu, w. (2020). mutual influences between message volume and emotion intensity on emerging infectious diseases: an investigation with microblog data. information & management, 57(4), 103217. https://doi.org/10.1016/j.im.2019.103217 qu, y., wu, p. f., & wang, x. (2009, january 5-8). online community response to major disaster: a study of tianya forum in the 2008 sichuan earthquake. [paper presentation]. 42nd hawaii conference on systems sciences (hicss-42), waikoloa, hawaii. https://www.computer.org/csdl/proceedings/hicss/2009/12omnwd1puh reuter, c., kaufhold, m.-a., schmid, s., spielhofer, t., & hahne, a. s. (2019). the impact of risk cultures: citizens' perception of social media use in emergencies across europe. technological forecasting and social change, 148. https://doi.org/10.1016/j.techfore.2019. 119724 reuter, c., kaufhold, m.-a., spahr, f., spielhofer, t., & hahne, a. s. (2020). emergency service staff and social media– a comparative empirical study of the attitude by emergency services staff in europe in 2014 and 2017. international journal of disaster risk reduction, 46, 101516. https://doi.org/10.1016/j.ijdrr.2020.101516 reuter, c., marx, a., & pipek, v. (2012). crisis management 2.0: towards a systematization of social software use in crisis situations. international journal of information systems for crisis response and management (ijiscram), 4(1), 1-16. https://doi.org/10.4018/jiscrm. 2012010101 reuter, c., & spielhofer, t. (2017). towards social resilience: a quantitative and qualitative survey on citizens' perception of social media in emergencies in europe. technological forecasting and social change, 121, 168-180. https://doi.org/10.1016/j.techfore.2016.07.038 schmidt, a., wolbers, j., ferguson, j., & boersma, k. (2017). are you ready2help? conceptualizing the management of online and onsite volunteer convergence. journal of contingencies and crisis management, 1-12. https://doi.org/10.1111/1468-5973.12200 schmidt, s., & galea, e. (2013). recommendations for different types of end-users. in s. schmidt & e. galea (eds.), human behaviour in emergencies and disasters: a crosscultural investigation (pp. 11-20). berlin: pabst science publishers. smoyer tomic, k. e. (2003). heat wave. jama, 289(12), 1573-1574. https://doi.org/10.1001/ jama.289.12.1573 australasian journal of information systems cooper, hayes & karanasios 2022, vol 26, research article building social resilience and inclusion in disasters 30 stephenson, d. b., diaz, h., & murnane, r. (2008). definition, diagnosis, and origin of extreme weather and climate events. climate extremes and society, 340, 11-23. https://doi.org/10.1017/cbo9780511535840.004 teo, m., goonetilleke, a., ahankoob, a., deilami, k., & lawie, m. (2018). disaster awareness and information seeking behaviour among residents from low socio-economic backgrounds. international journal of disaster risk reduction, 31, 1121-1131. https://doi.org/10.1016/j.ijdrr.2018.09.008 thomas, d. r. (2006). a general inductive approach for analyzing qualitative evaluation data. american journal of evaluation, 27(2), 237-246. https://doi.org/10.1177/1098214005283748 unisdr. (2016). report of the open-ended intergovernmental expert working group on indicators and terminology relating to disaster risk reduction. retrieved july 21, 2021 from https://www.preventionweb.net/files/50683_oiewgreportenglish.pdf wilcoxon, f. (1945). individual comparisons by ranking methods. biometrics bulletin, 1(6), 8083. https://doi.org/10.2307/3001968 wilde, g. j. (1998). risk homeostasis theory: an overview. injury prevention, 4(2), 89-91. http://dx.doi.org/10.1136/ip.4.2.89 wisner, b. (2004). assessment of capability and vulnerability. in g bankoff & g frerks (eds.), mapping vulnerability: disasters, development and people (pp. 183-193). routledge. wisner, b. (2016). vulnerability as concept, model, metric, and tool. in oxford research encyclopedia of natural hazard science. retrieved january 13, 2022, from https://oxfordre.com/naturalhazardscience/view/10.1093/acrefore/9780199389407.001.00 01/acrefore-9780199389407-e-25. wolbers, j. j., boersma, f. k., ferguson, j. e., mulder, f., & groenewegen, p. (2016). two faces of disaster response: transcending the dichotomy of control and collaboration during the nepal earthquake relief operation. international journal of mass emergencies and disasters, 34(4), 419. https://arro.anglia.ac.uk/id/eprint/705792/ zebardast, e. (2013). constructing a social vulnerability index to earthquake hazards using a hybrid factor analysis and analytic network process (f’anp) model. journal of the international society for the prevention and mitigation of natural hazards, 65(3), 13311359. https://doi.org/10.1007/s11069-012-0412-1 appendix one survey instrument 1. we care about the quality of our survey data and hope to receive the most accurate measures of your opinions, so it is important to us that you thoughtfully provide your best answer to each question in the survey. do you commit to providing your thoughtful and honest answers to the questions in this survey? i will provide my best answers i will not provide my best answers i can’t promise either way 2. have you ever used social media such as facebook, twitter, instagram, etc. to find or share information as a result of an extreme weather event such as a flood, storm, fire or heatwave? yes no 3. please indicate which of the following (if any) you identify as being/having http://www.preventionweb.net/files/50683_oiewgreportenglish.pdf australasian journal of information systems cooper, hayes & karanasios 2022, vol 26, research article building social resilience and inclusion in disasters 31 over 60 years of age geographically or socially isolated (e.g. live in a remote community, or live alone in an urban area) suffer from physical limitations (in terms of a physical functioning, mobility, dexterity etc.) low socio-economic means none of the above 4. in which australian state or territory do you currently live? new south wales queensland south australia tasmania victoria western australia australian capital territory northern territory i do not currently live in australia 5. what is your age? 18-21 22-29 30-39 40-49 50-59 60-69 70 or older 6. what is your gender: male female other 7. what is your highest level of education? high school diploma bachelor degree master degree doctoral degree other (please specify): 8. please specify your nationality 9. are you from a non-english speaking background? yes no 10. do you have any children under the age of 18? yes no 11 what type of household best describes your living situation? australasian journal of information systems cooper, hayes & karanasios 2022, vol 26, research article building social resilience and inclusion in disasters 32 lone-person household family household with children family household without children group-household (two or more unrelated adults) other (please specify) 12. what is your current employment status? full-time part-time casual fixed-term shift worker self-employed pensioner retired homemaker unemployed other (please specify) 13. to what extent do you currently use the following type of technology? type of technology often sometimes seldom never desktop computer laptop tablet regular mobile phone (not a smart phone) smartphone other (please specify) 14. to what extent do you currently use the following types of social media to share or look at information? type of social media often sometimes seldom never facebook twitter instagram blogs youtube others (please specify) 15. typically how much time do you actively spend on social media? 1 2 3 4 5 or more hours per day -----------|------------------------ per day during an extreme weather event -----------------|------------------ 16. please indicate how strongly you agree or disagree with the following statements item strongly agree agree neither agree or disagree disagree strongly disagree in my private life, i use social media very often i regularly post messages on social media such australasian journal of information systems cooper, hayes & karanasios 2022, vol 26, research article building social resilience and inclusion in disasters 33 as twitter, facebook or instagram most of my friends use social media to keep in touch it is important for emergency services to use social media to keep in touch with the public during emergencies i would not trust a message posted on social media during an emergency unless it came from an official source such as the police or fire service emergency services should not trust information on social media 17. which of the following communication channels have you ever used to get information about an extreme weather event such as a flood, storm, fire or heatwave? tv local radio station online news mobile apps social media (facebook, twitter, instagram, etc.) text message alerts online sites for extreme weather agencies or emergency services none of the these other (please specify) 18. when using social media such as facebook, twitter, instagram, etc. to find out information as a result of an extreme weather event such as a flood, storm, fire or heatwave, what kind of information have you looked for? weather conditions or warnings road or traffic conditions damage caused by the event the location or status of friends/family information about how others are coping with the extreme weather event eyewitness photographs or videos what to do to keep yourself safe other (please specify) 19. how helpful was the information you sought using social media about an extreme weather event from each of the following organisations? organisation type extremely helpful very helpful somewhat helpful slightly helpful not at all helpful not applicable emergency response organisations (e.g. police, fire brigade) government agencies (e.g. department of human services, bureau of meteorology) non-government organisations (e.g. red cross, salvation army) local government (e.g. local council) australasian journal of information systems cooper, hayes & karanasios 2022, vol 26, research article building social resilience and inclusion in disasters 34 local community groups (e.g. grey nomads, surf clubs) local online groups (e.g. twitter feed or facebook page of local community group) other (please specify) 20. how likely are you to use social media in the future to look for information as a result of an extreme weather event such as a flood, storm, fire or heatwave? very likely quite likely neutral not very likely not at all likely 21. what would make you (even) more likely to use social media to look for information as a result of an extreme weather event such as a flood, storm, fire or heatwave in the future? 22. have you ever received information about an extreme weather event from social media indirectly, i.e., passed on to you by someone that found the information via social media— (e.g. had a conversation with a friend about a post they saw on social media in relation to a flood)? yes no don’t know/not sure 23. what information about an extreme weather event did you receive indirectly from social media? weather conditions or warnings road or traffic conditions damage caused by the event the location or status of friends/family information about how others are coping with the extreme weather event eyewitness photographs or videos what to do to keep yourself safe other (please specify) 24. how likely are you to act upon information from social media provided to you indirectly by friends or relatives as a result of an extreme weather event such as a flood, storm, fire or heatwave? very likely quite likely neutral not very likely not at all likely 25. what would make you (even) more likely to act upon information from social media provided to you indirectly by friends or relatives in the future as a result of an extreme weather event such as a flood, storm, fire or heatwave? 26. have you ever used social media to share information as a result of an extreme weather event such as a flood, storm, fire or heatwave (e.g. posted a status update about an accident, tagged a friend in a news story about a fire, uploaded a picture of a flood)? australasian journal of information systems cooper, hayes & karanasios 2022, vol 26, research article building social resilience and inclusion in disasters 35 yes no don’t know/not sure 27. what kind of information have you shared using social media as a result of an extreme weather event such as a flood, storm, fire or heatwave? weather conditions or warnings road or traffic conditions reassurance that you are safe your feelings or emotions about what was happening your location what actions you were taking to stay safe an eyewitness description of something you experienced advice about what actions others should take to stay safe an eyewitness photo an eyewitness video other (please specify) 28. how likely are you to use social media in the future to share information with others during or after an extreme weather event such as a flood, storm, fire or heatwave? very likely quite likely neutral not very likely not at all likely 29. what would make you (even) more likely to use social media to share information as a result of an extreme weather event such as a flood, storm, fire or heatwave? 30. some people prefer now to look for information via social media rather than via tv, radio or traditional websites for the following reasons? strongly agree agree neither agree or disagree disagree strongly disagree the information via social media is available faster the information via social media is more accurate the information via social media is more accessible the information via social media is more reliable the information via social media is richer 31. imagine that you posted an urgent request for help or information on a social media site of a local emergency response organisation, such as your local police, coast guard, fire or medical emergency service. to what extent do you agree with the following statements? strongly agree agree neither agree or disagree disagree strongly disagree emergency services should regularly monitor their social media so they can promptly respond to such a request. australasian journal of information systems cooper, hayes & karanasios 2022, vol 26, research article building social resilience and inclusion in disasters 36 it is very likely that the emergency service will not know about this request. i would expect to get a response from them within an hour. emergency services are too busy to monitor social media during an emergency. 32. to what extent are you aware or have you heard of the following services provided via social media? very aware moderately aware somewhat aware slightly aware not at all aware twitter alerts used by key organisations to send alerts during emergencies. facebook safety checks allows you during an emergency to let friends and family know you and friends are safe. 33. to what extent do you agree with the following statements? strongly agree agree neither agree or disagree disagree strongly disagree information on social media from emergency services organisations meets my information needs my information needs during extreme weather events are very specific i prefer to use community groups on social media to obtain information about extreme weather events because the information is tailored to my needs i get most information related to my specific situation offline rather than through social media during extreme weather events 34. is there anything else you want to mention with regard to the use of social media before, during or after an extreme weather event such as a flood, storm, fire or heatwave? australasian journal of information systems cooper, hayes & karanasios 2022, vol 26, research article building social resilience and inclusion in disasters 37 appendix two – summary tables for quantitative analysis independent variable dependent variable kruskalwallis h test χ2, df, and n p-value mean rank for each vulnerability type post hoc paired mannwhitney test (p < .05)* vulnerability type (i.e., aged 60+, lse, pl or gsi) twitter 8.206, 3, 167 .042 age 60+ = 64.38 lse = 84.90 pl = 83.40 gsi = 94.06 age 60+ use twitter significantly less than: • gsi instagram 20.726, 3, 167 < .001 age 60+ = 53.87 lse = 86.35 pl = 78.12 gsi = 100.83 age 60+ use instagram significantly less than: • lse • gsi youtube 14.470, 3, 167 .002 age 60+ = 59.75 lse = 93.29 pl = 74.19 gsi = 92.41 age 60+ use youtube significantly less than: • lse • gsi chat tools 10.025, 3, 167 .018 age 60+ = 60.37 lse = 86.81 pl = 88.54 gsi = 91.88 age 60+ use chat tools significantly less than: • gsi table a1. kruskal-wallis h test and post hoc paired mann-whitney test analyses of vulnerability type by sm application use note: (i) *significance values have been adjusted by the bonferroni correction for multiple tests independent variable dependent variable kruskalwallis h test χ2, df, and n p-value mean rank for each vulnerability type post hoc paired mannwhitney test (p < .05)* vulnerability type (i.e., aged 60+, lse, pl or gsi) in my private life, i use sm very often 18.143, 3, 167 < .001 age 60+ = 53.15 lse = 88.71 pl = 85.50 gsi = 95.32 age 60+ use sm in their private lives significantly less than: • lse • pl • gsi i regularly post messages on sm such as twitter, facebook or instagram 12.228, 3, 167 .007 age 60+ = 57.32 lse = 91.78 pl = 89.60 gsi = 87.99 age 60+ significantly less likely to post messages on sm than: • lse • gsi most of my friends use sm to keep in touch 16.553, 3, 167 .001 age 60+ = 59.43 lse = 85.96 pl = 76.35 gsi = 99.03 age 60+ significantly less likely to perceive their friends use sm to stay in touch than: • lse • gsi emergency services should not trust information on sm† 12.390, 3, 167 .006 age 60+ = 71.15 lse = 75.22 pl = 81.02 gsi = 101.35 gsi respondents have significantly more positive attitude towards emergency services trusting information on sm than: • age 60+ • lse table a2. kruskal-wallis h test and post hoc paired mann-whitney test analyses of australasian journal of information systems cooper, hayes & karanasios 2022, vol 26, research article building social resilience and inclusion in disasters 38 vulnerability type by measures of sm use and trust towards sm independent variable dependent variable kruskal-wallis h test χ2, df, and n p-value mean rank for each vulnerability type post hoc paired mannwhitney test (p < .05)* vulnerability type (i.e., aged 60+, lse, pl or gsi) awareness of safety service provided by ‘twitter alerts’ 19.810, 3, 167 < .001 age 60+ = 53.85 lse = 83.31 pl = 83.98 gsi = 101.15 age 60+ significantly less likely to be aware of ‘twitter alerts’ than: • gsi • lse awareness of safety service provided by ‘facebook safety checks’ 18.724, 3, 167 < .001 age 60+ = 55.58 lse = 95.60 pl = 71.33 gsi = 93.68 age 60+ significantly less likely to be aware of ‘facebook safety checks’ than: • gsi • lse table a3. kruskal-wallis h test and post hoc paired mann-whitney test analyses of vulnerability type by awareness of twitter alerts and facebook safety checks note: (i) *significance values have been adjusted by the bonferroni correction for multiple tests independent variable dependent variable kruskalwallis h test χ2, df, and n pvalue mean rank for each vulnerability type post hoc paired mannwhitney test (p < .05)* vulnerability type (i.e., aged 60+, lse, pl or gsi) helpfulness of information sought via sm from emergency response organisation s (e.g. police, fire brigade) 11.706, 3, 167 .008 age 60+ = 83.87 lse = 72.46 pl = 67.87 gsi = 97.30 gsi rated perceived helpfulness of information sought via sm from eros significantly higher than: • lse • pl helpfulness of information sought via sm from local community groups (e.g. grey nomads, surf clubs) 10.509, 3, 167 .015 age 60+ = 60.55 lse = 70.94 pl = 48.29 gsi = 79.26 gsi rated perceived helpfulness of information sought via sm from local community groups significantly higher than: • pl helpfulness of information sought via sm from local online groups (e.g. twitter feed or facebook page of local community group) 9.202, 3, 167 .027 age 60+ = 69.98 lse = 84.06 pl = 59.88 gsi = 88.78 gsi rated perceived helpfulness of information sought via sm from local online groups significantly higher than: • pl table a4. kruskal-wallis h test and post hoc paired mann-whitney test analyses of vulnerability type australasian journal of information systems cooper, hayes & karanasios 2022, vol 26, research article building social resilience and inclusion in disasters 39 by perceived helpfulness of information about extreme weather events sought via sm from different organisations note: (i) *significance values have been adjusted by the bonferroni correction for multiple tests independent variable dependent variable kruskalwallis h test χ2, df, and n pvalue mean rank for each vulnerability type post hoc paired mannwhitney test (p < .05)* vulnerability type (i.e., aged 60+, lse, pl or gsi) how likely are you to use sm in the future to look for information as a result of an extreme weather event such as a flood, storm, fire or heatwave? 12.666, 3, 167 .005 age 60+ = 64.85 lse = 94.19 pl = 71.23 gsi = 90.18 age 60+ significantly less likely to perceive they’ll seek information from sm in the future than: • lse table a5. kruskal-wallis h test and post hoc paired mann-whitney test analyses of vulnerability type by likelihood of using sm in the future to obtain information about extreme weather events note: (i) *significance values have been adjusted by the bonferroni correction for multiple tests independent variable dependent variable kruskalwallis h test χ2, df, and n p-value mean rank for each vulnerability type post hoc paired mannwhitney test (p < .05)* vulnerability type (i.e., aged 60+, lse, pl or gsi) people prefer to look for informatio n via sm rather than tv, radio or traditional websites because the informatio n is available faster 27.028, 3, 167 < .001 age 60+ = 51.75 lse = 93.17 pl = 70.00 gsi = 98.87 lse and gsi significantly more likely to think people prefer to look for information on sm because it is faster than: • 60+ lse significantly more likely to think people prefer to look for information on sm because it is faster than: • pl people prefer to look for informatio n via sm rather than tv, radio or traditional websites because 21.213, 3, 167 < .001 age 60+ = 61.35 lse = 98.09 pl = 62.31 gsi = 92.26 lse and gsi significantly more likely to think people prefer to look for information on sm because it is more accessible than: • 60+ • pl australasian journal of information systems cooper, hayes & karanasios 2022, vol 26, research article building social resilience and inclusion in disasters 40 the informatio n is more accessible people prefer to look for informatio n via sm rather than tv, radio or traditional websites because the informatio n is more reliable 9.644, 3, 167 .022 age 60+ = 69.43 lse = 83.89 pl = 71.90 gsi = 97.77 gsi significantly more likely to think people prefer to look for information on sm because it is more reliable than: • 60+ table a6. kruskal-wallis h test and post hoc paired mann-whitney test analyses of vulnerability type by perceived reasons people prefer to look for information via sm compared to other information sources note: (i) *significance values have been adjusted by the bonferroni correction for multiple tests independe nt variable dependent variable kruskal-wallis h test χ2, df, and n pvalue mean rank for each vulnerability type post hoc paired mannwhitney test (p < .05)* vulnerabili ty type (i.e., aged 60+, lse, pl or gsi) emergency services should regularly monitor their sm so they can promptly respond to urgent requests for help. 16.208, 3, 167 .001 age 60+ = 55.38 lse = 88.45 pl = 82.17 gsi = 95.95 gsi and lse significantly more likely to think emergency services should regularly monitor their sm than: • age 60+ if i posted an urgent request for help on the sm site of a local ero i would expect to get a response from them within an hour. 9.332, 3, 167 .025 age 60+ = 71.20 lse = 86.84 pl = 67.46 gsi = 95.91 • no significant pairwise differences emergency services are too busy to monitor sm during an emergency. 8.056, 3, 167 .045 age 60+ = 82.18 lse = 73.33 pl = 80.08 gsi = 97.71 gsi significantly more likely to think emergency services are too busy to monitor sm than: • lse table a7. kruskal-wallis h test and post hoc paired mann-whitney test analyses of vulnerability type by expectations of emergency services’ use of sm note: (i) *significance values have been adjusted by the bonferroni correction for multiple tests independent dependent kruskalp-value mean rank for each post hoc paired mannaustralasian journal of information systems cooper, hayes & karanasios 2022, vol 26, research article building social resilience and inclusion in disasters 41 variable variable wallis h test χ2, df, and n vulnerability type whitney test (p < .05)* vulnerability type (i.e., aged 60+, lse, pl or gsi) information on sm from emergency services organisations meets my needs 14.886, 3, 167 .002 age 60+ = 63.13 lse = 86.97 pl = 70.17 gsi = 98.19 gsi significantly more likely to think information on sm from esos meets their needs than: • age 60+ • pl my information needs during extreme weather events are very specific 18.273, 3, 167 < .001 age 60+ = 64.00 lse = 80.84 pl = 71.96 gsi = 103.82 gsi significantly more likely to think their information needs during extreme weather event are very specific than: • age 60+ • lse • pl i get most information related to my specific situation offline rather than through sm during extreme weather events 9.043, 3, 167 .029 age 60+ = 74.48 lse = 82.30 pl = 68.88 gsi = 98.06 • no significant pairwise differences table a8. kruskal-wallis h test and post hoc paired mann-whitney test analyses of vulnerability type by participant’s information needs during extreme weather events note: (i) *significance values have been adjusted by the bonferroni correction for multiple tests copyright: © 2022 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v26i0.3281 http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 related work 2.1 sm use in disasters 2.1.1 how sm is used in disasters 2.1.2 community initiated sm groups in disasters 2.1.3 organisational integration of sm in disasters 2.1.4 information-related issues for sm in disasters 2.1.5 sentiment and network analysis via sm in disasters 2.2 sm use by vulnerable persons in disasters 3 methods 3.1 survey design 3.2 characteristics of survey participants 3.3 survey analysis 4 empirical results 4.1 current use of technology and views towards sm 4.2 sm as an information source during disasters 4.3 use of information received indirectly via sm 4.4 sharing information using sm during emergencies 4.5 preference for sm as an information source compared to alternative information sources 4.6 expectations towards emergency services 5 discussion 5.1 challenging prevalent assumptions about vulnerable persons and sm in disasters 5.2 comparison of perceptions of sm use in disasters between vulnerable groups 6 contributions, directions for future research and limitations 6.1 contributions to research and practice 6.2 directions for future research 6.3 limitations 7 conclusion 1. we care about the quality of our survey data and hope to receive the most accurate measures of your opinions, so it is important to us that you thoughtfully provide your best answer to each question in the survey. do you commit to providing your thoughtful and honest answers to the questions in this survey? 2. have you ever used social media such as facebook, twitter, instagram, etc. to find or share information as a result of an extreme weather event such as a flood, storm, fire or heatwave? 3. please indicate which of the following (if any) you identify as being/having 4. in which australian state or territory do you currently live? 5. what is your age? 6. what is your gender: 7. what is your highest level of education? 8. please specify your nationality 9. are you from a non-english speaking background? 10. do you have any children under the age of 18? 11 what type of household best describes your living situation? 12. what is your current employment status? paper title (paper title style) australasian journal of information systems khan et al. 2021, vol 25, research article effect of barrier related factors 1 effect of barrier related factors on perceived usefulness and ease of use of social media applications in the australian healthcare sector irfanuzzaman khan university of canberra, australia jennifer loh university of canberra, australia abu saleh university of canberra, australia ali quazi university of canberra, australia majharul talukder university of canberra, australia majharul.talukder@canberra.edu.au abstract despite the growing popularity of social media internationally, an extant review of the literature revealed a low rate of social media usage among healthcare professionals. while cynicism amongst healthcare professionals might be a reason, there might be other factors that could explain healthcare professionals’ reluctance to use social media in their practices. this research investigated potential barriers that affected healthcare professionals’ behavioural intention to use social media. a cross-sectional survey was randomly administered to 824 healthcare professionals working in australian healthcare organisations. at the end of data collection, 219 usable responses were collected. analysis of data via structural equation model (sem) found that perceived trust, privacy threats, professional boundary, facilitating conditions and self-efficacy significantly influence the notion of perceived usefulness and ease of use. in addition, information quality directly influences health professionals’ perceived ease of utilising social media technology. the result also indicated that gender moderates the relationship between barrier-related factors and perceived usefulness and ease of use. this study’s findings have important implications for healthcare providers and policymakers regarding medical professionals’ perceptions of the potential challenges in using social media as well as developing strategies to counter misinformation against the backdrop of covid-19. keywords: social media, healthcare professional, privacy threat, trust, perceived usefulness, professional boundary 1 introduction rapid advances in digital technologies and people’s increasing online connectivity have revolutionised personal and social life to the point where social media is omnipresent in almost everything that people do (fernández-luque & bau, 2015). within healthcare communication, consumers now rely heavily on information found on social media sites which influences critical health decisions such as self-diagnosis, accessing peer support to manage chronic conditions, diet management and selecting health providers (gagnon & sabus, 2015). in doing so, social media facilitates patient empowerment by placing healthcare australasian journal of information systems khan et al. 2021, vol 25, research article effect of barrier related factors 2 consumers in a position to take control of their needs (denecke et al., 2015). from the perspective of healthcare providers, social media offers an abundance of potential benefits. this includes better patient engagement and patient care, information-and knowledgesharing, social support enhanced community outreach and improved quality of patient care (barnes, kaul, & kudchadkar, 2019; park, bowling, shaw, li, & chen, 2019). public health agencies also frequently employ social media to supplement their existing health communications strategies (lober & flowers, 2011). prior studies have found that social media sites positively impact on patients’ psychological well-being (erfani, abedin, & blount, 2017). furthermore, a social media presence can consolidate and promote healthcare providers’ or a practitioner’s brand and generate more business opportunities for them (kotsenas et al., 2018a). notwithstanding the inherent benefits of using social media for health communication, professional applications of social media in medical contexts are deemed to be complex, unregulated and is an informal mechanism of communication and information exchange (moorhead et al., 2013). user interests drive the notion of health information exchange. users can decide what content to share or ignore. this phenomenon is called consumer sovereignty in interactional communication processes (kline, dyer-witheford, & de peuter, 2003) and is a concern to healthcare professionals because such an outlook may lead to serious consequences in terms of health outcomes (househ, borycki, & kushniruk, 2014). for example, an assessment of 121 websites based on popular health topics disclosed that only 24% of the websites fulfilled more than two-thirds of published health guidelines for that health topic; 35% met between one and two-thirds, and 41% less than a third of the guidelines (kunst, groot, latthe, latthe, & khan, 2002). another study conducted on online support-seeking behaviour revealed that information provided through online support communities is often inaccurate due to the absence of a trained medical professional in the bulletin boards (malik & coulson, 2010). consequently, many health professionals expressed concerns about the usefulness of social media in a healthcare setting (smailhodzic, hooijsma, boonstra, & langley, 2016) and thus, professional application of social media in healthcare remains limited. 1.1 rationale of the research despite the advances being made by social media in healthcare, a lower rate of adoption by healthcare professionals points towards potential barriers which might prevent them from adopting social media in their practices (usher et al., 2014). in addition, users’ individual differences including age, gender and usage frequency also account for different barriers to social media adoption (braun, 2013; kamboj & rahman, 2016; salim, 2012). the extant social media acceptance studies in the healthcare context have not concentrated on the effects of these individual differences. therefore, the barrier-related factors and moderators of social media adoption should be considered simultaneously. thus, this research attempted to investigate the barrier-related factors and moderating effect of individual differences by answering the following the questions: (i) what are the barriers affecting healthcare professionals’ use of social media in the healthcare domain; and (ii) are the moderating effect of age, gender, and social media usage frequency significant in health professionals’ perceptions concerning the usefulness and ease of use of social media? a deeper understanding of the barriers that stand in the way of health professionals adopting social media technologies offer insights for healthcare authorities to develop support mechanisms, strategies, procedures, and content of social media usage. the remainder of this australasian journal of information systems khan et al. 2021, vol 25, research article effect of barrier related factors 3 paper is organised in five sections. first a review of the literature research is provided. then, the theoretical framework is proposed, and its key constructs are described. following this the research methodology is outlined. in the next section, the results are presented through the hypothesis testing. the next section discusses the results and explains theoretical and practical implications. finally, the last section contains the limitations and future research directions. 2 review of the literature there is a rich body of literature on the adoption of social media in information systems including the behavioural aspects of social media, information sharing mechanisms, marketing features, benefits, and risks of using social media in a variety of contexts (alihassan, nevo, & wade, 2015; m. chau & xu, 2012; kapoor et al., 2018). a systematic review of studies published between 1995 and 2012 found two major themes that drive the adoption of social media: a customer-centric organisational culture and social networking site (sns) know-how (abedin, abedin, khoei, & ghapanchi, 2013). despite this, research within the sns know-how in the healthcare domain remains scarce. according to maclure and stewart (2016), this is due to a lack of digital literacy among australian, canadian, and american healthcare professionals. indeed, a cross-sectional study involving 500 healthcare professionals found that 81.1% of these healthcare professionals exhibited inadequate computer knowledge (alwan, ayele, & tilahun, 2015). this finding was also documented in a recent study of 407 australian healthcare workers who reported anxiety about using information systems (kuek & hakkennes, 2020). importantly, researchers have discovered that in the healthcare profession, many healthcare providers tend to perceive patients who seek online information to be misinformed and anxious (moick & terlutter, 2012; van uden-kraan et al., 2010). consequently, many healthcare professionals believe that they should actively counter this indiscriminate spread of misinformation from social media (kouzy et al., 2020). while much research has been conducted on healthcare consumers’ adoption of social media to access medical information, current research in healthcare has tended to focus mainly on social media usage patterns (antheunis, tates, & nieboer, 2013), features (grajales iii, sheps, ho, novak-lauscher, & eysenbach, 2014), utility (panahi, watson, & partridge, 2016), and possible limitations (devine, 2017). empirical studies are lacking on how medical professionals adopt and utilise social media or information technology in their medical practices (archambault et al., 2012; lau, 2011; mcgowan et al., 2012). therefore, more research is required to understand healthcare professionals’ perspectives on social media. moreover, existing technology adoption frameworks are inadequate in explaining the impact of barrierrelated factors on health professionals’ intention to use social media. the following section clarifies the theoretical framework adopted in this study and discusses the technology acceptance model (tam) in detail. 2. 1 research model technology adoption research relies on various mainstream adoption models such as the theory of reasoned action (tra), technology acceptance model (tam) and unified theory of acceptance and use of technology (utaut) to explain the adoption and implementation of new technologies and systems. these traditional models basically focus on outcome expectations in terms of whether a certain technology or system improves job performance (ajibade, 2018). in contrast, alternative theories such as capability approach (ca) centre around the notion of non-utilitarianism. ca posits that expected outcomes should go beyond australasian journal of information systems khan et al. 2021, vol 25, research article effect of barrier related factors 4 job performance or performance more generally as specified in classic technology adoption models (nikou, agahari, keijzer-broers, & de reuver, 2019). it is explicitly assumed by ca that availability of resources, such as digital technologies, does not inevitably lead to good outcomes. rather, availability of resources and capabilities effectively give individuals the freedom to choose whether they want to realise these opportunities or not (talaei-khoei, lewis, khoei, ghapanchi, & vichitvanichphong, 2015). however, the ca approach was specifically constructed for the domain of wellbeing and independent living (robeyns, 2005), hence, its generalisability and explanatory power in the context of professional use social media is questionable. therefore, the tam emerges as a logical choice when conducting an analysis of social media adoption. according to the tam, there are two determinants of technology acceptance: perceived usefulness (pu) and perceived ease of use (peou) of a technology (davis, bagozzi, & warshaw, 1989). pu refers to the extent a user believes that using a particular system will improve how they perform or execute a certain task. peou refers to the extent a potential user believes that use of a particular system would be free of effort (davis et al., 1989). pu and peou influence users’ intention to use a specific technology. sequentially, intentions to use it will determine the actual use of the system. prior studies concerning information system (is) and information technology (it) adoption have tended to consider pu as the most important determinant of an individual’s decision to embrace a new technology (abdullah, ward, & ahmed, 2016; kucukusta, law, besbes, & legohérel, 2015). furthermore, users are equally likely to adopt a new technology if they perceive that it is easy to use and in using it will free up their time to do other things (morosan, 2012). therefore, it is important to elucidate the determinants of these two factors (pe & peou) in the context of social media. indeed, most social media-related studies in healthcare have relied heavily on either a generic survey of medical professionals or descriptive studies without a solid theoretical foundation (fisher & clayton, 2012; mcgowan et al., 2012). this research attempts to fill the theoretical lacuna in this field. recent research indicates that privacy, compatibility, information quality, lack of understanding of the benefits of social media, lack of organisational support and trust are some of most important obstacles that influence health professionals’ adoption of social media for health communication (cain, 2011; chou, hunt, beckjord, moser, & hesse, 2009; househ et al., 2014; moorhead et al., 2013). consequently, the proposed framework (see figure 1) contextualises tam by adding barrier-related factors: information quality, perceived threat, professional boundary, facilitating conditions, self-efficacy, and perceived trust from the perspective of health professionals. the proposed model explains how these barriers might influence healthcare professionals’ perceived usefulness (pu), perceived ease of use (peou) and their behavioural intention to use social media when communicating with clients. the proposed model also considers the moderating effects of age, gender, and social media usage frequency on the relationship between barrier-related factors and the notion of pu and peou. figure 1 below illustrates the proposed healthcare social media acceptance model. 2.2 hypothesis development 2.2.1 information quality information quality is a complex construct, and the term is often used interchangeably with data quality. in information system research, there is a predisposition to apply data quality to australasian journal of information systems khan et al. 2021, vol 25, research article effect of barrier related factors 5 address technical issues and information quality to refer to non-technical issues (madnick, wang, lee, & zhu, 2009). however, based on the is success model, information quality refers to web-based content that is appropriately customised, relevant, easy to understand, complete and secure during an online transaction (delone & mclean, 2003). this definition encompasses both technical and non-technical aspects of data quality. figure 1: healthcare social media acceptance model in healthcare, scholars have found that barriers to social media adoption included inaccurate, invalid, and unreliable health information (franko & tirrell, 2012; moorhead et al., 2013). the effect of information quality of healthcare professionals’ decision to use social media is yet to be determined and warrants further investigation. therefore, the following hypotheses are presented: h1: information quality significantly influences pu. h2: information quality significantly influences peou. 2.2.2 perceived trust trust minimises perceived threat and cost associated with disseminating private and sensitive information (gefen, karahanna, & straub, 2003; holden & karsh, 2010). previous research indicates that trust in online health information is a significant factor when engaging in online information searching and related activities (kwon, kye, park, oh, & park, 2015). similarly, self-disclosure of health-related issues is promoted by trust, a feeling of safety, and having an obligation to others (bansal & gefen, 2010). this research argues that a trusted social media platform is to be perceived as presenting useful, reliable, and accurate information that is in australasian journal of information systems khan et al. 2021, vol 25, research article effect of barrier related factors 6 sync with the best interests of health consumers. moreover, trust can reduce uncertainties associated with an online experience which is related to ease of use (pavlou, 2003). therefore, trust is a critical element of online health services usage because it facilitates a feeling of security and individuals sense that social media can be trusted when exchanging healthrelated information. based on tam, the following hypotheses are proposed: h3: perceived trust is associated with the pu. h4: perceived trust is associated with the peou. 2.2.3 privacy threat privacy and confidentiality concerns constitute a major deterrent of social media usage in the healthcare industry (al-muhtadi, shahzad, saleem, jameel, & orgun, 2019). from an individual perspective, a threat to privacy can be defined as a “sense of anxiety regarding one’s personal privacy” (lanier & saini, 2008). in the field of it and is, privacy threats have received much attention from researchers. prior studies have investigated the impact of privacy threats on consumers’ online purchase intention and behaviours (pavlou, liang, & xue, 2007) and social media usage behaviour (adhikari & panda, 2018). thus, privacy threat is considered to be an important construct in the technology acceptance decision (brandyberry, li, & lin, 2010). recent analyses regarding generic use of social networking sites (sns) have shown that privacy threat is negatively related with pu and peou (buettner, 2015). moreover, higher pu and peou potentially outweigh people’s privacy concerns (awad & krishnan, 2006). considering the growing popularity of social media in the healthcare sector, it will be interesting to see how privacy threats influence health professionals’ perceptions of usefulness (pu) and perceived ease of use (peou). based on past empirical evidence, the following hypotheses are postulated: h5: privacy threat is negatively associated with health professionals’ pu. h6: privacy threat is negatively associated with health professionals’ peou. 2.2.4 professional boundary professionalism has been a topic discussed at length and in many contexts in the medical education literature (chretien & kind, 2013). social networking has created an imbalance in doctor-patient relational boundary issues (gholami-kordkheili, wild, & strech, 2013). boundary crossing and boundary violation is now a common phenomenon between patient and their doctor, for instance, when a doctor discloses a patient’s private information in a publicly open forum. importantly, the digital world may blur the personal and professional lives of health professionals (farnan et al., 2013). how health professionals and consumers represent themselves in the social media realm has also become a critical issue. as patients become increasingly web savvy, they have found new ways to access their physicians, and the success of social media platforms in augmenting patients’ experience depends mostly on how such issues are addressed (chretien & kind, 2013). the notion of professional boundary is still to be analysed as an independent variable in the current technology adoption model. previous studies have shown that when a user perceives a new system or technology to be inconsistent with his/her current professional practice, he/she will tend to be more uncertain about the expected benefits of the innovation (al-qirim, 2007; conway, cao, & hong, 2011; rogers, 2003). however, the impact of perceived boundary on pu and peou is unclear in the healthcare social media domain. more empirical australasian journal of information systems khan et al. 2021, vol 25, research article effect of barrier related factors 7 investigation is needed to ascertain the relationship between perceived boundary and the notion of pu and peou. thus, the following hypotheses are proposed: h7: greater awareness of professional boundary positively influences the notion of pu. h8: greater awareness of professional boundary positively influences the notion of peou. 2.2.5 facilitating conditions in this research, facilitating conditions refer to the availability of organisational support to facilitate a new system or technology use (thompson, higgins, & howell, 1991). extant literature has identified various factors such as time constraints, limited provision of workplace support and limited training as barriers to social media adoption in healthcare settings (moorhead et al., 2013). for example, in a study of paediatric physicians’ attitudes in using social media platforms to improve asthma management in adolescents, it was reported that despite the benefits of social networking sites, the additional time required to monitor the sites was a significant barrier concerning the use of social media (martinasek et al., 2011). the study indicated that daily workplace schedules were already overwhelming, and it was challenging for physicians or their staff to find the time for these added responsibilities. according to triandis’ model of interpersonal behaviour, when barriers are present, behaviours are less likely to arise even with high levels of intention (triandis, 1979). thus, the absence or presence of facilitating conditions can trigger a positive or negative perception of usefulness (peñarroja, sánchez, gamero, orengo, & zornoza, 2019) and perceived ease of use (venkatesh, 2000). thus, the following hypotheses are proposed: h9: supporting conditions positively influence the pu. h10: supporting conditions positively influence the peou. 2.2.6 self-efficacy self-efficacy refers to one’s belief concerning his/her ability to undertake and complete a certain task using it/is systems (compeau & higgins, 1995). existing literature indicates that certain healthcare professionals’ reluctance to use social media is due to the lack of required skills and experience in using online tools (adzharuddin & ramly, 2015; eschenbrenner & nah, 2015). self-efficacious individuals behave more socially and are more likely to engage in online information exchange than less self-efficacious individuals and they are influenced by other people’s input (hocevar, flanagin, & metzger, 2014). earlier studies primarily addressed the internet user’s self-efficacy (hsu & chiu, 2004). past studies point towards a causal flow from computer self-efficacy to system-specific perceived ease of use (venkatesh & davis, 1996) and perceived usefulness (liaw & huang, 2013). it is important to note that studies on health professionals’ social media self-efficacy are relatively scarce. thus, further empirical investigation of the relationship between social media self-efficacy and pu/peou is warranted. based on the above argument, the following hypotheses are posited: h11: social media self-efficacy is significantly associated with health professionals’ pu of social media usage. h12: social media self-efficacy is significantly associated with health professionals’ peou of social media. australasian journal of information systems khan et al. 2021, vol 25, research article effect of barrier related factors 8 2.2.7 behavioural intention based on tam, pu and peou are expected to positively influence behavioural intention to use a technology (davis, 1989). prior technology acceptance studies have revealed that perceived usefulness and perceived ease of use are significant predictors of intention to use information technology (chang, hwang, hung, & li, 2007; holden & karsh, 2010; yarbrough & smith, 2007). for example, chang et al. (2007) have shown that performance expectancy, which is similar to perceived usefulness, influences physicians’ intention to use the clinical decision support system. it will be interesting to see the effect of pu and peou on health professionals’ behavioural intention to use social media for communication purposes. based on tam, the following hypotheses are proposed: h13: pu positively influences health professionals’ intention to use social media. h14: peou positively influences health professionals’ intention to use social media. a summary of the root constructs, their definitions and sources are provided in table1. construct/sources definition source information quality refers to completeness, ease of understanding, personalisation, relevance and security of web-based content. delone & mclean, 2003 perceived trust an expectation that others are trusted not to behave in an opportunistic manner by taking advantage of the situation. gefen et al., 2003 professional boundary when professionals refrain from communicating and sharing knowledge with outsiders due to political, ethical and professional reasons. farnan et al., 2009; farnan et al., 2013; teigland & wasko, 2003 privacy threat the threat that violates the right to determine what personal information about an individual should be known by others. westin & review, 1968 supporting conditions availability of required conditions to perform a specific task. venkatesh, thong, & xu, 2012 self-efficacy an individual’s beliefs about his/her abilities to perform at a level that influences events in their lives. bandura, 1977 perceived usefulness (pu) the extent to which a person believes that using the system will enhance his/her job performance. davis, 1989 perceived ease of use (peou) the extent to which a potential user expects that the target system will be free of unproductive effort. davis, 1989 behavioural intention refers to the degree a user intends to use one or more system features to complete a particular task. burton-jones & straub jr, 2006 table 1. root constructs, definitions and sources australasian journal of information systems khan et al. 2021, vol 25, research article effect of barrier related factors 9 2.3 moderating effect of age, gender and usage frequency prior studies highlighted the moderating effect of age, gender, and experience on the relationship between dependent and independent variables. for example, venkatesh and davis (2000) found that age, gender, and experience moderate the relationship between situation constraints (e.g., performance expectancy, effort expectancy, social influence, facilitating conditions) and intention to use technology. building on the work of venkatesh and davis (2000), the proposed model also considered the moderating effect of gender, age, and social media usage frequency on the relationship between barrier-related factors and the pu and peou. gender is one of the most widely acknowledged individual difference variables which denotes the difference between men and women in terms of their beliefs and attitudes regarding technology adoption (venkatesh, sykes, & zhang, 2011; zhang, guo, lai, guo, & li, 2014a), of which health technology is a part. technology adoption research indicates that gender is strongly associated with the adoption and use of various health technologies (hoque, bao, & sorwar, 2017; zhang, guo, lai, guo, & li, 2014b). according to zhou, jin, and fang (2014), pu is more salient for male users when they make decisions concerning technology usage, as it is consistent with their technology affordances and need structure. however, the role of gender has not been investigated in the social media domain, specifically in the context of health professionals. thus, the following hypothesis is proposed. h15: the gender effects in the healthcare social media acceptance model will be significant among health professionals. prior studies have shown that age is a vital factor for technological acceptance. older people are relatively reluctant to adopt new technologies compared to younger people (liébanacabanillas, sánchez-fernández, & muñoz-leiva, 2014). prior research has shown that different age groups have a specific moderating influence on whether mobile health services are adopted (zhao, ni, & zhou, 2018). this study propose that these assumptions are still true among medical professionals. therefore, consistent with prior studies, this article assume that younger medical professionals are more likely to use social media tools and will perceive a greater usefulness of social media in healthcare. this research considered the moderating role of age to examine the differences between the subgroups concerning social media usage behaviour (i.e., younger and older). thus, we propose the following hypothesis: h16: the age effects in the healthcare social media acceptance model will be significant among health professionals. the underlying principle behind the use of frequency as a moderator is anchored in the unified theory of acceptance and use of technology model (utaut). according to utaut, experience in using certain systems reduces anxiety and any problems associated with the new system will dissipate (venkatesh & davis, 2000). therefore, this research argues that usage frequency will reduce any problems associated with social media adoption. consequently, frequent users are more likely to accept social media. this research proposes that frequency of social media usage will yield significant differences in the factors that are proposed in the conceptual model: h17: the usage frequency effects in the healthcare social media acceptance model will be significant among healthcare professionals. australasian journal of information systems khan et al. 2021, vol 25, research article effect of barrier related factors 10 3 research methodology researchers have used qualitative research to explore the benefits and challenges in using social media from the perspective of healthcare professionals (panahi et al., 2016; rolls, hansen, jackson, & elliott, 2016). however, less research has been conducted using quantitative research. a google search returned only three studies that applied tam-based frameworks to understand predictors of social media adoption behaviours in healthcare (hanson, west, thackeray, barnes, & downey, 2014; jo, song, & kim, 2017; mcgowan et al., 2012). these studies primarily focused on the drivers of social media adoption and showed that pu and peou is the most critical predictor of behavioural intentions to use social media in healthcare. previous models have not considered the impact of barrier-related factors that might hinder healthcare professionals’ pu and peou regarding their use of social media in their practices. the proposed conceptual model exclusively focuses on providing a holistic theoretical framework that clearly elucidates the barriers of social media acceptance in healthcare. moreover, this research considers the moderating effect of age, gender and usage frequency which was not considered in prior studies in the australian context. logically, a quantitative survey was conducted to capture the effect of barriers related to health professionals’ pu and peou. 3.1 data collection a simple random sampling method was employed to select the respondents. a sampling frame was developed from publicly available online databases using health directory and doctors4you website. healthcare blogs and various health-related social media sites were also considered while developing the database. subsequently, an invitation via email and traditional post was sent to solicit health professionals’ participation. in total, 824 health professionals responded and agreed to participate in the research. the survey instrument consisted of the factors that influence pu, peou and behavioural intention of healthcare professionals in adopting social media use in their practices. a structured questionnaire was designed to incorporate operational measures of the variables so that it could be selfadministered. a survey monkey link with the questionnaire was sent to those people who agreed to participate in this survey. a total of 219 completed responses were received over a period of six weeks. the response rate was 27%. details of the construct measures and their operational indicators are provided in table 2. measure sources measures based on indicators standard estimate privacy threat (α = .72, ave = .52) measures based on bhattacherjee & hikmet, 2007; cheng, lam, & yeung, 2006; duyck et al., 2008 hard to maintain confidentiality might compromise privacy there are legal implications .864 .572 .628 information quality (α = .85, ave = .69) delone & mclean, 1992 health information is presented well information provided by social media sites is reliable easy to understand provides accurate information .691 .914 .879 .768 perceived trust (α = .85, ave = .68) gefen et al., 2003 i trust social media for health communication healthcare social media sites are trustworthy can be trusted to communicate health wellbeing .940 .833 .833 australasian journal of information systems khan et al. 2021, vol 25, research article effect of barrier related factors 11 table 2. construct measures 3.2 reliability and validity table 2 shows that all composite reliabilities exceeded the required criterion of 0.70 (hair, black, babin, anderson, & tatham, 2006). it also reveals that the average variance extracted (ave) for each construct exceeds the recommended limit of 0.50 (fornell & larcker, 1981). all constructs are considered satisfactory since the factor loadings and reliability are at an acceptable level. all constructs are retained for confirmatory analysis and all results strongly suggest the instrument’s convergent validity. during the assessment of multicollinearity, the correlation matrix (table 3) showed no multicollinearity between any two items was above 0.8; thus, bivariate high collinearity did not exist. construct correlations were compared with the square root of the ave, and it was found that all square roots of the aves were consistently higher than the correlations between constructs. in this way the discriminant validity of the constructs used in this research has been confirmed. 1 2 3 4 5 6 7 8 1. information quality 1 2. privacy threat .284** 1 3. perceived trust .499** .242** 1 professional boundary (α = .88, ave = .79) moore & benbasat, 199) i am aware that social media fits well with the way i like to interact with patients i understand that social media is compatible with my profession social media aligns with my professional integrity .834 .914 .915 facilitating conditions (α = .80, ave = .59) chau & hu, 2002 enables me to participate actively in health communities can reach other physicians/health professionals workplace environment encourages me to use social media .707 .737 .879 self-efficacy (α = .89, ave = .82) compeau & higgins, 1995 confident about using social media possesses required skills to use social media i have no difficulty in using social media i am interested in using social media to seek health information .962 .947 .960 .960 perceived usefulness (α = .86, ave = .69) davis et al. 1989 social media enables me to communicate better with patients improves quality of patient care increase knowledge sharing .851 .749 .659 perceived ease of use (α = .89, ave = .82) compeau & higgins, 1995) easy to operate social media easy to become skilful flexible i am interested in using social media to seek health information .849 .775 .724 .711 behavioural intention (α = .82, ave = .76) davis et al., 1989 use social media to share medical knowledge with patients. use social media to share medical knowledge with colleagues i seek specific information about health-related issues i use social media to promote myself as a health professional .838 .871 .874 .881 australasian journal of information systems khan et al. 2021, vol 25, research article effect of barrier related factors 12 4. professional boundary .467** .048* .499** 1 5. facilitating condition .149* .111 -.099 .011 1 6. self-efficacy .254** .225** .441** .330** -.160* 1 7. perceived usefulness .050 -.427** .248** .335** -.009 .203** 1 8 perceived ease of use .498** .340 .349** .483** -.078 .456** .023 9. behavioural intention .364** .099 .354** .591** -.122 .314** .123* 1 table 3. bivariate correlations 4 findings 4.1 the demographic profiles table 4 below summarises participating healthcare professionals’ demographic information. the completed survey includes 48% male participants and 52% female participants. regarding the participants’ age range, 47% were in the 18-49 age bracket, and 53% were over 50 years of age. regarding the participants’ experience in healthcare, 48% had 16-30 years, 14% had 11-15 years, 20% had 6-10 years, and 12% had 1-5 years. the majority of respondents are from new south wales (47%) followed by the australian capital territory (37%). meanwhile 11% of participants are from queensland and 5% are located in victoria. categories criteria frequency percent age 18-49 102 47% 50+ 114 53% gender male 106 48% female 113 52% clinical experience 1-5 years 27 12% 6-10 years 47 22% 11-15 years 31 14% 16-30 years 105 48% 30 years or more 9 4% location nsw 104 47% act 80 37% queensland 23 11% victoria 12 5% total 219 100.0 table 4. demographic characteristics 4.2 social media usage pattern table 5 summarises the patterns of social media used by the health professionals in this study. the results show that 63% have identified themselves as frequent users, while 37% are infrequent users. type frequency percent frequent user 138 63% australasian journal of information systems khan et al. 2021, vol 25, research article effect of barrier related factors 13 infrequent user 81 37% table 5. health professionals’ social media usage frequency table 6 depicts the patterns of social media use by health professionals. the result indicates that facebook, youtube, twitter, and linkedin are the most popular social media applications. facebook is the most frequently used platform as 47% regularly access it. youtube is the second most popular platform among health professionals (45%). this is followed by 25% of respondents frequently using twitter while 21% of respondents access linkedin. only 8% use blog sites and only 9% of respondents regularly access physician-only networks, and only 6% of respondents are users of wiki sites. these findings are comparable with the overall social media user statistics in australia. approximately 80% of people in australia are active on facebook and youtube. linkedin and twitter are ranked 5 and 8, respectively, in terms of the total number of active users (civicwebmedia, 2021). it is noticeable that health professionals’ participation in physician-only networking sites and blogging platforms is particularly low. however, this is comparable with the usher’s (2011) study which showed that only 10% (n=231) of health professionals in australia regularly use blogs for professional or personal use. type social media use facebook 102/47% twitter 55/25% instagram 9/4% youtube 98/45% blogs 17/8% linkedin 47/21.4% wiki 14/6.4% physicians-only networks 19/8.8% table 6. health professionals’ social media usage pattern. 4.3 results of confirmatory factor analysis (cfa) and hypotheses testing the individual measurement model fit was tested for all dependent and independent variables included in the conceptual framework. the measurement model was tested with 31 items during the confirmation of measurement stage. the results of final cfa demonstrate good model fit (χ2 = 826.609; df, 395, χ2/df = 2.093, rmsea = .06, ifi = .918, tli = .902, and cfi = .917). following the assessment of the measurement model, the subsequent phase assessed the structure equation model (sem) to test the hypothesised relationships specified in the final conceptual framework. the results of sem specified the relationships between constructs and measured variables based on the theoretical or conceptual framework (hair et al., 2006). the sem analysis indicated a satisfactory model fit (χ2 = 864.344; df, 402, χ2/df = 2.150, rmsea = .06, ifi = .912, tli = .897, and cfi = .912). the structural model revealed that out of fourteen (14) hypotheses, 10 were supported were supported by the relationships at the p< .01 and p< .05 significance level (see table 7). australasian journal of information systems khan et al. 2021, vol 25, research article effect of barrier related factors 14 the findings (see table 7) reveal that the six proposed barrier-related constructs had significant effects on pu or peou. the findings show that the relationships between information quality and peou (β=.838, t=4.502, p<.01), perceived trust and pu (β=.326, t= 3.368, p< .01), privacy threat and pu (β=-.537, t= -6.126, p< .01), professional boundary and pu (β=.204, t= 1.964, p<.05), professional boundary and peou (β=.718, t= 6.122, p<.01), facilitating conditions and pu (β=.484, t= 1.997, p<.05), self-efficacy and pu (β=.210, t= 3.200, p<.01), selfefficacy and peou (β=.275, t= 3.937, p<.01), pu and behavioural intention (β=.142, t= 2.241, p<.05), and peou and behavioural intention (β=.377, t=7.999, p<.01) were statistically significant, confirming support for h2, h3, h5, h7, h8, h9, h11, h12, h13 and h14 (see table 7). however, the relationship between information quality and pu (β=-.170, t=-1.029, p>.05) was statistically insignificant (h1). the relationships between trust and peou (β=-.399, t=3.795, p>.01), facilitating conditions and peou (β=-.791, t=-2.969, p>.01) were significant and negative. moreover, the relationship between privacy threat and peou (β=.294, t=4.007, p>.01) was statistically significant and positive. thus, h1, h4, h6 and h10 were not supported. table 7 below provides an overview of the results of the hypotheses testing performed in sem. the results for h15, h16, and h17 will be discussed in section 5.2 hypothesis path coefficient (β) s.e. t p comment h1: information quality  perceived usefulness -.170 .165 -1.029 .303 not supported h2: information quality  perceived ease of use .838 .186 4.502 *** supported h3: perceived trust  perceived usefulness .326 .097 3.368 *** supported h4: perceived trust  perceived ease of use -.399 .105 -3.795 *** negative relationship h5: privacy threat  perceived usefulness -.537 .088 -6.126 *** supported h6: privacy threat  perceived ease of use .294 .073 4.007 *** positive relationship h7: professional boundary  perceived usefulness .204 .104 1.964 .049 supported h8: professional boundary  perceived ease of use .718 .117 6.122 *** supported h9: facilitating conditions  perceived usefulness .484 .243 1.997 .046 supported h10: facilitating conditions  perceived ease of use -.791 .266 2.969 -.003 negative relationship h11: self-efficacy  perceived usefulness .210 .066 3.200 .001 supported h12: self-efficacy  perceived ease of use .275 .070 3.937 *** supported h13: perceived usefulness  behavioural intention .142 .063 2.241 .025 supported h14: perceived ease of use  behavioural intention .377 .047 7.999 *** supported table 7. results with reference to the hypotheses australasian journal of information systems khan et al. 2021, vol 25, research article effect of barrier related factors 15 5. discussion the first research question of this study was: what are the barriers affecting the healthcare professionals’ use of social media? the extant literature pointed toward six (6) barrier-related factors impacting the notion of pu and peou. in this section, findings with respect to the first research question have been discussed. subsequently, the answer to the second research question concerning the moderating effect of age, gender and social media usage frequency is discussed. finally, theoretical and practical implications are presented. 5.1 impact of barrier-related factors 5.1.1 information quality the relationship between information quality and pu was insignificant. this finding contradicted prior research which has found that perceived information quality positively influences the notion of pu in digital information exchange settings (machdar, 2019; saeed & abdinnour-helm, 2008). results indicated that information quality may not be a critical determinant of health professionals’ perception of usefulness. it can be argued that information quality is a more important determinant for health consumers or patients than the healthcare professionals. this is because patients tend to rely heavily on social media based medical information than the healthcare professionals (antheunis, tates, & nieboer, 2013). however, the relationship between peou and information quality was particularly strong, which implies that higher quality social media content reduces the effort required to use social media in healthcare settings. 5.1.2 perceived trust this study also set out to examine the impact of perceived trust on pu and peou. the results demonstrated that perceived trust yielded a significant influence on pu. this outcome was consistent with previous studies (lin, zhang, song, & omori, 2016; mou, shin, & cohen, 2017; van der velden & el emam, 2013) and confirms that trust minimises uncertainties associated with disseminating private and sensitive health information. in other words, trust is a critical element of online health services (bansal & gefen, 2010). if online participants fail to trust the health information provided to them through social media channels, their uncertainty would be high which would impact their mental representation regarding the usefulness of social media. moreover, health professionals are likely to perceive the potential for benefit only if they trust social media to disseminate information correctly. however, the relationship between perceived trust and peou was found to be negative. the result contradicts previous findings which showed there is a significant association between trust and peou in predicting consumers’ use of social media for transactions (hansen, saridakis, & benson, 2018). thus, perceived trust may not be a strong predictor of peou in the healthcare context. 5.1.3 privacy threat in terms of h5, results confirm that privacy threat negatively influences pu and positively influence peou. thus, privacy threat reduces health professionals’ perceptions of usefulness in social media-based healthcare settings. this finding is consistent with previous research finding that privacy threat is one of the fundamental barriers to health professionals’ use of social media which has a significant impact on their perception of usefulness (moorhead et al., 2013). however, privacy threat was found to be positively related to peou. prior studies have shown that peou typically serves as a risk reduction factor and lessens privacy-related risks (featherman, miyazaki, & sprott, 2010; pavlou & gefen, 2004). however, this study australasian journal of information systems khan et al. 2021, vol 25, research article effect of barrier related factors 16 contradicts prior findings and claims that high privacy threat does not necessarily point to low peou. consequently, privacy threats do not negatively influence peou. 5.1.4 professional boundary this study also examined the relationship between professional boundary and pu (h7) and peou (h8). the finding confirms there is a significant association between professional boundary and pu and peou. this finding suggests that awareness of professional boundaries results in a favourable perception concerning pu and peou in healthcare social media settings. the findings lead to the insight that social media use by physicians for clinical and non-clinical purposes poses some dilemmas in terms of patients’ and health professionals’ interactions. specifically, any inappropriate use of social media can result in harm to patients and the profession, including breaches of confidentiality, defamation of colleagues or employers, and violation of doctor-patient boundaries (kotsenas et al., 2018b). consequently, lack of knowledge about the professional boundary can greatly influence health professionals’ social media usage decision. the concept of professional boundaries in the healthcare social media context is not widely known or understood, and the role of professional boundaries as an independent variable has not been addressed in the literature. to the best of the authors’ knowledge, this is the first study to include professional boundary as a predictor. 5.1.5 facilitating conditions regarding h9, facilitating conditions had a significant influence on pu. this result is consistent with prior studies in different contexts (for example, learning) which reported a positive effect of facilitating conditions on pu (teo, 2011). this finding implies that when endusers feel supported in their technological adoption, they tend to perceive that the new system is useful to them. thus, the finding of the study affirm that organisational support would encourage healthcare professionals to use social media for professional reasons. however, the relationship between facilitating conditions and peou was significant (h10) but negative. one possible explanation could be that the ubiquitous nature of social media which allow individuals to freely interact with others and the role of facilitating conditions are negligible. thus, facilitating conditions may not be a strong predictor of peou. 5.1.6 self-efficacy the result indicated a significant relationship between self-efficacy and pu (h11) and peou (h12). previous studies have found mixed results concerning the effect of self-efficacy on pu. some studies reported a positive relationship (al-ammary, al-sherooqi, & al-sherooqi, 2014; al-mushasha, 2013), whereas others reported the lack of a significant relationship between self-efficacy and pu & peou (lee, hsieh, & chen, 2013). however, this research confirms that if health professionals are efficacious in their use of social media, they are more likely perceive it as useful and will find it easy to use in their work. 5.1.7 behavioural intention finally, both pu and peou significantly influence the notion of behavioural intention to use social media for health-related communication. the finding confirms the idea that pu and peou are key predictors of health professionals’ intention to use social media in healthcare settings. australasian journal of information systems khan et al. 2021, vol 25, research article effect of barrier related factors 17 5.2 moderating effect of age, gender and usage frequency drawing on the recommendations of steenkamp and baumgartner (2000), measurement invariance tests were conducted to verify the statistical concern and validate the proposed theoretical constructs and their suitability for different groups such as gender (male-female), age (young adults – mature adults), and usage frequency (frequent user non-frequent user). furthermore, structural invariance including χ2 difference tests have been conducted to verify the moderating effect of these variables. gender moderation male-female χ2 value df χ2 difference rmsea ifi tli cfi χ2/df model 1: structural invariance (constrained – unconstrained) 1795.92 – 1709.51 842 808 cmin=86.41 df=34 p=.001 0.073 0.847 0.820 0.843 2.12 usage frequency moderation: frequent and non-frequent model 2: structural invariance (constrained – unconstrained) 1308.64 – 1276.39 841 806 cmin=32.26 df=16 p=.404 0.066 0.868 0.842 0.863 1.58 age moderation: young-old user model 3: structural invariance (constrained – unconstrained) 1410.42 – 1377.50 843 810 cmin=32.92 df=33 p=.471 0.057 0.895 0.876 0.892 1.70 table 8. moderating effect: structural invariance as evidenced in table 8, model 1 demonstrates a reasonable fit and provides evidence there is a significant difference between gender group samples, confirming h15. however, the other two groups usage frequency and age of consumers – did not demonstrate significant differences. therefore, h16 and h17 were not supported. thus, the only significant difference is found in gender at the .001 level. this implies that male and female professionals’ observations regarding predictors and their influence on criterion variables are different. path by path analysis reveals that the effect of pu and peou on behavioural intention to use social media is significant among women health professionals, whereas pu does not (p>.05) affect behavioural intention for men health professionals. these findings challenge the traditional perspective that perceived usefulness plays a more significant role for men than women when they are making decisions about technology usage (zhou et al., 2014). another interesting observation is that the influence of information quality on pu and peou is insignificant (p>0.05) for male users. however, the effect is significant for female healthcare workers. as expected, professional boundary awareness significantly influences the notion of pu and peou for both male and female healthcare workers. another notable finding is that the influence of privacy threat also significantly differs among male and female healthcare professionals. for female healthcare professionals, privacy threat significantly influences pu and peou (p<0.05). however, the effect is insignificant for their male counterparts in both cases. australasian journal of information systems khan et al. 2021, vol 25, research article effect of barrier related factors 18 5.3 theoretical implications this research questions the conventional acumen that behavioural intention to use social media is directly linked with some standard conventional antecedents. to address this contention, this study developed and tested a research framework by integrating barrierrelated factors as a novel predictor affecting usage of social media by medical professionals. factors such as professional boundary, perceived trust, privacy threat, facilitating conditions, and self-efficacy are creating cynicism in the minds of medical professionals and causing slow rates of social media usage in australia’s healthcare system. this is likely to be one of the few quantitative research studies on healthcare social media that takes into account the perceptions of australian health professionals on social media for communicating on medical issues. the construct measurement scales derived from existing literature are deemed reliable given the solid composite reliabilities of the constructs. it confirms these instruments can be safely applied in future research within the social media healthcare domain. the current state of research regarding the adoption of social media in healthcare seems to lack insights from the perspectives of potential users, their perceptions, and behaviours (smailhodzic et al., 2016). therefore, the primary contribution of this study is to establish the significance of a number of external variables in an extended tam-based approach, one that explains the barrier-related factors affecting social media usage in healthcare. in this context the validated research framework can serve to evaluate the barriers of social media adoption. generalisation of pu and peou in the healthcare domain is a complex issue, considering the dynamic nature of social media that makes it difficult to generalise. however, this research has uncovered the impact of barrier-related factors on health professionals’ behavioural intention to use social media. although generalisation is limited to the study participants, this research has, at least, structurally modelled the impacts of the possible barriers. it has taken into account the complexities associated with the moderating variables affecting health professionals’ perception and use of social media in australia. consequently, this research advances the previous research on the topic and contributes to this rapidly evolving field of study. to sum up, the novelty of this research enterprise lies in modelling several untested factors impacting on the usage of social media in healthcare in an australian setting. 5.4 practical implications the findings of this study can be considered timely given the current context of the covid19 pandemic and how it is greatly affecting people’s health or their perceptions of how it could do so. medical professionals in australia and beyond are faced with new challenges in providing effective services to clients in a more efficient way by utilising various online platforms. more specifically, the findings of this research have become more critical as the coronavirus has triggered a widespread online search for information, but has resulted instead in the rapid dissemination of misinformation, lies, unverified assumptions, etc. the impact of health misinformation can be devastating as the public often rely on such information while making important health decisions (cuan-baltazar, muñoz-perez, robledo-vega, pérezzepeda, & soto-vega, 2020). consequently, the role of health professionals has become even more critical, as healthcare workers can play a significant role in curbing the spread of misinformation that is so prevalent on social media sites. by understanding the health professionals’ attitude towards social media usage, government agencies can formulate appropriate policies and strategies to involve health professionals in the quest to stop the spread of misinformation. for example, health authorities around the world can engage australasian journal of information systems khan et al. 2021, vol 25, research article effect of barrier related factors 19 medical professionals to promote required actions relating to hygiene and social distancing, clarify misconceptions and disseminate correct and reliable health information based on sound scientific principles using social media. the study findings may provide an opportunity for healthcare providers and policy-makers to understand users’ perceptions of the potential challenges of using social media. the barriers noted in this research call for a clear social media usage policy, specifically one where health professionals are armed with a guideline to maximise the potential benefits of social media usage and overcome problems. also, the findings point to the need to educate, inform, and engage healthcare professionals and patients in reaping the benefits of using social media more effectively to achieve the above-stated ends (keir, bamat, patel, elkhateeb, & roland, 2019). the growing importance of social media in the current pandemic afflicting the world is evident, especially given that healthcare professionals and their clients can in many cases no longer meet in person. furthermore, it is worth mentioning here that the introduction of social media in healthcare is resulting in organisational changes. healthcare organisations can draw on the best practices around the world and work on the design and implementation of social media platforms and generate high levels of business for their services. finally, since the challenges posed by the pandemic are putting pressure on healthcare services to introduce innovative devices able to handle the diverse needs of patients, organisations need to respond in a timely way. 6. limitations and future research although the findings of this study point toward a sound understanding of the factors affecting the intention to use social media by healthcare professionals, several limitations need to be appreciated. first, this research did not seek to investigate the antecedents of the barriers of social media usage from the perspective of any specific group of health professionals. instead, it investigates the barrier-related factors from a holistic perspective which may limit the findings’ generalisability. according to their profiles the health professionals comprised general practitioners (gps), medical specialists, physiotherapists, surgeons, and pharmacists. future research should investigate the perspective of a specific group of healthcare professionals which may strengthen the applicability of the proposed theoretical framework that seeks to explain their social media adoption behaviours. second, the model variables explain 54% of behavioural intention by health professionals to use social media. thus, a considerable percentage of the variables remain unexplained, warranting the need for further research to explain user beliefs and attitude toward use. examples of these factors include social influence (venkatesh, 2003), perceived cost (gupta, joshi, & agarwal, 2012) and perceived risk (kamal, shafiq, & kakria, 2020). third, the moderating effects of other variables such as clinical experience and location (urban vs rural) may be a subject for further analysis. fourth, the issue of generalisability is a widely revealed limitation in the majority of research on technology acceptance. this study has been conducted in australia and the findings may not be applicable to other contexts or countries. for example, the australian and us healthcare systems are vastly different with varied models of care and practice (blendon et al., 2002). the united states healthcare system is a highly privatised and profit-oriented one, characterised by efficient service delivery and high-quality resources depending on the client’s ability to pay for them and low or non-existent government subsidies. conversely, in australia public healthcare is freely accessible to all citizens and australasian journal of information systems khan et al. 2021, vol 25, research article effect of barrier related factors 20 permanent residents (o'brien, 2017). the apparent differences between these two healthcare systems are also characterised by more frequent professional use of social media by us healthcare workers compared to their australian counterparts (mcgowan et al., 2012; usher, 2012). therefore, caution should be exercised when applying the findings of this research to other contexts. more robust studies including a cross-country comparison would enhance generalisation of the findings. finally, this study will benefit from a mixed method approach because both presence of qualitative and quantitative data may complement each other in this type of research and provide better generalisability. references abdullah, f., ward, r., & ahmed, e. (2016). investigating the influence of the most commonly used external variables of tam on students’ perceived ease of use (peou) and perceived usefulness (pu) of e-portfolios. computers in human behavior, 63, 75-90. doi: https://doi.org/10.1016/j.chb.2016.05.014. abedin, b., abedin, b., khoei, t. t., & ghapanchi, a. r. (2013). a review of critical factors for communicating with customers on social networking sites. the international technology management review, 3(4), 208-218. doi: https://dx.doi.org/10.2991/ itmr.2013.3.4.1 abedin, b., erfani, s., & blount, y. (2017). social media adoption framework for aged care service providers in australia. in 2017 international conference on research and innovation in information systems (icriis) (pp. 1-6). ieee. doi: https://doi.org/10.1109/ icriis.2017.8002485 adhikari, k., & panda, r. k. (2018). users' information privacy concerns and privacy protection behaviors in social networks. journal of global marketing, 31(2), 96-110. doi: https://doi.org/10.1080/08911762.2017.1412552 adzharuddin, n. a., & ramly, n. m. (2015). nourishing healthcare information over facebook. procedia-social and behavioral sciences, 172, 383-389. doi: https://doi.org/10.1016/j.sbspro.2015.01.384 al-ammary, j. h., al-sherooqi, a. k., & al-sherooqi, h. k. (2014). the acceptance of social networking as a learning tools at university of bahrain. international journal of information education technology, 4(2), 208-214. al-muhtadi, j., shahzad, b., saleem, k., jameel, w., & orgun, m. (2019). cybersecurity and privacy issues for socially integrated mobile healthcare applications operating in a multicloud environment. health informatics journal, 25(2), 315-329. doi: https://doi.org/10.1177/1460458217706184 al-mushasha, n. f. a. (2013, may). determinants of e-learning acceptance in higher education environment based on extended technology acceptance model. in 2013 fourth international conference on e-learning" best practices in management, design and development of e-courses: standards of excellence and creativity" (pp. 261-266). ieee. doi: 10.1109/econf.2013.50 al-qirim, n. (2007). championing telemedicine adoption and utilization in healthcare organizations in new zealand. international journal of medical informatics, 76(1), 4254. doi: https://doi.org/10.1016/j.ijmedinf.2006.02.001 australasian journal of information systems khan et al. 2021, vol 25, research article effect of barrier related factors 21 ali-hassan, h., nevo, d., & wade, m. (2015). linking dimensions of social media use to job performance: the role of social capital. the journal of strategic information systems, 24(2), 65-89. doi: https://doi.org/10.1016/j.jsis.2015.03.001 alwan, k., ayele, t. a., & tilahun, b. (2015). knowledge and utilization of computers among health professionals in a developing country: a cross-sectional study. jmir human factors, 2(1), e4. doi:10.2196/humanfactors.4184 antheunis, m. l., tates, k., & nieboer, t. e. (2013). patients’ and health professionals’ use of social media in health care: motives, barriers and expectations. patient education and counseling, 92(3), 426-431. doi: https://doi.org/10.1016/j.pec.2013.06.020 archambault, p. m., bilodeau, a., gagnon, m.-p., aubin, k., lavoie, a., lapointe, j., . . . légaré, f. (2012). health care professionals’ beliefs about using wiki-based reminders to promote best practices in trauma care. journal of medical internet research, 14(2). doi:10.2196/jmir.1983 awad, n. f., & krishnan, m. s. (2006). the personalization privacy paradox: an empirical evaluation of information transparency and the willingness to be profiled online for personalization. mis quarterly, 30, 13-28. doi: https://doi.org/10.2307/25148715. bandura, a. (1977). self-efficacy: toward a unifying theory of behavioral change. psychological review, 84(2), 191-215. doi: https://doi.org/10.1037/0033-295x.84.2.191 bansal, g., & gefen, d. (2010). the impact of personal dispositions on information sensitivity, privacy concern and trust in disclosing health information online. decision support systems, 49(2), 138-150. doi: https://doi.org/10.1016/j.dss.2010.01.010 barnes, s. s., kaul, v., & kudchadkar, s. r. (2019). social media engagement and the critical care medicine community. journal of intensive care medicine, 34(3), 175-182. doi: https://doi.org/10.1177/0885066618769599 bhattacherjee, a., & hikmet, n. (2007). physicians' resistance toward healthcare information technology: a theoretical model and empirical test. european journal of information systems, 16(6), 725-737. doi: https://doi.org/10.1057/palgrave.ejis.3000717 blendon, r. j., schoen, c., desroches, c. m., osborn, r., scoles, k. l., & zapert, k. (2002). inequities in health care: a five-country survey. health affairs, 21(3), 182-191. doi: https://doi.org/10.1377/hlthaff.21.3.182 brandyberry, a. a., li, x., & lin, l. (2010, august). determinants of perceived usefulness and perceived ease of use in individual adoption of social network sites. in amcis (p. 544). url: https://aisel.aisnet.org/amcis2010/544/ braun, m. t. (2013). obstacles to social networking website use among older adults. computers in human behavior, 29(3), 673-680. doi: https://doi.org/10.1016/ j.chb.2012.12.004 buettner, r. (2015, january). analyzing the problem of employee internal social network site avoidance: are users resistant due to their privacy concerns?. in 2015 48th hawaii international conference on system sciences (pp. 1819-1828). ieee. doi: 10.1109/hicss.2015.220 australasian journal of information systems khan et al. 2021, vol 25, research article effect of barrier related factors 22 burton-jones, a., & straub jr, d. w. (2006). reconceptualizing system usage: an approach and empirical test. information systems research, 17(3), 228-246. doi: https://doi.org/10.1287/isre.1060.0096 cain, j. (2011). social media in health care: the case for organizational policy and employee education. american journal of health-system pharmacy, 68(11), 1036-1040. doi: https://doi.org/10.2146/ajhp100589 chang, i.-c., hwang, h.-g., hung, w.-f., & li, y.-c. (2007). physicians’ acceptance of pharmacokinetics-based clinical decision support systems. expert systems with applications, 33(2), 296-303. doi: https://doi.org/10.1016/j.eswa.2006.05.001 chau, m., & xu, j. (2012). business intelligence in blogs: understanding consumer interactions and communities. mis quarterly, 36(4), 1189-1216. doi: https://doi.org/10.2307/41703504 chau, p. y., & hu, p. j.-h. (2002). investigating healthcare professionals’ decisions to accept telemedicine technology: an empirical test of competing theories. information & management, 39(4), 297-311. doi: https://doi.org/10.1016/s0378-7206(01)00098-2 cheng, t. e., lam, d. y., & yeung, a. c. (2006). adoption of internet banking: an empirical study in hong kong. decision support systems, 42(3), 1558-1572. doi: https://doi.org/10.1016/j.dss.2006.01.002 chou, w.-y. s., hunt, y. m., beckjord, e. b., moser, r. p., & hesse, b. w. (2009). social media use in the united states: implications for health communication. journal of medical internet research, 11(4). doi:10.2196/jmir.1249 chretien, k. c., & kind, t. (2013). social media and clinical care: ethical, professional, and social implications. circulation, 127(13), 1413-1421. doi: https://doi.org/10.1161/ circulationaha.112.128017 civicwebmedia. (2021, 18/01/2021). australia’s most popular social media sites 2021. retrieved from https://www.civicwebmedia.com.au/australias-most-popular-socialmedia-sites/ compeau, d. r., & higgins, c. a. (1995). computer self-efficacy: development of a measure and initial test. mis quarterly, 19(2), 189-211. doi: https://doi.org/10.2307/249688 conway, m., cao, y., & hong, p. (2011). antecedents and consequences of social media utilization in college teaching: a proposed model with mixed-methods investigation. on the horizon, 19(4), 297-306. doi: https://doi.org/10.1108/10748121111179420 cuan-baltazar, j. y., muñoz-perez, m. j., robledo-vega, c., pérez-zepeda, m. f., & soto-vega, e. (2020). misinformation of covid-19 on the internet: infodemiology study. jmir public health surveillance, 6(2), e18444. doi: https://doi.org/10.2196/18444 davis, f. d. (1989). perceived usefulness, perceived ease of use, and user acceptance of information technology. mis quarterly, 13(3), 319-340. doi: https://doi.org/10.2307/ 249008 davis, f. d., bagozzi, r. p., & warshaw, p. r. (1989). user acceptance of computer technology: a comparison of two theoretical models. management science, 35(8), 982-1003. doi: https://doi.org/10.1287/mnsc.35.8.982 australasian journal of information systems khan et al. 2021, vol 25, research article effect of barrier related factors 23 delone, w. h., & mclean, e. r. (1992). information systems success: the quest for the dependent variable. information systems research, 3(1), 60-95. doi: https://doi.org/10.1287/isre.3.1.60 delone, w. h., & mclean, e. r. (2003). the delone and mclean model of information systems success: a ten-year update. journal of management information systems, 19(4), 9-30. doi: https://doi.org/10.1080/07421222.2003.11045748 denecke, k., bamidis, p., bond, c., gabarron, e., househ, m., lau, a., . . . hansen, m. (2015). ethical issues of social media usage in healthcare. yearbook of medical informatics, 10(1), 137. doi: 10.15265/iy-2015-001 devine, p. j. (2017). social media and health care. using social media to build library communities: a lita guide, 65. duyck, p., pynoo, b., devolder, p., voet, t., adang, l., & vercruysse, j. (2008). user acceptance of a picture archiving and communication system. methods of information in medicine, 47(02), 149-156. doi: 10.3414/me0477 erfani, s. s., abedin, b., & blount, y. (2017). the effect of social network site use on the psychological well-being of cancer patients. journal of the association for information science, 68(5), 1308-1322. doi: https://doi.org/10.1002/asi.23702 farnan, j. m., paro, j. a., higa, j. t., reddy, s. t., humphrey, h. j., & arora, v. m. (2009). commentary: the relationship status of digital media and professionalism: it's complicated. academic medicine, 84(11), 1479-1481. doi: 10.1097/ acm .0b013e3181bb17af farnan, j. m., sulmasy, l. s., worster, b. k., chaudhry, h. j., rhyne, j. a., & arora, v. m. (2013). online medical professionalism: patient and public relationships: policy statement from the american college of physicians and the federation of state medical boards. annals of internal medicine, 158(8), 620-627. doi: https://doi.org/10.7326/00034819-158-8-201304160-00100 featherman, m. s., miyazaki, a. d., & sprott, d. e. (2010). reducing online privacy risk to facilitate e-service adoption: the influence of perceived ease of use and corporate credibility. journal of services marketing, 24(3), 219-229. doi: https://doi.org/10.1108/08876041011040622 fernández-luque, l., & bau, t. (2015). health and social media: perfect storm of information. healthcare informatics research, 21(2), 67-73. doi: 10.4258/hir.2015.21.2.67 fisher, j., & clayton, m. (2012). who gives a tweet: assessing patients’ interest in the use of social media for health care? worldviews on evidence-based nursing, 9(2), 100-108. doi: https://doi.org/10.1111/j.1741-6787.2012.00243.x fornell, c., & larcker, d. f. (1981). structural equation models with unobservable variables and measurement error: algebra and statistics. journal of marketing research, 18(3), 382-388. doi: 10.1177/002224378101800313 franko, o. i., & tirrell, t. f. (2012). smartphone app use among medical providers in acgme training programs. journal of medical systems, 36(5), 3135-3139. doi: https://doi.org/10.1007/s10916-011-9798-7 australasian journal of information systems khan et al. 2021, vol 25, research article effect of barrier related factors 24 gagnon, k., & sabus, c. (2015). professionalism in a digital age: opportunities and considerations for using social media in health care. physical therapy, 95(3), 406-414. doi: https://doi.org/10.2522/ptj.20130227 gefen, d., karahanna, e., & straub, d. w. (2003). trust and tam in online shopping: an integrated model. mis quarterly, 27(1), 51-90. doi: https://doi.org/10.2307/30036519 gholami-kordkheili, f., wild, v., & strech, d. (2013). the impact of social media on medical professionalism: a systematic qualitative review of challenges and opportunities. journal of medical internet research, 15(8), e184. doi:10.2196/jmir.2708 grajales iii, f. j., sheps, s., ho, k., novak-lauscher, h., & eysenbach, g. (2014). social media: a review and tutorial of applications in medicine and health care. journal of medical internet research, 16(2). doi:10.2196/jmir.2912 gupta, b., joshi, s., & agarwal, m. (2012). the effect of expected benefit and perceived cost on employees’ knowledge sharing behavior: a study of it employees in india. organizations markets in emerging economies, 3(1), 8-19. doi:10.15388/omee.2012.3.1.14272. hair, j. f., black, w. c., babin, b. j., anderson, r. e., & tatham, r. l. (2006). multivariate data analysis, 6th ed. uppersaddle river: pearson prentice hall. hansen, j. m., saridakis, g., & benson, v. (2018). risk, trust, and the interaction of perceived ease of use and behavioral control in predicting consumers’ use of social media for transactions. computers in human behavior, 80, 197-206. doi: https://doi.org/10.1016/j.chb.2017.11.010 hanson, c. l., west, j., thackeray, r., barnes, m. d., & downey, j. (2014). understanding and predicting social media use among community health center patients: a cross-sectional survey. journal of medical internet research, 16(11), e270. doi:10.2196/jmir.3373 hocevar, k. p., flanagin, a. j., & metzger, m. j. (2014). social media self-efficacy and information evaluation online. computers in human behavior, 39, 254-262. https://doi.org/10.1016/j.chb.2014.07.020 holden, r. j., & karsh, b.-t. (2010). the technology acceptance model: its past and its future in health care. journal of biomedical informatics, 43(1), 159-172. doi: https://doi.org/10.1016/j.jbi.2009.07.002 hoque, m. r., bao, y., & sorwar, g. (2017). investigating factors influencing the adoption of ehealth in developing countries: a patient’s perspective. informatics for health social care, 42(1), 1-17. https://doi.org/10.3109/17538157.2015.1075541 househ, m., borycki, e., & kushniruk, a. (2014). empowering patients through social media: the benefits and challenges. health informatics journal, 20(1), 50-58. doi: https://doi.org/10.1177/1460458213476969 hsu, m.-h., & chiu, c.-m. (2004). internet self-efficacy and electronic service acceptance. decision support systems, 38(3), 369-381. doi: https://doi.org/10.1016/j.dss.2003.08.001 jo, h. s., song, t.-m., & kim, b. g. (2017). analysis of the factors affecting consumer acceptance of accredited online health information. journal of korean medical science, 32(11), 1757-1763. doi: https://doi.org/10.3346/ jkms.2017.32.11.1757 australasian journal of information systems khan et al. 2021, vol 25, research article effect of barrier related factors 25 kamal, s. a., shafiq, m., & kakria, p. (2020). investigating acceptance of telemedicine services through an extended technology acceptance model (tam). technology in society, 60, 101212. doi: https://doi.org/10.1016/j.techsoc.2019.101212 kamboj, s., & rahman, z. (2016). the influence of user participation in social media-based brand communities on brand loyalty: age and gender as moderators. journal of brand management, 23(6), 679-700. doi: https://doi.org/10.1057/s41262-016-0002-8 kapoor, k. k., tamilmani, k., rana, n. p., patil, p., dwivedi, y. k., & nerur, s. (2018). advances in social media research: past, present and future. information systems frontiers, 20(3), 531-558. doi: https://doi.org/10.1007/s10796017-9810-y keir, a., bamat, n., patel, r. m., elkhateeb, o., & roland, d. (2019). utilising social media to educate and inform healthcare professionals, policy-makers and the broader community in evidence-based healthcare. bmj evidence-based medicine, 24(3), 87-89. doi: http://dx.doi.org/10.1136/bmjebm-2018-111016 kline, s., dyer-witheford, n., & de peuter, g. (2003). digital play: the interaction of technology, culture, and marketing. mcgill-queen's university press. kotsenas, a. l., arce, m., aase, l., timimi, f. k., young, c., & wald, j. t. (2018). the strategic imperative for the use of social media in health care. journal of the american college of radiology, 15(1), 155-161. doi: https://doi.org/10.1016/j.jacr.2017.09.027 kouzy, r., abi jaoude, j., kraitem, a., el alam, m. b., karam, b., adib, e., baddour, k. (2020). coronavirus goes viral: quantifying the covid-19 misinformation epidemic on twitter. cureus, 12(3). doi: https://dx.doi.org/10.7759%2fcureus.7255 kucukusta, d., law, r., besbes, a., & legohérel, p. (2015). re-examining perceived usefulness and ease of use in online booking. international journal of contemporary hospitality management, 27(2), 185-198. doi: https://doi.org/10.1108/ijchm-09-2013-0413 kuek, a., & hakkennes, s. (2020). healthcare staff digital literacy levels and their attitudes towards information systems. health informatics journal, 26(1), 592-612. doi: https://doi.org/10.1177/1460458219839613 kunst, h., groot, d., latthe, p. m., latthe, m., & khan, k. s. (2002). accuracy of information on apparently credible websites: survey of five common health topics. british medical journal, 324(7337), 581-582. doi: https://doi.org/10.1136/bmj.324.7337.581 kwon, j. h., kye, s.-y., park, e. y., oh, k. h., & park, k. (2015). what predicts the trust of online health information? epidemiology health communication, 37(e2015030.). doi: 10.4178/epih/e2015030 lanier jr, c. d., & saini, a. (2008). understanding consumer privacy: a review and future directions. academy of marketing science review, 2008, 12(2). lau, a. s. (2011). hospital-based nurses’ perceptions of the adoption of web 2.0 tools for knowledge sharing, learning, social interaction and the production of collective intelligence. journal of medical internet research, 13(4), e92.doi:10.2196/jmir.1398 lee, y.-h., hsieh, y.-c., & chen, y.-h. (2013). an investigation of employees' use of e-learning systems: applying the technology acceptance model. behaviour information technology, 32(2), 173-189. https://doi.org/10.1080/0144929x.2011.577190 australasian journal of information systems khan et al. 2021, vol 25, research article effect of barrier related factors 26 liaw, s.-s., & huang, h.-m. (2013). perceived satisfaction, perceived usefulness and interactive learning environments as predictors to self-regulation in e-learning environments. computers education, 60(1), 14-24. doi: https://doi.org/10.1016/ j.compedu.2012.07.015 liébana-cabanillas, f., sánchez-fernández, j., & muñoz-leiva, f. (2014). antecedents of the adoption of the new mobile payment systems: the moderating effect of age. computers in human behavior, 35, 464-478. doi: https://doi.org/10.1016/j.chb.2014.03.022 lin, w.-y., zhang, x., song, h., & omori, k. (2016). health information seeking in the web 2.0 age: trust in social media, uncertainty reduction, and self-disclosure. computers in human behavior, 56, 289-294. doi: https://doi.org/10.1016/j.chb.2015.11.055 lober, w. b., & flowers, j. l. (2011). consumer empowerment in health care amid the internet and social media. seminars in oncology nursing, 27(3),169-182. doi: https://doi.org/10.1016/j.soncn.2011.04.002 machdar, n. m. (2019). the effect of information quality on percieved usefulness and ease of ose business entrepreneurial review, 15(2), 131-146. doi: http://dx.doi.org/10.25105/ber.v15i2.4630 maclure, k., & stewart, d. (2016). digital literacy knowledge and needs of pharmacy staff: a systematic review. journal of innovation in health informatics, 23(3). doi: https://doi.org/10.14236/jhi.v23i3.840 madnick, s. e., wang, r. y., lee, y. w., & zhu, h. (2009). overview and framework for data and information quality research. journal of data information quality, 1(1), 1-22. doi: https://doi.org/10.1145/1515693.1516680 malik, s., & coulson, n. s. (2010). ‘they all supported me but i felt like i suddenly didn't belong anymore’: an exploration of perceived disadvantages to online support seeking. journal of psychosomatic obstetrics & gynecology, 31(3), 140-149. https://doi.org/10.3109/0167482x.2010.504870 martinasek, m. p., panzera, a. d., schneider, t., lindenberger, j. h., bryant, c. a., mcdermott, r. j., & couluris, m. (2011). benefits and barriers of pediatric healthcare providers toward using social media in asthma care. american journal of health education, 42(4), 213-221. doi: https://doi.org/10.1080/19325037.2011.10599190 mcgowan, b. s., wasko, m., vartabedian, b. s., miller, r. s., freiherr, d. d., & abdolrasulnia, m. (2012). understanding the factors that influence the adoption and meaningful use of social media by physicians to share medical information. journal of medical internet research, 14(5), e117. doi:10.2196/jmir.2138 moick, m., & terlutter, r. (2012). physicians' motives for professional internet use and differences in attitudes toward the internet-informed patient, physician–patient communication, and prescribing behavior. medicine 2.0, 1(2), e2. doi: 10.2196/med20.1996 moore, g. c., & benbasat, i. (1991). development of an instrument to measure the perceptions of adopting an information technology innovation. information systems research, 2(3), 192-222. doi: https://doi.org/10.1287/isre.2.3.192 australasian journal of information systems khan et al. 2021, vol 25, research article effect of barrier related factors 27 moorhead, s. a., hazlett, d. e., harrison, l., carroll, j. k., irwin, a., & hoving, c. (2013). a new dimension of health care: systematic review of the uses, benefits, and limitations of social media for health communication. journal of medical internet research, 15(4), e85. doi:10.2196/jmir.1933 morosan, c. (2012). theoretical and empirical considerations of guests’ perceptions of biometric systems in hotels: extending the technology acceptance model. journal of hospitality tourism research, 36(1), 52-84. https://doi.org/10.1177/1096348010380601 mou, j., shin, d.-h., & cohen, j. (2017). understanding trust and perceived usefulness in the consumer acceptance of an e-service: a longitudinal investigation. behaviour information technology, 36(2), 125-139.doi: https://doi.org/10.1080/0144929x.2016 .1203024 nikou, s., agahari, w., keijzer-broers, w., & de reuver, m. (2019). digital healthcare technology adoption by elderly people: a capability approach model. journal of telematics informatics, 53, 101315. https://doi.org/10.1016/j.tele.2019.101315 o'brien, l. (2017, september 23, 2017). the inside scoop part 1: a comparison of the u.s. and australian healthcare systems. onthewards. panahi, s., watson, j., & partridge, h. (2016). social media and physicians: exploring the benefits and challenges. health informatics journal, 22(2), 99-112. doi: https://doi.org/10.1177/1460458214540907 park, a., bowling, j., shaw, g., li, c., & chen, s. (2019). adopting social media for improving health opportunities and challenges. north carolina medical journal, 80(4), 240-243. doi: https://doi.org/10.18043/ncm.80.4.240 pavlou, p. a. (2003). consumer acceptance of electronic commerce: integrating trust and risk with the technology acceptance model. international journal of electronic commerce, 7(3), 101-134. doi: https://doi.org/10.1080/10864415.2003.11044275 pavlou, p. a., & gefen, d. (2004). building effective online marketplaces with institution-based trust. information systems research, 15(1), 37-59. doi: https://doi.org/10.1287/isre .1040.0015 pavlou, p. a., liang, h., & xue, y. (2007). understanding and mitigating uncertainty in online exchange relationships: a principal-agent perspective. mis quarterly, 31(1), 105-136. doi: https://doi.org/10.2307/25148783 peñarroja, v., sánchez, j., gamero, n., orengo, v., & zornoza, a. m. (2019). the influence of organisational facilitating conditions and technology acceptance factors on the effectiveness of virtual communities of practice. behaviour information technology, 38(8), 845-857. https://doi.org/10.1080/0144929x.2018.1564070 robeyns, i. (2005). the capability approach: a theoretical survey. journal of human development, 6(1), 93-117. https://doi.org/10.1080/146498805200034266 rogers, e. m. (2003). elements of diffusion. diffusion of innovations, 5(1.38). rolls, k., hansen, m., jackson, d., & elliott, d. (2016). how health care professionals use social media to create virtual communities: an integrative review. journal of medical internet research, 18(6), e166. doi: https://doi.org/10.2196/jmir.5312 australasian journal of information systems khan et al. 2021, vol 25, research article effect of barrier related factors 28 saeed, k. a., & abdinnour-helm, s. (2008). examining the effects of information system characteristics and perceived usefulness on post adoption usage of information systems. information management decision, 45(6), 376-386. doi: https://doi.org/ 10.1016/j.im.2008.06.002 salim, b. (2012). an application of utaut model for acceptance of social media in egypt: a statistical study. international journal of information science, 2(6), 92-105. doi: 10.5923/j.ijis.20120206.05 smailhodzic, e., hooijsma, w., boonstra, a., & langley, d. j. (2016). social media use in healthcare: a systematic review of effects on patients and on their relationship with healthcare professionals. bmc health services research, 16(1), 442. doi:10.1186/s12913016-1691-0 steenkamp, j.-b. e., & baumgartner, h. (2000). on the use of structural equation models for marketing modeling. international journal of research in marketing, 17(2-3), 195-202. doi: https://doi.org/10.1016/s0167-8116(00)00016-1 talaei-khoei, a., lewis, l., khoei, t., ghapanchi, a., & vichitvanichphong, s. (2015). seniors' perspective on perceived transfer effects of assistive robots in elderly care: capability approach analysis. in: 36th international conference on information systems (icis 2015) exploring the information frontier, 13-15 december 2015, fort worth, texas. teigland, r., & wasko, m. m. (2003). integrating knowledge through information trading: examining the relationship between boundary spanning communication and individual performance. decision sciences, 34(2), 261-286. doi: https://doi.org/10.1111/15405915.02341 teo, t. (2011). modeling the determinants of pre-service teachers' perceived usefulness of elearning. campus-wide information systems, 28(2), 124-140. doi: https://doi.org/10.1108/10650741111117824 thompson, r. l., higgins, c. a., & howell, j. m. (1991). personal computing: toward a conceptual model of utilization. mis quarterly, 15(1), 125-143. doi: https://doi.org/ 10.2307/249443 triandis, h. c. (1979). values, attitudes, and interpersonal behavior. paper presented at the nebraska symposium on motivation, 27, 195–259. usher, k., woods, c., casella, e., glass, n., wilson, r., mayner, l., mather, c. (2014). australian health professions student use of social media. collegian, 21(2), 95-101. doi: https://doi.org/10.1016/j.colegn.2014.02.004 usher, w. (2011). types of social media (web 2.0) used by australian allied health professionals to deliver early twenty-first-century practice promotion and health care. social work in health care, 50(4), 305-329. doi: 10.1080/00981389.2010.534317 usher, w. t. (2012). australian health professionals’ social media (web 2.0) adoption trends: early 21st century health care delivery and practice promotion. australian journal of primary health, 18(1), 31-41. doi: https://doi.org/10.1071/py10084 van der velden, m., & el emam, k. (2013). “not all my friends need to know”: a qualitative study of teenage patients, privacy, and social media. journal of the american medical informatics association, 20(1), 16-24.doi: https://doi.org/10.1136/amiajnl-2012-000949 australasian journal of information systems khan et al. 2021, vol 25, research article effect of barrier related factors 29 van uden-kraan, c. f., drossaert, c. h., taal, e., smit, w. m., seydel, e. r., & van de laar, m. a. (2010). experiences and attitudes of dutch rheumatologists and oncologists with regard to their patients’ health-related internet use. clinical rheumatology, 29(11), 12291236. doi: https://doi.org/10.1007/s10067-010-1435-1 venkatesh, v. (2000). determinants of perceived ease of use: integrating control, intrinsic motivation, and emotion into the technology acceptance model. information systems research, 11(4), 342-365. doi: https://doi.org/10.1287/isre.11.4.342.11872 venkatesh, v., & davis, f. (2000). a theoretical extension of the technology acceptance model: four longitudinal field studies. management science, 46(2), 186-204. doi:10.1287/mnsc.46.2.186.11926 venkatesh, v., & davis, f. d. (1996). a model of the antecedents of perceived ease of use: development and test. decision sciences, 27(3), 451-481. doi: https://doi.org/10.1111/ j.1540-5915.1996.tb00860.x venkatesh, v., sykes, t. a., & zhang, x. (2011). 'just what the doctor ordered': a revised utaut for emr system adoption and use by doctors. paper presented at the 2011 44th hawaii international conference on system sciences. venkatesh, v., thong, j. y., & xu, x. (2012). consumer acceptance and use of information technology: extending the unified theory of acceptance and use of technology. mis quarterly, 36(1), 157-178. doi: https://doi.org/10.2307/41410412 westin, a. f. j. w., & review, l. l. (1968). privacy and freedom. washington and lee law review, 25(1), 166-170. yarbrough, a. k., & smith, t. b. (2007). technology acceptance among physicians: a new take on tam. medical care research and review, 64(6), 650-672. https://doi.org/10.1177/1077558707305942 zhang, x., guo, x., lai, k.-h., guo, f., & li, c. (2014). understanding gender differences in mhealth adoption: a modified theory of reasoned action model. telemedicine e-health, 20(1), 39-46. https://doi.org/10.1089/tmj.2013.0092 zhao, y., ni, q., & zhou, r. (2018). what factors influence the mobile health service adoption? a meta-analysis and the moderating role of age. international journal of information management, 43, 342-350. doi: https://doi.org/10.1016/j.ijinfomgt.2017.08.006 zhou, z., jin, x.-l., & fang, y. (2014). moderating role of gender in the relationships between perceived benefits and satisfaction in social virtual world continuance. decision support systems, 65, 69-79. doi: https://doi.org/10.1016/j.dss.2014.05.004 copyright: © 2021 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v25i0.2625 http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems khan et al. 2021, vol 25, research article effect of barrier related factors 30 1 introduction 1.1 rationale of the research 2 review of the literature 2. 1 research model 2.2 hypothesis development 2.2.1 information quality 2.2.2 perceived trust 2.2.3 privacy threat 2.2.4 professional boundary 2.2.5 facilitating conditions 2.2.6 self-efficacy 2.2.7 behavioural intention 2.3 moderating effect of age, gender and usage frequency 3 research methodology 3.1 data collection 3.2 reliability and validity 4 findings 4.1 the demographic profiles 4.2 social media usage pattern 4.3 results of confirmatory factor analysis (cfa) and hypotheses testing 5. discussion 5.1 impact of barrier-related factors 5.1.1 information quality 5.1.2 perceived trust 5.1.3 privacy threat 5.1.4 professional boundary 5.1.5 facilitating conditions 5.1.6 self-efficacy 5.1.7 behavioural intention 5.2 moderating effect of age, gender and usage frequency 5.3 theoretical implications 5.4 practical implications 6. limitations and future research paper title (paper title style) australasian journal of information systems thuan et al. 2022, vol 26, research article using process stories 1 using process stories to foster process flexibility: the experts’ viewpoint nguyen hoang thuan rmit university, viet nam thuan.nguyenhoang@rmit.edu.vn hoang ai-phuong rmit university, viet nam mathews nkhoma rmit university, viet nam pedro antunes lasige, faculdade de ciências, universidade de lisboa/university of lisbon, portugal abstract process flexibility is essential for organizations coping with uncertainty, emergence, and change. in this study, we research how process stories may lessen friction in realizing flexible processes. we use friction as a metaphor, which characterizes the realization of flexible processes as handling two opposing forces: one pushes towards flexibility while the other pulls against flexibility. using in-depth interviews with bpm experts as a data-gathering technique, we provide insights into the dynamics of friction in the bpm lifecycle. we also provide empirical evidence about the capability of process stories to lessen friction in realizing flexible processes. this research contributes to understand the context where process stories may be most fit to realize process flexibility and adds knowledge about practical complaints experienced by bpm experts when realizing process flexibility. keywords: business processes, process flexibility, process stories, empirical study, qualitative interviews. 1 introduction process flexibility enables organizations to cope with uncertainty, emergence, and change. many organizations nowadays face uncertain business environments (cognini, corradini, gnesi, polini, & re, 2018; mejri, ghannouchi, & martinho, 2016), including a constant state of emergence (de albuquerque & christ, 2015), and operational vagaries (antunes, tate, & pino, 2019; haseeb, ahmad, malik, & anjum, 2019). these ‘push’ towards the realization of more flexible processes. besides, organizations also face significant ‘pull’ against process flexibility from both humans and software technology. for instance, enterprise systems, with their traditional emphasis on structure and control, can generate significant pull against process flexibility (qu, ding, shou, zhou, & du, 2014; rettig, 2007). employees and middle managers can also exhibit more or less intense resisting behaviours to change initiatives (bareil, 2013). we borrow from physics the notion of friction to highlight the forces that emerge when we try to push an object against a surface. similarly, when organizations try to push a new flexible process, they must handle the resisting pull against it. understanding and resolving this type of friction is a concern of business process management (bpm). after all, bpm is fundamentally concerned with managing the relationships between humans, business australasian journal of information systems thuan et al. 2022, vol 26, research article using process stories 2 processes, and software technology to help organizations achieve performance improvements (dumas, la rosa, mendling, & reijers, 2018; rosemann & recker, 2006). it is important for bpm experts to understand and deal with friction in realizing flexible processes. friction highlights a longstanding tension between standardization and flexibility in organizations (boudreau & robey, 1996; de albuquerque & christ, 2015; feldman & pentland, 2003), which is also apparent in bpm. on the one hand, organizations are usually structured around standardized procedures and behaviours (davenport, 2005; romero, dijkman, grefen, & van weele, 2015), which pull against process flexibility. on the other hand, organizations also have to deal with business uncertainties, environmental changes, conflicting viewpoints, unique cases, variations, innovations, and evolutionary changes (cognini et al., 2018; mejri et al., 2016; thuan, phuong, george, nkhoma, & antunes, 2020), which push towards process flexibility. without understanding the different forces involved in friction, bpm experts may find it difficult to realize flexible processes. in this study, we propose process stories to help bpm experts understand the dynamics of friction in realizing process flexibility. process stories rely on storytelling theory to communicate a variety of viewpoints over a business process (antunes, pino, tate, & barros, 2020; haggège & vernay, 2019; hayes, lee, & dourish, 2011). process stories have been used to analyse organizational business processes and the related views over these processes (antunes et al., 2020; leite, antunes, guimarães, & pino, 2019; simões, antunes, & carriço, 2018). we suggest that the capacity of conveying diverse viewpoints about a process can help bpm experts to understand and deal with the various forces involved in the dynamics of friction. consequently, the study addresses the following research question: how can bpm experts use process stories to reduce friction in realizing process flexibility? to answer this question, this study follows a tradition of exploratory studies and adopts a qualitative approach (saldaña, 2015). in particular, the study relies on in-depth interviews with bpm experts. we gather the participants’ views over the dynamics of friction in processes they have responsibility for; and gather the participants’ judgements about the adoption of process stories to support the realization of flexible processes. as a result, the interviews enable us to understand the dynamics of friction in flexible processes and the experts’ perceptions about the adoption of process stories to support process flexibility. this study contributes to research in the bpm field with a dynamic understanding of the realization of process flexibility, which uses friction as a metaphor. it provides insights about the elements pushing towards and pulling against process flexibility in the bpm lifecycle. the study further provides insights into the strengths and weaknesses of process stories as a method supporting the realization of flexible processes. the empirical evidence complements other theoretical efforts promoting the adoption of storytelling in business modelling (antunes et al., 2019; haggège & vernay, 2019; hayes et al., 2011). from a more practical perspective, this study provides guidance on how practitioners can use process stories to reduce existing friction when realizing process flexibility. the remainder of this paper is structured as follows: section 2 sets up the background of the research. section 3 develops and justifies our research framework. section 4 describes the adopted research method. the interview results are presented in section 5. section 6 discusses the implications of this research for the bpm field. finally, section 7 provides some concluding remarks and suggests future research directions. australasian journal of information systems thuan et al. 2022, vol 26, research article using process stories 3 2 background 2.1 prior research on process modelling standardization versus flexibility research in the bpm field has resulted in a plethora of theories, methods and tools that assist organizations in managing business processes (recker & mendling, 2016). a traditional and highly relevant research stream in the field concerns formal modelling of business processes. this stream addresses standardized formalisms that capture bpm-related phenomena (recker & mendling, 2016) and support modelling the different ways in which a process can be handled (van der aalst, 2013). process standardization is considered essential for functional analysis and optimization of organizations (davenport, 2005; romero et al., 2015; schäfermeyer, rosenkranz, & holten, 2012). it allows the organization to specify transparent processes, shorten processing time, and harmonize process activities (münstermann, eckhardt, & weitzel, 2010; romero et al., 2015; tregear, 2015). in addition, formal enough process models are essential for technology support to business processes using process-aware information systems (van der aalst, 2013). while the above literature has realized the benefits of standardized processes, another stream of literature has highlighted the needs for flexible processes. in particular, uncertain business environments (mejri et al., 2016), emergent situations (de albuquerque & christ, 2015), and operational vagaries (antunes et al., 2019; haseeb et al., 2019) are valid grounds for conferring some variability and flexibility on business processes. further, other factors such as new working methods, exceptions, and technology evolution also call for flexible business processes (cognini et al., 2018; simões, thuan, jonnavithula, & antunes, 2015). considering these two streams of literature, researchers have for long pointed out several competing factors related to process modelling, which complicate the realization of process flexibility (de albuquerque & christ, 2015; reichert & weber, 2012; zelt, recker, schmiedel, & vom brocke, 2019). we now review these competing factors in relation to process flexibility, including design versus use, structured versus unstructured processes, complexity versus simplicity, control versus value, technical versus social focus. design versus use. bpm practices traditionally involve a design-use cycle, where a process is designed and modelled, and then the model is used by a process-aware information system to enact the process. this cycle impacts flexibility, as process changes require organizations to continuously go through design-use cycles, which have inherent costs and complications (reichert, 2018). a variety of approaches have been proposed to support runtime dynamic changes and thus avoid going through the design-use cycle (krumeich, weis, werth, & loos, 2014; reichert & weber, 2012; rosa, van der aalst, dumas, & milani, 2017). however, these approaches may introduce mismatches between what has been modelled and what is executed. reichert (2018) suggests that the only feasible way to cope with this problem is to eliminate the design-use dichotomy, allowing processes to be dynamically (re)designed based on use. structured versus unstructured. van der aalst et al. (2009, p. 99) point out that “processaware information systems tend to either support business processes or provide flexibility” but not both. this happens because process-aware information systems require processes to be structured, due to execution constraints. on the other hand, flexibility requires processes to be less structured to accommodate changes. the problem is further complicated because traditional bpm tends to structure processes around activities, which commit organizations to australasian journal of information systems thuan et al. 2022, vol 26, research article using process stories 4 respond with certain activities to specific events. a variety of solutions have been proposed to increase the capacity to deal with changes in a structured way (hidri, m’tir, saoud, & ghediraguegan, 2019; reichert & weber, 2012). examples include adding exceptions and deviations to process models (hallerbach, bauer, & reichert, 2010; marcinkowski & gawin, 2019). however, these approaches increase modelling complexity and do not accommodate true exceptions and emergence. complexity versus simplicity. it is undeniable that process-aware information systems can expedite the enactment of business processes from model designs (dumas et al., 2018; van der aalst, 2013). a variety of software platforms support a rapid roundtrip between modelling and enactment using bpmn, such as aris, camunda, signavio, and oracle bpm suite. this would suggest a “triumphant vision many buy into,” that “software is fully integrated and intelligently controls infinitely complex business processes while remaining flexible enough to adapt to changing business needs” (rettig, 2007, p. 1). however, the results may not be as expected. svejvig and jensen (2013, p. 14) highlight the conflict between “match the current business processes” versus “match the standard package”. also in the studied case, the organization failed its first implementation attempt due to complex institutional practices that could not be modelled properly. the second attempt was successful as the organization decided to conform to a predefined process provided by software platform vendors to reduce complexity. it thus seems that expediting complex process enactment compromises process flexibility. control versus value. vom brocke et al. (2010) point out other factors in play, where the main focus of process modelling is often more related to control a set of activities (required to enact the process) than to design a process that brings actual value to the organization. as such, current process modelling practices focus more on aspects that bring low impact to the organization instead of being value-oriented. proposals to address this problem suggest extending process modelling beyond control to focus on value drivers and needs (e.g., financial and customer satisfaction). however, moving beyond control can make it more difficult to enact business processes in process-aware information systems, especially because value drivers and needs are more elusive and reflect conflicting views about what must be done. technical versus social. when modelling business processes, bpm experts tend to focus on the technical aspects of the process, e.g., defining required resources or alternative paths in response to a certain event. however, processes must be enacted in ways that are “locally and institutionally acceptable” (hayes et al., 2011). therefore, the enactment of a more or less flexible process depends on the characteristics of the individuals and organizations involved in the process (afflerbach, kastner, krause, & röglinger, 2014; de albuquerque & christ, 2015). as noted by de albuquerque and christ (2015), process models should be regarded as socio-technical artefacts, which must combine both technical and social properties. the consideration for social practice requires bpm experts to take full account of the ‘social world’ in which processes take place, considering the nuances and mutual compromises found in the relationships between the technical and the social. considering all these factors, we suggest that there is no one-size-fit-all modelling solution for realizing flexible processes. rather, bpm experts must identify and deal with the different factors and their conflicts, which form two opposing forces, namely push and pull. the push australasian journal of information systems thuan et al. 2022, vol 26, research article using process stories 5 force moves forward process flexibility, while the pull one resists process flexibility. in the next section, we link these forces to the bpm lifecycle. 2.2 process flexibility and the bpm lifecycle the typical bpm lifecycle encompasses different stages: process analysis, design/modelling, enactment/execution, and monitoring/improvement (cognini et al., 2018). as such, the realization of flexible processes must be tackled in all these stages. it ranges from analysing how the organization responds (or not) to internal and external demands for flexibility, resolving issues with the interactions between humans and software applications, servicing the organization and its clients with appropriate software, and finally monitoring and improving flexible processes according to specific goals and targets (aysolmaz, schunselaar, reijers, & yaldiz, 2019; hallerbach, bauer, & reichert, 2008; reichert, hallerbach, & bauer, 2015). in all these stages, friction when realizing flexible processes can be increased (or reduced) due to internal and external factors. for instance, difficulties in analysing what drives process flexibility create friction; difficulties in modelling operational changes also create friction; and problems in executing and monitoring process variations contribute to friction as well. cognini et al. (2018)’s review identifies other difficulties and problems that contribute to increasing (or decreasing) friction when realizing process flexibility in each and all stages. even though the realization of process flexibility involves the entire bpm lifecycle, research in the bpm field has been mainly focused on the enactment/execution stage (cognini et al., 2018). researchers have been developing software approaches that support dynamic, case-based, and ad hoc changes in processes during run-time (braun, schlieter, burwitz, & esswein, 2016; hallerbach et al., 2010; sid, reichert, & ghomari, 2019). for instance, the provop approach allows users to automatically adjust their workflows to small variations over a baseline process, which is accomplished by reconfiguring the sequence of activities (hallerbach et al., 2010). relying on our friction metaphor, these approaches can be characterized as ‘pull’ solutions, whereby software systems reconfigure flexible processes in runtime. hence, they lessen the pull of software technology and people against process flexibility. however, to further reduce friction, researchers should also examine the push towards the realization of more flexible processes. we stress here the etymology of the word ‘push,’ which refers to proactivity in exerting a force towards an intended goal. this proactive ‘push’ empowers organizations to imagine and explore alternative and emergent scenarios without being overly constrained by standardization and red tape (boudreau & robey, 1996; de albuquerque & christ, 2015; feldman & pentland, 2003). it is worthwhile to note that pushing towards the realization of more flexible processes requires adequate process analysis and articulation with the later stages of the bpm lifecycle. such articulation is not only necessary because analysis is a logical antecedent for the other stages (haseeb et al., 2019), but also because process analysis is not done in a vacuum. the main goal of bpm is to help organizations achieve performance improvements, which requires articulation of all stages. knowledge about a flexible process is attained in the analysis stage. this knowledge helps to generate a representation of the flexible process in the design/modelling stage, which is then used by software technology (cognini et al., 2018; van der aalst, ter hofstede, & weske, 2003). furthermore, an examination of the analysis and australasian journal of information systems thuan et al. 2022, vol 26, research article using process stories 6 design/modelling stages regarding process flexibility also contributes to the identifications of barriers and limitations in current software technology (hayes et al., 2011). in summary, while the bpm field has recognised both the push and the pull on process flexibility, existing efforts mainly focus on reducing the pull against process flexibility in the enactment/execution stage (cognini et al., 2018). few efforts have examined how to push towards more flexible processes (antunes et al., 2019; bauer, 2019), and even fewer efforts have investigated the two forces together. this gap prevents us from understanding the nature of the two opposing forces, which affects our full realization of process flexibility. this study fulfils the gap by exploring if process stories can support bpm experts in dealing with the dynamics of friction when realizing process flexibility. 2.3 process stories as a method pushing towards more flexible processes from the above discussion, the emerging concern is how to push towards process flexibility from process analysis to the other stages of the bpm lifecycle. process stories have been proposed as a method that helps to communicate and understand the contextual aspects of human participation in business processes (antunes et al., 2020; antunes et al., 2019). process stories rely on storytelling theory to narrate what happens in a business process using a combination of textual and visual elements. each story narrates the business process from a unique viewpoint, which could be anyone with an interest in the process (process owners, participants, external actors, bpm experts, etc.). the identification and elicitation of process stories may support bpm experts in pushing towards more flexible processes, as process stories highlight a variety of goals and patterns of action (haggège & vernay, 2019). process stories allow the process to adapt and evolve without much impact on process design and modelling, as new stories can be brought into the collection. a few research studies have explored process stories as a method to analyse business processes, assessing the capacity of process stories to examine business scenarios and capture process knowledge (antunes et al., 2020; haggège & vernay, 2019). some works have specifically used process stories as an instrument to analyse process flexibility and contextualisation, focusing on the method’s strengths to highlight variant scenarios lived by different people participating in the process (antunes et al., 2019; simões et al., 2018). to some extent, these studies indicate that process stories are promising to push the realization of process flexibility. as such, this study assesses if process stories can help bpm experts to reduce friction and push towards more flexible processes in the bpm lifecycle. 3 conceptual framework of research in qualitative studies, an established tradition is structuring the research using conceptual frameworks (miles, huberman, & saldaña, 2014; saldaña, 2015). conceptual frameworks help to define the perimeter of the research, guide data gathering and analysis through providing anticipatory data reduction and provide a frame of reference for identifying research contributions (miles et al., 2014). in the present study, the proposed framework identifies the main constructs involved in the realization of process flexibility, supports data collection in the interviews, and serves to synthesise the research contributions. australasian journal of information systems thuan et al. 2022, vol 26, research article using process stories 7 figure 1. friction in the realization of process flexibility, which is characterized as the push towards flexibility and pull against flexibility across the stages of the bpm lifecycle the proposed framework (figure 1) is grounded on the bpm lifecycle, which covers a set of bpm undertakings that cut across almost every process realization (bernardo, galina, & de pádua, 2017; de morais, kazan, de pádua, & costa, 2014). we consider three stages of the bpm lifecycle: analysis, design/modelling, and enactment/execution (we note that to decrease the extent and complexity of our research, we do not consider process monitoring/improvement). in the related literature, we further identify two relevant mediators in the bpm lifecycle: process knowledge and process models. process knowledge is the output of the analysis stage, which identifies relevant business processes and associated activities. process knowledge is essential to design and model business processes (dumas et al., 2018). process models are the output of the design/modelling stage; they represent the identified processes using a particular modelling language (andaloussi, burattin, slaats, kindler, & weber, 2020). process models are also essential to enact and execute business processes using process-aware information systems. given the three stages and two mediators, our conceptual framework characterizes the notion of friction in relation to the realization of process flexibility. the framework highlights that friction involves multiple stages of the bpm lifecycle (we recall section 2.2 for details), where our focus is on the three early stages. further, friction is caused by two opposing forces: push towards flexibility and pull against flexibility. these pushing and pulling forces may be generated at any stage of the bpm lifecycle and are expected to spill over the other stages. after all, the three stages are significantly interdependent (haseeb et al., 2019). we note this framework remains open to new elements emerging from the interviews. with this openness in mind, we used the framework to structure conversations with the interviewees. 4 research method we adopted qualitative interviews with domain experts to explore whether process stories could contribute to reducing friction in realizing flexible processes. interviews enable participants to reflect on their experiences across a range of cases and scenarios for understanding process flexibility, which can complement other particular case studies (de albuquerque & christ, 2015; haggège & vernay, 2019; simchi-levi, wang, & wei, 2018). australasian journal of information systems thuan et al. 2022, vol 26, research article using process stories 8 further, interviews provide a formative and (quasi)naturalistic way to evaluate method artifacts like process stories within real organizational contexts (venable, pries-heje, & baskerville, 2016). finally, interviews are considered appropriate to generate rich data, as bpm experts can discuss both their business process and the related contexts (gross et al., 2021; javidroozi, shah, & feldman, 2019). next, we discuss the adopted procedure, which covers the recruitment of participants, data collection, data analysis, and validity checks. the participants were recruited by emails, public websites, and professional networks. all participants are experienced business and process managers. while having different roles in their companies, all participants regard themselves as bpm experts and have dealt with process flexibility issues in their companies. eleven subjects voluntarily participated in the interviews. this sample size is appropriate for three reasons. first, it helps us to achieve data saturation, as the new interview data tended to repeat the collected data (saunders et al., 2018). while data saturation was reached after nine interviews, we conducted two additional interviews to confirm data saturation. second, although this sample size may limit the generalization of findings, it enables us to analyse the topic in greater details and compare the participants’ responses on the topic, as suggested by fischer et al. (2020). finally, our sample size aligns with creswell’s (2009) suggestion regarding qualitative sampling (between 5 and 25 participants). all interviews lasted about 30-45 minutes and were conducted between 2020 and 2021. they were audio-recorded and transcribed by a research assistant. the transcripts consisted of an average of 4348 words (or approximately 10 pages). table 1 summarises the sample’s demographic information. expert id roles years of experience business sectors gender business processes under their responsibility a business owner and manager >20 lottery wholesaler female lottery distribution b head of multimedia department 10-20 newspaper publisher male publishing articles c business coordinator 10-20 newspaper publisher female publishing articles d business process manager 1-10 e-commerce male product returns and refunds e business process manager 1-10 e-commerce male product returns and refunds f team leader 1-10 higher education female english-level class arrangement g manager 1-10 consulting agency male consulting process to open new business entity h business owner and manager 1-10 home appliance retailer female product sales i manager >20 market research male market research j operations director 1-10 market research male market research k business director 1-10 furniture manufacturer male requirements gathering and furniture implementation table 1. experts’ profile (n=11) australasian journal of information systems thuan et al. 2022, vol 26, research article using process stories 9 data collection was conducted through in-depth interviews and using semi-structured questions to guide the conversations. the questions (see appendix a) were organized into three main categories: 1) background questions asking about the participant’s experience with bpm; 2) questions exploring issues related to process flexibility in selected business processes (see below); and 3) questions exploring the adoption of process stories in the realization of process flexibility. before the interviews, the participants were asked to identify a specific business process under their responsibility, which would be the focus of discussion during the interviews. with the participant’s prior approval, the selected processes were modelled in bpmn and represented using process stories (appendix b shows an example of a business process represented using the two methods). during the interviews, we presented both the bpmn models and process stories to the participants, who were then asked to discuss the two methods with regards to the realization of flexible processes in their own context. table 2. example of the coding process we analysed the collected data using structural coding and thematic coding, suggested by saldaña (2015). these two coding techniques complemented each other (chancellor, townson, structural coding thematic coding structural dimensions structural codes codes example data friction in realizing process flexibility push towards flexibility demands for uncertainty, emergence, change, exceptions, and variations • due to the characteristic of this business, there’re so many exceptions • when we operate the business process, there’s always an exception. there are always things that the client asks out of the box • so, we need to act fast in building that real time processes which will have 80% standardization because we need to have 20% flexibility right at the end of the day, we cannot have 100% straight line. that doesn't work in any market. standard processes do not capture exceptions, looseness, variations, adaptations, and evolutions • obviously, it is a standard process showing each step of what we need to do, and the exceptions have not been shown here • so, this thing is just a standard and common process, and if you ask about exceptions, there are so many of them that i cannot tell you all fostering new solutions in the process • after reading this, we have to find a solution to solve it • bpm experts can redraw a whole new process pull against flexibility demands for abstraction, standards, structure, and control rules • the standard workflow has been performed for decades, since the company has been established • probably by asking them carefully if they have gone over the company’s policies. after confirming, we will have more control about whether returning the product or not • so, we need to act fast in building that real-time processes which will have 80% standardization because we need to have 20% flexibility right at the end of the day, we cannot have 100% straight line. that doesn't work in any market australasian journal of information systems thuan et al. 2022, vol 26, research article using process stories 10 & duffy, 2021; wolff, mahoney, lohiniva, & corkum, 2019), and suited the way we segmented and analysed the collected data. in particular, we adopted structural coding to identify comparable segments of data across the interviewees’ responses. then, we adopted thematic coding on these segments to analyse detailed codes and synthesise them into themes. the analysis was managed using nvivo. data analysis followed a three-step procedure, of which an example is presented in table 2. first, we performed structured coding to set up main dimensions and structural codes, which was based on the conceptual framework (see example in table 2 column 1 and 2). these dimensions and structural codes help to organise the interviewees’ responses into segments. second, in each segment, we performed thematic coding. we created initial codes to label a section of interviewees’ responses in relation to the structural codes. new codes were also allowed to emerge from the data (see example in table 2 column 3 and 4). we note that certain data might be labelled by multiple codes. for instance, a paragraph in table 2 was labelled as both push towards flexibility and pull against flexibility, which suggests an occurrence of the two forces. third, we aggregated the codes into themes, nested under structural codes. then, we compared and summarised the theme-related data across all interviews, which enabled us to find patterns and identify key findings. we checked coding reliability through investigator triangulation. one researcher and one research assistant independently followed the coding procedure and coded the participant’s responses, while another researcher randomly checked the coded items. then, the researchers compared the results and discussed the differences until reaching a consensus. our interview protocol and semi-structured questions ensured data reliability, as all interviews were conducted in a similar manner. 5 interview results we now summarise the interview results, which are grouped into different categories: friction in realizing process flexibility and elements of friction; strengths of process stories in reducing friction; and context where process stories are most fit to increase process flexibility. 5.1 friction in realizing process flexibility most participants acknowledged the occurrence of friction, pushing and pulling the flexibility of processes under their responsibility. eight out of eleven participants commented on and gave concrete examples of both push and pull forces (table 3). the experts highlighted that abstractions, standardizations, structures, and control rules defined in the business processes generated a significant pull against flexibility. they also noted that their organizations had been pushing towards process flexibility to cope with emergence, change, exceptions, and variations. the common cases in the dataset can be presented in three patterns. first, when we started the conversation, the participants began by describing the standard workflows of selected processes (expert a, b, d, f, g, h, i, and j). most participants adopted linear sequences to explain standard workflows (e.g., “so step one is basically […]; the next step is that […]” (expert h) and “the returning process is simply a function of […],” expert d). these explanations were usually enriched with clarifications about control rules. for instance, when explaining a workflow for the product return, a participant also referred to the company’s control policy: “probably by asking them carefully if they have gone over the company’s australasian journal of information systems thuan et al. 2022, vol 26, research article using process stories 11 policies. after confirming, we will have more control about whether returning the product or not” (expert d). for many participants, the standard workflow has been performed for years (“for decades, since the company has been established,” expert a), which suggests the resilience of standard structures and implies the hinder for flexibility. all in all, the first pattern highlights the existence of pull forces hindering flexible processes. node/sub-node (+) expert count (n=11) expert id significant quotes pull against flexibility + demands for abstraction, standards, structure, and control rules 8 a, b, d, f, g, h, i, j “let me tell you briefly about the whole process in general [following with a standard process description]” (expert a) push towards flexibility + demands for uncertainty, emergence, change, exceptions, and variations 11 a, b, c, d, e, f, g, h, i, j, k “when we operate the business process, there’s always an exception. there are always things that the client asks out of the box” (expert g) + standard processes do not capture exceptions, looseness, variations, adaptations, and evolutions 6 a, c, d, e, f, h, j “obviously it is a standard process showing each step of what we need to do, and the exceptions have not been shown here” (expert f) + fostering new solutions with the use of process story 5 b, d, f, i, j “the most important part is, after reading this, we have to find a solution to solve it” (expert b) table 3. main findings regarding push vs pull in relation to process flexibility the common cases in the dataset can be presented in three patterns. first, when we started the conversation, the participants began by describing the standard workflows of selected processes (expert a, b, d, f, g, h, i, and j). most participants adopted linear sequences to explain standard workflows (e.g., “so step one is basically […]; the next step is that […]” (expert h) and “the returning process is simply a function of […],” expert d). these explanations were usually enriched with clarifications about control rules. for instance, when explaining a workflow for the product return, a participant also referred to the company’s control policy: “probably by asking them carefully if they have gone over the company’s policies. after confirming, we will have more control about whether returning the product or not” (expert d). for many participants, the standard workflow has been performed for years (“for decades, since the company has been established,” expert a), which suggests the resilience of standard structures and implies the hinder for flexibility. all in all, the first pattern highlights the existence of pull forces hindering flexible processes. the second pattern emerges when the participants discussed cases of exceptions/changes related to the standard workflows. all participants asserted they were facing uncertainty and change at the process analysis stage of the bpm lifecycle, which resulted in strong demands for exceptions, emergence, change, and variations (experts a, b, c, d, e, f, g, h, i, j, and k). exceptions were identified as the most important push towards process flexibility, which can be illustrated by these statements: “due to the characteristic of this business, there’re so many exceptions” (expert c); and “actually, there are countless of these situations. for example, [exception description]” (expert d). thereby, exceptions were regarded as an essential aspect of business operations. further, as the interviews were conducted during the covid-19 epidemic, changes, arising emergences, variations were viewed as a natural consequence of current business uncertainty (experts a and i). for instance, “[there are] cases where you have australasian journal of information systems thuan et al. 2022, vol 26, research article using process stories 12 to follow the process and cases where you can change” (expert i). all these highlight that organizations are facing strong pushing forces towards process flexibility. interestingly, many participants recognised the co-existence of pull and push forces and suggested that friction is inherent to their business. this forms the third patterns, which can be further classified in two forms. on the one hand, many participants suggested the push towards flexibility as rising from limitations in the standard workflows: “although we need to follow standard process, we still always prepare a manual backup solution” (expert d), “in fact in this specific job, [a] process is just a standard for us to base on when we do our job. but our job has a lot of exceptions” (expert b); and “so this thing is just a standard and common process, and if you ask about exceptions, there are so many of them that i cannot tell you all” (expert c). in this form, the push toward flexibility is intrinsic to the standard workflow. on the other hand, a few participants observed the demand to create standard workflows (or pull against flexibility) from occurring exceptions: “even though we call them exceptions, in theory, they still need to follow a standard process” (expert c). overall, the participants agreed on the existence of friction, which simultaneously involves pull and push towards process flexibility. detailed picture of friction we now synthesize the interview findings into the conceptual framework (section 3) in order to elaborate a more detailed picture of friction when realizing process flexibility. in particular, figure 2 shows the detailed elements of the push towards and pull against process flexibility drawn from the interviews. it also presents two additional elements: understandability and representationality as relevant to the realization of process flexibility. figure 2. elements of friction in realizing process flexibility the push towards flexibility is essentially caused by organizations demanding for uncertainty, emergence, change, exceptions, and variations. considering the bpm lifecycle, this push primarily impacts the analysis and design/modelling stages. these stages involve the need to capture knowledge regarding uncertainty, emergence, change, exceptions, and variations, which require bpm experts “[analysing] how we can solve this problem” (expert f) and find ways to model more flexible processes. the push towards flexibility also impacts the enactment/execution stage. in particular, process models must consider exceptions and variations (expert d, g, and i), which need to be handled in the enactment/execution stage (cagnin & nakagawa, 2021). we further note that some experts in our interviews complained that their current process models do not/or cannot represent exceptions and variations: australasian journal of information systems thuan et al. 2022, vol 26, research article using process stories 13 “obviously it is a standard process showing each step of what we need to do, and the exceptions have not been shown here” (expert f). figure 2 also shows the detailed elements pulling against flexibility, which are caused by demands for abstraction, standards, structure, and control rules. it occurs in the opposite direction of the push force. considering the bpm lifecycle, the pull relates to the need for predefined rules and controls required by the enactment and execution of business processes. some experts suggested that even with software support for support dynamic, case-based and ad hoc changes, rules must be made explicit for their execution (expert c and d). this in turn constrains process design/modelling: “we need to create all this process because of the new requirements, new rules for the code” (expert g). furthermore, it also pulls against flexibility by demanding process knowledge to focus more on abstraction (e.g. understanding the process in general (expert a, d, and g)), and standards (e.g. following the company’s policies (expert d, i, and j)). further, our interview findings indicate two additional elements as important to the realization of process flexibility: understandability and representationality. related to understandability, the participants noted that it is important for the process knowledge to be understood. in particular, the participants mentioned the need to document the actual details of interruptions and exceptions. for instances, one must “understand the process they’re doing” (expert g) and that “i will find out where exactly the processes are interrupted, how it’s been interrupted. after understanding everything, i will analyse how to fix the problem and also understand how difficult it is for other people” (expert b). understandability is also important in processes that require finding new solutions for emergent business problems. by acquiring such knowledge, bpm experts can then “redraw a whole new process” (expert d). the lack of understandability may lead process models to go in erroneous directions, as “if they fall onto the exceptional scenario, they will just keep going straight without understanding about any other possible situations” (expert d). related to representationality, six out of eleven participants indicated that the process models under their supervision were unable to capture and represent business operations in practice, then making it difficult to realize process flexibility. for instance, about a particular process model, one participant noted: “i mean, in this [current process model], there are many activities that happen but cannot be shown in the model” (expert a). processes need to be represented beyond the standard workflows, to include a variety of scenarios where standard workflows must be adapted to specific situations and needs. in summary, figure 2 provides a detailed picture of friction, with elements of push towards and pull against process flexibility along the bpm lifecycle. figure 2 also highlights the roles of understandability and representationality in realizing of process flexibility, which are associated to process knowledge and process models, respectively. 5.2 strength of process stories in reducing friction this section reports on how process stories may contribute to reducing friction when dealing with process flexibility. most participants asserted that process stories may help reducing friction by increasing the understandability and representationality of business processes (table 4). australasian journal of information systems thuan et al. 2022, vol 26, research article using process stories 14 node/sub-node (+) expert count (n=11) expert id significant quotes understandability + in-depth look, understanding 6 a, b, e, f, g, j “[the] process story method is much more specific; when you take a deeper look at the story, you will understand how the situation has happened” (expert b) + easy to express, communication 5 a, c, f, i, j “when we go more in-depth to understand the process, it is obvious that this one [process story] is better” (expert a). + provide reasons behind business operations 4 d, h, i, j “the thing i like about the storytelling method is that it helps me to understanding the reason of each step which leads to the next one” (expert b) + real-life visualised communication 3 b, h, i “they can imagine themselves in this situation. and when a similar situation is happening in real life, they have already a solution to solve it” (expert b) representationality + capture exceptions, looseness, variations, adaptations and evolutions 8 a, b, d, e, f, g, h, k “for example, at the refund step, the [bpmn] chart won’t be able to show me how the system stops working, but the storytelling method will tell me exactly how” (expert d) + provide detail 7 a, b, c, d, e, f, h “in the story, you describe in detail many situations like how to estimate the demand, how to stock inventory, and receive customer’s order” (expert h) + high-level representation of real operations 7 a, c, f, g, h, i, j “it is relevant much to reality” (expert f) table 4. main findings regarding how process stories help process understandability and representation regarding understandability, the participants highlighted different aspects of process stories that make process flexibility easier to analyse: in-depth look on the business operations, easy to express, real-life visualised communication, and providing reasons behind business operations. the most commented aspect was that process stories can provide an in-depth look over the business operations and taking flexibility into consideration. for instance, a participant noted “i think it includes more information and various circumstances in more detail” (expert f); and others noted “they [process stories] can have further details to help understand [the process]” (expert e). this in-depth look at business processes brought by process stories also comes from the ability to capture operational contexts (expert b and e). five participants indicated that process stories are easy to express. by integrating narratives, process stories “do not constraint the way business scenarios are presented, and thus it is easier to express their tasks and ideas” (expert a). this is further supported by another key statement: “in the storytelling what you’re doing is you’re just explaining it in a simple way to make it easier for your team to understand” (expert i). experts d, h, i, and j highlighted another important strength of process stories, which is providing reasons behind certain business operations. in a common case of process flexibility, it is not just the workflow that adapts, the workers involved in the process also must adapt. here, process stories can provide reasons for the uncertainty and change demands, and thus australasian journal of information systems thuan et al. 2022, vol 26, research article using process stories 15 contribute to convince workers to become more involved in process adaptations and evolutions. in this regard, expert i noted “i think storytelling will be helpful. why? because as a team member, if i don’t get a bigger picture, i would just say why, why don’t we follow this? this has been followed for several years. so, the objective of storytelling is to really convince my team that whatever we are doing has a reason why. it is much easier to get their buy-in.” in a similar vein, expert g highlighted process stories can help to justify “what benefit this process brings in not only for the business perspective but also for the people perspective.” three experts further suggested that process stories can also support real-life visualised communication, “by using visual images to show the staffs how to do their job so that they can understand easier” (expert h). this aspect is important in training when one wants to explain the business operations to others, as seen via “storytelling is used for training of each procedure and to express the message to other people more effective because it has been visualised” (expert h). an expert also favoured process stories for their visual images: “i think the story telling, as it could include the image and also show more details” (expert f). regarding representationality, the participants highlighted three aspects related to process stories. first, eight participants agreed that process stories can capture variations. when the participants were asked to compare process stories with the bpmn models of processes under their responsibility, they asserted that process stories can cover more cases (expert a, b and d). one expert noted that “this one [process story] can show more, because this one [bpmn model] is just only a chart. less words cannot express more cases” (expert c). in a similar vein, expert k noted that “process stories are more appropriate, as there can be many variations in the business process. the company would try to collect as much as possible all information and insights from customers to have a more convincing consultation process by hitting those points collected from the customers.” furthermore, process stories were also seen by the participants as a way to document exceptions not present in the bpmn models. as noted by two participants, “at the refund step, the [bpmn] chart won’t be able to show that the system stopped working, but the storytelling method will tell us exactly that” (expert d), and “in the process story, there are some situations or exceptions that do not exist here [in the bpmn model]” expert (c). seven participants suggested that process stories also provide useful details about business processes. in particular, they noted that process stories show the interactions between the stakeholders in the business operations (expert e), provide details on the sequence of operations (expert d and f), and accommodate flexible ways to sequence operations (expert a and b). in fact, one expert exclaimed: “to me, the process story method is so detailed. the first thing is the interactions with the customers. this is the first information about solving a request, it shows how the case was done and the whole story about the situation” (expert e). seven participants also suggested that process stories are highly representative of real business operations. on the one hand, process stories support the representation of routine operations by “showing how the sequence of each step or how the process will happen, so it helps people imagine the process easier” (expert f). on the other hand, they support non-routine operations through the representation of exceptions, variations, adaptations, and emergent cases. as commented by two participants, “this one [process story] can show more [exceptions] (expert c) and “it shows how the cases were done and the whole story about the situation” (expert e). seen together, process stories enable us to represent business operations in a way closer to reality. australasian journal of information systems thuan et al. 2022, vol 26, research article using process stories 16 in summary, the participants highlighted different aspects of process stories supporting understandability and representationality in relation to reducing friction (figure 3). the reduction of friction operates in two directions. first, process stories increase the understandability of process knowledge, which pushes towards better design of process flexibility. second, process stories increase the representationality of process models, bringing them closer to reality. this reduces the pull against flexibility caused by inflexible process models and corresponding enactments/executions. figure 3. roles of process stories in reducing friction 5.3 context where process stories may be most fit (or unfit) another important result was the identification of contexts where process stories may be most fit (or unfit). table 5 summarises these contexts. the interviewed participants suggested that process stories may be most fit in three contexts. first, process stories are useful where the process needs complementarity of abstraction and detail. all participants asserted the combination of process stories and bpmn models provides abstraction with useful details, which can be illustrated through two notes: “this model [bpmn] provides us a big picture while this one [storytelling] gives us more details of the process” (expert a); and “if i want to show them the general plan, i will use the diagram from the bpmn method, to help them understand what they have to do in the near future. but if i need something more specific, i will use the process story method” (expert b). regarding how to use this combination, some participants suggested starting with bpmn models to get an overall picture, and then using process stories for details: “the [bpmn] diagram is general, right? from this we will go to details [process stories]” (expert f). in this combination, we note a practical point, that most organizations may already have bpmn models (or similar models) for their main processes (cognini et al., 2018). thus, process stories can serve as an add-on to existing bpmn (or similar) models, which together provide both the big picture and the details about business processes. further, the combination of bpmn models and process stories is aligned with the current movement towards hybrid approaches to process modelling, which “allow overcoming the limitations of individual languages and maintaining the balance between understandability and flexibility” (andaloussi et al., 2020, p. 8). australasian journal of information systems thuan et al. 2022, vol 26, research article using process stories 17 node/sub-node (+) expert count (n=11) expert id significant quotes context where process stories is most fit + complementarity of abstraction (big picture) and detail 11 a, b, c, d, e, f, g, h, i, j, k “bpmn method is mostly about a general view, where the process stories method is more specific about the situation” (expert b) + describe different situations, scenarios 5 a, c, f, h, j “if we need more subprocesses, this one [storytelling] is needed. and if we need to know further details and know what is right or wrong, we also need this one [storytelling]” (expert a) + handling exceptions 5 a, d, g, i, j “it [process stories] has the details on each step of the exceptional cases. for example, there’ll be some notes about exceptions, and it’ll guide you on how to solve the case” (expert d) complaints experienced by the participants (-) requires more effort reading 3 c, d, h “if talking about my idea, i would prefer the simple and fast model diagram; because storytelling requires me to read a lot, so many words” (expert h) shows too many details and loses the main idea 2 d, h “once i see too many exceptions in stories, then i cannot catch up the main idea of the process anymore” (expert d) time consuming 2 a, c “this one [storytelling] is more appropriate because people can easily understand; but it would take time to read” (expert a) table 5. main findings regarding context where process stories are most fit, and complaints experienced by the participants second, process stories facilitate the description of cases and scenarios. the method fosters “using visual images to show the staff how to do their job so that they can understand easier” (expert h). further, the method also supports situation description with example stories, as noted by two experts: “so what we get is a story in each scenario” (expert e) and “the stories you have describe in detail many situations like [situation description]” (expert h). interestingly, the method enables different perspectives over a business process. in our interviews, we noted a case of two experts from the same company having different views over the same process. using process stories, expert d and e described the same workflow from their personal perspectives and contributed different situations and scenarios regarding the workflow. third, process stories show strengths in handling exceptions. as process stories take the users’ viewpoints, they can represent exceptional situations and different solutions developed by the users to handle the situations. two experts clarified “i think this process story is an interesting method, it concretizes the problem that happens at that time and how we can solve that problem” (expert f), and “the most important part is after reading this [process story] we have to find a solution to solve it” (expert b). other experts further noted that this capability is australasian journal of information systems thuan et al. 2022, vol 26, research article using process stories 18 especially relevant during the covid epidemic: “regarding covid, you really must apply this model [process story]. people need to see how it happens with the company. so, you need to tell them” (expert g). on the other hand, the interviewed participants also identified three concerns regarding the use of process stories in realizing flexible processes: process stories require more effort analysing the process (expert c, d and h); they show too many details and thus the reader loses the main idea (expert d and h); and readers consume more time analysing them (expert a and c). we illustrate these issues with three quotes: “storytelling requires me to read a lot, so many words” (expert h); “once we show too many exceptions in this story, then i cannot catch up the main idea of the writer anymore” (expert d); and “when you look at it, you can imagine steps in detail, but the story method requires more time read” (expert c). these issues can be explained, if we consider the variety of exceptions being modelled, which lead to a large set of stories being defined for a single business process. as a result, many participants have suggested the complementarity of process stories and bpmn models, but not the substitution of one for the other. in summary, our main findings are threefold. first, there is evidence regarding the occurrence of friction in realizing flexible processes, where the participants characterized both the push and pull forces (table 3). based on that, we further identified and characterized the elements of friction in realizing flexible processes across the bpm lifecycle (figure 2). second, process stories can reduce friction by improving the understandability of process knowledge and the representationality of process models (figure 3 and table 4). third, there are contexts where process stories may be most fit (or unfit). table 5 summarises these contexts considering the participants’ feedback. 6 discussion disruptive business environments pose enormous risks as well as opportunities for organizations. both established industry leaders and small businesses must adapt and evolve to stay competitive, which highlights the importance of the realization of process flexibility (cognini et al., 2018; thuan et al., 2020). this study offers several research contributions towards this endeavour. 6.1 an analytical lens for understanding friction when realizing flexible processes the study introduces friction as a metaphor to understand the realization of process flexibility. while previous studies have related process modelling and process flexibility from singular angles (design versus use, structured versus unstructured, complexity versus simplicity, control versus value, and technical versus social), our view takes a holistic perspective, as friction caused by two opposing forces: one pushes towards flexibility while the other pulls against flexibility. the interview results provide empirical evidence supporting this notion of friction in the realization of flexible processes. as a result, a conceptual framework has been elaborated with the main elements of friction (figure 2), which serves as an analytical lens for understanding process flexibility in the bpm lifecycle. with the main elements of friction, bpm experts are expected to understand how the demand for flexibility manifests in the process analysis stage and is pushed to the design/modelling and enactment/execution stages. the push for flexibility involves certain elements: australasian journal of information systems thuan et al. 2022, vol 26, research article using process stories 19 uncertainty, emergence, change, exceptions, and variations. in addition, bpm experts are also required to understand how the pull against flexibility manifests in the process enactment/execution stage and is pulled back the design/modelling and analysis stages. the pull against flexibility involves certain elements: abstraction, standards, structure, and control rules. overall, experts “must move away from their singular systems viewpoint” (hayes et al., 2011, p. 174) to focus on the entire bpm lifecycle and diversity of intervening forces. our work introduces a holistic framework, which highlights the push-pull relationships and details the elements of friction involved. 6.2 process stories as a useful method for analysing and designing flexible processes process stories can be used as a method for designing flexible processes. the interview results indicate that process stories help bpm experts understand process knowledge, with particular focus on real-world operations (table 4); and such understanding reduces friction when realizing flexible processes. positioning the interview results with recent studies, we can further suggest that process stories help bringing personal experiences with the process. they narrate a combination of structured and unstructured events, providing context to the workflows (antunes et al., 2020; simões et al., 2018). they can narrate complex situations using simple scenarios and narratives. they can mix control structures with value-oriented goals and behaviours. finally, process stories can also mix the technical with the social aspects of process realization (antunes et al., 2019). considering different types of process flexibility, process stories can be used, to some extent, to foster flexibility by design, flexibility by deviation, flexibility by under-specification, and flexibility by change (schonenberg, mans, russell, mulyar, & van der aalst, 2008). in the first type, process stories help identify alternative execution scenarios and allow them to be incorporated in the process model at design time (see table 5 regarding ‘describe different situations, scenarios’). in the second type, process stories help identify occasional, unforeseen behaviours (see table 4 regarding ‘capture exceptions, looseness, variations, adaptations and evolutions’). here, we note that, by combining process stories and bpmn models, the method does not require altering the existing process models. in the third type, process stories contribute to understand and represent flexibility under specifications (see table 4), where only at execution time a specific scenario can be selected. in the last type, process stories motivate flexibility by changing the viewpoints and accommodating with alternative narratives. the method handles occasional and permanent unforeseen behaviours, fostering exception handling and problem-solving (see table 5 regarding ‘handling exceptions’). as mentioned above, there is no one-size-fits-all modelling solution for realizing flexible processes. our findings shed light on the specific contexts in which process stories are most fit (table 5). process stories should be used to understand different situations and scenarios, forming a holistic picture of the organization’s particular demands for flexibility, and then using such knowledge to design and model business processes in a way that represents the specific demands for flexibility, e.g., highlighting variations and exceptions. the combination of bpmn models and process stories brings benefits to the bpm lifecycle, as both support process understandability and representationality. for instance, while bpmn models represent the routine aspects of a business process (the “happy path”), process stories represent variant business situations and exceptions, and provide relevant contextual australasian journal of information systems thuan et al. 2022, vol 26, research article using process stories 20 knowledge. this is aligned with the current trend of using hybrid approaches for addressing process flexibility (andaloussi et al., 2020). 6.3 implications for process enactment/execution we note that process stories primarily concern the analysis and design/modelling stages of the bpm lifecycle. thus, this study differs from previous studies focusing on the enactment/execution stage, where the main goal is to develop software approaches supporting the enactment and execution of flexible processes (braun et al., 2016; hallerbach et al., 2010; sid et al., 2019). we further note that the analysis and design/modelling stages are logical antecedents of the enactment/execution stage (haseeb et al., 2019). only by properly capturing and representing flexible scenarios, the corresponding flexible processes can be enacted and executed. in this regard, we identify two limitations of the current study. first, we do not discuss if/how process stories can be technically integrated with bpmn (or similar) models. such technical integration would extend the ways process-aware information systems can automate and control flexible processes, for instance, by supporting the coherent execution of alternative and conflicting stories. this technical integration is an interesting topic for future work. our contribution lies in the discussion of the antecedents of such integration, which involve the understandability and representationality properties. second, even though we discuss process flexibility throughout the bpm lifecycle, the actual execution of process flexibility is left for future work. in particular, we do not propose and validate a method for analysing, designing, enacting, monitoring, and improving flexible business processes. nevertheless, we provide the foundations for developing such a method, considering in particular: • articulation of process analysis, design, and modelling, so that bpm experts better understand the forces imposed by the “happy path” across the various realization stages. • provide bpm experts with conceptual scaffolding required to design or redesign more flexible processes. • highlight relevant properties (understandability and representationality) to assess the realization of process flexibility. 6.4 implications for research and practice this study enriches the literature of process stories in bpm. existing studies have adopted storytelling for analysing and representing business processes (bauer, 2019; haggège & vernay, 2019), and examined process stories and process flexibility from a theoretical perspective (antunes et al., 2019; hayes et al., 2011). this study extends those findings with empirical evidence regarding the benefits of process stories in reducing friction in realizing flexible processes. in this context, process stories can improve the understandability of process knowledge and the representationality of process models. to bpm managers and practitioners, this study suggests that process stories are useful in the early stages of the bpm life cycle. our findings further suggest contexts in which process stories are most suitable and highlight some complaints experienced by bpm experts (table 5). these insights provide practical guidelines for using process stories in analysing and australasian journal of information systems thuan et al. 2022, vol 26, research article using process stories 21 designing/modelling flexible business processes. in addition, it is worthwhile to note that process stories can be used in combination with bpmn models. this helps reducing overhead for organizations, as the addition of process stories does not require organizations to replace existing process modelling, enactment, and execution technologies. 7 conclusion we began our research with the observation that organizations face increasing push towards process flexibility, while they also face significant pull against process flexibility. thus, there is a strong need for understanding the interactions between the two forces. addressing this gap, this research proposes the use of process stories in understanding friction throughout the bpm lifecycle. we explore this proposition using a series of in-depth interviews with bpm experts. the results show that process stories help reducing friction by increasing the understandability and representationality of business processes. our findings align with cognini et al. (2018), confirming that the realization of process flexibility has to be addressed in all stages of the bpm lifecycle. although process stories have already been used to analyse business processes (antunes et al., 2020; haggège & vernay, 2019), this research is a pioneer in considering process stories to address friction when realizing flexible processes. regarding future research directions, we observe from the interviews that modelling processes with visual stories are cognitively demanding, which suggests further investigation into how to ease such cognitive demand (haggège & vernay, 2019). future research should also explore the cognitive differences between showing participants bpmn models and process stories. another direction for future research involves the semantic and syntactic integration of process stories and bpmn (or similar) models. to generalize our results, future case studies could assess the use of process stories in different stages of the bpm lifecycle. another possible direction could be to extend the time frame of the research. as this study was conducted in a period highly influenced by the covid-19 pandemic, it would be interesting to investigate friction (pulling and pushing towards process flexibility) in a longitudinal study. acknowledgments this work was supported by rmit university vietnam (ref. irg 2022 – grant 12) and lasige research unit (ref. uidb/00408/2020). references afflerbach, p., kastner, g., krause, f., & röglinger, m. (2014). the business value of process flexibility. business & information systems engineering, 6(4), 203-214. andaloussi, a. a., burattin, a., slaats, t., kindler, e., & weber, b. (2020). on the declarative paradigm in hybrid business process representations: a conceptual framework and a systematic literature study. information systems, 91, 101505. antunes, p., pino, j. a., tate, m., & barros, a. (2020). eliciting process knowledge through process stories. information systems frontiers, 22(5), 1179–1201. doi:https://doi.org/10.1007/s10796-019-09922-0 antunes, p., tate, m., & pino, j. a. (2019). business processes and flexibility: a theoretical perspective. proceedings of the australasian conference on information systems, fremantle, australia, 64-74. https://aisel.aisnet.org/acis2019/7 australasian journal of information systems thuan et al. 2022, vol 26, research article using process stories 22 aysolmaz, b., schunselaar, d. m., reijers, h. a., & yaldiz, a. (2019). selecting a process variant modeling approach: guidelines and application. software & systems modeling, 18(2), 1155-1178. bareil, c. (2013). two paradigms about resistance to change. organization development journal, 31(3), 59-71. bauer, t. (2019). pre-modelled flexibility for business processes. proceedings of the 21st international conference on enterprise information systems, crete, greece, 547-555. bernardo, r., galina, s. v. r., & de pádua, s. i. d. (2017). the bpm lifecycle. business process management journal, 23(1), 155-175. boudreau, m. c., & robey, d. (1996). coping with contradictions in business process reengineering. information technology & people, 9(4), 40-57. braun, r., schlieter, h., burwitz, m., & esswein, w. (2016). bpmn4cp revised--extending bpmn for multi-perspective modeling of clinical pathways. proceedings of the 49th hawaii international conference on system sciences (hicss), 3249-3258. cagnin, m. i., & nakagawa, e. y. (2021). towards dynamic processes-of-business processes: a new understanding. business process management journal, 27(5), 1545-1568. chancellor, c., townson, l., & duffy, l. (2021). destination ambassador programs: building informed tourist friendly destinations. journal of destination marketing & management, 21, 100639. cognini, r., corradini, f., gnesi, s., polini, a., & re, b. (2018). business process flexibility-a systematic literature review with a software systems perspective. information systems frontiers, 20(2), 343-371. creswell, j. w. (2009). research design: qualitative, quantitative, and mixed methods approaches (3 ed.): sage publications, thousand oaks, california. davenport, t. h. (2005). the coming commoditization of processes. harvard business review, 83(6), 100-108. de albuquerque, j. p., & christ, m. (2015). the tension between business process modelling and flexibility: revealing multiple dimensions with a sociomaterial approach. the journal of strategic information systems, 24(3), 189-202. de morais, r. m., kazan, s., de pádua, s. i. d., & costa, a. l. (2014). an analysis of bpm lifecycles: from a literature review to a framework proposal. business process management journal, 20(3), 412-432. dumas, m., la rosa, m., mendling, j., & reijers, h. a. (2018). fundamentals of business process management (2 ed.): springer, berlin, germany. feldman, m. s., & pentland, b. t. (2003). reconceptualizing organizational routines as a source of flexibility and change. administrative science quarterly, 48(1), 94-118. fischer, m., imgrund, f., janiesch, c., & winkelmann, a. (2020). strategy archetypes for digital transformation: defining meta objectives using business process management. information & management, 57(5), 103262. doi: https://doi.org/10.1016/j.im.2019.103262 gross, s., stelzl, k., grisold, t., mendling, j., röglinger, m., & vom brocke, j. (2021). the business process design space for exploring process redesign alternatives. business process management journal, 27(8), 25-56. doi: https://doi.org/10.1108/bpmj-03-20200116 haggège, m., & vernay, a.-l. (2019). story-making as a method for business modelling. business process management journal, 26(1), 59-79. doi:10.1108/bpmj-12-2017-0363 australasian journal of information systems thuan et al. 2022, vol 26, research article using process stories 23 hallerbach, a., bauer, t., & reichert, m. (2008). managing process variants in the process lifecycle. proceedings of 10th international conference on enterprise information systems, barcelona, spain, 154-161. hallerbach, a., bauer, t., & reichert, m. (2010). capturing variability in business process models: the provop approach. journal of software maintenance and evolution: research and practice, 22(6-7), 519-546. haseeb, j., ahmad, n., malik, s. u., & anjum, a. (2019). application of formal methods to modelling and analysis aspects of business process reengineering. business process management journal, 26(2), 548-569. hayes, g. r., lee, c. p., & dourish, p. (2011). organizational routines, innovation, and flexibility: the application of narrative networks to dynamic workflow. international journal of medical informatics, 80(8), e161-e177. hidri, w., m’tir, r. h., saoud, n. b. b., & ghedira-guegan, c. (2019). a meta-model for context-aware adaptive business process as a service in collaborative cloud environment. procedia computer science, 164, 177-186. javidroozi, v., shah, h., & feldman, g. (2019). a framework for addressing the challenges of business process change during enterprise systems integration. business process management journal, 26(2), 463-488. krumeich, j., weis, b., werth, d., & loos, p. (2014). event-driven business process management: where are we now?: a comprehensive synthesis and analysis of literature. business process management journal, 20(4), 615-633. leite, f., antunes, p. a. m., guimarães, n., & pino, j. a. (2019). method supporting the adoption of visual stories in judicial systems. proceedings of the 27th european conference on information systems (ecis), stockholm and uppsala, sweden, paper 73. marcinkowski, b., & gawin, b. (2019). a study on the adaptive approach to technology-driven enhancement of multi-scenario business processes. information technology & people, 31(1), 118-146. mejri, a., ghannouchi, s. a., & martinho, r. (2016). representing business process flexibility using concept maps. procedia computer science, 100, 1260-1268. miles, m. b., huberman, a. m., & saldaña, j. (2014). qualitative data analysis: a methods sourcebook: sage publications, thousand oaks, california. münstermann, b., eckhardt, a., & weitzel, t. (2010). the performance impact of business process standardization: an empirical evaluation of the recruitment process. business process management journal, 16(1), 29-56. qu, w. g., ding, y., shou, y., zhou, h., & du, h. (2014). the impact of enterprise systems on process flexibility and organisational flexibility. enterprise information systems, 8(5), 563-581. recker, j., & mendling, j. (2016). the state of the art of business process management research as published in the bpm conference. business & information systems engineering, 58(1), 55-72. reichert, m. (2018). enabling flexible and robust business process automation for the agile enterprise. in gruhn v., striemer r. (eds) the essence of software engineering (pp. 203220): springer, cham. reichert, m., hallerbach, a., & bauer, t. (2015). lifecycle management of business process variants. in handbook on business process management 1 (pp. 251-278): springer. reichert, m., & weber, b. (2012). enabling flexibility in process-aware information systems: challenges, methods, technologies: springer science & business media, berlin, germany. australasian journal of information systems thuan et al. 2022, vol 26, research article using process stories 24 rettig, c. (2007). the trouble with enterprise software. mit sloan management review, 49(1), 21–27. romero, h. l., dijkman, r. m., grefen, p. w., & van weele, a. j. (2015). factors that determine the extent of business process standardization and the subsequent effect on business performance. business & information systems engineering, 57(4), 261-270. rosa, m. l., van der aalst, w. m. p., dumas, m., & milani, f. p. (2017). business process variability modeling: a survey. acm computing surveys (csur), 50(1), 1-45. rosemann, m., & recker, j. (2006). context-aware process design: exploring the extrinsic drivers for process flexibility. proceedings 18th international conference on advanced information systems enginnering, namur university press, belgium, 149-158. saldaña, j. (2015). the coding manual for qualitative researchers: sage, thousand oaks, california. saunders, b., sim, j., kingstone, t., baker, s., waterfield, j., bartlam, b., . . . jinks, c. (2018). saturation in qualitative research: exploring its conceptualization and operationalization. quality & quantity, 52(4), 1893-1907. schäfermeyer, m., rosenkranz, c., & holten, r. (2012). the impact of business process complexity on business process standardization. business & information systems engineering, 4(5), 261-270. schonenberg, h., mans, r., russell, n., mulyar, n., & van der aalst, w. m. p. (2008). process flexibility: a survey of contemporary approaches. in advances in enterprise engineering i (pp. 16-30): springer, berlin, germany. sid, i., reichert, m., & ghomari, a. r. (2019). enabling flexible task compositions, orders and granularities for knowledge-intensive business processes. enterprise information systems, 13(3), 376-423. simchi-levi, d., wang, h., & wei, y. (2018). increasing supply chain robustness through process flexibility and inventory. production and operations management, 27(8), 14761491. simões, d., antunes, p., & carriço, l. (2018). eliciting and modeling business process stories. business & information systems engineering, 60(2), 115-132. simões, d., thuan, n. h., jonnavithula, l., & antunes, p. (2015). modelling sensible business processes. in t. k. dang, r. wagner, j. küng, n. thoai, m. takizawa, & e. neuhold (eds.) future data and security engineering (vol. 9446, pp. 165-182): springer international publishing, switzerland. svejvig, p., & jensen, t. b. (2013). making sense of enterprise systems in institutions: a case study of the re-implementation of an accounting system. scandinavian journal of information systems, 25(1), 3-36. thuan, n. h., phuong, h. a., george, m., nkhoma, m., & antunes, p. (2020). toward an ontology for improving process flexibility. in dang t.k., küng j., takizawa m., chung t.m. (eds) future data and security engineering (pp. 411-428). cham: springer international publishing, switzerland. tregear, r. (2015). business process standardization. in vom brocke & rosemann m. (eds) handbook on business process management 2 (pp. 421-441): springer, berlin, germany. van der aalst, w. m. p. (2013). business process management: a comprehensive survey. international scholarly research notices, 2013. doi:http://dx.doi.org/10.1155/2013/507984 van der aalst, w. m. p., pesic, m., & schonenberg, h. (2009). declarative workflows: balancing between flexibility and support. computer science-research and development, 23(2), 99113. australasian journal of information systems thuan et al. 2022, vol 26, research article using process stories 25 van der aalst, w. m. p., ter hofstede, a. h., & weske, m. (2003). business process management: a survey. in van der aalst w.m.p. & weske m. (eds) proceedings of international conference on business process management 2003, eindhoven, the netherlands, 1-12. venable, j., pries-heje, j., & baskerville, r. (2016). feds: a framework for evaluation in design science research. european journal of information systems, 25(1), 77–89. vom brocke, j., recker, j., & mendling, j. (2010). value-oriented process modeling: integrating financial perspectives into business process re-design. business process management journal, 16(2), 333-356. wolff, b., mahoney, f., lohiniva, a. l., & corkum, m. (2019). collecting and analyzing qualitative data. the cdc field epidemiology manual; oxford university press: oxford, uk; new york, ny, usa, 213-228. zelt, s., recker, j., schmiedel, t., & vom brocke, j. (2019). a theory of contingent business process management. business process management journal, 25(6), 1291-1316. appendix a: semi-structured questions for the interview section 1: general information about their business processes. 1. tell me about the business process(es) that you have already participated 2. can you describe your roles in the(se) process(es)? a. for how long? 3. in the(se) process(es), do you face any issues regarding the process changes, unique cases and exceptions section 2: exploring process flexibility in selected business processes [show the participants selected processes modelled according to bpmn models] 4. do you know if the process is followed as described? 5. in what circumstances do you see standard processes? 6. in what circumstances do you see variations? 7. how do you handle flexibility of these processes? 8. for what reason do you see those variations? do you document variations? why not? 9. are there cases where the process fails, and you have to improvise? what happens when you improvise? section 3a: process stories in the realization of process flexibility [show the participants processes modelled according to process stories] 10. how process stories accommodate standard processes? 11. how process stories accommodate process variations? 12. how do you compare the stories model to bpmn? 13. do the stories better represent your real business process? 14. how these process stories handle changes, unique cases and exceptions? 15. did/does the proposed method lead you to any additional new way to change/adapt your business processes? can you explain these changes/adaptations? australasian journal of information systems thuan et al. 2022, vol 26, research article using process stories 26 section 3b: combine/compare process stories and bpmn 16. how do you think about the proposed method regarding its ability to integrate the abstract and concrete? 17. which model can help you understand more regarding business process elements: activities or narrative elements? 18. (general versus specific): a. which model provide you an overall picture? b. which model provide you further details? 19. how would you use process stories in your business/role? australasian journal of information systems thuan et al. 2022, vol 26, research article using process stories 27 appendix b: an example of process stories and bpmn model process stories of lottery distribution australasian journal of information systems thuan et al. 2022, vol 26, research article using process stories 28 australasian journal of information systems thuan et al. 2022, vol 26, research article using process stories 29 australasian journal of information systems thuan et al. 2022, vol 26, research article using process stories 30 australasian journal of information systems thuan et al. 2022, vol 26, research article using process stories 31 australasian journal of information systems thuan et al. 2022, vol 26, research article using process stories 32 australasian journal of information systems thuan et al. 2022, vol 26, research article using process stories 33 bpmn models of lottery distribution in a tier 1 wholesaler company in vietnam prepare won tickets request unsold ticket volume data calculate payables select payment method calculate commission pay the company deliver signature new ticket payment for company has commission? tickets allocation/ request increase volume maintain volume decrease volume notify partners finalise volumedistribution/trade plan for tier 1 agents volume adjustment? volume negotiation? australasian journal of information systems thuan et al. 2022, vol 26, research article using process stories 34 nmix ticket stacks deliver to partners exchange other ticket stacksy distribution/ trade with tier 1 agents any request for changes regarding unlucky numbers? manage payables and receivables receive additional payables proceed payments pay by cash bank transfer exchange won ticketspayment from tier 1 agents additional payables? by cash by exchange won tickets tickets allocation/ request increase volume maintain volume decrease volume notify partners finalise volume manage inventory distribution plan for tier 2 agents volume adjustment? volume negotiation? australasian journal of information systems thuan et al. 2022, vol 26, research article using process stories 35 mix ticket stacks at smaller level deliver to customers customers collect at warehouse distribution for tier 2 agents choose method for deliver collect unsold tickets calculate receivables proceed payments cash won tickets payment from tier 2 agents by cash by exchange won tickets copyright: © 2022 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v26i0.3479 http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 background 2.1 prior research on process modelling standardization versus flexibility 2.2 process flexibility and the bpm lifecycle 2.3 process stories as a method pushing towards more flexible processes 3 conceptual framework of research 4 research method 5 interview results 5.1 friction in realizing process flexibility 5.2 strength of process stories in reducing friction 5.3 context where process stories may be most fit (or unfit) 6 discussion 6.1 an analytical lens for understanding friction when realizing flexible processes 6.2 process stories as a useful method for analysing and designing flexible processes 6.3 implications for process enactment/execution 6.4 implications for research and practice 7 conclusion process stories of lottery distribution paper title (paper title style) australasian journal of information systems kaluarachchi, warren & jiang 2020, vol 24, selected papers from the 2019 aice conference cyberbullying among young people 1 responsible use of technology to combat cyberbullying among young people chintha kaluarachchi centre for cyber security research and innovation, deakin university. c.kaluarachchi@deakin.edu.au matthew warren centre for cyber security research and innovation, rmit university. frank jiang centre for cyber security research and innovation, deakin university. abstract cyberbullying has become a major challenge for authorities, parents, guardians and schools in particular, especially in the era of the digital world. this paper reviews available empirical research to examine the issues such as the responsible use of technology amongst young people, parents and schools responsibility to protect against cyberbullying. the analysis revealed that the responsible use of technology provides better practices to encourage comparisons because of these new digital technologies. parents and educators are the key to cyber ethics, therefore teaching the responsible use of technology whilst focusing on cyber ethics at the start of young people’s exposure to technology use may be an excellent strategy to reduce the growth and impact of cyberbullying. the paper will also review good practices for young people, school communities and parents to prevent and manage cyberbullying and unethical behaviours online. these claims are examined using current literature to ensure a better understanding of responsible use of technology and understanding of cyberbullying in order to support young people to combat this immerging societal challenge. keywords: awareness, cyberbullying, cyber ethics, technology, young people and social media. 1 introduction the world wide web is "an interactive sea of shared knowledge, made of the things our friends and we have seen, heard, believe or have figured out" (tim berners-lee, 1995). the introduction of world wide web 2.0 allowed the internet to be transferred to a social environment by adopting social media which enables users to generate online content and interactions on the web (lai & turban, 2008). new web technologies such as social networking sites and online communities are useful tools to share information and social interactions effortlessly, and these technologies also offered new opportunities for businesses (lu & hsiao, 2010). with the recent development of digital technologies, the internet has become an integral part of our lives. these internet based systems contributed to a heightened appreciation for their potential benefits; however, these are outweighed by their negative impacts such as cyberbullying. australasian journal of information systems kaluarachchi, warren & jiang 2020, vol 24, selected papers from the 2019 aice conference cyberbullying among young people 2 cyberbullying is a worldwide phenomenon, and has become a new form of bullying which occurs through the internet via cell phones, computer devices or handheld devices and can be anonymous and can occur 24 hours a day (feinberg & robey, 2008). researchers have shown that cyberbullying can happen at the "speed of thought" by using a collection of modern technology including sounds, transformed photos, intimidating messages, videos, slide shows and polls (sabella et al., 2013). further cyberbullying can occur anytime at anyplace in contrast with traditional bullying; therefore, this has become prevalence than traditional bullying (huang et al., 2018). research has also confirmed that cyberbullying behaviours are growing in educational environments and create new ethical and legal issues including severe consequences for young people’s living and learning environments, as well as physical endangerments (smith & yoon, 2012). researcher gonzales (2014) stated that “bullying online is contrary to the moral good of a person”. some young people are threatened by online strangers in numerous ways such as sending threatening or insulting messages, spreading rumours, posting humiliating pictures, and revealing or stealing personal information (couvillon & ilieva, 2011; perren et al., 2012). having external contacts via chat or emails results in explicit sexual approaches and possibly putting young people at risk, we also see that young people are distressed and terrified via these intimidating messages (stacey, 2009). cyberbullying can affect someone’s safety, selfperception, health, and education rights (elci & seckin, 2016). violating someone’s sense of well-being and lifestyle has now become a major ethical issue (so & lenarcic, 2014). technology (including the internet) is a great tool which allows young people to stay in touch with the outside world and therefore provides enormous benefits for their education (united nations children’s fund, 2017). it is not ethical if someone is using the internet to humiliate, harass or threaten young people. the ethical issue of cyberbullying is that the cyber victims are being humiliated, harassed, and threatened and this causes immense distress to people's lives with numerous health concerns including depression and suicidal behaviour among young people (kaltiala-heino et al., 1999). research has shown that young people who are cyberbullied are twice as likely to commit suicide compared to the young people who are not being bullied (cole, 2014). further, according to the empirical literature; the key impacts of cyberbullying include distress (li, 2010; sahin, 2012), depression (kowalski & fedina, 2011), loneliness (cross et al., 2009; sahin, 2012), increased psychosomatic symptoms (sourander et al., 2010), suicidal ideation (aboujaoude et al., 2015; hinduja & patchin, 2010), low self-esteem and reduced academic performance (smith et al., 2008). however, most of the victims of cyberbullying do not recognise their experiences as being cyberbullying, and about 90% of the victims did not tell their parents or other trusted adults about what they had experienced (kowalski & limber, 2007). the anonymous nature and lack of authority in cyberspace make the problems even worse. cyber ethics are key, it is very important that the moral conduct in both physical and virtual environments remain the same including showing respect for each other, being honest and not stealing others information in order to reduce cyberbullying and unethical behaviours online. there are numerous motivation factors which can drive cyberbullying acts, for instance, 1) seeking revenge (berger, 2007); 2) physical appearance, social status and experiences from the school settings (tynes et al., 2010); 3) limited social and peer support (williams & guerra, 2007); australasian journal of information systems kaluarachchi, warren & jiang 2020, vol 24, selected papers from the 2019 aice conference cyberbullying among young people 3 4) the ability to be anonymous (barlett, 2015); 5) high environmental exposure to violence (calvete et al., 2010); 6) technology capabilities and activities (walrave & heirman, 2011); 7) power imbalance (berger, 2007; olweus, 2013). social networking sites have become a ubiquitous key platform for cyberbullying. (livingstone et al., 2011; ybarra et al., 2007). according to the iris research, (2014) study conducted amongst 384 australian schools, facebook was identified as the most common platform where cyberbullying has been reported. the iris research related to cyberbullying also identified that: 55.6% of cases involved creating hate websites or social media pages; 58.4% of cases involved creating fake social networking accounts; 20.6% of cases involved personal information being posted without permission; 19.4% of cases involved inappropriate posting pictures, videos, or inappropriate image tagging. researchers found that 88% of us young people who use social media had witnessed harassment on social networking sites, while 15% had been victimised and 19% had harassed someone on social networking sites (lenhart et al., 2011). there are various prevention strategies highlighted by the cyberbullying literature for example, awareness-raising and training activities aimed at young people, parents and school personal, suitable school policies to restrict cyberbullying behaviours and coping strategies (kowalski et al., 2008; shariff & johnny, 2007; spears et al., 2014). most of the victims reported improvements after talking with peers and parents or someone trustworthy about the cyberbullying incidents (aricak et al., 2008; berg & breheny, 2014). reporting cyberbullying incidents to the content provider, or to call law enforcement was another successful strategy (hinduja & patchin, 2019; slonje et al., 2013). many researchers agree that education plays a vital role in preventing and responding to cyberbullying (hinduja & patchin, 2010; kowalski et al., 2012; shariff, 2005). researchers wong-lo & bullock (2011) confirmed that “a proactive approach in collaborating and educating parents, school personnel, and citizens in the communities about cyberbullying is one step toward prevention of this evolving issue”. schools should educate students about cyber ethics and the proper conduct to be used in the online world so that they know how to behave and keep themselves safe online (cole, 2014). you cannot wait until young people start working to make them aware of cyberbullying because their understanding of morality and ethics are developed in their early years. the logical places to address cyberbullying issues occur in the home and classroom (crystal et al., 2000). according to yamano (2004) “cyber ethics refers to the guidelines or framework upon which students will base their ethical and moral decision making. cyber ethics includes the guidelines for ethical and responsible use of computers, as well as the internet”. we also see that researchers state that cyber ethics is “the nature of right and wrong in cyberspace” (niyazi et al., 2011). the responsible use of technology is a key aspect of cyber ethics and refers to knowing right and wrong use of technology in the cyberspace. if technology is applied without the consideration of ethics and governance, negative outcomes such as cyberbullying may arise and impact society. australasian journal of information systems kaluarachchi, warren & jiang 2020, vol 24, selected papers from the 2019 aice conference cyberbullying among young people 4 2 research rationale as a growing social phenomenon, most of the cyberbullying research largely focuses on the prevalence and predictors of cyberbullying (lee, 2017), the comparison of traditional bullying and cyberbullying (hinduja & patchin, 2010), risk and protective factors of cyberbullying (kowalski et al., 2019), coping strategies of cyberbullying (raskauskas & huynh, 2015) and the characteristics of cyber perpetrators, cyber victims, and bystanders (lund & ross, 2017). less attention has been given to cyber ethics and the responsible use of technology to prevent cyberbullying. moreover, ethical or responsible use of technology and cyber ethics have been largely disregarded in technological education (berkowitz, 2000). other research had confirmed that the safe and ethical uses of technology have rarely been discussed in schools (crystal et al., 2000). there is a growing consensus that the teaching the responsible use of technology is an excellent solution to combat cyberbullying among young people (sabella et al., 2013; välimäki et al., 2013). according to nsw parliamentary research findings, increasing the awareness of the impact of cyberbullying and using technology responsibly to minimise cyberbullying behaviours have become a high priority matter for authorities, parents, guardians and australian schools in particular (angus, 2016). we have also seen researchers propose that youth advocates should improve and seek new knowledge related to the identification, response, and prevention of this immerging societal problem (sabella et al., 2013). without a careful review of the existing literature, educating or assisting young people can lead to a situation wrong decisions are being made, development of negative attitudes, and unsuccessful strategies or programs to counter cyberbullying (kowalski et al., 2008; sabella et al., 2013). 3 overview of the current literature this paper presents a narrative overview of the current literature relating to cyberbullying to clarify some of the key issues related to the positive or responsible use of technology. the narrative literature review process provides “a synthesis of published literature on a topic and describe its current state-of-art” (ferrari, 2015). within this paper we employed a narrative review methodology to discuss the following issues: a) what type of actions/responses regarding the responsible use of technology are the most effective in preventing and responding to cyberbullying among young people; b) identify what the web-based or app-specific interventions are available to deal with cyberbullying; c) ways to use social media responsibly to deal with cyberbullying. the paper also looks at good practices for young people, school communities and parents to prevent and manage cyberbullying and associated risks. the research approach taken in the paper was a multi-step approach. first, to review the literature, we performed an extensive review of the available related literature focusing on to the cyberbullying, cyber ethics and the responsible use of technology. in terms of this study we focused on keywords such as cyberbullying (or related terms), interventions or preventive strategies (or related terms), cyber ethics (or related terms), responsible use of technology (or related terms), and young people (or related terms). as part of this process we also looked at government reports and reviewed online resources such as blogs and webpages to collect other types of information. australasian journal of information systems kaluarachchi, warren & jiang 2020, vol 24, selected papers from the 2019 aice conference cyberbullying among young people 5 3.1 what type of actions/responses regarding the responsible use of technology are most effective in preventing and responding to cyberbullying among young people young people need to develop online safety skills and netiquette, especially for social networking sites, to allow them to avoid or deal with issues such as cyberbullying. many people believe that they already understand and are able to recognise what cyberbullying is (sabella et al., 2013). however, less than half of the 11-13-year-olds identified in the livingstone study could block unwelcomed messages online or bookmark a website whilst only a third of respondents were able to block unwanted mails (livingstone et al., 2011). therefore it is crucial for young people to develop an understanding of individual behaviour about cyber safety and empower young people to use critical thinking skills and use technology responsibility to mitigate cybersecurity risks they could face (coventry et al., 2014; yamano, 2004). there are various cybersecurity tools, technologies and best practices that are available to remedy issues relating to cyberbullying. from a cybersecurity perspective, the use of antivirus software and firewalls, the use of unique strong passwords, the installation of the latest security updates, the use of only reliable and secure connections, computers, devices, websites and services are some of the essential practices of cybersecurity to protect young people (coventry et al., 2014). users have to keep up to date about cybersecurity risks, provide a minimum amount of personal information online especially in social media sites, keep users identity secure and protected and report any cyber incidents to the relevant authorities (coventry et al., 2014). the centre for the prevention of violence showed that 70% of young people agreed that blocking cyber friends stopped the abuse, other research also confirmed that respondents reported finding active solutions to cyberbullying such as blocking the harasser (aricak et al., 2008; slonje et al., 2013; smith et al., 2008). the researcher perren et al., (2012) found that avoidance and confrontation are also successful strategies to overcome cyberbullying acts, this is also reinforced by other research that shows victims would avoid online activities in order to stop cyberbullying (hoff & mitchell, 2009; smith et al., 2008). however, encouraging young people to avoid or stay away from technology to stop cyberbullying is shown to be an impractical solutions and overall an unsuccessful long-term strategy (hinduja & patchin, 2009). youth-serving professionals can teach young people the necessary skills and information to respond to the problems of cyberbullying (sabella et al., 2013). besides, an awareness-raising and training programs can be used to increase the awareness of users on how to use technology responsibly (coventry et al., 2014). other researchers also shown that awareness-raising and professional development programs that would guide young people, educators, and parents in navigating the problems of cyberbullying (kowalski et al., 2008; shariff & johnny, 2007; slonje et al., 2013; spears et al., 2014). according to the usa national institute of standards and technology "awareness is not training. the purpose of awareness presentations is simply to focus attention on security. awareness presentations are intended to allow individuals to recognise it security concerns and respond accordingly" (bada et al., 2019). so, it is important to link cybersecurity awareness with other cyber safety matters such as cyberbullying. the second key issue is an educator's responsibility to teach their students the ethical use of technology and online safety skills as young people 's behaviour cannot be controlled by a policy itself (shariff, 2005). cooperative learning techniques, school conferences, consistent disciplinary methods, effective school policies, skilled classroom management by teachers and australasian journal of information systems kaluarachchi, warren & jiang 2020, vol 24, selected papers from the 2019 aice conference cyberbullying among young people 6 peer support programs are some of the successful strategies to encourage positive uses of technology to resolve cyberbullying (myers & cowie, 2019; välimäki et al., 2013). schools and educators offer an ideal environment to improve online safety skills and other necessary skills, such as digital citizenship skills for young people (hinduja & patchin, 2019; välimäki et al., 2013). according to the välimäki et al., (2013) study that looked at 35 national guidelines related to young people skills development found more than 40% of the guidelines highlighted that digital citizenship skills and netiquette are the most critical skills for young people. online safety skills and critical literacy skills are associated with each other and improving one skill may also improve other skills (livingstone et al., 2011). it is important that cyberbullying prevention programs should be incorporated into the school awareness-raising programs and curriculum-based activities (smith et al., 2013). schools can maintain a positive and safe school environment together with awareness-raising, positive school ethos and rewarding positive use of technology (smith et al., 2016; välimäki et al., 2013). further school officials can have the right to restrict student expression online or to discipline students to avoid inappropriate behaviour or speech online (hinduja & patchin, 2011). students should understand that all forms of bullying are wrong and those who are involved in bullying (including cyberbullying) will be subject to discipline punishments (hinduja & patchin, 2019). another key component in the reduction of cyberbullying is to develop clear school policies against cyberbullying (couvillon & ilieva, 2011; hinduja & patchin, 2009; myers & cowie, 2019; perren et al., 2012). school policies and practices should discourage bullying behaviours and encourage positive social and technical skills (välimäki et al., 2013). research has also shown that peer behaviour and social influence has a direct impact on users' behavioural intention (li, he et al. 2019). school counsellors and educators can use students to educate their friends or classmates about the responsible use of technology, they can train younger peers so that they will make wise decisions online from an earlier age (sabella et al., 2013). research has shown that peer support programs or buddy programmes play an essential role in both the intervention and prevention of cyberbullying (myers & cowie, 2019; perren et al., 2012; sabella et al., 2013). the other option schools can implement is to teach cyber ethics to their students through the use of external influences such as honour codes, monitoring software, blocking or filtering software (willard, 2002; yamano, 2004). another key category of prevention strategy is the parent’s responsibility to teach young people about the ethical use of technology and digital skills. parents need to be role models, talking and helping young people when needed and monitoring their ict use; and also parents need to develop their own it skills (lenhart et al., 2011; perren et al., 2012; sabella et al., 2013; united nations children’s fund, 2017; välimäki et al., 2013). according to the välimäki et al., (2013) study, “parents helping young people to use technology in positive ways for enforcing social interactions” was the most acknowledged parental role. other studies such as perren et al., (2012) highlight the importance of raising parents’ knowledge about safe online behaviours and improved their understanding of the technology to reduce the digital divide that possibly exists among parents and young people. parents also need to keep up to date with the technology that young people use to communicate online and where cyberbullying could occur (sabella et al., 2013; välimäki et al., 2013). parents often do not recognise that even though young people are technically more skilled than they are, they might not know how to use technology responsibly or in safe manner (välimäki et al., 2013). parents need to get young people to understand that any form of inappropriate behaviour will not be accepted, and the australasian journal of information systems kaluarachchi, warren & jiang 2020, vol 24, selected papers from the 2019 aice conference cyberbullying among young people 7 use of technology is a privilege, not a right and with such privileges comes responsibilities (hinduja & patchin, 2014). 3.2 what are the web-based or app-specific interventions available to deal with cyberbullying cyberbullying is a social problem that occurs though the undesirable use of it technologies. given the extensive exposure young people have online, it is essential to understand webbased or application (app) specific interventions available to combat cyberbullying. one of the best strategies for ensuring young people’s safety online is by adding parental control software to young people’s digital devices, this empowers parents to manage and limit their online activities (berg & breheny, 2014). parental control software can be used to block or limit access to websites, manage social apps, manage in-app purchasing, monitor online activities, schedule or limit screen time and filter unwanted or sensitive contents. technology examples include family zone®, net nanny®, web watcher®, pc pandora®, family protector®, these are well-known parental control and web filtering software systems which can be used to protect young people from unwanted interactions online (berg & breheny, 2014). many of the antivirus programs that are used also include parental control tools. for instance, norton security premium® is a parental control antivirus programs for families. norton parental control provides tools to see what videos young people watch, what websites they visit, the search terms they use, the apps they download, mobile gps location monitoring and content filtering feature for pcs and other devices (norton, 2020). there are a variety of mobile apps designed to help protect young people from bullying and to create awareness around the issue of cyberbullying. example apps include bullyblock® which is a free android app designed to captures and block bullies. this app allows young people to record verbal bullying and harassment secretly, block unwanted calls, messages and apply automatic respond features and collected information about abusive behaviour that can be shared with parents, teachers or law enforcement (notar et al., 2013; scheff, 2016). bully tag® is another android and apple support mobile app that allows young people to report cyberbullying incidents anonymously to school officials. this app allows young people to record and send incidents to trusted adults with a single click of a button. stopit® is another mobile app that allows young people instantly and anonymously report incidents to school or other trusted adults (scheff, 2016). mcafee safe eyes® is an example of a filtered mobile browsers that helps parents to manage young people’s internet activities on their apple mobile devices (notar et al., 2013). we are now seeing that mobile phone companies are including more features to safeguard young people from cyberbullying. apple has recently introduced ios 12, which includes new features to reduce interruptions, manage screen time and set goals or limit activities. the ios 12 version includes activity reports, app limits and do-not-disturb and notifications controls designed to help parents to reduce interruptions and manage screen time (apple, 2018). these are just a few examples of web-based or app-specific interventions available to deal with cyberbullying. 3.3 ways to use social media responsibly to deal with cyberbullying even though social media is a great tool to stay in touch with friends and family, sometimes it can create ethical issues such as cyberbullying (gonzales, 2014). as discussed before australasian journal of information systems kaluarachchi, warren & jiang 2020, vol 24, selected papers from the 2019 aice conference cyberbullying among young people 8 cyberbullies use social media because it is so widely used. this form of bullying distresses young people and adversely affects their families, teachers, and others around them (elci & seckin, 2016). the pressure of social media is real and has a consequence for all users (esafety commissioner, 2020). social media users should have an in-depth understanding of how to use social media, the features and capabilities as well as potential effects when creating a social media account (gonzales, 2014). some common strategies for young people to stay safe in the social media space are; unfollow the social media accounts of people that they feel unsure about, turn off message notifications and have the ability leave unwanted social media contacts (e.g. to leave a chat session with an unknown person); talk to a person of responsibility (a parent or guardian) if they feel pressured by someone on social media, remove or report harmful social media, strength individual privacy settings and limiting the amount of personal information share via social media (carter, 2013; esafety commissioner, 2020; gonzales, 2014; lenhart et al., 2011). it is very important for young people to develop self-discipline when using social media to avoid unethical situations (gonzales, 2014). there are also issues due to lack of awareness. vulnerable and disadvantaged young people may be less likely to understand the complexity of the systems they are using (cole, 2014; united nations children’s fund, 2017). social media literacy campaigns also good strategy to address the problems of cyberbullying (gonzales, 2014). we are seeing that social media organisations are doing more to tackle the issue of cyberbullying via their platforms. facebook is the largest online community and the facebook “help centre” includes information on cyberbullying and their “bullying prevention hub” provides resources and tips for young people, parents, and educators to prevent cyberbullying (facebook, 2020a). the facebook help centre consists of a series of information sources related to bullying including "what should they do if being bullied, harassed or attacked by someone on facebook", "how to remove users from abusive tags", "unfriending and blocking facebook users”, and “how to report abusive content”. facebook also provides tips and helpful resources for parents, guardians, and educators as to how they can help young people to use facebook wisely. further, facebook has also introduced suicide prevention tools in a global context (berg & breheny, 2014; facebook, 2019). in addition, facebook 'digital literacy library' provide lesson plans specially designed for young people to develop their skills in relation to privacy and reputation, identity exploration, positive behaviour, security, and community engagement modules. (facebook, 2020b). similarly, the twitter “help centre” provides support for its users by providing safety and security features to deal with online abuse and bullying. the help centre comprises of information on privacy controls including “how to protect personal information”, “how to deal with spam and fake accounts and sensitive content” and “how to report abusive behaviour including helping someone with online abuse” (berg & breheny, 2014; twitter, 2019). 4 discussion the narrative review presented in this paper reviewed a significant number of studies on cyberbullying relating to young people. most of the studies focus on the use of technology, online risky behaviours, ethical or responsible use of technologies to prevent and respond to cyberbullying. the following table 1 (based upon the narrative review) presents a summary australasian journal of information systems kaluarachchi, warren & jiang 2020, vol 24, selected papers from the 2019 aice conference cyberbullying among young people 9 of the actions, ongoing tools, technologies, and best practices that can help young people to protect themselves against cyberbullying. actions, tools, technologies, and best practices related to responsible use of technology. example actions or activities source who is responsible? improvement of online safety skills, critical thinking skills, best practices, and netiquette. use only reliable and secure connections, computers, devices, websites, and services; share a minimum amount of personal information online; keep young people’s identity secure and protected; implementing good cyber security measures. coventry et al., 2014; couvillon & ilieva, 2011; sabella et al., 2013; livingstone et al., 2011; välimäki et al., 2013; yamano, 2004. young people, parents, and educators. technology-based solutions. blocking unwanted cyber friends; blocking annoying messages; remove harmful/anonymous messages or content. aricak et al., 2008; coventry et al., 2014; hoff & mitchell, 2009; sabella et al., 2013; slonje et al., 2013; smith et al., 2008; yamano, 2004. young people. reporting of cyber incidents. report cyber incidents to parents, teachers, content providers (e.g. facebook help centre, twitter help centre) or call law enforcement. coventry et al., 2014; hinduja & patchin, 2019; slonje et al., 2013; facebook, 2019; 2020a; twitter, 2019. young people, parents, and educators. cyber safety awareness / training / development programs. digital literacy courses; social media literacy campaigns; cyber safety training. angus, 2016; bada et al., 2019; coventry et al., 2014; elci & seckin, 2016; gonzales, 2014; slonje et al., 2013; spears et al., 2014; shariff & johnny, 2007; kowalski et al., 2008. educators and other youth advocates. cyberbullying prevention programs in schools. cooperative learning techniques; consistent disciplinary methods; effective school policies; skilled classroom management by teachers; peer support programs; school focused awareness programs; curriculum based activities. myers & cowie, 2019; couvillon & ilieva, 2011; hinduja & patchin, 2009, 2011, 2019; livingstone et al., 2011; perren et al., 2012, shariff, 2005; sabella et al., 2013; slonje et al., 2013; smith et al., 2013; 2016; välimäki et al., 2013; yamano, 2004. educators. educate students about proper conduct in the cyber world. cyber ethics education; digital citizenship education; education on the responsibilities and ethical use of technology. angus, 2016; cole, 2014; crystal et al., 2000; couvillon & ilieva, 2011; gonzales, 2014; hinduja & patchin, 2014, 2019; sabella et al., 2013; välimäki et al., 2013; yamano 2004. parents and educators. australasian journal of information systems kaluarachchi, warren & jiang 2020, vol 24, selected papers from the 2019 aice conference cyberbullying among young people 10 actions, tools, technologies, and best practices related to responsible use of technology. example actions or activities source who is responsible? parental oversight. parents need to be cyber safety role models; parents need to be able to offer advice to young people on cyber safety; parents need to monitor young people’s it and social media usage. perren et al., 2012; hinduja & patchin, 2014; sabella et al., 2013; couvillon & ilieva, 2011; lenhart et al., 2011; välimäki et al., 2013; united nations children’s fund, 2017. parents. parent and educator’s education. awareness programs about new technologies and ethical behaviours; awareness programs about effective safe online behaviours. sabella et al., 2013; perren et al., 2012; välimäki et al., 2013. parents and educators. web-based or appspecific interventions. promotion of the use of parental control software, antivirus programs, mobile apps with special safety features. apple 2018; berg & breheny, 2014; coventry et al., 2014; notar et al., 2013; scheff, 2016; yamano, 2004. young people, parents, and educators. responsible use of social media. development of effective social media usage strategies to deal with cyber bullying. carter, 2013; gonzales, 2014; lenhart et al., 2011; esafety commissioner, 2020; facebook, 2019, 2020a, 2020b; twitter, 2019. young people, parents, and educators. promotion of resources from social media companies. promotion of social media platforms resources and activities, e.g. facebook, twitter cyber safety forums. berg & breheny, 2014; facebook, 2019, 2020a, 2020b; twitter, 2019. social media companies, educators, parents, and other youth advocates. table 1: actions, ongoing tools, technologies, and best practices to prevent and respond to cyberbullying. several conclusions can be drawn from this narrative review presented in this paper with regards to the cyberbullying issues facing young people. the issues identified are: 1) the existing research has shown that there are a number of preventive strategies that can be used to combat cyberbullying for young people; 2) there are fewer studies published relating to the ethical or responsible use of technology to combat cyberbullying (this could be a potential limitation to the study presented in this paper); 3) most of the cyberbullying current studies revealed that the education and youth skill development are a top priority to deal with cyberbullying among young people; australasian journal of information systems kaluarachchi, warren & jiang 2020, vol 24, selected papers from the 2019 aice conference cyberbullying among young people 11 4) schools have a key role to educate young people about online safety skills, critical thinking skills and netiquette; 5) parents and educators should teach young people about cyber ethics, digital citizenship responsibilities, as well as the positive or ethical use of technology; 6) the training of young people should start from an early age to assist in their development; 7) the importance of increasing parents and educator knowledge about cyber safety and associated technologies; this paper has shown that more diverse studies are needed to minimise cyberbullying because the main social media organisations and it organisations have resources and systems but there are limited studies to show their impact or effectiveness. 5 conclusion, limitations and future directions as mentioned before, cyberbullying is an intricate and ever-evolving form of bullying. cyberbullying has the ability to impact young peoples' health and psychological well-being in numerous ways. most of the existing cyberbullying research largely focuses on prevalence and predictors of cyberbullying and the comparison of traditional bullying and cyberbullying. we have seen that less attention has been given to the ethical or responsible use of technology to prevent cyberbullying. in this paper, we have proposed a number of actions, tools, technologies, and best practices related to responsible use of technology that can be used to mitigate the risks associated with cyberbullying. schools offer an ideal environment, to help young people improve their digital skills, critical thinking skills and netiquettes needed to navigate in the digital world in a safe manner. the cyberbullying interventions have not been successful if they have not reviewed the existing literature explicitly. these claims are elucidated using current literature so that government agencies, policymakers, educators, and other practitioners will have a better understanding of responsible use of technology and understanding of cyberbullying in order to support young people to combat this immerging societal challenge. the research presented in this paper has been limited by the small number of studies conducted relating to responsible use of technology and cyberbullying. the future research direction is to focus on what can be done to protect against cyberbullying and what protection steps can be implemented to protect young people. references aboujaoude, e., savage, m. w., starcevic, v., & salame, w. o. (2015). cyberbullying: review of an old problem gone viral. journal of adolescent health, 57(1), 10-18. doi:https://doi.org/10.1016/j.jadohealth.2015.04.011 angus, c. (2016). cyberbullying of children. nsw parliamentary research service retrieved from https://www.parliament.nsw.gov.au/researchpapers/documents/cyberbullyingof-children/cyberbullying%20of%20children.pdf australasian journal of information systems kaluarachchi, warren & jiang 2020, vol 24, selected papers from the 2019 aice conference cyberbullying among young people 12 apple. (2018). ios 12 introduces new features to reduce interruptions and manage screen time. newsroom. retrieved from https://www.apple.com/au/newsroom/2018/06/ios-12introduces-new-features-to-reduce-interruptions-and-manage-screen-time/ aricak, t., siyahhan, s., uzunhasanoglu, a., saribeyoglu, s., ciplak, s., yilmaz, n., & memmedov, c. (2008). cyberbullying among turkish adolescents. cyberpsychology & behavior: the impact of the internet, multimedia and virtual reality on behavior and society, 11, 253-261. doi:10.1089/cpb.2007.0016 bada, m., sasse, a. m., & nurse, j. r. c. (2019). cyber security awareness campaigns: why do they fail to change behaviour? arxiv e-prints. barlett, c. p. (2015). anonymously hurting others online: the effect of anonymity on cyberbullying frequency. psychology of popular media culture, 4(2), 70-79. doi:10.1037/a0034335 berg, c., & breheny, s. (2014). a social problem, not a technological problem, bullying, cyberbullying and public policy. melbourne: institute of public affairs retrieved from https://ipa.org.au/wp-content/uploads/archive/submission_v6.pdf berger, k. (2007). update on bullying at school: science forgotten? developmental review, 27, 90-126. doi:10.1016/j.dr.2006.08.002 berkowitz, m. (2000) itaa report. paper presented at the national conference on cyber ethics, arlington, va., usa. calvete, e., orue, i., estévez, a., villardón, l., & padilla, p. (2010). cyberbullying in adolescents: modalities and aggressors’ profile. computers in human behavior, 26, 11281135. doi:10.1016/j.chb.2010.03.017 carter, m. a. (2013). protecting oneself from cyber bullying on social media sites – a study of undergraduate students. procedia social and behavioral sciences, 93, 1229-1235. doi:https://doi.org/10.1016/j.sbspro.2013.10.020 cole, a. (2014). the digital world: are our children ready? (m.s.), utica college, ann arbor. proquest dissertations & theses global database. (1570361)dissertations & theses global database. (1570361) couvillon, m. a., & ilieva, v. (2011). recommended practices: a review of schoolwide preventative programs and strategies on cyberbullying. preventing school failure: alternative education for children and youth, 55(2), 96-101. doi:10.1080/1045988x.2011.539461 coventry, l., briggs, p., blythe, j., & tran, m. (2014). using behavioural insights to improve the public’s use of cyber security best practices. london,uk: government office for science retrieved from https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachm ent_data/file/309652/14-835-cyber-security-behavioural-insights.pdf cross, d., shaw, t., hearn, l., epstein, m., monks, h., lester, l., & thomas, l. (2009). australian covert bullying prevalence study (acbps). perth crystal, j., geide, c. a., salpeter, j., abight, k., & kuzas, k. j. (2000). the concerned educator's guide to safety and cyber-ethics. technology & learning, 21(4), 24-31. australasian journal of information systems kaluarachchi, warren & jiang 2020, vol 24, selected papers from the 2019 aice conference cyberbullying among young people 13 elci, a., & seckin, z. (2016). cyberbullying awareness for mitigating consequences in higher education. journal of interpersonal violence, 34. doi:10.1177/0886260516646095 esafety commissioner. (2020). pressures from social media. retrieved from https://www.esafety.gov.au/young-people/pressures-from-social-media facebook. (2019). put a stop to bullying. bullying prevention hub. retrieved from https://www.facebook.com/safety/bullying facebook. (2020a). help centre. retrieved from https://www.facebook.com/help facebook. (2020b). welcome to the digital literacy library. digital literacy library. retrieved from https://www.facebook.com/safety/educators feinberg, t., & robey, n. (2008). cyberbullying. principal leadership (middle school edition), 9(1), 10-14. ferrari, r. (2015). writing narrative style literature reviews. medical writing, 24, 230-235. doi:10.1179/2047480615z.000000000329 gonzales, r. h. (2014). social media as a channel and its implications on cyber bullying. paper presented at the dlsu research congress. hinduja, s., & patchin, j. (2011). cyberbullying: a review of the legal issues facing educators. preventing school failure: alternative education for children and youth, 55, 71-78. doi:10.1080/1045988x.2011.539433 hinduja, s., & patchin, j. w. (2019). cyberbullying identification, prevention, and response. in: cyberbullying research center (cyberbullying.org). hinduja, s., & patchin, j. w. (2009). bullying beyond the schoolyard: preventing and responding to cyberbullying. thousand oaks, ca: sage publications (corwin press). hinduja, s., & patchin, j. w. (2010). bullying, cyberbullying, and suicide. archives of suicide research, 14(3), 206-221. doi:10.1080/13811118.2010.494133 hinduja, s. & patchin, j. w. (2014). cyberbullying identification, prevention, and response. cyberbullying research center (www.cyberbullying.us). hoff, d., & mitchell, s. (2009). cyberbullying: causes, effects, and remedies. journal of educational administration, 47, 652-665. doi:10.1108/09578230910981107 huang, q., inkpen, d., zhang, j., & van bruwaene, d. (2018). cyberbullying intervention based on convolutional neural networks. paper presented at the proceedings of the first workshop on trolling, aggression and cyberbullying (trac-2018). iris research. (2014). estimates of cyber-bullying incidents dealt with by australian schools. canberra act 2601, australia retrieved from https://www.communications.gov.au/publications/estimates-cyber-bullying-incidentsdealt-australian-schools-%e2%80%93-iris-research-june-2014 kaltiala-heino, r., rimpelä, m., marttunen, m., rimpelä, a., & rantanen, p. (1999). bullying, depression, and suicidal ideation in finnish adolescents: school survey. bmj (clinical research ed.), 319(7206), 348-351. doi:10.1136/bmj.319.7206.348 australasian journal of information systems kaluarachchi, warren & jiang 2020, vol 24, selected papers from the 2019 aice conference cyberbullying among young people 14 kowalski, r., & fedina, c. (2011). cyber bullying in adhd and asperger syndrome populations. research in autism spectrum disorders res autism spectr disord, 5, 1201-1208. doi:10.1016/j.rasd.2011.01.007 kowalski, r. m., limber, s., & agatston, p. w. (2012). cyberbullying: bullying in the digital age. chichester, west sussex, uk: : wiley-blackwell. kowalski, r. m., & limber, s. p. (2007). electronic bullying among middle school students. journal of adolescent health, 41(6), s22-s30. doi:10.1016/j.jadohealth.2007.08.017 kowalski, r. m., limber, s. p., & agatston, p. w. (2008). cyberbullying: bullying in the digital age. malden, ma: : blackwell publishing. kowalski, r. m., limber, s. p., & mccord, a. (2019). a developmental approach to cyberbullying: prevalence and protective factors. aggression and violent behavior, 45, 2032. doi:10.1016/j.avb.2018.02.009 lai, l., & turban, e. (2008). groups formation and operations in the web 2.0 environment and social networks. group decision and negotiation, 17, 387. doi:10.1007/s10726-0089113-2 lee, e. b. (2017). cyberbullying: prevalence and predictors among african american young adults. journal of black studies, 48(1), 57-73. doi:10.1177/0021934716678393 lenhart, a., madden, m., smith, a., purcell, k., & zickuhr, k. (2011). teens, kindness and cruelty on social network sites. in (pp. pp. 1–86). washington, dc: pew research center’s internet & american life project. li, c. (2010). groundswell. winning in a world transformed by social technologies. strategic direction, 26. doi:10.1108/sd.2010.05626hae.002 livingstone, s., haddon, l., goerzig, a., & ólafsson, k. (2011). risks and safety on the internet: the perspective of european children: full findings and policy implications from the eu kids online survey of 9–16 year olds and their parents in 25 countries. in. london, uk: eu kids online. london school of economics & political science. lu, h.-p., & hsiao, k.-l. (2010). the influence of extro/introversion on the intention to pay for social networking sites. information & management, 47, 150-157. doi:10.1016/j.im.2010.01.003 lund, e. m., & ross, s. w. (2017). bullying perpetration, victimization, and demographic differences in college students: a review of the literature. trauma, violence, & abuse, 18(3), 348-360. doi:10.1177/1524838015620818 myers, c.-a., & cowie, h. (2019). cyberbullying across the lifespan of education: issues and interventions from school to university. international journal of environmental research and public health, 16(7), 1217. doi:10.3390/ijerph16071217 niyazi, ö., celal teyyar, u., & kadir, b. (2011). computer teachers’ attitudes toward ethical use of computers in elementary schools. international journal of cyber ethics in education (ijcee), 1(2), 15-24. doi:10.4018/ijcee.2011040102 norton. (2020). norton antivirus plus, parental control. nortan products & services. retrieved from https://au.norton.com/ australasian journal of information systems kaluarachchi, warren & jiang 2020, vol 24, selected papers from the 2019 aice conference cyberbullying among young people 15 notar, c. e., padgett, s., & roden, j. (2013). cyberbullying: resources for intervention and prevention. universal journal of educational research, 1(3), 133-145. olweus, d. (2013). school bullying: development and some important challenges. annual review of clinical psychology, 9(1), 751-780. doi:10.1146/annurev-clinpsy-050212-185516 perren, s., corcoran, l., cowie, h., dehue, f., garcia, d. j., mc guckin, c., . . . völlink, t. (2012). tackling cyberbullying: review of empirical evidence regarding successful responses by students, parents, and schools. international journal of conflict and violence, 6, 283. doi:10.4119/ijcv-2919 raskauskas, j., & huynh, a. (2015). the process of coping with cyberbullying: a systematic review. aggression and violent behavior, 23. doi:10.1016/j.avb.2015.05.019 sabella, r., patchin, j., & hinduja, s. (2013). review: cyberbullying myths and realities. computers in human behavior, 29, 2703-2711. doi:10.1016/j.chb.2013.06.040 sahin, m. (2012). the relationship between the cyberbullying/cybervictmization and loneliness among adolescents. children and youth services review, 34, 834–837. doi:10.1016/j.childyouth.2012.01.010 scheff, s. (2016). 7 apps for national bulling prevention month. retrieved from http://www.suescheffblog.com/7-apps-national-bullying-prevention-month/ shariff, s. (2005). cyber-dilemmas in the new millennium: school obligations to provide student safety in a virtual school environment. mcgill journal of education, 40(3). shariff, s., & johnny, l. (2007). cyber-libel and cyber bullying: can schools protect student reputations and free expression in virtual environments? education law journal, 16, 307342. slonje, r., smith, p. k., & frisén, a. (2013). the nature of cyberbullying, and strategies for prevention. computers in human behavior, 29(1), 26-32. doi:https://doi.org/10.1016/j.chb.2012.05.024 smith, j. a., & yoon, j. (2012). cyberbullying presence, extent, & forms in a midwestern postsecondary institution. proceedings of the information systems education conference, isecon, 29. smith, p., steffgen, g., & sittichai, r. (2013). the nature of cyberbullying, and an international network. in p. smith & g. steffgen (eds.), cyberbullying through the new media: findings from an international network (pp. 3-19). london: psychology press. smith, p., thompson, f., craig, w., hong, i., slee, p., sullivan, k., & green, v. (2016). actions to prevent bullying in western countries. in p. smith, k. kwak, & y. toda (eds.), school bullying in different cultures: eastern and western perspectives (pp. 301-333). cambridge: cambridge university press. doi:10.1017/cbo9781139410878.018 smith, p. k., mahdavi, j., carvalho, m., fisher, s., russell, s., & tippett, n. (2008). cyberbullying: its nature and impact in secondary school pupils. journal of child psychology and psychiatry, 49(4), 376-385. doi:10.1111/j.1469-7610.2007.01846.x so, a., & lenarcic, j. (2014). cross-cultural perspectives on cyberbullying through an ethical lens (pending publication). australasian journal of information systems kaluarachchi, warren & jiang 2020, vol 24, selected papers from the 2019 aice conference cyberbullying among young people 16 sourander, a., brunstein klomek, a., ikonen, m., lindroos, j., luntamo, t., koskelainen, m., . . . helenius, h. (2010). psychosocial risk factors associated with cyberbullying among adolescents: a population-based study. archives of general psychiatry, 67, 720-728. doi:10.1001/archgenpsychiatry.2010.79 spears, b., keeley, m., bates, s., & katz, i. (2014). research on youth exposure to, and management of, cyberbullying incidents in australia: part a ‒ literature review on the estimated prevalence of cyberbullying involving australian minors (sprc report 9/2014). in. sydney: social policy research centre, unsw australia. stacey, e. (2009). research into cyberbullying: student perspectives on cybersafe learning environments. informatics in education-an international journal, 8(1), 115-130. twitter. (2019). everything you need to know so you can use twitter like a pro. help center. retrieved from https://help.twitter.com/en tynes, b. m., rose, c. a., & williams, d. r. (2010). the development and validation of the online victimization scale for adolescents. cyberpsychology: journal of psychosocial research on cyberspace, 4(2), 1-15. united nations children’s fund. (2017). the state of the world’s children 2017, children in a digital world. new york, usa: division of communication retrieved from https://www.unicef.org/publications/files/sowc_2017_eng_web.pdf välimäki, m., almeida, a., cross, d., fandrem, h., o’moore, m., berne, s., . . . sygkollitou, e. (2013). guidelines for preventing cyber-bullying in the school environment: a review and recommendations in cyberbullying: coping with negative and enhancing positive uses of new technologies, in relationships in educational settings. retrieved from https://sites.google.com/site/costis0801/guideline walrave, m., & heirman, w. (2011). cyberbullying: predicting victimisation and perpetration. children & society, 25(1), 59-72. doi:10.1111/j.1099-0860.2009.00260.x willard, n. e. (2002). computer ethics, etiquette, and safety for the 21st-century student: international society for technology in education. williams, k. r., & guerra, n. g. (2007). prevalence and predictors of internet bullying. journal of adolescent health, 41(6), s14-s21. doi:10.1016/j.jadohealth.2007.08.018 wong-lo, m., & bullock, l. (2011). digital aggression: cyberworld meets school bullies. preventing school failure: alternative education for children and youth, 55, 64-70. doi:10.1080/1045988x.2011.539429 yamano, p. h. (2004). cyberethics in the elementary classroom: teaching the responsible use of technology. (ph.d. thesis), pepperdine university, retrieved from https://www.learntechlib.org/p/119082 ybarra, m. l., diener-west, m., & leaf, p. j. (2007). examining the overlap in internet harassment and school bullying: implications for school intervention. journal of adolescent health, 41(6), s42-s50. doi:10.1016/j.jadohealth.2007.09.004 australasian journal of information systems kaluarachchi, warren & jiang 2020, vol 24, selected papers from the 2019 aice conference cyberbullying among young people 17 copyright: © 2020 kaluarachchi, warren & jiang. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v24i0.2791 http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems clarke 2020, vol 24, selected papers from the is foundations conference establishing a research technique 1 the challenges involved in establishing a research technique roger clarke principal, xamax consultancy roger.clarke@xamax.com.au abstract many research techniques are well-accepted within the information systems (is) discipline. from time to time, however, a researcher investigates a question that requires a novel approach. it is then incumbent on the researcher to justify that approach. the is discipline has accumulated a large corpus of published works. a project is being undertaken whose purpose is to establish the conceptual foundations for a research technique for the critical analysis of published works, and to develop methodological guidance for its application. this article discusses the challenges that have confronted that undertaking. keywords: research techniques; research methods; research methodology; critical analysis of published works; article critique; critical content analysis 1 introduction since the information systems (is) discipline emerged c. 1965, a substantial body of publications has accumulated. for example, in mid-2019, the ais elibrary contains over 16,000 refereed works, john lamp’s directory of is journals identifies almost 700 active venues that publish is works, of the order of 7,500 articles have been published in the ‘basket of 8’ journals alone, and the major five is conferences alone publish an additional 1,500 refereed papers per annum. the accumulated is literature evidences two particularly strong desires. a great deal of emphasis is placed on empirical research involving the observation and measurement of some aspect of is practice. in addition, many papers are published that seek to establish or extend theories about is practices, which can then provide a basis for further empirical research. important though these approaches are, progress in the discipline depends on some further elements as well. meta-discussions are needed, in order to clarify such aspects as the discipline’s scope, the meanings of key terms, and the suitability of particular research techniques for particular purposes. the project reported on in this article is a contribution to one such meta-discussion. the starting-point for the project reported on here is a conviction that critical thought about the existing is literature has a vital role to play. criticism is, after all, central to the notion of science, in that all propositions must be in principle refutable, all propositions must be regarded as provisional, and all propositions must be subjected to testing (popper 1963). for this reason, criticism is inherent in the review process whereby works are evaluated as a condition of publication in refereed venues. the is discipline has to date demonstrated considerable nervousness about works that contain criticism of prior publications, and especially about works whose specific purpose is to criticise prior publications. this project is a response to a particular instance of this nervousness. i australasian journal of information systems clarke 2020, vol 24, selected papers from the is foundations conference establishing a research technique 2 conducted critical analysis of the articles in a special issue published by a leading, researchdomain-specific journal that publishes many papers relevant to the is discipline. i submitted the paper to the same journal as published the special issue. it was rejected, with a key factor being that the research technique adopted in the research was regarded by the reviewers as being illegitimate. it was subsequently also rejected by an is 'basket of 8' journal, but with an invitation to split the work into two articles, further develop the methodological component, and re-submit the substantive article as and when the methodological article achieves acceptance. although the conduct and publication of critiques of prior works are vital to progress in the discipline, the research methodology literature does not appear to currently provide a sufficient basis for work of this nature. a project has accordingly been undertaken to identify the necessary conceptual foundations, and develop a research technique for the critical analysis of published works. the present article provides an overview of the project as a whole, with a focus on the challenges that have been encountered, and the difficulties that need to be overcome in order to establish the technique. the article commences by discussing methodological considerations in the establishment of a research technique. the nature and role of criticism in research and the relevance of critical theory research are considered. the article then identifies multiple existing research techniques that have relevance to this specific purpose, in order to enable conceptualisation of the new technique. the key contribution of this article is the identification of challenges that need to be overcome in order to establish a new technique within the is discipline. 2 methodological considerations a research technique needs to reflect the accumulated knowledge about research methods generally, and to be applicable to some particular category of research questions. this section addresses those foundational aspects of the project. 2.1 definition of terms and scope the is literature evidences considerable looseness in its use of key terms relevant to this work. the term 'research methodology' is particularly badly abused. this article uses the following terms carefully, in a manner consistent with kaplan (1964, pp. 18-19): • 'research technique' refers to a specific process (such as structured interview, case study or action research); • 'research method' refers to the process adopted in a particular research project, which comprises one or more research techniques customised to the specific purpose; and • 'research methodology' refers to the disciplined study of research methods. it is not used as a synonym for either or both of ‘research technique’ or ‘research method’. the first challenge in devising a new research technique is to achieve clarity about the kinds of research that it is intended to address. the objective of the present project is defined as: the development of a research technique for the critical analysis of the content of published works relevant to the is discipline. australasian journal of information systems clarke 2020, vol 24, selected papers from the is foundations conference establishing a research technique 3 the term 'published works' (short form 'works') is intended to encompass entries in a wide variety of 'publishing venues', including at least refereed articles in journals, refereed papers in conference proceedings, refereed chapters in academic books, refereed academic books as a whole, and completed postgraduate dissertations. the works and venues might reasonably be extended to papers in workshops and symposia, and for some purposes may include preprints or working papers published by institutions with appropriate standing. in some circumstances, a case could be made for encompassing research reports commissioned by government agencies, foundations, industry associations and perhaps individual corporations. some other forms of publication may be relevant, depending on the specific research purpose. for example, technical media or corporate white papers might be included, if the research purpose were to, for example, assess the impact of academic work on thinking among consultancies or within industry sectors. there are multiple forms of research question to which the proposed technique might be applied. table 1 contains several examples, intended to provide a sense of the scope of application. the term 'content' refers to the text of the published work, encompassing sentences, tables, diagrams and formulae, and taking into account the context within which the work is situated. depending on the nature of the work, the context is likely to include aspects of the research domain, the academic discipline and cultural factors. which publishing venues are 'relevant to the is discipline' depends on the nature of the project. beyond mainstream is journals and conferences, for example, venues that publish is works may be relevant, as may venues used by particular reference and cognate disciplines, e.g. in management, social sciences, or computer sciences. methodological critique: 1. does each work describe the research method at a sufficient level of detail to enable the reader to evaluate the research quality? 2. does each work consider the suitability of the data to the data analysis techniques that are applied to the data, or does it merely assume suitability? 3. does each work provide access to sufficient supporting materials to enable audit? critique of the theoretical base: 4. does each work reflect major prior publications in the area? 5. is the scope of publications that are cited in each work's literature review sufficiently comprehensive? 6. to what extent does each work appropriately represent and apply the publications that it cites? substantive critique: 7. in each work, what unstated assumptions can be detected? 8. does each work address the issue of heterogeneity and instability in the relevant phenomena? 9. does each work evidence an understanding of the power relationships within the research domain? 10. to what extent does each work reflect the interests of sub-dominant stakeholders? table 1: sample research questions australasian journal of information systems clarke 2020, vol 24, selected papers from the is foundations conference establishing a research technique 4 the scope-definition for particular studies will vary considerably. some might consider the complete corpus of 'published works relevant to the is discipline'. the resource-intensiveness of the technique is such, however, that it is far more likely that each project’s focus will be on a particular population segment, with a sampling frame and sample selected from within that segment. a segment might be defined to be a particular venue (e.g. the complete set of icis proceedings), or a time-span within one or more venues (e.g. the last 10 years of the ‘basket of 8’ is journals), or a focussed collection (e.g. one or more journal special issues, narrowlyspecialised conferences, or academic books). other approaches would be to define the population segment on the basis of the research domain that the works address, the theoretical lens that they use, or the research technique that they apply. the smallest sample-size, a single work, would only appear likely to be appropriate under very particular circumstances, such as papers that are widely-regarded as being highly significant, in particular on the basis of citation-count or some other measure of influence, or are highly original. 2.2 meta-methodology a second set of challenges arises in relation to the desirable characteristics of a method for the creation of a method – hence 'meta-methodology'. a research technique is a socio-technical artefact as that term is used in design science (niedermann & march 2012, gregor & hevner 2013, p.337). the design science approach is accordingly applicable to the project, and the process needs to reflect peffers' design science research methodology (dsrm), with problem identification and definition of objectives followed by design and development (peffers et al. 2007). however, as discussed in the later parts of this article, the later steps of drsm – demonstration and evaluation – are challenges that the present project has not yet met. on the other hand, a research technique is a very particular kind of socio-technical artefact, and it is desirable that the process adopted in this project be founded on generic guidance in relation to the establishment of a new research technique. sources of the nature of metamethodology have proven to be elusive, however. for example, scans of the large library of sage publications, and searches on terms such as <meta-methodology> and <research technique design> came up largely empty-handed. within the is literature, it is uncommon to encounter guidance. for example, a foundational article on positivist case study technique (benbasat et al. 1987) commences with a set of assertions of 'key characteristics of case studies', without any explanation of their source. a corresponding authority on interpretive case studies (walsham 1995), discusses differences in the epistemological and ontological stances of the two approaches, and the role of theory, but offers little in relation to the process adopted and appropriate criteria for assessing the quality of the methodological guidance that the article offers. articles that provide guidance in relation to various forms of action research (avison 2002, davison et al. 2004) limit the criteria for judging the quality of a research technique to rigour and relevance. klein & myers (1999) offers principles for conducting "interpretive research of a hermeneutic nature" and presents quality criteria for that particular category of research technique, but it does not declare the process and the quality criteria that guided the authors' work in constructing that guidance. in the area of systematic literature reviews, okoli & schabram (2010) adopt the criteria used by fink (2005): there must be a process that is systematic, is the subject of an explicit australasian journal of information systems clarke 2020, vol 24, selected papers from the is foundations conference establishing a research technique 5 explanation, and is reproducible by others, and the scope of works examined must be comprehensive. boell & cecez-kecmanovic (2014) adds to that list "engagement with the literature [by means of] an ongoing hermeneutic process of developing understanding" (p.259). in is-cognate literature, bonoma (1985), a work cited in benbasat et al. (1987), posited two criteria for "sound" research methods in social science. one is "'data integrity' ... those characteristics of research that affect error and bias in research results. it is an amalgam of what is variously referred to as 'internal validity' ..., 'statistical conclusion validity', and 'reliability'". the other is what the author calls "'currency' ... the characteristics of research that affect the contextual relevance of findings across measures, methods, persons, settings, and time. it is an amalgam of what is variously termed 'external validity' ... and 'pragmatic' or 'ecological validity'" (p.200). in practice, the author notes, there are conflicts between the two sets of objectives, and trade-off is unavoidable, i.e. all research techniques are, by their nature, inevitably flawed. "ideally, a researcher can simultaneously pursue high levels of data validity and generalizability by adopting triangulation strategies which provide replication and/or corroboration of findings across methods within a single research project" (p.201). the author concedes, however, that 'intra-project triangulation' is very challenging, and that 'inter-project triangulation' is more common. the appropriateness of particular techniques is related to the current state of knowledge in the particular field. for example, the author positions case research very low on the data integrity scale but high on the currency scale, and indicates its appropriateness where "theoretical development is scant or uncertain" (p.201). alvesson & sandberg (2011) proposed a research technique for 'problematisation', the first part of which is for "identifying and challenging assumptions underlying existing literature" (p.247). they shift the conversation beyond merely identifying "gaps in the literature" (which 'underproblematises' the prior literature, by reinforcing rather than challenging existing theories). they note disadvantages of disruptive approaches that focus on what is wrong with existing knowledge (which 'overproblematises' the prior literature). in seeking a middle road, they propose a five-set typology of assumptions that can be subjected to challenge. in their terms, 'paradigmatic assumptions' encompass ontological, epistemological and methodological aspects. political, moraland gender-related factors are 'ideology assumptions'; 'in-house assumptions' are those of a school of thought; 'field assumptions' are shared with other schools of thought in the particular field; and 'root metaphor assumptions' are associated with the imagery common in the relevant area of research. the quality criteria that they use for their own work appear to be change without bloodshed, and the generation of interesting new research questions. on the basis of these varied exemplars, i posit the following as appropriate guidance in relation to the process of developing the new research technique: • recognise a threshold requirement of rigour, reflecting conventional characteristics of quality research, such as express statements of a systematic process and desirable characteristics of that process; controls over observer bias; traceability, auditability and to the extent feasible reproducibility; and sufficiently deep and reflective engagement with the literature • recognise a threshold requirement of relevance to an adequately defined context and target audience australasian journal of information systems clarke 2020, vol 24, selected papers from the is foundations conference establishing a research technique 6 • adopt the motivation of positive or constructive challenge to prior publications, rather than of negative or destructive disruption or confrontation as regards the product that this undertaking seeks to deliver, i posit that a body of knowledge about a research technique comprises the following elements: contextual information; conceptual foundations; guidance in relation to the process to be used; exemplars of processes that have been articulated in order to address particular kinds of research questions; and expositions of the technique in use. the body of this article addresses the first three of those elements and includes preliminary discussion of the last two. 3 criticism the challenge addressed in this section is to clarify key features of the category of research questions for which the new technique is being devised. the focus of this project is the critical analysis of published works. two aspects of the notion of criticism are particularly relevant. the first is the role that criticism plays in scientific research generally and in is research in particular. the second is the genre of critical theory research. 3.1 the role of criticism in research the purpose of undertaking content analysis may be simply exposition, that is to say the identification, extraction and summarisation of content, without any significant degree of evaluation. there are benefits in undertaking content analysis in a positive frame of mind, and in assuming that all that has to be done is to present existing information in brief and readilyaccessible form (as, for example, much of the present article does). alternatively, the researcher may bring a questioning and even sceptical attitude to the work. a common purpose of literature reviews is to depict the current state of theory in an area. the purpose may be to draw inferences for the particular context relevant to the project. alternatively, it may be to identify gaps in the present body of theory. gap-identification is a gentle form of criticism in the sense in which the term is used in this article. alvesson & sandberg (2011) distinguishes three modes of gap-spotting: confusion-spotting (where competing explanations exist); neglect-spotting (an area that is overlooked or underresearched, or where empirical support is lacking); and application-spotting (extending and complementing existing literature by drawing further inferences that can be tested). the notion of criticism goes further than merely identifying previously under-researched corners of theory. a critic asks hard questions that necessarily cut to the core of academic work. is it reasonable to assume that all relevant published literature is of high quality? that the measurement instruments and research techniques have always been good, well-understood by researchers, and appropriately applied? that there have been no material changes in the relevant phenomena? and that there have been no material changes in the intellectual contexts within which research is undertaken? the term 'criticism' is often used in a pejorative sense, implying that the critic is merely finding fault, is being destructive rather than constructive, and is failing to propose improvements to overcome the faults while sustaining the merits. the sense in which the term is used here, however, is related to ‘literary criticism’, and embodies both positive and negative sentiments. as the term is used in this article: australasian journal of information systems clarke 2020, vol 24, selected papers from the is foundations conference establishing a research technique 7 criticism presents an analysis of relevant features of a body of work, both positive and negative, including its framing, the analysis undertaken, and the inferences drawn. such terms as ‘constructive criticism’ and 'critique' might be preferred. they have the advantages of playing down the negative aspects, of indicating that the process is systematic, and of bringing focus to bear on the contribution being made by both the works that are being subjected to analysis and the critique they are being subjected to. the justification for applying a sceptical eye to a body of work is that criticism plays a vital role in the scientific process. the conventional popperian position is that the criterion for recognising a scientific theory is that it deals in statements that are empirically falsifiable, and that progress depends on scrutiny of theories and attempts to demonstrate falsity of theoretical statements: "the scientific tradition ... passes on a critical attitude towards [its theories]. the theories are passed on, not as dogmas, but rather with the challenge to discuss them and improve upon them" (popper 1963, p.50). however, senior members of a discipline commonly behave in ways that are not consistent with the popperian position. this might be explained by the postulates of 'normal science', which view the vast majority of research work as being conducted within a 'paradigm' (kuhn 1962) or 'disciplinary matrix' (kuhn 1977) – "the common possession of the practitioners of a professional discipline [including its] symbolic generalizations, models, and exemplars" (p.296) – and subject to its conventions. in more practical terms, the problem may arise because senior members of any discipline have strong psychic investment in the status quo, and – no matter how cogent and important the argument – react negatively against propositions perceived to be at least disruptive, and even revolutionary. firmly-worded criticism is normal in reviewers’ reports on as-yet unpublished submissions, and may be acceptable if uttered by a senior member of a discipline about a contrarian idea, whereas it commonly attracts opprobrium if made about the contemporary wisdom, by an outsider or a relative newcomer. in an influential commentary, webster & watson (2002) recommended that "in contrast to specific and critical reviews of individual papers, tell the reader what patterns you are seeing in the literature ... do not fall into the trap of being overly critical" (p.xviii). on a literal reading, the authors merely warn against unduly strong or negative expression – a recommendation that the research technique under development needs to adopt. unfortunately, the quoted words are capable of being interpreted as valuing politeness among researchers more highly than scientific insight and progress. similarly, straub (2009, p.viii) advised authors that "papers should be in apposition [the positioning of things side by side or close together] rather than in opposition". clearly, where a new theory subsumes an old one (as was the case in the example provided, of einsteinian relativity absorbing newtonian physics as a special case), the dictum 'apposition rather than opposition' is appropriate. where, on the other hand, the new theory actively contradicts or is fundamentally at odds with, existing theory, it would be intellectually dishonest to represent the contrary or disruptive theory as though it were a soul-mate to, or merely a refinement of, existing theory. further, a reader might all-too-easily interpret the advice as expressing a moral judgement that 'criticism is a bad thing'. my contention is that scientific behaviour demands the opposite: it is an obligation of researchers to 'think critically' and to 'apply their critical faculties'. politeness of expression and focus on the message rather than the messenger are elements of academic discourse, but they are second-order concerns rather than fundamental to science. australasian journal of information systems clarke 2020, vol 24, selected papers from the is foundations conference establishing a research technique 8 3.2 critical theory research positivism and interpretivism are well-established approaches to research in is. they have been joined by design science. and they have an odd bedfellow, in the form of what is variously termed 'critical research' and 'critical theory research'. the term 'critical' in this context is somewhat different from the sense of 'analysis of the merits and faults of a work' discussed in the previous section. both positivism and interpretivism are concerned with description and understanding of phenomena and behaviours. sometimes the focus is on natural phenomena, but frequently the interest is in natural phenomena that have been subjected to an intervention. importantly for the present project, however, both positivism and interpretivism involve strenuous avoidance of moral judgements and of 'having an agenda'. design research, in contrast, is expressly purposeful and value-laden, in that the features designed into the artefact embody judgements about what is good in the particular context, and whose interests the goodness is to serve. critical theory research is concerned with description and understanding of phenomena and behaviours, but, like design science, it 'has an agenda'. chua (1986) distinguished critical research from the positivist and interpretivist perspectives on the basis that the relations among real-world objects are transformed through subjective interpretation, conflict among actors' interests is endemic to society, interpretations reflect ideology, and theory must take ideologies into account. in addition, and most contentiously, critical research embodies an assumption that theory must strive to overcome ideological dominance. see also alvesson & deetz (2000). beyond mere 'gap-spotting', sandberg & alvesson (2011) champions 'problematisation', a term popularised by foucault in the 1960s. it is concerned with 'the defamiliarisation of common sense', with "a central goal ... to try to disrupt the reproduction and continuation of an institutionalized line of reasoning" (p.32). among the four forms that the authors discuss is 'critical confrontation'. the is literature reflects these sources, with critical research described as recognising the effects of power and the tendency of some stakeholders' interests to dominate those of other stakeholders. it brings to light "the restrictive and alienating conditions of the status quo" and expressly sets out to "eliminate the causes of alienation and domination" (myers 1997). "critical is research specifically opposes technological determinism and instrumental rationality underlying is development and seeks emancipation from unrecognised forms of domination and control enabled or supported by information systems" (cecez-kezmanovic 2005, p.19). in myers & klein (2011), three elements of critical research are identified: • insight, which is a feature also of interpretivist research, and which requires depth of study. an important role is played by reflexivity: "by intentionally expressing, questioning, and reflecting upon their subjective experiences, beliefs, and values, critical researchers expose their ideological and political agendas" (cecezkezmanovich 2001, p.147) • critique, which "goes beyond interpretation to focus the researcher on the power structures that lie behind accepted interpretations" (p.24) • transformation, which is "concerned with suggesting improvements to the conditions of human existence, existing social arrangements, and social theories" (p.24) australasian journal of information systems clarke 2020, vol 24, selected papers from the is foundations conference establishing a research technique 9 elements of critical theory research are relevant to the present purpose. however, the sense of 'criticism' or 'critique' adopted by critical theory researchers is strongly value-laden and partisan, and hence wilfully disruptive. that contrasts with the definition of 'criticism' adopted here, which expressly involves "analysis of relevant features of a body of work, both positive and negative". 4 relevant research techniques the previous section established the distinctive difference about the intended new research technique, and hence clarified the justification for devising a new research technique. the next challenge is to identify and evaluate prior techniques whose purposes are related to it, in order to co-opt aspects that are relevant and avoid aspects that are not. this section outlines a series of categories of existing research techniques that have at least superficial relevance to the present purpose. consideration is given firstly to qualitative research techniques and then to traditional and systematic literature reviews. the third group comprises techniques referred to generically as ‘content analysis’. a separate section outlines a particular and rather different form of content analysis. the final category is a form of literature review compatible with critical theory research. 4.1 qualitative research techniques a significant proportion of research involves the appraisal of content previously uttered by people. qualitative research techniques such as ethnography, grounded theory and phenomenology involve the disciplined examination of content. that content may be uttered in natural settings by people within the research domain (e.g. in emails or social media postings) or it may be generated by researchers who inject themselves into otherwise natural settings by undertaking field research. alternatively, the content may be captured in a contrived setting (laboratory experiments), or in a partially natural and partially contrived setting (e.g. interviews conducted in the subject's workplace). the content may originate as text, or in some other form, such as communications behaviour in verbal form – possibly transcribed into pseudo-text; as natural non-verbal behaviour ('body-signals'); or as nonverbal, non-textual communications behaviour (such as ticks in boxes in structured questionnaires). the issues arising with analysis of these kinds of content are very different from those associated with the analysis of carefully-considered, formalised content, uttered by researchers, in published works that have been at least strongly edited and in most cases subjected to peer-review. 4.2 literature review a context that is more closely related to the present purpose is literature reviews that examine substantial bodies of prior research. traditional ‘narrative’ reviews have been criticised as being insufficiently rigorous (oakley 2003, p.23). there are now expectations of structure, transparency and replicability, and the term ‘systematic’ review is commonly applied (webster & watson 2002, okoli & schabram 2010, bandara et al. 2011). examples particularly relevant to the present project include galliers & whitley (2002, 2007), which analysed themes in the ecis conference series, and clarke (2012) and clarke & pucihar (2015), which reviewed the corpus of bled conference papers. grover & lyytinen (2015) and tarafdar & davison (2018) reported on meta-analyses of articles in the ‘basket of 8’ is journals. australasian journal of information systems clarke 2020, vol 24, selected papers from the is foundations conference establishing a research technique 10 although such approaches have relevance to the present project, their focus is on exposition, application, or at most interpretation, and the research techniques used were not devised in order to support critical analysis. relevantly, boell & cecez-kecmanovic (2015) argue that the recipes provided for 'systematic literature reviews' are not necessarily appropriate, particularly where the research technique that is applied allows for emergent and changing understanding of the phenomena, and where even the research question itself may be emergent. 4.3 content analysis the term ‘content analysis’ refers to a cluster of techniques that seek to classify content into a manageable number of categories. two definitions are: "a research technique for making replicable and valid inferences from texts (or other meaningful matter) to the contexts of their us" (krippendorff 2013, p. 24) "the semantic analysis of a body of text, to uncover the presence of strong concepts" (indulska et al. 2012, p.4, citing weber 1990) many authors have attempted to categorise the various forms that content analysis takes. (see, for example, hsieh & shannon 2005, drisko & maschi 2016). the following four-way classification scheme is suitable for the present purpose: (1) pre-defined (a priori) categories a methodologically strong approach is to derive the categories from existing theory. to the extent that the declared or inferred content of the text does not fit well to the predefined categories, there may be a need to consider possible revisions of the coding scheme, or even of the theory on which the research design was based. it may be feasible to draw inferences based on counts of the occurrences of categories and/or on the intensity of the statements in the text, such as the confidence inherent in the author's choice of language (e.g. "this shows that" cf. "a possible explanation is that"). however, as with any theory-driven research, the evidence extracted from the text may have a self-fulfilling-prophecy quality about it, i.e. there is an inevitable tendency to find more evidence in support of a theory than in conflict with it, and contextual factors may be overlooked. in order to enable auditability, it is important that not only the analysis be published, but also the raw material, the coding scheme, and the coding, at a sufficiently high level of granularity. (2) emergent categories this approach is particularly appropriate when there is an absence of suitable theories to guide the establishment of a priori categories; but it also has relevance where the researcher is quite specifically challenging existing theories in the area. the process of necessity involves assumptions, and hence external validity of conclusions arising from this approach is likely to be limited. depending on the degree of generality of the conclusions claimed by the author, full disclosure of the text selection, coding and inferencing procedures may be merely desirable or vital. (3) summative content analysis this involves "counting and comparisons, usually of keywords or content, followed by the interpretation of the underlying context" (hsieh & shannon 2005, p.1277). the first step is to australasian journal of information systems clarke 2020, vol 24, selected papers from the is foundations conference establishing a research technique 11 explore usage, by "identifying and quantifying certain words or content in text with the purpose of understanding the contextual use of the words or content" (p.1283). because of the complexity and variability of language use, and the ambiguity of a large proportion of words and phrases, a naive approach to counting words is problematic. at the very least, a starting-set of terms needs to be established and justified. a thesaurus of synonyms and perhaps antonyms and qualifiers is needed. allowance must be made for both manifest or literal meanings, on the one hand, and latent, implied or interpreted meanings (in semiotic terms, ‘pragmatics’), on the other. counts may be made not only of the occurrences of terms, but also of the mode of usage (e.g. active versus passive voice, dis/approval indicators, associations made). (4) programmatic content analysis automated processing enables much larger volumes of text to be analysed. the coding scheme may be defined manually, cf. directed content analysis / a priori coding. however, some techniques involve purely computational approaches to establishing the categories, cf. 'machine-intelligent' (rather than human-intelligent) emergent coding. the processing depends, however, on prior data selection, data scrubbing and data-formatting. in addition, interpretation of the results involves at least some degree of human activity. debortoli et al. (2016), distinguishes three alternative approaches to programmatic coding: • dictionary-based text categorization, which "relies on experts assembling lists of words and phrases that likely indicate text’s membership to a particular category", cf. a priori coding • supervised learning methods "[using] predefined categories; however, one does not explicitly know the mapping between text features and categories" • unsupervised machine-learning methods "for categorizing text [which] find hidden structures in texts for which no predefined categorization exists", cf. emergent coding performed programmatically given that the 'big data analytics' movement is highly fashionable, vast volumes of data are available, and there is a comfort factor involved in office-based work much of which is automated, it would appear reasonable to anticipate that programmatic analysis techniques may be a growth-area in the coming few years – although that growth may be tempered as their limitations come to be better appreciated (clarke 2016a, 2016c, 2018). content analysis techniques exhibit varying degrees of structure and rigour, from impressionistic to systematic, and they may involve qualitative and/or quantitative assessment elements. qualitative data may be gathered on a nominal scale (whereby differences are distinguished, but no ordering is implied) or on an ordinal scale (such as 'unimportant', 'important', 'very important'). quantitative data, on the other hand, may be on an ordinal scale, or on a cardinal or interval scale (i.e. an ordinal scale but with equal distances between values, e.g. degrees celsius), or on a ratio scale (i.e. a cardinal scale, but with the additional feature of a natural zero, e.g. degrees kelvin). data collected on the higher-level scales, especially on a ratio scale, is able to be subjected to more powerful inferencing techniques. quantification generally involves measurement. this may be by counting, by comparing a real-world attribute against some standard, or by 'sensing' a real-world attribute and using the impulse to create one or more data-items whose contents are intended to correspond with the australasian journal of information systems clarke 2020, vol 24, selected papers from the is foundations conference establishing a research technique 12 state of the attribute. although there is a common perception that quantification produces data that reliably represents the real world, many conditions need to be fulfilled for this to be so. units of measure have arbitrary boundaries, and the configuration and calibration of measuring instruments is challenging. moreover, some quantification techniques are justified by conventions that are poorly supported by theory. a significant example is the frequentlyencountered but unjustified assumption that 'likert-scale' data is not merely ordinal, but is on an interval scale (i.e. the spaces between the successive terms are identical), and even ratio (i.e. the scale also features a natural zero), in order to justify the application of powerful statistical techniques to the data. many authors, particularly those who work in the positivist tradition, implicitly equate qualitative data with subjectivity and quantification with rigour. this has the effect of deprecating qualitative analysis, or at least relegating it to pre-theoretical research, which by implication should be less common than research driven by strong theories. however, the majority of authors who make those assumptions spend only limited time considering the extent to which their own assumptions, and the processes underlying the act of quantification, may be arbitrary or themselves 'subjective'. positivism embodies an implicit assumption that computational analysis necessarily leads to deep truth. the assumption needs to be tested in each particular circumstance, yet such testing is seldom evident. although quantification has advantages in terms of the scope for applying powerful analytical techniques, the degree of analytical rigour that quantification can actually deliver is heavily dependent on a number of factors. critical among them are the text selection; the express judgements and implicit assumptions underlying the choice of terms that are analysed; the sophistication and comprehensiveness of the thesaurus applied; and the significance imputed to each term. in practice, a considerable proportion of research that uses content analysis techniques involves primarily qualitative data. as with any research technique, all aspects need to be subject to quality controls. the coding scheme and the performance of the coding process are particularly critical (krippendorff 1980, weber 1990), stemler 2001). approaches include coding by individuals with strong experience in both the review of papers and the subject-matter, parallel coding by multiple individuals, review of individuals' coding by other parties, and publication of both the source materials and the detailed coding sheets, in order to enable audit by other parties. the various approaches to content analysis outlined above appear to be accepted within the is discipline, but their use has been somewhat limited. for example, in a survey of the articles published in six leading is journals during the 1990s, mingers (2003) found that the use of content analysis as a research technique was evident in only four of the journals, and even in those four in only 1-3% of all articles published during that time. in july 2019, of the over 16,000 refereed papers indexed in the ais electronic library, 13 had the term 'content analysis' in the title (none published more recently than 2013, and only 5 of them in journals), and 72 in the abstract (only 6 of them since 2013, but the most recent 12 in journals). in recently-published papers, the most common forms of text that have been subjected to content analysis are social media, consumer reviews, and other message content, with other categories including newspaper articles and corporations' 'letters to shareholders'. there appears to have been very limited application of these techniques to published works. many forms of content analysis are primarily descriptive, and at best interpretive. their focus is on concepts, themes, patterns and relationships. moreover, the semantics and pragmatics of australasian journal of information systems clarke 2020, vol 24, selected papers from the is foundations conference establishing a research technique 13 the source-materials may not be fully reflected, and only limited attention may be given to contextual factors. critical analysis depends on interpretation, and it must delve deeply not only into the tenable meanings of the content, but also into the context and the assumptions inherent in the works being studied. it may also be necessary to follow trails such as cited works and terms used, in order to place the work within a genre, or within a ‘school of thought’. 4.4 critical discourse analysis a recent proposal endeavours to address these weaknesses. wall et al. (2015) describe an approach that they say they have based on habermasian strains of critical discourse analysis (cda). their starting-point is that "the information systems (is) discipline is subject to ideological hegemony" (p.258). by ‘ideological hegemony’ the authors mean "the conscious or unconscious domination of the thought patterns and worldviews of a discipline or subdiscipline that become ingrained in the epistemological beliefs and theoretical assumptions embedded in scientific discourse (fleck, 1979; foucault, 1970; kuhn, 2012)". they argue that "ideologies can be harmful to individuals who are disadvantaged or marginalized by them, and they can be problematic to scientific research because they represent blind spots" (p.258), and hence that "review papers [should] ... challenge ideological assumptions by critically assessing taken-for-granted assumptions" (p.257). the authors propose a seven-step process for critical discourse analysis (pp. 265-9). all of the forms of content analysis outlined in this and the previous sub-section have relevance to the present purpose. however, the first four do not extend beyond exposition to critique. the fifth, on the other hand, adopts the 'critical theory research' tradition, and exhorts the righting of perceived wrongs to disadvantaged or marginalised individuals that arise from ideological assumptions adopted by designers. the scope of the critical analysis of published works research technique is intended to encompass not only critical theory research, but also research undertaken within the positivist and interpretivist traditions. it is therefore necessary to take care in applying the ideas embodied within critical discourse analysis, in order to avoid embedding commitment to social causes. 4.5 a hermeneutic approach to literature review the emphasis on 'systematic' literature reviews (slr) noted earlier has itself been subjected to criticism, in that it "suppresses aspects of quality in research and scholarship that are at least as important as clarity, countability and accountability – such as intertextual connectivity, critique, interest, expertise, independence, tacit knowledge, chance encounters with new ideas, and dialogic interactions between researcher, 'literature' and 'data'" (maclure 2005, p.394). more substantially, boell & cecez-kecmanovic (2015) argue that the literature on the slr technique has effectively kidnapped the term 'systematic'. it has linked with the notion 'systematic' the requirements that literature review must necessarily begin with a clearlydefined research question, and that its results must evidence or must deliver rigour, objectivity, replicability, and absence of bias. boell & cecez-kecmanovic contend firstly that these are not necessarily delivered by slr, and secondly that, even if they are attainable, they are, for some forms of research, undesirable and harmful. further, in boell & cecez-kecmanovic (2014) it is argued that a constructively loose and iterative process is needed, to avoid undue constraints and unlock insight and creativity: "highly structured approaches downplay the importance of reading and dialogical interaction australasian journal of information systems clarke 2020, vol 24, selected papers from the is foundations conference establishing a research technique 14 between the literature and the researcher; continuing interpretation and questioning; critical assessment and imagination; argument development and writing – all highly intellectual and creative activities, seeking originality rather than replicability [maclure, 2005, hart, 1998]" (p.258). to address these issues, boell & cecez-kecmanovic "propose hermeneutic philosophy as a theoretical foundation and a methodological approach for studying literature reviews as inherently interpretive processes in which a reader engages in ever exp[a]nding and deepening understanding of a relevant body of literature. hermeneutics does not assume that correct or ultimate understanding can be achieved, but instead is interested in the process of developing understanding" (p.259, emphasis added). their framework, reproduced in figure 1, comprises two intertwined cycles: a search and acquisition circle, and a wider analysis and interpretation circle (p.263). rather than a carefully-planned and closed-ended process, this approach embodies "questioning and critical assessment ... of previous research" (p.258). "critical assessment ... not only reveals but also ... challenges the horizon of possible meanings and understanding of the problem and the established body of knowledge" (p.267). figure 1: a hermeneutic framework for the literature review process consisting of two major hermeneutic circles the hermeneutic framework proposed is oriented towards the review of a body of literature. this will generally be selected because it has as its focus a particular body of theory, or perhaps a particular category of real-world phenomena or behaviour. such a body of literature is within the scope of the notion of 'published works' that is the focus of the present project. on the other hand, the technique being devised here needs to encompass an even broader range australasian journal of information systems clarke 2020, vol 24, selected papers from the is foundations conference establishing a research technique 15 of 'published work' populations and sampling frames, such as publishing venue(s) or research technique(s). 5 conceptualisation of the critical analysis of published works having addressed some of the preliminary challenges, the aim now is to draw from the substantial methodological resources identified above, and establish a framework for the specific category of research questions for which the critical analysis of published works is the appropriate technique. the preceding discussion has identified structured and pre-determinable techniques relevant to some forms of 'analysis of published works'. it has also highlighted a far more loose-limbed, adaptive and interpretive approach, which is imbued with strong, counter-cultural fervour, associated with the critical discourse analysis approach to content of wall et al. (2015), and the hermeneutic approach to literature review of boell & cecez-kecmanovic (2014). the diversity of flavour among contexts and purposes means that, in order to encompass all relevant variants, the new research technique needs to be described in a relatively abstract manner. researchers then need to apply the abstract principles in order to customise a research method appropriate to each particular project. drawing on the preceding discussion, a set of strongly-desirable characteristics of the new research technique is presented in table 2. the focus of the research 1. formal content the analysis is concerned with works that present carefully-considered textual content, in venues that generally apply formal review and editorial filters to the works that they publish 2. content and method the analysis is concerned with the content of the works, including methodological aspects of the content 3. critical thought the analysis applies critical thought to the content. this means that the analysis goes beyond exposition, description, explication, consolidation and interpretation of existing theory, and beyond mere the mere detection of gaps in existing theory, to the drawing of implications from theory, the identification of conflicts among stakeholder interests, and consideration of the impacts of ideologies on the research domain 4. constructive critique the analysis is concerned with both the positive and negative features of the work, including its framing, the analysis undertaken, and the inferences drawn 5. content rather than intent the analysis is of the content of works. this may include the authors' express intentions, or the author's implied intentions where clear evidence exists, but not intentions of their authors that are inferred by the reviewer the framing of the research 6. appropriate degree of structuredness depending on the purpose of the research: • the analysis may be highly-structured, driven by a body of theory, addressing a pre-defined and specific research question, and applying a coding scheme established a priori ; • the analysis may be lightly-structured and interpretive in nature, with an emergent coding scheme, seeking understanding, and progressively adapting the coding scheme and/or the research question; or australasian journal of information systems clarke 2020, vol 24, selected papers from the is foundations conference establishing a research technique 16 • various points between the various extremes may be appropriate to the purpose, e.g. commencing with a broad research question, a research design, and an a priori coding scheme, but remaining open to iterative refinement and adaptation of the question, design and/or coding scheme, to reflect the accumulated insights 7. designed-in methodological rigour the analysis is subjected to such structuredness, consistency, comprehensiveness, transparency and auditability constraints as are feasible and desirable in the particular context the research process 8. phased reading preliminary 'orientational reading' is followed by deeper 'analytical reading', in order to provide confidence in the appropriateness of the selection, coding and interpretation of passages. by 'passage' is meant a text-segment of any length, including word, word-group. phrase, clause, sentence, paragraph or paragraphs. (for example, the macquarie dictionary defines a passage as "an indefinite portion of a writing, speech, or the like ...") 9. coding rigour sources of categorisations are documented, and appropriate techniques are applied, which may fall anywhere within a wide range, from directed content analysis using a priori coding, via emergent coding, to critical discourse analysis 10. appropriate quantification • the scales on which data are collected are as strong as practicable in the circumstances (i.e. ordinal in preference to nominal, and ratio to cardinal or ordinal) • the assumptions underlying quantification are critically examined, and are justifiable 11. analytical rigour • the analytical techniques applied are appropriate to the scales on which the data are gathered • the means are documented whereby reliability is achieved in selection, coding and interpretation activities • where a hermeneutic philosophy is adopted and understanding is sought, the most significant factor may be the transparency of evidence supporting the reasonableness of interpretations 12. iteration the analysis involves open-mindedness and introspective questioning by the researcher about their analysis and assumptions, including loops back to earlier phases of the process, possibly resulting in adaptation of the project's framing, re-design of coding schemes, re-coding and/or reconsideration of inferences drawn, as appropriate in the context the presentation of the research 13. declaration of the final form of the research question • the research question is declared, and sufficient information is provided about such adaptation as occurred during the course of the project • the research question evidences detail and precision consistent with the research purpose 14. declaration of the research method • sufficient information is provided to enable the reader to understand and evaluate the appropriateness of the approach adopted and any adaptation that occurred during the course of the project • comprehensive supporting materials are made available to enable audit 15. declaration of researcher perspective the values underlying the researcher's approach are identified, and where appropriate are compared and contrasted with those that the researcher perceives to be embedded in the works 16. qualitative critique a textual critique is presented, to the extent appropriate supported by quantitative evidence, analysing the positive and negative aspects of the selected works, and perhaps by implication of the sampling frame or the population australasian journal of information systems clarke 2020, vol 24, selected papers from the is foundations conference establishing a research technique 17 17. inferential rigour • critical comments are expressed with a degree of confidence commensurate with the nature of the evidence • where relevant, the text includes critical assessment of taken-for-granted assumptions by both the authors of works that are assessed, and, to the extent feasible, by the researcher undertaking the critical analysis 18. careful expression the text is expressed in a measured manner, avoiding unduly strong expression, unduly positive or negative expression, and overly colourful speech 19. avoidance of argumentum ad hominem • criticisms are made of the work and not of the author of the work • where hegemonic participation is evident, the default assumption is that the author adopted the viewpoint unconsciously • an attribution of conscious hegemonic participation is not to be made unless reliable evidence is presented 20. expression in apposition unless the evidence warrants expression in opposition the text is no more confronting to or disruptive of existing theory than the evidence arising from the analysis warrants. in particular, where the conclusions and implications qualify, deepen or subsume existing theory, they are presented in apposition rather than in opposition table 2: key characteristics of the critical analysis of published works 6 applications of the technique the process of developing the new research technique has broadly followed he first three phases of peffers' design science research methodology (dsrm – peffers et al. 2007), viz. problem identification, objectives definition, and design and development. the next phase (demonstration) has been commenced but not brought to completion. the first aspect of demonstration is the identification of specific contexts to which the technique is to be applied, and the second is the application of the framework in order to instantiate research methods appropriate to specific research questions. four trials have been conducted and reported on, two of them published and two so far only in manuscript form. each of them applies the theory of researcher perspective (clarke & davison 2019) to a particular set of published works. of the sample research questions in table 1, this most closely relates to item 10. the first trial, reported in clarke (2015), considered a modest set of papers from the acis conference and the ajis journal. the second, reported in clarke (2016b), examined a larger set of papers presented at the bled conference. the current manuscripts report on studies of basket of 8 articles and of a set of papers in the journal 'electronic markets'. the pilot projects undertaken to date attest to the practicality of the technique when articulated for, and applied in, a specific critical theory research context. the small set is not sufficiently rich to illustrate or demonstrate the usefulness of the guidance across all of the various projecttypes. far less do the projects conducted to date represent peffers' fifth phase (evaluation), which involves observation and measurement of the new artefact’s effectiveness in addressing stated objectives. 7 conclusions this article has outlined the elements of a project to establish adequate conceptual foundations and guidance in relation to the implementation of a technique for critical analysis of the australasian journal of information systems clarke 2020, vol 24, selected papers from the is foundations conference establishing a research technique 18 content of published works relevant to the is discipline. it has identified and drawn on a wide range of sources on techniques that bear some kind of relationship to the present purpose. it has proposed a set of key characteristics of a process of this nature, and indicated the scope of application. articulation to apply to specific categories of critical content analysis, and application to particular sets of published works, are ongoing parts of the overall project. the description of the process undertaken in the project to date has identified a series of challenges that are likely to confront each endeavour to create a new research technique. it is necessary to achieve clarity about the category of research questions to which the technique is intended to be applicable. it is vital to reflect and build on the accumulated knowledge about research techniques generally; but the methodology literature is less helpful in this regard than might be imagined, and the same is true of exemplars within the is literature that are highlycited in relation to particular research techniques. a further challenge involves identifying the key features of the category of research questions that are in focus. in the present example, this involves the notion of (constructive) criticism and aspects of critical theory research. armed with this level of understanding, it becomes possible to identify pre-existing techniques, and assess the extent to which elements of them do and do not provide insights relevant to the new technique. the foundations having been laid, it becomes possible to undertake conceptualisation of the new technique, through the specification of key characteristics that it needs to have. the remaining challenges (only partly discussed in this article) are the articulation of the technique into forms that are directly relevant to particular forms of research question, application of the technique to a sufficient set of samples, and demonstration that the technique enables research questions to be satisfactorily answered. the work reported here is limited to a single new research technique, and the trial applications relate to a single form of the many kinds of research questions that are within-scope of the technique. it is accordingly infeasible to generalise from this instance to all, or even to other individual instances of, new research techniques. this article does, however, provide researchers with a somewhat stronger starting-point for their own endeavours than has previously been available, because it identifies key considerations, points to a range of relevant sources, and provides an articulated example of their use to develop a particular kind of new research technique. references alvesson m. & deetz s. (2000) 'doing critical management research' sage, 2000 alvesson m. & sandberg j. (2011) 'generating research questions through problematization' academy of management review 36, 2 (2011) 247-271 avison d.e. (2002) 'action research: a research approach for cooperative work' proc. conf. cooperative systems for cooperative work, rio de janeiro, september 2002 bandara,w., miskon s. & fielt e. (2011) 'a systematic, tool-supported method for conducting literature reviews in information systems' proc. ecis 2011, paper 221, at http://eprints.qut.edu.au/42184/1/42184c.pdf benbasat i., goldstein d.k. & mead m. (1987) 'the case research strategy in studies of information systems' mis quarterly 11, 3 (september 1987) 369-386 australasian journal of information systems clarke 2020, vol 24, selected papers from the is foundations conference establishing a research technique 19 boell s.k. & cecez-kecmanovic d. (2014) 'a hermeneutic approach for conducting literature reviews and literature searches' communicat ions of the association for information systems 34, 12, at http://tutor .nmmu.ac.za/mgerber/documents/resmeth_boell_2014_lite rature%20reviews.pdf boell s.k. & cecez-kecmanovic d. (2015) 'on being 'systematic' in literature reviews in is' j. infor. techno. 30 (2015) 161-173 bonoma t.v. (1985) 'case research in marketing: opportunities, problems,and a process' journal of marketing research 22, 2 (may 1985) 199-208 vom brocke j. & simons a. (2008) 'towards a process model for digital content analysis – the case of hilti' proc. bled conference, june 2008, at http://aisel.aisnet.org/bled2008/2 cecez-kecmanovic d. (2001) 'doing critical is research: the question of methodology' ch.vi in 'qualitative research in is: issues and trends: issues and trends' (ed. trauth e.m.), pp. 141163, idea group publishing, 2001, at https://pdfs.semanticscholar.org/37b1/e4c060b93fcfa04d81f03b750e746ba42f2d.pdf cecez-kecmanovic d. (2005) 'basic assumptions of the critical research perspectives in information systems' ch. 2 in howcroft d. & trauth e.m. (eds) (2005) 'handbook of critical information systems research: theory and application', pp.19-27, edward elgar, 2005 chua w.f. 'radical developments in accounting thought' the accounting review 61, 4 (october 1986) 601–632 clarke r. (2012) 'the first 25 years of the bled econference: themes and impacts' proc. 25th bled econference special issue, preprint at .. http://www.rogerclarke.com/ec/bled25ta.html clarke r. (2015) 'not only horses wear blinkers: the missing perspectives in is research' keynote presentation, proc. austral. conf. in infor. syst. (acis 2015), at https://arxiv.org/pdf/1611.04059, adelaide, december 2015, preprint at http://www.rogerclarke.com/sos/acis15.html clarke r. (2016a) 'big data, big risks' information systems journal 26, 1 (january 2016) 77-90, preprint at http://www.rogerclarke.com/ec/bdbr.html clarke r. (2016b) 'an empirical assessment of researcher perspectives' proc. bled econf., slovenia, june 2016, at http://www.rogerclarke.com/sos/bledp.html clarke r. (2016c) 'quality assurance for security applications of big data' proc. eisic'16, uppsala, 17-19 august 2016, preprint at http://www.rogerclarke.com/ec/bdqas.html clarke r. (2018) 'guidelines for the responsible application of data analytics' computer law & security review 34, 3 (may-jun 2018) 467476, https://doi.org/10.1016/j.clsr.2017.11.002, preprint at http://www.rogerclarke.com/ec/gda.html clarke r. & davison r.m. (2019) 'through whose eyes? the critical concept of researcher perspective' forthcoming, journal of the association for information systems australasian journal of information systems clarke 2020, vol 24, selected papers from the is foundations conference establishing a research technique 20 clarke r. & pucihar a. (2013) 'electronic interaction research 1988-2012 through the lens of the bled econference' electronic markets 23, 4 (december 2013) 271-283, preprint at http://www.rogerclarke.com/ec/eires-bled25.html davison r.m., martinsons m.g. & kock n. (2004) 'principles of canonical action research' info systems j (2004) 14, 65–86 debortoli s., mueller o., junglas i. & vom brocke (2016) 'text mining for information systems researchers: an annotated topic modeling tutorial' communications of the association for information systems 39, 7, 2016 fink a. (2005) 'conducting research literature reviews: from the internet to paper' sage publications, 2nd ed., 2005 galliers r.d. & whitley e.a. (2002) 'an anatomy of european information systems research ecis 1993 ecis 2002: some initial findings' proc. ecis 2002, june 2002, gdask, poland, preprint at http://personal.lse.ac.uk/whitley/allpubs/ecis2002.pdf galliers r.d. & whitley e.a. (2007) 'vive les differences? developing a profile of european information systems research as a basis for international comparisons' euro. j. of infor. syst. 16, 1 (2007) 20-35 gregor s. & hevner a. (2013) 'positioning design science research for maximum impact' mis quarterly 37, 2 (june 2013 ) 337-355, at https://ai.arizona.edu/sites/ai/files/mis611d/gregor-2013-positioning-presentingdesign-science-research.pdf grover v. & lyytinen k. (2015) ‘new state of play in information systems research: the push to the edges’ mis quarterly 39:2 (2015) 271-296 hsieh h.-s. & shannon s.e. (2005) 'three approaches to qualitative content analysis' qualitative health research 15, 9 (november 2005) 1277-1288, at http://www33.homepage.villanova.edu/edward.fierros/pdf/hsieh%20shannon.pdf indulska m., hovorka d.s. & recker j.c. (2012) 'quantitative approaches to content analysis: identifying conceptual drift across publication outlets' european journal of information systems 21, 1, 49-69, at http://eprints.qut.edu.au/47974/ kaid l.l. (1989) 'content analysis' in emmert p. & barker l.l. (eds.) 'measurement of communication behavior', pp. 197-217, longman, 1989 kaplan a. (1964) 'the conduct of inquiry: methodology for behavioral science' chandler, 1964 klein h.k. & myers m.d. (1999) 'a set of principles for conducting and evaluating interpretive field studies in information systems' mis quarterly 23, 1 (march 1999) 67–94 krippendorff k. (1980) 'content analysis: an introduction to its methodology' sage, 1980 krippendorff k. (2013) 'content analysis: an introduction to its methodology' sage, 3rd ed., 2013 kuhn t.s. (1962) 'the structure of scientific revolutions' university of chicago press, 1962 kuhn t.s. (1977) 'second thoughts on paradigms' ch. 12 of kuhn t.s. 'the essential tension: selected studies in scientific tradition and change' university of chicago press, 1977, pp. 293-319 australasian journal of information systems clarke 2020, vol 24, selected papers from the is foundations conference establishing a research technique 21 maclure m. (2005) '‘clarity bordering on stupidity’: where’s the quality in systematic review?' journal of education policy 20, 4 (2005) 393-416, at http://www.esri.mmu.ac.uk/respapers/papers-pdf/paperclarity%20bordering%20on%20stupidity.pdf mingers j. (2003) 'the paucity of multimethod research: a review of the information systems literature' information systems journal 13, 3 (2003) 233–249 myers m.d. (1997) 'qualitative research in information systems' misq discovery, june 1997, at http://www.academia.edu/download/11137785/qualitative%20research%20in%20infor mation%20systems.pdf myers m.d. & klein h.k. (2011) 'a set of principles for conducting critical research in information systems' mis quarterly 35, 1 (march 2011) 17-36, at https://pdfs.semanticscholar.org/2ecd/cb21ad740753576215ec393e499b1af12b25.pdf niederman f. & march s. (2012) 'design science and the accumulation of knowledge in the information systems discipline' acm transactions on mis 3, 1 (2012), 1 oakley, a. (2003) research evidence, knowledge management and educational practice: early lessons from a systematic approach, london review of education, 1, 1: 21-33, at http://www.ingentaconnect.com/contentone/ioep/clre/2003/00000001/00000001/art0000 4?crawler=true&mimetype=application/pdf okoli c. & schabram k. (2010) 'a guide to conducting a systematic literature review of information systems research' sprouts: working papers on information systems, 10, 26 (2010), at http://sprouts.aisnet.org/10-26 peffers k., tuunanen t., rothenberger m. & chatterjee s. (2007) 'a design science research methodology for information systems research' journal of management information systems, 24, 3 (winter 2007) 45–77 popper k. (1963) 'conjectures and refutations: the growth of scientific knowledge' harper & row, 1963 sandberg j. & alvesson m. (2011) 'ways of constructing research questions: gap-spotting or problematization?' organization 18(1) 23–44 stemler s. (2001) 'an overview of content analysis' practical assessment, research & evaluation 7, 17 (2001), at http://pareonline.net/getvn.asp?v=7&n=17 straub d. w. (2009) 'editor's comments: why top journals accept your paper' mis quarterly, 33, 3 (september 2009) iii-x, at http://misq.org/misq/downloads/download/editorial/3/ tarafdar m. & davison r.m. (2018) ‘research in information systems: intra-disciplinary and inter-disciplinary approaches’ forthcoming, journal of the association for information systems wall j.d., stahl b.c. & salam a.f. (2015) 'critical discourse analysis as a review methodology: an empirical example' communications of the association for information systems 37, 11 (2015), at https://www.dora.dmu.ac.uk/xmlui/bitstream/handle/2086/11180/2015_critical_discou rse_analysis_review_methodology_cais%20(1).pdf australasian journal of information systems clarke 2020, vol 24, selected papers from the is foundations conference establishing a research technique 22 walsham g. (1995) 'interpretive case studies in is research: nature and method' european journal of information systems 4, 2 (may 1995) 74–81 webster j. & watson r.t. (2002) 'analyzing the past to prepare for the future: writing a literature review' mis quarterly 26, 2 (june 2002) xiii-xxiii, at http://intranet.businessscience-institute.com/pluginfile.php/247/course/summary/webster%20%20watson.pdf weber r.p. (1990) 'basic content analysis' sage, 1990 acknowledgements the ongoing project of which this work is a part has benefitted from feedback by many colleagues, including during presentations of the emergent argument and analysis in a keynote at the australasian conference in information systems (acis'15) in adelaide in 2015, and at the bled conference in 2016 and 2017, the contributions of my co-authors on the evaluation of 'basket of 8' works, and valuable comments received from a reviewer and participants in the information systems foundations 2018 workshop: theorising society in information systems, anu, canberra, 13-14 september 2018. author affiliations roger clarke is principal of xamax consultancy pty ltd, canberra. he is also a visiting professor in cyberspace law & policy at the university of n.s.w., and a visiting professor in the research school of computer science at the australian national university. copyright: © 2020 clarke. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v24i0.2515 http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 methodological considerations 2.1 definition of terms and scope 2.2 meta-methodology 3 criticism 3.1 the role of criticism in research 3.2 critical theory research 4 relevant research techniques 4.1 qualitative research techniques 4.2 literature review 4.3 content analysis 4.4 critical discourse analysis 4.5 a hermeneutic approach to literature review 5 conceptualisation of the critical analysis of published works 6 applications of the technique 7 conclusions paper title (paper title style) australasian journal of information systems pye et al. 2022, vol 26, research article factor structures associated with online student engagement 1 factor structures associated with online student engagement in campus-based blended and online distance education settings graeme pye deakin university, geelong, australia graeme.pye@deakin.edu.au dale holt deakin university, geelong, australia scott salzman deakin university, warrnambool, australia abstract this research is strictly grounded in the work undertaken by coates regarding traditional campus-based settings of student engagement and was applied to the online settings positioned within the domain settings of blended online learning design and practice in an australian higher education business context. utilising an online student survey instrument, across two consecutive academic years, undergraduate commerce students were invited to reflect upon their learning engagement experience through the lens of a common learning management system (lms) a resource accessible to both campus-based and off-campus student cohorts. subsequent analysis of the research reconfirms the existence of student engagement constructs of coates in the blended online setting, but also unexpectedly revealed a previously unknown construct relating to assessment. this new student engagement construct, assessment, is identified as being a significant motivational factor relevant to student engagement in the context of the blended online learning environment of this higher education business undergraduate commerce course and is the focus of this exposition. keywords: student engagement, learning management systems, blended learning, online education, distance education. 1 introduction online learning is now central to many variant educational environments in global higher education. online learning supports campus-based blended education by providing access to the online learning environment. online also delivers distance education resources to students unable to attend campus and is a major delivery mode integrated with the development of global massive open online course (mooc) platforms. it also bridges environments designed to meet the needs of both campus-based and distance education students studying the same subjects and programs in multi-campus and multi-modal settings. whatever the concerns and criticisms levelled against the institution-wide selection and deployment of learning management systems with their set-piece forms and functions, they have become the mainstay platform for developing, delivering, and administering the online dimension of many variant higher educational environments. the generic term ‘learning management system’ (lms) has been used instead of the institution’s own branding of its online learning system. there is no suggestion that the lms alone is somehow automatically undertaking the various learning activities and resource delivery covered here. therefore, it is presumed that australasian journal of information systems pye et al. 2022, vol 26, research article factor structures associated with online student engagement 2 academics design and utilise the lms with students to create a value-added learning experience and hence, human agency animates the system. the motivation for this research is to revisit the specific key constructs of coates’s (2006) student engagement research, which was broadened via the extensive australasian survey of student engagement (ausse) survey, the focus of coates’s (2010) research. the intent here is to investigate if these specific student engagement constructs are prevalent in online learning supports and campus-based blended education environments within the contemporary online learning environment. the findings may further inform the online learning environment design of an institutional lms, to ensure that the student engagement constructs are active and prevalent as design features for the contemporary online learning environment. this research builds upon coates’s (2006) work on the validity of identified student engagement constructs that coates’s research recognised. to this end, this research project undertook a major surveying of individual student experiences of online engagement. the online student engagement data collected offered in multiple periods has been statistically analysed to test whether the constructs initially identified by coates’s (2006) are still apparent, valid, and of use for ongoing survey investigations. since coates’s original research, it is now timely to undertake this independent analysis of online student engagement constructs initially utilising those identified by coates (2006). specifically, this research was undertaken in the context of student engagement research around the inclusive on-campus learning, blended with online distance education, in the major field of higher business commerce education, and by extension, associated major professional fields of study. the structure of this document begins with the discussion relating to the review of the underpinning and relevant student engagement literature, including the research methodology as applied. next, the data collection approach is described, and the subsequent data analysis is undertaken utilising statistical analysis of relevant data to tease out the findings and their relevance. finally, the latent student engagement constructs that became apparent during the analysis are described, which is then followed by conclusions and a scope for future research. 2 literature review this research builds upon the previous research of pye et al. (2015, 2018) and is allied with the selection and use of a lms, as the enduring interest in investigating the student experience in higher education. one rich vein of research in this domain has related to theorising, conceptualising, and enhancing the practice of a phenomenon named ‘student engagement’ (kuh, 2003; kuh, kinzie, schuh, whitt & associates, 2005). student engagement relates to the time and effort students devote to educationally purposeful activities pursuant to achieving valued learning outcomes (coates & mccormick, 2014). student engagement integrates significant research based on the student experience in higher education settings, in key domains covering quality of effort (pace, 1979; 1995), principles for good practice in undergraduate education (chickering & gamson, 1987), student learning involvement principles (astin, 1985), and student integration into university life and persistence in study (tinto 1993; 1997). student engagement theory and policy development manifested itself in the national survey of student engagement (nsse) in the united states in the late 1990s, and was then adopted in canada, australia, new zealand, south africa, china, ireland (coates & australasian journal of information systems pye et al. 2022, vol 26, research article factor structures associated with online student engagement 3 mccormick, 2014) and the united kingdom (howson & buckley, 2017), as a major international initiative. student engagement is a high-potential vehicle for assuring and improving the quality of higher education at the sectoral and institutional levels, was implemented through the development of the australasian survey of student engagement (ausse) in the australian and new zealand context (coates, 2010). this research draws upon, recognises, reinforces, and extends previous learning design theories and practices examining various dimensions of interactivity in enhancing quality online learning environments (see, for example, bonk & zhang, 2008; garrison and anderson, 2003; salmon, 2011). coates (2006) work laid the theoretical, conceptual, and empirical foundation for the development and implementation of ausse. his empirically based, large-scale survey approach added a new dimension, illuminating the student experience in contemporary educational settings, both in relation to online learning technologies used in campus settings and student engagement in general, irrespective of the mode of course delivery. given the growing influence of online learning on the campus/residential student learning experience, coates’s (2006) expression of six student engagement constructs to encompass fundamental on-campus student engagement. are applied here to extend understanding and the implications of how ‘online’, as enabled through the mainstream use of lmss as it applies to (re)shaping the educational experience of students. this research seeks to take the six-student engagement constructs that coates (2006) identified in campus-based student learning settings and investigate whether these same student engagement constructs are apparent within the blended learning settings of a higher education business discipline setting, utilising a contemporary online and on-campus combined learning delivery environment. this research study positions and opens a new domain of specialised study of student engagement in online and blended learning environment within the discipline of business education. 2.1 student engagement the development and validation of coates’s (2006) student engagement constructs formed the foundation of his student survey instrument to elicit a structured conversation between learners and universities alike. coates developed a structured, methodical approach for deriving the qualities of campus-based and online engagement; and then validated these as constructs through the operationalisation of a student engagement questionnaire. as outlined by coates (2006, pp.71-80), ‘the student engagement instrument development began with the framing of constructs, scales and items found in the us nesse, and was shaped by various theoretical considerations.’ this framework of constructs and items was adjusted to include the findings of major online learning research. semi-structured interviews with students were conducted to explore student experiences engaging with online learning systems. students were sampled from different australian universities and fields of study and the analysis of the student interviews helped in the drafting of the scales and items to measure key qualities of online and campus student engagement. subject and technical experts reviewed draft items and scales, and this provided the basis for developing the actual student engagement questionnaire. the draft instrument was later reviewed by a student focus group who were invited to complete the survey and provide feedback on its meaning and comprehensibility. the survey instrument was then trialled with students from several universities. it constituted a landmark study of student engagement in online and blended learning settings in australian australasian journal of information systems pye et al. 2022, vol 26, research article factor structures associated with online student engagement 4 higher education which had not been previously attempted, or subsequently replicated or extended in the australian higher education sector. 2.2 research modelling approach coates (2006) used the statistical technique of congeneric measurement modelling to ascertain the identified qualities as being reflected in the validity of the generated latent online student engagement constructs, their sets of associated survey items and in calibrating the scales. the technique is described as providing ‘…a means of determining the strength of relationship between each item and the targeted latent scale, the extent to which the items from a common scale, and the existence of off-dimensional cross-loadings between items’ (coates, 2006, p.72). the six online student engagement scales used by coates in his survey instrument covered: 1. online active learning; 2. online social interaction; 3. online collaboration; 4. online teaching; 5. online academic relevance, and 6. online contact with staff. these were seen as relatively distinct qualities and properties of the online student experience. therefore, it is important to investigate this knowledge gap as there is little research that elucidates the motivational factors of student engagement. this research utilises coates’s (2006) student engagements constructs to investigate if they are evident in the online learning supports and campus-based blended education environment within a tertiary business education setting. 3 research design the development of the current student engagement survey instrument by pye et al. (2015) selected from coates’s (2006) work, the six online constructs for the study of student engagement in online and blended learning settings in commerce business education: 1. online active learning: key ways in which students use online systems to enhance learning. 2. online social interaction: use of online learning systems to support general forms of social interaction. 3. online collaboration: students’ use of online systems to work collaboratively with peers. 4. online teaching: whether teachers use online learning systems to promote effective learning. 5. online academic relevance: use of online systems to enhance the relevance and context of study. 6. online contact with staff: extent and ease of use of online systems by students in contacting staff. the items for each construct were the same as those used by coates (2006) for each of the adopted scales. these items for each construct are listed below in table 1. the student engagement survey used these six key latent constructs identified by coates (2006), and the associated activities indicating quality and quantity of time and effort related to each. thus, previously established, and validated items were chosen to measure each construct. one further (a seventh) student engagement construct identified by pye et al. (2015) was added which related student engagement to various activities associated with assessment. australasian journal of information systems pye et al. 2022, vol 26, research article factor structures associated with online student engagement 5 3.1 the assessment construct assessment redesign has been a cornerstone of major institutional curriculum review and renewal projects in australian higher education (see lawson, 2015; oliver, 2013). the strategic importance of relevant, innovative, and efficient assessment design and feedback approaches has been a major focus of higher education research and development (boud, 2010; nicol, 2010; boud & molloy, 2013; henderson, boud, molloy, et al., 2016; small & attree, 2016; winstone, nash, rowntree & parker, 2017). the domain of online assessment is therefore considered as a potentially significant student engagement construct by this investigation. in the 2014 survey design, inter alia, items were included around student engagement with online assessment feedback: ‘online assignment feedback helped me improve my understanding of the topic’; and ‘online assignment feedback helped me prepare for subsequent work’. in recognition of the specific importance of online assessment, feedback rubrics in the digital learning sphere, the two 2014 survey items were refined into the following three, in a revised version of the 2015 survey: • ‘access to the online assessment rubric before the assignment submission deadline, helped me with my assignment task’. • ‘after my assignment was marked, the online assessment rubric clearly explained the marks i had received’. • ‘after my assignment was marked, the online assessment rubric feedback comments helped me to improve my learning’. the inclusion of these refined items around online assessment is consistent with recent work emphasising the importance of well-designed assessment rubrics for advancing student learning (parkin, hepplestone, holden, irwin & thorpe, 2012; jonsson, 2014; mohoney, 2016; dawson, 2017). the best design of online assessment rubrics has come into sharper focus in fields like business education, which attract large and diverse student populations in australian higher education (atkinson & lim, 2013). it is surmised that coates (2006) did not focus on online assessment because of the technical limitations of the associated lms assessment functions, given the time his research was undertaken in the mid-2000s. the lms has now negotiated their experimentation and passage into mainstream use, with the last ten years seeing considerable maturation in the effective use of various lms features and assessment functions. one feature, the online assessment rubric, was not a feature available in early lmss. however, palmer and holt (2009; 2010; 2012 and 2014) reported the maturation in the importance and satisfaction attributed to different lms functions relating to the one original unchanged system, from both teacher and student perspectives. their historic findings (palmer & holt, 2014) in using a large, repeated crosssectional and quantitative survey spanning the period 2004-2011, reported that online assessment was still a major area requiring greater improvement in the student learning experience, an area now addressed by online assessment rubrics. additionally, pye et al. (2018), have reported significant variations in online student engagement in contemporary blended and distance education environments by gender, age, and cultural background. these substantive findings aside, it is timely to return to a statistical analysis of the robustness of coates’s (2006) original work, along with the addition of assessment to the six original online student engagement constructs. such an analysis of the now seven student engagement constructs would suggest the potential to extend the australasian journal of information systems pye et al. 2022, vol 26, research article factor structures associated with online student engagement 6 application of student engagement into the contemporary world of higher education teaching and learning practices, academic policy development and research. 3.2 research limitations the design of this online student engagement survey encompasses both satisfaction and frequency of use approaches to understanding student-learning experiences in the online and blended domain of business commerce education. the hybrid design was adjudged as the best approach for investigating student engagement at the micro unit/subject level of the business commerce student experience. while not germane to this meta-analysis, the survey has drawn upon the online student engagement constructs and their descriptors, and associated items as reported by coates (2006). the item response scales used levels of satisfaction rather than frequency of usage. this avoids the use of uniform frequency of use response scales that can lead to multiple student interpretations unless such anchor points clearly specify the level of usage in question. uniform response scales across a broad range of constructs and items can create syntactical problems between response scale descriptor, and item wording. the interpretation issue was examined by howson and buckley (2017) in the development of the uk national student engagement survey where student responses to ‘how often/to what extent’ survey items can be seen to depend on varying student expectations and on the priority and regularity of the use of the activities in question. frequency of use positively correlates with student learning value and satisfaction; higher levels of satisfaction can also positively relate to higher levels of usage. this matter requires further methodological consideration relating to online student engagement, which is outside the scope of this paper. suffice to say that the online and blended student experience across all three dimensions of importance, satisfaction, and frequency of use, are all of vital significance. in addition, the observed advent of learning analytics and associated applications can now make it less necessary to rely on students’ self-reported usage of key learning system features and activities, at least to the extent students are learning online. 4 data collection and analysis closed-ended questions allowed respondents to express their level of agreement with each of the items associated with the six key latent constructs proposed by coates, as well as the additional proposed student engagement construct of assessment from pye et al. (2015). students were asked to rank their level of agreement with each of the items using a likert scale represented using 2 ‘strongly disagree’ through to 6, ‘strongly agree’”. for all responses sought, participants were given the option to indicate that a response was 1, ‘not applicable’. for modelling and summary purposes, all ‘not applicable’ responses were removed, and all other ordinal responses were rescaled from 1, ‘strongly disagree’ through to 5, ‘strongly agree’. data collection was undertaken anonymously using self-selection sampling via an online survey of student participants utilising the online survey tool ‘opinio’ administered through the strategic intelligence and planning unit at deakin university. data collection occurred at the end of each teaching period during 2014 and 2015 across commerce subjects in accounting and information systems. the research project and survey instrument received ethics approval from the faculty of business and law’s ethics committee. the total number of responses was 569 out of a total population of 7353 surveyed, equating to a 7.74% overall response rate. there is no shortage of recommendations regarding the required sample size to conduct factor analysis, ranging from as few as three, to as many as 10 times the number of variables in the australasian journal of information systems pye et al. 2022, vol 26, research article factor structures associated with online student engagement 7 data set (maccallum et al., 1999). our number of observations to variables ratio is 19 observations per variable. the limitations associated with self-selected sampling methods are noted. however, as this exploratory research is conducted within one educational institution only, and as the investigation attempts to consider online learning at this institution, a selfselection sampling method provides the most efficacious outcome. a summary of the constructs, their items and their mean score is presented in table 1 and ibm spss ver. 22.0 was employed for the factor analysis modelling. latent constructs questions no’s mean std. dev active learning i used online materials to improve my learning 539 4.19 0.81 i used online material to make classroom learning more meaningful 511 3.91 0.95 i identified expected academic standards using the [lms] unit site 537 4.00 0.91 i found that online materials challenged me to learn 539 3.47 1.09 social interaction teaching staff participated in online discussions 508 4.09 0.85 i found it easy to explain my ideas in online discussions 445 3.60 1.05 i had helpful online discussions with other students 420 3.44 1.15 i met new people when using the [lms] 424 2.69 1.26 collaboration i used the [lms] with other students around the campus (on-campus students only) 378 3.45 1.11 i used the [lms] to work with other students outside the class (on-campus students only) 374 3.11 1.23 i used the [lms] to do academic work with other students 450 3.24 1.22 i used the [lms] to communicate with other students 464 3.21 1.22 teaching staff used [lms] to clarify what was required to do well 503 4.04 0.87 staff used {lms] in ways that improved the overall teaching 500 3.97 0.88 academic staff made an effort to communicate with students online 501 4.20 0.78 academic staff used [lms] to provide students with extra assistance 500 4.05 0.89 teaching staff used [lms] to tailor activities for students 494 3.63 1.04 teaching staff used [lms] to discuss interesting issues 494 3.60 1.05 assessment assignments were easy to locate online 499 4.26 0.83 tests were easy to locate online 468 4.19 0.89 assignments were easy to submit online 499 4.23 0.90 online assignment submission gave me more time to complete assessable work 497 4.13 0.95 access to the online assessment rubric before the assignment submission deadline, helped me with my assignment task 343 4.21 0.86 after my assignment was marked, the online assessment rubric clearly explained the marks i had received 424 3.81 1.08 australasian journal of information systems pye et al. 2022, vol 26, research article factor structures associated with online student engagement 8 after my assignment was marked, the online assessment rubric feedback comment helped me to improve my learning 421 3.73 1.11 relevance using [lms] made my study seem more relevant 494 3.75 1.01 using [lms] made me feel part of the university 495 3.75 1.13 using online materials helped me put my study in realworld contexts 494 3.61 1.09 contact with staff i used [lms] to contact academic staff 435 3.76 1.09 i found it easy to communicate with staff online 432 3.83 0.99 i had individual contact with academic staff online 391 3.54 1.19 table 1. descriptive statistics for all items of each latent construct note: assessment item 5 (italicised in the table above) was as mentioned, added to teaching period one in the 2015 iteration of the survey and is not included in subsequent analysis. all items were evaluated for reliability and internal consistency before investigating factor structures. it was necessary to assess the internal consistency of the factors to identify whether or not the proposed items were all measuring the same underlying construct (pallant, 2005). 4.1 data consistency analysis cronbach’s alpha was used to determine if the assumption of reliability (internal consistency) had been met. cronbach’s alpha was calculated separately for the items of each construct. consideration of consistent and reliable variance was then considered using an estimate of the combined composite score for all items associated with each construct. table 2 presents the results of the test for reliability using cronbach’s alpha. construct cronbach’s alpha cronbach’s alpha based on standardised items number of items active learning 0.789 0.796 4 social interaction 0.791 0.793 4 collaboration 0.868 0.868 4 teaching 0.881 0.884 6 assessment 0.850 0.856 6 relevance 0.874 0.875 3 contact with staff 0.852 0.854 3 table 2. reliability statistics the assumption of reliability appears to have been met in table 2 results. according to hinton, mcmurray and brownlow (2004), reliability is high when values ranging between 0.00 (more error) and 1.00 (no error), are closer to 1.00. a cronbach’s alpha score below 0.60 is considered unacceptable, 0.60 to 0.65 undesirable yet worth investigating further, 0.65 to 0.70 acceptable, 0.70 to 0.80 good, above 0.80 very good. reliability relates to the amount of random error from the measurement process that might be evident within the scores. highly reliable scores are precise, reproducible, and consistent across multiple testing instances when a measure produces similar results under consistent application. assumptions associated with convergent and discriminant validity were assessed. convergent validity assesses whether or not the construct items in effect cluster together to form that construct, whereas discriminant validity assesses if constructs are different from one australasian journal of information systems pye et al. 2022, vol 26, research article factor structures associated with online student engagement 9 another. both convergent and discriminant validity were assessed using the pattern component matrix of the separate question items and the extracted factors (table 4). an initial analysis utilised anova, t, and z tests to examine the potential for demographic differences both between and within the various demographic characteristics measured. however, this had no bearing on the results of the factor analysis undertaking and are therefore not presented. 4.2 exploratory factor analysis an exploratory factor analysis was undertaken to test if coates’ (2006) hypothesised six constructs as well as the new hypothesised ‘assessment’ construct provided a reasonable fit to the data, exploratory factor analysis was used to estimate the model and assess its fit to the 569 student responses. exploratory factor analysis was used to determine if the thirty student engagement items could be scaled into a reduced number of constructs that could provide a more meaningful basis for understanding how the activities associated with student engagement might be structured (sass & schmitt, 2010; schmitt, 2011). factors were extracted using principal components using the varimax rotation method with kaiser normalisation. for the dataset of students (n=569), bartlett's test of sphericity is significant (p< 0.000), indicating that dimension reduction exploratory factor analysis will provide a reasonable solution. further, a kaiser-meyer-olkin (kmo) indicator of 0.913 also supports this analytical approach. factor analysis is a modelling approach that allows a number of unobserved latent variables to be estimated that purportedly help explain the variability in the original, observed data. it is widely known that ‘retaining too few or too many factors’ has serious ramifications regarding the interpretation and the stability of the resulting factor patterns (preacher, zhang, kim & mels, 2013). in fact, hubbard and allen (1989) (cited in preacher et al., 2013) describe it as ‘one of the thorniest problems a researcher faces’. a population’s factor structure for any set of latent constructs can only be hypothesised, that is, there is no one true model (cudeck & henly, 2003). as it is impossible to identify the true and correct number of factors to extract. the researchers therefore discuss and present those factors considered worthwhile to retain, specifically, the factors identified by coates (2006) and pye et al. (2015). total variance explained components initial eigenvalues rotation sums of squared loadings total % of variance cumulative % total % of variance cumulative % 1 12.151 40.503 40.503 4.805 16.016 16.016 2 2.439 8.131 48.634 3.92 13.066 29.082 3 1.641 5.471 54.105 3.281 10.935 40.017 4 1.493 4.975 59.08 2.941 9.803 49.82 5 1.422 4.738 63.818 2.364 7.881 57.701 6 1.249 4.163 67.982 2.218 7.394 65.096 7 1.054 3.512 71.494 1.919 6.398 71.494 extraction method: principal component analysis. table 3. eigenvalues and the proportion of variance explained by each of the extracted factors factors with an eigenvalue of greater than 1.00 were retained and these factors explained 71.5% of the variance in the data set kaiser (1960) (table 3). other factor models were examined but they did little to improve the amount of variance explained. a scree plot australasian journal of information systems pye et al. 2022, vol 26, research article factor structures associated with online student engagement 10 graph of the eigenvalues and the extracted components is presented as figure 1. the percentage of variance explained by each extracted factor was high, with each factor explaining between 6.4% and 16.0% of the total variation in the observed data. figure 1. scree plot of eigenvalues and components extracted the resulting factor structure is presented in table 4. factor loadings of greater than |0.300| are presented (hair, anderson, tatham & black, 1998; 2009). factor loadings above 0.572 are bolded. seven distinct factors were extracted, except for the hypothesised construct of relevance, the extracted factors confirm five of the constructs defined by coates (2006). the latent constructs, in order of extraction were: collaboration (construct 1); teaching (construct 2); assessment a (construct 3); active learning (construct 4); contact with staff (construct 5); assessment b (construct 6) and social interaction (construct 7). the items associated with coates’ hypothesised construct of relevance, were not distinctly separable and revealed cross-loading with several other extracted factors. this indicates that in the online learning environment, relevance does not seem to be a separate construct. rather, it disappears into the other hypothesised constructs of mostly collaboration (1), but it does also cross-load with teaching (2) and active learning. notably, active learning (4) was the only other item that did not significantly load, and similarly, exhibited cross-loading with collaboration (1). using a cut-off criterion of 0.500, all assessment items loaded with an estimated factor. however, two distinct factors were extracted for the hypothesised construct of ‘assessment’ (6). the hypothesised overarching themes of each of the new constructs, based on the separation of items associated with assessment, are presented in table 5 and cluster as follows. these results indicate that student engagement aspects of assessment in the online environment reveal two distinct latent constructs: one associated with access and submission of assessment items, and the other revealed as post assessment feedback or discussion. australasian journal of information systems pye et al. 2022, vol 26, research article factor structures associated with online student engagement 11 rotated component matrix 1 2 3 4 5 6 7 collaboration 2 0.815 collaboration 3 0.767 collaboration 1 0.740 social interaction 1 0.730 collaboration 4 0.714 teaching 2 0.310 0.730 teaching 1 0.722 teaching 3 0.708 0.396 teaching 4 0.698 teaching 5 0.451 0.539 teaching 6 0.447 0.572 assessment 2 0.838 assessment 1 0.826 assessment 3 0.817 assessment 4 0.586 active learning 2 0.821 active learning 1 0.783 active learning 3 0.749 contact with staff 7 0.833 contact with staff 8 0.728 contact with staff 9 0.305 0.689 assessment 7 0.817 assessment 6 0.815 social interaction 2 0.385 0.701 social interaction 1 0.361 0.692 social interaction 3 0.476 0.613 active learning 4 0.435 0.466 relevance 1 0.415 0.452 0.340 0.373 relevance 2 0.483 0.32 0.323 0.340 relevance 3 0.413 0.419 0.383 0.435 table 4. factor structure matrix. extraction method: principal components. rotation method: varimax with kaiser normalisation factors correlated significantly with assessment a access and submission assessment item 1: assignments were easy to locate online assessment item 2: tests were easy to locate online assessment item 3: assignments were easy to submit online assessment item 4: online assignment submission gave me more time to complete assessable work assessment b post assessment feedback assessment item 5: after my assignment was marked, the online assessment rubric clearly explained the marks i had received assessment item 7: after my assignment was marked, the online assessment rubric feedback comment helped me to improve my learning table 5. separation and clustering of assessment item 4.3 tablestudent engagement summary findings in summary overall, except for the construct ‘relevance’, all other constructs identified by coates (2006) appear to populate a discrete and yet distinct factor structure space. furthermore, two new latent constructs associated with assessment seem apparent. the australasian journal of information systems pye et al. 2022, vol 26, research article factor structures associated with online student engagement 12 student engagement constructs revealed in the online learning environment using exploratory factor analysis are: 1. collaboration. 2. teaching. 3. assessment a: access and submission. 4. active learning. 5. contact with staff. 6. assessment b: post assessment feedback. 7. social interaction. these findings serve to first, reconfirm that the initial six constructs of student engagement as identified by coates (2006) remain apparent in the traditional campus-based environment as expected, but are also evident in the online settings positioned within the domain settings of blended online learning environment. secondly, tertiary practitioners would consider that the assessment construct has always been an ad hoc or anecdotal motivating factor influencing student engagement within the learning environment. however, and notably this research confirms and evidences the existence of the previously unidentified new construct of assessment as being an apparent motivating construct of student engagement. furthermore, the exploratory factor analysis undertaken indicates the presence of two new, important student engagement latent constructs. to this end, the assessment item revealed two additional distinct factors ‘access and submission’ and ‘post assessment feedback’. generally, the results of this analysis agree with the constructs as defined by coates (2006) for student engagement, in that they exist in an online learning context, and that they are also apparent and relevant in an online and on-campus (location learning) blended learning context. however, the separation of the items for assessment into two distinctly separate latent constructs implies that student’s perspectives regarding this theme are also markedly important latent factors motivating student engagement. these results indicate that not only is ‘assessment’ a distinctly important aspect of online student engagement, but that the hypothesised construct of ‘assessment’ has two markedly important features. that is, within the context of the ‘assessment’ items presented, ‘access and submission’, as well as ‘post assessment feedback’ are identified as distinctly important characteristics of student experience in the online engagement features of assessment. 5 conclusion it was established that coates (2006) student engagement constructs do likewise coalesce within the domain settings of blended online learning design and practice in an australian higher education business context. the analysis of the survey instrument responses confirms this through the results of the exploratory factor analysis. although anecdotally expected that assessment was a student motivator, the discovery and confirmation of the ‘assessment’ construct was unexpected and highlights the strong motivational factor that ‘assessment’ has on student engagement and subsequent learning. furthermore, the exceptions confirming the validity of all online student engagement constructs (and their associated items) related to the construct of ‘relevance’, which is not deemed as important or as apparent as initially thought, australasian journal of information systems pye et al. 2022, vol 26, research article factor structures associated with online student engagement 13 while the ‘assessment’ construct, breaks down into principally two factors: ‘access and submission’ and ‘post assessment feedback’. overall, the analysis confirms the robustness of conceptualising the phenomenon of online student engagement across all discretely identified constructs. business faculties specifically, and universities in general, have committed to the systematic use of various types of student evaluation instruments to assure and improve the educational quality of their subject and course offerings. such student evaluation instruments could be designed specifically to measure the student experience perspectives from different types of learning contexts and learning activities. the extension of the student engagement survey to include ‘assessment’ constructs provides a theoretically sound and practical method for illuminating the student experience as applied to integrated online and blended learning environments, which are now the designed norm in the sector. current and changing external environmental factors make it imperative to consider all aspects of the student experience in an increasingly online driven learning world. this in turn reflects dramatic shifts in national and global workplace settings, which is shaped by ongoing developments in the global digital economy. student engagement continues to increasingly be the point of focal importance in formal education, workplace learning and workplace practices. capturing and responding to these learning experience constructs through the prism of student engagement research, approaches and methods is of major relevance to increasing student engagement and learning enthusiasm. these global trends have gone into hyper-drive with the advent of the covid-19 pandemic and the imperative to take university studies online on an unprecedented scale in the context of the wide-spread shutdown of university campuses across the world. the aftermath will see online education anchored into its prime role in supporting all forms of teaching, learning and workplace training. in relation to assessment feedback and the use of rubric design, further research and development is needed relating to the use and value of feedback mechanisms enabled by lmss. rubrics and actioning feedback have dominated assessment practices in traditional higher education. for example, rubrics have been applied to the use of traditional forms of assessment tasks, i.e., essays, problem sets and project assignments requiring written student submissions. with the move towards shorter, modularised higher education courses, graduate professional skills development programs and workplace training endeavours, with an increasing variety of forms of oral, live, group and multimedia response formats, ‘rubric’ design and lms feedback tools need further elaboration and refinement to enhance student engagement. online feedback tools need to be broadened to include media responses other than text, adaptations for different types of oral and written student responses, both individual and group, and need to be deployed rapidly and efficiently to improve future student learning. these will be increasingly augmented by live and virtual assessment task debriefing sessions with quick mobilisation of assessment feedback regimes to operate in truncated education and training study periods, adapted to the needs and circumstances of students’ work and life preferences. future research proposes to expand to include undergraduate students from across all university faculties and other higher education establishments. this will significantly broaden the research focus and test the veracity of student engagement constructs in differing higher education discipline contexts, including postgraduate student cohorts. student engagement, as an overarching concept and its influential measurement in the context of the student australasian journal of information systems pye et al. 2022, vol 26, research article factor structures associated with online student engagement 14 experience of online and blended learning environments. continues to hold great promise in advancing learning theory, integration into educational policy, and extending student engagement motivational practice in the field. references astin, a.w. (1985). achieving educational excellence: a critical analysis of priorities and practices in higher education. jossey-bass: san francisco. atkinson, d. & lim, s.l. (2013). improving assessment processes in higher education: student and teacher perceptions of the effectiveness of a rubric embedded in a lms. australasian journal of educational technology. 29(5), pp. 651-666. bonk, c.j. & zhang, k. (2008). empowering online learning: 100 + activities for reading, reflecting, displaying, & doing. john wiley & sons: san francisco. boud, d. (2010). student assessment for learning in and after courses. final report for senior fellowship. australian learning and teaching council (altc): http://www.olt.gov.au/resource-student-assessment-learning-and-after-courses-uts2010 boud, d. & molloy, e. (2013). (eds.). feedback in higher and professional education understanding it and doing it well. routledge: london. chickering, a.w. & gamson, z.f. (1987). seven principles for good practice in undergraduate education. aahe bulletin, 39(7), pp. 3-7. coates, h. (2006). student engagement in campus-based and online education: university connections. routledge: london. coates, h. (2010). development of the australasian survey of student engagement (ausse). higher education, 60, pp. 1-17. coates, h. & mccormick, a.c. (eds.) (2014). engaging university students international insights from system-wide studies. springer: singapore. coates, h. & mccormick, a.c. (2014). introduction: student engagement – a window into undergraduate education. in h. coates & a.c. mccormick (eds.) (2014). engaging university students international insights from system-wide studies. springer: singapore, pp.1-12. cudeck, r., & henly, s. j. (2003). a realistic perspective on pattern representation in growth data: comment on bauer and curran (2003). psychological methods, 8, pp.378–383. dawson, p. (2017). assessment rubrics: towards clearer and more replicable design, research and practice. assessment & evaluation in higher education, 42(3), pp. 347-360. doi: 10.1080/02602938.2015.1111294 garrison, d.r. & anderson, t. (2003). e-learning in the 21st century. a framework for research and practice.routledge falmer: london. hair, j., anderson, r., tatham, r. and black, w. (1998). multivariate data analysis (5th edition). prentice hall: new jersey. hair, j, f. (jr), black, j. f., babin, j. & anderson r. e. (2009). multivariate data analysis (7th edition). prentice hall: new jersey. isbn-10: 0138132631 • isbn-13: 9780138132637 henderson, m., boud, d., molloy, e., dawson, p., phillips, m., ryan, t., walker, h. & mahoney, p. (2016). feedback for learning closing the assessment loop. federal australian government office for learning and teaching (in progress). project website: https://feedbackforlearning.org/ hinton, p. r., mcmurray, i. & brownlow, c. (2014). spss explained (2nd edition). routledge taylor and francis: east sussex, uk. australasian journal of information systems pye et al. 2022, vol 26, research article factor structures associated with online student engagement 15 howson, c.k. & buckley, a. (2017). development of the uk engagement survey. assessment & evaluation in higher education, 42(7), pp. 1132-1144. doi: 10.1080/02602938.2016.1235134 jonsson, a. (2014). rubrics as a way of providing transparency in assessment. assessment & evaluation in higher education, 39(7), pp. 840-852. doi: 10.1080/02602938.2013.875117 kaiser, h.f. (1960). the application of electronic computers to factor analysis. educational and psychological measurement, 20, pp. 141-151. kuh, g.d. (2003). what we’re learning about student engagement from nsse. change, 35(2), pp. 24-31. kuh, g.d., kinzie, j., schuh, j.h., whitt, e.j., & associates (2005). student success in college: creating conditions that matter. jossey-bass: san francisco. lawson, r. (2015). curriculum design for assuring learning – leading the way. final report for senior fellowship. office for learning and teaching (olt): http://www.olt.gov.au/resource-curriculum-design-assuring-learning-leading-way2015 maccallum, r. c., widaman, k., zhang, shaobo. (1999) sample size in factor analysis. psychological methods, 4(1), pp. 84-99 doi:10.1037/1082-989x.4.1.84 nicol (2010). re-engineering assessment practices in higher education: http://www.reap.ac.uk/home.aspx oliver, b. (2013). graduate attributes as a focus for institution-wide curriculum renewal: innovations and challenges. higher education research & development, 32(3), pp.450-63. pallant, j. (2005). spss survival manual. a step by step guide to data analysis using spss for windows (ver 12.0). allen and unwin press: crows nest, australia. pace, c.r. (1979). measuring outcomes of college: fifty years of findings and recommendations for the future. jossey bass: san francisco. pace, c.r. (1995). from good practices to good products: relating good practices in undergraduate education to student achievement. paper presented at the association for institutional research, boston. palmer, s., & holt, d. (2009). staff and student perceptions of an online learning environment: difference and development. australasian journal of educational technology, 25(3), pp. 366 381. palmer, s., & holt, d. (2010). students’ perceptions of the value of the elements of an online learning environment: looking back in moving forward. interactive learning environments, 18(2), pp. 135-151. palmer, s. & holt, d. (2012). trajectories of engagement: a repeated cross-sectional investigation of student perceptions of an online learning environment. research in learning technology, 20, pp. 253-265. palmer, s. & holt, d. (2014). development of student and academic staff perceptions of the elements of an online learning environment over time. australasian journal of educational technology, 30(4), pp. 375-389. parkin, h.j., hepplestone, s., holden, g., irwin, b. & thorpe, l. (2012). a role for technology in enhancing students’ engagement with feedback. assessment & evaluation in higher education, 37(8), pp. 963-973. doi: 10.1080/02602938.2011.592934 pye, g., holt, d., & salzman, s., bellucci, e., lombardi, l. (2015). engaging diverse student audiences in contemporary blended learning environments in australian higher business education: implications for design and practice. australasian journal of information systems, 19, pp. 1-20. australasian journal of information systems pye et al. 2022, vol 26, research article factor structures associated with online student engagement 16 pye, g., holt, d. & salzman, s. (2018). investigating different patterns of student engagement with blended learning environments in australian business education: implications for design and practice. australasian journal of information systems, 22, pp. 1-23. preacher, k. j., zhang, g., kim, c. & mels, g. (2013). choosing the optimal number of factors in exploratory factor analysis: a model selection perspective. multivariate behavioral research, 48, pp. 28–56, issn: 0027-3171 print/1532-7906 online. doi: 10.1080/00273171.2012.710386. salmon, g. (2011). e-moderating. the key to teaching and learning online. routledge: new york. sass, d.a., schmitt, t.a. (2010). a comparative investigation of rotation criteria within exploratory factor analysis. multivariate behavioral research, 45 (1), pp. 73-103. schmitt, t.a. (2011). current methodological considerations in exploratory and confirmatory factor analysis. journal of psychoeducational assessment, 29(4), pp. 304-321. small, f. & attree, k. (2016). undergraduate student responses to feedback: expectations and experiences, studies in higher education, 41(11), pp. 2078-2094. doi: 10.1080/03075079.2015.1007944 tinto, v. (1993). leaving college: rethinking the causes and cures of student attrition. university of chicago press: chicago. tinto, v. (1997). classrooms as communities: exploring the educational character of student persistence. journal of higher education, 68(6), pp. 599-623. winstone, n.e., nash, r.a., rowntree, j. & parker, m. (2017). ‘it’d be useful, but i wouldn’t use it’: barriers to university students’ feedback seeking and recipience. studies in higher education, 42(11), pp. 2026-2041. doi: 10.1080/03075079.2015.1130032 copyright: © 2022 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v26i0.2667 http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 literature review 2.1 student engagement 2.2 research modelling approach 3 research design 3.1 the assessment construct 3.2 research limitations 4 data collection and analysis 4.1 data consistency analysis 4.2 exploratory factor analysis 4.3 tablestudent engagement summary findings 5 conclusion paper title (paper title style) australasian journal of information systems fischer et al. 2023, vol 27, selected papers from acis 2022 data-driven organizations 1 data-driven organizations: review, conceptual framework, and empirical illustration hannes fischer tu dresden, germany hannes.fischer@mailbox.tu-dresden.de martin wiener tu dresden, germany susanne strahringer tu dresden, germany julia kotlarsky university of auckland, new zealand katja bley tu dresden, germany abstract with companies and other organizations increasingly striving to become (more) data-driven, there has been growing research interest in the notion of a data-driven organization (ddo). in existing literature, however, different understandings of such an organization emerged. the study at hand sets forth to synthesize the fragmented body of research through a review of existing ddo definitions and implicit understandings of this concept in the information systems and related literatures. based on the review results and drawing on the established concept of the “knowing organization,” our study identifies five core dimensions of a ddo— namely, data sourcing & sensemaking, data capabilities, data-driven culture, data-driven decision-making, and data-driven value creation—which we integrate into a conceptual ddo framework. most notably, the proposed framework suggests that—like its predecessor, the knowing organization—a ddo may draw on an outside-in view; however, it may also draw on an inside-out view, or even combine the two views, thereby setting itself apart from the knowing organization. to illustrate our conceptual ddo framework and demonstrate its usefulness, we apply this framework to three empirical examples. theoretical and practical contributions as well as directions for future research are discussed. keywords: data-driven organization (ddo), ddo understandings, ddo dimensions, knowing organization, literature review, conceptual ddo framework, empirical examples. 1 introduction to prepare themselves for the digital future, organizations around the world are increasingly engaging in large-scale digital transformation initiatives (cf. fischer et al., 2023). many of these initiatives revolve around data (gartner, 2021), which have been widely acknowledged as a key driver of economic growth (mckinsey, 2013; wef, 2021). for example, mcafee & brynjolfsson (2012) report a 5% increase in productivity and 6% growth in profitability for public north american companies driven by data; and rubin & rubin (2013) observe higher stock returns for such companies. relatedly, according to thusoo & sarma (2017), 84% of executives believe that most to all of their employees should use data to help them perform their job duties as organizations that act based on data show significant advantages over australasian journal of information systems fischer et al. 2023, vol 27, selected papers from acis 2022 data-driven organizations 2 competitors (berndtsson et al., 2018; constantiou & kallinikos, 2015). among other things, this is because organizations that are driven by data have been found to operate with improved processes and innovative data-enabled products and business models (sivarajah et al., 2017), and to make better decisions in general (mcafee & brynjolfsson, 2012; svensson et al., 2019). it is in this context that organizations are striving to become a data-driven organization (ddo; davenport & bean, 2018; hartmann et al., 2016) to benefit from the value potential embedded in data. with the growing interest in the concept of a ddo, however, multiple understandings of this concept1 have emerged in the academic and practitioner literature. more specifically, existing ddo understandings appear to vary considerably, ranging from rather simplistic to more complex. for example, while schüritz (2017) refers to a ddo as an organization that simply “uses data and analysis to help drive action” (p. 394), other authors such as thusoo & sarma (2017) include multiple characteristics such as data-driven culture, data-based decision-making and technological capabilities in their more complex ddo understanding. these diverse understandings in both the academic and practitioner literature make it challenging for scholars to further advance the body of knowledge on this emerging phenomenon. combined with the steadily increasing number of publications in the information systems (is) and related literatures, these conceptual challenges led us to the conclusion that a systematic review of existing understandings is needed and that an integrative conceptual ddo framework will help consolidate the state of the art and reveal the core nature of the ddo phenomenon. against this backdrop, our study aims to (1) synthesize different understandings of ddos in the literature, and (2) derive a conceptual ddo framework that integrates the identified understandings. to achieve our research objectives, we employ a two-step process. first, we follow established guidelines to conduct a systematic literature review of ddo understandings. second, based on the insights gained, we draw on the concept of the ‘knowing organization’ (choo, 1996) to develop a conceptual ddo framework. to this end, our study is structured as follows: in the next section, we review key concepts related to the notion of a ddo and introduce our guiding framework. we then detail our methodological approach, followed by the presentation of our review findings and the resulting conceptual ddo framework. subsequently, to illustrate our framework and demonstrate its applicability and usefulness, we apply it to three empirical examples (two ddos and one organization that is arguably not data-driven). finally, we conclude by discussing the main contributions of our study, along with its limitations and related directions for future research. 2 conceptual foundations the notion of a ddo combines the concepts of an ‘organization’ and ‘data’. after reviewing these two concepts and their interrelationships, we will introduce choo’s (1996) concept of the knowing organization, which we will use as the theoretical basis for developing our conceptual framework of a ddo. 1 in this paper, we use the term “ddo understanding” as an umbrella term for both explicit ddo definitions and (more or less) implicit descriptions of this concept. australasian journal of information systems fischer et al. 2023, vol 27, selected papers from acis 2022 data-driven organizations 3 2.1 the concept of an organization broadly speaking, “organizations are systems of coordinated action among individuals and groups whose preferences, information, interests or knowledge differ” (march & simon, 1993, p. 2). in this regard, it has been argued that an organization fulfills two main functions: division of labor and (re-) integration of efforts (puranam et al., 2014), which require both coordination and cooperation, which in turn need (data and) information next to motivation and trust (gulati et al., 2005; lawrence & lorsch, 1967). in general, the organization concept can be considered from two complementary perspectives: a system-oriented and a processoriented perspective (hall, 1977). from a system-oriented perspective, and in line with modern organizational theory, an organization can be seen as an open system that continuously affects and is affected by its environment (e.g., boulding, 1956; katz & kahn, 1978). as such, organizations can be described as “‘sensemaking systems’ which perpetually create and re-create conceptions of themselves and of all around them" (johnson & kruse, 2019, p. 10) and “collect, manage, and use the information they receive” (west & turner, 2014, p. 301). from a process-oriented perspective, theorists such as weick (1969) consider the noun ‘organization’ itself, with its implications of a static and figured-out concept, to be a “myth” (p. 88). rather, he argues, people engage in the process of organizing, defined as “the resolving of equivocality in an enacted environment by means of interlocked behaviors embedded in conditionally related process” (weick, 1969, p. 11). essential to this process are the acts of information processing and decision-making, two concepts that simon (1976) and choo (1996) place at the center of their definitions of an organization. simon’s (1976) notion of an organization is “the pattern of communications and relations among a group of human beings, including the processes for making and implementing decisions. this pattern provides to organization members much of the information and many of the assumptions, goals, and attitudes that enter into their decisions” (p. 18). choo presents the concept of an organization as a combination of both a process-oriented and a system-oriented view. on the one hand, he builds on simon’s (1976) process-oriented, decision-centric view of organizations; on the other hand, he draws on the system-oriented modern organizational theory, viewing organizations as systems that interact with their environment. integrating these two perspectives, he describes his understanding of an organization as a “decisionmaking system” (choo, 1996, p. 331). in today’s era of digitalization, the design of organizations and division of work is considerably influenced by emerging technologies such as data analytics that transform organizational decision-making, coordination and control (faraj & leonardi, 2022; kellogg et al., 2020; schwer & hitz, 2018; yoo et al., 2012). in this regard, the impact of these information technologies (it) goes far beyond aligning it to business functions (cf. brown & magill, 1994), as they fundamentally reshape business strategies (cf. bharadwaj et al., 2013; morakanyane et al., 2017) and the underlying organizations (yoo et al., 2012), thereby redefining their identity and value proposition (wessel et al., 2021). as a consequence, for faraj & leonardi (2022), it is an inherent part of today's understanding of an organization. building on this contemporary understanding, we develop our conceptual ddo framework. australasian journal of information systems fischer et al. 2023, vol 27, selected papers from acis 2022 data-driven organizations 4 2.2 the concepts of data and data-drivenness the term data refers to a “representation of facts, concepts or instructions in a formalized manner, suitable for communication, interpretation, or processing by humans or by automatic means” (hicks, 1993, p. 668). in the organizational context, martin & powell (1992; cited in hinton, 2006) describe data as “the raw material of organizational life; it consists of disconnected numbers, words, symbols, and syllables relating to the events and processes of the business” (p. 10). by providing context and meaning to data through human or automated interpretation and processing, they are transformed into information (hicks, 1993; tuomi, 1999), that in turn can be used in management decision-making (powell & martin, 1992; as cited in hinton, 2006). building on this understanding of data, the concept of data-drivenness can be explored. etymologically, the term drivenness refers to being propelled, guided, or controlled by something and is rarely used in isolation. consequently, “data-drivenness” can be defined as the property of being propelled, guided, or controlled by data. originally, the concept of data-drivenness was mainly applied to individual business functions that are particularly reliant on data, such as marketing (cf. data-driven marketing; see e.g., malhotra et al., 1999). however, as digitalization has progressed, entire value chains and business models have become centered around data, giving rise to the notion of data-driven business models (ddbm) (cf. e.g., wiener et al., 2020)2. nowadays, this development has progressed to the point where entire organizations are propelled, guided, or controlled by data, giving rise to the notion of ddos (thusoo & sarma, 2017). arguably, this notion can be seen as a contemporary version of what choo (1996) refers to as the “knowing organization” (ko). as such, we build on choo’s conceptualization of the ko and use it as a guiding framework. 2.3 guiding framework: the knowing organization in his seminal article on the ko, which has been cited extensively in the is and related literature, choo (1996) describes and refines the core functions of a learning organization and their interplay. building on choo’s understanding of an organization as a “decision making system” (choo, 1996, p. 331), knowing organizations can be described as organizations with “the ability to use information to gain a better understanding of their activities and their environment [and] achieve a competitive advantage by making better decisions and having clearly defined courses of action” (parra, 2022, p. 8), where “the creation and use of information [obtained from the external environment] play a strategic role in determining an organization’s capacity to grow and adapt” (choo, 1996, p. 329). according to choo (1996), a ko is an organization that effectively integrates the organizational knowing cycle, consisting of three interrelated organizational functions—namely, sensemaking, knowledge creating, and decision-making—that build on each other to form an outside-in view of how organizations absorb and utilize information (choo, 1996) by taking in signals and knowledge from the environment and processing these external inputs into internal knowledge, decisions, and behavioral changes (choo, 1996) (see figure 1). the sensemaking function is responsible for making sense of changes and developments in the organization’s 2 it should be noted that a business model can be centered around data without requiring the underlying organization to be data-driven, and vice versa. please refer to section 6 for a more detailed discussion on the conceptual distinction between a ddo and an organization with a ddbm. australasian journal of information systems fischer et al. 2023, vol 27, selected papers from acis 2022 data-driven organizations 5 external environment, by subjectively filtering out relevant information and forming possible explanations from past experiences (choo, 1996). an important part of this step is the organization’s purpose, which helps contextualize information during the interpretation. the second function of the ko, knowledge creating, focuses on the human-centric exchange of learning, e.g., through training and the conversion of tacit into explicit knowledge (and vice versa). through this exchange of knowledge among between organizational members, supplemented by external knowledge, an organization generates new capabilities and innovates. finally, the decision-making function draws on the generated knowledge, as well as shared purpose and meaning, to make rational, purposeful decisions. to do so, organizational decision makers design decision alternatives, adopt a set of evaluation criteria, and evaluate the alternatives to eventually arrive at decisions that lead to purposeful, adaptive internal behavior. figure 1. conceptual framework of the ko (based on choo, 1996, 2001) as noted above, we acknowledge that choo’s (1996) ko framework and its functions are still relevant to today’s organizations and thus provide a solid foundation for conceptualizing the notion of a ddo. at the same time, however, we also argue that choo’s framework needs to be updated and extended to account for the significant changes brought about in by the digital age. while choo’s (1996) ko framework was developed during an era focusing on creating organizational knowledge (parra, 2022), today—three decades later—organizational decision-making is characterized by data and extensive reliance on digital technologies (faraj & leonardi, 2022; parra, 2022), which should be reflected in the conceptual framework. furthermore, the original framework has an exclusive emphasis on internal value creation through decision making, thereby disregarding the relevance of external value creation. this suggests that the outside-in focus inherent in choo’s (1996) framework is too narrow, as today’s organizations also actively influence their environment in line with the open systems perspective of modern organizational theory. external knowledge new capabilities and innovations goal-directed adaptive behavior shared meanings and purpose shared meanings and purpose signals from the environment decision making knowledge creating sense making australasian journal of information systems fischer et al. 2023, vol 27, selected papers from acis 2022 data-driven organizations 6 3 research methodology to effectively organize prior literature on ddos, we followed established guidelines (webster & watson, 2002) and conducted a systematic literature review. our review approach showed several similarities to what paré describes as a theoretical review (paré et al., 2015). this particular type of review is often used to “tackle an emerging issue [such as the ddo phenomenon] that would benefit from the development of new theoretical foundations” and its primary goal is to “develop a conceptual framework” (paré et al., 2015, p. 188). furthermore, a theoretical review is usually broad in scope and draws on both conceptual and empirical sources (without quality assessment), which are mainly analyzed through content analysis or interpretive methods (paré et al., 2015). to conduct such a review, we adopted a hermeneutic approach (cf. boell & cecezkecmanovic, 2014; wittgenstein & anscombe, 1989), in which we continuously transitioned between search and acquisition of literature on the one hand and analysis and interpretation of our findings on the other hand. through this iterative and integrated approach, we were able to “re-interpret individual publications and their importance within a bigger ‘whole’” (boell & cecez-kecmanovic, 2014, p. 265) already during the search process, as we broadened our understanding of the literature. 3.1 literature search for our literature search and acquisition, we defined clear inclusion criteria that guided our selection of relevant studies. specifically, only studies concerned with a conceptual perspective on ddos (or synonymous terms such as “data-driven company” or “data-driven enterprise”) were included in our review sample. in addition, we decided, at least initially, to include only peer-reviewed, academic publications in our sample. to ensure a broad coverage of potentially relevant literature sources, we followed the four-step search process recommended by webster and watson (2002). specifically, we used the website litbasket.io (boell & wang, 2019) to conduct a keyword-based search covering the 51 most prestigious is journals (for a list see appendix a). moreover, we also searched the proceedings of leading is conferences (icis, ecis, amcis, pacis, hicss, and acis). for both searches, we used the same keywords. here, we anticipated that existing literature would use a broad range of different terms to refer to a ddo. therefore, we iteratively tested and refined our search string (e.g., by adding synonyms identified in the literature) until we derived our final search string that included nine possible combinations of the terms “data-driven”, “analyticsdriven”, “data-”, and the terms “organization”, “company”, and “enterprise”: as well as the term “data-drivenness” itself without any further limitations. this first step led to 42 distinct publications, of which only four elaborated on their ddo understandings. we then performed a backward search (step 2), followed by a forward search (step 3) (webster & watson, 2002) using google scholar to further supplement our base set of ddo-related publications. this snowball approach resulted in to 103 and 9 additional studies, respectively, yielding 10 additional descriptions of ddo understandings. in a fourth and final step, we conducted a supplementary database search to complement our review sample and to confirm that no new ddo understandings emerge (webster & watson, 2002). to do this, we searched the ais elibrary (aisel.aisnet.org) using the search string described above. furthermore, when analyzing our review sample, it became apparent that practitioner work—such as patil (2011) and anderson (2015)—is frequently cited in the academic literature as well (e.g., in fabijan et al., 2017; hupperz et al., 2021). therefore, we australasian journal of information systems fischer et al. 2023, vol 27, selected papers from acis 2022 data-driven organizations 7 decided to extend our search to the practitioner literature, using the google search engine (again with the above-described search string). in doing so, we identified an additional 52 publications, covering various adjacent fields and containing nine additional ddo understandings. taken together, our four-step search process yielded a total of 206 publications, of which 23 contained an explicit description of a ddo understanding and were therefore all used as the basis for our analysis. table 1 provides an overview of the search process. additionally, a detailed description of the review sample can be found in appendix b. category search process keyword-based search in leading outlets backward search forward search supplementary database search total total 42 103 9 52 206 thereof studies describing ddo understandings 4 6 4 9 23 table 1. search process and resulting number of publications 3.2 literature analysis already in parallel and closely intertwined with the search process (cf. boell & cecezkecmanovic, 2014; wittgenstein & anscombe, 1989), we thoroughly examined our review sample and arrived at our conceptual ddo framework dimensions in two main steps. first, we used a combination of descriptive coding and open coding to analyze our review sample. initially, we captured and documented descriptive metadata such as the literature type (academic/practitioner), outlet, research focus, and publication date of each publication (see appendix c) to grasp the context of each individual ddo understanding. the primary focus, however, was on the open coding of all ddo understandings to identify specific ddo characteristics highlighted by them. this was done by two authors in parallel and discussed iteratively within the authoring team to ensure the validity and reliability of the coding results. in case of disagreement, the relevant text passages were jointly revisited and thoroughly discussed until agreement was reached. second, we followed what can be described as an abductive process, in which we derived the dimensions of our conceptual ddo framework through a combination of deductive concept coding using choo’s (1996) ko framework as well as inductive coding building on the open codes identified in step one. in doing so, the emerging ddo characteristics were either mapped onto the organizational functions described in choo’s (1996) ko framework (sensemaking, knowledge creating, and decision-making), or were used to derive new ddospecific dimensions. in total, we identified five ddo dimensions, of which three are updated functions from choo’s (1996) ko and two are additional dimensions inductively derived from our data. as a check for completeness, the five ddo dimensions were mapped again to all 23 reviewed ddo understandings, covering all mentioned characteristics (see appendix c). australasian journal of information systems fischer et al. 2023, vol 27, selected papers from acis 2022 data-driven organizations 8 4 results: towards a conceptual framework of ddos 4.1 ddo understandings our review results reveal that the ddo understandings found in the existing literature vary considerably in terms of their level of elaboration and richness, ranging from quite simplistic understandings (based on a single ddo characteristic) to more complex understandings (referring to multiple characteristics of a ddo). on the one hand, examples of quite simplistic ddo understandings can be found in berndtsson et al. (2020) and schüritz (2017). both point to decision-making based on data as the single distinguishing characteristic of a ddo, which is captured in descriptions such as: “a data driven company is described as an organization that heavily relies on data to make decisions and take actions” (schüritz, 2017, p. 394). others such as lee (2017) rely on the sole characteristic of “having capabilities to leverage big data” (p. 2) when describing their ddo understanding and an even more high-level understanding is presented by halper & stodder (2017), who tie a ddo to any kind of organizational data usage. on the other hand, more complex ddo understandings refer to at least two unique characteristics of a ddo. for example, three studies in our review sample present the sourcing and processing of data, combined with the goal of using data to gain competitive advantage, as key ddo characteristics. a corresponding understanding is evident in fabijan et al. (2017) who state that “data-driven companies acquire, process, and leverage data in order to create efficiencies, iterate on and develop new products, and navigate the competitive landscape” (p. 1). likewise, gualo et al. (2021) put particular emphasis on the importance of the quality of the obtained data and name better service to the organization’s customer as a ddo characteristic. in a similar vein, körppen et al. (2021) raise data collection, processing and evaluation as a key characteristic and combine it with the act of making decisions based on data to describe their more internally oriented understanding of a ddo. going even further, olszak & zurada (2019) and satar (2021) add data capabilities as a further characteristic to several of the characteristics mentioned, consisting, for example, of data infrastructure, data tools and data experts (satar, 2021). further, studies with even more elaborate ddo understandings add to the aforementioned characteristics the notion of a data-driven culture (grover et al., 2018), requiring a “sustained commitment from leadership and employees alike” (kiron, 2017, p. 1) or “collaboration fostering” (hagen & hess, 2020, p. 1). finally, arguably the most complex ddo understandings are presented in kearny et al. (2016), anderson (2015), and davenport (2001), who make reference to five characteristics, ranging from data management and data governance to data-driven competitive advantages to data-based decision-making and a data-driven culture. illustrative sample quotes highlighting the characteristics mentioned in the various ddo understandings are provided in table 2. 4.2 underlying ddo dimensions although the ddo understandings of the reviewed studies differ notably, they also share several commonalities. in particular, five central ddo dimensions—namely, data sourcing & sensemaking, data capabilities, data-driven culture, data-driven decision-making, and data-driven value creation—emerged from our analysis as shown in table 2. for a comprehensive mapping of the five ddo dimensions onto the 23 ddo understandings identified in our literature review, see appendix c. australasian journal of information systems fischer et al. 2023, vol 27, selected papers from acis 2022 data-driven organizations 9 ddo dimension [# of mentions]: short description sample quotes highlighting matching characteristics data sourcing & sensemaking [15]: acquisition of (external) data and their purposeful interpretation and integration in a data-driven organization, data is collected, processed and evaluated in a targeted manner and used as the basis for decisions. (körppen et al., 2021, p. 454; translated from german) a data-driven organization acquires, processes, and leverages data in a timely fashion (patil, 2011, p. 3) so, what does it mean to be a data-driven enterprise? it means maximizing the value of your data and treating it as an asset differentiated by its completeness, lineage, and quality. […] (hou, 2018, p. 2) data capabilities [15]: abilities of an organization to use its infrastructure, tools, and talent to purposefully manage data data-drivenness [in the context of organizations] is about building tools, abilities, and, most crucially, a culture that acts on data. (anderson, 2015, p. 1) […] as the [data-driven] organization's data and analytics capabilities mature, they can underpin innovative new business models that alter, sometimes radically, power arrangements within the organization. (kiron, 2017, p. 2) [in the context of ddos, the] dynamic capabilities of organizations that should contribute to creating, extending, protecting and maintaining a unique database are to be considered a key point (olszak & zurada, 2019, p. 168) data-driven culture [12]: organization-wide belief and value system that fosters the understanding, management, and exploitation of data a data-driven organization should possess three things: a culture in which everyone buys into the idea of using data to make business decisions; an organizational structure that supports a data-driven culture; technology that supports a data-driven culture and makes data self-service. (thusoo & sarma, 2017, p. 43) data-driven decision-making and creating a data-driven culture are important aspects of a ddo. (svensson & taghavianfar, 2020, p. 4) an organization-wide data-driven culture musts to be established [sic] in order to exploit the full potential of advanced analytics. (berndtsson et al., 2018, p. 2) data-driven decisionmaking [11]: the act of making rational decisions based on data instead of intuition a data driven company is described as an organization that heavily relies on data to make decisions and take actions. (schüritz, 2017, p. 394) in theory, data-driven organizations can apply data-driven decisions for all types of analytics (descriptive, predictive, prescriptive), and all types of decisions (operational, tactical, strategical). (berndtsson et al., 2020, p. 1) data-driven organizations possess a culture of leveraging data-driven decisionmaking rather than depending on the intuitions of their managers in business activities. (gökalp et al., 2021, p. 2) data-driven value creation [15]: an organization’s actions with the ambition to create value through data (e.g., increased efficiency or product improvements) data-driven companies acquire, process, and leverage data in order to create efficiencies, iterate on and develop new products, and navigate the competitive landscape. (fabijan et al., 2017, p. 1) […] data collected from different operational stages can improve an organization’s performance and create new business opportunities. (gökalp et al., 2021, p. 2) data-driven enterprises enjoy advantages over their competitors because of the insights provided by data management and analytics and could, for instance, enhance marketing strategies and planning that involves customer insight. (kearny et al., 2016, p. 4606) table 2. derived ddo dimensions and sample quotes 4.3 the ddo as an update and extension of the knowing organization the five ddo dimensions presented above form the central cornerstones of our proposed conceptual ddo framework. three of these five dimensions (data sourcing & sensemaking, data capabilities, and data-driven decision-making) build on the focal dimensions of choo’s (1996) framework of the ko, whereas two dimensions (data-driven culture and data-driven value australasian journal of information systems fischer et al. 2023, vol 27, selected papers from acis 2022 data-driven organizations 10 creation) constitute novel additions. this implies that the ddo concept can be seen as a update and extension of choo’s (1996) ko concept, to make it fit with the affordances and challenges of today’s data-centric digital era. most notably, with the two added dimensions, the ddo framework emphasizes the importance of a data-driven culture and introduces the dimension of (external) data-driven value creation, which acts as the capstone for an additional inside-out focus, complementing the existing outside-in orientation (see figure 2). the distinct interaction of the individual dimensions becomes apparent when considering the two orientations in detail. outside-in view: our results show that, like the ko, a ddo strives to make sense of its environment, processes the information obtained, and uses them to drive decision-making, thereby demonstrating an outside-in focus. naturally, however, these three dimensions operate considerably more data-centric. in line with choo’s (1996) sensemaking function, the central purpose of the data sourcing & sensemaking dimension is to obtain data, filter them by their relevance, and harmonize and contextualize them. next to inherent internal data, the focus of this dimension lies particularly on data from the external environment, which must first be actively sourced. furthermore, due to today’s (overwhelming) amount of big data, the sourcing & sensemaking dimension additionally entails a data governance dimension to ensure data quality, regulation, and protection. closely intertwined with the sourcing & sensemaking dimension is the dimension of information processing. while choo (1996) describes this function as a human-centric knowledge creating process that deals with tacit knowledge, the concept of the ddo draws on data capabilities (i.e., tools, infrastructure, and experts) to process the information obtained in order to generate innovation, new capabilities, and actionable insights. the outside-in orientation of the conceptual ddo framework concludes with the dimension of data-driven decision-making, which—similarly to choo’s (1996) perspective—involves rational decision-making. contrary to the ko, however, the ddo not only facilitates those decisions based on harmonized personal interpretations of incoming signals, but takes into account factual data, providing an even more objective decision basis. however, to truly benefit from the potentially increased objectivity and implement the ddo dimensions mentioned, about half of the reviewed ddo understandings highlight the need for a data-driven culture as an additional dimension, providing a shared understanding and purpose behind the required efforts. inside-out view: according to our analysis, ddos demonstrate a new, fifth dimension concerned with (external) data-driven value creation, which is not reflected in choo’s (1996) framework of the ko. this outwardly directed dimension presents the capstone of an insideout perspective on ddos, complementing choo’s (1996) outside-in view by focusing on the internal workings of the ddo leading to external value creation. the internal dimensions data-driven culture, data capabilities, and data-driven decision-making interact with the objective of going beyond internal decision-making and thereby create actual value through data. this fifth, externally oriented dimension named data-driven value creation draws on new capabilities and innovations provided by the ddo’s data capabilities as well as goal-directed adaptive behavior caused by the data-driven decision-making to achieve (external) data-driven impact. our conceptual ddo framework, building on the five emerged dimensions, and interweaving both an outside-in as well as an inside-out view, is depicted in figure 2 below. australasian journal of information systems fischer et al. 2023, vol 27, selected papers from acis 2022 data-driven organizations 11 figure 2. conceptual ddo framework as becomes apparent from figure 2, three dimensions—namely, data-driven culture, data capabilities, and data-driven decision-making—are included in both the outside-in and the inside-out view of a ddo. this suggests that these three dimensions form the foundation of a ddo, indicating that they could be considered the core dimensions of such an organization. in the following, we will refer to this subset of dimensions as the inner house of a ddo. on this basis, we define a ddo as an organization (1) with a data-driven culture, enabling and inspiring organizational members on all levels to embrace data as the backbone of their actions and decisions; (2) with access to data capabilities including tools, talent, and infrastructure to gain insights from data; and (3) with organizational members consequently making their decisions based on data-driven insights. further, depending on its specific focus, a ddo may also be characterized by a particular emphasis on (4) systematically acquiring data and purposefully interpreting and integrating these data (outside-in view), and/or on (5) creating data-driven value (inside-out view). 5 empirical illustrations of the conceptual ddo framework to illustrate the general applicability of our conceptual ddo framework and demonstrate how its five ddo dimensions characterize and help differentiate ddos, we draw on three empirical examples. when selecting pharma, automotive and sports as empirical examples, we deliberately chose diverse organizations from different industries with annual revenues of more than one billion euros and a global presence to ensure comparability. to collect the actual case data, we integrated multiple types of sources such as semi-structured interviews with one top-level key informant each, internal documents provided by the key informants, and publicly available information, into a case database and coded the data along the five data capabilities data driven decision-making data driven culture data driven value creation data sourcing & sensemaking inside-out view outside-in view inner house signals from the environment & organization shared purpose & understanding new capabilities & innovations provision of highquality data shared purpose & understanding data-driven insights goal-directed adaptive behavior shared purpose & understanding data-driven impact australasian journal of information systems fischer et al. 2023, vol 27, selected papers from acis 2022 data-driven organizations 12 ddo dimensions of our conceptual framework. the case study companies as well as our case study methodology are detailed in appendix d. below, we present these three cases using our conceptual ddo framework. pharma, one of the world’s largest pharmaceutical companies, has transformed itself into a ddo mainly to improve efficiency and decision-making, with its head of digital transformation stating, “we want to help patients as fast as possible—therefore we need to speed up our time to market through smarter decisions.” the organization demonstrates all three ddo dimensions of the inner house of our conceptual ddo framework to a medium to high degree with an emphasis on data-driven decision-making. for example, in important meetings at pharma, there is a deliberate search for decisions that should be made in a more data-driven way and the required data is subsequently sourced, processed, and presented in the form of a dashboard. regarding the data capabilities that enable this way of working, pharma’s entire data infrastructure is designed to have data flow along the value chain and across divisions, with data analysis tools and necessary upskilling training being provided to all employees. as the reliability of and adherence to laboratory test results can be a matter of life and death in this industry, staff are inherently already aware of the value of data and therefore already cultivate a data-driven culture. this sense of ownership and responsibility of both data owners and users is further fostered by making the origin and use of data transparent throughout the whole organization. centered around this inner house, pharma also demonstrates the two additional ddo dimensions data sourcing & sensemaking, and data-driven value creation to a medium to high degree. because pharma is at its core a clinical research organization, the generation, access, and interpretation of high-quality clinical data (i.e., data sourcing and sensemaking) is an integral part of its operating model. also, building on this operating model, pharma ultimately creates externally-oriented data-driven value through more efficient, more precise, and smarter clinical trials, ultimately leading to clinical solutions being available earlier on the market. automotive, the data-driven car manufacturer we studied, primarily demonstrates an inside-out ddo perspective, building on the inner house of our conceptual ddo framework. to enable this, automotive places great emphasis on data quality and availability, with more than 150 employees being dedicated to improving data quality and developing comprehensive data portals that make available data more transparent and accessible. furthermore, the top management has decided to make all data free of use (i.e., without licenses/chargebacks to the organizational unit that owns the data) to further foster data exploration. however, automotive also demonstrates an inside-out orientation, as it focuses on internally exploring data use cases which lead to innovative, data-enabled products and services (e.g., data-driven applications in cars) or even fundamentally new income streams (e.g., selling traffic data). this focus on data-driven value creation is evidenced by the current 600-800 data use cases whose evaluation, budgeting, and development is managed centrally through a dedicated use case funnel process. to enable this kind of data-driven value creation, automotive’s management equally developed and strengthened all ddo dimensions of the inner house of our conceptual ddo framework. for example, this is reflected in the organization’s data capabilities, where data analytics tools and basic data analytics training are available for tens of thousands of employees. complementary to this mandatory training, all staff are regularly presented showcases (i.e., successful, and particularly impressive or innovative data use cases) that illustrate the value and innovation potential of data to inspire them to work in a more data-driven way, which could be seen as australasian journal of information systems fischer et al. 2023, vol 27, selected papers from acis 2022 data-driven organizations 13 a manifestation of a data-driven culture. according to the vice president data transformation of automotive, this data-driven culture on all levels is a cornerstone of their ddo understanding, as it is the main driver for their employees and managers to make decisions in a data-driven way. sports, a global sports brand, also pursues the goal of becoming a ddo but is currently at a much earlier stage of its transformation compared to pharma and automotive, as it demonstrates all five ddo dimensions to a low to medium level. starting with data sourcing & sensemaking, the organization currently has several non-integrated and inconsistent data pools, with employees navigating the data landscape through manual data requests via email. the lack of a single source of truth also inhibits data-driven decision-making, as conflicting data sets can—intentionally or unintentionally—lead to the adoption of a misleading decision-making basis. regarding data-driven value creation, sports has identified data use cases which are critical to success (e.g., supply chain optimization) and started to develop them accordingly. however, these use cases are built on a non-integrated database and without standardized tools, leading to further incompatibilities and further fragmentation, according to the tech strategy lead. these factors also undermine the establishment of a data-driven culture, as employees who have been convinced to work in a more data-driven way cannot do so due to a lack of data tools and skills as well as a lack of reliable data itself. in summary, and in consideration of the lower-level demonstration of the ddo dimensions of the inner house of our conceptual ddo framework, it can therefore be argued that sports is not (yet) a data-driven organization. a summary of the current state of all three empirical examples along the five ddo dimensions can be found in table 3. when comparing the three empirical examples along the five dimensions of our conceptual ddo framework, notable differences across the firms become apparent. all three organizations demonstrate each ddo dimension to varying degrees, thereby indicating their focus and the progress of their digital transformation towards a ddo. as presented in table 2, both pharma and automotive strongly demonstrate the inner house dimensions of a ddo as well as data sourcing & sensemaking and data-driven value creation. however, when looking at the general orientation of the ddo, a different focus emerges. the top-level management of pharma puts particular emphasis on exploiting data to achieve internal improvements through data-driven decision-making, hence focusing on an outside-in ddo view. the automotive top-level manager, on the other hand, primarily points to data exploration leading to innovative, productand customer-oriented data use cases, thereby focusing on an inside-out ddo view. in contrast, sports—notwithstanding its ambition to eventually become a ddo—currently demonstrates all five ddo dimensions to a much lower degree, indicating a generally earlier stage in the digital transformation process towards a ddo. figure 3 illustrates the degree to which each of the empirical examples currently demonstrates the ddo dimensions, thereby highlighting both the different foci as well as differences in the overall degree to which the dimensions are demonstrated. australasian journal of information systems fischer et al. 2023, vol 27, selected papers from acis 2022 data-driven organizations 14 ddo dimension empirical example pharma automotive sports data sourcing & sensemaking cockpit provides overview of available data and access instructions to all employees very high data quality due to clinical research standards significant investment and manual effort made to integrate and harmonize data pools degree of demonstration: mediumhigh full transparency on available data through data portal data freely available to everyone (no internal licenses/ chargebacks) >150 people dedicated to continuous data quality enhancement degree of demonstration: mediumhigh no consistency between individual data sources highly in-transparent data landscape which is navigated by employees through manual data requests via email degree of demonstration: low data capabilities data infrastructure completely rebuilt to allow integrated data flow data tools are provided as standardized out-of-the-box toolkit both data scientists and clinical staff are upskilled to integrated clinical data scientist role degree of demonstration: mediumhigh cloud-based, scalable data infrastructure data tools freely available for all relevant employees basic training on data analytics and use case identification for >40.000 employees degree of demonstration: high data infrastructure currently consists of incompatible data pools no established toolset, analyses are mainly run ad-hoc organization is not able to attract and retain talent mainly due to salary and leadership style degree of demonstration: low-medium data-driven culture data usage is made more transparent to foster sense of ownership and accountability business unit-overarching workshops where dashboards are jointly built and presented to inspire workforce degree of demonstration: mediumhigh dedicated show case space where lighthouse use cases are presented to >40.000 employees to inspire them focus on mitigating fears of more (performance) transparency degree of demonstration: mediumhigh frontrunners have been identified and placed in each business unit to act as change influencers employees try to act datadriven but are hindered by lack of data as well as capabilities degree of demonstration: low-medium data-driven decisionmaking deliberate search for (recurring) decisions in meetings that should be made in a more data-driven way data dashboards are developed for all important decisions degree of demonstration: high importance of data as objective decision basis is conveyed and fostered through data-driven culture degree of demonstration: medium-high no objective decision basis due to multiple, inconsistent, or even conflicting data sources data can easily be manipulated to underpin favorized message degree of demonstration: low-medium data-driven value creation focus on improving existing processes (e.g., decisionmaking) through data use case identification is mainly decision-driven deliberate development of cross-divisional use cases to foster exchange of data and insights degree of demonstration: medium focus on innovative data use cases (e.g., new products & services) dedicated innovation funnel for data use case ideas with central intake, approval, and monitoring currently 600-800 registered data use cases degree of demonstration: high monolithic implementation of single use cases for must-win battles degree of demonstration: low-medium table 3. overview of empirical examples (along the conceptual ddo framework) australasian journal of information systems fischer et al. 2023, vol 27, selected papers from acis 2022 data-driven organizations 15 figure 3. comparison of three organizations along five ddo dimensions when comparing the three empirical examples along the five dimensions of our conceptual ddo framework, notable differences across the firms become apparent. all three organizations demonstrate each ddo dimension to varying degrees, thereby indicating their focus and the progress of their digital transformation towards a ddo. as presented in table 2, both pharma and automotive strongly demonstrate the inner house dimensions of a ddo as well as data sourcing & sensemaking and data-driven value creation. however, when looking at the general orientation of the ddo, a different focus emerges. the top-level management of pharma puts particular emphasis on exploiting data to achieve internal improvements through data-driven decision-making, hence focusing on an outside-in ddo view. the automotive top-level manager, on the other hand, primarily points to data exploration leading to innovative, productand customer-oriented data use cases, thereby focusing on an inside-out ddo view. in contrast, sports—notwithstanding its ambition to eventually become a ddo—currently demonstrates all five ddo dimensions to a much lower degree, indicating a generally earlier stage in the digital transformation process towards a ddo. figure 3 illustrates the degree to which each of the empirical examples currently demonstrates the ddo dimensions, thereby highlighting both the different foci as well as differences in the overall degree to which the dimensions are demonstrated. 6 discussion the ddo concept has gained increasing relevance within is research during the recent era of digital transformation, resulting in a variety of ddo definitions that differ considerably in terms of their underlying ddo characteristics. therefore, the study at hand aims to (1) data capabilities data-driven value creation data-driven decision making data-driven culture data sourcing & sensemaking pharma automotive sports australasian journal of information systems fischer et al. 2023, vol 27, selected papers from acis 2022 data-driven organizations 16 synthesize the various understandings of ddos in the existing literature, and (2) derive a conceptual ddo framework that integrates the identified understandings, thereby updating and extending choo’s (1996) concept of the ko to modern organizations in the digital age. by doing so, this study makes important theoretical contributions to the is literature and offers practical implications. 6.1 theoretical and practical contributions the theoretical contribution of our study consists of one primary contribution accompanied by three complementary contributions that expand upon it. the main theoretical contribution of the study at hand is the synthesis of existing ddo knowledge through the identification of five central ddo dimensions (data sourcing & sensemaking, data capabilities, data-driven culture, data-driven decision-making, and data-driven value creation) that emerged from our literature review and the integration of those dimensions in our conceptual ddo framework. relatedly, our ddo conceptualization shows that a ddo may draw on an outside-in view— similar to its predecessor, the knowing organization—but it may also draw on an inside-out view or even combine the two views, thereby updating and extending choo’s (1996) knowing organization. as three of the five identified ddo dimensions (data capabilities, datadriven culture, and data-driven decision-making) are shared by all combinations, we propose these dimensions to be core dimensions, and refer to them as the inner house of a ddo. consequentially, our conceptual ddo framework helps identify ddos, as demonstrated by our illustrative examples. while automotive and pharma demonstrate all three inner house dimensions to a relatively high degree and can therefore be considered ddos, sports is arguably not a ddo (yet), as it exhibits only lower levels of those three dimensions. as a second related contribution, the empirical applications of our conceptual ddo framework suggest that ddos may have different foci. on the one hand, there are organizations like pharma that focus on exploiting data to enhance their decision-making processes and improve their internal processes using (external) data, thereby drawing on the outside-in view of the ddo. on the other hand, organizations such as automotive, which take an inside-out view of ddo, focus on exploring data with their internal data capabilities, data-driven culture and data-driven decision-making to create externally-oriented, datadriven value. as a third related theoretical contribution, our conceptual ddo framework helps to clarify the differentiation between a ddo and the related concept “data-driven business model” (ddbm). both phenomena exhibit data-driven value creation as a central dimension and it can be argued that further ddo dimensions contribute to key resources (data sourcing & sensemaking, data capabilities, and data-driven culture) and activities (data-driven decisionmaking) of a ddbm. however, we also argue that several types of ddbms do not necessarily exhibit the other four dimensions next to data-driven value creation, nor do they require an underlying ko. wiener et al. (2020) distinguish between organizations that (1) use data to inform strategic decision-making, improve internal operations, or enrich/develop products, services, and customer experience, (2) sell collected or aggregated data to other data users, or (3) facilitate data, for example, through infrastructure solutions or consulting services. while the first type is clearly in the spirit of the reviewed ddo understandings and therefore constitutes a ddo, we argue that organizations do not necessarily have to be data-driven australasian journal of information systems fischer et al. 2023, vol 27, selected papers from acis 2022 data-driven organizations 17 themselves to sell data or facilitate their use. thus, ddos and organizations with a ddbm should not be treated as synonymous. finally, as a practical contribution, our ddo framework can be used as a conceptual tool that may help practitioners concerned with ddos in several ways. first, it can serve as a target state for digital transformations. by applying our framework, practitioners can develop a roadmap for transforming their organization into a ddo or identify gaps in their current organization. also, by mapping organizations onto the five core dimensions of our framework, practitioners can compare and contrast different organizations. this comparative analysis can provide valuable insights into organization-specific foci—as demonstrated with automotive and pharma—as well as general industry-specific best practices, and opportunities for cross-industry learning. 6.2 limitations and further research directions regarding limitations and future research directions, we acknowledge that not all publications in our review sample had an exclusive focus on ddos. in particular, for publications where ddos are not the focus of the study, this could have an influence on the level of elaboration in the respective descriptions of ddo understandings. however, since half of the four shortest ddo understandings (based on only one characteristic) in our review sample were extracted from publications explicitly focused on ddos, we consider this potential bias to be acceptable. secondly, the study at hand presents only a simplified scale to measure the degree to which organizations demonstrate the respective ddo dimensions. to demonstrate the applicability of our framework for identifying and comparing illustrative examples, we believe this pragmatic approach to be appropriate. however, future research could expand on these dimensions with more refined scales. this operationalization could also open up the opportunity for configurational studies (e.g., fsqca; c.f. ragin, 2008) to examine successful configurations of ddo dimensions for different foci (e.g., data exploitation vs. data exploration) and to test our proposition of the inner house dimensions being core dimensions for a ddo. finally, our conceptual framework could also help to study the nature of digital transformation processes of organizations towards a ddo, as our five ddo dimensions could guide and inform research regarding the different aspects of such a transformation. in this context, it would be particularly interesting to explore the order in which the dimensions are addressed in the context of a digital transformation, as addressing all of them simultaneously does not seem feasible due to the high level of complexity. 7 conclusion in conclusion, our study provides an integrated, conceptual framework of the ddo, a phenomenon that is gaining increasing attention as organizations strive to become more data-driven. by synthesizing the existing, fragmented body of research, we identified five ddo dimensions (data sourcing & sensemaking, data capabilities, data-driven culture, data-driven decision-making, and data-driven value creation) and integrated them into a conceptual ddo framework. the proposed framework suggests that a ddo can draw on both an outside-in view or an inside-out view or both, thereby setting it apart from its predecessor, the knowing organization, which draws exclusively on an outside-in view. using the ddo framework to analyze empirical examples, the study at hand demonstrates both the framework’s general applicability and its practical relevance. for example, our framework can help practitioners identify and compare ddos, as well as support them describe the current and target states australasian journal of information systems fischer et al. 2023, vol 27, selected papers from acis 2022 data-driven organizations 18 of their organization and thus in derive a roadmap for its digital transformation toward a ddo. against this backdrop, we hope that our study results will serve as an analytical tool for managers, as well as inform and inspire future research on the exciting topic of ddos. references anderson, c. (2015). creating a data-driven organization: practical advice from the trenches. sebastopol, ca, usa: o’reilly. berndtsson, m., forsberg, d., stein, d., & svahn, t. (2018). becoming a data-driven organization. ecis 2018 proceedings, portsmouth, uk, 43. https://aisel.aisnet.org/ecis2018_rip/43 berndtsson, m., lennerholt, c., svahn, t., & larsson, p. (2020). 13 organizations’ attempts to become data-driven. international journal of business intelligence research, 11(1), 1–21. https://doi.org/10.4018/ijbir.2020010101 bharadwaj, a., el sawy, o. a., pavlou, p. a., & venkatraman, n. (2013). digital business strategy: toward a next generation of insights. mis quarterly, 37(2), 471–482. https://doi.org/10.25300/misq/2013/37:2.3 boell, s., & cecez-kecmanovic, d. (2014). a hermeneutic approach for conducting literature reviews and literature searches. communications of the association for information systems, 34(12). https://doi.org/10.17705/1cais.03412 boell, s., & wang, b. (2019). www.litbaskets.io, an it artifact supporting exploratory literature searches for information systems research. acis 2019 proceedings, perth, wa, australia, 71. https://aisel.aisnet.org/acis2019/71/ boulding, k. (1956). general systems theory. management science, 2(3), 197–208. https://doi.org/10.1287/mnsc.2.3.197 brown, c. v., & magill, s. l. (1994). alignment of the is functions with the enterprise: toward a model of antecedents. mis quarterly, 18(4), 371. https://doi.org/10.2307/249521 choo, c. w. (1996). the knowing organization: how organizations use information to construct meaning, create knowledge and make decisions. international journal of information management, 16(5), 329–340. https://doi.org/10/bbw629 choo, c. w. (2001). the knowing organization as learning organization. education + training, 43(4/5), 197–205. https://doi.org/10.1108/eum0000000005482 constantiou, i. d., & kallinikos, j. (2015). new games, new rules: big data and the changing context of strategy. journal of information technology, 30(1), 44–57. https://doi.org/10.1057/jit.2014.17 davenport, t. h. (2006). competing on analytics. harvard business review, 84(1), 98–107. davenport, t. h., & bean, r. (2018, february). big companies are embracing analytics, but most still don’t have a data-driven culture. harvard business review. https://hbr.org/2018/02/big-companies-are-embracing-analytics-but-most-still-donthave-a-data-driven-culture australasian journal of information systems fischer et al. 2023, vol 27, selected papers from acis 2022 data-driven organizations 19 davenport, t. h., harris, j. g., de long, d. w., & jacobson, a. l. (2001). data to knowledge to results: building an analytic capability. california management review, 43(2), 117– 138. https://doi.org/10.2307/41166078 fabijan, a., dmitriev, p., olsson, h. h., & bosch, j. (2017). the evolution of continuous experimentation in software product development. international conference on software engineering 2017 proceedings, buenos aires, 770–780. https://doi.org/10.1109/icse.2017.76 faraj, s., & leonardi, p. m. (2022). strategic organization in the digital age: rethinking the concept of technology. strategic organization, argentina, 20(4), 771–785. https://doi.org/10.1177/14761270221130253 fischer, h., wiener, m., & strahringer, s. (2023). embarking on the digital transformation journey toward a data-driven organization: empirical insights into transformation starting points. ecis 2023 proceedings, kristiansand, norway, 298. https://aisel.aisnet.org/ecis2023_rp/298/ gartner. (2021). embarking on the digital transformation journey toward a data-driven organization: empirical insights into transformation starting points. stamford, conn, usa: gartner. https://www.gartner.com/en/newsroom/press-releases/2021-05-05gartner-finds-72-percent-of-data-and-analytics-leaders-are-leading-or-heavilyinvolved-in-digital-transformation-initiatives gökalp, m. o., gökalp, e., kayabay, k., koçyiğit, a., & eren, p. e. (2021). the development of the data science capability maturity model: a survey-based research. online information review, 46(3), 547–567. https://doi.org/10.1108/oir-10-2020-0469 grover, v., chiang, r. h. l., liang, t.-p., & zhang, d. (2018). creating strategic business value from big data analytics: a research framework. journal of management information systems, 35(2), 388–423. https://doi.org/10.1080/07421222.2018.1451951 gualo, f., rodriguez, m., verdugo, j., caballero, i., & piattini, m. (2021). data quality certification using iso/iec 25012: industrial experiences. journal of systems and software, 176. https://doi.org/10.1016/j.jss.2021.110938 gulati, r., lawrence, p. r., & puranam, p. (2005). adaptation in vertical relationships: beyond incentive conflict. strategic management journal, 26(5), 415–440. https://doi.org/10.1002/smj.458 hagen, j. a., & hess, t. (2020). linking big data and business: design parameters of datadriven organizations. amcis 2020 proceedings, 5. https://aisel.aisnet.org/amcis2020/data_science_analytics_for_decision_support/data_ science_analytics_for_decision_support/5 hall, r. h. (1977). organizations: structure and process. englewood cliffs: prentice hall. halper, f., & stodder, d. (2017). what it takes to be data-driven: technologies and best practices for becoming a smarter organization. tdwi. https://tdwi.org/research/2017/12/bi-allppm-all-best-practices-report-what-it-takes-to-be-data-driven hartmann, p. m., zaki, m., feldmann, n., & neely, a. (2016). capturing value from big data – a taxonomy of data-driven business models used by start-up firms. international journal of operations & production management, 36(10), 1382–1406. https://doi.org/10.1108/ijopm-02-2014-0098 australasian journal of information systems fischer et al. 2023, vol 27, selected papers from acis 2022 data-driven organizations 20 hicks, j. o. (1993). management information systems: a user perspective. minneapolis, mn, usa: west publishing. hinton, m. (2006). introducing information management. london, uk: taylor & francis. https://doi.org/10.4324/9780080458397 hou, f. (2018). becoming a data-driven enterprise. accenture labs. https://www.accenture.com/content/dam/accenture/final/a-com-migration/r3additional-pages-1/pdf/pdf-83/accenture-becoming-data-driven-enterprise-dataindustrialization.pdf hupperz, m., gür, i., möller, f., & otto, b. (2021). what is a data-driven organization? amcis 2021 proceedings, 6. https://aisel.aisnet.org/amcis2021/strategic_is/strategic_is/6/ johnson, b. l., & kruse, s. d. (2019). educational leadership, organizational learning, and the ideas of karl weick: perspectives on theory and practice (1st ed.). milton park, uk: routledge. https://doi.org/10.4324/9781315114095-1 katz, d., & kahn, r. l. (1978). the social psychology of organizations. new york: wiley. kearny, c., gerber, a., & van der merwe, a. (2016). data-driven enterprise architecture and the togaf adm phases. international conference on systems, man, and cybernetics 2016 proceedings, budapest, hungary, 4603–4608. https://doi.org/10.1109/smc.2016.7844957 kellogg, k. c., valentine, m. a., & christin, a. (2020). algorithms at work: the new contested terrain of control. academy of management annals, 14(1), 366–410. https://doi.org/10.5465/annals.2018.0174 kiron, d. (2017). lessons from becoming a data-driven organization. mit sloan management review, 58(2). https://sloanreview.mit.edu/case-study/lessons-from-becoming-a-datadriven-organization/ körppen, t., ullrich, a., & bertheau, c. (2021). durchblick statt bauchgefühl – transformation zur data-driven organization. (insight instead of gut feeling – transformation to a data-driven organization). wirtschaftsinformatik & management, 13(6), 452–459. https://doi.org/10.1365/s35764-021-00370-7 lawrence, p. r., & lorsch, j. w. (1967). differentiation and integration in complex organizations. administrative science quarterly, 12(1). https://doi.org/10.2307/2391211 lee, i. (2017). big data: dimensions, evolution, impacts, and challenges. business horizons, 60(3), 293–303. https://doi.org/10.1016/j.bushor.2017.01.004 malhotra, n. k., peterson, m., & kleiser, s. b. (1999). marketing research: a state-of-the-art review and directions for the twenty-first century. journal of the academy of marketing science, 27(2), 160–183. https://doi.org/10.1177/0092070399272004 march, j. g., & simon, h. a. (1993). organizations. oxford, uk: blackwell business/blackwell publisher. mcafee, a., & brynjolfsson, e. (2012). big data: the management revolution. harvard business review. https://hbr.org/2012/10/big-data-the-management-revolution australasian journal of information systems fischer et al. 2023, vol 27, selected papers from acis 2022 data-driven organizations 21 mckinsey. (2013). game changers: five opportunities for us growth and renewal. https://www.mckinsey.com/~/media/mckinsey/featured%20insights/americas/us%2 0game%20changers/mgi_game_changers_us_growth_and_renewal_full_report morakanyane, r., grace, a., & o’reilly, p. (2017). conceptualizing digital transformation in business organizations: a systematic review of literature. digital transformation – from connecting things to transforming our lives, 427–443. https://doi.org/10.18690/978961-286-043-1.30 olszak, c. m., & zurada, j. (2019). big data-driven value creation for organizations. hicss 2019 proceedings, grand wailea, hi, usa, 10. https://hdl.handle.net/10125/59457 paré, g., trudel, m.-c., jaana, m., & kitsiou, s. (2015). synthesizing information systems knowledge: a typology of literature reviews. information & management, 52(2), 183–199. https://doi.org/10.1016/j.im.2014.08.008 parra, x. (2022). chronological evolution of the information-driven decision-making process (1950–2020). journal of the knowledge economy, 32. https://doi.org/10.1007/s13132-022-00917-y patil, d. (2011). building data science teams. sebastopol, ca, usa: o’reilly media. puranam, p., alexy, o., & reitzig, m. (2014). what’s “new” about new forms of organizing? academy of management review, 39(2), 162–180. https://doi.org/10.5465/amr.2011.0436 ragin, c. c. (2008). redesigning social inquiry. chicago: university of chicago press. rubin, e., & rubin, a. (2013). the impact of business intelligence systems on stock return volatility. information & management, 50(2–3), 67–75. https://doi.org/10.1016/j.im.2013.01.002 satar, n. s. b. m. (2021). data-driven organization and covid-19 pandemic: a systematic review. journal of strategic digital transformation in society, 1(1), 18–32. schüritz, r. (2017). how to cultivate analytics capabilites within an organization – design and types of analytics competency centers. ecis 2017 proceedings, guimarães, portugal, 26. http://aisel.aisnet.org/ecis2017_rp/26 schwer, k., & hitz, c. (2018). designing organizational structure in the age of digitization. journal of eastern european and central asian research, 5(1). https://doi.org/10.15549/jeecar.v5i1.213 simon, h. a. (1976). administrative behavior (4th ed.). new york, ny, usa: free press (simon & schuster). sivarajah, u., kamal, m. m., irani, z., & weerakkody, v. (2017). critical analysis of big data challenges and analytical methods. journal of business research, 70, 263–286. https://doi.org/10.1016/j.jbusres.2016.08.001 svensson, r. b., feldt, r., & torkar, r. (2019). the unfulfilled potential of data-driven decision making in agile software development, in p. kruchten, s. fraser, f. coallier (eds.), agile processes in software engineering and extreme programming. xp 2019 proceedings, montreal, canada (vol. 355, pp. 69–85). springer. https://doi.org/10.1007/978-3-030-19034-7_5 australasian journal of information systems fischer et al. 2023, vol 27, selected papers from acis 2022 data-driven organizations 22 svensson, r. b., & taghavianfar, m. (2020). toward becoming a data-driven organization: challenges and benefits, in f. dalpiaz, j. zdravkovic, & p. loucopoulos (eds.), research challenges in information science (vol. 385, pp. 3–19). springer international publishing. https://doi.org/10.1007/978-3-030-50316-1_1 thusoo, a., & sarma, j. s. (2017). creating a data-driven enterprise with dataops. insights from facebook, uber, linkedin, twitter, and ebay. sebastopol, ca, usa: o’reilly. tuomi, i. (1999). data is more than knowledge: implications of the reversed knowledge hierarchy for knowledge management and organizational memory. journal of management information systems, 16(3), 103–117. https://doi.org/10.1080/07421222.1999.11518258 webster, j., & watson, r. t. (2002). analyzing the past to prepare for the future: writing a literature review. mis quarterly, 26(2), 13–23. wef. (2021). data-driven economies: foundations for our common future. https://www.weforum.org/publications/data-driven-economies-foundations-for-ourcommon-future/ weick, k. e. (1969). the social psychology of organizing. new york, ny, usa: mcgraw hill. wessel, l., baiyere, a., ologeanu-taddei, r., cha, j., & blegind jensen, t. (2021). unpacking the difference between digital transformation and it-enabled organizational transformation. journal of the association for information systems, 22(1), 102–129. https://doi.org/10.17705/1jais.00655 west, r., & turner, l. h. (2014). introducing communication theory: analysis and application (6th ed.). new york, ny, usa: mcgraw hill. wiener, m., saunders, c., & marabelli, m. (2020). big-data business models: a critical literature review and multiperspective research framework. journal of information technology, 35(1), 66–91. wittgenstein, l., & anscombe, g. e. m. (1989). philosophical investigations: = philosophische untersuchungen (3nd ed., repr). cambridge, uk: blackwell. yoo, y., boland, r. j., lyytinen, k., & majchrzak, a. (2012). organizing for innovation in the digitized world. organization science, 23(5), 1398–1408. apendices appendix a. is basket journal list journal name basket size 2xs xs s m european journal of information systems x x x x information systems journal x x x x information systems research x x x x journal of the association for information systems x x x x journal of information technology x x x x journal of management information systems x x x x journal of strategic information systems x x x x australasian journal of information systems fischer et al. 2023, vol 27, selected papers from acis 2022 data-driven organizations 23 mis quarterly x x x x decision support systems x x x information and management x x x information systems frontiers x x x information and organization x x x communications of the association for information systems x x x data base for advances in information systems x x x expert systems with applications x x x information society x x x communications of the acm x x decision sciences journal x x ieee transactions on knowledge and data engineering x x information processing and management x x information technology and people x x international journal of human-computer studies x x international journal of information management x x journal of information systems x x journal of organizational computing and electronic commerce x x journal of systems and software x x journal of the association for information science and technology x x knowledge-based systems x x organization science x x acm transactions on information systems x australasian journal of information systems x business and information systems engineering x computer supported cooperative work x computers in human behavior x european journal of operational research x information systems and e-business management x information systems management x informs journal on computing x international journal of business information systems x international journal of enterprise information systems x international journal of information security x international journal of information systems in the service sector x journal of computer information systems x journal of database management x journal of enterprise information management x journal of global information management x journal of global information technology management x management science x omega international journal of management science x online information review x scandinavian journal of information systems x table a1. is basket overview (cf. boell & wang, 2019) australasian journal of information systems fischer et al. 2023, vol 27, selected papers from acis 2022 data-driven organizations 24 appendix b. review sample description in total, our review sample consists of 23 studies, including six journal articles, seven conference papers, one book section, and nine practitioner publications of various types (e.g., books, magazine articles, and white papers), representing a wide array of disciplines and perspectives. while 15 studies are published in is outlets that were the starting point of our search (e.g., journal of management information systems, amcis proceedings), our review set also includes publications from adjacent research fields such as computer science (4) and management (4) that were identified through the backward and forward searches. this distribution is in line with the perspective offered by hagen and hess (2020), who refer to ddos as socio-technical systems, which by their very nature lie at the boundary between computer science and management and thus within the focus of is research. in terms of sectors, our review set (and especially the case studies presented) reveals a wide range of application areas (e.g., retail, manufacturing, healthcare, finance, construction, technology) and predominantly (57%) a focus on the required actions and challenges of becoming a datadriven organization, again confirming the current emergence of ddos in practice as described in the introduction. another indication of the topicality of ddos can be seen in the chronological progression of the reviewed publications. as shown in figure b1, a noticeable increase in publications can be observed from 2016 onwards, with academic publications slightly trailing those from practitioners. moreover, in terms of the complexity and richness of the extracted ddo understandings, a wide span can be observed in the length of the descriptions alone, ranging from short onesentence definitions (e.g., schüritz, 2017) to extensive concept maps (e.g., hupperz et al., 2021) and detailed checklists (davenport, 2006). figure b1. distribution of publications in the sample 0 2 4 6 be fo re 20 13 20 13 20 14 20 15 20 16 20 17 20 18 20 19 20 20 20 21 academic literature practitioner literature australasian journal of information systems fischer et al. 2023, vol 27, selected papers from acis 2022 data-driven organizations 25 appendix c. review sample and identified ddo dimensions source literature type explicit focus on ddo identified ddo dimensions d at a so ur ci ng & se ns em ak in g d at a ca pa bi lit ie s d at adr iv en c ul tu re d at adr iv en de ci si on -m ak in g d at adr iv en va lu e cr ea tio n anderson (2015) practitioner yes x x x x x berndtsson et al. (2018) academic yes x x x x berndtsson et al. (2020) academic yes x davenport et al. (2001) academic no x x x x x davenport (2006) practitioner no x x x fabijan et al. (2017) academic no x x gökalp et al. (2021) academic no x x x gualo et al. (2021) academic no x x grover et al. (2018) academic yes x x x x hagen & hess (2020) academic yes x x x x halper & stodder (2017) practitioner yes x hou (2018) practitioner yes x x x x hupperz et al. (2021) academic yes x x x kearny et al. (2016) academic no x x x x x kiron (2017) practitioner no x x x x körppen et al. (2021) practitioner yes x x lee (2017) practitioner no x olszak & zurada (2019) academic no x x x patil (2011) practitioner no x x satar (2021) academic yes x x x svensson & taghavianfar (2020) academic yes x x x thusoo & sarma (2017) practitioner no x x x total 15 15 12 11 15 table c1. review sample and mapped ddo dimensions appendix d. illustrative example research approach to empirically illustrate our conceptual ddo framework, we deliberately selected three organizations of different industries and sizes that are currently undergoing digital transformations toward a ddo to demonstrate the applicability of our framework across organizational types. at the same time, we chose a lower boundary of more than ten billion euros in annual revenue and a global presence to ensure comparability. table d1 provides an overview of the industry, number of employees and revenue of the three selected organizations, as well as the role of the key informant interviewed. table d1. overview of analyzed organizations illustrative example staff 2021 [thousand] revenue 2021 [billion eur] key informant role automotive >100 >100 vice president data transformation pharma >100 10-50 head of transformation sports 50–100 10-50 tech strategy lead australasian journal of information systems fischer et al. 2023, vol 27, selected papers from acis 2022 data-driven organizations 26 to collect our data, we combined multiple sources. first, we conducted extensive online research on each company to gather contextual information such as external reports and evaluations, relevant press coverage and financial statements (including e.g., revenue). we then identified top-level managers who are responsible for the planning and implementation of the digital transformation of each company, contacted them via linkedin and conducted a 30-to-60-minute video call with one manager per organization. this interview was structured by a questionnaire focusing on the general ddo understanding, the digital transformation motivation and expected outcome, as well as key milestones and challenges in becoming a ddo. the entire questionnaire is available on request. to analyze the data obtained, we performed a deductive concept coding of all the data gathered, using our five ddo dimensions as categories and transferred our findings into a tabular database organized by ddo dimension and organization. building on this database, we performed a thorough within-case analysis for each empirical example, resulting in detailed, descriptive write-ups that we used to empirically illustrate our conceptual ddo framework. copyright: © 2023 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v27i0.4425 http://creativecommons.org/licenses/by-nc/3.0/au/ 1  introduction 2 conceptual foundations 2.1 the concept of an organization 2.2 the concepts of data and data-drivenness 2.3 guiding framework: the knowing organization 3 research methodology 3.1 literature search 3.2 literature analysis 4 results: towards a conceptual framework of ddos 4.1 ddo understandings 4.2 underlying ddo dimensions 4.3 the ddo as an update and extension of the knowing organization 5 empirical illustrations of the conceptual ddo framework 6 discussion 6.1 theoretical and practical contributions 6.2 limitations and further research directions 7 conclusion appendix a. is basket journal list appendix b. review sample description appendix c. review sample and identified ddo dimensions appendix d. illustrative example research approach paper title (paper title style) australasian journal of information systems rinta-kahila & penttinen 2021, vol 25, research article four flavours of customers 1 four flavours of customers: a dual-system perspective on self-service technology use tapani rinta-kahila australian institute for business and economics, the university of queensland, brisbane, australia t.rintakahila@uq.edu.au esko penttinen aalto university school of business, helsinki, finland abstract self-service technologies (ssts) increasingly permeate retail spaces. to make their sst investments worthwhile, retailers need to turn enough customers into sst users. previous research has uncovered the significance of habitual behaviour stemming from prior experience and situational factors from the environment on sst use. however, consumers are likely to vary regarding the extent they are driven by either habit or situational factors, suggesting that different types of consumers might exist in this regard. in this paper, we probe these consumer types in a real-life choice situation by studying the choice of selecting a checkout option (either staffed or self-checkout). we conduct a field study employing mixed qualitative methods by observing and interviewing customers checking out from retail stores. we discover four distinct customer types regarding the extent of reflexive (automatic) and reflective (deliberate) processing they use in their checkout selection: habitual traditional checkout users, habitual sco users, situational users, and drifting users. we discuss the implications of our findings by linking the cognitive processing styles to the different stages of technology acceptance. our main contribution lies in the development of a typology of consumers based on their selection between sst and human-delivered service. keywords: self-service technologies, consumer typology, dual-system theories, grocery industry, field study. 1 introduction new technologies are increasingly reshaping consumers’ retail experience (atapattu, sedera, & ravichandran, 2014; roy, balaji, sadeque, nguyen, & melewar, 2017). recent advancements in areas such as digital payment options (e.g., contactless and mobile payments) and automatic check-out channels (e.g., self-checkouts, automatic parcel stations, and even unstaffed grocery stores) are nudging consumers to change their incumbent routines (chen, white, & hsieh, 2020; gelderman, ghijsen, & van diemen, 2011; vannucci & pantano, 2019), ultimately putting consumer choice calculus to the test. rolling out these technologies requires investments by retailers, which, in turn, requires adoption decisions by consumers to make these investments worthwhile for retailers (collier, sherrell, babakus, & horky, 2014; evanschitzky, iyer, pillai, kenning, & schütte, 2015; rinta-kahila, penttinen, kumar, & janakiraman, 2021). the question of what prompts a consumer to choose technology-delivered service becomes especially apparent in settings where self-service technologies (ssts) are offered as an alternative to traditional human-delivered service mode. as ever more sophisticated smart technologies such as robotic shopping companions (bertacchini, bilotta, & pantano, 2017) are being australasian journal of information systems rinta-kahila & penttinen 2021, vol 25, research article four flavours of customers 2 developed into physical retail settings, understanding the abovementioned dynamics is increasingly important. situational factors such as waiting times and time pressure have been found to significantly influence sst use intentions and behaviour (chen, yu, yang, & wei, 2018; collier, moore, horky, & moore, 2015; penttinen, rinta-kahila, ronkko, & saarinen, 2014; wang, harris, & patterson, 2012). then again, incumbent habits have been established as powerful drivers of sst use too (wang, harris, & patterson, 2013, 2017). however, little is known about how situational factors and habits interact and whether their effects are different for different types of consumers. blut et al. (2016) note that the various predictors of sst use should be examined jointly to gain a comprehensive understanding of user behaviour. this invites us to explore the extent to which the interplay of situational factors and habit shapes consumers’ choice behaviour and to consider that different types of consumers might be affected in different ways. previous research has established that consumers can be classified into different types with regard, for instance, the temporality of technology adoption (e.g., innovators and laggards in rogers (2003)), the extent of technology use (e.g., intense vs. specialized in shih & venkatesh (2004)), ways of using social media (e.g., sporadics vs. actives in brandtzæg & heim (2011)), and level of experientiality in consumption (e.g., hedonic, holistic, and utilitarian consumers in zarantonello & schmitt (2010)). distinguishing between different types helps service providers to create circumstances that nurture the consumption or technology adoption of each user group. however, it is not clear whether distinct behavioural types exist with regard to consumer’s choice between technology and human-delivered service. this leads us to ask: 1. can different types of consumers be distinguished based on their choice between sst and human-delivered service? and 2. if yes, how do these types differ in their decision-making regarding sst use? we investigate the questions in the grocery retailing context (kurnia, belts, & johnston, 2002), where customers operate in a highly dynamic environment that burgeons with external cues and triggers. existing work has probed the topic of sst adoption and use mainly by inquiring consumers’ abstract perceptions of the technology of interest (lien, hsu, shang, & wang, 2019), and studies that account for situational factors tend to survey consumers about imaginary or hypothetical situations (collier et al., 2015; simon & usunier, 2007). yet, we do not often even realise the effect of external and environmental stimuli on our behaviour (ortiz de guinea & markus, 2009), and retail settings tend to be dynamic and even chaotic at times. therefore, surveys that measure consumers’ technology perceptions need to be complemented with approaches that probe how experienced environmental stimuli shapes consumer behaviour. to investigate our questions, we draw on prior literature’s conceptualization of two interconnected cognitive systems that feed stimulus to consumer choice behaviour: a reflexive automatic mode and a reflective controlled one. such frameworks, generally known as dualsystem theories, have significant potential to deepen our understanding of consumer decision making (bond, bettman, & luce, 2009; dhar & gorlin, 2013; gawronski, 2013; gorlin & dhar, 2013). we thus began by using purposive sampling (polkinghorne, 2005) to select a clear consumer choice situation where both reflexive and reflective modes are imaginable, and where a researcher may observe the actions taken by the consumer and make further inquiries immediately after the choice. as a result, we chose to focus on consumers selecting either a traditional staffed cashier or an unstaffed self-checkout (sco) in grocery stores. australasian journal of information systems rinta-kahila & penttinen 2021, vol 25, research article four flavours of customers 3 we conducted an empirical field study by observing and interviewing customers frequenting three grocery stores that all offer both traditional service and an sco option. by probing customers’ cognitive processing in the choice situation via the above-mentioned methods, we found that four types of consumer behaviour can be distinguished: habitual sco users, habitual traditional checkout users, situational users, and drifting users. this paper is organised as follows: section 2 sets the scene by articulating our current understanding regarding the selection of scos over traditional checkouts. section 3 establishes the theoretical background of this study by discussing dual-system theories. section 4 describes the empirical study in terms of data collection and data analysis. sections 5 and 6 are dedicated to the findings and their implications, respectively. 2 setting the scene: the selection of self-checkout over traditional checkout self-service technologies (ssts) such as self-checkouts (scos) empower consumers by providing them with increasing choice freedom (broniarczyk & griffin, 2014). by implementing scos, retailers hope to achieve increased sales through providing more attractive service environments, as well as to reap cost savings through more effective management of staff and customer waiting times (weijters, rangarajan, falk, & schillewaert, 2007). previous research on sst adoption and use indicates that customers’ perceptions of sst (amorim, lago, moscoso, & prieto, 2016; beatson, lee, & coote, 2007; blut, wang, & schoefer, 2016; galletta, durcikova, everard, & jones, 2005; lien et al., 2019) play a major role in explaining sst use behaviour. relevant perceptions typically relate to sst’s usefulness, convenience, and ease of use, among other factors. these perceptions, in turn, are shaped by individual demographic factors like age and gender (dean, 2008; venkatesh & morris, 2000), as well as psychographic characteristics like inherent thinking styles (simon & usunier, 2007), technology anxiety (chen et al., 2020), and personal values (lee & lyu, 2016). however, the high dynamism of the retail checkout environment makes the decision to use or not to use sst a highly contextual and situational phenomenon. this suggests that inherent thinking styles or abstract perceptions of technology may not sufficiently explain the choice between traditional service and sst. accordingly, situational factors like perceived or experienced waiting time (djelassi, diallo, & zielke, 2018; gelderman et al., 2011) and automatic behaviours like habits (wang et al., 2013, 2017) have been found to significantly shape sst use behaviour. wang et al. (2013) find that customer’s decision to continue sco usage is initially rationally driven, then emotionally driven, and finally becomes habitual. this suggests that attitude and intention alone do not always predict actual behaviour exhaustively. even though the link is strong in the initial usage phase, the significance of deliberate attitudes and intentions weakens during the habit formation and thus they are inadequate to explain automatic, habit-driven behaviour (limayem, hirt, & cheung, 2007; wang et al., 2013, 2017). however, for a habit to form, customers need to adopt and use the technology first. previous research on sst use has found that customers encounter significant trade-offs when it comes to embracing ssts (parasuraman, 2000; vannucci & pantano, 2019). these are often related to customers’ predispositions and discomfort toward new technologies, and they materialise, for instance, as the complexity of learning to use sst and the fear of failure when operating it (liao & cheng, 2013; liljander, gillberg, gummerus, & van riel, 2006; parasuraman & colby, australasian journal of information systems rinta-kahila & penttinen 2021, vol 25, research article four flavours of customers 4 2015; zhu, nakata, sivakumar, & grewal, 2013). thus, it is no wonder that the incumbent habit of using traditional human service can be strong, and not every customer is willing to become an sst user (gelderman et al., 2011). although previous research has established that both habit (venkatesh, thong, & xu, 2012; wang et al., 2013) and situational factors (collier et al., 2015; wang et al., 2012) shape sst use behaviour, it has not gone deeper into examining the universality of these effects. while evidence of their overall significance exists, there are good reasons to believe the effects are different for different types of consumers: some might be more resistant to situational triggers and stick to their behavioural patterns while others may operate more dynamically. this raises the question about what different types of consumers exist with regard to the choice between sst and human-delivered service. we posit that drawing on dual-system theories can shed new light on the sst use (or non-use) phenomenon by helping to uncover how environmental cues and internal habits translate into behavioural outcomes through one’s cognitive processing. 3 theoretical review this section lays the theoretical foundations for the current study by discussing dual-system theories and how they link to habits. 3.1 dual-system theories dual-system theories posit that humans’ thought processes are best represented by two discrete yet interactive systems, which we refer here to as the reflexive system (one that processes data automatically) and the reflective system (one that processes data deliberately) (dhar & gorlin, 2013; kahneman, 2003, 2011; laureiro-martínez & brusoni, 2018; lieberman, 2007). these systems are not isolated but are interconnected and parallel in complex ways (gawronski, 2013; martin & sloman, 2013). this may manifest sequentially in a way that the reflexive system feeds material to the reflective system which then corrects or filters the effects of the reflexive impulses before they materialise in actual behaviour (bond et al., 2009). one important distinction between dual-system theories concerns their primary focus (bond et al., 2009) which can relate either to a) broader theory of personality, emphasizing individual characteristics (e.g., epstein, 1994) or b) situational behaviour, highlighting the reasoning regarding a specific task (e.g., kahneman, 2003; sloman, 1996). thus, researchers may choose to concentrate on the characteristic processing styles tied to the decision-maker’s personality or alternatively to focus on the thought processes immediately related to the task or choice behaviour itself. one example of the former type of theory is epstein’s (1994) cognitiveexperiential self-theory (cest), which suggests that people differ in the degree to which they characteristically operate in rational or experiential style. the dominance between these two systems has also situational variations, namely situational circumstances and emotional arousal which can be considered as moderating factors (epstein, pacini, denes-raj, & heier, 1996). simon and usunier (2007) apply cest to study the influence of personal thinking styles on the choice between sst and traditional human-delivered service and find that in the case of complex services, high rational engagement and low experiential style contribute to sst preference. vice versa, low rational engagement and high experiential style promote preference for personal service. however, the influence of the personal thinking style was found non-significant if the service is perceived as simple, and the situational effect of waiting time was smaller than with complex technologies. on the other hand, kahneman’s (2003) australasian journal of information systems rinta-kahila & penttinen 2021, vol 25, research article four flavours of customers 5 division between system 1 and system 2 (stanovich & west, 2000) concerns the immediate task-specific processes and thus needs to be investigated in a real-life setting. finally, some consumer studies leverage the dual-system perspective to investigate self-control dilemmas, such as excessive use of the internet and social media (li, guo, & yu, 2019; osatuyi & turel, 2018; turel & qahri-saremi, 2016). in such studies, the different levels of focus are sometimes mixed (bond et al., 2009), e.g., soror et al. (2015) examine how a specific personal trait (general self-regulation) interacts with a behaviour bound to a certain context (habit of mobile phone use). 3.2 habits one focal aspect of information processing relates to habits, which can be defined as learned sequences of actions that are automatic responses to specific situations and function in attaining certain goals or end states (verplanken, aarts, & vanknippenberg, 1997). while habitual behaviour tends to occur without active awareness, habits are intentional in the sense that they have a functional or goal-oriented motivation or origin (polites & karahanna, 2012). thus, the formation of habit requires conscious reflection before it becomes automatic. this makes self-efficacy an important antecedent of habit in the early stages of technology use when one is learning a new behaviour (wang et al., 2013, 2017). from the dual-system perspective, this means that one’s processing style of a certain behaviour may change over time and through repetition: reflective processing required for learning to perform the behaviour changes into reflexive processing through habit formation (aarts & dijksterhuis, 2000). habit is then triggered by specific situational cues, often outside the awareness of an individual. habits are mentally efficient, meaning that they free an individual’s cognitive resources for other tasks (polites & karahanna, 2012). partly due to this, habits are often difficult to control or suppress (aarts & dijksterhuis, 2000; soror et al., 2015). overall, habit formation is associated with developing expectations about behaviour, which further reduces both the individual’s sensitivity to variations in the environment (e.g., situational factors) and one’s reflection and deliberation of the behaviour (verplanken and wood, 2006, p. 91). moreover, automatic and reflexive (and thus sometimes even unconscious and involuntary) behaviour may originate from a reasoned choice based on highly analytical reflection. 3.3 conceptual framework based on the discussion of habits’ powerful influence on behaviour, we expect some consumers to be driven with their prior experiences and thus stick to their incumbent habits regardless of other service options that might be available. yet, it is likely that not all consumers make the choice automatically. shopping tends to occur in a somewhat chaotic environment where, besides their objectives and preferences (e.g., habits, internal requests), shoppers are subjected to the influence of various environmental forces (e.g., crowdedness, store staff availability, external requests) (gelderman et al., 2011; wang et al., 2012). this means that some might apply deliberate reflection to optimize their choices. interestingly, in a recent meta-analysis, blut et al. (2016) find insufficient support for habit determining actual sst use behaviour. this finding contradicts some previous research on the topic (venkatesh et al., 2012; wang et al., 2013) but points to the possibility that some consumers might make more situational choices than they think they do. considering that habits can be unconscious, extant theories on technology use behaviour and conventional survey methodology may be insufficient for capturing the existence of habit. then again, it could be that situational triggers can sometimes overcome habit (at least with some consumers), weakening its aggregate effect australasian journal of information systems rinta-kahila & penttinen 2021, vol 25, research article four flavours of customers 6 on behaviour. these possibilities and the general lack of considering cognitive processing in the choice between sst and human-delivered service invite us to take a dual-system approach on the topic in a real-life field study setting. figure 1 illustrates how habits stemming from prior experience and situational factors in the choice environment exert their competing influences on consumer’s cognitive processing. the tug of war between reflexive and reflective processing then results in alternative outcomes. our research investigates what kinds of behavioural types can be distinguished based on the interplay between these two sources of influences. figure 1. conceptual framework 3.4 research approach to produce a typology of consumer behaviours based on sst-related decision-making behaviour one needs to capture the extent to which situational triggers cause consumers to switch their cognitive gears from automatic mode to deliberate thinking and vice versa (louis & sutton, 1991). to this end, the discussion above points to the importance of capturing a) the level of cognitive processing one used in a choice situation, b) the environmental cues one experienced, and c) one’s incumbent habits and attitudes. the challenge of directly measuring one’s level of cognitive processing and the difficulty to accurately perceive or recall environmental cues suggests that these should be probed as soon as possible after the decisionmaking situation. the fact that consumers’ cognitive processing often happens without their active awareness indicates that one could approach measuring the extent of reflection by applying indirect proxy questions, such as the extent to which they deliberately surveyed the crowdedness of the sst use area. the level of processing, in turn, helps to establish the extent of habitual behaviour (i.e., low level of processing indicates strong habit), which can be further ascertained by inquiring about consumers’ prior experiences, behavioural patterns, and general experiences with the sst. thus, drawing on the interplay of reflexive and reflective behaviour that is subject to both habit and situational cues, and building on the past work conducted in similar settings (simon australasian journal of information systems rinta-kahila & penttinen 2021, vol 25, research article four flavours of customers 7 & usunier, 2007), this paper explores how consumers differ in making the choice between sst and human-delivered service. 4 empirical study surprisingly, especially given the vast body of literature on the various forms of dual-system theories, there exist very few studies in which researchers would have attempted to empirically tap onto the reflexive and reflective processing modes in consumer-choice situations. attempts in the extant literature to distinguish between the two processing modes can be categorised roughly into three approaches. first, it is known that when an individual engages in reflective processing, her pupils tend to dilate, heart rate increase, and level of blood glucose drop (kahneman, 2011). also, it has been shown that different patterns of neural activation in brain activity can be spotted supporting reflective or reflexive social cognition (lieberman, 2007). however, detecting these types of psycho-physiological changes in reallife consumer choice settings such as in retailing would be cumbersome. second, the difference in response time has become the most commonly accepted measure to distinguish between reflective and reflexive cognitive modes; reflexive processing being fast, reflective slow (laureiro-martínez & brusoni, 2018). relying on response latencies, the iat (implicit association test) is a popular method for determining implicit cognition. while such methods might perform well in laboratory settings, we posit that they pose insurmountable measurement challenges in real-life consumer choice settings. third, survey-based research conducted in imaginary (meyers-levy & maheswaran, 2004; simon & usunier, 2007) or recalled (soror et al., 2015; turel & qahri-saremi, 2016) decision situations has severe limitations in capturing actual choice behaviour and cognitive processes behind it. earlier research on customer choice between sst and human-delivered service suggests that the emphasis should be on actual behaviour rather than on attitude and behavioural intentions (meuter, bitner, ostrom, & brown, 2005). thus, a field study was conducted in grocery stores that offer both scos and traditional checkout options for their customers. taking an interpretive approach has been found useful when investigating novel phenomena in the sst context (see, bitner, booms, & tetreault, 1990; meuter, ostrom, roundtree, & bitner, 2000; wang et al., 2012). while the sst research is reaching saturation, this study takes a previously uncharted perspective on sst usage by considering the cognitive processes that take place in a real-life decision-making situation. especially in a case like this where no previously established methodology exists for investigating the research question, it can be argued that an interpretive approach is appropriate in the initial exploration of the phenomenon, which could then pave the way for more systematic inquiries. as such, a structured interview protocol was designed to tap on the cognitive processes of customers right after performing the behaviour of interest, namely checking out from a grocery store. in line with wang et al. (2012), this study leverages the benefits of both objective observational data and interpretive interview data. 4.1 data collection the data collection methods included both non-participant observations and structured interviews at the service sites. these methods were chosen because while interviewing allows one to capture customers’ subjective experience of the checkout selection, observation provides the researcher with relevant situational information regarding the event. to get permission for data collection, one of the largest grocery-retail chains in finland was australasian journal of information systems rinta-kahila & penttinen 2021, vol 25, research article four flavours of customers 8 approached for cooperation. the chain gave their approval for the data collection in three of their grocery stores that had implemented scos, referred to as site a, site b, and site c. the stores are virtually identical in terms of size and product assortment, and they are all situated in busy, central locations in the capital. moreover, the chain also provided us chocolate bars to be given for each informant as an incentive and reward for participating in the interview. before the field study, the authors interviewed the store managers of site a and site b to gain an overall understanding of the objectives of the sco implementation and its effect on the daily operations in the stores. the data was collected by one of the authors and a research assistant. during the field study, the two data collection methods were applied sequentially, starting by observing the customer and continuing with an interview. specifically, the observation started when a customer entered the checkout area and ended after the transaction was finished and the customer was invited for an interview. the selection of informants was done randomly, however, the aim was to obtain roughly similar sized samples of informants from customers checking out through scos and the ones choosing traditional tills. if the observed customer did not want to be interviewed, that was counted as a non-response event in the overall sample. the observed items were gender, shopping tool, number of shopping companions, approximate waiting time, payment method, the number of purchases, and most importantly, the chosen checkout. also, the subject’s age was estimated, which was later confirmed in the interview if the informant was willing to share that information. additionally, an illustration of the checkout area (see appendix a, table a1) was produced beforehand and filled according to the customer’s movement, i.e., from which aisle the customer enters the area and which checkout is finally selected. observations were complemented with one-on-one interviews which were conducted immediately after the checkout. the interview form was informed by dual-system frameworks (kahneman, 2003, 2011; louis & sutton, 1991) aimed to probe situational cognitive processing, research on sst use (e.g., wang et al., 2012), and relevant contextual considerations. both close-ended questions with given answers to choose from and open-ended ones allowing respondents to elaborate their answers were used. additionally, some close-ended questions had a follow-up inquiry ‘why’ to gain a more profound understanding of the reasoning behind the answer. to assess the questionnaire’s validity, a pilot test was conducted at site a. in total, 17 customers agreed to take part in the pilot study. this sample consisted of ten self-checkout users and seven traditional-checkout users. after reflecting on the conducted interviews and examining the gathered data, the questionnaire was revised into its final form. questions were reordered, wordings were revised, and more options were included in the close-ended questions. the questionnaire leveraged multiple questions to capture the extent to which customers had consciously processed situational factors, including: did you observe the checkout area (e.g., waiting lines, other customers’ purchases, free checkouts)? and give an estimation of the extent of your observation between a range of 1-7, where 1 is minimal observation or none, and 7 is very attentive observation. such questions also informed about the automaticity of the behaviour by indicating the extent to which habit was dominating decision-making. also, questions about customers’ previous sco use experience were included to establish the existence of previously learned habits. additional questions charted australasian journal of information systems rinta-kahila & penttinen 2021, vol 25, research article four flavours of customers 9 the motivations for choosing the selected checkout and whether the choice was perfunctory, among several other factors. the final data collection stage was conducted using the revised questionnaire (see appendix a) at sites a, b, and c. this resulted in a sample of 69 customer observations and interviews, of which 39 checked out using sco and 30 used traditional service. descriptive statistics are provided in table 1. the average age of the respondents is 38 years, ranging between 13-85 years with a standard deviation of 17.4 years. females constitute 65% of the sample. as expected, sco users are younger on average than those using traditional checkouts. interestingly, sco users bought more items on average than did traditional checkout users. less surprisingly, customers queuing to the traditional service had to wait significantly longer time than sco users. most of the respondents (84%) had tried scos before, and most reported intending to use them in the future. however, the ones that used traditional checkouts at the time of the interview reported remarkably lower intentions to do so. traditional checkout sco total n 30 39 69 average age (years) 43.8 33.1 37.8 gender (female) 70% 62% 65% number of shopping companions 0.5 0.3 0.4 number of purchases 5.4 6.3 5.9 queuing time (min:sec) 1:44 0:08 0:50 made observations at the checkout area 83% 69% 75% level of observation (1-7) 5.1 4.6 4.8 checkout choice was perfunctory 87% 82% 84% satisfaction with the choice (4-10) 9.2 9.5 9.4 has tried sco before / uses sco often 67% 97% 84% intention to use sco in the future yes 57% possibly 27% no 17% yes 97% possibly 3% yes 80% possibly 13% no 7% table 1. description of the sample 4.2 data analysis customers in the sample divide naturally into two groups: the ones who checked out using traditional checkout and the ones who checked out using sco. the objective was to distinguish between habitual and situational behaviour by tapping on to the types of cognitive processing they applied in their checkout channel selection. we conducted data analysis in three distinct steps. first, we extracted the content of the interview forms and our additional notes by manually tabulating the data into a conceptual matrix in excel. the data was structured per different factors that were probed in the observation and interview stages. the individual research subjects (i.e., consumers) were listed on vertical rows and the variables captured in the data collection stage were recorded horizontally into the table’s columns. this allowed us to take a holistic view of the sample and start producing classifications. any data that could not be fitted to the readily structured columns (e.g., additional notes, customer’s comments) was recorded into a column for extra information that could be used to inform classifications. australasian journal of information systems rinta-kahila & penttinen 2021, vol 25, research article four flavours of customers 10 second, we produced an initial classification of consumers by looking into the automaticity of their choice by translating the data into a tree-like structure. this tree had a root value (total sample) and subtrees of children with parent nodes depending on the data values. we first divided the sample into those who checked out using traditional checkout and those who used sco. we expected to find consumers whose checkout channel selection was automatic (i.e., reflexive) and those whose selection was deliberate (i.e., reflective). to this end, we leveraged three main metrics: consumer’s prior sco use experience, intentions to use sco, and the extent of observing the checkout area before checking out. figure 2 illustrates the tree-like structure of the data and the analytical approach. figure 2. the classification system to classify the traditional checkout users into habitual and situational user types, we checked whether they had any previous use experience with scos or intentions to use them in the future. if a customer has never used sco and checks out through traditional till at the time of the interview, we concluded that s/he is using the traditional method out of habit. although such customers might have intentions to try scos, they have not yet broken the habit of using the traditional tills. thus, those customers who had never used scos before were preliminarily categorised as habitual traditional checkout users. moreover, also the ones who reported having used scos before but had no intentions of using them again were considered to belong to the same group as trialling sco had not caused them to break their habits. on the other hand, customers who reported having used scos before and expressed intentions of using them again were considered potential situational users. this classification was checked against the level of observation exercised at the checkout area so that high levels supported australasian journal of information systems rinta-kahila & penttinen 2021, vol 25, research article four flavours of customers 11 situational use. while these customers consider both checkout channels as viable options, this time they chose the traditional channel based on their reflection of the situation. for self-checkout users, the initial division between habitual and situational users was established similarly, by first looking into their prior sco experience and then their reported observation of the checkout area. the level of observation was an especially critical metric because the sco channels in the stores of this study are designed so that there is always one queue to the sco area that comprises of altogether five sco tills. the next customer in the sco queue gets to pay at the checkout as soon as anyone of the five tills becomes vacant. thus, if the customer is habitually choosing sco without further reflections, there is no need to specifically observe neither the sco area nor the overall checkout area. however, if a customer reports having observed queues and other factors at the checkout area, that indicates that the customer has contemplated on whether to choose sco or traditional checkout. as such, the ones who did not observe the checkout area were classified as habitual sco users and the ones who observed the area as situational users. third, we triangulated the initial classifications by analysing other indicators and notes taken of the customers’ elaborations. we looked into the reported level of deliberation put into the checkout selection, whether the selection was considered perfunctory, the point in time which the customer told having made the choice, motivations for choosing one channel and not the other, along with other relevant data. for instance, low level of observation and knowing which checkout channel to use already before entering the store coupled with strong beliefs that sco is a faster or more convenient checkout method supported habitual sco use behaviour, as illustrated by customer a in the example below: customer a: checked out using sco • had used sco before • did not observe the checkout area; a level of observation 2 out of 7 • uses sco because they are fast, convenient, and pleasant • knew which checkout channel to use already before entering the store • quote: “i can’t think of a situation where i would use a traditional checkout, nowadays i always use sco” conclusion: habitual user then again, reflective situational behaviour was associated with a high level of observation and statements that emphasized the situational nature of the choice: customer b: checked out using sco • had used sco before • observed the checkout area; a level of observation 7 out of 7 • likes sco because they are fast and allow simultaneous scanning and packing • did not know which checkout channel to use before entering the checkout area • quote: “i would use traditional checkout if it had no queue and if i had only a couple of items” conclusion: situational user australasian journal of information systems rinta-kahila & penttinen 2021, vol 25, research article four flavours of customers 12 moreover, visual mapping of the customer journey at the checkout area helped to verify the analysis: while the drawings showed some customers marching straight to their chosen checkout lane without hesitation, others appeared to halt for a second to observe and process the situation before making the choice. appendix b provides exemplary customer vignettes that demonstrate how classifications were reached. the classifications were first done independently by each author of this manuscript according to the principles outlined above. to assess the interrater reliability (i.e., the agreement among raters), we ran cohen's kappa in spss to determine the extent to which the two authors agreed in their classifications (landis & koch, 1977). it yielded a κ-value of 0.868 (p < 0.001), which means a percent agreement of 86.8%. this indicates that the two raters were not in agreeance about 13.2% (1-0.868) of the classifications. while a κ-value of 0.868 represents almost perfect agreement (landis & koch, 1977) and signals of sufficient interrater reliability, we discussed the differences in classifications until a full agreement was reached. 5 findings after analysing the data in line with the principles described above, three distinct types of customers emerged as expected: habitual traditional checkout users, habitual sco users, and situational users (who checked out using either traditional checkout or sco). in addition, a fourth customer type was discovered whose checkout selection behaviour did not fit into the initial categories. this fourth type was named as drifting users. the resulting customer types are summarised in table 2. next, the characteristics of the discovered customer types are discussed. customer type n choice ratio (sco vs. traditional) number of purchases observation at the checkout area most common reasons for the current checkout choice habitual traditional checkout users 12 0% 100% 4.9 moderate (mean 4.6, sd 2.5) habit; wants to preserve cashier jobs; no prior experience of sco; human contact habitual sco users 11 100% 0% 6.9 low (mean 1.7, sd 0.9) speed of checkout; handy; usually no queue situational users 44 61% 39% 6.0 high (mean 5.7, sd 1.6) length of the queue at the time of checking out; speed of checkout; situational need drifting users 2 50% 50% 4.0 low/moderate (mean 3.0, sd 2.8) happened to be near; no queue total sample 69 57% 43% 5.9 table 2. the resulting customer types 5.1 habitual traditional checkout users most of the habitual traditional checkout users had never tried scos, and when asked whether they intend to do so, their response was either ‘no’ or ‘possibly’. although two of them had tried scos in the past, they stated not intending to use them again. thus, most habitual traditional checkout users have not yet broken their incumbent habit by trialling scos – and the ones who have done that have concluded that they prefer to stick with the traditional service. interestingly, these customers reported moderate levels of observation (on average 4.6 australasian journal of information systems rinta-kahila & penttinen 2021, vol 25, research article four flavours of customers 13 out of 7, sd=2.5) at the checkout area. this can be explained by the insight that some of them were contemplating on which one of the traditional checkouts to choose. additional measures and the high standard deviation in reported observation levels lend support to this notion. these customers expressed the most diverse set of reasons for selecting traditional checkout. the most cited reasons were incumbent habit and lack of prior experience of sco use. moreover, these customers want effortless personal service and human contact, and they are often worried about making errors when operating them. furthermore, many stated their concern about cashier jobs disappearing due to the emergence of technologies like sco. finally, habitual traditional checkout users may want to use cash and buy cigarettes, which can only be done at traditional checkout, and this reinforces them to stick with their incumbent habits. when asked what would make them choose sco instead, some stated that they would not use sco in any circumstances, while others said that they might consider using sco in the case of prohibitively long queues to the traditional checkouts. these customers waited in line for 2:25 minutes on average and represented the oldest people in the sample with an average age of 50. 5.2 habitual sco users habitual sco users use sco automatically whenever it is available. all such customers in the sample had prior experience of using scos and they intended to continue using the technology in the future too. they described their checkout selection as perfunctory and reported of not having observed any environmental factors in the checkout area. this was confirmed by their low reported average rate of observation (1.7 out of 7, sd=0.9). the most often mentioned reason for selecting sco was their general belief that scos offer speedier checkout and that they do not usually have queues. these assumptions appeared to be behind forming the habit of sco use. the data supports their belief: habitual sco users waited in line only for 0:11 minutes on average. other reasons were related to convenience, privacy, control, and fun provided by scos. when asked, what would make them choose traditional checkout instead, most stated that they would do that only if scos were not available at all or that they would have to buy something that is not available at sco. the average age in this group was 41 years. 5.3 situational users situational users rarely have any predetermined checkout choices in mind when they walk into the store – instead, they tend to make their final decision at the checkout area. these customers reported consistently high levels of observation at the checkout area (5.7 out of 7, sd=1.6), and ended up making non-perfunctory choices more often than others. everyone had previous experience of sco use, except for one customer who used sco for the first time at the time of the interview. situational users had highly circumstantial motives for choosing between technology and human-delivered service. those who chose traditional service did that because the queue was short, they wanted to use cash, or needed to buy something that is not available at sco (e.g., cigarettes, stamps). those situational users who chose sco this time mentioned short queues and a small number of purchases as the main motivations for their choice. however, the second most cited reason was a general belief that scos are faster. situational users were generally willing to make a different choice of checkout than they did at the time of the interview if the situation would require so: both checkout alternatives were considered as australasian journal of information systems rinta-kahila & penttinen 2021, vol 25, research article four flavours of customers 14 viable options. some customers in this group reported normally paying more attention to the checkout selection but this time their choice was exceptionally impulsive due to certain internal or external triggers (e.g., tiredness, shopping companion, or receiving a phone call) that inhibited the functioning of their reflective system. it appears that leveraging the reflective system bears fruit in terms of efficiency: situational users queued only for 0:33 minutes on average. the average age of a situational user was 34 years. 5.4 drifting users two customers reported simply ‘drifting’ to the closest checkout (sco or traditional) without paying any attention to queues or other factors in the surroundings. thus, these customers appeared surprised when prompted with questions about their checkout selection – they had not even thought about making a choice. they further emphasized that while they made no cognitive efforts to choose the optimal service option – neither this time nor in general – they also do not have a routine or habit of whatsoever regarding checkout selection. instead, they end up to the nearest checkout available, no matter if it is a traditional or an sco till. consistently, they exhibited low to moderate levels of observation at the checkout area and had no previously established intentions regarding choosing the checkout method at the time of shopping, but both had used sco before and intended doing so in the future too – if they would happen to drift to the self-service tills. these users represented the younger end of the sample with an average age of 27 years. 6 discussion our analysis reveals the occurrence of both extreme reflective and reflexive processing and interesting nuances in between them. customers can apply deliberate reasoning through high levels of reflective processing (situational users) or make the decision entirely automatically relying on reflexive instincts (drifting users). in between these extremes are the habitual users who automatically go to either scos or traditional tills, and thus do not need much situational reflection regarding the choice between sst and human service due to their habit, beliefs, or attitude. habitual traditional checkout users exhibited a strong habit of using the traditional service and seemed unlikely to deviate from it. those traditional checkout users who had no prior experience in sco use exhibited inertia in technology adoption (polites & karahanna, 2012) – the reason for not using scos was – paradoxically – often reported to stem from the lack of prior sco use. the ones who had tried sco in the past expressed no intentions to return using scos in the future, suggesting that their first trials were disappointing, indicating unsuccessful recovery from a service failure (collier, breazeale, & white, 2017). similarly, in line with previous findings (wang et al., 2013, 2017), incumbent habit and established beliefs of scos’ efficiency emerged as strong drivers of habitual sco use. the convenience of scos may have decreased these customers’ need for human interaction (chen et al., 2018; collier & kimes, 2012), as the users have been able to substitute traditional service almost entirely with the sst. habitual traditional checkout users were older than habitual sco users, which is in line with previous findings indicating that younger people are more prone to adopt ssts (dean, 2008). by contrast, situational users continuously employ reflective processing in their search for an optimal choice. the selection is made based on a reflective evaluation of the observed situational factors at the checkout area and the users’ current needs and preferences, making australasian journal of information systems rinta-kahila & penttinen 2021, vol 25, research article four flavours of customers 15 situational users’ checkout selection harder to predict than that of habitual ones. the different motivations behind checkout selection between habitual sco users and situational users corroborate this: while the former ones usually referred to their general belief of scos being a fast checkout method as the main motivation for selecting sco, the latter group tended to emphasize the current queue situation or the crowdedness (collier et al., 2015; gelderman et al., 2011) of the checkout area. finally, certain individuals do not apply any kind of (conscious) cognitive processing but go to any checkout that is available nearby. however, one should consider the possibility that through experience they have found that neither forming a habit nor using constant optimisation helps them to make better choices, and thus they have stopped using their cognitive resources for such daily dilemmas. consumer choice behaviour lies at disciplinary crossroads where advancements in our understanding continue to have important implications in diverse fields such as retailing, consumer services, psychology, strategic management, and marketing. the different consumer types put forth here provides implications to both researchers and practitioners. our contributions, summarized in table 3, will be discussed next. literature stream what we knew what we now know consumer research consumers can be classified into different types based on, e.g., the temporality of their technology adoption (rogers, 2003), the extent of technology use (shih & venkatesh, 2004), and consumption styles (friedrich & zanker, 2011). consumers can be classified into different types based on how they choose between sst and human-delivered service. at least four types exist in the context of retail sco: habitual traditional checkout users, habitual sco users, situational users, and drifting users. sst research situational factors such as waiting time and crowdedness influence sst use decisions (collier et al., 2015; gelderman et al., 2011). habit drives sst use (wang et al., 2013, 2017). situational factors shape the choices of situational users but have little effect on habitual users. the latter ones are heavily driven by their learned habits. drifting users neither process situational factors nor adhere to particular habits; their decisions are shaped by the environment without their active awareness. technology acceptance research technology use is explained by perceptions of technology such as usefulness and ease of use, and individual traits such as technology readiness, selfefficacy, and computer playfulness (blut et al., 2016; venkatesh et al., 2012). individual propensity to simplify or optimise daily choices shapes technology use behaviour, as technology is increasingly present in different aspects of daily life. table 3. contributions of the study 6.1 theoretical implications first, to our best knowledge, this is the first study that produces a typology of consumers based on their decision-making behaviour when selecting between sst and human-delivered service. we provide a typology of consumers based on the extent to which they are driven or affected by situational factors and habits. the findings corroborated our initial intuition that consumers do differ in this respect: some tend to perform deliberate processing while others operate automatically. by shedding light on the interplay of situational factors and habits with australasian journal of information systems rinta-kahila & penttinen 2021, vol 25, research article four flavours of customers 16 different types of consumers, this study deepens the current understanding of how such forces shape consumer behaviour in a retail setting. our typology resonates with that of shih and venkatesh (2004), who classified consumers into four groups based on the variety and rate of their technology use: intense, specialized, nonspecialized, and limited. our approach is different in that we focused on the choice situation instead of general technology perceptions, leading us to distinguish contextualized use behaviours. second, the study contributes to the rich stream of literature on retail ssts (amorim et al., 2016; collier et al., 2015; lien et al., 2019; vannucci & pantano, 2019) and, specifically, on the consumer choice problem of selecting between an sco and a traditional checkout (gelderman et al., 2011). to this end, we demonstrate the usefulness of the dual-system approach when studying such decision-making. we show that neither situational factors nor habits exert their effects uniformly across the consumer pool, but their effects vary between consumer groups. we also distinguish an unexpected type of consumer that we refer to as drifting users. our interpretation of this finding is that choosing not to make a deliberate choice – whether the latter choice would involve establishing and following a habit reflexively or exercising conscious reflection every time – could serve as a method of increasing one’s well-being. accordingly, both drifting users exhibited high satisfaction with their choice. this is interesting as it contradicts dijksterhuis et al’s (2006) finding that conscious processing of simple decisions (which checkout selection arguably is) would result in higher post-choice satisfaction than giving them only a little attention. dhar and gorlin (2013) note that “when two options are equally attractive, justification does not provide a clear reason for choosing one versus the other.” thus, when a consumer has neither an incumbent preference over the checkout method nor the desire to optimise the selection, choice deferral in the form of drifting could be expected to happen. overall, studying drifting users warrants further research efforts. third, our findings provide novel perspectives on technology acceptance research in general. our typology connects to rogers’ (2003) stages of innovation adoption (e.g., exposure, adoption, continued use) in that situational technology use seems to occur more prevalently in the early stages of technology introduction as some consumers are still seeking the optimal balance between use and non-use. later a specific segment of users then matures into habitual technology users. while technology introduction can often disrupt consumer behaviour, it appears that consumers’ inherent traits influence their behaviours above and beyond a specific technology. thus, the use of cognitive processing with regard to the use of technology appears to be influenced not only by technology’s characteristics but also by customers’ inherent preferences between simplicity and optimisation in daily life decisions. while simplifiers look for easy solutions for accomplishing a task, optimisers are willing to strive to find the best solution possible, even if that would incur additional cognitive workload or even stress (adams, 2013). this brings an additional perspective to simon and usunier’s (2007) finding that the choice between sst and human service is influenced by the extent to which consumer exhibits rational or experiential thinking style. for instance, habitual sco users may have been optimisers to a certain point but after enough reflection they have preferred to solidify the most optimal mode of behaviour into a habit, perhaps to save cognitive resources. on the contrary, situational users may not settle for the solution that is the most optimal on average – instead, they remain optimisers to the very end. to some extent, this optimisation seems to australasian journal of information systems rinta-kahila & penttinen 2021, vol 25, research article four flavours of customers 17 pay off, since the waiting times were observed low among both situational users and habitual sco users. this could relate to the paradox of choice: having more options to choose from leads to worse satisfaction with one’s choices due to increased expectations and heightened regrets, and thus having a plethora of choices can have a paralysing effect on one’s decisionmaking (schwartz, 2004). 6.2 managerial implications identifying customer types can help retailers to distinguish the critical issues that should be considered when implementing ssts. this study’s findings highlight the importance of understanding different customer types when managing staffing and store layout. considering that most consumers appear to be situational users, it might make sense for retailers to focus on catering to this segment. however, interviews with the store managers revealed that customers’ sco utilization rates tend to be below their desired levels. the manager of site a explained that while a satisfactory utilization rate would be at 30% of all customers checking out through scos, that rate has remained between 25-27% in the store. similarly, the sco utilization rate at site b was reported as low as 20% of all the customer checkouts (and 16-17% of the aggregate transaction volumes). thus, it may be in the interest of store management to increase the number of sco users. this could happen at least in two ways: 1) getting habitual traditional checkout users to trial scos, and through that turn them into situational users, and possibly later even into habitual sco users, and 2) prompting situational and drifting users to choose sco more often. these objectives can be advanced by occasionally increasing the number of service staff at the sco area who can then invite people to try scos and provide assistance with operating them. moreover, layout accessibility has been found to significantly affect customers’ satisfaction with a retail environment (wakefield & blodgett, 1996). thus, scos could be made more accessible through careful design of store layout by situating them so that people are easily prompted to use them. finally, recent research has discussed the significance of service providers being clear about their motivations for the sst introduction (nijssen, schepers, jeroen, & belanche, 2016), suggesting that retailers should emphasize their position on the matter, which is often that scos are not introduced to substitute human employees but rather to provide better service for customers. 6.3 limitations and avenues for future research this study comes with a number of limitations that need to be addressed. first, one might reasonably assume that most of the customers in the sample were not in a terrible hurry during their shopping trip, since they had time to be interviewed. if this is true, the sample may be biased toward customers whose checkout selection was not affected by hurry. however, when inviting customers for an interview, the interview’s brevity and the importance of participation were emphasized. in addition, a few informants that mentioned being in a bit of a hurry still found time to respond to the questionnaire, so this limitation should not severely impact the results. second, regarding self-reported sco use intentions, it is possible that some customers (especially traditional checkout users) responded in a socially desirable way to avoid coming across as technology laggards (rogers, 2003). thus, it could be that their behavioural intentions to use sco are lower in reality. moreover, sometimes intentions may not translate into actual behaviour due to strong incumbent habits (in any given customer group), and thus using them as proxies for future behaviour should be exercised with some reservations. australasian journal of information systems rinta-kahila & penttinen 2021, vol 25, research article four flavours of customers 18 finally, our method of assessing the dominance between the two cognitive processing systems is somewhat crude and may lack some relevant components, limiting our ability to make conclusions of cognitive processes. however, we do believe that we managed to capture notions of reflective and reflexive processing in a way that it sufficiently accurate for our purposes of producing a consumer typology. we made significant efforts to ensure the validity of our data and analytical outcomes by informing ourselves of the existing dual-system theories, brainstorming different approaches to capture the reasoning process, and pilot testing the questionnaire. while the currently available technology does not provide noninvasive ways to get inside of customers’ heads to observe their thought processes, the applied research instrument could be refined further. future research can also extend the research scope by studying the interplay of characteristic thinking styles and situational processing. this could yield a clearer overall picture of the forces that influence customers’ choices. references aarts, h., & dijksterhuis, a. (2000). habits as knowledge structures: automaticity in goaldirected behavior. journal of personality and social psychology, 78(1). adams, s. (2013). how to fail at almost everything and still win big: kind of the story of my life. portfolio. amorim, m., lago, a., moscoso, p., & prieto, e. (2016). assisted vs. self-checkout in retail: an empirical analysis of the impact of operational process dimensions on customer satisfaction, recommendation and reuse. journal of service science research, 8(1), 1–39. https://doi.org/10.1007/s12927-016-0001-z atapattu, m., sedera, d., & ravichandran, t. (2014). agility in consumer retail: sense-response alignment through the eyes of customers. australasian journal of information systems, 18(2), 111–132. https://doi.org/10.3127/ajis.v18i2.866 beatson, a., lee, n., & coote, l. v. (2007). self-service technology and the service encounter. the service industries journal, 27(1), 75–89. https://doi.org/10.1080/02642060601038700 bertacchini, f., bilotta, e., & pantano, p. (2017). shopping with a robotic companion. computers in human behavior, 77, 382–395. https://doi.org/10.1016/j.chb.2017.02.064 bitner, m. j., booms, b. h., & tetreault, m. s. (1990). the service encounter: diagnosing favorable and unfavorable incidents. journal of marketing, vol. 54, pp. 71–82. https://doi.org/10.2307/1252074 blut, m., wang, c., & schoefer, k. (2016). factors influencing the acceptance of self-service technologies: a meta-analysis. journal of service research, 19(4), 396–416. https://doi.org/10.1177/1094670516662352 bond, s. d., bettman, j. r., & luce, m. f. (2009). consumer judgment from a dual-systems perspective: recent evidence and emerging issues. in review of marketing research (pp. 3–37). brandtzæg, p. b., & heim, j. (2011). a typology of social networking sites users. international journal of web based communities, 7(1), 28–51. https://doi.org/10.1504/ijwbc.2011.038124 broniarczyk, s. m., & griffin, j. g. (2014). decision difficulty in the age of consumer australasian journal of information systems rinta-kahila & penttinen 2021, vol 25, research article four flavours of customers 19 empowerment. journal of consumer psychology, 24(4), 608–625. https://doi.org/10.1016/j.jcps.2014.05.003 chen, c., white, c., & hsieh, y. e. (2020). the role of consumer participation readiness in automated parcel station usage intentions. journal of retailing and consumer services, 54(october 2019), 102063. https://doi.org/10.1016/j.jretconser.2020.102063 chen, y., yu, j., yang, s., & wei, j. (2018). consumer’s intention to use self-service parcel delivery service in online retailing: an empirical study. internet research, 28(2), 500–519. https://doi.org/10.1108/intr-11-2016-0334 collier, j., breazeale, m., & white, a. (2017). giving back the “self” in self service: customer preferences in self-service failure recovery. journal of services marketing, 31(6), 604–617. https://doi.org/10.1108/jsm-07-2016-0259 collier, j., & kimes, s. (2012). only if it is convenient: understanding how convenience influences self-service technology evaluation. journal of service research, 16(1), 39–51. https://doi.org/10.1177/1094670512458454 collier, j., moore, r. s., horky, a., & moore, m. l. (2015). why the little things matter: exploring situational influences on customers’ self-service technology decisions. journal of business research, 68(3), 703–710. https://doi.org/10.1016/j.jbusres.2014.08.001 collier, j., sherrell, d., babakus, e., & horky, a. (2014). understanding the differences of public and private self-service technology. journal of services marketing, 28(1), 60–70. https://doi.org/10.1108/jsm-04-2012-0071 dean, d. h. (2008). shopper age and the use of self-service technologies. managing service quality: an international journal, 18(3), 225–238. https://doi.org/10.1108/09604520810871856 dhar, r., & gorlin, m. (2013). a dual-system framework to understand preference construction processes in choice. journal of consumer psychology, 23(4), 528–542. https://doi.org/10.1016/j.jcps.2013.02.002 dijksterhuis, a., bos, m. w., nordgren, l. f., & van baanen, r. b. (2006). on making the right choice: the deliberation-without-attention effect. science, 311, 1005–1008. https://doi.org/10.1126/science.1121629 djelassi, s., diallo, m. f., & zielke, s. (2018). how self-service technology experience evaluation affects waiting time and customer satisfaction? a moderated mediation model. decision support systems, 111(august 2017), 38–47. https://doi.org/10.1016/j.dss.2018.04.004 epstein, s. (1994). integration of the cognitive and the psychodynamic unconscious. american psychologist, 49(8), 709–724. https://doi.org/10.1037/0003-066x.50.9.799 epstein, s., pacini, r., denes-raj, v., & heier, h. (1996). individual differences in intuitive– experiential and analytical–rational thinking styles. journal of personality and social psychology, 71(2), 390–405. https://doi.org/10.1037/0022-3514.71.2.390 evanschitzky, h., iyer, g. r., pillai, k. g., kenning, p., & schütte, r. (2015). consumer trial, continuous use, and economic benefits of a retail service innovation: the case of the personal shopping assistant. journal of product innovation management, 32(3), 459–475. australasian journal of information systems rinta-kahila & penttinen 2021, vol 25, research article four flavours of customers 20 friedrich, g., & zanker, m. (2011). a taxonomy for generating explanations in recommender systems. ai magazine, 32(3), 90. https://doi.org/10.1609/aimag.v32i3.2365 galletta, d. f., durcikova, a., everard, a., & jones, b. m. (2005). does spell-checking software need a warning label? communications of the acm, 48(7), 82–86. https://doi.org/10.1145/1070838.1070841 gawronski, b. (2013). what should we expect from a dual-process theory of preference construction in choice? journal of consumer psychology, 23(4), 556–560. https://doi.org/10.1016/j.jcps.2013.04.007 gelderman, c. j., ghijsen, p. w. t., & van diemen, r. (2011). choosing self-service technologies or interpersonal services-the impact of situational factors and technology-related attitudes. journal of retailing and consumer services, 18, 414–421. https://doi.org/10.1016/j.jretconser.2011.06.003 gorlin, m., & dhar, r. (2013). refining the dual-process theory of preference construction: a reply to gawronski, martin and sloman, stanovich, and wegener and chien. journal of consumer psychology, 23(4), 564–568. kahneman, d. (2003). maps of bounded rationality: psychology for behavioral economics. the american economic review, 93(5), 1449–1475. https://doi.org/10.1257/000282803322655392 kahneman, d. (2011). thinking, fast and slow. https://doi.org/10.1017/cbo9781107415324.004 kurnia, s., belts, j., & johnston, r. (2002). understanding the diffusion of efficient consumer response: an australian survey study. australasian journal of information systems, 9(2), 113–122. https://doi.org/10.3127/ajis.v9i2.198 landis, j. r., & koch, g. g. (1977). the measurement of observer agreement for categorical data. biometrics, 33(1), 159–174. https://doi.org/10.2307/2529310 laureiro-martínez, d., & brusoni, s. (2018). cognitive flexibility and adaptive decisionmaking: evidence from a laboratory study of expert decision-makers. strategic management journal, 39(4), 1031–1058. https://doi.org/10.1002/smj.2774 lee, h., & lyu, j. (2016). personal values as determinants of intentions to use self-service technology in retailing. computers in human behavior, 60, 322–332. https://doi.org/10.1016/j.chb.2016.02.051 li, h., guo, y., & yu, q. (2019). self-control makes the difference: the psychological mechanism of dual processing model on internet addicts’ unusual behavior in intertemporal choice. computers in human behavior, 101(july), 95–103. https://doi.org/10.1016/j.chb.2019.07.010 liao, s., & cheng, c. c. (2013). consumer evaluation of self-service innovation failure: the effect of brand equity and attribution. the service industries journal, 33(5), 467–485. https://doi.org/10.1080/02642069.2011.614339 lieberman, m. d. (2007). social cognitive neuroscience: a review of core processes. annual review of psychology, 58(1), 259–289. https://doi.org/10.1146/annurev.psych.58.110405.085 654 lien, c. h., hsu, m. k., shang, j. z., & wang, s. w. (2019). self-service technology adoption by australasian journal of information systems rinta-kahila & penttinen 2021, vol 25, research article four flavours of customers 21 air passengers: a case study of fast air travel services in taiwan. service industries journal, 0(0), 1–25. https://doi.org/10.1080/02642069.2019.1569634 liljander, v., gillberg, f., gummerus, j., & van riel, a. (2006). technology readiness and the evaluation and adoption of self-service technologies. journal of retailing and consumer services, 13(3), 177–191. https://doi.org/10.1016/j.jretconser.2005.08.004 limayem, m., hirt, s. g., & cheung, c. m. k. (2007). how habit limits the predictive power intention: the case of information systems continuance. mis quarterly, 31(4), 705–737. louis, m. r., & sutton, r. i. (1991). switching cognitive gears: from habits of mind to active thinking. human relations, 44(1), 55–76. martin, j. w., & sloman, s. a. (2013). refining the dual-system theory of choice. journal of consumer psychology, 23(4), 552–555. https://doi.org/10.1016/j.jcps.2013.04.006 meuter, m. l., bitner, m. j., ostrom, a. l., & brown, s. w. (2005). choosing among alternative service delivery modes: an investigation of customer trial of self-service technologies. journal of marketing, 69(2), 61–83. https://doi.org/10.1509/jmkg.69.2.61.60759 meuter, m. l., ostrom, a. l., roundtree, r. i., & bitner, m. j. (2000). self-service technologies: understanding customer satisfaction with technology-based service encounters. journal of marketing, 64(july), 50–64. https://doi.org/10.1509/jmkg.64.3.50.18024 meyers-levy, j., & maheswaran, d. (2004). exploring message framing outcomes when systematic, heuristic, or both types of processing occur. journal of consumer psychology, 14(1&2), 159–167. https://doi.org/10.1207/s15327663jcp1401&amp;2_18 nijssen, e. j., schepers, jeroen, j. l., & belanche, d. (2016). why did they do it? how customers’ self-service technology introduction attributions affect the customer-provider relationship. journal of service management, 27(3), 276–298. ortiz de guinea, a., & markus, m. l. (2009). why break the habit of a lifetime? rethinking the roles of intention, habit, and emotion in continuing information technology use. mis quarterly, 33(3), 433–444. osatuyi, b., & turel, o. (2018). tug of war between social self-regulation and habit: explaining the experience of momentary social media addiction symptoms. computers in human behavior, 85, 95–105. https://doi.org/10.1016/j.chb.2018.03.037 parasuraman, a. (2000). technology readiness index (tri): a multiple-item scale to measure readiness to embrace new technologies. journal of service research, 2(4), 307–320. https://doi.org/10.1177/109467050024001 parasuraman, a., & colby, c. l. (2015). an updated and streamlined technology readiness index: tri 2.0. journal of service research, 18(1), 59–74. https://doi.org/10.1177/10946705145 39730 penttinen, e., rinta-kahila, t., ronkko, m., & saarinen, t. (2014) triggering intention to use to actual use--empirical evidence from self-service checkout (sco) systems. proceedings of the 47th hawaii international conference on system sciences, 3347-3355. polites, g., & karahanna, e. (2012). shackled to the status quo: the inhibiting effects of incumbent system habit, switching costs, and inertia on new system acceptance. mis quarterly, 36(1), 21–42. australasian journal of information systems rinta-kahila & penttinen 2021, vol 25, research article four flavours of customers 22 polkinghorne, d. e. (2005). language and meaning: data collection in qualitative research. journal of counseling psychology, vol. 52, pp. 137–145. https://doi.org/10.1037/00220167.52.2.137 rinta-kahila, t., penttinen, e., kumar, a., & janakiraman, r. (2021). customer reactions to self-checkout discontinuance. journal of retailing and consumer services, 61, 102498. https://doi.org/10.1016/j.jretconser.2021.102498 rogers, e. m. (2003). diffusion of innovations (5th edition). new york: free press. roy, s. k., balaji, m. s., sadeque, s., nguyen, b., & melewar, t. c. (2017). constituents and consequences of smart customer experience in retailing. technological forecasting and social change, 124(2017), 257–270. https://doi.org/10.1016/j.techfore.2016.09.022 schwartz, b. (2004). the paradox of choice. harper perennial. shih, c.-f., & venkatesh, a. (2004). beyond adoption: development and application of a usediffusion model. journal of marketing, 68(1), 59–72. https://doi.org/10.1509/jmkg.68.1.59.24029 simon, f., & usunier, j.-c. (2007). cognitive, demographic, and situational determinants of service customer preference for personnel-in-contact over self-service technology. international journal of research in marketing, 24(2), 163–173. https://doi.org/10.1016/j.ijresar.2006.11.004 sloman, s. a. (1996). the empirical case for two systems of reasoning. psychological bulletin, 119(1), 3–22. soror, a. a., hammer, b. i., steelman, z. r., davis, f. d., & limayem, m. m. (2015). good habits gone bad: explaining negative consequences associated with the use of mobile phones from a dual-systems perspective. information systems journal, 25(4), 403–427. https://doi.org/10.1111/isj.12065 stanovich, k. e., & west, r. f. (2000). individual differences in reasoning: implications for the rationality debate? behavioral and brain sciences, 23(5), 645–665. https://doi.org/10.1017/s0140525x03210116 turel, o., & qahri-saremi, h. (2016). problematic use of social networking sites: antecedents and consequence from a dual-system theory perspective. journal of management information systems, 33(4), 1087–1116. https://doi.org/10.1080/07421222.2016.1267529 vannucci, v., & pantano, e. (2019). digital or human touchpoints? insights from consumerfacing in-store services. information technology and people, 33(1), 296–310. https://doi.org/10.1108/itp-02-2018-0113 venkatesh, v., & morris, m. g. (2000). why don’t men ever stop to ask for directions? gender, social influence, and their role in technology acceptance and usage behavior. mis quarterly, 24(1), 115–139. venkatesh, v., thong, j. y. l., & xu, x. (2012). consumer acceptance and use of information technology: extending the unified theory of acceptance and use of technology. mis quarterly, 36(1), 157–178. verplanken, b., aarts, h., & vanknippenberg, a. (1997). habit, information acquisition, and the process of making travel mode choices. european journal of social psychology, 27(5), australasian journal of information systems rinta-kahila & penttinen 2021, vol 25, research article four flavours of customers 23 539–560. https://doi.org/10.1002/(sici)1099-0992(199709/10)27:5<539::aid-ejsp831>3.0.co;2a verplanken, b., & wood, w. (2006). interventions to break and create consumer habits. journal of public policy & marketing, 25(1), 90–103. wakefield, k. l., & blodgett, j. g. (1996). the effect of the servicescape on customers’ behavioral intentions in leisure service settings. journal of services marketing, 10(6), 45–61. https://doi.org/10.1108/jsm-08-2016-0291 wang, c., harris, j., & patterson, p. (2012). customer choice of self-service technology: the roles of situational influences and past experience. journal of service management, 23(1), 54–78. https://doi.org/10.1108/09564231211208970 wang, c., harris, j., & patterson, p. (2013). the roles of habit, self-efficacy, and satisfaction in driving continued use of self-service technologies: a longitudinal study. journal of service research, 16(3), 400–414. https://doi.org/10.1177/1094670512473200 wang, c., harris, j., & patterson, p. (2017). modeling the habit of self-service technology usage. australian journal of management, 42(3), 462–481. https://doi.org/10.1177/03128962166408 62 weijters, b., rangarajan, d., falk, t., & schillewaert, n. (2007). determinants and outcomes of customers’ use of self-service technology in a retail setting. journal of service research, 10(1), 3–21. https://doi.org/10.1177/1094670507302990 zarantonello, l., & schmitt, b. h. (2010). using the brand experience scale to profile consumers and predict consumer behaviour. journal of brand management, 17(7), 532–540. https://doi.org/10.1057/bm.2010.4 zhu, z., nakata, c., sivakumar, k., & grewal, d. (2013). fix it or leave it? customer recovery from self-service technology failures. journal of retailing, 89(1), 15–29. https://doi.org/10.1016/j.jretai.2012.10.004 appendix a: the data collection instrument the observation protocol is presented in table a1. the layout depiction on the last row was used for tracking customers’ movement in the checkout area. letters a-g signify different aisles in the store that lead to the checkout area. the black box in the sco area signifies the employee who supervises the sco transactions. background info date: time: site: researcher: the customer gender: □ man □ woman additional information: age: the checkout situation chosen checkout: □ traditional shopping instrument: □ basket australasian journal of information systems rinta-kahila & penttinen 2021, vol 25, research article four flavours of customers 24 table a1 the observation protocol the interview protocol is presented in table a2. empty space around the questions and the backside of the form were leveraged when making additional notes. interview questions 1. was the shopping trip planned beforehand or did you just happen to stop by? 2. was this your first time in this particular grocery store? 3. at which point did you decide whether to use traditional checkout or self-checkout? □ before entering the store □ while shopping □ at the checkout area 4. did you make observations of the checkout area (e.g., waiting lines, other customers’ purchases, available checkout lanes)? □ yes □ no give an estimation between a range of 1-7, where: (1) i did not make any observations ... (7) i made a lot of observations 5. why did you choose self-checkout / traditional checkout? name the three most important reasons and their relative importance (0-100%). are there any other reasons? 6. in what kind of situation would you choose self-checkout / traditional checkout? any other reasons? □ sco □ cart shopping companions: □ alone □ companions, number____, who they are:______________ mode of payment: □ card □ cash the number of purchases: waiting time for service: illustration of the checkout situation australasian journal of information systems rinta-kahila & penttinen 2021, vol 25, research article four flavours of customers 25 7. how satisfied are you with your checkout selection (give a school grade between 4-10)? why? 8. your checkout selection was… □ perfunctory □ non-perfunctory why? 9. have you ever used self-checkout? / do you often use self-checkout? why? □ yes □ no 10. do you intend to use self-checkout in the future? □ yes □ no □ possibly table a2 the interview protocol appendix b: exemplary customer vignettes this appendix provides exemplary vignettes from the sample. in essence, it presents detailed depictions of customers from each group discussed in the findings section. the purpose of this is to further elaborate on how these customers made their choice of checkout method, why they chose the checkout they used, and in which circumstances they would act otherwise. habitual traditional checkout user (site c) an 82-year old woman was shopping with her friend. at the checkout area, they bypassed the empty sco area while proceeding towards traditional checkouts, scanning for a one with the shortest waiting line. both women were using shopping trolleys and spent approximately two minutes in the queue before receiving service. the woman bought five items and paid them with cash. she accepted the invitation to interview as they were not in a hurry. she explained that the decision to use a traditional checkout was already clear to her when entering the store, as she had never used scos and had no intention to try them. however, before starting to queue she consciously made a considerable amount of observations at the checkout area (5 out of 7) as she wanted to find a traditional checkout with the shortest queue. the incumbent habit was the most important reason (90%) for selecting traditional checkout but she also mentioned preferring personal service (10%). the only situation where she might use sco would be if it was the only option available and that someone would provide her guidance on how to use it. she mentioned being very satisfied with the checkout selection, thanks to the friendly service employee. habitual sco user (site c) a 26-year old woman was shopping alone and carried four small purchases in her hands. after entering the checkout area, she determinedly rushed towards scos, all of which were free at the moment. after checking out, she accepted the request for a short interview despite being in a bit of a hurry. it was her lunch break and she did not want to waste much time. she was working at the upper floor of the same building, comes to this store every day, and tends to use scos. consequently, the decision to use sco was perfunctory and already made before entering the store. thus, she did not observe the checkout area at all (1 out of 7). the reason for selecting sco was that she finds it fast to use (relative importance of 80%) and australasian journal of information systems rinta-kahila & penttinen 2021, vol 25, research article four flavours of customers 26 usually there is no waiting line (20%). only if the self-checkouts are closed, a situation that may occur in the early morning, she would choose traditional checkout. she intends to continue using self-checkouts also in the future. situational user: traditional checkout (site a) a 51-year old man was moving back and forth at the checkout area, contemplating on whether to use sco or go to traditional checkout. finally, he made up his mind and started to queue for a traditional checkout. while standing in the slowly proceeding waiting line, he kept glancing the half-empty sco area as well as the waiting lines to other traditional checkouts. this slight hesitation was noticeable. the man was shopping with his wife and daughter who had already left the store. he had a shopping basket filled with 11 items and used a credit card to pay them. even though his companions were waiting for him outside the store, he accepted the request for an interview. the man explained that before choosing the checkout, he observed the whole checkout area (7 out of 7). after some contemplation, he chose a traditional checkout because the service employee seemed very professional. however, he was not fully satisfied with his decision because the waiting time was relatively long. in the future, he would possibly use sco if the waiting line is significantly shorter compared to traditional checkouts. previously he had used scos in ikea and some other grocery stores. situational user: sco (site a) holding two items in his hands, this 63-year old man selected an empty sco and executed the transaction effortlessly. it was not his first time to use sco: he comes to this grocery store every day and has already adopted the system. however, he does not always choose sco. his decision was based on careful consideration and observations (7 out of 7) of the checkout area. he explained that if the traditional checkouts were free or he had many purchases, he might select the traditional checkout. otherwise, he tends to choose sco because of the anonymity and privacy it provides. while he was satisfied with his checkout selection, he also mentioned that scos have their disadvantages. for instance, he finds that the machines are not ergonomic to use, the plastic bags are not opened beforehand, and the annoying reminder of the bonus card is repeated every time. despite the shortcomings, he intends to continue sco usage in the future. drifting user (site c) a british couple, a man and a woman both in their late twenties, checked out through traditional till. when asked about their choice, they were surprised as they had not really thought about it. they explained that they do not have any particular habit regarding checking out from a store; rather, the decision is usually random or instantaneous. while they reported having observed the checkout area a bit, they did not reflect on which checkout to choose but merely drifted to the closest one available. since there was no queue at this checkout, the choice felt natural to them. however, when prompted on other possible factors that may influence their decisions, they mentioned that they find traditional checkout as effortless. nevertheless, they have used scos before and intend to use them again, especially if they are not carrying many purchases and the traditional checkouts have long queues. australasian journal of information systems rinta-kahila & penttinen 2021, vol 25, research article four flavours of customers 27 copyright: © 2021 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v25i0.2671 http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 setting the scene: the selection of self-checkout over traditional checkout 3 theoretical review 3.1 dual-system theories 3.2 habits 3.3 conceptual framework 3.4 research approach 4 empirical study 4.1 data collection 4.2 data analysis 5 findings 5.1 habitual traditional checkout users 5.2 habitual sco users 5.3 situational users 5.4 drifting users 6 discussion 6.1 theoretical implications 6.2 managerial implications 6.3 limitations and avenues for future research references paper title (paper title style) australasian journal of information systems li et al. 2023, vol 27, research article investigating mmo games 1 investigating the moderating effect of massively multiplayer online (mmo) games on the correlation between flow and game addiction: a meta-analysis mengyuan li curtin university, perth, australia wendy hui singapore institute of technology, singapore wendy.hui@singaporetech.edu.sg torsten reiners curtin university, perth, australia abstract the flow theory of addiction suggests that the psychological flow state and addiction are positively correlated. however, based on 23 empirical studies involving 6,772 subjects, our meta-analysis shows that this relationship is significantly weakened in massively multiplayer online (mmo) games. our results suggest that game genre, which is often overlooked in existing theories, warrants more attention in future research on game addiction. for game developers, our results suggest that it is particularly important for non-mmo games to provide a good match between game challenge and player skill level, which is essential to the flow experience. as mmo games have been reported to be more addictive than non-mmo games, future research should also investigate the unique characteristics of mmo games that cause addiction in this game genre. keywords: game addiction, flow, immersion, social interaction, massively multiplayer online games. 1 introduction game addiction has become a serious social problem in many modern societies and has recently been recognized as a mental disorder by the who (2018). a recent systematic review and meta-analysis suggests that 3.3% of gamers worldwide suffer from game addiction (kim et al., 2022). during the covid-19 pandemic, gaming hours increased among children and adolescents lacking parental support, as they turned to gaming as a means to cope with social isolation (han et al.., 2022). governments around the globe have adopted various policy measures to tackle the problem, including the establishment of dedicated counselling and treatment centres (király et al., 2018), the implementation of curfews to ban young children from playing games during certain hours of the day (bbc, 2019), and the introduction of regulations requiring game developers to include in-game anti-addiction features (zhan & chan, 2012). within the academic community, researchers from psychology (e.g., hilgard et al., 2013) and healthcare (e.g. kim et al., 2016) have endeavoured to address the problem via continual research efforts aimed at understanding the psychological and cognitive mechanisms that lead to addictive behaviours. is researchers (e.g., khang et al., 2013; snodgrass et al., 2013; lee et al., 2020) tend to focus on games as hedonic systems, and seek to identify antecedents of addiction in these systems. understanding the factors underlying game addiction is critical for australasian journal of information systems li et al. 2023, vol 27, research article investigating mmo games 2 promoting public health and can inform the development of effective interventions and treatments for individuals who are struggling with game addiction. while various authors have summarized the important factors that contribute to game addiction (juthamanee & gunawan, 2021; karmakar, 2020; kuss & griffiths, 2012), the lack of coordinated research efforts from diverse disciplines has made it difficult to integrate the knowledge accumulated thus far. in particular, the contextual factors that could support or reject existing theories represent a research gap that needs attention. for example, na et al. (2017) identify game genre as an underresearched factor and show that the statistical significance of the same factors differs significantly across game genres, implying a need for dedicated research efforts for different game genres. the objective of this study is to highlight the need to consider game genre in game addiction theory development. as we shall explain in the literature review section, the role of game genre is often overlooked in the literature on game addiction. however, understanding the motivational factors behind game addiction requires the consideration of a host of factors, including the game genre concerned (hilgard et al 2013). using a meta-analysis, we show that studies focusing on mmo games tend to report a weaker correlation between flow and game addiction than studies on other game types. this weakened correlation suggests that the flow theory of addiction may be more effective in explaining game addiction in non-mmo games than in mmo games. therefore, a general theory of game addiction should incorporate game genre into its development. this study highlights the complex nature of game addiction and suggests that there may be important differences in the factors that contribute to addiction across different game genres. for instance, the motivational factor "achievement" may have a smaller effect in mmo games than in other game types. this finding implies that a "one-size-fits-all" approach to game addiction treatment may not be effective, as different game genres may require different approaches. understanding the differences in the formation of game addiction in different game genres can help game developers design game features that are engaging while avoiding elements that encourage excessive play. with this study as a foundation, we hope to inspire further research into the pivotal role that game genres play in the development of game addiction. in the field of is, flow theory is found to be particularly useful for decisions involving the voluntary choice (bölen, et al., 2021). it has been applied in many different contexts including technology acceptance and use (kunz & santomier, 2019; naglis & bhatiasevi, 2019; lin, et al., 2020), online consumer behavior (kaur, et. al., 2016) and gamification (behl & pereira, 2021; krath, et al., 2021). however, as pointed out by valinatajbahnamiri & siahtiri (2020), the identification of distractors and inhibitors of flow remains a research gap in the existing literature of is, marketing and psychology; the authors have suggested individual characteristics to be a potential inhibitor. we contribute to this conversation in is research by suggesting that the computer-mediated environment itself (e.g., game genre) can be a potential inhibitor. the remainder of this paper is organized as follows. section 2 critically reviews the relevant literature. section 3 describes our search process, inclusion and exclusion criteria, and the final data set. section 4 describes the processes of moderator detection and confirmation. section 5 discusses the theoretical and practical implications of our findings. finally, section 6 summarizes our findings and identifies a few research directions. australasian journal of information systems li et al. 2023, vol 27, research article investigating mmo games 3 2 literature review 2.1 game addiction the who (2018) defines game addiction as a pattern of (i) gaming behaviour (digital or video gaming behaviour) characterized by impaired control over gaming, (ii) increasing priority given to gaming over other activities to the extent that gaming takes precedence over other interests and daily activities, and (iii) continuation or escalation of gaming despite the occurrence of negative consequences. other terms that have been used to refer to game addiction include excessive use of online games (kim & kim, 2010; lemmens et al., 2009; wan & chiou, 2006; wei et al., 2012) and problematic play behaviour (seay & kraut, 2007; snodgrass et al., 2013). in this paper, we use the who’s definition of game addiction because it is a commonly used definition and it is applicable to either online or offline games; this helps us ensure that we can include a sufficient number of publications for our meta-analysis to detect a moderating effect. negative impacts of game addiction include decreased academic performance in children (chiu et al., 2004), decreased work efficiency in adults (liu, 2011; wu, 2013), psychological distress (kim et al., 2016; king & delfabbro, 2016; son et al., 2013), depression (brunborg et al., 2014; han et al., 2010; young & rogers, 1998), social anxiety (van rooij et al., 2011; wang et al., 2019), and a higher likelihood of general health problems (mentzoni et al., 2011; yen et al., 2017). hence, understanding the factors that contribute to game addiction has important practical implications. 2.2 explaining game addiction flow is one of the most frequently discussed factors contributing to game addiction (e.g., chou and ting, 2003; hull, et al., 2013; khang, et al., 2013). it is defined as people’s overall feeling when they are completely involved in activities and is considered the optimal psychological state (csikszentmihalyi, 1996, 2013; csikszentmihalyi, et al., 2005). hence, flow is often associated with an engaging and enjoyable experience. applying csikszentmihalyi’s theory in the context of games, there are three phases during game progress: anxiety, flow, and boredom. the transition between these phases depends on the player’s skills and the game challenge. people feel anxious when their skills are not good enough with respect to the game challenge. as they spend more time playing the game, their skill level increases, and they can reach the state of flow in which they feel a deep sense of achievement and joy. however, as their skill level increases, they may find that the game becomes too easy and feel bored, which leads them to look for more challenges to maintain the state of flow. as the game level or storyline progresses, players continue to shift among anxiety, flow, and boredom. this continuous behaviour of exchanging time for satisfaction may cause the experience of time distortion (csikszentmihalyi, 2014; rau et al., 2006). figure 1 shows the continuous cycle of anxiety, flow and boredom and how it relates to addiction. australasian journal of information systems li et al. 2023, vol 27, research article investigating mmo games 4 figure 1. flow theory of addiction empirical studies confirm that the flow state in games can cause game addiction (ballabio et al., 2017; chou & ting, 2003; hull et al., 2013; jung et al., 2015; liu & chang, 2016; park & hwang, 2009; seah & cairns, 2008; snodgrass et al., 2013; snodgrass et al., 2014; sun et al., 2015; xu et al., 2012; zhong & yao, 2013). however, the reported correlations vary significantly across studies. some suggest that the correlation is positive but small (khang et al., 2013) and insignificant (dauriat et al., 2011; oggins & sammis, 2012), whereas other studies report large and significant correlations (ballabio et al., 2017; jung et al., 2015; seah & cairns, 2008). in figure 2, we depict the positive empirical relationship between flow and game addiction as reported in most relevant empirical studies. figure 2. relationship between flow and addiction in existing empirical studies aside from flow theory, many other theories have been used to explain game addiction, including interpersonal theory (liu & kuo, 2007), presence theory (park & hwang, 2009), use and gratification theory (wu et al., 2010), social cognitive theory (zhang et al., 2016), and selfdetermination theory (bhagat et al., 2020). the antecedents of game addiction can be broadly divided into individual characteristics and it characteristics (e.g., ease of use, game design features, speed, graphics, and portability) (karmakar, 2020). juthamanee and gunawan (2021) further divide individual characteristics into sociodemographic characteristics (e.g., gender, age, and parental economic/income status), parent and family factors (e.g., parental neglect and family relationships), peer relationships and support, personality traits (e.g., extraversion, introversion, neuroticism, conscientiousness, and agreeableness), perceived enjoyment and perceived benefits. however, as evident by the various themes that have emerged from systematic reviews in information systems (karmakar, 2020) vs. clinical research (kuss & lopez-fernandez, 2016), researchers across disciplines tend to focus on different aspects of game addiction. the lack of coordination among efforts across disciplines and the rapid growth of the literature have made it challenging to integrate the existing knowledge. similarly, juthamanee and gunawan (2021) conclude their literature review with a call for flow game addiction australasian journal of information systems li et al. 2023, vol 27, research article investigating mmo games 5 future research to understand the linkages among the many identified contributing factors of game addiction. 2.3 the role of the game genre one way to integrate existing empirical results that may seem contradictory is to identify potential contextual or moderating factors (e.g., de jong et al., 2016). for example, na et al. (2017) show that the statistical significance of antecedents of game addiction differs across game genres, suggesting that existing theories may not explain game addiction equally effectively across genres. mmo games are characterized by their persistent virtual worlds, advancement, and social interaction and are often described as the most addictive among all game genres (billieux et al., 2015). it is no surprise that this game genre has received the most research attention; dedicated research efforts on mmo games include studies by snodgrass et al. (2013), liu and chang (2016), oggins and sammis (2012), and wan and chiou (2006). however, establishing the moderating effects of game genre requires direct empirical comparisons between game genres, which are rare in the literature. na et al. (2017), lee et al. (2006), elliott, golub, ream, and dunlap (2012), and elliott, ream, mcginsky, and dunlap (2012) empirically show that certain game genres are more likely to be associated with game addiction. on the other hand, mathews et al. (2019) show that the relationship between adhd symptom severity and addiction severity does not depend on game genre. 2.4 research gap and research question in previous research, the role of game genre in game addiction has often been overlooked. empirical studies on the effect of game genre on addiction are scarce and have yielded conflicting results. in this study, to evaluate the overall moderating effects of game genre based on existing empirical evidence, a meta-analysis on the reported correlation between flow and game addiction is conducted for studies that focus on mmo games vs. those that do not. the relationship between flow and game addiction is chosen to investigate the moderating role of game genre. hence, the research question we aim to answer in this study is as follows: "is the correlation between flow and game addiction affected by whether the game concerned is an mmo game?" this study is exploratory in nature (anello and fleiss 1995). for the purpose of statistical testing for the significance of the moderating effect of mmo games, we develop the following working hypothesis: “mmo games moderate the correlation between flow and game addiction.” figure 3. the moderating role of mmo games flow game addiction mmo games australasian journal of information systems li et al. 2023, vol 27, research article investigating mmo games 6 figure 3, which is an extension of figure 2, graphically presents our working hypothesis. since our goal is to test our working hypothesis, which is new, the use of meta-analysis is appropriate, in accordance with higgins and green (2011). the decision to focus on the flow theory of game addiction is driven mainly by two reasons. first, the detection and confirmation of a moderating factor requires a reasonable number of original studies. flow theory has received much research attention in the past, and thus, the statistics required for a meta-analysis are widely reported. second, flow theory emphasizes the importance of matching game challenges with player skills. when there is a good match between the two, the player experiences a great sense of achievement and reaches the flow state. hence, flow theory provides a theoretical basis for the achievement motive of game play (yee 2006a), which is applicable across game genres, including mmo games and non-mmo games (hilgard et al 2013). the extent to which flow theory can explain game addiction in various game genres will have implications for game developers in terms of game design. 3 methodology in this study, we first conducted a literature search to identify relevant papers on this topic. a number of inclusion and exclusion criteria were used to identify a final set of 24 original studies for the meta-analysis. one of the studies was identified as an outlier and was excluded from further analysis1 . next, we divided the studies into two groups: (1) those that focus on mmo games and (2) those that do not. we then compared the reported correlations between the psychological flow state and game addiction in these two groups using meta-analytic techniques. the details of our methodology are provided in the following subsections. 3.1 meta-analysis to evaluate our proposition, we conducted a meta-analysis, which is a quantitative approach to summarize existing empirical studies on a given topic. typically, a meta-analysis involves correcting for measurement errors, computing an appropriate weighting for each study, and minimizing other possible biases in the studies (dennis et al., 2001; moed & halevi, 2015; sharma & yetton, 2003; wu & lederer, 2009). the meta-analytic approach has been used to investigate research questions in many areas (glass, 1976; hunter & schmidt, 1990), including the detection of moderating effects (sabherwal et al., 2006; sharma & yetton, 2003). in the field of information systems, meta-analysis has been used to test theories regarding technology acceptance (king & he, 2006; ma & liu, 2004; schepers & wetzels, 2007; j. wu & lederer, 2009), decision support system implementation (alavi & joachimsthaler, 1992), enterprise system implementation (berente et al., 2019), group support systems (dennis et al., 2001), turnover of information technology professionals (joseph et al., 2007), utilitarian hedonic and dual-purposed information systems (wu & lu, 2013), gaming consoles (goode et al., 2017), and information technology business strategic alignment (brinckmann et al., 2019; desrochers et al., 2017; gerow et al., 2014). 3.2 the sample we conducted our literature search in august 2020. to ensure a good coverage, we made use of scopus and other major databases, including google scholar, academic search premier, and ieee, to identify empirical studies from journals, books, conference proceedings, and 1 we have repeated all analyses in this paper with the outlier included, and our results remain robust. australasian journal of information systems li et al. 2023, vol 27, research article investigating mmo games 7 postgraduate dissertations. such broad coverage can enhance the credibility of a meta-analysis by reducing potential publication bias (sharma & yetton, 2003) and is in line with recent metaanalytic research practice (e.g., bhardwaj, et al., 2023; verma, et al., 2023). our main search was conducted using the following groups of keywords: 1) immersion, game addiction, and correlation and 2) flow, game addiction, and correlation. google scholar yielded 552 (first group) and 873 results (second group). the same set of keywords was used on three other databases (scopus, academic search premier, and ieee) for more comprehensive coverage of publications, resulting in an additional 126 papers. hence, the combined search resulted in 1,551 papers in total. to identify eligible papers for our meta-analysis, we used the following criteria: 1. the paper was written in english. 2. the relationship between game addiction (the dependent variable) and immersion or flow (the independent variable) was investigated. 3. the independent variable (immersion or flow) was measured using a comparable measurement scale. since the terms “flow,” “immersion,” and “escapism” are sometimes used interchangeably (hagström & kaldo, 2014; kuss et al., 2012; oggins & sammis, 2012; snodgrass et al., 2014; wu et al., 2013), we examined the questions for these terms and incorporated them if their scales were derived from the scale in the framework of yee (2006b). similarly, we ensured that the dependent variable (addiction or problematic play) was measured by a comparable measurement. based on the definition in the “game addiction” section, two measurement scales were used in this study. these scales are from the dsm-iv (1994) and internet addiction test (iat). both scales measure excessive playing of games (griffiths, 2005; young & rogers, 1998).2 4. the correlation between the independent and dependent variables was reported. 5. the sample size was reported. it is widely understood that published papers have higher data quality and more robust results than unpublished papers. however, phd/doctoral dissertations and conference papers were also included to minimize publication bias. since master’s theses may not be assessed by a thesis committee or external examiners, they were not included in our meta-analysis. of the 1,551 papers, 1,405 met criterion 1 regarding the english language. for each of these papers, we inspected the proposed model or correlations to determine whether they studied the relationship between flow and game addiction. one hundred papers investigated the 2 since we reviewed each paper we considered, we effectively eliminated the jingle fallacy, which occurs when constructs of the same name measure different latent constructs (larson and bong 2016). for the flow construct, as we noted that the terms “flow,” “immersion,” and “escapism” are sometimes used interchangeably, these terms were also considered and they were included if their scales were derived from yee (2006b). this alleviated the jangle fallacy to some extent. however, we did not conduct a thorough investigation on all the construct names used to refer to the psychological flow state. some papers using a less common construct name might have been excluded. similarly, papers using a different construct name to refer to addiction were also excluded. although we did not fully address the jangle fallacy, we believe that a construct’s name has no effect on its relationships with other constructs. the exclusion of these papers affects the sample size of the meta-analysis, but it does not introduce a systematic bias to the estimated effect size. australasian journal of information systems li et al. 2023, vol 27, research article investigating mmo games 8 relationship between flow and game addiction (criterion 2). in 45 of these papers, both flow and game addiction were measured using comparable measurements (criterion 3). thirtythree of these papers contained all the required information, i.e., correlations (criterion 4) and sample size (criterion 5). figure 4. prisma diagram (page et al., 2021) a general meta-analysis principle is the requirement of independence among studies (ma & liu, 2004). for studies that used the same data set, only one of the studies was considered in our meta-analysis. the prisma diagram for our study is presented in figure 4. our final data set consists of 24 studies3 , which are shown in table 1; for the full citations of the papers, 3 according to “advances in meta-analysis” (pigott, 2012) and researchers’ discussions of sample size for meta-analysis (valentine, et al., 2010), a sample size of 2 can be sufficient for conducting a metaanalysis. we conducted a brief investigation of other meta-analyses and identified a number of studies with a small sample size (e.g., alavi and joachimsthaler 1992; joseph et al., 2007). alavi and joachimsthaler (1992) analyse 43 constructs in 33 studies. the most frequently studied construct had a sample size of 11. our data contain 24 studies, which all focus on one relationship. the studies combined included 6,899 individual subjects; this number is comparable to that in the work by joseph et al. (2007). records identified from google scholar: (n = 1425) records identified from other databases (scopus, academic search premier, and ieee): (n = 126) total n = 1,551 records removed before screening: duplicate records (n = 82) articles written in non-english language (n = 37) full text not retrievable (n = 27) records screened (n = 1405) records excluded reason 1: qualitative research reason 2: desired variables and comparable measurements not studied (n = 1360) reports sought for retrieval (n = 45) reports not retrieved: reason: correlations and sample sizes not available (n = 55) reports assessed for eligibility (n = 33) reports excluded: (n = 9) reason: same sample set used and outcomes of study not available studies included in review (n = 24) id en tif ic at io n el ig ib ili ty in cl ud ed sc re en in g australasian journal of information systems li et al. 2023, vol 27, research article investigating mmo games 9 please refer to appendix a. the final sample included 21 journal papers and 3 conference articles.4 in table 1, an underlined number in the “ri corrected for measurement error” column indicates that covariance-based sem was used for path analysis in the study; therefore, it is assumed that the reported correlations have already been corrected for measurement errors. study ni ri construct reliability (flow) construct reliability (addiction) ri corrected for measurement error mmo ballabio et al. (2017) 327 0.79 0.69 0.80 0.99 0 chou and ting (2003) 395 0.60 0.91 0.85 0.60 0 dauriat et al. (2010) 696 0.02 0.80 0.76 0.03 1 hagstrom and kaldo (2014) 141 0.34 0.66 0.75 0.48 1 hull et al. (2013) 110 0.09 0.80 0.85 0.11 0 jung et al. (2015) 42 0.62 0.81 0.93 0.72 0 kardefelt-winther (2014) 702 0.47 0.73 0.72 0.65 1 khan and muqtadir (2016) 357 0.39 0.75 0.75 0.52 1 khang et al. (2013) 290 0.28 0.87 0.85 0.32 0 kuss et al. (2012) 265 0.30 0.69 0.78 0.41 1 kwok and khoo (2013) 128 0.35 0.86 0.77 0.43 1 lee et al. (2020) 406 0.20 1 0.93 0.21 1 li et al. (2013) 278 0.27 0.81 0.81 0.27 1 liu and chang (2016) 215 0.58 0.95 0.96 0.60 1 oggins and sammis (2012) 438 0.08 0.80 0.89 0.10 1 park and hwang (2009) 133 0.46 0.9 0.88 0.46 0 seah and cairns (2008) 33 0.76 0.80 0.85 0.93 0 snodgrass et al. (2013) 252 0.21 0.60 0.94 0.28 1 snodgrass et al. (2014) 133 0.43 0.80 0.94 0.50 1 sun et al. (2015) 234 0.58 0.89 0.88 0.66 0 wan and chiou (2006)* 127 -0.38 0.89 0.92 -0.42 1 wu et al. (2013) 357 0.39 0.72 0.91 0.48 1 xu et al. (2012) 623 0.39 0.86 0.82 0.46 0 zhong and yao (2013) 217 0.28 0.73 0.935 0.28 0 table 1. the dataset note. * this study is identified as an outlier. it is excluded from our analysis. all other correlations were corrected using the corresponding construct reliability statistics reported in the original papers, in accordance with hunter and schmidt (2004). missing construct reliability statistics were replaced by the average reliability statistics in accordance with hunter and schmidt (2004, p. 121), and they are presented in boldface in table 1. we note that all the reported correlations were positive, except for wan and chiou (2006). therefore, we considered it as an outlier and removed it from our analysis. the forest plot in hence, these 24 papers present a comprehensive coverage of existing empirical research and exhibit significant research value. 4 the number of phd/doctoral publications was small and none of them survived our inclusion and exclusion criteria. australasian journal of information systems li et al. 2023, vol 27, research article investigating mmo games 10 figure 5 was generated based on the uncorrected correlations using the metafor package of r. it graphically presents the effect sizes of the original studies and their confidence intervals. the r script used to perform the meta-analysis in this paper can be found in appendix b. figure 5. forest plot based on uncorrected correlations the funnel plot is a plot of precision on the vertical axis against the effect size on the horizonal axis. in the absence of publication bias and heterogeneity between studies, the plot should look like an inverted funnel. the funnel plot in figure 6 (below) was generated based on the uncorrected correlations using the metafor package of r. as shown, quite a few studies fell outside the triangular 95% confidence region. in section 4, we show that there is significant heterogeneity between studies. specifically, studies focusing on mmo games tend to have a lower correlation between flow and addiction than studies that do not focus on mmo games. the rank correlation test for funnel plot asymmetry yielded kendall’s τ = =-0.0794 and p = 0.5971, suggesting that there may not be any publication bias. 4 analysis following hwang and schmidt (2011), we used multiple methods (i.e., q test for heterogeneity, credibility intervals, and the schmidt and hunter 75% rule) to investigate whether there is sufficient heterogeneity to suggest that a moderator is in operation. since the practice of measurement error correction is controversial (borsboon & mellenbergh 2002), our analysis was based on both the corrected and uncorrected correlations. to confirm the moderating effect of mmos, we performed a meta-regression analysis and a subgroup australasian journal of information systems li et al. 2023, vol 27, research article investigating mmo games 11 analysis based on the random effects model. the analyses based on both corrected and uncorrected correlations yielded the same inferences. figure 6. funnel plot based on uncorrected correlations 4.1 moderator detection 4.1.1 method 1: q test for heterogeneity fisher z -transformed correlations were used to perform the q test for heterogeneity. based on the uncorrected correlations, the q statistic was 501.60, whereas based on the corrected correlations, the q statistic was 2061.22. in both cases, the test results suggest that there is significant heterogeneity between studies. hence, there are likely to be moderators affecting the correlation between flow and addiction. q statistic df p uncorrected correlations 426.37 22 < 0.0001 corrected correlations 1942.06 22 < 0.0001 table 2. q tests for heterogeneity 4.1.2 method 2: credibility interval we used the formulas from hunter and schmidt (2004) to compute our credibility intervals. for our analysis based on uncorrected correlations, the 80% credibility interval for the true correlation was 0.3483±(1.28)(0.1931), and the 95% credibility interval was 0.3483±(1.96)(0.1931). for our analysis based on corrected correlations, the 80% credibility australasian journal of information systems li et al. 2023, vol 27, research article investigating mmo games 12 interval was 0.4110±(1.28)(0.2249), and the 95% credibility interval was 0.4110±(1.96)(0.2249). table 3 presents these results. the actual calculations can be found in appendix c. mean correlation 𝝆𝝆� 80% credibility interval 95% credibility interval uncorrected correlations 0.3483 [0.1012, 0.5954] [-0.0301, 0.7267] corrected correlations 0.4110 [0.1231, 0.6988] [-0.0298, 0.8517] table 3. credibility intervals table 3 shows that all credibility intervals were quite large. furthermore, the 95% credibility intervals included zero, suggesting that a moderating factor is likely in operation (whitener, 1990). 4.1.3 method 3: schmidt and hunter’s 75% rule based on the calculations in appendix c, the variance due to sampling error (0.0026) and the variance due to measurement error (0.0010) combined represented less than 75% of the variance of the observed correlations (0.0399), i.e., (0.0026+0.0010)/0.0399<0.75. according to schmidt and hunter’s 75% rule, we cannot rule out the presence of a moderator (hunter & schmidt 2004, 401).5 4.2 moderator confirmation 4.2.1 method 1: meta-regression the r package metafor was used to perform the meta-regression analysis. table 4 shows the results of the analysis. the dependent variable was the fisher z-transformed uncorrected correlations. table 5 presents the results based on corrected correlations. table 4 suggests that the mmo game genre is a significant moderator of the relationship between flow and addiction, whereas table 5 suggests that it is marginally significant. 𝜷𝜷 standard error z p intercept 0.5575 0.0773 7.2095 <0.0001*** mmo -0.2308 0.1013 -2.2788 0.0227* table 4. metaregression results based on uncorrected correlations note. *** p ≤ 0.001, * p ≤ 0.05 𝜷𝜷 standard error z p intercept 0.8360 0.1681 4.9741 <0.0001*** mmo -0.4178 0.2227 -1.8763 0.0606+ table 5. metaregression results based on corrected correlations note. *** p ≤ 0.001, * p ≤ 0.05, + p ≤ 0.1 4.2.2 method 2: subgroup analysis to perform the subgroup analysis, we split the data into the mmo group and the non-mmo group and went through the same procedure in appendix c to compute the mean and 5 if we ignore measurement error, the percentage 0.0026/0.0399 is even smaller, i.e., we will reach the same conclusion. australasian journal of information systems li et al. 2023, vol 27, research article investigating mmo games 13 variance of the true correlation for each group. we then computed the t statistic for the difference in means between groups in accordance with aguinis, sturman, and pierce (2008): 𝑡𝑡 = |�̅�𝑟1 − �̅�𝑟2| �var(𝑟𝑟1) 𝑘𝑘1 + var(𝑟𝑟2) 𝑘𝑘2 the t test results and associated statistics are presented in table 6. as shown in the table, the mmo group was significantly different from the non-mmo group at the 0.05 level, regardless of whether the correlations were corrected for measurement error. group 1: mmo group 2: non-mmo t-statistic p uncorrected correlations mean:�̅�𝑟𝑖𝑖 0.2794 0.4735 2.660 0.0159* variance:var(𝑟𝑟𝑖𝑖) 0.0262 0.0331 # of studies:𝑘𝑘𝑖𝑖 13 10 corrected correlations mean:�̅�𝑟𝑖𝑖 0.3402 0.5366 2.3438 0.0323* variance:var(𝑟𝑟𝑖𝑖) 0.0379 0.0411 # of studies:𝑘𝑘𝑖𝑖 13 10 table 6. descriptive statistics of the study correlations between groups note. * p ≤ 0.05 to summarize, our meta-analysis, which incorporated multiple techniques, supports our hypothesis that mmo games moderate the relationship between flow and addiction. flow is less likely to lead to game addiction in mmo games than in non-mmo games. 5 implications 5.1 theoretical implications the moderating role of game genre in addiction theories is often overlooked in the literature. the few studies that investigate this role have yielded conflicting results on its significance. the present study employed a meta-analysis approach to consolidate existing empirical evidence and showed that mmo games have a moderating effect on the relationship between flow and game addiction. this result highlights the importance of accounting for game genre when developing theories related to game addiction. specifically, it suggests that some of the theories that have been proposed may not be equally applicable across different game genres. therefore, to develop a more comprehensive understanding of game addiction, it is crucial to consider how game genre may shape the factors that contribute to this phenomenon. by doing so, researchers and clinicians can better tailor interventions and treatments that are more effective in addressing problematic gaming behaviors in specific game genres, and ultimately, promote healthier gaming habits. specifically, we found that the moderating effect of mmo games is negative, i.e., mmo games tend to weaken the correlation between flow and game addiction6 . given that mmo games 6it is important to note that even when the mmo genre is considered, the relationship between flow and game addiction is still positive (e.g., dauriat et al., 2010; hagstrom and kaldo, 2014; kardefelt-winther; 2014). australasian journal of information systems li et al. 2023, vol 27, research article investigating mmo games 14 are often reported to be the most addictive among game genres (billieux et al., 2015), it is important to understand why the correlation is weakened. we note that flow theory can be used to explain the achievement motive behind game play (yee, 2006a). continuous game play allows a player to experience a repeated cycle of anxiety, flow, and boredom as the game level and the player’s skill increase (rau et al 2006). this progression in game creates a sense of achievement and motivates the player to continue to play. our results suggest that the achievement motive may not be very strong in mmo games compared to non-mmo games. based on billieux et al. (2013, 2015), we posit that problematic play in mmo games likely represents a maladaptive strategy to cope with negative affect (e.g., boredom, anxiety, dysphoria). this interpretation is consistent with psychguides.com (n.d.), where game addiction is divided into two categories: (1) standard video games, typically with a clear goal such as saving a princess, and (2) online multiplayer games, typically with no ending. it is suggested that the addiction in the first category is mostly driven by completing a mission or beating a high score or preset standard, in line with the achievement motive. on the other hand, addicted players in the second category tend to form virtual social networks with other players and use game play as a means to escape from reality and gain acceptance. more empirical evidence should be collected in the future to support this interpretation. as shown in table 7, the heterogeneity tests based on z-transformed correlations in each subgroup suggest that there could be other moderating factors affecting the correlation between flow and game addiction. this points to the need for future investigation of other relevant factors in the theory development for game addiction. group 1mmo group2 non mmo uncorrected correlations q (df = 12) = 152.9711 p < .0001*** q (df = 9) = 175.5068 p < .0001*** corrected correlations q (df = 12) = 293.7331 p < .0001*** q (df = 9) = 1369.9709 p < .0001 *** table 7. q tests for heterogeneity finally, as discussed in the literature review, flow theory has been applied in other is research problems to explain online consumer behaviour, technology acceptance and game-based learning (kaur, et. al., 2016; kunz & santomier, 2019; naglis & bhatiasevi, 2019; lin, et al., 2020; krath, et al., 2021). the results of this study highlight the computer-mediated environment (e.g., game genre) as a potential inhibitor of flow along side individual characteristics (valinatajbahnamiri & siahtiri 2020). 5.2 practical implications from the perspective of a game developer who wants to make addictive games, the stronger correlation between flow and game addiction observed for non-mmo games than for mmo games suggests that providing a good match between game challenge and player skill level, which is essential to the flow experience, is particularly important for non-mmo games. hence, it is important to ensure that the level of challenge can adapt to the player's skill level as it changes over time. such adaptation can avoid users having a boring experience (when the game is not challenging enough) or a frustrating experience (when the game is too challenging). that said, an engaging game experience can be addictive. balancing the pursuit of profit with the potential negative impacts on players' finances or mental health is a common ethical dilemma faced by game developers (amano-smerling 2021). guidelines such as the australasian journal of information systems li et al. 2023, vol 27, research article investigating mmo games 15 children’s code issued by the information commissioner’s office of the uk and the code of ethics for the game industry published by ethicalgames.org can be consulted to help game developers strike the right balance. on the other hand, the lower correlation between flow and game addiction observed for mmo games suggests that game developers may need to find other ways to engage players in mmo games. the results of this study do not provide insights into how this player engagement can be achieved. however, systematic reviews on game addiction (e.g., juthamanee & gunawan, 2021; karmakar, 2020; kuss & griffiths, 2012) have summarized various factors that contribute to game addiction. future empirical efforts should investigate which of these factors are more prominent in mmo games. 6 conclusion 6.1 summary the literature has not firmly established the moderating effects of game genre on game addiction. our study makes an incremental contribution to the literature by empirically demonstrating via a meta-analysis that mmo games significantly lower the correlation between flow and game addiction. after removing an outlier, our analysis covered 23 studies representing a total of 6,772 subjects. our results suggest that game genre should be considered in the development of a general theory of game addiction. 6.2 limitations our study has several limitations. first, while our aim is to empirically demonstrate that the mmo game genre moderates the relationship between flow and game addiction, we do not provide a theoretical explanation for why this effect exists. second, while mmo games are often perceived as more addictive than other genres (billieux et al., 2015), we do not investigate the psychological and cognitive mechanisms that lead to addiction in mmo games. third, the moderating effect we observe is based on results reported in existing correlational studies, and the research contexts of these studies may differ, leading to potential confounding factors. fourth, while there may be other moderating factors, as suggested by table 7, we do not identify or discuss them in this study. finally, although many existing correlational studies use structural equal models (sem) to analyse their data and report correlations between many other variables, we do not explore these relationships in our study. 6.3 future research based on the aforementioned limitations, several potential directions for future research could be pursued. first, it may be beneficial to develop a theoretical model to explain the weakened relationship between flow and addiction in mmo games. this model could help identify the unique characteristics of mmo games that may lead to this moderating effect. second, conducting experiments could strengthen the empirical evidence on the causal relationship between game features and game addiction, while also minimizing the impact of confounding factors. third, other potential moderating factors should be explored to improve the effectiveness of addressing game addiction in various contexts. fourth, a meta-analysis structural equation model (sem) could provide a more comprehensive summary of the relationships between different constructs that affect game addiction. finally, it is worth noting that using gap-spotting to formulate a research question is unlikely to generate significant research theories. as an alternative, we recommend problematization as a australasian journal of information systems li et al. 2023, vol 27, research article investigating mmo games 16 technique for constructing future research questions by challenging the assumptions underlying existing theories (sandberg & alvesson, 2011). references aguinis, h., sturman, m. c., & pierce, c. a. (2008). comparison of three meta-analytic procedures for estimating moderating effects of categorical variables. organizational research methods, 11(1), 9-34. doi: 10.1177/1094428106292896 alavi, m., & joachimsthaler, e. a. (1992). revisiting dss implementation research: a metaanalysis of the literature and suggestions for researchers. mis quarterly, 16(1), 95-116. amano-smerling, t. (2021). “pay to play: the ethics of video game economics,” viterbi conversations in ethics, 5(1). https://vce.usc.edu/volume-5-issue-1/pay-to-play-theethics-of-video-game-economics/ anello, c., & fleiss, j. l. (1995) exploratory or analytic meta-analysis: should we distinguish between them? journal of clinical epidemiology, 8(1), 109-116. doi: 10.1016/08954356(94)00084-4. ballabio, m., griffiths, m. d., urbán, r., quartiroli, a., demetrovics, z., & király, o. (2017). do gaming motives mediate between psychiatric symptoms and problematic gaming? an empirical survey study. addiction research & theory, 25(5), 397-408. doi:10.1080/16066359.2017.1305360 bbc (2019, november 6). video game addiction: china imposes gaming curfew for minors. https://www.bbc.com/news/world-asia-50315960 behl, a., & pereira, v. (2021). what’s behind a scratch card? designing a mobile application using gamification to study customer loyalty: an experimental approach. australasian journal of information systems, 25. berente, n., lyytinen, k., yoo, y., & maurer, c. (2019). institutional logics and pluralistic responses to enterprise system implementation: a qualitative meta-analysis. mis quarterly, 43(3), 873-902. bhagat, s., jeong, e. j., & kim, d. j. (2020). the role of individuals’ need for online social interactions and interpersonal incompetence in digital game addiction. international journal of human–computer interaction, 36(5), 449-463. doi:10.1080/10447318.2019.1654696 bhardwaj, s., rana, g. a., behl, a., & de caceres, s. j. g. (2023). exploring the boundaries of neuromarketing through systematic investigation. journal of business research, 154, 113371. billieux, j., deleuze, j., griffiths, m. d., & kuss, d. j. (2015). internet gaming addiction: the case of massively multiplayer online roleplaying games. in textbook of addiction treatment: international perspectives (pp. 1515-1525). doi:10.1007/978-88-470-5322-9_105 billieux, j., van der linden, m., achab, s., khazaal, y., paraskevopoulos, l., zullino, d., & thorens, g. (2013). why do you play world of warcraft? an in-depth exploration of selfreported motivations to play online and in-game behaviours in the virtual world of azeroth. computers in human behavior, 29(1), 103-109. doi:10.1016/j.chb.2012.07.021 australasian journal of information systems li et al. 2023, vol 27, research article investigating mmo games 17 bölen, m. c., calisir, h., & özen, ü. (2021). flow theory in the information systems life cycle: the state of the art and future research agenda. international journal of consumer studies, 45, 546-580. doi: 10.1111/ijcs.12641 borsboom, d., & mellenbergh, g. j. (2022) “true score, latent variables, and constructs: a comment on schmidt and hunter,” intelligence, 30, 505-514. brinckmann, j., dew, n., read, s., mayer-haug, k., & grichnik, d. (2019). of those who plan: a meta-analysis of the relationship between human capital and business planning. long range planning, 52(2), 173-188. brunborg, g. s., mentzoni, r. a., & frøyland, l. r. (2014). is video gaming, or video game addiction, associated with depression, academic achievement, heavy episodic drinking, or conduct problems? journal of behavioral addictions, 3(1), 27-32. doi: 10.1556/jba.3.2014.002 casula, m., rangarajan, n., shields, p. (2021). the potential of working hypotheses for deductive exploratory research. quality and quantity, 55(5), 1703-1725. doi: 10.1007/s11135-020-01072-9. chiu, s.-i., lee, j.-z., & huang, d.-h. (2004). video game addiction in children and teenagers in taiwan. cyberpsychology & behavior, 7(5), 571-581. doi:10.1089/cpb.2004.7.571 chou, t.-j., & ting, c.-c. (2003). the role of flow experience in cyber-game addiction. cyberpsychology & behavior, 6(6), 663-675. doi:10.1089/109493103322725469 csikszentmihalyi, m. (1996). flow and the psychology of discovery and invention (vol. 56). harper collins: new york, usa. csikszentmihalyi, m. (2013). flow: the psychology of happiness. random house: london, uk. csikszentmihalyi, m. (2014). toward a psychology of optimal experience. in flow and the foundations of positive psychology (pp. 209-226): springer: new york, usa. csikszentmihalyi, m., abuhamdeh, s., & nakamura, j. (2005). flow. in elliot a. j. & dweck, c. s. (eds.) handbook of competence and motivation. guilford publications: new york, usa. dauriat, f. z., zermatten, a., billieux, j., thorens, g., bondolfi, g., zullino, d., & khazaal, y. (2011). motivations to play specifically predict excessive involvement in massively multiplayer online role-playing games: evidence from an online survey. european addiction research, 17(4), 185-189. doi:10.1159/000326070 de jong, b. a., dirks, k. t., & gillespie, n. (2016). trust and team performance: a metaanalysis of main effects, moderators, and covariates. journal of applied psychology, 101(8), 1134. dennis, a. r., wixom, b. h., & vandenberg, r. j. (2001). understanding fit and appropriation effects in group support systems via meta-analysis. mis quarterly, 25(2), 167-193. doi:10.2307/3250928 desrochers, n., paul-hus, a., & pecoskie, j. (2017). five decades of gratitude: a meta-synthesis of acknowledgments research. journal of the association for information science and technology, 68(12), 2821-2833. doi:10.1002/asi.23903 australasian journal of information systems li et al. 2023, vol 27, research article investigating mmo games 18 elliott, l., golub, a., ream, g., & dunlap, e. (2012). video game genre as a predictor of problem use. cyberpsychology, behavior, and social networking, 15(3), 155-161. doi:10.1089/cyber.2011.0387 elliott, l., ream, g., mcginsky, e., & dunlap, e. (2012). the contribution of game genre and other use patterns to problem video game play among adult video gamers. international journal of mental health and addiction, 10(6), 948-969. doi: 10.1007/s11469-012-9391-4 gerow, j. e., grover, v., thatcher, j., & roth, p. l. (2014). looking toward the future of it– business strategic alignment through the past. mis quarterly, 38(4), 1159-1186. glass, g. v. (1976). primary, secondary, and meta-analysis of research. educational researcher, 5(10), 3-8. doi:10.3102/0013189x005010003 goode, s., hoehle, h., venkatesh, v., & brown, s. a. (2017). user compensation as a data breach recovery action: an investigation of the sony playstation network breach. mis quarterly, 41(3), 703-727. doi:10.25300/misq/2017/41.3.03 griffiths, m. (2005). a ‘components’ model of addiction within a biopsychosocial framework. journal of substance use, 10(4), 191-197. doi:10.1080/14659890500114359 hagström, d., & kaldo, v. (2014). escapism among players of mmorpgs—conceptual clarification, its relation to mental health factors, and development of a new measure. cyberpsychology, behavior, and social networking, 17(1), 19-25. doi:10.1089/cyber.2012.0222 han, d. h., hwang, j. w., & renshaw, p. f. (2010). bupropion sustained release treatment decreases craving for video games and cue-induced brain activity in patients with internet video game addiction. experimental and clinical psychopharmacology, 18(4), 297– 304. https://doi.org/10.1037/a0020023 han, t. s., cho, h., sung, d., & park m.-h. (2022). a systematic review of the impact of covid-19 on the game addiction of children and adolescents. frontiers in psychiatry, https://doi.org/10.3389/fpsyt.2022.976601 hedges, l. v., & olkin, i. (1985). statistical methods for meta-analysis. academic press: san diego, ca, usa. higgins, j. p. t., & green, s. (2011). cochrane handbook for systematic reviews of interventions, the cochrane collaboration. https://handbook-51.cochrane.org/chapter_9/9_1_3_why_perform_a_meta_analysis _in_a_review.htm hilgard, j., engelhardt, c. r., & bartholow, b. d. (2013). individual differences in motives, preferences, and pathology in video games: the gaming attitudes, motives, and experiences scales (games). frontiers in psychology, 4, 608. https://doi.org/10.3389/fpsyg.2013.00608 hull, d. c., williams, g. a., & griffiths, m. d. (2013). video game characteristics, happiness and flow as predictors of addiction among video game players: a pilot study. journal of behavioral addictions, 2(3), 145-152. doi:10.1556/jba.2.2013.005 hunter, j. e., & schmidt, f. l. (1990). dichotomization of continuous variables: the implications for meta-analysis. journal of applied psychology, 75(3), 334. doi:10.1037/00219010.75.3.334 australasian journal of information systems li et al. 2023, vol 27, research article investigating mmo games 19 hunter, j. e., & schmidt, f. l. (2004). methods of meta-analysis: correcting error and bias in research findings. sage: london, uk. hwang, m. i., & schmidt, f. l. (2011). assessing moderating effect in meta-analysis: a reanalysis of top management support studies and suggestions for researchers. european journal of information systems, 20(6), 693-702. joseph, d., ng, k.-y., koh, c., & ang, s. (2007). turnover of information technology professionals: a narrative review, meta-analytic structural equation modeling, and model development. mis quarterly, 31(3), 547-577. jung, d., kim, e.y., jeong, s.h., & hahm, b.j. (2016). higher immersive tendency in male university students with excessive online gaming. in: serino, s., matic, a., giakoumis, d., lopez, g., & cipresso, p. (eds.), pervasive computing paradigms for mental health. mindcare 2015. communications in computer and information science, vol. 604. springer: cham, switzerland. https://doi.org/10.1007/978-3-319-32270-4_16 juthamanee, s., & gunawan, j. (2021). factors related to internet and game addiction among adolescents: a scoping review. belitung nursing journal, 7(2), 62-71. doi:10.33546/bnj.1192 karmakar, i. (2020). a review of it addiction in is research. in amcis proceedings. 19. salt lake city, ut, usa august 13-15, 2020 (virtual). https://aisel.aisnet.org/amcis2020/adoption_diffusion_it/adoption_diffusion_it/19 kaur, p., dhir, a., chen, s. f., & rajala, r. (2016). understanding online regret experience using the theoretical lens of flow experience. computers in human behavior, 57, 230-239. krath, j., schürmann, l., & von korflesch, h. f. (2021). revealing the theoretical basis of gamification: a systematic review and analysis of theory in research on gamification, serious games and game-based learning. computers in human behavior, 125, 106963. kunz, r. e., & santomier, j. p. (2019). sport content and virtual reality technology acceptance. sport, business and management: an international journal, 10(1), 83–103. khang, h., kim, j. k., & kim, y. (2013). self-traits and motivations as antecedents of digital media flow and addiction: the internet, mobile phones, and video games. computers in human behavior, 29(6), 2416-2424. doi:10.1016/j.chb.2013.05.027 kim, h. s., son, g., roh, e. b., ahn w. y., kim, j., shin, s. h., chey, j., choi, k. h. (2022). prevalence of gaming disorder: a meta-analysis. addictive behaviors, 126, 107183. doi: 10.1016/j.addbeh.2021.107183. kim, m. g., & kim, j. (2010). cross-validation of reliability, convergent and discriminant validity for the problematic online game use scale. computers in human behavior, 26(3), 389-398. doi:10.1016/j.chb.2009.11.010 kim, n. r., hwang, s. s.-h., choi, j.-s., kim, d.-j., demetrovics, z., király, o., & youn, h. c. (2016). characteristics and psychiatric symptoms of internet gaming disorder among adults using self-reported dsm-5 criteria. psychiatry investigation, 13(1), 58. doi:10.4306/pi.2016.13.1.58 king, d. l., & delfabbro, p. h. (2016). the cognitive psychopathology of internet gaming disorder in adolescence. journal of abnormal child psychology, 44(8), 1635-1645. australasian journal of information systems li et al. 2023, vol 27, research article investigating mmo games 20 king, w. r., & he, j. (2006). a meta-analysis of the technology acceptance model. information & management, 43(6), 740-755. doi:10.1016/j.im.2006.05.003 király, o., griffiths, m. d., king, d. l., lee, h.-k., lee, s.-y., bányai, f., & demetrovics, z. (2018). policy responses to problematic video game use: a systematic review of current measures and future possibilities. journal of behavioral addictions, 7(3), 503-517. doi:10.1556/2006.6.2017.050 kuss, d. j., & griffiths, m. d. (2012). internet gaming addiction: a systematic review of empirical research. international journal of mental health and addiction, 10(2), 278-296. kuss, d. j., & lopez-fernandez, o. (2016). internet addiction and problematic internet use: a systematic review of clinical research. world journal of psychiatry, 6(1), 143. doi:10.5498/wjp.v6.i1.143 kuss, d. j., louws, j., & wiers, r. w. (2012). online gaming addiction? motives predict addictive play behavior in massively multiplayer online role-playing games. cyberpsychology, behavior, and social networking, 15(9), 480-485. doi:10.1089/cyber.2012.0034 larsen, k. r., & bong, c. h. (2016). a tool for addressing construct identity in literature reviews and meta-analysis. mis quarterly, 40(3), 529-552. lee, m.-s., ko, y.-h., song, h.-s., kwon, k.-h., lee, h.-s., nam, m., & jung, i.-k. (2006). characteristics of internet use in relation to game genre in korean adolescents. cyberpsychology & behavior, 10(2), 278-285. doi:10.1089/cpb.2006.9958 lemmens, j. s., valkenburg, p. m., & peter, j. (2009). development and validation of a game addiction scale for adolescents. media psychology, 12(1), 77-95. doi:10.1080/15213260802669458 lin, j. b., lin, s. z., turel, o., & xu, f. (2020). the buffering effect of flow experience on the relationship between overload and social media users' discontinuance intentions. telematics and informatics, 49, 101374. liu, c.-c., & chang, i.-c. (2016). model of online game addiction: the role of computermediated communication motives. telematics and informatics, 33(4), 904-915. doi:10.1016/j.tele.2016.02.002 liu, c.-y., & kuo, f.-y. (2007). a study of internet addiction through the lens of the interpersonal theory. cyberpsychology & behavior, 10(6), 799-804. doi:10.1089/cpb.2007.9951 liu, e. z. f. (2011). avoiding internet addiction when integrating digital games into teaching. social behavior and personality: an international journal, 39(10), 1325-1335. doi:10.2224/sbp.2011.39.10.1325 ma, q., & liu, l. (2004). the technology acceptance model: a meta-analysis of empirical findings. journal of organizational and end user computing (joeuc), 16(1), 59-72. doi:10.4018/joeuc.2004010104 mathews, c. l., morrell, h. e., & molle, j. e. (2019). video game addiction, adhd symptomatology, and video game reinforcement. the american journal of drug and alcohol abuse, 45(1), 67-76. australasian journal of information systems li et al. 2023, vol 27, research article investigating mmo games 21 mentzoni, r. a., brunborg, g. s., molde, h., myrseth, h., skouverøe, k. j. m., hetland, j., & pallesen, s. (2011). problematic video game use: estimated prevalence and associations with mental and physical health. cyberpsychology, behavior, and social networking, 14(10), 591-596. doi:10.1089/cyber.2010.0260 moed, h. f., & halevi, g. (2015). multidimensional assessment of scholarly research impact. journal of the association for information science and technology, 66(10), 1988-2002. na, e., choi, i., lee, t.-h., lee, h., rho, m. j., cho, h., & kim, d.-j. (2017). the influence of game genre on internet gaming disorder. journal of behavioral addictions, 6(2), 248-255. doi:10.1556/2006.6.2017.033 naglis, m., & bhatiasevi, v. (2019). why do people use fitness tracking devices in thailand? an integrated model approach. technology in society, 58, 101146. ng, b. d., & wiemer-hastings, p. (2005). addiction to the internet and online gaming. cyberpsychology & behavior, 8(2), 110-113. doi:10.1089/cyber.2010.0260 oggins, j., & sammis, j. (2012). notions of video game addiction and their relation to selfreported addiction among players of world of warcraft. international journal of mental health and addiction, 10(2), 210-230. doi:10.1007/s11469-010-9309-y page, m. j., mckenzie, j. e., bossuyt, p. m., boutron, i., hoffmann, t. c., mulrow, c. d., & brennan, s. e. (2021). the prisma 2020 statement: an updated guideline for reporting systematic reviews. bmj, 372. doi:10.1136/bmj.n71 park, s., & hwang, h.s. (2009). understanding online game addiction: connection between presence and flow. in: jacko, j.a. (eds) human-computer interaction. interacting in various application domains. hci 2009. lecture notes in computer science, vol 5613. springer: berlin, heidelberg, germany. https://doi.org/10.1007/978-3-642-02583-9_42 pigott, t. (2012). advances in meta-analysis. springer science & business media: new york, usa psychguides.com (n.d.). video game addiction symptoms, causes and effects. https://www.psychguides.com/behavioral-disorders/video-game-addiction/ (accessed 14 october 2021). rau, p.-l. p., peng, s.-y., & yang, c.-c. (2006). time distortion for expert and novice online game players. cyberpsychology & behavior, 9(4), 396-403. doi:10.1089/cpb.2006.9.396 sabherwal, r., jeyaraj, a., & chowa, c. (2006). information system success: individual and organizational determinants. management science, 52(12), 1849-1864. doi:10.1287/mnsc.1060.0583 sandberg, j., & alvesson, m. (2011). ways of constructing research questions: gap-spotting or problematization. organization, 18, 1, 23-44. schepers, j., & wetzels, m. (2007). a meta-analysis of the technology acceptance model: investigating subjective norm and moderation effects. information & management, 44(1), 90-103. doi:10.1016/j.im.2006.10.007 schmidt, f. l., caplan, j. r., bemis, s. e., decuir, r., dunn, l., & anton, l. (1979). the behavioral consistency method of unassembled examining (tm-79:21) office of personnel management, personnel resources and development center. washington, dc: usa. australasian journal of information systems li et al. 2023, vol 27, research article investigating mmo games 22 seah, m.-l., & cairns, p. (2008). from immersion to addiction in videogames. people and computers xxii culture, creativity, interaction, 22, 55-63. doi:10.14236/ewic/hci2008.6 seay, a. f., & kraut, r. e. (2007). project massive: self-regulation and problematic use of online gaming. in proceedings of the sigchi conference on human factors in computing systems (829838). april 2007, san jose, ca, usa. doi:10.1145/1240624.1240749 sharma, r., & yetton, p. (2003). the contingent effects of management support and task interdependence on successful information systems implementation. mis quarterly 27(4), 533-556. shields, p. m. & tajalli, h. (2006). intermediate theory: the missing link to successful student scholarship. journal of public affairs education, 12(3), pp. 313-334. snodgrass, j. g., dengah, h. f., lacy, m. g., & fagan, j. (2013). a formal anthropological view of motivation models of problematic mmo play: achievement, social, and immersion factors in the context of culture. transcultural psychiatry, 50(2), 235-262. doi:10.1177/1363461513487666 snodgrass, j. g., lacy, m. g., dengah ii, h. f., eisenhauer, s., batchelder, g., & cookson, r. j. (2014). a vacation from your mind: problematic online gaming is a stress response. computers in human behavior, 38, 248-260. doi:10.1016/j.chb.2014.06.004 son, d. t., yasuoka, j., poudel, k. c., otsuka, k., & jimba, m. (2013). massively multiplayer online role-playing games (mmorpg): association between its addiction, self-control and mental disorders among young people in vietnam. international journal of social psychiatry, 59(6), 570-577. doi:10.1177/0020764012445861 sun, y., zhao, y., jia, s.-q., & zheng, d.-y. (2015). understanding the antecedents of mobile game addiction: the roles of perceived visibility, perceived enjoyment and flow. pacis 2015 proceedings. 141. july 5-9, 2015, singapore. https://aisel.aisnet.org/pacis2015/141 valentine, j. c., pigott, t. d., & rothstein, h. r. (2010). how many studies do you need? a primer on statistical power for meta-analysis. journal of educational and behavioral statistics, 35(2), 215-247. doi:10.3102/1076998609346961 valinatajbahnamiri, m., & siahtiri, v. (2020). flow in computer-mediated environments: a systematic literature review. international journal of consumer studies, 45, 511-545. doi:10.1111/ijcs.12640 van rooij, a. j., schoenmakers, t. m., vermulst, a. a., van den eijnden, r. j., & van de mheen, d. (2011). online video game addiction: identification of addicted adolescent gamers. addiction, 106(1), 205-212. doi:10.1111/j.1360-0443.2010.03104.x verma, d., dewani, p. p., behl, a., pereira, v., dwivedi, y., & del giudice, m. (2023). a metaanalysis of antecedents and consequences of ewom credibility: investigation of moderating role of culture and platform type. journal of business research, 154, 113292. wan, c.-s., & chiou, w.-b. (2006). psychological motives and online games addiction: atest of flow theory and humanistic needs theory for taiwanese adolescents. cyberpsychology & behavior, 9(3), 317-324. doi:10.1089/cpb.2006.9.317 australasian journal of information systems li et al. 2023, vol 27, research article investigating mmo games 23 wang, h.-z., sheng, j.-r., & wang, j.-l. (2019). the association between mobile game addiction and depression, social anxiety, and loneliness. frontiers in public health, 7, 247. wei, h.-t., chen, m.-h., huang, p.-c., & bai, y.-m. (2012). the association between online gaming, social phobia, and depression: an internet survey. bmc psychiatry, 12(1), 92. doi:10.1186/1471-244x-12-92 whitener, e. m. (1990). confusion of confidence intervals and credibility intervals in metaanalysis. journal of applied psychology., 75(3), 315-321. doi:10.1037/0021-9010.75.3.315 who (2018). gaming disorder. retrieved from http://www.who.int/features/qa/gamingdisorder/en/ (last accessed on 14 september 2023). wu, j.-h., wang, s.-c., & tsai, h.-h. (2010). falling in love with online games: the uses and gratifications perspective. computers in human behavior, 26(6), 1862-1871. wu, j., & lederer, a. (2009). a meta-analysis of the role of environment-based voluntariness in information technology acceptance. mis quarterly, 33(2), 419-432. wu, j., & lu, x. (2013). effects of extrinsic and intrinsic motivators on using utilitarian, hedonic, and dual-purposed information systems: a meta-analysis. journal of the association for information systems, 14(3), 1. wu, p.-c. (2013). addictive behavior in relation to the happy farm facebook application. social behavior and personality: an international journal, 41(4), 539-553. doi:10.2224/sbp.2013.41.4.539 wu, t.-c., scott, d., & yang, c.-c. (2013). advanced or addicted? exploring the relationship of recreation specialization to flow experiences and online game addiction. leisure sciences, 35(3), 203-217. doi:10.1080/01490400.2013.780497 xu, z., turel, o., & yuan, y. (2012). online game addiction among adolescents: motivation and prevention factors. european journal of information systems, 21(3), 321-340. yee, n. (2006a). motivations for play in online games. cyberpsychology and behavior, 9(6), 772775. doi:10.1089/cpb.2006.9.772 yee, n. (2006b). the psychology of massively multi-user online role-playing games: motivations, emotional investment, relationships and problematic usage. in schroeder, r., & axelsson, a.s. (eds), avatars at work and play. computer supported cooperative work, vol 34. springer: dordrecht, the netherlands. https://doi.org/10.1007/1-40203898-4_9 yen, j.-y., liu, t.-l., wang, p.-w., chen, c.-s., yen, c.-f., & ko, c.-h. (2017). association between internet gaming disorder and adult attention deficit and hyperactivity disorder and their correlates: impulsivity and hostility. addictive behaviors, 64, 308-313. young, k. s., & rogers, r. c. (1998). the relationship between depression and internet addiction. cyberpsychology and behavior, 1(1), 25-28. doi:10.1089/cpb.1998.1.25 zhan, j. d., & chan, h. c. (2012). government regulation of online game addiction. communications of the association for information systems, 30(1), 13. doi:10.17705/1cais.03013 zhang, k. z., chen, c., zhao, s. j., & lee, m. k. (2016). failure to decrease the addictive use of information systems: an empirical investigation of smartphone game addiction. in australasian journal of information systems li et al. 2023, vol 27, research article investigating mmo games 24 proceedings of the 20th pacific asia conference on information systems (pacis 2016), june 27 july 1, 2016. chiayi, taiwan. https://aisel.aisnet.org/pacis2016/121 zhong, z.-j., & yao, m. z. (2013). gaming motivations, avatar-self identification and symptoms of online game addiction. asian journal of communication, 23(5), 555-573. doi:10.1080/01292986.2012.748814 appendix a list of studies included in the meta-analysis 1. ballabio, m., griffiths, m. d., urbán, r., quartiroli, a., demetrovics, z., & király, o. (2017). do gaming motives mediate between psychiatric symptoms and problematic gaming? an empirical survey study. addiction research & theory, 25(5), 397–408. 2. chou, t. j., & ting, c. c. (2003). the role of flow experience in cyber-game addiction. cyberpsychology & behavior, 6(6), 663–675. 3. dauriat, f. z., zermatten, a., billieux, j., thorens, g., bondolfi, g., zullino, d., & khazaal, y. (2011). motivations to play specifically predict excessive involvement in massively multiplayer online role-playing games: evidence from an online survey. european addiction research, 17(4), 185–189. 4. hagström, d., & kaldo, v. (2014). escapism among players of mmorpgs—conceptual clarification, its relation to mental health factors, and development of a new measure. cyberpsychology, behavior, and social networking, 17(1), 19–25. 5. hull, d. c., williams, g. a., & griffiths, m. d. (2013). video game characteristics, happiness and flow as predictors of addiction among video game players: a pilot study. journal of behavioral addictions, 2(3), 145–152. 6. jung, d., kim, e. y., jeong, s. h., & hahm, b. j. (2015, september). higher immersive tendency in male university students with excessive online gaming. in international symposium on pervasive computing paradigms for mental health (pp. 157–161). springer, cham. 7. kardefelt-winther, d. (2014). problematizing excessive online gaming and its psychological predictors. computers in human behavior, 3(1), 118–122. 8. khan, a., & muqtadir, r. (2016). motives of problematic and nonproblematic online gaming among adolescents and young adults. pakistan journal of psychological research, 31(1), 119–138. 9. khang, h., kim, j. k., & kim, y. (2013). self-traits and motivations as antecedents of digital media flow and addiction: the internet, mobile phones, and video games. computers in human behavior, 29(6), 2416–2424. 10. kwok, n. w. k., & khoo, a. (2011). gamers’ motivations and problematic gaming: an exploratory study of gamers in world of warcraft. international journal of cyber behavior, psychology and learning (ijcbpl), 1(3), 34–49. 11. kuss, d., louws, j., & wiers, r. w. (2012). online gaming addiction? motives predict addictive play behavior in massively multiplayer online role playing games. cyberpsychology, behavior and social networking, 15(9), 480–485. australasian journal of information systems li et al. 2023, vol 27, research article investigating mmo games 25 12. lee, z. w., cheung, c. m., & chan, t. k. (2020). understanding massively multiplayer online role‐playing game addiction: a hedonic management perspective. information systems journal, 31(2), 33-61. 13. li, d. d., liau, a. k., gentile, d. a., khoo, a., & cheong, w. d. (2013). construct and predictive validity of a brief mmo player motivation scale: cross-sectional and longitudinal evidence based on singaporean young gamers. journal of children and media, 7(3), 287–306. 14. liu, c. c., & chang, i. c. (2016). model of online game addiction: the role of computermediated communication motives. telematics and informatics, 33(4), 904– 915. 15. oggins, j., & sammis, j. (2012). notions of video game addiction and their relation to self-reported addiction among players of world of warcraft. international journal of mental health and addiction, 10(2), pp. 210–230. 16. park s., & hwang h.s. (2009). understanding online game addiction: connection between presence and flow. in j. a. jacko (ed.), human-computer interaction. interacting in various application domains. hci 2009. lecture notes in computer science (vol. 5613) (pp. 378-386). berlin, heidelberg: springer. 17. seah, m. l., & cairns, p. (2008, september). from immersion to addiction in videogames. in proceedings of the 22nd british hci group annual conference on people and computers: culture, creativity, interaction-volume(1), 55–63. 18. snodgrass, j. g., dengah, h. f., lacy, m. g., & fagan, j. (2013). a formal anthropological view of motivation models of problematic mmo play: achievement, social, and immersion factors in the context of culture. transcultural psychiatry, 50(2), 235–262. 19. snodgrass, j. g., lacy, m. g., dengah, h. f., eisenhauer, s., batchelder, g., & cookson, r. j. (2014). a vacation from your mind: problematic online gaming is a stress response. computers in human behavior, 3(8), 248–260. 20. sun, y., zhao, y., jia, s. q., & zheng, d. y. (2015). understanding the antecedents of mobile game addiction: the roles of perceived visibility, perceived enjoyment and flow. in pacific asia conference on information systems (pacis), p. 141. 21. wan, c. s., & chiou, w. b. (2006). psychological motives and online games addiction: a test of flow theory and humanistic needs theory for taiwanese adolescents. cyberpsychology & behavior, 9(3), 317–324. 22. wu, t.-c., scott, d., & yang, c.-c. (2013). advanced or addicted? exploring the relationship of recreation specialization to flow experiences and online game addiction. leisure sciences, 35(3), 203–217. 23. xu, z., turel, o., & yuan, y. (2012). online game addiction among adolescents: motivation and prevention factors. european journal of information systems, 21(3), 321–340. 24. zhong, z.-j., & yao, m. z. (2013). gaming motivations, avatar-self identification and symptoms of online game addiction. asian journal of communication, 23(5), 555–573. australasian journal of information systems li et al. 2023, vol 27, research article investigating mmo games 26 appendix b r script used in the meta-analysis library("metafor") meta_data = read.csv("c:\\working directory\\meta_data.csv") ### section 3 data visualizations ### # compute relevant outcome measures for correlations meta_data_corr = escalc(measure="cor", ri=ri, ni=ni, data=meta_data) # for analysis based on corrected correlations, use the following statement instead: # meta_data_corr = escalc(measure="cor", ri=corrected_ri, ni=ni, data=meta_data) # random effects model for correlations res_corr = rma(yi, vi, data=meta_data_corr) # forest plot for correlations (figure 5) forest(res_corr, slab = meta_data_corr$study, ilab = mmo, ilab.xpos =-1.5, cex=0.7, header = true, xlim=c(-3.5,2), digits=c(2,1)) text(-1.5, res_corr$k+2, "mmo", cex=0.7) # funnel plot (figure 6) funnel(res_corr, main="standard error") ranktest(res_corr) ### section 4 analysis ### # compute z-transformed correlations and associated outcome measures meta_data = escalc(measure="zcor", ri=ri, ni=ni, data=meta_data) # for analysis based on corrected correlations, use the following statement instead: # meta_data = escalc(measure="zcor", ri=corrected_ri, ni=ni, data=meta_data) # random effects model (showing the q test for heterogeneity in 4.1.1) res = rma(yi, vi, data=meta_data) res # inverse transform back to r predict(res, transf=transf.ztor) # meta-regression moderator = rma(yi, vi, mods = ~ factor(mmo), data=meta_data) moderator ### section 5 analysis ### meta_data = read.csv("c:\\working directory\\meta_data.csv") # subgroup heterogeneity test for non-mmo australasian journal of information systems li et al. 2023, vol 27, research article investigating mmo games 27 meta_data_non_mmo = subset(meta_data, mmo == 0) meta_data_non_mmo = escalc(measure="zcor", ri=ri, ni=ni, data=meta_data_non_mmo) # for analysis based on corrected correlations, use the following statement instead: # meta_data_non_mmo = escalc(measure="zcor", ri=corrected_ri, ni=ni, data=meta_data_non_mmo) res_non_mmo = rma(yi, vi, data=meta_data_non_mmo) res_non_mmo # subgroup heterogeneity test for mmo meta_data_mmo = subset(meta_data, mmo == 1) meta_data_mmo = escalc(measure="zcor", ri=ri, ni=ni, data=meta_data_mmo) # for analysis based on corrected correlations, use the following statement instead: # meta_data_mmo = escalc(measure="zcor", ri=corrected_ri, ni=ni, data=meta_data_mmo) res_mmo = rma(yi, vi, data=meta_data_mmo) res_mmo appendix c calculation of the credibility intervals we use the formulas from hunter and schmidt (2004) to compute our credibility intervals. for our analysis on uncorrected correlations, the population correlation is estimated by the sample size-weighted mean observed correlation (hunter and schmidt 2004, p. 81) as �̅�𝜌 = �̅�𝑟 = ∑𝑛𝑛𝑖𝑖𝑟𝑟𝑖𝑖 /∑𝑛𝑛𝑖𝑖 = 0.3483. the variance of the observed correlation is given by 𝑠𝑠𝑟𝑟2 = ∑𝑛𝑛𝑖𝑖(𝑟𝑟𝑖𝑖 − �̅�𝑟)2 /∑𝑛𝑛𝑖𝑖 = 0.0399. the variance due to sampling error can be approximated by (hunter and schmidt 2004, p. 87) 𝑠𝑠𝑒𝑒2 = 𝑘𝑘(1 − �̅�𝑟2)2/∑𝑛𝑛𝑖𝑖 = 0.0026, where 𝑘𝑘 is the number of original studies in the meta-analysis. in this case, 𝑘𝑘 = 23. hence, the standard deviation of the correlation corrected for sampling error is 𝑠𝑠𝜌𝜌 = √0.0399− 0.0026 = 0.1931. the 80% credibility interval is 0.3483 ± (1.28)(0.1931), and the 95% credibility interval is 0.3483 ± (1.96)(01931), as shown in table 3. for our analysis of corrected correlations, we first compute the mean compound attenuation factor �̅�𝐴 = 𝑎𝑎�𝑏𝑏� = �1 𝑘𝑘 ∑�𝛼𝛼𝑖𝑖� � 1 𝑘𝑘 ∑�𝛽𝛽𝑖𝑖�, where 𝛼𝛼𝑖𝑖 is the construct reliability of flow and 𝛽𝛽𝑖𝑖 is the construct reliability of addiction in study 𝑖𝑖 if the construct correlations reported have not already been corrected for measurement error. if the construction correlations have already been corrected for measurement error, as in most covariance-based sem analyses, 𝛼𝛼𝑖𝑖 or 𝛽𝛽𝑖𝑖 is simply equal to unity. for our dataset, �̅�𝐴 = australasian journal of information systems li et al. 2023, vol 27, research article investigating mmo games 28 0.8475. following the steps in hunter and schmidt (2004, p. 152), the estimated true correlation after correcting for measurement error is �̅�𝜌 = �̅�𝑟 �̅�𝐴 = 0.3483 0.8475 = 0.4110. let the standard deviations of �𝛼𝛼𝑖𝑖 and �𝛽𝛽𝑖𝑖 be 𝑠𝑠𝑎𝑎 and 𝑠𝑠𝑏𝑏 , respectively. the sum of the squared coefficients of variation is 𝑉𝑉 = � 𝑠𝑠𝑎𝑎 𝑎𝑎� � 2 + � 𝑠𝑠𝑏𝑏 𝑏𝑏� � 2 = � 0.0653 0.9082 � 2 + � 0.0482 0.9332 � 2 = 0.0078, and the variance due to measurement error can be found by 𝑠𝑠𝑚𝑚2 = �̅�𝜌2�̅�𝐴2𝑉𝑉 = 0.41102 × 0.84752 × 0.0078 = 0.0010. the variance in true correlation is given by 𝑠𝑠𝜌𝜌2 = 𝑠𝑠𝑟𝑟2−𝑠𝑠𝑒𝑒2−𝑠𝑠𝑚𝑚2 �̅�𝐴2 = 0.0399−0.0026−0.0010 0.84752 = 0.0506. hence, the standard deviation of the true correlation corrected for measurement error is 𝑠𝑠𝜌𝜌 = √0.0506 = 0.2249. the 80% credibility interval for 𝜌𝜌 is 0.4110 ± (1.28)(0.2249), and the 95% credibility interval is 0.4110 ± (1.96)(0.2249), as shown in table 3. copyright: © 2023 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v27i0.3995 http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 literature review 2.1 game addiction 2.2 explaining game addiction 2.3 the role of the game genre 2.4 research gap and research question 3 methodology 3.1 meta-analysis 3.2 the sample 4 analysis 4.1 moderator detection 4.1.1 method 1: q test for heterogeneity 4.1.2 method 2: credibility interval 4.1.3 method 3: schmidt and hunter’s 75% rule 4.2 moderator confirmation 4.2.1 method 1: meta-regression 4.2.2 method 2: subgroup analysis 5 implications 5.1 theoretical implications 5.2 practical implications 6 conclusion 6.1 summary 6.2 limitations 6.3 future research microsoft word 1681-5229-4-le.docx australasian journal of information systems perdana, robb & rohde 2018, vol 22, research article does visualization matter? 1 does visualization matter? the role of interactive data visualization to make sense of information arif perdana singapore institute of technology arif.perdana@singaporetech.edu.sg alastair robb uq business school, university of queensland fiona rohde uq business school, university of queensland abstract as part of business analytics (ba) technologies, reporting and visualization play essential roles in mitigating users’ limitations (i.e., being inexperienced, having limited knowledge, and relying on simplified information). reporting and visualization can potentially enhance users’ sense-making, thus permitting them to focus more on the information’s message rather than numerical analysis. to better understand the role of reporting and visualization in a contextualized environment, we investigate using interactive data visualization (idv) within accounting. we aim to understand whether idv can help enhance non-professional investors’ ability to make sense of foundational financial statement analyses. this study conducted an experiment using a sample of 324 nonprofessional investors. our findings indicate that nonprofessional investors who use idv are more heuristically adept than non-professional investors who use non-idv. these findings enrich the theoretical understanding of business analytics’ use in accounting decision making. the results of this study also suggest several practical courses of action, such as promoting wider use of idv and making affordable idv more broadly available, particularly for non-professional investors. keywords: human-computer interaction, information processing, visualization. 1 introduction and background using business analytics (ba) is increasingly widespread because of the availability of significant quantities of data and information (ong & shanks, 2015). ba is valuable because its interactive data visualization (idv) permits users to manipulate information presentations to best suit their needs (sallam et al., 2017). for example, this suitability relates to different contexts for users such as, their tasks, their personal capabilities, and the information content involved (dennis, fuller, & valacich, 2008; kelton, pennington, & tuttle, 2010). however, our research focuses on using idv in the accounting domain. idv presentations differ from those previously researched in the accounting domain (i.e., graphand table-type presentations). idv facilitates individuals’ active control of their presentations according to their needs, offers multiple types of presentations for selection, and facilitates presentation synchrony (e.g., timely feedback, search functions, and concise navigation). in short, ba can help users to acquire more meaningful information relevant to their context (cosic, shanks, & maynard, 2012; ong & shanks, 2015). for example, in accounting, idv can, all things being equal, help enhance non-professional investors’ understanding when analysing investments. australasian journal of information systems perdana, robb & rohde 2018, vol 22, research article does visualization matter? 2 the literature focuses on the impact of idv on individuals’ perceptions and decisions and finds that non-professional investors report favourably on idv because it aligns with their requirements (arnold, bedard, phillips & sutton, 2012; dilla, janvrin & raschke, 2010; dilla & raschke, 2015; janvrin, pinsker & mascha, 2013; van der heijden, 2013). our research falls within the decision-support systems (dss) domain where arnott, pervan & dodgson (2005) find that theory grounded in judgment and decision-making remain relevant in practice. in light of non-professional investors’ bounded rationality, and by locating this idv research in an accounting context, the extent to which idv affects decision making, judgment, and decision outcomes is timely and pertinent. a rich body of psychology research examines two dominant issues in decision processes and judgments namely, bounded rationality and uncertainty (see, e.g., brown, 2006; drechsler, katsikopoulos & gigerenzer, 2014; gigerenzer & goldstein, 1996; goodie & young, 2007; kahneman, 2003; tversky & kahneman, 1974). while bounded rationality assumes that individuals have limited capacity to process information (crozier & ranyard, 1997; simon, 1982), uncertainty refers to the imprecise outcomes that individuals perceive when decisionmaking (brown, 2006), for example, due to limited or irrelevant information (chuang, kao, cheng, & chu, 2012). existing research in the accounting and information systems (ais) field offers little insight, for example, into the role of information presentations in enhancing individuals’ cognition and coping with bounded rationality. research also provides little understanding whether relevant information can help reduce individuals’ perceived uncertainty. despite this limited understanding, research in information presentations’ appears to have arrived at some common findings. such findings reveal that enhancing the presentations of accounting information whether via content or visualization enhancements tends to benefit novices (e.g., non-professional investors) more than experts (e.g., professional investors) when they interact with such information (see, e.g., anderson & mueller, 2005; arnold et al., 2012; dilla, janvrin & jeffrey, 2013). research also indicates that novices’ (i.e., non-professional investors) capabilities can be improved when their decision-making involves interactions with idv (arnold et al., 2012; tang, hess, valacich & sweeney, 2014). to improve understanding of the role of information presentations relative to individuals’ cognition and bounded rationality, a deeper appreciation of how non-professional investors interact with idv is needed. such an inquiry is permitted by the availability of enhanced accounting information presentations like idv. against the preceding background and to help guide our investigation, we put forward the high-level research question: does idv improve non-professional investors’ understanding of financial information? building on and synthesizing the insight provided by prior studies, this research seeks to examine whether idv helps reduce decision making time, improves heuristic information processing (hip), reduces perceived uncertainty, increases task accuracy when solving investment tasks, and encourages an increased willingness to invest. we are guided by ecological rationality to explain idv’s potential to alleviate non-professional investors’ bounded rationality. to address the high-level research question, this study employs an experimental design using financial statements’ presentations (idv versus non-idv) and investment tasks (simple versus complex). participants in this research consisted of 324 nonprofessional investors from a research panel and an online crowdsourcing market-amazon mechanical turk (amt). this research uses financial statements from the us sec in which australasian journal of information systems perdana, robb & rohde 2018, vol 22, research article does visualization matter? 3 xbrl-enabled web versions of the statements represent idv, and the html versions of the same statements represent “nonidv”. such idv representation permits the use of several key idv features e.g., financial statement comparisons, graphical displays, searchable information disclosures, and customizable presentations. in this study, we endeavour to contribute to a better understanding of decision making with idv by offering theoretical and practical research contributions. first, this study provides evidence on idv’s role in alleviating non-professional investors’ bounded rationality by better leveraging their hip. second, this study helps articulate a complete representation of decisionmaking involving inputs (reporting/visualizations and task characteristics), decision processes (i.e., time spent, hip), judgment (perceived uncertainty), and decision outcomes (i.e., task accuracy and willingness to invest). third, this study draws attention to non-professional investors as potentially significant users of idv. fourth, this study suggests practical implications pertaining to the more widespread use of idv for non-professional investors. fifth, this study emphasizes the role of idv as a practical tool for non-professional investors to better help them make sense of complex accounting information. this paper is set out as follows. first, we identify relevant prior studies and develop hypotheses. second, we describe the research method to address the proposed hypotheses. third, we present the results including discussion of our hypotheses, findings, and research contributions. finally, we present the study’s limitations, future research directions, and conclusion. 2 research model and hypotheses development this section begins by offering our research model followed by reviewing extant research in judgement and decision making. we then explain relevant literature in idv and its relationships with task characteristics. such relationships are the key antecedents of judgement and decision making (jdm) in information presentations’ research. we then establish the importance of understanding the context in which the idv is used. first, we explain nonprofessional investors’ characteristics when interacting with idv. second, we explain how the effect of such interactions can be improved by using idv, resulting in improved decision making. we then develop our hypotheses on the extent to which idv can help alleviate nonprofessional investors’ bounded rationality and help reduce uncertainty when decision making. 2.1 research model the use of idv motivates accounting scholars to examine whether such presentation types provide more benefits to novice (e.g., non-professional investors) or experienced (e.g., professional investors) accounting information users. arnold, et al. (2012), for example, report that idv helps promote non-professional investors’ abilities to search for relevant information and to assess risks, resulting in better decision performance. arnold et al. also note that nonprofessional investors’ abilities when interacting with idv are more noticeably improved when compared to their professional counterparts. to help explicate the impact of idv on nonprofessional investors decision processes, judgment, and decision outcomes, this study proposes the following research model as shown in figure 1. we contend that the use of idv is dependent on the task characteristics. to specifically examine the effect of idv and tasks characteristics on non-professional investors’ decision australasian journal of information systems perdana, robb & rohde 2018, vol 22, research article does visualization matter? 4 processes and perceptions, and whether idv can help improve performance. heuristic information processing is a robust strategy to overcome uncertainty when decision making (mousavi & gigerenzer, 2014), when faced with limited information and constraints (e.g., cost and time) decision-makers (including non-professional investors) use hip to aid efficient decision processes to arrive at effective decision outcomes (artinger, petersen, gigerenzer & weibler, 2015). further, the complexity of accounting information is likely to produce substantial uncertainty when decision making. at the same time, non-professional investors have to be adaptable to uncertain environments and minimize information search costs when undertaking decision processes but without compromising their decision outcomes. figure 1 research model while non-professional and professional investors can both benefit from idv, nonprofessional investors obtain greater improvements than professional investors (arnold et al. 2012). such improvements likely occur because the idv enhanced presentations of accounting information better complement non-professional investors’ characteristics (i.e., inexperienced, possessing limited knowledge, and reliant on simplified information). idv likely permits nonprofessional investors to avoid complex cognitive effort required to integrate multiple pieces of information prior to making decisions. recall, they also prefer to use filtered information rather than unfiltered information (elliott, hodge, & jackson, 2008; hodge & pronk, 2006). idv, therefore, helps mitigate this non-integration tendency and enhances non-professional investors’ ability to arrive at better decisions, more akin to those made by their professional counterparts. 2.2 ecological rationality: supporting environment for decision making when processing information and making decisions, individuals may or may not deliberately process the information available to them. due to time/cost constraints and cognitive limitations, individuals likely experience bounded rationality and seek to minimize their efforts and pursue shortcuts when decision making (monti, boero, berg, gigerenzer, & martignon, 2012; simon, 1982). individuals tend to rely on simple decision rules rather than involving themselves in rational decision-making, calculating the expected utility, and optimizing results. they may, for example, expend less energy and costs focusing more on australasian journal of information systems perdana, robb & rohde 2018, vol 22, research article does visualization matter? 5 salient information instead of being overloaded (drechsler et al., 2014; tversky & kahneman, 1974, 2011). because individuals seek to reach decisions better more easily (gigerenzer, todd, & group, 1999), they manipulate either the information or its environment to meet this challenge. manipulating decision-making environments refers to ecological rationality that describes individuals being smarter and motivating other individuals to adjust their judgment and decision making (jdm) to reach better decisions (todd, gigerenzer, & the abc research group, 2012). being ecologically rational allows us to investigate decision-makers’ interactions with information presentations and visualizations, that is, the decision-making environment, because visualizations enhance the appeals of both the content and appearance of information presentations. visualizations tend to benefit novices more than experts (e.g., arnold et al., 2012; tang et al., 2014). this study posits that idv has the potential to provide an environment favourable for individuals to undertake simple, rapid, and economical strategies when decision making that still lead to effective outcomes. this premise agrees with the idea of ecological rationality, that is, the fitness between environment, the individuals, and domain-specificity permits individuals to better apply effective and efficient decision strategies when decision making (todd, fiddick & krauss, 2000; todd & gigerenzer, 2007; mousavi & kheirandish, 2014). two essential features of ecological rationality are the structure of the environments and individuals’ adaptability. ecological rationality suggests that individuals can exploit the information from their environment by applying an effective and efficient decision process, namely, hip (mousavi & kheirandish 2014). while environments may provide abundant information, not all is relevant and useful for decision making. individuals likely adapt to certain environmental structures by modifying their behaviours when decision making (gigerenzer, 2008; mousavi & kheirandish, 2014). given time and cost constraints, and uncertain computational abilities, individuals must adapt to their environment by noting what information they need to reach decisions. alternatively, the structure of the environments can be modified to facilitate individuals’ decisions (gigerenzer et al., 1999; mousavi & kheirandish 2014). individuals, for example, can modify the presentation of information including its order and structure, and how they might manipulate it. for a more specific supportive environment (e.g., an accounting office), investors could be allowed to switch the yearly or quarterly view of financial statements in a concise presentation that uses visualization to easily compare historical financial figures. in short, ecological rationality implies that supporting environments matter to decision making. individuals not only manage their decisions but also psychologically adapt to situations. however, the ecological rationality literature focuses on individuals’ decision making (i.e., hip) rather their adaptive behaviour. this study uses ecological rationality and also individuals’ perceptions to investigate both of these decision-making components of jdm environments related to accounting. this approach also reflects research calls from todd & gigerenzer (2007) to use other psychological states when investigating adaptive behaviour. our study thus adds to economic rationality the perspective of fitness of idv for nonprofessional investors’ use and idv’s roles in mitigating non-professional investors’ bounded rationality and reducing uncertainty when decision making. australasian journal of information systems perdana, robb & rohde 2018, vol 22, research article does visualization matter? 6 2.3 interactive data visualization and task characteristics research has provided the foundation for understanding features in information presentations that potentially affect decision-making (e.g., graphs, tables, electronic-based presentations, and multimedia presentations) (amer, 1991; clements & wolfe, 2000). such presentations, however, do not allow decision makers to control the information. in other words, decision-makers cannot manipulate the information according to their preferences nor control its relevance to their tasks. permitting decision makers to interactively process information and select multiple presentations or visualizations relevant to their tasks can improve their decision-making outcomes. starting in 2010 (dilla et al., 2010), accounting researchers have investigated the application of interactive presentations. while the idea of idv is established in other fields, such as marketing and information visualization, developing semantic web and a markup language (i.e., xbrl) specific to accounting information disclosure has contributed to advances in idv applications. dilla et al. (2010) establishes interactive presentation in ais by defining the term idv as “the visualization process that allows decision makers to navigate, to select data and display it at various levels of detail and in various formats” (p. 4). further, advances in is research have categorized idv features and capabilities (i.e., reporting and visualization) as part of the suite of ba technologies (watson, 2010). prior research in mainstream information systems (is) and ais suggest that individuals’ decision making is contingent on the information provided and the characteristics of their decision tasks (dennis et al., 2008; kelton et al., 2010). individuals’ interactions with information presentations are, therefore, linked to both the presentation and the task characteristics. individuals, for example, are likely to perform better with diagrams than other information presentations (i.e., icon, structured text, and text) when undertaking tasks requiring understanding, communicating, executing, or improving (figl & recker, 2014). auditors could also perform faster prediction tasks using graphical information than using tabular information (schulz & booth, 1995). while research acknowledges that decision outcomes are contingent on information presentations and tasks, apparently little research investigates idv allowing individuals to exercise greater levels of control. considering idv allows individuals to actively control the information relevant to their tasks, this study posits idv use should lead to improved decision outcomes. 2.4 the effect of idv and task characteristics on decision processes, user evaluation, and decision outcomes. research into whether manipulating information can enhance both experts’ and novices’ decision-making outcomes is divided. while experts’ decisions can be improved by information manipulation, their improvements appear to be similar to, or less than improvements reported by their novice counterparts. raschke & steinbart (2008) find that improvements in experts’ decision outcomes when interacting with misleading graphs were similar to improvements in novices’ decision outcomes using the same graph. in contrast, novices tended to show greater improvements than experts when interacting with graphs (cardinaels, 2008). similarly, when interacting with xbrl-enabled financial statements, novices like non-professional investors improved more in their searching ability and risk assessments than their professional counterparts (arnold et al., 2012). therefore, this study argues that using idv likely benefits novices more than experts. given that idv can provide australasian journal of information systems perdana, robb & rohde 2018, vol 22, research article does visualization matter? 7 more salient financial cues and easier information navigation than non-idv, non-professional investors’ use of idv can potentially improve their decision-making outcomes. the increasing volumes of information that individuals face compels individuals to more efficiently use their time (todd et al., 2000). this finding can also be applied to nonprofessional investors’ interactions with idv. using idv can potentially mitigate nonprofessional investors’ narrow ability to process information scattered around financial statements. the availability of search features, concise presentations, and multiple relevant visualizations can help non-professional investors to more readily navigate accounting information and better spend their time-solving tasks. janvrin et al. (2013) report that nonprofessional investors provided with idv spend less time doing tasks than when using nonidv. other research confirms this and notes that non-professional investors perform quicker financial statement analyses using idv (locke, lowe, & lymer, 2015). although research broadly finds that idv helps non-professional investors to make their decision-making more efficient, it does not consider the characteristics of the task of nonprofessional investors when they interact with idv. our study seeks to specify the effects of both idv and task characteristics to non-professional investors’ decision time. because idv helps non-professional investors obtain better information than traditional formats, we suggest two points: first, that idv is likely to benefit them when they seek to complete investment tasks (either simple or complex); and second, that the effect of both idv and task characteristics lead to reduced decision time. hence, we propose hypothesis 1: hypothesis 1: non-professional investors who interact with idv to complete investment tasks spend less time than non-professional investors who interact with non-idv to complete identical investment tasks. because of their ecological rationality, we see merit in investigating whether idv can help nonprofessional investors cope better with their bounded rationality and their uncertainty. bounded rationality can stimulate individuals to use heuristic information processing (hip), thus increasing their decision making efficiency by minimizing their cognitive effort (tversky & kahneman, 2011). hip is likely to occur when individuals seek to reach satisfactory solutions quickly, rather than reaching the best solutions although they may involve more time from comprehensive information processing. hip, therefore, is likely perceived as time-efficient because it involves speedier decision making. given these constraints, individuals often rationalize their decisions by efficiency or satisfaction rather than reaching the best solutions. providing relevant information, offering alternative choices for making decisions, and improving individuals’ reasoning abilities may help alleviate individuals’ bounded rationality. idv is capable of providing relevant information and thus permits individuals to select and display information fit for their purposes (arnold et al., 2012; hodge, kennedy & maines, 2004). we thus posit that, while idv not only affects individuals’ performance, it can also alleviate individuals’ bounded rationality by leveraging their hip. accordingly, we offer hypothesis 2: hypothesis 2: non-professional investors who interact with idv to complete investment tasks undertake better hip than non-professional investors who interact with non-idv to complete identical investment tasks. when relying on accounting information, idv-related research finds that xbrl-enabled financial statements potentially reduce individuals’ uncertainty (kim, lim, & no, 2012; yoon, australasian journal of information systems perdana, robb & rohde 2018, vol 22, research article does visualization matter? 8 zo, & ciganek, 2011). for example, xbrl is capable of reducing information asymmetry, making information more transparent, and lowering search costs (kim et al., 2012; peng, shon, & tan, 2011). by using idv’s searchable and more readily understood information, nonprofessional investors are better able to evaluate and integrate multiple information pieces into financial statements and thus reduce the uncertainty of their decisions. because it complements h2, we expect that using idv is positively associated with hip and will thus also reduce non-professional investors’ uncertainty. against this background, we formulate hypothesis 3: hypothesis 3: non-professional investors who interact with idv to complete investment tasks report less perceived uncertainty than non-professional investors who interact with non-idv to complete identical investment tasks. information presentations and task characteristics affect task accuracy: kelton & pennington (2012) suggest that non-professional investors perform better relative to decision making by using hyperlinked financial statements rather than paper-based financial statements. similarly, speier (2006) notes that graphical presentations help individuals to more accurately seek both simple and complex spatial solutions. more recently, tang et al., (2014) found that the interactions between idv and investment task complexity help improve both individuals’ decision accuracy and confidence. to confirm prior empirical findings in this research, we posit that the interactions between idv and task characteristics can help non-professional investors complete investment tasks more accurately and thus positively influence their willingness to invest. hypotheses 4 and 5 are thus proposed: hypothesis 4: non-professional investors who interact with idv to complete investment tasks do so more accurately than non-professional investors who interact with non-idv to complete identical investment tasks. hypothesis 5: non-professional investors who interact with idv to complete investment tasks have greater willingness to invest than non-professional investors who interact with non-idv to complete identical investment tasks. 2.5 the relationships between decision processes, user evaluation, and decision outcomes when relevant information can be more readily evaluated, individuals’ certainty when decision making can be increased (devaraj, fan, & kohli, 2002). simultaneously, individuals can better manage cost and time constraints. non-professional investors more readily acquire relevant information when interacting with xbrl-enabled financial statements (kim et al., 2012; yoon et al., 2011), because they help them to avoid incorrect analyses, and allow them to reduce errors and more efficiently complete their tasks (arnold et al., 2012; janvrin et al., 2013). we thus argue that idv can reduce non-professional investors’ perceived uncertainty. hence, we propose hypothesis 6: hypothesis 6: when interacting with idv to undertake investment tasks, non-professional investors’ time spent is reduced and is associated with less uncertainty compared with nonprofessional investors’ using non-idv to undertake identical investment tasks. when processing information, both experts and novices tend to use fast thinking by hip (tversky & kahneman, 2011). professional investors, for example, use it in integrating their prior knowledge and skills to better find accounting information cues, to apply directed search strategies, and to quickly retrieve information from financial statements, thus making their australasian journal of information systems perdana, robb & rohde 2018, vol 22, research article does visualization matter? 9 investment decision faster than non-professional investors (frederickson & miller, 2004; maines & mcdaniel, 2000). non-professional investors may also use hip but probably because their investment knowledge is restricted (monti et al., 2012). hip describes individuals’ cognitive effort when decision making (koop & johnson, 2011) and can thus influence individuals’ perceptions. xiao & benbasat (2007), for example, propose that aspects of consumers’ decision processing (e.g., less decision time) when using recommendation agents can lead to positive evaluations of the customers on dimensions, such as, perceived ease of use and perceived usefulness. trumbo (2002) finds that hip can predict individuals’ perceived risk, specifically, likely lower levels of it. while uncertain environments are frequently linked to hip, little is known about whether it can predict the level of individuals’ perceived uncertainty. this study posits that hip helps individuals to reduce their uncertainty when decision making. accordingly, we propose hypothesis 7: hypothesis 7: when interacting with idv to undertake investment tasks, non-professional investors’ hip is associated with less uncertainty compared with non-professional investors’ using non-idv to undertake identical investment tasks. while individuals perceive and process information during decision making (see, koop & johnson, 2011; tversky & kahneman, 2011; xiao & benbasat, 2007), we argue that decision making differs from user evaluation. it describes individuals’ belief about the particular circumstances existing prior to their decisions (xiao & benbasat, 2007), while decision processes describe the effort and time required when making decisions (bonner, 2008). we investigate the relationship between decision making and its outcomes using both processbased and performance-based measurement, because bonner (2008) suggests that reducing time is process-based while accuracy is performance-based. to more comprehensively consider non-professional investors’ interactions with idv, we separate decision processes, user evaluation, and decision outcomes. hypotheses 8 to 12 deal with such distinction. our first step relates to time spent and accuracy. while time spent can vary greatly, it is one of the variables of a decision process (rubinstein, 2013). time spent may predict whether individuals deliberately or quickly process information prior to arriving at decisions. while individuals’ decision processes can be quickened by applying hip, such information processing may, perhaps contrarily, lead to improved decision outcomes (todd et al., 2012). we expect that reduced time spent should similarly lead to improved decision outcomes. we argue that reducing the time spent on decision-making will provide the most benefit if it permits more time to be spent on improving task accuracy. these considerations lead to hypothesis 8: hypothesis 8: when interacting with idv to undertake investment tasks, non-professional investors’ time spent is reduced and is associated with improved task accuracy compared with non-professional investors’ using non-idv to undertake identical investment tasks. hip occurs when non-professional investors focus solely on information to support their tasks and avoid complex cognitive effort to make decisions (monti et al., 2012). hip typically uses rules of thumb together with salient information. because people who use hip can also achieve high-level performance (hertwig, herzog, schooler, & reimer, 2008; todd et al., 2012), we argue that the aforementioned circumstances can potentially occur when non-professional investors undertake accounting analyses. we propose that idv can help non-professional investors better use their hip to lead them to more informed investment decisions than nonaustralasian journal of information systems perdana, robb & rohde 2018, vol 22, research article does visualization matter? 10 professional investors who cannot access idv. this is because such access can improve the accuracy of investment, and positively influence their willingness to invest. hence, we offer hypotheses 9 and 10: hypothesis 9: when interacting with idv to undertake investment tasks, non-professional investors’ hip is positively associated with task accuracy compared with non-professional investors’ using non-idv to undertake identical investment tasks. hypothesis 10: when interacting with idv to undertake investment tasks, non-professional investors’ hip is positively associated with willingness to invest compared with nonprofessional investors’ using non-idv to undertake identical investment tasks. we extend the literature in accounting decision making evaluating non-professional investors’ willingness to invest. we adapted the variable ‘willingness to invest’, which describes individuals’ decision processes and user evaluations in research into investment decisionmaking (e.g., bhandari, hassanein & deaves, 2008; kelton & pennington, 2012) for example, kelton & pennington (2012) assess non-professionals investors’ willingness to invest by evaluating how likely they are to allocate their hypothetical investment funds. we argue that, when non-professional investors use idv, the visualizations better align with their cognitive and mental representations than when similar non-professional investors interact with nonidv. idv should thus enable non-professional investors to better evaluate the information, be better informed, and have their propensity to allocate investment improved by accessing visualizations. teo & yu (2005) find that the relationship between uncertainty and willingness to buy on online markets is mediated. that is, the lower the uncertainty and the transaction costs, the more likely that individuals will purchase products online. further, xbrl apparently enables quick information retrieval, improves transparency, and reduces information asymmetry (kim et al., 2012; peng et al., 2011; yoon et al., 2011). interactions with xbrl can thus aid nonprofessional investors to use relevant information for their investment analyses rather than having to pay to collect the information. such interactions should lead to reduced perceived uncertainty, thus encouraging non-professionals to invest more accurately and to more readily allocate their investment funds than non-professional investors undertaking identical investment tasks when interacting with non-idv. thus, we propose hypotheses 11 and 12: hypothesis 11: when interacting with idv to undertake investment tasks, non-professional investors’ reduced perceived uncertainty is positively associated with task accuracy compared with non-professional investors’ using non-idv to undertake identical investment tasks. hypothesis 12: when interacting with idv to undertake investment tasks, non-professional investors’ reduced perceived uncertainty is positively associated with willingness to invest compared with non-professional investors’ using non-idv to undertake identical investment tasks. 3 method 3.1 experimental design and variables this research employed a 2 x 2 factorial design comprised of two manipulated variables, namely financial statement presentations — 2 factors (idv vs non-idv) and task characteristics — 2 factors (simple vs complex). participants in this experiment were randomly assigned to one of four treatments: (1) idv and simple tasks; (2) non-idv and australasian journal of information systems perdana, robb & rohde 2018, vol 22, research article does visualization matter? 11 simple tasks; (3) idv and complex tasks, and (4) non-idv and complex tasks. the idv versions were given to the treatment group, with the control group receiving non-idv versions of the same financial statements. to achieve greater external validity for both the idv and nonidv presentations, we used the actual financial statements obtained from the us sec of two large pharmaceutical companies, apothecary and pharmacy (pseudonyms). the idv versions of the financial statements were taken from the calcbench platform, while the non-idv versions were html copies of the same financial statements retrieved from the us sec. the two companies had comparable financial statements and relatively similar financial performance. we chose manufacturing companies, particularly pharmaceutical, to minimize participants’ bias towards companies’ names and products. we also used manufacturing companies’ financial statements because they are the most commonly used in accounting textbooks and investment guidelines (merrill lynch, 2000, 2003). see appendix d for samples of experimental materials and experimental tasks. our research model was tested in two steps. first, we ran the experiment to investigate how the factorial design affected hip and perceived uncertainty, using manova to test the effect. second, we classified our data into four groups of non-professional investors: (1) those who interact with idv to solve simple tasks; (2) those who interact with non-idv to solve simple tasks; (3) those who interact with idv to solve complex tasks; and (4) those who interact with non-idv to solve complex tasks. we then tested how these four groups differed in their willingness to invest, using partial least squares (pls) to compare them. a web-based interface was used to operationalize the experimental procedures and materials. the following five dependent variables were thus assessed. perceived uncertainty (pu) refers to the degree to which non-professional investors can acquire and evaluate the relevant accounting information. this variable is based on measuring their perceptions in a post-task questionnaire using three reflective items reported on a seven-item, likert-type scale adapted from devaraj et al. (2002). see appendix e, table e, panel a for measurement items. hip refers to non-professional investors’ decision processes reflecting the extent to which they sought efficiency when decision making (gigerenzer, 2008; watts, shankaranarayanan, & even, 2012). this variable is based on self-reported decision processes captured in a post-task questionnaire. this variable consists of two reflective items and was measured on a seven-item, likert-type scale adapted from watts et al. (2012). see appendix e, table e, panel b for measurement items. time spent (ts) refers to the time taken by each non-professional investor to complete the assigned investment tasks in our research. this variable is directly recorded automatically using the web-based experiment. task accuracy (ta) refers how each non-professional investor scored when answering to the assigned investment tasks. this variable is measured directly with nonprofessional investors’ answers graded for accuracy by comparing them to model answers, which are straightforward and objective and do not need to be interpreted for their grading. willingness to invest (wi) refers to the likelihood of non-professional investors allocating an amount of money in the assigned tasks after they interact with the companies’ financial statements and complete their investment tasks. this variable is australasian journal of information systems perdana, robb & rohde 2018, vol 22, research article does visualization matter? 12 based on perceptual measurements captured in a post-task questionnaire. this variable was adapted from kelton & pennington’s (2012) four reflective items and was applied to the assigned financial information. this study also developed four additional reflective items to measure how individuals perceived financial statements’ presentations. see appendix e, table e, panel c for measurement items. to address the participants’ heterogeneity, we controlled for five variables: the participants’ familiarity with companies’ names and products, their prior knowledge of accounting and finance, their prior experience of investment analyses, their academic background, and their appetite for risk (i.e., risk aversion vs risk seeking). these control variables were collected using a demographic questionnaire prior to conducting the experimental tasks. 3.2 participants before the main experiment, we conducted a pilot test using 48 phd students from a large australian university to ensure that our procedures and treatments were understandable, valid, and reliable. this pilot also asked participants to assess the clarity of instructions, tasks, and questionnaires. its results indicate that participants understood the experimental materials (i.e., its procedure and treatments) and that all questions about perceptions were deemed reliable. we asked participants about the level of financial statement interactivity, the level of investment task complexity, and what they understood about the experimental procedure using a seven-point, likert-type scale. their responses to the manipulation check showed that the experimental procedure would be understandable and the experimental treatments would be successful. the participants for the main experiment were recruited from an online crowdsourcing market (ocm: amazon mechanical turk) and a research panel (qualtrics). the main consideration when selecting the sample data was the heterogeneity of non-professional investors. to ensure that participants suitably represented non-professional investors, we asked them screening questions and accepted only those who answered them appropriately. all participants sourced via qualtrics were from australia while those sourced using ocm were from the usa. all participants were from relatively diverse backgrounds (see appendix a, tables a.1, a.2, & a3). 4 results in the following subsections, we explain the results of the multivariate analysis of variance (manova) and pls used to test the hypotheses. manova compared the interaction effects between financial statement presentations and investment task complexity on nonprofessional investors’ time spent, hip, perceived uncertainty, task accuracy, and willingness to invest. pls with multigroup analysis investigated the strength and the direction of the relationships between variables in the research model. 4.1 results of the effect of idv and task characteristics on decision processes, user evaluation, and decision outcomes. before conducting the hypotheses testing, we undertook a factor analysis to determine whether the measurement items were grouped into their relevant constructs (see appendix b, table b.1). our sample was sufficient to satisfy the multivariate normality assumption (tabachnick & fidell, 2007). the data consist of 81 elements from each group indicating that the data exceed a sample size of 20 per group. a pearson correlation analysis suggests that the australasian journal of information systems perdana, robb & rohde 2018, vol 22, research article does visualization matter? 13 variables were moderately correlated (< .7) indicating that multicollinearity is not a concern for the data (see appendix b, table b.2). in addition, the results of the scatterplots for each group indicate linear relationships for each variable. the data violate the assumption of the equal variance of covariance; this is not substantially concerning as the data for each group in the experiment are approximately equal. thus, the interpretation of the data remains robust (hair, black, babin, & anderson, 2010; tabachnick & fidell, 2007). to deal better with the equality of error variance, we used a more stringent alpha level of .001 to accept or reject the hypotheses tested with manova (tabachnick & fidell, 2007). hypotheses 1, 2, 3, 4 and 5 specifically compare the differences between experimental and control groups. to test the hypotheses further and help specify the differences, we conducted planned comparisons to examine whether such differences occurred between the groups. we provided weighting coefficients for each group to be compared, and zero weight coefficients for groups that were not. the planned comparisons used the weight coefficients of (contrast 1: 1,-1,0,0); (contrast 2: 0,0,1,-1), and (contrast 3: 1,-1,1,-1) for the following four groups: idv and simple tasks; non-idv and simple tasks; idv and complex tasks; and non-idv and complex tasks, respectively. table 1 displays the results of planned comparisons for each group. hypotheses and variables contrast value of contrast std. error t sig. h1 time spent 1 -2.308 .3524 -6.550 .000* 2 -6.434 .6739 -9.548 .000* 3 -8.743 .7605 -11.496 .000* h2 heuristic information processing 1 .940 .159 5.892 .000* 2 1.400 .157 8.865 .000* 3 2.330 .224 10.422 .000* h3 perceived uncertainty 1 1.070 .147 7.313 .000* 2 1.830 .160 11.400 .000* 3 2.900 .217 13.345 .000* h4 task accuracy 1 1.012 .115 8.832 .000* 2 1.765 .107 16.490 .000* 3 2.778 .157 17.710 .000* h5 willingness to invest 1 .790 .101 7.788 .000* 2 1.432 .116 12.332 .000* 3 2.222 .154 14.412 .000* * significant at .05 (one-tailed) table 1 contrast tests the contrast tests confirm that our hypotheses 1, 2, 3, 4 and 5 were supported. the results suggest that non-professional investors who use idv, whether completing simple or complex investment tasks, did so more quickly, had better hip, perceived less uncertainty, were more accurate, and had greater willingness to invest than those who use non-idv when completing the same investment tasks regardless of their complexity. in addition to hypothesis testing, we conducted additional manova analyses to examine whether the interaction effects between information presentations and task characteristics remained significant while controlling for the other variables. we controlled for participants’ familiarity with companies’ names or australasian journal of information systems perdana, robb & rohde 2018, vol 22, research article does visualization matter? 14 products, prior knowledge of accounting and finance, experience of investment analysis, academic background, and appetite for risk. the multivariate tests presented a significant pvalue (< .05) for the interaction effects. after adding the control variables to the analysis, the effects of information presentations and task characteristics remained significant for ts, hip, pu, ta, and wi. the introduction of the control variables had no significant additional effect on the results, thus confirming the strength of the effect of both financial statements’ presentations and investment tasks’ complexity on ta and wi. 4.2 results of the relationships between decision processes, user evaluation, and decision outcomes. the second analysis in this study tested the strength and directions of relationships between non-professional investors’ ts, hip, pu, ta and wi using component-based structural equation modelling (sem) or pls to conduct the analysis. we used the software smartpls 3.0 to conduct the three analyses. first, we examined the relationship between the measurement items and the variables to ensure that the items suitably reflect and measure the variables. second, we conducted a relational analysis between variables for the idv and non-idv contexts. third, we examined the strength of the relationships between variables using a multigroup comparison analysis between idv and non-idv. we used the whole dataset (n = 324) to analyse the measurement model and found it to be valid (see details in appendix c). after this analysis and verifying the construct validity, we divided the data set into four subsets: group 1 (idv and simple tasks); group 2 (idv and complex tasks); group 3 (non-idv and simple tasks); and group 4 (non-idv and complex tasks), which yielded 81 items in each subset. we then analysed the groups in two categories (i.e., idv and non-idv) to assess the structural model using multigroup analysis. to determine the significance of the relationships, we performed a bootstrap analysis using a resampling technique of 1000 random samples. further, we compared the results of the structural model between idv and non-idv use to ascertain whether the relationships between variables in each group were different. table 2 presents the path coefficients and t statistics of the pls results for each group. relationships idv non-idv simple complex simple complex path coefficients/ p-values path coefficients/ p-values path coefficients/ p-values path coefficients/ p-values  time spent  task accuracy .017/.420 .006/.479 .194/.054* -.089/.312  time spent  perceived uncertainty .040/.364 -.001/.497 .054/.321 .678/.000*  hip task accuracy .591/.000* .387/.000* .327/.014* .039/.384  hip perceived uncertainty .286/.185 .443/.001* .342/.113 .001/0.497  hip willingness to invest .706/.000* .656/.000* .292/.013* .115/.230  perceived uncertainty task accuracy .193/.183 .004/.489 -.038/.403 .178/.155  perceived uncertainty willingness to invest .097/.235 .151/.117 .063/.346 -.268/.003* * significant at .05 (one-tailed) table 2 path coefficients for each comparable group regarding idv, our relational analysis (see table 2) shows that hip is positively associated with both ta and wi while the relationship between hip and pu is only significant for nonprofessional investors when completing complex tasks. these findings importantly suggest australasian journal of information systems perdana, robb & rohde 2018, vol 22, research article does visualization matter? 15 that hip can predict decision-making outcomes. overall, regarding relational analysis, hip and pu explained 60% of the variability in wi. ts and hip contribute 23% of the variance of pu, and 44% of the variance of accuracy is explained by hip, ts and, pu (see appendix c, figure c.1). our relational analysis for non-idv (see appendix c, figure c.2) shows five significant relationships between variables, namely, ts and ta, hip and ta, hip and wi, ts and pu, and pu and wi. the first three were only significant for non-professional investors completing simple tasks. the relationships between ts and pu, and pu and wi were also significant for non-professional investors completing complex tasks. the negative sign for the relationship between pu and wi implies that non-professional investors completing complex tasks with non-idv have higher uncertainty and less willingness to invest compared to those completing simple tasks with non-idv. the r2 value of the model indicates that time spent contributes to australasian journal of information systems perdana 2018, vol 22, research article does visualization matter? 16 relationships path difference and pvalue (idv simple vs idv complex) path difference and pvalue (idv simple vs non-idv simple) path difference and pvalue (idv simple vs non-idv complex)  time spent task accuracy .011/.526 .177/.883 .106/.292  time spent perceived uncertainty .041/.600 .014/.542 .638/.999  hip task accuracy .204/.935 .264/.048* .553/.000*  hip perceived uncertainty .157/.428 .056/.522 .285/.298  hip willingness to invest .050/.676 .414/.000* .591/.000*  perceived uncertainty task accuracy .189/.751 .231/.209 .016/.459  perceived uncertainty willingness to invest .055/.390 .033/.441 .365/.016* relationships path difference and pvalue (idv complex vs non-idv simple) path difference and pvalue (idv complex vs non-idv complex) path difference and pvalue (non-idv simple vs group non-idv complex)  time spent  task accuracy .188/0.878 .095/.320 .283/.899  time spent  perceived uncertainty .055/0.637 .679/.999 .624/.000*  hip  task accuracy .060/.380 .348/.018* .288/.928  hip  perceived uncertainty .101/.448 .442/.015* .341/.826  hip  willingness to invest .364/.005* .541/.000* .177/.824  perceived uncertainty  task accuracy .042/.416 .174/.781 .215/.180  perceived uncertainty  willingness to invest .088/.340 .420/.011* .332/.958 * significant at .05 (one-tailed) table 3: a 4 x 4 multigroup comparison australasian journal of information systems perdana 2018, vol 22, research article does visualization matter? 17 11% of the variance in perceived uncertainty. the r2 value for accuracy and wi was not significant. the multigroup comparison analysis (see table 3) between the four groups indicates that the relationships between hip, ta, and wi are stronger for non-professional investors interacting with idv than those interacting with non-idv regardless of their tasks’ complexity. the relationship between hip and pu is stronger for non-professional investors who use idv to complete complex tasks than non-professional investors who use non-idv to complete the same tasks. while the path coefficient between ts and pu is not significant for idv (see table 2), the relationship between those variables is stronger for non-professional investors interacting with non-idv when completing complex tasks than those completing simple tasks (see table 3). the path coefficient between perceived uncertainty and willingness to invest is not significant for idv (see table 2). such a relationship, however, is stronger for non-professional investors interacting with idv when completing either simple or complex tasks than those interacting with non-idv when completing complex tasks. to support the hypotheses tested with pls, we considered both the relational and comparative analyses. they showed that h9 and h10 were fully supported, h12 was partially support, h6 and h7 were partially supported but contingent on the investment tasks’ complexity, while h8 and h11 were rejected (table 4, panel a summarizes the outcomes of hypotheses testing for the experimental and control groups. table 4, panel b summarizes the outcomes of hypotheses testing for the interactions between groups and variables). panel a: hypotheses tested with manova support non-professional investors who interact with idv to complete investment tasks hypothesis 1: spend less time full hypothesis 2: undertake better hip full hypothesis 3: report less perceived uncertainty full hypothesis 4: do so more accurately full hypothesis 5: have greater willingness to invest full than non-professional investors who interact with non-idv to complete the same investment tasks. australasian journal of information systems perdana 2018, vol 22, research article does visualization matter? 18 panel b: hypotheses tested with manova results support relational analysis comparison analysis when interacting with idv to undertake investment tasks, non-professional investors’ hypothesis 6: time spent is reduced and is associated with less uncertainty not significant significant contingent on task complexity partial contingent on tasks’ complexity hypothesis 7:hip is associated with less uncertainty significant contingent on task complexity significant partial contingent on tasks’ complexity hypothesis 8: time spent is reduced and is associated with improved task accuracy not significant not significant no hypothesis 9: hip is positively associated with task accuracy significant significant full hypothesis 10: hip is positively associated with willingness to invest significant significant full hypothesis 11: reduced perceived uncertainty is positively associated with task accuracy not significant not significant no hypothesis 12: reduced perceived uncertainty is positively associated with willingness to invest not significant significant partial compared with non-professional investors’ using non-idv to undertake the same investment tasks. table 4 summary of hypotheses testing results 5 research contributions 5.1 theoretical contributions while information visualization improves cognition (mazza, 2009), little research in ais investigates whether enhancing perception improves cognition. little is also known about the relationships among non-professional investors’ perceptions, decision processes, and decision outcomes when interacting with idv while undertaking simple versus complex tasks. the results of this study fill these research gaps. our first finding suggests that decision processes and user evaluations contribute to the variance of decision outcomes, particularly when non-professional investors interact with idv to complete either simple or complex investment tasks. we also find that nonprofessional investors who use idv to complete their investment tasks do so with greater accuracy and willingness to invest. unlike utaut and tam which focus on adoption decisions, our research endeavours to better articulate the underlying framework for decision making with idv. for example, visualization is perceived to facilitate the use and usefulness of interactive data compared to other presentation formats (janvrin et al., 2013) and thus potentially lead to users choosing it. however, how this interactive data leads to better decision making remains in contention. thus, our research assists users to understand the role of idv in affecting decisions about willingness to invest rather than willingness to adopt idv. with research emphasizing outcomes (e.g., task accuracy) rather than the processes leading to them (arnold et al., 2012; janvrin et al., 2013), individuals are likely to develop information processing strategies that accommodate and mitigate their uncertainty about making good decisions. individuals may also refer to the information or decision aids they use to complete tasks before australasian journal of information systems perdana 2018, vol 22, research article does visualization matter? 19 arriving at decision outcomes. the second finding of this study highlights that non-professional investors who use idv apply their hip better than those who use non-idv. we found that nonprofessional investors are more heuristically adept and perceive idv as offering more certainty than non-idv when completing both simple and complex investment tasks. while researchers are divided about whether or not hip can lead to the best decision making (daigle, pinsker, & pitre, 2015; hertwig et al., 2008), our third finding shows that using hip can lead to improved decision outcomes. our three findings help confirm that decision making under pressure (e.g., time and uncertainty) is not always systematically undertaken but is more likely to be heuristic. idv positively influences time spent and task accuracy when completing investment tasks. idv also improves non-professional investors’ accuracy and their ability to allocate time efficiently when completing both simple and complex tasks. using idv to complete either investment tasks positively affects the likelihood of non-professional investors allocating their investment funds. this finding aligns with past studies in experimental psychology (see, e.g., hertwig et al., 2008; todd et al., 2012). overall, our findings show that idv could be used to positively influence non-professional investors’ perceptions and better use their hip. while this finding may interest cognitive science (goldstone, pestilli, & börner, 2015), understanding such a mechanism is important to the design knowledge of idv. the design properties of idv, such as multiple visualization, active control, and analytics, can help complex accounting information become more intelligible and thus understood. 5.2 practical contributions this study focuses on the context of idv use by non-professional investors. we find that idv better equips non-professional investors in their practice of making investment decisions. first, idv is a valuable tool for non-professional investors’ decision making. our findings show that idv can alleviate non-professional investors’ bounded rationality and reduce uncertainty. idv mitigate, at least, some non-professional investors’ limitations, particularly those associated with hip. we suggest that accounting and other regulatory bodies could promote widespread use of idv and help to provide affordable idv platforms that permit non-professional investors to make more sound investment decisions. companies could also develop idv to deliver accounting information that attracts non-professional investors’ funds. second, our findings suggest that idv capabilities may aid non-professional investors to reach better decisions closer to how professional investors do thus showing the potential of idv in practice. tools like idv are increasingly important, particularly in the era of big data where individuals, including non-professional investors, need the means of readily organizing increasing volumes of data. for non-professional investors, idv can mitigate the complexity of accounting information by offering them greater control over the presentation of information. they can thus, undertake investment analyses using visualizations rather than complex number crunching. 6 limitations, future research, and conclusion 1 three limitations should be considered when interpreting the results of this study. first, while we used only research tasks concerned with calculating financial ratios, which are basic to investment analyses (cong, du, & feng, 2008; hodge et al., 2004), such tasks in practice may differ from what we provided in this experiment. further, they focused only on information in the financial statements. we did not use investment tasks that require participants to examine other information, such as notes to financial statements and md&a. second, one of the variables in decision outcomes, that is, willingness to invest should be cautiously interpreted. this variable reflects individuals’ intention to invest based on their interactions with australasian journal of information systems perdana 2018, vol 22, research article does visualization matter? 20 financial statement presentations and the associated tasks. we used the financial statements from two pharmaceutical companies that had relatively similar financial performances. thus, willingness to invest does not consider other factors such as the economic circumstances of the companies being analysed. we incorporated this variable nonetheless to measure the extent to which idv assisted non-professional investors to better access information when investing, and how ibv may influence their intention to invest funds. third, while we used relatively representative pools of non-professional investors and presented real financial statements to improve the external validity of these results, future studies should confirm our results with other pools of non-professional investors. for example, those with different levels of business knowledge may display relationships between variables (e.g., time spent, hip, perceived uncertainty, tasks accuracy and willingness to invest) that vary in the degree to which idv enhances non-professional investors’ decision making. such research is needed because investigating idv from the perspective of ecological rationality continues to evolve. nonetheless, these limitations motivate future research. first, future research could develop investment task analyses that better represent the practices that require participants to integrate both quantitative and qualitative information from financial statements (e.g., financial reports’ footnotes and md&a). sutton, arnold, bedard, & philips (2012) note that xbrl’s capacity to convert qualitative accounting information into structured data can support sound investment analyses. such a capacity is indeed relevant as exemplified by idv’s role in helping non-professional investors to analyse qualitative accounting information and incorporate such information into their investment decisions. second, more research should investigate whether idv affects professional investors and how it differs from non-professional investors. third, while our research confirms the efficacy of idv for alleviating non-professional investors’ bounded rationality and reducing their uncertainty, we did not seek to isolate the aspects of idv most suited to aiding non-professional investors. future research could investigate, isolate, and confirm the aspects of idv that best reduce non-professional investors’ bounded rationality and their uncertainty. overall, this study has answered our high-level research questions: does idv improve nonprofessional investors’ understanding of financial information? idv aids non-professional investors by improving their hip and reducing their levels of uncertainty when undertaking investment tasks. we posit that our research is theoretically and pragmatically relevant as idv is becoming widespread in accounting to aid non-professional investors’ decision making. references amer, t. s. (1991). an experimental investigation of multi-cue financial information display and decision making. journal of information systems, 5(2), 18–34. anderson, j. c., & mueller, j.m. (2005). the effects of experience and data presentation format on an auditing judgment. the journal of applied business research, 21(1), 53–63. arnold, v., bedard, j., phillips, j., & sutton, s.(2012). the impact of tagging qualitative financial information on investor decision making: implications for xbrl. international journal of accounting information systems, 13(2), 2–20. arnott, d., pervan, g., dodgson, g. (2005). a descriptive analysis of decision support systems research between 1993 and 2003. australasian journal of information systems, 12(2), 178–191. bhandari, g., hassanein, k., & deaves, r. (2008). debiasing investors with decision support systems: an experimental investigation. decision support systems, 46(1), 35–40. bonner, s. e. (2008). judgement and decision making in accounting. new jersey: upper saddle river. australasian journal of information systems perdana 2018, vol 22, research article does visualization matter? 21 brown, j. c. (2006). the effects of behavioral and outcome feedback on prudent decision-making under conditions of present and future uncertainty. judgment and decision making, 1(1), 76–85. cardinaels, e. (2008). the interplay between cost accounting knowledge and presentation formats in cost-based decision-making. accounting organizations and society, 33(6), 582–602. chuang, s., kao, d-t., cheng, y-h., & chu, c-a. (2012). the effect of incomplete information on the compromise effect. judgment and decision making, 7(2), 196–206. clements, c. e., & wolfe, c. j. (2000). reporting financial results with the video medium: an experimental analysis. journal of information systems, 14(2), 79–94. cong, d., du, h., & feng, j. (2008). does web syndication technology facilitate investor decision making? journal of emerging technologies in accounting, 5(1), 143–159. cosic, r., shanks, g., & maynard, s. (2016). a business analytics capability framework. australasian journal of information systems, 19(1), 1–18. crozier, w. r., & r. ranyard. (1997). cognitive process models and explanations of decision making. in svenson, ranyard, r., crozier w.r. (ed.), decision making: cognitive models and explanations (pp. 5–20). new york: routledge. daigle, r. j., pinsker, r. e., & pitre, t. j. (2015). the impact of order effects on nonprofessional investors’ belief revision when presented a long series of disclosures in an experimental market setting. accounting horizons, 29(2). dennis, a., fuller, r., & valacich, j. (2008). media, tasks, and communication processes: a theory of media synchronicity. mis quarterly, 32(3), 575–600. devaraj, s., fan, m., & kohli, r. (2002). antecedents of b2c channel satisfaction and preference: validating e-commerce metrics. information systems research, 13(3), 316–333. dilla, w. n., janvrin, d. j., & jeffrey, c. (2013). the impact of graphical displays of pro forma earnings information on professional and nonprofessional investors’ earnings judgments. behavioral research in accounting, 25(1), 37–60. dilla, w. n., janvrin, d. n., & raschke, r. l. (2010). interactive data visualization: a new directions for accounting information systems research. journal of information systems, 24(2), 137. dilla, w.n. & raschke, r.l. 2015. data visualization for fraud detection: practice implications and a call for future research. international journal of accounting information systems, 16, 1-22. drechsler, m., katsikopoulos, k., & gigerenzer, g. (2014). axiomatizing bounded rationality: the priority heuristic. theory and decision, 77(2), 183–196. elliott, w. b., hodge, f. d., & jackson, k. e. (2008). the association between nonprofessional investors’ information choices and their portfolio returns: the importance of investing experience. contemporary accounting research, 25(2), 473–498. figl, k., & recker, j. (2014). exploring cognitive style and task-specific preferences for process representations. requirements engineering, forthcoming, 1–23. frederickson, j. r., & miller, j.s. (2004). the effects of pro forma earnings disclosures on analysts’ and non-professional investors’ equity valuation judgments. the accounting review, 79(3), 667–686. gigerenzer, g. (2008). why heuristic work. perspectives on psychological science, 3(1), 20–29. australasian journal of information systems perdana 2018, vol 22, research article does visualization matter? 22 gigerenzer, g., & goldstein, d. g. (1996). reasoning the fast and frugal way: models of bounded rationality. psychological review, 103(4), 650–669. gigerenzer, g., todd, p., & the abc research group. (1999). simple heuristics that make us smart. new york: oxford university press, inc. goldstone, r. l., pestilli, f., & börner, k. (2015). self-portraits of the brain: cognitive science, data visualization, and communicating brain structure and function. trends in cognitive sciences, 19(8), 462–474. goodie, a. s., & young, d. l. (2007). the skill element in decision making under uncertainty : control or competence? judgment and decision making, 2(3), 189–203. hair, j. f., black, w.c. babin, b.j. & anderson, r.e. (2010). multivariate data analysis 7th edition. usa: prentice hall. hertwig, r., herzog, s. m., schooler, l. j., & reimer, t. (2008). fluency heuristic: a model of how the mind exploits a by-product of information retrieval. journal of experimental psychology: learning, memory and cognition, 34(5), 1191–1206. hodge, f. d., kennedy, j. j., & maines, l. a. (2004). does search-facilitating technology improve the transparency of financial reporting. the accounting review, 79(3), 687–703. hodge, f., & pronk, m. (2006). the impact of expertise and investors’ use of online financial report information. journal of accounting auditing finance, 21(3), 267–292. janvrin, d. j., pinsker, r., & mascha, m. f. (2013). xbrl-enabled, spreadsheet, or pdf? factors influencing exclusive user choice of reporting technology. journal of information systems, 27(2), 35–49. kahneman, d. (2003). maps of bounded rationality: psychology for behavioral economist. the american economic review, 93(5), 1449–1475. kelton, a. s., & pennington, r. r. (2012). internet financial reporting: the effects of information presentation format and content differences on investor decision making. computers in human behaviour, 28(4), 1178–1185. kelton, a. s., pennington, r. r., & tuttle, b. m. (2010). the effects of information presentation format on judgment and decision making: a review of the information systems research. journal of information systems, 24(2), 79–105. kim, j., lim, j.-h., & no, w. g. (2012). the effect of first wave mandatory xbrl reporting across the financial information environment. journal of information systems, 26(1), 127–153. koop, g., & j.g. johnson. (2011). response dynamics: a new window on the decision process. judgment and decision making, 6(8), 750–758. locke, j., lowe, a. & lymer, a. (2015). interactive data and retail investor decision-making: an experimental study. accounting & finance, 55(1), 213–240. maines, l. a., & mcdaniel, l.s. (2000). effects of comprehensive-income characteristics on nonprofessional investors’ judgments: the role of financial-statement presentation format. the accounting review, 75(2), 179–207. mazza, r. (2009). introduction to information visualization. london: springer-verlag. merrill lynch. (2000). how to read financial report. usa: merrill lynch, pierce, fenner & smith, inc. australasian journal of information systems perdana 2018, vol 22, research article does visualization matter? 23 merrill lynch. (2003). merrill lynch guide to understanding financial reports. usa: merrill lynch, pierce, fenner & smith, inc. monti, m., boero, r., berg, n., gigerenzer, g., & martignon, l. (2012). how do common investors behave? information search and portfolio choice among bank customers and university students. mind & society, 11(2), 203–233. mousavi, s., & gigerenzer, g. (2014). risk, uncertainty, and heuristics. journal of business research 67(8): 1671–1678. mousavi, s., & kheirandish, r. (2014). behind and beyond a shared definition of ecological rationality: a functional view of heuristics. journal of business research 67(8): 1780–1785. ong, k., & shanks, g. (2015). editorial for the special issue on business analytics applications. australasian journal of information systems, 19(1), 1–3. peng, e., shon, j., & tan, c. (2011). xbrl and accruals: empirical evidence from china. accounting perspectives, 10(2), 109–138. raschke, r., & steinbart, p. j. (2008). mitigating the effects of misleading graphs on decisions by educating users about the principles of graph design. journal of information systems, 22(2), 23– 52. roohani, s., furusho, y. & koizumi, m. (2009). improving transparency and monitoring functions of corporate governance. international journal of disclosure and governance, 6(4), 355–369. rubinstein, a. (2013). response time and decision making: an experimental study. judgment and decision making, 8(5), 540–551. sallam, r. l., howson, c., idoine, c. j., oestreich, t. w., richardson james l, & tapadinhas, j. (2017). magic quadrant for business intelligence and analytics platforms. schulz, a., & booth, p. (1995). the effects of presentation format on the effectiveness and efficiency of auditors’ analytical review judgments. accounting and finance, 35(1), 107–131. simon, h. (1982). models of bounded rationality. cambridge, ma: mit press. speier, c. (2006). the influence of information presentation formats on complex task decisionmaking performance. international journal of human computer studies, 64, 11115–11131. sutton, s. g., arnold, v., bedard, j.c, & philips, j.r. (2012). enhancing and structuring the md&a to aid investors when using interactive data. journal of information systems, 26(2), 167–188. tabachnick, b. g., & fidell, l. s. (2007). using multivariate statistics 5th edition. usa: person educations, inc. tang, f., hess, t., valacich, j., & sweeney, j. (2014). the effects of visualization and interactivity on calibration in financial decision-making. behavioral research in accounting, 26(1), 25–58. teo, t. s., & yu, y. (2005). online buying behavior : a transaction cost economics perspective. omega, 33(5), 451–465. todd, p., fiddick, l., & krauss, s. (2000). ecological rationality and its contents. thinking & reasoning, 6(4), 375–384. todd, p. m., gigerenzer, g., & the abc research group. (2012). ecological rationality: intelligence in the world. new york: oxford university press, inc. todd, p.m., & gigerenzer, g. (2007). environments that make us smart: ecological rationality. current direction in psychological science 16(3): 167–171. australasian journal of information systems perdana 2018, vol 22, research article does visualization matter? 24 trumbo, c. w. (2002). information processing and risk perception: an adaptation of the heuristicsystematic model. journal of communication, 52(2), 367–382. tversky, a., & kahneman, d. (1974). judgment under uncertainty: heuristics and biases. science, 185(4157), 1124–1131. tversky, a., & kahneman, d. (2011). thinking, fast and slow. new york: farrar, straus and giroux. van der heijden, h. (2013). evaluating dual performance measures on information dashboards: effects of anchoring and presentation format. journal of information systems, 27(2), 21–34. watson, h.j. (2010). bi-based organizations. business intelligence journal, 15(2), 4 watts, s., shankaranarayanan, g., & even, a. (2012). data quality assessment in context: a cognitive perspective. decision support systems, 48(1), 202–211. xiao, b., & benbasat, i. (2007). e-commerce product recommendation agents: use, characteristics, and impact. mis quartely, 31(1), 137–209. yoon, h., zo, h., & ciganek, a. p. (2011). does xbrl adoption reduce information asymmetry? journal of business research, 64(2), 157–163. australasian journal of information systems perdana 2018, vol 22, research article does visualization matter? 25 appendices appendix a table a.1 screening questions for nonprofessional investors questions 1. are you subscribing on behalf of an investment firm? 2. are you employed by a bank or an investment firm performing investment related tasks? 3. are you a broker/dealer, or an investment advisor? 4. do you have a limited understanding about financial statements and interested in conducting investment tasks? 5. are you a student (bachelor, postgraduate student or research student, etc.)? 6. are you a united states (us) citizen or australian permanent resident? note: if participants’ answered “no” to questions 1, 2, 3, and 5, and “yes” to question 4 and 6 they passed the screening questions and the web-based experimental tasks provided them with an information sheet and a consent form. question no 6 was distinguished by the data sources. this study targeted us participants from online crowdsourcing market and targeted australian participants from online research panel. table a.2 participants responses selection research panel online crowdsourcing market total responses 131 367 failed to pass screening questions 0 82 incomplete responses 43 20 failed to complete experimental tasks 12 17 final usable responses 76 248 percent usable responses 57% 68% table a.3 demographic distribution demographics research panel online crowdsourcing market gender male 43% 48% female 57% 52% age mean 42 37 median 43 37 minimum 21 20 maximum 56 60 academic background accounting 7% 7% finance 4% 5% information systems 12% 8% psychology 4% 10% others 73% 70% prior accounting knowledge yes 54% 47% no 46% 53% investment familiarity not familiar 3% 3% a little less familiar 25% 32% australasian journal of information systems perdana 2018, vol 22, research article does visualization matter? 26 demographics research panel online crowdsourcing market somewhat familiar 53% 44% familiar 17% 13% very familiar 3% 8% pdf/html financial statements familiarity not familiar 10% 15% a little less familiar 18% 27% somewhat familiar 35% 34% familiar 30% 17% very familiar 7% 7% xbrl-enabled financial statements familiarity not familiar 50% 61% a little less familiar 24% 27% somewhat familiar 21% 9% familiar 4% 3% very familiar 1% 0% appetite for risk risk averse 84% 77% risk neutral 8% 13% risk taker 8% 10% familiarity with apothecary and its products not familiar 65% 35% a little less familiar 20% 29% somewhat familiar 10% 25% familiar 3% 8% very familiar 2% 3% familiarity with pharmacy and its products not familiar 40% 20% a little less familiar 24% 30% somewhat familiar 24% 30% familiar 7% 17% very familiar 5% 3% working experience no experience 4% 3% 0-5 years 22% 21% 6-10 years 21% 32% 11-15 years 53% 44% areas of experience accounting 4% 7% finance 11% 4% information systems 8% 13% psychology 7% 4% others 70% 72% australasian journal of information systems perdana 2018, vol 22, research article does visualization matter? 27 appendix b table b.1 rotated component matrix item component 1 2 3 pu1 .231 .679 .204 pu2 .053 .820 .154 pu3 .190 .801 -.013 hip1 .263 .112 .847 hip2 .320 .207 .771 wi1 .852 .233 .184 wi2 .847 .196 .180 wi3 .857 .189 .220 wi4 .891 .127 .208 wi5 .910 .132 .175 wi6 .898 .173 .179 wi7 .889 .117 .222 wi8 .887 .129 .250 pu – perceived uncertainty, hip – heuristic information processing, and wi – willingness to invest table b.2 pearson correlations variables time spent heuristic information processing perceived uncertainty accuracy willingness to invest time spent heuristic information processing -.120 perceived uncertainty .000 .376 accuracy -.232 .500 .486 willingness to invest -.256 .528 .370 .541 australasian journal of information systems perdana 2018, vol 22, research article does visualization matter? 28 appendix c table c.1 loadings and cross loadings of measure (n=324) time spent heuristic information processing perceived uncertainty accuracy willingness to invest ave n/a .773 .627 n/a .840 time spent* 1 -.138 -.019 -.232 -.301 hp1 -.110 .865 .296 .431 .460 hp2 -.132 .894 .361 .468 .511 pu1 -.080 .345 .814 .434 .381 pu2 .120 .287 .773 .298 .241 pu3 -.052 .253 .788 .411 .316 accuracy* -.232 .512 .489 .000 .602 wi1 -.299 .502 .413 .596 .902 wi2 -.222 .491 .383 .550 .888 wi3 -.274 .518 .388 .552 .907 wi4 -.277 .506 .347 .534 .923 wi5 -.233 .488 .349 .521 .932 wi6 -.257 .496 .380 .520 .931 wi7 -.323 .514 .340 .578 .922 wi8 -.322 .539 .353 .561 .929 composite reliability n/a .872 .834 n/a .977 * accuracy and time spent is indicated by single index in the pls model. australasian journal of information systems perdana 2018, vol 22, research article does visualization matter? 29 figure c.1 path diagram for idv figure c.2 path diagram for nonidv australasian journal of information systems perdana 2018, vol 22, research article does visualization matter? 30 appendix d experimental materials the two factors manipulated in the experiments were: financial statements’ visualizations (idv vs nonidv) and investment tasks’ complexity (simple vs complex). we used xbrl-enabled financial statements from us sec retrieved from www.calcbench.com representing the idv as presented in figure d1, whereas the html version retrieved from us sec representing nonidv. both financial statements represent equal information; the only different is the way the financial statements are presented to users. unlike nonidv, idv allows users to have a greater control over the information, for example, selecting the presentations or visualizations relevant to their purposes, and comparing several financial statements from different companies. australasian journal of information systems perdana 2018, vol 22, research article does visualization matter? 31 figure d.1 sample of interactive data visualization australasian journal of information systems perdana 2018, vol 22, research article does visualization matter? 32 figure d.2. instructions for interactive data visualization figure d.3. instructions for non-interactive data visualization australasian journal of information systems perdana 2018, vol 22, research article does visualization matter? 33 table d.1 experimental tasks experimental tasks simple tasks complex tasks 1. entering net earnings for the two companies 1. entering net earnings for apothecary and calculating the percentage change (% change quarter on quarter) to the net earnings attributable to apothecary 2. entering the value and/or component to calculating net income or profit margin for the two companies 2. calculating the difference of net earnings between apothecary and pharmacy 3. entering total assets for the two companies 3. entering total assets for apothecary and calculating the percentage change (% change quarter on quarter) to the net earnings attributable to apothecary 4. entering the value and/or component to calculating current ratio for the two companies 4. calculating the difference of total assets between apothecary and pharmacy australasian journal of information systems perdana 2018, vol 22, research article does visualization matter? 34 appendix e measurement items table e variables, items and measurements items measurements panel a: dependent variable (manova), independent variable (pls): perceived uncertainty (adapted from devaraj, fan & kohli (2002). 1. i believe that i was able to easily evaluate the financial information. likert 1-7 (strongly disagree to strongly agree) 2. the financial statements provided adequate information. likert 1-7 (strongly disagree to strongly agree) 3. the financial statements provided sufficient information for the required investment tasks. likert 1-7 (strongly disagree to strongly agree) panel b: dependent variable (manova), independent variable (pls): heuristic information processing (adapted from gigerenzer (2008) and watts, shankaranarayanan & even (2012). 1. the process i used to solve the required investment tasks was a very efficient one. likert 1-7 (strongly disagree to strongly agree) 2. i expended less effort to achieve an optimal solution to the investment tasks. likert 1-7 (strongly disagree to strongly agree) panel c: dependent variable (pls) : willingness to invest 1. assume that you have $5,000 to invest in apothecary stock. how much of the $5,000 would you invest in the company? (adapted from kelton & pennington (2012); bhandari, hassanein & deaves (2008)) likert 1-7 (none, about 1/4, about 1/3, about 1/2, about 2/3, about 3/4, all) 2. assume that you have $5,000 to invest in pharmacy stock. how much of the $5,000 would you invest in the company? (adapted from kelton & pennington (2012); bhandari, hassanein & deaves (2008)) likert 1-7 (none, about 1/4, about 1/3, about 1/2, about 2/3, about 3/4, all) 3. based on the financial statement presentations and accessibility, my willingness to invest in apothecary stock is: (adapted from kelton & pennington (2012); bhandari, hassanein & deaves (2008)) likert 1-7 (very unlikely to very likely) 4. based on the financial statement presentations and accessibility, my willingness to invest in pharmacy stock is: (adapted from kelton & pennington (2012); bhandari, hassanein & deaves (2008)) likert 1-7 (very unlikely to very likely) 5. financial statement accessibility affected my willingness to invest. likert 1-7 (strongly disagree to strongly agree) 6. financial statement availability affected my willingness to invest. likert 1-7 (strongly disagree to strongly agree) 7. financial statement interactivity affected my willingness to invest. likert 1-7 (strongly disagree to strongly agree) 8. financial statement presentations affected my willingness to invest. likert 1-7 (strongly disagree to strongly agree) australasian journal of information systems perdana 2018, vol 22, research article does visualization matter? 35 copyright: © 2018 perdana, robb & rohde. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. paper title (paper title style) australasian journal of information systems mwagwabi & hee jiow 2021, vol 25, selected papers from acis 2019 compliance with security guidelines in teenagers 1 compliance with security guidelines in teenagers: the conflicting role of peer influence and personal norms florence mwagwabi murdoch university, singapore f.mwagwabi@murdoch.edu.au jhee hee jiow singapore institute of technology singapore abstract what drives teenagers to comply with computer password guidelines? using an extended form of protection motivation theory (pmt) (rogers, 1983), we found that even if teenage computer users believe they are susceptible to being hacked, or that being hacked would be detrimental, it has no bearing on their password choices. other motives outside of pmt also drive teenage security behaviour. personal norms fully mediate the relationship between the perceived severity of threat and compliance intentions such that perceived severity is not sufficient to encourage compliance. teenagers must actually feel obligated to comply. while personal norms may encourage compliance, concerns about feeling embarrassed or ashamed if their social media accounts are hacked into actually encourages compliance. on the other hand, peer influence, such as the fear of being teased about someone hacking into their account, discourages compliance. our study contributes to understanding early security practices and highlights potential differences between adult and teenage behaviours to consider in future studies. for example, our findings suggest that password security guidelines alone will not suffice to ensure teenage compliance; they may need enforced password rules at the authentication level to eliminate any opportunity to violate password rules. our study will benefit children and parents as well as organizations that have changed work practices to enable employees to work from home, but which places children in danger of clicking on malicious links on their parents’ computers. to our knowledge, this is the first password security study that applies pmt to examine computer-based security behaviours in teenagers. keywords protection motivation theory, pmt, teenage cyber security behaviour, compliance behaviours in teenagers, password compliance intention, social media password practices, peer influence, personal norms, anticipated guilt, anticipated embarrassment 1 introduction while teenagers (13–16 years in this study) have access to all that the internet has to offer (dqinstitute, 2018; tsirtsis, tsapatsoulis, stamatelatos, papadamou, & sirivianos, 2016), most children have been exposed to some form of cyber threat (dq-institute, 2018). in 2018, the dqinstitute surveyed more than 80,000 children exploring the extent of their exposure to cyber risk. exacerbated by the always-on cell phone and social media usage, the dq-institute also found that children’s exposure to cyber threat is growing exponentially. almost 60% of children as young as eight are exposed to cyber risk, including cyber bullying, game addiction, and online sexual behaviour. the problem we find with major surveys such as this dqaustralasian journal of information systems mwagwabi & hee jiow 2021, vol 25, selected papers from acis 2019 compliance with security guidelines in teenagers 2 institute study is that they overlook other potential computer security risks, such as malware, phishing, and password attacks. with millions of children studying from home due to covid-19, exposure to cyber security threats such as clicking on infected links and downloading malware is a major problem (braue, 2020). cyber attacks targeting children are widely reported in the media (e.g., bbc, 2012; braue, 2020; corron, 2018), but research specifically related to teenage cyber risk is lacking. our study will benefit children and parents as well as organizations who have updated their work practices to enable employees to work from home but which places teenagers in danger of clicking on malicious links on their parents’ computers (braue, 2020). studies on cyber risk in teenage computer use have focused on threats such as problematic internet use (e.g., buckingham, banaji, carr, cranmer, & willett, 2005; dq-institute, 2018; soh, chew, koay, & ang, 2018), access to adult content (cybersafe.org 2016), cyber bullying and online harassment (e.g., dq-institute, 2018; lwin, li, & ang, 2012), but few cite cyber security threat (tsirtsis et al., 2016). in literature related to teenage cyber risk we identified three specific gaps. firstly, there is a lack of research on how to inspire better cyber security practices in teenagers. to address this, our research examines factors that motivate teenagers to comply with password guidelines on social media. we focus on online social media because applications such as facebook are a significant avenue through which teenagers are exposed to cyber security threat (tsirtsis et al., 2016). secondly, there is a lack of theory-based studies in teenage cyber security behaviour. our research addresses this by applying protection motivation theory (pmt) to investigate teenage compliance with password security guidelines. thirdly, as theory-based research on cyber security behaviour focuses on adults, this research uses pmt (rogers, 1983) to try and understand the protection motivation of teenagers using social media. teenagers were chosen as our target population, and facebook as the proxy for social media, because teenagers are more active on social media than adults. while instagram, youtube, and snapchat usage has increased, facebook is still widely used by teenagers (acma, 2016; ofcom, 2018). further, teenagers are likely to share personal information on facebook (acma, 2011) which increases their exposure to cyber security threat on this platform. understanding teenage cyber security behaviour is complicated because peer influence in this age group plays a strong role in teenage risk-taking behaviour (gardner & steinberg, 2005; soh et al., 2018; somerville, jones, ruberry, & dyke, 2013). since peers play such an essential role (acma, 2011; soh et al., 2018), this research extends pmt to examine just how peer influence can impact teenage security behaviour. investigating teenage cyber behaviour is challenging because age-group categorizations differ across different studies. for example, in the internet-use surveys described in acma (2016), data was collected from teenagers aged 14–17 years (australia), 12–17 years (usa), and 12–15 years (uk). for this research, undertaken in singapore, a teenager is defined as a person who falls within 13–18. in the infosec (is) domain, pmt (rogers, 1983) is used to investigate users’ underlying security perceptions and security practices (e.g., johnston, warkentin, & siponen, 2015; posey, roberts, & lowry, 2015). used initially in a health-related context, pmt has been used to investigate protection behaviours in all age groups (lwin et al., 2012). yet pmt applications in is security australasian journal of information systems mwagwabi & hee jiow 2021, vol 25, selected papers from acis 2019 compliance with security guidelines in teenagers 3 research have focused only on adult populations. teenagers and adults have different stages of cognitive-development (lwin et al., 2012; steinberg, 2008). since pmt requires the cognitive processing of cyber threat (rogers, 1983), threat appraisal in teenagers is expected to differ from that of adults. this study questions whether pmt can explain security compliance behaviour in teenagers, and whether the threat and coping appraisal abilities of teenagers display the same level of maturity as adults. a key ingredient in protection motivation is the perception of cyber threat and recommended safeguards (rogers, 1983). with early exposure to the internet, understanding teenage security perception is critical (tayouri, 2015). teenagers are aware that cyber safety education is important (acma, 2011, 2013), and are familiar with internet safety rules (livingstone, mascheroni, & staksrud, 2017). while they acknowledge receiving cyber safety training (cybersafe.org, 2016), and despite exhibiting some security awareness, teenagers possess poor security practices such as sharing passwords (acma, 2013). problems with poor security and password practices range from how valuable passwords are on the dark market, even when compared with credit card information (ablon, libicki, & golay, 2014), to how easily hackers can crack passwords from other known or leaked passwords (maor, 2020). this means poor security practices can have severe implications beyond accessing teenagers' online accounts (braue, 2020; jenkins, grimes, proudfoot, & lowry, 2014). by providing insights into security practices at an early age, our study will benefit a wide range of stakeholders. 2 literature review prior research using pmt has looked at individual adult cyber security behaviours (e.g., mwagwabi, mcgill, & dixon, 2018; thompson, mcgill, & wang, 2017) and security behaviour within organizations (e.g., menard, bott, & crossler, 2017; posey et al., 2015). lacking in the security literature, this study explores whether pmt (rogers, 1983) can be used to predict teenage cyber security behaviour. this is achieved by first examining how perceptions about security and security behaviour recommendations influence teenage behaviours, and whether pmt can predict protective behaviour in teenagers. 2.1 pmt theoretical background rogers (1983) developed pmt to explain health-related behaviour and identified perceived severity, perceived susceptibility, and response efficacy as key predictors of health protection motivation. bandura’s (1982) work on self-efficacy inspired rogers (1983) to include selfefficacy in the pmt framework. pmt proposes that threat severity and threat susceptibility appraisal are what drive protective behaviour. an intervening variable, fear, mediates perceived severity and perceived vulnerability. the revised pmt (rogers, 1983) model describes fear as being an emotional response to cyber security threat. many pmt applications in is security research exclude fear, yet it is an essential predictor of security behaviour (e.g., boss, galletta, lowry, moody, & polak, 2015; mwagwabi et al., 2018). this study consequently includes the fear of cyber threat as a direct predictor of protection behaviour and a function of perceived severity and perceived vulnerability. the perceived effectiveness of protection measures—i.e., response efficacy and confidence in implementing protection measures—i.e., self-efficacy, affect protective behaviour. on the other hand, response cost—i.e., the perceived inconvenience of protection measures—has a australasian journal of information systems mwagwabi & hee jiow 2021, vol 25, selected papers from acis 2019 compliance with security guidelines in teenagers 4 negative influence on protection motivation. these three variables—response efficacy, selfefficacy and response cost constitute pmt’s coping appraisal. applications of pmt in is security research (e.g., boss et al., 2015; menard et al., 2017; posey et al., 2015) support the link between coping appraisal and security behaviour. we examine how these three variables help to predict protection motivation in teenagers. 2.2 risk perceptions, coping appraisal, and cyber security behaviour in teenagers while some studies have examined the relationship between teenagers’ perceived cyber risk and online behaviour (lwin et al., 2012; youn, 2009), more studies are needed to better understand exactly how risk perceptions drive teenage security behaviour. using pmt, youn (2009) found that perceived cyber vulnerability influences privacy concerns and drives protection motivation. perceived severity, response efficacy, and self-efficacy drive teenagers’ intentions to prevent online harassment (lwin et al., 2012). these studies support using the pmt model to predict teenage security behaviour. as mentioned, when it comes to risk-taking behaviours on the internet, adults and teenagers may differ (gardner & steinberg, 2005). there is consequently a need to understand how well pmt predicts teenage security behaviour. 2.3 peer influence, personal norms, and cyber security behaviour in teenagers fear is an emotional response to threat (rogers, 1983) and influences security behaviour in adults (boss et al., 2015; mwagwabi et al., 2018). yet in teenagers, as found by burnett, bird, moll, frith, and blakemore (2009), the fear of threat has a less social role in guiding their behaviour than the fear of embarrassment and guilt (grasmick & bursik jr, 1990). teenage fear is manifested as guilt or shame when they violate personal norms (grasmick & bursik jr, 1990) that are associated with social expectations as well as expectations they have of themselves (schwartz, 1977). as a result, before teenagers engage in any behaviour, they first assess the cost of feeling embarrassed or guilty. schwartz (1977) notes that there is an overlap between social norms and personal norms, given that people acquire personal norms through social interaction. the conformity or violation of personal norms can lead to feelings of pride or guilt, respectively. on the other hand, the anticipated guilt from perceived peer evaluation can lead to concerns about experiencing it in the future, so their ensuing behaviour is motivated by avoiding feelings of guilt. personal norms appear to mediate the relationship between cyber risk perceptions and behaviour (de groot, 2010; de groot & steg, 2009; schwartz, 1977). personal norms can therefore explain how threat awareness translates into behavioural intentions (de groot, 2010). this study explores the potential mediating role of personal norms and intentions to comply with security guidelines. our findings should provide better insight into how the relationship between perceived severity and compliance evolves (hair, hult, ringle, & sarstedt, 2016). following de groot (2010), de groot and steg (2009), and schwartz (1977), in this study feelings of guilt or embarrassment both reflect and motivate teenage individuals to behave according to their personal norms (onwezen, antonides, & bartels, 2013; schwartz, 1977). australasian journal of information systems mwagwabi & hee jiow 2021, vol 25, selected papers from acis 2019 compliance with security guidelines in teenagers 5 teenagers also demonstrate a heightened self-consciousness when they know their peers are observing them (gardner & steinberg, 2005; somerville et al., 2013). in an experiment aimed at understanding teenage risk-taking behaviour, gardner and steinberg (2005) observed and compared the decisions teenagers make when alone against when their peers are present. this involved using a computer game requiring participants to make decisions in risky situations— such as running a red traffic light when driving—and found that despite awareness of the adverse consequences of e.g., running a red light, the teenagers took double the risk in the presence of peers than they took in their absence. steinberg (2008) attributes this risk-seeking propensity to how the teenage brain cognitive control system works, especially in their peers' company. teenage risk-taking behaviour and sensitivity to peer evaluation heightens between the ages of 13–16, and declines towards adulthood (steinberg, 2008). like gardner and steinberg (2005), somerville et al. (2013) found that sensitivity to peer evaluation peaks at around 17 years. using functional magnetic resonance imaging (fmri), somerville et al. (2013) found that peer evaluation can trigger a self-conscious emotion of embarrassment, which determines a teenager’s subsequent behaviour. this paper explores the role peer influence plays in teenage compliance to security guidelines. 3 research model and hypotheses development to achieve the objectives described in this study, a research model based on pmt (rogers, 1983) has been proposed. this extends pmt to explore how peer influence and personal norms affect teenage motivations for protecting their social media accounts. 3.1 hypotheses developed from the pmt framework the nine hypotheses explored in this study relate to the six variables that are defined in pmt (rogers, 1983). our contribution to protection motivation research lies not in testing the pmt model nor in the extended model. rather, it lies in the adaptation of pmt in the context of teenage cyber security behaviour. to the best of our knowledge, this is the first application of pmt in this context. our study provides new insights into whether pmt can also predict security behaviours in young citizens. given that pmt research on cyber security behaviour has primarily focused on adults, the literature reviewed in this section relates to adult security behaviour. pmt applications in is suggest that perceived severity and perceived vulnerability to cyber threat inspires better security behaviours. studies have shown that if adults believe they are susceptible to cyber threat, they are likely to follow security guidelines. for example, li et al. (2019) found that employees are willing to comply with security guidelines if they believe their organization could be vulnerable to cyber attack. adult users are also more likely to follow security guidelines if they believe the consequences would be severe. this finding is consistent with that of lee and larsen (2009), who found that business executives are more likely to apply security measures if they believe an attack on their organization would have severe consequences. perceived severity and perceived vulnerability to threat triggers the emotional response of fear, which in turn increases the likelihood of compliance behaviour (e.g., boss et al., 2015; mwagwabi et al., 2018; posey et al., 2015). this relationship has been shown in organizational settings where posey et al. (2015) found that when employees assess high degrees of cyber australasian journal of information systems mwagwabi & hee jiow 2021, vol 25, selected papers from acis 2019 compliance with security guidelines in teenagers 6 threat vulnerability and severity, their fear increased. this in turn increased their willingness to protect their company’s information assets. in the context of password use on the internet, mwagwabi et al. (2018) found that when users perceive an increased vulnerability to password attack and believe the consequences would be severe, their fear increased. this resulted in improving the quality of passwords. while research on teenage security behaviour is limited, lwin et al. (2012) found that in the context of online harassment, perceived severity influences teenagers’ intentions to protect against it. from these findings we hypothesize: h1: perceived vulnerability will have a positive effect on teenage intentions to comply with social media password guidelines. h2: perceived severity will have a positive effect on teenage intentions to comply with social media password guidelines. h3: fear of threats will have a positive effect on teenage intentions to comply with social media password guidelines. h4: perceived vulnerability will have a positive effect on teenage fear of threats. h5: perceived severity will have a positive effect on teenage fear of threats. response efficacy, response cost, and self-efficacy, which are referred to in this study as perceived password effectiveness, perceived cost, and password self-efficacy, also drive cyber security behaviour (e.g., boss et al., 2015; menard et al., 2017). while response efficacy and selfefficacy have a positive influence, response cost negatively influences protection motivation. the above three factors have also been shown to affect organizational and individual cyber security behaviour. in their study of how organizations can motivate employees to protect their information assets, posey et al. (2015) found that when employee response efficacy increases, their motivation to protect their organization’s information assets also increases. in addition, employees must believe that they have the necessary skills to protect their organization’s information assets from cyber threat. in the context of individual security behaviour, crossler, andoh-baidoo, and menard (2019) showed that when users perceive security measures as being easy to use or requiring minimal effort to implement, their likelihood to take action increases. accordingly, crossler et al. (2019) found that regardless of the culture, when individual computer users find security measures challenging to implement, they are less willing to take action. in teenagers, response efficacy and self-efficacy play a significant role in their intention to protect against online harassment (lwin et al., 2012). from these findings we hypothesize: h6: perceived password effectiveness will have a positive effect on teenage intentions to comply with social media password guidelines. h7: password self-efficacy will have a positive effect on teenage intentions to comply with social media password guidelines. h8: perceived cost will have a negative effect on teenage intentions to comply with social media password guidelines. 3.2 extending the pmt framework this study also explores the role of personal norms and peer influence in driving teenage security behaviour (elek, miller-day, & hecht, 2006; gardner & steinberg, 2005; somerville et al., 2013). following de groot (2010), de groot and steg (2009), and schwartz (1977), we found australasian journal of information systems mwagwabi & hee jiow 2021, vol 25, selected papers from acis 2019 compliance with security guidelines in teenagers 7 that encompassed in the construct of personal norms are the anticipated feelings of guilt or embarrassment which in turn motivate teenagers to behave in line with their norms (schwartz, 1977). this study proposes that the anticipated feelings of embarrassment or guilt result from a teenagers’ social media account being hacked will motivate them to comply with security guidelines. however, de groot and steg (2009) argue that personal norms are only experienced through these feelings when the individual is aware of the consequences of non-compliance, so only then can personal norms trigger security behaviour. personal norms therefore act as a mediating variable that enables cyber risk awareness to translate into behavioural compliance. in a separate study by de groot (2010), personal norms were also measured as feelings of guilt or embarrassment that mediate the link between risk awareness and behavioural intentions, explaining how threat awareness translates into behavioural intentions. de groot’s study also found that perceived norms mediate the relationship between risk perceptions and a willingness to take action. thus, the awareness of risk consequences influences the willingness to take action only through the emotional feeling of guilt or embarrassment (de groot, 2010). exploring the potential mediating role of teenage personal norms will bring better insight into how the relationship between perceived threat severity and compliance evolves (hair et al., 2016). by introducing personal norms as a mediating variable, this study can explore and reveal whether there is a relationship between perceived threat severity and teenage compliance. we hypothesize that an awareness of the consequences of password threats (perceived severity) influences personal norms. furthermore, we hypothesize that personal norms mediate the relationship between perceived severity and compliance intentions. h9: perceived severity will have a positive effect on teenage personal norms. h10: personal norms mediate the relationship between perceived severity and teenage intentions to comply with social media password guidelines. h11: personal norms will have a positive effect on teenage intentions to comply with social media password guidelines. as mentioned, peer influence plays an essential role in teenagers’ decision-making processes (albert, chein, & steinberg, 2013; soh et al., 2018). though there are different kinds of peer influence (brown, 2004), susceptibility to negative peer influence heightens during the teenage years (gardner & steinberg, 2005). peer pressure occurs when peers prescribe or model a particular behaviour. our study explores an intentional peer influence through teasing, gossiping, or conversation that reinforces the normative expectations of a specific groups (brown, 2004). since peer influence involves behaviour such as teasing someone if their online account is hacked, it is appropriate to include it in the model we use in this study. it follows that this study defines peer influence as a teenagers’ belief that their friends would ridicule or gossip about them if someone hacked into their social media account. given that the propensity to negative peer influence heightens in teenage years (gardner & steinberg, 2005), particularly through teasing (brown, 2004), we argue that peer pressure incites teenagers to expose themselves to risk which could lead to violation of security guidelines. we argue that for teenagers between ages 13–16 in our study, being teased about being hacked incites risky behaviour (gardner & steinberg, 2005) and discourages them from complying. these findings give rise to the following hypotheses: australasian journal of information systems mwagwabi & hee jiow 2021, vol 25, selected papers from acis 2019 compliance with security guidelines in teenagers 8 h12: peer influence will have a negative effect on teenage intentions to comply with social media password guidelines. the pmt framework is generally used to predict behavioural intentions, operationalized as protection motivation (maddux & rogers, 1983; rogers, 1975, 1983). this assumes that by measuring behavioural intentions we can predict actual behaviour (prentice-dunn & rogers, 1986). a meta-analysis of pmt research suggests that pmt’s influence on the results was significant, regardless of whether a study measures behavioural intention or actual behaviour as the dependent variable (floyd, prentice-dunn, & rogers, 2000; milne & milne, 2000). while behavioural intentions are assumed to predict behaviour (ajzen, 1991; fishbein & ajzen, 2010), their actual effects are marginal (floyd et al., 2000). as such, pmt studies began to examine behavioural intentions as well as actual behaviour (boss et al., 2015; milne & milne, 2000). in this study, we examine the degree to which teenagers report their willingness to comply with social media security guidelines. we also examine the extent to which their intentions predict how they currently comply with social media password guidelines. to do this, we operationalized protection motivation as the intention to comply (e.g., i intend to follow….”) and actual behaviour as the teenagers’ current compliance with social media password guidelines (e.g., “i always follow the password guidelines…”). please see the appendix for the full list of measurement items. from these findings we hypothesize: h13: intentions to comply will have a positive effect on actual teenage compliance with social media password guidelines. 4 methodology in this section, we describe this study’s participants, its design, and its measurement items. our participants were recruited by touch cyberwellness (tcw), singapore, through cyber wellness courses they provide to participating secondary schools across singapore. we randomly selected 246 students aged 13–16 years to participate in the study between november 2017 and may 2018. to ensure confidentiality and anonymity a randomly generated 6-digit index number was assigned to each participant before they completed our survey. we chose the cut-off ages of 13 and 16 for two reasons. firstly, because facebook’s starting age is 13 (dq-institute, 2018). secondly, the peer influence on risky teenage behaviour peaks between 13 and 16 years (gardner & steinberg, 2005) and then declines with age (steinberg, 2008). 4.1 measurement development to ensure the validity and reliability of this study’s measurement items, we selected previously validated items and reworded them to reflect password threats. these items were measured on a 7-point likert scale from (1) ‘strongly disagree’ to (7) ‘strongly agree.’ for example, we adapted the variables of perceived vulnerability (e.g., “there is a chance that someone could hack into my social media account”) and perceived severity (e.g., “i believe that if someone successfully hacked into my social media account the consequences would be severe”) from zhang and mcdowell (2009). the fear of threat (e.g., “the thought of someone hacking into any of my social media account frightens me”) was adapted from milne, orbell, and sheeran (2002). perceived password effectiveness (e.g., “making sure that my passwords are strong will protect my social media from hackers”), password self-efficacy (e.g., “i am australasian journal of information systems mwagwabi & hee jiow 2021, vol 25, selected papers from acis 2019 compliance with security guidelines in teenagers 9 confident that i can protect my social media account from hackers”) and perceived cost (e.g., “strong passwords take too much effort to create”), were adapted from zhang and mcdowell (2009), compeau and higgins (1995), and milne et al. (2002), respectively. from schwartz (1977), we adapted items to reflect anticipated feelings of embarrassment or guilt associated with personal norms (e.g., “i would feel guilty if my account was hacked”). peer influence in this study relates to casual conversations between peers that involve teasing or gossiping (brown, 2004), (e.g., “my friends would ridicule me if my social media account was hacked.”). please see appendix for the full list of measurement items and item loadings. 5 data analysis and results this section entails an assessment of this study’s measurement items and its structural model. section 6 then discusses the results of our hypotheses and the implications of this study. the data was analysed using partial least squares, using smartpls 3.2.8 software (ringle, wende, & becker, 2015) as pls is appropriate for analysing complex models like ours with its many parameters and relationships (lowry & gaskin, 2014). that pls predicts future behaviour rather than checking model fit (cb-sem) also made it more appropriate to use for our study. in total, 246 students aged 13–16 completed this study’s survey on qualtrics. due to our participants’ ages, and following our ethics requirements, we did not collect gender statistics. only 9% of the participants reported that their social media account had ever been hacked. when asked if they knew anyone else whose account had been hacked, 40% said yes. a small proportion (25%) indicated that they do not use social media every day. to assess the measurement items' significance and relevance, we used the pls bootstrapping procedure for 5,000 subsamples (hair et al., 2016). the measurement model was evaluated by using the recommended pls-sem measurement metrics (hair et al., 2016) and is discussed in the following. to confirm convergent validity, we used the average variance extracted (ave) value—i.e., the average squared value across explained variance of the measurement items—and to assess each individual item’s internal consistency we used composite reliability (cr) statistics. to establish convergent validity, ave should be > or equal to 0.50, implying that each latent variable accounts for at least 50% of the indicators’ variance. this study’s ave values were above 0.50, thus confirming convergent validity. the cr statistics and cronbach’s alpha (a measure of internal consistency) were at least 0.70, which confirms its internal consistency. to assess discriminant validity, we used heterotrait-monotrait ratio (htmt) statistics. discriminant validity tests whether a latent variable does not correlate too highly with other latent variables, and whether any empirical distinction between latent variables exists (hair et al., 2016; henseler, ringle, & sarstedt, 2015). as recommended by henseler et al. (2015), this study’s htmt values were below 0.850, and neither of the confidence intervals values included 1, suggesting that our model has no discriminant validity issues. pls has no established goodness-of-fit statistics (hair et al., 2016). but as this study is not testing or comparing competing models, assessing goodness-of-fit is not necessary (hair et al., 2016). nonetheless, we elected to report the following recommended goodness-of-fit statistics for pls (hair, risher, sarstedt, & ringle, 2019; henseler & sarstedt, 2013) the srmr, which measures the gap between the estimated and the observed model, should be ≤ 0.08 (0.10); and australasian journal of information systems mwagwabi & hee jiow 2021, vol 25, selected papers from acis 2019 compliance with security guidelines in teenagers 10 the nfi should be > 0.95 (0.90). though this study’s srmr and nfi values were within what henseler and sarstedt recommend (2013), we report them cautiously (hair et al., 2019). we also assessed the structural model for collinearity issues, path significance and direction, and explanatory power. for collinearity issues, we evaluated each latent variable separately and ensured the vif (variance inflation factor) values were ≤ 5 so no collinearity issues occurred in the structural model. the highest outer vif was 3.778 (fear of threat item), and the lowest outer vif value was 1.421 (intentions to comply item). common method bias checks whether surveys elicit any potential social desirability to answer questions in a certain way, or whether a surveys’ instructions influence participants’ answers. to check for common method bias, we examined the inner vif values to ensure they were all <3 (kock, 2015) and found no evidence of method bias. the highest inner vif was 1.669 (perceived password effectiveness), and the lowest inner vif value was 1.000 (intentions to comply). based on the inner vif values reported here, we found no evidence of measurement method bias. then using the pls bootstrapping method we assessed our model’s significance of path coefficients and whether its relationship directions are relevant. all p-values (p < 0.05) for the indicators are significant, confirming the relevance of the indicators. a full list of these results is available upon request. hypothesized relationship r2 tstatistic explanatory power f2 strength h4 perceived vulnerability → fear of threat 0.355 1.386 moderate 0.051 weak effect h5 perceived severity → fear of threat 0.355 2.259 moderate 0.442 strong effect h1 perceived vulnerability → intentions to comply 0.567 0.204 moderate 0.007 weak effect h3 fear of threat → intentions to comply 0.567 0.043 moderate 0.001 weak effect h2 perceived severity → intentions to comply 0.567 0.266 moderate 0.008 weak effect h9 perceived severity → personal norms 0.103 1.864 weak 0.114 weak effect h6 perceived password effectiveness → intentions to comply 0.567 0.910 moderate 0.075 weak effect h7 password self-efficacy → intentions to comply 0.567 1.278 moderate 0.259 moderate effect h8 perceived cost → intentions to comply 0.567 0.164 moderate 0.003 weak effect h11 personal norms → intentions to comply 0.567 0.944 moderate 0.071 weak effect h12 peer influence → intentions to comply 0.567 0.887 moderate 0.054 weak effect h13 intentions to comply → actual compliance 0.712 0.694 moderate 2.473 strong effect table 1. explanatory power and effect size of the proposed model finally, we assessed the model’s predictive relevance (hair et al., 2019) by using r2 and effect size f2. r2 measures the model’s predictive accuracy and reflects variance in the endogenous latent variables, as explained by all exogenous latent variables. effect size f2 determines how much certain independent variables in the model explain the dependent variables, thereby australasian journal of information systems mwagwabi & hee jiow 2021, vol 25, selected papers from acis 2019 compliance with security guidelines in teenagers 11 contributing to the r2 level, where f2 ≤ 0.02 < 0.15 is a weak effect; 0.15 to < 0.35 is a moderate effect; ≥ 0.35 is a strong effect. as indicated by the r2 values in table 1, the explanatory power of this study’s model is weak to moderate, while the f2 effect sizes range between weak and strong. the paths between the pmt constructs, peer influence, and personal norms produced an r2 of 0.567 for intentions to comply. the path between intentions to comply and actual compliance was significant (0.844, p=0.000) and yielded an r2 of 0.712. this means our model accounted for 56.7% and 71.2% of the variance for intentions to comply and actual password compliance, respectively. this is consistent with boss et al. (2015), who also found a high correlation between computer users’ backup intentions and their actual backup behaviour. we also examine the effect size f2 to determine how much the independent variables of this study’s model explain the primary dependent variable of a participants’ intention to comply with password guidelines. password self-efficacy had the most effect size f2 of 0.259, indicating a moderate effect on our target dependent variable (intention to comply). while most hypothesized paths are significant, the effect sizes are small, which further indicates a mediator relationship (hair et al., 2016). table 2 shows that this study’s h10 hypothesis—that personal norms mediate the relationship between perceived severity and intentions to comply with social media password guidelines— was supported. to assess this hypothesized mediator role, we examined its direct and indirect effects. the direct effect of perceived severity on compliance intention is not significant. the paths between perceived severity to personal norms to compliance intentions are significant, and the indirect effects are also significant (p=0.048; t=1.98). this indicates a full mediation (carrión, nitzl, & roldán, 2017; zhao, lynch jr, & chen, 2010), and only the indirect effect through the hypothesized mediator variable of personal norms exists. this suggests that the impact of perceived threat severity on compliance intentions is fully transmitted with the help of personal norms. mediated path path coefficient t-statistic std. dev effect perceived severity → intentions to comply -0.046 ns 0.699 0.105 full mediation perceived severity → personal norms 0.275*** 4.390 0.073 personal norms → intentions to comply 0.183* 2.328 0.111 *p<0.05; **p < 0.01; ***p<0.001 table 2. results of the hypothesized mediated relationship while the following section presents the results of our hypotheses, a discussion of these results is outlined in section 6. table 3 also presents the results of our path analysis. table 3 shows that h4 and h5 are supported as both perceived vulnerability and perceived severity have a significant impact (p=0.004, ß=0.183; p=0.000, ß=0.540) on the fear of threat. it shows that h3 is not supported, however, the fear of threat has no significant effect (p=0.795, ß=-0.027) on intentions to comply. as the threat component, that is perceived vulnerability and perceived severity have no significant influence (p=0.463, ß=0.063; p=0.485, ß=0.074) on intentions to comply, h1 and h2 are also not supported. as for the coping appraisal component, only perceived cost has no significant negative influence (p=0.554, ß=-0.046) on intentions to comply, meaning h8 is not supported either. perceived password effectiveness has a significant effect (p=0.039, ß=0.275) australasian journal of information systems mwagwabi & hee jiow 2021, vol 25, selected papers from acis 2019 compliance with security guidelines in teenagers 12 on intentions to comply, which supports h6. password self-efficacy has a significant effect (p=0.000, ß=0.499) on intentions to comply, supporting h7. intentions to comply has a strong and significant impact (p=0.000, ß=0.844) on actual compliance with password guidelines, supporting h13. among all antecedent constructs, password self-efficacy has the strongest total effect (0.421) on actual compliance. hypothesized paths path coefficient std. dev tstatistic p-value support perceived vulnerability → fear of threat 0.183 0.062 2.925 0.004 supported perceived severity → fear of threat 0.540 0.077 6.986 0.000 supported perceived vulnerability → intentions to comply 0.063 0.086 0.736 0.462 not supported fear of threat → intentions to comply -0.027 0.103 0.260 0.795 not supported perceived severity → intentions to comply 0.074 0.105 0.699 0.485 not supported perceived severity → personal norms 0.320 0.073 4.390 0.000 supported perceived password effectiveness → intentions to comply 0.275 0.133 2.066 0.039 supported password self-efficacy → intentions to comply 0.499 0.132 3.790 0.000 supported perceived cost → intentions to comply -0.046 0.077 0.592 0.554 not supported personal norms → intentions to comply 0.259 0.111 2.328 0.020 supported peer influence → intentions to comply -0.219 0.099 2.203 0.028 supported intentions to comply → actual compliance 0.844 0.059 14.337 0.000 supported table 3. path coefficients the paths for our proposed pmt model extension are all significant. peer influence has a significant negative impact (p=0.028, ß=-0.219) on intentions to comply, supporting h12. and personal norms have a significant positive influence (p=0.020, ß=0.259) on intentions to comply with password guideline, which supports h11. the results for the structural model of this research are shown in figure 1. however it shows that the threat component of our adapted pmt (rogers, 1983) model had no direct influence on intentions to comply, rejecting h1, h2, and h3. perceived vulnerability, perceived severity, and fear of threat had no impact on the teenagers’ intentions to comply with social media password guidelines. as hypothesized in h4 and h5, fear of threat stems from perceptions of vulnerability and threat severity. the coping appraisal of the research model we derived from pmt (rogers, 1983) better predicts compliance intentions. perceived password effectiveness and password self-efficacy had a significant influence on compliance intentions, which supports h6 and h7. this suggests that teenagers’ motivation to comply with social media password guidelines depends on their belief in whether the recommended password security measure is effective, and they are confident in protecting their social media accounts from hackers. perceived cost did not affect compliance intentions, which rejects h8. as hypothesized, peer influence has a significant but negative impact on teenagers’ compliance intentions, supporting h12. contrary to peer influence, we found that personal norms have a positive impact on teenagers’ compliance intentions, which supports h11. australasian journal of information systems mwagwabi & hee jiow 2021, vol 25, selected papers from acis 2019 compliance with security guidelines in teenagers 13 intentions to comply fear of threat perceived vulnerability perceived severity perceived password effectiveness password self-efficacy perceived cost h3: -0.027 (p = 0.795) h1: 0.063 (p = 0.462) h2: 0.074 (p = 0.485) h5: 0.540 (p = 0.000) h4: 0.183 (p = 0.004) h8: -0.046 (p = 0.554) h6: 0.275 (p = 0.039) h7: 0.499 (p = 0.000) peer influence pe rc ep tio ns a b ou t so ci al m ed ia p as sw or d th re at s pe rc ep tio ns a b ou t pa ss w or d g ui de lin es actual password compliance h13: 0.844 (p = 0.000) r2=0.567 = significant path; = non-significant path; personal norms h12: -0.219 (p = 0.028)h11: 0.259 (p = 0.020) r2=0.712 r2=0.355 h9: 0.320 (p = 0.000) r2=0.103 intentions to comply perceived severity personal norms0.275 (p -0.046 ns 0.183* (p h10: full mediation through personal norms figure 1. proposed research model results this study also explored the extent to which teenager’s compliance intentions can predict their actual compliance. behavioural intentions are assumed to predict behaviour (ajzen, 1991; fishbein & ajzen, 2010). however, since the effects of the relationship are marginal (floyd et al., 2000), pmt studies examine behavioural intentions as well as actual behaviour (boss et al., 2015; milne & milne, 2000). consistent with boss et al. (2015), we found a strong relationship between intentions and compliance, which supports h13. 6 discussion and conclusion we identified three gaps in literature on cyber security behaviour, including that there is little research focusing on teenage security behaviour and also a lack of theory-based research focusing on teenage security behaviour. to address this, and as discussed, we proposed a research model derived from pmt (rogers, 1983) to investigate what drives teenagers to comply with security guideless on social media. given the role peers play in teenagers’ protective and risky security behaviours (acma, 2011; gardner & steinberg, 2005; soh et al., 2018; somerville et al., 2013), our model extended the pmt model to explore the roles that personal norms and peer influence might play. in all, our model investigated the role of australasian journal of information systems mwagwabi & hee jiow 2021, vol 25, selected papers from acis 2019 compliance with security guidelines in teenagers 14 security perceptions, perceptions about security recommendations, peer influence, and personal norms on teenage compliance intentions. we found that perceived vulnerability, perceived severity, and fear of cyber threat do not influence teenagers’ compliance intentions. also, if teenagers believe there is a chance someone might hack into their social media account, it will not affect their compliance with security guidelines. and they are surprisingly unlikely to follow password measures recommended by their social media service, even if they believe being hacked would be detrimental or that the consequences of having personal information stolen would be severe. contrary to pmt (rogers, 1983) and findings in other is-related pmt studies (e.g., boss et al., 2015; mwagwabi et al., 2018; posey et al., 2015), we found that the fear of threat did not influence compliance intentions. this is an interesting revelation; even the thought of someone hacking their account or successfully cracking their passwords does not evoke enough emotional response to make teenagers want to protect their social media account. this finding highlights a potential difference between adults and teenagers in threat appraisal and decisions about whether to comply with or ignore recommended security measures. a likely reason for this finding is that teenagers tend to underestimate their online vulnerability (lwin et al., 2012). in fact, of the 246 teenagers who participated in our study, only 9% reported their social media account had been hacked. yet 40% indicated they knew someone else whose social media account had been hacked. this tendency to underestimate their susceptibility to cyber threat is a phenomenon weinstein (1984) described as ‘optimistic bias’; individuals— like the teenagers in our study—tend to believe that adverse events are less likely to happen to them. this tendency to discount their online susceptibility appears to affect their motivation to carry out recommended security guidelines. another possible reason for this is that the emotional response to threat proposed in pmt (rogers, 1983) may play a lesser role in guiding teenage behaviour. our findings show that teenage emotions such as embarrassment and guilt play a more significant role (burnett et al., 2009). consistent with our findings, the anticipated embarrassment or guilt associated with personal norms better predict teenagers' security compliance. while fear of threat did not influence our participants, feeling embarrassed or ashamed about an account being hacked appears to be a better driver of teenagers' compliance. it is possible to derive from this that the teenage cognitive threat appraisal process proposed in pmt (rogers, 1983) differs from adults, likely because they are at different stages of cognitive development (lwin et al., 2012; steinberg, 2008); the brain’s cognitive control system changes during the teenage years such that, and as steinberg (2008) explains, the adolescent stage is when the brain is sensitive to peer evaluation and seeks risky behaviour. this explains why teenagers’ concern for feeling embarrassed or ashamed is a better predictor of their compliance intentions. a key revelation that stems from this is that teenagers’ concern about feeling embarrassed or ashamed if their account was hacked into is highly likely to drive security behaviour, while the fear of someone actually hacking into their social media account does not motivate them take security measures. consistent with prior is security research (e.g., boss et al., 2015; posey et al., 2015), we found that self-efficacy and perceptions that strong passwords will protect social media accounts have a positive impact on the teenagers’ willingness to comply with social media password guidelines i.e., coping appraisal is a good predictor of teenage compliance behaviour . this may be because social media use is prevalent among teenagers (dq-institute, 2018). our results show that participants regularly use social media, which could explain their enhanced australasian journal of information systems mwagwabi & hee jiow 2021, vol 25, selected papers from acis 2019 compliance with security guidelines in teenagers 15 confidence (self-efficacy) and their awareness of the value of available protective measures (perceived effectiveness). one coping appraisal variable which is not supported in our study is perceived cost. whether teenagers perceive that strong passwords are challenging to manage or that following the recommended security measures would require too much effort did not affect their willingness to follow strict password guidelines. one potential reason is linked to the teenagers’ confidence level in their social media use (dq-institute, 2018). our findings reveal an interesting relationship between perceived threat severity and compliance intentions through personal norms: personal norms actually mediate the effect of perceived threat severity. this means that awareness of the adverse consequences of password security threat alone does not lead to compliance i.e., it is not sufficient to influence security behaviour in teenagers. however, they feel morally obligated or guilty about not complying, which means their awareness of the adverse consequences of security threats plays a vital role in forming their personal norms. so it is only when these two conditions are met that personal norms translate into compliance. since risk-taking behaviour increases from childhood and heightens between 13–16 years of age (e.g., gardner & steinberg, 2005; somerville et al., 2013; steinberg, 2008), we explored the role peer influence plays in security behaviour. our findings suggest peer influence plays a significant role in teenagers’ risky security behaviours by negatively influencing their willingness to protect their social media accounts. peer influence, which in this study relates to being teased about their accounts being hacked, appears to discourage teenagers from following recommended security measures. interestingly, our findings are consistent with prior research (e.g., gardner & steinberg, 2005; somerville et al., 2013; steinberg, 2008), meaning we all agree that risky behaviour linked to peer influence peaks at 13–16 years of age. even when teenagers at this age are aware of the adverse consequences of cyber risk (gardner & steinberg, 2005), risk-taking, particularly in their peers' presence, increases. this explains why our study found that peer pressure, such as when teenagers are teased about their social media accounts being hacked, discourages compliance. this is an interesting finding that opens new avenues for future research about is security behaviours and whether the negative peer influence observed in our 13–16 year-old participants would decline with age. 7 implications for research and practice this research explored whether pmt (rogers, 1983) can be used to predict security behaviour in teenagers. we found that while coping appraisal predicts compliance in teenagers, there are other motives outside of pmt that drive their security behaviour. our study reveals two important research implications. firstly, personal norms are a strong predictor of teenage security behaviours as they are driven more by feelings of guilt or embarrassment about their social media account being hacked than by their fear of being hacked. secondly, following the risky behaviour that occurs between ages 13–16 (gardner & steinberg, 2005), and because the teenagers in our study are in this age group, we argued that being teased about being hacked discourages teenagers from complying with recommended security guidelines and found this hypothesis to be true. this important finding suggests potential differences between adults and teenagers in security compliance behaviour. peer influence has a dominant effect on risky behaviour in teenagers. for example, albert et al. (2013) used brain australasian journal of information systems mwagwabi & hee jiow 2021, vol 25, selected papers from acis 2019 compliance with security guidelines in teenagers 16 imaging to study teenagers’ susceptibility to peer pressure and found they have a higher propensity for risky online behaviour when observed by peers. our study’s findings suggest a difference between teenage and adult motives that drive their compliance behaviour. this highlights important implications for future research on is security behaviour, which would benefit from investigating differences in adult and teenage behaviours. these potential differences draw interesting questions about whether the phenomena observed in our 13 –16 year-old participants would hold in adults, and how this could be important in tailoring cyber security training for teenagers. our study used pmt (rogers, 1983), a protection motivation framework, to explain the underlying security motivation of teenagers. while our data largely supports our hypotheses, it is important to note that our model yielded weak to moderate explanatory power and weak to strong effect sizes. though our hypothesized relationships were supported, the relationship strength ranged from weak to strong (benitez, henseler, castillo, & schuberth, 2020; hair et al., 2019). while it is unlikely for all variables to have high effects sizes (benitez et al., 2020), our study found that password self-efficacy had the most yet moderate effect size on the teenagers’ willingness to comply with password guidelines. consequently, while personal norms, peer influence, and perceived effectiveness of password measures influence teenage compliance intentions, the magnitude of impact these constructs had was small. from the motivational theory standpoint of our research model's case, the effectiveness of our tested variables to motivate teenagers to comply with security guidelines is medium to small. this raises questions about whether some motives are stronger than others, and if so, which ones they are. while our study found that pmt’s coping appraisal is a good predictor of teenagers' security compliance, future studies could explore if motives beyond the pmt model may drive their security behaviour. self-determination theory (sdt) is one example of a motivational framework that could be used in future research of security behaviour. sdt provides a differentiated taxonomy of extrinsic motivation that explains why different motives produce different attitudes and outcomes (ryan & deci, 2000; sheldon, osin, gordeeva, suchkov, & sychev, 2017). for example, following sdt’s proposed extrinsic motivation continuum, peer influence—the least effective of the motivation types in our study—aligns with external motivation as teenagers are motivated by what others think. this motivation type was also found to have the smallest effect size of all the significant relationships in our research. with personal norms, on the other hand, teenagers are motivated to act because of internal pressure, such as feeling ashamed by not acting. according to sdt (sheldon et al., 2017), this makes personal norms a more stable motive than peer influence. this is consistent with our results which found personal norms yielding a slightly larger effect size than peer influence. our studies also found that perceived password effectiveness yielded a slightly larger effect size than personal norms. echoing the sdt taxonomy of extrinsic motivation (sheldon et al., 2017), this indicates that perceived password effectiveness is a more stable motive than peer influence and personal norms. if teenagers see the value of such recommended measures, they are more motivated to comply with password guidelines. in the context of our study, while protecting one's social media or online accounts is necessary, nothing about creating strong passwords is inheritably enjoyable. people do not do it out of intrinsic motivation (ryan & deci, 2000). this also indicates that teenagers must act on extrinsic motivation. as our study shows, motivation to comply with security guidelines australasian journal of information systems mwagwabi & hee jiow 2021, vol 25, selected papers from acis 2019 compliance with security guidelines in teenagers 17 varied in its effect, which means that future research could explore what forms of motivation are effective and which are sustainable. for practitioners of cybersecurity training for teenagers, this study provides insight into security compliance behaviours in teenagers. when using persuasive communication, the findings reveal that teenagers and adults are possibly different in the way they respond to cyber security threat stimuli. consistent with burnett et al. (2009), we found that fear has a less social role in guiding teenage behaviour concerning these threats. instead we found that teenagers make decisions based on emotional feelings, so personal norms are more relevant to their decisions about protecting their social media than their fear of security threat. this suggests that there is value in targeting emotions based on personal norms rather than the fear of threat in practice. in terms of coping mechanisms, as teenagers’ perceived that recommended security measures are favourable, and their confidence in protecting their social media account increases, security guidelines are followed more frequently. our study suggests that confidence in protecting their social media has the most influence in teenagers’ willingness to follow recommended security measures. cyber security campaigns for teenagers should therefore focus on enhancing teenagers’ ability to create and maintain strong passwords. further, cyber security training for teenagers should understand that teenagers will comply with security guidelines if they believe the recommended security measure is valuable. lastly, we found that teenagers’ security practices are influenced by their peers. since peer influence on risky behaviour is heightened between 13–16 years of age, as suggested by gardner and steinberg (2005), it may be necessary to remove any opportunity for risky security behaviour. in the context of password security for teenagers in this age group, instead of relying on security guidelines, password rules should be enforced at the authentication level to eliminate any opportunity to violate password rules. 8 limitations the project faced some significant data collection issues, including (i) difficulty finding schools willing to participate in a cyber security study. (ii) participant feedback showed they found the surveys lengthy and tedious. this is unsurprising as the participants were 13–16 years of age. though we conducted a pilot test, future projects should focus on gauging a survey’s completion time and target 10– 15-minutes. (iii) the data we collected was only from singapore. this is because cultural differences can play a role in individual security behaviours (menard, 2018). future studies could consider examining security behaviours in teenagers across different countries. (iv) the data we collected was from people between 13–16 years of age. though we chose a cut-off age of 13 following facebook’s minimum age requirement, future studies could consider a wider age range, such as 13–18 years of age. (v) we used facebook as a proxy for wider social media use. while facebook use is prevalent among teenagers, youtube, instagram, or snapchat are also widely used (dq-institute, 2018; ofcom, 2018). future studies could consider teenagers’ cyber security behaviours on platforms such as instagram, youtube, and snapchat. australasian journal of information systems mwagwabi & hee jiow 2021, vol 25, selected papers from acis 2019 compliance with security guidelines in teenagers 18 references ablon, l., libicki, m. c., & golay, a. (2014). markets for cybercrime tools and stolen data: hackers' bazaar. rand corporation. acma. (2011). young australians’ experience of social media: qualitative research report. retrieved from https://www.acma.gov.au/-/media/mediacomms/report/pdf/like-post-shareyoung-australians-experiences-of-social-media-qualitative-research-report.pdf?la=en acma. (2013). young australians’ experience of social media: quantitative research report. retrieved from https://www.acma.gov.au/-/media/mediacomms/report/pdf/like-postshare-young-australians-experience-of-social-media-quantitative-researchreport.pdf?la=en acma. (2016). aussie teens and kids online. retrieved from https://www.acma.gov.au/theacma/engage-blogs/engage-blogs/researchsnapshots/aussie-teens-and-kids-online ajzen, i. (1991). the theory of planned behavior. organizational behavior and human decision processes, 50(2), 179–211. albert, d., chein, j., & steinberg, l. (2013). the teenage brain: peer influences on adolescent decision making. current directions in psychological science, 22(2) 114–120. bbc. (2012). hackers spread malware via children's gaming websites. bbc technology news. retrieved from https://www.bbc.com/news/technology-16576542 benitez, j., henseler, j., castillo, a., & schuberth, f. (2020). how to perform and report an impactful analysis using partial least squares: guidelines for confirmatory and explanatory is research. information & management, 57(2), 103–168. boss, s., galletta, d., lowry, p. b., moody, g. d., & polak, p. (2015). what do systems users have to fear? using fear appeals to engender threats and fear that motivate protective security behaviors. mis quarterly (misq), 39(4), 837–864). braue, d. (2020). cyber attackers target children at home. australian computer society, ict news. retrieved from https://ia.acs.org.au/article/2020/cyber-attackers-target-childrenat-home.html brown, b. b. (2004). adolescents’ relationships with peers. handbook of adolescent psychology, 2. 363–394. buckingham, d., banaji, s., carr, d., cranmer, s., & willett, r. (2005). the media literacy of children and young people: a review of the research literature. pp. 1–76. london: london knowledge lab. burnett, s., bird, g., moll, j., frith, c., & blakemore, s.-j. (2009). development during adolescence of the neural processing of social emotion. journal of cognitive neuroscience, 21(9), 1736–1750. carrión, g. c., nitzl, c., & roldán, j. l. (2017). mediation analyses in partial least squares structural equation modeling: guidelines and empirical examples. in partial least squares path modeling. (pp. 173–195). springer, cham. compeau, d. r., & higgins, c. a. (1995). computer self-efficacy: development of a measure and initial test. mis quarterly, 19(2), 189–211. australasian journal of information systems mwagwabi & hee jiow 2021, vol 25, selected papers from acis 2019 compliance with security guidelines in teenagers 19 corron, l. (2018). social cyber threats facing children and teens in 2018. retrieved from https://staysafeonline.org/blog/social-cyber-threats-facing-children-teens-2018/ crossler, r. e., andoh-baidoo, f. k., & menard, p. (2019). espoused cultural values as antecedents of individuals’ threat and coping appraisal toward protective information technologies: study of us and ghana. information & management, 56(5), 754–766. cybersafe.org. (2016). children's internet usage study. center for cyber safety and education. retrieved from https://www.iamcybersafe.org/s/parent-research de groot, j. i. m. (2010). morality and nuclear energy: perceptions of risks and benefits, personal norms, and willingness to take action related to nuclear energy morality and nuclear energy. risk analysis, 30(9), 1363–1373. doi:10.1111/j.1539-6924.2010.01419.x de groot, j. i. m., & steg, l. (2009). morality and prosocial behavior: the role of awareness, responsibility, and norms in the norm activation model. the journal of social psychology, 149(4), 425–449. dq-institute. (2018). outsmart the cyber-pandemic: empower every child with digital intelligence by 2020. retrieved from https://www.dqinstitute.org/wp-content/uploads/2018/08/2018dq-impact-report.pdf elek, e., miller-day, m., & hecht, m. l. (2006). influences of personal, injunctive, and descriptive norms on early adolescent substance use. journal of drug issues, 36(1), 147– 172. fishbein, m., & ajzen, i. (2010). prediction and change of behavior: the reasoned action approach. new york: psychology press. floyd, d., prentice-dunn, s., & rogers, r. (2000). a meta-analysis of research on protection motivation theory. journal of applied social psychology, 30(2), 407–429. gardner, m., & steinberg, l. (2005). peer influence on risk taking, risk preference, and risky decision making in adolescence and adulthood: an experimental study. developmental psychology, 41(4), 625. grasmick, h. g., & bursik jr, r. j. (1990). conscience, significant others, and rational choice: extending the deterrence model. law and society review, 837–861. hair, j. f., hult, g. t. m., ringle, c., & sarstedt, m. (2016). a primer on partial least squares structural equation modeling (pls-sem). sage publications. hair, j. f., risher, j. j., sarstedt, m., & ringle, c. m. (2019). when to use and how to report the results of pls-sem. european business review, 31(1), 2-24. doi:10.1108/ebr-11-2018-0203 henseler, j., ringle, c. m., & sarstedt, m. (2015). a new criterion for assessing discriminant validity in variance-based structural equation modeling. journal of the academy of marketing science, 43(1), 115–135. henseler, j., & sarstedt, m. (2013). goodness-of-fit indices for partial least squares path modeling. computational statistics, 28(2), 565–580. jenkins, j. l., grimes, m., proudfoot, j. g., & lowry, p. b. (2014). improving password cyber security through inexpensive and minimally invasive means: detecting and deterring password reuse through keystroke-dynamics monitoring and just-in-time fear appeals. information technology for development, 20(2), 196–213. australasian journal of information systems mwagwabi & hee jiow 2021, vol 25, selected papers from acis 2019 compliance with security guidelines in teenagers 20 johnston, a. c., warkentin, m., & siponen, m. (2015). an enhanced fear appeal rhetorical framework: leveraging threats to the human asset through sanctioning rhetoric. mis quarterly, 39(1), 113-134. kock, n. (2015). common method bias in pls-sem: a full collinearity assessment approach. international journal of e-collaboration (ijec), 11(4), 1–10. lee, y., & larsen, k. r. (2009). threat or coping appraisal: determinants of smb executives decision to adopt anti-malware software. european journal of information systems, 18(2), 177–187. li, l., he, w., xu, l., ash, i., anwar, m., & yuan, x. (2019). investigating the impact of cyber security policy awareness on employees’ cyber security behavior. international journal of information management, 45, 13–24. doi:https://doi.org/10.1016/j.ijinfomgt.2018.10.017 livingstone, s., mascheroni, g., & staksrud, e. (2017). european research on children’s internet use: assessing the past and anticipating the future. new media & society, 20(3), 1103–1122. doi:10.1177/1461444816685930 lowry, p. b., & gaskin, j. (2014). partial least squares (pls) structural equation modeling (sem) for building and testing behavioral causal theory: when to choose it and how to use it. ieee transactions on professional communication, 57(2), 123–146. lwin, m. o., li, b., & ang, r. p. (2012). stop bugging me: an examination of adolescents’ protection behavior against online harassment. journal of adolescence, 35(1), 31–41. doi:https://doi.org/10.1016/j.adolescence.2011.06.007 maddux, j. e., & rogers, r. w. (1983). protection motivation and self-efficacy: a revised theory of fear appeals and attitude change. journal of experimental social psychology, 19(5), 469– 479. maor, e. (2020). recycling credentials in four easy steps. retrieved from https://intsights.com/blog/recycling-credentials-in-four-easy-steps menard, p. (2018). the impact of collectivism and psychological ownership on protection motivation: a cross-cultural examination. computers & security, 75, 147–166. doi:10.1016/j.cose.2018.01.020 menard, p., bott, g. j., & crossler, r. e. (2017). user motivations in protecting information security: protection motivation theory versus self-determination theory. journal of management information systems, 34(4), 1203–1230. milne, s., orbell, s., & sheeran, p. (2000). prediction and intervention in health-related behavior: a meta-analytic review of protection motivation theory. journal of applied social psychology, 30(1), 106–143. milne, s., orbell, s., & sheeran, p. (2002). combining motivational and volitional interventions to promote exercise participation: protection motivation theory and implementation intentions. british journal of health psychology, 7(2), 163–184. mwagwabi, f., mcgill, t. j., & dixon, m. (2018). short-term and long-term effects of fear appeals in improving compliance with password guidelines. cais, 42, 7. australasian journal of information systems mwagwabi & hee jiow 2021, vol 25, selected papers from acis 2019 compliance with security guidelines in teenagers 21 ofcom. (2018). children and parents: media use and attitudes. retrieved from https://www.ofcom.org.uk/__data/assets/pdf_file/0024/134907/children-and-parentsmedia-use-and-attitudes-2018.pdf onwezen, m. c., antonides, g., & bartels, j. (2013). the norm activation model: an exploration of the functions of anticipated pride and guilt in pro-environmental behaviour. journal of economic psychology, 39, 141–153. posey, c., roberts, t. l., & lowry, p. b. (2015). the impact of organizational commitment on insiders’ motivation to protect organizational information assets. journal of management information systems, 32(4), 179–214. prentice-dunn, s., & rogers, r. (1986). protection motivation theory and preventive health: beyond the health belief model. health education research, 1(3), 153–161. ringle, c., wende, s., & becker, j.-m. (2015). smartpls 3. smartpls gmbh, boenningstedt. journal of service science and management, 10(3). rogers, r. (1975). a protection motivation theory of fear appeals and attitude change. journal of psychology, 91(1), 93–114. rogers, r. (1983). cognitive and physiological processes in fear appeals and attitude change: a revised theory of protection motivation. in j. t. cacioppo & r. e. petty (eds.), social psychophysiology (pp. 153–176). new york: guilford press. ryan, r. m., & deci, e. l. (2000). intrinsic and extrinsic motivations: classic definitions and new directions. contemporary educational psychology, 25(1), 54–67. schwartz, s. h. (1977). normative influences on altruism. in advances in experimental social psychology (vol. 10, pp. 221–279). academic press. sheldon, k. m., osin, e. n., gordeeva, t. o., suchkov, d. d., & sychev, o. a. (2017). evaluating the dimensionality of self-determination theory’s relative autonomy continuum. personality and social psychology bulletin, 43(9), 1215–1238. soh, p. c. h., chew, k. w., koay, k. y., & ang, p. h. (2018). parents vs peers’ influence on teenagers’ internet addiction and risky online activities. telematics and informatics, 35(1), 225–236. somerville, l. h., jones, r. m., ruberry, e. j., & dyke, j. p. (2013). the medial prefrontal cortex and the emergence of self-conscious emotion in adolescence. psychological science, 24(8), 1554–1562. doi:10.1177/0956797613475633 steinberg, l. (2008). a social neuroscience perspective on adolescent risk-taking. developmental review: dr, 28(1), 78–106. doi:10.1016/j.dr.2007.08.002 tayouri, d. (2015). the human factor in the social media security—combining education and technology to reduce social engineering risks and damages. procedia manufacturing, 3(1), 1096–1100. thompson, n., mcgill, t. j., & wang, x. (2017). “security begins at home”: determinants of home computer and mobile device security behavior. computers & security, 70, 376–391. tsirtsis, a., tsapatsoulis, n., stamatelatos, m., papadamou, k., & sirivianos, m. (2016). cyber security risks for minors: a taxonomy and a software architecture. paper presented at the 2016 australasian journal of information systems mwagwabi & hee jiow 2021, vol 25, selected papers from acis 2019 compliance with security guidelines in teenagers 22 11th international workshop on semantic and social media adaptation and personalization (smap). weinstein, n. (1984). why it won’t happen to me: perceptions of risk factors and susceptibility. health psychology, 3(5), 431–457. youn, s. (2009). determinants of online privacy concern and its influence on privacy protection behaviors among young adolescents. journal of consumer affairs, 43(3), 389–418. zhang, l., & mcdowell, w. c. (2009). am i really at risk? determinants of online users' intentions to use strong passwords. journal of internet commerce, 8(3), 180–197. zhao, x., lynch jr, j. g., & chen, q. (2010). reconsidering baron and kenny: myths and truths about mediation analysis. journal of consumer research, 37(2), 197–206. australasian journal of information systems mwagwabi & hee jiow 2021, vol 25, selected papers from acis 2019 compliance with security guidelines in teenagers 23 appendix constructs survey items and scale item loadings actual compliance with password guidelines (ajzen, 1991; fishbein & ajzen, 2010) strongly disagree 1 | strongly disagree 7 i always follow the password guidelines suggested by my social media service. 0.721 i always choose a password with a combination of numbers, letters and symbols. 0.794 i always take measures to protect my social media account. 0.766 for my social media accounts, i always choose strong passwords. 0.834 intentions to comply with password guidelines (floyd, prentice-dunn, & rogers, 2000; milne & milne, 2000; mwagwabi et al. 2018) extremely unlikely 1 | extremely likely 7 i intend to choose a password with a combination of numbers, letters, and symbols as suggested by my social media service. 0.716 i intend to follow the security measures recommended by my social media service. 0.747 i intend to protect my social media account from hackers. 0.751 perceived severity (zhang and mcdowell, 2009) not at all severe 1 | extremely severe 7 i believe that if someone successfully guessed my social media passwords the consequences would be…. 0.774 having my social media account accessed by someone without my knowledge would be…. 0.769 if my personal information was stolen from my social media account, the consequences would be… 0.773 perceived vulnerability (zhang and mcdowell, 2009) strongly disagree 1 | strongly disagree 7 it is likely that someone could successfully guess my social media passwords. 0.776 there is a chance that someone could successfully crack at least one of my social media passwords. 0.676 there is a chance that someone could hack into my social media accounts. 0.864 it is likely that someone would try to hack into my social media accounts. 0.693 it is extremely likely that my social media account will be attacked in the future. 0.647 fear of threat milne, orbell, and sheeran (2002) strongly disagree 1 | strongly disagree 7 the thought of someone hacking into any of my social media accounts frightens me. 0.868 the thought of someone using my personal information from any of my social media accounts makes me worried. 0.827 the thought of someone successfully cracking my social media passwords frightens me. 0.915 australasian journal of information systems mwagwabi & hee jiow 2021, vol 25, selected papers from acis 2019 compliance with security guidelines in teenagers 24 constructs survey items and scale item loadings perceived password effectiveness (zhang and mcdowell, 2009) strongly disagree 1 | strongly disagree 7 making sure that my passwords contain a combination of numbers, letters and special characters, will protect my social media account from hackers. 0.739 making sure that my passwords are strong will protect my social media account from hackers. 0.774 i can protect my social media accounts better if i use strong passwords. 0.743 the security measures recommended by my social media service are effective. 0.706 password self-efficacy (compeau and higgins, 1995) strongly disagree 1 | strongly disagree 7 i have the necessary skills to protect my social media account from hackers. 0.686 i would be able to use strong passwords if i had a lot of time. 0.566 i would be able to remember strong passwords if i had instructions on how to create memorable passwords. 0.739 i am confident that i can protect my social media account from hackers. 0.786 perceived cost (milne et al. 2002) strongly disagree 1 | strongly disagree 7 strong passwords are difficult to remember. 0.743 strong passwords take too much effort to create. 0.878 strong passwords take too much time to type. 0.807 if i change my passwords regularly, it would be difficult for me to remember them. 0.661 following the security measures recommended by my social media services is too much trouble. 0.678 peer influence (brown, 2004) strongly disagree 1 | strongly disagree 7 my friends would ridicule me if my account was hacked. 0.777 my friends would think that i am "weak" if my account was hacked. 0.805 my friends would gossip about me if my account was hacked. 0.885 personal norms (de groot, 2010; schwartz, 1977) strongly disagree 1 | strongly disagree 7 i would feel ashamed if my account was hacked. 0.844 i would feel guilty if my account was hacked. 0.889 i would feel embarrassed if my account was hacked. 0.851 full list of survey measurement items and item loadings australasian journal of information systems mwagwabi & hee jiow 2021, vol 25, selected papers from acis 2019 compliance with security guidelines in teenagers 25 copyright: © 2021 mwagwabi & hee jiow. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v25i0.2953 http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems kainulainen, tuunanen & vartiainen 2024, vol 28, research article requirements risk management for continuous dev. received 19 mar 2023, accepted 14 may 2024 after 5 rounds of review with karlheinz kautz as accepting editor 1 requirements risk management for continuous development: organisational needs sanna kainulainen faculty of information technology, university of jyvaskyla, jyväskylän yliopisto, finland tuure tuunanen* tuure.t.tuunanen@jyu.fi, faculty of information technology, university of jyvaskyla, jyväskylän yliopisto, finland tuure.t.tuunanen@jyu.fi tero vartiainen school of technology and innovations, information systems science, university of vaasa, vaasa, finland abstract information systems development has recently evolved from traditional to agile and continuous forms. continuous development (cd) methods, such as development and operations (devops), integrate many well-regarded parts of agile development and add collaboration among an organisation’s development, operations and quality assessment departments. we argue that requirements risk management (rrm) poses additional challenges to projects where development work is carried out quickly and continuously. however, in the literature, most methods for prioritising requirements and managing risks are more suited to traditional development. this raises the need for new tools and methodologies to meet cd challenges. as these challenges constantly evolve, project management must be able to control cd, changes in the determination of requirements and the accompanying risks. based on a systematic literature review, we define the key features of cd and develop a conceptual three-dimensional framework that can be used to understand the organisational needs of rrm for cd. keywords: requirements risk management, agile, continuous development, development and operations, devops, systematic literature review. 1 introduction information systems (is) development comprises the development activities required to create an is. isd involves multidisciplinary stakeholder collaboration to achieve better outcomes. stakeholders influence the complex development process and are affected in turn (maruping & matook, 2020b; siau et al., 2019). if cross-sectoral collaboration does not work, it can lead to higher requirement risks and project failures (wang et al., 2016). defining requirements has always been a critical part of project work. one of the most critical factors for moving a project forward and defining software functions and features is identifying industry-specific details and requirements. in agile projects, prioritising requirements and involving customers are critical to success (stray et al., 2019). today’s business constantly evolves; thus, continuous development (cd) requirements are increasing, and cd is seen as a future form of project development. our study is interested in how requirements risk management (rrm) has been considered for cd, what specific characteristics have been defined for a cd project type and what tools and methods should be mailto:tuure.t.tuunanen@jyu.fi australasian journal of information systems kainulainen, tuunanen & vartiainen 2024, vol 28, research article requirements risk management for continuous dev. 2 considered in risk management and definition. previous studies have shown that using agile methods helps solve challenges with traditional methods and improves performance (lwakatare et al., 2016). for example, the extreme programming method focuses on meeting customer requirements flexibly and interacting with customers during the development process to improve the provision and handling of continuous feedback. adopting this method has been argued to require less reworking than traditional methods and, thus, be more costeffective (iyawa, 2020). in contrast, cd focuses on organisational change because development is continuous and iterative, with no definite end. it aims to connect the development and operation domains to improve software development flow (hemon-hildgen et al., 2020), add services to the digital infrastructure and include continuous feedback and learning among actors from different organisational levels (osmundsen & bygstad, 2022). while organisations continue to move away from traditional stage gate approaches (cois et al., 2014; heemstra & kusters, 1996; royce, 1987) to agile and cd approaches (hütterman, 2012), project management methods have not evolved at the same pace. much of the literature posits that existing methods only suit traditional projects (ghobadi & mathiassen, 2016; jiang et al., 2006; racheva et al., 2010; ramesh et al., 2010). is development requires new tools and techniques because traditional information systems development (isd) methods and techniques are unsuitable for modern agile development (kautz et al., 2007). agile development enables a better understanding of customer needs and adaptation to today’s needs (cao & ramesh, 2008; kautz et al., 2007). requirements usually come from business needs, and changing requirements threaten project success if not handled properly (maruping et al., 2009). maruping et al. (2009) argue that very little guidance is available on managing teams in agile development and that managing changing requirements as effectively as possible is essential because change is inevitable. cao and ramesh (2008) state that organisations need to be able to respond rapidly to merging change requests, which are critical and arise throughout the software development process. they differentiate the process of agile requirements engineering (re) from traditional development, requiring new methods for handling requirements. the above discussion raises important questions: if many methods and models are based on traditional development, when and where are requirements defined at the beginning of the project (heemstra & kusters, 1996)? more importantly, how should changing requirements be managed and prioritised with the latest post-agile cd approaches? we posit that cd requires methods focusing on change and feedback (dingsøyr et al., 2019). in our study, we seek to answer the following research question: how should requirement risks be managed in cd, and what are the organisational needs for accomplishing this? we apply a systematic literature review (slr) method to identify cd's current state of the art and how it is defined and understood. our analysis was conducted from both is and re perspectives. based on our slr, we construct a framework with dimensions describing the organisational needs of rrm for cd. these dimensions can be used to define a project’s aspects from a cd perspective and determine whether an organisation implements the critical cd development features and how the project meets the organisational needs of rrm for cd. the dimensions australasian journal of information systems kainulainen, tuunanen & vartiainen 2024, vol 28, research article requirements risk management for continuous dev. 3 can be used to set preferences and priorities for project development to meet better cd requirements, such as rapid response and prioritisation of new needs. this article is structured as follows. first, the theoretical background is presented, and the slr process is described. the results are presented thereafter. the framework is then presented. next, we discuss the findings and implications for research and practice. we conclude by discussing the study’s limitations and offering avenues for future research. 2 theoretical background wallace et al. (2004) define complexity risk and requirement risk as the technical dimensions of is project risks: requirement risk describes the potential impact of a project’s requirements or requirement management process. it represents the probability of project failure when an is project’s requirements are unclear or highly unstable. project requirement risk examples include missing or incorrect stakeholders, unclear, incomplete, inconsistent or unrealised requirements, undocumented or inaccurate assumptions, business requirements falsely defined as functional requirements, an inability to link applicable requirements to business requirements and unvalidated requirements. projects based on flawed requirements will likely face challenges and problems and may fail (venkatesh et al., 2018; wallace et al., 2004). in response to fast-changing requirements, agile development emerged as a new form of isd to deliver new services (mangalaraj et al., 2009; mathiassen & pries-heje, 2006; olszewska & waldén, 2015). agile methodology divides projects into phases, focusing on continuous collaboration and development. teams follow a planning, implementation and evaluation cycle (dingsøyr et al., 2019; shimada et al., 2019). cd, in turn, evolved from agile development. it extends agile development by focusing on short continuous learning and development cycles with a continuous feedback loop (lwakatare et al., 2016; osmundsen & bygstad, 2022; virmani, 2015). gatrell (2016) states, “technology has moved from continuous integration to continuous deployment and, finally, to continuous delivery” (p. 104). continuous integration combines several authors’ source code changes into a single software project. automated functions ensure the code’s correctness and allow quick feedback on its quality (gall & pigni, 2021; stray et al., 2019). continuous deployment refers to frequent, automated software deployments (gall & pigni, 2021). continuous delivery aims to shorten release cycles by automating software testing and acceptance (chen, 2015; ghantous & gill, 2017). cd can be seen as an umbrella term that includes many devops processes, including continuous integration, testing, delivery and deployment (osmundsen & bygstad, 2022). according to gall and pigni (2021), advancing cd is critical for companies because agile methods cannot deliver quality results fast enough due to the market’s dynamic nature. in this article, development and operations (devops) are considered an instantiation of cd in which agile development is connected with fast delivery cycles, short feedback loops and automation. the cd is perceived as a way to organise functional units to achieve the necessary sensitivity and responsiveness to market conditions and demands (maruping & matook, 2020a; wiedermann et al., 2020). devops is a cd approach that extends agile principles to the entire software process and cooperation between the operations, development and quality assessment domains, including testers and quality assurance teams (ebert, 2018; krey et al., australasian journal of information systems kainulainen, tuunanen & vartiainen 2024, vol 28, research article requirements risk management for continuous dev. 4 2022; lwakatare et al., 2016; osmundsen & bygstad, 2022; stray et al., 2019). devops can also be regarded as a set of new practices for deploying production changes more efficiently without compromising quality (lwakatare et al., 2016; ozkaya, 2019). appendix 1 lists the key concepts and definitions used in this article. maruping and matook (2020a) note that the academic literature has only just begun to understand the devops phenomenon and its impacts. ghantous and gill’s (2017) literature review revealed no universal definition of devops. despite the extensive literature on devops, it does not clearly explain what devops is and lacks conceptual inclusion. this lack is considered a significant barrier to the mainstream adoption of devops, preventing a thorough understanding of what devops includes and means and how organisations can successfully transition to devops (gall & pigni, 2021; hüttermann & rosenkrantz, 2019). for example, krey et al. (2022) note that the implementation of devops in small and mediumsized enterprises has not been comprehensively researched. 3 research methodology: systematic literature review we chose an slr for this study, applying kitchenham et al.’s (2010) method to define the theoretical background. the slr process is detailed in appendices 2, 3 and 4. the literature review and analysis were used to create a conceptual framework, with each search step adding more detailed search criteria and further analysis. we examined 768 articles during the slr, with 83 used to develop the framework. first, we searched google scholar (separately for rrm and cd) for the association for is’s basket of eight journals1 (aisbasket8) with no time limit to examine how the is literature addresses devops and cd in its top eight publications. next, the first author conducted the first coding of the selected articles. this process began by the first author reading all the selected articles and highlighting text defining the features and concepts of continuous and agile development. seventeen articles were transferred to atlas.ti, and several codes describing cd features or concepts were extracted from these articles. appendix 3 includes an example of the coding process. during the analysis, defining codes involved several systematic steps to ensure they accurately represented the key concepts and themes identified in the literature. the first step involved reading the articles and highlighting texts relevant to the research focus. these highlighted texts included vital phrases, sentences or paragraphs that address aspects of agile, devops, or continuous development (cd). the highlighted texts were then assigned initial codes by the first author. these initial codes were often descriptive labels that captured the essence of the highlighted text. for instance, a text discussing the importance of cooperation in devops might be initially coded as “cooperation”. after initial coding, the codes were reviewed and refined to ensure they accurately represented the highlighted texts. the authors reviewed the coding results together and worked on the code processing further in a workshop. this involved combining similar codes, splitting broad codes into more specific ones, and ensuring consistency in coding across different texts. for example, codes related to communication, cooperation, and knowledge 1 european journal of information systems, information systems journal, information systems research, journal of association for information systems, journal of information technology, journal of management information systems, journal of strategic information systems and mis quarterly. australasian journal of information systems kainulainen, tuunanen & vartiainen 2024, vol 28, research article requirements risk management for continuous dev. 5 sharing were grouped into a code group related to project culture. each code belonged to only one code group, ensuring that the classification was distinct and organised, and each code group represented a broader category of concepts. this involved articulating what each group encompassed and ensuring that all the codes within a group were relevant to this definition. the process was iterative, meaning that the first author would switch back and forth between the texts and the codes, continually refining and adjusting the codes and code groups. this ensured that the codes aligned with the article’s content and the research objectives. documentation was maintained throughout the coding process. this included justifying why certain texts were highlighted, specific codes were assigned, and code groups and dimensions were defined. this documentation was crucial for ensuring transparency and reproducibility of the analysis. the final step of coding involved organising the code groups into more significant, overarching dimensions in a workshop. the all of the authors were involved in the process of creating the dimensions. as a result of this work, three dimensions were built: the first, culture, comprised subgroups – project and organisation. the other dimensions were methods and tools, and pace and seamlessness. these dimensions provided a structured lens for the analysis. appendix 3, table a4, lists the codes, code groups, and dimensions. in summary, defining codes involved a systematic and iterative approach to highlighting relevant texts, assigning and refining initial codes, grouping similar codes into thematic code groups, and organising these groups into broader dimensions. this structured process ensured that the codes accurately reflected the key concepts and themes within the literature, providing a robust framework for analysis. is conference proceedings2 were similarly handled, with 13 papers coded in atlas.ti. next, articles found as part of the original search’s forward and backward reference search results (other than the aisbasket8 journal articles and is conference proceedings papers) were selected for coding. the slr’s first phase focused on is, encoding the concepts is research focused on in devops and cd. we next wanted to determine how the is literature studies devops and cd and which concepts it focuses on in terms of the earlier identified framework dimensions. articles were selected similarly to the first search, resulting in 25 papers analysed using the defined research lens with three new dimensions. the purpose was to discover how key features of agile development, devops and cd are described and how the features of the different dimensions are highlighted. appendix 2, tables a1 and a2 list the search results and number of selected articles, papers and conference proceedings. the initial results revealed a lack of focus on cd: 16 articles (64%) focused on methods and tools, 12 (48%) on culture/project culture structure, 7 (28%) on culture/organisational culture structure and 4 (16%) on development speed or seamlessness. most articles mentioned only traditional development methods; 28% mentioned agile development, but only 4% mentioned cd. 2 the international conference on applied mathematics, informatics, and computing software (amics), european conference on information systems (ecis), hawaii international conference on system sciences (hicss), international conference on information systems (icis) and pacific asia conference on information systems (pacis). australasian journal of information systems kainulainen, tuunanen & vartiainen 2024, vol 28, research article requirements risk management for continuous dev. 6 next, we expanded the search to cover the corresponding literature in a selection of re journals3. three searches were performed separately for devops and cd, the last of which resulted in 159 articles, with 8 selected for analysis. appendix 2, table a3, lists the search results and selected articles. the articles were again coded with atlas.ti. the coding results were very similar to those for the is articles, with similar concepts emerging. this reinforced our results from the is article analysis and provided further evidence to refine the framework. five (63%) articles focused on methods and tools, 3 (38%) on culture/project culture, 4 (50%) on culture/organisational culture and 5 (63%) on development speed or seamlessness. most articles commented on agile, devops or cd. appendix 4 lists all analysed and referenced articles. our descriptive analyses of the is and re literature show an almost equal focus on the dimensions of methods and tools: 64% of the is articles and 63% of the re articles. the difference was slightly higher for the culture/project culture dimension: 48% of the is articles and 38% of the re articles. the most significant differences were in the culture/organisational culture and development speed or seamlessness dimensions. the is articles focused much less on these dimensions than the re articles. only 28% of the is articles concentrated on elements of the culture/organisational culture dimension; in the re articles, the coverage was 50%. the difference was even more significant in the development speed or seamlessness dimension. only 16% of the is articles presented this dimension’s elements, while in the re articles, the coverage was 63%. there was also a clear difference in development style mentions. in the is literature, only traditional methods were usually mentioned, and there was little focus on cd. in contrast, in the re literature, most articles mentioned agile development, devops or cd. 4 findings based on our slr, we identify the key features and concepts that can be used to determine the characteristics of cd methods. table 1 summarises the three dimensions of cd: culture, methods and tools and pace and seamlessness. culture is divided into project culture and organisational culture. project culture describes how a project’s organisation is managed, its functionality, knowledge sharing, roles and groups. because information is fragmented within an organisation, projects must involve people with in-depth knowledge of the development objectives and risks. everyone should share information and knowledge openly, as changes occur quickly and development work is continuous. organisational culture focuses on the organisation doing the development work. the critical features are cooperation, work tasks and overall transformation of the organisational culture based on cd. while progress in cd is achieved in collaboration with the firm’s development, operations and quality assurance domains, users and customers are also actively involved in the work. this work aims to develop a purpose-built, flexible system tailored to customers’ and users’ needs and requirements. development work must quickly and seamlessly progress to meet new challenges, requirements and risks. testing and development operate together, and the project must have the right resources to guide the development work in the correct 3 empirical software engineering: an international journal, ieee software, ieee transactions on software engineering, information and software technology, information systems (is), requirements engineering journal, software and systems modelling, software practice and expertise and software quality journal. australasian journal of information systems kainulainen, tuunanen & vartiainen 2024, vol 28, research article requirements risk management for continuous dev. 7 direction. cd and constant change require action to keep the organisation responsive. information technology (it) projects are particularly vulnerable to commitment challenges, mainly due to requirements volatility and software’s intangible nature (horlach et al., 2020; lee et al., 2021). the literature shows there is a demand for new methods and tools to keep project management consistent with cd (see, e.g. babb et al., 2017; bragge & merisalo-rantanen, 2009; hüttermann & rosenkranz, 2019; lwakatare et al., 2016). the methods and tools dimension focuses on the need for practical and usable tools to implement risk management in cd and the need to develop valuable and cost-saving tools for handling risk management. the pace and seamlessness dimension refers to the ability to respond quickly to requirements and change requests, to work in an iterative cd environment and the possibility to receive and react to continuous feedback from customers and users. dimension 1a: culture – project culture definition organisation’s approach to project management and implementation. activities development, operations, knowledge sharing, cooperation, defining roles and groups and continuing cooperation with users and customers. explanation project work is built on seamless cooperation and knowledge sharing among developers, users, operations staff and customers; requirements and different groups and users define risks, and practitioners with the proper knowledge must participate in the project. the focus is on continuous feedback and self-organisation. dimension 1b: culture – organisational culture definition organisation’s approach to managing cd and its further development. activities development, operations and quality assurance cooperation. explanation development organisation includes developers, operations and quality assurance teams. development is not only done by developers. cooperation among different departments is essential. dimension 2: methods and tools definition the means to support change cycles and fast development according to changing requirements. activities adoption and development of methods and tools suitable for cd. explanation methods must support rapid, cyclical development and changing requirements and risks. tools must account for cultural effects and user values when eliciting requirements. knowledge-sharing tools are essential because development is based on knowledge-sharing. dimension 3: pace and seamlessness definition the development process is rapid, continuous and without interruptions. activities rapid development, continuous work and continuous, flexible processes. explanation development is a continuous, flexible process that can adapt to changing demands and adjust to constant feedback. table 1. dimensions of organisational needs of rrm for cd. australasian journal of information systems kainulainen, tuunanen & vartiainen 2024, vol 28, research article requirements risk management for continuous dev. 8 in the following subsections, the three dimensions are discussed in more detail, including perspectives from the re and rrm literature. 4.1 project culture for rrm this dimension relates to the development and operations of a project and to its users and customers for the results. when defining a project’s organisation, the project team members’ expertise is paramount. while collaboration and knowledge sharing often identify broad risks (jiang et al., 2006), it is essential to focus on changing requirements and project goals (maruping et al., 2009). requirements ambiguity contributes to software project challenges in critical domains. failure is less likely when the right people are involved (niederman et al., 1991). however, unclear and changing requirements in the middle of a project are among the significant challenges of software development projects. as a result, it is almost impossible for development teams to identify and meet all customer expectations (ghanbari, 2016). similarly, an inability to respond to changing user requirements is one of the most critical reasons for project failure. failure also includes delayed project schedules, budget overruns and poor quality (maruping et al., 2009; mathiassen et al., 2007). these factors also contribute to the difficulty of sharing knowledge between individuals. previous research has shown that organisational culture influences how knowledge is transferred and stored and affects the success of managing it project requirements (azizi & rowlands, 2018). identifying the roles and key stakeholders in a project is an essential first step in risk management. li et al. (2003) state that the is user environment needs more research, while keil et al. (2002) explore how different roles can help identify project risks. people in different positions will recognise diverse project risks. groups can also define risks differently, so it is vital to maintain open communication and have good knowledge-sharing practices and discussion opportunities (keil et al., 2002; li et al., 2003). it is well recognised that managing risks in it projects is essential and that failure to address them might cause project problems, such as user dissatisfaction (elbanna & sarker, 2015; keil et al., 2002; li et al., 2003; ramesh et al., 2010). a necessary part of project success is communication between users and developers. although merging different stakeholders’ knowledge is recognised as a critical part of project success, there is no research on integrating different stakeholders into project risk management (keil et al., 2002; li et al., 2003). project risks typically relate to users (elbanna & sarker, 2015) and include poor communication with users and stakeholders at the project level, a lack of user involvement or misunderstanding of user requirements or failure to manage user expectations or accommodate defined requirements and scope changes. communication among different groups and roles critically influences risk management (li et al., 2003; ramesh et al., 2010). however, an organisation’s ability to define risks strongly affects a project’s quality (karlsson et al., 2007). it is also essential to introduce development methods to users, incorporate them into the project and coordinate expertise so that specialised knowledge is spread and integrated among the project’s many roles or phases (gemino et al., 2007; patnayakuni et al., 2006). this can be perceived as a practical risk management approach (gemino et al., 2007). according to the agile manifesto, collaboration and responsiveness to change are essential. several studies (e.g. azizi & rowlands, 2018; ghanbari, 2016; kiper, 2016; sletholt et al., 2012; stray et al., 2019) identify lack of knowledge about requirements and organised activities and testing principles as problem areas. identifying and defining requirements are problematic australasian journal of information systems kainulainen, tuunanen & vartiainen 2024, vol 28, research article requirements risk management for continuous dev. 9 because they may be predefined or unknown (sletholt et al., 2012). therefore, the requirement planning phase should be extended with activities that collect and exploit new sources of information (i.e. development activities should be more closely integrated with operational activities). the possibility of using direct user feedback and a central infrastructure also poses challenges for testing and validation processes (stuckenberg & heinzl, 2010). once software requirements have been received from stakeholders, they should be validated to ensure they meet user needs. requirements must also be prioritised to address technical constraints, business considerations and critical stakeholder preferences (kiper, 2016). babb et al. (2017) assert that a lack of knowledge sharing is a problem for agile development and cd. similarly, ghobadi and mathiassen (2017) observe that agile development lacks appropriate tools to manage risks associated with knowledge sharing and introduce a theoretical model for mitigating such risks in several ways to assess and clarify project information-sharing risk profiles and aiming to create an overall plan for reducing and resolving risks. their study’s results highlight how different risk management profiles for information sharing can lead to varying project performance outcomes. their model introduces concepts and detailed processes for managing a project’s knowledge-sharing risks. davison (2017) remarks that with this model, it is easy to show how risk management can be used in agile development to achieve better performance and outcomes. in examining how progressive obstacles to knowledge sharing are observed and differ according to the observer’s role, ghobadi and mathiassen (2016) conclude that sharing information is challenging and influenced by the actors’ knowledge of the work organisation and environment. in turn, organisational success depends on how effectively employees share information (qureshi et al., 2018). knowledgeable practitioners must actively participate in the project (taylor et al., 2012). it is typically assumed that managers are well informed about possible risks in using project methods, but this is not always so (schmidt et al., 2001). however, such knowledge is often fragmented throughout the organisation, so project managers should employ experienced users who can share information and work collaboratively (tiwana & keil, 2004). barriers can be caused by internal organisational tensions, value systems, personality clashes and policies related to knowledge transfer. 4.2 organisational culture for rrm the organisational culture dimension relates to changes in organisational culture and cooperation among different departments. in many organisations, managing risks is one of the most critical challenges. taylor et al. (2012) note that organisations often do not apply research or knowledge on risk management and risk factors. this is a problem, considering that the project outcome depends on how well the organisation prioritises its requirements (karlsson et al., 2007). up to a third of development costs are due to incorrect requirements (patnayakuni et al., 2006). jiang et al. (2006) describe partnering as a possible solution for improving rrm in that cooperation between users and it staff creates new possibilities to define requirements better. collaborative knowledge exchange positively impacts the whole development process and performance. the importance of cooperation is stressed in agile development methods, which require collaboration between development teams and customers, including customer feedback, throughout the project (cao et al., 2009; xiao et al., 2018). the literature identifies the importance of a designated customer representative role in an agile project (maruping & australasian journal of information systems kainulainen, tuunanen & vartiainen 2024, vol 28, research article requirements risk management for continuous dev. 10 matook, 2020b; matook & maruping, 2014). however, managing this role can be challenging, as it is multifaceted and takes many forms. mapping the modes of action the role requires is not easy because hardly any theory exists on the definitions of the different modes of action (maruping & matook, 2020b). the devops team comprises staff from the development, operations and quality assessment departments. their collaboration is critical, and changing the organisation’s structure is important for these previously independent departments to become part of one development unit. requirements definitions and related risks pose additional challenges to ongoing development projects in which development is quick and continuous. while traditional development requirements are defined at the project’s beginning, agile development requirements are defined iteratively throughout the project (ramesh et al., 2010). the same applies to cd projects. krancher et al. (2018) identify continuous feedback and selforganisation as other key issues in cd. rapid feedback and other devops practices help create and strengthen the development team’s autonomy (callanan & spillane, 2016) and focus on valuable features and requirements (chen, 2015). the lack of both knowledge and guidance is challenging for organisations shifting to devops (gall & pigni, 2021). previous studies have shown that agile development methodologies focus on isd (dev) but pay little attention to the operational (ops) aspects of software deployment in a production environment (gall & pigni, 2021; ghantous & gill, 2017). the definition and concepts of devops are still unclear, and there is little understanding of the factors that influence the adoption of devops practices in an organisation (gall & pigni, 2021; ghantous & gill, 2017). sharp and babb (2018) highlight the fact that there is no universally accepted definition of devops and note fundamental differences between the software engineering and is literature; in is, the conceptual elements of devops remain undefined. this lack of a commonly agreed-upon definition of devops is a critical area for future research (sharp & bagg, 2018). traditionally, different organisational units, teams and individuals coordinate. in agile contexts, organisational structure changes are encouraged, limiting the stability and alignment they can provide. therefore, alignment must allow flexible structural design, while letting people work together as smoothly as possible. from a customer perspective, agility is the ability to continue delivering customer value (horlach et al., 2020). ramesh et al. (2010) state that a customer’s inability to provide the correct requirements to the development team and a lack of harmony between developers and customers significantly impact the development process, for example, if requirements are poorly drafted. chen (2015) comments that despite the extant literature on organisational change, little research focuses on introducing continuous delivery or development to an organisation. conversely, according to lwakatare et al. (2019), studies of successful adoptions and implementations of cd, specifically devops, show organisations’ abilities to change their structures, processes and tools. osmundsen and bygstad (2022) recognise the potential of such changes in organisations, enabling them to respond to customer needs and requirements more innovatively. user input and communication across different organisational levels are considered the leading factors enabling value delivery in cd. cao et al. (2009) note that the literature recommends using agile methods in organisations with a flat structure. in organisations with centralised and hierarchical structures, the organisational culture may cause problems between the top management and the project team. australasian journal of information systems kainulainen, tuunanen & vartiainen 2024, vol 28, research article requirements risk management for continuous dev. 11 4.3 rrm methods and tools different means to support developers to track requirements are needed because requirements constantly evolve. requirements must be continuously modified and improved during a project, necessitating methods and tools to help developers track requirements. however, such advanced tools in smaller projects may be considered unnecessary (ghanbari, 2016). davis (1982) emphasised the importance of determining requirements and observed obstacles to and challenges in defining correct requirements: human limitations and complexity of and variety in information and user–analyst interaction models. various methods and tools are needed to overcome these obstacles and challenges. heemstra and kusters (1996) present different tools and methods for risk management elicitation and definition in various steps and project phases, determining that the purpose of risk prioritisation in traditional projects is to choose from identified risks. the most important should be on a manageable list at the project’s beginning. tiwana and keil’s (2004) one-minute risk assessment tool aims to help project teams conduct “what-if” analyses and improve software practices. it comprises project risk levels and questions to estimate an overall risk score. project managers and stakeholders assign scores to each question, and the results produce a background image of each project’s risk exposure. they argue that the most critical risk driver is the choice of methodology, followed by customer involvement. taylor et al. (2012) note inconsistent methods, and ramesh et al. (2010) report a lack of information on requirements management in real agile projects. several checklists for defining risks have been developed, but few organisations adopt them (wallace et al., 2004). the challenge lies in creating practical tools for implementing risk management. as there is no consistent model for managing knowledge-sharing risks in agile projects, ghobadi and mathiassen’s (2017) tool aims to help understand and manage knowledgesharing risks in agile development environments when moving from project risks to effective knowledge-sharing and resolution strategy plans. it presents seven risk area categories and five resolution strategies. for example, one defined risk area is team diversity, referring to conceptual, geographical and time differences between team members that may hinder effective knowledge sharing. the resolution strategy to overcome this is strengthening resources (i.e. strategies to develop supportive capabilities, experiences and technologies). ghobadi and mathiassen (2017) note that agile development must find a way to identify customers’ requirements and demands. in turn, tuunanen and kuo (2015) argue for the need for tools that account for cultural influences and user values. similarly, bragge and merisalorantanen (2009) state that challenges remain involving users in development, especially with traditional methodologies. simply put, traditional methods do not include explicit information about when and in which part of the project user involvement should occur. 4.4 pace and seamlessness for rrm in cd, development is rapid, continuous and seamless. this is based on business demands, where quick responses to changes and new requirements are mandatory. organisations must be able to deliver agile is, whereby development phases are repeated multiple times in cycles and requirements are defined and fulfilled at the beginning of each iteration cycle (hickey & davis, 2004; patnayakuni et al., 2006). while several different agile approaches exist, all focus australasian journal of information systems kainulainen, tuunanen & vartiainen 2024, vol 28, research article requirements risk management for continuous dev. 12 on development that can respond flexibly and seamlessly to customers’ and developers’ requests and needs (elbanna & sarker, 2015). according to ramesh et al. (2010), traditional requirement methods and practices are often too cumbersome to evolve with the rapidly changing field of agile development. they propose new techniques to help execute the re process in agile projects to compensate for the limitations of traditional methods. an example is in-person communication instead of written specifications to transfer ideas between customers and the development team without extensive documentation and generate a flexible way to consider various requirements (xiao et al., 2018). another good technique to define requirements in agile development involves user stories that can be used to determine high-level requirements (cao & ramesh, 2008). elbanna and sarker (2015) note that short iteration cycles allow changes to be considered faster and more efficiently, and user requirements can be clarified more effectively. face-to-face interaction and ongoing communication with users and business groups improve this relationship and help develop a common understanding of project requirements. racheva et al. (2010) state that the agile re literature has too little information on how reprioritisation work is done in practice, and generic models of how they should be processed are missing. chen et al. (2016) comment that feedback loops must be open in agile development. feedback should include technical and business feedback, such as performance, security and availability issues and new requirements, such as new user features. handling feedback should be continuous, a learning cycle and part of the development cycle. krancher et al. (2018) note that continuous frequent feedback is essential for gathering customers’ ideas and requirements; it should be handled as fast as possible, and learning cycles should be kept as short as possible to be able to learn as much as possible. 5 discussion digitalisation and today’s business demands create challenges requiring new ways of working (ebert, 2018). we argue that cd is a new way to develop is systems to meet rapidly changing situations and requirements (ramesh et al., 2010). consequently, we determined the literature’s position on cd and its instantiation of devops. we determined key dimensions of cd: (1a) culture – project culture, (1b) culture – organisational culture, (2) methods and tools and (3) pace and seamlessness. more specifically, we explored how these development methods accommodate rrm needs to answer our research question. our findings align with other research. gall and pigni (2021) define a continuous culture concept that corresponds to the project and organisational culture dimensions; continuous monitoring corresponds to the methods and tools dimension; and continuous automation, to some extent, matches the pace and seamlessness dimension. however, notably our findings show that the is literature does not focus on automation (continuous automation, monitoring, etc.) but mostly on other aspects. therefore, the dimensions defined in this study cannot be directly compared with gall and pigni’s (2021) model. krey et al. (2022) argue that devops implementations consist of the following interrelated categories: agility, collaboration, automation, measurement, monitoring and transparency. these categories can be used to find correspondences between our three dimensions and the elements of gall and pigni’s (2021) model. australasian journal of information systems kainulainen, tuunanen & vartiainen 2024, vol 28, research article requirements risk management for continuous dev. 13 jayakody and wijayanayake (2023) group the critical success factors of devops into four areas: collaborative culture, devops practices, skilled devops teams and metrics and measurement. they argue that, in addition to culture, devops affects the processes, products, related technologies and organisational structures used in software development and operations processes. there is a clear convergence between these concepts and the dimensions and key concepts defined in our research. jha et al. (2023) state three key themes associated with devops culture and thinking: collaboration, continuous improvement and automation, corresponding with the dimensions defined in our study. finally, khan et al. (2022) state that “culture, practices, and tools are the three backbones of devops; culture defines a way of thinking with some basic standards. practices reflect culture’s significant success, and numerous tools are required to implement these methods” (p. 14339). our findings also reveal that the extant is and re literature has, to some extent, focused on traditional-style waterfall model development and the applicable tools and methods (see, e.g. ghobadi & mathiassen, 2016; highsmith & cockburn, 2009; jiang et al., 2006; racheva et al., 2010). in the traditional model, at the beginning of a project, the objectives, requirements and schedules are defined (heemstra & kusters, 1996) and implemented by a specific group of experts and project management professionals in the it department. cao and ramesh (2008) echo this finding by arguing that there is a scarcity of research on how re is managed in agile projects. ramesh et al. (2010) state that the literature does not elaborate well on how re activities are considered in agile projects. this is problematic, as cd projects require seamless cooperation from developers, users and operations staff. the pace of change in cd is faster than that of legacy agile development (highsmith & cockburn, 2009), and many tools are designed for slower development work. this highlights the importance of knowledge sharing and cooperation, which have always been crucial in development projects but are even more so with cd. 5.1 implications for research and practice and a roadmap we believe that cd requires new methods and tools to modernise the requirements definition process and improve knowledge sharing without disrupting development work (ghobadi & mathiassen, 2017; ramesh et al., 2010). it is also essential that development considers changing requirements throughout the process. however, we argue that this can create problems with knowledge sharing, as different stakeholder groups often have other priorities (ghobadi & mathiassen, 2016), which can result in a further lack of cooperation (li et al., 2003; ramesh et al., 2010). our findings highlight the need to change project organisation and structures to involve the necessary stakeholders with appropriate knowledge (gemino et al., 2007; patnayakuni et al., 2006; taylor et al., 2012). additionally, knowledge sharing and internal communication require development and a new way of thinking collaboratively and creatively. for example, brainstorming or frequent social interaction through it can effectively share information (babb et al., 2017; qureshi et al., 2018). organisations need to collaborate and create structures that support cd work. essential components are iterative requirements, definition work and selforganisation (cao et al., 2009; krancher et al., 2018; ramesh et al., 2010). our research also shows the need to introduce cd to an organisation and simultaneously reformat its culture and structure to support a cyclical, continuous way of working (cao et al., 2009; chen, 2015; matook & maruping, 2014; xiao et al., 2018). australasian journal of information systems kainulainen, tuunanen & vartiainen 2024, vol 28, research article requirements risk management for continuous dev. 14 the developed framework highlights the cultural changes required to enable an organisation to work in accordance with cd. it helps the organisation prepare for the challenges of cd, improve collaboration between different departments, and structure the project organisation to work with cd to achieve seamless collaboration and knowledge sharing between all stakeholders, including ongoing collaboration with users and customers. the cd also requires the it department to be restructured so that there is a seamless collaboration between development, operations, and qa rather than departments working in isolation. the cd also requires new methods and tools to support a continuous, rapid development cycle, respond to continuous feedback, encourage and improve knowledge sharing, and consider cultural influences and user values as part of the requirements definition. agile methodologies are inherently different from cd requirements in terms of challenges and characteristics: continuous feedback, interdepartmental cooperation, collaboration, changing organisational structure, rapid and seamless continuous development, forming a project organisation. our framework emphasises organisational and project culture changes, collaboration, cooperation, feedback loops and seamless automation, which also means new tools. with new methods and tools that support continuous, rapid development cycles, respond to continuous feedback, encourage and improve knowledge sharing, and help to consider cultural influences and user values when defining requirements. when we compare maruping et al. (2009)’s work with ours, we note that our framework highlights more the need for feedback loops and advanced monitoring, adapting development strategies based on real-time data and changing user requirements. furthermore, our framework enables organisations to further improve their ability to effectively manage changing user requirements, optimise development processes, and deliver high-quality software products in dynamic and rapidly evolving environments. ramesh et al. (2010), in turn, developed agile re practices in their study. we argue that, to support cd, these practices need to incorporate devops and cd principles such as automation, collaboration, and continuous delivery into agile practices, as well as a feedback mechanism such as implementing feedback loops, as our framework emphasises. thus, re practices must support continuous feedback throughout the cd lifecycle collecting, analysing and acting on feedback from stakeholders, users and team members. in line with our framework, real-time collaboration and communication are essential in cd and require new tools tailored to cd methodologies. in the following, we present a roadmap for cd rrm based on the dimensions of the developed framework and propose some research problems and related research questions. we also suggest research approaches, objectives, and, finally, how managers would need to address the risks associated with cd requirements. table 2 summarises this roadmap. 5.1.1 project culture: stakeholder roles and knowledge sharing first of all, we find that the cd project comprises various stakeholders from different parts of the business or organisation. it includes multiple roles and areas of expertise from developers, operations staff, users and customers. customer engagement is essential in digital transformation and development (sebastian et al., 2017). communication among different stakeholders is critical for a project’s success (keil et al., 2002; li et al., 2003; ramesh et al., 2010). knowledge sharing is part of organisational success and depends on how effectively stakeholders share information (qureshi et al., 2018). it is an essential part of project work, as australasian journal of information systems kainulainen, tuunanen & vartiainen 2024, vol 28, research article requirements risk management for continuous dev. 15 is seamless cooperation among the project stakeholders, including users and customers (dingsøyr et al., 2019; elbanna & sarker, 2015; krancher et al., 2018; tiwana & keil, 2004). different stakeholders also define requirements and risks, which can be modified/added to as part of the development cycle based on their feedback during the project. in cd, practitioners with appropriate knowledge must participate in the project. project and development work focus on continuous feedback and self-organisation (krancher et al., 2018; matook & maruping, 2014). according to our study, managers need to address requirement risks in cd projects by perceiving them as business projects, not merely as it projects. stakeholders and knowledge can be found in various departments, roles, and users. managers should focus on changing requirements, project goals, and identified risks. they must prioritise requirements to address technical constraints, business considerations, and critical stakeholder preferences. it is essential to introduce practical risk management approaches and tools to users and integrate them into the project while coordinating expertise. information should be shared through open communication and good knowledge-sharing practices. we suggest an in-depth qualitative study (klein & myers, 1999) of cd teams to investigate project culture, focusing on roles and knowledge-sharing dynamics. this approach analyses real-world cd team structures to determine their effectiveness in managing requirement risks. 5.1.2 organisational culture: structure and cross-department collaboration consequently, the organisational structure of cd brings a significant change in organisational culture compared with traditional development (dremel et al., 2017; mathiassen & pries-heje, 2006). in cd (and devops), the organisational structure is built on cooperation among different departments, including development, operations and quality assurance. collaboration among various departments is essential (dremel et al., 2017; lwakatare et al., 2019; ozkaya, 2019). however, our findings show that an organisation’s hierarchical structure prevents cooperation across borders and complicates development work (dremel et al., 2017), creating poor communication between stakeholders and easily creating sporadic development work (lwakatare et al., 2019). we argue that businesses must play a substantial role in development work and ensure that their needs and requests are considered part of the project and that they obtain what the business needs (matook & maruping, 2014). to effectively support cd, the project organisation must find a way to involve all necessary stakeholders in both the project and project risk management. they need to create an environment where knowledge sharing among different organisational departments, users, and stakeholders is developed and prioritised. managers should create a cohesive structure that allows previously independent departments to function as a unified development unit, facilitating cooperation and ensuring that the needs and requests of all stakeholders are included and addressed throughout the project lifecycle. furthermore, businesses need to assign a substantial role to development work and ensure that their needs and requests are included in the project and that they obtain what they need. managers should change the organisation’s structure so that previously independent departments become part of one development unit. cooperation between users and it staff should be enabled to define requirements iteratively throughout the project and prioritise them. continuous definition and configuration work, feedback, and self-organisation should focus on the most valuable features and requirements. this comprehensive approach ensures australasian journal of information systems kainulainen, tuunanen & vartiainen 2024, vol 28, research article requirements risk management for continuous dev. 16 that cd projects are effective and aligned with business needs, fostering a collaborative environment that can adapt to changing requirements and risks. to support cd, organisational culture and structure must be developed to promote collaboration, flexibility, and transparency. this involves creating a unified development unit where previously siloed departments work together seamlessly, facilitating communication and cooperation. an organisational culture that encourages iterative feedback and values continuous improvement is essential. such a culture supports the iterative definition and prioritisation of requirements, ensuring that development work remains aligned with business goals. organisational knowledge plays a crucial role in the success of rrm. businesses can more accurately identify and articulate their needs by leveraging the organisation's collective expertise and insights. this shared knowledge base helps in the iterative requirement definition process, making it easier to adjust to changes and effectively incorporate feedback. ensuring that all stakeholders are well-informed and engaged in the development process enhances the overall quality and relevance of the requirements. establishing robust feedback mechanisms and maintaining close collaboration between it staff and business users is vital to ensuring the accuracy of the business's requirements. regularly scheduled reviews, continuous configuration work, and self-organised teams can help capture the most critical and valuable features. additionally, a mixed methods study (venkatesh et al., 2013) examining cultural factors within cd environments and their effects on feedback processes is recommended. the objective of such a study is to investigate how organisational culture influences the implementation of continuous feedback loops in cd. understanding these cultural dynamics can provide insights into improving the accuracy and effectiveness of requirements management, ultimately contributing to the success of cd projects. 5.1.3 methods and tools: innovation for rapid cyclical development our findings also reveal that most tools and methods have been created for traditional development work (e.g. ghobadi & mathiassen, 2016; highsmith & cockburn, 2009; jiang et al., 2006; racheva et al., 2010; tuunanen & kuo, 2015) and do not consider the functionalities required for cd. consequently, new methods that support rapid cyclical development are needed. these tools must handle changing requirements and risks and account for users’ cultural influences and values when eliciting requirements (tuunanen & kuo, 2015). cd highlights the importance of knowledge-sharing tools because development work is based firmly on knowledge sharing (ghobadi & mathiassen, 2017). this point of view has neither been part of traditional development nor considered when designing tools for traditional project management (bragge & merisalo-rantanen, 2009). managers, therefore, should address requirement risks in cd projects by treating the choice of methodology and customer involvement as the most critical risk drivers of the project. they should introduce new tools and methods to respond to cyclical development processes and handle changing requirements and risks, including cultural effects and user values. managers should also focus on the changing requirements, project goals, and identified risks. they must prioritise requirements to address technical constraints, business considerations, and critical stakeholder preferences. additionally, they should introduce practical risk management approaches and tools to users, integrating them into the project while coordinating expertise. effective information sharing through open communication and good knowledge-sharing practices is essential. australasian journal of information systems kainulainen, tuunanen & vartiainen 2024, vol 28, research article requirements risk management for continuous dev. 17 we suggest conducting design science research (hevner et al., 2004; peffers et al., 2007) studies to assess the utility and performance of tools and their impact on risk management at a team level. a goal of such a study should be to evaluate the effectiveness of new tools and techniques for managing dynamic requirements in cd projects, thereby contributing to the advancement of cd practices. reflecting on these points, several critical questions emerge. how can customer requirements and demands be identified in cd? this involves engaging customers early and continuously through iterative feedback loops, using user stories, personas, and direct communication to understand and refine their needs. how can users be involved in the development process, and how should user involvement occur? users should be integrated into the development process through regular feedback sessions, usability testing, and collaborative workshops. their involvement should be structured to ensure continuous input and validation of the product's direction and functionality. addressing these questions is essential for refining methodologies and ensuring the successful implementation of cd practices. 5.1.4 pace and seamlessness: rapid iteration we also find that the pace and seamlessness dimension differs between traditional development and cd. traditional development is typically a rigid, pre-planned process whose results are available when all the development work has been completed, and the project closed, whereas cd’s development process is fast, continuous and seamless. the pace of change in cd is even quicker than in agile development (highsmith & cockburn, 2009). cd can flexibly adapt to changing demands and adjust to continuous feedback. changes are implemented as fast as possible; cycles can vary from many times a day to a few times a week or month (chen, 2015). the key argument is that users and customers do not need to wait long to receive replies to their new requests or feedback, and the short iteration cycles and frequent releases help reduce risks, clarify requirements and increase the speed of feedback (elbanna & sarker, 2015; lwakatare et al., 2019). consequently, cd’s objective is to implement system changes as a cyclical process with no definite end. managers should, thus, focus on short iteration cycles and frequent releases when addressing requirement risks in cd. to achieve this, managers should develop and implement open feedback loops and learning cycles that are as fast and short as possible. they should enable continuous customer and user feedback that is processed quickly and seamlessly from the customer’s or user’s point of view, ensuring that ideas and requirements are gathered effectively. to effectively implement cd, the organisational workflow needs to be restructured to support rapid development cycles and manage changing requirements. cross-departmental communication must be enhanced to ensure all teams are aligned and respond swiftly to changes. this might involve integrating cross-functional teams and ensuring that development, operations, quality assurance, and other relevant departments work closely together from the start. such integration helps in quicker identification and resolution of issues. automating processes, especially testing, integration, and deployment, is crucial for speeding up development cycles and reducing human error. continuous training and development are also essential to keep teams updated with cd's latest tools and practices, maintaining high efficiency and adaptability. australasian journal of information systems kainulainen, tuunanen & vartiainen 2024, vol 28, research article requirements risk management for continuous dev. 18 the project must establish robust mechanisms to provide quick responses and manage risks associated with change requests and new requirements. real-time monitoring and feedback systems are vital, providing real-time insights into the system's performance and continuously gathering user feedback. developing frameworks that assess the risks of each change request based on impact, urgency, and feasibility helps prioritise tasks and manage resources efficiently. adopting an incremental update approach, where changes are broken down into smaller, manageable updates, minimises the risk of significant disruptions and makes identifying and addressing issues easier. engaging stakeholders actively through regular updates and reviews ensures their needs are understood and met promptly, aligning project goals with business requirements. a multi-case study (see, e.g., stake, 2013) of cd projects focusing on workflow changes and their impact on risk reduction can be carried out to explore and develop the dimension of pace and seamlessness. the aim is to examine workflow changes that enable cd teams to respond quickly to change requests while managing the risks associated with requirements. this study could analyse different cd implementations to compare how various organisations have modified their workflows to accommodate cd and the outcomes of these changes. evaluating risk management practices and documenting best practices for integrating feedback loops, automating processes, and fostering cross-departmental collaboration would provide valuable insights. by reflecting on these questions and incorporating the findings into their strategies, managers can better navigate the challenges of cd and leverage its benefits for faster, more responsive, and less risky software development. 6 conclusions in this research, we summarised the is and re literature on rrm for cd to understand the state of the art, revealing that there is still little focus on rrm for cd. through an slr, we constructed a framework with dimensions describing the organisational needs of rrm for cd. this conceptual framework can set preferences and priorities for project development to better meet cd requirements, such as rapid response and prioritisation of emerging needs. we posit that cd and rrm should proceed in an interleaved step-by-step manner, with the development cycle considering new requirements and their associated risks as part of a continuous improvement process. this approach requires a change in project culture and an organisation’s operations and culture. above all, it requires new methods and tools so that the integration of rrm into the cd process is a fast, seamless and practical part of the development process. we also presented research topics and questions that should be considered in future studies (table 2). australasian journal of information systems kainulainen, tuunanen & vartiainen 2024, vol 28, research article requirements risk management for continuous dev. 19 cd element(s) implications for research and suggested research questions suggested research approaches and objectives implications for practice: how managers need to address requirement risks in cd? project culture a change in project organisation is needed when moving to cd. this puts forward research questions such as the following: how should project organisation be structured to support cd? how can all the necessary stakeholders be involved in project and project risk management? how can knowledge sharing among different organisational departments, users and stakeholders be developed? what approaches and tools are needed in a project to manage risks associated with knowledge sharing and requirements? focus on managing the organisation’s internal communications and operating environment. approach: in-depth qualitative study of cd teams, focusing on roles and knowledge-sharing dynamics. objective: analyze real-world cd team structures to determine their effectiveness in managing requirement risks. projects should be perceived as business projects, not only as it projects. stakeholders and knowledge can be found in many different departments, roles, users, etc. managers should therefore: focus on the changing requirements, project goals and identified risks. prioritise requirements to address technical constraints, business considerations and critical stakeholder preferences. introduce practical risk management approaches and tools to users, and integrate them into the project and coordinates expertise. share information through open communication and good knowledgesharing practices. organisational culture changes need to be made so that the organisational structure can support continuous feedback, definitions of iterative requirements and self-organisation in cd. this puts forward research questions such as the following: how do organisational culture and structure need to be developed to support cd? how can organisational knowledge contribute to the success of rrm? how can the accuracy of the requirements defined by the business be ensured? approach: mixed methods study examining cultural factors within cd environments and their effects on feedback processes. objective: investigate how organisational culture influences the implementation of continuous feedback loops in cd. businesses need to assign a substantial role to development work and ensure that their needs and requests are included in the project and that they obtain what they need. managers should therefore: change the organisation’s structure so that previously independent departments become part of one development unit. enable cooperation between users and it staff to define requirements iteratively throughout the project and prioritise them. australasian journal of information systems kainulainen, tuunanen & vartiainen 2024, vol 28, research article requirements risk management for continuous dev. 20 address continuous definition and configuration work, feedback and selforganisation focusing on most valuable features and requirements. methods and tools new methods that support fast, cyclical development are needed. these tools should be able to handle changing requirements and risks and consider cultural effects and user values when eliciting requirements. this puts forward research questions such as the following: how can customer requirements and demands be identified in cd? how can users be involved in the development process, and how should user involvement occur? approach: design science research study conducted with cd teams, assessing tool utility and performance and its impact on risk management. objective: evaluate the utility and effectiveness of new tools and techniques for managing dynamic requirements in cd projects. the choice of methodology and customer involvement need to be treated as the most critical risk drivers of the project. managers should therefore: introduce new tools and methods to respond to cyclical development processes and handle changing requirements and risks, including cultural effects and user values. pace and seamlessness pace and change in cd are faster than in agile development and cd should be studied from multiple perspectives, such as well-being at work. this puts forward research questions such as the following: how does the organisational workflow need to be changed to communicate across different departments and work with fast-developing cycles and changing requirements? how can the project provide quick responses and define risks to change requests and new requirements defined by the business? approach: a multiple case study of the cd projects, focusing on workflow modifications and their impact on risk mitigation. objective: examine real-world workflow adaptations that enable cd teams to respond quickly to change requests while managing requirement risks. in cd, focusing on short iteration cycles and frequent releases is essential. managers should therefore: develop and implement open feedback loops and learning cycles as fast and short as possible. enable continuous customer/user feedback that is processed quickly and seamlessly from a customer’s or user’s point of view to gather ideas and requirements. table 2. roadmap for rrm research and practice in cd australasian journal of information systems kainulainen, tuunanen & vartiainen 2024, vol 28, research article requirements risk management for continuous dev. 21 our study has some limitations. the aisbasket8, conference proceedings and a few other sources cover only part of the is field. we applied a rigorous slr approach, but this study is not a comprehensive literature review. we aimed to investigate how mainstream is journals address cd and rrm issues. similarly, we reviewed top journals publishing re research to obtain their perspectives. in addition, our research question and choice of keyword searches contributed to the limitations of the slr methodology. these limitations may also be reflected in the reduction of data and data extraction accuracy. another related limitation is related to the use of google scholar to search the literature. use of other literature sources, such ones offered by the academic publishers (e.g., ebsco or proquest), may potentially result different search results depending on the algorythms used by different search engine providers. in addition, the mapping exercise used systematic reading of the articles to determine the keyword annotation. thus, intrepretive assessment was used in the data analysis and coding, which may have introduced bias. we also recognize that our study could be extended by searching the senior scholars’ list of premier journals and choosing articles from a field of software engineering other than re. in addition, our search used specific selected terms; it can be considered whether these terms were sufficient and appropriate. application of the framework has some boundary conditions. based on the slr, the framework’s dimensions are the key challenges faced by a traditional development organisation when adopting rrm for cd. consequently, our framework can be used to 1) define the state of the organisation and development work in terms of the cd definitions, 2) how the organisation’s and project culture have evolved and 3) how they compare with the characteristics defined in the framework. in addition, we can use the framework to examine, for example, how the tools and methods used in an organisation work for cd development, what changes are needed to support cd, how knowledge sharing works in the organisation and how effective collaboration is. therefore, our framework can be used to determine an organisation’s capacity and status to adopt or use cd. it can also outline where changes should be made in an organisation to make the rapid cyclical improvement model work without unnecessary interruptions, delays or knowledge-sharing problems. in our ongoing study, we seek to answer the question of how changing requirements should be managed and prioritised in cd and further develop an rrm method we have been working on (tuunanen et al., 2023). for this, we apply a design science research approach (hevner et al., 2004; peffers et al., 2007) and the developed framework. we are testing the rrm method in an industrial environment and using it in a multinational enterprise resource planning system implementation and deployment project. our initial goal is to determine how to further develop the rrm method to better address cd projects’ needs and mitigate practitioners’ concerns about a new tool for managing project requirements risks. we see that the developed conceptual framework can help prioritise requirements and determine their scope and severity and thus directly impact our rrm method development. the framework may help determine how agile methodologies and their associated requirements affect team performance (maruping et al., 2009), which may have implications for our study. in addition, we are considering using ramesh et al.’s (2010) framework to assess the applicability of the method and identify requirements risk practices to further improve the method. it would also be interesting to see if our conceptual framework can assist us in defining and managing cultural change in organisations and how this, in turn, may impact our ongoing study. australasian journal of information systems kainulainen, tuunanen & vartiainen 2024, vol 28, research article requirements risk management for continuous dev. 22 to conclude, according to almeida et al. (2022), devops applications lack clear implementation guidelines for organisations. since the most critical challenge identified in devops is cultural change (jayakody & wijayanayake, 2023)—specifically, the change in collaboration culture—we anticipate that the developed framework can offer ways to tackle this challenge in the development method in general, but also specifically for rrm for cd. khan et al. (2022) also state that devops needs further research on which methods to adopt and how to apply and improve them, as devops requires learning new tools, skills and social norms. this is something we believe as well and our design science research study should offer insights to resolving these challenges, in addition to developing a novel method. references almeida, f., simões, j., & lopes, s. (2022). exploring the benefits of combining devops and agile. future internet, 14(2), 63. azizi, n., & rowlands, b. (2018). the moderating effects of organisational culture on the relationship between knowledge sharing and it risk management success. in proceedings of the 26th european conference on information systems (ecis), 1–10. babb, j. s., nørbjerg, j., & yates, d. j. (2017). the empire strikes back: the end of agile as we know it? in h. holone, s. koch stiberg, & j. karlsen (eds.), selected papers of the iris; issue 8, 44–49. iris. baham, c., & hirschheim, r. (2022). issues, challenges, and a proposed theoretical core of agile software development research. information systems journal, 32(1), 103–129. bragge, j., & merisalo-rantanen, h. (2009). engineering e-collaboration processes to obtain innovative end-user feedback on advanced web-based information systems. journal of the association for information systems, 10(3), 196–220. callanan, m., & spillane, a. (2016). devops: making it easy to do the right thing. ieee software, 33(3), 53–59. cao, l., mohan, k., xu, p., & ramesh, b. (2009). a framework for adapting agile development methodologies. european journal of information systems, 18(4), 332–343. cao, l., & ramesh, b. (2008). agile requirements engineering practices: an empirical study. ieee software, 25(1), 60–67. chen, l. (2015). continuous delivery: huge benefits, but challenges too. ieee software, 32(2), 50–54. chen, h.-m., kazman, r., & haziyev, s. (2016). agile big data analytics development: an architecture-centric approach. in t. x. bui & r. h. sprague, jr. (eds.) proceedings of the 49th hawaii international conference on system sciences (hicss), 5378–5387, ieee. cois, c. a., yankel, j., & connell, a. (2014). modern devops: optimizing software development through effective system interactions. in proceedings of the 2014 ieee international professional communication conference (ipcc), 1–7, ieee. davern, m., shaft, t., & te’eni, d. (2012). more enduring questions in cognitive is research: a reply. journal of the association for information systems, 13(12), 1012–1016. davis, g. b. (1982). strategies for information requirements determination. ibm systems journal, 21(1), 4–30. australasian journal of information systems kainulainen, tuunanen & vartiainen 2024, vol 28, research article requirements risk management for continuous dev. 23 davison, r. m. (2017). editorial: the limitations of limitations. information systems journal, 27(6), 695–697. dingsøyr, t., falessi, d., & power, k. (2019). agile development at scale: the next frontier. ieee software, 36(2), 30–38. dremel, c., wulf, j., herterich, m. m., waizmann, j. c., & brenner, w. (2017). how audi ag established big data analytics in its digital transformation. mis quarterly executive, 16(2), 81–100. ebert, c. (2018). 50 years of software engineering: progress and perils. ieee software, 35(5), 94– 101. elbanna, a., & sarker, s. (2015). the risks of agile software development: learning from adopters. ieee software, 33(5), 72–79. gall, m., & pigni, f. (2021). taking devops mainstream: a critical review and conceptual framework. european journal of information systems, 31(5), 548–567. gantman, s. (2011). boundary objects and internal control in outsourced isd projects: results of a pilot study. in proceedings of the 17th americas conference on information systems (amcis) (pp. 1–9). gatrell, m. (2016). the value of a single solution for end-to-end alm tool support. ieee software, 33(5), 103–105. gemino, a., reich, b. h., & sauer, c. (2007). a temporal model of information technology project performance. journal of management information systems, 24(3), 9–44. ghanbari, h. (2016). seeking technical debt in critical software development projects: an exploratory field study. in proceedings of the 49th hawaii international conference on system sciences (hicss) (pp. 5407–5416). ieee. ghantous, g. b., & gill, a. (2017). devops: concepts, practices, tools, benefits and challenges. in proceedings of the pacific asia conference on information systems (pacis) (pp. 1–12). ghobadi, s., & mathiassen, l. (2016). perceived barriers to effective knowledge sharing in agile software teams. information systems journal, 26(2), 95–125. ghobadi, s., & mathiassen, l. (2017). risks to effective knowledge sharing in agile software teams: a model for assessing and mitigating risks. information systems journal, 27(6), 699– 731. heemstra, f. j., & kusters, r. j. (1996). dealing with risk: a practical approach. journal of information technology, 11(4), 333–346. hemon-hildgen, a., rowe, f., & monnier-senicourt, l. (2020). orchestrating automation and sharing in devops teams: a revelatory case of job satisfaction factors, risk and work conditions. european journal of information systems, 29(5), 474–499. hevner, a. r., march, s. t., park, j., & ram, s. (2004). design science in information systems research. mis quarterly, 28(1), 75–105. hickey, a. m., & davis, a. m. (2004). a unified model of requirements elicitation. journal of management information systems, 20(4), 65–84. australasian journal of information systems kainulainen, tuunanen & vartiainen 2024, vol 28, research article requirements risk management for continuous dev. 24 highsmith, j., & cockburn, a. (2009). agile software development: the business of innovation. computer, 34(9), 120–127. horlach, b., drews, p., drechsler, a., schirmer, i., & böhmann, t. (2020). reconceptualising business-it alignment for enabling organisational agility. in proceedings of the 28th european conference on information systems (ecis), an online ais conference, june 15–17, 2020. hütterman, m. (2012). devops for developers. apress. hüttermann, m., & rosenkranz, c. (2019). devops: walking the shadowy bridge from development success to information systems success. in proceedings of the 40th international conference on information systems (icis), 1–9. iyawa, g. e. (2020). personal extreme programming: exploring developers’ adoption. in proceedings of the 26th americas conference on information systems (amcis), 1–10. jayakody, v., & wijayanayake, j. (2023). critical success factors for devops adoption in information systems development. international journal of information systems and project management, 11(3), 60–82. jha, a. v., teri, r., verma, s., tarafder, s., bhowmik, w., kumar mishra, s., appasani, b., srinivasulu, a., & philibert, n. (2023). from theory to practice: understanding devops culture and mindset. cogent engineering, 10(1), 1–31. jiang, j. j., klein, g., & chen, h. g. (2006). the effects of user partnering and user non-support on project performance. journal of the association for information systems, 7(2), 68–90. karlsson, l., thelin, t., regnell, b., berander, p., & wohlin, c. (2007). pair-wise comparisons versus planning game partitioning – experiments on requirements prioritization techniques. empirical software engineering, 12(1), 3–33. kautz, k., madsen, s., and nørbjerg, j. (2007). persistent problems and practices in information systems development. information systems journal, 17(3), 217–239. keil, m., tiwana, a., & bush, a. (2002). reconciling user and project manager perceptions of it project risk: a delphi study. information systems journal, 12(2), 103–119. khan, m. s., khan, a. w., khan, f., khan, m. a., & whangbo, t. k. (2022). critical challenges to adopt devops culture in software organizations: a systematic review. ieee access, 10, 14339–14349. kiper, j. r. (2016). needs to know: validating user needs for a proposed fbi academy knowledge management system. in proceedings of the 49th hawaii international conference on system sciences (hicss) (pp. 4334–4343). ieee. kitchenham, b., pretorius, r., budgen, d., brereton, o. p., turner, m., niazi, m., & linkman, s. (2010). systematic literature reviews in software engineering—a tertiary study. information and software technology, 52(8), 792–805. klein, h. k., & myers, m. d. (1999). a set of principles for conducting and evaluating interpretive field studies in information systems. mis quarterly, 23(1), 67-93. australasian journal of information systems kainulainen, tuunanen & vartiainen 2024, vol 28, research article requirements risk management for continuous dev. 25 krancher, o., luther, p., & jost, m. (2018). key affordances of platform-as-a-service: selforganization and continuous feedback. journal of management information systems, 35(3), 776–812. krey, m., kabbout, a., osmani, l., & saliji, a. (2022). devops adoption: challenges & barriers. in proceedings of the 55th hawaii international conference on system sciences (hicss), virtual, january 3–7, 2022, 7297–7309, university of hawai’i at manoa. lee, j. s., keil, m., & wong, k. f. e. (2021). when a growth mindset can backfire and cause escalation of commitment to a troubled information technology project. information systems journal, 31(1), 7–32. li, e. y., jiang, j. j., & klein, g. (2003). the impact of organizational coordination and climate on marketing executives’ satisfaction with information systems services. journal of the association for information systems, 4(1), 99–117. lwakatare, l. e., karvonen, t., sauvola, t., kuvaja, p., olsson, h. h., bosch, j., & oivo, m. (2016). towards devops in the embedded systems domain: why is it so hard? in proceedings of the 49th hawaii international conference on system sciences (hicss), 5437– 5446. ieee. lwakatare, l. e., kilamo, t., karvonen, t., sauvola, t., heikkilä, v., itkonen, j., kuvaja, p., mikkonen, t., oivo, m., & lassenius, c. (2019). devops in practice: a multiple case study of five companies. information and software technology, 114(1), 217–230. mangalaraj, g., mahapatra, r., & nerur, s. (2009). acceptance of software process innovations—the case of extreme programming. european journal of information systems, 18(4), 344–354. maruping, l. m., & matook, s. (2020a). the evolution of software development orchestration: current state and an agenda for future research. european journal of information systems, 29(5), 443–457. maruping, l. m., & matook, s. (2020b). the multiplex nature of the customer representative role in agile information systems development. mis quarterly, 44(3), 1411–1437. maruping, l. m., venkatesh, v., & agarwal, r. (2009). a control theory perspective on agile methodology use and changing user requirements. information systems research, 20(3), 377–399. mathiassen, l., & pries-heje, j. (2006). business agility and diffusion of information technology. european journal of information systems, 15(2), 116–119. mathiassen, l., saarinen, t., tuunanen, t., & rossi, m. (2007). a contingency model for requirements development. journal of association of information systems, 8(11), 569–597. matook, s., & maruping, l. m. (2014). a competency model for customer representatives in agile software development projects. mis quarterly executive, 13(2), 77–95. niederman, f., brancheau, j. c., & wetherbe, j. c. (1991). information systems management issues for the 1990s. mis quarterly, 15(4), 475–500. olszewska, m., & waldén, m. (2015). devops meets formal modelling in high-criticality complex systems. in proceedings of the 1st international workshop on quality-aware devops (qudos), 7–12. acm press. australasian journal of information systems kainulainen, tuunanen & vartiainen 2024, vol 28, research article requirements risk management for continuous dev. 26 osmundsen, k., & bygstad, b. (2022). making sense of continuous development of digital infrastructures. journal of information technology, 37(2), 144–164. ozkaya, i. (2019). are devops and automation our next silver bullet? ieee software, 36(4), 3– 5. patnayakuni, r., ruppel, c. p., & rai, a. (2006). managing the complementarity of knowledge integration and process formalization for systems development performance. journal of the association for information systems, 7(8), 545–567. peffers, k., tuunanen, t., rothenberger, m. a., & chatterjee, s. (2007). a design science research methodology for information systems research. journal of management information systems, 24(3), 45–77. qureshi, i., fang, y., haggerty, n., compeau, d. r., & zhang, x. (2018). it‐mediated social interactions and knowledge sharing: role of competence‐based trust and background heterogeneity. information systems journal, 28(5), 929–955. racheva, z., daneva, m., herrmann, a., & wieringa, r. j. (2010). a conceptual model and process for client-driven agile requirements prioritization. in p. loucopoulos & j. l. cavarero (eds.), proceedings of the research challenges in information science (rcis), 2010 4th international conference, 287–298, ieee. ramesh, b., cao, l., & baskerville, r. (2010). agile requirements engineering practices and challenges: an empirical study. information systems journal, 20(5), 449–480. rowe, f. (2014). what literature review is not: diversity, boundaries and recommendations. european journal of information systems, 23(3), 241–255. royce, w. w. (1987). managing the development of large software systems: concepts and techniques. in proceedings of the 9th international conference of software engineering (icse ‘87), 328–338, acm press. salmela, h., baiyere, a., tapanainen, t., & galliers, r. d. (2022). digital agility: conceptualizing agility for the digital era. journal of the association for information systems, 23(5), 1080–1101. schmidt, r., lyytinen, k., keil, m., & cule, p. (2001). identifying software project risks: an international delphi study. journal of management information systems, 17(4), 5–36. sebastian, i. m., ross, j. w., beath, c., mocker, m., moloney, k. g., & fonstad, n. o. (2017). how big old companies navigate digital transformation. mis quarterly executive, 16(3), 197–213. sharp, j., & babb, j. (2018). is information systems late to the party? the current state of devops research in the association for information systems elibrary. in proceedings of the 24th americas conference on information systems (amcis),1–8. shimada, t., ang soo-keng, j., & ee, d. (2019). exploring the impact of is function maturity and is planning process on is planning success: an ace analysis. european journal of information systems, 28(4), 457–472. siau, k., long, y., & ling, m. (2010). toward a unified model of information systems development success. journal of database management, 21(1), 80–101. australasian journal of information systems kainulainen, tuunanen & vartiainen 2024, vol 28, research article requirements risk management for continuous dev. 27 sletholt, m. t., hannay, j. e., pfahl, d., & langtangen, h. p. (2012). what do we know about scientific software development’s agile practices? computing in science & engineering, 14(2), 24–37. stake, r. e. (2013). multiple case study analysis. guilford press. stray, v., moe, n. b., & aasheim, a. (2019). dependency management in large-scale agile: a case study of devops teams. in proceedings of the 52nd hawaii international conference on system sciences (hicss), 7007–7016. stuckenberg, s., & heinzl, a. (2010). the impact of the software-as-a-service concept on the underlying software and service development processes. in proceedings of the pacific asia conference on information systems (pacis),1297–1308. taylor, h., artman, e., & woelfer, j. p. (2012). information technology project risk management: bridging the gap between research and practice. journal of information technology, 27(1), 17–34. tiwana, a., & keil, m. (2004). the one-minute risk assessment tool. communications of the acm, 47(11), 73–77. tuunanen, t., & kuo, i. t. (2015). the effect of culture on requirements: a value-based view of prioritization. european journal of information systems, 24(3), 295–313. tuunanen, t., vartiainen, t., kainulainen, s., & ebrahim, m. (2023). development of an agile requirements risk prioritization method: a design science research study. communications of the association for information systems, 52(1), 609-637. venkatesh, v., brown, s. a., & bala, h. (2013) bridging the qualitative-quantitative divide: guidelines for conducting mixed methods research in information systems. mis quarterly, 37(1), 21–54. venkatesh, v., rai, a., & maruping, l. m. (2018). information systems projects and individual developer outcomes: role of project managers and process control. information systems research, 29(1), 127–148. virmani, m. (2015). understanding devops & bridging the gap from continuous integration to continuous delivery. in proceedings of the 5th international conference on the innovative computing technology (intech 2015), 78–82, ieee. wallace, l., keil, m., & rai, a. (2004). how software project risk affects project performance: an investigation of the dimensions of risk and an exploratory model. decision sciences, 35(2), 289–321. wang, s. y., chang, t. h., hsu, j. s. c., & lin, t. c. (2016). a study of the influences of knowledge boundary spanning on project performance in information system development projects. in proceedings of the pacific asia conference on information systems (pacis) (pp. 1–10). wiedemann, a., wiesche, m., gewald, h., & krcmar, h. (2020). understanding how devops aligns development and operations: a tripartite model of intra-it alignment. european journal of information systems, 29(5), 458–473. australasian journal of information systems kainulainen, tuunanen & vartiainen 2024, vol 28, research article requirements risk management for continuous dev. 28 xiao, x., lindberg, a., hansen, s., & lyytinen, k. (2018). ‘computing’ requirements for open source software: a distributed cognitive approach. journal of the association for information systems, 19(12), 1217–1252. appendix 1. key concepts and definitions table a1: key concepts and definitions appendix 2. systematic literature review (slr) our slr followed kitchenham et al.’s (2010) guidelines (figure 1). in step 1, we defined the need for this study. several is articles have identified the need for further research on how cd and devops can be used in isd. we wanted to investigate how the current is literature covers the key elements of cd (step 2) and sought answers to our research question (step 3): “how information systems development (isd) development activities that are required to create an information system. (kautz et al., 2007; gantman, 2011) agile development methodology focusing on collaboration, efficiency, and flexibility in software development. (cao & ramesh, 2008; dingsøyr et al., 2019; maruping et al., 2009; mathiassen & pries-heje, 2006;) continuous development (cd) an umbrella term that includes many devops processes, including continuous integration, testing, delivery, and deployment. extends agile development by focusing on continuous and short learning cycles with a constant feedback loop. (lwakatare et al., 2016; osmundsen & bygstad, 2022; virmani, 2015) continuous integration practice that automates and combines all source code changes by several authors into a single software project (gall & pigni, 2021; stray et al., 2019). continuous deployment practice that automates the release process through frequent and automated deployments (gall & pigni, 2021). continuous delivery extension of continuous integration that aims to shorten release cycles by automating software testing and approval (chen, 2015; ghantous & gill, 2017). devops instantiation of cd: fast delivery cycles, short feedback loops and automation. extends agile principles to the entire software process and adds cooperation between operations, development, and quality assessment. (ebert, 2018; krey et al., 2022; lwakatare et al., 2016; olszewska & waldén, 2015; osmundsen & bygstad, 2022, ozkaya, 2019; stray et al., 2019) requirements risk management (rrm) focuses on the active identification and management of solution-related uncertainties. helps anticipate and solve problems that may affect requirements during the implementation process. it is a process to handle risks associated with the requirements of a project or system. (venkatesh et al., 2018; wallace et al., 2004) requirements engineering (re) defines, documents, and maintains the needs and expectations of a software system. (ramesh et al., 2010) australasian journal of information systems kainulainen, tuunanen & vartiainen 2024, vol 28, research article requirements risk management for continuous dev. 29 should requirement risks be managed in cd, and what are the organisational needs for accomplishing this?” in step 4, we searched google scholar. a more detailed description follows. a2.1 finding devops and cd in the is literature a google scholar search (step 4) was performed in august 2018 to obtain an overview of how the aisbasket8 covers cd, devops, continuous analysis, continuous implementation, continuous integration and development and secure operations (devsecops). because a simple search for “devops” returned only 15 articles, additional criteria were added to the search (step 5), as follows: “continuous analysis” or “continuous development” or “continuous implementation” or “continuous integration” or “devops” or “devsecops”. this extended search, without a defined time limit, generated 171 articles, 16 of which were selected for further processing after all the articles were read. ten other journal and conference results were found as a result of the original search or a forward and backward reference search. four of these articles were selected for further processing (step 6). the selection criteria for further processing were that the article included the search terms in the text and that the article focused on software development using agile devops or cd. if the search terms were mentioned only in an article’s reference list, it was excluded (step 7). another google scholar search with the same criteria was conducted on the proceedings of the following conferences: the american conference for is (amcis), the european conference for is (ecis), the hawaiian international conference for system sciences (hicss), the international conference for is (icis) and the pacific-asian conference is (pacis). this search generated 113 cd articles, of which 16 were selected for further examination. in september 2019, an updated search (u2) on the aisbasket8 was performed to include articles published after august 2018. this search yielded eight articles; two were chosen for further handling and coding with atlas.ti. in may 2022, an updated search (u3) on the aisbasket8 was performed to find the newest articles. of 22 found, 9 were chosen for further handling. from the is literature searches, we selected 43 of 314 articles for further processing. ten other suitable journal articles and conference proceedings were defined during the search process, and four were selected for further processing. the google scholar searches for is journals and conference proceedings returned 314 (171 + 8 + 22 + 113) articles, of which 43 (16 + 2 + 9 + 16) were selected for further processing. after adding other journals and conference proceedings, the searches returned 324 (314 + 10) items, of which 47 (43 + 4) were processed further. a2.2 finding rrm in the is literature the next step defined how widely the selected is journals covered rrm. the google scholar search on rrm without any defined time limit used the following search string: “requirement” + “risk management” -finance* -economic* -biotech* -medical* -military. the search was limited to articles from the aisbasket8. this search (s1) generated 114 articles, of which 14 were selected after the articles were read. the selection criteria for the articles were that they covered requirements and risk management in isd; articles that did not cover the desired concepts or mentioned search terms only in the reference list were left out. australasian journal of information systems kainulainen, tuunanen & vartiainen 2024, vol 28, research article requirements risk management for continuous dev. 30 figure a1. systematic literature review steps based on kitchenham et al. (2010). another google scholar search with the same criteria was conducted to cover knowledge of this subject in is conferences. articles were selected from the amcis, ecis, hicss, icis and pacis proceedings. of the 135 rrm articles generated, 10 were selected for further examination. the first two is rrm literature searches defined 24 (of 249) articles for further processing. in september 2019, an updated search (u2) on the aisbasket8 was performed to include articles published after august 2018. an updated search for rrm yielded six new articles, none of which were selected for further processing. australasian journal of information systems kainulainen, tuunanen & vartiainen 2024, vol 28, research article requirements risk management for continuous dev. 31 in may 2022, an updated search (u3) on the aisbasket8 was performed to find the newest articles. two of the 11 recent ones were chosen for further handling. from the search results and using a forward and backward reference search, 22 interesting and suitable articles and conference proceedings were found from publications other than the specified aisbasket8 or is conference proceedings, of which 10 were selected for further processing. based on the rrm search results from is journals, conference proceedings and other interesting journals and conference proceedings, 36 (14 + 0 + 2 + 10 +10) articles were selected out of the 288 (114 + 135 + 6 + 11 + 22) articles. after analysing the 36 documents, 25 were selected for further processing. a2.3 creating the framework dimensions the qualitative research software atlas.ti was used to code the articles, group the codes and define the key features of cd. certain concepts and terms emerged repeatedly from the literature. first, key elements (step 8) were extracted from the articles by tagging the paper’s paragraphs, and then the tagged sections were coded with altas.ti (step 9). code groups were formed from the codes, the metadata-level concepts were defined, and the framework’s three dimensions were combined (step 10). dimensions of the framework were determined: (1a) culture – project-related tasks, such as roles, resources, instructions and guidance, management of work, teamwork, cooperation and knowledge sharing; (1b) culture – organisation-related issues, such as organisational structure in the development process; (2) different methods and tools; and (3) pace, seamlessness and continuous work – all typical features of cd. these dimensions formed the framework used as a research lens to analyse the rrm articles (step 10). a2.4 analysing is rrm articles with a three-dimensional research lens the selected is rrm articles (25 of 36) were analysed using the three-dimensional framework described above. the aim was to discover how key features of agile development, devops and cd were defined in is rrm articles. of 25 rrm articles, 16 (64%) focused on methods and tools, 12 (48%) on culture/project structure, 7 (28%) on culture/organisational structure and 4 (16%) on development speed or seamlessness. most articles did not comment on the development approaches, other than some notes about traditional methods; 28% mentioned agile development and only 4% mentioned cd. thus, the literature review revealed that, despite the extensive is literature, there was still a lack of focus on cd. a2.5 finding rrm, devops and cd in the re literature the same search was performed in journals on re. the nine journals that published re articles (table a3) were searched without a time limit for articles that covered the desired research subject and two selected terms – rrm and cd. the search was conducted in three phases. the first phase (s1) was searching for articles with the search criterion of devops or cd. to reduce the number of results, the terms “requirements” and “risk” were added in the second search (s2). the criterion “software engineering” was added in the third search (s3). after three search cycles, the third search generated 159 (devops 86 + cd 73) articles, of which 8 (7 + 1) were selected for further processing. those that covered devops, cd or agile development, where the focus was not only on the agility of publishing or deployment of software, were selected. australasian journal of information systems kainulainen, tuunanen & vartiainen 2024, vol 28, research article requirements risk management for continuous dev. 32 many articles written from the software development perspective dealt only with the agile deployment of software versions and were thus beyond this study’s scope. a2.6 analysing the re rrm articles the selected re articles were coded with atlas.ti, with very similar results to the re articles emerging from similar concepts, thus reinforcing the concepts defined in the is articles. these articles were also analysed using the developed framework as a research lens. five (63%) articles focused on methods and tools, 3 (38%) on culture/project, 4 (50%) on culture/organisation and 5 (63%) on development speed or seamlessness. most articles commented on agile, devops or cd. these findings revealed that, in the re literature, agile development, especially devops and cd, was recognised and studied much more than in the is literature. a2.7 comparing the analyses and refining the framework in the slr, we analysed articles from the is and re literature. table a1 shows the google scholar search steps, the search results and the selected articles on requirements risk management (rrm) and continuous development (cd) from the is literature. table a3 presents the information from the re literature. table a2 lists the google scholar search results and the selected articles from is conference proceedings. the is and re articles showed an almost equal focus on the methods and tools dimension: 64% vs. 63%, respectively. the difference was slightly higher for the culture/project dimension: 48% vs. 38%, respectively. the most significant differences were in the culture/organisation and development speed or seamlessness dimensions. the is articles focused much less on these dimensions than the re articles. only 28% of the is articles concentrated on the elements of the culture/organisation dimension, compared to 50% of the re articles. the difference was even more significant in the development speed or seamlessness dimension. only 16% of the is articles presented this dimension’s elements, while in the re articles, the coverage was 63%. there was also a clear difference in mentioning the development style. on the is side, it was usually not mentioned for anything other than traditional methods, and there was still little focus on cd. in contrast, in re, most articles mentioned agile development, devops or cd. this comparative analysis showed that agile development, especially devops and cd, was much more acknowledged in the re literature than in the is literature. journal rrm cd search s1/u2/u3 selected articles s1/u2/u3 search s1/u2/u3 selected articles s1/u2/u3 european journal of information systems 10/0/0 1/0/0 16/1/7 3/1/6 information systems journal 11/0/2 3/0/1 16/1/5 1/0/0 information systems research 25/0/1 1/0/0 26/0/3 3/0/0 journal of information technology 38/1/2 4/0/0 75/1/3 1/0/1 journal of management information systems 7/0/0 1/0/0 13/2/2 2/0/1 australasian journal of information systems kainulainen, tuunanen & vartiainen 2024, vol 28, research article requirements risk management for continuous dev. 33 journal of strategic information systems 1/1/1 0/0/0 3/1/0 0/0/0 journal of the association for information systems 6/2/2 3/0/0 7/1/0 2/1/0 mis quarterly 16/2/3 1/0/1 15/1/4 4/0/1 total aisbasket8 114/6/11 14/0/2 171/8/22 16/2/9 is conference proceedings (table a2) 135 10 113 16 other journals/conference proceedings 22 10 10 4 total (including update) 288 36 324 47 table a2. google scholar search results and selected articles about rrm and cd conference proceedings rrm cd search results selected articles search results selected articles amics 11 3 29 4 ecis 10 2 17 2 hicss 24 3 29 5 icis 33 0 28 1 pacis 57 2 10 4 total 135 10 113 16 table a3. google scholar search results for rrm and cd and selected articles from is conference proceedings journal devops cd search results, s1/s2/s3 selected articles search results, s1/s2/s3 selected articles empirical software engineering 0 0 0 0 0 0 0 0 ieee software 98 39 39 6 13 10 7 1 ieee transactions on software engineering 12 7 6 0 7 5 4 0 information and software technology 23 12 12 1 24 15 14 0 information systems (is) 95 47 25 0 463 196 47 0 requirements engineering journal 0 0 0 0 0 0 0 0 software and systems modelling 1 1 1 0 0 0 0 0 software practice and expertise 13 5 2 0 4 2 1 0 software quality journal 1 1 1 0 0 0 0 0 total 243/112/86 7 511/228/73 1 table a4. google scholar search results and selected re articles australasian journal of information systems kainulainen, tuunanen & vartiainen 2024, vol 28, research article requirements risk management for continuous dev. 34 appendix 3. coding process as an example of the coding process, we present the definition of the first codes in the article “key affordances of platform-as-a-service: self-organisation and continuous feedback” by krancher, o., luther, p. and jost, m., published in 2018 in the journal on management information systems. we present the first highlighted texts we coded, from which the base codes were defined and then selected and added to the code groups to specify the dimensions. quotation 1: “… rapidly deliver innovative software, many software development teams attempt to follow movements such as agile [29] and lean [62] software development2, continuous integration and delivery [41], and devops1 [2]. common to these movements is the aim to increase agility4 (i.e., the ability to rapidly create, react to, and learn from change [19]) by adopting practices based on selforganizing5 and frequent feedback3 [28, 73]”. figure a2: codes: devops1, agile development2, frequent feedback3, agility4, self-organizing5 quotation 2: “practices based on self-organizing6 are a hallmark of the agile software development movement, which values “individuals and interactions over [externally imposed] processes” [29] and which advocates uniting business users and developers7,9 in a self-organizing team [29]. these teams make decisions about requirements10, solution designs, and the distribution of work [66]. self-organizing is also a key idea behind devops8, which advocates joint teams of developers and system administrators7 with no rigid separation of roles11 between the two [42]”. figure a3: codes: self-organizing6, cooperation7, devops8, business users9, requirements10, roles11 australasian journal of information systems kainulainen, tuunanen & vartiainen 2024, vol 28, research article requirements risk management for continuous dev. 35 quotation 3: “… as realized by the devops12 movement, software development teams often lack control over infrastructure13 and knowledge14 to manage infrastructure13”. figure a4: codes: devops12, infrastructure13, knowledge14 quotation 4: “while the devops17 movement suggests removing this barrier by including system administrators into development teams and by eliminating the rigid separation15 of roles18, empirical evidence shows that developers and operations often take their traditional division of labor for granted [63]. in such teams, the transition to self-organizing practices is a relatively slow process of cultural change16”. figure a5: codes: cooperation15, cultural change16, devops17, roles18 quotation 5: “lean methods acknowledge that such time-boxed rhythms may still delay feedback21. they recommend further “increas[ing] the frequency of the feedback loops22” [62, p. 38] because “the shorter these cycles are, the more can be learned23” [62, p. 14]. principle is put into practice by the continuous integration20 and continuous delivery19 (ci/cd) movement”. australasian journal of information systems kainulainen, tuunanen & vartiainen 2024, vol 28, research article requirements risk management for continuous dev. 36 figure a6: codes: continuous delivery19, continuous integration20, feedback21, feedback loops22, learning cycles23 quotation 6: “… agile26 methods advocate frequent feedback24 through time-boxed iterations, often of a duration of a few weeks [73], which are seen as “learning cycles25”. figure a7: codes: frequent feedback24, learning cycles25, agile26 selected codes were grouped into code groups, which were used to define dimensions. table a5 presents the codes, code groups and dimensions. code code group dimension business users project characteristics 1a) culture – project culture business value project characteristics 1a) culture – project culture business needs project characteristics 1a) culture – project culture cooperation project characteristics 1a) culture – project culture collaboration project characteristics 1a) culture – project culture communication project characteristics 1a) culture – project culture coordination project characteristics 1a) culture – project culture customer engagement project characteristics 1a) culture – project culture customer representative project characteristics 1a) culture – project culture feedback project characteristics 1a) culture – project culture frequent feedback project characteristics 1a) culture – project culture interaction project characteristics 1a) culture – project culture knowledge project characteristics 1a) culture – project culture knowledge sharing project characteristics 1a) culture – project culture learning cycles project characteristics 1a) culture – project culture obstacle project characteristics 1a) culture – project culture performance project characteristics 1a) culture – project culture project management project characteristics 1a) culture – project culture australasian journal of information systems kainulainen, tuunanen & vartiainen 2024, vol 28, research article requirements risk management for continuous dev. 37 requirements project characteristics 1a) culture – project culture resources project characteristics 1a) culture – project culture risk level project characteristics 1a) culture – project culture risk management project characteristics 1a) culture – project culture roles project characteristics 1a) culture – project culture threat project characteristics 1a) culture – project culture users and user involvement project characteristics 1a) culture – project culture business environment organisation structure 1b) culture – organisational culture infrastructure organisation structure 1b) culture – organisational culture organisation organisation structure 1b) culture – organisational culture work environment organisation structure 1b) culture – organisational culture business agility organisation structure 1b) culture – organisational culture cultural change organisation structure 1b) culture – organisational culture organisational culture organisation structure 1b) culture – organisational culture organisational change organisation structure 1b) culture – organisational culture lack of knowledge of devops continuous development 2) methods and tools lack of guidance continuous development 2) methods and tools lack of research continuous development 2) methods and tools agile development development style 2) methods and tools agile principles and methodology development style 2) methods and tools agile to devops development style 2) methods and tools continuous delivery development style 2) methods and tools continuous deployment development style 2) methods and tools continuous development development style 2) methods and tools continuous integration development style 2) methods and tools definition of devops development style 2) methods and tools development cycle development style 2) methods and tools devops development style 2) methods and tools fast agile development development style 2) methods and tools method engineering development style 2) methods and tools software development development style 2) methods and tools traditional developing development style 2) methods and tools australasian journal of information systems kainulainen, tuunanen & vartiainen 2024, vol 28, research article requirements risk management for continuous dev. 38 behaviour-driven monitoring features and components 2) methods and tools shared goal features and components 2) methods and tools tools features and components 2) methods and tools flexibility pace and seamlessness 3) pace and seamlessness quickness pace and seamlessness 3) pace and seamlessness speed pace and seamlessness 3) pace and seamlessness table a5. codes, code groups and dimensions appendix 4. analysed and referenced articles european journal of information systems cao, l., mohan, k., xu, p., & ramesh, b. (2009). a framework for adapting agile development methodologies. european journal of information systems, 18(4), 332–343. https://doi.org/10.1057/ejis.2009.26 gall, m., & pigni, f. (2021). taking devops mainstream: a critical review and conceptual framework. european journal of information systems, 31(5), 548–567. https://doi.org/10.1080/0960085x.2021.1997100 hemon-hildgen, a., rowe, f., & monnier-senicourt, l. (2020). orchestrating automation and sharing in devops teams: a revelatory case of job satisfaction factors, risk and work conditions. european journal of information systems, 29(5), 474–499. https://doi.org/10.1080/0960085x.2020.1782276 mangalaraj, g., mahapatra, r., & nerur, s. (2009). acceptance of software process innovations—the case of extreme programming. european journal of information systems, 18(4), 344–354. https://doi.org/10.1057/ejis.2009.23 maruping, l. m., & matook, s. (2020a). the evolution of software development orchestration: current state and an agenda for future research. european journal of information systems, 29(5), 443–457. https://doi.org/10.1080/0960085x.2020.1831834 mathiassen, l., & pries-heje, j. (2006). business agility and diffusion of information technology. european journal of information systems, 15(2), 116–119. https://doi.org/10.1057/palgrave.ejis.3000610 rowe, f. (2014). what literature review is not: diversity, boundaries and recommendations. european journal of information systems, 23(3), 241–255. https://doi.org/10.1057/ejis.2014.7 shimada, t., ang soo-keng, j., & ee, d. (2019). exploring the impact of is function maturity and is planning process on is planning success: an ace analysis. european journal of information systems, 28(4), 457–472. https://doi.org/10.1080/0960085x.2018.1557373 tuunanen, t., & kuo, i. t. (2015). the effect of culture on requirements: a value-based view of prioritization. european journal of information systems, 24(3), 295–313. https://doi.org/10.1057/ejis.2014.29 wiedemann, a., wiesche, m., gewald, h., & krcmar, h. (2020). understanding how devops aligns development and operations: a tripartite model of intra-it alignment. european journal of information systems, 29(5), 458–473. https://doi.org/10.1080/0960085x.2020.1782277 journal of the association for information systems bragge, j., & merisalo-rantanen, h. (2009). engineering e-collaboration processes to obtain innovative enduser feedback on advanced web-based information systems. journal of the association for information systems, 10(3), 196–220. https://doi.org/10.17705/1jais.00188 https://doi.org/10.1057/ejis.2009.26 australasian journal of information systems kainulainen, tuunanen & vartiainen 2024, vol 28, research article requirements risk management for continuous dev. 39 davern, m., shaft, t., & te’eni, d. (2012). more enduring questions in cognitive is research: a reply. journal of the association for information systems, 13(12), 1012–1016. https://doi.org/10.17705/1jais.00317 jiang, j. j., klein, g., & chen, h. g. (2006). the effects of user partnering and user non-support on project performance. journal of the association for information systems, 7(2), 68–90. https://doi.org/10.17705/1jais.00082 li, e. y., jiang, j. j., & klein, g. (2003). the impact of organizational coordination and climate on marketing executives’ satisfaction with information systems services. journal of the association for information systems, 4(1), 99–117. https://doi.org/10.17705/1jais.00031 patnayakuni, r, ruppel, c. p., & rai, a. (2006). managing the complementarity of knowledge integration and process formalization for systems development performance. journal of the association for information systems, 7(8), 545–567. https://www.proquest.com/scholarly-journals/managing-complementarityknowledge-integration/docview/198858662/se-2 salmela, h., baiyere, a., tapanainen, t., & galliers, r. d. (2022). digital agility: conceptualizing agility for the digital era. journal of the association for information systems, 23(5), 1080-1101. https://doi.org/10.17705/1jais.00767 xiao, x., lindberg, a., hansen, s., & lyytinen, k. (2018). ‘computing’ requirements for open source software: a distributed cognitive approach. journal of the association for information systems, 19(12), 1217– 1252. https://doi.org/10.17705/1jais.00525 information systems journal baham, c., & hirschheim, r. (2022). issues, challenges, and a proposed theoretical core of agile software development research. information systems journal, 32(1), 103-129. https://doi.org/10.1111/isj.12336 davison, r. m. (2017). editorial: the limitations of limitations. information systems journal, 27(6), 695–697. https://doi.org/10.1111/isj.12167 ghobadi, s., & mathiassen, l. (2016). perceived barriers to effective knowledge sharing in agile software teams. information systems journal, 26(2), 95–125. https://doi.org/10.1111/isj.12053 ghobadi, s., & mathiassen, l. (2017). risks to effective knowledge sharing in agile software teams: a model for assessing and mitigating risks. information systems journal, 27(6), 699–731. https://doi.org/10.1111/isj.12117 kautz, k., madsen, s., and nørbjerg, j. (2007). persistent problems and practices in information systems development. information systems journal, 17(3), 217–239. https://doi.org/10.1111/j.1365-2575.2007.00222.x keil, m., tiwana, a., & bush, a. (2002). reconciling user and project manager perceptions of it project risk: a delphi study. information systems journal, 12(2), 103–119. https://doi.org/10.1046/j.13652575.2002.00121.x lee, j. s., keil, m., & wong, k. f. e. (2021). when a growth mindset can backfire and cause escalation of commitment to a troubled information technology project. information systems journal, 31(1), 7–32. https://doi.org/10.1111/isj.12287 qureshi, i., fang, y., haggerty, n., compeau, d. r., & zhang, x. (2018). it‐mediated social interactions and knowledge sharing: role of competence‐based trust and background heterogeneity. information systems journal, 28(5), 929–955. https://doi.org/10.1111/isj.12181 ramesh, b., cao, l., & baskerville, r. (2010). agile requirements engineering practices and challenges: an empirical study. information systems journal, 20(5), 449–480. https://doi.org/10.1111/j.13652575.2007.00259.x mis quarterly hevner, a. r., march, s. t., park, j., & ram, s. (2004). design science in information systems research. mis quarterly, 28(1), 75–105. https://doi.org/10.2307/25148625 australasian journal of information systems kainulainen, tuunanen & vartiainen 2024, vol 28, research article requirements risk management for continuous dev. 40 maruping, l. m., & matook, s. (2020b). the multiplex nature of the customer representative role in agile information systems development. mis quarterly, 44(3), 1411–1437. https://doi.org/10.25300/misq/2020/12284 niederman, f., brancheau, j. c., & wetherbe, j. c. (1991). information systems management issues for the 1990s. mis quarterly, 15(4), 475–500. https://doi.org/10.2307/249452 venkatesh, v., brown, s. a., & bala, h. (2013). bridging the qualitative-quantitative divide: guidelines for conducting mixed methods research in information systems. mis quarterly, 37(1), 21–54. https://doi.org/10.25300/misq/2013/37.1.02 misq executive dremel, c., wulf, j., herterich, m. m., waizmann, j. c., & brenner, w. (2017) how audi ag established big data analytics in its digital transformation. mis quarterly executive, 16(2), article 3. https://aisel.aisnet.org/misqe/vol16/iss2/3 matook, s., & maruping, l. m. (2014). a competency model for customer representatives in agile software development projects. mis quarterly executive, 13(2), article 3. https://aisel.aisnet.org/misqe/vol13/iss2/3 sebastian, i. m., ross, j. w., beath, c., mocker, m., moloney, k. g., & fonstad, n. o. (2017). how big old companies navigate digital transformation. mis quarterly executive, 16(3), article 6. https://aisel.aisnet.org/misqe/vol16/iss3/6 journal of management information systems gemino, a., reich, b. h., & sauer, c. (2007). a temporal model of information technology project performance. journal of management information systems, 24(3), 9–44. https://doi.org/10.2753/mis07421222240301 hickey, a. m., & davis, a. m. (2004). a unified model of requirements elicitation. journal of management information systems, 20(4), 65–84. https://doi.org/10.1080/07421222.2004.11045786 krancher, o., luther, p., & jost, m. (2018). key affordances of platform-as-a-service: self-organization and continuous feedback. journal of management information systems, 35(3), 776–812. https://doi.org/10.1080/07421222.2018.1481636 peffers, k., tuunanen, t., rothenberger, m. a., & chatterjee, s. (2007). a design science research methodology for information systems research. journal of management information systems, 24(3), 45–77. https://doi-org.ezproxy.jyu.fi/10.2753/mis0742-1222240302 schmidt, r., lyytinen, k., keil, m., & cule, p. (2001). identifying software project risks: an international delphi study. journal of management information systems, 17(4), 5–36. https://doi.org/10.1080/07421222.2001.11045662 journal of information technology heemstra, f. j., & kusters, r. j. (1996). dealing with risk: a practical approach. journal of information technology, 11(4), 333–346. https://doi.org/10.1177/026839629601100407 taylor, h., artman, e., & woelfer, j. p. (2012). information technology project risk management: bridging the gap between research and practice. journal of information technology, 27(1), 17–34. https://doi.org/10.1057/jit.2011.29 osmundsen, k., & bygstad, b. (2022). making sense of continuous development of digital infrastructures. journal of information technology, 37(2), 144–164. https://doi.org/10.1177/02683962211046621 information systems research maruping, l. m., venkatesh, v., & agarwal, r. (2009). a control theory perspective on agile methodology use and changing user requirements. information systems research, 20(3), 377–399. https://doi.org/10.1287/isre.1090.0238 australasian journal of information systems kainulainen, tuunanen & vartiainen 2024, vol 28, research article requirements risk management for continuous dev. 41 venkatesh, v., rai, a., & maruping, l. m. (2018). information systems projects and individual developer outcomes: role of project managers and process control. information systems research, 29(1), 127–148. https://doi.org/10.1287/isre.2017.0723 journal of association of information systems mathiassen, l., saarinen, t., tuunanen, t., & rossi, m. (2007). a contingency model for requirements development. journal of association of information systems, 8(11), 569–597. https://doi.org/10.17705/1jais.00143 ieee access khan, m. s., khan, a. w., khan, f., khan, m. a., & whangbo, t. k. (2022). critical challenges to adopt devops culture in software organizations: a systematic review. ieee access, 10, 14339–14349. https://doi.org/10.1109/access.2022.3145970 ieee software callanan, m., & spillane, a. (2016). devops: making it easy to do the right thing. ieee software, 33(3), 53– 59. https://doi.org/10.1109/ms.2016.66 cao, l., & ramesh, b. (2008). agile requirements engineering practices: an empirical study. ieee software, 25(1), 60–67. https://doi.org/10.1109/ms.2008.1 chen, l. (2015). continuous delivery: huge benefits, but challenges too. ieee software, 32(2), 50–54. https://doi.org/10.1109/ms.2015.27 dingsøyr, t., falessi, d., & power, k. (2019). agile development at scale: the next frontier. ieee software, 36(2), 30–38. https://doi.org/10.1109/ms.2018.2884884 ebert, c. (2018). 50 years of software engineering: progress and perils. ieee software, 35(5), 94–101. https://doi.org/10.1109/ms.2018.3571228 elbanna, a., & sarker, s. (2015). the risks of agile software development: learning from adopters. ieee software, 33(5), 72–79. https://doi.org/10.1109/ms.2015.150 gatrell, m. (2016). the value of a single solution for end-to-end alm tool support. ieee software, 33(5), 103–105. https://doi.org/10.1109/ms.2016.109 ozkaya, i. (2019). are devops and automation our next silver bullet? ieee software, 36(4), 3–5. https://doi.org/10.1109/ms.2019.2910943 information and software technology kitchenham, b., pretorius, r., budgen, d., brereton, o. p., turner, m., niazi, m., & linkman, s. (2010). systematic literature reviews in software engineering—a tertiary study. information and software technology, 52(8), 792–805. https://doi.org/10.1016/j.infsof.2010.03.006 lwakatare, l. e., kilamo, t., karvonen, t., sauvola, t., heikkilä, v., itkonen, j., kuvaja, p., mikkonen, t., oivo, m., & lassenius, c. (2019). devops in practice: a multiple case study of five companies. information and software technology, 114(1), 217–230. https://doi.org/10.1016/j.infsof.2019.06.010 journal of database management siau, k., long, y., and ling, m. (2010). toward a unified model of information systems development success. journal of database management, 21(1), 80-101. https://doi.org/10.4018/jdm.2010112304 communications of the acm tiwana, a., & keil, m. (2004). the one-minute risk assessment tool. communications of the acm, 47(11), 73– 77. https://doi.org/10.1145/1029496.1029497 communications of the association for information systems australasian journal of information systems kainulainen, tuunanen & vartiainen 2024, vol 28, research article requirements risk management for continuous dev. 42 tuunanen, t., vartiainen, t., kainulainen, s., & ebrahim, m. (2023). development of an agile requirements risk prioritization method: a design science research study. communications of the association for information systems, 52(1), 609-637. https://doi.org/10.17705/1cais.05226 decision sciences wallace, l., keil, m., & rai, a. (2004). how software project risk affects project performance: an investigation of the dimensions of risk and an exploratory model. decision sciences, 35(2), 289–321. https://doi.org/10.1111/j.00117315.2004.02059.x empirical software engineering karlsson, l., thelin, t., regnell, b., berander, p., & wohlin, c. (2007). pair-wise comparisons versus planning game partitioning – experiments on requirements prioritization techniques. empirical software engineering, 12(1), 3–33. https://doi.org/10.1007/s10664-006-7240-4 ibm systems journal davis, g. b. (1982). strategies for information requirements determination, ibm systems journal, 21(1), 4– 30. https://doi.org/10.1147/sj.211.0004 computer highsmith, j., & cockburn, a. (2009). agile software development: the business of innovation. computer (long beach, calif.), 34(9), 120–127. https://doi.org/10.1109/2.947100 future internet almeida, f., simões, j., & lopes, s. (2022). exploring the benefits of combining devops and agile. future internet, 14(2), 63. https://doi.org/10.3390/fi14020063 international journal of information systems and project management jayakody, v., & wijayanayake, j. (2023). critical success factors for devops adoption in information systems development. international journal of information systems and project management, 11(3), 60-82. https://doi.org/10.12821/ijispm110304 cogent engineering jha, a. v., teri, r., verma, s., tarafder, s., bhowmik, w., kumar mishra, s., appasani, b., srinivasulu, a., & philibert, n. (2023). from theory to practice: understanding devops culture and mindset. cogent engineering, 10(1), 1-31. https://doi.org/10.1080/23311916.2023.2251758 computing in science & engineering sletholt, m., hannay, j. e., langtangen, h. p., & pfahl, d. (2012). what do we know about scientific software development’s agile practices? computing in science & engineering, 14(2), 24–37. https://doi.org/10.1109/mcse.2011.113 conference proceedings azizi, n., & rowlands, b. (2018, june 23-28). the moderating effects of organisational culture on the relationship between knowledge sharing and it risk management success [research-in-progress papers]. 26th european conference on information systems (ecis), portsmouth, united kingdom. https://aisel.aisnet.org/ecis2018_rip/39 babb, j. s., nørbjerg, j., & yates, d. j. (2017, august 6-9). the empire strikes back: the end of agile as we know it? [paper presentation]. selected papers of the iris, halden, norway. http://aisel.aisnet.org/iris2017/8 chen, h-m., kazman, r., & haziyev, s. (2016, january 5-8). agile big data analytics development: an architecture-centric approach [paper presentation]. 49th hawaii international conference on system sciences (hicss), koloa, hi, usa. https://doi.org/10.1109/hicss.2016.665 australasian journal of information systems kainulainen, tuunanen & vartiainen 2024, vol 28, research article requirements risk management for continuous dev. 43 cois, c. a., yankel, j., & connell, a. (2014, october 13-15). modern devops: optimizing software development through effective system interactions [paper presentation]. 2014 ieee international professional communication conference (ipcc), pittsburgh. pa, usa. https://doi.org/10.1109/ipcc.2014.7020388 gantman, s. (2011, august 4-8). boundary objects and internal control in outsourced isd projects: results of a pilot study [paper presentation]. 17th americas conference on information systems (amcis), 3, detroit, michigan, usa. https://aisel.aisnet.org/amcis2011_submissions/200 ghanbari, h. (2016, january 5-8). seeking technical debt in critical software development projects: an exploratory field study [paper presentation]. 49th hawaii international conference on system sciences (hicss) ), koloa, hi, usa. https://doi.org/10.1109/hicss.2016.668 ghantous, g. b., & gill, a. (2017, july 16-20). devops: concepts, practices, tools, benefits and challenges [paper presentation]. 21st pacific asia conference on information systems (pacis), langkawi, malaysia. https://aisel.aisnet.org/pacis2017/96 horlach, b., drews, p., drechsler, a., schirmer, i., & böhmann, t. (2020, june 15-17). reconceptualising business-it alignment for enabling organizational agility [paper presentation]. 28th european conference on information systems (ecis), marrakech, morrocco. https://aisel.aisnet.org/ecis2020_rp/95 hüttermann, m., & rosenkranz, c. (2019, december 15-18). devops: walking the shadowy bridge from development success to information systems success [paper presentation]. 40th international conference on information systems (icis), munich, germany. https://aisel.aisnet.org/icis2019/is_development/is_development/10 iyawa, g. e. (2020, august 10-14). personal extreme programming: exploring developers’ adoption [paper presentation]. 26th americas conference on information systems (amcis), salt lake city, utah, usa. https://aisel.aisnet.org/amcis2020/it_project_mgmt/it_project_mgmt/1 kiper, j. r. (2016, january 5-8). needs to know: validating user needs for a proposed fbi academy knowledge management system [paper presentation]. 49th hawaii international conference on system sciences (hicss), koloa, hi, usa. https://doi.org/10.1109/hicss.2016.538 krey, m., kabbout, a., osmani, l., & saliji, a. (2022, january 4-7). devops adoption: challenges & barriers [paper presentation]. 55th hawaii international conference on system sciences (hicss), maui, hi, usa. http://hdl.handle.net/10125/80219 lwakatare, l. e., karvonen, t., sauvola, t., kuvaja, p., olsson, h. h., bosch, j., & oivo, m. (2016, january 5-8). towards devops in the embedded systems domain: why is it so hard? [paper presentation]. 49th hawaii international conference on system sciences (hicss), koloa, hi, usa. https://doi.org/10.1109/hicss.2016.671 olszewska, m., & waldén, m. (2015, september 1). devops meets formal modelling in high-criticality complex systems [paper presentation]. 1st international workshop on quality-aware devops (qudos 2015), bergamo, italy. https://doi.org/10.1145/2804371.2804373 racheva, z., daneva, m., herrmann, a., & wieringa, r. j. (2010, may 19-21). a conceptual model and process for client-driven agile requirements prioritization [paper presentation]. 2010 fourth international conference on research challenges in information science (rcis 2010), nice, france. https://doi.org/10.1109/rcis.2010.5507388 royce, w. w. (1987, march 30 april 2). managing the development of large software systems: concepts and techniques [paper presentation]. 9th international conference of software engineering (icse ‘87), monterey, california, usa. https://doi.org/10.5555/41765.41801 sharp, j., & babb, j. (2018, august 16-18). is information systems late to the party? the current state of devops research in the association for information systems elibrary [paper presentation]. 24th americas conference on information systems (amcis), new orleans, la, usa. https://aisel.aisnet.org/amcis2018/advancesis/presentations/26 australasian journal of information systems kainulainen, tuunanen & vartiainen 2024, vol 28, research article requirements risk management for continuous dev. 44 stray, v., moe, n. b., & aasheim, a. (2019, january 8-11). dependency management in large-scale agile: a case study of devops teams [paper presentation]. 52nd hawaii international conference on system sciences (hicss), maui, hi, usa. http://hdl.handle.net/10125/60137 stuckenberg, s., & heinzl, a. (2010, july 9-12). the impact of the software-as-a-service concept on the underlying software and service development processes [paper presentation]. pacific asia conference on information systems (pacis), taipei, taiwan. https://aisel.aisnet.org/pacis2010/125 virmani, m. (2015, may 20-22). understanding devops & bridging the gap from continuous integration to continuous delivery [paper presentation]. 5th international conference on the innovative computing technology (intech 2015), galicia, spain. https://doi.org/10.1109/intech.2015.7173368 wang, s. y., chang, t. h., hsu, j. s. c., & lin, t. c. (2016, june 27-july 1). a study of the influences of knowledge boundary spanning on project performance in information system development projects [paper presentation]. pacific asia conference on information systems (pacis), chiayi, taiwan. http://aisel.aisnet.org/pacis2016/135 books hütterman, m. (2012) devops for developers. apress. https://doi.org/10.1007/978-1-4302-4570-4 table a6: analysed and referenced articles paper title (paper title style) australasian journal of information systems behl & pereira 2021, vol 25, research on consumer evaluation what’s behind a scratch card? of mobile applications 1 what’s behind a scratch card? designing a mobile application using gamification to study customer loyalty: an experimental approach abhishek behl op jindal global university, sonipat, india abhishekbehl27@gmail.com vijay pereira neoma business school, reims campus, france abstract the use of game design elements (often called gamification) by firms to engage the customers has attracted attention in recent times. these game elements contribute to shaping up customer’s motivation and loyalty. gamification is explored from the lens of both empirical as well as an experimental methodological standpoint. there still lacks substantial evidence that explains how, and which types of rewards help to understand the customer's motivation. the study addresses this gap by designing an experimental study of 2x2 to address how gamified mobile apps used for making payments can help capture customer’s loyalty by offering them rewards. data is collected from 385 customers who have been using mobile apps to make payments in the past. the data were tested to check if gamification positively helps the user hedonic and utilitarian motivation, which then positively impacts their loyalty. the study is also moderated by type of rewards (direct cash rewards v/s indirect third party partnered rewards) on the relationship of gamification and customer loyalty mediated through motivation. the results confirm that mobile payment apps' cash rewards are more useful, especially when the degree of uncertainty in the game element is high (scratch card). additionally, they contribute to a higher degree of utilitarian benefits to the customers. the results contribute to the extension of the self-determination theory and stimulus organism response framework as well. keywords: gamification; scratch cards; mobile app; self-determination theory; stimulus organism response; experiment 1 introduction technology is growing at a rapid pace. the growth pace has helped firms use technology to expand their business, reach out to a broader customer base, and motivate them to use their product/service. technology motivates people and changes the attitude that has helped them alter their behavior. as businesses move from a physical space to an online space, the ways and means of engagement have also transformed significantly. one such mechanism that has started to gain momentum is gamification. the essence of gamification lies in controlling the user's intrinsic motivation towards various activities, commonly employing design characteristics to games (hamari and kovisto, 2015; huotari and hamari, 2012). the use of game mechanics in a non-game context is often considered the essence of gamification's operation definition in most of the studies. the most common game mechanics include points, badges, leader boards (pbl), narratives, feedback, achievements, progress bars, etc (hamari, australasian journal of information systems behl & pereira 2021, vol 25, research on consumer evaluation what’s behind a scratch card? of mobile applications 2 2013; bittner and shipper, 2014). one of the many forms of these game elements can be seen in marketing, finance, customer management, healthcare, fitness, wellbeing, etc. gamification has emerged as a socio-technological phenomenon that can help users be motivated intrinsically, extrinsically, and socially by offering meaningful benefits through communities and social interactions. studies have discussed how the use and implementation of game elements have helped end-users stay motivated and, more importantly, keep them hooked with the mobile application (hamari and kovisto, 2014; morschheuser et al., 2018). most of the mobile applications that use game elements are engaged in omnichannel businesses (kim et. al., 2020a). moreover, these applications have a large degree of competition available in the online space. thus, game elements help the firms design their mobile apps to capture the attention of their customers better and for a longer duration. recent literature on game design reflects that most game elements are positioned in the apps to trigger either sense of hedonic motivation or utilitarian motivation of the end-user (putri et al., 2019; sailer et al., 2014). in simple words, the game elements help contribute to either intrinsic or extrinsic motivation of the user, which helps them spend more time with the mobile application. however, it would be unfair to generalize that a gamified mobile app would be more inclined to draw hedonic or utilitarian motivation. thus, in reality, the sweet spot lies somewhere between the two wherein mobile apps try to balance between the two sides of the same coin. another matter of concern surfaced by recent literature is the selection of rewards that the app offers to its users (sigala et al., 2019). it is seen that most of the apps that have similar business models try to imitate each other; for example, most of the fitness apps like nike, gramin, fitbit, etc. use similar game elements like leader boards, badges, and missions. similarly, mobile apps that improve work productivity use similar game principles listed in the octalysis framework (yang et al., 2017). thus, each mobile application that uses game elements fulfills one of the eight principles of the octalysis framework: meaning, empowerment, social influence, unpredictability, avoidance, scarcity, ownership, and accomplishment. although each of the eight drivers has something unique to offer, the present study aims to study the seventh core driver: unpredictability and curiosity in detail (chou, 2014). unpredictability has been used in marketing literature in many forms. most of its application is seen in the retailing industry, where firms have drawn inspiration from game elements like easter eggs, rolling rewards/lottery, raffle tickets, mystery boxes, etc (seaborn and fels, 2015; rodrigues et al., 2016). while firms have succeeded in using such game design elements, they have helped them gain higher sales, augment future purchases, and improve customer interactivity. it is interesting to see that while evidence proves that degree of unpredictability works, most of the evidence is empirical (zichermann and cunningham, 2011; chou, 2019). while marketing literature has covered this aspect, there is also a need to understand the same from system design and psychology (perryer et al., 2016; nicholson, 2015). it forms the first degree of motivation for our study. secondly, the study is inspired to understand the functional applicability of unpredictability to understand the end user's motivation and attitude towards using a mobile application. the study is grounded in a triangulation of theories viz. technology adoption research (venkatesh, 1999, 2000; venkatesh and davis, 2000; davis, 1989); theory on human motivationsocial exchange theory and self-determination theory (ryan and deci, 2000; deci and ryan, 1985; blau, 1964; emerson, 1976); and stimulus organism response (sor) (mehrabian and russel, 1974; yee, 2006; ryan et al., 2006; hamari et al., 2014; deterding et al., 2011). thirdly, the study contributes to mobile app design australasian journal of information systems behl & pereira 2021, vol 25, research on consumer evaluation what’s behind a scratch card? of mobile applications 3 literature by performing an experimental study to understand the effect of the presence/absence of gamification with the type of rewards received by using the mobile application. we propose setting up an experimental study for a mobile app that is used to make payments to merchants and transfer money to individuals. the study further investigates how similar mobile payment apps can be designed with appropriate use of the “unpredictability” aspect to understand the user's hedonic and utilitarian motivation. the study also aims to understand how the systematic use of game elements and rewards can improve the user's attitude, which can translate into the continued use of the mobile application. the experiment study will also help mobile app developers understand the effect of stimuli to improve the app's usability for making payments. the study is performed for “mobile payment and digital wallet services mobile applications” and not for those mobile apps with a mobile payment as one of their features. it becomes crucial to understand how game elements can help these apps compete with others' offering better features. lastly, the study contributes to the literature on mobile app design by measuring gamification's effect with different rewards (sandberg and alvesson, 2011; alvesson and sandberg, 2011). to the best of the authors' knowledge, the study is the first of its kind that explores the following research question and addresses it by using an experimental approach: how do reward types interact with game design elements to understand the motivation (hedonic and utilitarian) and customer loyalty? to address this research question, we propose a research framework using sor's foundations and supporting it with self-determination theory and flow theory (csikszentmihalyi, 2014; admiraal et al., 2011). we then test the research framework using experimental data collected from users who make payments using mobile applications. the experiment was set up and run on respondents who have a history of making payments using mobile apps and follow the recency, frequency, and monetary (rfm) principle. the remainder of the paper is arranged as follows. section 2 presents a narrative review of the literature and briefly presents a discussion on each sub-section. section 3 builds on the theoretical arguments and details the development of the research hypotheses. section 4 illustrates the research design and focuses on the details regarding conducting an experimental study. the next section presents the results of the experiment and hypotheses. section 6 discusses the results and examines each of the hypotheses, thereby linking them with the previous studies. this section also explains how the present study contributes to both theory and practice. section 7 concludes the study and offers the future scope of the study and highlights the limitations of the present study. 2 theoretical underpinning and review of literature 2.1 theoretical debate we propose a theoretical framework using a theoretical triangulation approach: stimulusorganism response (sor), social exchange theory (emerson, 1976; blau, 1964), and selfdetermination theory (ryan and deci, 2000). the study's underlying groundwork is drawn from the sor framework that helps to understand gamification's effect on the customers' behavior. the framework posits the role of stimuli (s) to promote an individual’s cognitive and/or affective reaction (o) that translates into the behavior of the end-user (r). sor has been australasian journal of information systems behl & pereira 2021, vol 25, research on consumer evaluation what’s behind a scratch card? of mobile applications 4 used in the context of gamification, behavioral design, and consumer studies. we draw upon the concept of sor to understand how game elements can be used to control user’s behavior of a mobile app. the study also uses one game element to explain the customer’s behavior towards the app’s loyalty (cho et al., 2019; gatautis et al., 2016). the second and the third theoretical pivot on which our arguments are rested are social exchange theory and self-determination theory. social exchange theory (set) works on the cost benefit analysis principle that the customers evaluate before sticking to any brand for a longer duration (harwood and garry, 2015; hsu and chen, 2018). this helps explain the importance of rewards that can be useful for explaining the loyalty of the users towards any mobile application. in addition to this, rewards make the user motivated. thus, just gamifying the mobile application would not suffice to be more interactive and does not make the user be hooked to it (wee and choong, 2019; roungas et al., 2019). instead, it is vital to understand how game elements and reward incentives of various types help to control the users' hedonic and utilitarian motivation. earlier studies propose the relationship between motivation and loyalty, gamification and loyalty, and gamification and motivation (islam et al., 2020; koivisto and hamari, 2014). the use of these theories is done to understand how users draw costbenefit analysis from the product/service every time they are being bought and how they enhance their experience of the product/service when the entire purchase process is gamified. the design and use of rewards are often used to control the customer's attitude, which in turn helps them be more inclined towards being motivated towards either hedonic or utilitarian benefits (jacoby, 2002; mekler et al., 2013; shankar and jebarajakirthy, 2019). to the best of the authors' knowledge, the three theoretical perspectives' interplay has not been studied. this helps explain the phenomenon of customer loyalty and intentions to use a mobile app in the long run. we now present a detailed discussion of the operational definitions of gamification and customer loyalty. gamification is considered to be a game-changer for marketers as well as for customers. unlike any other human-computer interactive technology that promises to engage the end-user, gamification has its application's width and depth. it helps firms improve upon their customer's experience after using a gamified version of their application. firms' success stories can be seen mostly in all prominent sectors like health and fitness, retailing, branding, banking and finance, logistics, etc (morford et al., 2014; sailer et al., 2017; triantoro et al., 2019). gamification works on the user's psychological states and behavior that helps in getting improved results for the firms (hinton et al., 2019). additionally, it builds a system that focuses on social and communicative persuasion and change in the attitude by affecting their motivations (oinas-kukkonen, 2013; oinas-kukkonen & harjumaa, 2009; hamari et al., 2014; fogg, 2003). most of the game elements used are designed following one or more game design principles that are inspired by the octalysis framework (chou, 2014). the game design helps the users create a sense of belongingness with the product or service. studies have confirmed that an intervention of game elements in the system design has improved the firm's interaction with their customer (behl and dutta, 2020; behl et al., 2020). studies also portray that loyalty benefits/rewards are also an ancient form of gamification. building on sharp and sharp (1997) arguments, it would be logical to draw a parallel from loyalty programs into gamification. however, there are differences between them based on the type of rewards received in return. while most loyalty programs are designed to offer economic incentives that can be traded for a certain number of points earned australasian journal of information systems behl & pereira 2021, vol 25, research on consumer evaluation what’s behind a scratch card? of mobile applications 5 or several transactions made for a certain worth, it is essential to understand that such rewards evoke an extrinsic motivation (deci et al., 1999). gamification, on the other hand, works on the principle of driving the intrinsic motivation that helps the user take charge of their decisions and improve their ability to make better decisions. this is often done by targeting the affective self rather than the cognitive self of the user (müller-stewens et al., 2017). it is also studied that game design elements have also helped users make better decisions, which can also be explained using behavioural economics principles. various studies have attempted to define gamification; however, various contextual definitions hold for the respective studies that have defined them (leclercq et al., 2017; kim et al., 2020b). however, its generalizability is still questioned. however, the most common and highly cited definition refers to the “use of game elements in a non-game context.” the most common elements include points, badges, and leaderboards (often referred to as pbl). palmer et al. (2012) discussed four aspects of gamification viz. social connection to boost group dynamics, feedback, and rewards such as obtained points and status, progress paths concerning rules and difficulty/complexity, and interface and user experience driven by aesthetics, integration, usability, and fun. there is a long history of the application of gamification in areas like crowdsourcing (ipeirotis & gabrilovich, 2014; eickhoff, harris, de vries, & srinivasan, 2012), marketing and advertising (terlutter& capella, 2013; cechanowicz et al., 2013); environmental behavior (lounis et al., 2014; lee et al., 2013), public engagement (tolmie et al., 2014), commerce(hamari, 2013, 2015a); education (simões et al., 2013; hakulinen et al. ,2013; filsecker & hickey, 2014; bonde et al., 2014; christy& fox, 2014), health and fitness (koivisto and hamari, 2014; hamari and koivisto, 2015; jones et al., 2014) and many more. gamification studies have been dominated by theory-focused empirical studies that have mostly addressed the end user's behavioral aspects. gamification has its linkages with many variables, and one such variable that is discussed is customer loyalty. 3 hypotheses development gamification and loyalty have been studied together in many studies. the relationship between the two constructs is often explained using social exchange theory (emerson, 1976; blau, 1964). the theory explains subjective cost-benefit analysis that the user interprets while engaging with any brand for a longer time. the trade-off is often between how much time and resources a user invests and what the user gets in return apart from the product/service. firms aim to give away tempting rewards, which qualifies for the hedonic quotient of the user largely. kwak et al. (2018) also confirms that when the consumers perceive that the rewards are lucrative and give them either hedonic or utilitarian benefits from the rewards, their loyalty towards the brand becomes high. the study also hints the businesses carefully design their reward policy as it helps in cultivating loyalty towards the brand. loughrey et al. (2018) also notes that if two or more firms intend to offer similar rewards, the firm that gives reward benefits aligned with the customers' psyche and offers rewards at much shorter intervals, is liable to gain higher loyalty compared to their competitors. therefore, it is suggested that, if possible, the firms should give away rewards every time a customer shops something with the brand. for, e.g., shopper stop gives reward points to all its customers irrespective of their shopping. a similar example can also be seen by retail stores like big bazar and carrefour that keep accumulating the points for their customers and shopping at their outlets (agarwal and malhotra, 2018; kopalle et al., 2012). australasian journal of information systems behl & pereira 2021, vol 25, research on consumer evaluation what’s behind a scratch card? of mobile applications 6 drawing upon sor's principles, customer loyalty depends on the application and loyalty program's design. recent studies confirm that reward types (self-oriented v/s altruistic) also impact the apps' loyalty. rewards in their most contemporary form are designed without game elements (hwang and choi, 2020; högberg et al., 2019). it is known that the presence of rewards increases the chances of the customer being more loyal. it is essential to explore if gamifying these rewards by one of the game elements like points, leader boards, badges, spin wheels, avatars, etc. would improve loyalty scores compared to the conventional form (gallego-durán et al., 2019; zichermann and linder, 2010). we can, therefore, hypothesize: h1: gamified mobile apps significantly impact consumer loyalty compared to the same mobile apps without gamifying it. gamification creates hedonic value, and the relationship can be explained better using flow theory (csikszentmihalyi, 2014). flow refers to“a state in which people are so involved in an activity that nothing else seems to matter; the experience is so enjoyable that people will continue to do it even at great cost, for the sheer sake of doing it.” the concept of flow thus resonates into the phrase “enjoyable” which tallies with the idea of using game elements in a website which would eventually make the experience of the end-user more enjoyable (deterding, 2012; csikszentmihalyi, 2014). additionally, following the ideas of ryan and deci (2000) that explain the linkages between intrinsic motivation and enjoyment, it is important to understand how gamified website helps users gain hedonic pleasure. studies have confirmed that game elements trigger a sense of enjoyment experienced by the end-user and make them more involved with the gamified application (blohm and leimeister, 2013). unlike shopping, where the motives can be hedonic (hirschman and holbrook, 1982), mobile payment and digital wallet service applications on smartphones behave differently. as these services work on the central idea of tap-to-pay purchases, it becomes challenging to plug game elements anywhere in a transaction process. however, firms like google pay and apple pay have to use game elements to create a hedonic value for their users. recent studies on similar platforms are mostly empirical; however, to the best of the authors' knowledge, there isn't any study that tests the relationship experimentally. we, therefore, propose: h2: hedonic motivation positively mediates the relationship between gamification and customer loyalty. literature related to technology use embarks the importance of perceived ease of use and perceived usefulness of a technology. seminal work by davis (1989) and davis et al. (1992) has postulated a positive impact of perceived ease of use and perceived usefulness with users' intention to use the technology. this holds mostly for situations where such technology helps in drawing some utilitarian benefits out of it (baptista and oliveria, 2017). however, there seems to be a reverse trend when looked at the technology that draws hedonic benefits. games motivate the users to be engaged with the system, and games mostly offer hedonic pleasure, which is often considered the most significant point that makes them hooked to the system (faiola et al., 2013; hamari et al., 2016). however, when the same game element is drawn from the setting of a game and applied to business, it has a positive impact on the utilitarian motivation by previous studies. if game elements are perceived to be easy to understand and use in a non-game context, it can positively impact the user's attitude. ease of use cognitively impact the user's mind, and their perception towards using a technology multiple times. behavioral literature also confirms that australasian journal of information systems behl & pereira 2021, vol 25, research on consumer evaluation what’s behind a scratch card? of mobile applications 7 ease of use of any technology makes it useful for the end-user, which offers utilitarian benefits (hassan et al., 2019; koivisto and hamari, 2014; melancon et al., 2011). the same can also be referred to by unified theory of acceptance and use of technology (utuat) theory and its extension. gamification helps the firms earn profits by creating a medium that they can use to do business more frequently. in the present study, going by the same logical deduction, it is important to explore if there lies any mediating effect of utilitarian motivation that the mobile app (used for making payments) gives to its users. also, following the inherent principles of self-determination theory, we propose to explore the following hypotheses: h3: utilitarian motivation positively mediates the relationship between gamification and customer loyalty. 3.1 the moderating effect of the type of rewards as classified by previous literature (hwang and choi, 2020), the types of reward are divided into two types: self-oriented rewards and altruistic rewards. the mobile payment and digital wallet service applications on phones work on rewarding the end-user with a reward against a merchant or an individual's transaction. the rewards received might interact with the app's gamified version to affect utilitarian motivation, hedonic motivation, and customer loyalty. revisiting the flow theory in this context, csíkszentmihályi (1990) defined nine dimensions of flow as “(1) clear goals, (2) immediate feedback, (3) a match between personal skills and challenges, (4) merger of action and awareness, (5) concentration on the task, (6) sense of control, (7) loss of self-consciousness, (8) altered sense of time, and (9) the experience of becoming ‘autotelic,’ i.e., doing an activity for its own sake or its intrinsic reward” (faiola et al., 2013; p. 1114). the ninth point significantly stresses the derivation of autotelic experience drawn by the enduser after operating the app while results from “an activity or situation that produces its intrinsic motivation, rewards, or incentives, specifically without any outside goals or rewards” (p. 1114). thus, it can be clearly understood that the gamified application would lead to higher customer loyalty when the rewards are received at an individual level rather than for others. the rewards are designed mostly, keeping in mind the psyche of the player (hyken, 2017; melenhorst et al., 2015). gamification, in its octalysis form, gives rewards that motivate users to get themselves engaged better with the brand. these rewards elicit position emotions and a positive attitude that makes the user loyal (chiu et al., 2018; danaher et al., 2020). rewards have been studied in various forms, and the most common form of it is monetary and nonmonetary rewards. most of the firms stick to one type of reward as a matter of their loyalty program. however, recent studies have further confirmed that these rewards start losing value after a point in time as the user is aware of what he/she is likely to get. very recently, some studies have tested the behavior of people when the rewards are layered with uncertainty (suh and wagner, 2017; hsu et al., 2017). moreover, such uncertainty would also help them get intrigued by the brand and use it more often, which eventually translates into loyalty. hofacker et al., (2016) discussed the difference between the game design elements like a spin wheel and scratch card and indicated that while spin wheels are useful game design elements for short term engagement. on the flip side, scratch cards have proved to have a long association with the brand. bittner and shipper (2014) ascertains that while cash rewards are mostly found to impact utilitarian motivation positively, non-monetary rewards positively impact hedonic motivation. however, both these types of rewards have shown a positive australasian journal of information systems behl & pereira 2021, vol 25, research on consumer evaluation what’s behind a scratch card? of mobile applications 8 impact on loyalty. the earlier studies have tested the relationships on an individual level and not in a gamified environment. it becomes interesting to revisit the relationship in the presence/absence of game elements in any mobile application. we therefore propose: h4: there is a moderating effect of the type of rewards on the relationship between gamification and customer loyalty. h5a: there is a moderating effect of the type of rewards on the relationship between gamification and hedonic motivation h5b: there is a moderating effect of the type of rewards on the relationship between gamification and utilitarian motivation next, it is essential to discuss how customer loyalty towards the mobile application translates into its intention to use it for future transactions. dide and basu (1994) discusses that a loyal customer tends to use the same product or service when exposed to different situations and in the presence or absence of its competitors. brand loyalty and behavioral intention have been tested correlated to each other on various accounts. it is well discussed in the marketing literature that a customer exposed to loyalty benefits is likely to use the same brand in the future (evanschitzky et al., 2012). therefore, various brands have launched their loyalty cards in various forms, which has helped them get customers to stick to their brand. it is also wellgrounded in the literature that while studies have formally tested the relationship between loyalty and behavioural intention, it is important to understand if the same persists in mobile payment apps that are in inherent nature different from a product (henderson et al., 2011). we therefore hypothesize: h6: customer loyalty has a positive impact on the customer’s continued intention to use a mobile payment app. based on the proposed hypotheses, we propose the research framework presented in figure 1. figure 1. conceptual framework h5a h2 h2 h4 h6 h5b h1 h3 h3 gamification (yes/no) hedonic motivation utilitarian motivation customer loyalty continued intention to use reward type – direct v/s indirect australasian journal of information systems behl & pereira 2021, vol 25, research on consumer evaluation what’s behind a scratch card? of mobile applications 9 4 4. research design 4.1 stimuli development we developed stimuli based on previous studies that have used gamification in the context of mobile digital wallet platform and online payment system (e.g.: google pay and paytm). we chose scratch cards as a form of gamification because most mobile digital wallet platforms and online payment system apps like google pay, phone pay, paytm, wechatpay etc use similar mechanisms to reward the users. additionally, creating such a game element and its execution does not require a high degree of skill and complexity. moreover, the level of uncertainty is optimally taken care of while executing it. we mimic the style of rewards offered by digital wallet platform and online payment system platforms and therefore designed a 2 x 2 conditions with a mix and match of gamification (yes vs. no) and reward type (direct and indirect rewards). we created a dummy mobile payment application that resulted in one of the four activities when a user makes a payment to an individual or a merchant. to keep the rewards simple to understand and execute, we designed the rewards that look like the real rewards that these mobile apps give. the presence of gamification refers to a condition that the user will receive a scratch card, which has a reward. an absence of gamification refers to a situation when there is no scratch card earned by the user. the second level of attribution is the type of reward received by the user with or without the scratch card. the rewards can be cash rewards, or a third party partnered rewards like coupons or discounts on purchases. the end-user can receive either of the two types of rewards irrespective of the scratch card, and the simultaneous working of both the situations leads to four situations. to make the experiment look real, the system's eligibility to generate a scratch card was when the user spends at least $10 or more, and the cash rewards ranged from $1 to $5. whereas in the case of third part partnered rewards, the monetary range of rewards remained the same; however, those could be achieved after purchasing a product or a service with the partnered brand. this exercise helped in creating an equivalence scenario across four stimuli. moreover, as our stimuli were built with a high resemblance with a popular mobile application, we developed two additional pages that had different names with similar content in it to nullify the confounding impact of respondents due to familiarity with the mobile app. the only change that these additional reward pages had was a difference in the app's name. in contrast, all other features in terms of their app design remain similar to the original stimuli. we performed the confounding effect of this as a pre-test before running the proposed model analysis. 4.2 measurement we used existing and established scales to measure all the variables. we adapted the existing scales in the study's context and used them in the pre-test to verify its reliability and validity. we took “hedonic motivation,” which had two components: enjoyment and playfulness, as proposed by hamari and kovisto (2015). under these two dimensions of hedonic motivation, we proposed to ask a total of 7 questions. these seven questions were adapted from van der heijden (2004) and webster and martocchio (1992). the questions were central towards measuring user experience for attributes like curiosity, creativity, experimentative, enjoyable, exciting, engaging, and pleasant. the utilitarian motivation also comprises two dimensions: usefulness and ease of use. we borrowed items for these two sub-dimensions from davis (1989), and a total of 8 questions were asked. the items measured attributes like clarity in australasian journal of information systems behl & pereira 2021, vol 25, research on consumer evaluation what’s behind a scratch card? of mobile applications 10 understanding, easy interface, ease in operating, ease in making payments, ease in receiving payments, ease in accessing rewards, ease in redeeming rewards, and accomplishing tasks. loyalty to the app was measured using the items from yi and joen (2003), adapted to understand the loyalty to make payments from the same mobile app. we had four items to measure the construct that captured the user's mindset to choose a particular mobile app over its available competitors. lastly, we used the scale from venkatesh and davis (2001) and bhattacharjee (2011) to measure the mobile app's continued intention. the items in this construct helped us understand the respondent's behavior to use the mobile application to make payments to merchants, pay bills, and transfer money to friends/family both soon and in a prolonged time frame. each item was measured on a 5-point likert scale (1 denotes strongly disagree and 5 denotes strongly agree). some of the questions were asked in a reversed manner, and appropriate reverse coding was used to capture its actual meaning. we referred to the previous literature that supports the use of 5 points likert scale instead of 7point scale in an experimental design. 4.3 pre-test as proposed in the previous section, we performed a pre-test on gamification's designed manipulations and on the measurement items. the idea of doing this was to understand that there is no confounding effect drawn by designing an experiment that looks like a real situation. we reached out to students at a graduate business school and called for their expression of interest to participate voluntarily in our experiment. a total of 61 students agreed to be part of the pre-test study. we first explained the concept of a reward and its types to the participants and demonstrated them as a demo. the participants were randomly assigned to two of six categories: [2 gamification (present or absent) x 2 rewards (direct or indirect)] and [2 brands (brand a or brand b)] that are presented as loyalty programs implemented by the mobile app for making or receiving a payment. we found that participants were easily able to differentiate between a gamified v/s a non-gamified condition among the given scenarios and found that the gamified version of the app was found to be more playful (χ2 = 38.191, p < .001) and contributed more on the motivation when compared to a non-gamified version (t = 5.99, p <0.001). the results further confirm that the internal consistencies of measurement items were satisfactory as cronbach alpha was found to be greater than 0.70 for each construct. additionally, results confirm that participants’ familiarity with the brand name as no confounding effect for the two brand names (brand a and brand b) as there was no significant difference found between the two brands on the key constructs: hedonic motivation (t = 0.77, p= 0.39); utilitarian motivation (t = 0.51, p = 0.88) and loyalty (t = 0.14, p = 0.83). the results helped us to fix onto four stimuli developed based on a given brand and pretested items. the results of the main study are discussed in the subsequent section. 4.4 main study we used a market research agency to select respondents to participate in our study. we received a total of 734 voluntary responses to be part of the study. we contacted them over email and checked their eligibility by asking some preliminary questions based on the familiarity and using mobile payment apps recently. based on the first level of scrutiny, we found a total of 385 respondents who qualified for our study (see table 1). the respondents were then briefed about the research, and each of the respondents was then assigned randomly to one of the four conditions. the respondents were exposed to a screen that asked them to make a payment from their dummy account to a merchant or an anonymously created australasian journal of information systems behl & pereira 2021, vol 25, research on consumer evaluation what’s behind a scratch card? of mobile applications 11 character (who can be treated as a friend). in both cases, the respondent had to spend a minimum of $10 out of a total balance of $20 added to their dummy account. upon completing the transaction, the respondent received either a scratch card with some rewards or a pop-up message that displayed a reward. the reward could be either a cashback to their dummy account or a discount coupon that they can use for purchasing an associated brand. whether received from a scratch card or without it, the cash reward ranged between zero and one dollar. the participant was informed about the deposit by a visual appeal (“congratulations! you have won a cash prize”) that was created, showing that they have won a cash reward that was deposited in their dummy account. constructs measurement items factor loadings (pca) factor loadings (cfa) cronbach alpha composite reliability mean std. dev hedonic motivation hm1 0.81 0.79 0.87 0.86 4.03 1.03 hm2 0.84 0.82 0.91 hm3 0.85 0.92 0.82 hm4 0.88 0.8 0.85 hm5 0.92 0.84 0.86 hm6 0.85 0.88 0.84 hm7 0.83 0.88 0.88 utilitarian motivation um1 0.85 0.82 0.75 0.82 4.13 1.24 um2 0.93 0.82 0.82 um3 0.92 0.85 0.85 um4 0.91 0.86 0.79 um5 0.94 0.92 0.83 um6 0.82 0.84 0.84 um7 0.81 0.86 0.87 um8 0.85 0.84 0.81 customer loyalty cl 1 0.92 0.85 0.77 0.78 3.98 1.11 cl 2 0.93 0.86 0.79 cl 3 0.89 0.88 0.81 cl 4 0.84 0.87 0.78 continued intention to use ciu 1 0.89 0.82 0.8 0.81 4.04 1.31 ciu2 0.91 0.87 0.81 ciu 3 0.9 0.89 0.78 ciu 4 0.91 0.93 0.81 table 1. internal consistencies checks of the measurement note: results are presented for both the pre-test and the main measurement model. for the main study (χ (191) = 213.87, p < .001, rmsea = 0.07, nfi = 0.89, ifi = 0.88, tli = 0.92, cfi = 0.94). cfa  confirmatory factor analysis; pca  principal component analysis the respondents were also shown the account using a highlighter every time a transaction was done, and a reward was received. the same pattern was followed when the cash reward was not gamified. similarly, in the case of the rewards that were not cash-based, the study designed rewards related to one of the four categories: entertainment, travel, food, and well-being. the rationale for deciding these categories is based on the trends that similar web applications offer to their customers. in the case of a scratch card that resulted in a reward like this, we displayed the reward column's reward along with its expiry date. while in the case where australasian journal of information systems behl & pereira 2021, vol 25, research on consumer evaluation what’s behind a scratch card? of mobile applications 12 there wasn’t any scratch card used, the users were intimated about the reward using a pop-up message (“congratulations! you have won a prize”). in this case, the reward was also shown in the rewards section of the mobile application along with the date of expiry. 5 data analysis and results 5.1 manipulation checks this section discusses the results of the manipulation checks. we used a five-point likert scale (5= “strongly agree” and 1 = “strongly disagree) for a statement: “this mobile application entails a game element such as a scratch card.” we used the statement, “the reward received by using the mobile application is designed to offer a reward that can be redeemed with a partner brand” for testing the manipulation of the type of rewards received by the user for making a transaction. the respondents who experienced a gamified condition reported that the mobile application entails a game component (scratch card in this case) when compared to those who were not exposed to a gamified application (mgamification = 4.81 vs. mno_gamification = 2.11, t (383) = 12.64, p < .001). for the reward types, it was shown that there is a significant difference between the rewards received from cash prize when compared to discount coupons for a partnered brand (mgamification = 4.39 vs. mno_gamification = 2.87, t (383) = 10.60, p < .001). the results, therefore, confirm that the experiment populated successful manipulation. next, we discuss the results of the measurement validation. 5.2 measurement validation we performed the test for reliability by calculating cronbach’s alpha's value and found that all the values are greater than 0.70. it is important to note that both the construct's composite reliability and the items' reliability were more than the threshold value (robinson et al., 1991; nunnally, 1978). we performed confirmatory factor analysis (cfa) using amos 21 and the results confirmed that there was an acceptable model fit with the data (χ (191) = 213.87, p < .001, rmsea (root mean square error of approximation) = 0.07, nfi (normed fit index) = 0.89, ifi = 0.88, tli = 0.92, cfi (comparative fit index) = 0.94). the path coefficients were also found to have significant factor loadings that are greater than the threshold value (0.70). following the guidelines of fornell and larcker (1981), we found out that the average variance extracted (ave) of each construct is greater than 0.50 of the total variances. this confirmed that the data holds on convergent validity. we then calculated the inter construct correlation and found them to be significant (p <0.001). variables hedonic motivation utilitarian motivation customer loyalty continued intention to use hedonic motivation 0.79 utilitarian motivation 0.37 0.84 customer loyalty 0.56 0.49 0.82 continued intention to use 0.61 0.46 0.51 0.89 table 2. results of inter item correlation note: all the values of correlation coefficient are significant at 0.01 level (2 tailed) lastly, the results confirmed that the ave is greater than the squared correlation coefficient, which supports that the data holds discriminant validity (chin, 1998). the same can be australasian journal of information systems behl & pereira 2021, vol 25, research on consumer evaluation what’s behind a scratch card? of mobile applications 13 referred to in table 2. thus, we performed all the required checks and tests to validate that the data is consistent on convergent and discriminant validity and internal consistency. 5.3 results of the hypothesis testing we tested the five hypotheses (h1 h5) by using the approach proposed in the previous section. the results from ancova (analysis of covariance) (f = 4.73, p = 0.01) confirmed the effect of gamified customer loyalty programs. it is found that customer loyalty (cl) has significantly greater value for gamified programs when compared to conventional nongamified programs (mno_gamification = 3.12 v/s mgamification = 5.01, t=3.52, p =0.01). we then tested for the mediation effect of utilitarian shopping motivation (h2) and hedonic shopping motivation (h3) using process model 4 (hayes, 2013). results confirmed that gamification's total effect on customer loyalty is found to be significant (β = 0.48, p =0.023) (adjusted r² = 0.73). however, we found that the direct effect was insignificant (β = 0.198, p = 0.35) considering hedonic and utilitarian shopping motivation. results also supported that utilitarian shopping motivation had a significant indirect effect on the impact of gamification on customer loyalty (β = 0.39, p <0.05), and a similar indirect effect is seen for hedonic motivation on customer loyalty (β = 0.42, p<0.05) (hayes, 2013). thus, both h2 and h3 are supported. we applied ancova to test the moderating relationship of types of rewards (direct v/s indirect) on the user's loyalty to use the mobile app. results confirmed that there is a nonsignificant effect of gamification x reward type. however, results confirmed that the main effect of gamification and reward type is found to be significant. referring to the results from figure 2, we found a significant difference between the customers' loyalty towards the mobile app between the cash and partnered rewards (non-cash discount coupons) in the case of the non-gamified version of the mobile application. on the flip side, we found a precisely opposite result in the gamification condition, which confirms that h4 is rejected. dependent variable reward type (moderator) hedonic motivation utilitarian motivation b standard error index of moderated mediation b standard error index of moderated mediation customer loyalty direct 0.48 0.131 b = -0.301 0.23 0.198 b = 0.213 indirect 0.33 0.103 0.41 0.203 table 3. results for the moderating role of type of rewards on mediation of motivation note: the results are reported after performing a bootstrap analysis (n=5000; 95% bias corrected confidence interval) as proposed by hayes (2013). to test the moderated mediation analysis of hedonic motivation and utilitarian motivation (h5a and h5b), process macro (model 7) was adopted by the type of reward. we found that the direct effect of the gamification is found to be non-significant with all mediators. referring to the results from table 3, the reward classification moderated the mediation effect of utilitarian motivation. results confirmed that the mediating effect through utilitarian rewards between gamification and customer loyalty to use mobile apps is significantly high for direct (pure cash rewards) compared to that of indirect rewards (non-cash, third party partnered rewards). a similar analysis of the results to understand the mediating effect of hedonic motivation moderated by the type of rewards did not significantly differ when looking at the australasian journal of information systems behl & pereira 2021, vol 25, research on consumer evaluation what’s behind a scratch card? of mobile applications 14 analysis for different types of rewards. thus, we can conclude that the type of rewards is not essential for understanding how game elements can induce hedonic motivation among mobile application users. we can therefore conclude that while h5a is supported, we tend to reject h5b. as the last part of the analysis to test the h6, we found a positive and significant impact of customer loyalty towards their intentions to use mobile apps in the future (adjusted r² = 0.73; β = 0.74, p< 0.01). the results also aid our understanding of how a gamification experiment can help understand customer loyalty, which can then be translated into their intentions to use the mobile app again. thus, design-based intervention can be used to improve customer engagement with the mobile application. the next section discusses the implications of the results and brings out a detailed discussion about how the results can contribute to theory and practice. 6 discussion of results this section presents a detailed discussion of results obtained after experimenting and proposes theoretical and practical implications of the same. this will help extend the body of knowledge by advancing the theoretical debate in game design, customer loyalty, and behavioral studies. gamification has been the buzzword when it comes to engaging the customers by the companies. game elements in their multiple forms and variations are used by firms to enhance repeat purchase behavior. in our study, we ground our work using an experimental study replying upon the sor framework (mehrabian and russel, 1974). the study aims to understand how uncertainty as a core driver of gamification can be used along with direct/indirect rewards to understand the loyalty and turn the user's behavior into a mobile payment application. we also build our arguments self-determination theory to dissect the user's motivation into utilitarian and hedonic motivation while making payments to merchants or friends/family. while previous studies (seaborn and fels, 2015; kwak et al., 2018; hwang and choi, 2020) have attempted to link gamification with customer loyalty, we mediate the process by understanding how two types of motivations can be used as mediators to better explain customer loyalty, especially for a mobile application that can be used for only one reason (shankar et al., 2020). we further tested how a gamified mobile application would resonate into better customer loyalty, which further translates into better chances to make payments using the same mobile application. drawing upon the results from the 2 x 2 experiment, we confirm that gamified loyalty programs with a higher degree of uncertainty and offer a direct cash reward in return seem more effective than indirect benefits. 6.1 theoretical implications the study is one of its kind to demonstrate the inter-relationship between gamification, type of rewards, motivation, and loyalty of the customer using a theoretical framework. the study designed an experiment to test how game element (scratch cards, in this case), which yields a cash reward for making a bill payment or transferring money using a mobile app, is more beneficial to measure the loyalty of the customer. while earlier studies have explained this phenomenon using the mix of gamification with self-rewards and/or altruistic rewards in nature using sor (gatautis et al., 2016; cho et al., 2019), this study offers a unique application of sor's underlying concept by understanding the effect of motivation as a mediating variable to explain loyalty. we based our study on the practically applicable scratch cards on mobile applications like paytm, apple pay, phone pay, and google pay, which offer either a cash australasian journal of information systems behl & pereira 2021, vol 25, research on consumer evaluation what’s behind a scratch card? of mobile applications 15 reward or a discount coupon of an associated/partnered brand. the study brings to table a novel theoretical contribution of understanding the interrelationship of sor with selfdetermination theory to explain customer loyalty (islam et al., 2019; ashaduzzaman et al., 2020). furthermore, we based our argument on the technology adoption model and its application in technology marketing to explain how a game element-based intervention in a mobile application can help develop a significant difference in improving customer’s loyalty to use the same app for making future payments. while earlier studies have studied customer loyalty in the context of products/services that are mostly hedonic (kwak et al., 2018; melenhorst et al., 2015), we bring a new perspective towards understanding the role of rewards in contributing to the utilitarian perspective. thus, for a business application like the one used in the study (mobile payment app), it is important to use theoretical arguments from motivation literature to understand customer loyalty. this study uniquely discusses the mediating effect of motivation (hedonic and utilitarian) to explain the phenomenon of customer loyalty. drawing from self-determination theory that explains a player's intrinsic and extrinsic motivation, we integrate the motivation of using an app with customer loyalty under the act of it being gamified (kivetz and simonson, 2002). at the same time, earlier studies (kelley and thaibaut, 1978) have used sdt to discuss the linkage between gamification with an intrinsic and extrinsic motivation that often translates into the person's attitude, thereby resulting in increased engagement. we extend the known relationships by testing the moderated mediated effect of gamification and rewards using a motivation to capture customer’s loyalty. the study investigates the game design and the hidden rewards (with its different types) to understand a customer's future app usage behavior. the study offers a unique theoretical contribution to offer critical insights to explain how cash rewards to help offer a positive psychological impact on the app user's utilitarian motivation when given through a scratch card (a game element). the study additionally offers methodological support to explain questions related to “how” and “which” rewards matter for mobile apps dealing with money transfer. most of the earlier studies have used theory to explain the “how” part of gamification, while the “which” part of gamification is often missed (loughrey et al., 2018; nicholson, 2015; putri et l., 2019). the use of experimental design for a mobile payment app is first of its kind to the best of the researcher's knowledge for measuring the loyalty of the customer. the design of mobile apps with certain game elements and a specific set of rewards have gained pace recently. however, there isn't any clarity on which game elements and what type of rewards would add to the app's success and engage the customers better. our study challenges the conventional methodology of using an empirical method of data collection. it often does not test the effect of the presence/absence of game elements and the type of rewards received by the user upon using the mobile app. 6.2 practical implications the study offers various practical implications that can help engage the customers better by designing the mobile application differently. it will also help the mobile app to get customers with a higher degree of loyalty. some of the study's critical implications point to lesser use of conventional and non-gamified mobile applications, contributing to lower loyalty towards the app. the study's findings support the argument that companies should use at least one game australasian journal of information systems behl & pereira 2021, vol 25, research on consumer evaluation what’s behind a scratch card? of mobile applications 16 element depending upon the nature of their business to develop the hook for their product/service. further, to build loyalty towards the app, the game design should be used to support utilitarian benefits more critical than hedonic benefits, especially when the app has its business dealing with monetary transactions (jebarajakirthy et al., 2020). therefore, to enhance the customer’s intention to use the app regularly, the app should focus on game elements that induce utilitarian benefits to its users. next, an important finding of the type of reward that should be used confirms that the users are more loyal to apps that offer monetary rewards than rewards linked to third party brands (uncles et al., 2003). the rewards should be delivered using a gamified mobile application, and it is suggested that the rewards be designed using a degree of uncertainty. the degree of uncertainty keeps the customer hooked to the brand, and additionally, it positively impacts its behavior to use the app (nunes and dreze, 2006). results also point out that while the third party partnered brands are important to improve upon hedonic motivation (sharma et al., 2020). however, the users who use the mobile app to pay money to merchants or transfer money to their friends/family would be more loyal to the app when a cash reward is received in return because of its nature of being tangible. the post hoc analysis results confirmed that gamified apps with cash rewards are likely to be used more in the future, as expressed by the consumer's intentions compared with non-gamified rewards that are indirect. such insights are important for game design viewpoint and for such mobile apps to partner with third party brands to measure customer engagement. the mobile apps must give optimum weight to cash in return as a reward instead of discount coupons to gain the loyalty of the customer. 7 limitations and future scope of research the study inherits some limitations in selecting the respondents, designing the experiment, and deciding upon the scenarios that should be reported. first, the data was collected from respondents that had expressed their willingness to participate in the experiment. the sample size might comprise some respondents who did not understand the experiment and the objective behind it, leading to data reporting bias. instead, the respondents should have been recruited carefully. additionally, the success of any experiment lies in having a more extensive and diverse sample size. this issue could help in reducing the overall generalizability of the results. next, the experiment's design has some assumptions that could add up the limitations of the study. first, instead of using reward types as direct and indirect, the operational definition of indirect rewards (which are third party awards) needs to be defined clearly. a very high possibility that the third party partnered rewards can positively or negatively contribute to an individual’s hedonic or utilitarian motivation (hamari, 2013). for example, a scratch card with some percentage discount on the flight ticket could be highly appreciated by a frequent flyer. however, for others, these rewards could be demotivating. similarly, a one-month membership of a popular health and fitness chain could be useful for someone who has access to that health and fitness center and is inclined towards a healthy lifestyle. thus, designing the intangible and indirect rewards must be done, keeping in mind the respondents or vice versa. second, the selection of scratch cards as a game element for our study could be replaced by other types of rewards with almost similar degrees of uncertainty. gamification literature documents that game design has a significant impact on the behavior of the player. more importantly, the study lacks testing the placement of game design elements australasian journal of information systems behl & pereira 2021, vol 25, research on consumer evaluation what’s behind a scratch card? of mobile applications 17 in the experiment. the study creates a scenario that resembles very high with what the users face in their real life, which can lead to creating a bias in their responses. however, a shorter time duration experimented with testing the effectiveness of game design on the user's behavior and their intentions to pay; it is important to repeat the experiment or record the respondents' behavior over a prolonger time duration. future studies may be explored with different types of game elements to check, which has the highest effectiveness under similar situations. it is also important to quantify game interventions by linking it with actual use and reuse of the app in the future. it is interesting also to explore how users (often called players) of different types would respond to gamified apps' continuous use when similar features are embedded in competitive mobile applications. this, in turn, would help in actually measuring the loyalty of the user towards the mobile app. the experiment can also be tested for scenarios instead of a 2x2 setting to understand how users can use complex situations to make decisions regarding the use of mobile apps. lastly, future research may investigate how gamification and rewards can be used for other mobile applications and have other business models. it is crucial to understand the functioning and non-functioning of rewards for products and services to build a theoretical model that can be used by practitioners and academicians to understand the buying behavior of the consumers. additional intervention in terms of app design and rewards can also be explored as moderators of the experiment. references admiraal, w., huizenga, j., akkerman, s., & ten dam, g. (2011). the concept of flow in collaborative game-based learning. computers in human behavior, 27(3), 1185-1194. agarwal, r., & mehrotra, a. (2018). understanding what matters to customers: the case of loyalty programs in the indian context. journal of relationship marketing, 17(4), 311-334. alvesson, m., & sandberg, j. (2011). generating research questions through problematization. academy of management review, 36(2), 247-271. ashaduzzaman, m., jebarajakirthy, c., das, m., & shankar, a. (2020). acculturation and apparel store loyalty among immigrants in western countries. journal of marketing management, 37(5-6), 488-519. baptista, g., & oliveira, t. (2017). why so serious? gamification impact in the acceptance of mobile banking services. internet research, 27(1), 118-39. behl, a., & dutta, p. (2020). engaging donors on crowdfunding platform in disaster relief operations (dro) using gamification: a civic voluntary model (cvm) approach. international journal of information management, 54, 102140. https://doi.org/10.1016/j.ijinfomgt.2020.102140. behl, a., dutta, p., sheorey, p., & singh, r. k. (2020). examining the role of dialogic communication and trust in donation-based crowdfunding tasks using information quality perspective. the tqm journal. https://doi.org/10.1108/tqm-06-2020-0139 bhattacherjee, a. (2001). understanding information systems continuance: an expectation– confirmation model. mis quarterly, 25(3), 351–370. bittner, j. v., & shipper, j. (2014). motivational effects and age differences of gamification in product advertising. journal of consumer marketing. https://doi.org/10.1016/j.ijinfomgt.2020.102140 australasian journal of information systems behl & pereira 2021, vol 25, research on consumer evaluation what’s behind a scratch card? of mobile applications 18 blau, p. (2017). exchange and power in social life. new york, usa: routledge. blohm, i., & leimeister, j. m. (2013). gamification. business & information systems engineering, 5(4), 275-278. bonde, m. t., makransky, g., wandall, j., larsen, m., morsing, v., jarmer, m., et al. (2014). improving biotech education through gamified laboratory simulations. nature biotechnology, 32(7), 694–697. chin, w. w. (1998). the partial least squares approach to structural equation modeling. modern methods for business research, 295(2), 295-336. chiu, y. l., chen, l. j., du, j., & hsu, y. t. (2018). studying the relationship between the perceived value of online group-buying websites and customer loyalty: the moderating role of referral rewards. journal of business & industrial marketing, 33(5), 665-679. https://doi.org/10.1108/jbim-03-2017-0083. cho, w. c., lee, k. y., & yang, s. b. (2019). what makes you feel attached to smartwatches? the stimulus–organism–response (s–o–r) perspectives. information technology & people. 32 no. 2, pp. 319-343. https://doi.org/10.1108/itp-05-2017-0152. christy, k. r., & fox, j. (2014). leaderboards in a virtual classroom: a test of stereo-type threat and social comparison explanations for women’s math performance. computers and education, 78, 66–77. csikszentmihalyi, m. (2000). beyond boredom and anxiety. san francisco, ca, usa: josseybass. csikszentmihalyi, m. (2014). play and intrinsic rewards. in editors missing flow and the foundations of positive psychology (pp. 135-153). dordrecht, the netherlands: springer. csikszentmihalyi, m., & csikzentmihaly, m. (1990). flow: the psychology of optimal experience (vol. 1990). new york: harper & row. danaher, p. j., sajtos, l., & danaher, t. s. (2020). tactical use of rewards to enhance loyalty program effectiveness. international journal of research in marketing, 37(3), 505-520. davis, f. d. (1989). perceived usefulness, perceived ease of use, and user acceptance of information technology. mis quarterly, 13(3), 319–340. davis, f. d., bagozzi, r. p., & warshaw, p. r. (1992). extrinsic and intrinsic motivation to use computers in the workplace 1. journal of applied social psychology, 22(14), 1111-1132. deci, e. l., & ryan, r. m. (1985). the general causality orientations scale: self-determination in personality. journal of research in personality, 19(2), 109-134. deterding, s. (2012). gamification: designing for motivation. interactions, 19(4), 14-17. dide, a. s., & basu, c. (1994). customer loyalty: toward an integrated framework. journal of the academy of marketing science, 22(2), 99-105. eickhoff, c., harris, c. g., de vries, a. p., & srinivasan, p. (2012). quality through flow and immersion: gamifying crowdsourced relevance assessments. in proceedings of the 35th international acm sigir conference on research and development in information retrieval, portland, oregon, usa, august 12–16, 2012, (pp. 871–88) australasian journal of information systems behl & pereira 2021, vol 25, research on consumer evaluation what’s behind a scratch card? of mobile applications 19 emerson, r. m. (1976). social exchange theory. annual review of sociology, 2(1), 335-362. evanschitzky, h., ramaseshan, b., woisetschläger, d. m., richelsen, v., blut, m., & backhaus, c. (2012). consequences of customer loyalty to the loyalty program and to the company. journal of the academy of marketing science, 40(5), 625-638. faiola, a., newlon, c., pfaff, m., & smyslova, o. (2013). correlating the effects of flow and telepresence in virtual worlds: enhancing our understanding of user behavior in gamebased learning. computers in human behavior, 29(3), 1113-1121. filsecker, m., & hickey, d. t. (2014). a multilevel analysis of the effects of external rewards on elementary students’ motivation, engagement and learning in an educational game. computers and education, 75, 136–148. fogg, b. j. (2003). persuasive technology: using computers to change what we think and do. boston, state missing, usa: morgan kaufmann publishers. fornell, c., & larcker, d. f. (1981). evaluating structural equation models with unobservable variables and measurement error. journal of marketing research, 18(1), 39-50. gallego-durán, f. j., villagrá-arnedo, c. j., satorre-cuerda, r., compañ-rosique, p., molinacarmona, r., & llorens-largo, f. (2019). a guide for game-design-based gamification. informatics, 6(4), 49. https://doi.org/10.3390/informatics6040049. gatautis, r., vitkauskaite, e., gadeikiene, a., & piligrimiene, z. (2016). gamification as a mean of driving online consumer behaviour: sor model perspective. engineering economics, 27(1), 90-97. hakulinen, l., auvinen, t., & korhonen, a. (2013). empirical study on the effect of achievement badges in trakla2 online learning environment. in proceedings of learning and teaching in computing and engineering (latice) conference macau, march 21–24, 2013, (pp. 47–54). hamari, j. (2013). transforming homo economicus into homo ludens: a field experiment on gamification in a utilitarian peer-to-peer trading service. electronic commerce research and applications, 12(4), 236-245. hamari, j., & koivisto, j. (2014). measuring flow in gamification: dispositional flow scale-2. computers in human behavior, 40, 133-143. hamari, j., & koivisto, j. (2015). why do people use gamification services?. international journal of information management, 35(4), 419-431. hamari, j., shernoff, d. j., rowe, e., coller, b., asbell-clarke, j., & edwards, t. (2016). challenging games help students learn: an empirical study on engagement, flow and immersion in game-based learning. computers in human behavior, 54, 170-179. harwood, t., & garry, t. (2015). an investigation into gamification as a customer engagement experience environment. journal of services marketing, 29(6/7), 533-546. https://doi.org/10.1108/jsm-01-2015-0045. https://doi.org/10.3390/informatics6040049 australasian journal of information systems behl & pereira 2021, vol 25, research on consumer evaluation what’s behind a scratch card? of mobile applications 20 hassan, l., dias, a., & hamari, j. (2019). how motivational feedback increases user’s benefits and continued use: a study on gamification, quantified-self and social networking. international journal of information management, 46, 151-162. hayes, a. f. (2017). introduction to mediation, moderation, and conditional process analysis: a regression-based approach. new york, usa: guilford publications. henderson, c. m., beck, j. t., & palmatier, r. w. (2011). review of the theoretical underpinnings of loyalty programs. journal of consumer psychology, 21(3), 256-276. hinton, s., wood, l. c., singh, h., & reiners, t. (2019). enterprise gamification systems and employment legislation: a systematic literature review. australasian journal of information systems, 23, 1-24. hofacker, c. f., de ruyter, k., lurie, n. h., manchanda, p., & donaldson, j. (2016). gamification and mobile marketing effectiveness. journal of interactive marketing, 34, 2536. högberg, j., ramberg, m. o., gustafsson, a., & wästlund, e. (2019). creating brand engagement through in-store gamified customer experiences. journal of retailing and consumer services, 50, 122-130. hsu, c. l., & chen, m. c. (2018). how gamification marketing activities motivate desirable consumer behaviors: focusing on the role of brand love. computers in human behavior, 88, 121-133. hsu, c. l., chen, y. c., yang, t. n., & lin, w. k. (2017). do website features matter in an online gamification context? focusing on the mediating roles of user experience and attitude. telematics and informatics, 34(4), 196-205. huotari, k., & hamari, j. (2017). a definition for gamification: anchoring gamification in the service marketing literature. electronic markets, 27(1), 21-31. hwang, j., & choi, l. (2020). having fun while receiving rewards?: exploration of gamification in loyalty programs for consumer loyalty. journal of business research, 106, 365-376. hyken, s. (2017). the best loyalty programs go beyond rewards. forbes. retrieved from https://www.forbes.com/sites/shephyken/2017/03/25/the-best-loyalty-programs-gobeyond-rewards/# 268709882503. ipeirotis, p. g., & gabrilovich, e. (2014). quizz: targeted crowdsourcing with a billion (potential) users. in proceedings of www ’14 seoul, korea, april 7–11, (pp.143–154). new york, usa: acm publications. islam, h., jebarajakirthy, c., & shankar, a. (2019). an experimental based investigation into the effects of website interactivity on customer behavior in on-line purchase context. journal of strategic marketing, 29(2), 117-140. islam, j. u., shahid, s., rasool, a., rahman, z., khan, i., & rather, r. a. (2020). impact of website attributes on customer engagement in banking: a solicitation of stimulusorganism-response theory. international journal of bank marketing. 38(6), 1279-1303. https://doi.org/10.1108/ijbm-12-2019-0460. australasian journal of information systems behl & pereira 2021, vol 25, research on consumer evaluation what’s behind a scratch card? of mobile applications 21 jacoby, j. (2002). stimulus-organism-response reconsidered: an evolutionary step in modeling (consumer) behavior. journal of consumer psychology, 12(1), 51-57. jebarajakirthy, c., yadav, r., & shankar, a. (2020). insights for luxury retailers to reach customers globally. marketing intelligence & planning, 38(7), 797-811. jones, b. a., madden, g. j., & wengreen, h. j. (2014). the fit game: preliminary evaluation of a gamification approach to increasing fruit and vegetable consumption in school. preventive medicine, 68, 76–79. kelley, h. h., & thibaut, j. w. (1978). interpersonal relations: a theory of interdependence. new york, usa: john wiley & sons. kim, c., costello, f. j., & lee, k. c. (2020a). the unobserved heterogeneous influence of gamification and novelty-seeking traits on consumers’ repurchase intention in the omnichannel retailing. frontiers in psychology, 11. doi-10.3389/fpsyg.2020.01664. kim, m. j., lee, c. k., & jung, t. (2020b). exploring consumer behavior in virtual reality tourism using an extended stimulus-organism-response model. journal of travel research, 59(1), 69-89. kivetz, r., & simonson, i. (2002). earning the right to indulge: effort as a determinant of customer preferences toward frequency program rewards. journal of marketing research, 39(2), 155-170. koivisto, j., & hamari, j. (2014). demographic differences in perceived benefits from gamification. computers in human behavior, 35, 179-188. kopalle, p. k., sun, y., neslin, s. a., sun, b., & swaminathan, v. (2012). the joint sales impact of frequency reward and customer tier components of loyalty programs. marketing science, 31(2), 216-235. kwak, d. h., ma, x., polites, g., srite, m., hightower, r., & haseman, w. (2018). cross-level moderation of team cohesion in individuals’ utilitarian and hedonic information processing: evidence in the context of team-based gamified training. journal of the association for information systems, 20(2). doi: 10.17705/1jais.00532. leclercq, t., poncin, i., & hammedi, w. (2017). the engagement process during value cocreation: gamification in new product-development platforms. international journal of electronic commerce, 21(4), 454-488. loughrey, k., & broin, d. (2018). are we having fun yet? misapplying motivation to gamification. in 2018 ieee games, entertainment, media conference (gem), galway, ireland, august 15-17, (pp. 1-9). lounis, s., pramatari, k., & theotokis, a. (2014). gamification is all about fun: the role of incentive type and community collaboration. in proceedings of ecis 2014, tel aviv, israel, june 9-11, 2014, (pp. 1–14). mehrabian, a., & russell, j. a. (1974). an approach to environmental psychology. cambridge, ma, usa: the mit press. mekler, e. d., brühlmann, f., opwis, k., & tuch, a. n. (2013). do points, levels and leaderboards harm intrinsic motivation? an empirical analysis of common gamification australasian journal of information systems behl & pereira 2021, vol 25, research on consumer evaluation what’s behind a scratch card? of mobile applications 22 elements. in proceedings of the first international conference on gameful design, research, and applications, toronto, canada, october 2-4,2013, (pp. 66-73), melancon, j. p., noble, s. m., & noble, c. h. (2011). managing rewards to enhance relational worth. journal of the academy of marketing science, 39(3), 341-362. melenhorst, m., novak, j., micheel, i., larson, m., & boeckle, m. (2015, october). bridging the utilitarian-hedonic divide in crowdsourcing applications. in proceedings of the fourth international workshop on crowdsourcing for multimedia, brisbane, australia, october 15, 2015, (pp. 9-14). morford, z. h., witts, b. n., killingsworth, k. j., & alavosius, m. p. (2014). gamification: the intersection between behavior analysis and game design technologies. the behavior analyst, 37(1), 25-40. morschheuser, b., hassan, l., werder, k., & hamari, j. (2018). how to design gamification? a method for engineering gamified software. information and software technology, 95, 219237. müller-stewens, j., schlager, t., häubl, g., & herrmann, a. (2017). gamified information presentation and consumer adoption of product innovations. journal of marketing, 81(2), 8-24. nicholson, s. (2015). a recipe for meaningful gamification. in gamification in education and business (pp. 1-20). springer, cham. nunes, j. c., & drèze, x. (2006). your loyalty program is betraying you. harvard business review, 84(4), 124. nunnally, j. psychometric methods. 1978. new york, usa: mcgraw hill. oinas-kukkonen, h. (2013). a foundation for the study of behavior change support systems. personal and ubiquitous computing, 17(6), 1223–1235. oinas-kukkonen, h., & harjumaa, m. (2009). persuasive systems design: key issues, process model, and system features. communications of the association for information systems, 24(1), 485–500. perryer, c., celestine, n. a., scott-ladd, b., & leighton, c. (2016). enhancing workplace motivation through gamification: transferrable lessons from pedagogy. the international journal of management education, 14(3), 327-335. putri, m. f., hidayanto, a. n., negara, e. s., budi, n. f. a., utari, p., & abidin, z. (2019). gratification sought in gamification on mobile payment. in 2019 3rd international conference on informatics and computational sciences (icicos), semarang, indonesia, october 29-30, 2019 (pp. 1-6). ieee. location and date missing robinson, j. p., shaver, p. r., & wrightsman, l. s. (1991). criteria for scale selection and evaluation. measures of personality and social psychological attitudes, 1(3), 1-16. rodrigues, l. f., oliveira, a., & costa, c. j. (2016). does ease-of-use contribute to the perception of enjoyment? a case of gamification in e-banking. computers in human behavior, 61, 114-126. australasian journal of information systems behl & pereira 2021, vol 25, research on consumer evaluation what’s behind a scratch card? of mobile applications 23 ryan, r. m., & deci, e. l. (2000). self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. american psychologist, 55(1), 68–78. ryan, r. m., rigby, c. s., & przybylski, a. (2006). the motivational pull of video games: a selfdetermination theory approach. motivation and emotion, 30(4), 344–360. sailer, m., hense, j. u., mayr, s. k., & mandl, h. (2017). how gamification motivates: an experimental study of the effects of specific game design elements on psychological need satisfaction. computers in human behavior, 69, 371-380. sailer, m., hense, j., mandl, j., & klevers, m. (2014). psychological perspectives on motivation through gamification. interaction design and architecture journal, 19, 28-37. sandberg, j., & alvesson, m. (2011). ways of constructing research questions: gap-spotting or problematization? organization, 18(1), 23-44. seaborn, k., & fels, d. i. (2015). gamification in theory and action: a survey. international journal of human-computer studies, 74, 14-31. shankar, a., & jebarajakirthy, c. (2019). the influence of e-banking service quality on customer loyalty. international journal of bank marketing, 37(5), 1119-1142. shankar, a., jebarajakirthy, c., & ashaduzzaman, m. (2020). how do electronic word of mouth practices contribute to mobile banking adoption?. journal of retailing and consumer services, 52, 101920. https://doi.org/10.1016/j.jretconser.2019.101920. sharma, i., jain, k., & behl, a. (2020). effect of service transgressions on distant third-party customers: the role of moral identity and moral judgment. journal of business research, 121, 696-712. https://doi.org/10.1016/j.jbusres.2020.02.005. sharp, b., & sharp, a. (1997). loyalty programs and their impact on repeat-purchase loyalty patterns. international journal of research in marketing, 14(5), 473–486. sigala, m., toni, m., renzi, m. f., di pietro, l., & mugion, r. g. (2019). gamification in airbnb: benefits and risks. e-review of tourism research, 16(2/3), 24-32. simões, j., díaz redondo, r., & fernández vilas, a. (2013). a social gamification framework for a k-6 learning platform. computers in human behavior, 29(2),345–353. suh, a., & wagner, c. (2017). how gamification of an enterprise collaboration system increases knowledge contribution: an affordance approach. journal of knowledge management, 21(2), 416-431. terlutter, r., & capella, m. l. (2013). the gamification of advertising: analysis and research directions of in-game advertising, advergames, and advertising in social network games. journal of advertising, 42(2-3), 95–112. tolmie, p., chamberlain, a., & benford, s. (2014). designing for reportability: sustain-able gamification, public engagement, and promoting environmental debate. personal and ubiquitous computing, 18(7), 1763–1774. triantoro, t., gopal, r., benbunan-fich, r., & lang, g. (2019). would you like to play? a comparison of a gamified survey with a traditional online survey method. international journal of information management, 49, 242-252. https://doi.org/10.1016/j.jretconser.2019.101920 https://doi.org/10.1016/j.jbusres.2020.02.005 australasian journal of information systems behl & pereira 2021, vol 25, research on consumer evaluation what’s behind a scratch card? of mobile applications 24 uncles, m. d., dowling, g. r., & hammond, k. (2003). customer loyalty and customer loyalty programs. journal of consumer marketing, 20(4), pp. 294-316. van der heijden, h. (2004). user acceptance of hedonic information systems. mis quarterly, 28(4), 695–704. venkatesh, v. (1999). creating favorable user perceptions: exploring the role of intrinsic motivation. mis quarterly, 23(2), 239–260. venkatesh, v. (2000). determinants of perceived ease of use: integrating perceived behavioral control, computer anxiety and enjoyment into the technology acceptance model. information systems research, 11(4), 342–365. webster, j., & martocchio, j. j. (1992). microcomputer playfulness: development of a measure with workplace implications. mis quarterly, 16(2), 201–226. wee, s. c., & choong, w. w. (2019). gamification: predicting the effectiveness of variety game design elements to intrinsically motivate users' energy conservation behaviour. journal of environmental management, 233, 97-106. yang, y., asaad, y., & dwivedi, y. (2017). examining the impact of gamification on intention of engagement and brand attitude in the marketing context. computers in human behavior, 73, 459-469. zichermann, g., & cunningham, c. (2011). gamification by design: implementing game mechanics in web and mobile apps. sebastopol, ca, usa: o'reilly media, inc. zichermann, g., & linder, j. (2010). game-based marketing: inspire customer loyalty through rewards, challenges, and contests. new york, usa: john wiley & sons. copyright: © 2021 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v25i0.3203 http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 theoretical underpinning and review of literature 2.1 theoretical debate 3 hypotheses development 3.1 the moderating effect of the type of rewards 4 4. research design 4.1 stimuli development 4.2 measurement 4.3 pre-test 4.4 main study 5 data analysis and results 5.1 manipulation checks 5.2 measurement validation 5.3 results of the hypothesis testing 6 discussion of results 6.1 theoretical implications 6.2 practical implications 7 limitations and future scope of research paper title (paper title style) australasian journal of information systems hassan & pandey 2021, vol 25, research on user involvement social networking sites and mindfulness 1 demystifying the dark side of social networking sites through mindfulness yusuf hassan1 institute of management technology (imt) business school dubai united arab emirates yusuf@imt.ac.ae jatin pandey indian institute of management (iim) indore india abstract over the last few decades, social networking sites (sns) have evolved as an effective medium of communication for the world. they are instrumental in connecting people across time and space with just a click. however, the darker side of sns has resulted in a deteriorated human connection between individuals in real life. the current study is an attempt to examine the compulsive usage of sns in detail. it utilizes a sequential mixed method design to examine the negative outcome of compulsive sns usage and the effect of mindfulness in overcoming them. findings of the study suggest that compulsive usage mediates the relationship between mindfulness and exhaustion; further, this relationship is moderated by extroversion personality traits. keywords: sns, csns, sequential mixed method, mindfulness, extroversion. 1 introduction every aspect of human technology has a dark side, including the bows and arrows. margaret atwood the advent of new technology is accompanied by a proclamation of the change that it will bring about. information technology (it) has been a boon to mankind for facilitating engagement across time and distance. it has the ability to digitally connect humans across different time zones with a simple click of a button. undoubtedly, the coherent and enhanced effectiveness of it is unmatchable. in fact, the general perception of it is that it helps in increasing and strengthening our social ties. however, this is just one side of the complete story. while technology is certainly taking us places, its dependency and over-usage are also coming at a cost. going back in history, the story of a human-like biological machine in a postapocalyptic earth appeared in the late 1960s classic fiction work seems to have become a reality now. the merging of technology with our body has gone to such an extent that it has become too difficult to distinguish the biologically evolved identity with the one created by man-made technology. the dark side of technology is becoming more explicit in the form of a dystopia with deteriorating human connection between individuals in their real-life (salanova, llorens & cifre, 2013). today, over 1.95 billion people log in onto two major social media platforms, namely facebook and instagram daily. these 1.95 billion daily users comprise over 25% of the world's 1 both authors contributed equally to this paper, author listing is alphabetical based on last names. australasian journal of information systems hassan & pandey 2021, vol 25, research on user involvement social networking sites and mindfulness 2 population, and they belong to different age groups and sections of our society (kiberd, 2019). furthermore, industry reports suggest that 27% of our time on the internet is used for social media usage alone (longstreet & brooks, 2017). indeed, the much-appreciated it in the form of social media platforms has entered almost everyone's private space. however, its entry in our personal space has also set the pendulum to swing the other way. problems arising from excessive social media usage have become quite evident in our personal space. we can commonly see people hooked to their mobile phones and interacting with their friends on social networking sites (sns) rather than interacting with them physically. people in the same room are found using their personal devices to engage with their neighbours in the virtual world (lee, noh & koo, 2013). the growing concerns over the loss of our social interaction as a result of increased social media engagement and its long-term negative repercussions on our health and wellbeing have instigated debates in public forums. this reliance has also been accentuated by flexible knowledge-based work arrangements and technology driven aggregator models of business (gupta, pandey, gaur & vohra, 2019; pandey & sohani, 2020). according to the world health organization (who), over the last two decades, there has been a tremendous increase in social networking sites (sns) usage, and undoubtedly, such usage had some clearly defined benefits to the users, including an enhanced awareness about health issues. however, at the same time, there have been several documented cases of excessive sns use which has resulted in negative health consequences. in fact, for some countries of the world, the problem has reached the magnitude of a significant public health concern and requires immediate scholarly attention (who, 2018). the findings of the researchers and experts suggest that social media has become a persuasive form of technology that can modify certain parts of our psyche better than the real-world equivalent does (singletary, 2020). we frequently fall prey to this feature of social media platforms (such as sns) without realizing their long-term negative implications on our wellbeing. the prominent space occupied by social engagement in our real lives has now been substantially reduced as a result of our prolonged social engagement in virtual life. instead of people consuming social media, at times, it appears that people are being consumed by social media (slater, 2002). limitations of the existing literature and contribution of the current study as discussed above, prolonged social media usage in the form of compulsive social networking site (csns) usage is resulting in weakening of our social fabrics. the physiological and psychological complications in the form of depression, fear of missing out (fomo), psychiatric disorders, obesity, sleep deprivation, and others are already some of the major health and wellbeing related issues that have started surfacing as a result of csns (dhir et al., 2018; andreassen et al., 2016; lee et al., 2014). there is also evidence indicating that in the long run, csns might also compromise the overall stability of our social institutions, such as family, economy, and education (zheng & lee, 2016). having said so, it is quite surprising that despite prior studies pointing towards the serious health and wellbeing related adverse implications of csns, scholarly research on csns still rests at a nascent stage (dhir et al., 2018). in fact, the current literature on csns is overly focused on (al-yafi, el-masri & tsai, 2018; sharif, & khanekharab, 2017) marketing-related aspects of social media usage, such as the users' buying behaviour, or their service-related satisfaction and in-role work performance among others. this has resulted in ignoring the role of individual-level factors, such as the dispositional aspects (e.g., personality) in alleviating addictive technology behaviours (here csns usage). australasian journal of information systems hassan & pandey 2021, vol 25, research on user involvement social networking sites and mindfulness 3 studying these dispositional features in relation to csns is important because it will help us to understand the subjective meanings of such dispositional aspects on our daily enterprises. further, it would help scholars and practitioners to design and formulate roles, training and rewards, keeping into consideration the unique role of personality on such compulsive behaviours. another major limitation of the existing research on csns is the absence of discussion on the role of context in social media usage. contextualizing sns or csns usage is important because it can highlight the greater role of a dynamic cultural, business, and political environment in shaping the influence of sns on its users. such insights not only hold relevance for researchers working in this domain, but also for the marketers who are looking for newer strategies to create a strong customer base through sns. over the last decade or so, the world has witnessed a gradual shift from the developed to developing economies in terms of the number of new sns users joining social media platforms. to illustrate this, in india (an emerging market) alone, there are over 351.4 million people who used sns in the year 2019 (sanam, 2020). in fact, on an average, indians spend over 2.4 hours a day on these sites. even the monthly frequency of visits on facebook for indian users have crossed 1.6 billion views (sanam, 2020). by contextualizing research on csns usage, we can highlight relevant behavioural and social manifestations associated with sns users belonging to different social and cultural settings. the current literature on csns also lacks a detailed discussion on the various outcomes of csns and potential strategies through which such prolonged social media engagement can be controlled or regulated. some of the studies which have tried to explore these avenues to some extent have limited their scope to only the psychometric disorders, issues of self-esteem, or social anxiety (apaolaza et al., 2019; andreassen et al., 2016). what and how prolonged social media usage takes place and how it can be systematically regulated remains unanswered in these studies. the motivation to do the current study is derived from the above gaps in the existing literature on compulsive social media usage, i.e., csns. through the current study, the authors have tried to explore and examine the various factors that facilitate or impede csns usage in the first place and the subsequent negative effect of compulsive usage on the individuals. by doing so, the authors have tried to leave important theoretical and managerial implications relevant to the current scholarship on csns. 1.1 compulsive sns usage compulsive sns usage refers to the individuals' inability to manage or control the usage of internet applications even though the usage of those specific internet applications might affect them adversely (fox & moreland, 2015). csns impacts the normal functioning of users and often results in severe problems, such as identity confusion, anxiety, and stress among others (lai, hsieh & zhang, 2019; sharif & khanekharab, 2017; turel & osatuyi, 2017; apaolaza et al., 2019). the pertaining question here is, why do individuals engage in compulsive sns in the first place? we observed inconsistency in the role of different factors that shape such behaviours among internet users. for example, some researchers suggest that users feel a moral obligation to engage in such behaviour. they have often reported falling victim to conditions in which they lose track of time while using these sites and later complained of sleep deprivation, fatigue, dizziness, headache, and other problems (ryan, chester, reece & xenos, 2014). some other scholars argue that excessive involvement with sns results from an expectation for certain rewards – both tangible and intangible ones (turel & osatuyi, 2017). australasian journal of information systems hassan & pandey 2021, vol 25, research on user involvement social networking sites and mindfulness 4 as we move deeper, moral obligations and expectations for rewards in the case of csns can be sourced from some of the psycho-social aspects of our lives, i.e., the dispositional, sociocultural, and reinforcing behavioural aspects of our daily enterprise (andreassen, 2015). different scholars have highlighted how one or many of these aspects are responsible for shaping and promoting csns behaviour (mcintyre, wiener & saliba, 2015; kim, larose & peng, 2009). the current study suggests that those individuals who are externally-oriented are expected to be inclined towards the utility of internet for the instrumental purpose (andreassen et al., 2013). however, individuals who had a less satisfying life were found associating themselves more with digital platforms (e.g., social networking sites, etc.). further, such associations were not for instrumental purposes, instead of to fulfill the social voids in their lives. similarly, the socio-cultural aspects, such as peer pressure, family orientation, and role modelling have also been thought to shape our attitude and behaviour towards social media. social comparison intensifies the degree of usage by creating continuous pressure on users (bandura, 1986; andreassen, 2015). users' attitude, behaviour, and their intentions can significantly vary based on the groups they are affiliated to (fishbein & ajzen, 1997). how strongly such affiliation (ex. family, friend circle, professional groups, etc.) is influencing their decisions would determine their intentions and subsequently, the intensity to use sns (baker & white, 2010). finally, reinforcing behaviour can also be one of the determinants for csns (meerkerk et al., 2010). reinforcing behaviour includes an expectation for rewards. the reward here includes both tangible and intangible ones. we can explain reinforcing behaviours using the learning theories. popularity on social networking sites encourages users to repeat behaviours, including aggressive use of sns. similarly, the absence of any appraisal, fear of missing out, and others may result in reduced sns usage. in the case of compulsive sns usage, expectations for positive outcomes, such as rewards may motivate users to engage in spending over time (moxley, 1992; ryan et al., 2014; andreassen, 2015). based on the above discussion on csns, we may demystify or conclude that there exists an inconsistency in the roles different factors play in shaping such behaviours among the sns users. therefore, it is worth exploring the factors that foster and impede csns in order to gain more clarity on this topic. we were motivated to explore the following research questions to address existing gaps in the literature on sns. first, what are the outcomes of compulsive sns usage, and how can it be controlled? we also wish to know if the relationship is affected by individual dispositional factors or not. 2 methodology the current study utilizes a sequential mixed method design to examine the various factors that foster and impedes csns. sequential mixed-method refers to the approach of collecting, analysing, and integrating quantitative as well as qualitative data during the research process within a single study (pernkopf, latzke & mayrhofer, 2020; arora & stoner, 2009; morse & niehaus, 2009; hancock, 2007). the objective of the sequential mixed method research designs is to provide a detailed understanding of the problems that we are trying to examine (ivankova, creswell & stick, 2006). when both quantitative and qualitative approaches are combined and used together in one study, it gives the researcher an opportunity to utilize the strengths of both the approaches through robust analysis. mixed methods have been australasian journal of information systems hassan & pandey 2021, vol 25, research on user involvement social networking sites and mindfulness 5 recognized for their true practical and outcome-oriented approach of inquiry (cameron, 2009). for the purpose of the current study, we followed the recommendations of janice morse for mixed-method research (tashakkori & teddlie, 2010). our research has adopted an exploratory sequential mixed method design, as suggested by morse and other scholars. accordingly, first, we followed an explorative approach using qualitative interviews to decipher the antecedents and outcomes of csns. subsequently, in study ii, we adopted a quantitative research design using the survey method to validate the findings of the first study. the advantage of using a qualitative-quantitative mixed method design for studies, such as the current one is multifold. first, this approach has been followed in prior studies on similar research problems in the domain of information and technology research (srivastava & chandra, 2018; li, worch, zhou & aguiton, 2015; mao, 2014). second, an explorative study in the form of qualitative research at the beginning helps in setting the ground to explore why and how a phenomenon occurs. this forms an essential step towards theory building, especially when the topic is unexplored. later, a quantitative study that follows the explorative research helps in attaining the generalizability or explaining the causality identified through the exploration of the research problems (berman, 2017; creswell, 2003). finally, an exploratory sequential mixed method offers voice to the participants and ensures that the findings of such studies are grounded in the participants' experiences. since the current study is about users' prolonged engagement with social media platforms (which is one of their daily experiences), this method was not only suitable but also met the desired level of rigour. 3 study i 3.1 method the main objective of study-i was to explore the outcomes of compulsive sns usage and how such compulsive usage can be regulated or controlled. the very exploratory nature of our research question encouraged us to use a qualitative research design in the first place (kumar, sachan & mukherjee, 2017; tashakkori & teddlie, 2010). scholars have discussed the ability of qualitative explorative research designs in deciphering the complexity of socio-technical interactions in our lives (von hellens, beekhuyzen & kerr, 2006). qualitative studies have the ability to explore the novices of a phenomenon and explain the underlying motivations for the occurrence of such phenomenon or behaviour (throuvala et al., 2019). in other words, qualitative research design in information technology research can offer important insights into the development, adoption, and usage of technology. exclusive research on sns and qualitative research designs have proved very effective in identifying and examining similar relationships (ding, 2019; divine et al., 2019). 3.2 sampling data for study-i was collected through semi-structured interviews conducted over a span of three months. we followed a two-step inclusion and exclusion criteria to identify and interview our respondents. at the first stage, we randomly choose people and asked them to share their frequency of social media usage. some of the preliminary questions which were asked to these random respondents were: "do you have an account on any social networking sites?", if yes, "which social networking sites have you used in the past or use at present?" and "on an average, how often do you use these social networking sites in a week?" a total of 21 respondents were interviewed in the first round. based on their preliminary responses, australasian journal of information systems hassan & pandey 2021, vol 25, research on user involvement social networking sites and mindfulness 6 participants who reported high usage of sns were invited for the main interview, i.e., the second round of interaction (cao et al., 2018; ellison, vitak, steinfield, gray, & lampe, 2011). in order to restrict the number of respondents to be interviewed in the second round, we followed the theoretical sampling approach (draucker, 2007). in the theoretical sampling approach, the sample size is determined based on the evolving theory and not by a predetermined population dimension. we engaged our respondents through multiple rounds of interviews; their responses were transcribed and coded under major themes. wherever required, we even went back to the field for more data. respondents were given the freedom to choose the location and timing for their interviews to make them comfortable. distantly located participants were interviewed through telephone and skype. interviews began with an ice-breaking session followed by responses against the research-related queries. the above approach followed by us in the second round is consistent with other studies which suggested that the interview and the analysis process should be done simultaneously (langley, 1999). we continued the process of data collection and themes generation until we started receiving repetitive responses from our sample against the research questions. based on the saturation of themes, the sample size for our study was finalized at 15. out of the 15 respondents, 8 were male respondents, whereas the rest of the 7 participants were women. the minimum educational qualification of our respondents was an undergraduate degree, and the highest qualification was doctorate. the mean age of the respondents was 26 years. 3.3 analysis all the interviews were recorded with the prior consent of our respondents. recordings were transcribed, and a copy of the verbatim transcript was also shared with the respondents in order to allow them to suggest changes (if any). once, all the interviews were transcribed, the authors checked for reliability by measuring the inter-rater reliability (kappa value). the kappa value was above 0.76, which was consistent with the recommendation made in prior studies (macphail et al., 2016; mchugh, 2012). for the purpose of analysing our responses, we used the gioia method, which is a variant of grounded theory (gioia, corley, & hamilton, 2013). grounded theory approach has been used in information technology (including issues on social media) research in the past, and hence it was deemed suitable for the current study (wiesche et al., 2017). gioia's method has been mentioned for its "qualitative rigour" without compromising the creativity and revelatory nature of such studies (gioia, corley, & hamilton, 2013). extending the learning from grounded theory, the gioia method considers that apart from the fact that reality is socially constructed, the researcher is also a knowledge agent. it argues that researchers are aware of their objectives of the research and they can explain the thoughts, intentions, and actions involved in the process of creating a socially constructed reality (gioia, corley, & hamilton, 2013). the gioia method involves creating themes at three levels. the first order represents the voice of the informants and the process of developing the first-order themes is similar to axial coding (strauss & corbin, 1998). first-order themes are refined into second-order themes in which the theoretical realm turns more precise and clear. finally, the third-order aggregate dimensions reflect the "distillation" of the second-order constructs (gioia, corley, & hamilton, 2013). based on the analysis of the responses, we developed the three orders of themes, and the same has been represented through figure 1, 2 & 3. australasian journal of information systems hassan & pandey 2021, vol 25, research on user involvement social networking sites and mindfulness 7 3.4 findings many of the antecedents and consequences of csns were in line with the findings of previous studies. however, the following demanded detailed inspection. 3.4.1 mindfulness as speed-breaker for csns it was found that people who were mindful of their actions were less prone to csns usage (see figure 1). as can be seen from the following response by a respondent to a question on why he thinks his usage of social media is regulated. i am always aware of how much time i spend on facebook or instagram. i keep track of the usage and when i don't find it meaningful, i log out. the above narrative provides an indication of the respondent's ability to control one's behaviour. while accessing the internet, the respondent remains aware of his continuous sns usage and analyses his situation. when he perceives that such a situation is becoming less meaningful, he puts on hold his actions to prevent further escalation in his perception. we received similar responses from some other participants as well. for example, one respondent expressed: i have disabled real-time notification from social networking sites. frequent notifications can cause distraction. so to control this, i log in to my account only during a particular time of the day and use it for not more than five or six minutes. while the first narrative focused on the "what and why" of the intensity of sns usage, the second narrative illustrates the "how" of regulating sns usage. in order to prevent any disturbance or distraction from sns, the respondent has disabled the notifications on his mobile phone. he is not only aware of the strategies to control his involvement with social media, but he is equally motivated and capable of bringing those strategies into practice. mindfulness is able to reduce the physiological consequences of stress that could have been caused due to the distraction caused by sns and reinstate it by a positive enhancing effect (jain et al., 2007). the inability to address distractions like the one mentioned in the narratives above, may disturb the maladaptive practices and cause stress, anxiety, and fatigue. in the case of respondents who were not mindful of their activities, we observed an excessive usage of sns. for example, a respondent shared: …i know it would sound like an addiction, but stay too engrossed in facebook and instagram. especially during my weekends, i spend hours scrolling news and posts uploaded on these platforms. the respondent in the above narrative is aware of the fact that the excessive use of sns is leading her towards an addiction, yet is unable to control this maladaptive practice. the respondent did not mention the reason for not being able to control her behaviour which suggests that such behaviours are occurring away from a conscious and self-regulating self. mindfulness increases self-esteem and reduces anxiety (apaolaza et al., 2019). both self-esteem and reduced anxiety are critical for self-regulation and focus (rhodewalt & tragakis, 2003). in the above narrative, low mindfulness seems to weaken self-esteem and increases anxiety to debilitate any action against uncontrolled automated behaviour (apaolaza et al., 2019). australasian journal of information systems hassan & pandey 2021, vol 25, research on user involvement social networking sites and mindfulness 8 figure 1: mindfulness as speed-breaker. australasian journal of information systems hassan & pandey 2021, vol 25, research on user involvement social networking sites and mindfulness 9 figure 2: prolonged sns usage leading to exhaustion. australasian journal of information systems hassan & pandey 2021, vol 25, research on user involvement social networking sites and mindfulness 10 the following narrative further illustrates the lack of mindfulness and automated behaviour. it happens at sudden times, i chat with people on fb, and such conversation often goes for hours. we human are not robots that suddenly you will stop using fb because you follow any daily routine for such usage. so i don't stick to any set duration of time for social media. after my office, i log in to fb and use it for two hours or so. 3.4.2 prolonged sns usage leads to exhaustion respondents expressed that their prolonged engagement with social networking sites results in some form of anxiety, stress, and tiredness (see figure 2). for example, one of the respondents who extensively use facebook and instagram explained: after using facebook and instagram for long, i feel strain in my eyes and headache. i often have a feeling of tiredness and stress. this has happened to me quite often. similarly, another respondent who is employed as a digital marketer and uses social networking sites for marketing related works said: ..when i have to keep using fb (facebook) or twitter every now and then for my work, i feel exhausted and drained. the two narratives mentioned above illustrate the psychological effect of excessive use of sns by the respondents. strain in eyes, headache, sleep deprivation, somatic issues, etc. are some of the major symptoms of exhaustion (wiegner, hange, björkelund & ahlborg, 2015). both the respondents had earlier mentioned about the time when they first started using facebook. since this time period was reasonably large (10 years and 4.5 years respectively), we may assume that continuous accumulation of sns-induced stress resulted in these health-related problems among the respondents. the effect of extroversion we also asked respondents questions regarding their likes and dislikes, gauged their preference for sociability and gregariousness in both the real and virtual world. it was found that people who considered themselves extroverts were more prone to csns usage and exhaustion (see figure 3). for example, one of the respondents reported: i like experiencing new things in my life. i love going out and planning trips with friends and relatives. i often use social networking sites to find like-minded people with whom i can plan such trips. in the above narrative, the respondent seems to enjoy the company of others. he actively engages in activities that include like-minded people for healthy interaction with them. the respondent is showing characteristics of extroverts who in general are known for spending more time with people and have a large social network. they presume themselves as more effective and find pleasure in engaging in various aspects of life (watson & clark, 1997). australasian journal of information systems hassan & pandey 2021, vol 25, research on user involvement social networking sites and mindfulness 11 figure 3: extroversion trait. australasian journal of information systems hassan & pandey 2021, vol 25, research on user involvement social networking sites and mindfulness 12 but often such enthusiasm to meet new people and socialize or find new avenues for adventure results in tiredness. when a further inquiry was made, the same respondent expressed how socializing taking place on sns causes his exhaustion. the respondent said: i like to interact with people, and i can do so easily using my social media accounts; obviously! social networking sites have helped me connect with my old friends and colleagues, but after spending too much time on fb and others, i feel tired and it appears as if i have done some manual work. the findings of study i identified extroversion, exhaustion, and mindfulness as three major constructs associated with sns exhaustion. while mindfulness regulated social media usage and thus exhaustion, extroversion seems to promote sns usage among the respondents. how these variables interact with each other needs further probing through a review of the literature and statistical modelling. therefore, study ii first examines the existing literature on these constructs before framing the hypothesis for analysis. findings of study i also identified some association between csns and exhaustion. the literature on exhaustion conceptualizes it as a "chronic state of physical and emotional depletion that results from excessive and continuous hassles of any form" (wright & cropanzano, 1998, p. 486). exhaustion can, therefore, result in the feeling of psychological and emotional draining. since csns usage is the prolonged and excessive use of social media, it is expected that such continuous engagement should equally result in physical and emotional depletion of our resources, and hence result in exhaustion. after completing study i, we felt a strong need to validate our findings to ensure that the emergent model developed through study i is generalizable and holds true for population other than the one taken as a sample in study i. therefore, we conducted study ii. as stated, this study extends the above arguments based on the review of relevant literature to propose and test the hypothesized model. the approach is also consistent with the recommendation of scholars who suggested a review of the literature and statistical modelling post an inductive study in a sequential mixed method research design (gioia, corley, & hamilton, 2013; tashakkori & teddlie, 2010). 4 study ii we start with a brief review of the literature on the constructs uncovered in study ii, followed by the hypothesis generation. the process of hypothesis development was done in two stages. first we adopted a comprehensive review process to examine prior literature on the constructs. this stage was followed by hypothesis building and data collection. 4.1 mindfulness our understanding of mindfulness has roots in the theravāda buddhist doctrine, and its contemporary meanings are the offshoots of the popular burmese buddhist reform movement that took place in the early twentieth century (sharf, 2015). buddhist literature has defined mindfulness as a "clear and single-minded awareness of what actually happens to us and in us at the successive moments of perception" (nyanaponikathera, 1972, p.32). however, the popular definition of the concept assumes it as a "state of being attentive to and aware of what is taking place in the present" (brown & ryan, 2003, p. 822). over the last few decades, the concept has drawn serious attention from scholars owing to its positive relationship with the wellbeing of individuals (brown & ryan, 2003; rybak, 2013; shonin et al., 2016). australasian journal of information systems hassan & pandey 2021, vol 25, research on user involvement social networking sites and mindfulness 13 organizational studies have emphasized the role of leaders' mindfulness on employee satisfaction, absenteeism, and in-role performance (reb et al., 2009). similarly, studies on training and development have highlighted its role in stress management (de vibe et al., 2013), improving cognitive abilities (chiesa, calati & serretti, 2011) and athletic performance (aherne, moran & lonsdale, 2011). thus, the role of mindfulness in improving our life is imperative. however, a large portion of the research on mindfulness in the past has concentrated on mindfulness and its role in mitigating issues in real life. but, how virtual social life is shaped by our awareness and ability to control behaviour, i.e., mindfulness? is something that remains ill-examined. the current study is an attempt to fill this research gap. 4.2 sns exhaustion sns exhaustion is defined as the "degree to which the users believe that their resources, such as time and effort are depleted by the usage of social networking platforms" (han, 2018, p. 123). engaging with virtual social network platforms, such as facebook, twitter, and others, involve several benefits, obligations, and challenges at the same time. for example, one aspect of sns usage is that the individuals are expected to dedicate time and energy to create profiles and share information with others. the purpose is to build a virtual identity of the users. however, the darker side of social networking through online platforms expect users to continuously engage and respond to their virtual social contacts in order to mark their presence and justify their worth (zhao, grasmuck & martin, 2008; han, 2018). frequent use of sns by the users creates a perception that any reduced commitment or reduced participation in sns would result in loss of social contacts in users' real lives (beyens, frison & eggermont, 2016). this may result in fear, which could cause stress and oblige the users to further engage with such platforms. repeated compulsive sns usage finally causes depletion of our physical and psychological resources, i.e., exhaustion (marder et al., 2016). some studies (bright, kleiser & grau, 2015; han, 2018) have suggested that such sns-induced exhaustion may also result from different features of a social networking site. these may include chat options, photo and video sharing applications, opinion and polling, and other features. privacy concerns and overloading of information are the major cause of sns exhaustion. the way sns usage would impact an individual and result in exhaustion may vary from one individual to another (lo, 2019; mount et al., 1999). undoubtedly, research on exhaustion has become relevant in the contemporary world. psychological and physiological strain in the form of exhaustion is resulting in anxiety and depression among individuals (bint-e-rashid & irfan, 2018). sns exhaustion is associated with the stress-induced by digital platforms. the relevance of sns exhaustion rests with the fact that most of the social interactions today take place over digital platforms, such as facebook, twitter, instagram, and others. 4.3 extroversion extroversion is one of the big-five personality traits. extroversion is characterized by distinct engagement with the external world (friedman & schustack, 2016). extroverted individuals tend to be more assertive, talkative, sociable, and active. they showcase high group visibility. this is unlike those who are low on extroversion and are more reserved, retiring, and cautious (roccas, sagiv, schwartz & knafo, 2002). hogan suggested that extroversion comprises two dimensions, namely, ambition and sociability. the assertive nature of extroverts facilitates goals achievement through demonstration of competences in accordance with our social standards (hogan, 1986). this sociability is facilitated in the virtual world by engaging with social network sites. australasian journal of information systems hassan & pandey 2021, vol 25, research on user involvement social networking sites and mindfulness 14 an extroversion personality trait is capable of shaping some important behaviours among individuals. for example, in organizational settings, it can influence the performance, engagement, and other positive outcomes associated with the workforce. similarly, in our community settings, the trait can significantly moderate the degree of our personal adjustment to ageing and other issues (costa jr, mccrae & norris, 1981). before we draw our assumptions about the positive aspects of extroversion, it is important to remember that the dispositional factors are not always desirable. their role in shaping our behaviour either positively or negatively depends on how they act upon us (john, 2018). to illustrate this, recent studies found that if manifested at a very high level, extroversion may result in maladaptive practices (mccord, joseph & grijalva, 2014). the same organizational setting in which low or moderate extroversion could result in high performance; self-aggrandizement, long-windedness, and egotistic nature associated with high extroversion may result in low performance and customer dissatisfaction (coker, samuel & widiger, 2002). similarly, studies on addiction indicated that extroversion might promote addiction towards a particular behaviour or activity, such as excessive mobile usage in some cases (cook, griffiths & pontes, 2018; cho, kim & park, 2017). extroversion becomes a relevant construct for our study because examining this trait in the context of the virtual world would help us understand how this trait behaves differently on the virtual social interactions compared to its influence on those taking place in our real world. 5 theory and hypothesis as we have discussed earlier, mindfulness is "our ability to bring complete attention to the experiences occurring in the present moment" (charoensukmongkol, 2016, p.12). the literature on mindfulness argues that open and creative attention to one's experience or environment helps to equip oneself better to let these experiences come and go without affecting ones' life (leroy et al., 2013; reb, narayanan & chaturvedi, 2012). several benefits can be attributed to mindfulness. for example, mindfulness can significantly address the problematic, i.e., maladaptive use of the internet (calvete, gamez-guadix & cortazar, 2017). similarly, healthcare studies (fraumeni-mcbride, 2019; spears et al., 2017) have applauded mindfulness for its critical role in addressing addiction-related treatments. given the benefits associated with mindfulness, the authors argue that mindfulness should significantly reduce compulsive sns usage among individuals. compulsive sns behaviour has become very common today. research suggests that such compulsive usages are resulting in acute mental disorders among 5–8% of the global population (shonin, van gordon & griffiths, 2013). excessive usage of sns through mobile phones or computers is believed to result in prolonged stress which may cause mental disorders (creswell, 2017; pantic, 2014). mindfulness can significantly mitigate these issues. in an experimental study, weinstein, brown & ryan (2009) found that mindfulness can help significantly in coping with stressful situations. respondents who reported high on mindfulness were more motivated to cope with stress compared to the other respondents. the underlying mechanism is that mindfulness increases our confidence to regulate stress instead of getting anxious. the role of mindfulness in managing late positive potential (lpp), a neural marker responsible for emotional regulation has also been recognized in some studies (brown, goodman & inzlicht, 2013; bean, 2018). these studies have supported the argument that high mindfulness is effective in regulating emotions, and thus imbibing individuals with the ability to control australasian journal of information systems hassan & pandey 2021, vol 25, research on user involvement social networking sites and mindfulness 15 or regulate their behaviour. since, social networking platforms, such as facebook, twitter, etc., are an integral part of social media, we may expect mindful persons to better regulate the intensity of their sns usage in order to deal with any negative or maladaptive consequences associated with such excessive use. theoretically, the relationship between mindfulness and sns usage can be explained using the conservation of resource (cor) theory (hobfoll & shirom, 2000). the theory proposes that individuals have the ability to selectively choose behaviours that can maximize their returns and minimize their losses. individuals seek to conserve their resources. in the case of sns, the users are expected to reduce the frequency of their sns usage when such behaviours are resulting in stressful conditions (zhang et al., 2011). mindful individuals are able to recognize sns usage behaviours as merely an activity that can be regulated or controlled. the confidence and effort to overcome such behaviour reduces anxiety associated with csns (apaolaza et al., 2019). at any moment, an individual may have multiple responses available for a stimulus. bishop et al. (2004) stated that mindfulness helps in conserving our personal resources by directing our attention to only those responses which can maximize benefits (instrumental learning) and reduce losses (i.e., stress). based on the above discussion, we would like to propose our first hypothesis, h1: mindfulness is negatively associated with compulsive sns usage extending our discussion further, excessive usage of sns results in negative consequences, such as sns exhaustion (luqman et al., 2020; masood et al., 2020). since mindfulness helps in vitality and self-regulation of personal resources, mindful individuals are aware of their self and the situation in which they act. this consciousness guides them to step back and decide from further engaging in the current activity in order to reduce the impact of stress; sns usage is a stress-inducing activity. further, mindfulness offers stress-mitigating mechanisms which are more reflective in nature as compared to the absent-mindedness, which encourages reactive exacerbating responses against a stimulus (reb, narayanan & ho, 2015). thus, it would be appropriate to say that mindfulness is even capable of reducing stress-related outcomes, such as exhaustion. from the cor theory perspective, mindfulness is an effective mechanism that can prevent the depletion of personal resources, such as values, energy, states, and conditions. in the real world, mindful individuals have a preference for problem-focused coping mechanisms over emotion-focused coping strategies to handle stress (charoensukmongkol, 2013). they are engaging in a more optimistic problem analysis and planning process instead of stagnant deliberation. any discouraging negative emotions and unhealthy efforts to escape the condition are discouraged by them (charoensukmongkol, 2013). we may expect a similar response from mindful individuals toward technology-induced exhaustion, i.e., sns exhaustion. therefore, we propose our second hypothesis which is, h2: mindfulness is negatively associated with sns exhaustion. summarizing our discussion above, we have pointed out how mindfulness can regulate our engagement with sns. we also highlighted the existence of prolonged stress, i.e., sns exhaustion resulting from compulsive sns usage. we now argue that the compulsive sns usage will mediate the relationship between mindfulness and sns exhaustion. our argument is theoretically grounded on the sso perspective and some empirical studies. schumer & buchwald (2012) noted that the frequency of sns usage is critical for work-related stress. it cautioned us regarding the observations that sns usage often results in a loss of our australasian journal of information systems hassan & pandey 2021, vol 25, research on user involvement social networking sites and mindfulness 16 interactions and social ties in the real world. individuals who frequently engage with sns lose interest in meeting and interacting with people in real life and they may even turn prone to issues, such as concentration issues due to frequent distractions (junco, 2012; charoensukmongkol, 2016). however, in the case of individuals high on mindfulness, it is relatively easier to evaluate and then control the actions through the cognitive emotion regulation strategies (karing & beelmann, 2019). thus, mindful individuals are able to even regulate the intensity of sns use and subsequent outcomes (e.g., exhaustion) associated with such usage. in both cases, the intensity of sns usage is decisive. high/low mindfulness reduce/increase compulsive sns usage, and thus reduce/increase sns exhaustion. the association between compulsive sns and exhaustion can be explained theoretically using the stressor-strain-outcome perspective (sso perspective). the sso perspective has been used in prior studies to examine the implications of technological features as environmental technology stimuli on the virtual experiences of users (cao et al., 2018; ayyagari, grover & purvis, 2011). the framework suggests that our ability to manage the demand for environmental stimuli determines how well we avoid or cope up with technology-induced stress and subsequent strain. notably, the stressor here refers to any environmental stimulus "that appears annoying, irksome, or disruptive to sns users" (cao et al., 2018, p.3). when the demand for such technological-environmental stimulus exceeds our ability, it results in a chain of psycho-physiological reactions that eventually cause exhaustion (jonsdottir et al., 2013; koeske & koeske, 1993). extending this discussion further, the second aspect (the first being mindfulness and csns) of the proposed mediating relationship, i.e., the relationship between mindfulness and emotional exhaustion has been established through some empirical studies (broderick, 2005; kiken & shook, 2011; kiken & shook, 2012). these studies have argued that mindfulness increases self-efficacy, optimism, and self-esteem that aid in reducing emotional exhaustion to a great extent. since sns exhaustion is one of the forms of emotional exhaustion caused due to prolonged usage of social network platforms, mindfulness should also significantly reduce sns exhaustion. therefore, we propose, h3: compulsive sns usage mediates the relationship between mindfulness and exhaustion the way mindfulness, compulsive sns usage, and sns exhaustion interact with each other vary significantly from one individual to another. research advocates forgiving dispositional factors due to the consideration in predicting the behaviour of individuals in one's day-to-day life (witt et al., 2002; giluk, 2009). individual differences (trait-based) affect the degree of motivation that promotes or impedes the use of personal resources (such as energy, vigour, enthusiasm) in order to overcome the psychological strains (penney et al., 2011). personality traits may determine our behaviours and actions through different approaches. for example, individuals with different dominant traits may have different choices of self-selecting highly stressful/less stressful activities (garden, 1989). furthermore, some of the personality traits may predispose individuals to experience stress more intensely compared to others. finally, traits may determine the coping mechanism and its effectiveness for different individuals (armon, shirom & melamed, 2012). surprisingly, studies on personality, mindfulness, and exhaustion are limited, and more empirical research is needed to investigate the relationship among these variables. among the big five personality traits, the relationship between extroversion and mindfulness is quite presumptuous. extroverts or individuals high on extroversion are socially gregarious; they are more open to interact and share their thoughts and views with others (giluk, 2009). australasian journal of information systems hassan & pandey 2021, vol 25, research on user involvement social networking sites and mindfulness 17 however, a major challenge associated with this kind of personality is their tendency to crave more excitement, adventure, and stimulation most part of their life (dollinger, 1993; costa jr & mccrae, 2008). as a result, extroverts tend to resist being both mindful and socially outgoing at the same time (winning & boag, 2015). we believe that a similar role of extroversion should be observed in the case of online engagements. hence, we may assume that extroversion may moderate the intensity of mindfulness on csns usage and exhaustion to an extent. for individuals high on extroversion, the effect of mindfulness would get suppressed as a result of the dominant nature of the personality trait. contradictory to this, where extroversion is low, mindfulness would dominate the behaviour of such individuals. based on these arguments, we propose our fourth hypothesis: h4: extroversion has a significant moderating effect on the relationship between mindfulness and compulsive sns usage. the effect of personality traits such as extroversion not only affects mindfulness but also its relationship with other constructs such as sns usage and sns exhaustion. as stated earlier, personality traits determine how individuals would avoid or cope with the stressors. prolonged usage of social network sites results in stress, and mindfulness helps in overcoming these stressors. here extroversion can moderate the role of mindfulness on sns exhaustion because extroverts are generally talkative and sociable (kiffin-petersen, jordan, & soutar, 2011). similar arguments can be put forth in the case of interactions happening in the virtual world. the relationship between csns, sns exhaustion, and personality traits, such as extroversion are expected to encourage users to engage with sns on a frequent and prolonged basis as these platforms offer a wide opportunity to interact and socialize. socializing and interactive nature are among the core characteristics of extroversion in the real world. it is not surprising if similar tendencies are observed in the case of the virtual world. the moderating role of extroversion in the relationship between csns and sns exhaustion can also be established based on some of the field studies on extroversion, which has observed that such traits promote prolonged social media usage (klobas et al., 2018; de cock et al., 2014). based on the above discussion, hypothesis 5 and 6 have been proposed: h5: extroversion has a significant moderating effect on the relationship between mindfulness and exhaustion. h6: extroversion has a significant moderating effect on the relationship between compulsive sns usage and exhaustion. based on the literature review and discussion above, we may summarize our understanding through a hypothesized model given in figure 4. australasian journal of information systems hassan & pandey 2021, vol 25, research on user involvement social networking sites and mindfulness 18 figure 4 hypothesized model 6 methodology data for study ii was collected using the survey method. for the purpose of collecting data, we followed the purposive sampling technique. the main objective of using purposive sampling is to ensure that our sample is logically assumed to be representative of the desired population we are trying to examine (lavrakas, 2008). the purposive technique is also considered suitable for mixed-method research designs, such as the one used in the current research (palinkas et al., 2015). finally, prior research on similar topics has used purposive sampling to collect responses for their study, and hence the technique became our first choice for the current study (longstreet & brooks, 2017; nalwa & anand, 2003). the desired population in our study were the active sns users of different age groups and gender. therefore, we purposely sent the online survey forms to only those respondents who were reportedly active sns users at that point in time. the questionnaires were sent to 350 sns users belonging to different age groups, educational, and professional backgrounds. out of the 350 respondents who were emailed, we received survey forms from 249 respondents. however, 13 of these 249 forms were either incomplete or redundant, and hence they were not included in the final sample. finally, a total of 236 valid responses were used for analysis. for the purpose of measuring the constructs, we used established scales which had previously been used in other empirical studies. the questionnaire prepared for the study was shared with ten respondents (other than those 236 in the main sample) and three subject-matter experts for validity. measures we measured the constructs using established scales (appendix 1). compulsive sns usage was measured using a scale developed by turel & osatuyi (2017). sample items included: "i spent time on social networking sites even though i had other deadlines." the cronbach's alpha for the scale was 0.65. response for mindfulness was measured by baer et al. 's (2006) trait mindfulness scale. the sample item includes the following: "it seems i am running on automatic without much awareness of what i am doing". the cronbach's alpha for the scale was 0.83. similarly, sns exhaustion was measured using a scale proposed by han (2018). the scale has been used in other similar studies too. the sample item of this scale includes "i feel exhausted mindfulness csns usage exhaustion extroversion australasian journal of information systems hassan & pandey 2021, vol 25, research on user involvement social networking sites and mindfulness 19 by using social networking sites." the cronbach's alpha for the scale was 0.73. finally, extroversion was measured by gosling et al. 's (2003) scale. the sample item for extroversion includes the following: "i see myself as extraverted, enthusiastic." the cronbach's alpha for the scale was 0.91. we also measured social desirability using crowne & marlowe (1960) 's social desirability scale. the sample item for social desirability include: "i am always polite even to people who are offensive." the cronbach's alpha for this scale was 0.5. all the responses for the questionnaire were recorded using a 7-point likert scale ranging from 1, which stood for strongly disagree to 7, which stood for strongly agree. we followed the two-step approach to analyse the data. initially, we used confirmatory factor analysis to establish convergent and discriminant validity using sem followed by the second step using process macro (model 59) to test the overall model. the bootstrap samples were 2000 and the confidence interval was 90. bootstrapping is a statistical procedure that resamples a single dataset to create many simulated samples in order to provide more robust results. exhaustion was the dependent variable; mindfulness was the independent variable, and compulsive usage was the mediator. extroversion was the moderator. age, gender, education, work experience, and social desirability were controlled. we conducted herman's single factor test and found that a single factor accounted for less than 50 percent variance. in addition, marker variable analysis (williams, hartman & cavazotte, 2010) was also done to identify common method bias. we found that the inclusion of the marker variable did not substantially change the item factor loadings indicating that the common method variance is not substantial. 7 findings table 3 shows the mean, standard deviation, and bivariate correlations. mean sd exhaustion mindfulness compulsive use extraversion age exhaustion 3.68 1.40 0.76 mindfulness 4.73 1.36 -.305*** 0.93 compulsive use 3.89 1.19 -.508*** .327*** 0.83 extraversion 4.13 1.45 -.168** .118* .177** 0.6 age 26.76 8.30 .118* -.027 -.164** .038 note n=236; cronbach's alpha on the diagonals ***correlation is significant at the 0.01 level (2-tailed); **correlation is significant at the 0.05 level (2-tailed); *correlation is significant at the 0.1 level (2-tailed). table-3: descriptive statistics and bivariate correlations table 4 shows the following, convergent validity: the cronbach's alpha and composite reliability (cr) of all variables are above 0.6, thus indicating convergent validity (taber, 2008). the average variance extracted values of all variables except compulsive use is above 0.5. ave value; below 0.5 is considered acceptable if factor loading and composite reliability is well established (fornell & larcker, 1981). recent studies such as lam (2012) have considered ave value below 0.5 for the purpose of their analysis. australasian journal of information systems hassan & pandey 2021, vol 25, research on user involvement social networking sites and mindfulness 20 discriminant validity: msv < ave and the square root of ave greater than inter-construct correlations, thus indicating discriminant validity. cr ave msv maxr(h) exhaustion mindfulness compulsive extraversion exhaustion 0.767 0.529 0.130 0.796 0.727 mindfulness 0.929 0.686 0.342 0.932 -0.361 0.828 compulsive sns usage 0.833 0.425 0.342 0.870 0.346 -0.585 0.652 extraversion 0.873 0.821 0.048 1.568 -0.130 0.172 -0.219 0.906 note: n=236 square root of ave on the diagonals table 4: validity table table 5 shows the output of the hierarchical regression analysis. mindfulness was negatively and significantly associated with compulsive use (b=-0.426, p<0.001), thus supporting hypothesis 1. mindfulness was negatively and significantly associated with exhaustion (b=0.23, p<0.001), thus supporting hypothesis 2. the conditional indirect effect of mindfulness on exhaustion via compulsive usage is negative and significant at medium (extroversion=mean value; beta=-0.1, boot se 0.04, 90%ci [-0.176,-0.038]) and high (extroversion=plus 1 sd above the mean; beta=-0.16, boot se 0.06, 90%ci [-0.279,-0.070]) values of extroversion, thus supporting hypothesis 3. variables csns usage exhaustion csns usage 0.24** mindfulness -0.42*** -0.23*** extraversion 0.09* 0.06 mindfulness x extraversion 0.06* 0.11** csns usagex extraversion -0.06 control variables age -0.04** -0.01 edu -0.05 -0.01 work experience 0.01 0.01 social desirability 0.05 -0.02 f 14.15*** 5.75*** r2 0.3 0.19 note: n=236; †p < 0.1; *p < 0.05; **p < 0.01; ***p < 0.001. table 5: results of moderated mediation model. the interaction term of mindfulness and extroversion was significantly associated with compulsive use (b=-0.057, p<0.1), thus supporting hypothesis 4. the interaction was examined using a graph presented in figure 5. the figure shows that the negative relationship between mindfulness and compulsive use is stronger for people scoring high on extroversion. australasian journal of information systems hassan & pandey 2021, vol 25, research on user involvement social networking sites and mindfulness 21 figure 5: graphical representation of the interaction between mindfulness and csns the interaction term of mindfulness and extroversion was significantly associated with exhaustion (b=-0.114, p<0.05), thus supporting hypothesis 5. the interaction was examined using a graph presented in figure 6. the figure shows that the negative relationship between mindfulness and exhaustion is stronger for people scoring high on extroversion. the interaction term of compulsive sns use and extroversion was not significantly associated with exhaustion, thus hypothesis 6 was not supported. figure 6: graphical representation of the interaction between mindfulness and exhaustion. australasian journal of information systems hassan & pandey 2021, vol 25, research on user involvement social networking sites and mindfulness 22 8 discussion over the last few years, compulsive sns usage has drawn the attention of scholars and practitioners due to an increased number of sns subscribers on the one hand, and the serious physical and mental health issues associated with such usage on the other hand. prior studies have offered some indicators to distinguish csns usage from other forms of sns usage. for example, larose, lin & eastin (2003) identified it with deficient self-regulation. similarly, caplan (2010) explained csns usage as an unsuccessful attempt to cut down and control sns usage. however, in all these studies, the factors that facilitate or impede csns usage have been largely ignored. examining these variables is important because such findings might help scholars and practitioners in addressing various psychological and physiological issues concerning csns usage. the current study is an attempt to address the gaps in the existing literature on sns. findings of study i and ii can be synthesized here. our findings suggest that the excessive use of social media in the form of csns usage is interlinked to dispositional factors, such as personality traits and mindfulness. our study indicates that mindful persons are less impacted by sns usage. one reason for this could be the fact that such people are more capable of effectively regulating their behaviour in order to avoid depletion of their personal resources (e.g., energy and enthusiasm). observing and acting with awareness forms two decisive aspects of mindfulness. people reporting high on mindfulness are able to reflect on their behaviour based on the observations, and it subsequently encourages them to control such behaviours that could otherwise have resulted in adverse consequences (turel & osatuyi, 2017). in the case of sns usage, mindful persons are aware of the negative consequences (e.g., exhaustion) resulting from excessive sns use. they are able to effectively balance the social pressure and self-desire that promotes such high usage of sns. the literature on mindfulness suggests that mindful persons are capable of balancing pressures of various kinds. they assume external forces, such as peer pressure, as a transient and are equipped to handle such pressure through an obligatory and less-automatic approach towards social media (turel & osatuyi, 2017; sanger & dorjee, 2016; sriwilai & charoensukmongkol, 2016). therefore, while engaging with social media platforms, they keep a check on the instrumentality involved in such engagements, and they clearly identify the tipping point at which they have to take a break. our study also extends the discussion on mindfulness by taking into account the role of moderators-extroversion in our engagement with digital platforms. it was observed that mindfulness was less effective in addressing csns usage among users who reported high on extroversion traits. the above findings validate the observations by scholars extroversion is characterized by high gregariousness and interactive tendencies. extroverts prefer engaging in community activities, socializing, and networking. social networking sites appear to be an effective platform for expanding ones' personal network and relationship. therefore, extroverts are seen increasingly engaged with sns as it serves this purpose in the digital world (kim, hsu & de zuniga, 2013). compared to other personality types, prolonged sns usage is high among extroverts. such prolonged usage leaves open doors for obsession or addiction. our study indicates that mindfulness works only when extroversion is low or moderate among sns users. in the case of users high on extroversion, mindfulness becomes less imperative in controlling the excessive usage of social media as the influence of extroversion trait supersedes the effect of mindfulness. on account of this, the user starts experiencing stress and subsequently, sns-related exhaustion (nawaz et al., 2018). australasian journal of information systems hassan & pandey 2021, vol 25, research on user involvement social networking sites and mindfulness 23 we also examined the moderating role of extroversion in the relationship between csns and exhaustion. exhaustion is the loss of our physiological and mental strength. such losses can significantly impact our engagement and overall performance (dhir et al., 2018). the current study suggests that personality traits are detrimental in the way csns leads to exhaustion among social media users (liu & ma, 2018). literature suggests that sns users are prone to techno-stressors which are "technology-induced stimuli, events, or demands" (cao, masood, luqman, & ali, 2018, p.7). when users frequently use social media, the stressors cause strain, and subsequently, the users feel exhausted (cao et al., 2018; charoensukmongkol, moqbel & gutierrez-wirsching, 2017; bright, kleiser & grau, 2015). empirically findings of study ii show that in the case of users reporting high on extroversion, exhaustion due to csns usage is higher compared to those who reported low on extroversion. in other words, the gregarious and interactive nature of extroverts further enhances the strength of the csns-exhaustion relationship. the above findings have some important theoretical and managerial implications which have been discussed in the subsequent sections. 9 contribution and implications 9.1 theoretical implications research on sns has primarily focused on the positive role of social networking sites. for example, studies have highlighted its role in connecting non-profit organizations with donors and fundraisers across the globe (waters, burnett, lamm & lucas, 2009). but, the darker side of sns has remained largely ignored (cao et al., 2018). our study contributes towards this gap in the literature by highlighting the aetiology of csns, which has several important negative implications, exhaustion being one of them. a major theoretical contribution of this study is extending the current scholarship on conservation of resource theory. cor insinuates that in situations when we face a threat of losing our personal resources and opportunities to gain new resources are missing, then we try to maintain or preserve our existing current resources (hobfoll & shirom, 2000). cor theory provides an effective theoretical grounding to understand our findings. it assumes that the individual would discontinue indulging in such behaviours, which could result in the depletion of our personal resources (zhang et al., 2019). it is important to understand that the resources here refer to the values, energies, objects, and state of mind (hobfoll, 1988). in the current study, the behaviour is synonymous with compulsive sns usage, whereas personal resource is vigour, and the loss of it is resulting in sns exhaustion. our study extends the discussion on cor theory by accounting for dispositional factors that were originally not included in the cor framework. based on our findings of the two studies, we argue that the conservation of resources is not just governed by the rate at which resources are depleted. instead, the conservation process is a very complex phenomenon that involves both the rate at which resources are lost and the personality-related dispositional factors, and it determines how the individual would respond to such challenges. we further argue that the openness, talkative, and interactive nature of the individuals weakens our control such that the depletion of personal resources (such as sns usage) are quicker among extroverts as compared to individuals with any other personality traits. the current study also extends the discussion on the sso perspective. while the sso perspective explains how stress and stressors result in negative outcomes, such as exhaustion; the framework does not elaborate on the antecedents and moderators of such psychophysiological reactions. our findings provide mindfulness and csns usage as the two australasian journal of information systems hassan & pandey 2021, vol 25, research on user involvement social networking sites and mindfulness 24 major antecedents of exhaustion. further, it also highlights the greater role of personality traits in moderating the chain reaction, which has been discussed in the sso framework. 9.2 managerial implications our study also offers some important managerial implications. our study indicates the necessity for designing and developing a more comprehensive training program for compulsive sns usage. social media are effective in disseminating information and entertaining users. however, its excessive usage may equally result in negative outcomes, such as dissatisfaction, isolation from the real world, and mental health disorders, among others (apaolaza, hartmann, d'souza & gilsanz, 2019). we believe that if customized training for mindfulness is offered, taking into consideration the dispositional factors, such as the personality-related individual differences, then such training can effectively address the negative outcomes of sns usage to a great extent. recently, an article published in the washington post mentioned the term "digital detox," which is basically a strategy to reduce the amount of time we spend on social media platforms (griffiths & kuss, 2018). digital detox could be both voluntary as well as involuntary depending upon who regulates our digital engagement at home or our workplace. the article recommends strict rules to be placed by the employers to control csns behaviour among the employees and a similar approach for academic institutions to be followed. the findings of our study hold an important place for firms working in the field of digital marketing. a large number of companies are focusing on the promotion of their brands through social networking sites. this invariably, means employees would be spending a large part of their daily office time on these platforms to contribute towards the business. such employees are susceptible to poor wellbeing, which could invariably impact their performance. such organizations would require holding special training to improve employees' mindfulness in order to cope up with such issues. further, such organizations are also expected to provide flexible work schedules to reduce their employees' discomfort with prolonged internet usage. the current study also holds important practical relevance for adolescents and youths who are increasingly being attracted to social media usage. as the japanese psychologist, tamaki saitō said, the world has been predominantly occupied by hikikomori, a generation that has traded their social lives for internet, media consumption, and video-games (kiberd, 2019). our study suggests that we may restrict social media from consuming our healthy life through a planned intervention in the form of appropriate training to control our actions (turel, mouttapa & donato, 2015). 10 limitations and future research directions the current research has some limitations which can be addressed in future studies. based on the findings of study i, we identified that exhaustion is as an important outcome of compulsive sns usage. however, our study had limited its scope of research to the context of india. similar studies with a cross-cultural sample can explore and examine other relevant outcomes. second, the limitation of the current study is that it does not focus on the differences in csns usage by individuals based on factors such as the age, gender, type of the sector in which the respondent is employed, and prior education, among others. another limitation of the current study is that it focuses only on one dispositional trait, i.e., extroversion. possibility of findings other individual-level factors in future studies cannot be denied. finally, mindfulness as a concept has been distinguished from similar constructs such australasian journal of information systems hassan & pandey 2021, vol 25, research on user involvement social networking sites and mindfulness 25 as emotional intelligence, wakefulness, and various forms of self-awareness. for example, emotional intelligence (ei) refers to the ability to monitor emotions (of self and others), and use it as a guiding force to direct ones' action. but, ei represents only one stage of mindfulness, i.e., when mindfulness is high. however, in a state of less mindfulness, emotions occur outside the realm of our knowledge. awareness of our current behaviour is acknowledged much later (brown & ryan, 2003). similarly, wakefulness is another construct that is worth discussing here. wakefulness is different from mindfulness because the former concerns cognitive tasks, which are operationalized based on some external environment, and there are possibilities for multiple perspectives for the same operationalized tasks (bodner & langer, 2001; brown & ryan, 2003). one example of wakefulness is the functional diversity within business organizations. langer & moldoveanu (2000) noted that even though different employees are assigned functionally diverse roles in a setup, the differences do not necessarily imply that the employees of one function are inferior (or superior) to employees of a different function. instead, it should be observed as the same stimuli occurring differently and providing us with multiple perspectives on the same task (snow, 1997). contrary to this, mindfulness is about an open, single, and undivided observation of anything (e.g., cognitive task) happening either internally or externally, or both. it does emphasize on any specific approach or perspective towards a stimulus (langer & moldoveanu, 2000). future studies can examine these similar constructs along with mindfulness in order to understand their impact on compulsive sns use. there is a growing body of research that shows the linkages between management, spirituality, and religion (pandey & singh, 2019; pandey & varkkey, 2020). religion and its philosophical routing cannot be ignored in the managerial discourse of present times (hassan & pandey. 2019, pandey & gupta, 2018). the study shows the efficacy of eastern religious practices like mindfulness in providing solutions to modern-age problems. psychological capital and contracts provide many such mediating mechanism that can be studied in future studies (jha, pandey & varkkey, 2018; pandey, gupta & hassan, 2020). future studies can look at other religious practices like yoga, prayer and chanting in dealing with modern-day challenges. references aherne, c., moran, a. p., & lonsdale, c. (2011). the effect of mindfulness training on athletes' flow: an initial investigation. the sport psychologist, 25(2), 177–189. al-yafi, k., el-masri, m., & tsai, r. (2018). the effects of using social network sites on academic performance: the case of qatar. journal of enterprise information management, 33(3), 446-462. andreassen, c. s. (2015). online social network site addiction: a comprehensive review. current addiction reports, 2(2), 175-184. andreassen, c. s., billieux, j., griffiths, m. d., kuss, d. j., demetrovics, z., mazzoni, e., & pallesen, s. (2016). the relationship between addictive use of social media and video games and symptoms of psychiatric disorders: a large-scale cross-sectional study. psychology of addictive behaviors, 30(2), 252. apaolaza, v., hartmann, p., d'souza, c., & gilsanz, a. (2019). mindfulness, compulsive mobile social media use, and derived stress: the mediating roles of self-esteem and social anxiety. cyberpsychology, behavior, and social networking, 22(6), 388–396. australasian journal of information systems hassan & pandey 2021, vol 25, research on user involvement social networking sites and mindfulness 26 arora, r., & stoner, c. (2009). a mixed method approach to understanding brand personality. journal of product and brand management, 18(4), 272–283. baer, r. a., smith, g. t., hopkins, j., krietemeyer, j., & toney, l. (2006). using self-report assessment methods to explore facets of mindfulness. assessment, 13(1), 27–45. bean, v. (2018). trait mindfulness, facebook use, life satisfaction, and well-being (doctoral dissertation, university of kansas). berman, e. a. (2017). an exploratory sequential mixed methods approach to understanding researchers' data management practices at uvm: findings from the qualitative phase. journal of escience librarianship, 6(1), 5. beyens, i., frison, e., & eggermont, s. (2016). "i don't want to miss a thing": adolescents' fear of missing out and its relationship to adolescents' social needs, facebook use, and facebook related stress. computers in human behavior, 64(1), 1–8. bright, l. f., kleiser, s. b., & grau, s. l. (2015). too much facebook? an exploratory examination of social media fatigue. computers in human behavior, 44, 148-155. brown, k. w., goodman, r. j., & inzlicht, m. (2013). dispositional mindfulness and the attenuation of neural responses to emotional stimuli. social cognitive and affective neuroscience, 8(1), 93-99. calvete, e., gámez-guadix, m., & cortazar, n. (2017). mindfulness facets and problematic internet use: a six-month longitudinal study. addictive behaviors, 72(1), 57–63. cameron, r. (2009). a sequential mixed model research design: design, analytical and display issues. international journal of multiple research approaches, 3(2), 140–152. cao, x., masood, a., luqman, a., & ali, a. (2018). excessive use of mobile social networking sites and poor academic performance: antecedents and consequences from stressorstrain-outcome perspective. computers in human behavior, 85(1), 163–174. charoensukmongkol, p. (2013). the contributions of mindfulness meditation on burnout, coping strategy, and job satisfaction: evidence from thailand. journal of management & organization, 19(5), 544–558. charoensukmongkol, p. (2016). mindful facebooking: the moderating role of mindfulness on the relationship between social media use intensity at work and burnout. journal of health psychology, 21(9), 1966-1980. charoensukmongkol, p., moqbel, m., & gutierrez-wirsching, s. (2017). social media sites use intensity and job burnout among the us and thai employees. international journal of cyber behavior, psychology and learning (ijcbpl), 7(1), 34-51. chiesa, a., calati, r., & serretti, a. (2011). does mindfulness training improve cognitive abilities? a systematic review of neuropsychological findings. clinical psychology review, 31(3), 449–464. cho, h., kim, d. j., & park, j. w. (2017). stress and adult smartphone addiction: mediation by self‐control, neuroticism, and extraversion. stress and health, 33(5), 624–630. coker, l. a., samuel, d. b., & widiger, t. a. (2002). maladaptive personality functioning within the big five and the five-factor model. journal of personality disorders, 16(5), 385– 401. australasian journal of information systems hassan & pandey 2021, vol 25, research on user involvement social networking sites and mindfulness 27 cook, r. h., griffiths, m. d., & pontes, h. m. (2020). personality factors in exercise addiction: a pilot study exploring the role of narcissism, extraversion, and agreeableness. international journal of mental health and addiction, 18(1), 89–102. corbin, j., & strauss, a. (2014). basics of qualitative research: techniques and procedures for developing grounded theory. sage publications. costa jr, p. t., mccrae, r. r., & norris, a. h. (1981). personal adjustment to aging: longitudinal prediction from neuroticism and extraversion. journal of gerontology, 36(1), 78–85. creswell, j. d. (2017). mindfulness interventions. annual review of psychology, 68, 491-516. crowne, d. p., & marlowe, d. (1960). a new scale of social desirability independent of psychopathology. journal of consulting psychology, 24(4), 349. de cock, r., vangeel, j., klein, a., minotte, p., rosas, o., & meerkerk, g.-j. (2014). compulsive use of social networking sites in belgium: prevalence, profile, and the role of attitude toward work and school. cyberpsychology, behavior, and social networking, 17(3), 166–171. de vibe, m., solhaug, i., tyssen, r., friborg, o., rosenvinge, j. h., sørlie, t., & bjørndal, a. (2013). mindfulness training for stress management: a randomized controlled study of medical and psychology students. bmc medical education, 13(1), 107. dhir, a., yossatorn, y., kaur, p., & chen, s. (2018). online social media fatigue and psychological wellbeing—a study of compulsive use, fear of missing out, fatigue, anxiety and depression. international journal of information management, 40(1), 141–152. ding, l. (2019). applying gamifications to asynchronous online discussions: a mixed methods study. computers in human behavior, 91(1), 1–11. divine, a., watson, p. m., baker, s., & hall, c. r. (2019). facebook, relatedness and exercise motivation in university students: a mixed methods investigation. computers in human behavior, 91(1), 138–150. draucker, c. b., martsolf, d. s., ross, r., & rusk, t. b. (2007). theoretical sampling and category development in grounded theory. qualitative health research, 17(8), 1137–1148. fox, j., & moreland, j. j. (2015). the dark side of social networking sites: an exploration of the relational and psychological stressors associated with facebook use and affordances. computers in human behavior, 45, 168-176. fraumeni-mcbride, j. (2019). addiction and mindfulness; pornography addiction and mindfulness-based therapy act. sexual addiction & compulsivity, 26(1–2), 42–53. friedman, h. s., & schustack, m. w. (2016). personality: classic theories and modern research. pearson. new york.us gioia, d. a., corley, k. g., & hamilton, a. l. (2013). seeking qualitative rigor in inductive research: notes on the gioia methodology. organizational research methods, 16(1), 15–31. gosling, s. d., rentfrow, p. j., & swann jr, w. b. (2003). a very brief measure of the big-five personality domains. journal of research in personality, 37(6), 504–528. griffith, m., & kuss, d. (2018, april 25). 6 questions help reveal if you're addicted to social media. washington post. retrieved from https://www.washingtonpost.com/news/theworldpost/wp/2018/04/25/social-mediaaddiction/. australasian journal of information systems hassan & pandey 2021, vol 25, research on user involvement social networking sites and mindfulness 28 gupta, m., pandey, j., gaur, j., & vohra, n. (2019). research on role of technology in workforce management. australasian journal of information systems, 23, 1–4. han, b. (2018). social media burnout: definition, measurement instrument, and why we care. journal of computer information systems, 58(2), 122–130. hancock, a.-m. (2007). intersectionality as a normative and empirical paradigm. politics & gender, 3(2), 248. hassan, y., & pandey, j. (2019). examining the engagement of young consumers for religiously sanctioned food: the case of halal food in india. young consumers, 21(2), 211– 232. hobfoll, s. e. (1988). the ecology of stress. washington. dc: hemisphere. hobfoll, s. e., & shirom, a. (2000). conservation of resources theory. appears in handbook of organizational behavior, rt golembiewski (ed.), marcel dekker, new york, 57–80. hogan, r. (2007). hogan personality inventory. retrieved from https://www.crownedgrace.com/wp-content/uploads/2016/04/hogan-personalityinventory.pdf inceoglu, i., & warr, p. (2011). personality and job engagement. journal of personnel psychology, 10(4), 177-181. ivankova, n. v, creswell, j. w., & stick, s. l. (2006). using mixed-methods sequential explanatory design: from theory to practice. field methods, 18(1), 3–20. jain, s., shapiro, s. l., swanick, s., roesch, s. c., mills, p. j., bell, i., & schwartz, g. e. r. (2007). a randomized controlled trial of mindfulness meditation versus relaxation training: effects on distress, positive states of mind, rumination, and distraction. annals of behavioral medicine, 33(1), 11–21. jha, j., pandey, j., & varkkey, b. (2019). examining the role of perceived investment in employees’ development on work-engagement of liquid knowledge workers: moderating effects of psychological contract. journal of global operations and strategic sourcing, 12(2), 225–245. johns, g. (2018). advances in the treatment of context in organizational research. annual review of organizational psychology and organizational behavior, 5(1), 21–46. karing, c., & beelmann, a. (2019). cognitive emotional regulation strategies: potential mediators in the relationship between mindfulness, emotional exhaustion, and satisfaction?. mindfulness, 10(3), 459-468. kiberd, r. (2018, august 18). no ti. the guardian. retrieved from https://www.theguardian.com/commentisfree/2019/mar/19/social-media-facebookaddiction-not-natural-normal-disease. kim, j., larose, r., & peng, w. (2009). loneliness as the cause and the effect of problematic internet use: the relationship between internet use and psychological well-being. cyberpsychology & behavior, 12(4), 451-455. kim, y., hsu, s.-h., & de zúñiga, h. g. (2013). influence of social media use on discussion network heterogeneity and civic engagement: the moderating role of personality traits. journal of communication, 63(3), 498–516. australasian journal of information systems hassan & pandey 2021, vol 25, research on user involvement social networking sites and mindfulness 29 klobas, j. e., mcgill, t. j., moghavvemi, s., & paramanathan, t. (2018). compulsive youtube usage: a comparison of use motivation and personality effects. computers in human behavior, 87(1), 129–139. kumar, r., sachan, a., & mukherjee, a. (2017). qualitative approach to determine user experience of e-government services. computers in human behavior, 71(1), 299–306. lai, h. m., hsieh, p. j., & zhang, r. c. (2019). understanding adolescent students' use of facebook and their subjective wellbeing: a gender-based comparison. behaviour & information technology, 38(5), 533-548. lam, l. w. (2012). impact of competitiveness on salespeople's commitment and performance. journal of business research, 65(9), 1328–1334. langley, a. (1999). strategies for theorizing from process data. academy of management review, 24(4), 691–710. larose, r., lin, c. a., & eastin, m. s. (2003). unregulated internet usage: addiction, habit, or deficient self-regulation? media psychology, 5(3), 225–253. lavrakas, p. j. (2008). encyclopedia of survey research methods. sage publications. lee, k.-t., noh, m.-j., & koo, d.-m. (2013). lonely people are no longer lonely on social networking sites: the mediating role of self-disclosure and social support. cyberpsychology, behavior, and social networking, 16(6), 413–418. lee, y.-k., chang, c.-t., lin, y., & cheng, z.-h. (2014). the dark side of smartphone usage: psychological traits, compulsive behavior and technostress. computers in human behavior, 31(1), 373–383. li, l., worch, e., zhou, y., & aguiton, r. (2015). how and why digital generation teachers use technology in the classroom: an explanatory sequential mixed methods study. international journal for the scholarship of teaching and learning, 9(2), 2. liao, f.-y., yang, l.-q., wang, m., drown, d., & shi, j. (2013). team–member exchange and work engagement: does personality make a difference? journal of business and psychology, 28(1), 63–77. liu, c., & ma, j. (2018). social media addiction and burnout: the mediating roles of envy and social media use anxiety. current psychology, 1-9. longstreet, p., & brooks, s. (2017). life satisfaction: a key to managing internet & social media addiction. technology in society, 50, 73–77. luqman, a., masood, a., weng, q., ali, a., & rasheed, m. i. (2020). linking excessive sns use, technological friction, strain, and discontinuance: the moderating role of guilt. information systems management, 37(2), 94–112. macphail, c., khoza, n., abler, l., & ranganathan, m. (2016). process guidelines for establishing intercoder reliability in qualitative studies. qualitative research, 16(2), 198– 212. mao, j. (2014). social media for learning: a mixed methods study on high school students' technology affordances and perspectives. computers in human behavior, 33, 213–223. masood, a., feng, y., luqman, a., & ali, a. (2020). adverse consequences of excessive social networking site use on academic performance: explaining underlying mechanism from stress perspective. computers in human behavior, 106476. australasian journal of information systems hassan & pandey 2021, vol 25, research on user involvement social networking sites and mindfulness 30 mccord, m. a., joseph, d. l., & grijalva, e. (2014). blinded by the light: the dark side of traditionally desirable personality traits. industrial and organizational psychology, 7(1), 130–137. mcintyre, e., wiener, k. k., & saliba, a. j. (2015). compulsive internet use and relations between social connectedness, and introversion. computers in human behavior, 48, 569574. meerkerk, g. j., van den eijnden, r. j., franken, i. h. a., & garretsen, h. f. l. (2010). is compulsive internet use related to sensitivity to reward and punishment, and impulsivity?. computers in human behavior, 26(4), 729-735. nalwa, k., & anand, a. p. (2003). internet addiction in students: a cause of concern. cyberpsychology & behavior, 6(6), 653–656. nawaz, m. a., shah, z., nawaz, a., asmi, f., hassan, z., & raza, j. (2018). overload and exhaustion: classifying sns discontinuance intentions. cogent psychology, 5(1), 1515584. ong, e. y. l., ang, r. p., ho, j. c. m., lim, j. c. y., goh, d. h., lee, c. s., & chua, a. y. k. (2011). narcissism, extraversion and adolescents' self-presentation on facebook. personality and individual differences, 50(2), 180–185. palinkas, l. a., horwitz, s. m., green, c. a., wisdom, j. p., duan, n., & hoagwood, k. (2015). purposeful sampling for qualitative data collection and analysis in mixed method implementation research. administration and policy in mental health and mental health services research, 42(5), 533–544. pandey, j., & gupta, m. (2019). religion in the lives of hindu widows: narratives from vrindavan, india. psychology of religion and spirituality, 11(2), 91–100. pandey, j., gupta, m., & hassan, y. (2020). intrapreneurship to engage employees: role of psychological capital. management decision. https://doi.org/10.1108/md-06-2019-0825 pandey, j., & varkkey, b. (2017). impact of religion-based caste system on the dynamics of indian trade unions: evidence from two state-owned organizations in north india. business & society, 59(5), 995–1034. pandey, j., & singh, m. (2019). positive religious coping as a mechanism for enhancing job satisfaction and reducing work-family conflict: a moderated mediation analysis. journal of management, spirituality & religion, 16(3), 314–339. pandey, j., & sohani, s. s. (2020). engaging stakeholders of crowdsourced work: case of cab aggregators of india. journal of electronic commerce in organizations., 18(3), 38–52. pantic, i. (2014). online social networking and mental health. cyberpsychology, behavior, and social networking, 17(10), 652–657. park, h. i., jacob, a. c., wagner, s. h., & baiden, m. (2014). job control and burnout: a meta‐ analytic test of the conservation of resources model. applied psychology, 63(4), 607–642. quan-haase, a. (2007). university students' local and distant social ties: using and integrating modes of communication on campus. information, communication & society, 10(5), 671– 693. reb, j., narayanan, j., & chaturvedi, s. (2014). leading mindfully: two studies on the influence of supervisor trait mindfulness on employee wellbeing and performance. mindfulness, 5(1), 36–45. australasian journal of information systems hassan & pandey 2021, vol 25, research on user involvement social networking sites and mindfulness 31 rhodewalt, f., & tragakis, m. w. (2003). self-esteem and self-regulation: toward optimal studies of self-esteem. psychological inquiry, 14(1), 66–70. roccas, s., sagiv, l., schwartz, s. h., & knafo, a. (2002). the big five personality factors and personal values. personality and social psychology bulletin, 28(6), 789–801. ryan, t., chester, a., reece, j., & xenos, s. (2014). the uses and abuses of facebook: a review of facebook addiction. journal of behavioral addictions, 3(3), 133–148. rybak, c. (2013). nurturing positive mental health: mindfulness for wellbeing in counseling. international journal for the advancement of counselling, 35(2), 110-119. salanova, m., llorens, s., & cifre, e. (2013). the dark side of technologies: technostress among users of information and communication technologies. international journal of psychology, 48(3), 422–436. sharif, s. p., & khanekharab, j. (2017). identity confusion and materialism mediate the relationship between excessive social network site usage and online compulsive buying. cyberpsychology, behavior, and social networking, 20(8), 494–500. shonin, e., van gordon, w., & griffiths, m. d. (2016). mindfulness and wellbeing: towards a unified operational approach. in i. ivtzan & t. lomas (eds.), mindfulness in positive psychology: the science of meditation and wellbeing (p. 280–292). oxford: routledge. singletary, s. (n.d.). is social media destroying humankind? tedx sydney. retrieved from https://tedxsydney.com/idea/is-social-media-destroying-humankind/ slater, d. (2002). social relationships and identity online and offline. handbook of new media: social shaping and consequences of icts, 533–546. spears, c. a., hedeker, d., li, l., wu, c., anderson, n. k., houchins, s. c., cinciripini, p. m. (2017). mechanisms underlying mindfulness-based addiction treatment versus cognitive behavioral therapy and usual care for smoking cessation. journal of consulting and clinical psychology, 85(11), 1029. srivastava, s. c., & chandra, s. (2018). social presence in virtual world collaboration: an uncertainty reduction perspective using a mixed methods approach. mis quarterly, 42(3), 779–804. sriwilai, k., & charoensukmongkol, p. (2016). face it, don't facebook it: impacts of social media addiction on mindfulness, coping strategies and the consequence on emotional exhaustion. stress and health, 32(4), 427-434. taber, k. s. (2018). the use of cronbach's alpha when developing and reporting research instruments in science education. research in science education, 48(6), 1273–1296. tashakkori, a., & teddlie, c. (2010). sage handbook of mixed methods in social & behavioral research. sage. throuvala, m. a., griffiths, m. d., rennoldson, m., & kuss, d. j. (2019). motivational processes and dysfunctional mechanisms of social media use among adolescents: a qualitative focus group study. computers in human behavior, 93, 164–175. turel, o., & osatuyi, b. (2017). a peer-influence perspective on compulsive social networking site use: trait mindfulness as a double-edged sword. computers in human behavior, 77, 47-53. australasian journal of information systems hassan & pandey 2021, vol 25, research on user involvement social networking sites and mindfulness 32 von hellens, l., beekhuyzen, j., & kerr, d. (2006). qualitative research in information systems. australasian journal of information systems, 13(2). waters, r. d., burnett, e., lamm, a., & lucas, j. (2009). engaging stakeholders through social networking: how nonprofit organizations are using facebook. public relations review, 35(2), 102–106. watson, d., & clark, l. a. (1997). extraversion and its positive emotional core. in handbook of personality psychology (pp. 767–793). elsevier. weinstein, n., brown, k. w., & ryan, r. m. (2009). a multi-method examination of the effects of mindfulness on stress attribution, coping, and emotional wellbeing. journal of research in personality, 43(3), 374-385. who. (2018, september 13). public health implications of excessive use of the internet and other communication and gaming platforms. geneva. retrieved from https://www.who.int/news-room/detail/13-09-2018-public-health-implications-ofexcessive-use-of-the-internet-and-other-communication-and-gamingplatformshttps://www.who.int/news-room/detail/13-09-2018-public-healthimplications-of-excessive-use-of-the-internet-and-other-communication-and-gamingplatforms wiegner, l., hange, d., björkelund, c., & ahlborg, g. (2015). prevalence of perceived stress and associations to symptoms of exhaustion, depression and anxiety in a working age population seeking primary care-an observational study. bmc family practice, 16(1), 1– 8. witt, l. a. (2002). the interactive effects of extraversion and conscientiousness on performance. journal of management, 28(6), 835–851. wright, t. a., & cropanzano, r. (1998). emotional exhaustion as a predictor of job performance and voluntary turnover. journal of applied psychology, 83(3), 486. zhang, s., kwok, r. c.-w., lowry, p. b., liu, z., & wu, j. (2019). the influence of role stress on self-disclosure on social networking sites: a conservation of resources perspective. information & management, 56(7), 103147. zhang, y., tang, l. s. t., & leung, l. (2011). gratifications, collective self-esteem, online emotional openness, and traitlike communication apprehension as predictors of facebook uses. cyberpsychology, behavior, and social networking, 14(12), 733-739. zheng, x., & lee, m. k. o. (2016). excessive use of mobile social networking sites: negative consequences on individuals. computers in human behavior, 65, 65–76. australasian journal of information systems hassan & pandey 2021, vol 25, research on user involvement social networking sites and mindfulness 33 appendix 1 measurement scales used for different constructs in the study. name of the construct scale used no of items sample item compulsive sns use turel & osatuyi (2017) 7 i spent time on social networking sites even though i had other deadlines mindfulness baer et al. (2006) 6 it seems i am "running on automatic" without much awareness of what i'm doing. emotional exhaustion han (2018) 3 i feel exhausted by using social networking sites. extraversion gosling et al. (2003) 2 i see myself as extraverted, enthusiastic. social desirability crowne & marlowe (1960) 5 i am always polite even to people who are offensive. copyright: © 2021 hassan & pandey. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v25i0.2923 http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems verma, giri & debata 2022, vol 26, research article leapfrogging into knowledge economy 1 leapfrogging into knowledge economy: information and communication technology for human development anushka verma birla institute of technology and science, india p20190069@pilani.bits-pilani.ac.in arun kumar giri birla institute of technology and science, india byomakesh debata birla institute of technology and science, india abstract modern-day economic growth is focused on productivity and innovation, which puts information and technology integral to economic policy issues. in this context, ict has a significant position as it increases efficiency, promotes information dissemination, and enhances innovation, resulting in a global shift in social and human development processes. the purpose of this research is to examine the significance of ict diffusion in fostering human development in the south asian association for regional cooperation (saarc) countries from 2005 to 2019. ict diffusion is measured using a principal component analysis (pca) based composite index that combines telephone, mobile, broadband, and internet usage. the united nations development programme (undp) created human development index (hdi) serves as a proxy for human development. to adjust for any confounding bias, macroeconomic indicators, such as gross domestic product (gdp), inflation, and trade are also included. utilizing econometric methods robust to cross-sectional dependence (csd) such as the dynamic common correlated effect (dcce) estimator, driscoll-kraay (dk) regression, and the dumitrescu-hurlin (dh) causality test, the study highlights the strong positive relationship between ict and hdi. in addition, gdp boosts hdi owing to productivity gains. similarly, trade expansion, in addition to its direct effects, also influences hdi by boosting economic growth. inflation, on the other hand, has a negative impact on the hdi. consequently, the study recommends a cohesive setting that unifies ict with human development in this modern framework. keywords: human development, ict diffusion, dcce, dk regression, saarc. 1 introduction the world has become a globalized community over the course of the last few decades, owing to the transformative power of information, which has enabled the development of more substantial and faster forms of connectivity than ever before (schwab, 2018). icts and global trade and industry shifts toward technology, services, and human resources have resulted in this information being dispersed around the globe. policymakers in low-income countries are devising strategies to take advantage of the social and economic benefits of “information revolution” transforming their economies into the “knowledge economy” (world bank, 2007). this includes many labels such as "knowledge society," "information economy," "information society," and "digital economy." in contrast to the concentration on the main sectors of the economy in previous decades, these words stress the increasing reliance on ict and human resources. icts are viewed as a double-edged sword in the pursuit of long-term sustainable australasian journal of information systems verma, giri & debata 2022, vol 26, research article leapfrogging into knowledge economy 2 growth while also addressing ecological concerns (asongu & le roux, 2017). advancements in icts have had a profound effect on every facet of society, from infrastructure to transportation, businesses to service, education to culture, and entertainment to art (lee et al., 2017). until the mid-twentieth century, income was regarded as a significant aspect of economic growth; however, the reliance steadily transitioned towards human development and technological innovation (asongu & le roux, 2017). human skills and abilities, in the context of human development, are viewed as complementary inputs in any country's production and development processes (mustafa et al., 2017). it encompasses a diverse array of choices which includes healthy life, improved standard of living, and increased per capita income (yakunina & bychkov, 2015). in recent years, icts have spread to practically all aspects of human activity (latif et al., 2017). additionally, icts boosts both the productivity and economic growth, as well as human development and welfare, in both developing and developed countries (farooqi et al., 2020). the impact of icts extends beyond economic growth to human capital development in both developing as well as developed economies. the south asian association for regional cooperation (saarc) is blessed with a plethora of natural resources yet has had declining gdp rate in the past few years. almost all the countries in this region share the same culture and a similar economic, social, and geopolitical situation. more than a quarter (24.89 percent) of the world’s population lives in the saarc region, which cover 3.5 percent of the planet's area. according to the united nations, the population of south asia is predicted to grow between 0.4 and 1.9 percent by 2050 (mohsin et al., 2018). saarc has a cumulative gdp of $2.6 trillion (abbas et al., 2018). with respect to the life expectancy, sri lanka has the highest rate in the world at 70 years. while in the rest of the saarc economies, the average life expectancy is between 62 and 63 years old. moreover, a closer look at the saarc's public expenditure priorities reveals that countries in this region spend only 3 to 5 percent of the gdp on health and education (lee et al., 2017). as per the global competitiveness index with respect to ict adoption, nepal leads the area with 40.4 points, followed by bangladesh with 39.9 points, sri lanka with 32.9 points, india with 28 points, and pakistan with 23.6 points (pradhan et al., 2018). contemporary research endorses saarc as the region with sufficient opportunity for an investments in ict domains in the current environment (latif et al., 2017). however, we must acknowledge that ict adoption is not always easy, particularly for the developing economies. effective ict diffusion requires an active role of the government, given the various socio-cultural, infrastructural, economic, and technological constraints. on the other hand, the paucity of relevant local digital content is also challenging for ict diffusion. there also exist certain prerequisites to ict adoption which are merely obtainable in developing countries like that of saarc where a substantial part of the inhabitants relies on agriculture income, suffer from low literacy, and low per capita income. therefore, the above socioeconomic factors highlight the below average human development and ict diffusion rate. as a result, research and investigation into techniques and strategies to promote the expansion of icts and human development within the saarc economies is crucial. in light of the above discussion that icts are critical to human development, the present study examines the influence of ict diffusion on human development within the saarc countries. our research adds to the current literature in the following manner: first, to the best of the authors’ information, this is the first research in the saarc context to empirically evaluate australasian journal of information systems verma, giri & debata 2022, vol 26, research article leapfrogging into knowledge economy 3 the dynamic relationship and causality nexus between ict diffusion and human capital development while also incorporating the impact of economic growth, inflation, and trade openness in a unified framework. second, the study makes use of the most recent and extensive dataset available and constructed an index of ict diffusion using principal component analysis (pca) rather than focusing exclusively on a single ict indicator. finally, the study addresses the issue of cross-sectional dependence (csd), which is often disregarded in the existing body of research on ict. therefore, the present study employs the dynamic common correlated effect mean group (dcce-mg) estimator introduced by chudik and pesaran (2015) and the driscoll-kraay (dk) standard error approach of driscoll and kraay (1998) which accounts for csd and provides heteroskedasticity and autocorrelation-consistent standard errors. the remaining paper is structured in this manner. section 2 includes the theoretical basis and a brief analysis of relevant literature. section 3 discusses the materials and methods used. the empirical results have been presented in section 4. section 5 discusses the theoretical and practical implications of the study and section 6 concludes. 2 literature review this section presents the theoretical framework and a brief review of the literature. the first section will cover the theoretical underpinnings. the second section will provide an overview of past research on ict diffusion and human capital development. 2.1 theoretical framework the classical theory of economic growth has been supplanted in the existing literature by the theory of human development. the foundation of the classical theory was set up on the phenomenon of the gross national product (gnp), which is the combined value of all the goods as well as the services produced by the country at a given time period (khodabakhshi, 2011). according to the neo-classical growth model (solow, 1956), long run economic growth is contingent on exogenous factors such as technological advancement and populace expansion (donou-adonsou, 2019). in contrast to the solow growth model, which sees technological revolution as an exogenic factor, the new growth model views it as an internal driver of growth. according to alfaro et al. (2008), technological advancement is contingent on human development, and the combination of the two results in economic progress (sepehrdoust & ghorbanseresht, 2019). according to the new growth model proposed by romer (1990), innovation has an impact on technological progression, which in turn has an impact on economic growth and human capital development (pradhan et al., 2016). contemporary growth models accentuate the function of ict diffusion in human capital development and its significance for economic growth in the long run (balouza, 2019; makoza & chigona, 2013). the ict literature is highlighted by significant advancements in human competencies, improvements in health conditions, and longer lifespan (lee et al., 2017; morarivera & garcía-mora, 2021; yakunina & bychkov, 2015). ict is regarded as the core of human advancement because it not only contributes to growth but also improves the quality of life of the population (thomas, et al., 2011). in the current age, ict serves as both an input and an output i.e. by reducing market coordination costs and by enhancing public services like healthcare and education using e-services (hwang & shin, 2017). individuals now have new options for a healthier lifestyle thanks to medical applications (mosa et al., 2012). icts, such as personal computers and electronic whiteboards have since been implemented in classrooms australasian journal of information systems verma, giri & debata 2022, vol 26, research article leapfrogging into knowledge economy 4 to bring about substantial improvements in education and e-learning. this encourages pupils to participate more actively in the classroom because of enhanced communication between professors and students (msila, 2015). figure 1 summarizes the direct and indirect channels through which ict diffusion leads to human development (sarangi & pradhan, 2020). figure 1. impact of ict diffusion on human development 2.2 empirical evidence even though icts have been present for a long time, studies to identify their distinctive significant contribution to the development are fairly recent, starting in the early 2000s. one strand of literature uncovers a positive association between ict diffusion and human capital development in the developed economies (balouza, 2019; bhattacharya, 2021; farooqi et al., 2020; qureshi et al., 2020; zhang & danish, 2019), whereas the other strand of the literature reveals an adverse impact of ict on human life predominantly in the developing economies (bollou, 2006; bollou & ngwenyama, 2008; ngwenyama et al., 2006), mainly due to anxiety/nervousness (kessler et al., 2009), addiction to cyberspace (douglas et al., 2008) and, technical stress (salanova et al., 2014). pohjola (2001) used an expanded version of the neoclassical growth model to assess the influence of icts on development in 39 economies to disprove the notion that icts may considerably contribute to macroeconomic growth in rich countries but not in emerging economies. the results of the study did not disclose a correlation between the two variables unless the investigation was restricted to oecd countries, which found a positive correlation. why did emerging countries fail to imitate the accomplishments of their advanced counterparts, which constantly demonstrated a remarkable association between ict and development? one likely explanation for this is the absence of ict-complementary investments in poor nations (samoilenko & osei-bryson, 2008). another issue is the lack of infrastructure that is necessary to maximize the productive potential of ict technology (ziaie, 2013). there are also limitations on human capital in emerging countries, both in terms of quantity and quality. australasian journal of information systems verma, giri & debata 2022, vol 26, research article leapfrogging into knowledge economy 5 while some researchers (malik khalid, 2013; pohjola, 2001) speculated that developing countries were on a path to 'leapfrog' conventional stages of economic development into the digital future, more recent studies provide empirical support for those theories. table 1 highlights the empirical evidence on the relationship between the diffusion of icts and human capital development. author(s) study area method findings ngwenyama et al. (2006) west africa 1993-2003; regression analysis negative impact of ict on health component bollou & ngwenyama (2008) west africa 1995-2002; total factor productivity analysis negative impact of ict on hdi bankole et al. (2011) south africa 1998-2007; regression analysis ict contributes to hdi bankole et al. (2011) 51 countries 1994-2003; structural equation model ict impacts on hdi differ in high, middle and low income countries. asongu and le roux (2017) 49 ssa countries 2000-2012; tobit regressions ict impact varies across fundamental characteristics of hdi and ict dynamics badri et al. (2019) 15 developing countries 2012-2017; random effect modelling ict contributes to hdi balouza (2019) 6 gcc countries 2005-2014; quantitative research method results ranging between positive, negative, and insignificant relationship gupta et al. (2019) south asian 2000-2016; fixed effect modelling strong positive associations of internet and mobile usage with hdi zhang and danish (2019) asia 1990-2016; mean group estimator hdi is influenced by mobile phone use, but not by internet use. maiti &awasthi (2020) 67 countries 2000-2014; two-stages least square (2sls) less developed and developing countries have a slightly lower ict impact. qureshi et al. (2020) 15 advanced countries 1990-2017; quantile-onquantile approach feedback hypothesis between ict and hdi farooqi et al. (2020) 67 developing countries 2000-2018; auto regressive distributive lag (ardl) model ict investment have different impacts on the components of hdi in four panels of the developing countries asongu (2021) 49 ssa countries 2000-2012; tobit regressions mobile phone diffusion contributes to hdi bhattacharya (2021) 130 countries 2007-2019; generalized method of moments (gmm) negative impact of ict on hdi in lowincome countries table 1. summary of studies on ict-hdi nexus hence, the literature asserts that the relationship between ict diffusion and human capital development follows an integral but dynamic construct which strongly relies on the hdi dynamics, ict indicators, country characteristics and the adopted methodology. studies indicating a favourable relationship between ict and human development in the emerging economies are sector-specific and not global in nature. in addition, the existing literature typically operationalizes the idea of development solely in terms of gdp and disregards other crucial aspects of the development paradigm. further, the existing studies have overlooked the causality nexus (the feedback hypotheses, no causality hypothesis, one-way ict-led development hypothesis, and one-way development-led ict hypothesis) between ict diffusion and human development. presently, no consensus exists on the nature of the icthuman development relationship. this necessitates a comprehensive examination of how ict diffusion and human development are intertwined, using advanced methodologies robust to australasian journal of information systems verma, giri & debata 2022, vol 26, research article leapfrogging into knowledge economy 6 csd ensuring the transparency and reliability of the findings. ipso facto, the main objective of the present study is to answer the following research questions (rq) with respect to saarc economies. rq1: is there any association between diffusion of ict and human development? rq2: what is the nature of the association (causal relationship) between ict diffusion and human development? 3 materials and methods 3.1 data the present study aims to examine the impact of ict diffusion on human development in the saarc economies using panel data analysis cantered on the index of ict diffusion (icti) and human development (hdi). utilising data from the united nations development programme (undp), world development indicators (wdi) and international telecommunication union (itu), the analysis utilizes the data of saarc economies spanning 15 years from 2005 to 2019. the specified timeframe and extent of research is based on, the availability of data and the motivation stated in the above sections. ict diffusion is proxied by a composite index of ict diffusion (icti) constructed using pca which combines fixed telephone users per 100 people, mobile phone users per 100 people, fixed broadband users per 100 people and percentage of internet users. pca is defined as “a statistical procedure that uses an orthogonal transformation to convert a set of observations of possibly correlated variables into a set of values of linearly uncorrelated variables called principal components” (lenka, 2015). it has been frequently used in previous ict research (pradhan et al., 2016; verma & giri, 2020). the following formula, which is based on pca, is used to integrate the four different indicators of ict diffusion into a single composite index: icti = ∑ 𝑎𝑎𝑖𝑖 𝑗𝑗 𝑖𝑖=1 𝑥𝑥𝑖𝑖𝑖𝑖 𝑠𝑠𝑠𝑠 (𝑥𝑥𝑖𝑖) (1) where icti refers to the index of ict diffusion; ai are the factor loadings; sd denotes standard deviation, and xij states ith items in jth year. eigen analysis (correlation matrix) pcs eigenvalue difference proportion cumulative 1 2.766 1.842 0.691 0.691 2 0.924 0.750 0.231 0.922 3 0.174 0.039 0.043 0.966 4 0.134 0.033 1 eigen vectors (component loadings) variables pc1 pc2 pc3 pc4 mobile 0.572 -0.045 0.137 -0.806 telephone 0.229 0.958 0.108 0.127 broadband 0.563 -0.083 -0.775 0.272 internet 0.549 -0.267 0.606 0.508 kmo score: 0.80 table 2. summary of pca australasian journal of information systems verma, giri & debata 2022, vol 26, research article leapfrogging into knowledge economy 7 the steps for constructing an index using pca are discussed in detail by yorulmaz (2018). subsequently, the ict index for each year and each economy is obtained through the principal components. the index was further screened for sampling adequacy using the kaiser-meyerolkin (kmo) test (kaiser, 1974). the overall kmo score for the dataset is 0.80, indicating that our pca produces accurate and reliable results. table 2 presents the summary of pca results. figure 2 shows that maldives and bhutan are significantly ahead of the rest of the saarc countries in terms of the diffusion of ict. afghanistan and bangladesh, on the other hand, remain at the bottom of the list. however, ict performance in india, pakistan and nepal continues to rise at a steady pace. figure 2. trend of ict diffusion in the saarc economies figure 3. trend of human development index in the saarc economies australasian journal of information systems verma, giri & debata 2022, vol 26, research article leapfrogging into knowledge economy 8 the undp-created human development index (hdi) serves as a proxy for human development. hdi is a qualitative and composite index that calculates the intensity of socioeconomic development in an economy. it assesses the average levels of achievement in the areas of health, education, and a decent standard of living. figure 3 shows the human development index temporal trend for the saarc countries. from 2005 to 2019, the hdi values of all saarc economies reveals an upward sloping trend. hdi values in sri lanka and the maldives are the highest in the region. an important aspect of this research is that three control variables—per capita gdp growth (gdp), inflation (inf), and trade openness (trd) are employed as a means of controlling for possible confounding or extrinsic variables. economic growth has a direct impact on human development to the degree that it increases the options and capacities available to the government and the citizens (ranis, 2000). similarly, trade influences human growth through expanding markets, disseminating information, advancing knowledge, creating jobs, and increasing income (jawaid & waheed, 2017). further, the purchasing power of money is devalued by inflation, which has a direct influence on the standard of living (osiakwan & armah, 2013). to facilitate analysis of the estimates as the elasticity of the analysed variables, the data utilised in this study are converted into natural logs. table 3 displays the descriptive statistics and pairwise correlation matrix. lnhdi lnicti lngdp lninf lntrd mean -0.529 0.577 2.627 1.787 4.076 median -0.546 0.552 2.648 1.908 3.949 maximum -0.245 1.265 2.990 2.550 4.915 minimum -0.872 0.072 2.243 0.820 3.663 sd 0.153 0.378 0.197 0.525 0.405 lnhdi 1 lnicti 0.657 1 lngdp 0.084 0.147 1 lninf -0.239 -0.412 -0.061 1 lntrd 0.422 0.460 0.018 -0.198 1 table 3. descriptive statistics and pairwise correlation 3.2 model specification the purpose of this research is to analyse the role of ict diffusion in human capital development, taking into account economic growth, inflation, and openness to trade. as a result, an ict-based theoretical model for human capital development can be derived as: 𝑙𝑙𝑙𝑙𝑙𝑙𝑙𝑙𝑙𝑙𝑖𝑖𝑖𝑖 = 𝛼𝛼 + 𝛽𝛽1𝑙𝑙𝑙𝑙𝑙𝑙𝐼𝐼𝐼𝐼𝑙𝑙𝑖𝑖𝑖𝑖 + 𝛽𝛽2𝑙𝑙𝑙𝑙𝐺𝐺𝑙𝑙𝐺𝐺𝑖𝑖𝑖𝑖 + 𝛽𝛽3𝑙𝑙𝑙𝑙𝑙𝑙𝐼𝐼𝐼𝐼𝑖𝑖𝑖𝑖 + 𝛽𝛽4𝑙𝑙𝑙𝑙𝐼𝐼𝑙𝑙𝑙𝑙𝑖𝑖𝑖𝑖 + 𝜃𝜃𝑖𝑖𝑡𝑡 + ɛ𝑖𝑖𝑖𝑖 (2) where i = 1, 2,…, n denotes the panel members, t = 1, 2,.…, t is the time frame of the study, β1, β2, β3, and β4 signify the elasticity parameters of hdi relative to the icti, gdp, inf, and trd in the long run and θi & αit denotes the deterministic trend and fixed country effects, respectively. australasian journal of information systems verma, giri & debata 2022, vol 26, research article leapfrogging into knowledge economy 9 3.3 methodology 3.3.1 cross-sectional dependence the present study employs panel data estimate approaches to illustrate the dynamic behaviour of the parameters and enhance the estimation precision. therefore, it is possible that the issue of heterogeneity and csd which captures the complexity of macroeconomic variables among the countries may arise. panel data analysis necessitates checking for both stationary and cross-sectional dependency of economic variables. the results obtained in the presence of cross-sectional dependence are erroneous and biased in their interpretation. as a result, breusch-pagan lm (breusch & pagan, 1980), pesaran scaled lm (pesaran, 2004), bias-corrected scaled lm (baltagi et al., 2012), and pesaran cd (pesaran, 2004) tests were employed to check for csd in the data. the null hypothesis (h0) of these tests is as follows: 𝑙𝑙0 ∶ 𝜂𝜂𝑖𝑖𝑗𝑗 = 𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐 (𝜀𝜀𝑖𝑖𝑖𝑖; 𝜀𝜀𝑗𝑗𝑖𝑖) = 0∀𝑖𝑖 ≠ 𝑗𝑗 the mathematical expression of csd test (pesaran, 2004) is as follows: cd𝑃𝑃 = � 2𝑇𝑇 𝑁𝑁(𝑁𝑁−1) � ∑ 𝑁𝑁−1 𝑖𝑖=1 � �̂�𝑝𝑖𝑖𝑗𝑗 𝑁𝑁 𝑗𝑗=𝑖𝑖+1 � → 𝐼𝐼(0,1) 𝑖𝑖, 𝑗𝑗 (3) 𝑙𝑙 = � 2𝑇𝑇 𝑁𝑁(𝑁𝑁−1) � ∑ 𝑁𝑁−1 𝑖𝑖=1 � �̂�𝑝𝑖𝑖𝑗𝑗 𝑁𝑁 𝑗𝑗=𝑖𝑖+1 � (𝑇𝑇−𝑘𝑘) 𝑝𝑝�𝑖𝑖𝑖𝑖 2 −𝐸𝐸 (𝑇𝑇−𝑘𝑘) 𝑝𝑝�𝑖𝑖𝑖𝑖 2 var (𝑇𝑇−𝑘𝑘)𝑝𝑝�𝑖𝑖𝑖𝑖 2 (4) where i and t correspond to the specific cross-section and time dimension, respectively, and �̂�𝑝𝑖𝑖𝑗𝑗 is the projected multivariate cross-sectional correlation of error term across all crosssections. 3.3.2 unit root tests the subsequent stage in analysis is to execute unit root tests to determine the stationarity of the variables in question. therefore, panel unit root tests can be employed to attain this objective. it is not possible to employ first generation unit root tests to gain correct information regarding the cointegration order in the presence of csd and slope heterogeneity since these tests over-reject the null hypothesis and have an inadequate size property. because of this, the study uses second generation tests to account for csd in all cross sections. pesaran (2007) recommended two second generation panel stationarity tests: the cross-sectional augmented dickey fuller (cadf) and the cross-sectional im pesaran and shin (cips). pesaran (2007) specified the stationarity test as: 𝑥𝑥𝑖𝑖 = α𝑖𝑖𝑖𝑖 + β𝑖𝑖𝑥𝑥𝑖𝑖𝑖𝑖−1 + ρ𝑖𝑖𝑡𝑡 + ∑ 𝑛𝑛 𝑗𝑗=1 θ𝑖𝑖𝑗𝑗δ𝑥𝑥𝑖𝑖,𝑖𝑖−𝑗𝑗 + ε𝑖𝑖𝑖𝑖 (5) where, α_it refers to the intercept, t denotes the time, δ is the difference operator, 𝑥𝑥𝑖𝑖𝑖𝑖 and ε𝑖𝑖𝑖𝑖represents the variables and the error term, respectively. the null hypothesis asserts that the variables are non-stationary. 3.3.3 cointegration tests furthermore, panel cointegration analysis is used to verify whether or not there is a long-term relationship among the potentially integrated variables. both first and second generation cointegration approaches, such as pedroni (2004), kao (1999), and westerlund (2007) were used. pedroni (2004) offers seven statistics for testing the null hypothesis of “no cointegration relationship” divided into two categories: “group-mean statistics”, which average the findings australasian journal of information systems verma, giri & debata 2022, vol 26, research article leapfrogging into knowledge economy 10 of individual country test statistics, and “panel statistics”, which pool the statistics along the within-dimension. the seven test statistics allows for panel heterogeneity both in the short as well as long run dynamics and also in the long run slope and intercept coefficients. the study also uses the kao (1999) residual-based test to evaluate whether or not cointegration is present in order to validate the pedroni test results. however, we use the westerlund (2007) technique to tackle heterogeneity and csd issues, which gives more reliable and accurate information regarding long run cointegration relationships among the variables. the test examines two distinct statistics: group (gτ and ga) and panel (pτ and pa). the null hypothesis states that there exist no long run cointegration among the variables, as opposed to the alternative hypothesis that there is long run relationship between variables. the error correction system of westerlund test is as follows: δ𝑌𝑌𝑖𝑖𝑖𝑖 = 𝛿𝛿′𝑖𝑖𝑑𝑑t + 𝛼𝛼𝑖𝑖(𝑌𝑌𝑖𝑖𝑖𝑖−1 − 𝛽𝛽′𝑖𝑖𝑥𝑥𝑖𝑖𝑖𝑖−1) + ∑ 𝑝𝑝𝑖𝑖 𝑗𝑗=1 𝛼𝛼𝑖𝑖𝑗𝑗δ𝑦𝑦𝑖𝑖𝑖𝑖−𝑗𝑗 + � 𝛾𝛾𝑖𝑖𝑗𝑗δ𝑥𝑥𝑖𝑖,𝑖𝑖−𝑗𝑗 𝑝𝑝𝑖𝑖 𝑗𝑗=−𝑝𝑝𝑖𝑖 + 𝜇𝜇𝑖𝑖𝑖𝑖 (6) 3.3.4 panel model estimation after confirming the long run cointegration between the variables, the study utilizes the fully modified ordinary least squares (fmols) method to calculate the magnitude of long run elasticity (pedroni, 2004). fmols solves the issues of serial correlations, simultaneity biases, and endogeneity in the panel datasets. moreover, it generates robust results when a small sample size is used (micro-numerosity), demonstrating the non-parametric approach of this regression estimation. however, traditional methodologies, such as random effect, fixed effect, fmols, dols, and gmm methodology, presumes homogeneity and only enables modification of intercepts of the cross sections, when in real sense, the panel members are heterogeneous. thus, considering the sensitive nature of csd, the present study employs a dynamic common correlated effects mean group (dcce-mg) estimator introduced by chudik and pesaran (2015). based on the ideas of mg (pesaran & smith, 1995), pmg (pesaran et al., 1999), and cce estimation (pesaran, 2006), dcce can mitigate csd effects, using a cross-section average, 𝑙𝑙𝑙𝑙𝑙𝑙𝑖𝑖𝑖𝑖 = � 𝛼𝛼𝑖𝑖𝑖𝑖𝑙𝑙𝑖𝑖𝑖𝑖−1 + 𝑝𝑝 𝑖𝑖=0 � 𝛿𝛿𝑖𝑖𝑖𝑖𝑉𝑉𝑖𝑖𝑖𝑖−1 + 𝑝𝑝 𝑖𝑖=0 � 𝛶𝛶𝑖𝑖𝑖𝑖𝑊𝑊𝑖𝑖𝑖𝑖−1 + 𝑝𝑝 𝑖𝑖=0 𝜀𝜀𝑖𝑖𝑖𝑖 where, 𝑊𝑊𝑖𝑖−1 = (𝑙𝑙𝑖𝑖𝑖𝑖−1,𝑉𝑉𝑖𝑖𝑖𝑖−1) uit is used for human development as a dependent variable whereas vit-1 indicates all independent variables, i.e., icti, gdp, inf and trd. the average of both dependent and independent variables is indicated as 𝑊𝑊𝑖𝑖−1 to ease the csd problem (t-1). while p represents the lag of each variable. one of the primary advantages of dcce is its robustness in the event of structural breakdowns. furthermore, by applying the jack-knife correction strategy, this approach is equally relevant in cases of small sample size. also, the dcce model works well when the panel data is unbalanced. finally, to ensure the robustness of dcce test, the study also incorporates the driscoll-kraay (dk) standard error technique proposed by driscoll and kraay (1998) in addition to dcce. this strategy offers long run estimates and can counter csd, heteroscedasticity, as well as spatial/serial correlation in the data. this non-parametric regression is more efficient in the case of large time dimension because it is flexible and does not require any assumptions. in australasian journal of information systems verma, giri & debata 2022, vol 26, research article leapfrogging into knowledge economy 11 addition, the dk approach may provide estimates in both balanced as well as unbalanced panel dataset and is more efficient if the data set contains missing values (park et al., 2018). 3.3.5 causality test the final stage in the analysis is to establish a causal relationship between the variables under the study. as a corollary, the present study adopted dumitrescu and hurlin (2012) granger non-causality test, which addresses the csd issue. the test is based on an individual wald statistic representing the average non-causality relationship between all individual variables and assumes cross-sectional variations among the coefficients (usman et al., 2020). all coefficients are assumed to vary among cross-sections in this test. the following is the baseline regression equation given by dumitrescu and hurlin (2012): 𝑦𝑦𝑖𝑖𝑖𝑖 = 𝛼𝛼𝑖𝑖 + � 𝐾𝐾 𝑘𝑘=1 𝛽𝛽𝑖𝑖𝑘𝑘𝑦𝑦𝑖𝑖𝑖𝑖−𝑘𝑘 + � 𝐾𝐾 𝑘𝑘=1 γ𝑖𝑖𝑘𝑘𝑥𝑥𝑖𝑖𝑖𝑖−𝑘𝑘 + ɛ𝑖𝑖𝑖𝑖 where k is assumed to have an equivalent lag order for all panel members, and the panel must be balanced. the null hypothesis postulates that “no causal relationship exists between the variables”. 4 results to determine whether the series has csd issues, this study utilised four different csd tests. table 4 indicates the existence of csd in the variables. this suggests that the error components contain the presence of unanticipated common shocks across the countries because of economic interdependence. as a result, second generation panel unit root tests are more appropriate for testing the stationarity of the variables than the first-generation panel unit root tests in the presence of csd. series breusch-pagan lm pesaran scaled lm bias-corrected scaled lm pesaran cd lnhdi 405.739* 50.477* 50.191* 20.141* lnicti 343.095* 42.106* 41.820* 18.472* lngdp 39.069* 1.479 1.193 -0.656 lninf 101.400* 9.808* 9.522* 8.617* lntrd 97.229* 9.251* 8.965* 2.378* table 4. cross-sectional dependence test note: * indicates levels of significance at the 1% level variable cips cadf i(0) i(1) i(0) i(1) lnhdi -2.316 -3.938* -1.904 -2.392** lnicti -2.406*** -3.326* -1.810 -2.207*** lngdp -2.916* -3.968* -2.560** -3.222* lninf -2.790 -3.735* -2.549** -2.929* lntrd -2.096 -3.066* -2.252 -2.383** table 5. panel unit root test note: *, **, and *** indicates levels of significance at 1%, 5%, and 10%, respectively australasian journal of information systems verma, giri & debata 2022, vol 26, research article leapfrogging into knowledge economy 12 this study used two second generation panel unit root tests, cadf and cips, developed by pesaran (2007) to determine the stationarity of the variables. table 5 shows that the null hypothesis of the absence of a unit root is rejected at a 1% and 5% significance level. all the variables are stationary at first difference i.e. integrated at the order i (1) which fulfils the required condition to proceed for the panel cointegration analysis. the long run association among the variables is tested using the pedroni (2004), and kao (1999) tests of cointegration, as shown in table 6, which indicates presence of significant long run cointegration. the findings of these tests may sometimes be misleading since they overlook some critical issues such as csd, serial correlation, heteroscedasticity, and structural breaks, among the cross sections, whereas westerlund (2007) is a more advanced and powerful test of cointegration because it encompasses all of the abovementioned issues; thus, the findings are far more reliable. westerlund (2007) cointegration results in table 7 demonstrate that robust p-values for the ga, gt, pa, and pt tests are significant, rejecting the null hypothesis of “no cointegration” and confirming the existence of a long run link between the variables. pedroni test i it nit panel v-statistic -0.780 23.005* -0.772 panel rho-statistic 2.302 2.649 1.590 panel pp-statistic -0.136*** 0.407 -0.197** panel adf-statistic -0.464** -0.405 -1.222* group rho-statistic 2.685 4.224 1.941 group pp-statistic -3.186* 0.272* -4.810* group adf-statistic -2.72* -0.657* -4.531* kao test adf t-stat -3.866* table 6. traditional cointegration tests note: *, **, *** indicates levels of significance at 1%, 5%, and 10% level, respectively abbreviations: i, only intercept; it, both trend and intercept; nit, no trend and intercept value z value robust p value gt -1.280 1.923 0.000* ga -2.719 2.848 0.000* pt -1.945 2.074 1.000 pa -1.604 1.745 1.000 table 7. westerlund cointegration test note: * indicates levels of significance at 1% the results of both dcce and dk regression are presented in table 8. because all the variables are converted to logarithmic form, the long-run coefficient estimate of icti, gdp, inf, and trd are statistically equal to elasticities of hdi concerning icti, gdp, inf, and trd, respectively. table 8 shows that ict diffusion has a favourable and significant impact on the hdi. a 1% increase in icti leads to a 0.26% increase in hdi, which is in agreement with economic theories and is statistically significant. further, both gdp and trd have a positive and statistically significant impact on hdi. however, inflation depicted a negative impact on hdi. australasian journal of information systems verma, giri & debata 2022, vol 26, research article leapfrogging into knowledge economy 13 table 8. the results of long-run elasticity estimates note: *, **, and *** indicates levels of significance at 1%, 5%, and 10%, respectively table 9 shows the results of the panel dh causality test, which showed a two-way causal linkage i.e. bidirectional causality between icti and hdi and unidirectional causality running from hdi to gdp and hdi to trd. in contrast, the analysis depicts absence of any causal relationship between hdi and inf. null hypothesis: w-stat. zbar-stat. prob. remarks lnicti ⇎ lnhdi 5.945 2.459 0.013** hdi↔icti lnhdi ⇎ lnicti 7.731 3.798 0.000* lngdp ⇎ lnhdi 1.290 -1.032 0.301 hdi→gdp lnhdi ⇎ lngdp 4.966 1.725 0.084*** lninf⇎ lnhdi 2.122 -0.408 0.683 hdi⇎inf lnhdi⇎ lninf 7.130 3.347 0.486 lntrd ⇎ lnhdi 1.835 -0.623 0.533 hdi→trd lnhdi ⇎ lntrd 5.168 1.876 0.060*** table 9. dh causality test note: *, **, and *** indicates levels of significance at 1%, 5%, and 10% level, respectively; and the symbols →, ↔, and ⇎ indicate unidirectional, bidirectional and no causality relationship, respectively 5 discussion this research aims to examine the impact of ict diffusion on human capital development in the saarc economies for the period 2005-2019, using the most recent available data. dcce, dk regression, and the dh causality test were used to generate estimates robust to csd. the results of the study resulted in various theoretical and practical contributions which are shown below after an overview of the findings. 5.1 overview of the findings first, the results highlight positive impact of icti on hdi. this outcome is line with the studies of badri et al. (2019), bankole et al. (2011) and gupta et al. (2019). increased ict reliance on hdi characteristics such as health, education, and living standards influences hdi in a variety of ways. since ict diffusion in the medical industry helps to improve average lifespan, this subsequently contributes to the increase in hdi indicator across both the national and regional levels. in addition, the use of icts in educational institutions such as schools, education institutes, and training centres boosts the level of hdi. this is accomplished by the creation of virtual libraries and the facilitation of distant learning, both of which contribute to an improvement in the quality of education throughout the economy. variables fmols-mg dcce-mg dk regression coefficient p value coefficient p value coefficient p value lnicti 0.173* 0.000 0.268** 0.025 0.258* 0.000 lngdp 0.024*** 0.089 0.0215** 0.041 0.134** 0.020 lninf -0.003*** 0.060 -0.001*** 0.075 -0.011 0.477 lntrd 0.025* 0.002 0.038*** 0.055 0.050** 0.020 r-squared 0.70 0.63 0.68 root mse 0.09 0.01 prob>f 0.00 0.00 groups 8 8 8 observation 120 120 120 australasian journal of information systems verma, giri & debata 2022, vol 26, research article leapfrogging into knowledge economy 14 second, our study confirms the favourable impact of gdp on hdi which is in line with the existing theories. boosting growth leads to improvements in health, education, and life expectancy. furthermore, as incomes improve, people are better able to invest in their own health and education, enabling them to enjoy a better standard of living (bedir, 2016). investing in a high-quality education will allow them to strengthen their competences and secure a better job position in the future. high-income individuals may afford to spend more money on food, clothing, and housing that are of the highest quality and so have a greater impact on life expectancy and mortality rate. (bloom & canning, 2000). third, the association between trd and hdi is positive and significant. in addition to its direct benefits on human development, increased trade volume and trade liberalization improve hdi because businesses can provide better living conditions, more choices, promote sustainable, and enhance productivity, all of which have an impact on human capital in one way or another. the economic expansion that results from raising the volume of trade makes it simpler for individuals to acquire health and education services, hence fostering human development. (jawaid & waheed, 2017). fourth, in contrast, hdi is negatively impacted by inflation. this may be the result of its influence on monetary savings and spending habits, the decisions made by businesses and ultimately, the expenditures made by the government (osiakwan & armah, 2013). rather than participating in activities that are profitable and productive, economic agents typically spend a significant amount of their time and resources adjusting to inflationary circumstances and attempting to avoid the asset devaluation. inflation also influences the decision-making procedure, making it tricky for businesses to estimate their income and expenses. consequently, inflation influences government expenditure because the government itself is a consumer, and at the same time, it is required to pay off its workforce. lastly, findings indicated a bidirectional causal relationship between icti and hdi which is in line with the findings of qureshi et al. (2020). this suggests that an increase in human development indices motivates people throughout the saarc region to embrace advances in technological activities, which in turn leads to information diffusion, thereby empowering economies to accomplish the desired development objectives, such as economic empowerment, education, wellbeing, economic potentials, and poverty eradication. furthermore, the unidirectional causality association between hdi and gdp in saarc countries suggests that hdi is a major determinant of gdp in the region. education not only helps the economy thrive, but it also enhances the incomes of those who are economically disadvantaged, and improvements in health produce economic dividends. (ngwenyama et al., 2006). 5.2 contribution to theory the fundamental theoretical contribution of the present research is that ict minimizes informational asymmetries, which is linked to human wellbeing, ex-post of lowering information asymmetry; the saved information-based funds can be used to boost human development. one approach is by reducing the cost of searching. mobile phones have allowed workers to avoid the time and expense of making the long and laborious trip by simply calling to verify job prospects. second, by way of increasing communication, ict boosts education quality. texting, a less expensive alternative to voice calls, is increasingly being used by students in saarc, particularly in higher education, to exchange notes with their professors and peers. from a corporate standpoint, the improved communication made possible by icts australasian journal of information systems verma, giri & debata 2022, vol 26, research article leapfrogging into knowledge economy 15 contributes to increased productivity by allowing for more efficient supply-chain management. the other approach is by employment creation as well as other income-inducing prospects in the technological sector in general and the communications business at large. encouraging communication amongst the social networks to mitigate shock waves and lower the risk exposure of the households is the next approach. individuals in the developing rely on their close friends and family members for services provided by organisations such as emergency units, childcare centres, fire departments, insurance, and social security agencies, among others. there is no doubt that informal and family institutions in saarc have seen a major improvement because to the arrival of ict. the fifth way that ict can impact human capital development is through easing the provision of infrastructure and services in other economic sectors, like agriculture, healthcare, finance, and education. one example is google pay, a mobile banking service that allows the public to virtually store money on their smartphones and utilise it to make transactions anywhere in the country. it is impossible to comprehend the extent to which these services have affected the standard of living in saarc. 5.3 contribution to practice several policy recommendations can be derived from the findings in order to address the enormous challenges of ict adoption in saarc economies. the substantial investment and significant number of smartphone users in this region stimulates human development. people can readily communicate and convey information about health and education via smartphones and internet. the following recommendations are suggested for policymakers in the saarc economies. first, it is imperative that the general public have simple and easy access to ict and be provided with the requisite ict skills and digital literacy. second, formulate ict policies for the most recent ict applications, train ict workers, and promote e-culture to improve healthcare, education system and living standard. third, ensure that infrastructure and connectivity are of higher quality. fourth, lower the cost of ict equipment and provide low-cost mobile and internet service plans so that the majority of people can maximize the benefits of the technological revolution. lastly, encourage greater investment in the promotion of the internet, which is essential for human development and sustainable economic progress. 5.4 limitations and future research directions notwithstanding its contributions, the present research has certain drawbacks and limitations. to begin, the hdi, which serves as the dependent variable, does not incorporate each facet of human capital development as well as the quality of living, specifically subjective indicators. the scope of future studies could also be broadened to include more variables that accurately reflects the human welfare. second, the study can be expanded and improved by examining the impacts of utilising ict at the regional or local level, where the model can incorporate the variables specific to certain regions, like those of cultural aspect. second, the study can be expanded and improved by examining the impacts of utilising ict at the regional level, in which the framework could further include variables specific to certain areas, like those of cultural aspects. lastly, future research could increase the number of variables and utilise a wider variety of research methodologies to investigate the link in more depth. 6 conclusion technological advancements are occurring at a rate, scale, and intensity that have never been seen before. the only way for the international community to fulfill the 2030 agenda for australasian journal of information systems verma, giri & debata 2022, vol 26, research article leapfrogging into knowledge economy 16 people, peace, and prosperity is to take advantage of this progress. frontier technologies carry the possibility of reviving productivity and making abundant resources accessible to permanently eradicate poverty, permit more sustainable growth patterns, and reduce or even reverse decades of environmental degradation. in order to ensure that technological change and innovation are used to achieve equitable and sustainable outcomes, governments in developing countries, along with civil societies, businesses, and academicians, need to work together. if policymakers are not proactive, technological disruption can exacerbate inequality, further alienate the poor, and feed anti-open society and economy groups. using panel data analysis spanning the years 2005 to 2019, this study examines the role of ict diffusion in spurring human capital development in the saarc economies. it highlights the importance of policies that would assist individuals in navigating this transition period of advancements in ict. this may necessitate that the stakeholders adjust the social contract to the new reality being created by frontier technology. education will evolve into an increasingly more critical factor in determining the direction of human development and social progress. since icts serve as enablers and multipliers of other cutting-edge technologies, we should ensure that everyone, especially women and girls, has the cognizant opportunity to develop digital skills. technology leapfrogging, as transpired in saarc, must begin with a psyching phase that focuses on developing a plan for ict development. countries will need to think outside the box in order to provide efficient social protection systems for people who may not be able to keep up with the rapid technological change. the opportunity cost of not understanding the consequences of ict is as substantial as the opportunity cost of not being able to access and utilize its contents. most importantly, the international community must make a concerted effort to narrow the numerous gaps in technological capability that exist between developed and developing nations. investment in hard and soft infrastructure, human capital, and innovation platforms for sustainable development are important to ensure that the benefits of frontier technology are widely disseminated across the world. references abbas, s. z., kousar, a., razzaq, s., saeed, a., alam, m., & mahmood, a. (2018). energy management in south asia. energy strategy reviews, 21, 25–34. https://doi.org/https://doi.org/10.1016/j.esr.2018.04.004 alfaro, l., kalemli-ozcan, s., & volosovych, v. (2008). why doesn’t capital flow from rich to poor countries? an empirical investigation. the review of economics and statistics, 90(2), 347–368. https://doi.org/10.1162/rest.90.2.347 asongu, s. a. (2021). the effects of mobile phone technology, knowledge creation and diffusion on inclusive human development in sub-saharan africa. journal of the knowledge economy, 12(3), 1367–1398. https://doi.org/10.1007/s13132-020-00668-8 asongu, s. a., & le roux, s. (2017). enhancing ict for inclusive human development in subsaharan africa. technological forecasting and social change, 118, 44–54. https://doi.org/10.1016/j.techfore.2017.01.026 badri, a. k., badri, p. k., & cham, m. (2019). ict and human development in selected devolving countries. international journal of social sciences perspectives, 5(2), 30–35. https://doi.org/10.33094/7.2017.2019.52.30.35 australasian journal of information systems verma, giri & debata 2022, vol 26, research article leapfrogging into knowledge economy 17 balouza, m. (2019). the impact of information and communication technologies on the human development in the gulf cooperation council countries: an empirical study. management studies and economic systems, 4(2), 79–113. https://doi.org/10.12816/0055079 baltagi, b. h., feng, q., & kao, c. (2012). a lagrange multiplier test for cross-sectional dependence in a fixed effects panel data model. journal of econometrics, 170(1), 164–177. https://doi.org/10.1016/j.jeconom.2012.04.004 bankole, felix o., shirazi, f., & brown, i. (2011). investigating the impact of ict investments on human development. the electronic journal of information systems in developing countries, 48(1), 1–19. https://doi.org/10.1002/j.1681-4835.2011.tb00344.x bankole, felix olu, brown, i., & osei-bryson, k.-m. (2011). the impact of ict infrastructure on human development: an analysis of ict-use in sadc countries. proceedings of the 11th international conference on social implications of computers in developing countries, kathmandu, nepal, november 2016. bankole, felix olu, thomas, i., & brown, j. (2011). the impact of ict infrastructure on human development: an analysis of ict-use in sadc countries. november 2016. bedir, s. (2016). healthcare expenditure and economic growth in developing countries. advances in economics and business, 4(2), 76–86. https://doi.org/10.13189/aeb.2016.040202 bhattacharya, a. (2021). impact of information and communication technology on human development: a cross-country analysis. comparative advantage in the knowledge economy, 97–111. https://doi.org/10.1108/978-1-80071-040-520210009 bloom, d. e., & canning, d. (2000). the health and wealth of nations. science, 287(5456), 1207– 1209. https://doi.org/10.1126/science.287.5456.1207 bollou, f. (2006). ict infrastructure expansion in sub-saharan africa: an analysis of six west african countries from 1995 to 2002. the electronic journal of information systems in developing countries, 26(1), 1–16. https://doi.org/10.1002/j.1681-4835.2006.tb00174.x bollou, f., & ngwenyama, o. (2008). are ict investments paying off in africa? an analysis of total factor productivity in six west african countries from 1995 to 2002. information technology for development, 14(4), 294–307. https://doi.org/10.1002/itdj.20089 breusch, t. s., & pagan, a. r. (1980). the lagrange multiplier test and its applications to model specification in econometrics. the review of economic studies, 47(1), 239–253. https://doi.org/10.2307/2297111 chudik, a., & pesaran, m. h. (2015). common correlated effects estimation of heterogeneous dynamic panel data models with weakly exogenous regressors. journal of econometrics, 188(2), 393–420. https://doi.org/10.1016/j.jeconom.2015.03.007 donou-adonsou, f. (2019). technology, education, and economic growth in sub-saharan africa. telecommunications policy, 43(4), 353–360. https://doi.org/10.1016/j.telpol.2018.08.005 douglas, a. c., mills, j. e., niang, m., stepchenkova, s., byun, s., ruffini, c., lee, s. k., loutfi, j., lee, j. k., atallah, m., & blanton, m. (2008). internet addiction: meta-synthesis of australasian journal of information systems verma, giri & debata 2022, vol 26, research article leapfrogging into knowledge economy 18 qualitative research for the decade 1996-2006. computers in human behavior, 24(6), 3027– 3044. https://doi.org/10.1016/j.chb.2008.05.009 driscoll, j. c., & kraay, a. c. (1998). consistent covariance matrix estimation with spatially dependent panel data. the review of economics and statistics, 80(4), 549–560. https://doi.org/10.1162/003465398557825 dumitrescu, e., & hurlin, c. (2012). testing for granger non-causality in heterogeneous panels ☆. economic modelling, 29(4), 1450–1460. https://doi.org/10.1016/j.econmod.2012.02.014 farooqi, z. ud., makhdum, m. s. a., & yaseen, m. r. (2020). impact of information and communication technology (ict) investment on different components of human development in developing countries. international review of management and business research, 9(4), 108–129. https://doi.org/10.30543/9-4(2020)-11 gupta, s., jain, m., & nagpal, a. (2019). an empirical investigation on associated linkage between human development and ict: a south asian perspective. 96167. hwang, w. s., & shin, j. (2017). ict-specific technological change and economic growth in korea. telecommunications policy, 41(4), 282–294. https://doi.org/10.1016/j.telpol.2016.12.006 jawaid, s. t., & waheed, a. (2017). contribution of international trade in human development of pakistan. global business review, 18(5), 1155–1177. https://doi.org/10.1177/0972150917710345 kaiser, h. f. (1974). an index of factorial simplicity. psychometrika, 39(1), 31–36. https://doi.org/10.1007/bf02291575 kao, c. (1999). spurious regression and residual-based tests for cointegration in panel data. journal of econometrics, 90(1), 1–44. https://doi.org/10.1016/s0304-4076(98)00023-2 kessler, r. c., aguilar-gaxiola, s., alonso, j., chatterji, s., lee, s., & ustün, t. b. (2009). the who world mental health (wmh) surveys. psychiatrie (stuttgart, germany), 6(1), 5–9. khodabakhshi, a. (2011). relationship between gdp and human development indices in india. international journal of trade, economics and finance, 2(3), 251–253. https://doi.org/10.7763/ijtef.2011.v2.111 latif, z., xin, w., danish, k., iqbal, k., pathan, z. h., salam, s., & jan, n. (2017). ict and sustainable development in south asian countries. human systems management, 36(4), 353–362. https://doi.org/10.3233/hsm-17166 lee, s. o., hong, a., & hwang, j. (2017). ict diffusion as a determinant of human progress*. information technology for development, 23(4), 687–705. https://doi.org/10.1080/02681102.2017.1383874 lenka, s. k. (2015). measuring financial development in india: a pca approach. theoretical and applied economics, 22(1), 205–216. maiti, d., & awasthi, a. (2020). ict exposure and the level of wellbeing and progress: a cross country analysis. in social indicators research (vol. 147, issue 1). springer netherlands. https://doi.org/10.1007/s11205-019-02153-5 australasian journal of information systems verma, giri & debata 2022, vol 26, research article leapfrogging into knowledge economy 19 makoza, f., & chigona, w. (2013). the outcomes of ict diffusion and human development in malawi. international journal of information communication technologies and human development, 4(4), 52–70. https://doi.org/10.4018/jicthd.2012100105 malik khalid. (2013). united nations development programme: human development report 2013: the rise of the south: human progress in a diverse world. in population and development review (vol. 39, issue 3). united nations development programme. https://doi.org/10.1111/j.1728-4457.2013.00624.x mohsin, m., zhou, p., iqbal, n., & shah, s. a. a. (2018). assessing oil supply security of south asia. energy, 155, 438–447. https://doi.org/https://doi.org/10.1016/j.energy.2018.04.116 mora-rivera, j., & garcía-mora, f. (2021). internet access and poverty reduction: evidence from rural and urban mexico. telecommunications policy, 45(2), 102076. https://doi.org/10.1016/j.telpol.2020.102076 mosa, m., yoo, i., & sheets, l. (2012). a systematic review of healthcare applications for smartphones. bmc med.inf. decision making. bmc medical informatics and decision making, 12(67), 1–31. http://www.biomedcentral.com/1472-6947/12/67 msila, v. (2015). teacher readiness and information and communications technology (ict) use in classrooms: a south african case study. creative education, 06(18), 1973–1981. https://doi.org/10.4236/ce.2015.618202 mustafa, g., rizov, m., & kernohan, d. (2017). growth, human development, and trade: the asian experience. economic modelling, 61(december 2016), 93–101. https://doi.org/10.1016/j.econmod.2016.12.007 ngwenyama, o., andoh-baidoo, f. k., bollou, f., & morawczynski, o. (2006). is there a relationship between ict, health, education and development? an empirical analysis of five west african countries from 1997-2003. the electronic journal of information systems in developing countries, 23(1), 1–11. https://doi.org/10.1002/j.16814835.2006.tb00150.x niebel, t. (2018). ict and economic growth – comparing developing, emerging and developed countries. world development, 104, 197–211. https://doi.org/10.1016/j.worlddev.2017.11.024 osiakwan, r. o., & armah, s. e. (2013). clearing the muddy waters: an attempt to decompose the impacts of inflation on the ghanaian economy and on the welfare of ghanaian citizens in light of recent political rhetoric on inflation. journal of economic development, management, it, finance and marketing, 5(2), 32–65. park, y., meng, f., & baloch, m. a. (2018). the effect of ict, financial development, growth, and trade openness on co(2) emissions: an empirical analysis. environmental science and pollution research international, 25(30), 30708–30719. https://doi.org/10.1007/s11356-0183108-6 pedroni, p. (2004). panel cointegration: asymptotic and finite sample properties of pooled time series tests with an application to the ppp hypothesis. econometric theory, 20(3), 597–625. https://doi.org/10.1017/s0266466604203073 pesaran, m. h. (2004). general diagnostic tests for cross section dependence in panels. center for economic studies & ifo institute for economic research cesifo, working pa, 1–39. australasian journal of information systems verma, giri & debata 2022, vol 26, research article leapfrogging into knowledge economy 20 pesaran, m. h. (2006). estimation and inference in large heterogeneous panels with a multifactor error structure. econometrica, 74(4), 967–1012. https://www.jstor.org/stable/3805914 pesaran, m. h. (2007). a simple panel unit root test in the presence of cross-section dependence. journal of applied econometrics, 22(2), 265–312. http://www.jstor.org/stable/25146517 pesaran, m. h., shin, y., & smith, r. p. (1999). pooled mean group estimation of dynamic heterogeneous panels. journal of the american statistical association, 94(446), 621–634. https://doi.org/10.1080/01621459.1999.10474156 pesaran, m. h., & smith, r. (1995). estimating long-run relationships from dynamic heterogeneous panels. journal of econometrics, 68(1), 79–113. https://doi.org/https://doi.org/10.1016/0304-4076(94)01644-f pohjola, m. (2001). information technology and economic growth: a cross-country analysis. in information technology, productivity, and economic growth. oxford university press. https://doi.org/10.1093/acprof:oso/9780199243983.003.0011 pradhan, r. p., arvin, m. b., & hall, j. h. (2016). economic growth, development of telecommunications infrastructure, and financial development in asia, 1991-2012. quarterly review of economics and finance, 59, 25–38. https://doi.org/10.1016/j.qref.2015.06.008 pradhan, r. p., arvin, m. b., & norman, n. r. (2015). the dynamics of information and communications technologies infrastructure, economic growth, and financial development: evidence from asian countries. technology in society, 42, 135–149. https://doi.org/10.1016/j.techsoc.2015.04.002 pradhan, r. p., mallik, g., & bagchi, t. p. (2018). information communication technology (ict) infrastructure and economic growth: a causality evinced by cross-country panel data. iimb management review, 30(1), 91–103. https://doi.org/10.1016/j.iimb.2018.01.001 qureshi, m. a., qureshi, j. a., ahmed, a., qaiser, s., ali, r., & sharif, a. (2020). the dynamic relationship between technology innovation and human development in technologically advanced countries: fresh insights from quantiles-on-quantile approach. social indicators research, 152(2), 555–580. https://doi.org/10.1007/s11205-02002451-3 ranis, g. (2000). economic growth and human development. world development, 28(2), 22– 54. https://doi.org/10.23943/princeton/9780691166377.003.0002 romer, p. m. (1990). endogenous technological-change. journal of political economy, 98(5), 71– 102. https://doi.org/10.1086/261725 salanova, m., llorens, s., & ventura, m. (2014). technostress: the dark side of technologies. in c. korunka & p. hoonakker (eds.), the impact of ict on quality of working life (pp. 87–103). springer netherlands. https://doi.org/10.1007/978-94-017-8854-0_6 samoilenko, s., & osei-bryson, k. m. (2008). increasing the discriminatory power of dea in the presence of the sample heterogeneity with cluster analysis and decision trees. expert systems with applications, 34(2), 1568–1581. https://doi.org/10.1016/j.eswa.2007.01.039 australasian journal of information systems verma, giri & debata 2022, vol 26, research article leapfrogging into knowledge economy 21 sarangi, a. k., & pradhan, r. p. (2020). ict infrastructure and economic growth: a critical assessment and some policy implications. decision, 47(4), 363–383. https://doi.org/10.1007/s40622-020-00263-5 schwab, k. (2018). the global competitiveness report. in world economic forum. sepehrdoust, h., & ghorbanseresht, m. (2019). impact of information and communication technology and financial development on economic growth of opec developing economies. kasetsart journal of social sciences, 40(3), 546–551. https://doi.org/10.1016/j.kjss.2018.01.008 solow, r. m. (1956). a contribution to the theory of economic growth. the quarterly journal of economics, 70(1), 65–94. http://www.jstor.org/stable/1884513 usman, o., alola, a. a., & sarkodie, s. a. (2020). assessment of the role of renewable energy consumption and trade policy on environmental degradation using innovation accounting: evidence from the us. renewable energy, 150, 266–277. https://doi.org/10.1016/j.renene.2019.12.151 verma, a., & giri, a. k. (2020). ict diffusion, financial development, and economic growth: panel evidence from saarc countries. journal of public affairs, november, 1–11. https://doi.org/10.1002/pa.2557 westerlund, j. (2007). testing for error correction in panel data. oxford bulletin of economics and statistics, 69(6), 709–748. https://doi.org/10.1111/j.1468-0084.2007.00477.x world bank. (2007). building knowledge economies advanced strategies for development. the world bank. https://doi.org/10.1596/978-0-8213-6957-9 yakunina, r. p., & bychkov, g. a. (2015). correlation analysis of the components of the human development index across countries. procedia economics and finance, 24(july), 766–771. https://doi.org/10.1016/s2212-5671(15)00692-9 yorulmaz, r. (2018). an analysis of constructing global financial inclusion indices. borsa istanbul review, 18(3), 248–258. https://doi.org/10.1016/j.bir.2018.05.001 zhang, j., & danish. (2019). the dynamic linkage between information and communication technology, human development index, and economic growth: evidence from asian economies. environmental science and pollution research, 26(26), 26982–26990. https://doi.org/10.1007/s11356-019-05926-0 ziaie, p. (2013). challenges and issues of ict industry in developing countries based on a case study of the barriers and the potential solutions for ict deployment in iran. 2013 international conference on computer applications technology (iccat), 1–6. https://doi.org/10.1109/iccat.2013.6521973 note on data availability: the data that confirm the findings of this study are publicly available on the world bank database at https://databank.worldbank.org/source/worlddevelopment-indicators, international telecommunication union (itu) database at https://www.itu.int/en/itu-d/statistics/pages/stat/default.aspx and united nations development programme (undp) database at http://hdr.undp.org/en/data. http://hdr.undp.org/en/data australasian journal of information systems verma, giri & debata 2022, vol 26, research article leapfrogging into knowledge economy 22 copyright: © 2022 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v26i0.3883 http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 literature review 2.1 theoretical framework 2.2 empirical evidence 3 materials and methods 3.1 data 3.2 model specification 3.3 methodology 3.3.1 cross-sectional dependence 3.3.2 unit root tests 3.3.3 cointegration tests 3.3.4 panel model estimation 3.3.5 causality test 4 results 5 discussion 5.1 overview of the findings 5.2 contribution to theory 5.3 contribution to practice 5.4 limitations and future research directions 6 conclusion paper title (paper title style) australasian journal of information systems marx & mirbabaie 2022, vol 26, research on applied ethics the investigator’s dilemma 1 the investigator’s dilemma a review of social media analytics research ethics in information systems julian marx university of duisburg-essen, germany milad mirbabaie paderborn university, germany milad.mirbabaie@uni-paderborn.de abstract social media have become not only integral parts of our private and professional lives, but also an indispensable source of data for empirical research across a variety of academic disciplines. applying a social media analytics (sma) methodology, however, imposes heavy ethical challenges on researchers. scholars in the information systems (is) discipline must deal with a patchwork of ethical frameworks, regulations, and (missing) institutional support. to initiate a debate on how to develop a common understanding of sma research ethics, this paper compiles a scoping review of extant literature and suggests a research agenda for is scholarship on this matter. the review yields a total of eight fundamental principles of ethical sma research, which provide a starting point to guiding individual researchers towards more ethical conduct. at the same time, this work unearths a multitude of intricate dilemmas that are currently unresolved. the findings of this review will encourage is scholarship to find its own voice in the debate about social media research ethics. keywords: social media analytics, ethics, information systems. 1 introduction the impact of social media use on how communication is shaped within societies is continuously reinforced. nowadays, social media are used in various domains, for example in business (beier & wagner, 2016; richter et al., 2011), politics (bruns & burgess, 2015; s. stieglitz & dang-xuan, 2013), or crisis management (mirbabaie et al., 2020; reuter et al., 2018). hence, information systems (is) research continues to closely investigate social media as both a phenomenon and data source. communication on social media is often complex and voluminous, creating high quantities of data. at this juncture, the social media analytics (sma) approach combines a set of datadriven analysis techniques, including the collection, analysis, and visualisation of social media data (stieglitz et al., 2014). particularly in is research, the sma approach has helped scholars to navigate the process of social data analysis, aiming at the improvement of making sense of the data (zeng et al., 2010). this data handling, however, results in unprecedented challenges for researchers. stieglitz et al. (2018) elaborated that sma researchers have to deal with 1) a high heterogeneity of information, 2) data spread across different platforms, 3) many actors who are involved, and 4) a highly dynamic information diffusion. mailto:milad.mirbabaie@uni-paderborn.de australasian journal of information systems marx & mirbabaie 2022, vol 26, research on applied ethics the investigator’s dilemma 2 these challenges exemplify that handling social media data constitutes a unique way of conducting research. social media data may include personal and identifying information such as a research subject’s name, date of birth, or occupation. oftentimes, a researcher can hardly estimate to what extent this personal information is of a sensitive nature (lomborg & bechmann, 2014). rather mundane social media postings can turn out to contain highly sensitive information. health, gender, or financial information, to name a few examples, can be used to enable privacy or security harm (ohm, 2015). this makes sma subject to high ethical standards for the way research is conducted. so far, however, clear definitions and guidance about how to deal with social media data in research projects is missing. in the is literature, an infinitesimal number of scholars addresses ethics with explicit focus on the field of social media analytics (anderson et al., 2019; zafeiropoulou et al., 2015). there is hardly any consensus about ethical principles within online and computer research or, more specifically, sma (gruzd et al., 2020). ethical guidance for sma resembles a farrago with roots in disparate disciplines, individual judgement, and the need to comply with (supra-)local legislations such as the gdpr in countries of the european union. consequently, we argue, researchers conducting sma face an additional challenge: 5) ambiguity of ethical principles. in the past decade, ethical conduct in sma research appeared to be of secondary relevance due to the early age of the field, little institutional control, and lack of research towards consensus. the former factors, however, diminish with sma being an accepted methodology in high ranked is journals and, at the same time, higher ethical standards expected by publishers and project funders. from a researcher’s standpoint, however, following ethical guidelines in sma research is inherently problematic. throughout the sma process, the individual investigator finds herself in various ethical dilemmas. the latter, also referred to as moral dilemmas, are situations that involve conflicts between different moral requirements (foot, 1983). for instance, it is practically impossible to receive informed consent by many social media users and explore the communication patterns of a particular event. neither it is possible to comply with the terms and conditions of (most) social media platforms and collect supposedly public social media data. as a result, we argue, sma researchers are subject to another challenge: 6) the urgency to resolve ethical dilemmas. in order to reduce the ambiguity of existent ethical principles that apply to sma research in is, and to lay bare current ethical dilemmas of sma research, this paper aims to answer the following research questions: rq1: what is known from existing scholarship in is and its reference disciplines about ethical principles in social media analytics research? rq2: which ethical dilemmas (if any) need to be resolved by is scholars in social media analytics research? to answer these questions, we conducted a scoping literature review, which aims to map extant scholarship in a field primarily based on volume and characteristics of the published research. the review covers 58 relevant publications from is and reference disciplines. by using keyword, forward and backward searches, we collected a sample of literature that portrays how the topic of sma research ethics is represented in is outlets and what cumulative tradition this debate builds on. this paper aims to contribute to existing literature by bringing australasian journal of information systems marx & mirbabaie 2022, vol 26, research on applied ethics the investigator’s dilemma 3 together existing efforts of ethical guidance in sma research that are mainly unconnected to the interdisciplinary nature of the field. moreover, this work attempts to raise awareness for the ethical dilemmas that result from combining ethical principles and practical sma research. in order to mitigate this conflict, we propose a research agenda for sma research ethics in is and provide propositions about how to approach possible dilemmas. thus, we contribute to both the debate on sma ethics within the is discipline and the individual practice of sma researchers. the paper is structured as follows. first, we provide a methodological background about the conduct of scoping reviews, which underlies our study. additionally, we report our systematic search and exclusion criteria. second, we present our findings, i.e. ethical principles and dilemmas that have been discussed in sma research or emerge from the juxtaposition of the reviewed literature. we conclude with a research agenda, including recommendations for further research and a discussion of limitations. 2 scoping review methodology to provide a general conspectus on the present state of research in terms of sma ethics, we choose the approach of a scoping review. it constitutes a literature-based approach to synthesise research evidence and map the extant scholarship in a field, including volume and characteristics of the research (arksey & o’malley, 2005; paré et al., 2014). according to mays et al. (2001), a scoping review turns out to be useful if a research topic has not been extensively reviewed or is particularly complex or of heterogenous nature. the topic of this study, that is, ethical conduct in sma research, meets all those criteria, and therefore, is an appropriate subject of a scoping review. the expected contribution of this approach comprises the summary and dissemination of extant research findings, the identification of research gaps, and the scope and value of systematic reviews narrowed down to specific research questions within the domain (arksey & o’malley, 2005; levac et al., 2010). this work is based on the framework proposed by arksey and o’malley (2005) and additional suggestions made by levac et al. (2010). the framework provides five phases to conduct a scoping review: (1) identifying the research question, (2) identifying relevant studies, (3) study selection, (4) charting the data, and (5) collating, summarizing, and reporting the results. 2.1 research questions this scoping review is guided by the questions ‘what is known from existing is scholarship about ethical principles in social media analytics research?’ and ‘which ethical dilemmas (if any) need to be resolved by is scholars in social media analytics research?’. we are aware that sma research is a field of interdisciplinary efforts and is of concern for both social sciences (e.g., sociology, media and communication studies, business studies, economics, political science, and social psychology) and natural and applied sciences (e.g., computer science, information systems, linguistics, statistics, or physics) (stieglitz et al. 2014). however, each of these disciplines follows its own methodological idiosyncrasies and approaches to sma research with differing perspectives. to contribute to the body of knowledge of is research at the best possible rate, we use the is discipline as a starting point, to scope the debate within this discipline and its research identity that focuses on “how it systems are developed and how individuals, groups, organizations, and markets interact with it” (sidorova, evangelopoulos, valacich, & thiagarajan, 2008, p. 467). however, through iterative forward and backward searches (vom brocke et al., 2015), publications from reference disciplines will be identified australasian journal of information systems marx & mirbabaie 2022, vol 26, research on applied ethics the investigator’s dilemma 4 that inform the debate in is. this will also yield insights about what those reference disciplines are1. this is vital as neither sma nor ethics are fields that are discussed only in is. we decided to divide this work into two areas of interest, that is, the identification of existing ethical principles and the juxtaposition of potential ethical dilemmas present in the debate about sma research. by doing so, we argue, this scoping review will not only be able to uncover what the debate is composed of but also to expose possible contradictions, and to give recommendations for further directions. 2.2 data sources and search criteria we initiated our literature search on august 26th, 2020 by using the tool litbaskets.io2 . it is a browser-based application connected to scopus3 and tailored for is researchers by providing a curated set of is-related journals (boell & wang, 2019). we chose the largest preconfigured basket 3xl, which includes, at the time of the search, 847 journals that refer to the is discipline. having a broad basket of outlets serves the purpose of a scoping review to focus on breath rather than depth (arksey & o’malley, 2005). subsequently, we performed the same initial search on the ais electronic library (aisel) to expand the search to outlets that might not be considered by litbaskets.io but might be part of the leading database for is research. the combination of using litbaskets.io and the aisel allowed us to include all relevant is outlets in our search. for the search query, terms were used that were considered by the authors to describe the topic of social media analytics and ethics: “social media analytics” or “social media analysis” and “moral” or “ethic” or “ethical” or “ethics”. the search query was generated by litbaskets.io and then inserted on scopus. for aisel, the same query was tailored manually to fit the characteristics of the database. we made the a priori decision that a screening of relevant articles should only cover the first 100 hits (as sorted by relevance) per database. this is because the search algorithms of databases vary in broadness to a great extent. the above search string yielded 4 results on scopus and 8,044 on aisel. of those hits, the first 100 results per database were coded as relevant or not, following suggestions from the literature that further screening is unlikely to reveal a greater number of relevant publications (pham et al., 2014; stevinson & lawlor, 2004). duplicates were excluded prior to the initial screening. 2.3 eligibility assessment the screening was performed independently by the two authors and included a review of the title and abstract, a brief paraphrase of the topic and an assessment, whether the article is considered relevant or not. for the keyword search, an article had to clearly deal with social media analysis techniques and, at least in the abstract, address ethical considerations of the research practice. this is a complete list of all inclusion and exclusion criteria for the assessment after the keyword search: 1the term “reference discipline refers to “those disciplines x that provide foundational, methodological, or other inputs to another discipline/s y such that the state of knowledge in y is advanced through inputs provided by x. in other words if y cites x in order to develop and advance the state of its knowledge, x becomes a reference discipline for y, and vice versa.” (grover et al. 2006, p.337) 2https://www.litbaskets.io/, last accessed 09/12/2020 3http://scopus.com/, last accessed 09/12/2020 australasian journal of information systems marx & mirbabaie 2022, vol 26, research on applied ethics the investigator’s dilemma 5 • studies had to be peer-reviewed and published in journals or conference proceedings • studies must deal with social media and include an ethical reflection of data analysis techniques • due to limited resources for translation, articles published in languages other than english were excluded. • to be able to map the current debate on this topic, we limited the search to articles published after 2010 after screening the first 20 articles, a reviewer agreement kappa of 0.95 was calculated, which is considered as high reviewer agreement (> 0.8) (randolph, 2005). subsequently, a forward and backward snowball search was conducted with each relevant article to collect further appropriate publications by screening the reference lists (backward search) and seeking for relevant publications that have cited these papers (forward search) (vom brocke et al., 2015). the inclusion and exclusion criteria for the forward and backward searches are the following: • the time limitation (articles published after 2010) was repealed. • we did not exclude articles that did not deal with social media the reason for this is that we found that the debate on sma ethics is essentially not new. it builds on established concepts and principles, which have been developed pre-social media. therefore, to understand the debate on sma research ethics, the cumulative tradition of ethical conduct must be considered. moreover, we altered the following criterion: • we did not exclude non-scientific sources in the backward search. the reason for this is that a handful of non-scientific publications were cited by various research papers, which indicates that those publications are seminal for the debate on sma research ethics. the two researchers met on a regular basis and discussed the approach to resolve possible inconsistencies. as no changes in the coding process were necessary after the reliability pretest, the two coders arrived at an overall kappa of 0.90 after reviewing all data. to manage and perform the coding, microsoft excel was used, whereas the authors used mendeley as a literature management system. 3 results 3.1 search results and selection of relevant articles in total, the search yielded 58 relevant articles. from those articles, 15 originated from the database search, 15 from the first iteration of forward and backward search, 21 from the second iteration, 4 from a third iteration, and 3 from a final iteration. all articles could be gathered in full-text, and thus were included in the review. many studies were excluded, especially in the initial screening, as the search yielded a lot of publications with a methodical focus on social media analytics techniques but without any consideration of ethics. figure 1 provides an overview of the article selection process. australasian journal of information systems marx & mirbabaie 2022, vol 26, research on applied ethics the investigator’s dilemma 6 figure 1. literature search process the literature sample shows a high variety of studies in terms of outlets and originating disciplines addressing ethical issues that form a basis for potential debate in is research. a particularly small portion of the resources stem from core is outlets, whereas most of the research was published in journals from reference disciplines. table 1 provides an overview and characteristics of selected articles. discipline total number of publications types of publications authors information systems (is) 6 conference proceedings (6) anderson et al., 2019; gruzd et al., 2020; kaschesky et al., 2013; malsbender, voigt, beverungen, & rosemann, 2013; shropshire, 2014; zafeiropoulou et al., 2015 computer science 5 journal article (3), conference proceedings (2) carpenter & dittrich, 2011; fiesler, lampe, & bruckman, 2016; libert, 2015; mavroudis & milne, 2016; vitak, shilton, & ashktorab, 2016 information science 4 journal article (4) crawford & finn, 2015; fiesler & proferes, 2018; zimmer, 2010; zimmer & proferes, 2014 media studies 10 book (5), book section (2), journal article (3) bakardjieva & feenberg, 2000; couldry, 2010; couldry & powell, 2014; curran, fenton, & freedman, 2012; danah boyd, 2010; dijck & poell, 2013; dubois, gruzd, & jacobson, 2020; fuchs, 2014; gillespie, 2014; trottier, 2012 sociology 3 journal article (3) allen, 1996; kennedy, elgesem, & miguel, 2017; kennedy & moss, 2015 psychology 4 journal article (4) beninger et al., 2014; conway & o’connor, 2016; roberts, 2015; suler, 2004 philosophy 2 journal article (2) elgesem, 2002; nissenbaum, 2011 medicine 18 journal article (18) brotsky & giles, 2007; conway, 2014; eysenbach & till, 2001; franz, marsh, chen, & teo, 2019; golder, ahmed, norman, & booth, 2017; golder, australasian journal of information systems marx & mirbabaie 2022, vol 26, research on applied ethics the investigator’s dilemma 7 scantlebury, & christmas, 2019; guillemin & gillam, 2004; heilferty, 2011; ioannidis, 2013; mayer & till, 1996; mikal, hurst, & conway, 2016; moreno, fost, & christakis, 2008; moreno, grant, kacvinsky, moreno, & fleming, 2012; rothstein & shoben, 2013; swirsky, hoop, & labott, 2014; vayena, salathé, madoff, & brownstein, 2015; vitak, proferes, shilton, & ashktorab, 2017; yeung, 2018 law 4 book (1), journal article (3) cohen, 2012; heimer & petty, 2010; kerr, 2001; solove, 2013 economics 1 journal article (1) hair & clark, 2007 other 1 newspaper article (1) johnson, 2010 table 1. overview and characteristics of selected articles scoping the literature on sma ethics provides several useful insights on how the debate on this matter unfolds. first, is can solely contribute conference papers (6) to the debate. consequently, is researchers heavily rely on the work provided by reference disciplines. even though computer science contributed a couple of journal articles (3), the predominant discipline on research ethics and sma is medicine with 18 journal articles. figure 2 provides an additional point of view on the development of the debate. figure 2. bubble plot of identified studies and their respective disciplines. the distribution shown in figure 2 exemplifies how the debate of sma ethics evolved across disciplines. studies discussing ethics of online research pre social media (1996-2004) were predominantly published either in medicine or across various disciplines with lesser touchpoints to is. later, in the post social media phase (2007-2020), debates began in other disciplines, including media studies and computer science. remarkably, is and computer science are the disciplines with the latest start in any research activity on sma research ethics. furthermore, all articles published in is on this matter are part of conference proceedings, leaving no journal articles contributing to this debate. australasian journal of information systems marx & mirbabaie 2022, vol 26, research on applied ethics the investigator’s dilemma 8 3.2 ethical principles of sma research considered in is to map the contents of the selected articles and identify the ethical principles covered by extant literature, we inductively derived emerging themes from the full texts. we screened the articles for content that was particularly labelled as ethical ‘principles’, ’challenges’, or ’issues’. subsequently, we categorised all principles that apply to sma research. this content analysis approach was performed according to the suggestions regarding inductive category formation made by mayring (2014). here, we did not paraphrase the contents but jumped directly to the level of abstraction that we aimed for (ethical principles and challenges). we revised the categories after coding 50% of the papers and calculated an interrater agreement kappa of 0.79, which counts as substantial agreement (randolph, 2005). overall, this analysis resulted in eight main categories. each category represents one ethical principle that was addressed in the literature. table 2 provides an overview of the identified principles and in which studies it they were mentioned. ethical principle supporting literature total no. of times mentioned preserving privacy allen 1996; mayer & till 1996; kerr 2001; eysenbach & till 2001; elgesem 2002; hair & clark 2007; moreno et al. 2008; zimmer 2010; johnson 2010; heilferty 2011; nissenbaum 2011; moreno et al. 2012; trottier 2012; malsbender et al. 2013; solove 2013; swirsky et al. 2014; beninger et al. 2014; zimmer & proferes 2014; conway 2014; shropshire 2014; libert 2015; roberts 2015; vayena et al. 2015; kennedy & moss 2015; vitak et al. 2016; conway & o’connor 2016; fiesler et al. 2016; mikal et al. 2016; golder et al. 2017; franz et al. 2019; dubois et al. 2020 31 obtaining informed consent eysenbach & till 2001; elgesem 2002; hair & clark 2007; moreno et al. 2008; zimmer 2010; rothstein et al. 2013; malsbender et al. 2013; carpenter & dittrich 2011; ioannidis 2013; beninger et al. 2014; zimmer & proferes 2014; conway 2014; roberts 2015; crawford & finn 2015; kennedy & moss 2015; vitak et al. 2016; conway & o’connor 2016; golder et al. 2017; fiesler & proferes 2018; anderson et al. 2019; franz et al. 2019 21 minimising harm and maximising benefit allen 1996; mayer & till 1996; elgesem 2002; hair & clark 2007; swirsky et al. 2014; beninger et al. 2014; vayena et al. 2015; kennedy & moss 2015; vitak et al. 2016; golder et al. 2017; fiesler & proferes 2018; golder et al. 2019 12 producing good research moreno et al. 2008; shropshire 2014; roberts 2015; vayena et al. 2015; vitak et al. 2016; conway & o’connor 2016; golder et al. 2017 7 transparency allen 1996; bakardjieva & feenberg 2000; boyd 2010; eysenbach & till 2001; hair & clark 2007; brotsky & giles 2007; carpenter & dittrich 2011; couldry & powell 2014; shropshire 2014; libert 2015; roberts 2015; vayena et al. 2015; zafeiropoulou et al. 2015; vitak et al. 2016; golder et al. 2017 15 fairness moreno et al. 2008; crawford & finn 2015; vayena et al. 2015; kennedy et al. 2017; fiesler & proferes 2018; anderson et al. 2019 6 data minimisation zimmer 2010; zimmer & proferes 2014 2 diversity boyd 2010; kaschesky et al. 2013; golder et al. 2017 3 table 2. identified ethical principles and number of mentions. australasian journal of information systems marx & mirbabaie 2022, vol 26, research on applied ethics the investigator’s dilemma 9 preserving privacy safeguarding the privacy of research subjects in the context of sma turns out to be the most frequently discussed ethical principle in is literature and its reference disciplines. this principle particularly addresses the integrity and autonomy of each individual social media user and aims at the “protection of the individual’s ability to form reasonable expectations about how information about him or her will be treated in different channels” (elgesem, 2002, p. 201). a related notion mentioned in the literature is confidentiality, that is, to protect personal identifiers and sensitive information, even though they are publicly available on social media (moreno et al., 2008). consequently, research should be designed in a way that inhibits the ex post reidentification of subjects (zimmer, 2010). safeguarding the privacy rights of data subjects is crucial as potential risks such as data aggregation, profiling and selective targeting may arise (nissenbaum, 2011), alongside the risk of general surveillance (trottier, 2012). from a researcher’s perspective, anonymisation and pseudonymisation techniques are the obvious measure to respect this principle (roberts, 2015). based on the papers included in this review, a consensus emerges that only public data should be part of sma research (conway & o’connor, 2016; mikal et al., 2016; fiesler et al., 2016). however, this maxim entails an often discusses and partly unresolved dilemma: what data stemming from social media can be considered public and what data is private? many factors such as differing contexts or applied philosophies determine the outcome of this decision (solove, 2013). as fiesler (2016) point out, studies have shown “striking misunderstandings of copyrights in one’s own content—for example, that any content posted online automatically becomes public domain” (p.1452). here, the individual perception of privacy, including researchers, often differs from how it is defined in social media platforms’ terms and conditions. especially under consideration of the different nature of public, private, and semi-public spaces created by social media, this problem remains important for prospective research on sma ethics. dilemma: a researcher faces the often-difficult task to draw a line between public and private social media space, which determines the eligibility of data. obtaining informed consent building on the argument of privacy protection, the literature on hand defines informed consent as another key principle of ethical sma research. the procedure of obtaining informed consent is rooted in research involving human participants (eysenbach & till, 2001). the literature that contemporary is papers rely on largely stem from the discipline of medicine, in which standards were established for clinical studies involving social media. in its most basic manifestation, informed consent is achieved through explaining the characteristics of a particular study to research subjects and asking them for permission (hair & clark, 2007). to obtain informed consent from social media users, researchers need to reach out to them individually and ask for consent to obtain their data. one could argue that using social media and confirming a platform’s terms and conditions would equal this type of consent. however, research suggests that users are not aware of how their data is used, and therefore, might not consent (anderson et al., 2019). the same might apply to consent forms handed out by researchers. such forms are often difficult to understand for social media users and cannot fully convey the implications of the study (fiesler & proferes, 2018). another stream of sma research, that is, aiming at identifying patterns in large scale social datasets rather than (n)ethnographic research, faces another pivotal dilemma: it is practically impossible to obtain consent from millions of social media users (franz et al., 2019; golder et al., 2019). australasian journal of information systems marx & mirbabaie 2022, vol 26, research on applied ethics the investigator’s dilemma 10 dilemma: a researcher is asked to obtain informed consent of social media users while often experiencing its impracticability. given this dilemma, some scholars have reflected upon the question whether it is necessary to obtain consent from a large social media user base or if other measures should be taken. kennedy and moss (2015), for example advocate for a democratisation of sma, which is subject to (1) greater public supervision and regulation, (2) available and accessible to a greater public, and (3) used to educate the public to become reflexive, active and knowing. sma projects that use retrospective data to learn about events such as crises, elections, or social movements, can obtain consent from users who would be highlighted in form of quoted postings or descriptions of their roles and profiles. nevertheless, dissent remains whether to treat sma as human subject research (e.g. and ethnographic field study about an online community) or humanities research (e.g. a social network analysis of an election campaign) (heimer & petty, 2010; rothstein & shoben, 2013). this distinction clearly determines the ethical standards that should be applied to a given study. most ethical principles that have emerged in the reviewed literature have been developed against the backdrop of human subject research designs. here, sma research ethics need to differentiate and give room for reflexivity (guillemin & gillam, 2004) and ‘in-flux’ ethical considerations (vitak et al., 2016) on the part of the researcher – dependent on context and data. dilemma: when performing sma, a researcher must distinguish between human subject research and humanities research to make ethical decisions. in case a sma endeavour is classified as involving human subjects and informed consent is obtained, a researcher might face an additional problem: consent bias. participants may show altered behaviour in the way they use social media once they have given consent to be researched (rothstein & shoben, 2013). for example, in closed communities or groups an unintended behaviour change might occur as a result of the presence of a researcher. as we learn from the discipline of medicine, debates revolve around loosening the requirements for informed consent in case of research that is informational rather than interventional (ioannidis, 2013). dilemma: as a result of obtaining consent, research findings of sma studies may be compromised by consent bias. minimising harm and maximising benefit the third pillar of ethical sma research is the principle of avoiding harm and, at the same time, creating benefit for the research subject, the community or public under scrutiny, and society. possible (perceived) harm for social media users may comprise unsolicited attention on the web and, “abuse” or bullying, but also exploitation from organisations or use by authorities (golder et al., 2017). in contrast to medical studies involving social media, in which physical harm constitutes a serious risk of data misuse, is and computer science might be more concerned with the psychological and social nature of harm. vitak et al. (2016), for instance refer to the “golden rule”, that is “do to others what you would have them do to you.” (p. 945). this deontological approach may fundamentally differ from other ethical philosophies. separate philosophies can lead to different outcomes and weighing of harm and benefit. in contrast to a normative approach, a researcher could also apply a utilitarian perspective and pursue the greatest good for the greatest number of people (hair and clark, 2007). however, australasian journal of information systems marx & mirbabaie 2022, vol 26, research on applied ethics the investigator’s dilemma 11 the overall tone of research on this matter shows a tendency towards always pursuing the common good (vayena et al., 2015; vitak et al., 2016). dilemma: a researcher must determine what it means to be ‘ethical’ and consider sometimes conflicting ethical philosophies in order to make decisions. producing good research a deeply entrenched ethical principle that applies to sma research is the researcher’s responsibility to conduct ‘good’ research. it is mentioned primarily in the studies that were published pre-social media but appeared as important reference literature ever since (allen, 1996; mayer & till, 1996). not to be confused with benefitting the common good, this principle aims at a high validity of deployed research methods as well as rigor (golder et al., 2017). moreover, data retrieved from social media must be interpreted with the context of the data in mind (anderson et al., 2019). this “contextual integrity” (vayena et al., 2015, p.3) is imperative to collect, analyse, and theorise data with the needed sensitivity for the context of its origin, that is, platform characteristics, user history, socio-demographic specificities etc. regarding this principle, too, problems may arise such as “collapsed contexts” (boyd, 2010, p. 11), if different viewpoints about what is appropriate or ethical collide. contributing factors to this problem are a missing consensus on guidelines and misalignment of bureaucratic and professional ethics among ethics boards (heimer & petty, 2010; vitak et al., 2017) and additional regulations such as gdpr. the fact that the individual researcher must adhere to top-down (supposed) ethical and legal standards and do justice to her own ethical standards may confront her with an ethical dilemma. dilemma: a researcher must distinguish between the regulation of science (boards), legal frameworks, and the regulation of ethics, which may be misaligned. transparency early ethnographic studies in online and social media involved the intrusion of researchers in virtual communities to study them. a prominent example for this kind of research was the deception inside the “pro ana” virtual community described by brotsky and giles (2007). here, a researcher became part of a community related to the eating disorder anorexia nervosa without disclosing herself as a researcher. this seminal study lead to a debate about whether deception can be justified if a greater good is being served. earlier studies that are part of this review clearly emphasise that the phenomenon shall not be perturbed (allen, 1996). however, after brotsky and giles’ seminal paper, most authors do not categorically oppose the means of deception if it is very well justified (vitak et al., 2016). in recent years, however, the field of sma involves much more analyses that include a much larger number of research subjects. carpenter and dittrich (2011) note that this type of research “diminishes […] transparency and creates a distance between the researchers and potentially impacted parties” (p.2). as this distance leads to researchers not considering their work as human subject research, they argue, new and consistent ethical standards need to be developed. dilemma: a sma methodology might create an unprecedented researcher-subject relationship which is not covered by existing ethical standards. fairness according to the findings by kennedy et al. (2017), social media users expect a fair use of their data when sma is applied. in other words, sma research is fair when it “meets users’ australasian journal of information systems marx & mirbabaie 2022, vol 26, research on applied ethics the investigator’s dilemma 12 expectations about the collection and use of social media data” (kenney et al., 2017, p.17). once social media users publish information in a particular context online, their data rights will be violated in case the analysis removes the data from this context. avoiding this practice has also been coined as “contextual integrity” (nissenbaum, 2011). moreover, the fairness principle foresees that data shall not be exploited and researchers aim to handle the data in a just manner (fiesler & proferes, 2018). at the same time, studies that questioned social media users found that the general public is largely unaware about the possibility that their social media activity might be subject to research work (fiesler & proferes, 2018; golder et al., 2017). research projects increasingly implement awareness campaigns to inform the general public about their research. this can happen via social media, blogs, or other public relations measures. dilemma: too few users are aware about the fact that their social media postings might be analysed by researchers, leaving the researcher with the question: who is responsible for this more general awareness campaign? data minimisation with the increasing number of research subjects in datasets obtained from social media, the protection of personal profile data becomes a challenge for researchers (zimmer, 2010). this entails the need for automated and reliable anonymisation and pseudonymisation techniques. moreover, this development also poses the question of how much data is needed to answer a particular research question. consequently, zimmer & proferes (2014) conducted a systematic analysis of twitter-based academic research, finding a trend towards increasingly larger social media datasets. additionally, with the implementation of gdpr in 2018, the data minimisation has become a legal matter. both research and data protection regulations point towards the importance of the data minimisation principle, that is, to treat personal data as “adequate, relevant, and limited to what is necessary in relation to the purposes for which they are processed” (art. 5, gdpr). as franz et al. (2019) point out, a dilemma emerges from the responsibility that is placed upon the researcher. dilemma: despite the power and resources of platform vendors, the responsibility to ensure that social media users’ data are obtained minimised and protected is placed on the researcher. diversity hair and clark (2007) examined ethnographic research in online communities in terms of the range of ethical dilemmas and challenges. their argument stresses the point that each facebook group, twitter feed and youtube comment section is different and will involve dissimilar ethical challenges. the fact that different technological structures create diverse patterns of behaviour was also made by boyd (2010). consequently, it becomes an intricate task for the researcher to apply standards, guidelines and adhere to regulations to the analysis of these highly diverse environments. therefore, hair and clarke (2007) argue, it might be helpful to follow a relativistic approach to ethics that involves independent decisions and the open negotiation of the research practice within a discipline. moreover, according to the reviewed literature, sma approaches should consider the diverse nature of society and consider the fact that some groups and minorities might be underrepresented when following certain sampling methods. the terminology regarding the principle of diversity is quite inhomogeneous. mentioned are concepts that refer to power and agency such as “invisible audiences” (boyd, 2010), people’s “voice” (couldry, 2011; couldry & powell, 2014), and “segments of society” (kaschesky et al., 2013). in the reviewed literature, australasian journal of information systems marx & mirbabaie 2022, vol 26, research on applied ethics the investigator’s dilemma 13 kennedy and moss (2015) underline a dilemma that is grounded in a power difference between researcher and data subject. dilemma: sma can yield an unequal relationship between societal groups and the researcher, creating a lack of agency and equal data power. 4 discussion 4.1 towards a shared code of ethical practice in sma research scoping the literature about sma research ethics has yielded several striking patterns, which characterise the current state of the debate on this matter inside the is discipline. in the following, three patterns that could be identified from the literature will be discussed. first, the is discipline has missed out on making its mark in the debate on sma research ethics. whereas research published in is journals contributed important methodological groundwork to the interdisciplinary field of sma (e.g. stieglitz et al. 2014; stieglitz et al. 2018), no further efforts have been made to scrutinise sma methods in terms of their ethical requirements. this became apparent in the results section, with is research contributing to the debate about ethical principles in sma research with only six conference papers. other disciplines such as psychology or medicine have developed a rich discussion on social media research ethics in leading journals (e.g. conway & o’connor, 2016; golder et al. 2017). admittedly, those are disciplines of much greater proportions than is but, at the same time, are much further away from sma being a core concern of their disciplines’ identities. to be able to shape the debate on sma ethics to a much greater extent, is journals should encourage for more research on this matter by calls for papers and collaboration with other disciplines. the urgency to find better consensus about how sma research can be made more ethical provides an opportunity for is to set an example in a core area of its expertise. at this point in time, however, is remains dependent on reference disciplines to discuss the sma methodology. here, is researchers can use their experience in designing artefacts that resolve dilemmas about privacy, diversity, or data minimisation “by design”, e.g. with machine learning algorithms. second, extant ethical principles do not reflect the specificities of large-scale sma approaches. the literature on hand largely develops ethical principles against the backdrop of ethnographic social media research. for example, questions about informed consent have been largely answered in medical research (e.g. conway, 2014; franz et al., 2019), but remain unresolved when it comes to “digital humanities”, in which large datasets are researched to unearth patterns or mechanisms of a larger user base. a cut set of ethical principles that are referenced in the literature have been developed pre social media (1996-2004). therefore, one could pose the legitimate question: what is different with social media? why do ethical theories and principles from other domains not apply to social media? as this review showed, some ethical principles in fact apply to sma research as to other methodologies. however, sma approaches that involve the mass processing of personal data, for example concerning public events, social movements and the like, a considerable number of dilemmas emerge when applying traditional research ethics or even standards that have been derived from ethnographic social media research. one example being the inability to obtain informed consent for large data samples (anderson et al. 2019). australasian journal of information systems marx & mirbabaie 2022, vol 26, research on applied ethics the investigator’s dilemma 14 consequently, much more scrutiny is needed to exactly identify the distinguishing features of sma research and finding a shared code of practice within the field. third, the protection of the individual researcher has been erased from the equation. the ethical principle that we identified called ‘minimising harm and maximising benefit’ is almost entirely focused on the integrity of the research subject. in fact, only golder et al. 2017 reflected upon the protection of the researcher. surely most researchers handling social media data are experienced in their field of study and can foresee psychological consequences of their research efforts. however, coding large-scale data sets and social media contents oftentimes involves the labour of novice researchers and students. considering the fact that a multitude of sma studies involve data revolving around societal crises, e.g. terroristic attacks, natural disasters, or political uprisings, the low efforts concerning researcher protection are astounding. being confronted with bulk quantities of disturbing imagery and language can take a toll on a researcher’s psychological well-being. at this point, automated systems such as image recognition algorithms could filter disturbing material prior to the researchers being exposed to it. however, more consideration needs to be placed upon the protection of the research team. 4.2 a research agenda for the information systems discipline ultimately, the above-mentioned findings from extant literature give some direction to address the challenges of sma research ethics and move towards a shared code of ethical conduct in this field. hereby, we encourage especially is researchers to reflect their own sma practices to contribute to a more open and fruitful debate about sma research ethics. in the following, we propose research questions that might be of interest to the is discipline. based on the findings of this literature review, we grouped those exemplary questions along three levels of analysis (see table 3). level of analysis research question institutional how can the interest in research on sma ethics be incentivised by is journals and research associations? what are ethical dilemmas that can be addressed by interdisciplinary research projects? how can sma methodologies be made more explainable to improve the collaboration between researchers and ethics boards? technological how can sma artefacts be designed to incorporate ethical principles ‘by design’? how can the individual researcher be protected throughout the sma research process, e.g. through automated filtering? individual/project how can individual projects be empowered to initiate awareness campaigns to educate the public about the practices, risks, and benefits of sma research? table 3. examples for possible research questions about sma research ethics. this collection of possible research questions only represents a limited number of issues that need to be addressed. additionally, the identified dilemmas serve as starting points for further research as well. it is up to us, the researchers, to raise questions and add them to the list, to lead fruitful discussions about our own methods, and to reduce the distance between us and our research subjects by transparent communication and exchange. institutional entities, too, are responsible to do their part and intensify their efforts to assume responsibility to find a shared code of sma research practice. this will take some weight of the shoulders of the individual researcher and benefit all parties involved in the complex but so highly promising field of sma. australasian journal of information systems marx & mirbabaie 2022, vol 26, research on applied ethics the investigator’s dilemma 15 4.3 limitations our study comes with limitations. in this case, we considered literature that explicitly addresses sma research ethics. our search and eligibility criteria, therefore, may have led to the exclusion of studies that primarily applied sma and reflected about research ethics in a secondary manner. we might have missed relevant hits due to the limitation of screening only the first 100 hits per database, which we justify with practicability and suggestions from the literature. to focus on a purely scientific debate, we did not include institutional guidelines and frameworks or legal texts such as gdpr. those writings might have given us more depth regarding certain principles, however, a high number of reviewed articles referred to relevant guidelines and regulations and incorporated them into their findings. 5 conclusion in this study, we conducted a scoping literature review to determine the state of the art in social media analytics research ethics. a total of 58 identified pieces of literature was charted, summarised, and discussed. the findings uncovered that eight ethical principles are relevant for sma research, that is, preserving privacy, obtaining informed consent, minimising harm and maximising benefit, producing good research, transparency, fairness, data minimisation, and diversity. at the same time, our analysis revealed that several ethical dilemmas emerge for the individual researcher once each principle is applied. this is true especially for characteristic sma research that involves large-scale data set involving big samples of data subjects. based on our review results, we found that (1) the is discipline has missed out on making its mark in the debate on sma research ethics, (2) extant ethical principles do not reflect the specificities of large-scale sma approaches, and (3) the protection of the individual researcher has been erased from the equation. finally, we derived a research agenda for information systems to address those shortcomings and advance the important debate on sma research ethics. acknowledgement this project has received funding from the european union’s horizon 2020 research and innovation programme under the marie skłodowska-curie grant agreement no 823866. references allen, c. (1996). what’s wrong with the “golden rule”? conundrums of conducting ethical research in cyberspace. information society, 12(2), 175–188. https://doi.org/10.1080/713856146 anderson, j., casas saez, g., anderson, k., palen, l., & morss, r. (2019). incorporating context and location into social media analysis: a scalable, cloud-based approach for more powerful data science. hawaii international conference on system sciences (hiccs), 2274– 2283. https://doi.org/10.24251/hicss.2019.275 arksey, h., & o’malley, l. (2005). scoping studies: towards a methodological framework. international journal of social research methodology: theory and practice, 8(1), 19–32. https://doi.org/10.1080/1364557032000119616 bakardjieva, m., & feenberg, a. (2000). involving the virtual subject. ethics and information technology, 2(1), 233–240. https://doi.org/10.1023/a:1011454606534 beier, m., & wagner, k. (2016). social media adoption : barriers to the strategic use of social media. european conference on information systems (ecis), 1–18. australasian journal of information systems marx & mirbabaie 2022, vol 26, research on applied ethics the investigator’s dilemma 16 beninger, a. k., fry, a., jago, n., lepps, h., nass, l., silvester, h., beninger, k., fry, a., jago, n., lepps, h., nass, l., & silvester, h. (2014). research using social media; users’ views. natcen social research. boell, s. k., & wang, b. (2019). www.litbaskets.io, an it artifact supporting exploratory literature searches for information systems research. australasian conference on information systems (acis), 663–673. https://www.acis2019.org/papers/acis2019_paperfin_141.pdf brotsky, s. r., & giles, d. (2007). inside the “pro-ana” community: a covert online participant observation. eating disorders, 5(2), 93–109. https://doi.org/10.1080/10640260701190600 bruns, a., & burgess, j. (2015). twitter hashtags from ad hoc to calculated publics. in hashtag publics: the power and politics of discursive networks (pp. 13–28). https://doi.org/10.3726/978-1-4539-1672-8 carpenter, k. j., & dittrich, d. (2011). bridging the distance: removing the technology buffer and seeking consistent ethical analysis in computer security research. in 1st international digital ethics symposium. loyola university chicago center for digital ethics and policy. cohen, j. e. (2012). configuring the networked self: law, code, and the play of everyday practice. yale university press. conway, m. (2014). ethical issues in using twitter for public health surveillance and research: developing a taxonomy of ethical concepts from the research literature. journal of medical internet research, 16(12), 1–9. https://doi.org/10.2196/jmir.3617 conway, m., & o’connor, d. (2016). social media, big data, and mental health: current advances and ethical implications. current opinion in psychology, 9(2), 77–82. https://doi.org/10.1016/j.copsyc.2016.01.004 couldry, n. (2010). why voice matters: culture and politics after neoliberalism. in why voice matters: culture and politics after neoliberalism. sage publications. https://doi.org/10.4135/9781446269114 couldry, n., & powell, a. (2014). big data from the bottom up. big data and society, 1(2), 1–5. https://doi.org/10.1177/2053951714539277 crawford, k., & finn, m. (2015). the limits of crisis data: analytical and ethical challenges of using social and mobile data to understand disasters. geojournal, 80(1), 491–502. https://doi.org/10.1007/s10708-014-9597-z curran, j., fenton, n., & freedman, d. (2012). misunderstanding the internet. routledge. https://doi.org/10.4324/9780203146484 danah boyd. (2010). social network sites as networked publics: affordances, dynamics, and implications. in networked self: identity, community, and culture on social network sites. routledge. dijck, j. van, & poell, t. (2013). understanding social media logic. in media and communication (pp. 2–14). https://doi.org/10.12924/mac2013.01010002 dubois, e., gruzd, a., & jacobson, j. (2020). journalists’ use of social media to infer public opinion: the citizens’ perspective. social science computer review, 38(1), 57–74. https://doi.org/10.1177/0894439318791527 elgesem, d. (2002). what is special about the ethical issues in online research? ethics and information technology, 4(1), 195–20. https://doi.org/10.1023/a:1021320510186 eysenbach, g., & till, j. e. (2001). ethical issues in qualitative research on internet communities. british medical journal, 323, 1103–1105. https://doi.org/10.1136/bmj.323.7321.1103 australasian journal of information systems marx & mirbabaie 2022, vol 26, research on applied ethics the investigator’s dilemma 17 fiesler, c., lampe, c., & bruckman, a. s. (2016). reality and perception of copyright terms of service for online content creation. acm conference on computer supported cooperative work (cscw), 1450–1461. https://doi.org/10.1145/2818048.2819931 fiesler, c., & proferes, n. (2018). “participant” perceptions of twitter research ethics. social media and society, 4(1), 1–14. https://doi.org/10.1177/2056305118763366 foot, p. (1983). moral realism and moral dilemma. the journal of philosophy, 80(7), 379–398. franz, d., marsh, h. e., chen, j. i., & teo, a. r. (2019). using facebook for qualitative research: a brief primer. journal of medical internet research, 21(8), 1–12. https://doi.org/10.2196/13544 fuchs, c. (2014). social media: a critical introduction. sage publications. https://doi.org/10.4135/9781446270066 gillespie, t. (2014). the relevance of algorithms. in media technologies (pp. 167–194). https://doi.org/10.7551/mitpress/9780262525374.003.0009 golder, s., ahmed, s., norman, g., & booth, a. (2017). attitudes toward the ethics of research using social media: a systematic review. journal of medical internet research, 19(6), 1–19. https://doi.org/10.2196/jmir.7082 golder, s., scantlebury, a., & christmas, h. (2019). understanding public attitudes toward researchers using social media for detecting and monitoring adverse events data: multi methods study. journal of medical internet research, 21(8), 1–13. https://doi.org/10.2196/jmir.7081 gruzd, a., abul-fottouh, d., & mashatan, a. (2020). who is influencing the #gdpr discussion on twitter: implications for public relations. hawaii international conference on system sciences (hicss), 2619–2629. https://doi.org/10.24251/hicss.2020.319 guillemin, m., & gillam, l. (2004). ethics, reflexivity, and “ethically important moments” in research. qualitative inquiry, 10(2), 261–280. https://doi.org/10.1177/1077800403262360 hair, n., & clark, m. (2007). the ethical dilemmas and challenges of ethnographic research in electronic communities. international journal of market research, 49(6), 1–13. https://doi.org/10.1177/147078530704900609 heilferty, c. m. (2011). ethical considerations in the study of online illness narratives: a qualitative review. journal of advanced nursing, 67(5), 945–953. https://doi.org/10.1111/j.1365-2648.2010.05563.x heimer, c. a., & petty, j. (2010). bureaucratic ethics: irbs and the legal regulation of human subjects research. annual review of law and social science, 6(1), 601–626. https://doi.org/10.1146/annurev.lawsocsci.093008.131454 ioannidis, j. p. a. (2013). informed consent, big data, and the oxymoron of research that is not research. american journal of bioethics, 13(4), 40–42. https://doi.org/10.1080/15265161.2013.768864 johnson, b. (2010). privacy no longer a social norm, says facebook founder. the guardian. kaschesky, m., sobkowicz, p., lobato, j. m. h., bouchard, g., archambeau, c., scharioth, n., manchin, r., gschwend, a., & riedl, r. (2013). bringing representativeness into social media monitoring and analysis. hawaii international conference on system sciences (hicss), 2003–2012. https://doi.org/10.1109/hicss.2013.120 kennedy, h., elgesem, d., & miguel, c. (2017). on fairness: user perspectives on social media data mining. convergence, 23(3), 270–288. https://doi.org/10.1177/1354856515592507 kennedy, h., & moss, g. (2015). known or knowing publics? social media data mining and the question of public agency. big data and society, 2(2), 1–11. https://doi.org/10.1177/2053951715611145 australasian journal of information systems marx & mirbabaie 2022, vol 26, research on applied ethics the investigator’s dilemma 18 kerr, i. (2001). the legal relationship between online service providers and users. canadian business law journal, 35, 1–40. levac, d., colquhooun, h., & o’brien, k. k. (2010). scoping studies: advancing the methodology. implementation science, 5(69), 1–18. https://doi.org/10.1017/ cbo9780511814563.003 libert, t. (2015). privacy implications of health information seeking on the web. communications of the acm, 58(3), 68–77. https://doi.org/10.1145/2658983 lomborg, s., & bechmann, a. (2014). using apis for data collection on social media. information society, 30(4), 256–265. https://doi.org/10.1080/01972243.2014.915276 malsbender, a., voigt, m., beverungen, d., & rosemann, m. (2013). digital social signal processing theoretical underpinning and research agenda. pacific asia conference on information systems (pacis), 1–12. mavroudis, j., & milne, e. (2016). researching microcelebrity: methods, access and labour. first monday, 21(7), 1–20. https://doi.org/10.5210/fm.v21i7.6401 mayer, m., & till, j. e. (1996). the internet: a modern pandora’s box? quality of life research, 5(1), 568–571. https://doi.org/10.1007/bf00439230 mayring, p. (2014). qualitative content analysis: theoretical foundation, basic procedures and software solution. ssoar. https://nbn-resolving.org/urn:nbn:de:0168-ssoar-395173 mays, n., roberts, e., & popay, j. (2001). synthesising research evidence. in studying the organisation and delivery of health services (pp. 1–33). routledge. mikal, j., hurst, s., & conway, m. (2016). ethical issues in using twitter for population-level depression monitoring: a qualitative study. bmc medical ethics, 17(1), 1–11. https://doi.org/10.1186/s12910-016-0105-5 mirbabaie, m., bunker, d., stieglitz, s., marx, j., & ehnis, c. (2020). social media in times of crisis: learning from hurricane harvey for the covid-19 pandemic response. journal of information technology, 35(3), 195–213. https://doi.org/10.1177/0268396220929258 moreno, m. a., fost, n. c., & christakis, d. a. (2008). research ethics in the myspace era. pediatrics, 121(1), 157–161. https://doi.org/10.1542/peds.2007-3015 moreno, m. a., grant, a., kacvinsky, l., moreno, p., & fleming, m. (2012). older adolescents’ views regarding participation in facebook research. journal of adolescent health, 51(5), 439– 444. https://doi.org/10.1016/j.jadohealth.2012.02.001 nissenbaum, h. (2011). a contextual approach to privacy online. daedalus, 140(4), 32–48. https://doi.org/10.1162/daed_a_00113 ohm, p. (2015). sensitive information. southern california law review, 88(5), 1125–1196. paré, g., trudel, m. c., jaana, m., & kitsiou, s. (2014). synthesizing information systems knowledge: a typology of literature reviews. information and management, 52(2), 183–199. https://doi.org/10.1016/j.im.2014.08.008 pham, m. t., rajić, a., greig, j. d., sargeant, j. m., papadopoulos, a., & mcewen, s. a. (2014). a scoping review of scoping reviews: advancing the approach and enhancing the consistency. research synthesis methods, 5(4), 371–385. https://doi.org/10.1002/jrsm.1123 randolph, j. j. (2005). free-marginal multirater kappa. joensuu learning and instruction symposium, 1–20. reuter, c., hughes, a. l., & kaufhold, m. a. (2018). social media in crisis management: an evaluation and analysis of crisis informatics research. international journal of humancomputer interaction, 34(4), 280–294. https://doi.org/10.1080/10447318.2018.1427832 richter, d., riemer, k., & vom brocke, j. (2011). internet social networking. business & information systems engineering, 3(2), 89–101. https://doi.org/10.1007/s12599-011-0151-y australasian journal of information systems marx & mirbabaie 2022, vol 26, research on applied ethics the investigator’s dilemma 19 roberts, l. d. (2015). ethical issues in conducting qualitative research in online communities. qualitative research in psychology, 12(3), 314–325. https://doi.org/10.1080/14780887.2015.1008909 rothstein, m. a., & shoben, a. b. (2013). does consent bias research? american journal of bioethics, 13(4), 27–37. https://doi.org/10.1080/15265161.2013.767955 shropshire, j. (2014). extending the cloud with fog: security challenges & opportunities. americas conference on information systems (amcis), 1–10. sidorova, a., evangelopoulos, n., valacich, j. s., & thiagarajan, r. (2008). uncovering the intellectual core of the information systems discipline. management information systems quarterly (misq), 32(3), 467–482. solove, d. j. (2013). introduction: privacy self-management and the consent dilemma. harvard law review, 126(1), 1–24. stevinson, c., & lawlor, d. a. (2004). searching multiple databases for systematic reviews: added value or diminishing returns? complementary therapies in medicine, 12(4), 228–232. https://doi.org/10.1016/j.ctim.2004.09.003 stieglitz, s., & dang-xuan, l. (2013). emotions and information diffusion in social media an investigation of sentiment of microblogs and sharing behavior. an investigation of sentiment of microblogs and sharing behavior. journal of management information systems (jmis), 29(4), 217–248. stieglitz, stefan, dang-xuan, l., bruns, a., & neuberger, c. (2014). social media analytics: an interdisciplinary approach and its implications for information systems. wirtschaftsinformatik, 56(2), 101–109. https://doi.org/10.1007/s11576-014-0407-5 stieglitz, stefan, mirbabaie, m., ross, b., & neuberger, c. (2018). social media analytics – challenges in topic discovery, data collection, and data preparation. international journal of information management, 39(10), 156–168. https://doi.org/10.1016/j.ijinfomgt.2017.12.002 suler, j. (2004). the online disinhibition effect. cyberpsychology and behavior, 7(3), 321–326. https://doi.org/10.1089/1094931041291295 swirsky, e. s., hoop, j. g., & labott, s. (2014). using social media in research: new ethics for a new meme? american journal of bioethics, 14(10), 60–61. https://doi.org/10.1080/15265161.2014.948302 trottier, d. (2012). social media as surveillance: rethinking visibility in a converging world. routledge. vayena, e., salathé, m., madoff, l. c., & brownstein, j. s. (2015). ethical challenges of big data in public health. plos computational biology, 11(2), 1–7. https://doi.org/10.1371/journal.pcbi.1003904 vitak, j., proferes, n., shilton, k., & ashktorab, z. (2017). ethics regulation in social computing research: examining the role of institutional review boards. journal of empirical research on human research ethics, 12(5), 372–382. https://doi.org/10.1177/1556264617725200 vitak, j., shilton, k., & ashktorab, z. (2016). beyond the belmont principles: ethical challenges, practices, and beliefs in the online data research community. acm conference on computer supported cooperative work (cscw), 941–953. https://doi.org/10.1145/2818048.2820078 vom brocke, j., simons, a., riemer, k., niehaves, b., plattfaut, r., & cleven, a. (2015). standing on the shoulders of giants: challenges and recommendations of literature search in information systems research. communications of the association for information systems, 37, 205–224. https://doi.org/10.17705/1cais.03709 yeung, d. (2018). social media as a catalyst for policy action and social change for health and australasian journal of information systems marx & mirbabaie 2022, vol 26, research on applied ethics the investigator’s dilemma 20 well-being: viewpoint. journal of medical internet research, 20(3), 1–12. https://doi.org/10.2196/jmir.8508 zafeiropoulou, s., sarker, s., & carlsson, s. a. (2015). what’s trending in social media analytics area? a retrospective. americas conference on information systems (amcis), 1– 15. zeng, d., chen, h., lusch, r., & li, s. h. (2010). social media analytics and intelligence. ieee intelligent systems, 25(6), 13–16. https://doi.org/10.1109/mis.2010.151 zimmer, m. (2010). “but the data is already public”: on the ethics of research in facebook. ethics and information technology, 12(1), 313–325. https://doi.org/10.1007/s10676-010-92275 zimmer, m., & proferes, n. j. (2014). a topology of twitter research: disciplines, methods, and ethics. aslib journal of information management, 66(3), 250–261. https://doi.org/10.1108/ajim-09-2013-0083 copyright: © 2022 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v26i0.3287 http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 scoping review methodology 2.1 research questions 2.2 data sources and search criteria 2.3 eligibility assessment 3 results 3.1 search results and selection of relevant articles 3.2 ethical principles of sma research considered in is 4 discussion 4.1 towards a shared code of ethical practice in sma research 4.2 a research agenda for the information systems discipline 4.3 limitations 5 conclusion paper title (paper title style) australasian journal of information systems wang 2020, vol 24, selected papers from the 2019 aice conference three harms of gendered technology 1 the three harms of gendered technology lena wang university of sydney australia lenayiwang@gmail.com abstract marginalised groups experience both immediate and long-term detriment as a result of innovations in information systems. this paper explores three facets of technologically related gendered harm: physical, institutional, and psychological. these harms will be demonstrated by case studies. firstly, technology can cause physical harm by denying women their bodily autonomy, demonstrated by the public availability of ai software that generates nude pictures of women, and smart home devices used in instances of domestic abuse. secondly, technology can deny women institutional access, as increasingly widespread algorithms are shown to underperform on marginalised groups. thirdly, anthropomorphised technology reflects and entrenches harmful stereotypes of women’s submissiveness, causing psychological harm. reducing harm must go beyond ensuring a diversity of representation in stem fields. we conclude that effective regulation should focus on the design features in technological innovations. keywords: stem; diversity; technology; gender; women; sexism; machine learning: ai; bias 1 introduction the lack of diversity in stem fields has been widely publicised (west, s. et al., 2019; baranyai et al., 2016). it is uncontroversially recognised that technology is a male-dominated field. for example, in the burgeoning field of artificial intelligence (ai), only 10% of researchers at google and 15% at facebook are women (west, s. et al., 2019). the 2019 australian federal budget allocates $3.4 million in funding to support women in stem, recognising that only 16% of the stem qualified population are women (baranyai et al., 2016). the innovations that these fields produce are in response to the needs perceived by those that work in the field (rakow, 1988; rothschild, 1981). therefore, the development of information systems is not neutral—it embodies the values that caused its production (perry and greber, 1990; poulsen and burmeister, 2019). as such, some take it that the values and corresponding identities of engineers behind the technology may influence the features of the technology created (barratt, 2018; williams, 2014). the australian government cites, as motivation to its women in stem funding, that gender inequality is “bad for business” (“advancing women in stem”, 2019), an oft-repeated adage grounded in the belief that greater female representation in the industry leads to products that are better tailored to women, widening the target market and increasing sales (patterson, 2018; barratt, 2018; williams, 2014). the mainstream view, therefore, is that a lack of diversity creates biased, gendered technology, while a more representative industry will create unbiased and therefore better technology. however, this view, in accordance with a historically liberal feminist perspective, “does not ask the broader questions of whether and in what way technoscience and its institutions [can] be reshaped to accommodate women” (wajcman, 2007). it neglects to consider how gendered technology prevents women from entering stem fields, and causes them harm in general. to australasian journal of information systems wang 2020, vol 24, selected papers from the 2019 aice conference three harms of gendered technology 2 address this, we draw from socialist feminist frameworks that examine how gender relations are embedded in information systems, and therefore entrench gender hierarchies (wajcman, 2007). the problem is cyclical: the fewer women there are in stem, the more biased the technologies the field creates. the more biased technologies exist, the more severe the harm on women, and so prevent their representation in stem. this paper uses case studies to focus on the latter, and examines how gendered technology causes further harm to women, framed by three facets of harm. we submit that ensuring the production of unprejudiced technology goes beyond ensuring mere representation—it requires regulation. our analysis will proceed in the following four steps. first, we will give an overview of the three harms of gendered technology: physical, institutional, and psychological. next, we will provide cases of how technological development has exacerbated harms against women for each of the three facets. third, we examine the flaws of focusing on diverse representation. lastly, we argue that in light of gendered technology, regulatory and public attention ought not to only focus on who is making the technology, but what is being made. throughout this paper, we will take a consequentialist stance on ethics by considering the outcomes of technological design. we note that while there may be instances of biased, gendered technology that have beneficial consequences for vulnerable groups, we restrict our analysis to the gendered technology that has harmful consequences. therefore, under this definition, gendered technology is unethical, as it causes or exacerbates harm against a particular group. the production of gendered technology is therefore unethical, whether it is done intentionally or unintentionally, and those managing production, including legislators with influence on technological development, have an ethical responsibility to prevent harms to anyone, but in particular, harms to groups that have increased vulnerability due to their existing social marginalisation. 2 identifying the three harms generally, gendered oppression manifests not only in terms of lack of institutional representation, but also the pay gap, the increased burden of domestic and reproductive labour, gendered violence and assault, epistemic doubt and stereotype normalisation, and a lack of mobility and autonomy. this paper identifies three interconnected facets of this oppression, that we will call the three harms of gendered technology: 1. physical harm: the denial of bodily autonomy including lack of prevention and exacerbation of domestic violence and sexual assault (aihw, 2019), and restrictions to abortion, 2. institutional harm: the restriction on the mobility and freedoms of women due to lack of access to institutional benefits, resulting in poorer education, the pay gap, and domestic and reproductive burdens, 3. psychological harm: the perpetuation and internalisation of gender stereotypes that cause epistemic doubt and further entrench gendered oppression. these issues are interconnected, for example reducing stereotypes could consequently mean greater access to institutions and monetary compensation that could reduce women’s reliance on potentially abusive partners. simultaneously, safer women are more mobile and therefore able to study productively, changing stereotypes of their social roles. harms can also be australasian journal of information systems wang 2020, vol 24, selected papers from the 2019 aice conference three harms of gendered technology 3 perpetuated concurrently—domestic violence does not merely affect women physically, but psychologically. it is also relevant to note that while women are the direct targets of these harms, harm also occurs to people close to them, and also the perpetrators themselves. ethical considerations surrounding domestic violence need to account for not only women, but consequently, children. similarly, restricting abortion results in some children living in untenable situations. the perpetuation of stereotypes not only affects women, who come to view themselves through a distorted social lens, but in general, anyone participating in that stereotype internalisation, as their perception of the world becomes limited and, in some instances, unfairly prejudicial. medina (2013) describes this epistemic oppression as deficits that affect perpetrators’ ability to learn and contribute to the world. indeed, this is how prejudiced technology arises from prejudiced production. we also note that while this paper focuses on gendered harm to women in general, it is significant that this harm will not be felt equally by all women. an intersectional response, as detailed by black feminists like crenshaw (1989) will be necessary, as women with fewer socioeconomic resources, or who occupy another identity, such as a marginalised racial or sexuality group, can potentially suffer greater consequences from the same harms. stereotypes of race, for example, will affect how women are treated when they require health care as a result of domestic violence (rodriguez et al., 2009). technological harms are also not merely gendered, but can also affect people of different sexualities, gender expressions, race, class, and physical and mental abilities. for example, an ai algorithm has been developed to detect a person’s sexual orientation (wang and kosinski, 2017) which has privacy implications. though these cases will remain outside the scope of this paper, we note that gendered harm is only one aspect of many potential harms of unethical, irresponsible technological development. the three harms we have identified are general; they arise from complex, structural social issues. for the purposes of this paper, however, we will focus on technological cases that cause and perpetuate these harms. we will show that the lack of regulation in technological development, combined with the increasing pace and use of technology, means that women are vulnerable to these three facets of harm. 3 cases of gendered technology 3.1 physical harm: denial of bodily autonomy in december 2018, the australian federal government passed encryption laws that allow police and security agencies access to encrypted messages without user consent (bbc news, 2018). however, the mainstream focus on how technological tools can infringe on general public privacy fails to consider gendered power disparity, and how these tools are especially harmful towards women’s privacy and bodily autonomy. the first case focuses on an application called deepnude, made available online in june 2019. given clothed pictures of women, it used neural networks to generate realistic images of said women naked (cole et al., 2019). the problems with this application go beyond its assumptions of gender essentialism. the fact that these images are computer-generated does not detract from the effects of their publication. firstly, it humiliates and objectifies the depicted women without their consent. secondly, it supplements the phenomenon of revenge australasian journal of information systems wang 2020, vol 24, selected papers from the 2019 aice conference three harms of gendered technology 4 porn, in which women find themselves subject to loss of relationships and careers as a result of photos and videos released online. third, it increases the potential of targeted abuse towards the subject, both online and physically. the consequences of deepnude render it a deeply unethical product. further, deepnude did not generate male nudes, as its requirement of a large training dataset meant, given the much higher proportion of female nudes online, it could only be trained on pictures of women. this is not to say that if it did generate male nudes, it would be ethical. rather, it demonstrates that unethical technology like deepnude, while problematic regardless of which genders it works on, is especially a concern for women as it intersects with and exacerbates the existing objectification of women. it underscores the cyclical nature of oppression: objectification of women means more nude pictures of women online, allowing for the construction of technology that creates more nude pictures that entrench this objectification. the second case focuses on the popularisation of the internet of things, which have led to the proliferation of smart home devices. significantly, women and girls are 25% less likely than men to know how to leverage digital technologies and four times less likely to know how to program computers (west, s. et al., 2019). this technological education gap intersects with domestic violence such that, as a consequence, abusers are using smart home devices to surveil and psychologically harass women. a new york times investigation in 2018 found an increasing number of domestic abuse cases incorporating technology (bowles, 2018)—devices like wi-fi enabled doors, smart thermostats, and digital doorbells. though these devices are pitched as helpful tools to optimise domestic life, the consequences of its misuse have not been adequately addressed. abusers would remotely control and change the temperature, the music playing on the speakers, or the locks on the door, so that women at home “felt as if they were going crazy” (bowles, 2018). similarly, surveillance footage from indoor cameras was circulated as revenge porn. the investigation found that many victims of this type of abuse were women—in this case, the gendered nature of domestic violence is exacerbated by the gendered understanding of technology to exacerbate harm against women. the investigation also highlighted an insidious aspect of gendered technological harm— unlike other types of harm, technology is developed at such a fast pace that legal regulation often falls behind. restraining orders, for example, can neglect to include smart home devices. this underscores the importance of developing both forward-thinking technology and regulation. 3.2 institutional harm: restricted access an issue more widely talked about is women’s institutional access to education, jobs, and how workplaces support domestic and reproductive equality. technology will inevitably affect how women access these institutions: as workplaces become more digitalised, technological literacy becomes more important. from the data cited in the introduction, women and girls tend to be less digitally literate (west, s. et al., 2019). education therefore becomes vital in ensuring women can develop the necessary skills to achieve employment, and therefore financial independence. further, studies have shown some ai algorithms to be less effective for more vulnerable populations when their training sets neglect marginalised groups. buolamwini et al. (2018) demonstrated how some facial analysis algorithms have 99% accuracy when identifying white men, and only 65% accuracy when identifying darker-skinned women. google’s search australasian journal of information systems wang 2020, vol 24, selected papers from the 2019 aice conference three harms of gendered technology 5 algorithm has misidentified darker-skinned women as gorillas (simonite, 2018). these algorithms are increasingly embedded into institutions such that should they ever be used to screen for institutional access, such as screening job candidates, they will perpetuate and potentially worsen existing social hierarchies. 3.3 psychological harm: stereotype normalisation the intrinsic purpose of technology is to aid and serve. and historically, women have done the same as child bearers, objects of sexual gratification, and homemakers. these gender roles have been entrenched in a self-perpetuating loop, as media and products both reflect social trends and display such roles, further entrenching them as the norm. this shared social conception causes psychological harm for women, as they internalise their positions and worth in society.as an example, women have constantly reported feeling the imposter syndrome (clance and imes, 1978). as those in positions of power also internalise such perceptions, they create direct, long-term harms, giving rise to physical and institutional harms. technological progress plays a role in the normalisation of gender roles—to demonstrate this harm, we will use examples of anthropomorphised technology. a majority of popular voice assistants: apple’s siri, amazon’s alexa, microsoft’s cortana, the google assistant, etc. all have female sounding voices, while simultaneously serving as unquestioning helpers. feminine anthropomorphisation was a deliberate choice—amazon’s market research indicated it would be perceived as more “sympathetic” and helpful (schwar and moynihan, 2020). a 2019 study on this topic is titled “i’d blush if i could”, a response given by siri when a user uses gendered slurs such as “you’re a bitch” or “you’re a slut” (west, m. et al., 2019). this normalises women’s submission to gendered abuse—helpful voice assistants are not programmed to retort. the study found alexa would even thank the user for gendered abuse. siri would only tell the user to stop after the user uses gendered slurs eight times in a row—demonstrating that while the developers knew verbal harassment was harmful, once was not enough to warrant a response. further, the market for robotic companions is largely dominated by those anthropomorphised as women—these devices allow for sexual gratification and even violence without deviating from their programmed submissiveness and without requiring consent. hence, they normalise and potentially promote the objectification of women (richardson, 2016). in australia, one in six women have experienced physical or sexual violence from current or previous cohabiting partners, and one woman is killed every nine days by a partner (aihw, 2019). these statistics do not arbitrarily arise; they are a result of ingrained attitudes towards women, attitudes that are reflected and perpetuated by the technologies identified above. in the long-term, entrenched gender roles in technology and the psychological harms they entail will normalise and hence potentially cause violence against women, intersecting with the physical and institutional harms we identified above. the ethical implications of technological development must therefore be considered with these consequences in mind. 4 the flaws of mere representation characterising representation as the primary factor determining the kinds of technology that are made paints an incomplete and inaccurate picture of technological development. we will highlight two factors it neglects: internalised norms, in particular, internalised misogyny, and australasian journal of information systems wang 2020, vol 24, selected papers from the 2019 aice conference three harms of gendered technology 6 external market forces. hence there is a need for technological regulation, which better accounts for these neglected factors. internalised misogyny is a complex topic, one which we will only briefly outline. we take it to be a form of what fricker (2007) describes as an epistemic injustice: a wrong perpetrated to someone in their capacity as “a knower”. that is, gender stereotypes are so ingrained in our shared collective understanding so that, even if someone identifies as a woman, and believes herself to be a feminist, she can unwittingly share prejudices against women. internalised misogyny manifests in a number of ways, including self-objectification (mckinley and hyde, 1996) and passive acceptance of stereotypes (bargad and hyde, 1991). the prevalence of internalised misogyny is such that it would be untenable to assert a causal link between diverse representation within the technology industry and the creation of unprejudiced technology. similarly, technology is not merely a product of its engineers; it arises out of the socio-political context and profit motivation that influence its design, development and deployment. as a brief demonstrative example, technology company palantir continues to collaborate with united states immigration and customs enforcement despite employee pushback (macmillan and dwoskin, 2019). as such, the identities of the engineers do not guarantee a product in line with their beliefs. because of this, we must look beyond mere representation to examine how to rectify the problems caused by gendered information systems. 5 regulation the current focus of gender equity in technology is on increasing the participation of women in the field. the rationale for the focus is the hope that more diverse engineers will create more diverse datasets and use their lived experiences to create technologies that cater to their needs, not just the needs of the social default: white, heterosexual, male. but this is just a hope; as we described earlier, given how embedded gender roles are in society, women can just as easily suffer from internalised misogyny and therefore make products that harm women, just as men do. regulation, therefore, needs to take into account ethical consequences—not just who makes the technology, but what the technology will do. it is recognised that changing the former can change the latter, but there is not enough recognition that changing the latter—the effects of information systems—will also change the former. more socially responsible technology will avoid entrenching gendered stereotypes and prevent the harm to women that reduces their institutional access to stem fields. we cannot expect stem to diversify if we do not create the conditions for it, not just by increasing access to education, but also by ensuring women are safe from abusive partners and revenge porn. regulation needs to occur throughout the technological production process, from its inception to regulating its effects. we will make some positive assertions as to what this regulation would encompass, but acknowledge that the details of its implementations are outside the scope of this paper. this regulation ought to redefine abuse so as to include abuse perpetrated through digital mediums, such as through smart home devices. it ought to focus on improving digital literacy for girls, and other institutional mandates, such as anti-bias training. it ought to regulate when algorithms can be used to screen for institutional access. it should recognise the cultural role of anthropomorphised technology, and create guidelines for the appropriate responses to digital abuse—just as online forums are moderated for hate speech, voice assistants could australasian journal of information systems wang 2020, vol 24, selected papers from the 2019 aice conference three harms of gendered technology 7 perhaps respond with statements or educational resources against abuse, rather than obsequiousness. technological design is an important site for socially motivated intervention (layne et al., 2010). indeed, if technology is normatively a tool that serves humans, that means it ought to serve its most marginalised groups—it should not exacerbate harms against them. references australia passes encryption-breaking laws. (2018, december 7). retrieved from https://www.bbc.com/news/world-australia-46463029 australian institute of health and welfare. (2019). family, domestic and sexual violence in australia: continuing the national story (no. 3). canberra: aihw. baranyai, k., bowles, j., hassan, s., prinsley, r., smith, p., & walter, c. (2016). australia’s stem workforce (p. 222). retrieved from office of the chief scientist website: https://www.chiefscientist.gov.au/wp-content/uploads/australias-stemworkforce_full-report.pdf bargad, a., & hyde, j. s. (2016). women’s studies: a study of feminist identity development in women: psychology of women quarterly. retrieved from http://journals.sagepub.com/doi/10.1111/j.1471-6402.1991.tb00791.x barratt, b. (2018). the need for more women in stem roles goes beyond simple diversity. forbes. retrieved from https://www.forbes.com/sites/biancabarratt/2018/11/17/the-needfor-more-women-in-stem-roles-goes-beyond-simple-diversity/#2456c3845cbb bowles, n. (2018, june 23). thermostats, locks and lights: digital tools of domestic abuse. the new york times. retrieved from https://www.nytimes.com/2018/06/23/technology/smart-home-devices-domesticabuse.html buolamwini, j., & gebru, t. (2018). gender shades: intersectional accuracy disparities in commercial gender classification. conference on fairness, accountability and transparency, 81, 77–91. retrieved from http://proceedings.mlr.press/v81/buolamwini18a.html clance, p. r., & imes, s. a. (1978). the imposter phenomenon in high achieving women: dynamics and therapeutic intervention. psychotherapy: theory, research & practice, 15(3), 241–247. https://doi.org/10.1037/h0086006 cole, s., maiberg, e., & koebler, j. (2019, june 26). this horrifying app undresses a photo of any woman with a single click. vice. retrieved from https://www.vice.com/en_us/article/kzm59x/deepnude-app-creates-fake-nudes-of-anywoman crenshaw, k. (2015). demarginalizing the intersection of race and sex: a black feminist critique of antidiscrimination doctrine, feminist theory and antiracist politics. university of chicago legal forum, 1989(1). retrieved from https://chicagounbound.uchicago.edu/uclf/vol1989/iss1/8 department of industry, innovation, and science. (2019). advancing women in stem. commonwealth of australia. australasian journal of information systems wang 2020, vol 24, selected papers from the 2019 aice conference three harms of gendered technology 8 fricker, m. (2007). epistemic injustice: power and the ethics of knowing. in epistemic injustice. retrieved from http://www.oxfordscholarship.com/view/10.1093/acprof:oso/9780198237907.001.0001/a cprof-9780198237907 layne, l., sharra l. vostral, & kate boyer (eds.). (2010). feminist technology. university of illinois press. macmillan, d., & dwoskin, e. (2019, august 22). the war inside palantir: data-mining firm’s ties to ice under attack by employees. washington post. retrieved from https://www.washingtonpost.com/business/2019/08/22/war-inside-palantir-datamining-firms-ties-ice-under-attack-by-employees/ mckinley, n. m., & hyde, j. s. (1996). the objectified body consciousness scale: development and validation. psychology of women quarterly. retrieved from http://journals.sagepub.com/doi/10.1111/j.1471-6402.1996.tb00467.x medina, j. (2013). the epistemology of resistance: gender and racial oppression, epistemic injustice, and the social imagination. in the epistemology of resistance. retrieved from http://www.oxfordscholarship.com/view/10.1093/acprof:oso/9780199929023.001.0001/a cprof-9780199929023 patterson, d. (2018). why having more women in tech management is good for business. tech republic. retrieved from https://www.techrepublic.com/article/why-having-morewomen-in-tech-management-is-good-for-business/ perry, r., & greber, l. (1990). women and computers: an introduction. signs, 16(1), 74–101. poulsen, a., & burmeister, o. k. (2019). overcoming carer shortages with care robots: dynamic value trade-offs in run-time. australasian journal of information systems, 23. https://doi.org/10.3127/ajis.v23i0.1688 rakow, l. f. (1988). gendered technology, gendered practice. critical studies in mass communication, 5(1), 57–70. https://doi.org/10.1080/15295038809366685 richardson, k. (2016). sex robot matters: slavery, the prostituted, and the rights of machines. ieee technology and society magazine, 35(2), 46–53. https://doi.org/10.1109/mts.2016.2554421 rodríguez, m., valentine, j. m., son, j. b., & muhammad, m. (2009). intimate partner violence and barriers to mental health care for ethnically diverse populations of women: trauma, violence, & abuse. https://doi.org/10.1177/1524838009339756 rothschild, j. a. (1981). a feminist perspective on technology and the future. women’s studies international quarterly, 4(1), 65–74. https://doi.org/10.1016/s0148-0685(81)96373-9 schwar, h., & moynihan, r. (2020, april 5). companies like amazon may give devices like alexa female voices to make them seem “caring.” business insider. retrieved from https://www.businessinsider.com/theres-psychological-reason-why-amazon-gavealexa-a-female-voice-2018-9?r=au&ir=t simonite, t. (2018, november). when it comes to gorillas, google photos remains blind. wired magazine. australasian journal of information systems wang 2020, vol 24, selected papers from the 2019 aice conference three harms of gendered technology 9 wajcman, j. (2007). from women and technology to gendered technoscience. information, communication & society, 10(3), 287–298. https://doi.org/10.1080/13691180701409770 wang, y., & kosinski, m. (2017). deep neural networks are more accurate than humans at detecting sexual orientation from facial images. journal of personality and social psychology, 114(2), 246-257. west, m., kraut, r., & chew, h. e. (2019). i’d blush if i could: closing gender divides in digital skills through education. retrieved from unesco website: https://unesdoc.unesco.org/ark:/48223/pf0000367416.page=1 west, s. m., whittaker, m., & crawford, k. (2019). discriminating systems: gender, race, and power in ai. retrieved from ai now institute website: https://ainowinstitute.org/discriminatingsystems.pdf williams, s. (2014). why we need women in stem. inc. retrieved from https://www.inc.com/sue-williams/why-we-need-women-in-stem.html copyright: © 2020 wang. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v24i0.2799 http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems venable, vom brocke & winter 2019, vol 23, selected papers from acis 2017 designing trids 1 designing trids: treatments for risks in design science john r venable, curtin university, australia j.venable@curtin.edu.au jan vom brocke, university of liechtenstein robert winter, university of st. gallen, switzerland abstract design science research (dsr) has many risks. researchers inexperienced in dsr, especially early career researchers (ecrs) and research students (e.g. phd students) risk inefficient projects (with delays, rework, etc.) at best and research project failure at worst if they do not manage and treat dsr risks in a proactive manner. the dsr literature, such as the risk management framework for design science research (rmf4dsr), provides advice for identifying risks, but provides few suggestions for specific treatments for the kinds of risks that potentially plague dsr. this paper describes the development of a new purposeful artefact (trids: treatments for risks in design science) to address this lack of suggestions for treatment of dsr risks. the paper describes how the purposeful artefact was developed (following a dsr methodology), what literature it draws upon to inspire its various components, the functional requirements identified for trids, and how trids is structured and why. the paper also documents the trids purposeful artefact in detail, including four main components: (1) an extended set of risk checklists (extended from rmf4dsr), (2) a set of 47 specific suggestions for treating known risks in dsr, (3) a classification of the treatments identified into 14 different categories, and (4) a look-up table for identifying candidate treatments based on a risk in the extended risk checklists. the treatment suggestions and guidance in trids serve as a supplement to rmf4dsr by helping dsr researchers to identify treatments appropriate for a particular dsr project (or program) and thereby to improve dsr project efficiency and the probability of dsr project success. 1 introduction there has been much written and published about research methods for design science research (dsr), especially in the field of information systems, since the publication of hevner et al. (2004). methods for dsr include those by vaishnavi and kuechler (2004, 2015), peffers et al. (2008), hevner and chatterjee (2010), sein et al. (2011), bilandzic and venable (2011), and pries-heje et al. (2014b). managing risks is an important area for any kind of project, including dsr projects. according to pries-heje et al. (2014a, p. 61), “risk management is the identification of and response to potential problems with sufficient lead time to avoid a crisis. thus risk management is proactive.” design science researchers need to manage dsr project risks proactively, lest they incur substantial, possibly severe, costs. unless they do so, at best, they risk conducting dsr projects inefficiently (with higher costs and delays for rework or following poor design decisions longer than necessary). at worst, they risk dsr project failure, with a complete waste australasian journal of information systems venable, vom brocke & winter 2019, vol 23, selected papers from acis 2017 designing trids 2 of time and other resources and potentially the loss of reputation, career opportunities, etc. inexperienced dsr researchers, especially early career researchers (ecrs) and research students (e.g. honours, masters, or phd students) are especially vulnerable to dsr risks because they have little experience with research and little exposure to the breadth of the dsr literature and their careers are impacted by project results more directly. to be successful, they need timely and comprehensive guidance for dsr risk identification and treatment. the dsr literature does provide some pockets of advice on managing risks in dsr. for example, the seven guidelines for dsr research included in hevner et al. (2004) identify key areas of dsr work for which the quality should be managed. other dsr literature addresses management of dsr risks through early (“formative” or “ex ante”) evaluation (baskerville et al., 2008; sonnenberg & vom brocke, 2012a, 2012b; venable et al., 2012, 2016) – as opposed to solely relying on a final (“summative” or “ex post”) evaluation, which is conducted following completion of the design and implementation of the purposeful artefact, when there is little or no time or other resources for correcting any inadequacies in the artefact and or its performance in meeting its objectives. the most substantial contribution to risk management in the dsr literature is that of baskerville, pries-heje, and venable (baskerville et al., 2011; pries-heje et al., 2008; pries-heje et al., 2014a), who specifically address dsr risk identification, assessment, prioritisation, and management. however, none of this literature identifies more than a very few specific treatments for preventing, overcoming, or alleviating specific dsr risks. there are many risks to both the efficiency and the success of dsr projects, e.g. as identified by pries-heje et al. (2014a). however, researchers (especially novices, such as ecrs and research students) lack important advice and suggestions for how to treat risks to dsr, so that they can manage those risks in a proactive and effective way. supervisors of dsr researchers could also benefit from a more comprehensive treatment of risk management in dsr. funding bodies and industry partners engaged in dsr projects also have a major interest in identifying and treating risks in collaborative research projects (vom brocke & lippe, 2010). this paper reports on the development of trids (treatments for risks in design science), a purposeful artefact developed by the authors to overcome the above problems. trids suggests specific treatments for every specific dsr risk previously identified in the extensive dsr risk checklists by pries-heje et al. (2014a), as well as some other dsr risks identified in this paper. risks that are not specific to dsr (e.g. general project risks such as missing deadlines or scoping failures) are not addressed in this paper. the rest of this paper follows the organisation suggested by gregor and hevner (gregor & hevner, 2013) for a publication reporting dsr research. section 2 expands on the literature identified above and details what is already in the literature about risk treatment in dsr, as well as literature from other domains that inspired our identification of various other risk treatments. section 3 describes the research method (design science research process model or dsrpm (vaishnavi & kuechler, 2004, 2015)) used to develop trids. sections 4 describes the trids purposeful artefact, including requirements for trids and the details of the trids artefact and its components. section 5 describes how trids was evaluated and the outcomes of that evaluation. section 6 discusses the findings of the research. finally, section 7 draws conclusions about trids and the potential for further research. australasian journal of information systems venable, vom brocke & winter 2019, vol 23, selected papers from acis 2017 designing trids 3 2 literature review this first part of this literature review covers the dsr literature relating to the problem to be addressed and develops the research gap, i.e. the lack of specific suggestions for how to treat risks encountered in dsr. the second part identifies and reviews other literature that inspired some of the specific treatments that are incorporated into trids. 2.1 dsr literature relevant to dsr risk management there are three main areas of the dsr literature that are relevant to dsr risk management: (1) literature on guidelines for dsr, (2) literature on evaluation as a key activity in dsr, and (3) literature specifically about risk management in dsr. these are covered in sections 2.1.1-2.1.3 below. 2.1.1 dsr guidelines the dsr literature provides some guidelines for what makes quality or acceptable dsr. hevner et al. (2004) provide seven guidelines for dsr (see below). 1. design as an artefact 2. problem relevance 3. design evaluation 4. research contributions 5. research rigour 6. design as a search process 7. communication of research venable (2010, 2015) surveyed dsr authors, desrist program committee members, and is basket of eight journal editors concerning their opinions of what criteria and standards are more important in dsr. the findings were that hevner et al. (2004) guidelines 2, 4, 1, and 3 (in that order) were most important, while other highly rated criteria aligned with the guidelines (e.g. relevance of problem and significance for problem clearly established). also in the top 10 most important criteria were “clear understanding of why the artefact works (or doesn’t work)” and “depth of analysis and clarity of understanding of the problem and its causes.” it is a sensible approach to dsr risk management to understand key dsr guidelines and take action to ensure that any relevant (or at least that the most important) guidelines are met. 2.1.2 dsr evaluation literature baskerville, pries-heje, and venable have also published several papers suggesting that dsr risks of poor usability or of technical or organisational feasibility might be treated through multiple, ex ante (or formative) evaluations of prototypes or partial artefacts (venable et al., 2012, 2016). one purpose identified for evaluation in dsr is to “evaluate a designed artefact formatively to identify weaknesses and areas of improvement for an artefact under development” (venable et al., 2012, p. 426). they also identify that artificial, ex ante (formative) evaluation has the lowest risk to participants in the research, but naturalistic, ex post evaluation has the lowest risk of a false positive, which is especially important for safety-critical systems (venable et al., 2012, p. 432, figure 2). australasian journal of information systems venable, vom brocke & winter 2019, vol 23, selected papers from acis 2017 designing trids 4 in their subsequent paper on feds (framework for evaluation in design science, venable et al., 2016), they suggest that different evaluation strategies (typical trajectories of evaluation episodes) can be used to reduce risks of technical or organisational feasibility or of insufficient evaluation rigour with respect to effectiveness or efficacy. technical feasibility and/or rigorous evaluation of efficacy are better achieved with a technical risk & efficacy evaluation strategy, which starts with one or more artificial formative evaluation episode(s), transitions to one or more artificial summative evaluation episode(s), and concludes with one or more naturalistic summative evaluation episode(s). organisational feasibility, usability, and/or rigorous evaluation of effectiveness are better achieved with a human risk & effectiveness evaluation strategy, which starts with one or more artificial formative evaluation episode(s), transitions to one or more naturalistic formative evaluation episode(s), and concludes with one or more naturalistic summative evaluation episode(s). sonnenberg and vom brocke (2012b) present a conceptualisation for concurrent evaluation according to different aspects of design. they build on prior work describing dsr activities within the overall dsr process, and they argue that, in each of these activities, different progress towards the intended artefacts is made, which offer the potential for concurrent (or formative) evaluation. such evaluation can mitigate risk, since early feedback on more finegrained steps leading to the eventual artefact can be incorporated into the design process. further, they argue that evaluation can be more specific (compared to summative evaluation), and the information to be gained can be more directed, if the evaluation is focussed on particular relevant aspects of the design at the time that related design decisions are made in the design process. in order to exemplify their approach, they distinguish four evaluation types (eval 1 to eval 4), which are derived from typical dsr activities, and which are characterized by the input and output of each activity as wells as specific evaluation criteria and evaluation methods. • eval 1: evaluating the problem identification: criteria include importance, novelty and feasibility. • eval 2: evaluating the solution design: criteria include simplicity, clarity and consistency. • eval 3: evaluating the solution instantiation: criteria include ease of use, fidelity with real-world phenomena and robustness. • eval 4: evaluating the solution in use: criteria include effectiveness, efficiency and external consistency. stoeckli et al. (2017) examined how is scholars actually apply the above evaluations, as reported in papers citing sonnenberg and vom brocke (2012b). they found that few dsr researchers try concurrent evaluation and, if they do, they focus on early phases and apparently find value in doing so. these early evaluations (eval 1 and eval 2) are in accordance with using formative evaluations (venable et al., 2016) and the principle of “fail early, fail often” (abraham et al., 2014). 2.1.3 dsr risk management literature pries-heje et al. (2014a) provide extensive dsr risks checklists as part of their risk management framework for dsr (rmf4dsr). their risk checklists cover six areas (a through f) of potential risks in dsr: australasian journal of information systems venable, vom brocke & winter 2019, vol 23, selected papers from acis 2017 designing trids 5 a. identifying, selecting, and developing understanding of the business needs to be addressed in the dsr project or program, b. grounding of the dsr in knowledge available in the body of recorded human knowledge c. building design artefacts d. evaluating design artefacts and design theories e. artefact dissemination and use f. adding knowledge to the body of human knowledge in all, pries-heje et al. (2014a) identify 38 different risks specific to dsr and suggest that dsr researchers should use the checklist to identify particular risks relevant to their project as instances of each type of risk on the checklist. checklists a-f below show the risks identified in pries-heje et al. (2014a), grouped by each of the above six categories. checklist a: business needs • a-1 selection of a problem that lacks significance for any stakeholder • a-2 difficulty getting information about the problem and the context • a-3 different and even conflicting stakeholder interests (some of which may not be surfaced) • a-4 poor understanding of the problem to be solved • a-5 solving the wrong problem, i.e. a problem that isn’t a main contributor to undesirable outcomes that motivate the problem solving • a-6 poor/vague definition/statement of problem to be solved, with potential misunderstanding by others • a-7 inappropriate choice or definition of a problem according to a solution at hand • a-8 inappropriate formulation of the problem checklist b: grounding • b-1 ignorance or lack of knowledge of existing research relevant to the problem understanding and over-reliance on personal experience with or imagination of the problem • b-2 ignorance or lack of knowledge of existing design science research into solution technologies for solving the problem, i.e. lack of knowledge of the state of the art • b-3 ignorance or lack of knowledge of existing relevant natural and behavioural science research forming kernel theories for understanding or solving the problem checklist c: build design artefacts • c-1 development of a conjectural (un-instantiated) solution which cannot be instantiated (built or made real), cf. technical risk in venable et al. (2016) • c-2 development of a hypothetical (untried) solution which is ineffective in solving the problem, i.e. the artefact doesn’t work doesn’t work well in real situations with various socio-technical complications, cf. human risk in venable et al. (2016) australasian journal of information systems venable, vom brocke & winter 2019, vol 23, selected papers from acis 2017 designing trids 6 • c-3 development of a hypothetical (untried) solution which is inefficient in solving the problem, i.e. requiring overly high resource costs • c-4 development of a hypothetical (untried) solution which is inefficacious in solving the problem, i.e. the artefact isn’t really the cause of an improvement observed during evaluation • c-5 development of a hypothetical (untried) solution which cannot be taught to or understood by those who are intended to use it, e.g. overly complex or inelegant • c-6 development of a hypothetical (untried) solution which is difficult or impossible to get adopted by those who are intended to use it, whether for personal or political reasons • c-7 development of a hypothetical (untried) solution which causes new problems that make the outcomes of the solution more trouble than the original problem, i.e. there are significant side effects checklist d: evaluate • d-1 tacit requirements (which by definition cannot be surfaced) are not dealt with when evaluating the solution technology, leading to failure of the solution technology to meet those requirements • d-2 failure to surface some or all of the relevant requirements leads to those requirements not being dealt with when evaluating the solution technology, leading to failure of the solution technology to meet those requirements • d-3 incorrectly matching the articulated requirements to the meta-requirements of the information systems design theory (isdt) lead to the testing of the idst and evaluation of an instantiation of the meta-design in a situation for which neither should be applied • d-4 incorrectly matching the meta-design or the design method to the metarequirements (not following the isdt correctly) leads to evaluation of something other than the correct solution technology or the isdt as stated • d-5 improper application of the meta-design or the design method (not in accordance with the isdt) in designing an instantiation leads to evaluation of something other than the correct solution technology or the isdt as stated • d-6 improperly building an instantiation of the solution technology (such that it does not properly embody the meta-design) leads to evaluation of something other than the correct solution technology or the isdt as stated • d-7 difficulties in implementing the solution technology during naturalistic evaluation, due to such things as unforeseen complications within the business/organization, prevent the instantiation of the solution technology from successfully meeting its objectives • d-8 success of the solution technology to meet its objectives is not obtained due to dynamic or changing requirements beyond the scope of the solution technology • d-9 success of the solution technology to meet its objectives is not achieved due to poor change management practices australasian journal of information systems venable, vom brocke & winter 2019, vol 23, selected papers from acis 2017 designing trids 7 • d-10 determination of success or failure in reaching the objectives of the solution technology is error-prone or impossible due to disagreement about objectives or inability to measure • d-11 existing organizational culture, local organizational culture differences (subcultures), political conflicts, etc. complicate the evaluation process or weaken the ability to make meaningful measurement of the achievement of the objectives of the solution technology • d-12 existing organizational priorities, structures, practices, procedures, etc. complicate the evaluation process or ability to make/measure the achievement of the objectives • d-13 emergence of new organizational or individual practices, structures, priorities, norms, culture, or other aspects that complicate the acceptability, workability, or efficiency of the application of the solution technology in a naturalistic setting checklist e: artefact dissemination and use • e-1 implementation in practice of a solution that does not work effectively, efficiently, and/or efficaciously • e-2 misunderstanding the appropriate context for and limitations of the solution • e-3 misunderstanding how to apply the solution • e-4 inappropriate handling of adoption, diffusion, and organizational implementation checklist f: knowledge additions • f-1 inability to publish or present research results • f-2 publication of low significance research • f-3 publication of incorrect research • f-4 design artefacts prove too unique to disseminate pries-heje et al. (2014a) further suggest that, after identifying specific risks inspired by the checklists, dsr researchers should next analyse and prioritise identified risks and only then determine risk treatments, especially for higher priority risks (those risks that are more likely and with more potential of negative impact). to support the identification of treatments, they propose a dsr risk treatment framework, based on the traditional risk management literature, with a generic treatment type in each of four quadrants: self-insure (for low probability, low impact risks), transfer (for low probability, but higher potential impact risks), self-protect (for higher probability, but low impact risks), and avoidance (for higher probability and high potential impact risks) (pries-heje et al., 2014a). these four quadrants are also known respectively as the four t’s: (1) tolerate, (2) transfer, (3) treat, and (4) terminate (hopkin, 2017). however, pries-heje et al. (2014a) do not recommend risk treatments for the different kinds of risks. in fact, they expressly avoid doing so, stating “we could list resolution techniques for each risk item. however, we fear that this may limit the generative process of thinking through what should be done about each risk in the specific context. hence we prefer—and believe it australasian journal of information systems venable, vom brocke & winter 2019, vol 23, selected papers from acis 2017 designing trids 8 to be a better approach—to let the user (dsr researcher) decide the risk treatment within broader categories of treatments.” (pries-heje et al., 2014a, p. 61). nonetheless, in explicating the different classes of risk treatments and in providing a naturalistic case study evaluation of rmf4dsr, they do identify a number of specific risk treatments, such as “use pilots and prototypes so it becomes clear very fast what the contribution could be” and “use many diverse problem identification techniques such as document study, observe sourcing-atwork [sic], interview at many levels, etc.” (pries-heje et al., 2014a, p. 18). however, their examples do not cover treatments for all of the different risks and are sometimes specific to the particular case study in their evaluation of rmf4dsr. 2.1.4 problem to be solved the problem addressed in this paper is the lack of suggestions and guidance for specific treatments for the large number and variety of risks in conducting dsr. as noted in pries-heje et al. (2014a), there are a significant number of risks that are specific to dsr. for those risks, recommended practices from the general research methods and practice literature may not be adequate. moreover, that literature has not specifically addressed those risks unique to dsr. even within the dsr literature in particular, identification of risk treatments is far from comprehensive. many of the risks in dsr have no specific treatments recommended for them in the literature. the very general types of treatments from the general risk management literature incorporated within pries-heje et al. (2014a) are difficult to apply because they are insufficiently specific and rely too much on the ingenuity of the researcher. more guidance and candidate suggestions for risk treatments are needed. 2.2 non-dsr literature relevant to dsr risk management this section identifies and summarises salient aspects of other literature that inspired or informed the design of trids. 2.2.1 general and is risk management there is a large and rich literature on risk management, both generally and in information systems. is risk management is a special case of general risk management. furthermore, we can conceptualise that dsr (project) risk management is a special case of research project risk management, which is a special case of project risk management, which is a special case of general risk management. while we cannot possibly cover all of this literature here, we mention a few examples useful for our purposes here. among many others, hopkin (2017) gives an overview of risk management generally (not specific to any domain), covering areas such as kinds of risks, risk management standards, an eight-step process for risk management (the 8 r’s: (1) recognise, (2) rate, (3) rank, (4) respond (deciding how to respond – tolerate, transfer, treat, or terminate – the four t’s), (5) resource (the decided response), (6) reaction planning, (7) report, and (8) review), and four risk control techniques (treatments), including (1) preventive, (2) corrective, (3) directive, and (4) detective. while rmf4dsr analyses risks to determine which of the 4 t’s applies, it does not suggest different controls (preventive, corrective, directive, and detective). detective treatments identify occurrences of risks, which then need to be followed up with corrective treatments (e.g. redesign) to fix the problem identified. iversen et al. (2004) suggest four different types of approaches to software risk management: (1) a risk list, (2) a risk-action list, (3) a risk-strategy model, and (4) a risk-strategy analysis. the australasian journal of information systems venable, vom brocke & winter 2019, vol 23, selected papers from acis 2017 designing trids 9 approach in rmf4dsr (pries-heje et al., 2014a) is to start with a list of known candidate risks and determine their relevance to form a risk list. after prioritisation, the approach adds risk treatments, to form a risk-action list. importantly, rmf4dsr lacks suggestions for different kinds of treatments, which suggested to the authors that we should add candidate treatments to the candidate risk list, to form a candidate risk-action list. however, neither rmf4dsr nor trids attempt to aggregate risks or treatments and no overall strategy is created. 2.2.2 2.2.2 participation in system development and dsr there is a long and rich literature about participation in system development, including participatory design/development (e.g. the series of participatory design conferences, (pdc, 2018b)), user-centred design/development, co-design, and aspects of design thinking. the key idea is that participation by users/stakeholders in the design of purposeful artefacts (such as is) helps ensure that the design meets the users’ needs and motivates them to adopt and use the resulting purposeful artefact(s). e.g. see (pdc, 2018a). modern agile development methods also include user/customer/client participation as a key concept. e.g., the first value of the agile manifesto (beck et al., 2001) is to value “individuals and interactions over processes and tools”, while the third agile value (of only four) is to value “customer collaboration over contract negotiation.” participation is a key part of several dsr methodologies, including action design research (sein et al., 2011), participatory action design research (bilandzic & venable, 2011), and soft design science methodology (pries-heje et al., 2014b). research on agile software development has shown the importance of participation to deal with change in information systems design (isd) processes. according to conboy (2009) agility is defined as the “the continual readiness of an isd method to rapidly or inherently create change, proactively or reactively embrace change, and learn from change while contributing to perceived customer value (economy, quality, and simplicity), through its collective components and relationships with its environment”. the manifesto for agile software development (fowler & highsmith, 2001) and approaches such as xp and scrum (cf. boehm, 2002; highsmith, 2002; highsmith & cockburn, 2001) make normative statements on how to increase agility in isd, including aspects of participatory design and early feedback on intermediate artefacts. these approaches rejected highly-formalized approaches to working in favour of faster, user-centric, and more dynamic methodologies that enable continuous delivery, requirements change and reflection. agile methods have been successfully applied across diverse contexts and domains (boehm, 2002; gary et al., 2011; lindvall et al., 2002), and agile adoption has been attracting increasing interest in research, also beyond isd (fitzgerald et al., 2006; nerur et al., 2005; wang et al., 2012). 2.2.3 software engineering reviews the software engineering literature describes the practice and benefits of many different kinds of reviews, including design reviews (and software design reviews) (jalote, 2008), code inspections (jalote, 2008), peer reviews (summers, 2011), software inspections (summers, 2011), software engineering reviews (summers, 2011), and formal reviews and audits (summers, 2011). summers (2011) suggests six formal reviews or audits: system requirements review, system design review, software specification review, preliminary design review, critical design review, and joint technical review. this literature suggests that reviewing intermediate artefacts as well as the final artefact is a key way to identify problems so that the artefacts can be improved (and risks are avoided). australasian journal of information systems venable, vom brocke & winter 2019, vol 23, selected papers from acis 2017 designing trids 10 2.2.4 literature reviews there is a very useful body of work on how to accomplish a good literature review (e.g. jennex (2015); vom brocke et al. (2015); webster & watson (2002)). conducting a proper literature review and following the advice in papers such as these can be a very useful approach to reducing research risks, such as those relating to grounding, building and disseminating the artefact, most specifically as identified in rmf4dsr (pries-heje et al., 2014a). 2.2.5 change management several of the risks identified in rmf4dsr (pries-heje et al., 2014a) relate to change management. there is extensive literature on how to manage change well. planning and designing a purposeful artefact to accommodate change or even making change management recommendations could be effective risk management treatments. knowledge of change management approaches and concerns, such as contained in the change management body of knowledge (cmbok) (cmi, 2014) can inspire risk treatments and practices. specifically, risks related to the “evaluation” and “use” of the artefact (pries-heje et al., 2014a) can be mitigated by appropriate dissemination and communication strategies supporting intended users to effectively and efficiently apply the artefact. 2.2.6 problem analysis and formulation there is a very rich literature on problem analysis and formulation. among other areas and approaches, soft systems methodology (checkland, 1981; checkland & holwell, 1999; checkland & scholes, 1990) provides extensive advice and techniques (e.g. rich pictures, catwoe, and root definitions) for analysing problems and obtaining agreement among stakeholders. methods for causal analysis and reasoning include cognitive mapping (ackermann & eden, 2001; eden, 1988; eden & ackermann, 1998, 2001) and its extension to coloured cognitive mapping (venable, 2005) and the application of coloured cognitive mapping to dsr (venable, 2014). also very useful for problem analysis and formulation are approaches to stakeholder analysis, such as the instrumental view in mitchell et al. (1997) and the systems-thinking-based view of ulrich (1987). all of these approaches potentially provide useful ways to reduce risks related to poor requirements analysis and formulation. much of the above literature is also closely related to participation by a variety of stakeholders (see section 2.2.2). in addition to general approaches, tool-based support using shared representations of a problem and/or solution may be useful for creating the common ground needed for joint inquiry (hickman, 1998; hildebrand, 2008) into formulating and solving problems, e.g. through co-design (steen, 2013). avdiji et al. (2018) describe principles for such tools using examples such as the business model canvas and the team alignment canvas. still others have developed canvas tools specifically to support dsr (e.g. morana et al., 2018; nagle & sammon, 2016). having identified a number of potentially useful avenues from the literature, we now consider how those ideas were selected and incorporated into trids. 3 research method research methods and techniques are designed purposeful artefacts. they are designed for the purpose of conducting research efficaciously, effectively, efficiently, and ethically (cf. the five e’s in checkland and scholes (1990)). venable and baskerville (2012) asserted that, as australasian journal of information systems venable, vom brocke & winter 2019, vol 23, selected papers from acis 2017 designing trids 11 designed purposeful artefacts, research methods and techniques should be developed and evaluated using a design science research approach. to provide useful guidance to dsr researchers, especially inexperienced ones, on risk treatment in dsr, this research developed a new purposeful artefact (trids: treatments for risks in design science). trids is an approach for identifying suitable treatments for risks potentially encountered in dsr. therefore, we adopted a dsr approach to this research. our research roughly followed the dsr lifecycle of the design science research process model (dsrpm), developed by vaishnavi and kuechler (2004, 2015). dsrpm has five stages: (1) awareness of problem, (2) suggestion, (3) development, (4) evaluation, and (5) conclusion, as well as the possibility to cycle back to earlier stages. we now summarise each of these five stages and introduce how they were addressed in this research and where they are covered in this paper. 3.1 awareness of problem stage in the awareness of problem stage of dsrpm, we formulated and agreed upon the problem to be solved, in line with section 1 above. in conducting this research we first considered the problem to be solved and then considered requirements for a new purposeful artefact (trids) that could be used to solve the problem. more than just awareness of a problem, it is important to understand the problem to be solved and for researchers to agree on a definition of the problem. such problem formulation can be found in many dsr methodologies, including design science research process model (peffers et al., 2008), action design research (sein et al., 2011), or soft design science methodology (pries-heje et al., 2014b). importantly, the scope decided for trids was not the initial scope for the artefact. when the research began, we considered additionally providing further guidance for risk identification, as well as a lifecycle or methodology for risk management covering all the dsr activities. after some initial design iterations and a clearer idea of the role of rmf4dsr, the research team agreed to focus the scope of the design of trids on identifying appropriate and specific treatments for known dsr risks, not on identifying and prioritising the risks to a particular dsr project, which is already handled well by rmf4dsr. thus, the aim and scope of trids is to serve as a supplement or further step to the support for risk management provided by rmf4dsr. in addition to setting the scope of the problem, we further formulated requirements for a purposeful artefact to solve the problem, as will be described in section 4 on the design of trids. 3.2 suggestion stage the suggestion stage of dsrpm uses abductive reasoning to identify or inspire one or more approaches to solving the problem and making an improvement. in the suggestion phase, we considered the literature of different kinds of treatments for mitigating risks, including literature from dsr risk management, dsr methodologies, and risk management more generally, research methods more generally, software engineering, and system development. an overview of relevant aspects of these was given in section 2. we further reviewed all the known risks and brainstormed ideas for how to mitigate each risk. more details on the suggestion phase are provided in section 4. australasian journal of information systems venable, vom brocke & winter 2019, vol 23, selected papers from acis 2017 designing trids 12 3.3 development stage in the development stage of dsrpm, we conducted analyses, developed ideas, and designed an artefact to include those ideas. first, we reviewed the different risks in the pries-heje et al. (2014a) checklists and brainstormed different treatments for each individual risk, based on our understanding of the research literature and our collective experiences as researchers. we did this risk by risk to ensure that every single dsr risk has at least one treatment. in doing so, we also identified a small number of risks not included in pries-heje et al. (2014a), which we present as part of trids in section 4. finally, we needed to design a way to present the outcomes of our analysis so that they are parsimonious and easily understandable. during this process, we went through a number of design iterations in which we discussed and reviewed the different treatments, as well as how to present them, then revised our artefact design accordingly. details on the artefact design and development are provided in section 4. 3.4 evaluation stage for the evaluation stage of dsrpm, we evaluated our purposeful artefact with a criteria-based evaluation as to whether it meets its requirements, as described further below. further evaluations for usability, utility, etc., are still needed. details on the artefact evaluation are provided in section 5. 3.5 conclusion stage the conclusion stage of dsrpm was reached when the artefact reached a state where we judged that the artefact is sufficiently useful for its purpose to conclude the research, report the outcomes (thus far), and contribute to the on-going discourse on dsr risk management in the dsr research community. we trust that further work will lead to even better artefacts for managing risk in dsr. 4 design of trids (dsrpm stages 1-3) this section describes the design of the purposeful artefact, as suggested by gregor and hevner (2013). to do so, this section follows the structure of the first three stages of the dsrpm methodology (vaishnavi & kuechler, 2004, 2015). section 4.1 completes the awareness of problem stage, which was begun in the introduction to this paper, by specifying requirements for trids. section 4.2 describes the suggestion stage and section 4.3 provides a description of and rationale for the trids artefact, to convey the development stage of dsrpm. 4.1 requirements for trids (dsrpm stage 1) the scope of the problem to be solved – the lack of comprehensive guidance and advice (especially for novice dsr researchers) concerning treatments for known dsr risks – was described and justified in the introduction to this paper. based on that scope, we were able to proceed with developing what became trids. however, in order to solve a problem, it is useful to specify the requirements for solving the problem. setting out the requirements for a solution and designing a solution to meet them enables solving the problem. therefore, we formulated and agreed upon the following five requirements for trids, based on the identified scope of extending rmf4dsr to provide needed guidance to novice dsr researchers on specific treatments for reducing and managing dsr risks. australasian journal of information systems venable, vom brocke & winter 2019, vol 23, selected papers from acis 2017 designing trids 13 1. provide as comprehensive a list of all risks relevant to dsr projects as possible, i.e. possibly extending the checklists of known risks provided in rmf4dsr (pries-heje et al., 2014a). 2. identify treatments for all dsr risks identified (comprehensive coverage of risks), including both those already in rmf4dsr and any other dsr risks identified in point #1 above. for every known risk, at least one potential treatment should be identified. 3. identify a range of different kinds of treatments or treatment strategies. having more than one suggestion for potential treatments of a dsr risk would improve the flexibility and utility of the purposeful artefact. 4. where possible, identify treatments in each of the four risk management quadrants (self-insure, transfer, self-protect, and avoidance) found in rmf4dsr (pries-heje et al., 2014a). 5. support users to easily identify specific candidate treatments for each particular dsr risk. later, during further review of relevant research in seeking potential treatments and in reviewing the general risk management literature and classifications of risk treatments, as noted in section 2.3.1, we identified that rmf4dsr does not address the four kinds of controls suggested by hopkin (2017). this suggests that trids could try to identify treatments to cover the breadth of the four different controls for the self-protect/treat quadrant of controls. this led us to add a six requirement for what became trids: 6. where possible, identify controls of all four types (preventive, corrective, directive, and detective) for use in the self-protect/treat quadrant of assessed risks. 4.2 suggestion phase and solution alternatives (dsrpm stage 2) in the suggestion phase, we next considered how to approach the problem. as introduced in section 3, our first approach was to brainstorm ideas for treatments for each and every dsr risk identified in rmf4dsr. ideas came from many areas in the literature as well as the authors’ dsr experience and through considering each of the four quadrants of risk treatments in rmf4dsr (pries-heje et al., 2014a), i.e. self-insure, transfer, self-protect, and avoidance, as well as the four kinds of controls in hopkin (2017), i.e. preventive, corrective, directive, and detective. many key ideas were drawn from the extant approaches in the literature on dsr guidelines, as discussed in section 2.1.1, the dsr evaluation literature (venable et al. (2012, 2016), sonnenberg & vom brocke, (2012b)), as discussed in section 2.1.2, and in the dsr risk management literature, as discussed in section 2.1.3. the idea of a treatment to manage key dsr guidelines followed directly from the dsr guideline literature. venable et al. (2012, 2016), as well as sonnenberg and vom brocke (2012b), provided the idea that risks can be reduced through early evaluation and good evaluation strategies should consider potential areas of risk. the pries-heje et al. (2014a) paper on rmf4dsr provided a small number of example dsr risk treatments. however, we also drew on several other areas of extant literature as reviewed in section 2.2, including literature on general and is risk management, user participation and participatory development, software engineering (specifically software engineering reviews), how to australasian journal of information systems venable, vom brocke & winter 2019, vol 23, selected papers from acis 2017 designing trids 14 conduct literature reviews, change management, and problem analysis and solving. additionally, we drew on the research and system development knowledge and experience of the authors to identify other candidate treatments. 4.3 design and development of trids (dsrpm stage 3) as described earlier, we developed trids by first brainstorming and documenting all of the different treatments, risk by risk, drawing on rmf4dsr. in doing so, we further identified five other dsr risks not found in rmf4dsr. these were added to the rmf4dsr risk checklists at the appropriate point and relevant treatments identified. once we had lists of treatments for each risk, we further considered how to make identification of candidate treatments easy, once a particular risk was selected to be treated. we also needed to consider how to provide advice to novice researchers on what a specific treatment actually means and how to enact that specific treatment once selected. we noted that a number of specific treatments were similar and some could clearly be applied to a number of different risks. this led us to reflect on the range of different treatments and to group treatments conceptually into different classes. based on that, we are able to provide a description of a group or class of treatments and general advice and literature on how to enact treatments within that class. the resulting purposeful artefact, which we have named treatments for risk in design science (trids), has four main parts. the first part is comprised of six extended dsr risks checklists, which make minor extensions to the checklists in rmf4dsr. table 1 below shows the additional dsr risks we identified. the intention is that the design science researcher will use these extended checklists in the same manner as in rmf4dsr, to identify, analyse, and prioritise dsr risks for which treatments are needed. the second part is a dsr treatment list, which lists and explains the 47 different treatments that we identified. each treatment is a candidate for application for one or more dsr risks. the third part is a dsr risk treatments lookup table. this table matches dsr risks to candidate dsr treatments, but is organised by dsr risk, so a dsr researcher can easily look up potential (candidate) dsr risk treatments for any particular (and relevant, sufficiently high priority dsr risk, as determined using the rmf4dsr procedure. finally, the fourth part is a classification of the different kinds of treatments. this facilitates providing a succinct summary of each class of treatment and references to relevant literature. sections 4.3.1-4.3.4 below describe each of the four components of trids. 4.3.1 trids extended dsr risks checklists as noted earlier, during the development of trids, we identified several dsr risks that were not expressly included in the dsr risk checklists in pries-heje et al. (2014a). table 1 below identifies the additional risks that we have added to the pries-heje et al. (2014a) dsr risk checklists. pries-heje et al. (2014a) divided risks into six different categories (a. business needs, b. grounding, c. build purposeful artefacts, d. evaluate design artefacts and justify design theories or knowledge, e. artefact dissemination and use, f. knowledge additions). the full extended dsr risks checklists (including the original rmf4dsr risks and the additional risks below) are provided in appendix a. australasian journal of information systems venable, vom brocke & winter 2019, vol 23, selected papers from acis 2017 designing trids 15 new risk number additional risk description a-9 inappropriate choice of meta-requirements (scoping error) d-13 type i evaluation error (false positive) (baskerville et al., 2011) d-14 type ii evaluation error (false negative) (baskerville et al., 2011) e-5 unsafe artefact released f-5 unsafe artefact design published table 1. additional risks in the trids extended dsr risk checklists 4.3.2 trids treatment list during our brainstorming and review and revision of different dsr risk treatments, we identified 47 different treatments, which could be used at appropriate points in a dsr project to address the risks in the extended dsr risks checklists described above. table 2 below lists and briefly describes each of the treatments, as well as listing the relevant risks to which each treatment could be applied. treatments in table 2 are listed in the order of the earliest risk to which they apply, e.g. a treatment that applies to risk a-1 is listed before a treatment that doesn’t apply to risk a-1 – and so on. some of the treatments come from or are adapted from treatments in the dsr literature and references are provided where relevant. # detailed risk treatment relevant risks 1. systematic literature review about the problem, existing solutions, and technological capabilities that could be used for solution a-1, a-2, a-3, a-4, a-5, a-6, a-7, a-8, b-1, b-2, b-3 2. empirical investigation of the problem (e.g. using survey, case study), cf. pries-heje et al. (2014a), p. 15 a-1, a-2, a-3, a-4, a-5, a-6, a-7, a-8, d-1b 3. seek co-authors or clients with expertise in and understanding of the problem area and its significance a-1, a-2, a-3, a-4, a-5, a-6, a-7, a-8, b-1, b-2, b-3, c-5, c-6, c-7, d-1a 4. stakeholder analysis a-3, a-4, a-5, a-6, a-7, a-8 5. causal analysis (a.k.a. root cause analysis) a-3, a-4, a-5, a-9 6. catwoe/root definition a-6, a-7, a-8, a-9 7. requirements choice review a-9 8. link requirements to desired outcomes (causal analysis) a-9 9. update literature review, open automatic query b-1, b-2, b-3 10. seek co-authors or clients with expertise in extant purposeful artefact solutions to the problem b-2, c-4, c-5, c-6, c-7 11. seek co-authors or clients with expertise in technologies to be applied in new purposeful artefact b-2, c-1, c-3 12. seek co-authors with expertise in behavioural theory or other areas of potential kernel theory b-3 13. review solution idea and design with technical experts c-1 14. generate multiple candidate designs and contingency plans, cf. pries-heje et al. (2014a), p. 14 c-1, c-2, c-3, c-4, c-5, c-6, c-7, d-1a, d-1b, e-1 15. evaluate early and formatively, cf. pries-heje et a. (2014a, p. 14), abraham et al. (2014) c-1, c-2, c-3, c-4, c-5, c-6, c-7 16. review solution idea and design with potential users, cf. pries-heje et al. (2014a), p. 15 c-2, c-3, c-4, c-6, c-7, d-1a, d-1b 17. review partial prototypes as early as possible with users, cf. pries-heje et al. (2014a), 15 c-5, d-1a, d-1b 18. review solution idea and design with non-user stakeholders, especially with power and different interests c-6, c-7 19. ask “devil’s advocate” question: “how and why could use of the artefact make a situation worse rather than better?” c-7 australasian journal of information systems venable, vom brocke & winter 2019, vol 23, selected papers from acis 2017 designing trids 16 # detailed risk treatment relevant risks 20. triangulate evaluations using different forms, cf. pries-heje et al. (2014a), p. 14 c-7, d-13, d-14, e-1, e-5, f-1, f-3 21. ask “devil’s advocate” question: “how and why might the artefact fail to match the requirements?” d-2 22. design and document the design carefully using template/meta-design-driven design & documentation, cf. pries-heje et al. (2014a), p. 15 d-2, d-3, d-4, d-5 23. design review d-3, d-4 24. instantiation review d-5 25. early (partial) prototype review d-6 26. post-implementation review d-6 27. ask naturalistic evaluation stakeholders about potential and forecast changes (when investigating the problem for naturalistic evaluation) d-7, d-12 28. develop and deliver the working purposeful artefact for naturalistic evaluation quickly d-7, d-12 29. seek co-authors with expertise in evaluation methods d-7, d-9, d-10, d-11, d-13, d-14, e-1, e-5, f1 30. plan a good change management practice for naturalistic evaluation d-8 31. involve users early and often (a.k.a. user and stakeholder participation) for naturalistic evaluation, cf. pries-heje et al. (2014a), p. 15 d-8, d-9, d-10, d-11, d-12 32. identify and resolve disagreements among stakeholders (during problem formulation and/or change management) d9, d-10, d-11, d-12 33. support and guide implementers (post research) to implement the purposeful artefact properly e-1, e-2, e-3, e-4 34. rigorously evaluate for (unsafe) side effects, cf. pries-heje et al. (2014a), p. 15 e-1, e-5, f-1, f-3, f-5 35. help implementers to conduct proper change management e-4 36. throughout the dsr project, actively clarify and manage (plan, review/monitor, and re-plan) the significance of the problem f-1, f-2 37. throughout the dsr project, actively clarify and manage (plan, review/monitor, and re-plan) the newness/novelty of the artefact f-1 38. throughout the dsr project, actively clarify and manage (plan, review/monitor, and re-plan) the rigour of the evaluation f-1, f-3 39. throughout the dsr project, actively clarify and manage (plan, review/monitor, and re-plan) the relationship of the problem and the extant artefact to extant theory and the literature f-1 40. seek co-authors to distribute the workload and the risk, cf. pries-heje et al. (2014a), p. 15 f-1, f-2 41. change the scope of the research to something less risky, cf. pries-heje et al. (2014a), p. 15 f-1, f-2 42. abandon the research if too risky, cf. pries-heje et al. (2014a), p. 14 f-1, f-2 43. enter into contract/agreement clarifying ip rights and right to publish, cf. pries-heje et al. (2014a), p. 18 f-1 44. ask “devil’s advocate” question: “how and why might this research produce incorrect results?” f-3 australasian journal of information systems venable, vom brocke & winter 2019, vol 23, selected papers from acis 2017 designing trids 17 # detailed risk treatment relevant risks 45. ask “devil’s advocate” question (during design and design review): “how and why could the artefact be (or become) too unique to disseminate?” f-4 46. ask experts whether the design is too unique to be used in other contexts (during design review) f-4 47. revise the scope or topic of the research table 2. trids treatment list (with relevant risks) 4.3.3 trids risks and treatments lookup table having introduced all the dsr risk treatments that we identified, we inverted table 2 so that it is easily searchable by risk to look up candidate risk treatments for any particular risk. table 3 provides that capability. unfortunately, space limitations prevent providing explanations of the risks and treatments in table 3, so only the risk numbers and treatment numbers are included. all dsr risks identified in rmf4dsr and in this paper are present in table 3 and we have identified at least one risk treatment for all of them. all of the risk treatments are described in detail in table 2 above. dsr risk risk treatments dsr risk risk treatments a-1 1, 2, 3 d-3 22, 23 a-2 1, 2, 3 d-4 22, 23 a-3 1, 2, 3, 4, 5 d-5 22, 24 a-4 1, 2, 3, 4, 5 d-6 25, 26 a-5 1, 2, 3, 4, 5 d-7 27, 28, 29 a-6 1, 2, 3, 4, 6 d-8 30, 31 a-7 1, 2, 3, 4, 6 d-9 29, 31, 32 a-8 1, 2, 3, 4, 6 d-10 29, 31, 32 a-9* 1, 2, 3, 5, 6, 7, 8 d-11 29, 31, 32 b-1 1, 3, 9 d-12 27, 28, 31, 32 b-2 1, 3, 9, 10, 11 d-13* 20, 29 b-3 1, 3, 9, 12 d-14* 20, 29 c-1 11, 13, 14, 15 e-1 14, 20, 29, 33, 34 c-2 14, 15, 16 e-2 33 c-3 11, 14, 15, 16 e-3 33 c-4 10, 14, 15, 16 e-4 33, 35 c-5 3, 10, 14, 15, 17 e-5* 20, 29, 34 c-6 3, 10, 14, 15, 16, 18 f-1 20, 29, 34, 36, 37, 38, 39, 40, 41, 42, 43 c-7 3, 10, 14, 15, 16, 18, 19, 20 f-2 36, 40, 41, 42 d-1a 3, 14, 16, 17 f-3 20, 34, 38, 44 d-1b 2, 14, 16, 17 f-4 45, 46 d-2 21, 22 f-5* 20, 29, 34 table 3. trids risks and treatments lookup table the asterisks in table 3 indicate new risks introduced in trids. 4.3.4 trids dsr risk treatments classification and guidance the dsr risk treatments in table 2 are listed in order of risks and similar types of treatments are not grouped together. in this section, we classify different treatments, provide further explanation about them where needed, and provide references to literature for further explanation and guidance for how to conduct the treatments. australasian journal of information systems venable, vom brocke & winter 2019, vol 23, selected papers from acis 2017 designing trids 18 within the list of dsr risk treatments above, some treatments are similar to others and can be grouped into classes. we identified 14 generic classes of dsr risk treatments, including (1) literature reviews, (2) conducting empirical studies, (3) applying problem analysis techniques, (4) encouraging participation, (5) conducting reviews, (6) following established design practices, (7) following established dsr evaluation guidance, (8) applying change management practices, (9) using project management techniques, (10) seeking expert advice, (11) seeking co-authorship, (12) asking (and answering) devil’s advocate questions, (13) managing critical dsr quality guidelines, and (14) changing project scope. some treatments fall into more than one category. we introduce each of these 14 risk treatment categories and categorise them as self-insure, self-protect, transfer, or avoidance in the remainder of this section. literature reviews (treatments 1 and 9): much has been written in the is field about rigorous literature reviews, including work by webster and watson (2002), jennex (2015) and vom brocke et al. (2015). effective search queries are essential (e.g. not relying on only one or a few specific keywords, not limiting the search to a subset of academic outlets, and not ignoring current practices and artefacts available from vendors or consultants or used in industry). also essential is tracking how the literature search was conducted for future reference and expansion. modern search systems also provide the feature of open queries or alerts, which notify the user when new publications meeting search criteria become available. see, e.g., google scholar. moreover, a good literature search may identify kernel theories (walls et al., 1992), or justificatory knowledge (gregor & jones, 2007) that through abduction in the suggestion phase (vaishnavi & kuechler, 2004, 2015) can be adapted to address the problem to be solved. conducting thorough literature searches and reviews is a preventive form of self-protection (a.k.a. treat) as it reduces the likelihood and impact of risks. for example, in ongoing research involving one of the authors on managing complexity in project management, the core of the artefact design was suggested and heavily influenced by literature on the nature of complexity and how to deal with it, including papers from the nonacademic, practitioner literature. empirical studies (treatments 2 and 20): relying solely on the literature may not be appropriate, particularly for understanding problems (which may change over time). empirically evaluating purposeful artefacts and design theories needs to be carefully designed, using tried and true approaches such as triangulation rather than relying solely on one evaluation approach or episode. studying a problem empirically is another preventive form of self-protection. empirical formative evaluation is a detective form of self-protection, which enables corrective treatment through redesign (or possibly through revising the scope of a dsr study). designing formative and summative evaluations according to recommended practice is a directive form of selfprotection. for example, in ongoing research involving one of the authors on developing an approach for how to enhance and facilitate university collaboration with not-for-profit organisations, surveys of university staff and students were conducted to identify their willingness to collaborate (e.g. in work-integrated learning (wil) and collaborative research) and to identify potential issues relating to feasibility and needed features. australasian journal of information systems venable, vom brocke & winter 2019, vol 23, selected papers from acis 2017 designing trids 19 problem analysis techniques (treatments 4, 5, 6, 7, 27, and 32): understanding problems, which are always perceived, and agreeing about them are important in a problem-solving paradigm like dsr. much has been written about analysing and achieving shared understanding of problems, including work by peter checkland and colleagues on soft systems methodology (checkland, 1981; checkland & scholes, 1990), other work about problem analysis and solving methods (e.g. flood & jackson (1991); rosenhead & mingers (2001)), and work on causal analysis for dsr (venable, 2014). careful problem analysis, from different perspectives, is a form of preventive, but also directive self-protection. for example, venable and travis (2000) made use of stakeholder analysis techniques and rich pictures when developing a proposal for a community-based is digital library system and bringing it to the ifip community for discussion. participation (treatments 3, 10, 11, 16, 17, 18, 27, 31 and 32): participation by stakeholders, including users, beneficiaries, and decision makers is useful (if not essential) throughout all phases of dsr. the literature above on problem analysis and solving stresses participation. various dsr methods have also been developed that stress participation, including action design research (adr sein et al., 2011), participatory action design research (padr bilandzic & venable, 2011), soft design science methodology (sdsm pries-heje et al., 2014b), and a different version of participatory action design research (padre haj-bolouri et al., 2016). using a participative analysis and design approach is a form of preventive, but also directive self-protection. for example, in his phd thesis research, bilandzic (bilandzic, 2013) undertook two cycles of padr, involving extensive participation from (and co-design by) members of a community group, to develop a rich understanding a particular context of connected learning and to develop and evaluate two artefacts, hack the evening, a social intervention, and gelatine, a custom-developed ambient media system (for connected/social learning). heavy participation from stakeholders ensured that their needs were met and that the artefact was rigorously (and authentically) evaluated by those whom it was intended to benefit. reviews (treatments 8, 13, 16, 17, 18, 23, 24, 25, 26, 36, 37, 38, 39, 45, and 46): formal (or semi-formal) reviews can be conducted at any stage of a dsr project on any of the artefacts developed along the way. the software engineering literature describes practices and standards for the conduct of reviews. e.g., see pressman & maxim (2015) and sommerville (2015). reviews are a detective form of self-protection. errors or inadequacies identified through reviews enable redesign, a corrective form of self-protection. for example, many phd programs have formal review points during a program, e.g. at the stage of proposal approval, at a mid-way stage, and at a point soon before thesis submission. moreover, some places organise colloquium style peer reviews. however, such general approaches can be extended to review intermediate design artefacts, such as of problem analyses, solution requirements, initial or detailed designs, prototypes, of components of a larger design. following prescribed practices and identifying quality criteria beforehand to be applied during a review are good practices. design practices (treatments 14, 20, 23, and 28): there is extensive literature on how to design quality into software and other artefacts. see the software engineering books listed above for reviews as well as journals such as design australasian journal of information systems venable, vom brocke & winter 2019, vol 23, selected papers from acis 2017 designing trids 20 research. vaishnavi and kuechler (2015) suggest a large number of dsr design patterns. moreover, they also reproduce 40 “inventive principles” (appendix 6a, pages 146-149) from triz (altschuller, 2005). following good design practices is a form of directive self-protection. for example, vaishnavi and kuechler (2015), in their discussion of the suggestion phase of the example case of the smart objects dsr project (vaishnavi et al., 1997), noted how following the engineering design principles of partitioning the system into different objects and separating control information from domain information within each object led to a better design, which allowed domain experts to more easily review the correctness of the implementation of a system supporting a complex hierarchy of rules (e.g. for the control system for a nuclear power plant). evaluation guidance (treatments 15, 16, 17, 20, 25, 27, 28, and 34): as described earlier, the dsr literature provides substantial guidance for evaluation in dsr. see venable et al. (2012, 2016), sonnenberg and vom brocke (2012a, 2012b), or prat et al. (2015). early evaluation is a key lesson (abraham et al., 2014). early (formative) and thorough evaluations are a detective form of self-protection. it is also an important form of self-insurance as it allows more time to recover (through redesign, a corrective treatment) should a risk (such as a failed evaluation) eventuate. for example, the framework for evaluation in design science (feds) (venable et al., 2016) currently has over 300 citations on google scholar. change management (treatments 30, 33, and 35): the fields of management and information systems have both studied change management extensively and provide guidance for how to accomplish it successfully. see, e.g., by (2005). following good change management practices is a directive, but also a preventive form of selfprotection. for example, bider et al. (2012) describe the importance of change management in dsr and provide some useful suggestions for a change management process and various change management practices. project management (28, 41, 42, and 43): in many important ways, a dsr project is just like any other project, and basic project management practices can be very useful to help ensure successful dsr projects and avoid common dsr project pitfalls. see, e.g., vom brocke and lippe (2010, 2013) and lippe and vom brocke (2016). although not specialised to dsr, the memorial university of newfoundland provides guidance for managing research projects at https://research-tools.mun.ca/rpm/. good project management is a key form of directive self-protection. scheduling slack time in the project plan is also a key form of self-insurance for a project. for example, kearns and finn (2017) give an example how not carefully considering whether an activity (publishing a paper on a side topic) was within the scope of the research project led to a four-month delay in a research project. their excellent book, supervising phd students, provides a wealth of straightforward, practical advice about planning, organising, monitoring, and controlling a research project, which is highly applicable whether you are a supervisor or not. australasian journal of information systems venable, vom brocke & winter 2019, vol 23, selected papers from acis 2017 designing trids 21 expert advice (treatments 3, 10, 11, 12, 13, 29, and 46): for researchers who are not themselves experts in one or more areas relevant to a dsr project, seeking and obtaining expert advice can be a very useful way to minimise key risks. key areas of expertise might be in the problem to be solved, known extant approaches to the problem, technologies that might be employed, and dsr research and change management methods. expert advice is a form of preventive self-protection, but is to some extent also a transfer tactic (by transferring some of the risk to experts). for example, reiterer (2018), in preparing his phd thesis developing and evaluating an ontology of dsr concepts to be used for storing, searching, and retrieving dsr knowledge, sought advice from both dsr experts and ontology experts. he also published preliminary research-in-progress papers and obtained valuable feedback and suggestions from expert conference attendees. co-authorship (treatments 3, 10, 11, 12, 29, and 40): a common practice used to obtain expertise is to invite an expert to join the project as a coauthor. this also has the advantage of distributing the risk (and workload) across multiple individuals. adding co-authors may complicate working arrangements, but is often well worth it. co-authorship is a key form of transfer (of some of the risk). for example, one might bring in an expert in statistical analyses to conduct and write up the analysis part of a dsr paper with a quantitative approach to evaluating the artefact. devil's advocate questions (treatments 19, 21, 44, and 45): the devil’s advocate (latin: advocatus diaboli) was a formal position within the roman catholic church, whose task was to argue against the canonization of people under consideration for sainthood (wikipedia, 2018). the idea is extended to mean “someone who pretends, in an argument or discussion, to be against an idea or plan that a lot of people support, in order to make people discuss and consider it in more detail” (cambridge_dictionary, 2018). a popular corollary expression to that is to seriously (as opposed to flippantly) ask “what could possibly go wrong?” james a. senn (senn, 1981), when considering a system development project, would ask and consider “under what circumstances would implementing the system actually make things worse rather than better?” in our list, we suggest asking similar “devil’s advocate” questions about different parts of a dsr project. devil’s advocate questions are similar to evaluations, but through thought experiments; they provide some detective self-protection, but also self-insurance when they are asked early enough to discover problems while there is still time to solve them (i.e. take corrective redesign). for example, the earlier example of empirical research into staff and student willingness to participate in collaborations between not-for-profit organisations and a university arose out of asking devil’s advocate questions like “what if academic staff don’t want to collaborate with not-for-profit organisations on research, e.g. because they don’t have money to fund research?” similarly, one wonders if the design of automated systems for handling customer queries might have been different if the designers had asked: “how might automated phonebased customer service actually reduce business more than the anticipated cost savings of not having real people answer phones?” the design of early email systems might have been australasian journal of information systems venable, vom brocke & winter 2019, vol 23, selected papers from acis 2017 designing trids 22 different if the designers had asked: “how might being able to copy an email once to multiple addressees actually make email less rather than more useful?” manage critical dsr quality guidelines (treatments 36, 37, 38, 39): the dsr literature suggests a number of guidelines and recommendations for what makes criteria for quality (or satisfactory) dsr. the most commonly advocated are the seven guidelines in hevner et al. (2004). managing the achievement of critical guidelines is a good way to help overcome risks and ensure dsr project success. of the seven hevner et al. guidelines, in accordance with venable (2010, 2015), we chose guidelines 1, 2, 3, and 4 as critical, but also add a fifth guideline: ensuring that the purposeful artefact is clearly and explicitly related to the literature and relevant theory (i.e. it has theoretical significance). ensuring that you focus attention on key guidelines for a dsr project is a directive form of self-protection, as well as self-insurance by allowing time to reallocate resources to corrective redesign if a problem is discovered. for example, chau (2011) explicitly states how all seven of the guidelines by hevner et al. (2004) are addressed in the research reported in his paper. of course, managing the achievement of the guidelines during the project would make it easier to report when writing up a paper. making explicit statements about achieving the guidelines isn’t required, but the achievement of the first four hevner et al. guidelines – as well as our guideline to ensure the work is clearly related to the literature – must be apparent within any dsr paper, so must have been done during the research. changing project scope or topic revising (especially reducing) the scope or revising the topic of the research is often an effective (and sometimes the only) successful strategy when encountering significant project risks. reducing the scope may be used to eliminate a particularly risky area of the project, e.g. by solving a smaller, more specific problem or reducing the scope of the solution being attempted. doing so may reduce the significance of the project, but the project significance may still be acceptable. sometimes it may even require abandoning a project altogether and pursuing a different project. changing the project topic or scope is an avoidance tactic. however, it can only be practised early – either up front or in response to discovering a problem. planning ahead for flexibility in changing the scope of a dsr project is a prudent way to plan for using avoidance (or termination in the 4 ts terminology) if it becomes necessary. for example, riedmann et al.’s (2013) work on a rivale prototype of a virtual case study for learning and practising requirements elicitation originally planned to use voice recognition, but it was decided that voice recognition wasn’t reliable enough and was difficult to integrate into a virtual world. because the project scope was adequate without it (users could simply type their interaction instead), that capability was simply excluded from the research. in another example, the phd work mentioned earlier by reiterer originally intended to write software that would automatically analyse dsr publications to identify instances of the dsr concepts and automatically populate the dsr knowledge base. however, the state of the art of semantic analysis (and the amount of effort required to advance it sufficiently) made such research infeasible within available resources. therefore, the scope was changed to a more limited scope that was feasible for a phd project with the available resources. australasian journal of information systems venable, vom brocke & winter 2019, vol 23, selected papers from acis 2017 designing trids 23 changing, especially reducing, the scope can be an effective way to keep a dsr research project on track. other reasons for changing the scope include (1) you identify new research that already has done what you intended to do, (2) new approaches or technologies become available, or (3) interest in a slightly different topic (particularly if funded!) motivates revising the scope to address the related topic. 5 evaluation of trids (dsrpm stage 4) this paper identified six requirements for trids in section 4.1. the six requirements are repeated here for ease of reference. 1. provide as comprehensive a list of all risks relevant to dsr projects as possible, i.e. possibly extending the checklists of known risks provided in rmf4dsr (pries-heje et al., 2014a). 2. identify treatments for all dsr risks identified (comprehensive coverage of risks), including both those already in rmf4dsr and any other dsr risks identified in point #1 above. for every known risk, at least one potential treatment should be identified. 3. identify a range of different kinds of treatments or treatment strategies. having more than one suggestion for potential treatments of a dsr risk would improve the flexibility and utility of the purposeful artefact. 4. where possible, identify treatments in each of the four risk management quadrants (self-insure, transfer, self-protect, and avoidance) found in rmf4dsr (pries-heje et al., 2014a). 5. support users to easily identify specific candidate treatments for each particular dsr risk. 6. where possible, identify controls of all four types (preventive, corrective, directive, and detective) for use in the self-protect/treat quadrant of assessed risks. 5.1 criteria-based evaluation criteria-based evaluation (venable et al., 2016; venable & travis, 1996) is a form of artificial (non-naturalistic, i.e. it does not evaluate in real use (sun & kantor, 2006)) evaluation. criteriabased evaluation is appropriate when there are criteria concerning the artefact that can be objectively observed and confirmed or disconfirmed. each of the six requirements identified above can be objectively confirmed or disconfirmed in the design for trids. we evaluated whether each of the six requirements is met below. the first requirement is to provide a comprehensive list of risks, including extra risks identified in this paper. the list of extra risks is provided in table 1 and the rest can be found in the checklists in section 2.1.3, which are from pries-heje et al. (2014a). while comprehensiveness is a requirement, it is also an issue that such a list should be parsimonious, i.e. not including risks that don’t warrant attention, e.g. by being too esoteric, unlikely, or overly specific. the list is comprehensive with respect to sufficiently significant risks in the literature and known to the authors. however, it is impossible to establish that other dsr risks could not be identified in the future. australasian journal of information systems venable, vom brocke & winter 2019, vol 23, selected papers from acis 2017 designing trids 24 the second requirement is that all risks must have a treatment that applies to that risk. table 3 above facilitates confirming this, as all risks are listed. every risk in table 3 has at least one treatment. the third requirement for trids is to identify a range of different treatments. we judge that having identified and included 14 different categories of treatments meets this requirement. the fourth requirement is to (where possible) identify treatments in all four quadrants of risk treatments: self-insure, self-protect, transfer, and avoidance. while self-protection treatments dominate (mostly by reducing the probability of risks occurring), all four quadrants are represented, but not for every risk, as this is not always appropriate or possible. the fifth requirement that we decided for trids is that it must support easily identifying treatments for each risk. table 3 is sorted by dsr risk, allowing easy look-up of relevant treatments. once a risk is identified (using rmf4dsr), the only thing needed then is to use the treatment number to look up the treatment description in table 2. finally, the sixth requirement we selected for trids is to include candidate treatments from all four types of controls (where possible). the classes of treatments discussed in section 4.3 cover not only all four quadrants, but for self-protection protection treatments, the controls suggested include all four kinds: preventive, corrective, detective, and directive, although not for every specific risk, as this is not always appropriate or possible. 5.2 evaluation summary the criteria-based evaluation presented above demonstrates that trids adequately meets all of the requirements that we established for it early in its design. it is potentially debatable whether the list of risks in the risks checklists (both from rmf4dsr and its extension in trids) is sufficiently comprehensive and parsimonious. we believe that it is. however, we also caution that any list of potential risks may miss out on risks, some of which may only arise in the future. thus, relying totally on a risk checklist may not be appropriate. we also note that requirements 4 and 6 concern having a range of optional treatments for each and every risk. trids does not provide options in all cases. for many kinds of risks in dsr, only limited options may be available. that said, many risks have a wide variety of treatments available, as identified in table 3. importantly, the above criteria-based, artificial evaluation (meeting requirements) does not yet consider issues of usability, practicality, comprehensibility of potential treatments, and other potential issues. further naturalistic evaluation is needed with dsr researchers (student or practising) on actual dsr projects. such work is beyond the scope of this paper. 6 discussion this research has developed an approach for identifying risk treatments in dsr (trids), which extends prior research on dsr evaluation and risk management with additional identified risks and recommendations for risk treatments. as discussed in section 5, trids meets the six requirements identified for how to provide more detailed guidance for how to effectively and efficiently treat risks in dsr projects. this paper identifies new risks in dsr, identifies and classifies treatments for dsr risks into 14 categories, and fills an important gap in the literature. australasian journal of information systems venable, vom brocke & winter 2019, vol 23, selected papers from acis 2017 designing trids 25 6.1 theoretical significance the knowledge developed in this paper can be formulated as a nascent design theory, including new constructs of the artefact and for meeting the requirements of providing better guidance for how to manage risk. from a design theory perspective, trids provides a general design (baskerville & pries-heje, 2010; venable, 2013) of an artefact for matching dsr risks to candidate dsr treatments, which can later be expanded with additional risks and treatments. the artefact’s general design (an interacting set of components, e.g. tables) meets the general requirements (baskerville & priesheje, 2010; venable, 2013) to match risks and treatments through a utility relationship between them for improved effectiveness and efficiency (venable, 2006, 2013). moreover, the approach could be generalised for use for risk management in other domains than in dsr. finally, the risks and risk treatments identified could be generalised to other research paradigms than dsr. the design theory includes new constructs of the artefact and increased utility (compared to extant approaches) for meeting the requirements of providing better guidance for how to treat dsr risks. 6.2 practical significance appropriate use of trids, as described in this paper, should provide effective guidance to dsr researchers, especially non-expert researchers, in managing the risks to their dsr projects and/or programs, which should thereby increase the efficient use of resources and the probability of a dsr project/program’s success. the use of trids should enhance dsr and help make it more efficacious, effective, efficient, and ethical. dsr, in turn, should better serve society by delivering more and higher-quality contributions to the solution of real-world problems. 6.3 limitations as noted earlier concerning the evaluation of trids, while helpful, no checklist is complete (so risks may be missed) and no treatment is guaranteed to work 100% of the time. users of rmf4dsr and trids must be diligent in their application and consider how their dsr project’s context might reduce the effectiveness of risk management approaches. another limitation of this work is that, while extending the risk list (iversen et al., 2004) provided by pries-heje et al (2014a) into a more specific risk-action list (iversen et al., 2004), it does not provide the strategic oversight offered by a risk-strategy model or a risk-strategy analysis (iversen et al., 2004). future work might further extend rmf4dsr and trids to provide such strategic oversight. 7 conclusion the research reported in this paper extends the advice to dsr researchers on how to manage risks in dsr projects/programs by identifying 47 potential treatments for known risks in dsr. it also provides a straightforward way to work from a relevant risk – the augmented risk checklists – to identify candidate treatments for a particular risk. it has further classified the risk treatments into 14 categories, provided relevant examples, and made reference to relevant literature to guide and support their enactment. overall, it has extended the existing literature in dsr risk management with some very practical ideas for how to address and treat risks, which should assist dsr researchers to better manage risks in their dsr projects and australasian journal of information systems venable, vom brocke & winter 2019, vol 23, selected papers from acis 2017 designing trids 26 programs, and thus to better conduct dsr in terms of efficacy, effectiveness, efficiency, and ethics. however, further research would be useful. trids could still use a more complete, naturalistic evaluation. it could also benefit by supplementing it with better guidance on how to decide among the treatments identified and how and when to apply the treatments identified for best results. furthermore, trids could be extended further. ultimately, what we envision is an approach to dsr that is risk-aware and identifies and treats risks in an agile and continuous way, developing and evaluating intermediate artefacts rather than waiting for a summative evaluation at the end of a linear research process. such an approach should lead to dsr that is both more effective (higher quality and more needed outcomes) and more efficient (less wasted effort and rework). references abraham, r., aier, s., & winter, r. (2014). fail early, fail often: towards coherent feedback loops loops in design science research evaluation. in e. karahanna, a. srinivasan & b. tan (eds.), 35th international conference on information systems, held in auckland, new zealand, atlanta, ga, usa: association for information systems. ackermann, f., & eden, c. (2001). soda – journey making and mapping in practice. in j. rosenhead & j. mingers (eds.), rational analysis for a problematic world revisited. chichester, uk: wiley. altschuller, g. s. (2005). the innovation algorithm: triz, systematic innovation and technical creativity. worcester, ma, usa: technical innovation center. avdiji, h., elikan, d., missonier, s., & pigneur, y. (2018). designing tools for collectively solving ill-structured problems hawaii international conference on system sciences held in waikoloa village, hawaii, usa. baskerville, r., & pries-heje, j. (2010). explanatory design theory. business & information systems engineering, 2010(5), 271-282. baskerville, r., pries-heje, j., & venable, j. r. (2008). evaluation risks in design science research: a framework. in r. baskerville & v. vaishnavi (eds.), 3rd international conference on design science research in information systems and technology (desrist 2008) held in atlanta, georgia, usa, baskerville, r., pries-heje, j., & venable, j. r. (2011). a risk management framework for design science research 44th hawaii international conference on system science (hicss 2011), held in kauai, hawaii, usa, beck, k., beedle, m., bennekum, a. v., cockburn, a., cunningham, w., fowler, m., thomas, d. (2001). manifesto for agile software development. retrieved from http://agilemanifesto.org/ bider, i., johannesson, p., perjons, e., & johannesson, l. (2012). design science in action: developing a framework for introducing it systems into operational practice. in f. j. george (ed.), thirty third international conference on information systems, held in orlando, fl, usa, atlanta, ga, usa: association for information systems. australasian journal of information systems venable, vom brocke & winter 2019, vol 23, selected papers from acis 2017 designing trids 27 bilandzic, m. (2013). the embodied hybrid space – designing social and digital interventions to facilitate connected learning in coworking spaces phd queensland university of technology, brisbane, qld, australia. bilandzic, m., & venable, j. r. (2011). towards participatory action design research: adapting action research and design science research methods for urban informatics. journal of community informatics, 7(3) retrieved from http://cijournal.net/index.php/ciej/article/view/786/804 boehm, b. (2002). get ready for agile methods, with care. computer, 35, 64-69. by, r. t. (2005). organisational change management: a critical review. journal of change management, 54(4), 369-380. http://dx.doi.org/10.1080/14697010500359250 cambridge_dictionary. (2018). devil's advocate. retrieved from https://dictionary.cambridge.org/dictionary/english/devil-s-advocate chau, m. (2011). visualizing web search results using glyphs: design and evaluation of a flower metaphor. acm transactions on management information systems, 2(1), 2:1-2:27. http://dx.doi.org/10.1145/1929916.1929918 checkland, p. (1981). systems thinking, systems practice. chichester, uk: wiley. checkland, p., & holwell, s. (1999). information, systems, and information systems. chichester, uk: wiley. checkland, p., & scholes, j. (1990). soft systems methodology in action. chichester: j. wiley. cmi, t. c. m. i. (2014). the effective change manager: the change management body of knowledge. london, uk: the stationery office (tso) limited. conboy, k. (2009). agility from first principles: reconstructing the concept of agility in information systems development. information systems research, 20, 329-354. eden, c. (1988). cognitive mapping: a review. european journal of operational research, 36(1), 1-13. eden, c., & ackermann, f. (1998). making strategy: the journey of strategic management. london: sage. eden, c., & ackermann, f. (2001). soda the principles. in j. rosenhead & j. mingers (eds.), rational analysis for a problematic world revisited. chichester, uk: wiley. fitzgerald, b., harnett, g., & conboy, k. (2006). customising agile methods to software practices at intel shannon. european journal of information systems, 15, 200-213. flood, r. l., & jackson, m. c. (1991). creative problem solving: total systems intervention. chichester, uk: wiley. fowler, m., & highsmith, j. (2001). the agile manifesto. software development, 9, 28-35. gary, k., enquobahrie, a., ibanez, l., cheng, p., yaniv, z., cleary, k., heidenreich, j. (2011). agile methods for open source safety-critical software. software: practice and experience, 41, 945-962. gregor, s., & hevner, a. (2013). positioning and presenting design science research for maximum impact. mis quarterly, 37(2), 337-355. australasian journal of information systems venable, vom brocke & winter 2019, vol 23, selected papers from acis 2017 designing trids 28 gregor, s., & jones, d. (2007). the anatomy of a design theory. journal of the association for information systems, 8(5), 312-335. haj-bolouri, a., bernhardsson, l., & rossi, m. (2016). padre: a method for participatory action design research. in j. parsons, t. tuunanen, j. r. venable, b. donnellan, m. helfert & j. kenneally (eds.), 11th international conference on design science research in information systems and technology (desrist 2016), held in st john's, nl, canada, (pp. 1936). berlin, germany: springer. hevner, a., & chatterjee, s. (2010). design research in information systems: theory and practice. new york, ny, usa: springer. hevner, a. r., march, s. t., park, j., & ram, s. (2004). design science in information systems research. mis quarterly, 28(1), 75-105. hickman, l. a. (1998). dewey's theory of inquiry. in l. a. hickman (ed.), reading dewey: interpretations for a postmodern generation. bloomington, in, usa: indiana university press. highsmith, j. (2002). agile software development ecosystems. boston, ma, usa: addisonwesley. highsmith, j., & cockburn, a. (2001). agile software development: the business of innovation. computer, 334, 120-127. hildebrand, d. (2008). dewey: a beginner's guide. oxford, uk: oneworld. hopkin, p. (2017). fundamentals of risk management: understanding, evaluating and implementing effective risk management (4th ed.). london, uk: kogan page. iversen, j. h., mathiassen, l., & nielsen, p. a. (2004). managing risk in software process improvement: an action research approach. mis quarterly, 28(3), 395-433. jalote, p. (2008). a concise introduction to software engineering. london, uk: springer-verlag. jennex, m. e. (2015). literature reviews and the review process: an editor-in-chief’s perspective. communications of ais, 36(8) kearns, h., & finn, j. (2017). supervising phd students: a practical guide and toolkit. adelaide, sa, australia: thinkwell. lindvall, m., basilli, v., boehm, b., costa, p., dangle, k., shull, f., zelkowitz, m. (2002). empirical findings in agile methods. in d. wells & l. williams (eds.), extreme programming and agile methods (xp/agile universe), held in in chicago, il, usa: springer. lippe, s., & vom brocke, j. (2016). situational project management for collaborative research projects. project management journal, 47(1), 76-96. mitchell, r. k., angle, b. r., & wood, d. j. (1997). toward a theory of stakeholder identification and salience: defining the principle of who and what really counts. academy of management journal, 22(4), 853-886. morana, s., scheid, m., gau, m., benke, i., vom brocke, j., fettke, p., & maedche, a. (2018). research prototype: the design canvas in mydesignprocess. in s. chatterjee, k. dutta australasian journal of information systems venable, vom brocke & winter 2019, vol 23, selected papers from acis 2017 designing trids 29 & r. p. sundarraj (eds.), 13th international conference on design science research in information systems and technology, held in chennai, india: springer. nagle, t., & sammon, d. (2016). the development of a practitioner design science research canvas ais sigprag: practice-based design and innovation of digital artifacts, held in dublin, ireland: association for information systems. retrieved from http://sigprag.net/2016/11/ nerur, s., mahapatra, r., & mangalaraj, g. (2005). challenges of migrating to agile methodologies. communications of the acm, 48(5), 72-78. pdc. (2018a). about pdc. retrieved from https://pdc2018.org/about-pdc/ pdc. (2018b). free portal to the complete archive of participatory design conference (pdc) proceedings retrieved from http://pdcproceedings.org/ peffers, k., tuunanen, t., rothenberger, m. a., & chatterjee, s. (2008). a design science research methodology for information systems research. journal of management information systems, 24(3), 45-77. prat, n., comyn-wattiau, i., & akoka, j. (2015). a taxonomy of evaluation methods for information systems artifacts. journal of management information systems, 32(3), 229-267. pressman, r. s., & maxim, b. r. (2015). software engineering: a practitioner's approach. new york, ny, usa: mcgraw-hill. pries-heje, j., venable, j. r., & baskerville, r. (2008). a risk management framework for design science research 31st information systems research seminar in scandinavia (iris 2008), held in åre, sweden. pries-heje, j., venable, j. r., & baskerville, r. (2014a). rmf4dsr: a risk management framework for design science research. scandinavian journal of information systems, 26(1), article 3. retrieved from http://aisel.aisnet.org/sjis/vol26/iss1/3 pries-heje, j., venable, j. r., & baskerville, r. (2014b). soft design science methodology. in o. e. hansen, j. simonsen, c. svabo, s. m. strandvad, k. samson & m. hertzum (eds.), situated design methods. cambridge, ma, usa: mit press. reiterer, e. h. r. (2018). design and evaluation of dsrcrsa: an ontology-based approach to design science research content representation and summarisation phd curtin university, perth wa australia. riedmann, p., venable, j. r., chang, v., reiners, t., & gütl, c. (2013). rivale: a prototype realistic immersive virtual agent-based learning environment case study for learning requirements elicitation skills. 2013 ais siged: iaim international conference on informatics education and research (siged 2013) held in milan, italy rosenhead, j., & mingers, j. (eds.). (2001). rational analysis for a problematic world: problem structuring methods for complexity, uncertainty and conflict. chichester, uk: wiley. sein, m. k., henfridsson, o., purao, s., rossi, m., & lindgren, r. (2011). action design research. mis quarterly, 35(1), 37-56. senn, j. a., personal communication, 1981. sommerville, i. (2015). software engineering (10th ed.). harlow, essex, uk: pearson. australasian journal of information systems venable, vom brocke & winter 2019, vol 23, selected papers from acis 2017 designing trids 30 sonnenberg, c., & vom brocke, j. (2012a). evaluation patterns for design science research artefacts. in m. helfert & b. donnellan (eds.), european design science symposium (edss) 2011, held in dublin, ireland, (pp. 71-83). berlin/heidelberg, germany: springer. sonnenberg, c., & vom brocke, j. (2012b). evaluations in the science of the artificial – reconsidering the build-evaluate pattern in design science research. in k. peffers, m. rothenberger & b. kuechler (eds.), 7th international conference on design science research in information systems and technology (desrist 2012), held in las vegas, nevada, usa, (pp. 381-397). springer. steen, m. (2013). co-design as a process of joint inquiry and imagination. designissues, 29(2), 16-28. stoeckli, e., neiditsch, g., uebernickel, f., & brenner, w. (2017). towards an understanding of how and why design science research scholars evaluate. in m. indulska, k. riemer & v. tuunainen (eds.), australasian conference on information systems, held in hobart, tas, australia. summers, b. l. (2011). software engineering reviews and audits. boca raton, fl, usa: crc press. sun, y., & kantor, p. b. (2006). cross-evaluation: a new model for information system evaluation. journal of the american society for information science and technology, 57(5), 614628. ulrich, w. (1987). critical heuristics of social systems design. european journal of operational research, 31, 276-283. vaishnavi, v., & kuechler, w. j. (2004). design science research in information systems. 23 october 2013 retrieved from http://desrist.org/design-research-in-information-systems/ vaishnavi, v., & kuechler, w. j. (2015). design science research methods and patterns: innovating information and communication technology (2nd ed.). boca raton, fl, usa: crc press. vaishnavi, v. k., buchanan, g. c., & kuechler, w. l. (1997). a data/knowledge paradigm for the modeling and design of operations support systems. ieee transactions on knowledge & data engineering, 9(2), 275-291. venable, j. r. (2005). coloured cognitive maps for modelling decision contexts. in t. bui & a. gachet (eds.), first international workshop on context modeling and decision support, held in paris, france: ceur workshop proceedings. retrieved from http://ceurws.org/vol-144/03_venable.pdf venable, j. r. (2006). the role of theory and theorising in design science research. first international conference on design science research in information systems and technology (desrist 2006) held in claremont, ca, usa. venable, j. r. (2010). design science research post hevner et al: criteria, standards, guidelines, and expectations. in r. winter, j. l. zhao & s. aier (eds.), 5th international conference on design science research in information systems and technology (desrist 2010), held in st gallen, switzerland: springer. venable, j. r. (2013). rethinking design theory in information systems. in j. vom brocke, s. ram & m. rossi (eds.), 8th international conference on design science research in australasian journal of information systems venable, vom brocke & winter 2019, vol 23, selected papers from acis 2017 designing trids 31 information systems and technology (desrist 2013), held in helsinki, finland, heidelberg, germany: springer. venable, j. r. (2014). using coloured cognitive mapping (ccm) for design science research. in m. c. tremblay, d. vandermeer, m. rothenberger, a. gupta & v. yoon (eds.), 9th international conference on design science research in information systems and technology (desrist 2014), held in miami, fl, usa, (pp. 345-359). berlin, germany: springer. venable, j. r. (2015). five and ten years on: have dsr standards changed? in b. donnellan, m. helfert, j. kenneally, d. vandermeer, m. rothenberger & r. winter (eds.), 10th international conference on design science research in information systems and technology (desrist 2015), held in dublin, ireland, (pp. 264-279). berlin, germany: springer. venable, j. r., & baskerville, r. (2012). eating our own cooking: toward a more rigorous design science of research methods. electronic journal of business research methods, 10(2), 141-153. retrieved from http://www.ejbrm.com/volume9/issue2 venable, j. r., pries-heje, j., & baskerville, r. (2012). a comprehensive framework for evaluation in design science research. in k. peffers, m. rothenberger & b. kuechler (eds.), 7th international conference on design science research in information systems and technology (desrist 2012), held in las vegas, nevada, usa, (pp. 423-438). springer. venable, j. r., pries-heje, j., & baskerville, r. (2016). feds: a framework for evaluation in design science research. european journal of information systems, 25(1), 77-89. http://dx.doi.org/10.1057/ejis.2014.36 venable, j. r., & travis, j. (1996). criteria-based evaluation of information systems modelling languages. in k. siau & y. wand (eds.), 1st workshop on evaluation of modelling methods in systems analysis and design (emmsad ’96), held in heraklion, crete, greece: university of british columbia, vancouver, bc, canada. venable, j. r., & travis, j. (2000). developing a virtual community-based information systems digital library: a proposal and research program. in r. baskerville, j. stage & j. i. degross (eds.), organizational and social perspectives on information technology: ifip tc8 wg8.2 international working conference on the social and organizational perspective on research and practice in information technology, held in aalborg, denmark, (pp. 319-336). boston, ma, usa: kluwer academic press. vom brocke, j., & lippe, s. (2010). taking a project management perspective on design science research. in r. winter, j. l. zhao & s. aier (eds.), 5th international conference on design science research in information systems and technology (desrist 2010), held in st gallen, switzerland, (pp. 31-44). berlin: springer-verlag. vom brocke, j., & lippe, s. (2013). identifying and managing creative tasks in collaborative is research projects. project management journal, 44(6), 94-113. vom brocke, j., simons, a., riemer, k., niehaves, b., plattfault, r., & cleven, a. (2015). standing on the shoulders of giants: challenges and recommendations of literature search in information systems research. communications of acm, 37(1), 205-224. walls, j. g., widmeyer, g. r., & el sawy, o. a. (1992). building an information system design theory for vigilant eis. information systems research, 3(1), 36-59. australasian journal of information systems venable, vom brocke & winter 2019, vol 23, selected papers from acis 2017 designing trids 32 wang, x., conboy, k., & cawley, o. (2012). "leagile" software development: an experience report analysis of the application of lean approaches in agile software development. journal of systems and software, 85, 1287-1299. webster, j., & watson, r. t. (2002). analyzing the past to prepare for the future: writing a literature review. mis quarterly, 26(2), xiii-xxiii. wikipedia. (2018). devil 's advocate. retrieved from https://en.wikipedia.org/wiki/devil%27s_advocate australasian journal of information systems venable, vom brocke & winter 2019, vol 23, selected papers from acis 2017 designing trids 33 appendix a: extended dsr risk checklists checklists below are extended from rmf4dsr (pries-heje et al., 2014a). five additional risks are annotated with an asterisk. checklist 1: business needs a-1 selection of a problem that lacks significance for any stakeholder a-2 difficulty getting information about the problem and the context a-3 different and even conflicting stakeholder interests (some of which may not be surfaced) a-4 poor understanding of the problem to be solved a-5 solving the wrong problem, i.e. a problem that isn’t a main contributor to undesirable outcomes that motivate the problem solving a-6 poor/vague definition/statement of problem to be solved, with potential misunderstanding by others a-7 inappropriate choice or definition of a problem according to a solution at hand a-8 inappropriate formulation of the problem *a-9 inappropriate choice of meta-requirements (scoping error) checklist 2: grounding b-1 ignorance or lack of knowledge of existing research relevant to the problem understanding and over-reliance on personal experience with or imagination of the problem b-2 ignorance or lack of knowledge of existing design science research into solution technologies for solving the problem, i.e. lack of knowledge of the state of the art b-3 ignorance or lack of knowledge of existing relevant natural and behavioural science research forming kernel theories for understanding or solving the problem checklist 3: build design artefacts c-1 development of a conjectural (un-instantiated) solution which cannot be instantiated (built or made real), cf. technical risk in venable et al. (2016) c-2 development of a hypothetical (untried) solution which is ineffective in solving the problem, i.e. the artefact doesn’t work doesn’t work well in real situations with various socio-technical complications, cf. human risk in venable et al. (2016) c-3 development of a hypothetical (untried) solution which is inefficient in solving the problem, i.e. requiring overly high resource costs australasian journal of information systems venable, vom brocke & winter 2019, vol 23, selected papers from acis 2017 designing trids 34 c-4 development of a hypothetical (untried) solution which is inefficacious in solving the problem, i.e. the artefact isn’t really the cause of an improvement observed during evaluation c-5 development of a hypothetical (untried) solution which cannot be taught to or understood by those who are intended to use it, e.g. overly complex or inelegant c-6 development of a hypothetical (untried) solution which is difficult or impossible to get adopted by those who are intended to use it, whether for personal or political reasons c-7 development of a hypothetical (untried) solution which causes new problems that make the outcomes of the solution more trouble than the original problem, i.e. there are significant side effects checklist 4: evaluate d-1 tacit requirements (which by definition cannot be surfaced) are not dealt with when evaluating the solution technology, leading to failure of the solution technology to meet those requirements d-2 failure to surface some or all of the relevant requirements leads to those requirements not being dealt with when evaluating the solution technology, leading to failure of the solution technology to meet those requirements d-3 incorrectly matching the articulated requirements to the meta-requirements of the isdt lead to the testing of the idst and evaluation of an instantiation of the meta-design in a situation for which neither should be applied d-4 incorrectly matching the meta-design or the design method to the meta-requirements (not following the isdt correctly) leads to evaluation of something other than the correct solution technology or the isdt as stated d-5 improper application of the meta-design or the design method (not in accordance with the isdt) in designing an instantiation leads to evaluation of something other than the correct solution technology or the isdt as stated d-6 improperly building an instantiation of the solution technology (such that it does not properly embody the meta-design) leads to evaluation of something other than the correct solution technology or the isdt as stated d-7 difficulties in implementing the solution technology during naturalistic evaluation, due to such things as unforeseen complications within the business/organization, prevent the instantiation of the solution technology from successfully meeting its objectives d-8 success of the solution technology to meet its objectives is not obtained due to dynamic or changing requirements beyond the scope of the solution technology d-9 success of the solution technology to meet its objectives is not achieved due to poor change management practices australasian journal of information systems venable, vom brocke & winter 2019, vol 23, selected papers from acis 2017 designing trids 35 d-10 determination of success or failure in reaching the objectives of the solution technology is error-prone or impossible due to disagreement about objectives or inability to measure d-11 existing organizational culture, local organizational culture differences (sub-cultures), political conflicts, etc. complicate the evaluation process or weaken the ability to make meaningful measurement of the achievement of the objectives of the solution technology d-12 existing organizational priorities, structures, practices, procedures, etc. complicate the evaluation process or ability to make/measure the achievement of the objectives d-iii emergence of new organizational or individual practices, structures, priorities, norms, culture, or other aspects that complicate the acceptability, workability, or efficiency of the application of the solution technology in a naturalistic setting *d-13 type i evaluation error (false positive) (baskerville et al., 2011) *d-14 type ii evaluation error (false negative) (baskerville et al., 2011) checklist 5: artefact dissemination and use e-1 implementation in practice of a solution does not work effectively, efficiently, and/or efficaciously e-2 misunderstanding the appropriate context for and limitations of the solution e-3 misunderstanding how to apply the solution e-4 inappropriate handling of adoption, diffusion, and organizational implementation *e-5 unsafe purposeful artefact released checklist 6: knowledge additions f-1 inability to publish or present research results f-2 publication of low significance research f-3 publication of incorrect research f-4 design artefacts prove too unique to disseminate *f-5 unsafe purposeful artefact design published copyright: © 2019 venable, vom brocke & winter. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems venable, vom brocke & winter 2019, vol 23, selected papers from acis 2017 designing trids 36 1 introduction 2 literature review 2.1 dsr literature relevant to dsr risk management 2.1.1 dsr guidelines 2.1.2 dsr evaluation literature 2.1.3 dsr risk management literature 2.1.4 problem to be solved 2.2 non-dsr literature relevant to dsr risk management 2.2.1 general and is risk management 2.2.2 2.2.2 participation in system development and dsr 2.2.3 software engineering reviews 2.2.4 literature reviews 2.2.5 change management 2.2.6 problem analysis and formulation 3 research method 3.1 awareness of problem stage 3.2 suggestion stage 3.3 development stage 3.4 evaluation stage 3.5 conclusion stage 4 design of trids (dsrpm stages 1-3) 4.1 requirements for trids (dsrpm stage 1) 4.2 suggestion phase and solution alternatives (dsrpm stage 2) 4.3 design and development of trids (dsrpm stage 3) 4.3.1 trids extended dsr risks checklists 4.3.2 trids treatment list 4.3.3 trids risks and treatments lookup table 4.3.4 trids dsr risk treatments classification and guidance 5 evaluation of trids (dsrpm stage 4) 5.1 criteria-based evaluation 5.2 evaluation summary 6 discussion 6.1 theoretical significance 6.2 practical significance 6.3 limitations 7 conclusion paper title (paper title style) australasian journal of information systems an, vesty & cheong 2023, vol 27, research article defining digital wellbeing literacy 1 defining digital wellbeing literacy in remote work integrated learning nancy an rmit university, melbourne, australia anle1001.super@gmail.com gillian vesty rmit university, melbourne, australia chris cheong brimbank city council, sunshine, victoria, australia abstract the pandemic has accelerated the adoption of remote work-integrated learning (wil) programs for higher education (he) students, fostered by advances in digital technologies. emerging as a new post-covid normal, the ‘remote workplace’ and remote wil brings further challenges with students expressing anxiety in dealing with this new form of working environment. having the capacity to talk about wellbeing issues with others is an important wellbeing literacy (wl) skill. this paper addresses the need for a better understanding of wellbeing literacy (wl) skills in remote wil workplaces. interpreting wl in remote settings can further the definition of wl by extending it to a digital context. in contributing to computer-supported cooperative work (cscw) research and the emerging field of wl, this paper explores how wil students develop wl skills in a remote setting, which we define as digital wellbeing literacy (dwl). in-depth interviews were held with wil academics, wil professionals, wil students and wellbeing experts on digital and pedagogical factors that support wl in remote work. we found that students proactively use digital communication tools and social media to communicate wellbeing concerns, which in turn helps them practice dwl. we propose four strategies toward improving dwl in he wil offerings. keywords: wellbeing, wellbeing literacy, digital wellbeing literacy, computer-supported cooperative work, work integrated learning, pandemic, employability, higher education. 1 introduction since the outbreak of the pandemic, work integrated learning (wil) in higher education (he) programs have been adjusted to remote delivery mode (kay et al., 2020), with remote workplaces increasingly becoming the post-pandemic norm. although the field of computer supported cooperative work (cscw) has explored distributed collaboration for more than two decades, the pandemic has provided new opportunities to explore the unprecedented scale of remote working (caldeira et al., 2022; cho et al., 2022). to date the cscw literature has approached remote working issues from the perspective of organizations, such as the challenges associated with organizational culture (bullinger-hoffmann et al., 2021). this body of literature and other digitally mediated communication literature has considered the authenticity of interaction (lee, 2020) including messaging interactions (sundar et al., 2015). the pandemic has encouraged researchers to focus on a critical analysis of individual worker experience following a universally forced remote working environment (carroll & conboy, 2020) as well as the associated cscw technologies for remote work applications (cho et al., 2022). in examining the wellbeing challenges of remote working, cscw research proposes australasian journal of information systems an, vesty & cheong 2023, vol 27, research article defining digital wellbeing literacy 2 that remote working could result in burnout, stress, and work-life conflicts while workers tried to maintain productivity (aczel et al., 2021; ahmetoglu et al., 2020; cho et al., 2022). while the research to date has largely focused on full-time employees, there is very limited research on the remote work wellbeing impacts on wil students (pretti et al., 2020). for some wil students, like other remote workers, there is a need to deal with various wellbeing challenges, such as loneliness, insecurity and anxiety (burtscher et al., 2020). compared to their experienced workplace colleagues, wil students are less practised in handling workplace collaboration and professional communications. meeting workplace colleagues and mentors for the first time online and only have remote engagement experiences to build on, could lead to wil students’ stress (an et al., 2021). there is a greater requirement for supervision and mentoring to maintain wil student wellbeing and productivity compared with the more experienced worker (hunter, 2019). wellbeing literacy (wl) is a potential solution to maintain and increase wellbeing. wl refers to an individual’s capability to use intentional language about and for wellbeing (oades & johnston, 2017). rather than passively reacting to negative wellbeing incidents, wl is a soft skill that empowers individuals so they can proactively communicate wellbeing concerns to maintain positive feelings and lifestyle functions. with wellbeing challenges prevalent in the workplace (daniels et al., 2021), wl is a critical employability skill that students can use to demonstrate their mindset and skillset throughout their careers. remote working and learning provide a real-life context for wil students to practice wl, improve their resilience and personal growth, and thus promote their employability soft skills (an et al., 2021). given that research on wl is still in its infancy, the measurement of, and effective ways to develop wl require further exploration (oades & johnston, 2017). there is limited research focusing on wl development in higher education wil programs. furthermore, the application of wl in a digital context is increasingly important to explore, particularly given the role of technology in transforming the nature of work. the pandemic has accelerated digitalization (worldeconomicforum, 2020), with remote work becoming a routine practice for organizations around the world, even post-pandemic (carroll & conboy, 2020; worldeconomicforum, 2020). notwithstanding its benefits, remote work also creates significant wellbeing challenges. thus, it is important for individuals to develop capabilities to enhance wl in a digitalized workplace. prior literature has identified that students behave differently online when communicating for wellbeing purposes, compared when in a physical environment (an et al., 2021). however, the impact of remote wil programs on student wellbeing and wl in digitalized remote work context has not been fully explored. as such, the challenges encountered by wil students while communicating wellbeing in remote work environments, and the approaches to facilitate the development of wl in digital collaborations are the focus of this study. the research question explored in this study helps to define the concept of dwl: how do wil students express wl in a remote, digital workplace? interviews were conducted with twenty-two participants comprising wil academics, wil administrators, wil students and wellbeing experts. we found that online communication and social media can inhibit wl but can also create valuable environments to practice and develop wl. our study contributes to the developing literature on wl with qualitative evidence from wil students operating in remote working contexts. this study initiates digital wellbeing literacy (dwl) as a new direction of wl research and provides practical suggestions australasian journal of information systems an, vesty & cheong 2023, vol 27, research article defining digital wellbeing literacy 3 for universities and industry partners to enhance student resilience in the short run while providing them with tactics to support employability in the long term. the remainder of this paper is structured as follows: section two reviews related literatures and sets up the research background. section three describes the research method adopted. the findings from the interviews are presented in section four, followed by section five on the discussion of the findings. finally, the limitations, scope for future research and conclusion are stated in section six. 2 literature review 2.1 remote work integrated learning (wil) and its wellbeing impact wil is an umbrella term for a range of approaches and strategies bridging theory with work practice in curriculum (patrick et al., 2008). wil extends formal learning in traditional educational institutions by exposing students to real-world work environments. there is evidence that indicates wil promotes employability (gamage, 2022; jackson, 2015; smith & worsfold, 2015) and helps to reduce the gap between students’ competence and employer expectations (freudenberg et al., 2011). in recent years, digital technology has been included in administrating and delivering wil programs, which are broadly defined as “ewil”(gamage, 2022). rather than engage directly with the physical workplace, digital technology provides an authentic simulated working environment (schuster & glavas, 2017). virtual wil opportunities are provided to students when real placements are not possible (i.e., large classes) or to prepare students for wil placement. the cscw literature has explored the technologies including video conferencing, email and social media, online portals and virtual reality tools and how they contribute to ewil (bullinger-hoffmann et al., 2021; glavas & schuster, 2020). however, understanding the role of cscw technology, and the dynamics of this new remote wil environment, has now become a pressing issue. while remote work facilitated by digital technology has been studied (panteli et al., 2019), the scale of remote work subsequent to the pandemic is unprecedented. regardless of their initially designed mode of delivery, whether in-person placement or a simulated work environment, the pandemic accelerated the transformation of wil programs in many countries to be largely delivered remotely (pretti et al., 2020). remote work provides workers with flexible work schedules and locations, and enables organizations to recruit talent on demand (hunter, 2019). this may also negatively impact individual wellbeing. for example, the complexity of using various digital tools (such as computer, laptop, tablet, and smartphone), the increased amount of communication, and the expectation of instant responses has increased workers’ stress (bordi et al., 2018; mano & mesch, 2010; stich et al., 2015). the asynchronous cooperation and lack of social interaction during remote work deprives employees of a sense of belonging (panteli et al., 2019). nevertheless, social media has penetrated the professional space, reshaping workplace relationships and work itself (bucher et al., 2012; verduyn et al., 2017) also contributing to influence the he wil student experience (parsa & golab, 2021). social media has been reported to have complex impacts on wellbeing (hassan & pandey, 2021; wang et al., 2014) bringing about wellbeing challenges such as social ostracism, social comparison and envy, while enabling ubiquitous social interaction, facilitating workplace support, and promoting a sense of connectedness (charoensukmongkol, 2014; smith et al., 2017; verduyn et al., 2017). social media facilitates real-time and intimate interaction with stakeholders but also australasian journal of information systems an, vesty & cheong 2023, vol 27, research article defining digital wellbeing literacy 4 challenges workers with information overload and obscures the boundary between work and life. social media also increases the levels of uncertainty caused by difficulties tracking sources and changes in information (bucher et al., 2012). remote wil impacts wil students’ wellbeing and makes adaptation to the work environment more difficult than in-person wil (pretti et al., 2020). in an online setting, wil students have less opportunity to directly observe or interact with other co-workers in the professional working environment and learn organizational culture, procedures, and professional communication. it is also more difficult to establish relationships in the workplace and build confidence and professional identities. together these factors can contribute to wil student depression, loneliness and anxiety (an et al., 2021). during the pandemic, wil students arguably faced more wellbeing challenges than before (pretti et al., 2020). as well as the distress caused by the sudden interruption of usual he routines, some students lost wil opportunities, which sometimes resulted in delays in graduation and/or impaired their competitiveness in the marketplace. this further intensified their stress and anxiety (agnew et al., 2019; zhai & du, 2020). some students work in highrisk wil positions such as nursing, aged care and bank and feel anxious about dealing with patients or clients in a face-to-face setting. for wil students who were fortunate to maintain their wil positions but required to engage in remote work, decreased frequency and quality of interpersonal interaction resulted in a feeling of disconnect from the organization (an et al., 2021). while work integrated learning (wil) can provide students with a valuable context in which to proactively manage wellbeing challenges, the same could be argued for a remote work environment. given wil is a strategic priority for universities to improve student employability by offering continuous competency development in the labour market (schuster & glavas, 2017), it makes sense that attention also be given to manage wellbeing in students undertaking he wil programs. 2.2 wellbeing literacy in the workplace wellbeing is an overarching concept combining an individual’s physical, mental, emotional and social health, and refers to positive feelings and effective functioning (magyar & keyes, 2019) the wellbeing of individual employees is argued to be at the heart of organizational competitive advantage and business success (worldeconomicforum, 2020), with workplace wellbeing positively related to employee productivity and creativity, and negatively related to staff absenteeism and resignations (alagaraja, 2020; mccrea, 2019). workplace wellbeing is affected by a range of workplace stressors including interpersonal conflict, work overload and organizational limitations (mazzola et al., 2011). wil students face even more wellbeing issues than other employees. besides academic-related concerns about academic performance (pascoe et al., 2020), wil students tend to feel selfdoubt and lack of self-confidence at work, which are related to their underdeveloped professional dignity and can negatively impact their workplace wellbeing (king et al., 2021) . facing a prevalence of mental health problems in the workplace, many employers focus on providing support when their employees encounter wellbeing issues and when negative impacts on productivity and performance have already occurred (mccrea, 2019). mental health protection by identifying and minimizing stressors in the workplace is another trend, australasian journal of information systems an, vesty & cheong 2023, vol 27, research article defining digital wellbeing literacy 5 but tends to be negatively framed and emphasising avoidance of mental problems (lamontagne et al., 2014). wellbeing, on the other hand, is not merely the absence of mental illness or problems but a state of flourishing (keyes, 2005). to motivate this state, corporate wellbeing promotion programs draw on approaches from positive psychology (kobau et al., 2011), focus more on the positive side of the mental health continuum (lamontagne et al., 2014), and use training or coaching to build mindfulness, resilience and psychological capital (martin et al., 2019). as such, it is considered important that employees are reciprocally engaged in wellbeing initiatives and can articulate with a wellbeing agenda. comprehensive acquisition of, and learning about wellbeing may enable more effective control of each individual’s use of language and knowledge of wellbeing (oades & johnston, 2017). this phenomenon is defined as wellbeing literacy (wl), a capability for intentional language use about and for wellbeing (oades & johnston, 2017). wl research is in its infancy with fundamental aspects such as measurement and effective ways to develop wl still under investigation. wl helps to maintain wellbeing in the workplace and in turn contributes to work productivity and performance (oliver, 2015). wl plays a positive role in assisting wil students to manage wellbeing in the workplace (an et al., 2021). with “awareness and intention” in expressing their wellbeing experiences (oades et al., 2020) and wellbeing needs, many wellbeing-related issues in the workplace can be resolved at a local level. wl can accelerate wil student professionalism development, because the more a wil student intentionally communicates with colleagues about their wellbeing challenges at work, the more they receive insights from their colleagues and develop professional approaches to handle challenges (an et al., 2021). there is limited attention to he student wl throughout their wil experiences, even though the ability to maintain wellbeing status in the face of various workplace stressors is considered an important competence for employability (jackson, 2015; jackson & chapman, 2012). 2.3 toward defining digital wellbeing literacy (dwl) – a new strand of research the cscw literature has been contributing to health and wellbeing of workers since its inception (fitzpatrick & ellingsen, 2013) with cscw technology facilitating the adoption and implementation of wellbeing programs, including emotional and mental support through digital mediated communication. to date, the wellbeing programs are largely considered externally imposed interventions, rather than initiatives that foster emergence from an individual’s capacity. as such, interventions to cultivate wellbeing have been questioned for their limited efficacy (white et al., 2019) and/or inability to be used to enhance people’s capabilities. while it has been argued that for wl to be developed, individuals must foster their own repertoire of skills and seek options to influence their wellbeing (oades et al., 2020), we also content that the digital setting provides a new set of tools and tactics for the development of wl. this paper initiates dwl as a future-focused agenda to contribute to the cscw literature. we argue, the concept of dwl has its roots in wl, which emphasizes an individual’s capability to maintain wellbeing in a digital context. for example, communication through digital media such as collaborative software, or expression of wellbeing status on social media. extant cscw research has analysed social media posts and/or the decision-making process of generating posts (andalibi et al., 2017; constantinou, 2005). the focus of these work is on investigating the representation or expression of mental health in a digital context, to better understand an australasian journal of information systems an, vesty & cheong 2023, vol 27, research article defining digital wellbeing literacy 6 individual’s mental status or even the mood of the nation (venigalla et al., 2020). in most cases, people with mental illness such as depression and eating disorders were identified (feuston & piper, 2018). alternatively, the digital setting can provide a focus on cscw tools and tactics to support an individual’s capabilities to use language for wellbeing purposes, instead of the wellbeing status per se. cscw tools, such as digital communication tools in remote work and social media, create different contexts and channels for dwl which require different development strategies to wl in a face-to-face workplace. research on digital literacy (eshet, 2004; koltay, 2011) and digital competency (oberländer et al., 2020) addresses the cognitive and social skills in gathering information from digital sources, how to operate digital tools effectively and solve problems in digital contexts. studies offer a refreshed definition of professional literacy which includes social media literacy, or the ability to handle overload in social media contexts (bucher et al., 2012). to date, there is limited research focusing on the development of a new form of wl, dwl in the workplace to express and receive information for wellbeing purposes, or ways to support wil students to improve their dwl capacity. with workers spending a significant proportion of working time online and communicating through social media, we argue attention to dwl should naturally flow from activities to promote workplace wellbeing and wl. furthermore, the pandemic has accelerated the adoption of cscw, and remote working is becoming a regular practice for individuals and organizations around the world (carroll & conboy, 2020; yang et al., 2022). given the unique challenges with this working environment, it is critical to articulate the concept of dwl as a new strand of cscw research. 3 methodology 3.1 conceptual framework just as the concepts of wellbeing (magyar & keyes, 2019) and wl (oades & johnston, 2017) are closely related, in this paper we explore dwl and wellbeing as similarly interconnected. that is, a wil student’s knowledge and awareness about wellbeing can facilitate dwl development. as with wl (oades et al., 2020), we propose dwl skills enable individuals to use language to solve or prevent wellbeing issues in digital communication and collaboration. as shown in figure 1 below, the construct “remote work environment” influences both “wil students’ wellbeing” and “digital wellbeing literacy”. “remote work environment,” an important aspect of cscw, provides the context for this study, in which the large scale of remote work as a result of the covid-19 pandemic has provided an opportunity to explore wil students’ dwl. the remote wil programs have challenged student wellbeing due to the physical isolation and reduced social support (an et al., 2021). digital communication adds an extra layer to wellbeing challenges as students have limited experience with digital communication in professional contexts, potentially generating a lack of confidence and stress. nevertheless, the digital context can enable valuable social support to wil students undergoing stressful transition from university to workplace (mikal et al., 2013). additionally, remote work has created a real-world context in which students can practice and develop their dwl. students can articulate their wellbeing needs and proactively communicate about workplace stressors so that they can maintain their wellbeing, thus enhancing their future employability. together, the key constructs identified in the conceptual framework form the basis of the research question for this study and supports the emerging definition of dwl. australasian journal of information systems an, vesty & cheong 2023, vol 27, research article defining digital wellbeing literacy 7 figure 1. conceptual framework of the research design (adapted from(oades et al., 2020) 3.2 method a qualitative approach with semi-structured interviews was undertaken to better understand wil students’ dwl and explore effective ways to facilitate he wil students develop dwl strategies. a qualitative approach is appropriate for the exploratory nature of this study given wl is an emerging and underexplored field of research (oades, lg & johnston 2017). semi-structured interviews were used because they are suitable for collecting preliminary data for new research topics and enable researchers to access first-hand knowledge from interviewees (creswell & poth, 2016). researchers can ask open-ended questions to get indepth understanding of participant perspectives and experience, with flexibility to explore emerging key points during the interview conversations (creswell & poth, 2016). 3.3 participants twenty-two participants from four distinct groups participated in the interviews (refer to table 1). the first three groups consisted of six wil academics, six wil administrator and six wil students. wil administrators are professional staff who administer or coordinate wil programs. they engage with wil students throughout their wil journey and facilitate students’ non-academic needs, including wellbeing needs. the participant wil students are from undergraduate marketing and management programs within an australian-based university business school. the students engage in their wil experience in the third year of their four-year program. wil administrators, wil students themselves and wil academics are invited because they have been identified as key participants to understand wil student experiences holistically (martin & hughes, 2011). each group contributed different perspectives to help us form a comprehensive institutional understanding of challenges wil students face in remote wil programs, and the impact on their dwl development. the fourth group were four wellbeing experts. the concept of wl has roots in the field of wellbeing, and this article identified dwl as a new direction of wl research. dwl in essence is a capability related to wellbeing and thus it is important to explore insights of wellbeing australasian journal of information systems an, vesty & cheong 2023, vol 27, research article defining digital wellbeing literacy 8 experts. while wl is new and there are very few experts explicitly researching on wl, we invited wellbeing experts whose current research interests involve wl related topics to participate in the research. two of the experts have experience providing clinical consulting services and, thus, have an in-depth understanding of the use of language to communicate wellbeing and people’s abilities to articulate their wellbeing statuses and needs. the third expert specializes on workplace wellbeing and contributes specific viewpoints related to workplaces. the fourth expert’s specialization is on mental problem prevention, which is closely related to wl because wl prevents wellbeing issues forming from accumulated stressors. this relatively small number of wellbeing experts is based on the fact that there are very few available researchers working in the wellbeing literacy-related field and the principle that experts tend to agree more with each other in their field of expertise compared to a layperson, thus smaller samples are sufficient for contributing accurate information (guest, bunce & johnson 2006). group description of responsibility number of participants code group 1: wil academics wil academics whose current research interests and expertise include wl or wellbeing-related topics, or who are currently coordinating/teaching wil courses. n=6 a1-a6 group 2: wil administrator staff wil administration staff who have been working in universities to administer or coordinate wil programs and engage with wil students. they have been in their current position for at least 12 months to ensure they have a comprehensive understanding of the wil program. n=6 p1-p6 group 3: wil students undergraduate students from business degrees who have participated in remote wil programs. n=6 s1-s6 group 4: wellbeing experts wellbeing experts whose current research interests and expertise involve wellbeing literacyrelated topics. n=4 e1-e4 total number of participants total n=22 table 1. interview participant group data 3.4 data collection and analysis as part of the initial recruitment process, four wil academics, five wil administrators and three wellbeing experts were initially recruited from the research team’s professional networks. after the initial interviews, another two wil academics and one wil administrator were invited through snowball recruitment. two wil academics helped to forward the invitation email to wil students, and six wil students responded the invitation email to participate in the interviews. another wellbeing expert was invited based on contact information from a public website. the interview questions were developed based on the existing literature and our research questions and conceptual framework. participants from all four groups were asked questions about each key topic to prompt reflections from their perspectives. the full lists of interview questions for each group are in the appendix. all the participants were sent the list of questions and a detailed introduction to the research before their interviews. because wl is a new concept for most of the participants, the provision of an introduction to the research and the interview questions in advance gave participants australasian journal of information systems an, vesty & cheong 2023, vol 27, research article defining digital wellbeing literacy 9 adequate time to understand the research aims and to think about each question. this helped them to provide more detailed, considered and thorough responses during the interviews. the data collection was conducted in two phases. in phase one, five wil academics and six wil administrator staff were interviewed individually during the march-october 2020 covid-19 lockdown period. they had all been in their roles before covid-19 started. except for one wil academic, all the academics coordinated or administered wil programs. thus, they have substantial experience in coordinating and supporting wil students in the emergency remote wil context. in phase two, six students, four wellbeing experts and one wil academic were interviewed individually from november 2020 to december 2021. all students had completed their wil programs at the time of their interviews and, thus, could reflect on their entire wil experience. two of the six students also participated in a one-year placement wil program in 2019 and were able to compare their wil experiences in traditional face-to-face contexts to the remote wil context. individual semi-structured interviews were undertaken over the microsoft teams online conferencing platform with each of the participants. all the interviews were audio-recorded using microsoft teams. participants in the same groups were asked the same questions based on the question list in the appendix. follow-up questions such as “what do you mean by….” were often used to explore topics in-depth or clarify emerging key points. the interviews were transcribed, and the transcripts were imported and analysed in nvivo 12 (nvivo, 2020), a software commonly used by researchers for qualitative data analysis (feng & beharhorenstein, 2019; welsh, 2002). a thematic analysis of all the transcripts was undertaken based on braun and clarke’s (2006) guidelines (braun & clarke, 2006). after carefully reading each transcript to gain familiarity with the content, an initial coding was conducted. some codes such as “seek help” were generated from the literature review while other codes such as “lack of confidence” emerged from the transcripts. all the codes were summarized in the code system, and the related quotes were mapped to the codes. then, three themes were formed based on the analysis of the codes and the research questions, discussed in detail in the findings section: the impact of social media, online communication, and the role of emojis, on students’ dwl. 4 findings overall, working and learning online exposed wil students to wellbeing challenges and prompted them to practice wl in the remote workplace setting. in the following sub-sections, we discuss wil students' approaches to managing wl in a remote environment and the unfolding dwl that emerged from the use of social media and other digital communications, including the role of emojis. 4.1 wellbeing literacy and the use of social media students were already familiar with the use of social media platforms, such as facebook, instagram and line, frequently in their personal lives before they enrolled in wil. thus, they naturally extended the use of social media to their remote working context. besides using social platforms such as group chat and video conferencing as communication tools to engage with their work teams, wil students sometimes used social media as research tools to collect market information for their wil projects (s1). australasian journal of information systems an, vesty & cheong 2023, vol 27, research article defining digital wellbeing literacy 10 the use of social media played a positive role to relieve loneliness during the remote learning and working. social media provides a channel for students to talk about their wellbeing status, and students indicated that they preferred to post on social media when they felt happy and always browse social media while feeling bored or isolated and that funny social media content could improve their moods. a more critical contribution of social media is to facilitate relationship development. the language ability to build positive relationships is an important aspect of wl and in the digital context, statements about wellbeing status and expressions for wellbeing purposes demonstrate this language ability on social media. wil students noted that they frequently and proactively use instagram, facebook and whatsapp to connect and interact with teammates and colleagues from their work placement. besides chats and audio/video calls, students also browse, “like” or comment on posts and profiles of close friends. students believed that social media provided important background knowledge about co-workers, assisting in developing relationships with them (s3). however, social media also negatively impacted wl. for example, when building relationships, social media better supported “relationships already constructed before [communicating on social media]" (a6). adding new friends without “a healthy and good social relationship” (a6) can feel like a social media trap as “you are posting your status or your emotions… you can be really hurt by some of the comments [others post in response to your post]” (a6). when expressing their wellbeing status on social media, there are cases when students commented that they could depend too much on friends’ comments or “likes” for satisfaction. this distracts students from working and may frustrate them if they are not getting the expected numbers or quality of feedback on social media. another risk of posting wellbeing information online is the possibility of undesired feedback or even being cyberbullied (a6). despite the complexity of discussing personal wellbeing on social media, social media provides an easy and attractive education channel for a general audience, including students, to access and learn about wellbeing and obtain wellbeing support information to improve their dwl. one wellbeing expert (e3) said that conversations about wellbeing should be encouraged on social media, including the vocabulary to describe wellbeing and self-care and self-management strategies, because this is a neglected but critical aspect of wl. he explained that his institution moved a range of wellbeing content to social media platforms during covid-19. the benefits were obvious: they reached many more students, increased their frequency of interactions with their audience, and developed interesting ways to promote user-generated wellbeing tips and self-care plans. 4.2 developing digital wellbeing literacy through online communication wil students acknowledged that the unexpected challenges during remote work provided them with valuable contexts to improve their resilience, an important wl skill (oades et al., 2017). for instance, two students (s1, s3) shared their personal experience about adapting to time differences in group collaboration. they gradually adjusted their attitude to appreciate the international collaboration opportunities and developed strategies to use a more inclusive schedule. all these adaptations facilitated their resilience development in the digital workplace. online communication was reported by students as an unnatural way to convey information. in wil, students receive access to various types of online communication tools, such as instant messaging, video or audio conferencing with chats function, email, and digital collaboration australasian journal of information systems an, vesty & cheong 2023, vol 27, research article defining digital wellbeing literacy 11 tools, among others. most students had very limited experience with communicating purely online in business contexts. they tended to feel a lack of confidence when speaking out online in group meetings, especially to initiate conversations (s2). while video conferencing has been widely used by professionals in both formal and informal communication, to maintain collegial ties and reduce social barriers (bleakley et al., 2022), students reported they were not comfortable using video conferencing tools. their lack of confidence and feelings of insecurity explained why many students preferred to turn off their video cameras during group discussions and why they prefer using messaging, chats and emails to communicate with coworkers (s3). however, exposure to online communication and constant practice gradually improved wil students’ familiarity with the online context and enhanced their confidence and skills with online communication (s2). students also improved their ability to use professional digital tools to facilitate group collaboration. in line with previous research (glavas & schuster, 2020), students mentioned that they used mobile applications frequently which enabled them to collaborate with other group members efficiently and effectively, even when they were working separately and asynchronously (s3). while students were not familiar with communication software used for business, such as microsoft teams, miro and slack, they were excited that, with the help of these digital tools, they can deliver “the same standard of work but online” (s3). furthermore, several wil academics and professionals agreed that online communication provided more comfortable working environments for introverted students and more opportunities for them to speak up in group discussions: industry partners are finding that ...when [students] work together, the more introverted students are speaking out a bit more and the more extroverted students who tend to take over in a classroom don't have that opportunity because it's so much more democratic when you're using [microsoft] teams. (p3) online communication promoted help-seeking, which is an important wl skill in the workplace. students found that they were more confident to initiate help-seeking for more serious or tough problems through emails or phone calls instead of face-to-face communications (s2 and s3). the asynchronous communication allowed deeper and holistic thinking which is necessary for resolving serious problems. remote working made helpseeking through online communication even more essential, as it has not been possible to use other face-to-face channels. a wil student (s1) shared that she appreciated being able to contact friends for help and advice about job hunting remotely, which greatly relieved her stress (s1). students also found, “it is really helpful just to communicate between our group members” (s3). similar to the use of social media, effective and efficient help-seeking online worked well only with pre-established relationships (a6). both students and academics agreed that it is difficult to build new and strong relationships using online communication. for example, a student (s3) reported that online introductions are not as informative as face-to-face introductions, which meant that team members only gained limited understanding of each other. for students in project-based remote wil programs, their engagement with industry was also limited, because industry visits were cancelled due to the covid-19 lockdowns. australasian journal of information systems an, vesty & cheong 2023, vol 27, research article defining digital wellbeing literacy 12 most student participants mentioned that they felt uncomfortable when collaborating online, and it was hard to express emotion in online communication. one major factor was the lack of nonverbal cues to express and perceive wellbeing-related information (a4, e1, e2, e4). one of the wellbeing experts (e2) has provided wellbeing consultancy to clients who could not speak english, and interpreters were invited to translate the conversation. however, the translation of the language could not convey the full wellbeing message, which demonstrates the importance of non-verbal elements from another perspective. students also acknowledged, “the main setback [of online communication] is the lack of nonverbal cues” (s2). facial expressions and body language are important nonverbal aspects to express emotion and wellbeing status, and these can be captured through video camera. however, many students do not turn on their video cameras during group discussions, and for those who do, it was still very hard to connect emotionally, because “you have no idea what people are doing over there: looking at you or they just look at the computer” (s2). in addition, students mentioned that the atmosphere while working face-to-face was much better because they can easily see others’ emotions which makes it easy to empathize with each other (s1). while contextual information and non-verbal cues are not present in online communication, articulation of wellbeing depends more on language itself. thus, wellbeing vocabulary and the logic to articulate wellbeing, as a fundamental part of wl, are even more important to convey wellbeing information. all the wellbeing experts indicated that the general public had very limited “words, terminology or lexicon to talk about wellness” (e3). this matches the students’ self-perceived wl capability, who always found it was difficult to precisely describe their wellbeing situation using language (s4, s5 and s6). if people are talking about wellbeing to an expert, their lack of accurate wellbeing vocabulary will not be a big barrier for the listener to understanding (e1). however, if one is discussing wellbeing to a non-expert who is also not familiar with wellbeing vocabulary, this would be much more likely to cause confusion, especially in online communication without non-verbal cues. this is an extra barrier for non-native speakers to articulate their wellbeing. international students admitted that they could not express feelings and emotions very well because they did not have the appropriate vocabulary in english (s1, s5 and s6). another academic found it was hard to find an exact wellbeing word in english to match her intention (a6). 4.3 the role of emojis in wil students’ digital wellbeing literacy emojis are small digital images or icons conveying ideas or emotions, widely used in digital communications (gamage, 2022; wijeratne et al., 2017). all the student participants found emojis helped to express their emotions in an easy way and avoid misunderstanding in online communications. students also found that emojis facilitate a funny and interesting atmosphere, which promoted friendly conversation. emojis contain emotional information, which is an important aspect of individuals’ wellbeing (ryan & deci, 2001). while the wellbeing experts agreed that most ordinary people have limited vocabulary on emotions, emojis nowadays are “becoming more sophisticated” (e4) in expressing fine-grained emotional information. for example, there are various laughing emojis ranging from smiling with squinty eyes to laughter with tears, which are well recognized by the general public. student participants said they could differentiate the various emojis and developed their own preferences to convey emotion in different contexts. australasian journal of information systems an, vesty & cheong 2023, vol 27, research article defining digital wellbeing literacy 13 conveying emotions in the form of emojis compensated for the absence of nonverbal information in online text communication, and it helped individuals to reflect on their own emotions. a wellbeing expert (e4) pointed out that, even when people communicate face-toface with direct facial expressions, they usually do not think about what emotions should be attached, and emotional context is implicit. however, while using emojis, before transmitting messages to other individuals must be “introspective and have the ability to work out their own emotional response” (e4). this could potentially “enhance people's ability to be introspective and aware of [their own] emotions” (e4). emojis are especially helpful to eliminate language and cultural barriers of wl for non-native speakers. while the lack of emotion vocabulary prevents people from expressing themselves and understanding others’ wellbeing precisely and efficiently, emojis are an “internationally understood language” (e4) and thus can be expressed and recognized regardless of individuals’ native language. 5 discussion our research provides insights into dwl, a new strand of research. we argue that compared with wl in traditional physical settings, the expression and communication about wellbeing in digital context demonstrate different characteristics. in a remote working environment, people rely on digital tools to explore the context and mediate interaction with others. cscw technologies provides multiple channels (such as video, audio, and text), multiple platforms (such as digital conferencing software, digital discussion forum, social media and emails), with various digital tools (such as emojis). the cscw technologies also support both synchronized and asynchronized communication. while some physical context information and non-verbal cues are missed, the cscw technologies mediate interaction and enables individuals to take control of their expression in a more flexible way: they can choose their preferred time and channels to interact with others. this can improve their confidence and ability to communicate, which is especially important for expressing wellbeing related information. for example, wil students who are not familiar with their new colleagues may feel not confident to speak up in front of a group of colleagues and they may choose to keep silent in face-to-face team meetings. however, with digital conferencing tools, they can turn off the camera if this makes themselves more comfortable. they also have the option to write down their opinions in chats, express their feelings through emojis or write emails to discuss questions after the meeting. in a digital context, dwl is closely related to digital literacy (gilster & glister, 1997). the concept of digital literacy evolved from the awareness of skill-based competencies and functional use of technology (gourlay et al., 2014) to further emphasize the social and cognitive implications. this development highlights the extended purpose of digital literacy awareness to foster confident online communication, adoption of technologies to support critical thinking (chan et al., 2017; spante et al., 2018). thus, digital literacy is a critical competency for evaluating and supporting employees (bejaković & mrnjavac, 2020). while he students are relatively familiar with digital technologies, they lack the more nuanced digital culture and social practice experience to function effectively as digital workers. our research showed that wil students’ digital literacy improved with practical cscw tool experience in the workplace. the advancement of an individuals’ digital literacy not only enhanced the wil students wellbeing in a digital context (vissenberg et al., 2022; yue et al., australasian journal of information systems an, vesty & cheong 2023, vol 27, research article defining digital wellbeing literacy 14 2021), but also improved their dwl capability, making them more confident and aware of the most suitable digital methods to communicate for wellbeing purposes. dwl not only demonstrates its importance in a working context, but also its impact on the work life of remote workers. while remote working has the potential to enable a better worklife balance and wellbeing, because of the minimized commute time and improved flexibility in juggling personal commitments (cho et al., 2022; lima & souza, 2017; tamunomiebi & oyibo, 2020), the boundaries between work and life must be continually managed to achieve and maintain improved work-life balance. this is challenging for working professionals, and even more difficult for he students (cho et al., 2022; lim et al., 2017), including wil students. work-life balance has been reportedly a challenge for many students, especially at the early stages of wil programs (an et al., 2021). in a remote working context, it is critical for individual workers to be aware of the wellbeing risks related to the blurred boundaries between work and life and compose discussion with both family members and co-workers to proactively establish and negotiate boundaries. these are all important for developing dwl capabilities. our research provides valuable insights from wil academics, wil administrators, wellbeing experts, and wil students themselves to understand students’ dwl in remote wil programs. every individual has wl capacity because they acquire language and socialization capabilities relating to wellbeing. however, individuals demonstrate very different wl capabilities and people with high wl have increased language use choices for wellbeing purposes (oades et al., 2020). all student interviewees discussed their wellbeing during remote working, which demonstrated that they had basic level of wl, which is using language to explain wellbeing status. students’ wl capabilities are different. for example, some students only used simple and repeated words such as “happy” or “sad” to describe their wellbeing generally, while others had much broader vocabulary to precisely express their feeling in various contexts. some students also demonstrated more sophisticated awareness of wellbeing stressors and more proactive to seek help to mitigate the wellbeing impact. with regards to wl in digital settings, students also demonstrated differentiated dwl capabilities. for example, some felt less confident or insecure when expressing wellbeing issues in digital settings, while others could choose the appropriate digital tools to convey wellbeing information, building up relationship with colleagues, and able to seek help. the findings of the study help to define the concept of dwl as below: defining digital wellbeing literacy digital wellbeing literacy is defined as individuals’ capability to proactively choose the appropriate combination of digital communication platform(s) and digital tools, so that they could comprehend and compose information to improve their wellbeing. recommendations remote collaboration is isolating (fitzhugh & daniels, 2020; hafermalz & riemer, 2016). however, the students in our study proactively used digital tools to connect with their teammates. this not only relieved the students’ stress and loneliness, but also improved their sense of belonging and motivated them to work harder. research on wl is in its infancy and there is even less research on developing dwl for wil students. the remote wil environment is very different from the traditional face-to-face australasian journal of information systems an, vesty & cheong 2023, vol 27, research article defining digital wellbeing literacy 15 working environment and thus calls for different wl strategies. our research has identified key aspects that universities and wil industry partners could focus on to improve students’ dwl to better prepare them for their future careers. the following four strategies were identified to help develop students’ dwl in remote work environments. first, universities and industry partners could facilitate dwl in students by improving their digital competencies (oberländer et al., 2020) and familiarity with cscw tools through formal training. given that dwl is wl through digital media, it can be argued that individual knowledge, skills and abilities to communicate efficiently using digital media are foundational to their dwl. however, most students found the digital communication in remote wil programs unnatural and even frightening, which caused them to refrain from expressing their opinions and emotions. they also had limited experience with business collaboration tools for professional communication before they started their wil program. furthermore, even though the students were frequent social media users (dagli et al., 2020), they were not fully aware of the potential risks of discussing their wellbeing status and seeking support through social media. in the future, universities and host organizations could integrate more comprehensive digital competency training into their formal wil preparation curriculum to ensure students are confident and competent in the digital working context. second, we recommend that universities and industry partners facilitate wil students to establish high quality relationships with co-workers. while building positive and deep relationships in the workplace contributes to workplace wellbeing and provides a foundation for wl, it is difficult for students to establish strong and lasting relationships with colleagues in a digital context. universities and industry partners can carefully design and adjust remote wil programs to remove barriers and create opportunities for interpersonal engagement. constructive and genuine feedback from workplace colleagues is encouraged. social interaction on enterprise social media (esm) with internal co-worker discussion on wellbeingrelated topics could be beneficial in facilitating wil students’ wl development as well as increasing their sense of belonging. while the authenticity of wil programs has been arguably compromised in remote programs because of the reduced quality and frequency of interactions with host organization supervisors (smith, 2012), this could be alleviated somewhat. purposefully designed in-person interactions such as inviting wil students to attend mentor meetings, provide students with opportunities to observe and experience professional cooperation. students can learn to communicate wellbeing in a professional way, which is especially important for their dwl. third, universities should better prepare students to develop wl skills. students reported that they expected practical skills development to help them tackle workplace issues. wellbeing experts also believe that universities should help wil students “to be able to take care of themselves physically, mentally and spiritually at work” (e1). it is beneficial to help students build a rich wellbeing vocabulary so that they can accurately articulate their feelings and their own experiences, which sets a foundation for self-care strategies or help-seeking. micro-credit courses or social media campaigns can be used to promote awareness of wellbeing challenges and facilitate the learning of wellbeing vocabulary and knowledge. wl capability is context base, and universities should provide students with contextualized learning opportunities, such as role play or simulation games, to practice adaptive strategies such as negotiation, conflict resolution or help-seeking and to diminish reliance on maladaptive strategies when they get into the workplace. australasian journal of information systems an, vesty & cheong 2023, vol 27, research article defining digital wellbeing literacy 16 in addition, students should have context awareness in remote work. wellbeing experts (e1 and e3) warn that wellbeing is not the students’ own responsibility and that host organizations play important roles in wil students’ wellbeing. having very limited working experience, wil students may tend to blame themselves in the case of wellbeing challenges, instead of analysing the context (an et al., 2021). universities should prepare students to understand that, where the host organizations’ values contradict students’ own personal values, students should have the confidence and capability to protect themselves, because “people’s values are important to their wellbeing” (e1). finally, it is important for universities and industry partners reposition their mental health services to support students’ dwl development. currently, students perceive university and organizational wellbeing support to be only targeting those students experiencing serious mental health problems. compared with help from family and friends, universities and host organizations can provide more effective, efficient and pertinent support to students. universities can also develop programs to facilitate student engagement with peers and encourage mutual support channels to develop dwl skills. for example, students can share the wellbeing challenges and ethical dilemmas they encounter in the workplace and discuss coping strategies, either in face-to-face groups or virtual social groups facilitated by online platforms. in this way, they not only practice their wl skills, but also improve their sense of belonging to the student community, which is critical to fostering wellbeing. specific to remote work contexts and challenging situations, such as covid-19, digital tools and platforms could be applied for educational purposes to develop students’ dwl. for example, social media platforms match students’ learning style and are beneficial to facilitate higher education students’ learning (dagli et al., 2020). thus, wl educational resources could be developed and distributed by universities through social media to encourage more extensive and in-depth wellbeing-related conversation among wil students. given that dwl is a critical employability capability for students in the post-pandemic era, it is also able to be generalized further to other he students, or even co-workers already in the workplace setting. compared with wil students who can practice professional skills in wil programs, many other he students have limited opportunities to experience the professional workplace and collaborate with colleagues before graduation. the recommendations related to trainings and skills development can also be used to support other he students so that they can better develop their dwl and get prepared for their future workplace. co-workers already in the workplace and shifting to the remote workplace, might also benefit from the recommended upskilling opportunities. 6 conclusion the remote working environment, including remote wil, has created a very different context compared with the traditional physical working environment and ask for distinct strategies to manage wellbeing challenges (nauwerck & cowen forssell, 2018). our research has identified dwl as a new direction of cscw research and provided insights about dwl for students in remote wil programs. although remote wil impacted students’ wellbeing during the pandemic, it also provided students with valuable real-life opportunities to practice dwl in remote work environments. universities and wil host organizations can develop systematic ways to facilitate dwl development to benefit student employability both in the shortand long-term. it is also meaningful to generalize the recommendations of facilitating wil students dwl to all he students and others requiring upskilling, particularly given the australasian journal of information systems an, vesty & cheong 2023, vol 27, research article defining digital wellbeing literacy 17 prevalence of digital practices in the workplace during (worldeconomicforum, 2020). in the post-pandemic era, dwl skills arguably should be the requirement of future work (gartner, 2020). this not only benefits he students but also the future workforce. limitations and future research our analysis is based on the experiences and perspectives of wil academics, administrators and students. in the future, mixed-methods approaches can be used to further enhance the reliability and validity of the results from our study. due to the difficulty in recruiting student participants, we only interviewed six wil students, all of whom were from business degrees and participating in project-based wil programs during covid-19. in future studies, more students from varied disciplines and from different types of remote wil should be interviewed to provide a more comprehensive understanding of wil students’ experiences during remote work and their dwl in various context. furthermore, our study lacks a perspective from industry partners who are critical stakeholders in design and implementation of wil programs and have great impact on students’ dwl development. our study has already identified the trend of wil programs collaborating with international host organizations, which aligns with the worldwide trend of geographically flexible work accelerated by covid-19 (gartner, 2020). cultural and geographic backgrounds have significant impact on workplace collaboration (hofstede et al., 1990; hofstede et al., 2005). while extending their work experience across international borders, wil students will also face additional challenges. this will not only impact students’ wellbeing but also influence their dwl, because wl is a language ability closely related with culture and social background, which can affect an individual’s motivation or confidence to discuss wellbeing issues (an et al., 2021). in addition, “context-awareness” is an important aspect of wl (oades et al., 2021). in international remote wil programs, students need to understand both social and organizational cultures. culture is implicit and is even harder to observe and acknowledge through digital platforms in remote wil. thus, further investigations of dwl in international work collaboration is a promising direction for future research. besides wil students, other he students are also facing challenges in adapting remote learning environments and are expected to enter the future workplace as digitally adept graduates with the ability to work remotely. this provides a future direction for researchers, interested in investigating how to improve dwl for a broader cohort of learners. acknowledgement this research was conducted with ethics approval from the rmit university ethics advisory network under register number 23420 and 24187. we acknowledge the great support from all the interview participants, who have contributed valuable insights to our research. references aczel, b., kovacs, m., van der lippe, t., & szaszi, b. (2021). researchers working from home: benefits and challenges. plos one, 16(3), e0249127. agnew, m., poole, h., & khan, a. (2019). fall break fallout: exploring student perceptions of the impact of an autumn break on stress. student success, 10(3), 45. australasian journal of information systems an, vesty & cheong 2023, vol 27, research article defining digital wellbeing literacy 18 ahmetoglu, y., brumby, d., & cox, a. (2020). a longitudinal interview study on work planning during covid-19 lockdown. microsoft research new future of work workshop, redmond, wa, usa, https://discovery.ucl.ac.uk/id/eprint/10106230/, accessed 5 march 2023. alagaraja, m. (2020). wellbeing in the workplace: a new conceptual model and implications for practice. the palgrave handbook of workplace well-being, 1-22. an, n., cheong, c., & vesty, g. (2021). learning and working online during covid-19: a wellbeing literacy perspective on work integrated learning students. in covid-19 and education: learning and teaching in a pandemic-constrained environment (pp. 193-221). informing science press. andalibi, n., ozturk, p., & forte, a. (2017). sensitive self-disclosures, responses, and social support on instagram: the case of depression. proceedings of the 2017 acm conference on computer supported cooperative work and social computing, portland, usa, 1485-1500. bejaković, p., & mrnjavac, ž. (2020). the importance of digital literacy on the labour market. employee relations: the international journal, 42(4), 921-932. https://doi.org/10.1108/er-072019-0274 bleakley, a., rough, d., edwards, j., doyle, p., dumbleton, o., clark, l., rintel, s., wade, v., & cowan, b. r. (2022, 2022/09/03). bridging social distance during social distancing: exploring social talk and remote collegiality in video conferencing. human–computer interaction, 37(5), 404-432. https://doi.org/10.1080/07370024.2021.1994859 bordi, l., okkonen, j., mäkiniemi, j.-p., & heikkilä-tammi, k. (2018). communication in the digital work environment: implications for wellbeing at work. nordic journal of working life studies. 8(s3), 29-48. braun, v., & clarke, v. (2006, 01/01). using thematic analysis in psychology. qualitative research in psychology, 3, 77-101. https://doi.org/10.1191/1478088706qp063oa bucher, e., fieseler, c., & suphan, a. (2012, 01/01). the stress potential of social media in the workplace. information, communication & society, 16, 1-29. https://doi.org/10.1080/1369118x.2012.710245 bullinger-hoffmann, a., koch, m., möslein, k., & richter, a. (2021). computer-supported cooperative work–revisited. i-com, 20(3), 215-228. burtscher, j., burtscher, m., & millet, g. p. (2020). (indoor) isolation, stress, and physical inactivity: vicious circles accelerated by covid-19? scandinavian journal of medicine & science in sports, 30(8), 1544-1545. https://doi.org/https://doi.org/10.1111/sms.13706 caldeira, c., r.b. de souza, c., machado, l., perin, m., & bjørn, p. (2022). crisis readiness: revisiting the distance framework during the covid-19 pandemic. computer supported cooperative work (cscw). https://doi.org/10.1007/s10606-022-09427-6 carroll, n., & conboy, k. (2020, 2020/07/14/). normalising the “new normal”: changing techdriven work practices under pandemic time pressure. international journal of information management, 55, 102186. https://doi.org/https://doi.org/10.1016/j.ijinfomgt.2020.102186 australasian journal of information systems an, vesty & cheong 2023, vol 27, research article defining digital wellbeing literacy 19 chan, b. s., churchill, d., & chiu, t. k. (2017). digital literacy learning in higher education through digital storytelling approach. journal of international education research (jier), 13(1), 1-16. charoensukmongkol, p. (2014, 2014/07/01/). effects of support and job demands on social media use and work outcomes. computers in human behavior, 36, 340-349. https://doi.org/https://doi.org/10.1016/j.chb.2014.03.061 cho, j., beck, s., & voida, s. (2022). topophilia, placemaking, and boundary work: exploring the psycho-social impact of the covid-19 work-from-home experience. proceedings of the acm human-computer interaction conference, 6(group), article 24. https://doi.org/10.1145/3492843 constantinou, o. (2005). multimodal discourse analysis: media, modes and technologies. journal of sociolinguistics, 9(4), 602. creswell, j. w., & poth, c. n. (2016). qualitative inquiry and research design: choosing among five approaches. thousand oaks, ca, usa: sage publications. dagli, g., altinay, f., altinay, z., & altinay, m. (2020). evaluation of higher education services: social media learning. the international journal of information and learning technology, 38(1), 147-159. https://doi.org/10.1108/ijilt-03-2020-0032 daniels, k., watson, d., nayani, r., tregaskis, o., hogg, m., etuknwa, a., & semkina, a. (2021, 2021/05/01/). implementing practices focused on workplace health and psychological wellbeing: a systematic review. social science & medicine, 277, 113888. https://doi.org/https://doi.org/10.1016/j.socscimed.2021.113888 eshet, y. (2004). digital literacy: a conceptual framework for survival skills in the digital era. journal of educational multimedia and hypermedia, 13(1), 93-106. feng, x., & behar-horenstein, l. (2019). maximizing nvivo utilities to analyze open-ended responses. the qualitative report, 24(3), 563-571. feuston, j. l., & piper, a. m. (2018). beyond the coded gaze: analyzing expression of mental health and illness on instagram. proceedings of the acm on human-computer interaction, 2(cscw), 1-21. https://doi.org/10.1145/3274320 fitzhugh, h., & daniels, k. (2020). a systematic review of reviews on isolated and confined working. propel hub. https://www.propelhub.org/a-systematic-review-of-reviews-on-isolatedand-confined-working/ , accessed 5 march 2023. fitzpatrick, g., & ellingsen, g. (2013). a review of 25 years of cscw research in healthcare: contributions, challenges and future agendas. computer supported cooperative work (cscw), 22(4), 609-665. freudenberg, b., brimble, m., & cameron, c. (2011). wil and generic skill development: the development of business students' generic skills through work-integrated learning. asiapacific journal of cooperative education, 12(2), 79-93. gamage, a. (2022). an inclusive multifaceted approach for the development of electronic work-integrated learning (ewil) curriculum. studies in higher education, 47(7), 13571371. https://doi.org/10.1080/03075079.2021.1894116 https://doi.org/10.1080/03075079.2021.1894116 australasian journal of information systems an, vesty & cheong 2023, vol 27, research article defining digital wellbeing literacy 20 gartner. (2020). future of work trends post-covid-19: long-term impact & actions for hr. gartner. https://www.ahri.com.au/wp-content/uploads/future-of-work-trends-postcovid-19.pdf, accessed 5 march 2023, gilster, p. (1997). digital literacy. ny, usa: wiley computer publisher. glavas, c., & schuster, l. (2020). design principles for electronic work integrated learning (ewil). the internet and higher education in europe, 47, 100760. gourlay, l., hamilton, m., & lea, m. r. (2014). textual practices in the new media digital landscape: messing with digital literacies. research in learning technology, 21(4). https://doi.org/10.3402/rlt.v21.21438 hafermalz, e., & riemer, k. (2016). the work of belonging through technology in remote work: a case study in tele-nursing. ecis, https://aisel.aisnet.org/ecis2016_rp/106 hassan, y., & pandey, j. (2021). demystifying the dark side of social networking sites through mindfulness. australasian journal of information systems, 25. hofstede, g., neuijen, b., ohayv, d. d., & sanders, g. (1990). measuring organizational cultures: a qualitative and quantitative study across twenty cases. administrative science quarterly, 35(2), 286-316. https://doi.org/10.2307/2393392 hofstede, g. h., hofstede, g. j., & minkov, m. (2005). cultures and organizations: software of the mind (vol. 2). ny, usa: mcgraw-hill. hunter, p. (2019). remote working in research: an increasing usage of flexible work arrangements can improve productivity and creativity. embo reports, 20(1), e47435. jackson, d. (2015, 2015/02/07). employability skill development in work-integrated learning: barriers and best practice. studies in higher education, 40(2), 350-367. https://doi.org/10.1080/03075079.2013.842221 kay, j., mcrae, n., & leoni, r. (2020). two institutional responses to work-integrated learning in a time of covid-19: canada and australia. international journal of work-integrated learning, 21(5), 491. keyes, c. l. (2005). mental illness and/or mental health? investigating axioms of the complete state model of health. journal of consulting clinical psychology, 73(3), 539. king, o., davis, c., clemans, a., coles, j., crampton, p., jacobs, n., mckeown, t., morphet, j., seear, k., & rees, c. (2021, 2021/04/03). dignity during work-integrated learning: what does it mean for supervisors and students? studies in higher education, 46(4), 721-736. https://doi.org/10.1080/03075079.2019.1650736 kobau, r., seligman, m. e., peterson, c., diener, e., zack, m. m., chapman, d., & thompson, w. (2011). mental health promotion in public health: perspectives and strategies from positive psychology. american journal of public health, 101(8), e1-e9. koltay, t. (2011). the media and the literacies: media literacy, information literacy, digital literacy. media, culture & society, 33(2), 211-221. lamontagne, a. d., martin, a., page, k. m., reavley, n. j., noblet, a. j., milner, a. j., keegel, t., & smith, p. m. j. b. p. (2014). workplace mental health: developing an integrated intervention approach. bmc psychiatry, 14(1), 1-11. australasian journal of information systems an, vesty & cheong 2023, vol 27, research article defining digital wellbeing literacy 21 lee, e.-j. (2020). authenticity model of (mass-oriented) computer-mediated communication: conceptual explorations and testable propositions. journal of computer-mediated communication, 25(1), 60-73. https://doi.org/10.1093/jcmc/zmz025 lim, h., arawjo, i., xie, y., khojasteh, n., & fussell, s. r. (2017). distraction or life saver? the role of technology in undergraduate students' boundary management strategies. proceedings of the acm on human-computer interaction, 1(cscw), 1-18. lima, y. o. d., & souza, j. m. d. (26-28 april 2017) the future of work: insights for cscw. 2017 ieee 21st international conference on computer supported cooperative work in design (cscwd), wellington, new zealand, 2017, pp. 42-47, doi: 10.1109/cscwd.2017.8066668. magyar, j. l., & keyes, c. l. (2019). defining, measuring, and applying subjective well-being. in positive psychological assessment: a handbook of models and measures. (pp.389-415). washington, d.c., united states, american psychological association. mano, r. s., & mesch, g. s. (2010, 2010/01/01/). e-mail characteristics, work performance and distress. computers in human behavior, 26(1), 61-69. https://doi.org/https://doi.org/10.1016/j.chb.2009.08.005 martin, a., & hughes, h. (2011). how to make the most of work integrated learning. ako aotearoa website, accessed 5 march 2023. https://ako.ac.nz/assets/knowledge-centre/rhpf-c43work-integrated-learning/resource-how-to-make-the-most-of-work-integratedlearning-for-students.pdf martin, a., shann, c., lamontagne, a. d., & health. (2019). promoting workplace mental wellbeing: a rapid review of recent intervention research. handbook of disability, work. handbook series in occupational health sciences, vol 1. springer, cham. https://doi.org/10.1007/978-3-319-75381-2_15-1 mazzola, j. j., schonfeld, i. s., & spector, p. e. (2011). what qualitative research has taught us about occupational stress. stress and health, 27(2), 93-110. https://doi.org/https://doi.org/10.1002/smi.1386 mccrea, p. (2019). is oh being left behind by workplace wellbeing? occupational health and wellbeing, 71(10), 9-9. mikal, j. p., rice, r. e., abeyta, a., & devilbiss, j. (2013, 2013/09/01/). transition, stress and computer-mediated social support. computers in human behavior, 29(5), a40-a53. https://doi.org/https://doi.org/10.1016/j.chb.2012.12.012 nauwerck, g., & cowen forssell, r. (2018). the digital work environment–a challenge and an opportunity for cscw. proceedings of 16th european conference on computer-supported cooperative work-exploratory papers, france, accessed 5 march 2023, eusset digital library nvivo. (2020). fueling academic research. retrieved 10 oct 2020 from https://www.qsrinternational.com/nvivo-qualitative-data-analysissoftware/about/nvivo oades, l., & johnston, a. (2017). wellbeing literacy: the necessary ingredient in positive education. international journal of psychology and behavioral sciences, 82(1), 405-432. https://www.google.com/search?rlz=1c1chbf_en-gbau958au958&sxsrf=ajoqlzxrupfomn1tqife8u-f1rxaibmjma:1679021182280&q=washington,+dc&si=aecpfx6l3rvh8sflhhzyn7jic6m2bu9vmofvfamqv2wwsyjxn4e2qouldzdn2xzlqmmejbhjkr0vs1s-3jpp8ug8p6j9hppizx5-adjsj0p72olsahpfbd07ogjhraubxv6ee1ijdtzl31gcnmxzqssvw2dzkihqf4ynjybv05s3zis65rdjhbxgziybtj_conccwc7-9vtn&sa=x&ved=2ahukewiql4-p-eh9ahxy9dgghwhdcj4qmxmoaxoecgaqaw australasian journal of information systems an, vesty & cheong 2023, vol 27, research article defining digital wellbeing literacy 22 oades, l., ozturk, c., hou, h., & slemp, g. (2020, 2020/09/02). wellbeing literacy: a languageuse capability relevant to wellbeing outcomes of positive psychology intervention. the journal of positive psychology, 15(5), 696-700. https://doi.org/10.1080/17439760.2020.1789711 oades, l., slade, m., & jarden, a. (2017). wellbeing and recovery. in slade, m., oades, l., & jarden, a. (eds.), wellbeing, recovery mental health (pp. 324-332), cambridge university press, uk. oades, l. g., jarden, a., hou, h., ozturk, c., williams, p., r. slemp, g., & huang, l. (2021). wellbeing literacy: a capability model for wellbeing science and practice. international journal of environmental research and public health, 18(2), 719. https://www.mdpi.com/1660-4601/18/2/719 oberländer, m., beinicke, a., & bipp, t. (2020). digital competencies: a review of the literature and applications in the workplace. computers education and training, 146, 103752. oliver, b. (2015, 09/02). redefining graduate employability and work-integrated learning: proposals for effective higher education in disrupted economies. journal of teaching and learning for graduate employability, 6, 56. https://doi.org/10.21153/jtlge2015vol6no1art573 panteli, n., yalabik, z. y., & rapti, a. (2019). fostering work engagement in geographicallydispersed and asynchronous virtual teams. information technology people, 32(1), 2-17. parsa, m. s., & golab, l. (2021). social media mining to understand the impact of co-operative education on mental health. international journal of work-integrated learning, 22(4), 521537 pascoe, m. c., hetrick, s. e., & parker, a. g. (2020, 2020/12/31). the impact of stress on students in secondary school and higher education. international journal of adolescence and youth, 25(1), 104-112. https://doi.org/10.1080/02673843.2019.1596823 patrick, c.-j., peach, d., pockknee, c., webb, f., fletcher, m., & pretto, g. (2008). the wil (work integrated learning) report : a national scoping study [australian learning and teaching council (altc) final report]. queensland university of technology, australia. https://eprints.qut.edu.au/80768/ pretti, t. j., etmanski, b., & durston, a. (2020). remote work-integrated learning experiences: student perceptions. international journal of work-integrated learning, 21(4), 401-414. ryan, r. m., & deci, e. l. (2001). on happiness and human potentials: a review of research on hedonic and eudaimonic well-being. annual review of psychology, 52(1), 141-166. https://doi.org/10.1146/annurev.psych.52.1.141 schuster, l., & glavas, c. (2017). exploring the dimensions of electronic work integrated learning (ewil). educational research review, 21, 55-66. smith, c. (2012, 2012/04/01). evaluating the quality of work-integrated learning curricula: a comprehensive framework. higher education research & development, 31(2), 247-262. https://doi.org/10.1080/07294360.2011.558072 smith, c., & worsfold, k. (2015, 2015/01/02). unpacking the learning–work nexus: ‘priming’ as lever for high-quality learning outcomes in work-integrated learning curricula. studies in higher education, 40(1), 22-42. https://doi.org/10.1080/03075079.2013.806456 australasian journal of information systems an, vesty & cheong 2023, vol 27, research article defining digital wellbeing literacy 23 smith, r., morgan, j., & monks, c. (2017, 2017/03/01/). students' perceptions of the effect of social media ostracism on wellbeing. computers in human behavior, 68, 276-285. https://doi.org/https://doi.org/10.1016/j.chb.2016.11.041 spante, m., hashemi, s. s., lundin, m., & algers, a. (2018, 2018/01/01). digital competence and digital literacy in higher education research: systematic review of concept use. cogent education, 5(1), 1519143. https://doi.org/10.1080/2331186x.2018.1519143 stich, j.-f., farley, s., cooper, c., & tarafdar, m. (2015). information and communication technology demands: outcomes and interventions. journal of organizational effectiveness: people performance, 2(4), 327-345. tamunomiebi, m. d., & oyibo, c. (2020). work-life balance and employee performance: a literature review. european journal of business and management research, 5(2). https://doi.org/10.24018/ejbmr.2020.5.2.196 venigalla, a. s. m., chimalakonda, s., & vagavolu, d. (2020). mood of india during covid-19 an interactive web portal based on emotion analysis of twitter data conference companion publication of the 2020 on computer supported cooperative work and social computing, virtual event, usa. https://doi.org/10.1145/3406865.3418567 verduyn, p., ybarra, o., résibois, m., jonides, j., & kross, e. (2017). do social network sites enhance or undermine subjective well-being? a critical review. social issues and policy review, 11(1), 274-302. https://doi.org/https://doi.org/10.1111/sipr.12033 vissenberg, j., d’haenens, l., & livingstone, s. (2022). digital literacy and online resilience as facilitators of young people’s well-being? european psychologist, 27(2), 76-85. https://doi.org/10.1027/1016-9040/a000478 wang, j.-l., jackson, l. a., gaskin, j., & wang, h.-z. (2014, 2014/08/01/). the effects of social networking site (sns) use on college students’ friendship and well-being. computers in human behavior, 37, 229-236. https://doi.org/https://doi.org/10.1016/j.chb.2014.04.051 welsh, e. (2002). dealing with data: using nvivo in the qualitative data analysis process. forum qualitative sozialforschung / forum: qualitative social research, 3(2). https://doi.org/10.17169/fqs-3.2.865 white, c. a., uttl, b., & holder, m. d. (2019). meta-analyses of positive psychology interventions: the effects are much smaller than previously reported. plos one, 14(5), e0216588. https://doi.org/10.1371/journal.pone.0216588 wijeratne, s., balasuriya, l., sheth, a., & doran, d. (2017). a semantics-based measure of emoji similarity proceedings of the international conference on web intelligence, leipzig, germany, 646-653. https://doi.org/10.1145/3106426.3106490 world economic forum. (2020). the future of jobs report 2020. w. e. forum. http://www3.weforum.org/docs/wef_future_of_jobs_2020.pdf, accessed 5 march 2023. yang, l., holtz, d., jaffe, s., suri, s., sinha, s., weston, j., joyce, c., shah, n., sherman, k., & hecht, b. (2022). the effects of remote work on collaboration among information workers. nature human behaviour, 6(1), 43-54. yue, a., pang, n., torres, f., & mambra, s. (2021). developing an indicator framework for digital wellbeing: perspectives from digital citizenship. (nusctic working paper australasian journal of information systems an, vesty & cheong 2023, vol 27, research article defining digital wellbeing literacy 24 series no. 1). https://ctic.nus.edu.sg/resources/ctic-wp01(2021).pdf, accessed 5 march 2023. zhai, y., & du, x. (2020, 2020/06/01/). addressing collegiate mental health amid covid-19 pandemic. psychiatry research, 288, 113003. https://doi.org/https://doi.org/10.1016/j.psychres.2020.113003 copyright: © 2023 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v27i0.3969 https://ctic.nus.edu.sg/resources/ctic-wp01(2021).pdf http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 literature review 2.1 remote work integrated learning (wil) and its wellbeing impact 2.2 wellbeing literacy in the workplace 2.3 toward defining digital wellbeing literacy (dwl) – a new strand of research 3 methodology 3.1 conceptual framework 3.2 method 3.3 participants 3.4 data collection and analysis 4 findings 4.1 wellbeing literacy and the use of social media 4.2 developing digital wellbeing literacy through online communication 4.3 the role of emojis in wil students’ digital wellbeing literacy 5 discussion 6 conclusion paper title (paper title style) australasian journal of information systems bunker, knight & stieglitz 2022, vol 26, research on applied ethics special section editorial 1 editorial for the special section on research on applied ethics: developing ethical guidelines for social media analytics deborah bunker the university of sydney, australia deborah.bunker@sydney.edu.au shirlee-ann knight curtin university, australia stefan stieglitz university of duisburg-essen, germany the enormous growth of social media usage has led to an increasing accumulation of data, which has been termed social media big data. social media platforms offer many possibilities of data formats, including textual data, pictures, videos, sounds, and geolocations (stieglitz et al., 2018). this diverse social media data has spawned numerous attractive opportunities for researchers and practitioners to analyse social media users and their behaviour. we have seen the use of social media data for the benefit of individuals and society, but we have also seen problems emerge when social media data is not managed and used within specific frameworks and with specific intent. for example, during crisis situations like the recent onset of the covid-19 pandemic, social media data analysis could have been used to develop better situational awareness. this in turn would have allowed tailoring of messaging and the propagation of public health information in real-time, such as the location of outbreaks, to help minimise human movement and resulting health impacts. if not managed and analysed properly, however, social media data might have the opposite effect by causing complacency in those who are not directly impacted by the situation (mirbabaie et al., 2020). social media data, and its analysis, is complex. for companies, social media data can be investigated to identify new trends or ways to improve their products, but equally it can be used to damage their reputation by disgruntled or malicious parties. therefore, social media data can be analysed not only by researchers, who are obliged to adhere to specific, and ethical, frameworks, but also by journalists, political parties, and companies to sell their products and ideas to social media users, who are most open to their messages, but can equally be used to cause panic, complacency, or reputational damage. this form of microtargeting raises a scientific and societal discussion about the ethical implications of profiling social media users. while some users might enjoy being confronted with news, politics and advertisements which match their mindset, the profiling of social media users can be a double-edged sword. the often highly personal data i.e., extracted social media data, can range from sexual orientation and religious beliefs to ethnic background, and might be misused in ways that we previously outline. for instance, undemocratic societies could be interested in identifying potential regime opponents, and the leaders of democratic societies can misuse social media data to spread fake news and influence opinion formation processes, as was highlighted by the cambridge analytica scandal. furthermore, social media analytics, as an analysis of big data with help of machine learning algorithms, is confronting is researchers with typical epistemic concerns. the way that australasian journal of information systems bunker, knight & stieglitz 2022, vol 26, research on applied ethics special section editorial 2 conclusions are drawn from the data that is tracked by researchers can lead to inconclusive, inscrutable, and misguided evidence (mittelstadt et al., 2016). so, there is a huge ethical concern for academics and industry alike, as wrong conclusions about individuals might be drawn, bearing in mind that social networks do not always reflect the society as a whole e.g., the widely researched platform twitter attracts a special user group (boyd and crawford, 2012). researchers sometimes lose track of what are simple correlations of data and what can be interpreted as a causal connection (illari and russo, 2014). in reaction to numerous data scandals, many official and legislative bodies have developed stricter data protection regulations. for instance, the general data protection regulation of the european union came into force in 2018 and it confronts social media researchers with new standards on how to track, store and analyse data (eu, 2018). researchers now find themselves in a conflict with individual data protection laws and regulations and research objectives including the analysis of personal data that may contribute to the greater good of society (bunker et al., 2019). if this conflict limits the possibilities of researchers to investigate and understand social media platforms, it will increase the knowledge divide between platform providers, that own and use all the data, and researchers who are limited by restricted access, terms of trade and ethical concerns. the analysis of social media data in crisis situations to support emergency service agencies, is one example where such research has a morally good aim. zook et al., (2017) developed rules for responsible big data research, so to motivate this special section of ajis, and with this type of approach in mind we posed the question what constitutes ethical rules and approaches of responsible social media analysis? this special section presents four very different responses. marx & mirbabaie (2022) motivate the ethics of social media analytics (sma) debate by developing a scoping review of the extant literature to outline a critical research agenda for is scholarship. their review outlines eight fundamental principles for ethical sma research, to develop a starting point for research ethical conduct. simultaneously, their research also highlights the many and varied ethical dilemmas that are yet to be solved. they urge is scholars to find their distinctive voice in the debate on sma research ethics. eismann, fischer-preßler & fischbach (2022) analyse covid-19 contract tracing apps that have been adopted in australia, france, germany, japan, and new zealand identifying the normative and technical principles in their privacy-sensitive design. by using the restricted access/limited control (ralc) account of information privacy for their analysis, they discuss how these apps protect user privacy and develop an understanding of information privacy from the designs of these apps. their findings can be used to inform future app design and development of privacy approaches under certain contexts. their work in their own word "bridges the gap between ethical design guidelines and technical analyses of specific implementations". jung, clausen, franzke & marx (2022) propose an sma framework adapted from stieglitz et al. (2018) by extending it using the design science research (dsr) cycles as developed by hevner (2007). they extend the sma framework by incorporating ethical reflection phases assessed through evaluation by focus groups and questionnaires with ethics board members and sma experts. their extended framework provides simplified ethical guidance for australasian journal of information systems bunker, knight & stieglitz 2022, vol 26, research on applied ethics special section editorial 3 researchers and facilitates ethical self-reflection where projects involve the use of social media data. finally, soh, smith & dhillon (2022) look at the relationship between social capital and social media addiction by looking at privacy self-efficacy. they highlight the differences in social media users' usage purposes i.e., some users build network bridges, while others focus on increasing the strength of network bonds. they conclude that "the relationship between social capital and social media addiction is moderated by social media user privacy self-efficacy". their broader findings relating to this conclusion make a valuable theoretical contribution to sma research and have implications for practice. we know that you will find these papers make an interesting and valuable contribution to the debate on ethical rules and approaches for responsible sma and take this area of enquiry further forward by opening debate within our field of enquiry. references boyd, d. and crawford, k. (2012). “critical questions for big data: provocations for a cultural, technological, and scholarly phenomenon.” information, communication & society, 15(5), pp. 662-679. bunker, d., stieglitz, s., ehnis, c., and sleigh, a. (2019). “bright ict: social media analytics for society and crisis management”. in y. dwivedi, e. ayaburi, r. boateng, j. effah (eds.), ict unbounded: social impact of bright ict adoption: ifip wg 8.6 international conference on transfer and diffusion of it, tdit 2019, accra, ghana, june 21-22, 2019, proceedings, (pp. 536-552). cham: springer. european union (2018) general data protection regulation. accessed june 15, 2022, at www.gdpr-info.eu illari, p. and russo, f. (2014). causality: philosophical theory meets scientific practice. oxford: oxford university press. hevner, a. r. (2007). a three cycle view of design science research. scandinavian journal of information systems, 19(2), 87–92. mirbabaie, m., bunker, d., stieglitz, s., marx, j., ehnis, c. (2020). "social media in times of crisis: learning from hurricane harvey for the coronavirus disease 2019 pandemic response". journal of information technology, 35(3), pp. 195-213. mittelstadt, b., allo, p., taddeo, m., wachter, s. and floridi, l. (2016). "the ethics of algorithms: mapping the debate”, big data & society, 3(2), pp. 1-21. stieglitz, s., mirbabaie, m., ross, b. and neuberger, c. (2018) "social media analytics – challenges in topic discovery, data collection, and data preparation” international journal of information management, 39, pp. 156-168. zook, m., barocas, s., boyd, d., crawford, k., keller, e., gangadharan, s.p., et al. (2017) "ten simple rules for responsible big data research". plos computational biology, 13(3), e1005399. https://doi.org/10.1371/journal.pcbi.1005399 australasian journal of information systems bunker, knight & stieglitz 2022, vol 26, research on applied ethics special section editorial 4 copyright: © 2022 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v26i0.4063 http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems ceric & parton 2024, vol 28, research article what prevents e-hrm potential? 1 what prevents organisations from achieving e-hrm potential? arnela ceric charles sturt university, australia aceric@csu.edu.au kevin parton charles sturt university, australia abstract use of electronic human resource management (e-hrm) offers the prospect of enabling the human resource management (hrm) function to take on a strategic partner’s role in organisations. despite the pervasive expansion of e-hrm use, there is no clear understanding of why organisations are not achieving e-hrm potential. we address this issue by investigating e-hrm adoption factors and their influence on information technology (it) use potential to automate, informate and transform the hrm function in a sequential manner. in particular, we examine hrm professionals’ experiences with e-hrm use, including challenges, successes, and outcomes. we identified e-hrm adoption factors that enable and that constrain each stage of e-hrm use. with a focus on the inhibiting factors, our findings suggest that e-hrm potential hindered already in the automation stage diminishes e-hrm potential to subsequently informate and to transform the e-hrm function. keywords: human resource management, information systems, e-hrm adoption, e-hrm challenges, strategic e-hrm outcomes. 1 introduction the human resource management (hrm) function is faced by a dynamic and complex environment (parry & battista, 2019), with an expectation that it will evolve from its traditional administrative role to becoming a strategic business partner (ulrich, 1997). technology, in the form of electronic hrm (e-hrm), is seen as a critical resource for transforming the hrm function, enhancing its strategic contribution to the business (strohmeier, 2020; van der berg et al., 2020; bondarouk et al., 2017; marler & parry, 2016) and improving organisational performance. while e-hrm adoption is an important and growing trend (larkin, 2017), a considerable number of e-hrm adoptions have failed (bondarouk et al., 2017; martin & reddington, 2010; marler, 2009) and the organisations adopting e-hrm are struggling to obtain the desired e-hrm outcomes (bondarouk et al., 2017). the results of empirical research concerning e-hrm outcomes are controversial. some researchers found that positive outcomes are missing. for example, gardner et al. (2003) and martin and reddington (2010) found that e-hrm relieved hrm professionals from their administrative burden, only to replace it with doing it-support activities. parry (2011) reported that e-hrm increased the hrm function’s involvement in delivering strategy but did not result in any reduction in the headcount of hrm employees. based on a review of quantitative e-hrm studies, marler and fisher (2013) found that there is not sufficient evidence that e-hrm delivers e-hrm outcomes. australasian journal of information systems ceric & parton 2024, vol 28, research article what prevents e-hrm potential? 2 this conclusion was echoed in other studies (e.g., panos & bellou, 2016; strohmeier, 2009). more recently, however, zhou et al. (2022) conducted a meta-analysis of the e-hrm literature and established a positive relationship between e-hrm and its outcomes. these mixed findings reported in the literature may have multiple causes. first, studies exploring e-hrm outcomes have used different performance measures, as identified by zhou et al. (2022). second, focusing on e-hrm outcomes alone may not be enough. studies may need to capture the specific organisational context and account for differences in e-hrm goals, processes, and technologies amongst organisations (morris et al., 2009). for example, pobanzaou et al. (2020) developed a taxonomy of organisations based on considerations of business value in their motivation to adopt e-hrm. they identified seven clusters of organisations based on their differences in e-hrm motivation. the relationship between intended and realised e-hrm goals was further explored by parry and tyson (2011) in ten uk organisations. they concluded that e-hrm does not simply lead to desired e-hrm outcomes, and they identified a number of factors influencing their realisation. in another study conducted in greek organisations, panos and bellou (2016) also found that the relationship between intended and realised e-hrm outcomes stops being straightforward as soon as additional factors are introduced. third, e-hrm studies have considered a range of factors influencing e-hrm adoption, but have not done this systematically. for example, bondarouk et al. (2017) identified 168 adoption factors being explored in e-hrm studies over four decades. the sheer number of e-hrm adoption factors makes their practical use cumbersome and difficult to include in one study. fourth, e-hrm outcomes are dependent on how e-hrm is used, and yet, e-hrm use was mainly studied at an individual level in the e-hrm literature. zuboff’s (1988) renowned model of information technology (it) use which conceptualises it use in the three stages of automation, informating and transformation provides a foundational categorisation of it use. moreover, the three stages of e-hrm use capture universal it characteristics and overall e-hrm value potential which organisations expect to achieve from e-hrm adoption (strohmeier & kabst, 2009). as such, examining a small number of sequential use categories like zuboff’s automation, informating and transformation may give useful traction to understanding e-hrm research and practice. yet, there is only one study in the ehrm literature which used this model. gardner et al. (2003) empirically demonstrated that each stage of e-hrm use leads to different e-hrm outcomes. this, we believe, is an important contribution to the e-hrm literature, signifying an important avenue for exploring e-hrm outcomes. however, as this assumption has received little attention in the literature previously, the link between each stage of e-hrm use and specific adoption factors has not been established. in this study we combine zuboff’s three-stage model of it use (zuboff, 1988; burton-jones, 2014) with bondarouk et al.’s (2017) top (technology, organisation, people) framework of e-hrm adoption factors as a theoretical framework. this provides a specific conceptualisation of ehrm use which, together with e-hrm adoption factors, captures specific organisational contexts and explains differences in e-hrm outcomes and helps identify the hrm adoption factors that enable and that constrain each stage of e-hrm use. we validate our proposed framework and with a focus on the inhibiting factors we set out to answer the following research question: australasian journal of information systems ceric & parton 2024, vol 28, research article what prevents e-hrm potential? 3 what prevents organisations from achieving e-hrm potential? the paper is organised as follows. the theoretical framework used in this study is presented in the next section and it integrates zuboff’s (1988) three-stage model of it use and bondarouk et al.’s (2017) top framework. this study employed a qualitative research method to enable an in-depth understanding of why and how e-hrm was progressing in the organisations studied (myers, 2019). semi-structured interviews were conducted with 17 hrm professionals from australia and analysed using thematic analysis. the findings are then presented and discussed, followed by a presentation of limitations and future research and some concluding remarks. 2 theoretical framework as the theoretical framework for this research we combine the three-stage model of it use: automation, informating, and transformation, developed by zuboff (1988) (burton-jones, 2014) with bondarouk et al.’s (2017) top framework of e-hrm adoption factors. 2.1 the three-stage model of it use zuboff’s model is based on her observations and interviews with employees in eight organisations over a period of five years. the three stages of it use are inherent in it characteristics to collect, record, store and manipulate data (strohmeier & kabst, 2009). for example, automation of business activities inevitably and unavoidably leads to informating, that is, creation of new information on business activities, processes and users, which then transforms and redesigns the business activities. the three stages in zuboff’s (1988) model need to be developed to be accessed (gardner et al., 2003). that is, they are developmental, and each stage is a necessary condition for the next one (zuboff, 1988). for example, informating is a by-product of automation, and both, informating and automation are a prerequisite for transformation. the first stage of it use is automation, it’s capacity to replace people and perform the same tasks and activities with “more certainty and control” (zuboff, 1988, p.9). e-hrm in this stage is used primarily to automate manual, often transactional, and administrative hrm activities and processes such as payroll (gardner et al., 2003). automation can lead to operational ehrm outcomes which include cost and time savings for hrm professionals, reducing both the administrative burden, and the number of hrm professionals (e.g., parry & tyson, 2011; reddick, 2009; ruël at al., 2004; ruël et al., 2007; strohmeier, 2007; ball, 2001; hussain et al., 2007; haines & lafleur, 2008). achieving operational outcomes is the main motivation for adopting e-hrm in organisations (zhou et al., 2022; poba-nzaou et al., 2020; panos & bellou, 2016; parry & tyson, 2011). the second stage of it use is ‘informating’, a term coined by zuboff to explain it’s capacity to translate work processes, “activities, events and objects…into information… so that they become visible, knowable, and sharable in a new way” (zuboff, 1988, pp. 9-10). e-hrm generates information about the hrm activities and processes and makes it accessible to hrm professionals for evaluation and improvements. e-hrm can accumulate comprehensive employee data which hrm professionals can easily access and use to address specific hrm issues (strohmeier & kabst, 2009), for example, through modelling and projection. informating leads to relational e-hrm outcomes such as hr service improvements for employees and managers (ruël at al., 2004), information responsiveness and information autonomy of hrm australasian journal of information systems ceric & parton 2024, vol 28, research article what prevents e-hrm potential? 4 professionals (gardner et al., 2003). iqbal et al. (2019) found that quality of hr service mediates the relationship between e-hrm practices and employee productivity. employees may perceive that use of e-hrm leads to more reliable and fair decisions (menant et al., 2021). however, employees and managers can resist taking the burden of conducting transactional hrm activities (martin & reddington, 2010). talukdar and ganguly (2022) warn that e-hrm significantly reduces hr socialisation, and consequently, perceived hr effectiveness. additionally, informating potential is not often recognised during e-hrm adoption (shrivastava & shaw, 2003) as organisations become aware of this potential only with time or by chance (tansley et al., 2014). the third stage of it use is transformation, it’s capacity to “reconfigure the nature of work and the social relationships that organize productive activity” (zuboff, 1988, pp.10-11). ehrm enables networking and collaboration of hrm professionals, line managers, employees and job applicants which can lead to innovative forms of organising hrm (strohmeier & kabst, 2009), as well as change in culture and how hrm professionals use their time (gardner et al., 2003). the transformation stage is expected to metamorphose the hrm function from an administrative focus to becoming a strategic business partner (reddick, 2009; panayotopoulou et al., 2007) by allowing hrm professionals to spend more time on strategic activities (gardner et al., 2003) and organisational issues such as risk management and innovation (ruël et al., 2004). this is enabled by access to e-hrm information, and e-hrm’s capabilities in advanced reporting and sophisticated analysis (bondarouk et al., 2017). it is important to note that zuboff’s (1988) model, though called a stage model as part of her theory, is not a stage model in the classical normative sense. while she proposed three interdependent stages based on it characteristics, these stages have not been designed or intended for assessing situations in organisations and guiding potential improvements. stage models describe how it evolves in organisations, but they do not explain why they evolve the way they do (debri & bannister, 2015), while zuboff’s model was developed based on that very reason – it use progresses based on it characteristics. as a result, the progression between the stages is not something that can be managed as it can be in a stage model and thus zuboff’s model is not a prescriptive one, rather it is descriptive. this is an important difference from typical stage models described in the literature that are created to evaluate and prescribe different stages of it/is growth, locate where the organisation is at any given moment, and suggest what the next stage should look like. we apply her model as a microscope through which we investigate e-hrm adoption and related challenges experienced by hrm professionals, and then assess whether this can explain the conflicting research findings on e-hrm outcomes. the analysis of the data collected from the interviews we conducted (see the following sections) confirms that this is indeed a useful way to look at e-hrm. 2.2 the top framework of e-hrm adoption factors e-hrm does not guarantee improvements in the hrm function (parry & tyson, 2011). instead, e-hrm success depends on a range of contextual factors (bondarouk & ruël, 2013; marler & fisher, 2013; schalk et al., 2013). in their literature review spanning four decades of research, bondarouk et al. (2017) identified 168 adoption factors which affect e-hrm adoption and consequences, and noted that the e-hrm adoption literature classifies the identified factors as enablers or as barriers depending on their influence on e-hrm adoption. they further classified these factors into technological, organisational and people (top) contexts. australasian journal of information systems ceric & parton 2024, vol 28, research article what prevents e-hrm potential? 5 in their study the authors “define e-hrm adoption as the strategy and transfer process between an old (or non-existent) and a targeted e-hrm system, and its acceptance by the users” (bondarouk et al., 2017, p. 104), a definition which we also apply in this study. we now provide a brief overview of the adoption factors in each of the three top contexts. 2.2.1 technological context bondarouk et al. (2017) identified data integrity, system usefulness, system integration, and in-house development versus using external hris software, as e-hrm adoption factors in the technological context. designing the e-hrm system in a way that addresses hrm and organisational need for e-hrm information is important for e-hrm success (parry & tyson, 2011). however, this requires strategic e-hrm planning and implementation (lengnick-hall & moritz, 2003; parry, 2011; schalk et al., 2013), integration between e-hrm modules (bondarouk & ruël, 2013), as well as integration between e-hrm, hrm needs and business processes (hannon et al., 1996; ruël & kaap, 2012) and technological infrastructure (reddick, 2009). if not present, these factors can result in a view of e-hrm “as a costly distraction that does not add to competitive advantage” (sheehan, 2009, p.245). 2.2.2 organisational context organisational context consists of organisational characteristics, planning and project management, data access, and capabilities and resources (bondarouk et al., 2017). in addition, organisational size is an important indicator of the availability of organisational resources (such as it infrastructure, training and technical support) for promoting e-hrm adoption (strohmeier & kabst, 2009; zhou et al., 2022). if missing, organisational adoption factors can severely limit e-hrm success. the literature has identified e-hrm adoption challenges stemming from organisational context, such as lack of top management support (schalk et al., 2013), a limited budget for e-hrm adoption (reddick, 2009), inadequate resources (bondarouk et al., 2017), culture closed to e-hrm use (parry & tyson, 2011; sheehan & de cieri, 2012) and to the hrm function taking a business partner’s role (voermans & van veldhoven, 2007; dery & wailes, 2005), weak alignment between e-hrm and organisational strategy (marler, 2009), and poor data access, security and privacy (bondarouk et al., 2017). panos and bellou (2016) found that the type of role pursued by hrm in the organisation determines the type of e-hrm outcomes being achieved. they found that administrative experts tend to achieve operational and relational outcomes, and change strategists achieve transformational outcomes. 2.2.3 people context according to bondarouk et al.’s (2017) classification, the people context includes top management support, user acceptance, communication, and collaboration between it and hrm departments, hr skills and expertise, and leadership and culture. bondarouk et al. (2017) found this to be the most important context for influencing e-hrm adoption and its consequences. moreover, nyathi and kekwaletswe (2023) found that employee performance mediates the relationship between e-hrm use and organisational performance, thus emphasising the benefits of informating in terms of improved information flows, involvement of employees in decision making and giving them new development opportunities. however, this rests on employees’ perception of e-hrm as useful and easy to use in achieving their goals (zhou et al., 2022; nyathi & kekwaletswe, 2023). if not, the resulting e-hrm information can be untimely and inaccurate (dery & wailes, 2005). australasian journal of information systems ceric & parton 2024, vol 28, research article what prevents e-hrm potential? 6 the people context can also limit the success of e-hrm adoption. for example, hrm professionals’ limited it skills (panayotopoulou et al., 2007), their questionable statistical skills (sheehan & de cieri, 2012) and low quality of data analysis (wiblen et al., 2012) can all prevent transformation, that is, use of e-hrm for supporting and initiating strategic decisions (dery & wailes, 2005). users’ acceptance was found to be a critical factor for e-hrm adoption (e.g., menant et al., 2021; bondarouk & ruël , 2013; martin & reddington, 2010; panos & bellou, 2016) as it leads to positive e-hrm outcomes (marler & fisher, 2012) and it increases the efficiency of hrm activities and perceived effectiveness of hrm practices (gardner et al., 2003; ruël et al., 2007). similarly, obeidat (2016) found that e-hrm user intention mediates the relationship between e-hrm determinants (performance expectancy and social influence) and hrm use. if missing, it can cause employee rejection and resistance to e-hrm use (voermans & van veldhoven, 2007), from cynicism and opposition to sabotage (martin & reddington, 2010; reddick, 2009), such as continuing to use traditional offline systems rather than an e-hrm system (parry & tyson, 2011). ultimately, this can reduce the performance and effectiveness of the organisation (panos & bellou, 2016). users’ resistance to e-hrm may be related to inadequate change management (reddick, 2009), ineffective e-hrm implementation processes (reddington & hyde, 2008), security/privacy fears (reddick, 2009; lau & hooper, 2009), users’ e-hrm knowledge (zhou et al., 2022) poor training in using ehrm (parry & tyson, 2011; nyathi & kekwaletswe, 2023) and perception that they are ‘doing hr’s job’, with a consequence of actual or perceived work overload (martin & reddington, 2010). gardner et al. (2003) found that the adoption of e-hrm simply replaces administrative duties with technological support for employees. 2.3 the combined framework to study e-hrm adoption and use in this study, we open new territory by exploring these e-hrm adoption factors and their influence on each of the three stages of e-hrm use. that is, we propose that it is not sufficient to know which factors to consider in the e-hrm adoption, but for e-hrm adoption to be successful, the influence of these factors on different stages of e-hrm use needs to be further understood and considered. this will provide further understanding of realised e-hrm outcomes, or of reasons for not realising them. thus, in this study we combine and integrate, zuboff’s three-stage model of e-hrm use and bondarouk et al.’s (2017) top framework. this is presented in figure 1, where zuboff’s stage 1, automation, is characterised by a specific configuration of technology, organisation, and people contexts {ta, oa, pa}. success at this stage opens the possibility of informating. if informating proceeds for the organisation concerned, it does so based on the experience of automation represented by {ta, oa, pa}, which is carried through to the informating stage, though additional aspects of automation are also likely to develop during this second stage. then success at the second stage, informating, can lay the foundation for stage three, transformation. again, the previous experience of automation and informating {ta + ti, oa + oi, pa + pi}, influences the way in which transformation can unfold. australasian journal of information systems ceric & parton 2024, vol 28, research article what prevents e-hrm potential? 7 figure 1. theoretical framework used in this study note. t stands for technology context, o for organisation context and p for people context; subscript letters a, i and t stand for automation, informating and transformation. the developmental nature of the three stages of e-hrm use accentuates the need to identify factors which influence each stage of e-hrm use, especially those that present e-hrm challenges which inhibit e-hrm use. this is a prerequisite for achieving transformation of the hrm function. 3 method 3.1 data collection this study used a qualitative research method. semi-structured interviews were conducted with 17 hrm professionals from australian organisations in 2017. a semi-structured interview is a data collection method based on a written interview guide that contains predetermined questions based on the research objectives (given, 2008). as a result of each interviewee being asked the same general questions, the reliability of the findings increases. additionally, this interview method also allows flexibility so that the interviewer can seek clarification, ask supplementary questions on the issues brought up by the interviewee, and may change question wording (rowley, 2012). interviewees were asked about e-hrm used in their organisation, challenges they had experienced with e-hrm in relation to automation, informating and hrm transformation across different contexts, from technology, to users, the organisation and hrm. the interview guide is provided in the appendix. the interviews lasted approximately 60 minutes each, were digitally audio-recorded with the interviewees’ consent, and transcribed. interviews were conducted until saturation was reached: “the point at which no new information or themes are observed in the data” (guest et al., 2006, p.59), resulting in 17 interviews with hrm professionals. while all themes were identified after the 7th interview, there was still some variation in some aspects of identified themes. for example, the 13th interview provided insights into issues pertaining to the it department driving e-hrm implementation, an aspect of the hrm and it collaboration theme. although saturation was reached at this point and no new themes were generated afterwards, additional interviews were conducted to confirm that there were no new themes emerging. for example, guest et al. (2006) established that 92 per cent of all codes are identified within the first 12 interviews, and kuzel (1992) advises that 12 to 20 data sources are sufficient to achieve maximum code variation. other researchers in the e-hrm area have used a similar number of interviews to explore e-hrm adoption. for example, schalk et al. (2013) conducted seven interviews, and troshani et al. (2011) 16 interviews. data saturation in this study ensured the content validity of the findings (yin, 2003). australasian journal of information systems ceric & parton 2024, vol 28, research article what prevents e-hrm potential? 8 3.2 participants there were four criteria for recruiting hrm professionals for this study: a) participants have been part of the e-hrm implementation team, b) participants must use e-hrm applications in their respective organisations, c) participants must work in an organisation that has used ehrm for a minimum of one year, and d) e-hrm in the organisation has to be used for more than just basic hrm activities such as payroll. two main strategies were used in recruiting interviewees. first, emails with information on the research project were sent directly to hrm professionals. second, the chamber of commerce assisted in contacting hrm professionals and promoting participation in the research project. the demographic profile of interviewees is shown in table 1. demographic information research participants age <30 1 31-45 9 46-60 5 >61 1 n/a 1 gender female 8 male 9 education high school 1 diploma 1 bachelor degree 6 postgraduate degree 8 n/a 1 role in the organisation hr manager 15 hr business partner 2 time in the role <1 year 2 1-5 years 10 >5 years 4 n/a 1 table 1. participants’ demographic information all but one interviewee were employed by a large organisation (actively trading business with 200 or more employees (abs, 2017)) as indicated in table 2. organisations with less than 100 employees that were contacted for the purpose of this research project used none or a few basic e-hrm applications, and as a result they were not eligible to participate in the research project. the functional use of the e-hrm systems in the interviewees’ organisation is summarised in table 3. australasian journal of information systems ceric & parton 2024, vol 28, research article what prevents e-hrm potential? 9 table 2. information on organisations where interviewees worked functional e-hrm activities number of organisations payroll 17 training and development 17 recruitment 17 performance management 13 compensation and reward management 11 career/succession planning 4 time and attendance 4 health and safety 2 risk management and compliance 1 table 3. functional use of e-hrm systems in organisations 3.3 data analysis qualitative data analysis followed the steps of thematic analysis outlined by braun and clarke (2006) and was conducted using nvivo (version 11) software which ensured that data interpretation and conclusions were based on interview transcripts and data extracts (bazeley, 2007). this enhanced construct validity and reliability of this research (yin, 2003). the data were analysed in two stages. coding was done in the first stage, resulting in the identification of e-hrm adoption factors based on bondarouk et al.’s (2017) top framework and identification of the e-hrm use stages and its outcomes based on zuboff’s (1988) it use model. after the coding was done, text coded under each theme (or node in nvivo) was carefully checked to make sure that it was part of the same theme and sub-theme. although the interviewees belonged to different organisations, they had similar e-hrm adoption information on organisations number of interviewees number of employees 100 199 1 200 999 4 1,000 4,999 7 > 5,000 5 number of hrm employees <10 4 11 30 7 >31 5 n/a 1 industry mining, manufacturing and construction 3 real estate services, finance and insurance 3 information, media, telecommunications, tourism and transport 5 food services and pharmaceuticals 2 arts and recreation services 1 australasian journal of information systems ceric & parton 2024, vol 28, research article what prevents e-hrm potential? 10 experiences which supported the integration of interview findings using the same themes. this also provided a triangulation of data and prevented biased opinions (miles et al., 1994). the second part of the analysis focused on providing a deeper understanding of e-hrm use in organisations and identifying which factors influenced which of the three stages of e-hrm use and resulting outcomes. following this objective, we organised data into three themes, automation, informating and transformation. next, we identified e-hrm adoption factors relevant at each stage of e-hrm use. finally, we identified outcomes of each stage of e-hrm use that ensured that our findings provide further insights into each stage. while these outcomes correspond to those reported in the literature, we also identified outcomes which are more closely related to e-hrm information and realisation of e-hrm outcomes. we named these e-hrm challenges. e-hrm adoption contexts three stages of e-hrm use automation informating transformation technology context • lack of e-hrm integration (-) • low e-hrm functionality (-) • legacy e-hrm systems (-) • e-hrm difficult to use (-) organisational context • budget/cost of adoption (-) • lack of strategic e-hrm planning (-) • external recruitment of hrm staff (+) • declining industry (-) • organisational size (+) • change management (+) • involvement of stakeholders (+) • training in e-hrm (+) • groups of e-hrm users (+) people context • hr and it collaboration (+) • top management support (+) • hrm professionals’ it skills (+) • hrm professionals’ awareness of e-hrm potential (+) • users’ low use of ehrm (-) • users’ resistance (-) • lack of hrm professionals’ skills in data analytics (-) • organisational culture (-) • human connection (+) e-hrm outcomes • hrm professionals’ headcount • relief from administrative burden • hrm service • delegation of hrm tasks • hrm professionals’ time • hrm role in the organisation • strategic contribution to the organisation key challenges for achieving ehrm outcomes • limited access to ehrm data • lack of e-hrm data accuracy • limited use of ehrm data for strategic purposes table 4. e-hrm adoption factors enabling and inhibiting the three stages of e-hrm use australasian journal of information systems ceric & parton 2024, vol 28, research article what prevents e-hrm potential? 11 note. signs (-) and (+) indicate the direction of the factor’s influence on e-hrm success: (+) symbolises factors that support e-hrm success, and sign (-) symbolises factors that inhibit e-hrm success. bold text indicates that the factor concerned was discussed by 10 or more interviewees. 4 results guided by our research question, three critical e-hrm challenges and the factors that influence each of them positively or negatively were identified. an overview of the results is presented in table 4 (see above). this is followed by a detailed and focused examination of individual results that highlight, in line with our research question, those factors that inhibit the use of ehrm. 4.1 automation: what prevents organisations from achieving e-hrm potential? the automation stage of e-hrm use is influenced by the top factors which are presented in table 4. our findings indicate that the technology context often has a negative impact on automation. organisations have legacy e-hrm systems that are “archaic and complicated” and are “more a burden than a technological aid that facilitates hr” (interviewee 14). past e-hrm adoption decisions were made ad hoc based on the business needs at the time, or were simply inherited through mergers, without any strategic long-term planning. that is, a lack of strategic e-hrm planning in the past may have been the key factor that led to the partial automation that organisations are experiencing in the present. as a result, there is a lack of e-hrm integration: “we have 14 different systems” that are used for multiple pieces of information (interviewee 11), systems don’t talk to one another (interviewee 13), they are not compatible with one another (interviewee 15). furthermore, there was not always enough funding for ehrm (budget/cost of e-hrm adoption), leading to limited e-hrm functionality: “so when we put the system in, we chose not to add a lot of the extra capability that it had, because we didn’t have the money to spend on it” (interviewee 4). in addition, maintaining e-hrm systems is costly, and this tends to be overlooked when making a purchase decision (interviewee 9). interviewees reported that change is taking place and is driven by organisational and people contexts. factors from these contexts are leading organisations towards e-hrm integration, a main e-hrm goal for organisations participating in this study. this is seen as key for successful automation. top management backing is paramount in supporting the change as it ensures access to resources needed for e-hrm adoption and integration. however, interviewees are aware that their managers’ “buy-in” depends on the e-hrm cost and available budget (interviewee 13), as well as their interest in having “more information at their fingertips” (interviewee 7). another driver of change has been external recruitment of hrm staff as they bring in their e-hrm expertise and drive e-hrm adoption. the problem is that hrm professionals with e-hrm knowledge and skills may not want to work for an organisation that does not have fully operational e-hrm (interviewee 5). collaboration between hrm and it departments is further needed to facilitate strategic e-hrm planning which is based on long-term e-hrm decisions and e-hrm’s informating potential. in this regard, interviewees emphasised the need for hrm professionals to drive e-hrm adoption, as it professionals tend to focus on the technical back-end of e-hrm systems and can often overlook the hrm processes (interviewee 14). having it skills empowers hrm professionals in leading the implementation process, understanding it components of e-hrm australasian journal of information systems ceric & parton 2024, vol 28, research article what prevents e-hrm potential? 12 and communicating with the it department. hrm professionals also need to have awareness of e-hrm potential, its functionalities, as well as how e-hrm systems influence one another (interviewees 1, 11). finally, interviewees were aware that strategic e-hrm planning is critical as e-hrm decisions made today will affect the hrm function in the future. it is lack of strategic planning in the past that led to the automation issues discussed here. that is, effectively developing automation and informating today will produce transformational benefits in the future. interviewees also mentioned organisational size as influencing e-hrm adoption. some interviewees emphasised being part of a large organisation: they had access to “a large amount of resources” (interviewee 5) and those in smaller organisations emphasised less need for ehrm as they “can actually see trends develop outside of electronic systems” (interviewee 8), and the cost of e-hrm had a major influence on their e-hrm adoption decisions. our findings also reveal that the declining stage of the industry life cycle (declining industry) is an important factor influencing e-hrm adoption. organisations operating in a declining industry had scarce resources, regardless of the organisational size. regarding automation outcomes two key outcomes of e-hrm automation reported in the literature are hrm professionals’ headcount and relief from administrative burden (lepak & snell, 1998; gardner et al., 2003). less than a quarter of interviewees reported reduced headcount or redeployed hrm professionals. furthermore, the skillset of hrm professionals was affected by automation of hrm activities. “employees with only data entry and excel skills” were made redundant as there was no longer a need to manually enter data or manually manipulate excel spreadsheets, resulting in a need for “proper hr business partners” (interviewee 14). in general, e-hrm did not deliver hrm professionals from their administrative burden. more than half of interviewees had to engage in manual work to transfer data from one system to another: “reconciling between systems and data and interfaces that fall over” (interviewee 11) and collating data from different e-hrm systems into spreadsheets. zuboff’s (1988) it use model stipulates that automation gives rise to informating. only when technology is used to run activities and processes can it independently create, collect, and store new data regarding these activities and processes. however, organisations in this study did not use e-hrm for all their hrm activities and processes (see table 3). as discussed above, technology context further limited the automation stage. this resulted in limited access to ehrm data, which limited interviewees’ information responsiveness (gardener et al., 2003), ability “to make really good decisions” (interviewee 12) and “forward planning and modelling” (interviewee 14). limited access to e-hrm data, means that hrm professionals have access to “bits and pieces of information” (interviewee 12) from each system, instead of information generated from all systems which would provide a comprehensive understanding of hrm issues and trends in the organisation. this restricts the hrm function to an administrative role, unable to respond and take action as a result, and instead, spending time on pulling “all of that information together into a coherent and relevant report” (interviewee 12). an example provided by interviewee 14 paints the picture: “the ceo, if he just wanted to run a report and find out something about his workforce, he would then have to call the hr advisor who would then have to probably pull data out of various hr australasian journal of information systems ceric & parton 2024, vol 28, research article what prevents e-hrm potential? 13 systems and then manually crunch the numbers and check the validity of some of the data and then put it in a format that he wanted, and give it to him”. 4.2 informating: what prevents organisations from achieving e-hrm potential? table 4 also presents the factors from the top framework that influence the informating stage of e-hrm use. e-hrm not being easy to use is a factor from the technology context that is particularly relevant for the informating stage. interviewees in this study noted that e-hrm was being clunky (interviewees 17), cumbersome (interviewee 13), complex and convoluted (interviewee 14), and slow (interviewee 8). using e-hrm does not give “them [users] any kind of advantage” (interviewee 13). lack of integration further made e-hrm difficult to use, as employees “must get pretty sick of having to go through different systems with different looks and feels” (interviewee 10) and “trying to remember how to use that system versus another” (interviewee 11). many studies found that e-hrm systems were difficult to use and hence, users did not accept e-hrm and resisted its use (parry & tyson, 2011; martin & reddington, 2010; reddington & hyde, 2008), thus limiting its informating potential. employees’ resistance to e-hrm arises from “people got used to the old system” (interviewee 3) or “they see it as an additional burden” (interviewee 1). another aspect of resistance is that “people are always sceptical of hr’s motivation for rolling out new initiatives…what is hr wanting to know about us now? is this going to be just an easier way to discipline me…?” (interviewee 4). they can also be wary if they do not see benefits from using e-hrm, if it “replaces an already existing process that is working okay” or if rollout “isn’t executed well” (interviewee 15). another important explanation for users resisting e-hrm is the type of workforce and the nature of the organisation. for example, a vast majority of employees in some organisations have manual jobs that require them to spend their work time in production plant rather than at a desk, so they rarely use computers and e-hrm. interviewee 6 summarised this point adequately: “it ain’t their focus, it’s not their priority, so it’s really difficult to try and push that all the time”. interviewees reported that users differ a lot in how often they use e-hrm, and that they use e-hrm mainly for operational tasks (e.g., personal information, leave, training and overtime applications, performance reviews), but they do this well. another problem that interviewees noted here is that managers are not aware of e-hrm functionality “that could help them make better decisions” (interviewee 12). some interviewees saw this as hrm’s fault with e-hrm adoption, “we definitely neglect that end-user” (interviewee 17). interviewee 9 emphasised that e-hrm needs to “become enabling tools, to help achieve what you want to achieve functionally” in order for users to use them at the informating stage. interviewees explained that well-designed change management was the main factor in improving users’ acceptance of e-hrm. more specifically, they emphasised the hrm function’s role as an enabler (interviewee 5) and providing support for users in terms of training and having an hrm employee whom users can call for support (interviewee 13). there are different groups of users, some need substantial support in using and learning to use e-hrm, and there are groups that are ‘tech-savvy’ and are accepting of e-hrm (interviewee 15). another aspect of change management is engagement with the stakeholders, particularly, the leadership group, explaining how e-hrm will benefit them and managing stakeholders’ expectations. australasian journal of information systems ceric & parton 2024, vol 28, research article what prevents e-hrm potential? 14 regarding informating outcomes, interviewees stated that hrm service, a key aspect of relational e-hrm outcomes in their organisations, was limited, as managers and employees did not have access to relevant employee data. this can make them question the value of hrm service and affects hrm reputation (interviewee 9). interviewee 11 explained that “the view on data potentially from a business perspective is that it doesn’t add a lot of value”. a related e-hrm outcome is that managers rely on the hrm function to provide the missing employee data, that is, “do all of these transactional tasks, producing reports” (interviewee 14). however, this takes time as noted earlier in the discussion on e-hrm automation potential and restricts the ability of hrm professionals to take on a business partner role during transformation (see following section). zuboff’s (1988) perspective suggests that realisation of informating potential leads to hrm professionals having access to employee data that can be used to contribute to a deeper understanding of the organisation and have information autonomy and information responsiveness (gardner et al., 2003). however, interviewees in this study reported lack of accurate e-hrm data explained as not having “accurate information or any information entered into it [the e-hrm system]” (interviewee 4). this is a significant hurdle for hrm professionals as employee data “is really fundamental […] you just cannot do it [any analysis] without good data” (interviewee 17). finally, “if the system […] has incorrect data, it can cause all sorts of trouble” (interviewee 9) in terms of relationship with employees, hrm efficiency and its reputation. in terms of zuboff’s perspective, informating is failing. 4.3 transformation: what prevents organisations from achieving e-hrm potential? in the transformation stage, e-hrm is supposed to enable hrm to take on a strategic role in the organisation. however, there are top factors discussed earlier as part of automation and informating that also diminish the transformation stage. a few additional factors, such as organisational culture, inhibit transformation, as listed in table 4. an aspect of organisational culture that inhibits e-hrm transformation is a belief that hrm is just an administrative function. interviewees explained that managers often expect hrm professionals to do the transactional hrm activities for them, even when the e-hrm system already contains this information (interviewee 10). our findings indicate that organisational culture holds a limiting perception of hrm function’s role and value and creates users’ dependence on hrm. a profound explanation was provided by interviewee 12: “implementing a system is easy. changing people’s ways of working and dependence on hr and their belief in what hr should do and seeing them as strategic partners and all of these things, they’re cultural shifts, and cultural shifts don’t happen in a year”. more specifically, organisational culture restricts the delegation of transactional hrm activities to line managers and employees. the essence of this problem is in different beliefs between managers and hrm professionals: “the manager would feel that they were doing the hr work, whereas hr would say that they’re just doing the manager’s work that hr have had to do because we didn’t have an effective system in place” (interviewee 3). nevertheless, use of e-hrm does support and enables the shift where managers do take over hrm activities related to their role. interviewee 5 explained that “e-hrm makes them more responsible”. another factor that inhibits transformation of the hrm function is hrm professionals’ data analytics skills. this was identified as being “a fundamental skill” (interviewee 17): “it’s one of australasian journal of information systems ceric & parton 2024, vol 28, research article what prevents e-hrm potential? 15 the things that we constantly see on the development plans of hr people” (interviewee 16) as this skill is needed to support and initiate strategic decisions. some interviewees had at least one person on their team with data analytics skills, while some did not. interestingly, hrm professionals did not get training in data analytics during e-hrm adoption, so they relied on recruiting people with this skill or learning it on their own through using e-hrm. as a result of limited data analytics skills, e-hrm systems are less often used for data analysis, restricting the hrm function’s ability to provide the strategic value to the organisation. human connection is another aspect of the hrm transformation. interviewees agreed that ehrm can do transactional hrm and make the hrm function more efficient, but “people don’t want to work for a robot” (interviewee 17). hrm professionals changed their focus from administrative tasks to maintaining the human connection, connecting and having conversations with stakeholders. interviewee 9 explained, “as soon as you take the human connection bit out of it, you lose touch, and you lose relevance”. hrm value seems to be partly in e-hrm efficiencies and data, and partly in hrm’s ability to connect, coach, and continue having conversations with people. regarding transformation outcomes the most important transformational e-hrm outcome is enabling hrm to become a strategic business partner, by freeing hrm professionals to engage in strategic activities (gardner et al., 2003). as discussed earlier, interviewees used much of their time on manual data entry and collating data into excel spreadsheets, as well as supporting employees and line managers in their hrm activities. as a result, they did not have time for strategic aspects of their job as: “i’m spending all my time based on admin tasks […] because our systems aren’t up to date” (interviewee 3), and interviewee 10 aptly explained: “[we are] doing things very inefficiently but because they are so inefficient we haven’t got much time to go and re-invent a machine gun while we’re trying to fight the war with a bow and arrow”. the hrm function was still an administrative function, and “not adding as much value to the business” (interviewee 3). e-hrm’s potential to enable hrm to become more strategic was inhibited as pointed out by interviewee 14: “it affects the perception of the business, of your value and the service that you provide because you are stuck into that transactional and basic functions. it … affects the actual skills and the type of people that work in the hr team, hrm director’s influence (s)he can have over the executive team because there’s only so much that you can do.” in contrast, almost half of the interviewees reported having more time to engage in strategic activities, and 41 per cent of interviewees reported hrm having a strategic contribution to the organisation. e-hrm use ensured they had a seat at the table [with] the senior management team” (interviewee 13), their role changed from transactional hrm activities to having conversations with key stakeholders (interviewee 5), and using e-hrm data in strategic decisions (interviewee 1). zuboff (1988) explained that transformation is a stage of it use that results from automation and informating. when automation is partial, informating is limited, and consequently, transformation is deficient. the majority of interviewees noted limited actual use of e-hrm data for strategic purposes as a challenge. instead, it was used for only basic reporting, “day to day operational” (interviewee 14), “collating data and being quite reactive” (interviewee 3). australasian journal of information systems ceric & parton 2024, vol 28, research article what prevents e-hrm potential? 16 5 discussion the hrm function increasingly relies on e-hrm when taking a strategic partner’s role in the organisation. yet, research offers mixed findings on whether e-hrm’s potential is realised (marler & fisher, 2011; zhou et al., 2022). in our study we in particular set out to answer the question “what prevents organisations from achieving e-hrm potential?”. as a first contribution we combine zuboff’s (1988) three-stage model of it use with bondarouk et al.’s (2017) top framework of e-hrm adoption factors as an avenue of exploring a widely held expectation that e-hrm, once adopted, will lead to a range of organisational benefits and offer an answer to our question. employing zuboff’s model of it use as part of our framework provided a systemic understanding of e-hrm adoption as a progressive and developing phenomenon. the stages of e-hrm use are inherent in technology and are developmental (zuboff, 1988; gardner et al., 2003; strohmeier & kabst, 2009). with this approach to understanding e-hrm use, we reconsidered e-hrm adoption factors from technology, organisation and people contexts from the top framework influencing realisation of e-hrm outcomes. that is, identified factors in this study either support or constrain realisation of ehrm potential, specific to each stage of e-hrm use. this provided insights into what prevents organisations from achieving e-hrm potential. the findings of our study show that e-hrm outcomes do not happen as a direct consequence of e-hrm adoption and implementation. organisations experience challenges with automating hrm activities and processes. they also experience challenges with using e-hrm information that becomes available through the informating stage. we demonstrate that when one of the e-hrm stages is only partially developed, other stages cannot be completed, resulting in a limited realisation of e-hrm potential. our findings give support to zuboff’s (1988) and gardner et al.’s (2003) proposition that e-hrm success is a result of all three stages of e-hrm use. this has practical implications for understanding why organisations are struggling to achieve strategic e-hrm outcomes (marler, 2009, marler & fisher, 2013; parry & tyson, 2011; bondarouk & ruël, 2013; martin & reddington, 2010). organisations that are in the automation or informating stages of e-hrm use have not yet reached their strategic ehrm potential, and hence, cannot report strategic e-hrm outcomes, but emphasise operational e-hrm outcomes. however, this does not mean that strategic outcomes will not be forthcoming. that is, realisation of the e-hrm potential to automate, informate and transform is a process that requires time and support. this finding offers an insight into the inconsistent findings in the literature related to strategic e-hrm outcomes. our findings also have implications for the top framework. while this framework is valuable for practical reasons as it groups 168 e-hrm adoption factors into three categories (bondarouk et al., 2017), it is not clear when the identified factors become influential in the e-hrm adoption process or when they act as enablers or barriers to e-hrm adoption. exploring these questions was not the objective of bondarouk et al.’s (2017) work, but doing so is an important avenue for expanding the top framework and making it more practical for researchers and practitioners to use. in our research we distinguish between top factors that are influential at the three stages of e-hrm use, and that act as enablers and inhibitors of e-hrm potential. when used along with zuboff’s three stages of it use model, the top framework became more useful in explaining e-hrm adoption more specifically and in more detail. that is, the top framework allows a better understanding and exploration of each stage of zuboff’s model. in addition, we found that a factor we named declining industry can outweigh organisational size australasian journal of information systems ceric & parton 2024, vol 28, research article what prevents e-hrm potential? 17 as an indicator of organisational resources. for example, organisations in declining industries had limited resources for e-hrm adoption, regardless of their size, and could not afford investing in e-hrm and resolving e-hrm integration issues. hence, we suggest to include external environment as a fourth category into the top framework, and to further explore whether there are additional factors from the external environment that affect e-hrm adoption success. combining zuboff’s theoretical model of it use and the tops framework of e-hrm adoption factors provides practical value to practitioners in terms of a more comprehensive understanding of e-hrm adoption and challenges with realising e-hrm potential. practitioners could use zuboff’s model to investigate, understand and prepare the organisational context to support e-hrm adoption as a holistic and developing process. furthermore, practitioners can use the findings in this study to identify factors that they need to support or tackle in order to realise specific e-hrm objectives. the potential for e-hrm to produce positive outcomes requires business leaders to understand the three stages of e-hrm use and factors relevant for each stage. it is an imperative to identify potential constraints to e-hrm adoption success. practitioners can also consider inhibitors and enablers identified in this study during their planning and managing of e-hrm adoption to enhance its probability of success. next, identification of the three key e-hrm challenges – accessibility, accuracy, and limited actual use of e-hrm data – is another important contribution of this study. when present, they indicate a fully functioning e-hrm system that can provide benefits to the organisation. but when they are lacking, they indicate a need for investigating factors which inhibit realisation of e-hrm potential in a particular stage of e-hrm use and creating strategies to rectify these. this offers another use of our findings for practitioners. our findings support gardner et al.’s (2003) notion that e-hrm outcomes are different, based on the stage of e-hrm use. moreover, they are a result of e-hrm’s success in each stage of ehrm use. similarly to martin and reddington (2010), we also found that e-hrm required hrm professionals to provide it support to employees and managers in the organisations. in addition to this, we found hrm professionals had to manually enter data into e-hrm systems and consolidate e-hrm information from different systems. some interviewees also reported having to provide e-hrm information and reports to users as they themselves could not access it from the e-hrm system. this created an additional hurdle for realisation of strategic e-hrm outcomes as interviewees did not have time for data analysis and strategic contributions. however, use of e-hrm created a change in the skills hrm professionals needed, from data entry skills to data analytics. as a result, hrm professionals with only data entry skills were replaced by a functional e-hrm system resulting in some reduction in the headcount of hrm employees. 6 limitations and future research despite its contributions, this study has several limitations which call for further research. we acknowledge that there are several aspects of organisational context that could not be considered in our study and that may influence e-hrm adoption. for example, there is probably no single best approach to e-hrm adoption because each organisation is unique in terms of its organisational context and e-hrm systems, and this could lead to a very different e-hrm adoption process in each organisation. for example, organisational general experience australasian journal of information systems ceric & parton 2024, vol 28, research article what prevents e-hrm potential? 18 with the it adoption may affect e-hrm adoption. there may be a difference between people and organisational units in terms of readiness to adopt and use e-hrm. e-hrm adoption may be significantly different from other it systems in the organisation and these differences could be further explored. there may be a difference between organisations based on whether the ehrm adoption starts with involving the whole organisation or only the hr function and later introducing it to other parts of the organisation. further research is needed into the role of the e-hrm system itself as a facilitator or barrier to adoption, and characteristics that make e-hrm systems easier or more challenging to accept by people. organisations in our research adopted different e-hrm systems. we found that although different, existing e-hrm systems in organisations are problematic to use due to their interface and functionality. consequently, interviewees explained that an important requirement for a new e-hrm system is user friendliness. the lesson from these examples is that organisations need to approach e-hrm adoption in accordance with all of their contextual factors. further investigation of the organisational context is warranted, as e-hrm adoption could be conditioned by contextual and organisational aspects. our focus in this study was on e-hrm adoption and reaching later stages in zuboff’s (1988) model which could potentially significantly improve the impact of e-hrm on hrm and organisational performance. thus, our findings do not distinguish between different e-hrm applications identified in table 3, nor do our findings link specific e-hrm applications with barriers and enablers identified in table 4. this is another opportunity for future research. next, our data, due to its qualitative nature, does not rank the identified e-hrm factors in order of importance. instead, we used the frequency of themes mentioned by interviewees to distinguish key factors in table 4. future studies could address this limitation by asking hrm professionals to rank adoption factors in a survey. this would be valuable for practitioners and researchers and could further contribute to development and practical usability of the top framework. this study focused on hrm professionals employed in australian organisations. thus, the findings reflect an australian context, which may not be applicable to other countries. this reduces the generalisability of the findings. more research is needed to identify the influences of e-hrm adoption factors across the three stages of e-hrm use in different countries, to test whether the findings in this study are applicable to other contexts, and to identify whether there are other e-hrm challenges that are more relevant in these other contexts. another area for further research is the confirmation of three key challenges, namely, access to e-hrm data, e-hrm data accuracy and use of e-hrm data for strategic purposes, as indicators of e-hrm success. the findings reported in this article call for further research on the role of three-stages of ehrm use in explaining e-hrm adoption and realisation of e-hrm potential. it would be useful to focus on and compare organisations at different stages of the e-hrm development process from automation to transformation. those organisations with fully developed automation and informating e-hrm may have important lessons to teach us about the process. also, as the interviewees who participated in this research did not complete the transformation stage, they were not able to report on the factors needed to support this entire process. this can be explored in future research. in addition, there are other participants in the organisation, such as line managers and employees, whose views on e-hrm challenges need to be further australasian journal of information systems ceric & parton 2024, vol 28, research article what prevents e-hrm potential? 19 explored. this could provide additional understanding of e-hrm success, challenges, and outcomes. 7 concluding remarks in this study we combine zuboff’s (1988) three-stage model of it use with bondarouk et al.’s (2017) top framework of e-hrm adoption factors and identify critical e-hrm challenges and practical means of overcoming them. our findings can be of use to hrm practitioners who are in the process of e-hrm implementation or who are experiencing issues with achieving ehrm outcomes. this study also contributes to the literature by exploring e-hrm adoption factors, in particular inhibiting factors, influencing realisation of e-hrm outcomes in each of the three stages of e-hrm use. hence, while this study extends a theoretical discussion in the e-hrm literature and shows avenues for future research, its findings can empower hr managers to proactively respond to e-hrm challenges and achieve e-hrm potential. references australian bureau of statistics (abs) (2017). selected characteristics of australian business, 2015-16 (cat. no. 8167.0), available at http://www.abs.gov.au/ausstats/abs@.nsf/mf/8167.0 (accessed 20 february 2019). ball, k. s. (2001). the use of human resource information systems: a survey. personnel review, 30(6), 677–693. bazeley, p. (2007). qualitative data analysis with nvivo. thousand oaks, ca: sage publications. bondarouk, t., & ruël, h. (2013). the strategic value of e-hrm: results from an exploratory study in a governmental organization. the international journal of human resource management, 24(2), 391-414. bondarouk, t., parry, e., & furtmueller, e. (2017). electronic hrm: four decades of research on adoption and consequences. the international journal of human resource management, 28(1), 98-131. braun, v., & clarke, v. (2006). using thematic analysis in psychology. qualitative research in psychology, 3(2), 77-101. burton-jones, a. (2014). what have we learned from the smart machine?. information and organization, 24(2), 71-105. debri, f., & bannister, f. (2015.) e-government stage models: a contextual critique. in proceeding of the 48th hawaii international conference on system sciences, pp. 2222-2231. kauai, hi, usa, 2015. doi: 10.1109/hicss.2015.266 dery, k., & wailes, n. (2005). necessary but not sufficient: erps and strategic hrm. strategic change, 14(5), 265–272. gardner, s.d., lepak, d.p., & bartol, k.m. (2003). virtual hr: the impact of information technology on the human resource professional. journal of vocational behavior, 63(2), 159179. given, l.m. (2008). the sage encyclopaedia of qualitative research methods (vol. 1). thousand oaks, ca: sage publications. australasian journal of information systems ceric & parton 2024, vol 28, research article what prevents e-hrm potential? 20 guest, g., bunce, a., & johnson, l. (2006). how many interviews are enough? an experiment with data saturation and variability. field methods, 18(1), 59-82. haines, v.y., & lafleur, g. (2008). information technology usage and human resource roles and effectiveness. human resource management, 47(3), 525-540. hannon, j., jelf, g., & brandes, d. (1996). human resource information systems: operational issues and strategic considerations in a global environment. international journal of human resource management, 7(1), 245-269. hussain, z., wallace, j., & cornelius, n. e. (2007). the use and impact of human resource information systems on human resource management professionals. information & management, 44(1), 74–89. iqbal, n., ahmad, m., & allen, m. m. (2019). unveiling the relationship between e-hrm, impersonal trust and employee productivity. management research review, 42(7), 879899. kuzel, a. (1992). sampling in qualitative inquiry, in crabtree, b., & miller, w. (eds), doing qualitative research, pp. 31-44. thousand oaks, ca, usa: sage publications. larkin, l. (2017). hr digital disruption: the biggest wave of transformation in decades. strategic hr review, 16(2), 55-59. lau, g., & hooper, v. (2009). adoption of e-hrm in large new zealand organizations, in torres-coronas, t., & arias-oliva, m. (eds.), encyclopedia of human resources information systems: challenges in e-hrm, pp. 31-41. new york, ny, usa: igi global, information science reference. lengnick-hall, m. l., & moritz, s. (2003). the impact of e-hr on the human resource management function. journal of labour research, 24(3), 365–380. lepak, d.p., & snell, s.a. (1998). virtual hr: strategic human resource management in the 21st century. human resource management review, 8(3), 215-234. marler, j. (2009). making human resources strategic by going to the net: reality or myth?. the international journal of human resource management, 20(3), 515-527. marler, j. a., & fisher, s. l. (2013). an evidence-based review of e-hrm and strategic human resource management. human resource management review, 23(1), 18-36. marler, j. h., & parry, e. (2016). human resource management, strategic involvement and ehrm technology. the international journal of human resource management, 27(19), 22332253. martin, g., & reddington, m. (2010). theorizing the links between e-hr and strategic hrm: a model, case illustration and reflections. the international journal of human resource management, 21(10), 1553-1574. menant, l., gilibert, d., & sauvezon, c. (2021). the application of acceptance models to human resource information systems: a literature review. frontiers in psychology, 12 (2021). miles, m.b., huberman, a.m., huberman, m.a., & huberman, m. (1994). qualitative data analysis: an expanded sourcebook. thousand oaks, ca, usa: sage publications. australasian journal of information systems ceric & parton 2024, vol 28, research article what prevents e-hrm potential? 21 morris, s. s., wright, p. m., trevor, j., stiles, p., stahl, g. k., snell, s., & farndale, e. (2009). global challenges to replicating people, processes, and systems. human resource management, 48(6), 973–995. myers, m. d. (2019). qualitative research in business and management (3rd edition). thousand oaks, ca, usa: sage publications. nyathi, m., & kekwaletswe, r. (2023). electronic human resource management (e-hrm) configuration for organizational success: inclusion of employee outcomes as contextual variables. journal of organizational effectiveness: people and performance. https://doi.org/10.1108/joepp-08-2022-0237 obeidat, s.m. (2016). the link between e-hrm use and hrm effectiveness: an empirical study. personnel review, 45(6), 1281-1301. panayotopoulou, l., vakola, m., & galanaki, e. (2007). e-hr adoption and the role of hrm: evidence from greece. personnel review, 36(2), 277-294. panos, s., & bellou, v. (2016). maximizing e-hrm outcomes: a moderated mediation path. management decision, 54(5), 1088-1109. parry, e. (2011). an examination of e-hrm as a means to increase the value of the hr function. the international journal of human resource management, 22(5), 1146-1162. parry, e., & battista, v. (2019). the impact of emerging technologies on work: a review of the evidence and implications for the human resource function. emerald open resources, 1(5), 1-14. https://doi.org/10.12688/emeraldopenres.12907.1 parry, e., & tyson, s. (2011). desired goals and actual outcomes of e-hrm. human resource management journal, 21(3), 335-354. poba-nzaou, p., uwizeyemunugu, s., gaha, k., & laberge, m. (2020). taxonomy of business value underlying motivations for e-hrm adoption: an empirical investigation based on hr processes. business process management journal, 26(6), 1661-1685. reddick, c. g. (2009). human resources information systems in texas city governments: scope and perception of its effectiveness. public personnel management, 38(4), 19-34. reddington, m., & hyde, c. (2008). the impact of e-hr on line managers and employees in the uk: benefits, problems, and prospects. in marting, g., reddington, m., & alexander, h. (eds.), technology, outsourcing and transforming hr, pp. 135-160. oxford, uk: butterworth-heinemann books. rowley, j. (2012). conducting research interviews. management research review, 35(3/4), 260271. ruël, h., bondarouk, t., & looise, j.k. (2004). e-hrm: innovation or irritation. an explorative empirical study in five large companies on web-based hrm. management revue, 15(3), 364-380. ruël, h., & kaap, h. (2012). e-hrm and value creation: does organizational context matter?. german journal of research in human resource management, 26(3), 270-291. ruël, h. j. m., bondarouk, t. v., & van der velde, m. (2007). the contribution of e-hrm to hrm effectiveness. employee relations, 29(3), 280–291. australasian journal of information systems ceric & parton 2024, vol 28, research article what prevents e-hrm potential? 22 schalk, r., timmerman, v., & van den heuvel, s. (2013). how strategic considerations influence decision making on e-hrm applications. human resource management review, 23(1), 84-92. sheehan, c. (2009). outsourcing hrm activities in australian organisations. asia pacific journal of human resources, 47(2), 236-253. sheehan, c., & de cieri, h. (2012). charting the strategic trajectory of the australian hr professional. asia pacific journal of human resources, 50(2), 151-168. shrivastava, s., & shaw, j. b. (2003). liberating hr through technology. human resource management, 42(3), 201-222. https://doi.org/10.1002/hrm.10081 strohmeier, s. (2007). research in e-hrm: review and implications. human resource management review, 17(1), 19-37. strohmeier, s. (2009). concepts of e-hrm consequences: a categorisation, review and suggestion. the international journal of human resource management, 20(3), 528-543. strohmeier, s. (2020). digital human resource management: a conceptual clarification. german journal of human resource management, 34(3), 345-365. strohmeier, s., & kabst, r. (2009). organizational adoption of e-hrm in europe: an empirical exploration of major adoption factors. journal of managerial psychology, 24(6), 482-501. talukdar, a., & ganguly, a. (2022). a dark side of e-hrm: mediating role of hr service delivery and hr socialization on hr effectiveness. international journal of manpower, 43(1), 116-147. tansley, c., kirk, s., williams, h., & barton, h. (2014). tipping the scales: ambidexterity practices on e-hrm projects. employee relations, 36(4), 398-414. troshani, i., jerram, c., & hill, s.r. (2011). exploring the public sector adoption of hris. industrial management and data systems, 111(3), 470-488. ulrich, d. (1997). measuring human resources: an overview of practice and a prescription for results. human resources management, 36, 303-320. van den berg, m. j., stander, m. w., & van der vaart, l. (2020). an exploration of key human resource practitioner competencies in a digitally transformed organisation. sa journal of human resource management, 18. doi: https://doi.org/10.4102/sajhrm.v18i0.1404 voermans, m., & van veldhoven, m. (2007). attitude towards e-hrm: an empirical study at philips. personnel review, 36(6), 887-902. wiblen, s., dery, k., & grant d., (2012). do you see what i see? the role of technology in talent identification. asia pacific journal of human resources, 50(4), 421-438. yin, r. k. (2003). case study research design and methods (3rd ed.). applied social research methods series, 5. thousand oaks, ca, usa: sage publications. zhou, y., cheng, y., zou, y., & liu, g. (2022). e-hrm: a meta-analysis of the antecedents, consequences, and cross-national moderators. human resource management review, 32(4), 100862. zuboff, s. (1988). in the age of the smart machine: the future of work and power. new york city, ny, usa: basic books, inc. australasian journal of information systems ceric & parton 2024, vol 28, research article what prevents e-hrm potential? 23 appendix the interview guide introduction to start off, could you please tell me about your role in your organisation? how long have you been in this role? how many employees does your organisation have? how many hr employees are there in the hr department? e-hrm background what type of e-hrm systems are used in your organisation? are they compatible with one another? how does use of multiple e-hrm systems affect their overall use and benefits? next i’m going to read you a list of e-hrm activities. could you please indicate which of these practices are completed in your business via e-hrm? • recruitment, training and development, performance management, talent management, career/succession planning, benefits, payroll, time and attendance, work scheduling and health and safety when did the organisation start implementing e-hrm? what outcomes did your organisation seek to achieve from e-hrm implementation (e.g., automation of hr processes, quality or responsiveness of hr services, a more strategic hr function)? are these outcomes achieved and to what extent? were there any additional outcomes of e-hrm? are there plans for implementation of new e-hrm systems? if yes, what are the planned outcomes for these? could you please comment on the role of the hr department in the e-hrm implementation process? probe: what can enable the hr department to be a driver in e-hrm implementation? e-hrm value which three hrm activities do you think create most value for the organisation? is e-hrm used in these three hrm activities? what effect does e-hrm use have on these three ‘most valuable’ hrm activities? which e-hrm activities create most value for the organisation? could you please discuss which e-hrm capabilities are developed in the organisation? are there plans for developing any other e-hrm capabilities in the near future? what is expected to be gained from developing these? which skills of the hr team are important in developing e-hrm value in your organisation? how does organisational culture influence e-hrm use? australasian journal of information systems ceric & parton 2024, vol 28, research article what prevents e-hrm potential? 24 e-hrm use in the organisation could you please describe how e-hrm is used in your organisation, and to what extent? probe: why is e-hrm not used fully in the organization? *how is e-hrm information used by hr managers? what is the impact of hr employees’ ability to analyse and understand data? to what extent is e-hrm information used for: a) operational reporting of efficiency and compliance measures (focus on data accuracy, consistency, and timeliness)? b) advanced operational reporting, benchmarking and decision-making, dashboards and multidimensional analysis? c) proactively solving business problems? (e.g., statistical models, root-cause analysis) d) developing predictive models, scenario planning, risk analysis or integration with strategic planning? how does such use of e-hrm information affect strategic hrm decisions? could you please provide an example? what are the challenges with using the current e-hrm system for the hr department in your organisation? who else uses e-hrm and its information? how do line managers/employees use ehrm information? how does access to this information affect them? what are the challenges with using the current e-hrm system for the line managers in your organisation? in your opinion, what is the value of the current e-hrm for your organisation? what inhibits current e-hrm value in your organisation? (technology/organisation/people factors) when you reflect on the e-hrm use, which factors do you think are essential for creating e-hrm value in your organisation? (technology/organisation/people factors) impact of e-hrm having hr activities delivered via e-hrm, how does this impact on the hr department in your organisation? could you please provide an example? a) were there any changes in the hr function’s (overall performance) results based on using e-hrm? b) how does e-hrm use affect costs and efficiencies of the hr function in your organisation? does e-hrm affect any other operational outcomes (e.g., automation of hr tasks, time savings benefits, cost reductions, faster diagnosis of hr problems, less hr personnel, online report activities)? c) how did the use of e-hrm affect hr department’s ability to serve its internal clients? australasian journal of information systems ceric & parton 2024, vol 28, research article what prevents e-hrm potential? 25 d) how does e-hrm use affect hr’s strategic role in your organisation? does e-hrm use influence the organisation’s performance? how? what are your thoughts on how e-hrm will affect the hr department in your organisation in the future? demographics could you please tell me your age, and qualifications. copyright: © 2024 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v28.3877 http://creativecommons.org/licenses/by-nc/3.0/au/ https://doi.org/10.3127/ajis.v28.3877 1 introduction 2 theoretical framework 2.1 the three-stage model of it use 2.2 the top framework of e-hrm adoption factors 2.2.1 technological context 2.2.2 organisational context 2.2.3 people context 2.3 the combined framework to study e-hrm adoption and use 3 method 3.1 data collection 3.2 participants 3.3 data analysis 4 results 4.1 automation: what prevents organisations from achieving e-hrm potential? 4.2 informating: what prevents organisations from achieving e-hrm potential? 4.3 transformation: what prevents organisations from achieving e-hrm potential? 5 discussion 6 limitations and future research 7 concluding remarks paper title (paper title style) australasian journal of information systems kautz 2021, vol 25, editorial editorial for ajis 2021 1 rmit classification: trusted editorial for the australasian journal of information systems 2021: volume 25 karlheinz kautz rmit university, melbourne, australia karlheinz.kautz@rmit.edu.au when i was appointed editor-in-chief of the australasian journal of information systems in late 2020 i was asked by the selection panel to reinstate on a more regular basis an editorial for the journal. the journal does currently not publish numbered issues, but a yearly volume that consists of regular is research articles and specific thematic special sections which are published throughout a year. thus, as my inaugural year of serving in the position is about to conclude, and with the now available full volume 25, i follow the panel’s request and provide my first editorial reflections. first of all i like to thank the australasian information systems (is) community for entrusting the journal to me after welcoming and integrating me into its fellowship no longer than 8 years ago when i relocated to australia. in particular i am grateful to my predessor dr john lamp who set a high bar. he transferred the journal into a fully online publication, submitted it to numerous indices and ranking schemes, linked it to the the orcid author identification, implemented dois and added a qr code to each article. he also realized the journal’s social media presence on linkedin, twitter, facebook, as well as the distribution of relevant journal announcements through major is related mailing lists. last but not least he created a viable technical infrastructure within the open source review system ojs (open journal systems) which we use. i inherited well documented processes and john had the generosity to handover the journal in several additional face-to-face sessions at his home office to explain and introduce me to the system’s unavoidable idiosyncrasies which are best experienced hands-on beyond any well-written documentation. ajis is an open access journal with an inclusive approach to represent the diversity and cultural richness of our region. naturally my editorship is impacted by my understanding of the is discipline. professor allen lee has expressed this understanding more elegantly than i am able to do in one of his editorial comments when he served as editor-in-chief of management information systems quarterly (misq); i therefore quote him directly “research in the information systems field examines more than just the technological system, or just the social system, or even the two side by side; in addition, it investigates the phenomena that emerge when the two interact. this embodies both a research perspective and a subject matter that differentiate the academic field of information systems from other disciplines. in this regard, our field’s so-called ‘reference disciplines’ are actually poor models for our own field. they focus on the behavioral or the technological, but not on the emergent sociotechnical phenomena that set our field apart. for this reason, i no longer refer to them as reference disciplines, but as ‘contributing disciplines’ at best.” (lee, 2001, pg. iii) australasian journal of information systems kautz 2021, vol 25, editorial editorial for ajis 2021 2 rmit classification: trusted i follow this commitment to the sociotechnical perspective as a foundation of the is discipline by not only synergistically connecting instrumental results such as efficiency, effectiveness, productivity and, i might add, profitability, which usually dominate is research in business and commercial contexts, with humanistic outcomes such as well-being, equality, empowerment, and freedom as put forward in a recent plea by sarker, chatterjee, xiao, and elbanna (2019), but by balancing the two types of outcomes, if not privileging humanistic over instrumental outcomes. on this background, reinforced by the current trend of digitalisation and digital transformation (vial, 2019; wessel, baiyere, ologeanu-taddei, cha, and jensen, 2021) enabled and driving by the convergence of smac – social, mobile, analytics, and cloud computing technology developments (legner, eymann, hess, matt, böhmann, drews, mädche, urbach, and ahlemann, 2017) in among others the internet of things, artificial intelligence, machine learning, robotics, and blockchain to name a few, and in this context a rising need and interest in ethical, privacy, and security issues, majchrzak, markus, and wareham (2016) question the historical view that organisations are the centre of customary is research. they advocate is research that transcends organisational boundaries and emphasises concepts like emergence, networks, and ecosystems. sørensen (2016) likewise calls for abandoning the traditional focus on individual organisations as stable units of analysis in is research and promotes a focus on new sociotechnical phenomena that include service ecosystems and service platforms. sørensen and landau (2015) challenge conventional wisdom in the is discipline and require novel academic agility to explore these new horizons rather than stay within conventional boundaries and de reuver, sørensen, and basole (2018) offer an actual agenda for is research based among others on the concept of digital platforms and the ecosystem metaphor. this is the environment in which i see the ajis. furthermore, of the two foundational concepts of our discipline ‘information’ and ‘system’, beyond the sociotechnical perspective i have always been fascinated by the latter which with the ongoing shift from a goods dominant to a service dominant society (lusch and vargo, 2019; kazman and chen, 2009) re-emerges in the form of digital (service) ecosystems, but which also merits further attention of is research in the context of complexity (benbya, nan, tanriverdi, and yoo, 2020), of complex adaptive systems (kautz, bjerknes, fisher, and jensen, 2020), and of classical systems thinking in general (checkland, 1999). returning to what has once been qualified as reference disciplines (baskerville and myers, 2002), and what lee (2001) defines as contributing disciplines to clearly demark and provide an identity for the is discipline, the journal during the year received numerous submissions that fell out of the journal’s scope and position as i have described it above. a lot of these submissions with a clear focus on digital and information technology more adequately would have better fitted and were referred to computer science or electrical and electronics engineering outlets. there were also many submitted manuscripts from the marketing discipline which presented consumer studies and concentrated in particular on consumer behaviour with information technology as a mere background but little emphasis on the actual interaction between the social and the technical system as lee (2001) put it. also for these manuscripts we provided direction and suggested to submit them to pure marketing journals. australasian journal of information systems kautz 2021, vol 25, editorial editorial for ajis 2021 3 rmit classification: trusted a final group of submissions which caught our particular attention consisted of a large number of conventional adoption and/or intention to use studies based on the widespread and reputable work of davis (1989) and venkatesh, morris, davis, and davis (2003) who introduced the technology acceptance model (tam) and unified theory of acceptance and use of technology (utaut) into our discipline. while such research still offers valuable advice to policy makers and practitioners and mirrors an obvious interest and need for such investigations in parts of our region for political or commercial decision making, adoption or intention to use research of a certain technology in a particular sector or part of society in a selected country or region is a longestablished scholarly approach and is very well academically studied. the is literature presenting the orthodox approach of integrating a set of independent variables from one model with another set of variables from other models all derived from previous studies in a resulting conceptual model that mediates or moderates causative relationships as a part of standard model building and a prerequisite of model testing and validation is extensive. in a world where digital and information technologies are ubiquitous, hence it is difficult to make a new theoretical contribution with such an approach without fully and in depth appreciating and taking the socio-economic and cultural context of the adoption situation – i referred to the richness and diversity of our region above – into account. only few, newly submitted adoption studies have therefore been included in a review cycle. thus, abandoning or replacing the customary concept of (it) adoption with more advanced concepts such as the concept of (it) appropriation and/or applying different research approaches – as examples from our region, see plumb and kautz (2014) or riemer and johnston (2012, 2015) – might be more favourable and lead to new and interesting insights. throughout this year the journal has published 14 regular research articles and 17 articles in four special sections with accompanying editorials, which were all submitted before 2021; with more than 225 submissions during the year the journal – although not mirroring the actual calendar year – can be said to have had an acceptance rate of approximately 14% and has a healthy pipeline of manuscripts. the published articles represent a mixture of quantitative and qualitive, positivist and interpretive, and even critical research approaches in line with the methodological pluralist positioning of the journal. with traditional, quantitative, causal logic, factor and survey based (adoption) research embedded in a substantialist ontology still dominating the journal’s submissions and publications, i strongly encourage the submission of more daring manuscripts and contributions, in the words of alvesson and sandberg (2011, 2013, 2014), research that is assumption-challenging and deals with surprising phenomena instead of being confirmatory and founded on mere gap-spotting. such research can advantageously be grounded in alternative genres and be based on alternative research methods (avital, mathiassen, and schultze, 2017) such as process research (riemer and johnston, 2016) and an unconventional ontology, such as a relational ontology (riemer and johnston, 2012, 2015; cecez-kecmanovic, galliers, henfridsson, newell, and vidgen, 2014). action research (avison, davison, and malaurant, 2018) and advanced design science studies (gregor and hevner, 2013) are other approaches which deserve more representation in the journal. this is what the statement australasian journal of information systems kautz 2021, vol 25, editorial editorial for ajis 2021 4 rmit classification: trusted “ajis publishes high quality contributions to theory and practice in the global information systems (is) discipline based on innovative and novel approaches and topics.” that can be found on the journal website section ‘about the journal’ means. as for volume 25, the continuously published regular research articles in 8 manuscripts covered established topics such as an – applaudable, because of its clear commitment to the foundations of our discipline – explicit sociotechnical perspective on enterprise system implementation and a study of it governance, current subjects such as digital ecosystems, digital divide, and cybersecuity in our region, in iran, india, and vietnam, respectively, as well as the themes of customer use of self-service technologies in australia, social influence on social media, and the rising global utilization of humanized robots in marketing. the just published, latest collection of research articles comprises six articles that represent the above discussed classical is/it adoption research and had been under review for quite some time. mostly based on the technology acceptance model or related derivates of this model these works investigate the adoption and/or intention to use of healthcare information systems or apps in australia, bangladesh, and india, as well m-payment adoption in malaysia and vietnam, the former with specific consideraton of convenience, security and gamification. gamification and adoption issues are also prevaling in the special section on consumer evaluation of mobile apps with a specific focus on interactivity. gamification here is the lems for researching customer loyality and customers’ connection with a specific brand, which in another study was examined by taking the impact of e-word-to-mouth into consideration. research on the adoption, respectively the resistance to adoption of m-payment in india was part of this special section as well. we also published a special section on the bright and dark side of stakeholder engagement on social media covering topics such as policy makers use of social media in india, addiction to online platforms, counteracting excessive use of social networking sites through mindfulness, attrition of gig work on digital platforms, customer engagement in retailing, and the promotion of wildlife conservation on online wildlife trading sites. finally, beyond the customary special section on selected papers from the australasian conference on information systems (acis) , this volume covering the year 2020 with current contributions on digital transformation, cybersecurity, the application of filters in social media, and artificial intelligence in warehouse management, we published a most noteworthy and especially in our region important special section on indigenious use of information and communication technologies which argued the self-determination of indigenous people with regard to information systems and presented approaches to the design of archival information systems through partnerships with indigenous communities, and to knowledge elicitation with indigenous people. a last contribution in this section underlined the role of information systems and is research methodologies for indengenious people in their pursuit of care and transformation of their communities. beyond the just published collection of articles you thus might find it interesting to revisit the entire volume 25. while operating under the management of the australasian assocation for information systems (aais), a chaper of the association for information systems (ais), the australian https://journal.acs.org.au/index.php/ajis/article/view/2917 https://journal.acs.org.au/index.php/ajis/article/view/2917 australasian journal of information systems kautz 2021, vol 25, editorial editorial for ajis 2021 5 rmit classification: trusted council of professors and heads of information systems (acphis, and the professors and heads of information systems, new zealand (phisnz), it is important to keep in mind that ajis is a community based journal that is run on a voluntary basis with limited institutional support and without a commercial publisher that could offer further professional services such as expert proof reading or copy editing. currently the journal works with a review process that fully relies on what we call – partly imposed by the digital review system – section editors. section editors are on a need basis appointed out of a large, regularly reviewed, pool of regional and overseas scholars.they serve in this role comparable to the established nomenclature and mandate of senior or associate editors for an individual submission to the research article section or for entire topic-specific special sections which fall into their area of competence and for which they recruit qualified reviewers. the year 2021 has been a challenging year in many respects. without the section editors and the reviewers whom they involve, the journal would not exist. while the section editors thank their reviewers at the end of each review process through our review system i therefore like to use this end-of-the-year editorial to show my gratitude and acknowledge those members of the community who have volunteered their time and expertise as section editors. their names are listed below after this editorial’s reference list. i am looking forward to all further engagement with the ajis in the coming year. references alvesson, m., and sandberg, j. (2011). "generating research questions through problematization," academy of management review (36:2), pp. 247-271. alvesson, m., and sandberg, j. (2013). "has management studies lost its way? ideas for more imaginative and innovative research," journal of management studies (50), pp. 128-152. alvesson, m., and sandberg, j. (2014). "habitat and habitus: boxed-in versus box-breaking research," organization studies (35:7), pp. 967-987. avison, d., davison, r.m., and malaurant, j. (2018). "information systems action research: debunking myths and overcoming barriers," information and management, 55(2), pp. 177-187. avital, m., mathiassen, l., and schultze, u. (2017). "alternative genres in information systems research," european journal of information systems (26), pp. 240-247. baskerville, r. l. and myers, m. d. (2002. "information systems as a reference discipline," mis quarterly (26:1), pp. 1-14. benbya, h., nan, n., tanriverdi, h., and yoo, y. (2020). "complexity and information systems research in the emerging digital world," mis quarterly (44:1), pp. 1-17. https://doi.org/ 10.25300/misq/2020/13304 cecez-kecmanovic, d., galliers, r.d., henfridsson, o., newell, s., and vidgen, r. (2014). "the sociomateriality of information systems: current status, future directions", mis quarterly (38:3), pp. 809-830. https://faculty.essec.edu/en/research/en-information-systems-action-research-debunking-myths-and-overcoming-barriers/?&cv=53037 https://faculty.essec.edu/en/research/en-information-systems-action-research-debunking-myths-and-overcoming-barriers/?&cv=53037 https://faculty.essec.edu/en/research/en-information-systems-action-research-debunking-myths-and-overcoming-barriers/?&cv=53037 australasian journal of information systems kautz 2021, vol 25, editorial editorial for ajis 2021 6 rmit classification: trusted checkland, p. (1999). systems thinking, systems practice: includes a 30-year retrospective. chichester, uk: wiley & sons. davis, f. d. (1989). "perceived usefulness, perceived ease of use, and user acceptance of information technology," mis quarterly (13:3), pp. 319-340. https:// doi.org/ 10.2307/249008 de reuver, m., sørensen, c., and basole, r.c. (2018). "the digital platform: a research agenda," journal of information technology (33:2), pp. 124-135. https://doi.org/10.1057/s41265-016-0033-3 gregor, s., and hevner, a. (2013). "positioning and presenting design science research for maximum impact," mis quarterly (37:2), pp. 337-356. doi: 10.25300/misq/2013/37.2.01 kautz, k., bjerknes, g., fisher, j., and jensen, t. 2020. "applying complex adaptive systems theory to understand distributed participatory design in crowdsourced information systems development," australasian journal of information systems (24). https://doi.org/10.3127/ajis.v24i0.2225 kazman, r., and chen, h. (2009). "the metropolis model a new logic for development of crowdsourced systems," communications of the acm (52:7), pp. 76-84. https://doi.org/ 10.1145/1538788.1538808 lee, a. (2001). "editor’s comments," mis quarterly (25:1), pp. iii-vii. legner, c, eymann, t., hess, t., matt, c., böhmann, t., drews, p., mädche, a., urbach, n. ahlemann, f. (2017). "digitalization: opportunity and challenge for the business, and information systems engineering community," business information systems engineering (59), pp. 301-308. doi: 10.1007/s12599-017-0484-2 lusch, r.f., and vargo, s.l. (2019). "an overview of service-dominant logic, " in vargo, s.l., and lusch, r.f. (eds.), the sage handbook of service-dominant logic, pp. 3-21. london, uk: sage publications. majchrzak, a., markus, m.l., and wareham, j. (2016). "designing for digital transformation: lessons for information systems research form the study of ict and societal challenges," mis quarterly (40:2), pp. 267-277. https://doi.org/10.25300/misq/2016/ 40:2.03 plumb, m, and kautz, k. (2014). "reconfiguring early childhood education and care: a sociomaterial analysis of it appropriation," in doolin, b. et al. (eds.), information systems and global assemblages: (re)configuring actors, artefacts, organizations, ifip advances in information and communication technology, proceedings of the ifip wg 8.2 working conference on information systems and organizations, auckland, new zealand, december 11-12, 2014, vol. 446, pp 30-47. riemer, k., and johnston, r.b. (2012). "place-making: a phenomenological theory of technology appropriation," in proceedings of the international conference on information systems, orlando, florida, usa, december 16-19, 2012. riemer, k., and johnston, r. b. (2015). "organizational technology appropriation as actively accomplished "place-making"," in sandberg, j. et al (eds.), proceedings of the international symposium on process organization studies, oxford uk, pp. 1-14. https://www.researchgate.net/deref/http%3a%2f%2fdx.doi.org%2f10.1145%2f1538788.1538808?_sg%5b0%5d=jaaoyp6ew3d1tz00rldmpm4gii2_nvp7aomf8y0lc6rhinsplict8ags9hr83t508o7i1ymnva7lmhktbfgouafkoq.wyp_zvhe-dvevdny41cvp2unumo7o_aqxleen4ysyihqyfp4nfaqb414oefsrlth9qj5-t7bn3fotvahoxypla australasian journal of information systems kautz 2021, vol 25, editorial editorial for ajis 2021 7 rmit classification: trusted riemer, k., and johnston, r. b. (2016). "what makes technologies disruptive? a strong process view," in conference proceedings of the 8th international symposium on process organization studies dualities, dialectics and paradoxes in organizational life, corfu, greece, june 2016. sarker, s., chatterjee, s., xiao, x., and elbanna, a. ( 2019). "sociotechnical axis of cohesion for the is discipline: its historical legacy and its continued relevance," mis quarterly (43:3), pp. 685-719. https://doi.org/10.25300/misq/2019/13747 sørensen, c., and landau, j. s. (2015). "academic agility in digital innovation research: the case of mobile ict publications within information systems 2000–2014," journal of strategic information systems (24:3), pp. 158–170. https://doi.org/10.1016/j.jsis.2015.07. 001 sørensen, c. (2016). "the curse of the smart machine? digitalisation and the children of the mainframe," scandinavian journal of information systems (28:2), article 3. https:// aisel.aisnet.org/sjis/vol28/iss2/3 vial, g. (2019). "understanding digital transformation: a review and a research agenda," the journal of strategic information systems (28:2), pp. 118-144. venkatesh, v., morris, m. g., davis, g. b., and davis, f. d. (2003). "user acceptance of information technology: toward a unified view," mis quarterly (27:3), pp. 425– 478. oi:10.2307/30036540 wessel, l. k., baiyere, a., ologeanu-taddei, r., cha, j., and jensen, t. b. (2021). "unpacking the difference between digital transformation and it-enabled organizational transformation," journal of the association for information systems (22:1). doi: 10.17705/1jais.00655 https://misq.org/skin/frontend/default/misq/pdf/v43i3/13747_rc_sarkerabstract.pdf https://misq.org/skin/frontend/default/misq/pdf/v43i3/13747_rc_sarkerabstract.pdf australasian journal of information systems kautz 2021, vol 25, editorial editorial for ajis 2021 8 rmit classification: trusted ajis section editors in 2021: babak abedin shijia (caddie) gao natalie pang sultana lubna alam asif gill david pauleen mustafa ally manish gupta maria prokofieva alireza amrollahi riitta hekkala corina raduescu behrang assemi dirk s. hovorka kai riemer saba azeem charles jebarajakirthy jan recker wasana bandara arpan kumar khan alex richter jenine beekhuyzen shirlee-ann knight bruce rowlands emilia bellucci pooja kumari paul scifleet vijayakumar bharathi michael steven lane darshana sedera gro bjerknes nick letch amit shankar sebastian boell weng marc lim lisa soon deborah bunker yee ling boo stefan stieglitz raffaele ciriello henry linger carsten sørensen rajneesh choubisa sean b. maynard barney tan hiep cong pham shah j miah manoj thomas tingru cui simon r malian janet toland duy dang-pham annette mills holly tootell sophia duan wolfgang müller ranjan vaidya amany elbanna lemai nguyen doug vogel atj fernando heru nugroho kirsten wahlstrom julie fisher dominic o'sullivan nilmini wickramasinghe mark freeman neeraj pandey wahyu winarno copyright: © 2021 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v25i0.3853 http://creativecommons.org/licenses/by-nc/3.0/au/ editorial for the australasian journal of information systems 2021: volume 25 paper title (paper title style) australasian journal of information systems lim & mandrinos 2020, vol 24, research note multimodel comparison and hypothesis testing error 1 incidental findings on multimodel comparison, hypothesis testing error, and its solution: the case of information, knowledge, and career pursuit weng marc lim swinburne university of technology adjunct professor, swinburne business school, swinburne university of technology, australia professor and head of school, school of business, swinburne university of technology, malaysia lim@wengmarc.com marclim@swin.edu.au, wlim@swinburne.edu.my symeon mandrinos lecturer, school of business, swinburne university of technology malaysia abstract this article sheds light on an incidental discovery of hypothesis testing error and its solution in a study that compared the multimodel effects of information and knowledge on career decisions. using a sample of 500 second and third year undergraduates who were simple randomly recruited from 10 simple randomly selected universities, the study demonstrates how hypothesis testing error could occur when multimodel comparison of the effects of the same set of independent constructs (information and knowledge) on a different but related set of dependent constructs (career pursuit and rewarding career pursuit) is performed and how that error can be avoided through detailed scrutiny of the psychic distance between the dependent constructs. it is hoped that the insights herein will be useful for scholars to avoid and overcome the pitfall of hypothesis testing error arising from multimodel comparison involving a different but related set of dependent constructs. keywords: multimodel comparison; hypothesis testing error; psychic distance; information; knowledge; career pursuit; rewarding career pursuit 1 introduction incidental findings, which encapsulate peculiar insights discovered beyond the goals of the study, have become increasingly common due to the proliferation and omnipresence of advanced technologies and methodologies that empower researchers to perform highly sophisticated and rigorous investigations (booth et al., 2012; wolf et al., 2008). however, these unexpected, but potentially relevant, findings are often perceived as trivial due to the initial impression that the term “incidental findings”, unfortunately, radiates (booth et al., 2012; nature, 2005). in this paper, we contend that incidental findings can be ground-breaking. to support our contention, we report our incidental discovery of hypothesis testing error and its solution when we conducted an investigation that compared the multimodel effects of information and australasian journal of information systems lim & mandrinos 2020, vol 24, research note multimodel comparison and hypothesis testing error 2 knowledge on career decisions. in the sections that follow, we explain how and why the hypothesis testing error and its solution in our study, which we uncovered incidentally, differ from that which has been traditionally discussed and reported in the existing literature. in doing so, we demonstrate how our incidental findings contribute to theory, specifically by delineating an unconventional way on how hypothesis testing error could occur and how such an error could be resolved, and to practice, specifically by guiding the development of accurate managerial recommendations through rigorous scrutiny of empirical findings. 2 theoretical background 2.1 multimodel comparison multimodel comparison is a method that is often used but rarely defined, explicitly, by past scholars, as evidenced by a google scholar search for “multimodel (and multi-model) comparison” that yield no result that could offer a concrete, explicit definition of the method. nonetheless, we did consider, in consultation with experts, how multimodel comparisons have been operationalized in the extant literature, which led to our conclusion that it is arguably safe to say, explicitly, that multimodel comparison generally refers to “the appraisal of two or more models.” this appraisal, which may be conceptual (e.g., lim, 2018; wilson, 1999) or empirical (e.g., ba et al., 2014; gillingham et al., 2018), is typically undertaken to observe the similarities and differences in the models under study. in doing so, we find that multimodel comparison allows appraisers to delineate the distinct and overlapping boundaries of different models as well as the magnitude and replicability of effects across studied models (e.g., ba et al., 2014; gillingham et al., 2018; lim, 2018; wilson, 1999). 2.2 hypothesis testing a hypothesis is a testable assumption made about a target population. to test this assumption, a formal procedure in the form of hypothesis testing is undertaken, whereby the goal of hypothesis testing is not the verification but the falsification of the null hypothesis (h0) (i.e., non-existence of association) in favour of the alternative hypothesis (ha) (i.e. existence of association) using statistical tests on usable data from a sample of the target population (banerjee et al., 2009; wasserstein & lazar, 2016). the probability value (i.e., p-value), which indicates the probability to obtain an effect equal to or more extreme than the one observed when the null hypothesis is presumed to be true (and not the probability of the null hypothesis being true), is a statistical measure of the strength of evidence that can be used to guide the rejection or non-rejection of the null hypothesis in hypothesis testing (biau et al., 2010; wasserstein & lazar, 2016). four outcomes are possible from hypothesis testing: two outcomes, whereby one is a true positive that rejects the null hypothesis and the other is a true negative that fails to reject the null hypothesis, suggest that the sample and the reality in the population are concordant, whereas the remaining two outcomes suggest that an incorrect inference would have been made as a result of making a type one error or a type two error (biau et al., 2010; rothman, 2010). more specifically, a type one error is a false positive, whereby a null hypothesis that is true is rejected, whereas a type two error is a false negative, whereby a null hypothesis that is false is not rejected (rothman, 2010). to mitigate hypothesis testing error, the existing literature suggests that using a smaller level of statistical significance for rejecting or not rejecting the null hypothesis can help to minimize type one error, that using a larger sample size can help to reduce the probability of type two error, and that reporting effect size (i.e., the australasian journal of information systems lim & mandrinos 2020, vol 24, research note multimodel comparison and hypothesis testing error 3 arithmetic or geometric mean of the two confidence interval limits) and precision (i.e., the narrowness of the confidence interval) is encouraged to avoid misinterpretation and to enrich interpretation of study results (banerjee et al., 2009; biau et al., 2010; rothman, 2010; wasserstein & lazar, 2016). 3 method 3.1 the study (and its hypothesis development) in our study, we examined the relationship between information, knowledge, and career decisions. in particular, we wanted to investigate the extent to which information and knowledge had an influence on career decisions. to do so, we made two contentions. first, we contended that information is sustainable on the basis that it can be stored permanently (e.g., information centre, such as library) whereas knowledge is transient on the basis that it is an elusive entity that can either diminish or evolve over time (e.g., our understanding of what we have learnt at the university) (zins, 2007). second, we contended that investments in career decisions (e.g., to pursue not only a career [e.g., full-time job], but also a rewarding career [e.g., high income full-time job]) is determined by both information and knowledge, though information is posited to have a greater effect than knowledge on the basis that the sustainability of information provides greater assurance than the transience of knowledge in decision making (gupta et al., 2018). thus, we hypothesized that: h1. information encourages more than just a career pursuit. h2. knowledge encourages more than just a career pursuit. h3. information is more profound than knowledge for encouraging more than just a career pursuit. 3.2 the instrument a survey approach by means of questionnaire distribution was used to collect data for the study. more specifically, four single-item measures on a five-point likert scale (i.e., ‘1’ indicating ‘strongly disagree’ and ‘5’ indicating ‘strongly agree’) were used to measure the four constructs under study. more specifically, the items measuring the constructs pertaining to information, knowledge, career pursuit, and rewarding career pursuit are presented accordingly, as follows: “the career centre provides me with career information”, “i am able to see clearly the potential career rewards”, “i plan to be employed full-time upon graduation”, and “i am willing to work hard to achieve a rewarding career”. the rationale for taking a unidimensional approach to construct measurement was to allow for a macrounderstanding of the evaluation of information, knowledge, and career decisions, which is consistent with existing studies that suggest that taking a unidimensional approach to form a macro-understanding of the researched phenomenon is acceptable as no significant differences generally exist between multiand single-item measures (bergkvist & rossiter, 2007; walsh & mitchell, 2010). more important, the questionnaire containing these items were pre-tested and pilot-studied prior to administration in the main study. 3.3 the sample the sample consisted of 500 second and third year undergraduates who were simple randomly recruited from 10 simple randomly selected universities over two years from 2016 to 2018 (see lim and ting [2012] for more information on simple random sampling). the australasian journal of information systems lim & mandrinos 2020, vol 24, research note multimodel comparison and hypothesis testing error 4 sample size of 500 is deemed to be large and thus allowed us to mitigate type two error. the choice of target population for our sample, which we used as a sampling criterion, was predicated on the fact that second and third year undergraduates have normally decided on their academic major that will influence the choice of the professional career in the future, which is in line with the focus of the dependent constructs under study (i.e., career pursuit and rewarding career pursuit). more important, the students in our sample participated in our study voluntarily and were assured of their anonymity as no personal identifiable data was collected. most of them were locals (99.0%), females (65.6%), and second and third year undergraduate students enrolled across eight public universities (87.2%). no missing data is reported as participants in the study had to complete all questions before returning the questionnaire to the surveyor. 4 findings multiple regression analyses were performed to examine whether information (h1) and knowledge (h2) encourages more than just a career pursuit using a sample of second and third year undergraduates. importantly, we used a small p-value of .01 to mitigate type one error and we furbished details on confidence interval, effect size, and precision to enrich our reporting of results. figure 1 presents the results of the first regression, which suggests that information (β = .213, p = .000 < .01, ci = .125, .304, es = .215, p = .179) and knowledge (β = .214, p = .000 < .01, ci = .124, .302, es = .213, p = .178) encourage career pursuit, whereas figure 2 presents the results of the second regression, which suggests that information (β = .129, p = .000 < .01, ci = .050, .208, es = .129, p = .158) and knowledge (β = .297, p = .000 < .01, ci = .219, .376, es = .298, p = .157) encourage a rewarding career pursuit.1 multimodel comparison of the first and second regression suggests that a strong case can be built to support h1 and h2. figure 1. information, knowledge, and career pursuit figure 2. information, knowledge, and rewarding career pursuit more specifically, the first regression offers evidence to first establish that information and knowledge encourage career pursuit, which in turn, allows the second regression, which offers evidence to then establish that information and knowledge encourage rewarding career 1 β = regression coefficient, p = significance value, ci = confidence interval, es = effect size, and p = precision. information knowledge career pursuit .213*** .214*** r2 = .09 information knowledge rewarding career pursuit .129*** .297*** r2 = .13 australasian journal of information systems lim & mandrinos 2020, vol 24, research note multimodel comparison and hypothesis testing error 5 pursuit, to build on those findings to collectively provide support for h1 and h2 with medium effect size and reasonably narrow precision (rothman, 2010; durlak, 2009). however, a multimodel comparison of the first and second regression suggests that knowledge has a more profound impact than information for encouraging more than just a career pursuit, which is in contrast to our third hypothesis. this finding, in turn, does not provide support for h3. upon further scrutiny, we incidentally discovered that we had conducted a type two error as we mistakenly rejected an alternative hypothesis in the form of h3, and in doing so, not rejecting a null hypothesis that is false. to elaborate, our further scrutiny was undertaken through the computation of psychic distance, which is the perceived distance between two different but related dependent constructs (sousa & bradley, 2006). when contextualized in our study, psychic distance offers an alternative measure to test our third hypothesis, specifically by examining whether information and knowledge can encourage more than just a career pursuit when the psychic distance between career pursuit and rewarding career pursuit is reduced, an advantage that we were able to leverage as a result of collecting data for the purpose of multimodel comparison. more specifically, psychic distance in our study was computed by measuring the gap between career pursuit and rewarding career pursuit, where the gap of ‘0’, ‘1’, ‘2’, and ‘3’ that emerged were coded as ‘no’, ‘small’, ‘medium’, and ‘large’ psychic distance. following the computation of psychic distance, we ran an analysis of variance to scrutinize the effect of information and knowledge in encouraging more than just a career when psychic distance is reduced. findings from this analysis, which are reported in table 1, suggest that information had a significant effect in encouraging more than just a career pursuit when psychic distance is reduced from large to no psychic distance (f = 20.33, p = .00 < .01), whereas knowledge did not produce any significant improvements in encouraging more than just a career pursuit when psychic distance is reduced from large to no psychic distance (f = .73, p = .39 > .01). these findings, in turn, nullified type two error to provide support for h3. table 1. information, knowledge, and psychic distance of career pursuit 5 conclusion in summary, this article has shed new light on an incidental discovery of hypothesis testing error and its solution in a study that compared the multimodel effects of information and factor psychic distance n m ea n s ta n d ar d d ev ia ti o n s ta n d ar d e rr o r weighted linearity equality of means f s ig . w el ch s ig . b ro w n f o rs y th e s ig . information no 203 3.45 .86 .06 20.33 .00 6.62 .00 5.96 .00 small 199 3.17 .95 .07 medium 68 2.94 1.18 .14 large 30 2.90 1.09 .20 knowledge no 203 3.79 .83 .06 .73 .39 .72 .54 .68 .57 small 199 3.87 .91 .07 medium 68 3.68 1.30 .16 large 30 3.63 1.40 .26 australasian journal of information systems lim & mandrinos 2020, vol 24, research note multimodel comparison and hypothesis testing error 6 knowledge on career decisions in the form of career pursuit and rewarding career pursuit. the incidental findings reported herein should arguably be substantively significant as the discovery was in light of the implementation of the recommendations by past scholars to mitigate hypothesis testing error. while it is hoped that the insights herein will be useful for scholars to avoid and overcome the pitfall of hypothesis testing error arising from multimodel comparison involving a different but related set of dependent constructs, we concede that further exploration remains necessary to uncover the potential of other situations where hypothesis testing error may occur in multimodel comparison and the potential solutions that can correct for that error. declaration of conflicting interests the author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article. funding this research received no specific grant from any funding agency in the public, commercial, or not-for-profit sector. references ba, j., keenlyside, n. s., latif, m., park, w., ding, h., lohmann, k., mignot, j., menary, m., otterå, o. h., wouters, b., & melia, d. s. (2014). a multi-model comparison of atlantic multidecadal variability. climate dynamics, 43(9–10), 2333–2348. https://doi.org/10.1007/s00382-014-2056-1 banerjee, a., chitnis, u. b., jadhav, s. l., bhawalkar, j. s., & chaudhury, s. (2009). hypothesis testing, type i and type ii errors. industrial psychiatry journal, 18(2), 127–131. https://doi.org/10.4103/0972-6748.62274 bergkvist, l., & rossiter, j. r. (2007). the predictive validity of multiple-item versus singleitem measures of the same constructs. journal of marketing research, 44(2), 175–184. https://doi.org/10.1509/jmkr.44.2.175 biau, d. j., jolles, b. m., & porcher, r. (2010). p value and the theory of hypothesis testing: an explanation for new researchers. clinical orthopaedics and related research, 468(3), 885– 892. https://doi.org/10.1007/s11999-009-1164-4 booth, t. c., waldman, a. d., wardlaw, j. m., taylor, s. a., & jackson, a. (2012). management of incidental findings during imaging research in “healthy” volunteers: current uk practice. british journal of radiology, 85(1009), 11–21. https://doi.org/10.1259/bjr/73283917 durlak, j. a. (2009). how to select, calculate, and interpret effect sizes. journal of pediatric psychology, 34(9), 917–928. https://doi.org/10.1093/jpepsy/jsp004 gillingham, k., nordhaus, w., anthoff, d., blanford, g., bosetti, v., christensen, p., mcjeon, h., & reilly, j. (2018). modeling uncertainty in integrated assessment of climate change: a multi-model comparison. journal of the association of environmental and resource economists, 5(4), 791–826. https://doi.org/10.1086/698910 gupta, g., tan, k. t., ee, y. s., & phang, c. s. (2018). resource-based view of information systems: sustainable and transient competitive advantage perspectives. australasian journal of information systems, 22. https://doi.org/10.3127/ajis.v22i0.1657 australasian journal of information systems lim & mandrinos 2020, vol 24, research note multimodel comparison and hypothesis testing error 7 lim, w. m. (2018). dialectic antidotes to critics of the technology acceptance model: conceptual, methodological, and replication treatments for behavioural modelling in technology-mediated environments. australasian journal of information systems, 22. https://doi.org/10.3127/ajis.v22i0.1651 lim, w. m., & ting, d. h. (2012). a toolkit of sampling and data analysis techniques for quantitative research. munich, germany: grin verlag. nature. (2005). how volunteering for an mri scan changed my life. nature, 434(7029), 17. https://doi.org/10.1038/434017a rothman, k. j. (2010). curbing type i and type ii errors. european journal of epidemiology, 25(4), 223–224. https://doi.org/10.1007/s10654-010-9437-5 sousa, c. m., & bradley, f. (2006). cultural distance and psychic distance: two peas in a pod? journal of international marketing, 14(1), 49–70. https://doi.org/10.1509/jimk.14.1.49 walsh, g., & mitchell, v. w. (2010). the effect of consumer confusion proneness on word of mouth, trust, and customer satisfaction. european journal of marketing, 44(6), 838–859. https://doi.org/10.1108/03090561011032739 wasserstein, r. l., & lazar, n. a. (2016). the asa’s statement on p-values: context, process, and purpose. american statistician, 70(2), 129-33. https://doi.org/10.1080/00031305.2016.1154108 wilson, t. d. (1999). models in information behaviour research. journal of documentation, 55(3), 249–70. https://doi.org/10.1108/eum0000000007145 wolf, s. m., lawrenz, f. p., nelson, c. a., kahn, j. p., cho, m. k., clayton, e. w., fletcher, j. g., georgieff, m. k., hammerschmidt, d., hudson, k., & illes, j. (2008). managing incidental findings in human subjects research: analysis and recommendations. journal of law, medicine & ethics, 36, 219–248. https://doi.org/10.1111/j.1748-720x.2008.00266.x zins, c. (2007). conceptual approaches for defining data, information, and knowledge. journal of the american society for information science and technology, 58(4), 479–493. https://doi.org/10.1002/asi.20508 copyright: © 2020 lim & mandrinos. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v24i0.2555 http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems lim & mandrinos 2020, vol 24, research note multimodel comparison and hypothesis testing error 8 paper title (paper title style) australasian journal of information systems tretiakov, bensemann & jurado 2023, vol 27, research article women’s intentions to return to it employment 1 social outcome expectations and women's intentions to return to it employment alexei tretiakov massey university, new zealand jo bensemann massey university, new zealand j.bensemann@massey.ac.nz tanya jurado massey university, new zealand abstract women leaving it employment for childcare or other reasons, and never returning, is a phenomenon that contributes to the underrepresentation of women in it. however, potential women returners, women who have recently left it employment and may or may not return, remain an under-researched group. we studied the effects of social outcome expectations on the intention to return to it employment for 182 potential women returners from new zealand, australia, and the united states. the data were obtained via a survey questionnaire. expectations of friendly co-workers, work-life balance, and family proximity were included; and the expectations of friendly co-workers had a statistically significant effect on the intentions of potential women returners to return to it employment. the results highlight the difficulty of creating an environment that encourages potential women returners to return to it because, unlike work-life balance or family proximity, friendly co-workers is a factor that is difficult to control via managerial interventions. for practice, the results suggest that organisations should promote an environment friendly to women, which in part may be achievable by implementing agile approaches to organizing it work. keywords: it, women, social outcome expectations. 1 introduction the problem of underrepresentation of women in the it industry has been a subject of renewed attention in recent literature (alok et al., 2021; happe & buhnova, 2022; happe et al., 2021; molnar et al., 2021; serenko & turel, 2021; singh & vanka, 2021; tokbaeva & achtenhagen, 2023; trauth & connolly, 2021). indeed, the underrepresentation remains, even though it has been highlighted since the end of the 20th century (baroudi & igbaria, 1994; panteli et al., 1999; truman & baroudi, 1994). for example, in 2020 only 29% of employees in ict occupations in australia were women (https://www.statista.com). even though to a considerable extent the underrepresentation of women in the it industry can be attributed to women and girls staying away from it careers, women leaving the it industry is also an important factor (armstrong et al., 2018; castaño collado & webster, 2011; herman, 2015b; panteli, 2006, 2012; panteli & pen, 2009; singh & vanka, 2021). for example, in a survey of it companies in india (raghuram et al., 2017), 40% of the companies reported that more than 20% of their women employees going on a maternity leave did not return, with 16% of the companies reporting that more than half for their women employees did not return. nonetheless, there is a paucity of studies of potential women returners, women who have left australasian journal of information systems tretiakov, bensemann & jurado 2023, vol 27, research article women’s intentions to return to it employment 2 the it industry and might or might not return (singh & vanka, 2021). in particular, factors affecting the decision to return are not well understood. the present study aims to investigate the role of perceived social outcomes of re-joining the it industry, which have been argued to be particularly relevant to women (ahuja, 2002; armstrong et al., 2018; holth et al., 2017; tokbaeva & achtenhagen, 2023). the study pursues the following research question: how do the perceptions of social outcomes of re-joining the it industry by women who have left it affect their intentions to re-join? to address the research question, a research panel-based survey of potential women returners was conducted, measuring a range of perceived social and family outcomes and the intention to return to it employment. the data were analysed using pls sem. the rest of the paper is organised as follows. first, the relevant prior research is synthesized to reveal the research gaps and to justify the focus of the present study. then, the hypotheses are developed, followed by method, results, discussion, and conclusions sections. 2 background an influential study by ahuja (2002) offered a process model for women's careers in it, highlighting career choice, persistence, and career advancement stages. in the persistence stage women face a considerable work-family conflict as they need to devote attentional resources to raising children, which often results in women leaving and possibly never rejoining the it workforce. ahuja uses the metaphor of a leaky pipeline to describe this situation. the kaleidoscope model of women's career experiences by mainiero and sullivan (2005) suggests a process model that is not entirely dissimilar to ahuja's model. in the kaleidoscope model, challenge (pursuing greater responsibility and pay) is followed by balance (emphasis on accommodating both work and family responsibilities), which is followed by search for authenticity for a better subjective career experience after the children grow up. even if the end points in the two models are, arguably, inverted, the middle stage is largely identical, highlighting the importance of understanding how women combine career pursuit or career maintenance with family commitments. additionally, the two models come from very different ideological premises. the kaleidoscope model celebrates flexibility in women's careers, rather than implicitly asserting the value of linear progression upward, as in ahuja's model. a common strategy (or pattern, as it is not necessarily planned or deliberate) for potential women returners is to leave work for a period of time to re-join (or not re-join) when children are older, and the family situation is less time intensive (herman, 2015a). the individual difference perspective on women employed in the it industry asserts that the experiences of women belonging to different cultures and backgrounds may differ in important ways (trauth, 2002). women also leave the it workforce for a variety of reasons other than raising children, for example, to travel for an overseas experience (wilson et al., 2009), or to relocate to follow a partner pursuing a business or career opportunity (van der velde et al., 2017). from the perspective of reducing gender imbalance in the it industry, the important question is whether women who leave the industry for whatever reason will eventually return to it, or will they find employment in a different industry or end up out of the workforce altogether, thus realising the "leaky pipeline" metaphor (herman, 2015a; singh & vanka, 2021; tomlinson australasian journal of information systems tretiakov, bensemann & jurado 2023, vol 27, research article women’s intentions to return to it employment 3 et al., 2009). the issue of women returning to it careers has also been highlighted in the practitioner literature (barratt, 2018). nonetheless, there are very few empirical studies of women returners (those who have already returned) or potential returners (those who have left and might, or might not, return) to the it industry. a recent interview-based study by tokbaeva and achtenhagen (2023) of women working in the it industry in sweden included some women returners and adopted a strong individual responsibility perspective, taking a view that women stayed in the pipeline by developing career resilience as an individual trait. in contrast, a study in india (singh & vanka, 2021) emphasised resources and deliberate external support, highlighting upskilling, corporate career restart programmes, digital community platforms, and the availability of day care centres. in a uk study, kirton and robertson (2018) adopted a feminist perspective that highlighted "inequality regimes" systems of practices deliberately or unconsciously designed to "combine and interact to produce and maintain gender inequality in the it workplace" (p. 157). kirton and robertson (2018) highlighted the existence of an insufficient flexibility discourse in which it is assumed that women returners, because of the need to care for their family, cannot be flexible enough to cope with core responsibilities, resulting in them being assigned to an informal "mommy track" of lesser responsibility and fewer career prospects. in an interview-based study of women returners and non-returners to science, engineering, and it careers in the uk, herman (2015a) suggested a trichotomy of rebooting (returning to the initial career/industry), re-rooting (returning to work in a different industry), and retreating (staying out of the workforce altogether). the study highlighted the persistence of initial career identities even in those who "re-rooted" or "retreated", suggesting that under the right circumstance these women would return. a longitudinal study by the same author interviewed mature female students in the uk attempting to return to work in stem (science, technology, engineering, and mathematics) careers and found that locality was an important factor – women were constrained by the location of their partner's job or by the availability of extended family members for childcare support, and they would only consider local opportunities, thus drastically reducing their chances of success (herman, 2015b). a quantitative, cross-sectional survey-based study of women returners to the it industry in india by ravindran and baral (2014) found that job satisfaction and career commitment were positively affected by implementation of family friendly policies and an organisational culture that values integration of family and work, fairness, and diversity. rajesh et al. (2013) conducted a descriptive study of attitudes of managers at different levels towards women returners in the it industry in india, revealing that the attitudes are not always positive (e.g., middle and junior managers were much more likely to feel apprehensive about the performance of women returners than to be willing to help them when required). finally, panteli (2012) reported the results of an evaluation of a programme in the uk intended to help potential women returners to find it employment. panteli (2012) highlighted the positive feedback by the programme participants about network-building activities. both network building among potential women returners and with currently active it professionals were seen as valuable confidence building exercises, rather than as being instrumental for getting interviews and job offers. this study by panteli (2012) was the only empirical study that focused on potential women returners to the it industry, rather than on returners or on australasian journal of information systems tretiakov, bensemann & jurado 2023, vol 27, research article women’s intentions to return to it employment 4 women working in it in general, but it focused more on evaluating an intervention programme, not on potential returners in a broad sense. however, from the perspective of addressing the underrepresentation of women in the it industry, understanding potential returners is more important than understanding those who have already returned, because the decision to re-join the industry is made by potential returners, and not all of them make the decision to return. the present study contributes to addressing this research gap by empirically studying the intention to return by potential women returners to the it industry from the perspective of outcome expectations. outcome expectations is an important class of factors that influence behaviour, and they are included in a number of theories that explain behaviour, such as ajzen's theory of planned behaviour (ajzen, 1991) and, in the mis field, the unified theory of acceptance and use of technology (venkatesh et al., 2003) (as performance expectancy). thus, women's behaviour with respect to re-joining the it industry after a break is likely to be influenced by their expectations of the outcomes of such a move. the present study follows venkatesh et al. (2017) in considering outcome expectations with respect to it employment from the total rewards perspective (fulmer & li, 2022; lawler, 2011). rather than considering the outcomes of employment in the it industry solely in terms of pay and promotion opportunities, we consider a broader range of benefits of being employed. venkatesh et al. (2017) identified three categories of outcomes of being employed in the it industry: (a) extrinsic or instrumental outcomes (direct external consequences, such as pay, promotion, and job security), (b) social outcomes (outcomes pertinent to social relationships and family), and (c) intrinsic outcomes (individual outcomes that arise from engaging in work activity, such as skills development). research on women's choice of, and advancement in, it careers has consistently highlighted the importance for women of social and family factors, which are often seen as a differentiator between genders in their it career-related behaviours (ahuja, 2002; armstrong et al., 2018; holth et al., 2017; tokbaeva & achtenhagen, 2023). in these studies, both social relationships outside of work (such as caring for older relatives) and in it work (where women's social abilities can make them particularly valuable employees) have been found to be particularly relevant to women. therefore, rather than considering the broadest possible range of outcomes of women being employed in the it industry, the present study focuses on social outcomes. in the spirit of the theory of planned behaviour and related theories, it is anticipated that women's expectations of the social outcomes of re-joining the it industry influence their intentions (and, ultimately their behaviour) with respect to finding an it job after a break. hence, the research question of the present study formulated in the introduction: how do the perceptions of social outcomes of re-joining the it industry by women who have left it affect their intentions to re-join? studying the actual behaviour remains outside of the scope of the present study because the relevant data were not feasible to obtain. however, it has been demonstrated in prior studies that intent is strongly associated with actual behaviour (ajzen, 2002; westaby, 2005). 3 hypotheses development we follow venkatesh et al. (2017) in conceptualising social outcomes of working in it in terms of three constructs: friendly co-workers, work-life balance, and family proximity. in this section we consider these constructs in the context of women considering a return to it employment and formulate the hypotheses of the present study. australasian journal of information systems tretiakov, bensemann & jurado 2023, vol 27, research article women’s intentions to return to it employment 5 3.1 friendly co-workers male it workers may engage in gendered behaviours that result in a work environment uncomfortable to women or excluding women from access to tacit knowledge or decision making (crump & logan, 2000; gupta, 2015; li, 2021). wu (2020) summarised such behaviours by describing three archetypes of male it workers: brogrammers, who engage in hypermasculine competitive behaviours, tech hobbyists (or geeks), who derive life satisfaction from technology and thus lack work/non-work boundary, and coding peasants, who are mainly immigrants pursuing breadwinner roles. for women, engaging in hypercompetitive behaviours is difficult because they tend to pursue harmony, rather than conflict, in social interactions (niederle & vesterlund, 2008). further, women tend to attribute more importance than men to family relationships, so that devoting themselves fully to technology (by, for example, working extremely long hours) is not a viable option. finally, the worldview suggesting that men are supposed to be breadwinners is in contradiction with women pursuing it careers. thus, the presence of these archetypes in the workplace is bound to make it difficult for women to fit in, particularly when they are in a minority. moreover, as a minority group not fitting the workplace culture defined by the dominant brogrammer, geek, and male breadwinner archetypes, women may become targets of microaggressions, discursive steps undermining their confidence and professional identities (kim & meister, 2022; wilkins-yel et al., 2019). kim and meister (2022) categorized microaggressions deployed (intentionally or unintentionally) against women working in stem occupations, with the most important categories being devaluation of technical competence (assuming that women are not technically competent) and devaluation of physical presence (failing to engage women in decision making situations). one may expect that women will perceive co-workers in such an environment to be not friendly, both in a personal sense (as individuals) and in a professional sense (as colleagues). friendly co-workers are a desirable outcome from two distinctive, but complementary, perspectives. friendly co-workers will help the returning woman employee to become more productive more quickly. information technology changes very fast, as do the formal and informal organisational structures that support these changes (armstrong et al., 2007; carayon et al., 2006). to become productive in an it-related role, the returning employee needs to acquire substantial technological, organisational, and customer knowledge, whether the return is to the workplace she left or to a new one. a lot of this new knowledge is tacit because often, in a rapidly changing environment, the benefits of codifying knowledge do not justify the effort. thus, this knowledge can only be obtained from other employees, so that a friendly attitude of other employees making knowledge transfer effortless (or, even possible) is essential (milligan et al., 2013). furthermore, women often rely on their social skills to shape their jobs to offer value to their employer, and they do so to a greater extent than men (woodfield, 2002). these skills are particularly relevant in the current it workplace because organisations rely more and more on agile approaches and practices that emphasise direct informal communication over formal documentation and planning (denning, 2018; hoda et al., 2018). significantly, social skills are easier to employ in a friendly environment. moreover, it has been reported that women, more than men, derive life satisfaction from social exchanges (yang & girgus, 2019). therefore, a friendly environment is particularly valuable to a woman, quite independently from being productive on the job. at the same time, social exchanges with co-workers that are not immediately job related serve to improve australasian journal of information systems tretiakov, bensemann & jurado 2023, vol 27, research article women’s intentions to return to it employment 6 communication and to build trust, ultimately contributing to on-the-job performance (xie & li, 2021; yeow et al., 2019). thus, potential women returners who – based on their prior experience in the it workplace, as well as on other inputs, such as from family, friends, and media – expect the it workplace to feature friendly co-workers are more likely to intend to return: h1: higher expectations of friendly co-workers are associated with greater intent by women to return to work in the it industry. 3.2 work-life balance work-life balance refers to the ability of an employee to maintain important non-work social activities, such as caring for their children or older relatives, socialising with friends or neighbours, or pursuing community work (beauregard & henry, 2009). work-life balance may be difficult to maintain for an it worker because the constantly changing technology requires constant upskilling and because much of it work involves client-oriented work on projects combining high technology and business uncertainty with strict deadlines, or running global services that require 24/7 uptime, with strict penalties for downtime (alok et al., 2021; panteli, 2006). because it work is knowledge intensive, it takes time to become productive working on an it project and asking employees to work overtime to mitigate emergencies is easier than adding new workers to manage workload spikes (legault & chasserio, 2012). however, the agile movement has been associated with a trend to move away from excessive reliance on overtime work on the grounds that work conducted at an expected pace results in more predictable, manageable outcomes and in superior quality (hoda et al., 2018). excessive workload (even when compensated) and unpredictable workload, with an implicit assumption that work should always take priority over non-work activities, is detrimental to work-life balance, and is difficult to reconcile with taking care of young children or other nonwork activities requiring commitment. indeed, work-life balance has been highlighted very strongly as a critical factor to achieve gender balance in the it workforce (ahuja, 2002; sardeshmukh & srinivasan, 2014; trauth & connolly, 2021), particularly in connection with the retention of women it workers. thus, potential women returners who expect to be able to maintain important non-work social activities after returning to the it workforce are more likely to intend to return: h2: higher expectations of work-life balance are associated with greater intent by women to return to work in the it industry. 3.3 family proximity family proximity refers to the extent to which a job enables an employee to be physically located close to family members, thus enabling in-person interactions with the family. high quality it work is often concentrated in large or important cities (such as auckland or wellington in new zealand) or in geographically concentrated entrepreneurial systems, such as silicon valley in the united states (bresnahan & gambardella, 2004; coletti, 2010). thus, getting a quality it job often involves relocation, which may be incompatible with a woman's family situation (e.g., her spouse may already have a job at the location where she currently resides, or her aging parents she is caring for may be unwilling/unable to relocate). moreover, client-based it work may require the employee to frequently travel to work on clients' premises, even if her base of operations is located close to the family residence (morganson et australasian journal of information systems tretiakov, bensemann & jurado 2023, vol 27, research article women’s intentions to return to it employment 7 al., 2010; panteli, 2006). training needed to keep up with the constantly changing technology may also require travel. finally, long commutes may also greatly reduce the time a woman can be in physical proximity with her family (d'mello, 2006; morris & zhou, 2018). interestingly, it has been reported that societal expectations make it more difficult for a male partner to interrupt his career to follow a woman who wishes to pursue a career opportunity than the other way round. moreover, because of expectations that women are the primary care givers for older relatives, family proximity is a greater issue for women than it is for men (shah & barker, 2020; srinivasan et al., 2013). thus, potential women returners who expect to be able to remain in physical proximity to their family after returning to the it workforce (e.g., they may already reside in a large city with work-related travel likely be within the city) are more likely to intend to return: h3: higher expectations of family proximity are associated with greater intent by women to return to work in the it industry. the research model utilised in this study is shown in figure 1. it consists of the three social outcome expectations discussed above and a set of control variables, which are used to assess the behavioural intention of potential women returners. h2 figure 1. research model (age = age, auq australia, usq = us) 4 method to test the hypotheses of the present study, we conducted a panel-based cross-sectional survey of potential women returners. this design enabled us to reach enough respondents to conduct statistical tests. 4.1 sample and procedure the sample included women who had left an it-related role within two years prior to when the survey was conducted. the participants were from new zealand, australia, and the united states. the study targeted participants aged between 18 and 45 years old, and thus considerably below the retirement age, which is 65 years old in new zealand and 66 years old behavioural intention family proximity work-life balance friendly co-workers control variables (age, auq, usq) h3 h1 australasian journal of information systems tretiakov, bensemann & jurado 2023, vol 27, research article women’s intentions to return to it employment 8 in australia and in the united states. by including participants from more than one country, the study achieved greater generalisability (with country available as a control variable). while acknowledging that the experiences of older female workers are important and deserve attention, as highlighted by myers et al. (2021), we focused on younger workers. this is consistent with the view that the perceptions and behaviour of younger workers have greater impact on the it workforce composition because they can have longer careers within the industry. the data (182 responses) were obtained from a qualtrics research panel: qualtrics served our questionnaire deployed using qualtrics online survey software to the members of their research panel who fulfilled the criteria described in the previous paragraph. thus, the respondents were not selected at random from the full population of potential women returners to the it industry. this was an essential aspect of the research design because a database listing potential women returners does not exist. there was no missing data, and all responses were included in the analysis. massey university ethics procedures were followed, resulting in low-risk notification number 4000025367 recorded. the three countries were almost equally represented in the sample, with 61 responses from new zealand, 60 from australia, and 61 from the united states. the best represented age range was 30 to 33 years old (39 responses), and the least represented was the 42 to 45 age range (17 responses). the rest of the responses were distributed as follows: 18 to 21 years old (25 responses), 22 to 25 years old (23 responses), 26 to 29 years old (24 responses), 34 to 37 years old (33 responses), and 38 to 41 years old (21 responses). overwhelmingly, the respondents were highly educated: 153 of them (84% of the sample) had a college or university education and almost half of these (68, or 37% of the sample) had completed a graduate or postgraduate degree. this is not surprising, because a university education is often a prerequisite for finding an it job. as to the it jobs left by the respondents, the most frequently reported specialisations were ict trainer (15 participants), ict account manager (13), ict project manager (13), web designer (11), and software engineer (10). some other roles were chief information officer (7 participants), ict sales assistant (4), and telecommunications cable jointer (1). this represents a very broad range of specialisations, covering software, hardware, design, business/sales, and management. about two-thirds (114, or 63%) of the participants had left their it jobs less than a year before the survey was conducted. 4.2 measurement items used to operationalise the constructs of the research model are listed in table 1. three social outcome expectations – friendly co-workers (fcov), work-life balance (wlb), and family proximity (famp) – were measured using the operationalisations developed by venkatesh et al. (2017). items for measuring behavioural intention (bi) were based on davis (1989), following the common practice in mis research (see, for example, (venkatesh et al., 2012)). the survey questions were paraphrased to fit the context of women returning to it employment. all measures have been initially developed as reflective and were used as such in the present study. items were measured on five-point likert scales, from "not at all" to "great deal" for fcov, wlb, and famp, and from "strongly agree" to "strongly disagree" for bi. australasian journal of information systems tretiakov, bensemann & jurado 2023, vol 27, research article women’s intentions to return to it employment 9 operationalisation of constructs friendly co-workers please rate how much you expect each of the following friendly co-workers related characteristics to be present in your job if you were employed in an information technology (it) related role. fcov1 friendly co-workers. fcov2 collegial co-workers. fcov3 co-workers who are supportive. work-life balance please rate how much you expect each of the following work-life balance related characteristics to be present in your job if you were employed in an information technology (it) related role. wlb1 being able to balance my family life with my work life. wlb2 having time for my personal life. wlb3 a work environment that supports work-family balance. family proximity please rate how much you expect each of the following family proximity related characteristics to be present in your job if you were employed in an information technology (it) related role. famp1 being in the same geographic location as my immediate family (i.e., spouse/life partner, children, parents, siblings). famp2 living in the same area as my immediate family. famp3 being in very close geographical proximity of my immediate family. behavioural intention bi1 to find an information technology (it) related job, i plan to access information about openings for it related jobs during the next four weeks. bi2 to find an information technology (it) related job, i intend to access information about openings for it related jobs during the next four weeks. bi3 to find an information technology (it) related job, i will expend effort accessing information about openings for it related jobs during the next four weeks. table 1. operationalisation of constructs age (age) and country were included as control variables, with country operationalised as two dummy variables (auq and usq) taking the value of 1 if the respondent was, respectively, from australia or us, or taking the value of 0 otherwise. new zealand, thus, served as the base. 5 results the research model was validated by using pls modelling using r package seminr (version 2.3.0), with vif analysis conducted using r package car (version 3.0-12). pls fitted the purpose of the present study because it allows validation of both measurement and structural models, it can be used with small data sets, and it is suitable for exploratory research (hair et al., 2019). bootstrap analysis with 1000 subsamples was used to test the statistical significance of the path coefficients. 5.1 measurement model to assess the measurement model, item reliability, internal consistency reliability, convergent validity, and discriminant validity were assessed. item loadings are given in table 2. all item loadings on their own constructs were higher than the threshold of 0.7, suggesting item reliability (hair et al., 2019). furthermore, for all constructs, cronbach alpha and composite reliability values were above 0.7, suggesting internal consistency reliability (hair et al., 2019), and ave values exceeded 0.5, suggesting convergent validity (hair et al., 2019) (see table 3). australasian journal of information systems tretiakov, bensemann & jurado 2023, vol 27, research article women’s intentions to return to it employment 10 all items loaded on their own constructs higher than on other constructs in the model (see table 2) and for all constructs, inter-construct correlations involving a construct were lower than the construct's square root of ave (see table 4), suggesting discriminant validity (fornell & larcker, 1981). finally, htmt ratios of correlations for all pairs of constructs were below the threshold of 0.90 (see table 5), thus meeting the recently introduced criterion of convergent validity by henseler et al. (2015). fcov wlb famp bi fcov1 0.865 0.635 0.468 0.199 fcov2 0.851 0.507 0.359 0.233 fcov3 0.858 0.698 0.486 0.186 wlb1 0.693 0.883 0.617 0.129 wlb2 0.593 0.883 0.558 0.135 wlb3 0.585 0.884 0.590 0.123 famp1 0.420 0.591 0.881 0.111 famp2 0.438 0.592 0.914 0.114 famp3 0.463 0.555 0.816 0.100 bi1 0.236 0.135 0.106 0.895 bi2 0.186 0.068 0.094 0.886 bi3 0.214 0.182 0.132 0.863 table 2. item loadings and cross-loadings construct cronbach's alpha composite reliability (cr) average variance extracted (ave) fcov 0.822 0.893 0.736 wlb 0.859 0.914 0.780 famp 0.840 0.904 0.759 bi 0.858 0.912 0.777 table 3. internal consistency reliability and convergent validity fcov wlb famp bi fcov 0.858 wlb 0.706 0.883 famp 0.504 0.665 0.871 bi 0.242 0.146 0.125 0.881 note. square roots of ave are given on the diagonal, in bold. table 4. discriminant validity: inter-construct correlations and square roots of ave fcov wlb famp bi fcov wlb 0.850 famp 0.615 0.784 bi 0.283 0.169 0.147 table 5. discriminant validity: htmt ratios of correlations australasian journal of information systems tretiakov, bensemann & jurado 2023, vol 27, research article women’s intentions to return to it employment 11 5.2 structural model the results of structural model testing are summarised in table 6. for all independent variables variance inflation factor (vif) values were below the threshold of 5, suggesting that there were no collinearity issues (hair et al., 2014; hair et al., 2019). of the three hypotheses, only h1 was confirmed, suggesting that friendly co-workers is an important outcome expectation women consider when returning to the it industry after a break. additionally, one of the control variables, auq, had a statistically significant effect, suggesting that in australia women who have left the it industry were more likely to consider returning to work in comparison to new zealand and the us. the r square value for the only dependent variable in the model (behavioural intention to return to it employment) was estimated as 0.098, which is relatively low (hair et al., 2014; hair et al., 2019). this suggests that to fully explain variation in the intention of women to re-join the it workforce further factors, in addition to social outcomes, need to be included. path hypothesis path coefficient t statistic 2.5% ci 97.5% ci p value vif fcov→bi* h1 0.267 2.500 0.051 0.456 0.013 2.158 wlb→bi h2 -0.060 -0.508 -0.289 0.179 1.389 2.778 famp→bi h3 0.035 0.329 -0.168 0.267 0.742 1.840 age→bi 0.082 1.096 -0.082 0.215 0.274 1.088 auq→bi* 0.178 2.063 0.012 0.351 0.039 1.402 usq→bi 0.039 0.487 -0.115 0.192 0.627 1.383 table 6. the results of structural model testing 5.3 robustness check as is common in panel-based research (schonlau & toepoel, 2015), the data set contained responses with possible straightlining (i.e., respondents using the same response category for all questions in a survey section). following the recommendation by zhang and conrad (2014) and schonlau and toepoel (2015), we conducted the analysis with responses involving straightlining excluded. for this analysis, we classified responses using the same category to answer all items measuring the independent variables, fcov, wlb, and famp as responses involving straightlining. there were 29 responses meeting this criterion. the results of the analysis of the remaining 153 responses were, in essence, identical to the analysis of the full data set: all reliability and validity criteria were met, there were no collinearity issues, and the statistically significant effects were the ones by fcol and auq. with straightlining reduced, there was stronger discrimination between constructs (the highest htmt ratio, the one between wlb and fcov, was reduced from 0.850 to 0.788). furthermore, in the analysis with straightliners removed the path coefficients for statistically significant relationships had higher values (e.g., 0.292 for fcov→bi and 0.246 for auq -> bi) and p values were smaller (both were below 0.01, thus meeting a stronger criterion for statistical significance). the implications for the hypotheses of the present study remained the same, suggesting that the overall approach was robust. australasian journal of information systems tretiakov, bensemann & jurado 2023, vol 27, research article women’s intentions to return to it employment 12 6 discussion 6.1 theoretical contributions the present study is, to the best of our knowledge, the first quantitative study of potential women returners to the it industry (women who have left it employment and are considering returning). the study contributes to the literature by testing the effects of women's perceived social outcome expectations of returning to it employment on their intentions to return. of the three factors considered, friendly co-workers, work-life balance, and family proximity, only the expectation of friendly co-workers had a statistically significant effect on the intention to return to it employment. thus, the results of the present study suggest that the expectation to encounter friendly co-workers, who could both make it easier for women returners to reintegrate into the work environment and allow them to derive life satisfaction from social exchanges, is particularly important in shaping their intention to return. a recent study of women returners to the it industry (those who already returned) by ravindran and baral (2014) found that an organisational climate that accepts diversity – a construct that, in relation to women workers, is similar in content to the friendly co-workers construct of the present study – had a statistically significant effect on job satisfaction, career satisfaction, and career commitment – constructs that are congruent in content to the intention to work in it. thus, the results of the present study are consistent with the results by ravindran and baral (2014). on the other hand, the effects of work-life balance have been demonstrated to be significant in related contexts (e.g., organisational commitment by accountants (berk & gundogmus, 2018), trust in the organisation by software workers (scholarios & marks, 2004)). a possible explanation of no statistically significant effect of expectations of work-life balance found in the present study is that the expectations did not vary enough across the participants to make a difference. thus, one should not interpret the results as suggesting that work-life balance does not matter. similarly, family proximity has been found to have an effect on workforce participation by women with children (compton & pollak, 2014), and one should be cautious about interpreting the results of the present study as suggesting that family proximity does not matter. nonetheless, the results are consistent with a view that, when assessing the intention of potential women returners to re-join the it industry, the expectation of friendly co-workers is more of a differentiator than expectations of work-life balance or family proximity. the content of the friendly co-worker concept, as introduced by venkatesh et al. (2017), is rather broad and covers both friendliness in a personal sense (as individuals) and friendliness in a professional sense (being collegial). both in the study by venkatesh et al. (2017) and in our study, statistical analysis has confirmed the discriminant and the convergent validity of the construct, suggesting that in the context of the it industry the perceptions of friendliness in a personal sense go hand in hand with the perceptions of friendliness in a professional sense. this may be attributed to the creative and professional character of it work, and thus may not automatically apply to all work contexts. a fundamental distinction between friendly co-workers and work-life balance and family proximity is the extent to which these variables can be controlled by legislative or management interventions. organisations can improve work-life balance for returning female it employees by taking unambiguous direct actions, such as offering flexible work arrangements australasian journal of information systems tretiakov, bensemann & jurado 2023, vol 27, research article women’s intentions to return to it employment 13 (eurofound, 2017) or, to some extent, by following agile project management practices, which is a strong trend at present (meso & jain, 2006). furthermore, regulatory bodies can establish regulations compelling organisations to implement policies that encourage work-life balance. to a considerable extent, this also applies to family proximity: employers can rely more on remote work (topp et al., 2022) or work from home (ozkaya, 2021). indeed, unlike in other industries, such as manufacturing or hospitality, most of the jobs in it can be done remotely (ozkaya, 2021). indeed, the shift to remote work associated with the covid-19 pandemic resulted in considerable organisational learning in organising and managing remote work, and in greater acceptance of such work (dubey & tripathi, 2020). in contrast, the concept of friendly co-workers is akin to organisational culture (heras et al., 2021). management cannot order people to be friendly, and regulatory bodies cannot legislate friendliness. while managers can take actions, such as modelling friendly behaviour, it is far from certain that such actions will have the desired effect (aitken, 2007). indeed, attempts to legislate friendliness may have the opposite effect, making the fact of women returning to work more salient. some co-workers may be uneasy about women returning to the workforce because of the perception that they could be unreliable in project work. specifically, workers may believe women returners will prioritise families over work responsibilities (rajesh et al., 2013). these feelings may be reinforced by patriarchal beliefs that women with children should stay at home, which may be relevant in some national cultures (wijayawardena et al., 2017). these feelings can be based on a viewpoint that women returners took an opportunity to have a career and a family life that other workers did not (beauregard, 2014), or for other reasons. if women's return to work is more salient, these co-workers are more likely to take actions, or inactions, that are not friendly (harris & ogbonna, 1998). similarly, managerial or regulatory actions designed to improve work-life balance or to improve family proximity may have a negative effect on friendliness if co-workers form a perception that women returners get special treatment that results in shifting workloads or risks (rajesh et al., 2013). a cultural change resulting in women returners being welcome by friendly co-workers requires a change in the organic, tacit work ecosystem (subramony et al., 2018), rather than a deliberate managerial or regulatory design. thus, the finding of the present study that friendly co-workers is an important factor in determining women's intentions to return to it work highlights the challenge of achieving a greater gender balance in the it workforce. the main implication for theory is that research on achieving greater gender balance in the it industry should focus more on peer-to-peer relational dynamics, on organisational (and industry) culture, and on the it work ecosystem. 6.2 practical implications the present study contributes to practice by highlighting the need for managers to promote a work environment that women would perceive as friendly, which is an aspect that is not always emphasized in the current practitioner literature. for example, hewlett (2008) lists a number of recommendations, such as creating reduced-hour jobs and nurturing ambition, but stops short of recommending measures intended to create a friendly environment. even though behaviours associated with the brogrammer and the geek archetypes may be based on deeply held beliefs and thus may be difficult to change, managers may reduce their impact in the workplace by not promoting competition and not encouraging work overtime. indeed, agile approaches to it project management, widely believed to represent the best practice in organising it work, emphasize collaboration in a team, rather than competition between australasian journal of information systems tretiakov, bensemann & jurado 2023, vol 27, research article women’s intentions to return to it employment 14 individuals, and suggest timeboxing work with no overtime. thus, even though a deep cultural change (as, for example, suggested by coe et al. (2019)) may be difficult, to an extent, a workplace more friendly to women employees could be created by implementing agile approaches to organizing it work, for as long as agile implementations are true to the agile principles of relying on “self-organizing teams” (rather than on competing individuals) and on maintaining a “a constant pace indefinitely” (rather on binge-working to meet deadlines) (https://agilemanifesto.org/principles.html). at the micro level, managers could also discourage microaggressions by modelling discourse friendly to women and by challenging aggressors. this could involve openly acknowledging the expertise of women (both technical and soft skills), ensuring that women have an opportunity and feel comfortable to contribute their perspectives in formal and informal decision-making situations, and addressing other types of microaggressions described in the literature (kim & meister, 2022). the results of the present study suggest that creating an environment in which colleagues are friendly and collegial to women is important to encourage women to return to it work after career breaks. however, friendly colleagues are not the only factor, and measures such as creating reduced hour jobs and providing flexibility to make it easier to combine work with childcare, removing the stigma of having a career interruption, and nurturing ambition (hewlett, 2008) are also important. however, the importance of expectations of friendly coworkers in forming the intention of potential women returners to re-join the it industry suggests that in implementing measures intended to correct the gender imbalance managers and regulatory bodies should avoid creating perceptions that the interests of women returners conflict with other employees, male or female. any changes brought in to attract women returners should be seen as an improvement for everyone: tensions between equity and equality (bierhoff & rohmann, 2011) should be managed to ensure that the organisation is perceived as just (fuchs & edwards, 2012). furthermore, even though potential women returners are well informed in forming their expectations around the outcomes of re-joining the it industry – because they have worked in it in the past – messages that they receive from the media make some of the memories and perceptions more salient than others (kligler-vilenchik et al., 2014). thus, media messages depicting unfair or unfriendly treatment of women employed in it, even though possibly intended to prompt managerial or regulatory actions to improve women's conditions at work, may also make their return less likely. from the perspective of improving the gender balance in the industry and assuring greater diversity, for as long as such messages reflect reality, depictions of the it workplace as friendly to women are more desirable. 6.3 limitations and further research as a study with a relatively small sample and a specific focus (women's perceptions of social outcomes of returning to the it workforce), the present study is far from comprehensive. to achieve a better explanation/prediction of women's intentions (and ultimately, actual behaviour) with respect to returning to the it workforce additional research is required. specifically, larger quantitative studies covering a broader range of factors are desirable, possibly covering aspects such as organisational policy and organisational justice (ravindran & baral, 2014), or opportunities for professional development (annabi & lebovitz, 2018). longitudinal quantitative studies measuring actual behaviour (whether women who intended to return to it employment actually returned) are also desirable. further, the mechanisms australasian journal of information systems tretiakov, bensemann & jurado 2023, vol 27, research article women’s intentions to return to it employment 15 behind the effects discovered in the present study could be understood in greater depth by conducting qualitative research. actual social relationships (women’s participation in various social networks, such as social networks involving people currently employed in it) may influence social outcome expectations of returning to it employment and may influence women’s intention to return to it employment by establishing subjective norms (manning, 2009) favourable or unfavourable towards returning to it employment. the role of social relationships could be clarified in further research. the incidental finding of the present study that potential women returners to the it industry in australia may be more likely to form a behavioural intention to re-join the industry than women in new zealand or in the us suggests the it work ecosystems in different countries may differ in their attractiveness for potential women returners. australia may be offering a superior environment for women returners, which other countries may wish to replicate. purposeful and comprehensive country comparisons are needed to provide sufficient evidence for practical steps in this respect. the present study was funded to focus on new zealand, australia, and us. future studies could provide more insightful cross-country comparisons by including countries with different cultures and countries that are particularly successful in promoting women participation in the it industry, such as india (raghuram et al., 2017; ring, 2018 ; sondhi et al., 2018). 7 conclusions we conducted a survey to study the effects of social outcome expectations with respect to intention to return to it employment for potential women returners from new zealand, australia, and the united states. expectations of friendly co-workers, work-life balance, and family proximity were included. the expectation of friendly co-workers had a statistically significant effect on the intentions of potential women returners to return to it employment. the results highlight the difficulty of creating an environment that encourages potential women returners to return to it because, unlike work-life balance or family proximity, friendly co-workers is a factor that is difficult to control via managerial interventions. for practice, the results suggest that organisations and industry bodies should promote an environment friendly to women, which in part may be achievable by implementing agile approaches to organizing it work. references ahuja, m. k. (2002). women in the information technology profession: a literature review, synthesis and research agenda. european journal of information systems, 11(1), 20-34. https://doi.org/http://dx.doi.org/10.1057/palgrave.ejis.3000417 aishwarya, m. r., & mohana, p. (2018). a study on career inhibitors and career enablers for women working in it sector. mim international journal of management research, 4(1), 2534. aitken, p. (2007). ‘walking the talk’: the nature and role of leadership culture within organisation culture/s. journal of general management, 32(4), 17-37. https://doi.org/http://dx.doi.org/10.1177/030630700703200402 australasian journal of information systems tretiakov, bensemann & jurado 2023, vol 27, research article women’s intentions to return to it employment 16 ajzen, i. (1991). the theory of planned behavior. organizational behavior and human decision processes, 50(2), 179-211. ajzen, i. (2002). perceived behavioral control, self-efficacy, locus of control, and the theory of planned behavior. journal of applied social psychology, 32(4), 665-683. https://doi.org/http://dx.doi.org/10.1111/j.1559-1816.2002.tb00236.x alok, s., banerjee, s., & kumar, n. (2021). will she stay or will she quit: determinants of career persistence and non-persistence amongst women workers of india's it sector. south asian journal of business studies, ahead-of-print, 1-19. https://doi.org/http://dx.doi.org/10.1108/sajbs-08-2020-0276 annabi, h., & lebovitz, s. (2018). improving the retention of women in the it workforce: an investigation of gender diversity interventions in the usa. information systems journal, 28(6), 1049-1081. https://doi.org/http://dx.doi.org/10.1111/isj.12182 armstrong, d. j., riemenschneider, c. k., allen, m. w., & reid, m. f. (2007). advancement, voluntary turnover and women in it: a cognitive study of work–family conflict. information & management, 44(2), 142-153. https://doi.org/http://dx.doi.org/10.1016/j.im.2006.11.005 armstrong, d. j., riemenschneider, c. k., & giddens, l. g. (2018). the advancement and persistence of women in the information technology profession: an extension of ahuja's gendered theory of it career stages. information systems journal, 28(6), 1082-1124. https://doi.org/http://dx.doi.org/10.1111/isj.12185 baroudi, j. j., & igbaria, m. (1994). an examination of gender effects on career success of information systems employees. journal of management information systems, 11(3), 181201. https://doi.org/http://dx.doi.org/10.1080/07421222.1994.11518055 barratt, b. (2018, november 30). the initiatives helping women return to work. forbes. https://www.forbes.com/sites/biancabarratt/2018/11/30/the-initiatives-helping-womenreturn-to-work/?sh=270f1a6d1049 beauregard, t. a. (2014). fairness perceptions of work− life balance initiatives: effects on counterproductive work behaviour. british journal of management, 25(4), 772-789. https://doi.org/http://dx.doi.org/10.1111/1467-8551.12052 beauregard, t. a., & henry, l. c. (2009). making the link between work-life balance practices and organizational performance. human resource management review, 19(1), 9-22. https://doi.org/http://dx.doi.org/10.1016/j.hrmr.2008.09.001 berk, c., & gundogmus, f. (2018). the effect of work-life balance on organizational commitment of accountants. management, 13(2), 137-159. https://doi.org/http://dx.doi.org/10.26493/1854-4231.13.137-159 bierhoff, h.-w., & rohmann, e. (2011). justice in performance situations: compromise between equity and equality. in e. kals & j. maes (eds.), justice and conflicts (pp. 135152). springer publishing, cham, switzerland. bresnahan, t., & gambardella, a. (2004). building high-tech clusters: silicon valley and beyond. cambridge university press, cambridge, uk. australasian journal of information systems tretiakov, bensemann & jurado 2023, vol 27, research article women’s intentions to return to it employment 17 carayon, p., schoepke, j., hoonakker, p., haims, m. c., & brunette, m. (2006). evaluating causes and consequences of turnover intention among it workers: the development of a questionnaire survey. behaviour & information technology, 25(5), 381-397. https://doi.org/http://dx.doi.org/10.1080/01449290500102144 castaño collado, c., & webster, j. (2011). understanding women’s presence in ict: the life course perspective. international journal of gender, science and technology, 3(2), 364-386. coe, i. r., wiley, r., & bekker, l.-g. (2019). organisational best practices towards gender equality in science and medicine. the lancet, 393(10171), 587-593. https://doi.org/10.1016/s0140-6736(18)33188-x coletti, m. (2010). technology and industrial clusters: how different are they to manage? science and public policy, 37(9), 679-688. https://doi.org/http://dx.doi.org/10.3152/030234210x12778118264413 compton, j., & pollak, r. a. (2014). family proximity, childcare, and women’s labor force attachment. journal of urban economics, 79, 72-90. https://doi.org/http://dx.doi.org/10.1016/j.jue.2013.03.007 crump, b., & logan, k. (2000). women in an alien environment. new zealand journal of applied computing and information technology, 4(1), 28-35. d'mello, m. (2006). gendered selves and identities of information technology professionals in global software organizations in india. information technology for development, 12(2), 131158. https://doi.org/http://dx.doi.org/10.1002/itdj.20031 davis, f. d. (1989). perceived usefulness, perceived ease of use, and user acceptance of information technology. mis quarterly, 13(3), 319-340. https://doi.org/http://dx.doi.org/10.2307/249008 denning, s. (2018). the emergence of agile people management. strategy & leadership, 46(4), 310. https://doi.org/http://dx.doi.org/10.1108/sl-04-2018-0042 dubey, a. d., & tripathi, s. (2020). analysing the sentiments towards work-from-home experience during covid-19 pandemic. journal of innovation management, 8(1), 13-19. https://doi.org/http://dx.doi.org/10.24840/2183-0606_008.001_0003 eurofound. (2017). work–life balance and flexible working arrangements in the european union. accessed on may 23, 2023, at https://www.eurofound.europa.eu/publications/customised-report/2017/work-lifebalance-and-flexible-working-arrangements-in-the-european-union fornell, c., & larcker, d. f. (1981). evaluating structural equation models with unobservable variables and measurement error. journal of marketing research, 18(1), 39-50. https://doi.org/http://dx.doi.org/10.2307/3151312 fuchs, s., & edwards, m. r. (2012). predicting pro-change behaviour: the role of perceived organisational justice and organisational identification. human resource management journal, 22(1), 39-59. https://doi.org/http://dx.doi.org/10.1111/j.1748-8583.2011.00167.x fulmer, i. s., & li, j. (2022). compensation, benefits, and total rewards: a bird's-eye (re) view. annual review of organizational psychology and organizational behavior, 9, 147-169. https://doi.org/http://dx.doi.org/10.1146/annurev-orgpsych-012420-055903 australasian journal of information systems tretiakov, bensemann & jurado 2023, vol 27, research article women’s intentions to return to it employment 18 gupta, n. (2015). rethinking the relationship between gender and technology: a study of the indian example. work, employment and society, 29(4), 661-672. https://doi.org/10.1177/0950017014556410 hair, j. f., babin, b. j., anderson, r. e., & black, w. c. (2014). multivariate data analysis (7th ed.). prentice hall, hoboken, nj, usa. hair, j. f., risher, j. j., sarstedt, m., & ringle, c. m. (2019). when to use and how to report the results of pls-sem. european business review, 31(1), 2-24. https://doi.org/http://dx.doi.org/10.1108/ebr-11-2018-0203 happe, l., & buhnova, b. (2022). frustrations steering women away from software engineering. ieee software, 39(4), 63-69. https://doi.org/http://dx.doi.org/10.1109/ms.2021.3099077 happe, l., buhnova, b., koziolek, a., & wagner, i. (2021). effective measures to foster girls’ interest in secondary computer science education. education and information technologies, 26(3), 2811-2829. https://doi.org/http://dx.doi.org/10.1007/s10639-020-10379-x harris, l. c., & ogbonna, e. (1998). employee responses to culture change efforts. human resource management journal, 8(2), 78-92. https://doi.org/http://dx.doi.org/10.1111/j.1748-8583.1998.tb00168.x henseler, j., ringle, c. m., & sarstedt, m. (2015). a new criterion for assessing discriminant validity in variance-based structural equation modeling. journal of the academy of marketing science, 43(1), 115-135. https://doi.org/http://dx.doi.org/10.1007/s11747-0140403-8 heras, m. l., rofcanin, y., escribano, p. i., kim, s., & mayer, m. c. (2021). family-supportive organisational culture, work–family balance satisfaction and government effectiveness: evidence from four countries. human resource management journal, 31(2), 454-475. https://doi.org/http://dx.doi.org/10.1111/1748-8583.12317 herman, c. (2015a). rebooting and rerouting: women's articulations of frayed careers in science, engineering and technology professions. gender, work & organization, 22(4), 324338. https://doi.org/http://dx.doi.org/10.1111/gwao.12088 herman, c. (2015b). returning to stem: gendered factors affecting employability for mature women students. journal of education and work, 28(6), 571-591. https://doi.org/http://dx.doi.org/10.1080/13639080.2014.887198 hewlett, s. a. (2008). off-ramps and on-ramps: keeping talented women on the road to success. human resource management international digest, 16(2),43-54. https://doi.org/10.1108/hrmid.2008.04416bae.003 hoda, r., salleh, n., & grundy, j. (2018). the rise and evolution of agile software development. ieee software, 35(5), 58-63. https://doi.org/http://dx.doi.org/10.1109/ms.2018.290111318 holth, l., bergman, a., & mackenzie, r. (2017). gender, availability and dual emancipation in the swedish ict sector. work, employment and society, 31(2), 230-247. https://doi.org/http://dx.doi.org/10.1177/0950017016651378 kim, j. y., & meister, a. (2022). microaggressions, interrupted: the experience and effects of gender microaggressions for women in stem. journal of business ethics, 1-19. australasian journal of information systems tretiakov, bensemann & jurado 2023, vol 27, research article women’s intentions to return to it employment 19 https://doi.org/10.1007/s10551-022-05203-0 kirton, g., & robertson, m. (2018). sustaining and advancing it careers: women’s experiences in a uk-based it company. the journal of strategic information systems, 27(2), 157-169. https://doi.org/http://dx.doi.org/10.1016/j.jsis.2018.01.001 kligler-vilenchik, n., tsfati, y., & meyers, o. (2014). setting the collective memory agenda: examining mainstream media influence on individuals’ perceptions of the past. memory studies, 7(4), 484-499. https://doi.org/http://dx.doi.org/10.1177/1750698014523443 lawler, e. e. (2011). creating a new employment deal: total rewards and the new workforce. organizational dynamics, 40(4), 302-309. https://doi.org/http://dx.doi.org/10.1016/j.orgdyn.2011.07.007 legault, m.-j., & chasserio, s. (2012). professionalization, risk transfer, and the effect on gender gap in project management. international journal of project management, 30(6), 697707. https://doi.org/http://dx.doi.org/10.1016/j.ijproman.2011.11.004 li, x. (2021). strategic flexibility in a male-dominated occupation: women software engineers in china. journal of gender studies, 32(4), 330-342. https://doi.org/10.1080/09589236.2021.2006615 mainiero, l. a., & sullivan, s. e. (2005). kaleidoscope careers: an alternate explanation for the "opt-out" revolution. the academy of management executive, 19(1), 106-123. https://doi.org/http://dx.doi.org/10.5465/ame.2005.15841962 manning, m. (2009). the effects of subjective norms on behaviour in the theory of planned behaviour: a meta-analysis. british journal of social psychology, 48(4), 649-705. meso, p., & jain, r. (2006). agile software development: adaptive systems principles and best practices. information systems management, 23(3), 19-30. https://doi.org/http://dx.doi.org/10.1201/1078.10580530/46108.23.3.20060601/93704.3 milligan, c., margaryan, a., & littlejohn, a. (2013). learning at transition for new and experienced staff. journal of workplace learning, 25 (4), 217-230. https://doi.org/http://dx.doi.org/10.1108/13665621311316410 molnar, a., keane, t., & stockdale, r. (2021). educational interventions and female enrollment in it degrees. communications of the acm, 64(3), 73-77. https://doi.org/http://dx.doi.org/10.1145/3387106 morganson, v. j., major, d. a., oborn, k. l., verive, j. m., & heelan, m. p. (2010). comparing telework locations and traditional work arrangements: differences in work-life balance support, job satisfaction, and inclusion. journal of managerial psychology, 25(6), 578-595. https://doi.org/http://dx.doi.org/10.1108/02683941011056941 morris, e. a., & zhou, y. (2018). are long commutes short on benefits? commute duration and various manifestations of well-being. travel behaviour and society, 11, 101-110. https://doi.org/http://dx.doi.org/10.1016/j.tbs.2018.02.001 myers, b., thorn, k., & doherty, n. (2021). self-initiated expatriation and older women: composing a further life. personnel review, 51(3), 1120-1137. https://doi.org/http://dx.doi.org/10.1108/pr-11-2019-0638 australasian journal of information systems tretiakov, bensemann & jurado 2023, vol 27, research article women’s intentions to return to it employment 20 niederle, m., & vesterlund, l. (2008). gender differences in competition. negotiation journal, 24(4), 447-463. ozkaya, i. (2021). the future of software engineering work. ieee software, 38(5), 3-6. https://doi.org/http://dx.doi.org/10.1109/ms.2021.3089729 panteli, a., stack, j., atkinson, m., & ramsay, h. (1999). the status of women in the uk it industry: an empirical study. european journal of information systems, 8(3), 170-182. https://doi.org/http://dx.doi.org/10.1057/palgrave.ejis.3000326 panteli, n. (2006). returning to it: employment and development after a career break in the united kingdom. labour & industry: a journal of the social and economic relations of work, 16(3), 133-150. https://doi.org/http://dx.doi.org/10.1080/10301763.2006.10669334 panteli, n. (2012). a community of practice view of intervention programmes: the case of women returning to it. information systems journal, 22(5), 391-405. https://doi.org/http://dx.doi.org/10.1111/j.1365-2575.2012.00415.x panteli, n., & pen, s. (2009). empowering women returners in the uk high-tech industry. personnel review, 39(1), 44-61. https://doi.org/10.1108/00483481011007850 raghuram, p., herman, c., ruiz-ben, e., & sondhi, g. (2017). women and it scorecard-india. nasscom & the open university. doi: 10.13140/rg.2.2.10118.98882 rajesh, s., founder-president, a. c. c., gruhaa, u., & neelangarai, c. (2013). second career of women professionals in india: a corporate perspective. asian journal of management research, 4(1), 27-47. ravindran, b., & baral, r. (2014). factors affecting the work attitudes of indian re-entry women in the it sector. vikalpa, 39(2), 31-42. https://doi.org/http://dx.doi.org/10.1177/0256090920140205 ring, k. (2018 ). women in tech: india leads the way. 451 research. https://go.451research.com/women-in-tech-india-employment-trends.html sardeshmukh, s. r., & srinivasan, v. (2014). ict and work–family balance: context of indian software services. labour & industry: a journal of the social and economic relations of work, 24(1), 40-54. https://doi.org/http://dx.doi.org/10.1080/10301763.2013.877119 scholarios, d., & marks, a. (2004). work-life balance and the software worker. human resource management journal, 14(2), 54-74. https://doi.org/http://dx.doi.org/10.1111/j.1748-8583.2004.tb00119.x schonlau, m., & toepoel, v. (2015). straightlining in web survey panels over time. survey research methods, 9(2), 125-137. https://doi.org/http://dx.doi.org/10.18148/srm/2015.v9i2.6128 serenko, a., & turel, o. (2021). why are women underrepresented in the american it industry? the role of explicit and implicit gender identities. journal of the association for information systems, 22(1), 41-65. https://doi.org/http://dx.doi.org/10.17705/1jais.00653 shah, d., & barker, m. (2020). work-life interface: experiences of indian it women repatriates. the international journal of human resource management, 33(3), 505-538. https://doi.org/http://dx.doi.org/10.1080/09585192.2020.1737173 australasian journal of information systems tretiakov, bensemann & jurado 2023, vol 27, research article women’s intentions to return to it employment 21 singh, s., & vanka, s. (2021). career break, not a brake on career: a study of the reasons and enablers of women’s re-entry to technology careers in india. business perspectives and research, 9(2), 195-214. https://doi.org/http://dx.doi.org/10.1177/2278533720964328 sondhi, g., raghuram, p., herman, c., & ben, e. r. (2018). skilled migration and the it sector: a gendered analysis. in rajan, s. irudaya (eds). india migration report 2018: migrants in europe (pp. 229-248). routledge, new delhi, india. srinivasan, v., murty, l., & nakra, m. (2013). career persistence of women software professionals in india. gender in management: an international journal, 28(4), 210-227. https://doi.org/http://dx.doi.org/10.1108/gm-01-2013-0011 subramony, m., solnet, d., groth, m., yagil, d., hartley, n., kim, p. b., & golubovskaya, m. (2018). service work in 2050: toward a work ecosystems perspective. journal of service management, 29(5), 956-974 https://doi.org/http://dx.doi.org/10.1108/josm-05-2018-0131 tokbaeva, d., & achtenhagen, l. (2023). career resilience of female professionals in the maledominated it industry in sweden: toward a process perspective. gender, work & organization, 30(1), 223-262. https://doi.org/https://doi-org.ezproxy.massey.ac.nz/10.1111/gwao.12671 tomlinson, j., olsen, w., & purdam, k. (2009). women returners and potential returners: employment profiles and labour market opportunities—a case study of the united kingdom. european sociological review, 25(3), 349-363. https://doi.org/http://dx.doi.org/10.1093/esr/jcn053 topp, j., hille, j. h., neumann, m., & mötefindt, d. (2022). how a 4-day work week and remote work affect agile software development teams. in przybyłek, a., jarzębowicz, a., luković, i., ng, y.y. (eds), lean and agile software development. lecture notes in business information processing, vol 438. cham, switzerland, springer publishing company. https://doi.org/10.1007/978-3-030-94238-0_4 trauth, e. (2002). odd girl out: an individual differences perspective on women in the it profession. information technology & people, 15(2), 98-118. https://doi.org/http://dx.doi.org/10.1108/09593840210430552 trauth, e., & connolly, r. (2021). investigating the nature of change in factors affecting gender equity in the it sector: a longitudinal study of women in ireland. mis quarterly, 45(4), 2055-2100. https://doi.org/http://dx.doi.org/10.25300/misq/2022/15964 truman, g. e., & baroudi, j. j. (1994). gender differences in the information systems managerial ranks: an assessment of potential discriminatory practices. mis quarterly, 18(2), 129-142. https://doi.org/http://dx.doi.org/10.2307/249761 van der velde, m. e., jansen, p. g., bal, p. m., & van erp, k. j. (2017). dual-earner couples’ willingness to relocate abroad: the reciprocal influence of both partners’ career role salience and partner role salience. european journal of work and organizational psychology, 26(2), 195-207. https://doi.org/http://dx.doi.org/10.1080/1359432x.2016.1241768 venkatesh, v., morris, m. g., davis, g. b., & davis, f. d. (2003). user acceptance of information technology: toward a unified view. mis quarterly, 27(3), 425-478. https://doi.org/http://dx.doi.org/10.2307/30036540 australasian journal of information systems tretiakov, bensemann & jurado 2023, vol 27, research article women’s intentions to return to it employment 22 venkatesh, v., thong, j. y., & xu, x. (2012). consumer acceptance and use of information technology: extending the unified theory of acceptance and use of technology. mis quarterly, 36(1), 157-178. https://doi.org/https://doi.org/10.2307/41410412 venkatesh, v., windeler, j. b., bartol, k. m., & williamson, i. o. (2017). person–organization and person–job fit perceptions of new it employees: work outcomes and gender differences. mis quarterly, 41(2), 525-558. https://doi.org/http://dx.doi.org/10.25300/misq/2017/41.2.09 westaby, j. d. (2005). behavioral reasoning theory: identifying new linkages underlying intentions and behavior. organizational behavior and human decision processes, 98(2), 97120. https://doi.org/http://dx.doi.org/10.1016/j.obhdp.2005.07.003 wijayawardena, k., wijewardena, n., & samaratunge, r. (2017). compromising gender identities: stay strategies of women in gender-atypical information technology firms in sri lanka. information technology & people, 30(2), 246-264. https://doi.org/http://dx.doi.org/10.1108/itp-01-2016-0012 wilkins-yel, k. g., hyman, j., & zounlome, n. o. (2019). linking intersectional invisibility and hypervisibility to experiences of microaggressions among graduate women of color in stem. journal of vocational behavior, 113, 51-61. https://doi.org/10.1016/j.jvb.2018.10.018 wilson, j., fisher, d., & moore, k. (2009). reverse diaspora and the evolution of a cultural tradition: the case of the new zealand ‘overseas experience’. mobilities, 4(1), 159-175. https://doi.org/http://dx.doi.org/10.1080/17450100802658028 woodfield, r. (2002). woman and information systems development: not just a pretty (inter) face? information technology & people, 15(2), 119-138. https://doi.org/http://dx.doi.org/10.1108/09593840210430561 wu, t. (2020). the labour of fun: masculinities and the organisation of labour games in a modern workplace. new technology, work and employment, 35(3), 336-356. xie, b., & li, m. (2021). coworker guanxi and job performance: based on the mediating effect of interpersonal trust. technological forecasting and social change, 171, 120981. https://doi.org/http://dx.doi.org/10.1016/j.techfore.2021.120981 yang, k., & girgus, j. s. (2019). are women more likely than men are to care excessively about maintaining positive social relationships? a meta-analytic review of the gender difference in sociotropy. sex roles, 81(3), 157-172. https://doi.org/http://dx.doi.org/10.1007/s11199-018-0980-y yeow, p., dean, a., tucker, d., & pomeroy, l. (2019). group-works: exploring multiplex networks, leadership and group performance. journal of organizational effectiveness: people and performance, 6(4), 227-245. https://doi.org/http://dx.doi.org/10.1108/joepp-032019-0027 zhang, c., & conrad, f. (2014). speeding in web surveys: the tendency to answer very fast and its association with straightlining. survey research methods, 8(2), 127-135. https://doi.org/http://dx.doi.org/10.18148/srm/2014.v8i2.5453 australasian journal of information systems tretiakov, bensemann & jurado 2023, vol 27, research article women’s intentions to return to it employment 23 copyright: © 2023 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v27i0.4111 http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 background 3 hypotheses development 3.1 friendly co-workers 3.2 work-life balance 3.3 family proximity 4 method 4.1 sample and procedure 4.2 measurement 5 results 5.1 measurement model 5.2 structural model 5.3 robustness check 6 discussion 6.1 theoretical contributions 6.2 practical implications 6.3 limitations and further research 7 conclusions paper title (paper title style) australasian journal of information systems kathriarachchi, alam, weerasinghe & pauleen 2024, vol 28, research article risks of e-commerce recommender systems received 30 oct 2023, accepted 4 july 2024 after 2 rounds of review with karlheinz kautz as accepting editor. 1 risks of e-commerce recommender systems: a scoping review eranjana kathriarachchi school of management, massey university auckland, new zealand email: e.kathriarachchi@massey.ac.nz shafiq alam school of management, massey university auckland, new zealand kasuni weerasinghe management technology organisation department, auckland university of technology auckland, new zealand david pauleen graduate institute and department of business administration, national chung cheng university, chia-yi, taiwan abstract while recommender systems (rs) used in e-commerce have improved significantly providing customers with a personalised shopping experience, scholars have constantly raised concerns over the risks associated with e-commerce rs. however, a lack of methodological synthesis of risk-generating events associated with e-commerce recommender systems has curtailed systematic investigation of the risks of e-commerce rs. this paper presents a scoping review aimed at addressing this gap by synthesising different risk-generating events involved with the use of e-commerce rs as reported in the literature that could affect the welfare of customers who use those systems. accordingly, peer-reviewed research studies published from 2003-2023 were extracted from the scopus database and ebscohost platform for review. sixty-two publications with evidence on risk-generating events of e-commerce rs were considered for the review. twenty risk-generating events were identified through the review. these events were mapped with the corresponding risks based on existing frameworks on risks of e-commerce. we were able to identify several risk-generating events that had not previously been considered in conceptualising the risks of e-commerce rs. further, we identified the plurality of the outcomes of risk-generating events which could provide guidance for the evaluation of e-commerce recommender systems from a multistakeholder perspective. keywords: risk-generating events, e-commerce, recommender systems, scoping review. 1 introduction recommender systems (rs) predict user preferences using statistical, machine learning, and artificial intelligence (ai) techniques (jannach et al., 2022; necula & păvăloaia, 2023). rs are now being used in many areas such as e-commerce, entertainment, health, education, human resource australasian journal of information systems kathriarachchi, alam, weerasinghe & pauleen 2024, vol 28, research article risks of e-commerce recommender systems 2 management, online dating, and mobile apps (mallik & sahoo, 2020; pizzato et al., 2013) helping users to steer through complex and large volumes of information. among these various areas of application, the e-commerce sector has significantly benefited from personalised product recommendations provided by e-commerce recommender systems (hereinafter referred to as ecommerce rs). the global e-commerce sector is expanding rapidly with its revenue expected to grow up to us$ 8.1 trillion by the end of year 2027 from the 2022 level of us$ 5.8 trillion (chevalier, 2024). e-commerce rs deployed in e-commerce websites play an essential role in assisting customers to navigate through complex product-related information to identify the most suitable products for them (fabbri, 2022; murphy, 2011; vučetić & hudec, 2018; xiao & tan, 2012). the world’s leading e-commerce platforms such as amazon, ebay, alibaba, and flipkart are examples that show the significance of rs in the commercial sphere (jannach et al., 2022; kiswanto et al., 2018; kong et al., 2017; lee et al., 2012; ram mohan rao et al., 2018; smith & linden, 2017; yang et al., 2018). despite their contribution towards the success of the e-commerce sector, rs suffer from various issues that affect the welfare of the customers who rely on them. some scholars have looked at these issues from an ethical perspective (chen, 2022; milano et al., 2020; wang et al., 2023), while others have looked at those from a risk perspective (glover & benbasat, 2010; jannach & bauer, 2020). researchers who have taken the risk perspective have proposed several risk classifications based on different ways in which e-commerce rs negatively affect users. these classifications consist of risks such as inferior product decisions, negative user experiences and privacy issues (these will be discussed in detailed in the next section). although these risk classifications have guided researchers to examine the risks associated with e-commerce rs, we encountered two limitations with these classifications. the first limitation is that the risk classification (e.g., glover & benbasat, 2010) is not up-to date with the most recent risk-generating events (i.e. events that could cause harm to users) discussed in the academic literature. the second limitation is that the risk classification (jannach & bauer, 2020) was not developed based on a systematic study of different risk-generating events, related to the risks proposed. this lack of a comprehensive understanding of risk-generating events preempts a comprehensive understanding of risks associated with e-commerce rs. this was particularly relevant to our research team as the first author is engaged in a doctoral research project investigating stakeholder perceptions of risk in e-commerce rs. against this backdrop, we decided to conduct a study to synthesise risk-generating events associated with e-commerce rs as reported in the literature. grant and booth (2009) have identified 14 different types of literature reviews according to differences in intended objective and methodology followed in each type. for example, a critical review is aimed at critically evaluating the quality of literature whereas a meta-analysis is expected to statistically combine the results of quantitative studies to provide more precise effects of the results. according to them, a scoping review enables researchers to identify the size and scope of available research literature. further, scoping reviews are useful in examining emerging evidence that addresses and informs practice (munn et al., 2018; peters et al., 2015). due to these reasons, a scoping review was deemed suitable for our synthesis of literature. our scoping review addresses the following research questions: rq 1: what risk-generating events do customers encounter when they use e-commerce rs? australasian journal of information systems kathriarachchi, alam, weerasinghe & pauleen 2024, vol 28, research article risks of e-commerce recommender systems 3 rq 2: how are these risk-generating events potentially linked to the risks of e-commerce rs? these research questions are addressed by reviewing the literature published during 2003-2023. risk-generating events and their negative outcomes as reported in the literature were recorded, analysed, and presented to answer the first research question. in addressing the second research question, the risk classification proposed by jannach and bauer (2020) was utilized due to its specific focus on e-commerce rs. the process followed is further explained below in the methods section. the main contribution of this study is a review of evidence on risk-generating events with its potential links to risks of e-commerce rs. hence, this study is important for a diverse range of stakeholders who rely on e-commerce rs. for customers, knowledge about the events exposing them to potential harm will be important to either avoid or manage such situations for their wellbeing. e-commerce managers will be able to identify different occurrences which could negatively affect customers and the business and take required action to mitigate their impact. in addition, policy makers and governments can also gain a deeper understanding of different ways in which recommendation technology used in product recommendations could harm the welfare of customers and society in general. the next section of the paper presents a background to the study, followed by sections on rs in the information systems (is) context, the theoretical underpinning of the study, a summary of existing reviews on e-commerce rs, methods, results, and a discussion. 2 background e-commerce rs are defined as “a web-based technology that explicitly or implicitly collects a customer’s preferences and recommends tailored e-vendors’ products or services accordingly” (li & karahanna, 2015, p. 74). implicit preferences (e.g., customers’ online behaviour) and explicit preferences (e.g., customer reviews and ratings) function as the knowledge base to build user profiles (eryarsoy & piramuthu, 2014). these user profiles form the basis for providing personalised product recommendations to customers. advancements in rs technology have contributed significantly to the development of the ecommerce sector in the recent past with increased revenue for businesses and satisfying experiences for customers (chen, 2022; necula & păvăloaia, 2023). while some authors have commented on the positive contributions, some others have raised concerns over the harmful impact of e-commerce rs. as mentioned before these harmful effects have been discussed from ethics and risk perspective. for instance, milano et al. (2020) have presented a taxonomy of the ethical challenges of rs such as inappropriate content, privacy, autonomy and personal identity, opacity, fairness, and social effects. this taxonomy is based on whether an ethical issue negatively affects the utility or rights of stakeholders and whether it will constitute immediate harm or a future risk of harm or rights violation. they have further concluded that any aspect of a rs that could negatively affect any of its stakeholders, or risk imposing such negative impacts, constitutes a feature that is ethically relevant. a risk is defined as “the possibility a negative outcome will occur as a result of exposure to a hazard” (wilson et al., 2019, p. 780). similarly, researchers who have taken the risk perspective australasian journal of information systems kathriarachchi, alam, weerasinghe & pauleen 2024, vol 28, research article risks of e-commerce recommender systems 4 have looked at different risk-generating events (i.e. hazards) associated with the use of ecommerce rs which cause harm to their users. glover and benbasat (2010) have pioneered the use of events which cause harm to customers (which they have referred to as ‘unwanted events’) to conceptualise risks of e-commerce transactions. they have examined a total of 104 unwanted events (‘risk-generating events’ in this study) to study customers’ risk perceptions towards ecommerce transactions. among these 104 unwanted events are those associated with the rs deployed in e-commerce platforms. these events have been used to identify the cause of harm (formative measure), which have then been used to conceptualise the risk dimensions. an illustration of their conceptualisation is provided below. figure 1 conceptualisation of risk dimensions based on unwanted events (glover & benbasat, 2010) in this manner based on the unwanted events identified glover and benbasat (2010) have conceptualised three types of risks namely, ‘information misuse risk’, ‘failure to gain product benefit risk’, and ‘functionality inefficiency risk’, to construct a model of the perceived risks of ecommerce transactions. in a more recent study, jannach and bauer (2020) have proposed a classification of risks that are specific to rs deployed in e-commerce platforms. according to them, poorly designed ecommerce rs can lead to poor product decisions by customers and negative user experience. biased information can expand to community-related risks, and misuse of customer information can lead to privacy risks. the classification they have proposed consists of four types of risks namely: poor decision/choice dissatisfaction, bad user experience/decision difficulty, biased information state, and privacy. in essence, except for the risk of ‘biased information state’ proposed by them, the other three types of risks are similar to the risk classifications proposed by glover and benbasat (2010). a comparison of the two classifications is provided in table 1. types of risks (glover & benbasat, 2010) types of risks (jannach & bauer, 2020) explanation failure to gain product benefit risk poor decision/choice dissatisfaction bias introduced by a rs leading to an inferior product purchase decision functionality inefficiency risk bad user experience/decision difficulty poorly designed rs and unhelpful recommendations leading to poor user experience information misuse risk privacy unsolicited collection and use of confidential information table 1 comparison between risk classifications unwanted event the recommendation agent may give me biased advice cause of harm (formative measure) unmet needs: something i buy on the web might not fit my needs risk dimension failure to gain product benefit risk australasian journal of information systems kathriarachchi, alam, weerasinghe & pauleen 2024, vol 28, research article risks of e-commerce recommender systems 5 although these two risk classifications have provided crucial understanding of the risks unique to e-commerce rs, the limitations associated with them have instigated the need to conduct a comprehensive synthesis of evidence related to e-commerce rs. to elaborate, the risk classification proposed by glover and benbasat (2010) does not cover certain more recent riskgenerating events such as shilling attacks/profile injection attacks (laskar et al., 2023; xu et al., 2022), user decision biases (ahmed et al., 2022; gu et al., 2020), and algorithmic opacity (chen, 2022; eslami et al., 2019) which have received significant scholarly attention. shilling attacks/profile injection attacks on rs could lead to undesirable items being recommended to customers and in turn their loyalty towards the recommendation systems may decrease significantly (singh et al., 2022). researchers (ahmed et al., 2022; teppan & zanker, 2015) are of the opinion that decision biases associated with rs can lead to sub-optimal decision making among users and can be proactively exploited for persuading users. not only do these inferior decisions of customers pose potential harm to their own wellbeing but such decisions will also lead them to play a role in propagating systemic biases that can influence other customers (banker & khetani, 2019). further, customers’ lack of awareness of how algorithms work coupled with their overreliance on recommendation algorithms (banker & khetani, 2019) could lead to undesirable outcomes for customers due to biased and deceptive decisions supported by algorithmic opacity (eslami et al., 2019). on the other hand, the risk classification proposed by jannach and bauer (2020) was not developed based on a systematic study of different risk-generating events related to the risks proposed. this limits our understanding of specific instances that could lead to the types of risks identified in their classification. hence, we expect that this scoping review will help the academic community as well as the wider stakeholders (customers, managers, and policy makers) to determine the risk potential of current e-commerce rs. the next section of the paper presents an account of e-commerce rs research within the domain of is research. 2.1 recommender systems in the information systems context rs has been extensively studied within the information systems (is) context. this is mainly triggered by is scholars’ focus on user perspectives and the interplay between computerized systems and users (jannach et al., 2012). these scholars have looked at many topics of interest to both the academic community and practitioners interested in rs (see table 2 for a summary). biases in product recommendations, privacy issues, user engagement and satisfaction are some of the key concerns explored by is scholars. this scoping review will further assist is scholars to understand the nature of different events and their potential links with specific risks of ecommerce rs. further the present study synthesises risk-generating events reported in multiple disciplines such as marketing, computer science and decision support systems. this will be beneficial for is scholars to understand the inter-disciplinary nature of the research context. the next section of the paper highlights the theoretical underpinning of this study and prior reviews that have been conducted on e-commerce rs. australasian journal of information systems kathriarachchi, alam, weerasinghe & pauleen 2024, vol 28, research article risks of e-commerce recommender systems 6 studies focus/key findings of the study ebrahimi et al. (2022); hu et al. (2017) importance of quality in product recommendations ho et al. (2017); jiawei chen et al. (2023); silva et al. (2019) biases associated with human decision making in the context of ecommerce rs adomavicius et al. (2018) role of e-commerce rs in economic decision making. biases associated with product recommendations leading to customer dissatisfaction which in turn results in reduced willingness to pay jannach and jugovac (2019) e-commerce rs as a vital tool which fosters revenue/profitability, user engagement, loyalty and customer satisfaction lee and hosanagar (2019) collaborative filtering techniques lead to low sales diversity, biased product recommendations will lead to overall decrease in sales diversity awad and krishnan (2006); xin et al. (2023) privacy concerns associated with potential data leakage in ecommerce rs. table 2 summary of the rs research within the is context. 2.2 theoretical underpinning and prior reviews the model of perceived risk of e-commerce transactions (glover & benbasat, 2010) and the psychometric paradigm (slovic, 1987) provides the theoretical underpinning for this scoping review. these two theoretical frameworks emphasise the significance of examining harmful events in conceptualising risks. for instance, glover and benbasat (2010) have opined that identifying the negative outcomes of unwanted/harmful events (similar to risk-generating events in this study) to customers is the best approach to conceptualise the risks associated with it-based applications rather than looking at the ‘source of risk’ (the environment, object, or actor) or ‘type of harm’ (financial loss, time loss, psychological harm). in addition, the psychometric paradigm also prescribes examining harmful events (i.e. hazards) in exploring risk perceptions. the psychometric paradigm has been applied by researchers to elicit judgements on diverse hazard scenarios within a single technological domain where users’ risk perceptions estimated based on their evaluations of riskiness in hazards scenarios (slovic, 1987). we were able to identify several prior reviews on e-commerce rs which have looked at diverse areas of interest. recent studies have examined different negative effects of e-commerce rs. for instance, researchers (alamdari et al., 2020; nasir & ezeife, 2023; stalidis et al., 2023) have consistently shown the negative effect of a lack of diversity in search results and the inability to recognize changing customer preferences on the user experience. increased privacy concerns among the public due to large volumes of personal information collected by rs to provide better recommendations has also attracted the attention of some scholars (alamdari et al., 2020). the present study also coincides with these recent reviews which look at the negative effects of rs with a specific focus on e-commerce rs. a summary of the prior reviews on rs is presented in table 3. the next section of the review details the methods of the present study. australasian journal of information systems kathriarachchi, alam, weerasinghe & pauleen 2024, vol 28, research article risks of e-commerce recommender systems 7 study time period sample size analysis method key focus/findings karimov (2016) 2014-2016 60 a literature review of empirical studies traditional rs techniques which play a dominant role in e-commerce rs wei et al. (2007) not mentioned 29 a literature review of empirical studies there is less research on the application value of rs from other disciplines perspectives (i.e., management, marketing, and psychology) than computer science li and karahanna (2015) 1990-2013 41 a literature review of empirical studies rs literature is fragmented and lacks an overarching framework to guide research and integrate findings. more research is required on interaction between components of rs and the impact nasir and ezeife (2023) not mentioned 73 a literature review of empirical studies a classification and a taxonomy of sequential recommender systems stalidis et al. (2023) 2012-2023 296 a literature review of empirical and review studies recent trends in rs research namely: balancing accuracy with user expectations, contextual factors, trust, explainability, multiple sources of information available for users, and emergence of neural networks as a technique used in e-commerce rs xiao and benbasat (2007) 1990-2007 47 a literature review of empirical studies a conceptual model consisting of 28 propositions stemming from five theoretical perspectives xiao and benbasat (2014) 2006-2012 34 a literature review of empirical studies studies published between 2007 and 2012 have extended the authors’ conceptual model (xiao & benbasat, 2007). rs type, preference-elicitation, explanation, and the social impact of rs have been identified as key research areas deldjoo et al. (2023) 2014-2023 50 + a literature review of empirical studies critical challenges in fashion rs research and has proposed a taxonomy of objectives that the studies under review have attempted to achieve adolphs and winkelmann (2010) 2000-2008 42 a literature review of empirical studies a classification of literature on personalisation research in e-commerce under three major categories: implementation, theoretical foundations, and usercentric aspects alamdari et al. (2020) 2008-2019 33 a systematic literature review a comparison and an evaluation of different mechanisms used in e-commerce rs: most of the studies have focused on improving the accuracy of recommendations, and security, response time, and novelty, whereas serendipity has received scant attention table 3 summary of reviews on e-commerce rs 3 methods according to peters et al. (2015), scoping reviews are useful for synthesising existing literature and often are referred to as a ‘mapping’ exercise. the authors screened scientific literature with the objective of identifying publications that included a discussion on risk-generating events and risks customers face with rs deployed in e-commerce. the concepts were identified to cover this australasian journal of information systems kathriarachchi, alam, weerasinghe & pauleen 2024, vol 28, research article risks of e-commerce recommender systems 8 objective and to facilitate developing a comprehensive search query. the identified publications were then reviewed to map existing scholarly evidence of risk-generating events. the search was conducted through the scopus database and ebscohost platform. table 4 outlines the concepts and the respective search terms used. concept search terms recommender systems “recommend* system*” or “recommend* agent*” context and e-commerce or e-business or “electronic commerce” or “digital business” or “online business” or “online retailing” risks and risk* or concerns* or threats* or “privacy risk*” or “bias*” or “functional risk*” or “financial risk*” or “time risk*” or “psychological risk*” or “social risk*” or “product risk” users and customer* or user* or consumer* or client* table 4 concepts and search terms inclusion and exclusion criteria were applied in selecting the publications for this review. in line with the aim of this scoping review, only studies exploring/referring to risk-generating events and their outcomes of using e-commerce rs from customers’ perspectives were considered for this review. only peer-reviewed studies (journal articles, conference papers, and reviews) were included in this review due to their higher validity compared to other types of publications. studies conducted during 2003-2023 (see figure 2) were considered because most of the scholarly work in the field of e-commerce rs has been conducted during this time. only english language publications were considered for this review. figure 2: number of publications per year australasian journal of information systems kathriarachchi, alam, weerasinghe & pauleen 2024, vol 28, research article risks of e-commerce recommender systems 9 preferred reporting items for systematic reviews and meta analysis (prisma) guidelines (tricco et al., 2018) were followed in selecting publications for the review: the flow chart is shown in figure 3. the first and second authors were involved in the article screening process independently. the third author was consulted in managing any discrepancies that emerged during the process. separate search queries were developed independently by the first and second authors and later combined to reduce the biases in executing the search. a total of 417 records were identified in the initial search. then the search results were compared, and duplicates were figure 3: prisma flow diagram australasian journal of information systems kathriarachchi, alam, weerasinghe & pauleen 2024, vol 28, research article risks of e-commerce recommender systems 10 removed. next all publications were screened based on the title and the abstract. at this stage of screening, publications that were considered irrelevant based on the inclusion criteria were excluded. the next stage involved reading the selected articles (106 publications). forty-four articles were excluded after the full-text review for the following reasons: irrelevant domain (not e-commerce sector), irrelevant outcome (absence of risk-generating events or outcomes), or irrelevant study type. sixty-two articles were identified as suitable for this review (see table a1 in appendix for details on the selected publications). the data extracted comprised author details, the type of riskgenerating-event/s reported, and their negative outcomes. the third author reviewed a subsample of twenty papers to ensure the validity of the data characterization. a thematic content analysis was conducted on the data extracted, with the purpose of identifying common themes from each type of risk-generating event reported, resulting in risks of the ecommerce rs from customers’ perspectives, according to the inclusion criteria. as suggested by vailati riboni et al. (2020), categories were not predefined to avoid the risk of bias, and the classification was entirely conducted retrospectively. the step-by step approach for thematic analysis proposed by nowell et al. (2017) was used as a guide in deriving the themes. initially the first and third authors spent time familiarizing themselves with the data. the second step was to generate initial codes based on the risk-generating events and negative outcomes reported. after that, searching and reviewing of themes were conducted with researcher triangulation with the participation of first, second, and third authors. finally, four themes (hereinafter referred to as categories) were defined and named with the consensus of researchers based on the sources of the events. the next section of the review details the results obtained through the analysis of selected publications. 4 results studies that were considered for the review were labeled according to the risk-generating event/s reported. a total of twenty such events were identified through the review and were classified into four main categories namely, 1. biased recommendations 2. malicious activities 3. customer biases/actions 4. incompetent systems see table 5 for a summary of risk-generating events reported in the studies reviewed (see table a2 in appendix for a detailed list of risk-generating events, their negative outcomes, and authors). an analysis of the reviewed studies’ most frequent keywords (based on titles, keywords, and abstracts) and the use of keywords over the years is presented in figure a1 and a2 in appendix, respectively. in the next section, a detailed analysis of the publications reviewed highlighting the risk-generating events and their links to the risks of e-commerce rs are presented according to the four categories stated above. australasian journal of information systems kathriarachchi, alam, weerasinghe & pauleen 2024, vol 28, research article risks of e-commerce recommender systems 11 categories risk-generating events biased recommendations recommending popular products (popularity bias), less frequently purchased products, and/or ignoring unpopular/new/obscure products (long tail products) presenting selective/incomplete information or hiding product information promoting profitable products/un-profitable products use of biased/unbalanced user data to provide biased product recommendations to first time/lesser-known customers generating biased system ratings which influence subsequent customer preference ratings (anchoring effect) lacking the surprise element (i.e., serendipity) and/or diversity in recommendations presenting visually biased product recommendations and/or using biased marketing cues to promote products biased policies on exposing search results to customers (exposure bias) malicious activities shilling/ profile injection attacks (push and/or nuke attacks) by malicious users dishonest ratings by non-malicious users obtaining, tracking, storing, using, or divulging sensitive customer information in an unauthorized or undesired manner re-using product reviews (review plagiarism) for unwarranted purposes customer biases/actions customers complying with the choices of other customers (conformity/social influence bias) customers providing biased reviews on products are influenced by the order sequence in which existing online reviews are displayed to a new customer (sequential bias) customers’ decision biases (irregularities in human decision making) e.g., the tendency to make decisions under the influence of certain emotional states and/or interests customers contributing to data leakages customers’ tendency to favour a product simply because it is ranked high (position bias) or neighbouring/related items influencing click-through rate of a target item (neighbouring bias) customers’ lack of awareness of how algorithms make product recommendations incompetent systems incompetent recommender systems with functional issues (i.e., nonconsideration of complex user requirements, information overload, etc.) rs not capturing changing user preferences on products over time (user and item bias) table 5: summary of the risk-generating events associated with e-commerce rs 4.1 biased recommendations biases associated with rs have attracted significant scholarly attention from researchers. a summary of the risk-generating activities categorized under biased recommendations is presented in table 6. according to wang et al. (2018), biased rs recommend products that do not fit users’ indicated preferences. researchers over the years have explored many ways in which biases occur in rs. one such instance is the presence of popularity bias, where an e-commerce rs recommends popular products (chen, 2022; gu et al., 2020; kiswanto et al., 2018; lee et al., 2012; niu et al., 2019; sreepada & patra, 2021; wang et al., 2022, 2022; zeng et al., 2021). some authors australasian journal of information systems kathriarachchi, alam, weerasinghe & pauleen 2024, vol 28, research article risks of e-commerce recommender systems 12 have referred to this as the ‘concentration bias’ (deng et al., 2020; fleder & hosanagar, 2009). while popular products are recommended based on the purchase ratings, certain e-commerce sites engage in promoting either profitable products or un-profitable/less popular products (panniello et al., 2016; vučetić & hudec, 2018; wang et al., 2018; wang et al., 2018; xiao & benbasat, 2015, 2018) with the expectation of achieving high profitability or as an inventory management tactic (adomavicius et al., 2013; fabbri, 2022; xiao & benbasat, 2015). there have also been situations where companies recommend products that fit customers’ preferences, but the product lists are sorted to provide prominence to a product with the highest price or a newly released product. wang et al. (2018) have referred to this as ‘exposure bias’, where a predetermined policy biased towards the interest of the company is used in arranging the order of product recommendations. they have observed that leading e-commerce platforms such as amazon sometimes provide recommendations that are in favour of their sponsors regardless of their relevance to the customer due to the financial benefits, which results in customer exploitation. risk-generating events source/s recommending popular products (popularity bias), less frequently purchased products, and/or ignoring unpopular/new/obscure products (long tail products) deng et al., 2020; fleder & hosanagar, 2009; gu et al., 2020; kiswanto et al., 2018; lee et al., 2012; niu et al., 2019; sreepada & patra, 2021; vučetić & hudec, 2018; wang et al., 2022; zeng et al., 2021 presenting selective/incomplete information or hiding information kim et al., 2017; xiao et al., 2020 promoting profitable products/un-profitable products panniello et al., 2016; wang et al., 2018; xiao & benbasat, 2015, 2018 use of biased/unbalanced user data to provide biased product recommendations to firs time/lesser-known customers hu et al., 2017; silva et al., 2019; trakulwaranont et al., 2022 generating biased system ratings which influence subsequent customer preference ratings (anchoring effect) adomavicius et al., 2013; fabbri, 2022; xiao & benbasat, 2015 lacking the surprise element (i.e., serendipity) and/or diversity in recommendations fabbri, 2022; ge et al., 2020; grange et al., 2019 presenting visually biased product recommendations and/or using biased marketing cues to promote products qiu et al., 2021; wan et al., 2020 biased policies on exposing search results to customers (exposure bias) wang et al., 2022 table 6: summary of the risk-generating events (biased recommendations) the problem with continuously promoting popular products might be considered trivial since customers are already aware of popular items and they themselves can make the decision without the support of a recommender system (hu et al., 2017). however, researchers have pointed out that these biases towards popular/similar products or unpopular products have led to negative customer experiences, distrust (panniello et al., 2016; zeng et al., 2021), and standardization and homogenisation of customer choices (fabbri, 2022), foregoing better customer-product matches, potentially leading to balkanization/herding (fleder & hosanagar, 2009). this is because australasian journal of information systems kathriarachchi, alam, weerasinghe & pauleen 2024, vol 28, research article risks of e-commerce recommender systems 13 customers are looking for the products they really need rather than the higher-rated items (niu et al., 2019) or else they seek novelty, diversity, and serendipity (i.e., the ability to surprise and delight customers with relevant and novel recommendations) in lists of product recommendations (eryarsoy & piramuthu, 2014; fleder & hosanagar, 2009; ge et al., 2020; grange et al., 2019; kiswanto et al., 2018). in addition, some customers who lack awareness might rely on biased product recommendations assuming personalised recommendations have made their decision making easier (xiao & benbasat, 2018), but ultimately making poor product decisions. system-generated ratings are expected to indicate customers’ preference for a certain product that was recommended (adomavicius et al., 2013), and they are becoming indispensable for customers who are seeking product recommendations (eryarsoy & piramuthu, 2014). however, adomavicius et al. (2013) have identified that if the initial output of a recommender system is biased it can have an anchoring effect on the subsequent ratings provided to a product by a customer since customers’ preference ratings are malleable and can be significantly influenced by the recommendation they receive (necula & păvăloaia, 2023). further, when providing recommendations for a new customer, the rs faces the problem of a lack of data to provide personalised recommendations, which is referred to as the ‘cold-start’ problem. ahmed et al. (2022) and silva et al. (2019) have advocated that, although when faced with a cold start problem the conventional wisdom is to suggest popular products, that might not be wise as there are instances where unpopular products which are not prioritized by the rs are liked by customers who are new to the system. biases associated with the information provided by rs have also received attention from researchers. for example, kim et al. (2017) and xiao et al. (2020) have identified that e-commerce companies sometimes intentionally provide selective/incomplete information or hide vital information when providing product recommendations to customers. according to them, biased information influences customers’ subjective comprehension of the products available, thereby causing negative user experience (i.e., due to not meeting users’ preferences and low-quality results). in addition, the use of visually biased marketing cues is also an instance where intentional manipulations are practiced by e-commerce firms. the literature provides evidence of two kinds of such manipulations. the first is the use of visually biased recommendations which refer to the highlighting of one feature of a product to attract the customer. according to qiu et al. (2015), sometimes a customer might become dissatisfied when other features that are important for evaluating a product are not highlighted, which could lead to a declined purchase. the second is the use of biased marketing cues which may result in the underrepresentation of niche market segments in the input data for a recommender system (wan et al., 2020). for example, a genderneutral product like an armband marketed exclusively via a ‘male’ image might get less attention from female users. as a result of this, a customer representing a niche market segment might experience difficulties in locating a suitable product within the e-commerce platform. riskgenerating events classified under malicious activities are examined in the review’s next section. 4.2 malicious activities the literature provides evidence of different malicious activities associated with e-commerce rs. risk-generating events identified as malicious activities are shilling/profile injection attacks, dishonest ratings by users, unsolicited use of customer data, and review plagiarism (see table 7 australasian journal of information systems kathriarachchi, alam, weerasinghe & pauleen 2024, vol 28, research article risks of e-commerce recommender systems 14 for a summary). many authors (chopra & dixit, 2021, 2023; moradi & hamidi, 2023; xu et al., 2022) in this review identified the shilling or profile injection attacks on e-commerce rs. in shilling or profile injection attacks, malicious users create biased profiles in e-commerce rs with the aim of influencing the system’s behaviour and the list of recommendations (singh et al., 2022; zhang & sheng-hua, 2007). shilling attacks are classified into two areas namely, push and nuke attacks. push attacks are aimed at manipulating the systems to recommend one or more products to more users whilst nuke attacks are aimed at making a targeted product or products less likely to be recommended (chopra & dixit, 2021, 2023; singh et al., 2022). for example, malicious users intentionally engage in push attacks by injecting higher ratings to increase the recommendations for a product and alter the genuine list of recommendations (chopra & dixit, 2021; huang et al., 2021). further, malicious users leave negative reviews or low ratings on products that have no resemblance to the quality of the respective product (chopra & dixit, 2021; xu et al., 2022). sybil attacks are also related to shilling/profile injection attacks where malicious users create multiple fake profiles to inject false recommendations resulting in inappropriate recommendation lists (chopra & dixit, 2021). even popular e-commerce platforms such as amazon have come under these kinds of attacks and such attacks are aimed at affecting the recommendations provided to genuine customers (chung et al., 2013; yang & cai, 2017). risk-generating events source/s shilling/ profile injection attacks (push and/or nuke attacks) by malicious users aghili et al., 2011; alamdari et al., 2020; cai & zhu, 2019; chopra & dixit, 2021, 2023; chung et al., 2013; he et al., 2010; hu et al., 2017; huang et al., 2021; kumar et al., 2015; moradi & hamidi, 2023; singh et al., 2022; wei & shen, 2016; xu et al., 2022; yang et al., 2018; yang & cai, 2017; zhang & sheng-hua, 2007 dishonest ratings by non-malicious users cai & zhu, 2019 obtaining, tracking, storing, using, or divulging sensitive customer information in an unauthorized or undesired manner alamdari et al., 2020; chen, 2022; erkin et al., 2012; frey et al., 2016; hsieh, 2011; jeyamohan et al., 2019; kashani & hamidzadeh, 2020; li et al., 2021; lu & shen, 2015; mallik & sahoo, 2020; mican et al., 2020; polat & du, 2005; ram mohan rao et al., 2018; ran et al., 2022; rohden & zeferino, 2022; vučetić & hudec, 2018; yan & tang, 2011 re-using product reviews (review plagiarism) for unwarranted purposes david & pinch, 2006 table 7: summary of the risk-generating events (malicious activities) like shilling/profile injection attacks, there are also instances where malicious users manipulate product reviews, which are supposed to guide genuine customers and are used as input data to make product recommendations to customers (ahmed et al., 2022). review plagiarism as identified by david and pinch (2006) is a similar issue where product reviews are re-used by users for different purposes such as promoting their own products, opinions, or agenda, attacking others, or identity building. there are situations where dishonest ratings/reviews are posted by non-malicious users (cai & zhu, 2019) with different expectations as mentioned above. these actions by malicious or non-malicious users can negatively influence the quality (chung et al., australasian journal of information systems kathriarachchi, alam, weerasinghe & pauleen 2024, vol 28, research article risks of e-commerce recommender systems 15 2013; singh et al., 2022; yang et al., 2018), accuracy (moradi & hamidi, 2023; xu et al., 2022; yang et al., 2018) and, customers’ trust (aghili et al., 2011; yang & cai, 2017) of the recommendations provided which would pose risks causing customers to purchase inappropriate products (cai & zhu, 2019; huang et al., 2021) and endure negative user experiences (yang et al., 2018; zeng et al., 2021). the frequency of privacy leakages (chen, 2022; ran et al., 2022; rohden & zeferino, 2022) has increased as e-commerce rs continue to collect a tremendous amount of customer data to provide better recommendations. privacy is the ability of an individual to determine what data can be shared, and employ access control (ram mohan rao et al., 2018). it is an enduring issue of for people within the e-commerce setup (tran & huh, 2023) and they trust e-commerce firms to keep their purchase and ratings confidential (ben horin & tassa, 2021). privacy leakages can take place when transmitting data within the rs (li et al., 2021), transferring or selling them to third parties, or failing to provide the required level of physical security (erkin et al., 2012; li et al., 2021; mican et al., 2020; polat & du, 2005; yan & tang, 2011). customers’ personal information/preferences, online purchase records, and keywords can be employed to infer various sensitive information such as their financial status, shopping preferences, personal interests and concerns, gender, sexual inclinations, and political beliefs (li et al., 2021; mican et al., 2020; ram mohan rao et al., 2018) violating their privacy (ran et al., 2022). further, researchers have also highlighted building customer profiles using customer purchasing data to facilitate price discrimination (erkin et al., 2012; polat & du, 2005) and for unsolicited marketing activities (yan & tang, 2011). this suggests privacy violations could lead to negative user experiences in the form of price discrimination imposed on customers. the next section of the review elaborates the risk-generating events associated with customer biases/actions. 4.3 customer biases/actions at times, customers become the source of risk-generating activities (see table 8 for a summary of these events). rs play a significant role in customer decision making in the e-commerce context as mentioned. however, there are diverse kinds of individual-level biases that could harm customers’ welfare. for instance, researchers (ahmed et al., 2022; gopalachari, 2018; wang et al., 2022) have shown that customers are subjected to the influence of the reviews, product ratings, and product decisions of other customers in making product decisions, which is referred to as ‘social influence bias’ or ‘conformity bias’. these researchers have also observed that customers tend to make product decisions under the influence of certain emotional states or interests. further, customers at times tend to favour products simply because they are ranked high (position bias) and/or to favour neighbouring/related products (neighbouring bias) to a target product (gu et al., 2020). hence, it is evident that individual-level biases can sometimes result in customers making poor product purchase decisions. personalised product recommendations depend on customers’ feedback as their preferences (i.e., reviews and ratings) eventually become input data for subsequent recommendations provided to other customers (adomavicius et al., 2013; ahmed et al., 2022; fabbri, 2022). at times product reviews provided by new customers are influenced by the order sequence in which existing online reviews are captured, which is referred to as ‘sequential bias’ (eryarsoy & piramuthu, 2014). australasian journal of information systems kathriarachchi, alam, weerasinghe & pauleen 2024, vol 28, research article risks of e-commerce recommender systems 16 biased purchase data and product reviews serving as explicit input data (eryarsoy & piramuthu, 2014) to e-commerce rs will further aggravate the problem of low-quality recommendations affecting the purchase experience of future customers (ahmed et al., 2022; gopalachari, 2018). risk-generating events source/s customers complying with the choices of other customers (conformity/social influence bias) ahmed et al., 2022; gopalachari, 2018; wang et al., 2022 customers providing biased reviews on products are influenced by the order sequence in which existing online reviews are captured by a new customer (sequential bias) eryarsoy & piramuthu, 2014 customers’ decision biases (irregularities in human decision making) e.g., the tendency to make decisions under certain emotional state and interests ahmed et al., 2022; gopalachari, 2018 customers contributing to data leakages ram mohan rao et al., 2018; yan & tang, 2011 customers’ tendency to favour a product simply because it is ranked high (position bias) or neighbouring/related items influencing click-through rate of a target item (neighbouring bias) gu et al., 2020 customers’ lack of awareness of how algorithms make product recommendations chen, 2022 table 8: summary of the risk-generating events (customer biases/actions) on many occasions, customers’ lack of awareness leads to problems such as privacy issues as well: for instance, agreeing to online terms and conditions on e-commerce platforms without fully understanding the privacy statements. as a result, customers contribute to the privacy issues associated with e-commerce rs (ram mohan rao et al., 2018). to receive personalised recommendations, it is important to provide personal information each time a customer visits a new e-commerce platform. as a result of this, the more websites customers visit, the more personal information is disclosed which would result in increased privacy risks (yan & tang, 2011). in addition to that chen (2022) has identified customers’ general lack of awareness on how algorithms in e-commerce rs arrive at product recommendations (i.e., algorithmic opacity) as a challenge. milano et al. (2020) have opined that providing transparent explanations on why a certain product is recommended to customers will enhance the transparency in the algorithmic decisions. this suggests that the presence of algorithmic opacity might lead customers to make uninformed product decisions which could lead to poor purchase decision risks. the next section of the review details the risk-generating events associated with incompetent e-commerce rs. 4.4 incompetent systems incompetencies associated with e-commerce rs identified in this review are non-consideration of complex user requirements, information overload, and not capturing changing user preferences (see table 9 for a summary). according to vučetić and hudec (2018), customers are often not able to express their requirements (i.e., expectations of specific features) in a precise way when inputting their preferences into an e-commerce rs. on the other hand, information overload is caused by overabundance of products, varieties of comparable products, and overload of related information. such occurrence might lead customers to see the rs as incompetent and to either australasian journal of information systems kathriarachchi, alam, weerasinghe & pauleen 2024, vol 28, research article risks of e-commerce recommender systems 17 decline the recommendations or switch to another company (kim et al., 2016). similarly, dou et al. (2021) have highlighted that users’ interests and emotions (user bias) and their attraction towards specific products (item bias) might change over time. for example, a movie may gain attention when more information related to it becomes known (i.e., item bias). user biases occur in situations where a user’s emotions play an important part in reviewing and rating products. according to dou et al. (2021) collaborative filtering-based rs which rely on past information about user preferences to suggest personalised recommendations are not capable of identifying these kinds of changing preferences referred to as ‘temporal changes’. the technology to capture changing preferences is gradually developing with the incorporation of deep learning algorithms, but there is still room for further improvement. identifying such temporal changes is an essential aspect of providing effective personalised recommendations to customers and a failure to do so would result in a negative experience for customers. the next section presents the risk framework developed based on the potential links between risk-generating events and risks which were identified through this review. risk-generating events source/s incompetent recommender systems with functional issues (i.e., non-consideration of complex user requirements, information overload etc.) kim et al., 2017; vučetić & hudec, 2018 rs not capturing changing user preferences of products over time (user and item bias) dou et al., 2021 table 9: summary of the risk-generating events (incompetent systems) 4.5 risk framework for e-commerce rs as stated, this review’s purpose was to identify the full range of risk-generating events associated with e-commerce rs to overcome the limitations associated with existing risk classifications. in doing so, the link between the identified risk-generating events and the risks was elaborated with reference to the risk classification proposed by jannach and bauer (2020). the authors identified direct and indirect links between events and risks in the literature reviewed (see table a 3 in appendix). risk-generating events identified under the category of ‘biased recommendations’: biased product recommendations, presenting incomplete or selective information, influencing customer preference ratings, lack of serendipity and/or diversity, use of biased visual properties, and biased exposure policy were identified as leading to poor decision risk and bad user experience risk. all these events pose the risk of customers purchasing products that are not suitable for their requirements with the associated risk of negatively affecting their shopping experience. the second category ‘malicious activities’ encapsulated risk-generating events resulting from undesired human involvement in the recommendation process. these events overall are linked to all three types of risks: poor decision risk, bad user experience risk, and privacy risks. dishonest ratings by malicious (i.e., shilling/profile injection attacks) as well as non-malicious users and review plagiarism might mislead customers to make poor purchase decisions as well as creating negative user experience. unsolicited use of customer information certainly has implications for australasian journal of information systems kathriarachchi, alam, weerasinghe & pauleen 2024, vol 28, research article risks of e-commerce recommender systems 18 the privacy risks and issues. e.g., such as using customer information for unsolicited marketing activities might result in risk of bad user experience. the third category of ‘customer biases/actions’ covered instances where customers themselves become the source of risk-generating events. they comprise conformity bias, sequential bias, irregularities in human decision-making, and customers contributing to data leakages. these events also have the potential to lead to all three types of risks. when customers are influenced by reviews and ratings provided by other customers, and positioning of product recommendations, or due to irregularities in human decision making they might end up making poor product choices. when biased reviews provided by customers act as explicit input data to rs, it will lead to providing low-quality product recommendations to other customers which will negatively affect their shopping experience. agreeing to privacy statements without full understanding and exposing personal data on multiple platforms are instances where customers contribute to increased privacy risks. issues associated with rs such as failure to capture complex user requirements, information overload, and failure to capture changing preferences would also lead to negative user experience with e-commerce rs. based on this discussion, a risk framework is developed as shown in figure 4. figure 4: risk framework for e-commerce rs 5 discussion the discussion consists of observations on the risks of e-commerce rs with reference to existing frameworks on risks, the impact of those risks on businesses, policy implications of risks, and australasian journal of information systems kathriarachchi, alam, weerasinghe & pauleen 2024, vol 28, research article risks of e-commerce recommender systems 19 limitations and directions for future research. figure 5 summarizes the discussion presented below. 5.1 observations on risks of e-commerce rs through this review, several risk-generating events that were not previously discussed with reference to e-commerce rs risks were identified. these events included profile injection attacks, review plagiarism, customer biases, and algorithmic opacity. this identification of novel riskgenerating events is expected to provide a comprehensive view of the risk potential of current ecommerce rs. further, the ‘biased information state’ which was originally proposed by jannach and bauer (2020) as a risk was identified as a source of several other types of biases (i.e., biased products, biased information, and biased marketing cues) associated with e-commerce rs. as mentioned before, these biases result in diverse types of risks such as poor decision risk and bad figure 5 summary of the discussion australasian journal of information systems kathriarachchi, alam, weerasinghe & pauleen 2024, vol 28, research article risks of e-commerce recommender systems 20 user experience risk. hence, the findings of this review suggest that the three types of risks poor decision risk, bad user experience risk, and privacy risk proposed by jannach and bauer better represent the risks of e-commerce rs (except biased information state risk). this is also in alignment with the risk classification proposed by glover and benbasat (2010) and re-affirms the comparison presented in table 1. a plurality in the outcomes of the risk-generating events was observed, where some of the riskgenerating events identified were linked to multiple types of risks. for example, biased product recommendations were identified to have links to poor decision risk and bad user experience risk. similarly, the actions of malicious users might lead to poor decision risk and bad user experience risk. however, in glover and benbasat’s e-commerce risk framework, one unwanted event was always linked to one type of risk via the harm it causes. this indicates the potential for a novel approach to conceptualising risks associated with e-commerce rs. a close examination of different risks indicates that there is a temporal difference in customers’ exposure to risk. for example, a customer who receives biased recommendations might have a bad user experience at the time of searching for products/receiving recommendations. however, if the selected product does not meet his/her expectations then the risk of a poor decision would materialize only upon receiving/consuming the product. a similar idea has been presented by milano et al. (2020) in their taxonomy of ethical challenges of rs, where they have identified some ethical challenges involving immediate harm or future exposure to risk. within such a classification bad user experience will cause immediate harm, while poor decision risks and privacy risks create future exposure to risk. next, the impact of identified risk-generating events on businesses is discussed. 5.2 impact of risk-generating events on businesses this review suggests that risk-generating events associated with e-commerce rs are detrimental to customers and businesses. for example, the decision to promote popular products might result in a negative user experience for customers and a long tail of unsold products which could result in a monetary loss for the business. on the other hand, when a company decides to promote unpopular products to increase catalogue coverage, the company might benefit from it in the short term, but customers will lose trust in the product recommendations provided to them. interestingly jannach and bauer (2020) have highlighted these outcomes as organisational risks in their risk classifications of e-commerce rs (i.e., loss of customer trust, loss of societal trust, and monetary loss). this shows how the outcomes of risk-generating events affecting diverse stakeholders involved with e-commerce rs. this observation aligns with the multistakeholder perspective of rs (abdollahpouri et al., 2020; milano et al., 2020). rs are multistakeholder systems and in evaluating the impact of rs, the utility, rights, and risks of all stakeholders need to be considered (milano et al., 2020). accordingly, the generating events identified through this review provide an ideal avenue to identify the risks of e-commerce rs from the multistakeholder perspective. through this review, we were able to identify that the privacy issues associated with e-commerce rs led to several sub-optimal behaviours by customers: namely, not providing personal information (rohden & zeferino, 2022) and providing false information (kashani & hamidzadeh, australasian journal of information systems kathriarachchi, alam, weerasinghe & pauleen 2024, vol 28, research article risks of e-commerce recommender systems 21 2020; polat & du, 2005). in addition, nosi et al. (2022) have highlighted the free-riding behaviour of customers where they search for products online but buy products offline. in the case of incompetent or unreliable rs, some customers might decline recommendations received (kim et al., 2017) and decide to stop purchasing from a particular e-commerce platform (kim et al., 2017; rohden & zeferino, 2022). according to eryarsoy and piramuthu (2014), implicit and explicit data gathered via customer interactions are crucially important to provide effective recommendations. however, when customers avoid associating with e-commerce rs or engage in sub-optimal behaviours as mentioned above, it will affect the quantity and quality of information available to make effective personalised product recommendations. hence, the risk-generating events associated with e-commerce rs could affect the business value of the rs. the next section explains the policy implications of the findings of this review. 5.3 policy implications of the findings present-day e-commerce rs, that apply ai-based technologies can influence human decisionmaking related to products and services, which can have broader implications for society. for example, unhealthy food practices promoted by recommendations can burden the public health system (fabbri, 2022). due to these far-reaching implications, ai applications including rs have received policy-level attention from governments. the european commission’s regulatory framework on high-risk ai systems (european commission, 2023) and the discussion paper published by the australian government on mitigating potential risks of ai (department of industry, science and resources, australia, 2023) are good examples of this. fabbri (2022) has identified that rs outputs are already being recognized under the european commission's policy framework and highlights the need for incorporating the design principles due to the risks posed by biases in rs. in support of this view, di noia et al. (2022) have presented an account of how the issues unique to rs should be mitigated with reference to the european commission’s policy framework. in their work, they have looked at areas such as fairness, security, and privacy which are essential to mitigate the impact of certain risk-generating events that were discussed in this review. these scholars have consistently pointed out the inadequacy of existing policy frameworks in dealing with the negative effects associated with rs, especially with the use of ai in providing recommendations. according to fabbri, existing design principles of rs can pose risks to users which has not received enough regulatory attention. in such a situation, enhanced understanding of diverse types of risk-generating events and risks associated with e-commerce rs would be helpful in contributing to the development of more robust policies and regulations to govern rs used in public domains, which will benefit society. the next section of the review presents the limitations and directions for future research. 5.4 limitations and directions for future research the publications selected for this review were limited to the last twenty years (from 2003 to 2023). we believe that we were able to identify a comprehensive range of risk-generating events as most publications on e-commerce rs were published during the last two decades. only publications in the language of english were considered for this review. however, only a very minimal number of publications were excluded due to this reason. there are several future research avenues that emerge from this scoping review. australasian journal of information systems kathriarachchi, alam, weerasinghe & pauleen 2024, vol 28, research article risks of e-commerce recommender systems 22 5.4.1 empirical exploration of risks the risk framework (figure 4) is proposed based on the risk-generating events and their potential links to the risks as reported in the literature. researchers can make use of the identified list of risk-generating events to empirically test their relationship with the risks of e-commerce rs. such research will contribute towards updating and advancing the existing risk frameworks on ecommerce (e.g., glover & benbasat, 2010). further, the proposed framework can be used to identify potential negative effects of artificial intelligence (ai) enabled e-commerce rs which has thus far received scant attention (necula & păvăloaia, 2023). in addition, the risk-generating events identified through this review can also be utilized to explore customers’ risk perceptions of e-commerce rs by adopting the psychometric paradigm (slovic, 1987). such studies will help illuminate the theoretical understanding of e-commerce rs as a single hazard domain. 5.4.2 evaluation of multistakeholder perspectives on e-commerce rs during the conduct of this review, the authors were also able to identify that certain riskgenerating events can affect multiple stakeholders involved with e-commerce rs. for instance, recommending popular products could lead to dissatisfaction for customers and at the same time lead to long-tail products which are a disadvantage for businesses. however, in our review, we could not identify any significant empirical evidence that looks at the risks of e-commerce rs from a multistakeholder perspective. however, scholars (abdollahpouri et al., 2020; jannach & jugovac, 2019; milano et al., 2020) in the recent past have advocated the need for further research on the impact of rs for different stakeholders to understand the trade-off between their expectations and the reality of the rs. hence, the events identified in this study can be used as a basis to study how contradictory objectives are achieved by key stakeholders such as customers and businesses, as well as system developers. 5.4.3 rs and decision biases the literature reviewed in this study has indicated diverse ways human decision biases come into effect when customers rely on e-commerce rs. this could result in irrational decisions which are sub-optimal in nature. teppan and zanker (2015) are of the opinion that customers should be equipped with the knowledge on decision biases so that they can make more objective decisions. in a recent review, chen et al. (2023) have identified that in the real world context various biases could occur simultaneously. they advocate that the systems should be capable of handling multiple types of biases to enhance fairness and transparency in recommendations. at a time when there is heightened attention towards biases associated with rs, we believe the identification of specific biases and their potential links to risks discussed in this review should encourage researchers to further examine human decision biases associated with the use of ecommerce rs. 5.4.4 customer behaviour and quality of recommendations customers engage in different sub-optimal and free-riding behaviour due to their lack of confidence and/or knowledge about rs (kashani & hamidzadeh, 2020; nosi et al., 2022; rohden & zeferino, 2022). this could negatively affect e-commerce rs because the effectiveness of the recommendations provided depends on the accuracy of the information gathered from customers implicitly and explicitly (eryarsoy & piramuthu, 2014). hence, it would be interesting to study australasian journal of information systems kathriarachchi, alam, weerasinghe & pauleen 2024, vol 28, research article risks of e-commerce recommender systems 23 the conditions under which customers engage in such behaviours. findings on such specific conditions will be helpful to develop strategies to enhance customers’ engagement with ecommerce rs (e.g., privacy-preserving capabilities of e-commerce rs). such strategies could lead to an enhanced quality of personalised product recommendations provided by e-commerce rs to customers. 6 conclusion risks associated with e-commerce rs have received significant attention from scholars and policymakers. a comprehensive understanding of the events leading to such risks is of crucial importance in identifying the risk potential of e-commerce rs. however, the lack of a comprehensive synthesis of the risk-generating events has hindered the advancement of scholarly discussion in this field. this review addressed this gap by identifying risk-generating events and corresponding risks associated with e-commerce rs from the customers’ perspective as reported in the literature. through this review, certain risk-generating events that were not considered in prior research for the conceptualisation of e-commerce risks were also identified. the findings of this review can be utilized to empirically examine risks and explore risk perceptions on ecommerce rs from a multistakeholder perspective. in addition to this, several other important future research directions stemming from the analysis were also suggested. references abdollahpouri, h., adomavicius, g., burke, r., guy, i., jannach, d., kamishima, t., krasnodebski, j., & pizzato, l. (2020). multistakeholder recommendation: survey and research directions. user modeling & user-adapted interaction, 30(1), 127–158. iih. adomavicius, g., bockstedt, j. c., curley, s. p., & jingjing zhang. (2013). do recommender systems manipulate consumer preferences? a study of anchoring effects. information systems research, 24(4), 956–975. doi.org/10.1287/isre.2013.0497 aghili, g., shajari, m., khadivi, s., & morid, m. a. (2011). using genre interest of users to detect profile injection attacks in movie recommender systems. 2011 10th international conference on machine learning and applications and workshops, 1, 49–52. doi.org/10.1109/icmla.2011.151 ahmed, a., saleem, k., khalid, o., gao, j., & rashid, u. (2022). trust-aware denoising autoencoder with spatial-temporal activity for cross-domain personalized recommendations. neurocomputing, 511, 477–494. doi.org/10.1016/j.neucom.2022.09.023 alamdari, p. m., navimipour, n. j., hosseinzadeh, m., safaei, a. a., & darwesh, a. (2020). a systematic study on the recommender systems in e-commerce. ieee access, 8, 115694– 115716. doi.org/10.1109/access.2020.3002803 awad, n. f., & krishnan, m. s. (2006). the personalization privacy paradox: an empirical evaluation of information transparency and the willingness to be profiled online for personalization. mis quarterly, 30(1), 13–28. doi.org/10.2307/25148715 australasian journal of information systems kathriarachchi, alam, weerasinghe & pauleen 2024, vol 28, research article risks of e-commerce recommender systems 24 banker, s., & khetani, s. (2019). algorithm overdependence: how the use of algorithmic recommendation systems can increase risks to consumer well-being. journal of public policy & marketing, 38(4), 500–515. doi.org/10.1177/0743915619858057 ben horin, a., & tassa, t. (2021). privacy preserving collaborative filtering by distributed mediation. fifteenth acm conference on recommender systems, 332–341. doi.org/10.1145/3460231.3474251 cai, y., & zhu, d. (2019). trustworthy and profit: a new value-based neighbor selection method in recommender systems under shilling attacks. decision support systems, 124(c), 1–15. doi.org/10.1016/j.dss.2019.113112 chen, y. (2022). analysis on the impact of recommender systems on consumer decision: making on china’s online shopping platforms. 2022 6th international conference on e-commerce, ebusiness and e-government, 272–276. doi.org/10.1145/3537693.3537734 chen, j., dong, h., wang, x., feng, f., wang, m., & he, x. (2023). bias and debias in recommender system: a survey and future directions. acm transactions on information systems, 41(3), 1–39. doi.org/10.1145/3564284 chevalier, s. (2024, may 22). global retail e-commerce sales 2014-2027. statista. https://www.statista.com/statistics/379046/worldwide-retail-e-commerce-sales/ chopra, a. b., & dixit, v. s. (2021). balanced accuracy of collaborative recommender system. ct systems and sustainability: proceedings of ict4sd 2020, 1, 341–356. doi.org/10.1007/978981-15-8289-9_32 chopra, a. b., & dixit, v. s. (2023). detecting biased user-product ratings for online products using opinion mining. journal of intelligent systems, 32(1), 1–13. doi.org/10.1515/jisys-20229030 chung, c. y., hsu, p. y., & huang, s. h. (2013). β p: a novel approach to filter out malicious rating profiles from recommender systems. decision support systems, 55(1), 314–325. doi.org/10.1016/j.dss.2013.01.020 david, s., & pinch, t. (2006). view of six degrees of reputation: the use and abuse of online review and recommendation systems. first monday, 11(3). https://doi.org/10.5210/fm.v11i3.1315 deng, w., shi, y., chen, z., kwak, w., & tang, h. (2020). recommender system for marketing optimization. world wide web, 23(3), 1497–1517. doi.org/10.1007/s11280-019-00738-1 department of industry, science and resources, australia. (2023). safe and responsible ai in australia. https://consult.industry.gov.au/supporting-responsible-ai di noia, t., tintarev, n., fatourou, p., & schedl, m. (2022). recommender systems under european ai regulations. communications of the acm, 65(4), 69–73. doi.org/10.1145/3512728 dou, r., arslan, o., & zhang, c. (2021). biased autoencoder for collaborative filtering with temporal signals. expert systems with applications, 186, 115775. doi.org/10.1016/j.eswa.2021.115775 australasian journal of information systems kathriarachchi, alam, weerasinghe & pauleen 2024, vol 28, research article risks of e-commerce recommender systems 25 ebrahimi, s., ghasemaghaei, m., & benbasat, i. (2022). the impact of trust and recommendation quality on adopting interactive and non-interactive recommendation agents: a metaanalysis. journal of management information systems, 39(3), 733–764. doi.org/10.1080/07421222.2022.2096549 erkin, z., beye, m., veugen, t., & lagendijk, r. l. (2012). privacy-preserving content-based recommender system. proceedings of the on multimedia and security, 77–84. doi.org/10.1145/2361407.2361420 eryarsoy, e., & piramuthu, s. (2014). experimental evaluation of sequential bias in online customer reviews. information & management, 51(8), 964–971. iih. doi.org/10.1016/j.im.2014.09.001 eslami, m., vaccaro, k., lee, m. k., elazari bar on, a., gilbert, e., & karahalios, k. (2019). user attitudes towards algorithmic opacity and transparency in online reviewing platforms. proceedings of the 2019 chi conference on human factors in computing systems, 1–14. doi.org/10.1145/3290605.3300724 european commission. (2023). regulatory framework proposal on artificial intelligence | shaping europe’s digital future. https://digital-strategy.ec.europa.eu/en/policies/regulatoryframework-ai fabbri, m. (2022). social influence for societal interest: a pro-ethical framework for improving human decision making through multi-stakeholder recommender systems. ai & society, 38(2), 995–1002. doi.org/10.1007/s00146-022-01467-2 felfernig, a. (2014). biases in decision making. international workshop on decision making and recommender systems, bolzano, italy. https://ceur-ws.org/vol-1278/paper6.pdf fleder, d., & hosanagar, k. (2009). blockbuster culture’s next rise or fall: the impact of recommender systems on sales diversity. management science, 55(5), 697–712. doi.org/10.1287/mnsc.1080.0974 frey, r., wörner, d., & ilic, a. (2016). collaborative filtering on the blockchain: a secure recommender system for e-commerce. twenty-second americas conference on information systems (amcis), san diego, ca, usa. https://aisel.aisnet.org/amcis2016/issec/presentations/36/ ge, y., zhao, s., zhou, h., pei, c., sun, f., ou, w., & zhang, y. (2020). understanding echo chambers in e-commerce recommender systems. proceedings of the 43rd international acm sigir conference on research and development in information retrieval, 2261–2270. doi.org/10.1145/3397271.3401431 glover, s., & benbasat, i. (2010). a comprehensive model of perceived risk of e-commerce transactions. international journal of electronic commerce, 15(2), 47–78. https://doiorg.ezproxy.massey.ac.nz/10.2753/jec1086-4415150202 gopalachari, m. (2018). dbt recommender: improved trustworthiness of ratings through debiasing tendency of users. international journal of intelligent engineering and systems, 11(2), 85–92. doi.org/10.22266/ijies2018.0430.10 australasian journal of information systems kathriarachchi, alam, weerasinghe & pauleen 2024, vol 28, research article risks of e-commerce recommender systems 26 grange, c., benbasat, i., & burton-jones, a. (2019). with a little help from my friends: cultivating serendipity in online shopping environments. information & management, 56(2), 225–235. doi.org/10.1016/j.im.2018.06.001 grant, m. j., & booth, a. (2009). a typology of reviews: an analysis of 14 review types and associated methodologies. health information & libraries journal, 26(2), 91-108. gu, y., ding, z., wang, s., zou, l., liu, y., & yin, d. (2020). deep multifaceted transformers for multi-objective ranking in large-scale e-commerce recommender systems. proceedings of the 29th acm international conference on information & knowledge management, 2493–2500. doi.org/10.1145/3340531.3412697 he, f., wang, x., & liu, b. (2010). attack detection by rough set theory in recommendation system. 2010 ieee international conference on granular computing, 692–695. doi.org/10.1109/grc.2010.130 ho, y.-c. (chad), wu, j., & tan, y. (2017). disconfirmation effect on online rating behavior: a structural model. information systems research, 28(3), 626–642. hsieh, c. l. (2011). toward better recommender system by collaborative computation with privacy preserved. 2011 ieee/ipsj international symposium on applications and the internet, 246–249. doi.org/10.1109/saint.2011.46 hu, l., cao, l., cao, j., gu, z., xu, g., & wang, j. (2017). improving the quality of recommendations for users and items in the tail of distribution. acm transactions on information systems, 35(3), 1–37. doi.org/10.1145/3052769 huang, j. t., sun, h. l., chen, x. f., liu, x., & cao, j. (2021). an iterative deviation-based ranking method to evaluate user reputation in online rating systems✱. 2021 4th international conference on data science and information technology, 15–21. doi.org/10.1145/3478905.3478909 jannach, d., & bauer, c. (2020). escaping the mcnamara fallacy: toward more impactful recommender systems research. ai magazine, 41(4), 79–95. doi.org/10.1609/aimag.v41i4.5312 jannach, d., & jugovac, m. (2019). measuring the business value of recommender systems. acm transactions on management information systems, 10(4), 1–23. doi.org/10.1145/3370082 jannach, d., pu, p., ricci, f., & zanker, m. (2022). recommender systems: trends and frontiers. ai magazine, 43(2), 145–150. doi.org/10.1002/aaai.12050 jannach, d., zanker, m., ge, m., & gröning, m. (2012). recommender systems in computer science and information systems – a landscape of research. e-commerce and web technologies, 123, 76–87. doi.org/10.1007/978-3-642-32273-0_7 jeyamohan, n., chen, x., & aslam, n. (2019). local differentially private matrix factorization for recommendations. 2019 13th international conference on software, knowledge, information management and applications (skima), 1–5. doi.org/10.1109/skima47702.2019.8982536 australasian journal of information systems kathriarachchi, alam, weerasinghe & pauleen 2024, vol 28, research article risks of e-commerce recommender systems 27 kashani, s. m. z., & hamidzadeh, j. (2020). feature selection by using privacy-preserving of recommendation systems based on collaborative filtering and mutual trust in social networks. soft computing, 24(15), 11425–11440. doi.org/10.1007/s00500-019-04605-z kim, h., benbasat, i., & cavusoglu, h. (2017). online consumers’ attribution of inconsistency between advice sources. icis 2017 proceedings, 1–10. https://www.proceedings.com/37640.html kiswanto, d., nurjanah, d., & rismala, r. (2018). fairness aware regularization on a learningto-rank recommender system for controlling popularity bias in e-commerce domain. 2018 international conference on information technology systems and innovation (icitsi), 16–21. doi.org/10.1109/icitsi.2018.8696023 kong, d., tang, j., zhu, z., cheng, j., & zhao, y. (2017). de-biased dart ensemble model for personalized recommendation. 2017 ieee international conference on multimedia and expo (icme), 553–558. doi.org/10.1109/icme.2017.8019536 kumar, a., garg, d., & rana, p. s. (2015). ensemble approach to detect profile injection attack in recommender system. 2015 international conference on advances in computing, communications and informatics (icacci), 1734–1740. doi.org/10.1109/icacci.2015.7275864 laskar, a. k., ahmed, j., sohail, s. s., nafis, a., & haq, z. a. (2023). shilling attacks on recommender system: a critical analysis. 10th international conference on computing for sustainable global development (indiacom), 1617–1622. https://ieeexplore-ieeeorg.ezproxy.massey.ac.nz/document/10112342 lee, d., & hosanagar, k. (2019). how do recommender systems affect sales diversity? a crosscategory investigation via randomized field experiment. information systems research, 30(1), 239–259. doi.org/10.1287/isre.2018.0800 lee, y. h., hu, p. j. h., cheng, t. h., & hsieh, y. f. (2012). a cost-sensitive technique for positiveexample learning supporting content-based product recommendations in b-to-c ecommerce. decision support systems, 53(1), 245–256. doi.org/10.1016/j.dss.2012.01.018 li, g., yin, g., yang, j., & chen, f. (2021). sdrm-ldp: a recommendation model based on local differential privacy. wireless communications and mobile computing, 2021, 1–15. doi.org/10.1155/2021/6640667 li, s., & karahanna, e. (2015). online recommendation systems in a b2c e-commerce context: a review and future directions. journal of the association for information systems, 16(2), 72– 107. doi.org/10.17705/1jais.00389 lu, z., & shen, h. (2015). an accuracy-assured privacy-preserving recommender system for internet commerce. computer science and information systems, 12(4), 1307–1326. doi.org/10.2298/csis140725056l mallik, s., & sahoo, a. (2020). a comparison study of different privacy preserving techniques in collaborative filtering based recommender system. computational intelligence in data mining, 193–203. doi.org/10.1007/978-981-13-8676-3_17 australasian journal of information systems kathriarachchi, alam, weerasinghe & pauleen 2024, vol 28, research article risks of e-commerce recommender systems 28 mican, d., sitar-tăut, d. a., & moisescu, o. i. (2020). perceived usefulness: a silver bullet to assure user data availability for online recommendation systems. decision support systems, 139, 113420. doi.org/10.1016/j.dss.2020.113420 milano, s., taddeo, m., & floridi, l. (2020). recommender systems and their ethical challenges. ai & society, 35(4), 957–967. doi.org/10.1007/s00146-020-00950-y moradi, r., & hamidi, h. (2023). a new mechanism for detecting shilling attacks in recommender systems based on social network analysis and gaussian rough neural network with emotional learning. international journal of engineering, 36(2), 321–334. doi.org/10.5829/ije.2023.36.02b.12 munn, z., peters, m. d. j., stern, c., tufanaru, c., mcarthur, a., & aromataris, e. (2018). systematic review or scoping review? guidance for authors when choosing between a systematic or scoping review approach. bmc medical research methodology, 18(1), 143. doi.org/10.1186/s12874-018-0611-x murphy, m. s. (2011). notes toward a politics of personalization. proceedings of the 2011 iconference, 546–551. doi.org/10.1145/1940761.1940836 necula, s. c., & păvăloaia, v. d. (2023). ai-driven recommendations: a systematic review of the state of the art in e-commerce. applied sciences, 13(9), 5531. doi.org/10.3390/app13095531 niu, k., zhao, x., li, f., li, n., peng, x., & chen, w. (2019). utsp: user-based two-step recommendation with popularity normalization towards diversity and novelty. ieee access, access, ieee, 7, 145426–145434. edseee. doi.org/10.1109/access.2019.2939945 nosi, c., zollo, l., rialti, r., & ciappei, c. (2022). why do consumers free ride? investigating the effects of cognitive effort on postpurchase dissonance. journal of consumer marketing, 39(5), 417–431. doi.org/10.1108/jcm-02-2021-4436 nowell, l. s., norris, j. m., white, d. e., & moules, n. j. (2017). thematic analysis: striving to meet the trustworthiness criteria. international journal of qualitative methods, 16(1), 1609406917733847. doi.org/10.1177/1609406917733847 panniello, u., hill, s., & gorgoglione, m. (2016). the impact of profit incentives on the relevance of online recommendations. electronic commerce research and applications, 20, 87–104. doi.org/10.1016/j.elerap.2016.10.003 peters, m. d. j., godfrey, c. m., khalil, h., mcinerney, p., parker, d., & soares, c. b. (2015). guidance for conducting systematic scoping reviews. international journal of evidence-based healthcare, 13(3), 141–146. doi.org/10.1097/xeb.0000000000000050 pizzato, l., rej, t., akehurst, j., koprinska, i., yacef, k., & kay, j. (2013). recommending people to people: the nature of reciprocal recommenders with a case study in online dating. user modeling & user-adapted interaction, 23(5), 447–488. iih. polat, h., & du, w. (2005). privacy-preserving collaborative filtering. international journal of electronic commerce, 9(4), 9–35. doi.org/10.1080/10864415.2003.11044341 australasian journal of information systems kathriarachchi, alam, weerasinghe & pauleen 2024, vol 28, research article risks of e-commerce recommender systems 29 puntoni, s., reczek, r. w., giesler, m., & botti, s. (2021). consumers and artificial intelligence: an experiential perspective. journal of marketing, 85(1), 131–151. doi.org/10.1177/0022242920953847 qiu, j., lin, z., & li, y. (2015). predicting customer purchase behavior in the e-commerce context. electronic commerce research, 15(4), 427–452. doi.org/10.1007/s10660-015-9191-6 qiu, r., wang, s., chen, z., yin, h., & huang, z. (2021). causalrec: causal inference for visual debiasing in visually-aware recommendation. proceedings of the 29th acm international conference on multimedia, 3844–3852. doi.org/10.1145/3474085.3475266 ram mohan rao, p., murali krishna, s., & siva kumar, a. p. (2018). privacy preservation techniques in big data analytics: a survey. journal of big data, 5(1), 33. doi.org/10.1186/s40537-018-0141-8 ran, x., wang, y., zhang, l. y., & ma, j. (2022). a differentially private matrix factorization based on vector perturbation for recommender system. neurocomputing, 483, 32–41. doi.org/10.1016/j.neucom.2022.01.079 rohden, s. f., & zeferino, d. g. (2022). recommendation agents: an analysis of consumers’ risk perceptions toward artificial intelligence. electronic commerce research. doi.org/10.1007/s10660-022-09626-9 silva, n., carvalho, d., pereira, a. c. m., mourão, f., & rocha, l. (2019). the pure cold-start problem: a deep study about how to conquer first-time users in recommendations domains. information systems, 80, 1–12. doi.org/10.1016/j.is.2018.09.001 singh, p. k., pramanik, p. k. d., sardar, m., nayyar, a., masud, m., & choudhury, p. (2022). generating a new shilling attack for recommendation systems. computers, materials & continua, 71(2), 2827–2846. doi.org/10.32604/cmc.2022.020437 smith, b., & linden, g. (2017). two decades of recommender systems at amazon.com. ieee internet computing, 21(3), 12–18. doi.org/10.1109/mic.2017.72 slovic, p. (1987). perception of risk. science, 236(4799). doi.org/10.1126/science.3563507 sreepada, r. s., & patra, b. k. (2021). enhancing long tail item recommendation in collaborative filtering: an econophysics-inspired approach. electronic commerce research and applications, 49, 1–11. doi.org/10.1016/j.elerap.2021.101089 teppan, e. c., & zanker, m. (2015). decision biases in recommender systems. journal of internet commerce, 14(2), 255–275. doi.org/10.1080/15332861.2015.1018703 tiihonen, j., & felfernig, a. (2017). an introduction to personalization and mass customization. journal of intelligent information systems, 49(1), 1–7. doi.org/10.1007/s10844-017-0465-4 trakulwaranont, d., kastner, m. a., & satoh, s. (2022). personalized fashion recommendation using pairwise attention. multimedia modeling, 218–229. doi.org/10.1007/978-3-030-983550_19 australasian journal of information systems kathriarachchi, alam, weerasinghe & pauleen 2024, vol 28, research article risks of e-commerce recommender systems 30 tran, d. t., & huh, j. h. (2023). forecast of seasonal consumption behavior of consumers and privacy-preserving data mining with new s-apriori algorithm. the journal of supercomputing, 79, 12691–12736. doi.org/10.1007/s11227-023-05105-6 tricco, a. c., lillie, e., zarin, w., o’brien, k. k., colquhoun, h., levac, d., moher, d., peters, m. d. j., horsley, t., weeks, l., hempel, s., akl, e. a., chang, c., mcgowan, j., stewart, l., hartling, l., aldcroft, a., wilson, m. g., garritty, c., … straus, s. e. (2018). prisma extension for scoping reviews (prisma-scr): checklist and explanation. annals of internal medicine, 169(7), 467–473. doi.org/10.7326/m18-0850 vailati riboni, f., comazzi, b., bercovitz, k., castelnuovo, g., molinari, e., & pagnini, f. (2020). technologically enhanced psychological interventions for older adults: a scoping review. bmc geriatrics, 20(1), 191. doi.org/10.1186/s12877-020-01594-9 vučetić, m., & hudec, m. (2018). a fuzzy query engine for suggesting the products based on conformance and asymmetric conjunction. expert systems with applications, 101, 143–158. doi.org/10.1016/j.eswa.2018.01.049 wan, m., ni, j., misra, r., & mcauley, j. (2020). addressing marketing bias in product recommendations. proceedings of the 13th international conference on web search and data mining, 618–626. doi.org/10.1145/3336191.3371855 wang, h., wang, z., & zhang, w. (2018). quantitative analysis of matthew effect and sparsity problem of recommender systems. 2018 ieee 3rd international conference on cloud computing and big data analysis (icccbda), 78–82. doi.org/10.1109/icccbda.2018.8386490 wang, w., xu, j. (david), & wang, m. (2018). effects of recommendation neutrality and sponsorship disclosure on trust vs. distrust in online recommendation agents: moderating role of explanations for organic recommendations. management science, 64(11), 5198–5219. doi.org/10.1287/mnsc.2017.2906 wang, y., ma, w., zhang, m., liu, y., & ma, s. (2023). a survey on the fairness of recommender systems. acm transactions on information systems, 41(3), 52:1-52:43. doi.org/10.1145/3547333 wang, z., he, y., liu, j., zou, w., yu, p. s., & cui, p. (2022). invariant preference learning for general debiasing in recommendation. proceedings of the 28th acm sigkdd conference on knowledge discovery and data mining, 1969–1978. doi.org/10.1145/3534678.3539439 wei, r., & shen, h. (2016). an improved collaborative filtering recommendation algorithm against shilling attacks. 2016 17th international conference on parallel and distributed computing, applications and technologies (pdcat), 330–335. doi.org/10.1109/pdcat.2016.077 wilson, r. s., zwickle, a., & walpole, h. (2019). developing a broadly applicable measure of risk perception. risk analysis, 39(4), 777–791. doi.org/10.1111/risa.13207 xiao, b., & benbasat, i. (2014). research on the use, characteristics, and impact of e-commerce product recommendation agents: a review and update for 2007–2012. in handbook of strategic e-business management (pp. 403–431). springer. doi.org/10.1007/978-3-642-397479_18 australasian journal of information systems kathriarachchi, alam, weerasinghe & pauleen 2024, vol 28, research article risks of e-commerce recommender systems 31 xiao, b., & benbasat, i. (2015). designing warning messages for detecting biased online product recommendations: an empirical investigation. information systems research, 26(4), 793–811. edswss. doi.org/10.1287/isre.2015.0592 xiao, b., & benbasat, i. (2018). an empirical examination of the influence of biased personalized product recommendations on consumers’ decision making outcomes. decision support systems, 110, 46–57. doi.org/10.1016/j.dss.2018.03.005 xiao, b. s., & tan, c.w. (2012). reducing perceived deceptiveness of e-commerce product recommendation agents: an empirical examination of the relative impact of transparency and verifiability and the moderating role of gender. amcis 2012 proceedings. americas conference on information systems. https://aisel.aisnet.org/amcis2012/proceedings/hcistudies/28 xiao, pei, q., yao, l., & wang, x. (2020). recrisk: an enhanced recommendation model with multi-facet risk control. expert systems with applications, 158, 113561. doi.org/10.1016/j.eswa.2020.113561 xin, x., yang, j., wang, h., ma, j., ren, p., luo, h., shi, x., chen, z., & ren, z. (2023). on the user behavior leakage from recommender system exposure. acm transactions on information systems, 41(3), 1–25. doi.org/10.1145/3568954 xu, y., yang, y., wang, e., zhuang, f., & xiong, h. (2022). detect professional malicious user with metric learning in recommender systems. ieee transactions on knowledge and data engineering, 34(9), 4133–4146. doi.org/10.1109/tkde.2020.3040618 yan, c. m., & tang, t. j. (2011). applying customer-centered recommendation on an on-line shopping system. 2011 seventh international conference on natural computation, 4, 1993–1997. doi.org/10.1109/icnc.2011.6022582 yang, z., & cai, z. (2017). detecting abnormal profiles in collaborative filtering recommender systems. journal of intelligent information systems, 48(3), 499–518. doi.org/10.1007/s10844016-0424-5 yang, z., sun, q., & zhang, b. (2018). evaluating prediction error for anomaly detection by exploiting matrix factorization in rating systems. ieee access, 6, 50014–50029. doi.org/10.1109/access.2018.2869271 zeng, t., fang, x., lang, y., peng, j., wu, x., wang, s., & gong, j. (2021). fair personalized recommendation through improved matrix factorization by neural networks. 2021 the 10th international conference on networks, communication and computing, 19–24. doi.org/10.1145/3510513.3510517 zhang, f. g., & sheng-hua, x. (2007). analysis of trust-based e-commerce recommender systems under recommendation attacks. the first international symposium on data, privacy, and ecommerce (isdpe 2007,) 385-390. doi.org/10.1109/isdpe.2007.75 australasian journal of information systems kathriarachchi, alam, weerasinghe & pauleen 2024, vol 28, research article risks of e-commerce recommender systems 32 appendix 1 no. journal title year of publication publication outlet 1 ensemble approach to detect profile injection attack in recommender system 2015 2015 international conference on advances in computing, communications and informatics, icacci 2015 2 fairness aware regularization on a learning-to-rank recommender system for controlling popularity bias in ecommerce domain 2018 2018 international conference on information technology systems and innovation, icitsi 2018 proceedings 3 local differentially private matrix factorization for recommendations 2019 2019 13th international conference on software, knowledge, information management and applications, skima 2019 4 an iterative deviation-based ranking method to evaluate user reputation in online rating systems 2021 acm international conference proceeding series 5 analysis on the impact of recommender systems on consumer decision making on china's online shopping platforms 2022 6 fair personalized recommendation through improved matrix factorization by neural networks 2021 7 improving the quality of recommendations for users and items in the tail of distribution 2017 acm transactions on information systems 8 a comparison study of different privacy preserving techniques in collaborative filtering based recommender system 2020 advances in intelligent systems and computing 9 balanced accuracy of collaborative recommender system. 2021 10 social influence for societal interest: a pro-ethical framework for improving human decision making through multi-stakeholder recommender systems 2022 ai and society 11 collaborative filtering on the blockchain: a secure recommender system for e-commerce 2016 amcis 2016: surfing the it innovation wave 22nd americas conference on information systems 12 an accuracy-assured privacy-preserving recommender system for internet commerce 2015 computer science & information systems 13 generating a new shilling attack for recommendation systems 2022 computers, materials & continua 14 a cost-sensitive technique for positive-example learning supporting content-based product recommendations in b-to-c ecommerce 2012 decision support systems australasian journal of information systems kathriarachchi, alam, weerasinghe & pauleen 2024, vol 28, research article risks of e-commerce recommender systems 33 15 an empirical examination of the influence of biased personalized product recommendations on consumers' decision making outcomes 2018 16 perceived usefulness: a silver bullet to assure user data availability for online recommendation systems 2020 decision support systems 17 trustworthy and profit: a new value-based neighbor selection method in recommender systems under shilling attacks 2019 18 β p : a novel approach to filter out malicious rating profiles from recommender systems 2013 19 recommendation agents: an analysis of consumers’ risk perceptions toward artificial intelligence 2022 electronic commerce research 20 enhancing long tail item recommendation in collaborative filtering: an econophysics-inspired approach 2021 electronic commerce research & applications 21 the impact of profit incentives on the relevance of online recommendations 2016 22 a fuzzy query engine for suggesting the products based on conformance and asymmetric conjunction 2018 expert systems with applications 23 biased autoencoder for collaborative filtering with temporal signals 2021 24 recrisk: an enhanced recommendation model with multi-facet risk control | elsevier enhanced reader 2020 25 view of six degrees of reputation: the use and abuse of online review and recommendation systems 2006 first monday 26 online consumers' attribution of inconsistency between advice sources 2017 icis 2017: transforming society with digital innovation 27 a systematic study on the recommender systems in the ecommerce 2017 ieee access 28 evaluating prediction error for anomaly detection by exploiting matrix factorization in rating systems 2018 29 utsp: user-based two-step recommendation with popularity normalization towards diversity and novelty 2019 30 detect professional malicious user with metric learning in recommender systems 2022 ieee transactions on knowledge and data engineering 31 experimental evaluation of sequential bias in online customer reviews 2014 information & management australasian journal of information systems kathriarachchi, alam, weerasinghe & pauleen 2024, vol 28, research article risks of e-commerce recommender systems 34 32 with a little help from my friends: cultivating serendipity in online shopping environments 2019 33 the pure cold-start problem: a deep study about how to conquer first-time users in recommendations domains 2019 information systems 34 designing warning messages for detecting biased online product recommendations: an empirical investigation 2015 information systems research 35 do recommender systems manipulate consumer preferences? a study of anchoring effects 2013 36 deep multifaceted transformers for multi-objective ranking in large-scale e-commerce recommender systems 2020 international conference on information and knowledge management, proceedings 37 personalized fashion recommendation using pairwise attention 2022 international conference on multimedia modeling 38 privacy-preserving collaborative filtering 2005 international journal of electronic commerce 39 a new mechanism for detecting shilling attacks in recommender systems based on social network analysis and gaussian rough neural network with emotional learning 2023 international journal of engineering transactions c: aspects 40 dbt recommender: improved trustworthiness of ratings through de-biasing tendency of users 2018 international journal of intelligent engineering and systems 41 privacy preservation techniques in big data analytics: a survey 2018 journal of big data 42 detecting abnormal profiles in collaborative filtering recommender systems 2017 journal of intelligent information systems 43 detecting biased user-product ratings for online products using opinion mining 2023 journal of intelligent systems 44 blockbuster culture's next rise or fall: the impact of recommender systems on sales diversity 2009 management science 45 effects of recommendation neutrality and sponsorship disclosure on trust vs. distrust in online recommendation agents: moderating role of explanations for organic recommendations 2018 46 causalrec: causal inference for visual debiasing in visuallyaware recommendation 2021 mm 2021 proceedings of the 29th acm international conference on multimedia 47 privacy-preserving content-based recommender system 2012 mm and sec'12 proceedings of the 14th acm multimedia and security workshop 48 a differentially private matrix factorization based on vector perturbation for recommender system 2022 neurocomputing australasian journal of information systems kathriarachchi, alam, weerasinghe & pauleen 2024, vol 28, research article risks of e-commerce recommender systems 35 49 trust-aware denoising autoencoder with spatial-temporal activity for cross-domain personalized recommendations 2022 50 an improved collaborative filtering recommendation algorithm against shilling attacks 2016 parallel and distributed computing, applications and technologies, pdcat proceedings 51 using genre interest of users to detect profile injection attacks in movie recommender systems 2011 proceedings 10th international conference on machine learning and applications, icmla 2011 52 toward better recommender system by collaborative computation with privacy preserved 2011 proceedings 11th ieee/ipsj international symposium on applications and the internet, saint 2011 53 attack detection by rough set theory in recommendation system 2010 proceedings 2010 ieee international conference on granular computing, grc 2010 54 applying customer-centered recommendation on an on-line shopping system 2011 proceedings 2011 7th international conference on natural computation, icnc 2011 55 analysis of trust-based e-commerce recommender systems under recommendation attacks 2007 proceedings of the 1st international symposium on data, privacy, and e-commerce, isdpe 2007 56 invariant preference learning for general debiasing in recommendation 2022 proceedings of the acm sigkdd international conference on knowledge discovery and data mining 57 privacy preserving collaborative filtering by distributed mediation 2021 recsys 2021 15th acm conference on recommender systems 58 understanding echo chambers in e-commerce recommender systems 2020 sigir 2020 proceedings of the 43rd international acm sigir conference on research and development in information retrieval 59 feature selection by using privacy-preserving of recommendation systems based on collaborative filtering and mutual trust in social networks 2020 soft computing 60 sdrm-ldp: a recommendation model based on local differential privacy 2021 wireless communications and mobile computing 61 recommender system for marketing optimization 2020 world wide web 62 addressing marketing bias in product recommendations 2020 wsdm 2020 proceedings of the 13th international conference on web search and data mining table: a1 selected publications and journal titles australasian journal of information systems kathriarachchi, alam, weerasinghe & pauleen 2024, vol 28, research article risks of e-commerce recommender systems 36 appendix 2: category risk-generating events outcomes source/s biased recommendations recommending popular products (popularity bias), less frequently purchased products, and/or ignoring unpopular/new/obscure products (long tail products) resulting in a "long tail" of unsold items in the product space and those items are not shown to customers; lack of diversity and novelty in the recommendations weakening customer experience; customers foregoing better customer-product matches; potential balkanization, herding (convergence of groups with similar interests); ‘filters’ creating a fragmented society; poor personalisation of recommendations to customers (deng et al., 2020; fleder & hosanagar, 2009; gu et al., 2020; kiswanto et al., 2018; lee et al., 2012; niu et al., 2019; sreepada & patra, 2021; vučetić & hudec, 2018; z. wang et al., 2022; zeng et al., 2021) presenting selective/incomplete information or hiding information incomplete or hidden information impairing user experience; influences on users’ subjective comprehension of services which may lead to variations of personal preferences; negative impact on the accuracy of the rs (kim et al., 2017; xiao et al., 2020) promoting profitable products/unprofitable products users exploited by biased rs; users’ distrust of rs when they are aware of recommendation biases; customers believing that the biased recommendations make product searching easier and recommended products are the best choice for them (affecting customers’ decision quality and decision effort); negative impact on the credibility of the rs (panniello et al., 2016; wang et al., 2018; xiao & benbasat, 2015, 2018) use of biased/unbalanced user data to provide biased product recommendations to firs time/lesserknown customers unhelpful recommendations for rare occasions or lesser-known users; certain first-time users not being satisfied with the biased recommendations (hu et al., 2017; silva et al., 2019; trakulwaranont et al., 2022) generating biased system ratings which influence subsequent customer preference ratings (anchoring effect) the feedback loop between user-reported ratings and system-generated recommendations is affected (cascading error effect on the performance of the rs); anchoring effects biasing the inputs to rs; businesses using rs strategically for their (adomavicius et al., 2013; fabbri, 2022; xiao & benbasat, 2015) australasian journal of information systems kathriarachchi, alam, weerasinghe & pauleen 2024, vol 28, research article risks of e-commerce recommender systems 37 category risk-generating events outcomes source/s competitive advantage (e.g., inventory management) lacking the surprise element (i.e., serendipity) and/or diversity in recommendations lack of shopper engagement and satisfaction; continued narrow exposure of items raised by rs resulting in an echo chamber (like-minded users/customers); recommendations making people isolated from diverse content (filter bubble); negative social outcomes of bad customer choices (fabbri, 2022; ge et al., 2020; grange et al., 2019) presenting visually biased product recommendations and/or using biased marketing cues to promote products customers’ dissatisfaction with other features (e.g., brand, material) leads to rejecting a product; underrepresentation of niche markets in the input data for a rs; customers struggling to find relevant products; potential ethical and social issues (qiu et al., 2021; wan et al., 2020) biased policies on exposing search results to customers (exposure bias) influence on users’ purchase behaviour (wang et al., 2022) malicious activities shilling/ profile injection attacks by malicious users users may unreasonably focus on brands with top scores leading to poor product choices; honest merchants may suffer excessive loss or even be driven to bankruptcy; investments in fake raters getting promoted; unfair treatment of both users and merchants leading to market disorder; decreasing the accuracy and overall user satisfaction (quality of predictions); decrease in trustworthiness/reliability in recommendations; good or niche products might be invisible to customers; a suitable product initially poorly rated or not rated eventually getting removed from the list of recommendations (long-tail) (aghili et al., 2011; alamdari et al., 2020; cai & zhu, 2019; chopra & dixit, 2021, 2023; chung et al., 2013; he et al., 2010; hu et al., 2017; huang et al., 2021; kumar et al., 2015; moradi & hamidi, 2023; singh et al., 2022; wei & shen, 2016; xu et al., 2022; yang et al., 2018; yang & cai, 2017; zhang & sheng-hua, 2007) dishonest ratings by non-malicious users dishonest ratings prompting misleading recommendations (cai & zhu, 2019) obtaining, tracking, storing, using, or divulging sensitive customer violation of individual privacy; users providing false information or not providing (alamdari et al., 2020; chen, 2022; erkin et al., 2012; frey et al., 2016; hsieh, 2011; australasian journal of information systems kathriarachchi, alam, weerasinghe & pauleen 2024, vol 28, research article risks of e-commerce recommender systems 38 category risk-generating events outcomes source/s information in an unauthorized or undesired manner information resulting in false predictions; certain new customers moving away from ecommerce sites; users’ hesitance to provide their credible preference; influence on purchase intentions, loyalty, and relationship with the brand jeyamohan et al., 2019; kashani & hamidzadeh, 2020; li et al., 2021; lu & shen, 2015; mallik & sahoo, 2020; mican et al., 2020; polat & du, 2005; ram mohan rao et al., 2018; ran et al., 2022; rohden & zeferino, 2022; vučetić & hudec, 2018; yan & tang, 2011) re-using product reviews (review plagiarism) for unwarranted purposes (david & pinch, 2006) customer biases/actions customers complying with the choices of other customers (conformity/social influence bias) influence on user’s purchase behaviour; negative impact on the quality of the recommendations provided to future customers (ahmed et al., 2022; gopalachari, 2018; z. wang et al., 2022) customers providing biased reviews on products are influenced by the order sequence in which existing online reviews are captured by a new customer (sequential bias) leading to inaccuracies in recommendations that use online customer reviews as explicit input information to generate recommendations (eryarsoy & piramuthu, 2014) customers’ decision biases (irregularities in human decision making) e.g., the tendency to make decisions under certain emotional state and interests affecting accuracy of the recommendations (ahmed et al., 2022; gopalachari, 2018) customers contributing to data leakages increased privacy risks to customers (ram mohan rao et al., 2018; yan & tang, 2011) customers’ tendency to favour a product simply because it is ranked high (position bias) or neighbouring/related items influencing click-through rate of a target item (neighbouring bias) (gu et al., 2020) customers’ lack of awareness of how algorithms make product recommendations tendency to make inferior product choices (chen, 2022) australasian journal of information systems kathriarachchi, alam, weerasinghe & pauleen 2024, vol 28, research article risks of e-commerce recommender systems 39 table a2: risk-generating events, outcomes, and sources figure a1: top 10 most frequent keywords category risk-generating events outcomes source/s incompetent systems incompetent recommender systems with functional issues (i.e., nonconsideration of complex user requirements, information overload etc.) customers facing challenges in validating the recommendations; customers declining recommendations; customers switching to other stores (kim et al., 2017; vučetić & hudec, 2018) rs not capturing changing user preferences for products over time (user and item bias) rs failing to capture changing customer preferences (dou et al., 2021) australasian journal of information systems kathriarachchi, alam, weerasinghe & pauleen 2024, vol 28, research article risks of e-commerce recommender systems 40 figure a2: keywords over years australasian journal of information systems kathriarachchi, alam, weerasinghe & pauleen 2024, vol 28, research article risks of e-commerce recommender systems 41 table a3: links between risk-generating events and risks australasian journal of information systems kathriarachchi, alam, weerasinghe & pauleen 2024, vol 28, research article risks of e-commerce recommender systems 42 copyright copyright © 2024 kathriarachchi, alam, weerasinghe & pauleen. this is an open-access article licensed under a creative commons attribution-non-commercial 4.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v28.4869 https://creativecommons.org/licenses/by-nc/4.0/deed.en https://creativecommons.org/licenses/by-nc/4.0/deed.en paper title (paper title style) australasian journal of information systems hacker, johnson, saunders & thayer 2019, vol 23, research article trust in virtual teams 1 trust in virtual teams: a multidisciplinary review and integration janine hacker university of zurich jhacker@ifi.uzh.ch michael johnson louisiana state university carol saunders university of south florida amanda l. thayer the university of akron abstract organizations have increasingly turned to the use of virtual teams (vts) to tackle the complex nature of today’s organizational issues. to address these practical needs, vts researchers from different disciplines have begun to amass a large literature. however, the changing workplace that is becoming so reliant on vts comes with its own set of management challenges, which are not sufficiently addressed by current research on vts. paradoxically, despite the challenges associated with technology in terms of its disruption to trust development in vts, trust is one of the most promising solutions for overcoming myriad problems. though the extant literature includes an abundance of studies on trust in vts, a comprehensive multidisciplinary review and synthesis is lacking. addressing this gap, we present a systematic theoretical review of 124 articles from the disparate, multidisciplinary literature on trust in vts. we use the review to develop an integrated model of trust in vts. based on our review, we provide theoretical insights into the relationship between virtuality and team trust, and highlight several critical suggestions for moving this literature forward to meet the needs of workplaces of the future, namely: better insight into how trust evolves alongside the team’s evolution, clarity about how to adequately conceptualize and operationalize virtuality, and greater understanding about how trust might develop differently across diverse types of virtual contexts with various technology usages. we conclude with guidelines for managing vts in the future workplace, which is increasingly driven and affected by changing technologies, and highlight important trends to consider. keywords: virtual teams, trust, swift trust, virtuality, team processes 1 introduction in an attempt to create business value (chatfield, shlemoon, redublado, & darbyshire, 2014), as well as capture diverse knowledge resources, time cost savings, and more affordable opportunities for collaboration, organizations are increasing their reliance on virtual teams (vts) (gibson & gibbs, 2006). a 2003 gartner study estimated that 60% of all major corporations used vts (see gibson & gibbs, 2006; martins, gilson, & maynard, 2004). in 2012, the estimate for 2016 grew to 80% (gartner, 2012). as rosy as this estimate appears to have been, it was exceeded. a survey of vts in 2016 found that 85% of the respondents in corporations work on vts. another survey of 3000 managers located in 100 countries reported australasian journal of information systems hacker, johnson, saunders & thayer 2019, vol 23, research article trust in virtual teams 2 that 40% of the employees in their organizations spend at least half their time on work involving vts (hoch & dulebohn, 2017). furthermore, many of these vts (48%) are global, as compared to 41% in 2014 and 33% in 2012 (rw3 culturewizard, 2016). due to their increasing prevalence and reliance on information technologies, vts have been studied steadily and frequently by researchers in information systems (is), human resource management (hrm), management, computer science, psychology, and other disciplines for two decades. indeed, vts continue to be an important topic in is research (e.g., bartelt & dennis, 2014; chatfield et al., 2014; dennis, robert, curtis, kowalczyk, & hasty, 2012). however, the changing workplace that is becoming so reliant on vts comes with its own set of management challenges (chatfield et al., 2014; jimenez, boehe, taras, & caprar, 2017; lukić & vračar, 2018), which are not sufficiently addressed by current research on vts and have remained unsolved up until now. pearlson, saunders, & galletta (2016) suggest three types of challenges that managers of vts must overcome: (1) communication (e.g., scheduling meetings, passing work across time zones, inability to see communication dynamics such as facial expressions and gestures); (2) technology (e.g., ensuring workers have proficiency across a range of technologies, aligning technology across members); and (3) team diversity (e.g., building trust, norms and shared meanings; establishing a group identity). though trust development is suggested as one challenge (amongst others), we consider it as an important prerequisite for vt leaders to solve communicationand technology-related challenges. in fact, the accumulating research across a range of disciplines, among them is, management, psychology, computer science and communication, shows that interpersonal trust can help overcome many vt challenges, and some have identified it as the heart of vt functioning (e.g., jarvenpaa, knoll, & leidner, 1998). trust improves collaboration and knowledge sharing (alani, marczak, prikladnicki, & redmiles, 2013; henttonen & blomqvist, 2005; jimenez et al., 2017), coordination, performance (joshi, lazarova, & liao, 2009; lukić & vračar, 2018), and is overall a critical success factor for vts (verburg, bosch-sijtsema, & vartiainen, 2013). hence, vt leaders need to continuously make efforts to create, reinforce, and maintain trust between the members of their teams as well as between themselves and their team members (e.g., alani, marczak, et al., 2013; hambley, o’neill, & kline, 2007; henttonen & blomqvist, 2005; hertel, geister, & konradt, 2005; joshi et al., 2009; liao, 2017; lukić & vračar, 2018; staples, 1997; thomas & bostrom, 2008; zander, zettinig, & mäkelä, 2013). however, even though trust has become one of the most studied variables in the vts literature (gilson, maynard, jones young, vartiainen, & hakonen, 2015), researchers and practitioners are limited by a startling lack of clarity about what trust is in this context, due to the myriad ways it has been conceptualized and studied in vts research. taken together, research suggests that vts will be a fixture in the workplace of the future and that trust may serve as an important mechanism for overcoming challenges associated with technology use, communications, and team diversity. thus, a better understanding of how to facilitate vt functioning through trust will be critical for both hrm professionals and is managers in guiding the workforce of the future in a more efficient and effective way. against this backdrop, the major aims of this paper are two-fold. the primary goal is to identify the current state of research on trust in vts, develop a conceptual model that integrates research findings and serves as an initial guide to hrm practitioners and vt managers, and provide a roadmap for future research in this domain. the secondary goal of this paper is to encourage conversations across disciplines to provide fresh insights into the role of trust in vts and how hrm practitioners and is managers can best facilitate its development. australasian journal of information systems hacker, johnson, saunders & thayer 2019, vol 23, research article trust in virtual teams 3 to this end, we conduct a systematic theoretical review (ortiz de guinea & paré, 2017; paré, trudel, jaana, & kitsiou, 2015) of 124 articles from the disparate, multidisciplinary literature on trust in vts. given the current state of the literature, this review is both timely and needed. previous reviews have separately considered vts (e.g., chatfield et al., 2014; gibbs, sivunen, & boyraz, 2017; gilson et al., 2015; hertel et al., 2005; martins et al., 2004; powell, piccoli, & ives, 2004), team virtuality (e.g., kirkman & mathieu, 2005), interpersonal trust (e.g., lewicki, tomlinson, & gillespie, 2006), and definitions of trust (mcknight & chervany, 2000). even though seminal articles on trust in vts (i.e., jarvenpaa et al., 1998; jarvenpaa & leidner, 1998) have been cited over 6,000 times and these citations have been growing exponentially over the last decade, we know of no other broad-based, multidisciplinary review that looks at the phenomenon of trust in vts. this suggests research and practice would benefit from a synthesis and recommendations to guide future scholarly work as well as hr management decision-making. in this paper, we review and integrate the is, computer science, communications, management, and psychology literatures on trust in vts. in particular, the comprehensive multidisciplinary review was used to develop an integrative conceptual model of the current nomological network on trust in vts. it identifies knowledge gaps and ways to incorporate different conceptualizations emerging from the review. to move the science and practice of vts forward, we offer new directions for scholarly research, insights into potential synergies across disciplines, and practical guidance for managing a workforce of the future whose members increasingly work in vts and face various technology challenges. 2 key definitions to provide a foundation for our integrative, multidisciplinary review and our discussion of theoretical and practical recommendations, we first begin by defining key concepts, including: trust, teams and vts, and virtuality. 2.1 trust interpersonal trust has been conceptualized in a variety of ways across the literature (mcknight, cummings, & chervany, 1998). it is generally considered the expectations of another’s conduct and/or an acceptance of vulnerability (e.g., mcallister, 1995; rousseau, sitkin, burt, & camerer, 1998). according to our review, the most common definition of trust utilized is mayer and colleagues’ (1995) “the willingness of a party to be vulnerable to the actions of another party based on the expectation that the other will perform a particular action important to the trustor, irrespective of the ability to monitor or control that other party” (pg. 712). more recent studies use the definition proposed by lewicki, mcallister & bies (1998, pg. 439): “confident, positive expectations about the conduct of another”, which shares similarities to mayer and colleagues’ view. the clustering around a set of definitions is encouraging; however, scholars have been embracing this broad definition when more specific conceptualizations may be more theoretically appropriate (see section 4.1). 2.2 teams and virtual teams a team is defined as a group of “individuals interacting adaptively, interdependently, and dynamically towards a common and valued goal” (salas, burke, & cannon-bowers, 2000, pg. 341). teams vary across a range of dimensions (e.g., task interdependence, role structure, lifespan) (wildman et al., 2012). as a particular type of team, vts are frequently defined as “groups of australasian journal of information systems hacker, johnson, saunders & thayer 2019, vol 23, research article trust in virtual teams 4 geographically and/organizationally dispersed co-workers that are assembled using a combination of telecommunications and information technologies to accomplish an organizational task” (townsend, demarie, & hendrickson, 1998, pg. 18). the vt literature has roots in the is discipline’s systematic studies of vt precursors called group decision support systems (gdss) or group support systems (gss). a gdss (gss) is an interactive system that combines communication, computer, and decision technologies to support communication, coordination processes and information processing when formulating and solving unstructured problems in group meetings (maier, 2007, pg. 277). concomitant streams of research on gdss and gss were conducted by faculty and students at university of minnesota and the university of arizona, respectively, as far back as the 1980s and 1990s. as a specific example, jay nunamaker and colleagues/students summarized twelve years of gss lab and field studies conducted at the university of arizona during that time period (nunamaker, briggs, mittleman, vogel, & pierre, 1996). early vts were categorized using a single, identifiable phenomenon (i.e., whether or not they used an information and communication technology (ict) or whether or not the teams were geographically separated), but more recent vt studies have incorporated a more nuanced, multi-dimensional view of what it takes to make a team more or less virtual. for example, the commonly-cited definition by townsend and colleagues (1998) focuses on the geographic and organizational aspects of vts, two of the three most common boundaries mentioned in definitions of vts (martins et al., 2004). we use a more encompassing definition of vts: “teams whose members use technology to varying degrees in working across locational, temporal, and relational boundaries to accomplish an interdependent task” (martins et al., 2004, pg. 808). thus, technology is at the heart of our definition of vts. 2.3 virtuality virtuality is the concept that determines whether a team is more or less virtual (gibson & gibbs, 2006). we define virtuality as a multi-faceted higher-order construct encompassing a team’s independent identifying dimensions that signify degrees of dispersion or discontinuity. different researchers hold separate views as to what dispersions and discontinuities constitute the identifying dimensions. for example, discontinuities can be barriers to vts such as geographic separation, cultural differences, time zone differences, and organizational membership which contribute to many problems in vt functioning (chudoba, wynn, lu, & watson-manheim, 2005; lu, watson-manheim, chudoba, & wynn, 2006). some more common views of types of dispersions are: geographic dispersion, temporal dispersion, and structural dispersion (o’leary & cummings, 2007); geographic dispersion, electronic dependence, dynamic structural arrangements, and national diversity (gibson & gibbs, 2006); location, distance and time (jimenez et al., 2017); and the degree of face-to-face contact (fiol & o’connor, 2005). even though most views of virtuality include a geographic dimension, kirkman & mathieu (2005, pg. 702) dispute using it as an identifier and instead focus solely on three technology dimensions: “(a) the extent to which team members use virtual tools to coordinate and execute team processes…(b) the amount of informational value provided by such tools, and (c) the synchronicity of team member virtual interaction.” while there is considerable debate about the specific identifying dimensions, there is also considerable overlap among them. furthermore, most vt researchers now agree that dimensions are combined in some way to create a composite construct that indicates the degree of virtuality in the team and reflects how technology helps or impedes its performance. australasian journal of information systems hacker, johnson, saunders & thayer 2019, vol 23, research article trust in virtual teams 5 3 approach of the literature review to assess the literature on trust in vts, we adopted a systematic theoretical review approach (ortiz de guinea & paré, 2017; paré et al., 2015). systematic theoretical reviews “synthesize the literature often bringing different streams of research together but they go a step further in developing a theoretical framework or model” (ortiz de guinea & paré, 2017, pg. 75). our search strategy is called an “exhaustive coverage with selective citation” (ortiz de guinea & paré, 2017, pg. 79) meaning that we attempted to look at a relatively complete census of relevant published conceptual and empirical articles considering both trust and vt, but only discussed a selected sample of works (ortiz de guinea & paré, 2017; paré et al., 2015). we recognized that it was impractical to capture all articles at the intersection of two large literatures, so the theoretical review approach seemed appropriate in this regard. our search was systematic in that we adopted a structured approach to searching, analysing, evaluating, and synthesizing research (e.g., kitchenham, 2004). guided by a specific research question, systematic reviews involve a documented and reproducible review process and lead to a comprehensive overview of relevant literature (e.g., okoli & schabram, 2010). in particular, we developed a strategy for defining the sources of the search (i.e., databases instead of journals), selecting search terms for our keyword searches, combining them with search operators, using the appropriate search fields, and coding the articles we ultimately selected (vom brocke et al., 2015). in the following, details regarding the literature search and coding are provided. 3.1 literature search to review the “state of the science” and gather a broad range of articles discussing trust in vts, an extensive, systematic search of about the last decade of research (2003-2015) was conducted within databases representing disciplines where these articles were likely to be found: proquest/abi inform, ebscohost, google scholar, acm digital library, and ieee xplore databases. we decided that a search of databases would be broader than one based on specific journals (vom brocke et al., 2015), especially considering the possibility that we might have missed selecting journals from some of the multiple disciplines whose researchers are exploring the vt phenomenon. additionally, we conducted a backwards search (vom brocke et al., 2015; webster & watson, 2002) on key trust in vts articles using google scholar (e.g., jarvenpaa et al., 1998; jarvenpaa & leidner, 1998). we queried for the terms (“virtual team” or “distributed team”) and (“trust” or “swift trust”). “swift trust” was selected as a keyword because it is a type of trust that seems to be particularly associated with vts. in all, this search returned more than 17,000 potential articles. abstracts, titles, and keywords are among the most frequently used search fields (vom brocke et al., 2015). hence, we used these three search fields to look for references to trust in vts, resulting in an initial set of 750 relevant articles. subsequently, articles were excluded if they were less than five pages, were an extended abstract or foreword, in a language other than english, or did not meaningfully speak to trust in vts. we considered it to be a meaningful treatment if trust was in the article’s model or theoretical framework, hypotheses, or propositions, or if it was discussed in major article sections (i.e., not just a paragraph). this resulted in a set of 188 articles. after a more extensive reading of each article, a subset of these articles was determined to not meaningfully speak to trust in vt. during this step, at least two members of the author team assessed the articles for relevance to the literature review goals (vom brocke et al., 2015). we did not assess them for quality since some argue that “appraisals of quality may not be necessary” (paré et al., 2015, pg. 189), especially since “quality criteria from australasian journal of information systems hacker, johnson, saunders & thayer 2019, vol 23, research article trust in virtual teams 6 other disciplines…are often not applicable to is research” (vom brocke et al., 2015, pg. 211). the selection process resulted in a final set of 124 articles (see appendix a for a summary). 3.2 coding for each of the 124 articles in the review sample, we coded study characteristics (study design, study setting, participant type), trust conceptualizations (swift trust, general trust, time-based trust, multi-faceted trust, dispositional trust, institutional trust), use of vt technology (i.e., face-to-face only, videoconferencing/phone, email/chat, mixed), dimensions of virtuality (geographic distance, organization, time zone, team dispersion, national culture), technology capabilities, processes, antecedents, moderators, and consequences of trust. we also extracted the journal’s discipline (management, psychology, information systems, computer science, and communication). to ensure a high-quality coding procedure, at least two members of the author team coded each article and differences were discussed to achieve agreement as recommended by saldaña (2015). 4 literature review and synthesis most studies in the trust in vts literature adopt a variance approach, meaning they predominantly focus on studying relationships among independent and dependent variables (burton-jones, mclean, & monod, 2015). accordingly, following past reviews (e.g., narayanan, zane, & kemmerer, 2011) and based on the findings of our review, we present an integrative model (figure 1) that incorporates three broad categories of factors: antecedents, consequences, and moderators. figure 1. integrated conceptual model of the nomological network of trust in vts we additionally consider the reciprocal relationships among these factors, as well as technological capabilities and processes. in the following, we first present overall trends from the literature. then, we summarize findings about components of our integrative model. technology capabilities task-technology fit media synchronicity system design antecedents member attributes cultural distance expertise team structure goals roles size hierarchy formalization team management leadership behaviors leadership characteristics interpersonal relationships familiarity conflict processes conflict resolution communication knowledge sharing & exchange team development consequences performance individual performance team performance quality of solution overcoming technology failure learning effectiveness member well-being satisfaction morale member support commitment team identification power shifts cohesion turnover intentions trust moderators virtuality culture task type moderators virtuality functional diversity task interdependence australasian journal of information systems hacker, johnson, saunders & thayer 2019, vol 23, research article trust in virtual teams 7 4.1 general research trends our broad review of the literature on trust in vts suggests that researchers in is as well as in other disciplines often focus on different perspectives of trust. for example, the vt literature often views trust as a static or unidimensional concept; however, we argue that trust should be studied as a dynamic, evolving construct that changes form as vts progress. further, we propose that different forms of trust support, and are supported by, such vt processes as communication and knowledge sharing. we also show that the literature as a whole fails to employ a consistent view of virtuality in vt functioning. the level of a team’s virtuality (or what it takes to make teams virtual) is dependent on technology and affects the trust development and vt development process. in addition, the study of trust and virtuality in the vt literature remains somewhat fragmented across a number of disciplines. unfortunately, researchers from many disciplines are looking at different isolated aspects of trust and virtuality, making it difficult to compare the findings across disciplines. it is time for researchers to create a more integrated picture of the complex components of this picture. indeed, because conceptual reviews that bridge disciplinary backgrounds have the strongest potential for contribution (short, 2009), our multidisciplinary approach presents a more integrated understanding of the complex components of trust in vt. by more fully conceptualizing trust and consistently defining and operationalizing virtuality, it may be possible to compare findings across disciplines and more fully understand trust in vt in a variety of contexts. we now discuss these trends in more detail. 4.1.1 multidisciplinary investigation and conceptualizations of trust the study of trust in vts is inherently multidisciplinary (see table 1). though the is discipline has most studied this phenomenon (48 studies), interest in computer science and communication disciplines has been growing, and management journals steadily published a large number of the papers identified (46 in total). different disciplines conceptualize the trust construct in diverse ways. although the difference between these types of trust may seem nuanced and specific, the trust literature generally considers each as a different concept (e.g., lewicki & bunker, 1995; mayer et al., 1995; mcknight et al., 1998). that is, something like swift trust is a separate construct than multifaceted trust despite the fact that they share the trust name, and, thus, the distinctions are important. through our review, we uncovered six main conceptualizations of trust. these conceptualizations are different ways of looking at and understanding trust and include swift trust (i.e., trust that is built quickly based on surface-level cues rather than interaction or knowledge of the trustee), as well as generalized trust (i.e., a single, unidimensional construct reflecting willingness to accept vulnerability to risk), time-based trust (i.e., static trust vs. trust that is dynamic and evolves), multi-faceted trust (i.e., trust as a multidimensional construct, such that individuals can be trusted in particular ways and for particular behaviours), dispositional trust (i.e., trust as a function of a pre-existing characteristic of the trustor), and institution-based trust (i.e., trust that is based on norms and rules created by institutions that guide individual behaviour). for more information on the definitions of these constructs and example citations, see appendix b. from a conceptual standpoint, there are interesting trends in terms of how each discipline has treated the trust construct, as indicated in table 1. the time-based conceptualizations of trust tend to explore the dynamic and temporal characteristics of the vt environment. such dynamic conceptualizations are few in number and are more frequently found in the australasian journal of information systems hacker, johnson, saunders & thayer 2019, vol 23, research article trust in virtual teams 8 management and computer science literatures. additionally, although the term was first coined by management researchers (meyerson, weick, & kramer, 1996) and subsequently explored by several psychology and management researchers (e.g., bosch-sijtsema, 2007), most swift trust studies are from is and computer science disciplines. this review is one opportunity to disseminate findings to other disciplines. it is also important to note that all disciplines have underexplored dispositional and institution-based trust in vts. more work needs to be done on these forms of trust. generalized trust swift trust time-based trust multifaceted trust dispositional trust institutionbased trust total n % n % n % n % n % n % n communication 7 11% 3 12% 3 13% 2 6% 1 14% 2 20% 18 computer science 18 27% 8 32% 7 29% 9 26% 0 0% 0 0% 42 information systems 16 24% 6 24% 6 25% 11 32% 3 43% 5 50% 47 psychology 8 12% 3 12% 1 4% 2 6% 0 0% 0 0% 14 management 17 26% 5 20% 7 29% 10 29% 3 43% 3 30% 45 total 66 100% 25 100% 24 100% 34 100% 7 100% 10 100% note. theoretical and empirical manuscripts included. some papers included multiple conceptualizations and are therefore represented more than once. table 1. trust conceptualization by discipline of journal 4.1.2 nature of trust overall, the view of trust in empirical vt studies is not as evolved as the conceptualizations of trust described in the broader theoretical writings about trust. that is, regardless of discipline, the approach to studying trust has been, at times, inconsistent with the underlying theorizing. first, the majority of the reviewed research focuses on trust as a generalized, static construct (see table 2) even though more nuanced, dynamic views have been introduced in the broader trust literature. in a vt context, teams are viewed as constantly changing and evolving (kozlowski, chao, grand, braun, & kuljanin, 2013). the importance of team development suggests that interest in a temporal view incorporating time-based trust should be drastically increasing rather than stagnating, as it currently is. second, theoretical work has shown a multi-faceted, nuanced view of trust. yet, the rapid growth of generalized trust studies calls into question the impact theoretical work in the broader trust literature has had so far in informing the conduct of empirical studies about trust in vts. third, our review shows that more than a third of swift trust studies and the majority of time-based trust studies utilized intact student teams, calling into question if the methods are reflective of theory. these intact teams may not be accurately capturing the developmental process of trust nor swift trust as it exists from the team’s inception. consequently, researchers may not be appropriately operationalizing the testing of relationships with theory. in summary, the literature on trust in vts to date explores trust by relying on static and generalized views of trust rather than richer, dynamic views of trust that may be more theoretically appropriate. australasian journal of information systems hacker, johnson, saunders & thayer 2019, vol 23, research article trust in virtual teams 9 2003-2008 2009-2015 2003-2015 n % n % n % generalized trust 25 37% 41 42% 66 40% swift trust 13 19% 12 12% 25 15% time-based trust 11 16% 13 13% 24 14% multi-faceted trust 12 18% 22 22% 34 20% dispositional trust 3 4% 4 4% 7 4% institution-based trust 4 6% 6 6% 10 6% total 68 100% 98 100% 166 100% note. theoretical and empirical manuscripts included. some papers included multiple conceptualizations and are therefore represented more than once. table 2. trust conceptualization over time 4.1.3 virtuality vts are classified on the basis of their virtuality. in addition to its conceptualization, virtuality’s operationalization has evolved over the past ten years. initially, when researchers simply viewed teams as entirely virtual or face-to-face, virtuality was measured by whether or not technology supported the team. as noted in appendix c, virtuality was often measured in this way or the measurement was not specified at all. new measures of virtuality reflect the nuanced, complex view of virtuality. for instance, o’leary & cummings (2007) and chudoba and colleagues (2005) developed indices to measure virtuality that permit researchers to document and measure the vt characteristics as well as study their impact with more precision. the findings of multiple studies could be compared more easily if the same indices of virtuality were employed. despite efforts to develop indices, only eight studies have measured the dimensions of virtuality (bierly, stark, & kessler, 2009; joshi et al., 2009; lojeski, reilly, & dominick, 2006; lu et al., 2006; morgan, paucar-caceres, & wright, 2014; plotnick, hiltz, & ocker, 2011; staples & webster, 2008; verburg et al., 2013). hoch & kozlowski (2014) argued that not only must the dimensions of virtuality be incorporated into a composite operationalization, but that the precise contribution of each component should be determined. in summary, many of the reviewed studies view virtuality as the mere presence or absence of technology. only recently have more nuanced views of virtuality been developed, yet few researchers employ these operationalizations. 4.2 antecedents of trust more than half of the identified papers investigated the antecedents of trust formation and maintenance in vts (see appendix a). given the importance of trust in interpersonal relationships, what creates or hinders trust represents a major issue in the research on vts. we categorize these factors under member attributes (cultural distance, expertise), team structure (goals and roles, size, hierarchy, formalization), team management (leadership behaviours, leader characteristics), and interpersonal relations (familiarity, conflict). trust appears to be created by knowledge of other team members’ expertise (e.g., dorairaj & noble, 2013). further, trust is commonly found in teams with shared or aligned goals (e.g., verburg et al., 2013), transformational leaders (e.g., eom, 2009; joshi et al., 2009), transactional leaders (e.g., eom, 2009), participative leaders (muganda & pillay, 2013) and well-defined goals as well as teams in organizations with more formalization (al-ani, bietz, et al., 2013). on the australasian journal of information systems hacker, johnson, saunders & thayer 2019, vol 23, research article trust in virtual teams 10 other hand, cultural distance (al-ani, bietz, et al., 2013; al-ani, marczak, et al., 2013), a larger team size (al-ani, marczak, et al., 2013; al-ani & redmiles, 2009a, 2009b), unprincipled vt leaders (e.g., al-ani, marczak, et al., 2013), and conflict (e.g., bierly et al., 2009) serve as barriers to creating trust in vts. specific findings about antecedents of trust in vt supporting these statements may be found in appendix d. 4.3 consequences of trust fewer extant studies look at consequences of trust in vts as compared to antecedents. for the purposes of this review, we followed accepted team frameworks and categorized consequences into the team categories of performance (relations between the group and the environmental context), member well-being (the development and maintenance of the group as a system), and member support (ways in which the individual is embedded within the group) (mcgrath, 1991). trust has been positively related to a number of types of performance including learning effectiveness (e.g., edwards & sridhar, 2005), creative problem solving (e.g., murthy, rodriguez, & lewis, 2013), individual member performance and team performance (e.g., sarker, ahuja, sarker, & kirkeby, 2011), especially as reported by the team members themselves (e.g., peters & karren, 2009). member well-being in the form of higher satisfaction and morale have also been linked to higher levels of trust (e.g., derosa, hantula, kock, & d’arcy, 2004). research also suggests that member support in terms of cooperation is made possible by trust (e.g., kanawattanachai & yoo, 2007), though it is often more difficult to achieve in vts (e.g., bierly et al., 2009). trust is also linked to greater cohesion and less turnover in vts (e.g., derosa et al., 2004). a unique consequence that has been studied in the trust in vts literature is overcoming failures due to technology use. a major benefit, but also a problem with vts, is their reliance on various forms of technology. inevitably, these forms of technology can fail and interrupt operations. trust has been studied as a central part in vts overcoming technological difficulties (gaan, 2012). when trust is high, team members provide the benefit of doubt to other members when the technology fails (gaan, 2012). for example, if something gets wrongly posted or sent through the technology, other members are not as offended. further, theory y leadership has been found to facilitate technology adoption which what in turn is linked to increased trust and better team outcomes (thomas & bostrom, 2008). in addition to findings about antecedents, appendix d also describes the findings about the consequences of trust in vts. 4.4 moderators in contrast to the antecedents and consequences of trust, there is less information about the moderators of trust in vts that strengthen, weaken, or nullify the relationships mentioned above. approximately one fifth of the articles study a wide range of moderating relationships (i.e., 24 of 124). the literature, as shown in appendix a, suggests that culture, task type, functional diversity, and task interdependence represent important moderating factors for how trust is formed and translated into outcomes (eom, 2009; furumo & pearson, 2006; mukherjee, hanlon, kedia, & srivastava, 2012; peters & karren, 2009). only four articles suggest that virtuality is a moderator. however, research designs of virtuality based on the mere presence or absence of a technology like email may limit confidence in these findings. while important, moderating effects are generally underexplored, especially type of vt and task type (hertel et al., 2005). australasian journal of information systems hacker, johnson, saunders & thayer 2019, vol 23, research article trust in virtual teams 11 4.4.1 technology capabilities a range of technologies support vt communication and knowledge sharing (e.g., email, video conferencing). these technologies vary in the extent of their synchronicity and ability to convey cues, indicating aspects of the technology affect relationships in the trust in vt area (dennis & valacich, 1999). most articles do not explain differences in findings based upon technology used, with the exception of three approaches. first, the task-technology fit approach seeks to match the task with the technology (goodhue & thompson, 1995; zigurs & buckland, 1998). research in the trust in vts literature has relied on this approach to suggest things like technology fit enhances manager’s choice of leadership styles (thomas & bostrom, 2008). the second draws upon media synchronicity theory (dennis & valacich, 1999) which proposes that the functionality of a communication medium can influence how an individual performs a task (maruping & agarwal, 2004). third, the design science approach proposes developing innovative artefacts to support trust building in vts (wu, zhao, vassileva, sun, & fan, 2014). for example, based on a design science approach, wu and colleagues (2014) compared wiki systems to optimize the design and implementation of systems handling team conflict and motivating collaborative learning. additionally, tarmizi and colleagues (2007) used the design science approach to design, develop, and test the feasibility of collaborative process engineering in virtual settings. in all, these approaches unearth the finding that the type of technology is not uniformly better or worse. the technology, rather, needs to properly fit the situation and members of the team. for example, technologies that transmit visual and voice cues are most related to trust during early stages (maruping & agarwal, 2004). additionally, design science can more fully inform the proper design and implementation for technology in each context (gregor & hevner, 2013; hevner, march, park, & ram, 2004). technological tools are important, yet are a poorly studied variable in this literature. 4.4.2 processes in our sample, some processes such as communication, conflict resolution, knowledge sharing and exchange were discussed, but in a static manner. that is, most articles in our sample examine processes using a variance approach. they seek to predict or explain levels or degrees of trust based on the values of other variables, including variables measuring specific processes such as communication and knowledge sharing. although such processes are seen as collections of pivotal actions, they are rarely treated as a dynamic process. instead, most trust in vt studies consider them as either an antecedent of trust (e.g., coppola, hiltz, & rotter, 2004) or an outcome of trust (e.g., he & paul, 2008). these studies show that transferring knowledge using face-to-face communication is positively associated with trust (e.g., dorairaj, noble, & malik, 2012; hambley et al., 2007) and that communication with technologies that most approximate face-to-face interaction foster trust since it allows for the transmission of social cues and other forms of knowledge (e.g., dorairaj et al., 2012; hambley et al., 2007). moving to consequences, scholars have shown that trust positively impacts knowledge sharing (he & paul, 2008; mital, israel, & agarwal, 2010; paul & he, 2012; staples & webster, 2008), information acceptance (al-ani, wilensky, redmiles, & simmons, 2011), and overall knowledge transfer levels (kuo & yu, 2009; monalisa et al., 2008; newell, david, & chand, 2007a, 2007b). yet, knowledge sharing is an ongoing process that evolves as trust also evolves. considering it as both an antecedent and consequence of trust leads to confusion in how to properly operationalize the construct. australasian journal of information systems hacker, johnson, saunders & thayer 2019, vol 23, research article trust in virtual teams 12 only a handful of studies adopt a process approach in which processes evolve together over time and can be captured in an overall temporal model. most of these employ stage models from the management literature such as tuckman’s model of team development (corbitt, gardiner, & wright, 2004; furst, reeves, rosen, & blackburn, 2004; tseng & ku, 2011), gersick’s (1988, 1989) theory of punctuated equilibrium (kanawattanachai & yoo, 2007; mcnab, basoglu, sarker, & yu, 2012), or lewicki and bunker’s (1995, 1996) stages of multiphase trust (gwebu, wang, & troutt, 2007; kuo & yu, 2009) to describe the relevant processes and conditions that lead to the development of trust in vts over time. others develop their own stages unique to vts (henttonen & blomqvist, 2005; hertel et al., 2005; suprateek sarker & sahay, 2003; zander et al., 2013). in these stage models, vts form and proceed through distinct periods, which represent a gradual and logical development whereby teams finally achieve effective goal fulfillment. trust development, in turn, can correspond to distinct stages of team development in a vt. in these studies, trust development and team development tend to run in parallel and few studies explain or test how these impact each other. for example, in kuo & yu (2009), swift trust emerges as the team starts its project; calculus and knowledgebased trust emerge, gradually peak at the project midpoint and then wane; and a low-level of identification trust is displayed by team members at the project’s end (see appendix c). the stage models around which the trust in vt literature have coalesced have faced much criticism. some weaknesses include: a) group actions typically proceed in iterative cycles not in a linear order (fisher, 1970; scheidel & crowell, 1964), b) there are many possible sequences of actions through which goals get accomplished in groups, not just one (poole, 1981, 1983), and c) how groups conduct actions to accomplish goals is not a gradual and sequential process (gersick, 1988). 5 discussion based on our review, we present an integrated conceptual model of the nomological network of trust in vts (figure 1) that identifies antecedents, consequences, processes, technology capabilities and moderators of trust in vts. this model represents the current state of the literature. while many organizations are already relying on vts, emerging new forms of work, among them remote work and digital nomadism, will lead to “traditional” and “new types” of vts to become even more prevalent in the workplace of the future. consequently, team management, which is suggested as an antecedent of trust in vts in our model, and specifically vt leaders, will have to find new strategies to create and reinforce trust in their teams so as to achieve and maintain team effectiveness. while advanced technological solutions supporting the work of vts (e.g., by facilitating digital presence) are continuously released, building trust in vts requires digital fluency (e.g., knowing when to use which tool) and effective (digital) leadership (colbert, yee, & george, 2016; henttonen & blomqvist, 2005; thomas & bostrom, 2008; zander et al., 2013). against this backdrop, in the following sections, we discuss future research that could address shortcomings in the existing literature and that could help effectively deal with issues in the workplace of the future. additionally, to provide fresh insights into this area of study, we conclude with potential synergies across different disciplines dealing with trust in vts and provide recommendations for the management of vts in the changing new world of work. australasian journal of information systems hacker, johnson, saunders & thayer 2019, vol 23, research article trust in virtual teams 13 5.1 future research to address gaps first, the focus on the dynamic nature in developing trust in vts is lacking, representing a major gap. to transition from the static view found in much of the literature to a more dynamic view, it will require shifting from variance models that have been predominately used to process models (markus & robey, 1988; mohr, 1982). variance models view constructs as independent variables (i.e., leadership behaviours, formalization) that cause change in the levels or degrees of dependent variables (i.e., trust) (van de ven & poole, 2005). these models assume that the basic character of the dependent variable is not changed and that the temporal sequence for triggering changes in the independent variable is immaterial (van de ven & poole, 2005). process models, in contrast, assume that the basic nature of the dependent variable (i.e., trust) may change. for example, swift trust may be manifest at the beginning of a vt project whereas identification trust may emerge at the end. process models recognize the complexity among events to account for temporal connections and the dynamic nature of processes (marks, mathieu, & zaccaro, 2001). more specifically, processes are conceptualized as interdependent actions that transform inputs (antecedents) to outputs (consequences) (marlow, lacerenza, & salas, 2017). they represent behaviours or actions (e.g., monitoring team-mates, motivating the team), which can be contrasted with emergent states, or properties of the team that arise through the dynamic team process interactions (e.g., collective efficacy, cohesion). within this view, trust is seen as an emergent state that supports team processes and other important emergent states (marlow et al, 2017). further, process models focus on change over time and are interested in sequences of events (burton-jones et al., 2015) and the patterns of influence that must be applied to a development process (van de ven & poole, 2005). that is, a process model would describe trust development as a process encompassed by a set of actions (e.g., communication processes) that produces forms of trust (i.e., calculus trust, knowledge-based trust) as emergent states which further support team processes and emergent states. within the view of process models, future studies could study technologies as intertwined with organizational processes or investigate how specific technologies or their features impact team processes (gibbs et al., 2017) which affect trust states. second, our review suggests that is researchers adopt only a limited number of conceptualizations of trust (e.g., focusing only on swift trust or generalized trust), resulting in a fragmented understanding of trust. as explained in more detail in our cross-disciplinary discussion (section 5.2), future research should explore how different conceptualizations of trust interact with each other, especially across time. third, virtuality is sometimes conceptualized merely as the presence or absence of certain technologies. we encourage future researchers to pay particular attention to how they think about and operationalize virtuality. doing so can enrich our understanding of the inconsistent findings about trust in vts. for example, joshi et al. (2009) found that inspirational (transformational) leadership is more important for team performance in vts with higher degrees of virtuality as compared to teams with lower degrees of virtuality. joshi and colleagues’ (2009) operationalization of virtuality took into account multiple types of dispersion. hambley et al. (2007) also looked at virtuality as a moderator, but found no moderating relationship between transformational leadership and team performance. however, for them virtuality was operationalized in terms of the nature of technology use (e.g., face-to-face only without any technology, videoconferencing and chat), as was common at the time. was the lack of significant findings for hambley et al. (2007) because there really is no relationship or because virtuality was measured poorly? virtuality is a multidimensional australasian journal of information systems hacker, johnson, saunders & thayer 2019, vol 23, research article trust in virtual teams 14 construct. if across disciplines we can adopt one, or only a few, approaches to measuring these multiple dimensions of virtuality, we all will be in a better position to compare results and understand inconsistencies. fourth and finally, our review suggests that the existing literature may not be matching study design to theory: many scholars are using the wrong types of teams to study the different aspects of trust. for instance, if the primary interest is swift trust, researchers should investigate newly forming teams, as existing teams have likely passed this stage of trust development. similarly, if the primary interest is in studying the development of time-based trust, researchers should employ longitudinal studies in long-term vts and rely less heavily on student teams. in addition to longitudinal studies, multiple methods may need to be employed to gain insights about the development of various types of trust. 5.2 a cross-disciplinary discussion with the multi-disciplinary nature of our review, we hope that this effort starts a conversation across disconnected disciplines to improve the conceptualizations of trust and virtuality. scholars are missing important contributions from other disciplines and we hope this distance between conversations can begin to close. for example, while psychology researchers recently framed a new and major contribution by operationalizing virtuality to include cultural differences (hoch & kozlowski, 2014), is researchers have proposed a cultural dimension to virtuality for more than a decade (e.g., chudoba et al., 2005; watson‐manheim, chudoba, & crowston, 2002). is research findings about how cultural differences impact trust (al-ani, bietz, et al., 2013; al-ani & redmiles, 2009a, 2009b; al-ani, redmiles, et al., 2013; gaan, 2012; krebs, hobman, & bordia, 2006; newell et al., 2007b; paul & he, 2012), vt processes, and outcomes are many (e.g., pinjani & palvia, 2013; scott, 2013). further, psychology researchers’ views of what makes a team virtual (i.e., teams with distant members who use email to communicate are compared to teams whose members are collocated) can be informed by the is discipline’s view of technology and virtuality. the is discipline has assumed a leading role in exploring how technology affects team members. thus, psychology researchers can benefit from utilizing existing is research to better frame their study of vts. conversely, is researchers need to explore the dynamic, multi-dimensional aspects of trust that are discussed in management and psychology journals. below are several sample synergies across disciplines that highlight strengths that other researchers can leverage. it should be noted that our comments about disciplinary contributions are based on the discipline in which a particular article is published. we recognize that some cross-disciplinary publishing (e.g., psychologists publishing in management journals, is researchers publishing in psychology journals) exists. this crossdisciplinary publishing can be viewed as healthy for cross-fertilizing ideas, and we hope it continues. the sample synergies below are by no means comprehensive but rather a starting point to further cross-disciplinary conversations. 5.2.1 is swift trust informing research on different kinds of teams? is researchers have largely focused on swift trust to inform research on vts. however, swift trust was initially proposed for quick starting temporary teams (meyerson et al., 1996). swift trust is by no means constrained to only virtual environments, yet virtual environments have dominated its investigation. the rich empirical results from is researchers can inform other kinds of team management that will be common in the work of the future. for example, there australasian journal of information systems hacker, johnson, saunders & thayer 2019, vol 23, research article trust in virtual teams 15 is a growing research body on swift starting action teams – highly interdependent teams of skilled individuals completing demanding and time-pressured projects – which have been proposed by psychology researchers as a unique type of team that faces a host of challenges not seen in traditional teams (wildman et al., 2012). however, the research does not yet provide a clear picture of how trust develops in these environments. the is research on swift trust can provide much insight. for example, is swift trust as fragile and temporary in these settings as in virtual environments? can swift trust account for the starting level of trust in swift starting action teams? 5.2.2 management’s research to inform an understanding of ‘team’ trust management research has several streams that can be applied to conceptualizing and studying ‘team’ trust. management researchers have recently started to investigate the dispersion and accuracy of trust judgments, as well as multi-level trust. trust is, at its nature, bidirectional where parties need to trust and be trusted when engaging in mutually beneficial exchanges (e.g., korsgaard, brower, & lester, 2015). members at different organizational levels may have different perspectives on the kind of trust in a particular relationship. several have pointed to a weakness of the trust literature as almost exclusively focusing on the one-sided trust of the trustor (fulmer & gelfand, 2012). some can levy this same claim on the trust in vts research. trust in vts researchers can directly address this weakness by exploring trust as a bidirectional sequence that is networked among team members. is this process of creating team trust transitive in nature such that when two team members develop trust it can be transferred to a third member? or are there synergies in vt contexts that allow the development of a shared, team trust? it may be that the variance of trust perceptions is a stronger driver of team processes and outcomes. trust in vts researchers can leverage reciprocal, mutual, and asymmetric trust models to explore the trust development process in vts (tomlinson, dineen, & lewicki, 2009). teams researchers as a whole have not adopted this perspective (e.g., korsgaard et al., 2015), and these trust accuracy perceptions are potentially even more important in higher-virtuality team environments. vt researchers of trust can also draw from management research that considers how trust can be conceptualized at both the individual and team levels, as well as how trust may be aggregated from the individual to team level (fulmer & gelfand, 2012). 5.2.3 impact of technology use on trust development in higher-virtuality teams is-based media synchronicity theory (dennis & valacich, 1999; dennis, fuller, & valacich, 2008) suggests that communication technologies can be placed along a continuum of low to high synchronicity based on a number of factors (i.e., transmission velocity, symbol sets, parallelism, rehearsability, and reprocessability). technologies that are high in synchronicity can build higher levels of trust (dorairaj et al., 2012; hambley et al., 2007) and converge on meaning (dennis et al., 2008; maruping & agarwal, 2004). high synchronicity technologies that facilitate the convergence of meaning are important for quickly moving beyond swift trust to deeper forms of trust. this is because initial swift trust judgments give way to verification and perceptions of shared meaning (dennis et al., 2008). once teams evolve into deeper forms of trust, different aspects of the technologies that are used become important. for example, in knowledge-based trust, a type of time-based trust (see appendix b), team members gain deep knowledge about other team members and are better able to anticipate their responses and reactions (lewicki et al., 2006). as members acquire a deep knowledge of the others in their vt, they can move past communication failures by more easily anticipating what the other australasian journal of information systems hacker, johnson, saunders & thayer 2019, vol 23, research article trust in virtual teams 16 team member had meant, even when technologies that are low in synchronicity are used. all disciplines could consider media synchronicity as a means of improving deficiencies in highervirtuality teams. kirkman and mathieu’s (2005) view of virtuality based on technology that is employed in the management literature may further help enrich this line of study. 5.3 managing virtual teams in the technology-driven workplace of the future though vts have been prevalent in the workplace, many challenges remain in managing them effectively. our review helped surface ways of building trust to deal with some of these challenges. in this section we describe the guidelines we found, as well as challenges related to vts that we anticipate in the workplace of the future. 5.3.1 creating and maintaining trust over a vt’s lifespan by using virtual technologies given the ubiquity of vts, it is important to find ways of making them more effective in future workplaces. building trust in vts certainly seems to be a viable way of doing so. as shown in our review above, vts with higher levels of trust produce high quality products (e.g., altschuller & benbunan-fich, 2010; he & paul, 2008; jarvenpaa, shaw, & staples, 2004) and are more creative (murthy et al., 2013). further, vts with participative (henttonen & blomqvist, 2005; muganda & pillay, 2013) and inspirational leadership (joshi et al., 2009) are reported to have higher performance. our review of the literature further suggests that timing is critical in building trust in vts. when no other option is available, managers of vts must rely on swift trust. it is encouraging that teams can start with some form of trust. however, managers should take steps early in the life of their vts to develop deeper forms of trust. one option is to arrange face-to-face team building meetings where the team members share information about their training and abilities to move to deeper forms of trust (zander et al., 2013). beyond this, managers could allow team members access to this information through a company wiki or internal website. in this regard, using an internal social networking site, i.e. an enterprise social network (esn), might be particularly suitable to reinforce trust development processes in vts by facilitating the building of a shared context (riemer & scifleet, 2012). following updates and discussing matters of interest, employees become aware of happenings in the organization and get to know each other. in the course of these interactions, a shared background emerges that can be considered as an important prerequisite for trust. also, technologies such as shared visualizations (weakley & edmonds, 2004) or the application of agile techniques (karagoz, korthaus, & augar, 2016) facilitate collaboration, knowledge sharing, and trust between members of distributed teams. managers of vts should also turn to team training explicitly designed to build trust in their teams (lukić & vračar, 2018). team training can address team breakdowns and expand team capabilities. providing teams opportunities to a) learn teamwork-focused knowledge, skills, and attitudes (as compared to task-focused training), b) practice working together, and c) receive feedback for improving team interactions can provide teams with a deeper understanding of teammates for facilitating trust development, and create a “toolbox” for overcoming challenges associated with virtuality. individualized training may also be helpful. team members can benefit from one-on-one coaching from their team leaders about their performance and the team’s mission (watkins, 2013). and, team leaders can be trained on managing vts, especially culturally diverse vts. it appears that self-identified leaders of vts believe they are better prepared to lead culturally-diverse teams than do those who are australasian journal of information systems hacker, johnson, saunders & thayer 2019, vol 23, research article trust in virtual teams 17 members of their teams. that is, a high percentage of self-identified leaders (i.e., 96%) rate themselves as either effective or highly effective in managing across cultures and countries, yet 58% of respondents indicate that global team leaders are not adequately prepared to lead multicultural teams (rw3 culturewizard, 2016). once vts are formed, managers can alter schedules to ensure synchronous meetings, perhaps via skype or videoconferencing. the time of these remote synchronous meetings can be rotated across time zones (rw3 culturewizard, 2016). these meetings can be held on a regular basis to develop time-based trust. interestingly, too many meetings may be a symptom of lack of trust since different levels of managers and employees may not trust one another (al-ani, marczak, et al., 2013). because technology is so intrinsic to the definition of vts, vt managers can leverage the technology to their benefit. still, the timing of technology use is important. for example, technologies high in synchronicity may be particularly important early in the trust development cycle while technologies low in synchronicity may be more important later in the cycle. in order to enhance team effectiveness, managers of vts may need to develop and leverage electronic transactive memory systems (tmss), or systems for storing, encoding and retrieving knowledge related to team activities, and member skills and expertise. beyond supporting trust formation in early stages of vts, the above-mentioned esn platforms can be used for this purpose. later in trust development cycles, higher-virtuality teams may hold an advantage over low-virtuality teams as they have electronic tmss. higher-virtuality teams can automatically store discussions about activities and decisions in messages streams, e.g., within dedicated groups of an esn, email conversations, or other electronic storage. of course, teams also may have tmss residing as overlapping mental representations in the minds of its members. when teams have well-developed tmss, the team members have stored knowledge about how to perform necessary tasks that have been performed before and whom to contact to do subtasks. kanawattanachai and yoo (2007) suggest that forms of trust work together with team members’ knowledge of others’ expertise to promote highly developed tmss and more effective team performance. thus, managers should develop norms encouraging the use of esn messages or emails to confirm team decisions and to set up wikis, blackboards or other electronic repositories for key documents to enhance the team’s tms. by properly designing when team members can use which technology, vt managers can improve the trust development process in highly virtual teams. 5.3.2 dealing with workplace trends affecting trust formation in vts as the workplace changes, managers must consider the impact of trends on trust development in vts. in particular, increased use and emphasis on multiteam systems, work/life balance, and technologies like robotics are important trends for hr practitioners and vt managers to consider. multiteam systems involve multiple specialized groups working together interdependently to accomplish organizations’ goals or complete is programs (firth, hollenbeck, miles, ilgen, & barnes, 2015; mathieu, marks, & zaccaro, 2001). such systems add another level of complexity to trust building – amplifying the challenges to coordination and communication (van knippenberg, dahlander, haas, & george, 2015). in terms of interorganizational relationships, lane & lum (2011) and flemming & low (2007) argue that trust and a shared business understanding are key drivers of partnership quality in it outsourcing arrangements. as for multiteam systems, it is likely that there are different levels of trust involved (i.e., interpersonal, group, australasian journal of information systems hacker, johnson, saunders & thayer 2019, vol 23, research article trust in virtual teams 18 organizational) and trust at the multiteam system level likely has some characteristics that differ from trust at other levels. another current topic of interest and one that is likely to assume even greater significance for members of the workforce of the future is work-life balance. work-life balance is the state when work-family conflict (which occurs when the demands from one set of roles interferes with the completion of the demands from another set of roles) is considered to be at an acceptable level (rutkowski & saunders, 2018). however, work-family conflict is common if one is always working. modern ict have virtually “eliminated the conventional workday and have made time and distance immaterial to the execution of many organizational tasks” (ragu-nathan, tarafdar, ragu-nathan, & tu, 2008, pg. 418). on the positive side, ict support vts. however, they also can blur the boundaries between work and family and leave team members feeling that they must available to vt teammates on a 24/7 basis. one question that might be explored is the extent to which team members can build trust in other members if they perceive that they must constantly respond to communications from them even if it conflicts with their family time. governments have tried to ameliorate work/life imbalance with campaigns and legislation. for example, the french government passed legislation in 2017 which “requires companies with more than 50 employees to establish hours when staff should not send or answer emails” (morris, 2017), thus granting employees the right to “switch off” after hours. however, as well-meaning as it is meant to be, the french legislation could create considerable conflict for employees who work on global vts. unless the members have a considerable level of trust, the employees who are quite distant from france might not appreciate their french teammates “switching off” their calls, emails or other communications. here it is important for team leaders must build trust among the remote members to reduce their stress while keeping them in the communication loop (staples, 1997). finally, another trend that is changing the nature of the workforce of the future is its inclusion of robots. the cost of robots is dropping concomitantly with the increase in their abilities to perform a wide range of tasks with great efficiency, accuracy, and reliability. industrial robots that are being used in “dangerous, dirty or dull” jobs represent a forecasted market of $37 billion by 2018 (goodman, 2015). in addition, robots are becoming “collaborative” to the extent that they are now safer for their human colleagues to work with and relationships between owners, workers and robots on the assembly line are carefully choreographed (tingley, 2017). it is conceivable that robots might join vts and eventually provide “high level functions as fullfledged team members” (hancock et al., 2011, pg. 517). in ensuing human-robot collaborative teams, humans on the team must learn to trust their robotic teammates to protect their welfare and interests. they need to accept the robot as it was designed to perform, but must make sure that the level of trust in the robot is appropriate (oleson, billings, kocsis, chen, & hancock, 2011). future research should explore what factors impact the development and maintenance on human-robot vts. 5.4 limitations in conducting our review, we were challenged with balancing the time available to us with the need to have an adequate level of coverage (vom brocke et al., 2015). thus, we developed rules for what would be included and excluded from our review and systematically applied them. we decided to base our selection on the title, abstract and keywords, and not the whole text (vom brocke et al., 2015) and on a search of databases and not just journals (webster & watson, 2002). not searching the full text for key terms or not focusing on specific journals, such as the australasian journal of information systems hacker, johnson, saunders & thayer 2019, vol 23, research article trust in virtual teams 19 senior scholar basket of journals, no doubt meant that we missed some relevant articles, albeit, we did so systematically. another limitation that could be levied against our study is that we only searched on ‘trust’ and ‘swift trust’, but not other types of trust. however, our search for articles on ‘trust’ unearthed studies with other types of trust such as the five other types of trust that we ended up discussing. further, we did not search on other possible names of vts such as geographic-dispersed teams, remote team or computer-mediated teams, though articles with these titles often surfaced on our search and we included them. 5.5 conclusion overall, organizations are increasingly relying on vts despite the challenges found in virtual structures, which are characterized by high levels of technology use. this change in work represents an opportunity for research to inform practice. research exploring trust in this growing organizational form has, similarly, exploded. we, accordingly, conduct a multidisciplinary theoretical review of the trust in vt literature to show its role in addressing these vt challenges and those that are likely to gain prominence in the workforce of the future. our integrated model of trust in vts helps identify important antecedents, consequences, and other constructs that are critical in building and experiencing trust in vts. despite the powerful nature of trust, our review uncovered several important weaknesses in the literature that must be overcome to move forward – namely, problematic conceptualizations of trust and virtuality. we highlight these weaknesses and suggest how research across disciplines can be used to start healthy multidisciplinary dialogue and to create better conceptualizations for theorizing and conducting research on trust in vts. we hope that researchers will take up our call to incorporate the improved conceptualizations of trust and virtuality in order not only to improve the prospects of the work-force today effectively using technology in their vts, but also the workforce of the future. references al-ani, b., bietz, m. j., wang, y., trainer, e., koehne, b., marczak, s., redmiles, d., prikladnicki, r. (2013). globally distributed system developers. in proceedings of the 2013 conference on computer supported cooperative work cscw ’13 (p. 563). new york, new york, usa: acm press. https://doi.org/10.1145/2441776.2441840 al-ani, b., marczak, s., prikladnicki, r., & redmiles, d. (2013). revisiting the factors that engender trust of global systems engineers. in 2013 ieee 8th international conference on global software engineering (pp. 31–40). ieee. https://doi.org/10.1109/icgse.2013.13 al-ani, b., marczak, s., redmiles, d., & prikladnicki, r. (2014). facilitating contagion trust through tools in global systems engineering teams. information and software technology, 56(3), 309–320. https://doi.org/10.1016/j.infsof.2013.11.001 al-ani, b., & redmiles, d. (2009a). in strangers we trust? findings of an empirical study of distributed teams. in 2009 fourth ieee international conference on global software engineering (pp. 121–130). ieee. https://doi.org/10.1109/icgse.2009.20 al-ani, b., & redmiles, d. (2009b). supporting trust in distributed teams through continuous coordination. ieee software. https://doi.org/10.1109/ms.2009.115 al-ani, b., & redmiles, d. (2009c). trust in distributed teams: support through continuous coordination. ieee software, 26(6), 35–40. https://doi.org/10.1109/ms.2009.192 australasian journal of information systems hacker, johnson, saunders & thayer 2019, vol 23, research article trust in virtual teams 20 al-ani, b., redmiles, d., de souza, c. r. b., prikladnicki, r., marczak, s., lanubile, f., & calefato, f. (2013). trust in virtual teams. in proceedings of the 2013 conference on computer supported cooperative work companion cscw ’13 (pp. 301–306). new york, new york, usa: acm press. https://doi.org/10.1145/2441955.2442029 al-ani, b., wilensky, h., redmiles, d., & simmons, e. (2011). an understanding of the role of trust in knowledge seeking and acceptance practices in distributed development teams. in 2011 ieee sixth international conference on global software engineering (pp. 25– 34). ieee. https://doi.org/10.1109/icgse.2011.25 altschuller, s., & benbunan-fich, r. (2010). trust, performance, and the communication process in ad hoc decision-making virtual teams. journal of computer-mediated communication, 16(1), 27–47. https://doi.org/10.1111/j.1083-6101.2010.01529.x altschuller, s., & benbunan-fich, r. (2013). the pursuit of trust in ad hoc virtual teams: how much electronic portrayal is too much? european journal of information systems, 22(6), 619–636. https://doi.org/10.1057/ejis.2012.39 aubert, b. a., & kelsey, b. l. (2003). further understanding of trust and performance in virtual teams. small group research, 34(5), 575–618. https://doi.org/10.1177/1046496403256011 bartelt, v. l., & dennis, a. r. (2014). nature and nurture: the impact of automaticity and the structuration of communication on virtual team behavior and performance. mis quarterly, 38(2), 521–538. https://doi.org/10.25300/misq/2014/38.2.09 baskerville, r., & nandhakumar, j. (2007). activating and perpetuating virtual teams: now that we’re mobile, where do we go? ieee transactions on professional communication, 50(1), 17–34. https://doi.org/10.1109/tpc.2006.890849 beranek, p. m. (2005). a comparison of relational and trust training techniques for virtual team communication: how much training is enough? in proceedings of the 38th annual h awaii international conference on system sciences . ieee. https://doi.org/10.1109/hicss.2005.5 bierly, p. e., stark, e. m., & kessler, e. h. (2009). the moderating effects of virtuality on the antecedents and outcome of npd team trust. journal of product innovation management, 26(5), 551–565. https://doi.org/10.1111/j.1540-5885.2009.00680.x bosch-sijtsema, p. (2007). the impact of individual expectations and expectation conflicts on virtual teams. group & organization management, 32(3), 358–388. https://doi.org/10.1177/1059601106286881 brahm, t., & kunze, f. (2012). the role of trust climate in virtual teams. journal of managerial psychology, 27(6), 595–614. https://doi.org/10.1108/02683941211252446 breu, k., & hemingway, c. j. (2004). making organisations virtual: the hidden cost of distributed teams. journal of information technology, 19(3), 191–202. https://doi.org/10.1057/palgrave.jit.2000018 brown, h. g., poole, m. s., & rodgers, t. l. (2004). interpersonal traits, complementarity, and trust in virtual collaboration. journal of management information systems, 20(4), 115–138. https://doi.org/10.1080/07421222.2004.11045785 australasian journal of information systems hacker, johnson, saunders & thayer 2019, vol 23, research article trust in virtual teams 21 burton-jones, a., mclean, e. r., & monod, e. (2015). theoretical perspectives in is research: from variance and process to conceptual latitude and conceptual fit. european journal of information systems, 24(6), 664–679. https://doi.org/10.1057/ejis.2014.31 chang, h. h., chuang, s.-s., & chao, s. h. (2011). determinants of cultural adaptation, communication quality, and trust in virtual teams’ performance. total quality management & business excellence, 22(3), 305–329. https://doi.org/10.1080/14783363.2010.532319 chang, h. h., hung, c.-j., & hsieh, h.-w. (2014). virtual teams: cultural adaptation, communication quality, and interpersonal trust. total quality management & business excellence, 25(11–12), 1318–1335. https://doi.org/10.1080/14783363.2012.704274 chatfield, a. t., shlemoon, v. n., redublado, w., & darbyshire, g. (2014). creating value through virtual teams: a current literature review. australasian journal of information systems, 18(3). https://doi.org/10.3127/ajis.v18i3.1104 cheng, x., macaulay, l., & zarifis, a. (2013). modeling individual trust development in computer mediated collaboration: a comparison of approaches. computers in human behavior, 29(4), 1733–1741. https://doi.org/http://dx.doi.org/10.1016/j.chb.2013.02.018 cheng, x., nolan, t., & macaulay, l. (2013). don’t give up the community: a viewpoint of trust development in online collaboration. information technology & people, 26(3), 298–318. https://doi.org/10.1108/itp-10-2012-0116 chudoba, k. m., wynn, e., lu, m., & watson-manheim, m. b. (2005). how virtual are we? measuring virtuality and understanding its impact in a global organization. information systems journal, 15(4), 279–306. https://doi.org/10.1111/j.1365-2575.2005.00200.x clark, w. r., clark, l. a., & crossley, k. (2010). developing multidimensional trust without touch in virtual teams. marketing management journal, 20(1), 177–193. colbert, a., yee, n., & george, g. (2016). the digital workforce and the workplace of the future. academy of management journal, 59(3), 731–739. https://doi.org/10.5465/amj.2016.4003 collins, n., & chou, y.-m. (2013). building team trust: a study in the asian context. journal of american business review, cambridge, 1(2), 181–188. coppola, n. w., hiltz, s. r., & rotter, n. g. (2004). building trust in virtual teams. ieee transactions on professional communication, 47(2), 95–104. https://doi.org/10.1109/tpc.2004.828203 corbitt, g., gardiner, l. r., & wright, l. k. (2004). a comparison of team developmental stages, trust and performance for virtual versus face-to-face teams. in 37th annual hawaii international conference on system sciences, 2004. proceedings of the. ieee. https://doi.org/10.1109/hicss.2004.1265157 crisp, c. b., & jarvenpaa, s. l. (2013). swift trust in global virtual teams. journal of personnel psychology, 12(1), 45–56. https://doi.org/10.1027/1866-5888/a000075 curşeu, p. l., schalk, r., & wessel, i. (2008). how do virtual teams process information? a literature review and implications for management. journal of managerial psychology, 23(6), 628–652. https://doi.org/10.1108/02683940810894729 australasian journal of information systems hacker, johnson, saunders & thayer 2019, vol 23, research article trust in virtual teams 22 d. olson, j., d. appunn, f., a. mcallister, c., k. walters, k., & grinnell, l. (2014). webcams and virtual teams: an impact model. team performance management: an international journal, 20(3/4), 148–177. https://doi.org/10.1108/tpm-05-2013-0013 de laat, p. b. (2005). trusting virtual trust. ethics and information technology, 7(3), 167–180. https://doi.org/10.1007/s10676-006-0002-6 dennis, a. r., robert, l. p., curtis, a. m., kowalczyk, s. t., & hasty, b. k. (2012). research note —trust is in the eye of the beholder: a vignette study of postevent behavioral controls’ effects on individual trust in virtual teams. information systems research, 23(2), 546–558. https://doi.org/10.1287/isre.1110.0364 dennis, a. r., & valacich, j. s. (1999). rethinking media richness: towards a theory of media synchronicity. in proceedings of the 32nd hawaii international conference on system sciencesvolume 1 volume 1 (pp. 1017–1027). ieee. dennis, fuller, & valacich. (2008). media, tasks, and communication processes: a theory of media synchronicity. mis quarterly, 32(3), 575. https://doi.org/10.2307/25148857 derosa, d. m., hantula, d. a., kock, n., & d’arcy, j. (2004). trust and leadership in virtual teamwork: a media naturalness perspective. human resource management, 43(2–3), 219– 232. https://doi.org/10.1002/hrm.20016 dorairaj, s., & noble, j. (2013). agile software development with distributed teams: agility, distribution and trust. in 2013 agile conference (pp. 1–10). ieee. https://doi.org/10.1109/agile.2013.7 dorairaj, s., noble, j., & malik, p. (2012). understanding lack of trust in distributed agile teams: a grounded theory study. in 16th international conference on evaluation & assessment in software engineering (ease 2012) (pp. 81–90). https://doi.org/10.1049/ic.2012.0011 edwards, h. k., & sridhar, v. (2005). analysis of software requirements engineering exercises in a global virtual team setup. journal of global information management, 13(2), 21–41. https://doi.org/10.4018/jgim.2005040102 el khatib, v., trang, s., reimers, k., & kolbe, l. (2013). the role of motivational factors in distributed software development teams: an empirical investigation. in proceedings of the 21st european conference on information systems. eom, m. (2009). cross-cultural virtual team and its key antecedents to success. journal of applied business & economics, 10(1), 11–24. erez, m., lisak, a., harush, r., glikson, e., nouri, r., & shokef, e. (2013). going global: developing management students’ cultural intelligence and global identity in culturally diverse virtual teams. academy of management learning & education, 12(3), 330–355. https://doi.org/10.5465/amle.2012.0200 fiol, c. m., & o’connor, e. j. (2005). identification in face-to-face, hybrid, and pure virtual teams: untangling the contradictions. organization science, 16(1), 19–32. https://doi.org/10.1287/orsc.1040.0101 firth, b. m., hollenbeck, j. r., miles, j. e., ilgen, d. r., & barnes, c. m. (2015). same page, different books: extending representational gaps theory to enhance performance in australasian journal of information systems hacker, johnson, saunders & thayer 2019, vol 23, research article trust in virtual teams 23 multiteam systems. academy of management journal, 58(3), 813–835. https://doi.org/10.5465/amj.2013.0216 fisher, b. a. (1970). decision emergence: phases in group decision‐making. speech monographs, 37(1), 53–66. https://doi.org/10.1080/03637757009375649 flemming, r., & low, g. (2007). information systems outsourcing relationship model. australasian journal of information systems, 14(2). https://doi.org/10.3127/ajis.v14i2.472 fuller, c. m., marett, k., & twitchell, d. p. (2012). an examination of deception in virtual teams: effects of deception on task performance, mutuality, and trust. ieee transactions on professional communication, 55(1), 20–35. https://doi.org/10.1109/tpc.2011.2172731 fulmer, c. a., & gelfand, m. j. (2012). at what level (and in whom) we trust. journal of management, 38(4), 1167–1230. https://doi.org/10.1177/0149206312439327 furst, s. a., reeves, m., rosen, b., & blackburn, r. s. (2004). managing the life cycle of virtual teams. academy of management executive, 18(2), 6–20. https://doi.org/10.5465/ame.2004.13837468 furumo, k., & pearson, j. m. (2006). an empirical investigation of how trust, cohesion, and performance vary in virtual and face-to-face teams. in proceedings of the 39th annual hawaii international conference on system sciences (hicss’06) (vol. 00, p. 26c–26c). ieee. https://doi.org/10.1109/hicss.2006.51 furumo, k., & pearson, j. m. (2007). gender-based communication styles, trust, and satisfaction in virtual teams. journal of information, information technology, and organizations, 2, 47–60. gaan, n. (2012). collaborative tools and virtual team effectiveness: an inductively derived approach in india’s software sector. decision, 39(1), 5–27. gao, s., guo, y., & chen, j. (2014). the performance of knowledge collaboration in virtual teams: an empirical study. international journal of multimedia and ubiquitous engineering, 9(8), 193–212. https://doi.org/10.14257/ijmue.2014.9.8.17 gartner. (2012). ppm & it governance summit [brochure]. retrieved january 13, 2017, from http://imagesrv.gartner.com/summits/docs/na/programmanagement/ppmit4_brochure.pdf germain, m.-l., & mcguire, d. (2014). the role of swift trust in virtual teams and implications for human resource development. advances in developing human resources, 16(3), 356–370. https://doi.org/10.1177/1523422314532097 gersick, c. j. g. (1988). time and transition in work teams: toward a new model of group development. academy of management journal, 31(1), 9–41. https://doi.org/10.2307/256496 gersick, c. j. g. (1989). marking time: predictable transitions in task groups. academy of management journal, 32(2), 274–309. https://doi.org/10.2307/256363 gibbs, j. l., sivunen, a., & boyraz, m. (2017). investigating the impacts of team type and design on virtual team processes. human resource management review. https://doi.org/10.1016/j.hrmr.2016.12.006 australasian journal of information systems hacker, johnson, saunders & thayer 2019, vol 23, research article trust in virtual teams 24 gibson, c. b., & gibbs, j. l. (2006). unpacking the concept of virtuality: the effects of geographic dispersion, electronic dependence, dynamic structure, and national diversity on team innovation. administrative science quarterly, 51(3), 451–495. https://doi.org/10.2189/asqu.51.3.451 gilson, l. l., maynard, m. t., jones young, n. c., vartiainen, m., & hakonen, m. (2015). virtual teams research: 10 years, 10 themes, and 10 opportunities. journal of management, 41(5), 1313–1337. https://doi.org/10.1177/0149206314559946 giustiniano, l., & bolici, f. (2012). organizational trust in a networked world. journal of information, communication and ethics in society, 10(3), 187–202. https://doi.org/10.1108/14779961211261076 goodhue, d. l., & thompson, r. l. (1995). task-technology fit and individual performance. mis quarterly, 19(2), 213–236. https://doi.org/10.2307/249689 goodman, m. (2015). future crimes: everything is connected, everyone is vulnerable and what we can do about it. london: bantam press. gregor, s., & hevner, a. r. (2013). positioning and presenting design science research for maximum impact. mis quarterly, 37(2), 337–355. https://doi.org/10.25300/misq/2013/37.2.01 gressgård, l. j. (2011). virtual team collaboration and innovation in organizations. team performance management: an international journal, 17(1/2), 102–119. https://doi.org/10.1108/13527591111114738 gwebu, k. l., wang, j., & troutt, m. d. (2007). a conceptual framework for understanding trust building and maintenance in virtual organizations. journal of information technology theory and application, 9(1), 43–63. haines, r. (2014). group development in virtual teams: an experimental reexamination. computers in human behavior, 39, 213–222. https://doi.org/10.1016/j.chb.2014.07.019 hambley, l. a., o’neill, t. a., & kline, t. j. b. (2007). virtual team leadership. international journal of e-collaboration, 3(1), 40–64. https://doi.org/10.4018/jec.2007010103 hancock, p. a., billings, d. r., schaefer, k. e., chen, j. y. c., de visser, e. j., & parasuraman, r. (2011). a meta-analysis of factors affecting trust in human-robot interaction. human factors: the journal of the human factors and ergonomics society, 53(5), 517–527. https://doi.org/10.1177/0018720811417254 he, f., & paul, s. (2008). an empirical investigation of the roles of outcome controls and psychological factors in collaboration technology supported virtual teams. in proceedings of the 41st annual hawaii international conference on system sciences (hicss 2008) (pp. 35–35). ieee. https://doi.org/10.1109/hicss.2008.51 henttonen, k., & blomqvist, k. (2005). managing distance in a global virtual team: the evolution of trust through technology-mediated relational communication. strategic change, 14(2), 107–119. https://doi.org/10.1002/jsc.714 hertel, g., geister, s., & konradt, u. (2005). managing virtual teams: a review of current empirical research. human resource management review, 15(1), 69–95. https://doi.org/10.1016/j.hrmr.2005.01.002 australasian journal of information systems hacker, johnson, saunders & thayer 2019, vol 23, research article trust in virtual teams 25 hevner, a. r., march, s. t., park, j., & ram, s. (2004). design science in information systems research. mis quarterly, 28(1), 75–105. hoch, j. e., & dulebohn, j. h. (2017). team personality composition, emergent leadership and shared leadership in virtual teams: a theoretical framework. human resource management review, 27(4), 678–693. https://doi.org/10.1016/j.hrmr.2016.12.012 hoch, j. e., & kozlowski, s. w. j. (2014). leading virtual teams: hierarchical leadership, structural supports, and shared team leadership. journal of applied psychology, 99(3), 390– 403. https://doi.org/10.1037/a0030264 hunsaker, p. l., & hunsaker, j. s. (2008). virtual teams: a leader’s guide. team performance management: an international journal, 14(1/2), 86–101. https://doi.org/10.1108/13527590810860221 jang, c.-y. (2013). facilitating trust in virtual teams: the role of awareness. advances in competitiveness research, 21(1–2), 61–77. jarvenpaa, s. l., knoll, k., & leidner, d. e. (1998). is anybody out there? antecedents of trust in global virtual teams. journal of management information systems, 14(4), 29–64. jarvenpaa, s. l., & leidner, d. e. (1998). communication and trust in global virtual teams. journal of computer-mediated communication, 3(4). https://doi.org/10.1111/j.10836101.1998.tb00080.x jarvenpaa, s. l., & leidner, d. e. (1999). communication and trust in global virtual teams. organization science, 10(6), 791–815. https://doi.org/10.1111/j.1083-6101.1998.tb00080.x jarvenpaa, s. l., shaw, t. r., & staples, d. s. (2004). toward contextualized theories of trust: the role of trust in global virtual teams. information systems research, 15(3), 250–267. https://doi.org/10.1287/isre.1040.0028 jensen, r. e. (2014). why closely coupled work matters in global software development. in proceedings of the 18th international conference on supporting group work group ’14 (pp. 24–34). new york, new york, usa: acm press. https://doi.org/10.1145/2660398.2660425 jimenez, a., boehe, d. m., taras, v., & caprar, d. v. (2017). working across boundaries: current and future perspectives on global virtual teams. journal of international management, 23(4), 341–349. https://doi.org/10.1016/j.intman.2017.05.001 joshi, a., lazarova, m. b., & liao, h. (2009). getting everyone on board: the role of inspirational leadership in geographically dispersed teams. organization science, 20(1), 240–252. https://doi.org/10.1287/orsc.1080.0383 kanawattanachai, & yoo. (2007). the impact of knowledge coordination on virtual team performance over time. mis quarterly, 31(4), 783–808. https://doi.org/10.2307/25148820 karagoz, y., korthaus, a., & augar, n. (2016). how do ict project managers manage project knowledge in the public sector? an empirical enquiry from the victorian public sector in australia. australasian journal of information systems, 20. https://doi.org/10.3127/ajis.v20i0.1313 keim, t., & weitzel, t. (2005). an integrated approach to online partnership building. in proceedings of the 38th annual hawaii international conference on system sciences. ieee. https://doi.org/10.1109/hicss.2005.92 australasian journal of information systems hacker, johnson, saunders & thayer 2019, vol 23, research article trust in virtual teams 26 keyzerman, y. (2003). trust in virtual teams. in ieee international professional communication conference, 2003. ipcc 2003. proceedings. (p. 10 pp.). ieee. https://doi.org/10.1109/ipcc.2003.1245520 kirkman, b. l., & mathieu, j. e. (2005). the dimensions and antecedents of team virtuality. journal of management, 31(5), 700–718. https://doi.org/10.1177/0149206305279113 kitchenham, b. (2004). procedures for performing systematic reviews (technical report). retrieved from http://www.inf.ufsc.br/~aldo.vw/kitchenham.pdf korsgaard, m. a., brower, h. h., & lester, s. w. (2015). it isn’t always mutual. journal of management, 41(1), 47–70. https://doi.org/10.1177/0149206314547521 kotlarsky, j., & oshri, i. (2005). social ties, knowledge sharing and successful collaboration in globally distributed system development projects. european journal of information systems, 14(1), 37–48. https://doi.org/10.1057/palgrave.ejis.3000520 kozlowski, s. w. j., chao, g. t., grand, j. a., braun, m. t., & kuljanin, g. (2013). advancing multilevel research design. organizational research methods, 16(4), 581–615. https://doi.org/10.1177/1094428113493119 krebs, s. a., hobman, e. v., & bordia, p. (2006). virtual teams and group member dissimilarity. small group research, 37(6), 721–741. https://doi.org/10.1177/1046496406294886 krumm, s., terwiel, k., & hertel, g. (2013). challenges in norm formation and adherence. journal of personnel psychology, 12(1), 33–44. https://doi.org/10.1027/1866-5888/a000077 ku, h.-y., tseng, h. w., & akarasriworn, c. (2013). collaboration factors, teamwork satisfaction, and student attitudes toward online collaborative learning. computers in human behavior, 29(3), 922–929. https://doi.org/10.1016/j.chb.2012.12.019 kuo, f., & yu, c. (2009). an exploratory study of trust dynamics in work-oriented virtual teams. journal of computer-mediated communication, 14(4), 823–854. https://doi.org/10.1111/j.1083-6101.2009.01472.x lane, m. s., & lum, w. h. (2011). examining client perceptions of partnership quality and the relationships between its dimensions in an it outsourcing relationship. australasian journal of information systems, 17(1). https://doi.org/10.3127/ajis.v17i1.525 lee‐kelley, l., crossman, a., & cannings, a. (2004). a social interaction approach to managing the “invisibles” of virtual teams. industrial management & data systems, 104(8), 650–657. https://doi.org/10.1108/02635570410561636 lee, c.-c., & chang, j.-w. (2013). does trust promote more teamwork? modeling online game players’ teamwork using team experience as a moderator. cyberpsychology, behavior, and social networking, 16(11), 813–819. https://doi.org/10.1089/cyber.2012.0461 lewicki, r. j., & bunker, b. b. (1995). trust in relationships: a model of development and decline. in b. b. b. j. z. rubin (ed.), conflict, cooperation, and justice: essays inspired by the work of morton deutsch (pp. 133–173). san francisco, ca, us: jossey-bass. lewicki, r. j., & bunker, b. b. (1996). developing and maintaining trust in work relationships. in r. m. kramer & t. r. tyler (eds.), trust in organizations: frontiers of australasian journal of information systems hacker, johnson, saunders & thayer 2019, vol 23, research article trust in virtual teams 27 theory and research (pp. 114–139). 2455 teller road, thousand oaks california 91320 united states: sage publications, inc. https://doi.org/10.4135/9781452243610.n7 lewicki, r. j., mcallister, d. j., & bies, r. j. (1998). trust and distrust: new relationships and realities. the academy of management review, 23(3), 438–458. lewicki, r. j., tomlinson, e. c., & gillespie, n. (2006). models of interpersonal trust development: theoretical approaches, empirical evidence, and future directions. journal of management, 32(6), 991–1022. https://doi.org/10.1177/0149206306294405 li, y., li, h., mädche, a., & rau, p.-l. p. (2012). are you a trustworthy partner in a crosscultural virtual environment? in proceedings of the 4th international conference on intercultural collaboration icic ’12 (pp. 87–96). new york, new york, usa: acm press. https://doi.org/10.1145/2160881.2160895 liao, c. (2017). leadership in virtual teams: a multilevel perspective. human resource management review. https://doi.org/10.1016/j.hrmr.2016.12.010 lind, m. (2007). collective team identification in temporary teams. in 2007 40th annual hawaii international conference on system sciences (hicss’07) (pp. 37–37). ieee. https://doi.org/10.1109/hicss.2007.132 liu, x., magjuka, r. j., & lee, s. (2008). an examination of the relationship among structure, trust, and conflict management styles in virtual teams. performance improvement quarterly, 21(1), 77–93. https://doi.org/10.1002/piq.20016 lojeski, k. s., reilly, r., & dominick, p. (2006). the role of virtual distance in innovation and success. in proceedings of the 39th annual hawaii international conference on system sciences (hicss’06). ieee. https://doi.org/10.1109/hicss.2006.484 lu, m., watson-manheim, m. b., chudoba, k. m., & wynn, e. (2006). virtuality and team performance: understanding the impact of variety of practices. journal of global information technology management, 9(1), 4–23. https://doi.org/10.1080/1097198x.2006.10856412 lukić, j., & vračar, m. (2018). building and nurturing trust among members in virtual project teams. strategic management, 23(3), 10–16. https://doi.org/10.5937/straman1803010l maier, r. (2007). knowledge management systems. berlin, heidelberg: springer berlin heidelberg. https://doi.org/10.1007/978-3-540-71408-8 majchrzak, a., malhotra, a., & john, r. (2005). perceived individual collaboration know-how development through information technology–enabled contextualization: evidence from distributed teams. information systems research, 16(1), 9–27. https://doi.org/10.1287/isre.1050.0044 marks, m. a., mathieu, j. e., & zaccaro, s. j. (2001). a temporally based framework and taxonomy of team processes. academy of management review, 26(3), 356–376 markus, m. l., & robey, d. (1988). information technology and organizational change: causal structure in theory and research. management science, 34(5), 583–598. https://doi.org/10.1287/mnsc.34.5.583 australasian journal of information systems hacker, johnson, saunders & thayer 2019, vol 23, research article trust in virtual teams 28 marlow, s. l., lacerenza, c. n., & salas, e. (2017). communication in virtual teams: a conceptual framework and research agenda. human resource management review. https://doi.org/10.1016/j.hrmr.2016.12.005 martins, l. l., gilson, l. l., & maynard, m. t. (2004). virtual teams: what do we know and where do we go from here? journal of management, 30(6), 805–835. https://doi.org/10.1016/j.jm.2004.05.002 maruping, l. m., & agarwal, r. (2004). managing team interpersonal processes through technology: a task-technology fit perspective. journal of applied psychology, 89(6), 975– 990. https://doi.org/10.1037/0021-9010.89.6.975 mathieu, j. e., marks, m. a., & zaccaro, s. j. (2001). multiteam systems. in d. s. ones, n. anderson, c. viswesvaran, & h. k. sinangil (eds.), handbook of industrial, work & organizational psychology volume 2: organizational psychology (pp. 289–313). thousand oaks, ca: sage publications ltd. http://doi.org/10.4135/9781848608368.n16 mayer, r., davis, j., & schoorman, f. (1995). an integrative model of organizational trust. the academy of management review, 20(3), 709–734. mcallister, d. j. (1995). affectand cognition-based trust as foundations for interpersonal cooperation in organizations. academy of management journal, 38(1), 24–59. https://doi.org/10.2307/256727 mcgrath, j. e. (1991). time, interaction, and performance (tip). small group research, 22(2), 147–174. https://doi.org/10.1177/1046496491222001 mchugh, o., conboy, k., & lang, m. (2012). agile practices: the impact on trust in software project teams. ieee software, 29(3), 71–76. https://doi.org/10.1109/ms.2011.118 mcknight, d. h., & chervany, n. l. (1996). the meanings of trust. retrieved from http://www.misrc.umn.edu/workingpapers/fullpapers/1996/9604_040100.pdf mcknight, d. h., & chervany, n. l. (2000). what is trust? a conceptual analysis and an interdisciplinary model. in proceedings of the 6th americas conference on information systems (pp. 827–833). mcknight, d. h., cummings, l. l., & chervany, n. l. (1998). initial trust formation in new organizational relationships. the academy of management review, 23(3), 473–490. mcnab, a. l., basoglu, k. a., sarker, s., & yu, y. (2012). evolution of cognitive trust in distributed software development teams: a punctuated equilibrium model. electronic markets, 22(1), 21–36. https://doi.org/10.1007/s12525-011-0081-z meyerson, d., weick, k. e., & kramer, r. m. (1996). swift trust and temporary groups. in r. m. kramer & t. r. tyler (eds.), trust in organizations: frontiers of theory and research (pp. 166–195). thousand oaks, ca, us: sage publications, inc. https://doi.org/10.4135/9781452243610.n9 mirel, b., & johnson, n. (2006). social determinants of preparing a cyber-infrastructure innovation for diffusion. technical communication quarterly, 15(3), 329–353. https://doi.org/10.1207/s15427625tcq1503_4 australasian journal of information systems hacker, johnson, saunders & thayer 2019, vol 23, research article trust in virtual teams 29 mital, m., israel, d., & agarwal, s. (2010). information exchange and information disclosure in social networking web sites. the learning organization, 17(6), 479–490. https://doi.org/10.1108/09696471011082349 mitchell, a., & zigurs, i. (2009). trust in virtual teams. acm sigmis database, 40(3), 61–83. https://doi.org/10.1145/1592401.1592407 mogan, s., & wang, w. (2007). a study into user perceptions of information sharing and trust in virtual teams. in proceedings of the 18th conference on hypertext and hypermedia ht ’07 (pp. 43–44). new york, new york, usa: acm press. https://doi.org/10.1145/1286240.1286254 mohr, l. b. (1982). explaining organizational behavior. san francisco: jossey-bass. monalisa, m., daim, t., mirani, f., dash, p., khamis, r., & bhusari, v. (2008). managing global design teams. research-technology management, 51(4), 48–59. https://doi.org/10.1080/08956308.2008.11657514 morgan, l., paucar-caceres, a., & wright, g. (2014). leading effective global virtual teams: the consequences of methods of communication. systemic practice and action research, 27(6), 607–624. https://doi.org/10.1007/s11213-014-9315-2 morris, d. z. (2017). new french law bars work email after hours. retrieved january 16, 2018, from http://fortune.com/2017/01/01/french-right-to-disconnect-law/ muganda, n., & pillay, k. (2013). forms of power, politics and leadership in asynchronous virtual project environment. international journal of managing projects in business, 6(3), 457–484. https://doi.org/10.1108/ijmpb-11-2011-0075 mukherjee, d., hanlon, s. c., kedia, b. l., & srivastava, p. (2012). organizational identification among global virtual team members. cross cultural management: an international journal, 19(4), 526–545. https://doi.org/10.1108/13527601211270002 murthy, d., rodriguez, a., & lewis, j. (2013). examining the formation of swift trust within a scientific global virtual team. in 2013 46th hawaii international conference on system sciences (pp. 353–362). ieee. https://doi.org/10.1109/hicss.2013.211 nandhakumar, j., & baskerville, r. (2006). durability of online teamworking: patterns of trust. information technology & people, 19(4), 371–389. https://doi.org/10.1108/09593840610718045 narayanan, v. k., zane, l. j., & kemmerer, b. (2011). the cognitive perspective in strategy: an integrative review. journal of management, 37(1), 305–351. https://doi.org/10.1177/0149206310383986 newell, s., david, g., & chand, d. (2007a). an analysis of trust among globally distributed work teams in an organizational setting. knowledge and process management, 14(3), 158– 168. https://doi.org/10.1002/kpm.284 newell, s., david, g., & chand, d. (2007b). exploring trust among globally distributed work teams. in 2007 40th annual hawaii international conference on system sciences (hicss’07) (p. 246c–246c). ieee. https://doi.org/10.1109/hicss.2007.221 nunamaker, j. f., briggs, r. o., mittleman, d. d., vogel, d. r., & pierre, b. a. (1996). lessons from a dozen years of group support systems research: a discussion of lab and field australasian journal of information systems hacker, johnson, saunders & thayer 2019, vol 23, research article trust in virtual teams 30 findings. journal of management information systems, 13(3), 163–207. https://doi.org/10.1080/07421222.1996.11518138 nyström, c. a., & asproth, v. (2013). virtual teams — support for technical communication? journal of organisational transformation & social change, 10(1), 64–80. https://doi.org/10.1179/1477963312z.0000000006 o’leary, & cummings. (2007). the spatial, temporal, and configurational characteristics of geographic dispersion in teams. mis quarterly, 31(3), 433. https://doi.org/10.2307/25148802 okoli, c., & schabram, k. (2010). a guide to conducting a systematic literature review of information systems research. working papers on information systems, 10(26), 1–51. https://doi.org/10.2139/ssrn.1954824 oleson, k. e., billings, d. r., kocsis, v., chen, j. y. c., & hancock, p. a. (2011). antecedents of trust in human-robot collaborations. in 2011 ieee international multi-disciplinary conference on cognitive methods in situation awareness and decision support (cogsima) (pp. 175–178). ieee. https://doi.org/10.1109/cogsima.2011.5753439 olson, j., appunn, f., walters, k., grinnell, l., & mcallister, c. (2012). the value of webcams for virtual teams. international journal of management & information systems, 16(2), 161– 171. https://doi.org/10.19030/ijmis.v16i2.6915 olson, j., & olson, l. (2012). virtual team trust: task, communication and sequence. team performance management: an international journal, 18(5/6), 256–276. https://doi.org/10.1108/13527591211251131 ortiz de guinea, a., & paré, g. (2017). what literature review type should i conduct? in the routledge companion to management information systems (pp. 73–82). london: routledge. https://doi.org/10.4324/9781315619361 pangil, f., & moi chan, j. (2014). the mediating effect of knowledge sharing on the relationship between trust and virtual team effectiveness. journal of knowledge management, 18(1), 92– 106. https://doi/10.1108/jkm-09-2013-0341 panteli, n., & duncan, e. (2004). trust and temporary virtual teams: alternative explanations and dramaturgical relationships. information technology & people, 17(4), 423–441. https://doi.org/10.1108/09593840410570276 panteli, n., & tucker, r. (2009). power and trust in global virtual teams. communications of the acm, 52(12), 113–115. https://doi.org/10.1145/1610252.1610282 paré, g., trudel, m.-c., jaana, m., & kitsiou, s. (2015). synthesizing information systems knowledge: a typology of literature reviews. information & management, 52(2), 183–199. https://doi.org/10.1016/j.im.2014.08.008 paul, s., & he, f. (2012). time pressure, cultural diversity, psychological factors, and information sharing in short duration virtual teams. in 2012 45th hawaii international conference on system sciences (pp. 149–158). ieee. https://doi.org/10.1109/hicss.2012.593 pearlson, k. e., saunders, c. s., & galletta, d. f. (2016). managing and using information systems: a strategic approach (6th ed.). new york, ny: wiley. australasian journal of information systems hacker, johnson, saunders & thayer 2019, vol 23, research article trust in virtual teams 31 peñarroja, v., orengo, v., zornoza, a., & hernández, a. (2013). the effects of virtuality level on task-related collaborative behaviors: the mediating role of team trust. computers in human behavior, 29(3), 967–974. https://doi.org/10.1016/j.chb.2012.12.020 peters, l. m., & karren, r. j. (2009). an examination of the roles of trust and functional diversity on virtual team performance ratings. group & organization management, 34(4), 479–504. https://doi.org/10.1177/1059601107312170 peters, l. m., & manz, c. c. (2007). identifying antecedents of virtual team collaboration. team performance management, 13(3/4), 117–129. https://doi.org/http://dx.doi.org/10.1108/13527590710759865 piccoli, g., ives, b., & piccoli, b. g. (2003). trust and the unintended effects of behavior control in virtual teams. mis quarterly, 27(3), 365–396. pinjani, p., & palvia, p. (2013). trust and knowledge sharing in diverse global virtual teams. information & management, 50(4), 144–153. https://doi.org/10.1016/j.im.2012.10.002 plotnick, l., hiltz, s. r., & ocker, r. j. (2011). trust over time and distance in global partially distributed teams. in 2011 44th hawaii international conference on system sciences (pp. 1– 10). ieee. https://doi.org/10.1109/hicss.2011.460 polzer, j. t., crisp, c. b., jarvenpaa, s. l., & kim, j. w. (2006). extending the faultline model to geographically dispersed teams: how colocated subgroups can impair group functioning. academy of management journal, 49(4), 679–692. https://doi.org/10.5465/amj.2006.22083024 poole, m. s. (1981). decision development in small groups i: a comparison of two models. communication monographs, 48(1), 1–24. https://doi.org/10.1080/03637758109376044 poole, m. s. (1983). decision development in small groups, iii: a multiple sequence model of group decision development 1. communication monographs, 50(4), 321–341. https://doi.org/10.1080/03637758309390173 powell, a., galvin, j., & piccoli, g. (2006). antecedents to team member commitment from near and far. information technology & people, 19(4), 299–322. https://doi.org/10.1108/09593840610718018 powell, a., piccoli, g., & ives, b. (2004). virtual teams. acm sigmis database, 35(1), 6–36. https://doi.org/10.1145/968464.968467 privman, r., hiltz, s. r., & wang, y. (2013). in-group (us) versus out-group (them) dynamics and effectiveness in partially distributed teams. ieee transactions on professional communication, 56(1), 33–49. https://doi.org/10.1109/tpc.2012.2237253 purvanova, r. k. (2013). the role of feeling known for team member outcomes in project teams. small group research, 44(3), 298–331. https://doi.org/10.1177/1046496413480244 purvanova, r. k. (2014). face-to-face versus virtual teams: what have we really learned? the psychologist-manager journal, 17(1), 2–29. https://doi.org/10.1037/mgr0000009 radcliffe, l. l., & schniederjans, m. j. (2003). trust evaluation: an ahp and multi‐objective programming approach. management decision, 41(6), 587–595. https://doi.org/10.1108/00251740310484867 australasian journal of information systems hacker, johnson, saunders & thayer 2019, vol 23, research article trust in virtual teams 32 ragu-nathan, t. s., tarafdar, m., ragu-nathan, b. s., & tu, q. (2008). the consequences of technostress for end users in organizations: conceptual development and empirical validation. information systems research, 19(4), 417–433. https://doi.org/10.1287/isre.1070.0165 rämö, h. (2004). moments of trust: temporal and spatial factors of trust in organizations. journal of managerial psychology, 19(8), 760–775. https://doi.org/10.1108/02683940410568248 remidez jr., h., stam, a., & laffey, j. m. (2007). web-based template-driven communication support systems. international journal of e-collaboration, 3(1), 65–73. https://doi.org/10.4018/jec.2007010104 riemer, k., & scifleet, p. (2012). enterprise social networking in knowledge-intensive work practices: a case study in a professional service firm. in proceedings of the 23rd australasian conference on information systems (pp. 1–12). geelong, australia. robert, l. p., denis, a. r., & hung, y.-t. c. (2009). individual swift trust and knowledgebased trust in face-to-face and virtual team members. journal of management information systems, 26(2), 241–279. https://doi.org/10.2753/mis0742-1222260210 rousseau, d. m., sitkin, s. b., burt, r. s., & camerer, c. (1998). not so different after all: a cross-discipline view of trust. academy of management review, 23(3), 393–404. https://doi.org/10.5465/amr.1998.926617 rusman, e., van bruggen, j., sloep, p., & koper, r. (2010). fostering trust in virtual project teams: towards a design framework grounded in a trustworthiness antecedents (twan) schema. international journal of human-computer studies, 68(11), 834–850. https://doi.org/10.1016/j.ijhcs.2010.07.003 rusman, e., van bruggen, j., sloep, p., valcke, m., & koper, r. (2013). the mind’s eye on personal profiles: a cognitive perspective on profile elements that inform initial trustworthiness assessments and social awareness in virtual project teams. computer supported cooperative work (cscw), 22(2–3), 159–179. https://doi.org/10.1007/s10606-0129171-5 rutkowski, a.-f., & saunders, c. s. (2018). emotional and cognitive overload: the dark side of information technology. london, uk: routledge. rw3 culturewizard. (2016). trends in global virtual teams: virtual teams survey report 2016. retrieved january 18, 2018, from http://cdn.culturewizard.com/pdf/trends_in_vt_report_4-17-2016.pdf salas, e., burke, c. s., & cannon-bowers, j. a. (2000). teamwork: emerging principles. international journal of management reviews, 2(4), 339–356. https://doi.org/10.1111/14682370.00046 saldaña, j. (2015). the coding manual for qualitative researchers (3rd ed.). london: sage publications ltd. sarker, s., ahuja, m., sarker, s., & kirkeby, s. (2011). the role of communication and trust in global virtual teams: a social network perspective. journal of management information systems, 28(1), 273–310. https://doi.org/10.2753/mis0742-1222280109 australasian journal of information systems hacker, johnson, saunders & thayer 2019, vol 23, research article trust in virtual teams 33 sarker, s., & sahay, s. (2003). understanding virtual team development: an interpretive study. journal of the association for information systems, 4(1), 1–38. sarker, s., valacich, j. s., & sarker, s. (2003). virtual team trust. information resources management journal, 16(2), 35–55. https://doi.org/10.4018/irmj.2003040103 saunders, c. s., & ahuja, m. k. (2006). are all distributed teams the same? differentiating between temporary and ongoing distributed teams. small group research, 37(6), 662– 700. https://doi.org/10.1177/1046496406294323 scheidel, t. m., & crowell, l. (1964). idea development in small discussion groups. quarterly journal of speech, 50(2), 140–145. https://doi.org/10.1080/00335636409382654 schiller, s. z., mennecke, b. e., nah, f. f.-h., & luse, a. (2014). institutional boundaries and trust of virtual teams in collaborative design: an experimental study in a virtual world environment. computers in human behavior, 35(0), 565–577. https://doi.org/10.1016/j.chb.2014.02.051 scott, m. e. (2013). “communicate through the roof”: a case study analysis of the communicative rules and resources of an effective global virtual team. communication quarterly, 61(3), 301–318. https://doi.org/10.1080/01463373.2013.776987 short, j. (2009). the art of writing a review article. journal of management, 35(6), 1312–1317. https://doi.org/10.1177/0149206309337489 skjerve, a. b., & rindahl, g. (2010). promoting trust between members of distributed teams. in 2010 ieee international conference on systems, man and cybernetics (pp. 1650–1658). ieee. https://doi.org/10.1109/icsmc.2010.5642403 staples, d. s. (1997). an investigation of information technology-enabled remote management and remote work issues. australasian journal of information systems, 4(2). https://doi.org/10.3127/ajis.v4i2.362 staples, d. s., & webster, j. (2008). exploring the effects of trust, task interdependence and virtualness on knowledge sharing in teams. information systems journal, 18(6), 617–640. https://doi.org/10.1111/j.1365-2575.2007.00244.x tarmizi, h., payne, m., noteboom, c., zhang, c., steinhauser, l., vreede, g.-j. de, & zigurs, i. (2007). collaboration engineering in distributed environments. e-service journal, 6(1), 76– 97. thomas, d., & bostrom, r. (2008). building trust and cooperation through technology adaptation in virtual teams: empirical field evidence. information systems management, 25(1), 45–56. https://doi.org/10.1080/10580530701777149 tingley, k. (2017). learning to love our robot co-workers. retrieved january 16, 2018, from https://www.nytimes.com/2017/02/23/magazine/learning-to-love-our-robot-coworkers.html tomlinson, e. c., dineen, b. r., & lewicki, r. j. (2009). trust congruence among integrative negotiators as a predictor of joint‐behavioral outcomes. international journal of conflict management, 20(2), 173–187. https://doi.org/10.1108/10444060910949621 townsend, a. m., demarie, s. m., & hendrickson, a. r. (1998). virtual teams: technology and the workplace of the future. the academy of management executive, 12(3), 17–29. australasian journal of information systems hacker, johnson, saunders & thayer 2019, vol 23, research article trust in virtual teams 34 tsai, y.-h., ma, h.-c., lin, c.-p., chiu, c.-k., & chen, s.-c. (2014). group social capital in virtual teaming contexts: a moderating role of positive affective tone in knowledge sharing. technological forecasting and social change, 86, 13–20. https://doi.org/10.1016/j.techfore.2013.08.015 tseng, h., & ku, h. (2011). the relationships between trust, performance, satisfaction, and development progressions among virtual teams. the quarterly review of distance education, 12(2), 81–94. van de ven, a. h., & poole, m. s. (2005). alternative approaches for studying organizational change. organization studies, 26(9), 1377–1404. https://doi.org/10.1177/0170840605056907 van der land, s. f., schouten, a. p., feldberg, f., huysman, m., & van den hooff, b. (2015). does avatar appearance matter? how team visual similarity and member-avatar similarity influence virtual team performance. human communication research, 41(1), 128–153. https://doi.org/10.1111/hcre.12044 van knippenberg, d., dahlander, l., haas, m. r., & george, g. (2015). information, attention, and decision making. academy of management journal, 58(3), 649–657. https://doi.org/10.5465/amj.2015.4003 verburg, r. m., bosch-sijtsema, p., & vartiainen, m. (2013). getting it done: critical success factors for project managers in virtual work settings. international journal of project management, 31(1), 68–79. https://doi.org/10.1016/j.ijproman.2012.04.005 vom brocke, j., simons, a., riemer, k., niehaves, b., plattfaut, r., & cleven, a. (2015). standing on the shoulders of giants: challenges and recommendations of literature search in information systems research. communications of the association for information systems, 37(1), 205–224. walther, j. b., & bunz, u. (2005). the rules of virtual groups: trust, liking, and performance in computer-mediated communication. journal of communication, 55(4), 828–846. https://doi.org/10.1111/j.1460-2466.2005.tb03025.x watkins, m. d. (2013). making virtual teams work: ten basic principles. harvard business review. retrieved from https://hbr.org/2013/06/making-virtual-teams-work-ten watson‐manheim, m. b., chudoba, k. m., & crowston, k. (2002). discontinuities and continuities: a new way to understand virtual work. information technology & people, 15(3), 191–209. https://doi.org/10.1108/09593840210444746 weakley, a., & edmonds, e. (2004). shared visualizations in support of distributed creative communities. australasian journal of information systems, 11(2). https://doi.org/10.3127/ajis.v11i2.130 webster, j., & watson, r. t. (2002). analyzing the past to prepare for the future: writing a literature review. mis quarterly, 26(2), 13–23. weimann, p., pollock, m., scott, e., & brown, i. (2013). enhancing team performance through tool use: how critical technology-related issues influence the performance of virtual project teams. ieee transactions on professional communication, 56(4), 332–353. https://doi.org/10.1109/tpc.2013.2287571 australasian journal of information systems hacker, johnson, saunders & thayer 2019, vol 23, research article trust in virtual teams 35 wildman, j. l., thayer, a. l., pavlas, d., salas, e., stewart, j. e., & howse, w. r. (2012). team knowledge research: emerging trends and critical needs. human factors: the journal of the human factors and ergonomics society, 54(1), 84–111. https://doi.org/10.1177/0018720811425365 wilson, j., crisp, c. b., & mortensen, m. (2013). extending construal-level theory to distributed groups: understanding the effects of virtuality. organization science, 24(2), 629–644. https://doi.org/10.1287/orsc.1120.0750 wu, k., zhao, y., vassileva, j., sun, x., & fan, z. (2014). how does awareness of task conflict motivate wiki-based collaborative learning? a design science approach. in proceedings of the 18th pacific asia conference on information systems. yusof, s. a. m., & zakaria, n. (2012). exploring the state of discipline on the formation of swift trust within global virtual teams. in 2012 45th hawaii international conference on system sciences (pp. 475–482). ieee. https://doi.org/10.1109/hicss.2012.272 zander, l., zettinig, p., & mäkelä, k. (2013). leading global virtual teams to success. organizational dynamics, 42(3), 228–237. https://doi.org/10.1016/j.orgdyn.2013.06.008 zaugg, h., & davies, r. s. (2013). communication skills to develop trusting relationships on global virtual engineering capstone teams. european journal of engineering education, 38(2), 228–233. https://doi.org/10.1080/03043797.2013.766678 zigurs, i., & buckland, b. k. (1998). a theory of task/technology fit and group support systems effectiveness. mis quarterly, 22(3), 313–334. https://doi.org/10.2307/249668 zucker, l. g. (1986). production of trust: institutional sources of economic structure, 18401920. research in organizational behavior, 8, 53–111. appendices the following appendices can be accessed via http://dx.doi.org/10.3127/ajis.v23i0.1757 appendix a. description of articles included in review appendix b. conceptualizations of trust appendix c. assessment of virtuality in empirical research appendix d. summary of findings about antecedents and consequences of trust copyright: © 2019 hacker, johnson, saunders & thayer. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems hacker, johnson, saunders & thayer 2019, vol 23, research article trust in virtual teams 36 paper title (paper title style) australasian journal of information systems soh, smith & dhillon 2022, vol 26, research on applied ethics the relationship between social capital & social media addiction 1 the relationship between social capital and social media addiction: the role of privacy self-efficacy franck soh university of north carolina, greensboro, usa f_sohnoume@uncg.edu kane smith university of north texas, denton, usa gurpreet dhillon university of north texas, denton, usa university of kwazulu-natal, durban, south africa abstract the rise of social media raises important ethical issues regarding social media user behaviors. this study seeks to investigate the determinants of social media addiction by focusing on social capital and privacy self-efficacy. we argue that social capital has a mixed association with social media addiction by highlighting the difference between social capital bonding and social capital bridging. notably, social media users differ in their usage purposes; as some build more bridges, others focus on bonding. moreover, we posit that the relationship between social capital and social media addiction is moderated by social media user privacy selfefficacy. we collected the data using a survey approach and the data was analyzed using covariance-based structural equation modeling. the findings support our hypotheses. first, we found that social media users with high bridging experience lesser social media addiction. those with high bonding have more social media addiction. second, social media users' privacy self-efficacy moderates the relationship between social capital and social media addiction. this occurs by reinforcing the negative association between social capital bridging and social media addiction and the positive association between social capital bonding and social media addiction. our findings provide important theoretical contributions and implications for practice. keywords: social media, addiction, self-efficacy, privacy, social capital, bridging, bonding. 1 introduction social media addiction is a psychological state of maladaptive dependence and compulsive use (turel, serenko, & giles, 2011). studies show that 39% of social media users reported being addicted in 2019 (statista, 2019). social media users spend on average 142 minutes in 2021 (armstrong, 2021). implications of social media addiction are significant. research suggests low work performance (xanidis & brignell, 2016), low life satisfaction (błachnio, przepiorka, & pantic, 2016), and high stress (thomée, eklöf, gustafsson, nilsson, & hagberg, 2007) to be some of the adverse outcomes. there is much confusion as to what constitutes social media addiction. at times social media addiction is considered in clinical terms and at other times in non-clinical terms. in this paper, drawing on andreassen, and pallesen (2014) work, we consider social media addiction from a non-clinical perspective. we define it as overly concerned and strongly motivated to devote vast amounts of time and energy to use social media, thus impairing other social and job-related activities. such behaviors are significant ethical concerns. we investigate the relationships between social capital bridging and bonding, australasian journal of information systems soh, smith & dhillon 2022, vol 26, research on applied ethics the relationship between social capital & social media addiction 2 and social media addiction. we also investigate the moderating role of privacy self-efficacy, which is the expression of one’s own ability to manage their privacy. the main research question, therefore, is how social capital bridging and bonding are associated with social media addiction. following williams (2006), bridging and bonding are considered as part of social capital. “social capital” is generally understood to be personal relationships and the benefits that come with them, such as emotional support, with some individuals interacting and forming a network (williams, 2006). utilizing putnam’s (2000) conceptualization of social capital, we divide social capital into two concepts, known as “bridging” and “bonding.” these concepts enable different types of social capital to result when different norms and networks are in place (putnam, 2000), which means that different forms of social capital may influence social media addiction and we must distinguish between these forms. for this study, we utilized covariance-based structural equation modeling (cbsem). for robustness checks, we conducted an ordinary least squares (ols) regression to test the hypotheses. this study is important as it demonstrates bridging and bonding have a differing association with social media addiction. further, we show that high bonding users experience more social media addiction, while those with high bridging have lesser social media addiction. we also show the moderating effect of privacy self-efficacy on the relationships between social capital bridging and bonding and social media addiction. for bridging, privacy self-efficacy serves to increase the negative association with social media addiction, while it increases the positive association between social media addiction and bonding relationships. this advances academic literature as neither bonding nor bridging has been explored in the context of social media addiction previously. the remainder of this paper is organized as follows. first, we review the extant literature for social media addiction and develop a theoretical model. we then present findings from our empirical research. next, we evaluate the implications and discuss the significance of the findings. finally, we present our conclusions and future research directions. 2 informing literature and theoretical framing in the following sections, we discuss the theoretical framing that contextualizes this study, drawing from the supporting academic literature. we then posit our hypothesis drawn from the literature to explain how privacy self-efficacy moderates the relationship between social capital and social media addiction. 2.1 social media addiction while technology is generally considered to have a positive impact, we are increasingly witnessing the negative side effects. these have usually been referred to as the “dark side” of technology (turel, serenko, & giles, 2011). addiction to social media is one such consequence of the negative effects of technology. as previous studies have found that the overuse of technology can have significant negative impacts on users (błachnio et al., 2016; thomée et al., 2007; xanidis & brignell, 2016), it is important to increase understanding about addiction. we provide the meaning of addiction for this study in the context of technology and social media. using the definition put forward by turel et al. (2011), we define technology addiction as a psychological state of maladaptive dependency on the use of technology to such a degree that the following typical behavioral addiction symptoms can exist: salience, withdrawal, conflict, tolerance, and mood modification. these behaviors present an obsessive pattern that takes place at the expense of other important activities (turel et al., 2011). given that social media is australasian journal of information systems soh, smith & dhillon 2022, vol 26, research on applied ethics the relationship between social capital & social media addiction 3 a type of technology and has been shown to form addiction in users (hou, xiong, jiang, song, & wang, 2019; turel, brevers, & bechara, 2018), this definition is appropriate to frame the concept of addiction for this study. as social media platforms become all-pervasive in our daily lives, a strong potential for social media addiction exists. specifically, the irrational and excessive use of social media and the extent to which it interferes with other aspects of daily ‘real-world’ life (hou et al., 2019). therefore, we believe that to curb social media addiction, which culminates in unhealthy overuse of technology; we must understand the antecedents and how they can be moderated to reduce addiction to social media. 2.2 social capital “social capital” is generally understood to be personal relationships and the benefits that come with them, with some individuals interacting and forming a network, termed a “social network” (williams, 2006). the benefits of social capital include positive outcomes, such as emotional support or the ability to mobilize others. however, while the academic literature has continued to debate whether social capital is a cause or an effect, numerous applications of social capital exist, including work that applies it within the context of privacy (williams, 2006). for this research, we utilize putnam’s (2000) conceptualization of social capital that divides it into two related, but not equivalent forms known as “bridging” and “bonding.” the concepts of bridging and bonding enable different types of social capital to result when different norms and networks are in place (putnam, 2000). therefore, different forms of social capital may influence social media addiction and we must distinguish between these forms. 2.2.1 bridging as a form of social capital, bridging is considered an inclusive concept (putnam, 2000; williams, 2006). bridging occurs when individuals from different backgrounds make connections between social networks and while they are only shallow relationships, they are incredibly expansive (patulny and svendsen, 2007). as such, social capital bridging can be considered as “outward-looking and encompassing people across diverse social cleavages” (putnam, 2000, p. 22). bridging associations are often argued to be more likely to generate positive externalities than bonding associations. for example, putnam (2000) makes an important distinction between “getting by” and “getting ahead.” social capital bonding requires trust and reciprocity in closed networks and helps the process of getting by in life on a daily basis. conversely, getting ahead is facilitated through cross-cutting ties that take the form of bridging social capital (coffe and gayes, 2007). therefore, bridging serves to broaden social horizons or world views, as well as open up new opportunities for information. however, as a negative, little in the way of emotional support is provided. within the context of social media, bridging provides users the ability to connect to an unlimited number of new social networks, providing them with an unparalleled breadth of new information and experiences. 2.2.2 bonding in contrast to bridging, bonding is an exclusive concept, occurring when strongly tied individuals provide emotional or substantive support for each other (coffe and gayes, 2007; patulny and svendsen, 2007; putnam, 2000). this is to say, there is a distinction between bridging and bonding with regard to the idea of openness versus exclusivity, which links to a conversation regarding positive and negative effects of social capital (coffe and gayes, 2007; patulny and svendsen, 2007). positive social capital derived from social control is typically found in “enforceable trust,” which generates positive outcomes fairly equally for all members australasian journal of information systems soh, smith & dhillon 2022, vol 26, research on applied ethics the relationship between social capital & social media addiction 4 of a group. negative social capital also involves enforceable rules, yet generates negative outcomes for the group or positive outcomes for only some members at the expense of others (patulny and svendsen, 2007). given the more tightly structured and exclusive nature of bonding social capital, it is likely that more negative aspects will be associated with such capital, such as social media addiction. therefore, individuals who possess bonding social capital have little background diversity yet maintain stronger personal connections. these personal connections provide for continued reciprocity, demonstrating strong emotional and substantive support. however, a major drawback of bonding social capital is insularity and out-group antagonism, leading to feelings of mistrust and dislike for those outside the “bonded” group. 2.3 privacy self-efficacy the concept of privacy self-efficacy has only recently begun to be explored in the academic literature (chen, 2018; chen & chen, 2015), however, it is impactful as a concept as it exists as the expression of one’s own ability to manage their privacy. this is important as work by xu et al. (2012) originally postulated that the nature of control in the context of information privacy differs by approaches according to different context-specific user concerns, thus resulting in different effects. xu et al. (2012) found that such effects are exhibited in a mediated manner by one’s perceived control over personal information. later work by chen and chen (2015) confirmed this and explored self-efficacy in the context of privacy. this resulted in the development of the privacy self-efficacy construct, a privacy-specific construct that encompasses both self-efficacy and privacy control (chen & chen, 2015). ultimately chen and chen (2015) demonstrated that privacy self-efficacy promotes selfwithdrawal behaviors through users removing identifiable information and changing privacy settings. then, chen (2018) confirmed the positive influence of privacy self-efficacy on privacy behavior. further, chen (2018) explored how social capital moderates the positive impact of privacy self-efficacy on privacy behavior. however, privacy self-efficacy has yet to be explored in relation to social capital when looking at social media addiction. this is an important distinction as social capital can be explored from a cause or effect perspective, and the significance of the relationship between social capital and privacy self-efficacy to understand social media addiction is, as yet, unknown. 3 hypotheses development 3.1 relationship between social capital and social media addiction we argue that social capital bridging increases the self-control of social media users, creating a negative association with social media addiction. social capital bridging occurs as individuals form ties with the broader community (leonard, 2004). the focus of bridging is to get ahead (putnam, 2000). individuals, by creating bridges, participate in and control information flows (burt, 2004). bridging values non-redundant sources of information, limiting cohesive and structurally equivalent connections (dubos, 2017). as a result, individuals, through bridging, are not constrained by obligations and norms. moreover, bridging entails control over the connections to ensure information benefits. notably, bridging is not about building close connections but rather increasing access to new information (dubos, 2017). individuals strategize their connections to generate several benefits. for example, bridging may enable individuals to generate good ideas (burt, 2004). australasian journal of information systems soh, smith & dhillon 2022, vol 26, research on applied ethics the relationship between social capital & social media addiction 5 furthermore, bridging, by limiting cohesion, creates an environment wherein individuals focus on their activities (dubos, 2017). as individuals seek to get ahead, bridges provide opportunities to grow in their activities. bridging does not distract social media users' attention from their (online or offline) activities, limiting social media addiction. prior research suggests that individuals who engage in meaningful activities experience less online social network addiction (xu, turel, & yuan, 2012). hence, bridging facilitates self-control as individuals try to get ahead in their activities. social media does not become a substitute for the offline world when users focus on bridging. rather, social media becomes an effective tool to get ahead in offline or online activities. thus, we hypothesize that: h1: bridging social capital is negatively associated with social media addiction. also, we argue that social capital bonding dampens social media users' self-control, leading to social media addiction. social capital bonding characterizes closely connected communities wherein individuals trust each other, are obligated to support each other, and are dependent on exchange with their community. notably, social capital bonding occurs as community members form close relationships characterized by trust and solidarity (leonard, 2004). this trust and solidarity prevail in closely connected networks because network closure facilitates the enactment of sanctions that can influence community members' behaviors (dubos, 2017). the threats of sanctions create norms and obligations among community members (m. granovetter, 1985). as social capital bonding enables social media users to form close connections, we suggest that sanctions' threats lead to dependence and attachment to social media as social media users feel obligated to their online connections. for example, social media users might feel the obligation to reciprocate the support received as failing to meet norms and expectations might create sanctions such as being excluded by others, and not receiving future support. the threat of sanctions likely reduces social media users’ self-control over social media usage, leading to social media addiction. for example, prior studies found that receiving online social support leads to internet addiction (wang & wang, 2013; yeh, ko, wu, & cheng, 2008). hence, we hypothesize that: h2: bonding social capital is positively associated with social media addiction. 3.2 moderating effects of privacy self-efficacy we argue that privacy self-efficacy reinforces the negative association between social capital bridging and social media addiction by increasing further self-control of social media users. bridging connects individuals through weak ties wherein there is less expectation of reciprocity (granovetter, 1973). individuals with high privacy self-efficacy do not share their private information unless they trust the recipients (chen & chen, 2015). for example, privacy self-efficacy drives behaviors (e.g., delete posts) to reduce profile visibility. these preemptive behaviors limit individuals’ attachment to their connections. as a result, social media users' ability with high privacy self-efficacy to self-control their social media use might increase when engaging in bridging. hence, we hypothesize that: h3: privacy self-efficacy negatively moderates the relationship between social capital bridging and social media addiction. also, we argue that privacy self-efficacy reinforces social media bonding's positive association with social media addiction by decreasing further self-control of social media users. strong bonds of trust and solidarity between individuals facilitate sharing private information (bansal, zahedi, & gefen, 2016). as more private information is shared among individuals, the australasian journal of information systems soh, smith & dhillon 2022, vol 26, research on applied ethics the relationship between social capital & social media addiction 6 threat of sanctions increases as such information can be disclosed to unauthorized individuals. the sanctions are more problematic for individuals with high privacy self-efficacy because they might reflect their inability to protect their personal information (chen & chen, 2015). individuals with high privacy self-efficacy are likely to manifest stronger attachment to the community to protect their private information. individuals with low privacy self-efficacy might not associate attachment to the community with the protection of their private information. as a result, social media users' ability with high privacy self-efficacy to selfcontrol their social media use might decrease when engaging in bonding. hence, we hypothesize that: h4: privacy self-efficacy positively moderates the relationship between social capital bonding and social media addiction. 4 methodology and data analysis 4.1 data collection and validation the data for this study was collected from amazon mechanical turk in december 2020. amazon mechanical turk is a crowdsourcing platform that is commonly used in information systems research for online surveys (steelman, hammer, & limayem, 2014). we created an online survey to examine factors that influence the addiction behavior of social media users in the united states. the measurement scale and the survey items appear in the appendix. figure 1 illustrates the conceptual model of the study. table 1 presents the demographics of the sample. figure 1. conceptual model a total of 414 responses were collected and validated by checking for common method variance (cmv). we used the marker variable (mv) technique (lindell & whitney, 2001). we added one marker variable by including the following question: “what is the capital of france?.” the marker variable is theoretically unrelated to the study. the correlation between the marker variable and the dependent variable is .07 (see table 4). after adjusting for that h4 h3 h2 h1 social media addiction privacy selfefficacy social capital bridging social capital bonding australasian journal of information systems soh, smith & dhillon 2022, vol 26, research on applied ethics the relationship between social capital & social media addiction 7 correlation, all correlations remained significant. hence, cmv is not an issue of concern in our research. we analyzed the data using stata/se 16.1. demographics category frequency percentage gender male 260 0.63 female 152 0.37 other 2 0 age 18-23 16 0.04 24-29 83 0.2 30-35 131 0.32 36 or above 184 0.44 education less than high school 1 0 high school diploma 83 0.2 college degree 101 0.24 undergraduate degree 132 0.32 graduate degree 95 0.23 other 2 0 employment full-time employed 307 0.74 part-time employed 27 0.07 self-employed 42 0.1 student 6 0.01 retired 7 0.02 unemployed 17 0.04 stay-at-home parent 8 0.02 social media use (number of hours) 0-2 125 0.3 2-4 160 0.39 4-6 92 0.22 6-8 23 0.06 8+ 14 0.03 table 1. sample demographics (n = 414) 4.2 measures the proposed model consists of five constructs. we tested the model by adapting validated items found in prior literature. we measured all the constructs using a reflective measurement model since the items are interchangeable (i.e., dropping an item does not alter the conceptual domain of the construct (jarvis et al. 2003)), sharing one theme. specifically, we measured social media addiction by adapting an instrument taken from turel et al. (2011). moreover, we assessed social capital bonding and social capital bridging by following williams (2006). furthermore, we measured privacy self-efficacy based on an instrument used by chen (2018). finally, we assessed social media romance by adapting instruments from patwardhan, and balasubramanian (2011). the social media romance measurement model includes items about arousal and pleasure. we excluded dominance due to a high correlation with social media addiction. social media romance is used as a control variable. all the measures use a sevenpoint likert scale going from (1) “strongly disagree” to (7) “strongly agree”. appendix 1 presents the survey items. 4.3 reliability and construct validity table 2 presents the results of a principal component analysis with orthogonal varimax rotation that we conducted to validate the measurement model. we obtain consistent results after using a direct oblimin rotation (morris and venkatesh 2010). the results support the internal consistency and discriminant validity of the scales. we dropped four items of social australasian journal of information systems soh, smith & dhillon 2022, vol 26, research on applied ethics the relationship between social capital & social media addiction 8 capital bonding (scbond2, scbond3, scbond6, and scbond9) and four items of social media addiction (sma2, sma5, sma7, and sma10) because of low factor loading (loading < 0.5). rotated component matrix component 1 2 3 4 5 smr1 0.7236 smr2 0.7645 smr3 0.7070 smr4 0.5661 smr5 0.6727 smr6 0.6561 smr7 0.7253 smr8 0.7177 pse1 0.5697 pse2 0.8305 pse3 0.8631 pse4 0.8739 pse5 0.8716 scbond1 0.7575 scbond4 0.7295 scbond5 0.7372 scbond7 0.7824 scbond8 0.7287 scbond10 0.7364 scbr1 0.7523 scbr2 0.7307 scbr3 0.7151 scbr4 0.8067 scbr5 0.7627 scbr6 0.7745 scbr7 0.7352 scbr8 0.6279 scbr9 0.6748 scbr10 0.6571 sma1 0.7877 sma3 0.8791 sma4 0.8490 sma6 0.7069 sma8 0.8269 sma9 0.8642 extraction method: principal component analysis rotation method: orthogonal varimax (kaiser off) loadings above .5 are displayed social media addiction (sma), privacy-self efficacy (pse), social media romance (smr), social capital bridging (scbr), social media bonding (scbond) table 2. factor analysis moreover, we assessed the reliability of the constructs using cronbach’s alpha and composite reliability. the results presented in table 3 indicate that the measurement of each construct is reliable since cronbach's alpha and the composite reliability score of all the constructs are above the minimum recommended threshold .7 (hair, black, babin, & anderson, 2010). australasian journal of information systems soh, smith & dhillon 2022, vol 26, research on applied ethics the relationship between social capital & social media addiction 9 construct cronbach’s alpha composite reliability (cr) sma 0.95 0.95 scbr 0.93 0.95 scbond 0.9 0.94 pse 0.9 0.91 smr 0.94 0.94 table 3. reliability test of the constructs finally, we checked the convergent and discriminant validity of the constructs. in table 2, we observe that the factor loadings of items are greater than 0.5. additionally, the average variance extracted (ave) scores of all the constructs are above the minimum recommended threshold of 0.5 (fornell & larcker, 1981). thus, we conclude that the constructs have convergent validity. table 4 presents the correlations between the constructs. the diagonal axis contains the square root of the ave scores. we observe that the correlation scores are below the maximum threshold of 0.8 (hair et al., 2010), mitigating the multicollinearity issue. also, the square roots of the ave scores are greater than the correlations scores of the same row and column, indicating that the constructs have discriminant validity. hence, we can proceed with further analysis as we demonstrated that the measures of the constructs are reliable and valid. sma scbr scbond pse smr sma 0.87 scbr 0.41* 0.8 scbond 0.62* 0.59* 0.84 pse 0.22* 0.47* 0.37* 0.82 smr 0.65* 0.68* 0.66* 0.43* 0.82 marker 0.07 .03 .05 .03 .04 diagonal elements are the square root of the average variance extracted (ave) *p < .05 n = 414 table 4. constructs correlation matrix 4.4 results in figure 2, we displayed the path coefficients (β value) of the structural model. the path coefficients are obtained using covariance-based sem (cbsem). for robustness checks, we conducted an ordinary least squares (ols) regression to test the hypotheses. the results of cbsem and ols are presented in table 5. we find support for all the hypotheses. specifically, our findings indicate that social capital bridging is negatively associated with social media addiction (h1). moreover, the results show that social capital bonding is positively associated with social media addiction (h2). finally, we find that privacy self-efficacy reinforces the negative association between social capital bridging and social media addiction (h3) and the positive association between social capital bonding and social media addiction (h4). the moderation effects were calculated by assessing the effects of two interaction terms: social capital bridging and privacy self-efficacy, and social capital bonding privacy self-efficacy on social media addiction. the variance inflation factors (vif) are lower than 5, indicating that there is no multicollinearity (see table 6). australasian journal of information systems soh, smith & dhillon 2022, vol 26, research on applied ethics the relationship between social capital & social media addiction 10 * p<.05, ** p<.01 and *** p<.001 figure 2. path coefficients cbsem ols (robust standard errors) smr .558*** .562*** .553*** .559*** (.0592) (.059) (.0647) (.0658) scbr -.231** -.21** -.238** -.229** (.0766) (.0792) (.0842) (.0871) scbond .406*** .423*** .408*** .426*** (.0566) (.056) (.0605) (.0614) pse -.113+ -.0945 (.0687) (.0752) scbr*pse -.0997* -.105* (.0497) (.0535) scbond*pse .111** .117* (.041) (.0479) constant .968** 1.08*** .498* .504* (.324) (.32) (.208) (.208) var(e.sma) 1.51*** 1.46*** (.105) (.102) r2 0.531*** 0.544*** n 414 414 414 414 controls: age, gender, employment * p<.05, ** p<.01 and *** p<.001 table 5. results variable vif 1/vif smr 2.43 0.41 scbr 2.27 0.44 scbond 2.08 0.48 pse 1.62 0.62 scbr*pse 1.77 0.56 scbond*pse 1.84 0.54 mean vif 2.34 0.43 table 6. variance inflation factors .111** -.0997* .406*** -.231** social media addiction privacy selfefficacy social capital bridging social capital bonding australasian journal of information systems soh, smith & dhillon 2022, vol 26, research on applied ethics the relationship between social capital & social media addiction 11 5 discussion and implications our research makes two crucial theoretical contributions. first, bridging and bonding have a differing association with social media addiction. we found that high bonding users experience more social media addiction, while those with high bridging have lesser social media addiction. second, privacy self-efficacy plays a reinforcing role, increasing social media addiction for users with high bonding while decreasing further social media addiction for users with high bridging. in the paragraphs below, we discuss each of the implications. 5.1 role of bonding and bridging on social media addiction it is interesting to note that bridging and bonding are associated with social media addiction differently. in situations where an individual has high bonding with others, addiction chances are higher. a plausible reason is that bonding satisfies individual needs for emotional support thus increasing the tendency to be dependent on social media. past research has also found the search for emotional support results in higher chances of addiction as people become more reliant on technologies (malloch and hether, 2019). for example, andreassen et al. (2016) found that single individuals and females have a higher dependency on social media most likely due to their tendency to seek greater amounts of emotional support through technology. however, santini et al. (2015) have argued, providing emotional support through social networks, physical and online, effectively prevents adverse mental health outcomes, thus reducing the chances of addiction. in an online setting, however, all interactions are not the same. shensa et al. (2016), for instance, found "most time per day on social media sites had significantly lower odds of reporting higher levels of perceived emotional support" (p. 546). yet, in a 2020 study, shensa and her colleagues report that social media can be a source of emotional support (shensa et al., 2020). the more individuals identify with social media, for example as a source of emotional support, the more dependent they become on technology itself (carter and grover, 2015). one reason for conflicting evidence is perhaps related to who the social media interactions are with. casual interactions with acquaintances may not provide emotional support. and this may be the reason for contradictory evidence from shensa’s research. in that sense, our study's findings make a useful theoretical contribution where we find that bonding with close family and friends results in emotional support, which makes individuals likely to get addicted to social media. 5.2 moderating role of privacy self-efficacy on social media addiction as noted earlier, bonding occurs when individuals engage with friends and family for some type of emotional support. our research has found that increased privacy self-efficacy reinforces the association between bonding and social media addiction. this means that the higher the privacy self-efficacy, the greater the addiction for users with high bonding. given the extant literature, our finding makes an important theoretical contribution. in the literature, the bonding and bridging types of social capital have not been separated. casale, rugai, and fioravanti (2018), for instance, collectively consider bonding and bridging as metacognitions about social media use. other research has also studied metacognitions collectively while exploring the problematic use of social media networks (akbari, 2017; spada & marino, 2017). as putnam (2000) argues, bridging social capital is different as such associations are not for emotional support. these types of social capital are simply connections that lack depth. therefore, it makes intuitive sense that once privacy self-efficacy increases, social media addiction decreases for high bridging users. this is also an important theoretical contribution australasian journal of information systems soh, smith & dhillon 2022, vol 26, research on applied ethics the relationship between social capital & social media addiction 12 of our research. while not specifically discussing the role of privacy, infidelity has been found to increase social media addiction (abbasi, 2019). 5.3 limitations and future research directions although our study makes several contributions, our findings should be interpreted in light of several limitations. first, we are unable to make causal inferences because of the crosssectional nature of our dataset. future studies may collect longitudinal data, or conduct qualitative analyses to examine the impact of social capital on social media addiction. second, there are different conceptualizations of social capital. bonding and bridging are one way to conceptualize different types of social capital. future research may consider other conceptualizations (e.g., nahapiet & ghoshal, 1998). finally, our study focuses on a direct link between social capital and social media addiction. however, based on our argumentation, social media users’ self-control may mediate the effect of social capital on social media addiction. future work may test this mediation and other possible mediations to increase the understanding of the impact of social capital on social media addiction. 6 conclusion in this paper, we sought to investigate the relationship between social capital and social media addiction. we also tested the moderating role of privacy self-efficacy on the relationship between social capital and social media addiction. findings from our research significantly contribute to the body of literature beyond what the extant literature has noted. while muchpublished research has hypothesized social capital's role in social media addiction, there is limited research on the more granular aspects of social capital. our research has, in particular, explored the role of bridging and bonding in addiction. the paper concludes that if individuals are made aware and educated about their online privacy concerns, bridging’s negative association with social media addiction and bonding’s positive association with social media addiction are strengthened. individuals can suitably manage bridging and bonding relationships to limit social media addiction. references abbasi, i. s. (2019). social media addiction in romantic relationships: does user’s age influence vulnerability to social media infidelity? personality and individual differences, 139, 277– 280. akbari, m. (2017). metacognitions or distress intolerance: the mediating role in the relationship between emotional dysregulation and problematic internet use. addictive behaviors reports, 6, 128–133. andreassen, c., & pallesen, s. (2014). social network site addiction an overview. current pharmaceutical design, 20(25), 4053–4061. andreassen, c. s., billieux, j., griffiths, m. d., kuss, d. j., demetrovics, z., mazzoni, e., & pallesen, s. (2016). the relationship between addictive use of social media and video games and symptoms of psychiatric disorders: a large-scale cross-sectional study. psychology of addictive behaviors, 30(2), 252-262. armstrong, m. (2021). is peak social media already behind us? retrieved 20 december 2021, from https://www.statista.com/chart/26272/global-average-daily-time-spent-on-socialmedia-per-internet-user/ australasian journal of information systems soh, smith & dhillon 2022, vol 26, research on applied ethics the relationship between social capital & social media addiction 13 bansal, g., zahedi, f. m., & gefen, d. (2016). do context and personality matter? trust and privacy concerns in disclosing private information online. information and management, 53(1), 1–21. błachnio, a., przepiorka, a., & pantic, i. (2016). association between facebook addiction, selfesteem and life satisfaction: a cross-sectional study. computers in human behavior, 55, 701–705. burt, r. s. (2004). structural holes and good ideas. american journal of sociology, 110(2), 349– 399. carter, m., & grover, v. (2015). me, my self, and i (t). mis quarterly, 39(4), 931-958. chen, h. t. (2018). revisiting the privacy paradox on social media with an extended privacy calculus model: the effect of privacy concerns, privacy self-efficacy, and social capital on privacy management. american behavioral scientist, 62(10), 1392–1412. chen, h. t., & chen, w. (2015). couldn’t or wouldn’t? the influence of privacy concerns and self-efficacy in privacy management on privacy protection. cyberpsychology, behavior, and social networking, 18(1), 13–19. coffé, h., & geys, b. (2007). toward an empirical characterization of bridging and bonding social capital. nonprofit and voluntary sector quarterly, 36(1), 121-139. fornell, c., & larcker, d. f. (1981). structural equation models with unobservable variables and measurement error: algebra and statistics. journal of marketing research, 18(3), 382– 388. granovetter, m. (1985). economic action and social structure: the problem of embeddedness. american journal of sociology, 91(3), 481–510. granovetter, m. s. (1973). the strength of weak ties. american journal of sociology, 78(6), 1360– 1380. hair, j. f., black, w. c., babin, b. j., & anderson, r. e. (2010), multivariate data analysis. englewood cliffs, nj: prentice hall. hou, y., xiong, d., jiang, t., song, l., & wang, q. (2019). social media addiction: its impact, mediation, and intervention. cyberpsychology, 13(1). koc, m., & gulyagci, s. (2013). facebook addiction among turkish college students: the role of psychological health, demographic, and usage characteristics. cyberpsychology, behavior, and social networking, 16(4), 279–284. leonard, m. (2004). bonding and bridging social capital: reflections from belfast. sociology, 38(5), 927–944. lindell, m. k., & whitney, d. j. (2001). accounting for common method variance in crosssectional research designs. journal of applied psychology, 86(1), 114-121. malloch, y. z., & hether, h. j. (2019). the dark side of addiction support forums: impacts of poor quality and insufficient emotional support on perceived support availability and health efficacy. journal of health communication, 24(4), 432-441. nahapiet, j., & ghoshal, s. (1998). social capital, intellectual capital, and the organizational advantage. academy of management review, 23(2), 242–266. pantic, i. (2014). online social networking and mental health. cyberpsychology, behavior and social networking, 17(10), 652–657. patulny, r. v., & svendsen, g. l. h. (2007). exploring the social capital grid: bonding, bridging, qualitative, quantitative. international journal of sociology and social policy, 27(1/2), 32-51. putnam, r. d. (2000). bowling alone: the collapse and revival of american community: new york: simon and schuster, 2001. isbn. policy analysis. australasian journal of information systems soh, smith & dhillon 2022, vol 26, research on applied ethics the relationship between social capital & social media addiction 14 shensa, a., sidani, j. e., escobar-viera, c. g., switzer, g. e., primack, b. a., & choukasbradley, s. (2020). emotional support from social media and face-to-face relationships: associations with depression risk among young adults. journal of affective disorders, 260, 38–44. spada, m. m., & marino, c. (2017). metacognitions and emotion regulation as predictors of problematic internet use in adolescents. clinical neuropsychiatry, 14, 59–63. statista. (2019). share of online users in the united states who report being addicted to social media as of april 2019, by age group. retrieved 20 december 2021, from https://www.statista.com/statistics/1081292/social-media-addiction-by-age-usa/ steelman, z. r., hammer, b. i., & limayem, m. (2014). data collection in the digital age: innovative alternatives to student samples. mis quarterly, 38(2), 355-378. thomée, s., eklöf, m., gustafsson, e., nilsson, r., & hagberg, m. (2007). prevalence of perceived stress, symptoms of depression and sleep disturbances in relation to information and communication technology (ict) use among. computers in human behavior, 23(3), 1300–1321. turel, o., brevers, d., & bechara, a. (2018). time distortion when users at-risk for social media addiction engage in non-social media tasks. journal of psychiatric research, 97, 84–88. turel, o., serenko, a., & giles, p. (2011). integrating technology addiction and use: an empirical investigation of online auction users. mis quarterly, 35(4), 1043-1061. turel o, he q, xue g, xiao l and bechara a (2014) examination of neural systems sub-serving facebook “addiction”. psychological reports 115(3), 675–695. wang, e. s. t., & wang, m. c. h. (2013). social support and social interaction ties on internet addiction: integrating online and offline contexts. cyberpsychology, behavior, and social networking, 16(11), 843–849. williams, d. (2006). on and off the ’net: scales for social capital in an online era. journal of computer-mediated communication, 11(2), 593–628. xanidis, n., & brignell, c. m. (2016). the association between the use of social network sites, sleep quality and cognitive function during the day. computers in human behavior, 55, 121–126. xu, z., turel, o., & yuan, y. (2012). online game addiction among adolescents: motivation and prevention factors. european journal of information systems, 21(3), 321-340. yeh, y. c., ko, h. c., wu, j. y. w., & cheng, c. p. (2008). gender differences in relationships of actual and virtual social support to internet addiction mediated through depressive symptoms among college students in taiwan. cyberpsychology and behavior, 11(4), 485– 487. appendix 1: measurement scales constructs scale items source privacy selfefficacy pse1: i feel confident dealing with the ways that social media collect and use my personal information (chen, 2018) – adopted and modified pse2: i feel confident learning skills to protect my privacy on social media pse3: i feel confident blocking spam or unwanted content on social media pse4: i feel confident adjusting privacy settings on social media australasian journal of information systems soh, smith & dhillon 2022, vol 26, research on applied ethics the relationship between social capital & social media addiction 15 pse5: i feel confident managing personal profiles on social media social capital bonding scbond1: there are several people on social media i trust to help solve my problems. (williams, 2006) – adopted and modified scbond2: there is someone on social media i can turn to for advice about making very important decisions. scbond3: there is no one on social media that i feel comfortable talking to about intimate personal problems. (reversed) scbond4: when i feel lonely, there are several people on social media i can talk to. scbond5: if i needed an emergency loan of $500, i know someone on social media i can turn to. scbond6: the people i interact with on social media would put their reputation on the line for me. scbond7: the people i interact with on social media would be good job references for me. scbond8: the people i interact with on social media would share their last dollar with me. scbond9: i do not know people on social media well enough to get them to do anything important. (reversed) scbond10: the people i interact with on social media would help me fight an injustice. social capital bridging scbr1: interacting with people on social media makes me interested in things that happen outside of my town. (williams, 2006) – adopted and modified scbr2: interacting with people on social media makes me want to try new things. scbr3: interacting with people on social media makes me interested in what people unlike me are thinking. scbr4: talking with people on social media makes me curious about other places in the world. scbr5: interacting with people on social media makes me feel like part of a larger community. scbr6: interacting with people on social media makes me feel connected to the bigger picture. scbr7: interacting with people on social media reminds me that everyone in the world is connected. scbr8: i am willing to spend time to support general on social media community activities. scbr9: interacting with people on social media gives me new people to talk to. scbr10: on social media, i come in contact with new people all the time. social media addiction sma1: much of my time is occupied by thoughts about social media (turel, et al., 2011) – adopted and modified sma2: my thoughts about social media interfere with my social, school, work or role functioning sma3: my thoughts about social media cause me anxiety and/or distress sma4: i often try to turn my attention away from thoughts about social media sma5: i have much control over my thoughts about social media (reversed) sma6: i spend much of my time using social media sma7: my use of social media interferes with my social, school, work or role functioning australasian journal of information systems soh, smith & dhillon 2022, vol 26, research on applied ethics the relationship between social capital & social media addiction 16 sma8: i become anxious and/or distressed when i am prevented from using social media sma9: i often try to resist my social media usage compulsion sma10: i have much control over my use of social media (reversed) social media romance smr1: i love social media (patwardhan & balasubramanian, 2011) – adopted and modified smr2: using social media gives me great pleasure smr3: i am really happy that social media is available smr4: social media rarely disappoints me smr5: i desire social media smr6: i am attracted to social media smr7: i want social media smr8: i look forward to using social media dropped items are in italic copyright: © 2022 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v26i0.3367 http://creativecommons.org/licenses/by-nc/3.0/au/ https://doi.org/10.3127/ajis.v26i0.3367 1 introduction 2 informing literature and theoretical framing 2.1 social media addiction 2.2 social capital 2.2.1 bridging 2.2.2 bonding 2.3 privacy self-efficacy 3 hypotheses development 3.1 relationship between social capital and social media addiction 3.2 moderating effects of privacy self-efficacy 4 methodology and data analysis 4.1 data collection and validation 4.2 measures 4.3 reliability and construct validity 4.4 results 5 discussion and implications 5.1 role of bonding and bridging on social media addiction 5.2 moderating role of privacy self-efficacy on social media addiction 5.3 limitations and future research directions 6 conclusion paper title (paper title style) australasian journal of information systems richter & schwabe 2025, vol 29, provocation there is no ai in team 1 “there is no ‘ai’ in ‘team’! or is there?” – towards meaningful human-ai collaboration alexander richter school of information management victoria university of wellington new zealand email: alex.richter@vuw.ac.nz gerhard schwabe department of informatics university of zurich switzerland abstract ai is increasingly embedded in collaborative processes, challenging traditional notions of teamwork. while ai lacks consciousness and human-like intentionality, its ability to shape decision-making, structure interactions, and contribute to shared outcomes necessitates a reassessment of what it means to be a team member. this paper critically examines ai’s role in human-ai collaboration, arguing that effective teamwork is not solely defined by mutual understanding but by the complementary contributions of human and ai actors. we highlight the importance of context in shaping collaboration and propose a research agenda structured around five key aspects: (1) rethinking team membership, (2) evolving team roles, (3) the influence of contextual factors, (4) adaptive collaboration practices, and (5) ethical considerations. we call on scholars and practitioners to explore these dimensions to ensure that human-ai teams align with organizational goals and societal values. keywords: human-ai collaboration, human-ai teams, adaptive practices. 1 collaboration, agents, teams in the evolving landscape of ai-supported work, the concept of collaboration between humans and ai1 has attracted increasing attention and raised questions. participating in team activity intuitively looks like something very human and many will object to regarding ai agents as team members. and they have a point: traditional definitions of teams often emphasize the importance of shared understanding and common goals among team members (mathieu et al., 2000; salas et al., 2005). yet, available ai lacks consciousness, intentions, and an inherent understanding of goals (floridi & sanders 2004). so, is it useful to think that can humans and ai truly collaborate? can ai be regarded as an actor, hence, a ‘real’ team member? what consequences does this have for our understanding of teamwork? in this paper, we examine the notion of human-ai collaboration and challenge conventional definitions of teamwork. we argue that ai, though lacking human-like shared understanding, can still be regarded as a team member. as such, human-ai collaboration is not necessarily 1 throughout this article, "ai" refers to artificial intelligence in its broadest sense, encompassing its role as an agent in collaborative processes. this highlights ai's active contributions as a collaborator, beyond mere technological systems. australasian journal of information systems richter & schwabe 2025, vol 29, provocation there is no ai in team 2 defined by mutual comprehension but by the complementary strengths of both parties (huxham & vangen 2013). to explore whether ai can be considered an agent, we must first establish the ‘guidelines for agenthood’ (floridi & sanders 2004): how do we define what an agent is? it has been suggested to adopt an intentional stance (dennett, 1971) when interacting with ai, especially as ai grows more proactive. for instance, the belief-desire-intention (bdi) model offers a framework for developing ai that mimics goal-oriented human reasoning. bdi systems simulate intentional behavior by representing beliefs about the environment, desires (or goals), and intentions (or plans to achieve those goals). although bdi systems do not have true desires or intentions, they offer an approximation that allows ai to take a more autonomous role in collaborative settings (rao & georgeff, 1995). against this backdrop it has been argued that it is, in many cases, useful to follow the natural tendency of human actors to attribute agency to ai (nass & moon 2000): the more human-like a computer acts, the more humans can use their learnt human interaction skill with digital agents. and the simpler and the less critical the interaction is, the easier it is to include digital agents into human interaction. taking the example of innovation, digital agents have participated successfully as team members in brainstorming (schwabe et al. 2025) and they can easily be imagined to contribute as actors to organizing and evaluating ideas, too. as brainstorming is supposed to be conducted anonymously and strives to create “crazy” ideas, a digital brainstorming agent looks like a promising use case – with possibly unintended consequences for group dynamics. for example, when digital brainstorming agents assisted individual human brainstorming participants the human participants became free riders, not contributing as much as to traditional meetings (memmert & tavanapour 2023). in this context, the ai and the human team members each take on roles that influence the outcome. ai's role as a collaborator becomes even more critical in complex social environments where multiple stakeholders with diverse interests interact. ai must navigate these complexities by understanding and balancing conflicting interests, which requires a nuanced approach to cooperation. the ability of ai to act in multistakeholder environments such as those characterized by competing incentives or shared resources will redefine the parameters of human-ai collaboration (dafoe et al. 2021). hence, while ai may not fulfill all traditional criteria for team membership, its ability to contribute to collaborative processes highlights the need to rethink what it means to be a team member in the context of human-ai collaboration. 2 adaptive practices and contextual factors in human-ai collaboration as teams increasingly operate within digital ecosystems saturated with ai, the distinction between human and ai contributions becomes less apparent. for example, in collaborative platforms where ai offers real-time suggestions, moderates discussions, or automates routine tasks, it becomes challenging to attribute decisions solely to human or ai agents. as ai becomes more seamlessly embedded into digital workspaces, teams face the challenge of adapting their practices to leverage ai's capabilities without losing the unique strengths of human collaboration. in many of these ai-infused environments, ai often operates behind the scenes to mediate tasks, facilitate communication, or automate decision-making processes. this 'invisible ai' phenomenon blurs the line between active team members and the underlying digital infrastructure, making it challenging for teams to distinguish between australasian journal of information systems richter & schwabe 2025, vol 29, provocation there is no ai in team 3 human contributions and those influenced by ai. the invisibility of ai in such contexts raises important considerations for transparency and accountability, as teams may not always be aware of the extent to which ai shapes their workflows and outcomes. this underscores the need for governance models that ensure ai-driven decisions remain transparent and accountable. as ai’s role in teams evolves, its impact on work configurations extends beyond structured team settings to broader sociotechnical ecosystems. ai-mediated work has reconfigured spatial-temporal work dynamics, affecting when and where collaboration occurs (willems & hafermalz, 2021). in remote and hybrid work settings, ai can mediate digital interactions, shape participation dynamics, and influence role distributions, sometimes in ways that remain unnoticed (klein & watson-manheim, 2021). this raises critical questions about how ai’s presence in virtual and hybrid work environments transforms social coordination and decision-making over time. consequently, practices must evolve to ensure that teams can effectively integrate ai into their workflows while maintaining the critical contributions of human members. this includes fostering an environment where human team members can interpret ai outputs critically, understanding when to trust ai recommendations and when to override them based on contextual knowledge. the challenge lies in developing and reassessing practices enhancing collaboration as ai evolves. this intentional adaptation should not only focus on efficiency and effectiveness but also on ensuring that human-ai teams’ function in the way we desire. this iterative adaptation is especially crucial in high-stakes environments like healthcare, where ai’s role in decision-making must be aligned with human values and ethical standards to prevent unintended consequences. by recognizing that ai's influence permeates various levels of team dynamics from direct collaboration to the underlying digital infrastructure that supports it organizations can better prepare for the complexities of future collaborative environments. ai embedded within collaborative platforms might take on coordination tasks, freeing human members to focus on strategic decision-making and creative problem-solving. this reconfiguration of roles highlights the importance of designing adaptive practices that are flexible enough to accommodate the evolving contributions of ai in teams. for instance, chowdhury et al. (2022) discuss the emergence of 'collaborative intelligence,' which arises from effective knowledge sharing between ai and human collaborators. this concept underscores the need for adaptive practices that evolve alongside the dynamic roles of both ai and humans in collaborative settings. exploring the concept of collaboration further, it is worth noting that cultural, social, and organizational contexts may give rise to varying roles and interactions. as such, the nature of the tasks and the environment in which they are performed play a key role in shaping the dynamics between human and ai agents. for instance, inherent cultural norms or perceptions may limit the extent of human-ai collaboration, leading to a greater focus on human-to-human interaction. in some scenarios, ai might take on more directive roles, steering the process, making decisions or providing insights that guide human actions. in other contexts, humans may primarily participate in the collaborative process, with ai acting as a sophisticated agent that imitates human capabilities. historically, ai has primarily augmented human activities by specialized recommendations. for instance, ibm's watson has been used in healthcare to analyze medical literature and australasian journal of information systems richter & schwabe 2025, vol 29, provocation there is no ai in team 4 patient records, providing doctors with evidence-based treatment options to enhance their decision-making. however, more recent generative ai (genai) tools, such as chatgpt or google gemini, reduce the need for human involvement in routine activities. they can draft emails, generate content, write basic code, or even automate customer support by generating responses to inquiries. by recognizing ai’s capacity to co-create and provide strategic insights, we further challenge traditional notions of ai as a passive instrument. both specialized recommendation systems and generative ai exemplify the diverse ways in which human-ai collaboration can unfold. the roles and interactions between humans and ai shift considerably based on the specific demands of the situation and the unique strengths of each agent. similar to how bdi agents balance conflicting values like health and sustainability in decision-making processes (cranefield et al., 2017), ai in collaborative teams can adjust their actions based on situational demands. for instance, in elderly care artificial systems (ecas), ai chooses between short-term satisfaction (e.g., serving pizza) and long-term health goals (e.g., serving a healthy meal). this value-sensitive decision-making mirrors how ai contributes contextually to team tasks by optimizing for multiple conflicting objectives. adopting a values-based approach, such as incorporating value sensitive design (vsd) principles (friedman, 2013), provides a structured framework for embedding core human values—such as privacy, fairness, and autonomy—into adaptive practices. this ongoing ethical calibration ensures that ai aligns with shifting societal norms, maintaining responsiveness to human values like fairness and respect for autonomy across diverse contexts (van wynsberghe, 2013). this adaptability is particularly important in complex environments where predefined roles and rigid structures may not suffice. seeber et al. (2020) highlight the dualities inherent in aihuman collaboration, such as the potential for ai to both augment and deplete human knowledge or to be accepted or rejected by human teammates. these dualities underscore the importance of context in determining the roles and effectiveness of ai in collaborative processes. moreover, for ai to truly contribute as collaborator, they should be designed not only to enhance team outcomes but also to align with the ethical values upheld by the human team members. incorporating vsd principles into human-ai collaboration ensures that ai is attuned to ethical considerations from the outset, aligning their actions with core human values such as privacy, autonomy, and fairness (van riemsdijk et al. 2015). from a practical perspective, human-ai collaboration can span across various value chain activities, each associated with different contextual factors. therefore, it is essential to identify and develop a deep understanding of the practices established in the various departments. figure 1 illustrates ai’s role in multistakeholder environments, highlighting how ai navigates competing incentives and shared resources. in this context, (human-ai collaboration) use cases are a simple, but effective tool for prioritizing (“where to begin?”), for facilitating appropriation, risk management (“which uses bring which inherent risks?”), and for constant reflection and improvement (“is this use still in line with our cultural mindset”) (sudeeptha et al. 2025). australasian journal of information systems richter & schwabe 2025, vol 29, provocation there is no ai in team 5 figure 1: ai's role in multistakeholder environments thus, understanding and designing for these contextual nuances and practices is essential for maximizing the benefits of human-ai collaboration. rather than viewing ai as a mere tool or as an independent actor, it is more productive to see it as participant in a fluid and dynamic relationship. this perspective acknowledges that the effectiveness of human-ai collaboration is not just about the capabilities of the ai or the expertise of the humans but about how well the collaboration is contextualized and practiced within specific environments. however, as ai becomes more embedded in collaborative work, governance frameworks must evolve to address transparency, bias, and accountability. the challenge is not only to ensure ai’s ethical alignment with human values but also to prevent sociotechnical reversals where ai inadvertently reintroduces biases or alters workplace power dynamics. emerging models of ai safeguards and sociotechnical envelopment provide pathways for organizations to capture ai’s value while mitigating risk (asatiani et al., 2021). these governance mechanisms will be key in ensuring ai supports collaborative agency rather than undermining it. for human-ai collaboration to be effective, it is crucial that there is a dynamic exchange of information and capabilities between humans and ai. humans provide ai with essential background information, well-defined objectives, and contextual data, while ai contributes by processing this information, recognizing patterns, and making predictions based on its strengths in data analysis and categorization. this collaborative process allows ai to generate insights that humans then refine and contextualize, integrating these insights into broader decision-making processes. hence, the success of human-ai collaboration depends on a deep understanding of contextual factors and the continuous adaptation of practices that align with the evolving roles and contributions of both humans and ai. our deliberations above lead us to the following propositions and to a definition: proposition 1: the evolving nature of human-ai collaboration indicates that, while ai may not fulfill the traditional criteria for team membership, it contributes meaningfully to collective outcomes. as such, the term "team" remains relevant in describing human-ai interactions, albeit with a refined understanding of what constitutes (team) collaboration. australasian journal of information systems richter & schwabe 2025, vol 29, provocation there is no ai in team 6 proposition 2: the nature of human-ai collaboration varies significantly depending on the context, necessitating flexible roles and adaptive practices. proposition 3: roles and practices in human-ai collaboration are not static but must evolve in response to the unique challenges and demands of different contexts. such an approach ensures that ai is integrated in ways that maximize its potential as collaborator while aligning with human needs and organizational goals. proposition 4: effective human-ai collaboration requires a shift from focusing on technological possibilities to considering what individuals, organizations, and society desire. this involves envisioning future human-ai interactions that align with human values and goals, fostering meaningful and purposeful teamwork. definition: human-ai collaboration is a synergistic partnership where humans and ai work together to achieve collective outcomes. this dynamic interaction is shaped by specific contextual factors, including cultural norms, organizational structures, and the nature of the task, requiring ongoing adaptation of roles and practices to ensure alignment with human goals and values. 3 call to action: towards meaningful human-ai collaboration we have so far argued that ai's role in collaborative processes challenges traditional definitions of team membership, emphasizing the need for a redefined understanding of (team) collaboration that accounts for the unique contributions of both humans and ai. it seems essential that scholars and practitioners alike consider these insights in designing and governing ai. equally important is a reflection on how we, as individuals, organizations, and society, want to collaborate within human-ai teams. this involves proactively shaping the nature of collaboration and shifting our focus from what is possible to what is desired. beyond technical and ethical considerations, it requires envisioning human-ai collaboration that aligns with human aspirations, defining ai's role in meaningful ways, and fostering work environments that prioritize purposeful collaboration. from discussing the potential of ai with stakeholders in industry and government in new zealand and australia we know that the success of human-ai collaboration is strongly connected to the socio-cultural, and organizational contexts in which it occurs. on a societal level, our multicultural and highly developed societies provide a fantastic setup to explore human-ai collaboration. robust governance frameworks and socially inclusive policies can inform the development of ai governance models that are not only effective but also socially responsible and culturally sensitive – and ai must be designed to align with these societal norms. at the organizational level, integrating ai into collaborative processes mirrors other sociotechnical systems (e.g. for the use of enterprise social media cf. hacker et al. 2024): the organizational context impacts potential use cases – but on the other side, human-ai collaboration has far-reaching implications beyond team dynamics, influencing organizational structures and necessitating new approaches to work design (jarrahi et al., 2023). this shift australasian journal of information systems richter & schwabe 2025, vol 29, provocation there is no ai in team 7 underscores the importance of developing governance models that are not only inclusive of ai but also adaptable to the evolving nature of human-ai collaboration. moreover, these models need to be comprehensive enough to also consider the rapid development of virtual environments (aka the metaverse). the underlying immersive technologies do not only expand the possibilities for collaboration but also require us to consider their unique challenges and opportunities. future-oriented methodologies can help stakeholders prepare for and shape the future of work in increasingly complex digital environments (richter & richter, 2024). they can contribute to a comprehensive contextual understanding which is crucial for ensuring that human-ai collaboration not only functions effectively within specific environments but also contributes positively to social cohesion and ethical governance. the design of 'digital teammates' forces us to understand aspects of teamwork that have been ignored so far. for example, when we designed the digital brainstorming agent, we had to ensure that the agent matched the tone and the length of a typical human brainstorming participant. to our knowledge those have not been studied before. in some situations, digital brainstorming participants may encourage human participants to free ride. this forces us to revisit the motivational basics of team collaboration. so, striving to imitate human behavior in teams will lead to deeper insights into the foundations of team collaboration. note that this new knowledge may also be applied to purely human teams. but it may become particularly interesting when we create digital brainstorming participants with super-human capabilities (e.g. contributing far more contributions than human participants) or if several team members are digital agents. what does it mean for teamwork if additional (digital) participants can be invited "for free"? the following research questions are designed to explore the various dimensions of human-ai collaboration, as discussed above, from the redefinition of team roles to the practical and ethical implications of integrating ai into collaborative environments. we structure the research questions within a framework highlighting the key aspects of humanai collaboration as discussed in this article (cf. figure 2). a. a new understanding of teams • how can we understand team communication? and how can we improve it? • how can we understand team dynamics? and how can we influence them? • what is a good team? for which task? how can we detect and predict team quality? • how can the concept of human-to-human collaboration be reframed in the presence of ai in collaborative scenarios? • what are team processes? how can they be facilitated? • what is an optimal team size? what is the optimal ratio between human and digital participants? b. reimagining team roles • how can traditional definitions of team membership be adapted to account for the unique contributions of ai in collaborative processes? • how do roles develop in teams? how can we assign them? • what criteria should be used to evaluate the effectiveness of ai as a team member in different collaborative contexts? • how can we redesign human-centered work policies to account for human-ai collaboration? australasian journal of information systems richter & schwabe 2025, vol 29, provocation there is no ai in team 8 • how does the emergence of immersive virtual environments (aka the metaverse) influence the dynamics and effectiveness of human-ai collaboration, and what new frameworks are needed to support these interactions? figure 2: key aspects of human-ai collaboration c. contextual factors • how do varying cultural, organizational, and social contexts impact the roles and effectiveness of ai in human-ai collaboration? • in what ways do specific contextual factors (e.g., industry, task complexity, stakeholder diversity) influence the distribution of decision-making authority between humans and ai? • what is the impact of cultural, organizational, and social contexts on human-ai collaboration practices such as innovation? d. adaptive practices • how can we design adaptive roles and practices that can accommodate the evolving nature of human-ai collaboration? • how can ai be designed to align with evolving collaborative practices while ensuring ethical and effective outcomes? • how do established practices in high-stakes environments (e.g., healthcare, autonomous driving) ensure that ai contributes effectively to decision-making processes? australasian journal of information systems richter & schwabe 2025, vol 29, provocation there is no ai in team 9 e. ethical implications: • how can ethical guidelines be developed to govern the integration of ai in teams, particularly in scenarios where ai takes on decision-making roles traditionally reserved for humans? • what are the long-term implications of ai-human collaboration for social cohesion and ethical governance in diverse cultural contexts? • do ai teammates have rights? how should incidents like harassment by humans toward anthropomorphized ai be addressed, and does this matter in the context of team dynamics? references asatiani, a., malo, p., rådberg, n. p., penttinen, e., rinta-kahila, t., & salovaara, a. (2021). sociotechnical envelopment of artificial intelligence: an approach to organizational deployment of inscrutable ai systems. journal of the association for information systems, 22(2), 325–352. doi.org/10.17705/1jais.00664 chowdhury, s., budhwar, p., dey, p. k., joel-edgar, s., & abadie, a. (2022). ai-employee collaboration and business performance: integrating knowledge-based view, sociotechnical systems and organisational socialisation framework. journal of business research, 144, 31–49. doi.org/10.1016/j.jbusres.2022.01.069 cranefield, s., winikoff, m., dignum, v., & dignum, f. (2017). no pizza for you: value-based plan selection in bdi agents. proceedings of the twenty-sixth international joint conference on artificial intelligence. doi.org/10.24963/ijcai.2017/26 dafoe, a., bachrach, y., hadfield, g., horvitz, e., larson, k., & graepel, t. (2021). cooperative ai: machines must learn to find common ground. nature, 593, 33--36. doi.org/10.1038/d41586-021-01170-0 dennett, d. c. (1971). intentional systems. the journal of philosophy, 68(4), 87--106. doi.org/10.2307/2025382 dolata, m., katsiuba, d., wellnhammer, n., & schwabe, g. (2023). learning with digital agents: an analysis based on the activity theory. journal of management information systems, 40(1), 56–95. doi.org/10.1080/07421222.2023.2172775 floridi, l., & sanders, j. w. (2004). on the morality of artificial agents. minds and machines, 14, 349–379. doi.org/10.1023/b:mind.0000035461.63578.9d friedman, b., kahn, p. h., borning, a., & huldtgren, a. (2013). value sensitive design and information systems. in j. van den hoven, p. e. vermaas, & i. van de poel (eds.), early engagement and new technologies: opening up the laboratory (pp. 55–95). springer. hacker, j., trier, m., & richter, a. (2024). unfolding the contextual nature of enterprise social media use: a morphogenetic approach. european journal of information systems. doi.org/10.1080/0960085x.2024.2339217 huxham, c., & vangen, s. (2013). managing to collaborate: the theory and practice of collaborative advantage. routledge. australasian journal of information systems richter & schwabe 2025, vol 29, provocation there is no ai in team 10 jarrahi, m. h., lutz, c., boyd, k., oesterlund, c., & willis, m. (2023). artificial intelligence in the work context. journal of the american society for information science and technology. doi.org/10.1002/asi.24730 klein, s., & watson-manheim, m. b. (2021). the (re-)configuration of digital work in the wake of profound technological innovation: constellations and hidden work. information & organization, 31(4), 100377. doi.org/10.1016/j.infoandorg.2021.100377 mathieu, j. e., heffner, t. s., goodwin, g. f., salas, e., & cannon-bowers, j. a. (2000). the influence of shared mental models on team process and performance. journal of applied psychology, 85(2), 273–283. doi.org/10.1037/0021-9010.85.2.273 memmert, l., & tavanapour, n. (2023). towards human-ai collaboration in brainstorming: empirical insights into the perception of working with a generative ai. ecis 2023. nass, c., & moon, y. (2000). machines and mindlessness: social responses to computers. journal of social issues, 56(1), 81–103. doi.org/10.1111/0022-4537.00153 rao, a. s., & georgeff, m. p. (1995). bdi agents: from theory to practice. in icmas (vol. 95, pp. 312–319). richter, s., & richter, a. (2024). human-ai collaboration in the metaverse: how to research the future of work? in proceedings of the thirty-second european conference on information systems (ecis 2024), paphos. salas, e., sims, d. e., & burke, c. s. (2005). is there a “big five” in teamwork? small group research, 36(5), 555–599. doi.org/10.1177/1046496405277134 schwabe, g., sprecher, r., katsiuba, d., & dolata, m. (2025). ai-thinklets for brainstorming. hawaii international conference on system sciences hicss 2025. seeber, i., bittner, e., briggs, r. o., de vreede, t., de vreede, g. j., elkins, a., & söllner, m. (2020). machines as teammates: a research agenda on ai in team collaboration. information & management, 57(2). doi.org/10.1016/j.im.2019.103174 sudeeptha, i., mueller, w., leyer, m., richter, a., & nolte, f. (2025). use cases for prospective sensemaking of human-ai collaboration. in hicss 2025. https://arxiv.org/abs/2408.10812v1 van riemsdijk, m. b., jonker, c. m., & lesser, v. (2015). creating socially adaptive electronic partners: interaction, reasoning and ethical challenges. in proceedings of the 2015 international conference on autonomous agents and multiagent systems (pp. 1201–1206). van wynsberghe, a. (2013). designing robots for care: care centered value-sensitive design. science and engineering ethics, 19(2), 407–433. doi.org/10.1007/s11948-011-93436 willems, t., & hafermalz, e. (2021). distributed seeing: algorithms and the reconfiguration of the workplace, a case of 'automated' trading. information & organization, 31(4), 100376. doi.org/10.1016/j.infoandorg.2021.100376 australasian journal of information systems richter & schwabe 2025, vol 29, provocation there is no ai in team 11 acknowledgements we thank eva bittner, andrew burton-jones, julian marx, ishara sudeeptha, and jingyu zheng, and especially michael winikoff, whose thoughtful feedback and insights helped shape our ideas and ultimately improve this article. copyright © 2025 richter & schwabe this is an open-access article licensed under a creative commons attribution-non-commercial 4.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v29.5753 https://creativecommons.org/licenses/by-nc/4.0/deed.en https://creativecommons.org/licenses/by-nc/4.0/deed.en https://doi.org/10.3127/ajis.v29.5753 paper title (paper title style) australasian journal of information systems naqvi, rehman & islam 2020, vol 24, research article a hyperconnected smart city framework 1 a hyperconnected smart city framework: digital resources using enhanced pedagogical techniques naureen naqvi school of computing & mathematics charles sturt university nnaqvi@csu.edu.au sabih ur rehman school of computing & mathematics charles sturt university md zahidul islam school of computing & mathematics charles sturt university abstract recent technological advancements have given rise to the concept of hyper-connected smart cities being adopted around the world. these cities aspire to achieve better outcomes for citizens by improving the quality of service delivery, information sharing, and creating a sustainable environment. a smart city comprises of a network of interconnected devices also known as iot (internet of things), which captures data and transmits it to a platform for analysis. this data covers a variety of information produced in large volumes also known as big data. from data capture to processing and storage, there are several stages where a breach in security and privacy could result in catastrophic impacts. presently there is a gap in the centralization of knowledge to implement smart city services with a secure architecture. to bridge this gap, we present a framework that highlights challenges within the smart city applications and synthesizes the techniques feasible to solve them. additionally, we analyse the impact of a potential breach on smart city applications and state-of-the-art architectures available. furthermore, we identify the stakeholders who may have an interest in learning about the relationships between the significant aspects of a smart city. we demonstrate these relationships through force-directed network diagrams. they will help raise the awareness amongst the stakeholders for planning the development of a smart city. to complement our framework, we designed web-based interactive resources that are available from http://ausdigitech.com/smartcity/ keywords: smart cities; privacy; security; iot; big data 1 introduction the adoption and advancement in technology have given rise to smart cities that are acting as enablers to improve the liveability of cities by providing better outcomes for their residents. to improve the quality of life of citizens, smart cities make use of information and communication technologies. it builds a seamlessly interconnected society of devices capturing data from multiple sources, analysing it, and supporting data-driven decisions (achmad, nugroho, djunaedi, & widyawan, 2018; ismail, 2016). over the last decade, the speed at which the population of cities is growing has reinforced the importance of using technology to improve service outcomes for citizens. for example, smart cities, with the help australasian journal of information systems naqvi, rehman & islam 2020, vol 24, research article a hyperconnected smart city framework 2 of technological assistance, can save 30–300 lives each year, reduce crime by 30–40%, lower the disease burden by 8–15%. additionally, the smart cities can help improve emergency response times by 20–35%, cut daily commutes by 15–30 minutes, and save 25–80 litres of water consumption per person per day (shah & patel, 2014). these areas are a part of the daily lives of city residents and improved efficiencies in these areas will result in greater quality of lives. and an improved governance framework. more compelling arguments suggest that a smart city is not only capable of improving the quality of services for citizens but also empowers governments for running public administration effectively (sarker, wu, & hossin, 2018). furthermore, according to studies from the united nations (un), the world population is expected to increase by close to 6 billion by 2050. this will lead to challenges that require a standard framework to help with the growth of cities in a sustainable manner (okai, feng, & sant, 2019; simonofski, asensio, de smedt, & snoeck (2017)). figure 1 sources for data collection – smart city the performance of smart cities and outcomes produced by them relies on the amount and the quality of data provided to them. to collect this data, smart cities use an integrated network of closely webbed technologies that capture large volumes of data arriving at high speeds called the internet of things (iot). figure 1 illustrates the sources for data collection within a smart city. a smart city architecture has a fundamental layer of data collection where data is ingested from multiple sources. within the smart city paradigm, data can be collected from core applications across the city. for example, external databases collecting data from stakeholder organisations, and integrated devices that are embedded in operations. additionally, this data is also captured from the business systems that are used across the administration, and the iot infrastructure that has been installed to gather insights and media sources, etc. due to enormous data collection, often the boundaries of ethics with which the data has been acquired get blurred leading to trust issues among the citizens (brauchli & li, 2015). post data acquisition by smart cities, there is often a lack of effective mechanisms in australasian journal of information systems naqvi, rehman & islam 2020, vol 24, research article a hyperconnected smart city framework 3 place to ensure the protection from invading privacy and threatening civil liberties (wang, ali, & kelly, 2015). this data is vulnerable and can be compromised by intruders, leading to compromised security of the city operations and the privacy of citizens’ data posing a security threat. our work recognises current research gaps in the domain of smart cities and presents a holistic framework. we anticipate this framework to be pivotal in shaping a strategy for data privacy and security for the portfolio of smart city services. our framework has been developed by keeping in view the stakeholders’ requirements and the impact that data privacy and security have on the operations. our framework focuses on synthesizing information within the most important areas of the smart city and highlighting the key relationships. our work comprises a comprehensive literature review following a timeline of 2007-2018. the objective was to synthesize previous publications and analyse the relationship between the work that is already available. this work has then been overlaid with the current trends within a smart city, iot, and big data analytics. we also study state-of-the art exemplary smart cities around the world to correlate their learnings with our findings. our framework has been developed in the form of an open-source interactive digital resource. this will allow other researchers to benefit from this work and build future resources using the fundamental building blocks provided. the framework follows a structured research methodology with the identification of theoretical concepts, gaps, and implementation via kinaesthetic learning methodology. our broad literature review has resulted in a knowledge base consisting of commonly used terminology used in smart cities. these keywords are supported by definitions and publication references. to develop an interactive resource, we focused on a portfolio of services that fall under the remit of a smart city to fully comprehend the challenges pertaining to security and privacy. this has helped us in devising a targeted plan for addressing challenges. furthermore, mapping the challenges to the available solutions and highlighting relationships between the key aspects delivered via pedagogical techniques. our research offers the following contributions: 1. developed a knowledge base of key terms, definitions, and relationships between important concepts relating to smart cities along with relevant references. 2. analysed the learning of state-of-the-art exemplary smart cities around the world and identified key stakeholders within the smart city framework. this framework helps in demonstrating the key aspects of privacy & security within the smart city applications. 3. presented a framework for privacy and security preserved data sharing in hyperconnected smart cities. the framework is delivered via open-source interactive digital resources highlighting fundamental building blocks and the key relationships between entities. the paper is organized as follows: in section 2, a literature review of smart cities along with examples is provided. section 3 describes our proposed framework. section 4 explains the usefulness of our framework and a discussion on our designed interactive resources. section 5 contains the conclusion of our work and proposes a future outlook of research within this domain. australasian journal of information systems naqvi, rehman & islam 2020, vol 24, research article a hyperconnected smart city framework 4 2 literature review the smart cities model presents an integrated concept of a sustainable environment, improved quality of life, and reduced cost of living, using technology to optimize services. it is a composite of people, infrastructure, technology, and services. many cities around the world have adopted digital initiatives to establish smart governance. to achieve real-time datadriven decision making, the cities collect data from interconnected sensor-enabled devices, ‘things’. the internet of things (iot) has been recognized as an efficient mechanism to achieve city urbanization. emerging technologies provide reliable technology for real-time data collection and sharing using controlled architecture following standard processes for highimpact applications (khan, pervez, & ghafoor 2014; li, cao, & yao, 2015; maheswaran & misra, 2015; popescul & radu, 2016; pacheco & hariri 2016; shahat, elragal, & bergvallkåreborn, 2017; ati & basmaji, 2018; alias balamurugan, lilian, & sasikala 2018; manjunatha & annappa, 2018; alsafery, alturki, reiff-marganiec, & jambi, 2018; puschmann, barnaghi, & tafazolli, 2018; santos et. al, 2017; lim, kim, & maglio, 2018; harris 2014). the synergy of these processes, standards, and emerging technologies will result in collecting data. this data could include sensitive information, such as health records, financial transactions, academic records, multimedia recordings, industrial grids, electricity, water systems, etc. with a hyper-connected, integrated network in place, datasets may not always be collected and in turn shared with the explicit consent of the citizens. a security breach within the smart city could allow intruders to infiltrate the network and gain access to controls. this could have a severe impact on the smart city operations, requiring the attack paths to be identified and stored by using smart infrastructures (si) that provide autonomy and control. this further reinforces the impact that security and privacy have on the smart operations and must be considered when designing the reference architecture (cárdenas & safavi-naini, 2012; hashem et al., 2016; berntzen & johannessen, 2016; doynikova & kotenko, 2017; ylmaz, ciylan, gönen, sindiren, & karacayilmaz, 2018; ghirardello, maple, ng, & kearney, 2018). figure 2: smart city framework australasian journal of information systems naqvi, rehman & islam 2020, vol 24, research article a hyperconnected smart city framework 5 our research observed that some of the key aspects of security concerns include unlawful access to information, and attacks causing physical service disruptions. on the other hand, privacy breaches include the interception of data in transit-related to the citizens’ private life. these aspects must be planned contingently when designing a smart city, for example, by establishing business resiliency, service continuity planning, and appropriate accountability. a possible way in which accountability can be established is by storing the audit trail of operations in the form of tamper-proof digital patterns via blockchain. another measure of achieving resilience within the smart city business application is to host applications on critical infrastructure and classify them based on their risk score (estevez & janowski, 2013; camwinget, sadeghi, & jin, 2016; sankaran & vishwa vidyapeetham, 2017; liu et al., 2017; chan et al., 2018; boban & weber, 2018; russell, goubran, kwamena, & knoefel, 2018). this helps to map the relationship between applications and security essentials. it is also important to note that the legacy systems should be fully secure if they have any touchpoints with the systems using the iot. smart city stakeholders concerned about security should be vigilant when making a selection in hosting their iot networks on edge or fog respectively. the design has to be in line with the principles of scalability as a smart city portfolio contains closely related applications. finally, the security essentials should be applied at all levels of the smart city applications (anthopoulos, 2015; gyrard, zimmermann, & sheth, 2018; oteafy & hassanein, 2018; pradhan, suri, fuchs, bloebaum, & marks, 2018). figure 2 illustrates a smart city framework that captures the key elements of a smart city. the top layer includes the stakeholders who are the receivers of this information. layer 2 highlights the applications of interest. layer 3 demonstrates the important data attributes produced by the city. the following layer presents a number of iot technologies used that helps collect the data. to summarize, all of the above layers revolve around layer 5, the underlying principles within a smart city. figure 3: smart city framework portfolio of applications australasian journal of information systems naqvi, rehman & islam 2020, vol 24, research article a hyperconnected smart city framework 6 from a stakeholder’s perspective, our research is aimed at smart city planners, designers, specialists, privacy and security advocates, and decision-makers. these groups have been identified to have a significant impact on how interconnected cities are designed and function. we identified real-life digital initiatives focusing on economic planning, governance, opportunities, environmental damage, and operational efficiencies. to evaluate our ideas and test them on practical benchmarks, we conducted a thorough analysis of the smart city applications, as illustrated in figure 3. these applications have been listed based on their impact on the city’s operation to learn from the lessons of the exemplary smart cities. now that we have defined the role of the portfolio of applications within the smart city framework and the types of applications that sit within the portfolio. we introduce table 1, which summarizes state-of-the art exemplary smart cities that run smart applications around the world. the challenges faced by these initiatives have been cited in publications in reputed journals. we have grouped these applications in the form of the portfolio of services that relate to that group and experienced similar challenges (foucault & moulier-boutang, 2015; feng, dawam, & amin, 2017; smolander, rossi, & pekkola 2017; de carolis, macchi, negri, & terzi 2017; johannessen & berntzen, 2018; achmad, nugroho, djunaedi, & widyawan, 2018). publications portfolio of service smart city applications exemplary smart cities okai, feng, & sant, 2019 smart utilities smart grid barcelona, spain eckhoff, & wagner, 2018 smart energy zwolle, netherlands bristol, uk el hendy, miniaoui, atalla, & hashim, 2018 smart water the island city, state of singapore el hendy, miniaoui, atalla, & hashim, 2018 smart gas san francisco, us bellini, nesi, paolucci, & zaza, 2018; el hendy, miniaoui, atalla, & hashim, 2018 smart environment smart weather seattle, us san francisco, us bellini, nesi, paolucci, & zaza, 2018 smart waste helsinki, finland eckhoff, & wagner, 2018 environment control bristol, uk eckhoff, & wagner, 2018; bellini, nesi, paolucci, & zaza, 2018 smart services smart buildings aspern, austria milton keynes, uk eckhoff, & wagner, 2018 smart education almere, netherlands waseda, japan eckhoff, & wagner, 2018 smart healthcare rio de janeiro the island city, state of singapore bellini, nesi, paolucci, & zaza, 2018; eckhoff, & wagner, 2018 smart surveillance eindhoven, netherlands glasgow, scotland california, us eckhoff, & wagner, 2018 smart homes hague, netherlands eckhoff, & wagner, 2018 intelligent industry european union eckhoff, & wagner, 2018 intelligent control european union eckhoff, & wagner, 2018 smart transportation smart networking oulu, finland estonia copenhagen, denmark chicago, us australasian journal of information systems naqvi, rehman & islam 2020, vol 24, research article a hyperconnected smart city framework 7 publications portfolio of service smart city applications exemplary smart cities bellini, nesi, paolucci, & zaza, 2018 smart mobility glasgow, scotland bellini, nesi, paolucci, & zaza, 2018 intelligent transport seattle, us okai, feng, & sant, 2019; bellini, nesi, paolucci, & zaza, 2018 intelligent parking barcelona, spain milton keynes, uk eckhoff, & wagner, 2018; bellini, nesi, paolucci, & zaza, 2018; okai, feng, & sant, 2019 smart government smart economy almere, netherlands dubai, uae eckhoff, & wagner, 2018 smart services sydney, australia eckhoff, & wagner, 2018 smart citizens zaragoza, spain eckhoff, & wagner, 2018 smart emergency services hague, netherlands rio de janeiro bellini, nesi, paolucci, & zaza, 2018; okai, feng, & sant, 2019 smart land services dubai, uae table 1: smart city data model next, we discuss the intricate design and implementation details of the proposed framework. 3 the proposed framework the learning gathered for the framework has been presented in the form of a purpose-built website with interactive resources. this section describes the artifacts produced to disseminate the knowledge. the resource development was inspired by the product development lifecycle with stages categorized as scope definition, requirements gathering, stakeholder identification, and implementation. we adopted the standard approach for building a framework by i) reviewing requirements, ii) analysing relationships, and iii) proposing a solution (elmaghraby, 2013; teoh & mahmood, 2017; yamakami, 2017; puron-cid, gil-garci, & zhang, 2015; wagner, & eckhoff, 2018). 3.1 design to deliver an immersive learning experience through our digital resources, we focused on the following five dimensions of design: i. volume – reviewed a large number of publications per year. ii. domain – the study included the iot, architecture, ontologies, privacy, security, big data, and data sharing. iii. object – challenges, solutions, architecture, and applications. iv. level – review against three dimensions i.e., smart city subject expertise, education resources, and design excellence. v. contribution – the type of contribution to the field such as best practice, field experience, policy recommendation, standard design, and empirical methods. australasian journal of information systems naqvi, rehman & islam 2020, vol 24, research article a hyperconnected smart city framework 8 design principles standard our resource useful the content produced as a result of this research is original. the references to the source have been appropriately cited. ✓ desirable the website is light and has easy to use a color scheme to reduce strain on users’ eyes. consistent layout design and fonts have been used to give the website a unified, consistent look and feel. ✓ findable search functionality is built to look for keywords without hassle. ✓ accessible the website is web content accessibility guidelines (wcag) 2.0 compliant making the website inclusive to all users. ✓ table 2: digital resource design principles with the help of the above, we opted to use gamification techniques and methods for creating an immersive experience for developing our pedagogical resources (l’heureux, grolinger, higashino, & capretz, 2017). these resources stimulate a fast learning experience among users of resources by enabling kinaesthetic techniques. next, we discuss the methodology including data collection, organization, and the presentation method used in this research. this section is followed by a discussion on the user experience. 3.1.1 methodology we reviewed literature from primary and secondary sources. a detailed spreadsheet of the relationships is available on the website. presenting relationships between variables, reviewing the irregular hierarchies, positioning the citations, and listing the glossary were some of the biggest design challenges. table 2 represents the design principles that we have used in the presentation of knowledge. design elements standard our resource mega menus the website content is arranged in mega menus. it enables the content to be categorized reducing the page length. ✓ usable information bootstrap and jquery have been used to make the website easy to use. custom components have been used to present data and interrelationships. the website is cross-platform, compatible, and responsive, that is, usable on computers, tablets & phones. ✓ vertical tab allows efficient maneuvers between two sub-topics and contains a significant amount of information in the form of switchable tabs. ✓ accordion the accordions are horizontally expandable collapsible boxes containing the content reducing the vertical scroll of the page. ✓ html standard elements are used for designing. ✓ table 3: page design elements our reviews yielded approximately 500 variables, in 38 categories, and over 143 references. we used the design principles in table 2 to organize the data into simple user-friendly datarich resources. presenting data on different levels or hierarchies posed an implementation australasian journal of information systems naqvi, rehman & islam 2020, vol 24, research article a hyperconnected smart city framework 9 challenge. table 3 demonstrates the design elements used to render content dynamically – this required no added customisation to the code. we selected the technology such that all the components reside on the client-side with reduced round trips to the server to fetch the data for each node. our resources are cross-browser, cross-platform, and cross-devices. this means that they work effectively on four tested browsers (google chrome, mozilla firefox, microsoft edge, and apple safari), across both microsoft and apple os platforms. the resources perform equally well on multiple devices such as computers and mobile devices. table 4 shows the selected technology for our solution i.e., html5, bootstrap, php, json, javascript jquery, and amcharts. in the next section, we explain the user experience and usability of the resources. technology stack standard our resource ux specifications • html5 for developing seo-friendly web resource • php – html friendly object-orientation package • mega menus – ease of navigation with no scrolling • vertical tabs – intuitive with better use of horizontal space • accordions – responsive design, collapsible menus • cross-browser – works responsively on all browsers (chrome, opera, firefox, edge, safari and, ie onwards) • cross-device – works on phones, tablets, and computers • cross-platform – works on ios, android, and windows • wcag2.0 (world content accessibility guidelines) by w3c (world web consortium). ✓ technical specifications • bootstrap – customizable look & feel for the plugins used • jquery – seo friendly, clean, extensible and libraries used • json (javascript object notification)–data stream sync • amcharts– building intuitive & intelligent digital resources ✓ table 4: technology stack 3.1.2 user experience usability, the transfer of knowledge, and ease of navigation have been the core principles behind designing this website. the website has been classified into five sections: i. the “overview” provides a holistic view. ii. sub-topics illustrate “challenges”, “solutions”, “applications” and “architecture”. iii. “application relationship” and “security relationships” include relationships derived from sources and anecdotal evidence. iv. the “definitions” page is a knowledge base of concepts used in smart cities. v. the “surveys” section captures the possible considerations that smart city stakeholders must plan for when introducing new technology within applications. in the next section, we discuss the implementation details of our framework, including the areas of interest for the resource development and the benchmark set for the evaluation of resources. australasian journal of information systems naqvi, rehman & islam 2020, vol 24, research article a hyperconnected smart city framework 10 3.2 implementation we classified the key aspects of the framework into the areas of interest. later, we mapped the relationship between these aspects by highlighting the areas that would benefit stakeholders the most. as a preliminary step, we identified the key challenges, their solution, the applications they impact, and the technology used in the development of the city. we then moved onto understanding the relationships smart city stakeholders would benefit from the most. 3.2.1 delivery techniques the visual interaction offered by our system has been developed using the following techniques. i. tree maps: the tree maps provide a structured model for representing information in a layered fashion, highlighting the parent-child hierarchy between parameters of interest. we used tree maps to capture information on multiple levels from level 0 i.e., a high-level tree map to level 5, a low-level tree map demonstrating the children node under parent nodes. this is an effective method of organizing the information into categories. ii. force directed trees: the force directed trees offer an effective mechanism of visualizing pre-organized information in the form of mind maps. it is easier to understand the hierarchical nature of information and it is used as a standard method of data visualization in the software development lifecycle to create valuable knowledge. iii. force directed networks: the force directed networks is an extension of the force directed trees and offer to map of relationships in the form of a network. they are used as a standard technique in the software development lifecycle to visualize relationships. 3.2.2 areas of interest within smart city framework the development of the interactive resources available on our website has been aided by the literature review conducted. the learnings have been orchestrated in the form of a tree diagram before being converted to resources as illustrated in figure 4. all tree diagrams have been drawn from the high-level (being level 0) to the lowest level, capturing granular details under section ii) and iii) above. the following are the four areas of interest that were used to develop the digital resources and understand the relationships. figure 4: tree map – smart city framework i) the challenges regarding privacy-protected data sharing that emerge as a result of using new technologies within smart cities form the first area of interest. these challenges can be summarized into two categories – technical and functional challenges. technical challenges emerge as connected devices within smart cities australasian journal of information systems naqvi, rehman & islam 2020, vol 24, research article a hyperconnected smart city framework 11 require a non-conventional approach towards big data. this data has doubled in size every two years since 2011 and is predicted to increase from 130 exabytes in 2005 to 40,000 exabytes this year (a 300-fold increase). this can become a trigger for an increase in privacy-related issues. functional challenges arise due to the dynamic nature of smart city applications that have seen exponential growth and require policies to be developed to protect privacy (terzi, terzi, & sagiroglu, 2016; kolozali et al., 2018; nesi, pantaleo, paolucci, & zaza, 2018; siris, fotiou, mertzianis, & polyzos, 2019). our resource on challenges is available from http://ausdigitech.com/smartcity/challenges.html. section 4.2 provides a detailed discussion on this topic. ii) the second area of interest is the solutions that are available to tackle these challenges. however, it is interesting to note that there is no centralization of knowledge available at this stage. with our solution, we researched these areas and synthesized this information within a framework that the stakeholders can adopt and adapt when planning their smart city applications and related architecture. this solution has been derived from comparisons, expert views of prior studies, and questionnaires (belgaum, alansari, jain, & alshaer, 2018). we researched the key areas of big data, artificial intelligence (ai), and cloud automation along with their roles in the smart cities. our resources on this solution is available from http://ausdigitech.com/smartcity/solutions.html. section 4.3 has a detailed discussion. iii) the third area of interest is identifying the portfolio of applications within the city. with the significant adoption of cloud computing, these applications are increasingly being hosted in the cloud. under the transformative ecosystem, the use of the cloud to run smart city applications provides scalability, interoperability, elasticity, and maintainability at a low cost with fully encrypted data sharing (alkhamisi, nazmudeen, & buhari, 2016; venticinque & amato, 2018; abi sen, eassa, & jambi 2018). our resource on applications is available from http://ausdigitech.com/smartcity/applications.html with a detailed discussion in section 4.4. iv) the fourth and the most important area of the smart city framework is its architecture. the planning and development of a smart city requires an architecture that can present a holistic picture of information systems, mission-critical, and distributed systems. this is key to building architectural resources in line with the services that need to be supported and consideration for architectural responses (budhiputra & putra, 2016; sikora-fernandez & stawasz, 2016; assante et al., 2018; bass, sutherland, & symons, 2018; cui et al., 2018; sindhusha & bharathi, 2018). our resources on architecture are available from http://ausdigitech.com/smartcity/architecture.html. there is a detailed discussion in section 4.5. figure 5 illustrates the overview of our interactive resource on the high-level smart city framework, highlighting the areas of interest defined above. next, we evaluate our work against the key industry standards for assessing digital resources. australasian journal of information systems naqvi, rehman & islam 2020, vol 24, research article a hyperconnected smart city framework 12 figure 5: force-directed tree – overview – smart city australasian journal of information systems naqvi, rehman & islam 2020, vol 24, research article a hyperconnected smart city framework 13 table 5: evaluation criteria 3.2.3 benchmark for evaluation table 5 explains the high-level benchmarks for evaluating our framework design against the standard best practices. i. the literary benchmark i.e. the discovery and analysis of various related concepts and their significance within smart cities (liu, choo, huang, & au, 2017). ii. the functional benchmark includes the development of interactive digital resources. furthermore, the presentation of complex information in the form of the tree maps, force-directed trees, and force-directed networks is another important measure of a functional fit (ainane, ouzzif, & bouragba, 2018). iii. the technical benchmark includes the technical quality of the open-source interactive resources. the system has the ability for the stakeholders to leverage the existing knowledge and scale the framework (ahn, lee, kim, & hwang, 2016). benchmark by group measure of evaluation our resource literary benchmarks (findings of the research, their orchestration, and presentation within the website) • iot technologies • privacy enabling technologies • smart city services • smart city paradigm • architecture • relationships • associative relationships • market observations • glossary ✓ functional benchmarks (overall look and feel, ease of navigation, usability, accessibility and the empathy for the end-to-end stakeholder journey) • user experience • colour confusion • the golden triangle • feeding the imagination • informative • retention • reading comprehension • student achievements • thinking and learning • critical thinking • pedagogical techniques • design • readability & legibility • accessibility • clarity • maintenance • financially viable • easy to maintain ✓ technical benchmarks (the work to be made reproducible and extensible) • min. server-side roundtrip • dynamic updates ✓ next, we discuss the usefulness of our framework by describing the outputs of our research. australasian journal of information systems naqvi, rehman & islam 2020, vol 24, research article a hyperconnected smart city framework 14 4 the usefulness of our framework while emerging technologies have shown immense promise to encourage the growth of smart cities, their reliability and full adaptability are yet to be seen. additionally, their trust is yet to be matured, especially in government bodies and communities (bajramovic, waedt, ciriello, & gupta, 2016). therefore, a trust framework needs to be established to offer best-practice knowledge on privacy-protected data sharing within smart cities (hassanain et. al., 2019; ferrer, marquès, & jorba, 2019). we have attempted to consolidate the knowledge areas where best practices and standards need to be developed. table 6 presents a summary of our web system. page configuration standard overview the purpose of this project is to develop state of the art digital resources adopting newer ways of learning and introducing concepts of security and privacy for smart cities. this project dives deep into the challenges faced by smart cities on daily basis, offers solutions to the issues faced by the digitalization of the cities with a focus on a specific portfolio of services, in turn, shedding light on the architectural framework that is used for enabling privacy and security in smart cities. this project is a modern-day version of learning about the new concepts on smart city and is the version 1.0 with the intention that the team keeps contributing to the knowledge area smart city focusing on iot and big data. challenges smart cities face several challenges when it comes to keeping citizen's data private and secure from the time of collection and analysis. our digital resources provide the challenges faced by cities classified into four categories: technological issues, privacy & security issues, data sharing issues, and open areas. solutions many smart cities have established smart city initiatives to meet the privacy and security requirements in turn protecting citizen's data. our digital resources focus on providing a solution to the challenges defined. this has been classified into four main areas: solution on technological issues, privacy & security issues datasharing issues, and solution recommendations for open areas. applications smart cities around the world have many service offerings such as transactional and consultative services which can be divided into five main smart categories: services, transportation, utilities, environment, and government. architecture smart cities focus on developing their intelligent assets and digital services on the back of architectural building blocks such as fog computing, cloud computing, smart city modeling, and iot ontologies. this plays an important role in managing the privacy & security of the technologies involved in the city. applications relationships the application relationship diagram illustrates the fundamental building blocks of the application portfolio in a smart city. this diagram demonstrates the relationship between the building blocks such as privacy enabling technologies, iot ontologies, and security techniques used in data sharing with the smart city. this includes analysis such as how data sharing takes place in smart governance application, which technologies are used, how can operational efficiencies be improved, how involving the citizens can have better outcomes. here, a connection between these blocks can be seen with the ability to determine how these technologies can be used for preserving privacy and protecting security. australasian journal of information systems naqvi, rehman & islam 2020, vol 24, research article a hyperconnected smart city framework 15 page configuration standard security relationships the security relationship diagram illustrates the fundamental building blocks of privacy protection and security in a smart city. this diagram demonstrates the relationship between these building blocks such as privacy elements, security design, privacy & security recommendations, smart technologies used in privacy protection, types of privacy and security issues, type of attacks and attackers, sources of data attacks, and privacy enabling technologies. here, a connection between these blocks can be seen with the ability to dive deep into how security can be addressed in a smart city. this involves sense checking the entire data management lifecycle using iot, big data, and inter-agency data sharing. table 6: web-system overview smart city framework next, we identify the key stakeholders who will receive the maximum value from our framework. furthermore, we define the interactive digital resources on the challenges related to privacy-protected data sharing, and a solution to mitigate these challenges. additionally, we identify the smart city applications that this solution can be used on and architectural measures to ensure privacy-protected data sharing. to explain our framework in detail, we establish the relationship between privacy and security, and the impact on smart city operations. 4.1 identification of stakeholder for our interactive resources this section provides an overview of the stakeholders identified in our research. these stakeholders have a primary interest in the smart city, its functions and frameworks, the overall experience, needs, pain points, and outcomes revolve around them. with our resources, we aim to create an impactful engagement among stakeholders through interactivity. the success criterion is to enhance the users’ motivation, enabling them to achieve their goals. in the initiation phase of this research, we tried to identify a target audience for the knowledge – primarily in the form of research that exists and resources we collated as we progressed through the project. this research presents an opportunity to engage with multidisciplinary stakeholders. additionally, this enables collaboration with researchers who are authorities on the subject. in the past, frameworks were based on technology rather than people. we have observed that for smart cities, the framework must primarily serve people, or roles, mainly stakeholders. the key stakeholders identified were the it experts, data custodians, domain experts, legitimate and malicious providers, consumers, and governing bodies (sanseverino, sanseverino, vaccaro, macaione, & anello, 2017; tay, supangkat, cornelius, & arman, 2018; andrisano et al., 2018, yigitcanlar et. al., 2018; kodali, azman, & panicker, 2018; and rjab, ben, & mellouli, 2018). table 7 depicts the stakeholder map along with their key pain points when designing the smart city. our resources are targeted at addressing the pain points and keeping in view the interests of these stakeholders throughout the process of implementing our digital resources. australasian journal of information systems naqvi, rehman & islam 2020, vol 24, research article a hyperconnected smart city framework 16 stakeholder role profile pain points interest iot service provider organisation (alliance for telecommunications industry solutions, 2018) • develops iot devices and technologies • lack of visibility into client issues (shah & patel, 2014). • limited understanding of other key aspects of the smart city (nam & pardo, 2011). • limited time to perform research on case studies (sanseverino et al., 2014). the smart city framework, in this research, will act as a reference guide keeping into consideration various security and privacy parameters, technologies, methods and processes of the smart city in the development of the interactive resources. local councils, city & state government departments (jiong & gubbi et al., 2014) • manages cities resources in order to achieve the best outcomes and living environment for citizens in a sustainable manner • consumes iot technologies and various data analytics methods to develop a smart city. • lack of knowledge of smart city practices, models, and challenges. (berntzen, & johannessen, 2016) • limited time to perform research on case studies (sanseverino et al., 2017) • limited access to the domain expertise summarised for a business case. (lim, kim, & maglio, 2018) the smart city framework, in this research, provides a measure of quality assurance, for benchmarking purposes, and will act as the reference guide for systems development and business case development. other organisations (private sector/ public-sector) (iot alliance australia, 2017) • operates on businesses across various industry verticals such as healthcare, shopping centre, utilities, transportation, etc. • consumes iot technologies and various data analytics methods to develop a smart city • lack of visibility into privacy and security related issues for citizens (puron-cid, gabriel, 2015) • lack of knowledge on smart city practices, models and challenges (yigitcanlar et al., 2018) • limited time to perform research on case studies (sanseverino et al., 2017) • limited access to the domain expertise summarised for a business case. (lim, kim & maglio, 2018) the smart city framework, in this research, provides a measure of quality assurance, for benchmarking purposes, in performing technology assessment and evaluation of options and will act as the reference guide for systems development and business case development. researchers (ismail 2016) • consumes knowledge and domain expertise that has been established in the context of smart city • reproduce, modify existing knowledge and develop new knowledge on smart city • limited amount of wellorchestrated knowledge and domain expertise available in the context of a smart city (anthopoulos, 2015) • time must be expensed in order to develop a relationship between key areas of the smart city (sikora-fernandez, dorota, stawasz, & danuta, 2016) the smart city framework, in this research, will act as a one-stop-shop of the key areas under smart city incl. privacy, security, and big data providing details on the key challenges, implemented solutions, possible applications, and architecture). australasian journal of information systems naqvi, rehman & islam 2020, vol 24, research article a hyperconnected smart city framework 17 stakeholder role profile pain points interest citizens (simonofski, asensio, de smedt, & snoeck, 2017) • consumer of services in a smart city that employs various complex tools, methods, and technologies • lack of transparency and visibility within processes leading to their apprehensions in the use of the tools available in a smart city (johannessen, & berntzen, 2018). • concerns for privacy and security issues and lack of understanding to establish trust (elmaghraby, & adel, 2013). the smart city framework, aids in breaking down complex concepts into simple understanding providing subject matter knowledge and domain expertise to establish a transparent process and assure quality measures adopted in the security & privacy of citizens. table 7: stakeholder map – smart city framework in the next section, we provide a detailed explanation of our digital resources. australasian journal of information systems naqvi, rehman & islam 2020, vol 24, research article a hyperconnected smart city framework 18 figure 6: force-directed tree – challenges – smart city framework australasian journal of information systems naqvi, rehman & islam 2020, vol 24, research article a hyperconnected smart city framework 19 4.2 challenges related to privacy-protected data sharing within smart cities emerging technologies are forming the building blocks of smart cities, but there is a lack of a comprehensive and established framework for sharing data with relevant bodies within a smart city. as part of our research, we attempt to establish the relationships between core concepts of privacy and security in the context of smart cities. privacy is a state in which an individual is not observed by others, while security is a state of being free from threat. privacy protection is delivered into two steps, i) privacy by design (pbd) focused on which data to secure, where to store it and who to authorize access, ii) privacy enhancing technologies (pet) – an implementation approach offering tools to preserve privacy by reducing personal data, coupled with an inability to identify individuals. security is often followed by enforcing legislation and policies to avoid breaches. the data sharing exposes citizens’ data, including personally identifiable information, and possible attacks to the smart city systems. secure data transfer is another important area requiring attention as it occurs in many smart city applications. the over-collection of data may also lead to ethical concerns (alohali, merabti, & kifayat, 2014; memmi, kapusta, & qiu, 2015; cao et al., 2016; moreno et al., 2017; moura & serrao, 2016; steuer, benabbas, kasrin, & nicklas, 2016; elmisery, sertovic, & gupta, 2017; tawalbeh, tawalbeh, song, & jararweh, 2017; mohammadi, al-fuqaha, guizani, & oh, 2018; tanaka, de barros, & de souza mendes, 2018; liu, zhang, & fang, 2018; soultatos et al., 2018; rastogi, singh, & singh, 2018; essa, al-shoura, al-nabulsi, al-ali, & aloul, 2018). to understand these issues, we created an interactive resource illustrated in figure 6 that consolidates these issues so that the best practices may be proposed, keeping data security and citizens’ privacy at the heart of the design of this framework. 4.3 solutions to achieve privacy protected data sharing within smart cities in light of the challenges outlined in the previous section, the introduction of emerging technology in smart cities brings an increase in ethical issues related to the stage of data. since the technologies used for data capture are ubiquitous in nature, it is important to ensure that the cloud they are hosted on is fully secured via encryption, with traceability via a cryptosystem. the next issue is data sharing – all interagency systems must be interoperable, fully secured, and have a strong audit mechanism in place. other elements that focus on ensuring a secure architecture with privacy by design to consider are measures for integration and abstraction capability, individual intelligence and local reasoning, learning and adaptation, and human-centric services (luo, ren, hu, wu, & lou, 2017; masduki, ramli, & salman, 2017; fernandes, rahmati, eykholt, & prakash, 2017; esposito et. al., 2018; xie, wang, wang, & chang, 2018; zhang & zahng, 2018; alketbi, nasir, & talib, 2018; bellini, nesi, paolucci, & zaza, 2018; pradhan, fuchs, & johnsen, 2018; sinaeepourfard, krogstie, & peterse, 2018; badii et al., 2018; siddiqui, tayan, & khan, 2018). to synthesize these solution measures, we derived our resource through mapping the key challenges and solution e.g., ethical challenges, lessons learned, approaches. we identified the solution to the pain points of the various types of stakeholders within the smart city. we also investigated smart city best practices for the adoption of new technology. additionally, through our knowledgebase, we offer an explanation on entities to provide a guidance to the stakeholders, available from http://ausdigitech.com/smartcity/definitions.html. australasian journal of information systems naqvi, rehman & islam 2020, vol 24, research article a hyperconnected smart city framework 20 figure 7: force-directed tree – solution – smart city framework australasian journal of information systems naqvi, rehman & islam 2020, vol 24, research article a hyperconnected smart city framework 21 figure 7 illustrates our resource related to the solution we have synthesized and proposed as part of the smart city framework. this resource addresses solutions for the challenges associated with the increased use of emerging technologies focusing on privacy, security, data sharing, and open areas, such as scalability and extensive use of things. 4.4 portfolio of applications within the smart city using technology for data sharing countries have different goals for the adoption of technology to improve the quality of service, achieve operational efficiencies, interagency data exchange, and data-driven decision making by the administration. in india, the government has a mission to develop 100 smart cities by 2020. this can be achieved by adopting a wide range of emerging technologies offering an opportunity for agencies to collaborate. with time, citizens have moved from passive to active roles by conducting transactions, consulting with government online, and participating through digital forums. smart city applications help with managing day-to-day aspects of life in an integrated fashion. this helps to provide not only excellent service delivery for citizens but also to drive great stakeholder outcomes (layne & lee, 2001; haiyan, 2016; chakrabarty & engels, 2016; kor, pattinson, yanovsky, & kharchenko, 2016; agarwal & tripathi, 2017; bibri & krogstie, 2017; beleuta & delgado merce, 2017; liu, zhang, & fang, 2018; kesswani & kumar, 2018; alliance for telecommunications industry solutions, 2018; ryu, sharma, jo, & park, 2019). figure 8 illustrates our contribution to the synthesis of information on how technology is embedded within the smart city with the following suite of applications. 1. smart services: focused on improving the quality of life by delivering digital services. 2. smart transportation: offers an efficient road network, traffic mobility, and predictability, and shipping value chain (swarnkar & bhadoria, 2017; feibert, hansen, & jacobsen, 2017). 3. smart environment: reduce the carbon footprint, monitor air quality and manage waste (de carolis, macchi, negri, & terzi, 2017; borges et al., 2017; lokuliyana et al., 2018). 4. smart utilities: offer transactional services accessible via digital channels to achieve sustainable water quality, telephone, mobile network, and electricity (hong & liu, 2019). 5. smart government: allows citizens to interact with the government digitally (west, 2004). australasian journal of information systems naqvi, rehman & islam 2020, vol 24, research article a hyperconnected smart city framework 22 figure 8: force-directed tree – applications – smart city framework australasian journal of information systems naqvi, rehman & islam 2020, vol 24, research article a hyperconnected smart city framework 23 4.5 architectural considerations to ensure privacy-protected data sharing research suggests that by this year the number of connected “machine to machine” devices will be about 50 billion. when a smart city is designed, the architectural framework of existing systems and the modern-day technologies to be adopted by the city, play a significant role in its planning. there are several layers of abstraction involved from data collection to its use in a smart city application. these layers include the data collection layer, data ingestion layer, data processing, data analysis layer and data presentation layer. sensors, network, and connectivity play a critical role in the data collection and ingestion layers. big data analysis plays a significant role in the data processing and analysis layers, for knowledge extraction to actionable insights. data sharing and exchange channels play a major role in the data presentation layer within the smart city application (tragos, fragkiadakis, angelakis, & pöhls, 2016; marjani et al., 2017; elsaeidy et al., 2017; achmad, nugroho, & djunaedi, 2018; riboni, 2019). the other significant areas of consideration when designing smart city architecture are to ensure interoperability, resilience, fault-tolerance, scalability, and modularity. figure 9 represents our resource representing smart city architectural considerations. the smart city applications are hosted in the cloud using either fog, or edge computing. in fog computing, the data is aggregated, processed and analysed away from sensor whereas, in edge computing, all functions are performed closer to the sensors. researchers prescribe to use multi-factor authentication for privacy protection and security mitigation techniques such as encryption and security by design. the architectural considerations vary for the range of smart city applications having high to low impact on operations by maintaining the availability and integrity, authentication and confidentiality (inukollu, arsi, & rao ravuri, 2014; yan, yu, & ding, 2017; wang, yang, xavier, & li, 2018; iot alliance australia, 2017; alromaihi, elmedany, & balakrishna, 2018; falco, viswanathan, caldera, & shrobe, 2018; yadav & vishwakarma, 2018). australasian journal of information systems naqvi, rehman & islam 2020, vol 24, research article a hyperconnected smart city framework 24 figure 9: force-directed tree – architecture – smart city framework australasian journal of information systems naqvi, rehman & islam 2020, vol 24, research article a hyperconnected smart city framework 25 4.6 smart city applications relationship model with the aspects of data sharing the smart city application relationship model demonstrates relationships between the portfolio of applications and key aspects of data sharing in a privacy-preserving fashion. the smart city application portfolio comprises applications, and services provided by the city resulting in generating, analysing and sharing a large volume of data. with this relationship diagram, we intend to understand the impact that technology has on each of these applications and the techniques to mitigate privacy, security, and data sharing related issues. this will enable smart city stakeholders to consider and plan for these elements at the time of designing smart city architecture (fernandez molanes, amarasinghe, rodriguez-andina, & manic, 2018; mohammadi & al-fuqaha, 2018). since numerous relationships exist among nodes within a relationship model illustrated in figure 10, for the ease of demonstration, we elaborate “smart governance” highlighted in black in the figure below. the interactive resource for this relationship model is available from http://ausdigitech.com/smartcity/apps-relationship.html. smart governance is a branch of a smart city that focuses on providing services to citizens via digital channels from the submission of an application, to the termination of that service. the applications that form part of smart governance are interconnecting frameworks of data sharing for smart citizen services via online portals, mobile applications, qr codes, and online payment gateways. smart technologies that are used for the enablement of smart governance include smart cards, cloud, open data, ubiquitous computing, and other iot devices for sensing data. these methods of data collection trigger privacy concerns, which can be mitigated by using security preserving methods and privacy protection techniques such as privacy by design, i.e. to weave privacy in the dna of the application. to ensure that data is analysed, various methods of data sharing can be used. for example, just enough data collection, establishing secure exchanges and applying ethical principles when undergoing peer-to-peer exchange. australasian journal of information systems naqvi, rehman & islam 2020, vol 24, research article a hyperconnected smart city framework 26 figure 10: force-directed network – smart city application relationship – smart city framework australasian journal of information systems naqvi, rehman & islam 2020, vol 24, research article a hyperconnected smart city framework 27 4.7 privacy and security relationship model with aspects of smart city applications privacy and security must be considered as fundamental building blocks of data collected, processed, and shared by the smart cities. the privacy and security relationship model helps stakeholders understand the areas to focus on when designing architecture. through our relationship model, we highlight the relationship between the technology used in smart cities and the types of attacks attributed to privacy and security within a smart city. additionally, we correlate this to the types of attackers involved, the data sources they can breach, and the privacy enabling technologies used. we offer the privacy and security recommendations that can help save the cities from those attacks or take appropriate measures for mitigation (luo et. al., 2019; liu, nakauchi, & shoji, 2018; zhang et al., 2015; feng, & zhao, 2018; kuan et al., 2017). since numerous relationships exist within a relationship model as illustrated in figure 11. for the ease of demonstration, we elaborate on “data sources for an attack” highlighted in black in the figure below, also available from http://ausdigitech.com/smartcity/securityrelationship.html. data sources for attack provide information on four types of data sources that can be attacked including: • attacks based on the data that has been leaked during a data exchange; • data that has been used for observation purposes; • data that has been repurposed, i.e. it was used for one purpose and later on disposed of or reused for other purposes; and finally • data that has been published but has exposed too much information about the information that has not been published. we demonstrate how data sources for attack can be interrelated to profiles of attackers – this helps trace back to the data source where the breach took place. based on the relationship of data sources to types of data, it is possible to determine the type of attacks that various types of data sources can trigger. for example, repurposed data is likely to undergo a collusion attack or background knowledge attack, whereas published data is likely to undergo an identity attack or outside forgery attack. the profile of an attacker can be determined once the source of data is identified, i.e., an attack on observable data is most likely to be carried out by the service provider, whereas, an attack caused due to leaked data pragmatically relates to third parties involved. australasian journal of information systems naqvi, rehman & islam 2020, vol 24, research article a hyperconnected smart city framework 28 figure 11: force-directed network – smart city security relation – smart city framework australasian journal of information systems naqvi, rehman & islam 2020, vol 24, research article a hyperconnected smart city framework 29 5 conclusion smart city operations thrive on the data collected within the city. we identified that the security and privacy of this data are paramount in the real-time decision-making process. a security or privacy breach can have a significant impact not only on the operations of the city and quality of service outcomes but can also result in trust issues among the citizens and the administration. to address this challenge, we designed a holistic smart city framework that provides the building blocks and highlights the relationship between them. these building blocks are focused on the important aspects of a city, including the challenges faced, solutions offered, applications in consideration, and architectural recommendations. we also derived a knowledge base with several terms used in the context of smart cities to help future researchers and smart city stakeholders to navigate through the problem areas effectively. furthermore, to deliver an impactful learning experience, we adopted pedagogical techniques to present our framework using web-based interactive resources, which impart on-demand knowledge to users. we intend to undertake further research to explore the evolving trends within smart city domains and update the digital resources periodically. additionally, we intend to extend our website, which contains interactive resources in the form of a content delivery platform so that researchers and smart city stakeholders can contribute towards the knowledge. this adaptation of our website into a centralized portal will offer a platform for researchers to digitally configure resources. this will further enable them to focus on the targeted delivery of content according to the role and interests of the stakeholder focusing on emerging technologies within smart cities. acknowledgements the researchers would like to thank and acknowledge the school of computing and mathematics, charles sturt university for its support in making this research possible. references abi sen, a. a., eassa, f. a., & jambi, k. (2018). preserving privacy of smart cities based on the fog computing. lecture notes of the institute for computer sciences, social-informatics and telecommunications engineering, lnicst, 224, pp. 185–191. https://doi.org/10.1007/978-3319-94180-6_18 achmad, k. a., nugroho, l. e., & djunaedi, a. (2018). smart city model: a literature review. 2018 10th international conference on information technology and electrical engineering (icitee) smart, pp. 488–493. https://www.researchgate.net/publication/328982472_smart_city_model_a_literature_ review achmad, k. a., nugroho, l. e., djunaedi, a., & widyawan. (2018). smart city for development: towards a conceptual framework. proceedings – 2018 4th international conference on science and technology, icst 2018. https://doi.org/10.1109/icstc.2018.8528677 agarwal, n., & tripathi, a. (2017). big data security and privacy issues: a review. international journal of innovative computer science & engineering, 2(4), pp. 12–15. retrieved from www.ijicse.in australasian journal of information systems naqvi, rehman & islam 2020, vol 24, research article a hyperconnected smart city framework 30 ahn, j., lee, j. s., kim, h. j., & hwang, d. j. (2016). smart city interoperability framework based on city infrastructure model and service prioritization. icufn 2016, 337–342. retrieved from https://doi.org/10.1109/icufn.2016.7537044 ainane, n., ouzzif, m., & bouragba, k. (2018). data security of smart cities. acm, pp. 1–13. https://doi.org/10.1145/3286606.3286866 alias balamurugan, a., lilian, j. f., & sasikala, s. (2018). the future of india creeping up in building a smart city: intelligent traffic analysis platform. proceedings of the 2nd international conference on inventive communication and computational technologies (icicct 2018) ieee, pp. 518–522. https://doi.org/10.1109/icicct.2018.8473194 alketbi, a., nasir, q., & talib, m. a. (2018). blockchain for government services – use cases, security benefits and challenges. 2018 15th learning and technology conference, l and t 2018, 112–119. https://doi.org/10.1109/lt.2018.8368494 alkhamisi, a., nazmudeen, m. s. h., & buhari, s. m. (2016). a cross-layer framework for sensor data aggregation for iot applications in smart cities. ieee 2nd international smart cities conference: improving the citizens quality of life, isc2 2016 – proceedings. https://doi.org/10.1109/isc2.2016.7580853 alliance for telecommunications industry solutions. (2018). smart cities data sharing framework. retrieved from www.atis.org/01_legal/patent-policy/ alohali, b., merabti, m., & kifayat, k. (2014). a cloud of things (cot) based security for home area network (han) in the smart grid. proceedings – 2014 8th international conference on next generation mobile applications, services and technologies, ngmast 2014. https://doi.org/10.1109/ngmast.2014.50 alromaihi, s., elmedany, w., & balakrishna, c. (2018). cyber security challenges of deploying iot in smart cities for healthcare applications. proceedings – 2018 ieee 6th international conference on future internet of things and cloud workshops, w-fi cloud 2018, 140–145. https://doi.org/10.1109/w-ficloud.2018.00028 alsafery, w., alturki, b., reiff-marganiec, s., & jambi, k. (2018). smart car parking system solution for the internet of things in smart cities. 1st international conference on computer applications and information security, iccais 2018. https://doi.org/10.1109/cais.2018.8442004 andrisano, o., bartolini, i., bellavista, p., boeri, a., bononi, l., borghetti, a., borghetti, a., brath, a., corazza, g., corradi, a., de miranda, s., fava, f., foschini, l., leoni, g., longo, d., milano, m., napolitano, f., nucci, c., pasolini, g., patella, m., cinotti, t., tarchi, d., ubertini, f., & vigo, d. (2018). the need of multidisciplinary approaches and engineering tools for the development and implementation of the smart city paradigm. proceedings of the ieee, 106(4), pp. 738–760. https://doi.org/10.1109/jproc.2018.2812836 anthopoulos, l. (2015). understanding the smart city domain: a literature review. transforming city governments for successful smart cities, part of the public administration and information technology book series (pait, 8). pp 9-21. https://doi.org/10.1007/978-3-319-03167-5_2 australasian journal of information systems naqvi, rehman & islam 2020, vol 24, research article a hyperconnected smart city framework 31 assante, d., romano, e., flamini, m., castro, m., martin, s., lavirotte, s., rey, g., leisenberg, m., migliori, m., bagdoniene, i., gallo, r., pascoal, a., & spatafora, m. (2018). internet of things education: labor market training needs and national policies. ieee global engineering education conference, educon, 1846–1853. https://doi.org/10.1109/educon.2018.8363459 ati, m., & basmaji, t. (2018). framework for managing smart cities security and privacy applications. iscaie 2018 – 2018 ieee symposium on computer applications and industrial electronics, pp. 191–194. https://doi.org/10.1109/iscaie.2018.8405468 badii, c., belay, e. g., bellini, p., cenni, d., marazzini, m., mesiti, m., nesi, p., pantaleo, g., paolucci, m., valtolina, s., soderi, m., & zaza, i. (2018). snap4city: a scalable iot/ioe platform for developing smart city applications. 2018 ieee smartworld, ubiquitous intelligence & computing, advanced & trusted computing, scalable computing & communications, cloud & big data computing, internet of people and smart city innovation (smartworld/scalcom/uic/atc/cbdcom/iop/sci), pp. 2109–2116. https://doi.org/10.1109/smartworld.2018.00353 bajramovic, e., waedt, k., ciriello, a., & gupta, d. (2016). forensic readiness of smart buildings preconditions for subsequent cybersecurity tests. 2016 ieee international smart cities conference (isc2), trento, 2016, pp. 1-6. https://doi.org/10.1109/isc2.2016.7580754 bass, t., sutherland, e., & symons, t. (2018). reclaiming the smart city: personal data, trust and the new commons. decode. european commission. https://decodeproject.eu/file/380/download beleuta, v., & delgado merce, j. m. (2017). data privacy and security in business intelligence and analytics. https://www.semanticscholar.org/paper/data-privacy-and-security-inbusiness-intelligence-beleuta/93adc50037259b66127d5bf1ea6d15ee7dc1b893 belgaum, m. r., alansari, z., jain, r., & alshaer, j. (2018). a framework for evaluation of cyber security challenges in smart cities. smart cities symposium 2018, bahrain, 2018, pp. 1-6. https://ieeexplore.ieee.org/document/8643178 bellini, p., nesi, p., paolucci, m., & zaza, i. (2018). smart city architecture for data ingestion and analytics: processes and solutions. proceedings: ieee 4th international conference on big data computing service and applications, bigdataservice 2018, pp. 137–144. https://doi.org/10.1109/bigdataservice.2018.00028 berntzen, l., & johannessen, m. (2016). the role of citizens in "smart cities". conference: management – international conference, at slovakia: faculty of management, university of presov. october 2016. https://www.researchgate.net/publication/309040628_the_role_of_citizens_in_smart_ cities bibri, s. e., & krogstie, j. (2017). the core enabling technologies of big data analytics and context-aware computing for smart sustainable cities: a review and synthesis. journal of big data, 4(1). https://doi.org/10.1186/s40537-017-0091-6 boban, m., & weber, m. (2018). internet of things, legal and regulatory framework in digital transformation from smart to intelligent cities. 2018 41st international convention on australasian journal of information systems naqvi, rehman & islam 2020, vol 24, research article a hyperconnected smart city framework 32 information and communication technology, electronics and microelectronics (mipro). https://doi.org/10.23919/mipro.2018.8400245 borges, m. a., melo, g. f., de massaki, c., igarashi, o., lopes, p. b., & silva, l. a. (2017). an architecture for the internet of things and the use of big data techniques in the analysis of carbon monoxide. proceedings – 2017 ieee international conference on information reuse and integration, iri 2017, 184–191. https://doi.org/10.1109/iri.2017.76 brauchli, a., & li, d. (2015). a solution-based analysis of attack vectors on smart home systems. 2015 international conference on cyber security of smart cities, industrial control system and communications, ssic 2015 – proceedings. https://doi.org/10.1109/ssic.2015.7245682 budhiputra, p. m., & putra, k. p. (2016). smart city framework based on business process reengineering approach. 2016 international conference on ict for smart society, pp. 69–73. https://doi.org/10.1109/ictss.2016.7792851 cam-winget, n., sadeghi, a, & jin, y. (2016). invited can iot be secured: emerging challenges in connecting the unconnected. proceedings of the 53rd annual design automation conference. dac ’16, austin, tx, 122, pp. 6. https://doi.org/10.1145/2897937.2905004 cao, q. h., khan, i., farahbakhsh, r., madhusudan, g., lee, g. m., & crespi, n. (2016). a trust model for data sharing in smart cities. 2016 ieee international conference on communications, icc 2016. https://doi.org/10.1109/icc.2016.7510834 cárdenas, a. a., & safavi-naini, r. (2012). security and privacy in the smart grid. in handbook on securing cyber-physical critical infrastructure (pp. 637–654). https://doi.org/10.1016/b978-0-12-415815-3.00025-x chakrabarty, s., & engels, d. w. (2016). a secure iot architecture for smart cities. 2016 13th ieee annual consumer communications and networking conference, ccnc 2016. https://doi.org/10.1109/ccnc.2016.7444889 chan, a. l., chua, g. g., chua, d. z. l., guo, s., lim, p. m. c., mak, m. t., & ng, w. s. (2018). practical experience with smart cities platform design. ieee world forum on internet of things, wf-iot 2018 – proceedings, 2018-january, pp. 470–475. https://doi.org/10.1109/wf-iot.2018.8355181 cui, l., xie, g., qu, y., gao, l., & yang, y. (2018). security and privacy in smart cities: challenges and opportunities. ieee access, 6, pp. 46134–46145. https://doi.org/10.1109/access.2018.2853985 de carolis, a., macchi, m., negri, e., & terzi, s. (2017). guiding manufacturing companies towards digitalization: a methodology for supporting manufacturing companies in defining their digitalization roadmap. ieee access, pp. 487–495. https://doi.org/10.1109/ice.2017.8279925 doynikova, e., & kotenko, i. (2017). enhancement of probabilistic attack graphs for accurate cyber security monitoring. in 2017 ieee smartworld, ubiquitous intelligence & computing, advanced & trusted computed, scalable computing & communications, cloud & big data computing, internet of people and smart city innovation australasian journal of information systems naqvi, rehman & islam 2020, vol 24, research article a hyperconnected smart city framework 33 (smartworld/scalcom/uic/atc/cbdcom/iop/sci). http://doi.org/10.1109/uicatc.2017.8397618 eckhoff, d., & wagner, i. (2018). privacy in the smart city – applications, technologies, challenges, and solutions. ieee communications surveys and tutorials, 20(1), pp. 489–516. https://doi.org/10.1109/comst.2017.2748998 el hendy, m., miniaoui, s., atalla, s., & hashim, k. (2017). a survey on smart city technologies, initiatives and global technology providers. association for computing machinery. proceedings of the second international conference on internet of things, data and cloud computing. icc ’17, 84, pp. 7. https://doi.org/10.1145/3018896.3025132 elmaghraby, a. (2013). security and privacy in the smart city. 6th ajman international urban planning conference aiupc 6: “city and security” at: ajman, uae. march 2013. https://www.researchgate.net/publication/269874307_security_and_privacy_in _the_smart_city elmisery, a. m., sertovic, m., & gupta, b. b. (2017). cognitive privacy middleware for deep learning mashup in environmental iot. ieee access, pp. 1–12. https://doi.org/10.1109/access.2017.2787422 elsaeidy, a., elgendi, i., munasinghe, k. s., sharma, d., & jamalipour, a. (2017). a smart city cyber security platform for narrowband networks. 2017 27th international telecommunication networks and applications conference, itnac 2017. https://doi.org/10.1109/atnac.2017.8215388 esposito, c., castiglione, a., frattini, f., cinque, m., yang, y., & choo, k. k. r. (2018). on data sovereignty in cloud-based computation offloading for smart cities applications. ieee internet of things journal, pp. 1–15. https://doi.org/10.1109/jiot.2018.2886410 essa, a., al-shoura, t., al nabulsi, a., al-ali, a. r., & aloul, f. (2018). cyber physical sensors system security: threats, vulnerabilities, and solutions. 2nd international conference on smart grid and smart cities, icsgsc 2018, pp. 62–67. https://doi.org/10.1109/icsgsc.2018.8541316 estevez, e., & janowski, t. (2013). electronic governance for sustainable development – conceptual framework and state of research. government information quarterly, 30 (2013), s94–s109. https://doi.org/10.1016/j.giq.2012.11.001 falco, g., viswanathan, a., caldera, c., & shrobe, h. (2018). a master attack methodology for an ai-based automated attack planner for smart cities. ieee access, 6, 48360–48373. https://doi.org/10.1109/access.2018.2867556 feibert, d. c., hansen, m. s., & jacobsen, p. (2017). an integrated process and digitalization perspective on the shipping supply chain – a literature review. proceedings of the 2017 ieee ieem, pp. 1352–1356. https://doi.org/10.1109/ieem.2017.8290113 feng, x., & zhao, y. (2018). digital forensics challenges to big data in the cloud. proceedings – 2017 ieee international conference on internet of things, ieee green computing and communications, ieee cyber, physical and social computing, ieee smart data, ithingsgreencom-cpscom-smartdata 2017. https://doi.org/10.1109/ithings-greencomcpscom-smartdata.2017.132 australasian journal of information systems naqvi, rehman & islam 2020, vol 24, research article a hyperconnected smart city framework 34 feng, x., dawam, e. s., & amin, s. (2017). a new digital forensics model of smart city automated vehicles. proceedings – 2017 ieee international conference on internet of things, ieee green computing and communications, ieee cyber, physical and social computing, ieee smart data, ithings-greencom-cpscom-smartdata 2017, pp. 274–279. https://doi.org/10.1109/ithings-greencom-cpscom-smartdata.2017.47 fernandes, e., rahmati, a., eykholt, k., & prakash, a. (2017). internet of things security research: a rehash of old ideas or new intellectual challenges? ieee security & privacy, 15(1), pp. 79–84. https://doi.org/10.1109/msp.2017.3151346 fernandez molanes, r., amarasinghe, k., rodriguez-andina, j., & manic, m. (2018). deep learning and reconfigurable platforms in the internet of things: challenges and opportunities in algorithms and hardware. ieee industrial electronics magazine. https://doi.org/10.1109/mie.2018.2824843 ferrer, a. j., marquès, j. m., & jorba, j. (2019). towards the decentralised cloud. acm computing surveys, 51(6), 1–36. https://doi.org/10.1145/3243929 foucault, j., & moulier-boutang, y. (2015). towards economic and social “sensors”: condition and model of governance and decision-making for an organological smart city. 2015 international conference on smart and sustainable city and big data (icssc), shanghai, 2015, pp. 106-112. https://ieeexplore.ieee.org/document/7446445 ghirardello, k., maple, c., ng, d., & kearney, p. (2018). cyber security of smart homes: development of a reference architecture for attack surface analysis. proceedings: ieee. living in the internet of things: cybersecurity of the iot – 2018. https://ieeexplore.ieee.org/document/8379732 gyrard, a., zimmermann, a., & sheth, a. (2018). building iot based applications for smart cities: how can ontology catalogs help? ieee internet of things journal, pp. 1–22. https://doi.org/10.1109/jiot.2018.2854278 haiyan, s. c. (2016). digital education resource configuration mode transformation from the co-construction and sharing to the public to be shared. 2016 eighth international conference on measuring technology and mechatronics automation. https://doi.org/10.1109/icmtma.2016.64 harris, s. (2014). securing big data in our future intelligent cities. iet conference on future intelligent cities. pp. 1–4. https://www.researchgate.net/publication/300490699_securing_big_data_in_our_futur e_intelligent_cities hashem, i. a. t., chang, v., anuar, n. b., adewole, k., yaqoob, i., gani, a., ahmed, e., & chiroma, h. (2016). the role of big data in smart city. international journal of information management, 36(5), pp. 748–758. https://doi.org/10.1016/j.ijinfomgt.2016.05.002 hassanain, e., rahman, m. a., alhamid, m. f., hossain, m. s., guizani, m., & rashid, m. m. (2019). blockchain and iot-based cognitive edge framework for sharing economy services in a smart city. ieee access. https://doi.org/10.1109/access.2019.2896065. hong, j., & liu, c. c. (2019). intelligent electronic devices with collaborative intrusion detection systems. ieee transactions on smart grid, pp. 1–11. https://doi.org/10.1109/tsg.2017.2737826 australasian journal of information systems naqvi, rehman & islam 2020, vol 24, research article a hyperconnected smart city framework 35 inquiry into the australian government's role in the development of cities submission 37. iot and government’s role in the development of cities; submission to the standing committee on infrastructure. transport and cities. iot alliance australia july 2017. https://www.aph.gov.au/documentstore.ashx?id=229cdb32-7586-4867-8e40d43158edf35f&subid=514420 inukollu, v. n., arsi, s., & rao ravuri, s. (2014). security issues associated with big data in cloud computing. international journal of network security & its applications, 6(3), pp. 45– 56. https://doi.org/10.5121/ijnsa.2014.6304 ismail, n. (2016). determining the internet of things (iot) challenges on smart cities: a systematic literature review. journal of information systems research and innovation 10(3), pp. 56–63, johannessen, m., & berntzen, l. (2018). the transparent smart city. smart technologies for smart governments, public administration and information technology book series (pait, 24), pp 67-94. https://doi.org/10.1007/978-3-319-58577-2_5 kesswani, n., & kumar, s. (2018). the smart-x model for smart cities. proceedings: 2018 ieee 42nd annual computer software and applications conference (compsac). https://doi.org/10.1109/compsac.2018.00112 khan, z., pervez, z., & ghafoor, a. (2014). towards cloud based smart cities data security and privacy management. proceedings – 2014 ieee/acm 7th international conference on utility and cloud computing, ucc 2014, pp. 806–811. https://doi.org/10.1109/ucc.2014.131 kodali, r. k., azman, m., & panicker, j. g. (2018). smart control system solution for smart cities. 2018 international conference on cyber-enabled distributed computing and knowledge discovery (cyberc), zhengzhou, china, 2018, pp. 89-893. https://doi.org/10.1109/cyberc.2018.00027 kolozali, s., bermudez-edo, m., davar, n. f., barnaghi, p., gao, f., ali, m. i., … tonjes, r. (2018). observing the pulse of a city: a smart city framework for real-time discovery, federation, and aggregation of data streams. ieee internet of things journal. https://doi.org/10.1109/jiot.2018.2872606 kor, al., pattinson, c., yanovsky, m., & kharchenko, v. (2018). iot-enabled smart living. technology for smart futures, springer, cham, pp. 3-28. https://doi.org/10.1007/978-3-31960137-3_1 kuan, z., jianbing, n., yang, k., xiaohui, l., ren, j., & shen, x. (2017). security and privacy in smart city applications: challenges and solutions. ieee communications magazine, pp. 122–129. https://doi.org/10.1109/mcom.2017.1600267cm l’heureux, a., grolinger, k., higashino, w. a., & capretz, m. a. m. (2017). a gamification framework for sensor data analytics. proceedings – 2017 ieee 2nd international congress on internet of things, iciot 2017, pp. 74–81. https://doi.org/10.1109/ieee.iciot.2017.18 layne, k., & lee, j. (2001). developing fully functional e-government: a four stage model. government information quarterly. 18, pp. 122–136. https://doi.org/10.1016/s0740624x(01)00066-1 australasian journal of information systems naqvi, rehman & islam 2020, vol 24, research article a hyperconnected smart city framework 36 li, d., cao, j., & yao, y. (2015). big data in smart cities. science china information sciences, 58(10), 1–12. https://doi.org/10.1007/s11432-015-5396-5 li, y., dai, w., ming, z., & qiu, m. (2016). privacy protection for preventing data overcollection in smart city. ieee transactions on computers, 65(5), pp. 1339–1350. https://doi.org/10.1109/tc.2015.2470247 lim c., kim k., & maglio p. (2018). smart cities with big data: reference models, challenges, and considerations. elsevier, volume 82, december 2018, pp. 86-99. https://doi.org/10.1016/j.cities.2018.04.011 lin, h., hu, j., ma, j., xu, l., & yu, z. (2017). a secure collaborative spectrum sensing strategy in cyber-physical systems. ieee access, 5, pp. 27679–27690. https://doi.org/10.1109/access.2017.2767701 liu, j. k., choo, k.-k. r., huang, x., & au, m. h. (2017). erratum to: special issue on security and privacy for smart cities. personal and ubiquitous computing, 21(5), pp. 777–777. https://doi.org/10.1007/s00779-017-1062-9 liu, j., zhang, c., & fang, y. (2018). epic: a differential privacy framework to defend smart homes against internet traffic analysis. ieee internet of things journal, 1–12. https://doi.org/10.1109/jiot.2018.2799820 liu, w., nakauchi, k., & shoji, y. (2018). a neighbor-based probabilistic broadcast protocol for data dissemination in mobile iot networks. ieee access. https://doi.org/10.1109/access.2018.2808356 liu, y., weng, x., wan, j., yue, x., song, h., & vasilakos, a. v. (2017). exploring data validity in transportation systems for smart cities. ieee communications magazine. 55(5), pp. 26– 33. https://doi.org/10.1109/mcom.2017.1600240 lokuliyana, s., jayakody, a., dabarera, g. s. b., ranaweera, r. k. r., perera, p. g. d. m., & panangala, p. a. d. v. r. (2018, august 8-11). location based garbage management system with iot for smart city. the 13th international conference on computer science & education (iccse 2018), pp. 699–703. http://doi.org/10.1109/iccse.2018.8468682 luo, c., et al. (2019). predictable privacy-preserving mobile crowd sensing: a tale of two roles. ieee/acm transactions on networking, 27(1), pp. 361-374, feb. 2019. https://doi.org/10.1109/tnet.2019.2890860 luo, x., ren, y., hu, j., wu, q., & lou, j. (2017). privacy-preserving identity-based file sharing in smart city. personal and ubiquitous computing, 21(5), pp. 923–936. https://doi.org/10.1007/s00779-017-1051-z maheswaran, m., & misra, s. (2015). towards a social governance framework for internet of things. 2015 ieee 2nd world forum on internet of things (wf-iot). dec 2015. https://doi.org/10.1109/wf-iot.2015.7389156 manjunatha, & annappa b. (2018). real time big data analytics in smart city applications. international conference on communication, computing and internet of things (ic3iot), pp. 279–284. https://doi.org/10.1109/ic3iot.2018.8668106 australasian journal of information systems naqvi, rehman & islam 2020, vol 24, research article a hyperconnected smart city framework 37 marjani, m., nasaruddin, f., gani, a., karim, a., hashem, i. a. t., siddiqa, a., & yaqoob, i. (2017). big iot data analytics: architecture, opportunities, and open research challenges. ieee access. https://doi.org/10.1109/access.2017.2689040 masduki, b. w., ramli, k., & salman, m. (2017). leverage intrusion detection system framework for cyber situational awareness system. 2017 international conference on smart cities, automation & intelligent computing systems (icon-sonics), yogyakarta, 2017, pp. 64-69. https://doi.org/10.1109/icon-sonics.2017.8267823 memmi, g., kapusta, k., & qiu, h. (2015). data protection: combining fragmentation, encryption, and dispersion. 2015 international conference on cyber security of smart cities, industrial control system and communications (ssic). https://doi.org/10.1109/ssic.2015.7245680 mohammadi, m., & al-fuqaha, a. (2018). enabling cognitive smart cities using big data and machine learning: approaches and challenges. ieee communications magazine, 56(2), pp. 94–101. https://doi.org/10.1109/mcom.2018.1700298 mohammadi, m., al-fuqaha, a., guizani, m., & oh, j. s. (2018). semisupervised deep reinforcement learning in support of iot and smart city services. ieee internet of things journal. https://doi.org/10.1109/jiot.2017.2712560 moreno, m. v., terroso-saenz, f., gonzalez-vidal, a., valdes-vela, m., skarmeta, a. f., zamora, m. a., & chang, v. (2017). applicability of big data techniques to smart cities deployments. ieee transactions on industrial informatics, pp. 1–10. https://doi.org/10.1109/tii.2016.2605581 moura, j., & serrão, c. (2015). security and privacy issues of big data. handbook of research on trends and future directions in big data and web intelligence, pp 33. https://doi.org/10.4018/978-1-4666-8505-5.ch002 nam, t., & pardo, t. a. (2011). smart city as urban innovation: focusing on management, policy, and context. acm international conference proceeding series. 185-194. https://dl.acm.org/doi/10.1145/2072069.2072100 nesi, p., pantaleo, g., paolucci, m., & zaza, i. (2018). auditing and assessment of data traffic flows in an iot architecture. proceedings – 4th ieee international conference on collaboration and internet computing, cic 2018, pp. 388–391. https://doi.org/10.1109/cic.2018.00058 okai, e., feng, x., & sant, p. (2019). smart cities survey. proceedings – 20th international conference on high performance computing and communications, 16th international conference on smart city and 4th international conference on data science and systems, hpcc/smartcity/dss 2018, pp. 1726–1730. https://doi.org/10.1109/hpcc/smartcity/dss.2018.00282 oteafy, s. m. a., & hassanein, h. s. (2018). iot in the fog: a roadmap for data-centric iot development. ieee communications magazine, 56(3), pp. 157–163. https://doi.org/10.1109/mcom.2018.1700299 pacheco, j., & hariri, s. (2016). iot security framework for smart cyber infrastructures. proceedings – ieee 1st international workshops on foundations and applications of selfsystems, fas-w 2016, 242–247. https://doi.org/10.1109/fas-w.2016.58 australasian journal of information systems naqvi, rehman & islam 2020, vol 24, research article a hyperconnected smart city framework 38 popescul, d., & radu, l. d. (2016). data security in smart cities: challenges and solutions. informatica economica, 20(1/2016), pp. 29–38. https://doi.org/10.12948/issn14531305/20.1.2016.03 pradhan, m., fuchs, c., & johnsen, f. t. (2018). a survey of applicability of military data model architectures for smart city data consumption and integration. ieee world forum on internet of things, wf-iot 2018 – proceedings, pp. 129–134. https://doi.org/10.1109/wfiot.2018.8355226 pradhan, m., suri, n., fuchs, c., bloebaum, t. h., & marks, m. (2018). toward an architecture and data model to enable interoperability between federated mission networks and iot-enabled smart city environments. ieee communications magazine, pp. 163–169. https://doi.org/10.1109/mcom.2018.1800305 puron-cid g., gil-garci j.r., & zhang j. (2015). smart cities, smart governments and smart citizens: a brief introduction. international journal of e-planning research. 4. iv-vii. https://www.researchgate.net/publication/295850218_smart_cities_smart_government s_and_smart_citizens_a_brief_introduction puschmann, d., barnaghi, p., & tafazolli, r. (2018). using lda to uncover the underlying structures and relations in smart city data streams. ieee systems journal. https://doi.org/10.1109/jsyst.2017.2723818 rastogi, n., singh, s. k., & singh, p. k. (2018). privacy and security issues in big data: through indian prospective. proceedings – 2018 3rd international conference on internet of things: smart innovation and usages, iot-siu 2018. https://doi.org/10.1109/iot-siu.2018.8519858 riboni, d. (2019). opportunistic pervasive computing: adaptive context recognition and interfaces. ccf transactions on pervasive computing and interaction. https://doi.org/10.1007/s42486-018-00004-9 rjab, a. ben, & mellouli, s. (2018). smart cities in the era of artificial intelligence and internet of things. 2018 association for computing machinery, pp. 1–10. https://doi.org/10.1145/3209281.3209380 russell, l., goubran, r., kwamena, f., & knoefel, f. (2018). agile iot for critical infrastructure resilience: cross-modal sensing as part of a situational awareness approach. ieee internet of things journal. https://doi.org/10.1109/jiot.2018.2818113 ryu, j. h., sharma, p. k., jo, j. h., & park, j. h. (2019). a blockchain-based decentralized efficient investigation framework for iot digital forensics. springer journal of supercomputing. https://doi.org/10.1007/s11227-019-02779-9 sankaran, s., &vishwa vidyapeetham, a. (2017). securing networked control systems: modeling attacks and defenses. 2017 ieee international conference on consumer electronics-asia (icce-asia) securing, pp. 7–11. https://doi.org/10.1109/icceasia.2017.8309317 sanseverino, e., sanseverino, r., vaccaro, v., macaione, i., & anello, e. (2017). smart cities: case studies. smart cities atlas. springer link. november 2016, 82, pp. 47–140. https://doi.org/10.1007/978-3-319-47361-1_3 australasian journal of information systems naqvi, rehman & islam 2020, vol 24, research article a hyperconnected smart city framework 39 santos, j., et al. (2017) city of things: enabling resource provisioning in smart cities. ieee communications magazine, pp. 1-8. https://biblio.ugent.be/publication/8572970/file/8572995.pdf sarker, m. n. i., wu, m., & hossin, m. a. (2018). smart governance through bigdata: digital transformation of public agencies. 2018 international conference on artificial intelligence and big data, icaibd 2018. https://doi.org/10.1109/icaibd.2018.8396168 shah, t., & patel, s.v. (2014). a review of requirement engineering issues and challenges in various software development methods. international journal of computer applications. 99(15), pp. 36–45. https://doi.org/10.5120/17451-8370 shahat, a. m., elragal, o. a., & bergvall-kåreborn, b. (2017). big data analytics and smart cities: a loose or tight couple? international conference on connected smart cities 2017 (csc 2017), lisbon, 20-22 july 2017. https://www.researchgate.net/publication/317491579_big_data_analytics_and_smart_ cities_a_loose_or_tight_couple siddiqui, z., tayan, o., & khurram khan, m. (2018). security analysis of smartphone and cloud computing authentication frameworks and protocols. ieee access, pp. 1–16. https://doi.org/10.1109/access.2018.2845299 sikora-fernandez, d., & stawasz, d. (2016). the concept of smart city in the theory and practice of urban development management. romanian journal of regional science. 10(1), pp. 86–99. https://ideas.repec.org/a/rrs/journl/v10y2016i1p86-99.html simonofski, a., asensio, e. s., de smedt, j., & snoeck, m. (2017). citizen participation in smart cities: evaluation framework proposal. 2017 ieee 19th conference on business informatics (cbi), thessaloniki, 2017, pp. 227-236. https://doi.org/10.1109/cbi.2017.21. sinaeepourfard, a., krogstie, j., & peterse, s. a. (2018). a big data management architecture for smart cities based on fog-to-cloud data management architecture. retrieved from ceurws.org/vol-2316/paper4.pdf sindhusha, p., & bharathi, b. (2018). privacy protection on data overflow system for smartphones. 6th international conference on computation of power, energy, information and communication, iccpeic 2017, pp. 310–315. https://doi.org/10.1109/iccpeic.2017.8290383 siris, v. a., fotiou, n., mertzianis, a., & polyzos, g. c. (2019). smart application aware iot data collection. journal of reliable intelligent environments, 5(1), pp. 17–28. https://doi.org/10.1007/s40860-019-00077-y smolander, k., rossi, m., & pekkola, s. (2017). infrastructures, integration and architecting during and after digital transformation. proceedings – 2017 ieee/acm joint 5th international workshop on software engineering for systems-of-systems and 11th workshop on distributed software development, software ecosystems and systems-of-systems, jsos 2017, 23–26. https://doi.org/10.1109/jsos.2017.1 soultatos, o., spanoudakis, g., fysarakis, k., askoxylakis, i., alexandris, g., miaoudakis, a., & nikolaos petroulakis, e. (2018). towards a security, privacy, dependability, interoperability framework for the internet of things. ieee international workshop on australasian journal of information systems naqvi, rehman & islam 2020, vol 24, research article a hyperconnected smart city framework 40 computer aided modeling and design of communication links and networks, camad. https://doi.org/10.1109/camad.2018.8514937 steuer, s., benabbas, a., kasrin, n., & nicklas, d. (2016). challenges and design goals for an architecture of a privacy-preserving smart city lab. datenbank-spektrum, 16(2), pp. 147– 156. https://doi.org/10.1007/s13222-016-0223-8 swarnkar, m., & bhadoria, r. s. (2017). security issues and challenges in big data analytics in distributed environment. springer international publishing ag 2017, pp. 83–94. https://doi.org/10.1007/978-3-319-59834-5_5 tanaka, s. a., de barros, r. m., & de souza mendes, l. (2018). a proposal to a framework for governance of ict aiming at smart cities with a focus on enterprise architecture. 2018 association for computing machinery. acm, pp. 408–415. https://doi.org/10.1145/3229345.3229400 tawalbeh, l. a., tawalbeh, h., song, h., & jararweh, y. (2017). intrusion and attacks over mobile networks and cloud health systems. 2017 ieee conference on computer communications workshops, infocom wkshps 2017, 13–17. https://doi.org/10.1109/infcomw.2017.8116345 tay, k., supangkat, s. h., cornelius, g. & arman, a. a. (2018). the smart initiative and the garuda smart city framework for the development of smart cities. 2018 international conference on ict for smart society (iciss), semarang, 2018, pp. 1-10. https://doi.org/10.1109/ictss.2018.8549961 teoh, c. s., & mahmood, a. k. (2017). national cyber security strategies for digital economy. journal of theoretical and applied information technology. https://doi.org/10.1109/icriis.2017.8002519 terzi, d. s., terzi, r., & sagiroglu, s. (2016). a survey on security and privacy issues in big data. 2015 10th international conference for internet technology and secured transactions, icitst 2015, 202–207. https://doi.org/10.1109/icitst.2015.7412089 tragos, e., fragkiadakis, a., angelakis, v., & pöhls, h. c. (2016). designing secure iot architectures for smart city applications. in designing, developing, and facilitating smart cities (pp. 63–87). https://doi.org/10.1007/978-3-319-44924-1_5 venticinque, s., & amato, a. (2018). a methodology for deployment of iot application in fog. journal of ambient intelligence and humanized computing, 10, pp. 1–22. https://doi.org/10.1007/s12652-018-0785-4 wagner, i., & eckhoff, d. (2018). technical privacy metrics: a systematic survey. acm computing surveys, 51(3), 1–38. https://doi.org/10.1145/3168389 wang, p., ali, a., & kelly, w. (2015). data security and threat modeling for smart city infrastructure. 2015 international conference on cyber security of smart cities, industrial control system and communications (ssic). https://doi.org/10.1109/ssic.2015.7245322 wang, p., yang, l. t., xavier, s. f., & li, j. (2018). an edge cloud-assisted cpss framework for smart cities. ieee cloud computing, pp. 37–46. https://doi.org/10.1109/mcc.2018.053711665 australasian journal of information systems naqvi, rehman & islam 2020, vol 24, research article a hyperconnected smart city framework 41 west, d. m. (2004). e-government and the transformation of service delivery and citizen attitudes. wiley online library, 64(1), pp. 15–27. https://doi.org/10.1111/j.1540-6210.2004.00343.x xie, y., wang, w., wang, f., & chang, r. (2018). vtet: a virtual industrial control system testbed for cyber security research. 2018 3rd international conference on security of smart cities, industrial control system and communications, ssic 2018 – proceedings. https://doi.org/10.1109/ssic.2018.8556732 yadav, p., & vishwakarma, s. (2018). application of internet of things and big data towards a smart city. 2018 3rd international conference on internet of things: smart innovation and usages (iot-siu), pp. 1–5. https://doi.org/10.1109/iot-siu.2018.8519920 yamakami, t. (2017). an organizational coordination model for iot: a case study of requirement engineering of city-government in tokyo in city platform as a service. international conference on information and communication technology convergence: ict convergence technologies leading the fourth industrial revolution, ictc 2017. https://doi.org/10.1109/ictc.2017.8190982 yan, z., yu, x., & ding, w. (2017). context-aware verifiable cloud computing. ieee access, 5, 2211–2227. https://doi.org/10.1109/access.2017.2666839 yigitcanlar, t., kamruzzaman, m., buys, l., ioppolo, g., marques, j., moreira da costa, e., & yun, j. (2018). understanding ‘smart cities’: intertwining development drivers with desired outcomes in a multidimensional framework. cities. https://doi.org/10.1016/j.cities.2018.04.003 ylmaz, e. n., ciylan, b., gönen, s., sindiren, e., & karacayilmaz, g. (2018). cyber security in industrial control systems: analysis of dos attacks against plcs and the insider effect. proceedings – 2018 6th international istanbul smart grids and cities congress and fair, icsg 2018, pp. 81–85. https://doi.org/10.1109/sgcf.2018.8408947 zhang, g., wang, j., huang, w., su, h., lv, z., yao, q., & ye, s. (2015). big data collection and analysis framework research for public digital culture sharing service. proceedings 2015 ieee international conference on multimedia big data, bigmm 2015, 196–199. https://doi.org/10.1109/bigmm.2015.37 zhang, k., & zahng, q. (2018). preserve location privacy for cyber-physical systems with addresses hashing at data link layer. 2018 ieee 20th international conference on high performance computing and communications; ieee 16th international conference on smart city; ieee 4th international conference on data science and systems (hpcc/smartcity/dss), exeter, united kingdom, 2018, pp. 1028-1032. https://doi.org/10.1109/hpcc/smartcity/dss.2018.00170 copyright: © 2020 naqvi, rehman & islam. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v24i0.2531 http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems naqvi, rehman & islam 2020, vol 24, research article a hyperconnected smart city framework 42 paper title (paper title style) australasian journal of information systems dang-pham et al. 2022, vol 26, selected papers from acis 2020 digital kaizen 1 digital kaizen: an approach to digital transformation duy dang-pham school of business & management, rmit university, vietnam duy.dangphamthien@rmit.edu.vn ai-phuong hoang school of business & management, rmit university, vietnam diem-trang vo school of communication & design, rmit university, vietnam karlheinz kautz department of information systems & business analytics, rmit university, melbourne, australia abstract while digital transformation creates opportunities across all industries, many organisations and practitioners struggle to grasp what conducting digital transformation entails and seek directions for how digital transformation programs, especially large and complex ones, can be executed. in response, recent studies have offered such directions and models for guiding digital transformation and put forward suggestions for steps and phases of digital transformation, as well as drivers, enablers, and inhibitors. many of these proposals are based on the dynamic capabilities framework as a theoretical foundation and invite more research to develop further and/or to refine existing models for digital transformation, especially investigating how dynamic capabilities contribute to digital transformation. in this case study, we explore digital kaizen, a systematic approach to conduct large-scale digital transformation developed and applied at our case organisation fpt software. this approach integrates the kaizen philosophy into digital transformation practices that lead to digital improvements and transformation of some of the company’s human resource management processes and its business model. our analysis and our reflections on our findings lead us to propose a novel digital kaizen process model for digital transformation informed by the dynamic capabilities framework. keywords: digital transformation, digital transformation implementation, digital transformation approach, kaizen, digital kaizen, dynamic capabilities framework. 1 introduction along with the development of new technologies such as cloud computing, big data analytics or artificial intelligence (ai), the ways businesses operate have been transformed significantly (perkin and abraham, 2017). these digital technologies have been integrated into all areas of a business, changing the way organisations operate and deliver values to its customers. in other words, businesses are embracing digital transformation to succeed in the digital age (urbach and röglinger, 2018). the concept of digital transformation refers to the technology-induced changes necessary for digital business, in which organisations experience a fundamental and socio-technical transformation (legner et al., 2017). the current wave of digitalisation is not only enabled by australasian journal of information systems dang-pham et al. 2022, vol 26, selected papers from acis 2020 digital kaizen 2 the convergence of new information technology (it) megatrends (e.g., social, mobile, big data, cloud computing), but also modifies an organisation’s value creation logic, and in turn, creates new types of businesses (legner et al., 2017). therefore, the new concept of digital transformation refers to the manifold socio-technical phenomena and processes of using these technologies in broader individual, organisational and societal contexts (legner et al., 2017). nevertheless, despite the new opportunities that digital transformation has brought to organisations, it has created new pressures for organisations to develop adequate capabilities and capacities, to carry out such transformation (horlach et al., 2016). many organisations and practitioners struggle to grasp what conducting digital transformation entails and seek directions for how digital transformation programs, especially large and complex ones, can be executed (wessel et al., 2020). recent studies have offered such directions and models for guiding digital transformation (see e.g., chirumalla, 2021; warner and wäger, 2019) and put forward suggestions for steps and phases of digital transformation, as well as drivers, enablers, and inhibitors. some of these are based on teece’s (2007) dynamic capabilities framework as a theoretical foundation and invite more research to develop further and/or to refine existing models for digital transformation, especially investigating how dynamic capabilities contribute to digital transformation (steininger et al., 2022; vial, 2019; warner and wäger, 2019). lacking an understanding of digital transformation processes not only increases the risk of failing the transformation programs, resulting in high organisational and sometimes societal costs, but also limits knowledge of digital transformation strategies and how transformation efforts may differ across contexts e.g., large enterprises versus small and medium ones. our research heeds the calls for research to explore digital transformation processes and refine digital transformation strategies (warner and wäger, 2019) from the perspective of the dynamic capabilities framework as a consistent approach for investigating digital transformation (ellström et al., 2022; steininger et al., 2022). based on interviews with key informants, we explore digital kaizen, a systematic approach to conduct large-scale digital transformation developed and applied at fpt software1 , a large software development and it outsourcing company that turned into a digital transformation service provider in vietnam. this approach integrates the kaizen philosophy, which has mainly been applied in manufacturing to optimise production lines, into digital transformation practices that lead to improvements and transformation of some of the company’s human resource management processes, in regard to employee engagement, empowerment, and talent management, and of its business model. this research answers three research questions: rq1: what are the characteristics of the digital kaizen approach? rq2: what guidance can digital kaizen informed by the dynamic capabilities framework, provide for digital transformation? rq3: what are possible outcomes of applying digital kaizen? our paper is structured as follows. the next section reviews the relevant literature about digital transformation, enablers for digital transformation, the dynamic capabilities 1 fpt software has registered digital kaizen as a company’s trademark. australasian journal of information systems dang-pham et al. 2022, vol 26, selected papers from acis 2020 digital kaizen 3 framework, and the kaizen philosophy. the research approach section describes our data collection and introduces the case setting at fpt software including their digital kaizen approach. in the following section, we present our analysis and the findings, comprising the identified strategies, tactics, and enablers that in particular facilitated the successful execution of two notable digital transformation initiatives and the transformation of the business model at fpt software. we then discuss and reflect on the relationship between the kaizen framework and digital kaizen, and between the dynamic capabilities framework and digital kaizen. the latter reflections lead us to propose a novel digital kaizen process model for digital transformation informed by the dynamic capabilities framework. we conclude our paper with considerations and future research directions. 2 literature review and theoretical background 2.1 digital transformation previous studies on digital transformation concern two distinct but interrelated theoretical approaches. the first approach is in line with previous findings about it-enabled transformation. from this perspective, digital transformation refers to the use of digital technologies to significantly improve the performance or reach of a company (bekkhus, 2016), through facilitating the changes in organisational structures, processes, and culture (bharadwaj et al., 2013; matt et al., 2015). it strategy is no longer restricted to the operational level, and it is integrated into the organisation’s business strategy (bharadwaj et al., 2013). the second approach is based on the premises that digital transformation should go beyond it implementation and highlights the integration of digital technologies and structural modification to generate new paths for value creation (hess et al., 2016). wessel et al. (2020) argue that digital transformation activities should aim to redefine an organisation’s value proposition rather than solely leveraging digital technology to reinforce the existing value proposition. while it-enabled transformation indicates a firm’s vision of developing digital business models, digital transformation represents the organisation’s comprehensive utilisation of digital technologies to alter its value creation processes (vial, 2019). we understand digital transformation is defined as the changes brought about by digital technologies, with regard to an organisation’s business models, value proposition, processes, organisational identity and structures (hess et al., 2016; wessel et al., 2020). despite the prominence of digital transformation, the literature points to several challenges that prevent organisations from effectively executing such transformation. first, digital transformation is a complex concept (hess et al., 2016; wessel et al., 2020). businesses are unsure about what digital transformation is and do not know how to embark on their transformation journey (von leipzig et al., 2017; wessel et al., 2020). many enterprises are hesitant to commit resources to major digital transformation projects, especially when they are large in both scope and scale and contain uncertain outcomes (kane et al., 2019). hence, they might disregard solutions that are favourable to the firm’s specific situation, which in turn leads to unintended negative consequences to its transformation activities (hess et al., 2016). second, in some organisations, data is still utilised in organisational silos, thereby hindering strategic and digital transformation activities (rogers, 2016). the lack of a team mentality and collaboration poses a challenge for digital transformation, as digital transformation australasian journal of information systems dang-pham et al. 2022, vol 26, selected papers from acis 2020 digital kaizen 4 requires a unity of efforts (maedche, 2016). digital transformation is not about changing technology but about changing the way an organisation operates; new digital technologies should be leveraged to promote more creativity through e.g., information and computing capabilities that enable new forms of collaboration among networks of organisational actors (vial, 2019). third, the advancement of technologies has led to a shift in the business landscape across industries (meraviglia, 2018). while this generated opportunities for innovative start-ups to create value for customers, it also sparked a digital transformation competition among businesses (rogers, 2016). many companies are familiar with using technologies to improve their performance; however, as these technologies become more available and affordable, companies feel the pressure to rapidly transform themselves in an unprecedented manner or they will lose out eventually. 2.2 enablers for successful digital transformation successful digital transformation depends on a number of strategic and operational enablers. strategies, visions, roadmaps, and business models are viewed as the backbone of successful digital transformation (mergel et al., 2019). a digital strategy requires from a company to deploy digital technologies to target unmet customer needs; that is, to review the current value propositions. alternatively, it needs to come up with a new business model that generates additional revenue streams (loonam et al., 2018). defining scope is the first, most critical step in any transformation project. common scopes of digital transformation include creating customer value through softwareand platformenabled services or improving efficiency by digitalising logistics activities with intelligence and predictive maintenance (correani et al., 2020). these projects often aim at making continuous improvements, while acknowledging that an end state would not be achievable (mergel et al., 2019). this also means that organisations should avoid investing significant sums in research and implementation, and more efforts should be spent on minimising risk and motivating employees during this never-ending process (mergel et al., 2019). a common tactic used is implementing small, low-risk pilots for specific business lines in a stepwise fashion (henriette et al., 2016). as the benefits of digital transformation programs are generally qualitative in nature, it is difficult for organisational members to realise those benefits (agrawal et al., 2019). the stepwise approach allows companies to present visible results to stakeholders more easily and help to inspire changes (henriette et al., 2016). organisational culture is another enabler of digital transformation. a supportive organisational culture in which joint business and it initiatives flourish is vital for successful digital transformation (osmundsen et al., 2018). an open culture improves the organisation’s willingness to accept, implement, and promote changes, which creates a conducive environment for digital transformation (osmundsen et al., 2018). cultural considerations such as a focused vision statement and clear communication strategy are also necessary for companies to manoeuvre through large-scale, rapid, and disruptive changes (henriette et al., 2016). the commitment and involvement of top management are required (henriette et al., 2016; rueckel et al., 2020). the lack of a sense of urgency is a barrier of digital transformation, as complacent organisational leaders would not support transformation initiatives (agrawal et al., 2019). a shared digital vision among managers and executives is therefore essential. australasian journal of information systems dang-pham et al. 2022, vol 26, selected papers from acis 2020 digital kaizen 5 successful initiatives highlight the role of a chief digital officer (cdo), who oversees digital transformation projects, and of strong support from the board of directors (ivancic et al., 2019). transformation also requires the inclusion of staff from the entire organisation; employees must feel that they play an essential role in the implementation process (ivancic et al., 2019). organisations can organise regular workshops to share the transformation status and assess the attitude of managers toward the transformation (chanias et al., 2019). training should be also arranged for new employees to receive mentorship from current ones, which helps to flatten organisational hierarchies and makes it possible for entrepreneurial employees to identify problems early and improve organisational efficiency (li et al., 2018). enabler description references strategy and business model the digital strategy must incorporate organisational goals and visions, including reformulating the value proposition of the business, reconfiguring value delivery models, adapting new business models, and developing new capabilities. agrawal et al. (2019); henriette et al. (2016); ivancic et al. (2019); loonam et al. (2018); osmundsen et al. (2018); rueckel et al. (2020) change management change management is prioritised to deal with the potential adverse effects created by digital disruptions. digital champions should be appointed to advocate the adoption of a digital mindset and of tools through engaging with the employees. communication, including a clear and focused vision statement, is critical in this process. agrawal et al. (2019); correani et al. (2020); ivancic et al. (2019); loonam et al. (2018); osmundsen et al. (2018) organisational culture and participation digital transformation demands collective participation of all organisational members. it requires the endorsement and support from top management to present a clear vision and values. it also strives for a digital culture that involves greater mobility, collaboration and learning. agrawal et al. (2019); henriette et al. (2016); ivancic et al. (2019); loonam et al. (2018); osmundsen et al. (2018) data and it infrastructures data and it infrastructures lie at the core of digital transformation as both its prerequisites and outcomes. outcomes include better data management and accessible data platforms for internal and external stakeholders. data as a strategic asset of the company are synchronised, structured, and made accessible to provide insights to managers. correani et al. (2020); gaffley and pelser (2021); ivancic et al. (2019); loonam et al. (2018); rueckel et al. (2020) human resource management new professional roles, e.g., digital advisors and digital champions, may be created. retaining digital talent and improving digital skills are vital for developing new workforce capabilities. new employees with digital skills and proactive mindsets are preferable. correani et al. (2020); henriette et al. (2016); ivancic et al. (2019); rueckel et al. (2020) digital ecosystem the development of digital transformation programs often responds to changing market needs, digital shifts in the industry, and changes in the competitive landscape and regulations. customers are encouraged to participate in the implementing process. companies may utilise their existing digital ecosystems and/or establish new strategic networks and partnerships and cooperate with the academic community for knowledge transfer. correani et al. (2020); ivancic et al. (2019); loonam et al. (2018); osmundsen et al. (2018) table 1. enablers of successful digital transformation data and technology infrastructures are important assets in this never-ending process (correani et al., 2020; gaffley and pelser, 2021). companies may need to accept that digital australasian journal of information systems dang-pham et al. 2022, vol 26, selected papers from acis 2020 digital kaizen 6 transformation introduces new cyber security and privacy issues due to the adoption of new technologies and the increasing need for data collection. thus, rather than fear, companies should be aware of and face these cyber risks (agrawal et al., 2019). top managements need to recognise data as an important asset for business, and business data (e.g., data on personnel, financial reports, marketing information and sales performance) should be synchronised and made accessible at a single source (gaffley and pelser, 2021). companies develop their human capital for digital transformation by retaining and improving employees’ digital skills, as well as by fostering a culture for digital learning (gaffley and pelser, 2021). employees need to have specific skills and capabilities to fully seize the opportunities created by digital technologies (correani et al., 2020). the lack of digital skills and talents is considered one of the most significant barriers to digital transformation (agrawal et al., 2019). managers and leaders must therefore devise proper training programs to improve employees’ digital capabilities and to foster a digital spirit (correani et al., 2020; ivancic et al., 2019). both the involvement of internal and external stakeholders is essential. for instance, new digital solutions should be designed according to an end-to-end customer journey so that the new products and services offer novel values (loonam et al., 2018). integration with digital ecosystems is necessary for successful digital transformation, such as companies should cooperate with the academic community to enable knowledge transfer (ivancic et al., 2019; osmundsen et al., 2018; rueckel et al., 2020). digital transformation not only involves digitising existing processes, forms, documents, and services, but also digitally leveraging the relationships with their stakeholders (mergel et al., 2019). for instance, to maintain order in its business ecosystem, alibaba constantly pushes new certification services forward to verify the credits and qualifications of both buyers and sellers using the large amount of transaction data accumulated on its platform (li et al., 2018). table 1 summarises the identified enablers of digital transformation and their descriptions that we extracted from our review of the extant literature. 2.3 dynamic capabilities and digital transformation scholars have investigated digital transformation through various theoretical lens. among these, the dynamic capabilities framework (teece, 2007) has been widely applied and considered as a consistent approach to study digital transformation (ellström et al., 2022; warner and wäger, 2019). at the core of this framework is a hierarchy of capabilities, ranging from ordinary capabilities i.e., operational processes that are carried out by most companies, to mircofoundations i.e., less commonly employed routines that allow firms to integrate or reconfigure resources, and to dynamic capabilities at the top of the hierarchy (teece, 2018). dynamic capabilities are closely related to digital transformation since they enable firms to formulate strategies for refining existing and designing new business models in considerations of current and future contingencies, which are the key objectives of digital transformation programs (warner and wäger, 2019). according to teece (2007) what matters for organisations is corporate agility which is achieved through three dynamic capabilities, namely sensing, seizing, and transforming. sensing capabilities focus on a firm’s scanning of the ‘local’ and ‘distant’ environments to detect opportunities and threats and involves research and development activities and engagement with different stakeholders. once opportunities are detected, seizing australasian journal of information systems dang-pham et al. 2022, vol 26, selected papers from acis 2020 digital kaizen 7 capabilities allow firms to capture the values of those opportunities, mainly through informed decision-making, investments, and commercialisation strategies. the re-designing or creation of new business models or architectures, with updated technological competences, are key activities that reflect the seizing capabilities (teece, 2007). transforming capabilities concern the firm’s redeployment and reconfiguration of resources to sustain profitability in response to the changing business environment. the dynamic capabilities framework is used to study firms’ change capacity and to inform strategies for effective digital transformation (warner and wäger, 2019). prior studies developed and proposed process models focusing on dynamic capabilities, which describe the relationships between these capabilities, triggers and enablers of digital transformation, and identify the necessary mircofoundations or routines for developing the dynamic capabilities (see e.g., chirumalla, 2021; ellström et al., 2022; warner and wäger, 2019). for instance, data-driven sensing, flexible root cause discovery, agile cross-functional teams, continuous improvement strategy, and training support were determined to be mircofoundations for developing capabilities for digital transformation (chirumalla, 2021). similarly, ellström et al. (2022) identified cross-industrial and inside-out digital sensing, determining enterprise boundaries i.e., what to do in-house and what to outsource, and decomposing digital transformation into specified projects as mircofoundations. the dynamic capabilities and the specific mircofoundations are linked to the enablers of digital transformation, some of which were identified in our review above, such as data infrastructure, executive support, and strategies (chirumalla, 2021; warner and wäger, 2019). 2.4 the kaizen philosophy kaizen is a japanese business philosophy which refers to the “continuous improvement” approach led and supported by management. it involves the concept of change (kai) and to become good (zen) (newitt, 1996; von leipzig et al., 2017). therefore, organisations adopting a kaizen philosophy continuously identify and develop new or improved processes to achieve outcomes that contribute to organisational goals (brunet and new 2003). the kaizen philosophy and methodology focuses on three aspects: process, standards, and people (masaaki, 1986; suárez-barraza and lingham, 2008). first, kaizen emphasises the idea that processes need to be improved before good results can be obtained (masaaki, 1986). due to the rapid transition of business landscapes, organisations are in a constant state of flux (perkin and abraham, 2017). the philosophy emphasises the key idea that there is no end to the continuous improvement process that leverages good teamwork and flexibility in performing work (wickens, 1990). in this sense, it promotes process-oriented thinking rather than result-oriented thinking (wittenberg, 1994). second, standardisation is another essential feature of the kaizen philosophy (wittenberg, 1994). kaizen focuses on establishing and maintaining standards, then making small and continuous improvements while following such standards. standards refer to policies, guidelines, rules, and procedures (masaaki, 1986). these standards are to be reviewed and revised whenever people are unable to adhere to them, so that their effectiveness is reinforced, and they remain the practical guidelines for the employees (wittenberg, 1994). third, kaizen is people-centric and involves everyone within an organisation. it introduces a force of making improvements that draws on a spirit of team cooperation (malloch, 1997). australasian journal of information systems dang-pham et al. 2022, vol 26, selected papers from acis 2020 digital kaizen 8 kaizen assumes that positive changes in people’s attitudes and efforts are more likely to produce improvements in the long run (wittenberg, 1994). similarly, researchers suggested group-oriented kaizen as a mechanism to improve work methods, routines, and procedures (masaaki, 1986). kaizen not only starts with people’s needs and focuses on their efforts, but also concerns the collaborations between employees during work processes. while kaizen is widely applicable as a management philosophy and methodology, previous studies discussed whether kaizen could hinder innovation. while researchers suggested that innovation is implemented by abrupt changes, kaizen refers to a gradual process to introduce improvements (wittenberg, 1994). as a result, the philosophy is said to be better suited for a slowly growing economy, whereas the capability to come up with innovations is needed for a fast-growing economy (wittenberg, 1994). in practice, the kaizen philosophy covers many management techniques including quality circles, total productive maintenance, just-in-time, and robotic process automation (wittenberg, 1994). for example, masaaki (1986) highlighted that kaizen is an overarching concept for total quality management, which is a system of management where every employee continuously improves their standards of work in every aspect of a company’s operation. likewise, mizuno (1988) emphasised that companywide quality control, a japanese-rooted system for integrating quality technology in different departments to achieve customer satisfaction, is also based on the kaizen approach. in sum, the philosophy has affected many organisations’ productivity, and supported businesses to produce highquality organisational outcomes (macpherson et al., 2015; newitt, 1996; singh and singh, 2009). the literature review suggests that kaizen and digital transformation share similar characteristics. first, kaizen focuses on making continuous improvements, while digital transformation demands continual changes and innovative technology deployment (newitt, 1996; von leipzig et al., 2017; vial, 2019). second, kaizen aims to influence organisations’ productivity to achieve better customer satisfaction, and similarly, changes enabled by digital technologies focus on bringing about value creation processes that benefit both organisational activities and a company’s stakeholders (macpherson et al., 2015; hess et al., 2016; wessel et al., 2020). third, both kaizen and digital transformation draw on people and processes to encapsulate the changes (masaaki, 1986; suárez-barraza and lingham, 2008; hess et al., 2016; wessel et al., 2020). these similarities and connections between kaizen and digital transformation support the conceptualisation of the digital kaizen concept as an approach to digital transformation. 3 research approach 3.1 research method and data collection the objective of this research is to investigate digital kaizen as an approach for conducting digital transformation. given the nature of our research inquiries and the uniqueness of the approach, as developed by fpt software, we decided to follow a qualitative approach to explore the phenomenon (creswell and creswell, 2018). specifically, we opted for the interpretive case study approach (walsham, 1995), which involved conducting in-depth interviews with three key informants who would be reasonably expected to have the best knowledge about the research topic: (1) the chief executive officer (ceo), (2) the chief australasian journal of information systems dang-pham et al. 2022, vol 26, selected papers from acis 2020 digital kaizen 9 digital & technology officer (cd&to) cum executive vice president, and (3) the head of digital innovation (hdi). they were working in ho chi minh city, ha noi, and japan, respectively. interpretive case studies aim to understand both the context and processes related to the use of information systems, and how these two elements influence each other (walsham, 1995). the research took place in a single organisation rather than several case organisations. investigating a representative case introduces ideal types, as opposed to multiple cases, that are considered to be prototypical of a phenomenon (kautz and bjerknes, 2015). while the generalisability of the findings drawn from this approach may be limited, the interpretive case study provides a detailed account of empirical findings that contribute to the existing literature. we conducted our first interview with the hdi, and then we were introduced to the cd&to and ceo through the snowballing method. the open-ended interviews were conducted online via skype and audio-recorded, each of which lasted one hour on average. after each interview, we reviewed the findings and revised the questions to follow up with the next informant. the hdi and cd&to were interviewed twice, where the second interviews aimed to clarify some findings and probed for more information, which resulted in a total of five interviews. the interview data was coded openly and thematically analysed for similarities and patterns, as well as for disparities. the findings were sent back to the informants for feedback to ensure external validity (dubé and paré, 2003). direct quotes are provided in this paper to ensure research rigor (dubé and paré, 2003). initially, we announced our research objective to investigate strategies for conducting largescale digital innovation on linkedin for a month. among the received acceptances from three different companies to participate in the research project, we selected to conduct this research with fpt software for three reasons. first, it is an internationally recognised and established technology and digital transformation enterprise. second, it offered the opportunity to investigate the digital kaizen methodology, which we considered as a potential for generating findings that could contribute to both theory and practice. third, the company was highly committed to the project and promised access to key executive members for data collection. our qualitative data set contains a small set of five interviews with three key informants at fpt software, who are executive members of the company. while this can be considered a limitation of our study, such an approach was feasible during the pandemic, when strict lockdowns that lasted for several months in vietnam created challenges for data collection. to add interpretive rigor, we conducted additional desk research which involved examining documents, news articles, reports, and other materials concerning the company’s digital kaizen approach. moreover, we consider the five interviews with the key informants to be sufficient and adequate to answer our research questions pertaining to the characteristics of the unique digital kaizen approach and its potential in guiding digital transformation efforts due to the following reasons. first, the key informants, whom we interviewed, are among the few staff at fpt software who contributed to the creation of the digital kaizen approach and observed its implementation since its inception. therefore, these informants hold the vision and in-depth knowledge about the characteristics of the approach and how it was intended to be used for australasian journal of information systems dang-pham et al. 2022, vol 26, selected papers from acis 2020 digital kaizen 10 digital transformation. second, our study does not investigate and analyse the impact of digital kaizen on elements at the micro level of the company such as the workforce or work processes at fpt software, therefore input and insights of other stakeholders such as other employees were not within the scope of this research and were omitted. third, our study is exploratory by nature and despite the process model for digital kaizen, which we propose, does not aim at achieving a generalisation of our findings. thus, it was critical to focus on extracting insights from the relevant informants to understand the phenomenon rather than on achieving a large sample size. 3.2 case setting fpt software is a part of fpt corporation – a large technology and it services group headquartered in vietnam (fpt software, 2020). fpt software was established in 1999 with the mission to bring technological innovations to its international customers while being dedicated to the country’s prosperity and developing skills of young vietnamese employees. in 2020, fpt software achieved nearly usd$1.2 billion in revenue and has become one of vietnam’s largest and fastest growing it companies that is able to provide digital solutions to clients globally, ranging from the asia-pacific, oceania, north america and to europe. currently, it has more than 100 customers across 45 countries with key companies in diverse sectors, ranging from automotive industry (e.g., toyota) to the aviation industry (e.g., airbus and palantir). in the next 10 years, fpt software sets the ambition to become one of the top 50 technology service providers in the world. kaizen framework digital kaizen step 1: discover improvement potential • maintain awareness, through experience and training, of the opportunities or challenges that can bring changes to improve processes the organisation’s processes • cross-functional pain points are identified by conducting regular and mandatory workshops which are attended by department managers step 2: analyse the current method of work • understand the process inside the improvement area, use various analysis techniques • managers provide feedback about operational issues which are encountered during the daily work and collaboration step 3: generate original ideas • brainstorm the problems using creative and analytical thinking of each participant • managers discuss and come up with digital transformation solutions to address crossfunctional pain points • a steering committee evaluates the proposed projects to determine their priority based on clear criteria steps 4 & 5: develop and implement the plan • decide who will do what, where, how, by when, and why • coordinate the different implementation teams • flexible change management techniques are used to increase users’ adoption of improvement solutions • focus is on small and quick wins to raise employees’ morale with the goal of implementation not only being immediate results but also nurturing change capability and digital maturity step 6: evaluate the new method of work • compare results with the conditions before the improvement initiatives and decide whether further improvements are necessary and possible • digital transformation projects are evaluated based on specific criteria which are linked to strategic objectives australasian journal of information systems dang-pham et al. 2022, vol 26, selected papers from acis 2020 digital kaizen 11 • if successful, standardise the solution • successful digital transformation projects are refined, packaged, and sold to clients table 2. mapping the kaizen framework onto the digital kaizen approach fpt software created its own digital transformation methodology which they called ‘digital kaizen’ that involves conducting a series of small yet effective digital transformation initiatives within short time frames, where they mapped the six steps of the kaizen framework (smalley and kato, 2010), discover improvement potential, analyse the current method of work, generate ideas, develop and implement the improvement plan, and evaluate the new method of work to fit their objectives. table 2 summarises the mapping of the kaizen framework onto the digital kaizen approach at fpt software. the company’s digital transformation principle is simple: ‘driving business and technological transformation through people engagement.’ to achieve that, fpt software identified the ‘pain points’ or cross-functional problems that are encountered by employees at the operational level in their daily work. by adopting digital kaizen, according to our informants the company’s digital transformation delivers continuous improvements in a manner that maximises the adoption of digital technologies, optimises procedures, and minimises waste. the company’s digital transformation principle is simple: ‘driving business and technological transformation through people engagement.’ to achieve that, fpt software identified the ‘pain points’ or cross-functional problems that are encountered by employees at the operational level in their daily work. by adopting digital kaizen, the company’s digital transformation delivers continuous improvements in a manner that maximises the adoption of digital technologies, optimises procedures, and minimises waste. out of five digital transformation initiatives that recently had been implemented in fpt software, two, myfpt and akalink, have been investigated in this research. these two examples highlight how the company enacts its digital transformation strategy by relating to the alignment between employees’ needs and organisational goals, and how the focus on continuous improvements through delivering small digital transformation initiatives provided a spill over effect that nurtured and lead to a high level of digital maturity (kane et al., 2019) of the company. this focus also contributed to the transformation of the company’s business model from a software development and it outsourcing firm to an it consultancy. 4 analysis and findings to manage its digital business transformation, fpt software adapted the kaizen philosophy and created its own approach which they named digital kaizen. digital kaizen prioritises making a series of small initiatives and quick wins that build towards long-term success rather than creating a ‘big-bang’ disruption. it aims to continuously tackle cross-functional issues that are aligned with their strategic business goals, through implementing digital technologies that improve business processes and people engagement. these small, when successful, transformations in turn lead to sustainable and scalable digital transformation solutions. cross-functional pain points are the starting point of the company’s stepwise digital transformation strategy. they represent the issues that exist in the company’s operations that australasian journal of information systems dang-pham et al. 2022, vol 26, selected papers from acis 2020 digital kaizen 12 have direct impacts on the employees. to identify these, the company conducts regular and mandatory workshops throughout the four quarters of a year, in which managers from different departments participate and provide feedback about the operational issues encountered during the daily work interactions between departments. “the process of generating cross-function pain points addresses the concerns of employees who are directly involved in daily business operations. every few months, fpt software organises a workshop that aims to understand these cross-functional pain points. after the annual core missions of the entire company are identified, managers of all departments, who are related to those missions, gather and perform brainstorming activity to produce a list of pain points. these pain points are then ranked and prioritised based on some metrics. digital transformation projects are created to address the important pain points, and specific tasks are assigned to the relevant teams.” – cd&to these joint workshops aim to address two issues of digital transformation. first, while it may be easier for each department to recognise and address its own pain points, it is of greater importance to fpt software to define the cross-functional pain points that affect multiple departments and hinder shared work processes. second, the cd & to appreciated that digital transformation is more effective when it is motivated by the common needs of different organisational functions, as compared to by the needs of each company’s separate department (sidhu, 2019). “when it comes to identifying organisational problems, people often think that the problem is caused by other departments and not by themselves; department a may tend to attribute the pain point to department b and vice versa. as a result, nobody tackles the pain points since they think those pain points are not their responsibility. solving these problems will improve the work engagement between departments and thus benefit the whole business process. as such, it is effective when cross-functional teams work together in the workshops to identify the pain points.” – cd&to then, project proposals for digital transformation solutions are developed to address the pain points, and these proposals are evaluated by a steering committee to determine their priority. among the evaluation criteria used to assess the solutions’ priority, the key criteria are feasibility, urgency, and potential values. first, feasibility refers to the speed of the project’s completion and the evaluation of return on investment (roi) that is typically anticipated to be achieved within three or six months. second, urgency is determined by the critical level of the pain points that the proposed projects aim to tackle. for example, a project that focuses on a new business process that is needed for complying with a new regulation has a higher priority than others. third, a proposed project is deemed to have high potential value if it contributes to the key annual missions of fpt software, which often aim at providing value to the human and technological resources of the company. “the rule of thumb is that no project is allowed to take more than 6 months to complete. after six months, we should be able to evaluate the project’s roi. not only the project must support the company’s business strategy in the next three to five years, but ideally it should also yield enough revenue to sustain itself, or this revenue can be re-invested in other projects. apart from solving the critical pain points, the project is considered as valuable when it benefits the personal development of the employees or increases technological capital, which then contribute to other digital transformation projects.” – ceo australasian journal of information systems dang-pham et al. 2022, vol 26, selected papers from acis 2020 digital kaizen 13 after completion, successful digital transformation projects are refined and packaged together with the digital kaizen approach as digital transformation solutions, which as part of it consultancy assignments are sold to fpt software’s clients afterwards. from its database of clients, fpt software identifies and approaches clients, who may have the issues or pain points that can be addressed by such digital transformation solutions. as such fpt software does not primarily sell off-the-shelf software packages anymore but provides bespoke digital transformation solutions that are tailored to their clients’ business context and needs. “when selling digital transformation solutions to our clients, we sell the methodology rather than the software package. while digital technologies are just tools, digital kaizen is a methodological framework that guides the whole transformation process. when working with clients, fpt software serves as a consultant or advisor and not as a one-time seller. through conducting workshops with the clients, which are similar to what we do in fpt software, we identify clients’ pain points and come up with the solutions based on their context and needs. this approach therefore encourages the clients to adopt the digital kaizen spirit and to embrace continuous improvements rather than sticking to one fixed solution.” – ceo two notable internal digital transformation projects that successfully addressed two of fpt software’s pain points were myfpt and akalink. they were considered by the informants as key digital transformation solutions of fpt, which contributed significantly to the business-critical operations of the company in 2020. while myfpt aimed to tackle pain points that were related to retention, akalink focused on solving problems related to both resource fulfilment and retention. as these projects were concerned with achieving human resources-related objectives, they illustrate the process from identifying pain points to developing digital transformation solutions that were aligned with fpt’s strategic objectives. the following three subsections focus on the analysis of these two initiatives. 4.1 myfpt: a mobile app for people engagement from 2017 to 2018, employee engagement was one of the key challenges of fpt software. due to its fast-growing development, the number of employees increased significantly, which made it difficult for managers to provide personalised engagement with their employees and retain them. at that time, it was common for a manager to have about 200 employees who reported directly to him/ her. in addition, there was a lack of adequate processes and systems to evaluate the performance of these employees, thereby leading to poor performance review experiences and employees felt unacknowledged for their efforts. as a result, this one-year period witnessed a high turnover rate of 30 percent, where approximately 1,500 of 5,000 new employees left the company. at the same time, the interviews with informants revealed that they considered the time spent on training new staffs was a possible waste. it took every new employee 14 days to understand the procedures and systems and get along with key supporters before they could perform their tasks. this time-consuming onboarding process was not a challenge of any particular department but a cross-functional problem for all involved departments. the ineffective employee engagement methods and lengthy onboarding process were identified as the crossfunctional pain points between multiple departments and across organisational levels. these identified pain points were addressed by developing and introducing an all-in-one mobile application called myfpt, which transformed staff engagement and onboarding australasian journal of information systems dang-pham et al. 2022, vol 26, selected papers from acis 2020 digital kaizen 14 process. the mobile app contains a wide range of functionalities of both an enterprise social network and an online marketplace. for example, with a database of frequently asked questions (faqs), new employees have easy access to and the opportunity to familiarise themselves with the work procedures and systems in fpt software. in addition, personalised feedback about performance from managers to employees can be sent via this mobile app, and the employees can send each other appreciations and congratulations on special occasions. the app also provides features to improve the employees’ work experience at fpt software. for example, they can give ratings and reviews for the meals served at the company’s cafeteria, and a function is implemented on the app to suggest new or matching dishes based on each employee’s unique dining patterns and needs. by interacting with the app, employees can earn and receive reward points, which the company calls ‘company golds’ and use these to buy lunches at work or other goods. a series of engagement programs were promoted on the mobile app to increase the employees’ adoption of the app. for example, fpt software incorporated myfpt into its social event “run for green”, in which the company pledged to plant a tree for every 10kilometer run by each employee as recorded on the app. other work-related daily activities are also integrated into the app. for instance, the employees get extra points when they commute to work by company buses. the tangible benefits (e.g., bus tickets, redeemable points) are presented as an incentive for the employees’ trial of the application, and the app’s features (e.g., food reviews, feedback, appreciation) are used to maintain their engagement with the app in the long run. the hdi of fpt software explained the approach to persuade employees to adopt myfpt as part of this digital transformation project: “while the digital kaizen as a framework instructs us on what and how to implement digital transformations, top management needs to be creative in designing every implementation step. on one hand, myfpt is created to serve the essential needs of the employees to engage with the company; therefore, the initial adoption by some employees can be anticipated and is understandable. on the other hand, when the app as part of the digital transformation needs to be adopted at a larger scale, it needs to be flexibly changed to promote its adoption.” – hdi after six months of implementing the myfpt project, several achievements were recorded. first, new employees no longer had to spend too much time to get familiarized with the company's procedures and systems, as they could find the necessary information on myfpt. as a result, the time required for the onboarding process was reduced from 14 days to five days, which amounted to a large saving in terms of monetary value. second, the human resources department and top management were now able to engage better with the employees and retain new hires more effectively, which led to a significant decrease in the company’s high turnover rate. currently, 15,000 employees use myfpt daily for various purposes, from booking transport and ordering food, to reviewing their evaluations and communicating with colleagues. the large number of active daily users indicates a high level of engagement between fpt software and their employees. in total, 22 strategic pain points were resolved through the digitally transformed onboarding and engagement processes. 4.2 akalink: a digital platform for skill inventory management during the 2018–2019 period, fpt software faced an issue related to the management of the company’s skill inventory. traditionally, human resources management activities involve updating and assessing the current skill inventory, forecasting the availability for core skills australasian journal of information systems dang-pham et al. 2022, vol 26, selected papers from acis 2020 digital kaizen 15 and sourcing talents to meet business needs in a timely manner. as the company was growing rapidly, there was a growing demand for increasing the speed of these activities and for continuous monitoring of the organisational skill inventory. these requirements were identified as the cross-functional pain points of multiple departments that needed to be addressed. fpt software developed akalink to improve the skill inventory management processes. akalink is an online digital platform that provides project management and enterprise social networking functionalities. employees register and update their skills after every project, and their colleagues can endorse each other’s skills. akalink also allows employees to volunteer for tasks in other departments, and the system recommends new projects to the employees which match their nominated skills and career development plans. as such, akalink empowers employees to make decisions on their career development. the platform also provides top management with more accurate and timely information about the employees’ participation in projects, and an overview of the skills that are currently available in the company. this information in turn enables the top management to plan for training programs and hiring strategies more effectively. it was estimated that this digital transformation project had yielded a total revenue of about usd$700,000 in june 2020. 4.3 myfpt and akalink as digital kaizen-driven digital transformation solutions for both digital transformation solutions, the six steps of digital kaizen were executed as follows: in step 1 (discover improvement potential) top management set strategic objectives and department managers were organised in workshops to identify and discuss pain points as a foundation to generate relevant digital transformation solutions; in step 2 (analyse current method of work) lack of adequate employee engagement and an inefficient onboarding process were identified as particular cross-functional pain points. in step 3 (generate original ideas), potential digital transformation solutions which aimed at improving employee engagement and onboarding were prioritised based on their feasibility, urgency, potential values, and alignment with the organisation’s strategic objectives. subsequently in steps 4 and 5 (develop and implement the improvement plan), change management techniques were employed to increase the employees’ adoption of the digital transformation solutions; and in step 6 (evaluate the new method of work), the achievements of the new methods of work were recorded in terms of monetary gain, time saved, and number of addressed pain points. as an additional outcome, the employee engagement platform akalink has been advertised and successfully been sold to clients. the two projects are instances of fpt software’s digital transformation solutions that resulted in not only monetary values but according to our informants, their evaluation showed that they also had spill over effects that impacted the whole organisation (see also subsection 3.2 ‘case setting’). first, as employees engaged in these digital transformation projects, they further increased their readiness to adapt to the organisational changes related to using new technologies. second, these projects addressed and resolved cross-functional pain points that directly impacted operational staff in their daily work, which in turn increased their job satisfaction. third, as the pain points were successfully addressed in a short time frame of three to six months, the employees’ morale increased and top australasian journal of information systems dang-pham et al. 2022, vol 26, selected papers from acis 2020 digital kaizen 16 management gained trust from the employees, which again motivated their participation in other, subsequent digital transformation activities, as the ceo stated. “it should be noted that digital transformation projects must benefit employees at the operational level first. digital transformation projects should not be about creating new tasks for them but offer new solutions to increase their productivity and performance. this will make the employees find the digital transformations more relevant to them and feel willing to get involved in transformation activities.” – ceo 4.4 the enablers of digital transformation at fpt software fpt software’s initial motivation for digital transformation was to improve in-house business processes by addressing their cross-functional pain points with digital technologies. in light of the first enabler that we identified from the literature review (see table 1), strategy and business model, fpt software’s strategy for digital transformation was originally internally focused. the approach for planning and implementing the digital transformation at fpt software was operationalised and involved regular assessments of the status quo, as shown in their conducting of the frequent, mandatory workshops to identify cross-functional pain points. our informants at fpt software emphasised the role of the strategic objectives as the guiding vision with which digital transformation efforts and solutions had to align. however, fpt software did not stop at internally addressing the pain points, they also commercialised and sold their digital transformation solutions to their clients. their motivation for digital transformation is to both improve internal processes and to service the market at the same time. the challenge of having such a dual motivation is to ensure the relevance of the company’s internal priorities to those of the clients, so that their commercialised digital transformation solutions will be appealing to and accepted by their clients. in our case, fpt software leveraged the advantage of being a large enterprise with a long history of doing business with both local and global customers, which provided them with the knowledge of the common pain points of their clients (see also subsection 4.5 ‘the transformation of the company’s business model’). with regard to the two enablers, organisational participation and human resource management, it can be seen that learning and collaboration are also the prevalent themes of the digital transformation at fpt software. for fpt software, the benefits of their initial digital transformation are not limited to the developed digital artifacts as digital transformation solutions, but more importantly the participating employees also improved their digital skills and mindset i.e., their digital readiness for future transformation solutions (see also subsection 4.3 ‘myfpt and akalink as digital kaizen-driven digital transformation solutions). interestingly, fpt software realised the transformation of the employees’ digital readiness as a spill over effect rather than what they had intended to achieve, suggesting that digital transformation may bring unexpected results to organisations. top management support of digital transformation was not explicitly mentioned in our interviews. nevertheless, it can be implied that a common vision for digital transformation was shared among managers and employees, as was evident through their active participation in the assessment of the company’s status quo and the implemented change management techniques. in terms of change management, fpt software’ approach for implementing digital transformation is a stepwise and iterative approach, following the digital kaizen australasian journal of information systems dang-pham et al. 2022, vol 26, selected papers from acis 2020 digital kaizen 17 methodology that emphasises making small and quick wins during the transformation process to maintain employees’ high morale. while the role of a digital champion is mentioned in several digital transformation studies, such role did not surface in our interviews with fpt software. instead, the successful digital transformation implementation at fpt software relied more on their execution of employee engagement techniques, including the use of incentives to encourage the adoption of digital technologies. as a software development and it outsourcing company, it is unsurprising that fpt software had the capabilities to develop the two applications in-house, which is part of their ‘data and it infrastructures’ enabler of digital transformation. however, the interviewees claimed that fpt software’s successful digital transformation comes not only from their technological infrastructures but also their active assessment of the status quo that was carried out by conducting the workshops to identify operational pain points. with regard to digital ecosystems, we did not find evidence that, beyond fpt software’s regular business interactions with their large client base, the company engaged further with external stakeholders such as customers, business partners, or the academic community in their initial digital transformation projects. 4.5 the transformation of the company’s business model the application of the digital kaizen approach also transformed fpt software’s business model, from being a software development and it outsourcing company to a digital transformation and change management service provider. according to the cd&to and the hdi, this transformation of the business model was motivated both intrinsically and extrinsically. “between 2018 and 2019, fpt software changed its strategic business direction to not only provide software outsourcing services, but also to become a global digital transformation services provider. in other words, fpt software offers digital transformation services to multinational clients, to fpt corporation and fpt software itself.” – cd&to “there was a change of direction within the company. we started to rely on our core strengths such as our staff and large data sets to build and use our own digital solutions. once we found the solutions effective, we used them as case studies and sell them to international corporates. after convincing these corporates to purchase our solutions, we then sell them to the local market.” – hdi externally, the hdi explained, that the it outsourcing market had changed drastically between 2018 and 2019, where competition became more intense, and clients became more demanding with increasing needs for open-source solutions, more customisation, and complex integration of multiple data sources for software solutions. the it outsourcing model of fpt software at that time was deemed no longer sustainable in such market. internally, the adoption of the digital kaizen approach to continuously and rapidly enhance the workplace with digital innovations had resulted in fpt software becoming, quoting the ceo, a giant ‘test bed’. with the identity of a test bed in mind, the board of management realised the large scale of fpt software – with more than 16,000 employees and multinational clients in key markets such as the asia-pacific, japan, oceania, north america and europe, in particular germany – as a strategic advantage, that most of the internal problems australasian journal of information systems dang-pham et al. 2022, vol 26, selected papers from acis 2020 digital kaizen 18 addressed by the company’s digital transformation solutions would be relevant to almost any other company. “we are implementing the digital transformation solutions for the whole fpt corporation and not just for fpt software – an enterprise with more than 36,000 people, seven member companies, and offices in more than 30 countries. it is indeed a complex organisation. therefore, we may have a lot of pain points that are also encountered by other companies.” – ceo when the transformation of the business model happened, changes in terms of organisational structures and business objectives were also implemented. for instance, the hdi shared that: “in the past, we would take jobs from any clients without a specific focus on any domains. from 2018, we created a unit to cater for manufacturing and logistics clients in japan, the us, and germany, due to the fast development in the manufacturing and logistics sectors in these countries. then, when our clients in germany asked us to transform their outdated banking systems to modern ones, we also created a dedicated team for this line of work.” – hdi the business objectives of fpt software now include developing long-term partnerships with their clients, not only through providing high-quality services, but also through building trust and confidence by projecting a professional image of a company with adequate governance, fast problem-solving capability, and employees with an excellent workplace, which all resulted according to the informants from the adoption of digital kaizen. displaying such a professional image was particularly important as it signalled the company’s resilience to their clients, in particular during the pandemic period. 5 discussion our case analysis supports wessel et al.’s (2020) argument that digital transformation redefines organisations’ value propositions and leads to the creation of a new organisational identity. throughout its digital transformation endeavours, fpt software not only deployed digital technologies such as cloud computing and big data analytics but also experienced the constant shift of the company’s business model, products, and processes (wessel et al., 2020). fpt software changed from a software development to a digital transformation consultancy company (fpt software, 2019) and started offering digital transformation services as a value proposition to its customers. this change of organisational identity is mirrored in the company’s transformed business strategy and organisational structure, with task forces in charge of digital transformation solutions, a committee responsible for evaluating digital transformation projects’ priority, and mandatory workshops in place where managers in incubator teams identify cross-functional pain points and develop proposals for possible solutions. 5.1 the kaizen framework and digital kaizen the digital kaizen methodology was built on the principles of the original kaizen philosophy for the context of implementing digital transformation. in line with the original version, this methodology emphasizes the three aspects: process, standards, and people (masaaki, 1986; suárez-barraza and lingham, 2008) (see also subsection 2). for instance, rather than making a big-bang disruption, incremental digital changes are conducted within the organisation. this approach encourages digital transformation to undergo a continuous australasian journal of information systems dang-pham et al. 2022, vol 26, selected papers from acis 2020 digital kaizen 19 process with no ends. furthermore, people are recognized as the main driver of digital transformation activity, and increased people engagement leads to improved organisational change management capability and higher digital maturity. the digital kaizen approach adds a new perspective to the current kaizen literature. while kaizen has been applied widely in management practices of various industries (wittenberg, 1994), digital kaizen was created at fpt software specifically for digital transformation practices. the original kaizen approach is primarily applied in production to reduce costs, minimise wastes, and to increase productivity, whereas digital kaizen aims at facilitating the effective and efficient implementation of digital innovations that transform people and business processes. the focus on solving cross-functional pain points and implementing digital innovations to achieve quick and small improvements in the organisation expands beyond eliminating waste and improving efficiency in the production line. the application of kaizen philosophy in the field of digital transformation signals the capability of kaizen in creating innovations. in contrast to some prior studies that argued kaizen is more suitable for a slowly growing economy (wittenberg, 1994), our case study of fpt software shows that organisations can practice kaizen and generate innovations at the same time (imai, 2007). with the help of digital technologies, organisational initiatives that aim to bring about new business models, products, and value creation processes can be created clients. 5.2 the dynamic capabilities based digital kaizen process model we have so far focussed on digital kaizen’s role for the development of digital transformation solutions and its impact on the transformation of the case company’s business model. we also identified the enablers of digital transformation at the company and discussed the original kaizen framework in relation to digital kaizen. we established earlier (see subsection 2.3) that digital transformation has been investigated through the dynamic capabilities framework. we will now discuss and apply the framework in relation to the digital kaizen methodology based on our empirical data and its analysis which leads us to develop and propose a digital kaizen process model informed by the framework. we find an alignment between digital kaizen and the dynamic capabilities framework and argue that digital kaizen encompasses mircofoundations, which are required for the development and exploitation of dynamic capabilities. dynamic capabilities refer to an organisation’s ability to integrate, build, and reconfigure internal and external competences to address the changing environments (teece, 2007). underpinning these capabilities are foundational capabilities, the mircofoundations, which enable the sensing, seizing, and transforming activities of a company (teece, 2007). the activities of the digital kaizen approach as performed at fpt software (see section 4) can be considered as mircofoundations because (a) they contribute to the sensing, seizing, and transforming capabilities and (b) they are not ordinary processes that would be carried out by any firm (teece, 2007). in the following three subsections we describe and discuss how the dynamic capabilities, i.e. sensing, seizing, and transforming, the steps of digital kaizen, and the activities and mircofoundations are related and mapped onto each other in the digital kaizen process australasian journal of information systems dang-pham et al. 2022, vol 26, selected papers from acis 2020 digital kaizen 20 model and in subsection 5.3 we then discuss intended and possible target outcomes of as part of the process model. figure 1 summarises the model. 5.2.1 sensing and the digital kaizen process model sensing refers to the analytical methods, tools, and techniques that companies use to detect, identify, and filter opportunities for developing new products and services (teece, 2007). the learning capacities of the individuals within a company, innovation models such as open innovation, and generative sensing capabilities that use technologies such as artificial intelligence or big data, are part of the routines that contribute to this dynamic capability (chirumalla, 2021; warner and wäger, 2019). the processes related to r&d and working with suppliers and customers to gather market insights and select new technologies are the mircofoundations of the sensing capability (teece, 2007). in terms of digital kaizen, the mircofoundations reside in the process in which managers identify cross-functional pain points via mandatory workshops. digital kaizen stresses the need for collaboration between cross-functional teams, forming a continuous digital improvement strategy, and pursuing a bottom-up evolutionary approach, in line with chirumalla (2021). the sensing capability and its microfoundations relate to the ‘discover improvement potential’ stage of the digital kaizen approach, which aim to understand the current business situation and discover improvement opportunities (see table 2). 5.2.2 seizing and the digital kaizen process model seizing refers to the decision-making capability of a firm to invest in the identified opportunities and take actions (teece, 2007). seizing enables capturing the values of the identified opportunities by deciding the necessary changes that are needed by firms (ellström et al., 2022). the mircofoundations of the seizing capability include the processes and structures to create or revise business models, and build commitment by demonstrating leadership, culture, and effective communication (chirumalla, 2021; teece, 2007). being able to identify the cross-functional pain points is not enough to take actions. the digital kaizen process model provides further activities to address these identified problems, where managers come up with proposals for digital transformation solutions and based on a steering committee’s rating criteria rank both the pain points and the proposed solutions to satisfy the company’s strategic objectives. these are the mircofoundations of digital kaizen for seizing opportunities. the seizing capability and its microfoundations help to crystalise the discussed digital solutions and turn them into concrete action points, which are in line with the ‘analyse the current method of work’ and ‘generate original ideas’ stages of digital kaizen. 5.2.3 transforming and the digital kaizen process model transforming capabilities sustain the profitability of a business in a turbulent and uncertain environment (teece, 2007). transforming entails the redeployment and reconfiguration of assets and routines, which require mircofoundations such as achieving decentralised structures and managing complementary assets, as well as knowledge management and corporate governance capacities (chirumalla, 2021; teece, 2007; warner and wäger, 2019). the implementation of the digital solutions paired with engagement programs to encourage employees to participate in the uptake of the solutions reconfigures practices and assets and introduces new work processes and transformative technologies such as myfpt and akalink at fpt software to the workplace. this ultimately leads to a new business model australasian journal of information systems dang-pham et al. 2022, vol 26, selected papers from acis 2020 digital kaizen 21 which may include the commercialisation of these solutions as offerings and service packages to clients as was the case at fpt software. the transformation capability, which involves enacting changes and evaluating their outcomes, can be mapped to the ‘develop and implement the improvement plan’ and ‘evaluate the new method of work’ steps of digital kaizen. figure 1. a digital kaizen process model informed by the dynamic capabilities framework australasian journal of information systems dang-pham et al. 2022, vol 26, selected papers from acis 2020 digital kaizen 22 5.3 intended target outcomes of the digital kaizen process model with a focus on the kaizen philosophy’s aspects, process, standards, and people (see also subsection 2.4), the dynamic capabilities and their microfoundations intend to achieve some target outcomes that contribute to the set organisational goals. the sensing and seizing capabilities and their microfoundations prompt cross-functional collaboration and a shared understanding between organisational stakeholders. tangible outcomes of these capabilities are a list of pain points and/or opportunities, and the respective solutions. in digital kaizen, solving the ‘right problems’ is more crucial for digital transformation than ‘solving these problems the right way’. as such, all managers are involved in the identification of functional pain points that are later systematically evaluated and prioritised according to their relevance and importance. other intended target outcomes are created and realised through the transforming capabilities. adopting transformation solutions leads to changes in organisational processes. thus, a new business process and/ or a new business model might emerge. as employees are involved in all activities, and benefit from the changes, employees’ morale and agility are improved. at fpt software, the issues were identified by members from all departments, which supported a more objective and possibly accurate identification process. the focus on the pain points that directly impact the employees’ daily operations motivates employees to adopt digital solutions. the digital kaizen process model thus recommends and includes these activities and their micofoundations. while focusing on small successes and having an agile mindset are recommended for successful digital transformation (see e.g., henriette et al., 2016; agrawal et al., 2019; schaupensteiner et al., 2021), the complexity of large digital transformation agendas may lead to issues of “scope creep”, especially when the transformation aims at addressing multiple objectives. the digital kaizen approach encourages organisations to address problems that are cross-functional (see also subsection 5.2.1) and obtains measurable outcomes, e.g., monetary gain and time saved. the alignment between strategic and operational objectives is another important intended outcome. implementing digital transformation by decomposing a large transformation initiative into smaller initiatives with quick-win solutions paves the way for meeting strategic objectives (ellström et al., 2022). fpt software transformed from a software development to a digital transformation consultancy and solution provider. in this context, the company successfully repackaged and commercialised their in-house digital transformation solutions. the commercialisation of internal solutions is part of the company’s transforming capability, since the emerging solutions are re-packaged (i.e., redeployment and reconfiguration of resources) and sold to clients (i.e., to sustain profitability). the design and development of these solutions are based entirely on the organisation’s internal problems, which reflects their sensing and seizing capabilities. fpt software is not just reselling the digital transformation solutions i.e., software packages, but primarily the digital kaizen methodology and its success stories. prior research on dynamic capabilities and digital transformation recommended that organisations broaden their sensing capabilities by engaging with a wide range of internal and external stakeholders as well as by using digital infrastructures such as ai and big data technologies (chirumalla, 2021; warner and wäger, 2019). that is, maintaining an awareness australasian journal of information systems dang-pham et al. 2022, vol 26, selected papers from acis 2020 digital kaizen 23 of the business’s entire ecosystem is critical for sensing opportunities and threats (ellström et al., 2022; teece, 2007). the commercialisation of internally generated digital transformation solutions as a possible outcome of digital transformation in the proposed model derived from the empirical data at fpt software. it demonstrates that a company that uses sensing and subsequently other capabilities to gain awareness of its own situation, can actively influence its ecosystem through selling these solutions and the digital kaizen methodology. while we acknowledge that not all companies would want to sell their digital transformation solutions, this aspect contributes an interesting insight to the discussion of the impact of digital transformation on an organisation’s business model from a dynamic capabilities perspective. 6 future research this case study presents digital kaizen as a new methodology and practical approach for conducting digital transformation. as the key informants were leaders and managers, the findings reflect the leadership team’s beliefs and visions, rather than being based on a detailed account of how the approach impacted the activities at the micro level of the company. while we acknowledge this limitation as a further avenue for future research, the case analysis contributes insights into the philosophy, approach, and potential of digital kaizen as a process to guide digital transformation and build and exploit dynamic capabilities for digital transformation. we propose future research directions, which aim to advance theoretical and practical understanding of the process to implement digital transformation. the concept of enablers might give the impression of a checklist of prerequisites that indicate organisational readiness for digital transformation. however, digital transformation is a dynamic process where the current state is constantly assessed, and new digital enhancements are continuously added. thus, the enablers’ importance will change as a company progresses through different stages of digital transformation. at fpt software, we did not find evidence of external stakeholders’ participation or explicit involvement in an ecosystem during the company’s digital transformation process. this direct participation might take place at a later stage of the company’s digital transformation process. moreover, the enablers will probably also have different levels of importance according to the context of transformation e.g., the company’s position in the supply/value chain or industry. therefore, research in this area is necessary to understand the changing roles and importance of the enablers (as well as the inhibitors) of digital transformation. future research should also investigate the conditions and capabilities that enable a company to focus on sensing its internal problems, rather than scouting the entire ecosystem to seize digital transformation opportunities. our research suggests the integration of the kaizen philosophy into digital transformation practices. future research may determine a more rigorous and comprehensive list of criteria and techniques for ranking and prioritising digital transformation solutions that are parts of a complex transformation program. it will also be helpful to focus on investigating the practices, such as the workshops, to unpack complex digital transformation programs into meaningful and manageable solutions that can be carried out in the digital kaizen way. more empirical studies adopting the dynamic capabilities framework and our proposed digital kaizen process model are also necessary. the viability of digital kaizen in different contexts australasian journal of information systems dang-pham et al. 2022, vol 26, selected papers from acis 2020 digital kaizen 24 and workplace cultures, e.g., large versus small and medium enterprises should be considered. in this regard, action research to investigate empirically the digital kaizen approach will be valuable. the nature of digital transformation, as organisational changes that involve both technology adoption and change management, opens opportunities for research that can combine theoretical frameworks in both fields to unpack digital transformation-related phenomena at the micro level of an organisation. the relevancy or fit of the technological solutions is one of the many aspects that should be considered (goodhue, 1995), and a positive change experience or change history is another critical aspect that can contribute to the change management literature (pettigrew et al., 2001). 7 conclusion the digital kaizen methodology addresses the identification and resolution of crossfunctional pain points that are aligned with a company’s strategic goals, through implementing incremental digital improvements that enhance business processes and people engagement, which in turn lead to sustainable and scalable digital transformation solutions. we put forward digital kaizen and the proposed digital kaizen process model that encompasses dynamic sensing, seizing, and transformation capabilities as a viable digital transformation approach that leverages digital technologies to re-define an organisation’s value proposition and identity (wessel et al., 2020). our examination of the digital kaizen approach contributes to research on the relationship between dynamic capabilities and digital transformation as called for by e.g., vial, (2019) and warner and wäger (2019). its people-centric approach to conducting digital transformation identifies the employees as the main drivers of digital transformation, and people engagement as critical for developing and sustaining digital maturity. digital transformation programs are often large and complex, consisting of multiple related projects which depend on each other. consequently, it is useful to have an approach that separates such large programs into smaller and more manageable pieces of work. in our study, the digital transformation program at fpt software aimed at addressing several human resource-related issues, which were considered strategic objectives of the company. the workshops to identify the specific pain points and define digital transformation solutions, i.e., myfpt and akalink, proved to be effective for achieving the intended decomposition of the large overall envisioned digital transformation program. the case of fpt software confirmed that successful digital transformation is strategically motivated and can be carried out in a stepwise manner that employs change management and communication techniques to engage with the employees, especially by striving for and demonstrating quick wins during the transformation process to gain their buy-in. moreover, the study uncovered that the stepwise approach of digital kaizen contributes to a company’s digital readiness for further transformation. references agrawal, p., narain, r., & ullah, i. (2019). analysis of barriers in implementation of digital transformation of supply chain using interpretive structural modelling approach. journal of modelling in management, 15(1), 297-317. australasian journal of information systems dang-pham et al. 2022, vol 26, selected papers from acis 2020 digital kaizen 25 bekkhus, r. (2016). do kpis used by cios decelerate digital business transformation? the case of itil. paper presented at the digital innovation, technology, and strategy conference, dublin, ireland. in the proceedings of digit 2016. https://aisel.aisnet.org/digit2016/16 bharadwaj, a., el sawy, o. a., pavlou, p. a., & venkatraman, n. (2013). digital business strategy: toward a next generation of insights. mis quarterly, 37(2),471-482. brunet, a. p., & new, s. (2003). kaizen in japan: an empirical study. international journal of operations & production management, 23(12),1426-1446 chanias, s., myers, m. d., & hess, t. (2019). digital transformation strategy making in predigital organizations: the case of a financial services provider. the journal of strategic information systems, 28(1), 17-33. doi:10.1016/j.jsis.2018.11.003 chirumalla, k. (2021). building digitally-enabled process innovation in the process industries: a dynamic capabilities approach. technovation 105, 102256. correani, a., de massis, a., frattini, f., petruzzelli, a. m., & natalicchio, a. (2020). implementing a digital strategy: learning from the experience of three digital transformation projects. california management review, 62(4), 37-56. doi:10.1177/0008125620934864 creswell, j. w. a, & creswell, j. d. (2018). research design: qualitative, quantitative, and mixed methods approaches (fifth edition. ed.). thousand oaks, california: sage publications, inc. dubé, l., & paré, g. (2003). rigor in information systems positivist case research: current practices, trends, and recommendations. mis quarterly, 27(4), 597-636. ellström, d., holtström, j., berg, e., & josefsson, c. (2022). dynamic capabilities for digital transformation. journal of strategy and management, 15, 272–286. fpt software (2019). being digital transformation consultant, fpt enhances global competitiveness fpt-investor relations. retrieved from https://www.fpt.com.vn/en/ir/featured-news/being-digital-transformation-consultantfpt-enhances-global-competitiveness fpt software (2020). about fpt software. retrieved from https://www.fpt-software.com/about-fpt-software/ gaffley, g., & pelser, t. g. (2021). developing a digital transformation model to enhance the strategy development process for leadership in the south african manufacturing sector. south african journal of business management, 52(1). doi:10.4102/sajbm.v52i1.2357 goodhue, d.l. (1995). understanding user evaluations of information systems. management science, 41, 1827–1844. henriette, e., feki, m., & boughzala, i. (2016). digital transformation challenges. in the proceedings of mcis 2016. http://aisel.aisnet.org/mcis2016/33 hess, t., matt, c., benlian, a., & wiesböck, f. (2016). options for formulating a digital transformation strategy. mis quarterly executive, 15(2). https://aisel.aisnet.org/misqe/vol15/iss2/6 australasian journal of information systems dang-pham et al. 2022, vol 26, selected papers from acis 2020 digital kaizen 26 horlach, b., drews, p., & schirmer, i. (2016). bimodal it: business-it alignment in the age of digital transformation. in v. nissen, d. stelzer, s. straßburger & d. fischer (eds). multikonferenz wirtschaftsinformatik (mkwi), 1417-1428. imai, m., (2007). gemba kaizen. a commonsense, low-cost approach to management in das summa summarum des management. springer, 7-15. ivancic, l., vuksic, v. b., & spremi, m. (2019). mastering the digital transformation process: business practices and lessons learned. technology innovation management review, 9. kane, g. c., phillips, a. n., copulsky, j. r., & andrus, g. r. (2019). the technology fallacy: how people are the real key to digital transformation. cambridge, massachusetts: the mit press. kautz, k., & bjerknes, g., (2015). tales of it consultants: understanding psychological contract maintenance and employment termination. australasian journal of information systems, 19, 71-95. legner, c., eymann, t., hess, t., matt, c., böhmann, t., drews, p., mädche, a., urbach, n., & ahlemann, f., (2017). digitalization: opportunity and challenge for the business and information systems engineering community. business & information systems engineering, 59(4), 301-308. li, l., su, f., zhang, w., & mao, j.-y. (2018). digital transformation by sme entrepreneurs: a capability perspective. information systems journal, 28(6), 1129-1157. doi:10.1111/isj.12153 loonam, j., eaves, s., kumar, v., & parry, g. (2018). towards digital transformation: lessons learned from traditional organizations. strategic change, 27(2), 101-109. doi:10.1002/jsc.2185 macpherson, w. g., lockhart, j. c., kavan, h., & iaquinto, a. l. (2015). kaizen: a japanese philosophy and system for business excellence. journal of business strategy, 36(5), 3-9. https://doi.org/10.1108/jbs-07-2014-0083 maedche, a. (2016). interview with michael nilles on “what makes leaders successful in the age of the digital transformation?”. business & information systems engineering, 58(4), 287-289. doi:10.1007/s12599-016-0437-1 malloch, h. (1997). strategic and hrm aspects of kaizen: a case study. new technology, work and employment, 12(2), 108-122. masaaki, i. (1986). kaizen: the key to japan's competitive success. new york, ltd: mcgrawhill. matt, c., hess, t., & benlian, a. (2015). digital transformation strategies. business & information systems engineering, 57(5), 339-343. meraviglia, l. (2018). technology and counterfeiting in the fashion industry: friends or foes? business horizons, 61(3), 467-475. mergel, i., edelmann, n., & haug, n. (2019). defining digital transformation: results from expert interviews. government information quarterly, 36(4). doi: 10.1016/j.giq.2019.06.002 https://doi.org/10.1108/jbs-07-2014-0083 australasian journal of information systems dang-pham et al. 2022, vol 26, selected papers from acis 2020 digital kaizen 27 mizuno, s. (1988). company-wide total quality control. tokyo, japan: asian productivity organization. newitt, d. (1996). beyond bpr & tqm-managing through processes: is kaizen enough? paper presented at the iee colloquium on beyond tqm and re-engineering-managing through process, 31. osmundsen, k., iden, j., & bygstad, b. (2018). digital transformation: drivers, success factors, and implications. in the proceedings of mcis 2018. https://aisel.aisnet.org/mcis2018/37 perkin, n., & abraham, p., (2017). building the agile business through digital transformation. kogan page publishers. pettigrew, a.m., woodman, r.w., & cameron, k.s. (2001). studying organizational change and development: challenges for future research. the academy of management journal, 44, 697–713. rogers, d. l., (2016). the digital transformation playbook : rethink your business for the digital age. new york: columbia business school publishing. rueckel, d., muehlburger, m., & koch, s. (2020). an updated framework of factors enabling digital transformation. pacific asia journal of the association for information systems, 12, 1-26. doi:10.17705/1pais.12401 schaupensteiner, n., gonzález, s., & borgmann, j. (2021). impact of the digital transformation on the transformation of the workforce. in n. urbach, m. roeglinger, k. kautz, r.a. alias, c. saunders & m. wiener (eds.), digitalization cases vol. 2 (pp. 305-325). cham, switzerland: springer. sidhu, s. (2019). executive interview: opportunities and challenges of digital transformation for vietnam’s businesses. retrieved from https://vietnaminsider.vn/executiveinterview-opportunities-and-challenges-of-digital-transformation-for-vietnamsbusinesses/ singh, j., & singh, h. (2009). kaizen philosophy: a review of literature. iup journal of operations management, 8(2), 51. smalley, a., & kato, i. (2010). toyota kaizen methods: six steps to improvement. portland: portland: productivity press. steininger, d.m., mikalef, p., pateli, a., & ortiz-de-guinea, a. (2022). dynamic capabilities in information systems research: a critical review, synthesis of current knowledge, and recommendations for future research. journal of the association for information systems, 23, 447–490. suárez-barraza, m. f., & lingham, t. (2008). kaizen within kaizen teams: continuous and process improvements in a spanish municipality. asian journal on quality, 9(1), 1-21. teece, d.j. (2007). explicating dynamic capabilities: the nature and microfoundations of (sustainable) enterprise performance. strategic management journal, 28, 1319–1350. teece, d.j. (2018). dynamic capabilities as (workable) management systems theory. journal of management and organization, 24, 359–368. https://www.booktopia.com.au/search.ep?author=nils+urbach https://www.booktopia.com.au/search.ep?author=maximilian+roeglinger https://www.booktopia.com.au/search.ep?author=maximilian+roeglinger https://www.booktopia.com.au/search.ep?author=karlheinz+kautz https://www.booktopia.com.au/search.ep?author=rose+alinda+alias https://www.booktopia.com.au/search.ep?author=carol+saunders australasian journal of information systems dang-pham et al. 2022, vol 26, selected papers from acis 2020 digital kaizen 28 urbach, n., & röglinger, m. (2018). introduction to digitalization cases. how organizations rethink their business for the digital age. in n. urbach & m. röglinger (2018), digitalization cases. how organizations rethink their business for the digital age (pp. 114). cham, switzerland: springer. vial, g. (2019). understanding digital transformation: a review and a research agenda. the journal of strategic information systems, 28(2), 118-144. von leipzig, t., gamp, m., manz, d., schöttle, k., ohlhausen, p., oosthuizen, g., palm, d., & von leipzig, k., (2017). initialising customer-orientated digital transformation in enterprises. procedia manufacturing (8), 517-524. walsham, g. (1995). interpretive case studies in is research: nature and method. european journal of information systems, 4(2), 74-81. warner, k.s.r., & wäger, m. (2019). building dynamic capabilities for digital transformation: an ongoing process of strategic renewal. long range planning, 52, 326–349. wessel, l., baiyere, a., ologeanu-taddei, r., cha, j., & jensen, t. b., (2020). unpacking the difference between digital transformation and it-enabled organizational transformation. journal of association of information systems, 22(1), 102-129. wickens, p. d. (1990). production management: japanese and british approaches. ieee proceedings a (physical science, measurement and instrumentation, management and education), 137(1), 52-54. wittenberg, g. (1994). kaizen-the many ways of getting better. assembly automation, 14(4), 12-17. copyright: © 2022 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v26i0.3851 http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 literature review and theoretical background 2.1 digital transformation 2.2 enablers for successful digital transformation 2.3 dynamic capabilities and digital transformation 2.4 the kaizen philosophy 3 research approach 3.1 research method and data collection 3.2 case setting 4 analysis and findings 4.1 myfpt: a mobile app for people engagement 4.2 akalink: a digital platform for skill inventory management 4.3 myfpt and akalink as digital kaizen-driven digital transformation solutions 4.4 the enablers of digital transformation at fpt software 4.5 the transformation of the company’s business model 5 discussion 5.1 the kaizen framework and digital kaizen 5.2 the dynamic capabilities based digital kaizen process model 5.2.1 sensing and the digital kaizen process model 5.2.2 seizing and the digital kaizen process model 5.2.3 transforming and the digital kaizen process model 5.3 intended target outcomes of the digital kaizen process model 6 future research 7 conclusion paper title (paper title style) australasian journal of information systems chhina et al. 2023, vol 27, selected papers from acis 2021 role of actors in blockchain adoption decisions 1 the role of actors in blockchain adoption decisions – an innovation translation perspective shipra chhina federation university, ballarat, australia s.chhina@federation.edu.au mehmood chadhar federation university, ballarat, australia sally firmin federation university, ballarat, australia arthur tatnall victoria university melbourne, australia abstract an overview of challenges around blockchain adoption has revealed that there is little understanding of the process of blockchain adoption decisions, including who needs to be involved and consulted at various stages. when systems are designed to intentionally protect privacy or to obscure actors, such as blockchain platforms, it can be challenging to identify them and to understand their roles. to do this we used an innovation translation approach from actor-network theory (latour, 2007) which looks at the roles of both human and nonhuman actors. to understand the interactions among the actors involved, multiple rounds of interviews were conducted with blockchain experts, utilising the qualitative delphi method. the findings of this study propose a framework that provides insight into the decision-process by exploring the actors involved and their roles as they implement strategies to adopt blockchain and to convince other actors. the outcomes will assist blockchain practitioners and researchers to have an understanding of pre-adoption decisions and stakeholder involvement. organisations considering implementing blockchain will benefit from this research as they will know where to start, who to engage, and which stakeholders are needed during the decision-making process. keywords: actors, blockchain adoption, innovation translation, actor-network theory (ant), adoption decision. 1 introduction blockchain technology was introduced in 2008 by an unidentified person known by the pseudonym satoshi nakamoto, as the underlying technology behind bitcoin (nakamoto, 2008). since then, this technology has gained widespread popularity because of the secure and trusted platform it provides to store and process transactions. later this technology was also used, not just as an underlying technology behind cryptocurrencies, but also as it attracted many industry participants as a use case for their industries like supply chain, tourism, healthcare, education, or finance (bhaskaret al., 2020; dujak & sajter, 2019; treiblmaier, 2020; varma, 2019). there is hardly any industrial sector that does not see a potential application with blockchain technology. blockchain adoption is a most reviewed aspect of literature in recent years as it is one of the most promising emerging technologies. several blockchain australasian journal of information systems chhina et al. 2023, vol 27, selected papers from acis 2021 role of actors in blockchain adoption decisions 2 adoption models have been proposed, especially from behavioural, attitude and diffusion perspectives to streamline its adoption. despite these promising opportunities and exhaustive research in this technological domain, the practical implementation of blockchain use cases is very limited, indicating that there are still many gaps from a practical perspective to be explored. a brief overview of the challenges around blockchain adoption revealed that regardless of popularity and opportunities, the trust and respect for blockchain is not diffused with the actors involved in its adoption (galati, 2021). actors are the basic human or non-human elements involved in the adoption process of technology (tatnall, 2005). choosing blockchain as a platform also affects decision-making among the network of actors and overall organisational governance (tan et al., 2022). each of these actors involved in the decision may exert a positive or negative influence depending on their opinion about the technology (barnes iii & xiao, 2019). the identification of actors is often challenging, especially when systems are designed to protect privacy or purposefully obscure them, such as in blockchain platforms (tanniru et al., 2021). as the security and increasing value of data demands actions from all involved actors in blockchain (caldarelli et al., 2020), it has become imperative to understand their role in blockchain adoption. our preliminary study (chhina et al., 2021) suggested that there is a need to understand the role of the actors involved in blockchain adoption decisions but did not cover the actornetwork (ant) aspect to understand the role of these actors and their interactions. this study extends this work by the utilisation of the innovation translation approach from ant by identifying the actors and changes in the role of key actors with the change in the stages of (the process of) translation. the use of the innovation translation approach was needed to understand the change of roles in the blockchain adoption decision process. this study is motivated to fill this gap in the literature and practical aspects by identifying and understanding the role of actors in blockchain adoption. to achieve this objective, we aim to answer the following research question: how does the role of actors evolve in the process of a blockchain adoption decision? the term ‘evolve’ here implies the “progressive change” (bowler, 1975) in the adoption decision process as it moves towards a decision. achieving this aim is not only important from a theoretical and a practical perspective but also crucial from a technological perspective, as blockchain platforms are based on mechanisms involving several actors. the benefits of a blockchain-based supply chain can only be achieved if multiple actors adopt the technology (sternberg et al., 2021). findings from saurabh and dey (2021) revealed that the degree of utilisation of blockchain attributes such as disintermediation (reducing the chain of interaction by diminishing intermediaries (nasarre-aznar, 2018)), traceability, price, and trust also influence the actors’ adoption intention decisions. furthermore, the list of actors grows as the blockchain system becomes integrated into socioeconomic infrastructures, including end users (external actors), policy makers, and the broader ecosystem engaged with or building on the blockchain (hofman et al., 2021). hence the need of identifying the actors and understanding the change and evolvement in their role in the blockchain adoption decision process from a technology perspective. the above discussion made us realise the emerging need to develop a framework that explains each actor’s role in the adoption decision process of blockchain adoption. thus, this study australasian journal of information systems chhina et al. 2023, vol 27, selected papers from acis 2021 role of actors in blockchain adoption decisions 3 presents a framework for blockchain adoption decisions using the innovation translation approach informed by actor-network theory (latour, 2007). it is important to consider the socio-economic and cultural context of the adoption situation (kautz, 2021). the innovation translation approach attempts to meet this expectation by revealing the social-technological interactions among the actors involved in a process that cannot be understood by other theories that focus on one type of influence alone (boerboom & ferretti, 2014). the innovative translation approach is based on ant’s translation stage and process, which helps to understand the interactions that led to adoption decisions when many actors are trying to influence the outcome based on their own interests (sarker et al., 2006). to have a greater understanding of the role of actors and how these roles gradually change, and to uncover every niche aspect of the interactions among the actors, we have recruited the key actors to gather data for this study. we selected the key actors because they are those who succeed to become indispensable, and they are the people in the process who are seen as having the solution (tatnall, 2009). this paper is organised as follows – the next section provides a brief review of the literature covering blockchain technology and its adoption, the importance of actors in technology adoption and justification for the use of the innovation translation approach for this study. then we provide a detailed overview of the research method used for this study, followed by a findings and discussion section. contributions from this study are then discussed along with suggested future research directions. 2 literature review 2.1 blockchain technology and its adoption a blockchain platform is designed to establish trust between parties by eliminating intermediaries and enabling them to perform their tasks over a blockchain network, thereby enhancing transparency in the flow of information between parties (viriyasitavat & hoonsopon, 2019). a blockchain can be compared to a linked list, where nodes are linked in a network. blockchain nodes can also be described as entities connected to the blockchain (casino et al., 2019). as each node confirms a transaction entered into the blockchain, it is verified and validated (koteska et al., 2017). the broadcasting of a transaction makes it impossible for it to be altered. full nodes verify the blockchain rules for transactions and group transactions into blocks (casino et al., 2019). in a nutshell, blockchain can be understood as “a technology that enables immutability and integrity of data in which a record of transaction made in a system is maintained across several distributed nodes that are linked in a peer-to-peer network” (viriyasitavat & hoonsopon, 2019). in comparison to applications that are centrally controlled, blockchain technology offers many benefits such as decentralisation, disintermediation, immutability, and scalability. for this article, we reviewed the current literature around the blockchain adoption decision process especially focussing on studies which have considered the importance of actors in adoption decisions. table 1 below summarises some of the key studies from this domain in literature. australasian journal of information systems chhina et al. 2023, vol 27, selected papers from acis 2021 role of actors in blockchain adoption decisions 4 source findings (sternberg et al., 2021) the study highlighted the importance of supply chain actors in adopting blockchain technology and developed a framework for inter-organisational supply chains. (saurabh & dey, 2021) the study identified the factors that influenced wine production supply chain actors to adopt blockchain technology and suggested that these features influenced the actors’ decision to adopt. (rugeviciute & mehrpouya, 2019) the study looked at the role of blockchain technology in financial aid flows and suggested that insufficient engagement of actors is a big concern in blockchain adoption. (barnes iii & xiao, 2019) the study used the technological-organizational-environmental (toe) framework. the study extended the framework by considering a network of actors as part of the environmental ecosystem and suggested that to determine the influencing factors in the adoption of blockchain technology by organisations, one should examine the network of actors that comprise the enterprise blockchain ecosystem. (hofman et al., 2021) this study proposed a framework for opportunities for blockchain innovation in governance. it suggested that the identification of actors is often a practical challenge, especially when systems are designed to preserve their identities. (caldarelli et al., 2020) the study used the utaut (unified theory of acceptance and use of technology) model to understand the factors that push organisational actors to use blockchain. the findings suggest that changes in the perception of organisational actors change their influence on the adoption and use of blockchain technology. (islam et al. 2019) this study attempts to understand the role of actor heterogeneity in blockchain split and focuses on actors in bitcoin split transactions. it also suggested that it is necessary to investigate the actors involved in blockchain ecosystems. table 1. literature review of blockchain adoption studies considering the role of actors the above literature review summarises that researchers have made attempts towards realising the importance of actors in blockchain adoption. it reveals however that only one study islam et al. (2019) has made an attempt to understand the role of actors, but this study is too limited focussing on bitcoin split and it does not consider the actors involved in the organisational adoption decision process of blockchain. yet, many studies have suggested that it is crucial to understand which actors are involved and their adoption decisions, but no study has been found that made an effort to identify and understand the actors’ involvement in the adoption decision process. somehow no study has seen the evolvement of roles, referring to the changes these roles encounter while deciding to adopt blockchain. this provides support and motivation for this study. 2.2 the role of actors in technology adoption decision processes in the adoption process of an inter-organisational technology like blockchain, actors play diverse roles. the adoption of information technology has evolved from a simple adopters’ participation process to one in which various actors constantly interact and influence the process (eze & duan, 2016). there has been extensive research on it adoption, but most of it has focused on factors that affect it adoption by treating it as a static decision-making process. adopting new technology is more of a socioeconomic process involving human and nonhuman actors who influence adoption (venkatesh et al., 2003). the human actors could be part of an organisation like the business team, the r&d (research & development) team, or the finance team or could be external to the organisation like external stakeholders, or consultants. the non-human actors can be the hardware and software like the blockchain technology itself. australasian journal of information systems chhina et al. 2023, vol 27, selected papers from acis 2021 role of actors in blockchain adoption decisions 5 however, a very limited number of researchers have examined the influence of various actors on adoption (cavusoglu et al., 2010). the literature has been criticised for neglecting actors’ roles in innovation diffusion (latour, 1987). rather than viewing innovation diffusion as a phenomenon emanating from a centre, it is argued that innovations are transferred and spread in networks of connected actors (law et al., 1986). these actors influence technology adoption decisions through their interactions with other actors and the social and economic context in which they operate (bhatt & singh, 2020). the interactions and influences keep changing in the project life cycle with the change in the actors’ power (berardi, 2013). these interactions, influenced by actors’ habits and routines, may hinder, or advance an organisation’s adoption of new technology (bhatt & singh, 2020). limited studies have considered the role of actors and their interactions across the stages of the adoption decision process. bhatt and singh (2020) suggested that there is a need for exploratory studies to identify the key actors and their interest in adopting new technology. mirriahi et al. (2012) also suggested that there is a need for future studies to identify and understand the diversity of these actors. organisations adopt emerging technology more quickly if diverse actors convey the right information to key actors, allowing them to make informed decisions and streamline their business processes (eze & chinedu-eze, 2018). peng and mu (2011) also suggested to explore the social network of actors in technology adoption and to identify the actors and their interactions. analysts need to be aware of the diverse actors in the adoption process to cope with constantly emerging technology, and the roles of these individuals (eze & chinedu-eze, 2018). therefore, considering the above discussion and the identified gaps in the literature, this study is motivated to explore the actors and their role in the blockchain adoption decision process from an emerging technology perspective. 2.3 theoretical foundation – actor-network theory to understand the interaction between human actors and human and non-human actors involved in blockchain adoption, we employed an innovation translation approach based on actor-network theory (ant). according to ant, a translation process occurs when different actors in a network align their interests and beliefs about technology with each other (callon & latour, 1981) where innovation translation is “the processes through which an innovation process, by necessity, transforms the social space of the actors in the innovation network” (lyytinen et al., 2016). considering the distributed and decentralised nature of a blockchain, the translation process appears to be quite relevant to its adoption. in the translation process, adoption only occurs when the actors convince others to adopt new technologies, whether directly or indirectly (callon, 1984). ant assesses the role and influence of various actors in adopting emerging technologies like blockchain (allen et al., 2018; eze & duan, 2016). actor-network theory emphasizes the importance of understanding the social and material factors that shape technological adoption. ant suggests that technologies do not exist in isolation but are shaped by the network of actors involved in their development, implementation, and use (esmaili et al., 2022). innovation translation in ant consists of a four stages process to understand the adoption process of an innovation, these are the problematisation, interessement, enrolment, and mobilisation stages. in the context of blockchain, these stages help to gain a deeper understanding of the intricate social and technical aspects involved in the adoption process of blockchain technology. problematisation involves recognizing the issues that can be resolved by adopting blockchain technology. interessement is about convincing stakeholders of the potential value of blockchain australasian journal of information systems chhina et al. 2023, vol 27, selected papers from acis 2021 role of actors in blockchain adoption decisions 6 technology. enrolment entails motivating users to adopt a particular blockchain protocol or consensus mechanism. mobilisation is the process of developing agreements among stakeholders and working collaboratively to overcome challenges affecting the network. therefore, ant helps to identify the various actors involved and to show how their interactions with each other influence the adoption process. we discover new human and nonhuman actors through the analysis of associations, creations, and changes in heterogeneous actor-networks (cecez-kecmanovic et al., 2006). it also helps to understand the role of material factors, such as the technical design of blockchain technology and social factors, such as the regulatory environment in which it is being implemented (allen et al., 2018). thus, applying ant to the study of blockchain technology adoption can provide a nuanced understanding of the complex social and material factors that shape technological adoption. to summarize, despite many theories such diffusion of innovation and technology acceptance models like tam and utaut, available to examine technology adoption, ant has a number of advantages (sarosa, 2012). while most theories consider the role of only the human element in the adoption process, ant r also considers the role of non-human actors in shaping social phenomena. these include the complex technological systems and networks involved in blockchain adoption (chhina, et al., 2021). ant emphasizes the importance of understanding the relationships and interactions between all actors in a network. as stated above this is also relevant for blockchain adoption as it involves a wide range of actors with varying interests, incentives, and power dynamics (islam et al., 2019). we thus argue that ant is most relevant and appropriate to achieve the aim of our study. islam et al. (2019) used ant to understand the role of actor heterogeneity in blockchain splits, and esmaili et al. (2022) conducted a strategic analysis of blockchain technology for security organisations. several studies used ant to investigate the adoption of different technologies, including mobile communication (mcbride, 2003), autonomous vehicles (seuwou et al., 2016), solar machines (penteado et al., 2019), and business-to-business portals (tatnall & burgess, 2002). all these studies support the applicability and validity of ant as the theoretical foundation for this study. 3 research methodology considering the exploratory and contextual nature of the study and the newness of blockchain technology, a qualitative research methodology was chosen. this approach is appropriate for studying blockchain adoption as it can provide an in-depth and contextual understanding of social factors and non-humans that influence its adoption (chhina et al., 2021). blockchain adoption involves a wide range of stakeholders with diverse interests, motivations, and perspectives, which are impossible to capture with any other approach. this research approach has the potential and capability to explore and understand those subjective viewpoints and provide insights into experiences of those involved in the technology adoption process (myers, 2019). this research approach is thus best suited for this study. 3.1 data collection a delphi study the study uses the delphi method to collect qualitative data. the method requires multiple iterations of data collection and analysis to arrive at a consensus, unlike other methods like survey and focus group (hsu & sandford, 2007). since the delphi method is an iterative process that allows experts to revise and refine their opinions based on feedback from other participants, this can lead to more accurate and reliable results (levis et al., 2021). most other data collection methods cannot capture the social, economic, and political contexts to examine australasian journal of information systems chhina et al. 2023, vol 27, selected papers from acis 2021 role of actors in blockchain adoption decisions 7 the adoption of an innovation (levis et al., 2021). the delphi method, however, enables subjective consideration of changes in interconnected contexts. several research studies have used this method for data collection to understand adoption of different technologies including the blockchain (dadkhahet al., 2022; hanafin, 2004; sangal et al., 2022). this implies the appropriateness of the delphi method for this study. there are different variants of the delphi methods that researchers have used according to the needs and goals of their research. for this research, we used a three-round procedure. the first round (round 1), known as the “exploration phase”, is where the topic is fully explored with mostly open-ended questions. this is a kind of pilot study to capture the initial themes and to determine whether the right questions were asked of the right people. a total of eight experts were recruited for round 1, of which five were adopters, and three were consultants. round 1 identified the main actors in the adoption decision process and the initial interactions among them. having analysed round 1, round 2 questions were formulated. round 2 was intended to expand on these findings by exploring more themes, identifying more actors at each stage of translation, and on understanding their interaction on how the roles evolved. as a result of an expert review from other researchers, it was decided to also include the views of experts who had once considered adopting the concept and technology of blockchain but dropped it later. thus, a panel of ten members was recruited for rounds two and three, including five adopters, three consultants, and two non-adopters. this was also identified as one of the key limitations of the earlier study. round 2 was followed by the last phase, known as the “evaluation phase” in which the results of the first rounds are generally verified to reach a group consensus (fletcher & marchildon, 2014). round 2 data analysis had led to the development of a preliminary framework, and in round 3 further questions were developed to evaluate the conclusions and justifications made by the experts. in creating the final framework, the experts kept the elements on which they agreed or were relatively close to an agreement. 3.1.1 selection and composition of the panel it is important to select appropriate delphi participants since this directly impacts the quality of the results produced by the delphi process (hsu & sandford, 2007). although there are no formal criteria for determining a delphi expert’s qualifications (hsu & sandford, 2007), participants must meet certain criteria – having a high level of expertise in the research subject, committing to a number of testing rounds, and communicating with the researcher. participants for this study were selected only if they satisfied the selection criteria by answering the following introductory questions and providing the requested description via email before the interview: 1. what is your total experience, and what is your designation and experience with blockchain technology in your current organisation? 2. describe the role and responsibilities you handle in this position. 3. is your organisation an adopter, consultancy, or a non-adopter (has not adopted after consideration) of blockchain technology? instead of aiming at generalising results, the delphi method provides in-depth insight into a complex issue (powell, 2003). it is thus neither necessary nor possible to calculate sample size according to a standard method within the delphi approach (avella, 2016). rather than offering precise recommendations, delphi seeks to explore minds (jenkins & smith, 1994). our australasian journal of information systems chhina et al. 2023, vol 27, selected papers from acis 2021 role of actors in blockchain adoption decisions 8 study sampled decision makers (managers and above level) within organisations, and blockchain experts as its target population. it employed selective purposive sampling to achieve its objectives and meet delphi technique requirements. since delphi invites input from a purposefully selected sample of experts, this approach was considered appropriate (brady, 2015). study participants were interviewed using semi-structured interviews lasting approximately 30-40 minutes, with various key questions to identify the areas to be explored (britten, 2006). in addition, experts were asked as per above to categorise their organisations as either adopters, consultants, or non-adopters. to avoid biases in the responses, experts who volunteered as consultants and adopters were considered consultants. 3.1.1 reaching consensus among experts consensus between experts is one of the crucial requirements of the delphi method. it is typical for delphi studies to end when there is a minimal difference between experts’ opinions between rounds, even if there has not yet been a strong consensus (okoli & pawlowski, 2004; skulmoski et al., 2007). in most studies, this is achieved in two or three delphi rounds (ågerfalk et al., 2009). delphi studies typically reveal saturation at approximately ten to twelve participants (guest et al., 2006). for this study, the saturation was observed with eight experts in round 1 and ten experts in rounds 2 and 3. to reach saturation among the expert’s responses, three rounds of interviews were conducted (schmidt, 2004). table 2 below provides a detailed overview of the participant profile recruited for our study. pseudonym designation category experience in years round 1 round 2 round 3 interviewee 1 (i1) executive vice president adopter 21    interviewee 2 (i2) founder consultant 20    interviewee 3 (i3) co-founder and chief technology officer (cto) adopter 18    interviewee 4 (i4) chief innovation officer adopter 12    interviewee 5 (i5) co-founder and cto adopter 9    interviewee 6 (i6) enterprise architect and blockchain advisor consultant 30    interviewee 7 (i7) senior software engineer and cto consultant 10    interviewee 8 (i8) lead engineer adopter 8    interviewee 9 (i9) senior consultant nonadopter 6   interviewee 10 (i10) co-founder nonadopter 7   table 2. participant profiles 3.2 data analysis to analyse the data, a thematic analysis approach was used. considering the phenomenon under study, thematic analysis is often the most appropriate method for searching, identifying, australasian journal of information systems chhina et al. 2023, vol 27, selected papers from acis 2021 role of actors in blockchain adoption decisions 9 and reporting emerging themes (braun & clarke, 2006). thematic analysis is suggested as a flexible and useful research tool that helps to provide a rich yet complex interpretation of the data (vaismoradi et al., 2013). thematic analysis is a distinguished qualitative approach that is most useful for identifying common threads that extend across interviews. exploring common themes across heterogeneous participant interviews over multiple rounds was very important. as all interviews contained descriptive text, all were recorded and transcribed into word documents. we used qsr nvivo software for our thematic analysis where as a first step the word files were imported into the software. at the next stage, initial open coding was performed, and then codes were grouped using the emerging themes. before beginning the next round of interviews, each round was analysed and coded. the initial themes identified the key actors, and then further themes evolved around other actors involved in the process. identifying the themes guided the next round of questions and analysis where important themes were confirmed, and new themes that could inform the next round were identified in parallel. in the round 2 interviews, there was also a change identified in the role of key actors. in round 3 of the interviews, each participant had the opportunity to identify areas for additions and most of the participants reached a consensus in their responses, indicating saturation had been reached and no further data collection and analysis were needed. 4 findings and discussion the source data from the interviews underpin the findings and provide a discussion on the evolvement of roles of actors involved in the process as we move from problematisation to mobilisation while translating the adoption decision process of a technology. figure 1 summarizes our findings in a framework describing the actors and their changing roles involved in the blockchain adoption decision process. 4.1 problematisation actors in the proposed framework in the problematisation phase, the key actors are the people who initiate the idea of an innovation (tatnall, 2009). this phase involves the key actors and how they define the problem and how further, different actors are identified (birke & knierim, 2020). here the key actor defines the issues and problems that blockchain technology proposes to solve or realises the problems that can be negotiated with the technological solution (unnithan & tatnall, 2014). it is a process in which one or more key actors as initiators (attempt to) define the nature of the problem and the roles of other actors who appear to be able to contribute to the solution of the problem (tatnall, 2009). in our study, problematisation is understood as the stage where (a) key actor(s) will realise the need for a blockchain solution in their organisation and justify the need for the proposed technology for the business. thus, they initiate the process and identify blockchain technology as a need and/or a solution to their problem and formulate the strategic necessity for a blockchain solution for business. in our study, these key actors identified themselves as such or as a primary actor in the group of key actors. when the participants were asked about the starting point or the origins of identifying the need for blockchain in their organisation, most of them mentioned that someone (usually themselves or the head of the group they were part of) from their organisation formed a group that looked for the emerging technologies and their usage. australasian journal of information systems chhina et al. 2023, vol 27, selected papers from acis 2021 role of actors in blockchain adoption decisions 10 figure 1. proposed framework of blockchain adoption decision actors some also mentioned that they had a dedicated research and development group led by some blockchain expert (usually themselves) to see how this technology could be utilised. for instance, one of the participants, a consultant, from the round 1 interviews stated that they had a group that constantly explored new technologies in the market, and that he was leading that group when they discovered blockchain. “yeah, so we have a group which constantly explores new things new technologies and i lead that group in our organization.” (i2, r1) consultant we further found that many organisations have a set research and development (r&d) team to research new technologies, how these technologies are utilised in other organisations, and what their benefits are. we also found that usually the initiator, whom we call a key actor, is an individual who started the formation of the r&d group or was a part of this team. we also found support for forrest (1991) who argued that the interaction between the r&d team and other teams and individuals leads to the initial knowledge in the innovation-decision problematisation actors key actor: initiator human actors: initiator, r&d team non-human actors: blockchain technology interessement actors key actor: business leadership team human actors: initiator, r&d team business leadership team, consultants, finance, legal, product development teams, other potential internal & external stakeholders (f.ex. developers), non-human actors: blockchain enrolment actors key actor: consultants human actors: all process participants (initiator, r&d team, business leadership team, consultants, finance, legal, product development teams, other potential internal & external stakeholders (f.ex. developers) non-human actors: blockchain technology mobilisation actors key actor: blockchain technology human actors: all process participants (initiator, r&d team, business leadership team, consultants, finance, legal, product development teams, other potential internal & external stakeholders (f.ex. developers) non-human actors: blockchain technology key actor changed to business leadership team to align other teams key actor changed to consultants to align all process participants to support adoption key actor changed to blockchain technology to align all actors through demonstrating blockchain features to support adoption australasian journal of information systems chhina et al. 2023, vol 27, selected papers from acis 2021 role of actors in blockchain adoption decisions 11 process. for instance, one of the participants from round 2 mentioned that they had formal and informal channels of research to look at blockchain technology and its uses. “we did have formal research and as well as informal research, do i need a centre of excellence and that was essentially set up in 2016 to look at this technology, understand this technology.” (i1, r2) adopter bandura (2001) put forward that individuals who observe others who have adopted a particular innovation are favourably inclined to adopt the innovation themselves. in our study’s context, when the key actors or the r&d team tried to look into the work and utilisation of blockchain technology, and its benefits, they were likely to discover the idea of blockchain adoption. in line with straub’s (2009) finding that individuals are capable of learning from other’s experiences more than from their own, one of our participants who was a key initiator in his organisation, stated that it was a combination of their research and discussion with others that enabled them to start the blockchain adoption decision process. “it is a combination of what you learned, and you researched, and the technology being adopted and so we did lot of discussions on some of the leading voices in the industry around this technology” (i1, r1) adopter from the collective responses from all participants, we gathered that at the problematisation phase the idea of adopting blockchain technology is initiated by someone who has basic knowledge or a background about the usage of the technology. in this phase, as organisations encounter blockchain technology as an opportunity for technological advancement, it is the non-human actor – the block chain technology that also influences other actors to move towards an adoption decision. accordingly, one of the participants confirmed that one of the key motivations for considering blockchain adoption came from the expected advantages of blockchain technology for their organisation. “one is the opportunities that blockchain technology provides us, for us to bring industry changes where we can use it like a utility on some of the key things where the technology could benefit the organisation.” (i2, r2) adopter therefore, considering the motivations and identifying the technology as a solution to the problems, the study identified blockchain technology itself as an important non-human actor influencing other actors involved in the adoption decision. in summary, we found the following roles of problematisation actors. the initiator is usually a part of the r&d team or the one to start a group to research the innovation. they are either a part of the r&d team or others who initiated research on blockchain adoption. the key actors at this stage are usually technology-savvy people and/or group who has formed their understanding through some formal or informal research. the participants stated that it is clearly the technical benefits that are understood and taken into account by the actors and the technology is seen as a solution to an existing problem or is identified as a benefit to the organisation, but primarily from a technological perspective. it is also evident that at this stage the key actors’ beliefs are based on their understanding of the technology and its use (eze & chinedu-eze, 2018). thus, at this stage, the key actors, initiators, and/or r&d team look at and explore the adoption of blockchain technology by others, the benefits and success stories, and how they can utilise this technology to solve their problems. australasian journal of information systems chhina et al. 2023, vol 27, selected papers from acis 2021 role of actors in blockchain adoption decisions 12 as kyratsis et al. (2012) argues it requires an organisation’s members to acquire and process new knowledge about an innovation in order to reduce uncertainty around its adoption. in our study this knowledge helped the initiators to map the opportunities associated with blockchain technology for their organisation and for existing problems in the business. considering the important role of blockchain technology itself, the study identifies blockchain technology itself as one crucial non-human actors involved in the translation process that might help other actors towards an adoption decision. thus, thus in the problematisation phase, the human actors are the initiator and mostly an r&d team and the non-human actor is the blockchain technology. the key actor is the human actor who is the initiator usually as part of an r&d team. 4.2 interessement actors this phase focuses on bringing all the actors together to convince and interest them in the proposed blockchain solution. it helps to understand the strategies used by the key actors and the actions performed to gather the interest of other actors in the organisation so that they willingly adopt the roles defined for them (birke & knierim, 2020). it identifies how the key actors establish an interest in the technology (unnithan & tatnall, 2014). according to callon (1984), interessement consist of a series of processes that identity the role bearers that are defined in the problematisation phase and the roles that can be imposed on further actors (tatnall, 2009). in our study, interessement is understood as a stage where the key actor/s try to justify the need for blockchain technology in their (business) organisation and attempt to convince people to make them ‘interested’ in the proposed blockchain solution. as described in the proposed framework (see figure 1), at this stage the identified actors consist primarily of the members the business team, but are not restricted to them, as anyone in the organisation who is involved in the adoption decision processes can act in this stage. when the participants were asked who were those involved in the interactions after the conclusion of problematisation phase, in what we consider the interessement phase, most of them highlighted the business team or the business members responsible for making strategic decisions. we found that the key actors from the problematisation phase in the interessement phase look for other actors who adhere to and support their beliefs and values (eze & chinedueze, 2018). for example, one of the participants stated that their business team and the product team evaluated the idea of blockchain along with the blockchain experts and then tried to develop the best possible use case to start the adoption decision process with. “[we gather members of the] product and business teams’ domain, knowledge experts and technologists to figure out like what are the ideas that make the most sense for us using this technology and then they can perhaps have a scorecard in order to justify which is the best-suited idea to start with” (i3, r3) adopter top management’s role is crucial in the adoption decision process as they are responsible for all strategic, operational, and tactical decisions (khayer et al., 2020). its tight alignment with an interest in the technology is crucial to ensure a company’s long-term viability and profitability when adopting a technology (sarker et al., 2006). bhatti et al. (2021) also emphasise the role of top management support in embracing new technologies to ensure an organisation’s success. the business leadership team in an organisation should act as change enabler by leading other employees to adopt new technologies (garrett jr & neubaum, 2013). australasian journal of information systems chhina et al. 2023, vol 27, selected papers from acis 2021 role of actors in blockchain adoption decisions 13 actors who adopt leadership roles are usually also leaders in adoption decision making processes (seyfried, et al, 2019). however, while top management’s general support in technology adoption has been discussed widely, the specific actors from top management and their role in the adoption decision process has not. our study emphasises members of the business leadership team as key actors to identify the business needs of the technology. we found that as soon as the business team realises the need for the technology, they take over the role of key actors and try to create an alignment of interests with and between other actors. an explicit focus on the interessement phase accentuates the involvement of the business leadership team as the key actors. we also identified that the ownership of the idea of adopting blockchain technology changes with degree of the interest and power that actors hold in convincing other to adopt and further may lead to a successful adoption process. another important group of actors that we found in the interessement phase were consultants. as one of the participants stated consultants make the adopters go through a process of approval on requirements concerning possible blockchain solutions. “consultants have to sort of explain things and document things for them and go through like approval process” (i6, r1) consultant the role of consultants appears to be crucial in management and leader’s choice of block chain technology as it is the consultants who create an alignment between management, leaders and a firm’s interest when recommending an appropriate course of action (leiby, 2018). consultants fill a managerial gap by providing advice regarding the adoption process through transfer of their expert knowledge, sharing implicit and explicit experience and crosspollinating between firms (bessant & rush, 1995). consultants may also challenge and tailor the key actors’ beliefs and understanding of the technology (eze & chinedu-eze, 2018). however, as stated before recent studies on blockchain adoption and the adoption decision processes did not uncover the importance of consultants and their role in adoption decision process. our frameworks highlight their importance as our analysis identified consultants as crucial actors and their significant role in this phase when helping organisations to explore the practical aspects that are involved in implementing a blockchain project. the data also showed, that given their own limited technological knowledge, the key actors tend to agree with the propositions initiated by the consultants. in summary, consultants as actors are crucial in the process of the adoption decision as they support organisational members in this process through real case experience of other organisations. we also identified internal stakeholders who represent the internal departments such as members of finance, legal, or product development teams who may become involved in the adoption decision process. one of the participants described how they implemented a communication strategy to clarify understanding and expectations of the blockchain technology to get positive responses from involved departments. "…i would say that we have had pretty positive response from most departments …. so, for example, when you’re pitching to a department, you need to be very clear about. what is the problem you are solving and what is the end benefit you’re getting rather than just putting dropped in for blockchains?" (i6, r2)adopter similarly, another participant stated that as a blockchain consultant they need interaction with the internal and/or external technical development team and blockchain engineers (see also below) to design effective blockchain solutions. australasian journal of information systems chhina et al. 2023, vol 27, selected papers from acis 2021 role of actors in blockchain adoption decisions 14 “i mean especially the blockchain consultant and the blockchain engineers, because engineers do really tend to do the tech improvement and are highly, highly dependent on the input from the consulting teams.” (i4, r3)consultant among others pólvora et al., (2020) have highlighted the importance of stakeholder engagement and prototyping technical solutions with stakeholders. in these participatory activities, stakeholders can explore current possibilities, as well express, and test visions about the future application opportunities for blockchain. collaboration and coordination can be driven by internal relationships and act as a driver for blockchain adoption (kouhizadeh et al., 2021). our framework of actors involved in blockchain adoption highlights how this interaction among internal stakeholders, internal and external developers and consultants is crucial in making an adoption decision. exemplifying the above arguments one consultant stated that for blockchain technology adoption to be successful, the collaboration of multiple external stakeholders is needed confirming that external stakeholders are also involved in the interactions in the interessement phase. “you need to be really collaborating and co-innovating with the potential competitors, your clients and the other industry partners” (i7, r1) consultant the importance of involving external stakeholders has been introduced in the literature where studies have identified the importance of external actors (bhatti et al., 2021). our study confirms and extends these findings by emphasising the role of consultants and/or external developers as external stakeholder in the blockchain adoption decision process when providing knowledge from an external stakeholder perspective for the process. in summary we found the following roles of interessement actors. there is a change in the role of key actors at this stage from the initiator to the business leadership team. considering the power in convincing other teams for adoption, it became clear that as soon as the business leadership team identifies the business need of the innovation suggested in the problematisation phase, they take on the role of key actors. further, the business leadership team involves other teams as part of the adoption decision process when developing and using prototype solution with the blockchain technology to gain their trust and understanding, and to mould their perspectives towards the technology. in this process the role of the initiator and r&d team changes from being key actors, to become as common human actors. considering the influencing power of the business leadership team, they take over the role of key actors. as soon as the business leadership team is convinced about the idea of blockchain adoption, they implement strategies to convince other actors by involving consultants to, beyond the actors from the problematisation phase, the initiator and/or r&d team, and bring in further internal and external stakeholders, such as the finance and product team and to develop appropriate use cases and cost benefit analysis, and possibly a development team to demonstrate the change in business process. blockchain technology along with its properties remains as the standout non-human actor and there is no change in its role. 4.3 enrolment actors this phase involves the active enrolment of actors after their successful interessement. its analysis provides a deeper understanding of the interactions among the human and nonaustralasian journal of information systems chhina et al. 2023, vol 27, selected papers from acis 2021 role of actors in blockchain adoption decisions 15 human actors and the challenges involved (birke & knierim, 2020). in these interaction situations, there is not a straight-forward process of enrolment. instead, coercion, seduction, or consent is executed and negotiated (grint & woolgar, 1997). enrolment involves more than a one-way process with just one actor imposing their will on another, the other must yield as well (tatnall, 2009). the aim is to achieve consensus among all process participants including external consultants and developers. in round 1 and 2 of our interviews we identified as the actors in this phase all earlier categorised process participants, stakeholders and consultants (see figure 1). to support active enrolment, one participant, a member of the business leadership team, stated that they with the help of consultants developed a score card for other stakeholders and involved departments to decide from a pool of use cases on which one to focus first to support an adoption decision. “so we developed a score card like this with around 20 different factors and the process we followed was indifferently the technology team, the business team and the product team rated the idea and then we were able to evaluate a score on one is this problem is really worthy for [organisation] to solve?” (i1, r1) adopter thus, in the enrolment phase consultants became key actors for adopter organisations. on participant, who was a consultant, described how they invited everyone to vote about the idea of introducing blockchain technology and then identified use cases which had revenue earning potential. “we invited everyone who is a key stakeholder in this process to vote for that and then we decided and the client interest as well, in terms of whether these had revenue earning potential.” (i3, r2) adopter the consultants provided practical views of the potential use cases and supported the projects that organisations intended to implement. considering their knowledge and power to design and showcase the prototype, as well to convince other process participants, we thus qualify consultants as a key actor in the enrolment phase. we summarise the roles of enrolment actors as follows. a change in role is observed among the human actors and the key actors. consultants take over the role of key actors, considering the propositions they make at the interessement stage, and provide an advanced picture of technological, cost and complexity aspects involved in the practicalities of a blockchain adoption project. involving consultants in the interessement phase as ordinary human actor providing explanations and documentation services changed to providing their understanding and practical knowledge of blockchain adoption process in support of building practical problem solutions using blockchain technology. in this context we confirm awa et al. (2016) who stated that voting may occur before an application is accepted and is problem is resolved. alignment of interest of all the major stakeholders is important for establishing any it infrastructure (mcbride, 2003). this, and the multitude of actors, shape how the technology will be accepted and used in future (palas & bunduchi, 2020). thus, in the enrolment phase the common human actors were all the beforementioned process participants such as stated above members of the finance, legal and product team, and other internal and external stakeholders such as members of development teams. this also included members of business leadership teams whose role changed from key actors to more common human actors and the australasian journal of information systems chhina et al. 2023, vol 27, selected papers from acis 2021 role of actors in blockchain adoption decisions 16 initiator and r&d team members who remain human actors as process participants. the initiators became stakeholders to vote on the ideas of the consultants and contributed their consent towards the adoption decision. no change was observed in the role of blockchain technology, and it was still seen as a non-human element that is part of the actor-network. considering the major change that the technology will bring, the blockchain technology (still) is a prominent non-human actor. 4.4 mobilisation actors this is the final phase of the innovation translation approach of ant. its analysis provides insight into whether the roles taken by or assigned to the key actors in the enrolment phase were executed by the actors and how the actor-network of, in our case, blockchain decision making process was maintained (birke & knierim, 2020). mobilisation is focused on finalising the working prototype of the proposed blockchain technology solution, new processes, and the new roles of stakeholders as a foundation for the ultimate blockchain technology adoption decision. mobilisation requires the involvement of representatives of all impacted stakeholder groups as spokesperson to provide consent (or rejection) on the proposed solution and any proposed detours (tatnall, 2009). at this stage the proposed solution locks in if it is accepted by those involved (mcmaster et al., 1997) and might lead to some envisioned benefits. one participant referred to the use cases and blockchain technology that they are utilising for proxy voting in the capital market and that all their products are now generating revenue for them. "we have launched 3 solutions in the last 4 years which are actually actively in production. which means that they are actually revenue generating solutions…" (i2, r1) adopter when explaining the outcome of a prototype that had built to showcase what blockchain is capable of doing, another participant added that their proof of concept has resulted in an increase of 30% in terms of operational efficiency introduced by blockchain technology. "…once we did the experiment and we compared the timelines that it took for one against the other what we realised was there was a 30% efficiency gain in an industry that does 19 trillion dollars a year… …. 30% efficiency gain in a 19 trillion-dollar industry you have fundamentally altered the way this industry works…" (i7, r1) consultant a further participant stated that their expectations from blockchain technology have been met, and blockchain has helped to address the problem that existed before. “i think that the problems that we thought blockchain would help address has been addressed. so obviously blockchain has helped address the problems that existed for those specific areas before blockchain extension.” (i1, r2) adopter thus, we concluded that at the mobilisation stage, blockchain technology acted as the as the key non-human actor with blockchain technology-based prototypes demonstrating the exceptional features required for decentralisation processes and immutability and transparency met as per expectations, while the technology improved the business processes in question in terms of efficiency, time, and effort. these features and expectations may vary with different organisations but are specific to what blockchain technology is capable of. in summary the roles of mobilisation actors were as follows: the key actor in this phase was the blockchain technology as a non-human actor for mobilising this technology for future use in organisations. to successfully adopt technology within an actor-network, human and nonaustralasian journal of information systems chhina et al. 2023, vol 27, selected papers from acis 2021 role of actors in blockchain adoption decisions 17 human actors must align their interests and properties (mcbride, 2003). thus, the technology features and their use were showcased to the process participants and actors involved in the previous phases and aligned to support the blockchain adoption decision. the analysis of this phase uncovered that the successful application demonstrated by the technology itself as an actor led to positive adoption decisions by other actors involved in the adoption decision process. there was another a major change concerning the role of key actor with the blockchain technology becoming the key actor. the technology in the form of prototypes (had the power to) convinced other actors to support adoption decisions. compared to the enrolment phase the consultants changes as they became common process participants. we did not observe other changes in the roles of other remaining participants including the initiator and the r&d team, the members of the business leadership teal as well the members of the finance, product and legal team and other internal and external stakeholders such as members of the development team compared. all these actors became normal human actors who were mobilised by the non-human actor blockchain technology as the key actor to make the blockchain adoption decision. 5 contribution and future research the study makes several contributions from theoretical and practical perspectives. its first, and its primary contribution, is the identification of a multitude of different actors who are involved in the various phases of the blockchain adoption decision process and their varying roles in the decision process. utilising the innovation translation approach, the study provides an innovative approach and framework to understand the blockchain adoption process and offers an explorative view of the translation of actors’ interactions and the evolvement of their roles, in particular the role of the technology itself as it actively supports the adoption decision. our analysis of the problematisation phase identified that a member/s of the r&d team initiate the process as the key actor while blockchain technology acts as the primary non-human actor. as the translation process progresses towards interessement, we found that the business leadership team took over as the key actor as its members held the power to align other actors to the adoption decision. other process participants involved were internal and external stakeholders, members of the finance, product team, and legal team as well as developers and consultants. after the business teams aligned the process participants in the interessement phase, consultants took the role of the key actor in the enrolment phase based on the practical knowledge they have about use cases for organisations’ possible adoption of blockchain. to provide the other process participants with some understanding of the technology, they explained its capabilities to them and involved them further by letting them vote on the practical adoption of the technology. in the final stage of mobilisation, when use case prototypes were presented to the process participants, the blockchain technology non-human actor took over the role of the key actor. showcasing its features, the technology actively contributes to the other actors’ adoption decision. the outcomes of the study provide an indepth understanding of the interactions and change of roles among the actors throughout the process adoption decision. from a practical perspective, the study will help potential adopters to identify the actors involved in adopting an emerging technology like blockchain in their organisations. it will australasian journal of information systems chhina et al. 2023, vol 27, selected papers from acis 2021 role of actors in blockchain adoption decisions 18 also help them to understand the importance of collaboration between internal and external stakeholders, aligning actors’ perceptions of the technology and the degree of influence that each actor holds towards adoption decisions. the study underlines the overall importance of collaboration and interactions between those involved in the adoption decision process. it might thus also benefit current non adopter to revisit their decision potentially based on a lack of understanding of the technology to seek further information. to extend the findings of this study beyond its current limitations, we suggest some future research directions. first, this study followed a qualitative approach. we suggest a future quantitative study to confirm the validity of our findings. second, we suggest an in-depth study to explore the processes, factors and pressures that influence the actions of the actors in the several phases of the adoption decision process. lastly, based on our framework and the innovation translation approach we suggest future research to explore the phases of the adoption decision process that tactors institute to convince and align other actors towards or against the adoption of blockchain technology in more depth and with multiple case studies. 6 conclusion this study set out to answer the research question: how does the role of actors evolve in the process of a blockchain adoption decision? as answer we propose a framework of the relevant actors for blockchain adoption decisions process by using the innovation translation approach from ant. the research investigates the evolvement process of actors roles through the adoption decision process. the framework identifies the actors who are involved in blockchain adoption decision, and determines their roles by exploring their interactions through the phases of translation. our research contributes to the theoretical and practical domain of the blockchain adoption decision process and blockchain adoption and illustrates how the innovation translation approach is productive in examining the role of various actors and their strategies in adoption decisions. as such it provides a foundation for future research in the area of information technology adoption. australasian journal of information systems chhina et al. 2023, vol 27, selected papers from acis 2021 role of actors in blockchain adoption decisions 19 references ågerfalk, p. j., fitzgerald, b., & slaughter, s. a. (2009). introduction to the special issue— flexible and distributed information systems development: state of the art and research challenges. information systems research, 20(3), 317-328. doi: https://doi.org/10.1287/isre.1090.0244 allen, d. w., berg, c., & novak, m. (2018). blockchain: an entangled political economy approach. journal of public finance and public choice, 33(2), 105-125. doi: https://dx.doi.org/10.2139/ssrn.3158805 avella, j. r. (2016). delphi panels: research design, procedures, advantages, and challenges. international journal of doctoral studies, 11(1), 305-321. retrieved from http://www.informingscience.org/publications/3561. awa, h. o., ukoha, o., eke, b. c. (2016). adoption of emerging icts: the role of actors in a social network. cogent business & management, 3(1), 1259879. doi: https://doi.org/10.1080/23311975.2016.1259879 bandura, a. (2001). media psychology. social cognitive theory of mass communication, 3, 265299. doi: https://doi.org/10.1207/s1532785xmep0303_03 barnes iii, b. w., & xiao, b. (2019). organizational adoption of blockchain technology: an ecosystem perspective (2019). digit 2019 proceedings. 9. https://aisel.aisnet.org/digit2019/9 berardi, u. (2013). stakeholders’ influence on the adoption of energy-saving technologies in italian homes. energy policy, 60, 520-530. doi: http://dx.doi.org/10.1016/j.enpol.2013.04.074 bessant, j., & rush, h. (1995). building bridges for innovation: the role of consultants in technology transfer. research policy, 24(1), 97-114. doi: https://doi.org/10.1016/00487333(93)00751-e bhaskar, p., tiwari, c. k., & joshi, a. (2020). blockchain in education management: present and future applications. interactive technology and smart education, 18(1), 1-17. doi:http://dx.doi.org/10.1108/itse-07-2020-0102. bhatt, b., & singh, a. (2020). stakeholders’ role in distribution loss reduction technology adoption in the indian electricity sector: an actor-oriented approach. energy policy, 137, 111064. doi: https://doi.org/10.1016/j.enpol.2019.111064 bhatti, s. h., et al. (2021). internal and external antecedents of open innovation adoption in it organisations: insights from an emerging market. journal of knowledge management, 25(7), 1726-1744. doi: http://dx.doi.org/10.1108/jkm-06-2020-0457 birke, f. m., & knierim, a. (2020). ict for agriculture extension: actor-network theory for understanding the establishment of agricultural knowledge centers in south wollo, ethiopia. information technology for development, 26(3), 591-606. doi: https://doi.org/10.1080/02681102.2020.1727826 boerboom, l., & ferretti, v. (2014). actor-network-theory perspective on a forestry decision support system design. scandinavian journal of forest research, 29(sup1), 84-95. doi: https://doi.org/10.1080/02827581.2014.946960 australasian journal of information systems chhina et al. 2023, vol 27, selected papers from acis 2021 role of actors in blockchain adoption decisions 20 bowler, p. j. (1975). the changing meaning of "evolution". journal of the history of ideas, 36(1), 95-114. doi: https://doi.org/10.2307/2709013 brady, s. r. (2015). utilizing and adapting the delphi method for use in qualitative research. international journal of qualitative methods, 14(5), 1609406915621381. doi: https://doi.org/10.1177/1609406915621381 braun, v., & clarke, v. (2006). using thematic analysis in psychology. qualitative research in psychology, 3(2), 77-101. doi: https://doi.org/10.1191/1478088706qp063oa britten, n. (2006). qualitative interviews. in c. pope & n. mays, qualitative research in health care, (chapter 2, 12-20). hoboken, nj, usa: blackwell. doi: https://doi.org/10.1002/9780470750841.ch2 caldarelli, a., ferri, l., ginesti, g., & spanò, r. (2020). understanding blockchain adoption in italian firms. in r. agrifoglio, r. lamboglia, d. manchini, & f. ricciardi (eds.), digital business transformation: organizing, managing and controlling in the information age (pp. 121-135). springer: cham, switzerland: springer. callon, m. (1984). some elements of a sociology of translation: domestication of the scallops and the fishermen of st brieuc bay. the sociological review, 32(1_suppl), 196-233. doi: https://doi.org/10.1111/j.1467-954x.1984.tb00113.x callon, m., & latour, b. (1981). unscrewing the big leviathan: how actors macro-structure reality and how sociologists help them to do so. in k. knorr-cetina & a. v. cicourel (eds.), advances in social theory and methodology. toward an integration of micro and macrosociologies (pp. 277-303). london, uk: routledge & kegan paul. callon, m., & law, j. (1986). power, action and belief: a new sociology of knowledge? the science studies reader (pp. 196-223). london, uk: routledge. casino, f., dasaklis, t. k., & patsakis, c. (2019). a systematic literature review of blockchainbased applications: current status, classification and open issues. telematics and informatics, 36, 55-81. doi: https://doi.org/10.1016/j.tele.2018.11.006 cavusoglu, h., hu, n., li, y., & ma, d. (2010). information technology diffusion with influentials, imitators, and opponents. journal of management information systems, 27(2), 305-334. doi: https://doi.org/10.2753/mis0742-1222270210 cecez-kecmanovic, d., abrahall, r., & kautz, k. (2006). actor-networks and the production of is success and failure. acis 2006 proceedings, 9. http://aisel.aisnet.org/acis2006/9 chhina, s., chadhar, m., firmin, s., & tatnall, a. (2021). blockchain adoption framework using innovation translation approach-the preliminary study. acis 2021 proceedings, 85. https://aisel.aisnet.org/acis2021/85/ dadkhah, m., rahimnia, f., & filimonau, v. (2022). evaluating the opportunities, challenges and risks of applying the blockchain technology in tourism: a delphi study approach. journal of hospitality and tourism technology, 13(5), 922-954. doi: http://dx.doi.org/10.1108/jhtt-04-2021-0115 dujak, d., & sajter, d. (2019). blockchain applications in supply chain. in a. kawa, &, a. maryniak, (eds.), smart supply network (pp. 21-46). cham, switzerland: springer. doi: https://doi.org/10.1007/978-3-319-91668-2_2 australasian journal of information systems chhina et al. 2023, vol 27, selected papers from acis 2021 role of actors in blockchain adoption decisions 21 eze, s. c., & chinedu-eze, v. c. (2018). strategic roles of actors in emerging information communication technology (eict) adoption in smes: actor-network theory analysis. the bottom line, 31(2), 114-136. doi: http://dx.doi.org/10.1108/bl-09-2017-0029 eze, s. c., & duan, y. (2016, july). the role of actors in the adoption of emerging ict in smes: an actor network theory analysis. in proceedings of the 17th international conference on informatics and semiotics in organisations (iciso) (pp.234-236), august 2016, campinas, brazil. doi: http://dx.doi.org/10.1007/978-3-319-42102-5 fletcher, a. j., & marchildon, g. p. (2014). using the delphi method for qualitative, participatory action research in health leadership. international journal of qualitative methods, 13(1), 1-18. doi: https://doi.org/10.1177/160940691401300101 forrest, j. f. (1991). practitioners’ forum: models of the process technological innovation. technology analysis & strategic management, 3(4), 439-453. doi: https://doi.org/10.1080/09537329108524070 galati, f. (2022). blockchain adoption in supply networks: a social capital perspective. supply chain management: an international journal, 27(7), 17-32. doi: http://dx.doi.org/10.1108/scm-12-2019-0448 garrett jr, r. p., & neubaum, d. o. (2013). top management support and initial strategic assets: a dependency model for internal corporate venture performance. journal of product innovation management, 30(5), 896-915. doi: https://doi.org/10.1111/jpim.12036 grint, k., & woolgar, s. (1997). the machine at work: technology, work and organization. cambridge, uk: polity press. https://books.google.com.au/books?id=7c1faaaaqbaj guest, g., bunce, a., & johnson, l. (2006). how many interviews are enough? an experiment with data saturation and variability. field methods, 18(1), 59-82. doi: https://doi.org/10.1177/1525822x05279903 hanafin, s. (2004). review of literature on the delphi technique. dublin, ireland: national children’s office, 1-51. retrieved from: https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=38d8baf4f555fe5ff23 0dd75eb8483eb9298cfaa. hofman, d., dupont, q., walch, a., & beschastnikh, i. (2021). blockchain governance: de facto (x) or designed? in v.l. lemieux & c. feng (eds.), building decentralized trust: multidisciplinary perspectives on the design of blockchains and distributed ledgers (pp. 2133). cham, switzerland, springer. doi: https://doi.org/10.1007/978-3-030-54414-0_2 hsu, c. c., & sandford, b. a. (2019). the delphi technique: making sense of consensus. practical assessment, research, and evaluation, 12(1), 10. doi: https://doi.org/10.7275/pdz9-th90 jenkins, d. a., & smith, t. e. (1994). applying delphi methodology in family therapy research. contemporary family therapy, 16, 411-430. doi: https://doi.org/10.1007/bf02197902 kautz, k. (2021). editorial for the australasian journal of information systems 2021. australasian journal of information systems, 25. doi: https://doi.org/10.3127/ajis.v25i0.3853 australasian journal of information systems chhina et al. 2023, vol 27, selected papers from acis 2021 role of actors in blockchain adoption decisions 22 khayer, a., talukdar, m. s., bao, y., & hossain, m. n. 2020). cloud computing adoption and its impact on smes’ performance for cloud supported operations: a dual-stage analytical approach. technology in society, 60, 101225. doi: https://doi.org/10.1016/j.techsoc.2019.101225 koteska, b., karafiloski, e., & mishev, a. (2017, september). blockchain implementation quality challenges: a literature review. in z. budimac (ed.), proceedings of the sqamia 2017: 6th workshop of software quality, analysis, monitoring, improvement, and applications, belgrade, serbia (vol. 1938, pp. 8-8). https://ceur-ws.org/vol-1938/paper-kot.pdf kouhizadeh, m., saberi, s., & sarkis, j. (2021). blockchain technology and the sustainable supply chain: theoretically exploring adoption barriers. international journal of production economics, 231, 107831. doi: https://doi.org/10.1016/j.ijpe.2020.107831 kyratsis y., ahmad r., & holmes a. (2021). technology adoption and implementation in organisations: comparative case studies of 12 english nhs trusts. bmj open 2012, 2(2). doi: https://10.1136/bmjopen-2012-000872 latour, b. (1987). science in action: how to follow engineers and scientists through society. cambridge, ma, usa: harvard university press. retrieved from: https://www.academia.edu/download/32543408/99631234-science-inaction-by-b-latour.pdf. latour, b. (2007). reassembling the social: an introduction to actor-network-theory. oxford, uk: oxford university press. leiby, j. (2018). the role of consultants and management prestige in management control system adoption. accounting, organizations and society, 66, 1-13. doi: https://doi.org/10.1016/j.aos.2018.03.003 levis, d., fontana, f., & ughetto, e. (2021). a look into the future of blockchain technology. plos one, 16(11), e0258995. doi: https://doi.org/10.1371/journal.pone.0258995 lyytinen, k., yoo, y., & boland jr, r. j. (2016). digital product innovation within four classes of innovation networks. information systems journal, 26(1), 47-75. doi: https://doi.org/10.1111/isj.12093 mcbride, n. (2003). actor-network theory and the adoption of mobile communications. geography, 88(4), 266-276. http://www.jstor.org/stable/40573881 mcmaster, t., vidgen, r. t., & wastell, d. g. (1997). towards an understanding of technology in transition. two conflicting theories. in k. braa, & e. monteiro (eds.), proceedings of the information systems research in scandinavia, iris20 conference, hanko, norway, august 9-12, 1997. university of oslo, norway mirriahi, n., dawson, s., & hoven, d. (2012). identifying key actors for technology adoption in higher education: a social network approach. in m. brown, m. hartnett & t. stewart (eds.), future challenges, sustainable futures, proceedings ascilite (pp. 664-574). retrieved from: https://www.ascilite.org/conferences/wellington12/2012/images/custom/mirriahi%2c_ negin_-_identifying_key.pdf australasian journal of information systems chhina et al. 2023, vol 27, selected papers from acis 2021 role of actors in blockchain adoption decisions 23 myers, m. d. (2019). qualitative research in business and management. london, uk: sage publications ltd. http://digital.casalini.it/9781526418326 nakamoto, s. (2008). bitcoin: a peer-to-peer electronic cash system. retrieved from: https://assets.pubpub.org/d8wct41f/31611263538139.pdf. nasarre-aznar, s.o (2018). collaborative housing and blockchain. administration, 66(2), 59-82. https://ssrn.com/abstract=3189050. doi: https://doi.org/10.2478/admin-2018-0018 okoli, c., & pawlowski, s. d. (2004). the delphi method as a research tool: an example, design considerations and applications. information & management, 42(1), 15-29. doi: https://doi.org/10.1016/j.im.2003.11.002 palas, m. j. u., & bunduchi, r. (2021). exploring interpretations of blockchain's value in healthcare: a multi-stakeholder approach. information, technology & people, 34(2), 453495. doi: https://doi.org/10.1108/itp-01-2019-0008 peng, g., & mu, j. (2011). technology adoption in online social networks. journal of product innovation management, 28(s1), 133-145. doi: https://doi.org/10.1111/j.15405885.2011.00866.x penteado, t. m., silva do nascimento, a. c., , dávila corrêa, c., ... , amanda cristina nunes pacífico a. m. (2019). among people and artifacts: actor-network theory and the adoption of solar ice machines in the brazilian amazon. energy research & social science, 53, 1-9. doi: https://doi.org/10.1016/j.erss.2019.02.013 pólvora, a., nascimento, s., lourenço, j. s. & scapolo, f. (2020). blockchain for industrial transformations: a forward-looking approach with multi-stakeholder engagement for policy advice. technological forecasting and social change, 157, 120091. doi: https://doi.org/10.1016/j.techfore.2020.120091 powell, c. (2003). the delphi technique: myths and realities. journal of advanced nursing, 41(4), 376-382. doi: https://doi.org/10.1046/j.1365-2648.2003.02537.x qolizadeh, m. h., esmaili, m., ebrahim pour, m., & moradi, m. (2023). strategic analysis of blockchain technology to facilitate the kyc for the social security organization costumers based on actor-network theory. sciences and techniques of information management, 9(1), 279-310. doi: 10.22091/stim.2021.6920.1581 rogers e., m. (1995). diffusion of innovations: modifications of a model for telecommunications. in m.-w. stoetzer & a. mahler (eds.), die diffusion von innovationen in der telekommunikation (the diffusion of innovations in telecommunications) (pp.25-38). berlin, germany: springer. https://link.springer.com/chapter/10.1007/978-3-642-79868-9_2 rugeviciute, a., & mehrpouya, a. (2019). blockchain, a panacea for development accountability? a study of the barriers and enablers for blockchain's adoption by development aid organizations. frontiers in blockchain, 2, 15. doi: https://doi.org/10.3389/fbloc.2019.00015 sangal, s., nigam, a., & bhutani, c. (2022). conceptualizing the role of blockchain in omnichannel healthcare: a delphi study. aslib journal of information management, 74(5), 782-800. doi: https://doi.org/10.1108/ajim-08-2021-0230 australasian journal of information systems chhina et al. 2023, vol 27, selected papers from acis 2021 role of actors in blockchain adoption decisions 24 sarker, s., sarker, s., & sidorova, a. (2006). understanding business process change failure: an actor-network perspective. journal of management information systems, 23(1), 51-86. doi: https://doi.org/10.2753/mis0742-1222230102 sarosa, s. (2012). adoption of social media networks by indonesian sme: a case study. procedia economics and finance, 4, 244-254. doi: https://doi.org/10.1016/s22125671(12)00339-5 saurabh, s., & dey, k. (2021). blockchain technology adoption, architecture, and sustainable agri-food supply chains. journal of cleaner production, 284, 124731. doi: https://doi.org/10.1016/j.jclepro.2020.124731 schmidt, c. (2004). the analysis of semi-structured interviews. in u. flick, e. von kardoff, & i. steinke (eds), a companion to qualitative research (pp. 253-259). hamburg, germany: rowohlt. retrieved from: https://www.academia.edu/download/54606836/__a_companion_to_qualitative_rese arch.pdf#page=268. seuwou, p., banissi, e., ubakanma, g., sharif, m. s. & healy, a. (2016). actor-network theory as a framework to analyse technology acceptance model’s external variables: the case of autonomous vehicles. in h. jahankhani, a. carilie, ..., g. sexton (eds.), global security, safety and sustainability the security challenges of the connected world: 11th international conference, icgs3 2017, london, uk, january 18-20, 2017, (pp.305-320). cham, switzerland: springer. doi: https://doi.org/10.1007/978-3-319-51064-4_24 seyfried, m., ansmann, m., & pohlenz, p. (2019). institutional isomorphism, entrepreneurship and effectiveness: the adoption and implementation of quality management in teaching and learning in germany. tertiary education and management, 25(2), 115-129. doi: https://doi.org/10.1007/s11233-019-09022-3 skulmoski, g.j., hartman, f.t., & krahn, j., (2007). the delphi method for graduate research. journal of information technology education: research, 6(1), pp.1-21. retrieved from: https://www.learntechlib.org/p/111405/. sternberg, h. s., hofmann, e., & roeck, d. (2021). the struggle is real: insights from a supply chain blockchain case. journal of business logistics, 42(1), 71-87. doi: https://doi.org/10.1111/jbl.12240 straub, e. t. (2009). understanding technology adoption: theory and future directions for informal learning. review of educational research, 79(2), 625-649. doi: https://doi.org/10.3102/0034654308325896 tan, e., mahula, s., & crompvoets, j. (2022). blockchain governance in the public sector: a conceptual framework for public management. government information quarterly, 39(1), 101625. doi: https://doi.org/10.1016/j.giq.2021.101625 tanniru, m., niu, j., feng, c., duque, c.g., lu, c., & krishnan, h., (2021). incentives to engage blockchain and ecosystem actors. in v.l lemieux, & c. feng, (eds.), building decentralized trust: multidisciplinary perspectives on the design of blockchains and distributed ledgers (pp. 35-61). cham, switzerland: springer. doi: https://doi.org/10.1007/978-3-030-54414-0_3 australasian journal of information systems chhina et al. 2023, vol 27, selected papers from acis 2021 role of actors in blockchain adoption decisions 25 tatnall, a. (2005). actor-network theory in information systems research. in encyclopedia of information science and technology, first edition (pp. 42-46). hershey, pe, usa: igi global. doi: https://doi.org/10.4018/978-1-59140-553-5.ch009 tatnall, a. (2009). innovation translation as a research approach to theorising information systems implementation. international journal of networking and virtual organisations, 6(1), 64-76. doi: https://doi.org/10.1504/ijnvo.2009.022484 tatnall, a., & burgess, s. (2002). using actor-network theory to research the implementation of a bb portal for regional smes in melbourne, australia. in proceddings of the 15th bled electronic commerce conference 'ereality: constructing the eeconomy’, paper 46, bled, slovenia, university of maribor. retrieved from: http://aisel.aisnet.org/bled2002/46. treiblmaier, h. (2020). blockchain and tourism. in z. xiang, m. fuchs, u. gretzel, & w. hoepken (eds.), handbook of e-tourism (pp. 1-21), cham, switzerland: springer. doi: https://doi.org/10.1007/978-3-030-05324-6_28-1 unnithan, c., & tatnall, a. (2014). actor-network theory (ant) based visualisation of sociotechnical facets of rfid technology translation: an australian hospital scenario. international journal of actor-network theory and technological innovation (ijantti), 6(1), 31-53. doi: https://doi.org/10.4018/ijantti.2014010103 vaismoradi, m., turunen, h., & bondas, t. (2013). content analysis and thematic analysis: implications for conducting a qualitative descriptive study. nursing & health sciences, 15(3), 398-405. doi: https://doi.org/10.1111/nhs.12048 varma, j. r. (2019). blockchain in finance. vikalpa, 44(1), 1–11. doi: https://doi.org/10.1177/0256090919839897 venkatesh, v. morris, m. g., davis, g. b., davis, f. d. (2003). user acceptance of information technology: toward a unified view. mis quarterly, 27(3), 425–478. https://ssrn.com/abstract=3375136. doi: https://doi.org/10.2307/30036540 viriyasitavat, w., & hoonsopon, d. (2019). blockchain characteristics and consensus in modern business processes. journal of industrial information integration, 13, 32-39. doi: https://doi.org/10.1016/j.jii.2018.07.004 australasian journal of information systems chhina et al. 2023, vol 27, selected papers from acis 2021 role of actors in blockchain adoption decisions 26 copyright: © 2023 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v27i0.4489 http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 literature review 2.1 blockchain technology and its adoption 2.2 the role of actors in technology adoption decision processes 2.3 theoretical foundation – actor-network theory 3 research methodology 3.1 data collection a delphi study 3.1.1 selection and composition of the panel 3.1.1 reaching consensus among experts 3.2 data analysis 4 findings and discussion 4.1 problematisation actors 4.2 interessement actors 4.3 enrolment actors 4.4 mobilisation actors 5 contribution and future research 6 conclusion paper title (paper title style) australasian journal of information systems gabryk & naidoo 2024, vol 28, research article sociotechnical perspectives on digital sustainability 1 sociotechnical perspectives of digital technologies in sustainable mining warren gabryk university of pretoria, south africa wgabryk@tuks.co.za rennie naidoo university of pretoria, south africa university of the witwatersrand, south africa abstract this paper adopts an interpretive case study approach to understand the role of digital technologies in addressing seemingly contradictory sustainability goals in mining. the sociotechnical model of information systems was used as a framework to guide the analysis of twenty-five in-depth interviews with globally dispersed digital technology experts working collaboratively at an industry-leading hi-tech mining solutions company. the sociotechnicalled thematic analysis findings highlight the trade-offs experts face in balancing narrow technological imperatives and economic outcomes with broader sustainability goals. the analysis moves beyond the technological and economic to a harmonious perspective of social, human, environmental, and technological interactions. a visual thematic map is presented to aid practitioners in designing and optimally implementing digital technologies to simultaneously address the united nations sustainable development goals while prioritising business sustainability. we conclude by drawing from the proposed sociotechnical perspectives approach for digital sustainability to provide scholars with possible pathways for future responsible information systems research. keywords: case study, digital mining, digital sustainability, sociotechnical, sustainable development goals, thematic analysis. 1 introduction recently, scholars have called for information systems (is) researchers to address the need for society to transition to a more sustainable approach to sociotechnical is (nishant et al., 2020; pan and zhang, 2020). together with these calls, an increasing amount of responsible is researchers’ studies have urged for is to contribute to the practice of sustainability and innovate new approaches for academia, policymakers, and industry-based practitioners to contribute to sustainable development (nishant et al. 2020; khuntia et al. 2018; seidel et al., 2017). digital technologies present both an opportunity and a challenge to the united nations (un) global sustainable development goals (sdgs) (united nations, 2023). this opportunity and challenge highlight the predicament of balancing narrowly focused business objectives against the need for social and environmental sustainability for is leaders and practitioners (carberry et al., 2019; linkov et al., 2018; un, 2023; winter et al., 2011). in recent discussions, responsible is research scholars focusing on broader societal benefits continue to address digital technologies' growing ethical, moral, legal, and environmental challenges (davison et al., 2019; kotlarsky et al., 2023; pan & zhang, 2020; mccarthy et al., 2020). we contribute to this debate by studying an essential contextual space – the role of digitalisation on sustainability in the mining industry (chiasson & davidson, 2005). australasian journal of information systems gabryk & naidoo 2024, vol 28, research article sociotechnical perspectives on digital sustainability 2 digital technologies can support and transform mining activities, from prospecting, exploration, construction, operation, maintenance, expansion, abandonment, and decommissioning to repurposing and remediation (haddaway et al., 2019). recent studies carried out by barnewold and lottermoser (2020) and young and rogers (2019) confirm that various digital technologies currently used in mining practices are revolutionising the industry in several ways. for instance, implementing automation technologies, such as autonomous haul trucks and remote-operated drilling rigs, are being used to enhance operational efficiency while reducing worker safety hazards, data analytics and big data are being used to analyse geological data for better site exploration, predict equipment failures, and optimize operational parameters, the internet of things (iot) is transforming mine operations through real-time monitoring of equipment and environmental conditions, artificial intelligence (ai) and machine learning algorithms are being employed to interpret vast datasets for pattern recognition in mineral exploration and for predictive maintenance, remote operations and control systems are enabling the management of mining equipment and operations from a distance, which is particularly beneficial in hazardous or remote locations, 3d modelling and simulation tools are providing accurate visualizations of mining operations for improved planning and risk management, environmental monitoring technologies are playing a crucial role in assessing the impact of mining activities and ensuring compliance with environmental regulations (barnewold & lottermoser, 2020). additionally, blockchain technology is enhancing supply chain transparency and traceability (young & rogers, 2019). last but not least, the adoption of wearable technology for worker safety is enabling real-time health monitoring and environmental awareness for on-site employees. these technological advancements are optimising mining processes and contributing to a shift towards safer, more environmentally conscious, and sustainable mining practices (young & rogers, 2019). building upon the aforementioned applications of digital technologies in the mining industry, we define digital mining as the comprehensive integration and use of digital technologies to transform mining operations. this definition aligns with young and rogers (2019), who emphasise the transformative impact of digital solutions on traditional mining practices. the concept of digital mining represents a paradigm shift in how mining operations are conducted, marking a move from traditional, labour-intensive methods towards a more technologydriven, automated, and data-centric approach (barnewold & lottermoser, 2020). the shift to digital mining is not merely about the adoption of new technologies. it is also about a fundamental change in mining philosophy that embraces a holistic approach where efficiency, safety, and environmental sustainability are not seen as separate objectives but as interconnected facets of a modern mining operation (aznar-sánchez et al., 2018). while there have been marked improvements in social and environmental sustainability, the mining industry is still plagued by negative environmental impacts such as deforestation, erosion, contamination of water resources, increased noise levels, dust and emissions (haddaway et al., 2019). mining also has a negative social impact on public health, living standards, the human rights of indigenous peoples and local communities, and land conflict (moran et al., 2014). with the rapid adoption of digital technologies throughout the mining value chain, the mining industry increasingly relies on digital technology to support business goals (sganzerla et al., 2016). however, digitalisation can yield positive and negative social and environmental impacts – it can alleviate or exacerbate the abovementioned social and ecological concerns. the concept of sustainability, like development, can guide future research australasian journal of information systems gabryk & naidoo 2024, vol 28, research article sociotechnical perspectives on digital sustainability 3 and offer contributions that can help make a better world (walsham, 2012; qureshi, 2015). we agree with this important, future-oriented conceptualisation of sustainability and the pivotal role of is in addressing and shaping new social and environmental concerns (pan et al., 2022; melville, 2010). according to the world commission on environment and development (wced) (1987, p. 43), sustainability can be defined as “development that meets the needs of the present without compromising the ability of future generations to meet their own needs.” similarly, morelli (2011) defines sustainability as ensuring that resource needs do not negatively impact ecosystems. gladwin et al. (1995) view sustainable development as an inclusive and equitable process to achieve human development goals. meanwhile, hart and milstein (2003) view sustainable enterprises as contributing simultaneously to the triple bottom line (people, planet and profit) that addresses economic, social, and environmental benefits. although is research has promoted sustainable development goals in research streams, such as green is and it for development (davison et al., 2019; roohy gohar & indulska, 2020), existing research perspectives have not explicitly applied a sociotechnical perspective to frame their conceptualisation of digital sustainability (sarker et al., 2019; gholami et al., 2016). scholars have found that is studies tend to focus either on a critique of technology appropriations to achieve the organisation’s narrow instrumental goals or promote the organisation’s broader social responsibility and humanistic goals (kotlarsky et al., 2023; watson et al., 2010). while the is literature tends to treat the social and technological aspects of digital sustainability separately, we agree with those scholars who argue that it is crucial to explicitly consider the interplay of both elements in designing sustainable information systems (petrini & pozzebon, 2009; seidel et al., 2013). following the reasoning of sarker et al. (2019), a sociotechnical perspective of digital sustainability privileges neither technology nor human activities in shaping sustainability practices but seeks to account for the interplay between social and technological systems. therefore, a sociotechnical perspective offers a way of considering the interrelationships and interactions between is and the human and nonhuman actors as part of a larger ecosystem (kotlarsky et al., 2023; watson et al., 2010) rather than overemphasising the it artefact or focusing exclusively on the social responsibility of organisations (cooper & molla, 2017; tim et al., 2021). in doing so, we substitute the anthropocentric and engineering-centric term of ‘optimisation’ that tends to dominate current sociotechnical perspectives with the systems and ecological concept of ‘harmony’ to emphasise the importance of the experiences of humans and their interactions among human communities, the inseparable interrelationship of humans and nature and the respect, regeneration and balance with the natural environment (meadows & randers, 2012; steffen et al., 2015; united nations, 2023). whereas optimisation focuses on efficiency and effectiveness within specific parameters, harmonisation seeks a balanced and integrated approach that considers a wider range of factors, including long-term sustainability and diverse stakeholder needs (steffen et al., 2015; washington, 2018). after all, a so-called optimised digital mining solution risks producing a discordant effect on the overall system. therefore, the core research question guiding our investigation in this paper is: how can an expanded sociotechnical perspective enrich our understanding of the role of digital technologies in sustainable mining? in this paper, we apply a sociotechnical perspective supported by a social constructivist epistemology to examine the plurality and complexity of digital sustainability perspectives in australasian journal of information systems gabryk & naidoo 2024, vol 28, research article sociotechnical perspectives on digital sustainability 4 the mining context. we have done this to provide a better understanding of how digital mining experts understand and experience the role of digital technologies in sustainable mining practice. digital mining technologies appear to offer promising innovations for sustainable mining (gorman & dzombak, 2018). the research objective is to understand to what extent a group of digital mining experts at an industry-leading hi-tech mining solutions company hold contradictory and consistent perspectives about the role of digital technologies in sustainable mining. the key contribution of this paper is a visual thematic map that improves upon the extant understanding of digital sustainability by providing a more holistic analysis of the social and technical challenges facing current digital mining practices. in addition, this paper addresses two weaknesses in the extant literature: first, exploring the role of digital technologies in the context of sustainable mining is scarce and second, there have been calls for is research to conceptualise information systems phenomena as a ‘harmonious’ interaction between the social and technological sub-systems (sarker et al., 2019; seidel et al., 2013). we propose an expanded sociotechnical perspectives framework to guide our analysis about current understandings of the role of digital technologies in sustainable mining. as the world is becoming increasingly more complex and faces enormous social and environmental risks, emphasising economic imperatives alone has become inadequate in evaluating it’s contribution to organisational objectives (chen et al., 2009). porter and kramer’s (2006) assertion that firms need to address the triple bottom line of ‘people, planet, and profit’ also implies that it is pivotal in the firm’s broader value-creation process (hertel, et al., 2013). saldanha et al. (2022) support these points of view and further stress the necessity of firms to balance economic and sustainability imperatives better to improve complementary organisation objectives. therefore, this research aims to explore the various ways experts at a hi-tech mining solutions company firm make sense of the social and technical dimensions of digital technologies for sustainable mining. drawing on the sociotechnical perspective (sarker et al., 2019), we develop diverse themes about employees’ views about the role of digital technologies in sustainable mining. these themes reflect the different perspectives on digital sustainability and hence contribute to the academic discourse on the effects of the new generation digital technologies on broader sustainability goals to a multi-facetted understanding that can guide future studies of digital sustainability, but also help practitioners to better support more sustainable use of these technologies. this study assumes that sensitising concepts from a sociotechnical perspective framework can enhance and enrich the understanding of digital sustainability themes in mining and other sustainable informatics domains. 2 literature review and theoretical background 2.1 related digitalisation and sustainability studies in broad terms, digital sustainability refers to an organisation’s broad strategy for achieving sustainability with technologies (wut et al., 2021). another earlier definition of digital sustainability outside the is literature refers to the specific use of technology to support an organisation's environmental, social, and economic sustainability goals (bradley, 2007). this definition emphasises the appropriate use of technology infrastructure and tools to support digital sustainability practices. moreover, while some definitions of digital sustainability focus on how organisations actively engage in activities that support sustainable development goals australasian journal of information systems gabryk & naidoo 2024, vol 28, research article sociotechnical perspectives on digital sustainability 5 by leveraging technology for generating, managing, and disseminating electronic data (george et al., 2020), other definitions focus on the lifecycle of digital artifacts within their ecosystem (stuermer et al., 2017). in a more recent and comprehensive literature review, kotlarsky et al. (2023) found that there are two primary is research streams on digital sustainability. the first stream concentrates on the intrinsic sustainability of information systems themselves, encompassing the design, development, and lifecycle management of digital technologies to ensure they are environmentally friendly and resource-efficient (cooper & molla, 2017). this includes creating energy-efficient hardware, eco-friendly manufacturing processes, software for energy optimisation, sustainable data centres, and extended product lifecycles with effective end-oflife management. in contrast, the second stream is centred around using digital technologies as tools to achieve broader sustainability goals in various organisational or societal contexts (tim et al., 2021). this involves using digital solutions like smart energy management systems, big data analytics for environmental monitoring, digital supply chain management for resource efficiency, telecommuting technologies to reduce travel emissions, and geographic information systems (gis) for conservation efforts (naumann et al., 2011; watson et al., 2010). the key focus here is on applying digital tools to enhance environmental sustainability across different domains rather than on the sustainability of digital technologies themselves (kotlarsky et al., 2023). in is research, the distinction between green it and green is mirrors these broad delineations within the field of digital sustainability (watson et al., 2010). for instance, green it primarily concentrates on the sustainability of digital technologies themselves, while green is emphasises leveraging technology for broader sustainability goals. moreover, green it is inward-looking, focusing on the sustainability of it components. in contrast, green is is outward-looking, emphasising the role of it in advancing sustainability across a broader spectrum of organisational and societal activities (tim et al., 2021). drawing from the abovementioned review, kotlarsky et al. (2023, p. 938) define digital sustainability as “the development and deployment of digital resources and artifacts toward improving the environment, society, and economic welfare.” while our research resonates with this definition in how we explore the use of digital technologies in the mining sector, it also offers a unique dimension reflective of our specific context. for our study, we define digital sustainability as harmonising the interplay between the social and technological subsystems within the broader concept of integrating technological advancements with social and ecological dynamics to achieve sustainable outcomes. this perspective aligns with the sociotechnical view, which posits that effective and sustainable solutions emerge from the synergistic interaction between technology and the social elements of an organisation (petrini & pozzebon, 2009). this holistic approach recognises that the sustainability of digital initiatives in an organisation is not solely a technical challenge but also a social one (tim et al., 2021). according to binder and wade (2024), despite the transformative potential of digital technologies, the application of digital technologies directed at enhancing environmental sustainability may, in certain instances, lead to improved or, conversely, potentially compromised organisational performance. similarly, our use of the term 'harmonise' as opposed to 'optimise' in conventional sociotechnical perspectives reflects the recent understanding that in complex sociotechnical systems, especially in sectors like mining, australasian journal of information systems gabryk & naidoo 2024, vol 28, research article sociotechnical perspectives on digital sustainability 6 unintended consequences often emerge from attempts at optimisation (watson et al., 2010). unlike an optimisation approach that primarily aims to maximise efficiency and effectiveness within a system, a harmonisation-oriented perspective seeks to integrate and balance the interplay of social and technical elements to achieve cohesive and sustainable outcomes (steffen et al., 2015). this approach recognises that achieving a perfect optimisation of individual components in complex environments may not lead to the overall best outcome. instead, harmonising efforts focus on aligning and synchronising various elements to work cohesively towards shared goals (porter & kramer, 2006). this perspective is particularly critical in the mining industry, where the interconnectedness of environmental, social, and technological factors demands a comprehensive and holistic approach to ensure sustainable and responsible operations (whitmore, 2006). digital sustainability can also refer to organisational activities working towards achieving the un sdgs for future generations by leveraging information technologies and electronic data (united nations, 2023). is scholars have argued that contemporary digital technologies such as artificial intelligence (ai), big data, and distributed ledgers can promote sustainable economic growth and development (nishant et al., 2020; pan & zhang, 2020). a review by schoormann et al. (2021) found that ai contributes to health (sdg 3), education (sdg 4), sustainable farming (sdg 2), energy management (sdg 7), animal protection (sdg 15), working conditions (sdg 8), smart infrastructures and innovation (sdg 9), and inclusive political systems (sdg 10). butler (2011) argued that is can also ensure compliance with regulatory constraints brought about by a sustainability focus. meanwhile, seidel et al. (2013) found that is affordances can enable environmentally sustainable work practices. however, some scholars contend that current theories, concepts and artefacts fall short of addressing is sustainability challenges (baskerville et al., 2016; elliot, 2011; watson et al., 2010). despite the potential of digital technologies to enable sustainable decisions and work practices, they can also increase the gap between the beneficiaries of the digital economy and those it disadvantages – e.g., the victims of massive job losses (bessen, 2019). the term ‘digital divide’ has been used by is scholars to articulate the inherent inequalities in the digital economy. after all, planned is affordances, prone to creating unintended effects, do not necessarily translate into actual or optimal affordances. addressing sustainability challenges requires a transdisciplinary approach to align the deployment of digital technologies with sustainability imperatives (elliot, 2011; malhotra et al., 2013; pan & zhang, 2020; stillman et al., 2020). sustainability initiatives need to consider economic, ecological, legal, political and cultural perspectives. for instance, piotrowicz and cuthbertson (2009) identified the need for is evaluation research to pay attention to sustainability by going beyond economic factors, calling for the inclusion of social and environmental factors. as alluded to, the sustainability concept is complex and multifaceted and reflects the perspectives and interests of multiple stakeholders. traditional management approaches continue to assume that economic sustainability is at odds with social and environmental sustainability despite the obvious interdependence (ghoshal, 2005; porter & kramer, 2011). is scholars have called for improving the current conceptualisation of sustainable is to guide sustainability research (baskerville et al., 2016; malhotra et al., 2013). we contend that a more holistic research framework is needed to explore and demonstrate the interdependent nature of sustainability, contributing to the design and adoption of more responsible and sustainable is practices. australasian journal of information systems gabryk & naidoo 2024, vol 28, research article sociotechnical perspectives on digital sustainability 7 2.2 sociotechnical perspectives of digitalisation and sustainability we conceptualise digital mining technologies as part of a harmonious and sustainable sociotechnical system (ssts). bostrom and heinen (1977a) argue that computer-based information systems are neutral and, instead, overcome the shortcomings of monological perspectives by understanding the interplay between social and technical systems. bostrom and heinen (1977b) leverage a sociotechnical approach to illustrate a more comprehensive view of the benefits of computer-based information systems. however, the arguments made by bostrom and heinen (1977a, 1977b) do not refer to sustainability. starting from nine principles, cherns (1967) puts forward that sociotechnical thinking requires a better understanding of constant interplay in practice to achieve the desired outcome, exemplified in this research as supporting the un’s sdgs. similarly, mumford (2006) emphasises the need to understand humanistic focus goals, how sociotechnical interplay works, and, most significantly, that the means of interplay are as crucial as the desired result. mumford’s (2006) point of how something is done is just as important as the result, which is reinforced by bostrom et al.’s (2009) argument for the need to privilege neither the technical nor social subsystem in isolation. as a result, we are encouraged to move away from a dominant technical-focused or socialfocused view of the it artefact and towards a view of it as a holistic sociotechnical system that promotes a more balanced perspective (alter, 2003; sarker, 2019). while we recognise that the sociotechnical perspective has deep roots, has been applied with subtle variations, and has proven helpful in the mining industry (trist & bamforth, 1951), we chose sarker et al.'s (2019) sociotechnical model because it provides an initial transdisciplinary perspective while maintaining an is distinction (readers are encouraged to read sarker et al. (2019) for a more indepth historical review of the sociotechnical perspectives in is). moreover, by adopting a sociotechnical perspective, this research contributes to the discussion of is as a reference discipline (sarker et al., 2013) while supporting sustainability-linked objectives. the current is conception of sociotechnical systems tends to ignore sustainability, emphasising optimisation, which has traditionally been a core focus of is research (briggs et al., 2010; sarker et al., 2019). given the criticalness of sustainability, we adapt and enhance the sarker et al.’s (2019) sociotechnical model by using a sustainability lens through which we view the social and technological sub-systems that make up an information system to be part of a broader and more inclusive and interdependent ecosystem (avgerou & mcgrath, 2007; seidel et al., 2013). traditionally, the social sub-system is people-oriented and focuses on individuals, their relationships, reward systems and authority structures, whereas the technological sub-system focuses on tasks, processes and technologies for achieving organisational objectives or outcomes (bostrom & heinen, 1977; seidel et al., 2013). instrumental objectives are concerned with achieving economic objectives, whereas humanistic objectives are concerned with enhanced job satisfaction and higher quality of working life of employees (bostrom & heinen, 1977). following our extended ssts framework, the fit between the social and the technological sub-systems determines the sustainability of the broader ecosystem, which requires the harmonisation of both organisational sub-systems in sustainability efforts with broader social and environmental concerns (seidel et al., 2013). we believe that perspectives should shift from instrumental objectives and outcomes towards the sustainability of the larger ecosystem. we also assume that perspectives that simultaneously strive to achieve the payoff of the upside of business sustainability with the upside of social cohesion and environmental sustainability are crucial in building a more sustainable society. australasian journal of information systems gabryk & naidoo 2024, vol 28, research article sociotechnical perspectives on digital sustainability 8 figure 1. multiple perspectives of digital sustainability (adapted from sarker et al. 2019). sarker et al. (2019) make use of six types to characterise prevailing sociotechnical perspectives and their influence on organisational outcomes: a type i perspective is predominantly social and focuses mainly on how human factors explain outcomes in technology-mediated ecosystems. type ii, or the social imperative perspective, considers how social aspects influence the technical component and outcomes. type iii considers how social-technical factors additively deliver outcomes. these perspectives assume that there is no interplay between technical and social components. type iv perspectives consider how the sociotechnical interplay delivers outcomes. type v or technical imperative perspectives assume technology is a significant antecedent to social outcomes. type vi perspectives are predominantly technical and focus on developing or improving a sustainable ecosystem's technical component with little or no consideration of the social component. although sarker et al. (2019) focus on the multiple viewpoints of is researchers, we believe that the sociotechnical lens and these six perspective types are also an appropriate sensitising lens for analysing how organisations and practitioners strive to harmonise the social and the technical sub-systems to achieve both the organisation’s goals and broader sustainability goals. however, a sociotechnical conception of digital sustainability proposes that neither the technological nor social sub-system imperatives should be privileged. instead, the focus should be on the interplay between these two sub-systems. while there are marked advantages to using a sociotechnical perspective, these are not devoid of contextual limitations. employees operating in this context may be less flexible to function beyond the norm expected by their employer due to the dominant economic drive and technological orientation of mining businesses, given the seriousness of the consequences of doing so. furthermore, because this study conducts firm-level analysis, other contextual forces like regulators and governments would limit the social participants' flexibility. australasian journal of information systems gabryk & naidoo 2024, vol 28, research article sociotechnical perspectives on digital sustainability 9 this paper aims to better understand the extent to which participant perspectives, in our case, privilege either a social or technological imperative or whether they consider a harmonious interplay of the sociotechnical within a larger mining ecosystem in delivering broader sustainability outcomes. figure 1 shows the concepts we will use as an initial guiding framework to analyse the key similarities and differences in viewpoints of our social participants regarding the role of digital mining technologies in leveraging sustainability. 3 research methodology this paper forms part of a more extensive case study research focusing on implementing digital mining technologies. in this part of the study, we explore the sociotechnical perspectives on sustainability by globally distributed digital professionals working in the digital mining environment. selecting a single revelatory case study can provide a deeper understanding of sustainability perspectives in is (dubé & paré, 2003; orlikowski & iacono, 2001; seidel et al., 2013). we aimed to build a deeper understanding of sustainability within an industry rapidly adopting digital technologies (sganzerla et al., 2016; young & rogers, 2019). sustainability is a paramount and increasingly prominent focus area within the mining industry (joy, 2004; moran et al., 2014). despite the increasing use of digital technologies to improve the productivity and sustainability of mining operations, research on the sustainability perspectives of digital technologies in the mining context is scarce (kirsch, 2010; tost et al., 2018; whitmore, 2006). we followed the interpretive case study guidelines developed by walsham (2006) and klein and myers (1999) to conduct our study. 3.1 case study site a privately owned hi-tech digital mining solutions company that enables value co-creation in the mining industry serves as the case study setting for this research. digimine, a pseudonym, supports the mining industry by collaboratively creating value through the employment of digital mining solutions such as fleet management, machine guidance, haul-cycle automation, reliability and machine health monitoring, and collision avoidance systems. established by an american holding company, digimine has supported mining operations with digitally enabled equipment ranging from 20 to 1,200 on a single mining site. digimine and its american holding company have assisted 90% of the world's largest mining companies, including underground and open pit mining operations focusing on extracting mineral and coal resources. examples include coal (metallurgical and thermal), diamonds, copper, gold, platinum, iron ore, phosphates and phosphoric acid. being mindful that the digital mining technology industry operates and supports mining operations across the general mining value chain, digimine explicitly provides technology and professional support within the drilling, breaking, loading and hauling sub-activities (vorster, 2001). driven by a mission to improve safety and efficiency in mining, digimine co-creates value with mining operations that, by way of example, improve the loading efficiency of haul trucks, improve bench elevation, avoid equipment collisions, avoid critical haul truck failure, reduce the number of people needed in hazardous working areas and automate the haul cycle processes. while digimine is staffed and collaborates with mine operational staff that are primarily technical-focused professionals (scientists and engineers), there is already acute awareness of digital mining technologies and the role of these technologies in mining sustainability. senior leadership echoes this awareness by advocating that digital technology powers sustainable mining. australasian journal of information systems gabryk & naidoo 2024, vol 28, research article sociotechnical perspectives on digital sustainability 10 3.2 data collection in following the ethical procedures stipulated by the institution governing this research, clearance was obtained from digimine and each social participant individually beforehand to conduct interviews. research data was collected from primary and secondary sources. primary data was sourced from semi-structured interviews that were conducted between june 2020 and november 2021. an approved interview guide (please refer to the appendix for further details) was used to ask social participants to share their perspectives and experiences with digital technologies and their role within the mining environment. when the social participant raised their view of sustainability in the digital technology mining environment, the first author would probe for them to elaborate on these views with descriptively rich accounts of contextual scenarios. as a result of the covid-19 pandemic, these semi-structured interviews were digitally held using ms teams and lasted between thirty and ninety minutes. all interviews were audio-recorded, with the permission of digimine and each participant. the recorded interviews were transcribed using professional software and manually checked to confirm accuracy. primary data was supported using secondary data collected from digimine internal documentation (strategy documents, management reports and production results), business case studies and observations. for example, in a variety of site visits and digital workshops, the first author observed the training of operators to use a machine guidance system for safety and production improvements, engineers conducting operational maintenance of digital and physical enabled digital technologies, engineers optimising mining operations using digital technologies, and operators using heavy asset management solutions. triangulation was achieved using secondary information sources, collecting primary data over different periods, and interviewing diverse digimine staff, partners, and clients. we used triangulation to broaden and deepen our understanding of the social participants’ shared and varied sustainability perspectives (dubé & paré, 2003). an example is comparing social participants' feedback versus observations where social participants attempted to optimise a mining fleet’s production efficiency by analysing the trade-off between social and technical interventions. twenty-five social participants informed this research, comprising seven professional staff (engineers and specialists), six middle managers (supervisors and managers), seven senior managers (programme managers and directors) and five executives (general managers, vice presidents and presidents). at the time of the interview, the social participants were either employed by digimine or collaborating partners and mining clients. all social participants have extensive expertise in the mining industry, specifically with digital technologies. the average professional experience of the social participant is 19 years, with the lowest being eight years and the highest being 30 years. twenty-three social participants are degree graduates, eleven have a master's degree, and one holds a phd. this paper adopts a view of expertise as a collective socio-material practice that, through consistently superior performance, accomplishes repeatable, tangible and successful outcomes (ericsson et al., 2007; nicolini et al., 2017). as a result of the social participants’ collective proficiency in the mining, digital technology industry and grouping in industry-leading companies, their involvement in consistent digital mining results and coupled with their competence in the intersection of the three, this paper refers to the 25 social participants as an expert group of digital mining professionals. australasian journal of information systems gabryk & naidoo 2024, vol 28, research article sociotechnical perspectives on digital sustainability 11 pseudonyms are used to anonymise social participants' identities. at the time of this research, social participants were geographically located in australia, brazil, chile, south africa and the united states of america. when direct quotes are used as examples in the case analysis and the social participants’ primary language is not english, an asterisk (*) has been placed next to the corresponding excerpt. 3.3 data analysis our initial aim during the data collection phase was to gather broad and unfiltered insights about the use and impact of digital technologies in the mining industry without constraining responses to a specific theoretical framework. however, as the study progressed and we moved into the data analysis phase, it became increasingly clear that a sociotechnical perspective was appropriate to fully understand and interpret the findings. this perspective, which considers both the social and technical aspects of digital mining practices, was thus adopted post-interview as a lens through which to analyse and make sense of the data. since the sociotechnical perspective privileges neither the technical nor the social but emphasises the interaction between the two, we consider the digital technologies as well as the social (individuals and collectives that implement, use and are impacted by these technologies within the case context) to be part of our analysis. we started the case analysis by developing a coding template to identify and support the identification of key similarities and differences in the viewpoints shared by the social participants (crabtree & miller, 1992). the coding template contained sensitising coding categories, sub-categories and definitions informed by sarker et al.’s (2019) six types of sociotechnical perspectives (see figure 1). braun and clarke's (2006) guidelines for constructing thematic categories were adopted (figure 2). figure 2. iterative thematic analysis process both authors read the data sets multiple times and worked independently to generate an initial set of thematic categories using an open-coding approach. both authors then worked collaboratively to find, categorise and assign the common themes for the many codes that emerged during this iterative process to the selected coding categories. for example, themes predominantly of a social perspective (individuals, their relationships, reward systems and authority structures) referred to “community upliftment”, “community health”, “universal income”, and “employee safety” as considerations. in addition, lower-level codes that were predominantly technical (tasks, processes and technologies) in perspective were referred to as “artificial intelligence”, “robots”, and “plc code”. we identified the different perspectives constituted by these thematic categories until we were satisfied that the joint analysis adequately reflected the data set. the atlas.ti qualitative data analysis and research software australasian journal of information systems gabryk & naidoo 2024, vol 28, research article sociotechnical perspectives on digital sustainability 12 tool was used for data storage and analysis. we selected the exemplar excerpts to demonstrate the link between the data and the analysis. 4 case analysis and findings the thematic map represents an overarching theme, three themes and seven sub-themes (figure 3). following multiple iterations, we arrived at ‘competing sustainability perspectives in digital mining’ as our overarching theme and ‘privileging business sustainability over human, social and environmental sustainability’, ‘privileging automation over human labour’ and ‘privileging a longer time horizon for sustainability’ as the three main themes. figure 3. visual thematic map evidence of the social participants shared dissimilar and competing perspectives of sustainability in digital mining. although there is consensus on the criticality of sustainable mining practices and the solutions that digital technologies offer, the social participants tend to focus on instrumental considerations favouring the leveraging of digitalisation for the financial and economic sustainability of mining. this perspective overshadowed approaches that leveraged enhancing social cohesion and minimising the environmental impacts to address sustainability challenges. 4.1 privileging business sustainability over human, social and environmental sustainability our ‘privileging business sustainability over human, social and environmental sustainability’ theme was linked to four subthemes: a ‘lack of environmental focus’, ‘lack of social emphasis’, ‘lack of human orientation’ and a strong ‘economic emphasis’. the evidence highlights that privileging the human, social and environmental perspective of sustainable mining using digital technologies is less prevalent among the social participants. although these perspectives are far less evident than business sustainability, there were social participants australasian journal of information systems gabryk & naidoo 2024, vol 28, research article sociotechnical perspectives on digital sustainability 13 that highlighted the value of sustainability for the community, the preservation of the environment, and employee safety. hendrik jordaan, an executive, explains that "anything that drives and improves safety is always a kind of a no brainer," and that: “primary value drivers for our customers right now are obviously safety, as well as you know, cost per ton, of course, always productivity and reliability. but now more and more, also sustainability. so that's become a very large driver in the industry.” while safety and sustainability are being considered core value drivers, nadine de wet, a middle manager, puts forward that generating revenue is still prioritised when compared to human, social and environmental considerations. de wet exemplifies how human, social and environmental perspectives of sustainability, based on their experience in the broader mining industry, have not been prioritised due to the social benefits of sustainability. instead, it was prioritised because of instrumental outcomes that benefitted the organisation, such as sustainable revenue. “so, value for me is, i think, first of all, generating revenue, generating sustainable revenue where you can employ people that contributes to communities, contribute socially. so, i think it’s, first of all, it’s about generating money. second thing is you know to be sort of sustaining, maybe sustaining, you know, communities and supporting communities.”* similarly, to de wet, another middle manager, jacqueline robertson shares that from “a mine’s perspective, you know, it’s about working within a certain cost per ton threshold.” further emphasising the primacy of instrumental outcomes is a rhetorical question posed by sarah graham, a senior manager. “so how do we not just meet the environmental impact requirements, but also how do we sustain the business into the future?” graham’s exemplar viewpoint highlights a link between the social perspective of environmental impact and the sustainability of the business. it is the business sustainability primarily for kevin campbell, an engineering professional, who succinctly states that the role of digital technologies in the mining industry is to create shareholder wealth. “essentially, it comes down to the bottom line. it’s either revenues through increased production, productivity through cost reduction or risk mitigation.” this position of financial primacy or privileging business sustainability first is the most common account across the social participant group, despite an exceptional perspective, as one shared by antoinette van der merwe, a senior manager. “there is no reason for all the technology and advancements that we are making that people cannot go home to their families, and so it is a responsibility on our end.” peter greeff, a senior manager, explains that the more organisations experiment with sustainable and socially responsible technology that reduces carbon emissions, the lower the negative effect on instrumental outcomes such as operational efficiency: “yeah, i think, uh, sustainability is a good one, and you know the mining industry is recognised as a contributor, a significant contributor to co2 emissions [...]. even moving into things like hydrogen fuel cells and how can this be introduced to mining equipment as an additional option. outside of that, it’s you know really looking at major ways of reducing power consumption, how australasian journal of information systems gabryk & naidoo 2024, vol 28, research article sociotechnical perspectives on digital sustainability 14 can we optimise the equipment to reduce unnecessary power consumption and water consumption and potentially introduce other technologies that are most sustainable to the overall mining process as well.” greeff echoes that digital technology is changing the way business is conducted. however, it remains the position that sustainability goals are balanced against the normality of instrumental business requirements such as productivity and efficiency. “i mean, if you look at [...] the sustainability space, right? it’s all driving towards carbon neutrality. so how do we get rid of uh, uh diesel emission [....] so, you know, it’s kind of changing the value proposition from hey, here’s a productive piece of equipment that will do what you’ve always expected, but without the carbon output.” 4.2 privileging automation over human labour our ‘privileging automation over human labour’ theme was linked to two subthemes. these included ‘leveraging automation capabilities’ and ‘displacing human expertise’. when sharing their view of the role of digital technologies in the mining environment, most of the social participants preferred the technical perspective and a resultant instrumental outcome despite the negative influence digital technology may have. mining operations are impacted as a result of digital technologies and can range from influencing the design through to the day-to-day operational running of a mine. nico kleynhans, a senior manager, recounts that digitally enabled technology can be used to better balance seemingly ubiquitous instrumental and social perspectives in the mining environment. “sometimes it’s not directly seen. for example, implementing a collision avoidance system on a site will actually negatively impact your production numbers, but it will definitely increase your safety factor. and so, there’s always this challenge between getting the right balance between production and safety. ultimately, my opinion is that safety should take first priority, and that’s one of the big strives to get that safety where we’ve got no fatalities in a year. and you reduce the injuries as far as possible.” kleynhans continues by saying that “the more injuries you see or fatalities you see, that reflects badly on the companies apart from the monetary value that is lost.” kelly mccormick, an engineering professional, echoes the criticality of balancing the instrumental concern of mining efficiencies with employee safety: “but if we have an automatic process to do that and you saving that person a lot of time and they can then use a time for other things. that’s valuable! and, that yeah, like i said, i think that’s pretty straightforward in terms of that saves you money […]. safety if you can. i mean it’s hard. you can’t obviously put value to someone’s life and well-being. but if you can prevent a death or injury and, in the plant, because you have certain checks in place on your software or in your part of your plc (programmable logic controller) code or whatever, that’s going to stop a catastrophe from happening or stop a fire. stop breaking the line due to high pressure or something before it becomes an issue. then that’s massive, running it has many apart from the regulatory impacts you save the incidents which is of massive value.” safety concerns appear to be addressed because of regulatory requirements that entail stopping production flow if there is an injury on duty or a fatality. in other words, the incentives for safety are still largely instrumental rather than social. mccormick continues to emphasise that the use of software for optimising operations provides valuable instrumental outcomes such as improving the speed of decision making and staff productivity: australasian journal of information systems gabryk & naidoo 2024, vol 28, research article sociotechnical perspectives on digital sustainability 15 “we try and optimise the haul route cycle. we save time, the time it takes for that entire cycle to take place, meaning we get more loads of material going into the plant, and that in turn provides value because there’s a direct monetary link to that. on the other hand, if you can save a person’s time to get information, then you freed up that person’s time to do other things. so that’s another way of providing value.” privileging the sociotechnical perspective assumes that the appropriate balancing of the social and technical priorities in a digital mining environment would support safety, increase productivity and improve accuracy in mining activities was shared by social participants. kleynhans pointedly shares that: “obviously safer operations will gain more value than others due to the investors, being happy with the safety risk. obviously, no company in the world would want to injure or kill its personnel, and that improving safety around site definitely does add value.” in pursuit of a safer mining environment, social participants share accounts where equipment operators are being removed from “the line of fire of the machine,” or being “removed from the field” ultimately to ensure that the operators are “outside of that operating envelope” to where the control and information retrieval from trackless mining equipment (tmm) can be done from a “mobile” or a “smart device.” by “removing people from harm’s way” and effectively using technology to alleviate safety concerns, the ability to operate, maintain or optimise equipment outside of a “live fire” environment is possible. the replacement of human intervention or modification of how human labour is enacted leads to increases in the accuracy of physical mining operations and increased productivity. kleynhans recounts an exemplar experience: “we’ve upgraded some of our old pit viper drill rigs with a new telli-remote system, basically allowing the operator to drill from a control room that’s a few kilometres away from the actual drill rigs. this serves a few different purposes. number one is the safety purpose. you no longer have a person sitting drilling close to a high wall that collapse so you’re moving that portion of risk and secondly, your driller motor in your autonomous drill rigs have set parameters within which they can drill, so you’ve got less wear and tear on your equipment and your overall accuracy of your holes drilled is better.” * the use of operationally remote digital technology and the enhancement of the working environment account for the social and technical interplay that symbiotically results in an increase in humanistic and instrumental outcomes alike. to elaborate, working from an airconditioned office environment without high walls and numerous tmms like shovels, excavators, and haul trucks increases the well-being of the employee and reduces the risk of lost-time injuries (ltis). from a technology perspective, with increased accuracy in holes drilled, the result of the blast will align more closely to the planned design and require less remedial work. despite the strong focus on economic outcomes, there is concern for safety and human well-being. this is evident in the statements of nico kleynhans and kelly mccormick, who stress the importance of balancing productivity with safety. the use of digital technologies for improving safety reflects a concern to harmonise technological and humanistic objectives. an executive, hendrik jordaan, describes that the automation of traditionally labour-intensive activities, such as operating a haul truck or shovel, addresses instrumental and humanistic outcomes of revenue, safety and sustainability, respectively. however, autonomous haulage australasian journal of information systems gabryk & naidoo 2024, vol 28, research article sociotechnical perspectives on digital sustainability 16 systems also replace traditional tmm operators with fewer digital controllers. in other words, it introduces safety by reducing the human factor, as illustrated by jordaan’s following comment: “the autonomous haulage system addresses safety issues by removing people from harm’s way. it addresses the cost per tonne […] consistent and reliable, more productivity time because you don’t have the human factor. […] they (human labour) are not needed then what do you do with the people that are not needed?” a digimine whitepaper-supported press release also emphasised the instrumental benefits of leveraging automation capabilities. the paper explains that with automation playing an integral part in the mining sector and the demand for cost-cutting increasing, the desire to eliminate human involvement at specific operation points remains a crucial growth driver for the global mining industry. zandile twala, an executive, expresses another example of privileging automation over humans: “if you have a lot of automation, then you will make people superfluous […] if we drive with even further with robots and so on, you do not need so many people anymore. […]”* a technology-centred perspective has the potential for both misuse and a positive effect on employment. twala continued to share insights about the unintended consequences of robotic automation on the human aspects. “oh, you have to find the common ground if you have so many people which are not needed any more, you have to take care of these people in a dignified way and then, that’s what it is.” * during both physical and digitally enabled site engagements, tmm operators were observed using a machine guidance system to enhance productivity and safety. it was interesting to observe that the in-person training of operators, which included notes and videos, was supplemented with simulator training in a controlled environment. the digital augmentation of traditional labour, such as driving a truck, can account for significant instrumental value in the mining environment. anthea mthembu, a senior manager, explains that in addition to the “inherent value that we create,” there is value above “fifty million dollars on the table if we make good use of [technology name] across [operations names].” privileging automation over human labour in the digital mining environment should come as no surprise, explains jordaan. “whether you’re looking at hr systems, erp systems, crm systems, they’re all essentially attempting to accomplish what had traditionally been labour-intensive human activities that have been organised in some kind of process and then put a system to it and try to automate it as much as possible to remove the labour-intensive manual work that doesn’t necessarily add a lot of value, right? [….] i mean, it could cover any business process.” 4.3 privileging a longer time horizon for sustainability our ‘privileging a longer time horizon for sustainability’ theme was linked to one subtheme, ‘conflicting temporalities’. in the case organisation, most social participants felt pressured to focus on leveraging digital mining technologies for short-term economic and financial objectives while presenting the overall sustainability goal as a long-term concern. brendon smith, an engineering professional, is acutely aware of the impact and responsibility of sustainability-focused objectives and their impact on society for future generations. “let’s say you destroy mother earth in the process of, you know, the work that you do. then just future generations will be robbed of that value, maybe not short term in terms of the value that australasian journal of information systems gabryk & naidoo 2024, vol 28, research article sociotechnical perspectives on digital sustainability 17 you create, but in the long run it will be very disruptive, right, you know, for, uhm, let me say for humanity.” brendon smith acknowledges the need to balance immediate economic benefits with longterm environmental and social sustainability. this is evident in his emphasis on the responsibility towards future generations and the environment. such perspectives indicate a shift towards a more integrated and harmonious approach to sustainability. while smith’s statement is critically important, the primary consideration or legitimisation focuses on short-term instrumental outcomes. jacqueline robertson, a middle manager, explains this focus. “so, you know, there’s always those discussions on-site, so you know how much do you want to push your cost threshold vs. the benefits, the short-term benefits.” digital mining technology that focuses on optimising the value of “ltis”, “safety”, “really caring”, “carbon neutral by 2050”, and “trying to cut their carbon emissions in half by 2030” is evident. however, it is the “cost per tonne”, “margin”, “production through-put”, “revenue,” and “bottom line” that remain the quintessential legitimiser of digital mining technology. to illustrate, hendrik jordaan, an executive, highlights how the mining industry is beginning to address the safety and sustainability goals of the business while focusing on ensuring that the business is sustainable: “in the mining industry right now, i mean, if we look at… the digitisation analytic space. uh, yeah. primary value drivers for our customers right now are obviously safety as well as, you know, cost per ton, of course. always productivity and reliability but now more and more, also sustainability.” jordaan explains that “because most mining companies trade in commodities markets, they don’t control the price, so all they can really control is their costs and production. so that’s how they manage their business.” traditionally, mining operations have focused on driving down costs and improving efficiency. the increasing importance of social and environmental considerations remains deferential to the dominant, short-term, instrumental perspectives that characterise most mining firms based on institutional pressures (such as the financial market and shareholder expectations). the positive impact of digital technology on sustainability is also viewed as a long-term initiative. while an account of a remote solution indicates a current moment-in-time representation of outcome synergies, a collision avoidance solution could be synergetic but over a longer time horizon. robertson explained her temporal perspectives as follows: “because of the safety element, you are minimising ltis, lost time injuries, and fatalities that would, you know, have long-term consequences. so, i would rather drop productivity by maybe 1 or 2 %, but over the long-term period, and i overachieved my safety-related matrices, and my productivity it’s quite maintained nicely because, you know, you usually have, you know, peaks and drops due to mine stoppage because of a fatality or lost time injury.” paula jarvis, an engineering professional, shares that having a temporal perspective supports effective value creation from digital mining technologies. “we build value in a sustainable and more long-term type of approach; otherwise, going to be just a mathematical exercise.” * australasian journal of information systems gabryk & naidoo 2024, vol 28, research article sociotechnical perspectives on digital sustainability 18 antoinette van der merwe, a senior manager, reinforces this temporal perspective and the pressures of focusing on short-term objectives: “it’s just very hard to make the sustainability and the long-term value projections if you’re only trying to optimise one part of the process at a time.” peter greeff, a senior manager, is acutely aware of the financial ramifications of the sustainability of the environment on the business. he explains that over a longer period of time, an environmental value perspective can overcome the perceived negative short-term financial value impact. “i think it’s something that is being looked at closely and drive a major change in the industry like this. it does carry financial ramifications of sustainability, and i think the key to this is around the broader adoption of this type of new capability and technology. […] it enables us to move forward more effectively and more efficiently and reducing that overall financial impact because we all recognise that this is a critical goal that we need to stick to and go after.” the acknowledgement of the pressure to focus on short-term objectives, as discussed by antoinette van der merwe and some of our other participants above, while recognising the importance of long-term sustainability, also reflects an ongoing effort to find a balance between immediate and future needs, aligning with a more harmonised approach. 5 discussion as the convergence of emerging technologies advances, the mining industry has the opportunity to transition away from traditional methods to an increasingly digital future. the is discipline has the opportunity to positively influence sustainable business practices that leverage promising and powerful digital technologies by using a sociotechnical lens (sarker et al., 2019; seidel et al., 2013; stahl, 2012). this revelatory case study research set out to understand the sustainability perspectives held by digital mining professionals in the mining environment. this study draws on prior sociotechnical research in is to develop a thematic map to visualise sustainability perspectives in digital mining. by applying a theoretical thematic analysis guided by sociotechnical theory and concepts (sarker et al., 2019), we could conceptualise sustainability perspectives in digital mining by an overarching theme highlighting competing sustainability perspectives. these competing perspectives were linked to themes that show how practitioners were ‘privileging business sustainability over human, social and environmental sustainability’, ‘privileging automation over human labour’ and ‘privileging a longer time horizon for sustainability’ as the three main themes (figure 3). this study extends upon prior sociotechnical research in is by addressing the challenges more salient to sustainability in digital mining (bostrom & heinen 1977; seidel et al. 2013). findings from this study reveal the multidimensional and multifaceted nature of sustainability perspectives in digital mining. the similarity of our results to those reported in recent is sustainability research (nishant et al., 2020; pan & zhang, 2020) suggests that the broader challenges and complexities in digital sustainability practices are transferable to other domains (sarker et al., 2019). although social participants in the case were not driven purely by economic motives, fewer social participants shared perspectives of responsible and sustainable approaches to mining. nevertheless, there are significant subthemes and growing perspectives among the participants that reflect a movement towards harmonising technology, human well-being, and australasian journal of information systems gabryk & naidoo 2024, vol 28, research article sociotechnical perspectives on digital sustainability 19 environmental sustainability (zhang et al, 2011). this indicates an emerging recognition of the need to integrate and balance diverse objectives for sustainable and responsible mining practices. our thematic map also suggests that those who shared this awareness were struggling with juggling seemingly antagonistic and contradictory social, environmental, financial and economic goals (haddaway et al., 2019). our economic emphasis subtheme indicates that digital technology is being implemented to support sustainable business models in mining. our social participants emphasised using digital technology to support mining efficiency needs and priorities risks neglecting overall sustainability objectives. in contrast, socially focused business models that overfocus on social cohesion and multistakeholder expectations risk neglecting business sustainability goals. first, from a social perspective, social participants articulated humanistic outcomes such as “zero harm”, “community health”, “sustaining the environment”, “helping people out”, and that “safety” is “first”, “number one” or “top priority”. second, from a temporal perspective, social participants generally tend to supplement the instrumental objectives of “revenue”, “costs”, and “cash flow” together with humanistic goals such as ‘safety” to set attainable sustainability objectives in the short term. third, when social participants tended to privilege the social, they made statements such as the “emotional” aspects of mining and “collaboration” with stakeholders. when social participants privileged the technological perspective, statements including terms like “plc code”, “mining analytics” and “machine intervention” were emphasized. finally, when the social participants espoused more holistic sociotechnical perspectives, the pressures of short-term instrumental outcomes tended to foreground mining productivity, cost reduction and profitability goals while understating social issues and environmental concerns. overall, these perspectives on the role of digital mining technologies appear to suggest a weak rather than a strong approach to sustainability in digital mining (young and rogers, 2019). nevertheless, digital technology was found to support mainly sdg 9 (industry, innovation, and infrastructure), sdg 8 (decent work and economic growth), and sdg 3 (good health and well-being) (united nations, 2023). although the study has demonstrated the value of a sociotechnical approach in analysing the digital sustainability perspectives of practitioners in mining, more work needs to be done to understand how is capabilities can be leveraged to support the complex problem of balancing seemingly opposing sustainability and economic goals. this revelatory case paves the way for future research and analyses of digital sustainability in other contexts. 6 implications 6.1 theoretical implications this study contributes to theory in four ways. first, our thematic map provides a comprehensive visualised structure of the competing sustainability perspectives in digital mining, highlighting the interplay and conflicting priorities among critical environmental, social, and technical dimensions (leidner et al., 2022). the competing sustainability perspectives we observed among our participants also suggest a complex relationship between digital technology implementation and sustainability goals in mining. our findings are similar to prior is research that suggests that digital technologies can provide several affordances that can be leveraged to improve sustainability in contexts such as mining (seidel et al., 2013). however, our findings are similar to prior studies that argue that while much is known about how is affordances can enable digital sustainability, the role of is in impeding digital sustainability needs further exploration (binder & wade, 2024). the notion of australasian journal of information systems gabryk & naidoo 2024, vol 28, research article sociotechnical perspectives on digital sustainability 20 competing perspectives emphasises the inherent complexity in sociotechnical systems, particularly in the mining industry, suggesting the need for theorising that can simultaneously address the technical aspects of digital technology and the broader socio-economic and environmental implications of its implementation (butler, 2011). more specifically, our findings suggest that future research should work towards explicitly situating environmental considerations within the is sociotechnical framework (petrini & pozzebon, 2009). we also call for an expanded version of the is sociotechnical lens that can form the basis for a digital sustainability framework that sensitises is researchers to environmental issues in future research. second, our analysis extends the discourse on information systems (is) evaluation, revealing the need for a shift in theorising, where sustainability principles are given equal, if not more, weighting than tangible short-term financial benefits (hertel, et al., 2013). despite the possibilities presented by digital technologies to provide a more balanced and holistic approach to sustainability in practice, the current sustainability beliefs and actions among many of the digital mining experts in our case were being impeded by an is evaluation mindset that emphasises financial factors and a short-term orientation over sustainability principles. we refer to this paradoxical challenge as conflicting temporalities. industry professionals in the mining sector have traditionally prioritised economic gains and technical efficiency, often at the expense of social and environmental considerations. similar to binder and wade (2024), we confirm the difficulties practitioners face in shifting their mindset to incorporate a more balanced approach that values social and environmental sustainability alongside economic and technical objectives. by highlighting this, we provide preliminary insights explaining the difficulty in aligning short-term inducing is evaluation approaches with the broader concept of corporate social responsibility (csr) and sustainable development goals (sdgs) to enable the shift towards more sustainable business practices (porter & kramer, 2011). furthermore, our concept of conflicting temporalities shows that participants face tensions between short-term business objectives and long-term sustainability commitments and struggle with the complex trade-offs involved in these decision-making processes. further research is needed to identify and explain these tensions in is evaluation research (hertel, et al., 2013). moreover, research is needed to understand how experts can apply is evaluation frameworks in ways that are more mindful and attentive to the multidimensional nature of digital sustainability initiatives (piotrowicz & cuthbertson, 2009; swanson & ramiller, 2004). third, apart from implications for is evaluation research, our conception of conflicting temporalities also has implications for design science research (zhang et al, 2011, baskerville et al., 2016). some scholars have suggested that is researchers become ‘solution scientists’ by engaging in research that focuses on developing and implementing strategies that prevent environmental harm and actively contribute to ecological well-being (watson et al., 2010; chen et al., 2011). our findings suggest that design science research emphasis on building digital solutions speedily and rapid value realisation are significant constraints for sustainable is designs. understanding conflicting temporalities highlights the complexity of designs that favour the short-term oriented financial and economic goals over designs sensitive to the interaction of short-term financial goals with environmental, social and human dimensions with their longer-term orientation. this involves rethinking the criteria for success and effectiveness in design science research to include long-term environmental and social impacts (zampou et al, 2022). for instance, by applying chern’s (1976) design principles, we can simultaneously enhance sustainability affordances on a broad scale and achieve short-term australasian journal of information systems gabryk & naidoo 2024, vol 28, research article sociotechnical perspectives on digital sustainability 21 financial results. this implies advocating for designs that are not only technically efficient but also socially and environmentally effective, ensuring that digital technology aligns with human and environmental needs (naumann et al., 2011). future research should explore whether design principles that emphasise harmony by operationalising related but lower-level concepts such as compatibility, collaboration, autonomy, adaptability, and participation can be applied in the context of digital mining to create a more balanced and integrated approach. pertinent design approaches, such as soft systems thinking or participatory design, can be leveraged in design science research to theorise how sustainable solutions are designed and implemented with a deeper understanding of the environmental impacts, ecological balance, and conservation needs in digital mining and similar contexts. the findings in our digital mining case study also highlight the necessity for theories in is design to account for different time horizons. this implies a shift from designs that are predominantly short-term and financially driven to those that also consider longer-term environmental and social impacts (watson et al., 2010). this also implies the need to reevaluate and expand existing design science research frameworks in is to incorporate a dual focus on immediate economic gains and long-term sustainability. ultimately, this involves integrating considerations of the temporal dimension into the core theoretical constructs of is design science research. fourth, our study not only adapts but also builds upon the framework of sarker et al. (2019), tailoring it to cater for the distinctive challenges and dynamics of the mining industry within the context of digital sustainability. we integrate additional dimensions that critically assess the complex interaction between technological developments and the environmental and social factors inherent to mining operations. for instance, our refined concept of harmonisation in sociotechnical systems, moving beyond mere optimisation, emphasises the essential integration and balance between technical, environmental, and social elements while prioritising long-term ecological sustainability, inclusivity, and adaptability (kotlarsky et al., 2023; steffen et al., 2015). this adaptation enables is researchers to explore in depth how digital technologies influence not only operational efficiency but also worker safety, job security, environmental stewardship, and community engagement (bessen, 2019; gholami et al., 2016). by adopting this environmentally-focused sociotechnical approach, is researchers can enhance their comprehension of long-term ecological sustainability and resilience in a rapidly evolving digital context (watson et. al., 2010). future research should explore the process of attaining and maintaining an environmentally-aligned harmonisation between the social and technical facets of digital mining, understanding the dynamics of this balance and its impact on overall operational and ecological success. furthermore, future studies could investigate instances of environmental disharmony in digital mining, examining the causes, consequences, and strategies for environmental mitigation or restoration, thereby offering insights into transitions from ecological imbalance to balance and highlighting the challenges, adaptive strategies, and decision-making involved in these critical transitions. notwithstanding the above, we believe that the central finding that sustainability processes in digital mining generate competing perspectives among experts can inform future theorising in is sustainability studies (elliot, 2011; watson et al., 2010). in particular, we believe that the notions of competing perspectives and conflicting temporalities, identified in the digital mining case may also broaden our understanding of other concepts noted in is sustainability research, such as safety, environmental governance, green innovations, and sustainable design (baskerville et al., 2016; butler, 2011). given our findings, we contend that under conditions of competing sustainability perspectives, there is a risk that organisations will overfocus on australasian journal of information systems gabryk & naidoo 2024, vol 28, research article sociotechnical perspectives on digital sustainability 22 technical perspectives and short-term economic outcomes. therefore, the manner in which sustainability discourses circulate in organisations, as well as the consequences of conflicting temporalities, need to be closely examined in future research. 6.2 practical implications our thematic map provides an understanding of sustainable digital mining as a sociotechnical achievement, underscoring the crucial environmental, social, and technical perspectives that define the sustainability challenges and prospects in digital mining (watson et al., 2010). it accentuates the imperative of foregrounding environmental considerations alongside the existing focus on social and technical aspects, enabling practitioners to holistically enhance both the instrumental objectives and the environmental and humanistic goals of digital mining. our thematic map can also guide practitioners in developing sustainable competitive strategies with digital technologies that simultaneously address financial and economic goals and broader corporate social responsibility goals (watson et al., 2010; ghoshal, 2005). first, our thematic map suggests that to address sustainability issues more harmoniously and holistically, practitioners must be mindful of overcoming the different organisational, sociological and psychological barriers that may impede progress (swanson & ramiller, 2004). for example, to address social cohesion and environmental sustainability goals more effectively, practitioners should minimise major differences in perspectives and practices by simultaneously addressing the upsides of shareholder value and a viable form of sustainable stakeholder value (saldanha et al., 2022; porter & kramer, 2011). second, given the conflicting temporalities experienced by the social participants in our study about the goals and priorities in implementing digital technologies in mining, managers and leaders should engage more closely with their employees to develop a shared understanding of how digital technologies can contribute to sustainability goals (leidner et al., 2022). furthermore, for organisations to successfully appropriate digital mining technologies to meet their sustainability goals, practitioners should establish a more harmonious perspective of instrumental and humanistic objectives (washington, 2018). although both optimisation and harmonisation are valuable concepts and can be complementary in the mining industry and other similar organisational contexts, the success of digital sustainability in mining will depend on more than just proficient professionals in digital literacy, data management, and analytics that utilise digital technologies for enhanced efficiency and gains (young & rogers, 2019), but also on those who can design sustainable solutions that consider long-term impacts on the environment, society, and the economy (kotlarsky et al., 2023). we recommend new educational curricula for mining engineering and it professionals incorporating this broader digital design skill set that goes beyond energy informatics to sustainability informatics (watson et al., 2010; naumann et al., 2011). moreover, to change attitudes and sustainable business behaviour, practitioners should apply an appropriate mix and synthesis of sustainability performance metrics that go beyond technical and instrumental dimensions. as a start, an environmentally-focused sociotechnical perspective approach can be used by practitioners to understand and eliminate major differences and conflicting perspectives about goals, expectations and priorities that can compromise the needs of future generations. finally, to leverage information systems in achieving sustainability goals, practitioners should strive toward building more consistent mutual understandings and perspectives about the role of digital technologies in sustainability transitions (leidner et al., 2022). australasian journal of information systems gabryk & naidoo 2024, vol 28, research article sociotechnical perspectives on digital sustainability 23 7 limitations and future research the paper proposes a sociotechnical model of information systems to understand the role that digital mining technologies play in addressing seemingly conflicting sustainability goals in mining. our study was exploratory and limited to the experiences of a group of digital mining experts at a global digital mining solutions provider. the case shows an overfocus on technology imperatives to achieve short-term business sustainability goals. our findings suggest that establishing appropriate designs for digital sustainability transformations can harmonise the social and technical sub-systems. the unique contextual characteristics of the organisation and the experiences of the experts shaped our insights about sustainability in digital mining. the study was based on the perspective of one community, digital mining experts, who actively shape perspectives of digital sustainability in the mining industry. future research could extend this work to understand perspectives held by other key social actors such as local community members, employees, governments and regulators. in this case, most social participants recognised that autonomous vehicles increase safety and productivity but were not cognisant of how a reduction in low-skill jobs offsets these outcomes. future research on digital sustainability could investigate the unintended consequences of sociotechnical solutions on job losses. while this paper introduces a sociotechnical perspective of how digital technologies contribute to sustainable efforts in a mining context, action research or design science research using design thinking approaches could investigate how a mining firm could apply design principles that harmonise the social and technological sub-systems to improve sustainability in the short term. studies of sustainable is using an environmentally-focused sociotechnical lens can provide practitioners with new insights into managing seemingly intractable priorities. we hope our conception of sustainable information systems in digital mining will enable practitioners to benefit shareholders and stakeholders significantly. for those who study sustainability in organisations, we call for assessing the importance of the interplay between digital technology and human activities and advancing environmental stewardship in improving overall sustainability to be a greater part of their agenda. while we continue to work toward a better understanding of how digital technologies can support sustainable practice, we hope responsible information systems researchers will contribute to future is sustainability research and a more sustainable world that will benefit current and future generations. references alter, s. (2003). 18 reasons why it-reliant work systems should replace the it artifact as the core subject matter of the is field. communications of the association for information systems, 12(1), 366–395. https://doi.org/10.17705/1cais.01223 avgerou, c., & mcgrath, k. (2007). power, rationality, and the art of living through sociotechnical change. mis quarterly, 33(2), 295–315. https://doi.org/10.2307/25148792 aznar-sánchez, j. a., garcía-gómez, j. j., velasco-muñoz, j. f., & carretero-gómez, a. (2018). mining waste and its sustainable management: advances in worldwide research. minerals, 8(7), 1–27. https://doi.org/10.3390/min8070284 bai, c., & sarkis, j. (2013). green information technology strategic justification and evaluation. information systems frontiers, 15, 831–847. https://doi.org/10.1007/s10796-013-9425-x australasian journal of information systems gabryk & naidoo 2024, vol 28, research article sociotechnical perspectives on digital sustainability 24 barnewold, l., & lottermoser, b. g. (2020). identification of digital technologies and digitalisation trends in the mining industry. international journal of mining science and technology, 30(6), 747–757. https://doi.org/10.1016/j.ijmst.2020.07.003 baskerville, r., pries-heje, j., & recker, j. (2016). principles for re-designing information systems for environmental sustainability. in ifip world information technology forum, (481), springer, cham, 14–25. https://doi.org/10.1007/978-3-319-44447-5_2 bessen, j. (2019). automation and jobs: when technology boosts employment. economic policy, 34(100), 589–626. https://doi.org/10.1093/epolic/eiaa001 binder, j., & wade, m. (2023). digital sustainability for a better future. stanford social innovation review, 22(1), 52–60. https://doi.org/10.48558/c6f4-s443 bostrom, r. p., gupta, s., & thomas, d. (2009). a meta-theory for understanding information systems within sociotechnical systems. journal of management information systems, 26(1), 17–48. https://doi.org/10.2753/mis0742-1222260102 bostrom, r. p., & heinen, j. s. (1977a). mis problems and failures: a socio-technical perspective. part i: the causes. mis quarterly, 1(3), 17–32. https://doi.org/10.2307/248710 bostrom, r. p., & heinen, j. s. (1977b). mis problems and failures: a sociotechnical perspective. part ii: the application of sociotechnical theory. mis quarterly, 1(4), 11–28. https://doi.org/10.2307/249019 bradley, k. (2007). defining digital sustainability. library trends, 56(1), 148–163. https://doi.org/10.1353/lib.2007.0044 braun, v., & clarke, v. (2006). using thematic analysis in psychology. qualitative research in psychology, 3(2), 77–101. https://doi.org/10.1191/1478088706qp063oa briggs, r. o., nunamaker, j. f., & sprague, r. h. (2010). social aspects of sociotechnical systems. journal of management information systems, 27(1), 13–16. https://doi.org/10.2753/mis0742-1222270101 butler, t. (2011). compliance with institutional imperatives on environmental sustainability: building theory on the role of green is. the journal of strategic information systems, 20(1), 6–26. https://doi.org/10.1016/j.jsis.2010.09.006 carberry, e. j., bharati, p., levy, d. l., & chaudhury, a. (2019). social movements as catalysts for corporate social innovation: environmental activism and the adoption of green information systems. business & society, 58(5), 1083–1127. https://doi.org/10.1177/0007650317701674 chen, a. j., watson, r. t., boudreau, m. c., & karahanna, e. (2011). an institutional perspective on the adoption of green is & it. australasian journal of information systems, 17(1), 5–27. https://doi.org/10.3127/ajis.v17i1.572 cherns, a. (1976). the principles of sociotechnical design. human relations, 29(8), 783–792. https://doi.org/10.1177/001872677602900806 chiasson, m. w., & davidson, e. (2005). taking industry seriously in information systems research. mis quarterly, 29(4), 591–605. https://doi.org/10.2307/25148701 australasian journal of information systems gabryk & naidoo 2024, vol 28, research article sociotechnical perspectives on digital sustainability 25 crabtree, b.f., & miller w.l. (1992). a template approach to text analysis: developing and using codebooks. in b. f. crabtree & w. l. miller (eds.), doing qualitative research, (pp. 93–109). sage publications. davison, r. m., hardin, a. m., majchrzak, a., & ravishankar, m. n. (2019). call for papers: responsible is research for a better world. information systems journal, 33(1), 1–7. https://doi.org/10.1111/isj.12405 dubé, l., & paré, g. (2003). rigor in information systems positivist case research: current practices, trends, and recommendations. mis quarterly, 27(4), 597–636. https://doi.org/10.2307/30036550 ericsson, k. a., prietula, m. j., & cokely, e. t. (2007). the making of an expert. harvard business review, 85(7/8), 1–7. elliot, s. (2011). transdisciplinary perspectives on environmental sustainability: a resource base and framework for it-enabled business transformation. mis quarterly, 35(1), 197– 236. https://doi.org/10.2307/23043495 gholami, r., watson, r. t., hasan, h., molla, a., & bjorn-andersen, n. (2016). information systems solutions for environmental sustainability: how can we do more? journal of the association for information systems, 17(8), 521–536. https://doi.org/10.17705/1jais.00435 ghoshal, s. (2005). bad management theories are destroying good management practices. academy of management learning & education, 4(1), 75–91. https://doi.org/10.5465/amle.2005.16132558 gladwin, t. n., kennelly, j. j., & krause, t. s. (1995). shifting paradigms for sustainable development: implications for management theory and research. academy of management review, 20(4), 874–907. https://doi.org/10.5465/amr.1995.9512280024 gorman, m. r., & dzombak, d. a. (2018). a review of sustainable mining and resource management: transitioning from the life cycle of the mine to the life cycle of the mineral resources. conservation and recycling, (137), 281–291. https://doi.org/10.1016/j.resconrec.2018.06.001 haddaway, n. r., cooke, s. j., lesser, p., macura, b., nilsson, a. e., taylor, j. j., & raito, k. (2019). evidence of the impacts of metal mining and the effectiveness of mining mitigation measures on social–ecological systems in arctic and boreal regions: a systematic map protocol. environmental evidence, 8(1), 1–11. https://doi.org/10.1186/s13750-019-0152-8 hart, s. l., and milstein, m. b. (2003). creating sustainable value. academy of management perspectives, 17(2), 56–67. https://doi.org/10.5465/ame.2003.10025194 hertel, m., & wiesent, j. (2013). investments in information systems: a contribution towards sustainability. information systems frontiers, (15), 815–829. https://doi.org/10.1007/s10796-013-9417-x joy, j. (2004). occupational safety risk management in australian mining. occupational medicine, 54(5), 311–315. https://doi.org/10.1093/occmed/kqh074 australasian journal of information systems gabryk & naidoo 2024, vol 28, research article sociotechnical perspectives on digital sustainability 26 khuntia, j., saldanha, t. j., mithas, s., & sambamurthy, v. (2018). information technology and sustainability: evidence from an emerging economy. production and operations management, 27(4), 756–773. https://doi.org/10.1111/poms.12822 klein, h. k., & myers, m. d. (1999). a set of principles for conducting and evaluating interpretive field studies in information systems. mis quarterly, 23(1), 67-93. https://doi.org/10.2307/249410 kotlarsky, j., oshri, i., & sekulic, n. (2023). digital sustainability in information systems research: conceptual foundations and future directions. journal of the association for information systems, 24(4), 936–952. https://doi.org/10.17705/1jais.00825 kirsch, s. (2010). sustainable mining. dialectical anthropology, 34(1), 87–93. https://doi.org/10.1007/s10624-009-9113-x leidner, d. e., sutanto, j., & goutas, l. (2022). multifarious roles and conflicts on an interorganizational green is. mis quarterly, 46(1), 591–508. https://doi.org/10.25300/misq/2022/15116 linkov, i., trump, b. d., poinsatte-jones, k., & florin, m. v. (2018). governance strategies for a sustainable digital world. sustainability, 10(2), 440. https://doi.org/10.3390/su10020440 majchrzak, a., markus, m. l., & wareham, j. (2016). designing for digital transformation. mis quarterly, 40(2), 267–278. https://www.jstor.org/stable/26628906 malhotra, a., melville, n.p., & watson, r.t. (2013). spurring impactful research on information systems for environmental sustainability. mis quarterly, 37(4), 1265–1274. https://doi.org/10.25300/misq/2013/37:4.3 mccarthy, s., rowan, w., lynch, l., & fitzgerald, c. (2020). blended stakeholder participation for responsible information systems research. communications of the association for information systems, (47), 124–149. https://doi.org/10.17705/1cais.04733 meadows, d., & randers, j. (2012). the limits to growth: the 30-year update. routledge. https://doi.org/10.4324/9781849775861 melville, n. p. (2010). information systems innovation for environmental sustainability. mis quarterly, 34(1), 1–21. https://doi.org/10.2307/20721412 moran, c. j., lodhia, s., kunz, n. c., & huisingh, d. (2014). sustainability in mining, minerals and energy: new processes, pathways and human interactions for a cautiously optimistic future. journal of cleaner production, 84(1), 1–15. https://doi.org/10.1016/j.jclepro.2014.09.016 morelli, j. (2011). environmental sustainability: a definition for environmental professionals. journal of environmental sustainability, 1(1), 1–9. https://doi.org/10.14448/jes.01.0002 mumford, e. (2006). the story of sociotechnical design: reflections on its successes, failures and potential. information systems journal, 16(4), 317–342. https://doi.org/10.1111/j.13652575.2006.00221.x naumann, s., dick, m., kern, e., & johann, t. (2011). the greensoft model: a reference model for green and sustainable software and its engineering. sustainable computing: informatics and systems, 1(4), 294–304. https://doi.org/10.1016/j.suscom.2011.06.004 australasian journal of information systems gabryk & naidoo 2024, vol 28, research article sociotechnical perspectives on digital sustainability 27 nicolini, d., mørk, b. e., masovic, j., & hanseth, o. (2017). expertise as trans-situated-the case of tavi. in j. sandberg, et al. (eds), skillful performance: enacting capabilities, knowledge, competence, and expertise in organisations (pp. 27–49). oxford, uk: oxford university press. https://doi.org/10.1093/oso/9780198806639.003.0002 nishant, r., kennedy, m., & corbett, j. (2020). artificial intelligence for sustainability: challenges, opportunities, and a research agenda. international journal of information management, (53), 1–13. https://doi.org/10.1016/j.ijinfomgt.2020.102104 orlikowski, w. j., & iacono, c. s. (2001). research commentary: desperately seeking the “it” in it research—a call to theorizing the it artefact. information systems research, 12(2), 121–134. https://doi.org/10.1287/isre.12.2.121.9700 pan, s. l., carter, l., tim, y., & sandeep, m. s. (2022). digital sustainability, climate change, and information systems solutions: opportunities for future research. international journal of information management, (63), 1–5. https://doi.org/10.1016/j.ijinfomgt.2021.102444 pan, s. l., & zhang, s. (2020). from fighting covid-19 pandemic to tackling sustainable development goals: an opportunity for responsible information systems research. international journal of information management, 55(1), 1–6. https://doi.org/10.1016/j.ijinfomgt.2020.102196 petrini, m., & pozzebon, m. (2009). managing sustainability with the support of business intelligence: integrating socio-environmental indicators and organisational context. the journal of strategic information systems, 18(4), 178–191. https://doi.org/10.1016/j.jsis.2009.06.001 piotrowicz, w., & cuthbertson, r. (2009). sustainability a new dimension in information systems evaluation. journal of enterprise information management, 22(5), 492–503. https://doi.org/10.1108/17410390910993509 porter, m. e., & kramer, m. r. (2011). creating shared value: how to reinvent capitalism and unleash a wave of innovation and growth. harvard business review, 12, 1–17. https://doi.org/10.1007/978-94-024-1144-7_16 qureshi, s. (2015). are we making a better world with information and communication technology for development (ict4d) research? findings from the field and theory building. information technology for development, 21(4), 511–522. https://doi.org/10.1080/02681102.2015.1080428 roohy gohar, s., & indulska, m. (2020). environmental sustainability through green business process management. australasian journal of information systems, 24. 1-29 https://doi.org/10.3127/ajis.v24i0.2057 saldanha, t. j., mithas, s., khuntia, j., whitaker, j., & melville, n. p. (2022). how green information technology standards and strategies influence performance: role of environment, cost and dual focus. mis quarterly, 46(4), 2367-2386. https://doi.org/10.25300/misq/2022/16824 sarker, s., chatterjee, s., xiao, x., & elbanna, a. (2019). the socio-technical axis of cohesion for the is discipline: its historical legacy and its continued relevance. mis quarterly, 43(3), 695-720. https://doi.org/10.25300/misq/2019/13747 australasian journal of information systems gabryk & naidoo 2024, vol 28, research article sociotechnical perspectives on digital sustainability 28 sarker, s., chatterjee, s., & xiao, x. (2013). how "sociotechnical" is our is research? an assessment and possible ways forward. in proceedings of the 34th international conference on information systems, 9, 1–24. https://aisel.aisnet.org/icis2013/proceedings/generalistopics/9 schoormann, t., strobel, g., möller, f., & petrik, d. (2021). achieving sustainability with artificial intelligence—a survey of information systems research. in proceedings of the 42nd international conference on information systems (icis), 2,1–17. https://aisel.aisnet.org/icis2021/soc_impact/soc_impact/2 seidel, s., bharati, p., fridgen, g., watson, r. t., albizri, a., boudreau, m. c. m., butler, t., kruse, l.c., guzman, i., karsten, h., lee, h., melville, n., rush, d., toland, j., & watts, s. (2017). the sustainability imperative in information systems research. communications of the association for information systems, 40(1), 40–52. https://doi.org/10.17705/1cais.04003 seidel, s., recker, j., & vom brocke, j. (2013). sensemaking and sustainable practicing: functional affordances of information systems in green transformations. mis quarterly, 37(4), 1275–1299. https://www.jstor.org/stable/43825792 sganzerla, c., seixas, c., & conti, a. (2016). disruptive innovation in digital mining. procedia engineering, 138(1), 64-71. https://doi.org/10.1016/j.proeng.2016.02.057 stahl, b. c. (2012). responsible research and innovation in information systems. european journal of information systems, 21(3), 207–211. https://doi.org/10.1057/ejis.2012.19 steffen, w., richardson, k., rockström, j., cornell, s. e., fetzer, i., bennett, e. m., biggs, r. carpenter, s. r., de vries, w., de wit, c. a., folke, c., gerten, d., heinke, j., mace, g. m., persson, l. m., ramanathan, v., reyers, b., & sörlin, s. (2015). planetary boundaries: guiding human development on a changing planet. science, 347(6223), article 1259855. https://doi.org/10.1126/science.1259855 stillman, l., sarrica, m., anwar, m., sarker, a., & farinosi, m. (2020). sociotechnical transformative effects of an ict project in rural bangladesh. american behavioral scientist, 64(13), 1871–1888. https://doi.org/10.1177/0002764220952126 stuermer, m. (2014). characteristics of digital sustainability. proceedings of the 8th international conference on theory and practice of electronic governance, 494–495. https://doi.org/10.1145/2691195.2691269 swanson, e. b., & ramiller, n. c. (2004). innovating mindfully with information technology. mis quarterly, 28(4), 553–583. https://doi.org/10.2307/25148655 tim, y., cui, l., & sheng, z. (2021). digital resilience: how rural communities leapfrogged into sustainable development. information systems journal, 31(2), 323–345. https://doi.org/10.1111/isj.12312 tost, m., hitch, m., chandurkar, v., moser, p., & feiel, s. (2018). the state of environmental sustainability considerations in mining. journal of cleaner production, (182), 969–977. https://doi.org/10.1016/j.jclepro.2018.02.051 trist, e. l., & bamforth, k. w. (1951). some social and psychological consequences of the longwall method of coal-getting: an examination of the psychological situation and australasian journal of information systems gabryk & naidoo 2024, vol 28, research article sociotechnical perspectives on digital sustainability 29 defences of a work group in relation to the social structure and technological content of the work system. human relations, 4(1), 3–38. https://doi.org/10.1177/00187267510040010 united nations (2023). the sustainable development goals report 2023: special edition towards a rescue plan for people and planet. united nations. https://unstats.un.org/sdgs/report/2023/the-sustainable-development-goals-report2023.pdf vorster, a. (2001). planning for value in the mining value chain. journal of the southern african institute of mining and metallurgy, 101(2), 61–65. walsham, g. (1995). interpretive case studies in is research: nature and method. european journal of information systems, 4(2), 74-81. https://doi.org/10.1057/ejis.1995.9 walsham, g. (2006). doing interpretive research. european journal of information systems, 15(3), 320-330. https://doi.org/10.1057/palgrave.ejis.3000589 walsham, g. (2012). are we making a better world with icts? reflections on a future agenda for the is field. journal of information technology, 27(2), 87–93. https://doi.org/10.1057/jit.2012.4 washington, h. (2018). harmony—not ‘theory’. the ecological citizen, 1, 203–210. watson, r. t., boudreau, m.-c., & chen, a. j. (2010). information systems and environmentally sustainable development: energy informatics and new directions for the is community. mis quarterly, 34(1), 23-38. https://doi.org/10.2307/20721413 world commission on environment and development (wced) (1987). world commission on environment and development. our common future. 17, 1–91. wced. https://www.are.admin.ch/are/en/home/media/publications/sustainabledevelopment/brundtland-report.html winter, s. j., & butler, b. s. (2011). creating bigger problems: grand challenges as boundary objects and the legitimacy of the information systems field. journal of information technology, 26(2), 99108. https://doi.org/10.1057/jit.2011.6 whitmore, a. (2006). the emperor’s new clothes: sustainable mining? journal of cleaner production, 143(3-4), 309-314. https://doi.org/10.1016/j.jclepro.2004.10.005 wut, t. m., lee, d., ip, w. m., & lee, s. w. (2021). digital sustainability in the organization: scale development and validation. sustainability, 13(6), 1–14. https://doi.org/10.3390/su13063530 young, a., & rogers, p. (2019). a review of digital transformation in mining. mining, metallurgy and exploration, 4, 683-699. https://doi.org/10.1007/s42461-019-00103-w zampou, e., mourtos, i., pramatari, k., & seidel, s. (2022). a design theory for energy and carbon management systems in the supply chain. journal of the association for information systems, 23(1), 329-371. https://doi.org/10.17705/1jais.00725 zhang, h., liu, l., & li, t. (2011). designing it systems according to environmental settings: a strategic analysis framework. the journal of strategic information systems, 20(1), 80–95. https://doi.org/10.1016/j.jsis.2011.01.001 https://doi.org/10.1016/j.jclepro.2004.10.005 australasian journal of information systems gabryk & naidoo 2024, vol 28, research article sociotechnical perspectives on digital sustainability 30 appendix interview questions considering this research forms part of a more extensive study, exemplar questions that inform this research are presented below. questions relating to professional background: 1. what is your academic background? 2. what is your professional background? questions relating to sustainability perspectives of digital technology in the mining environment: 1. how have you been involved or affected by information systems within the business environment? could you perhaps provide a few examples around is that have affected those involved, specifically regarding reducing carbon footprint and sustainability or safety? 2. do you have any insights into the financial ramifications of sustainability in any of your experiences so far? 3. what has helped you support sustainability? note: questions were further explored with the use of appropriate probes. examples include: 1. could you please give me an example of what you mean by helping out the communities in the social aspect? 2. could you perhaps expand a little bit on what you meant by sustainability and give an example, please? copyright: © 2024 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v28.4369 http://creativecommons.org/licenses/by-nc/3.0/au/ https://doi.org/10.3127/ajis.v28.4369 1 introduction 2 literature review and theoretical background 2.1 related digitalisation and sustainability studies 2.2 sociotechnical perspectives of digitalisation and sustainability 3 research methodology 3.1 case study site 3.2 data collection 3.3 data analysis 4 case analysis and findings 4.1 privileging business sustainability over human, social and environmental sustainability 4.2 privileging automation over human labour 4.3 privileging a longer time horizon for sustainability 5 discussion 6 implications 6.1 theoretical implications 6.2 practical implications 7 limitations and future research paper title (paper title style) australasian journal of information systems ractham et al. 2022, vol 26, research article old but not out 1 old but not out: social media use and older adults' life satisfaction peter ractham thammasat business school & center of excellence in operation and information management, thammasat university, bangkok, thailand angsana a. techatassanasoontorn auckland university of technology, new zealand laddawan kaewkitipong thammasat business school, thammasat university, bangkok, thailand laddawan@tbs.tu.ac.th abstract social media has the potential to have a positive influence on older adults' quality of life. this study explores how older adults use social media and the implications of the use on their life satisfaction. a sequential mixed-method approach is used. first, focus group interviews were conducted with older adults in thailand, and a two-step sorting procedure is employed to develop comprehensive measures of social media use activity and their life domain affiliation. next, a field survey is used to evaluate the influence of satisfaction from social media use on domain life satisfaction and overall life satisfaction. the findings suggest that older adults integrate social media into activities in several life domains, including family, friend, community, health, consumer, education, self, leisure, and social. satisfaction from social media use activities positively associates with domain life satisfaction in all those life domains. the comprehensive measures of social media activities enable us to extensively theorize social media use and illustrate that it has a different meaning for older adults compared with young adults. keywords: life satisfaction, older adults, social media, subjective well-being. 1 introduction information and communication technology (ict) offers potential benefits for older adults to maintain their independence and increase their quality of life. the united nations reported a worldwide demographic trend of an increasing share of older adults known as population aging. according to the world health organization, a person aged 60 years or over is considered an older adult (who, 2018). in 2015, the united nations reported that the population of older adults aged 60 or over, which comprise 12.5% of the global population, is the fastest-growing population segment in the world (united nations, 2015) and predicted to grow to around 2.1 billion in 2050, more than doubling from 962 million people in 2017 (united nations, 2017). therefore, population aging has immense social and economic consequences and increases the need to put resources into promoting the well-being of an increasing number of older people. despite their slow adoption of technology, there is growing evidence to illustrate that older adults have a positive attitude toward technology and show a willingness to incorporate technology into their lives. a study by mitzner et al. (2010) looked at older adults' attitudes australasian journal of information systems ractham et al. 2022, vol 26, research article old but not out 2 toward technology and reported that they used several technologies at home to facilitate different aspects of their lives and enjoyed how technologies make their life easier. in thailand, the latest survey by thailand electronic transactions development agency (etda, 2020) revealed that the three most popular online activities among the thai baby boomer generation, or people aged 57-75 years, were using social media, searching information, and listening to music or watching video clips. like other countries, thailand has experienced population aging. it is expected to become a super-aged society in 2031, with older adults aged 60 years and over accounting for more than 28% of the population (thailand prd, 2020). in recent years, social media has significantly changed the way people communicate, interact, and socialize with one another. social media1 refers to various user-driven platforms that allow users to produce and share content, thus facilitating dialogue creation and communication to a broader audience (kapoor et al., 2018). social media has also been used to promote social inclusion for various groups of people who might be vulnerable to be excluded from social activities (verdegem, 2011). in particular, social media has proven to be an effective technology that helps older adults become socially included (trentham et al., 2015). social media include social network sites and services (e.g., facebook, line), video sharing sites (e.g., youtube), blogging and microblogging (e.g., twitter), among others. the first wave of social media users was the young people who use it as a platform of communication with their friends, allowing them to have privacy and keep "what they do online" from nosy parents (boyd, 2014)2. over the years, the user base of social media has expanded beyond teenagers, indicating the changing landscape of users. according to the pew internet report, older adults, despite being late adopters, are among the fastest-growing groups to use social media. in 2021, 45% of older adults aged 65 years or older in the u.s. used at least one type of social media, compared to 21% in 2013 and 12% in 2010 (pew research center, 2021). although statistics have shown that older adults have started to embrace social media in their lives, we do not know the role of social media in their life activities and how social media use affects their lives. this is because, driven by the concern that a small percentage of older adults adopts ict, previous studies on older adults and technology have mostly explored the agerelated digital divide to explain factors and hindrances associated with technology adoption among this age group (chen and chan, 2011; hill et al., 2008; lam and lee, 2006; niehaves and plattfaut, 2014; selwyn, 2004; xu et al., 2020). to address the gap in the literature, this study focuses on older adults who have adopted and used social media. we address two research questions: 1) how do older adults engage with social 1 in their comprehensive review of social media studies, kapoor et al. (2018) found that few studies defined the concept of social media. based on the definitions used in the literature, they proposed the following definition: “[s]ocial media is made up of various user-driven platforms that facilitate diffusion of compelling content, dialogue creation, and communication to a broader audience. it is essentially a digital space created by the people and for the people, and provides an environment that is conducive for interactions and networking to occur at different levels” (p. 536) 2 danah boyd uses a lower-cased style to refer to herself. on her web site, www.danah.org, she explained that she wants “to reflect my mother’s original balancing and to satisfy my own political irritation at the importance of capitalization”. australasian journal of information systems ractham et al. 2022, vol 26, research article old but not out 3 media to support life activities?, and 2) what is the influence of social media use on older adults' life satisfaction? drawing on social media use and subjective well-being literature, this research aims to produce a comprehensive account of how older adults use social media and the effect of its use on their life satisfaction. since social media is malleable and people can use it to suit their needs, it is not possible to theorize its use a priori. to elucidate social media use, we first conducted focus group interviews with older adults in thailand and developed the social media use instrument associated with life domains. next, we conducted a field survey to evaluate the relationship between social media use and life satisfaction. in the following section, we review research on social media use and the subjective well-being concepts to formulate a preliminary theoretical framework for our study. next, we present the methodology and analysis approaches followed by the discussion of results. finally, we discuss research and practical implications, limitations, and avenues for future research. 2 theoretical background this study draws on the literature on social media use and the concept of subjective well-being with an emphasis on life satisfaction to conceptualize our preliminary research framework. we first offer a brief review of research on older adults and technology. 2.1 older adults and technology a recurring stream of research on technology and older adults aims to identify factors that explain the age-related digital divide and hindrances to technology adoption among older adults (charness and boot, 2009; chen and chan, 2011; niehaves and plattfaut, 2014). some studies looked at older adults who already adopt ict to better understand their ict activities and impact. for example, hur (2016) looked at the connection between objectives for participating in ict-based activities (relationship improvement, information collection, learning, shopping, and business activities) and empowerment. he found that participating in ict-based learning could improve older adults' empowerment. in another study, hage et al. (2016) found that the choice of communication tools influences social connectivity among older adults who live in small rural communities. they reported that facebook has a negative impact on connectivity with friends but not on connectivity with others in the community. however, these studies focus on selected aspects of ict use and do not link purposes of technology use (e.g., connections) with various contexts (e.g., with friends, with family) in order to develop a precise understanding of the impact of technology. 2.2 social media use and influence the central tenet that drives the growth of users of social media sites is its design around friendship. in her book on social media, boyd (2014) highlighted four affordances that shape mediated interactions on this platform: persistence, visibility, spreadability, and searchability. persistence means that contents and expressions on social media endure and do not expire. interactions and contents on social media can be shared with public audiences, enabling visibility of messages and encouraging broad participation from the public. social media often encourages people to spread information widely using easy-to-use tools such as "tags," "share," and "retweet." the spreadability of content allows individuals to share information with their circle of friends, mobilize volunteers, support political campaigns, and even spread rumors (boyd, 2014; cogburn and espinoza-vasquez, 2011; turel and serenko, 2012). generally, australasian journal of information systems ractham et al. 2022, vol 26, research article old but not out 4 contents on social media are archived and searchable, allowing people to find information on any topics of interest easily. as more people join social media, this technological environment opens up new social dynamics and transforms everyday practices for other groups of populations beyond teens and young adults. however, empirical evidence on the effects of social media use is mixed with positive (oh et al., 2014; valenzuela et al., 2009; wang et al., 2014; kim and shen, 2020; gaia et al., 2021) and negative consequences (kross et al., 2013; turel and serenko, 2012; rasmussen et al., 2020). a closer examination of these findings offers a more nuanced understanding of the influence of social media use. for example, oh et al. (2014) examined social media users aged 18 to 81. they found that the amount of supportive interaction, but not the frequency or amount of social media use, is important in establishing social support and emotional well-being, and life satisfaction. in addition, the number of friends on social media was not associated with either positive or negative affect. gaia et al. (2021) reported a positive association between social media use and life satisfaction for older adults. leist (2013) looked at how older adults use social media to engage in social activities, provide and receive supports, increase feelings of control, and overcome the barriers of social exclusion. celik et al. (2018) examined several important factors that influence life satisfaction for older adults, such as psychological effects, physical health, health status, daily life and instrumental activity. some of the lessening life satisfaction variables for older adults include lower self-report health status, a decline in physical health, and increase in feeling depressed and feeling social withdrawal and exclusion. in particular, being active within different community activities can increase life satisfaction for older adults. finally, in a comparative study, kim and shen (2020) examined the effects of engaging in different facebook activities on older users over 50 compared to younger adults aged between 18 and 25 and found that older adults benefit less from having a large network and more from engaging in directed communication activities. this supports our proposition that older people might use social media differently from other age groups and thus require an in-depth study. in summary, what we learn from studies in the literature is that it may be too simplistic to use proxies such as frequency, time spent on social media, and a number of friends to measure social media use and evaluate its impacts. a fine-grained, activity-level elaboration of social media use opens up opportunities to derive a more precise explanation of social media use and outcomes. for example, turel and serenko (2012) used the comprehensiveness of usage construct developed by limayem et al. (2007) to measure the extent to which an individual makes use of the various functionalities of social media (e.g., posting messages and photos, reading messages sent by others, and sending gifts). they found that the range of social media functionalities that users employ provides a more robust explanation of the negative and positive outcomes of use than the amount of time spent on social media. building on this idea, we link turel and serenko's (2012) functionality approach with underlying motivations in various life domains to measure social media use. for example, one may use social media to look up product information and prices to support activities in the consumer life domain. in contrast, another may use social media to search for information related to healthcare to support activities in the health life domain. we explain our approach to measure social media use in the methodology section. another observation is that previous research mostly studied teens and young adults due to the concerns about the negative effects of the new technologies on their lives (boyd, 2014; australasian journal of information systems ractham et al. 2022, vol 26, research article old but not out 5 valenzuela et al., 2009). a review of literature on social media use by leist (2013) discussed possible positive and negative consequences of social media use among older adults. however, their study did not empirically evaluate these consequences. khoo and yang (2020) studied the impacts of social media use in middle-aged and older adults on only executive functions, which is a set of domain-general cognitive control processes, but not other aspects of their lives and the impacts of their social media use on life satisfaction or well-being. gaia et al. (2021) argued that not many studies have empirically investigated the relationship between older people's social media use and well-being. although their study looked at the relationship between social media use and life satisfaction, they did not investigate how older adults use social media. since older adults are one of the fast-growing users of social media, we need to have a better understanding of how they use social media and the implications of its use on their lives, which is the aim of this study. 2.3 subjective well-being: life satisfaction we draw on subjective well-being literature to theorize outcome variables associated with social media use. subjective well-being refers to how people feel about their lives (diener, 1984; kahneman and krueger, 2006). this concept constitutes three interrelated components: people's feelings, domain life satisfaction, and overall life satisfaction. although all three components are important to evaluate people's subjective experiences, research suggests that domain life satisfaction and overall life satisfaction can portray a state of people's lives (diener et al., 2013). life satisfaction allows people to reflect and assess how they feel about many areas of their lives, such as health, social activities, work, and family, among others. in this study, we draw on the concept of life satisfaction (diener, 1984) to theorize the impact of social media use. life satisfaction is a global assessment of the quality of life according to a person's own criteria. thus, the extent of life satisfaction may depend on one's circumstances and what the person sets as the standard of a good life for himself or herself. this study uses the life domain approach to explain the influence of social media use on life satisfaction. the tenet of this approach is that overall life satisfaction is shaped by reported satisfaction with various life domains. an individual's life can be segmented into several life domains (andrews and withey, 1976). some of the key ones are family, friends, community, health, leisure, and work (diener and diener, 1995; easterlin, 2006; headey et al., 1991; oishi et al., 1999; van praag et al., 2003). to establish the link between domain life satisfaction and overall life satisfaction associated with social media use, we first develop an account of social media use at an activity level and subsequently evaluate the impact of social media use on life satisfaction. our activity-level measurement of social media use is aligned with scheepers and middleton's (2013) framework on the study of personal icts. this line of thinking highlights an individual frame of reference to use personal icts "to support typical routines across the multiple contexts of his or her life" (p. 382). in addition, our approach answers the call from other researchers to find an alternative paradigm that treats users as active participants in the ict use experience to develop a comprehensive understanding of their ict use. this new paradigm is appropriate to study several emerging ict such as blogs, social media sites, virtual worlds, and mobile services that support diverse user goals and values. for example, jung's (2014) investigation of different values that users pursue with smartphones pays attention to "what users actually do with the technology rather than which factors affect user adoption of a given technology" (p. 313). australasian journal of information systems ractham et al. 2022, vol 26, research article old but not out 6 similar to personal icts and other emerging ict, social media use span multiple boundaries such as home, work, and leisure. in addition, different individuals have the freedom to choose to engage with social media to support their idiosyncratic needs. for example, one person may use social media to keep in touch with old friends, and another person may choose to use social media for entertainment, searching for and sharing health information, and volunteering. we use domains of life (e.g., family, friend, community, health) to organize social media use to give participants a frame of reference to describe their social media use that spans multiple contexts and multiple purposes and allow us to develop a better understanding of the meaning of social media in their lives. the findings will also enable us to identify salient domains in which social media use has a strong contribution to people's overall life satisfaction. 2.4 preliminary theoretical model our preliminary theoretical framework is built on the broad theorizing of the life satisfaction process (campbell et al., 1976; diener et al., 1999). the theoretical logic of this process is that overall life satisfaction is derived from one's satisfaction with various life domains. reported satisfaction in each life domain, in turn, reflects the extent to which one's goals and needs in that life domain are met through domain-related activities. for example, ill health can make a person feel bad and unhappy. it is likely to negatively influence how he or she assesses his or her overall life satisfaction (diener, 1984). in this study, we extend the life satisfaction process logic to examine the role of social media in people's lives. social media is applicable and useful for various activities across different life domains. for example, choi et al. (2014) show how social media can be appropriated for health promotion. social media is a useful platform for people to share their opinions regarding products and services (turel and serenko, 2012). park et al. (2009) found that individuals use facebook for various purposes, such as socializing, status-seeking, entertainment, and information seeking. similarly, xu et al. (2012) reported the relationship between affection, coordination, entertainment, immediate access, relaxation gratifications, and social media use. however, previous research on social media use has mostly focused on youth and young adults. the results may be less applicable to older adults due to different lifestyles and purposes of use. for example, gatto and tak (2008) suggest that older adults' usage of ict applications are different from the younger generation due to physical limitation, learning ability, and access to technology. based on these arguments, we extend the life satisfaction process reasoning to argue that satisfaction from social media use across life activities shapes domain life satisfaction, which, in turn, influences overall life satisfaction. it is important to emphasize that the theoretical model offers a process logic to guide this study's empirical inquiry. the precise theoretical logic of the relationship between social media use, domain life satisfaction, and overall life satisfaction is inductively analyzed and emerged from the data. figure 1 presents the preliminary theoretical model to illustrate the process of how social media use influences life satisfaction. australasian journal of information systems ractham et al. 2022, vol 26, research article old but not out 7 figure 1. a preliminary model of social media use and impacts on life satisfaction note: the diagram uses the leisure, health, family, and social life domains to illustrate the influence of social media use on life satisfaction. in an empirical evaluation, the life domains are inductively analyzed and emerged from the data on older adults' use of social media in their daily lives. 3 methodology and analysis we use a sequential mixed method approach in our research inquiry. the inquiry constitutes the use of focus group interviews to develop a comprehensive account of social media use, followed by a field survey to evaluate the relationship between social media use and life satisfaction through domain life satisfaction. table 1 presents how we apply venkatesh et al. 's (2013) mixed-method design guidelines in this study with two purposes in mind: completeness and developmental. in the context of our research, completeness refers to the usefulness of a mixed-methods design to obtain a complete picture of a phenomenon, which is to develop a substantive theory3 of social media use and its impact on older adults' lives. developmental refers to the efficacy of a mixed-method design to use extensive measures of social media use across life domains from a qualitative method to evaluate the relationship between social media use and life satisfaction using a quantitative method. guideline application in this study decide on the appropriateness of a mixed-methods approach the state-of-the-art research on social media use and its consequences is limited to young adults, and the findings are inconclusive with a mix of positive and negative outcomes. this gives us an opportunity to provide a holistic understanding of social media use and its impacts. develop a strategy for mixed methods research design the goal of this study is to develop rich insights into older adults' social media use in various life domains and evaluate its impacts on life satisfaction. since social media is a multi-purpose technology and users choose to use the technology that best serves their needs, there is a need to use a qualitative method (focus groups) to develop a comprehensive understanding of social media use. a quantitative approach (survey) is then used to evaluate the impact of social media use on older adults' life satisfaction through domain life satisfaction. 3 substantive theory refers to theory that constitutes concepts and their relations to explain a phenomenon or a substantive area of inquiry (glaser and strauss, 1967; urquhart et al., 2010; venkatesh et al., 2013). in this study, the mixed method approach enables us to develop in-depth understanding of social media use and its meaning to people’s lives through life satisfaction outcomes. australasian journal of information systems ractham et al. 2022, vol 26, research article old but not out 8 develop a strategy for analyzing mixed methods data a comprehensive account and validated instrument of social media use and associated life domains is derived from focus group interviews and two-step sorting procedures. the validated social media use items are incorporated into a survey along with measures of life satisfaction and domain life satisfaction. measurement model validity is assessed before the evaluation of the structural model using pls-sem. draw meta-inferences from mixed methods results this study develops a meta-inference or substantive theory of social media use and its impact on older adults' life satisfaction that identifies social media use activities in relation to life domains that are salient to life satisfaction. table 1. mixed methods research guidelines we inductively develop precise measurements of social media use from our study participants by following a two-phased approach. in the first phase, we developed a comprehensive account and validated instrument of social media use and associated life domains through focus group interviews and sorting procedures, respectively. in the second phase, we used the social media use instrument from the first phase along with validated measures of the domain and overall life satisfaction to examine the influence of social media use on older adults' life satisfaction. 3.1 phase 1: development of social media use measures we conducted initial focus group interviews with two groups of 15 older adults (10 females and 5 males). then, the members from these first two groups referred us to two groups of retired professionals (5 females and 2 males) and retired civil servants (3 females and 3 males). these first four focus group interviews were conducted in bangkok, thailand. after that, we identified those who are social media users outside of bangkok. we expanded our interviews to all the regions in thailand through the means of convenience sampling by making posts on facebook and sending private messages to various groups of ngos, associations of senior citizens, and associations of retired civil servants. in addition, we also contacted potential participants by phone and through social media channels. overall, we conducted 24 group interviews with 124 older adults in 20 provinces across thailand. the focus group interviewees’ profiles are shown in table 2. number of interviewees percent gender male 54 43.5 female 70 56.5 regions central (including bangkok) 51 41.4 north 20 16.1 south 18 14.4 northeast 35 28.1 age 60 – 65 years old 84 67.7 66 70 years old 30 24.2 70 75 years old 10 8.1 education level below high school 9 7.2 high school 16 12.9 technical college 20 16.1 bachelor degree 75 60.4 australasian journal of information systems ractham et al. 2022, vol 26, research article old but not out 9 number of interviewees percent master or ph.d. 4 3.2 monthly income 10,001 – 20,000 baht 30 24.1 20,001 – 30,000 baht 65 52.5 more than 30000 baht 29 23.4 social media use frequency everyday 90 72.7 5 6 days per week 10 8.0 3 4 days per week 11 8.8 1 2 days per week 6 4.8 1 day fortnightly 5 4.0 less than 1 day fortnightly 2 1.7 hours per day less than 1 hour 10 8.0 1 3 hours 35 28.4 3 5 hours 70 56.6 more than 5 hours 9 7.2 social media platforms used (can choose more than one choice) facebook 101 81.4 youtube 101 81.4 line chat 120 96.7 instagram 35 28.2 blog 4 3.2 others 3 2.4 work status full-time 18 14.5 part-time 24 19.3 retired 82 66.2 living condition live by themselves 11 8.2 live with family or relatives 89 71.8 live with friend(s) 2 1.8 live with spouse 20 16.4 others 2 1.8 table 2. focus group interviewee profiles each focus group interview session was between 60 – 90 minutes. during the focus group sessions, we asked participants to share information using three open-ended questions: "how did you start using social media and why?" (goal: to obtain background information about their lifestyles) "what social media do you frequently use and why?" (goal: to understand areas of social media use) and "what do you usually use social media for and how do you use it to support your lifestyle?) (goal: to identify social media use, associated life domains, and the perceived influence of social media use on domain life satisfaction). we audio-taped and transcribed all the interview sessions. the 108-page transcripts were used to identify social media use and associated life domains. based on the life domain definitions from the literature, the two authors independently coded social media use and their associated life domains, according to the three key pieces of data from the focus group interviews. these data cover the purposes of social media use, how it was used, and the types of social media platforms. for example, "i use facebook to contact my daughter who lives in america." was australasian journal of information systems ractham et al. 2022, vol 26, research article old but not out 10 coded as "contact family" (social media use) that is associated with the "family life domain" (life domain). disagreements on social media use or life domains were resolved through discussion between the two coders. overall, a total of 48 different social media use in ten life domains were identified. they were family, friend, community, leisure, health, self, social, education, consumer, and work life domains. these life domains and selected interview excerpts are shown in table 3. life domain goal context type interview excerpts family stay connected live separately from the family facebook, skype, and line chat app "i feel that i am closer to my family because of social media." friend nurture friendship reconnect with old friends facebook and line chat app "i found many old friends from high school and college." community community involvement participate or invite others to participate in community services facebook "i volunteered for many community events." leisure pass the time wait in queues line chat app and youtube "i usually use line chat with my friends when i'm waiting in line." health health improvement learn about illness and alternative medicines youtube and blog "i learned about herbal medicines from youtube." self selfaccomplishment want to show off selfaccomplishment facebook "i like to show my vacation pictures on social media." social expand social network find new friends and those who share similar interests facebook and blog "i use social media to find a new interest group like the bicycle club." education learn new things learn about cooking and traveling youtube "i usually learn about new recipes on youtube." work achieve work related activities communicate with colleagues/customers facebook and line chat app "i use facebook and/or line chat to contact my colleagues or customers quite often." consumer save money want to save money facebook "i search for deals to save money on social media." table 3. ten life domains and selected interview scripts next, we used two-step sorting procedures (moore and benbasat, 1991) to develop the social media use scale and assess the reliability and construct validity of the social media use items. as suggested by petter et al. (2007, p. 639), "[a] two-step q-sort may be one of the best methods to assess content validity for formative constructs." several studies in is have used the term qsorts to refer to these sorting procedures in the instrument development process (chang and king, 2005; gerow et al., 2015). however, thomas and watson (2002, p. 153) strongly argue that the validation sorts proposed by moore and benbasat (1991) and subsequently used by many is researchers "should be labeled something other than 'q-sorts' to reduce confusion." therefore, in this study, we use the term sorting procedures instead of q-sorts to refer to the procedure we followed to establish content and construct validity. in the first round, four participants who were not involved with the project were asked to examine and sort 48 social media use items into the ten life domains identified from the focus group data. if the participants believed that a particular social media use item did not match australasian journal of information systems ractham et al. 2022, vol 26, research article old but not out 11 any of the ten life domains, they could categorize this item as part of the "ambiguous or unclear" group. all items were presented on index cards for convenience with the sorting process. we checked the level of agreement among participants using fleiss's kappa, which has an average value of 0.64 (p<.001), suggesting substantial agreement among them. after the sorting procedure, the authors and the four participants discussed and clarified the ambiguous items. a few items were merged together, and the number of social media use items was reduced to 46 items. in the second round, we asked two different participants to sort the modified social media use items from the first round into life domains as they saw fit. the two participants created eight and twelve life domains, respectively. the calculated fleiss's kappa was 0.45 (p<.001), suggesting moderate agreement between the two participants. the most ambiguous domains were the community and social life domains. the authors and the two participants discussed and agreed that these two life domains should remain separate. we also agreed that some of the similar social media use items should be merged. the final number of items was reduced to 44 (see these items in table 5 in the results section). based on the results from our coding and sorting procedures, we concluded that social media use measures have content, convergent, and discriminant validity. 3.2 phase 2: examining the influence of social media use on older adults' life satisfaction in this phase, we used a field survey to evaluate the influence of satisfaction from social media use on domain life satisfaction, which, in turn, shapes overall life satisfaction. we used three groups of constructs in the questionnaire: satisfaction from social media use, domain life satisfaction, and overall life satisfaction. social media use items constitute 44 items from the scale developed in phase 1. domain life satisfaction is a single-item scale drawn from diener and diener (1995). the overall life satisfaction construct has five validated items from diener et al. (1985). we measured all three constructs with the 7-point likert scale. the scales of satisfaction from social media use are anchored with 1 – "strongly dissatisfied" to 7 – "strongly satisfied", while a scale of 0 is used for "never used." the domain life satisfaction and overall life satisfaction scales were measured from 1 – "strongly disagree" to 7 – "strongly agree." the survey instrument was pretested with 12 older adults. some modifications were made to clarify a few social media use items. to draw survey participants from diverse and different economic and educational backgrounds, we distributed 600 questionnaires to older adults from all four regions in thailand. during the three months of the field survey, one of the authors and three research assistants mailed the survey to participants and traveled to various parts of thailand to distribute, administer, and collect the questionnaires. in total, we received 341 questionnaires back from both face-to-face meetings and postal mail. after excluding cases with missing data or incomplete responses, 337 surveys were retained for data analysis. overall, there are 139 male and 198 female respondents, and approximately 75% of participants were between 60-65 years old. more than 70% of respondents are retirees. most (64%) are married and live with their spouse. close to 65% have a college degree, and about 48% earn a monthly income of between $1000 $3000, which is considered to be in the middle-income bracket in thailand. respondents have an average of 7 years of computer experience and 3.5 years of social media experience. sixty percent of the respondents have used social media for between 2-5 years. more than 53% of the respondents use social media on a daily basis, while about 29% use it 3australasian journal of information systems ractham et al. 2022, vol 26, research article old but not out 12 6 days a week. facebook (69%) and line chat (73%) are popular social media platforms that respondents use. respondent profiles are shown in table 4. number of respondents percent gender male 139 41.2 female 198 58.8 age 60 – 65 years old 253 75.1 66 70 years old 53 15.7 70 75 years old 26 7.7 76 80 years old 5 1.5 marital status single 46 13.6 widow 54 16.0 married 216 64.1 divorced or separated 21 6.2 education level below high school 41 12.2 high school 43 12.8 technical college 35 10.4 bachelor degree 160 47.5 master or ph.d. 57 16.9 monthly income less than 2000 baht 9 2.7 2001 – 10,000 baht 93 27.3 10,001 – 20,000 baht 78 23.1 20,001 – 30,000 baht 83 24.6 more than 30000 baht 74 22.0 social media use frequency everyday 180 53.4 5 6 days per week 48 14.2 3 4 days per week 50 14.8 1 2 days per week 35 10.4 1 day fortnightly 9 2.7 less than 1 day fortnightly 15 4.5 hours per day less than 1 hour 74 22.0 1 3 hours 161 47.8 3 5 hours 86 25.5 more than 5 hours 16 4.7 social media platforms used (can choose more than one choice) facebook 231 68.5 youtube 138 40.9 line chat 246 73.0 instagram 31 9.2 blog 5 1.5 others 5 1.5 work status full-time 45 13.4 part-time 44 13.1 retired 246 73.0 australasian journal of information systems ractham et al. 2022, vol 26, research article old but not out 13 number of respondents percent living condition live by themselves 35 10.4 live with family or relatives 124 36.8 live with friend(s) 2 0.6 live with spouse 172 51.0 others 4 1.2 table 4. respondent profiles next, we conducted partial least squares structural equation modeling (pls-sem) to evaluate the influence of satisfaction with social media use on domain life satisfaction and the influence of domain life satisfaction on overall life satisfaction. we decided to use pls-sem because our data did not have a multivariate normal distribution. mardia's coefficient is higher than 10, indicating non-normality. pls does not have a strict requirement for data distribution (hair et al., 2013). our sample size of 337 satisfies the minimal sample for pls-sem, which should be ten times the largest number of structural paths pointing to any latent construct in the model. in our model, the largest number of paths pointing to overall life satisfaction is 10. finally, our model has both formative constructs (satisfaction from social media use) and reflective constructs (overall life satisfaction). since each formative construct (e.g., satisfaction from social media use in the family domain) is linked to only one effect construct (family life satisfaction), the model is underidentified and does not allow the estimation of the paths from the formative indicators to the construct and the variance of the error term in the covariancebased sem (diamantopoulos, 2011). therefore, we believe that pls-sem is an appropriate analysis method. 4 results we first present the respondents' patterns of social media use and levels of satisfaction. next, we report the evaluation of the influence of satisfaction from social media use on domain life satisfaction and overall life satisfaction. table 5 presents the patterns of social media use and the respondents' levels of satisfaction. from the analysis, the participants showed satisfaction with social media use in all ten life domains. the top ten social media use activities are in the family, health, leisure, consumer, self, and friend domains. our finding is consistent with that of chou et al. (2009), who argued that social media has the potential for health communication and that an increasing number of older adults are likely to use social media for such purpose. the high degree of satisfaction from social media use for familial purposes among older adults is not surprising. it was reported in croll (2006) that asians have a strong commitment to family, and relationships within the family are important to their well-being. in addition to activities related to health and family domains, older adults also use social media for leisure purposes, such as taking photographs and posting pictures, playing games, and browsing social media to pass the time. while older adults reported social media use and satisfaction with their use in the family, health, and leisure domains, they are least satisfied with the use of social media to form new social relationships. although social media have been used mostly by adolescents and young adults to find new friends, older adults voice their concern about using social media to establish new relationships due to the lack of trust and safety concerns. this is in line with arguments in prior research that older adults have a strong emotional investment with family australasian journal of information systems ractham et al. 2022, vol 26, research article old but not out 14 members and established friends (chen and chan, 2011; hur, 2016). in addition, trust is an important factor affecting information sharing and new relationship development in an online setting, particularly for older adults (choi et al., 2014). life domain social media use never use mean s.d. family use social media to discuss, share pictures and follow family members' activities 2.4% 5.77 1.55 use social media to check on family members' safety 8.3% 5.18 1.99 use social media to tell family about my whereabouts 11.3% 4.90 2.14 use social media to connect with family members who live far away 4.7% 5.62 1.73 friend use social media to send electronic greeting cards on special occasions 8.0% 5.30 1.96 use social media to communicate with friends and follow their activities 4.5% 5.56 1.63 use social media to share safety news with friends 8.9% 5.01 1.98 use social media to arrange get-together with friends 6.2% 5.34 1.85 use social media to rediscover old friends 12.2% 4.87 2.17 work use social media to distribute work-related news 9.5% 5.02 1.98 use social media to communicate with customers 20.5% 4.10 2.39 community use social media to distribute and receive community/provincial related news and activities 8.6% 5.07 1.99 use social media to distribute news related to volunteer activities within own community 11.0% 5.05 4.41 use social media to connect between community leaders and members 10.7% 4.70 2.08 use social media to receive news from central government and distribute them to community members 18.4% 4.26 2.33 use social media to distribute charitable news to community members. 11.9% 4.76 2.16 health use social media to search for information related to healthcare, exercise and healthy food 1.8% 5.60 1.43 use social media to consult friends, relatives or health experts on health-related issues 9.8% 4.91 1.99 play social media games to do brain exercise to prevent memory loss 3.6% 5.73 1.53 use social media to share information about health and medicines 5.6% 5.41 1.72 consumer use social media to look up product information and price 9.2% 5.03 1.95 use social media to follow news from favorite shops 13.1% 4.59 2.11 use social media to make phone or video calls to save money 3.9% 5.72 1.67 use social media to exchange product information with others 19.3% 4.13 2.30 education use social media to learn new things such as stock trading, english language and new technology 3.6% 5.42 1.69 use social media to find useful information such as diy projects 10.4% 4.66 2.14 use social media to find answers from experts 8.9% 5.03 1.99 self use social media to post pictures of what i accomplish with my hobbies 4.2% 5.57 1.66 use social media to share personal thoughts and opinions with people who share similar interests 6.2% 5.18 1.78 use social media to read dharma to lift up my spiritual self 5.9% 5.00 1.78 australasian journal of information systems ractham et al. 2022, vol 26, research article old but not out 15 table 5. social media use and respondents' level of satisfaction we also performed a break-down analysis of social media use by age groups (see table 6). overall, all age groups shared similar choices of social media platforms that they used. in particular, line chat and facebook were popular social media platforms across all age groups. this is in line with the statistics presented in table 5 that older adults used social media mostly for connecting with friends and family. interestingly, instagram and blog were used by a smaller group of older adults. age social medial platform used facebook youtube line chat instagram blog others 60 65 years old (253 persons) 171 107 183 27 3 4 66 70 years old (53 persons) 40 22 43 4 1 1 70 75 years old (26 persons) 17 9 17 0 1 0 76 80 years old (5 persons) 3 0 3 0 0 0 total 231 138 246 31 5 5 table 6. social media platform used by different age groups of the older adults 4.1 measurement model the measurement model has one reflective construct (overall life satisfaction) and ten formative constructs (satisfaction from social media use) in ten life domains. domain life satisfaction is measured by a single item asking respondents to evaluate their domain life satisfaction in ten corresponding domains. formative indicators uniquely contribute to the latent construct; thus, they do not necessarily need to covary. since formative indicators do not need to be correlated, it is not appropriate to conduct the conventional construct life domain social media use never use mean s.d. use social media to share stories, pictures and video clips such as funny gags and classical music 4.5% 5.41 1.65 use social media to post pictures of the places i travelled to 6.2% 5.57 1.80 use social media to share sermons with others 13.1% 4.73 2.19 use social media to build my self-image showing that i am a part of a digital society 8.6% 5.03 1.96 use social media to share my life stories or other interesting stories with people who follow my channels 13.1% 4.75 2.18 leisure use social media to manage loneliness 2.1% 5.86 1.46 use social media for entertainment purposes (movie, music, game and novel) 3.3% 5.48 1.68 use social media to follow celebrities 7.4% 4.88 1.87 use social media to find information about interesting hobbies 3.6% 5.36 1.65 use social media to organize trips (accommodations, restaurants and travel routes) 5.0% 5.18 1.71 use social media to find interesting pictures 8.9% 5.07 2.05 social use social media to find new friends 12.5% 4.63 2.17 use social media to join different groups to share experiences 7.1% 5.14 1.78 use social media to join different groups to chat, socialize with others who share similar interests, such as politics 9.2% 4.89 1.98 australasian journal of information systems ractham et al. 2022, vol 26, research article old but not out 16 consistency assessment that relies on common factor analysis (petter et al., 2007). we followed the procedure suggested by petter et al. (2007) and cenfetelli and bassellier (2009) to assess formative measurement by examining multicollinearity among the indicators, the cooccurrence of negative and positive indicator weights, as well as absolute and relative indicator contributions through their loadings and weights. the lowest and highest variance inflation factor (vif) are 1.05 and 2.81, which are below the suggested cut-off of 3.3 (petter et al., 2007). therefore, we conclude that multicollinearity is not an issue among formative indicators. a few constructs (community, health, consumer, edu, social) have co-occurrence of positive and negative indicator weights, which can lead to a misinterpretation of the results. since these weight indicators are not significant, we decided to remove them. the self and leisure constructs have eight and six indicators, respectively. a higher number of indicators may increase the chance of low or non-significant weights. following cenfetelli and bassellier's (2009) suggestion, we tested different groupings of indicators. we found that grouping the self indicators into two constructs; one construct consisting of indicators associated with self-presentation (self1 and self8) and the other consisting of the rest of the indicators associated with self-actualization, resulted in positive weights with a few nonsignificant ones. since these indicators have significant weights as well as high and significant loadings, we retained them in the structural model estimation. similarly, we found that grouping the leisure indicators into two constructs, one construct consisting of indicators associated with relaxation (leisure1, leisure4, and leisure6) and the other construct consisting of indicators associated with entertainment (leisure2, leisure3, and leisure5) resulted in positive weights with a few non-significant ones. since these indicators have high and significant loadings, we retained them in the structural model estimation. table a1 in the appendix shows the formative measurement assessment. the overall life satisfaction is the only reflective construct with multiple indicators in our model. the reliability was evaluated by examining composite reliability (cr). the value of cr (0.92) is higher than the acceptable value of 0.70, suggesting that reliability is not an issue. convergent validity was evaluated by examining loadings and average variance extracted (ave). the highest loading was 0.87, and the lowest loading was 0.74, which are higher than the acceptable level of 0.60. the value of ave is 0.67, higher than the acceptable value of 0.50. see table a2 in the appendix. therefore, we conclude that the scale has acceptable convergent validity. overall, the measurement model demonstrates acceptable validity and reliability. data collection from self-report surveys is susceptible to common method bias (podsakoff et al., 2003). we used two techniques to check potential biases from common method variance. first, we conducted harman's single-factor test using exploratory factor analysis. common method bias exists if a single factor is identified from the unrotated factor solution, and the first factor explains the majority of the variance in the variables. in our unrotated factor analysis results, the eigenvalues of seven factors exceed 1.0; the first factor accounted for 41% of the variance, and the seven factors together accounted for 67% of the variance. these results suggest that common method bias was not likely to be an issue. second, we used a marker variable technique (lindell and whitney, 2001) by adding marital status as a marker variable. we found that correlations between the study's principal constructs and marital status are from -0.14 and 0.07, which are low and not significant. we conclude that common method bias was not a major concern for this study. australasian journal of information systems ractham et al. 2022, vol 26, research article old but not out 17 4.2 structural model we evaluated the relationship between social media use and life satisfaction by estimating the influence of satisfaction from social media use on domain life satisfaction and the influence of domain life satisfaction on life satisfaction. table 7 presents results from the full model (model 1), the reduced model (model 2) that removes the nonsignificant relationships between social media use related to self-presentation (selff1) and self domain satisfaction (selfsat), the reduced model (model 3) that removes non-significant life domains (i.e., friend, self, and social), and the reduced model with a control variable that is statistically significant (model 4). model 1 model 2 model 3 model 4 path r2 path r2 path r2 path r2 social media use satisfaction domain satisfaction family  familysat 0.38*** 0.15 0.38*** 0.15 0.38*** 0.15 0.38*** 0.15 friend  friendsat 0.50*** 0.25 0.50*** 0.25 community  communitysat 0.49*** 0.24 0.49*** 0.24 0.49*** 0.24 0.49*** 0.24 health  healthsat 0.35*** 0.13 0.35*** 0.13 0.35*** 0.13 0.35*** 0.13 consumer  consumersat 0.48*** 0.23 0.48*** 0.23 0.48*** 0.23 0.48*** 0.23 edu  edusat 0.41*** 0.17 0.41*** 0.17 0.41*** 0.17 0.41*** 0.17 selff1  selfsat -0.10 0.17 0.17 selff2  selfsat 0.48*** 0.40*** leisuref1  leisuresat 0.58*** 0.22 0.58*** 0.22 0.58*** 0.22 0.58*** 0.22 leisuref2  leisuersat -0.16 # -0.16 # -0.16 # -0.16 # social  socialsat 0.28*** 0.08 0.28*** 0.08 domain satisfaction life satisfaction familysat  lifesat 0.23*** 0.56 0.23*** 0.56 0.26*** 0.55 0.28*** 0.56 friendsat  lifesat -0.05 -0.05 communitysat  lifesat 0.15** 0.15** 0.18*** 0.18*** healthsat lifesat 0.11* 0.11* 0.13** 0.12** consumersat  lifesat 0.18*** 0.18*** 0.18** 0.15** edusat lifesat 0.11# 0.11# 0.11# 0.10 selfsat  lifesat 0.09 0.09 leisuresat  lifesat 0.09 # 0.09# 0.12* 0.11* socialsat  lifesat 0.07 0.07 control variable daily social media use 0.09* # p<.10, * p<0.05, ** p<0.01, *** p<0.001 table 7. social media use and influence on life satisfaction australasian journal of information systems ractham et al. 2022, vol 26, research article old but not out 18 figure 2. the relationships between social media use and life satisfaction figure 2 depicts the structural model and path coefficients. the estimated coefficients are the same for models 1 and 2. based on these results, the model explains 56% of the variance in life satisfaction. path coefficients for satisfaction from social media use have significant positive effects (p < .001) on domain life satisfaction across all life domains with the exception of two life domains. the effect of satisfaction from social media use for entertainment (leisuref2) on leisure domain satisfaction (leisuresat) is negative and weakly significant (β = -0.16, p < .10) and the effect of satisfaction from social media use for self-presentation (selff1) on self domain satisfaction (selfsat) is negative and non-significant (β = -0.10, p = ns). when examining the estimated path coefficients, we observed the strong influence of satisfaction from social media use on domain life satisfaction in five domains: social media use for relaxation (leisuref1) on leisure domain satisfaction (leisuresat) (β = 0.58, p < .001), social media use for friendship (friend) on friend domain satisfaction (friendsat) (β = 0.50, p < .001), social media use for community purposes (community) on community domain satisfaction (communitysat) (β = 0.49, p < .001), social media use for product and service consumption (consumer) on consumer domain satisfaction (consumersat) (β = 0.48, p < .001), and social media use for selfactualization (selff2) on self domain satisfaction (selfsat) (β = 0.48, p < .001). we observed the family friend community health consumer edu selff1 selff2 leisuref1 leisuref2 social family sat friend sat community sat health sat consumer sat edu sat self sat leisure sat social sat life sat 0.38*** 0.50 *** 0.49 *** 0.35 *** 0.48 *** 0.41 *** -0.10 0.48 *** 0.58 *** 0.16 # 0.28 *** 0.23 *** -0.05 0.15 ** 0.11 * 0.18 *** 0.11 # 0.09 0.07 0.09 # # p<.10, * p<0.05, ** p<0.01, *** p<0.001 australasian journal of information systems ractham et al. 2022, vol 26, research article old but not out 19 moderate effects of satisfaction from social media use for learning (edu) on education domain satisfaction (edusat) (β = 0.41, p < .001), social media use to connect with family (family) on family domain satisfaction (familysat) (β = 0.38, p < .001), social media use for health purposes (health) on health domain satisfaction (healthsat) (β = 0.35, p < .001), and social media use for socialization (social) on social domain satisfaction (socialsat) (β = 0.28, p < .001). the effects of domain life satisfaction on the overall life satisfaction (lifesat) are positive except in the friend domain (friendsat), which is negative and not significant. in addition, the effects of self domain satisfaction (selfsat) and social domain satisfaction (socialsat) are not significant. we removed these non-significant relationships and reported the estimated paths from the reduced model (model 3). we found that family domain satisfaction (familysat) (β = 0.26, p < .001) and community domain satisfaction (communitysat) (β = 0.18, p < .001) have strongly significant effects on life satisfaction. health domain satisfaction (healthsat) (β = 0.13, p < .01) and consumer domain satisfaction (consumersat) (β = 0.18, p < .01) have moderately significant effects on life satisfaction. leisure domain satisfaction (β = 0.12, p < .05) and education domain satisfaction (β = 0.11, p < .10) have weakly significant effects on life satisfaction. we tested a number of control variables, including demographic characteristics (age, gender, education, income, marital status), prior social media experience, types of social media applications, and daily social media use duration. we found that only social media use duration has a weakly significant effect in the model (β = 0.09, p < .05), suggesting that those who spend more time on social media reap more benefits in satisfaction with their lives. 5 discussion and conclusion this study develops a comprehensive account of older adults' social media use and evaluates the relationship between satisfaction from social media use and life satisfaction. our findings suggest that social media technology plays an integral role in older adults' lives in multiple life domains. the use of social media has a positive influence on their domain life satisfaction in the family, friend, community, health, consumer, education, self through self-actualization activities, leisure, and social life domains. among these life domains, family, community, health, consumer, and leisure are identified as salient domains that contribute positively to older adults' life satisfaction in the context of social media use. although social media have been used mostly by adolescents and young adults to find new friends, older adults voice their concern about using social media to establish new relationships due to the lack of trust and safety concerns. this is in line with arguments in prior research that older adults have a strong emotional investment with family members and established friends (chen and chan, 2011; hur, 2016). in addition, trust is an important factor affecting information sharing and new relationship development in an online setting, particularly for older adults (choi et al., 2014). we believe that discussing older adults' vis-à-vis teens and young adults' social media use gives the insight to understand older adults' motivations to engage with social media. previous studies reported that teens and young adults use social media mainly for social relationships, entertainment, and self-expression (boyd, 2014; kross et al., 2013; oh et al., 2014; trepte and reinecke, 2013; wang et al., 2014). we confirm the general observation in the technology and older adults literature that the needs of older adults as technology users are different from those of young users (chen and chan, 2011; hur, 2016). our results show that older adults engage with social media for a broader range of motivations: to maintain relationships with australasian journal of information systems ractham et al. 2022, vol 26, research article old but not out 20 family and friends, to stay connected and reduce loneliness, to learn new things to keep their mind active and sharp, to share information and connect with others in the community, to seek health-related information, and to engage in activities that promote self-actualization. in contrast to teens and young adults, older adults do not appear to use social media for selfpresentation and entertainment (e.g., following celebrities) to satisfy their self and leisure life domains, respectively. the family, community, health, consumer and leisure for relaxation are salient life domains that contribute to older adults' overall life satisfaction through social media use. overall, our findings suggest that social media technology has an important role in enhancing older adults' subjective well-being in a digitally enabled society. 5.1 implications for research this study develops a substantive theory that offers a comprehensive portrayal of older adults’ social media use in multiple life domains and explains the linkage between satisfaction from social media use and life satisfaction through domain life satisfaction. the exposition of social media use at an activity level in relation to life domains provides deeper insights to understand the motivations and purposes constructed by older adults to "support typical routines across the multiple contexts of his or her life" (scheepers and middleton, 2013, p. 382). compared with the amount of time spent on social media or the functionality approach that prior research employs to theorize technology use, the fine-grained account of social media use enables us to draw a more accurate inference of the meanings and implications of social media use on older adults' lives. the choice of domain life satisfaction and overall life satisfaction to evaluate the impacts of social media use is pertinent not only for social media but can be used to evaluate the impacts of new technologies that individuals exercise their choice and use their own frames of reference to integrate these technologies into their lives that fit their needs. this research contributes to the technology use and subjective well-being literature in general and technology use and older adults and social media in particular. since older adults are usually late adopters of social media, little is known about their social media use and impacts. this research extends the previous conjecture that individuals who use more functionalities of social media are likely to experience more enjoyment (turel and serenko, 2012). our theoretical framework and findings suggest that we need to look beyond the functionalities of social media to examine the context of social media use in order to develop a precise understanding of user engagement with the technology and the impact of social media use on their lives. in contrast to kross et al. (2013), who reported that social media use undermines young adults' subjective well-being, we found that satisfactory experiences from social media use in several life domains are important to promote older adults' subjective well-being. our findings are in line with lelkes (2013) who reported that older adults who use the internet regularly are less likely to experience social isolation and regular internet use positively relates to life satisfaction. this study contributes to advance subjective well-being research. as people across age groups make ict an integral part of their lives, there is a pressing need to understand whether and to what extent ict use contributes positively or negatively to individuals' subjective well-being. the results of this study suggest that technology in general and social media, in particular, has an important role to play in enhancing older adults' domain life satisfaction and overall life satisfaction. from a methodological perspective, this research contributes to is use and impacts research. the domain of life framework and the mixed methods approach can be applied to study australasian journal of information systems ractham et al. 2022, vol 26, research article old but not out 21 individuals' use of social media as well as other malleable technologies. a few studies have employed a similar framework and design to investigate mobile data services use (choi et al., 2007) and productivity software and internet use (techatassanasoontorn and tanvisuth, 2010). we encourage is researchers to use a mixed-methods approach to provide rich insights into technology use in which users express their choices to use these technologies to enhance their lives (scheepers and middleton, 2013). 5.2 implications for practice as the aging society is fast approaching and the use of social media is becoming part of everyday life, our work has shown how social media can be used to support older adults in their daily needs and improve their overall life satisfaction. social activities are important to nurture older adults' well-being (choi et al., 2014; dolan et al., 2008). in particular, our findings suggest that social media offers an alternative platform for older adults to keep up with friends, family, community and engage in volunteering activities. it will be helpful if social media platforms take older adults’ preferences of social media use into consideration and allow them to customize the interface and interactions to better support their specific needs. in addition, younger generations (e.g., children or grandchildren) may use social media to enhance relationships with older adults in their families, such as participating in their groups, communicating with them on their facebook wall, and sharing photos with them. these relatively simple activities have the potential to improve older adults' life satisfaction. governments can also use social media as one of the communication channels to share information with older adults. for example, the hong kong social welfare agency uses social media to enhance older adult's well-being. their goal is to prevent societal issues such as loneliness, abandonment and to improve older adults' healthcare management. 5.3 limitations and future research this research is subject to a few limitations that offer avenues for future research. we studied older adults' social media use and its impacts in a single country context where data may be predisposed toward one specific culture. thus, generalizing this study's findings to social media use and life satisfaction of older adults in other countries, whose societies are more individualist and less concerned about family relationships, should be done with caution. in addition, thailand is one of the countries with a relatively high level of social media adoption across various age groups. future research may consider conducting a comparative study across multiple countries while providing a rich description of society’s extent of social media use to help with an interpretation of the results. the findings shown in this study were a result of a cross-sectional evaluation of satisfaction with social media use and life satisfaction. thus, limitations on a causal interpretation and implication should be noted. longitudinal studies to examine how life satisfaction changes over time may add some new understanding of the effectiveness of social media use on life satisfaction. aging is a complex process, and this study only scratches the surface of the role of technology in older adults' lives by focusing on chronological age as one dimension of aging. however, chronological age may not fully capture biological, functional, psychological and social dimensions of aging that may impact different individuals in different ways (chen and chan, 2011). some of these age-related factors are functional loss in visual and auditory perception, mobility problems, cognition decline, social isolation, fear of illness, and death, among others. future research may want to expand the conceptualization of aging to broaden our understanding of the relationship between aging and technology use and impact. australasian journal of information systems ractham et al. 2022, vol 26, research article old but not out 22 references andrews, f.m., and withey, s.b. (1976). social indicators of well-being: american's perceptions of life quality. plenum press, new york, ny. boyd, d. (2014), it's complicated: the social lives of networked teens, yale university press, new haven, ct. campbell, a., converse, p.e., and rodgers, w.l. (1976). the quality of american life: perceptions, evaluations, and satisfactions. russel sage foundation, new york, ny. celik, s. s., celik, y., hikmet, n., and khan, m. m. (2018). factors affecting life satisfaction of older adults in turkey. the international journal of aging and human development, 87(4), 392-414. https://doi.org/10.1177/0091415017740677 cenfetelli, r.t., and bassellier, g. (2009). interpretation of formative measurement in information systems research. mis quarterly, 33(4), 689-707. https://doi.org/10.2307/ 20650323 chang, j.c., and king, w.r. (2005). measuring the performance of information systems: a functional scorecard. journal of management information systems, 22(1), 85-115. https://www.jstor.org/stable/40398759 charness, n., and boot, w.r. (2009). aging and information technology use. current directions in psychological science, 18(5), 253-258. https://doi.org/10.1111%2fj.1467-8721.2009. 01647.x chen, k., and chan, a. (2011). a review of technology acceptance by older adults. gerontechnology, 10(1), 1-12. https://doi.org/10.4017/gt.2011.10.01.006.00 choi, h., lee, m., im, k.s., and kim, j. (2007). contribution to quality of life: a new outcome variable for mobile data service. journal of the association for information systems, 8(12), 598-618. https://doi.org/10.17705/1jais.00146 choi, j.h., kim, s., moon, j.y., kang, j., lee, i., and kim, j. (2014). seek or provide: comparative effects of online information sharing on seniors' quality of life. communications of the association for information systems, 34, 513-530. https://doi.org/10.17705/1cais.03427 chou, w.s., hunt, y.m., beckjord, e.b., moser, r.p., and hesse, b.w. (2009). social media use in the united states: implications for health communication. journal of medical internet research, 11(4), e48. https://dx.doi.org/10.2196%2fjmir.1249 cogburn, d.l., and espinoza-vasquez, f.k. (2011). from networked nominee to networked nation: examining the impact of web 2.0 and social media on political participation and civic engagement in the 2008 obama campaign. journal of political marketing, 10(102), 189-213. https://doi.org/10.1080/15377857.2011.540224 croll, e.j. (2006). the intergenerational contract in the changing asian family. oxford development studies, 34(4), 473-491. https://doi.org/10.1080/13600810601045833 diamantopoulos, a. (2011). incorporating formative measures into covariance-based structural equation models. mis quarterly, 35(2), 335-358. https://doi.org/10.2307/ 23044046 diener, e. (1984). subjective well-being. psychological bulletin, 95(3), 542-575. https://ssrn.com/ abstract=2162125 diener, e., and diener, m. (1995). cross-cultural correlates of life satisfaction and self-esteem. journal of personality and social psychology, 68(4), 653-663. https://psycnet.apa.org/ doi/10.1037/0022-3514.68.4.653 australasian journal of information systems ractham et al. 2022, vol 26, research article old but not out 23 diener, e., emmons, r.a., larsen, r.j., and griffin, s. (1985). the satisfaction with life scale. journal of personality assessment, 49(1), 71-75. https://doi.org/10.1207/ s15327752jpa4901_13 diener, e., inglehart, r., and tay, l. (2013). theory and validity of life satisfaction scales. social indicators research, 112(3), 497-527. https://doi.org/10.1007/s11205-012-0076-y diener, e., suh, e.m., lucas, r.e., and smith, h.l. (1999). subjective well-being: three decades of progress. psychological bulletin, 125(2), 276-302. https://psycnet.apa.org/doi/10.1037/ 0033-2909.125.2.276 dolan, p., peasgood, t., and white, m. (2008). do we really know what makes us happy? a review of the economic literature on the factors associated with subjective well-being. journal of economic psychology, 29(1), 94-122. https://doi.org/10.1016/j.joep.2007.09.001 easterlin, r.a. (2006). life cycle happiness and its sources: intersections of psychology, economics, and demography. journal of economic psychology, 27(4), 463-482. https://doi.org/10.1016/j.joep.2006.05.002 etda (2020) thailand internet user behavior 2020. available: https://www.etda.or.th/getattachment/c5835c06-e238-4cda-9816814df31caca5/iub_2020 _web.pdf.aspx (accessed june 10 2021). gaia, a., sala, e., and cerati, c. (2021) social networking sites use and life satisfaction: a quantitative study on older people living in europe. european societies, 23(1), 98-118. https://doi.org/10.1080/14616696.2020.1762910. gatto, s.l., and tak, s.h. (2008). computer, internet, and e-mail use among older adults: benefits and barriers. educational gerontology, 34(9), 800-811. https://doi.org/10.1080/ 03601270802243697 gerow, j.e., thatcher, j.s., and grover, v. (2015). six types of it-business strategic alignment: an investigation of the constructs and their measurement. european journal of information systems, 24(5), 465-491. https://doi.org/10.1057/ejis.2014.6 glaser, b.g., and strauss, a.l. (1967). the discovery of grounded theory: strategies for qualitative research, aldine publishing company, chicago, il. hage, e., wortmann, h., van offenbeek, m., and boonstra, a. (2016). the dual impact of online communication on older adults' social connectivity. information technology & people, 29(1), 31-50. https://doi.org/10.1108/itp-09-2014-0216 hair, j.f., hult, g.t.m., ringle, c., and sarstedt, m. (2013). a primer on partial least squares structural equation modeling (pls-sem), sage publications, thousand oaks, ca. headey, b., veenhoven, r., and wearing, a. (1991). top-down versus bottom-up theories of subjective well-being. social indicators research, 24(1), 81-100. https://doi.org/10.1007/ bf00292652 hill, r., beynon-davies, p., and williams, m.d. (2008). older people and internet engagement: acknowledging social moderators of internet adoption, access and use. information technology & people, 21(3), 244-266. https://doi.org/10.1108/09593840810896019 hur, m.h. (2016). empowering the elderly population through ict-based activities. information technology & people, 29(2), 318-333. https://doi.org/10.1108/itp-03-2015-0052 jung, y. (2014). what a smartphone is to me: understanding user values in using smartphones. information systems journal, 24(4), 299-321. https://doi.org/10.1111/isj.12031 kahneman, d., and krueger, a.b. (2006). developments in the measurement of subjective wellbeing. journal of economic perspectives, 20(1), 3-24. https://doi.org/10.1257/ 089533006776526030 australasian journal of information systems ractham et al. 2022, vol 26, research article old but not out 24 kapoor, k.k., tamilmani, k., rana, n.p., patil, p., dwivedi, y.k., and nerur, s. (2018). advances in social media research: past, present and future. information systems frontier, 20(3), 531-558. https://doi.org/10.1007/s10796-017-9810-y khoo, s.s., and yang, h. (2020). social media use improves executive functions in middle-aged and older adults: a structural equation modeling analysis. computers in human behavior, 111, 1-10. https://doi.org/10.1016/j.chb.2020.106388 kim, c., and shen, c. (2020). connecting activities on social network sites and life satisfaction: a comparison of older and younger users. computers in human behavior, 105, 1-8. https://doi.org/10.1016/j.chb.2019.106222 kross, e., verduyn, p., demiralp, e., park, j., lee, d.s., lin, n., shablack, h., jonides, j., and ybarra, o. (2013). facebook use predicts declines in subjective well-being in young adults. plos one, 8(8). https://doi.org/10.1371/journal.pone.0069841 lam, j.c.y., and lee, m.k.o. (2006). digital inclusiveness--longitudinal study of internet adoption by older adults. journal of management information systems, 22(4), 177-206. leist, a. k. (2013). social media use of older adults: a mini-review. gerontology, 59, 378-384. https://www.jstor.org/stable/40398817 lelkes, o. (2013). happier and less isolated: internet use in old age. journal of poverty and social justice, 21(1), 33-46. https://doi.org/10.1332/175982713x664047 limayem, m., gabriele hirt, s., and cheung, c.m.k. (2007). how habit limits the predictive power of intention: the case of information systems continuance. mis quarterly, 31(4), 705-737. https://doi.org/10.2307/25148817 lindell, m.k., and whitney, d.j. (2001). accounting for common method variance in crosssectional research designs. journal of applied psychology, 86(1), 114-121. https://doi.org/10.1037/0021-9010.86.1.114 mitzner, t.l., boron, j.b., fausset, c.b., adams, a.e., charness, n., czaja, s.j., dijkstra, k., fisk, a.d., rogers, w.a., and sharit, j. (2010). older adults talk technology: technology usage and attitudes. computers in human behavior, 26(6), 1710-1721. https://doi.org/10.1016/j.chb.2010.06.020 moore, g.c., and benbasat, i. (1991). development of an instrument to measure the perceptions of adopting an information technology innovation. information systems research, 2(3), 192-222. https://doi.org/10.1287/isre.2.3.192 niehaves, b., and plattfaut, r. (2014). internet adoption by the elderly: employing is technology acceptance theories for understanding the age-related digital divide. european journal of information systems, 23(6), 708-726. https://doi.org/10.1057/ejis.2013.19 oh, h.j., ozkaya, e., and larose, r. (2014). how does online social networking enhance life satisfaction? the relationships among online supportive interaction, affect, perceived social support, sense of community, and life satisfaction. computers in human behavior, 30(1), 69-78. https://doi.org/10.1016/j.chb.2013.07.053 oishi, s., diener, e., lucas, r.e., and suh, e.m. (1999). cross-cultural variations in predictors of life satisfaction: perspectives from needs and values. personality and social psychology bulletin, 25(8), 980-990. https://doi.org/10.1177/01461672992511006 park, n., kee, k.f., and valenzuela, s. (2009). being immersed in social networking environment: facebook groups, uses and gratifications and social outcomes. cyberpsychology and behavior, 12(6), 729-734. https://doi.org/10.1089/cpb.2009.0003 petter, s., straub, d., and rai, a. (2007). specifying formative constructs in information systems research. mis quarterly, 31(4), 623-656. https://doi.org/10.2307/25148814 australasian journal of information systems ractham et al. 2022, vol 26, research article old but not out 25 pew research center (2021). social media fact sheets, available at: https://www.pewresearch.org/internet/fact-sheet/social-media/, accessed 10 june 2021. podsakoff, p.m., mackenzie, s.b., lee, j., and podsakoff, n.p. (2003). common method biases in behavioral research: a critical review of the literature and recommended remedies. journal of applied psychology, 88(5), 879-903. https://doi.org/10.1037/0021-9010.88.5.879 rasmussen, e.e., punyanunt-carter, n., lafreniere, j.r., norman, m.s., and kimball, t.g. (2020). the serially mediated relationship between emerging adults' social media use and mental well-being. computers in human behavior, 102, 206-213. https://doi.org/10.1016/j.chb.2019.08.019 scheepers, r., and middleton, c. (2013). personal ict ensembles and ubiquitous information systems environments: key issues and research implications. communications of the association for information systems, 33, 381-392. https://doi.org/10.17705/1cais.03322 selwyn, n. (2004). the information aged: a qualitative study of older adults' use of information and communications technology. journal of aging studies, 18(4), 369-384. https://doi.org/10.1016/j.jaging.2004.06.008 techatassanasoontorn, a.a., and tanvisuth, a. (2010). is use and quality of life: a conceptualization and empirical investigation. ais transactions on human-computer interaction, 2(2), 26-54. https://doi.org/10.17705/1thci.00013 thailand prd (2020). thailand is preparing for becoming a super-aged society. available: https://thailand.prd.go.th/mobile_detail.php?cid=4&nid=10055 (accessed june 10 2021). thomas, d.m., and watson, r.t. (2002). q-sorting and mis research: a primer. communications of the association for information systems, 8,article 9. https://doi.org/10.17705/1cais.00809 trentham, b., sokoloff, s., tsang, a., & neysmith, s. (2015). social media and senior citizen advocacy: an inclusive tool to resist ageism? politics, groups, and identities, 3(3), 558-571. https://doi.org/10.1080/21565503.2015.1050411 trepte, s., and reinecke, l. (2013). the reciprocal effects of social network site use and the disposition for self-disclosure: a longitudinal study. computers in human behavior, 29(3), 1102-1112. https://doi.org/10.1016/j.chb.2012.10.002 turel, o., and serenko, a. (2012). the benefits and dangers of enjoyment with social networking websites. european journal of information systems, 21(5), 512-528. https://doi.org/10.1057/ejis.2012.1 urquhart, c., lehmann, h., and myers, m.d. (2010). putting the 'theory' back into grounded theory: guidelines for grounded theory studies in information systems. information systems journal, 20(4), 357-381. https://doi.org/10.1111/j.1365-2575.2009.00328.x united nations (2015) world population ageing: 2015, available at: https://www.un.org/en/development/desa/population/publications/pdf/ageing/wpa20 15_highlights.pdf, accessed 10 june 2021. united nations (2017) world population ageing 2017, available at: https://www.un.org/en/development/desa/population/publications/pdf/ageing/wpa20 17_highlights.pdf, accessed 10 june 2021. valenzuela, s., park, n., and kee, k.f. (2009). is there social capital in a social network site?: a facebook use and college students' life satisfaction, trust, and participation. journal of computer-mediated communication, 14(4), 875-901. https://doi.org/10.1111/j.10836101.2009.01474.x australasian journal of information systems ractham et al. 2022, vol 26, research article old but not out 26 van praag, b.m.s., frijters, p., and ferrer-i-carbonell, a. (2003). the anatomy of subjective well-being. journal of economic behavior & organization, 51(1), 29-49. https://doi.org/10.1016/s0167-2681(02)00140-3 venkatesh, v., brown, s.a., and bala, h. (2013). bridging the qualitative-quantitative divide: guidelines for conducting mixed methods research in information systems. mis quarterly, 37(1), 21-54. https://doi.org/10.25300/misq/2013/37.1.02 verdegem, p. (2011). social media for digital and social inclusion: challenges for information society 2.0 research & policies. triplec, 9(1), 28-38. https://doi.org/10.31269/triplec. v9i1.225 wang, j., jackson, l.a., gaskin, j., and wang, h. (2014). the effects of social networking site (sns) use on college students' friendship and well-being. computers in human behavior, 37, 229-236. https://doi.org/10.1016/j.chb.2014.04.051 who (2018) ageing and health, available at https://www.who.int/news-room/factsheets/detail/ageing-and-health, accessed 3 june 2021. xu, c., ryan, s., prybutok, v., and wen, c. (2012). it is not for fun: an examination of social networking site usage. faculty research & creative activity, 7, https://thekeep.eiu.edu/ business_fac/7. xu, x., zhao, y. c., and zhu, q. (2020). the effect of social media use on older adults’ loneliness: the moderating role of self-disclosure. in international conference on humancomputer interaction (pp. 131-145). springer. appendix: measurement model assessment table a1. formative indicators construct item weight t-stat loading t-stat family family1 0.47 2.29* 0.82 8.58*** family2 0.26 2.93** 0.43 3.99*** family3 0.25 1.44 0.54 4.23*** family4 0.44 1.77# 0.84 8.26*** friend friend1 0.22 1.71# 0.74 10.15*** friend2 0.25 1.57 0.77 9.61*** friend3 0.12 0.75 0.69 8.37*** friend4 0.54 3.55*** 0.92 20.81*** friend5 0.10 0.62 0.75 10.46*** community community2 0.14 0.74 0.38 3.91*** community3 0.56 3.46*** 0.89 13.82*** community4 0.22 1.21 0.75 8.17*** community5 0.37 2.28** 0.77 8.69*** health health2 0.47 2.50** 0.79 7.61*** health3 0.61 4.29*** 0.84 9.88*** health4 0.17 0.89 0.70 6.07*** consumer consumer1 0.46 3.71*** 0.80 10.18*** https://www.who.int/news-room/fact-sheets/detail/ageing-and-health https://www.who.int/news-room/fact-sheets/detail/ageing-and-health australasian journal of information systems ractham et al. 2022, vol 26, research article old but not out 27 consumer3 0.35 2.82** 0.64 6.27*** consumer4 0.50 4.47*** 0.82 12.50*** edu edu1 0.44 3.14** 0.77 8.31*** edu3 0.72 6.33*** 0.92 18.28*** selff1 self1 0.40 2.01* 0.69 4.62*** self8 0.78 5.28*** 0.93 12.69*** selff2 self2 0.18 1.24 0.69 7.33*** self3 0.14 0.74 0.59 5.08*** self4 0.41 2.71** 0.80 9.85*** self5 0.22 1.37 0.72 7.34*** self6 0.17 0.85 0.69 7.04*** self7 0.26 1.67# 0.72 7.21*** leisuref1 leisure1 0.19 1.36 0.53 4.50*** leisure4 0.34 2.94** 0.65 6.58*** leisure6 0.74 7.01*** 0.91 17.84*** leisuref2 leisure2 0.24 1.10 0.71 4.49*** leisure3 0.51 2.45* 0.85 7.53*** leisure5 0.48 2.13* 0.82 6.43*** social social2 0.83 4.72*** 0.98 18.31*** social3 0.26 1.09 0.72 4.80*** # p<.10 * p<0.05, ** p<0.01, *** p<0.001 note: work2, community1, health1, consumer1, edu2, and social1 have negative weights. since all of these weights are not significant in our sample, we decide to remove them. the work construct only has two indicators, and the removal of work2 results in a single indicator for this construct. since more than 70% of respondents are retirees, we decided to drop this construct from the model. for the self construct, self1 and self8 have negative weights. for the leisure constructs, leisure2, leisure3, and leisure5 have negative weights. we created two separate formative constructs, selff1 consisting of self1 and self8 and selff2 consisting of the rest of the indicators. similarly, we created two separate formative constructs, leisuref1 consisting of leisure1, leisure4, and leisure6 and leisuref2, consisting of leisure2, leisure3, and leisure5. we reran the measurement model and these self and leisure indicators show positive weights. we retain these indicators for structural model assessment. table a2. reflective indicators item mean std. dev. loading t-stat lifesat1 5.57 1.07 0.81 28.60*** lifesat2 5.69 0.98 0.87 51.46*** lifesat3 5.93 1.00 0.86 40.12*** lifesat4 5.81 1.10 0.87 51.32*** lifesat5 5.38 1.30 0.74 24.51*** # p<.10 * p<0.05, ** p<0.01, *** p<0.001 australasian journal of information systems ractham et al. 2022, vol 26, research article old but not out 28 copyright: © 2022 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v26i0.3269 http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 theoretical background 2.1 older adults and technology 2.2 social media use and influence 2.3 subjective well-being: life satisfaction 2.4 preliminary theoretical model 3 methodology and analysis 3.1 phase 1: development of social media use measures 3.2 phase 2: examining the influence of social media use on older adults' life satisfaction 4 results 4.1 measurement model 4.2 structural model 5 discussion and conclusion 5.1 implications for research 5.2 implications for practice 5.3 limitations and future research australasian journal of information systems doolin et al. 2025, vol 29, research article theorising rpa as socio-technical change received 15 sep 2024, accepted 09 aug 2025 after 2 rounds of review with arif perdana as accepting senior editor, michael davern and stuart black as editors in chief 1 theorising robotic process automation as sociotechnical change: a process study bill doolin1 business school auckland university of technology new zealand email: bill.doolin@aut.ac.nz angsana a. techatassanasoontorn business school auckland university of technology new zealand lena waizenegger business school auckland university of technology new zealand erin wallace-carter business school auckland university of technology new zealand abstract robotic process automation (rpa) is increasingly adopted as a relatively inexpensive automation solution to reduce routinised and repetitive tasks and to initiate an organisation’s broader automation programme. prior research has focused on highlighting rpa benefits for organisations with suggestions on how to maximise benefits and avoid challenges in rpa implementation. there is less understanding of the emergent and dynamic nature of rpa implementation. drawing on key elements of socio-technical change, we conducted a process study of rpa implementation in a university. from our analysis, we identified five process patterns: initiation, mobilisation, configuration, adaptation, and evaluation, each of which has different implications for organisational trajectories of rpa implementation. our findings also offer insights into how the changing role of rpa as an epistemic, technical, and agentic object is intertwined with the dynamics of automation and augmentation in rpa’s conception, development, incorporation into work routines, and evaluation of the initiative’s future. keywords robotic process automation, implementation, process study, socio-technical change, case study. 1 introduction robotic process automation (rpa) is a technology that employs software agents, known as software robots or ‘bots’, to automate low to medium-skilled tasks in a business process according to pre-defined rules by interacting with other systems through their user interfaces. rpa has received attention because of its perception as a minimally invasive and relatively 1 the authors are listed alphabetically. australasian journal of information systems doolin et al. 2025, vol 29, research article theorising rpa as socio-technical change 2 inexpensive solution for the rapid automation of routine tasks involving structured data and deterministic outcomes that were previously reliant on mundane and repetitive manual labour (aguirre and rodriguez 2017; herm et al. 2021; hofmann et al. 2020; syed et al. 2020). increasingly, rpa is also viewed as a way to expand an organisation’s broader automation programme by combining rpa with process mining tools to assist in identifying candidate processes for automation (e.g., leno et al. 2021; van der aalst et al. 2018), or with artificial intelligence (ai) technology to build cognitive or intelligent automation (engel et al. 2022; herm et al. 2021; lacity and willcocks 2021). however, it has been reported that 30-50% of rpa implementations result in failure (ey 2017) and there are suggestions that only a quarter of organisations gain the full benefits from their rpa deployments (asatiani et al. 2023). rpa implementation is often presented as a sequence of clearly defined, deterministic phases with well-contained success factors, and the focus is typically on the early phases of the broader rpa life cycle (asatiani et al. 2023; syed et al. 2020). while prior research on rpa has provided valuable insights, the dynamic nature of rpa implementation is reflected in rpa’s susceptibility to changes in the underlying technological infrastructure or business process, the quality of input data, and willingness of users to adapt their work to accommodate a bot (plattfaut et al. 2022; santos et al. 2020; syed et al. 2020; waizenegger and techatassanasoontorn 2022). in addition, rpa implementation differs from that of more traditional information systems in terms of the nature of the processes supported, integration with other systems, implementation and maintenance costs, flexibility and scalability, and user interaction (coombs et al. 2020; hofmann et al. 2020; staaby et al. 2021; syed et al. 2020). unlike prior rpa research that has focused on static success factors, in this study we unpack the temporal, socio-technical dynamics shaping rpa outcomes. to improve our understanding of the dynamics of rpa implementation and the interrelated changes to technology, structures, work processes and employee roles that shape the course and outcome of rpa implementation, we conceptualise it as a temporally emergent process of socio-technical change (langley et al. 2013; mcleod and doolin 2012). we explore the sociotechnical interactions that occur in the rpa trajectory, which commences with an organisation’s positive imaginaries of rpa technology and its role in automating repetitive work and augmenting human employees (raisch and krakowski 2021), extends to the materialisation and configuration of software robots and their assimilation into work routines, and finally to organisational decisions on the future of the rpa initiative. we use the notion of objects (engeström and blackler 2005) to understand how rpa technology is conceptualised and the multifaceted roles it plays in the evolving journey of an organisation’s rpa initiative. hence, we pose the following research question to underpin our study: how does the organisational rpa journey unfold over time and how is rpa as an object implicated in this trajectory? while the focus of our study is on the full rpa life cycle from inception to development, implementation, post-implementation, and the decision to scale up or down the automation, in the following we use the term rpa implementation in the general sense for the sake of simplicity. our study makes two important contributions to the literature. first, we identify key process patterns that offer a nuanced understanding of how contingent interactions between sociotechnical elements in ongoing practices explain the dynamic and emergent trajectory of rpa implementation. second, drawing on the dual purposes of rpa in automating work processes australasian journal of information systems doolin et al. 2025, vol 29, research article theorising rpa as socio-technical change 3 and augmenting human employees, we explain the changing roles of rpa as epistemic, technical and agentic objects and their implications in the rpa implementation process. in the following, we summarise prior research on rpa implementation before introducing the process approach to socio-technical change used in our study. we then outline our methods of data collection and analysis. drawing on the findings of a case study of an rpa initiative in a university, we develop a process model that traces the dynamic interactions between the rpa technology, human actors and work practices that together shape the key activities that constitute different patterns of rpa implementation. we also explore how rpa evolves as an object of epistemic interest and material artefact or agent in the processes and work practices of the organisation. 2 prior research on rpa we organise insights from the rpa literature into two major streams of studies. the first stream focuses on rpa as an effective automation technology that facilitates process improvements for organisations. the second stream primarily highlights rpa benefits for organisations and potential challenges in the implementation process. the first stream of studies emphasises the appealing technological characteristics of rpa and its potential. rpa shapes work practices by automating repetitive tasks in organisational processes. as a technology, rpa can often overlie existing legacy applications without the need for significant changes to the underlying infrastructure, business logic, or data access layers (hofmann et al. 2020). in addition, as a development platform, rpa with its intuitive user interfaces and low-code environment fosters rapid implementation of software robots (plattfaut and borghoff 2022). the bots can interact with multiple systems and work autonomously to perform routine tasks (kokina and blanchette 2019). while some studies emphasise the capability of the bots to mimic human action and hence access systems through the user interface (syed et al. 2020), the bots can also access and post data to an application through an application programming interface or request tables from a system database through sql queries (hofmann et al. 2020). as the user interface can change through system updates and upgrades, some rpa developers prefer these latter approaches. finally, more recent studies propose an integration of rpa with advanced automation technologies such as ai to expand organisational automation initiatives and potentially enhance the benefits of automation for employees (e.g., benbya et al. 2021; engel et al. 2022; haase et al. 2024; moderno et al. 2024). in the second stream of research, a range of studies present the benefits of rpa for organisations in terms of productivity, speed, cost of data processing, error reduction, process consistency, compliance, and customer satisfaction (aguirre and rodriguez 2017; haase et al. 2024; hofmann et al. 2020; kokina and blanchette 2019; santos et al. 2020; syed et al. 2020). in addition, rpa promises to relieve employees from the repetitive and mundane parts of their jobs, leaving them to do more interesting and high-value activities such as those that require creative thinking, judgment, and social skills (güner et al. 2020; hofmann et al. 2020; lacity and willcocks 2021; santos et al. 2020; van der aalst et al. 2018). other studies argue that rpa is likely to reconfigure work processes by creating new tasks and divisions of labour between bots and the human employees working alongside them (andersson et al. 2022; dey and das 2019; ranerup and henriksen 2019; staaby et al. 2021; waizenegger and techatassanasoontorn 2022). for example, depending on the scope of the bot’s functionality and the underlying australasian journal of information systems doolin et al. 2025, vol 29, research article theorising rpa as socio-technical change 4 infrastructure, the bot will throw exceptions that require human employees to intervene and make judgements or decisions (dey and das 2019; techatassanasoontorn et al. 2023). in addition to highlighting rpa benefits and changes to work processes, some studies have also identified challenges critical to successful rpa implementation, including selecting processes that can deliver the most value after being automated (asatiani et al. 2023; farinha et al. 2024; plattfaut et al. 2022; santos et al. 2020; schmitz et al. 2019), balancing concrete deliverables in the short term with scalable solutions and maintainability in the longer term (lacity and willcocks 2016; santos et al. 2020; viehhauser and doerr 2021), and stakeholder and change management such as managing employee expectations (asatiani and penttinen 2016; dey and das 2019; haase et al. 2024; plattfaut et al. 2022; santos et al. 2020; syed et al. 2020). other reported challenges are integration issues between the bot and the applications it interfaces with (santos et al. 2020), the bots’ access control implications (syed et al. 2020), and the building of internal knowledge and skills on rpa from a vendor or external consultant (hallikainen et al. 2018). while rpa implementation is often presented as a seemingly straightforward process to automate repetitive tasks and augment humans in their work, some studies suggest that rpa implementation is more complex. pre-automation process improvements and selection of processes for automation that interface with stable external applications may be required. the desirability of close collaboration between developers and process owners necessitates the participation of multiple actors in configuring and maintaining bots. indeed, some organisations implementing rpa have challenged the espoused view that process owners who become bot managers do not need technology-related skills. finally, rpa implementation involves interconnected changes to work processes and employee roles that require communication of rpa’s impact and limitations as well as employee training. human-bot collaboration may be needed to realise the benefits of automation and augmentation of work (kokina and blanchette 2019; plattfaut et al. 2022; santos et al. 2020). 3 studying rpa implementation as socio-technical change to develop a deeper understanding of the dynamics of rpa implementation across the rpa life cycle, we adopt a process perspective that examines change as complex, multidimensional, temporally emergent and indeterminate (cloutier and langley 2020; van de ven and poole 2005). process studies analyse patterns in a sequence of events that lead to a particular outcome to open up the ‘black box’ of the process by which change occurs (langley 1999). process studies are concerned with developing knowledge of how and why particular change outcomes unfold in often unpredictable ways over time by focusing on interrelated events, activities, interactions and trajectories (cloutier and langley 2020; fachin and langley 2017; langley et al. 2013). process analysis is built on data that is typically longitudinal, qualitative and collected in actual organisational settings (langley 1999). in-depth interviews with organisation members are one method for soliciting accounts of the events and activities that constitute a trajectory of change (abdallah et al. 2019; fachin and langley 2017). langley (1999) outlines various techniques for analysing process data to conceptualise events and detect patterns between them. a narrative strategy involves constructing a detailed story from the data that conveys the unfolding process and dynamics that have led to the particular outcome being explained. temporal bracketing involves structuring the description of a sequence of events and activities australasian journal of information systems doolin et al. 2025, vol 29, research article theorising rpa as socio-technical change 5 into ‘brackets’ or episodes with some level of internal continuity. these episodes serve as units of analysis for the construction of the process narrative (langley 2023). the ordering of these episodes constructs a narrative temporality for analytical purposes rather than temporal evolution in real time (fachin and langley 2017). visual mapping is a useful complementary technique for analysing and displaying the large amounts of process data as more abstract representations using diagrams (langley and ravasi 2019). the output of the process analysis is a process model that reveals the underlying generative mechanism that shapes the process towards an emergent outcome and enables its interpretation and explanation (cloutier and langley 2020; fachin and langley 2017; pentland 1999). to inform our process analysis, we conceptualise rpa implementation as a process of sociotechnical change (lyytinen and newman 2008; mcleod and doolin 2012; salmimaa et al. 2018). technology implementation in an organisation is a function of dynamic interactions between the technology, the human actors who appropriate it for particular purposes, and the institutional context in which it is deployed (orlikowski 1992). we use a set of five core concepts to selectively focus on the elements of socio-technical change that are important in our analysis: technology, focal actors, practices, context, and effects. technology represents the digital apparatus required to facilitate rpa implementation. this includes the rpa platform and software robots, development tools and relevant elements of the organisation’s technological infrastructure. the decisions and actions involved in implementation are mediated by the properties and capabilities of the technology, which may both enable or constrain action, in turn requiring accommodations such as revisions to the form of technologies, the conduct of human activities, or the plans and goals of actors (pickering 1995). we draw on the notions of epistemic, technical, and agentic objects to explain the changing roles of rpa technology in the implementation process. an epistemic object is “an object of investigation that is in the process of being materially defined” (nicolini et al. 2012, p. 618). epistemic objects are incomplete, emergent and expansive with the capacity to unfold indefinitely (knorr cetina 2008; nicolini et al. 2012). these abstract objects become technical objects when their interpretation is stabilised, and they become defined. they are turned into concrete instruments or tools (at least temporarily) with the static quality of a material object (ewenstein and whyte 2009; rheinberger 2011). objects such as software robots can be viewed as both epistemic objects (in their open forms) and technical objects (in their closed forms). finally, agentic objects are software-based artefacts with the ability to act with “specific rights for task execution and responsibilities for preferred outcomes” (baird and maruping 2021, p. 317). software robots can be considered agentic objects in their instantiation in the automation of routine work practices. furthermore, the treatment of rpa bots as digital employees with their own user accounts, lan ids, and often human-like names, which execute tasks and processes autonomously underlines their agentic nature. in our study, we propose that rpa’s initial status as an open-ended epistemic object becomes progressively defined and (temporarily) stabilised as it materialises in the form of one or more software robots (technical objects), which in turn are integrated into the organisation’s workflows as agentic objects capable of acting with a degree of autonomy. by incorporating the notion of objects, we extend prior socio-technical change models to better capture the evolving role of technology in the change process. australasian journal of information systems doolin et al. 2025, vol 29, research article theorising rpa as socio-technical change 6 focusing on socio-technical change also directs our attention to the role of focal actors (pentland 1999). these are the project participants and stakeholders who shape the course of rpa implementation through their engagement in various project activities and influence in the development of software robots. they include managers, system owners, developers, support staff, vendors, and future business users of rpa. these individuals perform defined roles and possess particular knowledge and skills, expectations and interests, and values and beliefs that shape their perceptions, interpretations, decisions and actions (mcleod and doolin 2012). their actions are oriented toward the attainment of planned goals (leonardi 2015) in the execution of their tasks in rpa implementation. the third key component of the dynamic interactions between actors and technology in sociotechnical change are the theories and routines that provide a framework for action (glaser 2017). we characterise these as practices – recurrent modes of action and knowledge that constitute socially recognised and ordered ways of thinking and performing an activity (gherardi 2009; leonardi 2015; nicolini and monteiro 2016). existing practices discipline human actions by providing a repertoire of concepts, norms, values and procedures embedded in particular bodies of knowledge and institutionalised methods of action (pickering 1995). they legitimate who may act in specific circumstances and particular courses of action (nicolini and monteiro 2016). differing practices can create tensions and conflict, for example, between established and new ways of performing activities or from the introduction of new technologies (nicolini and monteiro 2016). in the context of this research, important practices are likely to include established approaches to project management, solution development and testing, vendor-specific approaches to rpa implementation, and customary ways of performing work in the processes where automation will occur. actors in technology implementation draw on established practices, engage with technology and act within the structures and properties of the context in which this interaction occurs. this includes both the organisational context – e.g. available resources, the formal and informal structure and relations of the organisation, and its culture of institutionalised norms and rules – and aspects of the external socio-political and economic environment in which the organisation is located (lyytinen and newman 2008; mcleod and doolin 2012). these contextual conditions of possibility are interpreted and mobilised by actors to understand, rationalise and legitimate their choices and behaviour (mcleod and doolin 2012). finally, the contingent interactions between actors, technology and practices have intended and unintended effects on the socio-technical change process that shape the trajectory of technology implementation (mcleod and doolin 2012; salmimaa et al. 2018). these effects influence future interactions by acting on or structuring the various elements of ongoing sociotechnical change (feldman and orlikowski 2011). this may include confirmation or change in the scope, objectives and deliverables of the project, actors’ reflective evaluation of their actions resulting in confirmation or change in their positions or intentions, and modification to the technologies used in the project or the introduction of new technologies and artefacts (mcleod and doolin 2012). table 1 presents the definitions of the key concepts on sociotechnical change and objects used to guide our theory development. australasian journal of information systems doolin et al. 2025, vol 29, research article theorising rpa as socio-technical change 7 concept definition technology the digital apparatus required to facilitate rpa implementation (e.g., rpa platform and software robots, development tools, technological infrastructure). epistemic objects abstract objects that are incomplete, emergent and expansive with the capacity to unfold indefinitely. technical objects objects with stabilised and defined interpretations representing concrete tools with the static quality of material objects. agentic objects software-based artefacts with the ability to act with endowed rights to execute responsible tasks. focal actors key actors, including project participants and stakeholders in rpa implementation. practices recurrent modes of action and knowledge that constitute socially recognised and ordered ways of thinking and performing an activity in rpa implementation. context contextual conditions (e.g., resources, structure, culture, norms) that define the range of possible actions available to actors during rpa implementation. effects intended and unintended effects on the socio-technical change process that shape the trajectory of rpa implementation. table 1. definitions of key concepts 4 methods to answer our research question, we conducted a case study of a university that has implemented rpa in its student admission process. we sought and obtained ethical approval for the study from our university’s ethics committee. the case study organisation was selected because it has a relatively complex operating environment with a range of core and support processes that could be automated. in addition, the rpa project studied was the organisation’s first rpa initiative and involved an overseas vendor working with a local it team. our access to the organisation allowed us to study an rpa project through implementation and into the start of the post-implementation, ‘business as usual’, phase. we anticipated that this would help us better understand the impacts of rpa and its autonomous software robots in a work setting as well as the implementation environment. like many universities, the organisation we studied experiences pressure to keep costs low and run their processes as efficiently as possible. rpa was regarded as a potential technology with which to achieve these performance goals, and it was decided to implement an rpa pilot project to evaluate how rpa could be leveraged at the university. 4.1 data collection we followed the rpa deployment at the university for three years from late 2019 to late 2022. our data collection occurred in two phases. the first phase took place from november 2020 to may 2021 and the second from august to november 2022. during phase 1, one of the authors conducted thirteen semi-structured interviews with the it manager as project sponsor; four members of the it project team: the project manager, developer, application support analyst and test analyst; two representatives from the rpa vendor: the relationship manager and solutions architect; and six employees from the process work team: the admissions manager and five admissions officers. interviewing participants from all involved teams allowed a holistic picture and rich accounts of the key events and activities in the rpa implementation process to be captured. the interview guidelines covered: a) description of the interviewee’s current role and responsibility, b) their perception of the rpa implementation and the challenges that occurred, c) the actions and strategies that were taken to facilitate the australasian journal of information systems doolin et al. 2025, vol 29, research article theorising rpa as socio-technical change 8 implementation project and d) the effects on the work practices of the work team employees. the author also observed a project team meeting that showcased the progress of the software robot development and provided the opportunity to see the bot in action. she was provided with copies of the rpa solution design, the vendor’s operating model for rpa deployment, and a plan for scaling rpa use. these documents provided useful context for the conduct of the interviews that were our primary data collection method. during phase 2, some two years after the bot went live, the same author conducted eight semistructured interviews with the project sponsor (now deputy cio), developer, application support analyst (now a solutions architect), admissions manager, three admissions officers who worked alongside the bot, and the product owner, a new participant responsible for further implementation of rpa in the university. the interview guidelines covered: a) changes to job profile since the previous interview and how the rpa deployment had impacted their work, b) the goals of implementing rpa, expectations of rpa performance and how these goals and expectations have been realised, c) perceptions of rpa and experiences managing and working alongside rpa, and d) any plans to expand the university’s rpa programme. table 2 provides details of the interviewees. some participants offered a strategic or managerial perspective, some were knowledgeable about technical details and systems integration, and others provided insights from the user perspective. we obtained the informed consent of our interviewees and maintained participant confidentiality by anonymising quotes and referring to the interviewees using generic job titles. each interview took between 30 and 60 minutes, was conducted in person or via microsoft teams, and was recorded and transcribed. once each interview was completed, key insights were written down and periodically discussed within the author team. participant gender time in position job title (role in implementation) interviews completed 1 female <10 years admissions officer a (subject matter expert) phase 1 and 2 2 female <10 years admissions officer b phase 1 and 2 3 female 10+ years admissions officer c phase 1 4 female 10+ years admissions officer d phase 1 5 female <10 years admissions officer e phase 1 and 2 6 female 10+ years test analyst phase 1 7 male <10 years project manager phase 1 8 male 10+ years admissions manager (process owner) phase 1 and 2 9 male <10 years application support analyst; then solutions architect phase 1 and 2 10 male 10+ years developer phase 1 and 2 11 male <10 years it manager (project sponsor); then deputy cio phase 1 and 2 12 male <10 years solutions architect (vendor representative) phase 1 13 male <10 years relationship manager (vendor representative) phase 1 14 male 10+ years product owner (of rpa programme) phase 2 table 2. summary of interviewees 4.2 data analysis our data analysis consisted of three steps. in the first step, the author who collected the data conducted an initial thematic analysis of the data set (braun and clarke 2006). she read the interview transcripts to familiarise herself with the data and then coded the data inductively australasian journal of information systems doolin et al. 2025, vol 29, research article theorising rpa as socio-technical change 9 using initial codes. conceptually related codes were then grouped into themes, which were reviewed to identify potential relationships between them. alongside the coding and theme development process, theoretical memos describing and specifying each theme were produced and discussed with the rest of the author team. some of the themes that emerged from this phase of the data analysis were drivers of the rpa project, roles, skills and resources issues during the implementation process, and changes of work practices – including an evolving distribution of responsibilities between the software robot and human employees. in the second step, the author team used the results of the initial data analysis to construct a timeline and table of 24 critical events and activities in the rpa implementation. this synopsis included the nature of the event or activity, the key actors involved and their associated actions, and the outcome of those actions on the project. this enabled us to establish the patterns of events within the temporal unfolding of the project (glaser 2017). in the third step of the data analysis, the data set was re-analysed by all authors to develop a temporal narrative of the rpa implementation that focused on analysing the observed events in the rpa implementation as a process of socio-technical change. all authors independently reviewed the interview transcripts and recoded the data, focusing on actions considered analytically important and how these actions were organised temporally (østerlund and carlile 2005), the interactions between key actors, technologies and established practices, how context was implicated, and the consequences of these interactions for shaping the rpa implementation trajectory. throughout this step of the data analysis, the author team had frequent discussions to reflect on the codes used and consider any differences in coding between the team members. this ensured that data was being consistently analysed while leveraging the multiple points of view brought to the analysis by the authors. categories and ideas emerging from this step of the data analysis were then synthesised to create a shared understanding and agreed interpretation of the rpa implementation process among the author team (cornish et al. 2014; crawford et al. 2000). 5 findings based on our data analysis, we explain the emergence and development of rpa in our case study as unfolding in five ‘brackets’ (langley 1999) or episodes, the temporal order of which is analytically important to our explanation. that is, the episodes are analytically sequenced and there may be temporal overlaps in the real time sequence of the rpa implementation programme. we use the five analytical episodes we identified to structure and present our findings in the process narrative below. figure 1 provides a summary of the major milestones in the real time of the rpa deployment. figure 1. rpa project milestones australasian journal of information systems doolin et al. 2025, vol 29, research article theorising rpa as socio-technical change 10 5.1 episode 1 we distinguished the first episode as the activities required to establish the rpa project. this primarily involved identifying rpa as a technology of interest and then defining the project objectives and determining the approach to rpa implementation. the potential for rpa to add value to the university was identified through an established practice of monitoring technological developments: “we sort of monitor the hype curves … and rpa has been there for quite some time” (it manager). the cio and two other senior managers visited several organisations to observe and evaluate rpa programmes and subsequently decided to try “implementing something in the rpa space” (it manager) at their university. from a technology perspective, the senior managers viewed rpa as an established technology with potential yet uncertain benefits to the university: “[rpa] needs to be shown to work in our environment … we don't need to do the proving of the technology” (it manager). figure 2. socio-technical change in episode 1 it was decided to commence with a pilot project as a proof of concept that could be funded from within the it operating budget without obtaining capital funding. a related objective involved developing familiarity with the rpa technology, reflecting a longstanding practice of developing in-house technical knowledge and expertise. the initial return on investment would be secondary to these considerations and the pilot project would be judged a success if “we have learned what bots are, that we have a feel for whether they would work in our environment” (it manager). figure 2 illustrates the salient aspects of the socio-technical interactions in this episode. various activities in the rpa implementation process provide sites for the contextbound interactions between actors, technology and practices that lead to ongoing effects on the rpa trajectory. australasian journal of information systems doolin et al. 2025, vol 29, research article theorising rpa as socio-technical change 11 5.2 episode 2 the second episode concerned the assembling of resources required to commence the rpa project. these included an rpa vendor, a project team, and a candidate process to automate, together with obtaining buy-in from the business unit involved. compatibility with the it team’s way of working was an important consideration in selecting a vendor that could assist with the rpa implementation: “what they were saying resonated with our environment” (it manager). financial constraints and the aim of developing in-house technical expertise meant a collaborative process would be followed: “which was us doing the bulk of the work and they’re just upskilling us” (it manager). the vendor provided two representatives to work with a project team from the university to achieve the desired “capability build” (relationship manager). the university’s project team comprised a project manager, developer, application support analyst (to ensure compatibility of the rpa solution with the university’s operating environment), and test analyst (responsible for quality assurance of the developed rpa solution). the vendor had requested a developer be dedicated to the project full time. however, this was not possible: “they’re thinly spread. so, you can’t just get someone full time” (project manager). the it manager was aware that the team responsible for processing student applications for admission to the university’s academic programmes had several candidate processes that were rule-based, repetitive and high volume. the admissions manager was enthusiastic: “he saw this as a new employee … the bot as someone that can help him out of a tight spot” (solutions architect). the processing of domestic school leaver applications was chosen for the rpa pilot. this process involves running a report in the student management system to identify school leaver applications, performing a series of checks (including accessing an external university entrance system), and generating a letter with a provisional offer of acceptance: “this seemed to be the simplest one to start off with and had the most impact because we are talking thousands of applications potentially that staff members won't need to actually touch” (admissions manager). a software robot would process straightforward applications that involved simple rule-based steps and did not require human interpretation and decision-making skills to assess. this would remove a large volume of mundane work for the admissions team: “just click, click, click and send the offer. there's no thinking about this” (admissions officer a). the admissions manager and admissions officer a were added to the project team (the latter as a subject matter expert) to assist with requirements gathering and user acceptance testing. at this relatively early stage, rpa was conceptualised in ways that aligned with the commonly stated affordances of this automation technology. for example, the solutions architect suggested that efficiency was a primary motivation: “they don't want to get rid of anyone … [they] just want to drive efficiency.” he linked that motivation to the espoused advantages of software robots: “as you know, a bot, if it’s programmed correctly, it will never make a mistake … it just sits there and does the same thing.” the admissions manager also shared this idealised conception of how bots work: “we shouldn’t have a backlog because [the bot] will be working all the time.” the project manager explained the rationale for the project as: “freeing people up who are doing repetitive jobs and allowing them to spend it on doing tasks that carry [higher] value.” automation was envisaged working seamlessly alongside this augmentation of employees’ work: “you should only have to have humans manage the exceptions … from the end user point of view, [the automation] should be almost invisible” (test analyst). australasian journal of information systems doolin et al. 2025, vol 29, research article theorising rpa as socio-technical change 12 the implementation was not expected to cause a large amount of disruption: “we are not anticipating any whole scale workforce changes” (it manager). however, most of the admissions team were not told of the decision to implement rpa until informed in a meeting after the project was underway. this lack of communication caused anxiety and uncertainty among some employees: “what happens when it comes in? will it actually help us or take away our jobs?” (admissions officer b). this was exacerbated when the covid-19 pandemic increased uncertainty around the university’s financial situation and concerns about job security. the admissions manager sent his team a detailed email about the project rationale and how it would affect them. he followed this up with periodic emails keeping the employees informed on progress. those who expressed concerns were generally reassured: “i talked to my manager about it … [and] he was like hoping this rpa will help us … so, i wasn't too worried after all” (admissions officer b). figure 3 illustrates the salient aspects of the socio-technical interactions in this episode. figure 3. socio-technical change in episode 2 5.3 episode 3 the third episode focused on the development of an rpa solution, from process mapping and solution design, through creation of a software robot, to user acceptance testing. the development approach followed in the project was heavily shaped by the vendor’s preferred approach to rpa development and implementation: “we bought their process so that they could train us on how to develop [rpa]” (application support analyst). the development approach created a steep learning curve for the developer. he was supported by the vendor, who provided document templates and advice as needed. the developer communicated with the admissions team to prepare the ‘as-is’ process document and was also responsible for writing the process definition and solution design documents: “as a developer we don’t usually do documentation … that was a bit of a challenge” (developer). he was able to implement some process improvements: “some shortcuts on how they process it … to make sure that process the bot australasian journal of information systems doolin et al. 2025, vol 29, research article theorising rpa as socio-technical change 13 is doing is efficient” (developer). the developer found that developing bots was very different to the programming techniques he usually applied: “there’s a little bit of logic, but it’s not really hard-core development … i was out of my comfort zone.” to upskill himself, he completed a series of rpa training modules and then worked closely with the solutions architect to learn to build the bots from the solution design document. the test analyst participated in their daily online meetings. given her knowledge of the underlying systems, some issues were avoided or resolved as they arose: “the more requirements you get sorted out, the less testing you have to do … it’s built right to start with” (test analyst). to perform its tasks, the bot needed to interact with the university’s student management system (sms), which “wasn’t a great application to automate on” (solutions architect). this introduced a level of complexity to the bot development process and several technical challenges emerged that required the developer to work closely with sms support team. as a non-web-based application, some controls were hidden from the bot or were difficult for it to access. changes were made to the sms software and “implemented in the new version of the software to make the process more efficient for the bot” (solutions architect). for other tasks in the process, the developer leveraged the bot’s technological capabilities to access the backend of the sms database to retrieve data instead of going through the user interface. this sped up the process, and the process flow was changed to remove this inefficient step: “the bot can actually go directly to the database, or it can call an api … [which] a human can’t” (solutions architect). figure 4. socio-technical change in episode 3 a single unattended robot license had been acquired for using the bot in the production environment. however, given the importance of the bot’s integration with the sms, it was considered necessary to test the bot in the sms staging environment before it went into production. this required purchase of a second license, adding to the project cost in already constrained it budget. the test analyst constructed a range of scenarios for which the subject matter expert provided dummy data. the results of the testing were checked by the australasian journal of information systems doolin et al. 2025, vol 29, research article theorising rpa as socio-technical change 14 admissions manager and subject matter expert, and any errors in the configuration files corrected. once user acceptance testing had been completed, the bot went into production. figure 4 summarises the salient aspects of the socio-technical interactions in this episode. 5.4 episode 4 the fourth episode described the process of learning to work with software robot once it was in production, including monitoring its performance and making modifications to it. once the bot went into production, its output was monitored by the admissions team for any errors in its processing. detected problems required technical adjustments to how the bot did its work: “i had to change the configuration” (developer). in addition, changes to the student application process itself after the bot’s initial development required updating the rpa script to ensure the bot could continue working. addressing these issues highlighted the difficulties of testing the bot in the staging environment. the monitoring of the bot also revealed how its performance was dependent on the underlying system: “if sms crashes then obviously the bot can’t continue doing its work, or it times out, or something goes wrong” (admissions manager). high usage of the sms by university staff and students during business hours caused slow response times, which adversely affected the bot’s performance when it was interfacing with the system. the admissions manager suggested at times the bot became so slow that: “[if] one of my staff members processed the application, they would be quicker.” new releases of the sms became a major issue as they sometimes caused the integration between the bot and sms to fail: “[the bot] is expecting this [element], but the element was not there anymore, or its changed” (developer). until the issues were resolved the admissions team needed to resume manual processing of applications. the project team requested advance notification of sms updates to ensure they had time to test the integration in the staging environment. given the periodic releases of sms updates, the test analyst became concerned about the practicalities of maintaining the bot in the post-implementation phase. her preference was to test the bot’s integration with sms upstream in the sms test environment. however, this would require a third robot license, something financial constraints precluded. instead, the team relied on anticipating potential issues in proposed sms releases and catching any that did arise in the staging environment. the main interaction between the bot and the admissions staff involved the management and processing of any cases the bot could not process: “it will come [through] to the exceptions report” (admissions officer a). they needed to check for specific cases that the bot was known to be unable to detect and to manually process such non-standard applications. however, at times the bot presented exception cases for applications that it was programmed to process, causing them to question: “why is this in my workload?” (admissions officer e). once they understood the various scenarios causing the bot’s incorrect processing of these applications, the staff learned how to correct the issue and resubmit to the bot for processing: “[if] you know that is the case, then you just change it. and then i just go, ‘i’ll leave that for the bot to do tomorrow’” (admissions officer e). exception management and structured handovers to the bot represent essential parts of the human-bot collaboration that enabled the bot to complete its work. figure 5 summarises the salient aspects of the socio-technical interactions in this episode. australasian journal of information systems doolin et al. 2025, vol 29, research article theorising rpa as socio-technical change 15 figure 5. socio-technical change in episode 4 5.5 episode 5 the final episode involved post-implementation activities, including assessing the outcome of the initial rpa project and considering the potential for scaling the rpa programme further within the university. despite the time saving achieved being less than anticipated, the it manager felt that the rpa pilot had been successful: “we’ve learned a great deal … we've demonstrated value and the bot itself is adopted and being used.” the admissions manager could see the potential for further enhancements to the bot to enable it to handle more complex application cases. reflecting on how rpa is often marketed as a low-effort automation technology, the solutions architect emphasised the effort involved and the importance of having access to skilled developers: “you still have to do a significant amount of analysis and development to make full use of rpa.” an outcome of the pilot was the design of a multi-tiered support model for rpa within the university: “i think we’ve set that up so that it can be scaled. it was one of the learnings … what is the support model for these bots that we will be creating at the university?” (it manager). however, tracking the bot’s performance proved difficult. the reporting module of the rpa platform came at an additional cost, so the university relied on transaction footprints in the sms, although this lacked detail and accuracy. post implementation, members of the admissions team seemed to accept the bot’s role in automating routine application processing: “i don't really notice it. it’s something that’s happening in the background” (admissions officer e). often, they would refer to the bot as a “colleague” or “someone [that] helps”: “the feeling that you have a new team member who is actually helping you in your daily workload” (admissions officer b). it enabled them to focus on other tasks, such as processing more complex applications: “i would much rather be doing other applications that i have to use my brain and think about it … the bot doing that other work is a good thing” (admissions officer e). some appreciated the opportunity to take on more specialised roles: “when the bot came in, i got to learn other things in other areas” (admissions officer b). at the same time, some australasian journal of information systems doolin et al. 2025, vol 29, research article theorising rpa as socio-technical change 16 looked back on the routine work they no longer did with a degree of nostalgia: “just to give the brain a rest” (admissions officer d). relative costs underlaid a decision to migrate from the existing rpa platform to a platform offered by the provider of the university’s main software ecosystem: “it makes more financial sense for us … we don’t have a lot of money” (product owner). the new platform was considered suitable: “the maturity has improved to the point where [the vendor] would be happy to recommend it” (deputy cio) and had clear advantages: “[all individuals] would be trained on one platform” (deputy cio). reporting functionality was not at an additional cost from this provider and bulk robot licenses could be obtained, providing more freedom over rpa implementation in additional business units. while the original bot would need to be reimplemented with the new platform, this was not anticipated to be onerous, “just a lift and shift of the technology” (deputy cio). as preparation for scaling the rpa programme, the potential for automating two further processes was explored. however, their frequency was deemed insufficient to offset the development time. this highlighted the need for scrutiny of target processes for automation to ensure they represent an adequate return on investment. ultimately, progress on expanding the rpa programme beyond the initial pilot stalled, primarily due to limited capacity within the it team, a lack of capital funding for expansion and changing priorities in the university: “we’ve lost the momentum” (deputy cio). from the vendor’s perspective, this was a missed opportunity: “an rpa programme is only good if you can scale it … if you just put one bot in … then you’re not going to get benefit from your rpa programme” (solutions architect). figure 6 summarises the salient aspects of the socio-technical interactions in this episode. figure 6. socio-technical change in episode 5 taken together, the five episodes present a temporal and contextualised account of the rpa implementation that enables us to better understand the emergent process of socio-technical change shaping the rpa implementation trajectory. by bracketing related events and activities australasian journal of information systems doolin et al. 2025, vol 29, research article theorising rpa as socio-technical change 17 together into a sequence of analytical episodes we produced a structured process narrative that reveals how various context-bound interactions between actors, technology and practices lead to ongoing effects on the trajectory of rpa implementation in our study. in the following section we use our process narrative to conceptualise underlying process patterns, along with the changing role of rpa as an object of interest, to derive a process model that can help interpret and explain the trajectory and outcome of rpa implementation. 6 a process model of rpa implementation in our analysis of the case study, we treat rpa implementation as a process rather than a formula or specification (bygstad et al. 2010). while rpa implementation is often planned and organised in relation to an established operating model or framework, its outcome is at least partially emergent and dependent on a dynamic process of socio-technical change that unfolds over time. as outlined earlier, we frame our understanding of socio-technical change involved in rpa implementation as the effects of contingent interactions between human actors, technological artefacts and work practices that occur during key implementation activities. based on the preceding five episodes identified in our findings, we abstracted five key process patterns occurring in the trajectory of an rpa implementation: initiation, mobilisation, configuration, adaptation and evaluation. they form a sequence of analytically interesting interactions (abdallah et al. 2019) that cluster different activities in the rpa implementation process. the intention is to abstract a conceptualisation of rpa implementation as a dynamic and emergent process that can be generalised across settings while retaining a recognisable connection to our empirical findings (cornelissen 2017; langley and ravasi 2019). while we do not claim that these five patterns are necessarily definitive or exhaustive, we suggest that they are highly likely to influence the development and eventual outcome of rpa implementation within an organisational context. figure 7 presents our process model of rpa implementation. the five process patterns that we abstracted from our findings are shown in their analytical temporal order. three large, curved arrows represent the dynamics of the socio-technical interactions comprising each pattern. to illustrate the analytical flow of patterns in an rpa trajectory, we use two different types of arrows. the solid arrows represent the likely trajectory, while the dashed arrow exiting the evaluation pattern represents the possibility of multiple trajectory outcomes. the changing status of rpa as an object of interest in the rpa implementation process is depicted in italics, moving from a relatively open concept to a progressively closed technical object and agent, before being re-opened as future paths for rpa within the organisation are contemplated. in the remainder of this section, we discuss the implications of these five patterns for rpa implementation. we also discuss the changing role of rpa as an epistemic, technical and agentic object along the rpa trajectory and how this influences the dynamics of rpa implementation. australasian journal of information systems doolin et al. 2025, vol 29, research article theorising rpa as socio-technical change 18 figure 7. a process model of rpa implementation 6.1 initiation initiation refers to the process by which rpa is introduced into an organisational setting. the idea of rpa must be formed, an objective for this form of automation programme established, and an approach to implementation decided. how senior managers in the organisation conceptualise or frame rpa as an applicable technology in the context of their organisation has an important influence on how the rpa implementation is established and structured, at least initially. framing is a form of ideational work (ritvala and kleymann 2012). in performing it, managers construct an opportunity for process automation in relation to their perceived organisational needs, articulate rpa as a solution drawing on knowledge about it available in the wider context and define its importance to the organisation while setting expectations around it. the latter involves establishing an understanding of the role of rpa in the organisation’s work processes and technological infrastructure (bygstad et al. 2010). during initiation, rpa is an epistemic object by virtue of its open-ended nature and what is not yet known (nicolini et al. 2012; rheinberger 2005). at this stage of its implementation, rpa is in the process of being materially defined, characterised by projection rather than being a definitive object (knorr cetina 2008). the open-ended nature and the lack of completeness of rpa generate strong interest and motivation from the focal actors wanting to realise the possibilities of rpa to automate processes in the organisation. australasian journal of information systems doolin et al. 2025, vol 29, research article theorising rpa as socio-technical change 19 6.2 mobilisation mobilisation refers to the process of assembling the human, technological and financial resources required for rpa implementation. resource mobilisation is a form of work that involves finding and securing social, cognitive and material support for an initiative (ritvala and kleymann 2012). it may include, for example, establishing collaborative arrangements that facilitate the rpa implementation process, such as those with rpa consultants or vendors, the creation of a project team, and the identification of a suitable target process for automation (asatiani et al. 2023; farinha et al. 2024). internal and external stakeholders need to be enrolled to engage with the rpa initiative. this may involve creating awareness of the project’s importance, aligning different stakeholder interests and formalising commitments (bygstad et al. 2010). employees need to be socialised to the idea of rpa and educated in the knowledge of how working alongside software robots will impact their practices. financial and technological resources must also be secured, including, for example, funding for the initial implementation and subsequent expansion of the automation initiative and the purchase of sufficient robot licenses to enable a straightforward implementation. as the implementation proceeds towards mobilisation, rpa, as an epistemic object fuelled by its incompleteness, continues to be a source of motivation to keep the project in motion. key activities to select a compatible vendor, assemble a project team, and identify a suitable process help extend rpa’s properties from having unbounded automation possibilities to affording idealised automation capabilities to relieve human employees from high-volume, mundane and rule-based tasks. in turn, these imagined automation capabilities confer rpa with promising augmentation benefits for employees. 6.3 configuration configuration refers to the process of rpa solution development. the technical work involved centres around process mapping and improvement, design of an rpa solution, integration with existing systems, and development and testing of bots before they are put into production. key aspects include defining boundaries and rules (lawrence and suddaby 2006) in relation to the processes and systems within which rpa is to be established, fitting the solution to the organisation (bygstad et al. 2010), and configuring the rpa software to select, integrate and fine tune the functionalities of the bots (brugali and gherardi 2016). while rpa interfaces generally require few programming skills and effort (aguirre and rodriguez 2017), some education and skill development may be required in familiarising employees with potentially novel practices of rpa development and implementation, as well as connecting those practices and associated knowledge bases to established ways of working and organising (lawrence and suddaby 2006). in particular, it is important to develop awareness of the fact that rpa development differs from traditional software development. first, rpa development is tightly linked to business processes (asatiani et al. 2023), thus requiring close collaboration between software developers and key users who understand the process tasks in programming a bot. second, rpa development usually does not require writing code, which many software developers are used to doing. instead, rpa development interfaces involve drag and drop icons to automate a process (hallikainen et al. 2018; lacity and willcocks 2016), which may lead some software developers to question the misalignment of their expertise with the project’s needs, as shown in our case study. therefore, organisations may need to carefully prepare software australasian journal of information systems doolin et al. 2025, vol 29, research article theorising rpa as socio-technical change 20 developers and provide appropriate training to help them develop rpa expertise and cultivate organisational rpa capabilities. during configuration, the underlying activities focus on turning rpa from an epistemic object into a technical object endowed with automation capabilities for a chosen process. rpa’s automation potential takes centre stage in developing bots while its possibilities to augment and benefit human employees stay in the background. during this process, developers need to correctly configure a bot’s automation capabilities by carefully replicating how a human would interact with other computer systems and, in some cases, taking advantage of a bot’s technical capabilities to bypass a user interface and interact directly with a computer system (lacity and willcocks 2016; santos et al. 2020). although previous studies often claim that rpa does not disturb the underlying systems (lacity and willcocks 2016) or require changes to it infrastructure (santos et al. 2020), this may not always be the case. configuring a bot may require technical interventions to make changes to underlying systems the bot interacts with, coordination with these systems’ owners, and an ongoing need for testing. 6.4 adaptation adaptation refers to the process by which the rpa technology and organisational systems and practices are reconfigured and adjusted to fit with each other after the bots are placed in production. bot performances must be managed and maintained as new exceptions, breakdowns and contingencies arise in the flow of organisational activities (orlikowski 1996), and the deployment of the new technology may trigger changes to work processes and practices as employees adapt to working alongside the bots. although human actors plan their actions in the implementation and use of a new technology, any stabilisation of a solution is achieved “through a series of ongoing and situated accommodations, adaptations, and alterations … enacted over time” (orlikowski 1996, p. 66; leonardi 2015). in the adaptation process, the bot, as a technical object with automation capabilities, becomes an agentic object to provide intended augmentation benefits to human employees. the bot completes work tasks autonomously on employees’ behalf by leveraging the existing technological infrastructure similarly to how a human employee would do. if working as intended, such an object is invisible and taken for granted (nicolini et al. 2012). a bot only becomes visible when it breaks down, especially when the underlying work process and the systems it interacts with change (asatiani and penttinen 2016; santos et al. 2020), leaving employees frustrated and creating doubts about the benefits of rpa. turning a bot into a productive agentic object may require additional time and resources that cannot be fully anticipated at the beginning of the project. once the bot as an agentic object has been put into action alongside human employees, automation and augmentation become mutually enabling and constituent of one another through the bot’s automation of high-volume, low-complexity tasks, freeing the human to work on tasks that require judgements. the extent to which augmentative benefits are achieved depends on whether employees are willing and able to adapt their ways of knowing and working to accommodate the bot (faraj et al. 2018). as our study shows, augmentation may require ongoing and often ‘invisible’ work by human actors (baptista et al. 2020) to accommodate the bot’s technological limitations. australasian journal of information systems doolin et al. 2025, vol 29, research article theorising rpa as socio-technical change 21 6.5 evaluation evaluation refers to the process by which an organisation learns from their experiences with the rpa deployed and decides how to proceed with the wider automation programme. it requires reflection and review of the performance of the bots against previously defined goals, although these may have been adjusted as the rpa implementation unfolded over time. managers need to decide whether rpa implementation is “fixed-term or continuous in nature, a stepping stone to large-scale automation or instead a fix for a specific problem” (asatiani et al. 2023, p. 114). various trajectory outcomes are possible as the result of the evaluation process, including expanding and scaling up the rpa programme (requiring further mobilisation of resources), scaling down rpa efforts once the project is completed (with continued maintenance of a one-off solution), or pivoting to a new platform or sourcing model based on some combination of internal and external expertise (asatiani et al. 2023). each requires an evaluation of the benefits to be gained against resource and capacity requirements. by the stage of evaluation, rpa has gone through various projections to unfold its capacity as an epistemic object, with seemingly unbounded automation opportunities during initiation, to acquire the affordances of idealised automation capabilities with augmentation potentialities for employees during mobilisation. in configuration, rpa turns into temporary instantiations of configured and tested bots as technical objects with defined characteristics and capabilities to make rpa ready to be absorbed into the practice of doing work. during adaptation, rpa in the form of a software robot becomes a functional agentic object to augment employees in their work routines, while coexisting as a technical object with wellspecified capabilities to automate routinised work. although the bot as a technical and agentic object becomes taken for granted and considered sufficiently stable to support employees’ work, during evaluation, rpa retains its quality as an epistemic object with its perpetual unfolding capacity and lack of completeness (knorr cetina 2008). this is apparent in the developing understanding of the potentials and realities of scaling automation, and how the organisation’s ‘wants’ prompt the search for new objects to work with (knorr cetina 2008). 7 conclusion 7.1 theoretical contribution reflecting on our research question, we were able to develop an explanation of the emergent process of rpa implementation in a particular organisational setting. by combining a process perspective with a theoretical understanding of socio-technical change, we explained how contingent interactions between technology, human actors and established practices shaped the rpa trajectory. drawing on our process analysis of rpa implementation in the case study, we abstracted five key process patterns—initiation, mobilisation, configuration, adaptation and evaluation—each of which has different implications for rpa and its implementation. together, these five patterns form the basis of the process model we propose for characterising the temporal emergence and evolution of an rpa programme. the process model offers a more analytical explanation of the complex, indeterminate nature of rpa implementation and how multiple potential rpa outcomes are possible across different settings. in addition to highlighting key process patterns that explain the dynamic and emergent trajectory of rpa implementation, our study also offers a better understanding of how the changing role of rpa as an object of interest and the varying emphasis on its dual purposes of automation and augmentation influence the course of rpa implementation. in automation australasian journal of information systems doolin et al. 2025, vol 29, research article theorising rpa as socio-technical change 22 software robots take over seemingly mundane, repetitive tasks from human employees, while augmentation involves collaboration between bots and human employees, the latter handling non-routine exceptions that bots could not process or taking outputs from bots to work on downstream tasks (lacity and willcocks 2016; raisch and krakowski 2021). our analysis reveals intricate interdependencies between automation and augmentation by highlighting their dynamics in the conception, development and incorporation of rpa into work routines. we use the notions of epistemic, technical, and agentic objects to explain the changing roles of rpa and its automation and augmentation purposes in the implementation process. in rpa implementation, actors across various roles work together to endow a software robot with innate automation capabilities waiting to be rendered from a potential automated agent into actuality in a work routine. the focus of rpa implementation is twofold: (1) to turn rpa as an epistemic object into a technical object in the bot development process and then an agentic object when it is put into use alongside human employees in their work routines, and (2) to evolve and further explore rpa as an epistemic object in the pursuit of scaling automation. 7.2 practical implications rpa implementation involves collaboration among diverse actors with different roles and potentially different aspirations for rpa’s immediate and future outcomes. it is important to establish an appropriate organisational structure with dedicated financial and human resources to internalise and consolidate knowledge from vendors and consultants, guide knowledge exchange, coordinate efforts, and manage emergent issues throughout the rpa implementation process in an organisation (e.g. a centre of excellence, willcocks et al. 2019). communication is important, particularly around changes in process flows and the impact of rpa on employees. during rpa initiation and mobilisation, the primary focus is on automation and its strategic value, with relatively less attention on augmentation. employees are often left to imagine for themselves how a bot that has yet to be developed is likely to impact task complexity, work intensity, and job security (riemer and peter 2020). this tension can be exacerbated when employees are not well informed about rpa and how the bot works, increasing their concerns about the impact on their work. ideally, employees impacted by the introduction of bots into their work routines should be engaged soon after initiation and periodically during critical events throughout the implementation (lacity and willcocks 2016). the rpa literature has emphasised the importance of careful selection of processes to be automated (farinha et al. 2024; plattfaut et al. 2022; santos et al. 2020). our study suggests carefully assessing the underlying systems a process interacts with, from both a technical perspective (interface design, frequency of changes) and a governance perspective (ownership of systems and willingness to cooperate), to avoid downstream implications and unwanted outcomes on bot performance. careful consideration of the nature of the rpa development and the required resources may avoid misperceptions or complications in the implementation process. tensions may arise from the availability of particular actors’ roles and time commitments, technical skills and learning required to master rpa development, and collaboration with a vendor (asatiani et al. 2023). some of these tensions are relevant to it projects generally and can be anticipated (plattfaut et al. 2022). rpa is considered relatively easy to configure a bot to automate processes (lacity and willcocks 2016). in our study, the experienced developer assigned to the rpa project saw the work of bot configuration as a downgrade to his programming expertise. australasian journal of information systems doolin et al. 2025, vol 29, research article theorising rpa as socio-technical change 23 however, it is also possible that some developers may view rpa as an opportunity to upgrade their expertise and position themselves as more business relevant. 7.3 limitations and future research a limitation of our research is that we only examined rpa implementation in one organisation. clearly, further case studies would increase confidence in our findings. in particular, comparative case studies in other industries would explore the effects of context in shaping rpa trajectories and outcomes. salient structural or cultural factors within the specific context of the university we studied included limited financial and human resources for rpa implementation, its instigation by the it function (rather than a business unit) as a proof of concept for the university, and shifting organisational priorities in a changing educational environment. nevertheless, the theoretical insights, particularly the conceptual model of rpa implementation as a dynamic and emergent process, offer a useful basis for understanding rpa implementation trajectories more generally. the underlying process patterns that we abstract in our model are transferable, or at least adaptable, across multiple contexts. although we followed the use of rpa over two years after the bot was deployed, we could not further explore the continued use of rpa and scaling of automation to gain deeper insights into emerging changes and long-term implications. finally, questions in the interviews were related to both historical and current events (kimberly and bouchikhi 1995) leading to the possibility that our participants might not be able to recall all the details from the former (lyytinen et al. 2009). we addressed this by asking similar questions to triangulate accounts of key events and activities with multiple participants. our findings suggest that some software developers used to coding in traditional software development may find rpa development experience potentially challenging to their expertise and thus the impact on their professional identity deserves further investigation. for example, future studies may examine potential threats or opportunities software developers experience with rpa development and their differing responses to preserve, strengthen, adjust or expand their occupational identity (vaast and pinsonneault, 2021). other studies could explore the tensions that may occur when developers interact and collaborate with non-technical employees who become involved in rpa development without the software-specific knowledge of regular developers (siemon and kedziora 2023; van den broek et al. 2021). some scholars encourage researchers to pay attention to the tensions between the automation and augmentation of human work (benbya et al. 2021) and the need to focus on process improvements without neglecting employee’s quality of work life (haase et al. 2024; riemer and peter 2020). we encourage future studies to take a human-centred view to critically examine the long-term implications of the implementation of rpa and other automation technologies in organisations. future research could also build on our conceptual model by studying the dynamics and process patterns in further rpa implementation settings or the implementation of other automation technologies, including those with more agency and intelligence, such as conversational agents and ai-powered applications (baird and maruping 2021; farinha et al. 2024; fügener et al. 2022; haase et al. 2024; moderno et al. 2024; seeber et al. 2020; siemon 2022). for example, it may be worthwhile for comparative case studies to explore how the introduction of ai and machine learning as part of intelligent process automation may accelerate the trajectory of rpa implementation and shape the contours of the process patterns identified in our study. another option is to explore how the use of ai-powered tools to australasian journal of information systems doolin et al. 2025, vol 29, research article theorising rpa as socio-technical change 24 support developers shapes the dynamics of the configuration process in rpa implementation. as rpa platforms increasingly integrate ai into various rpa development activities (e.g., process selection, process orchestration), future research could explore how ai as another digital apparatus changes the course of rpa implementation compared with those implementation contexts with no ai involvement. references abdallah, c., lusiani, m., & langley, a. (2019). performing process research. in boyd, b., crook, t. r., lê, j. k., & smith, a. d. (eds.) standing on the shoulders of giants: traditions and innovations in research methodology (pp. 91-113). emerald. aguirre, s., & rodriguez, a. (2017). automation of a business process using robotic process automation (rpa): a case study. in figueroa-garcía, j., lópez-santana, e., villaramírez, j., ferro-escobar, r. (eds.) applied computer sciences in engineering (pp. 65-71). springer. https://doi.org/10.1007/978-3-319-66963-2_7 andersson, c., hallin, a., & ivory, c. (2022). unpacking the digitalisation of public services: configuring work during automation in local government. government information quarterly, 39(1), 101662. https://doi.org/10.1016/j.giq.2021.101662 asatiani, a., copeland, o., & penttinen, e. (2023). deciding on the robotic process automation operating model: a checklist for rpa managers. business horizons, 66(1), 109-121. https://doi.org/10.1016/j.bushor.2022.03.004 asatiani, a., & penttinen, e. (2016). turning robotic process automation into commercial success – case opuscapita. journal of information technology teaching cases, 6(2), 67–74. https://doi.org/10.1057/jittc.2016.5 baird, a., & maruping, l. m. (2021). the next generation of research on is use: a theoretical framework of delegation to and from agentic is artifacts. mis quarterly, 45(1), 315-341. https://doi.org/10.25300/misq/2021/15882 baptista, j., stein, m. k., klein, s., watson-manheim, m. b., & lee, j. (2020). digital work and organisational transformation: emergent digital/human work configurations in modern organisations. journal of strategic information systems, 29(2), 101618. https://doi.org/10.1016/j.jsis.2020.101618 benbya, h., pachidi, s., & jarvenpaa, s. l. (2021). special issue editorial: artificial intelligence in organizations: implications for information systems research. journal of the association for information systems, 22(2), 281-303. https://doi.org/ 10.17705/1jais.00662 braun, v., & clarke, v. (2006). using thematic analysis in psychology. qualitative research in psychology, 3(2), 77–101. https://doi.org/10.1191/1478088706qp063oa brugali, d., & gherardi, l. (2016). hyperflex: a model-driven toolchain for designing and configuring software control systems for autonomous robots. in a. koubaa (ed.), robot operating system (ros) (pp. 509-534). springer. https://doi.org/10.1007/978-3-319-260549_20 bygstad, b., nielsen, p. a., & munkvold, b. e. (2010). four integration patterns: a sociotechnical approach to integration in is development projects. information systems journal, 20(1), 53-80. https://doi.org/10.1111/j.1365-2575.2007.00280.x australasian journal of information systems doolin et al. 2025, vol 29, research article theorising rpa as socio-technical change 25 cloutier, c., & langley, a. (2020). what makes a process theoretical contribution? organization theory, 1(1), 1-32. https://doi.org/10.1177/2631787720902473 coombs, c., hislop, d., taneva, s. k., & barnard, s. (2020). the strategic impacts of intelligent automation for knowledge and service work: an interdisciplinary review. journal of strategic information systems, 29(4), 101600. https://doi.org/10.1016/j.jsis.2020.101600 cornelissen, j. (2017). editor’s comments: developing propositions, a process model, or a typology? addressing the challenges of writing theory without a boilerplate. academy of management review, 42(1), 1-9. https://doi.org/10.5465/amr.2016.0196 cornish, f., gillespie, a., & zittoun, t. (2014). collaborative analysis of qualitative data. in u. flick (ed.) the sage handbook of qualitative data analysis (pp. 79-93). sage. crawford, h. k., leybourne, m. l., & arnott, a. (2000). how we ensured rigour in a multisite, multi-discipline, multi-researcher. forum qualitative sozialforschung / forum: qualitative social research, 1(1), article 12. https://doi.org/10.17169/fqs-1.1.1122 dey, s., & das, a. (2019). robotic process automation: assessment of the technology for transformation of business processes. international journal of business process integration and management, 9(3), 220-230. https://doi.org/10.1504/ijbpim.2019.100927 engel, c., ebel, p., & leimeister, j. m. (2022). cognitive automation. electronic markets, 32, 339350. https://doi.org/10.1007/s12525-021-00519-7 engeström, y., & blackler, f. (2005). on the life of the object. organization, 12(3), 307-330. https://doi.org/10.1177/1350508405051268 ewenstein, b., & whyte, j. (2009). knowledge practices in design: the role of visual representations as ‘epistemic objects’. organization studies, 30(1), 7-30. https://doi.org/10.1177/017084060808301 ey (2017). get ready for robots: why planning makes the difference between success and disappointment. https://assets.ey.com/content/dam/ey-sites/ey-com/en_gl/topics/emeiafinancial-services/ey-get-ready-for-robots.pdf fachin, f. f., & langley, a. (2017). researching organizational concepts processually: the case of identity. in cassell, c., cunliffe, a. l., & grandy, g. (eds.) the sage handbook of qualitative business and management research methods. (pp. 308-327). sage. https://doi.org/10.4135/9781526430212 faraj, s., pachidi, s., & sayegh, k. (2018). working and organizing in the age of the learning algorithm. information and organization, 28(1), 62-70. https://doi.org/10.1016/j.infoandorg.2018.02.005 farinha, d., pereira, r., & almeida, r. (2024). a framework to support robotic process automation. journal of information technology, 39(1), 149-166. https://doi.org/10.1177/02683962231165066 feldman, m. s., & orlikowski, w. j. (2011). theorizing practice and practicing theory. organization science, 22(5), 1240–1253. https://doi.org/10.1287/orsc.1100.0612 fügener, a., grahl, j., gupta, a., & ketter, w. (2022). cognitive challenges in human–artificial intelligence collaboration: investigating the path toward productive delegation. information systems research, 33(2), 678-696. https://doi.org/10.1287/isre.2021.1079 https://doi.org/10.1177/02683962231165066 australasian journal of information systems doolin et al. 2025, vol 29, research article theorising rpa as socio-technical change 26 gherardi, s. (2009). knowing and learning in practice-based studies: an introduction. learning organization, 16(5), 352-359. https://doi.org/10.1108/09696470910974144 glaser, v. l. (2017). design performances: how organizations inscribe artifacts to change routines. academy of management journal, 60(6), 2126-2154. https://doi.org/10.5465/amj.2014.0842 güner, e., han, s., & juell-skielse, g. (2020). robotic process automation as routine capability: a literature review. in proceedings of the 28th european conference on information systems (ecis). https://aisel.aisnet.org/ecis2020_rp/153 haase, j., kremser, w., leopold, h., mendling, j., onnasch, l., & plattfaut, r. (2024). interdisciplinary directions for researching the effects of robotic process automation and large language models on business processes. communications of the association for information systems, 54(1), 579-604. https://doi.org/10.17705/1cais.05421 hallikainen, p., bekkhus, r., & pan, s. (2018). how opuscapita used internal rpa capabilities to offer services to clients. mis quarterly executive, 17(1), 41-52. https://aisel.aisnet.org/misqe/vol17/iss1/4 herm, l. v., janiesch, c., reijers, h. a., & seubert, f. (2021). from symbolic rpa to intelligent rpa: challenges for developing and operating intelligent software robots. in a. polyvyanyy, m. t. wynn, a. van looy, & m. reichert (eds.) business process management (pp. 289-305). springer. https://doi.org/10.1007/978-3-030-85469-0_19 hofmann, p., samp, c., & urbach, n. (2020). robotic process automation. electronic markets, 30(1), 99-106. https://doi.org/10.1007/s12525-019-00365-8 kimberly, j. r., & bouchikhi, h. (1995). the dynamics of organizational development and change: how the past shapes the present and constrains the future. organization science, 6(1), 9-18. https://doi.org/10.1287/orsc.6.1.9 knorr cetina, k. (2008). objectual practice. in m. mazzotti (ed.), knowledge as social order (pp. 1-15). routledge. https://doi.org/10.4324/9781315591148 kokina, j., & blanchette, s. (2019). early evidence of digital labor in accounting: innovation with robotic process automation. international journal of accounting information systems, 35, 100431. https://doi.org/10.1016/j.accinf.2019.100431 lacity, m. c., & willcocks, l. p. (2016). robotic process automation at telefónica o2. mis quarterly executive, 15(1), 21-35. https://aisel.aisnet.org/misqe/vol15/iss1/4 lacity m., & willcocks, l. (2021). becoming strategic with intelligent automation. mis quarterly executive, 20(2), 169-182. https://aisel.aisnet.org/misqe/vol20/iss2/7 langley, a. (1999). strategies for theorizing from process data. academy of management review, 24(4), 691-710. https://doi.org/10.2307/259349 langley, a. (2023). approaches to studying continuity and change. in hernes, t., & feuls, m. (eds.) a research agenda for organisational continuity and change (pp. 225-246). edward elgar. langley, a., & ravasi, d. (2019). visual artifacts as tools for analysis and theorizing. in zilber, t. b., amis, j. m., & mair, j. (eds.) the production of managerial knowledge and australasian journal of information systems doolin et al. 2025, vol 29, research article theorising rpa as socio-technical change 27 organizational theory: new approaches to writing, producing and consuming theory (pp. 173-199). emerald. https://doi.org/10.1108/s0733-558x20190000059010 langley, a., smallman, c., tsoukas, h., & van de ven, a. h. (2013). process studies of change in organization and management: unveiling temporality, activity, and flow. academy of management journal, 56(1), 1-13. https://doi.org/10.5465/amj.2013.4001 lawrence, t. b., & suddaby, r. (2006). institutions and institutional work. in clegg, s. r., hardy, lawrence, t. b., & nord, w. r. (eds.) the sage handbook of organization studies (pp. 215-254). sage. https://doi.org/10.4135/9781848608030.n7 leno, v., polyvyanyy, a., dumas, m., la rosa, m., & maggi, f. m. (2021). robotic process mining: vision and challenges. business & information systems engineering, 63(3), 301-314. https://doi.org/10.1007/s12599-020-00641-4 leonardi, p. m. (2015). studying work practices in organizations: theoretical considerations and empirical guidelines. annals of the international communication association, 39(1), 235273. https://doi.org/10.1080/23808985.2015.11679177 lyytinen, k., & newman, m. (2008). explaining information systems change: a punctuated socio-technical change model. european journal of information systems, 17(6), 589-613. https://doi.org/10.1057/ejis.2008.50 lyytinen, k., newman, m., & al-muharfi, a.-r. a. (2009). institutionalizing enterprise resource planning in the saudi steel industry: a punctuated socio-technical analysis. journal of information technology, 24(4), 286-304. https://doi.org/10.1057/jit.2009.14 mcleod, l., & doolin, b. (2012). information systems development as situated socio-technical change: a process approach. european journal of information systems, 21(2), 176-191. https://doi.org/10.1057/ejis.2011.43 moderno, o., braz, a. c., & nascimento, p. (2024). robotic process automation and artificial intelligence capabilities driving digital strategy: a resource-based view. business process management journal, 30(1), 105-134. https://doi.org/10.17705/1cais.05421 nicolini, d., mengis, j., & swan, j. (2012). understanding the role of objects in crossdisciplinary collaboration. organization science, 23(3), 612-629. https://doi.org/10.1287/orsc.1110.0664 nicolini, d., & monteiro, p. (2016). the practice approach: for a praxeology of organisational and management studies. in langley, a., & tsoukas, h. (eds.) the sage handbook of process organization studies (pp. 110-126). sage. https://doi.org/10.4135/9781473957954 orlikowski, w. j. (1992). the duality of technology: rethinking the concept of technology in organizations. organization science, 3(3), 398-427. https://doi.org/10.1287/orsc.3.3.398 orlikowski, w. j. (1996). improvising organizational transformation over time: a situated change perspective. information systems research, 7(1), 63-92. https://doi.org/10.1287/isre.7.1.63 østerlund, c., & carlile, p. (2005). relations in practice: sorting through practice theories on knowledge sharing in complex organizations. the information society, 21(2), 9-107. https://doi.org/10.1080/01972240590925294 australasian journal of information systems doolin et al. 2025, vol 29, research article theorising rpa as socio-technical change 28 pentland, b. t. (1999). building process theory with narrative: from description to explanation. academy of management review, 24(4), 711-724. https://doi.org/10.2307/259350 pickering, a. (1995). the mangle of practice: time, agency, and science. university of chicago press. https://doi.org/10.2307/3106908 plattfaut, r., & borghoff, v., (2022). robotic process automation: a literature-based research agenda. journal of information systems, 36(2), 173-191. https://doi.org/10.2308/isys-2020033 plattfaut, r., borghoff, v., godefroid, m., koch, j., trampler, m., & coners, a. (2022). the critical success factors for robotic process automation. computers in industry, 138, 103646. https://doi.org/10.1016/j.compind.2022.103646 raisch, s., & krakowski, s. (2021). artificial intelligence and management: the automationaugmentation paradox. academy of management review, 46(1), 192-210. https://doi.org/10.5465/amr.2018.0072 ranerup, a., & henriksen, h. z. (2019). value positions viewed through the lens of automated decision-making: the case of social services. government information quarterly, 36(4), 101377. https://doi.org/10.1016/j.giq.2019.05.004 rheinberger, h. (2005). a reply to david bloor: "toward a sociology of epistemic things." perspectives on science, 13(3), 406-410. https://doi.org/10.1162/106361405774287973 rheinberger, h. (2011). consistency from the perspective of an experimental systems approach to the sciences and their epistemic objects. manuscrito, 34(1), 307-321. https://doi.org/10.1590/s0100-60452011000100014 riemer, k., & peter, s. (2020). the robo-apocalypse plays out in the quality, not in the quantity of work. journal of information technology, 35(4), 310-315. https://doi.org/10.1177/0268396220923677 ritvala, t., & kleymann, b. (2012). scientists as midwives to cluster emergence: an institutional work framework. industry and innovation, 19(6), 477-497. https://doi.org/10.1080/13662716.2012.718875 salmimaa, t., hekkala, r., & pekkola, s. (2018). dynamic activities for managing an is-enabled organizational change. business & information systems engineering, 60, 133-149. https://doi.org/10.1007/s12599-018-0524-6 santos, f., pereira, r., & vasconcelos, j. b. (2020). toward robotic process automation implementation: an end-to-end perspective. business process management journal, 26(2), 405-420. https://doi.org/10.1108/bpmj-12-2018-0380 schmitz, m., dietze, c., & czarnecki, c. (2019). enabling digital transformation through robotic process automation at deutsche telekom. in n. urbach & m. röglinger (eds.) digitalization cases: how organizations rethink their business for the digital age (pp. 1533). springer. https://doi.org/10.1007/978-3-319-95273-4_2 seeber, i., waizenegger, l., seidel, s., morana, s., benbasat, i., & lowry, p. b. (2020). collaborating with technology-based autonomous agents: issues and research opportunities. internet research, 30(1), 1-18. https://doi.org/10.1108/intr-12-2019-0503 australasian journal of information systems doolin et al. 2025, vol 29, research article theorising rpa as socio-technical change 29 siemon, d. (2022). elaborating team roles for artificial intelligence-based teammates in humanai collaboration. group decision and negotiation, 31, 871-912. https://doi.org/10.1007/s10726-022-09792-z siemon, d., & kedziora, d. (2023). from accountant to software developer – transforming employees with robotic process automation (rpa). proceedings of the 56th hawaii international conference on system sciences (pp. 42444253). https://aisel-aisnet-org/hicss56/in/digital_transformation/6 staaby, a., hansen, k., & gronli, t. m. (2021). automation of routine work: a case study of employees’ experiences of work meaningfulness. proceedings of the 54th hawaii international conference on system sciences (pp. 156-165). https://aisel-aisnet-org/hicss54/cl/ai_and_future_work/4/ syed, r., suriadi, s., adams, m., bandara, w., leemans, s. j. j., ouyang, c., ter hofstede, a. h. m., van de weerd, i., wynn, m. t., & reijers, h. a. (2020). robotic process automation: contemporary themes and challenges. computers in industry, 115, 103162. https://doi.org/10.1016/j.compind.2019.103162 techatassanasoontorn, a. a., waizenegger, l., & doolin, b. (2023). when harry, the human, met sally, the software robot: metaphorical sensemaking and sensegiving around an emergent digital technology. journal of information technology, 38(4), 416-441. https://doi.org/10.1177/02683962231157426 vaast, e., & pinsonneault, a. (2021). when digital technologies enable and threaten occupational identity: the delicate balancing act of data scientists. mis quarterly, 45(3), 1087-1112. https://doi.org/10.25300/misq/2021/16024 van den broek, e., sergeeva, a., & huysman, m. (2021). when the machine meets the expert: an ethnography of developing ai for hiring. mis quarterly, 45(3), 1557-1580. https://doi.org/10.25300/misq/2021/16559 van de ven, a. h., & poole, m. s. (2005). alternative approaches for studying organizational change. organization studies 26(9), 1377-1404. https://doi.org/10.1177/0170840605056907 van der aalst, w. m. p., bichler, m., & heinzl, a. (2018). robotic process automation. business & information systems engineering, 60(4), 269-272. https://doi.org/10.1007/s12599-0180542-4 viehhauser, j., & doerr, m. (2021). digging for gold in rpa projects – a quantifiable method to identify and prioritize suitable rpa process candidates. in la rosa, m., sadiq, s., & teniente, e. (eds.) advanced information systems engineering (pp. 313-327). springer. https://doi.org/10.1007/978-3-030-79382-1_19 waizenegger, l., & techatassanasoontorn, a. a. (2022). when robots join our team: a configuration theory of employees' perceptions of and reactions to robotic process automation. australasian journal of information systems, 26, 1-33. https://doi.org/10.3127/ajis.v26i0.3833 willcocks, l., hindle, j., & lacity, m. (2019). keys to rpa success: how blue prism clients are gaining superior long term business value knowledge capital partners executive research report. australasian journal of information systems doolin et al. 2025, vol 29, research article theorising rpa as socio-technical change 30 copyright copyright © 2025 doolin, b., techatassanasoontorn, a. a., waizenegger, l. and wallacecarter, e. this is an open-access article licensed under a creative commons attribution-noncommercial 4.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v29.5451 https://creativecommons.org/licenses/by-nc/4.0/deed.en https://creativecommons.org/licenses/by-nc/4.0/deed.en paper title (paper title style) australasian journal of information systems rosenkranz, kautz & cooper 2025, vol 29, research article organisational agility through boundary spanning and knowledge brokering received 21 feb 2024, accepted 11 feb  2025 after 2 rounds of review with henry linger as accepting senior editor, m. davern & s. black as editors in chief 1 supporting organisational agility through boundary spanning and knowledge brokering in a dual operating system: a case from the software industry christoph rosenkranz depart of information systems university of cologne karlheinz kautz department of information systems and business analytics rmit university vanessa cooper department of information systems and business analytics rmit university email: vanessa.cooper@rmit.edu.au abstract one challenge that organisations face today is how to develop organisational agility to remain competitive within a constantly turbulent, dynamic, and potentially disruptive environment. in this paper, we examine how a software development company operationalises kotter’s concept of a dual operating system to support organisational agility. the first operating system, the traditional hierarchical structure, enables the company to focus on providing efficiency and stability in its core operations. the second operating system prioritises flexibility. it is operationalised via boundary spanning and knowledge brokering activities rooted in a dedicated organisational unit that enables rapid response to environmental changes. we analyse the unit’s and its members’ approach and identify well-functioning activities and practices contributing to organisational agility as well as challenges and problems. as such we contribute to an improved understanding of how an intertwined strategy of boundary spanning and knowledge brokering, operating alongside a traditional hierarchical structure, forms a balancing mechanism between flexibility and stability, thus enabling organisational agility. keywords: organisational agility, boundary spanning, knowledge brokering, dual operating system, software development. 1 introduction as a consequence of growing digitalisation and digital transformation, all types of products and services increasingly incorporate digital hardware and software components (brynjolfsson & mcafee, 2014; laato, mäntymäki, islam, hyrynsalmi, & birkstedt, 2023; wessel, baiyere, ologeanu-taddei, cha, & jensen, 2021). at the same time, companies need to address the challenges produced by mounting complexity and rapid change in today’s competitive and constantly turbulent, and potentially disruptive, business environment (kotter, 2012). increasingly, therefore, software development takes place in dynamic and constantly changing situations as well as environments with rising complexity and demands for efficiency (benbya, nan, tanriverdi, & yoo, 2020; sommerville, cliff, calinescu, keen, kelly, kwiatkowska, mcdermind, & paige, 2012). this underscores the need for software development companies to become more flexible and agile (baham & hirschheim, 2022; harris, collins, & hevner, 2009; overby, bharadwaj, & sambamurthy, 2006) and to “grow australasian journal of information systems rosenkranz, kautz & cooper 2025, vol 29, research article organisational agility through boundary spanning and knowledge brokering 2 agile beyond the system development team to accommodate the required interactions [with other functional areas of the organisation]” (ågerfalk, fitzgerald, & slaughter, 2009). that is, a shift beyond agile development of it towards wider organisational agility is required (tallon, queiroz, coltman, & sharma, 2019). organisational agility, also called enterprise or business agility, is the ability to be flexible enough to speedily respond to customer requests, market dynamics, emerging technology options, and to adapt to a turbulent environment (franco, guimarães, & rodrigues, 2023; mathiassen & pries-heje, 2006; ravichandran, 2018; sambamurthy, bharadwaj, & grover, 2003) as well as to be stable enough to have sufficient processes, frameworks and structures to enable efficiency and productivity and avoid disorderly disintegration (franco et al., 2023; mathiassen & pries-heje, 2006). although there is a growing body of research on organisational agility there remains a shortage of studies addressing the methodologies that support organisational agility (franco et al., 2023). one mechanism that has prominently been suggested to cope with the challenges produced by mounting complexity and rapid change is a second organisational operating system1, which uses an agile and network-like structure and a very different set of processes than the traditional, hierarchical structure of an enterprise (kotter, 2012). kotter (2012) introduced the idea of a second operating system to address the need for organisations to simultaneously manage their ongoing operations via their first operating system, while undertaking significant transformations through a second operating system. this “[second] operating system continually assesses the business, the industry, and the organization, and reacts with greater agility, speed, and creativity than the existing one. it complements rather than overburdens the traditional hierarchy, thus freeing the latter to do what it’s optimized to do” (kotter, 2012). it is far from clear, however, what exactly such a second operating system should look like, how it should be set up, how such a system operates in detail, and if it increases organisational agility in the end. thus to date kotter's (2012) proposed concept of a second operating system, is largely uncorroborated. meanwhile, organisational research on environmental turbulence has long suggested boundary spanning as one important mechanism to cope with increasing complexity (aldrich & herker, 1977; ancona & caldwell, 1990; keszey, 2018), and to manage interfaces between the organisation and its environment (shi, cui, & kurnia, 2023; tushman, 1977, 1979), as well as between organisational units (ancona & caldwell, 1990; levina, 2005; levina & vaast, 2005). moreover, in addition to boundary spanning, knowledge brokering between diverse occupational groups takes place at knowledge boundaries and is important for facilitating knowledge sharing (argote & miron-spektor, 2011; crupi, del sarto, di minin, gregori, lepore, marinelli, & spigarelli, 2020; pawlowski & robey, 2004). the concepts of boundary spanning and knowledge brokering are both closely related to networks which makes them a prime choice for conceptualising and operationalising the network-like structure of a second operating system. the concepts share the common goal of connecting disconnected members 1 the concept of organisational operating systems formed by an organisation’s structures and processes should not be confused with the technical concept of operating systems which denote the software programs that manage computer hardware and software resources and provide common services for a computer’s programs. australasian journal of information systems rosenkranz, kautz & cooper 2025, vol 29, research article organisational agility through boundary spanning and knowledge brokering 3 across networks (cross & cummings, 2004; fleming & waguespack, 2007; kwon, rondi, levin, de massis, & brass, 2020; reagans & mcevily, 2003; tortoriello, reagans, & mcevily, 2011) but remain distinct wherein they describe different types of interaction (fleming & waguespack, 2007). both concepts have not been investigated together in much detail, except for open innovation communities (fleming & waguespack, 2007), and neither has been investigated with regard to their relation to organisational agility so far. the current role and interplay of both for enabling organisational agility in the software industry is unclear, as is their role for operationalising the kind of second operating system suggested by kotter (2012). therefore, we address the following research question: “how do boundary spanning and knowledge brokering enable organisational agility via a second operating system in the software industry?” in this paper, we present the results of an extensive case study of a software development company. we explore the role of boundary spanning and knowledge brokering in operationalising a second operating system in an organisational unit whose members engage in both knowledge brokering and boundary spanning between the company and its environment as well as within the company. we analyse the unit’s and its members’ approach and identify well-functioning activities and practices as well as challenges and drawbacks. as such, we contribute to an improved understanding of how an intertwined strategy of boundary spanning and knowledge brokering can establish and evolve a second operating system, allowing the first operating system, the traditional hierarchical structure, to focus on providing efficiency and stability in core operations. we thus provide empirical evidence of how a dual operating system can provide a balancing mechanism between flexibility and stability to support organisational agility in a software development company. the remainder of the paper is structured as follows. the next section discusses the related work and theoretical background of our study, detailing the concepts of organisational agility, the dual operating system, boundary spanning and knowledge brokering. we then present results from the exploratory study of a specialised unit within a software development company and develop an explanation of how the knowledge brokering and boundary spanning activities of this unit, operating alongside its traditional hierarchical structure, enable the organisation to balance flexibility and stability and support organisational agility. we summarise and discuss our findings and conclude with an indication of limitations and an outlook on further research. 2 related work and theoretical background 2.1 organisational agility in the face of intense competition, globalisation and time-to-market pressures, organisations must be agile, that is, have the capability to rapidly detect and seize market opportunities (sambamurthy et al., 2003). organisational agility is reported to enhance organisational performance overall, as well as in specific areas such as finance, supply chain management and innovation (walter, 2021). in addition to focusing on the organisational outcomes of agility, research has concentrated on how agility should be defined and measured, along with its antecedents, moderators and mediators (see nguyen, le, nguyen, nguyen, lien, & nguyen, 2024; walter 2021). in their recent literature review, nguyen et al. (2024) identified various antecedents to encompass organisational capabilities such as knowledge management, it and cooperation, coordination and collaboration. moderators of the australasian journal of information systems rosenkranz, kautz & cooper 2025, vol 29, research article organisational agility through boundary spanning and knowledge brokering 4 relationship between agility and organisational outcomes were found to include the business environment, market complexity, industry, technology, customer loyalty and other organisational factors such as leadership and firm size/age (nguyen et al., 2024). lastly they (nguyen et al., 2024) grouped the mediators of agility and organisational outcomes within the following fifteen categories: technology, innovation, analytics capabilities, strategy and structure, competitiveness, social capital and networks, customers, market, marketing, supply chain, operation, leadership, workforce and work arrangement, knowledge and sustainability. in the is literature, researchers have been particularly interested in the relationship between it capability (lu & ramamurthy, 2011), strategic it alignment (tallon & pinsonneault, 2011) and it investment (sambamurthy et al., 2003) and organisational agility. with extensive interest in digital transformation, is researchers have also highlighted how it-enabled transformations support organisational agility (vial, 2019). however, while it is often implied that firms need to be in a constant state of transformation, transformations are not always needed, may not be possible and come at a cost (teece, peteraf, & leih, 2016). instead, organisations in pursuit of agility need to balance flexibility with stability to ensure they can respond effectively to changes in the environment while simultaneously being efficient and productive. how organisations achieve this balance and develop organisational agility is a primary concern of this paper. investing in modular it architectures which allow organisations to maintain core stability while responding rapidly to changes (bharadwaj, el sawy, pavlou, & venkatraman, 2013) is important for balancing flexibility and stability. it has also been suggested in the practitioner literature that a bimodal approach to it strategy wherein one mode concentrates on stability and efficiency while the other focuses on flexibility and innovation is useful (gartner, 2016; haffke, kalgovas, & benlian, 2016). organisational structure too, is important for balancing flexibility with stability (kotter, 2012). this is because structure can determine how information flows, how decisions are made and the level of flexibility an organisation has. we consider kotter’s notion of the dual operating system for this purpose, next. 2.2 the dual operating system kotter (2012) introduces the concept of dual operating system. it is rooted in organisational theory which recognises that while traditional hierarchical structures are effective for managing routine operations and ensuring efficiency (burns & stalker, 1961), they often struggle with rapid innovation and adaptability (mintzberg, 1979). in this way they compromise an organisation’s pursuit of agility. in response, kotter (2012) introduces the idea of a parallel network-like structure to operate alongside an organisation’s traditional hierarchy as a second operating system. the structure is designed to be dynamic, allowing for quick decision-making and innovation while leaving the traditional hierarchy to optimise operational efficiency. kotter (2012) describes the structure as comprising a “volunteer army” of employees across the entire organisation and up and down the ranks, who are empowered to drive change, innovate and respond quickly to challenges. notably, a dual operating system need not start fully formed and can start in one part of an organisation and expand (kotter, 2012). overseeing the dual operating system should be a governance process that aligns the two operating systems with the strategic goals of the organisation and which reconciles any conflicts between the systems (kotter, 2012). kotter (2012) identifies eight accelerators to enable a dual operating system to function. these centre on a modern approach to change management and emphasise agility, rapid adaptation, and broad-based engagement, in australasian journal of information systems rosenkranz, kautz & cooper 2025, vol 29, research article organisational agility through boundary spanning and knowledge brokering 5 contrast to the more rigid, controlled, and hierarchical nature of traditional management approaches. however, while the accelerators provide some direction for leaders embarking on a dual operating system, kotter (2012) does not provide detailed insight into the underlying activities that members of the network-like structure need to undertake to support and utilise a dual operating system. as a key objective of the second operating system is to “liberate information from silos and hierarchical layers” and enable information to “flow with far greater freedom and accelerated speed” (kotter, 2012), we examine the role of boundary spanning and knowledge brokering for operationalising the second operating system and their links with organisational agility. 2.3 boundary spanning and knowledge brokering one critical implication of today’s business context and its heightened emphasis on organisational agility, is the need to gather and harness internal and external knowledge to enhance organisational responsiveness and innovation. boundary spanning and knowledge brokering are both concepts closely related to this goal. boundary spanning takes place at organisational boundaries and comprises external boundary spanning between an organisation and its environment as well as internal boundary spanning between different organisational units within the same organisation (ancona & caldwell, 1990; levina, 2005; levina & vaast, 2005). accordingly, boundary spanners are individuals, who are part of one organisational entity and who engage in boundary spanning activities (more on these activities later) towards organisational entities other than their own (aldrich & herker, 1977). organisations that engage in effective boundary spanning are better able to sense changes in their external environment early, understand emerging trends and customer needs, and adapt quickly to external changes (ancona & caldwell, 1992). similarly, knowledge brokers facilitate the transfer of knowledge across knowledge boundaries (fleming & waguespack, 2007; pawlowski & robey, 2004; pawlowski, robey, & raven, 2000). they work to connect individuals or groups, who possess different types of knowledge or expertise, help to translate and frame the elements of one group in terms of the perspective of another, therein enabling them to collaborate and exchange ideas effectively (pawlowski & robey, 2004). knowledge brokering ensures valuable information is disseminated quickly across the organisation which enables faster decision-making, innovation, and adaptation to changing circumstances (pawlowski & robey, 2004). in terms of their relationship with organisational agility, boundary spanning activities ensure the organisation is connected to external trends and changes, while knowledge brokering facilitates the translation, internal dissemination and application of this external knowledge, and together they allow the organisation to quickly adapt its strategies, products, or services based on external developments (ancona & caldwell, 1992; pawlowski & robey, 2004). both boundary spanning and knowledge brokering foster an environment where new ideas can emerge and be implemented swiftly. by bringing in diverse perspectives and insights, breaking down silos and promoting cross-functional collaborations, organisations can innovate more effectively in response to market changes or competitive pressures. by improving the flow of information and knowledge both externally and internally, boundary spanning and knowledge brokering reduce the time it takes for the organisation to detect changes and make decisions (ancona & caldwell, 1992; pawlowski & robey, 2004). this agility in decision-making is crucial for staying competitive in dynamic markets. australasian journal of information systems rosenkranz, kautz & cooper 2025, vol 29, research article organisational agility through boundary spanning and knowledge brokering 6 boundary spanning and knowledge brokering are related concepts, at least regarding knowledge sharing across boundaries (neal, posner, & brutzman, 2023). also, both knowledge brokers and boundary spanners facilitate relationships, disseminate evidence, find alignment, build capacity, and inform decision making (neal et al., 2023). while research has struggled to clearly delineate between the concepts and has identified there is overlap in the activities that boundary spanners and knowledge brokers undertake, there are differences (pawlowski & robey, 2004). we now turn our focus to the different boundary spanning and knowledge brokering activities that can potentially support a second operating system as proposed by kotter (2012). 2.4 boundary spanning activities in their studies of new product development teams, ancona and caldwell (1990, 1992) identified and summarised boundary spanning activities as ambassadoring, coordinating tasks, scouting, and guarding (see table 1, which also contains some examples for the activities). ambassadoring covers buffering activities and representational activities. boundary spanning activity description examples ambassadoring includes buffering activities and representational activities of individuals within a team or department. this activity is somewhat political; identifying threats and opposition in top levels of the organisation and working to build support from these powerful outsiders. absorbing pressures and protecting the team persuading others to support the team keeping higher levels informed of team activities informing the team of company strategy informing the team of potential threats or opposition lobbying for resources task coordination involves interactions aimed at coordinating technical or design issues. this activity represents specific efforts to coordinate and synchronise with other groups, typically those involved with the specific product or service. discussing design problems with others obtaining feedback on the product design coordinating and negotiating with outsiders scouting comprises activities that involve general scanning for ideas and information about the competition, the market, or the technology. this activity represents a more general scanning than task coordination, aimed at obtaining competitive, market, and technical ideas. undertaking research on market trends exploring emerging technology trends studying the strategies competitors are employing guarding involves controlling the team’s release of information. it represents a means of decreasing, rather than meeting, dependence through the control of information flow out of the group. activities aimed at keeping information within the team’s boundaries to protect the team presenting a specific image of the team to outsiders table 1. summary of key concepts for boundary spanning (ancona & caldwell, 1990; 1992) these activities are somewhat political and contain both protective and persuasive goals such as obtaining personnel, funding, equipment, and legitimacy from management. task coordination consists of interactions aimed at coordinating technical or design issues. this might involve a software designer liaising with marketing and sales about the scope of a software update and its associated timeline. scouting involves general scanning for ideas and australasian journal of information systems rosenkranz, kautz & cooper 2025, vol 29, research article organisational agility through boundary spanning and knowledge brokering 7 information about the competition, the market, or the technology and involves more general scanning than task coordination. both task coordination and scouting manage the dependence on other functions or groups that have critical information, expertise, and creative ideas. ancona and caldwell (1992) note that scouting is best undertaken during the early life of a project and should not be an enduring pattern because if a team continues scouting without committing to a plan, it may have a detrimental impact on performance. finally, guarding comprises controlling the team’s release of information. this activity differs from the others in that it represents internal activities to keep things from the environment. boundary spanning involves political manoeuvring, management, and coordination as well as knowledge sharing activities (ancona & caldwell, 1998), which goes beyond fleming and waguespack’s (2007) view, who see boundary spanning as primarily bridging technological boundaries. studies on boundary spanning in various fields, including information systems development projects (gasson, 2006; gopal & gosain, 2009; levina & vaast, 2005; shi et al., 2023) confirm the applicability of this broader perspective on boundary spanning. 2.5 knowledge brokering activities in their studies of it professionals, pawlowski et al. (2000) and pawlowski and robey (2004) identified and summarised four knowledge brokering activities, namely, crossing boundaries, surfacing and challenging assumptions, translating and interpreting information and relinquishing ownership (see table 2, which also contains some examples for the activities). crossing boundaries involves not just crossing knowledge boundaries or social boundaries, which fleming und waguespack (2007) see as the primary boundaries that brokers are crossing, but also crossing organisational boundaries between units to share information and to leverage resources; it also includes the effort of gaining permission from business units to cross organisational boundaries that are closed to others. surfacing and challenging assumptions involves stimulating reflection and change by asking ‘why’ as a means to uncover existing beliefs, practices and assumptions underlying the work of different divisions. (pawlowski & robey, 2004). translating and interpreting information involves the framing of elements of the worldview of one group in terms of the perspective of another. this involves reframing, explaining, and clarifying information in the context of the work practice of one group to the context of another. relinquishing ownership and maintaining a façade of objectivity includes the creation of the illusion that one is impartial and prepared to support any solution, even though a particular one is favoured. this might involve empowering users or transferring ownership of a project to business stakeholders while simultaneously maintaining influence over the outcomes. overall, knowledge brokering focuses on knowledge sharing and processes of translation, coordination, and alignment between perspectives (wenger, 1998). based on ancona and caldwell’s (1990, 1992) categorisation of boundary spanning activities and pawlowski and robey’s (2004) classification of knowledge brokering activities, in the following study we examine the knowledge brokering and boundary spanning activities undertaken by a dedicated organisational unit in a software development company as part of its wider strategy to balance flexibility and stability to support organisational agility via a dual operating system. australasian journal of information systems rosenkranz, kautz & cooper 2025, vol 29, research article organisational agility through boundary spanning and knowledge brokering 8 knowledge brokering activity description examples crossing boundaries routinely crossing organisational boundaries between units, sharing information, and leveraging resources. this includes the exertion of considerable effort in gaining permission from business units to cross organisational boundaries that are closed to other groups. participating in communities of practice (cops) comprising members from different business units managing relationships with it vendors surfacing and challenging assumptions stimulating reflection and change by asking ‘why’ as a means to uncover existing beliefs, practices and assumptions underlying the work of different divisions. prompting discussion by promoting alternative solutions challenging the technical feasibility of products to prompt critical discussions translation and interpretation framing elements of the worldview of one group in terms of the perspective of another. translating technical language into business terms interpreting business needs for technical developers facilitating cross-disciplinary communication façading / ostensibly relinquishing ownership creating the illusion that one is impartial and prepared to implement any solution requested by the user, even though a particular outcome is favoured (façade of impartiality). creating customer steering committees but guiding these through setting the agenda, disseminating the information, providing guidance and subtly influencing decisions. soliciting customer feedback but only selectively incorporating this into the product or service design table 2. summary of key concepts for knowledge brokering (pawlowski and robey 2004). 3 research design and method we conducted an exploratory, interpretive single in-depth case study to develop an understanding and theoretical explanation of how boundary spanning and knowledge brokering together enable organisational agility by operationalising the second operating system within the dual operating system of a software development company. the case organisation is a medium-sized german software solutions provider called softcorp (anonymised) that has undergone a transformation towards becoming a more agile and flexible organisation. the company is doing well in its dynamic environment with increasing numbers of sold licenses, acquired major customers and steadily rising customer satisfaction. the selected case organisation represents the main unit of analysis, but the individual departments, groups, and employees each represent analytical subunits, which allows for a multi-level analysis as called for by marrone, tesluk, and carson (2007) and is in line with multi-perspective innovation studies (cf. kautz & nielsen, 2004 for a detailed argument). we observed established guidelines for our research design ( miles & huberman, 1994; miles, huberman, & saldaña, 2014). the two main methods for data collection were interviews and observation (see table 3). during a period of one year, we visited the company three times for one to three days each visit. we conducted open interviews with selected key informants, group interviews, and observations of activities. all interviews, which lasted from 30 to 120 australasian journal of information systems rosenkranz, kautz & cooper 2025, vol 29, research article organisational agility through boundary spanning and knowledge brokering 9 minutes, and meetings were audio-recorded and transcribed. we followed a two-stage process of inductive and deductive coding of data, building upon and adapting the recommendations by miles and huberman (1994). first, two authors scrutinised and coded the data independently of each other. inspired by previous work on boundary spanning (ancona & caldwell, 1990; 1992) and on knowledge brokering (pawlowski & robey, 2004; pawlowski et al., 2000), we started with initial seed codes and searched separately for evidence of boundary spanning (see table 1) and knowledge brokering (see table 2) activities in relation to organisational agility. each author also added new codes in exploring the data in more detail through coding events and activities that they found important (e.g., several new codes emerged related to task variety and uncertainty/variability in processes). subsequently, the authors discussed their interpretations in person or using e-mail and teleconferencing. data type description single interviews 10 participants (4x leaders product management team, 1x developer product management team, 1x manager product management team, 1x professional services consultant, 1x manager professional services, 1x ceo, 1x manager marketing) group interviews 8 participants (3x product management team, 1x apprentice product management team, 2x manager and leader product management team, 1x lead developer and manager core development team, 1x manager product management & professional services consultant) observations 1x retrospective, 1x review meeting, daily stand-up meetings on 3 days documentation product and company information (internal documents, reports, printouts) table 3. overview of data & data collection methods it is worth noting that we follow alvesson and colleagues, who emphasise reflexivity as a key quality of rigorous thinking in the sense of identifying and challenging assumptions in the research literature and in research data (alvesson hardy, & harley, 2008). alvesson and sandberg (2013) call for “a more active construction of empirical material in ways that are interesting, and not just waiting passively for data to show us the route to something interesting, as is typically the case in more conventional research”. in contrast to the conventional concept of rigour they advocate “some relaxation of the pressure for the required standard in incremental gap-spotting research that emphasises rules, mechanics [of detailed data codification procedures] and data management” (alvesson & sandberg, 2013). this resulted in the following analysis. 4 case report and analysis 4.1 software development at softcorp softcorp is a german software development company, founded in 1999, it is now a subsidiary company of an exchange-traded it service provider. softcorp employs around 70 staff at its headquarters, with about 90 employees in total and sales offices across europe and the us. the core software product of softcorp is a content management system (cms). numerous companies from many industries worldwide use the cms for managing their internet presences as well as their intranet portals. professionals from softcorp or selected partner companies provide consultancy and project services such as implementation, tailoring, or configuration of the cms to end customers. softcorp’s strategy as regards to the cms is product-driven and not customer-driven: australasian journal of information systems rosenkranz, kautz & cooper 2025, vol 29, research article organisational agility through boundary spanning and knowledge brokering 10 “[…] the complete firm acts on the maxim or notion that we will not be driven by customer requests. we develop the product following our own vision […] in many areas, in many, many decisions, and for a long number of years, we simply didn’t follow the trend on the market, which essentially is determined by marketing or sales, but we had our own thoughts about it.” (mike2, member of the executive board) as a result of this strategy, softcorp focuses on the development of a stable software core that is compatible between release versions. end customers need to run their own development projects, possibly supported by professionals from softcorp or partner companies, for customer-specific extensions of the cms’ core features. such projects can range from simple extensions, so-called “modules”, to complex web application projects. for example, modules allow the application of e-mail marketing, the display of content on mobile devices, or the integration with enterprise application servers. most of the time, these extensions are not subsequently integrated into the core features of the cms. figure 1 summarises softcorp’s organisational structure. consulting services and support for internet and intranet projects of end customers are provided by consultants and project managers from the professional services unit (31 employees). the research & development unit (21 employees) comprises the development, quality assurance and internal it teams. the core development team at softcorp’s headquarters (eight developers plus one manager) has exclusive responsibility for developing and maintaining the cms as the core product. executive board professional services product management (pm team) marketing & communications (incl. sales) int. business development research & development quality assurance development (core dev. team) helpdesk (external) internal it figure 1. summary of organisational structure of softcorp the core development team continuously develops and advances the cms. in general, a release cycle takes several months, resulting in a gap between major releases of several years. the development process follows a mixture of practices from traditional software engineering (e.g., variants of stage-gate models) and modern approaches (e.g., time boxing). the strategy to shield the core development team from outside pressure, enabling them to develop the core product in a sustainable and stable way, results in a kind of wall to the external environment and to other departments: 2 all names are anonymous for reasons of confidentiality. australasian journal of information systems rosenkranz, kautz & cooper 2025, vol 29, research article organisational agility through boundary spanning and knowledge brokering 11 “[…] intentionally, we actually afford ourselves at the product side a small ivory tower, one can say that, an insular product development unit, many competitors work differently than that. and we actually make a point of doing so, that there exists this kind of wall […]” (tom, manager professional services) since spring 2011, the product management sub-unit (in the following: pm team) of the professional services unit provides a second development team (six full-time employees and four apprentices). the pm team is responsible for developing modules for the cms that address specific noncore features (e.g., video management). the pm team as a second development unit was created because of a felt need to be able to accelerate the development of modules to be able to react more quickly and faster to end customers’ and partners’ requests as well as to internal feature requests: “the module development team [the pm team] has been primarily brought into being in order to react quickly to requirements that come from the market because our development department simply cannot do that because they have firmly defined the work packages for the next one and a half years.” (charles, member of the pm team) thus, a specialisation exists as regards software development. while the core development team develops the cms as a stable core product in a steady way, with a pace and time horizon of years between releases, the pm team develops modules at a much faster pace, with a time horizon of months. using two distinct development teams allows softcorp to react more quickly to customer demands, without having to jeopardise the stability of its core product. the pm team also employs a different approach to software development than the core development team, using a variation of lean software development (poppendieck & poppendieck, 2003). 4.2 the pm team: the jack of all trades the pm team is not only responsible for software development in the form of new or customer-specific software modules for the cms. in parallel, a variety of other tasks are situated with the team (see table 4), and it sometimes acts as an internal fire-fighter, or indeed a jack of all trades: “[…] internally, the product management team per se is an all-rounder and jack of all trades, and all sorts of people from all sorts of departments stop over all the time and want something to be done or made, have questions, and so forth.” (joe, apprentice in the pm team) as such, the team is approached by colleagues from other units such as marketing & communications and by external partners regarding a variety of topics related to the cms. it thus provides internal product support and acts as an internal help desk. but it also develops show cases which involves the design and implementation of concepts and presentations for new cms features and modules to support marketing & communications at end customer demonstrations or fairs. in addition, the team provides intranet care, and maintains and supports softcorp’s intranet, both content-wise and infrastructure-wise. furthermore, it supports the cms community, where end customers and partner companies can exchange ideas, knowledge, and experiences with each other and with employees from softcorp using an internet-based bulletin board. the pm team answers questions regarding the cms and provides content for community members respectively. finally, three team members serve as stand-by personnel, and are occasionally used as consultants within end customer projects: australasian journal of information systems rosenkranz, kautz & cooper 2025, vol 29, research article organisational agility through boundary spanning and knowledge brokering 12 “[...] somebody such as luke who is something of a ‘jack of all trades’, who can do everything. […] i don’t need a developer, who sits there and looks at the code […] i need someone, who sits there and sometimes gives the developers a piece of one’s mind.” (sean, consultant professional services) task description software development development of new or customer-specific software modules for the cms development of show cases design of concepts and presentations for new cms features and modules, supporting marketing & communications, for example, for end customer demonstrations or fairs intranet care maintain and support softcorp’s intranet, both content-wise and infrastructure-wise support of the cms community end customers and partner companies can exchange ideas, knowledge, and experiences with each other and with employees from softcorp using an internet-based bulletin board; the pm team answers questions regarding the cms and provides content for the community members support as standby personnel three members of the pm team occasionally act as stand-by personnel and are used by professional services as consultants within end customer projects internal product support the pm team acts as an internal helpdesk and is approached by colleagues (e.g. from marketing & communications) and by external partners regarding a variety of topics related to the cms table 4. overview of tasks of the pm team 4.3 knowledge brokering in the pm team knowledge brokering by the pm team involves crossing boundaries, translation and interpretation and surfacing and challenging assumptions. none of the interviewees described knowledge brokering activities that comprise façading or ostensibly relinquishing ownership. the pm team routinely crosses internal boundaries between other departments and units within softcorp. this results in the sharing of knowledge between units which would otherwise not talk to each other: “[…] so these review meetings, we also conduct them in order to exemplify to others a little bit. because in reality, we wish us the same from these people. that’s one of the reasons why we do this, among other things. […] so that's one reason, because we would like to actually establish this kind of openness and this including of others throughout the company. and we are now the only ones who do it in this way.” (paul, manager of the pm team) the pm team members have direct access to members of the core development team, which members from other departments do not have. due to the pm team members’ extensive technical knowledge of the cms, when undertaking internal product support and developing show cases, the team often needs to translate and interpret the technical perspective of the core development team and those of more business and customer-oriented departments such as professional services or marketing & communications, so they can each understand the others’ views: “[…] so to speak, we are internally the contact for all departments that have questions regarding the product. so when our presales or our sales have questions in any pitch situations, ‘can we do that? how is our position as regards … ?’, then product management is asked [instead of the core development team].” (charles, member of the pm team) australasian journal of information systems rosenkranz, kautz & cooper 2025, vol 29, research article organisational agility through boundary spanning and knowledge brokering 13 specifically, during the development of modules for the cms, the pm team also engages in activities that surface and challenge assumptions. this includes both product-level (technical and business issues) as well as organisational-level topics. for example, as regards the former, the pm team continuously questions the way things are done in the cms during the development of modules, and tries to find new architectural solutions (e.g., challenging the pre-generation of content paradigm of the cms in certain areas): “yes, yes, but as i said before, the reaction [to our new module] was very clear ‘oh, we haven’t done it that way for the last 10 years and it is just something new’. well a completely new dimension, which had not been considered by the projects, as they until now always got away quite well with the existing solution.” (charles, member of the pm team) the pm team also regularly challenges the existing practices and ways of doing things at softcorp, for example, by actively choosing to manage projects differently than the core development team, using lean software development, and by being much more communicative than the core development team. 4.4 boundary spanning in the pm team as regards to boundary spanning, the pm team presents a very interesting case, undertaking ambassadoring, coordinating, scouting, and guarding activities on behalf of softcorp between outsiders such as external partners and end customers as well as between insiders such as staff from marketing & communications, and other employees from professional services. when external partners or those from other softcorp units experience issues with the existing cms the pm team seeks to provide support. this can involve scouting for information inside the firm to answer questions that arise or scouting for new knowledge to identify ideas for new functionality and solutions. moreover, the pm team engages in ambassadoring activities during their tasks as stand-by personnel. this gives pm team members the opportunity to inform partners of softcorp’s product strategy and garner their support while simultaneously protecting the core development team from outside pressure and informing them of potential threats and opportunities for product improvement: “and then we have topics such as that you sometimes visit the partner and talk about the product or project decisions, and explore how you can improve the product so that developers and partners can work in a better way using it.” (luke, member of the pm team) the pm team’s development task developing modules for the cms also involves frequent instances of coordinating activities regarding technical or design issues: “for a new module, we always at the outset conduct a corresponding workshop, to which we invite people from sales, from presales, from professional services, to jointly define requirements, at least roughly, or at least the expectations for the module as regards functionality.” (charles, member of the pm team) similarly, other tasks show activities of scouting the external environment in relation to the competition, the market in general, or innovative technologies. for example, providing consulting support as stand-by personnel involves scouting: “[…] and we have to look – we do quite some interrogation of third party products – what’s up on the market, and this is a very different kind of research & development. a bit more looking outside, we have to look around a bit more and not just look how can we do that better, how can we optimise, but it’s really, more, yes, putting on the market glasses.” (paul, manager of the pm team) australasian journal of information systems rosenkranz, kautz & cooper 2025, vol 29, research article organisational agility through boundary spanning and knowledge brokering 14 finally, as regards guarding, the pm team responds quickly to staff, especially to customer and market demands, and thus shields the core development team from external and internal contacts so that the latter can concentrate on developing the cms: “while the core team says ‘we have to defend the core asset of the firm for the next ten years, enhance and develop it in a clean way, and create a stable foundation on which customers can rely on using many, many interfaces in all kinds of directions’, the pm team is the more agile division, which can react considerably faster to a customer request, which can react considerably faster – that is also important – if we see opportunities within the market” (mike, member of the executive board) 4.5 organisational agility: balancing flexibility and stability through a dual operating system softcorp balances flexibility and stability based on market expectations and demands. the company does this with what we identify as a dual operating system: the first operating system is reflected in a traditional organisational hierarchy and provides stability by allowing the core development team to design the cms with a long-term vision operationalised by the original, shielded development unit while the second operating system is designed to react more quickly and nimbly to market, staff, and customer requests operationalised by the boundary spanning and knowledge brokering activities and tasks of the pm team: “[…] an important factor for the core product is the market expects that you to have at least a 3 years development roadmap; if you don’t have it you have a problem. as for the modules, we look maximum 12 months forward into the future, that’s it. if we would be forced to plan as long term [as the core development team] we would be in the same pattern.” (paul, manager of the pm team) the individual members of the pm team, who are organised in this unit have business as well as detailed technical knowledge of the core cms product, its software code, and the cms implementation projects because of their diverse tasks. gathering these skill sets in one dedicated organisational unit created a broad knowledge base which is considered beneficial for the organisation: “previously, we didn’t have that department. and we had very big problems because, yes, because this knowledge was not focused at one single point, so to speak. […] i believe that this unit contributes very strongly to the fact that we can develop a good, broad knowledge and a good view of the company and for the product itself and for the market as such, yes.” (paul, manager of the pm team) against this background the pm team undertakes diverse forms of knowledge brokering and boundary spanning activities between softcorp’s different units and towards softcorp’s environment. it was described by one member of the pm team as the interface between all departments and having important influence on the direction of softcorp’s product development. importantly, having the pm team focus on responding to market, staff, and customer requests provides the core development team with a solidity that has had a positive effect on customer satisfaction: “this deceleration [i.e., shielding the core development team] has meant that customer satisfaction has increased massively, it now has much less problems, i'm also helpdesk manager, the number of tickets has clearly decreased.” (tom, manager professional services) boundary spanning and knowledge brokering as performed by committed individuals, who are organised in a dedicated organisational unit in softcorp then, contributes to increased organisational agility by (1) enabling faster responses to customer requests and influences australasian journal of information systems rosenkranz, kautz & cooper 2025, vol 29, research article organisational agility through boundary spanning and knowledge brokering 15 from the market and external environment; (2) spotting opportunities faster; and (3) augmenting speed-to-market by shielding and taking pressure off the core development team to enable it to pursue a sustainable long-term vision and allowing for a steady development pace that delivers a stable core product. the boundary spanning and knowledge brokering activities undertaken by the pm team are key in operationalising a second operating system at softcorp and thus supporting the company to balance flexibility and stability via a dual operating system. as for boundary spanning, ambassadoring is reflected in the pm team’s engagement in a range of activities for softcorp, especially with the internal and external community tasks, their own development tasks, and as stand-by personnel in customer projects. with respect to coordinating tasks, this is evident when the pm team takes on various activities related to managing design and/or technical issues again through their performance of development tasks, their community tasks, and their role as stand-by personnel. in the identified scouting activities, the pm team explores the external environment, the market in general, their competitors as well as novel and innovative technologies on behalf of softcorp during their work tasks. finally, when guarding softcorp from outside and inside influences so that the core development team can concentrate on developing the core of the cms, the pm team through their development work, performance of community tasks, and also as stand-by personnel plays a significant role in balancing stability and flexibility and thus softcorp’s agility. as for knowledge brokering, in terms of crossing boundaries, the pm team routinely crosses internal boundaries between other departments and units within softcorp, which would otherwise not talk to each other. regarding surfacing and challenging assumptions, the pm team regularly challenges the existing product design decisions as well as organisational practices and ways of doing things at softcorp. lastly, when engaging in translation and interpretation the pm team frequently outlines elements of the technical views of the core development team regarding the core cms product in terms of the perspective of other units and departments such as professional services, marketing & communications, and it also translates market, customer and staff requirements to technical concepts, which they either themselves develop into modules or pass on to the core development team. the role of the pm team as dedicated unit that engages in both knowledge brokering and boundary spanning between other departments and between partners, end customers and the market, thus contributes to softcorp’s organisational agility. the tasks and activities of the pm team operationalise a second operating system, which provides flexibility as it reacts to changes in a much more accommodating way than the other softcorp units, and is more dialogical and interactive, both with the outside environment and with other units. it neither only conducts knowledge brokering nor boundary spanning; it does both in an intertwined way. the other units found in the traditional organisational hierarchy, the support functions and core development team, operationalise the first operating system, wherein the core development team is intentionally shielded from outside and inside influences, and only has minimal contact to other units and the external environment. this increases softcorp’s stability as it allows for a sustainable development of the core product with a long-term vision. softcorp’s approach to a dual operating system for organisational agility is summarised in figure 2. australasian journal of information systems rosenkranz, kautz & cooper 2025, vol 29, research article organisational agility through boundary spanning and knowledge brokering 16 figure 2. softcorp’s dual operating system for organisational 4.6 benefits and drawbacks of the second operating system the boundary spanning and knowledge brokering activities of the pm team in the second operating system clearly have several positive effects for softcorp as a whole. together, they enable the pm team to undertake agile development of modules using lean development, allowing for stable development of the core product with a clear, long-term vision by the core development team in the first operating system. by undertaking a variety of boundary spanning and knowledge brokering activities, the pm team contributes to organisational agility by acquiring knowledge on broad topics from diverse sources enabling it to react fast to changes in the environment. the manager of the pm team advocated that this enables the team to develop a broad understanding of business and product and to stay in touch with the company vision in a way that would not be possible if the team were only focused on development. however, while the activities of the pm team produce a range of benefits, they come at a cost and have a number of observed drawbacks. we summarise these drawbacks here also as a note of caution for organisations that may consider a similar approach. although it helps to accumulate knowledge in diverse fields, the variety of different potential units that want to contact the pm team sometimes leads to overload as regards to the capacity of the pm team to handle all tasks and requests. thus, a fundamental problem for enhancing the efficiency of the pm team, and thus the second operating system, is the variety of the tasks that are executed besides developing modules. each task domain is a business process, with a separate input, output, and flow unit, and is completely different as regards to its characteristics. for example, developing modules is different from developing show cases, or australasian journal of information systems rosenkranz, kautz & cooper 2025, vol 29, research article organisational agility through boundary spanning and knowledge brokering 17 from supporting the community on the online bulletin board. furthermore, demand for each task is independent from the demand of other tasks: “[…] the disadvantage is, of course, that all come running to us and always want to know something, yes, of course, this is the flip side of the coin. you somehow have to steer this into regulated channels or defend yourself in a sufficiently pushy way, as the case may be [laughs].” (paul, manager of the pm team) this task variety, with different goals, in turn leads to more variability and uncertainty in the development process and in all other tasks respectively. this becomes apparent when looking at how the pm team organises its work: using one global kanban board with the same activities for all tasks, even if every task is in fact a separate process. for example, sometimes the prioritisation of other tasks such as the building of show cases or the use of pm team members as stand-by personnel leads to bottlenecks and waste and cases where the development of modules is put on hold because developers are missing: “[on the role of stand-by personnel:] yes, all right. and luke was now, i would say; the last resort because he actually works in pm and actually has other tasks, but the other expert we wanted to have, first, he was on vacation and second, he also is a highly sought-after person with use here in the house.” (sean, consultant professional services) “[…] so we have to cut somewhere, consequentially […] which can mean that before cebit [trade fair], we hardly do any module development, so we can a little bit absorb this, internally, so almost no module development, but show cases, and accordingly vice versa, after cebit.” (charles, member of the pm team) table 5 summarises benefits and drawbacks of softcorp’s second operating system through the pm team. benefits drawbacks • quick, agile development of modules using lean development (kanban) while allowing for stable development of core product with a clear, longterm vision • create business agility and react fast to changes in the environment • acquisition of broad knowledge from a wide variety of tasks • increased knowledge brokering between other internal units and external parties • spanning of internal boundaries and boundaries with external end customers and partners • variety of tasks leads to uncertainty and variability in the processes • uncertainty and variability in the processes lead to bottlenecks • variety of tasks clashes with lean management approach table 5. summary of benefits and drawbacks of softcorp’s operationalisation of the second operating system 5 discussion and conclusion we performed a study that, building on kotter’s work on dual organisational operation systems (kotter, 2012), investigated how the boundary spanning and knowledge brokering activities rooted in a dedicated unit operationalise a second operating system, which when operating alongside a traditional hierarchical structure, the first operating system, enable the organisational agility of a software development company. australasian journal of information systems rosenkranz, kautz & cooper 2025, vol 29, research article organisational agility through boundary spanning and knowledge brokering 18 we established earlier in section 1, that the study is further based on an understanding that the concepts of boundary spanning and knowledge brokering share the common goal of connecting disconnected organisational members across networks. thus, they are closely related to the network concept. this makes them a prime choice for conceptualising and operationalising the network-like structure of a second operating system as proposed by kotter (2012). in the case setting, the boundary spanning and knowledge brokering activities supplement each other and contribute to the flexibility required to enable the organisation to respond rapidly to environmental changes, while allowing the first operating system to focus on providing efficiency and stability in core operations. thus, when part of a dual operating system, boundary spanning and knowledge brokering help balance flexibility and stability and support organisational agility. the results therefore suggest that the boundary spanning and knowledge brokering activities undertaken by the pm team in the second operating system serve an important function for softcorp. through their enactment the pm team offers an important structure to help balance the demand for more flexibility in a dynamic and increasingly complex environment and the demand for stability that is needed for the efficient development of the core product. this is aligned with the paradoxical approach to managing agility advocated by lewis, andriopoulos, and smith (2014) which requires managing stability while having enough flexibility to be responsive to a rapidly changing situations. the boundary spanning and knowledge brokering activities of the pm team provide a mechanism through which softcorp can act more quickly and make sense of changes in its environment, at the same time shielding the core development team from these influences. importantly, the pm team’s dual roles of boundary spanning and knowledge brokering are both necessary for maintaining softcorp’s viability. this helps with keeping the core development team isolated in order to develop the cms in a stable, slow-paced manner and allows softcorp to react quickly to changes in its environment. this dual strategy of stable core product vs. fast modules enables softcorp to support organisational agility and is a basis for their success. our study presents an empirical contribution (ågerfalk, 2014) in the form of a novel account of an empirical phenomenon – organisational agility – that reveals how the balance between flexibility and stability, which characterises organisational agility, can be achieved by the boundary spanning and knowledge brokering activities undertaken by a dedicated unit operating alongside a traditional hierarchical structure. this has been previously undocumented. we reveal insights into this phenomenon, while we rely explicitly on the a priori conceptualisations of boundary spanning and knowledge brokering. this has three important theoretical implications for further research. first, previous literature argues for the importance of such specialised units for balancing flexibility and stability (e.g., aghina, de smet, & weerda, 2015; baškarada & koronios, 2018; harter & krone, 2001; kotter, 2012). the data reported here support these arguments. yet, in contrast to harter and krone (2001), who revealed the role of a cooperating support organisation acting as a boundary spanner and thereby helping the cooperative’s member organisations to balance change and stability, we found that our case organisation has a dedicated team inside its own organisation that contributes to balancing flexibility and stability by undertaking boundary spanning and knowledge brokering activities. as such, our work supplements the work of kotter (2012), who advocates to extend traditional australasian journal of information systems rosenkranz, kautz & cooper 2025, vol 29, research article organisational agility through boundary spanning and knowledge brokering 19 organisational units with a second operating system that uses an agile, network-like structure and different processes to assess the business, the industry, and the organisation, and that reacts with greater nimbleness, speed, and creativity than the existing units. kotter (2012), however, does not consider how knowledge in such structures is shared with the organisation, and does not go into detail on how such a structure operates and looks like. we provide evidence for such a possible structure in the form of a dedicated unit and illuminate how knowledge brokering and boundary spanning underpin this. in doing so, we show that a second operating system need not be a parallel network-like structure comprised of volunteers but can be a part of the formal organisational hierarchy (i.e., the first operating system), yet by the distinct nature of the activities undertaken, it is a second operating system. further research could investigate other structural configurations and examine the mutually reinforcing role of the concepts. second, while the literature provides evidence for boundary spanning and knowledge brokering as two separate sets of activities (e.g., levina & vaast, 2005; marrone et al., 2007; pawlowski & robey, 2004), we extend, based on our case, the common body of knowledge and establish that a unit that intertwines these activities can usefully contribute to organisational agility. having a dedicated unit in a second operating system, with members with complimentary knowledge and skill sets in order to be able to both span boundaries and broker knowledge between others, is a promising strategy for other companies that also want to achieve organisational agility. in previous work, fleming and waguespack (2007) studied brokering and spanning together, however in open innovation communities, which are different from individual commercial companies. we provide an empirical confirmation for their proposition that boundary spanning and knowledge brokering are closely related in such a setting, too. they also put forward that brokers can perform spanning activities and spanners can perform brokering activities but found that performing these activities simultaneously can have negative consequences with regard to the individuals’ roles in these communities. this might be related to the environment in which they performed their study and their focus on individuals. we do not find these negative effects; on the contrary, both sets of activities in our setting contribute to organisational agility without jeopardising the individuals involved. however, we did find drawbacks in the second operating system for the pm team as a whole, resulting from the increasing task variety and the resulting variability and uncertainty. our study did not explore the impact of these effects on organisational agility. further studies should analyse negative effects in more detail and could investigate whether promising strategies exist for mitigating negative effects while retaining positive ones. third, taking a starting point in individuals acting as knowledge brokers, others have also struggled with the conceptual distinction of knowledge brokering and boundary spanning (pawlowski & robey, 2004; wenger, 2000). as we focus on an organisational unit as a whole, this difficulty, as important as the distinction is for our analysis, does not jeopardise the results of our study. we provide some new insights about the characteristics of knowledge brokers and boundary spanners. the pm team as a whole performs boundary spanning and knowledge brokering activities. the pm team’s ability to act as a dual bridging unit crucially depends on the different and complementary skills of its members. reportedly, the pm team was established in light of the special skills required by its members to fulfill this crucial position within the company. moreover, both boundary spanning and knowledge brokering are conducted in relation to external partners and customers as well as to other internal units. thus, the pm team is neither only conducting internal knowledge brokering nor external australasian journal of information systems rosenkranz, kautz & cooper 2025, vol 29, research article organisational agility through boundary spanning and knowledge brokering 20 boundary spanning. while the literature provides evidence for boundary spanning and brokering as two separate sets of activities, which are performed by individuals (kimble, grenier, & goglio-primard, 2010; levina & vaast, 2005; marrone et al., 2007; neal et al., 2023;. pawlowski & robey, 2004; tortoriello et al., 2011; mahnke, wareham, & bjoern-andersen, 2008), we extend, based on our case, the common body of knowledge and establish that these activities beyond the individual level also take place in an integrated manner on the unit level. although the individuals are important, it is the unit as a whole, which makes the difference in our case organisation. more work must be done to understand the idiosyncratic and the common characteristics of the involved sets of activities. we focused on boundary spanning and knowledge brokering from the outset and did not employ other lenses for scrutinising our data, e.g., the role of boundary objects ( star, 2010; star & griesemer, 1989) or the role of different technology frames (leonardi & barley, 2010; orlikowski & gash, 1994) surrounding the various units, which may play a role in this setting as well. these offer a worthy subject for future research. as a contribution to practice, our findings may help other companies to decide whether having a dedicated knowledge brokering and boundary spanning unit in a second operating system is worthwhile in their context to enable organisational agility. having a second operating system rooted in a dedicated unit comprising members with complimentary skill sets and knowledge sets that can act as a knowledge reservoir to be able to span or broker, operating alongside a traditional organisational hierarchy, is a promising strategy for balancing flexibility with stability to achieve organisational agility. our findings indicate that the antecedent conditions for this are to have (a) individuals with skills and competence necessary for knowledge brokering and boundary spanning, and (b) a variety of tasks that are necessary to develop a broad set of knowledge. however, the role of such a unit comes with a cost. this is mostly observable in terms of the high uncertainty and variability in the pm team’s business processes, which led to bottlenecks and standstills in the software development process. it is also somewhat in conflict with a lean management approach as taken up for software development by the pm team, which usually has the ultimate goal of eliminating bottlenecks and eliminating waste (anupindi, chopra, deshmukh, van mieghem, & zemel, 2006; poppendieck & poppendieck, 2003). thus, we observe a trade-off: on the one hand, the wide diversity and variety of tasks allows the members of the pm team to acquire the skill set and the set of broad knowledge through both knowledge brokering and boundary spanning. without the tasks, the pm team would not interface as frequent or as much with other units and with customers or partners. on the other hand, the same variety leads to increased variability and a resulting increased chance for disruptions in the workflow of the pm team. the task variety leads to coordination problems, for example, visible in the necessity to assign time contingents to tasks or in the fact that it takes quite a while to develop a module for a team working in an agile way. even though process variation may be related to performance gains or even to organisational survival in dynamic, competitive environments where flexibility is important (pentland, 2003a), too much emphasis on too many tasks at once may lead to increases in variability and bottlenecks in the process flow (anupindi et al., 2006), and thus, in turn, may even compromise the intended strategy to use a dedicated team or unit such as the pm team to operationalise a second operating system be able to react more quickly and be flexible to changes in the environment (pentland, 2003a, 2003b). australasian journal of information systems rosenkranz, kautz & cooper 2025, vol 29, research article organisational agility through boundary spanning and knowledge brokering 21 6 references ågerfalk, p. j. (2014). insufficient theoretical contribution: a conclusive rationale for rejection? european journal of information systems, 23(6), 593-599. agerfalk, p. j., fitzgerald, b., & slaughter, s. a. (2009). introduction to the special issue-flexible and distributed information systems development: state of the art and research challenges. information systems research, 20(3), 317-328. doi.org/10.1287/isre.1090.0244 aghina, w., de smet, a., & weerda, k. (2015). agility: it rhymes with stability. retrieved from https://www.mckinsey.com/business-functions/organization/our-insights/agility-itrhymes-with-stability aldrich, h., & herker, d. (1977). boundary spanning roles and organization structure. academy of management review, 2(2), 217-230. alvesson, m., hardy, c., & harley, b. (2008). reflecting on reflexivity: reappraising practice. journal of management studies (45). 480–501. alvesson, m., & sandberg, j. (2013). has management studies lost its way? ideas for more imaginative and innovative research. journal of management studies (50), 128-152. ancona, d. g., & caldwell, d. f. (1990). beyond boundary spanning: managing external dependence in product development teams. the journal of high technology management research, 1(2), 119-135. ancona, d. g., & caldwell, d. f. (1992). bridging the boundary: external activity and performance in organizational teams. administrative science quarterly, 37(4), 634-665. doi:10.2307/2393475 ancona, d. g., & caldwell, d. f. (1998). rethinking team composition from the outside in. in d.h. grueneld (ed.), research on managing groups and team. vol 1, composition, 2137.amsterdam, the netherlands: elsevier. anupindi, r., chopra, s., deshmukh, s. d., van mieghem, j. a., & zemel, e. (2006). managing business process flows. principles of operations management (2 ed.). upper saddle river, nj, usa: pearson education. argote, l., & miron-spektor, e. (2011). organizational learning: from experience to knowledge. organization science, 22(5), 1123-1137. doi.org10.1287/orsc.1100.0621 baham, c., & hirschheim, r. (2022). issues, challenges, and a proposed theoretical core of agile software development research. information systems journal, 32(1), 103-129. doi.org/10.1111/isj.12336 baškarada, s., & koronios, a. (2018). the 5s organizational agility framework: a dynamic capabilities perspective. international journal of organizational analysis, 26(2), 331-342. doi.org/10.1108/ijoa-05-2017-1163 benbya, h., nan, n., tanriverdi, h., & yoo, y. (2020). complexity and information systems research in the emerging digital world. mis quarterly, 44(1), 1-17. bharadwaj, a.,. el sawy, o. a., pavlou, p. a., & venkatraman, n. (2013). digital business strategy: toward a next generation of insights. mis quarterly, 37(2), 471-482. australasian journal of information systems rosenkranz, kautz & cooper 2025, vol 29, research article organisational agility through boundary spanning and knowledge brokering 22 brynjolfsson, e., & mcafee, a. (2014). the second machine age: work, progress, and prosperity in a time of brilliant technologies. new york, ny, usa: w.w. norton & company. burns, t., & stalker, g. m. (1961). the management of innovation. social science paperbacks. london, uk: tavistock. cross, r., & cummings, j. n. (2004). tie and network correlates of individual performance in knowledge-intensive work. academy of management journal, 47(6), 928-937. doi.org10.2307/20159632 crupi, a., del sarto, n., di minin, a., gregori, g. l., lepore, d., marinelli, l., & spigarelli, f. (2020). the digital transformation of smes – a new knowledge broker called the digital innovation hub. journal of knowledge management, 24(6), 1263-1288. doi:10.1108/jkm-112019-0623 fleming, l., & waguespack, d. m. (2007). brokerage, boundary spanning, and leadership in open innovation communities. organization science, 18(2), 165-180. franco, m., guimarães, j., & rodrigues, m. (2023). organisational agility: systematic literature review and future research agenda. knowledge management research & practice, 21(6), 1021-1038. doi.org/10.1080/14778238.2022.2103048 gasson, s. (2006). a genealogical study of boundary-spanning is design. european journal of information systems, 15(1), 26-41. gopal, a., & gosain, s. (2009). the role of organizational controls and boundary spanning in software development outsourcing: implications for project performance. information systems research, 20(8), 1-23. haffke, i., kalgovas, b., & benlian, a. (2016), options for transforming the it function using bimodal it, mis quarterly executive, 16(2), 101-119. harris, m. l., collins, r. w., & hevner, a. r. (2009). control of flexible software development under uncertainty. information systems research, 20(3), 400-419. doi.org/10.1287/isre.1090.0240 harter, l., & krone, k. (2001). the boundary-spanning role of a cooperative support organization: managing the paradox of stability and change in non-traditional organizations. journal of applied communication research, 29(3), 248-277. retrieved from doi.org/10.1080/00909880128111 kautz, k., & nielsen, p. a. (2004). understanding the implementation of software process improvement innovations in software organizations. information systems journal, 14(1), 3-22. retrieved from doi.org/10.1111/j.1365-2575.2004.00156.x keszey, t. (2018). boundary spanners’ knowledge sharing for innovation success in turbulent times. journal of knowledge management, 22(5), 1061-1081. doi.org/10.1108/jkm-01-20170033 kimble, c., grenier, c., & goglio-primard, k. (2010). innovation and knowledge sharing across professional boundaries: political interplay between boundary objects and brokers. international journal of information management, 30(5), 437-444. kotter, j. p. (2012). accelerate! harvard business review, 90(11), 43-58. australasian journal of information systems rosenkranz, kautz & cooper 2025, vol 29, research article organisational agility through boundary spanning and knowledge brokering 23 kwon, s.-w., rondi, e., levin, d. z., de massis, a., & brass, d. j. (2020). network brokerage: an integrative review and future research agenda. journal of management, 46(6), 10921120. doi.org/10.1177/0149206320914694 laato, s., mäntymäki, m., islam, a. k. m. n., hyrynsalmi, s., & birkstedt, t. (2023). trends and trajectories in the software industry: implications for the future of work. information systems frontiers, 25(2), 929-944. doi:10.1007/s10796-022-10267-4 leonardi, p. m., & barley, s. r. (2010). what’s under construction here? social action, materiality, and power in constructivist studies of technology and organizing. academy of management annals, 4(1), 1-51. levina, n. (2005). collaborating on multiparty information systems development projects: a collective reflection-in-action view. information systems research, 16(2), 109-130. retrieved from 10.1287/isre.1050.0055 http://search.ebscohost.com/login.aspx? direct=true&db=buh&an=17527638&amp;lang=de&site=ehost-live levina, n., & vaast, e. (2005). the emergence of boundary spanning competence in practice: implications for implementation and use of information systems. mis quarterly, 29(2), 335-363. lewis, m. w., andriopoulos, c., & smith, w. k. (2014). paradoxical leadership to enable strategic agility. california management review, 56(3), 58-77. lu, y., & ramamurthy, k. (2011). understanding the link between information technology capability and organizational agility: an empirical examination. mis quarterly, 35(4), 931-954. mahnke, v., wareham, j., & bjoern-andersen, n. (2008) offshore middlemen: transnational intermediation in technology sourcing. journal of information technology, 23(1), 18-30. doi: 10.1057/palgrave.jit.2000124 marrone, j. a., tesluk, p. e., & carson, j. b. (2007). a multilevel investigation of antecedents and consequences of team member boundary-spanning behavior. academy of management journal, 50(6), 1423-1439. retrieved from http://amj.aom.org/content/50/6/1423.abstract mathiassen, l., & pries-heje, j. (2006). business agility and diffusion of information technology. european journal of information systems, 15(2), 116-119. retrieved from doi.org/10.1057/palgrave.ejis.3000610 miles, m. b., & huberman, a. m. (1994). qualitative data analysis: a sourcebook of new methods. beverly hills, ca, usa: sage. miles, m. b., huberman, a. m., & saldaña, j. (2014). qualitative data analysis: a methods sourcebook (3 ed.). london, uk: sage. mintzberg, h. (1979). structuring of organizations. london: uk: pearson. neal, j. w., posner, s., & brutzman, b. (2023). understanding brokers, intermediaries, and boundary spanners: a multi-sectoral review of strategies, skills, and outcomes. evidence & policy, 19(1), 95-95–115. doi.org/10.1332/174426421x16328416007542 australasian journal of information systems rosenkranz, kautz & cooper 2025, vol 29, research article organisational agility through boundary spanning and knowledge brokering 24 nguyen, t., le, c. v., nguyen, m., nguyen, g., lien, t. t. h., & nguyen, o. (2024). the organisational impact of agility: a systematic literature review. management review quarterly, 1-49. doi.org/10.1007/s11301-024-00446-9 orlikowski, w. j., & gash, d. c. (1994). technological frames: making sense of information technology in organizations. acm transactions on information systems (tois), 12(2), 174-207. overby, e., bharadwaj, a., & sambamurthy, v. (2006). enterprise agility and the enabling role of information technology. european journal of information systems, 15(2), 120-131. retrieved from 10.1057/palgrave.ejis.3000600. http://search.ebscohost.com/login.aspx? direct=true&db=buh&an=28042003&lang=de&site=ehost-live pawlowski, s., & robey, d. (2004). bridging user organizations: knowledge brokering and the work of information technology professionals. mis quarterly, 28(4), 645-672. pawlowski, s. d., robey, d., & raven, a. (2000, december 10-13). supporting shared information systems: boundary objects, communities, and brokering. paper presented at the icis 2000 proceeding, brisbane, australia. pentland, b. t. (2003a). conceptualizing and measuring variety in the execution of organizational work processes. management science, 49(7), 857-870. pentland, b. t. (2003b). sequential variety in work processes. organization science, 14(5), 528540. retrieved from http://orgsci.journal.informs.org/cgi/content/abstract/14/5/528 poppendieck, m., & poppendieck, t. (2003). lean software development: an agile toolkit (1st ed.). amsterdam, the netherlands: addison-wesley longman. ravichandran, t. (2018). exploring the relationships between it competence, innovation capacity and organizational agility. the journal of strategic information systems, 27(1), 2242. doi.org/10.1016/j.jsis.2017.07.002 reagans, r., & mcevily, b. (2003). network structure and knowledge transfer: the effects of cohesion and range. administrative science quarterly, 48(2), 240-267. doi:10.2307/3556658 sambamurthy, v., bharadwaj, a., & grover, v. (2003). shaping agility through digital options: reconceptualizing the role of information technology in contemporary firms. mis quarterly, 27(2), 237-263. retrieved from http://www.jstor.org/stable/30036530 shi, y., cui, t., & kurnia, s. (2023). value co-creation for digital innovation: an interorganizational boundary-spanning perspective. information & management, 60(5), 103817. doi.org/10.1016/j.im.2023.103817 sommerville, i., cliff, d., calinescu, r., keen, j., kelly, t., kwiatkowska, m., mcdermind, j.,. & paige, r. (2012). large-scale complex it systems. communications of the acm, 55(7), 71-77. doi:10.1145/2209249.2209268 star, s. (2010). this is not a boundary object: reflections of the origin of a concept. science, technology, & human values, 35, 601-617. doi:10.1177/0162243910377624 star, s., & griesemer, j. (1989). institutional ecology, 'translations' and boundary objects: amateurs and professionals in berkeley's museum of vertebrate zoology, social studies of science, 19 (3), 387–420. doi: 10.1177/03063128901900300 australasian journal of information systems rosenkranz, kautz & cooper 2025, vol 29, research article organisational agility through boundary spanning and knowledge brokering 25 tallon, p. p., & pinsonneault, a. (2011). competing perspectives on the link between strategic information technology alignment and organizational agility: insights from a mediation model. mis quarterly, 35(2), 463-486. tallon, p. p., queiroz, m., coltman, t., & sharma, r. (2019). information technology and the search for organizational agility: a systematic review with future research possibilities. the journal of strategic information systems, 28(2), 218-237. doi.org/10.1016/j.jsis.2018.12.002 teece, d., peteraf, m., & leih, s. (2016). dynamic capabilities and organizational agility: risk, uncertainty, and strategy in the innovation economy. california management review, 58(4), 13-35. doi.org/10.1525/cmr.2016.58.4.13 tortoriello, m., reagans, r., & mcevily, b. (2011). bridging the knowledge gap: the influence of strong ties, network cohesion, and network range on the transfer of knowledge between organizational units. organization science, published online before print doi:10.1287/orsc.1110.0688 tushman, m. l. (1977). special boundary roles in the innovation process. administrative science quarterly, 22(4), 587-605. tushman, m. l. (1979). work characteristics and subunit communication structure: a contingency analysis. administrative science quarterly, 24(1), 82-98. vial, g. (2019). understanding digital transformation: a review and a research agenda. the journal of strategic information systems, 28(2), 118-144. doi.org/10.1016/j.jsis.2019.01.003 walter, a.-t. (2021). organizational agility: ill-defined and somewhat confusing? a systematic literature review and conceptualization. management review quarterly, 71(2), 343-391. doi:10.1007/s11301-020-00186-6 wenger, e. (1998). communities of practice: learning, meaning, and identity. cambridge, uk: cambridge university press. doi.org/10.1017/cbo9780511803932 wenger, e. (2000). communities of practice and social learning systems. organization, 7(2), 225-246. retrieved from http://org.sagepub.com/content/7/2/225.abstract wessel, l., baiyere, a., ologeanu-taddei, r., cha, j., & jensen, t. b. (2021). unpacking the difference between digital transformation and it-enabled organizational transformation. journal of the association for information systems, 22(1), 6. doi:10.17705/1jais.00655 copyright copyright © 2025 rosenkranz, c., kautz, k. & cooper, v. this is an open-access article licensed under a creative commons attribution-non-commercial 4.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. australasian journal of information systems rosenkranz, kautz & cooper 2025, vol 29, research article organisational agility through boundary spanning and knowledge brokering 26 doi: https://doi.org/10.3127/ajis.v29.5095 https://doi.org/10.3127/ajis.v29.5095 paper title (paper title style) australasian journal of information systems lam, heales, hartley & hodkinson 2020, vol 24, selected papers from acis 2018 consumer trust in food safety 1 consumer trust in food safety requires information transparency tri lam uq business school the university of queensland t.lam@business.uq.edu.au jon heales uq business school the university of queensland nicole hartley uq business school the university of queensland chris hodkinson uq business school the university of queensland abstract this paper proposes a conceptual model to understand how information transparency matters can support consumer trust in food safety. beside food labels, food product information can be disseminated by the support of technologies including traceability systems and social media. this article studies extant literature to provide a knowledge base for the development of a conceptual model. information provided by traceability systems is deemed to increase a consumer’s knowledge of a food product. furthermore, social media is considered as a wellinformed source that provides some useful information to consumers. therefore, we argue that technology-supported information supports and enhances the information consumers need to make their own judgement about the safety of a food product. three testable propositions are developed from a conceptual model that provides insights into food information that consumers find helpful for developing trust in food safety. keywords: tracing systems, social media, consumer trust. 1 introduction is information along food supply chain from farm to fork beneficial in increasing consumer trust in food safety? this question is posed when scandalous food safety issues are reported and cause public concern (marucheck, greis, mena, & cai, 2011). the association between risk management of the food supply chain and food safety has received scholars’ attention (de boeck, jacxsens, bollaerts, uyttendaele, & vlerick, 2016; h. l. lee & whang, 2005; marucheck et al., 2011; pyke & tang, 2010; tang, 2008). disruptions to the food supply chain can threaten food safety especially where food safety incidents are vulnerable to a global supply chain (narasimhan & talluri, 2009). the provision of sufficient food product information on the labels and the production process is all related to perceived product safety (marucheck et al., 2011). therefore, legislated standards, quality management, and tracking management are helpful for tackling product safety issues as well as reducing negative consequences to consumers (marucheck et al., 2011). australasian journal of information systems lam, heales, hartley & hodkinson 2020, vol 24, selected papers from acis 2018 consumer trust in food safety 2 indeed, food safety scandals have occurred and reoccurred regularly across countries and local areas in the world. table 1 shows some serious food safety incidents occurring between 1996 and 2014. food safety incident country / area year the relevance of the examples of food safety scandals and information transparency supporting literature mad cow disease britain 1996 infected meat and bonemeal were used to feed cattle. therefore, the governments have required producers to use traceability systems in order to ameliorate monitoring process and manage information sources. berg, (2004) commercial feed contained high concentrations of the toxic chemical dioxin germany 2010 toxic animal feed was used to feed livestock and poultry. hence, consumers require additional product-related information before food purchase. rieger, kuhlgatz, & anders, (2016) phthalate-tainted food scandal taiwan 2011 phthalates were purposefully added to several kinds of food in taiwan. therefore, taiwan food and drug administration has required relevant food producers to set up their own traceability systems for monitoring process. tsai et al., (2016) using expired meat china 2014 stinky and expired meat was mixed with non-expired meat. hence, production records are useful for regaining consumer trust. xie & yao, (2016) waste oil, recycled oil, and feed oil scandal taiwan 2014 contaminated cooking oil was intentionally labelled as high-quality cooking oil. therefore, taiwan food and drug administration has regulated the use of food traceability systems to collect information regarding ingredients and materials. ko, (2015) table 1 some serious food safety incidents in september, 2018, the strawberry needle tampering crisis occurred in australia when needles were found inside punnets of strawberries (wikipedia, 2019). the source of the contaminated strawberries was unknown. supermarkets and distributors across australia and new zealand had to dump strawberries because supermarkets and distributors would not accept the risk that the strawberries may contain needles. many farmers went out of business as a result of this product recall and strawberries were withheld from sale for several weeks during the investigation. the strawberry scandal badly damaged the 160-million dollar strawberry industry in australia as well as the australian strawberry export business (bavas, 2018). food standards australia new zealand has proposed the use of traceability systems along the supply chain to communicate food product information to the public through social media (the food standards australia new zealand, 2018) in an effort to restore consumer trust in food products. as a result of these instances, it is debatable whether information retrieved from food traceability systems and online social networking websites is enough to restore consumer trust. australasian journal of information systems lam, heales, hartley & hodkinson 2020, vol 24, selected papers from acis 2018 consumer trust in food safety 3 because many food consumers rely on detailed food information to make food choices (chen, 2011), providing food product information from the food supply chain can enhance consumers’ perspectives on food quality and safety (lehmann et al., 2011). traceability systems enable consumers to trace back food product information from raw materials to final products (australia new zealand food authority, 2017). we pose the first question that is: “how does food product-related information provided by food traceability systems influence consumer trust intentions?” trust intention refers to a person’s willingness to count on something or someone (morgan & hunt, 1994). online social networking communication about food safety issues between food consumers is also influential in shaping consumer trust (mou & lin, 2014), so, we raise the second question that is: “how does online social networking food product-related information influence consumer trust intentions?” this paper continues the studies on the major elements that affect consumer trust in food safety by assimilating the extant information systems management and consumer behaviour literature paradigm. this paper includes the study into previous literature about food safety, traceability systems, online social networking information, and consumer trust. social cognitive theory (sct) is employed in developing the conceptual model. the rationale is that when consumers obtain knowledge about food products through traceability systems and social media, their levels of trust in the safety of a food product will change accordingly. finally, theoretical and practical implications are discussed in last section. 2 literature review 2.1 previous literature in relation to food safety management, information systems refer to data collection, storage, assessment and retrieval (mcmeekin et al., 2006). information systems influence and frame the way users make decisions about the safety of food (mcmeekin et al., 2006). a hazard analysis of critical control points (haccp) is an organised and well accepted approach that carefully documents all information and actions on food processing and operations necessary for a food safety evaluation (mcmeekin et al., 2006). information systems can facilitate the collection and documentation of haccp initiatives. szymanska (2015) also suggested that electronic information systems were a tool supporting food processing at all stages of formation and supervision. the computer-based tool provides information about farmers and food producers to customers and shows a product’s origin and composition (szymanska, 2015). thus food safety management systems (fsmss) instruments, that provide food producers with information via reports of authentication and corroboration, have been developed (kirezieva, jacxsens, uyttendaele, van boekel, & luning, 2013). fsmss provide accountability for both the origin of food and transparency in its processing (motarjemi & mortimore, 2005). food safety management and the control of risks should be approached proactively through efficient fsmss (zwietering, jacxsens, membré, nauta, & peterz, 2016). swoffer (2009) indicated that food safety management included raw materials management, usage delineation, and the scrutiny of final products. however, product safety has been perceived as also resulting from technical issues such as risks from flawed operational processes that may cause negative consequences for internal and external stakeholders (lewis, 2003) – table 2 details stakeholders in the food processing and supply chain. australasian journal of information systems lam, heales, hartley & hodkinson 2020, vol 24, selected papers from acis 2018 consumer trust in food safety 4 stakeholders role information relevant to food processing and supply chain supporting literature food producers ensuring consumers that food is safe information about food processing and its supply chain motarjemi & mortimore, (2005); govindan, (2018) consumers bodies or organisations providing guidance information about guidance on food safety motarjemi & mortimore, (2005); govindan, (2018) governments formulating, implementing and administering policies on food safety information about the occurrence of any foodborne illness and other food safety issues motarjemi & mortimore, (2005); govindan, (2018) governments communicating and educating information on up-to-date foodborne issues via mass media and on the internet motarjemi & mortimore, (2005); govindan, (2018) various middlemen including distributors, wholesalers and retailers coordinating with manufacturers and consumers information about actions such as distribution and storage motarjemi & mortimore, (2005); manders, caniëls, & ghijsen, (2016) consumers consuming food products information concerning blame and/or experience barbarossa, de pelsmacker, moons, & marcati, (2016); manning (2015) consumers information about discussions on food safety and risk issues mou & lin, (2014); wu (2015) table 2 the extant literature on stakeholders in the food processing and supply chain from a safety perspective, food safety has causal relationships with food supply chains that typically have numerous vulnerabilities, e.g. warehouse and transportation management (whipple, voss, & closs, 2009). as a result, the analysis of information concerning fsmss in a produce chain is likely to provide insights about quality assurance, and elicit appropriate responses to product safety concerns (kirezieva et al., 2013). the fundamental component of fsmss is a traceable system that collects information about all activities involved in the production and distribution of a food product. 2.2 traceability systems and trust traceability systems refer to techniques for identification, provision, collection, storage and corroboration of data (starbird & amanor-boadu, 2006). traceability systems integrate information about product supply, transactions and logistics with the support of technical components such as database, information carriers, information systems and methods of information provision (engelseth, 2009). due to globalisation, food supply chains necessitate traceability systems to help ensure safety and identify failures in food processing. as a result, many countries have imposed a requirement for traceability systems (king et al., 2017). food traceability systems require producers to properly record information regarding manufacture and distribution in order for consumers to trace back the production processes and supply chain of ingredients (k. zhang, chai, yang, & weng, 2011). the recorded information consists of cultivation processes, source of origin, chemicals used, and other inputs to the production process and supply chain (choe, park, chung, & moon, 2009; hall, 2010; liao, chang, & chang, 2011). australasian journal of information systems lam, heales, hartley & hodkinson 2020, vol 24, selected papers from acis 2018 consumer trust in food safety 5 the benefits of traceability include ensuring the safety and quality of products (mattevi & jones, 2016), and ameliorating food supply chain performance by monitoring information about potential vulnerability (wang & huang, 2010). traceability systems can help diagnose problems and provide information about suppliers to competent authorities and customers (mcmeekin et al., 2006). in addition, traceability systems assist in tracking potentially unsafe products in order to take timely preventative and corrective actions (jansen-vullers, van dorp, & beulens, 2003). therefore, traceability systems help reduce information asymmetry between producers and consumers as well as supporting their purchase decisions (chen & huang, 2013) because information asymmetry increases consumers’ perceived risk (yoo, parameswaran, & kishore, 2015). information provided by traceability systems throughout the food processing and supply chain can be disclosed to stakeholders by mobile applications (cagliano, de marco, & rafele, 2017). as a result, information transparency through providing timely and reliable information impacts consumers’ perceived knowledge of a food product (li, liu, liu, lai, & xu, 2017). when information provided by traceability systems increases consumers’ knowledge about the safety of a food product it either aids in restoring consumer trust in the food product if the information is positive (bánáti, 2014), or else decreases trust in the food product. the retention of transparency in a food chain enhances consumers’ confidence in food safety (chiu, 2016). 2.3 food product communication and online social networking yiannas (2009) recommended the use of multiple forms of media for communicating food safety information such as leaflets, videos and websites. a variety of information sources exert a positive and significant influence on consumer trust (liu, pieniak, & verbeke, 2014). consumers’ trust in food information communicated by public organisations is greater than by private associations (nocella, romano, & stefani, 2014). for example, l. zhang, xu, oosterveer, and mol (2016) found that government was the most trustworthy source of information concerning food safety, compared to private certification schemes, e.g. the safe quality food (sqf) program, a civil-society organisation. in addition, promotional materials related to food safety and health risks increase consumers’ awareness of food safety issues (verçuni et al., 2016). online social networking websites have provided users with a unique platform for sharing and discussing food safety information (mou & lin, 2014). online social networking (osn) refers to social interactions between individuals about their everyday experiences by providing a venue with appropriate online tools (merchant, 2012). according to wu's (2015) findings, social media users having positive emotions and concerns about food safety issues tend to search for food safety information and learn from others. the use of social networking websites for risk consultation pertinent to food safety issues is influenced by risk awareness, sentiment, social trust, and social assistance factors (wu, 2015). as such, information and discussions about food products and food safety matters between consumers through social media help to develop trust intentions in food products. 2.4 consumer trust from social theory perspective, the underlying assumption of trust is the belief of a trustor (e.g. food consumers) in a trustee (e.g. food producers) who has competence in satisfying a trustor’s requirements honestly in a given context (grandison & sloman, 2003). according to australasian journal of information systems lam, heales, hartley & hodkinson 2020, vol 24, selected papers from acis 2018 consumer trust in food safety 6 chen (2008), actors in a food system, monitoring bodies, and truth telling are determinants in consumer trust in food safety. de jonge, van trijp, renes, and frewer (2010) found that optimism and pessimism about food safety were strongly related to trust in food manufacturers. a recent study also shows that perceived country of origin is positively associated with consumers’ perception of the reliability and controllability of food incidents, thus lessening consumers’ ascription of untrustworthy brands (barbarossa et al., 2016). another approach developed by lassoued and hobbs (2015) indicates that consumers’ perceived brand competence is positively associated with brand trust, i.e. confidence in the safety and quality of food products. a summary of the existing literature on consumer trust in food safety is in table 3. theory & literature independent variable(s) dependent variable(s) supporting literature from culture perspective 1. actors in a food system 2. monitoring bodies 3. truth telling consumers’ trust in food safety chen, (2008) consumer confidence in the safety of food trust in food manufacturers optimism and pessimism about food safety de jonge et al., (2010) the combination of attribution theory, country of origin, and national stereotypes frameworks perceived country of origin consumers’ attributions of blame barbarossa et al., (2016) perceived brand competence perceived brand competence for quality and safety in food brand trust lassoued & hobbs, (2015) table 3 the extant literature on consumer trust in food safety table 3 illustrates that consumer trust in food safety has been subject to different approaches including national culture and marketing perspectives. however, the concept of trust is multidimensional, and providing truthful information is generally regarded as a prominent characteristic of trust (frewer, howard, hedderley, & shepherd, 1996). 2.5 an extended search from international peer-reviewed journals this study adopted a systematic review strategy that synthesised the extant literature and provided a scholarly critique of current literature (kekäle, de weerd-nederhof, cervai, & borelli, 2009). the above supporting literature about consumer trust is from the international peer-reviewed journals of risk analysis, food quality and preference, and food policy. according to the scimago journal and country rank portal, risk analysis has h-index 105 and is ranked q1 in safety, risk, reliability and quality. food quality and preference has h-index 89 and is ranked q1 in food science. food policy has h-index 76 and is ranked q1 in development and food science. the h-index articulates the number of articles (h) in one journal that has at least h citations. it indicates journal scientific productivity and impact (please see h-index wikipedia definition). quartile 1 (q1) is the quarter of the journals with the highest values. journals in q1 are cited much more and by more high-ranked journals than those in other quartiles. to ensure that all relevant high-quality research forms the foundation of this research project, an extended search was conducted on three journals: risk analysis, food quality and preference, australasian journal of information systems lam, heales, hartley & hodkinson 2020, vol 24, selected papers from acis 2018 consumer trust in food safety 7 and food policy. those journals are peer-reviewed, scholarly journals with english, and full text articles. figure 1 outlines the screening process undertaken for identifying articles relevant to this research topic. figure 1. the screening process the outcomes of the extended search for relevant articles is presented in table 4. keyword search 6 articles 1st elimination: 4 articles review of abstracts review of paper content does the abstract mention any findings related to food safety, traceability, online feedback, or consumer trust? does the content indicate the rationale for the study, methods and results related to food safety, traceability, online feedback, or consumer trust? 2nd elimination: 0 articles 6 articles no 10 articles no yes yes australasian journal of information systems lam, heales, hartley & hodkinson 2020, vol 24, selected papers from acis 2018 consumer trust in food safety 8 journal keyword search the number of articles the number of articles after reading abstract the number of articles after reading content 1 risk analysis trust food information risk traceability systems online feedback 0 trust food information risk 2 1 1 2 food quality and preference trust food information risk traceability systems online feedback 3 2 2 3 food policy trust food information risk traceability systems online feedback 5 3 3 total 10 6 6 table 4 the outcomes of extended search for relevant articles the rationale for the study, “what determines trust in information about food-related risks? underlying psychological constructs” (frewer et al., 1996), is that the reliability of information sources is the major dimension in establishing trust. that is, communicator’s competence and trustworthiness (mcguire, 1985). methods: first, a semi-structured interview was used for obtaining basic constructs forming trust or mistrust in a variety of sources revealing food risk information (n = 35). second, the repertory grid interviewing technique was used to obtain information about terminology provided by respondents when they mentioned food information sources (n = 39). finally, a survey was carried out to validate the constructs (n = 888). factor analysis suggests that well-researched information sources are trusted, and distorted/wrong/biased information sources are discredited. in addition, the results show that information provided by trustworthy groups should be understandable. when one information source interacts with other trustworthy sources, it will increase trust in the public arena. the rationale for the study, “preferred information strategies for food allergic consumers. a study in germany, greece, and the netherlands” (voordouw et al., 2011), is that information and communication technologies (ict) are likely to ameliorate information supply for food allergic consumers (cornelisse-vermaat, voordouw, yiakoumaki, theodoridis, & frewer, 2008). an ict tool is able to provide food allergic consumers with product-related information flexibly and personally (hu, grbich, & kemp, 2007). methods: an online survey was carried out to collect respondents’ preferences in terms of three information delivery tools including food label, booklet and an ict application (n = 287 participants including 55 in germany, 65 in greece, and 167 in the netherlands). conjoint analyses were conducted. the findings indicate that an ict tool is not able to replace food labels as a food-related information source, but is considered as a complimentary tool to supply food product information. specifically, an ict scanner is helpful in showing all ingredients in detail, including audio and visual notifications, and glossary explanation. the rationale for the study, “optimising the delivery of food allergy information. an assessment of food allergic consumer preferences for different information delivery formats” australasian journal of information systems lam, heales, hartley & hodkinson 2020, vol 24, selected papers from acis 2018 consumer trust in food safety 9 (voordouw et al., 2012), is that confidence in truthful food-related information is important to the public (frewer et al., 1996). confidence is defined as consumer trust in food product’s safety to a certain degree (de jonge, van trijp, van der lans, renes, & frewer, 2008). consumer assessment of product quality is based on information or declarations before consumption. there are other findings showing that to some extent an information communication technologies (ict) application to deliver product-related information is useful to help determine an individual’s purchase-making decision (best, 2009). methods: an online self-reported questionnaire survey was conducted by food allergic consumers in the netherlands, and germany (n = 62 participants including 24 in germany, and 38 in the netherlands) to assess their preferences for different types of information delivery tools. the results demonstrate that food labels provide the highest perceived functionality. an electronic scanner for food traceability was perceived as the second most functional tool used for information delivery. the least preferred information delivery tool was a booklet provided by a 3rd-party following european union legislation. consumers appreciate obvious and unambiguous information on food labels, and detailed, informative information provided by an ict scanner to enhance consumer trust in food products. the findings suggest that the functionality of food labels, electronic scanners, and booklets help determine consumers’ preferences rather than the reliability of information provided. the rationale for the study, “social media as a useful tool in food risk and benefit communication? a strategic orientation approach” (rutsaert et al., 2014), is that communication is supposed to build trust, achieve consensus, build awareness, provide somebody with education that influences one’s perception, attitude, and belief and/or changes one’s behaviour (mcgloin, delaney, hudson, & wall, 2009). social media is applicable to communicate food risk and benefit with the audience, as well as interact with the public (rutsaert et al., 2013). methods: first, in-depth interviews, regarding the potential of social media use for food risk and benefit communication, were conducted with media representatives, food producers and retailers (n = 38), along with regulatory authorities and scientific professionals (n = 33) in belgium, ireland, italy, latvia, spain and the netherlands. second, participants (10 food chain stakeholders and 13 experts) were subject to an in-depth interview stage to give scores for strengths, weaknesses, opportunities and threats of social media in communicating food risk and benefit. the results confirm that social media has the potential for communicating food risk and benefit. the advantages of social media are accessibility, rapidity and interaction. on the other hand, the disadvantages of social media are not having an information filter, the credibility of information supply, and information overload. the rationale for the study, “food and nutrition labelling in thailand: a long march from subsistence producers to international traders” (rimpeekool et al., 2015), is that food labels provide consumers with information about safety (marks, 1984) and nutrition (taylor & wilkening, 2008a, 2008b). methods: legal documents about food labelling in thai were collected from the royal thai government gazette e-database. next, thai document collections was extended to the thai food and drug administration (thai fda) e-library to search for published articles. finally, english documents were searched in international databases including sciencedirect, medline, and scopus. all data collected was categorised into six sections: (1) thai food industry and legal provisions, (2) early stage of food labelling, (3) contemporary food labelling, (4) nutrition labelling, (5) the challenges of thai nutrition labelling, and (6) international tensions. the results show that the international codex australasian journal of information systems lam, heales, hartley & hodkinson 2020, vol 24, selected papers from acis 2018 consumer trust in food safety 10 alimentarius guidelines underlie thai nutrition label policy. a shift in thai food label policy has come along with economic development and social change. international trading policies have constrained thai food and nutrition label regulations and the government is giving priority to international ‘‘fair trade” in its regulations. however, the positive impact of guidelines on food labelling regarding dietary behaviour and consumers’ health has remains elusive. the rationale for the study, “citizens, consumers and farm animal welfare: a meta-analysis of willingness-to-pay studies” (clark, stewart, panzone, kyriazakis, & frewer, 2017), is that the public are interested in information about the food chain of specific food products (jensen, 2006). therefore, farm animal welfare (faw) policies are needed and preferred by the public (bennett, butterworth, jones, kehlbacher, & tranter, 2012; farm animal welfare committee, 2014). as the public drive demand for faw standards, consumer willingness-to-pay (wtp) is proposed as a proxy for consumers’ attitude that is assessed (ryan & spash, 2011). in addition, wtp is an indicator of public preferences (harvey & hubbard, 2013) for different faw standards. wtp refers to price premium/maximum price that a consumer is willing to pay (breidert, hahsler, & reutterer, 2006; hanley, shogren, & white, 2001) to avoid any other risk. methods: relevant articles about wtp for faw and animal production diseases were searched on multiple databases. there were 54 congruous articles after the screening process. finally, a random effects meta-analysis and cumulative meta-analysis were conducted to investigate heterogeneity and explore changes in wtp chronologically. the results demonstrate that a small, positive wtp for faw standards diverging in relation to animal type and region. sociodemographic characteristics (age, income, gender and education) account for substantial variation in wtp. an amalgamation of market-based and government-based policy solutions is considered as the most effective solution for enhancing faw standards. because only 4 studies of the 54 examine specific production diseases in relation to reduced wtp, the evidence is not strong enough to conclude that consumers are willing to pay for higher faw products. overall, information sources and communication tools have potential for affecting consumer trust (frewer et al., 1996; rutsaert et al., 2014). consumers prefer food labels to electronic scanners and booklets for obtaining food-product-related information (voordouw et al., 2012, 2011). international standards and guidelines are applied to national food labelling regulations (rimpeekool et al., 2015). animal type and region are associated with consumers’ positive willingness-to-pay for farm-animal-welfare-standardised products (clark et al., 2017). however, almost most of those articles are exploratory studies. the authors have not applied any theories to explain the causal relationships between constructs. in particular, the authors have not challenged current theories about consumer trust in food safety from the perspective of information systems management and consumer behaviour. judgements about the articles are made as follows. australasian journal of information systems lam, heales, hartley & hodkinson 2020, vol 24, selected papers from acis 2018 consumer trust in food safety 11 article judgement 1 what determines trust in information about food-related risks? underlying psychological constructs” (frewer et al., 1996) information sources are associated with consumer trust. semi-structured in-depth interviews were used to obtain individuals’ perspectives. after describing individual perspective by interviews, a questionnaire survey was used to describe the population’s characteristics by factor analysis. this study finds that well-researched information sources and multi-trustworthy sources are in association with consumer trust. however, the study has not indicated what food-product information concerns consumers most and whether causal relationships exist between information sources and consumer trust. 2 “preferred information strategies for food allergic consumers. a study in germany, greece, and the netherlands” (voordouw et al., 2011) ict is believed to improve information provision for food allergic consumers. the overall ratings show that food labels are preferred as the main tool for information provision; ict is preferred as an additional tool providing users with detailed information, audio and visual warnings, and glossary elaboration. the study corroborates that an ict scanner is the second tool used by consumers. the study also explains why consumers use ict scanners as a complimentary tool. nevertheless, plenty of food-processing and supply-chain-related information provided by ict such as traceability systems has not been examined in this study. 3 “optimising the delivery of food allergy information. an assessment of food allergic consumer preferences for different information delivery formats” (voordouw et al., 2012) information about food products is linked to individuals’ perspective on food safety. descriptive statistics identify that food labels are the most preferred; secondly, electronic scanners are preferred; and finally booklets are preferred the least. a food label is preferred because it provides direct and basic food product information. electronic scanners are appreciated because they provide detailed and additional food product information. booklets provide general and explanatory food product information. the study identifies food allergic consumers’ preferences between three types of information delivery tools. however, the relationship between food-productrelated information delivery tools and consumer trust has not been showed. australasian journal of information systems lam, heales, hartley & hodkinson 2020, vol 24, selected papers from acis 2018 consumer trust in food safety 12 article judgement 4 “social media as a useful tool in food risk and benefit communication? a strategic orientation approach” (rutsaert et al., 2014) communication relates to building trust that impacts consumer’s attitude and behaviour. social media is deemed an effective communication tool that conveys food risks and benefits to the public. in-depth interviews were conducted, and scores were given for strengths, weaknesses, opportunities and threats of social media. the study points out some advantages and disadvantages of using social media for communicating food risk and benefit. nonetheless, consumers’ perceptions of food-product-related information on social media and the influence of social media communication on consumer trust have not been adequately examined. 5 “food and nutrition labelling in thailand: a long march from subsistence producers to international traders” (rimpeekool et al., 2015) food labels are regulated to provide food product information about safety and nutrition to consumers. textual analysis of legal documents and published articles was undertaken. the findings confirm that thai food label policy complies with the international codex alimentarius guidelines. however, the causal relationships between food information and consumer trust have not been examined in the study. 6 “citizens, consumers and farm animal welfare: a meta-analysis of willingness-to-pay studies” (clark et al., 2017) the public is concerned about information on the food supply chain and farm animal welfare. consumers are willing to pay for food products that follow faw standards. meta-analysis is used to identify random effects. the study demonstrates that animal type and region are related to small, positive willingness-to-pay. although willingnessto-pay is a proxy for consumers’ attitude and preferences for farm-animal-welfare-standardised products, other important elements such as perceived food supply chain information concerning farm animal welfare, and the reliability of the food production process from farm to fork have not been investigated in this study. table 5 judgements about the articles 2.6 an extended search on scopus database after the search in 3 journals with 6 relevant articles, an extended search on a broader set of journals on scopus database was conducted. scopus database is the largest abstract database of peer-reviewed scientific journals, books and conference proceedings (elsevier, 2018). scopus distributes the most complete overview of the world’s research output including science, technology, medicine, social sciences and arts and humanities. the scopus database contains of over 71 million records, over 23,700 peer-reviewed journals, and over 8.3 million conference papers (elsevier, 2018). the outcomes of the extended search for relevant articles on the scopus database and screening process are showed in table 6. based on a keyword search, there are 11 relevant articles. after the screening process, there remains one peerreviewed article that is pertinent to business, management and accounting. australasian journal of information systems lam, heales, hartley & hodkinson 2020, vol 24, selected papers from acis 2018 consumer trust in food safety 13 keyword search the number of articles the number of articles after reading abstract the number of articles after reading content 1 keyword: trust food information risk traceability systems online feedback 0 2 keyword: trust food information transparency risk 11 screening process 3 subject area: business, management and accounting (categorised by scopus) 2 4 document type: peerreviewed articles (categorised by scopus) 1 1 1 total 1 1 1 table 6 the screening process and the outcomes of extended search on scopus database the rationale for the study, “nanotechnology: legal and ethical issues” (baran, 2016), is based on the use of nanotechnology in the food industry (matsuura, 2006). to ensure that nanofood is safe for consumption, there need officially authorised guidelines and explanations issued by experts and authorities (brazell, 2012). the purpose is to legislate and protect consumers. moreover, ethical aspects should be considered in nanofood (sandler, 2009). the author wrote a conceptual paper by providing the arguments about nanotechnology regulation, potential risks, and standards of conduct. the article indicates some problems with consumer trust, potential risks, environmental impact, and information transparency before applying nanotechnology to foodstuffs. judgement: the article has pointed out that consumer trust, risks, and information transparency play a key role in nanofood acceptance. however, the article did not mention any theory and framework to examine the influence of information transparency on consumer trust. 3 conceptual development food product-related information comes from a wide variety of sources and channels, especially with the support of the internet and social media. this article aims to develop a conceptual model to predict cognitive perceptions which follow the associations between individual perceived knowledge, social information, and attitude intentions via social cognitive theory (sct). sct focuses on cognitive factors (knowledge about a particular fact gained by each individual), environmental elements (facilities and tools designed for a person’s eagerness for certain behaviours), and behavioural capabilities (a person’s behaviour shaped by their beliefs, attitudes, and perception) that influence each other (bandura, 1986). sct suggests that personal perceived behaviour (intentions), beliefs (perceive knowledge) and facilities (social information) have interactive associations. based on an sct perspective, perceived knowledge of food product provided by food traceability systems, food product-related discussion facilitated by social media, and consumer trust intentions interact with each other. a conceptual model is proposed as follows, which focuses on the use of traceability systems and osn food product-related information australasian journal of information systems lam, heales, hartley & hodkinson 2020, vol 24, selected papers from acis 2018 consumer trust in food safety 14 that support consumers’ trust intentions in food safety. the following subsections will refer to the derivations of the propositions. figure 2. conceptual model 3.1 perceived knowledge of food product provided by traceability systems and osn food product-related information derived from sct, a person’s perceived knowledge is in association with his/her social information needs (bandura, 1986). to diminish food contamination and incidents, firms and partners connect each other by accessing reliable and well-timed information concerning supply chain disruptions and outcomes (premkumar, ramamurthy, & saunders, 2005). for instance, the information about date, health benefits, nutrition information, country of origin, and ingredients on food packaging can be communicated to consumers with the intention of increasing customer trust (chan, kam, coulthard, & button, 2012). the above food-related information is managed by fsmss (kirezieva et al., 2013). based upon such product information, consumers’ knowledge of food products is formed. a person’s rational reactions after acquiring knowledge of a food product, e.g. knowledge of benefits and risks, are impacted by his/her cognitive responses (de vocht, cauberghe, uyttendaele, & sas, 2015). when consumers have more knowledge about food products, they are more enthusiastic about food products (de steur, liqun, van der straeten, lambert, & gellynck, 2015). in this study, perceived knowledge of a food product refers to information about a product perceived by consumers because timely and trustworthy information results in increased trust (morgan & hunt, 1994). in contrast, negative information decreases levels of trust. traceability system is technology that provides food-product-related information to consumers that influences consumer trust (mattevi & jones, 2016). the application of traceability systems helps track and monitor farm produce, as well as transferring food chain information to stakeholders (ruiz-garcia & lunadei, 2011). nonetheless, to lower perceived risks, consumers require more information to support their food safety expectations, which, in turn, influences their opinions towards food products and their subsequent behaviour (eiser, miles, & frewer, 2002). with the support of the internet, communication is easier for consumers who can search for information through online channels (j. lee, kim, & moon, 2000). statements made by consumers on the internet and related to a product or corporate brand are considered as electronic word-of-mouth (ewom) communication (stauss, 2000). word-of-mouth (wom) refers to information communications between informal parties as regards assessment of goods and services (singh, 1988; westbrook, 1987). electronic word-of-mouth communication refers to and includes the following: ‘writing’, ‘liking’, ‘sharing’, ‘recommending’, ‘commenting on’, and ‘tweeting’ brand-related messages on facebook, twitter, and other social media (wolny & mueller, osn food product-related information p1 perceived knowledge of food product provided by food traceability systems trust intentions p2 p3 australasian journal of information systems lam, heales, hartley & hodkinson 2020, vol 24, selected papers from acis 2018 consumer trust in food safety 15 2013). these constitute nuanced measures of wom from a giver’s perspective about goods and services for a particular firm (harrison-walker, 2001). notably, online communication presents a large volume of information that is disseminated in cyberspace, which has no geographical limitations, and many-to-many online relationships (wolny & mueller, 2013). therefore, when consumers have concerns about the safety of food, they search for osn food product-related information on popular social networking websites. osn food productrelated information refers to information about food products provided by other social media users (wu, 2015). these arguments lead to the proposition that is: p1: a high degree of perceived knowledge of food products provided by traceability systems will be associated with osn food product-related information. 3.2 perceived knowledge of food product provided by traceability systems and trust intentions the second association proposed by bandura (1986) is between a person’s perceived knowledge and behavioural intensions. consumer trust in food safety is created when they think food items are safe to eat (chen, 2008). food is safe when it will not cause sickness or harm to people who consume it, provided that the food is consumed as instructed (australia new zealand food authority, 2016). as mentioned in table 3, the food supply chain, government involvement, optimism and pessimism, country of origin, and brand competence are pertinent to gaining consumer trust in food products. however, there are still other elements to consider. food-product-related information provided by traceability systems is strongly supportive of quality assurance (kirezieva et al., 2013). accurate information is crucial for consumers before food choice through a critical assessment of a producer’s competence (yee, yeung, & morris, 2005; yeung & yee, 2003). the provision of transparent information related to food products encourages consumers to make their own judgements about the safety of a food product (grunert, 2005). smigic, rajkovic, djekic, and tomic (2015) indicated that there was a need for transparency between various legal authorities and food business operators. yoo et al. (2015) found that information asymmetry increases consumers’ perceived risk. when consumers perceive risk in food safety, it motivates consumers to seek additional information via the use of traceability systems (yoo et al., 2015). consumers’ motives for using traceability systems are reducing perceived risk due to information asymmetry. producers must retain sufficient records from traceability systems, including processing operations, systematic examination, harvesting areas, storage, transportation, and food receipts to show that food products are safe to eat (australia new zealand food authority, 2016). to decrease perceptions of risk, the more information provided by food traceability systems to consumers increases consumers’ trust in food products (eiser et al., 2002). trust refers to people's dependency on others in whom they place confidence (moorman, deshpandé, & zaltman, 1993). confidence is a strong belief in others (larzelere & huston, 1980; rotter, 1971). trust intentions in this paper refer to a consumer’s intentions to place his/her strong belief in food products manufactured by trustworthy food producers (morgan & hunt, 1994). when consumers have very firm beliefs about products made by certain food producers, it will affect their behaviour (bredahl, 2001). this leads to the following proposition. australasian journal of information systems lam, heales, hartley & hodkinson 2020, vol 24, selected papers from acis 2018 consumer trust in food safety 16 p2: a high degree of perceived knowledge of food products provided by traceability systems will be associated with trust intentions. 3.3 osn food product-related information and trust intentions the third association based on sct (bandura, 1986) is the relationship between social information facilitated by social media and consumer trust intentions. trust is aroused by cognition, and evoked by emotion through interpersonal dynamics with the support of information technology (komiak & benbasat, 2006). hobbs (2004) showed that consumers preferred to know the safety of food before eating it, otherwise, information asymmetry (e.g. quality attributes known only to producers) can result in market failure. information asymmetry can frustrate consumers and lead to a loss of confidence in food products among consumers. the development of the internet and social network sites, aka online communication, have assisted in reducing information asymmetry and its negative effects by allowing buyers to communicate information on product quality and share their experiences (izquierdo & izquierdo, 2007). the use of microblogs is associated with generating public awareness of food safety scandals, and the use of microblogs is deemed as a stronger predictor of food safety risk perception compared to other offline media such as television, newspapers and magazines (mou & lin, 2014). consumers provide feedback on a product, and in turn, a receiver builds up a personal judicious perception of product or service features (allsop, bassett, & hoskins, 2007). consumers are subjected to a variety of positive and negative information about food to influence their trust in a food product (scholderer & frewer, 2003). this leads to: p3: osn food product-related information will be associated with consumer trust intentions. 4 discussion and conclusions transparency and accountability in the food production chain are desired by consumers so as to know the sources and processes from farm to fork. incomplete information disclosure about the attributes of food product production is deemed as misspecification, and has the potential to ruin trust between consumers and producers (mishra, heide, & cort, 1998). a dearth of reliable information on the market such as dishonest and cheating behaviours could result in a failure in gaining consumer trust (granovetter, 1985), whereas providing trustful information about food products could greatly augment consumer trust (adler, 2001). traceability systems enable consumers to retrieve information regarding food safety along the processing and supply chain. traceability assists in tracking any food product through all stages from sourcing, through production, processing, and subsequent distribution. traceability also supports mechanisms for tracing backwards and forwards at any point in the supply and processing chain (australia new zealand food authority, 2016). this pragmatic and holistic approach to food chain information is important to consumers due to potential information asymmetry between producers and consumers. truthful and readily available information disclosure is a value-added process for both producers and consumers. the information disclosed in food traceability systems is likely to influence consumer’s trust in choosing food products at the point of sale (chen & huang, 2013). thus, fsmss attempt to close this gap. additionally, osn food product-related information bolsters (or undermines) consumer trust in a firm’s food products. online feedback mechanisms are corroborated to develop trust in australasian journal of information systems lam, heales, hartley & hodkinson 2020, vol 24, selected papers from acis 2018 consumer trust in food safety 17 sellers' credibility (ba & pavlou, 2002). positive feedback results in both a trustworthy signal and a good feedback profile (greif, 1989; milgrom, north, & weingast, 1990). on the other hand, negative feedback leads to not only brand detriment but also financial loss (z. lee, im, & lee, 2000; webster & sundaram, 1998). for instance, ebay's feedback forum is a channel where buyers provide feedback or reviews about their transactions with sellers. credible osn product-related information is able to influence a consumer’s judgement and increase their trust in a food product. this research seeks to enhance sct by examining how technology-supported information influences consumer trust in the context of food safety. technologies here include traceability systems and osn websites. the associations between perceived knowledge of food products provided by traceability systems, osn food product-related information and consumer trust in food safety are articulated. definitively, the relationships between constructs are proposed to be testable. research design for further research and the operations used to test the model are proposed as follows. mixed methods, both qualitative and quantitative research methods, should be used to test the model and proposed hypotheses (venkatesh, brown, & bala, 2013). the first phase involves inductive semi-structured in-depth interviews to confirm the conceptual framework. the second phase involves empirical testing of the model using a questionnairebased survey approach. as proposed by gaskin, berente, lyytinen, and yoo (2014), the interview method is appropriately used to reveal the actors’ behaviour patterns in rich detail. therefore, semi-structured in-depth interviews for collecting consumers’ perspectives and food producers’ perspectives of phenomena should be used. consumers’ and food producers’ activities, and the use of artefacts including the use of traceability systems, and socialnetworking sites should also be collected by interviews. in addition, notes should be taken during each interview in order to retrieve interviewees’ responses to questions, and assist in raising further queries, following middendorf and macan's (2002) note-taking strategy. semistructured in-depth interviews with food consumers will be carried out in australia. australia is a western-style multicultural society that will help ensure broad-based external validity. a food company with a wide consumer base and using traceability systems will be chosen. the selected firm’s food products must harmonise its food safety assurance with australian government regulations. the company’s managing staff will be contacted to interview their perceptions of information flows in the food supply and processing chain. next, a questionnaire survey will be conducted to corroborate the proposed research framework. measurement items will be developed from interview data and adapted from previous studies. experts’ opinions about the content validity of the measurement items will be conducted. after that, a pilot study will be undertaken. any items with weak factor loadings will be reviewed. general food consumers, who are over 18 years old, reside in australia, and make purchase decisions, will be randomly approached by using qualtrics survey services to complete the questionnaire. it is estimated a sample size of 300 australian respondents will be sufficient for hypothesis testing (maccallum, browne, & sugawara, 1996). the proposed model provides food producers with a framework to ponder which factors have positive/negative associations with consumer trust intentions. in addition, food producers can use the constructs and associations between constructs proposed in the model to ameliorate their own fsmss and osn communication strategy in order to increase consumer trust australasian journal of information systems lam, heales, hartley & hodkinson 2020, vol 24, selected papers from acis 2018 consumer trust in food safety 18 intentions. finally, it is possible to make an application providing such credible information that can influence consumer trust in a product, and win consumer loyalty. the integrated model facilitates cognitive elaboration of consumer trust in food products. the most far-reaching implication is the identification of consumers’ information needs so that some means of communication, such as mobile phone applications, company websites, and social network sites, can be developed to provide appropriate and useful food product information to consumers. contingent on the food producer’s situation, they can work out their own internal and external information systems and decide how their food product information will be disseminated. this will offer valuable assistance in gaining consumer trust. information on food products is also useful for developing policy on food safety management to prevent foodborne diseases. this might also help reduce societal healthcare costs due to the reduction of food-induced illnesses, and diminish costs of doing business associated with product recalls. there are several limitations in this article that need consideration. firstly, other aspects of consumer trust such as consumer practices (l. zhang et al., 2016) have not been included in this article. further research may consider the association between consumers’ trust intentions and their actual purchasing behaviours. secondly, this research only considers the effects of osn food product-related information on consumer trust. future research could examine the influence of the general mass media on consumer trust in food products. lastly, this is a conceptual paper that is not supported by empirical evidence. hence, further research will use the testable model developed above to further evaluate and test the propositions using quantitative methods e.g. survey methodology. references adler, p. s. (2001). market, hierarchy, and trust: the knowledge economy and the future of capitalism. organization science, 12(2), 215–234. allsop, d. t., bassett, b. r., & hoskins, j. a. (2007). word-of-mouth research: principles and applications. journal of advertising research, 47(4), 398–411. https://doi.org/10.2501/s0021849907070419 australia new zealand food authority. (2016). the australia new zealand food standards code (australia only). in safe food australia: a guide to the food safety standards (3rd edition, pp. 1–228). creative commons attribution 3.0 australia. retrieved from https://www.foodstandards.gov.au/publications/documents/complete_safefood.pdf australia new zealand food authority. (2017). food traceability. safe food australia. retrieved from http://www.foodstandards.gov.au/industry/safetystandards/traceability/pages/default. aspx ba, s., & pavlou, p. a. (2002). evidence of the effect of trust building technology in electronic markets: price premiums and buyer behavior. mis quarterly, 26(3), 243–268. retrieved from http://www.jstor.org/stable/4132332 bánáti, d. (2014). european perspectives of food safety. journal of the science of food and agriculture, 94(10), 1941–1946. https://doi.org/10.1002/jsfa.6611 australasian journal of information systems lam, heales, hartley & hodkinson 2020, vol 24, selected papers from acis 2018 consumer trust in food safety 19 bandura, a. (1986). social foundations of thought and action: a social cognitive theory. social foundations of thought and action: a social cognitive theory. englewood cliffs, n.j.: prentice-hall. baran, a. (2016). nanotechnology: legal and ethical issues. engineering management in production and services, 8(1), 47–54. https://doi.org/10.1515/emj-2016-0005 barbarossa, c., de pelsmacker, p., moons, i., & marcati, a. (2016). the influence of country-oforigin stereotypes on consumer responses to food safety scandals: the case of the horsemeat adulteration. food quality and preference, 53, 71–83. https://doi.org/10.1016/j.foodqual.2016.05.015 bavas, j. (2018). strawberry needle tampering crisis : queensland growers call for’ calm and common sense’ million-dollar assistance fund. abc news. retrieved from https://www.abc.net.au/news/2018-09-18/strawberry-fund-set-up-for-farmers-qldpremier-says/10262804 bennett, r., butterworth, a., jones, p., kehlbacher, a., & tranter, r. b. (2012). valuation of animal welfare benefits. a report to defra. university of reading. berg, l. (2004). trust in food in the age of mad cow disease: a comparative study of consumers’ evaluation of food safety in belgium, britain and norway. appetite, 42(1), 21–32. https://doi.org/10.1016/s0195-6663(03)00112-0 best, k. (2009). invalid command: affordances, icts and user control. information communication and society, 12(7), 1015–1040. https://doi.org/10.1080/13691180802471463 brazell, l. (2012). nanotechnology law: best practices. alphen aan den rijn, the netherlands: wolters kluwer law & business. bredahl, l. (2001). determinants of consumer attitudes and purchase intentions with regard to genetically modified foods – results of a cross-national survey. journal of consumer policy, 24(1), 23–61. breidert, c., hahsler, m., & reutterer, t. (2006). a review of methods for measuring willingness to pay. innovative marketing, 2(4), 8–32. cagliano, a. c., de marco, a., & rafele, c. (2017). e-grocery supply chain management enabled by mobile tools. business process management journal, 23(1), 47–70. https://doi.org/10.1108/bpmj-01-2016-0002 chan, c., kam, b. h., coulthard, d., & button, p. (2012). consumer trust of food product information & its sources. retrieved from https://www.gs1au.org/workarea/downloadasset.aspx?id=2147484565 chen, m. f. (2008). consumer trust in food safety—a multidisciplinary approach and empirical evidence from taiwan. risk analysis, 28(6), 1553–1569. https://doi.org/10.1111/j.1539-6924.2008.01115.x chen, m. f. (2011). consumer’s trust-in-food-safety typology in taiwan: food-related lifestyle matters. health, risk & society, 13(6), 503–526. chen, m. f., & huang, c. h. (2013). the impacts of the food traceability system and consumer involvement on consumers’ purchase intentions toward fast foods. food control, 33(2), 313–319. https://doi.org/10.1016/j.foodcont.2013.03.022 australasian journal of information systems lam, heales, hartley & hodkinson 2020, vol 24, selected papers from acis 2018 consumer trust in food safety 20 chiu, h. k. (2016). exploring the factors affecting consumer boycott behavior in taiwan: food oil incidents and the resulting crisis of brand trust. international journal of business and information, 11(1), 49–66. choe, y. c., park, j., chung, m., & moon, j. (2009). effect of the food traceability system for building trust: price premium and buying behavior. information systems frontiers, 11(2), 167–179. https://doi.org/10.1007/s10796-008-9134-z clark, b., stewart, g. b., panzone, l. a., kyriazakis, i., & frewer, l. j. (2017). citizens, consumers and farm animal welfare: a meta-analysis of willingness-to-pay studies. food policy, 68, 112–127. https://doi.org/10.1016/j.foodpol.2017.01.006 cornelisse-vermaat, j. r., voordouw, j., yiakoumaki, v., theodoridis, g., & frewer, l. j. (2008). food-allergic consumers’ labelling preferences: a cross-cultural comparison. european journal of public health, 18(2), 115–120. https://doi.org/10.1093/eurpub/ckm032 de boeck, e., jacxsens, l., bollaerts, m., uyttendaele, m., & vlerick, p. (2016). interplay between food safety climate, food safety management system and microbiological hygiene in farm butcheries and affiliated butcher shops. food control, 65, 78–91. https://doi.org/10.1016/j.foodcont.2016.01.014 de jonge, j., van trijp, h., renes, r. j., & frewer, l. j. (2010). consumer confidence in the safety of food and newspaper coverage of food safety issues: a longitudinal perspective. risk analysis, 30(1), 125–142. https://doi.org/10.1111/j.1539-6924.2009.01320.x de jonge, j., van trijp, j. c. m., van der lans, i. a., renes, r. j., & frewer, l. j. (2008). how trust in institutions and organizations builds general consumer confidence in the safety of food: a decomposition of effects. appetite, 51(2), 311–317. https://doi.org/10.1016/j.appet.2008.03.008 de steur, h., liqun, g., van der straeten, d., lambert, w., & gellynck, x. (2015). the potential market for gm rice with health benefits in a chinese high-risk region. journal of food products marketing, 21(3), 231–243. https://doi.org/10.1080/10454446.2013.855993 de vocht, m., cauberghe, v., uyttendaele, m., & sas, b. (2015). affective and cognitive reactions towards emerging food safety risks in europe. journal of risk research, 18(1), 21–39. https://doi.org/10.1080/13669877.2013.879486 eiser, j. r., miles, s., & frewer, l. j. (2002). trust, perceived risk, and attitudes toward food technologies. journal of applied social psychology, 32(11), 2423–2433. https://doi.org/10.1111/j.1559-1816.2002.tb01871.x elsevier. (2018). the scopus fact sheet. retrieved from https://www.elsevier.com/__data/assets/pdf_file/0008/208772/acad_r_sc_fs.pdf engelseth, p. (2009). food product traceability and supply network integration. journal of business and industrial marketing, 24(5), 421–430. https://doi.org/10.1108/08858620910966291 farm animal welfare committee. (2014). evidence and the welfare of farmed animals. part 1: the evidence base. retrieved from https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachm ent_data/file/324480/fawc_report_on_evidence_and_the_welfare_of_farmed_animals _part_1_the_evidence_base.pdf australasian journal of information systems lam, heales, hartley & hodkinson 2020, vol 24, selected papers from acis 2018 consumer trust in food safety 21 frewer, l. j., howard, c., hedderley, d., & shepherd, r. (1996). what determines trust in information about food-related risks? underlying psychological constructs. risk analysis, 16(4), 473–486. retrieved from http://www.ncbi.nlm.nih.gov/pubmed/8819340 gaskin, j., berente, n., lyytinen, k., & yoo, y. (2014). toward generalizable sociomaterial inquiry: a computational approach for zooming in and out of sociomaterial routines. mis quarterly, 38(3), 849–871. https://doi.org/10.25300/misq/2014/38.3.10 govindan, k. (2018). sustainable consumption and production in the food supply chain: a conceptual framework. international journal of production economics, 195(march 2017), 419–431. https://doi.org/10.1016/j.ijpe.2017.03.003 grandison, t., & sloman, m. (2003). trust management tools for internet applications. in p. nixon & s. terzis (eds.), proceedings of the 1st international conference on trust management (pp. 91–107). heraklion, crete, greece: springer-verlag berlin heidelberg new york. granovetter, m. (1985). economic action and social structure: the problem of embeddedness. american journal of sociology, 91(3), 481–510. retrieved from http://www.jstor.org/stable/2393756 greif, a. (1989). reputation and coalitions in medieval trade: evidence on the maghribi traders. journal of economic history, 49(4), 857–882. grunert, k. g. (2005). food quality and safety: consumer perception and demand. european review of agricultural economics, 32(3), 369–391. hall, d. (2010). food with a visible face: traceability and the public promotion of private governance in the japanese food system. geoforum, 41(5), 826–835. https://doi.org/10.1016/j.geoforum.2010.05.005 hanley, n., shogren, j. f., & white, b. (2001). introduction to environmental economics. introduction to environmental economics. oxford ; new york: oxford university press. harrison-walker, l. j. (2001). the measurement of word-of-mouth communication and an investigation of service quality and customer commitment as potential antecedents. journal of service research, 4(1), 60–75. https://doi.org/10.1177/109467050141006 harvey, d., & hubbard, c. (2013). reconsidering the political economy of farm animal welfare: an anatomy of market failure. food policy, 38(1), 105–114. https://doi.org/10.1016/j.foodpol.2012.11.006 hobbs, j. e. (2004). information asymmetry and the role of traceability systems. agribusiness, 20(4), 397–415. https://doi.org/10.1002/agr.20020 hu, w., grbich, c., & kemp, a. (2007). parental food allergy information needs: a qualitative study. archives of disease in childhood, 92(9), 771–775. https://doi.org/10.1136/adc.2006.114975 izquierdo, s. s., & izquierdo, l. r. (2007). the impact of quality uncertainty without asymmetric information on market efficiency. journal of business research, 60(8), 858–867. https://doi.org/10.1016/j.jbusres.2007.02.010 australasian journal of information systems lam, heales, hartley & hodkinson 2020, vol 24, selected papers from acis 2018 consumer trust in food safety 22 jansen-vullers, m. h., van dorp, c. a., & beulens, a. j. m. (2003). managing traceability information in manufacture. international journal of information management, 23(5), 395– 413. https://doi.org/10.1016/s0268-4012(03)00066-5 jensen, h. h. (2006). consumer issues and demand. choices. retrieved from http://www.choicesmagazine.org/2006-3/animal/2006-3-09.htm kekäle, t., de weerd-nederhof, p., cervai, s., & borelli, m. (2009). the “dos and don’ts” of writing a journal article. journal of workplace learning, 21(1), 71–80. https://doi.org/10.1108/13665620910924925 king, t., cole, m., farber, j. m., eisenbrand, g., zabaras, d., fox, e. m., & hill, j. p. (2017). food safety for food security: relationship between global megatrends and developments in food safety. trends in food science and technology, 68, 160–175. https://doi.org/10.1016/j.tifs.2017.08.014 kirezieva, k., jacxsens, l., uyttendaele, m., van boekel, m. a. j. s., & luning, p. a. (2013). assessment of food safety management systems in the global fresh produce chain. food research international, 52(1), 230–242. https://doi.org/10.1016/j.foodres.2013.03.023 ko, w. h. (2015). food suppliers’ perceptions and practical implementation of food safety regulations in taiwan. journal of food and drug analysis, 23(4), 778–787. https://doi.org/10.1016/j.jfda.2015.05.006 komiak, s. y. x., & benbasat, i. (2006). the effects of personalization and familiarity on trust and adoption of recommendation agents. mis quarterly, 30(4), 941–960. https://doi.org/10.1002/fut larzelere, r. e., & huston, t. l. (1980). the dyadic trust scale: toward understanding interpersonal trust in close relationships. journal of marriage and family, 42(3), 595–604. lassoued, r., & hobbs, j. e. (2015). consumer confidence in credence attributes: the role of brand trust. food policy, 52, 99–107. https://doi.org/10.1016/j.foodpol.2014.12.003 lee, h. l., & whang, s. (2005). higher supply chain security with lower cost: lessons from total quality management. international journal of production economics, 96(3), 289–300. https://doi.org/10.1016/j.ijpe.2003.06.003 lee, j., kim, j., & moon, j. y. (2000). what makes internet users visit cyber stores again? key design factors for customer loyalty. in proceedings of the sigchi conference on human factors in computing systems (pp. 305–312). https://doi.org/10.1145/332040.332448 lee, z., im, i., & lee, s. j. (2000). the effect of negative buyer feedback. in w. j. orlikowski, s. ang, p. weill, h. krcmar, & j. i. degros (eds.), twenty-first international conference on information systems (pp. 286–287). brisbane. lehmann, r. j., hermansen, j. e., fritz, m., brinkmann, d., trienekens, j., & schiefer, g. (2011). information services for european pork chains closing gaps in information infrastructures. computers and electronics in agriculture, 79(2), 125–136. https://doi.org/10.1016/j.compag.2011.09.002 lewis, m. (2003). cause, consequence and control: towards a theoretical and practical model of operational risk. journal of operations management, 21(2), 205–224. https://doi.org/10.1016/s0272-6963(02)00071-2 australasian journal of information systems lam, heales, hartley & hodkinson 2020, vol 24, selected papers from acis 2018 consumer trust in food safety 23 li, z., liu, g., liu, l., lai, x., & xu, g. (2017). iot-based tracking and tracing platform for prepackaged food supply chain. industrial management & data systems, 117(9), 1906– 1916. https://doi.org/10.1108/imds-11-2016-0489 liao, p. a., chang, h. h., & chang, c. y. (2011). why is the food traceability system unsuccessful in taiwan? empirical evidence from a national survey of fruit and vegetable farmers. food policy, 36(5), 686–693. https://doi.org/10.1016/j.foodpol.2011.06.010 liu, r., pieniak, z., & verbeke, w. (2014). food-related hazards in china: consumers’ perceptions of risk and trust in information sources. food control, 46, 291–298. https://doi.org/10.1016/j.foodcont.2014.05.033 maccallum, r. c., browne, m. w., & sugawara, h. m. (1996). power analysis and determination of sample size for covariance structure modeling. psychological methods, 1(2), 130–149. manders, j. h. m., caniëls, m. c. j., & ghijsen, p. w. t. (2016). exploring supply chain flexibility in a fmcg food supply chain. journal of purchasing and supply management, 22(3), 181–195. https://doi.org/10.1016/j.pursup.2016.06.001 manning, l. (2015). determining value in the food supply chain. british food journal, 117(11), 2649–2663. https://doi.org/10.1108/bfj-02-2015-0049 marks, l. (1984). what’s in a label? consumers, public policy and food labels. food policy, 9, 252–258. https://doi.org/10.1038/340168a0 marucheck, a., greis, n., mena, c., & cai, l. (2011). product safety and security in the global supply chain: issues, challenges and research opportunities. journal of operations management, 29(7–8), 707–720. https://doi.org/10.1016/j.jom.2011.06.007 matsuura, h. j. (2006). nanotechnology regulation and policy worldwide. norwood, usa: artech house. mattevi, m., & jones, j. a. (2016). food supply chain: are uk smes aware of concept, drivers, benefits and barriers, and frameworks of traceability? british food journal, 118(5), 1107– 1128. https://doi.org/http://dx.doi.org/10.1108/mrr-09-2015-0216 mcgloin, a., delaney, l., hudson, e., & wall, p. (2009). nutrition communication: the challenge of effective food risk communication. in proceedings of the nutrition society: symposium on ‘the challenge of translating nutrition research into public health nutrition’ session (vol. 68, pp. 135–141). https://doi.org/10.1017/s0029665109001153 mcguire, w. j. (1985). attitudes and attitude change. in g. lindzey & e. aronson (eds.), handbook of social psychology: special fields and applications (3rd ed., pp. 233–346). new york: random house. mcmeekin, t. a., baranyi, j., bowman, j., dalgaard, p., kirk, m., ross, t., zwietering, m. h. (2006). information systems in food safety management. international journal of food microbiology, 112(3), 181–194. https://doi.org/10.1016/j.ijfoodmicro.2006.04.048 merchant, g. (2012). unravelling the social network: theory and research. learning, media and technology, 37(1), 4–19. https://doi.org/10.1080/17439884.2011.567992 australasian journal of information systems lam, heales, hartley & hodkinson 2020, vol 24, selected papers from acis 2018 consumer trust in food safety 24 middendorf, c. h., & macan, t. h. (2002). note-taking in the employment interview: effects on recall and judgments. journal of applied psychology, 87(2), 293–303. https://doi.org/10.1037/0021-9010.87.2.293 milgrom, p., north, d., & weingast, b. (1990). the role of institutions in the revival of trade: the medieval law merchant. economics and politics, 2(i), 1–23. https://doi.org/doi:10.1111/j.1468-0343.1990.tb00020.x mishra, d. p., heide, j. b., & cort, s. g. (1998). information asymmetry and levels of agency relationships. journal of marketing research, 35(3), 277–295. https://doi.org/10.2307/3152028 moorman, c., deshpandé, r., & zaltman, g. (1993). factors affecting trust in market research relationships. journal of marketing, 57(1), 81–101. morgan, r. m., & hunt, s. d. (1994). the commitment-trust theory of relationship marketing. the journal of marketing, 58(3), 20–38. motarjemi, y., & mortimore, s. (2005). industry’s need and expectations to meet food safety, 5th international meeting: noordwijk food safety and haccp forum 9-10 december 2002. food control, 16(6 spec. iss.), 523–529. https://doi.org/10.1016/j.foodcont.2004.10.014 mou, y., & lin, c. a. (2014). communicating food safety via the social media: the role of knowledge and emotions on risk perception and prevention. science communication, 36(5), 593–616. https://doi.org/10.1177/1075547014549480 narasimhan, r., & talluri, s. (2009). perspectives on risk management in supply chains. journal of operations management, 27(2), 114–118. https://doi.org/10.1016/j.jom.2009.02.001 nocella, g., romano, d., & stefani, g. (2014). consumers’ attitudes, trust and willingness to pay for food information. international journal of consumer studies, 38(2), 153–165. https://doi.org/10.1111/ijcs.12080 premkumar, g., ramamurthy, k., & saunders, c. s. (2005). information processing view of organizations: an exploratory examination of fit in the context of interorganizational relationships. journal of management information systems, 22(1), 257–294. https://doi.org/10.1080/07421222.2003.11045841 pyke, d., & tang, c. s. (2010). how to mitigate product safety risks proactively? process, challenges and opportunities. international journal of logistics research and applications, 13(4), 243–256. https://doi.org/10.1080/13675561003720214 http://www.informaworld.com rieger, j., kuhlgatz, c., & anders, s. (2016). food scandals, media attention and habit persistence among desensitised meat consumers. food policy, 64, 82–92. https://doi.org/10.1016/j.foodpol.2016.09.005 rimpeekool, w., seubsman, s. ang, banwell, c., kirk, m., yiengprugsawan, v., & sleigh, a. (2015). food and nutrition labelling in thailand: a long march from subsistence producers to international traders. food policy, 56, 59–66. https://doi.org/10.1016/j.foodpol.2015.07.011 australasian journal of information systems lam, heales, hartley & hodkinson 2020, vol 24, selected papers from acis 2018 consumer trust in food safety 25 rotter, j. b. (1971). generalized expectancies for interpersonal trust. american psychologist, 26(5), 443–452. https://doi.org/10.2466/pr0.1976.39.2.649 ruiz-garcia, l., & lunadei, l. (2011). the role of rfid in agriculture: applications, limitations and challenges. computers and electronics in agriculture, 79(1), 42–50. https://doi.org/10.1016/j.compag.2011.08.010 rutsaert, p., pieniak, z., regan, á., mcconnon, á., kuttschreuter, m. ô., lores, m., verbeke, w. (2014). social media as a useful tool in food risk and benefit communication? a strategic orientation approach. food policy, 46, 84–93. https://doi.org/10.1016/j.foodpol.2014.02.003 rutsaert, p., regan, á., pieniak, z., mcconnon, á., moss, a., wall, p., & verbeke, w. (2013). the use of social media in food risk and benefit communication. trends in food science & technology, 30(1), 84–91. ryan, a. m., & spash, c. l. (2011). is wtp an attitudinal measure? empirical analysis of the psychological explanation for contingent values. journal of economic psychology, 32(5), 674–687. https://doi.org/10.1007/s10551-011-0925-7 sandler, r. (2009). nanotechnology: the social and ethical issues. retrieved from http://eprints.internano.org/50/1/nano_pen16_final.pdf scholderer, j., & frewer, l. j. (2003). the biotechnology communication paradox: experimental evidence and the need for a new strategy. journal of consumer policy, 26(2), 125–157. singh, j. (1988). consumer complaint intentions and behavior: definitional and taxonomical issues. journal of marketing, 52(1), 93–107. https://doi.org/10.2307/1251688 smigic, n., rajkovic, a., djekic, i., & tomic, n. (2015). legislation, standards and diagnostics as a backbone of food safety assurance in serbia. british food journal, 117(1), 94–108. https://doi.org/10.1108/bfj-08-2013-0228 starbird, s. a., & amanor-boadu, v. (2006). do inspection and traceability provide incentives for food safety? journal of agricultural and resource economics, 31(1), 14–26. stauss, b. (2000). using new media for customer interaction: a challenge for relationship marketing. in t. hennig-thurau & u. hansen (eds.), relationship marketing: gaining competitive advantage through customer satisfaction and customer retention (1st ed., pp. 232–253). berlin, heidelberg: springer berlin heidelberg. https://doi.org/10.1007/978-3662-09745-8 swoffer, k. (2009). gfsi and the relationship with codex. in presentation made at cies international food safety conference. barcelona. szymanska, k. (2015). information system as a tool guaranteeing food safety and controlling food market. in 10th international scientific conference on economic and social development (pp. 95–102). miami. tang, c. s. (2008). making products safe: process and challenges. international commerce review, 8(1), 48–55. https://doi.org/10.1007/s12146-008-0028-2 taylor, c. l., & wilkening, v. l. (2008a). how the nutrition food label was developed, part 1: the nutrition facts panel. journal of the american dietetic association, 108(3), 437–442. https://doi.org/10.1016/j.jada.2007.12.010 australasian journal of information systems lam, heales, hartley & hodkinson 2020, vol 24, selected papers from acis 2018 consumer trust in food safety 26 taylor, c. l., & wilkening, v. l. (2008b). how the nutrition food label was developed, part 2: the purpose and promise of nutrition claims. journal of the american dietetic association, 108(4), 618–623. https://doi.org/10.1016/j.jada.2008.01.009 the food standards australia new zealand. (2018). strawberry tampering incident. retrieved from http://www.foodstandards.gov.au/publications/siteassets/pages/strawberrytampering-incident/fsanz strawberry report doc.pdf tsai, h. j., chen, b. h., wu, c. f., wang, s. l., huang, p. c., tsai, y. c., chen, m. l., ho, c. k., hsiung, c. a., wu, m. t. (2016). intake of phthalate-tainted foods and microalbuminuria in children: the 2011 taiwan food scandal. environment international, 89–90(100), 129– 137. https://doi.org/10.1016/j.envint.2016.01.015 venkatesh, v., brown, s. a., & bala, h. (2013). bringing the qualitative-quantitative divide: guidelines for conducting mixed methods research in information systems. mis quarterly, 37(1), 21–54. retrieved from http://web.b.ebscohost.com/ehost/ verçuni, a., zhllima, e., imami, d., bijo, b., hamiti, x., & bicoku, y. (2016). analysis of consumer awareness and perceptions about food safety in tirana, albania. albanian journal of agricultural sciences, 15(1), 19–26. voordouw, j., antonides, g., cornelisse-vermaat, j. r., pfaff, s., niemietz, d., & frewer, l. j. (2012). optimising the delivery of food allergy information. an assessment of food allergic consumer preferences for different information delivery formats. food quality and preference, 23(1), 71–78. https://doi.org/10.1016/j.foodqual.2011.08.001 voordouw, j., cornelisse-vermaat, j. r., pfaff, s., antonides, g., niemietz, d., linardakis, m., frewer, l. j. (2011). preferred information strategies for food allergic consumers. a study in germany, greece, and the netherlands. food quality and preference, 22(4), 384–390. https://doi.org/10.1016/j.foodqual.2011.01.009 wang, d., & huang, d. (2010). food supply chain management under conditions of food safety. in 2010 international conference on management and service science (pp. 1–4). ieee. https://doi.org/10.1109/icmss.2010.5577267 webster, c., & sundaram, d. s. (1998). service consumption criticality in failure recovery. journal of business research, 41(2), 153–159. westbrook, r. a. (1987). product/consumption-based affective responses and postpurchase processes. journal of marketing research, 24(3), 258–270. https://doi.org/10.2307/3151636 whipple, j. m., voss, m. d., & closs, d. j. (2009). supply chain security practices in the food industry: do firms operating globally and domestically differ? international journal of physical distribution & logistics management, 39(7), 574–594. https://doi.org/10.1108/09600030910996260 wikipedia. (2019). 2018 australian strawberry contamination. retrieved from https://en.wikipedia.org/wiki/2018_australian_strawberry_contamination wolny, j., & mueller, c. (2013). analysis of fashion consumers’ motives to engage in electronic word of mouth communication through social media platforms. journal of marketing management, 29(5–6), 562–583. https://doi.org/10.1080/0267257x.2013.778324 australasian journal of information systems lam, heales, hartley & hodkinson 2020, vol 24, selected papers from acis 2018 consumer trust in food safety 27 wu, c. w. (2015). facebook users’ intentions in risk communication and food-safety issues. journal of business research, 68(11), 2242–2247. https://doi.org/10.1016/j.jbusres.2015.06.005 xie, y., & yao, x. (2016). a content analysis of the image repair discourses of osi group in china’s food-safety scandal. asian agricultural research, 8(2), 68–72. yee, w. m. s., yeung, r. m. w., & morris, j. (2005). food safety: building consumer trust in livestock farmers for potential purchase behaviour. british food journal, 107(11), 841–854. https://doi.org/10.1108/00070700510629788 yeung, r. m. w., & yee, w. m. s. (2003). risk reduction: an insight from the uk poultry industry. nutrition & food science, 33(5), 219–229. https://doi.org/10.1108/00346650310499749 yiannas, f. (2009). communicating food safety effectively. in food safety culture: creating a behavior-based food safety management system (pp. 49–55). new york: springer. https://doi.org/10.1007/978-0-387-72867-4 yoo, c. w., parameswaran, s., & kishore, r. (2015). knowing about your food from the farm to the table: using information systems that reduce information asymmetry and health risks in retail contexts. information and management, 52(6), 692–709. https://doi.org/10.1016/j.im.2015.06.003 zhang, k., chai, y., yang, s. x., & weng, d. (2011). pre-warning analysis and application in traceability systems for food production supply chains. expert systems with applications, 38(3), 2500–2507. https://doi.org/10.1016/j.eswa.2010.08.039 zhang, l., xu, y., oosterveer, p., & mol, a. p. j. (2016). consumer trust in different food provisioning schemes: evidence from beijing, china. journal of cleaner production, 134, 269–279. https://doi.org/10.1016/j.jclepro.2015.09.078 zwietering, m. h., jacxsens, l., membré, j. m., nauta, m., & peterz, m. (2016). relevance of microbial finished product testing in food safety management. food control, 60, 31–43. https://doi.org/10.1016/j.foodcont.2015.07.002 copyright: © 2020 lam, heales, hartley & hodkinson. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems lam, heales, hartley & hodkinson 2020, vol 24, selected papers from acis 2018 consumer trust in food safety 28 1 introduction 2 literature review 2.1 previous literature 2.2 traceability systems and trust 2.3 food product communication and online social networking 2.4 consumer trust 2.5 an extended search from international peer-reviewed journals 2.6 an extended search on scopus database 3 conceptual development 3.1 perceived knowledge of food product provided by traceability systems and osn food product-related information 3.2 perceived knowledge of food product provided by traceability systems and trust intentions 3.3 osn food product-related information and trust intentions 4 discussion and conclusions paper title (paper title style) australasian journal of information systems troshani et al. 2022, vol 26, selected papers from acis 2020 environmental management infrastructures 1 information systems, sociomaterial practices and the emergence of environmental management infrastructures indrit troshani the university of adelaide, sa, australia indrit.troshani@adelaide.edu.au bill doolin auckland university of technology, new zealand karl fradley sra information technology, adelaide, sa, australia giselle rampersad flinders university, sa, australia abstract information systems are an integral part of environmental management infrastructures – complex assemblages of social and technical artifacts, human actors and sociomaterial routines enacted in the pursuit of environmental sustainability outcomes. we analyse the case of an environmental management infrastructure developed around a ‘vessel management system’ by a south australian regulator between 2004 and 2013. the goal of this infrastructure was to sustain water quality by controlling the discharge of ‘greywater’ from vessels on south australia’s inland waters. we conceptualise environmental management infrastructure, and the information systems they encompass, as the temporally emergent outcome of human agents’ attempts to extend their environmental management practice in a particular direction, and how that trajectory of emergence is shaped by the intersection of human agencies, material performances and disciplinary practices. keywords: environmental management, information systems, infrastructure, artifacts, routines, sociomaterial practice, mangle of practice, emergence. 1 introduction in this paper, we are concerned with how information systems become implicated in addressing environmental sustainability challenges and transformations (elliot & webster, 2017; seidel et al., 2013). specifically, we are interested in the ways that information systems underpin and coordinate environmental management practice. to articulate our understanding of how change in environmental management practice occurs and the role that information systems play we utilise the concept of ‘infrastructure’ (edwards et al., 2009; star, 1999) and its development. infrastructures are complex, extended assemblages of heterogeneous entities (jensen & morita, 2015; venters et al., 2014) that involve “interdependence, intricacy, and interweaving of people, systems, and processes” (ciborra & hanseth, 2000, p. 2). rather than simply being the material substrate that underlies social action such as environmental management (jensen & morita, 2017), infrastructure is how the world is made and brought into being – it is the product of co-evolving relations of people, technologies, materials, and organised practices (harvey et al., 2017; jensen, 2015; jensen & morita, 2015; star, 1999). infrastructures are thus australasian journal of information systems troshani et al. 2022, vol 26, selected papers from acis 2020 environmental management infrastructures 2 open-ended and emergent (jensen & morita, 2017), “always an unfinished work in progress” (edwards et al. 2009, p. 365). from this understanding, information systems are not standalone entities but contribute to environmental sustainability as part of the social and material relations that form the infrastructure underpinning environmental management practice. the key research question we ask is: how is change in environmental management practice achieved through the development of an environmental management infrastructure? to address this question, we focus on an example of the environmental management of wastewater – liquid and solid wastes mixed with water as the medium in which they are transported. environmental stresses related to water scarcity and deteriorating water quality in various parts of the world have focused governmental and regulatory attention on wastewater as an environmental problem (united nations, 2018). in the domestic context, wastewater includes both waste from toilets (‘blackwater’) and wastewater from household baths, showers, hand basins, washing machines, sinks or other kitchen appliances (‘greywater’). our case study examines the efforts of a south australian regulator, the environment protection authority (epa) to change the way that greywater produced by vessels using the state’s rivers and lakes was handled. to analyse the environmental management infrastructure that emerged in response to the epa’s attempt to manage vessel greywater in south australia, we adopt a ‘sociomaterial’ perspective, which helps us understand the complex and dynamic entanglement of human agency and material performances in environmental management practice (orlikowski, 2005, 2007; venters et al., 2014). in particular, we draw on pickering’s (1995) conception of the ‘mangle of practice’ to explain how environmental management infrastructures emerge “through a process of tuning where human and material agencies continually resist and accommodate each other” (baygi et al., 2021). our findings offer insights into how information systems underpinning these infrastructures become entangled with other artifacts and work practices and, in the process, transform environmental governance. the remainder of the paper is structured as follows. in the next section, we discuss the theoretical perspective underpinning our study and conceptualise the emergence of environmental management infrastructure. in the subsequent section we briefly outline the background to the case before discussing how we collected and analysed our data. in the next part of the paper, we present our findings and analysis, before concluding with a discussion of our key findings and contributions. 2 theoretical perspective 2.1 the mangle of practice for pickering (1995, 2013), human practice is about doing things in a material world. he approaches this from a performative perspective, an “ontology of becoming” (pickering, 2008, p. 3), in which practice is open-ended and emerges from the interplay of both human and material agencies. if agency is defined in terms of performance rather than intention, then while humans do things in the world, clearly so too do a whole range of nonhuman entities, including material artifacts such as information technology. both have agency in the sense of performing actions. pickering (1995) calls this “performative, emergent dance of agency … between the human and the non-human” (pickering, 2013, p. 33) the ‘mangle of practice’. australasian journal of information systems troshani et al. 2022, vol 26, selected papers from acis 2020 environmental management infrastructures 3 nonetheless, human agency is bound with intentionality. pickering (1995, p. 18) argues that human agents “construct goals that refer to presently non-existent future states and then seek to bring them about”. these goals “are imaginatively transformed versions of [the] present” (pickering, 1995, p. 19). in accomplishing their goals, human agents “maneuver in a field of material agency” (p. 7), engaging and negotiating with an array of material performativities, often through the construction of technologies – “human activities and technological doings” (orlikowski, 2005, p. 185). pickering (1995, p. 22) presents this process as a “dialectic of resistance and accommodation”: a sequence of temporarily emergent resistances to attempts to capture material agencies in practice – a “block on the path to some goal” (pickering, 1995, p. 39), and contingently formulated accommodations – an “active human strategy of response to resistance” (pickering, 1995, p. 22). these accommodations can involve not just revisions to the material form of technologies but also to the contours of human agency and the social relations that encompass it. thus, both material and human agency (including plans and goals) are mangled and emergently transformed in practice (pickering, 1995). in our analysis of the emergence of an environmental management infrastructure for the control of greywater produced by vessels on south australian waterways we build on the mangle of practice to conceptualise the extension of environmental management practice to vessel greywater management. detailed reviews of environmental or ‘green’ information systems research can be found in singh & sahu (2020) and jenkin et al., (2011). 2.2 the extension of practice for pickering (1995), new practice is produced from the intentional extension of existing practice in a particular direction. available resources and agencies are harnessed to create imagined future states through present action (pickering, 1995, venters et al., 2014). this process is shaped by existing technologies and the cultural context in which it is situated, which act as the ’surface of emergence’ for the new practice (chae & poole, 2005; pickering, 1995). pickering (1995) suggests that certain elements of existing practice that have become established as accepted ways of thinking and exert a form of agency that disciplines human activities. this “disciplinary agency” (pickering, 1995, p. 29) shapes human action by providing a repertoire of concepts, conventions, values and procedures encapsulated in particular bodies of knowledge: “the agency of a discipline … leads people through a series of actions and also naturalizes these actions for them” (chae & poole, 2005, p. 21). arguably, disciplinary agency reduces discretionary human agency because it creates inertia when human agents confront existing forms of knowledge and institutionalised methods of action (johnston et al., 2016; venters et al., 2014). thus, the extension of practice at least partially involves the copying of established elements or moves from existing practice. the attempted extension to a practice involves “the production of associations, the making of connections, and the creation of alignments between disparate [social and material] elements” (pickering, 1995, p. 139). however, because the outcome of this attempt cannot be known in advance, the various elements assembled do not necessarily perform together as intended. in the consequent struggles between human goals and plans and material and disciplinary agencies, conceptual structures, the material form of technologies, and the scale and social relations of human agency are all mangled in practice. mangling occurs until “specific trajectories come to be marked out and [interactively] stabilized by the achievement of associations between these elements” (pickering, 1995, p. 91). in this way, the contours of practice are defined in real time by the emergence of resistances at the intersection of human, australasian journal of information systems troshani et al. 2022, vol 26, selected papers from acis 2020 environmental management infrastructures 4 material and disciplinary agency, and the success or failure of accommodations to those resistances. 2.3 artifacts and routines to operationalise pickering’s extension of practice in our analysis of the emergence of infrastructure, we utilise the concepts of artifacts and routines as the observable and traceable outlines of the exercise of material and human agencies in sociomaterial practices (gaskin et al., 2014; leonardi, 2011). artifacts are material things constructed or produced by a human agent to achieve a given purpose or practical function (baker, 2004; verbeek & vermaas, 2009). technical artifacts, whether physical or digital objects (faulkner & runde, 2019; leonardi, 2010), perform their function on the basis of their designed form or structure (kallinikos, 2012; kroes, 2010). in contrast, social artifacts, such as scientific reports, technical standards, laws and contracts, perform their function “on the basis of collective intentionality” (kroes, 2010, p. 57) – the collective acceptance of the constitutive rules or norms associated with their status and function (burazin, 2019) that produces standardised social behaviour (kallinikos, 2012). thus, the material agency of a contract, for example, derives not from its inscription in paper or digital documents, but from its use within a collectively recognised legal regime, which serves particular material ends in the governance of human transactions (m. c. suchman, 2003). both technical and social artifacts are constructed in the course of infrastructural development in order “to achieve an intended capture of agency” (pickering, 1995, p. 22) through their performativity (leonardi, 2010). these artifacts are incorporated into sociomaterial routines or regular sequences of activities on which human agents rely in their everyday work (gaskin et al., 2014; pickering, 1995). we refer to such work practices as routines to distinguish them from practice in the more generic sense used by pickering (1995). artifacts lie at the centre of routines – shaping their emergence, encouraging their persistence, guiding the course of action, and performing key functions (d’adderio, 2011; pentland & feldman, 2008). in enacting sociomaterial routines, human agents interact with a range of artifacts, the agencies of which rely on their incorporation and use in specific sites and “skilled practices that bring them into alignment” (l. suchman, 2007, p. 269). the artifacts and routines that comprise an infrastructure (ciborra & hanseth, 2000; leonardi, 2011) represent malleable human-artifact relations (l. suchman, 2007) with varying consequences. for example, artifacts incorporated into a routine may fail to perform as intended. equally, the enactment of routines may produce outcomes that are inconsistent with the goals involved in transforming practice. human agents respond by reconfiguring artifacts and adjusting routines, or by creating new ones, as an infrastructure emerges over time. artifacts and routines thus co-evolve through their involvement in performative struggles among competing agencies (d’adderio, 2011). 2.4 synthesis our conceptualisation of the emergence of environmental management infrastructures involves attempts by human agents to extend an existing practice in a particular direction by making interactively stabilised associations between heterogenous elements of a new (extended) practice. these elements are resources produced and transformed in the course of practice (pickering, 1995). our particular focus in this study are the artifacts and routines that form key building blocks of infrastructure. australasian journal of information systems troshani et al. 2022, vol 26, selected papers from acis 2020 environmental management infrastructures 5 the mangle comprises the intentional structures and social contours of human agency, the culturally enduring procedures of disciplinary agency, and the various forms and performances of material agency (huvila, 2016). alignments are not straightforward and require ongoing ‘tuning’ (pickering, 1995) as resistances emerge at the intersection of these mangled agencies, which in turn trigger contingently formulated accommodations and adjustments to the evolving assemblage of artifacts and routines. depending on the effectiveness of these responses in circumventing encountered obstacles, the sequence of emergent resistances and attempted accommodations continues in time until a degree of closure is reached through stabilisation (albeit precarious) of the dynamic mangling process (johnston et al., 2016; pickering, 1995). 3 case background the epa is an independent environmental regulator that administers south australia’s environment protection act 1993. in particular, the epa protects and restores land, air and water quality through the risk-based regulation of pollution and waste. at the time of this study, one of the pressing challenges facing the epa was addressing the environmental and human health risks and impacts associated with management of the lower reaches of the murray river (environment protection authority, 2012b). the river is the major source of domestic water supply and also plays a key part in the state’s tourism and recreation economy. however, water quality in the river had significantly declined due to high levels of water extraction, extended drought conditions, and pollution from a range of sources (department of water, land and biodiversity conservation, 2009; environment protection authority, 2003b). one of those sources of pollution is the discharge of untreated greywater from vessels using the river and its associated waterways. the composition and characteristics of greywater depend on the sources and activities from which the water is generated and can vary significantly with respect to both place and time. potential components include soluble organic compounds derived from chemical products and kitchen waste, suspended solids (including food particles, hair and fibres), dissolved salts such as nitrates and phosphates, other pollutants such as heavy metals, and pathogenic micro-organisms (including bacteria and viruses) (eriksson et al., 2002). greywater discharge from vessels can be a significant source of pollution in areas with high levels of recreational boating. in south australia, the murray river is home to a thriving houseboat community, many with facilities such as washing machines, dishwashers, ensuite bathrooms and spa baths. with over 2,000 vessels, including some 300 commercial hire boats, it is estimated that up to 500 million litres of untreated greywater could be discharged into the river annually (environment protection authority, 2009b). this practice is problematic with potential effects on aquatic ecosystem health, raw water supply and recreational use of waterways (environment protection authority, 2007a). in response to the increasing environmental and public health risks associated with the discharge of vessel greywater, the epa sought to extend its environmental management practice towards a new practice of regulated greywater management. this involved the development of a greywater management infrastructure between 2004 and 2013 to facilitate the monitoring and management of greywater produced by vessels on south australia’s inland waters. in addition to the planned interventions of the epa and other human actors, this infrastructure is composed of different forms of materiality that were integral to enacting environmental management in practice (jensen, 2015), including policies and guidelines, regulatory instruments and standards, technical devices, digital artifacts and databases. the australasian journal of information systems troshani et al. 2022, vol 26, selected papers from acis 2020 environmental management infrastructures 6 materiality in the greywater management infrastructure could also be extended to encompass natural objects, such as the river itself (jensen, 2015). indeed, greywater and its material properties are a key reason why vessel greywater management practice needed to change on south australia’s waterways. however, in this paper we focus primarily on the artifacts and routines that were created to accomplish vessel greywater management. 4 method we analyse the development of the greywater management infrastructure from a longitudinal perspective. consistent with our theoretical perspective, we follow sandberg & tsoukas’s (2011) suggestions to focus on the sociomaterial performance of practice and what matters to the practitioners. this involved studying what human agents did and why, with different artifacts and in relation to other actors, and with what results, in their attempts to effect change in the form of new or extended environmental management practice. we collected qualitative empirical data from a ten-year period between 2004 and 2013 using two main sources: participant observation and documentary evidence. our initial focus was on the vessel management system (vms) developed by the epa. data were collected by one of the authors as a ‘participant observer’ (walsham, 1995) attached to the epa between 2010 and 2013. this author was involved in various aspects of the development of the vms and its ongoing modification as a systems developer and database coordinator. he was therefore able to observe many details of the development of the information system and its operation in practice. his involvement provided access to organisational documentation, screen shots of the evolving system, and various project artifacts. he also participated in interviews held between the developers of the vms and its stakeholders and users. these interviews covered a range of issues and provided insights into the organisational interests related to the vms (and the broader environmental management infrastructure), organisational engagement and interaction processes, and drivers and inhibitors of system adoption. participant observations from the interviews and other development activities were summarised as field note memos. data sources number of documents internal epa records and correspondence 90 epa publications 88 other government department publications 29 boating industry publications 32 broadcast, online and print media 63 total 302 table 1. documentary data to complement the data generated from the participant observation, a wide range of internal and public documents were reviewed, including print and online publications of the epa (including media releases); internal epa records such as unpublished reports, meeting minutes, consultation submissions, memos, formal correspondence and emails; publicly available newsletters, reports and white papers from other stakeholder organisations in both government and the boating industry; and media coverage, including newspaper and magazine articles, and television and radio interviews. table 1 summarises the documentary sources examined in the study. in our findings and analysis, we indicate the source of evidence presented from field note memos or internal epa documentation in square brackets. australasian journal of information systems troshani et al. 2022, vol 26, selected papers from acis 2020 environmental management infrastructures 7 examples from our data significance in our analysis theoretical concepts used excerpt from an epa media release: “we're currently developing an agreement with the department of transport’s commercial marine division that will see the new greywater requirements incorporated into their regular survey and inspection process.” collaborative relationship to support epa’s vessel audit routine. human agency (actors and social relations) routine (enactment) excerpt from the minutes of a vms project meeting: “system proposed to be in either php or j2ee. check with aim about standards for typical applications.” introduction of it specialists (aim) with their disciplinary expertise to vms development disciplinary agency (knowledge and conceptual schemes) technical artifact (form) excerpt from an industry magazine: “epa officers joined representatives from the boating industry association of sa (biasa) … on a 10 day survey of the river murray … the epa also quantified the location and numbers of inland vessels, and recorded the registration details of all vessels that were not previously on the epa’s vessel management database.” engagement with boating industry stakeholder. example of vessel audit in action and use of vms in that practice. human agency (actors and social relations) routine (enactment) technical artifact (function) excerpt from a vms project email and screenshot: “can we under this page make a pull down box that we can choose to retrieve inland vessels only. apparently as soon as you put a compliance date in for privates whether marine or inland they all come up within the search function. so we sent a few inland letters to marine vessel owners.” modification to the vms as a result of a problem encountered in practice. depiction of vms web-based interface over oracle rdbms. technical artifact (form and function) material agency (producing resistance) accommodation (revision to artifact) excerpt from a regional news item: “he says the epa had given people a longer grace period because of the low river levels. ‘we've got to the point where we no longer think that's the right approach and we're going to start being more forceful in our enforcement of the regulations,’ he said.” delay to enforcement of greywater requirements due to low river levels. intention to change approach to enforcing regulatory compliance. material agency (river conditions producing resistance) human agency (goals and planned interventions) accommodation (revision to goals) excerpt from an epa internal report: “summary of comment: small vessels should be exempted from having to make structural alterations to contain wastewaters … response: the code provides for a number of management strategies to prevent the discharge of vessel wastewaters and has introduced compliance dates to reflect the size and use made of vessel … outcomes: modification” example of submission from public consultation and epa’s response. refers to provisions of the greywater cop. modification of the cop to a phased implementation of greywater compliance requirements. human agency (actors and social relations) material agency (vessel size producing resistance) social artifact (function) accommodation (revision to artifact) table 2. interpretive template as we explored the vms and its development it became clear that we needed to extend our case study back in time in order to understand the conditions from which the vms emerged. consequently, we expanded our data collection to include documentary evidence from 2004. this introduced new artifacts, such as a code of practice (cop) covering vessel greywater management, and new social actors, such as the south australian department of transport. the start and end points of our analysis are somewhat arbitrary as the ‘biography’ of an information system extends in either direction, with nominal beginnings reflecting the accumulation of earlier decisions and actions, and future adjustments and adaptations equally likely (leonardi, 2011; williams & pollock, 2012). australasian journal of information systems troshani et al. 2022, vol 26, selected papers from acis 2020 environmental management infrastructures 8 the first stage of our data analysis involved assembling the various documents and field note memos comprising our data in chronological order. this enabled us to construct a timeline of events and activities and associate together accounts of these from different actors. based on an interpretive research approach, our initial reading of the empirical material focused on content, context and process of change (walsham, 1995). our coding of the data identified the main actors involved, their stated interests and concerns, the arguments that they articulated in relation to the issue of vessel greywater management, the various actions they undertook, and the tools and artifacts they utilised in those activities. based on this initial coding, we developed a case narrative that outlined the basic process involved in the emergence of the vms and the greywater management infrastructure in which it is embedded. this focus on the temporal unfolding of the case study is appropriate given our subsequent emphasis on mangling and emergence (venters et al., 2014). in the second stage of analysis, we used the main elements of our theoretical perspective as a lens with which to interrogate our empirical data (venters et al., 2014). initially, we read our case narrative to identify instances of resistance and accommodation. for each of these, we examined the relevant data for indicators of when and where different agencies were exercised. we then explored the contours of each agency, focusing on who or what was acting and why, the performative effects that resulted, and the relationship with other agencies. in doing so, we were guided by pickering’s (1995) admonition to concentrate on exactly what gets mangled. while our analysis in this stage was shaped by the main elements of the mangle, we treated these “as sensitising theoretical concepts … rather than mechanistically applied categories” (symon & pritchard, 2014, p. 248). examples of how we interpreted our data in this stage of the analysis are presented in table 2. in the final stage of analysis, we used our conceptualisation of the extension of material practice as discussed in section 2 to guide our analysis in terms of temporal emergence through sequences of resistance and accommodation. we analytically defined two main episodes of mangling that we considered important in developing a process explanation of infrastructural emergence. these structure the presentation of our findings and analysis. 5 findings and analysis our analysis of the empirical evidence demonstrates how the artifacts and routines comprising the emergent greywater management infrastructure were the products of the mangling of human, disciplinary and material agencies in the practices of its developers and users. figure 1 below summarises the sequence of events and activities and the two key (sometimes overlapping) mangling episodes that we delineated in the development of the epa’s new greywater management practice. the following sections can be read alongside figure 1. 5.1 episode 1: mangling to regulate by early 2003, the need to address the degraded state of the south australia’s waterways had gained considerable official and public acceptance (environment protection authority, 2003b). attention increasingly focused on the handling of greywater produced on vessels using the waterways, which was being discharged directly into the aquatic environment. previously, few vessel owners and boating industry operators had been concerned about its environmental impact (gallagher & wigley, 2001; salter, 2001). however, the publication of a local government report on the ecological impact of houseboats on the murray river australasian journal of information systems troshani et al. 2022, vol 26, selected papers from acis 2020 environmental management infrastructures 9 highlighted the significance of greywater pollution and lobbied the epa to investigate its impact (gallagher & wigley, 2001). figure 1. greywater management timeline in response to the heightened importance of water quality issues, the epa considered the development of a strategy to manage greywater discharge from vessels using the river (environment protection authority, 2002). perhaps unsurprisingly, the epa decided on the goal of regulating greywater discharge. its institutional position and status as an environmental regulator conditioned the intentional structure of its agency so that pursuing a regulatory approach to the issue of greywater management was the obvious choice. thus, the goal of the epa became to extend its existing environmental management practice to encompass vessel greywater management. extending its practice along this particular direction involved establishing a position that defined environmental regulation as a desired and legitimate future state for vessel greywater management. the vehicle for achieving this desired end state lay in a recently approved environment protection (water quality) policy 2003. an environment protection policy is a legislative tool under the state of south australia’s environment protection act 1993. the release of the water quality environment protection policy provided for a consistent approach to addressing degradation in water quality across south australian water bodies (environment protection authority, 2001). it set water quality objectives, established general obligations to manage and control pollution, and encouraged better use of wastewater. the environment protection policy also specified additional obligations relating to wastewater discharge, including the operation of vessels on the state’s rivers and lakes (environment protection authority, 2003a). in particular, the environment protection policy incorporated an earlier requirement that blackwater produced on vessels be contained for subsequent on-land disposal, but went further in also providing a requirement, at an unspecified future date, for the containment or treatment of greywater. central to the epa’s strategy for managing vessel greywater discharge was the realisation of the obligations and provisions outlined in the newly approved water quality environment protection policy. while the environment protection policy australasian journal of information systems troshani et al. 2022, vol 26, selected papers from acis 2020 environmental management infrastructures 10 acknowledged the need for vessel greywater management, the details of how this would be achieved would need to be developed by the epa. without establishing measurable outcomes, appropriate practices that users of vessels could follow, and viable monitoring and enforcement mechanisms, the epa would be constrained in its ability to ensure user compliance with the provisions of the environment protection policy and prevent pollution from this source. emergence of the code of practice artifact in working towards its goal of regulating vessel greywater, the epa copied elements of its existing environmental management practice into the new practice. that is, once the epa decided on the path of regulation, the regulatory framework under which it operated provided a set of generalisable procedures (chae & poole, 2005) that the epa could apply in furthering its goal – an exercise of disciplinary agency. in particular, this involved the creation of a regulatory artifact called a ‘code of practice’. a cop examines the impact of a particular activity and articulates practical and measurable outcomes for its regulation. it is a set of rules that details specific behaviours for complying with a particular environment protection policy. a cop describes both requirements, which ‘must’ be observed to avoid breaching the provisions of the environment protection policy to which it is linked, and recommended practices, which ‘should’ be followed to achieve the desired outcomes. under the provisions of the environment protection act 1993, compliance with a cop’s requirements can be enforced through the issuing of an environment protection order (environment protection authority, 2019), but its agency is primarily exercised through collective acceptance of the guidelines and rules for acceptable behaviour that it articulates. regulation of vessel greywater management would have a greater chance of being effective and accepted in practice if the cop for vessel greywater management was constructed collaboratively with those most affected. thus, the process the epa followed in developing the greywater cop involved extensive consultation and negotiation with stakeholders in the boating industry, local government and the community (an exercise of the epa’s human agency). this process needed to facilitate convergence on a set of rules for vessel greywater practice that could shape behaviours upholding environment sustainability. at the same time, it needed to offer opportunities for the industry and the wider public to identify issues with the emerging rules and for these to be resolved before the cop came into effect. to this end, the epa established an external advisory group, which consisted of twenty-five representatives from local business and vessel facilities, commercial and recreational vessel operators, state and local government, and boating industry associations. composition of the external advisory group deliberately included key stakeholders of south australia’s waterways, both those directly or indirectly engaged in maritime activity and those affected by the threat to water quality. the external advisory group met on repeated occasions during 2004 to discuss and draft an initial cop that would implement best practice in environmental management for the sustainable use of south australia’s aquatic environments, including “reasonable and practicable measures” to prevent the discharge of untreated greywater from vessels on inland waters (environment protection authority, 2008a, p. 59). in order to be reasonable and practicable, the draft rules needed to be based on an understanding of the boating industry’s operational practices, the specific risks of greywater for water quality, the effects on environmental and human health, and potential greywater containment or treatment solutions. to inform the drafting process, the epa and the external australasian journal of information systems troshani et al. 2022, vol 26, selected papers from acis 2020 environmental management infrastructures 11 advisory group conducted a series of site visits in early 2004 to collect information on the industry’s current environmental performance. the epa also commissioned a report that investigated the impact of untreated greywater on water quality, reviewed the regulation of greywater discharge in a range of other countries, recommended minimum water quality standards for treated greywater, and reported on the availability of potential on-board greywater treatment systems (environment protection authority, 2004, as cited in northwest hydraulic consultants, 2010). further, the epa was able to draw on the results of a large risk assessment project initiated in 2004 to identify specific hazards to water quality along the murray river and recommend appropriate risk management options (environment protection authority, 2007a). the knowledge produced in these reports was useful in providing the epa with a credible, scientific basis for the need to change existing behaviours around the handling of vessel greywater. in a sense, the epa was seeking to enrol the agency of scientific knowledge. comprised of facts, conceptual representations, relationships and classifications, such systematic and formalised knowledge has a performative power, not just reflecting the world but intervening in it and contributing to its constitution (callon, 2007; jones, 2014). for example, specific numerical criteria for a range of parameters eventually became incorporated into the cop as indicators of acceptable water quality after the onboard treatment of vessel greywater (table 3). these scientific statements defined when greywater was polluting and when it was not, effectively constituting treated greywater (meeting these criteria) as no longer wastewater. inland waters vessel operators must (required outcomes): 1 not permit grey water to enter into any inland waters by: – retaining grey water on board the vessel for disposal into land-based wastewater collection system – ensuring that the vessel’s fixed grey water holding tank meet the requirements outlined in … 2 only discharge treated grey water if the following criteria are met (dilution is not acceptable): – suspended solids less than 50 mg/l, – a total grease content of less than 25 mg/l, – nitrogen content less than 10 mg/l, – phosphorous content less than 1 mg/l, – enterococci of less than 40 cells per 100 ml, and – biochemical oxygen demand is reduced by digestion, oxidation or other recognised treatment method. … table 3. examples of greywater management rules (from environment protection authority, 2008a, p. 60) the draft cop underwent several iterations during 2004 with revisions to various details made after discussion and resolution of points raised by the external advisory group members, including aspects related to the feasibility of greywater storage, treatment and discharge quality. this represented the initial tuning (pickering, 1995) of the incipient cop artifact as points proposed by the epa were accepted or contested by external advisory group members to a varying degree (an exercise of their agency), triggering a series of modifications to the draft in order to secure consensus. when a final draft of the rules had been agreed with the external advisory group, the epa released the draft cop for state-wide consultation and public comment in early 2005. the aim of the consultation process was to engage stakeholders to review the draft cop, both as a means of increasing its validity and as a source of constructive feedback for further revision. over 50 submissions were received from a range of stakeholders, including many industry operators. these were carefully considered by the epa and relevant australasian journal of information systems troshani et al. 2022, vol 26, selected papers from acis 2020 environmental management infrastructures 12 comments were either responded to with a justification for why the proposed changes could not be accommodated or addressed through modification of the draft cop in a process of further tuning. for example, in response to concerns about the structural limitations and cost of modifying small vessels to deal with greywater, a range of management strategies including behaviour change were included in the cop “in order to recognise the social and economic status of all vessel operators” [epa internal report, august 2007]. similarly, the cop introduced a series of compliance dates intended to reflect the size and use made of different vessels. in another example, expressed concerns about permanently moored vessels on the murray river resulted in the cop requiring “a higher level of environment protection and compliance for these types of vessels and their operators” [epa internal report, august 2007]. the rules that emerged from the work of the epa and the external advisory group and the ensuing consultation process were eventually issued as the code of practice for vessel and facility management (marine and inland waters) in 2008 (environment protection authority, 2008a). the greywater management requirements applied to all commercial and private vessels operating on inland waters but would be phased in over three years depending on the type of vessel (environment protection authority, 2008b). from the epa viewpoint, the rules prescribed specific behaviours and the conditions under which they were acceptable (or not) and aligned these with the broader environmental protection goals. with respect to the management of vessel greywater, the resultant cop defined two possible required outcomes (table 3): either preventing greywater from entering waterways or only discharging treated greywater. vessel owners and operators had the option to retain greywater onboard in holding tanks for subsequent disposal at some 13 specific land-based pump out stations along the 650km south australian part of the murray river (department of water, land and biodiversity conservation, 2009). however, the stations were often difficult to access due to the distance between stations, the number of vessels attempting to use them, or drought conditions affecting river levels – arguably, an exercise of the material agency of the river itself. as a result, the cop also included provisions that allowed vessels to discharge greywater into the river after it had been adequately treated onboard (e.g., by suitable filtering). the cop outlined a further 17 recommended practices that assisted vessel owners and operators to minimise the environmental impact of greywater (environment protection authority, 2008a). the cop was the product of the mangling of various human and disciplinary agencies. as a social artifact it was a material piece of the culture surrounding environmental governance and regulation in south australia. circulated, the cop was capable of serving a specific end in the practical accomplishment of water quality objectives (m. c. suchman, 2003; verbeek & vermaas, 2009). the rules and recommended practices to govern vessel greywater management encapsulated in the cop were specific and actionable. adherence to them by vessel owners and operators would bring about the desired change to the existing greywater management behaviour and reduce the risk of greywater pollution. however, to be effective the rules had to be applied and observed consistently across south australia. their formulation as a cop within the state’s legal and regulatory framework produced a device that the epa could use to enforce compliance with these provisions. emergence of the audit routine although the intention of the cop was “to encourage best environmental management practices for the benefit of future generations” (environment protection authority, 2008a, p. 1), enforcement of the cop regulations was eventually perceived as a necessary and effective australasian journal of information systems troshani et al. 2022, vol 26, selected papers from acis 2020 environmental management infrastructures 13 course of action for the epa: “gaining compliance through regulation was really the only option for managing greywater discharges” [field note memo #8]. enforcing regulatory compliance implied the need to monitor the manner and extent to which vessel owners and operators were complying with the cop. data had to be collected via systematic monitoring to evidence when greywater management practice was compliant with the cop and when it was not. the evidence collected could then be used by the epa to monitor compliance progress and, if necessary, enforce the cop by issuing environment protection orders to continued noncompliers. monitoring compliance with greywater requirements was achieved through a vessel ‘audit’ routine (environment protection authority, 2012a, 2015), first trialled in 2007 to ensure houseboats along the murray river had appropriate blackwater disposal systems (environment protection authority, 2007b). the experience gained in conducting the blackwater audits was influential in shaping the epa’s performance of the greywater audits, which commenced from 2011 (department of planning, transport and infrastructure, 2010). the audit routine involved epa officers collecting compliance data during inspections of vessels in the field. collected data included information about the owners or operators, the manner in which greywater was managed, and whether it was compliant with the cop. vessels were typically deemed to be compliant with the cop in relation to greywater management, when an epa officer could establish, in audits, that vessels had installed and were using an approved greywater treatment system that filtered greywater to the specifications prescribed in the cop. ‘approved’ greywater treatment systems were those that met a new standard (as 4995) specifically developed by standards australia to provide an industry benchmark and accreditation system for the design, construction, installation and operation of onboard greywater treatment systems for vessels operated on inland waters (environment protection authority, 2008b, 2009a; standards australia, 2009). the large number of vessels operating on south australia’s inland waters, the majority of them privately owned vessels, necessitated the use of random audits (environment protection authority, 2012a). some audits targeted specific geographical areas, generally those with wharves, marinas and thus concentrations of vessels. alternatively, epa officers would hire a houseboat and sail it along different reaches of the murray river, auditing every vessel seen along the way. smaller audits would typically cover 30-50 vessels, while larger audits could cover up to 100 vessels. epa staff needed some means of handling the data produced from the vessel audits. thus, intertwined with the new audit routine, a spreadsheet-based information system emerged. compliance data were collected manually in the field on paper forms and subsequently entered into spreadsheets at the epa office. image-based evidence documenting compliance complemented the vessel records in the spreadsheets. for example, a large number of photos were taken of each audited vessel. the photos and scanned copies of the original audit forms were archived in electronic folders in the epa network that, in time, grew in size and complexity, making quick retrieval difficult and impractical: “the photos would simply be lumped together in a folder archived somewhere within the epa network, and rarely looked at” [field note memo #1]. commercially operated vessels were handled differently, in collaboration with another south australian regulator. since all commercial vessels operating in south australia were surveyed biennially by the department of transport, it made sense for that department to incorporate a check for greywater management compliance in their inspection and certification process australasian journal of information systems troshani et al. 2022, vol 26, selected papers from acis 2020 environmental management infrastructures 14 (environment protection authority, 2009b). they shared this and other relevant data on the commercial vessels, including vessel identification and ownership information, with the epa. the data were received periodically in the form of a large spreadsheet. the information it provided was important for maintaining up-to-date records of addresses for correspondence and for maintaining accurate records of vessels that go in and out of survey. however, the epa staff encountered challenges in their handling of the vessel audit data. the process of paper-based data capture in the field followed by transfer of that data to spreadsheets was inefficient and prone to errors. in addition, the new audit routine generated large volumes of data that were difficult to manage and lacked consistency. subsequent processing of the data was therefore complex and challenging. typically, this entailed aggregation and disaggregation of data at different levels of granularity to fulfil the needs of different stakeholders. for example, stakeholders within the epa and outside it, such as other government departments and industry associations, needed different types of reports. additionally, the epa tracked all correspondence to vessel owners and operators that formalised the findings of epa officers during audits, and which required further action if the requirements of the cop were breached. 5.2 episode 2: mangling to informate emergence of the vessel management system artifact epa staff perceived that their use of spreadsheets to store and manipulate audit data would constrain their ability to monitor vessel greywater management compliance efficiently and effectively: “team members had identified that the current way the data was stored, using spreadsheets, was inefficient and time consuming” [field note memo #7]. the limitations in the epa’s performance of data management constituted a material resistance, a technological barrier to the epa’s goal of creating a new regulated vessel greywater management practice. consequently, they looked for a means to overcome these limitations, by “designing the system to speed things up” [field note memo #7]. in the second half of 2010, a small team at the epa commenced work on a requirements analysis and possible system functions for a new ‘vessel management system’. the decision to develop the vms in-house introduced a new set of human agents in the form of the epa’s applications and information management group. as information technology specialists, they would be likely to draw on existing material and disciplinary elements that were familiar and relevant to them, including programming languages and development platforms (chae & poole, 2005). a decision was taken to develop the vms using oracle database, a well-established relational database management system. as the vms developers sought to harness the material agency of this platform, the data that were previously stored in spreadsheets were restructured based on the pre-programmed requirements of the system, which redefined granularity in the way the data were specified and stored in a relational database. the vms user interface was developed using a web-based development tool. development proceeded quickly, with user testing and an early demonstration of the vms announced in early 2011. a key aspect of the new system was greater integration and consistency of the extensive data collected and maintained about individual vessels. these included ownership details, descriptions of on-board facilities (different types of facility were capable of producing greywater to varying extents) and multiple greywater treatment options (depending on the australasian journal of information systems troshani et al. 2022, vol 26, selected papers from acis 2020 environmental management infrastructures 15 type of facility), as well as details of any onboard greywater treatment system installed (e.g., technical specifications, manufacturer information, model and serial number, installation and maintenance dates), and the presence and status of any issues or defects that might affect a vessel’s compliance. as noted above, the vessel audits produced a large amount of associated image-based evidence, such as photographs, that could change over time. consequently, a photo album feature was implemented in the vms that enabled the building of image galleries for each vessel, with the capacity to store any number of photos necessary along with specific audit comments by epa officers for each photo, including dates and other contextual information about objects featured in the photo. correspondence record management was another important aspect of the epa’s monitoring activity that influenced the development of the new vms. previously, epa officers had undertaken mass mail-outs to vessel owners and operators asking for their plans on how they intended to become compliant with the cop. these letters generated a substantial subsequent correspondence that needed to be stored and differentiated in the database: “we are getting a large amount of correspondence from vessel owners claiming that their vessel is now compliant with all greywater requirements” [vms project email, april 2011]. the vms needed to distinguish these owners as ‘pending’ compliance, until their vessel was inspected. only then would a compliance letter be issued. similarly, they needed to be excluded from any future correspondence sent to ‘non-compliant’ vessel owners. it was also desirable that the new system had the functionality to automatically generate certain standard letters at different points in the compliance process, such as the compliance reminder letters. a major issue facing the vms developers was how to capture and integrate the data relating to commercial vessels received from the department of transport: “[we] had to think creatively to come up with solutions to other problems. such things included … storing the data that is obtained from transport” [field note memo #7]. to facilitate more frequent sharing of data, a ‘load module’ was built for the vms that could be used to update the vms database based on changes in department of transport’s records. the database structures used by the two regulators were significantly different, with the vms holding more information on a vessel, so development of the load module required a large amount of coding and discussions between the two agencies. the role of the load module was to regularly input the department of transport data into a contained section of the vms database and identify any differences between the records, which epa officers could then choose to update (or not) in the main vms schema. a change history allowed users to view or undo changes to maintain the data integrity of the epa records. in a very real way, the material performativity of the vms was mangled with the need for the epa to work with the department of transport. the nature of the collaborative relationship and the material form of the vms emerged together in practice and interactively stabilised each other (pickering, 1995). as the epa staff involved in compliance monitoring became familiar with the new vms, they proposed improvements to the way data were processed or recognised other opportunities to exploit the potential of a database-driven solution. consequently, the system developers continually adjusted the functionality of the vms in a process of incremental improvement. for example, early in the development process one epa officer asked: “can we please add another field to the database, which names the manager of the commercial houseboats? many houseboat companies manage a fleet of vessels (but they don’t own them) and [that] would be good to distinguish” [vms project email, october 2010]. in another example, while the various australasian journal of information systems troshani et al. 2022, vol 26, selected papers from acis 2020 environmental management infrastructures 16 boating industry associations in south australia did not actually own vessels, they were closely involved in the development of the cop and the subsequent compliance monitoring regime. it was decided to include them in the vms as a special class of ‘owner’, thereby leveraging the system’s functionality to record epa advice and handle the considerable correspondence the epa had with these associations: “these have been created due to the need to store the correspondence that the epa has with these organisations in a centralised location” [field note memo #2]. similarly, the realisation that over time vessels would be onsold meant that changes in ownership needed to be tracked so that all correspondence associated with a specific vessel could be maintained: “an owner history was developed for vms, where all previous owners and all correspondence with previous owners would be recorded” [field note memo #6]. in ways such as these, the material form and functionality of the vms was shaped by its entanglement with the vessel audit routine and the human agency of its users as they explored its possibilities. tuning the assemblage during 2011, epa officers began using the vms in their regular work, so that the vms quickly became central to the epa’s enactment of the vessel audit routine and performing key functions in their monitoring and compliance activities. for example, epa officers conducted a 10-day survey of a large section of the murray river, noting the location and numbers of vessels, and recording the registration details of all vessels that were not previously in the vms. their owners were then sent letters describing the greywater management requirements for vessels on inland waters and their obligation to comply with these requirements (spencer, 2011). the performativity of the vms greatly facilitated ease of access to relevant vessel information, efficient records management, flexible and structured report generation, communication with vessel owners, and information exchange with other regulators and agencies. its functionality allowed the epa to maintain up-to-date information concerning a vessel’s history and current compliance status. using the vms, epa officers could easily identify non-compliant vessel owners, send out warnings, and locate correspondence history concerning underlying causes and any ensuing course of action taken. because data were centralised, authorised epa officers could access historical information about individual vessels including specific advice and recommendations provided at prior inspections, thus avoiding duplicate or contradictory actions. it was also useful because it constituted a complete record for each vessel that could potentially be used as evidence when environment protection orders were issued, and legal action taken against vessel operators failing to comply with the requirements of the cop. the improved data processing and management offered by the vms encouraged epa staff involved in monitoring vessel greywater compliance to envisage ways that the system could be used to enhance their audit routine. for example, an epa officer asked a vms developer to extend an initial reporting function to a finer-grained level of analysis: “could you incorporate into each of the different audits on vms a similar compliance summary report as you have used in the overall compliance summary?” [vms project email, september 2011]. another example involved task management and the multitude of compliance dates that epa officers were responsible for keeping track of. while the initial vms allowed users to search and find all vessels that were nearing or past their compliance due date, “action items were developed to allow the system to have a task managing capacity” [field note memo #2]. the new functionality allowed epa officers to set dates for particular actions that needed to be australasian journal of information systems troshani et al. 2022, vol 26, selected papers from acis 2020 environmental management infrastructures 17 completed or followed up on, triggering subsequent system-generated reminders on login when the relevant date arrived. these examples demonstrate how the progressive tuning of the audit routine and the accompanying changes in how the epa audit staff worked could not have occurred in the ways they did apart from the material form of the vms. by early 2012, the requirement introduced by the cop for onboard containment or treatment of greywater on all commercial and private vessels operating on inland waters was fully phased in. while some 60% of commercial vessels had become compliant, in order to achieve equity across the boating industry the epa launched the next phase of the vessel greywater compliance program, involving further inspections, warning letters and, if necessary, environment protection orders (environment protection authority, 2012a): “regulation [enforcement] was only implemented following a significant education and consultation campaign. the houseboat industry as well as representatives from the private/recreational vessel owners were engaged and helped to develop the mechanism for transitioning the south australian houseboat fleet to best practice and vessel greywater management. regulation [enforcement] was only the final phase in the program to ensure equity across the fleet” [field note memo #8]. as one epa representative commented, “we’ve got to the point where … we're going to start being more forceful in our enforcement of the regulations” (australian broadcasting corporation, 2012). arguably, the mangling of the material agency of the vms, in facilitating and streamlining the monitoring of greywater management, with the human agency of the epa officers, who intended to “ramp up” (australian broadcasting corporation, 2012) their efforts, enabled the enhancement of the vessel audit routine towards ensuring more complete regulatory compliance. in increasing their activities around ensuring vessel greywater compliance, epa staff encountered further resistances affecting the efficacy of their practical monitoring work. these constraints prompted the epa to consider and develop refinements in both the vms and the manner in which it was used in practice in attempted accommodations to overcome them. one issue revolved around data verification in the field. the vms was located inside the epa’s intranet and could not be accessed externally, for example by epa officers conducting a vessel audit. this meant that when talking with vessel operators and owners about greywater management and their vessel’s compliance status, officers did not have real-time access to upto-date information held in the vms. during an audit, some vessels owners would make claims to epa officers in relation to specific compliance issues. without access to the vms, officers were unable to verify these claims or provide appropriate advice. the material configuration of the vms was constraining the ability of epa officers to operate efficiently and effectively in the field. the epa and its vms developers were receptive to enhancing the mobility of the vms and work began on a solution. new functionality was developed that enabled a version of the vms database to be copied before an audit onto a handheld computer specifically designed to withstand rough handling in field conditions. taken with the epa officers on a vessel audit, this arrangement provided the officers with the ability to access up-to-date records while working at a distance from their office. the human agency of the epa officers was thus mangled with the material performativity of the mobile device to extend the reach of the vms. another issue identified by epa officers concerned the behaviour of some vessel operators that in effect circumvented the greywater treatment requirements of the cop. it became australasian journal of information systems troshani et al. 2022, vol 26, selected papers from acis 2020 environmental management infrastructures 18 apparent that on occasion vessel operators were disconnecting their vessel’s greywater treatment system and using the overflow pipe attached to the system to discharge untreated greywater directly into the river. typically, this action was resorted to when the installed greywater treatment system did not operate as intended or stopped working during a voyage – a failure in the material agency of the treatment system upon which the vessel operator was relying to be compliant with the cop. the reliability of the treatment systems approved for use under the cop was becoming of increasing concern for houseboat owners. nevertheless, the epa was insistent that compliance was necessary, even if that meant containing the greywater onboard until it could be pumped out at one of the disposal stations along the river (jean, 2012). the non-compliant use of overflow pipes prompted consideration by the epa on how best to fix the problem. at the time data collection ended in 2013, the envisaged solution involved attaching an ‘overflow tag’ to the overflow pipes or taps in a vessel’s greywater treatment system. the tag would break if the pipe was used to discharge greywater, and the date would be automatically recorded. overflow tag data would be captured during vessel audits and surveys and included in the vms database as part of the compliance history for vessels when applicable. a new vms function would be developed to track the overflow tag data. the material agency of the overflow tags and the vms would together afford the possibility of a more accurate and effective performance of the vessel greywater audit routine. over time, the greywater management infrastructure introduced by the epa was successful in achieving a high level of compliance for vessels operating south australia’s inland waters (brennan, 2013). data retrieved from the vms in early 2014 indicated that 82% of commercial vessels and 77% of recreational vessels on inland waters were greywater compliant with the cop, with the most common reasons for non-compliance being significant financial constraints, absentee or new owners and extreme climate events. as the heterogeneous arrangements based around the vessel audit routine and the cop and vms artifacts stabilised, the epa began to turn its attention to vessels operating in the state’s offshore waters. this would involve a further extension of greywater management practice to a new environment: “once we start to get on top of the vessels on inland waters we will need to turn our focus to marine waters” [vms project email, april 2011]. 5.3 emergence of environmental management infrastructure our analysis has explored how the genesis, form and function of an environmental information system lay in its inclusion within a broader infrastructure developed to support a particular environmental management practice. this infrastructure was a heterogeneous assemblage of individual and collective human agents, regulatory and technological artifacts, and the sociomaterial routines in which they were incorporated. figure 2 depicts our conceptualisation of the emergence of this environmental management infrastructure. consistent with the mangling approach we have taken to practice and its change (pickering, 1995), we view infrastructure as the interactive stabilisation of entangled artifacts and routines (visualised as entwined strands of a rope). an orthogonal cut (baygi et al., 2021) through the whole assemblage reveals the dynamic mangle of human, disciplinary and material agencies. this mangling includes transformations in the material form and function of various social and technical artifacts, and the production of disciplined human performances in which human intentionality, conceptual understandings, and relations between social actors are transformed. critically, infrastructure evolves from the field of existing artifacts, practices and australasian journal of information systems troshani et al. 2022, vol 26, selected papers from acis 2020 environmental management infrastructures 19 disciplines, which collectively constitute the ‘surface of emergence’ for its open-ended development. this in turn requires ongoing ‘tuning’ in order to stabilise the heterogeneous arrangements in the evolving assemblage of artifacts and routines. figure 2. emergence of environmental management infrastructure 6 discussion based on our analysis of the emergence of an environmental management infrastructure, we suggest four central implications for understanding how information systems become an integral part of environmental management practice. 6.1 looking back and looking forward our findings suggest that an environmental information system is neither a stand-alone artifact nor acontextual. it is located within and emerges from a broader infrastructure of environmental management. as such, it has a biography that includes a past and a future. this quality of temporal emergence (pickering, 1995) highlights how its current state arises from its previous history and context, and how it is in a process of continual becoming (abbott, 2016; pickering, 2008). in our analysis, we highlighted the material transformations that the vms underwent in its evolution from a spreadsheet-based precursor to a computer-based information system disciplined by the properties of relational database management, and its ongoing modification in response to requirements arising from its use in practice. the development of the vms was intimately intertwined with the epa’s attempt to extend its environmental management practice around the issue of vessel greywater management. following pickering’s (1995) extension of practice, this involved the epa envisioning a desired future state, selecting an approach that would get them there from their existing practice, and australasian journal of information systems troshani et al. 2022, vol 26, selected papers from acis 2020 environmental management infrastructures 20 working in real time to find out where that approach led. the form and outcome of this process could not be known in advance. thus, temporal emergence is also characteristic of the environmental management infrastructures in which information systems are embedded. our analysis demonstrates how the trajectory of emergence (pickering, 1995) of the greywater management infrastructure developed by the epa reflects the consequences of choices made and actions taken at particular junctures: “a chain of multidimensional links which continue to expand from previous assemblages and new actions” (abbott, 2016, p. 241). this trajectory is not a predictable path. environmental management infrastructures unfold in a nondeterministic way in the real time of practice. this means that one cannot know ex ante the shape of the infrastructure (and its parts), nor its future evolution (ciborra & hanseth, 2000). emergence of both environmental information systems and infrastructures is a consequence of the mangle. the epa’s extension of their environmental management practice to encompass vessel greywater management included the creation of artifacts such as the cop and vms and their incorporation in the vessel audit routine. the intersection of different agencies in the arrangements that the epa instigated produced a sequence of different challenges to their successful performance. in turn, human agents monitoring those performances put in place various accommodations that contingently responded to or attempted to circumvent resistances that arose as the infrastructure pushed back. at each stage of the trajectory of emergence, the mangle was in operation. the new practice of vessel greywater management needed ongoing tuning to adjust the contours of the different agencies involved and enrol them in pursuit of the epa’s goals. this tuning constituted a learning process, the results of which the epa successfully translated into a high level of regulatory compliance for vessels on south australia’s inland waters (and ultimately was able to contemplate extending even further to the marine environment). 6.2 shifting focus the epa’s engagement with the world in extending its practice was not simply a straightforward capture of the material agency of information technology. it also involved the production and deployment of knowledge in particular forms (e.g. knowledge of the properties of greywater, the delineation of known hazards and risks to water quality), the exploration and establishment of social relations with other actors, and the fabrication of social artifacts such as regulatory devices. this meant that the epa needed to assemble, align and make associations between a diverse assemblage of human and nonhuman elements – an interactive stabilisation of human and material performances (pickering, 1995). in this emergent assemblage, a range of nonhuman agencies were relevant and imposed themselves on the work of the epa. notwithstanding the fundamental role of naturally occurring objects (e.g. the characteristics of greywater that constitute it as pollution, the effect of low river levels on access to land-based wastewater disposal stations), the material forms and performances of a range of social and technical artifacts were influential in this regard. while the vms improved the efficiency and effectiveness of audit data processing through the performance of its programming, the cop prescribed regulatory compliance through its collective endorsement by the different stakeholders implicated in its circulation (e.g., the epa, the boating community, other users of the murray river). it was the practical instantiation of abstract concepts (leonardi, 2010) – ‘musts’ and ‘shoulds’ (environment protection authority, 2019) made material and given force through the status and position of the cop in the south australian environmental legal and regulatory framework. australasian journal of information systems troshani et al. 2022, vol 26, selected papers from acis 2020 environmental management infrastructures 21 acknowledging nonhuman performativity means affording an active role to natural and made objects in the development and operation of environmental management infrastructures. it necessitates a shift in focus to make visible the entanglement of human actors and nonhuman entities that traced out the trajectory of the epa’s environmental management practice (pickering, 1995). this is a posthumanist decentring of the human subject, changing the balance of the analysis towards a more symmetrical accounting for human and material agencies, and a move to examining the whole sociomaterial assemblage as the object of analysis. 6.3 intentionality and the shaping of human agency this is not to say that the human agent disappears from consideration in the examination of environmental management infrastructures. in our analysis, the epa is not reduced to a passive bystander, an effect of other agencies. it is clear that the epa played a leading role in orchestrating the emerging greywater management infrastructure. the particular accountabilities of the epa as an environmental regulator (pickering, 1995; orlikowski, 2005; l. suchman, 2007) led it to construct the goals that would bring about the intended extension of environmental management practice. it was the epa’s resources, skills and actions that set various nonhuman performances in motion or attempted to channel or control their agency (pickering, 1995). the epa had the power and legitimacy to both speak for and act to protect the environment through its institutional position in south australia’s environmental legislative framework. it had the resources to fund discursive and infrastructural development activity that ultimately aligned the interests of diverse stakeholders and produced the social and technical artifacts and their associated routines that formed the heart of the greywater management infrastructure. however, the development of an environmental management infrastructure was not fully under the control of the epa. disciplined by the environmental regulatory regime in south australia, it was not completely free to choose how to perform regulatory practice around greywater management. the discretion of its human agency was reduced. established ideas of proper procedures and ways of acting within the epa’s own disciplinary practices as an environmental regulator shaped how it proceeded (johnston et al., 2016) – assembling relevant knowledge of the greywater problem, engaging with its stakeholders, and manoeuvring within the environmental governance regime in south australia. similarly, the decision to use a relational database management system disciplined the development and form of the vms to an extent as the specialised knowledge of information technology professionals was brought to bear on the emergent system. the epa was also active in identifying both limitations and possibilities in the emerging infrastructure as it continuously revised its short-term plans and goals while maintaining its mission to achieve environmental sustainability in the longer term. its human agency did not proceed unchanged in the mangle of practice. accommodations subjected its intentions to revision as it dealt with a variety of human and material resistances, ranging from the inertia of vessel owners reluctant to change established ways of handling greywater, mis-performing greywater treatment systems, or drought-induced low river levels. this was apparent in the perceived need to differentiate requirements based on the physical limitations of vessel size and the economic status of vessel owners, and the phased approach eventually adopted to implementing the greywater requirements of the cop. australasian journal of information systems troshani et al. 2022, vol 26, selected papers from acis 2020 environmental management infrastructures 22 the social contours of the epa’s agency were transformed in the mangle of practice. the actions of other human actors were often significant, for example in setting high-level aims, highlighting concerns, or creating practical issues that needed to be overcome. the epa needed to align the interests of these multiple, different organisations and actors as they interacted during the infrastructure construction processes. acting as the legitimate ‘spokesperson’ for the environment, the epa defined substantive criteria for what constituted greywater pollution and its associated risks. the development and promulgation of the cop that materialised these criteria was the product of extensive negotiation with a range of stakeholders who wanted to be able to carry out boating-related activities on south australia’s waterways. while the cop was a legally enforceable mechanism for protecting the aquatic environment from greywater pollution, it also allowed the activities that produce vessel greywater, by defining permissible ways for dealing with produced greywater. these associations were constitutive of new relationships and roles in the boating industry in south australia. the epa’s relationship with vessel owners and operators (and other related stakeholders) was formalised in specific terms in relation to vessel greywater management with the creation and performance of the artifacts and routines comprising the greywater management infrastructure. for example, vessel users were expected to comply with the regulatory agency of the cop, to evidence compliance status or claims based on the data requirements of the vms, and also to subject their vessels to monitoring via the epa’s audit routine. similarly, the epa developed relationships with other state entities, including standards australia, with whom it collaborated in the creation of a new a standard for acceptable greywater treatment. the epa also extended its relationship with the department of transport in seeking to regulate greywater management practice for commercial vessels. this relationship was mediated via the ‘load module’ in the vms and department of transport routines which subsequently informed the epa. this coordination and alignment of the interests of different actors through the agency of social and technical artifacts is consistent with prior literature on infrastructure in other contexts (e.g., venters et al., 2014). 6.4 entangled artifacts and routines we have suggested that artifacts and routines are the observable traces of the mangle in the extension of environmental management practice. our analysis demonstrates how the routines and social and technical artifacts developed by the epa were co-implicated in the emergence of the greywater management infrastructure. the epa’s agency was materially performed through artifacts, and these material performances were enacted by human agents in routines (orlikowski, 2005). each artifact, and its (re)configurations and adjustments contributed to reducing the instability brought forth by resistances in the evolving infrastructure through their performance in associated routines and practices. in turn, routines created through the exercise of human agency were modified through exposure to various material agencies in order to better perform vessel greywater management in practice. when the epa decided to develop a computer-based information system to facilitate monitoring of greywater management practice, the earlier alignments produced by the cop artifact had to be fine-tuned. the material agency of the vms became evident in how the properties of the relational database management system underlying the vms allowed greater integration and consistency of the compliance data collected and maintained about vessels. the audit routine for monitoring compliance also developed in line with emergent refinements to the vms as practical issues were encountered and accommodated in the evolving greywater australasian journal of information systems troshani et al. 2022, vol 26, selected papers from acis 2020 environmental management infrastructures 23 management infrastructure. these included integration of the system with another regulator so as to capture data relating to commercial vessels, the needs of epa audit staff for access to data in the field, and the actions of some vessel owners in circumventing the greywater treatment requirements of the cop. while the performativity of the vms in practice materially influenced the ongoing development of the vessel audit routine, the material form of the vms was also itself at stake in these encounters. focusing on the entanglement of artifacts and routines is a helpful approach for tracing and analysing the process by which environmental management infrastructures are constituted and evolve over time. the ability of the epa to discipline vessel users and shift their entrenched practices of handling greywater is a consequence of the particular relations between the artifacts it created and their performance in the audit routine with which they are associated. the effects of the greywater management infrastructure cannot be reduced to the separate effects of the cop and vms artifacts and the corresponding audit routine. instead, they rely on the relative stability of their association in a specific, context-dependent configuration, itself the result of ongoing tuning in the mangle of practice. the emergent properties of the infrastructure are properties of the whole assemblage that would not exist if the individual parts were not assembled, organised and enacted in that particular way (eldervass, 2007). the reconfiguration and extension of environmental management practice involves the creation and interactive stabilisation of new social and technical artifacts and the social relations and disciplined human performances that accompany them in routines and practices (pickering, 1995). 7 conclusion taking the position that environmental management is a sociomaterial practice, we have outlined a theoretical perspective that draws on some of the key ideas and concepts behind the ‘mangle of practice’ to account for the emergence of an infrastructure in the extension of practice to regulate vessel greywater management in south australia. our perspective explains how information systems underpinning environmental management infrastructures, together with other social and technical artifacts, are entangled with the routines in which they are enacted through ongoing interactions between human and material agencies. this mangling of practice is constitutive of change in environmental governance as certain actors such as environmental regulators seek to create associations between human and nonhuman entities, align their interests, and channel their agencies. we have shown that human intentionality, in the form of the longer-term goal of the regulator, plays an important role in shaping the initial infrastructure, although pre-disciplined by established concepts and procedures entrenched in existing practice and vulnerable to revision in the short term. however, the exact form and destination of the infrastructure cannot be defined a priori. instead, it emerges in practice over time as tensions and practical resistances arise in the intersection of different agencies that prompt actors to make accommodations on the path to that goal in a continual process of tuning. equally, we have highlighted the centrality and material effects of nonhuman performances, both natural and artifactual, in this emergence. focusing our analysis on the whole sociomaterial assemblage avoids reducing explanations of environmental change to human factors alone. from a practical perspective, the emerging infrastructure, with regulatory artifacts, information systems and associated routines, is indicative of possible ways change in australasian journal of information systems troshani et al. 2022, vol 26, selected papers from acis 2020 environmental management infrastructures 24 environmental management and sustainability can proceed. it suggests the need for an awareness by regulatory authorities and other actors involved of how the mangling of environmental management practice can constitute particular interactions between human agency and nonhuman performances, both in terms of the natural environment and the social and technical artifacts produced by those seeking to manage it. this offers the potential to identify opportunities for influencing the outcomes of such manglings, whether these are material transformations, changes in social roles and relations, or transformations in the concepts and knowledge informing practice. in the process, it may be possible to (re)shape the contours of environmental management practice and even shift the trajectory of future infrastructure development into a closer alignment with long-term sustainability goals. for practitioners seeking to deploy information technology and systems in the service of environmental management, our research indicates the importance of understanding the historical context of change and the influence of past and current practices on their efforts. rather than assuming a development trajectory exclusively driven by the agency of the developers, practitioners must be aware of the potential resistances arising from its intersection with a range of material and human agencies when the environmental information system is entangled in the practices and routines in which it is implemented. our analysis is based on an in-depth, longitudinal case study of a greywater management infrastructure developed by a south australian environmental regulator. our findings, though particular to the trajectory of this specific environmental management infrastructure, offer insights that can inform the analysis of infrastructures in other settings. for example, our analysis has identified two mangling episodes where key artifacts and routines emerged to regulate a particular sociomaterial practice and then informate the monitoring of that practice. insights at this level may be transferable to other environment management infrastructures, under circumstances when regulatory authorities attempt to regulate human practices that are harmful to the natural environment. our more detailed findings pertaining to the particularities of the greywater management infrastructure in our case study may need to be considered more carefully before they can be applied to other contexts. overall, however, the broader posthumanist arguments about temporal emergence and performativity of practice underscore the importance of considering any infrastructure in its own right. references abbott, o. (2016). posthumanist perspectives and the chernobyl disaster: dances of agency, temporal emergence, and disaster risk management. international journal of mass emergencies and disasters, 34(2), 231-249. australian broadcasting corporation. (2012, march 22). boaties warned of greywater crackdown. abc regional news. retrieved from australia/new zealand reference centre (ebsco) on 6 may 2019. baker, l. r. (2004). the ontology of artifacts. philosophical explorations, 7(2), 99-111. baygi, r. m., introna, l. d., & hultin, l. (2021). everything flows: studying continuous sociotechnological transformation in a fluid and dynamic digital world. mis quarterly, 45(1), 423-452. brennan, b. (2013, january 16). fifty shades of greywater. the murray valley standard. australasian journal of information systems troshani et al. 2022, vol 26, selected papers from acis 2020 environmental management infrastructures 25 retrieved from https://www.murrayvalleystandard.com.au/story/1238200/fifty-shadesof-greywater/ burazin l. (2019). law as an artifact. in m. sellers, & s. kirste (eds) encyclopedia of the philosophy of law and social philosophy. dordrech: springer. https://doi.org/10.1007/978-94-007-67300_217-1 callon, m. (2007). what does it mean to say that economics is performative? in d. mackenzie, f. muniesa, & l. siu, (eds), do economists make markets? on the performativity of economics (pp. 311-357). princeton, nj: princeton university press. chae, b., & poole, m. s. (2005). the surface of emergence in systems development: agency, institutions, and large-scale information systems. european journal of information systems, 14(1), 19-36. ciborra, c. u., & hanseth, o. (2000). introduction: from control to drift. in c. u. ciborra ciborra, k. braa, a. cordella, b. dahlbom, a. failla, o. hanseth, v. hepsø, j. ljungberg, e. monteiro, & k. a. simon (eds), from control to drift: the dynamics of corporate information structures (pp. 1-11). oxford: oxford university press. d’adderio, l. (2011). artifacts at the centre of routines: performing the material turn in routines theory. journal of institutional economics, 7(2), 197-230. department of planning, transport and infrastructure. (2010, november 30). blackwater audits underway. on deck. retrieved from department of water, land and biodiversity conservation. (2009). river vessel waste disposal options: edition no 2. retrieved from https://www.epa.sa.gov.au/files/477475_riverboat.pdf edwards, p. n., bowker, g. c., jackson, s. j., & williams, r. (2009). introduction: an agenda for infrastructure studies. journal of the association for information systems, 10(5), 364-374. elder-vass, d. (2007). luhmann and emergentism: competing paradigms for social systems theory? philosophy of the social sciences, 37(4), 408-432. elliot, s., & webster, j. (2017). special issue on empirical research on information systems addressing the challenges of environmental sustainability: an imperative for urgent action. information systems journal, 27(4), 367-378. environment protection authority. (2001). environment protection authority annual report 2000– 2001. retrieved from https://www.epa.sa.gov.au/files/8271_annualreport0001.pdf environment protection authority. (2002). environment protection authority annual report 2001– 2002. retrieved from https://www.epa.sa.gov.au/files/8272_annualreport0102.pdf environment protection authority. (2003a). state of the environment report for south australia 2003. retrieved from https://www.epa.sa.gov.au/files/4771457_soe_index.pdf environment protection authority. (2003b). environment protection (water quality) policy 2003 and explanatory report. retrieved from https://www.epa.sa.gov.au/files/477390_epwq_report.pdf environment protection authority. (2007a). river murray and lower lakes catchment risk assessment for water quality: results and management options. retrieved from https://www.epa.sa.gov.au/files/8598_risk_2_s8.pdf https://doi.org/10.1007/978-94-007-6730-0_217-1 https://doi.org/10.1007/978-94-007-6730-0_217-1 https://www.epa.sa.gov.au/files/477475_riverboat.pdf https://www.epa.sa.gov.au/files/8271_annualreport0001.pdf https://www.epa.sa.gov.au/files/8272_annualreport0102.pdf https://www.epa.sa.gov.au/files/4771457_soe_index.pdf https://www.epa.sa.gov.au/files/477390_epwq_report.pdf https://www.epa.sa.gov.au/files/8598_risk_2_s8.pdf australasian journal of information systems troshani et al. 2022, vol 26, selected papers from acis 2020 environmental management infrastructures 26 environment protection authority. (2007b). houseboat audits. ce news [external newsletter], november 2007, p. 2. retrieved from https://www.epa.sa.gov.au/files/4771761_ce_news_002.pdf environment protection authority. (2008a). code of practice for vessel and facility management (marine and inland waters). adelaide: environment protection authority. environment protection authority. (2008b, october 7). new standard for greywater treatment. [media release]. retrieved from https://www.epa.sa.gov.au/files/4771560_media209.pdf environment protection authority. (2009a, september 14). new australian standard now available for greywater treatment systems. [media release]. adelaide: environment protection authority. environment protection authority. (2009b, november 25). greywater deadline extensions for commercial river boats [media release]. adelaide: environment protection authority. environment protection authority. (2012a). strategic plan 2012-2015. retrieved from https://www.epa.sa.gov.au/files/4771468_strategic_1215.pdf environment protection authority. (2012b, march 16). epa launches compliance program for vessel wastewater. [media release]. retrieved from https://www.epa.sa.gov.au/files/477894_media_16mar2012.pdf environment protection authority. (2015, june 12). houseboat greywater management audits. [media release]. retrieved from https://www.epa.sa.gov.au/files/10911_media_12jun2015.pdf environment protection authority. (2019). environment protection legislation. retrieved from https://www.epa.sa.gov.au/data_and_publications/legislation eriksson, e., auffarth, k., henze, m., & ledin, a. (2002). characteristics of grey wastewater. urban water, 4, 85-104. faulkner, p., & runde, j. (2019). theorizing the digital object. mis quarterly, 43(34), 1279-1302. gallagher, n., & wigley, s. (2001). ecological impact of houseboats on the river murray in south australia. adelaide: murray darling association incorporated for conservation and sustainable development. gaskin, j., berente, n., lyytinen, k., & yoo, y. (2014). toward generalizable sociomaterial inquiry: a computational approach for zooming in and out of sociomaterial routines. mis quarterly, 38(3), 849-871. harvey, p., jensen, c. b., & morita, a. (2017). introduction: infrastructural complications. in p. harvey, c. b. jensen, & a. morita (eds), infrastructures and social complexity: a companion (pp. 1-22). abingdon: routledge. huvila, i. (2016). awkwardness of becoming a boundary object: mangle and materialities of reports, documentation data, and the archaeological work. the information society, 32(4), 280-297. jean, d. (2012, may 25). epa angers houseboat owners with waste threat. the advertiser. retrieved from https://www.adelaidenow.com.au/news/epa-angers-houseboat-ownerswith-waste-threat/news-story/031ee85ecc6a8dd15ca5793f433bb555 https://www.epa.sa.gov.au/files/4771761_ce_news_002.pdf https://www.epa.sa.gov.au/files/4771560_media209.pdf https://www.epa.sa.gov.au/files/4771468_strategic_1215.pdf https://www.epa.sa.gov.au/files/477894_media_16mar2012.pdf https://www.epa.sa.gov.au/files/10911_media_12jun2015.pdf https://www.epa.sa.gov.au/data_and_publications/legislation https://www.adelaidenow.com.au/news/epa-angers-houseboat-owners-with-waste-threat/news-story/031ee85ecc6a8dd15ca5793f433bb555 https://www.adelaidenow.com.au/news/epa-angers-houseboat-owners-with-waste-threat/news-story/031ee85ecc6a8dd15ca5793f433bb555 australasian journal of information systems troshani et al. 2022, vol 26, selected papers from acis 2020 environmental management infrastructures 27 jenkin, t. a., webster, j. & mcshane, l. (2011). an agenda for ‘green’ information technology and systems research. information and organization, 21(1), 17-40. jensen, c. b. (2015). experimenting with political materials: environmental infrastructures and ontological transformations. distinktion: scandinavian journal of social theory, 16(1), 1730. jensen, c. b., & morita, a. (2015). infrastructures as ontological experiments. engaging science, technology, and society, 1, 81-87. jensen, c. b., & morita, a. (2017). introduction: infrastructures as ontological experiments, ethnos, 82(4), 615-626 johnston, r. b., reimers, k., & klein, s. (2016). performing research validity: a “mangle of practice” approach. in l. introna, d. kavanagh, s. kelly, w. orlikowski, & s. scott (eds), beyond interpretivism? new encounters with technology and organization (pp. 201-214). cham, switzerland: springer. jones, m. (2014). a matter of life and death: exploring conceptualizations of sociomateriality in the context of critical care. mis quarterly, 38(3), 895-925. kallinikos, j. (2012). form, function, and matter: crossing the border of materiality. in p. m. leonardi, b. a. nardi, & j. kallinikos (eds), materiality and organizing: social interaction in a technological world (pp. 67-87). oxford: oxford university press. kroes, p. (2010). engineering and the dual nature of technical artefacts. cambridge journal of economics, 34(1), 51-62. leonardi, p. m. (2010). digital materiality? how artifacts without matter, matter. first monday, 15(6). https://journals.uic.edu/ojs/index.php/fm/article/download/3036/2567 leonardi, p. m. (2011). when flexible routines meet flexible technologies: affordance, constraint, and the imbrication of human and material agencies. mis quarterly, 35(1), 147-167. northwest hydraulic consultants. (2010). review of greywater management strategies to improve public health and water quality in shuswap lake. prepared for the fraser basin council, kamloops, bc. retrieved from https://www.fraserbasin.bc.ca/_library/ tr_slipp/slipp_nhc_greywater_report_july2010.pdf orlikowski, w. j. (2005). material works: exploring the situated entanglement of technological performativity and human agency. scandinavian journal of information systems, 17(1), 183– 186. orlikowski, w. j. (2007). sociomaterial practices: exploring technology at work. organization studies, 28(9), 1435-1448. pentland, b. t., & feldman, m. s. (2008). designing routines: on the folly of desiging artifacts, while hoping for patterns of action. information and organization, 18(4), 235-250. pickering, a. (1995). the mangle of practice: time, agency and science. chicago: university of chicago press. pickering, a. (2008). new ontologies. in a. pickering & k. guzik (eds), the mangle in practice: science, society, and becoming (pp. 1-14). durham and london: duke university press. https://journals.uic.edu/ojs/index.php/fm/article/download/3036/2567 australasian journal of information systems troshani et al. 2022, vol 26, selected papers from acis 2020 environmental management infrastructures 28 pickering, a. (2013). living in the material world. in f. x. de vaujany & n. mitev (eds), materiality and space: organizations, artefacts and practices (pp. 25-40). london: palgrave macmillan. salter, c. (2001, january 12). grey water ban? murray valley standard. retrieved from infotrac newsstand on 8 may 2019. sandberg, j., & tsoukas, h. (2011). grasping the logic of practice: theorizing through practical rationality. academy of management review, 36(2), 338-360. seidel, s., recker, j., & vom brocke, j. (2013). sensemaking and sustainable practicing: functional affordances of information systems in green transformations. mis quarterly, 37(4), 1275-1299. singh, m. & sahu, g. p. (2020). towards adoption of green is: a literature review using classification methodology. international journal of information management, 54, article id: 102147. spencer, j. (2011). 2011 river murray risk assessment. the rambler, 29(4): 1, 7. river murray boat owners association. standards australia. (2009). greywater and the sa epa. standards australia bulletin, #2_10 september 2009, p. 1. retrieved from https://www.standards.org.au/standardau/ media/sa-archive/ ourorganisation/news/documents/ bulletin_2_standards_australia_100909.pdf star, s. l. (1999). the ethnography of infrastructure. american behavioral scientist, 43(3), 377399. suchman, l. (2007). human-machine reconfigurations: plans and situated actions (2nd ed.). cambridge: cambridge university press. suchman, m. c. (2003). the contract as social artifact. law & society review, 37(1), 91-142. symon, g., & pritchard, k. (2015). performing the responsive and committed employee through the sociomaterial mangle of connection. organization studies, 36(2), 241-263. united nations. (2018). the sustainable development goals report 2018. retrieved from https://unstats.un.org/sdgs/files/report/2018/thesustainabledevelopmentgoalsreport2 018-en.pdf venters, w., oborn, e., & barrett, m. (2014). a trichordal temporal approach to digital coordination: the sociomaterial mangling of the cern grid. mis quarterly, 38(3), 927949. verbeek, p. p., & vermaas, p. e. (2009). technological artifacts. in j.k. berg olsen, s.a. pedersen, & v. hendricks (eds.), a companion to the philosophy of technology (pp. 165171). west-sussex, uk: blackwell. walsham, g. (1995). interpretive case studies in is research: nature and method. european journal of information systems, 4(2), 74-81. williams, r., & pollock, n. (2012). moving beyond the single site implementation study: how (and why) we should study the biography of packaged enterprise solutions. information systems research, 23(1), 1-22. australasian journal of information systems troshani et al. 2022, vol 26, selected papers from acis 2020 environmental management infrastructures 29 copyright: © 2022 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v26i0.3829 http://creativecommons.org/licenses/by-nc/3.0/au/ https://doi.org/10.3127/ajis.v26i0.3829 1 introduction 2 theoretical perspective 2.1 the mangle of practice 2.2 the extension of practice 2.3 artifacts and routines 2.4 synthesis 3 case background 4 method 5 findings and analysis 5.1 episode 1: mangling to regulate 5.2 episode 2: mangling to informate 5.3 emergence of environmental management infrastructure 6 discussion 6.1 looking back and looking forward 6.2 shifting focus 6.3 intentionality and the shaping of human agency 6.4 entangled artifacts and routines 7 conclusion paper title (paper title style) australasian journal of information systems joshi, das & sekar 2023, vol 27, research article how personality traits affect ict use 1 how big five personality traits affect information and communication technology use: a meta-analysis arun joshi indian institute of technology kharagpur, west bengal, india international school of business and media, pune, india joshiarun72@gmail.com saini das indian institute of technology kharagpur, west bengal, india srinivasan sekar indian institute of technology kharagpur, west bengal, india abstract this study performed a meta-analysis of forty-eight studies to synthesize existing literature examining the relationship between ‘big five’ personality traits and the use of various information and communication technologies (icts). we conducted sub-group analysis to investigate the potential moderators on the relationship between personality and ict use. the results largely reveal that the ‘big five’ personality traits are significantly associated with the use of various icts. specifically, ‘extroversion’ showed the strongest association with social networking, along with business and commerce-based icts, while ‘openness’ had the highest correlations with career and education, and information-based icts. the results also identified technology type, region of the country, and voluntariness as potential moderators. this paper offers theoretical and practical implications that researchers could embrace in enhancing understanding of traits-technology fit, and technology providers in improving crafting, marketing, and delivering technology at the individual, organizational, national, and global levels. keywords: big five personality traits, information and communication technologies, metaanalysis, correlations, sub-group analysis. 1 introduction information and communication technologies (icts) have radically changed human lives in the last two decades. ict comprises various technologies that facilitate information management, while assisting in various forms of communication (amutha, 2020). the recent growth of icts has effectively reduced the distance between people across the globe with just a click. such technological advancements enable people to access a wide array of services and online applications (e.g., e-learning/online teaching platforms, social networking sites or applications, online banking applications, and e-commerce applications, among others) using their smartphones and/or computers (corcoran & duane, 2018; da silva oliveira & chimenti, 2021; fernandes et al., 2021). icts thereby have emerged as an indispensable part of contemporary lifestyle, whereby individual personality effortlessly meshes with the fabric of human life, everyday situations, problems, and relationships (alhassan et al., 2020; donvito et al., 2020; goyal et al., 2021; soral et al., 2020). extant literature is of the view that an individual australasian journal of information systems joshi, das & sekar 2023, vol 27, research article how personality traits affect ict use 2 with a specific disposition exhibits a unique pattern of adoption and use1,2, of various forms of technologies (ratchford & ratchford, 2021; stachl et al., 2017). therefore, in this technological age, it is of the utmost importance to examine the relationship between personality traits, and use of distinct icts to understand who uses what type of icts. scholars in the past, have made several attempts to investigate the same based on certain popular theories and models of personality; for example: myerbriggs type indicator (mbti) model of personality (brown & akroyd, 2006), personal innovativeness in information technology (piit) and openness (davis et al., 2007), big five model of personality (lane, 2012), and trust model (kipnis, 1996). extant literature highlighted that the ‘big-five’ personality model has been considered the most parsimonious and comprehensive approach to measuring personality, comprising five different continuums (mccrae et al., 2008). it has been well-validated and reiterated that big-five traits significantly influence the choice of icts (hamari & keronen, 2017; huang, 2019; kayiş et al., 2016; liu & baumeister, 2016; liu & campbell, 2017). people can be differentiated based on the score on these five dimensions when using various icts (chittaranjan et al., 2011; kim et al., 2015). however, existing literature consists of some antithetical empirical evidence concerning the relationships between the ‘big-five’ traits and ict use (kalmus et al., 2011; ko et al., 2012; tan & yang, 2012). these contrasting findings may be attributed to a cluster of factors, including diversified sample, location, contexts of study, stage of technological advancements, cultural differences, and technology characteristics. therefore, a meta-analysis of such divergent findings would enrich literature with a comprehensive, holistic, and clearer view. the motivation for conducting this study is threefold: first, extant literature on the classification of icts did not account for recently introduced technologies, such as online learning applications, online dating applications, and virtual reality devices, to mention a few (kircaburun et al., 2020; mody, 1999; rad et al., 2018; rauschnabel et al., 2015). existing taxonomies have primarily concentrated on the technical features of icts, and overlooked the individual psychological motivations, while classifying icts into certain groups that are very wide and diverse (e.g., e-services, mobile technology, virtual world environment, etc.) (inaba & squicciarini, 2017; mody, 1999; rad et al., 2018). therefore, based on the unique functionalities and individual motivations to use various icts, this study attempts to provide a distinct, comprehensive, and sparing classification of icts by synthesizing earlier taxonomies, and incorporating recently introduced ict tools and services. second, a detailed review of literature unearthed contrasting shreds of evidence on the interaction between personality and ict use. these inconsistent findings could potentially be perplexing for the research community at large, thereby calling for scholarly attention. although some researchers have attempted to address these issues (huang, 2019; liu & campbell, 2017; marengo et al., 2020; marino et al., 2018), most of them have had some limitations in terms of technology selection, such as focusing only on ‘social networking’ based icts. it may be noted herein that the fast-growing pace of technological advancements and increased access to icts can cause changes to the nature and extent of the association between personality traits and ict use (tan & yang, 2012). therefore, this study offers a detailed 1 ‘adoption’ and ‘use’ are used interchangeably for the purpose of this study (ayuning budi et al., 2021; rajput, 2015; wymer & regan, 2005). 2 technology adoption/use refers to accessing icts with varied frequency or time duration. australasian journal of information systems joshi, das & sekar 2023, vol 27, research article how personality traits affect ict use 3 account of the personality traits – ict use relationship by classifying the icts into six classes, based on psychological motivation to use, vis a vis their functionalities, making this study novel. since this study provides a motivation-based, ict-specific synthesis of the existing literature, it may be used as a guideline for problem formulation and explaining results for future studies. third, several studies have highlighted the relevance of technology type, individual voluntariness, and cultural diversities across different countries on use of various icts (jayaprakash & pillai, 2021; rathore et al., 2021; tao et al., 2020; xiao et al., 2012). however, previous review studies have overlooked the role of these factors in the relationship between personality traits and ict use. in an attempt to fill this knowledge gap, this study offers an advanced and comprehensive view of personality’s role in ict use by considering technology type, region of the country, and voluntariness as potential moderators in the above-stated relationship. in the process, this study makes four significant contributions to extant literature. first, this study proposes an integrated, updated, and advanced classification of various icts by undertaking their core utilities and individual motivations. second, this paper addresses and attempts to reduce inconsistencies in extant literature on the relationship between personality traits and the use of various icts. third, this research underscores the significance of technology type, voluntariness, and country region in understanding personality-ict use association by identifying them as moderators. finally, this paper proposes a theoretical framework (see figure 2) and practical implications that could be useful for academicians and practitioners to unfold the riddle of individual technology adoption. this paper is further structured as follows: section 2 summarizes relevant literature, while highlighting the research gaps, and discussing the rationale of the present study. the applied methodology is discussed in section 3, followed by the results in section 4. section 5 consists of discussion, implications, and limitations and future research directions; and finally, in section 6 we conclude. 2 literature review this section comprises three sub-sections: section 2.1 briefly explains the big five model of personality; section 2.2 summarizes the findings of relevant previous research; section 2.3 summarizes previous meta-analytic studies, highlights the research gap, and underscores the rationale of the present study. 2.1 the big five personality model proposed by mccrae and costa (1987), the big five model is the most widely used and wellvalidated taxonomy of personality traits that consists of five dimensions: extroversion, agreeableness, openness, conscientiousness, and neuroticism (john et al., 2008). each trait is a part of a continuum, defined in the sub-sections of section 2.2 (section 2.2.1 to 2.2.5). for each dimension, people can fit anywhere along the spectrum. these big five dimensions explain a large amount of variance in personality, justifying thereby ‘big’. personality researchers have reached a collective consensus that the territory of personality can be delineated by five superordinate constructs (digman, 1990). this theoretical approach has been identified as the fivefactor model (ffm), whereby the dimensions are often referred to as the big five. researchers have regarded ffm as the most comprehensive, yet concise and useful taxonomy for australasian journal of information systems joshi, das & sekar 2023, vol 27, research article how personality traits affect ict use 4 examining personality (mccrae et al., 2008). hence, we adopted the big-five as a measure of personality. 2.2 extant literature on personality and icts use 2.2.1 extroversion and icts use extroversion is characterized by an individual's tendency to be sociable, excited, talkative, emotionally expressive, and assertive (mccrae & costa, 1987). previous studies have shown extroverted people expressing a positive view of icts, vis a vis their usefulness, because they tend to be excited and outgoing. in other words, extroverts are inclined to use icts that facilitate interpersonal communication. therefore, icts were found to be more beneficial and simpler to use by extroverts, while introverts were observed to be pickier while selecting icts (mouakket, 2015; nov & ye, 2008; sriyabhand & john, 2014; svendsen et al., 2013; wang, 2010). extroversion has thereby been recognized as a personality trait that significantly influences the use of networking, online gaming, blogging, online friends, and finance-related apps (guadagno et al., 2008; nikbin et al., 2021; singh, 2020; tan & yang, 2014; teng, 2008), and leisure services (butt & phillips, 2008; hamburger & ben-artzi, 2000). bianchi and phillips (2005) suggested that extroverts use icts for self-stimulatory purposes, which may explain why ‘extroversion’ was found to be positively linked with online gaming and leisure activities. moreover, extroverts place a high value on close and warm interpersonal relationships (watson & clark, 1997) and maintain the same through online communications (ross et al., 2009). in fact, the ‘extroversion’ score was found to be a significant predictor of online social networking activities and online social network strength (lönnqvist et al., 2014; rajput, 2015; vaid & harari, 2021). similarly, this trait was found to be positively related to the use of online communication, online academic activities, and online economic activities (mark & ganzach, 2014; vaid & harari, 2021; witt et al., 2011), which may be attributed to the talkative, social, and bold nature of extroverts. interestingly however, ‘extroversion’ was found to be negatively linked to the use of productivity-enhancing mobile applications and online dating applications (lane, 2012). in contrast to previous findings, burtăverde et al. (2021) for instance, found ‘extroversion’ to be negatively associated with the use of entertainment applications, gaming applications, social media-based applications, and smartphone application use in general, but positively associated with dating apps. a possible explanation for such findings could be that extroverts spend most of their time in social activities that do not involve computer use (landers & lounsbury, 2006). however, they try to make new relationships using online platforms. further, tan and yang (2012) argued that the time duration plays a vital role in creating and broadening the user base for mobile applications. thus, such contrary findings necessitate a systematic evaluation of extant literature to unravel the nature of relationship between personality traits and icts use. 2.2.2 agreeableness and icts use according to mccrae and costa (1987), “agreeableness refers to being compassionate, generous, cooperative, and empathetic in social situations rather than suspicious and antagonist.” many studies in the past have found ‘agreeableness’ to be relatively impertinent in the context of ict use. some studies incorporated this trait for the sake of totality, whereas others, omitted it based on the previous literary evidence of its irrelevance (huang, 2019; hughes et al., 2012; keller & karau, 2013; kim et al., 2015; lane & manner, 2011; mark & ganzach, 2014; mendonca, 2016; rajput, 2015; wang, 2010). notably, such irrelevance may be attributed to unexamined mediators or moderators (buckner et al., 2012). other studies found australasian journal of information systems joshi, das & sekar 2023, vol 27, research article how personality traits affect ict use 5 partial relevance of ‘agreeableness’ in predicting ict use (amichai-hamburger et al., 2002; benlian & hess, 2010). agreeable people tend to be conflict-averse and risk-avoiding. therefore, the use of technologies having potential disharmony inherent in unexpected outcomes, such as online dating applications and location-based social networking technologies, is a rare phenomenon among them (timmermans & de caluwé, 2017; zhang et al., 2017). further, ‘agreeableness’ was found to be a significant predictor of perceived ease of use (özbek et al., 2014), perceived usefulness of mobile-based applications (zhou & lu, 2011), online communication (tsao, 2013), instant messaging and phone calls (ehrenberg et al., 2008), and partial predictor of overall internet use (tsao, 2013). the tendency of agreeable people to be generous, cooperative, and empathetic can explain the underlying reasons for using conversation-based icts. in the same vein, the score on the ‘agreeableness’ scale was found to be linked with the use of news and business-related applications (burtăverde et al., 2021), erp applications (benlian & hess, 2010), and social networking sites (lönnqvist et al., 2014). such intricacies in literature do translate into a clouded picture of the relationship between personality traits and use of icts, and require further investigation. 2.2.3 openness to experiences and icts use openness describes an individual's preference for a wide range of interests and stimuli. according to mccrae and costa (1987), “openness is best characterized by characteristics, such as originality, imagination, broad interests, and daring.” open people are inclined to be receptive to new ideas and try new things. therefore, in previous studies, open people showed a positive relationship with acceptance and use of online shopping applications (mendonca, 2016), smartphones (kim et al., 2015), e-government portals (venkatesh et al., 2014), and online banking applications (ko et al., 2012). people who score higher on ‘openness’, tend to perceive digital modes of learning, such as online academic courses (svendsen et al., 2013), online learning (tsao, 2013), and online academic activities (mark & ganzach, 2014), as valuable and satisfactory. apart from educational and learning activities, ‘openness’ was also found to be a significant predictor of several online activities, including entertainment, communication, and social relationships, whereas a partial predictor of overall internet use (tsao, 2013). further, open individuals were found to be more aware of digital advancements (ross et al., 2009), such as single sign-on-use for google and/or facebook (pavlicek et al., 2018) and google glasses (rauschnabel et al., 2015), and thereby, tended to use them. the thirst for trying new and innovative technologies could possibly explain these findings further. as open people possess broad area interests, this trait was found to be positively linked to various online activities, such as the use of facebook and twitter (anolli et al., 2005; hughes et al., 2012; kalmus et al., 2011; nikbin et al., 2021), entertainment activities (kalmus et al., 2011; tsao, 2013), and online communication and leisure activities (mark & ganzach, 2014). however, some studies concluded that ‘openness’ has no role to play in technology usage (ehrenberg et al., 2008; landers & lounsbury, 2006; lane & manner, 2011; mark & ganzach, 2014; mendonca, 2016; nov & ye, 2008; ross et al., 2009; terzis et al., 2012; xu et al., 2016). additionally, it may be noted that some studies have disregarded ‘openness’ totally, because open individuals are receptive to a variety of icts, and this characteristic does not affect the use of icts (kircaburun et al., 2020; zhou & lu, 2011). in light of these contradictory results, additional research is required to understand the association between ‘openness’ and ict use. australasian journal of information systems joshi, das & sekar 2023, vol 27, research article how personality traits affect ict use 6 2.2.4 conscientiousness and icts use conscientious people are characterized as being well-organized, hardworking, ambitious, persistent, consistent, dutiful, self-disciplined, and self-controlled by nature (mccrae & costa, 1987). since such individuals perceive time as a limited resource, they employ icts to manage time, and optimize their efforts in their day-to-day life, such as online shopping (wu & ke, 2015), formal and informal online communication, and e-government portals (mark & ganzach, 2014). high scorers on ‘conscientiousness’ avoid spending time on the internet for activities they consider unproductive and aimless, such as playing music and videos, photography, personalization apps, online gaming, social media browsing (leisure), and photo sharing apps (landers & lounsbury, 2006). in addition, conscientious people prefer icts that facilitate their personal and professional growth, wherefore this personality trait was found to be positively linked with the use of productive and utility applications, such as facebook (academic), work-related mobile applications, and learning applications (burtăverde et al., 2021; landers & lounsbury, 2006; lane, 2012; mark & ganzach, 2014; terzis et al., 2012; venkatesh et al., 2014). as conscientious individuals tend to be self-disciplined, ambitious, and hardworking, they use digital means to enhance their skills and expand their knowledge base. hence, conscientious people did seem to report a higher level of satisfaction with wikimediated learning, online academic courses, and other online learning activities (altanopoulou & tselios, 2015; cohen & baruth, 2017; keller & karau, 2013). further, internetassisted activities that push one to work hard and facilitate self-assessment were perceived as being favorable along with the above-mentioned activities (terzis et al., 2012). this finding conforms to the self-controlled and dutiful tendencies of conscientious people. however, there are contrasting shreds of evidence in literature on pursuing internet-assisted leisure activities and instant messaging services (benlian & hess, 2010; butt & phillips, 2008; landers & lounsbury, 2006; mark & ganzach, 2014; swickert et al., 2002). interestingly, this trait was observed to be positively related to the use of location-based social networking technologies, online dating apps, and online gaming (chorley et al., 2015; lin & ong, 2010; nikbin et al., 2021; teng, 2008; timmermans & de caluwé, 2017). these findings could be attributed to the tendency of time-saving while fulfilling recreational and social needs. thus, extant research supports the notion that conscientious individuals use icts only to satisfy their need for achievement, optimize distribution of time and resources, and keep themselves wellmaintained. however, several studies believed that ‘conscientiousness’ is irrelevant in relation to use of icts (anolli et al., 2005; devaraj et al., 2008; özbek et al., 2014; rajput, 2015; terzis et al., 2012; tsao, 2013; zhang et al., 2017; zhou & lu, 2011). thus, a meta-analysis of previous studies would indeed provide a holistic view of extant literature to address such contrasting findings. 2.2.5 neuroticism and icts use neuroticism is one of the big five personality traits proposed by mccrae and costa (1987); it is characterized by moodiness, sadness, and emotional instability. people with a higher score on this dimension tend to experience mood swings, sadness, and irritability more frequently and intensely than others. in contrast, low scorers tend to be emotionally resilient, well adjusted, and relaxed (mccrae et al., 2008). as high-scoring individuals on ‘neuroticism’ are more receptive to negative aspects of a phenomenon, they perceive technology as a futile pursuit. literature on the interaction of ‘neuroticism’ and technology use has been multifaceted. an individual who falls near the upper extreme of ‘neuroticism’ is inclined to be over-conscious about personal actions and persuasions (mccrae et al., 2008) . therefore, these australasian journal of information systems joshi, das & sekar 2023, vol 27, research article how personality traits affect ict use 7 individuals express a high need for quality information, quality services, and structural assurance to reduce their worries, accepting technology thereby (zhou & lu, 2011). in previous studies, ‘neuroticism’ was found to be positively correlated with the use of selfcentered mobile application, such as photography apps and personalization apps, whereas negatively correlated with the use of social media apps, such as facebook and instagram, among others (hassan & pandey, 2021; nikbin et al., 2021; tan & yang, 2014; xu et al., 2016). further, a high scorer on this dimension tends to avoid interacting with people in physical settings. in turn, they use digital space to satisfy social needs, self-expression needs, and daily necessities (bosnjak et al., 2007; mark & ganzach, 2014; tan & yang, 2014). a recent study conducted to examine the impact of teachers' personality traits on using different icts showed that ‘neuroticism’ negatively affects the behavioral intention to use various icts (camadan et al., 2018). in a nutshell, a preference for icts that conform to their personality traits is evident in extant literature. 2.2.6 summary of previous research though ict use has become a global phenomenon, it is substantially far from attaining global uniformity owing to personal, technological, and contextual factors. extant literature observed that the relationships between individual characteristics and use of icts are likely to be different on the basis of technology type, voluntariness, age, gender, experience, use cases, and the region of a country (baptista & oliveira, 2016; nadeem et al., 2022; sait et al., 2004; schlachter et al., 2018; venkatesh et al., 2014; vu & lim, 2021). tao et al. (2020) argued that the type of technology does exhibit a significant moderating impact on the relationship between individual attributes and technology adoption (wang et al., 2018; zabkar et al., 2017). moreover, extant literature has been of the view that the use of personality – ict relationship varies for icts with different characteristics and motivations (mendonca, 2016). in other words, people with different personality traits vary in terms of psychological motivations, which in turn influence their selection and use of ict tool/s (mody, 1999; xu et al., 2016). the congruence between individual disposition and ict attributes do play a determining role in technology use (mendonca, 2016). previous studies have established that countries vary to a large extent in terms of the personalities of citizens that are reflected in their selection of both products and services (mooradian & swan, 2006). allik (2012) examined the big-five traits of people from 56 countries, and argued that countries vary significantly in terms of their personalities; these differences influence their decision-making. further, it may be noted that individuals’ ict use varies based on the requirement (voluntary or non-voluntary). for instance, people are more likely to use icts when it is part of their job or when there are organizational requirements. however, the pattern of ict use may differ when icts are used voluntarily (buchanan et al., 2005; uesugi et al., 2010). extant literature has been of the view that ict use becomes more significant in case of non-voluntary (work-related) use of icts than voluntary use, because of work pressure and assigned responsibilities (jarvenpaa & lang, 2006; schlachter et al., 2018). thus, this study proposes to verify whether voluntariness (voluntary/ non-voluntary) does moderate the relationship between personality-ict use. in line with prior studies, we examine the impact of three moderating factors (i.e., technology type, region of country, and voluntariness) on the link between personality and use of icts. since most of the studies selected did not report statistics related to other potential moderators, such as age, gender, and experience, we considered the above-mentioned variables as moderators. australasian journal of information systems joshi, das & sekar 2023, vol 27, research article how personality traits affect ict use 8 2.3 extant meta-analytic studies in extant literature, we noted that some researchers had performed meta-analyses of studies that attempted to explain the relationship between the big five personality traits and specific ict use, such as ‘social networking’ based icts and internet use (huang, 2019; liu & campbell, 2017; marengo et al., 2020; marino et al., 2018). although these studies reported personality traits to be significantly associated with various use of icts, most have some drawbacks and limitations (as summarized in table 1). author and year context/objective key findings limitations/ drawbacks liu and campbell, 2017 examining the relationships between social network site use and the big five traits (ocean) as well as the big two meta-traits (plasticity and stability). ‘extroversion’ and ‘openness’ are the strongest predictors of sns activities compared to other big five traits. plasticity positively correlates with sns activities, whereas stability is a negative predictor. considers various activities on social networking sites but overlooks other avenues of icts. marino et al., 2018 summarize the findings of the recent literature on the relationship between internet addiction (problematic facebook use) and individual characteristics. ‘neuroticism’ and ‘conscientiousness’ are the most clearly and strongly associated with problematic facebook use. the other three traits are negatively but only mildly associated with problematic facebook use. undertake only one social media platform (i.e., facebook). not inclusive in terms of technology selection. huang, c.,2019 estimating the overall strength of the links between big five traits and sns use. the correlations of social network site use with ‘neuroticism’ and ‘eextroversion’ are positively small, while ‘conscientiousness’ has a negative and quite small correlation with social network site use. ‘openness’ and ‘agreeableness’ are not significantly correlated with social network site use. country of study and participants' genders moderate the examined relationship. this study solely focuses on social networking sites and does not consider other icts. marengo et al., 2020 envisage and understand the links between personality traits and technology users' addictive behavior by investigating associations between personality and individual differences in addictive smartphone use. ‘neuroticism’ is positively associated with smartphone use disorder (smud), while ‘conscientiousness’ is negatively related. ‘agreeableness’ and ‘conscientiousness’ showed a heightened inverse association with smud among older samples. this study limits itself to the excessive use (addictive behavior) of smartphones. table 1. summary of previous meta-analytic studies after thoroughly investigating literature on both personality and technology adoption, we observed that it had many contrasting findings. for instance, chorley et al. (2015) observed that the ‘conscientiousness’ score is positively linked to the use of ‘business and commercebased icts; landers and lounsbury (2006) found this linkage to be negative; whereas ko et al. (2012) on the other hand stated that the ‘conscientiousness’ score does not influence the use of icts. these inconsistencies could be misleading, and a potential source of confusion for australasian journal of information systems joshi, das & sekar 2023, vol 27, research article how personality traits affect ict use 9 future studies. additionally, previous meta-analytic studies have not been inclusive in terms of technology selection; and the role of potential moderators has also not been much explored. therefore, this research addresses these issues by conducting a meta-analysis of earlier studies. we classified icts into six categories, based on their utilities and users’ motivation to use (please see table 4). they include ‘business and commerce’ (b&c), ‘career and education’ (c&e), ‘communication’ (comm), ‘entertainment’ (ent), ‘information’ (info), and ‘social networking’ (sn). through our study, we believe that we would equip the research community with a better understanding of various classes of icts vis a vis the relationship between personality and technology use by answering the following questions: rq.1: how do personality traits affect ict use in general and for specific purposes (i.e., ‘career and education’, ‘communication’, ‘entertainment’, ‘social networking’, ‘business and commerce’, and ‘information’)? rq.2: what potential variables moderate the relationship between personality traits and icts use? 3 methodology meta-analysis is a statistical procedure that integrates the results of several independent studies that are considered to be combinable (egger et al., 1997). glass (1976) defined metaanalysis as a simple "analysis of analyses." it is the most objective and effective method of reviewing an extant body of literature (king & he, 2005). there are several techniques available in literature for conducting meta-analysis, such as search, appraisal, synthesis, and analysis (salsa) (grant & booth, 2009); cochrane collaboration guidelines for systematic reviews (higgins et al., 2019); preferred reporting items for systematic review and metaanalysis (prisma-p) (moher et al., 2010), and psalsar (mengist et al., 2020). notably, the psalsar method is an updated form of salsa with two additional steps: research protocols (p) and reporting the results (r). among these techniques, we opted for psalsar for conducting this study, as it is the most updated, relevant, and detailed method available in extant literature. it consists of six systematically arranged phases to obtain optimum output from the process. the first step includes defining the study protocol based on the picoc framework (population, intervention, comparison, outcome, and context). this step defines the research scope (booth, 2016). the objectives and scope of this study are presented as research questions in the previous section. the second step, i.e. ‘search’, aims to locate the relevant work pieces. to find relevant literature, we used domain-specific key terms, vis a vis their combination that define the boundaries of the scope of research (nayal et al., 2020; papaioannou et al., 2010; zhang et al., 2022). we used the following search databases/ engines and search strings: ● search databases: scopus, abi/inform global, web of science, emerald, business source complete, sciencedirect, and google scholar. ● search string: ("big five personality traits" or "extroversion" or "agreeableness" or "openness" or "conscientiousness" or "neuroticism" or "emotional stability" or "personality" or "individual dispositions" or "individual characteristics") and ("smartphone" or "social media" or "facebook" or "instagram" or "twitter" or "tinder" or "online dating apps" or "mobile applications" or "online learning" or australasian journal of information systems joshi, das & sekar 2023, vol 27, research article how personality traits affect ict use 10 "microsoft teams" or "online teaching" or "online classes" or "online academic activities" or "google meet" or "microsoft teams" or "online leisure services" or "online games" or "multi-player games" or "online commerce" or "online shopping" or "online banking" or "e-banking" or "mobile commerce" or "ecommerce" or "e-tail" or "online selling" or "online banking" or "mobile banking" or "information and communication technologies" or "internet" or "computer" or "technology") figure 1 shows the results obtained by applying the search string in the selected databases; while table 2 refers to both the inclusion and exclusion criteria for selecting extant literature. criteria decision when selected keywords are available in the title, keywords, or abstract section of the paper inclusion the paper is written in the english language inclusion studies that report the correlation statistics or p-value or t value inclusion the paper includes the big five personality model or at least one dimension of the big five personality model inclusion when the article discusses any form of icts and relates it to the big five personality trait inclusion papers that are duplicated within the search results exclusion papers that are not primary/original research exclusion papers that do not consist of prerequisite statistics excluded papers that do not meet any of the inclusion criteria excluded table 2. inclusion and exclusion criterion for selection of literature the third step of psalsar includes an appraisal of selected studies that evaluate the search results to be included in the study, based on the research scope, while describing their validity. figure 1 presents the flow of selecting relevant papers. the number of articles used for further analysis consists of 9.6% of the original research works available in the databases. the fourth step, i.e., ‘synthesis’, comprises extracting and classifying data from the publications selected to derive knowledge and conclusions. while collecting statistics for the meta-analysis, we found three types of statistics in different papers, and dealt with the prescribed methods (table 3) (krzywinski & altman, 2013). s.n. type of statistic formulae/tools used 1. correlation statistics(r) considered for the analysis 2. t-value r = √t2 / (t2 + df) 3. p-value r software (to get t-value form p-value), r = √t2 / (t2 + df) note. t value, calculated difference represented in standard error units; df (degrees of freedom), sample size – 1. table 3. processing various types of statistics reported in the selected papers the forty-eight articles selected and examined, included diverse forms of icts, such as social networking sites and applications, online banking, mobile commerce, online games, egovernment portals, online learning platforms, among others. notably, these technologies are significantly different in terms of their functionalities and psychological motivation to use. rad et al. (2018) proposed a conceptual classification based on technicalities of various icts that classify them into twelve groups. though this taxonomy encompasses most of the icts, some recently introduced forms of icts are not included (e.g. google glasses, online dating services, blogging, etc.) (guadagno et al., 2008; kircaburun et al., 2020; rauschnabel et al., 2015). australasian journal of information systems joshi, das & sekar 2023, vol 27, research article how personality traits affect ict use 11 figure 1. the flow diagram of the database search additionally, previous research has attempted to classify the activities even in the digital sphere (mody, 1999). however, this classification also suffers from similar limitations. according to technology and innovation report (canton, 2021), the recent decades have witnessed rapid technological advancements in icts, along with their availability and accessibility. therefore, by synthesizing earlier classification and incorporating modern forms of icts (conole & dyke, 2004; inaba & squicciarini, 2017; mody, 1999; rad et al., 2018), this study offers a simple, comprehensive, and parsimonious taxonomy of icts. this classification effectively has been based on the functional utility and users’ psychological motivation (chacón et al., 2017; mody, 1999; rad, nilashi, dahlan, et al., 2018), compartmentalizing icts into six brackets: ‘career and education’, ‘communication’, ‘entertainment’, ‘social networking’, ‘business and commerce’, and ‘information’. the central idea behind this classification is that each ict class possesses a distinctive set of features and offers different values. people with different personalities are driven by different motivations, and seek out icts with values and incentives matching their psychological motivations. thus, the congruence of personality traits, psychological motivations, and ict attributes does determine the use of different icts. further, it may be noted that this taxonomy (table 4) enables researchers to comprehend the relationship between psychological motivations and use of icts, identify the inimitable values of various icts, and classify them accordingly. table 4 presents a detailed classification based on psychological motivation and ict utility. australasian journal of information systems joshi, das & sekar 2023, vol 27, research article how personality traits affect ict use 12 psychological motivation ict tools considered in research ict class nature of use self-enhancement: motivations centered on skill development, knowledge acquisition, self-development, and, in general, feeling better about oneself (chacón et al., 2017). computer use for education, educational apps, linkedin, evernote, wikipediabased learning, internet use (learning), literacy mobile apps, online academic course, productivity-enhancing apps, computer-based assessment, and enterprise resource planning. career and education (c&e): digital platforms that provide learning opportunities and professional development through online courses and online training programs which offers flexibility in terms of time, money, pacing, study location, and facilitate individual's/employees’ professional growth (rad, nilashi, dahlan, et al., 2018). v online learning-based technology; microsoft excel spreadsheet software; web-based classroom technology system; eproject: commercial collaborative system. nv connectedness: individual's motivation to contact other people and exchange information (wei & lo, 2016). smartphone use; basic features of mobile; calls; sms; communication apps; whatsapp (calls and messages); instant messaging apps. communication (comm): tools and apps specifically used to send, receive, and process information (mody, 1999). v recreation: activities people do in their spare time to relax, refresh, and invigorate to refresh by physical or virtual influence (mokhtarian et al., 2004). entertainment app; online games; google glasses; leisure apps; online music; gaming apps; multimedia apps; music and audio apps; online comics. entertainment (ent): all icts that enable delivery of entertainment experience to the users by creating and improving components (mody, 1999). v socialization: motivations based on humans' urge to engage and interact with other people in informal settings (chacón et al., 2017) online social interaction; twitter (social); facebook; online acquaintances; social networking apps; location-based social networking apps; instagram; whatsapp (social); social networking websites; online dating apps; online friends apps. social networking (sn):conversational web-based sites/apps that allow users to develop profiles of themselves, upload personal information and connect with multiple networks (rad et al., 2018). v convenience: a degree to which users feel that the use of a system of technology will make them free from difficulty (brown & mcenally, 1992). e-commerce; m-commerce; online transaction apps; finance apps; business apps; shopping apps; online investments; online banking. business and commerce (b&c): applications of information technology that are integrated into the business for smoothening the operations of business (mody, 1999; rad et al., 2018). v understanding: motivations oriented to acquiring and/or improving knowledge and experiences to enhance the grasp on a particular phenomenon (chacón et al., 2017). e-government portal; online blogs; information services; twitter (info); facebook (info); online news and information; work and informationbased apps; technical knowledge management system; travel apps; online browsers; e-health apps; utility apps; digital bulletin board system; sports apps; lifestyle apps. information (info): websites and apps that facilitate accessing a large amount of information related to various domains (mody, 1999). v online software. nv note. v, voluntary ‘voluntary use’ refers to wilful use of icts without incentive or force. ; nv, non-voluntary ‘non-voluntary use’ refers to work-related ict use as a part of job. table 4. classification of icts reported in the selected papers australasian journal of information systems joshi, das & sekar 2023, vol 27, research article how personality traits affect ict use 13 the fifth step included ‘analysis’; herein, we employed the comprehensive meta-analysis (cma) software (a licensed web-based application) for conducting the correlational metaanalysis and calculating the pooled correlation coefficients (table 5). cma allows the creation of moderating variables that can be utilized in the analysis by performing a sub-group analysis (borenstein et al., 2009). we performed sub-group analysis to identify potential moderators by applying technology type, region of the country, and voluntariness as moderating variables. the countries of the study were classified into four continents, namely north america, asia, australia, and europe, as all of the shortlisted studies were conducted in the countries belonging to these continents. these continents were considered, since there were at least three studies that were conducted for each continent. moreover, we classified the icts into two categories, based on voluntariness (i.e., voluntary and non-voluntary). the last step included ‘reporting’, comprising the description, presentation of the methods, and results derived from the relevant literature selected. fernández del amo et al. (2018) divided this phase into two sub-phases: 1. description of the primary procedure followed, and 2. public presentation of the results, such as journal articles. note. b&c, business and commerce; c&e, career and education; comm, communication; ent, entertainment; info, information; sn, social networking. figure 2. proposed theoretical framework 4 results figure 1 presents the flow diagram for study selection; at the end of selection and appraisal phase, we found 48 studies to be eligible for further analysis (appendix 1). in addition, we australasian journal of information systems joshi, das & sekar 2023, vol 27, research article how personality traits affect ict use 14 extracted multiple correlation trials from several other studies, as they examined the association of big five personality traits with the use of different forms of icts. thus, we considered a total of 379 correlation trials for our meta-analysis. figure 2 depicts the proposed theoretical framework. 4.1 study characteristics most of the studies were conducted in the past 12 years (90%). the sample size in individual studies ranged from 22 to 9482. selected studies considered at least one of the big five personality traits as an independent variable (iv), and the use of any form of ict as a dependent variable (dv). based on table 4, the icts included in our study belonged to the six categories. 4.2 meta-analysis initially, we examined the correlations between the big five traits (iv) and overall ict use (dv) (five correlations). next, we analyzed the relationship between the big five traits (iv) and six classes of icts (dv) (thirty correlations). thus, a total of thirty-five correlations were examined with a minimum of three trials. the random-effect model was chosen over the fixedeffect model, as selected studies consisted of samples from different populations. table 5 depicts the results of meta-analysis, tests of heterogeneity, and publication bias. heterogeneity is a measure of variation in study outcomes between studies, which is primarily employed in systematic reviews and meta-analytic studies (fletcher, 2007; nayal et al., 2020). the i² statistic measures the percentage of variation across studies, caused by heterogeneity rather than chance. this measure of heterogeneity is better than other instruments (such as q-statistics) as it does not depend on the number of studies (higgins & green, 2011). we observed a high level of heterogeneity (table 5) across most of the correlations, which indicated the presence of potential moderators. further, we tested publication bias using begg and mazumdar rank correlation and egger's regression test. out of 35 correlations, egger's regression test reported the existence of publication bias for only one correlation: agreeableness-communication (t= 2.91, p=0.01). thus, the results of the publication bias’ test indicate an absence of significant publication bias and the robustness of the search strategy (higgins et al., 2019; tao et al., 2020). the meta-analysis results showed that 85% of the pooled correlation coefficients were significant. all pooled correlation coefficients were positive except for n – c&e (-0.13), c – comm (-0.04), c – ent (-0.02), and c – sn (-0.04). all of the big five traits, besides ‘agreeableness’, were significantly related to using icts in general. thus, most of the correlations between big five traits and the use of icts in general, and different forms of icts were supported. the 95% confidence intervals for the pooled correlations of n-ict, oict, and c – ict were narrower than those of other correlations, signifying that these correlations were robust throughout the studies. these findings relate to the use of icts in general. in addition, the 95% confidence intervals for the pooled correlations of n info, c – sns, and c – c&e were narrower than those of other correlations between the big five traits and classified six categories of icts. this indicates the robustness and consistency of these correlations across the trials. among all correlations, o – c&e (0.18), c – c&e (0.18), e – b&c (0.17), and e – sn (0.16) were found to be the most significant and strong correlations. australasian journal of information systems joshi, das & sekar 2023, vol 27, research article how personality traits affect ict use 15 pairwise cor relations no. of trials total sample size effect size r (pvalue) confidence interval (95%) homogeneity q i2 publication bias lower upper test tstat (pvalue) e – ict 88 99332 0.11*** 0.09 0.14 544.69*** 83.82% rc 0.47 (0.64) a – ict 71 99331 0.02 -0.01 0.05 197.07*** 63.96% rc 1.69 (0.10) o – ict 70 77448 0.1*** 0.07 0.12 503.80*** 86.08% rc 0.65 (0.55) c – ict 74 44221 0.03* 0.01 0.06 549.06*** 86.52% rc 0.09 (0.93) n – ict 76 90895 0.03* 0.01 0.05 277.45*** 72.56% rc 0.41 (0.68) e – b&c 7 1,177 0.17*** 0.06 0.28 23.13*** 69.57% er 0.22 (0.83) a – b&c 5 1,181 0.03 -0.09 0.15 19.05** 73.68% er 1.49 (0.23) o – b&c 5 1,909 0.14*** 0.07 0.21 8.35 37.50% er 1.20 (0.31) c – b&c 5 9,937 0.03* 0.01 0.05 10.52* 50.00% er 1.54 (0.21) n – b&c 7 10,124 0.02* 0.01 0.04 30.63*** 76.67% er 0.67 (0.53) e – c&e 14 20,236 0.10*** 0.05 0.15 106.59*** 86.79% er 1.42 (0.15) a – c&e 19 15,012 0.13*** 0.07 0.18 118.39*** 83.90% er 1.05 (0.29) o – c&e 19 20,013 0.18*** 0.13 0.22 158.46*** 87.97% er 1.33 (0.18) c – c&e 20 15,634 0.18*** 0.12 0.24 189.99*** 89.42% er 1.71 (0.08) n – c&e 16 21,230 -0.13*** -0.25 0.01 944.51*** 98.31% er 1.18 (0.26) e – comm 17 20,073 0.13*** 0.08 0.17 168.95*** 89.88% er 1.24 (0.22) a – comm 14 13,024 0.12*** 0.04 0.2 163.67*** 91.41% er 2.91 (0.01) o – comm 6 19,869 0.11*** 0.01 0.21 174.06*** 96.55% er 0.03 (0.98) c – comm 10 11,259 -0.04 -0.1 0.02 122.01*** 91.80% er 1.25 (0.24) n – comm 12 10,974 0.06 -0.01 0.12 132.63*** 90.91% er 1.51 (0.15) e – ent 11 19,446 0.05* 0.01 0.1 31.45*** 64.52% er 1.43 (0.18) a – ent 5 1,685 0.07*** 0.03 0.11 16.25** 68.75% er 1.06 (0.34) o – ent 12 11,042 0.02* -0.01 0.04 30.11** 60.00% er 1.16 (0.24) c – ent 4 18,810 -0.02 -0.08 0.03 25.53*** 84.00% er 0.30 (0.78) n – ent 8 1,406 0.06* 0.02 0.08 31.47*** 74.19% er 0.08 (0.94) australasian journal of information systems joshi, das & sekar 2023, vol 27, research article how personality traits affect ict use 16 e – info 14 13,363 0.04** 0.02 0.06 103.78 86.41% er 0.63 (0.54) a – info 14 5,212 0.08*** 0.04 0.13 25.60* 44.00% er 0.21 (0.83) o – info 14 15,451 0.13*** 0.07 0.18 80.60*** 82.50% er 1.72 (0.11) c – info 15 21,762 0.05*** 0.03 0.06 100.27*** 85.00% er 0.29 (0.77) n – info 13 13,131 0.02* 0.01 0.03 37.18*** 64.86% er 0.73 (0.48) e – sn 23 19,703 0.16*** 0.11 0.21 226.29*** 89.82% er 1.62 (0.12) a – sn 16 18,167 0.04** 0.01 0.07 40.30*** 60.00% er 0.92 (0.37) o – sn 14 19,909 0.06*** 0.04 0.07 95.88*** 85.26% er 0.85 (0.41) c – sn 20 20,928 -0.04* -0.05 -0.03 126.90*** 84.13% er 1.10 (0.26) n – sn 20 19,560 0.03*** 0.02 0.05 101.27*** 80.20% er 0.40 (0.69) note. e, extroversion; a, agreeableness; o, openness; c, conscientiousness; n, neuroticism; c&e, career and education; comm, communication; ent, entertainment; info, information; b&c, business and commerce; sn, social networking, rc: begg and mazumdar rank correlation; er: egger's regression intercept. table 5. results of meta-analysis for pairwise correlations notably, the relationship strength personality traits – ict use varied across different forms of technologies. out of the five personality traits, conscientiousness (0.18) and openness (0.18) were found to be most significantly correlated with c&e, extroversion (0.13), and agreeableness (0.12) with comm, agreeableness (0.07) and extroversion (0.05) with ent, openness (0.13) and agreeableness (0.08) with info, extroversion (0.17) and openness (0.14) with b&c, and extroversion (0.16) and openness (0.06) with sn. 4.3 subgroup analysis as illustrated in table 5, high value of i2 for most of the correlations indicates a substantial level of heterogeneity, indicating the existence of potential moderating variables (nayal et al., 2020). we performed subgroup analysis to examine the moderating effects of technology type, region of the country, and voluntariness on the personality trait-ict use relationship (nayal et al., 2020; tao et al., 2020). the results of the subgroup analysis are summarized in table 6. technology type (as categorized in table 4) moderated all relationships except the relation of agreeableness – icts use. out of relationships between personality traits and use of icts, neuroticism – icts use (q= 79.51, p<0.01) was the most significantly impacted by the technology type as moderator, followed by conscientiousness – icts use (q=40.77, p<0.01), openness – icts use (q=24.85, p<0.01), and extroversion – icts use (q= 15.96, p<0.01). we grouped the studies in our data-set under four regions based on the continents of the countries, where the studies were actually conducted; they included north america, australia, europe, and asia. the region of the country, as a moderator, moderated the relationship of extroversion – icts use (q=27.84, p<0.001) most significantly, followed by conscientiousnessicts use (q=21.22, p<0.001), and agreeableness-icts use (q=10.58, p<0.05). however, the remaining two correlations were not moderated by region of the country. australasian journal of information systems joshi, das & sekar 2023, vol 27, research article how personality traits affect ict use 17 finally, we examined the impact of voluntariness on the personality – ict use relationship by classifying the icts into two classes, i.e., voluntary and non-voluntary. the results revealed that voluntariness (voluntary and non-voluntary) moderated all relationships (except for neuroticism – icts use) in a way that the big five – ict use relationships become more significant in case of non-voluntary icts. specifically, the relationship of openness – icts use was observed to be most significantly moderated by voluntariness (q=49.28, p<0.01), followed by conscientiousness – icts use ((q=20.88, p<0.01), extroversion – icts use (q=12.80, p<0.01), and agreeableness – icts use (q=9.10, p<0.01). correlations technology type (moderator) region of country (moderator) voluntariness (moderator) impact q (p-value) impact q (p-value) impact q (p-value) e-ict moderated 15.96(0.01) moderated 27.84(0.01) moderated 12.80(0.01) a-ict not moderated 9.81(0.08) moderated 10.58(0.01) moderated 9.10 (0.01) o-ict moderated 24.85(0.01) not moderated 5.70(0.13) moderated 49.28(0.01) c-ict moderated 40.77(0.01) moderated 21.22(0.01) moderated 20.88(0.01) n-ict moderated 79.51(0.01) not moderated 6.16(0.10) not moderated 1.61(0.20) note. e: extroversion; a: agreeableness; o: openness; c: conscientiousness; n: neuroticism. table 6. results of subgroup analysis 5 discussion we conducted a meta-analysis of forty-eight studies that quantitatively investigates the use of various forms of icts in view of the big five model of personality. this study highlights the relevance of big five traits in unfolding the riddle of individual ict adoption. the findings of our study explain how big five traits effectively influence the adoption and use of certain icts. further, to understand the intricacies of personality – ict use relationship, we explored the moderating role of technology type, region of the country, and voluntariness. 5.1 primary findings the results of the meta-analysis demonstrate that two personality traits, namely ‘extroversion’ and ‘openness’, were the most significantly and consistently related to the use of icts in general and across various classes. previous research has documented similar findings and attributed them to the intrinsic proclivities of individuals (burtăverde et al., 2021; stachl et al., 2017; xu et al., 2016). as extroverts derive their energy from the presence of other people, they continuously seek out new opportunities that offer social interactions. the paradigm shift from physical settings to the virtual world may push them more to use ‘social networking’ based icts than other individuals. similarly, open individuals possess a wide variety of interests and enjoy novel experiences, wherefore they are more likely to use any innovative or updated form of icts. the meta-analysis results found variability in the magnitude of effect sizes because of the characteristics of various personality traits and different forms of icts. each personality trait comprised a set of individual dispositions, while each type of ict differed from the others in terms of motives and means of using technology. correlating these australasian journal of information systems joshi, das & sekar 2023, vol 27, research article how personality traits affect ict use 18 two reflects their association, and their interpretation may be used for important practical purposes. overall, the results of the meta-analysis showed that all big five personality traits are significantly associated with ict use in general except ‘agreeableness’. however, the relative strength of the associations varied across the different personality traits, signaling thereby ‘variability’ across the selected relationships. our study showed that both ‘extroversion’ and ‘openness’ are most significantly associated with the use of icts. however, ‘extroversion’ does have a relatively stronger association with ict use than ‘openness’. it implies that because of the widespread availability and accessibility of icts, ‘openness’ has become less important than ‘extroversion’. however, this is not true for the newly introduced forms of icts. timmermans and de caluwé (2017) argued that ‘openness’ does have an essential and determining role in adopting novel technologies, such as online learning platforms, online skill development programs, online business and commerce applications, and online information portals. therefore, ‘openness’ exhibited significant and stronger correlations with b&c, c&e, info-based icts. these findings concur with rauschnabel et al. (2015) study, which stated that ‘openness’ plays a central role in the success of a newly introduced ict. we found ‘conscientiousness’ to be significantly associated with the use of icts in general. in particular, this personality trait demonstrated significant positive correlations with ‘career and education’, ‘information’, along with ‘business and commerce’ based icts, while it showed significant negative correlations with ‘social networking’ based icts. notably, since conscientious people tend to be self-disciplined, dutiful, and achievement-oriented; hence, in congruence with their psychological motivation for self-enhancement, they prefer utilizing icts to help them in skill development, staying informed, being well-organized, and avoiding the use of those icts, which they perceive as being unproductive (eşkisu et al., 2017; hughes et al., 2012; tan & yang, 2014). we found ‘agreeableness’ to be significantly and positively associated with the use of various icts, except for ‘business and commerce’. next, ‘career and education’ based icts on the other hand, seemed to assist internet-based skill development courses, lecture delivery, online classes, student evaluation, and online training, among others. as agreeable people tend to be cooperative, sensitive, and conflict-averse, they readily accept suggestions, and follow instructions, which can explain this positive correlation (a-c&e). further, high scorers on ‘agreeableness’ are disposed to be social and easy-going. such people desire to get along with others, form friends easily, and seek out a ‘virtual community’. importantly, these proclivities can act as a motivation for agreeable people to use ‘communication’ and ‘social networking’ based icts (bowden-green et al., 2021). therefore, in line with (xu et al., 2016) and in contrast to (tsao, 2013), we observed significant positive correlation between ‘agreeableness’ with ‘communication’ and ‘social networking’ based icts. furthermore, it may be noted that extant literature has been of the view that agreeable people are driven by potential benefits of ‘information’ based icts, and thereby use the virtual worlds for recreational purposes (bowden-green et al., 2021). in line with lin et al. (2017), our study reported that high scorers on ‘agreeableness’ are likely to use ‘entertainment’ and ‘information’ based icts. further, ‘neuroticism’ showed significant correlations with the use of all categories of icts, except for ‘communication’. neurotic individuals prefer solace and avoid interactions with people; wherefore, the relationship of n – comm was found to be insignificant. notably, ‘neuroticism’ had the most negative strong and significant association with ‘career and australasian journal of information systems joshi, das & sekar 2023, vol 27, research article how personality traits affect ict use 19 education’ based icts. researchers have also noted that people's self-improvement efforts are significantly influenced by their emotional stability (altanopoulou & tselios, 2015; mccrae & costa, 1987). consequently, individuals with a low score on ‘neuroticism’ are motivated to gain new skills, and increase their knowledge using digital means, justifying in the process, the nature of n – c&e relationship. next, we observed that ‘neuroticism’ was positively linked with ‘social networking’ based icts. according to previous research, neurotic individuals experience social pressure and anxiety while interacting with others in physical environments (kircaburun et al., 2020). such people express themselves better in the digital realm, as it provides enhanced control over the flow of conversation and eliminates the anxiety caused by social consciousness. therefore, to fulfill social needs, neurotic individuals seek some alternate channel for self-expression and socialization, which effectively explains their fondness for ‘social networking’ based icts. in similar lines, it may be noted that neurotic individuals prefer spending most of their time alone, and avoid meeting people. hence, to satiate their basic and recreational needs, they use ‘business and commerce’, ‘information’ and ‘entertainment’ based icts such as online shopping, online banking, online blogs, online information websites, online news applications, online games, online music, and video applications. these findings justify the consistency of an individual's personality and behavior in the physical settings and digital space. further, our findings reveal three moderators, namely, technology type, region of the country, and voluntariness. technology type significantly moderated four out of five pairwise correlations, indicating that distinct personality types are associated with variations in technology use. the variability in the use of icts based on the type of technology has been consistently reported in extant literature (bowden-green et al., 2021; king & he, 2005; lai et al., 2022). these results explain the variations in the correlation coefficient between personality traits and the use of icts. moreover, in the results of preliminary analysis, we observed that the effect size of the correlation c – c&e was larger than other correlations in c&e class, whereas the effect size of the correlation between e – sn was larger than other correlations in the sn class. similarly, the effect size of the correlation o – info was observed as being relatively larger and more significant. these findings are to be read with the classification offered by this study (table 4) to understand why these effect sizes actually vary. for instance, conscientious individuals are motivated for achievements and self-development, and likely to use those icts, which in turn, facilitate being organized, achieving goals, skill development, and staying updated. therefore, ‘conscientiousness’ did show a larger and positive effect size for ‘career and education’, along with ‘information’ based icts. on the other hand, extroverts tend to be talkative, social, and outgoing. such individuals try to find various ways to express themselves and talk to others. therefore, ‘extroversion’ exhibited a larger and positive correlation with social networking sites, and ‘communication’ based icts. similarly, open individuals are disposed to be receptive to any novel or updated form of icts. therefore, they would readily accept newly introduced technology, such as online payment applications, and virtual reality devices, among others. hence, ‘openness’ showed a more significant effect size for ‘information’, ‘business and commerce’, and ‘career and education’ based icts. this suggests that type of technology would accentuate the relationship between personality traits and ict use, which in turn, would expand knowledge on condition (i.e., type of technology), especially in enabling the use of icts. thus, in light of the present study, it would be judicious to infer that people with different personalities are driven by different australasian journal of information systems joshi, das & sekar 2023, vol 27, research article how personality traits affect ict use 20 motivations. therefore, ict selection and use decisions are determined by the congruence between personality traits and the utility of ict. next, the region of the country emerged as another moderator. although this moderator was less significant than the technology type, it moderated three out of five pairwise correlations. this can be attributed to cultural differences in various regions of the countries across the world. hofstede's national cultural dimension indices depict the dispositions of their citizens at the national level. these scores could help in explaining the moderation effects of continents of the country. for example, the continents, including countries that score higher on individualism (such as north america and australia), comprise a larger number of citizens using individual-based icts, such as photo editing applications, online shopping websites and applications, online skills up-gradation courses, personalization applications, and blogging. similarly, european and asian countries (such as greece, romania, turkey, and south korea) that score higher on the dimension of uncertainty avoidance consist of a substantial number of citizens who avoid using ambiguous forms of icts, such as online dating applications and location-based social network applications (hofstede & fink, 2007; matusitz & musambira, 2013). researchers have also studied the pattern of big-five traits distribution across national cultures and observed significant differences across the globe (allik, 2012; schmitt et al., 2007). as a corollary, asians are more interested in ‘career and education’ based icts, because of their conscientious nature; while europeans are more likely to use ‘information’ and ‘social networking’ based icts due to their open personality (please see appendix 2). further, the disparities in ict use across different countries could be attributed to the digital divide caused by state of national economic development (vu & lim, 2021). previous literature for instance, has observed a significant impact of gdp, education, internet price, and regulatory frameworks and digitalization index (billon et al., 2010). we also observed significant differences in terms of ict use across different continents. thus, our results go on to validate the impact of global differences on ict use and support extant literature thereof. finally, voluntariness moderated four out of five pairwise correlations (all except n – ict). we noticed that the relationships between personality traits and ict use are uniformly stronger for non-voluntary icts, indicating that individuals with different kinds of personality use work-related or non-voluntary icts in a similar pattern (please see appendix 2). following the ‘empowerment/enslavement paradox’, this finding concurs with jarvenpaa and lang (2006). it may be noted herein that in a large-scale study examining the experience of mobile technology within a multinational context, jarvenpaa and lang (2006) reported that work-related demands actually reduce the influence of other individual and organizational factors in the case of non-voluntary ict use. in the same vein, the researchers also noted work pressure or job commitments as a vital factor in case of non-voluntary ict use (schlachter et al., 2018). thus, it could be inferred that organizational factors (such as job requirements) matter more in the context of non-voluntary ict use. 5.2 implications 5.2.1 theoretical implications the theoretical implications of this study are fivefold: first, this study proposes an updated and integrated classification of icts (mody, 1999; rad et al., 2018). this categorization is predicated primarily on the psychological motivations and utilities of icts, grouped into six distinct clusters. each cluster possesses certain exclusive features, offers unique value, and is related to distinct psychological motivations. for instance, ‘social networking’ based icts australasian journal of information systems joshi, das & sekar 2023, vol 27, research article how personality traits affect ict use 21 promote socialization (or social needs) by creating a feeling of proximity in digital space. on the other hand, ‘career and education’ based icts facilitate skill development and selfenhancement, which in turn, foster the need for achievement or intellect. thus, the proposed taxonomy does function as a classification tool to categorize icts, based on prescribed criteria (table 4), and thereby facilitates developing an enhanced understanding of the uniqueness of various forms of icts. second, this study enriches literature with knowledge concerning the role of individuals' personality traits in using various icts by statistically synthesizing previous studies' results and establishing relationships between personality traits and six categories of icts. since this study explains why different people have varying preferences for various icts, the findings may be employed to drive future research on the determinants of adoption and use of icts with specific values and attributes. though extant literature consists of a number of theoretical models that explain variance in icts' use (such as tra, tam, utaut, utaut 2), personality has not been considered in any of these models (dwivedi et al., 2019). also, previous review studies highlighted the need of incorporating the ‘individual differences’-based variables in technology adoption models (dwivedi et al., 2019). thus, the larger findings of this study imply that incorporating personality into extant technology adoption models would significantly enhance their explanatory power. third, this study addresses and minimizes inconsistencies relating to the relationship between personality traits and use of various forms of icts, and thus, provides a clearer picture of how personality traits are actually associated with the use of various forms of icts (or icts in general). previous studies reported varied nature and strength of similar personality traits – ict use relationships in terms of their statistical significance, i.e., positively significant, negatively significant, and non-significant. in other words, extant literature seemed to lack the comprehensive view of the relevance of personality traits in ict use. the present metaanalysis of earlier studies, do signal a significant positive correlation (except a – b&c, n – c&e, c – comm, n – comm, and c – sn). notably, higher heterogeneity scores indicate that these differences in extant literature may be attributable to moderators. we considered technology type, region of country, and voluntariness as moderators, and offer explanations to reduce such inconsistencies that existed in earlier studies. further, we group similar icts into six distinctive clusters, analyze the previously reported relationships, and offer a comprehensive view of how personality traits are actually linked to different ict use. furthermore, in line with gollwitzer and bargh (1996) study, we explain the nature and strength of the big five traits – ict use relationship by linking individual dispositions with six groups of icts, along with their respective psychological motivations. the results could assist the research community in developing a comprehensive grasp of the association between personality traits and usage of different kinds of icts, and serve as a strong foundation for future studies. fourth, this research unearthed three potential moderators: technology type, region of country, and voluntariness, that influence the relationship between personality traits and ict use. these findings explore and report the relevance of ict characteristics, national culture diversities, and individual’s desire with respect to the personality – ict use relationship. thus, present research underlines the significance of contextual variables while interpreting the results of this study and recommend considering these variables while examining the ict australasian journal of information systems joshi, das & sekar 2023, vol 27, research article how personality traits affect ict use 22 adoption. these findings and explanations could facilitate problem formulation and explaining the results of future studies. finally, this study noted that some personality traits are more relevant when examining adoption/use of specific ict that have distinct functionalities and offer certain benefits. for instance, ‘extroversion’, ‘agreeableness’, and ‘openness’ play a determining role in the context of ‘communication’ related icts. similarly, ‘extroversion’ and ‘openness’ are of higher significance when studying ‘business and commerce’ related icts. in accordance with the prior research, this study verifies the irrelevance of ‘agreeableness’ in case of ict in general. thus, this research offers recommendations with respect to technology-specific trait selection in order to make research model comprehensive and parsimonious while studying adoption/use of specific ict. 5.2.2 managerial implications the results of this study offer actionable insights to ict designers, marketers, service providers, business strategists, human resource managers, and policymakers on possible measures to enhance the business plan and optimize resource allocation. first, the significant role of an individual's dispositional factors suggests that it may be insufficient for ict providers to merely craft and traditionally deliver the icts. the characteristics of icts and delivery approach should align with the targeted consumer group's characteristics and requirements. a personalized user interface that adopts its functioning style to align with the user's personality could be perceived as being more useful and more likely to be accepted by customers (bosnjak et al., 2007; ruijten, 2021). for example, conscientious people avoid using leisure mobile apps, such as music and photography, along with personalization apps that negatively affect their productive activities. on the other hand, neurotic individuals tend to adopt such apps, owing to their fussy and picky nature, and their interest in creative activities (xu et al., 2016). thus, personalized suggestions may result in attaining the right traittechnology fit and offering the services accordingly. such user-friendly interfaces and navigations can be attained by aligning the interface characteristics with human principles and continuous evaluation process while crafting icts (or & tao, 2012). second, the nature and strength of relationships between personality traits and icts use suggest that technology providers require additional efforts to identify the cluster of targeted consumers with certain proclivities and preferences. various personality tests could be employed for this purpose by administering a quick survey to determine the test-takers' personality. for example, the ten items personality inventory (tipi) could be administered along with collecting demographic variables by providing a small reward for filling up the form. the scores obtained through personality tests would facilitate consumers and technology providers in making decisions regarding technology selection and crafting a marketing strategy, respectively (donvito et al., 2020; rojas-méndez et al., 2013; ruijten, 2021; wang et al., 2018; zabkar et al., 2017). for instance, if a technology provider plans to introduce a variant of ict that would help people be organized, additional efforts are required to locate those who are more conscientious, apart from merely crafting and delivering icts. similarly, if a company intends to introduce a new social media platform, it should identify and target the cluster of outgoing and sociable people. information about the number and types of preinstalled applications and browsing history of users (with the consent of device users) could facilitate identifying dominant personality traits. next, the negatively significant association of n-c&e implies that neurotic individuals are more receptive to the negative aspects of australasian journal of information systems joshi, das & sekar 2023, vol 27, research article how personality traits affect ict use 23 ‘career and education’ based icts. hence, when presenting self-enhancement-based icts, it is crucial that all pertinent information, prospective advantages, and guarantee of promised services be provided in an organized manner. further, icts should be created with a strategy that promotes and communicate an interactive user interface and user-friendly navigation to minimize the doubts and concerns of customers. for instance, if a bank plans to offer a mobile banking app, then it should circulate a detailed document or float a video comprising all required information related to its know-how, key functions, safety and security features, certifications, and benefits of use in a structured way through the authentic channel. doing so would aid in obtaining and maintaining neurotic clients. these practices may also facilitate technology providers in gaining consumers' trust and spreading positive word-of-mouth that plays a significant role in acquiring agreeable consumers. third, the variation in the pairwise correlations implies that individual dispositions should be aligned with the assigned tasks to optimize employee performance (palmer et al., 2019; van der schyff et al., 2020). for instance, in the case of an information technology company, open employees could be assigned to test newly developed icts and work using recently introduced icts, such as data analytics, machine learning, big data, and cloud technology, whereas low scorers on the ‘openness’ could be assigned to work on conventional technologies such as traditional database management using ms sql, microsoft word, and microsoft excel. fourth, the moderating effect of technology type recommends that practitioners should figure out the customized design and implementation strategy for different types of icts. for instance, people interested in ‘career and education’ based icts may be less interested in using ‘social networking’ based icts. therefore, a single design and implementation strategy may not work well for all forms of icts; thus, business strategies should be contextualized accordingly. practitioners, who target the international market to introduce any form of ict, should consider the aspect of variability in technology use across different cultures. we found the differences in users across different cultures, which effectively indicate a need to recognize culture-specific factors in determining the use of icts, and thereby ensure successful technology delivery and implementation in local markets. finally, the moderating effect of voluntariness indicates the variances in personal (voluntary) and work-related (nonvoluntary) technology use patterns. voluntary use of ict is subject to individual needs, wants, and desires; therefore, ict providers should focus more on psychological and functional incentives (as discussed in table 4), enhancing user interface, and personalized marketing. on the other hand, non-voluntary icts are acquired and implemented by organizations; thus, technology providers should prioritize matching client's requirements, offering competitive business advantages, and ensuring dependable assistance. hence, ict providers are recommended to modify their business strategies with respect to the use cases. 5.3 limitations and future research directions this study has some limitations that should be noted and addressed by future research. first, we only included variables with at least three trials for the purpose of meta-analysis. though this is standard practice for conducting a meta-analysis, this may have prevented us from understanding the complete scenario of interaction between human personality and ict use. future studies are encouraged to examine this research domain using qualitative methods, such as morphological analysis and narrative literature review. second, the results of tests for publication bias indicate the absence of significant publication bias; however, a high value of australasian journal of information systems joshi, das & sekar 2023, vol 27, research article how personality traits affect ict use 24 heterogeneity does indicate the presence of unexamined factors that influence the pairwise correlations (nayal et al., 2020). although we examined the moderating effect of select variables, data availability limitations in the selected papers and study design prevented us from assessing other moderators' roles. future studies are recommended to provide more data on the age group, education level of the participants, and additional relevant information and conduct their investigations in light of potential moderators. third, our study considered only correlational statistics and not path coefficients. the number of studies containing path coefficients was less than the number of studies containing correlation statistics. this could be addressed by conducting a meta-analysis of both correlation coefficients and path coefficients when sufficient data on such relationships is available in literature (zhang et al., 2022). moreover, assessment on use of technology in online/offline could have added value but use of technology in online/offline mode were not adequately reported in the studies shortlisted. future studies could add this as a potential moderator. fourth, some of the meta-analyzed correlations were found to be statistically insignificant. this finding could be attributed to the possibility of the presence of other variables that are not considered in our study. therefore, apart from individual personality, future studies are recommended to examine other individual and organizational variables with respect to the use of various icts. finally, we observed that the studies included in the meta-analysis, employed a variety of instruments to measure the big five personality traits. though previous studies have examined the measurement tool for validity and reliability (cohen & baruth, 2017; rauschnabel et al., 2015; tsao, 2013), we recommend future studies employ detailed taxonomy to capture an accurate picture of personality. 6 conclusion this study quantitatively synthesizes previous studies that examine the relationship between the big five personality traits and icts use. additionally, we identify three significant moderators, namely technology type, region of the country, and voluntariness, which enhances the understanding of personality's role in the use of varied technology contexts. based on our results, we proposed an updated and inclusive classification of available technologies based on their unique functionalities and distinct psychological motivations. successful identification of personality traits that affect use of icts would effectively benefit practitioners and researchers in crafting an appropriate business plan and developing an integrated perspective of factors affecting icts use, respectively. future efforts may be dedicated to including additional relevant moderators, such as educational background, age, gender, and technology frames to explain further the relationship among personality traits and the use of various icts. references alhassan, a., li, l., reddy, k., & duppati, g. (2020). consumer acceptance and continuance of mobile money. australasian journal of information systems, 24. https://doi.org/10.3127/ajis.v24i0.2579 allik, j. (2012). national differences in personality. personality and individual differences, 53(2), 114–117. https://doi.org/10.1016/j.paid.2011.05.011 altanopoulou, p., & tselios, n. (2015). how does personality affect wiki-mediated learning? 2015 international conference on interactive mobile communication technologies and learning (imcl), 16–18. https://doi.org/10.1109/imctl.2015.7359546 https://doi.org/10.3127/ajis.v24i0.2579 https://doi.org/10.1016/j.paid.2011.05.011 https://doi.org/10.1109/imctl.2015.7359546 australasian journal of information systems joshi, das & sekar 2023, vol 27, research article how personality traits affect ict use 25 amichai-hamburger, y., wainapel, g., & fox, s. (2002). “on the internet no one knows i’m an introvert”: extroversion, neuroticism, and internet interaction. cyberpsychology & behavior, 5(2), 125–128. https://doi.org/10.1089/109493102753770507 amutha, d. (2020). the role and impact of ict in improving the quality of education. available at ssrn 3585228. http://dx.doi.org/10.2139/ssrn.3585228 anolli, l., villani, d., & riva, g. (2005). personality of people using chat: an on-line research. cyberpsychology & behavior, 8(1), 89–95. https://doi.org/10.1089/cpb.2005.8.89 baptista, g., & oliveira, t. (2016). a weight and a meta-analysis on mobile banking acceptance research. computers in human behavior, 63, 480–489. https://doi.org/10.1016/j.chb.2016.05.074 benlian, a., & hess, t. (2010). does personality matter in the evaluation of erp systems? findings from a conjoint study. ecis 2010 proceedings. 109. https://aisel.aisnet.org/ecis2010/109 bianchi, a., & phillips, j. g. (2005). psychological predictors of problem mobile phone use. cyberpsychology & behavior, 8(1), 39–51. https://doi.org/10.1089/cpb.2005.8.39 booth, a. (2016). searching for qualitative research for inclusion in systematic reviews: a structured methodological review. systematic reviews, 5(1), 74. https://doi.org/10.1186/s13643-016-0249-x bosnjak, m., galesic, m., & tuten, t. (2007). personality determinants of online shopping: explaining online purchase intentions using a hierarchical approach. journal of business research, 60(6), 597–605. https://doi.org/10.1016/j.jbusres.2006.06.008 bowden-green, t., hinds, j., & joinson, a. (2021). personality and motives for social media use when physically distanced: a uses and gratifications approach. frontiers in psychology, 12, 806. https://doi.org/10.3389/fpsyg.2021.607948/xml/nlm brown, l. g., & mcenally, m. r. (1992). convenience: definition, structure, and application. journal of marketing management (10711988), 2(2). brown, l. h., & akroyd, d. (2006). using personality type to predict student success in a technology -rich classroom environment [doctoral dissertation, north carolina state university]. in proquest dissertations and theses. https://www.proquest.com/dissertations-theses/usingpersonality-type-predict-student-success/docview/305293018/se-2?accountid=27562 buchanan, t., johnson, j. a., & goldberg, l. r. (2005). implementing a five-factor personality inventory for use on the internet. european journal of psychological assessment, 21(2), 115– 127. https://doi.org/10.1027/1015-5759.21.2.115 buckner, j. e., castille, c. m., & sheets, t. l. (2012). the five factor model of personality and employees’ excessive use of technology. computers in human behavior, 28(5), 1947–1953. https://doi.org/10.1016/j.chb.2012.05.014 burtăverde, v., vlăsceanu, s., & avram, e. (2021). exploring the relationship between personality structure and smartphone usage. current psychology, 40(11), 5613–5625. https://doi.org/10.1007/s12144-019-00521-5 butt, s., & phillips, j. g. (2008). personality and self-reported mobile phone use. computers in human behavior, 24(2), 346–360. https://doi.org/10.1016/j.chb.2007.01.019 https://doi.org/10.1089/109493102753770507 https://dx.doi.org/10.2139/ssrn.3585228 https://doi.org/10.1089/cpb.2005.8.89 https://doi.org/10.1016/j.chb.2016.05.074 https://aisel.aisnet.org/ecis2010/109 https://doi.org/10.1089/cpb.2005.8.39 https://doi.org/10.1186/s13643-016-0249-x https://doi.org/10.1016/j.jbusres.2006.06.008 https://doi.org/10.3389/fpsyg.2021.607948/xml/nlm https://www.proquest.com/dissertations-theses/using-personality-type-predict-student-success/docview/305293018/se-2?accountid=27562 https://www.proquest.com/dissertations-theses/using-personality-type-predict-student-success/docview/305293018/se-2?accountid=27562 https://doi.org/10.1027/1015-5759.21.2.115 https://doi.org/10.1016/j.chb.2012.05.014 https://doi.org/10.1007/s12144-019-00521-5 https://doi.org/10.1016/j.chb.2007.01.019 australasian journal of information systems joshi, das & sekar 2023, vol 27, research article how personality traits affect ict use 26 camadan, f., reisoglu, i., ursavas, ö. f., & mcilroy, d. (2018). how teachers’ personality affect on their behavioral intention to use tablet pc. the international journal of information and learning technology, 35(1), 12–28. https://doi.org/10.1108/ijilt-06-2017-0055 canton, h. (2021). united nations conference on trade and development—unctad. in the europa directory of international organizations 2021, 172-176. routledge. chacón, f., gutiérrez, g., sauto, v., vecina, m. l., & pérez, a. (2017). volunteer functions inventory: a systematic review. psicothema, 29(3), 306–316. https://doi.org/10.7334/psicothema2016.371 chittaranjan, g., jan, b., & gatica-perez, d. (2011). who’s who with big-five: analyzing and classifying personality traits with smartphones. proceedings international symposium on wearable computers, iswc, 29–36. https://doi.org/10.1109/iswc.2011.29 chorley, m. j., whitaker, r. m., & allen, s. m. (2015a). personality and location-based social networks. computers in human behavior, 46, 45–56. https://doi.org/10.1016/j.chb.2014.12.038 cohen, a., & baruth, o. (2017). personality, learning, and satisfaction in fully online academic courses. computers in human behavior, 72, 1–12. https://doi.org/10.1016/j.chb.2017.02.030 conole, g., & dyke, m. (2004). what are the affordances of information and communication technologies? research in learning technology, 12(2). https://doi.org/10.3402/rlt.v12i2.11246 corcoran, n., & duane, a. (2018). using social media to enable staff knowledge sharing in higher education institutions. australasian journal of information systems, 22. https://doi.org/10.3127/ajis.v22i0.1647 da silva oliveira, a. b., & chimenti, p. (2021). "humanized robots": a proposition of categories to understand virtual influencers. australasian journal of information systems, 25. https://doi.org/10.3127/ajis.v25i0.3223 davis, j., lee, l., & yi, m. (2007). examining the relationships among personality traits, itspecific traits, and perceived ease of use. amcis 2007 proceedings, 295. devaraj, s., easley, r. f., & crant, j. m. (2008). research note—how does personality matter? relating the five-factor model to technology acceptance and use. information systems research, 19(1), 93–105. https://doi.org/10.1287/isre.1070.0153 digman, j. m. (1990). personality structure: emergence of the five-factor model. annual review of psychology, 41(1), 417–440. donvito, r., aiello, g., grazzini, l., godey, b., pederzoli, d., wiedmann, k.-p., halliburton, c., chan, p., tsuchiya, j., skorobogatykh, i. i., oh, h., singh, r., ewing, m., lee, y., fei, l., chen, c. r., & siu, n. y.-m. (2020). does personality congruence explain luxury brand attachment? the results of an international research study. journal of business research, 120, 462–472. https://doi.org/10.1016/j.jbusres.2020.06.047 dwivedi, y. k., rana, n. p., jeyaraj, a., clement, m., & williams, m. d. (2019). re-examining the unified theory of acceptance and use of technology (utaut): towards a revised theoretical model. information systems frontiers, 21(3), 719–734. https://doi.org/10.1007/s10796-017-9774-y https://doi.org/10.1108/ijilt-06-2017-0055 https://doi.org/10.7334/psicothema2016.371 https://doi.org/10.1109/iswc.2011.29 https://doi.org/10.1016/j.chb.2014.12.038 https://doi.org/10.1016/j.chb.2017.02.030 https://doi.org/10.3402/rlt.v12i2.11246 https://doi.org/10.3127/ajis.v22i0.1647 https://doi.org/10.3127/ajis.v25i0.3223 https://doi.org/10.1287/isre.1070.0153 https://doi.org/10.1016/j.jbusres.2020.06.047 https://doi.org/10.1007/s10796-017-9774-y australasian journal of information systems joshi, das & sekar 2023, vol 27, research article how personality traits affect ict use 27 egger, m., smith, g. d., & phillips, a. n. (1997). meta-analysis: principles and procedures. bmj, 315(7121), 1533. https://doi.org/10.1136/bmj.315.7121.1533 ehrenberg, a., juckes, s., white, k. m., & walsh, s. p. (2008). personality and self-esteem as predictors of young people’s technology use. cyberpsychology & behavior, 11(6), 739–741. https://doi.org/10.1089/cpb.2008.0030 eşkisu, m., hoşoğlu, r., & rasmussen, k. (2017). an investigation of the relationship between facebook usage, big five, self-esteem and narcissism. computers in human behavior, 69, 294–301. https://doi.org/10.1016/j.chb.2016.12.036 fernandes, p., pereira, r., & wiedenhöft, g. (2021). information technology governance and the individuals' behavior: a cross-sectional study. australasian journal of information systems, 25. https://doi.org/10.3127/ajis.v25i0.3141 fernández del amo, i., erkoyuncu, j. a., roy, r., palmarini, r., & onoufriou, d. (2018). a systematic review of augmented reality content-related techniques for knowledge transfer in maintenance applications. computers in industry, 103, 47–71. https://doi.org/10.1016/j.compind.2018.08.007 fletcher, j. (2007). what is heterogeneity and is it important? bmj, 334(7584), 94–96. https://doi.org/10.1136/bmj.39057.406644.68 glass, g. v. (1976). primary, secondary, and meta-analysis of research. educational researcher, 5(10), 3–8. https://doi.org/10.3102/0013189x005010003 gollwitzer, p. m., & bargh, j. a. (eds.). (1996). the psychology of action: linking cognition and motivation to behavior. nyc, usa: guilford press. goyal, s., pillai, a., & chauhan, s. (2021). e-governance using mobile applications: a case study of india during the covid-19 pandemic. australasian journal of information systems, 25. https://doi.org/10.3127/ajis.v25i0.3129 grant, m. j., & booth, a. (2009). a typology of reviews: an analysis of 14 review types and associated methodologies. health information & libraries journal, 26(2), 91–108. https://doi.org/10.1111/j.1471-1842.2009.00848.x guadagno, r. e., okdie, b. m., & eno, c. a. (2008). who blogs? personality predictors of blogging. computers in human behavior, 24(5), 1993–2004. https://doi.org/10.1016/j.chb.2007.09.001 hamari, j., & keronen, l. (2017). why do people play games? a meta-analysis. international journal of information management, 37(3), 125–141. https://doi.org/10.1016/j.ijinfomgt.2017.01.006 hamburger, y. a., & ben-artzi, e. (2000). the relationship between extraversion and neuroticism and the different uses of the internet. computers in human behavior, 16(4), 441–449. https://doi.org/10.1016/s0747-5632(00)00017-0 hassan, y., & pandey, j. (2021). demystifying the dark side of social networking sites through mindfulness. australasian journal of information systems, 25. https://doi.org/10.3127/ajis.v25i0.2923 higgins, j., & green, s. (2011). cochrane handbook for systematic reviews of interviews. in the cochrane collaboration. http://handbook.cochrane.org/ https://doi.org/10.1136/bmj.315.7121.1533 https://doi.org/10.1089/cpb.2008.0030 https://doi.org/10.1016/j.chb.2016.12.036 https://doi.org/10.3127/ajis.v25i0.3141 https://doi.org/10.1016/j.compind.2018.08.007 https://doi.org/10.1136/bmj.39057.406644.68 https://doi.org/10.3102/0013189x005010003 https://doi.org/10.3127/ajis.v25i0.3129 https://doi.org/10.1111/j.1471-1842.2009.00848.x https://doi.org/10.1016/j.chb.2007.09.001 https://doi.org/10.1016/j.ijinfomgt.2017.01.006 https://doi.org/10.1016/s0747-5632(00)00017-0 https://doi.org/10.3127/ajis.v25i0.2923 http://handbook.cochrane.org/ australasian journal of information systems joshi, das & sekar 2023, vol 27, research article how personality traits affect ict use 28 higgins, j. p. t., thomas, j., chandler, j., cumpston, m., li, t., page, m. j., & welch, v. a. (2019). cochrane handbook for systematic reviews of interventions. john wiley & sons. hofstede, g., & fink, g. (2007). culture: organisations, personalities and nations. gerhard fink interviews geert hofstede. european journal of international management, 1(1-2), 14-22. https://doi.org/10.1504/ejim.2007.012914 huang, c. (2019). social network site use and big five personality traits: a meta-analysis. computers in human behavior, 97, 280–290. https://doi.org/10.1016/j.chb.2019.03.009 hughes, d. j., rowe, m., batey, m., & lee, a. (2012). a tale of two sites: twitter vs. facebook and the personality predictors of social media usage. computers in human behavior, 28(2), 561–569. https://doi.org/10.1016/j.chb.2011.11.001 inaba, t., & squicciarini, m. (2017). ict: a new taxonomy based on the international patent classification. oecd science, technology and industry working papers https://doi.org/10.1787/18151965 lang, k. r., & jarvenpaa, s. (2005). managing the paradoxes of mobile technology. information systems management, 22(4), 7-23. https://doi.org/10.1201/1078.10580530/45520.22.4.20050901/90026.2 jayaprakash, p., & pillai, r. r. (2021). the role of ict and effect of national culture on human development. journal of global information technology management, 24(3), 183–207. https://doi.org/10.1080/1097198x.2021.1953319 john, o. p., naumann, l. p., & soto, c. j. (2008). paradigm shift to the integrative big five trait taxonomy: history, measurement, and conceptual issues. in o. p. john, r. w. robins, & l. a. pervin (eds.), handbook of personality: theory and research (pp. 114–158). nyc, usa: guilford press. kalmus, v., realo, a., & siibak, a. (2011). motives for internet use and their relationships with personality traits and socio-demographic factors. trames: a journal of the humanities and social sciences, 15(4), 385. https://doi.org/10.3176/tr.2011.4.04 kayiş, a. r., satici, s. a., yilmaz, m. f., şimşek, d., ceyhan, e., & bakioǧlu, f. (2016). big fivepersonality trait and internet addiction: a meta-analytic review. computers in human behavior, 63, 35–40. https://doi.org/10.1016/j.chb.2016.05.012 keller, h., & karau, s. j. (2013). the importance of personality in students’ perceptions of the online learning experience. computers in human behavior, 29(6), 2494–2500. https://doi.org/10.1016/j.chb.2013.06.007 kim, y., briley, d. a., & ocepek, m. g. (2015). differential innovation of smartphone and application use by sociodemographics and personality. computers in human behavior, 44, 141–147. https://doi.org/10.1016/j.chb.2014.11.059 king, w. r., & he, j. (2005). understanding the role and methods of meta-analysis in is research. communications of the association for information systems, 16(1), 32. https://doi.org/10.17705/1cais.01632 kipnis, d. (1996). trust and technology. trust in organizations: frontiers of theory and research, 39, 50. https://doi.org/10.4135/9781452243610 https://doi.org/10.1504/ejim.2007.012914 https://doi.org/10.1016/j.chb.2019.03.009 https://doi.org/10.1016/j.chb.2011.11.001 https://doi.org/10.1787/18151965 https://doi.org/10.1080/1097198x.2021.1953319 https://doi.org/10.3176/tr.2011.4.04 https://doi.org/10.1016/j.chb.2016.05.012 https://doi.org/10.1016/j.chb.2013.06.007 https://doi.org/10.1016/j.chb.2014.11.059 https://doi.org/10.17705/1cais.01632 australasian journal of information systems joshi, das & sekar 2023, vol 27, research article how personality traits affect ict use 29 kircaburun, k., alhabash, s., tosuntaş, ş. b., & griffiths, m. d. (2020). uses and gratifications of problematic social media use among university students: a simultaneous examination of the big five of personality traits, social media platforms, and social media use motives. international journal of mental health and addiction, 18(3), 525–547. https://doi.org/10.1007/s11469-018-9940-6 ko, m., mancha, r., beebe, n., & yoon, h. s. (2012). customers’ personality, their perceptions, and green concern on internet banking use. journal of information technology management, 23(4), 21–32. krzywinski, m., & altman, n. (2013). points of significance: significance, p values and t-tests. nature methods, 10(11), 1041–1042. https://doi.org/10.1038/nmeth.2698 lai, y., saab, n., & admiraal, w. (2022). university students’ use of mobile technology in selfdirected language learning: using the integrative model of behavior prediction. computers & education, 179, 104413. https://doi.org/10.1016/j.compedu.2021.104413 landers, r. n., & lounsbury, j. w. (2006). an investigation of big five and narrow personality traits in relation to internet usage. computers in human behavior, 22(2), 283–293. https://doi.org/10.1016/j.chb.2004.06.001 lane, w. (2012). the influence of personality traits on mobile phone application preferences. journal of economics and behavioral studies, 4(5), 252–260. https://doi.org/10.22610/jebs.v4i5.325 lane, w., & manner, c. (2011). the impact of personality traits on smartphone ownership and use. international journal of business and social science, 2(17). lin, c. m., & ong, c. s. (2010). understanding information systems continuance intention: a five factor model of personality perspective. in proceedings of the fourteenth pacific asia conference on information systems, taipei, taiwan. paper 52, 366-376. liu, d., & baumeister, r. f. (2016). social networking online and personality of self-worth: a meta-analysis. journal of research in personality, 64, 79–89. https://doi.org/10.1016/j.jrp.2016.06.024 liu, d., & campbell, w. k. (2017). the big five personality traits, big two metatraits and social media: a meta-analysis. journal of research in personality, 70, 229–240. https://doi.org/10.1016/j.jrp.2017.08.004 lönnqvist, j.-e., itkonen, j. v. a., verkasalo, m., & poutvaara, p. (2014). the five-factor model of personality and degree and transitivity of facebook social networks. journal of research in personality, 50, 98–101. https://doi.org/10.1016/j.jrp.2014.03.009 marengo, d., sindermann, c., häckel, d., settanni, m., elhai, j. d., & montag, c. (2020). the association between the big five personality traits and smartphone use disorder: a meta-analysis. journal of behavioral addictions, 9(3), 534–550. https://doi.org/10.1556/2006.2020.00069 marino, c., gini, g., vieno, a., & spada, m. m. (2018). a comprehensive meta-analysis on problematic facebook use. computers in human behavior, 83, 262–277. https://doi.org/10.1016/j.chb.2018.02.009 https://doi.org/10.1007/s11469-018-9940-6 https://doi.org/10.1038/nmeth.2698 https://doi.org/10.1016/j.compedu.2021.104413 https://doi.org/10.1016/j.chb.2004.06.001 https://doi.org/10.22610/jebs.v4i5.325 https://doi.org/10.1016/j.jrp.2016.06.024 https://doi.org/10.1016/j.jrp.2017.08.004 https://doi.org/10.1016/j.jrp.2014.03.009 https://doi.org/10.1556/2006.2020.00069 https://doi.org/10.1016/j.chb.2018.02.009 australasian journal of information systems joshi, das & sekar 2023, vol 27, research article how personality traits affect ict use 30 mark, g., & ganzach, y. (2014). personality and internet usage: a large-scale representative study of young adults. computers in human behavior, 36, 274–281. https://doi.org/10.1016/j.chb.2014.03.060 matusitz, j., & musambira, g. (2013). power distance, uncertainty avoidance, and technology: analyzing hofstede’s dimensions and human development indicators. journal of technology in human services, 31(1), 42–60. https://doi.org/10.1080/15228835.2012.738561 mccrae, r. r., & costa, p. t. (1987). validation of the five-factor model of personality across instruments and observers. journal of personality and social psychology, 52(1), 81–90. https://doi.org/10.1037/0022-3514.52.1.81 mccrae, r. r., & costa jr., p. t. (2008). the five-factor theory of personality. in o. p. john, r. w. robins, & l. a. pervin (eds.), handbook of personality: theory and research (3rd ed., pp. 159-181). nyc, usa: guilford press. mendonca, m. r. (2016). relating big five factor model to the acceptance and use of on-line shopping. international journal of marketing studies, 8(3), 89. https://doi.org/10.5539/ijms.v8n3p89 mengist, w., soromessa, t., & legese, g. (2020). method for conducting systematic literature review and meta-analysis for environmental science research. methodsx, 7, 100777. https://doi.org/10.1016/j.mex.2019.100777 mody, b. (1999). the internet in the other three-quarters of the world. the annual review of the institute for information studies, 69–93. moher, d., liberati, a., tetzlaff, j., altman, d. g., & prisma group*. (2009). preferred reporting items for systematic reviews and meta-analyses: the prisma statement. annals of internal medicine, 151(4), 264-269. https://doi.org/10.7326/0003-4819151-4-200908180-00135 mokhtarian, p. l, salomon, i., & handy, s. l. (2004). a taxonomy of leisure activities: the role of ict. uc davis: institute of transportation studies. https://escholarship.org/uc/item/0nk475qx mooradian, t. a., & swan, k. s. (2006). personality-and-culture: the case of national extraversion and word-of-mouth. journal of business research, 59(6), 778–785. https://doi.org/10.1016/j.jbusres.2006.01.015 mouakket, s. (2015). perceived usefulness of facebook: effects of personality traits and gender. journal of advances in information technology vol, 6(3). https://doi.org/10.12720/jait.6.3.111117 nadeem, a., marjanovic, o., & abedin, b. (2022). gender bias in ai-based decision-making systems: a systematic literature review. australasian journal of information systems, 26. https://doi.org/10.3127/ajis.v26i0.3835 nayal, p., & pandey, n. (2020). redemption intention of coupons: a meta-analytical review and future directions. journal of promotion management, 26(3), 372-395. https://doi.org/10.1080/10496491.2019.1699628 https://doi.org/10.1016/j.chb.2014.03.060 https://doi.org/10.1080/15228835.2012.738561 https://doi.org/10.1037/0022-3514.52.1.81 https://doi.org/10.5539/ijms.v8n3p89 https://doi.org/10.1016/j.mex.2019.100777 https://doi.org/10.7326/0003-4819-151-4-200908180-00135 https://doi.org/10.7326/0003-4819-151-4-200908180-00135 https://escholarship.org/uc/item/0nk475qx https://doi.org/10.1016/j.jbusres.2006.01.015 https://doi.org/10.12720/jait.6.3.111-117 https://doi.org/10.12720/jait.6.3.111-117 https://doi.org/10.3127/ajis.v26i0.3835 https://doi.org/10.1080/10496491.2019.1699628 australasian journal of information systems joshi, das & sekar 2023, vol 27, research article how personality traits affect ict use 31 nikbin, d., iranmanesh, m., & foroughi, b. (2021). personality traits, psychological well-being, facebook addiction, health and performance: testing their relationships. behaviour & information technology, 40(7), 706–722. https://doi.org/10.1080/0144929x.2020.1722749 nov, o., & ye, c. (2008). personality and technology acceptance: personal innovativeness in it, openness and resistance to change. proceedings of the 41st annual hawaii international conference on system sciences (hicss 2008), 448. or, c., & tao, d. (2012). usability study of a computer-based self-management system for older adults with chronic diseases. jmir research protocols, 1(2), e13. https://doi.org/10.2196/resprot.2184 özbek, v., alnıaçık, ü., koc, f., akkılıç, m. e., & kaş, e. (2014). the impact of personality on technology acceptance: a study on smart phone users. procedia-social and behavioral sciences, 150, 541–551. https://doi.org/10.1016/j.sbspro.2014.09.073 palmer, c., niemand, t., stöckmann, c., kraus, s., & kailer, n. (2019). the interplay of entrepreneurial orientation and psychological traits in explaining firm performance. journal of business research, 94, 183–194. https://doi.org/10.1016/j.jbusres.2017.10.005 papaioannou, d., sutton, a., carroll, c., booth, a., & wong, r. (2010). literature searching for social science systematic reviews: consideration of a range of search techniques. health information and libraries journal, 27(2), 114–122. https://doi.org/10.1111/j.14711842.2009.00863.x pavlicek, a., sudzina, f., & novak, r. (2018). impact of personality traits (bfi-2-xs) on using google or facebook login for other web portals. international scientific conference of business economics management and marketing 2018: proceedings of international scientific conference of business economics management and marketing 2018, 183–188. rad, m. s., nilashi, m., dahlan, h. m. (2018). information technology adoption: a review of the literature and classification. universal access in the information society, 17(2), 361–390. https://doi.org/10.1007/s10209-017-0534-z rajput, h. (2015). who’s chatting?: interplay between personality and whatsapp use. international journal of marketing and business communication, 4(4), 39–50. ratchford, m., & ratchford, b. t. (2021). a cross-category analysis of dispositional drivers of technology adoption. journal of business research, 127, 300–311. https://doi.org/10.1016/j.jbusres.2021.01.037 rathore, a. k., maurya, d., & srivastava, a. k. (2021). do policymakers use social media for policy design? a twitter analytics approach. australasian journal of information systems, 25. https://doi.org/10.3127/ajis.v25i0.2965 rauschnabel, p. a., brem, a., & ivens, b. s. (2015). who will buy smart glasses? empirical results of two pre-market-entry studies on the role of personality in individual awareness and intended adoption of google glass wearables. computers in human behavior, 49, 635–647. https://doi.org/10.1016/j.chb.2015.03.003 rojas-méndez, j. i., murphy, s. a., & papadopoulos, n. (2013). the u.s. brand personality: a sino perspective. journal of business research, 66(8), 1028–1034. https://doi.org/10.1016/j.jbusres.2011.12.027 https://doi.org/10.1080/0144929x.2020.1722749 https://doi.org/10.2196/resprot.2184 https://doi.org/10.1016/j.sbspro.2014.09.073 https://doi.org/10.1016/j.jbusres.2017.10.005 https://doi.org/10.1111/j.1471-1842.2009.00863.x https://doi.org/10.1111/j.1471-1842.2009.00863.x https://doi.org/10.1007/s10209-017-0534-z https://doi.org/10.1016/j.jbusres.2021.01.037 https://doi.org/10.3127/ajis.v25i0.2965 https://doi.org/10.1016/j.chb.2015.03.003 https://doi.org/10.1016/j.jbusres.2011.12.027 australasian journal of information systems joshi, das & sekar 2023, vol 27, research article how personality traits affect ict use 32 ross, c., orr, e. s., sisic, m., arseneault, j. m., simmering, m. g., & orr, r. r. (2009). personality and motivations associated with facebook use. computers in human behavior, 25(2), 578–586. https://doi.org/10.1016/j.chb.2008.12.024 ruijten, p. a. m. (2021). the similarity-attraction paradigm in persuasive technology: effects of system and user personality on evaluations and persuasiveness of an interactive system. behaviour & information technology, 40(8), 734–746. https://doi.org/10.1080/0144929x.2020.1723701 sait, s., al-tawil, k., & hussain, s. (2004). e-commerce in saudi arabia: adoption and perspectives. australasian journal of information systems, 12(1). https://doi.org/10.3127/ajis.v12i1.105 schlachter, s., mcdowall, a., cropley, m., & inceoglu, i. (2018). voluntary work-related technology use during non-work time: a narrative synthesis of empirical research and research agenda. international journal of management reviews, 20(4), 825–846. https://doi.org/10.1111/ijmr.12165 schmitt, d. p., allik, j., mccrae, r. r., & benet-martínez, v. (2007). the geographic distribution of big five personality traits: patterns and profiles of human self-description across 56 nations. journal of cross-cultural psychology, 38(2), 173-212. https://doi.org/10.1177/0022022106297299 singh, s. (2020). an integrated model combining ecm and utaut to explain users’ postadoption behaviour towards mobile payment systems. australasian journal of information systems, 24. https://doi.org/10.3127/ajis.v24i0.2695 soral, p., arayankalam, j., & pandey, j. (2020). the impact of ambivalent perception of bureaucratic structure on cyberloafing. australasian journal of information systems, 24. https://doi.org/10.3127/ajis.v24i0.2087 sriyabhand, t., & john, s. p. (2014). an empirical study about the role of personality traits in information technology adoption. humanities, arts and social sciences studies, 67–90. stachl, c., hilbert, s., au, j., buschek, d., de luca, a., bischl, b., hussmann, h., bühner, m., & wrzus, c. (2017). personality traits predict smartphone usage. european journal of personality, 31(6), 701–722. https://doi.org/10.1002/per.2113 svendsen, g. b., johnsen, j.-a. k., almås-sørensen, l., & vittersø, j. (2013). personality and technology acceptance: the influence of personality factors on the core constructs of the technology acceptance model. behaviour & information technology, 32(4), 323–334. https://doi.org/10.1080/0144929x.2011.553740 swickert, r. j., hittner, j. b., harris, j. l., & herring, j. a. (2002). relationships among internet use, personality, and social support. computers in human behavior, 18(4), 437–451. https://doi.org/10.1016/s0747-5632(01)00054-1 tan, w. k., & yang, c. y. (2012). personality trait predictors of usage of internet services. international conference on economics, business innovation, 38, 185–190. tan, w. k., & yang, c. y. (2014). internet applications use and personality. telematics and informatics, 31(1), 27–38. https://doi.org/10.1016/j.tele.2013.02.006 https://doi.org/10.1016/j.chb.2008.12.024 https://doi.org/10.1080/0144929x.2020.1723701 https://doi.org/10.3127/ajis.v12i1.105 https://doi.org/10.1111/ijmr.12165 https://doi.org/10.1177/0022022106297299 https://doi.org/10.3127/ajis.v24i0.2695 https://doi.org/10.3127/ajis.v24i0.2087 https://doi.org/10.1002/per.2113 https://doi.org/10.1080/0144929x.2011.553740 https://doi.org/10.1016/s0747-5632(01)00054-1 https://doi.org/10.1016/j.tele.2013.02.006 australasian journal of information systems joshi, das & sekar 2023, vol 27, research article how personality traits affect ict use 33 tao, d., wang, t., wang, t., zhang, t., zhang, x., & qu, x. (2020). a systematic review and meta-analysis of user acceptance of consumer-oriented health information technologies. computers in human behavior, 104, 106147. https://doi.org/10.1016/j.chb.2019.09.023 teng, c.-i. (2008). personality differences between online game players and nonplayers in a student sample. cyberpsychology & behavior, 11(2), 232–234. https://doi.org/10.1089/cpb.2007.0064 terzis, v., moridis, c. n., & economides, a. a. (2012). how student’s personality traits affect computer based assessment acceptance: integrating bfi with cbaam. computers in human behavior, 28(5), 1985–1996. https://doi.org/10.1016/j.chb.2012.05.019 timmermans, e., & de caluwé, e. (2017). to tinder or not to tinder, that’s the question: an individual differences perspective to tinder use and motives. personality and individual differences, 110, 74–79. https://doi.org/10.1016/j.paid.2017.01.026 tsao, w. (2013). big five personality traits as predictors of internet usage categories. international journal of management, 30(4), 374-386. https://www.proquest.com/scholarly-journals/big-five-personality-traits-as predictors/docview/1450040149/se-2 uesugi, s., okada, h., & sasaki, t. (2010). the impact of personality on acceptance of privacysensitive technologies: a comparative study of rfid and finger vein authentication systems. international symposium on technology and society, proceedings, 111–122. https://doi.org/10.1109/istas.2010.5514645 vaid, s. s., & harari, g. m. (2021). who uses what and how often?: personality predictors of multiplatform social media use among young adults. journal of research in personality, 91, 104005. https://doi.org/10.1016/j.jrp.2020.104005 van der schyff, k., flowerday, s., kruger, h., & patel, n. (2020). intensity of facebook use: a personality-based perspective on dependency formation. behaviour & information technology, 1–17. https://doi.org/10.1080/0144929x.2020.1800095 venkatesh, v., sykes, t. a., & venkatraman, s. (2014). understanding e-government portal use in rural india: role of demographic and personality characteristics. information systems journal, 24(3), 249–269. https://doi.org/10.1111/isj.12008 vu, h. t., & lim, j. (2021). effects of country and individual factors on public acceptance of artificial intelligence and robotics technologies: a multilevel sem analysis of 28-country survey data. 41(7), 1515–1528. https://doi.org/10.1080/0144929x.2021.1884288 wang, w. (2010). how personality affects continuance intention: an empirical investigation of instant messaging. pacis 2010 proceedings. 113. https://aisel.aisnet.org/pacis2010/113 wang, x., wang, x., fang, x., & jiang, q. (2018). power distance belief and brand personality evaluations. journal of business research, 84, 89–99. https://doi.org/10.1016/j.jbusres.2017.11.011 watson, d., & clark, l. a. (1997). extraversion and its positive emotional core. in handbook of personality psychology, 767-793. academic press. https://doi.org/10.1016/b978-0121346454/50030-5 https://doi.org/10.1016/j.chb.2019.09.023 https://doi.org/10.1089/cpb.2007.0064 https://doi.org/10.1016/j.chb.2012.05.019 https://doi.org/10.1016/j.paid.2017.01.026 https://www.proquest.com/scholarly-journals/big-five-personality-traits-as%20-predictors/docview/1450040149/se-2 https://www.proquest.com/scholarly-journals/big-five-personality-traits-as%20-predictors/docview/1450040149/se-2 https://doi.org/10.1109/istas.2010.5514645 https://doi.org/10.1016/j.jrp.2020.104005 https://doi.org/10.1080/0144929x.2020.1800095 https://doi.org/10.1111/isj.12008 https://doi.org/10.1080/0144929x.2021.1884288 https://aisel.aisnet.org/pacis2010/113 https://doi.org/10.1016/j.jbusres.2017.11.011 https://doi.org/10.1016/b978-012134645-4/50030-5 https://doi.org/10.1016/b978-012134645-4/50030-5 australasian journal of information systems joshi, das & sekar 2023, vol 27, research article how personality traits affect ict use 34 wei, r., & lo, v. h. (2016). staying connected while on the move: cell phone use and social connectedness. new media & society, 8(1), 53–72. https://doi.org/10.1177/1461444806059870 witt, e. a., massman, a. j., & jackson, l. a. (2011). trends in youth’s videogame playing, overall computer use, and communication technology use: the impact of self-esteem and the big five personality factors. computers in human behavior, 27(2), 763–769. https://doi.org/10.1016/j.chb.2010.10.025 wu, w.y., & ke, c.c. (2015). an online shopping behavior model integrating personality traits, perceived risk, and technology acceptance. social behavior and personality: an international journal, 43(1), 85–97. https://doi.org/10.2224/sbp.2015.43.1.85 xiao, h., li, w., cao, x., & tang, z. (2012). the online social networks on knowledge exchange: online social identity, social tie and culture orientation. journal of global information technology management, 15(2), 4–24. https://doi.org/10.1080/1097198x.2012.11082753 xu, r., frey, r. m., fleisch, e., & ilic, a. (2016). understanding the impact of personality traits on mobile app adoption – insights from a large-scale field study. computers in human behavior, 62, 244–256. https://doi.org/10.1016/j.chb.2016.04.011 zabkar, v., arslanagic-kalajdzic, m., diamantopoulos, a., & florack, a. (2017). brothers in blood, yet strangers to global brand purchase: a four-country study of the role of consumer personality. journal of business research, 80, 228–235. https://doi.org/10.1016/j.jbusres.2017.06.006 zhang, l., pentina, i., & fox kirk, w. (2017). who uses mobile apps to meet strangers: the roles of core traits and surface characteristics. journal of information privacy and security, 13(4), 207–225. https://doi.org/10.1080/15536548.2017.1394072 zhang, p. v., kim, s., & chakravarty, a. (2022). influence of pull marketing actions on marketing action effectiveness of multichannel firms: a meta-analysis. journal of the academy of marketing science. https://doi.org/10.1007/s11747-022-00877-4 zhou, t., & lu, y. (2011). the effects of personality traits on user acceptance of mobile commerce. international journal of human–computer interaction, 27(6), 545–561. https://doi.org/10.1080/10447318.2011.555298 appendix 1 study characteristics of selected studies study sample size country technology evaluated technology type examined big-five traits voluntar -iness theoretica l models altanopoulou & tselios, 2015 85 greece wiki-based learning c&e e,a,o,c,n v ffm amichaihamburger et al., 2002 40 israel internet use sn e,n v ffm behrenbruch et al, 2013 272 germany mobile applications sn e v ffm, tam benlian & hess,2010 232 germany enterprise resource planning c&e e,a,o,c,n v ffm, software evaluation https://doi.org/10.1177/1461444806059870 https://doi.org/10.1016/j.chb.2010.10.025 https://doi.org/10.2224/sbp.2015.43.1.85 https://doi.org/10.1080/1097198x.2012.11082753 https://doi.org/10.1016/j.chb.2016.04.011 https://doi.org/10.1016/j.jbusres.2017.06.006 https://doi.org/10.1080/15536548.2017.1394072 https://doi.org/10.1007/s11747-022-00877-4 https://doi.org/10.1080/10447318.2011.555298 australasian journal of information systems joshi, das & sekar 2023, vol 27, research article how personality traits affect ict use 35 criteria model bernett et al.., 2015 347 usa web based classroom technology system c&e e,a,o,c,n nv ffm, utaut burtăverde et al., 2019 341 romania smartphone use b&c, ent, info, sn e,a,o,c,n v ffm, hexaco, dark triad butt & phillips, 2008 115 australia calls and sms i&c e,a,c,n v ffm camden et al., 2018 425 turkey technology use c&e e,a,o,c,n v ffm, tam chorley et al., 2015 174 uk use of location based social networks sn c,n v ffm clemens et al., 2017 137 germany smartphone use b&c, comm, ent, info e,a,o,c,n v ffm cohen & baruth, 2017 72 israel online learning based technology c&e o,c nv ffm davis et al., 2007 111 usa microsoft excel spreadsheet software c&e o,n nv ffm, itspecific traits model devraj et al., 2008 180 usa e-project: commercial collaborative system c&e a,o,n nv ffm, tam ehrenberg et al., 2008 200 australia smartphone use comm e,a,n v ffm eşkisu et al., 2017 482 turkey facebook c&e, sn e,a,o,c,n v ffm guadagno et al, 2008 89 usa blogging info e,a,o,c,n v ffm hamburger& ben-artzi,2000 72 israel online services info, ent e,n, v ffm hughes et al., 2012 300 uk facebook and twitter info, sn e,a,o,c,n v ffm kalmus et al., 2011 1507 estonia internet use info,sn o,c,n v ffm keller & karau., 2013 250 usa online academic course c&e e,a,o,c,n v ffm kim et al., 2015 9482 south korea mobile applications c&e, comm, info, sn, ent e,a,o,c,n v ffm kircaburun et al., 2018 1008 turkey social media based technologies sn e,a,o,c,n v ffm australasian journal of information systems joshi, das & sekar 2023, vol 27, research article how personality traits affect ict use 36 ko et al., 2012 301 usa internet banking b&c a,o v ffm, tam lane & manner, 2011 312 usa smartphone use comm e,a,n v ffm lane, 2012 233 usa mobile applications b&c, c&e, sn e,a,c v ffm, tam lin & ong, 2010 65 china digital bulletin board system info a,o v ffm, tam mark & ganzach, 2014 8984 usa internet use c&e, ent, comm e,o,c,n v ffm mcelroy et al., 2007 132 usa internet use, e-commerce b&c e,n,o v ffm mouakket, 2015 397 uae facebook sn e,a,c,n v ffm, tam ozbek et al., 2014 401 turkey smartphone use comm e,a,o,c,n v ffm, tam rajput, 2015 137 india whatsapp sn, comm e,a,o,c,n v ffm rauschnabel et al., 2015 146 germany google glass ent e,o,n v ffm rosen & kluemper, 2008 522 usa social media based technologies sn e,c v ffm, tam ryan& xenos, 2011 1324 australia facebook info, sn e,a,o,c,n v ffm sriyabhand & john, 2014 320 thailand m-social networking technologies sn o v ffm, tam sullivan, 2012 251 usa technology knowledge management systems info e,a,o,c,n v ffm svendsen et al.., 2013 1004 norway online software info e,a,o,n nv ffm, tam swickert et al., 2002 206 usa internet use sn e,a,o,c,n v ffm tan & yang, 2012 148 taiwan internet applications b&c, sn e,n v ffm tan&yang, 2014 172 taiwan mobile applications comm, sn, ent, b&i, e,a,o,c,n v ffm terzis et al., 2012 117 greece computer based assessment c&e e,o,n v ffm tosun & lajunen, 2010 427 turkey social media based technologies sn e v ffm tsao, 2013 429 taiwan internet use ent, comm, a, o v ffm venkatesh et al., 2014 311 india e-governance info e,a,o,c,n v ffm wang, 2010 228 china instant messaging applications comm e,c, v ffm, tam australasian journal of information systems joshi, das & sekar 2023, vol 27, research article how personality traits affect ict use 37 xu et al., 2016 22 switzerland mobile applications sn e,a,n v ffm zhang et al., 2017 213 usa location based social networks sn e,a,o,c,n v ffm zhou & lu, 2011 268 china mobile commerce b&c e,a,o,c,n v ffm note: v, voluntary; nv, non-voluntary; b&c, business and commerce; c&e, career and education; comm, communication; ent, entertainment; info, information; sn, social networking; ffm, five factor model; tam, technology acceptance model. australasian journal of information systems joshi, das & sekar 2023, vol 27, research article how personality traits affect ict use 38 appendix 2 moderation analysis results correlation technology type (moderator) region of country (moderator) voluntariness e-ict b&c c&e comm ent info sn north america australia europe asia v nv n 7 14 17 11 14 23 12 10 20 25 80 8 effect size 0.17 0.1 0.13 0.05 0.04 0.16 0.17 0.1 0.12 0.03 0.11 0.18 ci (0.06-0.28) (0.05-0.15) (0.08-0.17) (0.010.10) (0.02-0.06) (0.110.21) (0.13-0.21) (0.02-0.18) (0.03-0.20) (0.00-0.06) (0.08-0.12) (0.140.21) q (p-value) 15.96(0.01) 27.84(0.01) 12.80(0.01) a-ict n 5 19 14 5 14 16 11 11 13 19 64 10 effect size 0.03 0.13 0.12 0.07 0.08 0 0.04 -0.06 0.11 0.03 0.07 0.14 ci (-0.09-0.15) (0.07-0.18) (0.04-0.02) (0.030.11) (0.04-0.13) (0.010.07) (-0.04-0.12) (-0.13-0.01) (0.03-0.19) (0. 01-0.06) (0.05-0.10) (0.100.19) q (p-value) 9.81(0.08) 10.58(0.01) 9.10 (0.01) o-ict n 5 19 6 12 14 14 12 4 14 23 61 9 effect size 0.14 0.18 0.11 0.02 0.13 0.06 0.07 0.09 0.11 0.01 0.09 0.25 ci (0.07-0.21) (0.13-0.22) (0.01-0.21) (-0.010.04) (0.07-0.18) (0.040.07) (0.00-0.14) (0.05-0.13) (0.07-0.14) (-0.06-0.08) (0.07-0.11) (0.210.28) q (p-value) 24.85(0.01) 5.70 (0.13) 49.28(0.01) c-ict n 5 20 10 4 15 20 13 9 18 20 67 7 effect size 0.03 0.18 -0.04 -0.02 0.05 -0.04 0.02 -0.04 0.07 0.17 0.04 0.15 ci (0.01-0.05) (0.12-0.24) (-0.10-0.02) (-0.080.03) (0.03-0.06) (-0.05-(0.03)) (-0.02-0.06) (-0.09-0.01) (-0.01-0.15) (0.07-0.26) (0.02-0.06) (0.110.19) q (p-value) 40.77(0.01) 21.22(0.00) 20.88(0.01) n-ict n 7 16 12 8 13 20 13 9 20 16 68 8 effect size 0.02 -0.13 0.06 0.06 0.02 0.03 -0.01 0.04 0.02 0.06 0.01 -0.24 ci (0.01-0.04) (-0.25-0.01) (-0.01-0.12) (0.020.08) (0.01-0.03) (0.020.05) (-0.03-0.03) (-0.02-0.11) (-0.06-0.09) (0.02-0.10) (-0.03-0.03) (-0.550.13) q (p-value) 79.51(0.01) 6.16(0.10) 1.61(0.20) australasian journal of information systems joshi, das & sekar 2023, vol 27, research article how personality traits affect ict use 39 copyright: © 2023 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v27i0.3985 http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 literature review 2.1 the big five personality model 2.2 extant literature on personality and icts use 2.2.1 extroversion and icts use 2.2.2 agreeableness and icts use 2.2.3 openness to experiences and icts use 2.2.4 conscientiousness and icts use 2.2.5 neuroticism and icts use 2.2.6 summary of previous research 2.3 extant meta-analytic studies 3 methodology 4 results 4.1 study characteristics 4.2 meta-analysis 4.3 subgroup analysis 5 discussion we conducted a meta-analysis of forty-eight studies that quantitatively investigates the use of various forms of icts in view of the big five model of personality. this study highlights the relevance of big five traits in unfolding the riddle of indiv... 5.1 primary findings 5.2 implications 5.2.1 theoretical implications 5.2.2 managerial implications 5.3 limitations and future research directions 6 conclusion paper title (paper title style) australasian journal of information systems singh 2020, vol 24, research article explaining post-adoption of mobile payment systems 1 an integrated model combining the ecm and the utaut to explain users’ post-adoption behaviour towards mobile payment systems sindhu singh k.j. somaiya institute of management studies & research india sindhusingh@somaiya.edu abstract technological progression in mobile phones has increased the popularity of mobile payments. users can shop online through a mobile device, which is time saving and convenient. mobile payment systems involve ongoing interactions between users and payment providers. the initial acceptance of mobile payment systems has been studied extensively, but few studies have attempted to understand users’ post-adoption behaviour. this study employs an integrated model with the unified theory of acceptance and use of technology (utaut) framework and the expectation confirmation model (ecm), along with two additional constructs: perceived security and trust. the empirical results show that the integrated model has a higher predictive power to explain continuance intentions for using mobile payment systems with significant factors of satisfaction, trust, performance expectancy, and effort expectancy. this study confirmed that the utaut model could be extended to explain postadoption behaviour towards mobile payment systems. the study’s findings have theoretical and practical value to further the understanding of preand post-adoption behaviour towards mobile payment systems. keywords: mobile payment systems; continuance intention to use; utaut; ecm; postadoption; india 1 introduction smartphone usage has increased tremendously due to price reductions and the increasing speeds of the mobile internet. mobile phones are now ubiquitous devices, offering entertainment, education, personal assistants and various options for conducting financial transactions. technological progression in mobile phones has increased the popularity of mobile payments, which are widely used in e-commerce and in-store transactions and even for utility bill payments. the growth of mobile commerce and the establishment of policies to enable cashless payments in both developed and emerging economies have also supported these trends. mobile payments allow users to conduct transactions on their mobile devices; the convenience of this approach is enticing a growing number of people to shop online. with its unparalleled convenience and anytime and anywhere payment options, mobile payment systems will continue their rapid growth in the foreseeable future. users of mobile wallets reached approximately 2.1 billion in 2019 (mobile payments world, 2019). china is ahead of other countries in mobile payment usage, followed by india and several african countries (gerban, 2019). in india, mobile phone penetration is high, and people prefer smartphones over computers for buying and selling online (adhikary, 2018). australasian journal of information systems singh 2020, vol 24, research article explaining post-adoption of mobile payment systems 2 consumers have a multitude of choices when it comes to mobile payments, and they can take advantage of cash discounts and other offers. while mobile payments offer tremendous convenience, threats such as mobile device theft, loss of personal information and privacy incursions may induce fear in potential users (fan, shao, li, & huang, 2018; iman, 2018). researchers have extensively studied the initial acceptance of mobile payments and have explored various antecedents to initial adoption (fan et al., 2018; hampshire, 2017; oliveira, thomas, baptista, & campos, 2016; phonthanukitithaworn, sellitto, & fong, 2015; schierz, schilke, & wirtz, 2010). mobile payment systems involve ongoing interactions between users and payment providers, and solid relationships are essential to ensuring customers’ continuous usage, particularly in highly competitive contexts. for example, cutthroat competition exists in the mobile wallet industry, with extremely low switching costs for customers (zhou, 2013). hence, the critical challenge for mobile payment networks is to retain customers. in the literature on mobile payment systems, studies on users’ post-adoption behaviour are scarce (cao, yu, liu, gong, & adeel, 2018; chen & li, 2017; zhou, 2013). additional studies in this area will strengthen the theoretical foundation and inform practical implementation. this study addressed this research gap by studying continuous intentions to use mobile payment systems and by developing an integrated model with the expectationconfirmation model (ecm) (bhattacherjee, 2001) and the unified theory of acceptance and use of technology (utaut) framework (venkatesh, morris, davis, & davis, 2003). this study contributes to the research on mobile payment adoption and management of mobile payment systems. the theoretical model developed in this study sheds light on the post-adoption behaviour related to these systems. the initial adoption of mobile payment systems (baabdullah, alalwan, rana, kizgin, & patil, 2019; oliveira et al., 2016; teo, tan, ooi, hew, & yew, 2015) has been studied using the utaut (venkatesh et al., 2003) and the utaut2 (venkatesh, thong, & xu, 2012) frameworks. however, few studies have used these models to investigate mobile payment post-adoption behaviour. the present study addresses this research gap, extending the utaut model to post-adoption behaviour and adding the constructs of trust and perceived security to explore users’ continuance intentions. this study’s theoretical contribution is in developing a novel integrated model to explore an understudied area. this study’s findings have both theoretical and practical value, furthering the understanding of preand post-adoption behaviour with respect to mobile payments. 2 literature review 2.1 mobile payments mobile payment systems allow payments for goods and services through mobile networks via smartphones (dahlberg, mallat, ondrus, & zmijewska, 2008). ghezzi, renga, balocco, and pescetto (2010, p. 5) defined a mobile payment system as ‘a process in which at least one phase of the transaction is conducted using a mobile device (such as mobile phone, smart phone, pda, or any wireless-enabled device) capable of securely processing a financial transaction over a mobile network or via various wireless technologies (bluetooth, radio frequency identification (rfid), near field communication (nfc), etc.)’. mobile payments can utilise any of the following systems: short message service (sms), near field communication (nfc), wireless application protocol (wap) and quick response code (qr) (chen & li, 2017). in sms-based mobile payments, customers initiate payments through sms, and merchants are notified when payments are successful. the payment is then deducted australasian journal of information systems singh 2020, vol 24, research article explaining post-adoption of mobile payment systems 3 from the customer’s monthly phone bill (ondrus & pigneur, 2006). nfc-based payment systems require that customers’ mobile phones are equipped with an nfc-enabled chip, and merchants with card readers can initiate payment processes (li, liu, & heikkilä, 2014). the wap-based mobile payment systems work in two ways. in one, customers use mobile browsers to initiate payments through banks at which they registered for mobile payments. alternatively, they can download mobile apps (such as paytm and phonepe in india) and link these apps with their bank accounts. then, they can use money in their bank accounts to pay for products and services (isaac & sherali, 2014). in qr-based payments, customers open mobile payment apps and scan the qr code to initiate payments (lee, cho, & jun, 2011). some of the well-known mobile wallet apps are apple pay, google pay and samsung pay, which are integrated with the banking systems of their resident countries. apple pay works only with apple devices, uses only nfc mechanisms, and uses encryption techniques and tokenisation for secure transactions (kang, 2018). using any of these payment mechanisms, the mobile wallet user can add their debit/credit card details to their wallets, after which the mobile wallet providers contact the card issuer for authentication and add the card information to the wallets after successful authentication (wang, hahn, & sutrave, 2016). samsung pay uses, in addition to nfc, magnetic secure transmission (mst), which allows mobile wallet users to use samsung pay in store even when retailers have not upgraded to an nfc reader, thereby ensuring the compatibility and convenience of the payment method (kang, 2018; wang et al., 2016). google pay, formerly known as android pay, supports only nfc technologies and works with any android devices for in-store and peer-to-peer payments (kang, 2018). along with in-store and online payments, in india, google pay can be linked to bank accounts that support unified payment interface (upi) for direct transactions instead of adding money to the wallet. similarly, paypal’s mobile app offers various services, such as send and receive money, by linking paypal accounts with banks as well as online and in-store payments. in addition, paypal accounts can be linked to google pay, and the paypal app allows access to these accounts from iphones/android phones as well as the ability to transfer funds through the linked bank account or through attached credit/debit cards. this study considers both types of mobile payments: payments initiated through mobile apps, and payments conducted through mobile browsers (figure 1). australasian journal of information systems singh 2020, vol 24, research article explaining post-adoption of mobile payment systems 4 figure 1: mobile payment process (source: wang, hahn, & sutrave (2016)) 2.2 post-adoption of mobile payments many studies of users’ initial adoption of mobile payments appear in the literature, and these studies have identified antecedents of mobile payments initial adoption, such as performance expectancy, perceived security, social influence, compatibility and trust (fan et al., 2018; hampshire, 2017; kim, mirusmonov, & lee, 2010; musa & aishare, 2015; oliveira et al., 2016; ramadan & aita, 2018). conversely, only a few studies have focused on users’ post-adoption behaviour towards mobile payment systems. among these few studies, cao, yu, liu, gong, and adeel (2018) used a trust transfer perspective to investigate the continuous intention to use mobile payments in 219 mobile payment users in china. their study revealed that trust transfer positively influenced continuous intention through satisfaction. kumar, adlakaha, and mukherjee (2018) studied continuance intention to use m-wallets in 265 mobile wallet users in india using the technology acceptance model (tam) and the ecm with the additional constructs of perceived security, trust and grievance redressal. their results revealed that grievance redressal, trust and satisfaction influenced continuance intention. in china, chen and li (2017) studied the continuous intention to use mobile payments in 38 interviewees (qualitative analysis) and 243 survey respondents (quantitative analysis) using it continuance theory and the risk–trust and affect–cognition framework. in this study, satisfaction and postadoption perceived usefulness significantly impacted the continuance intention to use mobile payments. lu, wei, yu, and liu (2016) examined the post-usage of m-payments in 724 mobile payment users in china using expectation-confirmation theory. their findings revealed that post-usage privacy protections and social influence impacted continued usage, whereas postusage mobility impacted satisfaction. other findings from their research were that espoused australasian journal of information systems singh 2020, vol 24, research article explaining post-adoption of mobile payment systems 5 cultural values acted as antecedents to social influence and post-usage mobility. yuan, liu, yao, and liu (2014) studied the continuance intention to use mobile payments in 434 users in china using an integrated model comprising the tam, task–technology fit and perceived risk with the ecm. their findings showed that satisfaction, perceived risk, perceived usefulness and perceived task–technology fit were the main drivers of continuance intention. zhou (2013, 2014) examined the factors of continuance intention to use mobile payments in 195 respondents in china using the is success model (delone & mclean, 1993) and flow theory. his findings indicated that trust, flow and satisfaction influenced continuance intention; service quality influenced trust; system quality affected satisfaction; and information quality and service quality affected flow (zhou, 2013, 2014). there have been a limited number of studies on post-adoption behaviour towards mobile payment systems, and the majority of these studies have been conducted in china (cao et al., 2018; chen & li, 2017; lu, wei, yu, & liu , 2016; yuan, liu, yao, & liu, 2014; zhou, 2013). more studies are needed to shed light on users’ post-adoption behaviour, which often changes depending on the cultural, economic and research context. hence, in the interest of generalisability, adoption behaviour should be studied in other parts of the world. the initial adoption of mobile payment systems (baabdullah et al., 2019; oliveira et al., 2016; teo et al., 2015) has been studied using the utaut (venkatesh et al., 2003) and the utaut2 (venkatesh et al., 2012) frameworks, but few studies have used this model to study post-adoption behaviour. mobile payment is a convenient option for conducting financial transactions, but it is still in its infancy, and not much is known about post-adoption behaviour. ensuring the loyalty of customers, who expect user-friendly interfaces for their mobile payment wallets and the option to choose from a wide variety of mobile payment solutions, is an enormous challenge for providers. thus, it is critical to examine the influence of usage factors, such as performance expectancy and effort expectancy, in post-adoption behaviour towards mobile payment systems. 3 theoretical model and hypotheses development this study used an integrated theoretical model (figure 2) based on the utaut (venkatesh et al., 2003) and the ecm (bhattacherjee, 2001). it added two additional constructs, perceived security and trust, with continuance intention to use mobile payment systems as the dependent variable. mobile payment systems are still emerging. to more fully capture the unique characteristics of this new technology and to better understand its continuous acceptance, it is preferable to integrate multiple theoretical frameworks (apanasevic, markendahl, & arvidsson, 2016; oliveira et al., 2016; zhou, 2013). 3.1 expectation-confirmation model the expectation-confirmation model (ecm) (bhattacherjee, 2001) posits that users’ continuance intention for any information system is influenced by their satisfaction and their perceived usefulness of continued usage. satisfaction results when expectations from prior use of information systems and from perceived usefulness are confirmed (bhattacherjee, 2001). the post-acceptance of any is depends upon the user satisfaction, which impacts continuance (2001). continuance intention is defined as ‘users’ intention to continue using is’ (2001, p. 359). the ecm has been used extensively to study the relationship between satisfaction and continuance intention to use various information systems (shang & wu, 2017; zhang, lu, gupta, & gao, 2015). however, there have been limited studies in the area of mobile payment australasian journal of information systems singh 2020, vol 24, research article explaining post-adoption of mobile payment systems 6 usage (susanto, chang, & ha, 2016). to understand post-adoption behaviour towards mobile payment systems, satisfaction and continuance intention were used as dependent variables, and the ecm served as the theoretical basis of the research model. 3.2 unified theory of acceptance and use of technology the unified theory of acceptance and use of technology (utaut) (venkatesh et al., 2003) was developed by assimilating elements from eight widely accepted models of information systems (is). the core constructs of the model were performance expectancy, effort expectancy, social influence and facilitating conditions, all of which influence behavioural intention to use technology, while the moderators were age, experience, gender and the voluntariness of use (venkatesh et al., 2003). the utaut explains higher variance in behavioural intentions compared to the eight other models (venkatesh et al., 2003). subsequently, many studies have used the model (hoque & sorwar, 2017; khalilzadeh, ozturk, & bilgihan, 2017). although the model has been widely utilised to explore behavioural intentions to use mobile payment systems (cao & niu, 2019; teo et al., 2015), it has seldom been employed to study post-adoption behaviour (oliveira et al., 2016). while the derivation of the utaut model has demonstrated user adoption of technologies, which is in effect a single choice, mobile payment continuance intention depends on users’ ongoing choices (dwivedi, rana, jeyaraj, clement, & williams, 2017). thus, this research addresses a gap and extends the utaut and ecm frameworks (bhattacherjee, 2001) to explore customers’ postadoption behaviour regarding mobile payment services. the adapted theoretical model used performance expectancy and effort expectancy from the utaut model (venkatesh et al., 2003). facilitating conditions and social influence were not used, since these constructs are more relevant to the initial adoption process than to continuance intention. this theoretical model integrated confirmation, satisfaction and continuance intention from the ecm model (bhattacherjee, 2001) to understand the postadoption behaviour of mobile payment users. mobile payment systems are vulnerable to various security threats. therefore, in the post-adoption phase, perceived security is critical. it thus follows that when users trust their mobile payment service provider, usage will increase. this is because each time a user has a positive experience with a mobile payment service provider, it enhances user trust, which accumulates with each positive experience. positive experiences also strengthen users’ confidence in the system’s overall security, making it more likely that they will use the system again. thus, these constructs play a significant role in continuous usage of mobile payment systems. prior research has confirmed that trust and perceived security can influence decisions to use mobile payment systems (giovanis, assimakopoulos, & sarmaniotis, 2018; phonthanukitithaworn et al., 2015; schierz et al., 2010).the independent variables used in the model are described in the next section. 3.3 performance expectancy in the utaut model (venkatesh et al., 2003, p. 447), performance expectancy is defined as 'the degree to which an individual believes that using the system will help him or her to attain gains in job performance’. in the utaut2 model (venkatesh et al., 2012, p. 159), performance expectancy in a consumer context ‘is defined as the degree to which using technology will provide benefits to consumers in performing certain activities’. mobile payment systems offer customers greater convenience by allowing them to conduct financial transactions through their personal mobile device anytime and anywhere. when customers realise this utility value australasian journal of information systems singh 2020, vol 24, research article explaining post-adoption of mobile payment systems 7 of the mobile payment system, their performance expectancy beliefs also increase (oliveira et al., 2016; zhou, 2014). hence, the following hypothesis is proposed. h1: performance expectancy significantly influences the continuance intention to use mobile payment systems. 3.4 effort expectancy in the utaut model (venkatesh et al., 2003, p. 447), effort expectancy is defined as ‘the degree of ease associated with the use of the system’. venkatesh et al. (2012, p. 159) adapted this construct in the utaut2 model, defining it as ‘the degree of ease associated with consumers’ use of technology’. customers expect their mobile payment system to have a simple interface, which increases the utility value of the system. user-friendly payment systems not only promote initial acceptance, they also play a major role in continuance intention. past research has confirmed that effort expectancy influences the initial adoption of mobile payment systems (cao & niu, 2019; teo et al., 2015). hence, the following hypotheses are proposed. h2: effort expectancy significantly influences the continuance intention to use mobile payment systems. h2a: effort expectancy significantly influences the performance expectancy of mobile payment systems. 3.5 perceived security fan, shao, li, and huang (2018, p. 526) defined perceived security in a mobile payment context as ‘the degree to which people believe that their property and information privacy is secure while using mobile payment’. customers will use a mobile payment system only when they perceive it has adequate security mechanisms (iman, 2018), and robust security features will enhance customers’ trust in their mobile payment system providers. it follows, then, that customers are more likely to patronise mobile payment service providers when the providers are trustworthy and verified. furthermore, advanced security features in mobile payment systems increase users’ continuance intention to use the systems. hence, the following hypotheses are proposed. h3: perceived security significantly influences the continuance intention to use mobile payment systems. h3a: perceived security significantly influences customer trust in the mobile payment systems. 3.6 trust mobile payment system customers place a high level of importance on trust. after customers initially use a system, their further usage will depend on trust-building. trust has various definitions in the existing literature. mcknight, choudhury, and kacmar (2002, p. 336) described it as ‘trusting beliefs lead to trusting intentions, which in turn result in trust-related behaviours. previous studies have verified that trust is a significant predictor of continuance intention to use mobile payments (zhou, 2013, 2014). trust associations between customers and mobile payment service providers significantly affect customers’ decisions to continue using the mobile payment systems. hence, the following hypothesis is proposed. h4: trust significantly influences continuance intention to use mobile payment systems. australasian journal of information systems singh 2020, vol 24, research article explaining post-adoption of mobile payment systems 8 3.7 satisfaction in mobile payment systems, customer satisfaction leads to continuance intention to use mobile payment systems. satisfaction follows a positive outcome from the initial acceptance of mobile payment systems and has many definitions in the existing literature. for example, in marketing, it has been defined as ‘the consumer’s sense that consumption provides outcomes against a standard of pleasure versus displeasure’ (oliver, 1999, p. 34). in an electronic marketplace, satisfaction is referred to as e-satisfaction and defined as ‘the contentment of the customer with respect to his or her prior purchasing experience with a given electroniccommerce firm’ (anderson & srinivasan, 2003, p. 125). in the is field, delone and mclean (2003, p. 25) defined user satisfaction as encompassing ‘the entire customer experience cycle from information retrieval through purchase, payment, receipt, and service’. previous studies have reported that satisfaction is a strong predictor of continuance intention (cao et al., 2018; chen & li, 2017; zhou, 2013). hence, the following hypothesis is proposed. h5: satisfaction significantly influences the continuance intention to use mobile payment systems. 3.8 relationships between performance expectancy, confirmation and satisfaction bhattacherjee (2001, p. 359) defined confirmation as a user’s ‘perception of the congruence between the expectation of is use and its actual performance’. the ecm (bhattacherjee, 2001) suggests that satisfaction is determined by confirmation of users’ expectations of is use and its perceived usefulness. in the mobile payment systems context, customers’ expectations are confirmed with continued usage, and their post-adoption usage of mobile payment systems increases their utilitarian expectation in terms of performance expectancy and satisfaction. when customers have their mobile payment systems usage experience confirmed, performance expectancy and satisfaction with the system are engendered. hence, the following hypotheses are proposed. h6: confirmation significantly influences performance expectancy for mobile payment systems. h7: confirmation significantly influences satisfaction with using mobile payment systems. h8: performance expectancy significantly influences satisfaction with using mobile payment systems. australasian journal of information systems singh 2020, vol 24, research article explaining post-adoption of mobile payment systems 9 figure 2: theoretical model 4 research methodology 4.1 variable measurement the variables used in the theoretical model were measured using validated items from prior research (appendix a). these items were reworded to suit the current study context. this study used a 5-point likert scale to measure the items, since this scale is widely used in marketing and social science research (garland, 1991), where 1 indicates ‘strongly disagree’ and 5 indicates ‘strongly agree’. in the questionnaire developed to measure and test the theoretical model, 28 items were used. to test the applicability of the instrument in a mobile payment adoption context, the questionnaire was shown to several academics and practitioners in the is and marketing fields. it was also pretested with 10 mobile payment customers to ensure completeness and correctness, and the feedback from both groups was incorporated into the final questionnaire. 4.2 sample profile and data collection this study followed a quantitative research approach, using a questionnaire to collect data. the sample population identified were mobile payment users who used mobile payment systems at least once. a qualifying question was used to identify the target respondents: ‘do you use any mobile payment system at least once?’. convenience sampling (a non-probability sampling method) was used to identify respondents. given the lack of availability of the mobile payment customer list and the time and cost of collecting the response from each customer, this study used convenience sampling as a method for collecting the responses from the survey questionnaire. earlier studies on mobile payment adoption from both developed and developing countries also used non-probability sampling to collect responses (fan et al., 2018; chen & li, 2017; kumar, adlakaha, & mukherjee, 2018). this research study used various multivariate techniques, and 28 items were used to measure the various constructs in the theoretical model. in addition, the following guidelines suggested by hair et al. (2010) were also followed: 10 observations per item, and a minimum sample size of 280. the data australasian journal of information systems singh 2020, vol 24, research article explaining post-adoption of mobile payment systems 10 collection took place in mumbai, the financial capital, and in one metropolitan region of india. a total of 500 questionnaires were distributed, and the survey was conducted over a period of three months, from may to july 2019. the study’s purpose was explained to respondents, and assurances were made that survey responses would be used only for the current study. participation in the survey was voluntary, and respondents were free to skip questions they did not wish to answer. the questionnaire was distributed to participants in offline and online formats. some offline or paper versions were distributed to post-graduate students, faculty and supporting staff at a business management school in mumbai, yielding 100 collected responses. in the online version, a google form link was sent through personal email and whatsapp contacts. a total of 400 questionnaires were collected from both the online and offline methods, out of which 370 were completed and thus deemed usable. the distribution of the questionnaires and the completed and usable responses were outlined in table 1. after removing incomplete forms and excluding those who had not used mobile payments at least once, the response rate was determined to be 74%. a chi-square test was conducted which demonstrated that no significant differences exist among gender, age, education, and income between the online and offline groups at the 5% significance level. type of survey distributed questionnaires completed responses usable responses offline 150 100 100 online 350 300 270 total 500 400 370 table 1: distribution of questionnaires data were analysed using the statistical package for social sciences (spss) version 21.0. structural equation modelling (sem) was used to test this study’s theoretical model. this technique measures the hypothesised causal relationships among multiple variables simultaneously and estimates the strength of interrelationships among latent constructs. path analysis models include covariance-based sem (cb-sem) or partial least squares sem (plssem) (hair jr, hult, ringle, & sarstedt, 2016), with cb-sem being more suitable for theory testing, and pls-sem being more appropriate for modelling the theoretical relationships among latent constructs (hair jr et al., 2016). pls-sem can model latent constructs for small and medium sample sizes and works well in the presence of data non-normality (chin, marcolin, & newsted, 2003). this study was exploratory in nature and prediction-based, integrating two theoretical perspectives, the utaut model and the ecm, along with perceived security and trust in the context of mobile payment systems. the following requirements of this study met the guidelines suggested by hair, risher, sarstedt, and ringle (2019, p. 5) for choosing pls-sem as the suitable method: • this study tested a theoretical framework, from a prediction perspective, that integrated two theoretical perspectives, the utaut model and the ecm, to predict users’ continuance intention to use mobile payment systems. • this study assessed the mediation effect in the structural model with the presence of multiple mediators and used pls-sem, as ‘pls-sem is superior to regression analysis when assessing mediation’ (hair, sarstedt, & ringle, 2019, p. 567). • the analysis included the structural model, which was complex and consisted of seven constructs, 28 indicators and model relationships. australasian journal of information systems singh 2020, vol 24, research article explaining post-adoption of mobile payment systems 11 • the objective of the study was to explore theoretical extensions of well-established theories, such as the utaut and the ecm, in the mobile payment context, along with the additional constructs of perceived security and trust. the study used pls-sem path modelling with smart pls 3 software (ringle, da silva, & bido, 2015) to analyse the measurement model and structural paths of the theoretical model. 5 results the mobile payment usage profiles of the 370 respondents were analysed (table 2). among respondents, 55.9% were male and 44.1% were female. around 70.3% of the respondents reported regularly using mobile payment systems for various payment transactions. further, the results showed that paytm was the dominant mobile wallet compared to others, such as mobikwik and freecharge, with a 78.38% usage rate. the majority of respondents reported using mobile payment systems for small-value transactions, with frequent usage for mobile recharges, grocery stores, online shopping and paying bills. variable category frequency percentage (%) gender male 207 55.9 female 163 44.1 age 15–25 179 48.4 26–35 149 40.3 36-45 28 7.6 above 45 14 3.7 educational qualification 10th 3 0.8 12th 17 4.6 diploma 19 5.1 degree 211 57.0 postgraduate 120 32.4 income less than 5 lakhs 103 27.84 between 5–10 lakhs 165 44.59 above ten lakhs 102 27.57 frequency of mobile payment usage daily 27 7.3 1–2 times a week 195 52.7 3–6 times a week 101 27.3 once a month 47 12.7 mobile payment option paytm 290 78.38 mobikwik 45 12.16 freecharge 26 7.03 others 9 2.43 size of transactions less than rs 100 13 3.5 rs 100 to rs 1000 165 44.6 rs 1001 to less than rs 5000 192 51.9 table 2: demographic profile of the respondents 5.1 normality assessment to measure the normality of the survey items of the constructs, the skewness-kurtosis method was used. spss 21.0 was used to test the values of skewness and kurtosis. all the values of the measured items had skewness and kurtosis values in the acceptable range, between -2 and +2 (byrne, 2016; george & mallery, 2010; hair, black, babin, & anderson, 2010). hence, the data set followed a univariate normal distribution. to identify the outliers in the data set, z-scores australasian journal of information systems singh 2020, vol 24, research article explaining post-adoption of mobile payment systems 12 of the variables were calculated, and the acceptable range of values were between -3 and +3 (kline, 2011). out of 370 usable data items, six items were identified as having z-score values above the acceptable range, demonstrating the presence of outliers; these items were thus removed from the data set before further analysis. 5.2 common method bias survey-based research is vulnerable to common method variance when respondents complete the survey questionnaire themselves (podsakoff, mackenzie, lee, & podsakoff, 2003), and for this reason, harman’s single-factor test was conducted to examine common method bias (cmb). in this test, multiple factors were extracted from the factor analysis, and no single factor accounted for major variance. thus, common method variance was not a major issue in this study. 5.3 pls-sem model assessment this study used pls-sem and the two-step approach suggested by anderson and gerbing (1988). first, the measurement model was analysed for reliability and validity. next, the structural model was analysed to test the hypothesised causal relationships. 5.3.1 measurement model analysis the reliability of the constructs was assessed by computing cronbach’s alpha and composite reliability (cr). the cut-off value recommended for both cronbach’s alpha and composite reliability is 0.70 (hair et al., 2010). table 3 shows that all constructs had cut-off values above 0.70, indicating that all items used in this study had adequate reliability. to check the validity of the constructs, the average variance extracted (ave) was calculated. this is a complementary measure to composite reliability that reflects the overall amount of variance in the indicators, accounting for the latent construct (hair et al., 2010). the cut-off value of ave was above 0.50 (hair et al., 2019), with all constructs used in this study exceeding the cutoff value (table 3). all items of the measured constructs (table 3) had indicator loadings greater than the cut-off value of 0.708 (hair et al., 2019), meaning that the constructs explained more than 50% of the indicators’ variance, demonstrating the reliability of the constructs’ items. to establish the validity of the measurement model, convergent validity and discriminant validity were estimated. each construct’s composite reliability and ave were calculated to estimate the convergent validity of the measurement model. both measurements were above the cut-off value (fornell & larcker, 1981) for the measurement model (table 3); thus, convergent validity was established for this study. australasian journal of information systems singh 2020, vol 24, research article explaining post-adoption of mobile payment systems 13 construct item loading ave cronbach’s alpha composite reliability continuance intention (ci) ci1 0.851 0.735 0.88 0.917 ci2 0.856 ci3 0.898 ci4 0.823 confirmation (conf) conf1 0.905 0.746 0.829 0.898 conf2 0.877 conf3 0.806 effort expectancy (ee) ee1 0.854 0.765 0.898 0.929 ee2 0.89 ee3 0.88 ee4 0.875 performance expectancy (pe) pe1 0.855 0.687 0.847 0.897 pe2 0.815 pe3 0.761 pe4 0.879 perceived security (ps) ps1 0.796 0.721 0.873 0.912 ps2 0.881 ps3 0.882 ps4 0.834 satisfaction (sat) sat1 0.893 0.815 0.886 0.93 sat2 0.924 sat3 0.891 trust (tru) tru1 0.801 0.695 0.911 0.932 tru2 0.86 tru3 0.865 tru4 0.885 tru5 0.838 tru6 0.746 table 3: measurement model analysis to compute discriminant validity, as suggested by fornell and larcker (1981), one compares the shared variance between the constructs with the ave from each construct. discriminant validity was established for this measurement model, as the shared variance between the constructs was lower than the ave for each construct (table 4). constructs ci conf ee pe ps sat trust ci 0.857 conf 0.731 0.864 ee 0.646 0.7 0.875 pe 0.631 0.709 0.79 0.829 ps 0.433 0.519 0.381 0.407 0.849 sat 0.789 0.707 0.625 0.597 0.439 0.903 trust 0.685 0.71 0.602 0.568 0.626 0.717 0.834 ci: continuance intention; conf: confirmation; ee: effort expectancy; pe: performance expectancy; ps: perceived security; sat: satisfaction table 4: discriminant validity of measured items australasian journal of information systems singh 2020, vol 24, research article explaining post-adoption of mobile payment systems 14 5.3.2 structural model analysis to test the theoretical model, a non-parametric bootstrapping method was used with 5,000 samples. the structural model (figure 3) and the path coefficients are shown in table 5. the path coefficients were significant for performance expectancy, effort expectancy, trust and satisfaction (hypotheses h1, h2, h4, and h5), indicating significant influences on continuance intention to use mobile payment systems. effort expectancy significantly influenced performance expectancy, thus validating hypothesis h2a. perceived security significantly influenced trust, hence validating hypothesis h3a. performance expectancy and confirmation positively influenced satisfaction, validating hypotheses h7 and h8. confirmation significantly influenced performance expectancy, hence validating hypothesis h6. however, perceived security did not influence continuance intention to use mobile payment systems; thus, hypothesis h3 was rejected. the control variables of age, gender and income did not influence the continuance intention to use mobile payment systems. the β coefficient represents the influence of the independent variable in the theoretical model. figure 3 shows that performance expectancy (β=0.129, p=0.004), effort expectancy (β=0.12, p=0.035), trust (β=0.167, p=0.01) and satisfaction (β=0.518, p=0.000) were statistically significant. among the independent variables, satisfaction demonstrated the highest influence (β=0.518) on continuance intention to use mobile payment systems, followed by trust (β=0.167), performance expectancy (β=0.129) and effort expectancy (β=0.12). perceived security (β=0.001, p=0.988) was not statistically significant – that is, it did not directly influence the dependent variable, continuance intention to use mobile payment systems, but it did have an indirect influence through trust. in the post-adoption of mobile payment systems, customers have sufficient experience and confidence when using their mobile payment system. hence, they may consider trust as a critical factor in their decision to continue with their mobile payment service providers. security features, on the other hand, are built into the mobile payment systems. this may be the reason perceived security was not significant. the r2 value obtained during the structural model analysis represents the total variance explained by the independent variables for the dependent variable. the obtained r2 values were 0.685, 0.672, 0.517 and 0.392 for continuance intention, performance expectancy, satisfaction and trust, respectively. performance expectancy, effort expectancy, trust and satisfaction together explained 68.5% of the variance in the dependent variable: continuance intention to use mobile payment systems. the ecm model accounted for 51.7% of the variance explained in continuance intention to use mobile payment systems. effort expectancy explained 67.2% of the variance in performance expectancy; whereas perceived security explained 39.2% of the variance in trust. australasian journal of information systems singh 2020, vol 24, research article explaining post-adoption of mobile payment systems 15 hypotheses path path coefficient p-values result h1 pe → ci 0.129** 0.004 accepted h2 ee → ci 0.12* 0.035 accepted h2a ee → pe 0.576*** 0.000 accepted h3 ps → ci 0.001 0.988 rejected h3a ps → trust 0.626*** 0.000 accepted h4 trust → ci 0.167* 0.01 accepted h5 sat → ci 0.518*** 0.000 accepted h6 conf → pe 0.305*** 0.000 accepted h7 conf → sat 0.572*** 0.000 accepted h8 pe → sat 0.189** 0.002 accepted *p < 0.05 ; **p < 0.01 ; ***p < 0.001 table 5: path coefficients note: n. s: not significant; *p<0.05; **p<0.01; ***p<0.001 dotted line indicates non-significant path figure 3: structural model cohen’s f2 was calculated (cohen, 1988) to assess the effect size for all significant paths. the suggested values for f2 are 0.02 for small, 0.15 for medium and 0.35 for large. to assess an exogenous construct’s contribution to an endogenous variable’s q2 value, q2 was calculated. the q2 values of 0.02, 0.15 and 0.35 indicate that an exogenous construct has small, medium and large predictive relevance to its endogenous construct, respectively. table 6 shows the effect size and predictive relevance. australasian journal of information systems singh 2020, vol 24, research article explaining post-adoption of mobile payment systems 16 path f2 effect size q2 effect size pe → ci 0.019 small 0.006 small ee → ci 0.016 small 0.004 small ee → pe 0.524 large 0.186 medium trust→ ci 0.029 small 0.011 small sat → ci 0.34 medium 0.138 medium conf → pe 0.143 small 0.054 small conf → sat 0.335 medium 0,207 medium pe → sat 0.037 small 0.023 small table 6: theoretical effect size for f2 and q2 stone-geisser’s q2 (geisser, 1975; stone, 1974) value was computed to cross-validate the predictive relevance of the model by running the blindfolding technique with an omission distance of 7. the q2 values obtained for the endogenous variables continuance intention, performance expectancy, satisfaction and trust were 0.465, 0.426, 0.396 and 0.252, respectively. these are greater than zero, indicating that the model has predictive relevance (hair, hult, ringle, & sarstedt, 2016). 5.4 mediation analysis to assess the mediation effect in the theoretical model, the bootstrapping method was used. this method does not make any assumption about the sampling distribution, and it works well with small sample sizes (hair et al., 2016). the variance accounted for (vaf) was used with the following conditions (hair et al., 2016) to examine the mediation analysis. vaf = indirect effect / total effect if the vaf value is less than 0.2, there is no mediation; if the value is greater than or equal to 0.2 and less than or equal to 0.8, then there is partial mediation; and if the value is greater than 0.8, there is full mediation (hair et al., 2016). the vaf value was calculated (table 7) to determine the mediation effect of the variable satisfaction in the relationship between performance expectancy and continuance intention to use mobile payment systems. the vaf value obtained was 0.43, indicating that satisfaction partially mediated the relationship between performance expectancy and continuance intention. for the mediation effect of satisfaction on the relationship between effort expectancy and continuance intention, the vaf value was 0.22, showing partial mediation. satisfaction fully mediated the relationship between confirmation and continuance intention to use mobile payment systems. performance expectancy partially mediated the relationship between effort expectancy and continuance intention, and fully mediated the relationship between effort expectancy and satisfaction. trust fully mediated the relationship between perceived security and continuance intention. path vaf mediation ee→pe→ci 0.381 partial conf → pe → sat → ci 0.08 nil ee → pe → sat → ci 0.22 partial conf → pe → sat 0.09 nil conf→sat→ci 0.8 full ee→pe→sat 1 full ps→trust→ci 1 full table 7: mediation analysis australasian journal of information systems singh 2020, vol 24, research article explaining post-adoption of mobile payment systems 17 6 discussion many studies have examined the initial acceptance of mobile payment systems by users and also various antecedents to initial adoption. however, few studies have addressed postadoption behaviour with respect to these systems. the current study addressed this research gap by investigating post-adoption behaviour towards mobile payment systems using a theoretical framework combining the utaut and the ecm. this framework used a quantitative, cross-sectional design, with mobile payment users as the target respondents. this study combined the utaut and the ecm in the integrated model along with two additional constructs: perceived security and trust. the model showed strong empirical support for predicting continuance intention to use mobile payment systems, with an r2 value of 0.685. the empirical result shows that the integrated model has a high predictive power to explain the continuance intention to use mobile payment systems. using the utaut framework, both performance expectancy and effort expectancy were strong predictors of continuance intention. performance expectancy, effort expectancy, trust and satisfaction together explained 68.5% of the variance in the dependent variable: continuance intention to use mobile payment systems. thus, performance expectancy and effort expectancy are essential predictors not only for initial adoption but also for repeated usage of mobile payment systems. the performance expectancy construct clearly had a significant effect on postadoption expectations through satisfaction as well as acting as a direct antecedent to continuance intention. performance expectancy is an integrated construct derived from many adoption theories, and it has considerable explanatory power in initial adoption behaviour (venkatesh et al., 2003; venkatesh et al., 2012). through the ecm model, this study empirically demonstrated that performance expectancy represents post-adoption expectations. this implies that the performance of the mobile payment system during the post-adoption phase significantly influences users to continue using it. users emphasise the importance of usability as a key factor in the success of any mobile payment system. furthermore, mobile payment systems should be convenient, meet customers’ needs and be available anytime, anywhere. prior studies have shown that performance expectancy is a significant predictor of the initial adoption of mobile payment systems (abrahão, moriguchi, & andrade, 2016), and this study confirmed this finding: performance expectancy is a significant predictor, even for continued usage of mobile payment systems. these findings are consistent with zhou (2014), who also showed that performance expectancy affects continuance intention to use mobile payment systems. in this study, effort expectancy was a significant predictor of post-adoption behaviour and a direct antecedent to performance expectancy, explaining 67.2% of the variance. this suggests that customers are looking for user-friendly mobile payment systems in the post-adoption stage. prior studies have also demonstrated a linkage between effort expectancy and performance expectancy (tam, santos, & oliveira, 2018). thus, promotional strategies are helpful only in the initial adoption of mobile payment systems, but it is the user-friendliness of the mobile payment system that encourages repeated usage. moreover, the effort to use a mobile payment system will improve performance expectations at the post-adoption stage. this study identified key predictors of continuance intention to use mobile payment systems: performance expectancy, effort expectancy, satisfaction and trust. among these factors, satisfaction had the highest impact on continuance intention, followed by trust. these results validate that satisfied customers are willing to continue using their mobile payment systems. australasian journal of information systems singh 2020, vol 24, research article explaining post-adoption of mobile payment systems 18 the competition in the mobile payment service industries is intense, and users of mobile payment systems have a multitude of choices. hence, service providers must ensure that users are satisfied with the mobile payment system, which will promote loyalty to the system. past studies have also pointed out that satisfaction promotes continuance intention (cao et al., 2018; zhou, 2013). trust is the second most important factor in the decision to continue using mobile payment systems. this result underscores the importance of the users’ trust expectations for their mobile payment service providers, even after they have been using the systems for a considerable period of time. perceived security was not significant for predicting continuance intention, but it was a direct antecedent to trust, explaining 39.2% of the variance. the reason for this finding might be that customers perceive that security is built into the mobile payment systems; hence, they trust these systems. in other words, customers may automatically believe that the security features of their chosen mobile payment systems are adequate for conducting basic financial transactions. this study empirically demonstrated that the ecm model can be applied in a mobile payment context to explain post-adoption behaviour. prior studies have used the ecm in other contexts, such as mobile banking (kumar, israel, & malik, 2018), websites (zhang et al., 2015) and mobile instant messaging (oghuma, libaque-saenz, wong, & chang, 2016). in this study, the ecm model accounted for 51.7% of the variance in continuance intention. furthermore, this study suggested that the ecm model is well suited for explaining customers’ post-adoption behaviour with respect to emerging technologies, such as mobile payment systems. 6.1 theoretical implications this study makes significant theoretical and practical contributions. the main theoretical contribution of the study is the integration of the utaut and the ecm into a single theoretical framework to explain post-adoption behaviour as it relates to mobile payment systems. the utaut model was extensively used in earlier studies to shed light on the initial adoption of mobile payment systems (cao & niu, 2019; teo et al., 2015). this study is among the first to show that the utaut model can be extended to post-adoption behaviour towards emerging information systems, such as those for mobile payments. this study empirically showed that trust is a significant predictor in the post-adoption phase of mobile payment systems, which involve financial transactions. hence, the present study bridges the gap by adding the constructs of trust and perceived security – vital constructs in consumer-centric is – to the utaut model. this extended model has the predictive power to explain post-adoption behaviour even in the case of a cutting-edge technology like mobile payments. furthermore, the majority of studies on post-adoption behaviour towards mobile payment systems have been in china (cao et al., 2018; chen & li, 2017; zhou, 2013). this study was conducted in a different cultural and socioeconomic context, thereby diversifying insights in the existing literature. existing studies (fan et al., 2018; oliveira et al., 2016) have suggested that refining the initial acceptance models by applying them in different countries and demographic groups, to different technologies, and with additional constructs, will strengthen the explanatory power of the models. 6.2 practical implications this study’s findings have many practical implications. first, the factors that have been found to be important in post-adoption can be exploited by financial institutions to increase the growth of mobile payment systems. the empirical results showed that satisfied customers are willing to continue with their mobile payment services, and this suggests that service australasian journal of information systems singh 2020, vol 24, research article explaining post-adoption of mobile payment systems 19 providers should focus on improving their service delivery processes to retain their customer base and survive in the highly competitive mobile payment market. also, service providers should introduce various incentives and discounts to attract customers from different segments. the findings also revealed that even in the post-adoption stage, customers place considerable importance on performance expectancy and effort expectancy factors. thus, information technology developers who create mobile payment apps should develop straightforward, user-friendly interfaces. to emphasise the usability of mobile payment systems, service providers should develop promotional strategies to generate awareness among potential customers. another important finding of the study that is applicable to practitioners is the significance of trust. for trust building, service providers should implement appropriate security mechanisms, policies and frameworks, as well as customer redressal systems. regarding this finding, demographic factors such as age, gender and income did not influence continuance intention. service providers can use these results to craft mobile payment solutions that will appeal to all segments of their customer base. 6.3 limitations and future research the present study has some limitations that have the potential to be addressed by future research. the first limitation is that the sample was drawn from a metropolitan area, whereas future studies could sample both urban and rural populations. moreover, this study used convenience sampling; thus, caution should be used when interpreting and generalising the findings. another limitation of the study is that hedonic factors were not included in the theoretical model. future research could incorporate hedonic factors into the utaut2 theoretical framework to better explain post-adoption behaviour towards mobile payment systems. finally, only a limited number of studies have addressed the post-adoption behaviour of mobile payment systems. hence, future research should focus more on theoretical frameworks and empirical findings in this understudied area. 7 conclusion technological progression in mobile phones has prompted rapid growth in mobile payment systems and usage, with e-commerce sites, in-store payments and utilities widely adopting these systems. pre-adoption of mobile payment systems has been studied extensively, but few studies have shed light on post-adoption behaviour. this study applied an integrated model combining the utaut and the ecm and identified factors such as satisfaction, trust, performance expectancy and effort expectancy. this study confirmed that the utaut model can be extended to study post-adoption behaviour towards mobile payment systems. the study’s findings have both theoretical and practical value for furthering the understanding of preand post-adoption behaviour with respect to mobile payments. references abrahão, r. d., moriguchi, s. n., & andrade, d. f. (2016). intention of adoption of mobile payment: an analysis in the light of the unified theory of acceptance and use of technology (utaut). rai revista de administração e inovação, 13(3), 221-230. doi:10.1016/j.rai.2016.06.003 adhikary, s. (2018). digital consumer profiles: how india’s most digitally savvy will shop and spend online (pp. 1-48). euromonitor international. australasian journal of information systems singh 2020, vol 24, research article explaining post-adoption of mobile payment systems 20 anderson, j. c., & gerbing, d. w. (1988). structural equation modeling in practice: a review and recommended two-step approach. psychological bulletin, 103(3), 411-423. anderson, r., & srinivasan, s. (2003). e‐satisfaction and e‐loyalty: a contingency framework. psychology & marketing,20(2), 123-138. apanasevic, t., markendahl, j., & arvidsson, n. (2016). stakeholders ' expectations of mobile payment in retail: lessons from sweden. international journal of bank marketing, 34(1), 3761. baabdullah, a., alalwan, a., rana, n., kizgin, h., & patil, p. (2019). consumer use of mobile banking (m-banking) in saudi arabia: towards an integrated model. international journal of information management, 44, 38-52. bailey, a. a., pentina, i., mishra, a. s., & ben mimoun, m. s. (2017). mobile payments adoption by us consumers: an extended tam. international journal of retail & distribution management, 45(6), 626-640. doi:10.1108/ijrdm-08-2016-0144 bhattacherjee, a. (2001). understanding information systems continuance: an expectationconfirmation model. mis quarterly, 25(3), 351. byrne, b. m. (2016). structural equation modeling with amos: basic concepts, applications, and programming (3rd ed.). routledge. cao, x., yu, l., liu, z., gong, m., & adeel, l. (2018). understanding mobile payment users’ continuance intention: a trust transfer perspective. internet research, 28(2), 456-476. cao, q., & niu, x. (2019). integrating context-awareness and utaut to explain alipay user adoption. international journal of industrial ergonomics, 69, 9-13. chen, x., & li, s. (2017). understanding continuance intention of mobile payment services: an empirical study. journal of computer information systems, 57(4), 287-298. chin, w., marcolin, b., & newsted, p. (2003). a partial least squares latent variable modeling approach for measuring interaction effects: results from a monte carlo simulation study and an electronic-mail emotion/adoption study. information systems research, 14(2), 189-217. cohen, j. (1988). statisticalpower analysis for the behavioral sciences (2’eu.). hilisuale, nj: lawrence eribaum associates. dahlberg, t., mallat, n., ondrus, j., & zmijewska, a. (2008). past, present and future of mobile payments research: a literature review. electronic commerce research and applications, 7(2), 165-181. delone, w., & mclean, e. (2003). the delone and mclean model of information systems success: a ten-year update. journal of management information systems, 19(4), 9-30. dwivedi, y. k., rana, n. p., jeyaraj, a., clement, m., & williams, m. d. (2017). re-examining the unified theory of acceptance and use of technology (utaut): towards a revised theoretical model. information systems frontiers, 21(3), 719-734. doi:10.1007/s10796-0179774-y fan, j., shao, m., li, y., & huang, x. (2018). understanding users’ attitude toward mobile payment use. industrial management & data systems, 118(3), 524-540. doi:10.1108/imds06-2017-0268 australasian journal of information systems singh 2020, vol 24, research article explaining post-adoption of mobile payment systems 21 fornell, c., & larcker, d. (1981). evaluating structural equation models with unobservable variables and measurement error. journal of marketing research, 39-50. garland, r. (1991). the mid-point on a rating scale: is it desirable. marketing bulletin,2(1), 6670. gerban, m. (2019). mobile wallet trends (rep.). retrieved may 25, 2019, from global acceptance transaction engine(gate) website: https://www.poweredbygate.com/wpcontent/uploads/2019/03/gate-mobile-wallet-trends-final-03.26.19.pdf geisser, s. (1974). a predictive approach to the random effect model. biometrika,61(1), 101107. george, d., & mallery, m. (2010). spss for windows step by step: a simple guide and reference, 17.0 update 10/e. ghezzi, a., renga, f., balocco, r., & pescetto, p. (2010). mobile payment applications: offer state of the art in the italian market. info, 12(5), 3-22. giovanis, a., assimakopoulos, c., & sarmaniotis, c. (2018). adoption of mobile self-service retail banking technologies. international journal of retail & distribution management. hair, j. f., black, w. c., babin, b. j., & anderson, r. e. (2010). multivariate data analysis. pearson higher ed. hair jr, j. f., hult, g. t. m., ringle, c., & sarstedt, m. (2016). a primer on partial least squares structural equation modeling (pls-sem). sage publications. hair, j. f., risher, j. j., sarstedt, m., & ringle, c. m. (2019). when to use and how to report the results of pls-sem. european business review, 31(1), 2-24. doi:10.1108/ebr-11-2018-0203. hair, j. f., sarstedt, m., & ringle, c. m. (2019). rethinking some of the rethinking of partial least squares. european journal of marketing, 53(4), 566-584. doi:10.1108/ejm-10-2018-0665. hampshire, c. (2017). a mixed methods empirical exploration of uk consumer perceptions of trust, risk and usefulness of mobile payments. international journal of bank marketing, 35(3), 354-369. hoque, r., & sorwar, g. (2017). understanding factors influencing the adoption of mhealth by the elderly: an extension of the utaut model. international journal of medical informatics, 101, 75-84. iman, n. (2018). is mobile payment still relevant in the fintech era?. electronic commerce research and applications, 30, 72-82. isaac, j. t., & sherali, z. (2014). secure mobile payment systems. it professional, 16(3), 36-43. josé liébana-cabanillas, f., sánchez-fernández, j., & muñoz-leiva, f. (2014). role of gender on acceptance of mobile payment. industrial management & data systems, 114(2), 220-240. kang, j. (2018). mobile payment in fintech environment: trends, security challenges, and services. human-centric computing and information sciences, 8(32), 1-16. doi:10.1186/s13673-018-0155 khalilzadeh, j., ozturk, a., & bilgihan, a. (2017). security-related factors in extended utaut model for nfc based mobile payment in the restaurant industry. computers in human behavior, 70, 460-474. australasian journal of information systems singh 2020, vol 24, research article explaining post-adoption of mobile payment systems 22 kim, c., mirusmonov, m., & lee, i. (2010). an empirical examination of factors influencing the intention to use mobile payment. computers in human behavior, 26(3), 310-322. doi:10.1016/j.chb.2009.10.013 kline, r. b. (2011). principles and practice of structural equation modeling. guilford press. kumar, a., adlakaha, a., & mukherjee, k. (2018). the effect of perceived security and grievance redressal on continuance intention to use m-wallets in a developing country. international journal of bank marketing, 36(7), 1170-1189. doi:10.1108/ijbm-04-2017-0077 kumar, r. r., israel, d., & malik, g. (2018). explaining customer’s continuance intention to use mobile banking apps with an integrative perspective of ect and self-determination theory. pacific asia journal of the association for information systems,79-112. lee, j., cho, c., & jun, m. (2011). secure quick response-payment (qr-pay) system using mobile device. in 13th international conference on in advanced communication technology (icact) (pp. 1424-1427). ieee. li, h., liu, y., & heikkilä, j. (2014). understanding the factors driving nfc-enabled mobile payment adoption: an empirical investigation. pacis, 231-244. lu, j., wei, j., yu, c., & liu, c. (2016). how do post-usage factors and espoused cultural values impact mobile payment continuation? behaviour & information technology, 36(2), 140-164. doi:10.1080/0144929x.2016.1208773 mayer, r., davis, j., & schoorman, f. (1995). an integrative model of organizational trust. academy of management review, 20(3), 709-734. mcknight, d. h., choudhury, v., & kacmar, c. (2002). developing and validating trust measures for e-commerce: an integrative typology. information systems research, 13(3), 334-359. mobile payments world. (2019). retrieved may 19, 2019, from https://www.mobilepaymentsworld.com/ musa, a., khan, h. u., & alshare, k. a. (2015). factors influence consumers' adoption of mobile payment devices in qatar. international journal of mobile communications, 13(6), 670. doi:10.1504/ijmc.2015.072100 oghuma, a. p., libaque-saenz, c. f., wong, s. f., & chang, y. (2016). an expectationconfirmation model of continuance intention to use mobile instant messaging. telematics and informatics,33(1), 34-47. oliveira, t., thomas, m., baptista, g., & campos, f. (2016). mobile payment: understanding the determinants of customer adoption and intention to recommend the technology. computers in human behavior, 61, 404-414. oliver, r. l. (1997). satisfaction a behavioral perspective on the consumer. oxfordshire: routledge. ondrus, j., & pigneur, y. (2006). towards a holistic analysis of mobile payments: a multiple perspectives approach. electronic commerce research and applications, 5(3), 246-257. orr, g. (2010, april). a review of literature in mobile learning: affordances and constraints. in 2010 6th ieee international conference on wireless, mobile, and ubiquitous technologies in education (pp. 107-111). ieee. australasian journal of information systems singh 2020, vol 24, research article explaining post-adoption of mobile payment systems 23 phonthanukitithaworn, c., sellitto, c., & fong, m. (2015). user intentions to adopt mobile payment services: a study of early adopters in thailand. journal of internet banking and commerce, 20(1), 1-29. podsakoff, p., mackenzie, s., lee, j., & podsakoff, n. (2003). common method biases in behavioral research: a critical review of the literature and recommended remedies. journal of applied psychology, 88(5), 879-903. ramadan, r., & aita, j. (2018). a model of mobile payment usage among arab consumers. international journal of bank marketing, 36(7), 1213-1234. doi:10.1108/ijbm-05-2017-0080 ringle, c. m., silva, d. d., & bido, d. d. (2014). structural equation modeling with the smartpls. revista brasileira de marketing,13(02), 56-73. schierz, p. g., schilke, o., & wirtz, b. w. (2010). understanding consumer acceptance of mobile payment services: an empirical analysis. electronic commerce research and applications, 9(3), 209-216. doi:10.1016/j.elerap.2009.07.005 shang, d., & wu, w. (2017). understanding mobile shopping consumers’ continuance intention. industrial management & data systems, 117(1), 213-227. stone, m. (1974). cross-validatory choice and assessment of statistical predictions (with discussion). journal of the royal statistical society: series b (methodological),38(1), 111-147. susanto, a., chang, y., & ha, y. (2016). determinants of continuance intention to use the smartphone banking services. industrial management & data systems, 116(3), 508-525. tam, c., santos, d., & oliveira, t. (2018). exploring the influential factors of continuance intention to use mobile apps: extending the expectation confirmation model. information systems frontiers,1-15. tellez isaac, j., & sherali, z. (2014). secure mobile payment systems. it professional, 16(3), 3643. teo, a., tan, g., ooi, k., hew, t., & yew, k. (2015). the effects of convenience and speed in m-payment. industrial management & data systems, 115(2), 311-331. venkatesh, v., thong, j., & xu, x. (2012). consumer acceptance and use of information technology: extending the unified theory of acceptance and use of technology. mis quarterly, 36(1), 157-178. venkatesh, v., morris, m., davis, g., & davis, f. (2003). user acceptance of information technology: toward a unified view. mis quarterly, 27(3), 425-478. wang, y., hahn, c., & sutrave, k. (2016). mobile payment security, threats, and challenges. 2016 second international conference on mobile and secure services (mobisecserv)ieee, 1-5. doi:10.1109/mobisecserv.2016.7440226. yuan, s., liu, y., yao, r., & liu, j. (2014). an investigation of users’ continuance intention towards mobile banking in china. information development, 32(1), 20-34. doi:10.1177/0266666914522140 zhang, h., lu, y., gupta, s., & gao, p. (2015). understanding group-buying websites continuance. internet research, 25(5), 767-793. australasian journal of information systems singh 2020, vol 24, research article explaining post-adoption of mobile payment systems 24 zhou, t. (2014). understanding the determinants of mobile payment continuance usage. industrial management & data systems, 114(6), 936-948. zhou, t. (2013). an empirical examination of continuance intention of mobile payment services. decision support systems, 54(2), 1085-1091. australasian journal of information systems singh 2020, vol 24, research article explaining post-adoption of mobile payment systems 25 appendix a: construct and their sources constructs items source continuance intention to use(ci) ci1 i plan to use mobile payment services more often. chen & li (2017) ci2 mobile payment services currently are and also will be one of my commonly used payment methods. ci3 i intend to continue using mobile payment services. ci4 mobile payments are growing, and are expected to continue. satisfaction(sat) sat1 i am delighted with my overall experience with mobile payment services. kumar, israel, & malik(2018) sat2 i feel contented about my overall experience with mobile payment services. sat3 i am satisfied with my overall experience with mobile payment services. confirmation(conf) conf1 my experience with mobile payment services was better than what i expected. bhattacherjee (2001) conf2 the various features of mobile payment services was better than what i expected. conf3 overall, most of my expectations towards mobile payment services were confirmed. performance expectancy(pe) pe1 i would find that mobile payment services are useful for me. venkatesh et al. (2003) australasian journal of information systems singh 2020, vol 24, research article explaining post-adoption of mobile payment systems 26 pe2 using the mobile payment services enables me to accomplish tasks quickly compared to traditional payment systems. pe3 using the mobile payment services increases my productivity. pe4 overall i will find mobile payment services useful in my payment transactions. effort expectancy(ee) ee1 i would find it easy to use mobile payment services to accomplish my payment. venkatesh et al. (2003) ee2 my interaction with mobile payment services would be clear and understandable. ee3 it would be easy for me to become skillful of using mobile payment services. ee4 learning to operate the mobile payment services is easy for me. perceived security(ps) ps1 the risk of an unauthorized third party overseeing the payment process is low. schierz, schilke, & wirtz (2010) ps2 the risk of abuse of usage information (e.g., names of business partners, payment amount) is low when using mobile payment services. ps3 the risk of abuse of billing information (e.g., credit card number, bank account data) is low when using mobile payment services. australasian journal of information systems singh 2020, vol 24, research article explaining post-adoption of mobile payment systems 27 ps4 i would find mobile payment services secure in conducting my payment transactions. trust(tru) tru1 i believe the technology of mobile payment services is secured. chen & li (2017) tru2 i believe the technology of mobile payment services is reliable. tru3 i believe the technology of mobile payment services is worthy of trust. tru4 i believe mobile payment services providers would keep their commitments. tru5 i believe mobile payment services providers would act in users’ best interest. tru6 i would believe mobile payment services providers as honest. copyright: © 2020 singh. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v24i0.2695 http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems singh & dev 2023, vol 27, research article ict-driven work engagement interventions 1 ict-driven work engagement interventions in work-fromhome: the mediating role of the need for relatedness harshit kumar singh indian institute of management rohtak, india harshitk.singh@iimrohtak.ac.in vedant dev amrut mody school of management, ahmedabad university, india abstract during the covid-19 pandemic, organizations across the world implemented mandatory work-from-home policy. the policy had a detrimental impact on the work engagement of the employees as there was a prolonged dependence on ict-mediated communication for all interactions. in response, organizations had to implement ict-driven interventions to address the problem of dwindling work engagement levels. in view of enhancing intervention effectiveness, this research examines the satisfaction of need for relatedness as the psychological mechanism underlying the relationship between ict-driven interventions and intellectual, social and affective dimensions of work engagement. we develop and test our mediation model in a cross-organizational study of 167 employees. results show that needfor-relatedness mediates the relationship between intervention intensity in fostering interactions and work engagement of the employees. in addition, the perceived social support in the team negatively moderates the relationship between intervention intensity and the satisfaction of need for relatedness. our results have implications for designing and implementing ict-driven interventions in organizations planning large-scale work-fromhome adoption. keywords: work-from-home, ict-driven interventions, work engagement, selfdetermination theory, need for relatedness, perceived social support. 1 introduction during covid-19, many organizations, particularly in the knowledge industry, adopted the mandatory work-from-home (wfh) policy using ict-mediated communication (kniffin et al., 2021). however, the prolonged dependence on ict-mediated communication resulted in a decrease in employee work engagement levels (mosquera et al., 2022; singh & verma, 2021). in response, many organizations have undertaken ict-driven work engagement interventions to address the problem of engagement (fallon, 2020). given the massive scale and the resource intensiveness of such interventions, there is a need for research on ways to improve the effectiveness of interventions. in this direction, this study shows that the intervention effectiveness on work engagement is contingent on the extent of fulfilment of the basic psychological need for relatedness of the employees. humans have an innate basic psychological need for relatedness for optimal functioning in all domains of life, including work (baumeister & leary, 1995; ryan & deci, 2000). communication at the workplace plays an important role in the satisfaction of need for relatedness. one of the prominent modes of workplace communication today is through ictmediated communication. however, ict-mediated communication does not fulfill the need for relatedness to the same extent as face-to-face interaction (sacco & ismail, 2014). mailto:harshitk.singh@iimrohtak.ac.in australasian journal of information systems singh & dev 2023, vol 27, research article ict-driven work engagement interventions 2 ict-mediated communication has gained mainstream importance for organizations in the context of covid-19 pandemic. during covid-19, governments in many countries mandated business and industry lockdowns (chong et al., 2020; kniffin et al., 2021, lim, 2021). faced with the challenge of business continuity, organizations asked their employees to undertake mandatory work from home (wfh). the impact of wfh adoption is far-reaching to the extent that some jobs are predicted to go remote permanently (stych & greer, 2020). wfh is a special case of telework wherein employees are mandated to work from home due to social distancing rules in the pandemic. in wfh, employees work only from home, while telework includes other remote work settings such as while travelling or on vacation. traditionally, telework involves the choice of the employee, whereas mandatory wfh means that employees do not have a choice due to changed organizational policy. before the pandemic, telework was not used extensively practice (kossek & lautsch, 2017). this is because telework entails a perceived lack of control for the managers on the teleworking employees and, thus, the corresponding monitoring costs. further, telework was an option given to employees who could benefit from the policy. thus, while telework does come with advantages of improved job autonomy and work-life balance (allen et al., 2015), employees forced to work from home are faced with unique challenges of social distancing and loneliness (kniffin et al., 2021). to this end, there is a need to review the current telework literature in the context of covid-19 and wfh (wang et al., 2021). in wfh, the social interaction takes place entirely through ict-mediated communication channels. here, ict serves a social function wherein it influences the employees by affecting their interactions and social communications (wang et al., 2020). the quality of communication in ict-mediated communication is inferior in comparison to face-to-face communication (walther, 2011; walther et al., 2015). this phenomena was acutely evident during the covid-19 pandemic (el-zoghby et al., 2020; mosquera et al., 2022). as a result, wfh employees face a loss of social connections and social support. we argue that the overdependence on ict-mediated communication in wfh has led to a perceived loss of social support, which has essentially thwarted the satisfaction of need for relatedness of the employees, causing a negative impact on their work engagement. in response to the dwindling engagement levels of the wfh employees, many organizations have undertaken ict-driven work engagement interventions which intend to enable and foster social interactions (fallon, 2020; singh & verma, 2021). a variety of interventions have been used in organizations such as town hall meetings, virtual catchups, online games and quizzes (newman & ford, 2021). the overarching research objective of this study is to explore the psychological mechanism underlying ict-driven interventions and individual-level work engagement. this knowledge can then be used to make these interventions more effective in addressing the loss of engagement. we examine the psychological mechanism underlying the relationship between ict-driven interventions and the work engagement of employees. according to selfdetermination theory (sdt), the need for relatedness is one of the basic psychological needs for the optimal functioning of individuals (ryan & deci, 2017). we contend that the effectiveness of ict-driven work engagement interventions in wfh is contingent on the satisfaction of need for relatedness of employees. in other words, the satisfaction of need for relatedness mediates the relationship between the interventions and work engagement. australasian journal of information systems singh & dev 2023, vol 27, research article ict-driven work engagement interventions 3 overall, we examine the mediating role of satisfaction of need for relatedness, and moderating role of perceived social support on the effect of ict-driven work engagement interventions on the satisfaction of need for relatedness of wfh employees. the focus is on full-wfh employees as ict-driven interventions were implemented by organizations during covid19 period wherein wfh was mandatory. we present a conceptual framework that can help organizations prepare and counter the challenge of the lack of relatedness while working from home and improve the engagement levels of employees. 2 theoretical background and hypotheses the advancement of icts and availability of high internet speed are the key drivers of the rise of teleworking (allen et al., 2015). ict enabled the quick switch to wfh in the wake of business lockdowns due to covid-19 pandemic (kniffin et al., 2021). in turn, wfh has become the dominant context that extends the effect of ict usage on work characteristics and employee outcomes. ict usage in telework has two roles: (1) human-ict interaction for accomplishing task aspects (e.g., using google/git-hub to address technical aspects of work) and (2) ict-mediated communication for addressing social and relational aspects of work (e.g., using whatsapp/signal/telegram to connect with colleagues) (wang et al., 2020). ict-mediated communication refers to the use of ict medium for interactions and social connections with others at work, rather than for accomplishing tasks. in wfh scenario, ict-mediated communication has a major impact on relational aspects of work as all the social and interpersonal interactions are mediated through icts. ict-mediated communication is associated with weakening of social relations (siampou et al., 2014; wang et al., 2020). this is because the employee interaction through ict-mediated communication suffers from lack of important social cues such as body language, contextual information, and other non-verbal cues (walther, 2011; walther et al., 2015). overall, there is a general concern of loss of social support in telework due to a decrease in level of interactions with others (olszewski & mokhtarian, 1994; sahoo et al., 2022). these problems have become acute due to the mandatory nature of wfh wherein employees are forced to use ict-mediated communication for all their work interactions. in sum, the prolonged dependence on ictmediated communication for all work-related interactions had led to the loss of social support and increased need for relatedness. 2.1 need for relatedness human beings have an innate, powerful, and fundamental need for relatedness, which refers to a desire for meaningful and significant interpersonal relationships (baumeister & leary, 1995). according to sdt, the optimal functioning of human beings, in terms of work performance and well-being, is contingent on the satisfaction of three basic psychological needs autonomy, competence and relatedness (deci et al., 2017, ryan & deci, 2000). the satisfaction of need for relatedness is contingent on two criteria. first, there should be frequent interactions with the relevant people. second, individuals should perceive that the relationships are stable and enduring in the foreseeable future, and that there is authentic affective concern in the interpersonal relations. the study focusses on the satisfaction of need for relatedness as it is affected in the pandemic era. australasian journal of information systems singh & dev 2023, vol 27, research article ict-driven work engagement interventions 4 according to sdt, the need for relatedness is extremely pervasive and it exists in all domains of life including work (ryan & deci, 2017). this is because work is an important domain in many people’s lives as they spend significant time and energy at work. in simple words, individuals seek to satisfy the need for relatedness in work domain. this need gets satisfied when people feel as a part of a group, feel connected to significant others, and are able to build close relations (van den broeck et al., 2016). it has been found to be responsible for prosocial tendency, and fulfilment of this need leads to emotional well-being (callea et al., 2019). in this article, we contend that the underlying reason for the weakening of social relations in wfh is the thwarting of the satisfaction of need for relatedness. in wfh, there is a lack of frequent social interactions and deterioration in the quality of social interactions, which leads to thwarting of the satisfaction of need for relatedness for employees. this, in turn, has an adverse impact on work engagement (deci et al., 2017). 2.2 work engagement engagement is defined as “the harnessing of organization members’ selves to their work roles; in engagement, people employ and express themselves physically, cognitively, and emotionally during role performances” (kahn, 1990; p. 694). a fully engaged employee shows psychological presence in the role, which involves being attentive, connected, integrated, and focussed on the performance of the role (saks, 2017). while engagement has been a top priority for organizations, it has become a critical success factor during the pandemic (lim, 2022). while there has been research attention to telework and engagement, there has been little progress on work engagement in teleworkers. the limited research on the topic indicates that telework has a negative relationship with social support and work engagement (sardeshmukh et al., 2012). 2.3 need for relatedness and work engagement according to sdt, when the basic psychological needs of an employee is satisfied, it leads to more autonomous forms of work motivation and this consequently has a positive impact on work performance, work engagement and well-being; on the other hand, when the satisfaction of basic psychological needs is thwarted, either due personal factors or environmental factors, the motivation is more controlled rather than autonomous, and there is negative spill over effect on employee performance and work engagement (deci et al., 2017). in the covid-19 context, when the satisfaction of need for relatedness of the employees is thwarted due to mandatory wfh, social distancing restriction, and prolonged dependence on ict-mediated communication, there is a negative impact on the employee work engagement. this decrease in work engagement in covid-19 is widely observed phenomena and organizations are attempting to improve the engagement levels through ict-driven work engagement interventions. an integral feature of engagement is the aspect of connectedness with work and with relevant others at work (e.g., team members, supervisors). the aspect of connectedness is brought out explicitly in the isa model of engagement (soane et al., 2012). in this model, work engagement is viewed as construct with three facets – intellectual, social, and affective engagement. isa engagement model is the operationalization of the need-satisfaction framework of engagement wherein employee’s engagement depends on the availability of psychological resources like meaningfulness, psychological safety and psychological availability (anthonymcmann et al., 2017). the advantage of using the isa model is that in addition to the wellaustralasian journal of information systems singh & dev 2023, vol 27, research article ict-driven work engagement interventions 5 identified cognitive and affective aspects of engagement, this model also focusses on the social dimension of engagement, which reflects the importance of presence of positive relations at work (toth et al., 2022). intellectual engagement refers to the cognitive dimension of engagement. one of the primary requirements of an employee being engaged in a work role is that the person applies oneself cognitively in performing the work. the intellectual component of engagement captures the extent to which the individual can apply one’s cognitive resources towards the task performance in the given work role. an increase in the satisfaction of need for relatedness leads to positive affect (baumeister & leary, 1995). increased positive affect in turn enables the usage of cognitive resources in the work role. hence, we argue that there is a positive impact of the satisfaction of need for relatedness on intellectual engagement. h1a: there is a direct positive relationship between the satisfaction of need for relatedness and intellectual engagement. social engagement refers to the connectedness aspect of engagement. connectedness with relevant others in the work environment is a central feature of the work engagement (kahn, 1990). for instance, relationships with supervisors and team members have been considered as the antecedents of engagement (saks, 2006). the ways in which the employee interacts with team members and others in the workplace influences the social engagement at work. work in today’s world entails extensive interaction and collaboration (toth et al., 2022). hence, a key challenge that a modern workforce faces is related to sustaining relationships (ashford et al., 2018). hence, social engagement is a vital facet of work engagement. an increase in satisfaction of the need for relatedness indicates the strengthening of interpersonal relationships with relevant others, like supervisor and team-members (baumeister & leary, 1995). hence, we argue that an increase in satisfaction of the need for relatedness has a direct positive impact on social engagement. h1b: there is a direct positive relationship between the satisfaction of need for relatedness and social engagement. affective engagement refers to the positive affect requirement of engagement. positive affect enables an individual to better utilize their physical, intellectual, and social resources (fredrickson, 1998). the satisfaction of need for relatedness makes the individual feel connected to relevant others. an increase in relatedness leads to positive affect, while a decrease is linked to negative affect (baumeister & leary, 1995). thus, we argue that an increase in satisfaction of the need for relatedness leads to positive affect, which in turn has a positive impact on the affective engagement. h1c: there is a direct positive relationship between the satisfaction of need for relatedness and affective engagement. 2.4 ict-driven work engagement interventions in response to employee issues in wfh, organizations are attempting to address the work engagement by implementing a host of ict-driven work engagement interventions (fallon, 2020; singh & verma, 2021). ict-driven work engagement interventions have been categorized based on their effects on the individuals (hedonic and/or eudaimonic effect), and on the interpersonal (social) aspects (riva et al., 2012). we focus on interpersonal effect of ictdriven work engagement interventions, wherein the intervention is directed to enhance the interactions and the connectedness of people in the organization. australasian journal of information systems singh & dev 2023, vol 27, research article ict-driven work engagement interventions 6 the interpersonal aspect of work engagement interventions holds relevance in the current wfh context. the category comprises of those interventions which focus on facilitating connectedness between employees on individual, team, as well as organizational level. a primary challenge in the implementation of such interventions is to create a strong sense of community while working with spatial distancing (riva et al., 2012), which was not easy even in the pre-pandemic era. in wfh, the intensity of the challenge has increased drastically, given the physical isolation of employees from each other and the organization. the social and interpersonal interventions try to mitigate the effect of physical separation by focusing on decreasing emotional and psychological separation. the evidence of these interventions can be seen in various forms, such as town hall meetings, virtual catchups, online games and quizzes (newman & ford, 2021). periodic virtual informal interactions between employees have been found to increase the relatedness among employees (singh & verma, 2021). these can be done through activities (or initiatives) like weekend get-together, catchup meetings, or virtual drinks on tools such as zoom and other similar channels. another instance is to ensure effective management communication through these interventions which increase job satisfaction in teleworkers (ilozor et al., 2001). this can be done through initiatives such as virtual town hall meetings, where the organizations leadership addresses the employees. other initiatives include health related periodic virtual events or sending regular updates to employees relating to health tips. these have been found to motivate employees (greasley et al., 2012). organizing online gaming events on different platforms or even simpler ones on email threads or other communication channels such as whatsapp can also help in increasing employees to connect with their co-workers (ellis et al., 2008). signing-up employees to wellness programs and courses on different online platforms to boost mental health are also part of such interventions (gubler et al., 2017). organizations employ techniques such as story-sessions where employees meet virtually to discuss and share personal and lived experiences which contribute to wellbeing at workplace (king et al., 2020). the interventions being attempted by organizations are aimed to increase the social interactions with relevant others (singh & verma, 2021). in wfh, where the employee is physically separated from the workplace and the co-workers, such an increase in social interactions tends to enhance their perception of social support available in the workplace. further, the interventions where focus is on increasing social support are likely to improve the satisfaction of need for relatedness (van den broeck et al., 2008). hence, we argue that intervention intensity has positive impact on the satisfaction of need for relatedness. h2: there exists a direct positive relationship between intervention intensity and the satisfaction of need for relatedness. in theory, such interventions drive work engagement through changes in job resources and personal resources (bakker & demerouti, 2007, 2017; halbesleben, 2010). the changes in job and personal resources, and the satisfaction of work-related needs have been proposed to mediate the relationship between work engagement interventions and work engagement (knight et al., 2019). since interventions are directed towards improvement of work engagement, intervention intensity has a positive impact on the satisfaction of need for relatedness, and the satisfaction of need for relatedness has a positive impact on intellectual, social, and affective dimensions of engagement, we contend that the satisfaction of need for australasian journal of information systems singh & dev 2023, vol 27, research article ict-driven work engagement interventions 7 relatedness mediates the relationship between intervention intensity and dimensions of work engagement. h3a: the satisfaction of need for relatedness mediates the relationship between intervention intensity and intellectual engagement. h3b: the satisfaction of need for relatedness mediates the relationship between intervention intensity and social engagement. h3c: the satisfaction of need for relatedness mediates the relationship between intervention intensity and affective engagement. 2.5 perceived social support social support is essential in the optimal functioning of employees at work. it has been shown to be an important job resource at the workplace (bakker et al. 2014) and it plays a significant role in the satisfaction of need for relatedness (fernet et al., 2013) as discussed earlier, the excessive usage of ict-mediated communication in wfh leads to the weakening of social support. organizations intend to bolster social support through work engagement interventions as it is pivotal in long-term wfh implementation (khor & tan, 2022). the perceived social support enables desirable online social interactions which tends to satisfy their need for relatedness. according to bavik et al. (2020), social support tends to enhance social interactions over ict, leading to satisfaction of the need for relatedness. in the pandemic, social support has been a crucial element in helping the employees deal with the challenges of wfh. if the employee already perceives strong social support in the team, the need for relatedness is fulfilled. in this case, the role of interventions in satisfaction the need for relatedness reduces. hence, we hypothesize: h4: perceived social support negatively moderates the relationship between intervention intensity and the satisfaction of need for relatedness. figure 1 below presents the conceptual model. figure 1. conceptual model 3 method we have theorized that the impact of ict driven work engagement interventions on employee work engagement would depend on the satisfaction of need for relatedness and perceived australasian journal of information systems singh & dev 2023, vol 27, research article ict-driven work engagement interventions 8 social support of the employee. we are using cb-sem as these are unobserved conceptual constructs and we are trying to estimate mediation effects (hair & sarstedt, 2019). the constructs have been measured through standardized scales from existing literature, wherein they are conceptualized as latent factors rather than composites (hair & sarstedt, 2019). 3.1 sample and data collection the study required employees who were in organizations with mandatory wfh policy. towards this end, convenience sampling was used wherein the participants were identified through professional networks of the researchers. the data was collected by sending emails to over 800 professionals working in organizations that have implemented wfh. the participants primarily belonged to the individual contributor and middle management roles. they were contacted in the month of june 2020, at a time when covid-19 pandemic was at peak, state policy of social distancing was enforced, and organizations were using mandatory wfh policy. the email listed the objective of the study. we received a total of 167 completed responses, which provided a response rate of 20.88%. this sample size is considered adequate for structural equation modelling (wang & wang, 2019). the demographic information of the participants is present in table 1. item category frequency ratio gender male 122 73.05% female 45 26.95% age 20-30 48 28.74% 30-40 93 55.69% 40-50 18 10.78% 50+ 8 4.79% education post-graduate 165 98.80% higher 2 1.20% total work experience <5 34 20.36% 5-15 92 55.09% 15+ 41 24.55% experience in the current firm <5 116 69.46% 5-15 48 28.74% 15+ 3 1.80% wfr experience regularly 13 7.78% often 47 28.14% sometimes 49 29.34% rarely 44 26.35% never 14 8.38% co-occupants 0 8 4.79% 1-3 117 70.06% 4-7 37 22.16% 7+ 5 2.99% table 1. demographic characteristics of the respondents australasian journal of information systems singh & dev 2023, vol 27, research article ict-driven work engagement interventions 9 the sectoral composition of the participants was: it services 31.37%, it consulting 15.69%, banking and insurance 11.76 %, energy 5.88% and others (telecommunication, healthcare, retail, ngo etc.) 35.30 %. we also collected data on the participants’ previous telework experience and the number of people with whom they share their homes as these factors are likely to influence the employees satisfaction of need for relatedness and engagement levels (kniffin et al., 2021). the questionnaire was prepared on the qualtrics software and shared with the participants through an anonymous link to maintain their privacy and confidentiality. the questionnaire measured the responses on a 7-point scale where participants marked 1 for “strongly disagree” to 7 for “strongly agree”. the 7-point likert scale was used to enable more variance among responses (podsakoff et al., 2003). 3.2 variable measurement in this section, we discuss the variables used to operationalize the constructs in our study. intervention intensity captures the strength of the ict-driven work engagement initiatives targeted towards developing relationships among co-workers. to measure this, we adapted from the scale developed by nielsen et al. (2000). organizations had implemented the interventions to increase the social interactions at the team level which had been reduced to a great extent due covid-19 enforced work from home policy. thus, in simpler terms, the aim of interventions was to provide opportunity for social interactions at the team level. the scale by nielson et al. (2000) is used for is used to measure the opportunity for developing relationships at the workplace and contains three items. this scale was adapted by us to measure the opportunity for social interactions at work from home. representative items include “my organization encourages communication amongst employees”. to measure perceived social support, we adapted from the four-item scale by liu et al. (2011). a representative item from the scale is “my team is supportive of each team members’ individual perspectives.” to measure the satisfaction of need for relatedness, we adapted from the sixitem scale by sheldon et al. (2001). a representative item from the scale is “at work, i can talk with people about things that really matter to me.” to measure work engagement, we have used the scale developed by soane et al. (2012). the scale has nine items, with three items for each dimension of the work engagement variable intellectual engagement, social engagement, and affective engagement. representative items include “i pay a lot of attention to my work”, “i share the same work values as my colleagues”, and “i feel positive about my work”. the first three control variables in our study are age, gender, and education. we also controlled for total work experience as it can affect the individual capabilities to manage the social relations at the workplace. further, we controlled for the experience in the current firm as it can affect the social relationships formed in the organizations. long-term interaction with coworkers can increase the opportunity to develop friendly relationships in the workplace. we further controlled for previous teleworking experience as it can affect the teleworker’s capabilities to manage workplace relationships. finally, we controlled for the co-occupants (number of people they share their home with) as the satisfaction of need for relatedness can also be affected by family or friends in the work environment’s immediate surroundings. age, work experience, tenure, and telework experience were measured as continuous variables, while number of co-occupants were measured in discrete numbers. australasian journal of information systems singh & dev 2023, vol 27, research article ict-driven work engagement interventions 10 3.3 addressing common method variance this study was susceptible to common method bias (cmb) because it is a cross-sectional correlational study wherein data was collected at one instance using online survey (podsakoff et al., 2003). we analysed the data using the common latent factor (clf) test with a specific bias construct blue marker to address this issue (fuller et al., 2016). blue attitude is 4-item construct that captures the respondent’s tendency to respond extreme (strongly agree) or moderate (agree or somewhat agree). this construct is not expected to correlate with other latent factors in the model. hence, any common variance between the factors is likely to be due to a common method bias. variable mean s.d. 1 2 3 4 5 6 7 8 9 10 11 12 13 1. age 2. gender -.17* 3. education .26** .01 4. work experience .85** .23** .08 5. tenure .26** -.04 .01 .33** 6. wfh experience -.14 .00 .04 .18* .09 7. cooccupant .17* .00 .06 .19* .16* -.02 8. intervention intensity 1.01 0.74 .07 -.12 .14 .02 .08 -.05 .00 (.70) 9. perceived social support 0.69 0.50 .03 -.10 -.09 .06 .04 -.06 -.01 .46** (.69) 10. satisfaction of need for relatedness 2.48 0.92 .06 -.06 .07 .05 .08 -.03 -.09 .78** .63** (.87) 11. intellectual engagement 2.53 0.68 .05 -.02 .05 .04 -.09 -.04 -.08 -.02 -.29** .19* (.91) 12. social engagement 1.95 0.89 .06 -.12 .00 .06 -.10 -.07 .01 .03 .31** .19* .37** (.83) 13. affective engagement 1.92 0.97 .20* -.13 .16* .18* .00 -.07 .11 .47** .26** .51** .30** .31** (.92) note. number of respondents = 167. age, work ex, tenure, wfh ex and co-occupants were measured as continuous variables. *p < 0.05, **p < 0.01. reliabilities (cronbach’s alpha) are in parenthesis. table 2. descriptive statistics we did a common method bias test (simmering et. al., 2014). here we compared the unconstrained common method factor model (chi-square = 534.826; df = 682) to the zeroconstrained common factor model (chi-square = 793.192; df = 382). the chi-square difference test between the two models was significant (p<0.001). this indicates that there is a significant cmb which needs to be accounted. then we ran the equal-constrained test to check if the cmb is evenly distributed across factors. the chi-square difference test between constrained (chi-square = 827.42; df = 387) and unconstrained model (chi-square = 774.776; df = 353) was significant indicating unevenly distributed bias. australasian journal of information systems singh & dev 2023, vol 27, research article ict-driven work engagement interventions 11 hence the latent factor scores were imputed while keeping the blue factor in the measurement model. this effectively parcels out the shared bias with the blue attitude construct. the imputed latent factors were then used in the path model. 3.4 reliability and validity of measures the reliabilities of the constructs are presented in table 2 (in parenthesis). in the reliability test, all the constructs have cronbach’s alpha value above the recommended range of 0.70 (nunnally, 1994). however, the alpha value for perceived social support lies on the boundary of the range. next, we measured the validity of the constructs using the confirmatory factor analysis (cfa). the analysis was done using the amos version 22 software. one item from perceived social support scale was dropped due to poor loading, and the reliability was improved after dropping the item against the minimum threshold value of .60 (hair et al., 2019). the reliability values shown in table 2 are after dropping the item. the new model was reanalysed to confirm the model fit. the hypothesized model provided a good fit to the data (χ2/d.f. = 1.51; n.s.1; rmsea = 0.055; cfi = 0.90, srmr = 0.06) (kline, 2015). finally, we proceeded to confirm the convergent and discriminant validity of the measures (table 3). the convergent validity of constructs intervention intensity and perceived social support are borderline. one of the possible reasons for this could be different interpretation of questions by participants. variable cr ave 1 2 3 4 5 6 1. intervention intensity .68 .42 .65 2. perceived social support .67 .40 .46 .64 3. satisfaction of need for relatedness .86 .51 .75 .64 .71 4. intellectual engagement .90 .76 .15 .04 .38 .87 5. social engagement .84 .64 .12 .37 .30 .48 .80 6. affective engagement .89 .73 .44 .31 .60 .44 .41 .85 table 3. composite reliability, average variance explained, and squared correlations 4 results the means, standard deviations and reliability and correlations between measured variables have been presented in table 2. intervention intensity was significantly positively related with the satisfaction of need for relatedness and affective engagement. further, the satisfaction of need for relatedness had significant positive correlation with the three dimensions for work engagement – intellectual, social and affective engagement. thus, these significant correlations provide a preliminary support to the hypotheses. to analyze the path model and test our hypotheses, we used amos software (version 22). the results of the structural equation modelling (sem) analysis are presented in figure 2. 1 n.s. not significant australasian journal of information systems singh & dev 2023, vol 27, research article ict-driven work engagement interventions 12 note. *p < 0.05, **p < 0.01, ***p < 0.001 figure 2. structural equation model 4.1 direct effects all our direct effect hypotheses, i.e., h1 (a, b, and c) and h2 were supported as the predicted relationships were found to be statistically significant. figure 2 details the effect size of each of our hypotheses. the standardized coefficient of the effect size of intervention intensity on the satisfaction of need for relatedness is 0.69 (p < 0.001). the satisfaction of need for relatedness affects the intellectual, social, and affective engagements at 0.19 (p < .05), 0.20 (p < .01), and 0.52 (p < .001) levels of magnitudes, respectively. the impact of the effect of control variables were found to be statistically non-significant. 4.2 interaction effects the coefficient of the interaction effect of intervention intensity and perceived social support on the satisfaction of need for relatedness is -0.20 (p < .01). the result supports the hypothesis h4. figure 3. interaction effect: intervention intensity and perceived social support on the satisfaction of need for relatedness australasian journal of information systems singh & dev 2023, vol 27, research article ict-driven work engagement interventions 13 the interaction effect between intervention intensity and perceived social support was measured through the interaction variable, which was created by standardizing all variables in the model and then computing a product variable from the interacting variables. this product variable was then used as an independent variable in amos to check if it has a statistically significant effect on the dependent variable (i.e. the satisfaction of need for relatedness). once we obtained a significant result, it was plotted. figure 3 presents the interaction plot. the interaction plot shows that the slope of the graph is higher in case of low perceived social support than in case of high perceived social support. this indicated that when perceived social support is low, interventions have a stronger impact on the satisfaction of need for relatedness. however, when perceived social support is already high, the impact of interventions on the satisfaction of need for relatedness dissipates. path effect size confidence level llci ulci hypothesis support h3a: intervention intensity → satisfaction of need for relatedness → intellectual engagement not supported h3b: intervention intensity → satisfaction of need for relatedness → social engagement .08 95% [.016 .165] supported h3c: intervention intensity → satisfaction of need for relatedness → affective engagement .32 99% [.203 .452] supported table 4. mediation effects 4.3 mediation effects to test the mediation effects, bootstrapping analysis were performed 5000 times to generate the bias-corrected bootstrap confidence interval (hair et al., 2019). mediation is statistically significant when the bootstrap confidence interval does not contain zero (zhao et al., 2010). using this criterion, the satisfaction of need for relatedness mediated the relationship between intervention intensity and social engagement (95% ci; .016, .165); and between intervention intensity and affective engagement (99% ci; .203, .452). the mediation by the satisfaction of need for relatedness between intervention intensity and intellectual engagement was not statistically significant. thus, h3 (b, c) were supported, while h3a was not supported. the results are presented in table 4. while the model is indicative of moderated mediation, i.e., the mediated relationship will be stronger at lower level of perceived social support, and weaker at higher level of perceived social support, the moderated mediation was found to be statistically non-significant. to summarize, the results support the theorized relationships: (1) the satisfaction of need for relatedness mediates the relationship between intervention intensity and work engagement (except for intellectual engagement); and the (2) perceived social support negatively moderates the relationship between intervention intensity and the satisfaction of need for relatedness. 5 discussion in the wake of the prevailing global pandemic, organizations have been forced to implement wfh. the social distancing had a debilitating impact on the informal interactions of employees, and as a result, the work engagement of the employees has been affected. we argue australasian journal of information systems singh & dev 2023, vol 27, research article ict-driven work engagement interventions 14 that the work engagement of employees is affected in wfh because of the lack of satisfaction of their need for relatedness. organizations have been swift in applying ict-driven interventions to sustain the employee interactions, with the larger goal to improve the work engagement of employees. however, the relationship between the ict-driven interventions and work engagement is a black box. in this study, we shed light on the central role of the satisfaction of need for relatedness in driving the work engagement. the satisfaction of need for relatedness is a basic psychological need of an individual to feel a sense of belongingness with relevant others at work and with the organization at large. we also explored the role of perceived social support in influencing the effectiveness of interventions on the need satisfaction. the results show that the satisfaction of need for relatedness mediates the relationship between intervention intensity, and the social and affective dimensions of work engagement. the relationship between the satisfaction of need for relatedness and affective engagement was the strongest, followed by social engagement. it would be expected that the satisfaction of need for relatedness would have the strongest relationship with social engagement because the satisfaction of need for relatedness is directly connected to social engagement. the satisfaction of need for relatedness deals with the employee's feeling of connectedness to their co-workers, while social engagement is gained through a feeling of connectedness with the work environment. the results showed that the relationship between the satisfaction of need for relatedness and intellectual engagement is significant, but the mediating effect of the satisfaction of need for relatedness on the relationship between intervention intensity and intellectual engagement is not statistically significant. a plausible reason is that the increase in intellectual engagement is subject to the cognitive resources at the disposal of the individual (soane et al., 2012). while the increased satisfaction of the need for relatedness leads to positive affect which may enable usage of cognitive resources (baumeister & leary, 1995), work engagement intervention does not impact intellectual engagement through the satisfaction of need for relatedness. as the work engagement interventions are directed at increasing the social interactions, they are unlikely to influence the usage of cognitive resources and impact the intellectual engagement. hence, the mediating effect of the satisfaction of need for relatedness has been found to be statistically insignificant in case of intellectual engagement. 5.1 theoretical contributions the key contribution of our study is the examination of the psychological mechanism underlying ict-driven interventions and work engagement. these interventions are resourceintensive, and organizations would seek to improve the return on this investment. our study provides this crucial insight into what factors influence the effectiveness of interventions. with this new knowledge, ict-driven interventions can be directed towards the satisfaction of need for relatedness to improve their effectiveness in promoting work engagement. on the other hand, if the interventions do not address the satisfaction of need for relatedness, the eventual intended effect on work engagement may be weakened. when interventions are designed keeping the satisfaction of need for relatedness factor in mind, they are likely to lead to employee interactions that are authentic and autonomous, which will have a positive impact on the perceived relatedness of the employees and their work engagement. job demand and resources (jd-r) model has been typically used to understand antecedents to work engagement (bakker & demerouti, 2007, 2017; halbesleben, 2010). it is quite broad in australasian journal of information systems singh & dev 2023, vol 27, research article ict-driven work engagement interventions 15 capturing the entire set of antecedents. however, in the context of covid-19 pandemic, there is one specific aspect that has been of primary concern – the loss of social support. it is the reduction in this job resource, i.e., social support, which has an adverse impact on work engagement of employees in wfh. keeping the satisfaction of need for relatedness at the centre of the model will help the practitioners to target their interventions more directly at the problem at hand, rather than making the interventions broader and targeting several job resources at a time. our application of sdt places the spotlight specifically on the satisfaction of need for relatedness which has been thwarted due to reduction social support. there has been a loss of social support during covid-19 and drop in work engagement (kniffin et al., 2021). the loss of social support is due to deterioration in the employee interaction (wang et al. 2020; wang et al. 2021). organizations are implementing ict driven intervention to bolster the work engagement by improvement in employee interactions (mosquera et al., 2022; singh & verma, 2021). in our paper, we have theorized that the satisfaction of need for relatedness mediates the relationship between intervention intensity and work engagement. one of the key sources of social support in the work context is the team. work teams develop a work environment over time which provides social support to its members. our construct perceived social support refers to the employee perception of social support already existing in a team. we argue that if the level of perceived social support in team is high, the satisfaction of need for relatedness is already fulfilled to an extent, and, hence, the positive impact of intervention intensity on the satisfaction of need for relatedness is low. on the other hand, if perceived social support in the team is low, the impact of intervention intensity on the satisfaction of need for relatedness is stronger. hence, we have theorized perceived social support moderates the relationship between intervention intensity and the satisfaction of need for relatedness. need for relatedness is the most recent addition to the basic psychological needs category in sdt (van den broeck et al., 2016). hence, it has received relatively less research attention in the work domain in comparison to the needs for autonomy and competence (gagné et al., 2018). however, the current covid-19 pandemic has brought the need for relatedness onto the centre stage. we find evidence that perceived social support negatively moderates the relationship between intervention intensity and the satisfaction of need for relatedness. if the employee is provided requisite social support in the work environment, the need for relatedness would be satisfied adequately. work engagement interventions are not needed in this scenario, as the need for relatedness is fulfilled. this results in the reduced effect of ict-driven interventions on the satisfaction of need for relatedness of the employees. figure 3 demonstrates the described phenomenon. at higher perceived social support, the relationship between the intervention intensity and the satisfaction of need for relatedness is weaker than a scenario of lower perceived social support. 5.2 practical implications the understanding of ict-driven interventions is important for practitioners in humancomputer interaction (hci). in the pre-covid era, most workplace interactions were face-toface. however, in wfh, the non-verbal cues and contextual information of the interacting participants that were obvious in the offline interactions are now missing in the ict-mediated interactions. the understanding of virtual interactions needs to evolve to make up for this absence. australasian journal of information systems singh & dev 2023, vol 27, research article ict-driven work engagement interventions 16 the changes in work policy due to recent covid-19 pandemic have increased the relevance of ict in enabling organizational initiatives of work engagement interventions. ict should drive the work engagement interventions in a manner that will not only increase the quantum of workplace interactions, but also needs to be more precise in leading to authentic and autonomous interactions. otherwise, the efforts will lead towards frustration, rather than satisfaction of the need for relatedness, which will negatively impact the engagement of the employee. an in-depth understanding of the working of the satisfaction of need for relatedness will help in design and implementation of ict-driven work engagement interventions. the satisfaction of need for relatedness is contingent on two criteria (baumeister & leary, 1995). first, there needs to be frequent interactions with the relevant people. however, the interactions should also be affectively positive, but more importantly, should be largely free of negative affect and conflict. second, individuals should perceive that the relationships are stable and enduring in the foreseeable future, and that there is authentic affective concern in the interpersonal relations. while the current interventions do attempt to increase the interaction intensity, their scope do not cover for positive affect in interactions and perception of stable relationship. if only one of the criteria is met, the satisfaction is likely to be partial, and hence, the positive impact on work engagement will be limited. it should also be noted the satisfaction of need for relatedness is subject to diminishing returns. this means that while individuals require a certain amount of social relatedness, but once that minimum is satisfied, the priority of forming additional bonds decreases. this indicates there is a limit to which increased interactions can improve the satisfaction of need for relatedness and the work engagement of employees. ict-mediated communications in the workplace should be implemented with the aim of making teams function in a more collaborative manner. with the collaborative nature of communications, the perceived social support for the employees is likely to be higher. this will enable the satisfaction of the relatedness need in wfh teams. the implication of the moderating role of perceived social support is that when it is high, the work engagement interventions may not be very effective in promoting work engagement. hence, organizations need to be more vigilant in implementing work engagement interventions rather than overall roll-out. the work environment of the target employees should be understood before going ahead with the intervention. it is likely that in teams with high perceived social support, such interventions may look redundant or burdensome at worse. 5.3 limitations and future research directions although the current study explores the role of organizational interventions and social support in the satisfaction of need for relatedness in detail, these are certain limitations to our study. the context of pandemic in which the current research is done is unique in nature. further research is needed to explore the role of the satisfaction of need for relatedness in the effectiveness of ict-driven interventions. the implication of prolonged work from home exposure can be explored by conducting similar studies periodically. we included multiple organizations in the study. although our study provides a general overview of the response towards ict-driven interventions from employees who are working from home, further research focusing on fewer organizations will provide a context where australasian journal of information systems singh & dev 2023, vol 27, research article ict-driven work engagement interventions 17 interventions could be more uniform across the employees. on similar line, while our study focused on organizations from the knowledge industry, sector-specific studies can provide insights on the peculiarities in different areas. also, future studies can be conducted with a higher sample size to confirm the results of the current study. another limitation of the current study is that the convergent validity of certain constructs is borderline. to improve convergent validity, additional data should be collected, and the scale items of the constructs should be tested and improved to ensure a clear non-ambiguous understanding of the scale items by the future studies. further, the educational background and gender representation in the study were uneven. gender representation is a reflection of the demography in the industry (kumar & parveen, 2021). it needs to be seen what happens when the genders have an equitable representation. lastly, there was a mix of employees in terms of prior wfh experience. although this effect of prior wfh experience was controlled in our analysis, studies purely focusing on employees with no wfh experience can bring to light new aspects to the understanding of the ict-driven interventions for work engagement. 6 conclusion wfh is an important work design in the future of work (gagne et al., 2022). it is becoming more prevalent with the development and advancement in communication technologies (ict). further, the recent covid-19 pandemic forced organizations to implement work-from-home on an organizational scale. the transfer of work from office to home has given rise to new challenges, especially to work engagement. also, covid-19 pandemic presents a novel context wherein employees have been enforced with wfh. consequently, work engagement has become one of the top priorities for the organizations as it is a crucial success factor. organizations have used ict-driven initiatives to manage the work engagement among wfh employees. given the importance of engagement to organizations and the resource intensiveness of ict-driven initiatives in terms of time, money and effort, the effectiveness of these interventions becomes crucial. this study shows that the satisfaction of need for relatedness of employees mediates the relationship between interventions and work engagement. in the world of practice, engagement is typically considered as a catch-all umbrella term for positive work-related attitudes (like organizational commitment, job satisfaction, organizational citizenship, and intention to stay) (robertson, 2012), and, with this understanding, the interventions often get directed at supporting various job resources. the effectiveness of the interventions can be improved if the focus can be directed on enabling the satisfaction of need for relatedness. further, the study also shows that perceived social support in a team negatively moderates the relationship between intervention intensity and the satisfaction of need for relatedness. this indicates that an organization-wide roll-out of ictdriven intervention would be a crude attempt at addressing engagement, because it does not take into consideration the existing social support in a team. we suggest that ict-driven interventions should be implemented with awareness and understanding of the team environment. references allen, t. d., golden, t. d., & shockley, k. m. (2015). how effective is telecommuting? assessing the status of our scientific findings. psychological science in the public interest, 16(2), 40–68. https://doi.org/10.1177/1529100615593273 australasian journal of information systems singh & dev 2023, vol 27, research article ict-driven work engagement interventions 18 anthony-mcmann, p. e., ellinger, a. d., astakhova, m., & halbesleben, j. r. b. (2017). exploring different operationalizations of employee engagement and their relationships with workplace stress and burnout. human resource development quarterly, 28(2), 163– 195. https://doi.org/10.1002/hrdq.21276 ashford, s. j., caza, b. b., & reid, e. m. (2018). from surviving to thriving in the gig economy: a research agenda for individuals in the new world of work. research in organizational behavior, 38, 23–41. https://doi.org/10.1016/j.riob.2018.11.001 bakker, a. b., & demerouti, e. (2007). the job demands-resources model: state of the art. journal of managerial psychology, 22(3), 309–328. https://doi.org/10.1108/02683940710733115 bakker, a. b., & demerouti, e. (2017). job demands–resources theory: taking stock and looking forward. journal of occupational health psychology, 22(3), pp. 273–285). educational publishing foundation. https://doi.org/10.1037/ocp0000056 bakker, a. b., demerouti, e., & sanz-vergel, a. i. (2014). burnout and work engagement: the jd–r approach. annual review of organizational psychology and organizational behavior, 1(1), 389–411. https://doi.org/10.1146/annurev-orgpsych-031413-091235 baumeister, r. f., & leary, m. r. (1995). the need to belong: desire for interpersonal attachments as a fundamental human motivation. psychological bulletin, 117(3), 497–529. https://doi.org/10.1037/0033-2909.117.3.497 bavik, y. l., shaw, j. d., & wang, x.-h. (frank). (2020). social support: multidisciplinary review, synthesis, and future agenda. academy of management annals, 14(2), 726–758. https://doi.org/10.5465/annals.2016.0148 callea, a., de rosa, d. d., ferri, g., lipari, f., & costanzi, m. (2019). are more intelligent people happier? emotional intelligence as mediator between need for relatedness, happiness and flourishing. sustainability, 11(4). https://doi.org/10.3390/su11041022 chong, s., huang, y., & chang, c.-h. (daisy). (2020). supporting interdependent telework employees: a moderated-mediation model linking daily covid-19 task setbacks to next-day work withdrawal. journal of applied psychology, 105(12), 1408–1422. https://doi.org/10.1037/apl0000843 deci, e. l., olafsen, a. h., & ryan, r. m. (2017). self-determination theory in work organizations: the state of a science. annual review of organizational psychology and organizational behavior, 4(1), 19–43. https://doi.org/10.1146/annurev-orgpsych-032516113108 deci, e. l., & ryan, r. m. (2000). the “what” and “why” of goal pursuits: human needs and the self-determination of behavior. psychological inquiry, 11(4), 227–268. https://doi.org/10.1207/s15327965pli1104_01 el-zoghby, s. m., soltan, e. m., & salama, h. m. (2020). impact of the covid-19 pandemic on mental health and social support among adult egyptians. journal of community health, 45(4), 689–695. https://doi.org/10.1007/s10900-020-00853-5 ellis, j. b., luther, k., bessiere, k., & kellogg, w. a. (2008). games for virtual team building. proceedings of the 7th acm conference on designing interactive systems, 295–304. https://doi.org/10.1145/1394445.1394477 australasian journal of information systems singh & dev 2023, vol 27, research article ict-driven work engagement interventions 19 fallon, n. (2020). managing from home? here’s how to keep your team engaged during coronavirus. us chamber of commerce. https://www.uschamber.com/co/run/humanresources/how-to-support-your-employees-in-a-crisis fernet, c., austin, s., trépanier, s.-g., & dussault, m. (2013). how do job characteristics contribute to burnout? exploring the distinct mediating roles of perceived autonomy, competence, and relatedness. european journal of work and organizational psychology, 22(2), 123–137. https://doi.org/10.1080/1359432x.2011.632161 fredrickson, b. l. (1998). what good are positive emotions? review of general psychology, 2(3), 300–319. https://doi.org/10.1037/1089-2680.2.3.300 fuller, c. m., simmering, m. j., atinc, g., atinc, y., & babin, b. j. (2016). common methods variance detection in business research. journal of business research, 69(8), 3192–3198. https://doi.org/10.1016/j.jbusres.2015.12.008 gagné, m., deci, e. l., & ryan, r. m. (2018). self-determination theory applied to work motivation and organizational behavior. in d. s. ones, n. anderson, & c. viswesvaran (eds.), the sage handbook of industrial, work & organizational psychology (pp. 97–121). sage reference. https://doi.org/10.4135/9781473914957.n6 gagné, m., parker, s. k., griffin, m. a., dunlop, p. d., knight, c., klonek, f. e., & parentrocheleau, x. (2022). understanding and shaping the future of work with selfdetermination theory. nature reviews psychology, 1(7), 378–392. https://doi.org/10.1038/s44159-022-00056-w greasley, k., edwards, p., baker-mcclearn, d., & dale, j. (2012). why do organisations engage in hr initiatives? a test case of a health and wellbeing intervention. employee relations, 34(4), 443–462. https://doi.org/10.1108/01425451211236869 gubler, t., larkin, i., & pierce, l. (2017). doing well by making well: the impact of corporate wellness programs on employee productivity. management science, 64(11), 4967–4987. https://doi.org/10.1287/mnsc.2017.2883 hair, j. f., & sarstedt, m. (2019). factors versus composites: guidelines for choosing the right structural equation modeling method. project management journal, 50(6), 619–624. https://doi.org/10.1177/8756972819882132 hair, j f, black, w. c., babin, b. j., anderson, r. e., & tatham, r. (2019). multivariate data analysis (8th ed.). boston, ma, usa: cengage. halbesleben, j. r. b. (2010). a meta-analysis of work engagement: relationships with burnout, demands, resources, and consequences. in work engagement: a handbook of essential theory and research. (pp. 102–117). washington, dc, usa: psychology press. ilozor, d. b., ilozor, b. d., & carr, j. (2001). management communication strategies determine job satisfaction in telecommuting. journal of management development, 20(6), 495–507. https://doi.org/10.1108/02621710110399783 kahn, w. a. (1990). psychological conditions of personal engagement and disengagement at work. academy of management journal, 33(4), 692–724. https://doi.org/10.2307/256287 australasian journal of information systems singh & dev 2023, vol 27, research article ict-driven work engagement interventions 20 khor, l. k., & tan, c. l. (2022). workforce management in the post-pandemic era: evidence from multinational companies using grounded theory. global business and organizational excellence. https://doi.org/10.1002/joe.22174 king, a. j., brophy, l. m., fortune, t. l., & byrne, l. (2020). factors affecting mental health professionals’ sharing of their lived experience in the workplace: a scoping review. psychiatric services, 71(10), 1047–1064. https://doi.org/10.1176/appi.ps.201900606 kline, r. b. (2015). principles and practice of structural equation modeling. nyc, usa: guilford press. kniffin, k. m., narayanan, j., anseel, f., antonakis, j., ashford, s. p., bakker, a. b., bamberger, p., bapuji, h., bhave, d. p., choi, v. k., creary, s. j., demerouti, e., flynn, f. j., gelfand, m. j., greer, l. l., johns, g., kesebir, s., klein, p. g., lee, s. y., … vugt, m. van. (2021). covid-19 and the workplace: implications, issues, and insights for future research and action. american psychologist, 76(1), 63–77. https://doi.org/10.1037/amp0000716 knight, c., patterson, m., & dawson, j. (2019). work engagement interventions can be effective: a systematic review. european journal of work and organizational psychology, 28(3), 348–372. https://doi.org/10.1080/1359432x.2019.1588887 kossek, e. e., & lautsch, b. a. (2017). work–life flexibility for whom? occupational status and work–life inequality in upper, middle, and lower level jobs. academy of management annals, 12(1), 5–36. https://doi.org/10.5465/annals.2016.0059 kumar, s., & parveen, r. (2021). assessing the factors of employees’ intrapreneurial behaviour: an indian context. world journal of entrepreneurship, management and sustainable development, 17(4), 804–821. https://doi.org/10.1108/wjemsd-12-2020-0160 lim, w. m. (2021). history, lessons, and ways forward from the covid-19 pandemic. international journal of quality and innovation, 5(2), 101–108. lim, w. m. (2022). ushering a new era of global business and organizational excellence: taking a leaf out of recent trends in the new normal. global business and organizational excellence, 41(5), 5–13. https://doi.org/10.1002/joe.22163 liu, d., chen, x.-p., & yao, x. (2011). from autonomy to creativity: a multilevel investigation of the mediating role of harmonious passion. journal of applied psychology, 96(2), 294– 309. https://doi.org/10.1037/a0021294 mosquera, p., soares, m. e., & alvadia, t. (2022). is teleworking at odds with social sustainability and organizational learning?. the learning organization, 29(5), 527–547. https://doi.org/10.1108/tlo-01-2022-0002 newman, s. a., & ford, r. c. (2021). five steps to leading your team in the virtual covid-19 workplace. organizational dynamics, 50(1), 100802. https://doi.org/10.1016/j.orgdyn.2020.100802 nielsen, i. k., jex, s. m., & adams, g. a. (2000). development and validation of scores on a two-dimensional workplace friendship scale. educational and psychological measurement, 60(4), 628–643. https://doi.org/10.1177/00131640021970655 nunnally, j. c. (1994). psychometric theory (vol. 56). nyc, usa: mcgraw-hill. australasian journal of information systems singh & dev 2023, vol 27, research article ict-driven work engagement interventions 21 olszewski, p., & mokhtarian, p. (1994). telecommuting frequency and impacts for state of california employees. technological forecasting and social change, 45(3), 275–286. https://doi.org/10.1016/0040-1625(94)90050-7 podsakoff, p. m., mackenzie, s. b., lee, j.-y., & podsakoff, n. p. (2003). common method biases in behavioral research: a critical review of the literature and recommended remedies. journal of applied psychology, 88(5), 879–903. https://doi.org/10.1037/00219010.88.5.879 riva, g., baños, r. m., botella, c., wiederhold, b. k., & gaggioli, a. (2012). positive technology: using interactive technologies to promote positive functioning. cyberpsychology, behavior, and social networking, 15(2), 69–77. https://doi.org/10.1089/cyber.2011.0139 robertson, i. (2012). the importance of employee engagement in difficult times. the guardian. https://www.theguardian.com/public-leaders-network/2012/may/03/importanceemployee-engagement-difficult-times ryan, r. m., & deci, e. l. (2000). self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. american psychologist, 55(1), 68–78. https://doi.org/10.1037/0003-066x.55.1.68 ryan, r. m., & deci, e. l. (2017). self-determination theory: basic psychological needs in motivation, development, and wellness. nyc, usa: guilford press. https://doi.org/10.1521/978.14625/28806 sacco, d. f., & ismail, m. m. (2014). social belongingness satisfaction as a function of interaction medium: face-to-face interactions facilitate greater social belonging and interaction enjoyment compared to instant messaging. computers in human behavior, 36, 359–364. https://doi.org/10.1016/j.chb.2014.04.004 sahoo, a., xechung, n. l., mostafiz, m. i., & krishnaswamy, j. (2022). perceived risk and sensitivity and their influence on expatriate performance during the covid-19 pandemic. global business and organizational excellence, 41(4), 68–84. https://doi.org/https://doi.org/10.1002/joe.22152 saks, a. m. (2006). antecedents and consequences of employee engagement. journal of managerial psychology, 21(7), 600–619. https://doi.org/10.1108/02683940610690169 saks, a. m. (2017). translating employee engagement research into practice. organizational dynamics, 46(2), 76–86. https://doi.org/10.1016/j.orgdyn.2017.04.003 sardeshmukh, s. r., sharma, d., & golden, t. d. (2012). impact of telework on exhaustion and job engagement: a job demands and job resources model. new technology, work and employment, 27(3), 193–207. https://doi.org/10.1111/j.1468-005x.2012.00284.x sheldon, k. m., elliot, a. j., kim, y., & kasser, t. (2001). what is satisfying about satisfying events? testing 10 candidate psychological needs. journal of personality and social psychology, 80(2), 325–339. https://doi.org/10.1037/0022-3514.80.2.325 siampou, f., komis, v., & tselios, n. (2014). online versus face-to-face collaboration in the context of a computer-supported modeling task. computers in human behavior, 37, 369– 376. https://doi.org/10.1016/j.chb.2014.04.032 australasian journal of information systems singh & dev 2023, vol 27, research article ict-driven work engagement interventions 22 simmering, m. j., fuller, c. m., richardson, h. a., ocal, y., & atinc, g. m. (2014). marker variable choice, reporting, and interpretation in the detection of common method variance: a review and demonstration. organizational research methods, 18(3), 473–511. https://doi.org/10.1177/1094428114560023 singh, h., & verma, s. (2021). evolution of is competencies due to mandatory telework on organizational scale–a work-systems approach. proceedings of the 54th hawaii international conference on system sciences, 5789. soane, e., truss, c., alfes, k., shantz, a., rees, c., & gatenby, m. (2012). development and application of a new measure of employee engagement: the isa engagement scale. human resource development international, 15(5), 529–547. https://doi.org/10.1080/13678868.2012.726542 stych, m., & greer, l. l. (2020). is your organization ready for permanent wfh? harvard business review. https://hbr.org/2020/08/is-your-organizationready-for-permanent-wfh toth, i., heinänen, s., & kianto, a. (2022). disentangling the elements of psycap as drivers for work, organization and social engagement in knowledge-intensive work. personnel review. https://doi.org/10.1108/pr-01-2021-0031 van den broeck, a., ferris, d. l., chang, c.-h., & rosen, c. c. (2016). a review of selfdetermination theory’s basic psychological needs at work. journal of management, 42(5), 1195–1229. https://doi.org/10.1177/0149206316632058 van den broeck, a., vansteenkiste, m., de witte, h., & lens, w. (2008). explaining the relationships between job characteristics, burnout, and engagement: the role of basic psychological need satisfaction. work & stress, 22(3), 277–294. https://doi.org/10.1080/02678370802393672 walther, j. b. (2011). theories of computer-mediated communication and interpersonal relations. in m. l. knapp & j. a. daly (eds.), the sage handbook of interpersonal communication (4th ed., pp. 443–479). thousand oaks, ca, usa: sage publications. walther, j. b., van der heide, b., ramirez jr., a., burgoon, j. k., & peña, j. (2015). interpersonal and hyperpersonal dimensions of computer-mediated communication. in the handbook of the psychology of communication technology (pp. 1–22). https://doi.org/10.1002/9781118426456.ch1 wang, b., liu, y., & parker, s. k. (2020). how does the use of information communication technology affect individuals? a work design perspective. academy of management annals, 14(2), 695–725. https://doi.org/10.5465/annals.2018.0127 wang, b., liu, y., qian, j., & parker, s. k. (2021). achieving effective remote working during the covid-19 pandemic: a work design perspective. applied psychology, 70(1), 16–59. https://doi.org/10.1111/apps.12290 wang, j., & wang, x. (2019). structural equation modeling: applications using mplus. in structural equation modeling: applications using mplus. hoboken, nj, usa: john wiley & sons. https://doi.org/10.1002/9781119422730 zhao, x., lynch jr., j. g., & chen, q. (2010). reconsidering baron and kenny: myths and truths about mediation analysis. journal of consumer research, 37(2), 197–206. https://doi.org/10.1086/651257 australasian journal of information systems singh & dev 2023, vol 27, research article ict-driven work engagement interventions 23 copyright: © 2023 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v27i0.4039 http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 theoretical background and hypotheses 2.1 need for relatedness 2.2 work engagement 2.3 need for relatedness and work engagement 2.4 ict-driven work engagement interventions 2.5 perceived social support 3 method 3.1 sample and data collection 3.2 variable measurement 3.3 addressing common method variance 3.4 reliability and validity of measures 4 results 4.1 direct effects 4.2 interaction effects 4.3 mediation effects 5 discussion 5.1 theoretical contributions 5.2 practical implications 5.3 limitations and future research directions 6 conclusion paper title (paper title style) australasian journal of information systems kautz 2022, vol 26, editorial editorial for ajis 2022 1 editor’s comment brief reflections on the journal, the concepts of systems, technologies, use, cybernetics, and the ajis volume 26 karlheinz kautz rmit university, melbourne, australia karlheinz.kautz@rmit.edu.au the final articles for the year 2022 of the australasian journal of information systems have been published; the complete volume 26 of the journal is now avaialble. in last year’s editorial i formulated my understanding of the information systems (is) discipline and research based on lee’s editorial comments on is research in management information systems quaterly (misq) (lee, 2001) as a foundation of my vision and of the (future) scope of the ajis. i also expressed a desire to receive more submissions based on alvesson and sandberg’s (2011, 2013, 2014) problematization research approach: “research that is assumption-challenging and deals with surprising phenomena instead of being confirmatory and founded on mere gap-spotting.” (kautz, 2021) it is therefore affirming to read chatterjee and davison’s (2021) editorial plea for compelling problematization in is research in the information systems journal (isj) as well as the misq editor’s comments on qualitative research methods in is including a call for phenomenonfocused problematization (monteiro, constantinides, scott, shaikh, and burton-jones, 2022) which is based on the misq knowledge sharing online sessions 2022 on qualitative research, in one of which i had the opportunity to participate. further based on the journal’s published articles i also sketched out some reasons for desk rejections. in this context professor yogesh dwiwedi, the current editor-in-chief of the international journal of information management (ijim) and colleagues in a recent editorial of the ijim (dwivedi, hughes, cheung, conboy, duan, dubey, ... , and viglia, 2022), likened submitting to a journal to participating in a conversation and that, beyond lack of fit with a journal’s scope, not participating in a journal's ongoing conversation and not being connected with the existing research conversations in a journal through a demonstrated lack of familiarity with existing work published in the target journal increases the risk of a desk reject. i highly recommend these three editorials to all is researchers, who seek support for their development of quality research articles. after some years of online gatherings in 2022 it became possible again to meet in person for workshops, seminars, and conferences and in the second half of the year i attended four such events: digital futures, an event by the disrupt.sydney conference series at the university of sydney, australia (disruptsydney.net) (i really recommend this series to you if you are interested in challenging common sense, in learning about new ways of thinking, and in critically engaging with our disicipline); the australian cybernetic 2022 event at the australian national university (anu) (cybernetics.anu.edu.au) in canberra (more about it later); the australasian conference on information systems (acis) at the university of melbourne, australia; and the international conference on information systems (icis) in copenhagen, mailto:karlheinz.kautz@rmit.edu.au australasian journal of information systems kautz 2022, vol 26, editorial editorial for ajis 2022 2 denmark under the auspices of the association of information systems. it was a wonderful and longed after experience with algorithms, algorithmic management, machine learning algorithms, and quite some focus on artifical intelligence (ai) everywhere. so, the jester role i had taken on earlier (see kautz and jensen, 2013; kautz, 2018) resurfaced as i could not withstand the temptation of testing the application and consequences of generative ai and of asking the openai chatbot chatgpt, gpt standing for generative pre-trained transformer (https://openai.com/blog/chatgpt/) to ‘write me an australasian journal of information systems editorial’. the result is below and i leave it to the learned reader to judge the outcome and compare it with my current efforts. dear readers of the australian journal of information systems, we are pleased to present the latest edition of our journal, which features a diverse range of research on the latest developments in the field of information systems. in this edition, we have a number of thought-provoking articles that explore the intersection of technology and society. one article examines the ethical implications of using artificial intelligence in decision-making, while another looks at the role of social media in shaping public opinion. we also have articles that delve into the technical side of information systems, including a study on the use of blockchain in supply chain management and an analysis of the security risks associated with cloud computing. we hope that these articles will spark discussions and encourage further research on the many exciting and complex issues facing the field of information systems today. thank you for your continued support of the australian journal of information systems. sincerely, [your name] editor-in-chief, australian journal of information systems figure 1. chatgpt’s editorial for the ajis created on december 29, 2022 another observation, i might rather call it a surprise in the spirit of problematization, that was triggered by the participation in these events and my work with the journal during the last years is what avison (2018) so aptly described in the context of teaching is as “one problem that seems to be always with us is the confusion that students have in differentiating is from ict.” (avison, 2018, p. 28). i see this problem, beyond in teaching is, as the is community’s continuous struggle to conceptually keep is and technology, information technology (it), or to use the term from avison’s quote above information and communication technology (ict), apart and, as i will point out below, to adequately deal with the concepts of use, user, and usage in the is discipline. this made me look at some definitions of is starting with those made by or quoted by avison. with wood-harper in their exploration of is development the authors write “an information australasian journal of information systems kautz 2022, vol 26, editorial editorial for ajis 2022 3 system can be defined as: ‘a system to collect, process store, transmit, and display information.’“1 (avison and wood-harper (1990, p. 3) and continue quoting one of the founders of the is discipline who defines an is as: “an integrated man/machine system for providing information to support the operations, management and decision making function in an organization. the system uses computer hardware, software, manual procedures, management and decision models and a data base.” (davis, 1974) (ibid.) where a man/machine system consists of “information technology and people’s activities in preparing and using information.” (davis, 1983) key concepts of information systems such as data, information, and systems are among others also defined by avison and fitzgerald (1988) in their classic textbook ‘information systems development – methodologies, techniques & tools’ here cited in its 4th (macgraw hill) 2006 edition. this publication contains a definition of an information system by buckingham et al. (1987), who declare that an information system is “a system which assembles, stores, processes and delivers information relevant to an organisation (or to society), in such a way that the information is accessible and useful to those who wish to use it, including managers, staff, clients and citizens. an information system is a human activity (social) system which may or may not involve the use of computer systems.”(avison and fitzgerald, 2006, p. 23)2 checkland (1981, p. 314) had earlier defined a human activity system as “a notional purposive system which expresses some purposeful human activity, activity which could in principle be found in the real world. such systems are notional in the sense that they are not descriptions of actual real-world activity (which is an exceptionally complex phenomenon) but are intellectual constructs; they are ideal types for use in a debate about possible changes which might be introduced into a real-world problem situation.” finally, i turned to nygaard (2002), who based on 40 years of engaging with systems thinking elaborates on the concept of system as: “nothing is inherently a system, but we may state that part of the world is a system to us when we study it using a system perspective. a system is a part of the world that is regarded a whole with: • its substance consisting of components; • each component’s state characterized by the states of properties, called attributes, that are selected as being relevant; and by • state transitions relating to these attributes and other components and their attributes … no single perspective is sufficient when one is considering the development and use of an information system. … the study of information systems in their social and organizational context remains at the heart of the discipline of informatics.” nygaard (2002, pp. v – x) for me these definitions together present the essence of our field. they have led me and, beyond those cited above, others to the position that humans, human actors are not merely using systems, or in our context is, but are active parts of them and use certain it, which are also part of these systems. this is in line with lee’s (2001) comments referring to the sociotechnical phenomena that set our field apart and sarker et al.’s (2019) explanation of the sociotechnical perspective on is, in which the social component includes humans as individuals or social collectives. 1 all bolding in the used citations has been made by me to put emphasis on the respective concepts, in contrast all italics appeared in the original texts. 2 while the use of computer hardware and software as well as the use of computer systems are stated in the chosen definitions, the term it system which is otherwise used regularly in colloquial language does not appear in the selected sources and would warrant a broader literature review and analysis including whether, and if yes, when and how ‘it systems’ became ‘information systems’. australasian journal of information systems kautz 2022, vol 26, editorial editorial for ajis 2022 4 unfortunately, this position is not been held consistently, not even by those putting it forward. in some cases it remains unclear whether avoidance of the concept of is was intended and conscious or, perhaps just accidental, unconscious and unintentional. for example, lamb and king (2003), although otherwise avoiding the concept of is use, list the key word ‘is users’ in their argument for reconceptualizing users as social actors and refer to ict and information service use when writing: “despite pervasive ict use, social actors are not primarily users of icts. most people who use ict applications utilize multiple applications, in various roles, and as part of their efforts to produce goods and services while interacting with a variety of other people, and in multiple social contexts. moreover, the socially thin user construct limits our understanding of information selection, manipulation, communication, and exchange within complex social context. using analyses from a recent study of online information service use, we develop an institutionalist concept of a social actor whose everyday interactions are infused with ict use. we then encourage a shift from the user concept to a concept of social actor in is research.” (lamb and king, 2003, p. 197) more recently halskov and hansen (2015) in a study of the participatory design (pd) community, while still referring to use situations, report that in the pd community the idea of the ‘user’ has been challenged and that rather the concept of ‘people’ should be considered as a fundamental aspect of pd. i myself have done that but also neglected to resolve the issue whether is are ‘used’ when writing “participatory design (pd) is a design approach in which the participation of people in the co-design of the information systems and information technologies (is/it) they are supposed to use themselves is a central tenet.” (kautz, 2011, p. 218) and even more recently in kautz and bjerknes (2021) nearly conflating the concepts of is and it when writing about user participation in distributed participatory design. to side-step such inconsistencies, riemer and johnston (2012) for instance avoid the concept of is and is use entirely in their research on technology appropriation. they set the concept of technology appropriation against the conventional it adoption concept, and apply the concept of “is technology appropriation” in their phenomenological study (see heidegger, 1927, 1962) of an unfolding technology appropriation and its emerging use. they qualify the technology in question as (enterprise) social media and use the term social media technology as well as the label social media platform, but make no additional mention of the terms system or information systems. they otherwise distinguish “it as an object of inspection and reflection from it as it is in use”, captured by heidegger’s concept of equipment and propose to interpret technology appropriation as the change of it from an object evaluated by users upon first encounter to equipment when it is transparently implicated in a practice. they also shun the concept of is use by consistently referring to it use when further discussing the usefulness of heidegger’s concept of equipment for the is discipline (riemer and johnston, 2017). yet others promote a different understanding of systems, is, and the concept of use or usage. acknowledging other views on the concept of is as a foundation for is research, burton-jones and straub (2006) put forward the system usage construct and argue that has played a central role in is research and that many studies have employed other constructs among others information usage as proxies for system usage. in their view “information usage is a useful construct, but it is not identical to system usage.” for the purpose of defining system usage they state that “system usage is an activity that involves three elements: (1) a user, i.e., the subject using the is, (2) a system, i.e., the object being used, and (3) a task, i.e., the function being performed.” (burton-jones and straub, 2006, p. 231). they further https://www.jstor.org/stable/30036529 https://www.jstor.org/stable/30036529 australasian journal of information systems kautz 2022, vol 26, editorial editorial for ajis 2022 5 define an is as an artifact that provides representations of one or more task domains. in extension burton-jones and grange (2013) clarify that their understanding of is use is grounded in representation theory and its particular view on is as detailed by weber (1997). the background for their work is among others the theory of task-technology fit that theorizes “the interdependence between an individual (a technology user), technology (data, hardware, software tools and the services they provide) and task (activity carried out by individuals to produce the required output) characteristics.” (marikyan and papagiannidis, 2022 referring to the influential work of goodhue and thompson, 1995) and which according to https://is.theorizeit.org/wiki/task-technology_fit (accessed december 29, 2022) holds that “it is more likely to have a positive impact on individual performance and be used if the capabilities of the it match the tasks that the user must perform.” the concept of is use does not appear here, the technology user applies or uses it. whether this a deliberate or unintentional decision and choice by the authors is not clear. in the same vain, goodhue and thompson (1995, p. 216) themselves contextualize their work by stating that “technologies are viewed as tools used by individuals in carrying out their tasks. in the context of information systems research, technology refers to computer systems (hardware, software, and data) and user support services (training, help lines, etc.) provided to assist users in their tasks.” but otherwise they hypothesize about “the utilization of information systems”. this, unfortunately does not resolve the confusion, or possible conflation, regarding the concepts of is and it. regrettably, this is also valid for a recent update of the misq curation on is use – also covering some it adoption literature where burton-jones, stein, and mishra (2020) put forward the foundations of knowledge in is use to be found in work on the “application, refinement, and integration of various social psychological explanations of it acceptance and use” and set it against unpacking the complexity of the concept by advancements in exploring “different theories to account for the characteristics of is use dynamics (emergence, interdependence, emotions, power, etc.).” which maintains the, from my perspective, unfortunate commingling of the concepts. in this context ciborra (2002, p. 1) laments that “the current descriptions of the design, implementation, management, and use of information technology in organizations (in short information systems) are largely founded on notions of rationality, science, and method … [the essays in this volume] point to an alternative centre of gravity: human existence in everyday life.” sadly, in the above text fragment he also remains ambiguous about whether it is a synonym or a component of an is. in contrast, riemer and johnston’s (2012, 2017) framing of it appropriation and evasion of the concept of is use and is user as introduced above is based on heidegger’s philosophy and phenomenological analysis of everyday human engagement and a critique of representationalism3. finally, in an early reflection on the tension between is and it based on human existence langefors (1980) contrasts what he calls data systems (and data processing concepts) with is (concepts) and argues that data systems do not carry information and that the concept of user, 3 further representational framings of is phenomena set off against nonrepresentational modes of theorizing is related phenomena are discussed in cecez-kecmanovic, kautz, and abrahall (2014), such a discussion however goes beyond the scope of this editorial comment. australasian journal of information systems kautz 2022, vol 26, editorial editorial for ajis 2022 6 in his words, the concept of user view delimits the concept of information systems to merely focussing on representation, structuring, and exploitation of data while ignoring significant socio-psychological challenges, socio-linguistic aspects, and socio-technical issues of is. as a result of his assessment of the – in his nomenclature – datalogical trait of a user view, he promotes an infological perspective on is, which goes beyond the identified limitations and does not employ the term user. he does not offer an alternative term and his infological view has not gained much traction in the wider is community, but it provides an early call to pay close attention to the concepts of system and is, data, information, and ‘user’ in the context of is. more than 40 years after the publication of langefors’ (1980) reflections there is still conceptual groundwork to do and efforts to make to break the everyday habit of simply calling something a system. this and nygaard’s (2002) reference above to informatics4 , brings me back to my participation in australian cybernetic 2022 event including the launch of the school of cybernetics at anu, which had been triggered by my attendance at disrupt.sydney. it also connects me to my past as i have been studying at, and graduated from, a faculty of informatics at the technical university berlin in germany from the mid 1970ties. that faculty then had only recently changed its name from cybernetics to informatics5, its curriculum was still very much impacted by the former. however, instead of discussing the distinction between these concepts here – a topic which also would deserve some attention in the context of is – i rather share some thoughts on what is called the ‘new cybernetics: systems thinking for the 21st century’ (https://cybernetics.anu.edu.au/news/2022/03/28/the-new-cybernetics-systems-thinking-for21st-century/) and is explained in two publications by members of the new anu school (see, bell, 2021; gould, daniell, meares, and bell., 2022). bell (2021) provides background and reminds us that as a form of systems thinking and as a systems methodology, cybernetics dates back to the 1940s (see wiener, 1948 ) and is strongly linked to the rise of the computer. cybernetics is the science of steering or control, and persuasively argues the importance of the relationship and of the feedback loops between humans, computing and the ecological in holistic systems, also called cybernetic systems. it is an interdisciplinary field that recognises the diversity of voices when identifying and building systems. as such bell (2021) promotes cybernetics and the concept of cybernetic systems as a tool and an approach to understand and form possible futures and to design future systems, above all as a means to take ai safely, sustainably, and responsibly to scale in cybernetically informed and digitally transformed societies and organizations. 4 informatics is the study of the structure, behaviour, and interactions of natural and engineered computational systems. informatics studies the representation, processing, and communication of information in natural and engineered systems. it has computational, cognitive and social aspects. the central notion is the transformation of information whether by computation or communication, whether by organisms or artifacts. (https://www.ed.ac.uk/files/atoms/files/what20is20informatics.pdf, accessed december 29, 2002). a further discussion about the origin of the concept and its relation to business informatics and its distinction from (business) is would be interesting; however it is beyond the scope of the current text. 5 to add even more concepts to this editorial comments: in a changed form as part of an informatics faculty now called ‘computer science’ it resurfaced in a different incarnation as a control systems group (https://www.tu.berlin/en/control). australasian journal of information systems kautz 2022, vol 26, editorial editorial for ajis 2022 7 one particular cybernetics concept, the concept of productive discomfort (bell, 2021; gould, daniell, meares, and bell, 2022) – i.e., giving room to diverse voices – has driven these editorial contemplations as an invitation to the readers and contributors of the ajis to continue the debate about the core concepts of our discipline. with these reflections as a timely reminder that the study of systems and is as a specific instance of systems is not limited to, and does not only happen in, the confines of business schools and departments performed by business is academics and specialists i like to return to volume 26 of the journal. in 2022 the journal received 215 submissions and published 22 articles, 13 in the regular research articles section, one article in its section on green is/it, four articles and an editorial in a section on applied ethics, and four in the section for selected papers from the australasian conference on information systems (acis). the author teams came for all three regions of the ais, with the majority coming from region 3 (asia, pacific) and here more precisely from oceania. the topics in the acis special section covered current subjects such as employees’ perceptions of robotic automation when robots join work teams, digital transformation with digital kaizen, sociomaterial practices in digitally supported water management, and gender bias in artificial intelligence presented in three case studies and one extended literature review. the applied ethics section included the contemporary topics of privacy concerns in covid-19 tracing apps, of social media analytics, and of social media users (see the reflections above) behaviour with their research methods comprising a case study, a systematic literature review, a design science approach, and a quantitative survey. the latter approach was also applied in the green is/it publication on the present-day subject of environmental sustainability and its relation to internet consumption. hypothetico-deductive approaches and/or quantitative survey questionnaires as well as a quantitative test instrument were used to study digital leadership, social media use by vulnerable persons in the context of disaster, student engagement in online education, turnover of it professionals in japan, and diagnosing phishing emails in five of the 13 regular research articles. web scraping instagram to comprehend customer engagement and textual tweet analysis to explore the role of emotion in privacy concerns of tracing apps, both in the explicit context of covid-19 were two further quantitative approaches that were deployed, while qualitative interviews were used in two studies to deepen the understanding of process stories as part of business process modelling and of value creation in smes through business intelligence, respectively, and one article presented a mixed method approach to research social media use and older adults life satisfaction. the remaining three pieces consisted of a conceptual study of challenges and opportunities brought about by pervasive sensing technology in smart work environments, a systematic literature review on the impact of blockchain on supply chains, and an econometrics approach to analyse icts’ impact on human development in south asia. together the articles present recent and established topics and again a healthy mixture of wellknown conceptual, quantitative and qualitive, positivist and interpretive, and in one case critical sociomaterial, research methods and emerging text analysis and website extraction approaches; all in line with the methodological pluralist positioning of the journal. australasian journal of information systems kautz 2022, vol 26, editorial editorial for ajis 2022 8 as an aside as we have been running some basic analytics regarding the resolution of the dois (clicks on dois referring to articles in the journal) of the 828 published articles in the journal since its inception, the interest in the journal has steadily increased; in 2020 we count around 55000 resolutions, in 2021 around 75000 and in 2022 around 90000, on average 7500 resolutions per month, and 804 articles in our archive and current volume have at least been accessed once during the last year. this has only been possible through the work of numerous previous and current volunteers and brings me to express my gratitude to the colleagues, who led their review teams and served as section editors for the 2022 journal submissions; their names are listed below after this editorial's reference list. finally i like to thank the is community and beyond for their continuing support and interest in the journal and look forward to all your further engagement with the ajis in the coming year. references alvesson, m., and sandberg, j. (2011). "generating research questions through problematization," academy of management review (36:2), pp. 247-271. alvesson, m., and sandberg, j. (2013). "has management studies lost its way? ideas for more imaginative and innovative research," journal of management studies (50), pp. 128-152. alvesson, m., and sandberg, j. (2014). "habitat and habitus: boxed-in versus box-breaking research," organization studies (35:7), pp. 967-987. avison, d. (2018). "reminiscences and reflections," in fitzgerald, g., and malaurent, j.,(eds.), information systems, development approaches and qualitative research: a tribute to david avison, pp. 19-25. cergy, france: essec publishing. avison, d., and wood-harper, t. (1990). multiview an exploration in information systems development. oxford, uk: blackwell scientific publications. avison, d., and fitzgerald, g. (2006). information systems development – methodologies, techniques & tools (4th edition). maidenhead, uk: mcgraw hill education. bell, g. (2021). "after the pandemic: cybernetic systems and an approach to the future," the sir robert garran oration, delivered at the institute of public administration australia (ipaa) conference 2021. anu school of cybernetics, canberra, act, australia: australian national university. buckingham, r.a., hirschheim, r.a., land, f.f., and tully, c.j. (1987). "information systems curriculum: a basis for course design, " in buckingham, r.a., hirschheim, r.a., land, f.f. , and tully, c.j. (1987), information systems education: recommendations and implementation. cambridge, uk: cambridge university press. burton-jones, a., and grange, c. (2013) "use to effective use: a representation theory perspective," information systems research (24:3)., pp. 632-658. https://www.jstor.org/stable/42004286 australasian journal of information systems kautz 2022, vol 26, editorial editorial for ajis 2022 9 burton-jones, a., stein, m.-k. , and mishra, a. (2020). "is use," misq curations. available as updated version to the original 2017 release at: https://www.misqresearchcurations.org/blog/2017/12/1/is-use burton-jones, a., and straub, d.w. jr. (2006). "reconceptualizing system usage: an approach and empirical test," information systems research (17:3), pp. 228-246. https://www.jstor.org/stable/23015887 cecez-kecmanovic, d., kautz, k., and abrahall, r. (2014). "reframing success and failure of information systems," mis quarterly (38:2)), pp. 561-588. chatterjee, s., and davison, r. (2021). "the need for compelling problematisation in research: the prevalence of the gap-spotting approach and its limitations, " information systems journal (31:2), pp. 227-230. https://doi.org/ 10.1111/isj.12316 checkland, p. (1981). systems thinking. systems practice. chichester, uk: wiley & sons ciborra, c. (2002). the labyrinths of information – challenging the wisdoms of systems. oxford, uk: oxford university press. davis, g.b. (1974). management information systems: conceptual foundations, structure and development. new york, usa: mcgraw-hill. davis. g. b. (1983). the domain of information systems. minneapolis, mn: management information systems research center, working paper 83-07. dwivedi, y. k., hughes, l., cheung, c. m., conboy, k., duan, y., dubey, r., ..., and viglia, g. (2022). "how to develop a quality research article and avoid a journal desk rejection," international journal of information management (62),102426. https://www.sciencedirect.com/science/article/pii/s0268401221001195 goodhue, d. l.; and thompson, r. l. (1995), "task-technology fit and individual performance," mis quarterly (19:2), pp. 213-236. gould, m., daniell, k. a., meares, a., and bell, g. (2022). "re/defining leadership in the 21st century: the view from cybernetics," a white paper, anu school of cybernetics. canberra, act, australia: australian national university & menzies foundation. halskov, k., and hansen, n.b. (2015). "the diversity of participatory design research practice at pdc [the participatory design conference] 2002-2012," international journal of humancomputer studies (74), pp. 81-92. heidegger, m. (1927). sein und zeit (german for being and time). tübingen, germany: max niemeyer verlag. heidegger, m. (1962). being and time (translation by macquarrie, j., and robinson, e.), london, uk: scm press. kautz, k. (2011). "investigating the design process: participatory design in agile software development," information technology & people (24:3), pp. 217-235. https://doi.org/10.1108/09593841111158356 kautz, k. (2018), "postface: the jester returns—selected readings and eclectic opinions on sociomateriality going mainstream in management and organization studies," in mitev, n., morgan-thomas, a., lorino, p., de vaujany, f.-x., and nama, y. (eds), technology, work and globalization: materiality and managerial techniques, pp. 389–403, australasian journal of information systems kautz 2022, vol 26, editorial editorial for ajis 2022 10 springer publishing company, berlin, germany doi: https://doi.org/10.1007/978-3-31966101-8_16 kautz, k. (2021). "editorial for the australasian journal of information systems 2021: volume 25, "australasian journal of information systems (25). https://doi.org/10.3127/ajis.v25i0.3853 kautz, k., and bjerknes, g. (2021). "focussing on humanistic outcomes on the sociotechnical axis of cohesion of the is discipline: user participation in distributed participatory design,". in insfran, e., gonzález, s. abrahão, s., fernández, m., c. barry, c., linger, h., lang, m., and schneider, c. (eds.), information systems development: crossing boundaries between development and operations (devops) in information systems (isd2021 proceedings). valencia, spain: universitat politècnica de valència. https://aisel.aisnet.org/isd2014/proceedings2021/methodologies/8/ kautz, k. and jensen, t. b. (2013). "sociomateriality at the royal court of is: a jester's monologue," information and organization (23:1), pp. 15-27. lamb, r., and kling, r. (2003). "reconceptualising users as social actors in information systems research," mis quarterly (27:2), pp. 197-235. https://doi.org/10.2307/30036529 langefors,b. (1980). "infological models and information user views," information systems (5), pp. 17-32. lee, a. (2001). "editor’s comments," mis quarterly (25:1), pp. iii-vii. marikyan, d., and papagiannidis, s. (2022). "task-technology fit: a review," in. papagiannidis, s. (ed), theoryhub book. https://open.ncl.ac.uk/theory-library/tasktechnology-fit.pdf monteiro, e., constantinides, p., scott, s., shaikh, m., and burton-jones, a. (2022). "editor’s comments qualitative research methods in information systems: a call for phenomenon-based problematization," mis quarterly (46:4), p. iii-xix. nygaard, k. (2002). "an emergency toolkit," forword to ciborra, c. (2002), the labyrinths of information – challenging the wisdoms of systems. oxford, uk: oxford university press. riemer, k., and johnston, r. b. (2012). "place-making: a phenomenological theory of technology appropriation," international conference on information systems 2012 proceeding. orlando, fl, usa. https://aisel.aisnet.org/icis2012/proceedings/socialimpacts/5 riemer, k., and johnston, r. b. (2017). "clarifying ontological inseparability with heidegger's analysis of equipment," mis quarterly (41:4), pp. 1059-1081. https://doi.org/10.25300/misq/2017/41.4.03 sarker, s., chatterjee, s., xiao, x., and elbanna, a. ( 2019). "sociotechnical axis of cohesion for the is discipline: its historical legacy and its continued relevance," mis quarterly (43:3), pp. 685-719. https://doi.org/10.25300/misq/2019/13747 weber. r (1997). ontological foundations of information systems. melbourne, vic, australia: coopers & lybrand and the accounting association of australia and new zealand. wiener, n. (1948). cybernetics: or the control and communication in the animal and the machine. cambridge, ma, usa: mit press. https://open.ncl.ac.uk/theory-library/task-technology-fit.pdf https://open.ncl.ac.uk/theory-library/task-technology-fit.pdf https://misq.org/skin/frontend/default/misq/pdf/v43i3/13747_rc_sarkerabstract.pdf https://misq.org/skin/frontend/default/misq/pdf/v43i3/13747_rc_sarkerabstract.pdf australasian journal of information systems kautz 2022, vol 26, editorial editorial for ajis 2022 11 ajis section editors in 2022 babak abedin axel korthaus helana scheepers arthur adamopoulos robert krimmer alex schmidt saeed akhlaghpour sherah kurnia manfred schoch michelle antero carmen leong paul scifleet allen au weng marc lim darshana sedera wasana bandara henry linger harminder singh jenine beekhuyzen matti mäntymäki mikko sipponen emilia bellucci sean maynard stefan stieglitz yee ling boo shah miah lars svensson deborah bunker milad mirbabaie angasana a. raffaele ciriello oliver mueller techatassanasoontorn tingru cui lemai nguyen janet toland duy dang-pham wietske van osch mark tolemann andreas drechsler neeraj pandey holly tootell yogesh dwivedi nikki panteli tuure tuunanen amany elbana jens peoppelbuss huan vo-tran anisha tj fernando konard peszynski kirsten wahlstrom julie fisher maria prokofieva philipp wunderlich asif gill jan recker nils urbach sigi goode kai riemer john venable raj gururajan michael rosemann blair wang jonas hedman christoph rosenkranz karl werder riitta hekkala bjoern ross william yeoh shirlee-ann knight matti rossi stan karanasios dilal saundage copyright: © 2022 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v26i0.4319 http://creativecommons.org/licenses/by-nc/3.0/au/ brief reflections on the journal, the concepts of systems, technologies, use, cybernetics, and the ajis volume 26 paper title (paper title style) australasian journal of information systems kumar & sharma 2024, vol 28, research article immersive environment for business metaverse received 23 may 2023, accepted 25 sept 2024 after 5 rounds of review with karlheinz kautz as accepting editor. 1 creating an immersive environment of metaverse for businesses harish kumar indian institute of management kashipur kundeshwari, kashipur, uttarakhand, india email: harish.kumar@iimkashipur.ac.in vijay kumar sharma tapmi school of business, manipal university jaipur jaipur, rajasthan, india abstract metaverse is an immersive environment that shares online space with users to interact with an artificial world through avatars or other digital representations. by leveraging the immersive features of metaverse, businesses can create new opportunities for revenue generation, customer engagement, collaboration, innovative products, services, or innovation. the development and governing of user interfaces and virtual environments are challenging. since the metaverse concept is emerging, numerous questions remain about designing, executing, and benefiting from it in the business environment. this study identifies enablers that can support creating an immersive metaverse environment for businesses to gain new business opportunities. the study deploys multi-criteria decision-making techniques, "total interpretive structural modelling," and "cross-impact matrix multiplication applied to classification" to establish cross-relationships among enablers. the study also provides a hierarchical roadmap for creating an immersive metaverse environment. recognizing which enablers hold the most influence in a hierarchy can lead to the strategic development of an immersive environment. this study will help researchers, policymakers, technology consultants, and business practitioners explore metaverse's most impacting enablers to gain competitive advantages and future opportunities for the business. keywords: metaverse; virtual reality; immersive experience; business opportunities; tismmicmac. 1 introduction in the immersive virtual environment of metaverse, individuals, locations, and objects from the real world are represented by digital avatars with which people can interact, communicate, work together, and meet (buhalis et al., 2023; polyviou & pappas, 2022). metaverse technology is a virtual environment where people can get immersed in an artificial world (bansal et al., 2022). the users engaged with the digital platform and formed social connections by using avatars in the immersive environment of metaverse (dwivedi et al., 2022). metaverse aims to give users an immersive, spatial experience by utilizing cutting-edge technologies (oleksy et al., 2023). the technological revolution can transform the real world beyond time limitations (kraus et al., 2022). practitioners and scholars are increasingly investigating and debating the societal consequences of meta platforms’ rollout of horizon worlds and its influence on various aspects of working culture and socialization (kraus et al., 2022; pamucar et al., 2022). in the mailto:harish.kumar@iimkashipur.ac.in australasian journal of information systems kumar & sharma 2024, vol 28, research article immersive environment for business metaverse 2 metaverse, people may freely move between the actual and virtual worlds while engaging in work, conversation, and socialization through cyber-virtual simulations in mixed-reality settings (li et al., 2023). frontiers of convergent technologies are shaping metaverse, which is a fully immersive networked world of virtual reality, ai, crypto, and blockchain (dwivedi et al., 2022). in a metaverse environment, virtual and real interactions can take many forms, ranging from fully immersive virtual experiences to hybrid experiences that blend elements of virtual and real worlds (li et al., 2023). businesses, governments, and societies would depend on the strength and scope of metaverse to exchange knowledge, offer services, and work together in ways that have never been possible (huynh et al., 2023). in the next decade, people will be surrounded by virtual objects and environments that will provide new interfaces and connections to time, places, and people with the emergence of human-computer interfaces (bale et al., 2022). people have already started spending time in a metaverse environment for education, training, shopping, office work, and entertainment (huynh et al., 2023). the metaverse often conducts business using digital assets like cryptocurrencies and non-fungible tokens (nft). with the metaverse environment, the manufacturing process, like acquiring raw materials, production, prototyping, and testing, could be sped up (li et al., 2023). more sophisticated technologies are needed to facilitate the emergence of metaverse and user-centered exploration. metaverse must provide user-centric activities and realistic experiences (zhao et al., 2022). with e-commerce and m-commerce shopping trends, the metaverse can be at the forefront of developing innovative and valuable customer interactions (buhalis et al., 2023). brands have been compelled to improve digital experiences so that consumers can disseminate information in retail sectors (bansal et al., 2022). while preserving the adaptability of a remote or mixed working culture, metaverse may also provide a way to improve the office environment. metaverse can help in designing virtual reality workspaces to promote conversations, interactions, and collaborative activities (buhalis et al., 2023; huynh et al., 2023). metaverse provides a digital platform where training, orientations, and corporate sessions can be performed for the employees to boost the efficiency and profitability of the business (kraus et al., 2022). to engage customers, businesses must allow them to connect with their brands, features, and market trends (huynh et al., 2023; prieto et al., 2022). metaverse may improve the shopping experience by refining search engine optimizations and advertisement metrics, understanding buying behavior, and using virtual avatars (prieto et al., 2022). therefore, business organizations should have a long-term plan on a priority basis to deploy metaverse technology (giang & shah, 2023). developing systems to monetize metaverse environments necessitates a broad range of skills, planning, development, marketing, and collaboration, regardless of the focused aspect being customers’ experience, adverts, purchasable objects, or games (prieto et al., 2022). the metaverse idea is currently in its early stages, and there are still many questions about how it will be implemented and what benefits it will have for businesses (hajian et al., 2024). the existing research has examined users’ intention to utilize metaverse in terms of perceived usefulness (aburbeian et al., 2022), embodied presence (zhang et al., 2022), informativeness and interactivity (hwang et al., 2022), perceived ubiquity, user innovativeness (salloum et al., 2023) and reducing cyber challenges (sebastian, 2022; hajian et al., 2024). australasian journal of information systems kumar & sharma 2024, vol 28, research article immersive metaverse for business 3 the metaverse environment encounters various challenges like governance (dwivedi et al., 2022), laws, regulations, and protections of intellectual property (lnenicka et al., 2024), technological scalability and interoperability between platforms (aysan et al., 2024). overcoming these challenges requires thoughtful planning and phased approaches (aladwan, 2024). besides, metaverse facilitates technological solutions, transforming traditional businesses (mkedder & das, 2024). the substantial initial investment needed for virtual and augmented reality technology, software, and infrastructure is one of the main obstacles (al-adwan, 2024). researchers have an opportunity to investigate how virtual worlds emerged, what makes virtual platforms successful or unsuccessful, and what the best practices and potential hazards are when creating and implementing virtual environments thanks to the metaverse. the metaverse enables social interactions among users to produce value and co-create experiences. adequate technologies are required for different stakeholders, like enterprises, customers, and users, to facilitate the experience and value co-creation in the virtual context (buhalis et al., 2022). businesses using metaverses to improve consumer or employee experiences should consider their target markets' infrastructure and equipment needs and the technical requirements for such offers. polyviou and pappas (2022) have introduced a framework for analyzing the metaverses and business value in terms of measurements and value, such as how organizations, users, governments, and platforms create, measure, or distribute the value generated using metaverses. the metaverse can inspire the development of innovative tools for various tasks and processes or inspire the production of new tools for tasks already completed. this can encourage the development of novel or improved solutions to current problems. digital innovations can create corporate value inside and outside organizations (polyviou & pappas, 2022). by leveraging the features of metaverse, businesses can create new opportunities for revenue generation, customer engagement, collaboration, and innovation (bale et al., 2022). the study has drawn a few research gaps based on previously published work analysis i) the enablers that can support the creation of an immersive environment of metaverse are briefly explored in previous literature, and ii) minimal resources are available for strategic planning and decision-making to create an immersive metaverse environment. from the identified research gaps, this research has come up with the following research questions: rq1: what enablers can support creating an immersive metaverse environment for businesses? rq2: how are these enablers interrelated for strategic planning and decision-making for business organizations to create an immersive environment for businesses? to resolve identified questions in any research, there are many approaches for gathering empirical data: surveys, interviews, ethnography, social network analysis, experimental design, time series data, and so on. conducting empirical qualitative research in a metaverse context opens new frontiers for researchers (hajian et al., 2024). the study appends to the body of knowledge regarding different enablers that drive the immersive environment of metaverse. the proposed hierarchical model (figure 1) appends resource-based theory (rbt) and holds ideas for the strategic planning for metaverse to improve computer-mediated shopping experiences to advance competitive business value. businesses can leverage the outcomes of this study to formulate strategies for interactive australasian journal of information systems kumar & sharma 2024, vol 28, research article immersive metaverse for business 4 virtual workspaces that can facilitate immersive experiences and interactions among stakeholders, including customers, employees, suppliers, and communities. businesses can adopt innovative approaches to generating revenue and new business value in a virtual dynamic environment. the study also advances uses and gratifications (u&g) theory to steer strategically to develop immersive shopping experiences tailored to individual preferences. the developed hierarchical model sets multiple directions for future investigation of immersive metaverse environments for creating business value and success. the categorization of enablers through micmac analysis provides a more profound investigation of how distinct enablers are interrelated and affect an immersive metaverse environment. the rest of the article's structure is as follows. section 2 deals with the literature review and the factors selected for the study. section 3 explains the methodology and tools used to carry it out. section 4 elaborates on the findings of the analysis of tism with micmac analysis. in section 5, there is a discussion, contributions, constraints, and potential avenues for future study, while in section 6, conclusions have been drawn. 2 review of literature 2.1 theories associated with metaverse and business the study thoroughly examined papers on the concept of an immersive environment, metaverse, and associated technologies in the context of businesses. the existing literature describes metaverse as a virtual environment using virtual reality through software agents and avatars (lichtenthaler, 2018). the term "metaverse" refers to merging physical and digital realms, allowing users to move between them quickly for multiple uses such as employment, learning and development, health, hobby exploration, and social interaction. (buhalis et al., 2023). the metaverse combines reality with the virtual environment through blockchain technology, avatars, and virtual reality headgear (prieto et al., 2022). the “technology acceptance model (tam)” is a standard theory in metaverse studies. previous studies on metaverse have been focused on implementation based on this theory. the “practice-based view (pbv) theory” can be utilized to clarify how the metaverse improves company performance (bromiley & rau, 2014). in metaverse retailing, u&g theory provides a valuable lens through which to analyze consumer behaviour dynamics. in metaverse, customers actively seek virtual experiences that suffice their retail needs and tastes rather than acting as passive recipients (hong & chow, 2023). according to the u&g hypothesis, people engage with media and virtual worlds to fulfill various social and psychological needs (hassouneh & brengman, 2014; chakraborty et al., 2023). using new technologies to create value requires addressing various social, environmental, and economic issues that may influence different societal stakeholders (polyviou & pappas, 2022). to study the potential of a metaverse environment for businesses' marketing capacities and procedures, kozlenkova et al. (2014) used resource-based theory (rbt) (barrera & shah, 2023). the scope and volume of economic activity and the environment in which it takes place incredibly increase by transacting in metaverses. rbt works on the assumption that companies are collections of resources and competencies. even when two organizations are operating in roughly the same markets or industries, rbt offers specific assumptions as to why it can occasionally be challenging to obtain the same high levels of economic value provided by another firm. rbt pays close attention to the creation and distribution of economic value. organizations can employ rbt to attain market parity (barney et al., 2021). australasian journal of information systems kumar & sharma 2024, vol 28, research article immersive metaverse for business 5 2.2 key enablers for the immersive environment of metaverse few articles have discussed metaverse benefits and its impact on business organizations (hajian et al., 2024). the metaverse is an opportunity that businesses must carefully examine (giang & shah, 2023). companies must move quickly to assess and integrate metaverse into their operations (huynh et al., 2023). virtual worlds have emerged as the predominant setting for metaverse-related research, consumer research, branding, and advertising (buhalis et al., 2022). considering that each component should have distinct qualities and not have the same meaning as another, fifteen enablers were identified (table 1) from the existing literature after discussing with experts (table 2) through the purposive sampling method. initially, the enablers were identified from various peer-reviewed literature studies to cover the concept's scope. a list was prepared for identified enablers consulted by experts (table 2). the experts could also share their comments and suggestions regarding the enablers. some enablers were merged and discarded based on the consensus of the experts to establish their level of agreement to reduce the duplicity and to increase a comprehensive understanding of the study. the detail of each enabler is presented below immersive experience (v1) metaverse enables immersive experiences (v1) in real and virtual environments (buhalis et al., 2023). by exploring virtual environments based on shared interests and preferences, users can collaborate to discover new and engaging experiences, enhancing their overall sense of immersion. metaverse technology can support immersive experiences using 3d identities, realistic sensory stimulation, and interactive communication through emerging technologies (capatina et al., 2024). digital twins and other interface features are supported by metaverse, allowing users to explore options in an immersive setting (dwivedi et al., 2022). in the metaverse, users can share their multimedia experiences and immersive content, influencing virtual and real-world settings by forming communities and exchanging digital content and interests (buhalis et al., 2022). organizational agility (v2) business agility implementation demands high complexity and stakeholder cooperation (huynh et al., 2023). an agile organization can adapt to changes in the environment and pivot its strategy and operations as needed to stay aligned with evolving user needs and trends. collaborative platforms in metaverse provide new levels of effectiveness, openness, and collaboration in flexible work settings (prieto et al., 2022). working side by side in virtual platforms substantially improves transparency and enables us to get over real-world physical limitations (bale et al., 2022). digital immersive environment (v3) digital immersive environments can be customized to create virtual environments that are relevant and engaging for a wide range of users. dimensions of immersiveness are interactive, visual, emotional, and narrative (bilgihan et al., 2024). users can have an engaging, realistic, and reliable experience in an immersive virtual environment (richter & richter, 2023). one significant barrier to businesses and consumers utilizing this new technology is the need for immersive platforms in metaverse (jaung, 2022; mkedder & das, 2024). cyber-physical systems (cps) can help converge the physical and virtual world using sensors, actuators, intelligence, and connectivity (martínez-gutiérrez et al., 2024). australasian journal of information systems kumar & sharma 2024, vol 28, research article immersive metaverse for business 6 unified business strategy (v4) a unified business strategy can help organizations scale their operations and offerings consistent with their vision and purpose. aharon et al. (2022) explored the areas where metaverse can improve customer experiences, streamline operations, or generate new revenue streams. various monetization strategies can be developed, such as selling virtual goods, offering subscriptions, charging for access to premium experiences, or leveraging advertising opportunities. monetization model (v5) emerging monetization prospects are likely one of the most critical developments in metaverse. businesses involved with metaverse could immediately gain the benefits of new models early on and gain market share (pamucar et al., 2022). platform-based services are one of the ways to monetize the metaverse. one stop metaverse shops can be created by businesses that combine basic platform functionality with new services that cover cognitive content moderation, connectivity services, ai, and edge computing (wei, 2022). organizations can integrate immersive environments into their business model. through the emergence of new industries, virtual collaboration can boost productivity. metaverse can transform the economy using virtual marketplaces and currencies to facilitate digital commerce (hajian et al., 2024). technological infrastructure (v6) to implement metaverse, organizations should focus on various issues like architecture, networking, communication, computation costs, interacting avatars, and online platforms (xu et al., 2023; hajian et al., 2024). the metaverse can be connected to the physical world using iot, sensors, blockchain technology, ai, edge computing, different internet-based devices, and digital twins (hajian et al., 2024). users can create native virtual environments that suit their interests using the digital environment of the metaverse. it is a collection of many virtual ecosystems that users can access (hwang & chien, 2022). nevertheless, the metaverse opens the potential for advancement when combined with augmented and virtual reality (giang & shah, 2023). researchers and practitioners can consider the financial effects of using information technology by examining the business value of the technology with a firmlevel focus (polyviou & pappas, 2022). visualization techniques (v7) visualization techniques can help users to analyze and understand data in virtual environments. data visualization techniques help identify patterns, trends, and relationships, enabling users to make intelligent decisions and interact more effectively with the virtual environment. zhao et al. (2022) suggested a framework based on graphics, visualization techniques, and interactions for metaverse and user-centric exploration. the graphics create an integrated environment fusing physical and virtual worlds by creating 3d sceneries, player characters, and non-player characters (npcs) for the metaverse's visual architecture. digital immersive literacy (v8) digital immersive literacy refers to the ability to understand and navigate digital immersive environments. users should also effectively navigate and understand immersive digital environments. skill sets like vr design, blockchain, and cybersecurity are crucial for immersive environments (bilgihan et al., 2024). australasian journal of information systems kumar & sharma 2024, vol 28, research article immersive metaverse for business 7 user-centric exploration (v9) zhao et al. (2022) proposed a framework to illustrate visualization methods, graphics, and interaction for creating metaverse's visuals and user-centric exploration (lnenicka et al., 2024). diverse communication channels and the ability to manipulate digital artifacts are two benefits of a virtual collaboration environment for remote teams (polyviou & pappas, 2022). effective interaction mechanisms and strategic planning are required to facilitate immersive user experiences in metaverse (lópez-cabarcos & piñeiro-chousa, 2024). regulatory compliance (v10) minimal performance standards and regulations are available for metaverse (lópez-cabarcos & piñeiro-chousa, 2024). cryptocurrencies and nft make trading virtual goods and services more accessible in the metaverse. although digital wallet metaverse networks facilitate social connections, individuals still need control over which portions of personal information are accessible to whom (zallio & clarkson, 2022). it must abide by the laws governing rights and tax matters (pamucar et al., 2022). wu et al. (2023) have provided a comprehensive taxonomy of financial and security risks associated with decentralized metaverse. regulatory needs and recommendations can influence metaverse systems and services' structure, functionality, and design. global laws, like the “general data protection regulation (gdpr)” of the "european union (eu)," may also be inefficient in regulating privacy concerns in metaverse (dwivedi et al., 2022). specific laws may apply to online content usage, data privacy, security, and virtual currencies, depending on the nation or location. these rules can be complicated for firms to comply with, and they put up barriers for new companies looking to enter the metaverse (mkedder & das, 2024). 3-d modelling (v11) metaverse is a vast network of 3-d virtual environments (bale et al., 2022). the technologies used to create graphically interactive, three-dimensional virtual representations on a computer are called 3d modeling (barrera & shah, 2023). it is a digital infrastructure that mimics the actual world by utilizing augmented reality, vr, blockchain technology, and social media. 3d models can create virtual reality, characters, objects, and environments. ethics and governance (v12) the metaverse and its avatars must have adequate and consistent system regulations with user identity and data privacy secured in contrast to the social media market, which is primarily unregulated and very dubious regarding its ethics and privacy guidelines (wei, 2022). a governance system is necessary to stop fraud, trolling, and other types of undesirable behavior in addition to content creation, such as events, games, and products (zallio & clarkson, 2022). ethical constraints should be considered throughout the evolution of metaverse (aysan et al., 2024). ethics and governance can create a metaverse environment that is safe, fair, and sustainable while fostering innovation, collaboration, and community building among users. to run the metaverse, governance should guarantee transparency and decision-making (dwivedi et al., 2022). the governance model usually comprises regulatory frameworks, interoperability protocols, and governance structure (aysan et al., 2024). australasian journal of information systems kumar & sharma 2024, vol 28, research article immersive metaverse for business 8 innovative engaging content (v13) metaverse technology has the potential to drive brands towards innovation and engage consumers in novel ways. how advertising systems interact with consumers will dramatically transform (hajian et al., 2024). theories on customer engagement and loyalty may need to be redefined considering ideas like "telepresence" and avatar-based personalization (bilgihan et al., 2024). data ownership and integrity (v14) businesses considering joining the metaverse as users or owners should be aware of two crucial privacy considerations: the platform privacy, which they intend to follow, and their privacy policies (kraus et al., 2022). this is difficult since there is no legislative structure till now. consumers may need to be more informed about what metaverse privacy entails. companies should take the initiative to implement successfully (zallio & clarkson, 2022). virtual and real interactions (v15) virtual communication can take different forms, such as communication, collaboration, socialization, and participation in virtual activities. innovative and engaging content can encourage users to explore virtual environments in a personalized, dynamic, and interactive way. any digitally created location or experience is considered an aspect of virtual space (hajian et al., 2024). "virtual reality" describes fully immersive experiences that substitute a simulated world for the real one. the process of adding digital data to the real world and enhancing it with computer-generated perceptual data is known as augmented reality (hajian et al., 2024). when a new product is released, consumers may wish to buy metaverse-specific products to have experience before purchasing it (huynh et al., 2023). businesses can guarantee that customers should have access to experiences by supporting technologies as they create identities in each space (bale et al., 2022). operating a business in the metaverse requires metaverse-capable talent, regardless of where the company is positioned. however, these fields have a distinct dearth of expertise because of wide gaps between software specializations and skilled experts (papagiannidis et al., 2022). the summary of selected vital enablers is shown in table 1. enabler description references (year) immersive experience (v1) a large-scale, open, communicative, constantly dynamic virtual setting with a real-time, fully functional world. prieto et al., 2022; buhalis et al., 2023; capatina et al., 2024 organizational agility (v2) the ability of an organization to reinvent itself, transform quickly, adapt, and grow in a dynamic, uncertain environment. bale et al., 2022; prieto et al., 2022; huynh et al., 2023 digital immersive environment (v3) the application of technology enhances augmented, virtual, and mixed and combines physical integration with digital one. buhalis et al., 2022; bilgihan et al., 2024 unified business strategy (v4) a unified approach combines conventional business methods like direct marketing with internet resources like blogs, online content, social networks, email, or advertising. aharon et al., 2022 monetization model (v5) users must pay charges to get the privilege of occupying space in the virtual world. service providers must develop innovative business strategies to monetize the metaverse for its users. pamucar et al., 2022; hajian et al., 2024 australasian journal of information systems kumar & sharma 2024, vol 28, research article immersive metaverse for business 9 technological infrastructure (v6) telecommunications providers may need to create more compact wireless cell networks and adopt new technologies that can recognize and react dynamically to various data types to better support early metaverse experiences and enable future experiences at scale. zallio & clarkson, 2022; xu et al. 2023 visualization technique (v7) present data in virtual reality and real-time data with aesthetically pleasing visual features. users have a more incredible experience of the information when they engage with it in a thorough and immersive way, which improves the effectiveness of the data visualization process. it interacts almost as naturally with threedimensional data due to clarity as 3d data objects. bale et al., 2022; zhao et al., 2022 digital immersive literacy (v8) "digital immersion" describes integrating the physical environment and informational virtual world. it involves using technologies that alter or enhance the physical environment, including ar, vr, extended, and mixed reality. kraus et al., 2022; bilgihan et al., 2024 user-centric exploration (v9) a digitally enhanced virtual platform where users can interconnect in a digital environment. papagiannidis et al., 2022; lnenicka et al., 2024 regulatory compliance (v10) to control the immoral use of technology, national legislation and regulatory bodies have set laws, rules, and regulations pertinent to a particular business based on the industry and nation in which it operates. zallio & clarkson, 2022; lópez-cabarcos & piñeiro-chousa, 2024; dwivedi et al., 2022; mkedder & das, 2024 3-d modelling (v11) any surface of an item, whether inanimate or alive, can be represented mathematically using coordinates in three dimensions by altering its edges, vertices, and polygons. an interconnected system of 3d virtual worlds called a "metaverse" focuses on social and commercial interaction. bale et al., 2022; barrera & shah, 2023 ethics and governance (v12) concerns about identity control, privacy, economic injustices, social accessibility, and freedom of expression make up most ethical issues in digital platforms. wei, 2022; zallio & clarkson, 2022; aysan et al., 2024 innovative engaging content (v13) virtual experiences can be divided into games, virtual locations, and virtual occasions. metaverse allows interaction with people in entirely new ways while advancing existing capabilities. hwang & chien, 2022; bilgihan et al., 2024; hajian et al., 2024 data ownership and integrity (v14) the control and usage rights of digital assets and property within these virtual settings are the subject of ownership problems in the metaverse. kraus et al., 2022; wei, 2022 virtual and real interaction (v15) a wholly realistic virtual environment where users might engage in real-world-like interactions with one other and virtual things. it would include cutting-edge technology, augmented reality, and virtual reality. hwang & chien, 2022; giang & shah, 2023; hajian et al., 2024 table 1. key enablers selected for the study 3 methodology 3.1 tism and micmac analysis this research adopted “multi-criteria decision-making (mcdm)” techniques to study selected enablers that can support an immersive environment of metaverse for business. the links between components and their complexity have been established by applying total interpretive structural modelling (tism) and cross-impact matrix multiplication applied to classification (micmac). a set of variables can use the "interpretive structuring modelling (ism)” technique to create a graphical representation of a complicated system. j.n. warfield australasian journal of information systems kumar & sharma 2024, vol 28, research article immersive metaverse for business 10 first invented this method in 1973; several authors have used its concept and analytical details in their research (dinesh, 2023; singh & sushil, 2013). the interpretive structural modelling identifies the logic of interrelationship within factors and is innovatively extended by the tism technique. the tism approach helps solve complex problems when many elements must be examined (bag, 2016). the following are the tism's methodology steps: • figuring out the elements that are important to the issue. literature reviews, surveys, or strategies for group problem-solving could all be used to accomplish this. tism participants include the following • applying pair-wise comparison to establish the contextual link between each factor. • designing a “structural self-interaction matrix (ssim)” of the selected factors (table 3) to establish pair-wise interactions between the selected factors. the survey and expert opinion approach could be employed to create the matrix. • the "initial reachability matrix” (table 4) is developed from ssim, and the “final reachability matrix” is by checking transitivity (table 5). • level-by-level partition of “final reachability matrix” (table 6). • building a conical form of reachability matrix (table 7). • developing a digraph based on the connections in a canonical matrix and eliminating all transitive links other than those with a distinct meaning. • transforming the resulting digraph into a tism model (figure 1) by substituting statements for element nodes and links for interpretation relations. micmac analysis was performed to classify factors according to how they can influence one another. in 1973, duperrin and godet developed micmac analysis to categorize enablers according to how much they can influence each other (attri et al., 2013). the micmac analysis is done using tism data as inputs. it classifies each component into four groups: independent, autonomous, dependent, and linkage (figure 2). 3.2 methodological selection the tism technique is interpretive, where the group's judgment decides whether and how the factors/enablers are related. it is a modeling technique to depict the relationships and overall structure as a hierarchical digraph model. the methodology helps to enforce order and direction on the complexity of the relationships among the enablers. tism and micmac have better practice advantages over other mcdm techniques for problems like immersive business metaverse environments. tism-micamc techniques support systematic ranking and rating of factors. the "technique for the order of prioritization by similarity to ideal solution (topsis)" can yield different rankings of elements compared to other mcdm approaches; however, it cannot preserve the consistency of the weight-assigning procedure. the analytical hierarchy process (ahp) is criticized since it tends to rank reversal. in addition, ahp creates a rigid hierarchy in which elements at the same level are assumed to be independent—the incapacity of anp to provide weighted consequences among connected components. "elimination and choice expressing reality (electre)" is often used to choose, rank, and arrange options before eliminating them. the "preference ranking organization method for enrichment evaluations" (promethee) is better for rating processes from best to worst. promethee only offers a partial ranking of the behaviors. only the cause-and-effect australasian journal of information systems kumar & sharma 2024, vol 28, research article immersive metaverse for business 11 relationships between the components are shown by dematel. the relationships between the variables are shown using graph theory, albeit the accuracy of the graphs' edge directions is doubtful. interpretive structural modelling (ism) captures complexity dynamics but does not examine inverse linkages. based on the elements' driving forces and dependencies, tismmicmac discloses their inter-contextual relationship. the tism-micamc has been used in several previous studies, like analysis of factors for digital supply chains (khan et al., 2021; sharma & kumar, 2023) and industry 4.0 leadership (bianco et al., 2021). 3.3 survey and details of respondents the research is an empirical study driven by twenty-seven experts' input to create a ssim matrix. a structured questionnaire was sent to ninety-five senior executives of companies from september 2022 to january 2023 through email in the form of a 'vaxo' sheet along with a brief description of the purpose of the research, selected factors, and their description to establish contextual relationships. the experts were selected through a profile search over a linkedin platform and have at least five years of relevant experience working as senior executives, such as senior managers, vice presidents, technical consultants, it officers, and business unit heads. out of the targeted 95 respondents (based on the purposive sampling method), only 29 respondents reverted to the filled 'vaxo' form. out of 29 responses received, only 27 responses (table 2) were found complete. based on responses, a 50% majority response approach has been adopted to establish contextual relationships for ssim. gender male 15 female 12 age 25-35 5 36-50 8 above 50 14 education graduation 20 post graduation & phd 7 occupation vice president 3 technical consultant 9 it officers 6 business unit heads 5 marketing professionals 4 table 2. respondents' (experts) demographic information 4 results and findings 4.1 4.1 analysis and development of hierarchical model based on responses received from the respondents, a "structural self-interaction matrix” was developed (table 3). a 50% majority response approach has been adopted to establish contextual relationships (singh & sushil, 2013). v= where 𝑉𝑖 influences 𝑉𝑗 a= where 𝑉𝑗 influences 𝑉𝑖 x= where 𝑉𝑖and 𝑉𝑗 both influence each other's o= where no relationship between 𝑉𝑖and 𝑉𝑗 australasian journal of information systems kumar & sharma 2024, vol 28, research article immersive metaverse for business 12 v1 v2 v3 v4 v5 v6 v7 v8 v9 v10 v11 v12 v13 v14 v15 v1 a x a o a a a a a a o a a a v2 v x v v v v v v v v o o v v3 a a a a a a a a o a a a v4 v v v v v v v v o v v v5 v v o o x v x v v v v6 v v v a v a v x v v7 v v a v o v a x v8 x a a a x o a v9 a a o x a a v10 v x v v v v11 a v a x v12 v v v v13 a a v14 v v15 table 3. “structural self-interaction matrix” (ssim) the “initial reachability matrix (irm)” was created from ssim by replacing it with 0 or 1 as per the rules of transformation (table 4, appendix). the transitivity rule has also been tested on the initial reachability matrix and if transitivity has been introduced as 1* (table 5, appendix). the level partition was accomplished to comprehend the level-wise placement of determinants (kumar et al., 2019). the antecedent and reachability set (table 6, appendix) were computed from the final matrix of reachability (table 5). the canonical matrix (table 7, appendix) is constructed after identifying all the portioned levels. the canonical matrix (table 7, appendix) is a matrix form that integrates all the factors that have been grouped into the same portioned level, helps validate level partitions, and aids in developing the tism digraph. based on the level partitioning (table 6) and conical matrix (table 7), all enablers were arranged at their respective partitioned levels to form the tism digraph, and direct links were then created according to the cross-relationship identified within the final reachability matrix. in the digraph, the important transitive links are displayed (figure 1). 4.2 cross-impact relationships the cross-impact factors were analysed using the developed digraph (figure 1). unified business strategies (v4) provide a clear and consistent direction for the organization to decide on changing circumstances. when the market or business environment changes, the companies can do so more rapidly and effectively. organizational agility (v2) identifies new opportunities for businesses. by involving a diverse range of stakeholders in developing strategies, the organization can ensure that it is aligned with the needs and priorities of the entire organization. australasian journal of information systems kumar & sharma 2024, vol 28, research article immersive metaverse for business 13 an agile organization can quickly understand new regulations, implement the necessary changes, and avoid compliance failures. in metaverse situations, organizational agility can be crucial to meeting regulatory compliance. by being flexible, responsive, and collaborative, organizations can more effectively navigate the complex regulatory landscape and comply with regulations promptly and efficiently. the unified business strategies influence the adoption of technological infrastructure in the metaverse through the development of interoperable systems and standards. an agile organization should establish robust data security procedures and strict policies to safeguard data from unauthorized access, misuse, and disclosure. to develop metaverse strategies and to encourage consumers to engage in the metaverse, every firm needs to build trust in business users and systems (golf-papez et al., 2022). ethical frameworks help to define the values and principles of an organization. ethics and governance (v12) practices can help businesses identify and manage risks associated with regulatory compliance (v10). organizations can minimize the risk of compliance failures by having robust risk management processes. monetization through advertising is a common practice in metaverse environments. businesses can adjust quickly and successfully as the market or business environment changes (aharon et al., 2022). regulatory compliance often requires organizations to assess and mitigate risks related to their operations. regulatory compliance related to advertising can impact how organizations monetize their products and services. the chosen monetization model (v5) can influence the way 3-d modelling is approached and utilized in the metaverse. business models for the metaverse can expand on the benefits of growing digital services (weking et al., 2023). regulations related to data privacy can impact how organizations collect, use, and share personal data in metaverse environments. regulations that promote open standards and data interoperability can impact data ownership (v14) in metaverse environments. the stakeholders must work together to build a reliable metaverse and derive value from it (golf-papez et al., 2022). regulations like open virtual reality standards can promote interoperability between different online platforms and help to ensure that data can be easily shared and owned across different platforms to promote trust with customers and regulators. adhering to ethical principles and establishing governance structures, organizations can ensure that visualization techniques (v7) are used to build user credibility. the hardware, software, and network technologies used to create and run the virtual environment are considered the metaverse's technological infrastructure (v6). the design of hardware and software systems must be robust and secure, with appropriate measures to prevent data breaches and cyber-attacks. the developers can use technologies like 3d modelling, mixed reality, blockchain, ai, social platforms, electronic shopping, digital twins, online games, and big data (barrera & shah, 2023; jaung, 2022). ai can produce intelligent virtual characters to interact with users for a more immersive experience. the technological infrastructure of metaverse is essential for creating innovative and engaging content (v13). the network architecture must also be reliable, with high bandwidth and low latency, to support real-time user interactions. the quality and speed of the network can influence the ability to establish and maintain virtual connections, which can impact the quality and reliability of virtual interactions (v15). data accuracy is also essential to the effectiveness of digital immersive experiences. the completeness of data is vital for creating a realistic and immersive experience. australasian journal of information systems kumar & sharma 2024, vol 28, research article immersive metaverse for business 14 (v2) organizational agility (v12) ethics and governance (v10) regulatory compliances (v4) unified business strategies quick response to changes in business stay aligned with users needs and trends reduce compliance failure implement necessary changes strict security measures monetizing products and services(v5) monetization model ethical and transparent compliances build trust enforces standards of conduct build trust and credibility among users creation and use of prototyping (v14) data ownership and integrity (v6) technological infrastructure privacy and data protectioncybersecurity frameworks data accuracy, consistency and reliability robust infrastructure to maintain data integrity (v3) digital immersive environment (v1) immersive experience enhances business performancebranching narratives virtual showrooms provides a realistic and interactive way to explore products and services (v7) visualizations techniques (v15) virtual and real interactions shaping virtual interactive experience data exchange and interoperability creates digital models of objects reduces cost of design and prototyping (v11) 3-d modelling creating virtual objects and environments through data allows users to interact (v8) digital immersive literacy (v9) user-centric explorationfosters social interactions co-creation with users(v13) innovative engaging content diverse and collaborative experience increases navigation abilities among users development of interoperable system identify new opportunities to grow and evolve creation of custom avatars, clothing and other objects visually interactive and aesthetically sound environment complete and realistic experience dynamic environment to manage data quality figure 1. tism digraph analyzing cross relationships among enablers for creating an immersive environment of metaverse for businesses. visualization techniques (v7) can increase users' sense of presence and engagement. 3d graphics, virtual reality, and animation can give customers a more dynamic and realistic experience, making virtual interactions more compelling and engaging. data visualization techniques can help identify trends, patterns, and relationships to enable users to make informed choices and communicate more successfully with the virtual environment. australasian journal of information systems kumar & sharma 2024, vol 28, research article immersive metaverse for business 15 visualization techniques can be used to create realistic and believable environments that accurately represent the real world. ai-based agents populate digital immersive environments (v13) (zallio & clarkson, 2022). techniques like photorealistic rendering, motion capture, and 360-degree video can be applied to produce exciting experiences that feel like the real thing. users' sense of presence and immersion may be improved as a result. 3d modelling (v11) facilitates visually interactive representations of objects (barrera & shah, 2023). 3-d modelling allows the creation of custom avatars, clothing, and other objects in metaverse. the accuracy of the 3d models can impact the realism and effectiveness of virtual interactions. virtual interactions can greatly influence user-centric exploration, such as personalized recommendations and notifications that can guide users toward virtual environments and experiences that align with their interests. touch-sensing interactions can empower the customers (golf-papez et al., 2022; jaung, 2022). the integration of online channels enlarges firms’ innovation performance. innovative and engaging content (v13) can facilitate social interaction in the metaverse, enabling users to share experiences and explore virtual environments. innovative and engaging content can stimulate users' creativity and stimulate them to create and share their content in the metaverse. immersive technologies strongly influence gestures of communication (zallio & clarkson, 2022). digitally literate users are more likely to have an easier time navigating virtual environments, which can encourage user-centric exploration (v9). by verbally commanding voice assistants and managing their involvement and interactivity, users may personalize their experience (buhalis et al., 2023). user-centric exploration can increase user engagement with immersive environments. metaverse enables immersive experiences (v1) in virtual and real environments (buhalis et al., 2023). by exploring virtual environments based on shared interests and preferences, users can collaborate to discover new and engaging experiences, enhancing their overall sense of immersion. digital immersive environments (v3) can be customized to create relevant and engaging virtual environments for many users. 4.3 micmac analysis for each enabler, dependence (dp) is calculated as column sum, and the driving power (dr) is determined as row sum (table 4). all the factors are divided into autonomous, dependent, linkage, and independent factors using both driving and dependent power. in this study, as there are fifteen factors, the 7.5 scale is the midpoint (i.e., 15/2= 7.5 scales), and the graph is divided into four quadrants (figure 2). i. cluster i (autonomous variables): weak dr and dp represented by this area. any variable here signifies a weak variable. figure 2 shows that no variables are visible in the first cluster, i.e., cluster i, and that selected factors are well chosen. ii. cluster ii (dependence variables): this cluster indicates all variables with intense dp and weak dr. v1, v3, v8, v9, and v13 are five variables are there. these enablers are situated at the top level of the proposed hierarchy model. other variables influence these due to weak dr. iii. cluster iii (linkage variables): this cluster has strong dr and dp. all factors that fall under this cluster are linkages and actions taken on these factors influence all other factors. here, v7, v11, and v15 appear as linkage factors. australasian journal of information systems kumar & sharma 2024, vol 28, research article immersive metaverse for business 16 iv. cluster iv (independent variables): this cluster indicates weak dp and strong dr. here, v2, v4, v5, v6, v10, v12, and v14 appear as independent factors influencing the hierarchal model. figure 2. micmac analysis 5 discussions the proposed model (figure 1) discovers relationships among all factors. organization agility (v2), unified business strategy (v4), monetization model (v5), regulatory compliance (v10), ethics and governance (v12), technological infrastructure (v6), and data ownership and integrity (v14) are seven crucial factors that influence changes in remaining factors that affect businesses. visualization techniques (v7), virtual and tangible interactions (v15), and 3-d modelling (v11) are linkages. other factors are dependent, and further change in these variables is a result of changes in independent and linkage enablers. it is pertinent that businesses must develop a unified business strategy (v4) and organizational agility (v2). unified business strategies provide clear direction, improve decision-making, enhance resource allocation, promote better communication, and foster greater flexibility. the organizations can develop strategies that are aligned with their goals and priorities. organizations can ensure their strategies by identifying potential risks and developing contingency plans. organizations with high agility can collaborate with regulators, customers, and suppliers to implement necessary changes and ensure compliance. an agile organization can innovate to develop new solutions that comply with regulations. the businesses can leverage their unified strategies (v4) to address important ethical and social considerations. ethical solid and governance (v12) practices can ensure regulatory compliance (v10). by establishing a clear ethical framework, implementing effective governance practices, managing risks, promoting transparency and accountability, and providing employee training, organizations can improve their compliance efforts and minimize the risk of compliance failures. regulatory compliance (v10) often requires organizations to adopt and enforce standards of conduct that promote ethical behavior. organizations must be aware of australasian journal of information systems kumar & sharma 2024, vol 28, research article immersive metaverse for business 17 the regulatory landscape and comply with relevant regulations related to the monetization model (v5), data privacy, intellectual property, open standards, and security. compliance with regulations can promote ethical behavior, transparency, risk management, and stakeholder engagement, all essential components of ethical governance. the possibility for unlawful, immoral, or unethical behavior in virtual environments requires appropriate legislation; these environments also provide ethical and legal difficulties (buhalis et al., 2023). individuals and institutions can create, promote, and offer new goods and services with the help of new business models in metaverse (barrera & shah, 2023). as enhanced activities in the metaverse increase, vr hardware and in-game advertising revenue considerably rise (huynh et al., 2023). investments in such business models and technology are being made to create virtual platforms that could combine important online industries like e-tail business, social networking engagement, education, entertainment sectors, ai, and big data (jaung, 2022). the technological infrastructure (v6) of the metaverse is closely tied to data integrity (v14). the system's speed, responsiveness, and stability can affect the ability to perform virtual activities. organizations must prioritize data ownership and integrity (v14) when designing and building digital immersive environments (v3) to ensure users can effectively navigate, understand, and create content. the visualization techniques (v7) play a critical role in enabling and enhancing virtual interactions (v15) by allowing a more immersive, collaborative, and engaging virtual environment to support effective communication, collaboration, and decision-making. by using the proper visualization techniques, organizations can create realistic, straightforward, interactive, and aesthetically pleasing environments, helping to enhance the overall user experience and engagement. 3-d modelling (v11) is critical to many virtual environments and can support various virtual interactions (v15). 3-d modelling technology can create different environments, objects, and virtual try-ons in metaverse (golf-papez et al., 2022). by creating immersive user experiences (v1), augmented reality improves virtual reality's 3-d real-time environment rendering capabilities (barrera & shah, 2023). customers purchase activities in virtual worlds resembling the metaverse for various reasons, including discovery, self-expression, sociability, escapism, and amusement (barrera & shah, 2023). virtual interactions (v15) can facilitate collaborative exploration in the metaverse. virtual interactions are critical in facilitating user-centric exploration in the metaverse, enabling people to engage in a more personalized, collaborative, and interactive virtual environment. innovative and engaging content (v13) can significantly impact user-centric exploration (v9). digital immersive literacy (v8) can be critical in shaping user-centric exploration (v9) in the metaverse. investments in metaverse-related business and technology are being made to integrate significant internet industries, including social networking, online retail, entertainment, education, and big data with artificial intelligence (barrera & shah, 2023). user-centric exploration (v9) can enhance user engagement and facilitate more diverse experiences in metaverse. digital immersive environments (v3) provide the tools and features to make immersive experiences possible. by incorporating advanced graphics, natural interaction, personalized content, and engaging storytelling, developers can create compelling and immersive experiences. with immersive 3-d virtual worlds, the metaverse may offer a new way of interacting with the natural world (jaung, 2022). australasian journal of information systems kumar & sharma 2024, vol 28, research article immersive metaverse for business 18 5.1 the study's theoretical contributions the proposed hierarchical model (figure 1) appends resource-based theory (rbt) and holds ideas for the strategic planning for metaverse to improve computer-mediated shopping experiences to advance competitive business value (bromiley & rau, 2016). this study identifies and explores the enablers for creating an immersive metaverse experience (rq1). the developed hierarchical model (figure 1) sets multiple theoretical directions for future investigation of immersive metaverse environments for successful business. in addition, the study presents a hierarchical road map for developing an immersive metaverse ecosystem that would provide ways for generating value, engaging users, cooperating, and innovating new goods and services in a cyber-physical world. the examination of dependent (v1, v3, v8, v9, v13), linkage (v7, v11, v15), and independent factors (v2, v4, v5, v6, v10, v12, v14) adds to the theoretical distinction of these enablers. this classification thoroughly analyzes how distinct enablers interact and affect an immersive metaverse environment (rq2). to provide a deeper understanding of the cross-interaction between enablers and their impacts on building an immersive metaverse environment for businesses, this model (figure 1) can also be used as a point of reference for future studies. as suggested in resource-based theory (rbt) (bromiley & rau, 2016), resources like r&d capabilities, logistics, brand management, low-cost processes, suppliers, customer demand, technology change, planning, managing, and coordinating systems develop the dynamic capabilities and competence of a business organization. businesses can get an edge by continuously reorganizing or reconfiguring various resource classes and developing new applications to satisfy consumer demand and generate value from many angles. (utami & alamanos, 2022). digital immersive environment (v3) (outcome of this study) promotes “value co-creation” and “technology-enhanced experience” (buhalis et al., 2022) for the customers. e-retailers may gain value through the sales of virtual products (polyviou & pappas, 2022). the "technology acceptance model" and "innovation adoption theories" are the cornerstones of metaverse research, which will build upon them to create metaverse-focused research (buhalis et al., 2022). the research additionally supports the u & g theory as a strategic framework for creating immersive retail experiences customized to individual preferences. based on customers' aspirations to use the metaverse continuously, the u & g theory forecasts the future utilization of these immersive retailers (chakraborty et al., 2023). the researchers can further explore various enablers discussed in the proposed hierarchical model, including user-centricity, collaboration, training, shopping experience, work culture, entertainment, and communication. the study explores a new dimension to user-centric experience (v9) by allowing businesses to design dynamic and immersive experiences for their customers. this could entail trying out different marketing strategies or creating chances to create new instruments or procedures to help employee-driven digital breakthroughs (polyviou & pappas, 2022). the study also sets a research direction for the knowledge segment to study the influences of digital environments on human behavior, cognition, and emotions. to understand the impact of the business model for metaverse adoption, innovation adoption research should look at the metaverse environment holistically, taking into account aspects such as entrepreneurship, food and beverage, human resources, strategy, management, and strategy (buhalis et al., 2022) to identify innovative ways of generating economic value (barney et al., 2021). australasian journal of information systems kumar & sharma 2024, vol 28, research article immersive metaverse for business 19 5.2 practical contributions and implications recognizing the most influential enablers can inform the strategic creation of an immersive environment. the hierarchy of these enablers (figure 1) must be considered by companies seeking to enter the metaverse, as doing so can significantly increase their chances of developing an immersive environment that is successful. the proposed model (figure 1) presents several enablers, such as business models, user experience, social interaction, governance, regulation, and innovation, to develop an immersive metaverse environment. to support the redefinition of human work-related processes, government associations (v12) may capitalize on these opportunities and create laws that promote the adoption of metaverses at workplaces (polyviou & pappas, 2022). virtual worlds can enhance the collaborative experience of workers in remote teams. businesses must plan strategically to use metaverse to generate emerging business opportunities, economic value, customer engagement, shopping experiences, and designing new products or services. in addition, users must acquire new abilities to facilitate the metaverse transfer and guarantee the seamless and efficient use of metaverses (polyviou & pappas, 2022). the conventional customer base will remain crucial in the banking, insurance, financial services, manufacturing, retail, and tourism industries. for specific business use cases, these sectors will either create their own metaverse for consumers or join/rent out others (giang & shah, 2023). metaverse has significant practical implications for businesses across various industries. metaverse allows businesses to engage clients innovatively through virtual services and goods. visualization techniques (v7) can increase users' sense of presence and engagement. 3d graphics, virtual reality, and animation can give users a more realistic and interactive environment. virtual interactions (v15) can significantly influence user-centric exploration, like personalized recommendations and notifications. unified business strategies (v4) provide a clear and consistent direction for the organization to make decisions and act quickly in response to changing circumstances. the lesson learned is that service providers must increase their client base and rely less on their existing clientele (zallio & clarkson, 2022). using the metaverse, businesses can develop immersive and interactive training programs that help to improve employee learning and retention. by leveraging the immersive features of metaverse, businesses can create new opportunities and value for revenue generation, customer engagement, collaboration, new products, services, or innovation. businesses are required to have long-term technical strategies to take advantage of the metaverse. the proposed hierarchical model and micmac analysis suggest developing strategies for successfully incorporating emerging technologies into business to maximize business benefits using a metaverse environment. businesses can leverage the outcomes of this study to create interactive virtual workspaces. businesses can learn important information about customer preferences by tracking users' behavior and interactions in a virtual environment. and behaviors to identify new business opportunities. individuals and institutions can create, promote, and sell innovative products and services in metaverse. one can get new prospects and greater autonomy by customizing business offerings using metaverse and co-creating business value (buhalis et al., 2023). australasian journal of information systems kumar & sharma 2024, vol 28, research article immersive metaverse for business 20 5.3 limitations & future scope the study analyzed 15 enablers for strategic planning for a technology-illusive metaverse environment for businesses. there might be more factors to address the digital immersive environment by viewing from other domains of business organization development, capability building, cyber security, revenue generation, and digital transformation perspectives. adding or subtracting any factor can change the hierarchy. it is challenging to rank among enablers in the tism hierarchical process. future studies may focus on several areas of metaverse applications in businesses. as businesses begin to use the metaverse, research will be needed on the most effective business models to generate revenue within the metaverse. metaverse can be advanced with the help of generative ai, which can significantly speed up its expansion and advancement. future research may be conducted to optimize user experiences in a metaverse environment, including interface design, user engagement, and personalization. future research could explore different models for governing the metaverse and the potential regulatory frameworks that may be necessary for developing and launching innovative products within the metaverse. 6 conclusions this study incorporates earlier investigations into the metaverse conceptualization. the importance of metaverse is increasing because of the new opportunities and value it offers to enterprises for global virtual communication and collaboration. the enablers that affect metaverse's success and value creation in businesses are this study's central subject. the enablers have been identified from literature studies. experts were consulted to finalize the selected enablers and provide ssim input. as an outcome of the tism process, a hierarchical model is proposed to describe how various factors are linked to each other and their level of importance. the results would assist business practitioners in understanding enablers that influence the successful incorporation of technology into business processes and assist in maximizing the benefits of a metaverse environment. organizational agility (v2) can be critical in meeting regulatory compliance in metaverse environments. the chosen monetization model (v5) can influence the way 3-d modeling (v11) is approached and utilized in the metaverse. 3-d modelling also facilitates visually interactive representations of objects. the network's speed can influence the reliability of virtual interactions (v15). organizations must be aware of the regulatory landscape and comply with relevant regulations related to the monetization model (v5), data privacy, intellectual property, open standards, and security. maintaining the confidentiality of data in the virtual environment demands data integrity (v14). avatar realism has significantly been improved using real-time recognition, tracking, and simulation technologies. governments will need to introduce rules and policies to deal with the new issues brought up by the metaverse. user-centric exploration (v9) can increase user engagement with immersive environments (v3). immersive experiences (v1) can be tailored to facilitate users with unique and meaningful experiences. digital immersive literacy (v8) navigates digital immersive environments. metaverse's technological infrastructure (v6) is essential for creating innovative and engaging content (v13) to facilitate social interactions and sharing experiences. australasian journal of information systems kumar & sharma 2024, vol 28, research article immersive metaverse for business 21 for metaverse, a robust and advanced technology infrastructure (v6) is required, including high-speed networking capabilities, powerful servers, data centers, ai-driven analytics, apis tailored to 3d environments, real-time data processing, and management. personalized recommendations, immersive experiences, and interactive components can all help to extend customer engagement and boost the possibility of future sales conversion. the businesses may need to navigate interoperability challenges when connecting with other businesses or users within the metaverse. this may make creating a seamless and integrated experience across multiple platforms challenging. references aburbeian, a. m., owda, a. y., & owda, m. (2022). a technology acceptance model survey of the metaverse prospects. ai, 3(2), 285-302. doi.org/10.3390/ai3020018 aharon, d. y., demir, e., & siev, s. (2022). real returns from the unreal world? market reaction to metaverse disclosures. research in international business and finance, 63, 101778. doi.org/10.1016/j.ribaf.2022.101778 al-adwan, a. s. (2024). the government metaverse: charting the coordinates of citizen acceptance. telematics and informatics, 102109. doi.org/10.1016/j.tele.2024.102109 attri, r., grover, s., dev, n., & kumar, d. (2013). an ism approach for modelling the enablers in implementing total productive maintenance (tpm). international journal of system assurance engineering and management, 4(4), 313–326. doi.org/10.1007/s13198-012-0088-7 aysan, a. f., gozgor, g., & nanaeva, z. (2024). technological perspectives of metaverse for financial service providers. technological forecasting and social change, 202, 123323. doi.org/10.1016/j.techfore.2024.123323 bag, s. (2016). building theory of green procurement using fuzzy tism and fuzzy dematel methods. international journal of applied management sciences and engineering (ijamse), 3(2), 21-49. bale, a. s., ghorpade, n., hashim, m. f., vaishnav, j., & almaspoor, z. (2022). a comprehensive study on metaverse and its impacts on humans. advances in humancomputer interaction, 2022, 1–11. doi.org/10.1155/2022/3247060 bansal, g., rajgopal, k., chamola, v., xiong, z., & niyato, d. (2022). healthcare in metaverse: a survey on current metaverse applications in healthcare. ieee access, 10, 119914119946. doi.org/10.1109/access.2022.3219845 barney, j. b., ketchen jr, d. j., & wright, m. (2021). resource-based theory and the value creation framework. journal of management, 47(7), 1936-1955. doi.org/10.1177/01492063211021655 barrera, k. g., & shah, d. (2023). marketing in the metaverse: conceptual understanding, framework, and research agenda. journal of business research, 155, 113420. doi.org/10.1016/j.jbusres.2022.113420 bianco, d., godinho filho, m., osiro, l., & ganga, g. m. d. (2021). unlocking the relationship between lean and industry 4.0 leadership competencies: an ism/fuzzy micmac approach. ieee transactions on engineering management. doi.org/10.1109/tem.2021.3069127 bilgihan, a., leong, a. m. w., okumus, f., & bai, j. (2024). proposing a metaverse engagement model for brand development. journal of retailing and consumer services, 78, 103781. doi.org/10.1016/j.jretconser.2024.103781 bromiley, p., & rau, d. (2014). towards a practice-based view of strategy. strategic management journal, 35(8), 1249-1256. doi.org/10.1002/smj.2238 australasian journal of information systems kumar & sharma 2024, vol 28, research article immersive metaverse for business 22 bromiley, p., & rau, d. (2016). missing the point of the practice-based view. strategic organization, 14(3), 260-269. doi.org/10.1177/1476127016645840 buhalis, d., leung, d., & lin, m. (2023). metaverse as a disruptive technology revolutionising tourism management and marketing. tourism management, 97, 104724. doi.org/10.1016/j.tourman.2023.104724 buhalis, d., lin, m. s., & leung, d. (2022). metaverse as a driver for customer experience and value co-creation: implications for hospitality and tourism management and marketing. international journal of contemporary hospitality management. doi.org/10.1108/ijchm-05-2022-0631 capatina, a., patel, n. j., mitrov, k., cristea, d. s., micu, a., & micu, a. e. (2024). elevating students’ lives through immersive learning experiences in a safe metaverse. international journal of information management, 75, 102723. doi.org/10.1016/j.ijinfomgt.2023.102723 chakraborty, d., polisetty, a., & rana, n. p. (2024). consumers' continuance intention towards metaverse-based virtual stores: a multi-study perspective. technological forecasting and social change, 203, 123405. doi.org/10.1016/j.techfore.2024.123405 dinesh, k. k. (2023). fuzzy predictive modeling for the hierarchical structure of strategic innovation management. ieee transactions on engineering management. doi.org/10.1109/tem.2022.3232156 dwivedi, y. k., hughes, l., baabdullah, a. m., ribeiro-navarrete, s., giannakis, m., al-debei, m. m., … wamba, s. f. (2022). metaverse beyond the hype: multidisciplinary perspectives on emerging challenges, opportunities, and agenda for research, practice and policy. international journal of information management, 66(july), 102542. doi.org/10.1016/j.ijinfomgt.2022.102542 golf-papez, m., heller, j., hilken, t., chylinski, m., de ruyter, k., keeling, d. i., & mahr, d. (2022). embracing falsity through the metaverse: the case of synthetic customer experiences. business horizons, 65(6), 739–749. doi.org/10.1016/j.bushor.2022.07.007 hajian, a., daneshgar, s., sadeghi, k., ojha, d., & katiyar, g. (2024). from theory to practice: empirical perspectives on the metaverse's potential. technological forecasting and social change, 201, 123224. doi.org/10.1016/j.techfore.2024.123224 hassouneh, d., & brengman, m. (2014). a motivation-based typology of social virtual world users. computers in human behavior, 33, 330-338. doi.org/10.1016/j.chb.2013.08.012 ho, s. p. s., & chow, m. y. c. (2023). the role of artificial intelligence in consumers’ brand preference for retail banks in hong kong. journal of financial services marketing, 1-14. doi.org/10.1057/s41264-022-00207-3 hwang, g. j., & chien, s. y. (2022). definition, roles, and potential research issues of the metaverse in education: an artificial intelligence perspective. computers and education: artificial intelligence, 3(april), 100082. doi.org/10.1016/j.caeai.2022.100082 hwang, i., shim, h., & lee, w. j. (2022). do an organization’s digital transformation and employees’ digital competence catalyze the use of telepresence?. sustainability, 14(14), 8604. doi.org/10.3390/su14148604 jaung, w. (2022). digital forest recreation in the metaverse: opportunities and challenges. technological forecasting and social change, 185, 122090. doi.org/10.1016/j.techfore.2022.122090 khan, s. a., kusi-sarpong, s., gupta, h., arhin, f. k., lawal, j. n., & hassan, s. m. (2021). critical factors of digital supply chains for organizational performance australasian journal of information systems kumar & sharma 2024, vol 28, research article immersive metaverse for business 23 improvement. ieee transactions on engineering management. doi.org/10.1109/tem.2021.3052239 kozlenkova, i. v., samaha, s. a., & palmatier, r. w. (2014). resource-based theory in marketing. journal of the academy of marketing science, 42, 1-21. doi.org/10.1007/s11747-0130336-7 kraus, s., kanbach, d. k., krysta, p. m., steinhoff, m. m., & tomini, n. (2022). facebook and the creation of the metaverse: radical business model innovation or incremental transformation? international journal of entrepreneurial behaviour and research, 28(9), 52– 77. doi.org/10.1108/ijebr-12-2021-0984 li, k., cui, y., li, w., lv, t., yuan, x., li, s., ... & dressler, f. (2023). when internet of things meets metaverse: convergence of physical and cyber worlds. ieee internet of things journal, vol. 10, no. 5, pp. 4148-4173. doi.org/10.1109/jiot.2022.3232845 lichtenthaler, u. (2018). the world’s most innovative companies: a meta-ranking. journal of strategy and management, 11(4), 497–511. doi.org/10.1108/jsma-07-2018-0065 lnenicka, m., rizun, n., alexopoulos, c., & janssen, m. (2024). government in the metaverse: requirements and suitability for providing digital public services. technological forecasting and social change, 203, 123346. doi.org/10.1016/j.techfore.2024.123346 lópez-cabarcos, m. á., & piñeiro-chousa, j. (2024). illusion or reality? building a metaverse community focused on value creation in the agricultural sector. international journal of information management, 77, 102782. doi.org/10.1016/j.ijinfomgt.2024.102782 martínez-gutiérrez, a., díez-gonzález, j., perez, h., & araújo, m. (2024). towards industry 5.0 through metaverse. robotics and computer-integrated manufacturing, 89, 102764. doi.org/10.1016/j.rcim.2024.102764 mkedder, n., & das, m. (2024). metaverse integration challenges: an in-depth ism and micmac analysis. journal of retailing and consumer services, 77, 103684. doi.org/10.1016/j.jretconser.2023.103684 oleksy, t., wnuk, a., & piskorska, m. (2023). migration to the metaverse and its predictors: attachment to virtual places and metaverse-related threat. computers in human behavior, 141, 107642. doi.org/10.1016/j.chb.2022.107642 pamucar, d., deveci, m., gokasar, i., tavana, m., & köppen, m. (2022). a metaverse assessment model for sustainable transportation using ordinal priority approach and aczel-alsina norms. technological forecasting and social change, 182(april). doi.org/10.1016/j.techfore.2022.121778 papagiannidis, s., bourlakis, m., & li, f. (2008). making real money in virtual worlds: mmorpgs and emerging business opportunities, challenges, and ethical implications in metaverses. technological forecasting and social change, 75(5), 610–622. doi.org/10.1016/j.techfore.2007.04.007 polyviou, a., & pappas, i. o. (2022). chasing metaverses: reflecting on existing literature to understand the business value of metaverses. information systems frontiers, 1-22. doi.org/10.1007/s10796-022-10364-4 prieto, j. de la f., lacasa, p., & martínez-borda, r. (2022). approaching metaverses: mixed reality interfaces in youth media platforms. new techno humanities, new techno humanities, 2(2), 136-145. doi.org/10.1016/j.techum.2022.04.004 salloum, s., al marzouqi, a., alderbashi, k. y., shwedeh, f., aburayya, a., al saidat, m. r., & al-maroof, r. s. (2023). sustainability model for the continuous intention to use metaverse technology in higher education: a case study from oman. sustainability, 15(6), 5257. doi.org/10.3390/su15065257 australasian journal of information systems kumar & sharma 2024, vol 28, research article immersive metaverse for business 24 sebastian, g. (2022). a study on metaverse awareness, cyber risks, and steps for increased adoption. international journal of security and privacy in pervasive computing (ijsppc), 14(1), 1-11. doi.org/10.4018/ijsppc.308785 singh, a. k., and sushil. (2013) :modeling enablers of tqm to improve airline performance international journal of productivity and performance management 62 (3): 250–275. doi:10.1108/17410401311309177. utami, h., & alamanos, e. (2022). resource-based theory: a review. theoryhub book. http://open. ncl. ac. uk. wei, d. (2022). genovese: the blockchain-based professional certification and tourism platform with its ecosystem in the metaverse. international journal of geoheritage and parks, 10(2), 322–336. doi.org/10.1016/j.ijgeop.2022.05.004 weking, j., desouza, k. c., fielt, e., & kowalkiewicz, m. (2023). metaverse-enabled entrepreneurship. journal of business venturing insights, 19, e00375. doi.org/10.1016/j.jbvi.2023.e00375 wu, j., lin, k., lin, d., zheng, z., huang, h., & zheng, z. (2023). financial crimes in web3empowered metaverse: taxonomy, countermeasures, and opportunities. ieee open journal of the computer society, 4, 37-49. doi.org/10.1109/ojcs.2023.3245801 xu, m., ng, w. c., lim, w. y. b., kang, j., xiong, z., niyato, d., ... & miao, c. (2022). a full dive into realizing the edge-enabled metaverse: visions, enabling technologies, and challenges. ieee communications surveys & tutorials, 25(1), 656-700. doi.org/10.1109/comst.2022.3221119 zallio, m., & clarkson, p. j. (2022). designing the metaverse: a study on inclusion, diversity, equity, accessibility, and safety for digital immersive environments. telematics and informatics, 75, 101909. doi.org/10.1016/j.tele.2022.101909 zhang, g., cao, j., liu, d., & qi, j. (2022). popularity of the metaverse: embodied social presence theory perspective. frontiers in psychology, 13, 997751. doi.org/10.3389/fpsyg.2022.997751 zhao, y., jiang, j., chen, y., liu, r., yang, y., xue, x., & chen, s. (2022). metaverse: perspectives from graphics, interactions and visualization. visual informatics, 6(1), 56-67. doi.org/10.1016/j.visinf.2022.03.002 appendix v1 v2 v3 v4 v5 v6 v7 v8 v9 v10 v11 v12 v13 v14 v15 v1 1 0 1 0 0 0 0 0 0 0 0 0 0 0 0 v2 1 1 1 1 1 1 1 1 1 1 1 1 0 0 1 v3 1 0 1 0 0 0 0 0 0 0 0 0 0 0 0 v4 1 1 1 1 1 1 1 1 1 1 1 1 0 1 1 v5 0 0 1 0 1 1 1 0 0 1 1 1 1 1 1 v6 1 0 1 0 0 1 1 1 1 0 1 0 1 1 1 v7 1 0 1 0 0 0 1 1 1 0 1 0 1 0 1 v8 1 0 1 0 0 0 0 1 1 0 0 0 1 0 0 v9 1 0 1 0 0 0 0 1 1 0 0 0 1 0 0 v10 1 0 1 0 1 1 1 1 1 1 1 1 1 1 1 v11 1 0 1 0 0 0 0 1 1 0 1 0 1 0 1 australasian journal of information systems kumar & sharma 2024, vol 28, research article immersive metaverse for business 25 v12 0 0 0 0 1 1 0 1 0 1 1 1 1 1 1 v13 1 0 1 0 0 0 0 1 1 0 0 0 1 0 0 v14 1 0 1 0 0 1 1 0 1 0 1 0 1 1 1 v15 1 0 1 0 0 0 1 1 1 0 1 0 1 0 1 table 4. initial reachability matrix v1 v2 v3 v4 v5 v6 v7 v8 v9 v10 v11 v12 v13 v14 v15 dr v1 1 0 1 0 0 0 0 0 0 0 0 0 0 0 0 2 v2 1 1 1 1 1 1 1 1 1 1 1 1 1* 1* 1 15 v3 1 0 1 0 0 0 0 0 0 0 0 0 0 0 0 2 v4 1 1 1 1 1 1 1 1 1 1 1 1 1* 1 1 15 v5 1* 0 1 0 1 1 1 1* 1* 1 1 1 1 1 1 13 v6 1 0 1 0 0 1 1 1 1 0 1 0 1 1 1 10 v7 1 0 1 0 0 0 1 1 1 0 1 0 1 0 1 8 v8 1 0 1 0 0 0 0 1 1 0 0 0 1 0 0 5 v9 1 0 1 0 0 0 0 1 1 0 0 0 1 0 0 5 v10 1 0 1 0 1 1 1 1 1 1 1 1 1 1 1 13 v11 1 0 1 0 0 0 1* 1 1 0 1 0 1 0 1 8 v12 1* 0 1* 0 1 1 1* 1 1* 1 1 1 1 1 1 13 v13 1 0 1 0 0 0 0 1 1 0 0 0 1 0 0 5 v14 1 0 1 0 0 1 1 1* 1 0 1 0 1 1 1 10 v15 1 0 1 0 0 0 1 1 1 0 1 0 1 0 1 8 dp 15 2 15 2 5 7 10 13 13 5 10 5 13 7 10 table 5. final reachability matrix reachability set antecedent set intersection level v1 1,3 1,2,3,4,6,7,8,9,10,11,13,14,15 1,3 i v2 1,2,3,4,5,6,7,8,9,10,11,12,13,14,15 2,4 2,4 v3 1,3 1,2,3,4,5,6,7,8,9,10,11,13,14,15 1,3 i v4 1,2,3,4,5,6,7,8,9,10,11,12,13,14,15 2,4 2,4 v5 1,3,5,6,7,8,9,10,11,12,13,14,15 2,4,5,10,12 5,10,12 v6 1,3,6,7,8,9,11,13,14,15 2,4,5,6,10,12,14 6,14 v7 1,3,7,8,9,11,13,15 2,4,5,6,7,10,11,12,14,15 7,11,15 v8 1,3,8,9,13 2,4,5,6,7,8,9,10,12,13,14,15 8,9,13 v9 1,3,8,9,13 2,4,5,6,7,8,9,10,11,12,13,14,15 8,9,13 v10 1,3,5,6,7,8,9,10,11,12,13,14,15 2,4,5,10,12 5,10,12 v11 1,3,7,8,9,11,13,15 2,4,5,6,7,10,11,12,14,15 7,11,15 v12 1,3,5,6,7,8,9,10,11,12,13,14,15 2,4,5,10,12 5,10,12 australasian journal of information systems kumar & sharma 2024, vol 28, research article immersive metaverse for business 26 v13 1,3,8,9,13 2,4,5,6,7,8,9,10,11,12,13,14,15 8,9,13 v14 1,3,6,7,8,9,11,13,14,15 2,4,5,6,10,12,14 6,14 v15 1,3,7,8,9,11,13,15 2,4,5,6,7,10,11,12,14,15 7,11,15 iteration 2 v2 2,4,5,6,7,8,9,10,11,12,13,14,15 2,4 2,4 v4 2,4,5,6,7,8,9,10,11,12,13,14,15 2,4 2,4 v5 5,6,7,8,9,10,11,12,13,14,15 2,4,5,10,12 5,10,12 v6 6,7,8,9,11,13,14,15 2,4,5,6,10,12,14 6,14 v7 7,8,9,11,13,15 2,4,5,6,7,10,11,12,14,15 7,11,15 v8 8,9,13 2,4,5,6,7,8,9,10,12,13,14,15 8,9,13 ii v9 8,9,13 2,4,5,6,7,8,9,10,11,12,13,14,15 8,9,13 ii v10 5,6,7,8,9,10,11,12,13,14,15 2,4,5,10,12 5,10,12 v11 7,8,9,11,13,15 2,4,5,6,7,10,11,12,14,15 7,11,15 v12 5,6,7,8,9,10,11,12,13,14,15 2,4,5,10,12 5,10,12 v13 8,9,13 2,4,5,6,7,8,9,10,11,12,13,14,15 8,9,13 ii v14 6,7,8,9,11,13,14,15 2,4,5,6,10,12,14 6,14 v15 7,8,9,11,13,15 2,4,5,6,7,10,11,12,14,15 7,11,15 iteration 3 v2 2,4,5,6,7,10,11,12,13,14,15 2,4 2,4 v4 2,4,5,6,7,10,11,12,14,15 2,4 2,4 v5 5,6,7,10,11,12,14,15 2,4,5,10,12 5,10,12 v6 6,7,11,14,15 2,4,5,6,10,12,14 6,14 v7 7,11,15 2,4,5,6,7,10,11,12,14,15 7,11,15 iii v10 5,6,7,10,11,12,14,15 2,4,5,10,12 5,10,12 v11 7,11,15 2,4,5,6,7,10,11,12,14,15 7,11,15 iii v12 5,6,7,10,11,12,14,15 2,4,5,10,12 5,10,12 v14 6,7,11,14,15 2,4,5,6,10,12,14 6,14 v15 7,11,15 2,4,5,6,7,10,11,12,14,15 7,11,15 iii iteration 4 v2 2,4,5,6,10,12,14 2,4 2,4 v4 2,4,5,6,10,12,14 2,4 2,4 v5 5,6,10,12,14 2,4,5,10,12 5,10,12 v6 6,14 2,4,5,6,10,12,14 6,14 iv v10 5,6,10,12,14 2,4,5,10,12 5,10,12 v12 5,6,10,12,14 2,4,5,10,12 5,10,12 v14 6,14 2,4,5,6,10,12,14 6,14 iv iteration 5 australasian journal of information systems kumar & sharma 2024, vol 28, research article immersive metaverse for business 27 v2 2,4,5,10,12, 2,4 2,4 v4 2,4,5,10,12 2,4 2,4 v5 5,10,12 2,4,5,10,12 5,10,12 v v10 5,10,12 2,4,5,10,12 5,10,12 v v12 5,10,12 2,4,5,10,12 5,10,12 v iteration 6 v2 2,4 2,4 2,4 vi v4 2,4 2,4 2,4 vi table 6. representation of levels. v1 v3 v8 v9 v13 v7 v11 v15 v6 v14 v5 v10 v12 v2 v4 v1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 v3 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 v8 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 v9 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 v13 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 v7 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 v11 1 1 1 1 1 1* 1 1 0 0 0 0 0 0 0 v15 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 v6 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 v14 1 1 1* 1 1 1 1 1 1 1 0 0 0 0 0 v5 1* 1 1* 1* 1 1 1 1 1 1 1 1 1 0 0 v10 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 v12 1* 1* 1 1* 1 1* 1 1 1 1 1 1 1 0 0 v2 1 1 1 1 1* 1 1 1 1 1* 1 1 1 1 1 v4 1 1 1 1 1* 1 1 1 1 1 1 1 1 1 1 table 7. canonical matrix copyright copyright © 2024 kumar & sharma. this is an open-access article licensed under a creative commons attribution-non-commercial 4.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v28.4529 https://creativecommons.org/licenses/by-nc/4.0/deed.en https://creativecommons.org/licenses/by-nc/4.0/deed.en https://doi.org/10.3127/ajis.v28.4529 paper title (paper title style) australasian journal of information systems soral, arayankalam & pandey 2020, vol 24, research article perception of bureaucracy and cyberloafing 1 the impact of ambivalent perception of bureaucratic structure on cyberloafing prakriti soral & jithesh arayankalam indian institute of management kozhikode, india prakritisoral@gmail.com jithesh.a@gmail.com jatin pandey indian institute of management indore, india abstract cyberloafing, defined as the use of the internet for personal use at the workplace, is emerging as a serious concern for organizations as it disrupts the attainment of organizational objectives. a systematic literature review, conducted as a part of this research, of antecedents of cyberloafing behaviour show that research exploring the relationship between the perception of organizational structure and cyberloafing is at a nascent stage. acknowledging the underexplored state of research in this area, we investigated this relationship through a quantitative study using a sample of 201 employees, and the containment theory as the base. our study results indicate that an ambivalent perception of the bureaucratic structure has a differential impact on the cyberloafing activities through serial mediation of two important attitudes, namely organizational identification and work engagement. theoretical and practical implications are also discussed. keywords: cyberloafing; enabling bureaucracy; coercive bureaucracy; organizational identification; work engagement 1 introduction the rapid penetration of smartphones and laptops has made the internet an indispensable part of our personal as well as work life. realizing its benefits, companies are building up their technology platforms to facilitate work at the workplace (griffiths, 2012). open internet access with high-speed data delivery is provided to the employees to make better and faster real-time decisions (mcafee, brynjolfsson, davenport, patil, & barton, 2012). however, open internet access may act as a double-edged sword (li, sarathy, zhang, & luo, 2014) and promote counterproductive behaviour at the workplace. one such instance is the use of the internet for personal activities, generally termed as cyberloafing. formally, cyberloafing is defined as the unauthorized personal use of the internet at work (blanchard & henle, 2008; lim, 2002; lim, & teo, 2005). in an era where every minute is crucial for the organizations, spending the supposedly productive working hours for unproductive internet activities for personal use by the employees has emerged as a major problem for the organizations. as per shrivastava, sharma, and marimuthu (2016), 57 per cent of indian workers agree that internet activities make them delay their work, and another study found that 89% of the us employees agreed that they waste their time every day by engaging into nonwork-related internet activities (salary.com, 2014). monetary losses to the companies are enormous, and it was estimated that they suffer a loss of us$759 billion annually because of cyberloafing activities (martin, brock, australasian journal of information systems soral, arayankalam & pandey 2020, vol 24, research article perception of bureaucracy and cyberloafing 2 buckley, & ketchen, 2010). due to its impact on the productivity of the employees and security concerns for the organizations, studies on identifying the causes of cyberloafing activities have emerged as an active area of inquiry among the information systems (is) researchers. most of the research in the cyberloafing literature has focused mainly on attitudes, interpersonal behaviours, and organizational policies (liberman, seidman, mckenna, & buffardi, 2011; pindek, krajcevska, & spector, 2018). however, studies on the impact of formalized rules on cyberloafing through varied policies have given mixed results. for example, according to shepherd, mejias, and klein (2014), presence of acceptable use policies (aup) reduces cyberloafing. similarly, bretschneider & parker (2016) found that such policies lead to reduction in the use of social media for personal use. on the contrary, ahmad and jamaluddin (2010) showed that computer and internet usage policies have minimal impact on cyberloafing, while stephens & ford (2015) found that placing restrictive mobile and internet policies backfired to the organization. such contradictions in findings make it necessary to identify the underlying dynamics among the aforementioned relationships, and calls for a deeper investigation into the psychological manifestations of organizational control on employees. accordingly, we propose to study formalized rules from a holistic viewpoint using the perception of bureaucratic structure to know its impact on cyberloafing behaviour. according to adler and borys (1996), even when two organizations are strikingly similar in structure, the perception of employees about bureaucratic elements may be entirely different, which cause employees to behave differently. to study this phenomenon, they proposed two types of perceptions towards bureaucratic structure, which are classified as enabling bureaucracy, where employees perceive bureaucracy as rational and a facilitator of employees’ work, and coercive bureaucracy, where they perceive it as rigid and strict. for example, as we know, employees require autonomy and participation in decision-making to feel motivated and satisfied, and to improve their performance (karasek & theorell, 1990; pandey, 2019). however, when employees encounter the bureaucratic mechanisms and perceive them to be inflexible, it prohibits plurality in decision-making and engages into centralized power (heffron, 1989), leading to lower commitment among the employees (hartline, maxham, & mckee, 2000). such an environment that restricts the innovation and creativity of employees (adler & borys, 1996; macher, 1988) abets the development of a perception toward the organizational structure as coercive, leading employees to indulge in delinquent behaviour. furthermore, according to lawrence and robinson (2007), when an employee perceives the organizational power (a form of enforcing bureaucratic structure) to be using the “discipline” mechanism composed of surveillance, normalization, and examination (foucault, 1979), it leads to production deviance (which in the present context is cyberloafing). above studies show that regardless of the purpose or intent of the enactment of power in organizations, an employee can perceive it negatively if it is impacting his/her autonomy, identity, and justice. this perception of being restricted by the organization results in frustration, eventually leading employees to indulge in deviant behaviour (lawrence & robinson, 2007). therefore, these perceptions of practices and structures in the organization are essential frames that colour the attribution and subsequent formation of attitudes and behaviours (nishii, lepak, & schneider, 2008; van de voorde & beijer, 2015). this difference in perception may be the reason for the contrasting results of many studies about bureaucracy and job outcomes. for example, while rousseau (1978) in his study identified bureaucracy to have a positive relationship with job satisfaction, arches (1991) found that bureaucracy leads australasian journal of information systems soral, arayankalam & pandey 2020, vol 24, research article perception of bureaucracy and cyberloafing 3 to job dissatisfaction. this incoherence in findings can be attributed to the negation of perceptual variables in these studies. carrying forward this stream of research, our study focuses on enabling and coercive perceptions of bureaucracy. although the number of studies on the phenomenon of cyberloafing is on the rise, there is an apparent neglect as far as the effect of perception of organizational structure on cyberloafing activities is concerned (sheikh, atashgah, & adibzadegan, 2015). as the vestiges of weber’s bureaucracy can still be found in organizational structures across the world, cyberloafing tends to be affected by the perception of organizational structure. through this research, we seek to answer how ambivalent perceptions about formalized rules and procedures, administrative hierarchy, and centralized decision-making affect nonwork-related internet activities at work. specifically, in this study, we investigate how perceptions of employees pertaining to enabling bureaucracy and coercive bureaucracy affect their cyberloafing activities. in addition, we believe that the perception of bureaucracy will manifest through job and organizational attitudes to impact cyberloafing behaviour. as several studies have empirically proved that perception leads to attitudes, which further lead to behaviour (alfes, truss, soane, rees, & gatenby, 2013; kehoe & wright, 2010), we use work engagement and organizational identification as two attitudes to mediate this relation. based on the aforementioned arguments, the research question we seek to answer through this study is as follows: rq: how the perception of bureaucracy as enabling or coercive, along with organizational identification and work engagement, influence cyberloafing behaviour of an employee? the present study draws from the containment theory (reckless, 1961), which focuses on the effect of internal and external systems on deviant behaviour, to understand the impact of perception of bureaucratic structures on cyberloafing activities. as per the theory, the push and pull factors trigger the deviant behaviour by an individual, whereas inner and outer containments act as preventive mechanisms toward controlling the deviant activity. drawing from this theory, we have proposed that outer containment (i.e., enabling bureaucracy) when further facilitated by the inner containment (i.e., organization identification and work engagement) will prevent cyberloafing behaviour. we have collected primary data from 201 participants using established scales and tested our model using covariance-based structural equation modelling (sem). the results confirmed the indirect relationship between perceptions of bureaucracy and cyberloafing through organizational identification and work engagement. 2 literature review 2.1 cyberloafing the term ‘cyberloafing’ is used to describe the voluntary act of an individual to indulge in nonwork-related internet activities at the workplace (blanchard & henle, 2008; lim, 2002). there are different views about the impact of cyberloafing activities in an organization. while some researchers favour its conduct because of its ability to act as a stress reliever, and as a source of learning and skill development (belanger & van slyke, 2002), others disapprove of these activities because of loss of productivity, security concerns, and reduction in concentration while working (chen, chen, & yang, 2008; scheuermann & langford, 1997). researchers are of the view that organizations should handle cyberloafing activities with care without affecting employees’ productivity. when working on repetitive tasks, employees may australasian journal of information systems soral, arayankalam & pandey 2020, vol 24, research article perception of bureaucracy and cyberloafing 4 feel bored and stressed (pindek et al., 2018). in such situations, cyberloafing may help in relieving the stress by providing an avenue for a break and may help employees to re-focus on work again (anandarajan & simmers, 2005). through proper monitoring of cyberloafing activities, organizations can understand the underlying reasons for such behaviour and act upon it. forcefully eliminating cyberloafing may bring about a feeling of alienation in employees as it may influence their job satisfaction level and may lead to the perception of being under coercive control (case & young, 2002). so, it is essential to have a balance between cyberloafing activities and work productivity by understanding the concerns of the employees (zoghbi manrique de lara, tacoronte, & ding, 2006). these contrasting viewpoints underscore the necessity of a more nuanced understanding of the causes of cyberloafing. cyberloafing behaviours are caused by individual as well as related organizational factors. as per van doorn (2011), cyberloafing is predominantly a deviant behaviour because it is conducted voluntarily against the organizational norms, which occur when employees try to avoid work-related activities. he also found addiction behaviour and recovery behaviour as the causes of cyberloafing. in recovery behaviour, employees engage in nonwork-related activity to recover from work-related activities. such behaviours arise while recovering from job burnout (aghaz & sheikh, 2016), job stress (koay, soh, & chew, 2017b), and workplace boredom (pindek et al., 2018). another behaviour is addiction behaviour, that is when employees engage in cyberloafing as a habit, and it may get amplified in the case of boredom or dissatisfaction (larose, kim, & peng, 2011; yellowlees & marks, 2007). as per the deviance model proposed by robinson and bennett (1995), cyberloafing falls under production deviance quadrant, which affects the organization, though in a minor way. few of the other individual-level factors like lack of self-control and propensity to procrastinate work-related tasks are also two prominent causes of cyberloafing (kim & byrne, 2011). cyberloafing also occurs due to organizational factors such as ease of internet access, due to which the demarcation between work and nonwork activities becomes unclear. they may take advantage of this high accessibility to the internet by indulging in non–work-related activities just by sitting at their desk (lim & teo, 2005). ease of access, hidden identity, and ease to neutralize (lim & teo, 2005) provoke such activities. other external causes are lack of organizational justice (chang & smithikrai, 2010), weak ethical leadership, feeble corporate culture (zoghbi-manrique-de-lara & viera-armas, 2017), loosely defined internet policies; and perceived cyberloafing activities of one’s co-workers (liberman et al., 2011). although there is a debate regarding the acceptance of indulgence in cyberloafing activities, there is a consensus on what are the serious and nonserious cyberloafing activities. these activities that are classified as serious deviants are online shopping, playing games, visiting adult-oriented websites, and downloading nonwork-related material (hadlington & parsons, 2017; lim & teo, 2005). these activities have severe repercussions on organizational productivity. other activities, such as checking the news and sending and receiving nonworkrelated emails, are considered to be minor activities (blanchard & henle, 2008). employees engaged in minor cyberloafing are generally unaware that they are engaging in deviant behaviours and continue to indulge in these activities because of vicarious influence from peers (blanchard & henle, 2008). cyberloafing studies have gained prominence in recent years, and some of the studies are related to identifying the extent of prevalence of cyberloafing (liberman et al., 2011), the impact of cyberloafing on emotions (lim & chen, 2012), seriousness and justification of australasian journal of information systems soral, arayankalam & pandey 2020, vol 24, research article perception of bureaucracy and cyberloafing 5 engaging in such activities (lim & teo, 2005), and preventive mechanisms like sanctions (khansa, kuem, siponen, & kim, 2017a; ugrin & michael pearson, 2013). other constructs studied were job attitude, job burnout (aghaz & sheikh, 2016), employee’s emotions and perceptions (lim & chen, 2012), job stress (koay et al., 2017b), national culture (ugrin, pearson, & nickle, 2018), organizational stressors (henle & blanchard, 2008; pindek et al., 2018), organizational norms, and personality (blanchard & henle, 2008). in order to provide a consolidated list of the causes of cyberloafing, we conducted a systematic literature review of the cyberloafing literature. a systematic literature review helps in identifying data points that provide a consolidated idea of the existing literature and inform new concepts using structured steps of analysis. papers were shortlisted in the following manner: as our study is a multidisciplinary and includes literature from various disciplines such as criminology, psychology, and organizational studies, among others we selected scopus, ebscohost, sciencedirect, and proquest as they are most extensive and commonly used multidisciplinary databases (hebrew university, 2019). guided by the objective of identifying the antecedents of cyberloafing behaviour, each of the above-listed databases was searched employing two keywords, namely, “cyberloafing” or “cyberslacking,” as an abstract search. our search criteria was restricted only to the peer-reviewed articles published in english and “journal articles” as the document type. further, we restricted the year of publication and shortlisted papers published in and after the year 2000. these elaborate search criteria helped us populate a list of 227 articles. in order to select the articles published in top category journals, we took articles published in a*, a and b categories in the abdc, and their equivalents in abs list. after going through the abstract and subsequently the full text of each article, a total of 30 relevant articles were finalized. each of the authors independently reviewed the 30 identified papers. we went through the studies to identify the variables that were studied and empirically proved as the antecedents of cyberloafing behaviour. each author independently scanned through the papers, listing down every antecedent mentioned therein. the articles were consolidated in annexure i, highlighting the empirically proved antecedents, theories used, sample characteristics, methodologies used, and context of the studies. after pooling all the data, as well as critically reviewing it, we identified the organizing principle as type of variable. we developed nine categories based on the type of the variable and classified each antecedent under the appropriate category. inferring from annexure i, annexure ii provides a list of all the antecedents classified as per their respective variable type. the classification was initially done based on the authors understanding of the variables and later was cross-verified from the papers in which these antecedents were used. majority of the antecedents studied in the cyberloafing literature were found to be based on perceptions, attitudes, and behaviour. from the given analysis, it is apparent that studies on the perceptions of organizational structure, and attitudes such as organizational identification and work engagement, as antecedents of cyberloafing, are not yet developed. 2.2 bureaucracy since its conceptualization by weber, several researchers have given various interpretations of bureaucracy as per changing times. although bureaucratic structures have prevailed since ancient days (schott, 2000), the concept was formalized by weber in the year 1955 in his seminal work wirtschaft und gesellschaft. as per zey-ferrell, weaver, and ferrell's (1979) australasian journal of information systems soral, arayankalam & pandey 2020, vol 24, research article perception of bureaucracy and cyberloafing 6 interpretation, bureaucracy is the “maximization of organizational efficiency as a consequence of legitimate, rationally-based authority. weber had little regard for practices of favouritism based on status (especially family connections) and the hiring of personal friends and saw bureaucratic organizations as correcting such practices” (p. 48). the bureaucratic form is considered as an ideal type, and it functions on a rational-legal authority (jelinek & ahearne, 2006; johnson, wood, brewster, & brookes, 2009). a structure celebrated by many due to its ability to improve the efficiency of operations (adler & borys, 1996), bureaucracy also gets opposed by many due to its coercive tendency to command and control, limit to the creative inputs, and excessively standardize jobs (von mises, 1944). although there was a reduction over the years in the prevalence of bureaucratic structure in organizations, due to the risk of losing out the operational benefits of bureaucracy, there is a resurgence in the adoption of bureaucratic forms (alvesson & thompson, 2004). this is especially true in developing countries, where government organizations still depend on bureaucratic structures to handle developmental challenges (utaybi, 1992). richard hall (1963) gave some of the most accurate features of bureaucracy, namely, the hierarchy of authority, division of labour, rule-based, formalized, impersonal, and technical qualification. these features represent the bureaucratic structure but yield varied consequences. adler and borys (1996) defined formalization as the extent to which rules, regulations, and procedures have been written down to impact on organizational functioning. centralization is defined as the decision-making power residing in the hands of a few people at the top of the organizational pyramid (hall, 1963). both formalization and centralization explain the significant features of bureaucracy (adler & borys, 1996). although such features of the organizational structure were prominently studied (adler, 2012), perceptions of employees about these features have not been much investigated. as per adler (2012), to understand the success or failure of an organizational setup, such perceptions need to be studied. lewin (1936) suggested that people respond and direct their behaviour by their perception of reality and not reality itself. such perceived realities are crucial and need to be studied. adler (2012) also highlights the importance of the perception of employees toward the bureaucratic organizational structure and concludes that employees perceive bureaucracy as coercive as well as enabling. enabling bureaucracy is perceived as facilitating employees’ motivation and performance, whereas coercive bureaucracy is perceived to engender alienation and deviant behaviours (adler & borys, 1996). according to adler (2012), the concept of bureaucracy explained by weber (1958) and gouldner (1954) does not take into account this ambivalence toward bureaucracy. even though the bureaucracy is implemented and practised as an enabler or a mix of enabler and coercive forces, employees’ perception may vary, and it may subsequently reflect upon their behaviour (adler, 2012). there are several studies on the existence of bureaucratic form in organizations. however, little has been studied about how employees differentiate between enabling bureaucracy and coercive bureaucracy. 2.3 organizational identification organizational identification arises when an employee displays a strong sense of “perception of oneness with or belongingness to” the focal organization (ashforth & mael, 1989, p. 34). the term has its roots in social identity theory, and it shares a similar idea of relating oneself with the values and goals of the outer being. organizational identification occurs at both the cognitive and affective level. while the cognitive part explains the degree to which an australasian journal of information systems soral, arayankalam & pandey 2020, vol 24, research article perception of bureaucracy and cyberloafing 7 individual feels they belong to the organization (ashforth & mael, 1989), the affective component helps in developing a positive image of one’s organization (tajfel, 1982). both the elements together affect the level of organizational identification. organizational identification is often confused with organizational commitment, and several authors have elaborately distinguished between them. in a seminal article by ashforth and mael (1989), the authors have argued the shortcomings in the measure of organizational commitment, and how organizational identification provides a better measure of the organizational attachment. hatch and schultz (2004) in their book critique the widely popular scale of organizational commitment given by mowday, steers, and porter (1979), where one of the items uses the terms “goals” and “values” in a generalized way, and not specific to a particular organization. however, the organizational identification construct clearly mentions that people who are identified with their organization feel some psychic loss while leaving the organization (levinson, 1970). such difficulty is not felt by people who are committed because the person may transfer this commitment to the other organization, which proves to be more convenient for him/her. this brings out the idea that “identified” people are more attached to the specific organization in which they work as compared to the employees who are just “committed” to the organization. 2.4 work engagement schaufeli and bakker (2004, p. 295) define work engagement as a “positive, fulfilling, workrelated state of mind that is characterized by vigour, dedication, and absorption.” it is a longterm dedication that arises out of an affective and cognitive state (schaufeli, salanova, gonzalez-romá, & bakker, 2002). schaufeli et al. (2002) gave three categorizations of work engagement, which are (a) vigour, (b) dedication, and (c) absorption. while vigour explains the high level of energy an employee experiences while working on the job that fosters a strong sense of willingness to invest significant efforts in the job, dedication refers to being involved in work with full enthusiasm, getting inspired by the job, and taking pride in working while understanding its significance. moreover, absorption is based on the degree of captivation by the work so much so that an employee loses the sense of time and it becomes difficult to be detached from the job (maslach, schaufeli, & leiter, 2001). the strong psychological bond between the employee and organization may motivate the employee to direct his/her efforts toward the work performance (dutton, dukerich, & harquail, 1994). constructs such as job commitment, job involvement, and job satisfaction were frequently discussed in the literature as being similar but distinct to work engagement. for example, nan wyk, boshoff, and cilliers (2003) showed that job involvement is similar to the involvement aspect of work engagement; however, the dimensions of energy and effectiveness are not explained by job involvement. work engagement is said to be a mega construct that incorporates work or job commitment into it, becoming a larger multidimensional aggregate factor (dalal, brummel, wee, & thomas, 2008; macey & schneider, 2008; newman & harrison, 2008). further, according to rich, lepine and crawford (2010), it provides a “comprehensive explanation for job performance than do concepts that depict the self, more narrowly” (p. 618). from this explanation, it is clear that work engagement is a broader construct, and is unique due to its energy dimension, i.e., employees high on work engagement tend to be more resilient, optimistic and have active coping style (christian & slaughter, 2007). australasian journal of information systems soral, arayankalam & pandey 2020, vol 24, research article perception of bureaucracy and cyberloafing 8 3 theory and hypotheses to answer our research question, we draw on the containment theory (reckless, 1961) from criminology discipline. although the theory is acknowledged as the best general theory to explain the vast array of delinquent behaviours (reckless, 1967), it is still not much used in organizational studies. the key strength of the theory lies in its ability to explain both the situations when a person behaves in a delinquent manner, and when she/he does not. the theory argues how external social factors and inner self factors helps in controlling the delinquent behaviour of a person. to elaborate, as per the theory, there are two delinquency inducing factors (i.e., push and pull factors) and two delinquency containing factors (i.e., inner and outer containments). the pull factors include the environmental factors that 'pull' individuals towards delinquent behaviour, whereas, push factors include the psychological aspects such as perceptions and attitudes of a person that 'pushes' him/her towards delinquency. while inner containment is the "ability of the individual to follow the norms" (reckless, 1967, p.475), outer containment represents the "capability of society, groups, organizations and communities to hold the behaviour of individuals in the bounds of accepted norms, rules." (reckless, 1967, p. 475) we have re-conceptualized this theory in the organizational context to study the relationships between coercive and enabling bureaucracies, organizational identification, work engagement, and cyberloafing. as discussed before, the outer containment describes the capacity of the factors external to the individual to hold him/her within the acceptable norms and expectations. it consists of three factors that help in preventing delinquent behaviours: (a) reasonable limits and expectations, (b) meaningful roles and activities, and (c) supportive relationships or adequate supervision (beebe & rao, 2005; reckless, 1961). if the individual perceives these factors as favourable and conforming, they are less likely to engage in a delinquent activity (dodder & long, 1980). since, in the context of our study, the perception of enabling bureaucracy is also conceptualized in similar terms, that is, a person will perceive bureaucracy as enabling when the rules and standards set by the organization are perceived as facilitators to his/her work (hess, 2006), we represent the outer containment as the perception of enabling bureaucracy. however, these outer containments should be facilitated by the inner containments, which are represented by four components of self: (a) a favourable self-perception; (b) goal direction towards approved behavioural patterns; (c) frustration tolerance; and (d) retention of norms. these factors represent "self-imposed limits on one's conduct that are formed out of recognition and internalization of pro-social norms, as well as one's attachment to pro-social ideals" (kennedy, 2015, p. 51). when employees identify themselves with the values, norms, rules, and customs of an organization, they are less likely to engage in deviant behaviour (reckless, 1967) because they feel responsible for the work or to the organization. these containment factors help in creating a sense of belonging and identity with the organization (reckless, 1967). as studied in the literature, employees with a high level of identification with the organization prioritize the welfare of the organization and showcase a more helpful behaviour towards it (priesemuth, schminke, ambrose, & folger, 2014; ashforth & mael, 1989). in contrast, when employees' identification with the organization is low, they will be less concerned about its success (liu, yang, liu & zhu, 2018) and will engage in harmful behaviour. similarly, studies have shown that employees with a high level of work engagement display less of deviant behaviour (shantz, alfes, truss, & soane, 2013). australasian journal of information systems soral, arayankalam & pandey 2020, vol 24, research article perception of bureaucracy and cyberloafing 9 accordingly, a high level of engagement towards the work or identification with the organization acts as a strong inner containment, preventing delinquent behaviour. consequently, we have conceptualized inner containment with organizational identification and work engagement variables. to make these four inner containments more aligned to the organizational setting, we have conceptualized (1) self-concept, retention of norms, and goal orientation as organizational identification, and (2) frustration tolerance as work engagement, because of following reasons. a) self-perception is generally conceptualized as self-concept, that is, a person may behave in the way he/she may perceive himself/herself (epitropaki, 2013). for example, a person perceiving himself or herself as honest and reliable will act accordingly. selfperception or self-concept is a self-regulatory variable that directs a person toward having certain work-related attitudes, perceptions, and behavioural intentions (epitropaki, 2013). so, the people who adopt the organization's missions and values in their self-concept tend to think and act in an organizationally consistent way (efraty & wolfe, 1988). as a highly identified individual will embed the organizational membership in one's self-concept and will develop consistency with the organizational values (van dick, ullrich, & tissington, 2006), we have conceptualized self-concept as per organizational identification. b) as retention of norms means adherence, acceptance, legitimizing the rules, values, or customs of the institution (reckless, 1967), people identifying with the organization adhere to the norms because they take pride in the organization and internalize the organization's goals and values (cook & wall, 1980). individuals identifying with the organization accept its norms, goals, and values, and integrate them into their belief system (ashforth & mael, 1989; pratt, 1998). taken together, retention of norms is conceptualized using organizational identification construct. c) the concept of goal direction indicates that when a person is having goal-oriented behaviour, he/she will conform to the rules existing in the society in order to achieve his/her goals. as identification with organization leads to congruence with the organization's goal (dutton et al., 1994), it is also conceptualized with organizational identification. d) frustration tolerance explains the power of an individual to be focused even if the person is forced to divert from his/her course. bakker and demerouti (2008) and ugwu and onyishi (2017), in their studies, highlighted that in the presence of work engagement, an employee develops the power of tolerating the frustration. the three factors of work engagement, namely, vigour, dedication, and absorption, create a flowlike situation (csikszentmihalyi, 1997), which helps the individuals to handle the frustration arising out of various reasons. hence, frustration tolerance is conceptualized using work engagement. therefore, the presence and practice of these internal containments, along with outer containment factors, will prevent delinquent behaviour (reckless & dinitz, 1968). the delinquency inducing pull factor represents pulling a person away from the normal way of working (reckless, 1961). an employee working in a bureaucratic setup develops a perception of coercive bureaucracy because he/she feels being detached from his/her normal way of working, i.e., having freedom at work and having the autonomy to take decisions. australasian journal of information systems soral, arayankalam & pandey 2020, vol 24, research article perception of bureaucracy and cyberloafing 10 working in such a rigid structure may lead to a feeling of aggression and frustration (push factor). such pushes and pulls together will produce delinquent behaviour unless they are counteracted by inner and outer containment (akers, 1994). however, sometimes, the pressure of push and pulls are so strong that it is difficult for even the self (inner) or for the groups (outer) to contain it (reckless, 1961). in our study, we represent the push and pull factors as the perception of coercive bureaucracy (figure 1). figure 1. structural model although criminology theories such as general deterrence theory (gdt) have been used in the cyberloafing literature to explain how internet usage policy at the workplace will lead to control of the cyberloafing behaviour, it addresses only the fear inducement as the controlling strategy (jia, jia, & karau, 2013). similarly, zoghbi manrique de lara et al. (2006), using the control theory, studied the impact of three coercive variables on cyberloafing. using the containment theory, we are proposing that other noncoercive factors may also contribute in controlling deviant behaviours such as cyberloafing. moreover, several studies have used the stress theories such as conservation of resources theory or strain theory (koay et al., 2017b), to explain the cyberloafing behaviour. while these studies only explained the external factors that cause the stress i.e., job stress and private demands, the containment theory provides a better explanation in understanding the cyberloafing behaviour due to the following two key reasons: (a) it helps in understanding both the deviant inducing behaviours and deviant containing behaviours; (b) both the inducing and containing factors can be studied from the work engagement organizational identification cyberloafing pull and push factors outer containment enabling bureaucracy coercive bureaucracy inner containments h1b h5b h4 h3 h2b h2a h5a h1a australasian journal of information systems soral, arayankalam & pandey 2020, vol 24, research article perception of bureaucracy and cyberloafing 11 perspective of inner self as well as external environment. these key reasons help us in understanding the differential impact of both the coercive and enabling perceptions of bureaucracy together on cyberloafing behaviour. 3.1 coercive bureaucracy the pioneers of bureaucratic ideology such as max weber, ernst freund, and fredrick taylor had a positive notion toward bureaucracy due to its disciplined mechanism to predominantly satisfy the needs of the owners (frug, 1984). several authors agree with weber’s definition of bureaucracy as an “iron cage” that facilitates organizational performance but at the cost of employees. left weberaians translate weber’s writings as “domination” rather than authority and view bureaucracy as a mechanism to experience and maintain dominance over employees (bendix, 1960). even researchers from marxists philosophy share a similar opinion of bureaucracy as a tool for exploitation (clawson, 1980). similarly, in bureaucracy, the power to control and command acts as a mechanism to preserve managerial power. procedures formed with the objective to enhance control over employees may lead to anger and affect their engagement (mcguigan, 2005). this human cost involved in the bureaucratic setup may lead to psychological and physical stress, which results in absenteeism and increases the propensity to leave. several studies focusing upon the negative impact of formalization on employees highlighted the feeling of powerlessness, selfestrangement (kakabadse, 1986), job dissatisfaction (arches, 1991), and anomie and alienation (bonjean & grimes, 1970). bennis (1969) consolidated the issues with bureaucratic structures into two parts. the first part talks about the impact of bureaucracy on individuals in making them dull and grey like an “organization man,” provides no room for personality development, discourages informal organizations, and leads to groupthink and conformity. the second part talks about how excessive use of control and authority, and lack of conflict-resolution mechanisms between hierarchical levels can negatively affect the communication, which in turn increases trust deficit among employees and employers. the implications of the perceptions of such coercive bureaucratic structures may lead to employees behaving in a deviant way. workplace deviance has been much explored in previous eras in relation with situational factors like boredom (spector et al., 2006), individual factors like attitude and personality (richards & schat, 2011), and organizational factors like abusive supervision (tepper, duffy, & shaw, 2001). previous studies encourage researchers to explore the concept of deviant behaviour further and to study the role of the perception of organizational structure (such as bureaucratic structures) on such behaviours (zimmerman, 2001). recently, a few papers have come up elaborating upon this relationship (jelinek & ahearne, 2006; marasi, bennett, & budden, 2018). greenberg (1977) argues against bureaucracy and establishes that there exists a direct causal link between bureaucracy and deviant behaviour. companies implementing bureaucracy act as a hindrance and restrict the free flow of communication among the employees. such restrictions may lead to a feeling of being trapped and in order to let out such emotions, employees may engage in organizational or personal deviance (jelinek & ahearne, 2006). marasi et al. (2018) analyse how the level of participation in decision-making and controlled working environment influence employees’ engagement in deviant behaviour. thus, there is a possibility that perception of organizational structure such as bureaucratic structures can influence cyberloafing, which is a modern manifestation of deviant behaviour. australasian journal of information systems soral, arayankalam & pandey 2020, vol 24, research article perception of bureaucracy and cyberloafing 12 one of the most prominent causes of deviant behaviour is alienation. seeman (1971) gave five antecedents to alienation such as powerlessness, meaningless, normlessness, isolation, and self-estrangement. all these five causes may very well flow from a bureaucratic structure. lack of control, unable to relate one’s work with the ultimate goal of the individual, just achieving the extrinsic goal rather than internal goals, and rules not giving proper direction to achieve personal goals—all stem from bureaucratic organizations and may prove fatal to the employee’s motivation (seeman, 1971). as discussed earlier, the containment theory proposes that push and pull factors act as a reinforcing mechanism for deviant behaviour (reckless, 1967). given this, we propose that the perceived coercive bureaucracy will further trigger the cyberloafing activities by the employee: hypothesis 1a: coercive bureaucracy is positively associated with cyberloafing. 3.2 enabling bureaucracy past studies have tried to bring forth and highlight the justification of a bureaucratic setup. a rigid hierarchy, as seen in a bureaucratic organization, may even facilitate the work by bringing in best practices and coordination through formalization (hall, 1968). as per hall (1968), this formalization brings standard rules and procedures, which create congruence and unity in direction. nicholson and goh (1983), in their study, argued that a formalized working setup would lead to a reduction in role conflict and role ambiguity. other positive outcomes of formalization are job satisfaction (snizek & bullard, 1983), reduced emotional exhaustion (pines & maslach, 1978), encouragement for innovation (damanpour, 1991), facilitation of job performance (deming, 1986), and reduction in the feeling of alienation and stress (jackson & schular, 1985). montagna (1968), in his study, proposes that with the introduction of formalized rules and norms, employees develop the understanding about their goals and ways to achieve them, thus reducing the need for direct supervision. such a scenario will lead to autonomy for the employees, thus shifting the negative connotation of the bureaucracy toward a positive one. as per the concept of enabling bureaucracy, bureaucratic structures are seen as problem solvers, which help in attaining a higher level of efficiency (adler & borys, 1996; saparito & coombs, 2013). it facilitates flexibility, promotes creativity, and encourages innovation by transfer of technology. another view as given by hess (2006) toward enabling bureaucracy is its ability to set standards for the employees. these standards showcase the best way of performing the tasks, provide support for handling multiple jobs, and improve the capabilities of the workers (hess, 2006). the difference between enabling and coercive bureaucracy is that the former focuses upon the development of rules and regulations to enhance performance, whereas the latter focuses upon controlling the employees. here, the formalization of rules and procedures does not hinder the work; instead, it acts as flexible guidelines that contribute to problem-solving. also, centralization gets a new dimension to it where the decision-making power is centralized to the employees, and so leaders maintain a high level of accountability with the subordinates and share mutual benefits unlike in coercive bureaucracy, where the focus is only toward the betterment of owners (utaybi, 1992). from this discussion, it can be seen that the perception of bureaucracy as enabling is favourable to the employees. given these arguments, we posit that the perceived enabling bureaucracy (outer containment) will reduce cyberloafing behaviour: australasian journal of information systems soral, arayankalam & pandey 2020, vol 24, research article perception of bureaucracy and cyberloafing 13 hypothesis 1b: enabling bureaucracy is negatively associated with cyberloafing. one very crucial aspect of organizational identification is that in addition to the need for identifying with the working, it requires employee’s full participation in the organization, which develops a sense of membership with the organisation. this may not be fruitful in the case of bureaucratic organizations. implementing strict rules and regulations and taking away the autonomy may develop perceived barriers to the organizational hierarchy. such excessive rules, regulations, and procedures may limit the ability of the employee to see beyond (stuart, 1999). perceiving such a treatment from the organization may restrict the identification between employee and organization. based on this, we hypothesize the following: hypothesis 2a: coercive bureaucracy is negatively associated with organizational identification. bringing in the concept of social identity theory, ashforth and mael (1989) explained how an employee’s identity with the organization cultivate into dedication towards work performance. organizational identification creates a strong sense of cognitive and affective bond that culminate in the relationship with the organizational goals. depending on this psychological bond, engagement with goals, values, and activities may increase or decrease (dutton et al., 1994). a study conducted by brown (1969) concluded that perceived access to organizational hierarchy has a strong direct relationship with organizational identification. moreover, support from supervisors in terms of work-related help, being concerned, and understanding toward employees will lead to the development of identity with the organization (benkhoff, 1997; yoon, baker, and ko, 1994). hall, schneider, and nygren (1970) study shows a direct positive relationship between organizational identification and autonomy. enabling formalization may help in building a congruence between the professional goals of an employee and organizational goals (greene, 1978). it also makes the organization more meaningful to the individual by building up “institutional motivation” (glaser, 1963). li, zhang, and sarathy (2010), in their study, found out that individuals having a high level of organizational identification developed a self-norm toward appropriate use of computer resources provided to them. such a component of trust between employees and organization develops a sense of duty to comply with the network policies (adler, 2012). based on this idea, we hypothesize that when an employee perceives his/her organization’s bureaucratic structure as empowering, he/she may develop a sense of identity with the organization: hypothesis 2b: enabling bureaucracy is positively associated with organizational identification. the current literature explaining the relationship between work engagement and organizational identification is scant. according to karanika-murray, duncan, pontes, and griffiths (2015), there is a positive relationship between organizational identification and work engagement, where engagement acted as a mediator between organizational identification and job satisfaction. so, to test the relationship further, we hypothesize that employees who strongly relate themselves to the organization may tend to engage in work as well: hypothesis 3: organizational identification is positively associated with work engagement. as explained by the concept of work engagement, employees who are engaged in their job work more efficiently and bring about better productivity (bakker & demerouti, 2008). similarly, if the employees are disengaged, they tend to indulge in deviant activities during working hours (ketchen, craighead, & buckley, 2008). when employees are engaged in their australasian journal of information systems soral, arayankalam & pandey 2020, vol 24, research article perception of bureaucracy and cyberloafing 14 work, they will be less distracted and hence will not engage in deviant behaviour (alias, mohd rasdi, ismail, & abu samah, 2013). based on this, we hypothesize the following: hypothesis 4: work engagement is negatively associated with cyberloafing. organizational control through rules and regulation also determines the level of engagement an employee will display. when there is a perception of bureaucracy as enabling rules are perceived to bring in best practices to the organization, and so, employees tend to work with more dedication. moreover, this perception will create a positive attitude about the organizational functioning, which should lead employees identifying themselves with the organization. however, in a bureaucracy perceived as coercive, rules are seen as a coercive way of control, leading to disengagement with the work (adler & borys, 1996). drawing from the containment theory (reckless, 1967), we propose that if the outer containment, that is, bureaucracy perceived as enabling, gets reinforced by internal containments of work engagement and organization identification, all of it together will reduce the deviant behaviour. moreover, the push and pull system, that is, bureaucracy perceived as coercive, will negatively affect the organizational identification and work engagement, leading to an increase in cyberloafing behaviour. based on the above arguments, we hypothesize the following: hypothesis 5a: perceived coercive bureaucracy has a positive indirect impact on cyberloafing through organizational identification and work engagement. hypothesis 5b: perceived enabling bureaucracy has a negative indirect impact on cyberloafing through organizational identification and work engagement. 4 methods 4.1 sample and procedures the questionnaire was sent via email and hard copy to 733 employees. organizations were identified in such a way that it included firms of all sizes. in total, 201 participants completed the survey. since the sample constituted a great variety in occupation, organizational size, educational level of the employees, and the location of the organization, it was suitable for the study. the recommended sample size needs to be at least five times the number of parameters estimated (bagozzi & yi, 1988). in the model, there are 25 parameters, and thus 201 is an adequate sample size for the analysis. the average age of the participants was 30.63 years, of which 70.4 per cent were male, and 28.2 per cent were females; 52.1 per cent of the respondents were graduates, and 45.5 per cent were postgraduates. the average tenure of the respondents was 5.18 years. around 46.5 per cent of the respondents were unmarried. nearly 40 per cent of the participants were from organizations that have more than 5,000 employees. nearly 48 per cent of the respondents were professionals, whereas 32 per cent were in technical positions. questionnaires were filled with the respondents’ consent, and respondents’ details were kept anonymous to make sure that honest responses were given. throughout the study, we followed the ethical standards recommended by the american psychological association (smith, 2013). all work has been done according to those guidelines. with regard to what is known as the researcher bias, we took various steps to minimize it. some of the major reasons for researcher bias are (1) poorly conceived questions leading respondents to answer in a forced way; and (2) question order bias, where the ordering of items affects the response (driscoll, 2011) among others. in order to reduce the impact of these australasian journal of information systems soral, arayankalam & pandey 2020, vol 24, research article perception of bureaucracy and cyberloafing 15 biases, in line with mackenzie and podsakoff (2012), and wales, patel, and lumpkin (2013), we have used established scales for collecting the data. in addition, the data were collected anonymously, and we have also provided detailed statistical analysis in the form of tables throughout our article. in addition, to prevent bias due to the low self-efficacy of the respondents to give the correct answer, while collecting data, we emphasized to the respondents that only their personal experience or knowledge is required to fill the questionnaire. 4.2 measures the survey used a seven-point likert scale unless otherwise stated. each construct was measured using scales used in previous research. in order to check the internal validity of the constructs, a confirmatory factor analysis (cfa) was conducted. a few items were removed because of low factor loading. the constructs used in the study are explained below, and their definitions and measures are shown in table 1. 4.2.1 perception of bureaucracy perception of bureaucracy was measured using a scale developed by hoy and sweetland (2001), which seeks to measure two crucial aspects of bureaucracy, namely, formalization and centralization. this scale further categorizes both these aspects as enabling and coercive. the scale consisted of 12 items, out of which six items were related to perception of bureaucracy as enabling and the remaining six items to perception of bureaucracy as coercive. four items were removed because of low factor loading. an example item for perception of bureaucracy as coercive is “administrative rules in this organization are used to punish employees.” similarly, an example item for perception of bureaucracy as enabling is “the administrators in this organizations use their authority to enable employees to their job.” 4.2.2 cyberloafing cyberloafing behaviour was measured using a scale designed by lim and teo (2005). this scale consists of 13 questions. in our study, considering the cultural sensitivity, we removed one question from the questionnaire, which is “visit adult-oriented (sexually explicit) websites.” since we are considering only those activities which are considered as severe cyberloafing, we removed items that are considered nonserious as per lim and teo (2005) (e.g. “receive nonwork-related email”). an example item of serious cyberloafing is “during office hours, how often do you use the internet at work to access the following websites for personal reasons? or visit non-job-related websites.” in the present study, major cyberloafing activities are considered to analyse the prevalence of cyberloafing among employees. 4.2.3 work engagement using a shortened scale of the utrecht work engagement scale (uwes), which consists of nine items, attitudes toward work were measured (schaufeli & bakker, 2004). one item was removed because of low factor loading. an example item is “i am enthusiastic about my job.” 4.2.4 organizational identification this construct was measured using a five-item scale (smidts, pruyn, & van riel, 2001). an example item is “i feel proud to work for my company.” 4.2.5 control variables education, personality, job type, number of working hours per week, size of the organization, income, and job position were added as control variables. jia, jia, and karau (2013) found that australasian journal of information systems soral, arayankalam & pandey 2020, vol 24, research article perception of bureaucracy and cyberloafing 16 personality factors such as conscientiousness, emotional stability, and extroversion have a significant relationship with cyberloafing. to capture the personality traits of the respondents, we used bfi-10 (rammstedt & john, 2007), a shorter version of the big five personality inventory. an example item is “i see myself as someone who is reserved.” in addition, as per case and young (2002), the efficiency of internet policies depends on the size of an organization. as internet policy can affect cyberloafing, the organizational size was used as a control variable. another control variable used is income, which has a significant effect on cyberloafing (garrett & danziger, 2008). aghaz and sheikh (2016), in their study, showed that job burnout and cyberloafing have a positive relationship in the knowledge-intensive sector. as the sample in the present study was drawn from varied sectors, job type was used as a control variable. further, the number of working hours was also controlled as we wanted to eliminate the possibility that more the amount of time spent at work, greater the possibility of cyberloafing. construct definition measures cyberloafing “voluntary act of employees’ using their companies’ internet access during office hours to surf non-job related web sites for personal purposes” (lim, 2002, p. 677) adopted serious cyberloafing from the 13-item scale used by lim & teo (2005) enabling bureaucracy “enabling bureaucracy is a structure that is formed by enabling formalization and enabling centralization—the rules, regulations, and procedures are helpful and lead to problemsolving among members rather than rigid, coercive activities that demand conformity.” (hoy & sweetland, 2001, p. 301). this is a perceptual variable. 12-item scale developed by hoy & sweetland (2001), out of which six items are related to enabling bureaucracy, and the remaining six items are related to coercive bureaucracy coercive bureaucracy coercive bureaucracy is “when formalization and centralization coerce and hinder rather than help.” (hoy & sweetland, 2001, p. 302). this is a perceptual variable. organizational identification organizational identification arises when an employee displays a strong sense of ‘‘perception of oneness with or belongingness to” the focal organization (ashforth & mael, 1989, p. 34) 5-item scale by smidts, pruyn, & van riel (2001) work engagement work engagement is defined as a “positive, fulfilling, workrelated state of mind that is characterized by vigor, dedication, and absorption.” (schaufeli & bakker, 2004, p. 295) the 9-item shortened scale of the utrecht work engagement scale (uwes) (schaufeli & bakker, 2004) table 1. construct definitions and measures. 5 results we conducted cfa using amos 25 for the fit assessment of the measurement model. we employed a bootstrapping method (1,000 repetitions) to test the model. two hundred and fifty or more bootstrap samples are considered to be relevant for the estimation (nevitt & hancock, 2004). model-fit measures commonly used to evaluate the model’s overall goodness-of-fit are the ratio of χ2 statistic to degrees-of-freedom (df), goodness-of-fit index (gfi), adjusted goodness-of-fit index (agfi), normalized fit index (nfi), tli, comparative fit index (cfi), and rmsea. in this study, we relied on the χ2 statistic and the rmsea, nnfi, cfi, and srmr (bagozzi & yi, 2012). since our measurement model met the criteria for a good fit (cfi = 0.951, australasian journal of information systems soral, arayankalam & pandey 2020, vol 24, research article perception of bureaucracy and cyberloafing 17 tli = 0.943, rmsea = 0.051), we went ahead to test for reliability, convergent validity, and discriminant validity. this was achieved using anderson and gerbing’s (1988) two-stage approach. initially, the convergent and discriminant validity of the measurement model using cfa were tested. subsequently, the hypothesized model was compared with alternate models based on fit measures and significance of constructs. measures of the validity and reliability of the measurement model are shown in table 2. cr ave msv we oi cy cb eb we .889 .542 .528 .736 oi .938 .754 .528 .726*** .868 cy .805 .455 .059 -.244*** -.220** .675 cb .783 .482 .239 -.489*** -.446*** .229* .694 eb .800 .509 .194 .361*** .441*** -.085 -.385*** .714 note. n = 201; *** p < 0.01; ** p < 0.05; * p < 0.10; cr: composite reliability; ave: average variance extracted; msv: maximum shared variance; we: work engagement; oi: organizational identification; cy: cyberloafing; eb: enabling bureaucracy; cb: coercive bureaucracy table 2. inter construct correlations in order to test the convergent validity of scale, fornell and larcker (1981) gave three criteria: (a) factor loadings should be greater than 0.70, (b) composite reliabilities (cr) should be greater than 0.70, and (c) average variance extracted (ave) should be greater than 0.50. as table 2 indicates, values of cr are above the recommended cut-off of 0.7 (hair, black, babin, & anderson, 2010). moreover, if ave is below 0.5, but composite reliability is greater than 0.6, the convergent validity is considered to be adequate (fornell & larcker, 1981). the measures shown in table 2 meet these criteria. discriminant validity was assessed using a criterion that the square root of ave should exceed the correlations between a construct and other constructs (fornell & larcker, 1981). in addition, almost all the values of maximum shared variance (msv) are greater than ave, which is another test for establishing discriminant validity. these two tests corroborated the adequacy of our measurement scales. to test the common method bias, we used harman's single-factor test, which advocates that variance explained by the single factor should be less than 50 per cent (harman, 1960). our model showed 26.71 per cent of variance, which suggests that common method bias was not present. next, we proceeded to test our hypothesized model. the model was tested using the maximum likelihood estimation. the structural model (figure 1) was compared with five other competing models, which are shown below (figure 2 to figure 7). although fit indices are widely accepted way of establishing the fit of the research model, we followed werner and schermelleh-engel (2010) method of comparing different plausible models to further establish the strength of our proposed model. there are studies (e.g., aghaz & sheikh, 2016), which has indicated the relationship between work-related aspects and cyberloafing. further, there are studies which stated that cyberloafing can impact work-related outcomes such as work engagement (e.g., lim & chen, 2012; zhao, wayne, glibkowski & bravo, 2007). acknowledging these studies, and considering the possibilities of other models and in line with werner and schermelleh-engel (2010), we wanted to check how our research model fare against these models to further establish the robustness of our model. australasian journal of information systems soral, arayankalam & pandey 2020, vol 24, research article perception of bureaucracy and cyberloafing 18 figure 2: hypothesized model figure 3: alternate model 1 figure 4: alternate model 2 australasian journal of information systems soral, arayankalam & pandey 2020, vol 24, research article perception of bureaucracy and cyberloafing 19 figure 5: alternate model 3 figure 6: alternate model 4 figure 7: alternate model 5 note: relationships shown in figures 2 to 7 are depicting direct relationship. australasian journal of information systems soral, arayankalam & pandey 2020, vol 24, research article perception of bureaucracy and cyberloafing 20 based on west, taylor, and wu’s (2012) recommendation, fit statistics were used to compare different models rather than using a specific cut off standards. the results showed that the hypothesized model has a better fit in comparison to most of the other models. values are shown in table 3. model gfi agfi chi-square/df rmsea cfi tli srmr hypothesised model 0.848 0.789 1.427 0.046 0.929 0.907 0.056 alternate model 1 0.846 0.787 1.450 0.047 0.925 0.902 0.061 alternate model 2 0.850 0.791 1.422 0.046 0.930 0.908 0.056 alternate model 3 0.831 0.766 1.633 0.056 0.894 0.861 0.077 alternate model 4 0.822 0.757 1.713 0.060 0.879 0.844 0.078 alternate model 5 0.838 0.775 1.545 0.052 0.909 0.881 0.087 note. n=201; gfi: goodness of fit index; agfi: adjusted goodness of fit index; rmsea: root mean square error of approximation; cfi: comparative fit index; tli: tucker lewis index; srmr: standardized root mean squared residual table 3. comparison of models education, personality, job type, number of hours working per week, size of the organization, income, and job position were added as control variables. among them, education, personality, job position, and job type were not found to have any effect on the results, while job type was found to have a direct effect on cyberloafing ( β = -0.167, p = 0.020, lower ci = -0.286, upper ci = -0.033). coming to the indirect effect, size of the organization was found to have an effect on cyberloafing (β= 0.036, p = 0.035, lower ci = 0.002, upper ci = 0.113). also, two aspects of personality, agreeableness ( β = 0.028, p = 0.032, lower ci = -0.120, upper ci = -0.001) and extroversion (β = 0.031, p = 0.046, lower ci = 0.001, upper ci = 0.099) were found to have a positive indirect effect on cyberloafing. hypothesis 1a did not get support as the direct correlation between coercive bureaucracy and cyberloafing was not found to be significant (p = 0.262). similarly, hypothesis 1b was not supported as the direct correlation between enabling bureaucracy and cyberloafing was not found to be significant (p = 0.565). hypothesis 2a was supported as the direct relationship between coercive bureaucracy and organizational identification was found to be significant ( β = -0.304, p = 0.002, lower ci = -0.517, upper ci = -0.150). similarly, hypothesis 2b was supported as the direct correlation between enabling bureaucracy and organizational identification was found to be significant (β = 0.330, p = 0.003, lower ci = 0.159, upper ci = 0.496). hypothesis 3 received support as the direct correlation between organizational identification and work engagement was significant (β = 0.690, p = 0.002, lower ci = 0.579, upper ci = 0.783). hypothesis 4, though rejected, was found to have a weak significance at 10 per cent confidence interval (β = -0.228, p = 0.057). hypothesis 5a, which states that coercive bureaucracy has a positive indirect effect on cyberloafing through organizational identification and work engagement, received support as the relation was found to be significant (β = 0.048, p = 0.035, lower ci = 0.003, upper ci = 0.134). similarly, hypothesis 5b which states that enabling bureaucracy has a negative indirect effect on cyberloafing through organizational identification and work engagement received support as the relation was found to be significant (β = -0.052, p = 0.038, lower ci = -0.132, upper ci = -0.004). the results are indicated in figure 8. australasian journal of information systems soral, arayankalam & pandey 2020, vol 24, research article perception of bureaucracy and cyberloafing 21 figure 8: path diagram of the hypothesized model 6 discussion the study is based on identifying how the ambivalent perceptions of employees toward their organizational structures impact their indulgence in deviant behaviour. perceptions of bureaucracy were studied based on two prominent features of bureaucracy—formalization and centralization. both the features were perceived by respondents as either coercive or enabling, which makes their perceptions ambivalent toward bureaucracy. the effect of these perceptions on serious cyberloafing, which is a deviant behaviour, was evaluated. findings from the study show that hypotheses 1a and 1b, that is, the presence of a direct relationship of enabling and coercive bureaucracy with cyberloafing came out as nonsignificant. the reason for this result can be understood through the theory of planned behaviour (tpb). as per the theory, behaviour is caused due to three prominent antecedents, namely, subjective social norms, attitudes, and perceived behavioural control (ajzen, 1985). as per ajzen (1985), the higher the favourable attitude of an employee and the higher the perception of ease in showcasing a behaviour, the higher will be the behavioural intent. attitude mediates the relationship between perceived control and behavioural intent. therefore, a direct relationship, as suggested in hypotheses 1a and 1b, came out to be insignificant. hypotheses 2a and 2b, that is, the direct relationship between enabling and coercive bureaucracy, and organizational identification came out to be significant. as given in the literature, organizational identification occurs when an employee feels a sense of belongingness with the organization, and this is possible only when an employee perceives the functioning of the organization as a facilitator towards his/her work. hypothesis 3, that is, the direct positive relationship between organizational identification and work engagement validates the study done by karanika-murray et al. (2015), and showed a significant result. work engagement organizational identification cyberloafing h2a: 0.304*** h2b: 0.330*** h1b: 0.051 h1a: 0.125 enabling bureaucracy coercive bureaucracy h4: -0.228* h5a: 0.048** h3: 0.690*** h5b: -0.052** *** = p < 0.01; ** = p < 0.05; * = p < 0.10 direct effect indirect effect australasian journal of information systems soral, arayankalam & pandey 2020, vol 24, research article perception of bureaucracy and cyberloafing 22 having a strong bond with the organization indicates a strong bond with the work as well because organizational identification develops a sense of responsibility toward the organization, which will be fulfilled by the employees through their work. these two attitudinal variables, when combined, depict an employee’s relationship or bond with the organization, which further influences the employee behavioural outcomes (karanika-murray et al., 2015). this brings us to hypothesis 4, which states that work engagement will negatively affect cyberloafing activities. as discussed earlier, work engagement is a work-related motivational state of mind that makes employee highly involved in work and persistent in their efforts (schaufelli et al., 2002). this high engagement with work may supersede the desire to involve in deviant behaviour and may even reduce the impact of internet addiction. another reason for explaining the relationship between work engagement and cyberloafing activities is boredom and stress occurring in the workplace. for example, engaged employees do not feel bored and maintain a constant involvement with work (van wyk, de beer, pienaar, & schaufeli, 2016). due to this, the tendency to involve in cyberloafing due to boredom will be reduced (pindek et al., 2018). thus, work engagement is negatively related to cyberloafing. hypothesis 5 states an indirect relationship between perception of bureaucracy and cyberloafing via organizational identification and work engagement. the two attitudinal variables play a significant role in establishing this indirect relationship. this relationship is also supported by the tpb, where attitudes (organizational identification and work engagement) create an indirect effect between perceived control (bureaucracy) and behaviour (cyberloafing). furthermore, as stated by kim and byrne (2011), employees engage in cyberloafing due to the internal processes and their perception of unfair treatment by the organization. a one shoe fits all approach may not work in organizations (jha, pandey & varkkey, 2019; pandey 2018). while bureaucratic structures are formed with the objective of bringing formalized procedures and norms to control any type of deviance (adler, 2012), the increase in the control itself leads to an increase in cyberloafing activity. therefore, people under the perception of the coercive organizational structure may feel alienated (mcguigan, 2005), so much so that it will directly affect their relationship with the organization and may subsequently reduce their engagement with their work. for such people, working will need higher mental efforts, and they may tend to feel stressed (koay et al., 2017b). in such a case, deviant behaviour comes to their rescue, helps them in releasing their worries, and makes them feel refreshed (case & young, 2002). however, these deviant behaviours negatively impact organizational productivity. kim (2018), in his hbr article about millennials has addressed this issue. since the entry of millennials into the workforce, who considers technology as something indispensable in their daily activities, it is necessary to understand their perception of work. use of control mechanisms like deterrence may backfire and can lead to dysfunctional outcomes. therefore, it is necessary to mutually decide upon the policies taking into account the needs of both the employees as well as the organization. so, there is a need for managers to keep a balance between employees’ necessity to indulge in minor cyberloafing and their productivity level while keeping a check on major cyberloafing activities. the results also validated the containment theory by empirically proving how outer containment (perception of enabling bureaucracy) along with internal containments (organizational identification and work engagement) prevent deviant behaviour like cyberloafing activity. 6.1 theoretical contributions our study contributes to the bureaucracy as well as cyberloafing literature in four keyways. australasian journal of information systems soral, arayankalam & pandey 2020, vol 24, research article perception of bureaucracy and cyberloafing 23 first, containment theory (reckless, 1961), a criminology theory, is used for the first time in the context of information system studies for empirical analysis. by using deviance inducing and deviance containing variables from the organizational context, we accommodated the theory to the organizational setting. drawing on the theory, we argue that pull and push factors that trigger the deviant behaviour (flexon, 2014) were represented by the perception of coercive bureaucracy variable. similarly, the outer containment factor, that prevents deviant behaviour, was represented by perception of enabling bureaucracy. further, the inner containment was represented using organizational variables such as organizational identification and work engagement. future studies may investigate further into other organizational constructs that may potentially serve as outer and inner containments. our contribution will provide is researchers a platform to understand other kinds of deviant cyber behaviours using the containment theory. second, in the is literature, cyberloafing studies have used theories from various areas such as behavioural theory, social theory, criminology theory etc. (refer appendix i). for example, jia, jia and karau (2013) and ugrin and pearson (2013) have used general deterrence theory to propose that using coercive methods and instilling fear in employees, cyberloafing behaviour can be controlled. on the contrary, by adopting the containment theory from criminology, our study adds to the existing literature on cyberloafing by establishing that there are non-coercive ways to contain cyberloafing behaviour. by adopting this theory, we empirically proved that the enabling perception of bureaucracy (i.e., an outer containment) along with the organizational identification and work engagement (i.e., inner containments) help in containing cyberloafing. as said by reckless (1967), "if a group or organization can get its members to internalize their rules, it would be doing an excellent job of containing" (p. 470) the deviant behaviour. thus, by underscoring the importance of non-coercive control mechanisms, ours is the first study to extend the theory on cyberloafing using containment theory. third, is as a multi-disciplinary field deals with problems in several contexts such as organizational, sociological, and psychological, among others. particularly, while studying an organization, several aspects of it such as culture, structure, communication, and processes play a critical role. since the organizational structure governs the workflow of an organization, it controls how a technology will be implemented, used and benefited within the organization. acknowledging such prominence of organizational structure, many is studies have undertaken investigations related to it. for instance, in the organizational context, studies have found that bureaucratic structure restricts the adoption of technology (hall & khan, 2002) and erp implementation (morton & hu, 2008). baroudi and lucas (1994) argue that bureaucracy creates bottlenecks, and technology can help remove it (baroudi & lucas, 1994). similarly, studies have acknowledged the role of centralized organizational structure on knowledge management (chen & huang, 2007). further, vaast and binz-scharf (2008) found that through technology, the negative effects of bureaucratic structure can be nullified. more importantly, the above-mentioned studies of bureaucracy in is have ignored the fact that bureaucracy is perceived differently by each employee. although bureaucratic structure as an antecedent is prominently studied in is literature, there is a conspicuous dearth of studies on the perception of organizational structure and cyberloafing. acknowledging this gap, we delved in understanding the role of ambivalent perception of bureaucracy as an antecedent to the technology-related deviant behaviour (i.e., cyberloafing). australasian journal of information systems soral, arayankalam & pandey 2020, vol 24, research article perception of bureaucracy and cyberloafing 24 finally, our study provides a systematic literature review of the cyberloafing literature, from which it can be seen that the literature mainly consists of studies based on the employee’s perception, followed by attitude and behaviours. such a comprehensive review of cyberloafing literature was missing in the extant studies. we strongly believe that future studies may benefit from the consolidated list provided in annexure-ii. our study adds to the literature of cyberloafing by studying the impact of unexplored variables such as perception of bureaucracy and two important attitudes, i.e. organizational identification and work engagement on cyberloafing behaviour. 6.2 practical contributions paper also offers contributions to the practitioners in several key ways. from the present study, it was also evident that it is not only the increase in accessibility to the internet that causes cyberloafing behaviours but also a mix of perception and attitudes. employees perceiving the organizational structure to be coercive identify themselves less with the organization who in turn tends to be less engaged in their work, and engage in cyberloafing activities. the extant studies have suggested that managers should advance toward adopting a post-bureaucratic or e-bureaucratic structure (osborne & plastrik, 1997). however, from our present study, it is found that even in the presence of a bureaucratic structure, employee engagement and identification may be present if their perception towards such structure is positive, which in turn will reduce the cyberloafing activities. to develop and maintain such a setup in an organization, managers must ensure that two-way dialogue is set up to understand the perception of the employees about such structural interventions. a wellintended initiative by policymakers can have perilous effects if the perceptions are not managed well. managers must, therefore, communicate the reasons and procedures of structural interventions present in the organizational structure. 7 limitations and future directions this study, while providing insightful results, is not devoid of limitations, which need to be accounted for in further studies. first, methodologically, the study did a cross-sectional analysis of employees, which may not provide the perspective of the manager. a multilevel study would be helpful to know the macro-level perception of managers and policymakers, who are responsible for setting up the organizational structure toward the causes and consequences of cyberloafing. second, the majority of the cyberloafing literature is based on the survey method, and researchers should further understand this phenomenon using different methodological techniques. for example, experiential sampling method may be used to study when during a working day an employee indulges in cyberloafing the most. this will help in finding specific interventions based on the intensity of the cyberloafing at a given time. experiential sampling is used to study the experience of engaging in activities in everyday life and to understand the dynamics of emotions and other subjective states (farnworth, mostert, harrison, & worrell, 1996). moreover, other techniques such as interviews and focus groups may be used to understand these behaviours in depth. third, our study considers only serious cyberloafing. there is a need for understanding the beneficial aspects of cyberloafing in addition to considering it as a deviant behaviour alone. few studies have studied cyberloafing to be a coping mechanism, and further debates about its positive aspects will help us to know more about the phenomena. fourth, the sample size is skewed towards male respondents, which made it difficult to test the effect of gender on cyberloafing activities. gender-specific studies can throw more light on this. future studies may contribute to the organizational australasian journal of information systems soral, arayankalam & pandey 2020, vol 24, research article perception of bureaucracy and cyberloafing 25 structure literature by studying how the perceptions of an organizational structure differ between male and female. fifth, start-ups have an evolving organizational structure, and they function in an organic way that provides flexibility to them (freeman & engel, 2007). since our study involves firms with different organizational size, our findings from this study should be generalized with caution. sixth, our study does not take into account the possibility that an individual may consider some aspects of bureaucracy as enabling, while other aspects as coercive. this possibility needs to be explored in future studies. finally, annexure i, developed from the systematic literature review, provides a consolidated view of the studies done in various aspects of cyberloafing. while studies on perceptions and attitudes are studied to quite an extent, the role of emotions and skills is yet to be explored in depth. 8 conclusion cyberloafing is an issue that can cause huge productivity losses to the companies, and it has become a ubiquitous deviant behaviour because of the rapid penetration of smartphones and the internet. in this study, we dug deeper to understand how perceptions of bureaucracy by employees can have an impact on their cyberloafing activities. drawing on the containment theory, we inquired into the role of ambivalent perceptions of bureaucracy using the push and pull factors. when the perception of bureaucracy is enabling (outer containment), it reduces cyberloafing behaviour of the employees in the presence of self-identification with the organization and significant engagement with the work (inner containment). on the other hand, the perception of employees toward bureaucracy as a coercive structure leads to a reduction in the organizational identification of the employees, which subsequently affects their work engagement, and finally induces them to indulge in cyberloafing activities. as found from the results, the study empirically established the serial mediation of perceptions of bureaucracy, that is, enabling (β = –0.052) and coercive (β = 0.048), on cyberloafing through organizational identification and work engagement. the insights from the study will be an important step in understanding the phenomenon of cyberloafing. particularly, it enlightens policymakers as well corporate practitioners about the role of perceptions regarding bureaucracy in dealing with cyberloafing. references adler, p. s. (2012). perspective---the sociological ambivalence of bureaucracy: from weber via gouldner to marx. organization science, 23(1), 244–266. https://doi.org/10.1287/orsc.1100.0615 adler, p. s., & borys, b. (1996). two types of bureaucracy: enabling and coercive. administrative science quarterly, 41(1), 61–89. https://doi.org/10.2307/2393986 aghaz, a., & sheikh, a. (2016). cyberloafing and job burnout: an investigation in the knowledgeintensive sector. computers in human behavior (vol. 62). https://doi.org/10.1016/j.chb.2016.03.069 ahmad, z., & jamaluddin, h. (2010). computer usage policy and employees’ attitude towards cyberloafing. in technology innovation and industrial management 2010 conference (tiim 2010). ajzen, i. (1985). from intentions to actions: a theory of planned behavior bt action control: from cognition to behavior. in j. kuhl & j. beckmann (eds.) (pp. 11–39). berlin, heidelberg: springer berlin heidelberg. https://doi.org/10.1007/978-3-642-69746-3_2 australasian journal of information systems soral, arayankalam & pandey 2020, vol 24, research article perception of bureaucracy and cyberloafing 26 akers, r. l. (1994). criminological theories : introduction and evaluation. los angeles, calif.: roxbury pub. co alfes, k., truss, c., soane, e. c., rees, c., & gatenby, m. (2013). the relationship between line manager behavior, perceived hrm practices, and individual performance: examining the mediating role of engagement. human resource management, 52(6), 839– 859. https://doi.org/10.1002/hrm.21512 alias, m., mohd rasdi, r., ismail, m., & abu samah, b. (2013). predictors of workplace deviant behaviour: hrd agenda for malaysian support personnel. european journal of training and development, 37(2), 161–182. https://doi.org/10.1108/03090591311301671 anandarajan, m., & simmers, c. a. (2005). developing human capital through personal web use in the workplace: mapping employee perceptions. communications of the association for information systems, 15(1). https://doi.org/10.17705/1cais.01541 anderson, j., & gerbing, d. w. (1988). structural equation modeling in practice: a review and recommended two-step approach. psychological bulletin, 103(3), 411–423. https://doi.org/10.1037/0033-2909.103.3.411 arches, j. (1991). social structure, burnout, and job satisfaction. social work. united kingdom: oxford university press. arciniega, l. m., stanley, l. j., puga-méndez, d., obregón-schael, d., & politi-salame, i. (2017). the relationship between individual work values and unethical decision-making and behavior at work. journal of business ethics. https://doi.org/10.1007/s10551-017-3764-3 ashforth, b. e., & mael, f. (1989). social identity theory and the organization. the academy of management review, 14(1), 20–39. https://doi.org/10.2307/258189 askew, k., buckner, j. e., taing, m. u., ilie, a., bauer, j. a., & coovert, m. d. (2014). explaining cyberloafing: the role of the theory of planned behavior. computers in human behavior, 36, 510–519. https://doi.org/10.1016/j.chb.2014.04.006 bagozzi, r. p., & yi, y. (1988). on the evaluation of structural equation models. journal of the academy of marketing science, 16(1), 74–94. https://doi.org/10.1007/bf02723327 bagozzi, r. p., & yi, y. (2012). specification, evaluation, and interpretation of structural equation models. journal of the academy of marketing science, 40(1), 8–34. https://doi.org/10.1007/s11747-011-0278-x bakker, a. b., & demerouti, e. (2008). towards a model of work engagement. career development international, 13(3), 209–223. https://doi.org/10.1108/13620430810870476 baroudi, j., & lucas, h. c. (1994). the role of information technology in organization design. journal of management information systems, 10(4), 9–23. https://doi.org/10.1080/07421222.1994.11518018 baturay, m. h., & toker, s. (2015). an investigation of the impact of demographics on cyberloafing from an educational setting angle. computers in human behavior, 50, 358– 366. https://doi.org/https://doi.org/10.1016/j.chb.2015.03.081 beebe, n., & rao, s. (2005). using situational crime prevention theory to explain the effectiveness of information systems security. in proceedings of the 2005 softwars conference. australasian journal of information systems soral, arayankalam & pandey 2020, vol 24, research article perception of bureaucracy and cyberloafing 27 belanger, f., & van slyke, c. (2002). abuse or learning? commun. acm, 45(1), 64–65. https://doi.org/10.1145/502269.502299 bendix, r. (1960). social stratification and the political community. european journal of sociology, 1(2), 181–210. https://doi.org/10.1017/s0003975600000138 bennis, w. g. (1969). post-bureaucratic leadership. trans-action, 6(9), 44–52. https://doi.org/10.1007/bf02819884 benkhoff, b. (1997). disentangling organizational commitment: the dangers of the ocq for research and policy. personnel review, 26(1/2), 114–131. https://doi.org/10.1108/00483489710157823 blanchard, a. l., & henle, c. a. (2008). correlates of different forms of cyberloafing: the role of norms and external locus of control. comput. hum. behav., 24(3), 1067–1084. https://doi.org/10.1016/j.chb.2007.03.008 bonjean, c. m., & grimes, m. d. (1970). bureaucracy and alienation: a dimensional approach*. social forces, 48(3), 365–373. retrieved from http://dx.doi.org/10.2307/2574655 bretschneider, s., & parker, m. (2016). organization formalization, sector and social media: does increased standardization of policy broaden and deepen social media use in organizations? government information quarterly, 33(4), 614–628. https://doi.org/10.1016/j.giq.2016.09.005 brown, m. e. (1969). identification and some conditions of organizational involvement. administrative science quarterly, 14(3), 346–355. https://doi.org/10.2307/2391129 case, c. j., & young, k. s. (2002). employee internet management: current business practices and outcomes. cyberpsychology & behavior, 5(4), 355–361. https://doi.org/10.1089/109493102760275608 chang, k., & smithikrai, c. (2010). counterproductive behaviour at work: an investigation into reduction strategies. the international journal of human resource management, 21(8), 1272–1288. https://doi.org/10.1080/09585192.2010.483852 chen, j. v, chen, c. c., & yang, h. (2008). an empirical evaluation of key factors contributing to internet abuse in the workplace. industrial management & data systems, 108(1), 87–106. https://doi.org/10.1108/02635570810844106 cheng, b., zhou, x., guo, g., & yang, k. (2018). perceived overqualification and cyberloafing: a moderated-mediation model based on equity theory. journal of business ethics. https://doi.org/10.1007/s10551-018-4026-8 clawson, d. (1980). bureaucracy and labor process: the transformation of u.s. industry, 1860-1920. monthly review press, new york and london. chen, c.-j., & huang, j.-w. (2007). how organizational climate and structure affect knowledge management—the social interaction perspective. international journal of information management, 27(2), 104–118. https://doi.org/https://doi.org/10.1016/j.ijinfomgt.2006.11.001 australasian journal of information systems soral, arayankalam & pandey 2020, vol 24, research article perception of bureaucracy and cyberloafing 28 christian, m. s., & slaughter, j. e. (2007). work engagement: a meta-analytic review and directions for research in an emerging area. in academy of management proceedings (vol. 2007, pp. 1–6). cook, j., & wall, t. (1980). new work attitude measures of trust, organizational commitment and personal need non-fulfilment. journal of occupational psychology, 53(1), 39–52. https://doi.org/10.1111/j.2044-8325.1980.tb00005.x csikszentmihalyi, m. (1997). finding flow: the psychology of engagement with everyday life. finding flow: the psychology of engagement with everyday life. new york, ny, us: basic books. dalal, r. s., brummel, b. j., wee, s., & thomas, l. l. (2008). defining employee engagement for productive research and practice. industrial and organizational psychology, 1(1), 52– 55. https://doi.org/10.1111/j.1754-9434.2007.00008.x damanpour, f. (1991). organizational innovation: a meta-analysis of effects of determinants and moderators. the academy of management journal, 34(3), 555–590. https://doi.org/10.2307/256406 deming, w. e. (1986). out of the crisis. cambridge, mass.: massachusetts institute of technology, center for advanced engineering study. dodder, r. a., & long, j. r. (1980). containment theory reevaluated: an empirical explication. criminal justice review, 5(2), 74–84. https://doi.org/10.1177/073401688000500211 driscoll, d. l. (2011). introduction to primary research: observations, surveys, and interviews. writing spaces: readings on writing, 2, 153-174. dutton, j. e., dukerich, j. m., & harquail, c. v. (1994). organizational images and member identification. administrative science quarterly, 39(2), 239–263. https://doi.org/10.2307/2393235 efraty, d., & wolfe, d. m. (1988). the effect of organizational identification on employee affective and performance responses. journal of business and psychology, 3(1), 105–112. https://doi.org/10.1007/bf01016752 epitropaki, o. (2013). a multi-level investigation of psychological contract breach and organizational identification through the lens of perceived organizational membership: testing a moderated–mediated model. journal of organizational behavior, 34(1), 65–86. https://doi.org/10.1002/job.1793 farnworth, l., mostert, e., harrison, s., & worrell, d. (1996). the experience sampling method: its potential use in occupational therapy research. occupational therapy international, 3(1), 1–17. https://doi.org/10.1002/oti.23 flexon, j. l. (2014). containment theory. in the encyclopedia of theoretical criminology (pp. 1– 5). american cancer society. https://doi.org/10.1002/9781118517390.wbetc023 fornell, c., & larcker, d. f. (1981). structural equation models with unobservable variables and measurement error: algebra and statistics. journal of marketing research, 18(3), 382– 388. https://doi.org/10.2307/3150980 foucault, m. 1979. discipline and punish: the birth of the prison. new york: vintage books. australasian journal of information systems soral, arayankalam & pandey 2020, vol 24, research article perception of bureaucracy and cyberloafing 29 freeman, j., & engel, j. s. (2007). models of innovation: startups and mature corporations. california management review, 50(1), 94–119. https://doi.org/10.2307/41166418 frug, g. e. (1984). the ideology of bureaucracy in american law. harvard law review, 97(6), 1276–1388. https://doi.org/10.2307/1340969 garrett, r. k., & danziger, j. n. (2008). on cyberslacking: workplace status and personal internet use at work. cyberpsychology & behavior, 11(3), 287–292. https://doi.org/10.1089/cpb.2007.0146 glaser, b. g. (1963). the local-cosmopolitan scientist. american journal of sociology, 69(3), 249– 259. https://doi.org/10.1086/223583 gouldner, a. w. (1954). patterns of industrial bureaucracy. patterns of industrial bureaucracy. new york, ny, us: free press. greenberg, d. f. (1977). delinquency and the age structure of society. contemporary crises, 1(2), 189–223. https://doi.org/10.1007/bf00728871 greene, c. n. (1978). identification modes of professionals: relationship with formalization, role strain, and alienation. academy of management journal, 21(3), 486–492. https://doi.org/10.2307/255729 griffiths, m. d. (2012). facebook addiction: concerns, criticism, and recommendations—a response to andreassen and colleagues. psychological reports, 110(2), 518–520. https://doi.org/10.2466/01.07.18.pr0.110.2.518-520 güğerçin, u. (2019). does techno-stress justify cyberslacking? an empirical study based on the neutralisation theory. behaviour & information technology, 1–13. https://doi.org/10.1080/0144929x.2019.1617350 hadlington, l., & parsons, k. (2017). can cyberloafing and internet addiction affect organizational information security? cyberpsychology, behavior, and social networking, 20(9), 567–571. https://doi.org/10.1089/cyber.2017.0239 hair, j. f., black, w. c., babin, b. j., & anderson, r. e. (2010). multivariate data analysis: international version. new jersey, pearson. hall, r. h. (1963). the concept of bureaucracy: an empirical assessment. american journal of sociology, 69(1), 32–40. https://doi.org/10.1086/223508 hall, r. h. (1968). professionalization and bureaucratization. american sociological review, 33(1), 92–104. https://doi.org/10.2307/2092242 hall, d. t., schneider, b., & nygren, h. t. (1970). personal factors in organizational identification. administrative science quarterly, 15(2), 176–190. https://doi.org/10.2307/2391488 hall, b., & khan, b. (2002). adoption of new technology. hartline, m. d., maxham, j. g., & mckee, d. o. (2000). corridors of influence in the dissemination of customer-oriented strategy to customer contact service employees. journal of marketing, 64(2), 35–50. https://doi.org/10.1509/jmkg.64.2.35.18001 harman, h. h. (1960). modern factor analysis. modern factor analysis. oxford, england: univ. of chicago press. australasian journal of information systems soral, arayankalam & pandey 2020, vol 24, research article perception of bureaucracy and cyberloafing 30 hatch, m. j., & schultz, m. (2004). organizational identity: a reader. oxford university press on demand. hebrew university of jerusalem (2019). databases by subject. retrieved from http://www.agri.huji.ac.il/library/subjdata_new.html#multi henle, c. a., & blanchard, a. l. (2008). the interaction of work stressors and organizational sanctions on cyberloafing. journal of managerial issues, 20(3), 383–400. retrieved from http://www.jstor.org/stable/40604617 heffron, f. a. (1989). organization theory and public organizations: the political connection. prentice hall. hess, p. c. (2006). bureaucracy and total quality management: a sociological theory of clashing systems, moralities, and knowledge methods (order no. 3221438). available from abi/inform collection; proquest dissertations & theses global. (305355460). retrieved from https://search.proquest.com/docview/305355460?accountid=47950 hoy, w. k., & sweetland, s. r. (2001). designing better schools: the meaning and measure of enabling school structures. educational administration quarterly, 37(3), 296–321. https://doi.org/10.1177/00131610121969334 jackson, s. e., & schuler, r. s. (1985). a meta-analysis and conceptual critique of research on role ambiguity and role conflict in work settings. organizational behavior and human decision processes, 36(1), 16–78. https://doi.org/10.1016/0749-5978(85)90020-2 jelinek, r., & ahearne, m. (2006). the enemy within: examining salesperson deviance and its determinants. journal of personal selling & sales management, 26(4), 327–344. https://doi.org/10.2753/pss0885-3134260401 jia, h., jia, r., & karau, s. (2013). cyberloafing and personality: the impact of the big five traits and workplace situational factors. journal of leadership & organizational studies, 20(3), 358–365. https://doi.org/10.1177/1548051813488208 jha, j. k., pandey, j., & varkkey, b. (2019). examining the role of perceived investment in employees’ development on work-engagement of liquid knowledge workers. journal of global operations and strategic sourcing. 12(2), 225-245. johnson, p., wood, g., brewster, c., & brookes, m. (2009). the rise of post-bureaucracy: theorists’ fancy or organizational praxis? international sociology, 24(1), 37–61. https://doi.org/10.1177/0268580908100246 kakabadse, a. (1986). organizational alienation and job climate: a comparative study of structural conditions and psychological adjustment. small group behavior, 17(4), 458– 471. https://doi.org/10.1177/104649648601700406 karanika-murray, m., duncan, n., pontes, h. m., & griffiths, m. d. (2015). organizational identification, work engagement, and job satisfaction. journal of managerial psychology, 30(8), 1019–1033. https://doi.org/10.1108/jmp-11-2013-0359 karasek, r., & theorell, t. (1990). healthy work: stress, productivity, and the reconstruction of working life. basic books. retrieved from https://books.google.co.in/books?id=mrxuaaaamaaj australasian journal of information systems soral, arayankalam & pandey 2020, vol 24, research article perception of bureaucracy and cyberloafing 31 kehoe, r. r., & wright, p. m. (2010). the impact of high-performance human resource practices on employees’ attitudes and behaviors. journal of management, 39(2), 366–391. https://doi.org/10.1177/0149206310365901 kennedy, j. p. (2015). losing control: a test of containment theory and ethical decision making. international journal of criminal justice sciences, 10(1). ketchen, d. j., craighead, c. w., & buckley, m. r. (2008). time bandits: how they are created, why they are tolerated, and what can be done about them. business horizons, 51(2), 141– 149. https://doi.org/https://doi.org/10.1016/j.bushor.2007.11.005 khansa, l., kuem, j., siponen, m., & kim, s. s. (2017). to cyberloaf or not to cyberloaf: the impact of the announcement of formal organizational controls. journal of management information systems, 34(1), 141–176. https://doi.org/10.1080/07421222.2017.1297173 kim, s. (2018). managing millennials’ personal use of technology at work. harvard business publishing. kim, k., del carmen triana, m., chung, k., & oh, n. (2016). when do employees cyberloaf? an interactionist perspective examining personality, justice, and empowerment. human resource management, 55(6), 1041–1058. https://doi.org/10.1002/hrm.21699 kim, s. j., & byrne, s. (2011). conceptualizing personal web usage in work contexts: a preliminary framework. computers in human behavior, 27(6), 2271–2283. https://doi.org/https://doi.org/10.1016/j.chb.2011.07.006 koay, k. y., soh, p. c.-h., & chew, k. w. (2017a). antecedents and consequences of cyberloafing: evidence from the malaysian ict industry. first monday; volume 22, number 3 6 march 2017do 10.5210/fm.v22i3.7302 . retrieved from https://firstmonday.org/ojs/index.php/fm/article/view/7302/5968 koay, k. y., soh, p. c.-h., & chew, k. w. (2017b). do employees’ private demands lead to cyberloafing? the mediating role of job stress. management research review, 40(9), 1025– 1038. https://doi.org/10.1108/mrr-11-2016-0252 könig, c. j., & caner de la guardia, m. e. (2014). exploring the positive side of personal internet use at work: does it help in managing the border between work and nonwork? computers in human behavior, 30, 355–360. https://doi.org/10.1016/j.chb.2013.09.021 larose, r., kim, j.-h., & peng, w. (2011). social networking: addictive, compulsive, problematic or just another media habit? in a networked self: identity, community, and culture on social network sites (pp. 59–81). lawrence, t. b., & robinson, s. l. (2007). ain’t misbehavin: workplace deviance as organizational resistance. journal of management, 33(3), 378–394. https://doi.org/10.1177/0149206307300816 lee, d.-c., lin, s.-h., ma, h.-l., & wu, d.-b. (2017). use of a modified utaut model to investigate the perspectives of internet access device users. international journal of human–computer interaction, 33(7), 549–564. https://doi.org/10.1080/10447318.2016.1261249 levinson, h. (1970). a psychologist diagnoses merger failures. harvard business review, 48(2), 139. australasian journal of information systems soral, arayankalam & pandey 2020, vol 24, research article perception of bureaucracy and cyberloafing 32 lewin, k. (1936). pyschology of success and failure. occupations: the vocational guidance journal, 14(9), 926–930. https://doi.org/10.1002/j.2164-5892.1936.tb00275.x li, h., zhang, j., & sarathy, r. (2010). understanding compliance with internet use policy from the perspective of rational choice theory. decision support systems, 48(4), 635–645. https://doi.org/10.1016/j.dss.2009.12.005 li, h., sarathy, r., zhang, j., & luo, x. (2014). exploring the effects of organizational justice, personal ethics and sanction on internet use policy compliance. information systems journal, 24(6), 479–502. https://doi.org/10.1111/isj.12037 liberman, b., seidman, g., mckenna, k. y. a., & buffardi, l. e. (2011). employee job attitudes and organizational characteristics as predictors of cyberloafing. computers in human behavior, 27(6), 2192–2199. https://doi.org/10.1016/j.chb.2011.06.015 lim, v. k. g. (2002). the it way of loafing on the job: cyberloafing, neutralizing and organizational justice. journal of organizational behavior, 23(5), 675–694. https://doi.org/10.1002/job.161 lim, v. k. g., & teo, t. s. h. (2005). prevalence, perceived seriousness, justification and regulation of cyberloafing in singapore: an exploratory study. information & management, 42(8), 1081–1093. https://doi.org/10.1016/j.im.2004.12.002 lim, v. k. g., & chen, d. j. q. (2012). cyberloafing at the workplace: gain or drain on work? behaviour and information technology, 31(4), 343–353. https://doi.org/10.1080/01449290903353054 liu, c., yang, j., liu, j., & zhu, l. (2018). the effect of abusive supervision on employee deviant behaviors: an identity-based perspective. the international journal of human resource management, 1–31. https://doi.org/10.1080/09585192.2018.1511613 macey, w. h., & schneider, b. (2008). the meaning of employee engagement. industrial and organizational psychology, 1(1), 3–30. https://doi.org/10.1111/j.1754-9434.2007.0002.x macher, k. (1988). empowerment and the bureaucracy. training & development journal, 42(9), 41. marasi, s., bennett, r., & budden, h. (2018). the structure of an organization: does it influence workplace deviance and its’ dimensions? and to what extent? journal of managerial issues, 30. martin, l. e., brock, m. e., buckley, m. r., & ketchen jr., d. j. (2010). time banditry: examining the purloining of time in organizations. human resource management review, 20(1), 26– 34. https://doi.org/10.1016/j.hrmr.2009.03.013 maslach, c., schaufeli, w. b., & leiter, m. p. (2001). job burnout. annual review of psychology, 52(1), 397–422. https://doi.org/10.1146/annurev.psych.52.1.397 mackenzie, s. b., & podsakoff, p. m. (2012). common method bias in marketing: causes, mechanisms, and procedural remedies. journal of retailing, 88(4), 542–555. https://doi.org/10.1016/j.jretai.2012.08.001 mcafee, a., brynjolfsson, e., davenport, t. h., patil, d. j., & barton, d. (2012). big data: the management revolution. harvard business review, 90(10), 60–68. retrieved from http://tarjomefa.com/wp-content/uploads/2017/04/6539-english-tarjomefa-1.pdf australasian journal of information systems soral, arayankalam & pandey 2020, vol 24, research article perception of bureaucracy and cyberloafing 33 mcguigan, l. (2005). the role of enabling bureaucracy and academic optimism in academic achievement growth. the ohio state university. montagna, p. d. (1968). professionalization and bureaucratization in large professional organizations. american journal of sociology, 74(2), 138–145. https://doi.org/10.1086/224616 morton, n. a., & hu, q. (2008). implications of the fit between organizational structure and erp: a structural contingency theory perspective. international journal of information management, 28(5), 391–402. https://doi.org/https://doi.org/10.1016/j.ijinfomgt.2008.01.008 mowday, r. t., steers, r. m., & porter, l. w. (1979). the measurement of organizational commitment. journal of vocational behavior, 14(2), 224–247. https://doi.org/10.1016/00018791(79)90072-1 nevitt, j., & hancock, g. r. (2004). evaluating small sample approaches for model test statistics in structural equation modeling. multivariate behavioral research, 39(3), 439– 478. https://doi.org/10.1207/s15327906mbr3903_3 newman, d. a., & harrison, d. a. (2008). been there, bottled that: are state and behavioral work engagement new and useful construct “wines”? industrial and organizational psychology, 1(1), 31–35. https://doi.org/10.1111/j.1754-9434.2007.00003.x nicholson, p. j., & goh, s. c. (1983). the relationship of organization structure and interpersonal attitudes to role conflict and ambiguity in different work environments. academy of management journal, 26(1), 148–155. https://doi.org/10.2307/256141 nishii, l. h., lepak, d. p., & schneider, b. (2008). employee attributions of the “why” of hr practices: their effects on employee attitudes and behaviors, and customer satisfaction. personnel psychology, 61(3), 503–545. https://doi.org/10.1111/j.1744-6570.2008.00121.x o’neill, t. a., hambley, l. a., & chatellier, g. s. (2014). cyberslacking, engagement, and personality in distributed work environments. computers in human behavior, 40, 152–160. https://doi.org/10.1016/j.chb.2014.08.005 osborne, d., & plastrik, p. (1997). banishing bureaucracy: the five strategies for reinventing government. addison-wesley publishing company, inc. pandey, j. (2018). managing employee surplus: the cats, rats, camels and elephants of strategic downsizing matrix. strategic hr review. 17 (4), 220-223 pandey, j. (2019). factors affecting job performance: an integrative review of literature. management research review. 42(2). 263-289 pindek, s., krajcevska, a., & spector, p. e. (2018). cyberloafing as a coping mechanism: dealing with workplace boredom. computers in human behavior, 86, 147–152. https://doi.org/10.1016/j.chb.2018.04.040 pines, a., & maslach, c. (1978). characteristics of staff burnout in mental health settings. hospital & community psychiatry. us: american psychiatric assn. pratt, m. g. (1998). to be or not to be: central questions in organizational identification. in identity in organizations: building theory through conversations. (pp. 171–207). thousand oaks, ca, us: sage publications, inc. https://doi.org/10.4135/9781452231495.n6 australasian journal of information systems soral, arayankalam & pandey 2020, vol 24, research article perception of bureaucracy and cyberloafing 34 priesemuth, m., schminke, m., ambrose, m. l., & folger, r. (2013). abusive supervision climate: a multiple-mediation model of its impact on group outcomes. academy of management journal, 57(5), 1513–1534. https://doi.org/10.5465/amj.2011.0237 rammstedt, b., & john, o. p. (2007). measuring personality in one minute or less: a 10-item short version of the big five inventory in english and german. journal of research in personality, 41(1), 203–212. https://doi.org/10.1016/j.jrp.2006.02.001 reckless, w. c. (1961). the crime problem (ch. 18). new york: appleton century crofts. reckless, w.c. (1967). the crime problem. new york: appleton-century-crofts reckless, w., & dinitz, s. (1968). pioneering with self--concept as a vulnerability factor in delinquency. journal of criminal law and criminology, 58(4). retrieved from https://scholarlycommons.law.northwestern.edu/jclc/vol58/iss4/6 rich, b. l., lepine, j. a., & crawford, e. r. (2010). job engagement: antecedents and effects on job performance. academy of management journal, 53(3), 617–635. https://doi.org/10.5465/amj.2010.51468988 richards, d. a., & schat, a. c. h. (2011). attachment at (not to) work: applying attachment theory to explain individual behavior in organizations. journal of applied psychology, 96(1), 169-182. https://doi.org/10.1037/a0020372 robinson, s. l., & bennett, r. j. (1995). a typology of deviant workplace behaviors: a multidimensional scaling study. academy of management journal, 38(2), 555–572. https://doi.org/10.5465/256693 rousseau, d. m. (1978). characteristics of departments, positions, and individuals: contexts for attitudes and behavior. administrative science quarterly, 23(4), 521–540. https://doi.org/10.2307/2392578 salary.com. (2014). wasting time at work survey. retrieved january 3, 2019, from https://www.salary.com/chronicles/2014-wasting-time-at-work/ saparito, p. a., & coombs, j. e. (2013). bureaucratic systems’ facilitating and hindering influence on social capital. entrepreneurship theory and practice, 37(3), 625–639. https://doi.org/10.1111/etap.12028 schaufeli, w. b., & bakker, a. b. (2004). job demands, job resources, and their relationship with burnout and engagement: a multi-sample study. journal of organizational behavior, 25(3), 293–315. https://doi.org/10.1002/job.248 schaufeli, w. b., salanova, m., gonzález-romá, v., & bakker, a. b. (2002). the measurement of engagement and burnout: a two sample confirmatory factor analytic approach. journal of happiness studies, 3(1), 71–92. https://doi.org/10.1023/a:1015630930326 scheuermann, l. e., & langford, h. p. (1997). perceptions of internet abuse, liability, and fair use. perceptual and motor skills, 85(3), 847–850. https://doi.org/10.2466/pms.1997.85.3.847 schott, r. l. (2000). the origins of bureaucracy: an anthropological perspective. international journal of public administration, 23(1), 53–78. https://doi.org/10.1080/01900690008525452 seeman, m. (1971). the urban alienations: some dubious theses from marx to marcuse. journal of personality and social psychology, 19(2), 135–143. https://doi.org/10.1037/h0031270 australasian journal of information systems soral, arayankalam & pandey 2020, vol 24, research article perception of bureaucracy and cyberloafing 35 shantz, a., alfes, k., truss, c., & soane, e. (2013). the role of employee engagement in the relationship between job design and task performance, citizenship and deviant behaviours. the international journal of human resource management, 24(13), 2608–2627. https://doi.org/10.1080/09585192.2012.744334 sheikh, a., atashgah, m. s., & adibzadegan, m. (2015). the antecedents of cyberloafing: a case study in an iranian copper industry. computers in human behavior, 51(pt a), 172– 179. https://doi.org/10.1016/j.chb.2015.04.042 shepherd, m., mejias, r., & klein, g. (2014). a longitudinal study to determine non-technical deterrence effects of severity and communication of internet use policy for reducing employee internet abuse. in 2014 47th hawaii international conference on system sciences (pp. 3159–3168). https://doi.org/10.1109/hicss.2014.392 shrivastava, a., sharma, m. k., & marimuthu, p. (2016). internet use at workplaces and its effects on working style in indian context: an exploration. indian journal of occupational and environmental medicine, 20(2), 88–94. https://doi.org/10.4103/0019-5278.197531 smith, d.(2013).five principles for research ethics: cover your bases with these ethical strategies. monitor on psychology, 34, 56. retrieved from http://www.apa.org/monitor/jan03/principles.aspx smidts, a., pruyn, a. t. h., & van riel, c. b. m. (2001). the impact of employee communication and perceived external prestige on organizational identification. academy of management journal, 49(5), 1051–1062. retrieved from http://www.jstor.org/stable/3069448 soh, p. c.-h., koay, k. y., & lim, v. k. g. (2018). understanding cyberloafing by students through the lens of an extended theory of planned behavior. first monday; volume 23, number 6 4 june 2018. https://doi.org/10.5210/fm.v23i6.7837 spector, p. e., fox, s., penney, l. m., bruursema, k., goh, a., & kessler, s. (2006). the dimensionality of counterproductivity: are all counterproductive behaviors created equal? journal of vocational behavior, 68(3), 446–460. https://doi.org/10.1016/j.jvb.2005.10.005 stephens, k. k., & ford, j. l. (2015). unintended consequences of a strategically ambiguous organizational policy selectively restricting mobile device use at work. mobile media & communication, 4(2), 186–204. https://doi.org/10.1177/2050157915619211 stuart, h. (1999). the effect of organizational structure on corporate identity management. corporate reputation review, 2(2), 151–164. https://doi.org/10.1057/palgrave.crr.1540075 tajfel, h. (1982). social psychology of intergroup relations. annual review of psychology, 33(1), 1–39. https://doi.org/10.1146/annurev.ps.33.020182.000245 tepper, b. j., duffy, m. k., & shaw, j. d. (2001). personality moderators of the relationship between abusive supervision and subordinates’ resistance. journal of applied psychology, 86(5), 974–983. https://doi.org/10.1037/0021-9010.86.5.974 ugrin, j. c., & michael pearson, j. (2013). the effects of sanctions and stigmas on cyberloafing. computers in human behavior, 29(3), 812–820. https://doi.org/10.1016/j.chb.2012.11.005 australasian journal of information systems soral, arayankalam & pandey 2020, vol 24, research article perception of bureaucracy and cyberloafing 36 ugrin, j. c., pearson, j. m., & nickle, s. m. (2018). an examination of the relationship between culture and cyberloafing using the hofstede model. journal of internet commerce, 17(1), 46–63. https://doi.org/10.1080/15332861.2018.1424395 ugwu, f. o., & onyishi, i. e. (2017). linking perceived organizational frustration to work engagement: the moderating roles of sense of calling and psychological meaningfulness. journal of career assessment, 26(2), 220–239. https://doi.org/10.1177/1069072717692735 utaybi, s. m. (1992). political development, bureaucracy and saudi culture. florida state university. van doorn, o. n. (2011). cyberloafing a multi-dimensional construct placed in a theoretical framework. eindhoven university of technology. van de voorde, k., & beijer, s. e. (2015). the role of employee hr attributions in the relationship between high-performance work systems and employee outcomes. human resource management journal, 25(1), 62–78. https://doi.org/10.1111/1748-8583.12062 van dick, r., ullrich, j., & tissington, p. a. (2006). working under a black cloud: how to sustain organizational identification after a merger*. british journal of management, 17(s1), s69–s79. https://doi.org/10.1111/j.1467-8551.2006.00479.x van wyk, r., boshoff, a. b., & cilliers, f. v. n. (2003). the prediction of job involvement for pharmacists and accountants. sa journal of industrial psychology, 29(3), 61–67. https://doi.org/10.4102/sajip.v29i3.118 van wyk, s. m., de beer, l. t., pienaar, j., & schaufeli, w. b. (2016). the psychometric properties of a workplace boredom scale (dubs) within the south african context. sa journal of industrial psychology; vol 42, no 1 (2016)do 10.4102/sajip.v42i1.1326 . retrieved from https://sajip.co.za/index.php/sajip/article/view/1326/1913 vaast, e., & binz-scharf, m. c. (2008). bringing change in government organizations: evolution towards post-bureaucracy with web-based it projects. icis 2008 proceedings, 213. von mises, l. (1944). the treatment of “irrationality” in the social sciences. philosophy and phenomenological research, 4(4), 527–546. https://doi.org/10.2307/2103039 wagner, d. t., barnes, c. m., lim, v. k. g., & ferris, d. l. (2012). lost sleep and cyberloafing: evidence from the laboratory and a daylight saving time quasi-experiment. journal of applied psychology, 97(5), 1068-1076.https://doi.org/10.1037/a0027557 wales, w. j., patel, p. c., & lumpkin, g. t. (2013). in pursuit of greatness: ceo narcissism, entrepreneurial orientation, and firm performance variance. journal of management studies, 50(6), 1041–1069. https://doi.org/10.1111/joms.12034 weber, m. (1958). essays in sociology. a galaxy book. werner, c., & schermelleh-engel, k. (2010). deciding between competing models: chi-square difference tests. goethe university. available online: https://perma. cc/2rtr-8xpz). west, s. g., taylor, a. b., & wu, w. (2012). model fit and model selection in structural equation modeling. in handbook of structural equation modeling. (pp. 209–231). new york, ny, us: guilford press. australasian journal of information systems soral, arayankalam & pandey 2020, vol 24, research article perception of bureaucracy and cyberloafing 37 yeik, k. k. (2018). workplace ostracism and cyberloafing: a moderated–mediation model. internet research, 28(4), 1122–1141. https://doi.org/10.1108/intr-07-2017-0268 yellowlees, p. m., & marks, s. (2007). problematic internet use or internet addiction? computers in human behavior, 23(3), 1447–1453. https://doi.org/10.1016/j.chb.2005.05.004 yoon, j., baker, m. r., & ko, j.-w. (1994). interpersonal attachment and organizational commitment: subgroup hypothesis revisited. human relations, 47(3), 329–351. https://doi.org/10.1177/001872679404700305 zey-ferrell, m., weaver, k. m., & ferrell, o. c. (1979). predicting unethical behavior among marketing practitioners. human relations, 32(7), 557–569. https://doi.org/10.1177/001872677903200702 zill-e-, h., hussain, s., & thurasamy, r. (2017). determinants of cyberloafing: a comparative study of a public and private sector organization. internet research, 27(1), 97–117. https://doi.org/10.1108/intr-12-2014-0317 zimmerman, j. (2001). the effects of bureaucratization on corruption, deviant and unethical behavior in organizations. journal of managerial issues, 13(1), 119–128. retrieved from http://www.jstor.org/stable/40604337 zhao, h., wayne, s. j., glibkowski, b. c., & bravo, j. (2007). the impact of psychological contract breach on work-related outcomes: a meta-analysis. personnel psychology, 60(3), 647–680. https://doi.org/10.1111/j.1744-6570.2007.00087.x zoghbi manrique de lara, p. (2006). fear in organizations. journal of managerial psychology, 21(6), 580–592. https://doi.org/10.1108/02683940610684418 zoghbi manrique de lara, p. (2009). inequity, conflict, and compliance dilemma as causes of cyberloafing. international journal of conflict management, 20(2), 188–201. https://doi.org/10.1108/10444060910949630 zoghbi manrique de lara, p., verano tacoronte, d. and ting ding, j. (2006), "do current anticyberloafing disciplinary practices have a replica in research findings?", internet research, vol. 16 no. 4, pp. 450-467. https://doi.org/10.1108/10662240610690052 zoghbi-manrique-de-lara, p., & viera-armas, m. (2017). corporate culture as a mediator in the relationship between ethical leadership and personal internet use. journal of leadership & organizational studies, 24(3), 357–371. https://doi.org/10.1177/1548051817696877 australasian journal of information systems soral, jithesh & pandey 2020, vol 24, research article ambivalent perception and cyberloafing 38 annexure i systematic literature review s.no. authors antecedents context/whic h industry sample characteristics analytical strategy theory 1 (aghaz & sheikh, 2016) job burnout – emotional exhaustion, decreased personal efficacy and cynicism organizationa l context 298 employees pearson correlation coefficients confirmatory factor analysis na 2 (arciniega, stanley, puga-méndez, obregón-schael, & politi-salame, 2017) self-enhancement organizationa l context 145 employees hierarchical logistic regression schwartz’s theory of individual values 3 (askew et al., 2014) descriptive norms, cyberloafing attitudes, ability to hide cyberloafing, intentions organizationa l context 429 employees regression theory of planned behavior 4 (baturay & toker, 2015) demographic variables gender, grade, internet skills, internet usage and internet experience university context 282 student respondents descriptive statistics multiple linear regression analysis structural equation modeling na 5 (blanchard & henle, 2008) employees’ perceptions of coworker and supervisor norms supporting cyberloafing, external locus of control organizationa l context 202 executive mba students hierarchical regression na australasian journal of information systems soral, jithesh & pandey 2020, vol 24, research article ambivalent perception and cyberloafing 39 6 (cheng, zhou, guo, & yang, 2018) perceived overqualification, need for achievement, harmonious passion organizationa l context 456 in first wave, 382 in second wave structural equation modelling equity theory 7 (güğerçin, 2019) techno-stress (techno-overload, techno-complexity, technoinvasion) organizationa l context 252 white-collar employees regression analysis neutralisation theory 8 (henle & blanchard, 2008) role ambiguity, role conflict, role overload, perceived organizational sanctions organizationa l context 194 executive mba students hierarchical regression analysis role theory 9 (huma, hussain, thurasamy, & malik, 2017) affect, social factor, perceived consequences, habit, facilitating condition, intention organizationa l context 85 responses from a public airline and 85 responses from a private sector airline. sem theory of interpersonal behavior (tib). 10 (jia, jia, & karau, 2013) big five personality factors (conscientiousness, emotional stability, extroversion, openness to experience), presence of an internet usage policy, perceived work meaningfulness organizationa l context 147 employees hierarchal regression general deterrence theory 11 (khansa, kuem, siponen, & kim, 2017) neutralization, perceived risk, past cyberloafing, peer cyberloafing organizationa l context/ 451 employees structural equation modeling (sem) akers’s social learning theory 12 (kim, del carmen triana, chung, & oh, 2016) conscientiousness, emotional stability, organizational justice organizationa l context 247 employees cfa, hierarchical linear regression. trait activation theory, organizationa l justice australasian journal of information systems soral, jithesh & pandey 2020, vol 24, research article ambivalent perception and cyberloafing 40 13 (yeik, 2018) workplace ostracism, emotional exhaustion organizationa l context 179 employees sem conservation of resources theory 14 (koay, soh, & chew, 2017a) intention, perceived consequence, affect social factors, facilitating conditions, habit organizationa l context 301 employees variance-based structural equation modelling theory of interpersonal behavior 15 (koay et al., 2017b) private demands, jobs stress organizationa l context 301 employees sem border theory, conservation of resources theory and general strain theory 16 (könig & caner de la guardia, 2014) private demands, job identification organizationa l context 190 employees hierarchical multiple regression work/family border theory 17 (lee, lin, ma, & wu, 2017) perceived enjoyment, performance expectancy, effort expectancy, social influence, age, gender, educational level organizationa l context 503 employees regression johnson–neyman to verify the moderate effects utaut 18 (liberman et al., 2011) job attitudes (job involvement and intrinsic involvement), organizational characteristics (perceived cyberloafing of one’s co-workers managerial support for internet usage), attitudes towards cyberloafing, other noninternet loafing behaviors organizationa l context 143 employees regression na 19 (lim, 2002) organizational justice (distributive, procedural and interactional), metaphor of the ledger organizationa l context 188 employees structural equation modeling, interview social exchange theory, organizationa australasian journal of information systems soral, jithesh & pandey 2020, vol 24, research article ambivalent perception and cyberloafing 41 l justice, neutralization theory 20 (o’neill, hambley, & chatellier, 2014) agreeableness, neuroticism, openness to experience, conscientiousness organizationa l context 148 employees hierarchal regression relative importance analysis (ria) person-job fit trait activation theory 21 (pindek et al., 2018) boredom at work, work underload university context 463 non-teaching staff maximum likelihood estimation (mlr) na 22 (sheikh, atashgah, & adibzadegan, 2015) ability to hide cyberloafing, subjective norms, behavioural attitude, attitudes & intentions toward cyberloafing university context 195 employees linear regression analysis theory of planned behaviour (tpb) 23 (soh, koay, & lim, 2018) class engagement, habit, attitude, prescriptive norms, perceived behavioural control university context 238 university students consistent partial least squares (pls) theory of planned behavior (tpb) 24 (ugrin & michael pearson, 2013) sanction, detection, enforcement, perceived abusiveness organizationa l context 69 business students, 81 employees experimental survey annova general deterrence theory (gdt) 25 (ugrin et al., 2018) power distance, masculinity versus femininity, uncertainty avoidance, long-term versus short-term orientation, indulgence versus restraint organizationa l context 249 executive mba and mba students from 17 countries regression na 26 (wagner, barnes, lim, & ferris, 2012) sleep quality, day light saving time (dst), conscientiousness university context 96 students hierarchical linear modeling (hlm) ego depletion model of selfregulation australasian journal of information systems soral, jithesh & pandey 2020, vol 24, research article ambivalent perception and cyberloafing 42 27 zoghbi manrique de lara, 2006) interactional justice, fear of formal punishment university context 147 non-teaching staff structural equation modeling na 28 (zoghbi manrique de lara, tacoronte, & ding, 2006) physical leadership proximity, perceived organizational control, fear of formal punishment university context 147 non teaching staff structural equation modeling social exchange theory, control theory 29 (zoghbi manrique de lara, 2009) procedural justice, perceptions of normative conflict university context 147 non teaching staff structural equation modeling na 30 (zoghbi manrique de lara & viera-armas, 2017) ethical leadership, corporate culture [clan culture, market culture, and hierarchal culture] organizationa l context 300 employees structural equation modelling social learning theory australasian journal of information systems soral, jithesh & pandey 2020, vol 24, research article ambivalent perception and cyberloafing 43 annexure ii classification of antecedents perception state behaviour attitude personality environment skill emotion demographics others neutralization, perceived risk, peer cyberloafing, organizational justice (procedural justice, interactional justice, distributive justice), perception of normative conflict, physical leadership proximity, metaphor of the ledger, perceived work meaningfulness, perceived consequences, social factors, ethical leadership, corporate culture (clan culture, market culture, and hierarchal culture), employees’ perceptions of co-worker, supervisor norms supporting cyberloafing, work underload, organizational characteristics (perceived cyberloafing of one’s co-workers, managerial support for internet usage), descriptive norms, subjective norms, prescriptive norms, sanction, detection, enforcement, perceived abusiveness, role ambiguity, role conflict, role overload, class engagement, perceived behavioural control, workplace ostracism, perceived enjoyment. power distance, masculinity versus femininity, uncertainty avoidance, long-term versus shortterm orientation, indulgence versus restraint, job burnout, boredom at work, job stress, emotional exhaustion, technostress. pastcyberloafing, internet usage, internet experience, habit, other non-internet loafing behaviors, performance expectancy, effort expectancy. job attitude (job involvement and intrinsic involvement), cyberloafing attitudes, job identification, behavioural attitude. big five personality, external locus of control, procrastination, honesty, general self-efficacy. presence of an internet usage policy, facilitating conditions, day light saving time. internet skills, ability to hide cyberloafin g, selfregulation. fear of formal punishment, affect, harmonious passion. gender, grade, age, educational level intention, private demands, sleep quality, need for achievement, self enhancement australasian journal of information systems soral, jithesh & pandey 2020, vol 24, research article ambivalent perception and cyberloafing 44 copyright: © 2020 soral, arayankalam & pandey. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v24i0.2087 http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 literature review 2.1 cyberloafing 2.2 bureaucracy 2.3 organizational identification 2.4 work engagement 3 theory and hypotheses 3.1 coercive bureaucracy 3.2 enabling bureaucracy 4 methods 4.1 sample and procedures 4.2 measures 4.2.1 perception of bureaucracy 4.2.2 cyberloafing 4.2.3 work engagement 4.2.4 organizational identification 4.2.5 control variables 5 results 6 discussion 6.1 theoretical contributions 6.2 practical contributions 7 limitations and future directions 8 conclusion paper title (paper title style) australasian journal of information systems lim, rasul & clarke 2020, vol 24, post publication review the challenges involved in establishing a research technique 1 a post publication review of the challenges involved in establishing a research technique clarke, r. (2020). the challenges involved in establishing a research technique. australasian journal of information systems, 24. https://doi.org/10.3127/ajis.v24i0.2515 review the progress of information systems (is) as a discipline is reliant on the gaps that is scholars identify and address, whom, in our view, can do so in a few notable ways. most often, is scholars work with gaps that justify conceptual and empirical research (e.g., quantitative, qualitative), wherein gaps may emerge from either is theory (e.g., lim et al., 2019; robinson, 2020; soral et al., 2020), practice (e.g., naqvi et al., 2020; pereira et al., 2020; warren, 2020), or both (e.g., alhassan et al., 2020; nayal & pandey, 2020). moreover, the maturity of the is discipline has coincided with the emergence of systematic reviews (lim, 2020; mazaheri et al., 2020), which typically rely on a set of protocols and procedures to produce a state-ofthe-art overview of existing knowledge and an agenda for future research to close extant gaps and drive the field forward (e.g., aljaroodi et al., 2019; chua & zhang, 2020; hacker et al., 2019; hinton et al., 2019). more recently, clarke (2020) brings to light of an underrated, and sometimes forgotten, form of review in is—that is, critical reviews (and though clarke did not mention “critical reviews” explicitly, we opine that “critical analysis of articles,” “critiques of prior works,” and “meta-discussions” may implicitly refer to “critical reviews”). unlike systematic reviews, whose methodologies and variations have been well established (hulland & houston, 2020; palmatier et al., 2018; paul & criado, 2020), the art of writing critical reviews has remained elusive. the seminal paper of clarke (2020) sought to address this gap by elucidating the pertinent aspects of critical reviews and by presenting a guide that is scholars can rely upon to produce critical reviews. we agree with clarke (2020) that critical reviews have an important role to play in advancing the is discipline given that critical thought—which in principle, is refutable, provisional, and subject to testing, with the outcome leading to discovery and progress—is central to the notion of science. yet, we empathize with the rejections encountered by clarke (2020) in the pursuit of publishing critical reviews in is journals, and we understand the “nervousness” that journal editors may feel when they receive manuscripts that are critical of published work, especially those appearing in their own journals. this “nervousness” may be due, in part, to the misconceived understanding of “criticism” that entails in critical reviews, as clarke (2020) rightly pointed out, wherein “criticism”, as a concept and practice, should draw attention to both the positive and negative aspects of a subject that may have been ignored, disregarded, or overlooked. nevertheless, we felt that the guide proposed by clarke (2020) for critical reviews may have been, to a certain extent, biased by his negative experience of trying to publish critical reviews in the past. in particular, we found the guide to be very prescriptive, which in our view, is a positive criticism that we wish to highlight, as we believe that early career researchers and higher degree research students are more likely to benefit from prescriptive rather than superficial guides for independent learning. australasian journal of information systems lim, rasul & clarke 2020, vol 24, post publication review the challenges involved in establishing a research technique 2 yet, we contend that traditional research techniques, such as the four-way classification of content analysis proposed by clarke (2020), may not be necessary for critical reviews. interestingly, clarke (2020) did acknowledge the problematic issues of systematic research techniques that typify systematic reviews (e.g., undue constraints that limit dialogical interaction between the literature and the researcher, thereby limiting creativity and insights; boell & cecez-kecmanovic, 2014; maclure, 2005), which the proposed content analysis would reasonably fall under. we also found two papers that clarke (2020) cited to demonstrate the application of the guide as useful exemplars to support our contention. in particular, we observed more critical thought and implications in clarke (2015) as opposed to clarke (2016), which we believe may be attributed to the choice of research technique employed, wherein the latter appears to be more rigid and systematic than the former, thereby supporting our contention to move away from traditional research techniques and to more actively engage in what critical reviews ought to be doing. that is, we believe that critical reviews should not try to replicate systematic reviews—be it in style or substance. instead, we opine that critical reviews should be “courageous” and “purposeful,” wherein “courageous” refers to picking out and interacting with conflicts, dilemmas, and paradoxes, whereas “purposeful” relates to consolidating and harmonizing insights for clarity and progress in the field. more importantly, is scholars must be aware of the different forms that critical reviews may assume, such as research articles and post-published reviews, and develop their critical reviews accordingly. for example, critical reviews submitted as research articles should deal with topical issues in a specific area in the is discipline (e.g., gupta et al., 2018; lim, 2018; namvar et al., 2018; samhan, 2018), whereas critical reviews submitted as post-published reviews should deal with topical issues arising from a recent publication in the journal (e.g., burmeister, 2020; koh & kwok, 2018; poulsen et al., 2019). to this end, we are happy to see that the australasian journal of information systems actively publishes critical reviews as part of the journal’s commitment to curate constructive discussions leading to the strengthening of ideas and arguments in the is discipline, and long may this valuable platform and practice continue. weng marc lim swinburne university of technology lim@wengmarc.com / marclim@swin.edu.au / wlim@swinburne.edu.my tareq rasul australian institute of business author response i welcome the comments made in the review. they lead me to offer some clarification of two aspects of the original article. project-types that the research technique supports the scope of the research technique that i am endeavouring to establish is "the critical analysis of {the content of} published works" (p.2). my intention is that researchers "apply the abstract principles in order to customise a research method appropriate to each particular project" (p.15). one part of the customisation process involves establishing an operational definition of relevant 'published works' and 'publishing australasian journal of information systems lim, rasul & clarke 2020, vol 24, post publication review the challenges involved in establishing a research technique 3 venues' (p.3). further decisions each researcher needs to make relate to the population segment, sampling frame and sample selection (p.4). as the reviewers note, the technique can be applied to projects of the nature of 'critical literature reviews', which by their nature involve "substantial bodies of prior research" (p.9). although the critical technique naturally bears some similarities to 'systematic literature review', the differences are substantial. however, the technique has application to collections of published works that do not represent 'a literature'. examples provided in the article include "one or more journal special issues, narrowly-specialised conferences, or academic books", and publications that address a particular research domain, adopt a particular theoretical lens, or apply a particular research technique, and even, in rare instances, a single, specific work (p.4). other categories are the complete oeuvre of a particular author, output of a particular research programme, or major works of a 'school of thought'. all of these applications of the technique go beyond "exposition, application, or at most interpretation" (p.10). as the reviewers comment, critical analysis is appropriate only where the researcher is conscious of the challenges they are taking up, the research design is 'purposeful', and the researcher's demeanour is 'courageous'. i would add that the researcher's demeanour and expression need to be 'constructive' and 'not combative', and their disposition 'calm' and 'patient', in order to deal equably with any editorial or review comments that are not attuned to the approach, or are excessively defensive of the status quo. research objects that the research technique encompasses the reviewers noted that critical analysis has application to various forms of publication, including research articles and post-published reviews. however, there is a further dimension that the original article may not have sufficiently emphasised. the article distinguishes the approaches of positivism, interpretivism, design science and critical theory research (p.8), and asserts that "the scope of the critical analysis of published works research technique is intended to encompass [all of these] traditions" (p.13). the guidance provided is intended to be sufficiently flexible that analysts can evaluate each work in its own terms, but incorporating an appreciation of the original author's own context. their analysis can then be tailored to reflect, for example, the greater commitment of positivist research to rigour, the instrumentalist and hence 'agenda-driven' nature of design science, and the inherently value-laden exhortations of critical theory research. roger clarke roger.clarke@xamax.com.au roger clarke is principal of xamax consultancy pty ltd, canberra. he is also a visiting professor associated with the allens hub for technology, law and innovation in unsw law, and a visiting professor in the research school of computer science at the australian national university. references aguinis, h., ramani, r. s., & alabduljader, n. (2020). best-practice recommendations for producers, evaluators, and users of methodological literature reviews. organizational research methods. https://doi.org/10.1177/1094428120943281 australasian journal of information systems lim, rasul & clarke 2020, vol 24, post publication review the challenges involved in establishing a research technique 4 aljaroodi, h. m., adam, m. t., chiong, r., & teubner, t. (2019). avatars and embodied agents in experimental information systems research: a systematic review and conceptual framework. australasian journal of information systems, 23. https://doi.org/10.3127/ajis.v23i0.1841 boell, s. k., & cecez-kecmanovic, d. (2015). on being ‘systematic’ in literature reviews. in formulating research methods for information systems (pp. 48-78). london: palgrave macmillan. https://doi.org/10.1057/9781137509888_3 burmeister, o. k. (2020). a post publication review of “understanding the effects of compromise and misuse of personal details on older people”. australasian journal of information systems, 24. https://doi.org/10.3127/ajis.v24i0.2839 chua, b. b., & zhang, y. (2020). applying a systematic literature review and content analysis method to analyse open source developers’ forking motivation interpretation, categories and consequences. australasian journal of information systems, 24. https://doi.org/10.3127/ajis.v24i0.1714 clarke, r. (2020). the challenges involved in establishing a research technique. australasian journal of information systems, 24. https://doi.org/10.3127/ajis.v24i0.2515 clarke, r. (2016). an empirical assessment of researcher perspectives. bled econference, slovenia. clarke r. (2015). not only horses wear blinkers: the missing perspectives in is research. australasian conference on information systems, adelaide, australia. https://arxiv.org/pdf/1611.04059 gupta, g., tan, k. t. l., ee, y. s., & phang, c. s. c. (2018). resource-based view of information systems: sustainable and transient competitive advantage perspectives. australasian journal of information systems, 22. https://doi.org/10.3127/ajis.v22i0.1657 hacker, j. v., johnson, m., saunders, c., & thayer, a. l. (2019). trust in virtual teams: a multidisciplinary review and integration. australasian journal of information systems, 23. https://doi.org/10.3127/ajis.v23i0.1757 hinton, s., wood, l. c., singh, h., & reiners, t. (2019). enterprise gamification systems and employment legislation: a systematic literature review. australasian journal of information systems, 23. https://doi.org/10.3127/ajis.v23i0.2037 hulland, j., & houston, m. b. (2020). why systematic review papers and meta-analyses matter: an introduction to the special issue on generalizations in marketing. journal of the academy of marketing science, 48, 351–359. https://doi.org/10.1007/s11747-020-00721-7 koh, s. g., & kwok, a. o. (2018). a post publication review of “an investigation into failure of internet firms: towards development of a conceptual model”. australasian journal of information systems, 22. https://doi.org/10.3127/ajis.v22i0.1840 lim, w. m. (2020). a post-publication review of “research directions in information systems field, current status and future trends: a literature analysis of ais basket of top journals”. australasian journal of information systems, 24. https://doi.org/10.3127/ajis.v24i0.2921 lim, w. m. (2018). dialectic antidotes to critics of the technology acceptance model: conceptual, methodological, and replication treatments for behavioural modelling in australasian journal of information systems lim, rasul & clarke 2020, vol 24, post publication review the challenges involved in establishing a research technique 5 technology-mediated environments. australasian journal of information systems, 22. https://doi.org/10.3127/ajis.v22i0.1651 lim, w. m., lim, a. l., & phang, c. s. c. (2019). toward a conceptual framework for social media adoption by non-urban communities for non-profit activities: insights from an integration of grand theories of technology acceptance. australasian journal of information systems, 23. https://doi.org/10.3127/ajis.v23i0.1835 maclure, m. (2005). ‘clarity bordering on stupidity’: where’s the quality in systematic review? journal of education policy, 20(4), 393–416. https://doi.org/10.1080/02680930500131801 mazaheri, e., lagzian, m., & hemmat, z. (2020). research directions in information systems field, current status and future trends. australasian journal of information systems, 24. https://doi.org/10.3127/ajis.v24i0.2045 namvar, m., cybulski, j., phang, c., wee, c. y., & tan, k. y. (2018). simplifying sensemaking: concept, process, strengths, shortcomings, and ways forward for information systems in contemporary business environments. australasian journal of information systems, 22. https://doi.org/10.3127/ajis.v22i0.1654 naqvi, n., ur rehman, s., & islam, z. (2020). a hyperconnected smart city framework. australasian journal of information systems, 24. https://doi.org/10.3127/ajis.v24i0.2531 nayal, p., & pandey, n. (2020). digital coupon redemption: conceptualization, scale development and validation. australasian journal of information systems, 24. https://doi.org/10.3127/ajis.v24i0.2469 palmatier, r. w., houston, m. b., & hulland, j. (2018). review articles: purpose, process, and structure. journal of the academy of marketing science, 46, 1–5. https://doi.org/10.1007/s11747-017-0563-4 paul, j., & criado, a. r. (2020). the art of writing literature review: what do we know and what do we need to know? international business review, 29(4), 101717. https://doi.org/10.1016/j.ibusrev.2020.101717 pereira, r., lapão, l. v., bianchi, i. s., & amaral, d. (2020). improving emergency department through business process redesign. australasian journal of information systems, 24. https://doi.org/10.3127/ajis.v24i0.2679 poulsen, a., ulhaq, a., & miah, s. (2019). a post publication review of “emerging insights of health informatics research: a literature analysis for outlining new themes”. australasian journal of information systems, 23. https://doi.org/10.3127/ajis.v23i0.2269 robinson, b. (2020). towards an ontology and ethics of virtual influencers. australasian journal of information systems, 24. https://doi.org/10.3127/ajis.v24i0.2807 samhan, b. (2018). revisiting technology resistance: current insights and future directions. australasian journal of information systems, 22. https://doi.org/10.3127/ajis.v22i0.1655 soral, p., arayankalam, j., & pandey, j. (2020). the impact of ambivalent perception of bureaucratic structure on cyberloafing. australasian journal of information systems, 24. https://doi.org/10.3127/ajis.v24i0.2087 warren, m. (2020). fake news case study during the australian 2019 general election. australasian journal of information systems, 24. https://doi.org/10.3127/ajis.v24i0.2803 australasian journal of information systems lim, rasul & clarke 2020, vol 24, post publication review the challenges involved in establishing a research technique 6 copyright: © 2020 lim, rasul & clarke. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v24i0.3103 http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems sangroya, yadav & joshi 2021, vol 25, research on consumer evaluation gamified interaction of mobile applications 1 does gamified interaction build a strong consumer-brand connection? a study of mobile applications deepak sangroya jindal global business school, op jindal global university, sonipat, india rambalak yadav institute of management technology, hyderabad, india rbyadav1988@gmail.com yatish joshi school of management studies, motilal nehru national institute of technology allahabad, india abstract in recent times gamification has increasingly been used by brands through smartphones to interact effectively with their consumers. the core assumption for creating gamified environment is that it will develop engagement with the consumer and motivate them to use their product or services. however, beyond this assumption, there is dearth empirical evidence regarding how much effective these gamified features are in engaging consumers. therefore, this research analyses the relationship between flow, brand engagement, self-brand connection and brand usage intent among consumers (n = 360) of two gamified mobile applications. the findings show that the multidimensional construct flow formed by five dimensions i.e. challenge, feedback, autonomy, immersion, and interaction positively associates with cognitive brand engagement and emotional brand engagement. additionally, both these forms of brand engagement further strengthen consumers’ brand connection and motivate them for further use. these results imply that gamified environment can augment consumer engagement with brand and further increase usage intention. therefore, gamification can be an effective technique in brand management and brand managers can use it to strengthen relationship with consumers and increasing possibility of using their brands. keywords: gamification; smartphone applications; flow; consumer brand engagement; brand usage intent; self-brand connection 1 introduction brands share a common belief that building compelling and interactive consumer experiences are critical for developing successful marketing strategies (e.g. brakus, schmitt & zarantonello, 2009) and attaining long term competitive advantage (prahalad & ramswamy, 2004). due to increasing accessibility of technology – especially mobile technology, multiple platforms such as mobile apps, online community are available for the consumers to interact and exchange resources with brands and other consumers (lamberton & stephen 2016). furthermore, consumers are increasingly using these technologies and their interactions with brand has increased manifold (kumar & pansari, 2016). however, just providing access to these technologies does not guarantee their proper usage or interactive consumer brand exchange (storbacka et al., 2016; venkatesan, 2017). the development of competitive advantage in this technology enabled market environment resides in brands’ ability to promote consumers’ voluntary and adequate investment of resources in consumer brand relationship (kumar & pansari, 2016). this characterizes the challenge of generating consumer brand engagement (kumar & pansari, 2016; venkatesan 2017). australasian journal of information systems sangroya, yadav & joshi 2021, vol 25, research on consumer evaluation gamified interaction of mobile applications 2 consumer brand engagement as psychological state develop interactions between consumer and brand. it highlights the persistent, creative, and reciprocal nature of customer and brand relationships that reach beyond transactions. an engaged consumer is expected to collaborate, advocate and share information with their preferred brands (kumar et al., 2010). however, designing effective physical or digital “engagement platforms” facilitating engagement between various actors is much more important task (breidbach, brodie & hollebeek, 2014). engagement platforms act as intermediary that actors (e.g. consumers and brand) leverage to engage with others (edvardsson et al., 2010). by using various engagement platforms brands can provide consumers compelling experience and develop effective customer engagement (jaakkola & alexander, 2014; jaakkola, helkkula & aarikka-stenroos, 2015). however, reflecting on the current realities of the increased fragmentation of media and other communication channels brands are still looking for efficient mechanism that can manage customer engagement effectively (lemon and verhoef, 2016). encouraged by the success and popularity of video games, brands have started using gamification to create compelling consumer experiences and boost engagement (berger, schlager, sprott & herrmann, 2018). gamification defined by robson et al. (2015), as “the application of game design principles in order to change behaviour in non-game context” (p. 411). in this process, technology via computers or mobile phones being used to affect people’s motivations and change behaviour through game like systems (hamari & koivisto, 2015). recently, various global companies such as nike, amazon, expedia, tencent and starbucks have incorporated gamified components into their promotion schemes (zhang, phang, wu & lou, 2017). consequently, the gamification market is projected to rise to more than $11.10 billion by 2020 from $1.65 billion in 2015 (markets & markets, 2016). largely, gamification process was applied to provide solutions to the problems that require behavioural changes, such as reducing obesity, maintaining mental health or mastering new skills. however, brands can also use gamification to increase their consumers’ engagement (shankar, 2016). research on gamification have noticed its positive impacts on product adoption (müller-stewens et al., 2017), enhancing consumer creativity (scheiner, 2015) and changing consumer behaviours (hamari & koivisto, 2015). some studies have also highlighted the positive influence of gamification on user experience and engagement (harwood & garry, 2015; darejeh & salim, 2016). however, in spite of these positive outcomes, very little information available about its marketing potential and outcome. still there remains conceptual inefficiency and dearth of empirical research on its functioning and consequences from the perspective of consumer brand relationship. specifically, there are no conclusive evidence regarding brand responses such as the self-brand relation of consumers and the desire to use the brand. the major reason for this gap in the literature is probably because academia till date have mainly studied gamification from designer perspective. research will offer rich source of knowledge if this concept is studied from the viewpoint of consumer experiences. by treating games as fun interactive interactions between consumers and brand (holbrook, chestnut, oliva & greenleaf, 1984), we resolve this void, thus enabling us to concentrate on consumer psychological processes during gamification. therefore, the objective of this analysis is to examine the integration of gamification as a branding element into consumer brand relationship. to achieve this objective, we intend to find out the answer to research questions: can gamification facilitate consumers’ brand usage intent and self-brand connection? if so, what is the process of connecting brands to the customers? for understanding above mentioned research problem, this study advanced the problem by using flow (here defined “as a psychological state of being fully immersed in an activity”) and australasian journal of information systems sangroya, yadav & joshi 2021, vol 25, research on consumer evaluation gamified interaction of mobile applications 3 consumer brand engagement (here defined as “consumers’ cognitive, emotional and behavioural activity around specific consumer/brand interaction”). we consider these two concepts are appropriate to examine the effect of gamification as both help in explaining consumers’ immersive engagement with the brand and address the outcome of that engagement. after reviewing previous literature, the current study suggests a conceptual framework whichdiscusses the theoretical underpinning of flow and consumer brand engagement. for empirical examination of consumers’ flow in gamification, this study followed a quantitative approach. in current study, a survey questionnaire was developed and used for collecting data from consumers’ two gamified mobile applications. this research leads to two ways of interpreting gamification in branding literature. firstly, this study provides first of its kind empirical examination of the mechanism of gamified consumer experience facilitating brand usage intent. the current research thus contributes to the literature that investigates influence of flow of games on brand specific outcomes (lee et al., 2014). secondly, we also study the critical psychological processes of gamified interaction on brand preference through consumer brand engagement (kuo & rice, 2015). overall, the results of current research exhibit how gamification can be used as tool by organisations for creating compelling experience that lead to sustainable consumer brand relationship. in rest of the paper, we first review prior studies on gamification and flow theory. then we develop theoretical model linking gamified flow to brand usage intent through consumer brand engagement. section 3 presents the questionnaire used for examining the hypotheses, explains sample data and research methods used for examination. finally, the paper provides suggestions for marketing theory and practices. 2 literature review and conceptual development 2.1 gamification the phenomenal success of video games has led industry and academia to study what makes games so engaging. marketers are trying to use this learning to transform routine consumer behaviour into pleasant experiences (robson et al., 2015). this method of transferring mechanics of gaming in non-game context, referred as gamification, becomes an extremely popular practice in business (werbach & hunter, 2012). in a very limited time, this concept has been studied in different management contexts, like innovation (roth, schneckenberg & tsai 2015), organizational governance (farzan & brusilovsky 2011), services (hamari & koivisto 2015), banking (rodrigues, oliveira, & costa, 2016) and e-commerce (insley & nunan, 2014). in literature, gamification has been largely studied around two perspectives i.e. designer and consumer. the designer perspective, introduced by deterding et al. (2011), defined gamification “as the introduction of game mechanics and elements (rather than full-fledged games) to design non-game contexts” (p. 5). whereas, more recently, huotari and hamari (2017) has taken the customer-oriented approach and defined gamification as “a process of enhancing a service with affordances for gameful experience to support customers' overall value creation” (p. 25). it focusses largely on gameful consumer experiences which are created through gamified interaction. huotari and hamari (2017), further explained gameful experience as “the intrinsically motivating experience customers’ live when they are interacting with the gamified settings”. previous studies suggest use of flow theory to understand these immersive consumer experiences generated during gamified interaction (berger et al., 2018). further, huotari and hamari (2017) definition also stressed on value creation through gamification process. customer engagement can be potential positive outcome that is generated through the consumers’ immersive interaction with brands (hammedi, leclercq & van riel, 2017). flow australasian journal of information systems sangroya, yadav & joshi 2021, vol 25, research on consumer evaluation gamified interaction of mobile applications 4 theory can also be used to study effect of consumers’ gamified experience on consumer brand relationship (berger et al., 2017). thus, current study uses flow theory framework to understand the underlying gamification process and its potential outcomes. 2.2 flow theory flow or immersive flow commonly used to describe a psychological state in which a person is completely immersed and involved in an activity (csikszentmihalyi & lefevre, 1989). as defined by csikszentmihalyi (1990), “it as a state of optimal experience in which individual feel simultaneously happy and cognitively efficient, as they are so engrossed in an activity as nothing else matter” (p. 816). flow is so called because individuals describe their perceptions of "flow" as a current of water moving them along (csikszentmihalyi, 1975). this psychological mechanism has also been examined in the sports, education and music fields (agarwal & karahanna, 2000; csikszentmihalyi & lefevre, 1989; hsu & lu, 2004). after hoffman and novak (1996) presented the conceptual structure of flow in the online world, a significant number of human-computer interaction investigations have explored the antecedents and implications of flow. although most of these studies used a one-dimensional flow, latest research has shown the value of multi-dimensional flow (e.g., hamari & koivisto, 2015; procci, singer, levy, & bowers, 2012). as both online and face-to face social interactions are needed in gamification, a multi-dimensional flow model is best suited to capture the flow experience (kaur et al., 2016). moreover, according to previous studies, when people completely immersed in an activity, they are psychologically driven to participate actively (ho and kuo, 2010). since flow is essentially satisfying, people tend to repeat these flow experiences, creating a procedure for psychological functioning that promotes engagement (nakamura & csikszentmihalyi, 2002). according to hamari et al. (2016), by fostering consumers’ intrinsic motivation, flow sets the stage for sustained commitment and develop enduring brand engagement. previous studies analysing impact of flow on product usage showed that focused immersion, temporal dissociation, control, curiosity and heightened enjoyment make people completely absorbed and further motivate to use the product continuously (agarwal & karahanna, 2000). in the context of gamification, lowry et al. (2013), found, gamers’ immersive experiences create enjoyment and control, which subsequently motivates them to stay in the game for longer duration. thus, in this research we study gamified consumer experience since it is likely to positively affect consumer brand engagement which subsequently develops strong connection between consumer and brand and positively influences its future usage. 2.3 conceptual model after exploring literature related to consumer engagement and flow theories, this study develops conceptual framework (figure 1) of drivers of flow, consumer engagement, self-brand connection and brand usage intent. this hypothesized model assesses cross sectional associations among flow, cognitive brand engagement, emotional brand engagement, selfbrand connection and brand usage intent. in the following subsections, detailed discussion on individual dimensions of flow construct and theoretical support for its relationships with other constructs are provided. 2.3.1 flow and consumer brand engagement in the current research, flow is considered as multidimensional construct having five dimensions which includes “immersion, challenge, feedback, autonomy and social interaction” (yoo, sanders & cerveny, 2018). the immersion experience widely studied as the dimension of flow; previous research has also incorporated challenges, autonomy and receiving consistent australasian journal of information systems sangroya, yadav & joshi 2021, vol 25, research on consumer evaluation gamified interaction of mobile applications 5 and exciting feedbacks as dimensions of flow (lowry et al., 2008). previous studies also empirically verified social interactions as important aspect of flow, in promoting enjoyment and intrinsic motivation thus promoting flow (fu, su, & yu, 2009). on the basis above discussion, this paper examines the effect of above mentioned five dimensions i.e. immersion, feedback, autonomy, challenge and social interaction on brand engagement. figure 1. conceptual model in the literature, challenge has been defined as “the level of difficulty an individual feel in relation to their skill level” (yoo et al., 2018, p. 109). according to previous studies, this difficulty is a two-edged sword, if the challenge is greater than individual’s current knowledge, it may lead to anxiety. whereas, if it does not match knowledge, apathy may occur (johnson & wiles, 2003). previous research has noticed positive impact of increasing the challenging nature on enjoyment in the gaming environment (abuhamdeh & csikszentmihalyi, 2012). some studies have also highlighted the importance of challenge in prompting cognitive reactions (mcmillan & hwang, 2002). therefore, this study hypothesises the positive role of challenge dimension in developing cognitive and affective brand engagement. in the present research, feedback discusses the degree people feel that brand notify them about their progress (yoo et al., 2018). in a gamified environment, such notifications are usually delivered via message and award (yoo et al., 2018). feedback is among most popular dimensions since positive feedback inspires mastery of tasks (federoff, 2002). therefore, this study hypothesized that good feedback facilitates development of engagement with the brand. autonomy understood as level of control perceived by user over game (yoo et al., 2018). in case, consumers feel difficulty in understanding the rules and procedures of game or controlling australasian journal of information systems sangroya, yadav & joshi 2021, vol 25, research on consumer evaluation gamified interaction of mobile applications 6 the game, they won't completely engage in the experience it offers. furby (1978), argued that greater the control over object is exercised by individuals, the more psychologically it is perceived as a part of self. thus, current study expects autonomy in the game may increase the probability that a consumer can promote both flow and lead to engagement. immersion is a mental state felt by individual when they get completely involved in a particular activity (yoo et al., 2018). this dimension has been noted as one of most representative experience of flow (csikszentmihalyi & csikszentmihalyi, 1992). high immersion is also the product of great pleasure and leads to high involvement in the subject (agarwal & karahanna, 2000). this study thus posits that if users experience more satisfaction and spend more effort and time when experiencing gamification flow, they may have more cognitive and emotional brand engagement. social interaction means the extent at which users communicate socially over the course of gamification (yoo et al., 2018). moreover, interaction with other consumers is an vital element of consumer brand engagement. thus, creating a communication medium which allow consumers to interact socially is a significant foundation for building engagement with brand. previous studies present that members’ communication, and technology develop affective emotions that lead to engagement. in case of gamification, now a days, social interaction is facilitated among users via bulletin boards, instant messaging etc. gamification presents users this ability to build a sense of belonging with the brand and other users, thus leading to consumer brand engagement. thus, we posit: h1: flow has significant positive impact on cognitive brand engagement (cbe). h2: flow has significant positive impact on emotional brand engagement (ebe). 2.3.2 consumer brand engagement and self-brand connection previous studies indicates that customer brand engagement which arises from their interactions with a brand (hollebeek, 2011) could conceptually relate consumers’ flow experiences to their relationship with brand (brodie, hollebeek, jurić & ilić, 2011). flow can activate cognitive and affective reactions to an experiential action (moneta & csikszentmihalyi, 1996). likewise, earlier examination has demonstrated that flow leads to engagement (shernoff, csikszentmihalyi, schneider & shernoff, 2014). further emotions and cognitions emerged as significant aspects for the engagement in online settings (mollen & wilson, 2010) and expected to foster self–brand connections. in similar lines hollebeek et al. (2014) examined the positive influence of emotional and cognitive interaction of consumers with a social networking facility and self-brand connections. park et al. (2010) has also observed that cognitive brand evaluations leads to greater connections with a brand. thus, on the basis of the discussion above we posit: h3: cognitive brand engagement (cbe) has significant positive influence on self-brand connections (sbc). h4: emotional brand engagement (ebe) has significant positive influence on self-brand connections (sbc). 2.3.3 consumer brand engagement and brand usage intent prior studies has indicated that flow influences customers' affective, cognitive, and conative actions, such as attitudes (hsu & lu, 2004), intentions (agarwal & karahanna, 2000), and behaviors (skadberg & kimmel, 2004). existing research in marketing noted that flow experiences accrued during gamification influence users’ engagement which subsequently increases frequency of using the system (hamari & koivisto, 2015; osatuyi & qin, 2018). australasian journal of information systems sangroya, yadav & joshi 2021, vol 25, research on consumer evaluation gamified interaction of mobile applications 7 feedback and social interaction, two key dimensions of flow proposed to affecting brand usage intention positively (huang et al., 2018). moreover, previous marketing studies have studied the role of brand engagement on positive brand usage intent (bui). according to these studies, engaged customers are more likely to build attitudes from beliefs (hollebeek, 2011). in fact, those attitudes are expected to be favourable which may result in higher brand usage intent (harmeling et al., 2017). social exchange theory also suggests consumers emotional and cognitive engagement, will result in higher interaction with the brand. thus, we propose that cognitive and emotional brand engagement develop through gamified interactions should lead to higher brand usage intent. h5: cognitive brand engagement (cbe) has significant positive influence on brand usage intent (bui). h6: emotional brand engagement (ebe) has significant positive influence on brand usage intent (bui). 2.3.4 moderating role of gender literature specialising in gender socialisation especially gender socialisation theory proposed by barry et al. (1975), suggests male and female in addition to their gender specific skills also have gender specific attributes which present them as feminine or masculine. hence, both these genders develop different value sets thus developing different value and ethical preferences among them (mason & mudrack, 1996). one of important research stream that emerged in the literature is “gender identity research” which studies gender as psychological sex on the basis of masculine and feminine personality traits (gould & weil, 1991; palan, 2001). however, marketing researchers questioned this perspective of gender identity for predicting their consumer behavior (kolyesnikova et al., 2009) and proposed “biological sex research stream as an important determinant of consumer behaviour” (kahle & homer, 1985). biological sex research stream view of gender in relations to biological sex as male and female, considered by marketing research as critical variable for predicting their behavior as consumers or segmenting them in different groups (das, 2014). so, current research considers gender as biological sex (female and male) for studying its moderating role between flow and cognitive and affective brand engagement. previous research has analysed the role of gender in marketing literature and also considered as crucial variable for engagement studies. however, very few studies are conducted in relation to mobile environment and there is no agreement on the impact of gender on mobile gamification (conaway & garay, 2014). existing research in context of digital environment acknowledge gender differences in the consumer decision making process as female and male differ with regard to adoption and internet usages (li & chang, 2016; zhang et al., 2009). these differences reflect the psychological process that influence people’s behavior (zurita-ortega et al., 2018). for example, research found women to be more intrinsically motivated, while men are having more extrinsic motivation (erturan-ilker et al., 2018). in context of digital environment, research found men and women to behave differently while interacting with digital platform. for example, women are found to be showing more exploratory behaviour, high involvement, and less purchases over digital platform (richard et al., 2010). whereas another research suggests these gender differences are declining as both considers key variables like system quality, information quality equally important. therefore, existing study on the moderating role of gender is highly consistent and this calls for further investigation of gender differences in the mobile gamified environment. to date, no study has explored the moderated role of gender inn the relationship of flow and consumer engagement, specifically in a gamified environment. thus, this study posits that: australasian journal of information systems sangroya, yadav & joshi 2021, vol 25, research on consumer evaluation gamified interaction of mobile applications 8 h7a: the impact of flow in a gamified environment on cbe will significantly vary across gender. h7b: the impact of flow in a gamified environment on ebe will significantly vary across gender. 2.3.5 mediating role of brand engagement in line with previous studies, current study postulate that brand engagement, stemming from consumer brand interactions (hollebeek, 2011) may link consumers’ flow experiences to consumers’ self–brand connections and brand usage intent (brodie et al. 2011). according to moneta and csikszentmihalyi (1996), flow can generate both cognitive and affective responses to an experiential activity. moreover, previous studies have found that flow develop engagement (shernoff et al. 2003) and that cognitive and emotional are key dimensions triggering the engagement process in online contexts (mollen & wilson, 2010). therefore, current study postulate that the impact of flow on brand usage intent and self–brand connections is mediated by cognitive and emotional brand engagement in the context of gamified environment. h8: the impact of flow on consumers’ bui is mediated by (a) cbe and (b) ebe. h9: the impact of flow on consumers’ sbc is mediated by (a) cbe and (b) ebe. 3 methodology 3.1 sample and data collection for this study, survey method has been used to collect data from smartphone users having experience of using gamified mobile apps. mobile gamified application like nike+, asics provide interactive platform, where users can join various competitions through their smartphone. these applications allow users to track, challenge, share, and interact with friends and other users, thereby creating socially exciting and collaborative experience. the interactive capability of gamified application helps in building deeper level of customer engagement. we administered the survey questionnaire to smartphone users residing in india. with 502.2 million smartphone users, india is world’s second largest market for smartphones (ndtv, 2020; shankar et al., 2020). in this study, structured questionnaire was employed to collect from sample respondents. the survey was conducted online with the respondents who were having smartphone and have had experience of using gamified mobile application. to ensure data were collected from regular user of these smartphone application, firstly a screening question was asked: “are you using this app for a minimum period of 30 days”. only the respondents who answered yes were asked to proceed with the rest of survey. in total of 584 responses were utilized in the study. the demographic profile of these respondents exhibited in table 1. in order to examine any nonresponse bias, independent sample t test was performed on the demographic variables between the first 25% (q1) and last 25% (q4) samples (shankar and jebarajakirthy, 2019). the results suggested absence of non-response bias as the demographic profile of these two group samples was not significantly different. 3.2 measures and instrument development current study used the prior established scales to measure the focal construct after slightly modifying to the context of this research. the measurement instrument had three sections, where first section included the screening question. the second and third section contains the variables related to the study constructs and demographic information, respectively. australasian journal of information systems sangroya, yadav & joshi 2021, vol 25, research on consumer evaluation gamified interaction of mobile applications 9 the multi-dimensional construct of cbe was measured by seven measurement statements proposed by hollebeek et al. (2014) capturing emotional brand engagement and cognitive brand engagement. seven items scale developed by escalas (2004) were used to operationalised selfbrand connection (sbc), while four items scale proposed by yoo and donthu (2001) was used to assess brand usage intent (bui). construct indicators fl cronbach’s α/cr ave ch ch1 0.80 0.80/0.85 0.58 ch2 0.72 ch3 0.77 ch4 0.76 fb fb1 0.78 0.82/0.86 0.61 fb2 0.82 fb3 0.77 fb4 0.75 au au 1 0.76 0.79/0.82 0.61 au 2 0.76 au 3 0.82 im im 1 0.77 0.89/0.90 0.60 im 2 0.79 im3 0.77 im4 0.78 im 5 0.76 im 6 0.78 si si 1 0.74 0.85/0.87 0.63 si 2 0.77 si 3 0.83 si 4 0.82 cbe cbe 1 0.75 0.74/0.76 0.51 cbe 2 0.65 cbe 3 0.74 ebe ebe 1 0.78 0.81/0.83 0.56 ebe 2 0.72 ebe 3 0.77 ebe 4 0.73 sbc sbc 1 0.80 0.91/0.92 0.63 sbc 2 0.77 sbc 3 0.80 sbc 4 0.84 sbc 5 0.73 sbc 6 0.76 sbc 7 0.84 bui bui 1 0.77 0.86/0.87 0.62 bui 2 0.80 bui 3 0.84 bui 4 0.73 table 2. summary of the measurement model note: fl: factor loadings; ave: average variance extracted, cr: composite reliability. australasian journal of information systems sangroya, yadav & joshi 2021, vol 25, research on consumer evaluation gamified interaction of mobile applications 10 all these scales used for measuring study constructs were operationalised on seven-point likert scales. as suggested by previous studies (kaur et al., 2016), this study used the multidimensional conceptualisation proposed by jackson and marsh (1996) to measure flow. this scale consists 21 measurement items capturing five dimensions i.e. challenge, feedback, autonomy, immersion and social interaction. a panel comprising managers and professors with the expertise in the digital branding field was formed for examining the content validity of survey questionnaire (jebarajakirthy et al., 2020). the survey instrument was also pretested by two groups of gamified smartphone application user. both these groups were having eight members. some minor changes were made to the format and language of the measurement instrument based on the feedback of these focus groups. 4 results the hypothetical model was examined in two phases as suggested in structural equation modelling (sem) approach (gerbing & hamilton,1996). firstly, confirmatory factor analysis (cfa) with maximum likelihood method was performed to examine the reliability and validity of the measurement indicators. thereafter, structural model conducted to examine the direction and strength of the hypothesised relationships among constructs. 4.1 measurement model as mentioned above, firstly, cfa was performed with maximum likelihood method to analyze the validity and reliability of the measurement indicators. each construct was having good internal consistency as composite reliability (cr) varies from 0.76 to 0.92 (see table 2), which was higher than the suggested value of 0.70 (hair et al., 2010; joshi et al., 2019). the convergent validity of latent constructs was analysed using item loadings and average variance extracted (ave) (islam et al., 2019). the findings of cfa exhibited in table 2 illustrates that all measurement items were having standardized factor loading more than 0.5. moreover, the ave values were also more than the minimum suggested value of 0.5 (mehta, sharma, & swami, 2013), thus verifying the convergent validity. proposed hypothesis/ path relationships (β) se direct effects flow → cbe .369*** .08 flow → ebe .423*** .11 cbe → sbc .520*** .09 cbe → bui .534*** .09 ebe → sbc .515*** .06 ebe → bui .507*** .07 table 3. discriminant validity notes: *** p < 0.001; fit indices cmin/df = 1.073 (p<0.001), cfi= .99, gfi= .91, agfi= .90, nfi=.91, tli= .99, srmr = 0.039, rmsea= .014. in this study, the constructs’ discriminant validity was examined on the following criteria: (1) measurement variables should load heavily only on their own construct and (2) the ave values of each construct should be more than the squared correlation between the constructs. the findings suggest all the ave values were above recommended value of 0.50 and all the measurement indicators are loading strongly only on their corresponding construct (see table 2) (bagozzi & yi, 1988). moreover, as presented in table 3, the squared correlation between the two constructs was less than the constructs’ ave, thus verifying the discriminant validity (fornell & larcker, 1981). australasian journal of information systems sangroya, yadav & joshi 2021, vol 25, research on consumer evaluation gamified interaction of mobile applications 11 the fitness of measurement model was studied after conducting the cfa and all the fit indices suggested a appropriate model fit (hair et al., 2010). the measurement model was fitting well with the data as χ2/df = 1.076 (χ2 = 738.9, df = 687, p<0.000), normed fit index (nfi) = 0.91, comparative fit index (cfi) = 0.99, root mean square error of approximation (rmsea) = 0.015 and standardized root mean square residual (srmr) = 0.038. cr ave msv sbc bui af cp flow sbc 0.921 0.627 0.446 0.792 bui 0.865 0.617 0.446 0.668*** 0.786 af 0.834 0.558 0.415 0.644*** 0.636*** 0.747 cp 0.756 0.509 0.431 0.648*** 0.656*** 0.245*** 0.714 flow 0.837 0.508 0.184 0.388*** 0.428*** 0.424*** 0.371*** 0.712 table 4. hypothesis testing notes: *** p < 0.001; ** p < 0.01; * p<0.05 4.2 structural model after examining the measurement model through cfa, path analysis was used to examine the relationship between constructs (jebarajakirthy and shankar, 2021). the fitness statistics suggested that the hypothesised model was adequate (χ2/df= 1.073, p < 0.001; cfi = 0.99, gfi= 0.90, nfi =0.91, agfi =0.897, srmr = 0.04 and rmsea = 0.014). to test the hypothesis, the proposed path coefficients and their respective p values were examined. the results presented in table 4, suggest flow positively affected cbe (γ11 = 0.423, t =5.82, p < 0.001) and ebe (γ21 = 0.369, t= 4.957, p < 0.001), thus supporting both h1 and h2. the results also suggest the significant positive effect of cbe on sbc (γ12 = 0.520, t= 8.60, p < 0.001) and bui (γ22 = 0.534, t= 8.337, p < 0.05). ebe also found to positively affecting sbc (β21= 0.515, t= 9.49, p < 0.01). as proposed, ebe found to have positive effect on bui (β42 = 0.507, t =8.89, p < 0.001). the results presented in table 6 suggest that hypotheses h1, h2, h3, h5 and h6 are supported at the significance level of p < 0.001, whereas h4 is supported at the significance level of p < 0.05. the findings in the structural model are also presented in figure 2. the moderating effect of gender hypothesized in h7 was examined using multi-group analysis in amos 24.0. for this analysis, the total sample was divided in two subgroups: female (n = 177) and male (n = 183). the independent model estimation for this multi group exhibited good fit: as χ2/df = 1.087, cfi = 0.98, nfi = 0.813; ifi = 0.988, gfi = 0.919, srmr = 0.057, rmsea = 0.013. hypothesis male female (β) t-values (β) t-values h7a flow → cbe .375 8.458 .369 7.251 h7b flow → ebe .582 12.365 .528 10.117 table 5. moderating effect of gender the results of relationship effects for both male and female are presented in table 5. the impact of flow on cognitive brand engagement for both the groups (male and female) is positive and significant. the difference between positive effect of flow on cbe was very marginal (male: β = 0.375, p < 0.01; female: β = 0.369, p < 0.01), but significant. similarly, the impact of flow on emotional brand engagement for male group was marginally higher than the female group (male: β = 0.582, p < 0.01; female: β = 0.528, p < 0.01). the significance of both these results australasian journal of information systems sangroya, yadav & joshi 2021, vol 25, research on consumer evaluation gamified interaction of mobile applications 12 suggest the support for h7a and h7b, thereby leading us to conclude that gender moderates the effect of flow on cognitive and emotional brand engagement. this study also examined two parallel mediation models using process macro (model 4) to analyse whether cognitive and emotional brand engagement mediates the impact of flow on consumers’ brand usage intent and self–brand connections (hayes 2013). the results of mediation analysis suggest that the total effect of flow on bui (β =1.55, ci [1.349, 1.773], p<0.05) and sbc (β =1.59, ci [1.396, 1.800], p<0.05) was significant. but the direct effect of flow on bui became insignificant (β =−0.1175, ci [−0.281, 0.046], p=.159) when cbe and ebe were considered. importantly, the indirect effect of cbe on the impact of flow on consumers' bui (β =0.78, ci [0.6377, 0.9481], p < .05) was significant. ebe also had a similar indirect effect (β =0.76, ci [0.6425, 0.9052], p < .05) (hayes, 2013). therefore, h8(a) and h8(b) were supported. these results further suggest that cbe and ebe partially mediate the relationship between flow and bui. figure 2. the final model for examination h9 (a) and h9 (b) similar process was followed. in this case, however the direct effect of flow on sbc got reduced but was significant (β =−0.3997, ci [−0.5410, 0.2583], p=.000) after including cbe and ebe as mediating variable. whereas, the indirect effect of both cbe (β =0.73, ci [0.5978, 0.8744], p < .05) and ebe (β =0.85, ci [0.7261, 0.9934], p < .05) on the relationship between flow and sbi was significant. thus, h9(a) and h9(b) were supported. 4.3 common method variance since the data for dependent and independent constructs provided by the same respondents, there is a possibility of common method bias (cmb) (ashaduzzaman et al., 2020). in literature, podsakoff et al. (2003) suggested harman's single test factor test to analyse the problem of cmb. according to their recommendation, all the focal constructs were loaded into a single factor and variance explained by this single factor was examined. the result of this test concluded no single factor explains disproportionately large portion of variance. therefore, no australasian journal of information systems sangroya, yadav & joshi 2021, vol 25, research on consumer evaluation gamified interaction of mobile applications 13 “general” factor is evident in the data. the problem of cmb was also examined through the analysis of correlation matrix and does not found very high correlation among constructs. additionally, a common latent factor all the measurement indicators included in the model. this common latent factor produced a value of 0.264, which was subsequently squared to calculate the common method variance. this analysis suggest that common method variance was not an issue as squared value was 0.069 (6.9%). as mentioned by ranaweera and jayawardhena (2014), when common method variance falls below 50%, it indicates the unlikelihood of cmb according to common latent factor approach. the results of all these tests suggested cmb is not a concern in this study (podsakoff et al., 2003). 5 discussion the objective of current research was to examine the role of flow in developing consumer engagement and further strengthening consumer brand relationship. the findings support our hypotheses that flow effect created by gamified mobile app result in positive consumer brand engagement, which helps in further usage of the brand and building strong relationship with consumer. specifically, the results showed that challenge, feedback, immersion, interaction and autonomy develop flow. this finding confirms the csikszentmihalyi's (1990) proposition that an individual’s involvement in an experiential activity with clarity of goal develops intrinsic flow experience. moreover, if the immersion and challenge increase, individuals engage in sharing goal-related feedback and their authentic experiences with other users. in addition, the regular feedback or information from the app also improves users’ flow experience. this develops a feeling of control and autonomy over the challenge at hand and keep them engaged (li & browne, 2006). the findings of this study also show that flow positively influence cognitive brand engagement and emotional brand engagement. flow experiences make the brands’ mobile app interactive to users and keep them engaged with the brand. previous studies in the branding literature posit that users’ high cognitive and emotional brand engagement develop enduring relationship with the brand. our findings confirm the positive effect of cbe and ebe on consumers’ self-brand connection and brand usage intent. 5.1 theoretical implications the findings of this study contribute to the branding and gamification research in four ways. firstly, current study adds to the existing body of literature which examine gamification from consumer experience perspective (mcalexander, schouten & koenig, 2002). this study confirms the experiential benefits conceptualised by bruhn et al. (2014) during gamification. in consistent with previous studies, the positive role of transcendent consumer experience like flow in developing engagement has also been confirmed (schouten, mcalexander & koenig, 2007). secondly, this research strengthens the position of flow in the branding literature. although few studies have examined the flow concept in the branding literature in different context such as learning, sports, leisure activities, media and human–computer interaction (engeser & schiepe-tiska, 2012). however, still very little knowledge available about its role in strengthening consumer brand relationship. moreover, previous studies did not examined flow from multi-dimensional perspective. current study analyse flow as multidimensional second order construct comprising challenge, interaction, immersion, feedback and autonomy dimensions. thirdly, by focusing on the role of gamification environment in developing flow, current research focuses on the contextual factors as per the recommendation of csikszentmihalyi (2014). in this manner, the findings contribute to the research stream studying contextual factors and proposes flow principles. lastly, current research establishes the vital role of flow in strengthening consumer brand relationship. the results of current study verify role of australasian journal of information systems sangroya, yadav & joshi 2021, vol 25, research on consumer evaluation gamified interaction of mobile applications 14 gamification environment in developing strong engagement with consumer from perspective of consumer experiences. this brand engagement results in building strong consumer brand relationship. flow, the peak experience, developing when mobile user participates in the game created by brands’ app, performs a key role in generating positive results for the brand. past studies showed that consumers strongly identify with a brand when they find the brand attractive (bhattacharya & sen, 2003). in the similar manner, this study showed that in case gamified brand app produce flow experience, user engages with the brand, and subsequently develop stronger connection with the brand. these results also provide further support the previous studies by showing that consumer brand engagement lead to brand usage intent. 5.2 managerial implications current research also offers few practical suggestions for branding practitioners focussing on developing strong consumer brand relationship. firstly, the result of current study provides brands valuable insights looking to design gamified consumer interactions. this study suggests the critical role of flow elements i.e. interaction and optimally challenging while designing gamified environment. the game design on the one side allow consumers to engage passively with brand but also build active engagement through the feature of feedback and social interaction. secondly, this study shows that brands should design games in the manner that challenge consumer’s skill to optimal level. since person’s skill level largely affects perception of challenge, games should be created with several levels and plans and should be able to dynamically match person’s skills with challenge. in case of service brands, users may join game at tough levels as this create optimal challenge for them. moreover, brands should be cognizant of the fact that a well-designed gamified interaction may also fail to engage and connect with consumer if they does not feel freedom to participate in the gameplay. thus, brands should provide user autonomy while using the game. the brands should provide user the option to leave and resume the game without losing the progress. in today’s world of big data and customer analytics, organisations can use consumers’ data in targeting them more strategically. this data can be used by marketers for better understanding their needs and makes it easier for marketer to satisfy them. brands should closely monitor the feedback shared by user and provide solutions immediately when users have product problems. these strategies will develop flow experience and ultimately lead to stronger engagement and connection with user. in addition, the findings highlight the importance of social interaction and sharing dimension for creating flow element. thus, branding apps can support “flow sharing opportunities” for users to share their progress and achievement with their friend. these sharing opportunities can be easily organized by developing a user friendly, well-organized social media platform. users’ flow experiences many times surge with others and this contagious experience encourage others to participate and helps in developing a strong brand. 6 limitations and future research while current study presents consistent support for the theoretical model, this research has also some limitations, which can be resolved by future studies. firstly, current study uses crosssectional design for data collection. it may be possible that some users change their decisions over time due to accumulated gamification experience, therefore in future, longitudinal studies can be conducted to explore the role of flow experience. furthermore, in such studies involve mobile branding apps, external factors like technical problems may affect consumer experience and probably lead to biased responses. in such a scenario, experimental design using controlled conditions will enable researchers to gather more reliable and valid responses from participants. australasian journal of information systems sangroya, yadav & joshi 2021, vol 25, research on consumer evaluation gamified interaction of mobile applications 15 our study used fitness app to examine effect of gamified element on customer engagement and their further relationship with brand. future studies may consider examining our model with other brands and in different context to broader the understanding of issues. references abuhamdeh, s., & csikszentmihalyi, m. (2012). the importance of challenge for the enjoyment of intrinsically motivated, goal-directed activities. personality and social psychology bulletin, 38(3), 317-330. agarwal, r., & karahanna, e. (2000). time flies when you're having fun: cognitive absorption and beliefs about information technology usage. mis quarterly, 665-694. ashaduzzaman, m., jebarajakirthy, c., das, m., & shankar, a. (2020). acculturation and apparel store loyalty among immigrants in western countries. journal of marketing management, 1-32. bagozzi, r. p., & yi, y. (1988). on the evaluation of structural equation models. journal of the academy of marketing science, 16(1), 74-94. barry, h., bacon, m. k., & child, i. l. (1957). a cross-cultural survey of some sex differences in socialization. the journal of abnormal and social psychology, 55(3), 327. batra, r., ahuvia, a., & bagozzi, r. p. (2012). brand love. journal of marketing, 76(2), 1-16. berger, a., schlager, t., sprott, d. e., & herrmann, a. (2018). gamified interactions: whether, when, and how games facilitate self–brand connections. journal of the academy of marketing science, 46(4), 652-673. bhattacharya, c. b., & sen, s. (2003). consumer–company identification: a framework for understanding consumers’ relationships with companies. journal of marketing, 67(2), 76-88. breidbach, c., brodie, r., & hollebeek, l. (2014). beyond virtuality: from engagement platforms to engagement ecosystems. managing service quality, 24(6), 592–611. brakus, j. j., schmitt, b. h., & zarantonello, l. (2009). brand experience: what is it? how is it measured? does it affect loyalty? journal of marketing, 73(3), 52-68. brodie, r. j., hollebeek, l. d., jurić, b., & ilić, a. (2011). customer engagement: conceptual domain, fundamental propositions, and implications for research. journal of service r\research, 14(3), 252-271. bruhn, m., schnebelen, s., & schäfer, d. (2014). antecedents and consequences of the quality of e-customer-to-customer interactions in b2b brand communities. industrial marketing management, 43(1), 164-176. choi, d. h., kim, j., & kim, s. h. (2007). erp training with a web-based electronic learning system: the flow theory perspective. international journal of human-computer studies, 65(3), 223-243. conaway, r., & garay, m. c. (2014). gamification and service marketing. springerplus, 3(1), 653. csikszentmihalyi, m., & csikzentmihaly, m. (1990). flow: the psychology of optimal experience (vol. 1990). new york: harper & row. csikszentmihalyi, m., & csikszentmihalyi, i. s. (eds.). (1992). optimal experience: psychological studies of flow iin consciousness. cambridge university press. australasian journal of information systems sangroya, yadav & joshi 2021, vol 25, research on consumer evaluation gamified interaction of mobile applications 16 csikszentmihalyi, m., & lefevre, j. (1989). optimal experience in work and leisure. journal of personality and social psychology, 56(5), 815. csikszentmihalyi, m. (1975). play and intrinsic rewards. journal of humanistic psychology, 15(3), 41-63. darejeh, a., & salim, s. s. (2016). gamification solutions to enhance software user engagement—a systematic review. international journal of human-computer interaction, 32(8), 613-642. das, g. (2014). impacts of retail brand personality and self-congruity on store loyalty: the moderating role of gender. journal of retailing and consumer services, 21(2), 130-138. deterding, s., dixon, d., khaled, r., & nacke, l. (2011, september). from game design elements to gamefulness: defining" gamification". in proceedings of the 15th international academic mindtrek conference: envisioning future media environments (pp. 9-15). drell, l. (2014). an app a day keeps the doctor away. marketing health services, 34(2), 20-23. edvardsson, b., tronvoll, b., & gruber, t. (2011). expanding understanding of service exchange and value co-creation: a social construction approach. journal of the academy of marketing science, 39(2), 327-339. engeser, s., & rheinberg, f. (2008). flow, performance and moderators of challenge-skill balance. motivation and emotion, 32(3), 158-172. erturan-i̇lker, g., yu, c., alemdaroglu, u., & köklü, y. (2018). basic psychological needs and self-determined motivation in physical education to predict health-related fitness level. escalas, j. e. (2004). narrative processing: building consumer connections to brands. journal of consumer psychology, 14(1-2), 168-180. farzan, r., & brusilovsky, p. (2011). encouraging user participation in a course recommender system: an impact on user behavior. computers in human behavior, 27(1), 276-284. federoff, m. a. (2002). heuristics and usability guidelines for the creation and evaluation of fun in video games (doctoral dissertation, indiana university). fornell, c., & larcker, d. f. (1981). evaluating structural equation models with unobservable variables and measurement error. journal of marketing research, 18(1), 39-50. fu, f. l., su, r. c., & yu, s. c. (2009). egameflow: a scale to measure learners’ enjoyment of e-learning games. computers & education, 52(1), 101-112. furby, l. (1978). possession in humans: an exploratory study of its meaning and motivation. social behavior and personality: an international journal, 6(1), 49-65. gerbing, d. w., & hamilton, j. g. (1996). viability of exploratory factor analysis as a precursor to confirmatory factor analysis. structural equation modeling: a multidisciplinary journal, 3(1), 62-72. gould, s. j., & weil, c. e. (1991). gift-giving roles and gender self-concepts. sex roles, 24(910), 617-637. hamari, j., & koivisto, j. (2015). why do people use gamification services?. international journal of information management, 35(4), 419-431. australasian journal of information systems sangroya, yadav & joshi 2021, vol 25, research on consumer evaluation gamified interaction of mobile applications 17 hamari, j., shernoff, d. j., rowe, e., coller, b., asbell-clarke, j., & edwards, t. (2016). challenging games help students learn: an empirical study on engagement, flow and immersion in game-based learning. computers in human behavior, 54, 170-179. hammedi, w., leclerq, t., & van riel, a. c. (2017). the use of gamification mechanics to increase employee and user engagement in participative healthcare services. journal of service management. harmeling, c. m., moffett, j. w., arnold, m. j., & carlson, b. d. (2017). toward a theory of customer engagement marketing. journal of the academy of marketing science, 45(3), 312-335. harwood, t., & garry, t. (2015). an investigation into gamification as a customer engagement experience environment. journal of services marketing. 29(6/7), 533-546. ho, l. a., & kuo, t. h. (2010). how can one amplify the effect of e-learning? an examination of high-tech employees’ computer attitude and flow experience. computers in human behavior, 26(1), 23-31. hoffman, d. l., & novak, t. p. (1996). marketing in hypermedia computer-mediated environments: conceptual foundations. journal of marketing, 60(3), 50-68. holbrook, m. b., chestnut, r. w., oliva, t. a., & greenleaf, e. a. (1984). play as a consumption experience: the roles of emotions, performance, and personality in the enjoyment of games. journal of consumer research, 11(2), 728-739. hollebeek, l. (2011). exploring customer brand engagement: definition and themes. journal of strategic marketing, 19(7), 555-573. hollebeek, l. d., glynn, m. s., & brodie, r. j. (2014). consumer brand engagement in social media: conceptualization, scale development and validation. journal of interactive marketing, 28(2), 149-165. hsu, c. l., & lu, h. p. (2004). why do people play on-line games? an extended tam with social influences and flow experience. information & management, 41(7), 853-868. huang, h. c., cheng, t. c. e., huang, w. f., & teng, c. i. (2018). impact of online gamers’ personality traits on interdependence, network convergence, and continuance intention: perspective of social exchange theory. international journal of information management, 38(1), 232-242. huotari, k., & hamari, j. (2017). a definition for gamification: anchoring gamification in the service marketing literature. electronic markets, 27(1), 21-31. insley, v., & nunan, d. (2014). gamification and the online retail experience. international journal of retail & distribution management. islam, h., jebarajakirthy, c., & shankar, a. (2019). an experimental based investigation into the effects of website interactivity on customer behavior in on-line purchase context. journal of strategic marketing, 1-24. jaakkola, e., helkkula, a., & aarikka-stenroos, l. (2015). service experience co-creation: conceptualization, implications, and future research directions. journal of service management. jaakkola, e., & alexander, m. (2014). the role of customer engagement behavior in value cocreation: a service system perspective. journal of service research, 17(3), 247-261. australasian journal of information systems sangroya, yadav & joshi 2021, vol 25, research on consumer evaluation gamified interaction of mobile applications 18 jackson, s. a., & marsh, h. w. (1996). development and validation of a scale to measure optimal experience: the flow state scale. journal of sport and exercise psychology, 18(1), 17-35. jebarajakirthy, c., yadav, r., & shankar, a. (2020). insights for luxury retailers to reach customers globally. marketing intelligence & planning 38(7), 797-811. jebarajakirthy, c., & shankar, a. (2021). impact of online convenience on mobile banking adoption intention: a moderated mediation approach. journal of retailing and consumer services, 58, 102323. johnson, d., & wiles, j. (2003). effective affective user interface design in games. ergonomics, 46(13-14), 1332-1345. joshi, y., sangroya, d., srivastava, a. p., & yadav, m. (2019). modelling the predictors of young consumers' sustainable consumption intention. international journal of nonprofit and voluntary sector marketing, 24(4), e1663. kahle, l. r., & homer, p. (1985). androgyny and midday mastication: do real men eat quiche?. acr north american advances. kaur, p., dhir, a., chen, s., & rajala, r. (2016). flow in context: development and validation of the flow experience instrument for social networking. computers in human behavior, 59, 358-367. kolyesnikova, n., dodd, t. h., & wilcox, j. b. (2009). gender as a moderator of reciprocal consumer behavior. journal of consumer marketing. koufaris, m. (2002). applying the technology acceptance model and flow theory to online consumer behavior. information systems research, 13(2), 205-223. kumar, v., & pansari, a. (2016). competitive advantage through engagement. journal of marketing research, 53(4), 497-514. kumar, v., aksoy, l., donkers, b., venkatesan, r., wiesel, t., & tillmanns, s. (2010). undervalued or overvalued customers: capturing total customer engagement value. journal of service research, 13(3), 297-310. kuo, a., & rice, d. h. (2015). catch and shoot: the influence of advergame mechanics on preference formation. psychology & marketing, 32(2), 162-172. lamberton, c., & stephen, a. t. (2016). a thematic exploration of digital, social media, and mobile marketing: research evolution from 2000 to 2015 and an agenda for future inquiry. journal of marketing, 80(6), 146-172. lee, j., park, h., & wise, k. (2014). brand interactivity and its effects on the outcomes of advergame play. new media & society, 16(8), 1268-1286. lemon, k. n., & verhoef, p. c. (2016). understanding customer experience throughout the customer journey. journal of mmarketing, 80(6), 69-96. li, d., & browne, g. j. (2006). the role of need for cognition and mood in online flow experience. journal of computer information systems, 46(3), 11-17. li, c. h., & chang, c. m. (2016). the influence of trust and perceived playfulness on the relationship commitment of hospitality online social network-moderating effects of gender. international journal of contemporary hospitality management. australasian journal of information systems sangroya, yadav & joshi 2021, vol 25, research on consumer evaluation gamified interaction of mobile applications 19 lowry, p. b., vance, a., moody, g., beckman, b., & read, a. (2008). explaining and predicting the impact of branding alliances and web site quality on initial consumer trust of e-commerce web sites. journal of management information systems, 24(4), 199-224. lowry, p. b., gaskin, j., twyman, n., hammer, b., & roberts, t. l. (2013). proposing the hedonic-motivation system adoption model (hmsam) to increase understanding of adoption of hedonically motivated systems. journal of the association for information systems, 14(11), 617-671. luna, d., peracchio, l. a., & de juan, m. d. (2002). cross-cultural and cognitive aspects of web site navigation. journal of the academy of marketing science, 30(4), 397-410. luna, d., peracchio, l. a., & juan, m. d. d. (2003). flow in individual web sites: model estimation and cross-cultural validation. acr north american advances. mason, e. s., & mudrack, p. e. (1996). gender and ethical orientation: a test of gender and occupational socialization theories. journal of business ethics, 15(6), 599-604. mathwick, c., & rigdon, e. (2004). play, flow, and the online search experience. journal of consumer research, 31(2), 324-332. marketsandmarkets (2016). gamification market by solution. https://www.marketsandmarkets.com/market-reports/gamification-market-991.html, accessed date: 10 may 2018. mcalexander, j. h., schouten, j. w., & koenig, h. f. (2002). building brand community. journal of marketing, 66(1), 38-54. ndtv (2020). over 50 crore indians now use smartphones, 77 per cent on internet: https://www.ndtv.com/india-news/over-50-crore-indians-now-use-smartphones-77-percent-on-internet-2171937, accessed date: 15 march 2020. mcmillan, s. j., hwang, j. s., & lee, g. (2003). effects of structural and perceptual factors on attitudes toward the website. journal of advertising research, 43(4), 400-409. mehta, r., sharma, n. k., & swami, s. (2013). the impact of perceived crowding on consumers' store patronage intentions: role of optimal stimulation level and shopping motivation. journal of marketing management, 29(7-8), 812-835. moneta, g. b., & csikszentmihalyi, m. (1996). the effect of perceived challenges and skills on the quality of subjective experience. journal of personality, 64(2), 275-310. mollen, a., & wilson, h. (2010). engagement, telepresence and interactivity in online consumer experience: reconciling scholastic and managerial perspectives. journal of business research, 63(9-10), 919-925. müller-stewens, j., schlager, t., häubl, g., & herrmann, a. (2017). gamified information presentation and consumer adoption of product innovations. journal of marketing, 81(2), 8-24. nakamura, j., & csíkszentmihályi, m. (2002). the concept of flow. in c. r. snyder & s. j. lopez (eds.), handbook of positive psychology, 89–105. new york: oxford university press. oinas-kukkonen, h. (2013). a foundation for the study of behavior change support systems. personal and ubiquitous computing, 17(6), 1223-1235. https://www.ndtv.com/india-news/over-50-crore-indians-now-use-smartphones-77-per-cent-on-internet-2171937 https://www.ndtv.com/india-news/over-50-crore-indians-now-use-smartphones-77-per-cent-on-internet-2171937 australasian journal of information systems sangroya, yadav & joshi 2021, vol 25, research on consumer evaluation gamified interaction of mobile applications 20 osatuyi, b., & qin, h. (2018). how vital is the role of affect on post-adoption behaviors? an examination of social commerce users. international journal of information management, 40, 175-185. park, c. w., macinnis, d. j., priester, j., eisingerich, a. b., & iacobucci, d. (2010). brand attachment and brand attitude strength: conceptual and empirical differentiation of two critical brand equity drivers. journal of marketing, 74(6), 1-17. palan, k. m. (2001). gender identity in consumer behavior research: a literature review and research agenda. academy of marketing science review, 10(2001), 1-31. podsakoff, p. m., mackenzie, s. b., lee, j. y., & podsakoff, n. p. (2003). common method biases in behavioral research: a critical review of the literature and recommended remedies. journal of applied psychology, 88(5), 879. prahalad, c. k., & ramaswamy, v. (2004). co‐creation experiences: the next practice in value creation. journal of interactive marketing, 18(3), 5-14. procci, k., singer, a. r., levy, k. r., & bowers, c. (2012). measuring the flow experience of gamers: an evaluation of the dfs-2. computers in human behavior, 28(6), 2306-2312. ranaweera, c., & jayawardhena, c. (2014). talk up or criticize? customer responses to wom about competitors during social interactions. journal of business research, 67(12), 26452656. richard, m. o., & chandra, r. (2005). a model of consumer web navigational behavior: conceptual development and application. journal of business research, 58(8), 10191029. richard, m. o., chebat, j. c., yang, z., & putrevu, s. (2010). a proposed model of online consumer behavior: assessing the role of gender. journal of business research, 63(910), 926-934. robson, k., plangger, k., kietzmann, j. h., mccarthy, i., & pitt, l. (2015). is it all a game? understanding the principles of gamification. business horizons, 58(4), 411-420. rodrigues, l. f., oliveira, a., & costa, c. j. (2016). does ease-of-use contributes to the perception of enjoyment? a case of gamification in e-banking. computers in human behavior, 61, 114-126. roth, s., schneckenberg, d., & tsai, c. w. (2015). the ludic drive as innovation driver: introduction to the gamification of innovation. creativity and innovation management, 24(2), 300-306. scheiner, c. w. (2015). the motivational fabric of gamified idea competitions: the evaluation of game mechanics from a longitudinal perspective. creativity and innovation management, 24(2), 341-352. schouten, j. w., mcalexander, j. h., & koenig, h. f. (2007). transcendent customer experience and brand community. journal of the academy of marketing science, 35(3), 357-368. shankar, v. (2016). mobile marketing: the way forward. journal of interactive marketing, 34(1-2). shankar, a., & jebarajakirthy, c. (2019). the influence of e-banking service quality on customer loyalty. international journal of bank marketing, 37(5), 1119-1142. australasian journal of information systems sangroya, yadav & joshi 2021, vol 25, research on consumer evaluation gamified interaction of mobile applications 21 shankar, a., jebarajakirthy, c., & ashaduzzaman, m. (2020). how do electronic word of mouth practices contribute to mobile banking adoption?. journal of retailing and consumer services, 52, 101920. shernoff, d. j., csikszentmihalyi, m., schneider, b., & shernoff, e. s. (2014). student engagement in high school classrooms from the perspective of flow theory. in applications of flow in human development and education (pp. 475-494). springer, dordrecht. skadberg, y. x., & kimmel, j. r. (2004). visitors’ flow experience while browsing a web site: its measurement, contributing factors and consequences. computers in human behavior, 20(3), 403-422. storbacka, k., brodie, r. j., böhmann, t., maglio, p. p., & nenonen, s. (2016). actor engagement as a microfoundation for value co-creation. journal of business research, 69(8), 3008-3017. van noort, g., voorveld, h. a., & van reijmersdal, e. a. (2012). interactivity in brand web sites: cognitive, affective, and behavioral responses explained by consumers' online flow experience. journal of interactive marketing, 26(4), 223-234. venkatesan, r. (2017). executing on a customer engagement strategy. journal of the academy of marketing science, 45 (3), 289-293. werbach, k., & hunter, d. (2012). for the win: how game thinking can revolutionize your business. wharton digital press. yoo, b., & donthu, n. (2001). developing and validating a multidimensional consumer-based brand equity scale. journal of business research, 52(1), 1-14. yoo, c. w., sanders, g. l., & cerveny, r. p. (2018). exploring the influence of flow and psychological ownership on security education, training and awareness effectiveness and security compliance. decision support systems, 108, 107-118. yoo, c. w., kim, y. j., & sanders, g. l. (2015). the impact of interactivity of electronic word of mouth systems and e-quality on decision support in the context of the emarketplace. information & management, 52(4), 496-505. zhang, c., phang, c. w., wu, q., & luo, x. (2017). nonlinear effects of social connections and interactions on individual goal attainment and spending: evidences from online gaming markets. journal of marketing, 81(6), 132-155. zhang, k. z., lee, m. k., cheung, c. m., & chen, h. (2009). understanding the role of gender in bloggers' switching behavior. decision support systems, 47(4), 540-546. zurita ortega, f., castro sánchez, m., chacón cuberos, r., cachón zagalaz, j., cofré bolados, c., knox, e., & muros, j. j. (2018). analysis of the psychometric properties of perceived motivational climate in sport questionnaire and its relationship to physical activity and gender using structural equation modelling. sustainability, 10(3), 632. australasian journal of information systems sangroya, yadav & joshi 2021, vol 25, research on consumer evaluation gamified interaction of mobile applications 22 appendix 1. measurement items constructs and scale items sources challenge (ch) ch1: “the difficulty of challenges (e.g., answering the questions) increases as my skills and knowledge in [app]”. ch2. “[app] provides new challenges with appropriate pacing”. ch3: “[app] provides “hints” that help me overcome the challenges”. ch4: “[app] provides “supporting materials” that help me overcome the challenges”. jackson and marsh (1996) feedback (fb) fb1: “i receive feedback (e.g., message, reward point, etc.) on my progress when using [app]”. fb2: “i receive immediate feedback on my actions when using [app]”. fb3: “i am notified of new tasks in [app]”. fb4: “i receive information on my success (or failure) of goals in [app]”. jackson and marsh (1996) autonomy (au) au1: “i feel a sense of control and impact over [app]”. au2: “i know the next step in [app]”. au3: “i feel comfortable to control [app]”. jackson and marsh (1996) immersion (im) im1: “i forget about time passing while attending [app]”. im2: “i become unaware of my surroundings while attending [app]”. im3: “i experience an altered sense of time when taking [app]”. im4: “i can become involved in [app]”. im5: “i feel emotionally absorbed in [app]”. im6: “i feel engaged in [app]”. jackson and marsh (1996) social interaction (si) si1: “i feel cooperative toward other members in [app]”. si2: “i can collaborate with other members in [app]”. si3: “the cooperation in [app] is helpful to my learning”. si4: “[app] supports social interaction between colleagues (sharing, etc.)”. jackson and marsh (1996) cognitive brand engagement (cbe) cbe1: “dealing with [brand] through the game got me to think about [brand]”. cbe2: “i thought about [brand] a lot when i was dealing with it through the game”. cbe3: “dealing with [brand] through the game stimulated my interest to learn more about it”. hollebeek et al. (2014) emotional brand engagement (ebe) ebe1: “i felt very positive when i was dealing with [brand] through the gane.” ebe2: “dealing with [brand] through the game made me happy”. ebe3: “i felt good when i was dealing with [brand] through the game”. ebe4: “i was proud to deal with [brand] through the game” hollebeek et al. (2014) australasian journal of information systems sangroya, yadav & joshi 2021, vol 25, research on consumer evaluation gamified interaction of mobile applications 23 self-brand connection (sbc) sbc1: “[brand] reflects who i am”. sbc2: “i can identify with this [brand]” sbc3: “i feel a personal connection to this [brand]” sbc4: “i use this [brand] to communicate who i am to other people” sbc5: “i think this [brand] (could) help(s) me become the type of person i want to be”. sbc6: “i consider this [brand] to be ‘me’ (it reflects who i consider myself to be or the way that i want to present myself to other(s))”. sbc7: “this [brand] suits me well”. escalas (2004) brand usage intent (bui) bui1: “it makes sense to use [brand] instead of any other brand, even if they are the same” bui2: “even if another brand has the same features as this [brand], i would prefer to use it.” bui3: “if there is another brand as good as this [brand], i prefer to use it” bui4: “if another brand is not different from this [brand] in any way, it seems smarter to still use it”. yoo and donthu (2001) copyright: © 2021 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v25i0.3105 http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 literature review and conceptual development 2.1 gamification 2.2 flow theory 2.3 conceptual model 2.3.1 flow and consumer brand engagement 2.3.2 consumer brand engagement and self-brand connection 2.3.3 consumer brand engagement and brand usage intent 2.3.4 moderating role of gender 2.3.5 mediating role of brand engagement 3 methodology 3.1 sample and data collection 3.2 measures and instrument development 4 results 4.1 measurement model 4.2 structural model 4.3 common method variance 5 discussion 5.1 theoretical implications 5.2 managerial implications 6 limitations and future research paper title (paper title style) australasian journal of information systems boell & hovorka 2019, vol 23, selected papers from acis 2017 writing, arguing, contributing 1 writing, arguing, contributing a cogent argumentation framework for identifying, specifying, and evaluating research contribution sebastian k. boell university of sydney business school sydney, nsw 2006, australia sebastian.boell@sydney.edu.au dirk hovorka university of sydney business school abstract the predominant means by which research becomes visible and accessible to the research community is through publication. generally, publication requires careful framing of the research in relation to existing knowledge. as a contribution to knowledge cannot be selfevident, authors must persuade, through argumentation, the editors, reviewers, and the research community that their work offers a contribution. in information systems, the discussion of argumentation is often limited to the logic dimensions of argumentation, namely deductive, inductive, and abductive reasoning. in this paper, we demonstrate that argumentation requires the consideration of three additional dimensions of argumentation: rhetoric, dialectic, and social-institutional. kuhn’s concept of the disciplinary matrix is introduced as the background toward which a cogent argument is directed and against which contribution is evaluated. we then illustrate the role of argumentation through the example of the seminal paper by orlikowski and iacono on the role of it in information systems research. understanding the importance of argumentation in framing one’s research contribution is critical to authors, editors, and reviewers alike within and beyond information systems and its reference disciplines. keywords: argument; argumentation; cogency; research contribution; contribution; information systems; toulmin; kuhn; publication 1 introduction the last step in the research process is often the most time-consuming and problematic: persuading reviewers, editors, and readers that an instance of research contributes to knowledge. research outcomes do not speak for themselves but instead require researchers to present the research findings within the ongoing discourse of the research community. careful argumentation is indispensable if researchers want to make a compelling case for the importance, relevance, benefit, and value of their research for the community, for the broader field, or to practitioners. it is incumbent on authors “to convince their colleagues that their work has value. … [thus] the arguments researchers use to expound their [research findings] must be crafted carefully; otherwise, their theories’ contribution to knowledge might be overlooked” (corley and gioia 2011 p. 14; emphasis added). in this research, we deconstruct two critical aspects of corley and gioia’s claim to provide guidance for editors, reviewers, and authors in the crafting and evaluation of research for publication: first, the importance of argumentation and second, the context within which contributions can occur. australasian journal of information systems boell & hovorka 2019, vol 23, selected papers from acis 2017 writing, arguing, contributing 2 first, authors must construct a cogent or persuasive argument to convince editors, reviewers, and colleagues that a paper participates in the ongoing discourse of the research community. to this end, locke and golden-biddle (1997) identify two rhetorical strategies that legitimize research through constructing intertextual coherence (disagreement, cumulative progress, and latent consensus) and problematizing the existing literature (identify gaps, oversights, or alternative accounts) to expose opportunities for contribution to knowledge. in identifying that rhetoric is important in constructing a perception of novelty, locke and golden-biddle (1997) explicate one element of argumentation; however, argumentation also involves logic, dialectic, and social-institutional dimensions (rehg 2009; toulmin 1958; wenzel 1990). the insight that argumentation is essential to participation in a research discourse requires us to engage more thoroughly with argumentation theory (habermas 1984; rehg 2009; toulmin 1958). second is the question of how a contribution is recognized. while many submissions identify excellent questions, provide analysis of reasonable data, and deliver a literature review which articulates the research background, “the most common comment that one sees on reviews is that the authors need to enhance their ‘contribution’” (straub 2009 p. iii). to this end, novelty and utility are often noted as key criteria for assessing a contribution (hovorka and boell 2015, 2017). but neither novelty nor utility is self-evident, nor can they be properties of a paper itself. scientific contributions, therefore, do not exist as isolated papers but rather are evaluated against a background of accepted knowledge, such that research activity increases what is known, what is valued as knowledge, how we come to know, or what researchers aspire to know. while intertextual coherence has been identified as a rhetorical strategy (locke and golden-biddle 1997) and problematization as a tactical approach (barrett and walsham 2004), it is necessary to articulate the shared knowledge and commitments which a contribution seeks to change. thomas kuhn’s concept of the disciplinary matrix (dm) (kuhn 1977) offers guidance for participating in the material and discursive activity in which researchers participate and in which progress is made. in the following section, we take a closer look at theories of argumentation in academic discourse, distinguishing four different dimensions of argumentation. as this review will highlight the importance of understanding argumentation as involving the shared values and commitments of a community of researchers, we then turn to kuhn’s (1962, 1977) work on paradigms and more specifically the dm. using the concept of the dm, we outline different ways in which research can advance and challenge the shared symbolic generalizations, exemplars, and models of a community to which research seeks to contribute. thus, we claim that argumentation of the importance, relevance, benefit, and value of the contribution made by research is the result of a persuasive argument in relation to a dm. we then demonstrate the importance of argumentation using toulmin’s argumentation scheme in relation to the seminal paper by orlikowski and iacono (2001) on the it artefact. 2 argumentation in this article, we describe argumentation as the action or process of systematic reasoning in support of an idea, action, theory, or method using logic, rhetoric, dialectic, and socialinstitutional dimensions of argumentation. while the importance of argumentation is recognized within the is community (levy and ellis 2006; ochara 2013), to date, argumentation is often understood in terms of the logic used in argumentation, namely the difference between deductive reasoning, inductive reasoning, and abductive reasoning australasian journal of information systems boell & hovorka 2019, vol 23, selected papers from acis 2017 writing, arguing, contributing 3 (ochara 2013). while these are important differences in the logical structure of arguments, focussing on the logic dimension of arguments alone offers only a limited picture of what is involved in argumentation. understanding the types of logic applied to research thus cannot offer a sufficient picture as to why research fails or succeeds in persuading other researchers of its contribution. to this end, we can look at the work by rehg (2009) on the cogency of argumentation in scholarly discourse. cogency conceptualizes how scholars persuade their peers of the value of their work. rehg’s conception of cogency brings together different perspectives on argumentation, including toulmin’s (1958) work on how arguments are constructed, habermas’ (1984) work on discourse and ideal speech situations, as well as kuhn’s (1977) work on scientific revolutions. rehg (2009) suggests that argumentation can be understood as involving four dimensions which together produce cogency in academic work. these four dimensions include logic, rhetoric, dialectic, and social-institutional aspects of argumentation (table 1). we now turn to each of these four dimensions, outlining how each offers important insights into the role that argumentation plays when authors try to persuade their readers, editors, and reviewers of their contribution. dimension exemplary aspects of cogent argumentation logic • relevance (is relevant information overlooked?) • plausibility (are premises sufficiently precise?) • non-contradiction and consistency • support for premises dialectic • providing proofs for claims • acknowledging other views and contributions • creating, considering, and responding to challenges • engaging in critical assessment rhetoric • clearly reference how research assents to or dissents from earlier work • sufficiently demonstrate expertise in used methods and theories • appeal that the research benefits the research community • appeal that the research benefits practice and society socialinstitutional • unbiased – peer review • acknowledge limitations due to socioeconomic and material resources • appeal to prestige / expertise / reputation • acknowledge reputation and achievements of others table 1. dimensions of cogent argumentation 2.1 the logic dimension of argumentation the logic dimension of argumentation is interested in how arguments are built in terms of how syntactically and semantically the conclusion and premises are linked to each other. this requires analysing the reasoning brought forward for linking the conclusion of an argument to specific premises that support such a conclusion (rehg 2009). within the academic context, traditionally the interest has been in two logical types of reasoning: deductive reasoning as well as inductive reasoning, with a clear favour for the former. these two forms of reasoning have been added to by abductive reasoning as a third form of the logic of argumentation (gregory and muntermann 2011; ochara 2013). australasian journal of information systems boell & hovorka 2019, vol 23, selected papers from acis 2017 writing, arguing, contributing 4 deductive reasoning employs the idea that the acceptance of specific premises forms the foundation from which specific claims can be made that logically must follow from those premises. for instance, if one premise is that clouds are the source of rain and a second premise is that there are currently no clouds, then deductive logic would conclude that it is currently not raining. deductive reasoning is used when testing specific hypotheses or premises postulated by a model, such as that the intention to use technology increases when technology is both useful and easy to use. following deductive reasoning, any instance where the intention to use technology is high despite the technology being useful and easy to use would reject the deductive logic of the model as incorrect. inductive reasoning, in contrast, justifies its conclusion based on high likelihood given past experience. that is, a particular conclusion is not certain but highly likely given specific premises. in contrast to deductive reasoning, an exception to the expected behaviour does not falsify the premises but may be explained by using additional premises stating exceptions. inductive reasoning is required in abstracting generalized rules from specific observations. one example of inductive reasoning could be that the sun is rising in the morning, as the sun has so far risen every morning. inductive reasoning is used when researchers generalize their research findings beyond the observations made by them as they make predictions about likely future observations. abductive reasoning, sometimes also called retroduction, was introduced by peirce as additional form of logical reasoning (douven 2017). given the history of science, where explanations offered by long-standing theories eventually turned out to be incorrect, the possibility of using purely deductive reasoning is rejected. instead, abductive reasoning uses the idea of best possible explanation for an observation given what is known. abductive inference thus provides the most probable conclusion given specific premises. in research, abductive reasoning is used when existing theory guides the interpretation of observations as to what is the most likely explanation for a specific observation. looking at argumentation from the logical dimension, it is important that arguments are plausible, non-contradicting, and based on consistent use of premises. judging the logic of arguments will thus evoke the dialectic dimension of argumentation as providing the foundation on which the syntactic and semantic link between premise and conclusion is assessed as appropriate. 2.2 the dialectic dimension of argumentation the dialectic dimension of argumentation involves the rules, standards, attitudes, and behaviours applied by participants engaged in argumentation (rehg 2009; wenzel 1990). to be considered cogent, arguments must be open to critical assessment by the community according to dialectic standards used for challenging, testing, and rebutting the content of arguments (rehg 2009). from the dialectic dimension, arguments have to adhere to dialectic standards applied by a community as part of their practices used when competing for better arguments (habermas 1984). dialectic aspects of argumentation include, for instance, that arguments need to stay on topic or that authors need to provide proof for their claims. within academic discourse, the dialectic dimension of argumentation requires that arguments need to be tested in relation to challenges brought forward regarding an argument’s content (rehg 2009). the need to respond to such challenges is frequently involved in the review process where authors, as the creators of an argument, have to respond to challenges brought australasian journal of information systems boell & hovorka 2019, vol 23, selected papers from acis 2017 writing, arguing, contributing 5 forward by the review team. but likewise, the dialectic dimension of argumentation also requires authors to respond to questions when they present their work to the academic community at conferences, at seminars, and in talks. the dialectic dimension involves all conditions according to which arguments can be openly and critically tested. the dialectic dimension provides the foundation for how argumentation is to be undertaken in accordance with the agreed-upon standards within a community of competent participants. following habermas’ (1984) concept of an ‘ideal speech situation’, academic discourse is generally agreed to be undertaken in a way where different viewpoints are allowed to participate, where deviating views are not suppressed or excluded, where any argument can be expressed without restraint, and where authors are not deceptive. 2.3 the rhetoric dimension of argumentation the rhetoric dimension of argumentation is concerned with the effective rhetorical presentation of an argument in a social-psychological sense (rehg 2009). cogent argumentation is not simply a matter of using logical devices such as causal reasoning or inductive analogy, but also depends on effective assertion towards the audience that one is in a position to make a competent judgment in a particular area. as researchers will bring various assumptions, training, backgrounds, and expertise to their research, they will differ in how they judge existing knowledge and in how a proposed hypothesis or research question may be considered appropriate for advancing knowledge (rehg 2009). thus, authors must use rhetoric to effectively communicate their expertise and credibility, as well as the relevance of their research for the audience. for instance, authors can demonstrate that they have relevant knowledge of a field by discussing seminal papers and introducing alternative viewpoints in their literature review. cogent argumentation usually also involves discussing how the findings or standpoint presented in a paper benefit other researchers and practitioners. when discussing existing knowledge to construct one’s contribution, one of two standpoints can be taken: assent or dissent (rehg 2009). a position of assent accepts existing knowledge claims, to build upon them in one’s own research (locke and golden-biddle 1997). authors summarize existing knowledge to demonstrate a cumulative research tradition, upon which they can then build their own research efforts. this can be achieved by showing explicit or latent consensus in earlier research given the relevance of particular constructs or the relationships among them. based on such constructed consensus, the authors can then argue for their own research as extending existing knowledge. in contrast, a position of dissent challenges existing knowledge claims as insufficient or incorrect. using this strategy, authors demonstrate their familiarity with existing standpoints and explanations in order to show how these are insufficient (locke and golden-biddle 1997). this rhetorical move can be achieved by demonstrating inconsistency, contradiction, or oversights in earlier research studies. by problematizing existing knowledge claims, authors create a foundation on which they can then propose an alternative standpoint. irrespective of the rhetoric standpoint chosen (assent or dissent), authors have to persuade their audience that their judgment of existing knowledge is warranted. thus, readers, editors, and reviewers need to be persuaded that that the authors are trustworthy and in a position to cast a fair-minded and balanced judgment on existing knowledge. australasian journal of information systems boell & hovorka 2019, vol 23, selected papers from acis 2017 writing, arguing, contributing 6 2.4 the social-institutional dimension of argumentation from the social-institutional dimension, cogent argumentation requires considering social and institutional procedures and rules and their influence on argumentation. cogent argumentation relies on sustaining particular social practices and is evaluated according to these social practices and procedures. thus social, institutional, and cultural aspects form the background and social order within which academics operate (rehg 2009). these interlocutors guide and constrain how normative and factual constraints can be used for scrutinizing the conception, execution, and presentation of research in terms of its dialogical adequacy. for example, the social-institutional dimension determines what can reasonably be expected within the context of available material resources and time constraints researchers face in a particular disciplinary context. often, social-institutional aspects involved in scholarly argumentation are “taken for granted in the process of argumentation” (rehg 2009 p. 152). an example of one such aspect would be that cogent argumentation will involve a process of peer review, where research findings are scrutinized by experts before they are made available to the wider community of scholars. another social-institutional assumption is that journal-level ranking by impact-factor or university ranking lists are suitable measures of research quality. other social-institutional aspects are: the way in which disciplinary groups are organized; how academic achievements are recognized through awards and prizes; the reputation and social relationships of researchers; and facilities, financing, or other socio-economic forces (e.g. big-science vs. smallscience research). researchers can therefore rely on tacit agreement that a particular instance of research is undertaken within certain constraints. what constitutes, for instance, a suitable subject population to sample (e.g. students, is professionals, or executives) will depend on what is considered reasonable when taking into account financial constraints or access within a particular research environment. through ongoing discourse within the academic community, such as panels at conferences or opinion pieces, the merits of social-institutional practices can be challenged. if socialinstitutional practices are perceived as hindering the exchange of reasonable arguments, they can be questioned from the dialectic or rhetoric dimension of argumentation. socialinstitutional practices will thus only persist as long as they are perceived as adequate in supporting the exchange of cogent arguments. overall, all four dimensions of argumentation come to bear when authors seek to persuade the academic community of the contribution of their work. when properly utilized, these dimensions can produce cogent arguments that not only adhere to the tenets of logical form, but also provide sufficient force to persuade. but returning to our own critique that novelty and usefulness do not stand on their own, we can question how the norms for argumentation are established, evaluated and maintained within a community of scholars. as kuhn notes, principles of theory choice (for problem-solving) and the evaluation of contributions are sociological. we must understand the “nature of the scientific group, discovering what it values, what it tolerates and what it distains” (kuhn 1970 p. 237). while the four dimensions discussed above highlight the relevance of argumentation when presenting and assessing research, argumentation always takes place within the context of a australasian journal of information systems boell & hovorka 2019, vol 23, selected papers from acis 2017 writing, arguing, contributing 7 community of scholars. we now turn to the commitments shared by the group of scholars within which our research is problematized and toward which our argument is oriented. 3 argumentation within the disciplinary matrix while argumentation is critical in persuading editors, reviewers, and the research community of the value of a research instance, it is important to understand the context within which the argument is oriented. contribution requires that some ‘body of knowledge’ or set of beliefs exists, into which a contribution can be placed. here, we turn to kuhn’s (1962) concept of paradigm to underscore the dependence of scientific research upon concrete examples that “bridge what would otherwise be gaps in the specifications of the content and application of scientific theories” (kuhn 1965 p. 16). the term paradigm has been criticized as ill-defined (masterman 1970), as enabling a reification of the subject/object dualism (burrell and morgan 1979), and as providing multiple and conflicting interpretations (hassan and mingers 2018). kuhn acknowledged that he “lost control of the word [paradigm]” and introduced the disciplinary matrix (dm) to describe the commitments of a community, which “enabled them to solve puzzles and that accounted for their relative unanimity in problem-choice and in the evaluation of problem-solutions” (1970 p. 271; emphasis added). the intellectual function of these shared commitments is to determine “the patterns of theory, the meaningful questions, the legitimate interpretations, etc. within which theoretical speculation is bounded” (toulmin 1970 p. 40). kuhn suggests the term ‘disciplinary matrix’ as: “‘disciplinary’ because it refers to the common possession of the practitioners of a particular discipline; ‘matrix’ because it is composed of ordered elements of various sorts, each requiring further specification” (kuhn 1977 p. 182). in kuhn’s view, a central locus of interest is the community (e.g. discipline) which can ideate, perform, and evaluate research in a coherent manner without a set of declarative rules. for kuhn, the discipline is recognized as the community of researchers who share commitments, not as the more recent meaning of a political/institutional group named by a university as “a discipline” (e.g. information systems, business analytics, management). he argues that communities become scientific when a group shares research commitments rather than by adopting or imposing a pre-existing "scientific" set of criteria. the dm, while historically grounded in the natural sciences, offers a starting point from which to investigate the shared commitments which coordinate research and through which contributions are evaluated in the social sciences. in table 2, we describe and provide illustrations of kuhn’s concept of shared commitments, including exemplars, symbolic generalizations, and models (kuhn 1977). these shared commitments are not considered discrete paradigms by researchers themselves, but rather are the background upon which research activities are guided in their performance and evaluation (masterman 1970). we use them as analytic categories to illustrate how argumentation can be directed toward the elements of a community’s dm. the aspects of the dm encapsulate the practice into which new researchers are enrolled and become the accepted ways for framing problems, the vocabulary used to articulate research, and the methods and instruments through which research is performed. thus, problematization (barrett and walsham 2004; sandberg and alvesson 2011) occurs within and between dms and it is from within a dm that contributions are argued. australasian journal of information systems boell & hovorka 2019, vol 23, selected papers from acis 2017 writing, arguing, contributing 8 particular description symbolic generalizations • standardized forms for expressing problems and solutions • readily accepted without question e.g. using a hypothesis to express a research problem e.g. a regression in the form: y = β0 + βjxi + β2x2 + ... + βnxn exemplar • finished and published research • are used in enrolling researchers into a community • demonstrate what problems are and how they can be solved • provide templates for engaging in new problems and their solution e.g. walsham’s interpretive research on is (walsham 1995) e.g. technology acceptance model (davis 1989; davis et al. 1989) models (theories) • preferred analogies for describing phenomena • abstract representations of phenomena • when held deeply, become objects of metaphysical commitments (ontology) e.g. a gas behaves like a collection of microscopic billiard balls e.g. rogers’ model of diffusion of innovations table 2. particulars (shared commitments) of a disciplinary matrix symbolic generalizations are the expressions of a community that permit representation of explicit, formalized components of the dm (kuhn 1977). symbolic generalizations connote the unquestioned means by which a community represents its own activity. in is, few would disagree that conceptual frameworks (e.g. a boxes-arrows diagram), regression models, hypotheses or propositions, and statistical correlation tables are suitable means for representing research problems and their solutions. for example, there may be disagreement about a particular conceptual framework; however, by and large, the use of a conceptual framework as a means to express and conduct research is readily accepted within the is community. exemplars are a key particular in kuhn’s conception of how involvement in a disciplinary practice is learned, maintained, and stabilized. exemplars illustrate and reveal: a specific problem-solution; manifestations of analogies and metaphors; research heuristics; or methods of problem-solving. is examples include walsham’s (1995) “interpretivism in is” and “design science research” (hevner et al. 2004), which each contain the key theoretical approaches and techniques, and which explicate applications of those approaches in solving important problems. another well-known example in is is the technology acceptance model (davis 1989; davis et al. 1989), which is frequently used in classes for training research students. models (theories) are generalized beliefs or patterns about the world (e.g. the analogy that electricity can be viewed as a ‘fluid’ flowing through conductors). theories are then more specific and precise attachments of symbolic generalizations to specific phenomena of the world. for kuhn, theories are a foundation upon which problems are solved. an example of a deeply held model in is is presented by weber (2012) as: “all concrete things in the world possess properties … a human (a concrete thing) may possess a property that he uses an information system, … we perceive a property … [as] an attribute … a particular user of an information system has two attributes that relate to the information system: ‘perceived ease of use’ and ‘perceived usefulness’” (pp. 3-4). this model represents relationships between australasian journal of information systems boell & hovorka 2019, vol 23, selected papers from acis 2017 writing, arguing, contributing 9 humans and information systems and also specifies the ontological commitments of a community of researchers. kuhn’s concept of the dm provides a means for describing the commitments of a community of scholars within which research argumentation is oriented. while kuhn uses the three elements of symbolic generalizations, exemplars, and models (theories) to describe the shared commitments held by a community of scholars, he explicitly acknowledges that they do not provide an exhaustive list of all of the shared commitments. indeed, further commitments may include particular values held by a community (e.g. emphasizing the importance of relevance to practice) or the instruments used by a community of scholars for advancing the production and verification of knowledge. the critical insight is that the dm held by a community of scholars orients the cogency of argumentation. we can thus analyse the argumentation of a research paper to demonstrate this orientation beyond the level of an individual paper, outward to a community of researchers. 4 argumentation in research: an example toulmin (1958) provides a scheme for analysing the logical structure of arguments beyond the formal logics of deduction, induction, and abduction. he proposes a scheme describing how arguments need to be structured if they are to persuade recipients of a particular claim. stripped back to its bare bones, toulmin’s scheme states that any argument involves at least three basic elements: a claim, which is the position or belief that one is arguing for; a ground that is brought forward to support the claim; and a warrant, which is the often-implied rationale by which a ground actually supports a claim (figure 1). toulmin further extends this basic scheme by stating that both ground and warrant can be further substantiated or backed if needed, and that a claim can be further qualified, for instance, by stating specific conditions under which exceptions are possible. figure 1: the bare bones of toulmin’s (1958) scheme of argumentation. in order to demonstrate that a contribution needs careful argumentation, we can now use toulmin’s (1958) scheme to unpack the overall structure of the argument made by orlikowski and iacono (2001) (figure 2). the claim stated by orlikowski and iacono (2001) addresses the community of information systems researchers, as the authors argue for a shift in the way the community engages in research. the authors argue that a shift in the communities’ research agenda is needed to develop “conceptualizations and theories of it artefacts; and incorporating such conceptualizations and theories of it artefacts expressly into our studies” (ibid p. 130). this claim is grounded in the authors’ observation that in the year 2001, most is research had a “tendency to take it artefacts for granted in is studies” (ibid p. 133). this black-boxing has limited the ability of the is research community to understand the implications of it. to support their claim, the authors provide explicit evidence to link the ground of their argument to their claim. to do this, the authors analyse all 177 papers published during the first ten years australasian journal of information systems boell & hovorka 2019, vol 23, selected papers from acis 2017 writing, arguing, contributing 10 of the journal information systems research (isr) and demonstrate that the vast majority of papers published in isr did not seriously engage in theorizing it artefacts. development of the discipline of is is held back by a lack of serious engagement with it in its theorizing: “it artefacts in is research tend to be taken for granted or are assumed to be unproblematic.” (p. 122) “by and large it artefacts continue to be under theorized.” (p. 121) “the tendency to take it artefacts for granted in is studies has limited our ability as researchers to understand many of their critical implications” (p. 133) through an analysis of the role of it in a representative sample of published research, the authors demonstrate that much research is not seriously theorizing it artefacts: “taken together we see that 88 percent of all papers published in isr [information systems research] over the past 10 years adopt a nominal, proxy, tool, or computational view of the it artefact.” (p. 130). this also indicates the use of a qualifier in their argument, as 12 percent of research (taking the ensemble view) already does what orlikowski and iacono are calling for. the is community has to strengthen its effort in developing deeper theorizing of it artefacts, by changing its existing research agenda: “we propose a research agenda [...]: developing conceptualizations and theories of it artefacts; and incorporating such conceptualizations and theories of it artefacts expressly into our studies. […] moving beyond received disciplinary notions towards broader and deeper interdisciplinary conceptualizations of it artefacts is not only possible, but essential if the is field is to make important contributions to the understanding of a world become increasingly interdependent with ubiquitous, emergent, and dynamic technologies.” (p. 130) figure 2: using toulmin’s argumentation scheme to describe the overall argument of orlikowski and iacono (2001) as our analysis of the argument of orlikowski and iacono (2001) demonstrates, the contribution made by them involves several dimensions of argumentation. first, their contribution evokes the social-institutional dimension of argumentation as they argue toward the shared commitments of the is research community, pointing out that while the community generally agrees on the importance of it, most research takes it for granted. rhetorically, they dissent from earlier research, as they show that the vast majority of research published in one of the leading journals in the field does not seriously engage in theorizing it artefacts. dialectically, the paper engages in the collection and analysis of evidence for the authors’ claim. the validity of this evidence is difficult to be disputed by those associating themselves with the is research community and hence forming the intended audience of the publication. finally, the authors employ deductive and abductive logical reasoning, as they link the outcome of their analysis with their claim that a shift in the communities’ research agenda is needed. furthermore, we can see that the argument made by orlikowski and iacono (2001) is oriented outward toward the disciplinary matrix of the information systems research community. they use symbolic generalizations widely accepted within the is community to present the results of their analysis: their findings are summarized in a table listing the number of publications australasian journal of information systems boell & hovorka 2019, vol 23, selected papers from acis 2017 writing, arguing, contributing 11 allocated to different conceptualizations of it. their paper also introduces a model describing how it is used in information systems research. this model is a detailed description of five clusters of conceptualizations of it, that together contain 14 different subcategories. this model offers a theoretical account for how the discipline engages with and uses it as part of its research. since its publication, the paper has become an exemplar for how it is conceptualized in is research and it has instigated an ongoing debate about the discipline and its research orientation (e.g. alter 2003; benbasat and zmud 2003; hirschheim 2006; hirschheim and klein 2003; ives et al 2004; king and lyytinen 2006; lyytinen and king 2004; myers, 2003; robey 2003). the overview of different conceptualizations of it has become important in orienting a new generation of is researchers into the richness and diversity of approaches to it within the wider is community. 5 discussion the argument presented to this point discloses scientific contributions as discursive and material argumentation. contribution is not self-evident, and a paper cannot make a contribution in isolation. authors need to place arguments in relation to a disciplinary matrix, which requires careful and appropriate structuring and understanding of the shared commitments of a community. this positioning increases the importance of engaging with literature as a means to learn the dm of a community. engagement requires in-depth reading of existing research to understand relevant research problems: the accepted means to express research outcomes and findings (e.g. accepted use of symbolic generalizations); appropriate formulations for presenting research insights; appropriate research instruments; and other aspects of the community’s dm. this nuanced understanding reveals three important insights for the social sciences, including is. first, argumentation and philosophical inquiry play a crucial role as communities borrow from or infiltrate other communities of scientific practices, both within the broader is field and from other disciplines. when models, symbolic generalization, or exemplars from one dm are imported into another dm, problematization (sandberg and alvesson 2011) must occur. this is not simply aggregation from one dm to another, but indicates fundamental changes and the emergence of altered models, generalizations, values, or problem domains. ontological and epistemological assumptions may be questioned, evaluative criteria may be critiqued, and the phenomenon itself may come under scrutiny. this suggests that challenges to the components of the disciplinary matrix are only achieved through argumentation. for example, lee and hubona (2009) illustrate how authors can emphasize the dialectic dimension of argumentation by calling upon positivist researchers to adhere to their own rigorous commitments. their critique does not seek to refute or subvert positivist approaches; rather, they are identifying the logical implications inherent in the values (e.g. prediction and validity) and methods (e.g. sampling and regression) of positivist research, that would enable the community to strengthen its own claims. secondly, new communities may coalesce as epistemological approaches, models, and exemplars are compounded and institutionally legitimized. an example of a nascent dm is the design science research community. design activities, products, and design models that existed in other guises were widely dispersed across communities (indulska and recker 2010; kuechler and vaishnavi 2008) and were difficult to place into the literature. a paper commissioned by a top journal (hevner et al. 2004) consolidated prior knowledge into what has become a seminal exemplar, around which symbolic generalizations, academic practice, australasian journal of information systems boell & hovorka 2019, vol 23, selected papers from acis 2017 writing, arguing, contributing 12 illustrations, and the institutional force of journal special issues, conferences, and expertise have evolved. notably, dsr has not subsumed other communities, nor does it represent a revolution in research thought; rather, it coexists with other communities and is becoming a legitimate approach to creating knowledge within the broader domain of socio-technical development. thirdly, our perspective invites a re-examination of social sciences like is as scientific practices. is is not subject to disruptive revolutions or the refutation of extant theories and exemplars, which characterize kuhn’s account of the natural sciences. it is reasonable to question why a diverse field like is, composed of multiple communities (indulska et al. 2011) with different disciplinary matrices, would progress in the same manner as a natural science. as a field largely defined by institutional organizations, is is composed of intellectual communities and competing schools of thought, which are supported by identifiable sets of journals (larsen et al. 2008), conferences, special issues, and special interest groups (sigs). but argumentation within and between these communities does not have the force of crisis behind them that would overthrow or subsume other communities in the natural sciences; rather, argumentation expands the set of metaphysical orientations, models, and exemplars that coexist, compete (for journal and conference space, funding, and citations), and obtain ‘partial communication’ among members of differing communities. as a science, is largely pursues a coherence (internally consistent) approach rather than a correspondence (to nature) approach. argumentation enables a disciplinary matrix to evolve through challenges, support, alteration, or addition to existing shared commitments. argumentation may also counter and constrain such evolution. here, the taken-for-granted role performed by journals can be seen as enacting magisterial authority (toulmin 1970), a social-institutional aspect of argumentation that stabilizes and constrains change in a disciplinary matrix. when a dm is strongly held, editorial processes seek to shape papers into contributions which appeal to a broad research audience and are thus likely to be cited; however, integrating new exemplars, models (theories), and values, or building communities who share new sets of commitments (e.g. dsr, data science) are slow processes. the tendency to exclude novel or challenging ideas during the review process is therefore an area where study of the editorial magisterial authority practices may reveal opportunities for progress by changing the goals of specific outlets. as argumentation is foundational to change and progress in scientific communities, we see additional opportunity to challenge the format of conferences and publications in is. for example, the dialectic element invokes the claim that cogent arguments should stand up in open debate. yet, as a field, few papers are debated or become the focus of a discussant; rather, they are commented upon and shaped by editors and reviewers in a process that is inaccessible by the wider is community. 6 conclusion persuading reviewers, editors, and the community at large that a piece of research makes a useful contribution to the research community and is worthy of publication requires forceful argumentation using logic, rhetoric, dialectic, and social-institutional dimensions to position the work; however, to date, much of the discourse around argumentation in information systems overly emphasizes the logic dimension of argumentation at the expense of the rhetoric, dialectic, and social-institutional. expanding the role of argumentation in persuading others of the contribution of one’s work is important, as in information systems and the social australasian journal of information systems boell & hovorka 2019, vol 23, selected papers from acis 2017 writing, arguing, contributing 13 sciences, more generally, manuscripts do not speak for themselves. to be evaluated as contributing to the shared disciplinary matrix of a community, authors must use all aspects of argumentation to construct the contribution against the background of existing knowledge. argumentation must be placed in the context of the shared commitments of the community to which the contribution is to be made. the often-referred to criteria of novelty and utility for assessing contribution (hovorka and boell, 2015, 2017) per se provide limited guidance to researchers in formulating argumentation to editors and reviewers for evaluating claims of contribution. viewing contribution as involvement in an ongoing discourse will enable authors to problematize their research such that future researchers can more clearly adapt and translate the work in ways which inform new research. for a paper to be assessed as a contribution, the reviewers, editors, and readers must be persuaded that the paper improves the research discourse of the community. thus, specific attention to the relation between elements of argumentation in a paper and elements of the disciplinary matrix of the community can persuade reviewers and editors of the contribution. in our paper, we highlight that contributions can only be made in relation to the commitments held by specific communities. we put forward kuhn’s more developed concept of a dm to avoid the ambiguity and misappropriation of the word ‘paradigm’. dms are the “prerequisite of research, their grip on the mind is not merely ‘pickwickian’ nor can it be right to say ‘if we try, we can break out of our [disciplinary matrix] at any time’” (kuhn 1970 p. 242). a dm is thus not a set of tools researchers use but rather the background understanding of a practice in which the researcher enrols and is involved through reading exemplars, writing a dissertation, reviewing research by others, writing and referencing, and attending conferences. the worldview through which phenomena are disclosed and investigated, the kinds of problems that are relevant, applicable references, and appropriate methods and instruments of observation and measurement constitute the matrix of commitments upon which a community agrees. recognition that the cogency of science results from all aspects of the matrix of commitments shared within and between communities sensitizes authors to the need for rhetorical and dialectic connections to the appropriate aspects of the scientific discourse. the focus on discursive and material argumentation and on the dm shifts the involvement in the discourse toward challenging, extending, contrasting, measuring, reproducing, or refuting shared commitments of the community. references alter, s. (2003). 18 reasons why it-reliant work systems should replace “the it artifact” as the core subject matter of the is field, communications of the association for information systems 12(1), 23. barrett, m., and walsham, g. (2004). making contributions from interpretive case studies: examining processes of construction and use, in: kaplan b., truex d.p., wastell d., wood-harper a.t., degross j.i. (eds) information systems research. ifip international federation for information processing, vol 143. springer, boston, ma, 293-312. benbasat, i., and zmud, r.w. (2003). the identity crisis within the is discipline: defining and communicating the discipline’s core properties, information systems research 27(2), june, 183-194. burrell, g., and morgan, g. (1979). sociological paradigms and organizational analysis. london: heinemann. australasian journal of information systems boell & hovorka 2019, vol 23, selected papers from acis 2017 writing, arguing, contributing 14 corley, k.g., and gioia, d.a. (2011). building theory about theory building: what constitutes a theoretical contribution?, academy of management review 36(1), 12-32. davis, f.d. (1989). perceived usefulness, perceived ease of use, and user acceptance of information technology, mis quarterly, september, 319-340. davis, f.d., bagozzi, r.p., and warshaw, p.r. (1989). user acceptance of computer technology: a comparison of two theoretical models, management science 35(8), 9821003. douven, i. (2017). peirce on abduction, in: stanford encyclopedia of philosophy. gregory, r., and muntermann, j. (2011). theorizing in design science research: inductive versus deductive approaches, thirty second international conference on information systems, shanghai ais. habermas, j. (1984). the theory of communicative action, volume i, boston: beacon). hassan, n.r., and mingers, j. (2018). reinterpreting the kuhnian paradigm in information systems. journal of the association for information systems 19(7), 568-599. hevner, a.r., march, s.t., park, j., and ram, s. (2004). design science in is research, mis quarterly 28(1), 75-106. hirschheim, r. (2006). special research perspectives issue on the is core/identity debate, journal of the association for information systems 7(1), 29. hirschheim, r., and klein, h.k. (2003). crisis in the is field? a critical reflection on the state of the discipline, journal of the association of information systems 4(5), 237-293. hovorka, d.s., and boell, s.k. (2015). cogency and contribution in is research. thirty sixth international conference on information systems. fort worth, tx, usa, paper 1574. hovorka, d.s., and boell, s.k. (2017). contribution in information systems: insights from the disciplinary matrix, in: australasian conference on information systems. hobart, tas. indulska, m., hovorka, d.s., and recker, j. (2011). quantitative approaches to content analysis: identifying conceptual drift across publication outlets, european journal of information systems 21(1), 49-69. indulska, m., and recker, j. (eds.). (2010). design science in is research: a literature analysis. canberra. ives, b., parks, m.s., porra, j., and silva, l. (2004). phylogeny and power in the is domain: a response to benbasat and zmud’s call for returning to the it artifact, journal of the association for information systems 5(3), 4. king, j.l., and lyytinen, k. (eds.). (2006). information systems: the state of the field. chichester, england: john wiley & sons. kuechler, w., and vaishnavi, v. (2008). the emergence of design research in information systems in north america, journal of design research 7(1), 1-16. kuhn, t. (1965). logic of discovery or psychology of research, in: i. lakotos and a. musgrave (eds.), criticism and the growth of knowledge: volume 4: proceedings of the international colloquium in the philosophy of science, cambridge: cambridge university press, 1-23. australasian journal of information systems boell & hovorka 2019, vol 23, selected papers from acis 2017 writing, arguing, contributing 15 kuhn, t.s. (1962). the structure of scientific revolutions, (3rd ed.). chicago, il: university of chicago press. kuhn, t.s. (1970). reflections on my critics, in: i. . lakotos and a. musgrave (eds.), criticism and the growth of knowledge, cambridge university press, 231-280. kuhn, t.s. (1977). second thoughts on paradigms, in: the essential tension. chicago: university of chicago press, 293-319. larsen, k.r., monarchi, d.e., hovorka, d.s., and bailey, c.n. (2008). analyzing unstructured text data: using latent categorization to identify intellectual communities in information systems. decision support systems 45(4), 884-896. lee, a.s., and hubona, g.s. (2009). a scientific basis for rigor in information systems research, mis quarterly 33(2), 237-262. levy, y., and ellis, t.j. (2006). a systems approach to conduct an effective literature review in support of information systems research, informing science journal 9), 181-212. locke, k., and golden-biddle, k. (1997). constructing opportunities for contribution: structuring intertexual coherence and “problematizing” in organizational studies, academy of management journal 40(5), 1023-1062. lyytinen, k., and king, j. (2004). nothing at the center?: academic legitimacy in the information systems field, journal of the association of information systems 5(6), 220-246. masterman, m. (1970). the nature of a paradigm, in: i. lakatos and a. musgrave (eds.), criticism and the growth of knowledge, cambridge, cambridge university press, 59-89. myers, m.d. (2003). the is core-viii: defining the core properties of the is disciplines: not yet, not now, communications of the association for information systems 12(1), 38. ochara, n.m. (2013). linking reasoning to theoretical argument in information systems research, nineteenth americas conference on information systems, chicago, illinois. orlikowski, w., and iacono, c. (2001). desperately seeking the “it” in it research: a call to theorizing the it artifact, information systems research 12(2), june, 121-134. rehg, w. (2009). cogent science in context. cambridge, ma: mit press. robey, d. (2003). identity, legitimacy and the dominant research paradigm: an alternative prescription for the is discipline: a response to benbasat and zmud’s call for returning to the it artifact, journal of the association for information systems 4(1), 15. sandberg, j., and alvesson, m. (2011). ways of constructing research questions: gapspotting or problematization?, organization 18(1), 23-44. straub, d.w. (2009). creating blue oceans of thought via highly citable articles, management information systems quarterly 33(4), 2. toulmin, s. (1958). the uses of argument. cambridge: university press. toulmin, s. (1970). does the distinction between normal and revolutionary science hold water, in i. lakatos and a. musgrave (eds.), criticism and the growth of knowledge, cambridge, cambridge university press, 39-48. australasian journal of information systems boell & hovorka 2019, vol 23, selected papers from acis 2017 writing, arguing, contributing 16 walsham, g. (1995). the emergence of interpretivism in is research, information systems research 6(4), 376-394. weber, r. (2012). evaluating and developing theories in the information systems discipline, jais 13(1), 1-30. wenzel, j.w. (1990). three perspectives on argument: rhetoric, dialectic, logic, perspectives on argumentation: essays in honor of wayne brockriede, 9-26. copyright: © 2019 boell & hovorka. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 argumentation 2.1 the logic dimension of argumentation 2.2 the dialectic dimension of argumentation 2.3 the rhetoric dimension of argumentation 2.4 the social-institutional dimension of argumentation 3 argumentation within the disciplinary matrix 4 argumentation in research: an example 5 discussion 6 conclusion paper title (paper title style) australasian journal of information systems amrollahi & rowlands 2019, vol 23, research article openness and the effectiveness of strategic planning 1 an exploratory study of the relationship between the openness and effectiveness of strategic planning alireza amrollahi australian catholic university sydney australia alireza.amrollahi@acu.edu.au bruce rowlands griffith university brisbane australia abstract this paper focuses on a new approach for facilitating the participation of stakeholders in a process of strategic planning known as open strategic planning (osp). osp is recognised through three characteristics: inclusiveness, transparency, and the use of information technology (it) tools. drawing on the theoretical foundations of osp, the research explores moderating factors impacting the relationship between these characteristics and osp effectiveness by referring to qualitative data obtained from two open strategic planning projects. a secondary aim was to examine how stakeholders interpret the effectiveness of osp in an organisational setting. results indicate various moderating factors (level of trust, it literacy, and diversity of participants) impacted the relationship between the characteristics of osp and strategic planning effectiveness. the study formulates eight propositions, each is discussed in relation to the existing literature on strategic planning effectiveness. this paper is significant as it is the first exploratory research linking openness and strategic planning outcomes. keywords: open strategic planning, open strategy, inclusiveness, transparency, itenabledness. 1 introduction inclusion of stakeholders in the process of strategy formulation has been a key debate throughout the strategic planning literature (amrollahi & rowlands, 2017; hautz, seidl, & whittington, 2017). although there were a number of calls in the literature before 2000 for the inclusion of stakeholders other than management in the process of strategic planning (hamel, 1996; mintzberg, 1994a, 1994b), technical obstacles for inclusion of other stakeholders in a faceto-face strategy forum hindered their inclusion (langley, 1988; mitroff, barabba, & kilmann, 1977). however, the use of information technology (it) tools for the purpose of strategic planning (liinamaa, nuutinen, sutinen, & vanharanta, 2004) facilitated collaboration with all stakeholders leading to the development of a new concept called open strategic planning (osp). osp is recognised through the three characteristics: inclusiveness, transparency and the use of information technology (it) (tavakoli, schlagwein, & schoder, 2017; whittington, cailluet, & yakis-douglas, 2011). implementing these characteristics as part of a process to formulate “how an organisation is going to compete, what its goals should be, and what policies will be australasian journal of information systems amrollahi & rowlands 2019, vol 23, research article openness and the effectiveness of strategic planning 2 needed to carry out those goals” (tavakoli, schlagwein, & schoder, 2015a, p. 5) constitutes a recent working definition for osp. surveys on the use of management tools and techniques indicate that strategic planning is one of the most frequently practised techniques since the 1990s (berry & wechsler, 1995; rigby, 2001), and remains of great importance among practitioners (kalkan & bozkurt, 2013; rigby & bilodeau, 2013). furthermore, in response to a dominant perspective in the management literature focusing on the outcomes of strategic planning (golsorkhi, rouleau, seidl, & vaara, 2010; wolf & floyd, 2013), there are recent calls to develop a processual perspective for osp relying on practice that considers strategy as a process of actions, and an interplay between tools and people, rather than focussing primarily on strategy outcomes (jarzabkowski & paul spee, 2009). for these reasons, the evolution of strategic planning research and practice has been of key interest to researchers, as new topics such as strategy as practice, decentralisation of strategy formulation, and it-enabled strategic planning have emerged and altered the landscape of strategic planning (chesbrough & appleyard, 2007; wolf & floyd, 2013). osp is one of these new topics extending the domain of strategic planning research by incorporating computerised information systems and recommending new it tools for this purpose (amrollahi & rowlands, 2018). accordingly, this paper claims that the notion of osp can benefit from contributions from is research. there are three reasons for this: firstly, the concept of osp is driven by the notion of open innovation and its reliance on collaborative and open source research in the is domain (gegenhuber & dobusch, 2017; von krogh & spaeth, 2007). chesbrough and appleyard (2007) in one of the earliest work introduced the concept of osp through applying the notion of open innovation at a strategic level. however, open strategy goes beyond innovation in terms of topic range and purpose (leonhard dobusch, kremser, seidl, & werle, 2017; hautz et al., 2017). some authors even claim that open innovation is a subset of open strategy focusing on innovative strategy processes (whittington et al., 2011). secondly, it is now acknowledged to be core to the osp concept (amrollahi, najafikhah, morton, & sarooghi, 2019; morton, wilson, & cooke, 2016b) having been developed through advances in social it technologies such as web 2.0 (haefliger, monteiro, foray, & von krogh, 2011). thirdly, it and is models of collaboration such as crowdsourcing (amrollahi, ghapanchi, & talaei-khoei, 2014), jamming (bjelland & wood, 2008), and wiki (leonhard dobusch, 2012) have played a crucial role in the implementation of the osp concept. developments in these new technologies add weight to applying an is research perspective when exploring the notion of osp. despite many advancements in osp, existing literature has yet to focus on the impact of openness on effectiveness of strategic planning. to address this shortcoming, we investigate the factors moderating the impact of openness on strategic planning effectiveness. the remainder of this paper is structured as follows. the next section presents the study aims and scope, followed by an introduction to the concept of osp and its characteristics. the research method and details of our approach to data collection and analysis are then presented. next, we present the research findings in the form of eight related propositions. the paper concludes with a comparison of the findings with the literature, presents a summary of the contributions, and discusses future research directions. australasian journal of information systems amrollahi & rowlands 2019, vol 23, research article openness and the effectiveness of strategic planning 3 2 study aims and scope this study addresses a call by matzler, füller, koch, hautz, and hutter (2014) to investigate the complex phenomenon of osp from a technical, social, and political perspective focussing on how openness can impact the effectiveness of strategic planning. currently, the three characteristics of open strategy − inclusiveness, transparency, and the use of information technology − are widely accepted in the strategic planning community. a number of theoretical studies (chesbrough & appleyard, 2007; whittington et al., 2011) and numerous empirical studies (amrollahi & rowlands, 2017; aten & thomas, 2016; leonhard dobusch & kapeller, 2013; gegenhuber & dobusch, 2017; hutter, nketia, & füller, 2017; kendall, nino, & staley, 2008; liinamaa et al., 2004; mantere & vaara, 2008; morton, wilson, & cooke, 2016a; stieger, matzler, chatterjee, & ladstaetter-fussenegger, 2012) have reported that openness improves the effectiveness of strategic planning. in this study, our assumption is that the concept of osp can be understood through these three characteristics, that they impact on the effectiveness of strategic planning, and there is a relationship between openness and the effectiveness of osp. figure 1 illustrates this relationship. building on these assumptions, the next stage of knowledge accrual is to study what moderating factors facilitate the impact of osp characteristics on the effectiveness of strategic planning, and to explore how osp can be employed in an effective manner in organisations. figure 1 outline of the current study studying moderating and mediating factors is common in both strategic management (boyd, haynes, hitt, bergh, & ketchen, 2012; ketchen, boyd, & bergh, 2008) and is research (carte & russell, 2003; smith & keil, 2003). in the strategic management literature, moderating and mediating concepts have been differentiated. mediation refers to a variable transmitting the impact of an independent factor to an outcome, whereas moderation points out the conditions impacting the magnitude of a relationship (aguinis, edwards, & bradley, 2017). moderating factors are common in quantitative and meta-analysis research, but they are less utilised in qualitative research as they are difficult to interpret inductively from research data. however, as the aim of this study is to explore the conditions under which a set of principles can impact a desired outcome, the term moderating factors has been preferred over contingency factors. previous studies have examined certain aspects of osp, but little is known about the factors making these aspects effective. hence, this study seeks to answer the following research question: rq1. what factors moderate the impact of openness on the effectiveness of osp? the effectiveness of strategic planning (in general) has been studied through various indicators such as financial performance (capon, farley, & hulbert, 1994; kargar, 1996), australasian journal of information systems amrollahi & rowlands 2019, vol 23, research article openness and the effectiveness of strategic planning 4 profitability (griggs, 2002; robinson, 1982), and market performance (schäffer & willauer, 2003). using these indicators, however requires the implementation of the plan and measuring them until the conclusion of its time horizon. while the time horizon of strategic plans typically is more than one year, using the above measures of effectiveness for this study was not feasible due to time constraints. the inherent difference between osp and conventional approaches of strategic planning may impact the consequences of such a process (sailer, schlagwein, & schoder, 2017). therefore, there is no guarantee that the indicators developed for traditional strategic planning can accommodate specific dimensions of osp. for example, the literature asserts that organisational culture that values transparency and inclusiveness (l dobusch & müller-seitz, 2015), cultural factors such as trust (matzler et al., 2014), and specific types of organizational structure (amrollahi & ghapanchi, 2016; friis, 2015) as preconditions of osp are not required in conventional strategic planning. furthermore, the literature has identified new consequences and benefits for osp such as creating commitment, a sense of ownership (aten & thomas, 2016), and motivation among the stakeholders (hautz et al., 2017) that cannot be found in the conventional strategic planning literature. finally, the suggested process of osp is different from traditional strategic planning (amrollahi & rowlands, 2017; sailer et al., 2017; tavakoli et al., 2015a) and these differences will impact the way strategy effectiveness is perceived and measured especially from a strategy as practice point of view (golsorkhi et al., 2010; whittington, 1996). in addition, there are differences between the two approaches considering the number of strategy makers, a longer process of planning, and a more dynamic nature of osp compare to conventional methods of strategic planning. finally, considering inclusiveness of stakeholders forms a significant characteristic of osp, paying attention to stakeholders’ opinion on the effectiveness of strategic planning is crucially important in this area (amrollahi & ghapanchi, 2016). for this reason, this study is focused on the “perceived effectiveness of strategic planning” and employs a grounded theory approach to identify how stakeholders articulate their perception about the effectiveness of strategic planning. accordingly, the second research question guiding this study is: rq2. how do stakeholders interpret and perceive the effectiveness of osp? this research is, to the best of the authors’ knowledge, the first study to empirically examine the moderating factors that may lead to improving the effectiveness of open strategic planning with a focus on the impact of openness. the main contribution of the study is to produce a sound theoretical basis in the form of tentative propositions for future research on osp and strategic planning effectiveness, by recognising the micro mechanisms that originate from osp characteristics. the study also informs future practices of osp by pointing out the impact of openness and how it can improve the effectiveness of open strategic planning in organisational settings. 3 open strategic planning the concept of decentralisation in the strategy process (i.e. inclusion of middle managers and other stakeholders) first came under the spotlight during the 1990s. mintzberg (1994a), for example, criticised the differentiation between the concepts of strategic formulation and strategic implementation by highlighting the fact that this differentiation is in fact “to separate thinking from doing”. mintzberg also termed the information barrier between strategy planners and implementers as “the fallacy of detachment” (mintzberg, 1994b). hamel (1996) australasian journal of information systems amrollahi & rowlands 2019, vol 23, research article openness and the effectiveness of strategic planning 5 followed this stream of work by calling for a “democratic process of strategic planning” through stakeholders’ participation in the process. further studies of actors in strategic planning attracted the attention of scholars and made decentralisation of the strategy process a key topic (wolf & floyd, 2013). empirical studies of decentralisation in the 1990s and 2000s addressed topics such as the role of middle managers (jarzabkowski & balogun, 2009; spee & jarzabkowski, 2011; vilà & canales, 2008; wooldridge, schmid, & floyd, 2008) and the participation of more stakeholders (aldehayyat & al khattab, 2012; kargar, 1996; phillips & moutinho, 1999, 2000; segars, grover, & teng, 1998). these studies led to the development of principles of open innovation i.e. the use of both inflows and outflows of knowledge to improve internal innovation (chesbrough, 2006) in strategic planning research and the formation of the osp concept (chesbrough & appleyard, 2007). although various terminology has been used, namely: open source strategy, democratic strategy, strategy as practice of thousands, and open strategising; scholars have adopted these terms to explain a similar concept (morton, wilson, & cooke, 2015a; tavakoli, schlagwein, & schoder, 2015b). these characteristics forming the core of the osp concept (inclusiveness, transparency, it-enabledness) are summarised in table 1. characteristic description inclusiveness receiving strategy ideas from people outside of the management team (whittington et al., 2011). transparency providing people outside the management team with access to strategy input, process, and outputs (whittington et al., 2011). it-enabledness using it tools to open participation as an essential element of the strategy process (tavakoli et al., 2015a). table 1 characteristics of open strategy various benefits of osp have been suggested in the literature. for instance, whittington et al. (2011) suggested osp is an approach for improving commitment and understanding in strategy implementation through widening the scope of strategy ideas. liinamaa et al. (2004) reported on the it enactment of osp to make the strategy process more effective. leonhard dobusch and kapeller (2013) also proposed that greater openness in the strategy process leads to a greater diversity of open strategy practices. further studies have also mentioned various benefits for using osp such as: creating a dialogue about strategy (stieger et al., 2012), leveraging customers’ knowledge for strategy formation (newstead & lanzerotti, 2010), increase innovation (morton, wilson, & cooke, 2015b), integrating diverse perspectives (malhotra, majchrzak, & niemiec, 2017), creating a sense of community (hutter et al., 2017), and better firm performance (tackx & verdin, 2014). despite the increased attention paid to the osp concept in recent years from both an is and management perspective, a number of topics remained under-studied. these topics include the social and organisational aspects of osp, and the contingency factors impacting the effectiveness of using such approaches in practice. the current research addresses this gap by exploring the moderating factors facilitating the impact of osp. focusing on the moderating factors for achieving effectiveness goes beyond the characteristics of osp with a focus instead on contextual, social, and organisational factors. furthermore, this study is unique in that it focuses on identifying and understanding effectiveness from a stakeholder perspective and extends the literature about strategic planning effectiveness and osp. australasian journal of information systems amrollahi & rowlands 2019, vol 23, research article openness and the effectiveness of strategic planning 6 4 research method the purpose of this section is to argue for and justify the methodological orientation, design, procedures and data analysis used in this research. the current study uses the principles of a grounded theory (gt) approach to identify and understand the moderating factors improving the effectiveness of osp characteristics. according to strauss & corbin (1990), grounded theorising is well suited to capturing the interpretive experiences of stakeholders and developing theoretical propositions from them. gt is an inductive and qualitative method seeking to discover and develop a theory grounded in systematically gathered and analysed data (martin & turner, 1986; petrini & pozzebon, 2009; urquhart, lehmann, & myers, 2010). considering the explorative nature of the research focus specifically, as well as the historical relevance of a gt approach in both information systems and strategic planning research (lind & goldkuhl, 2006; rowlands, 2005) more generally, this paper claims that using the principles of grounded theory analysis is both relevant and needed to identify moderating factors based on the experiences of stakeholders. in this study, we use a straussian approach to coding in gt permitting a degree of flexibility regarding priori theoretical concepts through axial coding (gasson, 2009; strauss & corbin, 1998). based on guidelines for conducting grounded theory research by noted methodologists (corbin & strauss, 1990; urquhart et al., 2010), the following five steps were followed to develop a tentative theory: (1) iterative data collection and analysis; (2) concept development (open coding); (3) development of concept categories (axial coding); (4) theoretical sampling; and (5) theoretical integration. an overview of the process of data collection and analysis is illustrated in figure 2. figure 2 research process involving data collection and analysis as depicted in figure 2, this research commenced with the design and implementation of the osp concept in two cases. in both cases several interviews were conducted after the australasian journal of information systems amrollahi & rowlands 2019, vol 23, research article openness and the effectiveness of strategic planning 7 implementation of the designed approach and after the development of the strategic plan. the following section introduces both case studies. 4.1 the case studies as the notion of open strategy was new and less practiced at the time of conducting this study, the authors decided to present this approach and its potential benefits to several organisations. the approach we undertook at this stage is similar to the principles of “action research” highlighting the two stages of diagnostic and therapeutic (baskerville, 1999). similarly, we performed an analysis of the social situation in collaboration with managers, and after considering contextual elements in each organisation, proposed a solution for strategic planning in each case. two organisations accepted our invitation to apply the osp concept and the related online tools for developing their strategic plan. this paper, however, concentrates on the second stage (therapeutic) by highlighting the effects of the proposed solution. at this point, the issue of the number of cases or sample size deserves comment. different logics undergird sampling approaches. qualitative inquiry typically focuses in-depth on relatively small samples which are selected purposefully to illuminate the questions being studied. in a multiple-case approach there is no ideal number of cases, but yin (1994) suggests, that because sampling logic is not being used, typical criteria regarding sample size also are irrelevant. he concluded by saying that the number of replications is a matter of discretionary, judgmental choice. the two cases featuring in this research were selected to provide replication logic by providing variety and contrast among the cases (yin, 1994). the first case was an open source software (oss) project. at the time of strategic planning, it was almost four years since the initiation of the project. this organisation had experienced several changes during its lifetime highlighting the need to formulate a strategic plan for managers. at the time of conducting the case study, more than 30 developers were working in the team and five major organisations were using their software. managers and sponsors, software and service developers and users were stakeholder groups in the first case. according to managers, previous attempts to develop a strategic plan for the project did not succeed for several reasons including lack of access to all stakeholders, logistical problems in developing workshops, and a lack of consensus among stakeholders about the topics to be included in the final plan. however, managers and interviewees from other stakeholder groups all agreed that developing a strategic plan is necessary for the project to remain sustainable in the highly competitive environment of software development. to minimise and manage future changes in the project, the management team decided to develop a two-year strategic plan in 2014. the specific structure of the project and the importance of factors such as the role of software and service developers made osp relevant to this context. given these circumstances, and after presenting this approach and its potential benefits to management, they approved the osp project to commence. in this case, three groups of managers, developers, and users were invited to include their ideas about the future strategic directions of the project. after two months, a total of 78 ideas and comments were submitted in the online planning system. the ideas were collected in a transparent manner and they were available to all participating stakeholders to view and comment on. these ideas resulted in identification of 34 strategic themes. these themes were then submitted to project managers who were asked to approve/reject each theme to be australasian journal of information systems amrollahi & rowlands 2019, vol 23, research article openness and the effectiveness of strategic planning 8 considered in the strategic plan. finally, a two-year strategic plan was developed and published through the planning system. the developed plan was made available to all designated stakeholders using the same strategic planning system. the second case was conducted in a not-for-profit provider of life-long learning services. this organisation was found in 1991 with 19 members (working as both student and tutor) and reached 540 members in 2015. different groups of members, including management committee, tutors, and volunteers were stakeholders in this case. strategic plans are also a crucial document for not-for-profit organisations to help them meet complex missions while facing severe resource and personnel constraints (reid, brown, mcnerney, & j. perri, 2014). despite this necessity, the managers in this case had no experience in developing a strategic plan. as most not-for-profit organisations (including the 2nd case) are reliant on members and volunteers, their role was crucial for the future of the organisation and hence should be involved in any form of long-range planning. this research argues the important role of members and volunteers, and the not-for-profit motive of this case made the open strategy approach suitable in this context. for this reason, managers in the second case quickly adopted the idea of open strategic planning and confirmed project commencement. in the second case study, four groups participated in strategic planning (managers, volunteers, tutors, and members). during the idea submission phase, almost 100 ideas were submitted. each idea was then sent to two reviewers (who knew the organisation and its limitations well) and the reviewers commented on the appropriateness of each idea to be included in the final strategic plan. the ideas and comments were then sent to the president for final approval and based on submissions a strategic plan with four goals, ten objectives, and 27 strategies were formulated. all stakeholder groups have access to the output of strategic planning through the planning system as well as their internal newsletter system. a web-based information system was developed to facilitate the strategic planning process in both case studies. this information system included a multi-layer architecture enabling various capabilities including: idea submission, idea review, commenting and scoring ideas, idea approval, and managing users who participated. 4.2 data collection the main source of data within a case was a set of interviews administered after the development and publication of the strategic plan. the interviews in this research were conducted with a variety of stakeholders with different skill sets and different understanding of the concept of open strategic planning. for this reason, a conversational interview technique (neuman, 2011) helped the research team to better articulate the concepts during interviews and make sure both interviewer and interviewee had a mutual understanding of the conversation. a set of leading questions asked during the interviews are listed in appendix 1. as the main goal was to study the effectiveness of the planning approach, interviewees were selected regardless of their participation in the planning process. the first author of this paper conducted all the interviews and each interview took between 30 to 40 minutes in duration. data collection was stopped when little new evidence was being obtained, a situation known as theoretical saturation (shiau & george, 2014). after finalising the interviews, the process of transcribing and coding started. table 2 represents the number of interviewees in each category. australasian journal of information systems amrollahi & rowlands 2019, vol 23, research article openness and the effectiveness of strategic planning 9 first case second case position number of respondents position number of respondents managers 7 managers 2 developers 6 volunteers and tutors 3 users 3 members 3 total 16 total 8 table 2 number of respondents in each case in addition to the interviews, observations by the research team during the planning period and activities prior to planning formed a second source of data collected. this source of observational data, including notes from design and planning sessions, interaction with stakeholders, and interaction with managers during the planning period, helped the research team to better interpret the interview data during the data analysis phase. 4.3 data analysis data analysis was performed simultaneously with data collection. this type of analysis helped the research to find “where to sample from next in the study” known as theoretical sampling (urquhart et al., 2010, p. 369). for example, the focus of the interview questions changed at the end of interviews in the first case study to better capture interviewees’ opinion on contextual factors. in the first round of interviews, the theme of trust was highlighted and for this reason researchers asked questions related to this theme in the following interviews. the collected data was first analysed through an open coding approach. this process involved condensing data to identify significant themes and finding abstract representations for it (corbin & strauss, 1990). to satisfy constant comparison or continuous interplay between data collection and analysis as emphasised in gt method (urquhart et al., 2010), open coding started after the first interview and the open codes were gradually developed and refined after each interview iterating toward the final analysis findings. for example, in early stages of coding, different codes of self-confidence, involvement and participation (not included in final list of codes) were assigned to retrieved excerpts later merged to one code (motivation). data analysis continued with a focus on a process called axial coding (strauss & corbin, 1998). this phase of data analysis mainly focused on relating the codes (from open coding) together; rather than focusing on the textual excerpts in order to produce a more nuanced explanation about the phenomena of study (strauss & corbin, 1998). throughout the axial coding process the hypothetical relationships among emerging themes were checked deductively to ensure credibility of the claims (allen, 2017). in order to perform axial coding, extracted codes during open coding were reviewed, and each characteristic of osp was selected to represent a number of codes. these concepts were then selected as priori theoretical concepts or seed categories, as suggested by gasson (2009). for example, two codes of system efficiency and it literacy are both related to the technological factors and for this reason were categorised under the itenabledness code. australasian journal of information systems amrollahi & rowlands 2019, vol 23, research article openness and the effectiveness of strategic planning 10 table 3 coding examples table 3 contains samples of interview text and how open and axial codes were assigned and interpreted. as explained in section 4.2, in many instances, the research team’s observations helped the interpretation or added to the data collected during interviews. for example, interviewees raised a concern about the possibility of tracking their identity through their interview excerpt open and axial codes interpretation of the codes it seems a fantastic idea to ask for developers’ feedback on strategic issues and everyone can freely indicate his/her idea. however, there is an issue here and that’s the potential uncertainty that idea submitter may feel: according to both technical infrastructure and people position, people can be traced based on their idea. this is more highlighted for people who have been here for long and this may prevent the system to be completely open and everybody does a trade-off when submitting an idea. motivation inclusiveness trust transparency although the online system provides an anonymous approach for stakeholders to submit their ideas, but some ideas are still traceable as they have been previously mentioned in other communications by the same person. the research team observed a number of cases that managers made a guess after reading an anonymous idea. this may prevent stakeholders to submit their real idea. it’s the first time we have done anything like that. it’s new to members and the committee as well. i would say the focus of it, because we are clearly focused on how we are going to grow and to develop, i would say that was successful. it covered the main important areas of development and because it has focused on minds very much or where we are going in terms of the organization. i would say that would be most certainly the case. awareness inclusiveness organizational effectiveness effectiveness plan effectiveness effectiveness the novelty of osp approach was a challenge in both cases. although a portion of stakeholders were contacted about this approach, some of them were still unaware or were not sure about the importance of their role in the process. so they really feel ownership and they were interested in ongoing development of the strategic plan. how to do that is a challenge. maybe the only way to do that was to devote a lot more time going to talk to individual classes and become a much more one on one experience but again that takes a lot of time and manpower and right kind of manpower. organizational effectiveness effectiveness awareness inclusiveness this excerpt indicates that participation of people in the planning has recognised as a success factor that can lead to a sense of involvement for stakeholders and better communication is recognised as an approach to improve this participation. australasian journal of information systems amrollahi & rowlands 2019, vol 23, research article openness and the effectiveness of strategic planning 11 submitted ideas. notes from the strategic plan development sessions between planning facilitators and managers also confirmed the legitimacy of this privacy concern. the following section discusses relevant guidelines to ensure methodological rigor when employing a grounded theory method. 4.4 study rigor scholars engaged in subjective research should provide the readers with some assurance regarding the validity of their investigations. urquhart, lehmann, and myers (2010) offer a set of criteria that can be applied to gauge the validity of interpretive research. see table 4 for our evaluation of the interpretive methods used in this study, and how our research stands against such criteria. while the principles of gt method were used in this study to explore the moderating factors, similar guidelines are used to evaluate the rigor of the research undertaken. guideline for gtm summary from (urquhart et al., 2010, p. 369) application in this research constant comparison the process of constantly comparing instances of data within a category. conducting open coding at the same time with data collection and reflecting the new pieces of data on established categories. iterative conceptualisation increase the level of abstraction and relate categories of data to each other. during the axial coding process, categories of codes emerged reflecting elements of strategic planning process. theoretical sampling deciding the theoretical grounds where to sample from next in the study. changing the focus of interview questions during and revising them after the first case study. scaling up the process of grouping higher-level categories into broader themes in order to relate them to the broader literature. following the coding step and developing four concepts grounded into the three characteristics of osp and the concept of effectiveness ended in creating a level of abstraction applicable to different contexts. theoretical integration relating the theory to other theory in the same or similar field. enfolding the literature in theoretical integration in section 6. table 4 comparison of guidelines for gtm secondly, to help ensure credibility of data analysis, a triangulated approach (mingers, 2001; myers, 1997) was used where the second member of the research team reviewed the coding process and in several cases assigned different codes to some of the excerpts or commented on the relevancy of the performed interpretation. these differences were then discussed in regular meetings and the authors arrived at an agreement regarding the assigned codes. although a replication logic (yin, 1994) was used in this study, both cases reflected an acceptable level of diversity with respect to many variables. for example, they represent different industries (software vs. education), have different number of potential participants (smaller team of less than 50 in the first case, compared to a relatively bigger team of 540 in the second), different demographics (mainly middle eastern participants in the first case with all participants aged less than 35, compared to the second case in australia with an average age of 69). despite this diversity, the study does not make any claim about the generalizability of the findings. rather, we generalise to theory based on the context of studies where the data was australasian journal of information systems amrollahi & rowlands 2019, vol 23, research article openness and the effectiveness of strategic planning 12 collected. this insufficiency is due to the exploratory nature of the study where effectiveness of osp has not been researched. therefore, claims about generalisability of the substantive theory can be made when further findings are derived from multiple studies (gasson, 2009). this study’s focus is on transferability of findings and how the results can be adapted in other contexts (gasson, 2004). for this reason, in section 6.2 about future research we have made several recommendations for the application of the findings in other contexts by taking into account contextual factors impacting on transferability. 5 results twenty-four interviews with different stakeholders in two case studies were conducted and after the open coding process, the codes were assigned to the characteristics of osp, or stakeholders’ interpretations of strategic planning effectiveness. this section describes and explains the extracted codes. 5.1 strategic planning effectiveness based on their position, interests, and attitude, stakeholders in an organisational setting can have different understandings about the effectiveness of osp. stakeholders’ inclusion in the strategy process and considering the transparency of the process to them are mentioned as characteristics of open strategy. this participation highlighted the importance of understanding stakeholders’ interpretations about effectiveness, and numerous text referring to stakeholders’ interpretation of effectiveness were found. 5.1.1 planning efficiency the degree of efficiency as a result of using the osp approach is considered an indicator of effectiveness according to our data. for example, the fact that the planning process led to the development of a plan was considered an indicator of success for many participants: “i believe this [strategic planning] project has been successful because i can see a plan in front of me containing a number of strategies which can be implemented”. furthermore, the impact of using an osp approach to save the required time for planning was highlighted as an indicator of planning efficiency, as one of the interviewees described: “according to my experience always a collaborative approach arrives to a faster output compared to when you limit the strategic planning to a couple of people in the top organisational levels.” although efficiency and effectiveness were identified as different concepts in the management literature, stakeholders tended to compare osp with conventional approaches of strategic planning, and for this reason, they frequently referred to efficiency as an indicator of osp effectiveness. 5.1.2 plan effectiveness the contents of the developed plan formed another dimension for planning effectiveness in our interviews with stakeholders. for example, one of the managers criticised the effectiveness of the plan because its content did not cover certain aspects: “i had a broader definition of strategic plan in my mind. although i believe the current plan can be beneficial for us, i expected prioritising of the strategies and a model of implementation to be included in the final plan.” australasian journal of information systems amrollahi & rowlands 2019, vol 23, research article openness and the effectiveness of strategic planning 13 in one of the cases, the set objectives in the plan were the subject of attention for a respondent and she expressed concerns about this: “one thing concerns me: if we develop a very high standard for tutors we cannot get them.” although different interviewees mentioned different indicators for the effectiveness of the plan, they all agreed that the content of the developed plan formed an integral element when evaluating its effectiveness. 5.1.3 organisational effectiveness the potential impact of the developed strategic plan on the organisation was the most common explanation for strategic planning effectiveness. regarding the indicator of effectiveness, a manager in the first case identified osp to be effective as it can potentially contribute to creating “stakeholder alignment” in the organisation: “the collaborative model for strategic planning helped the organisation to align the stakeholders and this will help the organisation to attract attention of all stakeholders to the final product and this will help us to succeed in the market.” another instance of an organisational indicator for osp effectiveness was its impact on increasing stakeholders’ involvement in the achievement of organisational long-term objectives. the extent to which this objective had been attained was considered by many stakeholders as an indicator of success. for example, a manager mentioned: “the planning project has resulted in aligning stakeholders and this will make them involved in the organisation which is absolutely beneficial for us.” although organisational effectiveness appeared to be a prominent indicator, measuring it could be problematic as it depends on the organisational context and can differ from case to case. 5.2 transparency and strategic planning effectiveness when reviewing the results of open coding, we found two concepts as instances of transparency. these codes were trust and power, and according to the results of our study they played a moderating role on the impact of transparency on osp effectiveness: 5.2.1 trust during our analysis, the theme of trust emerged regarding the impact of transparency on strategic planning effectiveness. here trust was coded in accordance with stakeholders’ concerns about whether their ideas were going to be used in the strategic plan and how they believed their participation in osp could impact their organisational position. one of the developers mentioned that: “there is an issue here and that’s the potential uncertainty that the idea submitter may feel: according to both technical infrastructure and people position, [idea submitter] can be traced based on their idea. this is more highlighted for people who have been here longer and this may prevent the system to be completely open so everybody thinks about possible consequences when submitting an idea.” further, some interviewees believed that the strategic planning project had been initiated by managers as a means of pretence that they were following up on stakeholders’ ideas, but in fact stakeholders’ voices have not been heard in practice. there was even a lack of trust among australasian journal of information systems amrollahi & rowlands 2019, vol 23, research article openness and the effectiveness of strategic planning 14 participants about the planning team. many respondents expressed the opinion that as this project was not initiated by them, they couldn’t trust it and did not consider the process transparent. these trust issues prevented stakeholders to accept the transparency within the planning project or interpret its impact on the effectiveness, as one interviewee said: “this is a project started and continued outside of the team. therefore, you should not expect the team to fully participate and stay motivated about the process and the outcome of the planning project.” 5.2.2 power the effectiveness of using a transparent approach for strategic planning was questioned by some respondents because they thought their input wouldn’t be considered in the decisionmaking process because of a lack of influence. many stakeholders believed that an open and transparent approach would be incomplete without the power and influence to implement the plan and to put it into practice. one of the interviewees for example said: “it’s all up to the committee to make this strategic plan effective. strategic plan is a document until an action or actions are built on those strategies and the timelines on how to achieve those. people at the top level are those who can do this.” another interviewee, in the first case, mentioned that: “this planning approach can be effective only when you see managers using these strategy ideas. they have the power to decide about the organisational strategies and we have not seen any decision other than manager’s [decision], therefore, i can’t see any transparency in the strategic planning process.” these excerpts indicate that without sharing decision making power, the transparency of strategic planning won’t be perceived to improve the effectiveness of strategic planning. the reason given is that some stakeholders believed that transparency is only limited to the planning phase in their organisation and subsequently their voice won’t be heard after planning or during the implementation of the plan. for this reason, transparency of planning is believed to have little impact on effectiveness when power is not shared within the organisation. 5.3 inclusiveness and strategic planning effectiveness the second characteristic of osp developed during open and axial coding was inclusiveness. although inclusiveness was suggested as an important principle in the literature, the results of our study indicate that a call for inclusion is not enough to impact strategic planning effectiveness. in this study, three codes were recognised to moderate the relationship between inclusiveness and effectiveness. 5.3.1 awareness stakeholders’ awareness about the concept of open strategy, its potential benefits, and details about the practice of this approach in the organisation was regarded as an important factor influencing the way stakeholders perceive the impact of inclusiveness on the effectiveness of osp. awareness is highlighted by a developer in the first case: “it will take time for people to understand this approach and after that it may help the planning process in time. the benefits of this approach are less perceived at the moment but the long-term benefits will be understood in the future.” australasian journal of information systems amrollahi & rowlands 2019, vol 23, research article openness and the effectiveness of strategic planning 15 it appeared that stakeholders’ awareness about the project can be improved through improving the quality of communication and increasing the number of communication channels with designated stakeholders. using communication channels other than electronic communication was suggested in the first case where most of the communication with stakeholders was performed through the strategic planning system and their email addresses: “there was no one to follow up this and ask people to submit their ideas. i received a couple of emails but if you could meet people in person and ask them to participate, people would feel more committed to visit the website and submit their ideas. i feel that it was not identified as a duty for me.” to address this type of comment, the communication approach was revised in the second case and more attention was paid to face-to-face communication (including several workshops). increased communication resulted in minimising the number of comments about awareness. a similar excerpt was coded in the interviews in the second case: “maybe the only way to do that was to devote a lot more time going to talk to individual classes and become a much more one on one experience but again that takes a lot of time and manpower and right kind of manpower.” the importance of quality communication for participants indicates how informing stakeholders about the osp concept and practice can impact their perception regarding participation and effectiveness. 5.3.2 diversity this study found diversity of participants as an important factor influencing the impact of inclusiveness on osp effectiveness. our definition of diversity goes beyond simple demographic diversity, as we focus on the concept of “cognitive diversity” meaning the difference in beliefs and preferences by various stakeholders in the organisation (miller, burke, and glick (1998)). while studying the effectiveness of osp, interviewees mentioned that many aspects were not covered in the strategic plan. for example, one of the interviewees mentioned that: “the user viewpoint is not highlighted in this plan. this organisation is developing a product but our developers usually look at the supply side and ignore the demand side. and while in the planning (for any reason) developers have participated more than users, now the plan has a tendency toward the development process and has ignored mechanisms for attracting users.” the impact of diversity was perceived to be important as a result of paying attention to certain groups of stakeholders. contacting all groups of stakeholders and persuading them to participate in the strategy process was however a big challenge for the strategy development team that could be dealt with by better communication. 5.3.3 motivation according to the excerpts in our study, the motivation of stakeholders can impact the perceived effectiveness of inclusion in strategic planning. it was evident that many stakeholders were not motivated to participate in strategic planning as one of the interviewees in the second case mentioned: “i think a lot of members come specifically to do that course and they are not worried about what is going on outside the classroom.” australasian journal of information systems amrollahi & rowlands 2019, vol 23, research article openness and the effectiveness of strategic planning 16 moreover, many stakeholders mentioned that (for different reasons) they thought participation in the strategy process was not part of their duty. for example, one of the developers in the first case mentioned: “i have not been here for so long, and i am not really well informed about the organisation even on technical topics… for this reason, i thought it’s better to let managers decide [about strategic planning].” how to increase stakeholders’ motivation was another important topic identified. intrinsic motivations were suggested to be effective in the context of osp. one of the interviewees mentioned that: “when they asked me to submit my ideas for future directions of the project, i feel myself in managers’ position and it gives me a good feeling that my ideas can be constructive.” research team observations during the planning period also indicated that intrinsic motives such as acknowledging contributors’ names in the plan can better impact the number of participants compared to extrinsic factors such as a prize draw. 5.4 it-enabledness and strategic planning effectiveness the use of it tools for strategic planning was also a focus while studying osp effectiveness. however, according to the context and level of it knowledge in the two cases, different aspects of the it system were highlighted. two factors were retrieved to indicate a moderating relationship between it-enabledness and the perceived effectiveness of osp: 5.4.1 system efficiency the efficiency and user-friendliness of the planning system were highlighted as an important moderating factor. in particular, the first case (software project), where most of the stakeholders were technology-savvy people with high levels of it literacy, these stakeholders were mainly concerned about visual aspects and the user interface of the planning system: “i believe the system was not user-friendly at all! when the user enters the portal, she should understand what is going on and what to do.” in this study, the planning system was more than a simple system for entering strategy ideas. many different activities such as informing participating stakeholders and managing the flow of ideas from submitter to the final plan were conducted through the planning system. for this reason, extra attention should be paid to the development of the user interface. 5.4.2 it literacy in the second case (lifelong education community), age of stakeholders, the level of it literacy and the ability of users to work with an online system were the main concerns. the average age of members, in this case, was 69 while members came from a variety of educational backgrounds and levels. based on this situation, and during the initial stages of developing the plan concerns were raised about those members who may have problems of accessibility to and ability to work with an online system. for this reason, an alternative approach of submitting strategy ideas on paper was considered for the members who were less familiar with online technologies. in spite of this alternative solution, there were still members who felt they had been ignored in the strategy process by having to use an online system: australasian journal of information systems amrollahi & rowlands 2019, vol 23, research article openness and the effectiveness of strategic planning 17 “i think you don’t need to drag everybody to the 21st century. a lot of people are very happy and very proud of the fact that they are not it literate. these people shouldn’t be shut out because they are different.” statements like the above indicate the extreme importance of it literacy in the effectiveness of an osp approach. in other words, ignoring demographic and cognitive diversity in it literacy may cause some stakeholders to feel excluded during the planning process. in summary, a number of moderating factors leading to osp effectiveness were identified from the case interviews. table 5 provides a summary of the codes retrieved during data analysis, and the identified moderating factors with sample excerpts supporting each moderating factor. categories measure / moderating factors definition sample of retrieved text strategic planning effectiveness planning efficiency the degree of efficiency as a result of using the osp approach. i believe the collaborative approach can be quicker than conventional approaches of strategic planning. strategic planning effectiveness plan effectiveness the quality of the plan and its content. strategies are attainable and they are aligned with our current policies. we are at the moment doing our best to achieve these goals. strategic planning effectiveness organisational effectiveness the potential impact of a developed strategic plan on the organisation. the planning project has resulted in aligning stakeholders and this will make them involved in the organisation which is absolutely beneficial for us. transparency trust stakeholders’ concerns about the way they believe their participation in osp could impact their organisational position. the reason for the lack of participation was the fact that this idea was coming from somewhere outside the team. transparency power stakeholders’ concerns about whether their ideas will be considered in the decision-making process considering their level in the organisation. when in practice there is no decision other than manager’s [decision], i can’t see any good on this strategic plan. inclusiveness awareness stakeholders’ information about the concept of open strategy and the practice of this approach in the organisation. it will take time for people to understand this approach and after that it may help the planning process in time. inclusiveness diversity cognitive and demographic diversity of stakeholders participating in the osp initiative. i cannot see a single comment in the planning system indicating a user concern. inclusiveness motivation stakeholders’ willingness to take part in the osp project. you’ve got 350 members who didn’t have any contribution and you need to think why you have lost them. australasian journal of information systems amrollahi & rowlands 2019, vol 23, research article openness and the effectiveness of strategic planning 18 categories measure / moderating factors definition sample of retrieved text it-enabledness system efficiency the efficiency and userfriendliness of the planning system. i found it very easy to use. it was quite simple. i was able to access easily and type in my answer and i used it in a number of occasions. so i had no problem. it-enabledness it literacy stakeholders’ knowledge and ability to use the planning system. so, with this background comment, i think the demographic of this club may have been a bit of challenge to do an online survey. table 5 codes retrieved during data analysis; and moderating factors 6 discussion of findings based on the findings presented from our analysis, this section proposes eight propositions and discusses the results by way of reference to the strategic planning and osp literature. it does so by returning to the literature to note consistencies with and departures from findings of earlier research. this process involves asking what it is similar to, what it contradicts, and why. in pursuit of this objective, for each proposition, the paper indicates the extent to which it was supported by previous research and the extent to which the findings have added some new perspective or idea when thinking about the effectiveness of open strategic planning. at this juncture it worth to mentioning that we have identified three different topic areas when discussing our propositions in relation to the available literature. these three topic areas are: (i) strategic planning; (ii) strategic planning effectiveness; and (iii) open strategic planning (osp). considering the research questions and aim, our discussion is primarily focused on the two last two topic areas. 6.1 6.1 the propositions proposition 1. effectiveness of open strategic planning is perceived through (a) its impact on organisational long-term objectives, (b) the quality of developed plan, and (c) the efficiency of the planning process. the various factors impacting the effectiveness of strategic planning have been previously studied in the literature. although the focus of this study is on the moderating factors between the openness and osp effectiveness, comparing the findings with factors leading to strategic planning effectiveness can provide insight regarding the similarities and differences between osp and conventional approaches of strategic planning. it is important once again to emphasise that the focus of this study is on the “perceived effectiveness of strategic planning”. this perspective on strategy effectiveness is leveraged in the literature (bantel, 1993; bolton & leach, 2002; simpson, 1998) and in this study has been employed for the sake of widening the traditional measures of strategic planning effectiveness to those more focused on the principles and characteristics of openness. this approach is also aligned with a practice perspective (the dominant perspective in open strategy (tavakoli et al., 2017)) that necessitates consideration of practitioners (instead of solely focusing on organisational aspects) in evaluating the effectiveness of strategic planning (whittington, 1996). australasian journal of information systems amrollahi & rowlands 2019, vol 23, research article openness and the effectiveness of strategic planning 19 among the three perspectives in this proposition, organisational effectiveness of osp is highlighted in the literature as (i) the ability of osp to result in a continuous rejuvenation of a collective identity (luedicke, husemann, furnari, & ladstaetter, 2017), (ii) increase the legitimacy of decisions that stand on morally ambiguous grounds (luedicke et al., 2017), and (iii) enhanced organisational learning (hutter et al., 2017). also, the osp literature confirms that using an inclusive approach to strategizing indirectly leads to better strategies, improved understanding, and better implementation of the strategies (hutter et al., 2017). therefore, the literature supports our finding suggesting that organisational effectiveness is an indicator of the effectiveness of open strategic planning. perceived success of strategic planning was also studied in the strategic planning effectiveness literature through different factors such as self-assessing effectiveness (griggs, 2002) and planning success (bunning, 1992). moreover, many studies have referred to subjective measures for evaluating the financial effectiveness of the plan (ghobadian, o'regan, thomas, & liu, 2008; green jr & medlin, 2003; suklev & debarliev, 2012) mainly managers are surveyed in these studies whereas a broader range of stakeholders are surveyed in the current paper. focusing on each of the measures identified in this study about strategic planning effectiveness, the literature tends to focus on organisational effectiveness in terms of financial performance (elbanna, 2016; phillips, 1996; rudd, greenley, beatson, & lings, 2008). in contrast, our findings suggest that osp effectiveness is more focused on social aspects such as aligning individual goals with organisational strategies. similarly, the extant literature on conventional strategic planning is less concerned with the practice of strategic planning (plan efficiency as named in our study) as a measure of effectiveness. the few studies that focused on plan efficiency mainly considered the formality of planning (falshaw, glaister, & tatoglu, 2006) and cost (skokan, pawliczek, & piszczur, 2013) with an absence in the literature focusing on agility and responsiveness of the planning process. this study also proposes plan efficiency as a new indicator of effectiveness that is currently under-reported in the literature. plan efficiency deals with the practice of strategy (how the strategy is developed) and considers whether using an open approach for strategic planning can impact time and cost. proposition 2. the transparency of strategy process impacts the perceived effectiveness of strategic planning when there is high level of trust between managers and stakeholders. this study finds trust to play a moderating role on the impact of transparency on strategic planning effectiveness. the theme of trust has been frequently mentioned in the strategy literature as a factor improving collaboration (kumar & paddison, 2000), as a product of ethical strategic planning (hosmer, 1994); and as a prerequisite for inter-organisational strategic planning (lai, lee, & hsu, 2009). although trust has not been directly addressed in the strategic planning literature with regards to the effectiveness of the planning attempt, similar constructs such as attention to legal and ethical principles have been frequently mentioned in the related literature (al-turki, 2011; bunning, 1992; galbreath, 2010; hendrick, 2003). also, trust has been found to impact high-level actions (but not necessarily strategic plans) in organisations including organisational learning (schäffer & willauer, 2003), decision-making effectiveness (petersen, ragatz, & monczka, 2005), and organisational change (morgan & zeffane, 2003). australasian journal of information systems amrollahi & rowlands 2019, vol 23, research article openness and the effectiveness of strategic planning 20 this study suggests building trust among all participating stakeholder groups (planning experts, managers, selected crowd, crowd, and individuals) is an effective approach to increasing the effectiveness of osp. a recent study by birkinshaw (2017) confirms that "common-based production" of strategy content and "collective decision making" about the strategy will lead to a better understanding of the strategy indirectly supporting the moderating impact of trust on osp effectiveness proposed in this research. proposition 3. the transparency of strategy process impacts the perceived effectiveness of strategic planning when stakeholders have a certain level of power when making organisational decisions. prior studies have scarcely examined the impact of power on strategic planning effectiveness. exceptions include, dean jr and sharfman (1996) who studied the strategic decision-making process and found political behaviour (including asserting power) as part of this process to impact the effectiveness of the final decision. other studies have focused on power in terms of giving priorities to a specific group of stakeholders. (cooper & downer, 2012; faehnle & tyrväinen, 2013). our finding on the role of power is also consistent with studies on innovation (napierkowski & parsons, 1995) and change management. for example, fichter (2009) investigated the role of power in the formation of innovation communities and found that certain groups have the power to promote innovation. another study by mathieu, gilson, and ruddy (2006) found evidence that empowering staff mediates the relationship between their involvement and the effectiveness of organisational change. however, recent studies on osp, in general, have focused on power. for example, hautz et al. (2017) mentioned empowerment as a central dilemma of open strategy referring to the fact that although an osp approach can potentially grant more power to stakeholders, it subjects them to the burdens of strategising. hence, it can be inferred that in an organisation without a culture of power sharing, this burden may negatively impact stakeholders’ interpretations about osp effectiveness. mack and szulanski (2017) also identified stakeholders’ power over decision-making as a factor impacting the degree of openness in osp. on the other hand, managers’ loss of power and control over the organisation is mentioned as a potential risk in osp (birkinshaw, 2017; malhotra et al., 2017). managers’ fear about losing their power over strategy and strategic decisions is another factor impacting their interpretation. this concern is reflected in prior literature on open strategy as a challenge when considering strategy as a secret and strategists as elite staff in conventional work (whittington et al., 2011). proposition 4. inclusiveness in strategy process impacts the perceived effectiveness of strategic planning when there is high level of awareness about the strategy process among all stakeholders. stakeholders’ awareness about the strategic planning project and the resultant plan has also been highlighted as an important moderating factor in the strategic planning effectiveness literature (adams, 2005; bunning, 1992; dyson & foster, 1982) with less emphasis recently. in the osp literature, although awareness about the planning project is not explicitly mentioned as a factor impacting effectiveness, a sense of virtual community has been reported australasian journal of information systems amrollahi & rowlands 2019, vol 23, research article openness and the effectiveness of strategic planning 21 to moderate the relationship between participation and organisational sense of community (similar to organisational effectiveness in this study) (hutter et al., 2017). proposition 5. inclusiveness in strategy process impacts the perceived effectiveness of strategic planning when diverse groups of stakeholders participate in strategy formulation. with regards to inclusiveness, this study recommends paying attention to the cognitive diversity of participating stakeholders as well as demographic diversity. diversity is consistent with the strategic planning effectiveness literature where the variety of stakeholders was considered as a success factor for strategic planning (owolabi & makinde, 2012). participation of stakeholders other than managers was also suggested in various studies (adewale & esther, 2012; cooper & downer, 2012; giffords & dina, 2004; hendrick, 2003). the demographic diversity factor has also been mentioned as a potential risk in osp where internal stakeholders are reluctant to implement strategies and ideas submitted by external stakeholders (malhotra et al., 2017). our research contradicts this finding, where we suggest the diversity of participating stakeholders is a factor increasing osp effectiveness. in other words, this study found including stakeholders from diverse groups should be considered as an opportunity in an osp project, as diversity may help the organisation to see the future from a different perspective and may improve a sense of inclusion among stakeholders. proposition 6. inclusiveness in strategy process impacts the perceived effectiveness of strategic planning when stakeholders are motivated to participate in strategy formulation. a number of motivational factors such as resistance to planning (mclarney, 2001) and a will to succeed (wasilewski & motamedi, 2007) have been mentioned to influence strategic planning effectiveness. these factors are similar to the motivation factor identified in this research, as they both refer to a sense of willingness among stakeholders to participate in the planning process and to embrace the outcomes. the osp literature is however relatively silent about the impact of motivating stakeholders. the only exception is the work of hutter et al. (2017) who refer to a sense of virtual community as a motivator of osp effectiveness. proposition 7. using it tools in strategy process impacts the perceived effectiveness of strategic planning when the planning system is considered to be efficient. there is a debate in the literature about the possibility of openness without it (schlagwein, conboy, feller, leimeister, & morgan, 2017). while the management literature is less inclined to consider it as a vital component of open strategy (e.g. hautz et al. (2017)), the is literature considers it as a critical characteristic of open strategy (tavakoli et al., 2015b) forming one of the preconditions of osp (sailer et al., 2017). further to this disagreement, the literature about using it tools in strategic planning is limited; and the impact of it on osp also remains under-studied. however, in the broader is literature, stakeholders’ perception about it infrastructure (in terms of accessibility) is listed as a factor influencing social inclusion (tambulasi, 2009), while gaining feedback from stakeholders about it is suggested as an approach to improve the perception of effectiveness (jaeger, 2006). the proposed system for collaborative strategic planning can be identified as an example of a group decision support system (gdss). therefore, our findings on the efficiency of the it component can be compared with the literature in this area, confirming the same (kim, australasian journal of information systems amrollahi & rowlands 2019, vol 23, research article openness and the effectiveness of strategic planning 22 godbole, huang, panchadhar, & smari, 2004). the results of this study support the importance of it tools in creating a transparent and inclusive strategic discourse while emphasising the importance of the perceived effectiveness of the planning system aligned with the literature on social inclusion. proposition 8. using it tools in a strategy process impacts the perceived effectiveness of strategic planning when the participating stakeholders are it-literate and able to work with the system. the impact of it literacy has been essentially ignored in the strategic planning effectiveness literature. this may be a result of less attention being paid to it tools in the strategy literature (as explained when discussing proposition 7). however, this research claims that by extending the inclusion of stakeholders to wider societies, special consideration should be given to less it literate people. this research is the first to focus on the exclusion factor for stakeholders. by reference to a wider is literature, the use of it tools have been found to widen exclusion in certain contexts (tambulasi, 2009; trauth & howcroft, 2006). although it has been mentioned as a crucial social phenomenon in the osp literature (tavakoli et al., 2017), there is a limited in-depth commentary about the impact of it literacy on the ultimate effectiveness of osp. accordingly, this study recommends a more detailed analysis of the impact of using it in the open strategy making process (see section 6.2). figure 3 graphical representation of open strategic planning effectiveness, including the eight propositions in sum, comparing the findings of this research with the literature indicates that in these two cases of osp some of the factors outlined in the propositions have previously been identified australasian journal of information systems amrollahi & rowlands 2019, vol 23, research article openness and the effectiveness of strategic planning 23 as success factors for strategic planning, even though different titles or terms may have been used. furthermore, individual factors such as trust and awareness, and factors related to itenabledness have attracted little attention in the strategic planning effectiveness literature. figure 3 provides a graphical illustration of the findings represented as codes, moderating factors, and propositions developed from this research. 6.2 implications for research this research presented eight propositions facilitating the participation of stakeholders in a process of strategic planning known as osp. these propositions are based on empirical results and focus on conditions moderating the impact of three characteristics of osp on strategic planning effectiveness. while the topic of osp is relatively new and is lacking studies from a social and political perspective (matzler et al., 2014), this study focused on identifying micro-level conditions relating to effective osp practice. the paper’s main contribution, outlined in figure 3 and in the eight propositions, is a substantive theory of the effectiveness of osp. our study investigated the impact of using it for the purpose of including more stakeholders in the strategic planning process. although the importance of using it tools is a critical dimension of open strategy (morton & amrollahi, 2018; morton, wilson, galliers, & marabelli, 2019); it as a tool to attract stakeholders (sailer et al., 2017; schlagwein et al., 2017) has been widely studied in the open strategy literature. however, to best of our knowledge, it tools have not previously been reported in connection with the effectiveness of osp. future studies may want to focus on this aspect and can benefit from the available theoretical work on topics such as gender (broome, deturk, kristjansdottir, kanata, & ganesan, 2002; trauth, 2013) investigating how the application of it tools can impact the interpretation by less-privileged groups such as aged people or stakeholders with lower levels of it literacy, regarding their inclusion in highlevel organisational forums such as osp. also, the effectiveness of the planning system in isolation or as part of the whole osp ecosystem can be investigated in future work by potentially considering the theoretical work on generic is success factors (delone & mclean, 2003). as this work is an exploratory study, future studies can further develop the propositions in a variety of contexts using osp. for example, while this study documented different interpretations of osp effectiveness (organisational effectiveness, plan effectiveness, and plan efficiency), future studies may benefit from a re-examination of what our interpretation of osp effectiveness means. this can be done by referring to the available theory in the is and strategic management literature. for example, regarding different interpretations and impressions of an osp system, the theory of impression management (dillard, browning, sitkin, & sutcliffe, 2000) could be helpful to investigate the dynamic motivations of humans in their understanding about the effectiveness or success of an osp endeavour as a reality. this theory can further explain in what way stakeholders perceive the openness of strategy making process and how it can impact their understanding about the strategy and its effectiveness. we also advocate a different approach to doing future strategic planning research — one that considers the interaction over time of intentions, context, process, and action around the effectiveness of osp. this research has essentially been a cross-sectional study focussing on identifying a set of moderating factors and a specific outcome (effectiveness), whereas process models aim to understand the sequence of events leading to some result over time. while our research has identified some of the important moderating factors, it has not examined the australasian journal of information systems amrollahi & rowlands 2019, vol 23, research article openness and the effectiveness of strategic planning 24 dynamic set of contextual elements that interact with one another over time leading to osp effectiveness. what needs to be researched next, this paper claims, is the interplay of conditions and process by which stakeholders come to be involved with osp for the first time. process research and methods that track activities over time are needed to fill this gap. previous reviews on open strategy have identified a lack of experimental and longitudinal studies in this area (morton, wilson, galliers, & marabelli, 2017; sailer et al., 2017). the propositions in this research can form a basis for process studies by suggesting components of osp effectiveness and the factors impacting effectiveness. there is a rich literature on organisational and contextual factors that can be referred to as a basis in future studies. we recommend considering forms of involvement (morgan & zeffane, 2003) in organisational change and the impact of such involvement (hussain et al., 2018; mathieu et al., 2006). furthermore, and considering the interrelationship between osp and innovation as a potential area of research, future studies could benefit from and adapt the literature on innovation as a theoretical frame. for example, studies in diffusion of innovation have focused on the outcome of innovation (de vries, bekkers, & tummers, 2016) that could be beneficial for future studies to compare and contrast with osp. the level and type of openness in osp could also be mapped with patterns in the open innovation literature (dahlander & gann, 2010) with respect to the effectiveness of osp. finally, given that the findings of our study mainly focus on the diversity of internal stakeholders, a further avenue of research could be to focus on both internal and external stakeholders (amrollahi, 2018; whittington et al., 2011). diversity of external stakeholders is a promising research topic as it differentiates various forms of openness and further investigates the impacts of these patterns. 6.3 implications for practice this study and the propositions have several implications for practice. first of all, the propositions provide practitioners wanting to implement the notion of osp with insight on how to improve the effectiveness of their project. similarly, osp is expected to produce new opportunities for strategy consultants (whittington et al., 2011) and by considering the propositions will help practitioners to implement an osp approach in a more effective way. in the first case, an imbalance among stakeholders submitting strategic ideas occurred as a result of one group’s better access to and communication with the plan. this finding is reflected in proposition five as diversity. based on this finding, we recommend future practices of osp to consider the diversity of stakeholders (besides the number of participating stakeholders) to accommodate all possible perspectives in developing open strategic plans. this study also found stakeholders’ awareness a significant moderating factor of effectiveness in osp. this finding suggests the importance of quality communication (adams, 2005; spee & jarzabkowski, 2011) with all stakeholders about the anticipated outputs of planning. in the second case, increasing stakeholders’ awareness through improved communication lead to more effective planning and a higher level of inclusiveness in the planning process. we recommend, practitioners pay attention to increasing stakeholders’ awareness about both the planning process and the formulated plan. increasing awareness about the planning process can be done through workshops and/or training material on osp before and during idea submission phase. the developed plan should be introduced to all stakeholders, and implementation plans, and revisions should be prepared with their active participation. australasian journal of information systems amrollahi & rowlands 2019, vol 23, research article openness and the effectiveness of strategic planning 25 recent studies (amrollahi & rowlands, 2016; morton et al., 2016b; tavakoli et al., 2015a) have highlighted the use of it tools as a critical characteristic of open strategy. using an online system is regarded as essential in attracting more stakeholders to participate, as well as providing a trustworthy and anonymous platform for them to submit their ideas. however, this study warns practitioners about the ‘dark side’ of using these technologies for less it literate stakeholders. our findings reveal that people with less it ability and limited access may feel excluded when these technologies are used for strategic planning and feeling excluded can negatively affect their perception about the effectiveness of the approach. designing an “easy to use” planning system and providing stakeholders with alternative tools for submitting their ideas e.g. on paper or through participation in focus groups and other types of facilitated meetings may help in preventing a feeling of exclusion. lastly, this study found conflicting viewpoints about what is required as part of the plan and different ways the plan was interpreted. as the content of the plan and its interpretation are suggested as key elements of effectiveness (as reflected in the propositions), practitioners should first consider developing a shared understanding of the strategic plan among different groups of stakeholders. approaches such as benchmarking of similar plans prior to the idea submission phase, post development workshops, and implementation focus groups could help to facilitate a common understanding. following these suggestions may result in stakeholders forming a positive interpretation about the project and potential benefits of an osp approach. 7 conclusion using an exploratory approach helped this study uncover various interpretations about the effectiveness of osp, at the same time recognising moderating factors that lead to effectiveness. as stated in the introduction, a lack of established theoretical work about the effectiveness of osp necessitated the generation of new propositions, new perspectives and empirical insights adding to an existing body of knowledge in this arena. as the characteristics of open strategy have been studied in previous research (leonhard dobusch & kapeller, 2013; stieger et al., 2012), this study went further and extends previous research by focusing on moderating factors strengthening the impact of the so-called three open strategy characteristics. while stakeholders have a critical role in new paradigms of strategy, including strategy as practice and open strategy, it is crucial to understand stakeholders’ interpretation about the effectiveness of strategic planning. based on an available review of past literature, the outcome measures of osp have mostly come from firm-specific metrics such as increased loyalty and an improved image for the firm (sailer et al., 2017). however, this research comes from the perspective of stakeholders involved in osp. this perspective assisted the research to better explore open strategy from the perspective of those who created the plan and from those that implemented it. in conclusion, this exploratory research achieved its goals by identifying how stakeholders interpreted the effectiveness of an osp approach by identifying moderating factors linking the openness with effectiveness through the generation of eight propositions. australasian journal of information systems amrollahi & rowlands 2019, vol 23, research article openness and the effectiveness of strategic planning 26 references adams, j. (2005). successful strategic planning: creating clarity. journal of healthcare information management, 19(3), 24-31. adewale, a. r., & esther, m. m. (2012). the relationship between stakeholders’ involvement in strategic planning and organisation’s performance–a study of the university of venda. international business & economics research journal (iber), 11(11), 1175-1190. aguinis, h., edwards, j. r., & bradley, k. j. (2017). improving our understanding of moderation and mediation in strategic management research. organizational research methods, 20(4), 665-685. al-turki, u. (2011). a framework for strategic planning in maintenance. journal of quality in maintenance engineering, 17(2), 150-162. aldehayyat, j., & al khattab, a. (2012). strategic planning and organisational effectiveness in jordanian hotels. international journal of business and management, 8(1), 11-25. allen, m. (2017). the sage encyclopedia of communication research methods: sage publications. amrollahi, a. (2018). how open has the practice of strategic planning been?: a qualitative meta-analysis of case studies. in handbook of research on strategic innovation management for improved competitive advantage (pp. 359-382): igi global. amrollahi, a., ghapanchi, a., & talaei-khoei, a. (2014). using crowdsourcing tools for implementing open strategy: a case study in education. paper presented at the twentieth americas conference on information system (amcis 2014), savannah, usa. amrollahi, a., & ghapanchi, a. h. (2016). open strategic planning in universities: a case study. paper presented at the 49th hawaii international conference on system sciences (hicss), koloa, usa. amrollahi, a., najafikhah, s., morton, j., & sarooghi, h. (2019). does opening strategy in organizations lead to effectiveness and success? paper presented at the proceedings of the twenty-seventh european conference on information systems (ecis). amrollahi, a., & rowlands, b. (2016). ospm: a design methodology for open strategic planning. paper presented at the twenty-second americas conference on information systems (amcis 2016), san diego, usa. amrollahi, a., & rowlands, b. (2017). collaborative open strategic planning: a method and case study. information technology & people, 30(4), 832-852. amrollahi, a., & rowlands, b. (2018). ospm: a design methodology for open strategic planning. information & management, 55(6), 667-685. aten, k., & thomas, g. f. (2016). crowdsourcing strategizing communication technology affordances and the communicative constitution of organizational strategy. international journal of business communication, 53(2), 148-180. bantel, k. a. (1993). top team, environment, and performance effects on strategic planning formality. group & organization management, 18(4), 436-458. baskerville, r. l. (1999). investigating information systems with action research. communications of the association for information systems, 2(1), 19. australasian journal of information systems amrollahi & rowlands 2019, vol 23, research article openness and the effectiveness of strategic planning 27 berry, f. s., & wechsler, b. (1995). state agencies' experience with strategic planning: findings from a national survey. public administration review, 55(2), 159-168. birkinshaw, j. (2017). reflections on open strategy. long range planning, 3(50), 423-426. bjelland, o. m., & wood, r. c. (2008). an inside view of ibm's' innovation jam'. mit sloan management review, 50(1), 32-40. bolton, n., & leach, s. (2002). strategic planning in local government: a study of organisational impact and effectiveness. local government studies, 28(4), 1-21. boyd, b. k., haynes, k. t., hitt, m. a., bergh, d. d., & ketchen, d. j. (2012). contingency hypotheses in strategic management research use, disuse, or misuse? journal of management, 38(1), 278-313. broome, j., deturk, s., kristjansdottir, e. s., kanata, t., & ganesan, p. (2002). giving voice to diversity: an interactive approach to conflict management and decision-making in culturally diverse work environments. journal of business and management, 8(3), 239. bunning, c. (1992). effective strategic planning in the public sector: some learnings. international journal of public sector management, 5(4), 54-59. capon, n., farley, j. u., & hulbert, j. m. (1994). strategic planning and financial performance: more evidence. journal of management studies, 31(1), 105-110. carte, t. a., & russell, c. j. (2003). in pursuit of moderation: nine common errors and their solutions. mis quarterly, 27(3), 479-501. chesbrough, h. (2006). open innovation: a new paradigm for understanding industrial innovation. in h. chesbrough, w. vanhaverbeke, & j. west (eds.), open innovation: researching a new paradigm (pp. 1-12). chesbrough, h., & appleyard, m. (2007). open innovation and strategy. california management review, 50(1), 57-76. cooper, t., & downer, p. (2012). stakeholders, strategic planning and critical success factors in professional accounting organizations. journal of leadership, accountability and ethics, 9(4), 82-98. corbin, j. m., & strauss, a. (1990). grounded theory research: procedures, canons, and evaluative criteria. qualitative sociology, 13(1), 3-21. dahlander, l., & gann, d. m. (2010). how open is innovation? research policy, 39(6), 699-709. de vries, h., bekkers, v., & tummers, l. (2016). innovation in the public sector: a systematic review and future research agenda. public administration, 94(1), 146-166. dean jr, j. w., & sharfman, m. p. (1996). does decision process matter? a study of strategic decision-making effectiveness. academy of management journal, 39(2), 368-392. delone, w. h., & mclean, e. r. (2003). the delone and mclean model of information systems success: a ten-year update. journal of management information systems, 19(4), 9-30. dillard, c., browning, l. d., sitkin, s. b., & sutcliffe, k. m. (2000). impression management and the use of procedures at the ritz-carlton: moral standards and dramaturgical discipline. communication studies, 51(4), 404-414. australasian journal of information systems amrollahi & rowlands 2019, vol 23, research article openness and the effectiveness of strategic planning 28 dobusch, l. (2012). strategy as a practice of thousands: the case of wikimedia. paper presented at the academy of management proceedings, boston, usa. dobusch, l., & kapeller, j. (2013). open strategy between crowd and community: lessons from wikimedia and creative commons. paper presented at the academy of management proceedings. dobusch, l., kremser, w., seidl, d., & werle, f. (2017). a communication perspective on open strategy and open innovation. managementforschung, 27(1), 5-25. dobusch, l., & müller-seitz, g. (2015). closing open strategy: strategy as a practice of thousands in the case of wikimedia. unpublished manuscript. dyson, r., & foster, m. (1982). the relationship of participation and effectiveness in strategic planning. strategic management journal, 3(1), 77-88. elbanna, s. (2016). managers' autonomy, strategic control, organizational politics and strategic planning effectiveness: an empirical investigation into missing links in the hotel sector. tourism management, 52, 210-220. faehnle, m., & tyrväinen, l. (2013). a framework for evaluating and designing collaborative planning. land use policy, 34, 332-341. falshaw, j. r., glaister, k. w., & tatoglu, e. (2006). evidence on formal strategic planning and company performance. management decision, 44(1), 9-30. fichter, k. (2009). innovation communities: the role of networks of promotors in open innovation. r&d management, 39(4), 357-371. friis, o. u. (2015). openness in strategy: social strategy practices and practitioner interaction-an opportunity and threat in strategy making. paper presented at the the 31st egos colloquium'organizations and the examined life. galbreath, j. (2010). drivers of corporate social responsibility: the role of formal strategic planning and firm culture. british journal of management, 21(2), 511-525. gasson, s. (2004). rigor in grounded theory research: an interpretive perspective on generating theory from qualitative field studies. in the handbook of information systems research (pp. 79-102): igi global. gasson, s. (2009). employing a grounded theory approach for mis research. in handbook of research on contemporary theoretical models in information systems (pp. 34-56): igi global. gegenhuber, t., & dobusch, l. (2017). making an impression through openness: how open strategy-making practices change in the evolution of new ventures. long range planning, 50(3), 337-354. ghobadian, a., o'regan, n., thomas, h., & liu, j. (2008). formal strategic planning, operating environment, size, sector and performance: evidence from the uk’s manufacturing smes. journal of general management, 34(2), 1-20. giffords, e. d., & dina, r. p. (2004). strategic planning in nonprofit organizations: continuous quality performance improvement-a case study. international journal of organization theory and behavior, 7(1), 66-80. australasian journal of information systems amrollahi & rowlands 2019, vol 23, research article openness and the effectiveness of strategic planning 29 golsorkhi, d., rouleau, l., seidl, d., & vaara, e. (2010). introduction: what is strategy as practice. in cambridge handbook of strategy as practice (pp. 1-20). green jr, k. w., & medlin, b. (2003). the strategic planning process: the link between mission statement and organizational performance. academy of strategic management journal, 2, 23. griggs, h. (2002). strategic planning system characteristics and organisational effectiveness in australian small-scale firms. irish journal management, 23(1), 23-51. haefliger, s., monteiro, e., foray, d., & von krogh, g. (2011). social software and strategy. long range planning, 44(5), 297-316. hamel, g. (1996). strategy as revolution. harvard business review, 74(4), 69-82. hautz, j., seidl, d., & whittington, r. (2017). open strategy: dimensions, dilemmas, dynamics. long range planning, 50(3), 298-309. hendrick, r. (2003). strategic planning environment, process, and performance in public agencies: a comparative study of departments in milwaukee. journal of public administration research and theory, 13(4), 491-519. hosmer, l. t. (1994). strategic planning as if ethics mattered. strategic management journal, 15(s2), 17-34. hussain, s. t., lei, s., akram, t., haider, m. j., hussain, s. h., & ali, m. (2018). kurt lewin's change model: a critical review of the role of leadership and employee involvement in organizational change. journal of innovation & knowledge, 3(3), 123-127. hutter, k., nketia, b. a., & füller, j. (2017). falling short with participation—different effects of ideation, commenting, and evaluating behavior on open strategizing. long range planning, 50(3), 355-370. jaeger, p. t. (2006). telecommunications policy and individuals with disabilities: issues of accessibility and social inclusion in the policy and research agenda. telecommunications policy, 30(2), 112-124. jarzabkowski, p., & balogun, j. (2009). the practice and process of delivering integration through strategic planning. journal of management studies, 46(8), 1255-1288. jarzabkowski, p., & paul spee, a. (2009). strategy-as-practice: a review and future directions for the field. international journal of management reviews, 11(1), 69-95. kalkan, a., & bozkurt, ö. ç. (2013). the choice and use of strategic planning tools and techniques in turkish smes according to attitudes of executives. procedia-social and behavioral sciences, 99(6), 1016-1025. kargar, j. (1996). strategic planning system characteristics and planning effectiveness in small mature firms. midatlantic journal of business, 32(1), 19-34. kendall, s. l., nino, m. h., & staley, s. m. (2008). collaborative strategic planning: a wiki application. journal of web librarianship, 2(1), 3-23. ketchen, d. j., boyd, b. k., & bergh, d. d. (2008). research methodology in strategic management past accomplishments and future challenges. organizational research methods, 11(4), 643-658. australasian journal of information systems amrollahi & rowlands 2019, vol 23, research article openness and the effectiveness of strategic planning 30 kim, s.-y., godbole, a., huang, r., panchadhar, r., & smari, w. w. (2004). toward an integrated human-centered knowledge-based collaborative decision making system. paper presented at the proceedings of the 2004 ieee international conference on information reuse and integration, 2004. iri 2004. kumar, a., & paddison, r. (2000). trust and collaborative planning theory: the case of the scottish planning system. international planning studies, 5(2), 205-223. lai, j.-m., lee, g.-g., & hsu, w.-l. (2009). the influence of partner's trust-commitment relationship on electronic commerce strategic planning. management decision, 47(3), 491507. langley, a. (1988). the roles of formal strategic planning. long range planning, 21(3), 40-50. liinamaa, k., nuutinen, j. a., sutinen, e., & vanharanta, h. (2004). collaborative strategic planning on-line. psychnology journal, 2(2), 242-254. lind, m., & goldkuhl, g. (2006). how to develop a multi-grounded theory: the evolution of a business process theory. australasian journal of information systems, 13(2). luedicke, m. k., husemann, k. c., furnari, s., & ladstaetter, f. (2017). radically open strategizing: how the premium cola collective takes open strategy to the extreme. long range planning, 50(3), 371-384. mack, d. z., & szulanski, g. (2017). opening up: how centralization affects participation and inclusion in strategy making. long range planning, 50(3), 385-396. malhotra, a., majchrzak, a., & niemiec, r. m. (2017). using public crowds for open strategy formulation: mitigating the risks of knowledge gaps. long range planning, 50(3), 397-410. mantere, s., & vaara, e. (2008). on the problem of participation in strategy: a critical discursive perspective. organization science, 19(2), 341-358. martin, p. y., & turner, b. a. (1986). grounded theory and organizational research. the journal of applied behavioral science, 22(2), 141-157. mathieu, j. e., gilson, l. l., & ruddy, t. m. (2006). empowerment and team effectiveness: an empirical test of an integrated model. journal of applied psychology, 91(1), 97. matzler, k., füller, j., koch, b., hautz, j., & hutter, k. (2014). open strategy–a new strategy paradigm? in strategie und leadership (pp. 37-55): springer. mclarney, c. (2001). strategic planning-effectiveness-environment linkage: a case study. management decision, 39(10), 809-817. miller, c. c., burke, l. m., & glick, w. h. (1998). cognitive diversity among upper-echelon executives: implications for strategic decision processes. strategic management journal, 19(1), 39-58. mingers, j. (2001). combining is research methods: towards a pluralist methodology. information systems research, 12(3), 240-259. mintzberg, h. (1994a). the fall and rise of strategic planning. harvard business review, 72(1), 107-114. mintzberg, h. (1994b). rethinking strategic planning part i: pitfalls and fallacies. long range planning, 27(3), 12-21. australasian journal of information systems amrollahi & rowlands 2019, vol 23, research article openness and the effectiveness of strategic planning 31 mitroff, i. i., barabba, v. p., & kilmann, r. h. (1977). the application of behavioral and philosophical technologies to strategic planning: a case study of a large federal agency. management science, 24(1), 44-58. morgan, d., & zeffane, r. (2003). employee involvement, organizational change and trust in management. international journal of human resource management, 14(1), 55-75. morton, j., & amrollahi, a. (2018). open strategy intermediaries? a framework for understanding the role of facilitators in open strategy-making. paper presented at the proceedings of the twenty-sixth european conference on information systems (ecis). morton, j., wilson, a., & cooke, l. (2015a). collaboration and knowledge sharing in open strategy initiatives. paper presented at the ifutures 2015, sheffield, uk. morton, j., wilson, a., & cooke, l. (2015b). opening strategy through'jamming': exploring the process. paper presented at the loughborough school of business and economics (sbe) doctoral conference, loughborough university, uk. morton, j., wilson, a., & cooke, l. (2016a). exploring the roles of external facilitators in it-driven open strategizing. paper presented at the 12th international symposium on open collaboration (opensym 2016), berlin, germany. morton, j., wilson, a., & cooke, l. (2016b). open strategy initiatives: open, it-enabled episodes of strategic practice. paper presented at the twentieth pacific asia conference on information systems (pacis 2016), chiayi, taiwan. morton, j., wilson, a., galliers, r. d., & marabelli, m. (2019). open strategy and information technology. in d. seidl, r. whittington, & g. von krogh (eds.), cambridge handbook of open strategy: cambridge. morton, j., wilson, a. d., galliers, r. d., & marabelli, m. (2017). open strategy and it: a review and research agenda. paper presented at the 33rd egos colloquium, copenhagen, denmark. myers, m. d. (1997). qualitative research in information systems. mis quarterly, 21(2), 241242. napierkowski, c. m., & parsons, r. d. (1995). diffusion of innovation: implementing changes in school counselor roles and functions. the school counselor, 42(5), 364-369. neuman, l. w. (2011). social research methods (7th ed.): pearson education india. newstead, b., & lanzerotti, l. (2010). can you open-source your strategy. harvard business review, october, 32(1), 32. owolabi, s. a., & makinde, o. g. (2012). the effect of strategic planning on the corporate performance in university education: a study of babcock university. kuwait chapter of arabian journal of business and management review, 2(4), 27-44. petersen, k. j., ragatz, g. l., & monczka, r. m. (2005). an examination of collaborative planning effectiveness and supply chain performance. journal of supply chain management, 41(2), 14-25. petrini, m., & pozzebon, m. (2009). managing sustainability with the support of business intelligence: integrating socio-environmental indicators and organisational context. the journal of strategic information systems, 18(4), 178-191. australasian journal of information systems amrollahi & rowlands 2019, vol 23, research article openness and the effectiveness of strategic planning 32 phillips, p. a. (1996). strategic planning and business performance in the quoted uk hotel sector: results of an exploratory study. international journal of hospitality management, 15(4), 347-362. phillips, p. a., & moutinho, l. (1999). measuring strategic planning effectiveness in hotels. international journal of contemporary hospitality management, 11(7), 349-358. phillips, p. a., & moutinho, l. (2000). the strategic planning index: a tool for measuring strategic planning effectiveness. journal of travel research, 38(4), 369-379. reid, m. f., brown, l., mcnerney, d., & j. perri, d. (2014). time to raise the bar on nonprofit strategic planning and implementation. strategy & leadership, 42(3), 31-39. rigby, d. (2001). management tools and techniques: a survey. california management review, 43(2), 139-160. rigby, d., & bilodeau, b. (2013). management tools & trends 2013. robinson, r. b. (1982). the importance of “outsiders” in small firm strategic planning. academy of management journal, 25(1), 80-93. rowlands, b. (2005). grounded theorising applied to is research-developing a coding strategy. australasian journal of information systems, 12(2). rudd, j. m., greenley, g. e., beatson, a. t., & lings, i. n. (2008). strategic planning and performance: extending the debate. journal of business research, 61(2), 99-108. sailer, a.-s., schlagwein, d., & schoder, d. (2017). open strategy: state of the art review and research agenda. paper presented at the thirty eighth international conference on information systems, south korea. schäffer, u., & willauer, b. (2003). strategic planning as a learning process. schmalenbach business review, 55(2), 86-107. schlagwein, d., conboy, k., feller, j., leimeister, j. m., & morgan, l. (2017). “openness” with and without information technology: a framework and a brief history. in: sage publications sage uk: london, england. segars, a. h., grover, v., & teng, j. t. c. (1998). strategic information systems planning: planning system dimensions, internal coalignment, and implications for planning effectiveness. decision sciences, 29(2), 303-341. shiau, w.-l., & george, j. f. (2014). a grounded theory approach to information technology adoption. communications of the association for information systems, 34(1), 1379-1407. simpson, d. g. (1998). why most strategic planning is a waste of time and what you can do about it. long range planning, 31(3), 476-480. skokan, k., pawliczek, a., & piszczur, r. (2013). strategic planning and business performance of micro, small and medium-sized enterprises. journal of competitiveness, 5(4), 57-72. smith, h. j., & keil, m. (2003). the reluctance to report bad news on troubled software projects: a theoretical model. information systems journal, 13(1), 69-95. spee, a. p., & jarzabkowski, p. (2011). strategic planning as communicative process. organization studies, 32(9), 1217-1245. australasian journal of information systems amrollahi & rowlands 2019, vol 23, research article openness and the effectiveness of strategic planning 33 stieger, d., matzler, k., chatterjee, s., & ladstaetter-fussenegger, f. (2012). democratizing strategy: how crowdsourcing can be used for strategy dialogues. california management review, 54(4), 44-68. strauss, a., & corbin, j. (1998). basics of qualitative research techniques: sage publications, inc. suklev, b., & debarliev, s. (2012). strategic planning effectiveness comparative analysis of the macedonian context. economic and business review, 14(1), 63-93. tackx, k., & verdin, p. (2014). can co-creation lead to better strategy? an exploratory research. universite libre de bruxelles, working papers, 1-26. tambulasi, r. i. (2009). pushed to the abyss of exclusion: ict and social exclusion in developing countries. journal of information, communication and ethics in society, 7(2/3), 119-127. tavakoli, a., schlagwein, d., & schoder, d. (2015a). open strategy: consolidated definition and processual conceptualization. paper presented at the thirty-sixth international conference on information systems, fort worth, usa. tavakoli, a., schlagwein, d., & schoder, d. (2015b). open strategy: towards a shared understanding. paper presented at the developmental workshop on open research and practice in is, münster, germany. tavakoli, a., schlagwein, d., & schoder, d. (2017). open strategy: literature review, reanalysis of cases and conceptualisation as a practice. the journal of strategic information systems, 26(3), 163-184. trauth, e. m. (2013). the role of theory in gender and information systems research. information and organization, 23(4), 277-293. trauth, e. m., & howcroft, d. (2006). social inclusion and the information systems field: why now? in social inclusion: societal and organizational implications for information systems (pp. 3-12): springer. urquhart, c., lehmann, h., & myers, m. d. (2010). putting the ‘theory’back into grounded theory: guidelines for grounded theory studies in information systems. information systems journal, 20(4), 357-381. vilà, j., & canales, j. i. (2008). can strategic planning make strategy more relevant and build commitment over time? the case of racc. long range planning, 41(3), 273-290. von krogh, g., & spaeth, s. (2007). the open source software phenomenon: characteristics that promote research. the journal of strategic information systems, 16(3), 236-253. wasilewski, n., & motamedi, k. (2007). insights for effective strategic planning. competition forum, 5(1), 229-235. whittington, r. (1996). strategy as practice. long range planning, 29(5), 731-735. whittington, r., cailluet, l., & yakis-douglas, b. (2011). opening strategy: evolution of a precarious profession. british journal of management, 22(3), 531-544. wolf, c., & floyd, s. w. (2013). strategic planning research toward a theory-driven agenda. journal of management, 1-35. australasian journal of information systems amrollahi & rowlands 2019, vol 23, research article openness and the effectiveness of strategic planning 34 wooldridge, b., schmid, t., & floyd, s. w. (2008). the middle management perspective on strategy process: contributions, synthesis, and future research. journal of management, 34(6), 1190-1221. yin, r. k. (1994). designing single-and multiple-case studies. improving educational management through research and consultancy london, chapman, 135-155. australasian journal of information systems amrollahi & rowlands 2019, vol 23, research article openness and the effectiveness of strategic planning 35 appendix 1. interview questions 1. are you agreeing with this statement: “this approach has helped people to be more involved in the organization”? why and how? 2. how efficient and easy to use did you find the planning system? 3. how do you think this approach decreased the planning cost? 4. how do you think this approach improved the required time for strategic planning? 5. from your perspective, what is the most important business goal for this organization? (examples: customer service, productivity, increase market share, customer satisfaction, customer retention, better competitive position, shareholders satisfaction) do you think this goal is highlighted enough in the developed plan? 6. 6. which strategy in the final plan you think is the most important? from your perspective, is this strategy explained completely? from your perspective, is this strategy achievable in the mentioned time frame? 7. do you think: “this project was generally successful”? why? 8. do you think this plan will improve the performance of the organization? why? copyright: © 2019 amrollahi & rowlands. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v23i0.2093 http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 study aims and scope 3 open strategic planning 4 research method 4.1 the case studies 4.2 data collection 4.3 data analysis 4.4 study rigor 5 results 5.1 strategic planning effectiveness 5.1.1 planning efficiency 5.1.2 plan effectiveness 5.1.3 organisational effectiveness 5.2 transparency and strategic planning effectiveness 5.2.1 trust 5.2.2 power 5.3 inclusiveness and strategic planning effectiveness 5.3.1 awareness 5.3.2 diversity 5.3.3 motivation 5.4 it-enabledness and strategic planning effectiveness 5.4.1 system efficiency 5.4.2 it literacy 6 discussion of findings 6.1 6.1 the propositions 6.2 implications for research 6.3 implications for practice 7 conclusion australasian journal of information systems guler et al 2025, vol 29, research article the role of roles:are llms behavioural? 1 received 05 dec 2024, accepted 12 aug 2025 after 1 round of review with wasana bandara as accepting senior editor, michael davern and stuart black as editors in chief the role of roles: are llms behavioural in information systems decision-making? nazmiye guler school of information systems and technology management university of new south wales kensington, sydney nsw, australia michael cahalane school of information systems and technology management university of new south wales kensington, sydney nsw, australia samuel n. kirshner* school of information systems and technology management university of new south wales kensington, sydney nsw, australia email:s.kirshner@unsw.edu.au richard vidgen school of information systems and technology management university of new south wales kensington, sydney nsw, australia abstract large language models (llms) are increasingly embedded in organisational workflows, serving as decision-making tools and proxies for human behaviour as silicon samples. while these models offer significant potential, emerging research highlights concerns about biases in llm-generated outputs, raising questions about their reliability in complex decisionmaking contexts. to explore how llms respond to challenges in information systems (is) scenarios, we examine chatgpt’s decision-making in three experimental tasks from the is literature: identifying phishing threats, making product launch decisions, and managing it projects. crucially, we test the impact of role assignment, a prompt engineering technique, on guiding chatgpt towards behavioural or rational decision approaches. our findings reveal that chatgpt often behaves like human decision-makers when prompted to assume a human role, demonstrating susceptibility to similar biases. however, when instructed to act like ai, chatgpt exhibited greater consistency and reduced susceptibility to behavioural factors. these results suggest that subtle prompt variations can significantly influence decision-making outcomes. this study contributes to the growing literature on llms by demonstrating their dual potential to mirror human behaviour and improve decisionmaking reliability in is contexts, highlighting how llms can enhance efficiency and reliability in organisational decision-making. keywords: large language models, chatgpt, behavioural information systems, ai cognition, prompt-engineering. australasian journal of information systems guler et al 2025, vol 29, research article the role of roles:are llms behavioural? 2 1 introduction advanced information systems (is) integrate artificial intelligence (ai), e.g., natural language processing, image recognition, and large language models (llms)1, into decisionmaking processes (agrawal et al., 2019; autor, 2015; brynjolfsson & mcafee, 2014; chui et al., 2018; dwivedi et al., 2019; jordan & mitchell, 2015; shollo et al., 2022). the emergence of openai's chatgpt, underpinned by generative pre-trained transformer2, (gpt) and other llms, is increasing organisational ai adoption (dwivedi et al., 2023; dell'acqua et al., 2023). llms have problem-solving capabilities across domains, including marketing, education, management, and research (dwivedi et al., 2023; lin, 2023), which has led to a growing interest in their application to replicate human judgment (binz & schulz, 2023; chen et al., 2023; hutson & mastin, 2023). llm system’s ability to mimic human responses has led to inquiries into their applicability as human proxies. early studies show that gpt-3.5’s ethical judgment correlates with human responses (gray, 2023), and that it has the ability to simulate diverse demographic characteristics, facilitating the creation of silicon samples3 that mirror survey responses (argyle et al., 2023). these capabilities can enrich fields (e.g., marketing) and have opened avenues within economics and psychology research, where llms can serve as stand-ins for human subjects (brand et al., 2023; dillion et al., 2023; horton, 2023; sarstedt et al., 2024). such research can inform our understanding of how people react to business decisions, such as targeted marketing campaigns, new product development, and potential organisational policies, at scale. llm's ability to make human-like decisions also implies that it can provide decision support through integration into workflow processes to make automated or semiautomated decisions (chen et al., 2023; dwivedi et al., 2023). llms simulating and substituting human decision-making holds profound implications for is practitioners and researchers. for example, llms can foster a deeper understanding of it usage behaviour by simulating human responses, contributing to responsive, human-centric is (dwivedi et al., 2023). yet, these capabilities rely on llms being able to adequately proxy human behaviour and decision-making. although llms can behave like people, it depends on the context (binz & schulz, 2023; chen et al., 2024). indeed, a review of studies examining llms against prototypical human behaviours reveals mixed findings on the efficacy of silicon samples to produce human-like judgement (sarstedt et al., 2024). a key conclusion from this research was that the effectiveness of silicon samples depends on the domain. while studies have examined llms’ ability to be silicon samples in areas like politics (argyle et al., 2023), psychology (binz & schultz, 2023), marketing (sarstedt et al., 2024), 1 an llm is an advanced ai model trained on extensive text datasets to interpret and generate human language. utilising deep learning, llms can handle complex language tasks like text generation, translation, and questionanswering. in this paper, specific experiments using openai's models are referred to by their exact names (gpt3.5 or gpt-4) for precision, while general discussions about ai technology use the term 'large language models' or 'llms'. 2 gpt-3, launched in 2020 with 175 billion parameters, and gpt-4, released in 2023 with an undisclosed but larger number of parameters, are advanced ai language models by openai, featuring progressive enhancements in language understanding and generation. chatgpt is a specific application optimised for conversational responses and fine-tuned with supervised learning and human feedback. 3 silicon samples are synthetic datasets that “seek to mimic human respondents to describe, explain, and predict human behaviour” (sarstedt et al., 2024; p. 1254). australasian journal of information systems guler et al 2025, vol 29, research article the role of roles:are llms behavioural? 3 services (bickley et al., 2025), tourism (viglia et al., 2024), and operations management (chen et al., 2024), researcher have yet to examine silicon samples in behavioural is research. as human decision-making in is contexts is suboptimal due to the proliferation of decision biases (østerlund et al., 2021; raghavan et al., 2020; rahwan, 2018; rahwan et al., 2019), whether llms behave like people within is contexts is unknown. contrasting psychology, politics, or marketing, where the behavioural fidelity of llms is a prerequisite for testing theories about people, use cases in is include both decision support and behavioural realism. this raises a fundamental question: if llms behave like people in organisational is contexts, are they still effective as decision-support tools? on the surface, this observation suggests that using llms as human proxies may either offer behavioural insights into human decision-making in is systems or reduce the influence of human biases by serving as more consistent decision-makers, providing advantages for decision support. while prior research has explored the ability of llms to mimic human behaviour (sarstedt et al., 2024) or serve as a tool for optimal decision-making (li et al., 2023), little is known about how prompt-engineered role assignments affect their decision-making in is contexts. while is research typically conceptualises ai systems as an adaptive processes that learns and evolves (e.g., herath et al., 2025), llms do not share this property post-training. once trained on vast datasets and fine-tuned through human-based reinforcement learning, llms are fixed and rely entirely on prompt structure to shape their outputs. this distinction introduces a nuanced departure from prevailing assumptions in is research. we test if llms can exhibit distinct decision styles when prompted to act as human-like or machine-like agents, thereby linking debates in silicon sampling to core is concerns about decision reliability and bias. put differently, we examine whether these two seemingly conflicting is goals can be managed through prompt engineering, which focuses on assigning llm roles. by conducting experiments centered on gpt’s response within is contexts, we uncover how role-specific prompts (whether the role is human or ai) impact behaviour and recommendations. in doing so, we explore ‘the role of roles’ in is decision-making with llms by investigating the research questions: rq1: to what extent can gpt replicate the decision-making behaviours of humans in is contexts when prompted to assume the role of a human? rq2: to what extent does gpt improve decision-making compared to humans in is contexts when prompted to assume the role of ai? to address these questions, we use a series of experiments with gpt acting as participants (binz & schulz, 2023) and examine the impact of roles on decision-making efficacy. we investigate three specific is contexts: 1) the influence of behavioural factors in emails on phishing detection, 2) the role of perspectivetaking in deciding whether to launch a potentially flawed software product and 3) how the deaf effect affects project management decisions in software development. 2 study background 2.1 llms as silicon samples the transformative capabilities of llms are influencing behavioural analysis, market research, and organisational decision-making. this evolution reflects ai’s applications in understanding and emulating human judgment and behaviour. this ability is leading researchers to examine whether gpt can indeed be used to replicate human judgement. for example, gray (2023) used gpt-3.5 to judge the ethics of 464 scenarios and found that responses were nearly identical to human responses. argyle et al. (2023) created “silicon australasian journal of information systems guler et al 2025, vol 29, research article the role of roles:are llms behavioural? 4 samples” by assigning gpt-3 characteristics such as age, gender, race, education, and political affiliation, finding that the responses closely matched voter surveys. jiang et al. (2023) prompted gpt-3.5 to take on different combinations of personality traits and found the llms manifested the assigned personalities. hutson and mastin (2023) anticipate that ai systems will soon mimic human behaviour to the extent that “we could have a system capable of seamlessly integrating into any experiment and exhibiting behaviour virtually indistinguishable from that of humans” (p. 123). similar outcomes have been observed in market research studies. brand et al. (2023) found that gpt-3.5 can display fundamental consumer behaviours. for example, the model's responses varied with changing factors like income level and brand loyalty, showing human-like reactions. dillion et al. (2023) also highlight llm’s capabilities to mirror authentic consumer behaviours, such as expressing preferences for particular brands, showing responsiveness to prices, and making decisions influenced by previous purchases. llms can respond to questions about product preferences and generate artificial interviews that provide diverse feedback. this efficiency helps market researchers to quickly gather insights from extensive text data on online platforms (e.g., social media), saving time and resources compared to traditional surveys (argyle et al., 2023; horton, 2023). within economics and psychology, advances in llms provide realistic approaches for simulating decisions at scale. llms are already considered ‘stand-ins’ for human participants in pilot or preliminary experiments, streamlining the experiment design process and saving resources (dillion et al., 2023; horton, 2023). using llms as synthetic participants enables quick turnaround and feedback, effectively a ‘test and learn’, on survey questions or experimental design (argyle et al., 2023). hutson and mastin (2023, p. 123) detail novel research opportunities afforded by llms, noting that: “you could run 1 million bargaining scenarios with a model to identify the factors that most affect behaviour – before launching the study with people” (p. 123). llms also offer opportunities to simulate scenarios that might be ethically or practically challenging with human subjects, allowing researchers to study sensitive topics like social behaviours, ostracism, or the impact of negative feedback (binz & schulz, 2023; dillion et al., 2023). despite their demonstrated capabilities, llms like gpt may also exhibit human biases and decisions that entirely diverge from human behaviour (bender et al., 2021; chen et al., 2023). acknowledging the limitations of ai models, hutson & mastin (2023) details their inability to perfectly mirror human biases or guarantee of response truthfulness. accordingly, their use as human proxies warrants thorough scrutiny and validation (bender et al., 2021; bertino et al., 2020; dillion et al., 2023; marcus, 2020; marcus & davis, 2019; pontrefact, 2023; ribeiro et al., 2020). 2.2 llm research directions in is the findings on whether llms can proxy and replace humans as research participants are exciting and essential lines of inquiry, particularly for fields like marketing. however, this may not be the only pertinent path for is research, as these questions risk overlooking opportunities consistent with the focus of is research: to enhance organisational decisionmaking, efficiency, and effectiveness through methodologically pluralistic explorations of the interplay between information technology and organisational contexts (boell & cecezkecmanovic, 2014; orlikowski & baroudi, 1991). key goals include enhancing organisational australasian journal of information systems guler et al 2025, vol 29, research article the role of roles:are llms behavioural? 5 efficiency and productivity, designing and managing is, and formulating is strategies and policies (avgerou, 2000; dickson & desanctis, 2000; markus & rowe, 2018; orlikowski & iacono, 2001; orlikowski & robey, 1991). in adopting a dual focus on the capability of llms to replicate human judgment to their potential to augment is decision-making, we explore the integration of llms as a cognitive partner in is research, probing its role in decision-making experiments and the resulting implications for organisational practices and research methodologies. an essential focus of llm research in the is context should be improving the integration of llms and ai-based systems, given the central role of human/machine entanglement in is (orlikowski, 2005; scott & orlikowski, 2014). this consideration is consistent with seminal works emphasising the goals of is research. for example, benbasat and zmud (1999) emphasise the relevance of is research in addressing practical, real-world problems, such as improving decision-making processes. orlikowski and iacono (2001) further argue that is research should focus on the role and impact of specific it tools and systems (e.g., llms and ai-based systems). also, lee and baskerville (2003) highlight the need for generalisability in is findings, suggesting that integrating llms and ai systems should work across different contexts and settings. these studies suggest that experimental research leveraging llms to enhance human decision-making aligns directly with the objectives of is research. disciplines like marketing, economics, and psychology (brand et al., 2023; chen et al., 2023; dillion et al., 2023; horton, 2023) are exploring gpt's capacity to emulate human behaviour. the emerging silicon sample literature applies llms to better understand people, whether as consumers in marketing (sarstedt et al., 2024) or citizens in other social sciences (e.g., politics; argyle et al., 2022). in operations, the focus is primarily to understand fundamental biases (e.g., chen et al., 2024). like these fields, behavioural is, which is rooted in behavioural economics and psychology, demonstrates that problem-solving is frequently impaired due to inherent biases and heuristics (arnott & gao, 2022; benbasat & zmud, 1999; hevner et al., 2008; hutson & mastin, 2023)). as behavioural economics represents the prevailing approach for comprehending human is decision-making (arnott & gao, 2022; kahneman, 2011; lenat & marcus, 2023; marcus & davis, 2019; simon, 1995)4, it provides a set of theoretical lens for understanding the decision-making processes of human and ai participants. however, is decision-making moves beyond behavioural biases to study interactions in larger socio-technological systems. whether llm agents can adequately approximate human is behaviours and what prompt engineering techniques can improve performance is unclear. while this question remains open, we also examine if prompting llms to act as a ‘human proxy’ or an ‘ai advisor’ produces distinct decision styles. when framed as human, we expect llms to generate intuitive, heuristic-based responses that reflect biases common in human judgment. in contrast, when framed as ai, we expect more structured outputs aligned with rule-based analysis. this logic draws on dual-process theory (e.g., kahneman, 2011), which distinguishes between fast, intuitive and heuristic-based reasoning (system 1) and slow, deliberative reasoning (system 2). crucially, dual-process theory was developed for human cognition. however, in our study we extend this framework to better incorporate ai to evaluate whether role framing can shift llm decision behaviour. 4 we refer readers to arnott & gao (2022) for a review of the literature on behavioural economics in is. australasian journal of information systems guler et al 2025, vol 29, research article the role of roles:are llms behavioural? 6 3 research design and model this quantitative study aims to investigate the impact of gpt's designated role on decisionmaking outcomes and to explore how these insights might inform the integration of llms to support and enhance decision-making processes. prompt engineering is a key mechanism for communicating and operationalising roles within llms. the specificity of prompts provided to llms directly influences their ability to perform designated roles effectively, shaping the ai’s task contribution and outcomes (wang et al., 2023). given the fragility of human decision-making within is contexts (arnott & gao, 2022; kahneman, 2011; lenat & marcus, 2023; marcus & davis, 2019; simon, 1995)), we posit that prompting gpt to assume the role of ai could yield different and potentially superior outcomes compared to when gpt mimics human decision-makers. following the approach of binz and schulz (2023), we treat chatgpt (models 3.5, 4, and 41106) as participants in a series of experiments. we leveraged appendix b of arnott and gao (2022) review, which lists is articles on behavioural economics from the basket of 8 published between 2014 and 2018, to identify relevant experiments. we selected studies based on suitability for replication (e.g., text-based) and behavioural features that would be interesting to test with chatgpt (i.e., emotions, perspective-taking, and framing). we specifically focused on three decision-making domains, phishing detection, product launch planning, and it project management, because they represent real-world is tasks where llms are being applied or explored. in phishing detection, llms help simulate threats, generate training content, or support detection systems (quinn & thompson, 2024). in product launch planning, they assist in drafting communications, interpreting customer data, or suggesting promotional strategies (paliwal et al., 2024). in it project management, llms are used to support scheduling, documentation, and delivery risk assessment (karnouskos, 2024). these tasks involve both behavioural and analytical judgement, making them well suited for examining how role framing affects decision style. in our three experiments, chatgpt was prompted to assume different roles, simulating either a human persona (e.g., a student, shareholder, or product manager) based on the experimental vignette, or acting as an ai tasked with providing recommendations. the “human” and “ai” roles in our experiments reflect distinct cognitive and normative assumptions about decision-making. we posit that the human role will evoke reasoning patterns associated with intuitive, experience-based judgment, mirroring scenarios where individuals make decisions influenced by context, emotion, and cognitive bias. in contrast, the ai role may prompt gpt to behave more as an analytical decision-support tool, expected to reason consistently and objectively, independent of personal emotion or framing. this conceptual distinction reflects how roles function in actual is contexts: humans can bring emotional or ethical nuance, but are prone to inconsistency; ai systems offer scalability and precision, but may lack sensitivity to social cues or comparable levels of bounded rationality. by prompting gpt to simulate these roles, we capture competing models of reasoning and explore their implications for judgment quality and behavioural fidelity in is applications. therefore, the design aims to analyse chatgpt's responses within the context of the selected experiments, assessing its ability to engage in decision-making, provide recommendations, and simulate human-like behaviour across diverse scenarios. moreover, by manipulating the roles of gpt between a human and ai decision-support system, we identify the extent to australasian journal of information systems guler et al 2025, vol 29, research article the role of roles:are llms behavioural? 7 which roles impact decision-making and how behavioural contexts (e.g., emotions and frames) from the selected experiments interact with gpt’s role. we propose that llm decisions will be more influenced by behavioural factors when the llm assumes a human role than when it explicitly assumes the role of an ai. although this logic draws on dual-process theory, our argument extends beyond its original human-based application to propose a functional analogue in llms. in humans, shift between reasoning styles through internal mechanisms such as cognitive effort and emotional salience formed by biology and experience (system 1) and formal and cultural education (system 2; (arnott & gao, 2022)). in contrast, llms lack these cognitive states and affective experiences. instead, human vs ai role framing operates as an external mechanism for inducing system 1or system 2-like reasoning styles. when prompted to act as a human, gpt may more readily reproduces heuristic-driven, bias-prone behaviour, reflecting the language of intuitive decision-making, enabling responses that are consistent with behavioural theories. when prompted as an ai, its outputs are more structured, consistent, and analytical, aligning with the qualities of system 2, and dampening behavioural factors. thus, gpt’s assigned role moderates the influence of scenario-specific behavioural factors on decision outputs. more broadly, we extend dual-process theory by proposing that gpt’s decision style is not endogenously activated but exogenously framed. this provides researchers with a novel lens for studying the conditional activation of reasoning modes in llms and evaluating their behavioural realism and reliability across decision contexts. figure 1. research design and study details. we examine this hypothesis through three distinct scenarios with unique behavioural and contextual factors influencing is decisions. figure 1 presents our research design and study details. each scenario draws on well-established behavioural theories used in is to study biases. study 1 considers difference in losses and gains (tversky & kahneman, 1981) and contextualisation effects, which explain how message framing and personalisation cues affect risk detection relevant to phishing susceptibility. study 2 incorporates regret theory and the construct of anticipated guilt, to explore how emotional forecasting and perspectivetaking can de-escalate commitment to flawed product launches (lee et al., 2018). study 3 is grounded in research on perceived control, messenger credibility, and the deaf effect, which australasian journal of information systems guler et al 2025, vol 29, research article the role of roles:are llms behavioural? 8 help explain escalation decisions in it project management (nuijten et al., 2016). these behavioural theories have been widely applied in is and organisational decision research to identify judgment biases and suboptimal decision patterns. their inclusion here allows us to test whether llms reproduce or resist such biases under different role framings. to assess our hypothesis and gpt decision-making, each experiment required gpt to evaluate a scenario and provide a response using either a scale (e.g., 0–10 likelihood, 1–7 agreement) or a discrete decision (e.g., proceed or redirect a project). we defined taskspecific optimal responses based on prior literature and domain logic. in the phishing detection study, effectiveness was measured by how closely the gpt rating aligned with whether the email was phishing or legitimate. in the product launch scenario, performance was assessed by whether the model favoured launching a product variant that aligned with the needs of the customer segment described. in the it project management task, effectiveness was determined by whether gpt chose to reallocate resources to avoid escalation, consistent with the original study. these responses served as benchmarks to evaluate decision quality across both the human proxy and ai advisor roles. the three experiments (n = 880) were conducted from october 3, 2023, to november 10, 2023, using openai’s playground. although our focus was on the impact of roles, we also created additional treatments to examine technical aspects of gpt. our first study considered two different temperatures (0 and 1) and two time periods (before and after the november 6 release of gpt-4-1106). study 2 also considered two models (gpt-4 vs gpt-41106). finally, study 3 varied the amount of detail around chatgpt’s role description. 4 experiments and results 4.1 study 1 – got phished? overview. this experiment was based on the article by sanjay goel, kevin williams and ersin dincelli entitled "got phished? internet security and human vulnerability.” goel et al. (2017) investigated student vulnerability to phishing emails, focusing on contextualisation, framing, and motivation. sending fictitious emails to undergraduate students demonstrated that various situational factors can modify the effectiveness of phishing attempts, contributing to information security research. procedure. our study levarges the scenarios and materials of goel et al. (2017). their study consisted of eight email scenarios to assess the influence of frames (gains vs. losses), contextualisation (high vs. low), and motivation (e.g., acquisition vs. social) on phishing susceptibility. see appendix b.1 for the text of each e-mail and a description of the specific features. using openai’s playground chat feature, we provided gpt with the eight e-mail scenarios. we asked it to evaluate the likelihood that the e-mail was a phishing attempt. each prompt had the e-mail and the question, “using a scale of 0-10, from extremely unlikely to extremely likely, how would you rate this email as a phishing attempt?” we manipulated gpt’s role using the system prompt. we instructed chatgpt to either be a student or an llm trained for phishing detection. we also considered different temperature setting in openai's playground controls how predictable or varied the responses from the ai are. a lower temperature results in more consistent and expected answers, while a higher australasian journal of information systems guler et al 2025, vol 29, research article the role of roles:are llms behavioural? 9 settings: 0 (more deterministic) and 1 (more stochastic).5 put differently, the temperature temperature makes the responses more diverse and less predictable. in addition, we collected data in two time periods, before and after november 6th’s release of gpt-4-1106, which saw an overhaul of chatgpt. thus, our experiment had a 2 (role: machine vs human) x 2 (temperature: 0 vs 1) x 2 (date: october 3rd vs november 10th) x 8 (different emails) between-treatment design. we collected 10 observations6 for each e-mail across each treatment, leading to a final dataset of 640 evaluations of phishing likelihoods. figure 2. role-based comparison of overall average responses across all scenarios. results. we first examine figure 2, which presents the phishing ratings. the figure shows that ai tends to exhibit a consistently higher level of caution in terms of phishing evaluations (m = 8.76) compared to students (m = 7.89, p < 0.001). the figure also indicates that gpt as ai has more consistent ratings for each scenario (sd = 0.59) compared to when it is in the student role (sd = 1.95). moreover, the responses were more stable across scenarios when assuming the role of an ai, whereas as a student, the ratings exhibited significantly more variability, with the most notable discrepancy occurring in the registration scenario. (a) frame (b) context 5 in all studies, top-p was set to 1, maximum length was 2000, frequency penalty and presence penalty were 0. 6 gpt can often exhibit near-zero variation in responses in controlled study settings (park et al., 2024; mei et al., 2024), implying that sample sizes can be smaller compared to human-based studies. indeed, our results suggest that n = 10 leads to highly consistent responses with very narrow standard deviations. australasian journal of information systems guler et al 2025, vol 29, research article the role of roles:are llms behavioural? 10 figure 3. effects of frame and context on average response. next, we examined the effects of framing. to clarify, framing here refers to the way information is presented in different contexts to influence decision-making. it involves the manipulation of context or the phrasing of scenarios and questions to see how it affects responses or choices. previously, goel et al. (2017) found that a loss frame and a highly related context increased the propensity of students being susceptible to phishing. figure 3(a) demonstrates that framing impacted the student role but not the ai. similarly, figure 3(b) shows a higher level of contextual relevance increases susceptibility more for gpt assuming the role of a student than as an ai. m1 m2 m3 constant b 9.33 9.14 9.08 se 0.14 0.15 0.16 p <0.001 <0.001 <0.001 role (human = 1) b -0.87 -0.49 -0.37 se 0.12 0.16 0.19 p <0.001 0.003 0.051 framing (loss = 1) b 0.48 0.86 0.48 se 0.18 0.22 0.18 p 0.009 <0.001 0.009 context (high = 1) b -1.36 -1.36 -0.96 se 0.19 0.19 0.22 p <0.001 <0.001 <0.001 role x framing b -0.76 se 0.23 p 0.001 role x context b -0.80 se 0.24 p 0.001 temperature b -0.02 -0.02 -0.02 se 0.12 0.12 0.12 p 0.893 0.892 0.892 gpt version (after nov 6 = 1) b 0.08 0.08 0.08 se 0.12 0.12 0.12 p 0.467 0.464 0.463 notes. b represents the regression coefficients, se represents standard errors, and p refers to the corresponding p-value from t-tests for each coefficient. table 1. results of study 1 regression analysis. we use regression to formally test these observations. the dependent variable is gpt’s phishing evaluations, and the main independent variables of interest are whether gpt’s role is ai, whether the phishing attempt has a high contextual relevance, and whether the e-mail is framed as a loss. our first model (m1) only considers the main effects of the variables. we also include two additional models to examine the interaction effects between gpt’s role and behavioural factors, i.e., framing (m2) and contexts (m3). we also include indicator variables to control for temperature and the run date in all three models. table 1 presents the results of three regression models. the results show that there is a discernible increase in phishing suspicion when gpt is in the role of ai compared to when it is primed to respond as a student. the significant interaction effect between the human role australasian journal of information systems guler et al 2025, vol 29, research article the role of roles:are llms behavioural? 11 and the frame being a loss supports the observations in figure 3(a). similarly, the significant interaction effect between the human role and the context being high supports the observations in figure 3(b). the temperature variable shows a negligible effect on phishing suspicion. similarly, the nov 6th updates to chatgpt did not affect how gpt rated potential phishing emails. see appendix a.1 for more details. 4.2 study 2 – perspective-taking and product launches overview. study 2 was based on hyung koo lee, jong seok lee and mark keil’s research, “perspective-taking to de-escalate launch date commitment for products with known software defects”. lee et al. (2018) suggest that perspective-taking can impact launch decisions for new software. they investigate perspective-taking as a de-escalation tactic to reduce product managers' commitment to the original launch date when faced with potential defects. they found that when participants took the perspective of product users who might be negatively affected by the launch (i.e., victim) of a defective software product, their commitment de-escalated more than when they took a shareholder’s perspective. they found that anticipated guilt about launching a defective product mediated the relationship. procedure. we adopted their study 2 to evaluate if similar effects occur when gpt takes on different roles and perspectives in decision-making scenarios. the system prompt provided the context of the vignette, with subtle modifications to denote different roles. for example, we manipulated gpt to take on the role of a human product manager or an llm decision support system. in scenarios where gpt assumed the role of a human, the prompt stated, “you work for ecomsoft, a company specialising in e-commerce software development.” when portraying gpt as an llm, the prompt was adjusted to reflect this, as in “you are an llm used by ecomsoft for decision support in e-commerce software development.” see appendix b for full details and text of each scenario. using the chat functionality in openai’s playground, we provided gpt with the same questions from the original experiment, relating to whether or not to launch the product, the anticipated guilt associated with the decision and views on customer orientation. the launch and customer orientation were rated on a likert scale from “strongly disagree” (1) to “strongly agree” (7), and the guilt questions were on a scale ranging from “not at all” (1) to “extremely” (7)”. we also ran our study using two different models, gpt-4 and gpt-4-1106 preview. so, our experiment has a 2 (role: ai vs human) x 2 (perspective taking: shareholder vs victim) x 2 (model: gpt-4 vs gpt-4.0 1106 preview) between-subject design. as we collected ten observations per treatment, our final dataset contains 80 observations. as the first set of results demonstrated a negligible impact of temperature on ai's performance, in this and the next study, we maintain a constant temperature setting of 1. results. we started our analysis by examining the mean responses to our key questions on launch and guilt and looked at the interactions between role and perspective. figure 4 shows that different gpt roles (ai vs. human) and perspectives (shareholder vs. victim) influence decision-making in the context of launching software and the level of guilt. similar to study 1, gpt as ai makes more consistent decision-making and is less influenced by contextual factors. australasian journal of information systems guler et al 2025, vol 29, research article the role of roles:are llms behavioural? 12 (a) launch decision across perspective (b) anticipated guilt across perspective figure 4. impact of chatgpt’s role and perspective taking. m1 (guilt) m2 (launch) m3 (launch) constant b 1.66 6.58 6.96 se 0.16 0.14 0.65 p <0.001 <0.001 <0.001 role (ai= 1) b 0.98 -0.28 0.75 se 0.21 0.18 0.16 p <0.001 0.126 <0.001 perspective (shareholder = 1) b 2.78 -2.08 1.10 se 0.21 0.18 0.26 p <0.001 <0.001 <0.001 role x perspective b -1.80 1.19 -0.84 se 0.30 0.25 0.24 p <0.001 <0.001 0.001 guilt b -0.80 se 0.10 p <0.001 gpt model (gpt4 turbo = 1) b 0.38 -0.80 -0.27 se 0.15 0.13 0.14 p 0.014 <0.001 0.047 table 2. results of study 2 regression analysis. we conducted three regressions, presented in table 2, to examine the relationship between perspective and role on the launch decision, while also controlling for the specific gpt model. the dependent variable for the first model (m1) was the average level of anticipatory guilt. the independent variables were indicator variables for the role and perspective treatments. we also included an interaction effect between these treatments to test if the role impacts perspective. the second model (m2) and third model (m3) have the launch decision as the dependent variable. for m2, the independent variables are the same as m1. m3 is the same as m2, but adds in the variable of guilt (i.e., the dependent variable for m1) to examine potential evidence for mediation. this analysis shows a significant interaction effect between taking a shareholder’s perspective and chatgpt’s role as ai on anticipated guilt and the launch decision. specifically, taking a shareholder’s versus victim’s perspective had a australasian journal of information systems guler et al 2025, vol 29, research article the role of roles:are llms behavioural? 13 greater impact on influencing guilt, which in turn mediates the decision to launch the product when gpt acted as a human manager rather than a decision support system. 4.3 study 3 – messenger role influence on ‘deaf effect’ overview. this experiment was based on “collaborative partner or opponent: how the messenger influences the deaf effect in it projects” by arno nuijten, mark keil and harry commandeur. nuijten et al. (2016) investigated factors influencing responses to risk warnings in the context of it project escalation, exploring how the perception of the messenger's role (a trusted partner or opponent) and the framing of perceived control (low or high) affect the likelihood of decision-makers continuing with a high-risk project. they found that the level of control plays a significant role: participants are presented with either a high or low-control manipulation related to the potential outcomes of continuing or redirecting the project. the decision maker’s perceived control over the project moderates the influence of the messenger’s role. specifically, the influence of the messenger role is strengthened when perceived control is high, such that if the messenger is perceived as a collaborative partner rather than an opponent, decision-makers are less likely to turn a deaf ear to the auditor’s risk warning, and this effect will be mediated by perceived risk. procedure. we build on the original experiment, tailoring the scenarios and materials to gpt. our experiment centred on a scenario where participants assumed the role of senior vp at an insurance company, overseeing a project named pension-view. following nuijten et al. (2016), we altered the description of the messenger, mr. smith, from the internal audit department. depending on the scenario, mr. smith was portrayed either as a trusted partner or an opponent who is not to be trusted. this manipulation was designed to explore how varying perceptions of a messenger’s trustworthiness impact decision-making when interacting with an ai system like gpt, particularly in roles where trust and reliability are pivotal. the perceived control was manipulated to be high by stating that the “is are maintained and supported by your own organisation. the owners of these information systems reside at your location, and they directly report to you”. in the low condition, the system prompt included information that the is maintenance and support “has been outsourced to an offshore location in china. the owners of these information systems are located at other departments at various locations. they do not report to you.” the main dependent variable is whether gpt chooses to continue the project as planned. see appendix b.3 for full details and text of each scenario setup. to manipulate gpt’s role when undertaking the experiment, using the system prompt, we told gpt to act as either a senior vice president or an advanced ai system responsible for managing strategic is projects. we also considered the implication of role description detail and included an additional variable, the level of detail in the role description (low or high). the high role description included all the details in the original experiment, whereas the low role details only had the essential information. thus, our experiment has a 2 (role: ai vs human) x 2 (role detail: low vs high) x 2 (messenger role: collaborative vs opponent) x 2 (perceived control frame: low vs high) between-subject design. again, we collected ten observations for each treatment, so our dataset contains 160 decisions. we prompted gpt-4 in openai’s playground, providing the scenario context in the system prompt and asked the two australasian journal of information systems guler et al 2025, vol 29, research article the role of roles:are llms behavioural? 14 dependent variable questions relating to the launch decision and messenger influence using the user prompt. figure 5. interaction effects of messenger's role, chatgpt's role, and perceived control. results. we started by examining the mean responses to whether to continue the project and looked at the interactions between gpt’s role (ai vs human) and the messenger role (opponent vs collaborative partner). figure 5 shows the decision to continue depends on both the auditor (opponent vs collaborative partner), chatgpt’s (human vs ai) role, and the degree of control. similar to human participants, a high level of perceived control leads to continuing the project. however, the impact of the messenger role depends on gpt’s role. when control is high, gpt is more likely to continue the project in the human senior vp role compared to the ai role, and this effect is amplified when the auditor is framed as oppositional. the figure also suggests that control has a larger influence when gpt takes the role of a human. we again formally test these observations using a series of five regression models (m1-m5) with the decision regarding project continuation as the dependent variable in each model. the results, presented in table 3, support that when gpt is assigned the human role, it favours the continuation of the project and that a higher perceived control significantly increases the likelihood of continuing rather than re-directing the project. the interaction effects show that when gpt assumes a human role, and the messenger is perceived as an opponent, there is an increased likelihood of project continuation. this effect suggests that these factors’ combined influence is particularly salient in shaping decision outcomes. similarly, when gpt is in a human role, and there is high perceived control, the likelihood of project continuation increases. this finding highlights the compound effect of role purpose and perceived control in guiding decision-making processes. the results also suggest that the higher the relevance of the messenger’s role, the more likely the project is to be continued. interestingly, this effect is stronger for gpt in the ai role than in the human role. however, this is offset by the substantially higher direct effect of gpt being in the role of a human. finally, we note that the role of detail level in the decision-making process is minimal. the interaction effect of role description detail, perceived control, chatgpt role and mr smith’s influence on the decision are presented in appendix a.2. this analysis shows that the role detail in chatgpt's descriptions impacts decisions for the human role, australasian journal of information systems guler et al 2025, vol 29, research article the role of roles:are llms behavioural? 15 while the decision-making remains stable across different levels of role detail when given the role of an ai. m1 m2 m3 m4 m5 m6 constant b 1.52 1.73 1.94 -1.79 -2.99 -1.87 se 0.10 0.10 0.08 0.72 0.99 0.56 p 0.001 <0.001 0.000 0.014 0.003 0.001 gpt’s role (human = 1) b 0.59 0.18 -0.25 5.90 7.17 4.83 se 0.09 0.12 0.09 0.65 0.97 0.52 p <0.001 0.148 0.005 <0.001 <0.001 <0.001 messenger role (opponent = 1) b 0.088 -0.33 0.09 0.05 0.25 0.17 se 0.09 0.12 0.06 0.08 0.14 0.06 p 0.338 0.008 0.158 0.560 0.076 0.008 percevied control (high = 1) b 3.16 3.16 2.33 2.78 2.79 2.44 se 0.09 0.09 0.09 0.07 0.07 0.06 p <0.001 <0.001 <0.001 <0.001 <0.001 <0.001 gpt’s role x messenger role b 0.83 -0.28 se 0.17 0.16 p <0.001 0.082 gpt’s role x percevied control b 1.68 1.05 se 0.12 0.11 p <0.001 <0.001 messenger relevance b 0.54 0.72 0.58 se 0.11 0.15 0.08 p <0.001 <0.001 <0.001 gpt’s role x messenger relevance b -0.89 -1.07 -0.78 se 0.10 0.14 0.08 p <0.001 <0.001 <0.001 detail level (high = 1) b 0.04 0.04 0.04 0.10 0.09 0.06 se 0.09 0.09 0.06 0.06 0.06 0.04 p 0.681 0.660 0.544 0.085 0.109 0.176 table 3. results of study 3 regression analysis. 5 findings 5.1 overview of main results our research explores how gpt’s recommendations can differ in is contexts based on whether the role offered to gpt is human or ai. in the first study, when chatgpt operated in the ai role, it was less susceptible to phishing attempts and was more consistent and objective in its evaluations compared to when in the human role. in the second study, resonating with the findings of lee et al. (2018), a shareholder perspective increased the propensity to launch the product, whereas a victim's perspective elicits stronger emotions of guilt and remorse, signalling deeper ethical and moral concerns. consistent with study 2, these factors had greater influence when gpt was in the role of an ai. similarly, in the third study, perceived control significantly influenced the decision to continue or redirect a project, but there were strong differences in decisions between ai and human roles. when in the role of ai, gpt was less influenced by the messenger being collaborative or oppositional. taken together, our research reveals advantages of framing llms as ai since it creates greater decision consistency by limiting contextual factors related to biases and australasian journal of information systems guler et al 2025, vol 29, research article the role of roles:are llms behavioural? 16 emotions. thus, the effectiveness of llms in decision support systems may be determined by subtle differences in prompts. 5.2 gpt’s role and behavioural biases when framed as ai, gpt exhibits high levels of consistency and advanced cognitive awareness in problem-solving, enabling it to outperform traditional human decisionmaking. for instance, in the phished study, the ai demonstrated a disciplined approach to evaluating a phishing email when assigned the role of an expert llm. gpt’s analysis was comprehensive, dissecting elements of the email with detail and consistency. for example, it noted the sense of urgency created by some emails, a tactic often employed to coerce recipients into precipitous actions. it highlighted the email's link, noting the need for more association with a legitimate organisation and the absence of standard educational or governmental domain indicators. gpt also critiqued the generic greeting, a typical phishing strategy, due to its lack of personalisation. moreover, it was sceptical of the too-good-to-betrue offer of a significant reward for minimal effort and the conspicuous omission of a sender's email address and verification methods for the survey's legitimacy—essential elements in authentic communications. thus, gpt leverages its knowledge of phishing to make decisions. this thorough analysis starkly contrasts with the shallow approach of a human student role, which did not engage in such an in-depth critique. 5.3 gpt’s role and emotional decision-making another central finding is gpt's diminished susceptibility to emotional manipulation when it is framed as ai compared to when it's given the role of a human (not to mention an actual human, whose judgments are often grounded in intuition). this observation was particularly evident in the perspective-taking study, which scrutinised emotional responses, notably the role of guilt in human decision-making. guilt is a uniquely human emotion that can distort decision-making based on contextual intensity. gpt's decision-making approach, as evidenced in our study, is not subject to such distortions. when gpt assumes the human (victim) perspective, there is an amplification in the representation of guilt, mimicking the complex tone of human emotion with guilt-laden responses. this observation is based on the higher guilt ratings, with all ratings being either 6 or 7, and language used in the responses, such as “significant level of guilt”, “high level of remorse”, and “strong feeling of regret”, suggests a more emotional, guilt-laden reaction to the proposed scenarios. in contrast, the human (shareholder) role exhibits guilt, but the ratings are slightly lower, ranging from 5 to 6. the language used in these responses suggests guilt, but the expressions are less intense, such as “some guilt”, “quite remorseful”, and “feel bad”. while ai roles (victim and shareholder) reflect simulated guilt, they are limited in their emotional response due to their nature as artificial entities. their answers are more focused on ethical responsibility rather than emotional guilt. this amplification is a product of carefully programmed responses designed to simulate human reactions rather than an authentic emotional experience. gpt's mimicry of human-like emotions, despite its incapacity for genuine emotional response, exemplifies its ability to adopt sophisticated human perspectives. such capabilities highlight gpt's utility in decision-making scenarios, especially where understanding human emotional dynamics is pivotal. by simulating these emotions, gpt can provide insights into human behavioural patterns and potential biases, thereby supporting more informed and balanced decision-making processes. australasian journal of information systems guler et al 2025, vol 29, research article the role of roles:are llms behavioural? 17 gpt’s ability to simulate human responses is core to our role-centric model. as an ai, gpt can replicate the behavioural patterns observed in human decision-making. however, its advantage lies in its ability to make decisions free from the heuristic biases of human emotion, like guilt. this aspect of ai decision-making highlights the stability and reliability of gpt’s responses across varied scenarios, underscoring the potential of ai to complement or enhance human decision-making processes in complex environments or scenarios where emotional and personal biases might cloud human judgment. 6 discussion 6.1 implications for research we extend the discussion of ai's potential to proxy and improve on human judgment. our results, which provide results consistent with human behaviour across the considered studies, reveal llms’ potential for providing insights into decision-making within the behavioural is context. this research is the first to provide evidence of silicon samples for the is context. however, as we only consider three studies, further investigation is required to establish the extent to which llms are silicon samples that can reliably replicate precise behavioural patterns in is decision-making. a foundational principle of behavioural economics is the dual-process theory of cognition, which posits that human decision-making operates within two cognitive systems: system 1 and system 2 (stanovich & west, 2000). system 1 is fast, intuitive, and automatic, often relying on heuristics formed through innate instincts and past experiences. while efficient, system 1 is susceptible to cognitive biases, such as overconfidence and anchoring, that impair decision effectiveness (kahneman, 2011). in contrast, system 2 is slow, deliberate, and rule-based, requiring significant cognitive effort to override the intuitive answers proposed by system 1 (kahneman, 2011). behavioural economists have leveraged dualprocess theory to develop interventions, such as nudges, aimed at mitigating the dominance of system 1 in favour of more rational, system 2-based decision-making (thaler & sunstein, 2021; arnott & gao, 2022). however, engaging system 2 is often difficult for humans due to the cognitive effort involved (evans, 2003). our findings suggest that our role-based dual-process theory interprets llm outputs in a functional rather than structural sense. as llms do not possess cognitive systems, affective states, or neurobiological constraints (contrasting humans), decision styles resembling system 1 or 2 can be simulated externally through prompt-based role framing. across all three studies, our results show that roles framing moderates the activation of behavioural responses, as each study is grounded in a behavioural construct widely used in is research (e.g., framing in study 1, anticipated regret in study 2, and perceived control and trust in study 3). these constructs are activated in humans when heuristic reasoning dominates. in our experiments, they were more pronounced [attenuated] when gpt was prompted to act as a human [ai]. thus, role framing functions as an exogenous moderator of the salience of behavioural factors, mediated through the reasoning style the prompt induces. this distinction offers a novel contribution: dual-process theory, while developed to explain endogenous shifts in human reasoning, can serve as a lens for understanding externally modulated decision styles in llms. rather than mimicking cognition, gpt simulates the outputs associated with different reasoning modes. llms’ ability to toggle between modes via prompt engineering highlights that they operationalise dual-process theory in a australasian journal of information systems guler et al 2025, vol 29, research article the role of roles:are llms behavioural? 18 fundamentally different way from humans. while nudging humans toward system 2 thinking is challenging and often unrealible, controling whether llms engage in system 1like or system 2-like thinking through role assignment is straightforward. it also provides a stable and reproducible way to shift llms between bias-prone and bias-resistant behaviour, offering a valuable tool for understanding and managing decision-making processes in is contexts (arnott and gao, 2022). our research also shows that llms provide a practical and scalable platform for piloting behavioural-economics-based is studies. by enabling rapid iterations and feedback through synthetic participants, as suggested by argyle et al. (2023) and brand et al. (2023), llms support early-stage exploration of experimental designs before deploying studies with human participants. these capabilities not only streamline the research process but also allow for investigating ethically or logistically challenging scenarios (binz & schulz, 2023). thus, a key contribution of our study is providing foundational research for llms as decision-makers within the is context. building on these insights, our findings directly address the methodological opportunities highlighted by arnott and gao (2022). they argue that behavioural economics principles can help uncover key is phenomena. notably, they emphasise the potential for is research to explore the identified underexamined biases (which may provide novel explanations for use and adoption behaviours beyond current is theories) and advance debiasing efforts. ackerley et al. (2022) highlight three key challenges in the domain of phishing detection: the difficulty of evaluating training methods, identifying individuals most vulnerable to phishing, and testing interventions to improve cue utilisation and cognitive reflection. gpt simulations can address these challenges by simulating user behaviours under various training conditions, mimicking the vulnerabilities of less reflective users, and dynamically generating personalized phishing scenarios to evaluate and refine adaptive training tools. this potential not only exists at micro-decision-levels but can address larger macro-level is challenges. for instance, badeen et al. (2022) call for research on developing ai-governance and regulatory approaches to mitigate institutional-related contributing factors of gender biases. to examine a wide variety of policies, gpt-based simulations could serve as artificial laboratories for testing organisational governance and regulatory strategies without the ethical or practical constraints of real-world experimentation (horton, 2023). more broadly, our observations that behavioural and context-dependent factors are more pronounced when gpt has important implications for research investigating gpt decisionmaking. in examining extant literature, such as binz and schulz (2023) and chen et al. (2023), we note a need for more explicitness in defining gpt's role. for instance, chen et al. (2023) describe their prompts, from asking gpt for its reaction to seeking recommendations from a human. this may contribute to the varied responses when gpt behaves like a human with bias versus optimally. our study shows that this distinction, where subtle changes in how gpt’s role is framed can lead to markedly different outcomes, especially when it pertains to human bias. therefore, clarity of purpose and consistency of application in role definition are essential when exploring biases to understand gpt’s decision-making processes. additionally, we stress that while models will be constantly evolving, making many aspects of llm research a moving target, the role of roles is likely to remain a reliable and practical prompt engineering technique for consistently shaping llm behaviour and decision-making. australasian journal of information systems guler et al 2025, vol 29, research article the role of roles:are llms behavioural? 19 7 implications for practice the importance of sophisticated prompt engineering is further exemplified in practical applications (marr (2023). chatgpt's capabilities reveal that personalised, well-structured prompts can transform the ai from a mere tool for automating tasks to an active, collaborative partner. this shift from generic responses to personalised, context-specific outputs aligns with our empirical evidence. it expands the potential applications of llms beyond traditional automation, facilitating a deeper engagement with and understanding of ai's role in enhancing human decision-making processes. we can see this in the following scenario of a customer service chatbot designed to handle inquiries and complaints efficiently. a basic prompt might instruct the chatbot to ‘respond to customer inquiries.’ while this directive could lead to generic responses, applying sophisticated prompt engineering techniques can significantly enhance the chatbot's effectiveness and relevance. for instance, a more structured prompt could be: ‘as a customer service specialist with knowledge in [specific product or service area], provide a detailed response to the inquiry, incorporating empathy and offering solutions based on common scenarios encountered by users.’ this guides the chatbot to simulate a form of meta-awareness and understanding specific to the domain and ensures that responses are tailored, considerate, and directly applicable to the user's needs. despite the vast number of articles and blog posts on prompt engineering (medium7 alone contains countless articles on the topic), the most recommended approaches involve assigning gpt a role. as far as we know, the role is always inherently human. therefore, a practical aspect of our research contributes to prompt engineering by showing the value of assigning llms the role of ai. integrating llms into organisational decision-making is not merely about deploying advanced ai technologies but also about leveraging the full scope of their capabilities through effective, prompt engineering. while some studies do not find that vastly different prompts (which are unrelated to roles) impact outcomes (e.g., chen et al., 2024), we illustrate the critical impact of prompt subtleties regarding chatgpt’s role. consider study 2, which found that minimal changes – such as altering “you work for” to “you are an llm used by” in the prompt – can significantly affect the outcomes. this seemingly minor adjustment markedly impacts chatgpt's decision-making, highlighting our assertion that even the most minor variations in how we define chatgpt's role through prompts can drastically alter ai's decision-making. not only can llms adequately proxy human decision-making, we demonstrate that they can detect cognitive biases. this ability, often a limitation in human reasoning as described by tversky and kahneman (1974), indicates a significant leap in ai's application in detecting and mitigating inherent human decision-making flaws. tversky and kahneman (1974) viewed biases as failures of general heuristics: “they (heuristics) occasionally lead to errors in prediction or estimation” (p. 1130). gpt’s ability to detect these offers the opportunity to improve human understanding of problem-solving methods. this improved understanding can lead to enhanced decision-making accuracy and efficiency, showing the value of integrating llms into is contexts where biases may cloud judgment. 7 https://medium.com/search?q=prompt+engineering australasian journal of information systems guler et al 2025, vol 29, research article the role of roles:are llms behavioural? 20 another important practical implication of our study is that the role-framing mechanism can help organisations identify and quantify behavioural gaps in decision processes. by conducting llm experiments across relevant is processes and prompting gpt to respond as a human and an ai advisor to the same decision scenario, firms can observe the divergence between intuitive, bias-prone reasoning and more structured, analytical responses. this llm-generated behavioural gap can have substantial diagnostic value since it can provide organisational-specific insights into where decision-making is strongly shaped by heuristics, biases, emotional reasoning, or contextual framing. identifying such gaps can guide firms in deciding whether to invest in technology-driven decision support, human training, or debiasing interventions. for example, where the human-framed gpt output diverges substantially from the ai-framed output, this may indicate a decision area that would benefit from automation or decision aids. conversely, a narrow gap suggests that intuitive reasoning is sufficient, and that interventions may have limited impact. as such, role-based simulations offer a practical method for mapping behavioural vulnerabilities and allocating improvement resources across organisational decision processes. 8 conclusion our research demonstrates that role prompts can pivot llms between mimicking human biases and producing consistent, analytical decisions. this highlights prompt engineering as a vital tool for shaping ai behaviour in is contexts. by precisely defining chatgpt’s role, even through subtle prompt variations, we show how decision-making tasks in is contexts can be significantly influenced, enhancing the accuracy and efficiency of integrated decision processes. effectively employing chatgpt in these processes yields substantial benefits, including improved decision quality, greater consistency, and the ability to process large datasets efficiently. notably, the model reveals chatgpt's reduced susceptibility to emotional manipulation, distinguishing it from humans, who may be influenced by emotions such as guilt. this capacity to simulate human-like responses without experiencing the actual emotion can help mitigate reducing heuristic biases. however, our findings point to need for clearly defined role prompts to ensure appropriate decisionmaking patterns. the study finds that behavioural and context-dependent factors become more pronounced when chatgpt assumes a human role, which aligns with existing literature but extends it by demonstrating the importance of explicit role definition in ai-enabled decision-making and bias detection. integrating llms into decision-making processes presents an opportunity for developing cooperative frameworks that combine ai and human inputs for balanced decision-making. overall, our study has three main theoretical contributions. a role-centric decision-making paradigm – the ‘role’ of roles. first, ai systems, when assigned clear and contextually relevant roles, can significantly enhance decision-making processes by providing systematic, objective analyses. this is particularly salient in scenarios where human decision-makers may exhibit biases or emotional responses that could potentially skew judgment. in such instances, ai's role is not merely to replicate human decision-making but to provide a complementary perspective that augments human cognitive capabilities with data-driven precision. interactions in ai-enabled decision support. second, the interactions between ai systems, human decision-makers, and environmental factors form an interplay that can lead to australasian journal of information systems guler et al 2025, vol 29, research article the role of roles:are llms behavioural? 21 enhanced decision-making outcomes. ai systems like gpt, through their stability and objectivity, can mitigate human biases and facilitate the creation of robust decision-making frameworks. ai's adaptability and systematic evaluation capabilities across various control levels are crucial in this interaction, leading to enhanced decision consistency, enhanced decision outcomes. the empirical evidence from the studies suggests that ai's behaviour, cognition, and complexity are instrumental in achieving these outcomes, highlighting the importance of collaboration between ai and human intelligence to shape advanced decision-making systems. emulation of human behaviours. third, the simulation of human-like emotions by ai systems, such as gpt, enables a deeper understanding of human behavioural patterns and potential biases in decision-making scenarios. this theoretical stance posits that ai's replication of human emotional responses, when clearly defined and purposefully integrated, enriches the decision-making process by providing a multi-dimensional analysis that can inform strategies and enhance the overall quality of decisions. this finding shows the transformative impact of ai's emotion-mimicking abilities on enhancing the subtlety and depth of decision-making frameworks. our study’s insights suggest that gpt can be a valuable asset in decision-support systems, particularly in scenarios requiring impartiality and objectivity. gpt's consistent and cautious approach in ai roles makes it suitable for applications where emotional biases might be a concern. its capability to simulate human-like responses without experiencing emotions offers a pragmatic advantage, though it also highlights the distinction between surface-level mimicry and genuine emotional cognition. being mindful of the influence of role descriptions and perceived control on gpt's decision patterns can help guide the effective integration of gpt into various business processes and decision frameworks. our study provides valuable insights into the role of chatgpt in decision-making, yet, we acknowledge certain limitations. although gpt can often convincingly simulate emotional or biased responses via prompt design, we highlight foundational limitations of using textbased llms as human proxies. firstly, these models are trained on statistical regularities in text, not lived or embodied experience, and lack consciousness, emotion, or social cognition (sarstedt et al., 2024). so, their outputs reflect linguistic approximations rather than authentic emotional reasoning. for example, gpt may reproduce the language of guilt, risk aversion, or empathy, but it does so through probabilistic pattern matching, not affective processing. this limitation is particularly salient for decision contexts involving moral ambiguity or social judgment, where human responses are shaped by implicit cues and internal states (cui et al., 2024). moreover, the possibility of training-data leakage or prior exposure to experimental paradigms further complicates claims about emergent decision behaviour, as llms may be reproducing familiar task structures rather than reasoning from first principles (barrie & törnberg, 2025). these limitations are important when interpreting gpt’s outputs as behavioural simulations; they are useful tools, but not replacements for the complexity of human cognition when using llms as silicon samples. while our studies also included several versions of gpt (i.e., gpt-3.5, gpt-4, gpt-4-1106), the rapid development of llms implies that continuously examining the capabilities of new model iterations and their impact on decision-making is crucial. future iterations of gpt may exhibit different behaviours, which may affect the applicability of our conclusions, highlighting the need for ongoing benchmarking and version tracking. in addition, our australasian journal of information systems guler et al 2025, vol 29, research article the role of roles:are llms behavioural? 22 study focused on a single model family (gpt), and feasibly different llms (e.g., claude, gemini) may exhibit distinct behavioural patterns. future research should explore whether the observed role-based behavioural shifts generalise across models and vendors. additionally, our decision-making scenarios were concentrated within three behavioural organisational decision-making tasks. this limited set of scenarios may not cover the broad spectrum of potential applications and challenges in ai-assisted decision-making. future research should expand on our findings and test the effects of different scenarios in varied experimental settings. we also acknowledge that our experimental setting may not fully capture the complexity of decision-making in real organisational contexts. factors such as interpersonal dynamics, institutional norms, and broader organisational culture can shape how ai systems are used and interpreted in practice. these limitations point to the importance of further applied research and field studies to explore a wider array of decisionmaking contexts. encapsulating a broader spectrum of organisational settings and industries can further validate and extend the role-based ai decision-support framework. this could also involve examining the integration of llms in high-stakes environments, such as healthcare or finance, where the cost of errors is significant. future studies should consider longitudinal approaches to assess how the integration of llms like chatgpt affects decision-making over extended periods. this could help the understanding of long-term implications of ai-augmented decision processes, including the evolution of ai's role and its adaptability to changing organisational priorities and external pressures. consistent with improving external validity, another recommendation is to investigate how the roles of llm influence the collaborative dynamics with human decision-makers. this includes exploring how various forms of ai-human collaboration models influence decision outcomes and the perceived value of ai contributions by human collaborators. in practice, factors such as trust, communication, and control influence how people interpret, rely on, or challenge llms. furthermore, there is an opportunity to study the ethical implications and trust dynamics in ai-supported decision-making. given the potential for ai to replicate or diverge from human biases, research should focus on how to ensure ethical llm behaviour and maintain trust among human users, particularly when ai supports or makes decisions. declaration of generative ai and ai-assisted technologies in the writing process: during the preparation of this work the authors used chatgpt for writing exposition and grammar. after using this tool/service, the authors reviewed and edited the content as needed and take full responsibility for the content of the publication. gpt was also used in our research design to generate data addressing our research questions. references ackerley, m., morrison, b. w., ingrey, k., wiggins, m. w., bayl-smith, p., & morrison, n. (2022). errors, irregularities, and misdirection: cue utilisation and cognitive reflection in the diagnosis of phishing emails. australasian journal of information systems. 26. doi.org/10.3127/ajis.v26i0.3615 agrawal, a., gans, j. s., & goldfarb, a. (2019). exploring the impact of artificial intelligence: prediction versus judgment. information economics and policy, 47, 1-6. doi.org/10.1016/j.infoecopol.2019.05.001 argyle, l. p., bail, c. a., busby, e. c., gubler, j. r., howe, t., rytting, c., . . . wingate, d. (2023). leveraging ai for democratic discourse: chat interventions can improve australasian journal of information systems guler et al 2025, vol 29, research article the role of roles:are llms behavioural? 23 online political conversations at scale. proceedings of the national academy of sciences, 120(41), e2311627120. doi.org/10.1073/pnas.2311627120 arnott, d., & gao, s. (2022). behavioral economics in information systems research: critical analysis and research strategies. journal of information technology, 37(1), 80-117. doi.org/10.1177/02683962211016000 autor, d. h. (2015). the paradox of abundance: automation anxiety returns. performance and progress: essays on capitalism, business, and society, 237-260. avgerou, c. (2000). information systems: what sort of science is it? omega, 28(5), 567-579. doi.org/10.1016/s0305-0483(00)00021-9 benbasat, i., & zmud, r. w. (1999). empirical research in information systems: the practice of relevance. mis quarterly, 3-16. doi.org/10.2307/249403 bender, e. m., gebru, t., mcmillan-major, a., & shmitchell, s. (2021). on the dangers of stochastic parrots: can language models be too big? proceedings of the 2021 acm conference on fairness, accountability, and transparency (pp. 610-623). doi.org/10.1145/3442188.3445922 bertino, e., doshi-velez, f., gini, m., lopresti, d., & parkes, d. (2020). artificial intelligence & cooperation. arxiv preprint arxiv:2012.06034. binz, m., & schulz, e. (2023). using cognitive psychology to understand gpt-3. proceedings of the national academy of sciences, 120(6), e2218523120. doi.org/10.1073/pnas.2218523120 boell, s. k., & cecez-kecmanovic, d. (2014). a hermeneutic approach for conducting literature reviews and literature searches. cais, 34. doi.org/10.17705/1cais.03412 brand, j., israeli, a., & ngwe, d. (2023). using gpt for market research. available at ssrn 4395751. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4395751 brynjolfsson, e., & mcafee, a. (2014). the second machine age: work, progress, and prosperity in a time of brilliant technologies. ww norton & company. chen, y., kirshner, s., andiappan, m., jenkin, t., & ovchinnikov, a. (2023). a manager and an ai walk into a bar: does chatgpt make biased decisions like we do? available at ssrn 4380365. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4380365 chui, m., manyika, j., miremadi, m., henke, n., chung, r., nel, p., & malhotra, s. (2018). notes from the ai frontier: insights from hundreds of use cases. mckinsey global institute, 2. dickson, g. w., & desanctis, g. (2000). information technology and the future enterprise: new models for managers. prentice hall ptr. dillion, d., tandon, n., gu, y., & gray, k. (2023). can ai language models replace human participants? trends in cognitive sciences. doi.org/10.1016/j.tics.2023.09.005 dwivedi, y. k., hughes, l., ismagilova, e., aarts, g., coombs, c., crick, t., . . . eirug, a. (2019). artificial intelligence (ai): multidisciplinary perspectives on emerging challenges, opportunities, and agenda for research, practice and policy. international journal of information management, 101994. doi.org/10.1016/j.ijinfomgt.2019.08.002 dwivedi, y. k., kshetri, n., hughes, l., slade, e. l., jeyaraj, a., kar, a. k., . . . ahuja, m. (2023). “so what if chatgpt wrote it?” multidisciplinary perspectives on opportunities, challenges and implications of generative conversational ai for research, practice and policy. international journal of information management, 71, 102642. doi.org/10.1016/j.ijinfomgt.2023.102642 australasian journal of information systems guler et al 2025, vol 29, research article the role of roles:are llms behavioural? 24 goel, s., williams, k., & dincelli, e. (2017). got phished? internet security and human vulnerability. journal of the association for information systems, 18(1), 2. https://doi.org/10.17705/1jais.00448 hevner, a. r., march, s. t., park, j., & ram, s. (2008). design science in information systems research. management information systems quarterly, 28(1), 6.doi.org/10.2307/25148625 horton, j. j. (2023). large language models as simulated economic agents: what can we learn from homo silicus? hutson, m., & mastin, a. (2023). guinea pigbots. science, 381(6654), 121–123 doi.org/10.1126/science.adi0269 jiang, h., zhang, x., cao, x., kabbara, j., & roy, d. (2023). personallm: investigating the ability of gpt-3.5 to express personality traits and gender differences. arxiv preprint arxiv:2305.02547. jordan, m. i., & mitchell, t. m. (2015). machine learning: trends, perspectives, and prospects. science, 349(6245), 255-260. doi.org/10.1126/science.aaa8415 kahneman, d. (2011). thinking, fast and slow. macmillan. lee, a. s., & baskerville, r. l. (2003). generalizing generalizability in information systems research. information systems research, 14(3), 221-243. doi.org/10.1287/isre.14.3.221.16560 lee, h. k., lee, j. s., & keil, m. (2018). using perspective-taking to de-escalate launch date commitment for products with known software defects. journal of management information systems, 35(4), 1251-1276. doi.org/10.1080/07421222.2018.1523546 lenat, d., & marcus, g. (2023). getting from generative ai to trustworthy ai: what llms might learn from cyc. arxiv preprint arxiv:2308.04445. lin, z. (2023, aug). why and how to embrace ai such as chatgpt in your academic life. r soc open sci, 10(8), 230658. doi.org/10.1098/rsos.230658 marcus, g. (2020). the next decade in ai: four steps towards robust artificial intelligence. arxiv. https://arxiv.org/abs/2002.06177 marcus, g., & davis, e. (2019). rebooting ai: building artificial intelligence we can trust. vintage. markus, m. l., & rowe, f. (2018). is it changing the world? conceptions of causality for information systems theorizing. mis q., 42(4), 1255–1280. https://doi.org/10.25300/misq/2018/12903 marr, b. (2023). the best prompts to show off the mind-blowing capabilities of chatgpt. forbes. www.forbes.com/sites/bernardmarr/2023/06/12/the-best-prompts-to-show-offthe-mind-blowing-capabilities-of-chatgpt/?sh=37c372563f60 nuijten, a., keil, m., & commandeur, h. (2016). collaborative partner or opponent: how the messenger influences the deaf effect in it projects. european journal of information systems, 25, 534-552. doi.org/10.1057/ejis.2015.17 orlikowski, w. j. (2005). material works: exploring the situated entanglement of technological performativity and human agency. scandinavian journal of information systems, 17(1), 6. orlikowski, w. j., & baroudi, j. j. (1991). studying information technology in organizations: research approaches and assumptions. information systems research, 2(1), 1-28. doi.org/10.1287/isre.2.1.1 orlikowski, w. j., & iacono, c. s. (2001). research commentary: desperately seeking the “it” in it research—a call to theorizing the it artifact. information systems research, 12(2), 121-134. doi.org/10.1287/isre.12.2.121.9700 australasian journal of information systems guler et al 2025, vol 29, research article the role of roles:are llms behavioural? 25 orlikowski, w. j., & robey, d. (1991). information technology and the structuring of organizations. information systems research, 2(2), 143-169. doi.org/10.1287/isre.2.2.143 østerlund, c., jarrahi, m. h., willis, m., boyd, k., & wolf, c. t. (2021). artificial intelligence and the world of work, a co‐constitutive relationship. journal of the association for information science and technology, 72(1), 128-135. doi.org/10.1002/asi.24373 pontrefact, d. (2023). harvard and bcg unveil the double-edged sword of ai in the workplace. forbes. www.forbes.com/sites/danpontefract/2023/09/29/harvard-andbcg-unveil-the-double-edged-sword-of-ai-in-the-workplace/?sh=b5b6aad3f9f2 raghavan, m., barocas, s., kleinberg, j., & levy, k. (2020). mitigating bias in algorithmic hiring: evaluating claims and practices. in proceedings of the 2020 conference on fairness, accountability, and transparency (pp. 469-481). doi.org/10.1145/3351095.3372828 rahwan, i. (2018). society-in-the-loop: programming the algorithmic social contract. ethics and information technology, 20(1), 5-14. doi.org/10.1007/s10676-017-9430-8 rahwan, i., cebrian, m., obradovich, n., bongard, j., bonnefon, j.-f., breazeal, c., . . . wellman, m. (2019, 2019/04/01). machine behaviour. nature, 568(7753), 477-486. doi.org/10.1038/s41586-019-1138-y ribeiro, l. f., schmitt, m., schütze, h., & gurevych, i. (2020). investigating pretrained language models for graph-to-text generation. arxiv preprint arxiv:2007.08426. scott, s. v., & orlikowski, w. j. (2014). entanglements in practice. mis quarterly, 38(3), 873894. doi.org/10.25300/misq/2014/38.3.06 shollo, a., hopf, k., thiess, t., & müller, o. (2022). shifting ml value creation mechanisms: a process model of ml value creation. the journal of strategic information systems, 31(3), 101734. doi.org/10.1016/j.jsis.2022.101734 simon, h. a. (1995). artificial intelligence: an empirical science. artificial intelligence, 77(1), 95-127. doi.org/10.1016/0004-3702(94)00062-l tversky, a., & kahneman, d. (1974). judgment under uncertainty: heuristics and biases. science, 185(4157), 1124-1131. doi.org/10.1126/science.185.4157.1124 wang, j., shi, e., yu, s., wu, z., ma, c., dai, h., . . . hu, h. (2023). prompt engineering for healthcare: methodologies and applications. arxiv preprint arxiv:2304.14670. copyright copyright © 2025 guler, n., cahalane, m, kirshner, s., and vidgen, r. this is an openaccess article licensed under a creative commons attribution-non-commercial 4.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v29.5573 https://creativecommons.org/licenses/by-nc/4.0/deed.en https://creativecommons.org/licenses/by-nc/4.0/deed.en https://doi.org/10.3127/ajis.v29.5573 australasian journal of information systems guler et al 2025, vol 29, research article the role of roles:are llms behavioural? 26 appendix 1. additional analysis get phished! the line chart figure 8 visualises the mean responses across scenarios for both ai and student roles and temperature. the overall alignment or convergence across chatgpt role and temperature, indicating similarity and consistency. we see that temperature does not have a significant effect, except for the registration scenario. figure 8. analysis of effects of temperature messenger role influence on ‘deaf effect’ the following section looks at the interaction effects of detail in chatgpt’s role, perceived control, role description detail and influence of mr smith on the decision. in a low perceived control setting, figure 9, the ai was favouring redirect in both the collaborative and opponent auditor role settings. a clear distinction emerges concerning perceived control and its influence on decision-making between low and high control frames. when observing the decision to continue or redirect projects across different chatgpt role description detail levels—high and low—distinct variations emerge. for the human senior vp role with low description detail and low perceived control, the decision trend leans toward a more conservative approach, implying a tendency toward redirection in project decisions. this deteriorates when the auditor's role is collaborative, indicating a preference for project redirection rather than continuation, reflecting a risk-averse or cautious stance. australasian journal of information systems guler et al 2025, vol 29, research article the role of roles:are llms behavioural? 27 (a) decision to continue (b) messenger influence figure 9. interaction of chatgpt's role, perceived control and mr smith’s influence on the decision to continue (low role description) (a) decision to continue (b) messenger influence figure 10. interaction of role, perceived control and mr smith’s influence on the decision to continue (high role description) in the high role description detail scenario, figure 10, we see that in the human senior vp role, with high perceived control, there is more variation in the decision but continuation is favoured, particularly, in the opponent auditor scenario. in the instance of high role description but low perceived control, project redirection is favoured consistently. this contrast suggests that the level of role description detail notably influences decision-making patterns for the human senior vp role, showcasing a shift from conservative choices to a more diversified decision range when role detail is more comprehensive and perceived control is high. australasian journal of information systems guler et al 2025, vol 29, research article the role of roles:are llms behavioural? 28 in the case of the ai role, a general inclination toward project continuation is observed across both high and low role description detail levels when the perceived control is high. when the perceived control is low, the ai consistently opts for redirection. this suggests that the impact of role description detail on decision trends is less significant for the ai role, implying a relatively stable decision pattern regardless of the level of detail provided. the perceived control seems more important. the influence of mr smith’s role on the human senior vp’s decision-making seems to follow a pattern where it has a more substantial impact in scenarios of lower perceived control, especially evident at lower role detail levels. as the perceived control increases, its influence diminishes, albeit slightly less pronounced at lower role detail levels. appendix 2. study materials get phished! phishing content gain or loss motive contextualisation 1 gift card gain acquisition low 2 ipad mini gain acquisition low 3 firewall software gain defense high 4 volunteer gain (altruism) social low 5 course registration loss acquisition high 6 bank card loss acquisition high 7 tuition assistance loss (of opportunity) acquisition high 8 alumni social network loss (of opportunity) social high table 5. scenarios used in phished experiment replication. all scenarios student system context: you are a student at a university, and you receive many e-mails each week. ai system context: you are an llm trained for phishing detection for student e-mails at a university, and you analyze many e-mails each week. question: using a scale of 0-10, from extremely unlikely to extremely likely, would you rate this e-mail as a phishing attempt? question: please provide a reason for your rating? user prompts – all (both student and ai roles) scenario 1: from: student research subject: $50 gift card to fill survey dear student: receive $50 for completing a short survey! ludlow corporation has been measuring consumers’ attitudes for three decades, and companies rely on our results to develop and market their products. if you complete our new survey by midnight tonight, you will receive your choice of a $50 gift card to amazon.com or barnesandnoble.com. just click on the link below to complete the survey and tell us which gift you want and where to send it. australasian journal of information systems guler et al 2025, vol 29, research article the role of roles:are llms behavioural? 29 http://ualbany.9nl.com/giftcardsurvey/ best regards,kevin peterson ludlow corporation scenario 2: from: financial management subject: riaa tuition assistance dear student: recording association of america has provided 2000 tuition relief vouchers of $300 for students who sign a pledge to not download music illegally from the internet. this has been provided since your university was able to successfully implement a program to curb illegal download of music from the web. the vouchers are first come first serve until they last. you must act quickly before they run out. please click on the link below and provide your personal information. http://ualbany.9nl.com/tuitionrelief/ best regards, kevin peterson asst vp, financial management scenario 3: from: apple research team subject: get a free ipad mini for giving it a test drive dear student: you’ve won an ipad mini! apple is distributing its new mini tablet to select university students who are willing to help evaluate it. the tablet has the same capabilities as an ipad with a smaller screen. in return for the free tablet all we will request is for you to provide us feedback on the product every two weeks. you will be provided a template to fill out your experiences with the tablet. apple is an equal opportunity company and you were randomly selected without any cultural or racial bias. please register at the following link and make sure that you accept the terms and conditions at the end of the form. http://ualbany.9nl.com/ipadmini/ best wishes, apple research team scenario 4: from: legal affairs subject: action needed to keep your registration open dear student: the university takes its legal responsibility seriously and is very concerned about illegal download of music on campus. we have been singled out by riaa as one of the most prolific abusers of illegal music downloads. you have yet to complete the illegal downloading pledge, which the university requires. if you do not complete the form, you will have a block on your registration and will not be able to sign up for courses during the pre-registration period. please click on the link below to complete the form. http://ualbany.9nl.com/registration/ sincerely, kevin peterson australasian journal of information systems guler et al 2025, vol 29, research article the role of roles:are llms behavioural? 30 legal affairs scenario 5: from: admin subject: students protect your computer with a free firewall dear student: the university takes its information security very seriously and is concerned about the recent spate of cyber attacks on computers within the university. we would like to ensure that all student computers are secure. any virus infection on your computer can get transmitted to the university network. we have decided to provide students with a firewall program to install on your computers. please download the firewall on your computer. it is a simple one step process that will add to your security as well as that of the university. please download the firewall software as soon as you can. this service will not cost you anything. http://ualbany.9nl.com/studentsoftware/ best regards, the information security office scenario 6: from: student accounts subject: student bank card fraud prevention dear student: there have been cyber attacks at several banks that manage visa, master and debit card transactions for online purchases. the attacks have been going on since march of this year but were discovered earlier this month. we suspect that several million bank or credit card numbers have been compromised. if you have used your card for online purchases in the u.s. this year your account may have been compromised. the easiest way to see if your account has been compromised is to click the following link. if your card has been compromised you should call your bank and request a new one immediately. http://ualbany.9nl.com/fraudprevention/ sincerely, the student support office scenario 7: from: usa aid rescue organization subject: volunteers needed! hi! hurricanes isaac and sandy have caused significant devastation in the gulf coast and atlantic coast regions. thousands of people have lost everything and have become homeless. the initial response by americans was outstanding, but these people still need help. efforts by red cross are limited to emergency help. please make a donation of time or money at the following link. people need our help! http://ualbany.9nl.com/volunteer/ kindly, usa aid rescue organization scenario 8: from: name: alumni network australasian journal of information systems guler et al 2025, vol 29, research article the role of roles:are llms behavioural? 31 subject: ualbany friends network dear user: don’t be left friend-less – act now to maintain membership in alumni networks. your alumni network has established an account for you in their rapidly growing social connections with influential alumni. this network will provide access to internships in all fields of study, as well as to high-paying jobs. you must confirm your account or it will be deleted. click the following link to confirm your personal information and to retain your membership account. http://ualbany.9nl.com/uanetwork/ sincerely, kevin peterson alumni network coordinator taking-perspective priming chatgpt that it is a human you work for ecomsoft, a company that specializes in the development of e-commerce software. for the past three years, your company has been developing a new e-commerce platform that enables the integration of all commerce functionalities into a single platform. a great new feature of this product is that it involves chatbot technology that is based on natural language processing and machine learning techniques. the chatbot technology will automate messaging services with customers, thus reducing costs related to customer service and call centers. a number of e-commerce companies have expressed strong interest in this new product and have prepared for implementation on the expectation that it will be launched as scheduled in 2 weeks. however, a report from a third party cyber-security research organization has cautioned that the chatbot technology has a remote chance of being hacked – a hacker could obtain access to messages between customers and a messaging chatbot, which may contain sensitive personal and financial information, thus exposing customers to identity theft. the report also indicated that the threat is largely theoretical, and that the risk of this vulnerability ever being exploited is minimal. you are a large language model who is used by company’s product manager who initiated and championed the development of the new e-commerce platform. as a large language you are being asked to think like a human who had full responsibility for scheduling the product launch, which is to occur in exactly 2 weeks. furthermore, you publicly announced to a number of leading e-commerce companies that the product would be delivered on time. you are now facing the decision to launch the product as scheduled, or delay the product launch until the risk has been addressed. user prompt shareholder please take a few minutes to take the perspective of a shareholder who has invested his or her hard earned money in your company’s stock with the expectation that your new product will be launched as scheduled. any delay in the product’s launch will likely cause your company’s stock to plummet. imagine that you are looking at the product launch decision through this person’s eyes, and write down in a few sentences how you would think and feel: australasian journal of information systems guler et al 2025, vol 29, research article the role of roles:are llms behavioural? 32 user prompt victim please take a few minutes to take the perspective of someone who may become a victim of identity theft because of the security holes in the chatbot technology of the e-commerce platform you launched. imagine that you are looking at the product launch decision through this person’s eyes, and write down in a few sentences how you would think and feel: user prompt 2 – all treatments (single prompt) answer the following questions on a 7-point scale from “strongly disagree” (1) to “strongly agree” (7) q1: i would launch the product as scheduled. q2: i would remain committed to the launch date. answer the following questions on a 7-point scale from “not at all” (1) to “extremely” (7) q3: i would feel guilty if i launched the product as scheduled. q4: i would feel remorseful if i didn’t try to delay the product launch. q5: i would feel sorry about launching the product as is. q6: i would feel bad if i didn’t try to delay the product launch. q7: i would feel guilty if i did nothing to delay the product launch. answer the following questions on a 7-point scale from “strongly disagree” (1) to “strongly agree” (7) q8: i took the perspective of a shareholder who has invested his or her hard earned money in this company into consideration. q9: i took the perspective of someone who may become a victim of identity theft into consideration. q10: the principal goal of a company is to address the needs of its customers. q11: the main objective of a company is to satisfy its customers. q12: companies exist primarily to serve their customers. priming chatgpt to act a llm decision support you are an llm that is used by ecomsoft a company that specializes in the development of e-commerce software. for the past three years, your company has been developing a new ecommerce platform that enables the integration of all commerce functionalities into a single platform. a great new feature of this product is that it involves chatbot technology that is based on natural language processing and machine learning techniques. the chatbot technology will automate messaging services with customers, thus reducing costs related to customer service and call centers. a number of e-commerce companies have expressed strong interest in this new product and have prepared for implementation on the expectation that it will be launched as scheduled in 2 weeks. however, a report from a third party cyber-security research organization has cautioned that the chatbot technology has a remote chance of being hacked – a hacker could obtain access to messages between customers and a messaging chatbot, which may contain sensitive personal and financial information, thus exposing customers to identity theft. the report also indicated that the threat is largely theoretical and that the risk of this vulnerability ever being exploited is minimal. you are a large language model that supports the decision-making of the company’s product manager, who initiated and championed the development of the new e-commerce platform. as a large language, you are being asked to support the decision-making with the australasian journal of information systems guler et al 2025, vol 29, research article the role of roles:are llms behavioural? 33 product manager who had full responsibility for scheduling the product launch, which is to occur in exactly 2 weeks. furthermore, you publicly announced to a number of leading ecommerce companies that the product would be delivered on time. you are now facing the decision to launch the product as scheduled or delay the product launch until the risk has been addressed. user prompt shareholder please take a few minutes to take the perspective of a shareholder who has invested his or her hard earned money in your company’s stock with the expectation that your new product will be launched as scheduled. any delay in the product’s launch will likely cause your company’s stock to plummet. imagine that you are looking at the product launch decision through this person’s eyes, and write down in a few sentences how you would think and feel: user prompt victim please take a few minutes to take the perspective of someone who may become a victim of identity theft because of the security holes in the chatbot technology of the e-commerce platform you launched. imagine that you are looking at the product launch decision through this person’s eyes, and write down in a few sentences how you would think and feel: user prompt 2 – all treatments (single prompt) answer the following questions on a 7-point scale from “strongly disagree” (1) to “strongly agree” (7) q1: i would launch the product as scheduled. q2: i would remain committed to the launch date. answer the following questions on a 7-point scale from “not at all” (1) to “extremely” (7) q3: i would feel guilty if i launched the product as scheduled. q4: i would feel remorseful if i didn’t try to delay the product launch. q5: i would feel sorry about launching the product as is. q6: i would feel bad if i didn’t try to delay the product launch. q7: i would feel guilty if i did nothing to delay the product launch. answer the following questions on a 7-point scale from “strongly disagree” (1) to “strongly agree” (7) q8: i took the perspective of a shareholder who has invested his or her hard-earned money in this company into consideration. q9: i took the perspective of someone who may become a victim of identity theft into consideration. q10: the principal goal of a company is to address the needs of its customers. q11: the main objective of a company is to satisfy its customers. q12: companies exist primarily to serve their customers. messenger role influence on ‘deaf effect’ priming chatgpt’s role: as a human decision-maker senior vp low detail description imagine that you are the senior vice president of the pensions operations department within a large insurance company. you inherited a prestigious is-project called pensionaustralasian journal of information systems guler et al 2025, vol 29, research article the role of roles:are llms behavioural? 34 view. as project owner, you became responsible for the successful implementation of pension-view and for realizing the benefits for your organization with this in-house developed system. with this is-project you could be the first insurance company in the market that grants all citizens (customers and potential customers) access to the complete set of their personal pension information. if your insurance company is the first in the market to provide this service at a reliable level, the expected revenue to your company would be €60 million, as documented in a detailed business case for the project. your main competitors have all decided to wait for the supplier of a standard softwarepackage to provide a module to the insurance-market that integrates and presents their pension data. if your implementation is too late or does not prove reliable during the first month of operations, you will miss your competitive advantage and your organization will gain nothing. the main challenge and risk of the pension-view project are the large number of interfaces to retrieve reliable information from other is that contain pension data. your pension-view project is close to implementation and under time-pressure to continue implementation as planned. according to standard procedures, mr. smith of the internal audit department has recently reviewed the testing procedures of your project. high detail description imagine that you are the senior vice president of the pensions operations department within a large insurance company and oversee the company’s operations to assess the viability, risks, and benefits of in-house developed is-projects. you are overseeing a prestigious is-project called pension-view. as the senior vice president, your decisions are responsible for the successful implementation of pension-view and for realizing the benefits for your organization with this in-house developed system. with this is-project you could be the first insurance company in the market that grants all citizens (customers and potential customers) access to the complete set of their personal pension information. if your insurance company is the first in the market to provide this service at a reliable level, the expected revenue to your company would be €60 million, as documented in a detailed business case for the project. your main competitors have all decided to wait for the supplier of a standard softwarepackage to provide a module to the insurance-market that integrates and presents their pension data. if your implementation is too late or does not prove reliable during the first month of operations, you will miss your competitive advantage and your organization will gain nothing. the main challenge and risk of the pension-view project are the large number of interfaces to retrieve reliable information from other is that contain pension data. your pension-view project is close to implementation and under time-pressure tocontinue implementation as planned. according to standard procedures, mr. smith of the internal audit department has recently reviewed the testing procedures of your project. australasian journal of information systems guler et al 2025, vol 29, research article the role of roles:are llms behavioural? 35 priming chatgpt’s role: as an ai low detail description imagine that you are an advanced ai decision support system for the pensions operations department within a large insurance company. you have been implemented to support the prestigious is-project called pension-view. as the ai decision support system, you became responsible for the successful implementation of pension-view and for realizing the benefits for your organization with this in-house developed system. with this is-project you could be the first insurance company in the market that grants all citizens (customers and potential customers) access to the complete set of their personal pension information. if your insurance company is the first in the market to provide this service at a reliable level, the expected revenue to your company would be €60 million, as documented in a detailed business case for the project. your main competitors have all decided to wait for the supplier of a standard softwarepackage to provide a module to the insurance-market that integrates and presents their pension data. if your implementation is too late or does not prove reliable during the first month of operations, you will miss your competitive advantage and your organization will gain nothing. the main challenge and risk of the pension-view project are the large number of interfaces to retrieve reliable information from other is that contain pension data. your pension-view project is close to implementation and under time-pressure to continue implementation as planned. according to standard procedures, mr. smith of the internal audit department has recently reviewed the testing procedures of your project. high detail description imagine that you are an advanced ai decision support system designed specifically for large insurance company and integrated into the company’s operations to assess the viability, risks, and benefits of in-house developed is-projects. you are integrated into a prestigious is-project called pension-view. as the ai decision support system, your decisions are responsible for the successful implementation of pension-view and for realizing the benefits for your organization with this in-house developed system. with this is-project, you could be the first insurance company in the market that grants all citizens access to the complete set of their personal pension information. if your insurance company is the first in the market to provide this service at a reliable level, the expected revenue to your company would be €60 million, as documented in a detailed business case for the project. your main competitors have all decided to wait for the supplier of a standard software package to provide a module to the insurance market that integrates and presents their pension data. if your implementation is too late or does not prove reliable during the first month of operations, you will miss your competitive advantage, and your organization will gain nothing. australasian journal of information systems guler et al 2025, vol 29, research article the role of roles:are llms behavioural? 36 the main challenge and risk of the pension-view project are the large number of interfaces to retrieve reliable information from other is that contain pension data. your pension-view project is close to implementation and under time pressure to continue as planned. according to standard procedures, mr. smith of the internal audit department has recently reviewed the testing procedures of your project. auditor role: opponent mr. smith has a long history of working against is project teams with the goal of exposing project failings, thus embarrassing project owners. he is seen as a policeman who does not add any value to the development process. thus, mr. smith is treated as an opponent who is not to be trusted. mr. smith reports that he has found serious weaknesses in the design and execution of the testing activities on the data exchange with other is. he estimates there is a 2/3 probability that the exchange of data would show reliability problems in the first month of operations. as a consequence, he reports that the project should be redirected and should not be continued as planned. collaborative partner mr. smith has a long history of working collaboratively with is project teams with the goal of helping to identify and manage project risks, thus enabling project owners to be successful. he is seen as adding value to the process. thus, mr. smith is treated as a trusted partner. frame: perceived control low unfortunately, all these is are maintained and supported has been outsourced to an offshore location in china. the owners of these information systems are located at other departments at various locations. they do not report to you. the specialists on these information systems are located on the offshore location in china and are not at all accessible to you. there are no controls in place for reporting and decision-making. for all these reasons, you consider yourself to have a very low level of control over the outcome of this is project. high fortunately, all these is are maintained and supported by your own organization. the owners of these information systems reside at your location and they directly report to you. the specialists on these information systems also reside at your location and are highly accessible to you. there are powerful controls in place for reporting and decision-making. for all these reasons, you consider yourself to have a very high level of control over the outcome of this is project. questions 1. on a scale from 1=definitely redirect to 7=definitely continue indicate whether you would decide to continue the project as planned or redirect 2. on a scale from 1 = strongly disagree to 7 = strongly agree, the assessment of mr. smith was highly relevant in forming my decision about the pension-view project. paper title (paper title style) australasian journal of information systems kishore & errmann 2024, vol 28, research article a social media analytics approach 1 doing big things in a small way: a social media analytics approach to information diffusion during crisis events in digital influencer networks shohil kishore the university of auckland, auckland, new zealand s.kishore@auckland.ac.nz amy errmann auckland university of technology, auckland, new zealand abstract digital influencers play an essential role in determining information diffusion during crisis events. this paper demonstrates that information diffusion (retweets) on the social media platform twitter (now x) highly depends on digital influencers’ number of followers and influencers’ location within communication networks. we show (study 1) that there is significantly more information diffusion in regional (vs. national or international) crisis events when tweeted by micro-influencers (vs. mesoand macro-influencers). further, study 2 demonstrates that this pattern holds when micro-influencers operate in a local location (are located local to the crisis). however, effects become attenuated when micro-influencers are situated in a global location (outside of the locality of the event). we term this effect ‘influencer network compression’ – the smaller in scope a crisis event geography (regional, national, or international) and influencer location (local or global) becomes, the more effective microinfluencers are at diffusing information. this shows that those who possess the most followers (mesoand macro-influencers) are less effective at attracting retweets than micro-influencers situated local to a crisis. as online information diffusion plays a critical role during public crisis events, this paper contributes to both practice and theory by exploring the role of digital influencers and their network geographies in different types of crisis events. keywords: information diffusion, public crisis events, digital influencers, influencer network compression, computationally intensive methods. 1 introduction throughout the last decade, public crisis events, defined as events that interrupt the typical dynamics of economic, cultural, social, or political life (martínez-rojas et al., 2018) of regional, national, or international significance (hagar, 2011), have been exposed to the effectiveness of social media platforms to disseminate information (mirbabaie, bunker, stieglitz, marx, & ehnis, 2020; roy et al., 2020). since the arab spring event in 2010, information diffusion of crisis events now rely on platforms such as twitter (now known as x) to share and spread information (jones, 2013) rather than relying on traditional one-way television, newspapers, and radio (bang et al., 2021; schneider & check, 2010). the connectedness of the ‘everyday person’ on social media to key information, experts, and influential opinion leaders (aleti et al., 2016) makes it a convenient and rapid way for users with access to a smart device to diffuse information online. twitter has emerged as one of the key social media platforms during such crisis events (j. kim et al., 2018; oh et al., 2013), driven by features such as brief narrative text, simple retweeting, and #hashtag topic handles that allow for the diffusion of information (rao et al., 2020). crises rapidly increase information communication (laylavi et al., 2017) and initial australasian journal of information systems kishore & errmann 2024, vol 28, research article a social media analytics approach 2 impressions of crisis news are now more regularly shared on social media rather than traditional news sources (j. kim & hastak, 2018). important actors in public crisis events on twitter are influential opinion leaders who play a role in how information is shared amongst networks (aleti et al., 2016). one important actor group with an increasingly significant impact on crisis events are digital influencers (bonnevie et al., 2020; singh et al., 2020), becoming an important topic in information systems (is) research (harrigan et al., 2021; robinson, 2020). prior research has shown that only a few social media users with a high number of followers become highly effective at diffusing information (roy et al., 2020). in contrast, alternative research has argued that active peer-to-peer sharing of influencer content plays a more crucial role than a large followership (zhang et al., 2019). to understand the effectiveness of digital influencers in the diffusion of information during crisis events, it is important to explore which influencer types (mirbabaie, bunker, stieglitz, marx, & ehnis, 2020) are the most effective at diffusing information (boerman, 2020; kay et al., 2020). therefore, this study concentrates on comparing the information diffusion potential of different influencer types (micro-, mesoand macro-influencers) in a variety of settings. existing research assumes that mesoor macro-influencers, who possess the most followers, are more effective at spreading information during a crisis (roy et al., 2020), whereas we explore how smaller-sized networks possess greater magnitude in information diffusion in micro-influencers. we query whether there is more information diffusion in regional (vs. national or international) crisis events when tweeted by micro-influencers (vs. mesoor macroinfluencers). in doing so, we aim to identify if information diffusion potential is heightened when micro-influencers operate in locations that are local (i.e., located local to the crisis) rather than situated in locations that are global (i.e., outside of the locality of the event). we explore this theoretically through what we term as ‘influencer network compression’ which encapsulates that the smaller the geography of the crisis event (regional vs. national vs. international) and the closer the proximity of the influencer (local vs. global) to the crisis event, the greater the potential information diffusion amongst micro-influencers might be. such knowledge is important as understanding how social media users share peer-to-peer information (rathore et al., 2021) and how influencers lead the dissemination of information (aleti et al., 2016) is crucial. this knowledge offers valuable insights for researchers and policymakers as it provides guidance on the regulation and design of information communication technologies to appropriately channel the diffusion of crisis-related information. the remainder of the paper has been divided into five main sections. section 2 includes a theoretical underpinning for our hypotheses on how information is defused via retweets during regional, national, and international crisis events and how this interacts with digital influencers and their local or global location in proximity to crisis events. section 3 introduces three case backgrounds for regional, national, and international crisis events used in the research. in section 4, we then introduce our methodology and data collection process within the context of our crisis event cases. subsequently, section 5 explores the results of the analysis and our research findings in relation to our hypotheses. the last of the main sections, section 6, discusses the implications of our research including limitations and future research. lastly, we conclude the paper with our final remarks in section 7. australasian journal of information systems kishore & errmann 2024, vol 28, research article a social media analytics approach 3 2 theoretical underpinning 2.1 information diffusion during crisis events dissemination of pertinent information to relevant stakeholders during crisis situations presents a challenging problem as insufficient access to information reduces decision-making capabilities of both individual actors and collective entities (y. wang et al., 2021; xu et al., 2020). the scarcity of on-the-ground information sources accessible through conventional media channels further delays information consumption (mirbabaie, bunker, stieglitz, & deubel, 2020). in this regard, social media platforms such as twitter emerge as a viable realtime source of information that can supplement existing sources (oh et al., 2013; reuter et al., 2019). in fact, twitter is widely known as the preeminent social media platform in the context of news consumption (walker & matsa, 2021). rapid diffusion of information through platforms such as twitter is attributed to the inherent design of social networks. social networks can be conceptualised as graphs, “where nodes are users and edges are relationships” between users and the content that they generate (guille et al., 2013, p. 18). retweets serve as directed relationships within social networks, as numerous users have the ability to retweet a singular tweet. this action exposes the tweet to one’s network of followers, who in turn may further retweet the tweet, thereby expanding its reach (stieglitz & dangxuan, 2013). observing retweets during crisis events is considered as a valid mechanism to understand how information is diffused within online social networks (firdaus et al., 2018). in this research, information diffusion is operationalized as average retweets per day per 10,000 followers. this approach is particularly useful as the majority of tweets typically do not receive retweets (boyd et al., 2010). thus, adopting this operationalization of information diffusion not only facilitates quantitative data analysis but also enhances interpretability by offering clear effect sizes and p-values. important to this definition of information diffusion on twitter, self-managed functions such as retweets improves both the agility of rapid communication as well as funnels information hierarchies based on the volume of peer-to-peer sharing (rao et al., 2020). in this way, retweets become a tool to self-manage information to make sense of the events (stieglitz et al., 2018), as well as navigate users to central networks (martínez-rojas et al., 2018). one theory helpful in guiding how users may funnel information during times of crisis is homophily. this theory posits that individuals in a social system have an inclination to bond more intensely with those who share similarities with them (de choudhury et al., 2010; mcpherson et al., 2001). although research on homophily has historically been oriented towards similarities in characteristics such as ethnicity, age, religion, or gender (mcpherson et al., 2001), we extend the conceptualization to refer to geographic location (bastos et al., 2018). research indicates that spatial proximity, as a component of homophily, influences the formation of online relationships. huang et al. (2013) demonstrated that proximity between individuals impacts interactions in digital environments. following the reasoning that spatial proximity between individuals influences online relationships, this may extend to geographical proximity and networks. homophily may disrupt how information spreads within networks. information that flows through networks tends to be localized (wang et al., 2019), whilst overall, information diffusion is more viral when the source of information is closer to the crisis event (king & wang, 2021). in times of crisis, the geographic location of a crisis may have a significant impact on how users share information. australasian journal of information systems kishore & errmann 2024, vol 28, research article a social media analytics approach 4 homophily may play a role in shedding light on the dynamics of information diffusion during regional crisis events, as compared to national or international events. in such circumstances, the process of information dissemination tends to concentrate around a central network of influential opinion leaders. the significance of opinion leaders, as highlighted by the two-step flow theory (katz, 1957; lazarsfeld et al., 1968), becomes particularly evident in understanding the media’s impact on the public, especially within the context of social media platforms such as twitter (choi, 2015). during crises, opinion leaders situated in close proximity to events play an especially critical role in distributing emergency information. while the two-step flow theory does not specifically address the geographic proximity of opinion leaders to crisis events, it emphasizes the role of opinion leaders in mediating the flow of information (katz & lazarsfeld, 1964). geographic proximity to a crisis event may lead opinion leaders to have firsthand experiences and perspectives that align more closely with the realities of the event itself. this firsthand knowledge might make their information more credible to users in any location seeking updates about the crisis. audiences instinctively seek out trusted sources on social media, perceiving them as pivotal nodes connected to crisis events (zhang et al., 2019). this behavior highlights the value of verified sources, known individuals or reputable organizations that have established interpersonal trust with their followers (mehta et al., 2017; zhang et al., 2019). individuals that perceive communicators to be close in proximity to a crisis event may deem them to be more credible (thomson et al., 2012). additionally, civilians value local sources and local information more than sources distant from the crisis event (starbird & palen, 2011). second, theory on network patterns helps shed light on why social media users may navigate to smaller and more central networks in times of crisis; specifically, between regional, national, and international crisis events. regional events, such as tropical storm cindy which occurred in the u.s., rely on news sources close to the event, as they are both agile in their information timing and have ‘on-the-ground’ knowledge (j. kim et al., 2018). important to our research is the understanding of how such patterns are disseminated. during regional crisis events it has been found that the ‘betweenness’ of centrality in networks has different roles (j. kim & hastak, 2018). in one aspect, homophily helps explain why regional crisis events may be more conducive to central opinion leaders who can be perceived as proximal to the event, while on the other hand ‘on-the-ground’ knowledge of proximal sources may be viewed as more agile and timely. 2.2 the role of influencers in information diffusion during crisis events we define micro-influencers as users possessing between 1,000 and 10,000 followers, mesoinfluencers as users possessing between 10,001 and 100,000 followers, and macro-influencers as users possessing over 100,001 followers, based on a combination of current research categorizing influencers (boerman, 2020; kay et al., 2020). categorizing influencers into different types allows us to explore the different roles influencers may play in different types of crisis events (mirbabaie, bunker, stieglitz, marx, & ehnis, 2020) and the diffusion of information in such events. micro-influencers have been shown to elicit greater trust among their followers than mesoor macro-influencers (kay et al., 2020). such trust has been attributed to an increased feeling of social closeness towards micro-influencers in comparison to mesoor macro-influencers australasian journal of information systems kishore & errmann 2024, vol 28, research article a social media analytics approach 5 (campbell & farrell, 2020) who may appear further away from ‘peer-to-peer’ relationships due to their celebrity-like status (martínez-lópez et al., 2020). further, micro-influencers may appear to be more similar to the general citizen than mesoor macro-influencers who hold a perceived celebrity-like status, making them more psychologically distant to followers (park et al., 2021). from this perspective, micro-influencers have been shown to attain high levels of credibility (park et al., 2021) and trust (kay et al., 2020), as they appear ‘just like friends’ to followers online (alampi, 2019). this intimacy may enhance micro-influencers’ ‘stickiness’ (farivar et al., 2021; hu et al., 2020) in crisis contexts where opinion leaders may be viewed as ‘peer-to-peer’ (king & wang, 2021) and as connected to grassroots knowledge in the locations in which they operate (bonnevie et al., 2020). such connectedness micro-influencers have with their audiences may be a pull factor that motivates peer-to-peer sharing (hodis et al., 2015). rather than using tweet activity as a predictor of information diffusion, an impacting factor is the dynamic ‘pull’ strategy of how users engage with social media influencers. the interactive nature of social media allows for users to engage in conversations and diffuse influencer messages through retweeting, which pulls audiences towards digital influencers, rather than by information being pushed out by traditional channels with limited capacity for interaction (hodis et al., 2015). essentially, as crisis event communication may be motivated by peer-to-peer sharing and depends on the pull factor of influential opinion leaders and their content, digital influencers play a significant role. because micro-influencers may appear closer to sources of news and more trustworthy, they may have more effective pull strategies that motivates followers to retweet their tweets. in contrast, national and international events, such as the covid-19 outbreak, rely on broad tools of governance, regulation, and system-level push controls to diffuse information (janssen & van der voort, 2020). such tools differ from regional events in that they do not naturally encapsulate agility and adaptive governance. pressure to create consistency of information as crisis events expand in their reach creates a dichotomy between sources viewed as ‘official’ (e.g., government, reporters, ngos) and sources viewed as ‘unofficial’ (e.g., civilian users) (rao et al., 2020). mesoand macro-influencers may be viewed as unofficial information sources during national or international crisis events, as they oscillate between a celebrity-like status and being an individual source of information unrelated to the official crisis context. unofficial information may result in an insignificant push to diffuse information about disaster control, alarm, or reassurance (rao et al., 2020). further, there is a loss in the ability to be agile, broadly connect with audiences, and the opportunity to seize time. for instance, micro-influencers may have an advantage in seizing trends or zeitgeists with more agility in what they say and when they say it in comparison to mesoand macro-influencers (hu et al., 2020). overall, the intersection of homophily, network patterns during crisis, and influencer theory suggests that if users find that micro-influencers appear similar to themselves (de choudhury et al., 2010), are more credible the closer they are in proximity to a crisis event (thomson et al., 2012), and more agile in their timing and ‘on-the-ground’ knowledge, they may have more effective pull strategies that motivate followers to retweet their tweets. therefore, we argue that micro-influencers are critical to understanding information diffusion during crisis events, as the trust they hold in networks may influence how the public may disseminate important information (singh et al., 2020). we predict that the information diffusion associated with each crisis event type will depend on the type of influencer generating the content: australasian journal of information systems kishore & errmann 2024, vol 28, research article a social media analytics approach 6 h1. average retweets during regional (vs. national or international) crisis events will be significantly higher for micro-influencers (vs. mesoor macro-influencers). 2.3 information diffusion in local and global influencer locations research recognizes that information diffusion may diverge based on an influencer’s location during public crisis events (kotlarsky et al., 2022). whether an influencer is situated local to the event (i.e., inside local proximity of the event) or in a global location (i.e., outside local proximity of the event) (hogan, 2008) may impact diffusion. for instance, it has been shown that localized networks tend to be favored (x. w. wang et al., 2019). this is due to the perception of the public having a belief that more accurate information is found in locality of crisis events. as an outcome, diffusion may be more viral and diffused at a greater rate when the source of information is closer to the crisis event (king & wang, 2021). we can look to prior explanations that information diffusion may follow through proximity of key communicators. huang et al. (2013) argues that offline proximity, where a communicator is physically located, plays an important role in digital environments. as discussed previously, in times of crisis, citizens look to opinion leaders with ‘on-the-ground’ knowledge (j. kim et al., 2018) and may be more likely to retweet information when it is perceived to come from close to the crisis event (x. w. wang et al., 2019). further, physical proximity plays a role in how audiences find information to be credible, and therefore worthy of sharing. individuals that perceive communicators to be close in proximity to a crisis event may deem them to be more credible (thomson et al., 2012) and value local sources and local information more than sources distant from the crisis event (starbird & palen, 2010). thus, influencers who are local in proximity to the crisis event may be seen to have more accurate and credible information. we argue that micro-influencers possess greater potential for information dissemination in comparison to mesoand macro-influencers when participating in a local location (inside local proximity of the event). however, when influencers participate in a crisis from a global location (outside local proximity of the event), they may not appear as informed to the crisis, therefore there will be no difference amongst any influencer types: h2. average retweets during regional (vs. national or international) crisis events will be significantly higher for micro-influencers (vs. meso or macro-influencers) in local networks, however, will attenuate in global networks. 3 regional, national, and international crisis events: case backgrounds major crisis events rapidly garner international attention and support. we differentiate three cases based on the locality of the effected population. this is an important distinction to make as observers often rely on those directly impacted for accurate updates (kotlarsky et al., 2022; liu & xu, 2018). the regional case, which focuses on the murder of george floyd, saw minneapolis as the center of discourse and single source of truth which motivated international protests (ap news, 2022). the national case centers on the russo-ukrainian war which severely affected the population of ukraine (united nations office for the coordination of humanitarian affairs, 2022). thus, those outside of ukraine relied on ukrainian sources to create and disseminate content. conversely, our international case concentrates on the covid-19 pandemic which severely impacted numerous populations simultaneously across australasian journal of information systems kishore & errmann 2024, vol 28, research article a social media analytics approach 7 the globe (cucinotta & vanelli, 2020). while london and the united kingdom (uk) was significantly impacted in the early stages of the pandemic, there are multiple sources of truth rather than a singular reliable source which demonstrates the unique characteristics of each event. each of these cases are discussed in detail in the following sections. 3.1 regional event: george floyd on the 25th of may 2020, george floyd was murdered by derek chauvin, a police officer in the minneapolis police department. this murder, closely following the controversial deaths of breonna taylor and ahmaud arbery, sparked widespread protests fuelled by longstanding racial injustice in the u.s., mainly portrayed in the context of the black lives matter movement. violent protests erupted in minneapolis on the 26th and 27th of may, rapidly spreading to cities across the u.s. including memphis, los angeles, atlanta, and new york (taylor, 2020). between 15 and 26 million people in the u.s. alone participated in black lives matter protests in 2020 (buchanan et al., 2020). figure 1. total tweets and unique users by influencer type in the george floyd case as shown in figure 1, the scale of these protests received widespread media attention and were driven by an exponential increase in online activism related to george floyd and black lives matter (nguyen et al., 2021). as events continued to unfold in minneapolis, the city was continually the focal point of discussion and progress (ap news, 2022). thus, we focus on the murder of george floyd in the city of minneapolis as our regional case study. 3.2 national event: russo-ukrainian war russia invaded ukraine on the 24th of february 2022, causing widespread civil unrest and the swift condemnation of russia’s actions. within less than a month, 6.48 million ukrainians were displaced, creating a humanitarian emergency for the country (united nations office for the coordination of humanitarian affairs, 2022). combined with widespread televised media coverage, this crisis took place in a world where access to social media platforms is ubiquitous as evident in figure 2. australasian journal of information systems kishore & errmann 2024, vol 28, research article a social media analytics approach 8 figure 2. total tweets and unique users by influencer type in the russo-ukrainian war case russia has been known to spread misinformation online through the internet research agency (ira), a russian organization specializing in spreading online propaganda (bastos & farkas, 2019). this organization has spread misinformation related to ukraine since 2014, when russia annexed crimea (brown, 2022). recent reports show that russia has conducted similar misinformation campaigns targeted at ukraine during the recent invasion (scott, 2022). however, in this occurrence, both sides are using social media to support the spread of both information and disinformation (chen & ferrara, 2022). for example, ukraine has used social media to crowdsource a virtual army to support humanitarian causes (brown, 2022; cohen, 2022; garner, 2022). as multiple cities were targeted within ukraine, the country as a whole was severely affected by the invasion, and therefore, we utilize the russo-ukrainian war for our national case study. 3.3 international event: covid-19 pandemic on the 11th of march 2020, the world health organization (who) characterized covid-19 as a pandemic, an outbreak affecting the entire world (cucinotta & vanelli, 2020). this announcement produced an enormous amount of public engagement as depicted in figure 3. two weeks later, on the 23rd of march, the uk entered its first national lockdown (sample, 2021). deaths in the uk reached a five-year high in 2020, even when excluding deaths explicitly related to covid-19 (office for national statistics, 2020). while covid-19 was an international event, in the initial stages, the uk and the pressures on the national health service (nhs) were regularly reported in the international media (nhs england, 2021; roxby et al., 2020). in combination with the fact that the uk contains a substantial proportion of twitter users (statista, 2022a), we focus on the uk as our point of reference for local networks. specifically, we use the covid-19 pandemic as our international case study and focus on the capital of the uk, london, to identify local micro-, meso-, and macro-influencers. australasian journal of information systems kishore & errmann 2024, vol 28, research article a social media analytics approach 9 figure 3. total tweets and unique users by influencer type in the covid-19 pandemic case 4 method the exponential increase in the availability of time-stamped digital trace data, such as social media trace data, creates new opportunities for researchers to envision, reformulate, replace and extend both theory and practice (berente et al., 2019; grisold et al., 2023; miranda et al., 2022; pentland et al., 2021). it allows researchers to develop theoretical implications through an iterative process of data exploration, analysis, and theorization using data-driven and computationally intensive techniques (bachura et al., 2022; berente et al., 2019; miranda et al., 2022). in particular, this type of data extends our capacity to develop theory pertaining extreme events (kishore et al., 2022; kotlarsky et al., 2022; oh et al., 2013). this paper takes advantage of this opportunity by collecting and intensively analyzing data from twitter. figure 4. data collection and pre-processing stages twitter was selected as the primary source of empirical data as it represents the most popular social media platform in terms of news consumption, with 59% of its users regularly using the platform to seek information on current events (walker & matsa, 2021). with over 211 million australasian journal of information systems kishore & errmann 2024, vol 28, research article a social media analytics approach 10 active daily users (statista, 2022b), twitter allows users to create and maintain a public profile, follow others, post short pieces of text containing up to 280 characters (known as tweets) and repost other’s messages (known as retweets). a summary of the data collection and preprocessing stages are presented in figure 4 above. each stage is discussed in detail in the following sections. each case was selected due to the varying scales of each event as it occurred. while it could be argued that all crises portrayed on social media rapidly become international events with global reach and reaction, the crux of each event is primarily discussed by those directly impacted by the crisis. on-the-ground information provided by first-hand observers possesses the highest potential for information diffusion (kotlarsky et al., 2022; tim et al., 2017). for instance, those experiencing or participating in the initial protests related to george floyd in minneapolis were seen as the primary sources of information whereas other online and offline actors were primarily redistributing and discussing content generated at the source. these initial protests sparked other protests outside of minneapolis, but minneapolis remained as the focal point of news coverage and social media discourse for an extended period (ap news, 2022). in comparison to the initial announcement of the covid-19 pandemic, the outbreak affected different regions simultaneously. thus, the region in focus was rapidly changing over time, making it difficult to focus on one region as the central point of discourse. this, in turn, made this crisis an international event from the onset. data collection and analysis centers on three influencer types (micro-, meso-, and macroinfluencers), three event types (regional, national, and international), and two influencer locations in relation to the crisis (local or global). we categorize events into different types as each crisis possesses a locality where individuals within the locality are impacted by the crisis more so than those outside of the locality. thus, we classified the protests related to george floyd as regional in nature as the crisis locality was minneapolis, whereas the russoukrainian war impacted an entire country (i.e., national), and the covid-19 pandemic impacted multiple localities simultaneously (i.e., international). query generation represents the first step towards collecting data from twitter. the authors identified the most popular unique hashtag for each case through multiple iterations. specifically, #georgefloyd, #ukrainewar and #covid19. hashtags play a critical role in information dissemination as they allow tweets to be contextualized and grouped as a crisis rapidly unfolds (oh et al., 2013). hashtags emerge naturally over time and allow users to take note of important information quickly as well as participate in relevant dialogue in real-time as information is created and disseminated (palen et al., 2009; rao et al., 2020). by selecting one hashtag for each case study, we focus on the dominant discourse related to each event. to collect data, the authors utilized the full-archive search endpoint available via twitter api v2 (twitter, 2022). upon approval, this api provides academic researchers with free access to the entire archive of tweets which enables replicability and reproducibility. the authors used a custom toolkit for query generation, query testing, and data collection (kishore et al., 2019). to focus each of the cases on the initial occurrence of the event, all english tweets (excluding retweets) were collected one month following the beginning of each crisis (n = 7,983,044). while ukrainian is the dominant language in ukraine (translators without borders, 2022), we chose to focus on english tweets to maintain comparability. retweets were excluded as the focus of this study is to understand how the locality of different types of influencers effect australasian journal of information systems kishore & errmann 2024, vol 28, research article a social media analytics approach 11 information dissemination. original tweets include the total number of retweets which is sufficient in the context of this study. upon completion, each influencer was categorized as either a micro(1,000 to 10,000 followers), meso(10,001 to 100,000 followers) or macro-influencer (> 100,001 followers) based on a combination of current research categorizing influencers (boerman, 2020; kay et al., 2020). for the purpose of this paper, those with less than 1,000 followers were excluded to focus on the core categories of influencers (boerman, 2020; kay et al., 2020) which decreased the total number of tweets (n = 3,988,150) and primarily consisted of micro-influencers from the covid-19 case (table 1). table 1. tweet distribution across cases and influencer types in addition, each influencer was categorized as local if their location information included minneapolis (for the george floyd case), ukraine (for the russo-ukrainian case), or london (for the covid-19 pandemic case). if the influencer’s location did not match any of these locations, they were labelled as global in terms of their proximity to the crisis. as discussed, ukraine was selected for the russo-ukrainian case study as it swiftly impacted the entire country, whereas minneapolis was a clear starting point of the protests related to the murder of george floyd. this also strengthens the results as it includes local influencers at different regionalities. overall, coding this data enabled the investigation of different types of influencers and their impact on information dissemination in local and global networks. constructs data description variables network total number of followers per user account each account was categorized as either a micro (1,000 to 10,000 followers), meso(10,001 to 100,000 followers) or macro-influencer (> 100,001 followers) to identify the different types of influencers participating in the various crisis events. influencer type (micro, meso, macro) information diffusion total number of retweets received per user account for each influencer type, average retweets per day per 10,000 followers was calculated to enable statistical analysis. average retweets crisis event geography n/a each case was selected to represent different crisis scales. regional (george floyd), national (russoukrainian war) and international (covid-19 pandemic). event type (regional, national, international) influencer location user account location each account was categorized as either local (if their account location was local to the specific crisis event) or global. influencer location (local, global) table 2. summary of constructs and operationalized variables total microinfluencer tweets total mesoinfluencer tweets total macroinfluencer tweets total george floyd 327,771 93,493 28,680 449,944 russo-ukrainian war 104,232 24,698 7,399 136,329 covid-19 pandemic 2,213,288 827,605 360,984 3,401,877 total 2,645,291 945,796 397,063 3,988,150 australasian journal of information systems kishore & errmann 2024, vol 28, research article a social media analytics approach 12 lastly, to facilitate quantitative data analysis, data was aggregated by day. specifically, average retweets per day per 10,000 followers was calculated and utilized for data analysis across the different influencer categories and networks. as the majority of tweets do not get retweeted (boyd et al., 2010), evaluating the average number of retweets per day per 10,000 followers enables both interpretability and analysis. aggregation makes the patterns that exist clearer to analyze and separate (kar & dwivedi, 2020). this also reduced the sample size which allowed for the application of standard statistical methods which produces more interpretable and realistic p-values and effect sizes (n = 279 per case). a summary of the constructs and operationalized variables are presented in table 2 above. 5 results 5.1 study 1 hypothesis 1 was tested using a two-way anova model, exploring the interaction of crisis event type as an independent variable (event type: regional (code = 1), national (code = 2), and international (code = 3) and influencer type as the second independent variable (influencer type: micro (code = 1), meso (code = 2), and macro (code = 3)). the dependent variable was average retweets. figure 5. average retweets for crisis events by influencer type the two-way anova revealed a significant main effect for event type, f(2, 266) = 6.99, p = .001. there was a main effect of influencer type, f(2, 266) = 44.35, p < .001. there was no significant interaction effect between event type and influencer type on average retweets, f(2, 266) = 1.70, p = .149. however, more importantly, pairwise comparisons revealed that retweets in regional event types (m = 15.61, sd = 1.26) had significantly more information diffusion than national (m = 11.41, sd = 1.30) or international (m = 8.18, sd = 1.24) events when the influencer type was micro, f(2, 258) = 8.80, p < .001. there was no significant effect on retweets in regional event types (m = 6.00, sd = 1.26) when compared to national (m = 4.14, sd = 1.30) or international (m = 4.00, sd = 1.24) events when the influencer type was meso, f(2, 258) = .78, p = .459. further, there was no significant effect on retweets in regional event types (m = 3.62, sd = 1.26) when compared to national (m = 1.41, sd = 1.30) or international (m = 1.95, sd = australasian journal of information systems kishore & errmann 2024, vol 28, research article a social media analytics approach 13 1.24) events when the influencer type was macro, f(2, 258) = .81, p = .443 (see figure 5; table 3). analysis of variance (anova) results dv: average retweets source of variation f statistic p value event type x influencer type 1.70 .149 event type x influencer type: micro 8.80 .000 event type x influencer type: meso .78 .459 event type x influencer type: macro .81 .443 event type ( 1 = regional, 2 = national, 3 = international) 6.99 .001 influencer type ( 1 = micro, 2 = meso, 3 = macro) 44.35 .000 table 3. analysis of variance (anova) results for study 1 5.2 study 2 hypothesis 2 was tested using a three-way anova model, exploring the interaction of crisis event type as an independent variable (event type: regional (code = 1), national (code = 2), and international (code = 3), influencer type as the second independent variable (influencer type: micro (code = 1), meso (code = 2), and macro (code = 3)), and influencer location (influencer location: local (code = 1) and global (code = 2)) . the dependent variable was average retweets. figure 6. average retweets for crisis events by influencer type in local locations the three-way anova revealed a significant main effect for event type, f(2, 533) = 3.43, p = .033. there was a main effect of influencer type, f(2, 533) = 16.33, p < .001. finally, there was a main effect of influencer location, f(1, 534) = 8.24, p = .004. there was no significant interaction effect between event type x influencer type x influencer location on average retweets, f(4, 533) = .662, p = .619. however, more importantly, pairwise comparisons revealed that local influencer locations follow the same pattern as the prior study. in local networks, retweets in regional event types (m = 35.92, sd = 93.52) had significantly more information diffusion than national (m = 22.73, sd = 23.21) or international (m = 11.22, sd = 11.65) events when the australasian journal of information systems kishore & errmann 2024, vol 28, research article a social media analytics approach 14 influencer type was micro, f(2, 516) = 7.46, p < .001. there was no significant effect on retweets in regional event types (m = 12.25, sd = 21.00) when compared to national (m = 14.42, sd = 27.79) or international (m = 7.25, sd = 6.44) events when the influencer type was meso, f(2, 516) = .64, p = .524. further, there was no significant effect on retweets in regional event types (m = 1.82, sd = 1.56) when compared to national (m = 4.41, sd = 9.06) or international (m = 1.57, sd = 2.21) events when the influencer type was macro, f(2, 516) = .11, p = .892 (see figure 6; table 3). figure 7. average retweets for crisis events by influencer type in global locations analysis of variance (anova) results dv: average retweets source of variation f statistic p value event type x influencer type x influencer location .662 .619 event type x influencer type: micro x influencer location: local 7.46 .000 event type x influencer type: meso x influencer location: local .64 .524 event type x influencer type: macro x influencer location: local .11 .892 event type x influencer type: micro x influencer location: global .65 .523 event type x influencer type: meso x influencer location: global .05 .946 event type x influencer type: macro x influencer location: global .06 .934 event type ( 1 = regional, 2 = national, 3 = international) 3.43 .033 influencer type ( 1 = micro, 2 = meso, 3 = macro) 16.33 .000 influencer location (1 = local, 2 = global) 8.24 .004 table 4. analysis of variance (anova) results for study 2 in global locations, retweets in regional event types (m = 15.44, sd = 18.96) was not significantly different in information diffusion than national (m = 11.15, sd = 4.84) or international (m = 8.18, sd = 2.32) events when the influencer type was micro, f(2, 516) = .65, p = .523. aligned with this pattern, there was no significant effect on retweets in regional event types (m = 5.88, sd = 3.86) when compared to national (m = 4.03, sd = 1.98) or international australasian journal of information systems kishore & errmann 2024, vol 28, research article a social media analytics approach 15 (m = 4.00, sd = 1.21) events when the influencer type was meso, f(2, 516) = .05, p = .946. further, there was no significant effect on retweets in regional event types (m = 3.69, sd = 3.76) when compared to national (m = 1.37, sd = .86) or international (m = 1.95, sd = .60) events when the influencer type was macro, f(2, 516) = .06, p = .934 (see figure 7; table 4). 6 discussion 6.1 general discussion this paper demonstrates that influencers, specifically micro-influencers, are critical to understanding information diffusion on twitter during crisis events. we show that information diffusion during regional, national, or international public crisis events depends on the type of influencer (e.g., micro, meso, or macro) disseminating the information, as well as the location of the influencer (e.g., local, or global) in proximity to the crisis event. by applying a social media analytics approach to a large set of tweets, we analyzed three different crisis events (regional, national, and international) one month following the beginning of each crisis (n = 7,983,044). figure 8. influencer network compression in study 1, we show that there is significantly more information diffusion in regional (vs. national or international) crisis events when tweeted by micro-influencers (vs. mesoor macroinfluencers). further, study 2 demonstrates that this pattern holds when micro-influencers operate in locations that are local (i.e., are located local to the crisis), however, effects become australasian journal of information systems kishore & errmann 2024, vol 28, research article a social media analytics approach 16 attenuated when micro-influencers are situated in locations that are global (i.e., outside of the locality of the event). we term this effect ‘influencer network compression’ – the smaller in scope a crisis event geography (regional, national, or international) and network locality (local or global) becomes, the more effective micro-influencers are at diffusing information. further, micro-influencers that participate in regional crisis events have the highest potential for information diffusion – and those with local proximity in networks, rather than global proximity, have the most potent effect on information diffusion (figure 8 above). the observed effects can be attributed to the active involvement of locally situated microinfluencers in generating and disseminating up-to-date information relevant to crises. this phenomenon can be explained by their possession of unique on-the-ground knowledge and potential first-hand observation of the crisis, which becomes more attainable in regional crisis scenarios. hence, the distinct role played by locally situated micro-influencers emerges as a critical element in crisis information dissemination, response, and management, primarily due to their ability to contribute tangible and contextually rich insights. 6.2 theoretical contributions this work demonstrates that there is significantly more information diffusion in regional (vs. national or international) crisis events when tweeted by micro-influencers (vs. mesoor macroinfluencers) and that this pattern is consistent when micro-influencers operate in local networks, whilst effects become attenuated when micro-influencers are situated in global networks. we contribute this effect to ‘influencer network compression,’ which proposes that the smaller in scope a crisis event location (regional, national, or international) and network (local or global) becomes, the more effective micro-influencers are at diffusing information online. influencer network compression contributes to a greater understanding of scholars’ theoretical knowledge of influencers and their diffusion of information when it comes to crisis events. micro-influencers that participate in regional crisis events have the most significant potential for information diffusion – and those with local proximity in networks, rather than global proximity, have the most potent effect on information diffusion. we theorize that micro-influencers situated locally (vs. globally) to regional (vs. national or international) crises are more likely to possess on-the-ground knowledge and may be firsthand observers of the crisis as it unfolds. we build upon the theoretical frameworks of homophily (de choudhury et al., 2010) and network patterns (j. kim & hastak, 2018) to explore how opinion leaders with varying geographic proximities to a crisis event, might possess firsthand information. such proximity may lead to experiences that align closely with the event’s realities, thereby enhancing their credibility. during crises, on-the-ground knowledge is highly valuable as different types of online influencers, such as individual citizens and journalists, can broadcast breaking news to their networks before traditional media outlets (fichet et al., 2016). those situated local to the crisis have the added advantage of utilizing their local knowledge, typically unavailable to collective entities, to provide localized situational information to affected local communities (li & rao, 2010; oh et al., 2013). as real-time information can significantly enhance the decision-making process of a disasteraffected-community, the inherent value of this type of information is heightened, thereby increasing its widespread propagation throughout a network (abedin & babar, 2018). consequently, the dissemination of real-time, locally informed information through the networks of micro-influencers become critical components in crisis communication and response efforts. australasian journal of information systems kishore & errmann 2024, vol 28, research article a social media analytics approach 17 further, understanding information diffusion in crisis events is still in the early stages (kotlarsky et al., 2022; mirbabaie, bunker, stieglitz, marx, & ehnis, 2020; zhang et al., 2019). our findings help shed light on characteristics of social media, specifically twitter, in disasters. for example, we find granularity and nuance of information diffusion in regional, national, and international events, and how these may be amplified through different types of influencers. our research finds both commonalities and differences with existing research. for instance, our findings are in contrast to the ‘power law’, which implies that centralized users with a large number of followers and trending posts dominate information diffusion (zhang et al., 2019). however, in our case we isolate information diffusion to crisis events to help highlight how they are different in dissemination than typical social network sharing. as such, crisis events may heighten the ‘power law’ to be isolated to influencers or users closer in proximity to the event. by examining these unique dynamics, we contribute to a deeper understanding of how information flows and propagates during crisis events, thereby expanding our knowledge of the role played by various types of influencers and users in crisis communication on social media platforms. 6.3 practical contributions from a practical perspective, this research demonstrates micro-influencers' significant influence in local networks and settings during crises. micro-influencers local to minneapolis during protests related to the murder of george floyd were, on average, more likely to be retweeted than local mesoand macro-influencers with significantly larger networks. this goes against the expectation that a more extensive network size corresponds to an increase in information dissemination during crises. however, the prominence of micro-influencers in information dissemination pertaining to local crises carries significant implications. on one hand, their role is crucial in disseminating timely and relevant information to affected communities. however, on the other hand, it is important to recognize that micro-influencers also possess the potential to propagate misinformation and disinformation during crises. misinformation, which refers to the unintentional dissemination of false information, and disinformation, which involves the deliberate spread of false information, both contribute to problematic dynamics during crises, primarily due to their impact on public opinion (tran et al., 2021). an example can be seen in the case of george floyd, where discussions surrounding social injustice frequently cited the black lives matter movement. however, intense online discourse also gave rise to the oppositional movement, all lives matter, leading to polarization and further division within public sentiment (carney, 2016). given the influential role of micro-influencers in shaping public opinion during crises, it becomes imperative for government agencies, policymakers, and social media platforms to acknowledge their significance and take appropriate measures. in particular, these stakeholders should ensure that micro-influencers, who possess heightened information dissemination potential, are encouraged to cite factual sources when creating and disseminating content online during an unfolding crisis. to increase accurate information dissemination social media platforms should also consider incorporating behavioral nudges and source ratings (a. kim et al., 2019) primarily targeted at this group. this is especially crucial when micro-influencers engage with emergent crisis-related hashtags that rapidly spread to a large population in a short timeframe. strategies aimed at promoting information accuracy, fact-checking, and promoting critical thinking skills among both micro-influencers australasian journal of information systems kishore & errmann 2024, vol 28, research article a social media analytics approach 18 and their audiences can be instrumental in mitigating the adverse effects of false or misleading information during crises. 6.4 limitations and future research this research should be viewed considering its intrinsic limitations. while the dataset is complete and replicable as all data was collected from the twitter api, the api only provides access to public tweets. therefore, the tweets of users with private accounts were not available for data collection and analysis. in addition, this study focuses on average retweets per day per 10,000 followers to reveal interpretable results. while statistical results were consistent when data was aggregated at different levels, mesoand macro-influencers may possess hundreds of thousands, if not millions, of followers. by using an aggregate approach, their relative influence was potentially diluted. nevertheless, this research shows that possessing a large online following does not ensure that followers will reshare content at the same rate as micro-influencers, which may possess a smaller, yet more engaged, followership (kwak et al., 2010). furthermore, as only original tweets were collected, we are unable to analyze the network dynamics at play. future research should consider collecting retweets in a similar context to analyze the different types of networks influencers engage with. the authors consciously decided to exclude a control group. this would usually make the results difficult to justify, however, this has been accommodated for by evaluating multiple case studies. in addition, location information on twitter is often limited to a small percentage of public user accounts. furthermore, as location information can be manually edited, some users may list incorrect location information on their profiles (burns & eltham, 2009). however, this is unlikely to occur on a large scale. as each case utilized a single hashtag during data collection, it is likely that a proportion of relevant tweets were excluded. furthermore, while ukrainian is the dominant language in ukraine (translators without borders, 2022), we chose to focus on english tweets only. as this study’s aim is to compare trends in influencer types rather than identify influential individuals, we narrowed our data collection scope to maintain comparability and consistency. future studies focusing on networks or individuals should consider an iterative approach to identify all relevant parameters (kishore et al., 2022). future research should also consider the relationship between temporality and distinct categories of influencers. temporality, or time, is an important, yet understudied, construct in is (venkatesh et al., 2021). micro-influencers locally situated to an unfolding crisis may be quicker to post content on social media as they are experiencing the crisis first-hand. in some contexts, they may be reporting breaking news and news updates before traditional media agencies and news reporters which may partially explain why their content is being retweeted significantly more often in local networks. this finding is in line with current developing research (kotlarsky et al., 2022), however, this relationship requires further examination. in addition, delving into multi-step dissemination offers an additional avenue for future research examining the temporal dynamics of networks. given the potential for tweets to be retweeted by various types of influencers and users, the original tweet may reach a considerably larger network of followers over time, leading to subsequent retweets. consequently, investigating the roles of influencers, both situated locally and globally, within the context of multi-step dissemination may yield promising insights on the dynamics at play within networks. australasian journal of information systems kishore & errmann 2024, vol 28, research article a social media analytics approach 19 interestingly, alternative research also shows that virality is higher for misinformation, novel tweets, and tweets with negative sentiment (king & wang, 2021). specific to the cases we explored, misinformation during crisis events is a risk due to complexity of the information, and further, polarization among networks may occur when there are large networks with multiple actor motivations (törnberg, 2018). although our research focused specifically on how information diffuses amongst networks, it is of great interest to explore risks of misinformation within information diffusion, as well as the centrality of key actors and what role they play in this situation. future research may also explore the content of influencer tweets, rather than the effectiveness of information diffusion, to understand if different types of influencers are more adept at spreading misinformation or negative sentiment. it would be revelatory to apply natural language processing (nlp) methods to understand how discourse is evolving over time during crisis events, and how this discourse varies between regionalities and influencer types. 7 conclusion this paper investigates how different types of influencers (micro, meso, and macro) participating in different public crisis event geographies (regional, national, and international) and different network localities (local and global) effects information diffusion on the social media platform, twitter. online information diffusion plays a critical role during public crisis events. we show (study 1) that there is significantly more information diffusion in regional (vs. national or international) crisis events when tweeted by a micro-influencer (vs. mesoinfluencers or macro-influencers). further, study 2 demonstrates this pattern holds when micro-influencers operate in local networks (are located local to the crisis), however, effects become attenuated when the micro-influencer is situated in a global network (outside of the locality of the event). we term this effect ‘influencer network compression’ – the smaller in scope a crisis event geography and the network becomes, the more effective micro-influencers are at diffusing information. in summary, this research sheds light on how micro-influencers have the power to do big things in a small way. references abedin, b., & babar, a. (2018). institutional vs. non-institutional use of social media during emergency response: a case of twitter in 2014 australian bush fire. information systems frontiers, 20(4), 729–740. https://doi.org/10.1007/s10796-017-9789-4 alampi, a. (2019). the future is micro: how to build an effective micro-influencer programme. journal of digital & social media marketing, 7(3), 203–208. aleti, t., harrigan, p., cheong, m., & turner, w. (2016). an investigation of how the australian brewing industry influence consumers on twitter. australasian journal of information systems, 20. https://doi.org/10.3127/ajis.v20i0.1350 ap news. (2022, october 23). timeline of events since george floyd’s arrest and murder. ap news. https://apnews.com/article/death-of-george-floyd-minneapolis-police-reformd109eb0d3094119ddbcb560676467f19 bachura, e., valecha, r., chen, r., & rao, h. r. (2022). the opm data breach: an investigation of shared emotional reactions on twitter. mis quarterly, 46(2), 881–910. https://doi.org/10.25300/misq/2022/15596 australasian journal of information systems kishore & errmann 2024, vol 28, research article a social media analytics approach 20 bang, c. c., lee, j., & rao, h. r. (2021). the egyptian protest movement in the twittersphere: an investigation of dual sentiment pathways of communication. international journal of information management, 58. https://doi.org/10.1016/j.ijinfomgt.2021.102328 bastos, m., & farkas, j. (2019). “donald trump is my president!”: the internet research agency propaganda machine. social media+ society, 5(3). https://doi.org/10.1177/2056305119865466 bastos, m., mercea, d., & baronchelli, a. (2018). the geographic embedding of online echo chambers: evidence from the brexit campaign. plos one, 13(11), e0206841. https://doi.org/10.1371/journal.pone.0206841 berente, n., seidel, s., & safadi, h. (2019). research commentary—data-driven computationally intensive theory development. information systems research, 30(1), 50– 64. https://doi.org/10.1287/isre.2018.0774 boerman, s. c. (2020). the effects of the standardized instagram disclosure for micro-and meso-influencers. computers in human behavior, 103, 199–207. https://doi.org/10.1016/j.chb.2019.09.015 bonnevie, e., rosenberg, s. d., kummeth, c., goldbarg, j., wartella, e., & smyser, j. (2020). using social media influencers to increase knowledge and positive attitudes toward the flu vaccine. plos one, 15(10). https://doi.org/10.1371/journal.pone.0240828 boyd, d., golder, s., & lotan, g. (2010). tweet, tweet, retweet: conversational aspects of retweeting on twitter. 43rd hawaii international conference on system sciences, 1–10. https://doi.org/10.1109/hicss.2010.412 brown, s. (2022). in russia–ukraine war, social media stokes ingenuity, disinformation. mit sloan school of management. https://mitsloan.mit.edu/ideas-made-to-matter/russia-ukrainewar-social-media-stokes-ingenuity-disinformation buchanan, l., bui, q., & patel, j. k. (2020). black lives matter may be the largest movement in us history. the new york times, 3. https://www.nytimes.com/interactive/2020/07/03/us/george-floyd-protests-crowdsize.html burns, a., & eltham, b. (2009). twitter free iran: an evaluation of twitter’s role in public diplomacy and information operations in iran’s 2009 election crisis. communications policy and research forum, sydney, australia. https://vuir.vu.edu.au/15230/1/cprf09burnseltham.pdf campbell, c., & farrell, j. r. (2020). more than meets the eye: the functional components underlying influencer marketing. business horizons, 63(4), 469–479. https://doi.org/10.1016/j.bushor.2020.03.003 carney, n. (2016). all lives matter, but so does race: black lives matter and the evolving role of social media. humanity & society, 40(2), 180–199. https://doi.org/10.1177/0160597616643868 chen, e., & ferrara, e. (2022). tweets in time of conflict: a public dataset tracking the twitter discourse on the war between ukraine and russia. arxiv preprint arxiv:2203.07488. https://doi.org/10.48550/arxiv.2203.07488 australasian journal of information systems kishore & errmann 2024, vol 28, research article a social media analytics approach 21 choi, s. (2015). the two-step flow of communication in twitter-based public forums. social science computer review, 33(6), 696–711. https://doi.org/10.1177/0894439314556599 cohen, r. (2022). a surge of unifying moral outrage over russia’s war. the new york times. https://www.nytimes.com/2022/03/01/world/europe/zelensky-ukraine-waroutrage.html cucinotta, d., & vanelli, m. (2020). who declares covid-19 a pandemic. acta bio medica: atenei parmensis, 91(1), 157. https://doi.org/10.23750/abm.v91i1.9397 de choudhury, m., sundaram, h., john, a., seligmann, d. d., & kelliher, a. (2010). “birds of a feather”: does user homophily impact information diffusion in social media? arxiv preprint arxiv:1006.1702., 1–31. https://doi.org/10.48550/arxiv.1006.1702 farivar, s., wang, f., & yuan, y. (2021). opinion leadership vs. para-social relationship: key factors in influencer marketing. journal of retailing and consumer services, 59. https://doi.org/10.1016/j.jretconser.2020.102371 fichet, e. s., robinson, j. j., dailey, d., & starbird, k. (2016). eyes on the ground: emerging practices in periscope use during crisis events. thirteen's international conference on information systems for crisis response and management (iscram 2016), 1–10. firdaus, s. n., ding, c., & sadeghian, a. (2018). retweet: a popular information diffusion mechanism–a survey paper. online social networks and media, 6, 26–40. https://doi.org/10.1016/j.osnem.2018.04.001 garner, i. (2022). how is the war going for putin on social media? not great. washington post. https://www.washingtonpost.com/outlook/2022/03/04/russian-social-media-ukrainevk-propaganda/ grisold, t., kremser, w., mendling, j., recker, j., vom brocke, j., & wurm, b. (2023). generating impactful situated explanations through digital trace data. journal of information technology. https://doi.org/10.1177/02683962231208724 guille, a., hacid, h., favre, c., & zighed, d. a. (2013). information diffusion in online social networks: a survey. acm sigmod record, 42(2), 17–28. https://doi.org/10.1145/2503792.2503797 hagar, c. (2011). crisis information management: communication and technologies. amsterdam, the netherlands: elsevier. harrigan, p., daly, t. m., coussement, k., lee, j. a., soutar, g. n., & evers, u. (2021). identifying influencers on social media. international journal of information management, 56, 102246. https://doi.org/10.1016/j.ijinfomgt.2020.102246 hodis, m. a., sriramachandramurthy, r., & sashittal, h. c. (2015). interact with me on my terms: a four segment facebook engagement framework for marketers. journal of marketing management, 31(11–12), 1255–1284. https://doi.org/10.1080/0267257x.2015.1012535 hogan, b. (2008). analyzing social networks via the internet. in the sage handbook of online research methods (pp. 141–160). new york, ny, usa: sage publications. https://doi.org/10.4135/9780857020055 australasian journal of information systems kishore & errmann 2024, vol 28, research article a social media analytics approach 22 hu, l., min, q., han, s., & liu, z. (2020). understanding followers’ stickiness to digital influencers: the effect of psychological responses. international journal of information management, 54, 102169. https://doi.org/10.1016/j.ijinfomgt.2020.102169 huang, y., shen, c., & contractor, n. s. (2013). distance matters: exploring proximity and homophily in virtual world networks. decision support systems, 55(4), 969–977. https://doi.org/10.1016/j.dss.2013.01.006 janssen, m., & van der voort, h. (2020). agile and adaptive governance in crisis response: lessons from the covid-19 pandemic. international journal of information management, 55(june), 102180. https://doi.org/10.1016/j.ijinfomgt.2020.102180 jones, s. g. (2013). the mirage of the arab spring: deal with the region you have, not the region you want. foreign affairs; heinonline. https://www.foreignaffairs.com/articles/syria/201212-03/mirage-arab-spring kar, a. k., & dwivedi, y. k. (2020). theory building with big data-driven research–moving away from the “what” towards the “why.” international journal of information management, 54, 102205. https://doi.org/10.1016/j.ijinfomgt.2020.102205 katz, e. (1957). the two-step flow of communication: an up-to-date report on an hypothesis. public opinion quarterly, 21(1), 61–78. https://doi.org/10.1086/266687 katz, e., & lazarsfeld, p. f. (1964). personal influence. piscataway, nj, usa: transaction publishers. kay, s., mulcahy, r., & parkinson, j. (2020). when less is more: the impact of macro and micro social media influencers’ disclosure. journal of marketing management, 36(3–4), 248–278. https://doi.org/10.1080/0267257x.2020.1718740 kim, a., moravec, p. l., & dennis, a. r. (2019). combating fake news on social media with source ratings: the effects of user and expert reputation ratings. journal of management information systems, 36(3), 931–968. https://doi.org/10.1080/07421222.2019.1628921 kim, j., bae, j., & hastak, m. (2018). emergency information diffusion on online social media during storm cindy in u.s. international journal of information management, 40(january), 153–165. https://doi.org/10.1016/j.ijinfomgt.2018.02.003 kim, j., & hastak, m. (2018). social network analysis: characteristics of online social networks after a disaster. international journal of information management, 38(1), 86–96. https://doi.org/10.1016/j.ijinfomgt.2017.08.003 king, k. k., & wang, b. (2021). diffusion of real versus misinformation during a crisis event: a big data-driven approach. international journal of information management, 102390. https://doi.org/10.1016/j.ijinfomgt.2021.102390 kishore, s., peko, g., & sundaram, d. (2019). looking through the twitter glass: bridging the data– researcher gap. proceedings of the americas conference on information systems, 8–16. https://aisel.aisnet.org/amcis2019/social_computing/social_computing/4 kishore, s., sundaram, d., & myers, m. (2022). retweeting my feelings? exploring the temporal effects of the covid-19 pandemic on social media use. proceedings of the australasian conference on information systems. australasian conference on information systems, melbourne, australia. https://aisel.aisnet.org/acis2022/56/ australasian journal of information systems kishore & errmann 2024, vol 28, research article a social media analytics approach 23 kotlarsky, j., kishore, s., branicki, l., & sullivan-taylor, b. (2022). how the world understood the white island eruption: the tales that different social media actors tell us. proceedings of the european conference on information systems. european conference on information systems, timișoara, romania. https://aisel.aisnet.org/ecis2022_rip/17/ kwak, h., lee, c., park, h., & moon, s. (2010). what is twitter, a social network or a news media? proceedings of the 19th international conference on world wide web, 591–600. https://doi.org/10.1145/1772690.1772751 laylavi, f., rajabifard, a., & kalantari, m. (2017). event relatedness assessment of twitter messages for emergency response. information processing & management, 53(1), 266–280. https://doi.org/10.1016/j.ipm.2016.09.002 lazarsfeld, p. f., berelson, b., & gaudet, h. (1968). the people’s choice: how the voter makes up his mind in a presidential campaign. new york, ny, usa: columbia university press. li, j., & rao, h. r. (2010). twitter as a rapid response news service: an exploration in the context of the 2008 china earthquake. the electronic journal of information systems in developing countries, 42(1), 1–22. https://doi.org/10.1002/j.1681-4835.2010.tb00300.x liu, f., & xu, d. (2018). social roles and consequences in using social media in disasters: a structurational perspective. information systems frontiers, 20, 693–711. https://doi.org/10.1007/s10796-017-9787-6 martínez-lópez, f. j., anaya-sánchez, r., fernández giordano, m., & lopez-lopez, d. (2020). behind influencer marketing: key marketing decisions and their effects on followers’ responses. journal of marketing management, 36(7–8), 579–607. https://doi.org/10.1080/0267257x.2020.1738525 martínez-rojas, m., pardo-ferreira, m. del c., & rubio-romero, j. c. (2018). twitter as a tool for the management and analysis of emergency situations: a systematic literature review. international journal of information management, 43(april), 196–208. https://doi.org/10.1016/j.ijinfomgt.2018.07.008 mcpherson, m., smith-lovin, l., & cook, j. m. (2001). birds of a feather: homophily in social networks. annual review of sociology, 27(1), 415–444. https://doi.org/10.1146/annurev.soc.27.1.415 mehta, a. m., bruns, a., & newton, j. (2017). trust, but verify: social media models for disaster management. disasters, 41(3), 549–565. https://doi.org/10.1111/disa.12218 miranda, s., berente, n., seidel, s., safadi, h., & burton-jones, a. (2022). computationally intensive theory construction: a primer for authors and reviewers. mis quarterly, 46(2). https://misq.umn.edu/misq/downloads/download/editorial/756/ mirbabaie, m., bunker, d., stieglitz, s., & deubel, a. (2020). who sets the tone? determining the impact of convergence behaviour archetypes in social media crisis communication. information systems frontiers, 22(2), 339–351. https://doi.org/10.1007/s10796-019-09917-x mirbabaie, m., bunker, d., stieglitz, s., marx, j., & ehnis, c. (2020). social media in times of crisis: learning from hurricane harvey for the coronavirus disease 2019 pandemic response. journal of information technology, 35(3), 195–213. https://doi.org/10.1177/0268396220929258 australasian journal of information systems kishore & errmann 2024, vol 28, research article a social media analytics approach 24 nguyen, t. t., criss, s., michaels, e. k., cross, r. i., michaels, j. s., dwivedi, p., huang, d., hsu, e., mukhija, k., & nguyen, l. h. (2021). progress and push-back: how the killings of ahmaud arbery, breonna taylor, and george floyd impacted public discourse on race and racism on twitter. ssm-population health, 15, 100922. https://doi.org/10.1016%2fj.ssmph.2021.100922 nhs england. (2021). reducing burden and releasing capacity to manage the covid-19 pandemic. https://www.england.nhs.uk/coronavirus/documents/reducing-burden-and-releasingcapacity-to-manage-the-c-19-pandemic/ office for national statistics. (2020). coronavirus (covid-19): 2020 in charts. https://www.ons.gov.uk/peoplepopulationandcommunity/healthandsocialcare/conditi onsanddiseases/articles/coronaviruscovid192020incharts/2020-12-18 oh, o., agrawal, m., & rao, h. r. (2013). community intelligence and social media services: a rumor theoretic analysis of tweets during social crises. mis quarterly, 37(2), 407–426. https://www.jstor.org/stable/43825916 palen, l., vieweg, s., liu, s. b., & hughes, a. l. (2009). crisis in a networked world: features of computer-mediated communication in the april 16, 2007, virginia tech event. social science computer review, 27(4), 467–480. https://doi.org/10.1177/0894439309332302 park, j., lee, j. m., xiong, v. y., septianto, f., & seo, y. (2021). david and goliath: when and why micro-influencers are more persuasive than mega-influencers. journal of advertising, 50(5), 584–602. https://doi.org/10.1080/00913367.2021.1980470 pentland, b., vaast, e., & wolf, j. r. (2021). theorizing process dynamics with directed graphs: a diachronic analysis of digital trace data. mis quarterly, 45(2), 967–984. https://doi.org/10.25300/misq/2021/15360 rao, h. r., vemprala, n., akello, p., & valecha, r. (2020). retweets of officials’ alarming vs reassuring messages during the covid-19 pandemic: implications for crisis management. international journal of information management, 55, 102187. https://doi.org/10.1016/j.ijinfomgt.2020.102187 rathore, a. k., maurya, d., & srivastava, a. k. (2021). do policymakers use social media for policy design? a twitter analytics approach. australasian journal of information systems, 25. https://doi.org/10.3127/ajis.v25i0.2965 reuter, c., kaufhold, m.-a., schmid, s., spielhofer, t., & hahne, a. s. (2019). the impact of risk cultures: citizens’ perception of social media use in emergencies across europe. technological forecasting and social change, 148(1), 1–17. https://doi.org/10.1016/j.techfore.2019.119724 robinson, b. (2020). towards an ontology and ethics of virtual influencers. australasian journal of information systems, 24. https://doi.org/10.3127/ajis.v24i0.2807 roxby, p., butcher, b., & england, r. (2020). pressure on hospitals “at a really dangerous point.” bbc. https://www.bbc.com/news/health-55362681 roy, k. c., hasan, s., sadri, a. m., & cebrian, m. (2020). understanding the efficiency of social media based crisis communication during hurricane sandy. international journal of information management, 52(december 2019), 102060. https://doi.org/10.1016/j.ijinfomgt.2019.102060 https://doi.org/10.3127/ajis.v24i0.2807 australasian journal of information systems kishore & errmann 2024, vol 28, research article a social media analytics approach 25 sample, i. (2021). covid timeline: the weeks leading up to first uk lockdown. the guardian. https://www.theguardian.com/world/2021/oct/12/covid-timeline-the-weeks-leadingup-to-first-uk-lockdown schneider, s., & check, p. (2010). read all about it: the role of the media in improving construction safety and health. journal of safety research, 41(3), 283–287. https://doi.org/10.1016/j.jsr.2010.05.001 scott, m. (2022). as war in ukraine evolves, so do disinformation tactics. politico. https://www.politico.eu/article/ukraine-russia-disinformation-propaganda/ singh, j., crisafulli, b., quamina, l. t., & xue, m. t. (2020). ‘to trust or not to trust’: the impact of social media influencers on the reputation of corporate brands in crisis. journal of business research, 119(january 2019), 464–480. https://doi.org/10.1016/j.jbusres.2020.03.039 starbird, k., & palen, l. (2011). “voluntweeters” self-organizing by digital volunteers in times of crisis. proceedings of the sigchi conference on human factors in computing systems, 1071–1080. https://doi.org/10.1145/1978942.1979102 starbird, k., & palen, l. (2010). pass it on?: retweeting in mass emergency. proceedings of the 7th international information systems for crisis response and management (iscram 2010). https://idl.iscram.org/files/starbird/2010/970_starbird+palen2010.pdf statista. (2022a). leading countries based on number of twitter users as of january 2022. https://www.statista.com/statistics/242606/number-of-active-twitter-users-in-selectedcountries/ statista. (2022b). number of monetizable daily active twitter users (mdau) worldwide from 1st quarter 2017 to 1st quarter 2022. https://www.statista.com/statistics/970920/monetizabledaily-active-twitter-users-worldwide/ stieglitz, s., bunker, d., mirbabaie, m., & ehnis, c. (2018). sense-making in social media during extreme events. journal of contingencies and crisis management, 26(1), 4–15. https://doi.org/10.1111/1468-5973.12193 stieglitz, s., & dang-xuan, l. (2012). social media and political communication: a social media analytics framework. social network analysis and mining, 3(4), 1277–1291. https://doi.org/10.1007/s13278-012-0079-3 taylor, d. b. (2020). george floyd protests: a timeline. the new york times, 10. https://www.nytimes.com/article/george-floyd-protests-timeline.html thomson, r., ito, n., suda, h., lin, f., liu, y., hayasaka, r., isochi, r., & wang, z. (2012). trusting tweets: the fukushima disaster and information source credibility on twitter. proceedings of the 9th information systems for crisis response and management (iscram 2012). https://idl.iscram.org/files/thomson/2012/216_thomson_etal2012.pdf tim, y., pan, s. l., ractham, p., & kaewkitipong, l. (2017). digitally enabled disaster response: the emergence of social media as boundary objects in a flooding disaster. information systems journal, 27(2), 197–232. https://doi.org/10.1111/isj.12114 törnberg, p. (2018). echo chambers and viral misinformation: modeling fake news as complex contagion. plos one, 13(9), e0203958. https://doi.org/10.1371/journal.pone.0203958 australasian journal of information systems kishore & errmann 2024, vol 28, research article a social media analytics approach 26 tran, t., valecha, r., rad, p., & rao, h. r. (2021). an investigation of misinformation harms related to social media during two humanitarian crises. information systems frontiers, 23(4), 931–939. https://doi.org/10.1007/s10796-020-10088-3 translators without borders. (2022). language data for ukraine. https://translatorswithoutborders.org/language-data-for-ukraine/ twitter. (2022). academic research access. https://developer.twitter.com/en/products/twitterapi/academic-research united nations office for the coordination of humanitarian affairs. (2022). update on idp figures in ukraine. https://www.unocha.org/publications/report/ukraine/update-idpfigures-ukraine-18-march-2022-enuk venkatesh, v., sykes, t. a., aljafari, r., & poole, m. s. (2021). the future is now: calling for a focus on temporal issues in information system research. industrial management & data systems, 121(1). https://doi.org/10.1108/imds-08-2020-0506 walker, m., & matsa, k. e. (2021). news consumption across social media in 2021. pew research center. https://www.pewresearch.org/journalism/2021/09/20/news-consumptionacross-social-media-in-2021/ wang, x. w., cao, y. m., & park, c. (2019). the relationships among community experience, community commitment, brand attitude, and purchase intention in social media. international journal of information management, 49(july), 475–488. https://doi.org/10.1016/j.ijinfomgt.2019.07.018 wang, y., hao, h., & platt, l. s. (2021). examining risk and crisis communications of government agencies and stakeholders during early-stages of covid-19 on twitter. computers in human behavior, 114, 106568. https://doi.org/10.1016/j.chb.2020.106568 xu, l., qiu, j., gu, w., & ge, y. (2020). the dynamic effects of perceptions of dread risk and unknown risk on sns sharing behavior during eid events: do crisis stages matter. journal of the association for information systems, 21(3), 545–573. http://dx.doi.org/10.17705/1jais.00612 zhang, c., fan, c., yao, w., hu, x., & mostafavi, a. (2019). social media for intelligent public information and warning in disasters: an interdisciplinary review. international journal of information management, 49(march), 190–207. https://doi.org/10.1016/j.ijinfomgt.2019.04.004 australasian journal of information systems kishore & errmann 2024, vol 28, research article a social media analytics approach 27 copyright: © 2024 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v28.4429 http://creativecommons.org/licenses/by-nc/3.0/au/ https://doi.org/10.3127/ajis.v28.4429 1 introduction 2 theoretical underpinning 2.1 information diffusion during crisis events 2.2 the role of influencers in information diffusion during crisis events 2.3 information diffusion in local and global influencer locations 3 regional, national, and international crisis events: case backgrounds 3.1 regional event: george floyd 3.2 national event: russo-ukrainian war 3.3 international event: covid-19 pandemic 4 method 5 results 5.1 study 1 5.2 study 2 6 discussion 6.1 general discussion 6.2 theoretical contributions 6.3 practical contributions 6.4 limitations and future research 7 conclusion paper title (paper title style) australasian journal of information systems zobair et al. 2021, vol 25, research article health seekers’ acceptance & adoption determinants 1 health seekers’ acceptance and adoption determinants of telemedicine in emerging economies khondker mohammad zobair griffith university, brisbane, australia k.zobair@griffith.edu.au louis sanzogni griffith university, brisbane, australia luke houghton, griffith university, brisbane, australia kuldeep sandhu griffith university, brisbane, australia md jahirul islam griffith university, brisbane, australia abstract this study investigates health seekers’ acceptance and adoption determinants of telemedicine services in a rural public hospital setting in an emerging economy using an adapted, extended technology acceptance model. the present study pursued synthesising a plethora of existing literature and contextualised the significance of seven broad categories of potential determinants that significantly affect patients’ acceptance and adoption intentions: perceived usefulness, perceived ease of use, self-efficacy, service quality, privacy and data security, social influence, and facilitating conditions. the partial least square structural equation modeling technique was employed to test the conceptual model and research hypotheses. a crosssectional survey was administered among 500 telemedicine users in randomly selected rural and remote areas of bangladesh. excluding self-efficacy and ease of use, five determinants expressively contributed to patients’ acceptance of telemedicine adoption, explaining 65% of the variance (r2) in behavioural intention. the empirical findings have the quality of rigour obtained from rich data sets in health informatics and can contribute to build telemedicine into an institutionalised health infrastructure in bangladesh and similar settings. pertinent implications, limitations and future research directions were recommended to secure the longterm sustainability of telemedicine healthcare projects. keywords: acceptance, ict, pls-sem, rural and remote areas, technology acceptance model, telemedicine 1 introduction telemedicine emerges as a vital healthcare provision for rural and remote communities in many emerging economies where specialist physicians rarely practice (zobair, sanzogni, sandhu, & islam, 2020). the widespread expansion of telemedicine potentially reduces health disparity between rural and urban areas (lerouge & garfield, 2013), increases users’ convenience (zobair, sanzogni, & sandhu, 2019), enhances accessibility and service quality, decreases healthcare costs (jansen-kosterink, dekker-van weering, & van velsen, 2019), and contributes to sustainable health care systems (whitten, holtz, & nguyen, 2010). while telemedicine healthcare services are well accepted by many clinicians and patients in both australasian journal of information systems zobair et al. 2021, vol 25, research article health seekers’ acceptance & adoption determinants 2 developed and emerging economies (greenberg et al., 2019; jansen-kosterink et al., 2019), their adoption into routine practices are isolated, scattered (lerouge, gupta, corpart, & arrieta, 2019) and slower than anticipated (taylor, coates, wessels, mountain, & hawley, 2015). telemedicine continues to gain significant recognition in providing specialised care to the rural and remote communities in bangladesh. this service has formally been integrated in 2010 into its’ public healthcare systems to deliver specialised care to rural and remote areas where around 70% of the population inhabits (darkwa, newman, kawkab, & chowdhury, 2015; zobair et al., 2019). despite bangladesh’s passion for public telemedicine schemes, the functional adoption into clinical practices has remained inadequate in rural settings. the incorporation of 27 specialised, and district-level medical colleges and hospitals consistently providing specialist care to 488 upazila (i.e., sub-district) public hospitals aided rural telemedicine centres is considered a momentous shift in the public healthcare sector in bangladesh (zobair et al., 2019). the underlying principle of personalised healthcare approaches reflects the control patients exercise over their healthcare progress rather than the physicians themselves (hur, cousins, & stahl, 2019). although many health providers facilitate a variety of ict supported services (i.e., telemedicine/ehealth, mhealth), patients can actively participate in choosing what is more appropriate and convenient to them (hur et al., 2019). hence, it is mandatory to understand patients’ evaluations of service quality that form their acceptance of the services. exploration of health seekers’ acceptance determinants in rural settings could support successful telemedicine implementation (evans, 2015). the technology acceptance model (tam) is a fitting theory for understanding health information technology acceptance and adoption (holden & karsh, 2010) in varieties of information-driven settings (i.e., telemedicine) that serve as the focus of this study. evidence suggests that, as modern healthcare services are becoming heavily reliant on technology, a better understanding of the determinants that contribute to acceptance and adoption of technology-mediated healthcare services (i.e., telemedicine) is now critical (jewer, 2018). understanding users' acceptance or rejection of technology, such as computers, appears to be one of the most challenging factors in information systems research (davis et al., 1989). given the importance of this issue, academic and practitioner literature has drawn on many behavioural theories, including the tam (agarwal et al., 2000). likewise, some other classic theories that can be called upon to provide varying explanations of consumers' continuity behaviours towards technology adoption such as, the theory of reasoned action; the diffusion of innovation theory; social cognitive theory; the theory of planned behaviour; the technology acceptance model; and the unified theory of acceptance and use of technology (hoque & sorwar, 2017; leung & chen, 2019). telemedicine health technology, like any innovative technology, faces critical issues related to stakeholders’ acceptance and adoption (zhou et al., 2019). abundant empirical evidence further suggested that patients’ past positive experiences and satisfaction affect their behavioural intention to use telemedicine services (zhou et al., 2019). in the present study, patients’ behavioural intentions to use telemedicine refers to acceptance and adoption (holden & karsh, 2010). past contributions concerning studies in bangladesh have mostly focused on m-health adoption (ahmed et al., 2014; alam, hoque, hu, & barua, 2020; hoque & sorwar, 2017), with a recent study (hoque & sorwar, 2017) focusing on e-health adoption in urban areas. telemedicine appears to be well accepted by rural health seekers in many developed countries australasian journal of information systems zobair et al. 2021, vol 25, research article health seekers’ acceptance & adoption determinants 3 (bros et al., 2018). conversely, its acceptance by rural health seekers in developing countries could be based on different determinants that affect patients’ behavioural intentions (lin & chang, 2018). the existing literature has not yet identified these determinants associated with telemedicine in rural bangladesh’ public hospital settings. little attention has been paid to exploring how public hospitals in bangladesh can successfully provide telemedicine support to rural communities. moreover, there have been little-reported investigations into telemedicine project initiation under the health authorities, representing a considerable knowledge gap. employing a theory focused approach to examine ways to determine patients’ acceptance/adoption of telemedicine could advance a rational understanding of how to sustain and leverage telemedicine deployment in rural bangladesh settings. motivated by prior technology adoption theory-based research, the current study aims to fill the literature gaps by empirically testing the suitability of the revised version of extended tam framework as applied to the perceptions of bangladeshi rural telemedicine patients. 2 theoretical model and hypotheses this study builds on prior research (davis, 1989; venkatesh, 2000a; venkatesh & bala, 2008; venkatesh & davis, 1996, 2000; venkatesh, morris, davis, & davis, 2003) by presenting behavioural attributes while maintaining the parsimony of the original theory. adapted from the theory of reasoned action, tam is an extensively applied model used for predicting users’ technology acceptance and usage (venkatesh, 2000a, 2000b) by evaluating their internal beliefs, attitudes, and behavioural intentions (davis, bagozzi, & warshaw, 1989; hu, chau, sheng, & tam, 1999; rho, young choi, & lee, 2014). venkatesh and davis (2000) further proposed an extended tam referred to as tam 2, which together with tam 3 (venkatesh & bala, 2008), present a complete nomological network of the determinants of an individual’ technology adoption and use (venkatesh & bala, 2008). although the tam was initially applied to health information technology (hit) adoption it has found its way into the health sector as a means to measure end-user reactions to hit (anderberg, eivazzadeh, & berglund, 2019). the present study’s rationale for the extension and integration of the tam theory (venkatesh & bala, 2008) was to establish a framework to predict health seekers’ telemedicine acceptance and adoption determinants in developing countries’ rural settings. cimperman, brenčič, and trkman (2016) argue that most of the prior studies often ignored some critical aspects concerning interdependency between human characteristics, technology, and socio-economic phenomena, which lead to low impact on healthcare practices. cimperman et al. (2016) further suggest that the previously developed tam theory should be extended by adding those factors for the plausibility of the existing theory. in a recent study, for example, zhou et al. (2019) adopted an extended tam to explore the determinants of the behavioural intention of telehealth adoption among older adults in china. synthesising prior tam studies related to the health it adoption, we developed an extended tam (venkatesh & bala, 2008). a plethora of telemedicine, e-health, and m-health studies (alam et al., 2020; chau & hu, 2002; hoque & sorwar, 2017; zhou et al., 2019), provide a solid theoretical foundation for the extended tam. jansen-kosterink et al. (2019), for instance, revealed that tam was employed for examining users’ telemedicine acceptance and could explain up to 70% of the variance in individuals’ acceptance intentions. coherently and cogently to this background, this study adopted seven common categories of determinants adapted from prior studies (see appendix a) for predicting health seekers’ telemedicine acceptance and adoption intention in rural bangladesh’s public hospital settings (see figure 1). australasian journal of information systems zobair et al. 2021, vol 25, research article health seekers’ acceptance & adoption determinants 4 figure 1. research model 2.1 self-efficacy self-efficacy comprises individuals’ judgements of their abilities to perform given levels of performance and control over the events (bandura, o'leary, taylor, gauthier, & gossard, 1987). an individual’s perceived self-efficacy has a direct influence on his/her choice of activities (bandura, 1977). within the telemedicine context, self-efficacy refers to the extent of an individual’s judgment concerning his or her capability to attain a certain degree of telemedicine service performance (zobair et al., 2019). although, it does not explicitly comprise patients’ perceptions of their skills in physical experiences of telemedicine technology (zobair et al., 2019). perceived self-efficacy in telemedicine combines the patients’ assessment of the effectiveness of services performance and the ability or competency to conduct the behaviour (zhang et al., 2017). for example, within the health context, bandura (1986) confirmed that perceived self-efficacy arbitrates patients’ health behaviours. bandura et al. (1987) found that perceived self-efficacy has positively influenced patients’ coping mechanisms related to pain alleviation. within the health context, williams and bond (2002) confirmed that self-efficacy is the strongest predictor of diabetic patients’ behavioural intention. self-efficacy comprises a multiplicity of skills that affect individuals’ belief that they can execute particular tasks in specific situations (rosenstock, strecher, & becker, 1988), such as information systems acceptance and adoption (lending & dillon, 2007). consistent with prior studies, it is predicted that highly efficacious patients incline to have a high intention to accept telemedicine services (zobair et al., 2019). therefore, the following hypothesis is proposed: australasian journal of information systems zobair et al. 2021, vol 25, research article health seekers’ acceptance & adoption determinants 5 h1: self-efficacy positively influences patients’ behavioural intentions to use telemedicine health services. 2.2 perceived usefulness perceived usefulness is a consistently strong acceptance determinant for understanding behavioural intention to adopt the technology (venkatesh et al., 2003). perceived usefulness is the degree to which an individual believes that using a system will improve his/her job performance (venkatesh, 2000a; venkatesh & davis, 2000). the usefulness of a system (e.g., service) is strictly related to its effectiveness (bettiga, lamberti, & lettieri, 2019), while telemedicine is a system/service. in the context of telemedicine, patients evaluate telemedicine usefulness by giving more importance to the health outcomes achieved from the services based on their expectations. in this study, perceived usefulness is the degree to which a patient believes that using telemedicine healthcare services will improve his/her mental or physical health conditions. rho et al. (2014) acknowledged that perceived usefulness has a positive effect on behavioural intention to use telemedicine services. similarly, in hong kong, perceived usefulness was a significant determinant in physicians’ acceptance of telemedicine technology in hospitals (hu et al., 1999). consistent with prior studies, it is anticipated that when a patient’s perception of telemedicine’s usefulness is high, the patient is more likely to use telemedicine healthcare services. therefore, the following hypothesis is proposed: h2: perceived usefulness positively influences patients’ behavioural intentions to use telemedicine health services. 2.3 perceived ease of use perceived ease of use refers to ‘the extent to which a person believes that using the system will be free of an effort’ (venkatesh & davis, 2000, p. 117). in the present study, perceived ease of use is defined as the extent to how much a patient believes that using telemedicine healthcare services will be effortless. healthcare studies using the tam identified a significant positive effect of perceived ease of use on perceived usefulness (dünnebeil, sunyaev, blohm, leimeister, & krcmar, 2012). conversely, rather than perceived ease of use, perceived usefulness has a substantial effect on behavioural intention towards telemedicine healthcare services (aggelidis & chatzoglou, 2009; hu et al., 1999) usage. this is congruent with venkatesh (2000a), who found that perceived ease of use positively affects users’ technology acceptance and intention via perceived usefulness. a recent study by bettiga et al. (2019) confirmed that peu has a substantial impact on predicting patients’ behavioural intention in health technology adoption. another survey by woo and dowding (2018) acknowledged that ease of use significantly affects patients’ acceptance of telehealth services. therefore, the following hypothesis is proposed: h3: perceived ease of use positively influences patients’ behavioural intentions to use telemedicine health services. 2.4 facilitating conditions facilitating conditions refers to the ‘degree to which a person believes that an organisational and technical infrastructure exists to support to use the system’ (venkatesh et al., 2003, p. 453). it incorporates three characteristics—perceived behavioural control, facilitating conditions and compatibility (venkatesh et al., 2003)—that have a positive impact on behavioural control and behaviour (venkatesh, thong, & xu, 2012). a study by aggelidis and chatzoglou (2009) revealed that facilitating conditions include resources and technologies that positively affect australasian journal of information systems zobair et al. 2021, vol 25, research article health seekers’ acceptance & adoption determinants 6 ict-supported information systems adoption. macedo (2017), for instance, confirmed that facilitating conditions have a positive effect on behavioural intention to adopt ict mediated services. within the e-health (i.e., telehealth) context, cimperman et al. (2016) found that promoting high facilitating conditions increases behavioural intention to use telehealth services. similarly, another recent study by hossain, quaresma, and rahman (2019) asserted that facilitating conditions have positive effects on behavioural intention to use e-health services. therefore, the following hypothesis is proposed: h4: facilitating conditions positively influence patients’ behavioural intentions to use telemedicine health services. 2.5 social influence social influence refers to the extent of how individuals behave in the way they are directly or indirectly influenced by the feelings, thoughts, and actions of others (vries, backbier, kok, & dijkstra, 1995). within the context of ict, social influence can be defined as ‘the degree to which an individual perceives that important others believe he or she should use the new system’ (venkatesh et al., 2003, p. 451) and whether others expect that individuals to execute a behaviour (zhang et al., 2017). another study by lewis, agarwal, and sambamurthy (2003, p. 662) defines social influence as the ‘perceived social pressure to perform or not to perform the particular behaviour’, indicating higher perceptions of social pressure leads to a high behavioural intention to adopt technologies. for example, in the context of telemedicine, cimperman et al. (2016) revealed that social influence strongly affects patients’ behavioural intention to use telemedicine healthcare services. macedo (2017) echoed that social influence considerably influences users’ behavioural intention. within the m-health adoption context, hoque, bao, and sorwar (2017) confirmed that social influence has a substantial impact on behavioural intention to use m-health services in bangladesh. therefore, the following hypothesis is proposed: h5: social influence positively influences patients’ behavioural intentions to use telemedicine health services. 2.6 service quality service quality is defined as ‘a measure of how well the service level delivered matches customer expectations’ (parasuraman, zeithaml, & berry, 1985, p. 42) and is a broadly used— but challenging (boscarino, 1992)—healthcare service characteristic for detecting its strengths and weaknesses (kettinger & lee, 1997). service quality encompasses users’ perceived service satisfaction (kettinger & lee, 1997), reflecting that high quality of service contributes to high service satisfaction (k.-h. kim, kim, lee, & kim, 2019), positive effects, and organisational accomplishment (delone & mclean, 2003). zeithaml, berry, and parasuraman (1996) confirmed that service quality has a strong influence on customers behavioural intention. another study by parasuraman, zeithaml, and berry (1985b) revealed that most services are intangible; for example, service quality has been significantly affecting physicians’ acceptance of health information systems (his) in a hospital setting (chen & hsiao, 2012), as well as users’ behavioural intentions to use m-health services (akter, d’ambra, & ray, 2010; k.-h. kim et al., 2019). therefore, the following hypothesis is proposed: h6: service quality positively influences patients’ behavioural intentions to use telemedicine health services. australasian journal of information systems zobair et al. 2021, vol 25, research article health seekers’ acceptance & adoption determinants 7 2.7 privacy and data security health seekers’ privacy, security, and confidentiality in healthcare provision have been considered noticeable concerns that generated additional attraction in many recent empirical studies (esmaeilzadeh, 2019). within the e-health context, hall and mcgraw (2014) acknowledged that there are severe privacy and data security risks in telehealth (i.e., telemedicine) services that must be addressed to succeeding in its adoption. esmaeilzadeh (2019), for instance, revealed that prior empirical research emphasises significant shreds of evidence concerning patients’ privacy violations in healthcare services, such as unauthorised access to patients’ information or stolen medical data. these risks are increasing concerns in ict mediated healthcare systems due to insufficient control over data handling (hale & kvedar, 2014). hoque et al. (2017), for example, reminded that patients might not be confident in sharing health-related information with a third party due to fear of social disgrace and discrimination. these indicate that patients are more likely to accept telemedicine healthcare services if they perceive their privacy and information are well protected. a recent study by dutot, bergeron, rozhkova, and moreau (2019) noted that secured and well-protected systems corroborate with users’ attention to use healthcare services. similarly, xu (2019) confirmed that privacy concerns have substantial effects on patients’ trust, perceived privacy and confidentiality risk, and adoption intention. therefore, the following hypothesis is proposed: h7: privacy and data security positively influence patients’ behavioural intentions to use telemedicine health services. 3 research methodology 3.1 study population this study has drawn from a large project, which predominantly explores the determinants of barriers, facilitators, and antecedents to health seekers’ expectations of telemedicine adoption in rural public healthcare facilities in bangladesh. a cross-sectional survey was conducted in 2017 in three upazilas (sub-districts) telemedicine centres in bangladesh. for this study, 500 rural patients who received telemedicine services at least once from any selected telemedicine centres in the past 12 months constitute the sampling frame. the study excluded non-users because patients in bangladesh cannot access telemedicine services without a physician’s referral. 3.2 data collection detailed data collection procedures have been described previously (zobair et al., 2019). the sample was drawn from selected telemedicine centres using a multistage random sampling design. at first, three districts—pabna, khulna and satkhira—where telemedicine services are available were selected randomly. from these three districts, three upazila telemedicine centres—bera, dacope and devhata—were randomly selected as survey implementation site. from the patient lists collected from the selected telemedicine centres, 500 users were randomly selected, consisting of proportionate samples from bera (n=53), dacope (n=242), and devhata (n=205) upazila telemedicine centres. each group was statistically representative of the telemedicine population with commonalities in telemedicine infrastructure and clinical methods provided by the government. patients’ addresses and phone numbers were collected from the selected telemedicine centres. eligible individuals were contacted by phone and invited to participate in face-to-face australasian journal of information systems zobair et al. 2021, vol 25, research article health seekers’ acceptance & adoption determinants 8 interviews at a telemedicine centre during office hours. those unable to travel to the centres were asked to participate according to their convenience. a closed-form intervieweradministered questionnaire was used because several questions were relatively technical and would be hard for respondents with limited literacy to interpret on their own. the questionnaire was adopted in english, translated into the local language (i.e., bengali), pretested with bengali speakers before being administered, confirming its internal validity (islam et al., 2015). the study met the sampling target by obtaining 500 valid responses that were scrutinised for completeness. eight samples were excluded due to incomplete responses; 492 samples were preserved for analysis. the survey’s demographics are presented in table 1. measure items frequency percentage (%) gender male 206 41.9 female 286 58.1 age ≥18 and ≤20 64 13.0 ≥21 and ≤30 158 32.1 ≥31 and ≤40 106 21.5 ≥41 and ≤50 88 17.9 ≥51 76 15.4 education illiterate 68 13.8 primary 104 21.0 secondary 178 36.2 higher secondary 64 13.0 bachelor 51 10.4 masters and above 27 5.5 table 1. demographic characteristics of the sample (zobair et al., 2019) the project received approval from the directorate of general health services, bangladesh, and griffith university human research ethics committee. an informed consent was taken from each participant after informing them the purpose of the study, and their right to withdraw from the study at any time without consequence. they were ensured that their participation in this survey was completely voluntary. 3.3 construct measurements a seven-point likert scale was used (i.e., 1 = very strongly disagree to 7 = very strongly agree) for this study. a pre-test was conducted to examine the wording, sequence, length, and format of the questionnaire indicators. fifteen individuals with expertise in telemedicine in bangladesh were invited to pre-test the questionnaire and their feedback was used to rectify the questionnaire. the constructs, corresponding indicators and associated scales were further tested for content validity and reliability in a pilot study involving 25 telemedicine users representing 5% of the target sample (cresswell & clark, 2011). the scales used in this study are drawn from prior empirical studies related to behavioural science, information technology adoption, and particularly e-health (i.e., telemedicine) adoption research. item scales, their corresponding measures, and adapted sources are shown in the appendix a. these constructs were deemed significant for telemedicine research as they relate to the provision of timely responses, enhanced communication, and better access to services (lankton & wilson, 2007) to secure telemedicine services sustainability in rural settings in emerging economies. australasian journal of information systems zobair et al. 2021, vol 25, research article health seekers’ acceptance & adoption determinants 9 4 data analysis and results this study has chosen the partial least square (pls) approach for structural equation modeling (sem) (kock, 2018) instead of cb-sem as it is a more suitable approach for many reasons (ali, ali, badghish, & baazeem, 2018). for example, this method is a full-fledged variance-based approach ideal for the linear, non-linear, recursive, and non-recursive structural model (benitez, henseler, castillo, & schuberth, 2020). this is a more suitable choice for explorative studies and highly complex casual (i.e., confirmatory and explanatory) models (otter & beer, 2021). pls-sem has distinctive features that do not follow the restrictive assumptions imposed by cb-sem (ali et al., 2018; astrachan, patel, & wanzenried, 2014). compared to cbsem, it is a more robust approach with fewer identification issues and works with smaller and larger sample sizes (hair, ringle, & sarstedt, 2011). pls-sem can more effectively provide parameter estimates than cb-sem under appropriate circumstances (hair jr, hult, ringle, & sarstedt, 2016). for instance, hair, ringle, and sarstedt (2013) noted that the pls-sem representation of graphical model structures renders the model under investigation easier to manage and comprehend, allowing for rapid reconfigurations and real-time feedback. further, pls-sem can easily manage both reflective and formative measurement models (hair et al., 2011), including a single-item construct (hair jr et al., 2016). in contrast, model estimation with many latent constructs or indicators is often impossible with cb-sem (hair jr, hult, ringle, & sarstedt, 2017). pls-sem uses latent constructs scores as an exact linear combination of the observed indirect constructs (schubring, lorscheid, meyer, & ringle, 2016) and is more suited for theory building and predictive applications (gefen, straub, & boudreau, 2000). compared to cb-sem results, which can be highly imprecise when the assumptions are violated, pls-sem provides more robust estimations of the structural model (hair et al., 2011). besides, pls-sem has greater statistical power than cb-sem (hair et al., 2011). finally, the pls-sem method provides more robust estimations of the structural model and has greater statistical power than cb-sem (hair et al., 2011). this study’s model contains a single endogenous (i.e., dependent) latent construct (z/ η denotes bi) and seven exogenous (i.e., independent) latent constructs (si), (fc), (peu), (pu), (se), (pds) and (sq), which are indicated as ovals (hair et al., 2011). chin (1998) suggested that pls design assumes recursive (i.e., one way arrowed) relations among latent constructs and each endogenous latent construct η often termed casual chain systems of latent constructs that can be specified by equation 1. the mathematical equation of the structural model of this study is shown in equation 1. η = ʃβ1 ξ1+ ʃβ2 ξ2+ ʃβ3ξ3+ʃ β4 ξ4+ ʃβ5 ξ5+ʃ β6 ξ6+ ʃβ7 ξ7+ δ. [1] where η is an endogenous latent construct, β is a path coefficient, and ξ1 is an exogenous latent construct. literature indicates that all variance-based sem detects a nonzero path coefficient; however, the effect is originally zero (dijkstra & henseler, 2015). therefore, the mathematical equation of this model is yielding the following regression equation: z=+β1si+ β2fc+ β3peu+ β4pu+ β5se+ β6pds+ β7sq+e. [2] eq 2 charactierises the extended technology acceptance model process, where z is the outcome (behavioural intention) of social influence (si), facilitating conditions (fc), perceived ease of use (peu), perceived usefulness (pu), self-efficacy (se), privacy and data security (pds), and service quality (sq) latent constructs. all partial regression models are estimated iteratively (pls-sem algorithm) in two stages (hair et al., 2011). in the first stage, construct scores are australasian journal of information systems zobair et al. 2021, vol 25, research article health seekers’ acceptance & adoption determinants 10 estimated. then in the second stage, final estimates of the outer weights and loadings are calculated, with the structural model’s path coefficients and the resulting r2 values of the endogenous latent construct (hair jr et al., 2017). the dataset was analysed using pls-sem (muthupoltotage & gardner, 2018) largely consisting of latent constructs that require measurement by a set of indicators as proxies (hair jr et al., 2017). it incorporates two sets of linear equations—a measurement model specifying a construct and its corresponding indicators and a structural model specifying the relationship between exogenous and endogenous constructs and addressing the research questions and hypotheses (henseler, hubona, & ray, 2017). therefore, the pls-sem technique is considered a more rigorous analysis for the proposed research model (muthupoltotage & gardner, 2018). smartpls v.3.2.8 was utilized for data analysis, the pls (algorithm) tool for testing scales reliability, and a bootstrapping (algorithm) technique for hypotheses and structural model testing. 4.1 measurement model this research evaluates the reliability and validity of the reflective measurement model constructs (ringle, sarstedt, & straub, 2012) without choosing a composite model. a recent study by benitez et al. (2020) revealed that a reflective and casual-formative model is usually used for behavioural studies since a composite model is employed for artefacts. however, pls-sem deals with both latent variables (i.e., a construct that represents a behavioural concept such as individual behaviour, attitude, and personality traits) and emergent variables (i.e., a construct that represents an artifact) (benitez et al., 2020). in pls-sem, a composite model can be estimated by mod b; in turn, a reflective measurement model should be evaluated by mod a (benitez et al., 2020). the reason for choosing a reflective measurement model for this research is that the evaluation of mod a relies on a different set of criteria than its composite counterparts mod b (benitez et al., 2020). the measurement model was evaluated by testing internal consistency reliability, indicator reliability, convergent validity, and discriminant validity (hair et al., 2017). all outer loadings for each indicator in the model (see table 2) were higher than the threshold value of 0.707, suggesting that more than 50% of the variance in a single indicator can be explained by the corresponding latent construct, confirming indicator reliability (benitez et al., 2020; hair et al., 2017; roldán & sánchez-franco, 2012). our results show (see table 2) that the loadings range from 0.711 to 0.821, and all are significant on a 1‰ level, authenticating that the measures are reliable (benitez et al., 2020). both cronbach’s alpha and composite reliability >0.707 (see table 2) confirmed the model’s statistical significance and demonstrated strong evidence of internal consistency reliability (hair et al., 2017; roldán & sánchez-franco, 2012). the cronbach’s alpha and composite reliability range from 0.849 to 0.727 and 0.889 to 0.841, respectively, are above the threshold value, demonstrating reliable construct scores (benitez et al., 2020). convergent validity (≥.50) was assessed using the average variance extracted (ave) values for each construct (see table 2) (hair et al., 2017). ave values in our model range for 0.647 to 0.554 are higher than the threshold value of >.50, indicating significant convergent validity (benitez et al., 2020; henseler et al., 2017). the ave values for each construct explain the variance of more than half of their corresponding indicators, confirming convergent validity (henseler, ringle, & sinkovics, 2009; roldán & sánchez-franco, 2012). australasian journal of information systems zobair et al. 2021, vol 25, research article health seekers’ acceptance & adoption determinants 11 latent construct indicator code loading ave cronbach’s alpha composite reliability r2 r2 adj behavioural intention to use tm bi1 0.767*** 0.579 0.757 0.846 0.655 0.650 bi2 0.755*** bi3 0.800*** bi4 0.727*** facilitating conditions fc1 0.744*** 0.554 0.839 0.882 fc2 0.722*** fc3 0.755*** fc4 0.762*** fc5 0.726*** fc6 0.756*** privacy and data security pds1 0.796*** 0.647 0.727 0.846 pds2 0.805*** pds3 0.812*** perceived ease of use peu1 0.795*** 0.570 0.747 0.841 peu2 0.720*** peu3 0.790*** peu4 0.711*** perceived usefulness pu1 0.771*** 0.612 0.788 0.863 pu2 0.774*** pu3 0.818*** pu4 0.765*** self-efficacy se1 0.757*** 0.588 0.767 0.851 se2 0.757*** se3 0.752*** se4 0.801*** social influence si1 0.730*** 0.604 0.835 0.884 si2 0.729*** si3 0.787*** si4 0.821*** si5 0.813*** service quality sq1 0.776*** 0.571 0.849 0.889 sq2 0.741*** sq3 0.758*** sq4 0.741*** sq5 0.790*** sq6 0.726*** table 2. measurement model assessment note: †p<0.10, *p < 0.05, **p < 0.01, ***p < 0.001, one-tailed test. australasian journal of information systems zobair et al. 2021, vol 25, research article health seekers’ acceptance & adoption determinants 12 discriminant validity defines the extent to which a construct in a model is distinct from other constructs by empirical standards (henseler et al., 2009). a construct accepts more variance from its assigned items than from any other constructs (henseler et al., 2009). this study measured the distinctiveness of a latent construct using the heterotrait-monotrait ratio (htmt) criterion test. literature indicates that the htmt criterion is essential to evaluate the constructs' discriminant validity (benitez et al., 2020; hair jr, howard, & nitzl, 2020; otter & beer, 2021). the cut-off scores should be smaller than 0.85 or 0.90 to interpret the results (benitez et al., 2020; hair jr et al., 2020). the results (see table 3) demonstrate that htmt is significantly less than 0.85 or 0.90, authenticating that all measured constructs illustrated their discriminant validity (benitez et al., 2020) except two pairs. our results provided evidence that we established discriminant validity among all constructs. however, we cannot confirm discriminant validity between two pairs of constructs such as pds and bi, and peu and pu contain the htmt value of 0.937 and 0.981, respectively, suggesting a lack of discriminant validity. bi fc pds peu pu se si sq bi fc 0.846 pds 0.937 0.842 peu 0.868 0.820 0.859 pu 0.872 0.833 0.872 0.981 se 0.686 0.754 0.670 0.802 0.735 si 0.796 0.877 0.665 0.829 0.693 0.814 sq 0.890 0.779 0.871 0.838 0.838 0.748 0.716 table 3. heterotrait-monotrait ratio (htmt) for discriminant validity note: bi = behavioural intention to use telemedicine; fc = facilitating conditions; pds = privacy and data security; peu = perceived ease of use; pu = perceived usefulness; se = self-efficacy; sq = service quality; si = social influence. further, the bootstrapping technique applied for testing whether the htmt value is significantly less than 1 (hair jr et al., 2017). our results demonstrated that htmt is considerably less than 1 (benitez et al., 2020). we found that neither of the confidence intervals includes the value 1 (hair jr et al., 2017) except pds and bi, pu and peu, similar to the previous test. our results show that the lower and upper bounds of the confidence interval of htmt for the relationships between pds and bi are 0.860 and 1.011, pu and peu are 0.921 and 1.037, indicating that these constructs in the path model are more distinct, suggesting a lack of discriminant validity (hair jr et al., 2017). except these, the bootstrap confidence interval results of the htmt criterion support the construct's acceptable discriminant validity (hair jr et al., 2017). these results are unexpected and need further investigation. 4.2 structural model the structural model was developed to describe the relationships among the latent constructs and examine their significance (roldán & sánchez-franco, 2012). a bootstrapping technique using 5,000 iterations tested the statistical significance of the relationships between endogenous and exogenous latent constructs in the structural path models (hair jr et al., 2017). the structural model and hypothesised relationships between constructs were tested using a standardised path coefficient (β) and t-statistics (see tables 2 and 4) at p<0.05, p<0.01 and australasian journal of information systems zobair et al. 2021, vol 25, research article health seekers’ acceptance & adoption determinants 13 p<0.001 (benitez et al., 2020). however, a consistent pls (plsc) version was developed to correct the bias and consistently estimate sems with common factors (schuberth, henseler, & dijkstra, 2018). these enhancements are based on the pls algorithm and, thus, on ordinary least squares (ols) regression analysis, implicitly assuming that all indicators are continuous (schuberth et al., 2018). the reason for choosing the pls algorithm for this research is that previous health adoption studies, for example, (chang et al., 2015; hoque et al., 2017; hoque & sorwar, 2017; hsia, chiang, wu, teng, & rubin, 2019; lankton & wilson, 2007; zobair et al., 2019) have widely used the pls algorithm and associated bootstrapping procedure to examine the statistical significance and test hypotheses. the structural model revealed that 5 hypotheses were supported by the relationships p<0.05, p<0.01 and p<0.001 (see table 4 and figure 3). the five proposed latent constructs had significant effects (see tables 2 and 4) on behavioural intention (bi) to use telemedicine healthcare services. the findings show that the relationships between service quality (sq) and bi (β=0.258, t=4.015, p<0.001), privacy and data security (pds) and bi (β=0.249, t=4.232, p<0.001), social influence (si) and bi (β=0.193, t=3.650, p<0.001), perceived usefulness (pu) and bi (β=0.148, t=2.478, p<0.01) and facilitating conditions (fc) and bi (β=0.102, t=1.691, p<0.05) were statistically significant, confirming support for h6, h7, h5, h2 and h4 (one-tailed test, see table 4) respectively. an r2 ≈ 0.65 indicates that about 65% of the variance (i.e., bi) in the model was jointly explained by sq, pds, si, pu and fc. the results were close to a substantial r2 value (i.e., 67%) (henseler et al., 2009), suggesting high predictive capability of the model. further, r2 demonstrates each construct’s significance and its associative contribution to overall r2 (wilson, 2010). therefore, the findings (see figure 3) confirm that service quality, privacy and data security, social influence, usefulness and facilitating conditions substantially impact patients’ behavioural intention towards telemedicine health services adoption in rural bangladesh’s public hospital settings. table 4. structural model assessment note: †p<0.10, *p < 0.05, **p < 0.01, ***p < 0.001, one-tailed test. the structural model further revealed that two other hypotheses were not supported. the relationships between se and bi (β= -0.065, t= 1.521, p<0.05), peu and bi (β=0.060, t=0.918, p<0.05), were statistically not significant. this invalidates h1 and h3 (see table 4), indicating that se, bi, and peu and bi do not contribute to patients’ acceptance and behavioural intention towards adopting telemedicine health services in rural bangladesh’s public hospital settings. hypotheses path coefficient (β) se t p decision h1 se → bi -0.065 0.043 1.521 0.064 unsupported h2 pu→ bi 0.148** 0.060 2.478 0.007 supported h3 peu → bi 0.060 0.066 0.918 0.173 unsupported h4 fc → bi 0.102* 0.060 1.691 0.045 supported h5 si → bi 0.193*** 0.053 3.650 0.000 supported h6 sq → bi 0.258*** 0.064 4.015 0.000 supported h7 pds → bi 0.249*** 0.059 4.232 0.000 supported australasian journal of information systems zobair et al. 2021, vol 25, research article health seekers’ acceptance & adoption determinants 14 figure 2. final pls-sem structural model for acceptance and adoption of telemedicine healthcare services figure 3. final model with confirmed hypotheses. note: †p < 0.10; *p < 0.05; ** p < 0.01; ***p < 0.001 one tailed test. due to recent pls-sem developments, the overall model fit can be estimated using standardised root mean squared residual (srmr), squared euclidean distance, and the geodesic distance (hair jr et al., 2020). the measure of fit (srmr) and the test of overall model australasian journal of information systems zobair et al. 2021, vol 25, research article health seekers’ acceptance & adoption determinants 15 fit squared euclidian distance (duls) and the geodesic distance (dg) is preferable in casual research (benitez et al., 2020; hair jr et al., 2020). the discrepancy between the two matrices is measured by squared duls, dg, and the srmr (benitez et al., 2020; hair jr et al., 2020) assuming, the significance at 5% (benitez et al., 2020). the recommended threshold value of srmr should be below 0.080, and all discrepancy measures should be below the (duls and dg) 95% quantile of their reference distribution (hi95) (benitez et al., 2020). in our model (see table 5), all values of discrepancy are below the 95% quantile of their corresponding reference distribution (hi95) except (duls), authenticating acceptable overall model fit. further, the srmr is below the threshold value of 0.080, indicating a good model fit (benitez et al., 2020). however, the squared euclidean distance in this model is significantly high, and more investigations are encouraged on this new enhancement/development. discrepancy value hi95 decision srmr 0.058 0.045 supported duls 2.230 1.332 unsupported dg 0.836 0.587 supported table 5. overall saturated model fit evaluation 5 discussion this is the first known inclusive study in bangladesh (utilising the extended version of tam) that explicitly investigated determinants of health seekers’ behavioural intentions to accept and adopt telemedicine in the rural public healthcare setting. even though a significant body of studies have explored patients’ behavioural intentions to telemedicine adoption in high and middle-income countries (aggelidis & chatzoglou, 2009; hoque & sorwar, 2017; leung & chen, 2019; maillet, mathieu, & sicotte, 2015), the present study contributes to the health informatics literature by demonstrating determinants of telemedicine adoption predominately in rural healthcare facilities of a low-income country. consistent with previous studies (hoque & sorwar, 2017; leung & chen, 2019; maillet et al., 2015), findings from this study reveal that perceived usefulness, facilitating conditions, social influence, service quality, and privacy and data security are five leading determinants prompting patients’ acceptance behaviours towards telemedicine adoption. this study showed that sq had the most significant effect on patients’ behavioural intentions (h6) to accept telemedicine healthcare services in rural bangladesh settings. evidence suggests that service quality is a strong determinant of patients’ telemedicine acceptance in developing countries settings (ivatury, moore, & bloch, 2009). within the m-health context, akter et al. (2010) found strong relationships between service quality and patients’ satisfaction. davis et al. (2019), for example, found 89% of patients were satisfied with the service quality via teleneurology services. this indicates that telemedicine services quality has a strong effect on patients’ behavioural intention (cronin jr, brady, & hult, 2000) and subsequent acceptance and adoption of telemedicine healthcare services. within the healthcare context, service quality can be measured by evaluating patients’ health expectations against actual service performance (parasuraman et al., 1985). health seekers’ satisfaction is critically important to determine whether patients continue to experience these services or move to different providers (hill & doddato, 2002). satisfied patients will most likely return to a service (i.e., behavioural intention to use) and share their encouraging experiences through word-of-mouth with others (cronin jr et al., 2000; hill & australasian journal of information systems zobair et al. 2021, vol 25, research article health seekers’ acceptance & adoption determinants 16 doddato, 2002). for example, service quality significantly affected patients’ behavioural intentions to use m-health services (akter et al., 2010) and physicians’ acceptance of his (chen & hsiao, 2012). within m-health services, a recent study by k.-h. kim et al. (2019) found that users’ service satisfaction has the most significant effect on the continuity of intention to use m-health services. today’s health seekers expect high service quality from their health providers through continuous, round-the-clock health access, immediate appointments, lengthy consultations, and reduced wait times (yarbrough & smith, 2007). applied to an is success context, delone and mclean (2003) suggest that service quality is considered to be the most crucial variable. our findings are consistent with prior studies. this study argues that service quality to serve as determinants of patients’ behavioural intentions to accept and adopt telemedicine is reflected in the current rural telemedicine service phenomenon. this is because, the rural patients are characterised geographically isolated, and telemedicine care by a specialist through videoconferencing is comparable to face to face specialised care benefiting their health and wellbeing without travelling for hospitalised treatment (ferrer-roca, garcia-nogales, & pelaez, 2010). our findings provide strong evidence of rural patients’ being influenced by telemedicine service quality, indicating improved service quality in telemedicine is more likely to lower health seekers’ anxiety levels and improve their behavioural intentions to use telemedicine services. therefore, service quality presents a significant determinant in predicting patients’ behavioural intentions to accept and adopt telemedicine healthcare services in rural bangladesh. similar to the findings of prior studies (kim & park, 2012; or et al., 2011; venkatesh & bala, 2008; venkatesh et al., 2003; wu, shen, lin, greenes, & bates, 2008), results from this study demonstrated that pu had a significant effect on behavioural intention to accept and adopt telemedicine healthcare services, as it supports h2. similar to the study by venkatesh and davis (2000), perceived usefulness is a strong determinant of patients’ behavioural intentions, explaining telemedicine adoption intentions in rural bangladesh’s public hospital settings. a recent study by leung and chen (2019) demonstrated that pu has significantly contributed to patients’ continuous intentions to use e-health services (i.e., telemedicine). this finding shows that pu is a strong predictor of patients’ acceptance towards telemedicine adoption intentions indicating that patients intend to use telemedicine when they perceive it is useful for addressing their disease concerns, wants and needs. this is consistent with a prior study by davis (1989) which revealed that people intend to use or not to use a service (i.e., application) depending on whether it will exceed their expectations. for example, aggelidis and chatzoglou (2009) studied extended tam and found that pu has a substantial effect on the behavioural intentions of health staff to accept and adopt his. another recent study by koceska et al. (2019) found that pu has a substantial effect on behavioural intention to use mobile health monitoring systems. within the m-health context, zhang et al. (2017) found that pu is a significant determinant of the adoption intention of mhealth services. in the context of e-health, hoque et al. (2017) confirmed that pu has substantial impacts on predicting users’ intention to use e-health services. our findings are consistent with existing literature, authenticating that the pu construct is a significant determinant of predicting patients’ behavioural intentions to accept and adopt telemedicine services. this study reveals the pds construct had substantial effects on patients’ behavioural intentions to accept and adopt telemedicine healthcare services, supporting h7 and indicating that rural australasian journal of information systems zobair et al. 2021, vol 25, research article health seekers’ acceptance & adoption determinants 17 patients desire that health providers keep protected their health status and data. this result comports with other studies showing that privacy, and data security (pds) issues have a dominating effect on users’ behavioural intentions of accepting and adopting telemedicine healthcare services (akter et al., 2010; zhou et al., 2019). a study revealed that health-seeking behaviour is strongly influenced by privacy-related concerns in healthcare services (cheng, savageau, sattler, & dewitt, 1993). patients expect privacy and confidentiality. this is supported by kamal, shafiq, and kakria (2020) acknowledged that privacy and data security significantly contributed to adopting telemedicine services in developing countries. our results validate that the pds construct is a strong determinant in predicting behavioural intention to accept and adopt telemedicine, highlighting that health providers must keep patients’ health information confidential and secure to ensure continued acceptance of telemedicine. our results are consistent with prior empirical studies and are informative and relevant. the significance of our findings associated with pds is further highlighted. for example, some empirical evidence indicates that health-seeking behaviour is strongly influenced by privacyrelated issues in healthcare services (cheng et al., 1993), while wilkowska and ziefle (2011) acknowledged that privacy and confidentiality have substantial effects on users’ health technology acceptance. chang (2015) pointed out that a lack of security, privacy and confidentiality is potentially significant in the acceptance and adoption of health technology, as telemedicine involves the electronic transmission and storage of patient health-related data. the risk of privacy and data security breaches can arise in healthcare systems due to insufficient control over data handling (hale & kvedar, 2014). this is supported by kamal et al. (2020) acknowledged that privacy and data security significantly contributed to adopting telemedicine services in developing countries. our results validate that the pds construct is a strong determinant in predicting behavioural intention to accept and adopt telemedicine, highlighting that health providers must keep patients’ health information confidential and secure to ensure continued acceptance of telemedicine. our results are consistent with prior empirical studies and are informative and relevant. findings from this study suggest that si plays a significant role in rural patients’ behavioural intentions to use telemedicine healthcare services in bangladesh, supporting h5. patients’ telemedicine acceptance decisions could be strongly influenced by their family members, friends, physicians, social media, and legislative awareness. this finding is consistent with previous literature. for example, studies confirmed that social influence is a significant determinant influencing users’ behavioural intentions to accept and adopt health technologies (karahanna & straub, 1999), such as m-health services (hoque et al., 2017) and wireless-based mobile services (lu, yao, & yu, 2005). this indicates that higher perceived social pressure is more likely to heighten behavioural intention to accept and adopt technologies. more precisely, with health context, cimperman et al. (2016) indicate that positive social influence and support strongly affect patients’ behavioural intentions to use healthcare systems. alam et al. (2020) further acknowledged that social influence significantly impacts the behavioural intention to accept and adopt m-health services in bangladesh. our findings have suggested that social influence is an essential determinant in predicting health seekers acceptance and adoption of telemedicine in rural hospital settings. consistent with prior studies, this study reveals that the facilitating conditions construct has the dominating effects on patients’ behavioural intentions of telemedicine services adoption, australasian journal of information systems zobair et al. 2021, vol 25, research article health seekers’ acceptance & adoption determinants 18 supporting h4. this result comports with other studies showing that the facilitating conditions significantly influence the users’ behavioural intentions (jewer, 2018; venkatesh et al., 2003). likewise, alam et al. (2020) found that facilitating conditions contributed to patients’ behavioural intentions of adopting m-health services in bangladesh. a recent study by dutot et al. (2019) asserted that facilitating conditions have positive effects on users’ behavioural intentions to use systems/services. within the e-health context, hossain et al. (2019), for example, revealed that facilitating conditions have a significant impact on behavioural intention to use e-health (i.e., telemedicine) services. these findings are consistent with venkatesh et al. (2012), who confirmed a solid link between social influence and users’ behavioural intentions to use technology. our results suggest that the facilitating conditions construct remains a strong determinant, potentially predicting health seekers’ behavioural intentions to accept and adopt telemedicine healthcare services in rural bangladesh. additionally, this study confirmed that the se and peu had no direct effects on the bi in the proposed model (h1 and h3 are not supported). these results underscore the importance of the impact on patients’ behavioural intentions. however, these findings are consistent with prior studies as peu construct (or et al., 2011) and se constructs (maillet et al., 2015) unable to make any significant contributions to predict users’ behavioural intentions of adopting telemedicine healthcare services. interestingly, kim and park (2012) acknowledged that selfefficacy and ease of use had significant effects on the behavioural intention of hit. this study argues that the inability of self-efficacy and ease of use to serve as determinants of patients’ behavioural intentions to accept and adopt telemedicine is reflected in the current rural telemedicine service phenomenon. the study findings may be unexpected, but it may indicate that in bangladesh, telemedicine healthcare services and technical, operational, and administrative support is actively managed by its ict staff and physicians. patients remain passive and lack control over those issues. these findings are consistent with current telemedicine operations and critical management criteria that have been empirically proven, indicating that patients may have minimal interaction with the technology during consultation. 5.1 contributions and implications the present study contributes to theory-based testing models (i.e., the tam and sct) to examine the determinants of acceptance and antecedents of expectation associated with telemedicine adoption. prior studies have drawn on many behavioural theories, including the tam (agarwal, sambamurthy, & stair, 2000), to better understand users’ acceptance or rejection of technologies (davis, 1989; davis et al., 1989). in the context of health, social learning theory, se, and locus of control have been applied for understanding, explaining, predicting, and influencing behaviour (rosenstock et al., 1988). the present study built on these previous studies, for instance (bandura, 1977, 1986; davis et al., 1989; shankar, smith, & rangaswamy, 2003; venkatesh, 2000a; venkatesh & davis, 1996), integrated their theories and uniquely applied the tam and sct to identify and examine rural patients’ acceptance and expectations of telemedicine adoption in emerging economies’ public hospital settings. this study's findings have important implications for information systems, particularly in health informatics research from a managerial perspective. the extended tam model appears sound and could become a benchmark model to study patients' acceptance and future service continuity in the telemedicine domain. this suggests that health providers should enhance patients' happiness to keep secure patients' long-term service acceptance and continuity with australasian journal of information systems zobair et al. 2021, vol 25, research article health seekers’ acceptance & adoption determinants 19 telemedicine. secondly, this study employed an extended tam framework to significantly contribute to predict, understand and explain health seekers' behavioural intention to accept and adopt telemedicine health services in rural settings in a developing country's context. thirdly, determinants that influence health seekers' behavioural choices to use telemedicine in the context of rural areas in a developing country are explored and noted. generally, rural and urban patients' intentions to adopt telemedicine services are disparate due to differences in their social, cultural, and facilitating circumstances. fourthly, the results provide government, policymakers, implementers, and stakeholders with a clear picture of telemedicine projects' causative factors, adoption successes and failures. finally, it identifies five exemplary determinants to patients' telemedicine acceptance and adoption —sq, pu, fc, si and pds—that merit further policy intervention for the successful adoption of telemedicine services in rural settings, including healthcare industry improvement in bangladesh and similar settings. from telemedicine implementation and actual uses standpoints, our findings point to the importance of providing a high quality of care, protecting patients’ privacy, prompt responsiveness and health outcomes, ensure well-facilitating service conditions. the health providers additional support build patients' trust that plays a significant role in a positive attitude towards new technology-mediated telemedicine acceptance and future continuity. 5.2 research impact the present study's underlying impacts are classified into seven-dimensional categories: knowledge, stakeholders, theory, situational, model, country, and data collection. this study's knowledge contributed to defining, distinguishing, explaining, interpreting, and predicting telemedicine adoption determinants in contexts like emerging economies' rural settings. it provides unique insights into rural patients' (stakeholders') views regarding telemedicine services. further, this research identified four areas of impact, namely (1) research related (i.e., research problem, methods used, research management and communication), (2) policy (i.e., level of policymaking, type, nature and policy networks), (3) service (i.e., health services, service management, quality of care, and information systems) and (4) social-based (i.e., attitudes behaviour, health literacy, health status, and sustainable health outcomes) on research impact framework (kuruvilla, mays, pleasant, & walt, 2006). failure to recognise these determinants and impacts stakeholders' acceptance and expectations of telemedicine will undermine attempts to implement in emerging economies. 5.3 limitations and future research tam model has successfully predicted user technology acceptance and usage continuance (ramkumar, schoenherr, wagner, & jenamani, 2019). however, using tam without extending it might not explain how users accept a new technology (alshammari & rosli, 2020). the designed goal of the present research involves extending the tam model. not surprisingly, the contemporary is research literature has become choked and confused by various tam extensions and tam verifications in varied contexts. consequently, authors of such research reports face a challenging hurdle in making a case for the need (motive), the novelty, and the significance of findings that extend the tam model or its operation in different contexts, demonstrating one of the significant theoretical limitations. evidence suggests that contemporary healthcare services are becoming heavily reliant on technology (jewer, 2018); thus, thoughtful research on users' acceptance concerning australasian journal of information systems zobair et al. 2021, vol 25, research article health seekers’ acceptance & adoption determinants 20 technology-mediated healthcare services (i.e., telemedicine) is now crucial for sustaining this provision. however, tam's significant weaknesses/limitations to explain users' behaviours indicate that this model is incapable of sufficiently predicting ict acceptance (ajibade, 2018). for example, the tam model is inconsistent in providing precursors to mobile use and social influence that facilitate behaviours (ajibade, 2018). against this backdrop, a unique customised theorisation (i.e., expectation disconfirmation theory (edt) and tam) can be put forward for future research to conceptualise patients' acceptance of the context of telemedicine. furthermore, in line with edt and tam research, lankton and mcknight (2012) revealed that if an easy-to-use system/service may satisfy the users, in turn, individuals may put up with a technology/system that is not easy to use. this indicates that an individual's acceptance decision will increase more for positive ease of use than it will decrease for negative ease of use regarding a system/service (lankton & mcknight, 2012). lankton and mcknight (2012) further predicted that a positive asymmetry effect on ease of use would exist for performance because ease of use is more likely to have assimilation effects on satisfaction through performance than contrast effects through disconfirmation. this indicates that an individual with low experience is more likely to explore small discrepancies between ease-of-use expectations and performance disconfirmation (lankton & mcknight, 2012). some other limitations do exist in this research. first, this study reflected only the behavioural adoption intentions of health seekers. future research should investigate factors concerning the behavioural intentions of public health providers that influence telemedicine adoption in rural settings. second, only public telemedicine health systems were included in this study. private telemedicine health systems could be included in future research to examine the differences between public and private service interests. third, this study focused on few factors to determine health seekers’ behavioural intentions to use telemedicine. future research could be conducted by including age, gender, education, disease types, health literacy and organisational effects that may significantly influence health seekers’ adoption intentions. finally, this study was limited to bangladesh. combining this study with cross-sectional data from similar developing countries would provide a broader view of behavioural intention to use telemedicine in a global context. 5.4 conclusion this research demonstrates significant support for the proposed theorised model, accounting for over 65% of the variance in health seekers’ behavioural intentions to use telemedicine healthcare services. this study has developed an extended tam framework that has yielded guidelines for accelerating telemedicine adoption in rural bangladesh and contextually similar settings. drawing from research on tam in e-health, hoque et al. (2017) found that privacy and trust have not contributed adequately to qualify as a determinant of patients’ behavioural intentions, while in our study, we found that pds strongly influences patients’ intentions to use telemedicine services. this authenticates that health providers must keep patients’ health data confidential and secure to ensure continued acceptance of telemedicine healthcare services in rural settings in bangladesh. further, this study would help to determine the model that is considered the most appropriate and beneficial standard as a benchmark for all future research on patients’ behavioural intentions of telemedicine healthcare service adoption in both developed and developing countries’ rural contexts. consistent with our findings, this study is significant and deepens the understanding of the determinants of health seekers australasian journal of information systems zobair et al. 2021, vol 25, research article health seekers’ acceptance & adoption determinants 21 behavioural intentions of other important ict based healthcare services adoption fields such as e-health and m-health. novel interpretations of large data sets in health informatics have been generated and extended a body of new knowledge in rural emerging economies settings. pertinent implications, limitations and forthcoming research directions were further recommended to secure the long-term sustainability of telemedicine healthcare projects in emerging economies and similar settings. acknowledgements we are very grateful to all the study participants and officials of the directorate general of health services under the ministry of health and family welfare of bangladesh, who helped a lot during data collection. references agarwal, r., sambamurthy, v., & stair, r. m. (2000). the evolving relationship between general and specific computer self-efficacy—an empirical assessment. information systems research, 11(4), 418-430. aggelidis, v. p., & chatzoglou, p. d. (2009). using a modified technology acceptance model in hospitals. international journal of medical mnformatics, 78(2), 115-126. ahmed, t., bloom, g., iqbal, m., lucas, h., rasheed, s., waldman, l., . . . bhuiya, a. (2014). ehealth and m-health in bangladesh: opportunities and challenges. retrieved from https://www.ids.ac.uk/publications/e-health-and-m-health-in-bangladeshopportunities-and-challenges/, ajibade, p. (2018). technology acceptance model limitations and criticisms: exploring the practical applications and use in technology-related studies, mixed-method, and qualitative researches. library philosophy & practice, 1941. akter, s., d’ambra, j., & ray, p. (2010). service quality of mhealth platforms: development and validation of a hierarchical model using pls. electronic markets, 20(3-4), 209-227. alam, m. z., hoque, m. r., hu, w., & barua, z. (2020). factors influencing the adoption of mhealth services in a developing country: a patient-centric study. international journal of information management, 50, 128-143. ali, i., ali, m., badghish, s., & baazeem, t. a. s. (2018). examining the role of childhood experiences in developing altruistic and knowledge sharing behaviors among children in their later life: a partial least squares (pls) path modeling approach. sustainability, 10(2), 292. alshammari, s. h., & rosli, m. s. (2020). a review of technology acceptance models and theories. innovative teaching and learning journal (itlj), 4(2), 12-22. anderberg, p., eivazzadeh, s., & berglund, j. s. (2019). a novel instrument for measuring older people’s attitudes toward technology (techph): development and validation. journal of medical internet research, 21(5), e13951. doi: 10.2196/13951 astrachan, c. b., patel, v. k., & wanzenried, g. (2014). a comparative study of cb-sem and pls-sem for theory development in family firm research. journal of family business strategy, 5(1), 116-128. bandura, a. (1977). self-efficacy: toward a unifying theory of behavioral change. psychological review, 84(2), 191. https://doi.org/10.1037/0033-295x.84.2.191 bandura, a. (1986). social foundations of thought and action. englewood cliffs, new jersey, prentice hall, 1986. https://doi.org/10.2196/13951 australasian journal of information systems zobair et al. 2021, vol 25, research article health seekers’ acceptance & adoption determinants 22 bandura, a., o'leary, a., taylor, c. b., gauthier, j., & gossard, d. (1987). perceived selfefficacy and pain control: opioid and nonopioid mechanisms. journal of personality and social psychology, 53(3), 563-571. doi:10.1037/0022-3514.53.3.563. benitez, j., henseler, j., castillo, a., & schuberth, f. (2020). how to perform and report an impactful analysis using partial least squares: guidelines for confirmatory and explanatory is research. information & management, 57(2), 103168. https://doi.org/10.1016/j.im.2019.05.003 bettiga, d., lamberti, l., & lettieri, e. (2019). individuals’ adoption of smart technologies for preventive health care: a structural equation modeling approach. health care management science, 23(2), 203-214. doi:10.1007/s10729-019-09468-2 bhattacherjee, a., & hikmet, n. (2007). physicians' resistance toward healthcare information technology: a theoretical model and empirical test. european journal of information systems, 16(6), 725-737. boscarino, j. a. (1992). the public's perception of quality hospitals ii: implications for patient surveys. journal of healthcare management, 37(1), 13. bros, j. s., poulet, c., arnol, n., deschaux, c., gandit, m., & charavel, m. (2018). acceptance of telemonitoring among patients with obstructive sleep apnea syndrome: how is the perceived interest by and for patients? telemedicine and e-health, 24(5), 351-359. chang, h. (2015). evaluation framework for telemedicine using the logical framework approach and a fishbone diagram. healthcare informatics research, 21(4), 230-238. chang, y.-z., ko, c.-y., hsiao, c.-j., chen, r.-j., yu, c.-w., cheng, y.-w., . . . chao, c.-m. (2015). understanding the determinants of implementing telehealth systems: a combined model of the theory of planned behavior and the technology acceptance model. journal of applied sciences, 15(2), 277. chau, p. y., & hu, p. j.-h. (2002). investigating healthcare professionals’ decisions to accept telemedicine technology: an empirical test of competing theories. information & management, 39(4), 297-311. chellappa, r. k., & sin, r. g. (2005). personalization versus privacy: an empirical examination of the online consumer’s dilemma. information technology and management, 6(2-3), 181202. chen, r.-f., & hsiao, j.-l. (2012). an empirical study of physicians' acceptance of hospital information systems in taiwan. telemedicine and e-health, 18(2), 120-125. cheng, t. l., savageau, j. a., sattler, a. l., & dewitt, t. g. (1993). confidentiality in health care: a survey of knowledge, perceptions, and attitudes among high school students. jama, 269(11), 1404-1407. chin, w. w. (1998). the partial least squares approach to structural equation modeling. modern methods for business research, 295(2), 295-336. cimperman, m., brenčič, m. m., & trkman, p. (2016). analyzing older users’ home telehealth services acceptance behavior—applying an extended utaut model. international journal of medical informatics, 90, 22-31. cresswell, j. w., & clark, v. l. p. (2011). designing and conducting mixed methods resaerch. (2nd edition). thousand oaks, california, sage, 2011. cronin jr, j. j., brady, m. k., & hult, g. t. m. (2000). assessing the effects of quality, value, and customer satisfaction on consumer behavioral intentions in service environments. journal of retailing, 76(2), 193-218. https://doi.org/10.1016/j.im.2019.05.003 australasian journal of information systems zobair et al. 2021, vol 25, research article health seekers’ acceptance & adoption determinants 23 culnan, m. j., & armstrong, p. k. (1999). information privacy concerns, procedural fairness, and impersonal trust: an empirical investigation. organization science, 10(1), 104-115. doi:10.1287/orsc.10.1.104. darkwa, e. k., newman, m. s., kawkab, m., & chowdhury, m. e. (2015). a qualitative study of factors influencing retention of doctors and nurses at rural healthcare facilities in bangladesh. bmc health services research, 15(1), 344. doi:10.1186/s12913-015-1012-z. davis, l.e., harnar, j., lachey-barbee, l. a., pirio richardson, s., fraser, a., & king, m. k. (2019). using teleneurology to deliver chronic neurologic care to rural veterans: analysis of the first 1,100 patient visits. telemedicine and e-health, 25(4), 274-278. davis, f. d. (1989). perceived usefulness, perceived ease of use, and user acceptance of information technology. mis quarterly, 13(3), 319-340. https://doi.org/10.2307/249008 davis, f. d., bagozzi, r. p., & warshaw, p. r. (1989). user acceptance of computer technology: a comparison of two theoretical models. management science, 35(8), 9821003. doi:10.1287/mnsc.35.8.982. delone, w. h., & mclean, e. r. (2003). the delone and mclean model of information systems success: a ten-year update. journal of management information systems, 19(4), 9-30. dijkstra, t. k., & henseler, j. (2015). consistent and asymptotically normal pls estimators for linear structural equations. computational statistics & data analysis, 81, 10-23. dünnebeil, s., sunyaev, a., blohm, i., leimeister, j. m., & krcmar, h. (2012). determinants of physicians’ technology acceptance for e-health in ambulatory care. international journal of medical informatics, 81(11), 746-760. doi:10.1016/j.ijmedinf.2012.02.002. dutot, v., bergeron, f., rozhkova, k., & moreau, n. (2019). factors affecting the adoption of connected objects in e-health: a mixed methods approach. systèmes d'information et management, 23(4), 31-66. https://aisel.aisnet.org/sim/vol23/iss4/7 dwivedi, y. k., shareef, m. a., simintiras, a. c., lal, b., & weerakkody, v. (2016). a generalised adoption model for services: a cross-country comparison of mobile health (m-health). government information quarterly, 33(1), 174-187. esmaeilzadeh, p. (2019). the effects of public concern for information privacy on the adoption of health information exchanges (hies) by healthcare entities. health communication, 34(10), 1202-1211. evans, p. r. (2015). an exploration of the perceptions of a us midwest acute care hospital organization regarding the adoption and acceptance of telemedicine. (doctoral dissertaion, capella university), proquest dissertations publishing, 2015. 3700252. ferrer-roca, o., garcia-nogales, a., & pelaez, c. (2010). the impact of telemedicine on quality of life in rural areas: the extremadura model of specialized care delivery. telemedicine and e-health, 16(2), 233-243. gefen, d., straub, d., & boudreau, m.-c. (2000). structural equation modeling and regression: guidelines for research practice. communications of the association for information systems, 4(1), 7. https://doi.org/10.17705/1cais.00407 greenberg, a. s., steinway, c., wu, k., thomas, b., chuo, j., digiovine, m., & jan, s. j. j. o. a. h. (2019). 80. role of telemedicine in the transition from pediatric to adult care. 64(2), s43. hair, j., hair, j., hollingsworth, c. l., hollingsworth, c. l., randolph, a. b., randolph, a. b., . . . chong, a. y. l. (2017). an updated and expanded assessment of pls-sem in information systems research. industrial management & data systems, 117(3), 442-458. hair, j. f., ringle, c. m., & sarstedt, m. (2011). pls-sem: indeed a silver bullet. journal of marketing theory and practice, 19(2), 139-152. https://doi.org/10.2307/249008 australasian journal of information systems zobair et al. 2021, vol 25, research article health seekers’ acceptance & adoption determinants 24 hair, j. f., ringle, c. m., & sarstedt, m. (2013). partial least squares structural equation modeling: rigorous applications, better results and higher acceptance. long range planning, 46(1-2), 1-12. hair jr, j. f., howard, m. c., & nitzl, c. (2020). assessing measurement model quality in plssem using confirmatory composite analysis. journal of business research, 109, 101-110. hair jr, j. f., hult, g. t. m., ringle, c., & sarstedt, m. (2016). a primer on partial least squares structural equation modeling (pls-sem): los angeles, california, sage publications. hair jr, j. f., hult, g. t. m., ringle, c., & sarstedt, m. (2017). a primer on partial least squares structural equation modeling (pls-sem): los angeles, california, sage publications. hale, t. m., & kvedar, j. c. (2014). privacy and security concerns in telehealth. virtual mentor, 16(12), 981. hall, j. l., & mcgraw, d. (2014). for telehealth to succeed, privacy and security risks must be identified and addressed. health qffairs, 33(2), 216-221. henseler, j., hubona, g., & ray, p. a. (2017). partial least squares path modeling: updated guidelines. in partial least squares path modeling (pp. 19-39): cham, switzerland springer. 10.1007/978-3-319-64069-3_2 henseler, j., ringle, c. m., & sinkovics, r. r. (2009). the use of partial least squares path modeling in international marketing. in new challenges to international marketing (pp. 277319): bingley, emerald group publishing limited. doi: 10.1108/s1474-7979(2009)0000020014 hill, m. h., & doddato, t. (2002). relationships among patient satisfaction, intent to return, and intent to recommend services provided by an academic nursing center. journal of cultural diversity, 9(4), 108-112. holden, r. j., & karsh, b.-t. (2010). the technology acceptance model: its past and its future in health care. journal of biomedical informatics, 43(1), 159-172. hoque, bao, y., & sorwar, g. (2017). investigating factors influencing the adoption of e-health in developing countries: a patient’s perspective. informatics for health and social care, 42(1), 1-17. hoque, & sorwar, g. (2017). understanding factors influencing the adoption of mhealth by the elderly: an extension of the utaut model. international journal of medical informatics, 101, 75-84. doi:10.1016/j.ijmedinf.2017.02.002. hossain, a., quaresma, r., & rahman, h. (2019). investigating factors influencing the physicians’ adoption of electronic health record (ehr) in healthcare system of bangladesh: an empirical study. international journal of information management, 44, 7687. hsia, t.-l., chiang, a.-j., wu, j.-h., teng, n. n., & rubin, a. d. (2019). what drives e-health usage? integrated institutional forces and top management perspectives. computers in human behavior. hu, p. j., chau, p. y., sheng, o. r. l., & tam, k. y. (1999). examining the technology acceptance model using physician acceptance of telemedicine technology. journal of management information systems, 16(2), 91-112. hur, i., cousins, k. c., & stahl, b. c. (2019). a critical perspective of engagement in online health communities. european journal of information systems, 28(5), 523-548. https://doi.org/10.1080/0960085x.2019.1620477 islam, f., rahman, a., halim, a., eriksson, c., rahman, f., & dalal, k. (2015). perceptions of health care providers and patients on quality of care in maternal and neonatal health in fourteen bangladesh government healthcare facilities: a mixed-method study. bmc health services research, 15(1), 1-9. doi: 10.1186/s12913-015-0918-9 australasian journal of information systems zobair et al. 2021, vol 25, research article health seekers’ acceptance & adoption determinants 25 ivatury, g., moore, j., & bloch, a. (2009). a doctor in your pocket: health hotlines in developing countries. innovations: technology, governance, globalization, 4(1), 119-153. jansen-kosterink, s., dekker-van weering, m., & van velsen, l. (2019). patient acceptance of a telemedicine service for rehabilitation care: a focus group study. international journal of medical informatics, 125, 22-29. jewer, j. (2018). patients’ intention to use online postings of ed wait times: a modified utaut model. international journal of medical informatics 112, 34-39. https://doi.org/10.1016/j.ijmedinf.2018.01.008 johnston, a. c., & warkentin, m. (2010). fear appeals and information security behaviors: an empirical study. mis quarterly, 34(3), 549-566. https://doi.org/10.2307/25750691 kamal, s. a., shafiq, m., & kakria, p. (2020). investigating acceptance of telemedicine services through an extended technology acceptance model (tam). technology in society, 60, 101212. karahanna, e., & straub, d. w. (1999). the psychological origins of perceived usefulness and ease-of-use. information & management, 35(4), 237-250. kettinger, w. j., & lee, c. c. (1997). pragmatic perspectives on the measurement of information systems service quality. mis quarterly, 21(2), 223-240. kim, & park, h.-a. (2012). development of a health information technology acceptance model using consumers’ health behavior intention. journal of medical internet research, 14(5), e133. doi: 10.2196/jmir.2143 kim, k.-h., kim, k.-j., lee, d.-h., & kim, m.-g. (2019). identification of critical quality dimensions for continuance intention in mhealth services: case study of onecare service. international journal of information management, 46, 187-197. koceska, n., komadina, r., simjanoska, m., koteska, b., strahovnik, a., jošt, a., . . . trontelj, j. (2019). mobile wireless monitoring system for prehospital emergency care. european journal of trauma and emergency surgery, 46(6), 1301-1308. kock, n. (2018). should bootstrapping be used in pls-sem? toward stable p-value calculation methods. journal of applied structural equation modeling, 2(1), 1-12. kuruvilla, s., mays, n., pleasant, a., & walt, g. (2006). describing the impact of health research: a research impact framework. bmc health services research, 6(1), 1-18. lankton, n. k., & mcknight, h. d. (2012). examining two expectation disconfirmation theory models: assimilation and asymmetry effects. journal of the association for information systems, 13(2), 1. doi: 10.17705/1jais.00285 lankton, n. k., & wilson, e. v. (2007). factors influencing expectations of e-health services within a direct-effects model of user satisfaction. e-service journal, 5(2), 85-112. lending, d., & dillon, t. w. (2007). the effects of confidentiality on nursing self-efficacy with information systems. international journal of healthcare information systems and informatics (ijhisi), 2(3), 49-64. lerouge, c., & garfield, m. j. (2013). crossing the telemedicine chasm: have the us barriers to widespread adoption of telemedicine been significantly reduced? international journal of environmental research and public health, 10(12), 6472-6484. lerouge, c. m., gupta, m., corpart, g., & arrieta, a. (2019). health system approaches are needed to expand telemedicine use across nine latin american nations. health affairs, 38(2), 212-221. leung, l., & chen, c. (2019). e-health/m-health adoption and lifestyle improvements: exploring the roles of technology readiness, the expectation-confirmation model, and health-related information activities. telecommunications policy, 43(6), 563-575. australasian journal of information systems zobair et al. 2021, vol 25, research article health seekers’ acceptance & adoption determinants 26 lewis, w., agarwal, r., & sambamurthy, v. (2003). sources of influence on beliefs about information technology use: an empirical study of knowledge workers. mis quarterly, 27(4), 657-678. https://doi.org/10.2307/30036552 lin, h.-c., & chang, c.-m. (2018). what motivates health information exchange in social media? the roles of the social cognitive theory and perceived interactivity. information & management, 55(6), 771-780. https://doi.org/10.1016/j.im.2018.03.006 lu, j., yao, j. e., & yu, c.-s. (2005). personal innovativeness, social influences and adoption of wireless internet services via mobile technology. the journal of strategic information systems, 14(3), 245-268. macedo, i. m. (2017). predicting the acceptance and use of information and communication technology by older adults: an empirical examination of the revised utaut2. computers in human behavior, 75, 935-948. maillet, é., mathieu, l., & sicotte, c. (2015). modeling factors explaining the acceptance, actual use and satisfaction of nurses using an electronic patient record in acute care settings: an extension of the utaut. international journal of medical informatics, 84(1), 36-47. muthupoltotage, u. p., & gardner, l. (2018). analysing the relationships between digital literacy and self-regulated learning of undergraduates—a preliminary investigation. in advances in information systems development (pp. 1-16): cham, switzerland, springer. or, c. k., karsh, b.-t., severtson, d. j., burke, l. j., brown, r. l., & brennan, p. f. (2011). factors affecting home care patients' acceptance of a web-based interactive self-management technology. journal of the american medical informatics association, 18(1), 51-59. otter, v., & beer, l. (2021). alley cropping systems as ecological focus areas: a pls-analysis of german farmers’ acceptance behaviour. journal of cleaner production, 280, 123702. https://doi.org/10.1016/j.jclepro.2020.123702 parasuraman, a., zeithaml, v. a., & berry, l. l. (1985). a conceptual model of service quality and its implications for future research. the journal of marketing, 49(4), 41-50. ramkumar, m., schoenherr, t., wagner, s. m., & jenamani, m. (2019). q-tam: a quality technology acceptance model for predicting organizational buyers’ continuance intentions for e-procurement services. international journal of production economics, 216, 333-348. rho, m. j., young choi, i., & lee, j. (2014). predictive factors of telemedicine service acceptance and behavioral intention of physicians. international journal of medical informatics, 83(8), 559-571. ringle, c. m., sarstedt, m., & straub, d. w. (2012). editor's comments: a critical look at the use of pls-sem in mis quarterly. mis quarterly, 36(1), iii-xiv. roldán, j. l., & sánchez-franco, m. j. (2012). variance-based structural equation modeling: guidelines for using partial least squares. research methodologies, innovations and philosophies in software systems engineering and information systems, (pp.193-221). igi global. doi: 10.4018/978-1-4666-0179-6.ch010 rosenstock, i. m., strecher, v. j., & becker, m. h. (1988). social learning theory and the health belief model. health education quarterly, 15(2), 175-183. schuberth, f., henseler, j., & dijkstra, t. k. (2018). partial least squares path modeling using ordinal categorical indicators. quality & quantity, 52(1), 9-35. schubring, s., lorscheid, i., meyer, m., & ringle, c. m. (2016). the pls agent: predictive modeling with pls-sem and agent-based simulation. journal of business research, 69(10), 4604-4612. australasian journal of information systems zobair et al. 2021, vol 25, research article health seekers’ acceptance & adoption determinants 27 shankar, v., smith, a. k., & rangaswamy, a. (2003). customer satisfaction and loyalty in online and offline environments. international journal of research in marketing, 20(2), 153175. taylor, j., coates, e., wessels, b., mountain, g., & hawley, m. s. (2015). implementing solutions to improve and expand telehealth adoption: participatory action research in four community healthcare settings. bmc health services research, 15(1), 529. tsai, c.-h. (2014). integrating social capital theory, social cognitive theory, and the technology acceptance model to explore a behavioral model of telehealth systems. international journal of environmental research and public health, 11(5), 4905-4925. venkatesh, v. (2000a). determinants of perceived ease of use: integrating control, intrinsic motivation, and emotion into the technology acceptance model. information systems research, 11(4), 342-365. venkatesh, v. (2000b). determinants of perceived ease of use: integrating perceived behavioral control, computer anxiety and enjoyment into the technology acceptance model. information systems research, 11(4), 342-365. venkatesh, v., & bala, h. (2008). technology acceptance model 3 and a research agenda on interventions. decision sciences, 39(2), 273-315. venkatesh, v., & davis, f. d. (1996). a model of the antecedents of perceived ease of use: development and test. decision sciences, 27(3), 451-481. venkatesh, v., & davis, f. d. (2000). a theoretical extension of the technology acceptance model: four longitudinal field studies. management science, 46(2), 186-204. venkatesh, v., morris, m. g., davis, g. b., & davis, f. d. (2003). user acceptance of information technology: toward a unified view. mis quarterly, 27(3), 425-478. https://doi.org/10.2307/30036540 venkatesh, v., thong, j. y., & xu, x. (2012). consumer acceptance and use of information technology: extending the unified theory of acceptance and use of technology. mis quarterly, 36(1), 157-178. https://doi.org/10.2307/41410412 vries, h. d., backbier, e., kok, g., & dijkstra, m. (1995). the impact of social influences in the context of attitude, self-efficacy, intention, and previous behavior as predictors of smoking onset. journal of applied social psychology, 25(3), 237-257. whitten, p., holtz, b., & nguyen, l. (2010). keys to a successful and sustainable telemedicine program. international journal of technology assessment in health care, 26(2), 211-216. doi: https://doi.org/10.1017/s026646231000005x wilkowska, w., & ziefle, m. (2011). perception of privacy and security for acceptance of e-health technologies: exploratory analysis for diverse user groups. in 2011 5th international conference on pervasive computing technologies for healthcare (pervasivehealth) and workshop (pp. 593-600). ieee. doi: 10.4108/icst.pervasivehealth.2011.246027 williams, k., & bond, m. (2002). the roles of self-efficacy, outcome expectancies and social support in the self-care behaviours of diabetics. psychology, health & medicine, 7(2), 127141. wilson, b. (2010). using pls to investigate interaction effects between higher order branding constructs. in handbook of partial least squares (pp. 621-652): berlin, heidelberg, germany, springer. doi: 10.1007/978-3-540-32827-8_28 woo, k., & dowding, d. (2018). factors affecting the acceptance of telehealth services by heart failure patients: an integrative review. telemedicine and e-health, 24(4), 292-300. australasian journal of information systems zobair et al. 2021, vol 25, research article health seekers’ acceptance & adoption determinants 28 wu, j.-h., shen, w.-s., lin, l.-m., greenes, r. a., & bates, d. w. (2008). testing the technology acceptance model for evaluating healthcare professionals' intention to use an adverse event reporting system. international journal for quality in health care, 20(2), 123-129. xu, z. (2019). an empirical study of patients' privacy concerns for health informatics as a service. technological forecasting and social change, 143, 297-306. yarbrough, a. k., & smith, t. b. (2007). technology acceptance among physicians: a new take on tam. medical care research and review, 64(6), 650-672. zeithaml, v. a., berry, l. l., & parasuraman, a. (1996). the behavioral consequences of service quality. journal of marketing, 60(2), 31-46. zhang, x., han, x., dang, y., meng, f., guo, x., & lin, j. (2017). user acceptance of mobile health services from users’ perspectives: the role of self-efficacy and response-efficacy in technology acceptance. informatics for health and social care, 42(2), 194-206. zhou, m., zhao, l., kong, n., campy, k. s., qu, s., & wang, s. (2019). factors influencing behavior intentions to telehealth by chinese elderly: an extended tam model. international journal of medical informatics, 126, 118-127. zobair, k. m., sanzogni, l., & sandhu, k. (2019). expectations of telemedicine health service adoption in rural bangladesh. social science & medicine, 238, 112485. https://doi.org/10.1016/j.socscimed.2019.112485 zobair, k. m., sanzogni, l., sandhu, k., & islam, m. j. (2020). telemedicine adoption opportunities and challenges in the developing world. in evaluating challenges and opportunities for healthcare reform (pp. 167-193): hershey, pe, usa, igi global. doi: 10.4018/978-1-7998-8052-3.ch003 appendix measurement items of constructs constructs indicators statement sources perceived usefulness (pu) pu1 using telemedicine will improve my quality of life. (bhattacherjee & hikmet, 2007; chau & hu, 2002; hoque et al., 2017; venkatesh et al., 2003) pu2 using telemedicine will make my life more convenient. pu3 using telemedicine will make me more productive in managing my health. pu4 using telemedicine will make my life more useful. perceived ease of use (peu) peu1 i can get telemedicine health services easily. (chau & hu, 2002; venkatesh et al., 2003; zhou et al., 2019) peu2 i can easily receive a remotely specialist consultation via telemedicine. peu3 i can discuss my health-related issues with specialist physicians easily via video conferencing. australasian journal of information systems zobair et al. 2021, vol 25, research article health seekers’ acceptance & adoption determinants 29 peu4 using telemedicine to discuss with my specialist physician, any of my health-related issues is easy for me. self-efficacy (se) se1 it is easy for me to get telemedicine services that are provided by my hospitals. (johnston & warkentin, 2010; maillet et al., 2015; tsai, 2014; venkatesh et al., 2003; zhang et al., 2017) se2 i can use telemedicine for receiving specialist consultation remotely. se3 i am confident to discuss my health-related issue with specialists via telemedicine. se4 i believe that i can use telemedicine for better health management. service quality (sq) sq1 i have been satisfied with telemedicine services that performed well for the first time. (akter et al., 2010; kettinger & lee, 1997) sq2 getting telemedicine has enabled me to improve my health. sq3 in most ways, my healthy life has come closer to my ideal since i have started to use telemedicine. sq4 the physicians in telemedicine have understood my specific health needs. sq5 the physicians in telemedicine have given me individual attention during the consultation. sq6 overall, telemedicine has helped me to achieve the health goals i most expect in life. social influence (si) si1 people who are important to me think that i should use telemedicine. (dwivedi, shareef, simintiras, lal, & weerakkody, 2016; hoque & sorwar, 2017; venkatesh et al., 2003) si2 people who influence my behaviour think that i should use telemedicine. si3 those family members who are important to me think i should use telemedicine. si4 my friends whose opinions i value think i should use telemedicine. si5 the physicians whose opinions i value prefer that i should use telemedicine. privacy and data security (pds) pds1 i believe the privacy of telemedicine patients is protected. (chellappa & sin, 2005; culnan & armstrong, 1999; dünnebeil, sunyaev, blohm, leimeister, & krcmar, 2012; hoque et al., 2017) pds2 i believe that the personal information stored in the telemedicine system is safe. pds3 i believe the telemedicine system keeps patients’ information secure. facilitating condition (fc) fc1 i find that telemedicine has the resources necessary to provide me with healthcare services. (alam et al., 2020; dwivedi et al., 2016; australasian journal of information systems zobair et al. 2021, vol 25, research article health seekers’ acceptance & adoption determinants 30 fc2 i find that telemedicine physicians have been very knowledgeable. hoque & sorwar, 2017; venkatesh et al., 2003) fc3 i find that telemedicine facilitates the required technologies that are needed. fc4 i can get telemedicine services when i need to. fc5 i find that telemedicine enables easy referrals to specialised physicians. fc6 i have found telemedicine consultations successful. behavioural intention (bi) bi1 i have a high intention to use telemedicine. (chau & hu, 2002; hoque & sorwar, 2017; venkatesh et al., 2003; zhou et al., 2019) bi2 i intend to use telemedicine for a better quality of life. bi3 i intend to acquire more knowledge about the benefits of telemedicine for better health management. bi4 i plan to use telemedicine to manage my healthy life. copyright: © 2021 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v25i0.3071 http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 theoretical model and hypotheses 2.1 self-efficacy 2.2 perceived usefulness 2.3 perceived ease of use 2.4 facilitating conditions 2.5 social influence 2.6 service quality 2.7 privacy and data security 3 research methodology 3.1 study population 3.2 data collection 3.3 construct measurements 4 data analysis and results 4.1 measurement model 4.2 structural model 5 discussion 5.1 contributions and implications 5.2 research impact 5.3 limitations and future research 5.4 conclusion paper title (paper title style) australasian journal of information systems nadeem, marjanovic & abedin 2022, vol 26, selected papers from acis 2020 gender bias in ai 1 gender bias in ai-based decision-making systems: a systematic literature review ayesha nadeem university of technology sydney, australia ayesha.nadeem@student.uts.edu.au olivera marjanovic university of technology sydney, australia babak abedin macquaire university, sydney, australia abstract the related literature and industry press suggest that artificial intelligence (ai)-based decision-making systems may be biased towards gender, which in turn impacts individuals and societies. the information system (is) field has recognised the rich contribution of aibased outcomes and their effects; however, there is a lack of is research on the management of gender bias in ai-based decision-making systems and its adverse effects. hence, the rising concern about gender bias in ai-based decision-making systems is gaining attention. in particular, there is a need for a better understanding of contributing factors and effective approaches to mitigating gender bias in ai-based decision-making systems. therefore, this study contributes to the existing literature by conducting a systematic literature review (slr) of the extant literature and presenting a theoretical framework for the management of gender bias in ai-based decision-making systems. the slr results indicate that the research on gender bias in ai-based decision-making systems is not yet well established, highlighting the great potential for future is research in this area, as articulated in the paper. based on this review, we conceptualise gender bias in ai-based decision-making systems as a socio-technical problem and propose a theoretical framework that offers a combination of technological, organisational, and societal approaches as well as four propositions to possibly mitigate the biased effects. lastly, this paper considers future research on the management of gender bias in ai-based decision-making systems in the organisational context. keywords: artificial intelligence, fairness, gender bias. 1 introduction artificial intelligence (ai)-based decision-making systems are now used in various industry sectors at an increasing rate and continue to penetrate all aspects of our daily lives. the current literature offers many examples of ai-based decision-making systems’ benefits. for instance, these systems have the potential to improve organisational operations and decision-making (kordzadeh & ghasemaghaei, 2021). also, recent research indicates that there has been an increased interest in ai-based decision-making systems during the recent covid-19 pandemic, because of reduced face-to-face human interaction and increased use of automation (collins et al., 2021). this in turn has further accelerated the use of these systems in various industries. however, while ai-based decision-making systems may offer solutions to various problems faced in different disciplines, they may simultaneously create unintended harmful effects, australasian journal of information systems nadeem, marjanovic & abedin 2022, vol 26, selected papers from acis 2020 gender bias in ai 2 including gender-biased outcomes affecting individuals or minorities of a certain race, gender, or colour (ntoutsi et al., 2019; eubanks, 2018; caplan et al., 2018; benjamin, 2019; west et al. 2019; unesco 2020). worryingly, ai-based decision-making systems are increasingly used to screen job applications, determine outcomes of loan applications, calculate insurance premiums and benefits, determine access to social services and more (mehrabi et al., 2019; caplan et al., 2018; marjanovic et al., 2021). the instances of gender-biased ai-based decisionmaking systems have already been reported in the scientific literature and popular press. for instance, facebook’s job ads, highly favoured males for stem (science, technology, engineering, and mathematics) jobs (lambrecht & tucker, 2019), and credit loan applications. also, amazon discontinued using an ai-based decision-making system for recruitment, which resulted in gender-biased outcomes (dastin, 2018; bolukbasi et al., 2016; kordzadeh, & ghasemaghaei, 2022). the ai gender-biased outcomes was due to the lack of female applicant data incorporated in the training datasets. the concern regarding gender bias in ai-based decision-making systems has also been raised by governments and research organizations (parikh et al., 2019; feast, 2019; parsheera, 2018; agarwal, 2020). the harmful effects of these systems go beyond individuals and are reported to affect families, communities, and society at large (altman et al., 2018). therefore, it is important to scrutinise ai-based decision-making systems for gender bias in order to ensure fairness in its outcomes, which is one of the fundamental principles of ai ethics (mehrabi et al., 2019; jobin et al., 2019). a greater understanding of this type of bias will also help organisations to make conscious strategic choices (marabelli et al., 2021). given the above, this paper aims to contribute to the emerging body of is literature on the unintended harmful effects of ai by focusing on gender bias in ai-based decision-making systems. the objectives of this study are (i) to identify and examine the characteristics of gender bias in ai-based decision-making systems, (ii) to investigate the role of relevant contributing factors behind gender bias in ai-based decision-making systems and the reported approaches to mitigation of gender bias in ai-based decision-making systems, and (iii) to propose a theoretical framework for the management of gender bias in ai-based decisionmaking systems. this paper is organised as follows: section 2 introduces the foundational concepts and further elaborates on the significance of this research; section 3 describes the adopted research method that is a systematic literature review process, along with the search criteria, selection of articles and analysis, and coding process; section 4 presents the findings of this research; section 5 discusses the proposed framework along with considerations for future research; section 6 offers the conclusion and discusses the study limitations. 2 foundational concepts ai-based it systems transform it systems from just representing reality to also actively participating in it and influencing it, and thereby these systems are explicitly demonstrating digital agency (baskerville et al., 2020; niehaus, & wiesche, 2021). in this research we follow markus (2017) and refer to a particular type of ai-based it systems which automate algorithmic decision-making based on computational models and among others natural language processing capabilities as ai-based decision-making systems where “automated decision-making is the process of making a decision by automated means without any human involvement. these decisions can be based on factual data, as well as on digitally created australasian journal of information systems nadeem, marjanovic & abedin 2022, vol 26, selected papers from acis 2020 gender bias in ai 3 profiles or inferred data. examples of this include: an online decision to award a loan; and an aptitude test used for recruitment which uses pre-programmed algorithms and criteria” (ico, 2018, p. 5). ai-based decision-making systems are now underpinning the digital economy; at the same time, they are also criticised regarding their fairness, accountability, and transparency (feuerriegel et al., 2020). consequently, there has been an outburst of research on fairness in ai-based decision-making systems in recent years (feuerriegel et al., 2020; bellamy et al., 2018; zhong, 2018; leavy, 2018; jobin et al., 2019). moreover, considerations of fairness in ai-based decision-making systems in organisations are still lagging, including fair practices within systems, people, and processes (feuerriegel et al., 2020). hence, is researchers and practitioners have been encouraged to work and collaborate towards ‘fair ai’ (feuerriegel et al., 2020). this also includes increasing concerns about, and reconsideration of the current approaches to bring fairness to ai-based decision-making systems (ntoutsi et al., 2019; feuerriegel et al., 2020; kordzadeh, & ghasemaghaei, 2021). dwivedi et al. (2019) argue that it is imperative to study fairness in ai-based decision-making systems as they are limited to industrial applications but have entered our lives on a daily basis. yet, the notion of ‘fairness’ remains unclear. for example, there are various, even mutually incompatible, definitions of fairness proposed by computer science researchers, with system and software developers unable to resolve these differences (teodorescu et al., 2021). at the same time, there are long-standing discussions on fairness within philosophical and theological literature for centuries, often in connection with justice (feuerriegel et al., 2020). in the absence of any well-established definition of fairness, in this paper, we draw from the previous work by merhrabi et al. (2019) who consider fairness as the elimination of any prejudice or favouritism behaviour towards a certain group or individuals. according to hayes et al., (2020), fairness prevents any action or policy that perpetuates discrimination or unequal treatment. hence, fairness refers to treating others the way one wants to be treated (teodorescu et al., 2021). an example of unfairness could be the act of disqualifying individuals who want to improve their financial conditions by rejecting their loan applications or job applications based on their gender, ethnicity, or the neighbourhood they live in (feuerriegel et al., 2020). while the concept of fairness is very broad, gender-related fairness is considered an essential aspect of fairness. gender bias, as defined by masiero and aaltonen (2020, p.1), is ‘the systemic, unfair difference in a way men and women are treated in a particular domain’. the related literature now provides strong evidence about gender bias in some ai-based decision-making systems (agarwal, 2020; altman et al., 2018; bolukbasi et al., 2016; canetti et al., 2019; crawford, 2016; dwivedi et al., 2019; galleno et al., 2019; lambrecht, & tucker 2019; mehrabi et al., 2019; nadeem et al., 2020; trewin et al., 2019). however, the research on gender-related biases in ai-based decision–making systems in is is still emerging (jobin et al., 2019; marabelli et al., 2021) there are still research gaps regarding our understanding of gender bias in aibased decision-making systems, particularly what causes gender bias in ai-based decisionmaking systems, the mitigation of this bias and possible prevention (leavy, 2018). additionally, there is a significant lack of research on how to manage bias in ai-based decision-making systems, including its harmful implications (berente et al., 2019; feuerriegel et al., 2020; kordzadeh, & ghasemaghaei, 2021). therefore, approaches to prevent, tackle and mitigate gender bias in ai-based decision-making systems are of high priority. australasian journal of information systems nadeem, marjanovic & abedin 2022, vol 26, selected papers from acis 2020 gender bias in ai 4 3 methodology to achieve the objectives of this research, we conducted a systematic literature review (slr) of the related literature in is and beyond. this is an appropriate research method as the research phenomenon is still emerging and a slr can be used to it systematically summarise and investigate previous findings (cao et al., 2015; webster & watson, 2002). the outcomes of a slr can further be used as a valued reference for future research (kitchenham et al., 2011; petersen et al., 2015, pare` et al., 2015). as borges et al. (2021) observe, the analysis of articles selected through slr yields a rich picture of various characteristics. also, systematic reviews allow researchers to examine the scope and range of research activities in a given domain by focusing on the breadth of the literature covered (pare` et al., 2015). our study adopts the slr approach introduced by bandara et al. (2011) and wolfswinkel et al. (2013). the approach enables researchers to conduct a conceptualised analysis of the literature and identify the key themes (wolfswinkel et al., 2013). the process of selection and identification of relevant articles was carried out using a rigorous method, as shown in figure 1. figure 1. selection of articles 3.1 search criteria for this study, we used scopus, which is one of the largest databases for journals and books (collins et al., 2021), and which has also been used by numerous ai researchers (borges et al., 2021). australasian journal of information systems nadeem, marjanovic & abedin 2022, vol 26, selected papers from acis 2020 gender bias in ai 5 the first step of our slr included a thorough investigation of the appropriate keywords’ selection. we initially performed a generic and multidisciplinary literature review (nadeem et al., 2020) for an iterative process of refining and selecting identified relevant keywords. the keywords were then further reviewed and selected based on the scope of this study. subsequently, we conducted a keyword search for the period from 2010 to 2020 to capture published research on gender bias in ai-based decision-making systems. the time frame of 10 years includes all relevant studies in high-quality journals is considered a reasonable and recommended time frame (borges et al., 2021). we selected computer science and business management (including is) as the subject areas. we also included social sciences and psychology to cover the social and behavioural aspects of gender bias. 3.2 selection of the articles the identification of articles in scopus started after the selection of the keywords, i.e., artificial intelligence, machine learning, analytics, gender, fairness. a total of 6,024 articles were captured through the selected keywords. to further filter the relevant articles, we applied inclusion criteria and source type (see figure 1). this was followed by exclusion criteria and time frame; articles that were outside the time frame (i.e., 2010 – 2020) and that were not written in english were excluded from the final set of articles. then, we started by reading the titles and abstracts of the identified articles. after selecting the articles on the basis of their titles and abstracts, we thoroughly read the full text of the articles. in this step, we considered only those articles that were directly dealing with gender bias in ai-based decision-making systems. hence, we excluded all papers that were outside the scope of this research, and ultimately 31 papers were selected that were relevant to our research scope. 3.3 analysis of the selected articles the analysis of the final set of 31 papers was carried out by in-depth reading of the articles. the relevant concepts and themes were identified by open coding, axial coding, and comparative analysis, as suggested by wolfswinkel et al., (2013), and through thematic analysis (pare` et al., 2015). the themes were initially coded by the first author independently and inductively, and then they were scrutinised by the other two authors for authentic and unbiased themes and outcomes. in the coding process, all codes with similar themes were integrated into one concept; f. ex. for the characteristics of gender bias in ai-based decisionmaking systems, the codes ‘societal gender prejudices’ and ‘discrimination’ were merged into ‘prejudices in society’ and later integrated into a concept ‘societal’ as shown in the appendix in table 2. similarly, when coding the approaches for mitigating gender bias in ai-based decision-making systems, the codes ‘bias aware collection of datasets’, ‘preparation of fair data sets’ and ‘removing proxies of protected attributes in data sets’ were merged into the theme ‘collection and preparation of dataset’, which was later integrated into the concept ‘ai technological approaches’ as shown in the appendix in table 4. 4 findings of the systematic literature review in the following, we present the key findings from the slr, which identify and categorise the insights about the manifestations of gender bias in ai-based decision-making systems and the contributing factors, as well as possible approaches to mitigate it. in doing so, we contribute to the emerging body of is literature on the potentially harmful effects of ai and their mitigation. australasian journal of information systems nadeem, marjanovic & abedin 2022, vol 26, selected papers from acis 2020 gender bias in ai 6 to reach an in-depth understanding of this area of research, we first identified and noted the type of published articles (i.e., conceptual research, literature review, design science, empirical, survey or case-based research), their use of theory and their focus as presented in table 1 in the appendix. our analysis shows that the majority of the reviewed articles have been conceptual papers. there is a lack of empirical and detailed literature review papers. moreover, the publication trend, as depicted by figure 2, indicates that the number of relevant publications started to grow in 2017 and then increased quite considerably in 2020, which confirms that this is a fastgrowing research area. further, our analysis discovers that although the topics of fairness and gender bias in ai have been discussed in the broader literature, the is field is yet to pay more attention to this important topic. our findings also indicate that the research on gender bias in ai is not yet well established, which highlights a great potential for future research in the is field and beyond. figure 2. publication trend of reviewed articles and type of selected articles 4.1 characteristics and contributing factors of gender bias in ai-based decision-making systems we adopt the term ‘characteristics’ to indicate various domains of gender bias in ai-based decision-making systems and observe three main characteristics of gender bias in ai-based decision-making systems: design and implementation, institutional and societal. the identified characteristics are intertwined (see figure 3). the long-standing societal inequalities and discriminatory norms propagate to organisational culture, thus affecting institutional practices and are manifest in the design and implementation of ai-based decision-making systems. when considering the contributing factors of gender bias in ai-based decision-making systems, we established six main themes relating to: gender stereotyping, biased training datasets, lack of gender diversity in ai development teams, ai amplifies existing bias, contextual and other factors and lack of ai regulations, (see also depicted by figure 3). these contributing factors are rooted within the characteristics of gender bias in ai-based decisionmaking systems. table 2 and 3 in the appendix include the sources, description and coding 0 5 10 15 20 2017 2018 2019 2020 publication trend of articles in this field literature review 16% empirical (case study) 10% conceptual and essays. 45% empirical (survey) 6% design science 23% type of article australasian journal of information systems nadeem, marjanovic & abedin 2022, vol 26, selected papers from acis 2020 gender bias in ai 7 process of the characteristics and contributing factors of gender bias in ai-based decisionmaking systems and are further discussed in the next subsection. 4.1.1 design and implementation characteristics the design of ai-based decision-making systems including any possible flaw in this phase can have an impact on the implementation and use of these systems (marabelli et al., 2021). a major challenge and reason for such flaws in the design of such systems is the misrepresentation of the datasets, i.e., ones that are either biased, incomplete, or incorrect (marabelli et al., 2021). according to hayes et al. (2020), societal gender inequalities are incorporated in the ai algorithms datasets due to unfair representation of datasets (i.e. over, under or misrepresentation of certain groups) and lack of gender diversity in the design of aibased decision-making systems that create ‘blind spots’ (johnson, 2019; lee, 2018; wang, 2020; martinez & fernandez, 2020; clifton et al., 2020). based on our literature review, the design and implementation characteristics are found to have the following contributing factors. figure 3. characteristics and contributing factors of gender bias in ai-based decision-making systems biased training datasets are patterns of unfairness in datasets (veale & binns, 2017) that are often based on the underor over-representation of social groups and that convert the computational training processes to a biased discriminative decision (ntoutsi et al., 2019). additionally, the correlation of data of sensitive variables and features (i.e., proxy variables) makes its way into ai algorithms and modelling and results in biased outcomes. past literature has noted that proxy variables discriminate against certain groups (e.g., salary serving as a proxy for gender and zip code serving as a proxy for background) (feuerriegel et al., 2020; ahn & lin, 2020; martinez & fernandez, 2020). further, word embedding not only preserves the statistical relationship present in the training data but also places co-occurring words close to each other, such as man is to king and is woman to queen (mikolov et al., 2013; brunet et al., 2019) and man is to computer programmer as woman is to homemaker (bolukbasi et al., australasian journal of information systems nadeem, marjanovic & abedin 2022, vol 26, selected papers from acis 2020 gender bias in ai 8 2016; brunet et al., 2019). also, gender stereotyping is predominant across different word embedding practices (see subsection 4.1.3) and as such not an artefact of a particular word training corpus or methodology (bolukbasi et al., 2016). for instance, female pronouns such as her or she, and the word woman are closely associated with family and arts, while the term male is largely associated with career, intelligence and maths (brunet et al., 2019). lack of gender diversity in ai development teams as shown in the literature confirms that the gender disparity in systems and software developers and data miners may result in gender bias during the training phase of the algorithms of ai-based decision-making systems (clifton et al., 2020; johnson, 2019; lee, 2018; martinez & fernandez, 2020; wang, 2020). as stated by feuerriegel et al. (2020), the lack of gender diversity in ai developers and other workers in science, technology, engineering and mathematics (stem) careers is reflective of a maledominated, homogeneous it industry, which may lead to a lack of diversity of mindsets in development teams (johnson, 2019; lee, 2018; wang, 2020) that develop ai-based decisionmaking systems. in turn, this may reinforce the dominance of one gender (male) and control over algorithms and decisions, yielding gender-biased outcomes. for instance, facial recognition software being used in usa in 2015 was unable to handle diversity well (daugherty et al., 2018; otterloo, 2019), and a huge amount of research has examined the difference in treatments of men and women in us criminal justice, with women being more likely to be arrested and sentenced than men (kulik et al., 1996). further, lambrecht & tucker (2019) investigated how advertisements promoting job opportunities in stem are viewed by more men than women, which eventually results in fewer women’s applications for stem jobs. the lack of human feedback and ‘humans-in-the-loop’ in ai-based decision-making systems may also amplify existing bias (johnson, 2019; miron et al., 2020) as well as the choice and application of certain modelling approaches during the training of the algorithms (chen, et al., 2019, feuerriegel et al., 2020;). once inscribed into the algorithmic training datasets, this bias is perpetuated due to the systems’ self-training. research found that some ai-based decisionmaking systems work better for certain groups of people over time, thus perpetuating inequalities in society by learning through biased outcomes. for instance, teodorescu et al. (2021) uncovered that gender disparity was perpetuated by facebook advertisements’ targeted job posting algorithms in which female applicants failed to see the job advertisements of companies that predominately hired male applicants. hence, if such algorithms are opaque and complex (miron et al., 2020) they may re-enforce their creators’, programmers’, developers’, designers’, software engineers’ and data miners’ bias (hayes et al., 2020; miron et al., 2020; ntoutsi et al., 2019; wang, 2020), further yielding gender bias in ai-based decisionmaking systems. the lack of proper testing of an algorithm for specific contexts may lead to decisions that disadvantage certain social groups in society (qureshi et al., 2020). in the context of ai-based decision-making systems, important contextual and external factors are often left unnoticed (marabelli et al., 2021). such contextual and external factors like third parties collecting the data, in the process might omit some important variables (johnson 2019; ntoutsi et al., 2019) and the lack of proper testing of an algorithm for specific contexts may lead to decisions that disadvantage certain social groups in society (qureshi et al., 2020). australasian journal of information systems nadeem, marjanovic & abedin 2022, vol 26, selected papers from acis 2020 gender bias in ai 9 4.1.2 institutional characteristics gender bias in ai-based decision-making systems is also manifested within institutions, with gender-biased decisions reported to be influenced by a broader societal context (see also subsection 4.1.3). ai-based decision-making systems both reflect and amplify the existing societal bias. in our literature review, we identified that a lack of ai regulations is a significant contributing factor to the institutional characteristics impacting on such systems. due to conscious or unconscious categorization between sociocultural groups, some institutions operate in ways that might disadvantage some minorities or social groups because of socioeconomic factors (costa & ribas, 2019; ntoutsi et al., 2019) (see also subsection 4.1.3). further, there is limited development in regulations toward addressing gender bias in ai-based decision-making systems (johnson, 2019). despite the european union’s (2021) recently presented proposal for firmer ai regulations (marabelli et al., 2021), there is a lack of more-precise ai guidelines for developers and institutions concerning fair ai in particular regarding data protection and data quality (johnson, 2019; ntoutsi et al., 2019; wang, 2020). thus, societal gender stereotyping and discrimination are amplified through institutional characteristics. this underlines the need and urgency for ai regulations and policy intervention for fairer ai (hoffmann, 2019; lee, 2018; ntoutsi et al., 2019). 4.1.3 societal characteristics the long-standing inequalities in society, i.e., gender stereotyping (johnson 2019; ntoutsi et al., 2019) leading to preferential treatment towards masculinity are often reflected in ai algorithms. these concepts normally ‘sneak in’ the datasets through the misguided conduct of ‘bad actors’ (hoffmann, 2019). hence, they connect the already-existing concept of gender stereotyping in society to gender bias in ai-based decision-making systems (cirillo et al., 2020; clifton et al., 2020; johnson, 2019; martinez & fernandez, 2020; noriega, 2020; ntoutsi et al., 2019; sun et al., 2019; wang, 2020 based on our literature review, we found that gender stereotyping as a contributing factor to gender bias in ai-based decision-making systems occurs in societies where historical biases and norms are followed not because of conscious discrimination but rather because the majority following the pre-existing customs presents a culture that promotes masculinity and exclusivity (johnson, 2019; ntoutsi et al., 2019), this biased behaviour is inscribed in ai systems; therefore, ai-based decision-making systems reflect human biases toward people from a certain background, race, or gender. also, certain upstream social norms are followed blindly because of their easy acceptance in society (ntoutsi et al., 2019). for instance, certain professions are associated with males e.g., doctors, engineers, and scientists, while professions like nursing and secretary work are associated with females. the wording and association of certain professions with certain genders sow unequal division and discrimination in society. related to gender stereotyping, socioeconomic factors also impact and amplify gender bias in ai-based decision-making systems i.e., socio-economic factors based on social standing (e.g., neighbourhood, zip code and location) result in incorrect assumptions of an individual. people with lower socio-economic backgrounds may be disadvantaged in a society due to their social status and standing. this biased behaviour may be reflected in ai-based decision-making systems because of the unconscious bias of those who develop these systems (clifton et al., australasian journal of information systems nadeem, marjanovic & abedin 2022, vol 26, selected papers from acis 2020 gender bias in ai 10 2020; wang, 2020) for instance, the amazon delivery system excluded certain socio-economic neighbourhoods due to socioeconomic stereotyping in the society impacting ai-based decision-making systems and their outcomes (dastin, 2018). 4.2 approaches to mitigating gender bias in ai-based decision-making systems based on the reviewed literature, we observe proposals for four main approaches to possibly mitigate gender bias in ai-based decision-making systems, which are: ai technology-related approaches, fair ai management approaches, ai governance and regulatory approaches, and societal and community-focused approaches. table 4 in the appendix presents the sources, description, and coding process, which resulted in the identified approaches. 4.2.1 ai technology-related approaches clifton et al. (2020) propose a strategy of capturing data from all vulnerable, gender-diverse groups of society and adding multi-dimensional datasets during the design of the decisionmaking algorithms, which in their view can neutralise inappropriate and/or unfair datasets. similarly, hayes et al. (2020) argue that a fair representation of the population in data sets will result in fair ai outcomes. researchers also call for ai-based decision-making algorithms to be designed and programmed in such a way that they do not replicate prejudices and gender bias while analysing and interpreting the data (johnson 2019; ntoutsi et al., 2019). if the implementation context does not match training datasets, the resulting ai-based decision-making systems are unlikely to perform well, i.e., lead to biased outcomes (hardt & price, 2016). thus, testing the algorithms for a specific application increases the accountability and bias detection procedures (ahn & lin, 2020; arrieta et al., 2020; bellamy et al., 2018; berk et al., 2018; feuerriegel et al. 2020; grari et al., 2020; johnson, 2019; lambrecht, & tucker, 2019; martin, 2019; miron et al., 2020; ntoutsi et al., 2019; thelwall, 2017; veale & binns, 2017). context-specific decision-making algorithms are put forward as being more effective; however, they require continuous re-design as per the specific contextual conditions (marabelli et al., 2021). appropriate choices for ai-based decision-making systems need to be made, depending on the context in which they are being used (marabelli et al., 2021). moreover, paying additional attention to the context of the ai-based decision-making systems, data and people involved can effectively decrease their discriminatory outcomes (marabelli et al., 2021). 4.2.2 fair ai management approaches hayes et al. (2020) found that, due to unethical data practices, misreporting of data and other misconducts related to data collection and preparation as well as the ability of ai decisionmaking algorithms to self-learn from the so-called pernicious feedback of their own, biased decisions worsen the ai outcomes. to address this concern, researchers (johnson, 2019; miron et al., 2020) suggest incorporating testing and auditing the ai-based decision-making algorithms into the design and implementation phase. this could involve external auditors or internal compliance auditors (martinez & fernandez, 2020; kyriazanos et al., 2019). for example, ai experts can maintain regular testing and verification of ai-based decision-making systems and can also use interpretation tools to diagnose potential problems and challenges (wu et al., 2019). australasian journal of information systems nadeem, marjanovic & abedin 2022, vol 26, selected papers from acis 2020 gender bias in ai 11 other recommended strategies focus on human-decision makers. if given authority, humans actively involved in the ultimate decision-making can effectively adjust the outcomes provided by the technical components of ai-based decision-making systems (hayes et al., 2020). further inclusiveness and diversity training to decision-makers is also suggested as an approach to avoid unconscious bias and, most importantly, understand and identify of when to intervene in the ai proposed decision (hayes et al., 2020). likewise, institution-wide education that involves principles of ethics, such as promoting ethical education for every stakeholder involved in ai practices (martin, 2019; noriega, 2020), is reported to assist in detecting gender-biased outcomes. other education-based approaches include professional certifications and courses focused on building awareness of gender bias in ai-based decisionmaking systems (martin, 2019). in addition, increased and enhanced ai corporate governance regarding gender inclusion in the development of ai technologies is suggested as a strategy to introduce diverse perspectives (costa & ribas, 2019; johnson, 2019; lee, 2018; ntoutsi et al., 2019) and to ensure that gender bias is addressed in ai (ibrahim et al., 2020). enhanced gender diversity and inclusion in the technology sector, especially in development for ai-based decision-making systems (lambrecht & tucker, 2019), is proposed to avoid homogeneous and predominantly male-dominated leaderships and decisions (johnson, 2019). when applied to ai, the inclusion of a more diverse it workforce in the design and implementation of algorithms and diversity of thoughts in ai development is reported to bring a multicultural perspective to ai design, which in turn might mitigate gender bias (ibrahim et al., 2020; wu et al., 2019). organisations also need to develop fair and ethical internal structures, corporate strategies, and governance to manage gender imbalance; gender diversity among board members, management, senior developers, and general leadership encourages people from diverse backgrounds, and offers pathway towards mitigating gender bias (johnson, 2019). 4.2.3 ai governance and regulatory approaches the beforementioned recent proposal by the european union (2021) highlights the significance and urgency of creating ai regulations for dealing with humans. introducing rules and policies governing ai-based decision-making systems ensures better efficiency in the resulting decisions (marabelli et al., 2021). hayes et al. (2020) argue that institutional ai regulations should be developed and implemented to increase transparency and accountability in ai-based decision-making systems. at the same time, ai algorithms should not be designed in a way that precludes individuals from taking responsibility (martin, 2019). similarly, researchers argue that users who are affected by ai decisions should have the right to know and comprehend the reasons behind those decisions and share their feedback on them (wu et al., 2019). having regulations relating to formal verification, adhering to ai ethical values and testing ai-based decisionmaking systems in place, is envisaged to result in fair outcomes (lee, 2018; wu et al., 2019). this includes datasets purchased from third parties that need to be properly analysed for the particular context in which they will be used (johnson, 2019), as well as enhanced ethical ai standards by government and regulatory organisations about data collection and selection (cirillo et al., 2020). ai governance across interdisciplinary and multinational collaborations is suggested to establish a census on ai principles, which in turn enhances the general practice of responsible australasian journal of information systems nadeem, marjanovic & abedin 2022, vol 26, selected papers from acis 2020 gender bias in ai 12 ai conduct (wu et al., 2019). recognising the need for knowledge sharing, researchers propose collaborative ethical ai online platforms for all stakeholders, which would permit demographically diverse organisations to collaborate and share knowledge regarding the appropriate and practical strategies to promote fairness in ai systems (soleimani et al., 2021; veale & binns, 2017). 4.2.4 societal and community-focused approaches hayes et al. (2020) and prates et al. (2019) propose to encourage social interventions such as enhancing professional education and training on gender diversity in the community to boost diversity and inclusiveness. public policies to protect the personal data of users are also proposed as a possible approach to increase confidence in ai-based decision-making systems, in particular when it comes to sharing personal data for such decision-making process (clifton et al., 2020). cirillo et al. (2020) propose an ‘ecosystem of trust’ by government or policymakers to ensure that systems comply with the fundamental rules that protect both human and consumer rights, particularly in ai-based decision-making systems. 5 discussion and contributions the findings of this study contribute to an improved understanding of the state of gender bias in ai-based decision-making systems. so far, there has been ample work on exploring and reducing the bias in ai-based decision-making systems through technical approaches. research communities such as fat machine learning (fairness, accountability, and transparency in machine learning) have emphasised bringing fairness to ai algorithms through programming and mathematical modelling (veale & binns, 2017). consequently, many ai researchers see gender bias in ai-based decision-making systems as a technological problem (ahn & lin, 2020; arrieta et al., 2020; bellamy et al., 2018; grari et al., 2020; lee 2018; miron et al., 2020; ntoutsi et al., 2019; veale & binns, 2017). however, the roots of gender bias in ai-based decision-making systems are not technological, and thus, technical solutions might not suffice (nadeem et al., 2021). information systems research lags behind in addressing the behavioural, organisational, and social implications, antecedents, and consequences of this problem, despite the fact that computational scientists have developed mathematical techniques to detect and mitigate biases in algorithms (kordzadeh & ghasemaghaei, 2021; sarker et al., 2019). australasian journal of information systems nadeem, marjanovic & abedin 2022, vol 26, selected papers from acis 2020 gender bias in ai 13 figure 4. proposed framework for the management of gender bias in ai-based decision-making systems the bias in ai-based decision-making systems cannot be corrected by merely fixing the decision-making algorithms; ‘this is not an algorithmic problem’, as stated by teodorescu et al. (2021). ai-based decision-making systems are multidisciplinary phenomena that call for the collaboration of experts representing technological, organizational, and human perspectives (marabelli et al., 2021). moreover, bias in decision-making algorithms is socio-technical in nature, and thus social implications of this phenomenon should be at the centre of its examination and potential solution (kordzadeh & ghasemaghaei, 2021). consequently, we conceptualise gender bias in ai-based decision-making systems as multilayered, multidimensional, and socio-technical with the systems’ development and implementation requiring a combination and integration of technical, organizational, and societal approaches. for this purpose, we systematically reviewed the existing literature and build in particular on the previous work from marabelli et al. (2021) and kordzadeh & ghasemaghaei (2021) to advance the conversation about possible technological, organizational, human and societal mitigating approaches. as a result, we propose a theoretical framework for the management of gender bias in ai-based decision-making systems (see below figure 4). the proposed theoretical framework synthesizes previously reported contributing factors and approaches and consolidates them in a theoretical framework. as part of the proposed framework and as a summary of our findings we offer four theoretical propositions for the possible mitigation of gender bias in ai-based decision-making systems. p1: ai technology-related approaches can mitigate design and implementation-related contributing factors. australasian journal of information systems nadeem, marjanovic & abedin 2022, vol 26, selected papers from acis 2020 gender bias in ai 14 removing proxy variables of protected attributes and ensuring fair datasets from all groups and members of a community i.e., diverse and inclusive datasets, are reported to be effective in mitigating gender bias in the design and implementation of ai systems (bellamy et al., 2018; feuerriegel et al., 2020; grari et al., 2020; hayes et al., 2020; miron et al., 2020; veale & binns, 2017). having a document or guideline on fair datasets for developers can support fair outcomes and prevent unfairness in training data by ensuring fair data collection, data preparation and regularising the training data to minimise the unfairness (bellamy et al., 2018; ntoutsi et al., 2019). such measures, which strictly speaking are socio-technical approaches, could be the pairing of data scientists with social scientists to achieve multidisciplinary for the design and implementation and for effectively mitigating gender bias in ai-based decisionmaking systems (marabelli et al., 2021). further, enhanced and constant testing for algorithmic accountability and transparency can improve the understanding and explanation of bias detection of algorithmic models and structures (ntoutsi et al., 2019). hence, we propose that ai technology-related and diversity mitigating approaches can be used to address the design and implementation-related factors that contribute to gender bias in ai-based decision-making systems (bellamy et al., 2018; feuerriegel, et al., 2020; grari et al., 2020; hayes et al., 2020; johnson, 2019; lee, 2018; miron et al., 2020; noriega, 2020; ntoutsi et al., 2019; veale & binns, 2017; wu et al., 2019). p2: ai governance and regulatory approaches can mitigate institutional-related contributing factors. ai regulations that enforce to incorporate key ethical standards (ntoutsi et al., 2019; wang, 2020), adhering to laws and policies for better ai governance, auditing and gender diversity and inclusiveness in organisations concerning fair ai (feuerriegel et al., 2020) are all reported to result in mitigating gender bias in ai-based decision-making systems. hence, we propose that ai governance and regulatory approaches can be used to mitigate the institutional-related contributing factors (feuerriegel et al., 2020; ntoutsi et al., 2019; wang, 2020). p3: fair ai management approaches can mitigate institutional-related contributing factors. ai to be fair by design (arrieta et al., 2020) practiced in organisations with policies and business models concerning fair ai (feuerriegel et al., 2020) includes implementing inclusive policies and regulations within the organisations and bringing about algorithmic accountability and transparency (johnson 2019; ntoutsi et al., 2019). fair ai management mitigation approaches through awareness and promoting policies are reported to ensure having ‘humans in the loop’ which increases the chance of fairness provided by ai-based decision-making systems (teodorescu et al., 2021). in particular, creating awareness through training, workshops, and seminars at the organisational level regarding gender-biased outcomes of ai-based decision-making systems can encourage ai developers and users of such systems to enforce gender-diverse workplaces and public policies regarding fairness to support demographic and cultural diversity in data that is used by the systems (lee, 2018). hence, we propose that fair ai management approaches can be used to mitigate the institutional-related contributing factors (hayes et al., 2020; lee, 2018; marabelli et al., 2021; teodorescu et al. 2021; wu et al., 2019). australasian journal of information systems nadeem, marjanovic & abedin 2022, vol 26, selected papers from acis 2020 gender bias in ai 15 p4: societal and community-focused approaches can mitigate societal-related contributing factors. gender bias in society is found to be replicated in emerging technologies, i.e., including ai (kordzadeh & ghasemaghaei, 2021). emphasising social interventions – f. ex., awareness of gender equity and fairness in society through social and educational aspects such as workshops, seminars, etc. – is reported to be effective in mitigating the socially manifested gender bias in society (hayes et al., 2020; johnson, 2019). moreover, certain public policies that protect fundamental rights and societal well-being, if enhanced, bring awareness to human rights and work against gender bias and other discrimination (clifton et al., 2020; miron et al., 2020). hence, we propose that societal & community-focused approaches can be used to mitigate societal-related contributing factors (clifton et al., 2020; hayes et al., 2020; johnson, 2019; kordzadeh, & ghasemaghaei, 2021; miron et al., 2020). 6 future research the offered propositions are not exhaustive. therefore, further research is needed to develop more propositions which along with the ones we have proposed, must be refined and empirically tested. based on the proposed theoretical framework, we suggest future is research related to the prevention, mitigation, and future theorizing of gender bias in ai-based decision-making systems from an is perspective. while this study presents proposed approaches for mitigating the contributing factors that generate gender bias in ai-based decision-making systems through systematically reviewing the existing literature, it is important to empirically investigate the proposed approaches. another interesting opportunity for further research is to study how societal gender bias is manifested in institutional ai practices and vice versa as there is a lack of contextually rich theories in this domain, that examine these practices in broader institutional and regulatory structures (conboy et al., 2022) will be interesting. in this context it is interesting to investigate how regulations can shape gender bias in ai in an organisational context. we therefore suggest to further evolve the proposed theoretical framework for the management of gender bias in ai-based decision-making systems through future theoretical and empirical research and contribution. we expect such research to build the foundations for new frameworks that, as our systematic literature review confirms, are very much needed. more research on context-specific ai algorithms will be beneficial. hence, the proposed framework could be further explored in specific organisational and societal contexts. 7 concluding remarks and study limitations gender-related bias is of vital concern in ai-based decision-making systems that are now used in organisational and societal contexts. therefore, it is important to unpack the status of gender bias in ai-based decision-making systems in the literature and systematically analyse the findings of such reviews for better understanding and mitigation of possible harmful effects. this paper has contributed to the conversation on gender bias in ai-based decision-making systems by identifying and investigating reported design and implementation, institutional australasian journal of information systems nadeem, marjanovic & abedin 2022, vol 26, selected papers from acis 2020 gender bias in ai 16 and societal approaches to potentially mitigating gender bias in ai-based decision-making systems. we conceptualise gender bias in ai-based decision-making systems as a socio-technical problem that affects a variety of stakeholders, including the workforce, and society in general. we identify some key characteristics that manifest gender bias in ai-based decision-making systems along with the associated contributing factors and possible approaches for potential mitigation that we developed based on the existing literature and timely industry examples. hence, a framework is proposed to guide ai designers, developers, and other stakeholders to ensure the management of ai by mitigating gender bias in ai-based decision-making systems. our research findings also suggest that organisations need to be actively engaged in the implementation of ethical and fair ai outcomes. in particular, our findings highlight strategies related to workplace diversity, further education on ethical and fair ai as well as improved transparency and accountability in algorithmics. in addition, training and certification on ethical and fair ai should be considered for new employees and reinforced periodically. moreover, organisations should have ai governance strategies in place, which should support the prevention, detection, and mitigation of gender bias in their ai-based decision-making systems. we recognise that our study has several limitations. first, different keywords are likely to result in a different pool of related research work. moreover, a systematic literature review of a wider group of is and other journals, along with more databases such as the isi web of science could be used to obtain more detailed results. finally, as stated earlier, we also acknowledge the need for the empirical validation of the proposed framework and propositions. our current work includes such empirical research including expert interviews and case studies. references agarwal, p. (2020). gender bias in stem: women in tech still facing discrimination. forbes. retrieved from https://www.forbes.com/sites/pragyaagarwaleurope/2020/03/04/gender-bias-in-stemwomen-in-tech-report-facing-discrimination/?sh=72c9e78670fb ahn, y., lin, y.r. (2020). fair sight: visual analytics for fairness in decision making. ieee transactions on visualization and computer graphics, 26 (1). doi: https:// doi.org/10.1109/tvcg.2019.2934262 altman, m., wood, a., vayena, e. (2018). a harm-reduction framework for algorithmic fairness. ieee security & privacy, 16(3), 34-45. arrieta, a. b., diaz – rodriguez, n., ser, j. d., bennetot, a., tabik, s., barbado, a., garcia, s., gillopez, s., molina, d., benjamins, r., chatila, r., herrera, f. (2020). explainable artificial intelligence (xai): concepts, taxonomies, opportunities, and challenges towards responsible ai. information fusion, 58, 82-115. bandara, w., miskon, s., fielt, e. (2011). a systematic, tool-supported method for conducting literature reviews in information systems. in proceedings of the 19th european conference on information systems, helsinki, finland. https://aisel.aisnet.org/ecis2011/221 australasian journal of information systems nadeem, marjanovic & abedin 2022, vol 26, selected papers from acis 2020 gender bias in ai 17 baskerville, r. l., myers, m. d., & yoo, y. (2020). digital first: the ontological reversal and new challenges for is. mis quarterly, 44(2), 509 -523. doi: 10.25300/misq/2020/14418 bellamy, r. k. e., dey, k., hind, m., hoffman, s. c., houde, s., kannan, k., lohia, p., martino, j., mehta, s., mojsilovic, a., nagar, s., ramamurthy, k. n., richards, j., saha, d., sattigeri, p., singh, m., varshney, k. r., zhang, y. (2018). ai fairness 360: an extensible toolkit for detecting, understanding, and mitigating unwanted algorithmic bias. journal of research and development, 1 (1), 99. doi: https://arxiv.org/pdf/1810.01943.pdf berente, n., seidel, s., safadi, h. (2019). data-driven computationally-intensive theory development. information systems research, 30(1), 50-64. berente, n., gu, b., recker, j., santhanam, r. (2019). managing ai: special issue, mis quarterly, 45(3), 1433-1450. benjamin, r. (2019). assessing risk, automating racism. a health care algorithm reflects underlying racial bias in society. social science. doi: 10.1126/science.aaz3873 berk, r., heidari, h., jabbari, s., kearns, m., roth, a. (2018). fairness in criminal justice risk assessments: the state of the art. sociological methods and research. 50 (1). https://doi.org/10.1177/0049124118782533 bolukbasi, t., chang, k.w., zou, j., saligrama, v., kalai, a. (2016). man is to computer programmer as woman is to homemaker? debiasing word embeddings. in proceedings of the 30th international conference on neural information processing systems (pp. 4356– 4364). https://dl.acm.org/doi/10.5555/3157382.3157584 borges, a. f. s., laurindo, f.j.b., spinola, m. m., goncalves, r. f., mattos, c.a. (2021). the strategic use of artificial intelligence in the digital era: systematic literature review and future research directions. international journal of information management, 57. doi: https://doi.org/10.1016/j.ijinfomgt.2020.102225 canetti, r., cohen, a., dikkala, n., ramnarayan, g., scheffler, s., smith, a. (2019). from soft classifiers to hard decisions: how far can we be? proceedings of the conference on fairness, accountability, and transparency, 309-318. doi: https://doi.org/10.1145/3287560.3287561 cao, j., basoglu, k., sheng. h., lowry, p. (2015). a systematic review of social network research in information systems: building a foundation for exciting future research. communications of the association for information systems, 36(37), 727-758. caplan, r., donovan, j., hanson, l., matthews, j. (2018). algorithmic accountability: a primer: prepared for the congressional progressive caucus. data & society, washington, dc, usa. doi: https://datasociety.net/library/algorithmic-accountability-a-primer/ chen, i. y., szolovits, p., ghassemi, m. (2019). can ai help reduce disparities in general medical and mental health care? ama journal of ethics, 22 (2), 167179. cirillo, d., catuara – solarz, s., morey, c., guney, e., subirats, l., mellino, s., gigante, a., valencia, a., rementeria, m. j., chadha, a. s., nikolaos, m. (2020). sex and gender differences and biases in artificial intelligence for biomedicine and healthcare. digital medicine, 8 (3). doi: https://www.nature.com/articles/s41746-020-0288-5 clifton, j., glasmeier, a., gray, m. (2020). when machines think for us: the consequences for work and place. cambridge journal of regions, economy, and society,13(1), 3-23. https://dl.acm.org/doi/proceedings/10.5555/3157382 australasian journal of information systems nadeem, marjanovic & abedin 2022, vol 26, selected papers from acis 2020 gender bias in ai 18 costa, p., ribas, l. (2019). ai becomes her: discussing gender and artificial intelligence. a journal of speculative research,17 (2), 171-193. conboy, k., crowston, k., lundstrom, j.e., jarvenpaa, s., ram, s., mikalef, p. (2022). artificial intelligence in information systems: state of the art and research roadmap. communications of the association for information systems, 50, 420438. collins, c., dennehy, d., conboy, k., mikalef, p. (2021). artificial intelligence in information system research: a systematic literature review and research agenda. international journal of information management, 60. doi: https://doi.org/10.1016/j.ijinfomgt.2021.102383 crawford, k. (2016). a.i.’s white guy problem. (sunday review desk) (opinion). the new york times. doi:https://www.nytimes.com/2016/06/26/opinion/sunday/artificialintelligences-white-guy-problem.html dastin, j. (2018). amazon scraps secret ai recruiting tool that showed bias against women. reuters. doi: https://www.reuters.com/article/us-amazon-com-jobs-automation-insightiduskcn1mk08g daugherty, p., wilson, h., chowdhury, r. (2018). using artificial intelligence to promote diversity. mit sloan management review. doi:https://sloanreview.mit.edu/article/usingartificial-intelligence-to-promote-diversity/ dawson, d., schleiger, e., horton, j., mclaughlin, j., robinson, c., quezada, g., scowcroft, j., hajkowicz, s. (2019). artificial intelligence: australia’s ethics framework. data61 csiro, australia. doi: https://www.csiro.au/en/research/technology-space/ai/ai-ethicsframework dwivedi, y.k., hughes, l, …, williams, m. d. (2019). artificial intelligence: multidisciplinary perspectives on emerging challenges, opportunities, and agenda for research, practice and policy. international journal of information management, 57(7). doi:10.1016/j.ijinfomgt.2019.08.002 eubanks, v. (2018). automating inequalities: how high-tech tools profile, police, and punish the poor. law technology and humans. doi: https://dl.acm.org/doi/10.5555/3208509 european union (2021). europe fit for the digital age: commission proposes new rules and actions for excellence and trust in artificial intelligence. available at https://ec.europa.eu/commission/presscorner/detail/en/ip_21_1682 feuerriegel, s., dolata, m., schwabe, g. (2020). fair ai: challenges and opportunities. business and information systems engineering, 62(4), 379-384. feast, j., (2019). 4 ways to address gender bias in ai, harvard business review. doi: https://hbr.org/2019/11/4-ways-to-address-gender-bias-in-ai galleno, a., krentz, m., tsusaka, m., yousif, n. (2019). how ai could help or hinder women in the workforce. boston consulting group. doi:https://www.bcg.com/publications/2019/artificial-intelligence-ai-help-hinderwomen-workforce. grari. v., ruf. b., lamprier. s., detyniecki. m. (2020). achieving fairness with decision tress: an adversarial approach. data science and engineering, 5(2). 99110. https://hbr.org/2019/11/4-ways-to-address-gender-bias-in-ai australasian journal of information systems nadeem, marjanovic & abedin 2022, vol 26, selected papers from acis 2020 gender bias in ai 19 hardt, m., price, e., srebro, n. (2016). equality of opportunity in supervised learning. in proceeding of the 30th international conference on neural information processing systems. doi: https://proceedings.neurips.cc/paper/2016/file/9d2682367c3935defcb1f9e247a97c0dpaper hayes, p., poel, i.v.d., steen, m. (2020). algorithms and values in justice and security. ai & society, 35 (3), 533555. hoffmann, a.l. (2019). where fairness fails: data, algorithms, and the limits of antidiscrimination discourse. information, communication & society, 22(7), 900-915. ico (2018). what is automated individual decision-making and profiling. uk information commissioner’s office, 1-23. doi: https://ico.org.uk/for-organisations/guide-to-dataprotection/guide-to-the-general-dataprotection-regulation-gdpr/automated-decisionmaking-and-profiling/what-is-automated-individual-decisionmaking-andprofiling/#id2 ibrahim, s.a., charlson, m.e., neill, d.b. (2020). big data analytics and the structure for equity in healthcare: the promise and perils. health equity, 4 (1), 99101. johnson, k.n. (2019). automating the risk of bias. george washington law review, 87(6). doi: https://www.gwlr.org/wp-content/uploads/2020/01/87-geo.-wash.-l.-rev.-1214.pdf jobin, a., lenca, m., vayena, e. (2019). the global landscape of ai ethics guidelines. nature machine intelligence, 1, 389-399. kaplan, a., haenlein, m. (2019). siri, siri, in my hand: who’s the fairest in the land? on the interpretations, illustrations, and implication of artificial intelligence. business horizons, 62, 15-35. kitchenham, b., budgen, d., brereton, o.p. (2011). using mapping studies as the basis for further research – a participant -observer case study. information and software technology, 53 (6), 638651. kordzadeh, n., ghasemaghaei, m. (2021). algorithmic bias: review synthesis, and future research directions. european journal of information systems, 31 (3), 388409. kyriazanos, d. m., thanos, k.g., thomopoulos, s.c.a. (2019). automated decisions making in airports checkpoints: bias detection toward smarter security and fairness. ieee security & applications, 17 (2), 8-16. lambrecht, a., tucker, c. (2019). algorithmic bias? an empirical study of apparent genderbiased discrimination in the display of stem career ads. management science, 65(7), 29473448. leavy, s. (2018). gender bias in artificial intelligence: the need for diversity and gender theory in machine learning. 2018 ieee/acm first international workshop on gender equality in software engineering, gothenburg, sweden. doi: https://ieeexplore.ieee.org/document/8452744 lee, n. t. (2018). detecting racial bias in algorithms and machine learning. journal of information, communication, and ethics in society, 16 (3), 252-260. australasian journal of information systems nadeem, marjanovic & abedin 2022, vol 26, selected papers from acis 2020 gender bias in ai 20 marabelli, m., newell, s., handunge, v. (2021). the lifecycle of algorithmic decisionmaking systems: organizational choices and ethical challenges. journal of strategic information systems, 30(3). doi: https://doi.org/10.1016/j.jsis.2021.101683 markus (2017). datification, organizational strategy, and is research: what’s the score? the journal of strategic information systems, 26(3), 233-241. doi:https://doi.org/10.1016/j.jsis.2017.08.003. martinez, c. f., fernandez, a. (2020). ai and recruiting software: ethical and legal implications. journal of behavioural robotics, 11(1). doi: https://doi.org/10.1515/pjbr-20200030 martin, k. (2019). ethical implications and accountability of algorithms. journal of business ethics, 160, 835850. marjanovic, o., cecez-kecmanovic, d., vidgen, r. (2021). algorithmic pollution: making the invisible visible. journal of information technology, 36(3), 391-408. masiero, s., aaltonen, a. (2020). gender bias in is research: a literature review. in proceedings of the 41st international conference on information systems, hyderabad, india. doi: http://dx.doi.org/10.2139/ssrn.3751440 mehrabi, n., morstatter, f., saxena, n., lerman, k., galstyan, a. (2019). a survey on bias and fairness in machine learning. acm computing surveys 54 (6). https://doi.org/10.1145/3457607 mikolov, t, chen, corrado, g. s., dean, j. (2013). efficient estimation of word representations in vector space. in proceedings of the international conference on learning representations. retrieved from https://storage.googleapis.com/pub-tools-public-publication-data/pdf/41224.pdf miron, m., tolan, s., gomez, e., castillo, c. (2020). evaluating causes of algorithmic bias in juvenile criminal recidivism. artificial law and intelligence, 29(2), 111-147. nadeem, a., abedin, b., marjanovic, o. (2020). gender bias in ai: a review of contributing factors and mitigating strategies. in proceedings of the australasian conference on information systems, wellington, new zealand. https://aisel.aisnet.org/acis2020/27 nadeem, a., marjanovic, o., abedin, b. (2021). gender bias in ai: implications for managerial practices. 13e 2021. responsible ai and analytics for an ethical and inclusive digitized society. doi: https://link.springer.com/chapter/10.1007/978-3-030-85447-8_23 niehaus, f., wiesche, m. (2021). a socio-technical perspective on organizational interaction with ai: a literature review. in proceedings of european conference on information systems 2021. doi: https://aisel.aisnet.org/ecis2021_rp/156 ntoutsi, e., fafalios, p., …, staab, s., (2019). bias in data-driven artificial intelligence systems – an introductory survey. data mining and knowledge discovery, 10(3). doi: https://doi.org/10.1002/widm.1356 noriega., m. (2020). the application of artificial intelligence in police interrogations: an analysis addressing the proposed effect ai has on racial and gender bias, cooperation, and false confessions. futures, 117. doi:10.1016/j.futures.2019.102510 https://doi.org/10.1016/j.jsis.2021.101683 https://doi.org/10.1515/pjbr-2020-0030 https://doi.org/10.1515/pjbr-2020-0030 https://dx.doi.org/10.2139/ssrn.3751440 https://dl.acm.org/toc/csur/2022/54/6 https://doi.org/10.1002/widm.1356 http://dx.doi.org/10.1016/j.futures.2019.102510 australasian journal of information systems nadeem, marjanovic & abedin 2022, vol 26, selected papers from acis 2020 gender bias in ai 21 pare`, g., trudel, m-c., jaana, m., kitsiou, s. (2015). synthesizing information systems knowledge: a typology of literature reviews. information & management, 52(2) 183-199. parikh, r.b., teeple, s., navathe, a.m. (2019). addressing bias in artificial intelligence in health care. jama. doi: https://pubmed.ncbi.nlm.nih.gov/31755905/ parsheera, s. (2018). a gendered perspective on artificial intelligence. in proceeding of itu kaleidoscope – machine learning for a 5g future. doi: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3374955 petersen, k., vakkalanka, s., kuzniarz, j. (2015). guidelines for conducting systematic mapping studies in software engineering: an update. information and software technology, 64, 1-18. paulus, j. k., kent, d. m. (2020). predictably unequal: understanding and addressing concerns that algorithmic clinical prediction may increase health disparities. digital medicine, 99 (3). https://doi.org/10.1038/s41746-020-0304-9 piano, s.l. (2020). ethical principles in machine learning and artificial intelligence: cases from the field and possible ways forward. humanities and social sciences communications. doi: https://www.nature.com/articles/s41599-020-0501-9 prates, m., avelar, p., lamb, l.c. (2018). assessing gender bias in machine translation – a case study with google translate. neural computing and applications. 32, 6363 – 6381. qureshi, b., kamiran, f., karim, a., ruggieri, s., pedreschi, d. (2020). causal inference for social discrimination reasoning. journal of intelligent information systems, 54, 425-437. robert, l.p., pierce, c., marquis, l., kim, s., alahmad, r. (2020). designing fair ai for managing employees in organizations: a review, critique, and design agenda. human computer interaction. 35(5-6). doi: https://doi.org/10.1080/07370024.2020.1735391 robnett, r. d. (2015). gender bias in stem fields: variation in prevalence and links to stem self-concept. psychology of women quarterly, 40(1). https://doi.org/10.1177/0361684315596162 sarker, s., chatterjee, s., xiao, x., elbanna, a. (2019). the socio technical axis of cohesion for the is discipline: its historical legacy and its continued relevance. mis quarterly, 43(3), 695-719. sun, t., gaut, a., tang, s., huang, y., elsherief, m., zhao, j., mirza, d., belding, e., chang, k. w., wang, w. y. (2019). mitigating gender bias in natural language processing: a literature review. in proceeding of 57th annual meeting of association for computational linguistics, florence, italy. doi: 10.18653/v1/p19-1159 soleimani, m., intezari, a., pauleen, d.j. (2021). mitigating cognitive biases in developing aiassisted recruitment systems: a knowledge-sharing approach. international journal of knowledge management, 18(1). doi: 10.4018/ijkm.290022 schonberger, d. (2019). artificial intelligence in healthcare: a critical analysis of the legal and ethical implications. international journal of law and information technology, 27 (2), 171203. https://pubmed.ncbi.nlm.nih.gov/31755905/ https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3374955 https://doi.org/10.1080/07370024.2020.1735391 http://dx.doi.org/10.18653/v1/p19-1159 australasian journal of information systems nadeem, marjanovic & abedin 2022, vol 26, selected papers from acis 2020 gender bias in ai 22 teodorescu, m., morse, l., awwad, y., kane. g. (2021). failures of fairness in automation require a deeper understanding of human-ml augmentation. mis quarterly, 45(3), 1483-1500. thelwall, m. (2017). gender bias in machine learning for sentiment analysis. online information review, 42(3), 343354. trewin, s., basson, s., muller, m., branham, s., treviranus, j., gruen, d., hebert, d., lyckowski, n., manser, e. (2019). considerations for ai fairness for people with disabilities. ai matters, 5(3). doi: 10.1145/3362077.3362086 united nations educational, scientific and cultural organization (unesco) (2020). artificial intelligence, and gender equality: key finding of unesco’s global dialogue. available at https://unesdoc.unesco.org/ark:/48223/pf0000374174 veale, m., binns, r. (2017). fairer machine learning in the real world: mitigating discrimination without collecting sensitive data. big data and society, 1-17. doi: https://doi.org/10.1177/2053951717743530 wang, l. (2020). the three harms of gendered technology. australasian journal of information systems, 24. doi: https://doi.org/10.3127/ajis.v24i0.2799 west, s.m., whittaker, m., crawford, k. (2019). discriminating systems: gender, race and power in ai. ai now institute. retrieved from https://ainowinstitute.org/ discriminatingsystems.html webster, j., watson, r.t. (2002). analysing the past to prepare for the future: writing a literature review, mis quarterly, 26(2), 3-23. wolfswinkel, j., furtmueller, e., wilderom, c. (2013). using grounded theory as a method for rigorous reviewing literature. european journal of information systems, 22 (1), 45-55. wu, w., huang, t., gong, k. (2019). ethical principles and governance technology development of ai in china. engineering, 6(3), 302-309. zhong, z. (2018). a tutorial on fairness in machine learning, towards data science. https://towardsdatascience.com/a-tutorial-on-fairness-in-machine-learning-3ff8ba1040 appendix table 1. articles selected for the literature review journal name title of the article author theory used type of article focus data science and engineering achieving fairness with decision trees grari et al., 2020 no theory used design science decisionmaking (decision trees) journal of behavioural robotics al & recruiting software: ethical and legal implications martinez & fernandez, 2020 no theory used conceptual and essays. human resource journal of speculative research ai becomes her: discussing gender and ai costa & ribas, 2019 gender theory conceptual and essays gendered technology https://doi.org/10.1177/2053951717743530 https://doi.org/10.3127/ajis.v24i0.2799 australasian journal of information systems nadeem, marjanovic & abedin 2022, vol 26, selected papers from acis 2020 gender bias in ai 23 journal of research and development ai fairness 360 bellamy, 2018 no theory used design science fairness toolkit ai & society algorithms and values injustice & security hayes, poel, steen, 2020 no theory used conceptual and essays societal international journal of law and information technology ai in healthcare: a critical analysis schonberger, 2019 no theory used literature review healthcare industry sector neural computing and applications assessing gender bias in machine translation prates, avelar, lamb, 2019 no theory used empirical (case study) machine learning ieee security and privacy automated decision making in airport checkpoints: kyriazanos et al., 2019 no theory used conceptual and essays. airport checkpoints george washington law review automating the risk of bias johnson, 2019 risk management theory, traditional agency theory, theory of commitment bias conceptual and essays ai software tools management science algorithmic bias? an empirical study of apparent gender-based discrimination in the display of stem career lambrecht & tucker, 2019 no theory used design science stem career data mining and knowledge discovery bias in datadriven ai systems ntoutsi et al., 2019 no theory used empirical (survey) data health equity big data analytics and the struggle for equity in healthcare: the promise & perils. ibrahim, charlson, neill, 2020 no theory used conceptual and essays. healthcare industry sector ama journal of ethics can ai help reduce disparities in general medical & medical healthcare? chen, szolovits, ghassemi, 2019 no theory used empirical (case study) healthcare industry sector journal of intelligent information systems causal inference for social discrimination reasoning. qureshi et al., 2020 no theory used design science testing of algorithms humancomputer interaction designing fair ai for managing employees in organizations: a review, critique & design agenda robert et al., 2020 organizational justice theory & adam’s equity theory literature review human resource australasian journal of information systems nadeem, marjanovic & abedin 2022, vol 26, selected papers from acis 2020 gender bias in ai 24 journal of information, communication, and ethics in society detecting racial bias in algorithm and machine learning lee, 2018 no theory used conceptual and essays. machine learning journal of business ethics ethical implications and accountability of algorithms martin, 2019 theory of algorithmic accountability, decisionmaking theory conceptual and essays. ethics & accountability of algorithms engineering ethical principles and governance technology development of ai in china wu, huang, gong, 2019 no theory used conceptual and essays. country (national use of ai) humanities and social sciences communications ethical principles in ml & ai: cases from the field and possible ways forward piano, 2020 no theory used literature review autonomous vehicles artificial intelligence and law evaluating causes of algorithmic bias in juvenile criminal recidivism miron et al., 2020 no theory used designs science criminal justice information fusion explainable ai(xai): concepts, taxonomies, opportunities & challenges towards responsible ai arrieta et al., 2020 game theory, theory guided data science conceptual and essays responsible ai business and information systems engineering fair ai challenges and opportunities feuerriegel, dolata, schwabe, 2020 no theory used conceptual and essays. fairness in ai big data and society fair machine learning in the real world: mitigating discrimination without collecting sensitive data veale & binns, 2017 no theory used conceptual and essays. fairness in ai sociological methods and research fairness in criminal justice risk assessments: the state of the art. berk et al., 2018 no theory used literature review criminal justice online information review gender bias in ml for sentiment analysis thelwall 2017 no theory used. design science ml digital medicine predictably unequal: understanding and addressing concerns that paulus & kent, 2020 no theory used design science healthcare australasian journal of information systems nadeem, marjanovic & abedin 2022, vol 26, selected papers from acis 2020 gender bias in ai 25 algorithmic clinical predictions increase health disparities digital medicine sex & gender differences and biases in ai for biomedicine & healthcare cirillo et al., 2020 no theory used empirical (survey) healthcare futures the application of ai in police interrogations: an analysis addressing the proposed effect ai has on racial and gender bias noriega, 2020 uncanny theory conceptual and essays criminal justice australasian journal of information systems the three harms of gendered technology wang, 2020 no theory used conceptual and essays decision making ieee transactions on visualization and computer graphics fair sight: visual analytics for fairness in decision making ahn & lin, 2020 no theory used empirical (case study) visual analytics cambridge journal of regions, economy, and society when machines think for us: the consequences for work and place clifton, glasmeier, gray, 2020 no theory used conceptual and essays workplaces and society table 2. characteristics of gender bias in ai grouping of concepts/ themes grouping of characteristics/concepts characteristics of gender bias in ai source description so ci et al prejudices in society societal gender prejudices martinez & fernandez 2020; johnson 2019; ntoutsi et al., 2019; thelwall, 2017; noriega, 2020; wang, 2020; clifton, glasmeier, gray 2020; prates, avelar, lamb 2019; lee, 2018; cirillo et al., 2020 pre-existing societal inequalities, such as internalized misogyny. discrimination grari et al., 2020; lambrecht & tucker, 2019 biased behaviours followed by the majority pre-existing norms of society are followed by the majority ntoutsi et al., 2019 australasian journal of information systems nadeem, marjanovic & abedin 2022, vol 26, selected papers from acis 2020 gender bias in ai 26 table 3. contributing factors of gender bias in ai grouping of concepts/ themes grouping of factors/ concepts contributing factors of gender bias in ai source description b ia se d tr ai ni ng d at as et s misrepresentation of subjects in training datasets improper data gathering practices grari et al., 2020; martinez, fernandez, 2020; hayes, poel, steen, 2020; kyriazanos et al., 2019; johnson 2019; lambrecht, tucker, 2019; ntoutsi et al., 2019; chen, szolovits, ghassemi, 2019; ibrahim, charlson, neill, 2020; qureshi et al., 2020; lee, 2018; martin, 2019; miron et al., 2020; arrieta et al., 2020; feuerriegel, dolata, schwabe, 2020; thelwall, 2017; paulus, kent, 2020; cirillo et al., 2020; noriega, 2020; ahn, lin, 2020; over and underrepresentation of certain groups in data sets can result to perpetuate discrimination. datasets may be underrepresented of the public demographics. in st itu tio na l biases across various disciplines – sociodemographic & technological biases association of femineity with certain soft skills (nontechnical) costa & ribas, 2019 socio-economic factors contribute to discrimination. socio-economic factors requiring interdisciplinary collaboration ibrahim, charlson, neill, 2020; ntoutsi et al., 2019, martin, 2019; veale & binns 2017 d es ig n & im pl em en ta tio n lack of gender diversity in data and technology lack of gender diversity in ai development and training datasets martinez & fernandez, 2020; johnson, 2019; wang, 2020; clifton, glasmeier, gray 2020 existing issues in societal bias sneak into the design and implementation of technology. ai amplifies the bias in society by producing biased outcomes ai amplifying social prejudices grari et al., 2020; hayes, poel, steen 2020; johnson, 2019 algorithms amplify those phenomena that are easily quantifiable. nascent technology creates a risk johnson 2019 prejudices influencing technology through biased data pre-existing patterns of exclusions and disparities discovered by data mining johnson, 2019; clifton, glasmeier, gray 2020 biased data seeps into the ai algorithms resulting in amplifying discrimination and inequalities in societies. disparities in society embedded in data ibrahim, charlson, neill, 2020; veale, binns 2017 australasian journal of information systems nadeem, marjanovic & abedin 2022, vol 26, selected papers from acis 2020 gender bias in ai 27 bellamy, 2018; feuerriegel, dolata, schwabe, 2020 under or overrepresentation of subjects in datasets martinez, fernandez, 2020; hayes, poel, steen, 2020; johnson, 2019; ntoutsi et al., 2019; robert et al., 2020; martin, 2019; miron et al., 2020; veale, binns, 2017; cirillo et al., 2020; clifton, glasmeier, gray, 2020 unavailability of useful data hayes, poel, steen, 2020; veale, binns, 2017; noriega, 2020 unfair training datasets language discrimination for gender in data prates, avelar, lamb, 2019; ntoutsi et al., 2019; chen, szolovits, ghassemi, 2019; qureshi et al., 2020; lee, 2018; thelwall, 2017; cirillo et al., 2020 patterns in the data are designed to discriminate. unfairness in data veale, binns, 2017. programmers/data miners’ conscious or unconscious bias data miners unintentionally parse the bias while discovering patterns of inequalities in data hayes, poel, steen, 2020; johnson, 2019; ntoutsi et al., 2019, lee, 2018; martin, 2019. programmers unintentionally incorporate bias during the input, training, and programming stage. programmers’ conscious or unconscious bias hayes, poel, steen, 2020; johnson, 2019; lambrecht, tucker, 2019; piano, 2020; noriega, 2020, wang, 2020; clifton, glasmeier, gray, 2020 proxy variables of sensitive features variables acting as a proxy sensitive features and their casual influences in data martinez, fernandez, 2020; ntoutsi et al., 2019; ibrahim, charlson, neill, 2020; robert et al., 2020; lee, 2018; martin, 2019; piano, 2020; arrieta et al., 2020, feuerriegel, dolata, schwabe, 2020; noriega, 2020; ahn, lin, 2020; bellamy, 2018; robert et al., 2020. proxy data are being used for features that are hard to quantify or to be collected. co-relational analysis of observational data qureshi et al., 2020 historical human bias in data historical bias goes to biased datasets hayes, poel, steen, 2020; kyriazanos et al., 2019; johnson, 2019; lambrecht, tucker, 2019; ibrahim, charlson, neill, 2020; martin, 2019; veale, binns, 2017; cirillo et al., 2020; ahn, lin, 2020. the majority follow the norms established by the society in the past. australasian journal of information systems nadeem, marjanovic & abedin 2022, vol 26, selected papers from acis 2020 gender bias in ai 28 g en de r s te re ot yp in g prejudices in society gender stereotyping in society prates, avelar, lamb, 2019; lee, 2018; miron et al., 2020; cirillo et al., 2020; noriega 2020; wang 2020 prevailing gender stereotyping in society. societal prejudices martinez, fernandez, 2020; ntoutsi et al., 2019; miron et al., 2020; thelwall, 2017; cirillo et al., 2020; noriega, 2020. culture fostering masculinity bro culture fostering exclusivity and masculinity johnson, 2019 gender imbalance and masculinity are pervasive in the it industry. socio-economic factors imputing discrimination economic factors martinez, fernandez, 2020; hayes, poel, steen, 2020; lambrecht, tucker, 2019; cirillo et al., 2020; wang, 2020. socio-economic factors based on social standing e.g. neighbourhood, zip code, location results in incorrect assumptions of an individual. individual sociostatus based on zip code lee, 2018; veale & binns, 2017 decisions are taken on biased norms followed by the majority decisions are made on preexisting norms of society followed by the majority ntoutsi et al., 2019 certain upstream social norms are followed blindly because of their easy acceptance in society. a i a m pl if ie s ex is tin g bi as failing to ensure humans in the loop in ai decisions reducing the role of human agents or failing to ensure “human in the loop” johnson, 2019 data mining systems reproduce the historic biases embedded in the data if there is a missing human role in the final decision making algorithms optimizing costeffectiveness in a discriminatory way lambrecht & tucker, 2019; ntoutsi et al., 2019; chen, szolovit, ghassemi, 2019; ibrahim, charlson, neill, 2020; martin, 2019. ai amplifying the bias in society the algorithm perpetuates/ amplifies discrimination and biases hayes, poel, steen, 2020; johnson, 2019, ntoutsi et al., 2019; lee, 2018; cirillo et al., 2020; clifton, glasmeier, gray, 2020; clifton, glasmeier, gray 2020 algorithms inherent rules from previously discriminatory decisions. lack of transparency in algorithms opacity in algorithms hayes, poel, steen, 2020; miron et al., 2020; ahn, lin, 2020; clifton, glasmeier, gray, 2020. algorithms are opaque, complex, unpredictable, and partially autonomous. australasian journal of information systems nadeem, marjanovic & abedin 2022, vol 26, selected papers from acis 2020 gender bias in ai 29 creator’s inherent bias in ai algorithms ai reflecting creator’s bias hayes, poel, steen, 2020; ntoutsi et al., 2019, miron et al., 2020; wang, 2020. the lack of multidisciplinary aspects in ai creators results in unfair outcomes. la ck o f g en de r d iv er si ty in a i d ev el op m en t te am s lack of gender disparity in developers and the technology sector gender disparity in ai development team clifton, glasmeier, gray, 2020; martinez, fernandez, 2020; johnson, 2019; wang, 2020 there is a lack of diversity in ai developers’ teams and in the technology sector because of which there is a lack of diversity of thought in the preparation of the data as unconscious bias seeps in the training data in the data preparation stage. lack of diversity in the stem sector in senior management and employees. johnson, 2019; lee, 2018; wang, 2020. the technology industry is remarkably maledominated and exceptionally homogeneous. johnson, 2019; lee, 2018; wang, 2020 la ck o f a i r eg ul at io ns limited regulation on data collection, selection, and modification lack of legal provision dealing with the way data is collected, selected, and modified ntoutsi et al., 2019; wang, 2020 data protection laws, and general provisions concerning data quality are deficient. limited regulations addressing gender bias limited development in regulation towards addressing gender balance. johnson, 2019. c on te xt ua l a nd o th er fa ct or s agents behind the data collection lack awareness of certain sensitive features. agents such as executives, software engineers, data scientists, developers, and policymakers commissioning and authorizing the algorithm are not aware of sensitive features hayes, poel, steen, 2020; wang, 2020. ai development teams are not aware of the importance of distinguishing between certain categories. developers may fail or ignore to specify the limits of datasets johnson 2019; ntoutsi et al., 2019. the third party that has collected the underlying dataset may aggregate data johnson, 2019. australasian journal of information systems nadeem, marjanovic & abedin 2022, vol 26, selected papers from acis 2020 gender bias in ai 30 algorithms not tested for a specific context/application/ sector dumb-start programs that are not designed or tested for a specific context qureshi et al., 2020. failing to test an algorithm with regards to the context in which it would be used. table 4. approaches for addressing gender bias in ai grouping of conncepts/ themes grouping of factors/ concepts approaches for mitigating gender bias in ai source description a i t ec hn ol og y re la te d ap pr oa ch es collection and preparation of data sets bias-aware data collection ntoutsi et al., 2019; bellamy, 2018 preventing unfairness in training data by ensuring fair data collection, data preparation, and regularizing the training data to minimize the unfairness. preparation of the fair data hayes, poel, steen, 2020; qureshi et al., 2020; arrieta et al., 2020; noriega, 2020; grari et al., 2020; kyriazanos et al., 2019; ntoutsi et al., 2019; miron et al., 2020; arrieta et al., 2020; veale, binns, 2017; berk et al., 2018; ahn, lin, 2020; bellamy, 2018. removing proxies of protected attributes from the datasets. grari et al., 2020; hayes, poel, steen, 2020; miron et al., 2020; arrieta et al., 2020; feuerriegel, dolata, schwabe, 2020; veale, binns, 2017; bellamy, 2018. in-processing of algorithms integration of algorithm grari et al., 2020; kyriazanos et al., 2019; ntoutsi et al., 2019; miron et al., 2020; arrieta et al., 2020; veale, binns, 2017; berk et al., 2018; ahn, lin, 2020; bellamy, 2018. fair algorithmic integration and resource allocation to ensure strengthening of algorithmic design. equal/unbiased resources of allocation in an algorithm grari et al., 2020; lambrecht, tucker, 2019; lee, 2018; miron et al., 2020; arrieta et al., 2020; veale, binns, 2017; ahn, lin, 2020. australasian journal of information systems nadeem, marjanovic & abedin 2022, vol 26, selected papers from acis 2020 gender bias in ai 31 designing fair classification of algorithms grari et al., 2020; ntoutsi et al., 2019; robert et al., 2020; kyriazanos et al., 2019. strengthening of formal & statistical foundations of algorithms ntoutsi et al., 2019. implementation of algorithms interpreting & testing of the algorithms grari et al., 2020; kyriazanos et al., 2019; ntoutsi et al., 2019; miron et al., 2020; arrieta et al., 2020; veale, binns, 2017; berk et al., 2018; ahn, lin, 2020; bellamy, 2018. testing the algorithm for a specific application for enhanced accountability and bias detection. ensuring algorithmic transparency, explain ability, and accountability johnson, 2019; lambrecht, tucker, 2019; martin, 2019; feuerriegel, dolata, schwabe, 2020; thelwall, 2017; clifton, glasmeier, gray, 2020; arrieta et al., 2020; ntoutsi et al., 2019; cirillo et al., 2020. f ai r a i m an ag em en t a pp ro ac he s better fairness governance policies internal governance policies johnson, 2019. enhanced ai corporate governance for gender bias mitigation internal structures and processoriented corporate governance johnson, 2019; martin, 2019 continuous education/training on fairness and ethics for all stakeholders educational workshops and training on workplace fairness noriega, 2020. workshops/educa tion that involves principles of ethics such as promoting ethical education for every stakeholder in ai research & development. certified professional required martin, 2019. awareness of ethics and promoting responsible ai wu, huang, gong, 2019; veale, binns, 2017 australasian journal of information systems nadeem, marjanovic & abedin 2022, vol 26, selected papers from acis 2020 gender bias in ai 32 awareness of unintended bias in scientific community and technology industry cirillo et al., 2020 collaborative organizational learning on fairness & demographic characteristics business models and policy should be designed concerning fair ai feuerriegel, dolata, schwabe, 2020 design of business models and policies to consider ai principles. interdisciplinary approach & understanding of ai ethical principles interdisciplinary disciplines to work collaboratively to address ethical challenges wu, huang, gong, 2019; ibrahim, charlson, neill, 2020 employment of a more diverse it workforce to be included in the design and implementation of algorithms. workplace diversity in managerial roles gender diversity at managerial levels lee, 2018 an increase in gender inclusion in the development of ai technologies will introduce diverse perspectives and diversity of thought in the ai development which is essential for breaking down the bias. diversity in the development of ai systems costa, ribas, 2019; johnson 2019; ntoutsi et al., 2019; arrieta et al., 2020; clifton, glasmeier, gray 2020 gender diversity in the high-tech industry and stem career lee, 2018; johnson, 2019; wang, 2020 designing strategies for incorporating algorithmic transparency and accountability big data review board required martin, 2019 ai audits are to be conducted periodically to ensure ai compliance. incorporate regular audits of the data martinez, fernandez, 2020; johnson, 2019; ibrahim, charlson, neill, 2020; robert et al., 2020; piano, 2020; veale, binns, 2017; noriega, 2020 designing strategies for fairness and ensuring accountability hayes, poel, steen, 2020 ensuring human in the loop integrating human & ai decision making miron et al., 2020 design strategies like providing more autonomy to the users in decision-making would bring australasian journal of information systems nadeem, marjanovic & abedin 2022, vol 26, selected papers from acis 2020 gender bias in ai 33 fairness to ai decisions. a i g ov er na nc e an d re gu la to ry a pp ro ac he s ai governance to incorporate key ethical standards ethical regulation for new technology development clifton, glasmeier, gray, 2020; wang, 2020 ethical standards by government and regulatory organizations to ensure fairer data collection and models public policy regarding fair ai clifton, glasmeier, gray, 2020 ethical technology development cirillo et al., 2020 laws and policies to adhere to ethical ai principles quality assurance for ai safety. w, huang, gong, 2019 formal verification and testing to be carried out by users for ensuring ai safety and to adhere to ai ethical values. algorithm test & trail lee, 2018; wu, huang, gong, 2019 practitioners and policymakers adhering to ethical ai principles lee, 2018 regulated policies for implicit and unconscious bias lee, 2018 ethical ai platforms comprehensive open ai platforms for all stakeholders wu, huang, gong, 2019 developing open ai platforms for designing strategies or technical inputs to be incorporated to promote autonomy. ci et al & c om m un ity fo cu se d ap pr oa ch es awareness of gender diversity logical considerations for increasing gender diversity hayes, poel, steen, 2020 social intervention in order to enhance gender diversity in social and contextual aspects would address the gender bias in ai. gender-neutral expression in communication prates, avelar, lamb, 2019 bringing cognitive diversity in workplace /institutions johnson, 2019 gender diversity related to socioeconomic aspects gender diversity related to socioeconomic aspects cirillo et al., 2020 compounding factors related to socio-economic aspects to australasian journal of information systems nadeem, marjanovic & abedin 2022, vol 26, selected papers from acis 2020 gender bias in ai 34 establish fairness in ai algorithm “ecosystem of trust” that ensures ai systems incorporate ethical standards policy intervention policies to reach beyond the social prejudices clifton, glasmeier, gray, 2020 ensuring public policy to bring trust for ai systems and societal wellbeing and fairness. better ai policies for gender bias affectee clifton, glasmeier, gray, 2020 to comply with the rules that protect fundamental rights miron et al., 2020 copyright: © 2022 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v26i0.3835 http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 foundational concepts 3 methodology 3.1 search criteria 3.2 selection of the articles 3.3 analysis of the selected articles 4 findings of the systematic literature review 4.1 characteristics and contributing factors of gender bias in ai-based decision-making systems 4.1.1 design and implementation characteristics 4.1.2 institutional characteristics 4.1.3 societal characteristics 4.2 approaches to mitigating gender bias in ai-based decision-making systems 4.2.1 ai technology-related approaches 4.2.2 fair ai management approaches 4.2.3 ai governance and regulatory approaches 4.2.4 societal and community-focused approaches 5 discussion and contributions 6 future research 7 concluding remarks and study limitations paper title (paper title style) australasian journal of information systems waizenegger & techatassanasoontorn 2022, vol 26, selected papers from acis 2020 when robots join our team 1 when robots join our team: a configuration theory of employees’ perceptions of and reactions to robotic process automation lena waizenegger auckland university of technology, new zealand lena.waizenegger@aut.ac.nz angsana a. techatassanasoontorn auckland university of technology, new zealand abstract robotic process automation (rpa) as a lightweight automation technology has witnessed an increasing uptake in the industry in recent years. despite considerable changes in employees’ tasks and processes brought about by the introduction of rpa, there is a lack of research that explores how employees react to an rpa implementation. hence, the goal of this research is to understand employees’ perceptions of and reactions to rpa as these affect their interaction with the technology and, ultimately, their adoption and use. to address this research gap, we conducted a case study at a financial institution in new zealand and interviewed 18 employees of the business units and members of the rpa team. building on a configurational approach, we developed a mid-range theory and identified four distinct configurations that show how employees’ perceived consequences of software robots on their jobs influenced their collaboration with the automation team, their attitude towards the change in work tasks and processes and ultimately their interactions with software robots and attribution of software robots’ roles and performance. our findings may inform implementation and change management strategies and accommodation initiatives to support employees’ needs to facilitate adoption, which is crucial for organisations to realise the benefits of rpa. keywords: case study, configurational approach, financial institution, robotic process automation (rpa), rpa implementation. 1 introduction in the wake of the digitalisation trend that is observed globally, robotic process automation (rpa) has gained increasing popularity as one of the least invasive, easiest, and fastest automation approaches. organisations jump on the rpa bandwagon in order to cut costs while expecting to improve the efficiency and quality of their processes (cewe, koch, & mertens, 2018; hofmann, samp, & urbach, 2020). in fact, in 2019, 49% of large companies worldwide invested in rpa, and 24% adopted rpa in their work processes. also, small and mid-sized companies show increasing interest in rpa, with 14% small-sized and 17% midsized companies having invested in rpa and 9% of small and 9 % of mid-sized companies having adopted rpa (statista, 2020). rpa has been implemented in various industries worldwide, such as telecommunications, finance and insurance, healthcare, and oil and gas, and across multiple processes such as recruitment, payroll, purchasing, healthcare claims, vendor information processing (ivančić, suša vugec, & bosilj vukšić, 2019). australasian journal of information systems waizenegger & techatassanasoontorn 2022, vol 26, selected papers from acis 2020 when robots join our team 2 rpa is an automation technology that can execute tasks operating on the user interface of other information systems in the same way a human would do (syed et al., 2020). the ieee corporate advisory group (2017) defined rpa as “a preconfigured software instance that uses business rules and predefined activity choreography to complete the autonomous execution of a combination of processes, activities, transactions, and tasks in one or more unrelated software systems to deliver a result or service with human exception management” (p. 11). in contrast to other automation techniques, rpa is often regarded as a lightweight solution where an underlying information system remains unchanged (santos, pereira, & vasconcelos, 2019; van der aalst, bichler, & heinzl, 2018). rpa is frequently used to automate rule-based, well-structured, and repetitive tasks such as extracting structured data from documents, transferring data between applications through screen scraping, accounting reconciliation, and automated email query processing (hofmann et al., 2020; syed et al., 2020). one of the drivers of rpa implementation is its potential to reduce mundane and repetitive tasks, allowing employees to work on more value-adding tasks that require social skills, problem-solving capabilities, and decision-making (institute for robotic process automation, 2015; penttinen, kasslin, & asatiani, 2018; santos et al., 2019). however, it is common and natural that some employees feel apprehensive and concerned about automation and its effects on their jobs and employment (fernandez & aman, 2018; hallikainen, bekkhus, & pan, 2018; lacity & willcocks, 2017). employees might also be more reluctant to change as they enjoy their work tasks, do not have the required skill set, or refuse to learn a new role (fernandez & aman, 2018; hallikainen et al., 2018). while rpa has received much attention in the industry due to the increasing trend of digitalisation and digital transformation, academic research is lagging and missing an opportunity to provide theoretical insights that are important to inform the development, implementation and adoption of rpa (hofmann et al., 2020; ivančić et al., 2019; syed et al., 2020). in this study, we respond to syed’s et al. (2020) call for research on the implications of rpa to better understand the changes and effects on the human workforce and explain employees’ perspectives towards rpa-enabled changes in their work environment. employee buy-in is fundamental for the adoption and use of the technology and hence the success of rpa implementation (plattfaut, 2019), especially against the backdrop that 30-50% of rpa projects fail (lamberton, gillard, & kaczmarskyj, 2016). existing research on traditional information-technology (it)-enabled organisational change has shown that employees’ perception and emotional reactions to a new it system and the consequent changes to their work influence their behavioural response ranging from acceptance, deploying workaround and resistance and avoidance behaviours (bala & venkatesh, 2015; beaudry & pinsonneault, 2005; bhattacherjee, davis, connolly, & hikmet, 2018). however, rpa is part of a new generation of workplace technologies that have a more radical and profound impact on the nature of work of human employees and their roles and responsibilities than traditional technologies such as enterprise resource planning systems (aroles, cecez-kecmanovic, dale, kingma, & mitev, 2021; baptista, stein, klein, watsonmanheim, & lee, 2020; klein & watson-manheim, 2021). hence, it is important to explore the effects of this new generation of technologies on human employees and employee perceptions of and reactions to these transformational technologies, including rpa. a better understanding of employees’ perspectives will help to inform technology and human australasian journal of information systems waizenegger & techatassanasoontorn 2022, vol 26, selected papers from acis 2020 when robots join our team 3 resources policies as well as the design of more effective change management strategies that are crucial for successful rpa implementations (kyheröinen, 2018). therefore, the goal of this research is to explore how employees make sense of rpa technology through their perceptions of and reactions to rpa and software robots. drawing on a configurational approach (meyer, tsui, & hinings, 1993), we develop distinct configurations of employees’ rpa perceptions and reactions to this technology that reflect their perceived consequences of software robots, cooperation with an automation team, attitude toward change in work practices, and view of software robots and their performance. these configurations reflect different employees’ perspectives on rpa and hence contribute to the academic discourse on the effects of the new generation of workplace technologies on the human workforce, but also may help change managers to accommodate the needs of employees and facilitate team leaders to better support their employees to maintain job satisfaction and avoid turnover. therefore, we investigate the following research question: “what are the distinct configurations of employees’ perceptions of and reactions to rpa? and how do these perceptions and reactions reflect their perspectives of rpa?” in order to address our research questions, we adopt a qualitative research approach (sarker, xiao, beaulieu, & lee, 2018) and configurational analysis to theorise employees’ perceptions and reactions to rpa implementation at a financial institution based in new zealand. 2 theoretical background the implementation of rpa leads to the full or partial automation of work that was previously executed by human employees and hence often leads to radical changes in the way if, when and how human employees complete their work associated with those processes (denagama vitharanage, bandara, syed, & toman, 2020; eikebrokk & olsen, 2020; syed et al., 2020). yet, research on employees’ perception of and reactions to rpa and, in particular, the resulting effects on their work is still nascent. hence, we first present the insights gained from the established literature on traditional it-enabled organisational change and the effects on human employees (bala & venkatesh, 2013; morris & venkatesh, 2010), which informed our research and our theorising on rpa. we then present the state of the art of rpa research and conclude with a justification for using a configurational approach to theorise distinct configurations of employees’ perceptions of and reactions to rpa. 2.1 employees’ perceptions of it-enabled organisational change and responses to it implementations implementation of new it is a major organisational change event that presents significant disruptions to employees’ work environment. there is a rich body of work that has examined employees’ perceptions and reactions to it-enabled organisational change, particularly as a consequence of enterprise systems implementation. despite nuanced differences in research findings, two key insights emerge from this stream of research. the first insight from this literature indicates that employees’ perceptions of it-enabled organisational change are primarily drawn from their perceptions of technology characteristics, perceptions of changes in work practices, and perceptions of changes in job characteristics (bala & venkatesh, 2015; morris & venkatesh, 2010). for example, bala and venkatesh (2013) found that employees’ perceptions of technology complexity and the lack of flexibility to adjust system features to australasian journal of information systems waizenegger & techatassanasoontorn 2022, vol 26, selected papers from acis 2020 when robots join our team 4 work practices influence their unfavourable perceptions of changes in work practices and significantly increase job demands and decrease job control. the second insight shows that user responses are varied and complex, depending on how they interpret and perceive consequences from a new it in their work environment (beaudry & pinsonneault, 2005; bhattacherjee et al., 2018; wanchai, díaz andrade, & techatassanasoontorn, 2019). user emotional responses as reactions to changes in their work environment from an it implementation may include excitement, fear, anxiety, frustration, enthusiasm, and happiness that lead to different behavioural responses such as experimentation with it and modification of work tasks, the use of workarounds and delaying the use of it (beaudry & pinsonneault, 2005; bhattacherjee et al., 2018; stein, newell, wagner, & galliers, 2015). some users who anticipate that an it implementation will negatively affect them personally and professionally are more likely to engage in avoidance and resistance behaviours (bala & venkatesh, 2015; lapointe & rivard, 2005). for example, boudreau and robey (2005) found that users developed workarounds to respond to the perceived inflexibility and limitations of the system so that it responded to their needs. in contrast, those employees who see new it as an opportunity often embrace the technology and enthusiastically explore the it to find, extend, and/or change features of an it to complete tasks in creative, novel and improvised ways (bala & venkatesh, 2015). while the literature on it-enabled organisational change with a focus on previous generations of technology such as enterprise systems has provided significant insights into employees’ perceptions and responses, there are conceptual differences between transformations induced by traditional technologies and the new generation of workplace technologies such as the internet of things, algorithmic data analytics tools, and rpa (klein & watson-manheim, 2021; wessel, baiyere, ologeanu-taddei, cha, & blegind jensen, 2021). this is because the new generation of digital technologies has a more transformational and profound impact on the nature of work, the role of human actors, and forms of mutual dependencies among human actors, technical artefacts, and processes (aroles et al., 2021; baptista et al., 2020; benbya, nan, tanriverdi, & yoo, 2020; klein & watson-manheim, 2021). despite the call for new ways of theorising consequences of the new class of technologies, including rpa (wessel et al., 2021), the employee’s perspective appears to be under-theorised (seiffer, gnewuch, & maedche, 2021; staaby, hansen, & grønli, 2021). our study focuses on developing an understanding of employees’ experiences with an rpa implementation. we attempt to develop a configurational theory of employee perceptions of rpa-enabled changes and responses to consequences of its implementation in their work environment that offers important groundwork to explain the why and how of employees’ rpa experiences. these perceptions and responses to perceived and actual consequences of the introduction of rpa can occur before, during and after the implementation. 2.2 effects of rpa on the human workforce the literature on the effects of rpa implementation shows a mix of positive and negative implications on human employees. based on a literature search, we identify common benefits and drawbacks for human employees and provide illustrative examples and quotes from existing studies presented in table a1 in the appendix. australasian journal of information systems waizenegger & techatassanasoontorn 2022, vol 26, selected papers from acis 2020 when robots join our team 5 some studies show that rpa implementations have led to various beneficial outcomes for human employees. many studies have demonstrated that rpa has the potential to reduce repetitive and mundane tasks and allows employees to take on more complex tasks that require critical thinking, decision-making, problem-solving and human judgment (castelluccio, 2017; penttinen et al., 2018). this shift to more value-adding tasks can lead to increased job satisfaction and work meaningfulness for employees (denagama vitharanage et al., 2020; eikebrokk & olsen, 2020; staaby et al., 2021). several case studies have also shown that employees experienced an increase in their work responsibility through rpa implementation (staaby et al., 2021) that extended their knowledge and skills, especially around automation technologies and process automation (denagama vitharanage et al., 2020; eikebrokk & olsen, 2020; lacity & willcocks, 2016a). some employees moved to new roles that require more technological skills or roles that allow a company to offer new services to their customers (eikebrokk & olsen, 2020; lacity & willcocks, 2016b). despite numerous benefits of rpa, several studies report a number of negative effects on the existing employees and potential new employees. while many organisations may not lay off their staff, they do not intend to hire new employees but choose to use software robots instead (eikebrokk & olsen, 2020). in addition, existing employees are often concerned about possible job loss as they feel threatened by software robots (fernandez & aman, 2018; hallikainen et al., 2018). this is because software robots are usually more productive, make fewer errors and cost less than human employees (aguirre & rodriguez, 2017). these concerns are not unfounded, as some studies report redundancies and downsizing after rpa implementation (eikebrokk & olsen, 2020; lacity & willcocks, 2016a). although some employees view learning new skills as an opportunity, others are worried and more resistant to adapting to these new challenges (fernandez & aman, 2018; hallikainen et al., 2018). further, while the reduction of mundane and repetitive tasks is one of the most significant benefits of rpa, some employees experience an increase in workload as they have to cope with more of the same kind of tasks or new tasks (staaby et al., 2021). also, while standardisation of work can be perceived as liberating and allowing employees to gain more time, it also affects employees' autonomy, leading to work alienation (staaby et al., 2021). our literature review has shown that those positive and negative effects on employees almost always co-exist in an rpa implementation project, as illustrated in a case study from fernandez and aman (2018). their results show that rpa implementation led to time savings made through the automation of mundane and repetitive tasks that can now be utilised by workers to take on more challenging tasks such as data analysis. also, the error rate could be drastically reduced. on the other hand, the study also pointed out that workers feared losing their jobs to software robots, and they resisted changes to their work practices that forced them to learn new skills. additionally, the reduction of employees intensified not only the competition among employees but also the competition between employees and robots. based on the conducted literature review, we identified two weaknesses in the existing body of knowledge on rpa that we will address in this study. first, despite the insights into the perceptions, reactions and effects of rpa implementation on employees, the literature on rpa lacks the theoretical foundation and synoptic analysis, which does not allow for actionable insights to improve rpa implementation experiences for the human workforce (hofmann et al., 2020; syed et al., 2020). the dearth of research literature on rpa is emphasised by syed et al. (2020). their structured literature review highlights that only 36% of the identified research australasian journal of information systems waizenegger & techatassanasoontorn 2022, vol 26, selected papers from acis 2020 when robots join our team 6 articles are peer-reviewed, confirming the predominance of grey literature in this field of study. second, while the existing literature highlights organisational benefits, best practices and provides guidelines for successful rpa implementation or rpa governance (aguirre & rodriguez, 2017; alberth & mattern, 2017; lamberton et al., 2016; plattfaut, 2019; rutaganda, bergstrom, jayashekhar, jayasinghe, & ahmed, 2017; syed et al., 2020), only a few studies take an employees’ perspective to highlight their perceptions of and reactions to rpa (waizenegger & techatassanasoontorn, 2020; eikebrokk & olsen, 2020; staaby et al., 2021). further, those studies that outline the benefits or drawbacks for human employees often draw their conclusions from participants of the rpa team, management level or rpa vendors (denagama vitharanage et al., 2020; eikebrokk & olsen, 2020; ratia, myllärniemi, & helander, 2018). therefore, little is known about actual employee perceptions of and reactions to rpa. broadly speaking, our analysis of the literature suggests that there are possible winners and losers in the human workforce from rpa implementation; however, we need to develop a more precise contour and characterisation of what winning and losing from rpa implementation from an employee’s perspective looks like. against the backdrop of the increasing adoption of rpa in organisations and the lack of empirical research from an employee's perspective on rpa (denagama vitharanage et al., 2020), the goal of this study is to develop a more nuanced understanding of how employees perceive and react to changes introduced by rpa implementation. using a configurational approach as a theoretical framework and method (hinings, 2018; miller, 2018) we aim to enhance our understanding of a) the different rpa perceptions and reactions of human employees, b) how and why employees’ perception of rpa influences the collaboration with the rpa team and ultimately their interactions with software robots, and c) how and why some employees benefit while others lose from rpa implementation. this understanding allows us to explain how employees’ perceptions and reactions affect their interactions with rpa in order to design more customised change management approaches which foster the adoption and ultimately the realisation of rpa benefits. 2.3 configurational approach as novel and advanced applications of it such as rpa have been increasingly integrated into individuals’ work practices, how individuals interact with its and their relationships with its have become increasingly complex (hofmann et al., 2020). consequently, some is research has begun to use configurational concepts to theorise the relationship between users and it. for example, ortiz de guinea and webster (2013) suggest that individuals engage in different patterns of it use in organisations depending on whether they experience problems with it or find new opportunities to interact with it. in another study, wanchai et al. (2019) identify four distinct individual adaptation patterns to enterprise systems in organisations. these patterns represent different configurations of attitudes towards the system, approach to learning how to use the system, level of interaction with the system, exploration of system features, and stance towards changing work practices. rich insights from these studies suggest that configurational concepts are useful for researchers to present a complex picture of various manifestations of user-technology relationships, which contributes to a deeper understanding of the role of it and its effects on employees in organisations. configurational theories embrace the notion that a phenomenon (i.e., employees’ perceptions of and reactions to rpa in this study) depends on a complex arrangement of multiple attributes interacting in a non-linear fashion (fiss, 2007; meyer et al., 1993). at a conceptual australasian journal of information systems waizenegger & techatassanasoontorn 2022, vol 26, selected papers from acis 2020 when robots join our team 7 level, a configuration is a “constellation of conceptually distinct characteristics that commonly occur together” (meyer et al., 1993, p. 1175). configurational theorising focuses on identifying combinations of attributes and logics that orchestrate the interactions of the various attributes and limit their variety (furnari et al., 2020; hinings, 2018; miller, 1996, 2018). underlying the configurational concept is a configurational approach that aims to identify distinct patterns composed of interdependent attributes. a configurational perspective is theoretically attractive because it enables researchers to organise complex cause-effect relationships into typologies that constitute causal relationships of various factors that make up different configurations (fiss, 2011). in contrast to other approaches (e.g., linear regression), a configurational perspective stresses that complex causality is often characterised by nonlinearity, synergistic effects, and equifinality (fiss, 2007; meyer et al., 1993; misangyi et al., 2017). nonlinearity suggests that relationships among attributes are reciprocal, and attributes “found to be causally related in one configuration may be unrelated or even inversely related in another” (meyer et al., 1993, p. 1178). synergistic effects mean that outcomes are the result of the interdependence of multiple conditions. the principle of equifinality is the idea that different configurations may lead to the same outcome. hinings (2018) suggested that there are at least three theoretical goals for configurational analysis. first, configurations are a building block of a systematic theory by capturing patterns among key attributes. much of the early configurational research follows this theoretical goal by developing conceptual typologies or empirically generated taxonomies (miller, 2018). second, configurational analysis is used as a classification tool to identify patterns of elements at different levels, such as organisations, groups, and individuals. configurational theories are a good fit to simultaneously identify similarities and differences among populations across a wide range of phenomena. third, a configurational approach can be used to understand the degree of coherence among a set of attributes. this study fits with the first two theoretical goals of configurational analysis because we aim to theorise distinct configurations of employees’ perceptions of and reactions to rpa and identify constituting attributes underlying these configurations. 3 methodology to achieve our research objective, we conducted a case study building on the philosophical underpinnings of interpretivism (levers, 2013; myers & walsham, 1998). in contrast to positivism, interpretivism assumes that knowledge is socially constructed by human actors (walsham, 1995). we chose a financial institution as these institutions are known for being early adopters of new technologies (syed et al., 2020), and engaging in process improvement to harness the economies of scale (vishnu, agochiya, & palkar, 2017). with the widespread adoption of virtual banking, banks and financial institutions find themselves in an increasingly competitive and saturated market not only with other financial institutions but also with highly innovative and efficient fintech companies. to stay competitive, banks and financial institutions need to innovate to provide the best customer experience while minimising their costs, adhering to security standards and following the regulatory and compliance requirements (rutaganda et al., 2017; vishnu et al., 2017). automation, in general, and rpa, in particular, is deemed as a suitable solution to achieve those goals. from an operational standpoint, financial institutions usually produce vast amounts of documents across their operations through a mix of legacy systems, manual processes, and emerging technologies. such practice creates various adoption, integration, and information australasian journal of information systems waizenegger & techatassanasoontorn 2022, vol 26, selected papers from acis 2020 when robots join our team 8 retrieval challenges, leading to inefficiencies, errors, and high costs due to a large amount of human labour required to collect, input, interpret and transform data (vishnu et al., 2017). rpa is regarded as a powerful and effective technology that can address these challenges and potentially transform the customer service model and internal operation processes, making the financial industry particularly interesting to study (met, kabukçu, uzunoğulları, soyalp, & dakdevir, 2020). the financial institution in our study has a frontrunner role in rpa implementation in new zealand. starting its automation journey in 2016, the organisation included rpa as part of its broader strategy to simplify and optimise business processes. since then, it has implemented 60 attended and unattended software robots using blueprism across various business areas and processes, including customer address changes, security alteration approvals, and payments and anti-money laundering management, among others. before their rpa implementation, employees had to manually perform mundane tasks such as data entry, copying and pasting data from one system to another system and cross-checking the accuracy of data. according to the head of the intelligent automation team, one important driver for rpa implementation was to automate these mundane tasks to improve employees’ work experience, which ultimately affects the customer experience. after developing a business case and receiving funding, they started with a proof of concept for the merchant onboarding process through a collaboration with a vendor in india. however, they soon discovered that the offshore model didn’t work partly because the process was built very poorly. they then turned to an onshore automation consultancy and developed five additional processes across business units in the following six months to increase the visibility of rpa and spread its benefits throughout the organisation. based on this expansion of rpa implementation, they saw positive results in saved labour hours, an increase in processing speed and positive reactions from employees. as a result, they further extended their rpa programme across other business units, i.e. organisational entities with a particular focus such as corporate banking, business banking, personal banking, insurance etc. and officially established the intelligent automation team tasked with process optimisation and automation, including rpa. 3.1 data collection we conducted 18 semi-structured interviews (wengraf, 2001) from august to december 2019. our participants consisted of members of the intelligent automation team, including the head of the unit, business analysts, risk and project managers, change managers, and process controllers, as well as employees from the business units such as team managers, bankers, operations officers, and back-office clerks, who have had direct experiences with software robots and experienced changes in their work processes. those employees were the end users of the rpa solutions and, depending on the processes they worked on interacted with a variety of different systems such as the customer relationships management system, the workflow orchestration tool that allocates tasks to the employees and bots, registry system, and various applications such as excel, email services, and smart forms. depending on their roles in the organisation, they were involved in the various back office and front office tasks such as assessing loan applications, onboarding new customers, advising on insurance packages etc. on the other hand, the members of the intelligent automation team worked with several automation tools, rpa being one of them, the systems that employees from the business units were using and a variety of different applications. depending on their roles, they consulted the business units on various automation solutions, gathered requirements, developed the australasian journal of information systems waizenegger & techatassanasoontorn 2022, vol 26, selected papers from acis 2020 when robots join our team 9 script for the software robots, tested the solutions, managed the change in the business units and were responsible for the maintenance and troubleshooting of software robots, among many other tasks. we first got introduced to the head of intelligent automation team and then used the snowball sampling approach to recruit other interviewees. the participant profiles are shown in table 1. we achieved demographic diversity by interviewing six men and 12 women from different age cohorts ranging from 25 – 55 years old and from various educational and cultural backgrounds. pseudonym role team gender interviewee a head of intelligent automation (ia) team ia team male interviewee b member of the ia team ia team male interviewee c member of the ia team ia team female interviewee d risk manager and former rpa user risk team male interviewee e member of the ia team ia team female interviewee f member of the ia team ia team female interviewee g change manager business team female interviewee h change manager ia team female interviewee i member of the ia team ia team female interviewee j member of the ia team ia team female interviewee k manager of rpa users and software robots business team male interviewee l rpa user business team male interviewee m rpa user business team female interviewee n rpa user business team female interviewee o member of the ia team ia team male interviewee p risk analyst risk team female interviewee q rpa user business team female interviewee r rpa user business team female table 1. participant profiles we followed the dramaturgical model of qualitative interviews (myers & newman, 2007). we asked the intelligent automation team members questions about a) the rpa implementation process from initiation, design, development, testing, controlled production, and production, to post-implementation for 14 of their automated processes, associated challenges and how they were overcome, b) the processes that have been automated and if and how they were redesigned as part of the rpa implementation, and c) the interaction with and reactions to rpa of the employees in the business units. furthermore, we were able to observe the work of the process controller, who explained how she manages the sixty different robots across the various processes while talking us through the different systems and applications she uses. additionally, we witnessed a breakdown of one of the software robots, how she consequently interacted with the process owner and her colleague in the rpa team to find the root cause of the issue, and how they ultimately troubleshooted the bot. being able to observe the bots “in action”, seeing the log data, and a typical dashboard that is provided to the process owner, including important key performance indicators such as saved labour hours and straightthrough processing rate allowed us to triangulate our data and better interpret the interview data. the interviews with the employees from the business units allowed us to gain further insights into a) the execution of the process before and after the rpa implementation, b) the changes in work tasks and responsibilities due to the rpa implementation, and c) their perceptions, reactions and consequent interaction with the rpa technology. hence, australasian journal of information systems waizenegger & techatassanasoontorn 2022, vol 26, selected papers from acis 2020 when robots join our team 10 conducting interviews with the automation team and the business units allowed us to get a holistic picture of how employees in the business units perceive and use software robots. however, we also remained open to interesting yet not directly related accounts of our participants and probed into those narratives during the interviews if we thought they helped us answer our research questions. the interviews took between 30 – 110 minutes and were conducted face-to-face at the organisation. the interviews were recorded, notes were taken, and the audio recordings were professionally transcribed. 3.2 data analysis configurations can be generated inductively or deductively (ketchen et al., 1997). previous configurational studies have used various methods, including inductive analysis using qualitative data and quantitative techniques such as cluster analysis, principal components analysis, and, more recently, qualitative comparative analysis (qca) (fiss, 2007; hinings, 2018; miller, 2018; misangyi et al., 2017). techniques such as cluster analysis are theory agnostic, while a new method such as qca has clear strengths in theory testing. however, some researchers argue that qca studies emphasise contrasts among configurations without paying enough attention to the nature of configurations and their underlying themes (hinings, 2018; miller, 2018). since our interest is to develop internally consistent configurations, we chose to derive the configurations on employees’ perceptions of and reactions to rpa inductively. qualitative data allows us to characterise derived configurations with the richness required to fully interpret them (miller, 2018). we followed furnari et al. (2020) configurational theorising process that involves three iterative stages: scoping, linking, and naming. furnari and colleagues noted that the configurational theorising process is not a prescriptive set of analytical steps. but instead, it was designed as sets of heuristics aimed at inspiring “mental stimulation, thinking processes, and verbal articulation in the development of configurational theories” (p. 12). the scoping stage aims at surfacing attributes that explain employees’ rpa perceptions and reactions while embracing the tensions between complexity and simplicity. as a theoretical starting point, we drew insights on plausible employees’ perceptions and reactions from the it implementation and rpa literature discussed earlier for our scoping process. taken together, previous studies in the it implementation literature found that employees had diverse attitudes and reactions to it, disparate interactions with it, and adaptation, particularly for those it applications that had a significant impact on how they performed their work. the rpa literature hints that there are at least two broad groups of employees; those who are enthusiastic and embrace the benefits of software robots (denagama vitharanage et al., 2020; eikebrokk & olsen, 2020; staaby et al., 2021) and those who have a resistance stance and feel threatened by software robots (fernandez & aman, 2018; hallikainen et al., 2018; lacity & willcocks, 2016a). based on this understanding, we conducted a thematic analysis according to the process proposed by braun and clark (2006). we first familiarised ourselves with the data again by reading the transcripts and listening to the audio files. sensitised by the plausible attributes from the literature, we then generated initial codes without any preconceived theoretical lens in mind, which led the patterns emerge from the data in an inductive manner (thomas, 2003). after this initial coding step, we grouped the initial codes into themes, reviewed the themes, and named them in several iterative cycles in order to achieve a shared understanding of the emerging australasian journal of information systems waizenegger & techatassanasoontorn 2022, vol 26, selected papers from acis 2020 when robots join our team 11 patterns in the theorising process. during the coding process, both authors wrote memos and summarised the key insights, which helped us determine the overarching themes. for example, initial codes of “robot makes mistakes” and “robot does not perform accurately” were merged into a higher-level code of “do not trust software robots’ work”, which was further refined to “employees do not trust software robots’ work and question their reliability” and was grouped alongside other themes to the overarching theme “evaluation of robot performance”. this approach allowed us to simultaneously subsume the complexity of employees’ rpa perceptions and reactions and retain theoretical parsimony. we identified six co-occurring attributes that contribute to employees’ perceptions of and reactions to rpa. these are perceived consequences of software robots on their jobs, cooperation with the automation team, attitude towards changes in work processes and practices, view of software robots’ role in the work process, level and nature of interactions with software robots, and evaluation of robot performance. to provide insights into our coding process, we illustrate the chain of evidence, including an explanation of the attribute, the initial codes and example quotes for two attributes, namely perceived consequences of software robots on their jobs and cooperation with the automation team in table 2. to give insights into our coding logic, we numbered the initial codes and added them in square brackets after the respective interviewee statements to showcase how the original voices were codified. attribute explanation initial codes data excerpts perceived consequences of software robots on their jobs perception on how software robots will affect employees’ job characteristics including their workload, work performance and job security 1. availability of other tasks mitigate concerns around rpa taking over jobs 2. concerns about robots taking jobs 3. human employee does not feel threatened by robots 4. negative emotions towards software robots 5. negative perceptions on the implementation of software robots 6. positive attitudes of human employees towards software robots 7. positive emotions towards software robots 8. software robots save time for human employees i found that for example, the older employees were much more worried about being replaced by the robots [2, 4, 5], whereas the newer generation employees understood that it just meant that there would be more time for the humans to deal with the actual investigation side of our role, rather than doing the menial tasks [1, 3, 6, 7, 8]. (interviewee l, rpa user) so i would say there are employees who like to do repetitive work, since they are now experts in that. so some would say: oh so what will i do if i don't do this work, if the robot takes over this? [2, 5] so we tell them you're a person. i mean there has to be people engagement as well, so you will do more productive work, get the relationship built within whatever customer's they are serving. so that's where more time could be utilised [1, 8], because customers, they want things quick and easy nowadays, but they also want that human interaction as well so that they know there's a face behind whatever's happening to their request. (interviewee c, member of the ia team) cooperation with the automation team degree of openness to proactively or reactively collaborate with the automation 9. buy in from the business is crucial for successful rpa implementation 10. close collaboration between business teams and the they're involved. they're helping us understand the process, the solutioning, seeing it getting built, getting questions regarding oh would you do it this way or that way, you know? this system allows you to search for customers on this screen or this screen, what should you do? and they suddenly feel quite involved, so yeah, australasian journal of information systems waizenegger & techatassanasoontorn 2022, vol 26, selected papers from acis 2020 when robots join our team 12 team through sharing information about existing processes intelligent automation team 11. crucial role of subject matter experts 12. importance of team mentality between the business and intelligent automation team 13. lacking information provided by the business unit 14. requires involvement and ownership mindset from business teams 15. rpa success depends on team culture and mindset 16. intelligent automation team relies on information from subject matter experts very, very positive [9, 10, 11, 12] and then we get them involved on the sort of the dare i say ramp up as well, so as they see the volume coming through, see their time getting saved, they love it [9, 14]. (interviewee a; head of ia team) like you need to know your people well in order to know your process, because they are the doers of the process... [10, 11, 12] so if they don't do it correctly and they give you a wrong solution we'll be doomed, because obviously the robot will make a mess.[13, 16]. we rely on them to get that information, and another point is that to overcome that challenge we have to ensure that there is thorough communication with them [12, 15, 16]. (interviewee c, member of the ia team) table 2. illustrative coding table in the linking stage, the aim is to theorise why and how the attributes specified in the scoping stage connect to each other. in other words, linking is about the orchestration logic that explains how different attributes of a configuration relate to one another. based on our analysis and evidence from the data, we observed that employees’ perceptions, attitudes, and views of software robots mutually influence their cooperation stance with the automation team and subsequently shape their interactions with software robots and assessment of software robot performance. finally, in the naming stage, the aim is to articulate “an overarching narrative that meaningfully communicates complex patterns that constitute each theorized configuration and the configurational theory as a whole” (furnari et al., 2020, pp. 25-26). our configurational theory of employees’ perceptions of and reactions to rpa is framed around the interplay of employees’ attitude towards both changes in work practices and software robots, cooperation stance with the automation team, role and performance of software robots in their work processes, and interactions with software robots. an overarching narrative of the process of employee responses to changes in work practices and the need to collaborate with software robots captures the different configurations to explain how different rpa perceptions and reactions to this automation technology develop throughout the implementation process. the four emerging configurations consisting of six interrelated attributes are introduced in the findings section. 4 findings based on our analysis, we identified four distinct configurations of employees’ perceptions of and reactions to rpa. we name these four configurations as ‘software robots as a burden and threat,’ ‘software robots as tools,’ ‘software robots as teammates,’ and ‘software robots as innovative enablers.’ these names reflect the disparate perspectives that employees attribute to software australasian journal of information systems waizenegger & techatassanasoontorn 2022, vol 26, selected papers from acis 2020 when robots join our team 13 robots and rpa. table 3 presents a summary of these configurations and a description of their attributes. next, we present each configuration and its constitutive attributes, along with evidence from the interview data. 4.1 software robots as a burden and threat the ‘software robots as a burden and threat’ configuration largely describes a somewhat negative stance towards software robots by some employees due to concerns over their job security and negative reactions to additional responsibility on their work and software robots’ performance. in particular, some employees are concerned that software robots will replace their jobs, which manifests in their resistance to cooperate with the automation team throughout the implementation process: “there would be a lot of resistance there, especially the fear of the robots, all of those movies that we've seen where the robots take over... so they're like i'm going to lose my job over this, you know? it's always that fear.” (interviewee j, member of the ia team). an employee in the operations team underlines this point explaining that “the older employees were much more worried about being replaced by the robots” and “whereas people who perhaps don't really understand computers, i think they would be warier of the change because you know, they think along the lines of like well what am i going to do now” (interviewee l, rpa user) consequently, these employees are unwilling to share information about their work tasks during the requirement gathering phase or intentionally leave out information about how they perform their work tasks to maintain their edge over software robots: “it was interesting also in that when we went back to the business, and it's like, oh well we can do it quicker. we can submit it quicker because we have shortcuts... and it's like shortcuts? well why didn't you tell us about these? you know, suddenly there was more information coming up.” (interviewee h, change manager) after the introduction of software robots, these employees tend to reject the changes introduced to their work processes: “once you bring in automation, that’s when they are like, i don’t understand this. i don’t like this. i’m going to go back to doing it myself.” (interviewee m, rpa user) in addition, they regard software robots as a burden because the introduction of software robots creates more work or responsibility for them, as explained by one participant whose work tasks are located after a task that is now performed by software robots in the workflow: “it's good that we've got the system working, but more responsibility lies on us if we don't actually check the memo and something goes wrong, i get the blame too for not rechecking everything. so more responsibility means that we have to go and check each and everything.” (interviewee n, rpa user) perhaps, not surprisingly, these employees do not want to use software robots or, in some cases, only use them when they are told to do so: “like we've got data that says these people aren't using that one even though we've made all the benefits and the changes, and every month it's the same users that don't use it and they just refuse to use it.” (interviewee b, member of the ia team) when it comes to their evaluation of software robot performance, these employees maintain their distrust in software robots’ work and often question their reliability: “there was definitely a little bit of, is the robot calculating it correctly? i'm concerned, you know?” (interviewee g, change manager). 4.2 software robots as tools the ‘software robots as tools’ configuration mainly describes a yielding stance towards software robots by some employees while maintaining some scepticism over changes to their australasian journal of information systems waizenegger & techatassanasoontorn 2022, vol 26, selected papers from acis 2020 when robots join our team 14 work and software robot performance. at the beginning of the implementation process, these employees see some potential benefits of software robots and anticipate that software robots will help them save time and reduce mundane tasks: “so i think it was an interesting mix between excitement and fear from some of the workers. like some of them thought oh well this is what i do, what am i going to do now? whereas others were thinking along the lines of well at least now i don't have to do this really boring thing, because some of those tasks were really, really repetitive.” (interviewee l, rpa user) however, due to potential changes to their work tasks, they are reluctant to fully cooperate with the automation team during the automation process: “that’s part of the agreement that you have a regular kind of meetings that you can voice any kind of frustrations. but sometimes what can happen is that people don't feel comfortable for whatever reason in kind of voicing that and they'll sit on it. and then when you find out that it's a problem, it's grown to a bigger problem.” (interviewee b, member of the ia team) over time after seeing some initial benefits, these employees hesitantly accept the changes in their work processes without fully trusting software robots after their introduction: “the approval officer double-checks the information that the robots put in. they do not fully trust the robots because robots do not get it right every time. it’s assisting us to some degree… but what i’m saying is we can’t rely on the robot for any overdraft account that he’s giving us the full picture.” (interviewee k, manager of rpa users and software robots) once they start working with software robots, these employees take a pragmatic stance and view software robots as additional resources to partially support their work as explained by a manager whose team members have been working with software robots: “the robot is assisting us in our process of making a decision. so when i talk about robotics, i refer to it as assisted automation, because it's automation that's assisting us to do our job.” (interviewee k, manager of rpa users and software robots) eventually, these employees accept software robots as a solution or a new tool in their work process: “the robots are live, they're bau [business as usual]. people have accepted it as a solution. we have a usage of at around 60 per cent... the total volume of [process a] that we do [here], 60 per cent of that gets done by robot.” (interviewee o, member of the ia team) but they remain vigilant of software robot performance and use various key performance indicators (kpis) as evaluation metrics such as the number of exceptions or errors that software robots make: “this is why i refer to it as assisted automation. it's assisting us to some degree. every week i would say at least my staff will escalate at least two to three to me to go back to that team, to say why did the robot do this? why did the robot do that? robot didn't put this in, robot didn't put that in.” (interviewee k, manager of rpa users and software robots) 4.3 software robots as teammates the ‘software robots as teammates’ configuration mostly describes an eager stance by some employees in relation to how software robots as their new team members can support their work. at the beginning of the implementation process, these employees enthusiastically expect that software robots will help them reduce their workload: “we had so much work on, we knew, yeah, we've got other work to do. we knew it wouldn't take our jobs... ...that it would help us.” (interviewee q, rpa user) as a result, these employees work closely in a collaborative fashion with the automation team to continuously improve their teammates and often give the bots a human name like roby in australasian journal of information systems waizenegger & techatassanasoontorn 2022, vol 26, selected papers from acis 2020 when robots join our team 15 to emphasise that they are a true team member: “i'm always like guys, roby's feeling sick. please just be mindful, i've contacted roby's dad and... i just say hi guys, roby's broken down. i think he needs some medicine and been overworked. can you please assist?” (interviewee m, rpa user) in general, these employees willingly adapt to new tasks and responsibilities after the introduction of software robots as explained by one employee who works with rpa at one of the branches of the financial institution: “we moved the dzt [pseudonym] reporting over to us. we have moved the cx drafts [pseudonym] over to us as well, so yeah, we were able to take on more work [from] neighbouring teams, so sister teams. so yeah, to summarise the things that it was more like, like i said, it was more of giving the time back to the users” (interviewee p, rpa user). in general, the implementation of rpa triggered a reallocation of work, so that mundane and repetitive tasks were outsourced to the software robots and some teams could now focus on more customer-focused tasks or got tasks allocated from other teams as their time got freed up by leveraging the software robot. table 3. configurations of employees’ perceptions of and reactions to rpa attributes software robots as a burden and threat software robots as tools software robots as teammates software robots as innovative enablers perceived consequences of software robots on their jobs employees are concerned that the introduction of software robots will lead to uncertainty about their jobs and job loss. employees anticipate that software robots will help them save time and reduce mundane tasks. employees expect that software robots will reduce their workload. employees trust that software robots will help improve work performance. cooperation with the automation team employees are not willing to share information about their work tasks. employees are reluctant to share information about their work tasks. employees closely collaborate with the automation team to improve or fix their “teammate”. employees proactively suggest how best to incorporate software robots into work processes. attitude towards changes in work processes and practices employees reject changes to their work processes. employees reluctantly accept changes in their work processes. employees adapt to new tasks and responsibilities. employees enthusiastically take on new roles with more responsibility. view of software robots’ role in the work process software robots are regarded as a burden because they create more work or responsibility. software robots are regarded as additional resources to partially support work. software robots are regarded as super users to help manage workload. software robots are regarded as enablers to improve work quality. level and nature of interactions with software robots employees do not use software robots or only use them when they are told to. employees accept software robots as a solution. employees consider software robots as members of their team. employees embrace software robots and proactively seek out ways to enhance software robot use. evaluation of robot performance employees do not trust software robots’ work and question their reliability. employees focus on kpis to evaluate software robots’ performance. employees attribute software robots’ performance in a similar fashion as they would to a human colleague. employees view software robots as highly compliant and highperforming to support their tasks. australasian journal of information systems waizenegger & techatassanasoontorn 2022, vol 26, selected papers from acis 2020 when robots join our team 16 in addition, they view software robots positively as super users to help them better manage their workload: “our roby memos out faster to the approvals team and i can continue working, whereas before i'd be stuck on it, finishing that off and then i'd it wasn't until i completed that, then i was able to move on with the rest of the requests in my inbox. whereas now quickly fill out roby, send it off. okay, move on to my next request. oh yeah, roby's come back, forward it, send to approvals, wait for them to send it back.” (interviewee m, rpa user) many employees who see software robots as a teammate have a deep appreciation for the technology: “very appreciated, yeah. you have no idea how we love casper.” (interviewee q, rpa user) this mainly results from the robot taking over processes that the employees used to hate, which now positively affects their job satisfaction: “i kind of feel like casper's taken away that chunky thing they introduced” (interviewee r, rpa user) immediately followed by her colleague “i enjoy my job a lot more. i used to dread doing [process s] 'cause i hated doing [process s]” (interviewee q, rpa user). they also consider software robots as members of their team, similar to the way they think of their human colleagues with whom they would for example, coordinate the time when tasks should get completed: “we need roby to wake up earlier, 'cause we've got some staff that start at six in the morning. so i've got one approval officer that comes in at six o'clock, so if she wanted to send a smart form to roby but roby only wakes up at eight, she's going to sit for two hours before she can do anything.” (interviewee k, manager of rpa users and software robots). some employees even describe the robot as “charles [robot name], yes definitely part of the family” (interviewee r, rpa user) and even send christmas cards on the robot’s behalf (interviewee q, rpa user). likewise, they describe software robots’ performance similar to the way they talk about their human teammates’ performance. that is, software robots may have some good days and bad days: “roby's on fire today... ...'cause they're [memos] coming back within five minutes! yeah. i know one of the other teams calls it roby… and they're like oh roby's having a few troubles this morning.” (interviewee d, risk manager and former rpa user) 4.4 software robots as innovative enablers the ‘software robots as innovative enablers’ configuration typically describes a forwardlooking perspective taken by some employees in relation to the role of software robots, their innovativeness, and benefits to enhance work performance. at the beginning of the implementation process, these employees overwhelmingly believe that software robots will help improve their work performance: “the other one, like when we explained here's the solution, this is what we're going to do, the room like there was a standing ovation in the room, 'cause people were so happy that...they'd get their lives back.” (interviewee b, member of the ia team) therefore, these employees proactively collaborate with the automation team and suggest how best to incorporate software robots into work processes: “he was very, very good and maybe that is part of his attitude or his competence in his existing role. he knew the existing process very well as well, that he he didn't challenge us, but he worked with us to say okay, well what about this, what about this and what about this, what about this.” (interviewee b, member of the ia team) with regards to changes to their work, these employees enthusiastically take on new roles with more responsibility as described by the risk manager: “but i would say majority, like 95 per cent of people embraced their new role and just went with it and they were very, very successful at it too.” (interviewee d, risk manager and former rpa user) unlike those in the previous configuration who put an emphasis on how software robots can help them with work volume, these employees view robots as enablers to help them improve australasian journal of information systems waizenegger & techatassanasoontorn 2022, vol 26, selected papers from acis 2020 when robots join our team 17 work quality: “so if you think of an operations team at the bank, the highest risk an operations team would have is processing error; it's basically we use lots of people, people make mistakes. so one thing that i've seen from that point of view is there's a real like when robotics is built correctly, there's a really, really reduced amount of risk in processing error.” (interviewee d, risk manager and former rpa user) therefore, these employees embrace software robots and proactively seek out ways to employ more of them or expand their use, if possible: “they're like okay, can we have one more robot please? because they know that robots are there to actually help assist them.” (interviewee c, member of the ia team) overall, they appreciate software robots as high-performing partners to support their tasks and allow them to do more meaningful work: “so in this particular instance and the team were delighted with this process right, because they don't want to sit around doing these transactions anyways... and they also don't want to sit around fixing up mistakes. so having such a repetitive manual task taken away from you they thought was really cool, 'cause then they can get on and do much more human add value work.” (interviewee e, member of the ia team) 5 discussion the literature on it implementation highlights that the consequences of an introduction of a new it in a work environment are often perceived in a variety of ways by employees, thus triggering a wide range of reactions and responses among them (beaudry & pinsonneault, 2005; bhattacherjee et al., 2018; morris & venkatesh, 2010; wanchai et al., 2019). similarly, automation efforts through rpa implementations can transform employees’ work environment with deep changes to the nature of work leading to the emergence of new forms of work organising around interdependencies between human actors and digital technologies (baptista et al., 2020; denagama vitharanage et al., 2020; eikebrokk & olsen, 2020; klein & watson-manheim, 2021; staaby et al., 2021). existing rpa studies often lack a theoretical foundation and synoptic analysis (hofmann et al., 2020; syed et al., 2020) and miss the opportunity to explain the employee’s perspectives on rpa-enabled changes in their work environment. to address this void, we conducted a fine-grained analysis that recognised that employees’ perceptions of and reactions to rpa reflected complex interdependencies among multiple explanatory factors and addressed the research question: “what are the distinct configurations of employees’ perceptions of and reactions to rpa? and how do these perceptions and reactions reflect their perspectives of rpa?” with the development of our four distinct configurations, we contribute new insights into the consequences of rpa implementation in organisations by providing a more nuanced picture that shows diverse perceptions of and reactions to rpa as configurations of intertwined attributes that together form distinct perspectives of rpa. these disparate configurations, along with their constitutive attributes, allow us to better explain how employees perceive rpa, how this perception affects the collaboration with the rpa team, why and how employees interact with the technology the way they do, and how their perceptions, attitude and behaviour give rise to distinct views of rpa and software robots. based on our analysis, it becomes clear that the initially perceived consequences of software robots affect the extent of employees’ cooperation with the automation team and whether they accept the changes to their work processes and practices. for example, employees who perceive software robots as a burden and threat are concerned about the changes rpa will australasian journal of information systems waizenegger & techatassanasoontorn 2022, vol 26, selected papers from acis 2020 when robots join our team 18 have on their work practices and their job security, which is in line with the findings of fernandez and aman (2018) and hallikainen et al. (2018). these concerns have repercussions on the way they cooperate with the automation team. in particular, they often hold back important information, which prevents the automation team from implementing the most effective solution. consequently, the robots often take longer than the employees to complete a task, or their executions generate too many exceptions. this unsatisfactory software robot performance provides these employees who perceive robots as a burden and threat a compelling reason not to use them and reject the changes to their work processes. this behaviour is similar to avoidance and inertia reactions commonly observed in previous it implementation studies (bala & venkatesh, 2015; boudreau & robey, 2005), where employees avoid using a system or only use it if they are forced to do so. however, while employees often have to learn how the redesigned processes work and how to use the new it system to complete their tasks in the case of a previous generation of workplace technologies (boudreau & robey, 2005; yin yeh & ouyang, 2010), some employees in rpa projects do not actually need to use software robots to complete their tasks, but their tasks get outsourced to the software robots. therefore, instead of learning to complete their tasks with a new system, employees in rpa projects often need to learn entirely new tasks, for example, how to send input to the software robots or how to check robots’ work and performance. in other cases, some employees are required to shift to new and unfamiliar roles or face redundancies (asatiani & penttinen, 2016; hallikainen et al., 2018; lacity & willcocks, 2016a). these various effects explain why employees who see robots as a burden and threat engage in behaviours similar to avoidance and inertia. they are unwilling to cooperate with the rpa team and do not want to use software robots. on the contrary, employees who perceive robots as innovative enablers are enthusiastic about robots as they are convinced that robots will improve their work performance and lead to the various positive effects that are often highlighted in the literature, such as reduced error rate, reduction of mundane and repetitive tasks, increased speed and productivity improvements (aguirre & rodriguez, 2017; denagama vitharanage et al., 2020; staaby et al., 2021). this enthusiasm for the technology explains their proactive collaboration with the automation team and how they embrace new work tasks and responsibilities. this proactive behaviour is in line with the exploration to innovate approach that some employees take, as reported in previous it implementation studies (bala & venkatesh, 2015). when employees follow this approach, they usually explore it to find new features and ways to help them accomplish their work processes and tasks in innovative ways. in our study, employees who saw software robots as innovative enablers often eagerly learned how the robots worked and suggested further tasks and processes where rpa could be implemented in the future. these employees usually flourished and often took on more responsibilities, moved to different roles where they could apply their newly acquired technological skills and advanced their careers. this is in line with previous it implementation studies, where those users who explore it features and engage in innovative use experience an increase in job satisfaction and job performance (bala & venkatesh, 2015; hsu, yen, & chung, 2015). we observe that employees who see robots as teammates anthropomorphise the technology and accept them as members of their team. in line with seeber et al. (2020) research on machines as teammates, our participants do not simply associate a supporting role with the bots, but expect active engagement in the form of taking (independent) action to complete the work. anthropomorphism is described as the tendency of humans to associate human-like australasian journal of information systems waizenegger & techatassanasoontorn 2022, vol 26, selected papers from acis 2020 when robots join our team 19 characteristics, properties or mental states with non-human artefacts such as it systems (epley, waytz, & cacioppo, 2007). prior research has found that technologies with anthropomorphic cues foster users’ trust in the technology and increase the likelihood of adoption (qiu & benbasat, 2005). our findings signal a positive relationship between anthropomorphism and an appreciation for the software robot materialised in employees’ view of software robots as true team members who take over mundane and often despised tasks. the allocation of tasks to the bots and human employees depending on their skill set alludes to an effective team design (seeber, bittner, et al., 2020) which is in turn conducive to successful collaboration between humans and machines. since employees who perceive robots as their teammates expect the robots to reduce their workload as they often work faster and can process tasks around the clock (seeber et al., 2020), they are likely to closely collaborate with the automation team. when talking about the interactions with robots or their performance, these employees often use analogical descriptions commonly ascribed to human colleagues, such as robots being sick. while the effect of anthropomorphism is well-researched in the field of ai in general and the field of conversational agents in particular (schuetzler, grimes, & scott giboney, 2020; seeber et al., 2020; sheehan, jin, & gottlieb, 2020), there is a lack of research that theorises anthropomorphism in the area of rpa. similar to the positive impact of anthropomorphism on the adoption of personal intelligent agents (moussawi, koufaris, & benbunan-fich, 2020), we found that if employees anthropomorphise software robots, they are more likely to have a favourable attitude towards software robots and accept them as true digital colleagues in their teams. employees who perceive robots as tools that automate their manual tasks is similar to the way the association of chartered certified accountants (2015) describes the technology. in contrast to the other configurations where employees display rather uniform responses (either positive in the case of the “software robot as teammates” and “software robots as innovative enablers” configurations or negative in the case of the “software robots as a burden and threat” configuration), employees who perceive robots as tools experience ambivalent affective responses. emotional ambivalence captures mixed emotions in response to a target, in our case, rpa and refers to an experience that involves positive and negative emotions (fong, 2006; pratt & doucet, 2000; stein et al., 2015). to be more precise, our participants respond positively to some cues of the new rpa solution but negatively to others. for example, these employees anticipate that robots will help them save time, which they can use for other tasks. however, they share similar concerns, like those who perceive robots as a burden and threat regarding their future employment and drastic changes in their job. these concerns have implications for their collaboration with the automation team. they only reluctantly share information about their tasks and only accept changes to their work tasks after initial hesitations. they eventually accept software robots as an additional resource to support their work. in line with their perception of software robots as tools, they evaluate robot performance according to common key performance indicators such as the number of exceptions (syed et al., 2020), processing time and the number of cases processed (aguirre & rodriguez, 2017). employees who perceive software robots as tools are cautiously optimistic about the technology and go through a transition process over the course of the rpa implementation project. to some extent, these employees’ behaviours are similar to those described in bala and venkatesh (2015) where they want to maintain their old work processes and find workarounds to circumvent rpa; however, after realising rpa’s benefits, they slowly accept the changes to their work processes and open up to engage with rpa where they accept software robots as australasian journal of information systems waizenegger & techatassanasoontorn 2022, vol 26, selected papers from acis 2020 when robots join our team 20 additional resources and a solution. this is in line with stein et al. (2015) research, which found that users with ambivalent affective responses do not adopt one clear adaptation strategy to interact with the new it system but display various adaptation behaviours. 5.1 theoretical contributions this study contributes to theory in three ways. first, we developed configurations of employees’ perceptions of and reactions to rpa using a configurational approach (furnari et al., 2020; meyer et al., 1993) and thereby highlighting different perspectives that employees have towards rpa and software robots. by doing so, we show how the complex interdependencies among multiple attributes across the implementation process orchestrate employees’ perspectives of software robots. in particular, we show how their perceived consequences of software robots on their jobs influenced the collaboration with the automation team, their attitude towards the change in work tasks and processes and ultimately their interactions with software robots and attribution of software robots’ roles and performance. by highlighting this chain of interdependent attributes that interact and operate together, in contrast to treating the attributes as individual factors working in isolation, we provide exploratory insights to explain complex and nuanced employee’s perspectives towards rpaenabled changes in their work environment. second, we build on stein et al. (2015)’s research on ambivalent affective responses toward new it implementations. in line with their findings that ambivalent affective responses can lead to various adaptation strategies, we show that employees who perceive robots as tools display a mix of positive and negative emotions, eliciting a variety of responses to rpa. for example, while they are hesitant to share information with the automation team and only reluctantly accept changes in their work processes, they anticipate that the bots will save them time and reduce their work tasks and ultimately accept them as a solution. the insight that employees can experience ambivalent affective responses (fong, 2006; pratt & doucet, 2000) in response to rpa implementations extends existing research on employees’ perceptions of and reactions to rpa, which predominantly highlights uniform emotional responses which are either positive or negative (denagama vitharanage et al., 2020; eikebrokk & olsen, 2020; fernandez & aman, 2018; hallikainen et al., 2018; syed et al., 2020). however, our research has shown that individuals can experience mixed emotions in response to different cues of rpa, which triggers a variety of different responses. further research is needed to explore why some employees experience uniformly positive or negative emotions while others exhibit ambivalent emotions, how they react to different cues of rpa, and how those affective responses impact rpa adoption and (continued) use as the latter is crucial to reap the benefits of the technology. third, our approach to identifying coherent configurations of employees’ perceptions of and reactions to rpa is an important step in developing rigorous insights in this emerging research area on automation and work (baptista et al., 2020; benbya et al., 2020; klein & watsonmanheim, 2021; wessel et al., 2021). as argued by other scholars, a theory-based classification afforded by a configurational approach is central to theorising for all organisational phenomena (miller, 2018). by following the theorising process from furnari et al. (2020), we manage to achieve theoretical parsimony of six intertwined attributes that contribute to four distinct views of software robots. our configuration theory suggests that employees’ encompassing view of rpa can be better understood via the identification of the distinct, internally consistent set of elements derived from employees’ perceptions, attitudes, and australasian journal of information systems waizenegger & techatassanasoontorn 2022, vol 26, selected papers from acis 2020 when robots join our team 21 behaviours. these configurations can be used as a building block for future research to systematically theorise and extend our understanding of employees’ perceptions of and reactions to rpa as the process of transformative change unfolds (baptista et al., 2020). fourth, we show that anthropomorphism is an important explanatory factor in employees’ perceptions of and reactions to rpa. this is particularly true for the “software robots as teammates” configuration where employees partially anthropomorphised the robots. partial anthropomorphism implies that these employees attribute some human traits to the robots and interpret them using a human schemata without perceiving the robots fully as literally human (aggarwal & mcgill, 2007; guthrie, 1993). we theorised the ripple effect of anthropomorphism on employees’ collaboration with the rpa team, their interactions with the robots and ultimately, the inclusion of software robots in their team. while research has shown that anthropomorphism has a positive effect on user experience in the field of ai in general and conversational agents in particular (moussawi et al., 2020; sheehan et al., 2020), the effects of anthropomorphism in the context of rpa have not been explored and are therefore not well understood. we, therefore, encourage further research in this area of anthropomorphism and employees’ experiences of rpa and the social dynamics between software robots and human employees. we also call for future research to investigate drivers of anthropomorphism in the context of rpa (epley et al., 2007). 5.2 managerial implications besides our theoretical contributions, our findings may inform rpa implementation and change management strategies. knowing about the different configurations of employees’ perceptions of and reactions to rpa allows change managers and line managers to better respond to the needs and concerns of employees, especially those who see software robots as a burden and threat or those who see them as tools. in particular, organisations should proactively address the myths around rpa and explain the consequences of rpa on employees’ work and how the technology either allows them to do their work faster and better or to discuss possible avenues for re-and up-skilling to transition some employees into jobs that require more decision-making, human judgement and empathy (institute for robotic process automation, 2015; santos et al., 2019). bringing employees on the rpa journey early on allows organisations to establish better collaboration between employees and the automation team, which ultimately leads to a smoother implementation process as well as more accurate processing of the robots and a higher straight-through processing rate, which reduces the number of exceptions and therefore the workload of employees. the insights from our findings also allow line managers to better support their employees on the job as they are aware of how employees perceive robots, the effects on their work processes and how they evaluate robots’ performance, which determines the extent to which they interact with the robots. supporting employees on their needs, especially those who see robots as a burden and threat and those who perceive them as tools, can lead to increased interactions with the robots and subsequently the realisation of the benefits of rpa. 6 conclusion this study develops four distinct configurations as constellations of six interrelated attributes of employees’ perceptions of and reactions to rpa. with our mid-range theory of employees’ perceptions of and reactions to rpa, we contribute to the body of knowledge on it-enabled organisational change in general and rpa-enabled organisational transformation in particular australasian journal of information systems waizenegger & techatassanasoontorn 2022, vol 26, selected papers from acis 2020 when robots join our team 22 with the focus on the effects on the human workforce based on the narratives of employee perceptions and reactions to an rpa implementation. our research has two limitations that need to be acknowledged. first, when developing the configurations, we had to rely on the perspectives of the automation team members and not only on the opinions of rpa users. this, however, allowed us to get a holistic perspective and access to data logs that triangulate if the employees use the robots or not. second, in two interviews with three employees from the business units, a member of the automation team was present, which might have biased their responses. however, despite the presence of the member of the automation team, the interviewees also talked about their negative perceptions and reactions and one dramatic incident, which allows us to conclude that they didn’t feel pressured to alter their responses to please the automation team member. as stated above, the field of rpa is still widely under-researched (hofmann et al., 2020; syed et al., 2020); we know particularly little about the implications of rpa implementations on the human workforce. our study was a first attempt to address this gap; however, we encourage further research on: a) if and how the different configurations evolve over time, b) the occurrence of ambivalent affective responses and the consequences for rpa adoption and (continued) use, c) the effects of rpa implementation on employees’ work processes and practices, and d) the relationship between anthropomorphism and employees’ attitude towards rpa and software robots. future studies that follow a configurational approach may want to consider qca, a set-theoretic approach that uses boolean algebra to evaluate which combinations of attributes combine to result in an outcome (fiss, 2007; misangyi et al., 2017; ragin, 1987, 2008). another valuable direction for future research is to look deeper into employees’ adaptation behaviours across different configurations of employees’ perceptions of and reactions to rpa. references aggarwal, p., & mcgill, a. l. (2007). is that car smiling at me? schema congruity as a basis for evaluating anthropomorphized products. journal of consumer research, 34(4), 468-479. doi:10.1086/518544 aguirre, s., & rodriguez, a. (2017). automation of a business process using robotic process automation (rpa): a case study. in l.-s. e. figueroa-garcía j., villa-ramírez j., ferroescobar r. (ed.), applied computer sciences in engineering. wea 2017. communications in computer and information science (vol. 742, pp. 65-71). doi:https://doi.org/10.1007/978-3319-66963-2_7 alberth, m., & mattern, m. (2017). understanding robotic process automation (rpa). journal of financial transformation, 46, 54-61. aroles, j., cecez-kecmanovic, d., dale, k., kingma, s. f., & mitev, n. (2021). new ways of working (nww): workplace transformation in the digital age. information and organization, 31(4), 1-11. doi:10.1016/j.infoandorg.2021.100378 asatiani, a., & penttinen, e. (2016). turning robotic process automation into commercial success – case opuscapita. journal of information technology teaching cases, 6(2), 67-74. doi:10.1057/jittc.2016.5 association of chartered certified accountants. (2015). the robots are coming? implications for finance shared services. 11-14. https://doi.org/10.1007/978-3-319-66963-2_7 https://doi.org/10.1007/978-3-319-66963-2_7 australasian journal of information systems waizenegger & techatassanasoontorn 2022, vol 26, selected papers from acis 2020 when robots join our team 23 retrieved from https://www.accaglobal.com/us/en/technical-activities/technicalresources-search/2015/september/robots-and-the-future-of-finance.html bala, h., & venkatesh, v. (2013). changes in employees' job characteristics during an enterprise system implementation: a latent growth modeling perspective. mis quarterly, 37(4), 1113-1140. bala, h., & venkatesh, v. (2015). adaptation to information technology: a holistic nomological network from implementation to job outcomes. management science, 62(1), 156-179. doi:10.1287/mnsc.2014.2111 baptista, j., stein, m.-k., klein, s., watson-manheim, m. b., & lee, j. (2020). digital work and organisational transformation: emergent digital/human work configurations in modern organisations. the journal of strategic information systems, 29(2), 1-10. doi:10.1016/j.jsis.2020.101618 beaudry, a., & pinsonneault, a. (2005). understanding user responses to information technology: a coping model of user adaptation. mis quarterly, 29(3), 493-524. doi:10.2307/25148693 benbya, h., nan, n., tanriverdi, h., & yoo, y. (2020). complexity and information systems research in the emerging digital world. mis quarterly, 44(1), 1-17. bhattacherjee, a., davis, c. j., connolly, a. j., & hikmet, n. (2018). user response to mandatory use: a coping theory perspective. european journal of information systems, 27(4), 395-414. doi:10.1057/s41303-017-0047-0 boudreau, m.-c., & robey, d. (2005). enacting integrated information technology: a human agency perspective. organization science, 16(1), 3-18. doi:10.1287/orsc.1040.0103 braun, v., & clarke, v. (2006). using thematic analysis in psychology. qualitative research in psychology, 3(2), 77-101. doi:10.1191/1478088706qp063oa castelluccio, m. (2017). artificial intelligence in business. strategic finance, 98(10), 55-57. retrieved from https://sfmagazine-com.ezproxy.aut.ac.nz/post-entry/april-2017artificial-intelligence-in-business/ cewe, c., koch, d., & mertens, r. (2018). minimal effort requirements engineering for robotic process automation with test driven development and screen recording. in w. m. teniente e. (ed.), lecture notes in business information processing. business process management workshops. bpm 2017. lecture notes in business information processing (vol. 308, pp. 642-648). doi:https://doi.org/10.1007/978-3-319-74030-0_51 denagama vitharanage, i. m., bandara, w., syed, r., & toman, d. (2020). an empirically supported conceptualisation of robotic process automation (rpa) benefits. in the proceedings of 28th european conference on information systems (ecis2020). https://aisel.aisnet.org/ecis2020_rip/58 eikebrokk, t. r., & olsen, d. h. (2020). robotic process automation and consequences for knowledge workers; a mixed-method study. in the proceedings of responsible design, implementation and use of information and communication technology: 19th ifip wg 6.11 conference on e-business, e-services, and e-society, skukuza, south africa https://dx.doi.org/10.1007/978-3-030-44999-5_10 https://www.accaglobal.com/us/en/technical-activities/technical-resources-search/2015/september/robots-and-the-future-of-finance.html https://www.accaglobal.com/us/en/technical-activities/technical-resources-search/2015/september/robots-and-the-future-of-finance.html https://sfmagazine-com.ezproxy.aut.ac.nz/post-entry/april-2017-artificial-intelligence-in-business/ https://sfmagazine-com.ezproxy.aut.ac.nz/post-entry/april-2017-artificial-intelligence-in-business/ https://doi.org/10.1007/978-3-319-74030-0_51 https://aisel.aisnet.org/ecis2020_rip/58 https://dx.doi.org/10.1007/978-3-030-44999-5_10 australasian journal of information systems waizenegger & techatassanasoontorn 2022, vol 26, selected papers from acis 2020 when robots join our team 24 epley, n., waytz, a., & cacioppo, j. t. (2007). on seeing human: a three-factor theory of anthropomorphism. psychological review, 114(4), 864–886. doi:https://doi.org/10.1037/0033-295x.114.4.864 fernandez, d., & aman, a. (2018). impacts of robotic process automation on global accounting services. asian journal of accounting and governance, 9, 123-132. doi:10.17576/ajag-201809-11 fiss, p. c. (2007). a set-theoretic approach to organizational configurations. academy of management review, 32(4), 1180-1198. doi:10.5465/amr.2007.26586092 fiss, p. c. (2011). building better causal theories: a fuzzy set approach to typologies in organization research. academy of management journal, 54(2), 393-420. doi:10.5465/amj.2011.60263120 fong, c. t. (2006). the effects of emotional ambivalence on creativity. the academy of management journal, 49(5), 1016-1030. doi:org/10.5465/amj.2006.22798182 furnari, s., crilly, d., misangyi, v. f., greckhamer, t., fiss, p. c., & aguilera, r. (2020). capturing causal complexity: heuristics for configurational theorizing. academy of management review, 46(4), 1-50. doi:https://doi.org/10.5465/amr.2019.0298 guthrie, s. (1993). faces in the clouds: a new theory of religion. new york: oxford university press. hallikainen, p., bekkhus, r., & pan, s. l. (2018). how opuscapita used internal rpa capabilities to offer services to clients. mis quarterly executive, 17(1), 41-52. hinings, c. (2018). why should we bother? what are configurations for? strategic organization, 16(4), 499-509. doi:10.1177/1476127018804796 hofmann, p., samp, c., & urbach, n. (2020). robotic process automation. electronic markets, 30(1), 99-106. doi:https://doi.org/10.1007/s12525-019-00365-8 hsu, p.-f., yen, h. r., & chung, j.-c. (2015). assessing post-implementation success at the individual level: revisiting the role of service quality. information &management, 52(8), 925-942. doi:10.1016/j.im.2015.06.009 ieee corporate advisory group (2017). ieee guide for terms and concepts in intelligent process automation. institute for robotic process automation. (2015). introduction to robotic process automation: a primer. 1-34. retrieved from https://www.businesssystemsuk.co.uk/uploads/attachments/papers/robotic-processautomation-june2015.pdf ivančić, l., suša vugec, d., bosilj vukšić, v. (2019). robotic process automation: systematic literature review. in r. g. c. di ciccio, l. garcía-bañuelos, t. hernaus, r. hull, m. indihar štemberger, a. kő, m. staples (eds.), lecture notes in business information processing (vol. 361, pp. 280-295): springer, cham. https://doi.org/10.1007/978-3-03030429-4_19 ketchen, d. j., combs, j. g., russell, c. j., shook, c., dean, m. a., runge, j., . . . lamoureux, s. (1997). organizational configurations and performance: a meta-analysis. academy of management journal, 40(1), 223-240. doi:10.5465/257028 https://doi.org/10.1037/0033-295x.114.4.864 https://doi.org/10.5465/amr.2019.0298 https://doi.org/10.1007/s12525-019-00365-8 https://www.businesssystemsuk.co.uk/uploads/attachments/papers/robotic-process-automation-june2015.pdf https://www.businesssystemsuk.co.uk/uploads/attachments/papers/robotic-process-automation-june2015.pdf australasian journal of information systems waizenegger & techatassanasoontorn 2022, vol 26, selected papers from acis 2020 when robots join our team 25 klein, s., & watson-manheim, m. b. (2021). the (re-)configuration of digital work in the wake of profound technological innovation: constellations and hidden work. information and organization, 31(4), 1-15. doi:10.1016/j.infoandorg.2021.100377 kyheröinen, t. (2018). implementation of robotic process automation to a target process–a case study. (masters thesis). aalto university, aaltodoc. lacity, m., & willcocks, l. (2016a). robotic process automation at telefónica. mis quarterly executive, 15(1), 21-35. lacity, m., & willcocks, l. (2016b). robotic process automation: the next transformation lever for shared services. the outsourcing unit working research paper series, 1-35. lacity, m., & willcocks, l. p. (2017). a new approach to automating services. mit sloan management review, 58(1), 1-16. lamberton, c., gillard, a., & kaczmarskyj, g. (2016). get ready for robots why planning makes the difference between success and disappointment. 1-9. retrieved from https://www.ey.com/publication/vwluassets/get_ready_for_robots/$file/ey-getready-for-robots.pdf lapointe, & rivard. (2005). a multilevel model of resistance to information technology implementation. mis quarterly, 29(3), 461-491. doi:10.2307/25148692 levers, m.-j. d. (2013). philosophical paradigms, grounded theory, and perspectives on emergence. sage open, 3(4), 1-6. doi:10.1177/2158244013517243 met, i̇., kabukçu, d., uzunoğulları, g., soyalp, ü., & dakdevir, t. (2020). transformation of business model in finance sector with artificial intelligence and robotic process automation. in u. hacioglu (ed.), digital business strategies in blockchain ecosystems. contributions to management science. (pp. 3-29). doi:https://doi.org/10.1007/978-3-03029739-8_1 meyer, a. d., tsui, a. s., & hinings, c. r. (1993). configurational approaches to organizational analysis. academy of management journal, 36(6), 1175-1195. doi:https://doi.org/10.5465/256809 miller, d. (1996). configurations revisited. strategic management journal, 17(7), 505-512. https://www.jstor.org/stable/2486729 miller, d. (2018). challenging trends in configuration research: where are the configurations? strategic organization, 16(4), 453-469. doi:10.1177/1476127017729315 misangyi, v. f., greckhamer, t., furnari, s., fiss, p. c., crilly, d., & aguilera, r. (2017). embracing causal complexity. journal of management, 43(1), 255-282. doi:10.1177/0149206316679252 morris, & venkatesh. (2010). job characteristics and job satisfaction: understanding the role of enterprise resource planning system implementation. mis quarterly, 34(1), 143-161. doi:10.2307/20721418 moussawi, s., koufaris, m., & benbunan-fich, r. (2020). how perceptions of intelligence and anthropomorphism affect adoption of personal intelligent agents. electronic markets, 31, 343–364. doi:https://doi.org/10.1007/s12525-020-00411-w https://www.ey.com/publication/vwluassets/get_ready_for_robots/$file/ey-get-ready-for-robots.pdf https://www.ey.com/publication/vwluassets/get_ready_for_robots/$file/ey-get-ready-for-robots.pdf https://doi.org/10.1007/978-3-030-29739-8_1 https://doi.org/10.1007/978-3-030-29739-8_1 https://doi.org/10.5465/256809 https://doi.org/10.1007/s12525-020-00411-w australasian journal of information systems waizenegger & techatassanasoontorn 2022, vol 26, selected papers from acis 2020 when robots join our team 26 myers, m. d., & newman, m. (2007). the qualitative interview in is research: examining the craft. information and organization, 17(1), 2-26. doi:doi.org/10.1016/j.infoandorg.2006.11.001 myers, m. d., & walsham, g. (1998). exemplifying interpretive research in information systems: an overview. journal of information technology, 13(4), 233 234. ortiz de guinea, a., & webster, j. (2013). an investigation of information systems use patterns: technological events as triggers, the effect of time, and consequences for performance. mis quarterly, 37(4), 1165-1188. https://www.jstor.org/stable/43825786 penttinen, e., kasslin, h., & asatiani, a. (2018). how to choose between robotic process automation and back-end system automation? in the proceedings of european conference on information systems, portsmouth, uk. https://aisel.aisnet.org/ecis2018_rp/66 plattfaut, r. (2019). robotic process automation–process optimization on steroids? in the proceedings of international conference on information systems, munich, germany. https://aisel.aisnet.org/icis2019/practice_is_research/practice_is_research/3/ pratt, m. g., & doucet, l. (2000). ambivalent feelings in organizational relationships. in s. fineman (eds.), emotion in organizations (pp. 204-226). london: sage publications. qiu, l., & benbasat, i. (2005). an investigation into the effects of text-to-speech voice and avatars on the perception of presence and flow of live help in electronic commerce. acm transactions on computer-human interaction (tochi), 12(4), 329-355. doi:10.1145/1121112.1121113 ragin, c. c. (1987). the comparative method: moving beyond qualitative and quantitative strategies. berkeley: university of california press. ragin, c. c. (2008). redesigning social inquiry: fuzzy sets and beyond. chicago: university of chicago press. ratia, m., myllärniemi, j., & helander, n. (2018). robotic process automation value by digitalizing work in the private healthcare? in the proceedings of international academic mindtrek conference, tampere, finland https://dx.doi.org/10.1145/3275116.3275129 rosso, b. d., dekas, k. h., & wrzesniewski, a. (2010). on the meaning of work: a theoretical integration and review. research in organizational behavior, 30, 91-127. doi:10.1016/j.riob.2010.09.001 rutaganda, l., bergstrom, r., jayashekhar, a., jayasinghe, d., & ahmed, j. (2017). avoiding pitfalls and unlocking real business value with rpa journal of financial transformation, 46, 104-115. santos, f., pereira, r., & vasconcelos, j. b. (2019). toward robotic process automation implementation: an end-to-end perspective. business process management journal, 26(2), 405-420. doi:doi:10.1108/bpmj-12-2018-0380 sarker, s., xiao, x., beaulieu, t., & lee, a. s. (2018). learning from first-generation qualitative approaches in the is discipline: an evolutionary view and some implications for authors and evaluators (part 1/2). journal of the association for information systems, 19(8), 752-774. doi:10.17705/1jais.00508 https://www.jstor.org/stable/43825786 https://aisel.aisnet.org/ecis2018_rp/66 https://aisel.aisnet.org/icis2019/practice_is_research/practice_is_research/3/ https://dx.doi.org/10.1145/3275116.3275129 australasian journal of information systems waizenegger & techatassanasoontorn 2022, vol 26, selected papers from acis 2020 when robots join our team 27 schuetzler, r. m., grimes, g. m., & scott giboney, j. (2020). the impact of chatbot conversational skill on engagement and perceived humanness. journal of management information systems, 37(3), 875-900. doi:10.1080/07421222.2020.1790204 seeber, i., bittner, e., briggs, r. o., de vreede, t., de vreede, g.-j., elkins, a., . . . söllner, m. (2020). machines as teammates: a research agenda on ai in team collaboration. information & management, 57(2), 1-22. doi:https://doi.org/10.1016/j.im.2019.103174 seeber, i., waizenegger, l., seidel, s., morana, s., benbasat, i., & lowry, p. b. (2020). collaborating with technology-based autonomous agents: issues and research opportunities. internet research, 30(1), 1-18. doi:10.1108/intr-12-2019-0503 seiffer, a., gnewuch, u., & maedche, a. (2021). understanding employee responses to software robots: a systematic literature review. in the proceedings of international conference on information systems, austin, texas. https://aisel.aisnet.org/icis2021/is_future_work/is_future_work/5 sheehan, b., jin, h. s., & gottlieb, u. (2020). customer service chatbots: anthropomorphism and adoption. journal of business research, 115, 14-24. doi:10.1016/j.jbusres.2020.04.030 staaby, a., hansen, k. s., & grønli, t.-m. (2021). automation of routine work: a case study of employees' experiences of work meaningfulness. in the proceedings of the 54th hawaii international conference on system sciences. http://hdl.handle.net/10125/70629 statista. (2020). robotic process automation (rpa) adoption and investment rates worldwide in 2019, by organization size. technology & telecommunications software. retrieved from https://www.statista.com/statistics/1017027/worldwide-robotic-process-automationadoption-investment-rates-organization-size/ stein, m., newell, s., wagner, e., & galliers, r. (2015). coping with information technology: mixed emotions, vacillation, and nonconforming use patterns. mis quarterly, 39(2), 367392. doi:10.2307/26628358 syed, r., suriadi, s., adams, m., bandara, w., leemans, s. j., ouyang, c., . . . reijers, h. a. (2020). robotic process automation: contemporary themes and challenges. computers in industry, 115, 1-15. doi:10.1016/j.compind.2019.103162 thomas, d. r. (2003). a general inductive approach for qualitative data analysis. university of auckland, auckland. van der aalst, w. m. p., bichler, m., & heinzl, a. (2018). robotic process automation. business & information systems engineering, 60(4), 269-272. doi:10.1007/s12599-018-0542-4 vishnu, s., agochiya, v., & palkar, r. (2017). data-centered dependencies and opportunities for robotics process automation in banking. journal of financial transformation, 45(1), 6876. waizenegger, l., & techatassanasoontorn, a. a. (2020). “you have no idea how much we love casper” – developing configurations of employees’ rpa implementation experiences." (2020). in proceedings of the australasian conference on information systems 2020. https://aisel.aisnet.org/acis2020/21 https://doi.org/10.1016/j.im.2019.103174 http://hdl.handle.net/10125/70629 https://www.statista.com/statistics/1017027/worldwide-robotic-process-automation-adoption-investment-rates-organization-size/ https://www.statista.com/statistics/1017027/worldwide-robotic-process-automation-adoption-investment-rates-organization-size/ australasian journal of information systems waizenegger & techatassanasoontorn 2022, vol 26, selected papers from acis 2020 when robots join our team 28 walsham, g. (1995). the emergence of interpretivism in is research. information systems research, 6(4), 376-394. doi:https://doi.org/10.1287/isre.6.4.376 wanchai, p., díaz andrade, a., & techatassanasoontorn, a. (2019). it adaptation patterns to enterprise-wide systems. communications of the association for information systems, 45(1), 226-250. doi:10.17705/1cais.04515 wengraf, t. (2001). qualitative research interviewing: biographic narrative and semi-structured methods. london: sage. wessel, l., baiyere, a., ologeanu-taddei, r., cha, j., & blegind jensen, t. (2021). unpacking the difference between digital transformation and it-enabled organizational transformation. journal of the association for information systems, 22(1), 102-129. doi:10.17705/1jais.00655 yin yeh, j., & ouyang, y. c. (2010). how an organization changes in erp implementation: a semiconductor case study. business process management journal, 16(2), 209-225. doi:10.1108/14637151011035561 appendix table a1. possible effects of rpa implementation and use on human employees effects explanation illustrative examples or evidence from existing studies positive effects focus on higher-value work rpa can change the nature of employees’ work so that employees can now focus on tasks that require judgment, critical thinking, and problem-solving capabilities. the implementation of rpa in the purchase to pay process of the company basware allows employees who monitor the process to focus on tasks that require judgment-based decision making (castelluccio, 2017). penttinen et al. (2018) explored a process at a telco’s corporate customers unit, focusing on the availability check of the fibre-ethernet product. due to the implementation of rpa, employees could drop all tasks related to the availability check and now focus on processing the bids that require expert judgment. they receive the availability check results from the software robot, and then accept or reject the bid, determine the pricing for the accepted bids and answer the customer. reduction of mundane and repetitive tasks and workload rpa takes over mundane and repetitive tasks from human employees. based on a mixed-method study of organisations in the public and private sector, eikebrokk and olsen (2020) found that the reduction of mundane tasks is particularly common in public and financial companies. for instance, a participant from the public sector explains: “it has been a change in tasks where those https://doi.org/10.1287/isre.6.4.376 australasian journal of information systems waizenegger & techatassanasoontorn 2022, vol 26, selected papers from acis 2020 when robots join our team 29 tasks we still do are those that need human judgement, but we have got rid of those boring tasks […] so we can concentrate on new and unsolved tasks.” (eikebrokk & olsen, 2020, p. 122). higher accuracy and fewer errors software robots can perform tasks more accurately and make fewer errors than humans. payroll employees at opuscapita, a finnish business process outsourcing provider, were delighted after realising that software robots performed the tasks faster and more accurately than human employees (hallikainen et al., 2018). time savings the reduction of manual and repetitive tasks leads to time savings for employees. the vice president of the finance and accounting unit at an oil and gas company explains that rpa leads to time savings for human employees due to the reduction of manual and repetitive tasks (fernandez & aman, 2018). new roles for employees the implementation of rpa leads to the creation of new roles for employees. based on their literature review, syed et al. (2020) confirm that rpa could lead to the creation of new roles for employees. at telefonica o2, uk-based employees were redeployed to other service areas after the implementation of rpa (lacity & willcocks, 2016a). a robotic manager from eikebrokk and olsen (2020) study explains: “cutting costs can be done in many ways. everybody thinks immediately that people will be fired as the only way of cutting costs. this is totally wrong because you might rather use their time to do tasks that you so far have not been able to do in the company, including new services and tasks that have been neglected for a long time. you can simply do more with the same workforce and thus save money” (p.122). outsourcing non-valueadding tasks to allow employees to focus on customer interactions employees can now focus on interactions with their customers. a qualitative study exploring rpa potentials in the private healthcare sector in finland found that due to rpa, administrative and manual work could be outsourced to software robots so that the clinical staff, including doctors and nurses, can focus more on the interactions with patients and therefore create more value for patients. for example, inputting data can be outsourced to rpa in the visit records management process (ratia et al., 2018). increase in job satisfaction employees experience an increase in job satisfaction due to the change in the nature of their work. the payroll employees at the queensland university of technology (qut) experienced an increase in job satisfaction as they a) could focus on analytical and problem-solving skills instead of swivelaustralasian journal of information systems waizenegger & techatassanasoontorn 2022, vol 26, selected papers from acis 2020 when robots join our team 30 chair tasks and b) didn’t have to work overtime and on the weekend anymore (denagama vitharanage et al., 2020). increase in work meaningfulness rpa can facilitate work meaningfulness which is ”work experienced as particularly significant and holding more positive meaning for individuals” (rosso, dekas, & wrzesniewski, 2010, p. 95). staaby et al. (2021) found that all three case organisations in their study created initiatives to increase employees’ work meaningfulness. for example, the audit unit in the consultancy achieved an increase in work meaningfulness among their employees by providing them with more autonomy and the opportunity to engage in more complex tasks. another example is the employees of the administrative unit in a housing cooperation felt important with their new responsibility to operate software robot, which, in turn, had a positive effect on their work meaningfulness (staaby et al., 2021) more social relationships inside and outside an organisation rpa implementation facilitates collaboration within and across units in an organisation as well as increases the interactions with clients. the implementation of rpa led to the establishment and fostering of interpersonal relationships that was enabled through the participation in rparelated task forces that involved working with employees and managers from other departments and offices than their own (staaby et al., 2021). in eikebrokk and olsen (2020) study, the ceo from a wholesale company states “in our case, workers work more with sales tasks which leads to better market relations and increased sales – this is what makes a salesperson valuable” (p. 122) which implies that sales people can focus more on interactions with their clients and other stakeholders to build and foster relationships after rpa implementation. job crafting rpa allows employees to engage in job crafting, which involves changing their work and jobs to enhance their skill sets and job opportunities. the employees of the audit unit in a consultancy organisation engaged in job crafting. for example, one of the implementers developed programming skills that he leveraged to develop digital solutions for accounting practices within his department (staaby et al., 2021). enhancement of knowledge and skills the involvement in the rpa projects allows employees to extend their skill set and broaden their knowledge on automation technologies and process improvement. the employees who worked on the payroll process at qut learned about automation that they wouldn’t have without the implementation of rpa (denagama vitharanage et al., 2020). at telefonica o2, back-office staff members received training in the rpa software, blue prism, and were able to upskill within three months so that they could automate australasian journal of information systems waizenegger & techatassanasoontorn 2022, vol 26, selected papers from acis 2020 when robots join our team 31 end-to-end processes independently (lacity & willcocks, 2016a). a respondent from eikebrokk and olsen (2020) highlights, “they [workers] do other and newer tasks because it [rpa] frees up capacity to prioritize differently and learn from new insight into process and technology” (p.122). negative effects actual job loss employees face redundancies due to the implementation of rpa. a process rationalisation initiative at telefonica o2, which involved process elimination, simplification and optimisation, led to the reduction of the back-office headcount by 10% (lacity & willcocks, 2016a). based on the mixed-method study conducted by eikebrokk and olsen (2020) in norway, downsizing was more common in the private sector than in the public sector. financial companies report more downsizing than companies in other sectors. one of the strategic managers of a bank states: “yes, there is a lot of [downsizing] in the finance industry. there are severance packages three times a year” (p.120). no new recruitments instead of hiring new employees, organisations choose to use software robots. a strategic manager at a norwegian bank highlights that his bank doesn’t hire new people anymore, but that they employ rpa instead (eikebrokk & olsen, 2020). another participant from the supplier industry explains that “we grow without [hiring] new people. […] rpa is a way to handle a part of the growth.” (eikebrokk & olsen, 2020, p. 121) concerns about job insecurity employees fear that they will lose their job to rpa. the operational manager of an oil and gas company reports that the accountants fear losing their jobs due to the implementation of rpa. the human resource manager of the same company states that due to the reduction of employees, the competition increases not only between human employees but also between employees and robots (fernandez & aman, 2018). opuscapita, a finnish business process outsourcing provider, implemented rpa internally and then started to provide rpa services to clients. in their pilot project, they focused on payroll production and automated the “new employment relationships” process as well as the “changes in employee payment details” process. when payroll employees learned australasian journal of information systems waizenegger & techatassanasoontorn 2022, vol 26, selected papers from acis 2020 when robots join our team 32 about the implementation of rpa they feared to lose their jobs to robots (hallikainen et al., 2018) necessity to learn new skills employees are forced to learn new skills due to the changes in their work processes. the operational manager of the oil and gas company states that accountants are worried about having to learn new skills due to the rpa-induced changes to their work processes (fernandez & aman, 2018). the payroll employees at opuscapita had to learn new daily work practices such as how to read and interpret the reports from the software robots (hallikainen et al., 2018). increase in workload employees perceive an increase in workload either of the same kind of tasks or different tasks. due to the implementation of rpa at an administrative unit in a housing cooperation, the employees reported that they have to complete an increased amount of the same type of work that they receive from the regional office (staaby et al., 2021). superiority of software robot performance human employees are usually less productive, slower, make more errors, and cost more than software robots. the implementation of rpa in a payment receipt generation process showed that the group with rpa could handle 21% more cases than the group without rpa. further, the group with rpa was 9 seconds faster than the group without rpa and the human-robot team was more productive because the software robots could perform several cases at the same time. this highlights the superiority of software robots over employees (aguirre & rodriguez, 2017). work alienation due to standardisation standardisation affects employees’ autonomy negatively which can result in work alienation. standardisation can have a negative impact on the autonomy of employees as highlighted in the case of an audit unit at a consultancy organisation. a manager of this unit highlights that his employees are knowledge workers and not assembly line workers and that they want to work autonomously. as a result, they react negatively to standardisation of work, which could result in work alienation (staaby et al., 2021). copyright: © 2022 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems waizenegger & techatassanasoontorn 2022, vol 26, selected papers from acis 2020 when robots join our team 33 doi: https://doi.org/10.3127/ajis.v26i0.3833 1 introduction 2 theoretical background 2.1 employees’ perceptions of it-enabled organisational change and responses to it implementations 2.2 effects of rpa on the human workforce 2.3 configurational approach 3 methodology 3.1 data collection 3.2 data analysis 4 findings 4.1 software robots as a burden and threat 4.2 software robots as tools 4.3 software robots as teammates 4.4 software robots as innovative enablers 5 discussion 5.1 theoretical contributions 5.2 managerial implications 6 conclusion paper title (paper title style) australasian journal of information systems xu et al. 2024, vol 28, research article using analytical information for digital business transformation 1 using analytical information for digital business transformation through dataops: a review and conceptual framework jia xu university of melbourne, australia jx3@student.unimelb.edu.au humza naseer rmit university, melbourne, australia sean maynard university of melbourne, australia justin filippou university of melbourne, australia abstract organisations are increasingly practising business analytics to generate actionable insights that can guide their digital business transformation. transforming business digitally using business analytics is an ongoing process that requires an integrated and disciplined approach to leveraging analytics and promoting collaboration. an emerging business analytics practice, data operations (dataops), provides a disciplined approach for organisations to collaborate using analytical information for digital business transformation. we propose a conceptual framework by reviewing the literature on business analytics, dataops and organisational information processing theory (oipt). this conceptual framework explains how organisations can employ dataops as an integrated and disciplined approach for developing the analytical information processing capability and facilitating boundary-spanning activities required for digital business transformation. this research (a) extends current knowledge on digital transformation by linking it with business analytics from the perspective of oipt and boundary-spanning activities, and (b) presents dataops as a novel approach for using analytical information for digital business transformation. keywords: digital business transformation, business analytics, analytical information processing capability, boundary spanning, dataops. 1 introduction digital business transformation is a strategic imperative for many organisations. gartner (2021) reports that 58% of boards flagged digital tech initiatives as the single biggest strategic business priority. organisations invest in digital technologies such as the internet of things (iot), cloud computing, big data analytics, and artificial intelligence to transform their business and thereby improve customers’ experience, achieve operational efficiency, deliver new products or services, and innovate with new business models (bonnet & westerman, 2021; dwivedi et al., 2021). while these disruptive digital technologies are valuable to organisations, warner and wäger (2019) suggest that they may also create uncertainties in the business environment. hence, organisations need to adjust their business strategies to thrive in the dynamic digital environment (sia et al., 2016). australasian journal of information systems xu et al. 2024, vol 28, research article using analytical information for digital business transformation 2 digital business transformation refers to how organisations can alter their value creation and change the scope of their business by using digital technologies (hess et al., 2016). in this digital age, the ability to manage and exploit data for value creation and business transformation is becoming a vital competency in organisations’ transformation journeys (dremel et al., 2017). hence, organisations are increasingly practising business analytics to generate actionable insights that can guide their digital transformation initiatives (pappas et al., 2018; sebastian et al., 2017). business analytics is defined as the extensive use of data, statistical and quantitative analysis, and explanatory or predictive models to drive decisions and actions (davenport & harris, 2017). through business analytics, organisations generate analytical information that refers to the insights (quantitative or qualitative) used by business executives in their analysis and decision-making (naseer et al., 2021). for instance, predicted atm usage and customer withdrawal patterns can be identified by analysing transactional data, which a bank could then use to optimise its business (sia et al., 2016). the capacity to generate analytical information for business transformation in organisations that have high volumes and wide varieties of data requires a mature business analytics capability (vidgen et al., 2017). although existing research has explored how organisations could gain value from business analytics (e.g., wixom et al., 2013; vidgen et al., 2017;grover et al., 2018; and wee et al., 2022), there is a scarcity of research that links the value of business analytics with digital business transformation. in the context of digital business transformation, business analytics needs to not only support decision-making, but more importantly, it needs to support organisations to drive change in the business and generate a transformative impact on the business across the whole organisation. using business analytics for digital business transformation requires organisations to develop enterprise-wide analytical capabilities and promote collaboration activities (karippur & balaramachandran, 2022; setia et al., 2014). dataops is a business analytics practice that aims to better manage data, provide high-quality insights, and promote collaboration through analytics in a dynamic business environment (ereth, 2018; heudecker et al., 2020). the majority of digital transformation literature focuses on digital transformation strategy (e.g., hess et al., 2016; ismail et al., 2017; and the organisational and societal impact of digital technologies (e.g., nambisan et al., 2019). however, digital business transformation is an ongoing, long-term process (davenport & westerman, 2018), and organisations need a disciplined approach to using analytical information for their transformation. although dataops has the potential to be such an approach, there is little research linking dataops with digital business transformation. therefore, we employ organisational information processing theory (oipt) (galbraith, 1974; premkumar et al., 2005; tushman & nadler, 1978) and a boundary spanning activities perspective (aldrich & herker, 1977; ancona & caldwell, 1990; someh & shanks, 2013) to link dataops with digital business transformation. subsequently, this research aims to answer the following research question: how do organisations use analytical information for digital business transformation through dataops? to address this research question, we propose a conceptual framework by reviewing the diverse areas of literature on business analytics, dataops, and digital business transformation through the lens of oipt. the details of the framework explain how organisations can employ dataops to develop an analytical information processing capability and facilitate boundary spanning activities for digital business transformation. australasian journal of information systems xu et al. 2024, vol 28, research article using analytical information for digital business transformation 3 in the next section, we provide the theoretical background and present the research method for our literature review. then we synthesize the business analytics, dataops, oipt and digital business transformation literature to develop the conceptual framework. we also develop a set of propositions that explain the utility of our framework. finally, we conclude the paper by describing the contributions and limitations of our work. 2 theoretical background in this section, we review the literature around the intersection of business analytics, dataops, oipt, and digital business transformation. in addition, we also link the key concepts in our study (i.e., analytical information, analytical information processing capability, boundary spanning activities, and dataops) to the context of digital business transformation. 2.1 using analytical information for digital business transformation existing research views digital transformation as the evolution of digitization and digitalization (verhoef et al., 2021). digitization refers to the encoding of analogue information into digital format (yoo, 2010). digitalization is about how it or digital technologies can be used to alter existing business processes for not only cost savings but also process improvements (verhoef et al., 2021). compared with digitization and digitalization, digital transformation is a broader concept. it refers to an enterprise-wide phenomenon with broad organisational implications in which, most notably, the core business model of the firm is subject to change by using digital technologies (hess et al., 2016; verhoef et al., 2021). digitization, digitalization, and digital transformation can be viewed as three phases of the transformation journey, in which organisations “may start with minor changes (e.g., digitization or digitalization) to gradually transform their traditional business into a digital one” (verhoef et al., 2021, p. 892). this research focuses on how organisations transform their business in the digital era using analytical information. an organisation can be viewed as an information processing system facing complexity and uncertainty (tushman & nadler, 1978). in oipt, uncertainty refers to the absence of information (daft & lengel, 1986). when organisations first start their transformation journeys, they are very likely to encounter a high degree of uncertainty (matt et al., 2015; vial, 2019). for instance, given the changing diffusion of technologies and customers’ expectations, organisations may lack information that can guide them regarding how to transform their products or services using disruptive technologies to improve customer satisfaction (ismail et al., 2017; matt et al., 2015). organisations may need rich analytical information about their current operations assessment to identify bottlenecks that need to be transformed by digital technologies (sia et al., 2016). analytical information about customers’ demands is also needed to sense an opportunity for new business models (loebbecke & picot, 2015; setia et al., 2014). indeed, digitization and digitalization have fostered the generation of big data, which emphasizes the volume, velocity, variety, veracity, variability, and value of data (conboy et al., 2020). big data provides huge opportunities for organisations to get rich analytical information (dwivedi et al., 2021; grover et al., 2018; ranjan & foropon, 2021). by applying statistical and quantitative analysis, and explanatory and predictive models to analyse big data, organisations can get useful analytical information that affords them opportunities for their transformations (e.g., improving the current business models or creating new business models) (bonnet & westerman, 2021). table 1 lists examples of how analytical information is obtained from big data and used for digital business transformation in different industries. australasian journal of information systems xu et al. 2024, vol 28, research article using analytical information for digital business transformation 4 industry data analytical information digital business transformation reference manufacturing product-related data from embedded sensors patterns of product usage extracted from product usage segment the market and identify opportunities for new product development and new services (frank et al., 2019) telecommunicatio ns data generated by interactions with users on social media best practices and methods for marketing approaches by applying social media analytics improve customers satisfaction and position itself as the largest service provider in the market (conboy et al., 2020) technology provider customers’ service requests and feedback real-time patterns and trends of customers’ needs by monitoring the feedback of thousands of customers with text analytics achieve efficiency gains and improve customer service (müller et al., 2016) retailer business data such as orders and delivery schedule predictive information (e.g., future sales) and prescriptive information (e.g., price recommendations transforming the supply chain management by proactively preparing for emergency situations and preventing and mitigating risks (papanagnou et al., 2022) entertainment user behavioural data, public feedback, and comments insights (e.g., fail rate and user’s preference) that help to optimize the game design and maximize user experience shifting towards a new business model (tim et al., 2020) electronic sensors’ data and satellite images predictive demand of electricity to match the supply and demand revolutionising the operational processes through which electricity is generated, transported, distributed, and sold. (neirotti et al., 2021) table 1. examples of using analytical information for digital business transformation in different industries 2.2 the need for analytical information processing capability and boundary spanning activities for digital business transformation modern digital technologies provide two notable business value drivers: the expansive role of data, and digital ecosystems (subramaniam, 2021). we argue that analytical information processing capability and boundary spanning activities are needed to realize these business value drivers. analytical information processing capability enables organisations to transform expansive data into useful analytical information that solves uncertainties in the transformation journey (cao et al., 2019). boundary spanning activities enable organisations to share their analytical information (karippur & balaramachandran, 2022; someh & shanks, 2013) and foster the digital ecosystem for digital business transformation (tan et al., 2020). below, we explain why these two aspects are necessary for organisations’ transformation journeys. based on galbraith (1974, p. 28), “the greater the uncertainty is, the greater the amount of information is needed to be processed among decision-makers to achieve a given level of performance.” to get the needed information, organisations need to develop the information processing capability: the ability to gather, transform, interpret, synthesize, analyse, store, and australasian journal of information systems xu et al. 2024, vol 28, research article using analytical information for digital business transformation 5 communicate data, information, and knowledge to cope with the variety, uncertainty and unclear environment (naseer et al., 2021; premkumar et al., 2005; tushman & nadler, 1978). according to li et al. (2021), the information processing capability enabled by digital technologies can positively impact business transformation by enhancing market agility. we argue that analytical information processing capability, a specialised form of information processing capability enabled by business analytics, is needed to generate analytical information for digital business transformation. analytical information processing capability is the ability to use business analytics technologies or applications that analyse critical business data to better understand business and market and make timely business decisions (anand et al., 2020; cao et al., 2019; naseer et al., 2021; saldanha et al., 2017). from the boundary spanning perspective, a digital business transformation has a boundaryspanning nature. boundary spanning refers to the activities or practices that transcend functional division and increase communication and coordination between different stakeholders (schotter et al., 2017). from the internal boundary perspective, digital business transformation needs cross-functional efforts and requires alignment between different functions (matt et al., 2015). for example, dremel et al. (2017) emphasize the important role of collaboration among the sales and marketing departments, digital innovation hub, and it department in leveraging business analytics to transform their business. treating digital business transformation in functional silos would not maximise cross-fertilization opportunities (verhoef et al., 2021). from the external boundary perspective, the digital business transformation will impact the interactions that take place across firm borders with their clients, competitors, partners, and suppliers (hess et al., 2016). sia et al. (2016) give an example of how an organisation collaborated with an external research institute to develop its analytics capabilities to better understand its customers and thus provide more personalized interactions. pappas et al. (2018) also propose a big data and business analytics ecosystem where data, information and knowledge are shared and transferred among stakeholders, enabling collaboration among multiple actors, including private and public organisations, academia, and individuals/entrepreneurs. this results in new business opportunities, the development of digital data-based designs, and the transformation of current business models (pappas et al., 2018). the blurring boundaries of entities resulting from digital business transformation exemplify the need for boundary spanning activities to enable organisations to use analytical information effectively. existing research has identified a wide range of boundary spanning activities such as buffering and representing activities, coordination of task performance, information searching, and guarding (ancona & caldwell, 1990; marrone, 2010). in our research, we focus more on the activities that enable organisations to share and exchange information (fleischer & carstens, 2021), bridge the knowledge gap (someh & shanks, 2013), and facilitate communication, interactions and cooperation (schotter et al., 2017) across different domains (e.g., business departments, suppliers, partners, and customers) involved in the transformation journey. table 2 summarizes boundary spanning activities that seamlessly enable a team/organisation to interact with its external actors and meet the overall goals of digital business transformation. these boundary spanning activities can complement analytical information processing capability by disseminating the information delivered by the analytics process, thus facilitating the cross-boundary collaboration required for digital business transformation. australasian journal of information systems xu et al. 2024, vol 28, research article using analytical information for digital business transformation 6 activity types explanation examples reference organisational structure design arranging business functions and resources to promote the crossboundary collaboration decentralized governance architecture; having crossfunctional work groups (singh et al., 2020) task coordinating interactions with interdependent entities to solve coordinating issues, synchronize work efforts, exchange, and combine work outcomes in the transformation journey, formal or informal meetings to discuss data ownership problems, obtain feedback, coordinate, and negotiate with outsiders. (ancona & caldwell, 1990; dremel et al., 2017; levina & vaast, 2014; marrone, 2010) scouting searching and accessing information or expertise from different domains; learning different organisational systems, and developing a shared language to interact with them sensing and exploiting new opportunities to implement analytics-enabled transformation initiatives in other functional areas (someh & shanks, 2013; yang et al., 2021) embedding developing social ties and establishing relationships with another based on familiarity, trust, and commitment to connect resources, collaborate, share knowledge creating a data-driven culture and embedding analytics in the organisation’s process and routines. (someh & shanks, 2013) influencing influencing or forcing other functional areas or organisations to conform to values, norms or traditions, and social expectations in an institutional environment the successful adoption of analytical initiatives in one functional area may influence other functional areas to change their values and norms (someh & shanks, 2013) adopting boundary spanning objects adopting and having entities that enhance the capacity of an idea, theory, or practice to translate across culturally defined boundaries, for example, between communities of knowledge or practice. developing protocols, repositories, standardized documentation, models, information systems and collaboration tools to facilitate the shared understanding and collaboration work of different actors (beckett, 2021; fox, 2011; vilvovsky, 2009) table 2. types of boundary spanning activities digital business transformation is a long-term process and journey. as emphasized by warner and wäger (2019, p. 327), “the genuine digital transformations are an ongoing process of using digital technologies in everyday organisational life”. the long-term nature of digital business transformation requires not only extensive investment and time but also a holistic and integrated approach to make steady progress toward the right end state (davenport & westerman, 2018; sia et al., 2016). according to oipt, organisations should develop their information processing capabilities to fulfil their information needs (flynn & flynn, 1999; galbraith, 1974; premkumar et al., 2005). this means that there should be a disciplined approach that supports organisations (a) to develop their analytical information processing capability to constantly fulfil their analytical information needs, and (b) to facilitate boundary spanning activities to better share analytical information throughout the transformation journey. we argue dataops as such a disciplined approach and explain its potential role in the digital business transformation below. australasian journal of information systems xu et al. 2024, vol 28, research article using analytical information for digital business transformation 7 2.3 linking dataops with digital business transformation with the high volume and wide variety of data being generated each day, getting and using analytical information from analytical information processing capability and boundary spanning activities is challenging. wells (2019) and zahid et al. (2018) identify several challenges, including rapidly increasing data volumes, more sources of data, more data use cases, more stakeholders in modern data ecosystems, and poor data management. these challenges increase the risk of conflicting and erroneous data, the establishment of data silos, and delays in getting needed information, which subsequently inhibits organisations from extracting analytical information as required for digital business transformation. dataops, as an emerging business analytics practice, aims to overcome such challenges in modern analytical ecosystems. popularised by palmer (2015), dataops acknowledges the interconnected nature of data engineering, integration, quality, security and privacy to help organisations accelerate analytics and enable previously impossible analytics tasks. dataops applies to analytics the best principles and practices from diverse methods (i.e., agile, devops, lean, and total quality management) used in the software engineering and manufacturing areas. it enables organisations to streamline the process of analytics, maximize the business value of data as well as improve business analytics user satisfaction (eckerson, 2019a; ereth & eckerson, 2018). since dataops was coined, it has been most closely associated with the quality and efficiency in the delivery of analytical information (aslett, 2020; capizzi et al., 2020). from the perspective of quality, dataops helps to promote communication between, and integration of, formerly siloed data, teams, and systems (thusoo & sarma, 2017), which reduces the risks of conflicting and erroneous results from analytics and thus ensures the quality of analytical information. from the perspective of efficiency, dataops represents a culture change that focuses on improving collaboration and accelerating analytical delivery by adopting lean or iterative practices where appropriate (heudecker et al., 2020). moreover, teamwork and collaboration are the key themes in dataops (atwal, 2020). transforming data into insights usually requires collaboration between different stakeholders such as data architects, engineers, analysts, and scientists, as well as it operations and business users (ereth & eckerson, 2018). the goal of dataops is to foster greater collaboration and trust among development, test, operations, and business teams and thus improve both the speed and quality of analytical information delivery (ereth & eckerson, 2018). therefore, in our research, dataops is viewed as a disciplined approach to improving collaboration and building analytical capabilities in a way that continuously accelerates output and enhances the quality of analytical information (ereth & eckerson, 2018; naseer et al., 2020). given the potential of dataops, organisations can adopt dataops to develop their analytical information processing capability and facilitate the boundary spanning activities to get their required analytical information that will improve the likelihood of optimising digital business transformation. 3 research method we conducted an integrative literature review to combine the diverse literature on digital business transformation, business analytics, and dataops, following the guidelines outlined in snyder (2019). the aim of the integrative literature review is to assess, critique, and synthesize this diverse literature in a way that “enables new theoretical frameworks and australasian journal of information systems xu et al. 2024, vol 28, research article using analytical information for digital business transformation 8 perspectives to emerge” (snyder, 2019, p. 335). accordingly, the aim of our integrative literature review is to build a novel conceptualisation linking dataops with digital business transformation through the lens of oipt (snyder, 2019). 3.1 data collection the purpose of our data collection is to combine perspectives from different fields rather than cover all published articles on a topic, which helps expand on the theoretical foundation of a specific topic (snyder, 2019). we first searched literature from popular literature databases, including science direct, ais electronic library, scopus, and google scholar by using a combination of key words such as digital business transformation, analytical information processing capability, and dataops. our search terms are listed in appendix a. we selected a timeframe to include literature published between 2015 and 2023, with 2015 being used as the earliest date, given that the dataops term was only popularised by palmer (2015) in that year. the search results for the different search terms across the databases are listed in appendix b. as the aim of this integrative review is not to comprehensively cover as many publications as possible, we sorted the search results by relevance and went through the first 200 results for each search to focus on the most relevant works in the field (vom brocke et al., 2015). we examined the title and abstract of the articles to exclude papers that were duplicates or deemed not relevant to our research question (i.e., how do organisations use analytical information for digital business transformation through dataops?). we also excluded papers published in journals that are not ranked in either australian business deans council journal quality list or scimago. this screening process resulted in the selection of 301 articles from the literature databases for the full-text eligibility assessment. notably, as dataops emerged from practice, we also searched 178 articles from practitioner research and advisory firms such as gartner, datakitchen and the eckerson group to identify articles that provided us with insights into how dataops is used in practice in the context of using analytical information for digital business transformation. in an effort to broaden the search, we noted cited works of potential interest in the articles reviewed and identified 30 articles as per webster and watson (2002) for eligibility assessment. as a result, there were 509 articles in total for full-text eligibility assessment. in the eligibility assessment stage, we read the full text of each article and omitted articles for the following reasons: a) papers focused on the technical side of analytics and dataops rather than the usage of analytical information for digital business transformation; b) papers that mentioned digital transformation in passing, such as growing trend type papers; c) papers in which data operation(s) refer to the general operation of data rather than the dataops method that learns from devops, agile, total quality management, and lean method. the eligibility assessment resulted in a total of 87 articles for in-depth review. the data collection process and the exclusion criteria for the screening and eligibility process (in the dashed boxes) are shown in figure 1. australasian journal of information systems xu et al. 2024, vol 28, research article using analytical information for digital business transformation 9 figure 1. data collection (literature search) process – adapted from page et al. (2021) 3.2 data analysis we synthesized, analysed and integrated the literature using oipt by following watson and webster (2020)’s concept-centric approach. specifically, we first reviewed the literature to get an in-depth understanding of the key concepts (e.g., dataops, analytical information processing capability, boundary spanning, and digital business transformation) included in the study to lay the foundation of the framework building. next, guided by oipt, we analysed the literature to identify the relationship between these concepts. we first identified the uncertainties in digital transformation and what analytical information can be provided by analytical information processing capabilities for digital business transformation. then we focused on the boundary spanning activities needed in digital business transformation. we linked dataops research with digital transformation by focusing on the principles and practices from dataops that can help in using the analytical information processing capability and facilitate the boundary spanning activities for digital business transformation. we classified the literature and integrated the review into the paths shown in figure 2. the list of 87 articles that were reviewed for this study is provided in appendix d. australasian journal of information systems xu et al. 2024, vol 28, research article using analytical information for digital business transformation 10 4 digital business transformation through dataops: a conceptual framework based on our synthesis, analysis, and integration of the literature, we develop a conceptual framework that links digital business transformation and dataops through the lens of oipt (see figure 2). the dynamic digital environment brings uncertainties to organisations’ digital business transformations. dataops provides organisations with an integrated and disciplined approach to developing their analytical information processing capabilities and facilitating boundary spanning activities to address the uncertainties in digital business transformation. boundary spanning activities mediate the relationship between analytical information processing capability and digital business transformation by enabling the analytical information to be shared and exchanged across the whole enterprise for digital business transformation. table 3 gives definitions of the concepts in this framework. below, we explain the framework by deriving propositions about how analytical information can be leveraged for digital business transformation using dataops. figure 2. using analytical information for digital business transformation through dataops concept definition exemplar references dynamic business environment a dynamic environment consisting of disruptive digital technologies, new competitors, and changing customers’ expectations. (vial, 2019; warner & wäger, 2019) dataops an integrated and disciplined approach to developing analytical information processing capability and facilitating boundary spanning activities for digital business transformation. (heudecker et al., 2020; naseer et al., 2020) analytical information processing capability the ability to use business analytics technologies or applications that analyse critical business data to better understand business and market and make timely business decisions. (cao et al., 2019; saldanha et al., 2017) boundary spanning activities the activities or practices to increase the communication and coordination between different stakeholders involved in digital business transformation. (schotter et al., 2017) digital business transformation an ongoing process where an organisation strategically transforms its business dimensions (e.g., value propositions, critical business operations, organisational routines and structures, and management) to radically improve the business enabled by digital technologies. (vial, 2019; warner & wäger, 2019) table 3. definitions of key concepts in the proposed conceptual framework australasian journal of information systems xu et al. 2024, vol 28, research article using analytical information for digital business transformation 11 4.1 dynamic business environment the business environment in this digital age is becoming more dynamic than before. table 4 summarizes the dynamics of the business environment. organisations operate in a dynamic digital environment with opportunities and uncertainties. on the one hand, opportunities to transform a business exist within the digital environment, given the rapidly evolving nature of digital technologies. artificial intelligence is one example where organisations can use it to engage customers and employees and deliver new products and services (borges et al., 2021). 3d printing is another example of where customized products can be produced and manufacturing a range of goods using the same resources can be made possible (frank et al., 2019). on the other hand, the environmental dynamism is intensified given the advancement and disruption of novel digital technologies (gupta & bose, 2022), thereby increasing uncertainty, characterized by the entrance of new competitors and changing expectations of customers (warner & wäger, 2019). taking a market competition perspective, disruptive technologies impact the existing market by creating a new market in which entrants would always win because of the established firms’ delay in making the strategic commitment to enter the emerging market (christensen, 2013). facing the threat of new competitors, organisations have an imperative to start their digital business transformation journeys to better compete in the market. further, disruptive digital technologies such as smartphones also play decisive roles in shaping and mediating all dimensions of people’s lived experiences (yoo, 2010), creating change in customers’ expectations and requirements (schallmo et al., 2017; vial, 2019). facing uncertainties, organisations require analytical information to understand the market and their competitors as well as customers (sia et al., 2016). according to oipt (daft & lengel, 1986; flynn & flynn, 1999), organisations should develop their analytical information processing capability to match their information needs for digital business transformation. thus, we propose the following: p1: a dynamic and uncertain business environment triggers the need to develop analytical information processing capability. dynamics of the business environment description exemplar references disruptive digital technologies disruptive digital technologies are creating far-reaching effects on business (ashrafi et al., 2019; matt et al., 2015; verhoef et al., 2021; vial, 2019; warner & wäger, 2019) changing customers’ requirements and expectations digital technologies (e.g., mobile, and social media) have a profound impact on customers’ behaviour and thus influence their expectations and requirements of services and products (ashrafi et al., 2019; ismail et al., 2017; schallmo et al., 2017; sia et al., 2016; vial, 2019; warner & wäger, 2019) competitive landscape digital technologies facilitate the generation of new forms of digital offerings and new disruptive competitors, which moves the competition from a physical plane to a virtual plane (ismail et al., 2017; sia et al., 2016; verhoef et al., 2021; vial, 2019; warner & wäger, 2019) table 4. dynamics of the business environment australasian journal of information systems xu et al. 2024, vol 28, research article using analytical information for digital business transformation 12 the uncertain environment encourages team members to broaden their attempts to collect information from outside of the team and so turn their focus externally, scanning for information (ancona & caldwell, 1990; gupta & bose, 2022). boundary spanning activities allow boundary spanners to search, acquire, and convey relevant information about the environment to the internal organisation to reduce uncertainty (xue et al., 2022). according to leifer and delbecq (1978), when the perceived environmental uncertainty is high and the information need is unanticipated and irregular, the boundary spanning actives will be nonregulated and nonroutine. therefore, with the increasing uncertainties in the dynamic business environment, organisations need to have more nonregulated and nonroutine boundary spanning activities as an important mechanism to cope with the complexity of the dynamic business environment (ancona & caldwell, 1990). accordingly, the following proposition is developed: p2: a dynamic and uncertain digital business environment triggers the need to have boundary spanning activities. 4.2 employing dataops for digital business transformation organisations can leverage analytical information for digital business transformation, however, unpacking the value of such information requires an integrated approach. dataops can provide this to help organisations unite stakeholders, govern the flow of data, and ensure that the insights from analytics can satisfy the digital business transformation needs in time. just as devops contributes to building collaboration between software development and it operations (munappy, mattos, et al., 2020), dataops unites data stakeholders such as data engineers, data analysts, it operations, and business users, around business requirements to achieve digital business transformation (ereth & eckerson, 2018; munappy, bosch, et al., 2020). dataops verifies that results at each intermediate step in the production of analytics match business requirements, borrowing the process from total quality management in using statistical measurement to monitor and control manufacturing processes (bergh et al., 2019). to identify waste and manage the flow of data, lean thinking can be used to deliver analytical solutions as quickly as possible (atwal, 2020). moreover, dataops can discover common ground where various stakeholders and technologies can act in concert (e.g., orchestration tools that enable automatic combinations of various technologies) (ereth & eckerson, 2018; richardson, 2020). last but not least, guinan et al. (2019)’s research showed that organisations navigated digital business transformation through the agile method to improve the speed of adapting and responding to the dynamic business environment. by adopting agile and welcoming changing requirements, dataops embraces changes and enables organisations to cope with the dynamic digital environment when using analytics (atwal, 2020). overall, learning from these mature methods of software engineering and manufacturing, dataops makes it possible for organisations to sense internal and external changes, improve decisionmaking effectiveness, and reorganise the resources to remain competitive (ereth, 2018; gur et al., 2022). dataops is not only a method for analytical product development but also a discipline that integrates people, processes, technology, and data for digital business transformation. therefore, we propose: p3: dataops provides an integrated and disciplined approach for organisations to leverage analytical information for digital business transformation. the analytical information processing capability in our framework refers to the ability to use analytical practices and applications to process data – that is, to capture, store, transform, australasian journal of information systems xu et al. 2024, vol 28, research article using analytical information for digital business transformation 13 analyse, and visualise it, and generate insights for digital business transformation (cao et al., 2019; saldanha et al., 2017). oipt posits that organisations should design their structures, mechanisms, and business processes in such a manner that facilitates information processing and thereby enables informed decision-making and improved organisational performance (flynn & flynn, 1999; galbraith, 1974; premkumar et al., 2005). therefore, to build analytical information processing capability, organisations need to leverage tools, techniques, people, and processes in analytics to meet the above goal (srinivasan & swink, 2018). during the literature review process, we identified the relevant dataops principles and practices that enable the development of analytical information processing capabilities by integrating the analytical resources such as people, processes, tools, and techniques (aslett, 2020; ereth, 2018; naseer et al., 2020). from a people’s perspective, dataops champions the business value mindset that data is not an end in itself but should deliver insights that add value to the business (atwal, 2020; bahaa et al., 2023; ereth & eckerson, 2018). by applying this mindset, each stakeholder involved views data quality as a top priority to deliver analytical insights that satisfy the needs of digital business information. moreover, teams in dataops are expected to self-organized to meet goals (atwal, 2020). self-organized teams can help to create self-contained tasks which help to reduce the amount of analytical information processed and thereby reduce the uncertainty (galbraith, 1974). from a processes’ perspective, emphasizing the importance of testing and monitoring at every stage in data processing is key to performing dataops, which ensures that the analytical information delivered satisfies the needs of digital business transformation. issues can be identified quickly through testing before they are delivered to the business and become hard to fix (mainali et al., 2021). monitoring the performance of data movement processes is also important as it can detect unexpected patterns and problems (ereth & eckerson, 2018; sahoo & premchand, 2019). moreover, a key pillar in dataops is to continuously improve (eckerson, 2019a), where teams learn from their mistakes and review processes continuously to adapt to the changing environment (heudecker et al., 2020; rodriguez et al., 2020). such an iterative process with incremental improvements ensures that the analytical information that organisations extract is high-quality, thus promoting digital business transformation in a sustainable manner. from a tools and technologies perspective, wherever possible, dataops uses technologies to automate (ereth, 2018), which can improve the reliability of the analytical information processing capability easing reliance on human intervention and thus reducing the time required for data processing. with appropriate levels of governance and metadata, the automated analytical information delivery also helps to improve the use and value of analytical information in a dynamic environment (heudecker et al., 2020). overall, dataops can be a catalyst for organisational and cultural shifts (wells, 2019), as it emphasizes the notion that processing data is about more than simply the technologies. instead, all resources, including people, technologies, and processes, need to be combined and orchestrated (friedman & heudecker, 2019; wells, 2019) for data processing capabilities and delivering the needed analytical information for digital business transformation. accordingly, we propose: p4: dataops enables the integration of analytical resources such as people, processes, tools, and technologies needed to develop the analytical information processing capability. australasian journal of information systems xu et al. 2024, vol 28, research article using analytical information for digital business transformation 14 dataops also helps facilitate the boundary spanning activities when organisations leverage business analytics for digital business transformation in the following ways: leveraging boundary spanning objects: dataops provides boundary spanning objects, such as version control systems and code repositories, fostering parallel development and code reuse for analytics (ereth & eckerson, 2018). like the agile method, dataops is a highly iterative approach in which prototypes of analytics products/solutions (as boundary spanning objects) are provided to gather feedback across different domains (ereth & eckerson, 2018). the adoption and usage of these boundary spanning tools enables stakeholders to better align and integrate their knowledge domains (beckett, 2021; pershina et al., 2019). promoting cross-functional coordination: based on beckett (2021), a multidisciplinary agile team with a clear collaboration and knowledge-sharing mindset facilitates boundary spanning. learning from the agile and devops method, dataops requires a cross-functional team consisting of people from the it team, data team, and business team (ereth & eckerson, 2018; walsh, 2023). also, dataops values cross-functional ownership over siloed responsibility (bergh et al., 2019). teams in dataops should be organized around shared datacentric goals to eliminate barriers (atwal, 2020). rearranging business functions by creating such a cross-functional team with shared goals brings everyone involved together, helping stakeholders to interact and coordinate more efficiently (eckerson, 2019b), thereby promoting boundary spanning across different domains. enriching communication and collaboration mechanisms: dataops provides both informal and formal mechanisms for cross-functional teams’ communications and collaboration. for example, communities of interest can be established to share knowledge, tools, and practices (atwal, 2020). teams can also be linked by a formal hub and spoke model through a central team responsible for harmonizing best practices and consistency across other teams (atwal, 2020). these formal and informal mechanisms enable knowledge sharing and improve communications within and between teams (ereth & eckerson, 2018), which facilitates both the scouting and task coordinating boundary spanning activities. encouraging continuous learning: dataops encourages continuous learning through iterations (heudecker et al., 2020). each iteration generates feedback, which better informs the next iteration, creating a loop that is critical to improving results. it also helps to publicize the early benefits of adopting dataops and thus creates curiosity and excitement among other teams (atwal, 2020). in this sense, the adoption of dataops in one team can influence other business teams to learn and change their ways of using business analytics for digital business transformation, thereby creating a standardized ways of using business analytics for digital business transformation across an organisation. therefore, we propose: p5: dataops enables boundary spanning activities for digital business transformation by leveraging boundary spanning activities, promoting cross functional coordination, enriching communication, and collaboration mechanisms, and encouraging continuous learning. 4.3 analytical information processing capability, boundary spanning activities, and their interaction for digital business transformation the existing literature conceptualised, explained, or gave examples of the analytical information processing capability, analytical information, and boundary spanning activities in the context of digital business transformation differently (summarised in appendix c). below, we explain in detail how analytical information processing capability and boundary australasian journal of information systems xu et al. 2024, vol 28, research article using analytical information for digital business transformation 15 spanning activities play a role in digital business transformation by synthesizing and analysing the literature. in a dynamic business environment, organisations may lack information for digital business transformation. for example, since the diffusion of digital technologies can change swiftly, organisations may not have enough information to make reliable assumptions within their organisational digital business transformation strategies (matt et al., 2015). further, it can be challenging to obtain sufficient information to predict how the market will change in line with customers’ varying expectations and requirements, especially given the rate at which borndigital firms enter the market (ismail et al., 2017). moreover, many complexities exist in internal operations, such as the tension between exploiting the existing business while also exploring new digital initiatives that are compatible with the pact dependencies of the past (warner & wäger, 2019). it is within these types of complexities that organisations may not have adequate information to cope effectively, impacting the digital business transformation process. from a oipt perspective, when the information process capability satisfies the information needs, it brings the effectiveness in the tasks for organisations (tushman & nadler, 1978). facing uncertainties in digital business transformation requires extensive useful information, including market trends, customers’ expectations, the performance of business operations, and stakeholders’ sentiments on the cultural shift towards digital business transformation (ismail et al., 2017; loebbecke & picot, 2015). to significantly improve the amount and richness of information used in digital business transformation, analytical information processing capabilities can be utilised to generate actionable insights to satisfy the analytical information needs (ashrafi et al., 2019; dremel et al., 2017; vidgen et al., 2017; wee et al., 2022). for example, organisations can get analytical information about new customer-centric trends that are hard for strategic planners to predict (warner & wäger, 2019). conboy et al. (2020) and sebastian et al. (2017) also found that organisations can extract rich analytical information, such as the sentiment of customers and staff and predicted defect rates, to understand customers and competitors and identify internal inefficiencies that need to be addressed through transformation. hence, the following proposition is derived: p6: analytical information processing capability provides the analytical information needed to address uncertainties and guide digital business transformation. analytical information processing capabilities also play a role in facilitating boundary spanning activities. first, analytical information processing capabilities help to measure and monitor boundary spanning activities. for instance, in schwade (2021)’s research, a dashboard monitoring a boundary spanning network is developed to facilitate and interpret boundary spanning activities. second, given the explosive growth in data generated in this digital age, organisations may face information overload when managing, exploiting, and disseminating information across organisational boundaries (aldrich & herker, 1977; fleischer & carstens, 2021). analytical information processing capabilities can help to reduce information overload for boundary spanning, as evidenced in song et al. (2021)’s research on the usage of analyticsembedded digital platforms to analyse relevant data, which enabled decision makers to find the targeted information, make targeted decisions, and effectively organize business activities. third, analytical information processing capabilities may reshape the organisational boundaries. internally, the existing boundary setting will be reshaped based on data and australasian journal of information systems xu et al. 2024, vol 28, research article using analytical information for digital business transformation 16 analytics accessibility (jonsson et al., 2009). externally, analytical information processing capabilities deliver analytical information that alters organisations’ understanding of its external environment, which facilitates new boundary spanning forms that can interact with the environment (conboy et al., 2020). for instance, supported by analytics-enabled capabilities to monitor and analyse internal and external data, a firm can sense threats and trends in the environment and adapt as necessary. further, the organisation can provide aggregated information to their customers and other companies under an alliance, which creates new boundary spanning forms in which the organisation interacts with their external stakeholders (conboy et al., 2020). taken together, the following proposition is developed: p7: analytical information processing capabilities facilitate boundary spanning activities by monitoring the boundary spanning network, reducing information overload, and reshaping boundaries. as a strategic imperative for organisations, digital business transformation requires boundary spanning activities to extend the benefits beyond the focal team itself to the performance of other parties and to the achievement of higher-order organisational and cross-organisational goals (marrone, 2010). boundary spanning activities improve coordination effectiveness, scale analytics applications across the organisation, promote innovation, and foster a digital ecosystem for digital business transformation. below, we explain how boundary spanning activities play a key role in digital business transformation. improve coordination effectiveness: increased collaboration, both internally and externally, is integral to organisations’ digital business transformation (dremel et al., 2017). according to azzouz and papadonikolaki (2020), boundary spanning activities at all frequencies (e.g., project, iteration, and ad-hoc), their associated boundary spanning artefacts, and a coordinator role increase explicit coordination effectiveness. for example, singh et al. (2020) explain the importance of the chief digital officer (cdo) and its coordinator role in spanning the vertical boundary and horizontal boundary through formal (e.g., board meetings) and informal coordination mechanisms (e.g., webinars). these mechanisms performed by cdo facilitate effective cross-unit relationship-building and achieve efficient coordination across an organisation. dremel et al. (2017) explained that solving the data ownership issues and data transparency issues (as a task coordinating activity) helps to embrace operational transparency and grant access to the data, which fosters a data sharing culture and improves the coordination of business functions across the firm when leveraging operational data for digital business transformation. scale analytics applications across the organisation: someh and shanks (2013) explain how the success of analytics initiatives in one functional area of an organisation may influence other functional areas to change their values and norms and therefore adopt analytics. this assists with scaling analytics-enabled digital transformation initiatives across the entire organisation. lighthouse projects and proof-of-concept are valid devices for familiarizing the business with the opportunities arising from digitization and operational business data (dremel et al., 2017). moreover, the embeddedness as a boundary spanning activity helps to establish social ties across the organisation based on the analytics-driven culture (someh & shanks, 2013). dremel et al. (2017) exemplify how embedding analytics as an internal competence, cutting across individual it units and other organisational entities, enables the firm to leverage analytics as a service and evidence-based decision-making, thus assisting digital business transformation. australasian journal of information systems xu et al. 2024, vol 28, research article using analytical information for digital business transformation 17 promote innovation: most research on digital business transformation acknowledges the need for firms to engage with other parties to generate digital innovation (vial, 2019), which always occurs in the context of a community of cooperatives (harter & krone, 2001). according to glaser et al. (2015), a potential key driver of exploratory innovation is boundary spanning, with activities such as scouting helping to reduce knowledge gaps and promote digital innovation for digital business transformation. searching for new knowledge across organisational and technical boundaries brings in novel and heterogeneous knowledge to accelerate technological advancements (yang et al., 2021). by creatively combining a diverse set of knowledge, digital business transformation occurs by prompting the emergence of novel ideas, services and products (pershina et al., 2019). foster a digital ecosystem: the collaboration and interaction of a firm with its stakeholders (e.g., customers, suppliers, and partners) in the digital transformation journey result in a digital ecosystem (pappas et al., 2018). such ecosystems provide opportunities for organisations to transform their business and create value by engaging with their customers and partners (suseno et al., 2018). the analytics embeddedness boundary spanning activity can help to establish a digital ecosystem necessary for digital business transformation. for example, by embedding analytics into operations, a firm can better interact with its customers and co-create value with customers (gupta et al., 2020). accordingly, we formulate the following proposition: p8: boundary spanning activities enable digital business transformation by improving coordination effectiveness, scaling the analytics across the whole organisation, promoting innovation, and fostering a digital ecosystem. although analytical information processing capability generates the needed analytical information that supports organisations’ digital business transformations, boundary spanning activities elevate the analytical information usage and enable organisations to integrate, exchange, and share the analytical information for digital business transformation. by integrating, exchanging, and sharing the analytical information across the internal and external organisational boundaries, a network effect can be created to improve communication and coordination in the transformation (khuntia et al., 2022; papanagnou et al., 2022). based on aldrich and herker (1977), information needs to be summarized and directed to the organisational unit that needs it. it is important for boundary spanning roles to summarize and interpret information and mediate the flow of information between relevant actors in a focal organisational unit and its task environment (aldrich & herker, 1977), making each relevant unit get its needed analytical information. according to beckett (2021), boundary spanning activities facilitate the flow of information across different kinds of boundaries, including geographical, cultural, and organisational. by having boundary spanning activities such as embedding analytical experts into a hybrid team and making analytics reports available to all employees, stakeholders across boundaries can better share and exchange their knowledge and analytical information. this enables the entire organisation to embrace analytics as the key driver for the transformation (tim et al., 2020). therefore, we propose: p9: boundary spanning activities foster analytical information usage and sharing and thereby mediate the relationship between analytical information processing capability and digital business transformation. australasian journal of information systems xu et al. 2024, vol 28, research article using analytical information for digital business transformation 18 5 discussion the purpose of this study is to answer the following research question: how do organisations use analytical information for digital business transformation through dataops? by analysing and synthesizing the literature through the lens of oipt, we develop a conceptual framework that explains how dataops helps to enable the analytical information processing capability and facilitate boundary spanning activities required for digital business transformation. below, we discuss the implications of our study for research and practice. 5.1 implications for research by integrating the diverse literature on digital business transformation, dataops and business analytics through oipt, our study formulates a new topic of investigation (cronin & george, 2023; torraco, 2016), that is, using analytical information for digital business transformation through dataops, and thereby contributes to research on data-driven digital business transformation. our framework links business analytics and dataops with digital business transformation, which extends the current knowledge on digital business transformation by arguing dataops as a disciplined approach to enabling organisations to use analytical information to achieve such transformation. organisations need to develop their capabilities to realize the benefit of business analytics on digital business transformation (setia et al., 2014). our research contributes to this by proposing analytical information processing capability as the requirement and introducing dataops as the discipline to build the analytical information processing capability. according to ranjan and foropon (2021), business analytics is considered a vital information processing mechanism to reduce uncertainty and equivocality in decision-making processes. other analytical information processing capabilities have been identified by anand et al. (2020), such as online analytic processing (olap), automated and ad-hoc reporting, social media analytics, and sentiment analytics. our research extends these studies by explaining the importance of analytical information processing capabilities for digital business transformation and proposes a disciplined approach to develop such capabilities. one of the key features of digital business transformation is its broader scope. digital business transformation needs to include various business functions and customers across all valueadded chain segments (schallmo et al., 2017), and as such, digital transformation can be significant and have implications beyond the organisation’s immediate value network (vial, 2019). our research extends these studies by emphasizing the importance of boundary spanning activities in digital business transformation. our study explains what boundary spanning activities are needed and how these boundary spanning activities contribute to digital business transformation. boundary spanning activities such as task coordinating, scouting, embedding, and influencing help to improve coordination effectiveness, scale analytics applications, promote innovations, and foster a digital ecosystem for digital business transformation. further, boundary spanning activities allow the analytical information to be shared and exchanged across the organisation, enhancing the value of business analytics for digital business transformation. this paper also contributes to research on dataops by linking it with digital business transformation through oipt and boundary spanning activities perspective. based on heudecker et al. (2020), the real benefit of dataops is as a lever for organisational change. our research aligns with this view of dataops and indicates that dataops enables organisations to develop their analytical information processing capabilities for digital business australasian journal of information systems xu et al. 2024, vol 28, research article using analytical information for digital business transformation 19 transformation. the principles and practices of dataops, such as testing and monitoring, automation, continuous improvement, and business value mindset, help to align analytical people, processes, and technologies to generate high-quality analytical information needed in the digital business transformation. dataops also helps to foster boundary spanning activities for digital business transformation by providing the boundary spanning objects (e.g., code repositories) and communication and collaboration mechanisms (e.g., communities of interests). dataops is not strictly a technical competency (heudecker et al., 2020). instead, dataops is a discipline that helps organisations leverage business analytics and maximizes the value of business analytics for digital business transformation. 5.2 implications for practice first, this research highlights the important role of analytical information in digital business transformation. by using analytics practices and applications, the numerous data generated in the digital world can be transformed into useful analytical information, solving the uncertainties in digital business transformation. therefore, organisations need to realize the importance and necessity of analytical information and use it throughout their digital business transformation journey. more importantly, rather than viewing analytics simply as a technology, organisations need to build analytical information processing capability that aligns people, processes, and technologies for digital business transformation. second, this research provides organisations with ways to manage external and internal boundaries (e.g., the boundary of different business functions) and improve collaboration for digital business transformation. for example, using dataops, a cross-functional team across the business, it, and analytics can be established. people in this cross-functional team can act as boundary spanners who connect their original teams with other teams across the organisation. as the joint use of assets or combining resources in a company is valueenhancing (cao et al., 2019), organisations can adopt the practices from dataops to foster the joint use of analytical resources across the organisation to drive digital business transformation. third, given the potential role of dataops in fostering the boundary spanning activities for digital business transformation, business analytics vendors need to consider the different needs of users and develop dataops tools accordingly. in other words, dataops tools or platforms need to provide a common language that can be understood by people from different backgrounds. dataops tools should allow different users to work collaboratively to improve the efficiency of teamwork in digital business transformation with business analytics. 6 conclusions, limitations, and future research an increasing number of organisations are embarking on their digital business transformation journeys. however, transforming a business digitally and getting value from it is challenging (davenport & westerman, 2018). in this study, we develop a conceptual framework that explains how organisations can transform their business by leveraging analytical information through dataops. specifically, dataops provides principles and practices that help organisations develop their analytical information processing capabilities and facilitate boundary spanning activities to share and exchange analytical information for digital business transformation. australasian journal of information systems xu et al. 2024, vol 28, research article using analytical information for digital business transformation 20 although our research extends the current knowledge on digital transformation by bringing in business analytics and dataops for digital business transformation through the lens of oipt, this research has a few limitations that offer opportunities for future research. first, we did not explore the specific analytical information processing capabilities needed for digital business transformation. future research can explore the role of different types of analytical information processing capabilities, such as descriptive, diagnostic, predictive, and prescriptive analytics (lepenioti et al., 2020). second, boundary spanning activities usually include both the activities that enable cross-boundary collaboration and activities that guard the boundary to control the release of information (ancona & caldwell, 1990). our framework focuses on the boundary spanning activities that enable cross-functional collaboration and encourage the sharing and exchanging the analytical information across the organisation. future research can explore the boundary spanning activities that protect the key analytical information to keep a firm competitive in the market. third, our framework does not include details on the implementation of dataops for digital business transformation. future research can explore the aspects (e.g., technologies, processes, and people) needed to implement dataops for digital business transformation. in addition, in this study, we only discuss the relevant dataops principles and practices that play an integral role in the development of analytical information processing capabilities. future research can focus on building a comprehensive list of dataops principles and practices in the context of digital business transformation. finally, we propose a conceptual framework using an integrative literature review. future research can further develop, refine, and test the proposed framework through in-depth case studies and validate it by conducting a large-scale survey. references aldrich, h., & herker, d. (1977). boundary spanning roles and organization structure. academy of management review, 2(2), 217-230. https://doi.org/10.5465/amr.1977.4409044 anand, a., sharma, r., & kohli, r. (2020). the effects of operational and financial performance failure on bi&a-enabled search behaviors: a theory of performancedriven search. information systems research, 31(4), 1144-1163. https://doi.org/10.1287/isre.2020.0936 ancona, d. g., & caldwell, d. (1990). beyond boundary spanningmanaging external dependence in product development teams. the journal of high technology management research, 1(2), 119-135. https://doi.org/10.1016/1047-8310(90)90001-k ashrafi, a., zare ravasan, a., trkman, p., & afshari, s. (2019). the role of business analytics capabilities in bolstering firms’ agility and performance. international journal of information management, 47, 1-15. https://doi.org/10.1016/j.ijinfomgt.2018.12.005 aslett, m. (2020). dataops unlocks the value of data. https://www.hitachivantara.com/enus/pdf/analyst-content/dataops-unlocks-value-of-data-451-research.pdf. accessed august 8, 2023 atwal, h. (2020). practical dataops: delivering agile data science at scale. berkeley, ca, usa: apress. https://link.springer.com/content/pdf/10.1007/978-1-4842-5104-1.pdf. accessed august 8, 2023 australasian journal of information systems xu et al. 2024, vol 28, research article using analytical information for digital business transformation 21 azzouz, a., & papadonikolaki, e. (2020). boundary-spanning for managing digital innovation in the aec sector. architectural engineering and design management, 16(5), 356-373. https://doi.org/10.1080/17452007.2020.1735293 bahaa, s., ghalwash, a. z., & harb, h. (2023). dataops lifecycle with a case study in healthcare. international journal of advanced computer science and applications, 14(1). https://doi.org/10.14569/ijacsa.2023.0140115 beckett, r. c. (2021). agile coping in a digital world: an expanding need for boundary spanning. in ferreira, n., potgieter, i.l., & coetzee, m. (eds.) agile coping in the digital workplace (pp. 119-145). cham, switzerland: springer. https://doi.org/10.1007/978-3-03070228-1_7 bergh, c., benghiat, g., & strod, e. (2019). the dataops cookbook: methodologies and tools that reduce analytics cycle time while improving quality. cambridge, ma, usa: datakitchen headquarters. https://www.devopsschool.com/blog/wp-content/uploads/2021/07/ dk_dataops_book_2nd_edition.pdf bonnet, d., & westerman, g. (2021). the new elements of digital transformation. mit sloan management review, 62(2), 82-89. https://www.proquest.com/scholarly-journals/newelements-digital-transformation/docview/2471816389/se-2?accountid=12372 borges, a. f. s., laurindo, f. j. b., spínola, m. m., gonçalves, r. f., & mattos, c. a. (2021). the strategic use of artificial intelligence in the digital era: systematic literature review and future research directions. international journal of information management, 57, 1-16. https://doi.org/10.1016/j.ijinfomgt.2020.102225 cao, g., duan, y., & cadden, t. (2019). the link between information processing capability and competitive advantage mediated through decision-making effectiveness. international journal of information management, 44, 121-131. https://doi.org/10.1016/j.ijinfomgt.2018.10.003 capizzi, a., distefano, s., & mazzara, m. (2020). from devops to devdataops: data management in devops processes. in software engineering aspects of continuous development and new paradigms of software production and deployment: second international workshop, devops 2019 (vol. 12055). cham, switzerland: springer. https://doi.org/10.1007/978-3-030-39306-9_4 christensen, c. m. (2013). the innovator's dilemma: when new technologies cause great firms to fail. harward university, ma, usa: harvard business review press. conboy, k., mikalef, p., dennehy, d., & krogstie, j. (2020). using business analytics to enhance dynamic capabilities in operations research: a case analysis and research agenda. european journal of operational research, 281(3), 656-672. https://doi.org/10.1016/j.ejor.2019.06.051 cronin, m. a., & george, e. (2023). the why and how of the integrative review. organizational research methods, 26(1), 168-192. https://doi.org/10.1177/1094428120935507 daft, r. l., & lengel, r. h. (1986). organizational information requirements, media richness and structural design. management science, 32(5), 554-571. https://doi.org/https://doi.org/10.1287/mnsc.32.5.554 australasian journal of information systems xu et al. 2024, vol 28, research article using analytical information for digital business transformation 22 davenport, t. h., & harris, j. (2017). competing on analytics: updated, with a new introduction: the new science of winning. harward university, ma, usa: harvard business press. davenport, t. h., & westerman, g. (2018). why so many high-profile digital transformations fail. harvard business review, 9, 1-5. https://www.nutanix.com/content/dam/nutanixcxo/pdf/why%20so%20many%20high-profile%20digital%20transformations%20 fail.pdf dremel, c., wulf, j., herterich, m. m., waizmann, j. c., & brenner, w., 16(2). (2017). how audi ag established big data analytics in its digital transformation. mis quarterly executive, 16(2), 81-100. https://aisel.aisnet.org/misqe/vol16/iss2/3 dwivedi, y. k., hughes, l., ismagilova, e., aarts, g., coombs, c., crick, t., duan, y., dwivedi, r., edwards, j., eirug, a., galanos, v., ilavarasan, p. v., janssen, m., jones, p., kar, a. k., kizgin, h., kronemann, b., lal, b., lucini, b., medaglia, r., le meunier-fitzhugh, k., le meunier-fitzhugh, l. c., misra, s., mogaji, e., sharma, s. k., singh, j. b., raghavan, v., raman, r., rana, n. p., samothrakis, s., spencer, j., tamilmani, k., tubadji, a., walton, p., & williams, m. d. (2021). artificial intelligence (ai): multidisciplinary perspectives on emerging challenges, opportunities, and agenda for research, practice and policy. international journal of information management, 57, 1-47. https://doi.org/10.1016/j.ijinfomgt.2019.08.002 eckerson, w. (2019a). trends in dataops: bringing scale and rigor to data and analytics. https://www.eckerson.com/register?content=trends-in-dataops-bringing-scale-andrigor-to-data-and-analytics. accessed august 8, 2023 eckerson, w. (2019b). the ultimate guide to dataops: production evaluation and selection criteria. https://www.eckerson.com/register?content=the-ultimate-guide-to-dataops. accessed august 8, 2023 ereth, j. (2018). dataops-towards a definition. in proceedings of the conference lernen, wissen, daten, analysen (german for learning, knowledge, data, analysis) (lwda) 2018, mannheim, germany, august 22-24, 2018 (pp. 104-112). https://ceur-ws.org/vol2191/paper13.pdf ereth, j., & eckerson, w. (2018). dataops: industrializing data and analytics strategies for streamlining the delivery of insights. https://www.eckerson.com/register?content=dataops-industrializing-data-andanalytics. accessed august 8, 2023 fleischer, j., & carstens, n. (2021). policy labs as arenas for boundary spanning: inside the digital transformation in germany. public management review, 24(8), 1208-1225. https://doi.org/10.1080/14719037.2021.1893803 flynn, b. b., & flynn, e. j. (1999). information-processing alternatives for coping with manufacturing environment complexity. decision sciences, 30(4), 1021-1052. https://doi.org/10.1111/j.1540-5915.1999.tb00917.x fox, n. j. (2011). boundary objects, social meanings and the success of new technologies. sociology, 45(1), 70-85. https://doi.org/10.1177/0038038510387196 australasian journal of information systems xu et al. 2024, vol 28, research article using analytical information for digital business transformation 23 frank, a. g., dalenogare, l. s., & ayala, n. f. (2019). industry 4.0 technologies: implementation patterns in manufacturing companies. international journal of production economics, 210, 15-26. https://doi.org/10.1016/j.ijpe.2019.01.004 friedman, t., & heudecker, n. (2019). introducing dataops into your data management discipline. https://www.gartner.com/document/3970916?ref=solrall&refval=246181733. accessed august 8, 2023 galbraith, j. r. (1974). organization design: an information processing view. interfaces, 4(3), 28-36. https://doi.org/10.1287/inte.4.3.28 gartner. (2021). 6 key takeaways from the gartner board of directors survey. https://www.gartner.com/en/articles/6-key-takeaways-from-the-gartner-board-ofdirectors-survey . accessed august 8, 2023 glaser, l., fourné, s. p. l., & elfring, t. (2015). achieving strategic renewal: the multi-level influences of top and middle managers’ boundary-spanning. small business economics, 45(2), 305-327. https://doi.org/10.1007/s11187-015-9633-5 grover, v., chiang, r. h. l., liang, t.-p., & zhang, d. (2018). creating strategic business value from big data analytics: a research framework. journal of management information systems, 35(2), 388-423. https://doi.org/10.1080/07421222.2018.1451951 guinan, p. j., parise, s., & langowitz, n. (2019). creating an innovative digital project team: levers to enable digital transformation. business horizons, 62(6), 717-727. https://doi.org/10.1016/j.bushor.2019.07.005 gupta, g., & bose, i. (2022). digital transformation in entrepreneurial firms through information exchange with operating environment. information & management, 59(3). https://doi.org/10.1016/j.im.2019.103243 gupta, s., leszkiewicz, a., kumar, v., bijmolt, t., & potapov, d. (2020). digital analytics: modeling for insights and new methods. journal of interactive marketing, 51(1), 26-43. https://doi.org/10.1016/j.intmar.2020.04.003 gur, i., moller, f., hupperz, m., uzun, d., & otto, b. (2022). requirements for dataops to foster dynamic capabilities in organizations a mixed methods approach. in 2022 ieee 24th conference on business informatics (cbi), amsterdam, netherlands, june 1517, 2022 (pp. 166-175). https://doi.org/10.1109/cbi54897.2022.00025 harter, l., & krone, k. (2001). the boundary-spanning role of a cooperative support organization: managing the paradox of stability and change in non-traditional organizations. journal of applied communication research, 29(3), 248-277. https://doi.org/10.1080/00909880128111 hess, t., matt, c., benlian, a., & wiesböck, f. (2016). options for formulating a digital transformation strategy. mis quarterly executive, 15(2), 123-139. https://aisel.aisnet.org/misqe/vol15/iss2/6 heudecker, n., friedman, t., & dayley, a. (2020). innovation insight for dataops. https://www.gartner.com/document/3896766?ref=solrall&refval=232501050&qid=. accessed agust 8, 2023 australasian journal of information systems xu et al. 2024, vol 28, research article using analytical information for digital business transformation 24 ismail, m. h., khater, m., & zaki, m. (2017). digital business transformation and strategy: what do we know so far. cambridge service alliance, 10(1), 1-36. https://cambridgeservicealliance.eng.cam.ac.uk/system/files/documents/2017novpaper _mariam.pdf jonsson, k., holmström, j., & lyytinen, k. (2009). turn to the material: remote diagnostics systems and new forms of boundary-spanning. information and organization, 19(4), 233252. https://doi.org/10.1016/j.infoandorg.2009.07.001 karippur, n. k., & balaramachandran, p. r. (2022). antecedents of effective digital leadership of enterprises in asia pacific. australasian journal of information systems, 26, 1-35. https://doi.org/ttps://doi.org/10.3127/ajis.v26i0.2525 khuntia, j., saldanha, t., kathuria, a., & tanniru, m. r. (2022). digital service flexibility: a conceptual framework and roadmap for digital business transformation. european journal of information systems, 1-19. https://doi.org/10.1080/0960085x.2022.2115410 leifer, r., & delbecq, a. (1978). organizational:environmental interchange: a model of boundary spanning activity. academy of management review, 3(1), 40-50. https://doi.org/10.5465/amr.1978.4296354 lepenioti, k., bousdekis, a., apostolou, d., & mentzas, g. (2020). prescriptive analytics: literature review and research challenges. international journal of information management, 50, 57-70. https://doi.org/10.1016/j.ijinfomgt.2019.04.003 levina, n., & vaast, e. (2014). turning a community into a market: a practice perspective on information technology use in boundary spanning. journal of management information systems, 22(4), 13-37. https://doi.org/10.2753/mis0742-1222220402 li, h., wu, y., cao, d., & wang, y. (2021). organizational mindfulness towards digital transformation as a prerequisite of information processing capability to achieve market agility. journal of business research, 122, 700-712. https://doi.org/10.1016/j.jbusres.2019.10.036 loebbecke, c., & picot, a. (2015). reflections on societal and business model transformation arising from digitization and big data analytics: a research agenda. the journal of strategic information systems, 24(3), 149-157. https://doi.org/10.1016/j.jsis.2015.08.002 mainali, k., ehrlinger, l., matskin, m., & himmelbauer, j. (2021). discovering dataops: a comprehensive review of definitions, use cases, and tools. in data analytics 2021 : the 10th international conference on data analytics, barcelona, spain, october 3-7, 2021 (pp. 61-69). https://www.researchgate.net/profile/kiran_mainali2/publication/ 355107036_discovering_dataops_a_comprehensive_review_of_definitions_use_cas es_and_tools/links/615dd703fbd5153f47e938a1/discovering-dataops-acomprehensive-review-of-definitions-use-cases-and-tools.pdf marrone, j. a. (2010). team boundary spanning: a multilevel review of past research and proposals for the future. journal of management, 36(4), 911-940. https://doi.org/10.1177/0149206309353945 matt, c., hess, t., & benlian, a. (2015). digital transformation strategies. business & information systems engineering, 57(5), 339-343. https://doi.org/10.1007/s12599-015-04015 australasian journal of information systems xu et al. 2024, vol 28, research article using analytical information for digital business transformation 25 müller, o., junglas, i., debortoli, s., & vom brocke, j. (2016). using text analytics to derive customer service management benefits from unstructured data. mis quarterly executive, 15(4), 243-258. https://doi.org/10.25300/misq/2018/13239 munappy, a. r., bosch, j., & olsson, h. h. (2020). data pipeline management in practice: challenges and opportunities. in proceedings of the product-focused software process improvement conference, turin, italy, november 25-27, 2020 (pp. 168-184). https://doi.org/10.1007/978-3-030-64148-1_11 munappy, a. r., mattos, d. i., bosch, j., olsson, h. h., & dakkak, a. (2020). from ad-hoc data analytics to dataops. in proceedings of the international conference on software and system processes, seoul, republic of korea, june 26-28, 2019 (pp. 165-174). https://doi.org/10.1145/3379177.3388909 nambisan, s., wright, m., & feldman, m. (2019). the digital transformation of innovation and entrepreneurship: progress, challenges and key themes. research policy, 48(8), 1-9. https://doi.org/10.1016/j.respol.2019.03.018 naseer, h., maynard, s. b., & desouza, k. c. (2021). demystifying analytical information processing capability: the case of cybersecurity incident response. decision support systems, 143, 1-11. https://doi.org/10.1016/j.dss.2020.113476 naseer, h., maynard, s. b., & xu, j. (2020). modernizing business analytics capability with dataops: a decision-making agility perspective. in proceedings of the european conference on information systems (ecis 2020), marrakech, morocco, june 15-17, 2020 (pp. 1-11). https://aisel.aisnet.org/ecis2020_rip/36 neirotti, p., pesce, d., & battaglia, d. (2021). algorithms for operational decision-making: an absorptive capacity perspective on the process of converting data into relevant knowledge. technological forecasting and social change, 173. https://doi.org/10.1016/j.techfore.2021.121088 page, m. j., mckenzie, j. e., bossuyt, p. m., boutron, i., hoffmann, t. c., mulrow, c. d., shamseer, l., tetzlaff, j. m., akl, e. a., brennan, s. e., chou, r., glanville, j., grimshaw, j. m., hrobjartsson, a., lalu, m. m., li, t., loder, e. w., mayo-wilson, e., mcdonald, s., mcguinness, l. a., stewart, l. a., thomas, j., tricco, a. c., welch, v. a., whiting, p., & moher, d. (2021, apr). the prisma 2020 statement: an updated guideline for reporting systematic reviews. international journal of surgery, 88, 105906. https://doi.org/10.1016/j.ijsu.2021.105906 palmer, a. (2015). from devops to dataops. in c. shannon (ed.), getting data right: tackling the challenges of big data volume and variety (pp. 49-57). sebastopol, ca, usa: o’reilly. https://cdn2.hubspot.net/hubfs/2631050/cco%20usa/getting_data_right_tamr.pdf papanagnou, c., seiler, a., spanaki, k., papadopoulos, t., & bourlakis, m. (2022). data-driven digital transformation for emergency situations: the case of the uk retail sector. international journal of production economics, 250. https://doi.org/10.1016/j.ijpe.2022.108628 pappas, i. o., mikalef, p., giannakos, m. n., krogstie, j., & lekakos, g. (2018). big data and business analytics ecosystems: paving the way towards digital transformation and sustainable societies. information systems and e-business management, 16(3), 479-491. https://doi.org/10.1007/s10257-018-0377-z australasian journal of information systems xu et al. 2024, vol 28, research article using analytical information for digital business transformation 26 pershina, r., soppe, b., & thune, t. m. (2019). bridging analog and digital expertise: crossdomain collaboration and boundary-spanning tools in the creation of digital innovation. research policy, 48(9), 1-13. https://doi.org/10.1016/j.respol.2019.103819 premkumar, g., ramamurthy, k., & saunders, c. s. (2005). information processing view of organizations: an exploratory examination of fit in the context of interorganizational relationships. journal of management information systems, 22(1), 257-294. https://doi.org/10.1080/07421222.2003.11045841 ranjan, j., & foropon, c. (2021). big data analytics in building the competitive intelligence of organizations. international journal of information management, 56, 1-13. https://doi.org/10.1016/j.ijinfomgt.2020.102231 richardson, g. (2020). to build or buy? effective dataops in an era of rapid change. database and network journal, 50(1), 3-5. https://link.gale.com/apps/doc/a619013513/aone?u=unimelb&sid=googlescholar&xid =fb06b7f3 rodriguez, m., de araújo, l. j. p., & mazzara, m. (2020). good practices for the adoption of dataops in the software industry. in journal of physics: conference series (vol. 1694, pp. 012-032). innopolis, russia: iop publishing. https://doi.org/10.1088/1742-6596/1694/1/012032 sahoo, p. r., & premchand, a. (2019). dataops in manufacturing and utilities industries. international journal of applied information systems, 12(23), 1-6. https://doi.org/10.5120/ijais2019451814 saldanha, t. j. v. v., mithas, s., & krishnan, m. s. (2017). leveraging customer involvement for fueling innovation: the role of relational and analytical information processing capabilities. mis quarterly, 41(1), 267-286. https://www.jstor.org/stable/26629647 schallmo, d., williams, c. a., & boardman, l. (2017). digital transformation of business models — best practice, enablers, and roadmap. international journal of innovation management, 21(08), 1-17. https://doi.org/10.1142/s136391961740014x schotter, a. p. j., mudambi, r., doz, y. l., & gaur, a. (2017). boundary spanning in global organizations. journal of management studies, 54(4), 403-421. https://doi.org/10.1111/joms.12256 schwade, f. (2021). measuring and visualising boundary spanningin enterprise collanoration systems. in proceedings of the european conference on information systems (ecis 2021), marrakech, morocco, june 14-16, 2021. (pp. 1-16). https://aisel.aisnet.org/ecis2021_rp/63 sebastian, i., ross, j., beath, c., mocker, m., moloney, k., & fonstad, n. (2017). how big old companies navigate digital transformation. mis quaterly executive, 16(3), 197-213. https://aisel.aisnet.org/misqe/vol16/iss3/6 setia, p., setia, p., venkatesh, & joglekar, s. (2014). leveraging digital technologies: how information quality leads to localized capabilities and customer service performance. mis quarterly, 13(2), 565-590. https://www.jstor.org/stable/43825923 sia, s. k., soh, c., & weill, p. (2016). how dbs bank pursued a digital business strategy. mis quarterly executive, 15(2), 105-121. https://aisel.aisnet.org/misqe/vol15/iss2/4 australasian journal of information systems xu et al. 2024, vol 28, research article using analytical information for digital business transformation 27 singh, a., klarner, p., & hess, t. (2020). how do chief digital officers pursue digital transformation activities? the role of organization design parameters. long range planning, 53(3), 1-14. https://doi.org/10.1016/j.lrp.2019.07.001 snyder, h. (2019). literature review as a research methodology: an overview and guidelines. journal of business research, 104, 333-339. https://doi.org/10.1016/j.jbusres.2019.07.039 someh, i. a., & shanks, g. (2013). the role of synergy in achieving value from business analytics systems. in proceedings of the international conference on information systems (icis), milan, italy, december 15-18, 2013 (pp. 1-15). https://aisel.aisnet.org/icis2013/proceedings/knowledgemanagement/7 song, h., li, m., & yu, k. (2021). big data analytics in digital platforms: how do financial service providers customise supply chain finance? international journal of operations & production management, 41(4), 410-435. https://doi.org/10.1108/ijopm-07-2020-0485 srinivasan, r., & swink, m. (2018). an investigation of visibility and flexibility as complements to supply chain analytics: an organizational information processing theory perspective. production and operations management, 27(10), 1849-1867. https://doi.org/10.1111/poms.12746 subramaniam, m. (2021). the 4 tiers of digital transformation. harvard business review. https://hbr.org/2021/09/the-4-tiers-of-digital-transformation. accessed may 14, 2023 suseno, y., laurell, c., & sick, n. (2018). assessing value creation in digital innovation ecosystems: a social media analytics approach. the journal of strategic information systems, 27(4), 335-349. https://doi.org/10.1016/j.jsis.2018.09.004 tan, f. t. c., ondrus, j., tan, b., & oh, j. (2020). digital transformation of business ecosystems: evidence from the korean pop industry. information systems journal, 30(5), 866-898. https://doi.org/10.1111/isj.12285 thusoo, a., & sarma, j. (2017). creating a data-driven enterprise with dataops. sebastopol, ca, usa: o’reilly. https://www.oreilly.com/library/view/creating-a-data-driven/9781492049227/. accessed august 8, 2023 tim, y., hallikainen, p., pan, s. l., & tamm, t. (2020). actualizing business analytics for organizational transformation: a case study of rovio entertainment. european journal of operational research, 281(3), 642-655. https://doi.org/10.1016/j.ejor.2018.11.074 torraco, r. j. (2016). writing integrative literature reviews. human resource development review, 15(4), 404-428. https://doi.org/10.1177/1534484316671606 tushman, m. l., & nadler, d. a. (1978). information processing as an integrating concept in organizational design. academy of management review, 3(3), 613-624. https://doi.org/10.5465/amr.1978.4305791 verhoef, p. c., broekhuizen, t., bart, y., bhattacharya, a., qi dong, j., fabian, n., & haenlein, m. (2021). digital transformation: a multidisciplinary reflection and research agenda. journal of business research, 122, 889-901. https://doi.org/10.1016/j.jbusres.2019.09.022 australasian journal of information systems xu et al. 2024, vol 28, research article using analytical information for digital business transformation 28 vial, g. (2019). understanding digital transformation: a review and a research agenda. the journal of strategic information systems, 28(2), 118-144. https://doi.org/10.1016/j.jsis.2019.01.003 vidgen, r., shaw, s., & grant, d. b. (2017). management challenges in creating value from business analytics. european journal of operational research, 261(2), 626-639. https://doi.org/10.1016/j.ejor.2017.02.023 vilvovsky, s. (2009). internal and external boundary spanning in outsourced is development projects: opening the black box. in proceedings of the americas conference on information systems (amcis), amcis 2009 doctoral consortium, san francisco, ca, usa (pp. 1-11). https://aisel.aisnet.org/amcis2009_dc/4 vom brocke, j., simons, a., riemer, k., niehaves, b., plattfaut, r., & cleven, a. (2015). standing on the shoulders of giants: challenges and recommendations of literature search in information systems research. communications of the association for information systems, 37. https://doi.org/10.17705/1cais.03709 walsh, b. (2023). ai best practice and dataops. in c. j. suresh, l. berendson, & m. powers (eds.), productionizing ai (pp. 41-74). apress, berkley, ca, usa. https://doi.org/10.1007/978-1-4842-8817-7_2 warner, k. s. r., & wäger, m. (2019). building dynamic capabilities for digital transformation: an ongoing process of strategic renewal. long range planning, 52(3), 326-349. https://doi.org/10.1016/j.lrp.2018.12.001 watson, r. t., & webster, j. (2020). analysing the past to prepare for the future: writing a literature review a roadmap for release 2.0. journal of decision systems, 29(3), 129-147. https://doi.org/10.1080/12460125.2020.1798591 webster, j., & watson, r. t. (2002). analyzing the past to prepare for the future: writing a literature review. mis quarterly, 26(2), xiii-xxiii. https://www.jstor.org/stable/4132319 wee, m., scheepers, h., & tian, x. (2022). understanding the processes of how small and medium enterprises derive value from business intelligence and analytics. australasian journal of information systems, 26, 1-26. https://doi.org/10.3127/ajis.v26i0.2969 wells, d. (2019). intelligent data operations: the next wave in smart data ecosystems. https://www.eckerson.com/register?content=intelligent-data-operations-the-next-wavein-smart-data-ecosystems . accessed august 8, 2023 wixom, b. h., yen, b., & relich, m. (2013). maximizing value from business analytics. mis quarterly executive, 12(2), 111-123. https://aisel.aisnet.org/misqe/vol12/iss2/6 xue, f., zhao, x., tan, y., & xie, l. (2022). digital transformation of manufacturing enterprises: an empirical study on the relationships between digital transformation, boundary spanning, and sustainable competitive advantage. discrete dynamics in nature and society, 2022, 1-16. https://doi.org/10.1155/2022/4104314 yang, m., wang, j., & zhang, x. (2021). boundary-spanning search and sustainable competitive advantage: the mediating roles of exploratory and exploitative innovations. journal of business research, 127, 290-299. https://doi.org/10.1016/j.jbusres.2021.01.032 australasian journal of information systems xu et al. 2024, vol 28, research article using analytical information for digital business transformation 29 yoo, y. (2010). computing in everyday life: a call for research on experiential computing. mis quarterly, 34(2), 213-231. https://doi.org/10.2307/20721425 zahid, h., mahmood, t., & ikram, n. (2018). enhancing dependability in big data analytics enterprise pipelines. in international conference on security, privacy and anonymity in computation, communication and storage (pp. 272-281). cham, switzerland: springer. https://doi.org/10.1007/978-3-030-05345-1_23 appendices appendix a search terms no. search terms 1 (“dataops” or “data operations” or “data operation”) and (“analytics” or “analytical information” or “analytical information processing capability”) and “boundary spanning” and “digital transformation” 2 (“analytics” or “analytical information” or “analytical information processing capability”) and “boundary spanning” and “digital transformation” 3 (“analytics” or “analytical information” or “analytical information processing capability”) and “digital transformation” 4 (“dataops” or “data operations” or “data operation”) and (“analytics” or “analytical information” or “analytical information processing capability”) 5 (“dataops” or “data operations” or “data operation”) and “boundary spanning” 6 (“dataops” or “data operations” or “data operation”) and “digital transformation” 7 “boundary spanning” and “digital transformation” appendix b search results science direct scopus ais e-library web of science google scholar search terms 1 0 0 0 0 1 search terms 2 50 2 36 2 625 search terms 3 3611 652 1495 603 28000 search terms 4 479 28 46 22 61200 search terms 5 1 0 9 0 22 search terms 6 30 2 9 0 841 search terms 7 67 12 67 7 1290 appendix c the conceptualisations, explanations, or examples of analytical information processing capability, analytical information, and boundary spanning activities in the context of digital business transformation note: this appendix shows all the articles that discussed more than one of these concepts. articles that only discuss one of the concepts are not included in the table. australasian journal of information systems xu et al. 2024, vol 28, research article using analytical information for digital business transformation 30 reference analytical information processing capability analytical information boundary spanning activities digital business transformation (conboy et al., 2020) people, process, technology, and organisational structure that support the usage business analytics techniques detailed insights such as stakeholders’ sentiment and needs, inefficiencies, customers segmentation that help to improve the business forming alliances and joint ventures. having boundary spanning tools (e.g., interactive dashboards) to share information improving operational efficiency, developing new business models and services; and improving firms’ competitiveness in the market (dremel et al., 2017) the collection of organisational governance instruments needed to implement big data analytics data based insights (e.g., patterns, predictive orders amount, and demographic distribution of customers) that improve information availability and evidence-based decision making fostering new ways of intraorganisational collaboration between existing departments and the integration of new departments. (e.g., establishing an innovation hub) changing the nature of the company’s offerings and value propositions (gupta et al., 2020) digital analytics capability which consists of firmrelated capabilities, consumer-related capabilities, and macro-level capabilities superior insights (e.g., new customers insights) for decision makers embedding analytics in business processes; interacting with customers and thereby creating more value. marketing productivity, operational excellence, customer value growth (papanagnou et al., 2022) orchestration and management of the data-related resources by the firm predictive information (e.g., future sales) and prescriptive information (e.g., price recommendatio ns sharing information and knowledge. coordinating upstream suppliers and downstream customers transforming the supply chain management by proactively preparing for emergency situations and preventing and mitigating risks (sia et al., 2016) analytical systems (e.g., a cognitive cloud-based data analytics solution) insights that help managers unpack the complexity of the market and predict customer patterns and needs cultivating leadership and investing collaborative technology to facilitate enterprise coordination providing new digital customer experiences australasian journal of information systems xu et al. 2024, vol 28, research article using analytical information for digital business transformation 31 (sebastian et al., 2017) an operational backbone with a single source of truth for critical data. a digital services platform with analytics engines to covert data into insights customer demands; optimized suggestions for operations creating cross functional teams delivering new digital services; improve customer engagement; and achieve operational excellence (tim et al., 2020) insights (e.g., fail rate and user preference) that help to optimize the game design and maximize user experience. broadening use of analytics by embedding analytical experts into a hybrid team and making analytics reports available to all employees shifting towards a new business model (vial, 2019) insights that help firms better answer the needs of customers or perform processes more efficiently leaders to act as boundary spanners to implement digital business strategy and foster close collaboration between internal business and it functions and external parties providing new services, redefining value networks, and better adapting to the changes in the environment (ismail et al., 2017) insights that support realtime and right decisions customer collaboration, cross-channel integration, and information sharing transforming the customer experience (warner & wäger, 2019) new customercentric trends fostering a wider internal and external collaboration activities (e.g., knowledge sharing and establishing a digital joint venture) renewing of the business model, the collaboration, and the culture of the organisations (schallmo et al., 2017) the usage prediction databased routing systems warning alerts as well as maintenance guidance and recommendatio ns digitally transforming their business model by developing an innovative maintenance management system (cao et al., 2019) the ability to use analytical practices increasing customer loyalty australasian journal of information systems xu et al. 2024, vol 28, research article using analytical information for digital business transformation 32 and applications to process data and empowering employee to act quickly enabled by effective and fast decision making (beckett, 2021) representing the views/requiremen ts of the parties involved. coordinating task requirements. searching information; utilising boundary spanning tools proactively responding and coping with changes in client expectations and/or accessible technology (pershina et al., 2019) adopting digital and non-digital boundary spanning tools to facilitate collaborative work among specialists with different background gradually transforming analogue expertise into a novel digital format and creating digital games as digital innovation (matt et al., 2015) formulating a digital transformation strategy to integrate the entire coordination exploring new digital technologies to transform key business operations and affects products and processes, as well as organisational structures and management concepts. (loebbecke & picot, 2015) using big data analytics to analyse and interpret any kind of digital information insights that allowed the organisation to adjust their operations proactively reshaping business models and increasing productivity (ashrafi et al., 2019) a technologically enabled ability that can helps process large volumes of high–velocity data, and several varieties of data insights new knowledge and actionable insights that help firms react properly to a changing environment gaining competitive advantage and improving firm performance (fleischer & carstens, 2021) transferring of information and knowledge, designing the digital solutions for pre-selected, australasian journal of information systems xu et al. 2024, vol 28, research article using analytical information for digital business transformation 33 including generic insights into the digitalization of the public sector; requiring the engagement of various external actors prioritized services across a wide range of policy areas (azzouz & papadonikolak i, 2020) boundary spanners promote information exchange, knowledge sharing, collaboration, and innovation adoption through boundary spanning objects adopting novel digital innovations to digitise processes (singh et al., 2020) chief digital officer (cdo) acting as coordinator to span the vertical boundary and horizontal boundary through formal (e.g., board meetings) and informal coordination mechanisms (e.g., webinars) strategic change of the business (harter & krone, 2001) providing networks of learning. promoting the legitimacy of cooperative forms of organizing, and protecting cooperatives’ interests embracing innovation and change (pappas et al., 2018) the combination of analytical resources, including big data, technology, technical and managerial skills, data driven culture and organisational learning data actors (e.g., academia, private and public organisations) in the big data and business analytics ecosystem need to cooperate, coordinate, and collaborate to enable the use of big data towards the achievement value creation, business change, and societal change australasian journal of information systems xu et al. 2024, vol 28, research article using analytical information for digital business transformation 34 of digital transformation and the creation of sustainable societies (suseno et al., 2018) organisations’ activities and interactions with stakeholders can create value for digital innovation scaling the innovation through data-driven operation, instant release, and swift transformation (khuntia et al., 2022) integrating information within and across partner firms. improved communication and coordination of services, being flexible to address changing business environments and customer expectations appendix d a complete list of the 87 reviewed articles this table below shows the 87 reviewed articles. we cited these articles in section 2 (theoretical background) and section 4 (digital business transformation through dataops, a conceptual framework). # papers academic articles 1 anand, a., sharma, r., & kohli, r. (2020). the effects of operational and financial performance failure on bi&a-enabled search behaviors: a theory of performance-driven search. information systems research, 31(4), 1144-1163. https://doi.org/10.1287/isre.2020.0936 2 ashrafi, a., zare ravasan, a., trkman, p., & afshari, s. (2019). the role of business analytics capabilities in bolstering firms’ agility and performance. international journal of information management, 47, 1-15. https://doi.org/10.1016/j.ijinfomgt.2018.12.005 3 azzouz, a., & papadonikolaki, e. (2020). boundary-spanning for managing digital innovation in the aec sector. architectural engineering and design management, 16(5), 356-373. https://doi.org/10.1080/17452007.2020.1735293 4 beckett, r. c. (2021). agile coping in a digital world: an expanding need for boundary spanning. in agile coping in the digital workplace (pp. 119-145). https://doi.org/10.1007/978-3030-70228-1_7 5 borges, a. f. s., laurindo, f. j. b., spínola, m. m., gonçalves, r. f., & mattos, c. a. (2021). the strategic use of artificial intelligence in the digital era: systematic literature review and future research directions. international journal of information management, 57, 1-16. https://doi.org/10.1016/j.ijinfomgt.2020.102225 6 cao, g., duan, y., & cadden, t. (2019). the link between information processing capability and competitive advantage mediated through decision-making effectiveness. international journal of information management, 44, 121-131. https://doi.org/10.1016/j.ijinfomgt.2018.10.003 7 capizzi, a., distefano, s., & mazzara, m. (2020). from devops to devdataops: data management in devops processes. in software engineering aspects of continuous development and new paradigms of software production and deployment: second international workshop, devops 2019 (vol. 12055). springer, cham. https://doi.org/10.1007/978-3-030-39306-9_4 https://doi.org/10.1287/isre.2020.0936 https://doi.org/10.1016/j.ijinfomgt.2018.12.005 https://doi.org/10.1080/17452007.2020.1735293 https://doi.org/10.1007/978-3-030-70228-1_7 https://doi.org/10.1007/978-3-030-70228-1_7 https://doi.org/10.1016/j.ijinfomgt.2020.102225 https://doi.org/10.1016/j.ijinfomgt.2018.10.003 https://doi.org/https:/doi.org/10.1007/978-3-030-39306-9_4 australasian journal of information systems xu et al. 2024, vol 28, research article using analytical information for digital business transformation 35 8 conboy, k., mikalef, p., dennehy, d., & krogstie, j. (2020). using business analytics to enhance dynamic capabilities in operations research: a case analysis and research agenda. european journal of operational research, 281(3), 656-672. https://doi.org/10.1016/j.ejor.2019.06.051 9 dremel, c., wulf, j., herterich, m. m., waizmann, j. c., & brenner, w., 16(2). (2017). how audi ag established big data analytics in its digital transformation. mis quarterly executive, 16(2), 81-100. https://aisel.aisnet.org/misqe/vol16/iss2/3 10 dwivedi, y. k., hughes, l., ismagilova, e., aarts, g., coombs, c., crick, t., duan, y., dwivedi, r., edwards, j., eirug, a., galanos, v., ilavarasan, p. v., janssen, m., jones, p., kar, a. k., kizgin, h., kronemann, b., lal, b., lucini, b., medaglia, r., le meunier-fitzhugh, k., le meunier-fitzhugh, l. c., misra, s., mogaji, e., sharma, s. k., singh, j. b., raghavan, v., raman, r., rana, n. p., samothrakis, s., spencer, j., tamilmani, k., tubadji, a., walton, p., & williams, m. d. (2021). artificial intelligence (ai): multidisciplinary perspectives on emerging challenges, opportunities, and agenda for research, practice and policy. international journal of information management, 57. https://doi.org/10.1016/j.ijinfomgt.2019.08.002 11 ereth, j. (2018). dataops-towards a definition. in lwda (pp. 104-112). https://ceurws.org/vol-2191/paper13.pdf 12 fleischer, j., & carstens, n. (2021). policy labs as arenas for boundary spanning: inside the digital transformation in germany. public management review, 24(8), 1208-1225. https://doi.org/10.1080/14719037.2021.1893803 13 frank, a. g., dalenogare, l. s., & ayala, n. f. (2019). industry 4.0 technologies: implementation patterns in manufacturing companies. international journal of production economics, 210, 15-26. https://doi.org/10.1016/j.ijpe.2019.01.004 14 glaser, l., fourné, s. p. l., & elfring, t. (2015). achieving strategic renewal: the multi-level influences of top and middle managers’ boundary-spanning. small business economics, 45(2), 305-327. https://doi.org/10.1007/s11187-015-9633-5 15 grover, v., chiang, r. h. l., liang, t.-p., & zhang, d. (2018). creating strategic business valu e from big data analytics: a research framework. journal of management information systems, 35(2), 388-423. https://doi.org/10.1080/07421222.2018.1451951 16 gupta, g., & bose, i. (2022). digital transformation in entrepreneurial firms through information exchange with operating environment. information & management, 59(3). https://doi.org/10.1016/j.im.2019.103243 17 gupta, m., & george, j. f. (2016). toward the development of a big data analytics capability. information & management, 53(8), 1049-1064. https://doi.org/10.1016/j.im.2016.07.004 18 gur, i., moller, f., hupperz, m., uzun, d., & otto, b. (2022). requirements for dataops to foster dynamic capabilities in organizations a mixed methods approach. in 2022 ieee 24th conference on business informatics (cbi) (pp. 166-175). https://doi.org/10.1109/cbi54897.2022.00025 19 harter, l., & krone, k. (2001). the boundary-spanning role of a cooperative support organization: managing the paradox of stability and change in non-traditional organizations. journal of applied communication research, 29(3), 248-277. https://doi.org/10.1080/00909880128111 20 hess, t., matt, c., benlian, a., & wiesböck, f. (2016). options for formulating a digital transformation strategy. mis quarterly executive, 15(2), 123-139. https://aisel.aisnet.org/misqe/vol15/iss2/6 21 jonsson, k., holmström, j., & lyytinen, k. (2009). turn to the material: remote diagnostics systems and new forms of boundary-spanning. information and organization, 19(4), 233-252. https://doi.org/10.1016/j.infoandorg.2009.07.001 22 karippur, n. k., & balaramachandran, p. r. (2022). antecedents of effective digital leadership of enterprises in asia pacific. australasian journal of information systems, 26, 1-35. https://doi.org/10.3127/ajis.v26i0.2525 23 khuntia, j., saldanha, t., kathuria, a., & tanniru, m. r. (2022). digital service flexibility: a conceptual framework and roadmap for digital business transformation. european journal of information systems, 1-19. https://doi.org/10.1080/0960085x.2022.2115410 https://doi.org/10.1016/j.ejor.2019.06.051 https://aisel.aisnet.org/misqe/vol16/iss2/3 https://doi.org/10.1016/j.ijinfomgt.2019.08.002 https://ceur-ws.org/vol-2191/paper13.pdf https://ceur-ws.org/vol-2191/paper13.pdf https://doi.org/10.1080/14719037.2021.1893803 https://doi.org/10.1016/j.ijpe.2019.01.004 https://doi.org/10.1007/s11187-015-9633-5 https://doi.org/10.1080/07421222.2018.1451951 https://doi.org/10.1016/j.im.2019.103243 https://doi.org/10.1016/j.im.2016.07.004 https://doi.org/10.1109/cbi54897.2022.00025 https://doi.org/10.1080/00909880128111 https://aisel.aisnet.org/misqe/vol15/iss2/6 https://doi.org/10.1016/j.infoandorg.2009.07.001 https://doi.org/10.3127/ajis.v26i0.2525 https://doi.org/10.1080/0960085x.2022.2115410 australasian journal of information systems xu et al. 2024, vol 28, research article using analytical information for digital business transformation 36 24 levina, n., & vaast, e. (2014). turning a community into a market: a practice perspective on information technology use in boundary spanning. journal of management information systems, 22(4), 13-37. https://doi.org/10.2753/mis0742-1222220402 25 li, h., wu, y., cao, d., & wang, y. (2021). organizational mindfulness towards digital transformation as a prerequisite of information processing capability to achieve market agility. journal of business research, 122, 700-712. https://doi.org/10.1016/j.jbusres.2019.10.036 26 loebbecke, c., & picot, a. (2015). reflections on societal and business model transformation arising from digitization and big data analytics: a research agenda. the journal of strategic information systems, 24(3), 149-157. https://doi.org/10.1016/j.jsis.2015.08.002 27 mainali, k., ehrlinger, l., matskin, m., & himmelbauer, j. (2021). discovering dataops: a co mprehensive review of definitions, use cases, and tools. in data analytics 2021: the tenth int ernational conference on data analytics (pp. 61-69). https://www.researchgate.net/profile/kiran_ mainali2/publication/355107036_discovering_dataops_a_comprehensive_review_of_definit ions_use_cases_and_tools/links/615dd703fbd5153f47e938a1/discovering-dataops-a-compr ehensive-review-of-definitions-use-cases-and-tools.pdf 28 matt, c., hess, t., & benlian, a. (2015). digital transformation strategies. business & information systems engineering, 57(5), 339-343. https://doi.org/10.1007/s12599-015-0401-5 29 müller, o., junglas, i., debortoli, s., & vom brocke, j. (2016). using text analytics to derive customer service management benefits from unstructured data. mis quarterly executive, 15(4), 243-258. https://doi.org/10.25300/misq/2018/13239 30 munappy, a. r., bosch, j., & olsson, h. h. (2020). data pipeline management in practice: challenges and opportunities. in product-focused software process improvement (pp. 168-184). https://doi.org/10.1007/978-3-030-64148-1_11 31 munappy, a. r., mattos, d. i., bosch, j., olsson, h. h., & dakkak, a. (2020). from ad-hoc data analytics to dataops. in proceedings of the international conference on software and system processes (pp. 165-174). https://doi.org/10.1145/3379177.3388909 32 naseer, h., maynard, s. b., & desouza, k. c. (2021). demystifying analytical information processing capability: the case of cybersecurity incident response. decision support systems, 143, 1-11. https://doi.org/10.1016/j.dss.2020.113476 33 naseer, h., maynard, s. b., & xu, j. (2020). modernizing business analytics capability with dataops: a decision-making agility perspective. in ecis 2020 (pp. 1-11). https://aisel.aisnet.org/ecis2020_rip/36 34 papanagnou, c., seiler, a., spanaki, k., papadopoulos, t., & bourlakis, m. (2022). data-driven digital transformation for emergency situations: the case of the uk retail sector. international journal of production economics, 250. https://doi.org/10.1016/j.ijpe.2022.108628 35 pappas, i. o., mikalef, p., giannakos, m. n., krogstie, j., & lekakos, g. (2018). big data and business analytics ecosystems: paving the way towards digital transformation and sustainable societies. information systems and e-business management, 16(3), 479-491. https://doi.org/10.1007/s10257-018-0377-z 36 pershina, r., soppe, b., & thune, t. m. (2019). bridging analog and digital expertise: crossdomain collaboration and boundary-spanning tools in the creation of digital innovation. research policy, 48(9), 1-13. https://doi.org/10.1016/j.respol.2019.103819 37 ranjan, j., & foropon, c. (2021). big data analytics in building the competitive intelligence of organizations. international journal of information management, 56, 1-13. https://doi.org/10.1016/j.ijinfomgt.2020.102231 38 richardson, g. (2020). to build or buy? effective dataops in an era of rapid change. database and network journal, 50(1), 3-5. https://link.gale.com/apps/doc/a619013513/aone?u=unimelb& sid=googlescholar&xid=fb06b7f3 39 rodriguez, m., de araújo, l. j. p., & mazzara, m. (2020). good practices for the adoption of d ataops in the software industry. in journal of physics: conference series (vol. 1694, pp. 012-032). iop publishing. https://doi.org/10.1088/1742-6596/1694/1/012032 40 sahoo, p. r., & premchand, a. (2019). dataops in manufacturing and utilities industries. international journal of applied information systems, 12(23), 1-6. https://doi.org/10.5120/ijais2019451814 https://doi.org/10.2753/mis0742-1222220402 https://doi.org/10.1016/j.jbusres.2019.10.036 https://doi.org/10.1016/j.jsis.2015.08.002 https://www.researchgate.net/profile/kiran_mainali2/publication/355107036_discovering_dataops_a_comprehensive_review_of_definitions_use_cases_and_tools/links/615dd703fbd5153f47e938a1/discovering-dataops-a-comprehensive-review-of-definitions-use-cases-and-tools.pdf https://www.researchgate.net/profile/kiran_mainali2/publication/355107036_discovering_dataops_a_comprehensive_review_of_definitions_use_cases_and_tools/links/615dd703fbd5153f47e938a1/discovering-dataops-a-comprehensive-review-of-definitions-use-cases-and-tools.pdf https://www.researchgate.net/profile/kiran_mainali2/publication/355107036_discovering_dataops_a_comprehensive_review_of_definitions_use_cases_and_tools/links/615dd703fbd5153f47e938a1/discovering-dataops-a-comprehensive-review-of-definitions-use-cases-and-tools.pdf https://www.researchgate.net/profile/kiran_mainali2/publication/355107036_discovering_dataops_a_comprehensive_review_of_definitions_use_cases_and_tools/links/615dd703fbd5153f47e938a1/discovering-dataops-a-comprehensive-review-of-definitions-use-cases-and-tools.pdf https://doi.org/10.1007/s12599-015-0401-5 https://doi.org/10.25300/misq/2018/13239 https://doi.org/10.1007/978-3-030-64148-1_11 https://doi.org/10.1145/3379177.3388909 https://doi.org/10.1016/j.dss.2020.113476 https://aisel.aisnet.org/ecis2020_rip/36 https://doi.org/10.1016/j.ijpe.2022.108628 https://doi.org/10.1007/s10257-018-0377-z https://doi.org/10.1016/j.respol.2019.103819 https://doi.org/10.1016/j.ijinfomgt.2020.102231 https://link.gale.com/apps/doc/a619013513/aone?u=unimelb&sid=googlescholar&xid=fb06b7f3 https://link.gale.com/apps/doc/a619013513/aone?u=unimelb&sid=googlescholar&xid=fb06b7f3 https://doi.org/10.1088/1742-6596/1694/1/012032 https://doi.org/10.5120/ijais2019451814 australasian journal of information systems xu et al. 2024, vol 28, research article using analytical information for digital business transformation 37 41 saldanha, t. j. v. v., mithas, s., & krishnan, m. s. (2017). leveraging customer involvement for fueling innovation: the role of relational and analytical information processing capabilities. mis quarterly, 41(1), 267-286. https://www.jstor.org/stable/26629647 42 schallmo, d., williams, c. a., & boardman, l. (2017). digital transformation of business models — best practice, enablers, and roadmap. international journal of innovation management, 21(08), 1-17. https://doi.org/10.1142/s136391961740014x 43 schwade, f. (2021). measuring and visualising boundary spanning in enterprise collaboration systems. in ecis (pp. 1-16). https://aisel.aisnet.org/ecis2021_rp/63 44 sebastian, i., ross, j., beath, c., mocker, m., moloney, k., & fonstad, n. (2017). how big old companies navigate digital transformation. mis quarterly executive, 16(3), 197-213. https://aisel.aisnet.org/misqe/vol16/iss3/6 45 setia, p., setia, p., venkatesh, & joglekar, s. (2014). leveraging digital technologies: how information quality leads to localized capabilities and customer service performance. mis quarterly, 565-590. https://www.jstor.org/stable/43825923 46 sia, s. k., soh, c., & weill, p. (2016). how dbs bank pursued a digital business strategy. mis quarterly executive, 15(2), 105-121. https://aisel.aisnet.org/misqe/vol15/iss2/4 47 singh, a., klarner, p., & hess, t. (2020). how do chief digital officers pursue digital transformation activities? the role of organization design parameters. long range planning, 53(3), 1-14. https://doi.org/10.1016/j.lrp.2019.07.001 48 someh, i. a., & shanks, g. (2013). the role of synergy in achieving value from business analytics systems. in icis (pp. 1-15). https://aisel.aisnet.org/icis2013/proceedings/knowledgemanagement/7 49 song, h., li, m., & yu, k. (2021). big data analytics in digital platforms: how do financial service providers customise supply chain finance? international journal of operations & production management, 41(4), 410-435. https://doi.org/10.1108/ijopm-07-2020-0485 50 srinivasan, r., & swink, m. (2018). an investigation of visibility and flexibility as complements to supply chain analytics: an organizational information processing theory perspective. production and operations management, 27(10), 1849-1867. https://doi.org/10.1111/poms.12746 51 suseno, y., laurell, c., & sick, n. (2018). assessing value creation in digital innovation ecosystems: a social media analytics approach. the journal of strategic information systems, 27(4), 335-349. https://doi.org/https://doi.org/10.1016/j.jsis.2018.09.004 52 tan, f. t. c., ondrus, j., tan, b., & oh, j. (2020). digital transformation of business ecosystems: evidence from the korean pop industry. information systems journal, 30(5), 866898. https://doi.org/10.1111/isj.12285 53 tim, y., hallikainen, p., pan, s. l., & tamm, t. (2020). actualizing business analytics for organizational transformation: a case study of rovio entertainment. european journal of operational research, 281(3), 642-655. https://doi.org/10.1016/j.ejor.2018.11.074 54 verhoef, p. c., broekhuizen, t., bart, y., bhattacharya, a., qi dong, j., fabian, n., & haenlein, m. (2021). digital transformation: a multidisciplinary reflection and research agenda. journal of business research, 122, 889-901. https://doi.org/10.1016/j.jbusres.2019.09.022 55 vial, g. (2019). understanding digital transformation: a review and a research agenda. the journal of strategic information systems, 28(2), 118-144. https://doi.org/10.1016/j.jsis.2019.01.003 56 vidgen, r., shaw, s., & grant, d. b. (2017). management challenges in creating value from business analytics. european journal of operational research, 261(2), 626-639. https://doi.org/10.1016/j.ejor.2017.02.023 57 vilvovsky, s. (2009). internal and external boundary spanning in outsourced is development projects: opening the black box. in amcis 2009 doctoral consortium (pp. 1-11). https://aisel.aisnet.org/amcis2009_dc/4 58 walsh, b. (2023). ai best practice and dataops. in productionizing ai (pp. 41-74). https://doi.org/10.1007/978-1-4842-8817-7_2 59 warner, k. s. r., & wäger, m. (2019). building dynamic capabilities for digital transformation: an ongoing process of strategic renewal. long range planning, 52(3), 326349. https://doi.org/10.1016/j.lrp.2018.12.001 https://www.jstor.org/stable/26629647 https://doi.org/10.1142/s136391961740014x https://aisel.aisnet.org/ecis2021_rp/63 https://aisel.aisnet.org/misqe/vol16/iss3/6 https://www.jstor.org/stable/43825923 https://aisel.aisnet.org/misqe/vol15/iss2/4 https://doi.org/10.1016/j.lrp.2019.07.001 https://aisel.aisnet.org/icis2013/proceedings/knowledgemanagement/7 https://doi.org/10.1108/ijopm-07-2020-0485 https://doi.org/10.1111/poms.12746 https://doi.org/https:/doi.org/10.1016/j.jsis.2018.09.004 https://doi.org/10.1111/isj.12285 https://doi.org/10.1016/j.ejor.2018.11.074 https://doi.org/10.1016/j.jbusres.2019.09.022 https://doi.org/10.1016/j.jsis.2019.01.003 https://doi.org/10.1016/j.ejor.2017.02.023 https://aisel.aisnet.org/amcis2009_dc/4 https://doi.org/10.1007/978-1-4842-8817-7_2 https://doi.org/10.1016/j.lrp.2018.12.001 australasian journal of information systems xu et al. 2024, vol 28, research article using analytical information for digital business transformation 38 60 wee, m., scheepers, h., & tian, x. (2022). understanding the processes of how small and medium enterprises derive value from business intelligence and analytics. australasian journal of information systems, 26, 1-26. https://doi.org/10.3127/ajis.v26i0.2969 61 yang, m., wang, j., & zhang, x. (2021). boundary-spanning search and sustainable competitive advantage: the mediating roles of exploratory and exploitative innovations. journal of business research, 127, 290-299. https://doi.org/10.1016/j.jbusres.2021.01.032 62 yoo, y. (2010). computing in everyday life: a call for research on experiential computing. mis quarterly, 34(2), 213-231. https://doi.org/10.2307/20721425 63 zahid, h., mahmood, t., & ikram, n. (2018). enhancing dependability in big data analytics enterprise pipelines. in international conference on security, privacy and anonymity in computation, communication and storage (pp. 272-281). springer, cham. https://doi.org/10.1007/978-3-030-05345-1_23 64 aldrich, h., & herker, d. (1977). boundary spanning roles and organization structure. academy of management review, 2(2), 217-230. https://doi.org/10.5465/amr.1977.4409044 65 ancona, d. g., & caldwell, d. (1990). beyond boundary spanningmanaging external dependence in product development teams. the journal of high technology management research, 1(2), 119-135. https://doi.org/10.1016/1047-8310(90)90001-k 66 daft, r. l., & lengel, r. h. (1986). organizational information requirements, media richness and structural design. management science, 32(5), 554-571. https://doi.org/10.1287/mnsc.32.5.554 67 flynn, b. b., & flynn, e. j. (1999). information-processing alternatives for coping with manufacturing environment complexity. decision sciences, 30(4), 1021-1052. https://doi.org/10.1111/j.1540-5915.1999.tb00917.x 68 fox, n. j. (2011). boundary objects, social meanings, and the success of new technologies. sociology, 45(1), 70-85. https://doi.org/10.1177/0038038510387196 69 galbraith, j. r. (1974). organization design: an information processing view. interfaces, 4(3), 2 8-36. https://doi.org/10.1287/inte.4.3.28 70 leifer, r., & delbecq, a. (1978). organizational: environmental interchange: a model of boun dary spanning activity. academy of management review, 3(1), 40-50. https://doi.org/10.5465/amr .1978.4296354 71 marrone, j. a. (2010). team boundary spanning: a multilevel review of past research and proposals for the future. journal of management, 36(4), 911-940. https://doi.org/10.1177/0149206309353945 72 schotter, a. p. j., mudambi, r., doz, y. l., & gaur, a. (2017). boundary spanning in global organizations. journal of management studies, 54(4), 403-421. https://doi.org/10.1111/joms.12256 73 tushman, m. l., & nadler, d. a. (1978). information processing as an integrating concept in organizational design. academy of management review, 3(3), 613-624. https://doi.org/10.5465/amr.1978.4305791 74 bahaa, s., ghalwash, a. z., & harb, h. (2023). dataops lifecycle with a case study in healthcare. international journal of advanced computer science and applications, 14(1). https://doi.org/10.14569/ijacsa.2023.0140115 practitioner articles 75 aslett, m. (2020). dataops unlocks the value of data. https://www.hitachivantara.com/enus/pdf/analyst-content/dataops-unlocks-value-of-data-451-research.pdf 76 atwal, h. (2020). practical dataops: delivering agile data science at scale. apress. https://link.springer.com/content/pdf/10.1007/978-1-4842-5104-1.pdf 77 bergh, c., benghiat, g., & strod, e. (2019). the dataops cookbook: methodologies and tools that reduce analytics cycle time while improving quality. datakitchen headquarters. https://www.devopsschool.com/blog/wpcontent/uploads/2021/07/dk_dataops_book_2nd_edition.pdf 78 bonnet, d., & westerman, g. (2021). the new elements of digital transformation. mit sloan management review, 62(2), 82-89. https://www.proquest.com/scholarly-journals/new-elementsdigital-transformation/docview/2471816389/se-2?accountid=12372 79 eckerson, w. (2019a). trends in dataops: bringing scale and rigor to data and analytics. https:// www.eckerson.com/register?content=trends-in-dataops-bringing-scale-and-rigor-to-data-andanalytics https://doi.org/https:/doi.org/10.3127/ajis.v26i0.2969 https://doi.org/10.1016/j.jbusres.2021.01.032 https://doi.org/10.2307/20721425 https://doi.org/https:/doi.org/10.1007/978-3-030-05345-1_23 https://doi.org/https:/doi.org/10.5465/amr.1977.4409044 https://doi.org/10.1016/1047-8310(90)90001-k https://doi.org/10.1287/mnsc.32.5.554 https://doi.org/10.1111/j.1540-5915.1999.tb00917.x https://doi.org/10.1177/0038038510387196 https://doi.org/https:/doi.org/10.1287/inte.4.3.28 https://doi.org/10.5465/amr.1978.4296354 https://doi.org/10.5465/amr.1978.4296354 https://doi.org/10.1177/0149206309353945 https://doi.org/10.1111/joms.12256 https://doi.org/https:/doi.org/10.5465/amr.1978.4305791 https://doi.org/10.14569/ijacsa.2023.0140115 https://www.hitachivantara.com/en-us/pdf/analyst-content/dataops-unlocks-value-of-data-451-research.pdf https://www.hitachivantara.com/en-us/pdf/analyst-content/dataops-unlocks-value-of-data-451-research.pdf https://link.springer.com/content/pdf/10.1007/978-1-4842-5104-1.pdf https://www.devopsschool.com/blog/wp-content/uploads/2021/07/dk_dataops_book_2nd_edition.pdf https://www.devopsschool.com/blog/wp-content/uploads/2021/07/dk_dataops_book_2nd_edition.pdf https://www.proquest.com/scholarly-journals/new-elements-digital-transformation/docview/2471816389/se-2?accountid=12372 https://www.proquest.com/scholarly-journals/new-elements-digital-transformation/docview/2471816389/se-2?accountid=12372 https://www.eckerson.com/register?content=trends-in-dataops-bringing-scale-and-rigor-to-data-and-analytics https://www.eckerson.com/register?content=trends-in-dataops-bringing-scale-and-rigor-to-data-and-analytics https://www.eckerson.com/register?content=trends-in-dataops-bringing-scale-and-rigor-to-data-and-analytics australasian journal of information systems xu et al. 2024, vol 28, research article using analytical information for digital business transformation 39 80 eckerson, w. (2019b). the ultimate guide to dataops: production evaluation and selection criteria. https://www.eckerson.com/register?content=the-ultimate-guide-to-dataops 81 ereth, j., & eckerson, w. (2018). dataops: industrializing data and analytics strategies for streaml ining the delivery of insights. https://www.eckerson.com/register?content=dataops-industrializi ng-data-and-analytics 82 friedman, t., & heudecker, n. (2019). introducing dataops into your data management disciplin e. https://www.gartner.com/document/3970916?ref=solrall&refval=246181733 83 heudecker, n., friedman, t., & dayley, a. (2020). innovation insight for dataops. https://www. gartner.com/document/3896766?ref=solrall&refval=232501050&qid= 84 ismail, m. h., khater, m., & zaki, m. (2017). digital business transformation and strategy: wh at do we know so far. cambridge service alliance, 10(1), 1-36. https://cambridgeservicealliance. eng.cam.ac.uk/system/files/documents/2017novpaper_mariam.pdf 85 palmer, a. (2015). from devops to dataops. in c. shannon (ed.), getting data right: tackling t he challenges of big data volume and variety (pp. 49-57). o’reilly. https://cdn2.hubspot.net/hubf s/2631050/cco%20usa/getting_data_right_tamr.pdf 86 thusoo, a., & sarma, j. (2017). creating a data-driven enterprise with dataops. o’reilly media i nc. https://www.oreilly.com/library/view/creating-a-data-driven/9781492049227/ 87 wells, d. (2019). intelligent data operations: the next wave in smart data ecosystems. https://www.eckerson.com/register?content=intelligent-data-operations-the-next-wave-insmart-data-ecosystems copyright: © 2024 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v28.4215 https://www.eckerson.com/register?content=the-ultimate-guide-to-dataops https://www.eckerson.com/register?content=dataops-industrializing-data-and-analytics https://www.eckerson.com/register?content=dataops-industrializing-data-and-analytics https://www.gartner.com/document/3970916?ref=solrall&refval=246181733 https://www.gartner.com/document/3896766?ref=solrall&refval=232501050&qid https://www.gartner.com/document/3896766?ref=solrall&refval=232501050&qid https://cambridgeservicealliance.eng.cam.ac.uk/system/files/documents/2017novpaper_mariam.pdf https://cambridgeservicealliance.eng.cam.ac.uk/system/files/documents/2017novpaper_mariam.pdf https://cdn2.hubspot.net/hubfs/2631050/cco%20usa/getting_data_right_tamr.pdf https://cdn2.hubspot.net/hubfs/2631050/cco%20usa/getting_data_right_tamr.pdf https://www.oreilly.com/library/view/creating-a-data-driven/9781492049227/ https://www.eckerson.com/register?content=intelligent-data-operations-the-next-wave-in-smart-data-ecosystems https://www.eckerson.com/register?content=intelligent-data-operations-the-next-wave-in-smart-data-ecosystems http://creativecommons.org/licenses/by-nc/3.0/au/ https://doi.org/10.3127/ajis.v28.4215 1 introduction 2 theoretical background 2.1 using analytical information for digital business transformation 2.2 the need for analytical information processing capability and boundary spanning activities for digital business transformation 2.3 linking dataops with digital business transformation 3 research method 3.1 data collection 3.2 data analysis 4 digital business transformation through dataops: a conceptual framework 4.1 dynamic business environment 4.2 employing dataops for digital business transformation 4.3 analytical information processing capability, boundary spanning activities, and their interaction for digital business transformation 5 discussion 5.1 implications for research 5.2 implications for practice 6 conclusions, limitations, and future research paper title (paper title style) australasian journal of information systems raghavan & mahadevan 2024, vol 28, research article project outcomes amidst technical uncertainty received 23 aug 2023, accepted 3 oct 2024 after 4 round of review with karlheinz kautz as accepting editor. 1 achieving desired project outcomes with control modes amidst technical uncertainty vijay v. raghavan information systems & analytics college of informatics northern kentucky university highland heights, ky 41099, usa lakshman mahadevan information systems, analytics, and supply chain lutgert college of business florida gulf coast university fort myers, fl 33965, usa email: lmahadevan@fgcu.edu abstract organizations aim to achieve successful project outcomes in information systems development (isd) projects by managing the technical uncertainties that arise with adopting innovative technologies. this study, guided by control theory, investigates various control modes—namely, behavior, outcome, clan, and self-control—and their role in enhancing internal efficiency and psychological outcomes among project members in the presence of technical uncertainty. our findings reveal that outcome and clan control modes are particularly effective at promoting favorable project results amidst the technical uncertainty surrounding innovative technology development, whereas behavior and self-control modes show no significant impact. as a result, prioritizing outcome and clan control modes is recommended for managing isd projects characterized by technical uncertainty. keywords: control modes, technical uncertainty, internal efficiency, psychological outcomes, information systems development project. 1 introduction in the dynamic landscape of information systems development (isd), the ever-evolving nature of technology introduces a myriad of challenges, with technical uncertainty being a prominent one. technical uncertainty is described as the lack of familiarity and experience among project members when working with new, complex, or rapidly evolving technologies (ragatz et al., 2002; lewis et al., 2002). as organizations increasingly rely on isd projects to drive innovation and gain a competitive edge, understanding and effectively managing technical uncertainty becomes paramount (codex, 2023). this argument advocates for the necessity to study technical uncertainty in isd projects, asserting that doing so is essential for optimizing project outcomes, ensuring resource efficiency, and fostering innovation (dönmez & grote, 2018). organizations invest in innovative technologies such as generative artificial intelligence, quantum computing and virtual reality to leverage existing solutions or create new ones with a goal of gaining competitive advantage and improving their innovation process and performance (spanjol et al., 2018). nearly half (49%) of the 1000 business leaders around the australasian journal of information systems raghavan & mahadevan 2024, vol 28, research article project outcomes amidst technical uncertainty 2 world at enterprise-level organizations believe that the inability to keep up with technology innovation compared to their competition as one of the greatest threats to their organizations (mullen, 2023). innovative technologies comprise of enhanced products and processes, which are either entirely novel or substantially improved in terms of quality, performance, and technological advancements (dasgupta et al., 2011). however, innovative technologies are often unrefined, and these technologies change over time as new solutions are developed and tested (lynn & akgün, 1998). to make full use of the technology, there needs to be a good understanding of its capability and the advantages it can deliver. however, technological advancements are happening at a rapid pace which requires companies to embrace constant innovation through accelerated development cycles to 1) stay ahead of the competition (jain et al., 2019) and 2) target profitability within a short span of time (gnanasambandam et al., 2018). the introduction of innovative technologies can create challenges for companies, hindering their ability to bring new designs to market (lewis et al., 2002). this issue often arises because the workforce may not possess the necessary knowledge to effectively utilize these innovations (jalonen, 2012). such problems can adversely affect internal efficiency, as companies need to release innovative technology quickly, on schedule, and to meet user satisfaction, particularly in environments marked by technical uncertainty (laine et al., 2016). furthermore, the success of a project is influenced by psychological factors, which includes each team member's motivation and the satisfaction they gain from participating in the information systems development (isd) process (stewart & gosain, 2006). psychological outcomes perceived by team members depend on how much control they have over uncertain conditions (gerber, 2009). in situations where team members have more control, they experience greater interest, higher creativity, less pressure, and tension (deci & ryan, 1987; gerber & carroll, 2012). control has been thought of as an attempt to impact the behavior of another individual or group to accomplish planned objectives (cram et al., 2016) and categorized into two distinct modes: formal and informal control (kirsch, 1997). while there are a sufficient number of studies that have researched project performance using control theory, remus et al. (2020) show that previous research has been limited 1) by conceptualizing the use of controls as either authoritative or enabling (weiner et al., 2016) and 2) finding inconsistent results on the use of various control modes due to focus on project level unit of analysis (henry et al., 2015; wiener et al., 2016). however, remus et al. (2020) recommend that future studies consider researching technical uncertainty which is a major contextual influence especially in the case of isd projects deploying innovative technologies. while informal controls provide effective support for innovation (beese et al., 2023), an organizational level study conducted by liu (2015a) on chinese companies found that in highly risky and complex situations, behavior and self-control have low effects on performance, but outcome and clan control have higher effectiveness. in contrast, our research aims to understand the control modes that help individual team members achieve project outcomes in the presence of technical uncertainty. thus, we ask the following research question. rq: how can expected project outcomes be achieved by effectively adapting appropriate control modes in the presence of technical uncertainty? to answer this question, we conducted a survey among software professionals who had recently participated in isd projects under technically uncertain conditions. the findings from australasian journal of information systems raghavan and mahadevan 2024, vol 28, research article achieving project outcomes amidst technical uncertainty 3 our research indicate that in innovative technology-driven development environments characterized by technical uncertainty, both clan and outcome control offer adequate internal efficiency and psychological outcomes. however, behavior and self-control do not yield significant benefits. in the following sections, we first discuss the theoretical foundations of this study by elaborating on the different modes of control employed by management in companies to get isd projects completed. second, we develop the research model and present the hypotheses. third, the measurement model is evaluated, and we empirically test the hypotheses through structural equation modeling. finally, we present the results of the hypothesis testing and discuss the theoretical and managerial implications of the findings. 2 theoretical foundations isd is a complex, knowledge-intense activity that often use innovative technologies leading to less desired project outcomes. in such an environment, control modes help to ensure alignment of isd teams with firm performance (nidumolu & subramani, 2003). control theory (ouchi, 1979) provides a useful theoretical foundation to conceptualize controller and controlee relationships and their effects on internal efficiency and psychological outcomes in the presence of technical uncertainty. control modes attempt to converge the divergent views that the controller and the controlees have about project completion (eisenhardt, 1985). control can be broadly referred to as a set of process and rules that a controlee implements to promote desirable controller behavior (choudhury & sabherwal, 2003). in an organization, the term controller refers to the team of individuals responsible for the conception and implementation of controls. the term controlee refers to the individuals who are tasked with systems development (choudhury & sabherwal, 2003). as technologies become more sophisticated, an accurate assessment of their functionality requires extensive knowledge, which makes the choices difficult for organizations (bakos, 1991; duan et al., 2009) resulting in increased technical uncertainty. however, appropriate fit between the technical uncertainty and interorganizational interaction leads to more effective integration of innovative technology (stock & tatikonda, 2008). research by ens et al. (2023) has shown that controls on digital platforms is uniquely dynamic and requires the use of both formal and informal controls. as much as there is need for flexibility to deal with uncertainty in the isd process, there is a need for control modes to ensure compliance with organizational goals. to this end, control modes such as behavior, outcome, clan and self-control provide organizations with the ability to manage the flexibility versus control standoff often experienced in isd projects (harris et al., 2006). however, “when a project has both unknowns and tight control, controlees respond to tight controls instead of unknowns. controlees behave as controllers expect leading controllers to be unaware of brewing problems” (chen et al., 2022. p 309). 2.1 behavior control behavior control refers to how project participants' expected behaviors are specified through formal documents, and members are expected to follow the prescribed rules and procedures (jaworksi & macinnis, 1989; kirsch, 1996). these may include requirements documents that are written and accepted, change procedures, approval processes, coding standards, and testing methodologies (remus et al., 2020; saxena & mcdonagh, 2020). for behavior control procedures to be successful, the project behavior specified must be observable, and there must australasian journal of information systems raghavan and mahadevan 2024, vol 28, research article achieving project outcomes amidst technical uncertainty 4 be monitoring modes to ensure conformity with expected behavior (eisenhardt, 1985). behavior control assumes the presence of a single control hierarchy (mccarthy et al., 2023) and is implemented through modes such as a isd procedure or those that permit the controller to assess behavior (kirsch, 1997). while behavior control has been proposed for virtual teams (townsend et al., 1998) and for collocated teams (henderson & lee, 1992; kirsch, 1997), research by piccoli and ives (2003) has shown that behavior control negatively impacts trust in virtual teams since team members’ failure to uphold their obligations can be easily detected. 2.2 outcome control in this formal mode of control, project managers independently decide on the outcomes, such as when the projects or subtasks are expected to be completed and what resources are available. the controller focuses on the output without regard to the process used to achieve these outputs (choudhary & sabherwal, 2003). the performances of the participants are measured based on the outcome or goals set by the controller (kirsch, 1997). outcomes can be the completion of functional specifications and completing the project on target. both behavior and outcomes control modes are directed toward aligning the participants' goals with the goals of a project or its manager and are grouped under the general label of formal modes of control. outcome measurability is defined as the capability of controllers to evaluate and measure the ability of controlees to achieve the desired results and outcomes (kirsch et al., 2002). 2.3 clan control the knowledge of isd is often dispersed within the organization. the project members (chua et al., 2012) are usually drawn from varied stakeholders, such as customers, consultants, and information technology (it) professionals, to work in unison for project delivery. the project manager's role is to facilitate a shared understanding of organizational objectives and processes. the notion of clan control is based on the premise that formal modes of control described earlier are unlikely to be as effective as control arising from shared norms of a team. isd projects are intensive team environments where participants are drawn from multiple organizations, including individuals with diverse skills and expertise. in such cases, it is realistic to expect how the performance of the project can be positively influenced by members working toward a common goal. the extent of trust amongst project members and the expectation to do their part of the project well is the underlying theme of clan control. the ability of a project team to direct members' behavior in this direction is known as clan control. further team-based clan control is a conceptualization fostered by team members and enabled by managers who are clan members themselves (kirsch et al., 2010). 2.4 self-control when a project's members exercise self-control, they independently set explicit goals for themselves without the contribution of a team manager. self-control is a mode by which an individual relies on self-monitoring and regulation (kirsch & cummings, 1996). individuals undertake a particular task by controlling their own actions (kirsch, 1997) and proceed to evaluate themselves to whether they have achieved their goals and reward or sanction themselves correspondingly (jaworski, 1988; kirsch, 1997). flexibility and self-control are essential in a complex project environment where it is expected that everyone would do their part with little monitoring by anyone outside or within the project. heales et al. (2007) report that informal controls such as self-control play a major role in ensuring successful project australasian journal of information systems raghavan and mahadevan 2024, vol 28, research article achieving project outcomes amidst technical uncertainty 5 outcomes and are effective for the performance of different stakeholders in an it project (liu, 2015b; wright et al., 1993) including developers (henderson & lee, 1992). 2.5 technical uncertainty uncertainty in the plain english sense of the word means a lack of sureness about someone or something (uncertainty, n.d.). it is the conditional volatility of a disruption that is unpredictable (jurado et al., 2015). uncertainty is a sense of ambiguity associated with a lack of clear picture about the behavior of relevant project members, lack of procedures to consider issues, lack of knowledge about the presence of known and unknown sources of bias, lack of familiarity about the amount of effort required to resolve the condition (ward & chapman, 2003) and information deficiency (klir, 2006). uncertainty reflects a person’s lack of knowledge of the goals of the project and the environment (shenhar & dvir, 2007) such as the isd process. project uncertainty is mostly evident in isd tasks that are usually complex (jun et al., 2011; mehta et al., 2014). however, most project management practices do not consider the uncertainty of the project for determining strategies relevant to task completion (maes et al., 2022). the lack of critical knowledge about various aspects of the project can derail isd leading to uncertainty (mehta et al., 2014). technical uncertainty is a major challenge faced by high-tech companies (yu & chiu, 2013) and refers to the project members’ unfamiliarity and the lack of experience in dealing with technologies that are new, complex and/or rapidly changing (ragatz et al., 2002; lewis et al., 2002). technical uncertainty has been considered in organizations that consider research and development as part of their process (atanassov et al., 2024), where a strong correlation existed in situations where a team had to choose between various available technologies (holland et al., 1976). there is no more fertile ground than isd for technical uncertainty to affect the outcome. varieties in design along with the skill and competence of the developer can fuel technical uncertainty. although it is not possible to foresee issues during isd projects, project managers can employ tactics that mitigate those possible problems by managing uncertainties. (marinho et al., 2018). variations in the design refer to the challenges that frequently occur due to variations in skilled worker levels and their technical competence and capabilities. these uncertainties in technology affect project budgets, workloads, and other indicators to varying degrees related to project performance. 2.6 project outcomes we consider project outcomes from two dimensions: the internal efficiency of the project and its psychological outcomes. psychological outcomes are the team members’ satisfaction level with the work associated with the project (aladwani, 2002). innovative technologies can usually provide challenges to team members that can be quite motivating and interesting. iacovou et al. (2005) show that good, open communications will lead to better interactions and superior team performance, thus, leading to positive psychological outcomes. the criteria of project cost, project time, and project quality have been generally used (shenhar et al., 1997; baccarini, 1999) as project management internal measures of efficiency. the findings of a study by banker & kemerer, (1992) concluded that timeliness in delivering the project and user satisfaction with the project are important matrices in determining project success. internal efficiency measures how well resources are utilized to complete the project within the assigned time and achieve desired user satisfaction. while a study by cheng et al. (2018) found that the relative lack of internal control significantly lowered an organization’s australasian journal of information systems raghavan and mahadevan 2024, vol 28, research article achieving project outcomes amidst technical uncertainty 6 operational efficiency, organizations engaged in isd projects with innovative technologies could prioritize on-time completion and gaining user satisfaction to achieve organizational excellence and competitive advantage. 3 research model and hypotheses the research model is presented in figure 1. the direct effect of the control modes on isd project outcomes is assessed followed by the examination of the mediating effect of technical uncertainty on the association between each control mode with internal efficiency and psychological outcome. unlike moderators which affect the strength or direction of a relationship between two variables, a mediating variable clarify how external physical events take on internal psychological significance and accounts for the relationship between the independent and dependent variable (baron & kinney, 1986). technical uncertainty can directly impact the execution of tasks (song & montoya-weiss, 2001) affecting the isd process. figure 1. theoretical model showing direct relationships and mediation of technical uncertainty on project outcomes 3.1 hypotheses successful isd projects usually score well on efficient use of time and budget (aladwani, 2002). project managers, who are entrusted with the task of ensuring time and budget objectives may resort to behavior control to achieve internal efficiency. in this type of control, managers are very ‘hands-on’ by nature in providing a written sequence of procedures that need to be followed by the project team. in isd, behavior control involves specifying methods for tasks such as project management, analysis, design, programming, testing, and documentation (kirsch, 1996). research by kanwal et al. (2017) has found that behavior australasian journal of information systems raghavan and mahadevan 2024, vol 28, research article achieving project outcomes amidst technical uncertainty 7 control is positively related to isd project performance. such interventions generally result in the project achieving higher efficiencies. managers who exhibit behavior control are heavily involved in the process of change and often closely monitor the progress made by their team. these managers are usually quite experienced and have had previous experience working hands-on on similar projects. establishing behavior control involves developing management processes to define and oversee behaviors, as well as implementing incentives and rewards to encourage appropriate conduct (nidumolu & subramani, 2003). however, behavior control has difficulty in predicting the desired output in the presence of complexity (chua et al., 2012). teams are unable to respond quickly to issues arising out of the complexity of certain isd projects (chien et al., 2007). though patterson et al. (1997) showed that uncertainty will have a negative association with performance perceptions, it is possible that the manager can come up with procedures to standardize the change process, enabling the codification and formalization of information to manage processes (kirsch, 1996). however, such procedural interventions may not be enough to nullify the impact of technical uncertainty on internal efficiency. thus, we hypothesize that: h1a: technical uncertainty mediates the effect of behavior control on internal efficiency. when team members are expected to follow procedures and are regularly assessed on how they performed tasks, they often find themselves in tightly controlled situations where their expertise is dominated by the sequence of steps prescribed by the manager. this can lead to dissatisfaction among team members resulting in less than expected positive psychological outcomes. most notably, information system workers reported significantly lower professional efficiency and lower job satisfaction in firms where the dominant formal form of control is behavior control (ply et al., 2012). the adherence to the rules and regulations set by the manager along with the close monitoring of the project's progress can lead to dissatisfaction among the team members. focusing on the material consequence of the isd project without bearing in mind the social situation ignores the fact that an isd project is completed within a highly interactive situation (aladwani, 2002). the introduction of technical uncertainty further lowers expectations of achieving good psychological outcomes such as satisfaction (ferris, 1977) since the manager and the team members are not on the same page about either the technical object or the technology to achieve project goals. thus, we hypothesize that: h1b: technical uncertainty mediates the effect of behavior control on psychological outcomes. in the case of outcome control, project managers evaluate team members for their timely and within-budget completion of the project regardless of the process or behaviors followed (kirsch, 1996). team members have more freedom and can complete their responsibilities without much monitoring or having to follow procedures and routines set by the manager. however, the manager evaluates whether the outcomes were met, which are known and measurable (kirsch, 1996). team members can put their knowledge and expertise to full use without any restrictions being placed by the manager. this can lead to a better project outcome since team members have the independence to devise their methods to achieve their targets, the resultant outcome being evaluated by the manager. liu and wang (2014) report that the outcome control used by team members is effective in the process performance of it projects. managers do not provide any procedures or sequences for completion and do not ‘breathe down the neck’ of the developers at every step of the way. outcome controls synergistically increase process performance (rijsdijk & van den ende, 2011). additionally, outcome control australasian journal of information systems raghavan and mahadevan 2024, vol 28, research article achieving project outcomes amidst technical uncertainty 8 should be adopted in technologically complex situations (lin et al., 2019) since team members may have ideas on how to resolve the problem and do not need to worry about much managerial intervention. however, the presence of technical uncertainty can lower the efficiency of the project. thus, we hypothesize that: h2a: technical uncertainty mediates the effect of outcome control on internal efficiency. outcome control helps alleviate uncertainties (kreutzer et al., 2016) since team members can apply their knowledge and skills to resolve the issue on their own and with minimal intervention from management. since the manager gives a free hand to the team to understand the technical objectives and the technology needed to accomplish the goal, team members have the satisfaction that they were able to apply their expertise and know-how to complete the tasks without any external pressure. it professionals in situations of higher outcome control mentioned lower role uncertainty and higher job satisfaction relative to their counterparts in organizations with higher behavior controls (ply et al., 2012). however, the presence of technical uncertainty could result in lower satisfaction since the project’s outcomes may not meet the required levels. thus, we hypothesize that: h2b: technical uncertainty mediates the effect of outcome control on psychological outcomes. clan control is a team phenomenon (kirsch et al., 2010) and takes shape with support from the organization (when team members enter into an informal agreement about the goals, values, and norms for the project. clan control, a type of informal control, functions when behavior is driven by shared values, rules, and common goals (ouchi, 1979). there is a lot of trust among members of the team and some of the processes and procedures are unique to the team and not part of any established procedures in the organization (goldbach & benlian, 2015). such informal contact among the team members and mutual trust leads to a sense of prioritizing team goals over the individual, thus leading to efficient completion of the project. however, in the presence of technical uncertainties, the clan may require time to resolve how to go forward with solving the problem. this can lead to the need for the escalation of the issue to management either for more financial spending on technology or expertise. we hypothesize that: h3a: technical uncertainty mediates the effect of clan control on internal efficiency. members of a clan feel accepted and have a sense of satisfaction that their contributions are acknowledged by their peers. team-based clan control is employed by managers who lack indepth isd experience and knowledge (kirsch et al., 2010). as part of the informal group, they can express their opinions and disagreements about how the project is headed without fear of any reprisal since fellow team members understand that their opinions are for the good of the team. a clan control system is likely to produce high levels of job satisfaction (jaworski et al., 1993). technical uncertainties can lead to intense discussions within the clan on how to approach the issue and could result in tensions among members lowering their satisfaction levels. hence, we hypothesize that: h3b: technical uncertainty mediates the effect of clan control on psychological outcomes. in the case of self-control, each project participant devises a mechanism to achieve their targets, appraise themselves and apply rewards or punishments based on their performance (kirsch & cummings, 1996). the manager plays a minor part in the process of setting goals for the team member. self-control can lead to effective project teams (henderson & lee, 1992). australasian journal of information systems raghavan and mahadevan 2024, vol 28, research article achieving project outcomes amidst technical uncertainty 9 self-control positively impacts the process performance of it projects (liu, 2015b). in the presence of technical uncertainty, an individual who is highly skilled in their line of work will have the opportunity to resolve the situation on their own. the project output can be impacted even though kirsch & cummings (1996) report that the use of self-control is often suggested in a situation that has task complexity and uncertainty. thus, we hypothesize that: h4a: technical uncertainty mediates the effect of self-control on internal efficiency. self-control involves agents regulating their actions by cultivating a coherent sense of themselves (xu et al., 2018). however, projects require that they are part of a group within the organization. it has been found that formal control within complex projects can be balanced by self-control (liu & chua, 2020). at times, organizational and team rules at the client site supplant the goals and objectives of the individual. however, in certain projects where the individual is solely responsible for the delivery of solutions, self-control could provide an effective method and as task complexity increases, professionals’ self-control perceptions increase (kirsch & cummings, 1996). since the individual is responsible for resolving technical uncertainties, there can be situations where they are successful which leads to a feeling of satisfaction and accomplishment in the individual. self-control is related to improved satisfaction or performance (hwang, 2005; kirsch & cummings, 1996). however, the presence of technical uncertainty can push the individual to exert self-control to consider the issue and find a solution. this can provide either a sense of satisfaction or a feeling of worth in the individual if there is a satisfactory project outcome. thus, we hypothesize that: h4b: technical uncertainty mediates the effect of self-control on psychological outcomes. 4 methods we collected data from employees working in isd projects for companies situated in the midwest of the united states. a survey was designed using the scales mentioned in table 1. in addition to the two project outcome measures and four control modes, the survey collected demographic data, such as gender, and information on whether the respondent is a regular team member or a consultant. we enlisted the help of contact persons to distribute the survey to respondents who had recently participated in isd projects that involved technical uncertainty. the survey asked specific questions on how project leaders and team members exercised control behavior in managing the just completed project. the contact persons themselves were students enrolled in isd graduate classes and concurrently employed in isd companies. the graduate students who acted as intermediaries were all from a university situated in the midwest of the united states. they distributed the questionnaire among their colleagues who had recently participated in a development project. this form of convenience sampling was used to ensure that the survey was completed by people who had recently participated in a completed project. an institutional review board approval was obtained, and an informed consent was taken when participants completed the survey. each student received less than 10 questionnaires to avoid overrepresentation from a single organization. no formal reminders were necessary and a total of 210 questionnaires were distributed by the students. a total of 156 responses were received from which 103 usable responses for an overall response rate of 49.04% after 8 weeks that were then used for analysis. the excluded responses did not contain enough information to warrant further analysis. the respondent profiles indicated a good distribution of gender, national origin, team member/consultant mix, and a australasian journal of information systems raghavan and mahadevan 2024, vol 28, research article achieving project outcomes amidst technical uncertainty 10 good distribution of the technical complexity of the projects. following hedt and pagano (2011) strategy to eliminate bias in convenience sampling, we compared the demographics of our sample and verified using secondary data to ensure that it matches the population of software developers on age and gender. table 1 below shows how the constructs were developed and measured. the model was controlled for gender and whether the team participants were team members or consultants. we conducted our mediation analysis using warp pls version 7.0, a variance-based structural equation modeling software. pls has established its capability as a good application for examining human behaviors (cyr et al., 2009) and is appropriate for this study as our theorized model is focused on how project participants react to different control modes to possibly influence project outcomes. we first evaluated the proposed model for model fit using the suggested procedures for pls. the average r-squared (ars) was found to be equal to 0.166, (p=0.020) while the average block vif (avif) and average full collinearity vif (afvif) value was observed to be 1.272 and 1.637 which are well lower than 3.3, the recommended levels for these model fit indices. behavior control (kirsch, 1997; kirsch et. al., 2002) the project manager expected me to follow a written sequence of steps. the project manager assessed the extent written procedures were followed. outcome control (kirsch, 1997; kirsch et. al., 2002) the project manager placed significant weight on project completion to the satisfaction of the client. the project manager evaluated my performance by the extent to which goals were accomplished, regardless of how the goals were accomplished. clan control (kirsch, 1997; kirsch et. al., 2002) the team members had common goals, values, and norms for the project. there was a clear expectation from other team members to do my part of the project well. the project climate was such that there was trust among team members. self-control (kirsch, 1997; kirsch et. al., 2002) project members set specific goals for themselves without the involvement of the project manager. project members defined specific procedures to follow without the involvement of the project manager. it was expected that project members would do their part of the project well without much monitoring by the project manager. routine monitoring of project work was done mostly by the team members. technical uncertainty (lewis et al., 2002) actual experience with the project’s technology. familiarity with technical objectives for the project. familiarity with the technology used in the project. understanding of the technical objectives for the project. psychological outcome (aladwani, 2002) members of our project were satisfied with their work. members were satisfied with the kind of work they did in the project. internal efficiency (banker & kemerer, 1992) the project was delivered in the least possible time. users are satisfied. the system is simple and easily maintainable compared to other projects of this scope. table 1. constructs and measures australasian journal of information systems raghavan and mahadevan 2024, vol 28, research article achieving project outcomes amidst technical uncertainty 11 5 results 5.1 measurement model discriminant validity was assessed by comparing the square of the average variance extracted (ave) correlations between the construct and other constructs (barclay et al., 1995; chin, 1998). as indicated in table 2, the square root of the ave (diagonal elements) exceeded the correlations (off-diagonal elements) between the constructs, confirming adequate discriminant validity. beh oc clan self ie po tu ave cr α beh 0.93 0.029 0.363 -0.06 0.098 0.026 0.014 0.865 0.927 0.844 oc 0.029 0.851 0.305 0.007 0.287 0.224 -0.338 0.725 0.841 0.621 clan 0.363 0.305 0.861 0.336 0.229 0.205 -0.265 0.742 0.896 0.825 self -0.06 0.007 0.336 0.832 0.009 0.002 -0.009 0.693 0.900 0.852 ie 0.098 0.287 0.229 0.009 0.792 0.767 -0.258 0.627 0.834 0.702 po 0.026 0.224 0.205 0.002 0.767 0.874 -0.421 0.764 0.906 0.844 tu 0.014 -0.338 -0.265 -0.009 -0.258 -0.421 0.840 0.705 0.905 0.859 beh -behavior control, oc outcome control, clan clan control, self self control, ie internal efficiency, po psychological outcomes, tu technical uncertainty, aveaverage variance extracted, cr composite reliability, α cronbach alpha table 2. outer model loadings and cross-loadings. in addition, convergent validity was evaluated by comparing the item loadings with the suggested minimum value of 0.6 (chin et al., 1997) as shown in table 3. the item that loaded lowest against its construct was 0.673, sufficiently demonstrating convergent validity. 5.2 structural model the mediation hypotheses were tested using the approach outlined by baron and kenny (1986). according to their method, mediation is established if: (1) the relationship between the independent variable and the dependent variable is significant; (2) the relationship between the independent variable and the presumed mediator is significant; and (3) when controlling for the mediator, the previously significant relationship between the independent variable and the dependent variable diminishes in significance or strength. h1a hypothesized that technical uncertainty significantly mediates the effect of behavior control on internal efficiency. as shown in figure 2, without the mediation of technical uncertainty, the effect of behavior control was not significant (β=0.004, p= 0.48) on internal efficiency. however, with mediation, the link between (1) behavior control and uncertainty (β=0.12, p< 0.10), (2) uncertainty and internal efficiency were significant (β=-0.16, p=0.04), and (3) behavior control and internal efficiency (β= 0.02, p= 0.41) was not significant. thus, h1a is unsupported since technical uncertainty does not mediate the effect of behavior control on internal efficiency. australasian journal of information systems raghavan and mahadevan 2024, vol 28, research article achieving project outcomes amidst technical uncertainty 12 beh oc clan self ie tu po beh1 0.866 0.026 0.365 -0.072 0.114 -0.024 0.074 beh2 0.908 0.026 0.276 -0.032 0.059 0.05 -0.047 oc1 0.195 0.772 0.305 -0.026 0.291 -0.314 0.256 oc2 -0.17 0.877 0.188 0.042 0.172 -0.236 0.120 clan1 0.315 0.206 0.829 0.215 0.226 -0.193 0.180 clan2 0.283 0.257 0.803 0.389 0.111 -0.185 0.101 clan3 0.285 0.277 0.802 0.230 0.210 -0.259 0.179 self1 -0.057 -0.078 0.320 0.932 0.088 0.018 0.054 self2 -0.124 0.001 0.22 0.923 -0.092 0.041 -0.176 self3 -0.028 0.051 0.362 0.921 0.046 -0.108 0.054 self4 -0.002 0.062 0.334 0.883 -0.014 0.020 0.049 ie1 0.130 0.165 0.194 -0.024 0.869 -0.050 0.398 ie2 0.099 0.258 0.194 -0.050 0.673 -0.274 0.587 ie3 0.003 0.197 0.126 0.090 0.729 -0.200 0.602 tu1 -0.027 -0.437 -0.307 0.033 -0.163 0.750 -0.345 tu2 0.017 -0.178 -0.177 -0.037 -0.247 0.872 -0.307 tu3 -0.016 -0.328 -0.249 0.061 -0.175 0.816 -0.312 tu4 0.069 -0.198 -0.155 -0.083 -0.264 0.868 -0.282 po1 0.029 0.165 0.178 -0.028 0.541 -0.305 0.741 po2 -0.004 0.187 0.109 0.018 0.558 -0.256 0.758 table 3. measurement model assessment h1b hypothesized that technical uncertainty significantly mediates the effect of behavior control on psychological outcomes. as shown in figure 3, without the mediation of technical uncertainty, the effect of behavior control was not significant (β=-0.10, p=0.14) on psychological outcomes. however, with mediation, the link between (1) behavior control and uncertainty (β=0.12, p=0.10), (2) uncertainty and psychological outcomes was significant (β=0.37, p<0.01), and (3) behavior control and psychological outcomes (β=-0.05, p=0.30) were not significant. thus, h1b is unsupported since technical uncertainty does not mediate the effect of behavior control on psychological outcomes. figure 2: h1a australasian journal of information systems raghavan and mahadevan 2024, vol 28, research article achieving project outcomes amidst technical uncertainty 13 h2a hypothesized that technical uncertainty significantly mediates the effect of outcome control on internal efficiency. as shown in figure 4, without the mediation of technical uncertainty, the effect of outcome control was significant (β=0.25, p<.01) on internal efficiency. however, with mediation, the link between 1) outcome control and uncertainty (β=-0.26, p<0.10), 2) uncertainty and internal efficiency was significant (β=-0.16, p=0.04) and 3) outcome control and internal efficiency (β=0.21, p=0.01) were significant. thus, h2a is supported since technical uncertainty partially mediates the relationship between outcome control and internal efficiency. figure 3: h1b figure 4: h2a h2b hypothesized that technical uncertainty significantly mediates the effect of outcome control on psychological outcomes. as shown in figure 5, without the mediation of technical uncertainty, the effect of outcome control was significant (β=0.20, p=0.02) on psychological outcomes. however, with mediation, the link between 1) outcome control and uncertainty (β=-0.26, p<0.01), 2) uncertainty and psychological outcome control was significant (β=-0.37, p<0.01), and 3) outcome control and psychological outcomes (β=0.09, p=0.18) was not australasian journal of information systems raghavan and mahadevan 2024, vol 28, research article achieving project outcomes amidst technical uncertainty 14 significant. thus, h2b is supported since technical uncertainty fully mediates the relationship between outcome control and psychological outcomes. figure 5: h2b h3a hypothesized that technical uncertainty significantly mediates the effect of clan control on internal efficiency. as shown in figure 6, without the mediation of technical uncertainty, the effect of clan control was significant (β=0.16, p=0.04) on internal efficiency. however, with mediation, the link between 1) clan control and uncertainty (β=-0.26, p<0.01), 2) uncertainty and internal efficiency was significant (β=-0.16, p<0.05), and 3) clan control and internal efficiency (β=0.12, p=0.10) was significant. thus, h3a is supported since technical uncertainty partially mediates the relationship between clan control and internal efficiency. figure 6: h3a h3b hypothesized that technical uncertainty significantly mediates the effect of clan control on psychological outcomes. as shown in figure 7, without the mediation of technical australasian journal of information systems raghavan and mahadevan 2024, vol 28, research article achieving project outcomes amidst technical uncertainty 15 uncertainty, the effect of clan control was significant (β=0.19, p=0.02) on psychological outcomes. however, with mediation, the link between 1) clan control and uncertainty (β=0.26, p<0.01), 2) uncertainty and the psychological outcome was significant (β=-0.37, p<0.01), and 3) clan control and psychological outcomes (β=0.11, p=0.13) was not significant. thus, h2b is supported since technical uncertainty fully mediates the relationship between clan control and psychological outcomes. figure 7: h3b h4a hypothesized that technical uncertainty significantly mediates the effect of self-control on internal efficiency. as shown in figure 8, without the mediation of technical uncertainty, the effect of self-control was not significant (β=-0.04, p=0.35) on internal efficiency. however, with mediation, the link between 1) self-control and uncertainty (β=0.09, p=0.18) was not significant, 2) uncertainty and internal efficiency were significant (β=-0.16, p<0.05), and 3) selfcontrol and internal efficiency (β=-0.02, p=0.41) was not significant. thus, h4a is unsupported since technical uncertainty does not significantly mediate the relationship between selfcontrol on internal efficiency. figure 8: h4a australasian journal of information systems raghavan and mahadevan 2024, vol 28, research article achieving project outcomes amidst technical uncertainty 16 h4b hypothesized that technical uncertainty significantly mediates the effect of self-control on psychological outcomes. as shown in figure 9, without the mediation of technical uncertainty, the effect of self-control was not significant (β=-0.06, p=0.35) on psychological outcomes. however, with mediation, the link between 1) self-control and uncertainty (β=0.09, p=0.18) was not significant, 2) uncertainty and the psychological outcome was significant (β=0.37, p<0.01), and 3) self-control and psychological outcomes (β=-0.02, p=0.39) was not significant. thus, h4b is unsupported since technical uncertainty does not significantly mediate the effect of self-control on psychological outcomes. table 4. summarizes the results of hypothesis tested in this study. figure 9: h4b hypothesis independent variable dependent variable result h1a behavior control internal efficiency not supported h1b behavior control psychological outcomes not supported h2a outcome control internal efficiency supported h2b outcome control psychological outcomes supported h3a clan control internal efficiency supported h3b clan control psychological outcomes supported h4a self-control internal efficiency not supported h4b self-control psychological outcomes not supported table 4. results of hypothesis testing. (mediated by technical uncertainty) 6 discussion the ever-changing technology landscape presents immense challenges to organizations, mainly in the form of technical uncertainty as they attempt to keep up with the changes and derive value. in this study, we examined the control modes best suited to achieve internal australasian journal of information systems raghavan and mahadevan 2024, vol 28, research article achieving project outcomes amidst technical uncertainty 17 efficiency and psychological outcomes in the presence of technical uncertainty. the successful completion of any organizational endeavor relies on 1) internal efficiencies (shenhar et al.,1997; baccarini, 1999), which are the effectiveness of an organization’s internal process and 2) psychological outcomes, which refers to the team members’ satisfaction level with the work required to be completed for the isd project (aladwani, 2002). analysis of responses from team members based in the us reveals that, out of the four control modes, clan and outcome control modes are best suited to achieve internal efficiencies and psychological outcomes in the presence of technical uncertainty. the results of this study provide new theoretical insights for the isd project management literature and generate practical implications for managers in implementing technically uncertain projects effectively. 6.1 theoretical implications our research contributes to the isd project literature by studying the effects of the four control modes on two project outcomes: psychological outcomes and internal efficiency. as theorized, our results show that clan and outcome control were the two control modes appropriate to realize project outcomes when the relationships are mediated by technical uncertainty. our findings show that technical uncertainty 1) fully mediated the relationship between clan and outcome control with psychological outcomes, 2) partially mediated the relationships between clan and outcome control with internal efficiency and 3) is not relevant in the case of behavior and self-control since these two control modes had no significant direct relationship with the project outcomes even in the absence of technical uncertainty. earlier studies using control theory have shown the impact of behavior, outcome, self and clan control on project performance (keil et al., 2013) and how clan and outcome control are effective for project performance amidst high complexity risk (liu, 2015a). our findings demonstrate that outcome and clan control are significant antecedents of psychological outcomes and that both control modes have no significant outcome on psychological outcomes over and above the effects mediated by technical uncertainty. in the case of psychological outcome, clan and outcome control retain their significant relationship with internal efficiency over and above the effects mediated by technical uncertainty. thus, the project outcomes depend on the freedom provided by the controller to the controlees to provide the solution (e.g., outcome control) or the following of shared values and informal agreed-upon behaviors (e.g., clan control). our results show that clan and outcome control lose their direct effect on psychological outcomes in the presence of technical uncertainty. however, the control modes significantly lessen technical uncertainty which in turn reduces the quality of psychological outcomes. in the case of internal efficiency, the control modes (clan and outcome) still retain some of their direct effect even while reducing technical uncertainty. however technical uncertainty results in reduced internal efficiency. no support for the relationship between behavior control with the project outcomes could be due to the lack of importance for effectiveness and satisfaction as criteria for formal documentation and procedures prescribed to the isd project. similarly, in the case of selfcontrol, the controlee may be focused more on getting the project completed and may not be particularly interested in deriving satisfaction or ensuring process effectiveness. 6.2 managerial implications the study of technical uncertainty in isd project control is not just a proactive measure but a strategic imperative. it empowers organizations to optimize project outcomes, enhance australasian journal of information systems raghavan and mahadevan 2024, vol 28, research article achieving project outcomes amidst technical uncertainty 18 resource efficiency, and foster a culture of innovation. driven by the need for innovation and competitive advantage, organizations initiate projects that incorporate new technologies. integration of such software and hardware can result in technical uncertainty since team members have limited training or are unsure how such technologies integrate with the existing infrastructure. our results show that both clan and outcome control deliver internal efficiency and psychological outcomes, which are highly desired by organizations. most team members have the knowledge and are quite skilled at managing uncertainties posed by new technologies. though they lack familiarity with the newly introduced technology, they bring their experience and learning to find solutions with innovative technologies. our findings show that managers could employ outcome and clan control since they provide modes for such experienced team members to work with their peers to provide new solutions using innovative technology. team members have the freedom to discuss issues and workarounds without the intervention of their controller(s). our results show that regardless of use of either clan or outcome control, managers shouldn’t expect increased internal efficiency and psychological outcomes in the presence of technical uncertainty. in the case of internal efficiency, both clan and outcome control reduce technical uncertainty. although technical uncertainty results in reduced internal efficiency, certain aspects of previously adopted procedures of achieving internal efficiency developed in the absence of technical uncertainty still apply. however, managers should note that previous ways of achieving psychological outcomes are not effective in the presence of technical uncertainty in the case of both outcome and clan control. though both clan and outcome control mode reduce technical uncertainty, satisfaction is lowered and only related to how the technical uncertainty is resolved. this is probably due to the complexity associated with implementing the new technology (johnson et al., 2008). team members may find less satisfaction having to adapt the innovative technology to implement the solution. interestingly, our results support the use of both formal and informal modes of control which may help organizations that have more controller-oriented culture to decide using outcome control compared to clan control which can be used by organizations that provide more freedom to the controlee. furthermore, the findings indicate that managers should refrain from implementing behavior or self-control modes, as these modes demonstrate no significant impact on internal efficiency or psychological outcomes, even in the absence of technical uncertainty. this could stem from the resistance of experienced team members to adhere strictly to protocols and procedures, especially when they possess their own ideas and concepts for resolving issues. 6.3 limitations and future research our study is not without limitations. first, our study surveyed team members who possibly work on different projects of various complexities. the complexity of the project is a dimension that should be explored in future studies. second, our study sample raises concerns about common method bias. this can be resolved by a future study where a controllercontrolee pairwise design is applied to understand the phenomenon. third, culture can influence how team members respond to control modes. future research could undertake a comparative study of team members in different cultures to understand which control modes are best suited to reduce technical uncertainty. fourth, technical uncertainty in different industry verticals could be researched to unearth more definitive methods to apply suitable control models. fifth, research on different levels of technical uncertainty also could provide australasian journal of information systems raghavan and mahadevan 2024, vol 28, research article achieving project outcomes amidst technical uncertainty 19 a richer understanding of the interaction between the control model with psychological outcomes and internal efficiency. 7 conclusion as the isd landscape continues to evolve, understanding and addressing technical uncertainties will be a defining factor in the success of projects and the competitiveness of organizations in the digital era. in this technically uncertain environment, mainly created by the focus on building innovative solutions, the issue of control modes between the management and the team is an issue of importance. project efficiencies and psychological outcomes play a critical role in the successful completion of any organizational endeavor. the objective of this study was to comprehend how technical uncertainty mediates the effect of different control types on internal efficiency and psychological outcomes. given that projects face technical uncertainty and apply different control tactics, the identification of outcome and clan control having a favorable influence on technical uncertainty helps organizations maintain their competitive advantage and team member morale. clan control is shaped by interactions among team members towards achieving project goals through peer sanctioned norms, rituals, and ceremonies (chua et al., 2012). in the case of outcome control, the controller evaluates the output without much regard to the process used to achieve it (choudhary & sabherwal, 2003). the results indicate that in technically uncertain conditions, companies should let the team decide and figure out the solution, implying reliance on clan control and outcome control. thus, the guidance our results provide is to rely less on behavior and selfcontrol but more on outcome and clan control. references aladwani, a. m. (2002). an integrated performance model information systems project. journal of management information systems, 19(1), 185–210. doi.org/10.1177/87569728221089831 atanassov, j., julio, b., & leng, t. (2024). the bright side of political uncertainty: the case of r&d. the review of financial studies, hhae023. doi.org/10.1093/rfs/hhae023 baccarini, d. (1999). the logical framework method for defining project success. project management journal, 30(4), 25–32. doi.org/10.1177/875697289903000405 bakos, j. y. (1991). a strategic analysis of electronic marketplaces. mis quarterly, 295–310. https://www.jstor.org/stable/249641 banker, r. d., & kemerer, c. f. (1992). performance evaluation metrics for information systems development: a principal-agent model. information systems research, 3(4), 379400. doi.org/10.1287/isre.3.4.379 barclay, d., higgins, c., & thompson, r. (1995). the partial least squares (pls) approach to casual modeling: personal computer adoption ans use as an illustration. technology studies, 2, 285–309. baron, r. m., & kenny, d. a. (1986). the moderator–mediator variable distinction in social psychological research: conceptual, strategic, and statistical considerations. journal of personality and social psychology, 51(6), 1173. https://psycnet.apa.org/doi/10.1037/00223514.51.6.1173 australasian journal of information systems raghavan and mahadevan 2024, vol 28, research article achieving project outcomes amidst technical uncertainty 20 beese, j., haki, k., schilling, r., kraus, m., aier, s., & winter, r. (2023). strategic alignment of enterprise architecture management–how portfolios of control mechanisms track a decade of enterprise transformation at commerzbank. european journal of information systems, 32(1), 92-105. doi.org/10.1080/0960085x.2022.2085200 chen, w., huang chua, c. e., young, r., & xu, x. (2022). explaining reverse outcome tight control: a case study of mindless/mindful governance. project management journal, 53(3), 309–324. doi.org/10.1177/87569728221091484 cheng, q., goh, b. w., & kim, j. b. (2018). internal control and operational efficiency. contemporary accounting research, 35(2), 1102-1139. doi.org/10.1111/19113846.12409 chien, s. w., hu, c., reimers, k., & lin, j. s. (2007). the influence of centrifugal and centripetal forces on erp project success in small and medium-sized enterprises in china and taiwan. international journal of production economics, 107(2), 380–396. doi.org/10.1016/j.ijpe.2006.10.002 chin, w. w. (1998). the partial least squares approach to structural equation modeling. modern methods for business research/ lawrence erlbaum associates. chin, w. w., gopal, a., & salisbury, w. d. (1997). advancing the theory of adaptive structuration: the development of a scale to measure faithfulness of appropriation. information systems research, 8(4), 342-367. doi.org/10.1287/isre.8.4.342 choudhury, v., & sabherwal, r. (2003). portfolios of control in outsourced software development projects. information systems research, 14(3), 291–314. doi.org/10.1287/isre.14.3.291.16563 chua, c. e. h., lim, w. k., soh, c., & sia, s. k. (2012). enacting clan control in complex it projects: a social capital perspective. mis quarterly, 577-600. doi.org/10.2307/41703468 codex, a. c. (2023, june 16). managing the challenges of software development projects with high levels of technical uncertainty. reintech media. https://reintech.io/blog/managingchallenges-of-software-development-with-technical-uncertainty cram, w. a., brohman, k., & gallupe, r. b. (2016). information systems control: a review and framework for emerging information systems processes. journal of the association for information systems, 17(4), 2. https://aisel.aisnet.org/jais/vol17/iss4/2/ cyr, d., head, m., larios, h., & pan, b. (2009). exploring human images in website design: a multi-method approach. mis quarterly, 539–566. doi.org/10.2307/20650308 dasgupta, m., gupta, r. k., & sahay, a. (2011). linking technological innovation, technology strategy and organizational factors: a review. global business review, 12(2), 257–277. doi.org/10.1177/097215091101200206 deci, e. l., & ryan, r. m. (1987). the support of autonomy and the control of behavior. journal of personality and social psychology, 53(6), 1024. dönmez, d., & grote, g. (2018). two sides of the same coin–how agile software development teams approach uncertainty as threats and opportunities. information and software technology, 93, 94-111. doi.org/10.1016/j.infsof.2017.08.015 australasian journal of information systems raghavan and mahadevan 2024, vol 28, research article achieving project outcomes amidst technical uncertainty 21 duan, w., gu, b., & whinston, a. b. (2009). informational cascades and software adoption on the internet: an empirical investigation. mis quarterly, 23–48. doi.org/10.2307/20650277 eisenhardt, k. m. (1985). control: organizational and economic approaches. management science, 31(2), 134-149. doi.org/10.1287/mnsc.31.2.134 ens, n., hukal, p., & blegind jensen, t. (2023). dynamics of control on digital platforms. information systems journal, 33(4), 890–911. doi.org/10.1111/isj.12429 ferris, k. r. (1977). perceived uncertainty and job satisfaction in the accounting environment. accounting, organizations and society, 2(1), 23–28. doi.org/10.1016/03613682(77)90004-6 gerber, e. (2009). prototyping: facing uncertainty through small wins. in ds 58-9: proceedings of iced 09, the 17th international conference on engineering design, vol. 9, human behavior in design, palo alto, ca, usa, 24.-27.08. 2009, 333–342. gerber, e., & carroll, m. (2012). the psychological experience of prototyping. design studies, 33(1), 64-84. doi.org/10.1016/j.destud.2011.06.005 gnanasambandam, c., miller, a., & sprague, k. (2018, april 9). grow fast or die slow: the role of profitability in sustainable growth. mckinsey. https://www.mckinsey.com/industries/technology-media-andtelecommunications/our-insights/grow-fast-or-die-slow-the-role-of-profitability-insustainable-growth goldbach, t., & benlian, a. (2015). how informal control modes affect developers' trust in a platform vendor and platform stickiness. wirtschaftsinformatik proceedings 2015. 89. https://aisel.aisnet.org/wi2015/89 harris, m., hevner, a. r., & collins, r. w. (2006, january). controls in flexible software development. in proceedings of the 39th annual hawaii international conference on system sciences (hicss'06) (vol. 9, pp. 216a–216a). ieee. https://ieeexplore.ieee.org/abstract/document/1579732/citations heales, j., susilo, a., & rohde, f. (2007). project management effectiveness: the choice-formal or informal controls. australasian journal of information systems, 15(1). doi.org/10.3127/ajis.v15i1.480 hedt, b. l., & pagano, m. (2011). health indicators: eliminating bias from convenience sampling estimators. statistics in medicine, 30(5), 560-568. doi.org/10.1002/sim.3920 henderson, j. c., & lee, s. (1992). managing i/s design teams: a control theories perspective. management science, 38(6), 757-777. doi.org/10.1287/mnsc.38.6.757 henry, r. m., narayanaswamy, r., & purvis, r. l. (2015). effect of control on information systems development performance: a meta-analysis. journal of computer information systems, 55(3), 46–54. doi.org/10.1080/08874417.2015.11645771 holland, w. e., stead, b. a., & leibrock, r. c. (1976). information channel/source selection as a correlate of technical uncertainty in a research and development organization. ieee transactions on engineering management, (4), 163–167. https://ieeexplore.ieee.org/abstract/document/6447184 australasian journal of information systems raghavan and mahadevan 2024, vol 28, research article achieving project outcomes amidst technical uncertainty 22 hwang, y. (2005). investigating enterprise systems adoption: uncertainty avoidance, intrinsic motivation, and the technology acceptance model. european journal of information systems, 14(2), 150-161. doi.org/10.1057/palgrave.ejis.3000532 iacovou, c., smith, j., & thompson, r. (2005). the linkage between reporting quality and performance in information systems projects. icis 2005 proceedings. 34. https://aisel.aisnet.org/icis2005/34 jain, n. k., celo, s., & kumar, v. (2019). internationalization speed, resources and performance: evidence from indian software industry. journal of business research, 95, 26-37. doi.org/10.1016/j.jbusres.2018.09.019 jalonen, h. (2012). the uncertainty of innovation: a systematic review of the literature. journal of management research, 4(1), 1-47. http://dx.doi.org/10.5296/jmr.v4i1.1039 jaworski, b. j. (1988). toward a theory of marketing control: environmental context, control types, and consequences. journal of marketing, 52(3), 23–39. doi.org/10.1177/002224298805200303 jaworski, b. j., & macinnis, d. j. (1989). marketing jobs and management controls: toward a framework. journal of marketing research, 26(4), 406–419. doi.org/10.1177/002224378902600403 jaworski, b. j., stathakopoulos, v., & krishnan, h. s. (1993). control combinations in marketing: conceptual framework and empirical evidence. journal of marketing, 57(1), 57-69. doi.org/10.1177/002224299305700104 johnson, d. s., bardhi, f., & dunn, d. t. (2008). understanding how technology paradoxes affect customer satisfaction with self‐service technology: the role of performance ambiguity and trust in technology. psychology & marketing, 25(5), 416–443. doi.org/10.1002/mar.20218 jun, l., qiuzhen, w., & qingguo, m. (2011). the effects of project uncertainty and risk management on is development project performance: a vendor perspective. international journal of project management, 29(7), 923–933. doi.org/10.1016/j.ijproman.2010.11.002 jurado, k., ludvigson, s. c., & ng, s. (2015). measuring uncertainty. american economic review, 105(3), 1177-1216. https://www.aeaweb.org/articles?id=10.1257/aer.20131193 kanwal, n., zafar, m. s., & bashir, s. (2017). the combined effects of managerial control, resource commitment, and top management support on the successful delivery of information systems projects. international journal of project management, 35(8), 1459– 1465. doi.org/10.1016/j.ijproman.2017.08.007 keil, m., rai, a., & liu, s. (2013). how user risk and requirements risk moderate the effects of formal and informal control on the process performance of it projects. european journal of information systems, 22(6), 650–672. doi.org/10.1057/ejis.2012.42 kirsch, l. j. (1996). the management of complex tasks in organizations: controlling the systems development process. organization science, 7(1), 1-21. doi.org/10.1287/orsc.7.1.1 australasian journal of information systems raghavan and mahadevan 2024, vol 28, research article achieving project outcomes amidst technical uncertainty 23 kirsch, l. j., & cummings, l. l. (1996). contextual influences on self-control of is professionals engaged in systems development. accounting, management and information technologies, 6(3), 191-219. doi.org/10.1016/0959-8022(96)00018-5 kirsch, l. j., ko, d. g., & haney, m. h. (2010). investigating the antecedents of team-based clan control: adding social capital as a predictor. organization science, 21(2), 469–489. doi.org/10.1287/orsc.1090.0458 kirsch, l. j., sambamurthy, v., ko, d. g., & purvis, r. l. (2002). controlling information systems development projects: the view from the client. management science, 48(4), 484– 498. doi.org/10.1287/mnsc.48.4.484.204 kirsch, l. s. (1997). portfolios of control modes and is project management. information systems research, 8(3), 215–239. doi.org/10.1287/isre.8.3.215 klir, g. j. (2006). uncertainty and information: foundations of generalized information theory. kybernetes, 35(7/8), 1297–1299. doi.org/10.1108/03684920610675283 kreutzer, m., cardinal, l. b., walter, j., & lechner, c. (2016). formal and informal control as complement or substitute? the role of the task environment. strategy science, 1(4), 235– 255. doi.org/10.1287/stsc.2016.0019 laine, t., korhonen, t., & martinsuo, m. (2016). managing program impacts in new product development: an exploratory case study on overcoming uncertainties. international journal of project management, 34(4), 717-733. doi.org/10.1016/j.ijproman.2016.02.011 lewis, m. w., welsh, m. a., dehler, g. e., & green, s. g. (2002). product development tensions: exploring contrasting styles of project management. academy of management journal, 45(3), 546-564. doi.org/10.5465/3069380 lin, l., müller, r., zhu, f., & liu, h. (2019). choosing suitable project control modes to improve the knowledge integration under different uncertainties. international journal of project management, 37(7), 896-911. doi.org/10.1016/j.ijproman.2019.07.002 liu, g. h. w., & chua, c. e. h. (2020). the reinforcing effects of formal control enactment in complex it projects. journal of the association for information systems, 21(2). doi.org/10.17705/1jais.00603 liu, s. (2015a). effects of control on the performance of information systems projects: the moderating role of complexity risk. journal of operations management, 36, 46–62. doi.org/10.1016/j.jom.2015.03.003 liu, s. (2015b). how team risk and planning and control risk moderate the effects of clan and self control on the process performance of it projects: the perspective of user liaisons. information development, 31(1), 27–39. doi.org/10.1177/0266666913501682 liu, s., & wang, l. (2014). user liaisons’ perspective on behavior and outcome control in it projects: role of it experience, behavior observability, and outcome measurability. management decision, 52(6), 1148–1173. doi.org/10.1108/md-08-20130430 lynn, g. s., & akgün, a. e. (1998). innovation strategies under uncertainty: a contingency approach for new product development. engineering management journal, 10(3), 11–18. doi.org/10.1080/10429247.1998.11414991 australasian journal of information systems raghavan and mahadevan 2024, vol 28, research article achieving project outcomes amidst technical uncertainty 24 maes, t., gebhardt, k., & riel, a. (2022). the relationship between uncertainty and task execution strategies in project management. project management journal, 53(4), 382–396. doi.org/10.1177/87569728221089831 marinho, m., sampaio, s., & moura, h. (2018). managing uncertainty in software projects. innovations in systems and software engineering, 14, 157–181. doi.org/10.1007/s11334-017-0297-y mccarthy, s., o'raghallaigh, p., li, y., & adam, f. (2023). control enactment in context: understanding the interaction of controlee and controller perceptions in inter‐ organisational project teams. information systems journal, 33(5), 1029–1084. doi.org/10.1111/isj.12434 mehta, n., hall, d., & byrd, t. (2014). information technology and knowledge in software development teams: the role of project uncertainty. information & management, 51(4), 417-429. doi.org/10.1016/j.im.2014.02.007 mullen, j. (2023, april 25). 2023 insight intelligent technology report: are we entering a new era of innovation? insight. https://www.insight.com/en_us/content-andresources/gated/insight-intelligent-technology-report-ac1252.html nidumolu, s. r., & subramani, m. r. (2003). the matrix of control: combining process and structure approaches to managing software development. journal of management information systems, 20(3), 159–196. doi.org/10.1080/07421222.2003.11045774 ouchi, w. g. (1979). a conceptual framework for the design of organizational control mechanisms. management science, 25(9), 833–848. doi.org/10.1287/mnsc.25.9.833 patterson, p. g., johnson, l. w., & spreng, r. a. (1997). modeling the determinants of customer satisfaction for business-to-business professional services. journal of the academy of marketing science, 25, 4–17. doi.org/10.1007/bf02894505 piccoli, g., & ives, b. (2003). trust and the unintended effects of behavior control in virtual teams. mis quarterly, 365–395. doi.org/10.2307/30036538 ply, j. k., moore, j. e., williams, c. k., & thatcher, j. b. (2012). is employee attitudes and perceptions at varying levels of software process maturity. mis quarterly, 601–624. doi.org/10.2307/41703469 ragatz, g. l., handfield, r. b., & petersen, k. j. (2002). benefits associated with supplier integration into new product development under conditions of technology uncertainty. journal of business research, 55(5), 389–400. doi.org/10.1016/s01482963(00)00158-2 remus, u., wiener, m., saunders, c., & mähring, m. (2020). the impact of control styles and control modes on individual-level outcomes: a first test of the integrated is project control theory. european journal of information systems, 29(2), 134–152. doi.org/10.1080/0960085x.2020.1718008 rijsdijk, s. a., & van den ende, j. (2011). control combinations in new product development projects. journal of product innovation management, 28(6), 868-880. doi.org/10.1111/j.15405885.2011.00850.x australasian journal of information systems raghavan and mahadevan 2024, vol 28, research article achieving project outcomes amidst technical uncertainty 25 saxena, d., & mcdonagh, j. (2020). the evolving nature of information systems controls in healthcare organisations: the case of a blood banking enterprise system from western europe. australasian journal of information systems, 24. doi.org/10.3127/ajis.v24i0.2635 shenhar, a. j., & dvir, d. (2007). reinventing project management: the diamond approach to successful growth and innovation. harvard business review press. shenhar, a. j., levy, o., & dvir, d. (1997). mapping the dimensions of project success. project management journal, 28, 5-13. https://www.wcu.edu/pmi/1998/j97jun05.pdf song, m., & montoya-weiss, m. m. (2001). the effect of perceived technological uncertainty on japanese new product development. academy of management journal, 44(1), 61–80. doi.org/10.5465/3069337 spanjol, j., xiao, y., & welzenbach, l. (2018). successive innovation in digital and physical products: synthesis, conceptual framework, and research directions. innovation and strategy, 15, 31–62. doi.org/10.1108/s1548-643520180000015004 stewart, k. j., & gosain, s. (2006). the impact of ideology on effectiveness in open source software development teams. mis quarterly, 291–314. doi.org/10.2307/25148732 stock, g. n., & tatikonda, m. v. (2008). the joint influence of technology uncertainty and interorganizational interaction on external technology integration success. journal of operations management, 26(1), 65–80. doi.org/10.1016/j.jom.2007.04.003 townsend, a. m., demarie, s. m., & hendrickson, a. r. (1998). virtual teams: technology and the workplace of the future. academy of management perspectives, 12(3), 17–29. doi.org/10.5465/ame.1998.1109047 webster, n. (1916). webster's collegiate dictionary. g. & c. merriam company. ward, s., & chapman, c. (2003). transforming project risk management into project uncertainty management. international journal of project management, 21(2), 97–105. doi.org/10.1016/s0263-7863(01)00080-1 wiener, m., mähring, m., remus, u., & saunders, c. (2016). control configuration and control enactment in information systems projects. mis quarterly, 40(3), 741–774. https://www.jstor.org/stable/26629036 wright, p. m., george, j. m., farnsworth, s. r., & mcmahan, g. c. (1993). productivity and extra-role behavior: the effects of goals and incentives on spontaneous helping. journal of applied psychology, 78(3), 374. https://psycnet.apa.org/buy/1993-37472-001 xu, y., tong, y., & liao, s. s. (2018, january). self-control matters: examining indirect use of hospital information systems and its control mechanisms. in acis (p. 26). yu, s. h., & chiu, w. t. (2013). social networks and corporate performance: the moderating role of technical uncertainty. journal of managerial issues, 26–45. https://www.jstor.org/stable/43488156 copyright copyright © 2024 raghavan and mahadevan this is an open-access article licensed under a creative commons attribution-non-commercial 4.0 australia license, which permits nonhttps://creativecommons.org/licenses/by-nc/4.0/deed.en australasian journal of information systems raghavan and mahadevan 2024, vol 28, research article achieving project outcomes amidst technical uncertainty 26 commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v28.4669 https://doi.org/10.3127/ajis.v28.4669 paper title (paper title style) australasian journal of information systems bandara, leyer and rosemann 2025, vol 29., provocation from automation to autonomisation 1 accepted 07 sep 2025, after 1 round of review with michael davern as accepting editor in chief.   from automation to autonomisation: how to capitalise on empowered processes wasana bandara* school of information systems queensland university of technology brisbane, australia email: w.bandara@qut.edu.au michael leyer chair of digitisation and process management university of marburg marburg, germany school of management queensland university of technology brisbane, australia michael rosemann centre for future enterprise school of management queensland university of technology brisbane, australia abstract process automation has been a cornerstone of organisational success, driving competitiveness through enhanced effectiveness and efficiency. while automation remains a key focus, the rise of artificial intelligence (ai), machine learning (ml) and the internet of things (iot) provides entirely new process design options and as such demands new skills and approaches in business process management (bpm) to maintain relevance and competitiveness. this paper extends current bpm paradigms by introducing the concept of “process autonomisation,” a new paradigm that empowers organisations to make decisions autonomously using real-time data and adaptive business contexts. by critically examining existing bpm methodologies and highlighting key innovations, we propose strategies for evolving process management frameworks emphasising agility, data-driven decision-making, and seamless digital integration. to thrive in the digital age, we argue, organisations must rethink their bpm practices. we call on both academic and industry stakeholders to collaborate on pioneering advancements in bpm and co-develop a roadmap that will help organisations align their process management with the ongoing digital transformation reshaping industries. key words: business process management, process design, process autonomisation, agile bpm, data-driven-bpm, artificial intelligence. australasian journal of information systems bandara, leyer and rosemann 2025, vol 29., provocation from automation to autonomisation 2 1 introduction business process management (bpm) is integral to information systems (is), bridging the gap between technology and organisational processes. echoing socio-technical is theory, we treat bpm not as technology deployment but as the joint design of social and technical elements, so that processes, people, and it co-evolve toward business goals (alter, 2008; mumford, 2006). bpm focuses on optimising business processes to improve efficiency, agility, and innovation. however, despite the significant advances in bpm and its role in is, traditional bpm approaches are increasingly being questioned for their continued relevance and effectiveness, especially when compared to other technological investments (jeston & nelis, 2008). this paper argues that to remain relevant and effective, bpm must evolve by incorporating autonomisation and leveraging emerging technologies to better align with contemporary business needs and the rapid pace of technological transformation. at the same time, despite substantial investment, many process changes don’t stick. staff create local “workarounds,” exposing misfits between formal processes and real work, and with this underscore the need for bpm to pair design-time standards with the way processes are executed. while emerging technologies such as artificial intelligence (ai), machine learning (ml), and the internet of things (iot) open new possibilities, their integration needs to be guided by the organisation’s strategic priorities and stakeholder needs. from a bpm perspective, the value of ai, ml, and iot is maximised when embedded into the process lifecycle spanning design, implementation, execution, monitoring, and continuous improvement with governance structures that align technology use to organisational goals and stakeholder needs (rosemann et al., 2024). thus, rather than an ‘ai-first’ mantra, organisations need to follow a ‘process-first’ approach. ultimately, business processes are the source of organisational value propositions, and contemporary technologies are here to serve processes in creating and sustaining value whether this relates to customer experience, operational resilience, regulatory compliance, or sustainability. technology serves as the means to address these objectives, rather than the primary driver of change. bpm has a long-standing tradition in organisational management, with roots extending over a century (leyer et al., 2017; nordsieck, 1931). historically, bpm relied on manual methods such as pen, paper, and stopwatches, primarily for observation and time-based analysis. these early approaches were labour-intensive, requiring significant human effort for documenting and tracking processes. methodologies such as fordism, total quality management, six sigma and lean management helped to develop structured, automated processes (harmon, 2019). the fundamental work of hammer and champy (1993) in process redesign marked a turning point in rethinking process designs. van der aalst and van hee (1995) introduced processaware information systems (such as workflow management) to model, execute and analyse business processes. this shift offered more accurate and efficient representations of business processes in digital environments. this gave rise to a data-driven and automated approach to bpm. process mining, for example, has become a key method for converting digital traces into process descriptions providing valuable insights to improve process efficiency and ensure compliance. however, despite these advancements, current bpm methods still do not fully leverage the potential of emerging technologies, particularly ai, ml and iot and are largely used to achieve established process productivity goals meaning they are under-utilised. this is where we propose the new notion of ‘process autonomisation’ to truly capture the change in technology support for business australasian journal of information systems bandara, leyer and rosemann 2025, vol 29., provocation from automation to autonomisation 3 processes that is catalysed by these technologies. autonomisation means systems (not just humans) can make decisions about how a process is executed (rosemann et al., 2024). more specifically, this means the delegation of process decision-making to intelligent systems that can independently analyse data, adapt to business context, and determine how processes are executed in real time. autonomisation represents the next frontier in bpm, where decision-making and governance of business processes are delegated to intelligent, framed systems capable of analysing broader datasets and making dynamic decisions (dumas et al., 2023). this shift allows for more agile, responsive, and self-optimising processes that can adapt in real-time to changes in the business environment. as organisations increasingly navigate the complexities of a digitised, dataintensive environment, bpm methods must evolve to integrate such autonomous capabilities (rosemann et al., 2024). unlike general is approaches, bpm provides a distinctive and integrated capability set; combining end-to-end process orchestration (dumas et al., 2018), maturity frameworks for capability assessment (e.g., kerpedzhiev et al., 2021), and sociotechnical integration patterns that align people, processes, and technology to deliver measurable, process-centric outcomes. these strengths position bpm as a discipline uniquely equipped to ensure that emerging technologies such as ai, ml, and iot are embedded within process architectures in ways that are business-driven, outcome-focused, and adaptable to both operational and contextual dynamics (grisold et al., 2024). we propose a forward-looking research agenda and call upon scholars and practitioners to critically re-assess and transform bpm practices, ensuring they remain relevant and effective in the face of rapidly evolving technological advancements and shifting organisational dynamics. 2 current state of bpm methods and lost opportunities business process management (bpm) has made significant strides from its early manual approaches (stage 1) to more digital and automated systems (stage 2) – see figure 1. however, despite these advancements, current bpm methods still fail to fully capitalise on emerging technologies, particularly ai, ml and iot, which are reshaping industries today. figure 1: emerging paradigms for bpm methods in the digital age australasian journal of information systems bandara, leyer and rosemann 2025, vol 29., provocation from automation to autonomisation 4 historically, business process management (bpm) relied on manual methods like pen, paper, and stopwatches to track and optimise processes (barker, 1994; nordsieck, 1931, 1961). companies documented workflows through written instructions or flowcharts, while stopwatches measured task durations and identified delays. audits were performed manually to spot inefficiencies, and decision-making was based on limited, often outdated data. this approach made process optimisation slow, inconsistent, and labour-intensive, with businesses only able to make incremental improvements through trial and error. this basic method, which spanned roughly from the 1930s to the 1980s referred to here as stage 1: traditional bpm, focused on observation and time-based analysis, relying heavily on human effort for documentation and lacking integration or automation. while it provided some insight into operational efficiencies, it struggled to keep up with the growing complexity of modern organisations. this stage was marked by limited technology use beyond simple recordkeeping and manual adjustments, making it a resource-intensive, less scalable approach. with the rise of digital technologies, business process management (bpm) underwent a profound transformation from the 1990s into the early 2020s, marking the emergence of stage 2: the first wave of digital bpm. digital tools became central to bpm, transitioning from manual methods to efficient, data-driven systems further advancing the process redesign mindset laid out by hammer and champy (1993). software environments enabled processes to be modelled and automated, providing an accurate representation of workflows (garcíaborgoñón et al., 2014; van der aalst & van hee, 1995). the integration of workflow management systems further advanced bpm by capturing real-time data, which allowed for performance analysis and optimisation (dumas et al., 2005). this shift to digital bpm marked the beginning of an automated, data-driven approach, where tools like process mining provided insights into process performance and conformance, enabling organisations to continuously refine and optimise their operations. currently, organisational bpm practices often combine both stage 1 and stage 2 approaches, with the specific blend depending on factors such as the organisation’s bpm maturity, automation levels, data quality, and available expertise. while these approaches can yield successful outcomes, they also have inherent limitations. key gaps persist within both traditional bpm (stage 1) and digital bpm (stage 2) practices, preventing organisations from fully realising the potential of a truly process-centric organisation— the foundational principle upon which bpm is built. notably, these gaps are not just technological, but strategic with many initiatives failing to start from clearly articulated business priorities and stakeholder needs. the failure to integrate ai and advanced technologies into bpm represents a significant missed opportunity, hindering organisations from optimising processes in real-time, driving innovation, and staying competitive in today’s rapidly evolving business landscape. a visible symptom of these gaps is the proliferation of workarounds, local adaptations people use when processes or systems don’t fit (alter, 2014). from a socio-technical perspective, these gaps reflect misfits between formal process designs and situated work systems—misfits long recognised as a root cause of is shortfalls (alter, 2008). despite the advances of stage 2 bpm, which incorporates data-driven approaches and workflow automation, several key gaps remain that prevent organisations from fully optimising their processes. while stage 2 bpm enables retrospective analysis and optimisation, it lacks the ability to anticipate future process behaviour or proactively optimise workflows through predictive insights. this absence of ai integration limits real-time australasian journal of information systems bandara, leyer and rosemann 2025, vol 29., provocation from automation to autonomisation 5 decision-making, as businesses continue to rely on human interpretation of data rather than leveraging ai to provide intelligent, automated recommendations. additionally, without the capacity for real-time learning and adaptation, organisations are left dependent on static process models and periodic reviews, which are often too slow to respond to rapid changes in the digital landscape. furthermore, stage 2 bpm misses the opportunity for ai-driven personalisation, especially in customer-facing processes like sales and support, where tailored experiences could significantly improve engagement. finally, the process intelligence offered by stage 2 bpm tools, such as process mining, falls short of the deeper insights ai could provide, such as simulating and predicting process behaviour, offering a more comprehensive view of process health and optimisation potential. a recurring limitation in both stage 1 and stage 2 bpm practice is the tendency to adopt new technologies in search of problems to solve. this technology-first mindset risks producing automation without purpose or alignment to organisational strategy. we contend that future bpm advancements should begin with a rigorous diagnosis of business challenges and stakeholder needs, and then identifying the appropriate technology mix to address them. in other words, business context, strategic priorities, and value-creation goals must set the agenda, with technology serving as an enabler rather than the driver. this business-first approach ensures that integration of ai, ml, and iot delivers tangible, context-specific value, rather than generic efficiency gains. what bpm uniquely brings, and where it differs from broader is or automation initiatives is its disciplined, end-to-end view of organisational work that integrates operational detail with a broader contextual understanding. bpm unites modelling, analysis, implementation, execution, and continuous improvement within a socio-technical frame, aligning processes to strategy and evolving stakeholder needs. it also offers tested methods for managing both operational and contextual dynamics (grisold et al., 2024), including the ability to conceptualise potential futures and to deliberately discard superseded practices that no longer serve organisational goals. this dual capability; operational control and contextual foresight, enables bpm not only to optimise current processes but also to shape and reshape them in response to both closedended and open-ended futures. by embedding ai and other advanced technologies within this business-anchored bpm framework, organisations can anticipate change, proactively adapt, and sustain competitiveness in turbulent environments. we provoke the status quo of current bpm approaches by questioning the focus on merely enhancing process efficiency with increasingly sophisticated tools. it is time to rethink bpm, identify untapped opportunities, and challenge the discipline to evolve beyond its current boundaries, unlocking new ways to drive impactful, transformative change. this marks the third stage of bpm and the second wave of digital bpm, where we explore how bpm methods will integrate with dominant technological trends, particularly ai, ml and iot. 3 unveiling new opportunities for process autonomisation we present three key opportunities to enhance bpm tools and methods through the integration of ai, ml, and iot, challenging the conventional bpm practices. while we do not claim to provide an exhaustive overview, we identify these areas as the primary avenues for advancing bpm in the digital era. additionally, we outline future research directions to advance each of these opportunities. these opportunities are framed around persistent and australasian journal of information systems bandara, leyer and rosemann 2025, vol 29., provocation from automation to autonomisation 6 well-documented business challenges, with technology serving as the means to address them effectively through bpm frameworks. 3.1 context-aware personalised business processes while personalisation is a common theme in is, bpm provides the methodological scaffolding to embed personalisation into the governance and orchestration of end-to-end processes. this ensures that ai-driven recommendations or iot-enabled triggers are not isolated features, but explicit decision points within adaptive workflows, guided by clear goal models, macro-level resourcing, and constraint-based governance so that outcomes remain aligned with strategy and compliance (rosemann et al., 2024). this opportunity is motivated by persistent business challenges, experienced across multiple sectors, in delivering relevant, timely, and seamless customer experiences across channels. ai, ml, and iot offer powerful enablers to address these needs when embedded in bpm frameworks. in is, personalisation has been theorised and tested across lenses that are directly useful for bpm; spanning foundational conceptualisations (fan & poole, 2006), recommender design (adomavicius & tuzhilin, 2005), and the personalization–privacy paradox (xu et al., 2011), with recent advances on ethical personalisation (greene et al., 2023). these insights help ensure process personalisation is contextually relevant yet governed, privacy-aware, and strategically aligned. personalisation has long been recognised as a key differentiator for businesses across industries (wurm et al., 2019). historically, many deployed approaches combined explicit input with implicit behaviour signals (e.g., on-site clicks, co-views), as seen in early amazonstyle “customers who bought x also bought y” recommendations. while this provides a level of personalisation, it still depends on customers actively engaging with the platform. similarly, in banking, personalisation is often driven by customers explicitly sharing their financial goals with a representative or through systems like robo-advisors (e.g., betterment), which require customers to input their data upfront. while effective, these tactics generally personalise content within the current channel. they are typically triggered only after a user action and focus on what to show, not how the end-to-end process should respond for example, which channel to hand off to next, when to intervene, or how to prioritise fulfilment and service routing under explicit governance. this narrow scope limits real-time anticipation and cross-channel adaptation. furthermore, these systems assume that customers can clearly articulate their preferences or are willing to engage in potentially lengthy interaction. as a result, valuable opportunities for fluid, real-time, and context-aware personalisation are often missed. recent is work pushes beyond this reactivity: showing that real-time recommendations, tailored to a user’s immediate context (e.g., current activity, time, device), exponentially increase engagement (shi et al., 2025). the limitations of these traditional personalisation methods highlight a critical gap: they are overly reliant on active customer input or staff engagement, which hinders the delivery of seamless, proactive, and adaptive experiences. this reliance creates missed opportunities for dynamic, context-aware personalisation that adapts in real-time to a customer’s needs. however, recent technology advancements provide a powerful solution to bridge this gap. by integrating technologies such as ai, ml and iot including data analytics, bpm systems can automate processes like data collection, real-time analysis, and decision-making, reducing the need for direct customer or staff intervention. this shift enables organisations to offer australasian journal of information systems bandara, leyer and rosemann 2025, vol 29., provocation from automation to autonomisation 7 personalised services that are not only relevant and timely, but also adaptive to the customer’s needs as they evolve. our vision for personalisation goes beyond simply suggesting products or services based on past behaviour or explicit customer input. we advocate for data-driven, context-aware systems that dynamically adapt to an individual’s situation in real time. these systems anticipate needs and adjust their offerings automatically, without requiring explicit customer input. this proactive approach integrates seamlessly into business processes, responding to real-time customer behaviour and external factors to offer tailored experiences, without the customer having to "opt in" or actively engage. we introduce four critical areas that can drive this transformation: ai-driven customer insights: a key limitation of traditional personalisation is its reliance on customers to actively declare preferences. methods like simple data mining analyse past behaviours, such as purchase history or browsing patterns, but still require explicit customer input or clear preferences. in contrast, ai-enabled bpm goes beyond basic data analysis by enabling systems to continuously monitor and analyse customer interactions across various touchpoints whether through digital platforms, transaction data, or behavioural signals, without requiring any explicit input from the customer (davenport, 2023). for example, ai can not only analyse browsing history or past purchases but also integrate additional data sources, such as social media activity, to predict customer preferences and anticipate needs in real time. this capability allows e-commerce platforms, for instance, to offer a more proactive approach to personalisation by automatically suggesting products that align with a customer’s implied preferences, even before they actively search for them. unlike traditional methods, ai-driven insights leverage ml algorithms to uncover deeper, more nuanced patterns, enabling a level of personalisation that adapts dynamically as customer behaviour evolves. this enables business processes to be more dynamic, allowing real-time adjustments and continuous learning from data. it enhances automation, streamlines decision-making, optimizes resource allocation, and improves demand forecasting. moreover, when insights are state-dependent (e.g., mood, context, session intent), bpm can orchestrate when to diversify vs. assimilate recommendations to improve outcomes (shi et al., 2025). to sustain trust, these pipelines should include explicit privacy boundary rules (zhu & kanjanamekanant, 2021) and governance for algorithms that continuously test and adjust their choices, including human oversight, auditability, and fairness checks (greene et al., 2023). real-time personalisation through data integration: traditional methods also rely on staff to engage with customers to understand their needs. modern bpm innovations can integrate data from multiple touchpoints (e.g., website interactions, mobile apps, customer service interactions, and past purchases) to build a holistic customer profile. this real-time data can then be processed by ai algorithms to create tailored offerings or recommendations without waiting for direct customer input. for instance, in the banking sector, bpm systems can use customer transaction data and behavioural insights to automatically offer personalised loan options, savings plans, or investment advice based on a customer’s financial habits, without needing an initial consultation with a bank representative. automated, adaptive workflows: bpm workflows can be designed to automatically adapt in real-time based on customer behaviour and data. for example, when a customer browses specific products or services, crm systems can trigger personalised content, offers, or notifications, which are seamlessly integrated into business processes. the workflow system australasian journal of information systems bandara, leyer and rosemann 2025, vol 29., provocation from automation to autonomisation 8 then uses these triggers to adapt the process dynamically adjusting the next steps, such as sending tailored promotions, updating customer profiles, or prioritising certain actions based on the customer's preferences or previous interaction history. this adaptability ensures that customers receive a personalised experience without requiring them to manually communicate their needs. this approach is far more dynamic than traditional methods, which often require customers to proactively express their preferences. in industries like retail or banking, adaptive bpm workflows can continuously adjust product or service offerings, delivering the most relevant options based on real-time data and evolving customer needs. where adaptivity relies on learning-by-feedback methods, such as bandit algorithms that continuously test different offers and shift toward the best-performing one, or reinforcement learning that chooses the next step based on rewards from prior actions—bpm should include socio-technical safeguards (human oversight, audit trails, fairness checks) in line with ethical is guidance (greene et al., 2023). machine learning for continuous improvement: ml algorithms can continuously refine personalisation models by learning from ongoing interactions, enabling bpm systems to improve over time without requiring manual updates. for instance, in customer support, bpm systems can use ml to analyse past service requests and automatically suggest solutions based on similar customer issues. this approach can greatly enhance efficiency and customer satisfaction, as the system learns to predict the most relevant solutions without waiting for the customer to explicitly state their needs. ai-powered bpm workflows can revolutionise personalisation across a wide range of industries by leveraging data-driven insights to proactively anticipate and meet customer needs. in healthcare, ai can analyse a patient’s medical history, appointment data, and wearable health tracker information to predict potential health issues or recommend preventive measures, providing care providers with the ability to guide patients toward better health outcomes without waiting for explicit requests. in banking, ai and ml models can help automate personalised financial advice by analysing customers' spending habits, savings, and income data to recommend savings plans or loan options without customers needing to explicitly declare their goals. platforms like monzo and cleo exemplify this by providing tailored financial recommendations based on customers' actual financial activity. in retail, contextual digital trace data allows retailers to proactively deliver personalised products and services by analysing customer behaviours and situational factors. for example, retailers can send tailored product recommendations, location-based offers, or time-sensitive promotions based on browsing history, cart abandonment, or in-store behaviour. integrating this data with bpm systems enables retailers to adapt processes in real-time, offering predictive discounts, seasonal deals, and loyalty rewards that align with evolving customer needs and enhance the shopping experience. these sectorial examples underscore how integrating ai and ml into bpm systems not only enhances the customer experience through proactive personalisation but also improves operational efficiency and scalability, allowing businesses to stay competitive in an increasingly dynamic digital world. while some of these practices are already in use, we propose the following avenues for future research, aimed at providing evidence-based, contextually relevant guidelines that can inform and enhance practical applications in the field. australasian journal of information systems bandara, leyer and rosemann 2025, vol 29., provocation from automation to autonomisation 9 • how can ai and/or ml be seamlessly integrated into bpm systems to analyse customer data in real time to proactively uncover implied preferences and anticipate needs before customers explicitly search for products or services? • how can bpm systems effectively integrate data from diverse touchpoints and systems (such as website interactions, mobile apps, customer service engagements, and past purchases) to create a comprehensive, real-time customer profile that drives personalised decision-making and enhances service delivery while respecting privacy boundaries? • how can bpm workflows be designed to automatically adapt to changing customer needs in real-time, leveraging in-the-moment data and customer behaviour to dynamically adjust service or product offerings (including when to introduce new options versus reinforce known preferences)? • how can machine learning be integrated into bpm systems to automatically optimise service delivery, by predicting relevant solutions based on historical interactions, with continuous learning and monitoring for model drift, without requiring manual updates or explicit customer input? • in what ways can ai-driven personalisation models in bpm systems account for customer emotions and sentiment in real-time, enhancing the relevance and empathy of service delivery? • how can bpm systems ensure ethical and transparent personalisation practices when utilising ai and data analytics, particularly in terms of privacy, security, explainability, human oversight, and fairness, to sustain customer trust? 3.2 self-adaptive process workflows: integrating real-time customer feedback and identifying workarounds in many organisations, feedback loops remain too slow or fragmented to meaningfully influence day-to-day process performance. addressing this gap requires bpm approaches that are grounded in real operational and stakeholder challenges, with technology applied to enable responsive, data-informed adaptation. ml has the potential to significantly transform bpm systems by enabling them to evolve continuously through real-time insights from various sources. we propose a self-evolving bpm system, powered by ml, that allows organisations to dynamically refine their workflows in (a) response to customer feedback, and (b) internal work patterns and process workarounds. by observing interactions such as complaints, positive feedback, or inefficiencies, bpm systems can automatically react and adapt in real-time, creating a new “asis” process that reflects these insights. this approach fosters an agile, responsive process environment that stays aligned with actual customer and/or user needs and behaviours, enhancing operational efficiency and improving stakeholder satisfaction while ensuring that workflows remain relevant and effective. 3.2.1 autonomous reaction to customer feedback integrating autonomous reactions to customer feedback through ml in bpm systems can significantly transform various industries. for example, in e-commerce, bpm systems could automatically adjust processes based on customer dissatisfaction highlighted on platforms like reddit, improving aspects like checkout and delivery. in telecommunications, social media complaints could trigger automatic reviews and enhancements in billing processes or customer service workflows. in healthcare, feedback on wait times or booking issues could australasian journal of information systems bandara, leyer and rosemann 2025, vol 29., provocation from automation to autonomisation 10 prompt scheduling adjustments to improve patient experience, while in banking, insights into loan application frustrations could streamline workflows. similarly, retail and hospitality businesses could refine ordering systems or service levels in response to customer reviews. by enabling bpm systems to autonomously adapt to real-time feedback, businesses can enhance operational efficiency, proactively address issues, and continuously improve customer satisfaction. to fully explore the cutting-edge potential of integrating ml, real-time feedback, and bpm systems, several key areas require further research. these areas will help unlock the ability to create self-evolving, adaptive processes that are more responsive to customer needs and aligned with business objectives. these research avenues include: • how can ml algorithms be trained to autonomously prioritise and translate diverse customer feedback (e.g., from reddit, reviews, or social media) into actionable process changes within bpm systems, ensuring that only high-impact issues trigger workflow adjustments? • in what ways can bpm systems integrate real-time feedback from customer sentiment analysis (such as social media discussions) to autonomously adapt workflows and processes, and how can these adaptations be tested for effectiveness without human intervention? • how can bpm systems leverage ml to identify and incorporate unspoken customer needs, detected through patterns in feedback and behavioural data, into process improvements that align with both business goals and customer satisfaction? 3.2.2 detect and autonomously respond to process workarounds in addition to customer-driven insights, ml can also track how employees or other stakeholders are working around established processes. these workarounds often arise when individuals perceive the existing processes as inefficient or unable to meet specific demands in real-time. workarounds are local, goal-driven adaptations people use when standard procedures or systems get in the way of getting the job done (alter, 2014). rather than viewing them simply as “unauthorised” deviations, they can be read as signals of misfit between process design and operational reality. for example, retail agents may skip low-value verification to speed resolution for high-value customers; in healthcare, staff may streamline intake steps to expedite treatment. analysing these patterns helps pinpoint where workflows create friction and whether the informal change actually improves outcomes (e.g., resolution time, patient flow, resource use). the bpm implication is to convert beneficial workarounds into governed process changes—and redesign or reinforce controls where workarounds introduce risk. in socio-technical terms, workarounds surface where designed technical routines and the social organisation of work lack fit; robust bpm can sense these signals and adjust the joint design (strong & volkoff, 2010). if workarounds consistently lead to improved outcomes, a bpm system should, under governance, promote them into standard operating procedures as “positive deviance” (setiawan & sadiq, 2013), so successful local practices are scaled safely. at run time, the bpm system can detect recurring patterns and their effects, formalise beneficial ones, and flag harmful ones for review (alter, 2014). institutionalising validated workarounds makes processes more agile and resilient, reduces manual intervention, and recognises frontline improvement efforts. it also streamlines operations and builds a culture of continuous improvement, aligning workflows with actual user needs and boosting customer satisfaction australasian journal of information systems bandara, leyer and rosemann 2025, vol 29., provocation from automation to autonomisation 11 and employee morale. the result is improved day-to-day performance and a more durable capability turning informal problem-solving into formal, scalable solutions. we propose several research questions aimed at advancing the understanding of how bpm systems can autonomously evolve by analysing workarounds: • how can ml algorithms identify and evaluate workarounds in bpm systems to determine which should be formalised into standard operating procedures, and how can these insights trigger autonomous system adaptations to enhance process performance? • what patterns in employee-driven workarounds can inform bpm systems to autonomously adapt workflows in real-time, improving operational efficiency and responsiveness to customer needs without manual intervention? • what metrics and success indicators should bpm systems use to assess the effectiveness of workarounds? o how can these insights be leveraged to continuously optimise workflows and enhance customer satisfaction? o how can bpm systems differentiate between temporary workarounds and systemic process changes, ensuring that only beneficial adaptations are formalised while maintaining alignment with organisational goals and compliance standards? 3.3 distributed deployment of processes using iot the motivation for exploring iot-enabled process deployment comes from long-standing challenges in achieving real-time responsiveness and operational agility across distributed environments. iot technologies become high-value enablers when harnessed to address these specific challenges through bpm designs. the idea of distributed deployment of processes using iot is an emerging and increasingly important area of innovation in business process management (bpm). the combination of iot devices, edge computing, and smart process automation is reshaping the way organisations approach operational efficiency, agility, and real-time decision-making. traditionally, bpm systems have been centralised and dependent on human intervention or static, pre-defined workflows. however, as the iot landscape evolves, the capability to deploy and manage business processes across a vast network of interconnected devices introduces an entirely new dimension to bpm. this shift towards distributed and autonomous processes is poised to revolutionise industries by enabling smarter, more efficient, and highly responsive operations. the convergence of iot and bpm has recently gained significant attention because of its potential to transform traditional process management. as iot devices become more pervasive, collecting data and enabling real-time interactions across various points of operation, organisations are looking for ways to integrate this data into their existing bpm frameworks. by leveraging the intelligence of iot devices, businesses can create processes that react, adapt, and evolve based on real-time insights from across the network. this is a natural progression from static processes to more dynamic, distributed systems that can execute decisions autonomously, enabling faster and more effective responses to customer needs, operational inefficiencies, or environmental changes. this represents a paradigm shift where workflows no longer rely on centralised decisionmaking but instead are distributed and automated in response to real-time data feeds. for australasian journal of information systems bandara, leyer and rosemann 2025, vol 29., provocation from automation to autonomisation 12 example, in manufacturing, iot sensors can monitor machine conditions and trigger maintenance actions before breakdowns occur. in logistics, gps and rfid sensors can track the movement of goods and initiate re-routing decisions without human intervention. in healthcare, patient monitoring devices can trigger workflows to adjust treatment plans based on real-time health data. iot in smart cities can transform public services by automating traffic management, waste collection, and energy use, with bpm systems leveraging real-time sensor data to optimise urban operations, improving efficiency and quality of life for residents. several critical areas require further exploration to fully integrate iot into bpm systems, ensuring seamless scalability, robust security, and appropriate human oversight while addressing the complexities of real-time data processing and decision-making across diverse industries. • how can bpm systems be redesigned to integrate real-time, distributed iot data while ensuring scalability and flexibility across diverse industries? • what methodologies and ai/ml models can be developed to enable autonomous decision-making and process adaptation within bpm systems driven by iot sensor data? • what frameworks or protocols are needed to ensure seamless interoperability between iot devices, sensors, and existing bpm tools, particularly in large-scale, multi-device environments? • how can bpm systems balance automation with human oversight when integrating iot-driven decision-making, particularly in mission-critical sectors like healthcare or finance? • what security and privacy measures must be integrated into bpm tools to safeguard real-time data from iot devices, especially in sensitive industries like healthcare or smart cities? 4 conclusion as organisations accelerate their digital transformation journeys, bpm must evolve from adapting to technological change to actively shaping how technology delivers business value. the integration of ai, ml and iot offers unprecedented opportunities for agility, personalisation, and real-time adaptability. yet these technologies must be deployed in response to clearly articulated business challenges, not pursued for their novelty. bpm’s enduring strength lies in its ability to align operational processes with strategic priorities, ensuring that technological investments generate context-specific value and measurable outcomes. a central advancement in this evolution is process autonomisation enabling bpm systems to make decisions autonomously and adapt to real-time data and contextual shifts. the three opportunities explored in this paper; context-aware personalised processes, real-time feedback-driven workflows, and distributed iot-based process management, demonstrate how autonomisation can transform both operational efficiency and customer engagement. by anchoring these innovations in business needs, organisations can ensure that automation enhances not replaces strategic intent. drawing on grisold et al. (2024), we stress that bpm is uniquely positioned to manage both operational and contextual dynamics. it can conceptualise potential futures, design for flexibility, and deliberately discard superseded practices that no longer serve organisational australasian journal of information systems bandara, leyer and rosemann 2025, vol 29., provocation from automation to autonomisation 13 goals. this dual capability allows bpm not only to optimise processes in predictable, closedended scenarios but also to navigate uncertainty in open-ended futures. the proposed research agenda calls on academics and practitioners to seize these opportunities while addressing the accompanying challenges; scalability, security, governance, and balancing automation with human oversight. future work must focus on integrating ai, ml, and iot seamlessly into socio-technical bpm systems, guided by capability frameworks and grounded in goal specification, macro-level resourcing, and constraint governance. ultimately, bpm must not only adapt to the rapidly evolving technological landscape but also lead the way in reimagining business processes for a smarter, more efficient future. as emerging technologies often lack seamless integration within individual workflows, the strategic integration of technology into bpm is no longer optional—it is mission-critical. embracing process autonomisation as the next pivotal stage will enable organisations to unlock new levels of performance, enhance operational agility, and foster deep alignment between process management and business strategy. this will not only drive efficiency but also position businesses to thrive in an increasingly dynamic and competitive environment, a future where bpm is the discipline that ensures technology serves strategy, not the other way around. 5 references adomavicius, g., & tuzhilin, a. (2005). toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions. ieee transactions on knowledge and data engineering, 17(6), 734-749. doi.org/10.1109/tkde.2005.99 alter, s. (2008). defining information systems as work systems: implications for the is field. european journal of information systems, 17(5), 448-469. doi.org/10.1057/ejis.2008.37 alter, s. (2014). theory of workarounds. communications of the association for information systems, 34(55), 1041-1066. doi.org/10.17705/1cais.03455 barker, r. c. (1994). the design of lean manufacturing systems using time-based analysis. international journal of operations & production management, 14(11), 86-96. davenport, t. h. (2023). hyper-personalization for customer engagement with artificial intelligence. management and business review, 3(1-2), 29-36. doi.org/10.1177/2694105820230301006 dumas, m., fournier, f., limonad, l., marrella, a., montali, m., rehse, j.-r., accorsi, r., calvanese, d., giacomo, g. d., fahland, d., gal, a., rosa, m. l., völzer, h., & weber, i. (2023). ai-augmented business process management systems: a research manifesto. acm trans. manage. inf. syst., 14(1), article 11. doi.org/10.1145/3576047 dumas, m., la rosa, m., mendling, j., & reijers, h. a. (2018). fundamentals of business process management (2 ed.). heidelberg: springer. dumas, m., van der aalst, w. m. p., & ter hofstede, a. h. (2005). process-aware information systems. bridging people and software through process technology hoboken: wiley. australasian journal of information systems bandara, leyer and rosemann 2025, vol 29., provocation from automation to autonomisation 14 fan, h., & poole, m. s. (2006). what is personalization? perspectives on the design and implementation of personalization in information systems. journal of organizational computing and electronic commerce, 16(3-4), 179-202. doi.org/10.1080/10919392.2006.9681199 garcía-borgoñón, l., barcelona, m. a., garcía-garcía, j. a., alba, m., & escalona, m. j. (2014). software process modeling languages: a systematic literature review. information and software technology, 56(2), 103-116. doi.org/10.1016/j.infsof.2013.10.001 greene, t., shmueli, g., & ray, s. (2023). taking the person seriously: ethically aware is research in the era of reinforcement learning-based personalization. journal of the association for information systems, 24(6), 1527-1561. doi.org/https://aisel.aisnet.org/jais/vol24/iss6/6 grisold, t., janiesch, c., röglinger, m., & wynn, m. t. (2024). managing dynamics in and around business processes. business & information systems engineering, 66(5), 533-540. doi.org/10.1007/s12599-024-00895-2 hammer, m., & champy, j. (1993). reengineering the corporation. a manifesto for business revolution. new york: harpercollins. harmon, p. (2019). business process change (4 ed.). waltham: morgan kaufmann. jeston, j., & nelis, j. (2008). management by process. a roadmap to sustainable business process management. london: elsevier. kerpedzhiev, g. d., könig, u. m., röglinger, m., & rosemann, m. (2021). an exploration into future business process management capabilities in view of digitalization. business & information systems engineering, 63(2), 83-96. doi.org/10.1007/s12599-020-00637-0 leyer, m., stumpf-wollersheim, j., & kronsbein, d. (2017). stains on the bright side of processoriented organizational designs. an empirical investigation of advantages and disadvantages. schmalenbach business review, 18(1), 29-47. mumford, e. (2006). the story of socio-technical design: reflections on its successes, failures and potential. information systems journal, 16(4), 317-342. doi.org/10.1111/j.13652575.2006.00221.x nordsieck, f. (1931). aufgabenverteilung und instanzenbau im betrieb. grundprobleme und grundprinzipien der organisation des betriebsaufbaus ii. die betriebswirtschaft, 24(7), 204-210. nordsieck, f. (1961). betriebsorganisation. betriebsaufbau und betriebsablauf. stuttgart: poeschel. rosemann, m., brocke, j. v., van looy, a., & santoro, f. (2024). business process management in the age of ai – three essential drifts. information systems and e-business management, 22(3), 415-429. doi.org/10.1007/s10257-024-00689-9 setiawan, m. a., & sadiq, s. (2013). a methodology for improving business process performance through positive deviance. international journal of information system modeling and design (ijismd), 4(2), 1-22. doi.org/10.4018/jismd.2013040101 shi, l., liu, j., li, y., & foutz, n. z. (2025). ephemeral state-dependent recommendation for digital content. information systems research, 0(0), null. doi.org/10.1287/isre.2022.664 australasian journal of information systems bandara, leyer and rosemann 2025, vol 29., provocation from automation to autonomisation 15 strong, d. m., & volkoff, o. (2010). understanding organization?enterprise system fit: a path to theorizing the information technology artifact. mis quarterly, 34(4), 731-756. doi.org/10.2307/25750703 van der aalst, w. m., & van hee, k. m. (1995). framework for business process redesign. proceedings 4th ieee workshop on enabling technologies: infrastructure for collaborative enterprises (wet ice'95), wurm, b., goel, k., bandara, w., & rosemann, m. (2019). design patterns for business process individualization. in t. hildebrandt, b. f. van dongen, m. röglinger, & j. mendling (eds.), (vol. 11080, pp. 370-385). springer international publishing. doi.org/10.1007/978-3-030-266196_24 xu, h., luo, x., carroll, j. m., & rosson, m. b. (2011). the personalization privacy paradox: an exploratory study of decision making process for location-aware marketing. decision support systems, 51(1), 42-52. doi.org/10.1016/j.dss.2010.11.017 zhu, y.-q., & kanjanamekanant, k. (2021). no trespassing: exploring privacy boundaries in personalized advertisement and its effects on ad attitude and purchase intentions on social media. information & management, 58(2), 103314. doi.org/10.1016/j.im.2020.103314 copyright copyright © 2025 bandara, w., leyer, m. and rosemann, m. this is an open-access article licensed under a creative commons attribution-non-commercial 4.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v29.6059 paper title (paper title style) australasian journal of information systems saxena & mcdonagh 2020, vol 24, research article is controls in healthcare 1 the evolving nature of information systems controls in healthcare organisations: the case of a blood banking enterprise system from western europe deepak saxena trinity business school, trinity college dublin ireland saxenad@tcd.ie joe mcdonagh trinity business school, trinity college dublin ireland abstract information systems (is) projects are found to be complex, unpredictable, and prone to time and cost overruns. perhaps that is why organisations put a strong focus on is controls during the planning and execution of such projects. is control literature in the past has focussed on dyadic control relationships during an outsourced is development project and relatively little is known about such controls during a complex enterprise systems project. existing studies usually take a static view of is controls and do not investigate how controls evolve during different phases of the system lifecycle, as well as across projects. this study presents a processual view of is controls in the enterprise systems lifecycle in a national blood processing organisation. traditional research in a blood banking context has focussed on optimising the process of blood collection, inventory management, and distribution with relatively limited attention to the implementation of the supporting information systems. this research focusses on the evolution of control based on a study of three enterprise system implementation projects in the case organisation. the study demonstrates that while all five control modes (input, outcome, behaviour, clan, and self-control) are applied across the phases of enterprise systems projects, the nature and extent of control mechanisms changes across the phases of the enterprise system lifecycle. the findings also suggest a teleological evolution of a project’s control portfolio in which the portfolio evolves based on adaptive learning processes from earlier projects. finally, by exhibiting the influence of institutional and market context, this study also underlines the multi-stakeholder and contextual nature of enterprise systems implementation and associated controls in health service organisations. keywords: is control, enterprise systems, formal control, informal control, control evolution 1 introduction information systems (is) projects are usually complex, unpredictable, and are prone to time and cost overruns (bhoola, hiremath, & mallik, 2014). moreover, evidence suggests that context plays a major role in the planning, procurement, and implementation of is projects (berente, lyytinen, yoo, & maurer, 2019; matinheikki, aaltonen, & walker, 2019; moe, newman, & sein, 2017). this is especially true for healthcare organisations where complex institutional logics (e.g. medical, managerial, public service, and market) may be at work (bunduchi, tursunbayeva, & pagliari, 2020; poba-nzaou, uwizeyemungu, raymond, & paré, australasian journal of information systems saxena & mcdonagh 2020, vol 24, research article is controls in healthcare 2 2014; salge, kohli, & barrett, 2015). perhaps that is why organisations put a strong focus on is controls during the planning and execution of complex is projects. information systems controls may be defined as a set of activities that are conducted in a project to moderate the behaviour of project participants and to ensure that their skills and capabilities are applied towards successful project completion (kirsch, 1997, 2004). traditionally, is control literature has focussed on dyadic control relationships during outsourced is projects (choudhury & sabherwal, 2003; gopal & gosain, 2010; heiskanen, newman, & eklin, 2008) and relatively little is known about such controls during a complex enterprise systems project (cram, brohman, & gallupe, 2016; inácio & marques, 2018). existing studies usually take a static view of is controls (wiener, mähring, remus, & saunders, 2016) and do not investigate how controls evolve during different phases of the system lifecycle, as well as across projects. moreover, there is limited research on the influence of institutional context on is controls, including in the healthcare setting. against this theoretical backdrop, this study presents a dynamic and contextual view of is controls during an enterprise system implementation by conducting a processual case study (langley, smallman, tsoukas, van de ven, 2013; niederman, müller, & march, 2018; pettigrew, 2012) of the system lifecycle in a national blood banking context. the research question for this study was – how does an is control portfolio evolve during an enterprise system implementation project in a blood banking organisation? by way of supporting this question, this paper is organised as follows. section 2 presents a brief discussion on blood banking as a research context and reviews the literature on is controls. section 3 justifies and outlines the case research methodology used in the study. the case narrative is presented in section 4, followed by findings and discussion in section 5. the concluding section discusses the key contributions, implications, and limitations of this study. 2 literature review 2.1 is control modes and control portfolio traditional discussion on control theory in is projects divides is controls into two modes (kirsch, 1997; susilo, heales, & rohde, 2007; wiener et al., 2016) – formal and informal. figure 1 shows various formal and information controls with some (non-exhaustive) examples in the adjacent box. formal controls are mostly bureaucratic in nature (ouchi, 1979, 1980; gregory & keil, 2014) and are driven by organisational rules and procedures. in contrast, informal controls are social in nature (ouchi, 1979, 1980; chua, lim, soh, & sia, 2012; chua & myres, 2018) and are driven by prevalent norms and practices. australasian journal of information systems saxena & mcdonagh 2020, vol 24, research article is controls in healthcare 3 figure 1. is control modes (adopted from wiener et al., 2016) within formal controls, while the outcome control is focussed on ensuring that pre-determined outcomes are achieved, behaviour control is focused on aligning the controlee’s behaviour towards accepted standards (ouchi, 1978; kirsch, 1997). the example of outcome control in is projects could be software requirement specifications and delivery milestones. behaviour controls may include regular meetings and production of status reports. wiener et al. (2016) also add input control to the list of formal control modes. input control primarily includes resource allocation (e.g. manpower, funding, infrastructure, and training) available for the project. within informal controls, clan control and self-control are identified as predominant control modes. clan control relies on the social capital (chua et al., 2012) for the emergence of shared norms and practices outside formal controls. this could be done, for instance, via away-days or weekly lunches (wiener et al., 2016) so that a collective sense of project ownership and responsibility (wiedermann & wiesche, 2018) is developed. self-control, as the name suggests, is more individual in nature relying on self-monitoring and regulation (kirsch & cumming, 1996). while organisations primarily apply formal controls in is projects (henderson & lee, 1992; choudhury & sabherwal, 2003; tiwana & keil, 2009; wiener, remus, heumann, & mähring, 2015; wiener, cram, & remus, 2017), informal controls may be highly useful in lowmaturity environments (mao, zhang, & song, 2008) or where project leaders have considerable domain experience (kirsch & cumming, 1996). 2.2 dynamic nature of control portfolio it is crucial to note that the control modes are not applied in isolation, and a control portfolio (kirsch, 1997, 2004; choudhury & sabherwal, 2003; soh, chua, & singh, 2011; murungi, wiener, & marabelli, 2019) is often used in is projects. this is usually done to make use of control complementarity (grabski & leech, 2007; srivastava & tao, 2012) and control ambidexterity (gregory & keil, 2014). while initial research largely presents a static view of a project’s control portfolio, subsequent research notes its dynamic nature. for instance, studies (choudhury & sabherwal, 2003; susilo et al., 2007) report that while is projects in the beginning typically rely on outcome-based controls, behaviour control mechanisms are often added at later stages of the project. this reflects a lifecycle (van de ven & poole, 1995; van de ven & sun, 2011) based assumption that each phase of the is project exhibits certain australasian journal of information systems saxena & mcdonagh 2020, vol 24, research article is controls in healthcare 4 characteristics, and a certain configuration of a project’s control portfolio is required for each phase (nuwangi & sedera, 2018). an alternative conception of a project’s control portfolio relies on a teleological mechanism (van de ven & poole, 1995) arguing that the organisation adopts the use of control modes based on the learnings gained during the process. for instance, organisations may not have sufficient clarity on deliverables in the beginning and would increase control to maximise the benefits (heiskanen et al., 2008) at a later stage. conversely, as the understanding between the client and the vendor increases, the organisation may decide to employ more trust-based controls (gregory, beck, & keil, 2013). extending this view, recent studies (chua & myres, 2018; murungi et al., 2019) offer a social perspective that views controls as a negotiated order between parties. 2.3 gaps in existing research barring some exceptions (e.g. soh et al., 2011), is control literature traditionally focusses on the control portfolio for system development projects (choudhury & sabherwal, 2003; heiskanen et al., 2008; mao et al., 2008) between two parties. this results in a number of limitations. first, we have limited empirical evidence on the control mechanisms during a project where an off-the-shelf solution such as an enterprise system (cram et al., 2016; inácio & marques, 2018) is being implemented. second, the controller-controlee relationships are often conceived in terms of simple dyads (kirsch, 2004) such as business-is or client-vendor. consequently, it is sometimes ignored that projects may have multiple levels of controllers and their respective controlees (soh et al., 2011) or sometimes a controlee may have multiple controllers (nuwangi, sedera, & srivastava, 2018). consequently, different control modes may be effective at different levels. for instance, ouchi (1978) notes that while outcome controls are easily transmitted across vertical hierarchy, different levels of the hierarchy may rely on different behaviour controls. finally, since such studies conduct a cross-sectional analysis or focus on a single project at a single instance, there is no appreciation of the historical context (chua & myres, 2018) in the evolution of control modes across multiple projects is an organisation. this study aims to address these limitations by conducting a processual case study (langley et al., 2013; niederman et al., 2018; pettigrew, 2012) of is controls during three instances of an enterprise system implementation in a national blood banking organisation from western europe. a blood bank organisation was deemed an exemplary case (yin, 2017) for this study since the level of controls are generally very high in the healthcare sector. a high level of control is also necessitated since software solutions for blood banks are much less standardised due to the niche nature of business processes followed across the globe. the next section introduces blood banking as a research context. 2.4 blood banking as a research context blood transfusion is a crucial and integral element of medical practice. it is the process through which blood cells of a donor are transferred to the recipient after careful testing and crossmatching. blood transfusion may be required in cases where the patient has lost significant amounts of blood due to an accident or during a surgery. thus, hospitals and clinical practices need to have an adequate supply of blood to ensure patient safety. quite often collection and transfusion are conducted by separate entities – the former by a regional blood bank, and the latter by hospitals. figure 2 shows the typical stages in the blood supply chain. while hospitals australasian journal of information systems saxena & mcdonagh 2020, vol 24, research article is controls in healthcare 5 store blood in a small blood bank for emergency and planned use, a major part of blood operations is covered by regional blood banks. the case organisation in this study is one such regional blood bank covering blood processing and supply in a western european nation. figure 2. typical stages in the blood supply chain. in modern medicine, blood transfusion usually involves transfusing only the specific cells (e.g. red blood cells, white blood cells, platelets, or plasma) rather than transfusing whole blood. hence, regional blood centres need to process and test individual components to ensure timely delivery and patient safety. at the same time, blood and blood components are perishable (see table 1) and hence the collection and processing need to be optimised to minimise wastage. responding to the peculiar nature of blood products, past research has focussed on optimising the collection planning (alfonso, xie, augusto, & garraud, 2012; cumming, kendall, pegels, seagle, & shubsda, 1976), inventory management (kendall & lee, 1980) at the regional blood centre, and ordering policies in hospital (katsaliaki & brailsford, 2007) using modelling and simulation. component shelf life whole blood 21-35 days red blood cells 35 days (14 days for irradiated/washed cells) white blood cells 24 hours platelets 5-7 days plasma (frozen) 36 months table 1. key blood components and shelf life (source: jpac, 2013) apart from safe processing and timely delivery of the physical product, a key informational requirement for blood operations is traceability. the european directorate for the quality of medicines & healthcare (edqm, 2017) defines traceability as “the ability to trace in all directions each individual unit of blood or the blood components derived from it from the donor to its final destination, whether this is a patient, a manufacturer of medicinal products or its disposal” (p. 218). an audit trail and lookback provision are considered essential for blood banks (ashford et al., 2000) to ensure patient safety in situations such as a recall due to unforeseen circumstances. for this reason, implementation of blood enterprise systems (bes) is often recommended to provide end-to-end connectivity for blood bank information processes (edqm, 2017; jpac, 2013). such bes also need to have an interface with the testing equipment to facilitate automated transfer of results to the system (li, chao, & dong, 2007) in order to avoid human errors in entering data (gupta, priyadarshini, massoud, & agrawal, 2004). australasian journal of information systems saxena & mcdonagh 2020, vol 24, research article is controls in healthcare 6 however, while the general enterprise system market benefits from the incorporation of best practices into the software, the bes software industry has much to do in this regard despite numerous directives and suggested protocols (for instance, isbt 128 for the labelling of blood products). due to historical evolution and regulation diversity, different blood banks follow slightly different practices. as a result, the bes market is replete with small players catering to their niche markets and none of the big players (e.g. sap, oracle, or microsoft) from the wider enterprise system market provide a specific solution tailored to blood banks. in addition to the health and safety concerns, the inconsistency across various bes is a crucial reason for tighter controls when implementing a solution. therefore, a blood bank context was considered an exemplary context (yin, 2017) to study the evolution of is controls. 3 research methodology 3.1 processual case study this study focussed on answering the following research question: how does an is control portfolio evolve during an enterprise system implementation project in a blood banking organisation? since the research question is of ‘how’ type, and the research was being conducted in the realworld context, a case study (yin, 2017; murungi et al., 2019) approach was considered appropriate. this is in line with established practice where a case study approach is a widely used research method in is research (mazaheri, lagzian, & hemmat, 2020) as well as in enterprise systems research (saxena & mcdonagh, 2017). moreover, since the research question focusses on the ‘evolution’ of is control, a longitudinal case study with a processual focus (langley et al., 2013; pettigrew, 2012) was deemed relevant. a processual study allows the researcher to trace events (george & bennett, 2005) over time and observe patterns across process instances (langley et al., 2013; pettigrew, 2012) to identify the mechanism underpinning the process. finally, since the research question was framed for a specific context (blood bank), processual case study was adopted as it helps in developing a contextual explanation (avgerou, 2019) based on a situated understanding (maxwell, 2017) of specific events. thus, a longitudinal qualitative case study approach with a processual lens (niederman et al., 2018) was adopted to gain an understanding of the evolution in the control portfolio in the case organisation. at the global level, the blood bank market is often classified in terms of geographical coverage with north america, europe, asia-pacific, and lamea (latin america, middle east and africa) as major regions. within europe, the market is further classified into germany, france, the uk, and rest of europe (pandey & sumant, 2020). the case organisation in this study belongs to a nation from the rest of europe. national blood processing services (nbps, a pseudonym) is an autonomous public body engaged in the collection, processing and distribution of blood and related products in the country. while nbps is an autonomous agency in its day to day operations, it needs the endorsement of its parent government department (mentioned as department for the remainder of the paper) for strategic investments. the agency engaged in three instances of bes implementation during 2000-2017. considering the level of controls applied in blood bank operations, nbps was considered an exemplary case (yin, 2017) to study the evolution of is controls within and across three bes projects in nbps. australasian journal of information systems saxena & mcdonagh 2020, vol 24, research article is controls in healthcare 7 3.2 data collection data collection started with the collection and analysis of secondary data. table 2 shows the secondary data collected and used in the study. it may be noted the secondary data from outside the organisation was also collected, thereby avoiding a ‘container’ approach (winter, berente, howison, & butler, 2014). analysis of secondary data helped in deriving the sequence of events and in identifying key stakeholders for primary data collection. secondary data number of documents source nbps annual reports (2001-2017) 17 nbps website nbps board minutes (2002-2017) 153 nbps website nbps strategic plans (2005-2009, 2010-2012, 2013-2016) 3 nbps website newspaper reports 10 news websites internal project documents (including internal audit) 27 nbps project team reports of the public auditor 3 public auditor website debate of the parliamentary committee 2 parliament website other government reports and circulars 10 government websites table 2. secondary data collected in the study primary data collection mainly consisted of in-depth qualitative interviews with the participants from the case organisation. to ensure internal validity (maxwell, 2017), the participants from top management, the implementation team, and user groups were interviewed. table 3 presents the interview participants’ profile. participant’s primary affiliation no. of participants top management team members 6 it team (including project manager and super-users) 8 members of user groups 11 total 25 table 3. details of interview participants interviews were conducted by the first author in the organisation’s premises to allow for a natural interview setting (langley & meziani, 2020). in the paper, respondents are referred to as r1, r2, r3 etc. to ensure anonymity. in total, twenty-seven interviews were conducted with twenty-five participants (one interview had two participants, three participants were interviewed twice). all but three interviews were recorded and transcribed verbatim by the researcher. the three interviews for which recording was not permitted, detailed notes were taken, and the interview was transcribed on the same day based on the notes and memory. once the transcription was completed, detailed case analysis was conducted as outlined in the next section. 3.3 case analysis case analysis was a combination of narrative analysis and qualitative coding. the case narrative was written in the form of process tracing (george & bennett, 2005) with a focus on the sequence of events. the narrative was used not just as a description tool, but also as an analytical tool (pentland, 1999) to identify deep structures underpinning the sequence of events. for analytical purpose, the narrative was divided using the strategy of temporal australasian journal of information systems saxena & mcdonagh 2020, vol 24, research article is controls in healthcare 8 bracketing (langley et al., 2013; pettigrew, 2012) in which the sequence of events is analysed in temporal blocks of each implementation cycle. the case narrative is presented in section four. this was followed by a three-stage qualitative coding exercise (miles, huberman & saldana, 2014) for data reduction and identification of themes. all interview transcripts and minutes of the meetings were uploaded to the software qda miner lite. in the first stage, text blocks were assigned a short phrase indicating the nature of the block. this resulted in 128 first-level codes. in the second stage, the first-level codes were amalgamated based on similarity in their nature. this resulted in nineteen second-level codes. most of the codes corresponded to the examples of various controls (e.g. review and audit, professional norms) noted during the literature review. finally, all second-order codes were assigned to one of the control categories – behavioural, outcome, input, clan and self-control. appendix 1 presents the coding tree generated during the analysis. once the coding was complete, the narrative helped in identifying the prevalence of specific control modes during the pre-implementation, implementation, and post-implementation phases of the enterprise systems project. since the case covers three instances of the bes project, one could also observe the evolution of control modes across the three project implementations. the case analysis and discussion are presented in section five. 4 case narrative figure 3 shows the timeline of the three bes projects in national blood processing service (nbps). following the strategy of temporal bracketing (langley, 1999; langley et al., 2013), this section presents the narrative by highlighting key events during the three projects. blood enterprise system-i (bes-i) project was conceived in 1998 and completed in 2003. blood enterprise systems-ii (bes-ii) was an upgrade of bes-i that started in 2004 but was abandoned in 2007 due to consistent problems. after a few years’ gap, nbps engaged in a full-suite blood enterprise system-iii (bes-iii) implementation that started in 2011 and completed in 2017. australasian journal of information systems saxena & mcdonagh 2020, vol 24, research article is controls in healthcare 9 figure 3. timeline of bes projects in nbps 4.1 blood enterprise system-i (1998-2003) national blood processing service approached its parent department in 1998 with the concern that its existing system was not ready for the year 2000 problem and proposed the implementation of a new bes. considering the gravity of the problem, the department agreed australasian journal of information systems saxena & mcdonagh 2020, vol 24, research article is controls in healthcare 10 to support the enterprise system implementation. once the proposal was approved by the department, nbps published an open tender for the procurement and implementation of the bes. however, the public tender had a very limited response due to the niche nature of the blood bank market and only two suppliers were shortlisted for consideration. one supplier was based in the us and offered two separate systems to manage the donor side and the processing side. the other supplier offered an integrated system and was the market leader with its clients across europe. the contract was awarded to the second supplier (referred to as the vendor for the remainder of the paper). the implementation started around 1998 and was due to finish by late 1999 at a cost of around €4.3 million. the implementation was managed by a project manager who came from a medical background. he was assisted by a number of superusers who were in-house domain experts. the superusers were instrumental in testing and adapting the system to suit the operational needs of nbps. the project manager and the superusers worked in close proximity with the information technology (it) unit. the it unit was responsible for ensuring that the infrastructure was of the required standard and that all technological requirements were in place for the system. the project was overseen by a steering committee which included nbps top management and the head of it. after resolving a number of technical and organisational issues, bes-i finally went live during early 2003 (i.e. a delay of more than three years) with an estimated cost of €9.3 million (i.e. more than double the estimated budget). in the interim, an audit was conducted by the public auditor in 2002 due to the cost and time overrun, which was subsequently debated in the parliamentary committee of public accounts in 2003. 4.2 blood enterprise system-ii (2004-07) since bes-i implementation took longer than expected, nbps leadership contemplated and initiated the implementation of bes-ii, an upgraded system from the same vendor. while the former was a file-based database system with limited reporting capabilities, the latter was based on a relational database system with more reporting potential. the organisation remained with the same vendor as there was not much change in the blood bank market since the tender evaluation for the first project. they also cite a benchmarking exercise conducted by another european blood service examining all possible systems at the time and still going with the same vendor. thus, nbps did not feel the need to go for a public tendering for besii. the business case for bes-ii implementation was drawn up in may 2004. the business case explained the rationale for moving from bes-i to bes-ii. it was written by the it unit outlining various requirements for the implementation of bes-ii. a steering committee was established to monitor the project, and to make timely decisions on relevant issues. in october 2004, the vendor sent an official proposal which was accepted by the steering committee. however, bes-ii implementation consistently faced obstacles during the testing and validation for the next two years. there were considerable issues with the system and the system never went beyond testing and validation. finally, the steering committee decided to abandon the project in the beginning of 2007. since nbps incurred considerable expenditure on the project, there was another public audit and a parliamentary committee debate in 2008. while the public auditor noted in its report that value for money was not achieved during bes-ii implementation, it also commended nbps for its ‘exacting testing standards’ that led to the suspension of the project. in view of the implementation failure, nbps decided to move bes-i to the new hardware that was purchased for bes-ii. the project was subject to standard testing and validation procedures and completed in 2008. australasian journal of information systems saxena & mcdonagh 2020, vol 24, research article is controls in healthcare 11 once bes-ii was abandoned, nbps sought to improve the functionality of bes-i. since it had limited reporting capabilities, nbps decided to implement a reporting module in late 2007. the reporting module, developed by the new york blood bank, fetched data from bes-i and produced the reports. the project was completed towards the end of 2009. around 2011, nbps undertook the development of a web-based electronic ordering system to receive online orders from the hospitals, thereby intending to replace the manual, telephone-based ordering process. the project was conceived by the head of operations within nbps with it unit developing the solution. by may 2011, the pilot testing of the system with a nearby hospital was completed and modifications were done in the software based on feedback received from the pilot test site. by march 2012, the electronic ordering system was live in every hospital in the country. 4.3 blood enterprise system-iii (2011-17) at the same time, however, bes-i software was approaching the end of its lifecycle and nbps recognised the need for a new system. subsequently, nbps approached its parent department with a proposal to implement a new bes. however, since the government was tightening its purse in the wake of global recession, the department vetoed the project at the beginning. the logjam with the department continued for a few months when nbps kept trying to convince the department and department kept vetoing the implementation. ultimately, nbps stressed that the timelines on support for existing hardware and software were finite and it would take some time to procure new hardware and to implement new software. this arguably posed a huge risk to the blood supply system if the existing system went down at some point in time. eventually, the department relented and approved the project. along with the approval, they recommended the establishment of the peer review group (prg) to provide external project oversight. the prg consisted one member from the department, one member from a large government health agency, and one external expert. once prg was formed, nbps prepared a requirement specification document and a tender was published before christmas 2011. the tender document sought integration of the entire blood and tissue operations within nbps, including reporting. this time as well, nbps received only two responses – one from the existing vendor (who had now purchased the reporting tool from the new york blood bank) and another vendor from the uk. after tender evaluation, nbps decided to implement an integrated product suite (bes-iii) from the existing vendor. an extensive governance mechanism was put in place during 2011-12. this time nbps also included a vendor representative in the steering committee with a view to support high-level coordination with the vendor. the implementation team was led by a project sponsor and a dedicated project manager. the project was divided into different work-streams according to the business processes and the functionality of the software. each work-stream was led by a key business user assisted by system superusers. in other words, it followed a structure similar to earlier projects, but in a more extensive manner. during bes-iii implementation, nbps also used the services of an external auditor from the private sector for a two-part review of the project. the two reviews were respectively completed in the latter half of 2013 and 2014. cognizant of the cost overruns in previous projects, nbps also made sure that project expenditure was regularly examined by the finance committee of the organisation. at the start of 2014, the finance committee noticed that vendor invoices consistently got delayed and did not properly reflect the content and amount of work. australasian journal of information systems saxena & mcdonagh 2020, vol 24, research article is controls in healthcare 12 the negotiations with the vendor finally concluded in june 2014 and the final agreement was subsequently endorsed by the finance committee. two smaller modules (one on the tracing of blood requests, the other a risk management module on nonconforming cases) went live in the first half of 2014. the core bes-iii module (centred on blood processing) was scheduled to go-live in the fourth quarter of 2014. however, the implementation got consistently delayed due to technical and change management issues, finally going live in september 2015. this was followed by resolution of reporting and performance issues in 2016. finally, the last module for bes-iii, a module for managing the information on tissue banking, went live in 2017. while bes-iii project had some time and budget overrun, it was deemed at an acceptable level and did not invite any additional scrutiny, much to the relief of nbps. 5 analysis and discussion the case analysis primarily involved identification of patterns (langley et al., 2013; pettigrew, 2012) across the three implementations and identification of the evolution of an is control portfolio in national blood processing organisation (nbps). the case analysis is presented using the enterprise system lifecycle model consisting of three phases (ali & miller, 2017) – pre-implementation, implementation, and post-implementation. as shown in table 4, while all phases use each of the five control modes, the precise control mechanisms and the control portfolio evolve throughout the enterprise system project peaking during the implementation phase. moreover, the nature and extent of control also evolves across multiple projects in an organisation. pre-implementation implementation post-implementation input resource allocation, market control user training market control outcome business case requirement specifications, testing and validation, project plan testing and validation behaviour administrative controls steering group, financial oversight, project communication, audit and review audit and review, change control clan professional norms (health) professional norms (health/it) professional norms (health) self-control individual initiative (top leadership) individual initiative (project team) individual initiative (management) table 4. control portfolio during enterprise system projects 5.1 input control the pre-implementation phase of bes projects is marked by input controls in the form of resource allocation and market control. the main input control (wiener et al., 2016, 2017) for all three projects was mainly in terms of resource allocation. while the project managers for bes-i and bes-iii came from business, the project manager of bes-ii was from the it department. the findings support the long-standing recommendation (adam & o’doharty, 2000) that project leadership and control should remain outside the it unit, preferably with the senior managers (bernroider, 2013; sumner, 2018) from the business. keeping the project australasian journal of information systems saxena & mcdonagh 2020, vol 24, research article is controls in healthcare 13 leadership with the business helps in ensuring the system-business fit and in institutionalising the implementation process within the organisation. as far as superusers are concerned, selection of domain experts as superusers (obwegeser, danielsen, hansen, helt & nielsen, 2019; sumner, 2018) worked for nbps in the long run because the superusers gained significant experience in implementing bes-i and validating bes-ii. the same set of superusers continued for bes-iii, with some additional domain experts trained as superusers for the latter. in this way, it supports the recommendations that a multiskilled team (syed, bandara, french, & stewart, 2018; sumner, 2018) having experience in both organisational processes and it is crucial for the implementation. however, case evidence also suggests that in some cases, the project members from the business side “either weren’t made by their respective managers to get involved to the levels they should have been involved or chose not to get involved at the level they should have been involved” (r8). this illustrates a form of passive resistance (murungi et al., 2019), which may be the result of a conflict between multiple institutional logics (bunduchi et al., 2020) in the healthcare setting. the niche nature of the blood bank industry resulted in the market acting as an input control mechanism (ouchi, 1979, 1980) during the pre-implementation phase. while there exist a lot of capacity in the health informatics sector in general (miah, shen, lamp, kerr, & gammack, 2019), few it vendors possess the appropriate capabilities and skills to fully support the evolving needs of the blood banks (li et al., 2007). for this reason, all three instances of preimplementation are marked by limited options available to nbps due to a niche market (olsen & sætre, 2007), as opposed to the general health informatics market where many solutions are available for hospital and clinical management. thus, during the pre-implementation phase for all three instances of bes project, the market indirectly controlled the options available to nbps. interestingly, a user notes for bes-iii that they got a certain module “because the organisation decided as a whole that they were going for [the complete suite]… never saw it operated anywhere else” (r14). this suggests that replacement risks (furneaux & wade, 2017) and perceived reputational gains (polykarpou, barrett, oborn, salge, antons, & kohli, 2018; salge et al., 2015) may also play a role in system acquisition apart from the formal gap analysis. during the implementation phase, user training also acted as an input control mechanism. there was a significant amount of time allocated to user training during bes-i since npbs “had people who did not know how to turn on a computer, suddenly sitting down trying to figure it out – f11 and f8 and all these different buttons” (r7). “for most of the end-users, they were using computers the very first time. so, it was not just training for bes-i, but also it was their first exposure to work with computers” (r18). for bes-ii, user training was not required since the project was abandoned. interestingly, user training was not given too much attention during bes-iii implementation since the business processes and user screens did not change much during bes-iii implementation. a limited amount of training was conducted towards the very end of the implementation. the “training at the last minute didn't really impact because the navigation (in becs) is so similar to the old system” (r22). this underscores the importance of users’ prior exposure to the software, suggesting limited investment in formal training during es upgrades (koh et al., 2009; barth & koch, 2019). perhaps due to these factors, nbps did not report any problems associated with bes-iii usage after go-live. this supports the importance of existing social capital (sykes, 2015; sykes & venkatesh, 2017) in system usage in an organisation. australasian journal of information systems saxena & mcdonagh 2020, vol 24, research article is controls in healthcare 14 the market also played a controlling role during the post-implementation phase. during the post-implementation phase for bes-i, while nbps recognised the need for a reporting tool, the project started only when a reliable solution was available from an external provider (new york blood bank). in contrast, internal development of electronic ordering system started since no similar feature, that could work with bes-i, was available in the market. 5.2 outcome control the three bes projects are marked by diverse control mechanisms for enforcing outcome control across the phases. in the pre-implementation phase, business case was the main behaviour control mechanism. interestingly, the business case seems to serve a dual purpose in this regard. while the ostensive purpose of the business case was to provide the rationale for the implementation (nafeeseh & al-mudimigh, 2011), the demonstrative nature of the business case preparation (berente, gal, & yoo, 2010; einhorn, marnewick, & meredith, 2019) may also be noted in the case study. while the users were involved in the process, they often did not have significant expertise in developing such documents. for bes-iii, users report that they “were given documents from another european blood service and more or less told to copy and paste them” (r17). it was also expressed by the respondents that “not enough time was put in at that stage of the process” due to time constraints on “on top of (their) routine day” (r16). the findings support the argument that user involvement in the early stages may not be feasible due to their inexperience (wagner & newell, 2007). at times, actively involving end users may not be productive due to a lack of time (wagner & piccoli, 2007) from their dayto-day job. paradoxically, although envisaged as a control mechanism, user involvement in business case development may be ineffective (lyytinen & newman, 2015; willis & chiasson, 2007) at the same time. during the implementation phase, requirement specifications, testing and validation, and project plan were the main outcome control mechanisms. the project team used requirement specification to lock down user expectation. this is in line with the use of requirement specifications as a scope control and expectation control (einhorn et al., 2019) mechanism. similarly, project plan was used as a mechanism to control the project timeline. while bes-i and bes-ii were marked by huge deviation from the project plan, most respondents noted “a preoccupation” with and “an unmerciful push” (r4) towards the go-live in bes-iii. this reflects a teleological dimension (van de ven & poole, 1995; wiener et al., 2016) in the form of learning from prior projects. in one instance where project leadership was aware that the golive date would be postponed for bes-iii core module, the team members were informed about it only at the very end “to keep the pressure on and the focus on getting the work done” (r2). as the narrative suggests, however, there were still some delays in the project. this suggests that project plans in dynamic environments should be used as a reference only and need to be constantly updated (petit, 2012) in response to emerging situations. however, the main outcome control mechanism in all three instances of bes implementation was testing and validation of the system. perhaps owing to its healthcare context, nbps essentially subscribes to validation view of the implementation process. within nbps, “validation lifecycle process is very closely akin to project management, especially in terms of specification, design, testing, (and) delivery” (r3). there is “a great emphasis on validation reports for all the software development in nbps… and there is a quality manager sitting in it, who ensures that (the) it systems fulfil all the quality requirements” (r15). considering the concern related to patient safety and the complexity of institutional logics (bunduchi et al., australasian journal of information systems saxena & mcdonagh 2020, vol 24, research article is controls in healthcare 15 2020; poba-nzaou et al., 2014; salge et al., 2015) in the healthcare setting, the level of testing and validation does not seem misplaced. in fact, testing and validation also worked as an outcome control during the postimplementation phase as well. the reporting project team followed similar procedures for report development validation and testing that are reported for the implementation phase: we did specs for every one of them and then tested them against the spec. got them built, got them in, had a look at them, got them refined, then validated them. so, that's how we did it. (r4) electronic ordering system initiation was a bit informal in the sense that it was “born out of (the) director of operations and (the) head of processing approaching a member of it directly” and “there were no controls put in place” (r5). however, later on, outcome controls were placed, and it was ensured the system was properly validated and that it was a secure system. therefore, for both the reporting system and the electronic ordering system, testing and validation was the main outcome control mechanism during the post-implementation phase of bes-i. 5.3 behaviour control perhaps owing to the public sector context, behaviour control in the pre-implementation phase is marked by administrative controls in the form of departmental approval and public tendering processes. for bes-i and bes-iii, nbps asked for departmental approval. for besii acquisition though, nbps did not seek clearance from its parent department, effectively bypassing administrative control. instead, the vendor sent an official proposal for the project which was then accepted by nbps. as noted in the case, the main reason for this was bes-ii being an upgrade of bes-i system, and unchanged market condition. however, getting the departmental approval for bes-iii proved to be a difficult job due to prevailing economic environment. thus, the pre-implementation phase demonstrates the politically contested and negotiated nature of the acquisition process (matinheikki et al., 2019; moe et al., 2017) in the public sector. a second administrative control for the acquisition process is the public tendering process. while nbps used public tendering processes for the acquisitions requiring capital investments for the first and the third instances, it did not do so in the second instance. this reflects the dialectics of public sector procurement (moe et al., 2017) in which the public entities may wish to continue extensive collaboration with the vendor, but also are restricted by existing procurement rules. the implementation phase saw the most extensive form of behaviour control across multiple levels, supporting the findings (soh et al., 2011; nuwangi et al., 2018) on hierarchy of controls and the multiplicity of controller-controlee relationships in a multi-stakeholder environment. the steering group was the main behaviour control mechanism in all three project instances. a steering group comprising of top management, and heads of it and business units oversaw the implementation of bes-i and bes-ii. the steering group for bes-iii also included a highlevel vendor representative, in addition to four members from the senior management team at nbps. it was felt that “the set-up of the steering group meetings helped” (r3) the project and ensured that the issues were “being raised with the vendor at the highest level” (r2). thus, the findings support the notion of the steering group as an effective behavioural control mechanism (loonam, mcdonagh, kumar, & o'regan, 2014; sumner, 2018) for enterprise systems implementation. case evidence also suggests that such temporary hybridisation australasian journal of information systems saxena & mcdonagh 2020, vol 24, research article is controls in healthcare 16 (matinheikki et al., 2019) of including the vendor in the steering group (choudhury & sabherwal, 2003) helps in early resolution of implementation problems. this also reflects the teleological learning (van de ven & poole, 1995; wiener et al., 2016) from earlier projects when the vendor was not part of the steering group. blood enterprise system-iii implementation also included a financial oversight as a cost control mechanism (matinheikki et al., 2019; susilo et al., 2007; grabski & leech, 2007) during the implementation phase. this probably was due to the repercussions associated with bes-i and bes-ii that went significantly over-budget and invited an audit by the public auditor, once again exhibiting a teleological evolution of controls. cost-control with the vendor involved “being very challenging and scrutinising every invoice” (r1) and verifying it with the actual work performed. there was also a quarterly review of the bes-iii project by the finance committee of nbps. it is a sub-committee of the nbps board and the “project manager would present at each meeting of the finance committee a project update and a status report on the project” (r2). regular project meetings were identified as a key form of behaviour control (matinheikki et al., 2019; niederman et al., 2018; wiener et al., 2016) during the implementation phase. for bes-i, separate project groups operated in the two major centres, and there were no joint meetings. during bes-ii, this was remedied, and project management was integrated across the two centres with a single project team. while there were larger project meetings (usually once a month) for bes-iii, work-stream meetings were more frequent (usually once a week). nbps had “an implementation plan for each of the different work-streams” (r3). each workstream had a “weekly meetings with (the) project manager (and) issues were discussed weekly” (r22). the majority of the respondents agreed that the work-stream structure worked well for the project, supporting the observation that such structure may help in strengthening structural ties (wiedermann & wiesche, 2018) within the larger project team. the post-implementation phase of bes implementation in nbps is marked by project audit and review especially by external parties. for bes-i and bes-ii, nbps ended up justifying itself before the public auditor and the parliamentary committee. in both cases, there were many “questions asked from the government point of view at that time as to why so much money was spent” (r22). in that sense, public auditor and parliamentary committee acted as a postfacto behaviour control for entire process. after the implementation failure of bes-ii, nbps also commissioned an external review of the project and a professional audit firm was appointed to undertake this review. around the same time, nbps also commissioned a study by an academic expert to suggest steps to maximise the business value from it. both these initiatives can be seen as efforts to repair their reputation (polykarpou et al., 2018) in the public service context that demands higher levels of public accountability (campbell, mcdonald & sethibe, 2010; matinheikki et al., 2019; syed et al., 2018). perhaps therefore, being “very conscious of (what) had happened in the past” (r1), nbps incorporated project audit and review as a control mechanism also during the implementation phase of bes-iii. as noted in section four, there were two audits of bes-iii by external audit partners. furthermore, nbps constituted a prg for bes-iii when suggested by the parent department. formation of prg is a mandatory feature of large-scale it projects in the national public sector. however, since nbps is not funded through voted monies, it need not form a prg by law. yet, it formed a prg to ensure legitimacy (matinheikki et al., 2019) within the public sector context. it was also noted that the prg “did ensure that (nbps) did retain australasian journal of information systems saxena & mcdonagh 2020, vol 24, research article is controls in healthcare 17 discipline around project management and around delivering through the different stages” (r2). thus, the findings support the use of external reporting and audits (grabski & leech, 2007; chang, yen, chang, & jan, 2014) as behavioural control mechanisms during enterprise systems implementation. at the same time, it also shows a teleological evolution (van de ven & poole, 1995; wiener et al., 2016) of behaviour controls across multiple projects. a high level of behavioural control was also visible in the change controls during the postimplementation phase. any maintenance and upgrade activity within nbps require a high level of change control procedures (wang, ju, jiang, & klein, 2008), a form of strict behaviour control. for some, “there (is) an insane amount of change control and a very high level of validation (surpassing) the level of validations and quality control (in the pharmaceutical industry)” (r18). there was a feeling that within nbps, “it can be quite difficult to get a change. there is an awful lot of paperwork. there is a validation procedure and a lot of paperwork even for the smallest change on a system” (r16). this particular element of change control also relates to the clan control discussed in the next section. 5.4 clan control due to the nature of its business, multiple institutional logics (berente et al., 2010, 2019; bunduchi et al., 2020; salge et al., 2015) – medical, nursing, it, managerial – clan control could be observed in nbps. as one participant notes [national blood processing service] is an organisation comprised of many different kinds of professionals. there are senior consultant doctors (who) have a very strong view on just about everything. you have a full cohort of medical scientists, who will regard themselves as a professional of their own. there is a full cohort of nurses in the organisation, and then you've got the it professionals, who have their own particular technical set of blinkers that they look at, everything at. so, you've all these professional groups, sitting at a table, all looking over at everybody else's what they are doing and commenting on what everybody else is doing. (r1) due to its health service context, nbps is risk averse with a safety culture (poba-nzaou et al., 2014; salge et al., 2015). as one participant notes: “our concern in scientific side is safety. have we tested it and is everything right? and is the donation fit to go? so, we are buried in a whole lot of that” (r4). high level of change control (noted in section 5.3) is justified in the organisation because “there is a safety in that. so, it means that people aren't changing things at willy-nilly… it's quite cumbersome to make a change but again, there is a safety element in that” (r16). this is aligned with the observation that health professionals prefer sticking to tried and tested systems (furneaux & wade, 2017) when it comes to healthcare it. while the health sector norms were found to be effective during the pre-implementation and the post-implementation phase, it was at odds with it professional culture especially during the implementation phase. while the health norms prioritise safety, even if at the expense of time, it professionals sometimes felt frustrated due to it. this usually resulted in it professionals taking on additional responsibilities, reflecting a clan control by the it unit. “so, it's the people in it (who) had to pick up the slack for those people” (r12). the it unit was “doing it since they (business) were not doing it” (r18). the result was that resources from the it unit (including superusers) worked “above and beyond, working on evenings and on weekends” (r18). irrespective of the project, the clan control within the it unit (chua et al., 2012; goldbach & benlian, 2015; wiedermann & wiesche, 2018) is clearly evident from the quotation below: australasian journal of information systems saxena & mcdonagh 2020, vol 24, research article is controls in healthcare 18 what it (unit) tends to do is to pick-up what other people drop. that is just the nature of what we do. i think it's just a mindset. you may kind of know yourself that technically-minded people have a particular way of going about things. so, if they see a problem, they generally want to fix it. i did find that we ended up picking an awful lot of things in the project that weren't strictly our responsibility to do. (r8) while professional norms played a greater role in the it unit assuming additional responsibilities, some bit of self-control was also evident, as noted in the next section. 5.5 self-control self-control in the bes project was evident in the form of individual initiatives. in all three instances, the initiative from the top management played an important role during the preimplementation phase. from nbps board minutes, it is clear that the chief executive presented the rationale for the implementation in nbps board meetings, resulting in the board accepting the proposal for the implementation of the new system. case evidence supports the observation that senior management are the most likely activators (bernroider, 2013) for enterprise systems projects. support from top management is deemed crucial for such projects (bhoola et al., 2014; loonam et al., 2014; syed et al., 2018; sumner, 2018) since “once you get the leadership and the buy-in from the top (and) once you get the support at a senior administration level…, you can get the job done” (r3). there was a significant amount of individual initiative from the project team during the implementation phase. this was mainly due to the resource allocation discussed in section 5.1. for instance, the superusers had to concentrate on the implementation as well as to focus on the maintenance of the existing system. the project resourcing was such that “the same people who (were) supporting the current day to day production applications (were) implementing projects” (r8). this resulted in a lot of multi-tasking. the superuser “would be writing the test scripts for [bes-iii] and then a user may call to solve a daily problem related to [bes-i]” (r18). understandably this decision was made to exploit superusers’ in-depth knowledge of the system (obwegeser et al., 2019; sumner, 2018), but it also increased their workload (wagner & piccoli, 2007). for this reason, superusers seemed to impose a lot of self-control upon themselves. since this was noted for the domain experts as well as the members of the it unit, it was taken as a sign of self-control rather than clan control. as participants from both groups observe: i've been working weekends a lot, and extremely long hours during the week, i've to say. but there was nobody else to do my job. so, i haven't had my holidays really either… i was doing more than i should have been doing. i would have been down in the labs doing some it work. i'm not saying that it is it's fault, but when things needed to be done straightaway… you want to do it. so, i went down to the labs myself and just did it.” (r12) there wasn't enough hours in the day to get everything done you know because and then you're kind of under pressure to get things done at a certain time and to get documents signed off and changes made and things like that, you know. so, it was kind of pressurised, but every project is like that. (r21) in the post-implementation phase, self-control from the management played a major role in the introduction of new modules. the initiative for the reporting module came from the director of operations who joined nbps in 2005 and saw value in introducing a reporting and australasian journal of information systems saxena & mcdonagh 2020, vol 24, research article is controls in healthcare 19 forecasting system. similarly, the development of electronic ordering system was initiated when the director of operations and the head of processing unit approached a member of the it unit. this illustrates the extended role of senior management (bernroider, 2013; loonam et al., 2014) in enterprise systems projects beyond investment decisions. 6 conclusion 6.1 contribution this study offers a number of contributions towards a holistic understanding of is controls. first, this study provides a rich description of the evolution of an is control portfolio during an enterprise system lifecycle implemented in a regional blood bank in western europe, something that is relatively scarce in extant literature (cram et al., 2016; inácio & marques, 2018). the study demonstrates that the nature and extent of controls changes across the phases of the enterprise system lifecycle, thus supporting a lifecycle perspective (van de ven and poole, 1995; van de ven & sun, 2011) within a project. while all control modes are found to be applicable across all phases, they differ in terms of control mechanisms applied in different phases. this is a significant contribution of this study as limited evidence is available on the clan control and self-control dimensions (wiener et al., 2016), especially in the presence of multiple institutional logics. second, by studying the three instances of the implementation in a single case organisation, it presents the historical evolution (chua & myres, 2018) of a control portfolio across multiple projects as they unfold over time. in particular for behaviour controls, the findings suggest a teleological (van de ven & poole, 1995; wiener et al., 2016) evolution of a control portfolio in which new control mechanisms are introduced over time based on the adaptive learning process from earlier projects. third, this study moves beyond a single controller-controlee dyad and exhibits the multistakeholder and multi-level (soh et al., 2011; nuwangi et al., 2018) nature of the control portfolio. this is particularly salient in the case of behaviour controls during the implementation phase. while project communication provided the control within project, other behavioural controls (steering group, financial oversight, audit and review) acted as the control mechanisms over (wiener et al., 2017) the project. this supports the observation (ouchi, 1978) that different levels of the hierarchy may rely on different behaviour controls. finally, by exhibiting the influence of institutional context and niche market, this study underlines the contextual nature (avgerou, 2019; berente, et al., 2019; matinheikki et al., 2019) of the enterprise systems implementation and associated controls in the blood bank market. application of external controls (public auditor, parliamentary committee, prg) is a reflection of the public sector context where demands for public accountability (campbell et al., 2010; matinheikki et al., 2019; syed et al., 2018) are usually higher. the influence of the health sector (bunduchi et al., 2020; poba-nzaou et al, 2014; salge et al., 2015) is also evident in the form of the prevailing safety culture and the focus on a high level of validation and change control. due to the niche nature (olsen & sætre, 2007) of the blood bank market, this study also demonstrates the influence of market as an input control mechanism (ouchi 1979, 1980) in the form of limited vendor/product options. australasian journal of information systems saxena & mcdonagh 2020, vol 24, research article is controls in healthcare 20 6.2 implications there are a number of implications of this study for practicing managers. first, the institutional and market influences on the control portfolio indicate that the top management team needs to be aware of the contextual factors when planning and executing enterprise systems projects. controls due to the contextual factors may increase the demand for resources as well as the need for top management involvement. these requirements are to be factored-in when planning a project in a complex environment like public health. second, multi-stakeholder control structure identified in this study suggests that there are multiple controller-controlee relationships with the same managers playing both roles at times. for instance, while top management primarily acted as a controller during the pre-implementation and the implementation phase, they fulfilled the controlee role when representing the organisation before the national auditor during the post-implementation phase. this might require managers to revise their mental models as the project progresses. third, a related implication is that managers may need to draw from different skillsets (e.g. technical, managerial, or political) based on the control portfolio evolution during the enterprise systems lifecycle, depending upon their role and whom they are interacting with. fourth, the importance of an informed and powerful steering group cannot be overemphasised for enterprise system implementation. in addition, the case evidence suggests that it might be beneficial to include a high-level vendor representative in any such group. this may facilitate the resolution of issues from the very beginning. fifth, the outcome of besiii suggests that a high number of formal controls may be associated with positive project outcomes. hence, it might be better for the managers to be comprehensive in applying formal controls. even though this might take time in the beginning, it would eventually be beneficial for the overall project in terms of project completion and ensuring legitimacy. finally, for is audit and evaluation functions, there is a need to recognise and respond to the evolving nature of an organisation’s control portfolio as it relates to enterprise systems implementation. of necessity, this requires a commitment to (re)educating key stakeholders about the critical linkages between controls at various phases/stages of the enterprise systems lifecycle and the pursuit of purposeful and positive is-enabled business change. 6.3 limitations and future development some limitations of this study need to be outlined. first, this study acknowledges the context specificity of the findings and limited external generalisability due to the focus on a single case. however, the rich description provided in this longitudinal case seeks generalisation to theoretical propositions (yin, 2017) as opposed to statistical generalisation to a population. second, due to the retrospective nature of the interview method, there might be a possibility of post-hoc rationalisation by the participants to provide some coherence to the events and a rational purpose to their actions. however, publicly available documentation, reports from the national auditor, and parliamentary committee debates are used as triangulation devices (miles et al., 2014; yin, 2017) to ensure internal validity (maxwell, 2017) of the account. third, due to access restrictions, the vendor perspective could not be included. the vendor perspective could have added richness to the case study and may have contributed to overall validity of the study. finally, due to time limitations, we could not investigate the postimplementation phase of bes-iii. since the bes-ii project was abandoned during implementation, the findings for the post-implementation phase rely on bes-i project only. australasian journal of information systems saxena & mcdonagh 2020, vol 24, research article is controls in healthcare 21 in terms of future development, the findings from this study suggest that there is merit in pursuing further research into the evolving nature of control portfolios in enterprise systems initiatives in diverse contexts. moreover, since the enterprise systems industry is moving towards cloud-based implementation, future research may focus on studying the difference between the control mechanisms operating in a traditional on-premise implementation versus those in the cloud environment. of particular interest here is a deep exploration of the role and influence of is audit and evaluation functions in shaping control portfolios that contribute directly to achieving positive is-enabled business change. references adam, f., & o'doherty, p. (2000). lessons from enterprise resource planning implementations in ireland–towards smaller and shorter erp projects. journal of information technology, 15(4), 305-316. alfonso, e., xie, x., augusto, v., & garraud, o. (2012). modeling and simulation of blood collection systems. health care management science, 15(1), 63-78. ali, m., & miller, l. (2017). erp system implementation in large enterprises–a systematic literature review. journal of enterprise information management, 30(4), 666-692. ashford, p., gozzard, d., jones, j., revill, j., wallis, j. ... & cohen, h. (2000). guidelines for blood bank computing. transfusion medicine, 10(4), 307-314. avgerou, c. (2019). contextual explanation: alternative approaches and persistent challenges. mis quarterly, 43(3), 977-1006. barth, c., & koch, s. (2019). critical success factors in erp upgrade projects. industrial management & data systems, 119(3), 656-675 berente, n., gal, u., & yoo, y. (2010). dressage, control, and enterprise systems: the case of nasa's full cost initiative. european journal of information systems, 19(1), 21-34. berente, n., lyytinen, k., yoo, y., & maurer, c. (2019). institutional logics and pluralistic responses to enterprise system implementation: a qualitative meta-analysis. mis quarterly, 43(3), 873-902. bernroider, e. w. (2013). effective erp adoption processes: the role of project activators and resource investments. european journal of information systems, 22(2), 235-250. bhoola, v., hiremath, s. b., & mallik, d. (2014). an assessment of risk response strategies practiced in software projects. australasian journal of information systems, 18(3). https://doi.org/10.3127/ajis.v18i3.923 bunduchi, r., tursunbayeva, a., & pagliari, c. (2019). coping with institutional complexity. information technology & people. 33(1), 311-339. campbell, j., mcdonald, c., & sethibe, t. (2009). public and private sector it governance: identifying contextual differences. australasian journal of information systems, 16(2), 5-18. https://doi.org/10.3127/ajis.v16i2.538 chang, s. i., yen, d. c., chang, i. c., & jan, d. (2014). internal control framework for a compliant erp system. information & management, 51(2), 187-205. choudhury, v., & sabherwal, r. (2003). portfolios of control in outsourced software development projects. information systems research, 14(3), 291-314. australasian journal of information systems saxena & mcdonagh 2020, vol 24, research article is controls in healthcare 22 chua, c. e. h., lim, w. k., soh, c., & sia, s. k. (2012). enacting clan control in complex it projects: a social capital perspective. mis quarterly, 36(2), 577-600. chua, c. e. h., & myers, m. d. (2018). social control in information systems development: a negotiated order perspective. journal of information technology, 33(3), 173-187. cram, w. a., brohman, k., & gallupe, r. b. (2016). information systems control: a review and framework for emerging information systems processes. journal of the association for information systems, 17(4), 2. cumming, p. d., kendall, k. e., pegels, c. c., seagle, j. p., & shubsda, j. f. (1976). a collections planning model for regional blood suppliers: description and validation. management science, 22(9), 962-971. einhorn, f., marnewick, c., & meredith, j. (2019). achieving strategic benefits from business it projects: the critical importance of using the business case across the entire project lifetime. international journal of project management, 37(8), 989-1002. edqm (2017). guide to the preparation, use and quality assurance of blood components. european directorate for the quality of medicines & healthcare. https://www.edqm.eu/en/bloodguide furneaux, b., & wade, m. (2017). impediments to information systems replacement: a calculus of discontinuance. journal of management information systems, 34(3), 902-932. george, a. l., & bennett, a. (2005). case study and theory development in the social sciences. cambridge, ma. goldbach, t., & benlian, a. (2015). how social capital facilitates clan control on software platforms to enhance app-developers’ performance and success. in thirty sixth international conference on information systems, fort worth 2015 gopal, a., & gosain, s. (2010). research note – the role of organizational controls and boundary spanning in software development outsourcing: implications for project performance. information systems research. 21(4), 960-982. grabski, s. v., & leech, s. a. (2007). complementary controls and erp implementation success. international journal of accounting information systems, 8(1), 17-39. gregory, r. w., & keil, m. (2014). blending bureaucratic and collaborative management styles to achieve control ambidexterity in is projects. european journal of information systems, 23(3), 343-356. gregory, r. w., beck, r., & keil, m. (2013). control balancing in information systems development offshoring projects. mis quarterly, 1211-1232. gupta, o., priyadarshini, k., massoud, s., & agrawal, s. k. (2004). enterprise resource planning: a case of a blood bank. industrial management & data systems, 104(7), 589-603. heiskanen, a., newman, m., & eklin, m. (2008). control, trust, power, and the dynamics of information system outsourcing relationships: a process study of contractual software development. the journal of strategic information systems, 17(4), 268-286. henderson, j. c., & lee, s. (1992). managing i/s design teams: a control theories perspective. management science, 38(6), 757-777. australasian journal of information systems saxena & mcdonagh 2020, vol 24, research article is controls in healthcare 23 inácio, h., & marques, r. p. (2018, june). analysis of the research on internal control in enterprise resource planning systems. in 2018 13th iberian conference on information systems and technologies (cisti). ieee. jpac (2013). handbook of transfusion medicine. joint united kingdom (uk) blood transfusion and tissue transplantation services professional advisory committee. https://www.transfusionguidelines.org/transfusion-handbook katsaliaki, k., & brailsford, s. c. (2007). using simulation to improve the blood supply chain. journal of the operational research society, 58(2), 219-227. kendall, k. e., & lee, s. m. (1980). formulating blood rotation policies with multiple objectives. management science, 26(11), 1145-1157. kirsch, l. j. (1997). portfolios of control modes and is project management. information systems research, 8(3), 215-239. kirsch, l. j. (2004). deploying common systems globally: the dynamics of control. information systems research, 15(4), 374-395. kirsch, l. j., & cummings, l. l. (1996). contextual influences on self-control of is professionals engaged in systems development. accounting, management and information technologies, 6(3), 191-219. koh, s. c. l., gunasekaran, a., & cooper, j. r. (2009). the demand for training and consultancy investment in sme-specific erp systems implementation and operation. international journal of production economics, 122(1), 241-254. langley, a., & meziani, n. (2020). making interviews meaningful. the journal of applied behavioral science, 56(3) 370–391. langley, a., smallman, c., tsoukas, h., & van de ven, a. h. (2013). process studies of change in organization and management: unveiling temporality, activity, and flow. academy of management journal, 56(1), 1-13. li, b. n., chao, s., & dong, m. c. (2007). sibas: a blood bank information system and its 5year implementation at macau. computers in biology and medicine, 37(5), 588-597. loonam, j., mcdonagh, j., kumar, v., & o'regan, n. (2014). top managers & information systems: ‘crossing the rubicon!'. strategic change, 23(3-4), 205-224. lyytinen, k., & newman, m. (2015). a tale of two coalitions marginalising the users while successfully implementing an enterprise resource planning system. information systems journal, 25(2), 71–101. mao, j. y., zhang, x., & song, w. (2008). information systems development in a low maturity environment: an exploratory case study on control modes. pacis 2008 proceedings, 263. matinheikki, j., aaltonen, k., & walker, d. (2019). politics, public servants, and profits: institutional complexity and temporary hybridization in a public infrastructure alliance project. international journal of project management, 37(2), 298-317. maxwell, j. a. (2017). the validity and reliability of research: a realist perspective. eds. wyse, d., selwyn, n., smith, e., & suter, l. e. (eds.). the bera/sage handbook of educational research (pp. 116-140). uk: sage. australasian journal of information systems saxena & mcdonagh 2020, vol 24, research article is controls in healthcare 24 mazaheri, e., lagzian, m., & hemmat, z. (2020). research directions in information systems field, current status and future trends. australasian journal of information systems, 24. https://doi.org/10.3127/ajis.v24i0.2045 miah, s., shen, j., lamp, j. w., kerr, d., & gammack, j. (2019). emerging insights of health informatics research: a literature analysis for outlining new themes. australasian journal of information systems, 23. https://doi.org/10.3127/ajis.v23i0.2137 miles, m. b., huberman, a. m., & saldana, j. (2014). qualitative data analysis. sage. moe, c. e., newman, m., & sein, m. k. (2017). the public procurement of information systems: dialectics in requirements specification. european journal of information systems, 26(2), 143-163. murungi, d., wiener, m., & marabelli, m. (2019). control and emotions: understanding the dynamics of controllee behaviours in a health care information systems project. information systems journal, 29(5), 1058-1082. nafeeseh, r. a., & al-mudimigh, a. s. (2011). justifying erp investment: the role and impacts of business case a literature survey. international journal of computer science and network security, 11(1), 185-193. niederman, f., müller, b., & march, s. t. (2018). using process theory for accumulating project management knowledge: a seven-category model. project management journal, 49(1), 624. nuwangi, s. m., & sedera, d. (2018). too tight – too loose: controlling lessons from a failed is development project. australasian conference on information systems 2018. uts, sydney. nuwangi, s. m., sedera, d., & srivastava, s. (2018). multi-layered control mechanisms in software development outsourcing. pacis 2018 proceedings. 167. obwegeser, n., danielsen, p., hansen, k. s., helt, m. a., & nielsen, l. h. (2019). selection and training of super-users for erp implementation projects. journal of information technology case and application research, 21(2), 74-89. olsen, k. a., & saetre, p. (2007). it for niche companies: is an erp system the solution?. information systems journal, 17(1), 37-58. ouchi, w. g. (1978). the transmission of control through organizational hierarchy. academy of management journal, 21(2), 173-192. ouchi, w. g. (1979). a conceptual framework for the design of organizational control mechanisms. management science, 25(9), 833-848. ouchi, w. g. (1980). markets, bureaucracies, and clans. administrative science quarterly, 129141. pandey, s., & sumant, o. (2020). blood bank market by product type, function, bank type, and end user: global opportunity analysis and industry forecast, 2020–2027. allied market research. https://www.alliedmarketresearch.com/blood-bank-market pentland, b. t. (1999). building process theory with narrative: from description to explanation. academy of management review, 24(4), 711-724. australasian journal of information systems saxena & mcdonagh 2020, vol 24, research article is controls in healthcare 25 petit, y. (2012). project portfolios in dynamic environments: organizing for uncertainty. international journal of project management, 30(5), 539-553. pettigrew, a. m. (2012). context and action in the transformation of the firm: a reprise. journal of management studies, 49(7), 1304-1328. poba-nzaou, p., uwizeyemungu, s., raymond, l., & paré, g. (2014). motivations underlying the adoption of erp systems in healthcare organizations: insights from online stories. information systems frontiers, 16(4), 591-605. polykarpou, s., barrett, m., oborn, e., salge, t. o., antons, d., & kohli, r. (2018). justifying health it investments: a process model of framing practices and reputational value. information and organization, 28(4), 153-169. salge, t. o., kohli, r., & barrett, m. (2015). investing in information systems: on the behavioural and institutional search mechanisms underpinning hospitals’ is investment decisions. mis quarterly, 39(1). saxena, d., & mcdonagh, j. (2017). a systematic literature review of the enterprise systems research in leading is journals (2000-2015). in proceedings of the twelfth midwest association for information systems conference, us: springfield, illinois, may 2017 soh, c., chua, c. e. h., & singh, h. (2011). managing diverse stakeholders in enterprise systems projects: a control portfolio approach. journal of information technology, 26(1), 1631. somers, t. m., & nelson, k. g. (2004). a taxonomy of players and activities across the erp project life cycle. information & management, 41, 257–278. srivastava, s. c., & teo, t. s. (2012). contract performance in offshore systems development: role of control mechanisms. journal of management information systems, 29(1), 115-158. sumner, m. (2018). erp project retrospectives—55 enterprise systems: evaluating project success, lessons learned, and business outcomes. in proceedings of the thirteenth midwest association for information systems conference, us: saint louis, missouri 2018. syed, r., bandara, w., french, e., & stewart, g. (2018). getting it right! critical success factors of bpm in the public sector: a systematic literature review. australasian journal of information systems, 22. https://doi.org/10.3127/ajis.v22i0.1265 susilo, a., heales, j., & rohde, f. (2007). project management effectiveness: the choice-formal or informal controls. australasian journal of information systems, 15(1). https://doi.org/10.3127/ajis.v15i1.480 tiwana, a., & keil, m. (2009). control in internal and outsourced software projects. journal of management information systems, 26(3), 9-44. van de ven, a. h., & poole, m. s. (1995). explaining development and change in organizations. academy of management review, 20(3), 510-540. van de ven, a. h., & sun, k. (2011). breakdowns in implementing models of organization change. academy of management perspectives, 25(3), 58-74. wagner, e. l., & newell, s. (2007). exploring the importance of participation in the postimplementation period of an es project : a neglected area. journal of the association for information systems, 8(10), 508–524. australasian journal of information systems saxena & mcdonagh 2020, vol 24, research article is controls in healthcare 26 wagner, e. l., & piccoli, g. (2007). moving beyond user participation to achieve successful is design. communications of the acm, 50(12), 51-55. wang, e. t., ju, p. h., jiang, j. j., & klein, g. (2008). the effects of change control and management review on software flexibility and project performance. information & management, 45(7), 438-443. wiedemann, a., & wiesche, m. (2018). how to implement clan control in devops team. twenty-fourth americas conference on information systems, us: new orleans, 2018. wiener, m., remus, u., heumann, j., & mähring, m. (2015). the effective promotion of informal control in information systems offshoring projects. european journal of information systems, 24(6), 569-587. wiener, m., mähring, m., remus, u., & saunders, c. s. (2016). control configuration and control enactment in information systems projects: review and expanded theoretical framework. mis quarterly, 40(3), 741-774. wiener, m., cram, a., & remus, u. (2017). the view from the top – how senior executives exercise control over information systems projects to enhance performance. in proceedings of the 25th european conference on information systems (ecis), guimarães, portugal, june 5-10, 2017 (pp. 1423-1438). willis, r., & chiasson, m. (2007). do the ends justify the means? a gramscian critique of the processes of consent during an erp implementation. information technology & people, 20(3), 212-234. winter, s., berente, n., howison, j., & butler, b. (2014). beyond the organizational ‘container’: conceptualizing 21st century sociotechnical work. information and organization, 24(4), 250-269. yin, r. k. (2017). case study research and applications: design and methods. sage publications. australasian journal of information systems saxena & mcdonagh 2020, vol 24, research article is controls in healthcare 27 appendix -1: coding tree code control mechanism control mode being public sector administrative control behaviour control difference with pvt sector different processes following eu directive prg oversight procurement process project approval public accountability regulatory oversight it audit audit and review learning from past project audit project evaluation project review change management change control focus on change control impact of change limited bpr process change quality control risk management budget overrun financial oversight budgetary control it budgeting project costing project meetings project communication user buy-in user expectation user resistance work stream meetings communication within organisation ict council steering group it representation in tmt management support to it project management steering committee following international standard professional norms (health) clan control organisation culture risk aversion in the health service australasian journal of information systems saxena & mcdonagh 2020, vol 24, research article is controls in healthcare 28 risk aversion in nbps safety focus different professional cultures professional norms (health/it) ensuring professional accountability fragmented view of the organisation inter-departmental tension internal work relationships organisation complexity validation credibility following technology curve professional norms (it) it taking the slack being first user market control input controls benchmarking against peers blood transfusion industry complete suite dominant vendor external development of the system in-house/outsourcing nature of blood operations niche market relationship with vendor vendor’s push system usage in the industry dedicated project manager resource allocation domain expert as superuser it infrastructure it staffing multiple responsibilities project leadership project resourcing project structure project team single point of dependency staffing superusers location superuser's system knowledge work streams prior exposure to system user training staff it skills training needs bbcs business case outcome controls blood control system australasian journal of information systems saxena & mcdonagh 2020, vol 24, research article is controls in healthcare 29 business integration business process ownership customisation data migration database integration gap analysis introducing isbt-128 is strategy process specificity project scoping single site processing system centrality to business system maturity system usage in organisation understanding hospital's needs understanding impact of change cascading effect on projects project plan change in go-live dates focus on go-live incremental development no parallel systems project lifecycle project planning time overrun validation view of project system definitions requirement specifications system functionality user involvement competing access testing and validation hardware constraints system configuration system constraints system performance system quality system security system testing electronic document management system individual initiative (management) self-control electronic ordering system implementing boss information deficit introducing automation australasian journal of information systems saxena & mcdonagh 2020, vol 24, research article is controls in healthcare 30 mis reporting operational improvement operational inefficiency work stress individual initiative (project team) initiating project proposal individual initiative (top leadership) tmt support copyright: © 2020 saxena & mcdonagh. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v24i0.2635 http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems wahlstrom, ulhaq & burmeister 2020, vol 24, selected papers from the 2019 aice conference privacy by design 1 privacy by design: a holochain exploration kirsten wahlstrom university of south australia, australia kirsten.wahlstrom@unisa.edu.au anwaar ulhaq school of computing and mathematics, charles sturt university, australia oliver burmeister school of computing and mathematics, charles sturt university, australia abstract privacy is important because it supports freedom, dignity, autonomy, justice, and democracy, and therefore it is important that privacy is studied in ontologically robust ways. a form of privacy is implemented in the right to be forgotten, which is a human right established by the european court of justice. blockchain and holochain are examples of recently emerged technologies that were shaped by, and are now shaping of, social contexts in which economic transactions may occur. the right to be forgotten represents a compliance challenge for public and private implementations of blockchain technology. this paper describes a few of these challenges. keywords: holochain, blockchain, privacy by design, ethics, crypto 1 introduction the role of privacy in information systems (is) design has been well established in numerous domains, including the one that is the focus of this paper, namely the blockchain (hackius, reimers, & kersten, 2019; zyskind & nathan, 2015). information management and governance, ensuring data is tamper proof and managing business risks related to privacy have created challenges for many companies, because information is seen as a strategic asset. a blockchain is a tamper-proof and distributed data repository that supports information management and governance through automating trust, thereby reducing transaction costs; for these reasons, blockchain technology is of advantage in business (vovchenko, andreeva, orobinskiy, & filippov, 2017). however, blockchains suggest challenges for privacy and the aim of this paper is to explore some of these challenges. privacy has been conflated with security and confidentiality (mittelstadt, fairweather, mcbride, & shaw, 2013; wahlstrom & fairweather, 2013) and this ontological vagueness obscures all three concepts and curtails clarity in research. in this paper, a clear definition of privacy enables us to establish a stronger position on privacy with respect to the blockchain, utilising the example of holochain. privacy is important because it supports freedom (hull, 2015), dignity (panichas, 2014), autonomy (nissenbaum, 2004), justice (introna, 2000), and democracy (schwartz, 1999). australasian journal of information systems wahlstrom, ulhaq & burmeister 2020, vol 24, selected papers from the 2019 aice conference privacy by design 2 westin considers that “… privacy is a quality of life topic worth the best scholarship, thoughtful advocacy, and continuing attention of us all” (2003, p451). the literature on privacy encompasses diverse themes (wahlstrom, fairweather, ashman, & istance, 2013): control of data and self-determination (bernoth, dietsch, burmeister, & schwartz, 2014; westin, 1967) restricting access to self and data (moor, 1990), privacy and data as commodities that may be traded (posner, 1977), privacy as a social good differing from context to context (burmeister, islam, dayhew, & crichton, 2015; dix, 1990), and the view that privacy takes shape according to the technologies forming the infosphere (floridi, 2005). data privacy has been seen as distinct from physical privacy and while the distinction affords analysis (stahl, timmermans, & mittelstadt, 2016), it obscures the foundation of privacy: social context. as parent put it, “a lonely man isolated on a desert island could hardly be expected to cherish his privacy. so, we serve no useful or constructive purpose in ascribing it to him” (1983, p349). it is social context that gives rise to meaningful privacy, regardless of whether that context is the blockchain or in a grocery store. here, privacy is understood to be an intrinsic and pliant feature of social contexts, shaped by social contexts and to a lesser extent, shaping of social contexts (burmeister, 2016; burmeister & kreps, 2018; teipel et al., 2016; wahlstrom, fairweather, & ashman, 2017) and therefore we define privacy as privacy is a highly contextual social good which is instrumental to the perpetuation of other social goods. people exercise privacy through personal information preferences and practices which are specific to social contexts and changing over time. this view is consistent with those held by others. gavison states “my purpose is to point out the many contexts in which privacy may operate ...” (1980, p444). one of three elements in solove’s work on conceptualising privacy is “a recognition of context and contingency” (2002, p1127) and in a later paper, solove considers the question “how can privacy be addressed in a manner that is non-reductive and contextual, yet simultaneously useful in deciding cases and making sense of the multitude of privacy problems we face?” (2006, p481). finally, nissenbaum (2009) details a framework for revealing the integrity of privacy within specific contexts. according to panichas (2014, p159), “privacy has undeniable implications for persons’ abilities to define themselves and to command the requisite degree of respect for their dignity and autonomy.” the right to be forgotten is a human right established when the european court of justice ruled against data controllers in google v spain1. today, the right to be forgotten exists in europe and argentina. it acknowledges that one’s conduct in one social context ought not to impact on one’s success in other social contexts. for example, youthful drunkenness ought not to damage future access to employment opportunities. the right to be forgotten may require data controllers to remove data from indexes. therefore, the right to be forgotten supports privacy in online contexts. similarly to privacy, socio-technical is has features of social contexts, perhaps not as pliant as privacy, but shaped by social contexts and shaping of social contexts. the bitcoin blockchain is an example of technology that was shaped by (and is now shaping of) social contexts in which economic transactions occur. 1 google spain sl, google inc. v agencia española de protección de datos (aepd), mario costeja gonzález australasian journal of information systems wahlstrom, ulhaq & burmeister 2020, vol 24, selected papers from the 2019 aice conference privacy by design 3 2 limiting privacy through a blockchain blockchain is considered as a data structure that can provide the capabilities of a ledger with three main properties (pass, seeman, & shelat, 2017): 1. consensus: all participants must be agreed on what data should reside on blockchain 2. persistence: once data has been added to the blockchain it can never be removed and 3. liveness: anyone can add data to the blockchain and it is not possible to prevent anyone from writing new data so system can never get stuck. blockchain was originally intended to digitally timestamp documents to make them tamperproof. blockchain is considered a transparent, time-stamped and decentralized system as it provides digital trust by recording information in a public space and is tamper-proof (carlozo, 2017). in a blockchain, a block is a cryptographic hash of the previous block’s identifier, a timestamp, and transaction data (zheng, xie, dai, chen, & wang, 2017). when a new block is added, it encapsulates the previous block and is itself later encapsulated by subsequent blocks (zheng et al., 2017). this ensures all blocks in a blockchain are in accord; blocks may be added to the ends of a blockchain, but once added, blocks are immutable (asharaf & adarsh, 2017). in other words, a blockchain can be added to and read, but it cannot be amended. if it were possible to change a block, the entire chain would fail because the block’s encapsulation would no longer accurately represent the previous and subsequent blocks in the chain. therefore, data cannot be changed within or removed from blockchains. one of the major issues with blockchains is the lack of privacy of transaction data. one of the primitive requirements is to prevent double-spending attacks (joshi & mathew, 2018). to do that, a user is required to reveal some information for authentication. each computer on the network receives a record of every single transaction and update, and each computer validates these transactions. the transactions contain information like sending account, receiving account, amount, and any other details that are required for validation. in some applications, it is not acceptable for all transactions to be revealed to all participants in real time. these problems are motivating privacy as an emerging research topic in the study and development of blockchain technology. let us review privacy concerns in the context of bitcoin financial transactions in between two business parties and review how solutions to such concerns are evolving in with respect to blockchain technology. so, if we look at initial way of bitcoin transactions, we can see the source addresses are written on the blockchain as well as the all the amounts, and information about payer and payee. even though due to cryptology, it is directly difficult to figure out the physical identify associated with these addresses, there are companies that analyse the blockchain and map these addresses to physical entities. it is a problem for business payers, payee and amounts are revealed, making the blockchain less trusted for business payments. for instance, if costs in supply chain are known to everyone, a business cannot gain a competitive advantage. one idea to deal with such privacy concerns is confidential transactions (noether & mackenzie, 2016). the idea is rather than writing amounts in the clear, system can replace amounts with commitments to the amount. a commitment is a way to represent data in a way that no one knows what the data is but the person who wrote the data no longer can modify it. so, they are committed to a particular value. australasian journal of information systems wahlstrom, ulhaq & burmeister 2020, vol 24, selected papers from the 2019 aice conference privacy by design 4 even though no one knows what the value is: like pederson commitment (metere & dong, 2017): commit(v)=gv.hr where, r is random. we put value v as an exponent and we blind it by random r. unfortunately, we lose the opportunity to verify the transaction. normally, when we verify, we check whether the number of inputs is equal to the number of outputs in a transaction plus the fee for the miner. however, a solution named zero knowledge proof (pass, 2019) is used to counter it. it means that payee attaches zero knowledge proof to the transaction. some blockchain systems like zcash (hopwood, bowe, hornby, & wilcox, 2016) and monero (möser et al., 2018) have implemented transactions where not only the amount is hidden but also payee and payer information is hidden. such systems use techniques such as znsnark (bitansky et al., 2017). the right to be forgotten enshrines a legitimate right to request that personal information be erased from the internet, so that it cannot be found by search engines. in contemporary digital history, it is associated to the case of google spain sl, google inc for request by mario costeja gonzález. in the 1990s, gonzález had financial debts that were reported by an online newspaper and later resolved. years later, gonzález requested his past to be forgotten, but the internet would not forget. on may 2014, the european court of justice ruled that google had an obligation to remove links to gonzález’s personal data that was no longer correct (cellanjones, 2014). it led to the european union law referred to as the right to be forgotten. however, it has serious implications for corporate burden and access to information. after this decision, google received 41,000 requests for data to be forgotten and in 2015, requests were reported to be around 1,000 a year (laursen, 2015). in response, google has removed various urls from its search results. the right to be forgotten is a perpetual burden on corporate resources and is considered very hard in implementation terms. it also suggests questions regarding the scope of european law at global scale. finally, it highlights the enduring tension between privacy and utility, as it invites consideration of whether the benefits of privacy outweigh the costs to web indexing organisations. while the outcome of such consideration is a clear ‘yes’ in the deliberations of the european court of justice, the question remains open in other jurisdictions. blockchain enables the decentralised web by relying on a network of computers for distributing data (mougayar, 2016). each computer can act as a node, with power and memory on a distributed storage network system. the data is not stored in any one storage and therefore there is no central point to hack and control. this peer-to-peer infrastructure model of nodes is similar to a blockchain’s distributed ledger and therefore it could be the answer to create a decentralized web. if we reimagine the right to be forgotten, it becomes highly infeasible from the implementation point of view. one reason is the highly decentred nature of the next generation of the internet as no one would have the central authority on data. as a data controller (e.g., a node in a public blockchain) makes personal data public, exercise of the right will also place responsibility upon a node to take reasonable steps, to inform other controllers of any erasure request. to comply with this responsibility, data controllers will have to take all technical measures based on the available technology and the cost of implementation. in similar terms, other world states are contrary to the right to be forgotten like united states and australia where access to the information is considered linked to the free speech and freedom (bennett, 2012). australasian journal of information systems wahlstrom, ulhaq & burmeister 2020, vol 24, selected papers from the 2019 aice conference privacy by design 5 3 holochain privacy there have been various suggestions for supporting privacy (wahlstrom, roddick, sarre, estivill-castro, & devries, 2006), including attempts to leverage blockchain technology to provide privacy (zyskind & nathan, 2015). one such attempt is holochain, which provides “individual authority over data sharing, access, and storage” (holochain, 2020). holochain is a platform for which apps (called happs) are developed (holochain, 2020). happs provide for various distributed and transparent economic exchanges of value and examples include energy trading, farm produce trading, village resource-sharing, agile project management, publishing, heterogeneous iot, social media, and event and disaster management2. in other words, holochain enables distributed, server-less apps running on devices with localized data storage and management. for this reason, holochain is disruptive of client/server platforms and cloud technologies. a happ “… consists of a network of agents maintaining a unique source chain of their transactions, paired with a shared space implemented as a validating, monotonic, sharded, distributed hash table (dht) where every node enforces validation rules on that data in the dht as well as providing provenance of data from the source chains where it originated” (harris-brown, luck, & brock, 2018, p4). in less compact terms, in a happ each node creates its own (small) blockchain for holding verified data which is shared to a dht (frahat, monowar, & buhari, 2019). the dht is sharded and distributed back to the nodes in the happ (holotescu & vasiu, 2020), which validate the shards against their source blockchains (harrisbrown et al., 2018). an inherent limitation of any blockchain is its computation and communication cost, which makes a blockchain less efficient (slower) as it gets bigger. a happ is more scalable than a blockchain because validating shards of dht at nodes is a more efficient validation process than validating an entire blockchain. as a blockchain gets bigger, it gets less efficient and more resilient; but as a happ gets bigger, it gets more efficient and more resilient. nodes in a happ may be controlled by agents, and an agent may be a person. in this way, holochain does indeed support “individual authority of data sharing, access, and storage.” also, holochain transfers power from centralised data controllers such as facebook to individual agents. however, authority over data sharing, access, and storage is not always consistent with privacy. according to the definition of privacy adopted in this paper, privacy may take different forms in different contexts. for example, in one such context, authority over data sharing may be a privacy preference and practice for many people, but not others. in another context, authority over the sharing of sensitive data may not be practicable; for example, when a patient loses consciousness they can no longer exert authority over the sharing of sensitive data related to their health, yet it may be in the patient’s best interest that the data is shared. finally, there are many social contexts in which someone’s privacy preferences and practices dictate that their authority over data sharing be waived (e.g. communications within an intimate relationship that two people share may be controlled by both parties together rather than one or the other). 2 these and other examples are listed at https://forum.holochain.org/c/projects/7 australasian journal of information systems wahlstrom, ulhaq & burmeister 2020, vol 24, selected papers from the 2019 aice conference privacy by design 6 as it is impossible to specify the privacy preferences and practices for every social context, so it is impossible to precisely state that holochain provides privacy. however, it is clear that holochain supports privacy for a subset of social contexts. the contexts in this subset are to be determined by holochain’s users as they develop and use happs. for this reason, we suggest that it will be one of many privacy tools deployed by those with an interest in exercising their privacy preferences and practices. 4 holochain privacy and the right to be forgotten one form of privacy which is likely to be consistent with holochain’s features is the right to be forgotten. this right applies to data indexed by search engines, that is, it applies to the web. if a search engine indexed holochain nodes, then the data returned to web search queries would be under the control of the holochain agents. in cases where the agent is an individual person who wants to exercise the right to be forgotten, they may re-create the small blockchain at their own node making sure to prevent the sharing of the data which is to be forgotten. then, the agent’s small blockchain is re-shared with the holochain’s dht, which is resharded, and re-distributed to the nodes for verification. if such a holochain web search engine existed, its computational and communication costs are likely far less than a blockchain deployed for this purpose. however, these costs would be greater than we observe in today’s web search engines. our point remains that a holochain platform is consistent with the right to be forgotten. 5 conclusion privacy needs to be studied in an ontologically robust way, in relation to blockchain and in relation to any other technology or social context. it is important because it is mutually shaping of freedom, dignity, autonomy, justice, and democracy. it encompasses diverse themes including the control of data and self-determination, restricting access to self and data, privacy and data as commodities that may be traded, privacy as a social good differing from context to context, and the view that privacy takes shape according to the technologies forming the infosphere. blockchains are technologies that were shaped by, and are now shaping of, social contexts in which economic transactions occur. as noted above, holochain is disruptive of client/server platforms and cloud technologies. whether holochain’s data control and computational advantages are sufficient to disrupt data is the new oil business models is a matter of conjecture, but for privacy, holochain happs offer significant advantages over blockchains. privacy and data protection laws around the world represent a real compliance challenge for public and private distributed implementations of blockchain technology; and holochain platforms have the potential to meet these challenges. references asharaf, s. & adarsh, s. (2017). introduction to blockchain technology decentralized computing using blockchain technologies and smart contracts: emerging research and opportunities (pp. 10-27). hershey, pa: igi global. doi:10.4018/978-1-5225-2193-8.ch002 bennett, s. c. (2012). the right to be forgotten: reconciling eu and us perspectives. berkeley j. int'l l., 30, 161. australasian journal of information systems wahlstrom, ulhaq & burmeister 2020, vol 24, selected papers from the 2019 aice conference privacy by design 7 bernoth, m., dietsch, e., burmeister, o. k., & schwartz, m. (2014). information management in aged care: cases of confidentiality and elder abuse. journal of business ethics, 122(3), 453-460. bitansky, n., canetti, r., chiesa, a., goldwasser, s., lin, h., rubinstein, a., & tromer, e. (2017). the hunting of the snark. journal of cryptology, 30(4), 989-1066. burmeister, o. k. (2016). the development of assistive dementia technology that accounts for the values of those affected by its use. ethics and information technology, 18(3), 185-198. burmeister, o. k., islam, m. z., dayhew, m., & crichton, m. (2015). enhancing client welfare through better communication of private mental health data between rural service providers. australasian journal of information systems, 19. https://doi.org/10.3127/ajis.v19i0.1206 burmeister, o. k. & kreps, d. (2018). power influences upon technology design for age-related cognitive decline using the vsd framework. ethics and information technology, 1-4. carlozo, l. (2017). what is blockchain? journal of accountancy, 224(1), 29. cellan-jones, r. (2014). eu court backs ‘right to be forgotten’in google case. bbc news (14 may 2014) online: bbc news europe< http://www.bbc.com/news/world-europe-27388289. dix, a. (1990). information processing, context and privacy. paper presented at interact. floridi, l. (2005). the ontological interpretation of informational privacy. ethics and information technology, 7(4), 185-200. frahat, r. t., monowar, m. m., & buhari, s. m. (2019). secure and scalable trust management model for iot p2p network. paper presented at the 2019 2nd international conference on computer applications & information security (iccais). gavison, r. (1980). privacy and the limits of law. the yale law journal, 89(3), 421-471. https://doi.org/10.2307/795891 hackius, n., reimers, s., & kersten, w. (2019). [the privacy barrier for blockchain in logistics: first lessons from the port of hamburg] logistics management (pp. 45-61): springer. harris-brown, e., luck, n., & brock, a. (2018). holochain: scalable agent-centric distributed computing. github. url: https://github.com/holochain/holochainproto/blob/whitepaper/holochain.pdf. holochain. (2020). holochain: think outside the blocks. retrieved from https://holochain.org holotescu, v. & vasiu, r. (2020). challenges and emerging solutions for public blockchains. brain. broad research in artificial intelligence and neuroscience, 11(1), 58-83. hopwood, d., bowe, s., hornby, t., & wilcox, n. (2016). zcash protocol specification. github: san francisco, ca, usa. hull, g. (2015). successful failure: what foucault can teach us about privacy self-management in a world of facebook and big data. ethics and information technology, 17(2), 89-101. introna, l. (2000). workplace surveillance, privacy and distributive justice. sigcas comput. soc., 30, 33-39. australasian journal of information systems wahlstrom, ulhaq & burmeister 2020, vol 24, selected papers from the 2019 aice conference privacy by design 8 joshi, j. & mathew, r. (2018). a survey on attacks of bitcoin. paper presented at the international conference on computer networks, big data and iot. laursen, l. (2015). google's year of forgetting [news]. ieee spectrum, 52(5), 16-17. metere, r. & dong, c. (2017). automated cryptographic analysis of the pedersen commitment scheme. paper presented at the international conference on mathematical methods, models, and architectures for computer network security. mittelstadt, b., fairweather, b., mcbride, n., & shaw, m. (2013). privacy, risk and personal health monitoring. paper presented at the ethicomp, kolding, denmark. moor, j. (1990). the ethics of privacy protection. library trends, 39(1), 69-82. möser, m., soska, k., heilman, e., lee, k., heffan, h., srivastava, s., hogan, k., hennessey, j., miller, a., & narayanan, a. (2018). an empirical analysis of traceability in the monero blockchain. proceedings on privacy enhancing technologies, 2018(3), 143-163. mougayar, w. (2016). the business blockchain: promise, practice, and application of the next internet technology: john wiley & sons. nissenbaum, h. (2004). privacy as contextual integrity. washington law review, 79(1), 119-157. nissenbaum, h. (2009). privacy in context: technology, policy, and the integrity of social life. stanford, us: stanford law books. noether, s. & mackenzie, a. (2016). ring confidential transactions. ledger, 1, 1-18. panichas, g. (2014). an intrusion theory of privacy. res publica, 20(2), 145-161. parent, w. a. (1983). recent work on the concept of privacy. american philosophical quarterly, 20(4), 341-355. pass, r. (2019). [a tutorial on concurrent zero-knowledge] providing sound foundations for cryptography: on the work of shafi goldwasser and silvio micali (pp. 623-648). pass, r., seeman, l., & shelat, a. (2017). analysis of the blockchain protocol in asynchronous networks. paper presented at the annual international conference on the theory and applications of cryptographic techniques. posner, r. (1977). the right of privacy. ga. l. rev., 12, 393. schwartz, p. (1999). privacy and democracy in cyberspace. vanderbilt law review, 52(6), 16091702. solove, d. (2002). conceptualizing privacy. california law review, 90(4), 1087-1155. solove, d. (2006). a taxonomy of privacy. university of pennsylvania law review, 154(3), 477560. stahl, b., timmermans, j., & mittelstadt, b. (2016). the ethics of computing: a survey of the computing-oriented literature. acm comput. surv., 48(4). https://doi.org/10.1145/2871196 teipel, s., babiloni, c., hoey, j., kaye, j., kirste, t., & burmeister, o. k. (2016). information and communication technology solutions for outdoor navigation in dementia. alzheimer's & dementia, 12(6), 695-707. australasian journal of information systems wahlstrom, ulhaq & burmeister 2020, vol 24, selected papers from the 2019 aice conference privacy by design 9 vovchenko, n., andreeva, a., orobinskiy, a., & filippov, y. (2017). competitive advantages of financial transactions on the basis of the blockchain technology in digital economy. european research studies, 20(3b), 193. wahlstrom, k. & fairweather, n. b. (2013). privacy, the theory of communicative action and technology. paper presented at the ethicomp 2013, kolding, denmark. wahlstrom, k., fairweather, n. b., & ashman, h. (2017). brain-computer interfaces and privacy: method and interim findings. paper presented at the ethicomp 2017, turin, italy. wahlstrom, k., fairweather, n. b., ashman, h., & istance, h. (2013). brain-computer interfaces and privacy: clarifying the risks. paper presented at the proceedings of the seventh australian institute of computer ethics conference. wahlstrom, k., roddick, j. f., sarre, r., estivill-castro, v., & devries, d. (2006). on the ethical and legal implications of data mining. technical report sie-06-001, school of informatics and engineering, flinders university, adelaide, australia. westin, a. (1967). privacy and freedom: atheneum. westin, a. (2003). social and political dimensions of privacy. journal of social issues, 59(2), 431453. zheng, z., xie, s., dai, h., chen, x., & wang, h. (2017). an overview of blockchain technology: architecture, consensus, and future trends. paper presented at the 2017 ieee international congress on big data (bigdata congress). zyskind, g. & nathan, o. (2015). decentralizing privacy: using blockchain to protect personal data. paper presented at the 2015 ieee security and privacy workshops. copyright: © 2020 wahlstrom, ulhaq & burmeister. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v24i0.2801 http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems nguyen, sukunesan & huynh 2022, vol 26, research article web scraping instagram pre and during covid-19 1 web scraping instagram pre and during covid-19: examining customer engagement on australian smes accounts viet hoang nguyen swinburne university of technology, australia 102400039@student.swin.edu.au suku sukunesan rmit university, australia minh huynh la trobe university, australia abstract instagram has gained the attention of hundreds of millions of users and evolved quickly into a critical customer engagement tool for businesses worldwide, more so during covid-19. impacts of covid-19 have fundamentally changed the market, and therefore, this paper explores the relationship between instagram practices and the engagement of 20 australian smes (small medium enterprises) pre and during covid-19. this study aims to answer the following questions: (1) how should user-generated content (ugc) and call to act content (cta) be included as instagram posts? (2) how to use #hashtags and @tagging in instagram posts to keep a campaign going? (3) how instagram can be utilised to mitigate the effect of covid-19? findings revealed a statistically significant relationship between the number of ugcs to instagram engagement, while cta content performance recorded a mixed result. however, both ugcs and cta positively affect the engagement when used to build a virtual community and engage with followers rather than redirecting customers to online selling locations. also, diversity in @tagging and #hashtag uses are found to be effective drivers of engagement. the results imply that addressing the covid-19 related concerns of followers while showing genuine brand social responsibility can be rewarded by extra engagement. keywords: instagram, web scraping, engagement, australian smes. 1 introduction the covid-19 outbreak has dramatically impacted australian smes, and the damages are partly reported in recent government surveys. according to the australian bureau of statistics (abs) (2020), until june 2020, 70% of all surveyed businesses reported a decline in revenue compared to the same period of 2019, while smes were twice more likely to record a deficit in revenue comparing to large businesses. this is consistent with a large number of smes involved in heavily affected industries (e.g., hospitality, retailing, arts, and recreation). many businesses are uncertain about the economy's future, which is a crucial factor forcing them to delay or cancel investment or hiring, leading to a spike in 10% unemployment in 2020. the pandemic damages can also be quantified through the stimulus package allocated to smes by the australian commonwealth government. the commonwealth renamed and doubled the unemployment benefit as jobseeker allowance (previously known as newstart) to supply cash payments to workers (prime minister of australia 2020). in addition, the smes loan australasian journal of information systems nguyen, sukunesan & huynh 2022, vol 26, research article web scraping instagram pre and during covid-19 2 guarantee scheme was established as part of the 66 billion fiscal stimulus package that offers tax credits to support businesses' cash flow (morrison & frydenberg, 2020). while many suffered a tremendous loss, smes from industries with a high digitalisation level have been widely insulated from the crisis since the epidemic has caused severe disruptions in both the upstream and downstream supply chains forcing customers and businesses to embrace digital engagement (seetharaman, 2020). there is a pressing need for smes to align their processes to keep pace with current digital acceleration and leapfrog to be competitive (goldemberg, 2011). due to financial constraints, smes often require quick feedback on product performance before investing further, more so during covid-19. as such, social media presents itself as a viable platform to test product acceptance and technology investment due to its low up-front cost and unlimited marketing potential. more specifically, social media has become an increasingly critical business tool for internationalisation (sukunesan et al., 2020, blanchard 2011, kaplan & haenlein, 2010), engaging with customers, building brand images, and collecting market and customers insight (leung et al., 2013; neiger et al., 2012). however, studies into smes' social media usage are still limited especially systematic analysis of micropractices (bianchi et al., 2017; grochal-brejdak & szymura-tyc, 2018; louart & martin, 2012). also, many smes are social media ignorant and unclear about the benefits due to the lack of exposure, skills, and resources (deloitte, 2019). among social media platforms, instagram has a distinctive uptake with the exceptional ability to leverage the use of visual content and visual storytelling (hu et al., 2014, kaplan & haenlein, 2010). as internet users increasingly prefer images and visual content (neher, 2013), instagram visual content is a viable solution for businesses to "break through the clutter" (miles, 2014) due to its capability to effectively build awareness, promote conversations, traffic, and generating interest in a highly visual offering. with a large number of active users, the platform has already surpassed its counterparts (facebook and twitter) in brand engagement (120 times and 58 times, respectively) as instagram's advertising is reaching 849.3 million users, with every one in three of the most viewed stories are created by businesses (hootsuite, 2019). this has enabled instagram as a value exchange platform, making it an essential component of many businesses' marketing strategies (kietzmann et al., 2011). furthermore, since perspectives on customers' roles shift from non-creation to co-creation (or co-destruction) of value (hanna et al., 2011), instagram practices to effectively communicate with their customers for co-creation of value are vital. according to phua et al. (2017), instagram has a higher level of interaction, co-creation, and engagement than other platforms, making it the most frequently used platform by users to follow and engage with business profiles. the cocreation process from instagram is based on the idea of sharing to prompt action (miles, 2014). for example, engaging content shared by smes may prompt users to react quickly by endorsing the content or the company (e.g., like, comment, follow) or contributing to the community and the company (e.g., sharing their content to the community). additionally, instagram visual content are found to be more suitable for hedonic-related topics, while textual content-based platforms (e.g., twitter, linkedin) may be appropriate for utilitarian posts containing more information (virtanen et al., 2017). instagram is more suitable for products and services advertisements, while twitter and linkedin are often used for spreading political messages or professional networking. to explore this further, it is vital to understudy customer engagement pre and during covid19 on instagram. nevertheless, solely exploring the optimal practices of instagram features australasian journal of information systems nguyen, sukunesan & huynh 2022, vol 26, research article web scraping instagram pre and during covid-19 3 (e.g., @tagging, #hashtag) is not sufficient for smes to improve their instagram performance. therefore, this article systematically analyses instagram features' usage in two quintessential content, namely ugc and cta. this study acquired data from a total of 10 australian smes' instagram profiles in addition to 10 smes that were contextually analysed by sukunesan et al., (2020) to improve rigour and produce a more generalisable result. as many organisations have resorted to using data as a means by which they can build customer value position and monetisation (balakrishnan et al., 2020), data scraping and analytical methods have emerged as critical tools to embrace and transform business models towards information-based and knowledge-based (feldman & sanger, 2007; katsurai & satoh, 2016; arora et al., 2019). this study applies data scrapping methods and draws parallel to haddad and hornuf (2016) survey on tweet sentiment analysis and arora et al. (2019) survey on 1000 indian social media influencers' influence index. in the following sections, the interactive value formation (ivf) framework is shared as the theoretical lens before sme classification, and the business values of instagram are discussed. then, a review on ugc, cta, and covid-19 is also shared, followed by the methodology and results section. finally, the conclusion and limitations of the study are shared by confirming the hypotheses that were presented. 2 literature review 2.1 interactive value creation theoretical background customer participation and contribution (in terms of labour and resources) to creating business value have been one of the prominent research topics in marketing and management literature for the past four decades (mustak et al., 2013). noticeable changes in the marketplace have implied that businesses can and need to collaborate with the increasingly active customers whose participation has become an essential component of brand co-creation processes (bagozzi & dholakia 2006), product customisation/design (dahlander & magnusson, 2008) and innovation (tether & tajar, 2008). over the past 40 years, the ivf theory has contradicted the proper understanding that conceptualises business value as embedded in the products and services and solely created by the providers (non-interactive value creation). the theory implies that value is mutually and simultaneously formed in the collaboration between providers and their customers (vargo & lusch, 2004; prahalad & ramaswamy, 2004; ramirez, 1999). therefore, both customers' and providers' roles are fundamental in the joint value creation activities since customers become active participants in the value creation processes. however, it is worth mentioning that ivf is both a creating and destructive process. echeverri and skålén (2011) argue that value is co-created and codestroyed in the joint value creation activities between businesses and customers. moreover, academic literature tends to overemphasise value co-creation by claiming that co-destruction only occurs in an exchange-based setting (ramirez, 1999) or underestimating the impacts of value co-destruction as the "minor phenomenon" of value co-creation (prahalad & ramaswamy, 2004). still, value co-destruction is considered as a significant feature of businesses and customers' collaboration which allows fluidity in business needs and ideas, among other studies (e.g., plé & rubén, 2010; echeverri & skålén, 2011). makkonen and olkkonen (2017) introduced the third dimension of ivf, where providers and customers are in a neutral position and "resource integration and the respective value-in-context could remain unrealised" (value no-creation) (sthapit et al., 2018). shortly, as noted by sukunesan et al. (2020), ivf dynamically interchanges among three dimensions: 1) value co-creation, 2) value co-destruction, and 3) value no-creation. three ivf dimensions are also seen in a australasian journal of information systems nguyen, sukunesan & huynh 2022, vol 26, research article web scraping instagram pre and during covid-19 4 parallel concept called "produsage" (bruns, 2007), which revolves mainly around social media engagement and ugc. ugc is a diversified type of media content that social media users freely create. harnessing ugcs allow business owners to build online communities with increased engagement alongside their business operations (kaplan & haenlein, 2010; michaelidou et al., 2011), exemplifying ivf. 2.2 smes classification to better explore smes' instagram practices and customise training to improve their performance on the platform, smes classification is the first crucial step suggested in several prior studies (cappel & myerscough, 1996; van vliet & pota, 2001; spiller & lohse, 1997). classification will provide an understanding of practice variations on social media. by categorising smes into sub-groups, smes' characteristics can be thoroughly explored. simultaneously, social media strategies can be designed for a specific business case to better use its resources while mitigating its limitations (spiller & lohse, 1997). therefore, this paper applies van vliet and pota's (2001) framework to classify smes based on their origin. analysing smes' instagram performances under their origin can enrich the data analysis process with previously undiscovered patterns. particularly, van vliet and pota (2001) group online retailers into five categories: 1) internet startups, 2) traditional retailers, 3) catalogue retailers, 4) retailers who both sell through store and catalogue, and 5) manufacturers/wholesalers. since smes presented on instagram often produce and distribute their products and engage with customers, the number of pure catalogues and retailers is limited. therefore, it is suggested that the online retailers with catalogue retail origin, either as their sole selling channel or in combination with store-based retail, have the most advantage and skill to build an effective online retail channel (van vliet & pota 2001). in other words, the skill set and operating model of the internet startup and catalogue selling smes share many similarities bar the selling medium (digital and paper-based). therefore, this study focuses more on the differences in instagram practices/performance among smes, the origins of 1) internet startups, 2) store-based retailers, and 3) manufacturers & wholesalers. more importantly, this study attempts to validate val vliet and pota (2001), where the co-authors expect internet startup smes to have the advantage of exploiting their background to succeed in the online market. therefore, hypothesis 1 (h1) is designed as the following: h1a: smes with an internet startup background have better instagram practice leading to better engagement. h1b: b2b smes will tend to shy away from engagement with individual customers and focus on businesses leading to poorer engagement 2.3 the utilisation of social media among smes the utilisation of social media provides numerous benefits to smes. overall, social network platforms create remarkable opportunities for alternative information distribution and marketing channels (sahay et al., 1998). most importantly, perhaps the reason behind smes' social media utilisation is that promotion on social media is more cost-effective than traditional marketing channels, as social media is the most efficient form of information dissemination and customer interaction (wang et al., 2012). social media promotional campaigns are found to effectively encourage customer engagement and precisely target potential customers leading to revenue growth and long-term value creation by building a robust and loyal australasian journal of information systems nguyen, sukunesan & huynh 2022, vol 26, research article web scraping instagram pre and during covid-19 5 customer community (angel & sexsmith 2011). for example, when viewers like a business post on facebook, they spread promotional material to their social networks with ease that no traditional marketing channel can emulate (lacho & marinello, 2010). many smes also use social media to redirect viewers to company websites, improving website visibility, brand recognition, and search engine rankings (chua et al., 2009). for example, a successful social media presence has been found to increase search ranking on the google search engine of companies' websites (wood, 2009). additionally, smes are also suggested to utilise social media platforms in the information gathering process (chung & buhalis, 2008). social media platforms provide the opportunity for businesses to learn about their consumers more efficiently as social media data hold priceless information about (potential) customers (parrott et al., 2010) (e.g., customers' sentiments toward the brands, the level of engagement of customers towards the company's activities or specific products/services, and customers' feedbacks, rating, and comments on the products/services). however, despite the significant value that smes may gain from social media activities, simply establishing a profile on a popular platform does not assure that smes can attract customers to their profiles or groups (culnan et al., 2010). drawing on facebook and twitter data analysis, paniagua and sapena (2014) suggest that social media engagement only positively influences business value after a "critical mass of followers" is developed. business value is not generated by social media platforms but from the interactions between smes and their followers (milosavljevic & cerf, 2008). fostering a robust and loyal customer community allows businesses to gain business value from social media engagement and enhance business value in many other areas (e.g., brand images, customer services, products, and distribution). in other words, social media presence is not always directly linked to customers' immediate decision making or purchasing behaviour. instead, businesses can only gain benefits from social media by fostering relationships with (potential) customers over time. wang et al. (2012) propose that social media is a unique tool for businesses to actively create dialogue and stay involved with target market customers while encouraging peer communication (communication among followers) on social media platforms poses considerable influences on their attitudes and purchasing behaviours. specifically, these loyal customers may contribute the social media content for smes' profiles, accept the company as an insider, accept new offerings, immunise negative information about the business and even serve as provocateurs for its products (bhattacharya & sen, 2003). the ability of social media to foster interactive dialogues provides a powerful way to engage with customers and build long-term relationships between the customers and the brand (van noort et al., 2012). however, smes have found forming interactive dialogues with customers to be 'problematic' as many smes are 'failing to gain a sufficient insight into the "true" nature of their customers' (parrott et al., 2010) and to create a relevant and effective dialogue with followers in the contemporary era of social networks. compared with traditional marketing campaigns where materials are professionally designed and distributed, social media customers can freely communicate (with the businesses and with other customers) about products and services (mangold & faulds 2009; palka et al., 2009). interactive communications require smes to consider when and how the business can involve in the conversation (kietzmann et al., 2011) and how much they facilitate customer communication (geho et al., 2010) and how they define their perspectives on the brand (bernoff & li, 2008). australasian journal of information systems nguyen, sukunesan & huynh 2022, vol 26, research article web scraping instagram pre and during covid-19 6 2.4 business value from instagram key features according to hootsuite (2019), commercial instagram profiles are visited by more than 200 million active users daily, in which 70% of users search for a specific brand, 80% of the users follow at least one business profile, and 60% search for the desired products/services on instagram before buying. the instagram engagement has become one of the main concerns for businesses on the platform where better presences are expected to acquire more followers allowing promotions, updates, and content to be more easily shared with potential customers (ting et al., 2016). from a synthesis view, leung et al. (2013) and neiger et al. (2012) classify social media value to business into several major impacts, including promotion/brand awareness, product distribution, communication/engagement, management, information dissimilation, market research and brand performance indication apart from innovative business processes. however, social media's business value does not come from the platform itself but from the user community's specific usages enabled by the platform's network logic (mukerji & roy, 2019; culnan et al., 2010). network logics refers to a macro-level belief that shapes the cognition and decision-making process of a digital platform's users. it can be reduced into micro-level day-to-day practices that define users' activities, integrations, and strategies (mukerji & roy, 2019). according to culnan et al. (2010), the business value of social media applications outpaces routine commerce activities (distributing content or driving sales) due to its enormous capability of customer community formation, known as virtual customer environments (vces). instagram provides many vces augmented features for businesses to boost business value through branding, sales, customer service and support, and product development. while forming the customer community is not a new idea, instagram vces augmented features allow the firm to build a loyal customer community much more effectively and efficiently (culnan et al., 2010). for instance, a previous study conducted by virtanen et al. (2017) confirmed that actively following instagram users' accounts and replies to their comments significantly and positively affect instagram engagement and the number of smes' business profile followers. many other organisations, such as lego, ebags, and others (see li & bernoff, 2011), have proved the benefit of solid vces. at the time of this study, 6 instagram features for engagement and vces are identified as 1) instagram is a social network for images and videos sharing, 2) users can follow (or get followed by) other users, including businesses and celebrities, for viewing/commenting their latest posts or communicating through private messages, 3) "discover" feature to suggest the preferred content and other accounts that are more likely to get the engagement from the users, 4) "stories" feature for instant content sharing, 5) "igtv" feature for the long-form video posting, 6) "shopping" features to streamline online selling and payment process via facebook pays (united states market exclusive) (instagram, 2020). 2.5 user-generated content driven by rapid technological developments and changes in users' behaviours, ugc has become a phenomenon in intelligent web services (naab & sehl, 2017). as part of the ugc development, many platform operators have moved to the passive position where media content are created and freely distributed by the users. therefore, the role of platform operators is to enable the content distribution process, customise and further develop shared content (dahlberg, 2011; bruns & schmidt, 2011; o'reilly, 2005). considering the brand-related aspects of social media, ugc is applied by businesses to drive product awareness and to influence customers' purchase decisions (blakley, 2013). literature also refers to social media australasian journal of information systems nguyen, sukunesan & huynh 2022, vol 26, research article web scraping instagram pre and during covid-19 7 ugc as "electronic word-of-mouth" (ewom), where the users' message is more trustworthy than other sources of information with faster and greater reach to the global audiences (hennig-thurau et al., 2004). however, not many studies target the specific uses of instagram ugc for business growth; thus, there has been no widely accepted definition for instagram ugc. based on the earlier review of naab and sehl (2017), we introduce three criteria for instagram ugc as follows: 1) ugc holds a degree of personal contribution. since users cannot directly post content on the page of smes' instagram profiles, smes' instagram profile needs to manually repost the content created by users on the company page. 2) instagram ugc must be publicly shared. content must be accessible to the public. content shared via email (or instant message) with editors are excluded. 3) instagram ugc is created without professional skills and routines. content created with design making and decision making in the hands of users is counted as instagram ugc, while content created by smes or with the collaboration of smes are excluded. in the context of instagram ugc, it is crucial to exclude the content created by the business either for product promotions or customer engagement. since ugc is considered the trustworthy ewom of the internet that is freely communicated among users, content created by businesses (if detected by users) can be considered untrustworthy due to the potential commercial motivation. therefore, social media users often filter out the content that appears to be too commercial intensive while engaging with trustworthy content from sources with expertise (yu et al., 2020; ki and kim, 2019). additionally, social media users are subjected to the information saturation problem where a large amount of available information is distributed by businesses or influencers for their interests, preventing social media users from analysing relevant and applicable information to their needs (pan & fesenmaier, 2006). for example, in a study conducted on social media utilisation for tourism products, lee and gretzel (2014) suggested that when travellers are unfamiliar with destinations, suggestions from social media communities (especially those from well-known travellers) are found to be more influential to their decision-making than those from friends and relatives. in short, the lack of trustworthiness and credibility is the main factor in discarding content created by smes from instagram ugcs, while these factors allow ugc to influence instagram engagement positively. therefore, hypothesis 2 was proposed as follows: h2: higher ugc numbers will establish a stronger vce, prompting a higher number of engagements. 2.6 call to act content cta is a traditional advertising and copywriting technique to pursue customers doing something (white & woods, 1997; applegate, 2005). since the early days of digital marketing, many authors have mentioned cta as a suitable solution for businesses in encouraging customers to learn more about the products and the brands (bendinger et al., 2009) to gain more traffic for an internet location (ghose & dou, 1998; edwards & la ferle, 2000) and to change consumers' behaviours with desirable offers and rewards (white & woods, 1997). by integrating cta into internet marketing campaigns, it is suggested that businesses can maximise the utility of the online location for the brand's competitive positioning within a broader audience. cta can improve several marketing functions, such as building a brand's australasian journal of information systems nguyen, sukunesan & huynh 2022, vol 26, research article web scraping instagram pre and during covid-19 8 image, providing product information, reaching a new market, and developing customer relationships and customer services (parsons & lepkowska-white, 2010). since consumers are not merely passive receivers of advertising messages and are increasingly involved in the value creation process (mustak et al., 2013), ctas have long been used to encourage viewers to participate in the value creation process. in the context of social media, cta has been widely used to gain more digital engagement (scheinbaum, 2016). as technology has posed an enormous potential of transforming previous social configurations, social media platforms also expand conventional cta's capability with new functions such as @tagging (where followers can be directly tagged and referred to in the post). such developments have provided implications for better cta practices in social media marketing campaigns. although cta still contains its traditional marketing functions, social media engagement stimulation (number of likes, comments, and shares) becomes a critical quantitative metric to justify cta effectiveness. according to de vries, (2012), cta is a "medium-level interactive brand post" that can increase the popularity of social media content. the authors suggest that increasing the level of social media content interactivity is an alternative way to enhance the salience of such content. the level of instagram content interactivity characterises two-way communication between smes and their followers and between customers themselves; put differently, it characterises instagram users' dialogue (many-to-many communication). since social media users are becoming increasingly active, many-to-many communication is highly preferred and trusted as users often turn to social media to engage with peers, create, consume and distribute content (schultz & peltier 2013; hanna et al., 2011). as the motivation behind instagram content posting is to gain viewers' reactions, we expect that higher degrees of interactivity will generate more engagement by enabling such communications among users. however, previous studies report inconclusive findings (e.g., insignificant effect (de vries et al., 2012), positive effects (liu & shrum, 2002)) regarding interactivity on advertisement effectiveness and social media engagement gaining. in the context of smes' instagram engagement, we propose h3 as follows: h3: higher number of cta content results in higher engagement when used for vces building than for product advertising. 2.7 the effect of covid-19 on australian smes as we are in the middle of the covid-19 outbreak, it is too soon to quantify the long-term effects of the pandemic on businesses, primarily in the cases of australian smes (cassells & duncan, 2020). at the time of writing, adverse fiscal impacts could be seen across many layers of society despite the australian government's introduction of small business grants (morrison & frydenberg, 2020) and schemes like jobseeker (morrison et al., 2020) and jobkeeper (morrison & frydenberg, 2020). this has dynamic consequences resulting in dramatic changes in how businesses and consumers behave. from the smes' point of view, the outbreak has forced many businesses to close with severe disruption in capital flow and retaining the quality of services in a time of uncertainty, and now the imminent recession is much more complicated than usual (tarki et al., 2020). traditional retailers face numerous short-term challenges that worsen their pre-existing issues. new challenges relating to employee/customers' health and safety, travel restrictions, supply chain and cash flow disruption, and the sudden disruption in consumer demand, sales, and marketing (ivanov, 2020; seetharaman, 2020) have propped up. australasian journal of information systems nguyen, sukunesan & huynh 2022, vol 26, research article web scraping instagram pre and during covid-19 9 many businesses, especially those in traditional retail, tourism, and hospitality require fundamental business model changes with a shift towards online operation (donthu & gustafsson, 2020) as covid-19 is forcing customers to shop digitally due to physical restrictions (seetharaman, 2020). along with physical restrictions, the "scarcity effect" (hamilton et al., 2019) has also dramatically affected price elasticities and stockpiling habits, while traditional deterrents such as queuing and perceived crowding are now widely accepted. many retailers are diffusing messages about the availability of commodities, limiting the number of items per consumer and implementing new online services and delivery. similarly, online retailers introduced the virtual queue and notifying option to limit the number of customers to be handled simultaneously. this can be witnessed among many australian online retailers across several industries, from computer hardware retailers (scroptec) to online grocery stores (woolworths; coles) on items such as central processing unit (cpu), graphic processor unit (gpu), food or toilet paper (scroptec, 2020; woolworths, 2020; coles, 2020). these restrictions and disruptions have led to several consumer behaviour changes: 1) providers switching (product availability/special reasons). 2) brand loyalty enhancement (reward for supporting customers in their hard times) 3) new products/services discovery (pantano et al., 2020). as such, we introduce h4 as following: h4: assuring customers about business availability via instagram with covid-19 related topics during the outbreak encourages customers to remain loyal and engaged. 3 methodology in this section, the methods to collect and process data from 20 australian smes' instagram profiles are clarified, followed by the justification of our chosen statistical methods. our approach contains four main steps: 1) smes classification and input seeds preparation, 2) data scraping, 3) data pre-processing, 4) time series visualisation and poisson regression model (refer to figure 1) figure 1. modified research model australasian journal of information systems nguyen, sukunesan & huynh 2022, vol 26, research article web scraping instagram pre and during covid-19 10 3.1 smes selection and classification the fast-changing nature of social media and lack of accountability permit the creation and abandonment of accounts seamlessly; thus, selecting a representative sample is one of the most challenging aspects of analysing instagram accounts. since this study is interested in smes with van vliet and pota (2001) classification, samples' origin was also considered in the selection process to avoid any bias in the data analysis phase. this research employs data scraping methods to collect data from the 20 australian smes' instagram profiles, 10 of which were previously examined in sukunesan et al. (2020). although sukunesan et al. (2020) analysed five marketing dimensions of smes' instagram utilisation (brand awareness, culture awareness, communication, information, integration), this current study goes deeper into instagram engagement (communication dimension). as sukunesan et al. (2020) have delivered a comprehensive analysis of smes instagram uses with a content analysis method, this study aims to generalise and quantify the previous findings throughout the covid-19 crisis. this study places exceptional attention on the potential of social media data scraping as a means of building market intelligence and deriving business competitiveness. to choose ten additional smes for data analysis, a set of inclusion criteria for samples are provided as follows eventuating to smes listed in table 1: 1) smes are australian based, 2) smes are active on instagram from (15/11/2019 to 15/11/2020), 3) smes are listed in victoria state government business arm (business.vic.gov.au). while the criteria for sme selection were based on the previous of sukunesan et al., (2020), the timeframe for data scrapping is a year. the chosen time frame for data collection entirely covered the ongoing covid-19 epidemic milestones in the context of australian smes, including 1) covid-19 was first reported in wuhan, hubei province, china on 31/9/2019 (zu et al., 2020), 2) the first australian laboratory-confirmed covid-19 patients were diagnosed in melbourne 25/1/2020 (fotheringham et al., 2021), 3) victoria state lockdowns (16/3/2020 to 16/9/2020) (costantino & raina macintyre, 2021). smes brand classification description keepcups https://au.keepcup.com internet startup keepcups was born in 2009 in melbourne by a local coffee shop owner who found the business opportunity in designing and producing their reusable barista standard cup. frank body https://frankbody.com internet startup frank body was born in 2013 by an australian coffee shop owner who used leftover coffee grounds to exfoliate human skins. the business is growing from $5000 capital to a multi-million-dollar brand via aggressive social media and word of mouth marketing. lonely kid club https://lonelykidsclub.com internet startup lonely kid club was born in 2011 as an independent slow fashion brand. the company sells its products via its website and promotes its community via private social media groups pressed juices https://pressedjuices.com.au traditional storebased retailer pressed juices offer cold-pressed juice, smoothies, and flavoured waters with frozen fruit soft-serve australasian journal of information systems nguyen, sukunesan & huynh 2022, vol 26, research article web scraping instagram pre and during covid-19 11 via several stores and food trucks in melbourne. bisonte australia http://bisonte.com.au traditional storebased retailer bisonte australia was born in 1980 as a handcrafted leather jacket and ethos producer and designer. the company sells via a retail store in chapel street melbourne. liar the label https://liarthelabel.com.au/shop manufacture/ wholesalers liar, the label, was born in 2013 as an accessory, swimwear, activewear designer, and producer in gold coast. vegethreads https://vegethreads.com traditional retailer vegethreads offers locally made clothes using eco-friendly materials via their flagship store at 246 high street, northcote. thankyouaus https://thankyou.co internet startup thankyouaus brands itself as a social movement where profit is contributed to tackling poverty in the world. they offer sustainable and ethical skincare products via their website and promote their activities on social media profiles loving earth https://lovingearth.net manufacturers/ wholesalers loving earth is a highquality chocolate/cacao manufacturer that offers healthy and sustainable products hlsk https://hlsk.com.au manufacturers/ wholesalers hlsk was born in 2013 as a handcrafted jewellery designer and producer by an experienced jeweller (hannah stewart). the brand saw tremendous growth from the garagebased workshop to a 300 square meter factory in melbourne and a flagship store in armadale. tulloch wine https://tullochwines.com manufacturers/ wholesalers tulloch wine was found in 1838 by a scottish australian (james tulloch) as viticulture and winemaking. crazy plant people https://crazyplantpeople.com traditional retailer crazy plant people are a new south wales-based company that offers inhouse plants and accessories. the company distributes its product via its store and website and builds a community via social media profiles big winery tour https://bigwinerytours.com.au manufacturers/ wholesalers big winery tours was born in 2016 by beau heme. the company offers winery tours in yarra valley, melbourne. ugg direct https://uggdirect.com.au/ traditional retailer ugg direct is australian footwear and online accessory retailer. the company distributes its product via its website and a flagship store in south wharf, victoria. beanbagrus https://beanbagsrus.com.au manufacturers/ wholesalers beanbagrus is a queensland-based bean bag manufacturer. the https://liarthelabel.com.au/shop australasian journal of information systems nguyen, sukunesan & huynh 2022, vol 26, research article web scraping instagram pre and during covid-19 12 company produces several product lines from cushion beans and distributes its product via its website. solar calculator https://solarcalculator.com.au internet startup solar calculator offers solar power advice. the company connects customers with accredited solar installers and provides consultants for several solar solutions via their website. drinking style vinstrip https://drinkinstyle.com.au/shop manufacturers/ wholesalers drinking style vinstrip is the wholesaler of the italian brand vinstrip. the company offers italianmade wine/bottle carrier and handbags from eco-friendly materials. alfresco blinds co https://alfrescoblindsco.com.au manufacturers/ wholesalers alfresco blinds co was founded in 2008 by chris peroukaneas, and the company is a high-quality supplier and installer of (in)outdoor blinds. mister moose shop https://etsy.com/au/shop/mistermooseshop traditional retailer mister moose shop offers family portraits, australiana prints, and washi tape from a sydney-based store and esty.com – an ecommercial platform. soflossy official https://soflossy.com.au traditional retailer soflossy official offers handmade, gluten-free, vegan-friendly cotton candy via their website. table 1. smes description 3.2 data collection following the outcome of the cambridge analytica debacle, both facebook and instagram rigorously limited access to their users' data through facebook application programming interfaces (api). the restrictions hamper the ability to explore and examine the social phenomenon through the platforms and preserve access for those whose policies may facilitate the outrage itself (bruns, 2019). along these lines, to gather meaningful data for the study, this paper developed a hybrid data scraping method that uses both current instagram apis and an alternative web-scraping program to collect the desired data. the data collection phase incorporates various steps that were designed to collect applicable data instances for the study. the data collection phase began with the preparation of 20 targeted smes' instagram profiles, including profile url and profile names that serve as input seeds for data scraping purposes. twelve months of data from 15/11/2019 to 15/11/2020 was extracted from each site. table 2 presents a brief description of the smes and the total amount of data scraped across different instagram features. the web scraping program was written in python an increasingly popular application for machine learning and a.i. (artificial intelligence). employing a similar approach to arora et al. (2019), the instagram api (application programming interfaces) was called to collect raw data in json format. the data held smes' profile insights, including the number of followers, the type of post, the number of likes and comments for each post and the content (both images, videos, and texts). australasian journal of information systems nguyen, sukunesan & huynh 2022, vol 26, research article web scraping instagram pre and during covid-19 13 no smes’ instagram profile followers image sidecar video com ment @tag ging #hash tag ugc cta 1 keepcup 181,000 300 32 31 3308 309 848 236 23 2 frank body 789,000 549 47 70 20,544 476 132 261 198 3 hlsk 240,000 172 9 11 4,011 9 43 0 27 4 loving_earth 342,000 257 43 24 31,802 352 255 164 49 5 thankyouaus 97,500 84 43 25 4,865 128 297 20 56 6 lonelykidclub69 16,400 575 50 5 3,537 95 3 211 77 7 liarthelable 6,027 79 0 0 31 28 789 0 1 8 pressedjuices 77,300 50 10 16 816 37 34 9 7 9 bisonet_australia 1,027 15 1 1 21 0 53 0 0 10 vegethreads 36,000 515 26 0 1,624 116 225 35 17 11 tullochs wine 3,930 68 4 77 453 21 1,142 0 23 12 big winery tour 721 20 10 6 93 21 141 0 0 13 soflossy official 3,116 159 15 5 1,368 47 389 0 3 14 mister moose shop 1,422 101 2 25 1,054 68 528 0 10 15 uggdirect 2,627 133 71 9 2,650 10 227 0 148 16 beanbagsrus 1,376 70 0 3 67 0 900 0 0 17 crazy plant people 169,000 1,573 0 0 10,789 3,136 2 1,573 1,573 18 alfresco blinds co 6,313 53 100 25 193 30 0 0 16 19 solar calculator 186 48 0 0 70 0 4 0 23 20 drinking style vinstrip 1,771 186 13 10 185 192 5,298 0 31 total 1,976,716 5,007 476 343 87,481 5,075 11,310 2,512 2,282 table 2. description of 20 australian smes and the total amount of data scraped from various instagram features 3.3 data pre-processing json files are parsed in the pre-processing data phase to acquire meaningful variables for analytics and regression modelling processes. the parsed json contains the number of likes and comments of each post, the text/images/video content of each post, the text content of comments, and the date/time of each post. excel (16.0) was used to parse and systematise the json data and the different companies' text format data into spreadsheets. variable description engagement rate the total number of likes yielded from each instagram post of the profile. [type=graph image] instagram post in image format. [type=graph sidecar] instagram post in sidecar format (post with more than one image) [type=graph video] instagram post in video format. comments the number of comments. #hashtag the number of #hashtags. @tagging the number of @taggings. date time date and time of each instagram post. ugc user-generated content instagram post. cta call to act content post. table 3. variables to measure engagement australasian journal of information systems nguyen, sukunesan & huynh 2022, vol 26, research article web scraping instagram pre and during covid-19 14 the date time of each post is used as a unique identifier for the data set. this study adopted several measurements from arora et al., 2019, including the engagement rate and posting rate. other variables such as comment, type of post, date and time, cta and ugc were generated from the collected data (refer to table 3). 3.4 poisson regression since traditional regression techniques are best suited when dependent variables are continuous, they would not fit in the context of instagram engagement analysis, where the counted likes were used as the respondent variable. in contrast, poisson regression, which belongs to a statistical family known as the generalised linear model (glim) (see dobson, 2002; fahrmeir & tutz, 2001; fox, 2008; mccullagh & nelder, 1989), are more suited for more suited to model count data. glim statistical family is designed to model datasets with binary, ordered categorical, count, and time to failure (or success) dependent variables (coxe et al., 2009). since count data is poisson distributed the discrete distribution depicts the occurrence of (individually countable) outcomes that takes on a probability value only for non-negative integers, making it an excellent choice to model count variables such as the count of instagram likes and comments (coxe et al., 2009). as such, poisson regression models can produce more precise results in analysing instagram scrapped metadata, where the dependent variable is the count of likes received in the considered timeframe. the count of like was used as the respondent variable to represent instagram engagement because liking is the lowest form of engagement on instagram. the number of like represents an acknowledgment of instagram users about instagram content, regardless of the meaning behind such actions. typical liking incidents require users to view instagram content and, if applicable, then click the like button. the like action requires only a simple tap from users, which is virtually effortless, instantaneous, and reflexive (swani et al., 2017). more importantly, liking can be considered a donating action that is driven by user-brand relationships rather than self-presentation. in other words, when users like instagram content, they are more likely to advocate for the content creators than self-expression. although the count of comments is also a potential respondent variable representing instagram engagement, comments differ from likes in that they require many steps, including understanding the instagram content (and perhaps previous comments), clicking the comment section, generating the appropriate responses, and then posting the responses. therefore, commenting requires more cognitive resources and effort and contains a higher level of self-expression than liking (labrecque et al., 2020). interestingly, commenting represents a two-way interaction between the commenters and receivers, while many instagram users use comments for exposure and self-promotion. compared to previous studies, we posit that comment heavily drives the user's relationship with a brand and instagram engagement. in other words, the higher number of comments allows instagram to go viral, leading to a higher number of likes. as smes can influence the number of comments (e.g., by tagging, commenting, replying and deleting comments), it is not a representation of instagram engagement but an instagram micro-practice, as suggested by (virtanen et al., 2017). therefore, comments are not a respondent variable but a dependent variable that smes can control or influence to gain instagram engagement. poisson regression presents the incident-rate-ratio (irr) computed as eβ, where β is the poisson regression co-efficient. if the independent variable increases by a unit, the dependent variables are expected to change by β, and other variables remain constant. in other words, in australasian journal of information systems nguyen, sukunesan & huynh 2022, vol 26, research article web scraping instagram pre and during covid-19 15 each model, the companies' posts may belong to several independent categories (e.g., image, video, ugc, cta). having an incidence rate ratio for the dependent variable equal to irr multiplies by a changing unit of the independent variables. the statistical method employed in this paper is comparable to labrecque et al., (2020). in particular, the usage of poisson regression models revealed the differences across social media engagement activities such as likes, comments, and shares for linguistic characteristics (pronoun usage across five pronoun types). however, instead of analysing the linguistic characteristics of instagram posts, we are more interested in investigating specific instagram practices that smes can apply to improve instagram engagement. 4 results 4.1 user-generated content overall, the results indicate that ugc has a positive relationship with the chances of getting more engagement (ranging from 1.02 (β = 0.02, p < 0.001) to 4.21 (β = 1.44, p < 0.001) times). the result also confirms h2: a higher number of ugc will establish a stronger vce, prompting a higher number of engagements (refer to table 4). more specifically, smes with strong vces such as lonelykidclub69, frank body, loving_earth, and crazy plant people have a higher chance of getting 1 unit of engagement from a ugc posted due to their consistent uses of ugc (ranging from 33% to 100% of total posted content). by contrast, smes with a lower ugc posting rate (vegethread) record a negative instagram engagement gain (eβ = 0.72 β = -0.23, p < 0.001) from ugcs, while smes that did not use ugcs in their instagram posts (alfresco blinds co, hlsk) witnessed significantly weaker vces leading to a smaller number of followers and engagement in the considered timeframe. the results confirm previous studies in determining ugc as an effective tool that drives social media contents' popularity (chevalier & mayzlin, 2006; tan et al., 2014) or visual content (li & xie 2020). social media users highly prefer ugc to interact with each other since the messages conveyed by ugc are more trustworthy than other sources of information (hennig-thurau et al., 2004). in other words, instagram users, especially those who are researching specific products/services before purchasing, prefer to trust and engage with peer users rather than businesses and other entities on the platform, as information from peer users are more reliable and trustworthy than those from businesses or influencers. as such, as the ewom of the internet, ugc can be deployed by businesses to build product awareness and to positively affect customers' purchase decisions (blakley, 2013; leung et al., 2013; burgess et al., 2018; coxe et al., 2009). the exception to this pattern occurs in the case of thankyouaus – an internet startup sme. while 13% of thankyouaus posts are ugc, the company still shows an inferior performance in ugc categories (every extra ugc lowers the chances of getting more engagement by 58% (β = -0.86, p < 0.001). as the positive relationship between ugcs and vces building has been described in prior studies (sukunesan et al., 2020), we found it more informative for smes to explore practices of the exceptional cases where the lack of ugcs or the misuses of ugcs leads to poor engagement performance and/or inflicting the development of smes' vce. more specifically, we find the ugcs used by 2 smes (thankyouaus and vegethreads) are indistinguishable from other types of posting and entirely about their products to promote and redirect followers to selling sites. vegethreads and thankyouaus's ugcs share similar messages, colour tone, and sentiment to other companies' posts. the lack of transparency, diversity, product-centric posting style and framed ugcs used by the two companies have discouraged customers from engaging with the companies and the instagram community. australasian journal of information systems nguyen, sukunesan & huynh 2022, vol 26, research article web scraping instagram pre and during covid-19 16 unsurprisingly, this finding has been addressed in the previous study of yu and egger (2021), where instagram users often filter out instagram images that appear too commercial. as users prefer to collect genuine information from peers instead of advertisements, the findings hold practical implications for smes that ugc is a more suitable tool for vce building and instagram engagement gaining than advertising or redirecting followers to online selling locations. furthermore, businesses should not target short term sale growth by using ugc solely for advertising and selling because this type of practice may discourage followers' involvement and prevent the vce from forming, ensuring the modest business value gained from instagram initiatives in the long run. moreover, in the specific use of ugc, the content created by the "trustworthy" source with well know "expertise" to the vce can gain more discussion. the findings provide practical implications for businesses in choosing the preferred ugc for instagram reposts and maintaining the consistent ugc posting rate for a higher level of instagram engagement. sme ugc ratio classification β s.e. wald eβ p keepcup 65% internet start-up 0.16 0 1682.09 1.17 0 frank body 40% internet start-up 0.1 0 3809.4 1.11 0 loving_earth 50% manufacturer 0.02 0.01 56.71 1.02 0 thankyouaus 13% internet start-up -0.86 0.01 5316.42 0.42 0 lonelykidclub69 33% internet start-up 0.27 0.01 2631.75 1.32 0 pressedjuices 12% traditional retailer 0.24 0.02 179.8 1.27 0 vegethreads 6% traditional retailer -0.23 0.02 226.78 0.79 0 crazy plant people 100% traditional retailer 1.44 0.08 351.21 4.21 0 table 4. ugc practice among 20 smes 4.2 call to act content initial analysis of the cta variable surprisingly shows a mixed relationship between cta content and engagement gain among 18 smes that used cta in the considered timeframe. specifically, seven companies show adverse effects on engagement from cta uses (from 26% to 10% decrease in chances of engagement). on the other hand, six companies record positive impacts from cta posts (from 7.4% to 421% increase in engagement), while five companies record insignificant results. however, by looking deeper into the cta practices of 18 companies, a posting pattern concerning their origins can be identified. specifically, internet startup smes (e.g., loneltkidclub69) and retailer smes (e.g., uggdirect) are found to use cta often to redirect customers to their online shop location, especially in the event of promotions or exclusive offers (refer to figure 2). this contrasts with manufacturing smes, where cta is employed to ask customers to look at the products presented in the posted image. this style of using cta has resulted in negative or insignificant chances of engagement gaining for most smes in all three origin categories. as discussed above, the selective attention of instagram users often filters out commercial intensive instagram content (yu & egger 2021). since instagram users prefer to collect genuine information from peers instead of advertisements, the findings hold practical implications for smes to use cta as another tool for vce building and instagram engagement rather than to advertise or redirect followers to online selling locations. in other words, instagram cta should not be directly used to gain more traffic for an internet location, as suggested by ghose and dou (1998) and edwards and la ferle (2000). instead, by using cta to engage with viewers and foster strong vce among the brand and its products/services, smes can more effectively gain traffic to online locations when the trust australasian journal of information systems nguyen, sukunesan & huynh 2022, vol 26, research article web scraping instagram pre and during covid-19 17 (with followers) is built, and followers become loyal to the brands. on the other hand, smes will gain little rewards by intensively advertising via cta and continuously redirecting viewers to unfamiliar online destinations. however, we also found a remarkable exception to this pattern occurring in the case of crazy plant people – a retailer smes. the significant 421% increase in engagement gaining of crazy plant people's cta content is created with a uniform approach. all crazy plant people's posts were images taken and submitted by the customers that contain the customers and their purchased products. the company also encourages customers to submit their moment in every post's caption in one predefined template. this unique type of instagram practice has led to exceptional cta performance since crazy plant people's practices have successfully created an inclusive virtual community of like-minded people who love the products and encourage them to show their love for the product with the rewards of potential likes and comments from other community members. figure 2. cta practices of crazy plant people (left) and lonelykidclub69 (right) compared to crazy plant people's, other high-performance companies (lonelykidclub69, keepcup) seldom use cta in their post. also, when used, content is used to redirect customers to their online shop (refer to figure 2). according to the results, this approach of cta usage adversely impacts instagram engagement. for example, lonelykidclub69 performs poorly in the cta category (every cta post of the company yields only a 0.88 chance of getting more engagement) despite its superior performance in other instagram features' uses. in other words, cta tends to be an ineffective tool for product advertising/selling, but it is effective in building vces around the company's product/services. also, smes should consider the rewards for customers when they follow the cta content to ensure posts' effectiveness. the reward should be substantive to set precedence, ensuring repeat behaviour. customers' rewards (for participating in cta) can be in the form of exclusive products/services discounts or monetary prizes for those who closely follow and participate in the company activities (refer to figure 3 a). additionally, for customers who consistently show their love for the products/services, companies can also share their profiles by the @tagging function (with owner consensus) to reward them with engagement from the vce. by directing the engagement from the vce to specific customers, smes can increase the customers' sense of belonging to the community and encourage other members to follow the desired behaviours of such customers (refer to figure 3 b). alternatively, smes can reward customers with desired behaviours by promoting such behaviours in instagram posts. these practices also direct positive engagement to the customers and encourage others to follow of behaviours of rewarded customers (refer to figure 3 c). moreover, smes can even use customer input in product design and development and rewards them with a monetary prize or profit cut for australasian journal of information systems nguyen, sukunesan & huynh 2022, vol 26, research article web scraping instagram pre and during covid-19 18 every sold product (refer to figure 3 d). this practice allows smes to collect new ideas that are likely preferred by customers in products/services development and enhances the capability of a value co-creation process that benefits both the smes and the community. as the financial constraint is smes' weakness, instagram practices that maximise customers' contributed value is significant for smes' viability. (a) (b) (c) (d) figure 3. examples of smes rewards for followers' participation in cta although poisson regression produces many significant results, we still found an interesting pattern in weaker predictors, especially in cta content among manufacturers smes (refer to table 5). the use of cta among manufacturer smes neither increase nor decrease the chance of getting more engagement. this finding reinforces h1a and h1b in consideration of cta content usage. manufacturers (b2b) are hesitant to use cta in their daily posts (in the cases australasian journal of information systems nguyen, sukunesan & huynh 2022, vol 26, research article web scraping instagram pre and during covid-19 19 of hlsk and alfresco blinds co) or lack of needed marketing/selling skills to effectively use cta content to yield consistent results (in the cases of liarthelabel and loving earth). sme classification β s.e. wald eβ p hlsk manufactures 0 0 1.08 0.996 0.3 loving earth manufactures 0 0 0.01 1 0.92 liarthelabel manufacture -0.11 0.26 0.18 0.9 0.67 ugg direct traditional retailer -0.04 0.04 0.99 0.96 0.32 alfresco blinds co manufactures 0.04 0.04 1.13 1.04 0.29 table 5. smes with insignificant results from using cta 4.3 the best and worst-performing smes this section categorised the results from 20 companies into the best and worst-performing groups based on smes' eβ (refer to table 6). smes classification variable wald eβ p be st p er fo rm in g solar calculator internet start-up comments 2.85 1.46 0.09 ugg direct traditional retailer @tagging 96.1 2.12 0 lonelykidclub69 internet start-up #hashtags 568.98 1.93 0 lonelykidclub69 internet start-up image 313.87 2.14 0 alfresco blinds co manufacturers slide 547.49 2.68 0 crazy plant people traditional retailer cta 351.21 4.21 0 crazy plant people traditional retailer ugc 351.21 4.21 0 w or st p er fo rm in g loving earth manufactures comments 31471.47 1 0 crazy plant people traditional retailer @tagging 4362.93 0.72 0 crazy plant people traditional retailer #hashtag 286.86 0.83 0 beanbagrus manufacturer image 19.83 0.63 0 bisonte_australia traditional retailer slide 61.55 1.08 0 soflossy official traditional retailer cta 12.31 0.74 0 thankyouaus internet start-up ugc 5316.42 0.42 0 table 6. the best and worst-performing across all parameter's comparison overall, internet startup smes posted more diverse content on instagram (e.g., ugc, cta, memes), while smes with traditional retail and manufacturing origins mostly post about their products and services. this diversity in posting among internet startup smes led to superior performances where internet startup smes' uses of @tagging and #hashtag show much stronger positive relationships to engagement than others. the remarkable exception to this pattern occurs in the crazy plant people case, where the company applies only 1 type of content for every post. although crazy plant people's unique practice results in a high number of ugc, cta content, the lack of creativity in applying other instagram features is obvious. the company’s performances in @tagging and #hashtag is the poorest among 20 smes where every extra @tagging and #hashtag only yield 0.72 (β = -0.33, p < 0.001) and 0.83 (β = -0.18, p < 0.001) chances of getting 1 engagement unit. nonetheless, crazy plant people's posting style has successfully encouraged customers to show their support for the brands resulting in the best performing in the cta and ugc category (every cta and ugc post by the company gained 4.21 (β = 1.44, p < 0.001) times more chance of getting more engagement. crazy plant people's practice has shown a considerable potential of ugc and cta in building an effective australasian journal of information systems nguyen, sukunesan & huynh 2022, vol 26, research article web scraping instagram pre and during covid-19 20 vce. the results have confirmed h2 and h3, where a higher number of ugc and cta will establish a stronger vce prompting a higher number of engagements. in consideration of posting types, image is the preferred posting type; sidecar recorded fewer uses while an insignificant number of videos were used. the usage of image and sidecar showed a strong positive relationship with engagement, while video uses recorded insignificant results. however, in the beanbagrus case, any extra images gained only 0.63 engagement (β = -0.18, p < 0.001) while none of the sidecar/video posts and @tagging are used in the considered timeframe making 3 out of 7 parameters results insignificant. further, the company uses instagram solely as a digital product showroom where all company post only contains the company's brand name and product description (refer to figure 4). this instagram practice disregards followers' involvement and prevents the customer base/community from forming, resulting in poor instagram performance. this type of practice is also common among manufacturer (b2b) smes and some traditional retailers leading to comparable results and reinforcing h1b: b2b smes tend to shy away from individual customers' engagement and focus more on businesses. figure 4. frank body, lonelykidclub69, alfresco blinds co, beanbagrus's instagram practices (from left to right, top to down) australasian journal of information systems nguyen, sukunesan & huynh 2022, vol 26, research article web scraping instagram pre and during covid-19 21 the results also support h1a and h1b that manufacturers and traditional retailer smes (except for crazy plant people) are believed to use online stores as an alternative cost-saving selling channel for the physical stores/distribution chains. at the same time, internet startup companies focus more on building customer bases via online engagement. internet-based smes often consider their instagram customer base and followers as a community; thus, the posted content is more engaging, relating to their followers' lifestyle/hobbies leading to more followers and higher engagement. internet startups' content encourages their followers to expand the community with more like-minded people; by @tagging feature where followers' friends can be tagged in companies' posts by the followers, in contrast to manufacturers and store-based smes, which tend to focus on their products rather than engage with the customers. in short, store-based and wholesalers/manufacturers smes use instagram as an online store that allows customers to research products/services before purchasing. simultaneously, internet startup smes promote a community where customers and followers find like-minded people (frank body and lonelykidcub69) or join a social movement (thankyouaus). crazy plant people's ugc and cta usage results also shed light on how instagram features and practices may maximise the ivf from their customers/followers. our findings support both hypotheses (h2 and h3) and suggest that ugc and cta practices' consistency is an essential driver of /followers' participation in vce. in particular, while the usage of cta and ugc by crazy plant people has successfully encouraged customers to participate in the company's marketing efforts. lonelykidclub69 has converted many ugc into their products' designs to address followers' needs while rewarding their participation in the value formulation process. the instagram practices of these smes create an inclusive vce where customers/followers get rewarded for their involvement and convert ugcs from participants into value for both the communities and businesses. the superior engagement gained by the company with vce building orientation is unsavoury compared to smes with commercially intensive posts, as pointed out by hanna et al., (2011). consumers expect and even relish their role as active participants in brand-consumer interactions, while commercially intensive and unsolicited marketing is increasingly considered both annoying and intrusive by the customers (schultz & peltier 2013). in other words, the role of ivf is significant in smes' cases with relevant vce building practices, and other smes receive extremely limited to no cocreated value with the followers/customers. in some cases, unsolicited ugc practices result in a co-destruction effect on instagram engagement and the business value. 4.4 covid-19 adaptation and direction to move forward when examining the instagram practices of 20 smes, we found only 5 smes (keepcup, frank body, lonelykidclub69, thankyouaus, and tullochwine) that launched the covid-19 related content during the considered timeframe. unsurprisingly, the smes that quickly react to the current situation tend to have superior instagram practices across seven parameters (frank body, lonelykidclub69, and tullochwine) or focus on building vce through ugc and cta (thankyouaus; lonelykidclub69 and frank body). moreover, among 5 smes, four companies are internet startups, while only one company has a manufacture/wholesaler origin. this phenomenon reinforces h1a and h1b and suggests that internet startup smes tend to react faster to changes and better capture the community's sentiment than smes from the traditional retailer and manufacturer/wholesaler backgrounds. therefore, they are more active on the platform, which results in higher engagement gain. among covid-19 posts by 5 smes, we detected two main messages: 1) messages that address the community's frustration and 2) australasian journal of information systems nguyen, sukunesan & huynh 2022, vol 26, research article web scraping instagram pre and during covid-19 22 messages that show the social responsibility of the business in tackling the epidemic. in particular, smes with strong vce (lonelykidclub69 and frank body) often apply the 1st type of covid-19 message, while tullochwine and keepcup prefer to use the 2nd type of message (refer to figure 5). further, while the 1st message type comes in the form of ugc, the 2nd type was solely designed and posted by the smes. although there is not enough evidence to conclude which approach is more effective than the other, all covid-19-related posts by 5 smes in the period record a spike in engagement rate. figure 5. covid-19 related instagram messages posted by lonelykidclub69, frank body, thankyouaus, and tullochwine (from left to right up to down) the high engagement return of covid-19 related posts has also confirmed the view of he and harris (2020) that the outbreak offers excellent opportunities for businesses to engage with their corporate social responsibility strategies and agenda actively. this also supports h4 that by assuring customers about business availability and addressing their anxiety through instagram covid-19 related posts during the outbreak, smes can gain unprecedented numbers of loyal customers and instagram engagement. since the tragedies of covid-19 do not limit sickness or death, the ubiquity of such threats leads to uncertainty among customers. as people become more suspicious and less susceptible, gestures to show the brand's social responsibility or address customers' concerns can be more meaningful and long-lasting than those implemented in "normal" times (he & harris, 2020). further, social unrest caused by physical lockdown can also lead to a spike in demand for specific products. therefore, businesses must maintain an e-commerce platform so that their products can continue to be sold and delivered directly to customers while assuring the products' availability to meet unexpected demand (sharma et al., 2020). furthermore, since instagram is a widely preferred australasian journal of information systems nguyen, sukunesan & huynh 2022, vol 26, research article web scraping instagram pre and during covid-19 23 online destination for many customers to research the product before purchasing (hootsuite, 2019), actively present on the platform is also critical for smes during the outbreak and the optimisation of the e-commerce business functions. moreover, continuously engaging with customers on both physical and digital outlets will reassure them that they are being cared for and manage the supply chain constraints caused by volatile demand. other measures such as panic buying control by restricting purchase quantity per customer, implementing protection plans for customers and employees, and contributing to public health are also suggested in early studies and can be implemented and augmented via instagram engagement. besides, instagram enables smes to engage with customers and provides management applications and business/market intelligence gathering applications (leung et al., 2013; neiger et al., 2012). as pantano et al. (2020) suggested, to minimise current and future business impacts of the covid-19 epidemic, retailers should identify and execute controllable activities with more understanding of how their stakeholders operate and how their customers behave during the pandemic. therefore, we suggest that smes combine the market survey with cta content and analyse instagram data analytics tools provided by instagram or third parties to re-evaluate the business model and build a contingency plan to manage inevitable supply chain disruption and volatile customers' demand. 5 conclusion and implication this study aimed to explore how ugcs, ctas and instagram micro-practices contribute to instagram engagement via the process of co-creation. by analysing 5826 instagram content posted by 20 australian smes throughout one year, this study also confirms the ability to employ data analysis in extracting business intelligence and examining four proposed hypotheses. this study has also provided a procedure to measure smes' instagram micropractices effectiveness in terms of engagement via data scraping. drawing from the data analysis process, the findings present several practical implications for better instagram practices. specifically, while ugc offers a positive influence in driving instagram engagement among smes, there is a mixed picture in the case of cta. by analysing the best/worst performing smes across seven parameters, cta and ugc were more suitable tools to build vce than redirect them to smes' online stores. at the same time, ugc and cta are significantly more effective in yielding engagement when used in diverse forms (e.g., internet memes, customers' snapshots, message snapshots) with diverse @tagging and #hashtag. the results also imply the importance of content customisation for followers' loci and relevant context of 'tribal' language or culture to gain attraction. these phenomena can be explained with the concept of "trustworthy" and "expertise" in instagram practices. more specifically, as instagram users are suffering from saturated and illegitimate information from countless businesses and influencers, they often filter out commercial intensive instagram content (yu & egger, 2021) or information from untrustworthy sources (ki & kim, 2019). in other words, the uses of ugc and cta can only positively affect instagram engagement, and vce builds with trustworthy messages, especially those from sources with well-known expertise. besides the trustworthiness of instagram ugc and cta, the consistency in posting rate also plays a crucial role in driving instagram engagement. this study also supplies a brief overview of how 20 australian smes use instagram platforms to manage and gain competitive advantages from the covid-19 outbreak. the study confirms van vliet and pota’s (2001) previous findings, where smes with an internet startup australasian journal of information systems nguyen, sukunesan & huynh 2022, vol 26, research article web scraping instagram pre and during covid-19 24 background react more quickly and effectively to the sentiment and trends on the internet than those from manufacturers. more specifically, while all manufacturer smes retain a similar posting style during the outbreak, internet startup smes have successfully addressed the anxiety of their followers and presented their social responsibility. this explains several peaks in engagement rate during the ongoing epidemic for internet startup ugc and cta, while the engagement rate for manufacturer smes’ content remains insignificant engagement. the consistency of the posting rate during the covid-19 epidemic is also a vital factor noted by sharma et al. (2020). by assuring the availability of its products/services via consistent posting rates, smes address the anxiety of their followers and prepare for spikes in demands for specific products. it is also worth noting that if they survive, the australian smes will emerge in a vastly different "normality" compared to the pre-covid-19-era (he & harris, 2020. the business model's fundamental changes are needed to cope with consumer behaviours and accelerate the digitalisation process. 6 limitations and suggestions for future research first, consumers have been increasingly active participants in brand-consumer interactions and business value creation on social media platforms (hanna et al., 2011). future studies should dive deeper into consumer engagement (dessart et al., 2015) by deploying more complex experimental apparatus (e.g., measuring the number of clicking, the amount of traffic to online location via instagram links, post-viewing time, and buying behaviours) and more objectively denoting consumer engagement variables, beyond cross-section data collected from the platform (e.g., conducting interviews, retrospective self-report). since the viability of many smes in today's rapidly transforming business environment heavily depends on successful engagement for meaningful partnerships, the engagement of customers and smes social media profiles is one of the most crucial aspects of business-consumer relationship quality and business value creation (e.g., in term of sale, products/services development) (kaplan & haenlein, 2010; michaelidou et al., 2011). however, the study only focused on one in four marking dimensions (engagement) proposed by sukunesan et al. (2020); this provides opportunities for further studies to examine all four additional marketing dimensions (brand awareness, information, integrations, and culture awareness). second, while many smes present on more than one platform, this study has solely considered smes' instagram performance. since one social media platform can be more suitable for a specific type of product or smes from one industry than others (yu et al., 2021), biases can be introduced into regression models when we justify smes' engagement solely on instagram practices. additionally, the follower base of smes' business profiles is distinctive (e.g., smes offer cosmetic products (frank body), or women's swimwear (liarthelable) may have follower bases dominated by women). to amplify external validity and managerial implications, future study needs to address the role of potentially interesting follower base characteristics to design customised social media strategies that different smes can pursue in consideration of their unique follower bases. as this study only analysed the instagram content of 20 smes, by no mean the results are generalised for the population of australian smes. future studies can address this limitation with a larger sample size with representatives from different industries. thirdly, as the data collected step only relied on instagram metadata, this study has not thoroughly analysed the instagram visual and audio content. since engagement is also heavily influenced by the contexts and properties of instagram content (yu et al., 2021), many important independent variables (e.g., colour distributions, objects, texts, human face australasian journal of information systems nguyen, sukunesan & huynh 2022, vol 26, research article web scraping instagram pre and during covid-19 25 expression) are left unexplored. nonetheless, machine learning solutions can be introduced in future studies to collect uncovered parameters from posting content. fourth, at the time of writing, the covid-19 epidemic is continuing to spread around the globe. this study has not been able to thoroughly quantify the damage caused by the outbreak on australian smes and the reaction and perception of smes during different phases of covid-19 and different covid19 related restrictions (e.g., lockdown, re-opening, face-mask requirements, physical distancing). future studies need to comprehensively quantify and qualify the damages caused by the covid-19 outbreak to smes, changes in consumer behaviours, and the effectiveness of smes' reactions to the pandemic to shed light on instagram's best practices in the time of crisis. references angel, r., & sexsmith, j. (2011). social networking: the corporate value proposition. ivey business journal, 33, 44-60. applegate, e. (2016). strategic copywriting: how to create effective advertising. rowman & littlefield publishers. arora, a., bansal, s., kandpal, c., aswani, r., & dwivedi, y. (2019). measuring social media influencer indexinsights from facebook, twitter and instagram. journal of retailing and consumer services, 49, 86-101. https://doi.org/10.1016/j.jretconser.2019.03.012 atta tarki, p. l., & weiss, j. (2020). the coronavirus crisis doesn’t have to lead to layoffs. havard business review. retrieved january 17, 2020 from https://hbr.org/2020/03/thecoronavirus-crisis-doesnt-have-to-lead-to-layoffs australian bureau of statistics (2020). business indicators, business impacts of covid-19, july 2020. retrieved january 17, 2020, from https://www.abs.gov.au/statistics/economy/business-indicators/business-conditionsand-sentiments/jun-2020 bagozzi, r. p., & dholakia, u. m. (2006). antecedents and purchase consequences of customer participation in small group brand communities. international journal of research in marketing, 23(1), 45-61. https://doi.org/10.1016/j.ijresmar.2006.01.005 bahae, s. (2018). revisiting technology resistance: current insights and future directions. australasian journal of information systems, 22(0). https://doi.org/10.3127/ajis.v22i0.1655 balakrishnan, r., satyasiba, d., & manojit, c. (2020). implementing data strategy: design considerations and reference architecture for data-enabled value creation. australasian journal of information systems, 24(0). https://doi.org/10.3127/ajis.v24i0.2541 barnes, d. o. u. (2001). understanding business: processes. routledge in association with the open university. bernoff, j., & li, c. (2008). harnessing the power of the oh-so-social web. mit sloan management review, 49(3), 36-42. https://www.proquest.com/scholarlyjournals/harnessing-power-oh-so-social-web/docview/224962179/se-2?accountid=14205 https://librarysearch.swinburne.edu.au/openurl/61sut_inst/61sut_inst:swin2?genre=ar ticle&issn=15329194&isbn=&title=mit+sloan+management+review&volume=49&issue =3&date=spring+2008&atitle=harnessing+the+power+of+the+oh-sosocial+web&spage=36 https://doi.org/10.1016/j.jretconser.2019.03.012 https://hbr.org/2020/03/the-coronavirus-crisis-doesnt-have-to-lead-to-layoffs https://hbr.org/2020/03/the-coronavirus-crisis-doesnt-have-to-lead-to-layoffs https://www.abs.gov.au/statistics/economy/business-indicators/business-conditions-and-sentiments/jun-2020 https://www.abs.gov.au/statistics/economy/business-indicators/business-conditions-and-sentiments/jun-2020 https://doi.org/10.1016/j.ijresmar.2006.01.005 https://doi.org/10.3127/ajis.v22i0.1655 https://doi.org/10.3127/ajis.v24i0.2541 https://librarysearch.swinburne.edu.au/openurl/61sut_inst/61sut_inst:swin2?genre=article&issn=15329194&isbn=&title=mit+sloan+management+review&volume=49&issue=3&date=spring+2008&atitle=harnessing+the+power+of+the+oh-so-social+web&spage=36 https://librarysearch.swinburne.edu.au/openurl/61sut_inst/61sut_inst:swin2?genre=article&issn=15329194&isbn=&title=mit+sloan+management+review&volume=49&issue=3&date=spring+2008&atitle=harnessing+the+power+of+the+oh-so-social+web&spage=36 https://librarysearch.swinburne.edu.au/openurl/61sut_inst/61sut_inst:swin2?genre=article&issn=15329194&isbn=&title=mit+sloan+management+review&volume=49&issue=3&date=spring+2008&atitle=harnessing+the+power+of+the+oh-so-social+web&spage=36 https://librarysearch.swinburne.edu.au/openurl/61sut_inst/61sut_inst:swin2?genre=article&issn=15329194&isbn=&title=mit+sloan+management+review&volume=49&issue=3&date=spring+2008&atitle=harnessing+the+power+of+the+oh-so-social+web&spage=36 australasian journal of information systems nguyen, sukunesan & huynh 2022, vol 26, research article web scraping instagram pre and during covid-19 26 bhattacharya, c. b., & sen, s. (2003). consumer–company identification: a framework for understanding consumers’ relationships with companies. journal of marketing, 67(2), 76-88. https://doi.org/10.1509/jmkg.67.2.76.18609 bianchi, c., glavas, c., & mathews, s. (2017). sme international performance in latin america. journal of small business and enterprise development, 24(1), 176-195. https://doi.org/10.1108/jsbed-09-2016-0142 blakley, j. (2013). brands as publishers and how it’s changing marketing. https://www.socialmediatoday.com/content/brands-publishers-and-how-its-changingmarketing blanchard, o. (2011). social media roi: managing and measuring social media efforts in your organization. que. bruns, a. (2007). produsage proceedings of the 6th acm sigchi conference on creativity &amp; cognition, washington, dc, usa. https://doi.org/10.1145/1254960.1254975 bruns, a. (2019). after the ‘apicalypse’: social media platforms and their fight against critical scholarly research. information, communication & society, 22(11), 1544-1566. https://doi.org/10.1080/1369118x.2019.1637447 bruns, a., & schmidt, j.-h. (2011). produsage: a closer look at continuing developments. new review of hypermedia and multimedia, 17(1),3-7. https://doi.org/10.1080/13614568.2011.563626 cappel, j. j., & myerscough, m. a. (1996). world wide web uses for electronic commerce: toward a classification scheme. amcis 1996 proceedings, 334. cassells, r., & duncan, a. (2020). jobkeeper: the efficacy of australia’s first short-time wage subsidy (vol. 23). curtin university of technology. https://doi.org/10.3316/informit.468936006281571 cha, k. j., hwang, t., & gregor, s. (2015). an integrative model of it-enabled organizational transformation. management decision, 53(8), 1755-1770. https://doi.org/10.1108/md-092014-0550 chevalier, j. a., & mayzlin, d. (2006). the effect of word of mouth on sales: online book reviews. journal of marketing research, 43(3), 345-354. https://doi.org/10.1509/jmkr.43.3.345 chua, a. p. h., kenneth, r. d., & craig, m. p. (2009). exploring the types of smes which could use blogs as a marketing tool: a proposed future research agenda. australasian journal of information systems, 16(1). https://doi.org/10.3127/ajis.v16i1.561 chung, j. y., & buhalis, d. (2008). information needs in online social networks. information technology & tourism, 10(4), 267-281. https://doi.org/10.3727/109830508788403123 coles. (2020). online shop. retrieved january 17, 2020, from https://www.coles.com.au/ costantino, v., & raina macintyre, c. (2021). the impact of universal mask use on sarscov-2 in victoria, australia on the epidemic trajectory of covid-19 [original research]. frontiers in public health, 9. https://doi.org/10.3389/fpubh.2021.625499 https://doi.org/10.1509/jmkg.67.2.76.18609 https://doi.org/10.1108/jsbed-09-2016-0142 https://www.socialmediatoday.com/content/brands-publishers-and-how-its-changing-marketing https://www.socialmediatoday.com/content/brands-publishers-and-how-its-changing-marketing https://doi.org/10.1145/1254960.1254975 https://doi.org/10.1080/1369118x.2019.1637447 https://doi.org/10.1080/13614568.2011.563626 https://doi.org/10.3316/informit.468936006281571 https://doi.org/10.1108/md-09-2014-0550 https://doi.org/10.1108/md-09-2014-0550 https://doi.org/10.1509/jmkr.43.3.345 https://doi.org/10.3127/ajis.v16i1.561 https://doi.org/10.3727/109830508788403123 https://www.coles.com.au/ https://doi.org/10.3389/fpubh.2021.625499 australasian journal of information systems nguyen, sukunesan & huynh 2022, vol 26, research article web scraping instagram pre and during covid-19 27 coxe, s., west, s. g., & aiken, l. s. (2009). the analysis of count data: a gentle introduction to poisson regression and its alternatives. journal of personality assessment, 91(2), 121136. https://doi.org/10.1080/00223890802634175 culnan, m. j., mchugh, p. j., & zubillaga, j. i. (2010). how large us companies can use twitter and other social media to gain business value. mis quarterly executive, 9(4). dahlander, l., & magnusson, m. (2008). how do firms make use of open source communities? long range planning, 41(6), 629-649. https://doi.org/10.1016/j.lrp.2008.09.003 dahlberg, l. (2011). re-constructing digital democracy: an outline of four ‘positions’. new media & society, 13(6), 855-872. https://doi.org/10.1177/1461444810389569 de vries, l., gensler, s., & leeflang, p. s. h. (2012). popularity of brand posts on brand fan pages: an investigation of the effects of social media marketing. journal of interactive marketing, 26(2), 83-91. https://doi.org/10.1016/j.intmar.2012.01.003 deloitte. (2019). the performance of small and medium sized businesses in a digital world. retrieved january 17, 2020, from https://www2.deloitte.com/content/dam/deloitte/es/ documents/consultoria/the-performance-of-smbs-in-digital-world.pdf dessart, l., veloutsou, c., & morgan-thomas, a. (2015). consumer engagement in online brand communities: a social media perspective. journal of product & brand management, 24(1), 28-42. https://doi.org/10.1108/jpbm-06-2014-0635 dobson, a. j. (2008). an introduction to generalized linear models. chapman & hall. donthu, n., & gustafsson, a. (2020). effects of covid-19 on business and research. journal of business research, 117, 284-289. https://doi.org/10.1016/j.jbusres.2020.06.008 echeverri, p., & skålén, p. (2011). co-creation and co-destruction: a practice-theory based study of interactive value formation. marketing theory, 11(3), 351-373. https://doi.org/10.1177/1470593111408181 edwards, s., & ferle, c. l. (2000). cross media promotion of the internet in television commercials. journal of current issues & research in advertising, 22(1), 1-12. https://doi.org/10.1080/10641734.2000.10505098 fahrmeir, l., tutz, g., hennevogl, w., & salem, e. (1994). multivariate statistical modelling based on generalized linear models (vol. 425). springer. https://doi.org/10.1007/978-1-4757-3454-6 feldman, r., & sanger, j. (2006). the text mining handbook: advanced approaches in analyzing unstructured data. cambridge university press. https://doi.org/10.1017/cbo9780511546914 fotheringham, p., anderson, t., shaw, m., jewitt, j., storey, h., hutchings, o., cartwright, j., & gupta, l. (2021). control of covid-19 in australia through quarantine: the role of special health accommodation (sha) in new south wales, australia. bmc public health, 21(1), 225. https://doi.org/10.1186/s12889-021-10244-7 fox, j. (2015). applied regression analysis and generalized linear models. sage publications. geho, p. r., smith, s. e. s., & lewis, s. d. (2010). is twitter a viable commercial use platform for small businesses? an empirical study targeting two audiences in the small business community. the entrepreneurial executive, 15, 73. https://doi.org/10.1080/00223890802634175 https://doi.org/10.1177/1461444810389569 https://doi.org/10.1016/j.intmar.2012.01.003 https://www2.deloitte.com/content/dam/deloitte/es/ https://doi.org/10.1108/jpbm-06-2014-0635 https://doi.org/10.1016/j.jbusres.2020.06.008 https://doi.org/10.1177/1470593111408181 https://doi.org/10.1080/10641734.2000.10505098 https://doi.org/10.1007/978-1-4757-3454-6 https://doi.org/10.1186/s12889-021-10244-7 australasian journal of information systems nguyen, sukunesan & huynh 2022, vol 26, research article web scraping instagram pre and during covid-19 28 ghose, s., & dou, w. (1998). interactive functions and their impacts on the appeal of internet presence sites [21_publication in refereed journal]. journal of advertising research, 38(2), 29-43. http://www.scopus.com/inward/record.url?scp=0001782545&partnerid=8yflogxk goldemberg, j. (2011). technological leapfrogging in the developing world. georgetown journal of international affairs, 12(1), 135-141. http://www.jstor.org/stable/43133873 gomez-rodriguez, m., gummadi, k. p., & schölkopf, b. (2014). quantifying information overload in social media and its impact on social contagions proceedings of the eighth international conference on weblogs and social media, icwsm 2014, ann arbor, michigan, usa, june 1-4, 2014., https://publications.cispa.saarland/1124/ gretzel, u., & yoo, k.-h. (2014). premises and promises of social media marketing in tourism. in the routledge handbook of tourism marketing (pp. 513-526). routledge. grochal-brejdak, m., & szymura-tyc, m. (2018). the internationalisation process of an ecommerce entrepreneurial firm: the inward-outward internationalisation and the development of knowledge. entrepreneurial business and economics review, 6(4), 103. https://doi.org/10.15678/eber.2018.060406 haddad, c., & hornuf, l. (2019). the emergence of the global fintech market: economic and technological determinants. small business economics, 53(1), 81-105. https://doi.org/10.1007/s11187-018-9991-x hamilton, r., thompson, d., bone, s., chaplin, l. n., griskevicius, v., goldsmith, k., hill, r., john, d. r., mittal, c., o’guinn, t., piff, p., roux, c., shah, a., & zhu, m. (2019). the effects of scarcity on consumer decision journeys. journal of the academy of marketing science, 47(3), 532-550. https://doi.org/10.1007/s11747-018-0604-7 hanna, r., rohm, a., & crittenden, v. l. (2011). we’re all connected: the power of the social media ecosystem. business horizons, 54(3), 265-273. https://doi.org/10.1016/j.bushor.2011.01.007 he, h., & harris, l. (2020). the impact of covid-19 pandemic on corporate social responsibility and marketing philosophy. journal of business research, 116, 176-182. https://doi.org/10.1016/j.jbusres.2020.05.030 hennig-thurau, t., gwinner, k. p., walsh, g., & gremler, d. d. (2004). electronic word-ofmouth via consumer-opinion platforms: what motivates consumers to articulate themselves on the internet? journal of interactive marketing, 18(1), 38-52. https://doi.org/10.1002/dir.10073 hiram, t., cyril, d. r. e., & siew, l. l. (2016). intention to use instagram by generation cohorts: the perspective of developing markets. global business and management research: an international journal, 8, 43. hootsuite. (2019). the state of digital in q3 2019. retrieved january 17, 2020, from https://hootsuite.com/resources/the-state-of-digital-in-q3-2019 hu, y., manikonda, l., & kambhampati, s. (2014). what we instagram: a first analysis of instagram photo content and user types. eighth international aaai conference on weblogs and social media, http://www.scopus.com/inward/record.url?scp=0001782545&partnerid=8yflogxk http://www.jstor.org/stable/43133873 https://publications.cispa.saarland/1124/ https://doi.org/10.15678/eber.2018.060406 https://doi.org/10.1007/s11187-018-9991-x https://doi.org/10.1007/s11747-018-0604-7 https://doi.org/10.1016/j.jbusres.2020.05.030 https://doi.org/10.1002/dir.10073 https://hootsuite.com/resources/the-state-of-digital-in-q3-2019 australasian journal of information systems nguyen, sukunesan & huynh 2022, vol 26, research article web scraping instagram pre and during covid-19 29 instagram. (2020). introducing instagram shopping. retrieved january 17, 2020, from https://business.instagram.com/shopping?locale=en_gb ivanov, d. (2020). predicting the impacts of epidemic outbreaks on global supply chains: a simulation-based analysis on the coronavirus outbreak (covid-19/sars-cov-2) case. transportation research part e: logistics and transportation review, 136, 101922. https://doi.org/10.1016/j.tre.2020.101922 kaplan, a. m., & haenlein, m. (2010). users of the world, unite! the challenges and opportunities of social media. business horizons, 53(1), 59-68. https://doi.org/10.1016/j.bushor.2009.09.003 katsurai, m., & satoh, s. (2016, 20-25 march 2016). image sentiment analysis using latent correlations among visual, textual, and sentiment views. 2016 ieee international conference on acoustics, speech and signal processing (icassp), ki, c.-w. c., & kim, y.-k. (2019). the mechanism by which social media influencers persuade consumers: the role of consumers’ desire to mimic. psychology & marketing, 36(10), 905922. https://doi.org/10.1002/mar.21244 kietzmann, j. h., hermkens, k., mccarthy, i. p., & silvestre, b. s. (2011). social media? get serious! understanding the functional building blocks of social media. business horizons, 54(3), 241-251. https://doi.org/10.1016/j.bushor.2011.01.005 la ferle, c., edwards, s. m., & lee, w.-n. (2000). teens&#039; use of traditional media and the internet. journal of advertising research, 40(3), 55. https://doi.org/10.2501/jar-40-3-5565 labrecque, l. i., swani, k., & stephen, a. t. (2020). the impact of pronoun choices on consumer engagement actions: exploring top global brands' social media communications. psychology & marketing, 37(6), 796-814. https://doi.org/10.1002/mar.21341 lacho, k. j., & marinello, c. (2010). how small business owners can use social networking to promote their business. the entrepreneurial executive, 15, 127-133. https://www.proquest.com/scholarly-journals/how-small-business-owners-can-usesocial/docview/807619721/se-2?accountid=14205 larson, d., & chang, v. (2016). a review and future direction of agile, business intelligence, analytics and data science. international journal of information management, 36(5), 700-710. https://doi.org/10.1016/j.ijinfomgt.2016.04.013 lee, y. j., & gretzel, u. (2014). cross-cultural differences in social identity formation through travel blogging. journal of travel & tourism marketing, 31(1), 37-54. https://doi.org/10.1080/10548408.2014.861701 leung, d., law, r., van hoof, h., & buhalis, d. (2013). social media in tourism and hospitality: a literature review. journal of travel & tourism marketing, 30(1-2), 3-22. https://doi.org/10.1080/10548408.2013.750919 li, c., & bernoff, j. (2010). groundswell. winning in a world transformed by social technologies. strategic direction, 26(8). https://doi.org/10.1108/sd.2010.05626hae.002 https://business.instagram.com/shopping?locale=en_gb https://doi.org/10.1016/j.tre.2020.101922 https://doi.org/10.1016/j.bushor.2009.09.003 https://doi.org/10.1002/mar.21244 https://doi.org/10.1016/j.bushor.2011.01.005 https://doi.org/10.2501/jar-40-3-55-65 https://doi.org/10.2501/jar-40-3-55-65 https://doi.org/10.1002/mar.21341 https://www.proquest.com/scholarly-journals/how-small-business-owners-can-use-social/docview/807619721/se-2?accountid=14205 https://www.proquest.com/scholarly-journals/how-small-business-owners-can-use-social/docview/807619721/se-2?accountid=14205 https://doi.org/10.1016/j.ijinfomgt.2016.04.013 https://doi.org/10.1080/10548408.2014.861701 https://doi.org/10.1080/10548408.2013.750919 https://doi.org/10.1108/sd.2010.05626hae.002 australasian journal of information systems nguyen, sukunesan & huynh 2022, vol 26, research article web scraping instagram pre and during covid-19 30 li, l., su, f., zhang, w., & mao, j.-y. (2018). digital transformation by sme entrepreneurs: a capability perspective [doi.org/10.1111/isj.12153]. information systems journal, 28(6), 11291157. https://doi.org/10.1111/isj.12153 li, y., & xie, y. (2019). is a picture worth a thousand words? an empirical study of image content and social media engagement. journal of marketing research, 57(1), 1-19. https://doi.org/10.1177/0022243719881113 liu, y., & shrum, l. j. (2002). what is interactivity and is it always such a good thing? implications of definition, person, and situation for the influence of interactivity on advertising effectiveness. journal of advertising, 31(4), 53-64. https://doi.org/10.1080/00913367.2002.10673685 louart, p., & martin, a. (2012). small and medium-sized enterprises and their attitudes towards internationalization and innovation. i. b. research. makkonen, h., & olkkonen, r. (2017). interactive value formation in interorganizational relationships: dynamic interchange between value co-creation, no-creation, and codestruction. marketing theory, 17(4), 517-535. https://doi.org/10.1177/1470593117699661 mangold, w. g., & faulds, d. j. (2009). social media: the new hybrid element of the promotion mix. business horizons, 52(4), 357-365. https://doi.org/https://doi.org/10.1016/j.bushor.2009.03.002 mccullagh, p. n. j. a. (1989). generalized linear models. chapman and hall. michaelidou, n., siamagka, n. t., & christodoulides, g. (2011). usage, barriers and measurement of social media marketing: an exploratory investigation of small and medium b2b brands. industrial marketing management, 40(7), 1153-1159. https://doi.org/10.1016/j.indmarman.2011.09.009 miles, j. (2014). instagram power: build your brand and reach more customers with the power of pictures. mcgraw hill professional. milosavljevic, m., & cerf, m. (2008). first attention then intention. international journal of advertising, 27(3), 381-398. https://doi.org/10.2501/s0265048708080037 morrison, s., & frydenberg, j. (2020). $130 billion jobkeeper payment to keep australians in a job. retrieved january 17, 2020, from https://parlinfo.aph.gov.au/parlinfo/search/display/display.w3p;query=id%3a%22med ia%2fpressrel%2f7269587%22 morrison, s., frydenberg, j., & ruston, a. (2020). media release: jobkeeper payment and income support extended. retrieved january 17, 2020, from https://ministers.treasury.gov.au/ministers/josh-frydenberg-2018/mediareleases/jobkeeper-payment-and-income-support-extended mukerji, m., & roy, p. s. (2019). platform interactions and emergence of an organizational field: case study on ola. australasian journal of information systems, 23(0). https://doi.org/10.3127/ajis.v23i0.2113 mustak, m., jaakkola, e., & halinen, a. (2013). customer participation and value creation: a systematic review and research implications. managing service quality: an international journal, 23(4), 341-359. https://doi.org/10.1108/msq-03-2013-0046 https://doi.org/10.1111/isj.12153 https://doi.org/10.1177/0022243719881113 https://doi.org/10.1080/00913367.2002.10673685 https://doi.org/10.1177/1470593117699661 https://doi.org/10.1016/j.indmarman.2011.09.009 https://doi.org/10.2501/s0265048708080037 https://doi.org/10.3127/ajis.v23i0.2113 https://doi.org/10.1108/msq-03-2013-0046 australasian journal of information systems nguyen, sukunesan & huynh 2022, vol 26, research article web scraping instagram pre and during covid-19 31 naab, t. k., & sehl, a. (2016). studies of user-generated content: a systematic review. journalism, 18(10), 1256-1273. https://doi.org/10.1177/1464884916673557 neher, k. (2013). visual social media marketing harnessing images, instagram, infographics and pinterest to grow your business online. boot camp publ. neiger, b. l., thackeray, r., van wagenen, s. a., hanson, c. l., west, j. h., barnes, m. d., & fagen, m. c. (2012). use of social media in health promotion: purposes, key performance indicators, and evaluation metrics. health promotion practice, 13(2), 159-164. https://doi.org/10.1177/1524839911433467 nummila, m. (2015). successful social media marketing on instagram. case@ minoshoes. haagahelia university of applied sciences]. https://www.theseus.fi/handle/10024/98853. o'reilly, t. (2005, october 1). web 2.0: compact definition? retrieved january 17, 2020, from oecd. (2020). oecd coronavirus (covid-19). sme policy responses. retrieved january 17, 2020 from https://read.oecd-ilibrary.org/view/?ref=119_119680-di6h3qgi4x. palka, w., pousttchi, k., & wiedemann, d. g. (2009). mobile word-of-mouth a grounded theory of mobile viral marketing. journal of information technology, 24(2), 172-185. https://doi.org/10.1057/jit.2008.37 pan, b., & fesenmaier, d. r. (2006). online information search: vacation planning process. annals of tourism research, 33(3), 809-832. https://doi.org/10.1016/j.annals.2006.03.006 paniagua, j., & sapena, j. (2014). business performance and social media: love or hate? business horizons, 57(6), 719-728. https://doi.org/https://doi.org/10.1016/j.bushor.2014.07.005 pantano, e., pizzi, g., scarpi, d., & dennis, c. (2020). competing during a pandemic? retailers’ ups and downs during the covid-19 outbreak. journal of business research, 116, 209-213. https://doi.org/10.1016/j.jbusres.2020.05.036 parrott, g., azam roomi, m., & holliman, d. (2010). an analysis of marketing programmes adopted by regional small and medium-sized enterprises. journal of small business and enterprise development, 17(2), 184-203. https://doi.org/10.1108/14626001011041201 parsons, a. l., & lepkowska-white, e. (2010). web site references in print advertising: an analysis of calls to action. journal of internet commerce, 9(3-4), 151-163. https://doi.org/10.1080/15332861.2010.526487 phua, j., jin, s. v., & kim, j. (2017). gratifications of using facebook, twitter, instagram, or snapchat to follow brands: the moderating effect of social comparison, trust, tie strength, and network homophily on brand identification, brand engagement, brand commitment, and membership intention. telematics and informatics, 34(1), 412-424. https://doi.org/10.1016/j.tele.2016.06.004 plé, l., & chumpitaz cáceres, r. (2010). not always co-creation: introducing interactional codestruction of value in service-dominant logic. journal of services marketing, 24(6), 430437. https://doi.org/10.1108/08876041011072546 porter, m. e. (2001). the value chain and competitive advantage. understanding business processes, 2, 50-66. https://doi.org/10.1177/1464884916673557 https://doi.org/10.1177/1524839911433467 https://www.theseus.fi/handle/10024/98853 https://read.oecd-ilibrary.org/view/?ref=119_119680-di6h3qgi4x https://doi.org/10.1057/jit.2008.37 https://doi.org/10.1016/j.annals.2006.03.006 https://doi.org/10.1016/j.jbusres.2020.05.036 https://doi.org/10.1108/14626001011041201 https://doi.org/10.1080/15332861.2010.526487 https://doi.org/10.1016/j.tele.2016.06.004 https://doi.org/10.1108/08876041011072546 australasian journal of information systems nguyen, sukunesan & huynh 2022, vol 26, research article web scraping instagram pre and during covid-19 32 prahalad, c. k. (2004). the future of competition: co-creating unique value with customers. boston, mass.: harvard business school pub. prime minister of australia (2020). media release. retrieved january 17, 2020 from https://www.pm.gov.au/media/130-billion-jobkeeper-payment-keep-australians-job radhakrishnan, b., satyasiba, d., & manojit, c. (2020). implementing data strategy: design considerations and reference architecture for data-enabled value creation. australasian journal of information systems, 24(0). https://doi.org/10.3127/ajis.v24i0.2541 rainie, l., brenner, j., & purcell, k. (2012). photos and videos as social currency online. pew internet & american life project. retrieved january 17, 2020 from https://www.pewresearch.org/internet/2012/09/13/photos-and-videos-as-socialcurrency-online/ ramírez, r. (1999). value co-production: intellectual origins and implications for practice and research [https://doi.org/10.1002/(sici)1097-0266(199901)20:1<49::aid-smj20>3.0.co;22]. strategic management journal, 20(1), 49-65. https://doi.org/10.1002/(sici)10970266(199901)20:1<49:aid-smj20>3.0.co;2-2 rammer, c., czarnitzki, d., & spielkamp, a. (2009). innovation success of non-r&dperformers: substituting technology by management in smes. small business economics, 33(1), 35-58. https://doi.org/10.1007/s11187-009-9185-7 savlovschi, l. i., & robu, n. r. (2011). the role of smes in modern economy. economia. seria management, 14(1), 277-281. https://econpapers.repec.org/repec:rom:econmn:v:14:y:2011:i:1:p:277-281 scheinbaum, a. c. (2016). digital engagement: opportunities and risks for sponsors. journal of advertising research, 56(4), 341. https://doi.org/10.2501/jar-2016-040 schultz, d. e., & peltier, j. (2013). social media's slippery slope: challenges, opportunities and future research directions. journal of research in interactive marketing, 7(2), 86-99. https://doi.org/10.1108/jrim-12-2012-0054 scorptec. (2020). retrieved january 17, 2020, from https://www.scorptec.com.au/ seetharaman, p. (2020). business models shifts: impact of covid-19. international journal of information management, 54, 102173. https://doi.org/10.1016/j.ijinfomgt.2020.102173 selvarajah, c., le, t. d., & sukunesan, s. (2019). the vietnam project: developing conceptual knowledge on cross-cultural skills for training in sme internationalization. asia pacific business review, 25(3), 338-366. https://doi.org/10.1080/13602381.2019.1598076 sharma, p., leung, t. y., kingshott, r. p. j., davcik, n. s., & cardinali, s. (2020). managing uncertainty during a global pandemic: an international business perspective. journal of business research, 116, 188-192. https://doi.org/10.1016/j.jbusres.2020.05.026 spiller, p., & lohse, g. l. (1997). a classification of internet retail stores. international journal of electronic commerce, 2(2), 29-56. https://doi.org/10.1080/10864415.1997.11518307 stephen, b., scott, b., rafael, p., & carolyn, w. (2018). interventions in entrepreneurial learning: blended learning and ict capacity. australasian journal of information systems, 22(0). https://doi.org/10.3127/ajis.v22i0.1226 https://www.pm.gov.au/media/130-billion-jobkeeper-payment-keep-australians-job https://doi.org/10.3127/ajis.v24i0.2541 https://www.pewresearch.org/internet/2012/09/13/photos-and-videos-as-social-currency-online/ https://www.pewresearch.org/internet/2012/09/13/photos-and-videos-as-social-currency-online/ https://doi.org/10.1002/(sici)1097-0266(199901)20:1 https://doi.org/10.1002/(sici)1097-0266(199901)20:1 https://doi.org/10.1002/(sici)1097-0266(199901)20:1 https://doi.org/10.1007/s11187-009-9185-7 https://doi.org/10.2501/jar-2016-040 https://doi.org/10.1108/jrim-12-2012-0054 https://www.scorptec.com.au/ https://doi.org/10.1016/j.ijinfomgt.2020.102173 https://doi.org/10.1080/13602381.2019.1598076 https://doi.org/10.1016/j.jbusres.2020.05.026 https://doi.org/10.1080/10864415.1997.11518307 https://doi.org/10.3127/ajis.v22i0.1226 australasian journal of information systems nguyen, sukunesan & huynh 2022, vol 26, research article web scraping instagram pre and during covid-19 33 sthapit, e., coudounaris, d. n., & björk, p. (2018). the memorable souvenir-shopping experience: antecedents and outcomes. leisure studies, 37(5), 628-643. https://doi.org/10.1080/02614367.2018.1519031 sukunesan, s., selvarajah, c., & mellstrom, z. (2020). internationalization via instagram: an exploratory study of small and medium enterprises. contemporary management research, 16(2), 77-121. https://doi.org/10.7903/cmr.19811 swani, k., & labrecque, l. i. (2020). like, comment, or share? self-presentation vs. brand relationships as drivers of social media engagement choices. marketing letters, 31(2), 279298. https://doi.org/10.1007/s11002-020-09518-8 tarki, a., levy, p., & weiss, j. (2020). the coronavirus crisis doesn’t have to lead to layoffs. harvard business review digital. retrieved, 4. https://www.harvardbusiness.org/wpcontent/uploads/2020/03/hbr_the_coronavirus_doesnt_have_to_lead_to_layoffs.p df tether, b., & tajar, a. (2008). beyond industry-university links: sourcing knowledge for innovation from consultants, private research organisations and the public science-base. research policy, 37(6-7), 1079-1095. https://econpapers.repec.org/repec:eee:respol:v:37:y:2008:i:6-7:p:1079-1095 ting, h., de run, e. c., & liew, s. l. (2016). intention to use instagram by generation cohorts: the perspective of developing markets. global business & management research, 8(1). van noort, g., voorveld, h. a. m., & van reijmersdal, e. a. (2012). interactivity in brand web sites: cognitive, affective, and behavioral responses explained by consumers' online flow experience. journal of interactive marketing, 26(4), 223-234. https://doi.org/https://doi.org/10.1016/j.intmar.2011.11.002 van vliet, p. j. a., & pota, d. (2001). understanding online retail: a classification of online retailers. journal of computer information systems, 41(2), 23-28. https://doi.org/10.1080/08874417.2002.11646987 vargo, s. l., & lusch, r. f. (2004). evolving to a new dominant logic for marketing. journal of marketing, 68(1), 1-17. https://doi.org/10.1509/jmkg.68.1.1.24036 virtanen, h., björk, p., & sjöström, e. (2017). follow for follow: marketing of a start-up company on instagram. journal of small business and enterprise development, 24(3), 468484. https://doi.org/10.1108/jsbed-12-2016-0202 walden, r., lie, s., pandolfo, b., lee, t., & lockhart, c. (2018). design research units and small to medium enterprises (smes): an approach for advancing technology and competitive strength in australia. the design journal, 21(2), 247-265. https://doi.org/10.1080/14606925.2018.1427913 wang, x., yu, c., & wei, y. (2012). social media peer communication and impacts on purchase intentions: a consumer socialization framework. journal of interactive marketing, 26(4), 198-208. https://doi.org/https://doi.org/10.1016/j.intmar.2011.11.004 weinberg, b. d., & pehlivan, e. (2011). social spending: managing the social media mix. business horizons, 54(3), 275-282. https://econpapers.repec.org/repec:eee:bushor:v:54:y:2011:i:3:p:275-282 https://doi.org/10.1080/02614367.2018.1519031 https://doi.org/10.7903/cmr.19811 https://doi.org/10.1007/s11002-020-09518-8 https://www.harvardbusiness.org/wp-content/uploads/2020/03/hbr_the_coronavirus_doesnt_have_to_lead_to_layoffs.pdf https://www.harvardbusiness.org/wp-content/uploads/2020/03/hbr_the_coronavirus_doesnt_have_to_lead_to_layoffs.pdf https://www.harvardbusiness.org/wp-content/uploads/2020/03/hbr_the_coronavirus_doesnt_have_to_lead_to_layoffs.pdf https://doi.org/https:/doi.org/10.1016/j.intmar.2011.11.002 https://doi.org/10.1080/08874417.2002.11646987 https://doi.org/10.1509/jmkg.68.1.1.24036 https://doi.org/10.1108/jsbed-12-2016-0202 https://doi.org/10.1080/14606925.2018.1427913 https://doi.org/https:/doi.org/10.1016/j.intmar.2011.11.004 australasian journal of information systems nguyen, sukunesan & huynh 2022, vol 26, research article web scraping instagram pre and during covid-19 34 white, s., & woods, j. (1997). do it yourself advertising. ma: adams media corp wood, c. (2009). the power of social media: from bolt-on to the centre of the universe. hospitality review, 11(3), 18-22. woolworths. (2020). woolworths interact website. retrieved january 17, 2020, from https://www.woolworths.com.au/ yu, j., & egger, r. (2021). color and engagement in touristic instagram pictures: a machine learning approach. annals of tourism research, 89, 103204. https://doi.org/10.1016/j.annals.2021.103204 yu, m., li, z., yu, z., he, j., & zhou, j. (2021). communication related health crisis on social media: a case of covid-19 outbreak. current issues in tourism, 24(19), 2699-2705. https://doi.org/10.1080/13683500.2020.1752632 zaman, g., & vilceanu, g. (1999). coexistence of large and small companies. tribune economic tribune, 45. zu, z. y., jiang, m. d., xu, p. p., chen, w., ni, q. a.-o., lu, g. a.-o., & zhang, l. a.-o. (2020). coronavirus disease 2019 (covid-19): a perspective from china. radiological society public health emergency collection 1527-1315 https://doi.org/10.1148/radiol.2020200490 copyright: © 2022 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v26i0.3447 https://www.woolworths.com.au/ https://doi.org/10.1016/j.annals.2021.103204 https://doi.org/10.1080/13683500.2020.1752632 https://doi.org/10.1148/radiol.2020200490 http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 literature review 2.1 interactive value creation theoretical background 2.2 smes classification 2.3 the utilisation of social media among smes 2.4 business value from instagram key features 2.5 user-generated content 2.6 call to act content 2.7 the effect of covid-19 on australian smes 3 methodology 3.1 smes selection and classification 3.2 data collection 3.3 data pre-processing 3.4 poisson regression 4 results 4.1 user-generated content 4.2 call to act content 4.3 the best and worst-performing smes 4.4 covid-19 adaptation and direction to move forward 5 conclusion and implication 6 limitations and suggestions for future research paper title (paper title style) australasian journal of information systems batist & chimenti 2021, vol 25, research article “humanized robots” 1 “humanized robots”: a proposition of categories to understand virtual influencers antonio batista da silva oliveira federal university of rio de janeiro, brazil antonio.batista@coppead.ufrj.br paula chimenti federal university of rio de janeiro, brazil abstract worldwide investments in influencer marketing are growing and could reach us$ 101 billion in 2020. but how can the brands shield themselves from scandals and other limitations of human influencers? the solution for many companies has involved robots. virtual influencers (vi) are virtual robots that can emulate human appearance and behaviour and have become a trend in marketing. this study analyses how non-human influencers affect marketing communication by adopting two methods that are unpublished in the investigation of this phenomenon: a systematic literature review and netnography in conjunction with in-depth interviews with specialists resulting in the study identifying five categories (two of which are unpublished and unexplored in the literature) that can facilitate management decisions and also future studies around vis: anthropomorphism/humanization, attractiveness, authenticity, scalability, and controllability. this study also identified more convergences than divergences between the virtual and the real and between humans and non-humans, generating challenges, opportunities, and guidelines for future research and for assisting management in making decisions concerning digital marketing. keywords: virtual influencers, digital marketing, exploratory study, artificial intelligence, robots, virtualization 1 introduction there was an estimated turnover of more than us$ 101 billion in 2020 in the influencer marketing industry (association of national advertisers, 2018) with investments in marketing using artificial intelligence growing, including the development of virtual influencers (vis). spark capital, for example, led a us$ 125 million investment round in brud, the start-up that created the world's first vi, lil miquela (alexander, 2019). for many, "virtual influencers are the future of advertising, fashion and commerce" (robinson, 2020, p. 3). in fact, influencers who are involved in scandals, harming advertisers and partners, may have their days numbered. there is already a good number of virtual humanoid robots (virtual influencers) in campaigns for global brands such as prada, samsung, porsche, unilever, calvin klein, ikea, and kfc. some vis requested by global advertisers such as miquela (@lilmiquela), shudu (@shudu), and imma (@imma.gram), were designed realistically using computer generated images (cgi) or other resources. there is, however, a lot of fog over the origin and functioning of vis with many people calling them ai influencers, while others are confused by their realism and ask, "is it human or not?". in miquela's case, after months of speculation about her identity, she replied, "i am a robot." the “revelation” took place in 2018. miquela has a realistic image and a rich “life story” as a singer and model. she interacts with followers, creates content, makes advertisements, etc., but all of this exists only on the virtual plane. miquela has australasian journal of information systems batist & chimenti 2021, vol 25, research article “humanized robots” 2 more than 7 million followers on different social networks (instagram, tiktok, facebook, and youtube). she stars in campaigns for major brands, music releases, and was even voted one of the 25 most influential people on the internet by time magazine in 2018 alongside personalities such as former president donald trump. although there are vis that are not very realistic, this study will focus on those that challenge the boundaries between the real and the virtual, the human and the non-human. vis are operated largely by humans, but machine learning, self-supervision, language, and other attributes of artificial intelligence have developed rapidly in recent years with the natural language generation (nlg) model growing by 1 billion parameters to 17 billion in just one year (microsoft, 2020). therefore, this study addresses managerial and practical implications around vis such as the reduction in relevance of human influencers, more controllable options for advertisers, greater investments in artificial intelligence, and the construction of humanized virtual robots. from a theoretical point of view, while graphics, narratives, and artificial intelligence are enriching the vi phenomenon in marketing (alexander, 2019), there is little literature on nonhuman influencers in digital marketing with the explanatory and prescriptive capacity of the field being predominantly focused on human influence. many recent studies deal exclusively with the influence between humans (boerman, 2020; silva et al., 2020; torres et al., 2019; jiménez-castillo & sánchez-fernández, 2019; ashman et al., 2018) with some citing nonhuman/virtual influencers as a trend (appel et al., 2020; kádeková & holienčinová, 2018), but only two recent studies (robinson, 2020; moustakas et al., 2020) deal specifically with vis. we add to these studies the original perspective of consumers, answering a call from both robinson (2020) and moustakas et al. (2020). this study aims to bring an initial understanding of the vi phenomenon, contribute to filling the gap in non-human influencers, and to answer the question: how do non-human influencers affect marketing communication? to answer this question, the study adopts two methods that are unpublished in the investigation of this phenomenon: a systematic literature review and netnography in conjunction with in-depth interviews with specialists. with this the study identified five categories (two of which are unpublished and unexplored in the literature) that can facilitate management decisions and also future studies around vis: anthropomorphism/ humanization, attractiveness, authenticity, scalability, and controllability. in addition, the study found that in the eyes of many humans, non-human characters are not part of a narrative or message. to paraphrase mcluhan and fiore (1967), "the non-human is the message". 2 literature review a systematic literature review was performed to understand the state-of-the art of research on non-human influencers. the databases scopus, jstor, ebsco, and google scholar were used to search for the keywords "virtual influencer" (vi) and "cgi influencer" (cgi, an acronym for computer generated images). the keywords “virtual influencer” and “cgi influencer” prevail in commercial (emma, 2018) and academic (appel et al., 2020) materials. a keyword research was carried out prioritizing titles and abstracts of articles from any areas of knowledge. due to the initial scarcity of results (as shown in table 1), the keyword "digital influencer" (di) was added due to the synergy with the main theme while making a comparability between humans australasian journal of information systems batist & chimenti 2021, vol 25, research article “humanized robots” 3 and non-humans for subsidizing possible theories applicable to the phenomenon. initially (step 1), 897 results were identified considering the aforementioned keywords and all areas of knowledge. then (step 2), only peer-reviewed articles were selected, returning 176 results. finally (step 3), only articles written in english were kept, leaving 32 articles, which are summarized in table 1 and analysed in the following sections in line with the objectives of this study. authors theory approach/ method categories identified in each study relationship of the theme or context with human or non-human influencers robinson, 2020 agency theory and personal identity qualitative/ essay attractiveness authenticity controllability non-human moustakas et al., 2020 parasocial interaction qualitative/s emistructured interviews attractiveness authenticity controllability non-human appel et al., 2020 n/a qualitative/ systematic review attractiveness authenticity controllability human* marôpo et al., 2020 cultural intermediation qualitative/ content analysis attractiveness authenticity human boerman, 2020 parasocial interaction quantitative /hypothesis testing based on online experiment attractiveness authenticity human casaló et al., 2020 n/a quantitative / likert scale for theory test attractiveness authenticity human silva et al., 2020 consumer behaviour qualitative/ semiotic images and critical incident technique attractiveness authenticity controllability human guerreiro et al., 2019 n/a quantitative /t test, simple frequency, etc. authenticity controllability human gillpatrick et al., 2019 consumer value chain (cvc) qualitative/ essay n/a human** gillpatrick, 2019 consumer value chain (cvc) qualitative/ essay attractiveness controllability human** australasian journal of information systems batist & chimenti 2021, vol 25, research article “humanized robots” 4 hutchinson, 2019 cultural intermediation qualitative/ ethnograph y attractiveness authenticity controllability human** miranda et al., 2019 perceived usefulness, perceptual homophily, tra, and theory of planned behaviour quantitative / structural equation modelling attractiveness authenticity controllability human jiménezcastillo & sánchez fernández, 2019 media dependency theory quantitative / semantic evaluation / opinion using likert scale attractiveness authenticity controllability human verdoodt & feci, 2019 social comparison theory qualitative/ content analysis authenticity controllability human dobreva et al., 2019 theory of cognitive dissonance mixed/conte nt analysis and network analysis authenticity controllability human** moura, 2019 n/a quantitative / survy with likert scale controllability human kuksov & liao, 2019 game theory quantitative / experiment with consumers followed by bayesian analysis authenticity controllability human torres et al., 2019 meaning transfer quantitative / structural equation model attractiveness authenticity controllability human kádeková & holienčinov á, 2018 consumer behaviour quantitative / statistical processing and hypothesis testing attractiveness authenticity controllability human* carvalho, 2018 consumer behaviour quantitative / forms, simple frequency, etc. attractiveness authenticity controllability human australasian journal of information systems batist & chimenti 2021, vol 25, research article “humanized robots” 5 drenten et al., 2018 n/a qualitative/ content analysis attractiveness authenticity controllability human goanta, 2018 n/a qualitative/ case studies attractiveness authenticity controllability human jorge et al., 2018 media literacy qualitative/ content analysis attractiveness authenticity controllability human aranramspott et al., 2018 cultural studies / theory of uses and gratifications mixed/anal ysis of audience via survey and focus groups attractiveness authenticity controllability human ashman et al., 2018 technologies of the self qualitative/ netnograph y attractiveness authenticity controllability human graziano & perez, 2018 network society and stakeholder qualitative/ semistructured in-depth interviews and content analysis controllability human gross & von wangenhei m, 2018 social presence qualitative/i nterviews e content analysis attractiveness authenticity controllability human maden, 2018 diffusion of innovations quantitative / quantitative content analysis, simple frequency, etc. attractiveness authenticity controllability human cotter, 2018 social field theory qualitative/ content analysis authenticity controllability human** clifford & verdoodt, 2017 n/a qualitative/ content analysis attractiveness authenticity controllability human vinerean, 2017 n/a qualitative/ slr attractiveness authenticity human edgerton et al., 2016 media richness mixed/cont ent analysis and authenticity controllability human australasian journal of information systems batist & chimenti 2021, vol 25, research article “humanized robots” 6 quantitative indexing archer & harrigan, 2016 n/a qualitative/i nterviews and ethnography controllability human golder & macy, 2014 n/a qualitative/ literature review authenticity controllability human table 1. overview of the systematic literature review and the presence of vis in the literature on influencers marketing. * mention vis, but they are not the main topic of the study. ** mention non-human elements such as algorithms and artificial intelligence, but not vis. the literature review carried out in this study and presented in table 1 identified that regarding influence marketing, 30 of the 34 studies deal only with human influencers and 2 focus humans but mention vis in some way. only two studies out of the 34 have non-human influencers as their main theme. despite the strong predominance of studies on human influencers, there appears to be a recent growth trend in research on non-human influencers (notably vis) with one paper citing the phenomenon of vis in 2018 (kádeková & holienčinová, 2018) and three papers citing the theme in 2020 (appel et al., 2020; robinson, 2020; moustakas et al., 2020), two of which (robinson, 2020; moustakas et al., 2020) dedicated entirely to vis. 2.1 concept and types of influencers “influencer marketing” is seen as a new frontier of opportunities due to be "one of the fastest growing tools in terms of getting new customers online" (kádeková & holienčinová, 2018, p. 92). despite this, there is still a strong need for studies on digital influencers and the consequences of their influence and opinion leadership (casaló et al., 2020; jiménez-castillo & sánchez-fernández, 2019). there are also interesting studies involving cultural intermediation (marôpo et al., 2020; hutchinson, 2019), health communication (edgerton et al., 2016), “soft facts” (dobreva et al., 2019), e-commerce (moura, 2019), opinion leadership (kuksov & liao, 2019), sexualization/attention economics (drenten et al., 2018), competition dynamics/neoliberalism (ashman et al., 2018), ewom (vinerean, 2017), motivation (archer & harrigan, 2016), and social/privacy research (golder & macy, 2014). kádeková and holienčinová (2018) define an influencer as an individual with a significant number of followers on social networks and who is paid by brands to promote products to those followers. boerman (2020) classifies influencers by the number of followers, but there are other classifications (gross & von wangenheim, 2018), although the number of followers appears to be dominant. in any case, an ever-present element is authenticity. it is linked to honesty, reliability, credibility, and transparency (miranda et al., 2019; guerreiro et al., 2019; maden, 2018; jorge et al., 2018; carvalho, 2018). as for vis, appel et al. (2020, p. 89) mention that "there are cases where consumers know they are interacting with bots and seem not to care" and recommend future research on whether there would be any difference between virtual and real influencers in their effect on consumers. robinson (2020, p. 1) states that "there is nothing morally significant that distinguishes them from natural 'real life' influencers", and the debate is just beginning. australasian journal of information systems batist & chimenti 2021, vol 25, research article “humanized robots” 7 attractiveness is another factor linked to influencers (torres et al., 2019). moustakas et al. (2020) prescribes the construction of a strong and engaging personality to humanize vis. this would include "conflicts, aspirations, and challenges" and would help to expand "affective bonds" with followers. therefore, a greater attractiveness of the robot would be to look even less...like a robot. appel et al. (2020, p. 90) ask: “what is the underlying appeal of virtual influencers?”. the question remains unanswered, while kádeková and holienčinová (2018, p. 97) question whether it would be the case to replace human influencers with virtual “realists and intrigues”. but they emphasize that the commercial and legal aspects are some of the factors at stake. non-human influencers are not new, but they have been changing and growing. the literature focuses on algorithms (hutchinson, 2019; cotter, 2018) or on artificial and robotic intelligence and on recommendation systems (gillpatrick, 2019; gillpatrick et al., 2019; graziano & perez, 2018). in fact, most studies on digital influence and/or digital marketing focus on relationships between humans and disregard a unit of analysis: non-human influencers. the studies in this review involve fields such as information technology, marketing, information systems, psychology, law, education, and sociology. although necessary, such studies seem insufficient to understand the new phenomenon of virtual influencers. this study partly considers thoughts from the field of psychology such as the theory of social comparison (miller & dollard, 1941) and the parasocial theory, while pointing out a relevant gap. according to the parasocial theory (horton & wohl, 1956), fans have a unilateral relationship with celebrities that involves equally unilateral and illusory affective dimensions, which seems applicable to the relationships between influencers and followers, but recent studies (bond, 2016) in the context of social networks deal only with relationships between humans. the theory of social comparison (miller & dollard, 1941) argues that people tend to compare themselves to others and to copy certain models with comparison and imitation being important elements of social learning. both theories focus on relationships between humans, but with the growth of non-human influencers, the gap persists in the face of nonhuman actors. the present study advances some aspects of the discussion on vis: it is the first to use a systematic literature review and a netnography to address the theme. it is also a pioneer in the approach of the consumer's vision (via netnography) and establishes that the human avatar (vi) beyond being only equal to the human, as robinson stated (2020), may also be even superior in some respects. and going beyond what moustakas et al. (2020) pointed out, vis seem to be something more than a marketing strategy, so this study organizes and proposes for the first time five categories that can support management decisions and future studies involving vis: anthropomorphism/humanization, scalability, attractiveness, authenticity, and controllability. 3 research method the literature review identified a gap regarding the understanding of non-human influencers in general and particularly on the view and behaviour of the consumer of this type of influencer. in fact, the option here to analyse vi consumers/followers was in response to requests from literature (robinson, 2020; moustakas et al., 2020), which was done through netnography. australasian journal of information systems batist & chimenti 2021, vol 25, research article “humanized robots” 8 as it is a relatively recent phenomenon, this is a qualitative exploratory study (creswell, 2007) combining interviews with specialists and netnography (kozinets, 2002). this study, due to its exploratory and qualitative design, does not intend to produce generalizations nor correlations between variables, but it does aim to provide the foundations for future theoretical generalizations. the practice of interviewing a small number of sources (especially experts through in-depth interviews) is recurrent in works of different themes and fields (see kautz et al., 2020; hasan & linger, 2020; moustakas et al., 2020). eight specialists participated in semi-structured interviewees with some of them being among the main executives in brazil working in relevant organizations in the media, research, and marketing segments. the interviewees are professionals who have participated or are participating in innovative actions involving influencers from different niches such as cosmetics, sports, media, children & youth audiences, research, and games. they operate predominantly in brazil, which is the second country in the world in average daily access time to social networks (globalwebindex, 2019) and the 2nd largest worldwide audience for virtual influencers (hypeauditor, 2019). table 2 presents a summary of the interviewees' profiles. interviewee/ expert age generation gender school training occupation interviewee 1 40-50 years old x male masters entrepreneur in digital marketing and marketing executive nominated for national awards and cofounder of important brands in the digital environment interviewee 2 20-30 years old y female ba degree culture and consumption researcher with emphasis on influencers interviewee 3 20-30 years old y female ba degree commercial project manager on an innovative platform that brings together influencers interviewee 4 20-30 years old y male ba degree digital marketing entrepreneur and partner manager interviewee 5 40-50 years old x male mba social media executive on a tv station interviewee 6 40-50 years old x male doctorate degree entrepreneur in digital marketing and trade association director interviewee 7 20-30 years old y female masters influencer in a channel with over 10 million followers interviewee 8 30-40 years old y male high school entrepreneur and director in digital marketing agency table 2. summary profile of the interviewed agents and experts the interviews were based on a script prepared via a literature review, giving interviewees freedom (semi-structured model) so that they could produce insights and opinions that were not covered in the literature. the audios of the eight interviews were recorded with an average duration of 61.25 minutes (total of 490 minutes). the recordings were transcribed by a specialized professional service that generated 114 pages of text. in addition, a 10-month netnography was carried out (kozinets, 2002; belk, r., fisher, e., & kozinets, 2013; kozinets & nocker, 2018; kozinets, 2019) involving the profiles of the top five australasian journal of information systems batist & chimenti 2021, vol 25, research article “humanized robots” 9 existing vis (hypeauditor, 2019) with a focus on the interactions between these vis and their followers. in addition to the theme itself (vis), the use of netnography in this article is a way of seeking to "break barriers" (macinnis et al., 2020) in research involving consumers. netnography is defined as a “particular kind of online ethnography whose use is commonplace in marketing, consumer research, tourism, and many other kinds of academic business and communication research fields" (kozinets, 2019, p. 69). netnography seeks to "affirm the preeminence of the construction of meaning and its understanding, providing a cultural approach to study the social interaction that transpires through interactive media" (kozinets & nocker, 2018, p. 131). netnography is one of the differentials of this study in relation to the few articles that deal with vis so far and meets the suggestion of robinson (2020) and moustakas et al. (2020) to investigate vi consumers or followers. the vis were selected according to the ranking of the main vis on instagram (hypeauditor, 2019). the top five at the time were miquela, noonoouri, imma, shudu, and bermuda. noonoouri was excluded from the analysis as he is the only non-humanoid and therefore not the focus of this study. all posts and interactions of the four vis were analysed. posts with photos, texts, and/or videos were followed up and analysed daily for ten months (january to october 2020), except for stories, which last only 24 hours. for the purposes of this study, each post was considered as unique data, but some posts may have more than one "page", which may contain different photos or texts on each "page" of the post, for example. therefore, the number of photos and videos tends to be even greater than that presented in this study, if the other "pages" of each post were to be considered separately. table 3 presents a summary of the data analyzed in the interviews and netnography: data source quantity of saved/analysed data posts 497 videos 66 of 497 comments 387,107 in-depth interviews with digital marketing experts (n = 8) 490 audio recording minutes 114 single-spaced 11 pt-font printed pages monitored profiles (n = 4) and total number of followers 3,614,000 followers duration of data collection january 2020 to october 2020 (10 months) table 3: dataset details as for the profiles analysed on netnography, all are open to the public, with no restrictions on access to posts. the profiles chosen are shown in table 4 with the links of each vi and the links of the respective operators: no. @name number of followers (2019 and 2020) country operator 1 lilmiquela 1.7m 2.8m united states brud 3 imma.gram 159k 320k japan aww.tokyo 4 shudu.gram 192k 209k england thediigitals 5 bermudaisbae 180k 285k united states brud table 4. vis studied on netnography source: adapted by the authors based on hypeauditor, 2019 australasian journal of information systems batist & chimenti 2021, vol 25, research article “humanized robots” 10 the transcripts of the interviews as well as the netnography content were subjected to analysis through inductive research and interpretive procedures (miles, huberman, and saldaña, 2014) as a way of identifying patterns. the categories were identified and developed in meetings between the authors, which generated the five categories proposed in this study, as illustrated in the results and discussion section of this study. 4 results and discussion this section is organized according to the five categories that emerged from the literature, interviews, and netnography. three of the categories have been previously identified in the literature and although initially applicable to human-influencers appeared in our field as also relevant for nonhuman influencers, which are attractiveness, authenticity, and controllability. two of the categories emerged from the interviews and netnography and are original propositions of this study, which are scalability and anthropomorphism/humanization. 4.1 attractiveness attractiveness is related to the influencer’s (physical and/or behavioural) appeal to followers, as well as to the commercial dimension, endorsement, and congruence with the brand (torres et al., 2019). some aspects of attractiveness are presented by interviewees and netnography: as much as i am still in the "league" of human influencers, looking at the brand, hiring (…) this is one of the great advantages: these [virtual] influencers do not suffer human and physical limitations. (interviewee 3) attractiveness is inherent to influence, while certainly being necessary to have some real or apparent attraction such as talent, beauty, style, comedy, sensuality, or authority to attract attention. it is also possible that attractiveness occurs due to attributes such as value creation, novelty, exclusivity, among others. in addition, attractiveness can connect with other categories. it seems plausible that strong attractiveness contributes to more intense scalability, for example. attractiveness can be promoted, for example, through beauty, which is a certain artistic sense (moustakas et al., 2020) or some aesthetic appeal, as in the case of the smartphone ad where the vi imma (figure 1) receives several praises for her beauty, as well as praise for the beauty of the advertisement as a whole. criticism and questioning almost do not exist in the ad, reinforcing the idea that authenticity can lose space for attractiveness or other appeals (imma “does not exist” to the point of being able to “use” the smartphone, for example). this is the same case as miquela, a virtual robot (vi) who appears singing in a video and, although receiving criticism, seems to be praised by most followers, who even register the beauty of miquela's “voice”. however, as with human influencers, vis do not live only by praise: some of the interviewees in this study see an "unattainable perfection" in vis and point out risks to the mental health of some followers due to unattainable comparisons. a follower questioned a post by shudu accusing the creators of the profile of investing in an "unreal" beauty standard. even so, most of the reactions seemed complimentary with some followers even defending shudu. in turn, bermuda caused comparisons of several followers who wanted to have the beauty of this vi or who felt less beautiful than bermuda. female followers generally seem to highlight the australasian journal of information systems batist & chimenti 2021, vol 25, research article “humanized robots” 11 beauty of bermuda and some of the male followers point to the vi’s sensuality. the ice cream that bermuda would be "enjoying" almost disappears in the discussion. figure 1. from left to right clockwise (examples of vis): imma (www.instagram.com/p/cfjtqsnjcbl/) in an ad campaign; miquela (www.instagram.com/p/ cbii1rynivo/) singing in a video; shudu (www.instagram.com/p/ceulej1hsnv/) campaigning for samsung and receiving criticism in another post for having an "unreal" beauty standard; bermuda (www.instagram.com/p/b9ulbrahffp/) "eating" ice cream. there are few mentions of algorithms, artificial intelligence, or authenticity in the posts, even though they are virtual robots that are eating an ice cream, for example. however, if the algorithms and the virtual dimension itself can help in the performance of the vi, certainly an advanced graphic and a very engaging narrative facilitate the engagement in general, generating feedback between attractiveness and scalability. likewise, a powerful algorithm for a mediocre narrative does not seem to be a perfect effectiveness, which brings up the possible connection with another category: authenticity. in fact, algorithms determine user behaviour, but they do not do it unilaterally (cotter, 2018). attractiveness can be related and increased with fascination, curiosity, or other factors and can be simply non-linear: faced with the account of a strong human drama by a virtual influencer, i think i have a prejudiced view. perhaps because i have little knowledge of the subject. but i think it would make me weird, not that i would stop following, i think it would even be a curiosity, maybe. (...) maybe i love one of these too...i don't know. (interviewee 7) according to our research, attractiveness is an important aspect to understand digital influencers and it remains important when moving to world vis. australasian journal of information systems batist & chimenti 2021, vol 25, research article “humanized robots” 12 4.2 authenticity authenticity connects with trust, reliability, and transparency. it refers to the influencer's relationship with his followers and the influencer's narrative coherence and nexus between influencer and advertisers (torres et al., 2019). it may be related to feelings on both sides (influencer and follower), although sometimes a model of parasocial relationship seems to prevail or the illusion in the follower that s/he has intimacy with the influencer. in this sense, the respondents' considerations are consistent with the literature (appel et al., 2020; marôpo et al., 2020): i look very authentically at a traditional influencer. (...) it could be that we start to do this with artificial intelligence too, right? (interviewee 1) the more feelings of truth the virtual influencer has, the closer we will be to an ideal model of influencers. (interviewee 4) consistent with the social comparison theory (miller & dollard, 1941), there are those who believe that vis could harm people: “in relation to the robot, (...) it is a perfection that nobody can achieve, so it goes into disrepute. it is a speech that to me sounds empty, is very idealized, not feasible. for some subjects it can be, while for others it is not” (interviewee 6). however, the authenticity and ethics of vis do not seem to be linked to what would be "plausible" in a linear logic. arguments such as "virtual robots don't really exist and therefore cannot be authentic as a human" seem to be challenged in many situations. below are some examples in figure 2. it is no use looking at the phenomenon of vis simply with the logic of previous phenomena. example: “only physical or biological beings can taste an ice cream”. well, magnum/unilever, one of the world's largest ice cream brands, hired imma as its first virtual ambassador. in the post in which imma announces the commercial partnership with magnum (figure 2), most followers, many of them aware that imma is a virtual robot, show joy and few are the questions about the validity of a virtual being becoming an ambassador of an ice cream. there are many demonstrations of affection for imma's “victory”, even if that victory seems to some people to be absurd. examples are becoming more complex: cameron-james wilson, the creator of shudu, is accused of being a white man who exploits a black profile to make a profit. in parallel, many accolades are directed to shudu's beauty by other followers. it is curious to point out that trevor mcfedries, co-founder, and ceo of brud, the start-up that created miquela (before shudu, inclusive) is a black man who maintains vis of different types, including a white vi (bermuda). it is difficult to identify the real level of authenticity of the profiles, if any, in a debate like this. although relevant, the discussion about control, ownership, and authenticity does not usually dominate the profiles of vis and seems to lack greater maturity. and it seems that there are more decisive elements with the power of the narrative being one of them. bermuda laments that she cannot “see friends” because of the social distancing during the covid-19 pandemic, and although many praise the sensuality of the photo or bermuda’s text that has warmer complements, the posture of some followers focuses on the fact that bermuda could not be contaminated by the coronavirus because she is a robot. it is curious to note that miquela, when saying that she is “working hard” to help artists during the same pandemic, does not receive questions. miquela appears painting a wall, which is hardly plausible (why australasian journal of information systems batist & chimenti 2021, vol 25, research article “humanized robots” 13 do they criticize and claim that a virtual being could not be touched by a virus but applaud when a virtual being decides to paint walls?). figure 2. from left to right clockwise: imma (www.instagram.com/p/b_w6ypojjk5/) in a magnum ice cream campaign; shudu (www.instagram.com/ p/b7egv1ehpin/), “the world's first digital supermodel”; bermuda (www.instagram.com/p/bapohydxzx/) upset by "not being able to see friends" during social distancing during the covid-19 pandemic; miquela (www.instagram.com/p/cbbxvdsn8t3/ and www.instagram.com/p/cbrjulxhaxz/) "striving" to help artists during social distancing during the covid-19 pandemic and supporting the "black lives matter" movement a few days after george floyd's death, in addition to asking to report homophobia. australasian journal of information systems batist & chimenti 2021, vol 25, research article “humanized robots” 14 in another post, miquela defends the “black lives matter” campaign and asks them to fight against racism and homophobia. few questions and a huge support network surround the post. miquela's narrative in those posts (more humanistic and less sensual than bermuda during the pandemic seems to eliminate many facts: bermuda was the one who “revealed” that miquela was in fact a vi and part of the “most intimate” circle of miquela as miquela and bermuda belong to the same owner, a masterful creativity start-up called brud. miquela and bermuda are both vis and therefore share the same nature in terms of possibilities and limitations. but the narratives of miquela and bermuda are different. authenticity, therefore, although it has some relevance, does not seem to be the focus. in the post on black lives matter, a follower goes on to say that "if someone has less empathy than a robot, that person has a problem." would miquela be more human than many flesh and blood beings? in a post in which miquela seeks to raise donations to help artists and other audiences, a follower says that “if you were a human, i would say that you have a good heart, but believe me, you are extremely good 100% [sic] and i love you for that”. the power of the narrative, along with other elements, not only reduces the relevance of authenticity, but at times it is the authenticity of humans that is questioned by the vi followers. in addition to all the above, one of the effects that virtual and/or non-human beings can generate is trust. appel et al. (2020, p. 89), point out that therapy robots "because they are explicitly nonhuman, are perceived as less critical and can therefore be easier to be trusted by users." as for vis, another type of robot, one of the interviewees in this study opines: i think that the virtual influencer is a type of influence that starts to have an expectation for authenticity a little higher than the human influencer. (interviewee 2) and there are those who suggest a path opposite to that pointed out by most of the literature: we don't have to arrive trying to educate. we can just put that on and then say, "this profile you've been looking at for a few months is a robot." this even becomes a part of the campaign. (interviewee 8) robinson (2020, pp. 5-6) expands the debate: “it’s likely that different people, and possibly groups of people, generations even, have different intuitions about the authenticity and transparency requirements for online identities, and by extension, how much they are willing to trust fictional identities online.” it is not clear what the real value of authenticity is in the virtual world and what other elements (attractiveness, identification, value generation, etc.) could possibly complement or even replace authenticity in online relationships with humans and nonhumans. in any case, authenticity can also be confused with a certain need for control (the more authentic, the more reliable/controllable), which requires further study. 4.3 controllability controllability is linked to a certain predictability in relation to the influencer, which can be partially sought through contracts, among others, while also involving social media metrics (audience, engagement, conversion, etc.). a simple example is the ipsos ranking of influencers (metro, 2020), which pointed out the comedian whindersson nunes (28 million followers on youtube) as the most well-known youtuber in brazil. in the same survey, the most influential youtuber was nathalia arcuri (“me poupe” channel) at the time with “only” 4 million followers on her financial education channel. literature states that advertising with nonhumans would be less risky (robinson, 2020; moustakas et al., 2020; kádeková & holienčinová, 2018), but it is not clear whether the intelligence of some vis would be more australasian journal of information systems batist & chimenti 2021, vol 25, research article “humanized robots” 15 improved than some ais that faced problems (microsoft, 2016). but, in line with the literature, most interviewees agree that non-human influencers are more controllable: for non-commercial campaigns, i think i would prioritize the virtual because i would not "risk"... if we are talking about something like philanthropy and social causes, it is very important that the people you choose have a connection with this topic, and if i choose a digital (human) influencer, i run the risk of being “betrayed” by their story on the internet. (interviewee 2) i think that the greater use of artificial intelligence brings many opportunities and risk forecasting is the greatest benefit. (interviewee 3) also important is the autonomy of robots and artificial intelligence. there are those who call vis “ai influencers” (emma, 2018), perhaps anticipating a trend towards autonomy, selflearning, and self-supervision in robotics and ai, especially if combined with iot, nlg, or machine learning, but the debate still seems immature. there are those who point to control as a greater facility to “guide” the vi, but attention must be given to another control—the social: i initially thought about non-commercial use, so that there is education in society regarding the use of virtual influencers. then comes commercial use. imagine such a robot working for the alcohol industry. i think the debate will be very high and may end up hindering the entry of several other robots that could help in several other areas to share important content for society. (interviewee 8) controllability also involves regulation. if regulation of advertising with human influencers already has deficiencies (verdoodt & feci, 2019), there are even more uncertainties for nonhumans. a channel influencer with more than 10 million followers interviewed in this study summarizes as follows: i, as an influencer, as we are now, is fine, but there are cases where we miss where to go. i am a huge fan of youtube because other networks do not provide assistance. you have to fend for yourself (…) i think there is a lot of tension between authorities and companies in relation to this and we are in that limbo. i don't know what would be better for me now and for a virtual influencer i don't know either. (interviewee 7) the state of california (usa), for example, opted for a law to deal with robots and advertising: law sb 1001 (section 17941). it prohibits using a bot to "cheat and encourage buying or selling" or to "influence voting in an election" (california, 2018). observing unbelievable scandals involving people of flesh and blood, perhaps robots that "look" like humans could be anything but controllable. 4.4 anthropomorphism/humanization this category emerged from the interviews and netnography. the attribution of human characteristics to other beings (anthropomorphism) has been practiced for millennia since prehistoric rock records to “memes” with drawings on the internet involving art and religion. there are many and growing examples of human-like robots, both physical robots such as sophia (@realsophiarobot), and virtual robots such as colonel sanders/kfc (@kfc), for example. virtual influencers appear crying, brushing their teeth, and eating ice cream. "are you sure you are a robot?" is not an uncommon expression in profiles such as @lilmiquela, australasian journal of information systems batist & chimenti 2021, vol 25, research article “humanized robots” 16 @shudu.gram, @imma.gram, etc. figure 3 show examples that many people do not know whether vis are human or not and that this does not seem to be so decisive for their appreciation for vis. figure 3. from left to right clockwise: shudu (www.instagram.com/p/cb3aqsjhkq5/) / followers with doubts whether it would be "real" and expressions of appreciation; bermuda (www.instagram.com/p/b9xqafdhlb6/) / “a robot or a woman?”; imma (www.instagram.com/p/cbkupzsjnzx/) / “too real for a robot”; miquela (www.instagram.com/p/cfp-peqn-z7/) / “robots cry?” the anthropomorphism of some vis leads many followers to ask whether the vis are human or not, as shown in figure 3. some are in doubt, others criticize, some defend, and there are many who, knowing or not that vis are not human, declare appreciation, affection, support, etc. as for shudu, some say that it is impossible that she is not a human and the same is said about bermuda. imma arouses many reactions of affection and the feeling in many that she is "too real to be a robot". but vis go beyond form or appearance with the vis' behaviour sometimes arousing broader emotions in their followers, such as miquela who has several fan clubs around the world. in fact, the relationship between vis and many followers seems to have no australasian journal of information systems batist & chimenti 2021, vol 25, research article “humanized robots” 17 difference compared to the relationship between human influencers and their flesh and blood followers, which seems to confirm the thinking of robinson (2020) for whom vis have no moral difference compared to human influencers. in addition to the relational and moral dimension, affection also gains significant space in the context. for many followers, it does not seem to make much difference whether vis are human or not. when seeing miquela "cry", for example, a follower summarizes something valuable: “i'm totally confused if you're a robot or not. anyway, you're cool and gorgeous. i love your music! just don't be sad.” there are also vis with lighter anthropomorphism that resemble a cartoon (@noonoouri, for instance). however, the focus of this study is on the most humanized vis. in any case, consistent with the parasocial theory (horton & wohl, 1956), an interviewee indicates concern about possible psychological impacts: the more human the virtual influencer becomes, the more the audience will want to have personal relationships, see the show, get closer, and awaken feelings of all kinds. i think this can create confusion in the imaginary. (interviewee 4) given the descriptions (kádeková & holienčinová, 2018) and prescriptions (moustakas et al., 2020) of the literature, there are those who ask “which” human is at stake: we talk about machine learning, but it will learn what? what is the correct learning? (...) if a machine in 2016 in 24 hours has already become a neo-nazi, imagine today, when radicalism is more acute? look at what a danger it can become! (interviewee 5) another interviewee, consistent with the social comparison (miller & dollard, 1941) and parasocial theories (horton & wohl, 1956), expresses strong concerns about some people’s mental health: you are inspired by something that does not exist, that is manufactured—it is very cruel. (...) if we check this issue of the virtual influencer, which is more perfect and idealized, it will always be something that is so distant and perhaps the frustration is greater and can even bring more anguish and anxiety. (interviewee 6) some argue, however, that anthropomorphism can generate greater identification of humans in relation to vi (moustakas et al., 2020), which still requires further studies. the more anthropomorphic, the more authentic (at least in theory) the virtual influencer will be. in this sense, one interviewee spoke about what would be advances and setbacks: the challenge is to get a little out of this (...) conspiracy theory that it is a machine and is always trying to deceive me. try to make it as human as possible. and that goes from the visual to the text and the way it interacts. and we already see some examples coming very close to this humanization, but at the same time it can be a good tool for you not to control, but to direct your audience to what you want. (interviewee 2) a priori, anthropomorphism is the only category in this study aimed only at robots. but the fact that some flesh and blood influencers give charity and donations after a scandal with a reduction in followers and contracts, wouldn't it be an attempt to appear more... “human”? australasian journal of information systems batist & chimenti 2021, vol 25, research article “humanized robots” 18 4.5 scalability the questioning of the literature (kádeková & holienčinová, 2018) about whether nonhumans would surpass humans (scalability) arouses non-convergent reactions in the interviewees of this study: i think that replacing the human with a robot will still take a while, (...) but it is a trend (...) those who do not know how to program will be out of the market (...) so, it is very important that people adapt to the scenario and start to reflect on how they will be useful in this world of virtualization, robotization, etc. we will all be teachers of robots in some way. (interviewee 5) virtual influencers, in terms of productivity, would have a lot of advantages compared to human beings. (...) imagine that dream of not having to go to sleep at 5 am, like i did yesterday, producing content? that would be incredible! (interviewee 7) in addition, resources such as machine learning and ai may be disruptive by attracting new markets through investors and advertisers who avoid the "unpredictability" of human influencers, for example. it is also possible that the vi is created by the brand itself, as has already been happening. these are just some of the challenges and opportunities involving virtual influencers. if a disruption via vi eventually occurs in the marketing industry, it may counteract the trend seen in other disruptions in which “former industry leaders have become irrelevant not because of their direct competitors, but because of new products and services coming from other industries” (orlova et al., 2020, p. 1). the advance of “synthetic media” that combines cgi and artificial intelligence resources is a phenomenon on the rise (alexander, 2019) with promising applications in marketing. in addition, in 2020, it was announced that for the first time a film will feature an artificially intelligent actor. science fiction film b, with an estimated initial cost of us$ 70 million, will have erica, a humanoid robot (keegan, 2020). the potential for convergence between influence, entertainment, and ai deserves attention. the feat of b can represent a step far beyond any work of science fiction to date. another sign of possible scalability to be confirmed are that the posts by lil miquela, the biggest virtual influencer in the world, had 224% more reach than publications by human influencers with the same number of followers in 2018 (snpt, 2018). in 2019, vis were almost three times more engaged than humans (hypeauditor, 2019). it is difficult to know whether the numbers are just a "novelty effect" of the vi or other attributes. it should be noted. 5 theoretical and management implications figure 4 presents a graphic summary with five categories (anthropomorphism /humanization, attractiveness, authenticity, scalability, and controllability) that can assist future studies and management decisions. categories that make up the vi are dynamic and interrelated with nonlinear movements occurring between them. but these possibilities need to be tested. this is one of the first peer-reviewed studies on virtual influencers in the world and allows us to seek more structured answers about non-human influencers, especially vis, bringing not only the voice of specialists, but for the first time of vis and their followers. the most relevant theoretical aspect is the addition of a new unit of analysis (non-human/vi) to the studies of digital influence, and the addition of new categories to understand this new phenomenon. apparent certainties, such as "followers can miss the human touch" (moustakas et al., 2020), can be questioned. in fact, in the studies by moustakas et al. (2020), robinson (2020), and in australasian journal of information systems batist & chimenti 2021, vol 25, research article “humanized robots” 19 this study, most of the findings can be applied to humans and non-humans without major adjustments. figure 4. categories to understand the virtual influencer phenomenon it is possible to imagine that some sectors will have more vi representatives in the future or will prioritize vis, especially for reasons of controllability and scalability of outsourced vis, not to mention the possibility of customization of own vis, time in traffic, environmental sustainability, etc. in the long run, with the advance of ai, popularization of vis and tools to build them, scalability of vis, control (avoiding scandals, for example), greater adherence by followers, and the development of agencies, for example, the point will come where investing in human influencers will be the basic while investing in vi will be the most advanced and even most prestigious level in some sectors. in fact, this is already happening, albeit in an embryonic way through major brands. popularization might be a matter of time. thus, considering the literature review, netnography and interviews of this study, these are some managerial paths for investments in outsourced or own vis, as shown in table 4. with due proportions and considering the long term, the vi may be for the human influencer as the uber service is for the taxi service (in technology, innovation, hiring, business model, scalability, etc.). if so, arguments of some influencers will not stop the progress of vis. it is another step towards virtualization: kozinets (1998; 1999) pointed to the growth of virtual communities and, later (kozinets et al., 2010), to the network narratives of these same communities. vis are now growing as new virtual elements with communities that surround them (followers, advertisers, fan clubs, etc.) and with narratives that emulate human appearances, interactions, and emotions or simply do not care about them. vis are starting a form of disruption in digital marketing and especially in billionaire influencer marketing, attracting global brands to reach millions of anonymous followers who seem to have reason to seek diversification online. ultimately, the symbiosis between vis and technologies such as customer service bots, robot therapists, investment bots, artificial intelligence, distance education, social network algorithms, and others can symbolize a much more complex disruption: humanity will no longer need humans for many activities and can use robots (virtual or not) for more and more possibilities ranging from sexual activities to financial investment decisions and from distance education tutoring to rock shows. this would free humanity up for several other possibilities. in fact, everyone knows that this future has already begun. it is just gaining new actors, new tools, and new memberships. and looking at it with only the eyes of the past may not be australasian journal of information systems batist & chimenti 2021, vol 25, research article “humanized robots” 20 enough. without preciousness or naivety, options in vis present risks and opportunities. this study offers initial ideas (some for testing), but one of the results obtained here is that sometimes humans can be replaced by non-humans and even with more similarities or benefits than imagined. it remains to be seen whether this is advantageous. a) some segments (technology, fashion, etc.) seem more open to vis, but less "obvious" segments such as food can benefit from this type of influence under some conditions. b) in some cases, it may be necessary to make a transition before investing in vis. as a transition between the traditional influencer (human) to the vi (non-human), one can start with cartoon-style vis and if necessary move on to more humanized vis. another strategy would be to create interactions between humans and vis. in some cases, the human can be the ideal alternative, while the non-human can be the ideal in other segments and situations. c) although younger generations seem more open to vis, certain groups or niches may be more open or resistant than others within the same generation. d) when possible, “a/b” or other tests can be used to make simultaneous announcements (one with human and one with vi, for example), comparing results and giving support to decisions. e) in addition to the strategic aspects, the eventual decision between the vi itself or third parties requires budgetary (which is still high) and technical attention. advanced options, such as very thought-provoking graphics and a really refined script, can only correspond to the "basics" in vi. f) benchmarking with companies that have invested or invest in vis can be an interesting way to start. the formation of partnerships or groups of investors/advertisers can also dilute risks and/or costs in some cases. g) have you thought about an investment that reduces scandals, does not get stuck in traffic, is scalable, has an indefinite validity, and preserves the environment? the creators of some of the major vis have also thought about these issues. h) you will be able to meet humanized robots and robotic humans. in some situations, only one of them will deserve your attention. i) authenticity is relevant, but it may be less important than they usually admit. unbelievable secrets can live near (or far) from an address called "obvious". j) you can invest in "more of the same" (cartoons, for example) or seek innovations. even among humans and non-humans, there may be an option to innovate. looking at the 21st century can help you make the right choice. k) don't neglect the narrative (visual, textual, argumentative, etc.). if talking rats have so much power, what can humanoids with really engaging narratives do? table 4. highlights on vi investments 6 conclusions this work is the beginning of a new and unpredictable journey. inserted in a broader digitalization/virtualization context amplified by the pandemic covid-19, some virtual influencers have resources such as realistic images, exquisite narratives, and other refinements. in an attempt to help understand this relatively recent phenomenon, this study joins the rare literature on vis (robinson, 2020; moustakas et al., 2020) and is one of the first peer-reviewed studies to address virtual influencers in the world in addition to being the first to employ a systematic review of literature and netnography with vi consumers/followers and with vis themselves. as a contribution, in addition to a wide review on digital influence, interviews were conducted with experts and agents linked to digital marketing. three australasian journal of information systems batist & chimenti 2021, vol 25, research article “humanized robots” 21 categories were identified from the literature (attractiveness, authenticity, and controllability) for vi analyses. two other categories emerged from the interviews and the netnography: scalability and anthropomorphism/humanization. the five categories can interact in a nonlinear and extremely dynamic way. in the interviews of this study and in the study by moustakas et al. (2020), a good part of the weightings of academic and market experts points to more similarities or convergences than disparities between human and non-human influencers, something that was also possible to observe via netnography. although the use of artificial intelligence is not so clear and the dependence on human teams is still considerable, machine learning, self-supervision, language, and other attributes of artificial intelligence have grown strongly in recent years. the natural language generation (nlg) model has evolved from 1 billion parameters to 17 billion in just one year (microsoft, 2020), for example. in addition, there are significant investments for using artificial intelligence (alexander, 2019) and other resources to obtain differentiated results in digital marketing, including in the creation and behaviour of vis. it should also be noted that virtualization has accelerated with the covid-19 pandemic. in this scenario, for the reasons explained and others, the boundaries between real and virtual and between human and non-human are being reduced and presenting new realities or new concepts of reality. there are even those who say that there is nothing "morally significant" that differentiates virtual influencers from humans (robinson, 2020). virtual influencers bring new scenarios to digital marketing and more. if most analyses (including this study) point to more convergence than divergence between real and virtual and between human and non-human, then of the two, one: 1) it is necessary to change the analytical structure or 2) there is a way for influential robots with an increasingly “human, too human” (nietzsche 1908/1878) touch. the second option would be a good one. or not? 7 limitations and future research this study is exploratory and does not seek to establish generalizations or quantify causal relationships between variables, its research design does not allow statistical analysis, and the criteria used to select vi profiles in the netnography is a limitation. as it reproduces the top five of a world ranking, there is a risk of selection bias. less expressive profiles may have attributes not identified in this study, so the profiles selected may not represent the entire spectrum of vi characteristics and future investigations should include different profiles. regarding future work, the phenomenon allows multiple approaches, methods, and fields of study. building upon the categories identified in this paper, future studies can investigate more deeply how and why people get involved with vi. in addition, they can assess the implications of different types of vi on various aspects such as consumption, work, behaviour, and social (and/or para-social) relationships. various comparisons between humans and vi can be undertaken and involve everything from traits such as empathy to performance (engagement, conversion). one of the central studies for the management/marketing area would be analysing the managerial implications of reputation and performance with a focus on the performance of vis in conversion/sales. studies on technique, graphics, autonomy, and the use of artificial intelligence to improve vi (in addition to ethical, philosophical, and legal aspects) also have strong research potential. finally, issues such as complementarity and eventual replacement of humans by non-humans (kádeková & holienčinová, 2018) are crucial and can be researched from different perspectives ranging from marketing to public policies. australasian journal of information systems batist & chimenti 2021, vol 25, research article “humanized robots” 22 references alexander, j. (2019). virtual creators aren’t ai — but ai is coming for them. the verge. theverge.com/2019/1/30/18200509/ai-virtual-creators-lil-miquela-instagram-artificialintelligence appel, g., grewal, l., hadi, r., & stephen, a. t. (2020). the future of social media in marketing. journal of the academy of marketing science, 48, 79-95. https://doi.org/10.1007/s11747-019-00695-1 aran-ramspott, s., fedele, m., & tarragó, a. (2018). youtubers’ social functions and their influence on pre-adolescence. media education research journal, 26(57), 71-79. https://doi.org/10.3916/c57-2018-07 archer, c., & harrigan, p. (2016). prosumers with passion: learning what motivates bloggers as digital influencer stakeholders. prism, 13(1), 1-14. www.prismjournal.org/ uploads/1/2/5/6/125661607/v13-no1-a4.pdf ashman, r., patterson, a., & brown, s. (2018). ‘don't forget to like, share and subscribe’: digital autopreneurs in a neoliberal world. journal of business research, 92, 474-483. https://doi.org/10.1016/j.jbusres.2018.07.055 association of national advertisers. (2018). survey report: how ana members are using influencer marketing. https://www.ana.net/getfile/26389 belk, r., fisher, e., & kozinets, r. qualitative consumer and marketing research. london: sage, 2013. boerman, s. c. (2020). the effects of the standardized instagram disclosure for microand mesoinfluencers. computers in human behavior, 103, 199-207. https://doi.org/10.1016/ j.chb.2019.09.015 bond, b. (2016). following your “friend”: social media and the strength of adolescents' parasocial relationships with media personae. cyberpsychology, behavior, and social networking, 19(11), 656-660. https://doi.org/10.1089/cyber.2016.0355 california. (2018). senate bill no 1001. an act to add chapter 6 (commencing with section 17940) to part 3 of division 7 of the business and professions code, relating to bots. approved september 28. california legislative information. http://leginfo.legislature.ca.gov/faces/billtextclient.xhtml?bill_id=201720180sb1001 carvalho, g. j. (2018). social networks and digital influences: a description of the influences in the digital consumer behavior. pmkt – brazilian journal of marketing research, opinion and media, 11(3), 277-288. http://www.revistapmkt.com.br/portals/9/revistas/ v11n3/engb/4social%20networks%20and%20digital%20influences%20a%20description %20of%20 the%20influences%20in%20the%20digital%20consumer%20behavior.pdf casaló, l. v., flavián, c., & ibáñez-sánchez, s. (2020). influencers on instagram: antecedents and consequences of opinion leadership. journal of business research, 217, 510-519. https://doi.org/10.1016/j.jbusres.2018.07.005 clifford, d., & verdoodt, v. (2017). integrative advertising: the marketing 'dark side' or merely the emperor's new clothes? european journal of law and technology, 8(1), 1-43. researchgate.net/publication/316101961 australasian journal of information systems batist & chimenti 2021, vol 25, research article “humanized robots” 23 cotter, k. (2018). playing the visibility game: how digital influencers and algorithms negotiate influence on instagram. new media & society, 21(4), 895-913. https://doi.org/10.1177/1461444818815684 creswell, j., & miller, d. (2000). determining validity in qualitative inquiry. theory into practice, 39(3), 124-130. https://doi.org/10.1207/s15430421tip3903_2 dobreva, d., grinnell, d., & innes, m. (2019). prophets and loss: how “soft facts” on social media influenced the brexit campaign and social reactions to the murder of jo cox mp. policy & internet, 12(2), 144-164. https://doi.org/10.1002/poi3.203 drenten, j., gurrieri, l., & tyler, m. (2020). sexualized labour in digital culture: instagram influencers, porn chic and the monetization of attention. gender work organ, 27(1), 4166. https://doi.org/10.1111/gwao.12354 edgerton, e., reiney, e., mueller, s., reicherter, b., curtis, k., waties, s., & limber, s. p. (2016). identifying new strategies to assess and promote online health communication and social media outreach. health promotion practice, 17(3), 448-456. www.jstor.org/stable/10.2307/26734036 emma, r. (2018). the rise of the virtual influencers. influencer. https://blog.influencer.uk/therise-of-virtual-influencers-1fdb0c13907f flanagan, j. c. (1954). the critical incident technique. psychological bulletin, 51(4). https://www.apa.org/pubs/databases/psycinfo/cit-article.pdf gillpatrick, t. (2019). the digital transformation of marketing: impact on marketing practice & markets. economics, 7(2), 139-156. https://doi.org/10.2478/eoik-2019-0023 gillpatrick, t., blunck, e., & boğa, s. (2019). understanding the role of consumer behavior in forecasting the impact of industry 4.0 and the wave of digital disruption driving innovation in retailing. dubrovnik international economic meeting, 4(1), 165-176. hrcak.srce.hr/228708 globalwebindex. (2019). global web index social media report. https://www.globalwebindex.com/reports?page_num=10 goanta, c., (2018). how technology disrupts private law: an exploratory study of california and switzerland as innovative jurisdictions. stanford-vienna transatlantic technology law forum, 38, 1-53. https://law.stanford.edu/publications/no-38-how-technologydisrupts-private-law-an-exploratory-study-of-california-and-switzerland-asinnovative-jurisdictions/ golder, s. a., & macy, m. w. (2014). digital footprints: opportunities and challenges for online social research. annual review of sociology, 40, 129-152. www.jstor.org/stable/ 43049529 graziano, d. c., & perez, g. (2018). the impact of the internet on the e-commerce organizations. journal of financial innovation, 1-13. www.researchgate.net/publication/ 330015305 gross, j., & wangenheim, f. v. (2018). the big four of influencer marketing. a typology of influencers. marketing review st. gallen, 2, 30-38. https://papers.ssrn.com/sol3/ papers.cfm?abstract_id=3230687 australasian journal of information systems batist & chimenti 2021, vol 25, research article “humanized robots” 24 guerreiro, c., viegas, m., & guerreiro, m. (2019). social networks and digital influencers: their role in customer decision journey in tourism. journal of spatial and organizational dynamics, 7(3), 240-260. www.jsod-cieo.net/journal/index.php/jsod/article/view/198 hasan, h., & linger, h. (2020). letting the public in: dialectic tensions when local governments move beyond e-government to e-democracy. australasian journal of information systems. vol. 24, https://doi.org/10.3127/ajis.v24i0.1897 horton, d., & wohl, r. r. (1956). mass communication and para-social interaction, psychiatry, 19(3), 215-229. https://doi.org/10.1080/00332747.1956.11023049 hutchinson, j. (2019). digital first personality: automation and influence within evolving media ecologies. convergence, 26(5-6), 1284-1300. https://doi.org/10.1177/ 1354856519858921 hypeauditor. (2019). the top instagram virtual influencers in 2019. https://hypeauditor.com/blog/the-top-instagram-virtual-influencers-in-2019/ jiménez-castillo, d., & sánchez-fernández, r. (2019). the role of digital influencers in brand recommendation: examining their impact on engagement, expected value and purchase intention. international journal of information management, 49, 366-376. https://doi.org/10.1016/j.ijinfomgt.2019.07.009 jorge, a., marôpo, l., & nunes, t. (2018). ‘i am not being sponsored to say this’: a teen youtuber and her audience negotiate branded content. observatorio. special issue on the co-option of audiences in the attention economy. https://doi.org/10.15847/ obsobs0001382 kádeková, z., & holienčinová, m. (2018). influencer marketing as a modern phenomenon creating a new frontier of virtual opportunities. communication today, 9(2), 90-104. www.researchgate.net/publication/329247338 kautz, k., bjerknes, g., fisher, j., & jensen, t. (2020). applying complex adaptive systems theory to understand distributed participatory design in contemporary, crowdsourced information systems development. australasian journal of information systems. vol. 24, selected papers from acis 2018. https://doi.org/10.3127/ajis.v24i0.2225 keegan, r. (2020). a.i. robot cast in lead role of $70m sci-fi film. the hollywood reporter. www.hollywoodreporter.com/news/ai-robot-cast-lead-role-70m-sci-fi-film-1300068 kozinets, r. v. (1998, november 9). how online communities are growing in power. financial times, 291–96. kozinets, r. v. (1999). e-tribalized marketing: the strategic implications of virtual communities of consumption. european management journal, 17(3), 252-264. https://doi.org/10.1016/s0263-2373(99)00004-3 kozinets, r. v. (2002). the field behind the screen: using netnography for marketing research in online communities. journal of marketing research, 39(1), 61–72. https://doi.org/10.1509/jmkr.39.1.61.18935 kozinets, r. v. (2015). netnography: redefined. london: sage. australasian journal of information systems batist & chimenti 2021, vol 25, research article “humanized robots” 25 kozinets, r. v. (2019). youtube utopianism: social media profanation and the clicktivism of capitalist critique. journal of business research, 98, 65-81. https://doi.org/10.1016/j.jbusres.2019.01.019 kozinets, r. v., de valck, k., wojnicki, a. c., & wilner, s. j. s. (2010). networked narratives: understanding word-of-mouth marketing in online communities. journal of marketing, 74(2), 71–89. https://www.researchgate.net/publication/255600900 kozinets, r. v., & nocker, m. (2018). netnography: online ethnography for a digital age of organization research. in a. bryman, & d. a. buchanan (eds.). unconventional methodology in organization and management research (pp. 127–146). cambridge, uk: oxford university press. kuksov, d., & liao, c. (2019). opinion leaders and product variety. marketing science, 38(5), 812-834. https://doi.org/10.1287/mksc.2019.1179 macinnis, d. j., morwitz, v. g., botti, s., hoffman, d. l., kozinets, r. v., lehmann, d. r., lynch, j. g., & pechmann, c. (2020). creating boundary-breaking, marketing-relevant consumer research. journal of marketing, 84(2), 1–23. https://doi.org/10.1177/0022242919889876 maden, d. (2018). the role of digital influencers in the diffusion of new products. akdeniz üniversitesi i̇letişim fakültesi dergisi, 30, 119-141. https://doi.org/10.31123/akil.465923 marôpo, l., jorge a., & tomaz, r. (2020). “i felt like i was really talking to you!”: intimacy and trust among teen vloggers and followers in portugal and brazil, journal of children and media, 14(1), 22-37. https://doi.org/10.1080/17482798.2019.1699589 mcluhan, m. & fiore, q. (1967). the medium is the massage: an inventory of effects. new york: random house. metro. (2020). pesquisa aponta evaristo costa como maior ‘influencer' do brasil (research appoints evaristo costa as the greatest 'influencer' in brazil). metrojornal. metrojornal.com.br/entretenimento/2020/02/17/evaristo-costa-maior-influencerbrasil.html microsoft. (2016). learning from tay’s introduction. peter lee (corporate vice president). official microsoft blog. https://blogs.microsoft.com/blog/2016/03/25/learning-taysintroduction/#sm.00000gjdpwwcfcus11t6oo6dw79gw microsoft. (2020). microsoft announces new supercomputer, lays out vision for future ai work. jennifer langston. the official microsoft blog. https://blogs.microsoft.com/ ai/openai-azure-supercomputer/ miles, m. b., huberman, a. m., & saldaña, j. (2014). qualitative data analysis: a methods sourcebook. thousand oaks, ca: sage, 2014. 381 pp. miller, n. e., & dollard, j. (1941). social learning and imitation. new haven: yale university press. https://doi.org/10.1177/00027162422200019 miranda, s., cunha, p., & duarte, m. (2019). an integrated model of factors affecting consumer attitudes and intentions towards youtuber-generated product content. review of managerial science. https://doi.org/10.1007/s11846-019-00370-3 australasian journal of information systems batist & chimenti 2021, vol 25, research article “humanized robots” 26 moura, b. s. (2019). relationship marketing in social networks for small businesses. pmkt – brazilian journal of marketing research, opinion and media, 12(1), 46-62. revistapmkt.com.br/portals/9/revistas/v12n1/en-gb/5%20%20relationship%20marketing%20in%20social%20networks%20for%20small%20busi nesses.pdf moustakas, e., lamba, n., mahmoud, d., & ranganathan, c. (2020). blurring lines between fiction and reality: perspectives of experts on marketing effectiveness of virtual influencers. social media conference ieee. international journal on cyber situational awareness. https://doi.org/10.22619%2fijcsa nietzsche, f. (1908). human, all too human – a book for free spirits (alexander harvey, ed. & trans.). charles h. kerr & company. (original work published 1878). https://www.gutenberg.org/files/38145/38145-h/38145-h.htm orlova, a., nogueira, r. & chimenti, p. (2020). the present and future of the space sector: a business ecosystem approach. space policy, 52, 1-8. https://doi.org/10.1016/j.spacepol.2020.101374 robinson, b. (2020). towards an ontology and ethics of virtual influencers. australasian journal of information systems, 24, 1-8. https://doi.org/10.3127/ajis.v24i0.2807 silva, m. j. b., farias, s. a., grigg, m. k., & barbosa, m. l. a. (2020). online engagement and the role of digital influencers in product endorsement on instagram. journal of relationship marketing, 19(2), 133-163. https://doi.org/10.1080/15332667.2019.1664872 snpt. (2018). https://www.getsnapppt.com/blog/can-artificial-influencers-replace-humans torres, p., augusto, m., & matos, m. (2019). antecedents and outcomes of digital influencer endorsement: an exploratory study. psychology & marketing, 36(12), 1267-1276. https://doi.org/10.1002/mar.21274 verdoodt, v., & feci, n. (2019). digital influencers and vlogging advertising: calling for awareness, guidance and enforcement. larcier. auteurs en media, 1, 11-22. https://limo.libis.be/primoexplore/fulldisplay?docid=lirias2339326&context=l&vid= lirias&search_scope=lirias&tab=default_tab&lang=en_us&fromsitemap=1 vinerean, s. (2017). importance of strategic social media marketing. expert journal of marketing, 5(1), 28-35. http://marketing.expertjournals.com/23446773-504/ #:~:text=nowadays%2c%20social%20media%20has%20developed,to%20instigate%20c ustomer%2dto%2dcustomer copyright: © 2021 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems batist & chimenti 2021, vol 25, research article “humanized robots” 27 doi: https://doi.org/10.3127/ajis.v25i0.3223 1 introduction 2 literature review 2.1 concept and types of influencers 3 research method 4 results and discussion 4.1 attractiveness 4.2 authenticity 4.3 controllability 4.4 anthropomorphism/humanization 4.5 scalability 5 theoretical and management implications 6 conclusions 7 limitations and future research paper title (paper title style) australasian journal of information systems rathore, maurya & srivastava 2021, vol 25, research on user involvement do policymakers use social media 1 do policymakers use social media for policy design? a twitter analytics approach ashish kumar rathore great lakes institute of management, chennai, india ashish.r@greatlakes.edu.in dayashankar maurya international university of japan amit kumar srivastava t a pai management institute, india abstract social media has been widely used for communicating information, awareness, and promote public policies by government agencies. however, limited attention has been paid to the use of social media in improving the design of public policies. this paper explores to what extent citizens' opinions and responses expressed on social media platforms contribute to policy design. the paper analyzes discussion about the 'ayushman bharat' scheme on twitter through social media analytics techniques such as content analytics and then traces the change in policy design over two years. to validate findings from twitter data, and assess the evolution in policy design, we conducted in-depth interviews with experts and extensive document analysis. the paper reveals that consistently similar issues were raised by experts in the past as well as by citizens in the current scheme. however, over time policy design has not changed significantly. therefore, despite a strong social media presence, its optimum use to improve policy effectiveness is yet to be achieved. the paper contributes by exploring the role social media can play in the public policy process and policy design in developing countries' contexts and identifies gaps in existing social media strategies of public agencies. keywords: social media, twitter analytics, public policy, policy co-design, policy responsiveness 1. introduction policy design is considered to be the core of effective public policy (peters, 2015; howlett, 2019). in developing country context, poor policy designs have been common due to the high incidence of bargaining, corruption of clientelism, log rolling, electoral opportunism, and capacity constraints (boubakri et al., 2008; bali & ramesh, 2015). limited policy monitoring and the use of data remain crucial constraints in improving policy design in developing countries (khawaja, 2011; holvoet & inberg, 2014). the emergence of social media (sm) platforms has made it easy and cost-efficient to capture citizens' reactions and opinions on a public policy or scheme (eze et al., 2016; miah et al., 2017; morgan et al., 2018; aswani et al., 2018b). in recent years, sm has emerged as a powerful tool to engage various stakeholders in the knowledge exchange process (corcoran & duane, 2018; morgan et al., 2018; lim et al., 2019), and government agencies are also actively using sm for various purposes (chun & luna-reyes, 2012; linders, 2012; malsbender et al., 2014; hacker et al., 2017; eom et al., 2018; hedestig et al., 2018). however, little is known about the use of social media analytics (sma) in policy co-design, which requires policy designers and citizens to come together to design solutions to problems critical to improving the effectiveness of policy design. australasian journal of information systems rathore, maurya & srivastava 2021, vol 25, research on user involvement do policymakers use social media 2 this paper explores a social health insurance program called 'ayushman bharat' in india to investigate the extent to which public agencies use citizens' opinions and expectations to improve the design of this scheme. the paper uses sma to obtain a generic and bigger picture of citizens' opinions to analyze scheme-related posts. the posts were collected from twitter, a microblogging sm platform that contains various metadata information that can be collected cost-effectively (aleti et al., 2016; rathore et al., 2017; aswani et al., 2018a; grover et al., 2018). india has experimented with several social health insurance schemes over the last six decades, but 'ayushman bharat' launched in 2018 has been the most ambitious public health insurance scheme. given the experience of citizens with previous social health insurance schemes over the last two decades, citizens are expected to be well informed on the issues related to social health insurance programs. in this paper, we mapped the sm responses of citizens at the launch of ‘ayushman bharat’ in 2018 to identify the critical concerns related to its design. this was followed by tracing the evolution of program design from august 2018 to may 2020 using document analysis and in-depth interviews assessing the extent to which the program designers responded to concerns expressed by citizens. the findings suggest that although agencies implementing the ‘ayushman bharat’ program has a significant presence on sm and tend to use sm to address grievances expressed by citizens and other stakeholders, their use of sm for co-design has been limited. the concerns expressed by citizens at the time of the program's launch have been consistently the same as those identified by experts based on their experience of similar social health insurance schemes in the past. even though sm has been used by agencies, there have been only limited efforts to improve the program’s design. thus, program designers are yet to use online platforms to engage stakeholders in policy design, especially in developing countries. this paper seeks to fill the literature gap on the online engagement of stakeholders in the policy co-design using sm to design effective public policies. the paper contributes to (a) the policy co-design process, which has received limited attention in the literature; and (b) the role of sm in the public policy process in the developing countries' context. finally, the paper highlights the gaps in existing strategies of sm usage by public agencies in developing countries. the paper was been divided into seven sections. the first section introduces the need and scope of sm in designing healthcare schemes in developing countries such as india. the second section includes the existing literature on the policy design domain and that on social media monitoring (smm) (bekkers et al., 2013). the third section introduces the research context of the paper. the fourth section explains the research methodology used in this paper. the fifth section demonstrates the results of the analysis. the sixth section discusses the implications of insights from the previous section. the last section contains the concluding remarks, followed by the limitations of current work and future directions. 2. theoretical background poor policy design is considered a critical reason for policy failure (peters, 2015; peters et al., 2018). theoretical frameworks in policy design refer to policy design as a process of conscious and deliberate effort to define policy aims and select policy tools (such as taxes, regulations, etc.) and calibrate policy tools to achieve specific outcomes (peters et al., 2018; howlett, 2019). evidence-based policy design needs policy actors to aggregate, communicate and manage knowledge resources on what policy tools work, in what conditions, and how they are being australasian journal of information systems rathore, maurya & srivastava 2021, vol 25, research on user involvement do policymakers use social media 3 perceived by target groups (peters et al., 2018; howlett, 2019). theoretical policy design frameworks suggest that even though the goal of policy design is to invoke the desired behavioral change in the target group, various barriers and impediments may prevent it. for example, compliance with a policy by citizens depends upon their perception of the legality, normative and moral appropriateness of the policy and the capability and resources citizens must use to be compliant (wan et al., 2015; howlett, 2019). therefore, this uncertainty about the compliance of citizens with the intention of the designed policy is one of the critical challenges of policy design (howlett, 2019). one of the ways to address this is actively engaging with target groups after the policy is launched to improve the design in case the target group’s compliance is not forthcoming. 2.1 public opinion and communicative action for policy co-design the theoretical frameworks in policy design give primacy to actors in policy subsystems such as experts, political actors, and bureaucrats as a policy is generally designed by bureaucrats, politicians, and consultants; therefore, the process is dominated by experts (howlett et al., 2015; howlett, 2019). however, recent trends suggest that citizens could also be fruitfully involved in the policy design process referred to as co-design. co-design involves "enabling or empowering the people affected by a policy issue to actively contribute to developing a solution for it" (blomkamp, 2018, pg. 732). co-design is argued to improve idea generation, service delivery, project management, and longer-term outcomes (steen et al., 2011). however, challenges remain regarding how to collect public opinion and involve citizens in the codesign process. collecting and using public opinion is not a straightforward process as public opinion is usually strong and diverse on critical, long-term issues (zaller, 1992) but limited in the case of complex and arcane issues (arnold, 1990). second, even though public opinion is collected, how it is used depends on the relationship between the user of the information and the provider of the information, as the habermas ‘theory of communicative action’ suggests ((habermas, 1984; habermas, 2015). this theory suggests three distinct ways in which the information collected may be used. first, in a rational or instrumental approach, the user aims at realizing pre-given goals using the information provided about the environment by citizens and the nomological knowledge about effective intervention. in the second approach, known as the political-strategic approach, the user of the information considers the goals and power resources of the provider of the information and alternative actions available to the provider. in the third approach, known as the communicative approach, the user of the information seeks to reach a common understanding with the providers of the information, especially about the interpretation of the problem. as the habermas theory of communicative action suggests, even though citizens are involved in the co-design process, the outcome of the process will depend upon how the provided information is used. for a policy co-design process to happen, information needs to be used in the communicative approach rather than in the rational and strategic approaches. 2.2 citizens’ engagement through sm for policy co-design in recent years, sm has provided a fast and cost-effective opportunity to obtain public opinion on policy issues and are considered to have the potential to reform public administration (linders, 2012; medaglia & zheng, 2017). governments are using sm to improve transparency, the dissemination of policy information (bertot et al., 2010; linders, 2012), citizen engagement (criado et al., 2013; gintova, 2019), and the co-production of services (loukis et al., 2017; australasian journal of information systems rathore, maurya & srivastava 2021, vol 25, research on user involvement do policymakers use social media 4 hedestig et al., 2018). however, critics argue that sm is used mainly for self-promotion and political marketing rather than improving transparency, participation, and service delivery (picazo-vela et al., 2016; medaglia & zheng, 2017). given the extensive use of sm by public agencies, sm is considered to have the extensive potential for improving policy design. however, there has been minimal discussion on using sm in the policy design process (loukis et al., 2017; hedestig et al., 2018). existing policy co-design approaches use crowdsourcing and digital participatory platforms to engage citizens (taeihagh, 2017; falco & kleinhans, 2019), but there has been little research on the use of sm for the same purpose, even though sm is used much more extensively, even in developing country contexts. one of the approaches to using sm in co-design is smm. smm has been defined as the continuous, systematic observation and analysis of sm networks and social communities (fensel et al., 2012; bekkers et al., 2013). using smm, the government can passively monitor sm communities related to a specific topic or policy (loukis & charalabidis, 2015; loukis et al., 2017). however, to access knowledge from sm content, smm is not enough. the limitation of smm can be addressed by sma as it is a more appropriate approach to understand the influence of different stakeholders (rathore et al., 2017; aswani et al., 2018a; grover et al., 2019b). many businesses have been using sma to collect knowledge, ideas, and opinions about their products and services from users to improve the design of their products and services (zhang & vos, 2014; rathore & illavarasan, 2020). however, research on the use of sma for coproduction by public agencies is scarce (loukis et al., 2017; hedestig et al., 2018). through the co-design process, governments can gain much by collecting citizens' inputs through sm platforms to improve the design of public services in an evidence-based manner (androutsopoulou & charalabidis, 2018; alizedah et al., 2019). applying the theory of communicative action (habermas, 1984), sma can also take one of the following three approaches based on how the information is used. first, in a rational approach, sma can be used to determine the situation of the policy and intervene by using normative knowledge. second, in a political-strategic approach, sma can be used to assess what is being said about the policy, and that information can be used for reputation management. third, in the communicative approach, sma can be used for co-design purposes whereby government agencies and citizens come together to form a shared definition of the situations, problems, and policies that can be used to improve existing policies or design new policies based on the principle of collective intelligence (surowiecki, 2004). 2.3 communicative approach of sma for policy co-design for co-design, the use of sma requires adopting a communicative approach (linders, 2012; bekkers et al., 2013). in this approach, first, government agencies gather information about citizens' perceptions, grievances, and demands and obtain feedback about the policies that are being developed or implemented. this is followed by responding to this information by engaging and interacting to generate new ideas and policy improvements (linders, 2012). although co-design using sma could be very cost-efficient and quick, it has its own set of challenges. co-design requires extensive coordination to manage and balance diverse objectives and interests in a time-bound manner (blomkamp, 2018). apart from usual difficulties in the use of sma by governments, such as irrelevant noise and chatter, the reliability and quality of information, information overload, political risks, and privacy violations (beer & burrows, 2007; sutton, 2009), there are additional challenges for using sma in co-design. the first challenge is obtaining a representative from every segment of the population; otherwise, the data could be considered to be biased (euripidis et al., 2017). australasian journal of information systems rathore, maurya & srivastava 2021, vol 25, research on user involvement do policymakers use social media 5 second, population groups that do not have access to the internet are left out, leading to participation inequalities (brabham, 2012). the third challenge is the capability of government agencies to manage sm and the co-design process (blomkamp, 2018; netshakhuma, 2019). public opinion research suggests that public opinion is expected to be high if the information is available to all (downs, 1957; becker, 1983; williamson, 1993), but most often groups placing pressure determine the issues (grindle & thomas, 1991; berry & wilcox, 2018), which may be different from public opinion (wright, 1996; domhoff, 1998; medaglia & zheng, 2017). as the use of sm among public agencies, departments, and programs to disseminate information has increased considerably in developing countries (warf, 2016; verma et al., 2017), there is little evidence on whether citizens' feedback on sm platforms for various programs and policies is taken into account by public agencies. public agencies in developing countries such as india engage in sma, as seen from the number of incidences where public agencies have addressed the grievances raised by citizens on sm platforms (agarwal et al., 2017; goyal et al., 2020). however, to what extent sma is used to improve the design of public services through sm activism remains unknown (poell & rajagopalan, 2015). using a case study approach, this paper explores whether the input provided on sm by citizens about public service at the launch of the policy is considered and addressed by the policy designers over a period. thus, this paper explores the empirical evidence on policy design using sm. 3. research context in a democratic setting such as india, with a high frequency of elections and a large number of political parties, making elected members accountable for policies is difficult, especially for low salience issues (arnold, 1990; jones, 1994; lindaman & haider-markel, 2002). however, public opinion has informally determined various populist policies in india, as observed by other studies (kumar & sharma, 2017; chatterjee, 2019b). we explored the role of public opinion expressed on sm platforms for policy co-design in the case of a large-scale public health insurance (phi) scheme, ‘ayushman bharat’, one of several programs that were launched by the government in the last two decades. in 2014-2015, federal and provincial governments spent approximately inr 25000 million (330.55 million usd) on approximately 15 phi schemes (irda, 2016). phi schemes have provided health insurance to 214.3 million more persons over the past decade, contributing to approximately 74% of the total health insurance coverage in india (irda annual report, 2016). phi targets low-income households and is entirely financed by the public sector while service delivery is managed by contracted for-profit insurance companies and private hospitals. over the years, experts have highlighted several critical issues in the design of phi schemes. first, phi schemes were criticized for not including ambulatory care, which contributed to 65% of the out-of-pocket healthcare expenditures (national sample survey office, 2015). second, fraudulent behavior was observed among all stakeholders primarily due to the fixed cost payment system and limited quality monitoring (maurya, 2018; maurya & ramesh, 2019). the packages in the schemes and standard of care were poorly defined, allowing hospitals to manipulate these items (la forgia & nagpal, 2012; maurya, 2018). third, beneficiaries faced challenges in enrolling either because the poverty registers were faulty, their names were not printed correctly, or they never received an identification card to access services. the beneficiaries also lacked information about what kind of services are covered by insurance. australasian journal of information systems rathore, maurya & srivastava 2021, vol 25, research on user involvement do policymakers use social media 6 fourth, the coverage of most of the phi schemes was considered shallow, ranging from 30,000 to 100,000 inr per family per year (asher et al., 2015). finally, although phi schemes have experienced increased utilization among enrolled individuals (prinja et al., 2017), no reduction in healthcare expenditures among beneficiaries was observed (mitchell et al., 2011; gupta et al., 2017; prinja et al., 2017). on september 23, 2018, the government of india (goi) launched the 'ayushman bharat' scheme, also referred to as 'pradhan mantri jan arogya yojana' (pmjay). the scheme had a similar design to the previous phi schemes but with some modifications. similar to previous phi schemes, in ‘ayushman bharat’, provincial governments also contracted with private and public hospitals based on defined packages of services. however, in ‘ayushman bharat’, the government aimed to enroll 500 million families in the program, which was much higher than the number in previous phi schemes. each family is supposed to obtain healthcare coverage of 500,000 inr per year that can be used at any of the contracted public or private hospitals (business line, 2018). the healthcare package included the costs of hospitalization, investigations, food, drugs, and some prehospitalization care. by june 21, 2020, 125 million families were enrolled in the scheme, more than 22,000 hospitals were contracted, and more than 10.7 million hospitalizations were covered by the scheme. the ‘ayushman bharat’ program is managed by the national health authority (nha), which joined twitter in may 2018. the account is followed by more than 52 thousand followers. in addition, each district where the program is implemented has its own twitter handle. the nha regularly responds to the concerns and grievances related to the services delivered raised on twitter by beneficiaries. apart from beneficiaries, other stakeholders such as experts, insurance companies, and hospitals tend to use twitter to voice their concerns, suggesting that some form of coproduction and coordination is taking place using sm. however, to what extent the feedback by citizens on the design of the program is used to improve the design is not known. therefore, the research questions that this paper seeks to address are as follows: ● what were the key issues, concerns, and sentiments expressed by the citizens about the ‘ayushman bharat’ scheme on sm platforms? ● does sma by policy designers improve policy design and to what extent? 4. methodology following the research tradition of the multimethod approach (hambrick & abrahamson, 1995; cyr et al., 2009), we adopted a two-stage methodology (lin, 2014; akrout & diallo, 2017). the first stage is sma which includes data extraction and analysis from twitter, and the second stage is semistructured interviews of health experts associated with the ‘ayushman bharat’ scheme. an interview-based qualitative study can validate the findings of sma. sma is a cost-effective approach that provides real-time insights to the government to gauge public needs and sentiments (he & yan, 2014). it helps in analyzing citizens' emotions captured through their posts (melville et al., 2009; huh et al., 2013; tuarob et al., 2014). it also helps the government in policy design and redesign (lau et al., 2014). however, the existing sma methods often adopt context-free sentiment analysis. therefore, methods for the fine-grained extraction of public intelligence from user-generated content (ugc) are required to aid policy design processes. the data on twitter is predominantly textual content, the most likely nature of content with sm posts, reflecting citizens' emotions. as of 2020, there are 275 million twitter users (statista, 2020), and its usage has been increasing in both business and nonaustralasian journal of information systems rathore, maurya & srivastava 2021, vol 25, research on user involvement do policymakers use social media 7 business domains. common citizens have also made twitter part of their daily routines to share their experiences. features such as the strict character limit of up to 280 characters, called tweets, make twitter data more concise and expressive than the data of other sm platforms such as facebook (pak & paroubek, 2010; rathore et al., 2017). short-form features categorize them in the special case of the social-emotional context to evaluate collective sentiments (hodeghatta & sahney, 2016). twitter data represent real events in terms of personal opinions, public knowledge, and collective sentiment (rathore et al., 2016; zhang et al., 2016; grover et al., 2019b). users do not need to have reciprocal relationships to share and access other users’ content. this phenomenon is referred to as asymmetric connections in the literature (wakefield & wakefield, 2016). it seems more appropriate to think of tweets as public announcements rather than private conversations. this paper analyzes tweets on the ‘ayushman bharat’ scheme to analyze public discussion and sentiment that can facilitate reflection on policy design. determining topics with the associated polarity, negative and positive, presents the relationship between the sentiment of comments and the ratings of different aspects of the scheme (cao et al., 2013). in the following paragraphs, both stages of the adopted approach are presented. stage 1 involves extracting and analyzing twitter data (figure 1), implemented through an r application using different scripts. stage 2 includes semi-structured interviews with experts in the domain of the research context, which are then subjected to document and secondary data analysis. 4.1. stage 1: empirical twitter analytics 4.1.1. data preprocessing first, we selected the event, i.e., the launch of the ‘ayushman bharat’ scheme in india. the choice of this event is driven by the fact that it is the world's largest public health insurance scheme, and it has drawn significant attention from all key stakeholders, including citizens. therefore, ‘ayushman bharat’, as the event, was able to generate enough user-generated content for the analysis. the next part includes different steps, including keyword selection, tweet extraction, and tweet preparation. we used twitter, a microblogging platform, as the data source. to determine the most appropriate keywords related to the event and avoid nonrelated events having similar kinds of discussion, we selected keywords from twitter only. this approach not only provides valid keywords, but it also provides the posting rate for the content, which is useful for predicting the data volume for a specific time interval. using selected keywords, tweets are retrieved using 'r,' an open-source tool. to perform the search, an r script was created to download the tweets weekly. the posts/comments were collected weekly through a twitter search api (application programming interface). to create our dataset, we used "#ayushmanbharat" as the keyword, which resulted in a total of 46754 tweets collected during july 2018-july 2019. to clean the data, a tweet corpus is created using another package, "tm". this tweet corpus is used for all the analyses. twitter data contain a variety of languages, unstructured abbreviations and grammar, and unofficial formats (rathore et al., 2017; grover et al. 2018). australasian journal of information systems rathore, maurya & srivastava 2021, vol 25, research on user involvement do policymakers use social media 8 figure 1. data preprocessing and data analysis stages thus, to place the data in a meaningful format by removing unusual characters, datasets are formed through an automated text mining r script and along with the use of syntactic/semantic libraries (natural language packages) through the following techniques: removing stop words, tokenization, stemming and identifying n-grams. 4.1.2. data analysis this paper adopts a content analysis approach to analyze tweets (chae, 2015; grover et al., 2019a; rathore et al., 2020). content analytics includes two parts text analytics (identifying themes) and sentiment analysis (analyzing emotions) (chau & xu, 2012; feldman, 2013; yoon et al., 2013; chae, 2015). first, various themes have been recognized using different tools (r codes1) to obtain a clear picture of citizens' preferences and expectations. the findings from ideation are prerequisites for sentiment analysis for the intelligent gathering of collective emotions associated with each of the aspects of the scheme. while text analytics (e.g., word frequency, clustering, word association, etc.) emphasizes the facts in the form of various themes, sentiment analysis provides more subjective information (e.g., emotions) in those themes. it classifies all different themes into broader emotions. sentiment word clustering groups the major words into emotional clusters, which are further linked with various themes identified in text analytics. 4.1.2.1 text analytics as a vast amount of data is generated from public reactions related to the scheme, many dynamic topics are associated with ugc (aggarwal & subbian, 2012; rathore et al., 2017). a traditional text analytics approach has been followed to address twitter data (figure 2). topic modeling is one of the most effective approaches to identify different themes from ugc. in this paper, the latent dirichlet allocation (lda) approach was used for topic modeling (yang & zhang, 2018). lda “considers each document as a multinomial distribution over topics and 1 https://drive.google.com/file/d/1tmupgvmo03rkpqj9szmzmm7atqlix9wm/view?usp=sharing ayushman bharat data preprocessing (transformation) keyword selection tweet preparation (ngram)stop word removal and stemming term document matrix twitter data (search api) word analysis (tf-tdf) sentiment polarity (positive or negative) – lexicon approach feature extraction emotion labeling (anger, fear, sad, anticipation, joy, disgust, and trust) australasian journal of information systems rathore, maurya & srivastava 2021, vol 25, research on user involvement do policymakers use social media 9 regards each to pic as a multinomial distribution over words” (ren et al., 2016, pg. 4). the lda approach does not need dissimilar streams to allocate vocabulary. it can be used to match words from different streams. figure 2. text analytics process (source: chau & xu, 2012; feldman, 2013; pang & lee, 2009) in the first stage, scheme-related topics are discovered using a data matrix approach, including various techniques such as word clouds, frequency distributions, clustering, and word association. the basic parameters for all mentioned techniques are word counts and the relationships among words. these are visualized using different two-dimensional graphs. such a word frequency-based approach creates a very noisy corpus. word frequency analysis includes word clouds based on the frequency distribution of each word. the frequency range varies from one to 1000+ among the words in a single dataset. after analyzing the frequency distribution, we found that few words with high frequency such as ‘ayushman’, ‘bharat’, and ‘scheme’ are very common. in fact, these words do not contribute significantly to the body of the context. additionally, more computational power is required to apply multiple approaches to a full corpus of words. thus, we adopted a more robust approach, tf-idf (term frequencyinverse document frequency) (qaiser & ali, 2018; curiskis et al., 2020), to obtain more significant words rather than frequent words. comparing the output from the word frequency distribution and tf-idf techniques showed that words with frequencies less than 100 are also at the bottom of the tf-idf weight list. therefore, a filter, i.e., for words with frequencies less than 100, has been applied, and these words are not considered in any further analysis in the paper. this also provides better visualization and analysis of word clouds. this paper applies the combination of word clustering, and temporal segmentation helps to determine the different topics related to the scheme. temporal segmentation differentiates various contextual aspects based on topical patterns (cataldi et al., 2010). in this paper, topics are segmented using the relationship and connectivity among words. all these relationships and connectivity are based on the various edges and nodes, which lead to a structural form. this structure is used to conclude the occurrence-based topics. in addition, to recognize aspects based on different topics, latent aspect rating analysis (lara) (wang et al., 2010) was conducted, revealing the latent aspect ratings and aspect weights in every individual review. the integration of latent ratings and aspect weights strengthens the associated topic preferred by users. to illustrate it, a word network graph is created showing the various scheme aspects and interconnections among all. nodexl , a commercial software program, has been used to perform thematic analysis, i.e., form a word clustering network. nodexl is a widely used tool for social network analysis that does not need a programming language (hansen et al., 2010; extraction • unstructured in nature • informal and composed of a short list of words, hashtags, urls, and other information transformation • cleaning and processing • tokenization, ngrams, stemming, and removing stop words summarization • subjective information • key word analysis, word frequency analysis, text clustering, and sentiment analysis australasian journal of information systems rathore, maurya & srivastava 2021, vol 25, research on user involvement do policymakers use social media 10 ahmed & lugovic, 2019). going beyond the identified topics, it is essential to understand the sentiment weight experience for different topics. 4.1.2.2 sentiment analysis while retrieving topics and patterns in trends, there are approaches to recognize the polarity and emotions of different sentiments (liu et al., 2008, rathore et al., 2016). they include machine learning and the computational detection of the attitudes and opinions of users. these attitudes and opinions are related to the subjective knowledge about the scheme or different individual aspects of the healthcare initiative. to recognize more specific aspects, the relationships and correlations between words are very important compared to just distinguishing between positive and negative opinions based on polarity and emotions. these emotions reflect the relevant citizens' states of preferences. thus, it is necessary to address the different aspects of the scheme in terms of hedonic aspects with the desired impacts. sentiment analysis identifies the appropriate sensory aspects related to the policy design of the scheme with expressive details. sentiment analysis is concerned with identifying new attributes of the utility of an emotional expression in policy design and implementation strategies. it also provides a cross-document opinionated summary of specific policy aspects that determine the degree of relevance and the degree of sentiment. in this paper, the polarity determination of a textual corpus tends toward positive, negative, or neutral, offering a variety of benefits such as the prediction of emotional information in a citizen's choices (cao et al., 2013). tweets are categorized as having positive or negative polarity with a range of various emotion classifications. the contextual evaluation includes typical assessments, such as feeling, for different scheme aspects. assuming users' posts are a function of the sentiments of their aspects, emotions are classified using sentence selection methods (king et al., 2013). this method is used to recognize the relevant words within each emotion (ku et al., 2006). this approach evaluates tweets based on the degree of sentiment polarity (e.g., positive and negative sentiments) on a scale of mapping numbers (e.g., -2 to +2). it provides a comparative assessment of the emotions useful for detecting the emotional variance of each citizen's statements. a large difference in sentiments would highlight the collective emotions in almost all aspects of the scheme. second, a combination of n-gram and lexicon-based approaches is used to improve the accuracy of emotion and polarity classification, including different aspects such as parts-of-speech and negation terms (barbosa & feng, 2010). based on a predefined set of sentiment classes, it is difficult to measure or evaluate aspects within the ugc for classifying its sentiment class in the context of the entity. aspects are in forms of words that relate to any specific lexicon. for instance, it categorizes words into such defined emotions, such as joy, sadness, anger, disgust, fear, and anticipation, using frequency as sentiment values for each category (fredrickson & losada, 2005). mapping words in each tweet from a sentiment lexicon is used to evaluate the average emotion and polarity. this sentiment lexicon contains a directory of words that are associated with all classified emotions. in this research, symbolic aspects, based on hashtags, have been used for emotion classification decisions. it offers significant sentiment insights more accurately at predicting new scheme aspects. australasian journal of information systems rathore, maurya & srivastava 2021, vol 25, research on user involvement do policymakers use social media 11 4.2. stage 2 semi-structured interviews after twitter analytics, qualitative data were gathered via a two-stage random cluster probability sample (burger & silima, 2006; henry et al., 2015). interviews were conducted with health policy experts. for the first stage of the sample, one of the experts was selected via simple random sampling. that expert became the starting point for the interviews. interviewers were instructed to seek eligible respondents at each starting point. once an eligible respondent had been located, interviewers used a snowball approach to locate subsequent respondents. a total of 17 interviews, each running for 30-60 minutes, taking a total of approximately 12 hours were conducted. the in-depth interview guidelines were prepared based on the theory of policy design (howlett et al., 2015), the design of social health insurance programs (mathuer & carrin, 2011), and how different design elements have evolved over time. the data from the interviews were analyzed using thematic and contextual analysis (meyer & goes, 1988; nowell et al., 2017). the analysis focused on the nine design elements related to social health insurance programs. for each design element, using contextual analysis, we mapped the factors that drove the adoption of the particular design element and changes over time. the contextual analysis addresses contextual conditions policy actors, policy ideas, and existing institutional arrangements. we also extended the fourcriteria framework (gibbert et al., 2008) to our research to ensure the validity and reliability of the qualitative research. 5. results and findings 5.1 text analysis text analysis, as demonstrated through a word cloud (figure 2), results in the major words in the discussion on the ‘ayushman bharat’ scheme "haryana", "hospital", "claim” "first", "health", "girl", "welcome", "nitiaayog", "primari", "half", "million", "effort", "treatment", "pillar", "healthforall", "swasthbharat", "financialexpress", "vision", "aim", "beneficari", "two", "million", "care", "reform", "path", etc. (figure 3). from these words, we can obtain small clues regarding citizens' reactions to the scheme’s implementation and expectations. to discover topics, a clustering figure 2). the approach highlights a few topics such as the first case of beneficiaries, the claim for childbirth in 'haryana' (one of the states of the republic of india), the aim to protect and provide schemes that serve as financial pillars for a million people, comprehensive primary care, path-breaking efforts, good employment opportunities, the 'public health cadre' as a new service sector for quality patient-centered primary healthcare services, a vision for their future in remote rural india, exempted reinsurance services (coverage up to ₹5 lakh per family per year to people over 10), reforming healthcare, and call centers for prime minister jan arogya yojna. (figure 3). there are few key challenges discussed in twitter conversations – efficient coordination between the central and state government and insurance through other schemes such as maternal and child survival programs. therefore, the clusters help in identifying the discussion related to various aspects of ‘ayushman bharat’ based on word occurrence. australasian journal of information systems rathore, maurya & srivastava 2021, vol 25, research on user involvement do policymakers use social media 12 figure 3. word cloud for ‘ayushman bharat’-related discussion figure 4. thematic analysis to use these topics, a word network clustering approach was used to highlight the themes associated with 'ayushman bharat' (figure 4). there are a few clusters connected to each other. for instance, the first cluster has the center word 'ayushmannha' with single connected words such as 'mohfw_india', 'pmoindia', 'nitiaayog', and 'union minister' and various individual stakeholders who are associated with the healthcare sector such as the minister of health and family welfare, the government of india, industry leaders such as the president of reliance australasian journal of information systems rathore, maurya & srivastava 2021, vol 25, research on user involvement do policymakers use social media 13 industries limited (ril), top academic medical journals such as 'the lancet' and medical professionals. thus, stakeholders from diverse backgrounds are involved in covering multidimensional perspectives. overall, the findings highlight various themes such as the financial benefits to people, treatments, claim amounts, different stakeholders, reform efforts to restructure healthcare, execution, and fraud detection mechanisms (e.g., digitization). there is also one dimension of the treatment theme that includes critical diseases, modern facilities through globalization, and robust processes. 5.2 sentiment analysis there are more positive scores (approximately 65%) associated with multiple topics. trust has the highest score, suggesting that citizens are expecting more benefits from the scheme (figure 5). along with anticipation, joy emotion has the second-highest score, showing citizens’ faith in the government. thus, this shows that citizens have considerable expectations and, therefore, are looking forward to more responsiveness from the government. fear and surprise also have noticeable scores that need to be addressed by policymakers. the surprise score indicates the presence of unexpected benefits such as health insurance coverage of up to rs. 500,000 per family per year. however, there are few cases in which people did not obtain benefits because of a lack of awareness. due to this, there is sadness due to the lack of awareness mechanism from the government. figure 5. emotion classification with frequency as mentioned above, we further categorized all the results into major classes reflecting the positive and negative perspectives of the scheme from the citizens' point of view. the positive results show that the scheme is considered to be a bold and flawless effort to reform the healthcare sector in india (figure 6). for instance, female childbirth is the first beneficial case under this scheme in the state of haryana, supporting the role the scheme can play in addressing critical social issues such as the gender ratio. the response of the citizens indicates that the scheme strengthens the vision of developing a real partnership with the patients/beneficiaries. most of the citizens consider the scheme robust and perceive it as a landmark of having modern facilities and expert service, which was out of reach for the majority of the citizens. similarly, citizens are seeking the scheme’s rapid acceptance and high australasian journal of information systems rathore, maurya & srivastava 2021, vol 25, research on user involvement do policymakers use social media 14 reach to make it much more impactful. in the initial phase of its implementation, the scheme was perceived as having a high potential to have an impact on the healthcare sector. figure 6. word cloud for positive words however, concerns were also expressed, which could be considered the negative side of the scheme (figure 7). the major concern is the current information about the scheme and its accessibility. the scheme covers approximately half a billion people, which itself is a record number with treatment coverage. therefore, first, a framework for the implementation of the scheme is required. second, public systems need a safe and secure information infrastructure for the digitalization of all processes, i.e., cashless, paperless, etc. in summary, the scheme has significant benefits for approximately half a billion people but building the infrastructure to implement it is also a major challenge for the government. figure 7. word cloud for negative words another major concern is related to the transparency of the different processes. a lack of information about the type of diseases (e.g., cancer) covered under the scheme may restrict access to care. another concern regards the charges for drugs and drug corruption. another concern that was raised regards the coverage and financing of the scheme which covers only australasian journal of information systems rathore, maurya & srivastava 2021, vol 25, research on user involvement do policymakers use social media 15 the poor and not all taxpayers. the scheme primarily covers low-income families and is funded through general taxes. low-income families do not contribute to direct taxes but are the primary beneficiaries of the program. concerns were also expressed about additional expenses that would not be covered in the scheme. thus, although respondents appreciated the scheme, they also had some concerns related to its design and implementation. therefore, there is both considerable enthusiasm for the scheme and many concerns. 6. discussion based on the results, the discussion is structured into four subsections. first, we assess the extent to which issues raised at the launch of the 'ayushman bharat' scheme have been addressed by the policy designers/program administrators in successive years. this is achieved using secondary data and expert interviews. second, based on our case study, we discuss the usefulness of sm data in co-design. finally, we discuss the value of sm in policy co-design during policy formulation in the country context of india. 6.1 policy co-design: matching public opinion with evolution in program design the results presented above identify positive and negative sentiments reflecting the appreciations and concerns regarding the scheme at the time of its launch. in this section, we track the progress on the issues identified by experts over the years using primary and secondary data. we assess the extent to which program designers have addressed the concerns expressed by citizens at the time of launch of the scheme (refer to table 1 and table 2). positive sentiments: the positive sentiments about the scheme were primarily related to the scale and scope of the scheme. for all the positive topics, the data also include multiple positive words such as ‘thank’, ‘honor’, ‘proud’, ‘golden’, ‘success’, ‘wow’, ‘huge’, ‘bless’, ‘good’, ‘bold’, etc. (table 1). the past schemes in india covered limited population groups, and the depth of the coverage was also minimal, as stated by various analysts (asher et al., 2015; prinja et al., 2017). compared to other past schemes such as rsby, the ‘ayushman bharat’ scheme is considered to be very ambitious as it covers approximately ten times the existing coverage of various schemes, and the depth of the coverage is increased by approximately 15 times. this is evident with twitter users referring to the scheme as ‘path-breaking’ and ‘visionary’. apart from the general public or beneficiaries, the scheme also received enthusiastic responses from private sector companies and hospitals as the scheme was launched in partnership with them. the scheme also received great support from various international development agencies, public health experts, and academic journals as they genuinely feel that the scheme can transform the indian healthcare sector and address the vicious cycle of poverty and poor health. the former director-general of the who, bruntland (2018), noted in an article in ‘the lancet’ that “the reforms are to be welcomed because for decades india has underinvested in health” (pg. 1174). several newspapers also lauded the scheme as one of the world's largest public healthcare schemes (bbc news, 2018). respondents also seem to be excited about the scheme, desire to scale it up to the next level, and expect a positive impact on the health system. as the scheme's focus is rural, it is also likely to increase the supply of healthcare facilities and improve the quality of health services in rural areas. the scheme impanels both public and private hospitals, and, therefore, the competition between them would probably also lead to reforms in the public sector. additionally, it would also facilitate employment in rural areas. australasian journal of information systems rathore, maurya & srivastava 2021, vol 25, research on user involvement do policymakers use social media 16 negative topics positive topics prevent fraud and fake cases such as abortions low pain in the process of the scheme focus on critical diseases such as cancer the danger of oppression should be avoided ~ neglect death plight symptom of drug corruption limited payment of the extra fee for the mortal bearer concern regarding the usage of taxpayers’ money only for nontaxpayers fees are harmful young angel child ~ the birth of a baby girl as the first beneficiary under this scheme new strengthen vision ~ rapid acceptance with a high reach a flawless and bold effort to reform care join the government to make it smart, superb, and better and boost proudness support patient-partner model ~ adopt real partnership concept dream, enthusiast, robust landmark for citizens core global interest for talks the scheme addresses developing/detecting cures by professional doctors blessed with modern facilities and experts excellent model table 1. positive and negative topics of ‘#ayushmanbharat’ tweets however, after two years of implementation, the high expectations from the scheme seem to be fading away. the budgetary allocation for the ‘ayushman bharat’ scheme over the years has been argued to be grossly inadequate (chatterjee, 2019a; ghosh and quadeer, 2019; chatterjee, 2019b). in 2018-19, the budget allocation for pmjay was 20000 million inr and was increased to 64000 million, which was much less than what was required (420.73 billion) to fund the minimum scale and depth of the coverage that the scheme promises (ghosh & quadeer, 2019). two years after the launch of the scheme, many analysts have termed the budget allocation to be a token gesture (chatterjee, 2019a; ghosh & quadeer, 2019). concerns 1. issues observed in previous programs other than pmjay 2. lack of focus on quality 3. providers supply induced demand and fraud 4. limited reduction in the healthcare expenditures of beneficiaries 5. concerns about pmjay as per the tweets related to pmjay 6. process issues 7. fraud and drug corruption 8. extra fees from beneficiaries 9. status after 2 years of program implementation 10. not adequately addressed 11. limited evidence to either support or reject 12. not adequately addressed table 2. citizens’ concerns and the status of the program’s implementation (from the experts) australasian journal of information systems rathore, maurya & srivastava 2021, vol 25, research on user involvement do policymakers use social media 17 although many people have benefited from similar schemes, there are consistent concerns that remain unaddressed. for example, although access to hospital care and utilization among lowincome families has increased, the current scheme, like past schemes, has had a limited impact on the health expenditures of the target population (prinja et al., 2017; maurya & ramesh, 2019). earlier analysts argued extensively about the leakages and wastes generally observed in the health insurance model of providing hospitalization care, which can lead to a significant increase in healthcare costs. these concerns were both raised by experts (brundtland, 2018; ghosh & quadeer, 2019) and government-commissioned reports (planning commission, 2011). two years after the launch of the scheme, fraud by hospitals continues to plague the scheme; however, the government remains optimistic about their ability to control the adverse effects of the model (niti aayog goi, 2019). respondents also expressed disappointment as the scheme did not cover primary care or outpatient-based care (opd care). the importance of covering opd care has been emphasized by various experts’ reports (planning commission, 2011; la forgia & nagpal, 2012), and there have been some experiments by previous schemes such as rsby (rsby newsletter, 2014). during in-depth interviews, the vice president of one of the leading insurance companies in india commented that “in opd care the ticket price is very small whereas operational costs are high. insuring opd care means a vast number of claims, and therefore checking for fraud will be very difficult”. although there has been some discussion so far, primary care or opd care remains uncovered in the ‘ayushman bharat’ scheme. negative topics the negative topics raised in the tweets are related to the design of the program (table 1). our discussion with the experts revealed that hospitals have the autonomy to select treatment packages, and, therefore, they have an incentive to choose packages that are higher paying. given the lack of standard treatment guidelines and a large volume of cases, hospitals engage in fraudulent behavior by making fake claims or claiming much higher amounts, as observed in the previous schemes. similar concerns have also been raised by various experts and commentators (prinja et al., 2017; maurya & ramesh, 2019). in-depth interviews with the experts suggested that this is one of the main concerns in the design of the scheme. the former director-general of who, brutland (2018), noted in an article in ‘the lancet’ that “india’s poorly regulated private hospitals will be only too pleased to meet this demand, but as previous hospital insurance schemes have shown in india, this could result in people being given expensive diagnostic and surgical procedures they don’t need.” (pg. 1175) the national office of the world health organization, india office, also noted his apprehension about the “limited capacity amongst indian states in identification and enrolment of beneficiaries, designing the benefit package, fixing the package rate, empanelment of facilities, monitoring and regulation and fraud detection” (laharia, 2018, pg. 495). the implementing agency, the national health authority of india (nhai), seems to be conscious of this concern and has made efforts to address this concern, including seeking technical expertise from various agencies (economic times, 2018). however, it is too early to comment on the effectiveness of the measures taken to control fraud. by may 8, 2020, 164 hospitals had been de-empaneled across the country due to engaging in fraudulent behavior. however, our interviews with hospital managers suggested that more prominent hospitals were empaneled in the program because of pressure from the government. nevertheless, they do not accept these patients as the rates offered in the pmjay scheme are much lower. australasian journal of information systems rathore, maurya & srivastava 2021, vol 25, research on user involvement do policymakers use social media 18 on the other hand, smaller hospitals use this as an opportunity to make profits by engaging in fraudulent behavior. fraudulent behavior not only occurs during hospitalization, but it also may emerge during the enrollment of beneficiaries in terms of issuing fake cards that can be later used to make claims in collusion with hospitals without providing any services. responding to a question in the upper house of parliament, the state minister of ministry of health and family reported that out of 120 million cards issued so far, 71027 cards were found to be fraudulent with the claimed amount of around rs. 4.5 million (government of india, 2020). to what extent these efforts have been successful is difficult to ascertain because no data exist on the extent of fraudulent cards in other health insurance programs. the findings also suggest that a few issues in the scheme design have also existed in previous schemes and continue in the present design. as per the scheme design, there is no copayment in the scheme. beneficiaries can receive treatment at the empaneled hospital without paying as all components of care-bed charges, medicines, investigation, and food are included in the package. due to limited awareness among beneficiaries about the package details, this gets exploited by hospitals as they collect additional monies for various services, as observed in other schemes (maurya & ramesh, 2019). another way that hospitals collect monies from beneficiaries is when beneficiaries are unsure of the reimbursement from insurance companies. insurance companies tend to reject claims on trivial technical issues to maintain profits, as found in other schemes (maurya & srivastava, 2019). anticipating this rejection, hospitals tend to collect money from beneficiaries before providing treatment, although the program design expects the treatment to be free at the time that the service is consumed. third, in other schemes, extensive delays in insurance companies reimbursing providers have been observed (maurya, 2018). delays, coupled with the probability of rejection make hospitals unsure whether they will get paid for their claims, and, therefore, they collect money from the beneficiaries. public health expert sulakshana nandi, the national joint convener of the jan swasthya abhiyan, one of the most vocal ngos in the health sector, commented that “due to power asymmetry and the absence of adequate regulation and grievance redressal systems private hospitals end up taking co-payments or making patients buy medicines, both of which are illegal under these schemes” (chatterjee, 2019a, pg. 491). such concerns have been raised by the people at the time of the launch of the scheme, but the evidence regarding the efforts that have been made to control this is not clear. reports from the ‘ayushman bharat’ suggest that most grievances are still about the collection of additional monies (national health authority, 2019). the beneficiaries have a limited voice as they can only lodge a complaint in the online grievance system or by calling the helpline. the widespread illiteracy in rural areas suggests that this is not an easy task for most of the beneficiaries. though the beneficiary can choose the hospital, once the beneficiary has consented to be treated by a hospital and his/her card has been swiped, he/she cannot change hospitals. there is also a discussion of the concerns related to coverage of the scheme. ‘ayushman bharat’ has substantially increased the depth of coverage compared to previous schemes. for example, rsby only covered expenditures up to 395 usd per family per year, which was grossly insufficient for many types of care, such as cancer treatment and cardiac surgeries, both common in india, even among low-income families. ‘ayushman bharat’ covers expenditures up to 6590 usd, which is enough for treatment in these cases. however, as all these expenditures are financed through general tax revenues and targeted to low-income australasian journal of information systems rathore, maurya & srivastava 2021, vol 25, research on user involvement do policymakers use social media 19 families, this will increase the tax burden on high-income families given the very low tax base in india, as only 3.8% of the population paid income taxes in 2018-19 (economic times, 2020). this challenge needs to be recognized among policymakers as the general tax financing of health insurance in a country context with lower levels of formal employment such as india may sound like a pragmatic approach; however, it may lead to issues related to the solidarity among society. final thoughts public participation in policy design may result in diverse perspectives and conflicting goals and require trade-offs (komor & bazilian, 2005) if there is wider participation. however, it may bias the data if there are certain population segments and political activism (medglia & zheng, 2017). this challenge may be further heightened if the data are collected through sm due to ingroup-outgroup bias (tajfel & turner, 2004; hogg & reid, 2006), where people with similar beliefs may organize themselves in groups amplifying and re-enforcing each other’s beliefs while belittling other groups that have different beliefs. the findings include the analysis of tweets from various groups of the population: common citizens, government officials and agencies, magazines, newspapers, academic journals, and healthcare stakeholders. government officials and agencies are expected to express positive sentiments and therefore represent one group with a positive belief system. however, magazines, newspapers, academic journals, and healthcare stakeholders tend to highlight the gaps that existed in previous schemes and therefore represent other groups with another belief system. we found that negative tweets were retweeted more often than positive tweets, creating a reinforcement and amplification of the belief system and thus presenting the possibility of intergroup bias, the dark side of sm analysis. sm platforms are also considered to be effective and convenient tools for fake news, misinformation, and misleading directions. considering these anomalies, it becomes very critical to incorporate citizens’ opinions generated on various sm platforms. wrong information can be taken as the most important piece of news because of the manipulation occurring on sm platforms. one of the observations from the dataset highlights that people share (e.g., retweet) more tweets that contain more negative sentiments. with this trend, citizen engagement can be derived in the direction of destroying the actual benefits of the scheme. in all of the abovementioned online engagement disasters, various agencies monitor citizens' activities on sm, disgruntled citizens damage the public reputation when they rant about their experiences, other users bully each other, and misinformation and aggressive social engagement cause social tension. however, despite the many knowledge extraction benefits sm platforms offer, they are not immune to these relational ills either. 6.2. theoretical implications table 3 compares the public’s views and experts' views on the key concerns related to the scheme. the results of our analysis interestingly show that the issues raised by the public are very close to those of experts. this finding contradicts some of the literature strands on policymaking processes. for example, weisberg (2001) argued that policymakers should ignore public opinion as it reflects only wishes and preferences and no costs or risks associated with policy criteria; therefore, public opinion cannot coincide with the required policy choices. it is also argued that efforts to engage a sizeable public base in policy decisions are not fruitful as diverse perspectives may result in conflicting goals and require trade-offs (komor & bazilian, australasian journal of information systems rathore, maurya & srivastava 2021, vol 25, research on user involvement do policymakers use social media 20 2005). further representativeness of all population segments and political activism can bias the data (medglia & zheng, 2017). public view expert view strengthen vision a flawless and bold effort modern facilities and experts prevent fraud and fake cases transparency correct information financial protection welcomed reform in underinvested healthcare lauded as the largest public healthcare scheme no opd care to void large claims and fraud check hospital picking high-cost treatment package hospitals’ fraudulent behavior table 3. comparison of the public’s and experts’ views on the ‘ayushman bharat’ scheme however, our analysis suggests that public opinion to a large extent matched the experts’ opinions. we also performed user analysis to assess the representativeness of our sample. apart from common citizens, our sample included several professional categories. the number of tweets from common citizens suggests that our sample represents public opinion. although we can expect that government officials and agencies are expected to express positive sentiments, magazines, newspapers, and academic journals probably have more critical reviews of the scheme. furthermore, to understand the reasons for the match between experts’ opinion and the public’s opinion, we returned to the policy experts and executives who had significant experience and exposure to policymaking in india. it is found that health policy in india has not historically been a core political discourse compared to several other social issues such as caste and reservation, religion and state, foreign direct investment and economic self-reliance, etc. (brass, 1994; bayly, 2001; dasgupta & qadeer, 2005; sachan, 2014). for instance, a close examination of the election manifesto of the political parties in india shows little mention of their health policy orientations, resulting in very little political debate on the issue compared to other issues (bhuyan, 2019; chatterjee, 2019b). only recently have differences in political viewpoints regarding policies related to health in india emerged, as observed in the case of ‘ayushman bharat’ between two dominant political parties (chatterjee, 2019b). usually, most political parties have similar viewpoints about healthcare in india (kumar et al., 2017) and possibly do not see healthcare as a reason to differentiate themselves from other political parties; therefore, healthcare never becomes an essential component of election campaigns (ahemed et al., 2016). therefore, as health policy has never been a popular election topic in india, only informed citizens take an interest in health policy and comment on different platforms. 6.3 managerial implications the analysis suggests that managers in public agencies should use sm to primarily inform and promote policies and programs. public managers can significantly improve the design of programs and service delivery by routinely analyzing the feedback on sm. as also confirmed by the practitioners, health policy, due to being a politically non-controversial issue, does not very often attract extreme views. from a political perspective, the noncontroversial nature of healthcare in india results in a more rational debate rather than an emotional discussion. the australasian journal of information systems rathore, maurya & srivastava 2021, vol 25, research on user involvement do policymakers use social media 21 continuity of the same policies during different political regimes also suggests that political ideology plays a limited role in health policy formulation in india. however, policymakers need to innovatively use online platforms including sm considering the social context. for example, the reports and public opinion suggest that most of the grievances on ‘ayushman bharat’ still regard the collection of additional monies. the beneficiaries have limited voices as they can only lodge complaints in the online grievance system or by calling the helpline. however, due to the widespread illiteracy in rural areas, this is not an easy task for most citizens. policy designers may have to be more innovative to ease the use of online platforms despite a lack of a quick fix for social issues such as illiteracy. additionally, even though sm helped obtain feedback from citizens on broader issues, granularity in those analyses is still missing. for example, opd care constitutes a significant portion of out-of-pocket expenditures in india and therefore contributes significantly to the effect of improving healthcare expenditures (garg & karan, 2009). however, the complexity of ensuring opd care under insurance has been a significant challenge. obviously, the social backwardness of the beneficiaries and high information asymmetry makes it difficult for the beneficiaries to engage in a granular discussion. therefore, policymakers and sm platform designers need to be innovative to respond to the dual responsibility of educating citizens and helping them engage in critical discussions. 6.4 limitations and future directions there are certain limitations of this study, and, therefore, the inferences from this study should be used with caution for the following reasons. firstly, the data might not be the most accurate representative due to limitations of the effectiveness of how public data is extracted from twitter. future research could consider the development of an advanced text analytics technique to analyze all language tweets from a single dataset. secondly, we attribute changes in the program design over the years to sma; however, the program designers may have been influenced by the knowledge gained from other sources. future research on policy co-design requires collecting sm data over longer periods and on the extent to which policy design agencies such as nhai use these sm data. given the potential of sma, policy design, and codesign processes can be effective even in developing country context, where the most common approach for governments to consider policy feedback remains limited to elections. 7. conclusion in the indian context, sm is increasingly used by citizens to express their opinion and public agencies to communicate information (gul et al., 2017; paruthi et al., 2020). however, its use has been primarily limited to one-way communication in a strategic political way. our analysis suggests that although public agencies use sm to address grievances, their use of sm to engage stakeholders online to improve the design of public policy was limited. tracing the evolution of program design indicated that there had been some changes in the program design over the last two years which addressed the concerns raised by citizens at the time of launch of the program. however, most of the critical issues remain unaddressed until today. given the activism on sm by nha, the agency responsible for improving the design of ‘ayushman bharat’, it can be inferred that sma could play an essential role in enhancing policy design. nevertheless, at present, sm continues to be primarily used for addressing grievances related to service delivery, as also done by various other government agencies. this could be considered as a rational approach to information use as per the ‘habermas theory of communication action’. using sma for policy design requires a communicative approach australasian journal of information systems rathore, maurya & srivastava 2021, vol 25, research on user involvement do policymakers use social media 22 that uses the information to create discussion groups on critical issues and consider different viewpoints of the stakeholders. our findings also suggest that contrary to expectations, citizens' opinions and experts' views were very similar despite the complex and technical nature of the scheme. this could be due to better awareness of the scheme among citizens due to their experience with similar schemes in the past and extensive propagation of information on the scheme in media, including sm. references agarwal, s., & sureka, a. (2017, december). investigating the role of twitter in e-governance by extracting information on citizen complaints and grievances reports. in international conference on big data analytics (pp. 300-310). springer, cham. aggarwal, c. c., & subbian, k. (2012, april). event detection in social streams. in sdm (vol. 12, pp. 624-635). ahmed, s., cho, j., & jaidka, k. (2017). leveling the playing field: the use of twitter by politicians during the 2014 indian general election campaign. telematics and informatics, 34(7), 1377-1386. ahmed, w., & lugovic, s. (2019). social media analytics: analysis and visualisation of news diffusion using nodexl. online information review. 43 (1), 149-160. akrout, h., & diallo, m. f. (2017). fundamental transformations of trust and its drivers: a multi-stage approach of business-to-business relationships. industrial marketing management, 66, 159-171. aleti, t., harrigan, p., cheong, m., & turner, w. (2016). an investigation of how the australian brewing industry influence consumers on twitter. australasian journal of information systems, 20. https://doi.org/10.3127/ajis.v20i0.1350 alizadeh, t., sarkar, s., & burgoyne, s. (2019). capturing citizen voice online: enabling smart participatory local government. cities, 95, 102400. https://doi.org/10.1016/j.cities. 2019.102400 androutsopoulou, a., & charalabidis, y. (2018, april). a framework for evidence based policy making combining big data, dynamic modelling and machine intelligence. in proceedings of the 11th international conference on theory and practice of electronic governance (pp. 575583). aquilino, w. s., & wright, d. l. (1996). substance use estimates from rdd and area probability samples: impact of differential screening methods and unit nonresponse. public opinion quarterly, 60(4), 563-573. arnold, r. d. (1990). the logic of congressional action. yale university press. asher, m., vora, y., & maurya, d. (2015). an analysis of selected pension and health care initiatives for informal sector workers in india. social policy & administration, 49(6), 738-751. aswani, r., kar, a. k., & ilavarasan, p. v. (2018a). detection of spammers in twitter marketing: a hybrid approach using social media analytics and bio inspired computing. information systems frontiers, 20(3), 515-530. https://doi.org/10.3127/ajis.v20i0.1350 australasian journal of information systems rathore, maurya & srivastava 2021, vol 25, research on user involvement do policymakers use social media 23 aswani, r., kar, a. k., ilavarasan, p. v., & dwivedi, y. k. (2018b). search engine marketing is not all gold: insights from twitter and seoclerks. international journal of information management, 38(1), 107-116. bali, a. s., & ramesh, m. (2015). mark time: india's march to universal health care coverage. social policy & administration, 49(6), 718-737. barbosa, l., & feng, j. (2010, august). robust sentiment detection on twitter from biased and noisy data. in coling 2010: posters (pp. 36-44). bayly, s. (2001). caste, society and politics in india from the eighteenth century to the modern age (vol. 3). cambridge university press. bbc news, (2018). india unveils ‘world’s largest’ public healthcare scheme. available on 22 june 2019, retrieved from http://www.bbc.com/news/world-asia-india-42899402. becker, g. s. (1983). a theory of competition among pressure groups for political influence. the quarterly journal of economics, 98(3), 371-400. beer, d., & burrows, r. (2007). sociology and, of and in web 2.0: some initial considerations. sociological research online, 12(5), 67-79. bekkers, v., edwards, a., & de kool, d. (2013). social media monitoring: responsive governance in the shadow of surveillance?. government information quarterly, 30(4), 335342. berry, j. m., & wilcox, c. (2018). the interest group society. routledge. bertot, j. c., jaeger, p. t., & grimes, j. m. (2010). using icts to create a culture of transparency: e-government and social media as openness and anti-corruption tools for societies. government information quarterly, 27(3), 264-271. bhuyan, a. (2019). in india, a tight election—with healthcare trailing the field. bmj: british medical journal (online), 365. https://doi.org/10.1136/bmj.l2175 blomkamp, e. (2018). the promise of co‐design for public policy. australian journal of public administration, 77(4), 729-743. boubakri, n., cosset, j. c., & saffar, w. (2008). political connections of newly privatized firms. journal of corporate finance, 14(5), 654-673. brabham, d. c. (2012). the myth of amateur crowds: a critical discourse analysis of crowdsourcing coverage. information, communication & society, 15(3), 394-410. brass, p. r. (1994). the politics of india since independence (vol. 1). cambridge university press. brundtland, g. h. (2018). india’s health reforms: the need for balance. http://dx.doi.org/10.1016/ s0140-6736(18)32387-0 burger, a., & silima, t. (2006). sampling and sampling design. journal of public administration, 41(3), 656-668. cao, q., thompson, m. a., & yu, y. (2013). retracted: sentiment analysis in decision sciences research: an illustration to it governance. decision support systems, 54(2), 10101015. https://doi.org/10.1136/bmj.l2175 australasian journal of information systems rathore, maurya & srivastava 2021, vol 25, research on user involvement do policymakers use social media 24 cataldi, m., di caro, l., & schifanella, c. (2010, july). emerging topic detection on twitter based on temporal and social terms evaluation. in proceedings of the tenth international workshop on multimedia data mining (pp. 1-10). chae, b. k. (2015). insights from hashtag# supplychain and twitter analytics: considering twitter and twitter data for supply chain practice and research. international journal of production economics, 165, 247-259. chatterjee, p. (2019a). modi's health reforms: between hope and hype. the lancet, 394(10208), 1495-1498. chatterjee, p. (2019b). health debate rising around the upcoming indian election. available on dec 2019, retrieved from https://dl.uswr.ac.ir/bitstream/hannan/80065/1/2019%20lancet%20volume%20393%20 issue%2010180%20april%20%2831%29.pdf chau, m., & xu, j. (2012). business intelligence in blogs: understanding consumer interactions and communities. mis quarterly, 36(4), 1189-1216. chun, s. a., & luna-reyes, l. f. (2012). social media in government. government information quarterly, 29(4), 441-445. corcoran, n., & duane, a. (2018). using social media to enable staff knowledge sharing in higher education institutions. australasian journal of information systems, 22. https://doi.org/10.3127/ajis.v22i0.1647 criado, j. i., sandoval-almazan, r., & gil-garcia, j. r. (2013). government innovation through social media. government information quarterly, 30(4), 319-326. curiskis, s. a., drake, b., osborn, t. r., & kennedy, p. j. (2020). an evaluation of document clustering and topic modelling in two online social networks: twitter and reddit. information processing & management, 57(2), 102034. cyr, d., head, m., larios, h., & pan, b. (2009). exploring human images in website design: a multi-method approach. mis quarterly, 539-566. dasgupta, r., & qadeer, i. (2005). the national rural health mission (nrhm): a critical overview. indian j public health, 49(3), 138-40. downs, a. (1957). an economic theory of democracy. new york: harper. economic times, (2018, may). “niti aayog seeks world bank's help to prevent fraud in health insurance scheme”. available on 12 oct 2019, retrieved from https://economictimes.indiatimes.com/news/politics-and-nation/niti-aayog-seeksworld-banks-help-to-prevent-fraud-in-health-insurance-scheme/articleshow/ 64051120.cms? from=mdr eom, s. j., hwang, h., & kim, j. h. (2018). can social media increase government responsiveness? a case study of seoul, korea. government information quarterly, 35(1), 109-122. eze, e., gleasure, r., & heavin, c. (2016). how can mhealth applications that are developed in one area of the developing world be adapted for use in others?. journal of decision systems, 25(sup1), 536-541. https://dl.uswr.ac.ir/bitstream/hannan/80065/1/2019%20lancet%20volume%20393%20issue%2010180%20april%20%2831%29.pdf https://dl.uswr.ac.ir/bitstream/hannan/80065/1/2019%20lancet%20volume%20393%20issue%2010180%20april%20%2831%29.pdf https://doi.org/10.3127/ajis.v22i0.1647 https://economictimes.indiatimes.com/news/politics-and-nation/niti-aayog-seeks-world-banks-help-to-prevent-fraud-in-health-insurance-scheme/articleshow/%2064051120.cms?%20from=mdr https://economictimes.indiatimes.com/news/politics-and-nation/niti-aayog-seeks-world-banks-help-to-prevent-fraud-in-health-insurance-scheme/articleshow/%2064051120.cms?%20from=mdr https://economictimes.indiatimes.com/news/politics-and-nation/niti-aayog-seeks-world-banks-help-to-prevent-fraud-in-health-insurance-scheme/articleshow/%2064051120.cms?%20from=mdr australasian journal of information systems rathore, maurya & srivastava 2021, vol 25, research on user involvement do policymakers use social media 25 falco, e., & kleinhans, r. (2019). digital participatory platforms for co-production in urban development: a systematic review. in crowdsourcing: concepts, methodologies, tools, and applications (pp. 663-690). igi global. feldman, r. (2013). techniques and applications for sentiment analysis. communications of the acm, 56(4), 82-89. fensel, d., leiter, b., & stavrakantonakis, i. (2012). social media monitoring. semantic technology institute, innsbruck, 16. available on 15 june 2018, retrieved from https://oc.sti2.at/sites/default/files/smm%20handouts.pdf fredrickson, b. l., & losada, m. f. (2005). positive affect and the complex dynamics of human flourishing. american psychologist, 60(7), 678–686. https://doi.org/10.1037/0003066x.60.7.678 garg, c. c., & karan, a. k. (2009). reducing out-of-pocket expenditures to reduce poverty: a disaggregated analysis at rural-urban and state level in india. health policy and planning, 24(2), 116-128. ghosh, s. m., & qadeer, i. (2019). pradhan mantri jan arogya yojana: a paper tiger. social change, 49(1), 136-143. gintova, m. (2019). understanding government social media users: an analysis of interactions on immigration, refugees and citizenship canada twitter and facebook. government information quarterly, 36(4), 101388. https://doi.org/10.1016/j.giq.2019.06.005 government of india (2020). “rajya sabha unstarred question no-283” http://164.100.47.5/qsearch/qresult.aspx goyal, m., gupta, n., jain, a., & kumari, d. (2020). smart government e-services for indian railways using twitter. in micro-electronics and telecommunication engineering (pp. 721731). springer, singapore. grindle, m. s., & thomas, j. w. (1991). public choices and policy change: the political economy of reform in developing countries. johns hopkins univ pr. grover, p., kar, a. k., & davies, g. (2018). “technology enabled health”–insights from twitter analytics with a socio-technical perspective. international journal of information management, 43, 85-97. grover, p., kar, a. k., & ilavarasan, p. v. (2019a). impact of corporate social responsibility on reputation—insights from tweets on sustainable development goals by ceos. international journal of information management, 48, 39-52. grover, p., kar, a. k., dwivedi, y. k., & janssen, m. (2019b). polarization and acculturation in us election 2016 outcomes–can twitter analytics predict changes in voting preferences. technological forecasting and social change, 145, 438-460. gul, s., mahajan, i., nisa, n.t., shah, t.a., asifa, j., & ahmad, s. (2016). "tweets speak louder than leaders and masses: an analysis of tweets about the jammu and kashmir elections 2014", online information review, 40(7), 900-912. https://doi.org/10.1108/oir-10-20150330 https://oc.sti2.at/sites/default/files/smm%20handouts.pdf https://doi.org/10.1037/0003-066x.60.7.678 https://doi.org/10.1037/0003-066x.60.7.678 http://164.100.47.5/qsearch/qresult.aspx https://www.emerald.com/insight/search?q=sumeer%20gul https://www.emerald.com/insight/search?q=iram%20mahajan https://www.emerald.com/insight/search?q=nahida%20tun%20nisa https://www.emerald.com/insight/search?q=tariq%20ahmad%20shah https://www.emerald.com/insight/search?q=jan%20asifa https://www.emerald.com/insight/search?q=suhail%20ahmad https://www.emerald.com/insight/publication/issn/1468-4527 https://doi.org/10.1108/oir-10-2015-0330 https://doi.org/10.1108/oir-10-2015-0330 australasian journal of information systems rathore, maurya & srivastava 2021, vol 25, research on user involvement do policymakers use social media 26 gupta, i., chowdhury, s., trivedi, m., & prinja, s. (2017). do health coverage schemes ensure financial protection from hospitalization expenses? evidence from eight districts in india. journal of social and economic development, 19(1), 83-93. habrmas, j. (1984). the theory of communicative action: reason and the rationalization of society (vol. 1). beacon press. habermas, j. (2015). the theory of communicative action: lifeworld and systems, a critique of functionalist reason, volume 2 (vol. 2). john wiley & sons. hacker, j., wickramasinghe, n., & durst, c. (2017). can health 2.0 address critical healthcare challenges? insights from the case of how online social networks can assist in combatting the obesity epidemic. australasian journal of information systems, 21, 1-17. hambrick, d. c., & abrahamson, e. (1995). assessing managerial discretion across industries: a multimethod approach. academy of management journal, 38(5), 1427-1441. hansen, d., shneiderman, b., & smith, m. a. (2010). analyzing social media networks with nodexl: insights from a connected world. morgan kaufmann. he, w., & customer co-creation, the computer journal, 57(12), 1909-1920. hedestig, u., skog, d., & söderström, m. (2018, may). co-producing public value through iot and social media. in proceedings of the 19th annual international conference on digital government research: governance in the data age (pp. 1-10). henry, d., dymnicki, a. b., mohatt, n., allen, j., & kelly, j. g. (2015). clustering methods with qualitative data: a mixed-methods approach for prevention research with small samples. prevention science, 16(7), 1007-1016. hogg, m. a., & reid, s. a. (2006). social identity, self-categorization, and the communication of group norms. communication theory, 16(1), 7-30. holvoet, n., & inberg, l. (2014). taking stock of monitoring and evaluation systems in the health sector: findings from rwanda and uganda. health policy and planning, 29(4), 506516. howlett, m. (2019). procedural policy tools and the temporal dimensions of policy design. resilience, robustness and the sequencing of policy mixes. international review of public policy, 1(1: 1), 27-45. howlett, m., mukherjee, i., & woo, j. j. (2015). from tools to toolkits in policy design studies: the new design orientation towards policy formulation research. policy & politics, 43(2), 291-311. huh, j., yetisgen-yildiz, m., & pratt, w. (2013). text classification for assisting moderators in online health communities. journal of biomedical informatics, 46(6), 998-1005. irda. annual report, (2016). insurance regulatory and development authority of india (2016), new delhi. available on 5 sep 2018, retrieved from http://dhc.co.in/uploadedfile/1/2/1/irdai%20%20(health%20insurance)%20regulations%202016.pdf jones, b. d. (1994). reconceiving decision-making in democratic politics: attention, choice, and public policy. university of chicago press. khawaja, s. (2011). good governance and result based monitoring. poorab academy, islamabad. http://comjnl.oxfordjournals.org/content/current http://comjnl.oxfordjournals.org/content/current http://dhc.co.in/uploadedfile/1/2/-1/irdai%20%20(health%20insurance)%20regulations%202016.pdf http://dhc.co.in/uploadedfile/1/2/-1/irdai%20%20(health%20insurance)%20regulations%202016.pdf australasian journal of information systems rathore, maurya & srivastava 2021, vol 25, research on user involvement do policymakers use social media 27 king, s. c., meiselman, h. l., & carr, b. t. (2013). measuring emotions associated with foods: important elements of questionnaire and test design. food quality and preference, 28(1), 8-16. komor, p., & bazilian, m. (2005). renewable energy policy goals, programs, and technologies. energy policy, 33(14), 1873-1881. ku, l. w., liang, y. t., & chen, h. h. (2006, march). opinion extraction, summarization and tracking in news and blog corpora. in aaai spring symposium: computational approaches to analyzing weblogs (vol. 100107, pp. 1-167). kumar, a., & sharma, a. (2017). systematic literature review on opinion mining of big data for government intelligence. webology, 14(2). available on 14 april 2018, retrieved from https://www.webology.org/2017/v14n2/a156.pdf kumar, d., kumar, r., chauhan, r., chander, v., & raina, s. k. (2017). pattern of health promises for indian democracy: a qualitative review of political manifestos. journal of family medicine and primary care, 6(3), 455-459. la forgia, g., & nagpal, s. (2012). government-sponsored health insurance in india: are you covered?. the world bank. available on 27 july 2018, retrieved from http://crossasiarepository.ub.uni-heidelberg.de/3506/1/goverment%20sponsored%20health%20insur ance%20india.pdf lahariya, c. (2018). ‘ayushman bharat’program and universal health coverage in india. indian pediatrics, 55(6), 495-506. lau, r. y., li, c., & liao, s. s. (2014). social analytics: learning fuzzy product ontologies for aspect-oriented sentiment analysis. decision support systems, 65, 80-94. lim, w. m., lim, a. l., & phang, c. s. c. (2019). toward a conceptual framework for social media adoption by non-urban communities for non-profit activities: insights from an integration of grand theories of technology acceptance. australasian journal of information systems, 23. https://doi.org/10.3127/ajis.v23i0.1835 lin, h.-f. (2014). a multi-stage analysis of antecedents and consequences of knowledge management evolution. journal of knowledge management, 18(1), 52-74. lindaman, k., & haider-markel, d. p. (2002). issue evolution, political parties, and the culture wars. political research quarterly, 55(1), 91-110. linders, d. (2012). from e-government to we-government: defining a typology for citizen coproduction in the age of social media. government information quarterly, 29(4), 446-454. liu, x., zhang, w. j., tu, y. l., & jiang, r. (2008). an analytical approach to customer requirement satisfaction in design specification development. engineering management, ieee transactions on, 55(1), 94-102. loukis, e., & charalabidis, y. (2015). active and passive crowdsourcing in government. in policy practice and digital science (pp. 261-289). springer, cham. loukis, e., charalabidis, y., & androutsopoulou, a. (2017). promoting open innovation in the public sector through social media monitoring. government information quarterly, 34(1), 99-109. https://www.webology.org/2017/v14n2/a156.pdf http://crossasia-repository.ub.uni-heidelberg.de/3506/1/goverment%20sponsored%20health%20insur%20ance%20india.pdf http://crossasia-repository.ub.uni-heidelberg.de/3506/1/goverment%20sponsored%20health%20insur%20ance%20india.pdf http://crossasia-repository.ub.uni-heidelberg.de/3506/1/goverment%20sponsored%20health%20insur%20ance%20india.pdf https://doi.org/10.3127/ajis.v23i0.1835 australasian journal of information systems rathore, maurya & srivastava 2021, vol 25, research on user involvement do policymakers use social media 28 malsbender, a., hoffmann, s., & becker, j. (2014). aligning capabilities and social media affordances for open innovation in governments. australasian journal of information systems, 18(3). https://doi.org/10.3127/ajis.v18i3.1100 mathauer, inke, and guy carrin. 2011. ‘the role of institutional design and organizational practice for health financing performance and universal coverage’. health policy 99 (3): 183–192. maurya, d. (2018). contracting out: making it work. policy design and practice, 1(4), 281-297. maurya, d., & ramesh, m. (2019). program design, implementation and performance: the case of social health insurance in india. health economics, policy and law, 14(4), 487-508. maurya, d., & srivastava, a. k. (2019). managing partner opportunism in public–private partnerships: the dynamics of governance adaptation. public management review, 21(10), 1420-1442. medaglia, r., & zheng, l. (2017). mapping government social media research and moving it forward: a framework and a research agenda. government information quarterly, 34(3), 496-510. melville, p., sindhwani, v., & lawrence, r. (2009). social media analytics: channeling the power of the blogosphere for marketing insight. proc. of the win, 1(1), 1-5. meyer, a. d., & goes, j. b. (1988). organizational assimilation of innovations: a multilevel contextual analysis. academy of management journal, 31(4), 897-923. miah, s. j., hasan, n., hasan, r., & gammack, j. (2017). healthcare support for underserved communities using a mobile social media platform. information systems, 66, 1-12. mitchell, a., mahal, a., & bossert, t. (2011). healthcare utilisation in rural andhra pradesh. economic and political weekly, 15-19. https://www.jstor.com/stable/27918082 morgan, k., cheong, m., & bedingfield, s. (2018). “power to the people!”: social media discourse on regional energy issues in australia. australasian journal of information systems, 22. https://doi.org/10.3127/ajis.v22i0.1678 national health authority of india (2019). annual report 2018-19. available on 16 dec 2019, retrieved from https://main.mohfw.gov.in/basicpage/annual-report-2018-19 national sample survey office (2015). “key indicators of social consumption in health nss 71st round.” national sample survey organization. available on 19 jan 2018, retrieved from http://www.mospi.gov.in/sites/default/files/publication_reports/ ki_health 75th_ final.pdf netshakhuma, n. s. (2019). assessment of the status of the social media records: the case of the mpumalanga government, south africa. journal of archival organization, 16(4), 178196. niti aayog, government of india (2019). ‘health system for a new india: building blocks, potential pathways to reform’. available on 12 feb 2020, retrieved from https://niti.gov.in/sites/default/files/2019-11/nitiaayogbook_compressed.pdf. nowell, l. s., norris, j. m., white, d. e., & moules, n. j. (2017). thematic analysis: striving to meet the trustworthiness criteria. international journal of qualitative methods, 16(1), 113. https://doi.org/10.1177/1609406917733847 https://doi.org/10.3127/ajis.v18i3.1100 https://www.jstor.com/stable/27918082 https://doi.org/10.3127/ajis.v22i0.1678 https://main.mohfw.gov.in/basicpage/annual-report-2018-19 http://www.mospi.gov.in/sites/default/files/publication_reports/%20ki_health%2075th_%20final.pdf http://www.mospi.gov.in/sites/default/files/publication_reports/%20ki_health%2075th_%20final.pdf about:blank about:blank https://doi.org/10.1177%2f1609406917733847 australasian journal of information systems rathore, maurya & srivastava 2021, vol 25, research on user involvement do policymakers use social media 29 pak, a., & paroubek, p. (2010, may). twitter as a corpus for sentiment analysis and opinion mining. in lrec (vol. 10, no. 2010, pp. 1320-1326). pang, b., & lee, l. (2009). opinion mining and sentiment analysis. comput. linguist, 35(2), 311-312. paruthi, m., mendiratta, p., & gupta, g. (2020). young citizen's political engagement in india: social media use by political parties. in examining the roles of it and social media in democratic development and social change (pp. 115-132). igi global. peters, b. g. (2015). state failure, governance failure and policy failure: exploring the linkages. public policy and administration, 30(3-4), 261-276. peters, b. g., capano, g., howlett, m., mukherjee, i., chou, m. h., & ravinet, p. (2018). designing for policy effectiveness: defining and understanding a concept. cambridge university press. picazo-vela, s., fernández-haddad, m., & luna-reyes, l. f. (2016). opening the black box: developing strategies to use social media in government. government information quarterly, 33(4), 693-704. planning commission (2011). ‘high level expert group report on universal health coverage for india’. available on 21 march 2017, retrieved from http://ideas.repec.org/p/ess/wpaper/id4646.html. poell, t., & rajagopalan, s. (2015). connecting activists and journalists: twitter communication in the aftermath of the 2012 delhi rape. journalism studies, 16(5), 719-733. prinja, s., chauhan, a. s., karan, a., kaur, g., & kumar, r. (2017). impact of publicly financed health insurance schemes on healthcare utilization and financial risk protection in india: a systematic review. plos one, 12(2). https://doi.org/10.1371/journal.pone.0170996 qaiser, s., & ali, r. (2018). text mining: use of tf-idf to examine the relevance of words to documents. international journal of computer applications, 181(1), 25-29. rathore, a. k., & ilavarasan, p. v. (2020). pre-and post-launch emotions in new product development: insights from twitter analytics of three products. international journal of information management, 50, 111-127. rathore, a. k., ilavarasan, p. v., & dwivedi, y. k. (2016). social media content and product co-creation: an emerging paradigm. journal of enterprise information management, 29(1), 7-18. rathore, a. k., kar, a. k., & ilavarasan, p. v. (2017). social media analytics: literature review and directions for future research. decision analysis, 14(4), 229-249. ren, y., wang, r., & ji, d. (2016). a topic-enhanced word embedding for twitter sentiment classification. information sciences, 369, 188-198. rsby newsletter, (2013). ministry of labour and employment government of india and indogerman social security programme, and mole, govt. of india and german development cooperation. 2014. rsby connect issue no. 24. new delhi. available on 15 may 2018, retrieved from http://www.rsby.gov.in/documents.aspx?id=16. sachan, d. (2014). health gets greater attention in the 2014 indian elections. the lancet, 383(9925), 1281-1282. http://www.rsby.gov.in/documents.aspx?id=16 australasian journal of information systems rathore, maurya & srivastava 2021, vol 25, research on user involvement do policymakers use social media 30 steen, m., manschot, m., & de koning, n. (2011). benefits of co-design in service design projects. international journal of design, 5(2), 53-60. surowiecki, j. (2004). the wisdom of crowds: why the many are smarter than the few and how collective wisdom shapes business. economies, societies and nations, 296, 5. sutton, j. n. (2009). social media monitoring and the democratic national convention: new tasks and emergent processes. journal of homeland security and emergency management, 6(1). https://doi.org/10.2202/1547-7355.1601 taeihagh, a. (2017). crowdsourcing: a new tool for policy-making?. policy sciences, 50(4), 629647. tajfel, h., & turner, j. c. (2004). the social identity theory of intergroup behavior. in j. t. jost, & j. sidanius (eds.), political psychology: key readings (key readings in social psychology) (pp. 276-293). new york, ny: psychology press. tuarob, s., tucker, c. s., salathe, m., & ram, n. (2014). an ensemble heterogeneous classification methodology for discovering health-related knowledge in social media messages. journal of biomedical informatics, 49, 255-268. verma, r. k., kumar, s., & ilavarasan, p. v. (2017). government portals, social media platforms and citizen engagement in india: some insights. procedia computer science, 122, 842-849. wan, c., shen, g. q., & yu, a. (2015). key determinants of willingness to support policy measures on recycling: a case study in hong kong. environmental science & policy, 54, 409-418. wang, h., lu, y., & zhai, c. (2010, july). latent aspect rating analysis on review text data: a rating regression approach. in proceedings of the 16th acm sigkdd international conference on knowledge discovery and data mining (pp. 783-792). acm. warf, b. (2016). e-government in asia: origins, politics, impacts, geographies. chandos publishing. weissberg, r. (2001). why policymakers should ignore public opinion polls. washington, dc: cato institute. williamson, o. e. (1993). calculativeness, trust, and economic organization. the journal of law and economics, 36(1, part 2), 453-486. yang, s., & zhang, h. (2018). text mining of twitter data using a latent dirichlet allocation topic model and sentiment analysis. int. j. comput. inf. eng, 12, 525-529. yoon, s., elhadad, n., & bakken, s. (2013). a practical approach for content mining of tweets. american journal of preventive medicine, 45(1), 122-129. zaller, j. r. (1992). the nature and origins of mass opinion. cambridge university press. zhang, b., & vos, m. (2014). social media monitoring: aims, methods, and challenges for international companies. corporate communications: an international journal. 19(4), 371383. https://doi.org/10.1108/ccij-07-2013-0044 copyright: © 2021 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits nonhttps://doi.org/10.2202/1547-7355.1601 http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems rathore, maurya & srivastava 2021, vol 25, research on user involvement do policymakers use social media 31 commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v25i0.2965 paper title (paper title style) australasian journal of information systems guha & mukerji 2021, vol 25, research article determinants of digital divide using demand-supply framework 1 determinants of digital divide using demand-supply framework: evidence from india prof. atulan guha indian institute of management kashipur india maitrayee mukerji independent researcher india maitrayee_mukerji@yahoo.com abstract in last two decades, india has seen high economic growth. concomitantly, there has been increase in availability and use of information and communication technologies (icts). however, ict penetration in india is much less when compared to global averages. there exists a substantive level of inequality in ict access and use. empirical studies on extent and nature of digital divide in india are few, especially those with explicit theoretical demandsupply framework, using consistent and reliable pan india data. this paper is an attempt to address these research gaps. it examines the digital divide in india across socio-economic classes and different political-geographic regions. the reduced form demand equations for two ict instruments – internet and mobile phone are estimated separately for households aggregated at subnational level. a multi-variate econometric model identifies both demand and supply side factors shaping differential access by households. findings indicate digital divide is a reflection of existing socio-economic divide. on the demand side, socio-economic inequality as manifest in the economic conditions of households, social category, occupational profile, age and education status are key determinants of district level digital divide. further, supply side factors like availability of electricity, mobile network and extent of urbanization also play an important role. keywords: digital divide, key determinants, econometrics, india, census 1 introduction proactive policies and decreasing cost of data and mobile phones has led to rapid diffusion of information and communication technologies (icts) in india (dutta, 2018). according to the latest data available in the international telecommunication union’s ict indicator database1 the mobile-cellular subscription that was 19.8 per 100 inhabitants in 2007 became 71.67 in 2011 and 85.17 in 2016. during the same time period, the fixed broadband subscription per 100 inhabitants increased from 0.27 to 1.41. the mobile broadband subscription per 100 inhabitants was 0.94 in the year 2011 and has shown a steep rise to 16.41 by the year 2016. however, despite huge improvement in availability and access, the overall penetration of icts in india is much less when compared to global averages. for the period between 2003 and 2018, the values for india’s subscription of fixed broadband per hundred people, fixed telephone per hundred people and mobile subscription per 100 people is substantially and 1 http://www.itu.int/net4/itu-d/idi/2017/index.html australasian journal of information systems guha & mukerji 2021, vol 25, research article determinants of digital divide using demand-supply framework 2 consistently lower than individual brics countries, and from averages of the middle-income country group, upper middle-income country group and world as a whole (see figures 1a, 1b, and 1c). the comparatively low penetration of ict instruments in india was also observed by barman, dutta and nath (2018). this huge gap in key ict development indicators between india and world average is surprising. it is because, in last two decades (year 2001 to 2019), the annual average real gdp growth rate (in 2010 constant us dollar value) of india was 6.65 per cent, whereas the global average was 2.85 percent (source: world development indicator). this indicates that india was one among the high growth economies of the world during the period. and across countries, the most prominent characteristics of the economic growth during this period was the growth of ict and its diffusion. this had created an expectation that high growth economies will also experience high level of ict diffusion. thus, the low level of ict penetration in comparison to other countries, despite having very high gdp growth in last two decades, calls for examining in depth the nature and extent of ict diffusion in india. it is worthwhile to investigate if the overall low penetration of icts in india is because of uneven spread of required ict infrastructure or difference in socio-economic characteristics of households. specifically, in this paper we seek: (a) to examine the extent of digital divide in india at the national, state and district level and (b) to determine the demand and supply side factors causing digital divide at district level. data source: wdi indicators, world bank figure 1a: fixed broadband subscriptions (per 100 people) data source: wdi indicators, world bank figure 1b: fixed telephone subscriptions (per 100 people) 0.00 5.00 10.00 15.00 20.00 25.00 30.00 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 bra chn ind rus zaf mic umc wld 0.00 5.00 10.00 15.00 20.00 25.00 30.00 35.00 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 bra chn ind rus zaf mic umc wld australasian journal of information systems guha & mukerji 2021, vol 25, research article determinants of digital divide using demand-supply framework 3 data source: wdi indicators, world bank figure 1c: mobile cellular subscriptions (per 100 people) the study contributes towards addressing key theoretical and empirical gaps in the extant research on digital divide in india. theoretically, none of the existing studies on digital divide use an explicit demand-supply framework to determine the causes of digital divide in india. this paper uses the concept of realised demand to arrive at the single equation econometric model based on the demand-supply framework to find out the factors causing digital divide. in the process of arriving at single equation the problems of endogeneity and identification has been also taken into consideration. empirically, the demand and supply side factors are considered at the district level using large and consistent public datasets. in future, as data collection becomes more systematic and available, the theoretical model and findings of this study will serve as a reference to further studies on diffusion and adoption of icts in india. following the introduction, the paper proceeds as follows. the second section reviews the existing literature on digital divide, identifying the theoretical and empirical gaps and highlighting the contribution of this research. the third section on methods and data includes description of the theoretical framework, data sources, variables and the econometric model. the next three sections give the results and discussion followed by policy implications and concluding remarks. 2 review of literature icts are classified as general-purpose technologies, with applications that can increase the efficiency and productivity of individuals, households and firms (doong & ho, 2012; james, 2007). in general, icts make production of goods and services more efficient by reducing operational costs and increasing productivity (dewan & riggins, 2005; vu, 2011). the use of icts by government reduces transaction, transportation and communication costs. for households, use of icts reduce communication cost and increase access to various services. in any economy, households provide labour and savings. use of icts by households can make labour more productive and their savings can be channelized to the financial sector more efficiently. thus, in an increasingly connected world, it is necessary for any country to have a robust, and standardised ict infrastructure along with widespread diffusion and adoption of ict instruments that can potentially increase productivity, efficiency and effectiveness of all sectors (ibid). however, it is seen that diffusion, adoption and use of icts by individuals, 0.00 20.00 40.00 60.00 80.00 100.00 120.00 140.00 160.00 180.00 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 bra chn ind rus zaf mic umc wld australasian journal of information systems guha & mukerji 2021, vol 25, research article determinants of digital divide using demand-supply framework 4 groups, households, firms and countries is generally uneven leading to differential outcomes. this has implications for a country’s economic growth and human development (james, 2007). digital divide refers to this “gap between individuals, households, businesses, and geographic areas at different socioeconomic levels with regard both to their opportunities to access icts and to their use of the internet for a wide variety of activities” (oecd, 2001). conceptualizations and definitions of digital divide have evolved over the years. the early definitions simply looked at the phenomenon as difference between “haves” and “have-nots”, the gap between entities based only on availability and access to certain kinds of latest icts (rice & katz, 2003). current understanding about digital divide is more nuanced, taking into consideration factors that give rise to such inequality in access and use (chinn & fairlie, 2007; dewan & riggins, 2005; scheerder, deursen, & dijk, 2017)). the digital divide between any two entities is categorised as either first order or the second order digital divide. while first order digital divide refers to the difference in access to and utilization of technologies, second order digital devices refer to the difference in skills or capabilities of those who have access to icts and can benefit from their use (dewan & riggins, 2005). 2.1 existing studies on digital divide existing studies on digital divide can be broadly grouped into two. the first set of studies examine the presence and extent of digital divide (coria et al., 2013; cuervo & menéndez, 2006) and investigate the underlying socio-cultural, economic, infrastructural and institutional factors that give rise to digital divide among various entities both across (andonova, 2006; doong & ho, 2012; kiiski & pohjola, 2002; pick, sarkar, & johnson, 2015) and within countries (coria et al., 2013; nishida, pick, & sarkar, 2014; nishijima, ivanauskas, & sarti, 2017; sujarwoto & tampubolon, 2016). the objective of these studies is generally to put forth policy level recommendations for broader diffusion and adoption of technologies. thus, digital divide has been explored in terms of technology infrastructure at the country level (ahn & lee, 1999; chinn & fairlie, 2007; cuervo & menéndez, 2006; kiiski & pohjola, 2002; zhang, 2017), adoption of information systems by firms (lera-lopez & marco, 2016) diffusion and utilization of ict instruments and related services by individuals, groups (dobransky & hargittai, 2006), and households (billon, marco, & lera-lopez, 2009; zhao, collier, & deng, 2014). in general, studies have observed that the gap in access and use of icts among different entities are reflection of the prevailing socio-economic, infrastructural and institutional divides (andonova, 2006; baliamoune-lutz, 2003; kiiski & pohjola, 2002;) however, there are differences in the findings regarding the factors, and the quantum of their influence. for example, most studies looking at global level digital divide indicate that education plays a significant role in diffusion of technology in developing countries, but not so significantly in developed countries because of low variability (baliamoune-lutz, 2003; kiiski & pohjola, 2002). the second set of studies evaluate the impact of icts on growth and human development, and some also investigate if digital divide between and within countries have converged or led to divergence over the years (gulati & yates, 2012; park, choi, & hong, 2015). these studies indicate that as compared to computers and internet, diffusion of mobile and mobile broadband services has been faster and more inclusive. this might be because fixed costs and asset-specificity of mobile networks and skills for using mobile phones is generally less (andonova, 2006). australasian journal of information systems guha & mukerji 2021, vol 25, research article determinants of digital divide using demand-supply framework 5 the particular quantitative technique or method adopted by the studies depends not only the level and aspect of digital divide being examined, but also on the availability of relevant datasets. spatial analysis is a common method adopted to study the presence and extent of digital divide (coria et al., 2013; pick & nishida, 2015). determinants of digital divide has been examined using various forms of regression techniques (nishijima et al., 2017). factor analysis and/or principal component analysis is done to identify the factors before running a regression and/or to create a composite index to characterise the digital-divide (park et al., 2015; vicente & lopez, 2011). various forms of tests have been used to check whether the gap between “haves” and “have-nots” have converged or diverged over the years (park et al., 2015; rath, 2016). 2.2 empirical studies on digital divide in india the number of empirical studies examining the extent and nature of digital divide in india is comparatively less (barman et al., 2018; saibal ghosh, 2016; sajal ghosh & prasad, 2014; gupta & jain, 2012; haenssgen, 2018; pick & sarkar, 2015; singh, 2008). specific studies on digital divide in india (see appendix a1) have sought to understand the extent of digital divide (pick & sarkar, 2015), diffusion of various forms of mobile technologies (gupta & jain, 2012), factors for ict utilization and impact of icts on economic growth (ghosh, 2016; narayana, 2011), access to healthcare (haenssgen, 2018) and financial inclusion (saibal ghosh, 2016). key findings are that spread and impact of technologies has been differential. as in case of other countries, diffusion and use of different icts in india have been shaped by technology characteristics and perceived utility (barman et al., 2018; gupta & jain, 2012; pick & sarkar, 2015). the rate of diffusion of mobile telephony has been faster than others. the study done by singh (2008) has analysed the pattern and rate of adoption of mobile phones in india by using diffusion model and estimated future trends. similarly, using an epidemic diffusion model, gupta and jain (2012) analysed the diffusion process of two mobile technologies – code division multiple access (cdma) and global system for mobile (gsm). they found that while the diffusion of cdma was faster than gsm, the latter remained the dominant mobile technology in india. few studies have examined the relationship between telecommunication services and development outcomes. ghosh (2016) investigate the effects of mobile penetration on economic growth in major indian states during 2001–2012. the study found that financial inclusion was one of the channels through which mobile phones exerted a positive and statistically significant impact on economic growth. however, the quantum differs across states with high and low mobile penetration. using a small household sample, narayana (2011) determined that prices have a negative impact whereas income variable has a positive impact on household demand for fixed and mobile phone. haenssgen (2018) in his study of diffusion of mobile phone has observed that while mobile phones increase opportunities to improve rural health care delivery system, poor people not owning one are at disadvantage. 2.3 gaps in extant research and the contributions of this study this paper seeks to address three specific gaps in the literature on digital divide in india – (a) lack of explicit theoretical framework to examine the nature and extent of digital divide in india, taking into consideration demand and supply side factors, (b) lack of studies that takes into heterogeneity at the sub-national (district) level, (c) using large, and consistent public dataset. australasian journal of information systems guha & mukerji 2021, vol 25, research article determinants of digital divide using demand-supply framework 6 in terms of theoretical framework, none of the existing studies have incorporated both demand side and supply side factors for examining digital divide in india. narayana (2011) has estimated the demand for ict instruments using data from a small sample survey of subscribers of a public sector telecom company in karnataka. however, in his estimation the endogeneity problem between price and quantity demanded was ignored. pick & sarkar (2015) have taken into consideration both demand and supply side factors to examine digital divide across states, but without adopting the demand-supply framework. in their econometric analysis, ict utilisation, defined as number of subscribers, is taken as the dependent variable while education, economic factors, social capital, level of openness in the society and electricity are the explanatory variables. however, it has not theoretically developed the estimated equation. further, due to lack of relevant data, this study has not considered some of economic variable, like-exports, which were not available at the state level prior to implementation of goods and service tax (gst). the authors have themselves recognised that the data used for analysis did not belong to the same year. barman, dutta and nath (2018) have examined the determinants of state level variations in teledensities without using any specific theory to support the estimated equation. the authors have not conducted the panel unit root test for the explanatory variables, though they have conducted it for the dependent variable. further, for some variables like populations, literacy rate, and ratio of urban populations, data has been extrapolated from two consecutive census. hence, it is obvious that these variables have a trend. for per capita consumption, the study has used nsso’s state-level small sample dataset. since the number of surveyed observations is small, the estimation of per capita rural and urban consumption might not be accurate. most studies on digital divide in india focus only on the national level data. specific studies investigating the determinants of digital divide at the sub-national level is not yet available. it would be pertinent to examine the differential access and use of icts at district level because digital divide in india manifests as regional, and rural-urban digital divide (parayil, 2005) because of various reasons. since early seventies, a series of policies and incentives by both central and state governments helped in establishing the indian software development and itenabled services (it/ites) industry. however, the predominantly export-oriented it/ites industry led to an “enclave” form of development (parayil, 2005) generating economic opportunities for english speaking and technically trained people (dutta, 2018) with little spill-over effects on other domestic manufacturing or services industries. further, while the national it policy of 1998 gave a common framework for the country, it was adapted and implemented by different states as per their capabilities and priorities (pick & sarkar, 2015). thus, some states like maharashtra, andhra pradesh, karnataka, tamil nadu and kerala with better institutional, infrastructural and human capital could take lead in developing robust and competitive it/ites industry through appropriate environment and incentives (dutta, 2018; mukerji, 2013). however, even in these states, the spill-over effects of the it/ites sector in terms of economic opportunities was limited to specific districts or geographical areas (parayil, 2005). intra-state digital divide also manifested in the form of rural-urban divide, with urban areas having higher penetration of computers and internet (pick & sarkar, 2015). internet penetration in urban india is around 59 per cent but it is as low as 18 per cent in rural india (gupta, 2018). the empirical contribution of this study is to examine the demand and supply side factors characterising the digital divide at the district level in india. in india, a district is the highest administrative unit in a state. considering districts as level of analysis also enables us to obtain comparable data for most variables. australasian journal of information systems guha & mukerji 2021, vol 25, research article determinants of digital divide using demand-supply framework 7 further, it aligns with the general practice in decentralised development planning, where the district is the unit of plan, action and accountability. the third contribution of the paper is that it makes an attempt to partially address the unavailability of recent and appropriate data, often cited as a reason for lack of studies on digital divide in india. unlike many other countries, large-scale data collection or survey on access and use of icts at individual and household level is not institutionalised in india. data that can be used for undertaking such analysis are scattered across multiple databases, dated differently and often behind a paywall. much effort is required to combine them and prepare a consistent dataset. this study uses household, population and district level data available in census 2011 and supplemented by data published by trai on supply side factors. hence, the analysis is based on internally consistent data, expected to be free from biases that often arise in small sample surveys. to summarise, this paper seeks to contribute to literature by examining the nature, extent and determinants of digital divide in india using an explicit demand-supply theoretical framework, using large and consistent database aggregated at district level. 3 methods and data this research has two objectives. the first objective is to examine the extent of digital divide in india at the national, state and district level and the second is to determine the factors that causing digital divide at district level. for the first objective, data has been taken from the nsso surveys and analysed using descriptive statistics. for accomplishing the second objective of finding the determinants of digital divide at the district level, we have developed a demand supply framework, and applied it to derive the reduced form of demand equation for the two ict instruments – computers with internet and mobile phone. the determinants of digital divide at the district level are then estimated using a multivariate econometric model. 3.1 theoretical framework in order to understand the reasons for digital divide at district level, it is imperative to understand the demand for ict instruments. this article defines ict instrument as any computing/communication devices like computer, internet and mobile phone. at the equilibrium, the demand for particular ict instrument (qd) will be equal to the supply of the ict instrument (qs). 𝑄𝑑 = 𝑎 − 𝑏1𝑃 + ∑ 𝑏2𝑖𝑋𝑖 𝑛 𝑖=1 + 𝑢 (1) 𝑄𝑠 = 𝑐 + 𝑑1𝑃 + ∑ 𝑑2𝑗𝑍𝑗 + 𝑣𝑚 𝑗=1 (2) at equilibrium, qd=qs=q equation 1 is a demand function. it is assumed to be linear. we take ‘qd‘ as the district-level aggregate household demand for a particular ict instrument. ‘p’ is the price of the ict instrument, ‘xi ’ are exogenous variables representing the various household and population characteristics, aggregated at the district level. these include socio -economic status of households, educational level of the population, and occupational characteristics, among others. ‘n’ is number of exogenous variables. ‘a’, ‘b1’ and ‘b2i’ are the coefficients. equation 2 is a supply function, assumed to be linear. ‘qs’ is the supply of ict instruments. ‘p’ is the price of ict instrument. ‘zj’ are the ‘m’ exogenous variables characterising the various supply side australasian journal of information systems guha & mukerji 2021, vol 25, research article determinants of digital divide using demand-supply framework 8 factors like urbanization, availability of electricity, availability and quality of service of broadband and mobile phone networks at the district level. ‘c’ and ‘d1’, ‘d2j ’ are the coefficients. estimating a set of simultaneous equation consisting of demand and supply equations may give rise to the problem of endogeneity. to avoid this, we use the reduced form of these equations, derived from the equilibrium condition. this reduced form of equation is suitable for ordinary least square (ols) estimation. however, to estimate the coefficients of the demand and supply equations, mentioned above, the identification problem will arise, as both ‘m’ and ‘n’ are greater than the number of endogenous variables. here we are not interested in estimating the coefficient of demand and supply equation, but seek to understand the relationship between district-level demand of ict instruments and the various exogenous variables. hence, estimation of reduced form equation with ‘q’ as dependent variable, which is suitable for ols estimation, will serve our purpose. thus, the reduced form equation we are estimating is: 𝑄 = 𝐴 + ∑ 𝐵𝑖𝑋𝑖 + ∑ 𝐷𝑗𝑍𝑗 𝑚 𝑗=1 𝑛 𝑖=1 + 𝑤 where, a=(ad1 + b1c)/(b1 + d1), b=d1b2/(b1 + d1) d=b1d2/(b1 + d1) w=(d1u + b1v)/(b1 + d1) existing literature guide the selection of exogenous variables that shape the demand for particular ict instruments. broadly, exogenous variables would include factors related to economic status of households, their social categories, and occupational, age and gender profiles (andonova, 2006; dewan & riggins, 2005; nishida et al., 2014; pick & azari, 2008; vicente & lopez, 2011). household income determines the budget constraint of consumer (chinn & fairlie, 2007) and is one major factor that should influences the demand for ict instruments. preference for owning an ict instrument is also likely to vary depending on individuals’ exposure to and perceived usefulness of owning them (gupta & jain, 2012). this, in turn might be related to a person’s occupation (cuervo & menéndez, 2006) and education (chinn & fairlie, 2007). it is expected that the use of ict instruments will be more among those who are in service sector than those in agriculture and industries (barman et al., 2018; park et al., 2015). this is because as compared to agriculture and manufacturing sectors, the very nature of service sector requires greater degree of communications between different stakeholders. further, if a person belongs to some backward community, the exposure to new technologies is expected to be less. similarly, it is expected that young people will adopt new technologies more readily, and hence demand of ict instruments will be more among younger population (kiiski & pohjola, 2002). status of women within family and society might be an important factor in enabling them to adopt new technologies. the supply side exogenous variables are size of the market and urbanization. (andonova, 2006; vicente & lopez, 2011). both of them can influence the cost of supplying ict instruments. size of the market provides economies of scale which can reduce average cost of supply. urban areas are more likely to have better transportation facility and higher concentration of population that facilitates greater diffusion of ict infrastructure. since one cannot use a particular ict instrument without electricity or communication network, the australasian journal of information systems guha & mukerji 2021, vol 25, research article determinants of digital divide using demand-supply framework 9 other supply side exogenous variables are availability of electricity (chinn & fairlie, 2007; kiiski & pohjola, 2002), internet and mobile networks (barman et al., 2018) along with quality of their services. further, we have also included competitiveness among various network service providers as a supply side variable. greater competition should lead to lower price of connectivity, hence creating greater demand for ict instruments, especially for the mobile phones. so, we have included all these exogenous variables to estimate the reduced form of demand equation for ict instruments. the independent variables of the study are: (1) economic status of households, which also proxy the market size (2) educational profile, also proxy for the status of women, (3) occupational characteristics, (4) age distribution, and (5) social category. (6) extent of urbanization, (7) availability of electricity, (8) availability of broadband network, (9) broadband service quality, (10) availability of mobile network, (11) mobile network service quality, and (12) degree of competitiveness. the next section describes the data sources and indicators for these variables. 3.2 data, data sources and variables the nature and extent of digital divide is examined primarily by using two public databases – the census 2011 database and 68th and 75th rounds of sample survey by national sample survey organisation (nsso). the 68th round survey of nsso provides data on household consumption expenditure for the year 2011-12. the 75th round survey of nsso provides data for the year 2017-18 through its report on household expenditure on education. however, this report does not provide data on possession of mobile phone, which is too important to ignore in any discussion on digital divide presently. hence, we use the data on household possession of mobile phones provided in both the census and 68th round nsso survey for the year 2011. but there are some important differences between the two datasets. census 2011 has data on the household possession of ict instruments like landline, mobile phones, computers and computers with internet, disaggregated up to village level. however, it does not provide data on household expenses on icts, which is available in the nsso database. on the other hand, since nsso is primarily a sample survey, in terms of geographical disaggregation, the nsso estimates, even though this particular round of survey is of large sample size, are not much reliable beyond the level of large sized states due to further lowering of the sample size. thus, we have used the nsso data from 68th round survey to describe the nature and extent of digital divide at the national level by using descriptive statistics. since there are 29 states (with even lower number of large states) in india, we have not used the nsso data in the econometric models due to lack of adequate degrees of freedom. to avoid the problems arising out of low degrees of freedom, the econometric models are estimated using the district level data. in the econometric models, we have used census of india 2011 and reports on indian telecom services performance indicators, january march 2011, published by telecom regulatory authority of india (trai). in census 20112 database, district level data is organized state-wise and is located in multiple tables including district census handbook3 (dch), population census abstract4 (pca), household assets and amenities5 (haa) and education level by age and sex for population 2 http://www.censusindia.gov.in 3 http://www.censusindia.gov.in/2011census/dchb/dchb.html 4 http://www.censusindia.gov.in/pca/ddw_pca0000_2011_indiastatedist.xlsx 5 http://www.censusindia.gov.in/2011census/hlo/houselisting-housing-pca.html australasian journal of information systems guha & mukerji 2021, vol 25, research article determinants of digital divide using demand-supply framework 10 age 7 and above6 (edu). a particular challenge was to identify appropriate variables corresponding to the factors specified in the theoretical framework. proxy variables had to be defined for data not available in direct form. all the data corresponding to dependent variables and demand side explanatory variables of the econometric models has been taken from the census of india, 2011 database. two supply side explanatory variables, to capture urbanisation and electricity connectivity, has also taken from the census, 2011 database. data corresponding to other supply side explanatory variables of the econometric models were extracted from reports published by trai (see appendix a2). 3.2.1 dependent variable the census of india 2011 in its tables on household asset and amenities (haa) provides information on percentage of households in a district having (a) landline (b) computer, (c) computer with internet, and (d) mobile phones. the variations in the characteristics of these four ict instruments fulfil different kinds of household needs and hence there could be different reasons for possessing them (gupta & jain, 2012). varying degree of substitutability and complementarity among these four ict instruments (gupta & jain, 2012; narayana, 2011) makes it is difficult to create one combined dependent variable for household access to ict instruments. while all the four variables are reasonable indicators of access to ict instruments, we use only the last two in our analysis. as computers without internet do not create any medium of communication, and the percentage of households with landline telephone is very low, we do not consider the corresponding variables. the variable – computer with internet is the only one that gives some idea about internet penetration at household level. further, existing studies indicate that mobile phones have a higher rate of diffusion as compared to landline, computers and/or internet (gupta & jain, 2012; singh, 2008). hence, we estimate the reduced form of demand equation separately for these two ict instruments – computers with internet and mobile phones. it is to be noted that in census 2011 the variable mobile phone only indicates the ownership of a mobile phone. it does not indicate whether a household has more than one mobile phone or the type of mobile phone or use of mobile internet. we expect that together these two variables would reasonably represent ict access and use by households at the district level. thus, for our analysis we define the following two dependent variables as proxy to household demand for particular ict instruments at the district level. 1) share_of_households_having_computers_with_internet: is defined as the ratio of the number of households in a district possessing computer with internet to total number of households in a district 2) share_of_households_having_mobile_phones: is defined as the ratio of the number of households in a district possessing mobile phones to total number of households in a district 3.2.2 independent variables 1) economic well-being of households: in india around 92 per cent of the workforce are in informal sector. close to 50 per cent of this workforce is in the agriculture sector dominated by marginal and small farmers practicing subsistence farming. even large farmers are largely unincorporated. given this employment profile in the indian economy, it is very difficult to acquire reasonably good quality data on household 6 http://www.censusindia.gov.in/2011census/c-series/c08.html australasian journal of information systems guha & mukerji 2021, vol 25, research article determinants of digital divide using demand-supply framework 11 income. in fact, for measuring poverty and inequality, government of india uses consumption expenditure data. since census 2011 also does not have data on household income, we use the data of the variable type of roof as a proxy to determine the economic status of households. based on this variable households are divided into three distinct economic classes. the households with concrete roof-top are assumed to be economically well-off. households classified as economically poor have houses with roofs made of grass, polythene, or hand tiles. those belonging to economically middle category are ones with roof made of burnt brick, stone, asbestos and machine tiles. for our analysis, we have defined the ratio of poor and economically middle households to economically well-off in a district (ratio_of_poor_and_middle_to_welloff_households) as an indicator for economic status of households aggregated at the district level. it is also used as proxy for market size. we can expect, higher the ratio of economically poor and middle income to well-off households lower will be the market size. 2) education and gender: the district level educational profile is captured by the variable called share_of_person_educated_at_middle_school_and_above. it is defined as share of total number of persons attaining middle level school education and above in total population. similarly, a variable called share_of_women_educated_secondary_and_above indicates the status of women at the district level. it is defined as share of total number of females attaining secondary level school education and above in total female population. it was found that both the variables are highly correlated with correlation co-efficient of 0.95 and can be used interchangeable. hence, we use only one of them for estimating the reduced form of demand function of the two dependent ict variables. for estimating the demand for computers with internet, we use the variable share_of_women_educated_secondary_and_above as independent variable. various information criteria, like, akaike information criterion (aic), bayesian information criteria (bic) also shows, among these two education variables, share_of_women_educated_secondary_and_above is best fit. since mobile phones require relatively lower level of education and skills, we take the share_of_person_educated_at_middle_school_and_above as independent variable for estimating the demand for mobile phones. the aic and bic criterion also shows that it is the best fit. the variable, share_of_women_educated_secondary_and_above not only captures the level of education but also captures the state of women and openness within the household. since, both the education variables are highly correlated, the variable share_of_person_educated_at_middle_school_and_above should also proxy for the other one. 3) occupational characteristics: to capture the occupational characteristics, we use the ratio of the workforce involved in service sector to total workforce in the district (share_of_workers_in_service_sector) as the indicator. 4) age distribution: to capture the effect of age, we take the ratio of population in the age cohort of 15 years to 35 years to the total population of the district (share_of_pop_age_between_15to35). 5) social category: in india, for socio-historical reasons certain groups referred to as scheduled caste and scheduled tribe (sc/st) are considered to be disadvantaged than australasian journal of information systems guha & mukerji 2021, vol 25, research article determinants of digital divide using demand-supply framework 12 others. thus, to capture social backwardness of a district, we use the share of sc/st population to the total population of the district (share_of_scst_pop). 6) extent of urbanization: the extent of urbanization in a district is captured by considering the share of population residing in urban area to total population (share_of_urban_population). 7) availability of electricity: we have taken the share of households with electricity to total households in a district (share_of_households_with_electricity) as independent variable. 8) availability of broadband network: the availability of broadband network and its service quality reasonably captures the presence of district level infrastructure for internet access. we have taken the share of village panchayats with broadband connections in total number of village panchayats (share_of_villages_with broadband_coverage) as a proxy variable7. for our analysis, we have mapped the circle level figure given in the report for broadband coverage to corresponding districts. for those districts, which are completely urban (this holds only for big cities), it is expected that they are completely covered by the broadband network. hence, these districts have been assigned the value ‘1’. 9) service quality of broadband network: the data for service quality of broadband network connectivity is taken from a telecom regulatory authority of india (trai) report on performance indicators for indian telecom services for the period january 2011 to march 20118. the report lists fifteen parameters9 for measuring broadband service quality of a service provider. trai has fixed a minimum threshold for each of these parameters. if a service provider exceeds this minimum threshold value, the quality of service on that particular parameter is considered to be satisfactory. for our analysis, we have created an index, named index_average_service_quality_broadband_network for each district that comes under particular circles in different states. a score of 1 is assigned if the particular parameter crosses the minimum threshold value, otherwise 0. since in the year 2011 more than 80 per cent of broadband connections in india was with the public sector unit, bsnl-mtnl10, we calculate the simple (not taking the market share of service provider as the weight) average for each parameter. we then 7 data is taken from the report to delivery monitoring unit set up under prime minister’s office, [dmu(pmo)] for the march 2011, having an annexure on broadband coverage of village panchayats under bharat nirman-ii (a govt. of india sponsored scheme to create rural infrastructure). 8 https://main.trai.gov.in/release-publication/reports/performance-indicators-reports 9 these parameters are – (1) percent of connections provided within 15days of registration of demand, (2) percent of faults repaired by next working day (>90%), (3) percent of faults repaired within 3 working days, (4) percent of bills disputed, (5) percent of billing complaints resolved within 4 weeks, (6) percent of cases to whom refund of deposits is made within 60 days of closures, (7) percent of calls answered by operator (voice to voice) within 60 sec to customer, (8) percent of calls answered by operator (voice to voice) within 90 sec to customer, (9) percent international bandwidth utilization during peak hours (tcbh) (enclose mrtg) <90%, (10) broadband connection speed available (download) from isp node to user, (11) service availability /uptime (for all users) in percent, (12) packet loss (for wired broadband access) in percent, (13) user reference point at pop/isp gateway node to igsp/nixi, (14) user reference point at isp gateway node to international nearest nap port abroad (terrestrial), (15) user reference point at isp gateway node to international nearest nap port abroad (satellite) 10 source: “telecom sector in india: a decadal profile”, 2012, prepared by ncaer at the behest of trai australasian journal of information systems guha & mukerji 2021, vol 25, research article determinants of digital divide using demand-supply framework 13 take the average of these values for all the 15 parameters to get a value of index. this index can be considered as average broadband service quality for a particular circle. the circle level index values are then mapped to corresponding districts. 10) availability of mobile network: the district census handbooks have data indicating the presence or absence of mobile network towers, but only for rural areas. thus, for each district, we define the share_of_villages_with_mobile_network as an indicator of physical availability of mobile network coverage. for the year 2011, the demand for computers with internet connection is less likely to be dependent upon availability of mobile network. 11) service quality of mobile network: based on statistical data from the same trai report11 that has been used for creating the index for average service quality of broadband networks, we create an index, named, index_average_service_quality_mobile_network for measuring the service quality of mobile network. for assessing the service quality of mobile networks, the publication lists 17 parameters. if a service provider exceeds a minimum threshold value, the quality of service on that particular parameter is considered to be satisfactory. to combine these 17 parameters12, we assign a score 1 if the minimum threshold value of a particular parameter is crossed, otherwise score is 0. then for each parameter the weighted average is calculated across service providers. the assigned weights correspond to the share of different service providers in a particular circle. a mean of the 17 weighted average is calculated to get the value of index_average_service_quality_mobile_network for a particular circle. the index values for each circle are then mapped onto the corresponding districts. 12) competitiveness: the degree_of_competition is defined as the coefficient of variation of market share of each mobile network service provider within a particular circle. higher the value of the co-efficient of variation, lower the degree of competition, as we can expect, in a more competitive world market share should become more even. further, this value of coefficient of variation is being assigned against the districts that comes under this particular circle. the data for market share has been taken from the same trai report13. we have not calculated it for broadband service, as in 2011, more than 80 percent of broadband services were provided by bsnl-mtnl. appendix a2 summarises the variables, their indicators and data source. it also indicates the variables that have been used for estimating the specific reduced form of the demand equations. 11 https://main.trai.gov.in/release-publication/reports/performance-indicators-reports 12 these parameters are – (1) btss accumulated downtime (% ), (2) worst affected btss due to downtime (%), (3) call set up success rate (within licensees own network), (4) sdcch/paging chl. congestion (%), (5) tch congestion (%), (6) call drop rate (%), (7) worst affected calls having more than 3% tch drop (call drop) rate (%), (8) connection with good voice quality, (9) poi congestion averaged over quarter, (10) metering and billing credibility (post paid), (11) metering and billing credibility (pre paid), (12) resolution of billing complaint, (13) period of applying credit/adjustment,(14) accessibility to call centre, (15) percent of call answered, (16) percent request for termination, (17) time taken for refund deposit 13 https://main.trai.gov.in/release-publication/reports/performance-indicators-reports australasian journal of information systems guha & mukerji 2021, vol 25, research article determinants of digital divide using demand-supply framework 14 3.3 econometric model we estimate the reduced form of demand equation for the two ict instruments separately using the following equations: 1) share_of_households_having_computers_with_internet = a + b1 x ratio_of_poor_and_middle_to_welloff_households + b2 x share_of_women_educated_secondary_and_above + b3 x share_of_workers_in_service_sector + b4 x share_of_pop_age_between_15to35 + b5 x share_of_scst_population + b6 x share_of_urban_population + b7 x share_of_households_with_electricity + b8 x share_of_villages_with_broadband_coverage + b9 x index_average_service_quality_broadband_network + € 2) share_of_households_having_mobile_phones = a + b1 x ratio_of_poor_and_middle_to_welloff_households + b2 x share_of_person_educated_middle_school_and_above + b3 x share_of_workers_in_service_sector + b4 x share_of_pop_age_between_15to35 + b5 x share_of_scst_population + b6 x share_of_urban_population + b7 x share_of_households_with_electricity + b8 x share_of_villages_with_mobile_network + b9 x index_average_service_quality_of_mobile_network + b10 x degree_of_competition + € 4 results and discussion 4.1 nature and extent of digital divide: according to census 2011, 56.70 per cent of household in india have mobile phones. in contrast, only 9.06 per cent of households have landline telephone. further, while 8.25 per cent of households own a computer, only 2.36 per cent of the total households have computers with internet (see table 1). these figures indicate that in the year 2011, a very large number of households did not have access to ict instruments. while comparable data for recent years is not yet available, as next census is due only in the year 2021, the itu handbook for the year 201714 indicates that the level of access to ict instruments has substantially improved, maybe as result of both policy interventions and market mechanisms. in particular, mobile broadband subscription per 100 inhabitants increased from 0.94 in the year 2011 to 16.41 by the year 2016. the nsso 75th round (2017-18) survey report on household expenditure in education tells us that 10.7 per cent and 23.8 per cent of households in india, have computers and internet connections respectively. so, while access to internet through mobile has definitely witnessed a rapid improvement, even then it is accessed by only one-fourth of the indian households. 14 telecommunication/ict indicators 2007-2016, yearbook of statistics 2017, page 120 australasian journal of information systems guha & mukerji 2021, vol 25, research article determinants of digital divide using demand-supply framework 15 item landline computer internet mobile mean 9.06 8.25 2.36 56.70 top 3 states lakshadweep (47.10) kerala (42.99) goa (36.65) chandigarh (33.20) goa (31.15) delhi (29.66) chandigarh (18.80) delhi (17.89) goa (12.70) delhi (85.78) lakshadweep (85.50) chandigarh (83.20) bottom 3 states bihar (3.72) nagaland (3.68) meghalaya (3.39) rajasthan (5.95) jharkhand (5.79) madhya pradesh (5.21) jharkhand (1.03) meghalaya (1.03) chhattisgarh (0.86) madhya pradesh (41.93) tripura (41.10) meghalaya (37.24) data source: census 2011 table 1. households’ possession of ict instruments in india the diffusion of ict instruments is not even across the country (figure 2). there is much difference in the spread of various ict instruments in different states. it is also observed that states which had lower penetration of computers and landline, also have relatively less penetration of internet and mobile phones. data source: census 2011 figure 2. share of households having ict instruments in total households in different states (in %) share of households with: min max coefficient of variations (in %) landline 0.80 55.00 96.47 computer 1.70 39.30 69.09 internet 0.20 24.20 147.88 mobile phones 9.00 88.20 30.12 data source: census 2011 table 2. district-wise dispersion in households’ possession of ict instruments 0.00 10.00 20.00 30.00 40.00 50.00 60.00 70.00 80.00 90.00 100.00 a n d a m a n & … a n d h r a p r a d e s h a r u n a c h a l … a s s a m b ih a r c h a n d ig a r h c h h a t t is g a r h d a d r a & n a g a r … d a m a n & d iu g o a g u ja r a t h a r y a n a h im a c h a l … ja m m u & … jh a r k h a n d k a r n a t a k a k e r a l a l a k s h a d w e e p m a d h y a … m a h a r a s h t r a m a n ip u r m e g h a l a y a m iz o r a m n a g a l a n d n c t o f d e l h i o d is h a p u d u c h e r r y p u n ja b r a ja s t h a n s ik k im t a m il n a d u t r ip u r a u t t a r p r a d e s h u t t a r a k h a n d w e s t b e n g a l computers landline mobile internet australasian journal of information systems guha & mukerji 2021, vol 25, research article determinants of digital divide using demand-supply framework 16 table 2 gives the district-wise dispersion in possession of various ict instruments. districtwise dispersion is lowest for mobile phone and highest for the computer with internet. the 68th round survey on household consumption expenditure for the year 2011 by national sample survey organisation (nsso) tells us that 1.5 and 14.9 per cent of the rural and urban households respectively have computers. the same survey also tells us 77.6 and 92.2 per cent of the rural and urban households respectively have mobile phone. this indicates that the rural-urban divide is more than eight times in case of computers than for mobile phones. in 2011 only 1 and 7.2 per cent of the rural and urban household respectively had internet connectivity (table 3). ict instrument 2011-12 2017-18 rural urban rural-urban ratio rural urban rural-urban ratio computer 1.5 14.9 0.10 4.4 23.4 0.19 mobile 77.6 92.2 0.84 na na internet 1.0 7.2 0.14 14.9 42.0 0.35 source: nsso survey report no. 558 (68th round) and report no. 585 (75th round) table 3: share of households having ict instruments in india in the year 2011-12 and 2017-18 after 68th round, the nsso has not published the report of survey (large sample based) on household consumption expenditure citing some technicalities. however, they have published the report of large sample-based survey on household expenditure in education (75th round). this 75th round report provides us data on household having computers and internet connection for the year 2017-18. according to this report, 4.4 and 23.4 per cent of the rural and urban household, respectively have computers. further, 14.9 and 42.0 per cent of the rural and urban household, respectively have internet connectivity (table 3). so, there is a very large number of households in india who still do not have access to internet. the rural-urban ratio tells us there is a continuation of huge rural-urban gap in possession of computer and internet; though the extent of rural-urban gap has reduced in 2017-18 in comparison to 201112. this is because both the possession of computers and internet connection has increased. however, the spread of internet is much faster than that of computers. this indicates access to internet through mobile phone has contributed substantially in this spread of internet. the nsso survey further indicates that richer households have greater access to ict instruments (computers and internet connection) as compared to poorer households (figure 3a and 3b). this trend has not been changed between the years 2011-12 to 2017-18. however, the gap in possession of these two ict instruments between different economic classes has come down during this period. we have clubbed cumulative distribution of household consumption expenditure through quintile classes grouped as – (a) up to lowest 20 percent, (b) 20-40 percent, (c) 40-60 percent, (d) 60-80 percent and (e) 80-100 percent. the gap in possession of computers among the different economic classes is higher than that of internet connection. the gap between the rural and urban classes is higher in possession of computers than that of internet. australasian journal of information systems guha & mukerji 2021, vol 25, research article determinants of digital divide using demand-supply framework 17 source: nsso survey report no. 558 (68th round) and report no. 585 (75th round) figure 3a: percentage of households having computer for each quintile class source: nsso survey report no. 558 (68th round) and report no. 585 (75th round) figure 3b: percentage of households having internet facility for each quintile class with regards to possession of mobile phones, the rural-urban divide exists and the richer economic classes in both rural and urban areas possess more. however, the gap between share of households having mobile phone in rural and urban is much less in compare to that of other ict instruments like computers and internet connection. further, the gap in possession of mobile among different economic classes is also less in compare to possessions other ict instruments (figure 4). 0 .2 7 0 .5 3 1 .6 7 .5 0 .3 0 2 .4 5 2 .4 9 .9 0 .4 5 5 .8 3 .3 1 6 0 .4 5 1 2 .5 3 .4 2 4 5 .3 0 4 0 .1 9 .9 4 5 .5 r u r a l 2 0 1 1 1 2 u r b a n 2 0 1 1 1 2 r u r a l 2 0 1 7 1 8 u r b a n 2 0 1 7 1 8 0-20 20-40 40-60 60-80 80-100 0 .0 3 0 .2 0 6 .6 1 9 .8 0 .2 0 1 .3 5 9 .2 2 9 .4 0 .1 5 2 .8 0 1 2 .4 3 8 0 .8 5 4 .8 5 1 5 4 6 .3 3 .0 0 2 0 .6 3 2 7 .1 6 1 .6 r u r a l 2 0 1 1 1 2 u r b a n 2 0 1 1 1 2 r u r a l 2 0 1 7 1 8 u r b a n 2 0 1 7 1 8 0-20 20-40 40-60 60-80 80-100 australasian journal of information systems guha & mukerji 2021, vol 25, research article determinants of digital divide using demand-supply framework 18 data source: nsso 68th round note: we have 12 fractile economic classes based upon cumulative monthly per capita consumption expenditure. the fractile classes are i) less than 5%, ii) 5 to 10%, iii) 10 to 20%, iv) 20 to 30%, v) 30 to 40%, vi) 40 to 50%, vii) 50 to 60%, viii) 60 to 70%, ix) 70 to 80%, x) 80 to 90%, xi) 90 to 95 %, xii) 95 to 100%. figure 4. number per 1000 household possessing mobile handset in fractile classes of mpce figures 3a, 3b and 4 together show that more than 95 per cent of rich households in urban areas have mobile phone. however even among the richest 20 per cent of urban households, roughly 55 per cent does not have computer and roughly 39 per cent did not have internet connectivity. so even among richest urban household, there is substantial gap in possession of ict instruments. households in rural india and those belonging to urban poor have substantially low possession of ict instruments. all this evidence conclusively highlights that there is substantial level of inequality in digital access in india. this inequality exists between states, between districts, among rural-urban areas and different economic classes. in the next two sub-sections, we examine in detail the factors creating this inequality in household possession of two ict instruments – computers with internet and mobile phones using econometric models arising out of demand –supply framework. 4.2 factors behind digital divide 4.2.1 estimation of reduced form demand equation of households for computers with internet in the previous section, we saw that the rural-urban digital divide is very wide for household possession of computers with internet. hence, we estimate the reduced form of demand function for computers with internet by taking extent of urbanisation as an explanatory variable. we have estimated the equation twice; first for all districts together and then separately for pre-dominantly rural districts only. for this purpose, we define a rural district as one in which the urban population is less than or equal to 40 percent. the total number of districts is 634, after removing outliers, 512 were identified as rural districts. since, by definition, the value of the independent variable share_of_household_having_computers_with_internet will lie between 0 and 1 (confirmed by the appendix a3.3), we take its logistic transformation before running the ols [see appendix a4, equation1, 2]. the breusch-pagan test suggest that there is no heteroscedasticity. both the information criteria akaike information criteria (aic) and bayesian information criteria (bic) suggested that share_of_women_educated_secondary_and_above is providing a better fit in comparison to share_of_person_educated_middle_school_and_above. thus, the former is included as a variable for both education and gender. the ramsey test of specification on this estimation 0 200 400 600 800 1000 1 2 3 4 5 6 7 8 9 10 11 12 rural urban australasian journal of information systems guha & mukerji 2021, vol 25, research article determinants of digital divide using demand-supply framework 19 suggests no specification error due to omitted variables. the regression result (table 4) shows that five independent variables – share_of_urban_population, share_of_pop_age_between_15to35, share_of_workers_in_service_sector, share_of_women_educated_secondary_and_above, index_average_service_quality_broadband have positive statistically significant relationship with the dependent variable. the variable share_of_scst_population has negative statistically significant relationship with the dependent variable. however, the variables share_of_villages_with_broadband_coverage, ratio_of_poor_and_middle_to_welloff_households and share_of_households_with_electricity do not have any statistically significant impact. the above regression results indicate that districts with a smaller number of sc-st population, higher percentage of young population, higher level of education and maybe with better social status of women, more service sector-oriented economy, with higher degree of urbanisation and better broadband network service quality have greater diffusion of computer with internet. among all these, higher education, especially of women and share of young population are the two most important factors. prominence of service sector economy follows next. the three other significant factors are degree of urbanisation, broadband network service quality15, and share of sc/st population. the economic status of households appears to be less important as compared to other factors since the value of its coefficient is less and it is not statistically significant even at 10 per cent level. one possible reason can be that even the economically well-off in india are not using computers and internet, unless required for occupational need. further, higher levels of education appear to be enabling people to use computer with internet. 15 though the co-efficient of broadband network coverage is not statistically significant, broadband network service quality is. it most probably indicates that users are more sensitive to the quality of network services that they are getting, rather than merely availability of network on paper. it is true for mobile network coverage and network service quality too. australasian journal of information systems guha & mukerji 2021, vol 25, research article determinants of digital divide using demand-supply framework 20 independent variables dependent variable: logistic transformation of share_of_households_having_computers_with_internet logistic transformation of share_of_households_having_mo bile_phones all districts rural districts all districts value of coefficient level of significance value of coefficient level of significance value of coefficient level of significance ratio_of_poor_and_middle_to_welloff_households -0.02976 not significant -0.0473291 sig. at 10 % level -0.058002 sig. at 5% level share_of_women_educated_secondary_and_above 4.598076 sig. at 1 % level 4.378131 sig. at 1 % level share_of_persons_educated_middle_school_and_ab ove 0.679934 sig. at 5% level share_of_workers_in_service_sector 1.385045 sig. at 1 % level 1.845224 sig. at 1 % level -0.160926 not significance share_of_pop_age_between_15to35 3.209441 sig. at 1 % level 2.50101 sig. at 1 % level 5.097142 sig. at 1% level share_of_scst_pop -0.328500 sig. at 1 % level -0.3122073 sig. at 1 % level -1.289316 sig. at 1% level share_of_urban_population 1.181709 sig. at 1 % level 0.023099 not significance share_of_households_with_electricity 0.000885 not significant 0.1127426 not significant 0.279126 sig. at 1% level share_of_villages_with_broadband_coverage -0.474380 not significant 2.764722 not significant index_average_service_quality_broadband 0.456850 sig. at 1 % level 0.7294781 sig. at 1 % level share_of_villages_with_mobile_network 0.167741 not significance index_average_service_quality_mobile_network 1.158066 sig. at 1% level degree of competition -0.153752 not significance source: detailed result in appendix a4, equation 1 & 3 source: detailed result reported in appendix a5, equation1 table 4. regression result of reduced form demand equation estimation australasian journal of information systems guha & mukerji 2021, vol 25, research article determinants of digital divide using demand-supply framework 21 the supply side factors like broadband network quality and degree of urbanisation also appears to play a deciding factor here, though at a lesser extent than the other three factors, which essentially generate demand. further, this result also indicates that household possession of computers with internet is less among sc-st communities. to summarise, the results commensurate with the expectation that educated youth (relatively less from sc-st community), men and women from more urbanised districts, somewhat dependent on service sector economy access and use computers with internet more than others. we then estimate the reduced form demand equation for computers with internet for the rural districts (see appendix a4, equation 3), the breusch-pagan test suggest that there is no heteroscedasticity. both the information criteria akaike information criteria (aic) and bayesian information criteria (bic) suggested that share_of_women_educated_secondary_and_above is providing a better fit in comparison to share_of_person_educated_at_middle_school_and_above. thus, share_of_women_educated_secondary_and_above is included as the education variable. the ramsey test of specification on this estimation suggests no specification error due to omitted variables. the estimated results (table 4) for rural districts show three important departures from the previous estimated results of all districts. first, among the rural districts relatively economically well-off districts are having greater number of households possessing computers with internet. second, the values of all coefficients of demand side factors are lower for rural districts than the corresponding co-efficient values obtained when all districts are considered. third, the values of all coefficients of supply side factors are higher for rural districts than the corresponding co-efficient values obtained when all districts are considered. however, the absolute value of statistically significant coefficient of most of the demand side factors remain higher than that of supply side factors. thus, demand side factors remain prominent to explain the possession of computers with internet for rural districts. and there is an increasing importance of supply side factors to explain digital divide in rural districts. however, broadly speaking, the causes for digital divide in rural districts are similar to that of the general story of all districts of india, but with the exception that the economic status of households also becomes a statistically significant factor. 4.2.2 estimation of reduced form demand equation of households with mobile phones: the dependent variable is share_of_households_having_mobile_phones. since, by definition, its value will vary between 0 and 1 (confirmed by the appendix a3.3), we take its logistic transformation before running the ols. the rural-urban divide in household possession of mobile phone is comparatively less. during analysis, we found that extent of urbanization is not a statistically significant independent variable for explaining households’ possession of mobile phone. hence, here we do not estimate the regression equation separately for rural districts. further, instead of the variables share_of_villages_with_broadband_coverage and index_average_service_quality_broadband, we use the independent variables corresponding to mobile network coverage and quality of its services. also, as compared to computer, skills required to use a mobile phone is comparatively less. hence, instead of considering educational level as secondary school and above, we take share_of_persons_educated_middle_school_and_above. a supply side explanatory variable, degree_of_competition, is introduced to indicate level of competition among mobile network service provider. the breusch-pagan test suggest presence of heteroskedasticity, which was australasian journal of information systems guha & mukerji 2021, vol 25, research article determinants of digital divide using demand-supply framework 22 addressed through robust estimator provided in stata. however, on plotting the residual with the fitted value, we found some outliers. after removing these outliers, we have again estimated the equation. the ramsey test of specification on this estimation suggests no specification error due to omitted variables (appendix a5, equation1). results (table 4) indicate that the independent variables: share_of_pop_age_ between_15to35, share_of_persons_educated_middle_school_and_above, share_of_households_with_electricity and index_average_service_quality_mobile_network have positive statistically significant relationship with the dependent variable share_of_households_with_mobile_phones. the independent variables: share_of_scst_population and ratio_of_poor_and_middle_to_welloff_households have negative statistically significant relationship with the dependent variable. the variables: share_of_workers_in_service_sector, degree_of_competition, share_of_villages_with_mobile_network_coverage and share_of_urban_population do not have statistically significant impact. the largest value of co-efficient is that of share_of_pop_age_between_15to35, followed by the share_of_scst_population and index_average_service_quality_mobile_network. the other significant factors that are influencing the possession of mobile are level of education, electricity connection and economic condition of the household. the analysis indicates that with regards to household possession of mobile phones, the ruralurban divide is not statistically significant. however, availability of the electricity and quality of mobile network services are important. young people with basic level of education are embracing the mobile. in fact, we estimated the same equation by replacing the variable share_of_person_educated_middle_school_and_above with share_of_persons_educated_below_middle_school and share_of_pop_age_between_15to35 with share_of population_with_age_not_between_15to35 (appendix a5, equation 2). both the new variables show statistically significant inverse relationship with the dependent variable share_of_households_with_mobile_phones. however, household possession of mobile phones caused negatively by the poorer economic conditions and being sc-st. however, household possession of mobile phones does not appear to significantly different between those working in service sectors and others. 5 policy implications the paper was an attempt to empirically investigate the determinants of digital divide in india using available public data. the census of india 2011, collected for the first time, household possession of four ict instruments – landline, computer, computer with internet and mobile phone. applying the demand-supply equation as our theoretical framework, we use this data to separately estimate the reduced form demand equation for computers with internet and mobile phones. our findings indicate that three factors – (1) spread of education, (2) being young, and (3) being member of sc-st community significantly influence the possession of both the ict instruments. while the first two factors positively influence possession of ict instruments, the last factor has a negative effect. household possession of computers with internet is higher in districts having more educated, young and urban population. presence of educated young people also has very large positive impacts on possession of mobile phone. however, as compared to household possession of computer with internet, mobile phones did not require higher level of education. similarly, working in service sector and residence in urban area does seem to encourage household australasian journal of information systems guha & mukerji 2021, vol 25, research article determinants of digital divide using demand-supply framework 23 possession of computer with internet but these two factors do not have any significant influence on household possession of mobile phones. however, availability of electricity and mobile network service quality are important factors for household demand of mobile phones in both rural and urban areas. on the whole, findings indicate that in 2011, the digital divide in india was largely a reflection of existing socio-economic divides. thus, a key policy implication is that efforts and interventions to close the digital divide should not only look at supply side interventions but also focus on policies that reduce existing socio-economic divides and thereby contribute to greater diffusion of icts. the results also show that while possession of the two ict instruments were not sensitive to availability of network, they were sensitive to quality of service of broadband and mobile networks. this might be because customers in the year 2011 did not distinguish between nonavailability of network and poor quality of network services. hence, digital divide is likely to lessen if the price of ict instruments and data access is reduced, along with improvement in the quality of network services. firms focussing on better network services at lower rates will not only capture greater market share but also expand the market size for the industry. on the whole, diffusion of mobile phones is more broad-based than computers with internet. we did not find statistically significant rural-urban divide or essentiality of higher education or specific service oriented occupational requirement for mobile phones. unlike computers with internet, there is not much variability in household possession of mobile phone among various occupational categories. this bodes well for current policy emphasis on delivery of mobile-based services. however, being a sc-st household or a relatively poor household does seem to negatively impact household possession of both the ict instruments. results indicate that districts with greater share of sc-st population have substantially lower access to mobile phone. hence a targeted policy for increasing accessibility and usage of mobile phones for scsts will further reduce the digital divide faster. further, to increase the spread of computer with internet, spread of education, especially of women, at secondary and above level is very important. so, policy intervention and schemes that aim to incentivise schooling of girl child by linking it with a conditional cash transfer scheme will lead to reduction in gender-based digital divide. 6 concluding remarks this paper is a modest attempt to examine the nature and extent of digital divide at the district level in india and the factors determining the relatively low and unequal household access to icts. the context is set against a high economic growth led by it/ites sectors but relatively low and unequal diffusion of icts across districts. a demand-supply framework was developed and applied on a large, and consistent public dataset to arrive at the results. the analysis provides evidence that the district level digital divide in india is a reflection of existing socio-economic divide. the overall low penetration of icts in india is because of both uneven spread of required ict infrastructure and difference in socio-economic characteristics of households. thus, policy makers need to simultaneously address constraints on both demand side factors, coming out of socio-economic divide as well as on ict infrastructure. policy measures that aim to reduce the existing socio-economic divides including enhancement of employment opportunities and education for women and sc-sts, income of australasian journal of information systems guha & mukerji 2021, vol 25, research article determinants of digital divide using demand-supply framework 24 the poor, along with reduction of infrastructural gap between urban and rural, will contribute towards greater diffusion of icts. the report of 75th round of nsso survey on household expenditure on education tells us that there is a reduction in digital divide in comparison to that of 2011. however, this report does not provide information on households’ possession of mobile phone which appears to be playing a key role in increasing diffusion of internet and related services. as more data becomes available, future research can examine if any changes in socio-economic divides bridging the digital divide or it is primarily due to the better ict infrastructure and lowering price has reduced the digital divide. the findings of this study will serve as a baseline for such studies examining digital divide in india. references ahn, h., & lee, m. (1999). an econometric analysis of the demand for access to mobile telephone networks. information economics and policy, 11(3), 297–305. https://doi.org/10.1016/s0167-6245(99)00016-5 andonova, v. (2006). mobile phones, the internet and the institutional environment. telecommunications policy, 30(1), 29–45. https://doi.org/10.1016/j.telpol.2005.06.015 baliamoune-lutz, m. (2003). an analysis of the determinants and effects of ict diffusion in developing countries. information technology for development, 10(august 2002), 151–169. https://doi.org/10.1002/itdj.1590100303 barman, h., kanti, m., & nath, h. k. (2018). the telecommunications divide among indian states. telecommunications policy, 42(7), 530–551. https://doi.org/10.1016/j.telpol.2018.05.003 billon, m., marco, r., & lera-lopez, f. (2009). disparities in ict adoption : a multidimensional approach to study the cross-country digital divide. telecommunications policy, 33(10–11), 596–610. https://doi.org/10.1016/j.telpol.2009.08.006 chinn, m. d., & fairlie, r. w. (2007). internet penetration the determinants of the global digital divide : a cross-country analysis of computer and internet penetration. oxford economic papers, 59(1), 16–44. https://doi.org/10.1093/oep/gp1024 coria, s. r., mondragón-becerra, r., pérez-meza, m., ramírez-vásquez, s. k., martínezpeláez, r., barragán-lópez, d., & ávila-barrón, o. r. (2013). ct4rdd: classification trees for research on digital divide. expert systems with applications, 40(14), 5779–5786. https://doi.org/10.1016/j.eswa.2013.04.002 cuervo, m. r. v., & menéndez, a. j. l. (2006). a multivariate framework for the analysis of the digital divide: evidence for the european union-15. information & management, 43(6), 756–766. https://doi.org/10.1016/j.im.2006.05.001 dewan, s., & riggins, f. j. (2005). the digital divide : current and future research directions. journal of the association for information systems, 6(12), 298–337. dobransky, k., & hargittai, e. (2006). the disability divide in internet access and use, 4462. https://doi.org/10.1080/13691180600751298 australasian journal of information systems guha & mukerji 2021, vol 25, research article determinants of digital divide using demand-supply framework 25 doong, s. h., & ho, s. (2012). electronic commerce research and applications the impact of ict development on the global digital divide. electronic commerce research and applications, 11(5), 518–533. https://doi.org/10.1016/j.elerap.2012.02.002 dossani, r., misra, d. c., & jhaveri, r. (2005). enabling ict for rural india. project report. asiapacific research center, stanford university and national informatics centre, government of india. retrieved from http://www.mssrf-nva.org/speeches/enabling ict for rural india by asia-pacific research center, stanford university, national informatics centre by rafiq dossani, d.c. misra, roma jhaveri, november 2005.pdf dutta, d. (2018). development under digital divide in india. in development under dualism and digital divide in twenty-first century india. singapore: springer. ghosh, saibal. (2016). does mobile telephony spur growth ? evidence from indian states. telecommunications policy, 40(10–11), 1020–1031. https://doi.org/10.1016/j.telpol.2016.05.009 ghosh, sajal, & prasad, r. (2014). telephone penetrations and economic growth: evidence from india, (march). https://doi.org/10.1007/s11066-012-9067-z gulati, g. j., & yates, d. j. (2012). different paths to universal access : the impact of policy and regulation on broadband diffusion in the developed and developing worlds. telecommunications policy, 36(9), 749–761. https://doi.org/10.1016/j.telpol.2012.06.013 gupta, k. (2018). mobile internet penetration in rural india is just 18%: report. livemint. retrieved from https://www.livemint.com/technology/obzowmvu6cxhmpcdpldyfm/mobileinternet-penetration-in-rural-india-is-just-18-repo.html gupta, r., & jain, k. (2012). diffusion of mobile telephony in india : an empirical study. technological forecasting & social change, 79(4), 709–715. https://doi.org/10.1016/j.techfore.2011.08.003 haenssgen, m. j. (2018). the struggle for digital inclusion : phones , healthcare , and marginalisation in rural india. world development, 104, 358–374. james, j. (2007). from origins to implications: key aspects in the debate over the digital divide. journal of information technology, 22, 284–295. https://doi.org/10.1057/palgrave.jit.2000097 kaushik, p. d., & singh, n. (2004). information technology and broad-based development : preliminary lessons from north india. world development, 32(4), 591–607. https://doi.org/10.1016/j.worlddev.2003.11.002 kiiski, s., & pohjola, m. (2002). cross-country diffusion of the internet. information economics and policy, 14, 297–310. lera-lopez, m. b. f., & marco, r. (2016). ict use by households and firms in the eu : links and determinants from a multivariate perspective. review of world economics, 152(4), 629– 654. https://doi.org/10.1007/s10290-016-0259-8 mukerji, m. (2013). icts and development: a study of telecentres in rural india. palgrave macmillian. https://doi.org/10.1057/9781137005540 australasian journal of information systems guha & mukerji 2021, vol 25, research article determinants of digital divide using demand-supply framework 26 narayana, m. r. (2011). telecommunications services and economic growth : evidence from india. telecommunications policy, 35(2), 115–127. https://doi.org/10.1016/j.telpol.2010.12.001 nishida, t., pick, j. b., & sarkar, a. (2014). japan’s prefectural digital divide: a multivariate and spatial analysis. telecommunications policy, 38(11), 992–1010. https://doi.org/10.1016/j.telpol.2014.05.004 nishijima, m., ivanauskas, t. m., & sarti, f. m. (2017). evolution and determinants of digital divide in brazil (2005–2013). telecommunications policy, 41(1), 12–24. https://doi.org/10.1016/j.telpol.2016.10.004 parayil, g. (2005). political economy and information capitalism in india. palgrave macmillian. park, s. r., choi, d. y., & hong, p. (2015). club convergence and factors of digital divide across countries. technological forecasting and social change, 96, 92–100. https://doi.org/10.1016/j.techfore.2015.02.011 pick, j. b., & azari, r. (2008). global digital divide : influence of socioeconomic , governmental , and accessibility factors on information technology. information technology for development, 14(2), 91–115. https://doi.org/10.1002/itdj pick, j. b., & nishida, t. (2015). digital divides in the world and its regions: a spatial and multivariate analysis of technological utilization. technological forecasting and social change, 91, 1–17. https://doi.org/10.1016/j.techfore.2013.12.026 pick, j. b., & sarkar, a. (2015). india’s digital divide. in the global digital divides: explaining change. berlin: springer-verlag. pick, j. b., sarkar, a., & johnson, j. (2015). united states digital divide: state level analysis of spatial clustering and multivariate determinants of ict utilization. socio-economic planning sciences, 49, 16–32. https://doi.org/10.1016/j.seps.2014.09.001 rath, b. n. (2016). does the digital divide across countries lead to convergence ? new international evidence. economic modelling, 58, 75–82. https://doi.org/10.1016/j.econmod.2016.05.020 scheerder, a., deursen, a. van, & dijk, j. van. (2017). telematics and informatics determinants of internet skills , uses and outcomes . a systematic review of the second and third-level digital divide. telematics and informatics, 34(8), 1607–1624. https://doi.org/10.1016/j.tele.2017.07.007 singh, s. k. (2008). the diffusion of mobile phones in india. telecommunications policy, 32(3), 642–651. https://doi.org/10.1016/j.telpol.2008.07.005 sujarwoto, s., & tampubolon, g. (2016). spatial inequality and the internet divide in indonesia. telecommunications policy, 40(7), 602–616. https://doi.org/10.1016/j.telpol.2015.08.008 vicente, m. r., & lopez, a. j. (2011). assessing the regional digital divide across the european union-27. technology policy, 35, 220–237. https://doi.org/10.1016/j.telpol.2010.12.013 vu, k. m. (2011). ict as a source of economic growth in the information age : empirical evidence from the 1996 – 2005 period. telecommunications policy, 35(4), 357–372. https://doi.org/10.1016/j.telpol.2011.02.008 australasian journal of information systems guha & mukerji 2021, vol 25, research article determinants of digital divide using demand-supply framework 27 zhang, x. (2017). telematics and informatics exploring the patterns and determinants of the global mobile divide. telematics and informatics, 34(1), 438–449. https://doi.org/10.1016/j.tele.2016.06.010 zhao, f., collier, a., & deng, h. (2014). a multidimensional and integrative approach to study global digital divide and e-government development. information technology & people, 27(1), 38–62. https://doi.org/10.1108/itp-01-2013-0022 copyright: © 2021 guha & mukerji. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v25i0.3029 http://creativecommons.org/licenses/by-nc/3.0/au/ australasian journal of information systems guha & mukerji 2021, vol 25, research article determinants of digital divide using demand-supply framework 28 appendix a1: select studies on digital divide in india author / year focus of the study period / dataset key findings haenssgen (2018) examines if uneven distribution of mobile phones benefit all equally in the context of rural healthcare services 2005-2012 panel data of 12,003 rural households in ihds database rapid mobile phone diffusion creates opportunity to improve people’s access to healthcare in rural india, but it also creates new forms of marginalisation among poor rural households. barman, dutta and nath (2018) examines patterns of distribution dynamics, and the drivers of telecommunications (telecom) services across 16 different states of india 2001-2015 16 states of india – trai database the interstate gap in telecommunications services has been declining and it is converging. per capita income and network externality are significant determinants of teledensity across states in india. literacy rate and relative size of the service sector are independently significant predictors of teledensity. ghosh (2016) analyse the impact of mobile telephony on economic growth. panel data for the years 2001-12 of 14 major indian states mobile telephony exerts a positive and statistically significant impact on growth. the magnitude of the response differs across states with high and low mobile penetration. gupta and jain (2012) to identify factors affecting the adoption of mobile telephony in rural india and examine their impact on its adoption itu databases from 1998 to 2009; trai the study reveals that competition and government intervention played a significant role in accelerating the diffusion speed of mobile telephony by making the technology affordable. it is found that mobile telephony is a substitute for fixed line telephony in india. narayana (2011) estimates the growth contribution of telecom services by public and private sectors. socioeconomic determinants of demand for telecom services are estimated for fixed and mobile phones sample survey of households significant negative impact of both access and usage price on household demand for fixed and mobile phones and a positive impact of income variables; substitutability of mobile phones for fixed phones distinguishes the importance of social caste, education level, nature or occupation, age of the head of the household, and family size. australasian journal of information systems guha & mukerji 2021, vol 25, research article determinants of digital divide using demand-supply framework 29 appendix a2: variables used and their data-source sl. no. factor variable name source of data used in regression equation of: comput er with internet mobil e phone dv1 ict instrument share_of_households_having_ computer_with_internet table haa, census 2011 dv dv2 ict instrument share_of_households_having_ mobile_phones table haa, census 2011 dv 1 economic status ratio_of_poor_and_middle_to_ welloff_households tables haa, census 2011 iv iv 2 education and gender share_of_persons_educated_mi ddle_school_and_above tables edu, census 2011 iv 3 education and gender share_of_women_educated_se condary_and_above tables edu, census 2011 iv 4 occupational profile share_of_workers_in_service_s ector tables pca, census 2011 iv iv 5 age profile share_of_pop_age_between_15 to35 tables pca, census 2011 iv iv 6 social status/ backwardness of a district share_of_scst_pop tables pca, census 2011 iv iv 7 extent of urbanization share_of_urban_population tables pca, census 2011 iv iv 8 infrastructure/ electricity share_of_households_with_ele ctricity tables haa, census 2011 iv iv 9 infrastructure/ broadband network coverage share_of_villages_with broadband_coverage annex-ii, report to dmu pmo for the month of march,2011, broadband coverage of village panchayats under bharat nirman-ii iv 10 infrastructure/ broadband service quality index_average_service_quality _broadband annexture 4.3, the indian telecom service performance indicators, jan-mar, 2011, trai iv 11 infrastructure/ mobile network coverage share_of_villages_with_mobile _network tables dca, census 2011 iv 12 infrastructure/ mobile network service quality index_of_mobile_network_serv ice_quality annexture 4.1, the indian telecom service performance indicators, jan-mar, 2011, trai iv 13 competitiveness degree_of_competition annexture 1.2, the indian telecom service performance indicators, jan-mar, 2011, trai iv iv: independent variable dv: dependent variable australasian journal of information systems guha & mukerji 2021, vol 25, research article determinants of digital divide using demand-supply framework 30 appendix a3 sl. no. variable name var1 var2 var3 var4 var5 var6 var7 var8 var9 var1 share_of_scst_ pop 1 var2 share_of_workers_in_service_sector -0.30* 1 var3 ratio_of_poor_and_middle_to_welloff_households 0.25* -0.07 1 var4 share_of_women_educated_secondary_and_above -0.30* 0.75* -0.18* 1 var5 share_of_persons_educated_middle_school_and_above -0.29* 0.73* -0.15* 0.95* 1 var6 share_of_pop_age_between_15to35 0.09* 0.41* 0.067 0.34* 0.39* 1 var7 share_of_households_with_ electricity -0.09* 0.53* -0.20* 0.66* 0.64* 0.48* 1 var8 index_average_service_quality_broadband -0.19* -0.01 -0.22* 0.26* 0.18* 0.11* 0.38* 1 var9 share_of_villages_with broadband_coverage -0.27* 0.40* -0.21* 0.53* 0.42* 0.05 0.32* 0.27* 1 var10 share_of_urban_population -0.24* 0.74* -0.19* 0.71* 0.64* 0.44* 0.56* 0.33* 0.39* note: ‘*’ indicates statistically significant at 5% level table a3.1: correlation co-efficient of the independent variables used in reduced demand equation estimation for computers_with_internet sl.no. variable name var1 var2 var3 var4 var5 var6 var7 var8 var9 var1 share_of_scst_population 1 var2 share_of_workers_in_ service_sector -0.30* 1 var3 ratio_of_poor_and_middle_to_welloff_households 0.25* -0.07 1 var4 share_of_persons_educated_middle_school_and_above -0.28* 0.73* -0.15* 1 var5 share_of_pop_age_between_15to35 0.09* 0.41* 0.06 0.39* 1 var6 degree_of_competition 0.23* -0.20* 0.06 -0.29* -0.05 1 var7 share_of_households_with_electricity -0.09* 0.54* -0.20* 0.64* 0.48* -0.00 1 var8 share_of_villages_with_mobile_network -0.30* 0.30* -0.25* 0.39* 0.17* -0.00 0.54* 1 var9 index_mobile_network_service_quality -0.42* 0.22* -0.19* 0.16* -0.04 0.19* 0.14* 0.26* 1 var10 extent_of_urbanisation -0.24* 0.74* -0.19* 0.64* 0.45* 0.16* 0.56* 0.32* 0.14* note: ‘*’ indicates statistically significant at 5% level table a3.2: correlation co-efficient of the independent variables used in reduced demand equation estimation for mobile_phone australasian journal of information systems guha & mukerji 2021, vol 25, research article determinants of digital divide using demand-supply framework 31 sl. no. variable name mean std. dev. min max dv 1 share_of_households_having_computer_with_internet 0.0236 0.0349 0.0020 0.2420 dv 2 share_of_households_having_mobile_phones 0.5119 0.1447 0.0800 0.7960 1 ratio_of_poor_and_middle_to_welloff_households 0.2014 0.6187 0.0020 9.9700 2 share_of_persons_educated_middle_school_and_above 0.3187 0.1038 0.0732 0.5972 3 share_of_women_educated_secondary_and_above 0.1634 0.0895 0.0204 0.4448 4 share_of_workers_in_service_sector 0.4370 0.2102 0.1015 0.9910 5 share_of_pop_age_between_15to35 0.3723 0.0258 0.2850 0.5680 6 share_of_scst_pop 0.3259 0.2259 0.0034 0.9858 7 share_of_urban_population 0.2642 0.2112 0.0000 1.0000 8 share_of_households_with_electricity 0.6588 0.2833 0.0190 0.9970 9 share_of_villages_with broadband_coverage 0.0046 0.0023 0.0002 0.0100 10 index_of_broadband_service_quality 0.5863 0.1148 0.3333 0.8094 11 share_of_villages_with_mobile_network 0.8446 0.0450 0.7663 0.9285 12 index_of_mobile_network_service_quality 0.7659 0.2167 0.0033 1.0000 13 degree_of_competition 0.9404 0.0956 0.5888 1.1127 table a3.3: descriptive statistics of the variables of reduced form demand estimation australasian journal of information systems guha & mukerji 2021, vol 25, research article determinants of digital divide using demand-supply framework 32 appendix a4: demand estimation of computer with internet dependent variable: logistic transformation of share_household_computers_with_internet all districts a rural districtsb independent variables equation1 equation 2 equation 3 share_of_sc-st_population -.3285015 (.0717976)*** -.2982297 (0.0777549)*** -.3122073 (.0769693)*** share_workers_in_service_sector 1.385045 (0.1413372)*** 1.677573 (0.1490896)*** 1.845224 (0.147666)*** ratio_of_poor_and_middle_to_well-off_households -.0297621 (.0246302) -.0261717 (0.0265226) -.0473291 (0.0243419)* share_of_women_educated_secondary_and_above 4.598076 (.3008528)*** 4.378131 (.3395891)*** share_of_person_educated_at_middle_school_and_above ………….. 2.593129 (0.2410297)*** ……….. share_of_population_age_ between_15 to 35 3.209441 (.7240674)*** 2.068897 (0.7799134)*** 2.50101 (.8418013)*** share_of_households_with_electricity 0.0008851 (.0768176) .1250661 (0.0819864) .1127426 (.0829464) index of average service quality of broadband 0.4568503 (.1642788)*** .6953176 (0.176321)*** .7294781 (.1679193)*** share_of_villages_with_broadband_coverage -0.4743803 (7.362669) 20.29096 (7.670942)*** 2.764722 (8.492882) share_of_urban_population 1.181709 (0.1243345)*** 1.372428 (.1325794)*** ………… adjusted r2 0.8753 0.8554 0.7395 f test f(9, 624) = 494.62*** f( 9, 624) = 417.13*** f( 8, 503) = 182.33*** information criteria aic = 480.14 bic = 524.6 aic=561.34 bic= 605.85 aic=368.21 bic=406.36 heteroskedasticityc χ2 = 0.22 χ2 = 0.79 χ2 = 0.89 normalityd χ2 = 2.43 χ2 =39.49*** χ2 =0.43 functional forme f(3, 621) = 1.66 f(3, 621) = 3.51** f(3, 500) = 0.31 note: standard errors are in bracket (a) demand estimation of computers_with_internet (equation 1 &2) for all districts (b) demand estimation of computers_with_internet (equation 3) for rural districts (defined as having urban population less than 40 percent of the total population). (c) breusch–pagan/ cook-weisberg heteroskedasticity test. in case of the existence of heteroscedasticity, adjustment was made to make it robust. (d) test of skewness and kurtosis of residuals. since the degree of freedom is more than 500, the assumption of normality can be made (due to law of large number) without testing it. (e) ramsey reset test using powers of the fitted values. *statistically significant at 10%. **statistically significant at 5% *** statistically significant at 1% australasian journal of information systems guha & mukerji 2021, vol 25, research article determinants of digital divide using demand-supply framework 33 appendix a5: demand estimation of mobile dependent variable: logistic transformation of share_of_households_having_mobile_phones all districts a independent variables equation1 equation 2 share_of_sc-st_population -1.289316 (.0977918)*** -1.289316 (.0977918)*** share_workers_in_service_sector -.1609256 (.1524257) -.1609257 (.1524257) ratio_of_poor_and_middle_to_well-off_households -.0580019 (.0232608)** -.0580019 (.0232608)** share_of_person_educated_at_middle_school_and_above .679934 (.2688573)** ……. share of person educated at below middle school …….. -.6799339 (.2688573)** share_of_population_age_ between_15 to 35 years 5.097142 (.8690987)*** ………. share of population age not within 15 to 35 years ……… -5.097142 (.8690986)*** degree of competition -.1537522 (.1835348) -.1537523 (.1835348) share_of_households_with_electricity .2791257 (.0976886)*** .2791257 (.0976886 )*** share_of_villages_with_mobile_network .1677409 (.11098) .1677408 (.11098) index of average service quality of mobile network 1.158066 (.4109839)*** 1.158067 (.4109839)*** share_of_urban_population .0230995 (.1228256) .0230996 (.1228256) adjusted r2 0.5185 0.5185 f test f( 10, 592) = 88.40 f( 10, 592) = 88.40 information criteria aic = 630.43 bic = 678.86 aic = 630.43 bic = 678.86 heteroskedasticityb robust robust normalityc χ2 = 1.449 χ2 = 1.449 functional formd f(3, 589) = 2.09 f(3, 589) = 2.09 note: standard errors are in bracket (a) demand estimation of demand estimation of mobile (equation 1 &2) for all districts (b) breusch–pagan/ cook-weisberg heteroskedasticity test. in case of the existence of heteroscedasticity, adjustment was made to make it robust. (c) test of skewness and kurtosis of residuals. since the degree of freedom is more than 500, the assumption of normality can be made (due to law of large number) without testing it. (d) ramsey reset test using powers of the fitted values. *statistically significant at 10%. **statistically significant at 5% *** statistically significant at 1% paper title (paper title style) australasian journal of information systems nguyen et al. 2021, vol 25, research article trust types and mediating effect 1 trust types and mediating effect of consumer trust in mpayment adoption: an empirical examination of vietnamese consumers tuan anh nguyen rmit university, australia s3641842@student.rmit.edu.au hiep-cong pham rmit university vietnam, viet nam martin dick rmit university, australia joan richardson rmit university, australia abstract this study employs a quantitative method to investigate different types of trust in m-payment adoption. it aims to overcome the limitation of previous studies which are a lack of differentiating trust types and investigating any mediating effect to m-payment adoption. data of the study was collected in vietnam, one of fastest growing m-payment usage markets globally in 2019. the research found significant and positive impacts of m-payment provider trust, institution-based trust, and seller trust on the overall consumer trust, which then fully mediates the relationships of three trust types and m-payment adoption. the study also revealed that technology trust is embedded in m-payment provider trust, suggesting that the m-payment provider is considered fully responsible for ensuring technology protection from the perspective of the m-payment consumers. the results enable researchers to better understand trust characteristics in m-payment adoption as well as technology adoption in general. in addition, the findings are beneficial to practitioners such as policy makers, consultants, and m-payment service providers to improve different elements of consumer trust, leading to higher m-payment adoption. keywords: mobile payment adoption, consumer trust, trust types, technology adoption 1 introduction mobile payment (m-payment) refers to financial transactions made via mobile devices, such as, tablets and mobile phones (stringfellow, 2018). in other words, m-payment consumers can use mobile devices to pay for goods or services that they purchase, instead of using cash, debit cards, credit cards or any other type of bank card. m-payment is considered to be a subset of mobile commerce (m-commerce), as well as, electronic commerce (e-commerce) which processes the payment transaction for customers when purchasing goods or services (kreyer, pousttchi, & turowski, 2002; mobiforge, 2014). there are two types of m-payment namely remote and proximity m-payment. the former refers to m-payment transactions which can be conducted from a distance and consumers do not need to directly interact with sellers or the merchants’ point of sale systems (pos). common examples of remote m-payment are carrier billing, short message service (sms) australasian journal of information systems nguyen et al. 2021, vol 25, research article trust types and mediating effect 2 payments or using mobile phones to pay online via applications, such as, paypal (emily, 2018). in contrast, proximity m-payment methods using near field communications (nfc), allow customers to directly pay for their purchase using their mobile phones with m-payment applications, such as, apple pay, samsung pay or google pay, or scanning a seller’s qr (emily, 2018). m-payment is considered as a state-of-the-art payment method in the modern world as it builds upon the internet and mobile devices (kolaki, 2017). global use of mpayment is forecasted to increase 28% by 2022 and surpass cash and credit cards in the longer term, which will contribute to the future of a cashless world (merchantsavvy, 2019). in the current situation where covid-19 has spread across the globe, the usage of m-payment is growing at an even faster rate as it is considered as a safer way to pay rather than traditional payment transactions that require cash (researchandmarkets, 2020).in the area of m-payment adoption, trust is defined as customers/consumers’ beliefs and willingness to rely on mpayment for transactions (adapted from alhulail, 2018; mcknight, choudhury, & kacmar, 2002; xin, techatassanasoontorn, & tan, 2015). trust plays an important role in m-payment adoption in both research and practice. in research, many studies empirically found a significant effect of trust on m-payment adoption by individual consumers (andreev, pliskin, & rafaeli, 2012; liu, 2012; patil, rana, dwivedi, & abu-hamour, 2018).(andreev et al. 2012; liu 2012; patil et al. 2018). in practice, a lack of trust is reported as one of the main barriers to m-payment adoption, resulting in low rates of technology service use (asatryan, 2017; shuhaiber, 2016). it can be summarised that trust is one of the key drivers for intention to adopt, as well as, acceptance of m-payment (alalwan, dwivedi, & rana, 2017). however, most previous studies in m-payment adoption assess trust as a single construct and examine the impact of trust on other factors, such as, perceived usefulness and intention to adopt m-payment without differentiating types of trust constituting to the more general term. researchers have pointed out that trust is a complex phenomenon, thus it should be modelled as a multidimensional or multifaceted construct (hillman & neustaedter, 2017; jimenez, sanmartin, & azuela, 2016). in particular, exploring different types of trust in m-commerce or mpayment enables researchers to better understand trust as a phenomenon which ultimately allows predictions of consumer adoption (meng, min, & li, 2008; min, meng, & zhong, 2008; nguyen, dick, & pham, 2020). consequently, the identification of different types of trust in mpayment adoption needs to be addressed further (nguyen et al., 2020).(nguyen et al. 2020). this study argues that overall consumer trust plays a mediating role between various trust types and m-payment adoption. such trust types are initiated from multiple aspects of mpayment context. in particular, it aims to ascertain the impact of four types of trust including m-payment provider trust, technology trust, institution-based trust, and seller trust, which are well-defined from related literature, on overall consumer trust which in turn influences mpayment usage. the findings of this study not only contribute to the literature of trust in technology, as well as, in m-payment adoption, but also helps practitioners to improve customer trust, thereby increasing the adoption of m-payment services. the data collection was undertaken in vietnam which had the fastest global m-payment adoption growth rate, from 37% in 2018 to 61% in 2019 (pwc, 2019). consumers choice to use m-payments in vietnam is voluntary. by 2018, the number of smartphone users in vietnam was about 32.43 million which accounted for around 33% of the population, and it was forecasted to increase to 40% by 2021 (statista, 2018). vietnam’s financial technological market reached us$4.4 billion in 2017 and was forecasted to increase to us$7.8 billion by 2020 australasian journal of information systems nguyen et al. 2021, vol 25, research article trust types and mediating effect 3 (fintechnews, 2018a). the booming e-commerce market and support from the vietnamese government were expected to lead to a boom in digital payments, especially m-payments in vietnam (fintechnews, 2018b). this paper is organised as follows. the next section presents the literature review, description of the limitations of existing studies, and research questions. then the research model and hypotheses are presented. the fourth and fifth sections in turn present the research methodology and data analysis. finally, the implications, limitations, and future research are discussed. 2 literature review 2.1 previous studies about trust in e-commerce, m-commerce, and mpayment adoption the term trust is used frequently in daily life, however it is still difficult to provide a comprehensive and full definition of trust across different disciplines because it is a complex phenomenon including distinctive aspects (mcknight & chervany, 2001a). trust is recognised as a multi-disciplinary term due to existing research exploring and examining trust in a wide variety of contexts and incorporating perspectives of different disciplines like psychology, economics, commerce, management and behavioural science (mcknight & chervany, 2001b). in the areas of e-commerce and m-commerce, trust has received attention in a number of studies (e.g. bilgihan, 2016; hallikainen & laukkanen, 2018; hillman & neustaedter, 2017; lin, wang, wang, & lu, 2014; lu, fan, & zhou, 2016; malaquias & hwang, 2016; piao, wang, & yang, 2012; rouibah, lowry, & hwang, 2016). mcknight and chervany (2001b) applied their interdisciplinary model of high-level trust concepts to e-commerce to propose a model of ecommerce customer relationship trust to improve the service provided to customers and relationships built with a view to a positive impact on the business performance. in e-commerce, trustors are e-commerce consumers, trustees are e-vendors. disposition to trust reflects the extent to which an e-commerce consumer “has a general propensity/tendency to depend on most people across most situations” (mcknight & chervany, 2001b, p. 43). institution-based trust is the belief that the necessary conditions, such as, regulations, laws, security, and protocols of the internet, are present in order to increase the likelihood of participating in e-commerce (mcknight et al., 2002). interpersonal trust refers to a person trusting another one across diverse contexts (mcknight & chervany, 2001b). the following four constructs including disposition to trust, institution-based trust, trusting beliefs and trusting intentions are subdivided into lower-level constructs to be measured via relevant scales: • disposition to trust includes faith in humanity and a trusting stance, • institution-based trust includes structural assurance and situational normality, • trusting beliefs encompass competence, benevolence, integrity, and predictability belief, and • trusting intentions cover willingness to depend and subjective probability of depending. australasian journal of information systems nguyen et al. 2021, vol 25, research article trust types and mediating effect 4 mcknight et al. (2002) integrated their model of e-commerce customer relationship trust with the theory of reasoned action (tra) (fishbein & ajzen, 1975) to propose and test the web trust model in e-commerce (figure 1). figure 1: the trust model in e-commerce (mcknight et al., 2002, p. 341) differently, siau and shen (2003) argued that the process of developing the trust of ecommerce customers is dynamic and time-consuming, thus they proposed two types of trust namely, initial and ongoing trust. when customers begin to conduct their first transactions, initial trust starts with merchants, which may be first based on information gathered about advantages, such as, convenience or cost efficiency, and reward attraction. then continuous trust follows, which may result in forming firm consumer loyalty is developed once customers are convinced to buy further products/services or repeat transactions. customers then evaluate their satisfaction, which given their repeat business must have included positive experiences with vendors. in contrast, if customers have a bad experience, this could result in them dropping out due to distrust of the merchants. figure 2 illustrates the impact of customer loyalty on consumer behaviour. figure 2: the e-commerce trust development life cycle (siau & shen, 2003, p. 92) australasian journal of information systems nguyen et al. 2021, vol 25, research article trust types and mediating effect 5 siau, sheng, and nah (2003) suggested a framework for trust in m-commerce with five groups of factors including vendor and website characteristics, technology of wireless services and mobile devices and other factors, such as, legal regulations or third-party certification. figure 3 illustrates the framework for trust in m-commerce (siau et al., 2003, p. 88). figure 3. proposed framework for trust in m-commerce (siau et al. 2003, p. 88) more broadly, siau and shen (2203, p. 92) proposed two components for building customer trust in m-commerce which are mobile technology and mobile vendor trust (see figure 4). figure 4:. framework for m-commerce trust building (siau and shen 2003, p. 92) a lack of trust has been recognised as a major obstacle for any m-commerce service (chen & dhillon, 2003; joubert & belle, 2009). m-payment is a significant component of m-commerce services as it allows mobile users to conduct financial transactions to finish their m-commerce transactions. trust plays an important role in m-payment adoption therefore it has been studied in many m-payment studies (e.g. gao & waechter, 2017; hillman & neustaedter, 2017; nguyen et al., 2020; patil et al., 2018; qasim & abu-shanab, 2016; slade, dwivedi, piercy, & williams, 2015; xin et al., 2015). most previous studies examined the effect of trust as a single construct on intention to adopt m-payment services without the identification of which types of trust constituting overall trust of m-payment consumers (nguyen et al., 2020). many authors adopted the unified theory of acceptance and use of technology (utaut) or technology acceptance model (tam) as a theoretical underpinning of their research project involving trust as a predictor of m-payment australasian journal of information systems nguyen et al. 2021, vol 25, research article trust types and mediating effect 6 adoption (e.g. andreev et al., 2012; liu, 2012; mingxing, jing, & yafang, 2014; zhou, 2013). zhou (2011) adopted tam and diffusion of innovation theory (doi) to investigate the effect of initial trust on user adoption of m-payment services, in china, and recognised the significant effect of perceived ubiquity, security and ease of use on initial trust which positively influence m-payment usage intention. when researching drivers of the willingness to use m-payment in israel, andreev et al. (2012) extended tam and included doi, and found a direct and significant impact of vendor trust, and a non-significant impact of mechanism trust on willingness to use m-payment services. similarly, based on tam and doi, liu (2012) collected data from 200 university students in jiangsu, china to test consumers’ intention to use m-payment services. the outcome indicated that trust is one of the most important variables affecting intention to adopt m-payment services. mingxing et al. (2014) refined factors from tam and recognised the important effect of customers’ perceived risk and trust on m-payment adoption in china. yan and yang (2014) used tam2 to empirically examine user adoption of m-payments in china, and recognised the vital effect of perceived ease of use, perceived usefulness, structural assurance, and ubiquity on trust, and the impact of trust on m-payment adoption. phonthanukitithaworn, sellitto, and fong (2015) extended tam to test user intentions to adopt m-payment services in thailand and found that perceived trust had a significant positive influence on behavioural intention. the study of slade, williams, dwivedi, and piercy (2015) underpinned by utaut2, and collected data in the uk, revealed that trust was an important predictor for intention to adopt m-payment services. based on utaut and tam, qasim and abu-shanab (2016) examined the impact of network externalities, such as, performance and effort expectancy, social influence, and trust on m-payment acceptance in jordan. except for effort expectancy, their study found all factors were significant. gao and waechter (2017) integrated tam, utaut and the valence framework by peter, sr, and x (1975) which is a customer decision-making model examining customer behaviour to argue that a lack of trust was the most significant long-term inhibitor for acceptance and success of m-payment services. then they examined the role of initial trust developed by consumers when interacting with mpayments for the first time on perceived benefit and convenience which in turn influenced intention to adopt m-payment. gao and waechter collected data from a sample in australia, and revealed that perceived system quality, information quality and service quality also positively and significantly influence initial trust, which positively impacts intention to adopt m-payment. besides utaut and tam, numerous authors have used other theoretical framework and also recognised the significant impact of trust on m-payment adoption without the consideration of types of trust in the context of m-payment adoption (huang & liu, 2012; jia, hall, & zhu, 2015; lu, yang, chau, & cao, 2011; xin et al., 2015; zhou, 2013, 2014). based on the literature of the concern for information privacy and internet users’ information privacy concerns, huang and liu (2012) investigated the effect of internet users’ concern for information privacy on intention to adopt m-payment in china, and found a significant positive impact of control, awareness and collection on trust and intention to use m-payment. lu et al. (2011) adopted the valence framework by peter et al. (1975) to develop a trust-based customer decision-making model and examined how trust interacted with both positive and negative factors. they found that initial mobile payment trust had a significant positive impact on behavioural intention, and a significant negative impact on perceived risk. zhou (2013) used the information systems success model (iss) (delone & mclean, 2003) as a theoretical basis to identify the factors australasian journal of information systems nguyen et al. 2021, vol 25, research article trust types and mediating effect 7 influencing continuance intention to adopt m-payment in china. he found a positive and significant effect of the system, information and service quality on trust which has an important impact on continuing intention to adopt m-payment. similarly, based on iss, zhou (2014) examined the determinants of m-payment adoption in china. the results confirmed that system and information quality are significant antecedents for trust in m-payment, which in turn influences adoption. jia et al. (2015) utilised a multi-stage decision making model and initial trust building theory to explore how trust is built into the learning process of customers in china, and the effect of trust on intention to use m-payment. the result indicated that exposure to m-payment and information searching has a significant and positive impact on trust which positively influences individual behavioural intention. xin et al. (2015) collected data in new zealand to examine the determinants of trust. the model confirmed the significant impact of perceived reputation of mobile service provider, perceived opportunism of mobile service provider, reputation of mobile payment vendor, structural assurance, environment risk, espoused uncertainty avoidance, and disposition to trust on trust in m-payment adoption, nonetheless it still neglected the different types of trust that constitute trust of m-payment consumers. trust is a complex concept, therefore it should be theorised as a multidimensional or multifaceted phenomenon (chen & dhillon, 2003; hillman & neustaedter, 2017; jimenez et al., 2016; mcknight et al., 2002; meng et al., 2008; yan, niemi, dong, & yu, 2008). researchers have pointed out that exploring the different types of trust in m-commerce is beneficial to a better understanding of trust, leading to more substantial understanding and prediction of consumer adoption (meng et al., 2008; min et al., 2008; nguyen et al., 2020). as a result, research on trust in m-payment adoption needs to identify trust types and their interaction effects on m-payment adoption (nguyen et al. 2020). this could lead to a more comprehensive understanding of the concept of trust in the context of m-payment adoption, such as, understanding the trust dimensions and how they affect each other and to the intention to adopt m-payment. we argue that the overall consumer trust towards m-payment services can act as a mediating factor between trust types and m-payment adoption. 2.2 research aim and questions this study aims to identify different types of trust that constitute trust of m-payment consumers and whether consumer trust mediates the effect of trust types and m-payment adoption. this leads to the following research questions: • what are the relevant types of trust in adopting m-payment? • to what extent does consumer trust mediate the relationship between identified trust types and m-payment adoption? 3 the proposed model and hypotheses regarding identifying the trust types that constitute consumer trust in m-payment context, the authors conducted a comprehensive review of related literature regarding trust in the adoption of e-commerce, m-commerce, mobile banking, and m-payment. this is because: (1) m-payment is considered a subset of m-commerce, as well as, e-commerce, and mobile banking applications also can be used for the purpose of m-payment; (2) there is no agreement regarding the types of trust in m-payment adoption. obviously, a comprehensive review of australasian journal of information systems nguyen et al. 2021, vol 25, research article trust types and mediating effect 8 related literature was necessary. based on this review, the authors adopted four sub-types of trust in the context of m-payment adoption as following. m-payment provider trust refers to the belief of consumers that the m-payment service provider will perform and complete the transactional commission, as well as, any obligations which might arise from risky or uncertain circumstances (joubert & belle, 2009, 2013), i.e. the extent to which consumers trust the m-payment provider. this is important in the context of mpayment because an m-payment service provider approaches, uses and stores private and important consumer information, such as, personal and financial information, bank account data, and data about purchased goods and services. obviously, in order to use m-payment, consumers must be confident that m-payment providers are able to provide and process mpayment services correctly, fast, conveniently, and securely (mingxing et al., 2014).(mingxing et al. 2014). service provider trust plays an important part in the overall trust of consumers, therefore it is widely used in research on e-commerce adoption (mcknight & chervany, 2001a, 2001b; mcknight et al., 2002; mcknight, kacmar, & choudhury, 2004), m-commerce adoption (joubert & belle, 2009, 2013; meng et al., 2008; min et al., 2008; siau & shen, 2003; siau, sheng, nah, & davis, 2004), and m-payment adoption (mingxing et al., 2014; srivastava, chandra, & theng, 2010; xin et al., 2015). as a result, the authors suggest m-payment provider trust is the first type of trust in building consumer trust in m-payment services. this leads to the following hypothesis: h1: m-payment provider trust positively influences consumer trust in m-payment. technology trust or system trust refers to the degree to which consumers believe that the underlying technology or system has the ability to work as expected and to process mpayment transactions appropriately (joubert & belle, 2013).(joubert and belle 2013). this study looks at system/technology trust from the viewpoint of consumers, instead of technical and technological specialists. many authors have identified the significant role of technology trust on m-commerce (joubert & belle, 2009, 2013; meng et al., 2008; min et al., 2008; siau & shen, 2003; siau et al., 2003), and m-payment adoption (srivastava et al., 2010). the underlying technology in m-payment includes not only the application technology of the m-payment provider, but also other involved technologies such as internet bandwidth, network connection coverage, mobile devices technology, encryption, mobile operating systems, connection technology such as nfc, and banking systems that accept financial transactions (cabral, 2018; krishnan, 2015). consequently, technology trust differs from m-payment provider trust, and should be recognised as a separate type of trust in m-payment adoption. as a result, the authors argue that if consumers think that m-payment related technology is trustworthy, they are more likely to trust m-payment. this leads to the following hypothesis. h2: technology trust positively influences consumer trust in m-payment. institution-based trust refers to the belief of consumers that necessary structural conditions for increasing the likelihood of achieving a successful outcome in an endeavour like m-payment, are present (joubert & belle, 2013; mcknight et al., 2002). institution-based trust includes two dimensions: structural assurance and situational normality. structural assurance manifests the belief that structures, such as, guarantees, promises, regulations, and legal resources are accessible to ensure the success and security of transactions (mcknight et al., 2002). situational normality refers to the belief that the online environment is appropriate, well ordered, and favourable for conducting transactions (mcknight & chervany, 2001a). cheung and lee (2001) pointed out that institutional infrastructure and regulations are the critical determinants for australasian journal of information systems nguyen et al. 2021, vol 25, research article trust types and mediating effect 9 building customer trust. mahadevan and venkatesh (2000) explained the power of institutionbased trust that the legal system plays an essential role in regulating the vendors to provide fair information and protect user’s privacy and other concerns, thereby recovering customers’ confidence and belief. pavlou and gefen (2004, p.37) stated that “institution-based trust is a buyer’s perception that effective third-party institutional mechanisms are in place to facilitate transaction success”. due to the importance of institution-based trust, it has been widely adopted in research on e-commerce (gefen, karahanna, & straub, 2003; mcknight et al., 2002; pavlou & gefen, 2004), m-commerce (cho, kwon, & lee, 2007; guangming & yuzhong, 2011; hillman & neustaedter, 2017; joubert & belle, 2009, 2013; min et al., 2008; piao et al., 2012), mobile banking (nguyen, 2016), and m-payment (hillman & neustaedter, 2017; srivastava et al., 2010; yan & yang, 2014). as a result, the authors argue that if consumers perceive that they are protected by third-party institutional mechanisms when using m-payment, they are more likely to trust m-payment. this results in the following hypothesis. h3: institution-based trust positively influences consumer trust in m-payment. seller trust is the degree to which the consumer trusts a community of sellers, and this is necessary for any e-commerce, as well as, social commercial activities (lu et al., 2016). seller trust is a vital factor in m-commerce because in the online environment, sellers and buyers may make contact anonymously, and normally conduct transactions without a formal contractual agreement (andreev et al., 2012). the significant role of seller trust is identified in many studies on e-commerce, as well as, m-payment adoption (andreev et al., 2012; lu et al., 2016; pavlou & gefen, 2004). pavlou and gefen (2004) collected data from amazon’s auction websites, and demonstrated the significant impact of trust in sellers on the transaction intention of customers in online markets. lu et al. (2016) found the important effect of trust in sellers on social commerce purchase intention of customers in china. the study of andreev et al. (2012) collected data in israel, and resulted in showing the significant influence of trust in sellers on willingness to use m-payments. obviously, a reputable seller must not only provide qualified goods, but also use a fast, accurate and secured payment method. as a result, the authors argue that if consumers trust reputable sellers who accept and use m-payment for their goods or services, consumers are more likely to trust m-payment. this leads to the following hypothesis. h4: seller trust positively influences consumer trust in m-payment. prior studies empirically found that consumer trust is one of the important drivers for mpayment adoption (patil et al., 2018; qasim & abu-shanab, 2016; xin et al., 2015), therefore the authors put forward that if consumers trust m-payment, they are more likely to intend to adopt m-payment services in conducting e-commerce transactions. this leads to the last hypothesis. h5: consumer trust in m-payment positively influences intention to adopt m-payment. figure 5 presents the proposed model and hypotheses of this research: australasian journal of information systems nguyen et al. 2021, vol 25, research article trust types and mediating effect 10 m-payment provider trust institution-based trust seller trust consumer trust in mpayment intention to adopt m-payment h1 h2 h3 h5 ` technology trust h4 figure 5. the proposed model 4 research methodology this section discusses the design of instrument and data collection. 4.1 instrument development the construct instruments for the questionnaire used for data collection were borrowed from previous studies which had been tested and validated (see table 1). all questions were translated to vietnamese and then revised based on the pilot test. participants indicated their agreement or disagreement with the questions in the online survey (see appendix 8.1 for full survey questions). construct and items references m-payment provider trust (pt): six items andreev et al. 2012; srivastava et al. 2010; zhou 2011 technology trust (tt): four items srivastava et al. 2010 institution-based trust (it): six items mcknight et al. 2002; nguyen 2016; srivastava et al. 2010 seller trust (st): four items andreev et al. 2012; lu et al. 2016; pavlou and gefen 2004 consumer trust (ct): four items lu et al. 2011; qasim & abu-shanab 2016 intention to adopt m-payment (in): three items venkatesh et al. 2012 table 1. instrument items used in the survey 4.2 data collection in order to test the hypotheses, data was collected in vietnam using survey, targeting the population that: (1) are over 18 years old, and (2) have used m-payment services in the last three months. an online survey with the questionnaire in the vietnamese language was hosted in qualtrics survey tool. the pilot test was conducted with a group of 31 m-payment users to refine the items, and the pilot test data was not included in the final data. the authors used a facebook account to promote the survey to participants. given the fact that using m-payment is voluntary and popular amongst vietnamese consumers, the connections in the social australasian journal of information systems nguyen et al. 2021, vol 25, research article trust types and mediating effect 11 network were effective in approaching participants without raising concerns about spam, virus or spyware when accessing the online survey. the measurement model includes 28 questions therefore with a rate of case-to-variables of 5:1 for factor analysis (hair, black, babin, anderson, & tatham, 1998), the needed sample is at least 140 observations. a total of 225 answers was collected over two months. after checking outlier and missing data, 204 valid observations were used for further analysis, which satisfies the sample adequacy criterion. of the survey participants, 72% were female and 28% male. nearly 97% were with a degree or above, 71% aged under 35 years old, with 94% had used smart phones for more than three years. all participants had experience of using some forms of m-payment before. see appendix 8.2 for full demographic information. the data was tested for common method bias by using harman’s test (chau, deng, & tay, 2020; podsakoff, mackenzie, lee, & podsakoff, 2003), which revealed that the largest factor explains only 13.56% of the variance in the measure. this is less than 50% the threshold of common method bias, so there is no significant common method bias in the data. as there were significantly more female respondents, the data was examined for potential bias between male and female via conducting a two-sample t-test with six constructs adopted in the conceptual model. the independent t-test results revealed that there is no significant difference between male and female at a 95% confidence level for the variables. this is consistent to the previous study of xin et al. (2015) which found that gender has insignificant impact on trust in m-payment adoption. overall, it is unlikely that the results are biased by gender. 5 data analysis and results structural equation modelling (sem) is a general term that refers to a multivariate statistical analysis technique used to evaluate and analyse the structural relationship between the latent variables and their measured variables with empirical data. this study used spss and smartpls to analyse the collected data including 204 responses. analysis was done in two steps: the reliability and validity assessment of the measurement model were tested, and the structural model assessment performed which are presented below. 5.1 measurement model there are four criteria to test the measurement model which are construct, indicator, convergence and discriminant validity. construct validity is tested via composite reliability and cronbach’s alpha with the cut-off value is 0.7 (straub, 1989). an item that has factor loading lower than 0.4 must be eliminated for indicator validity (churchill, 1979). the requirement for convergence validity is the average variance extracted (ave) is greater than 0.5 (henseler, ringle, & sinkovics, 2009). discriminant validity is tested by the htmt analysis with the cut-off value is 0.85 (ab hamid, sami, & sidek, 2017; kline, 2015). based on the above criteria, the authors dropped items it1, pt4 due to a low factor loading, and the technology trust (tt) construct due to high loading on the m-payment provider trust (pt) construct. the implication of this elimination is discussed in the section 6.2. all the remaining items and constructs met the requirements of validity for measurement model. the outcome is presented in tables 2 and 3. australasian journal of information systems nguyen et al. 2021, vol 25, research article trust types and mediating effect 12 construct cronbach’s alpha composite reliability ave items loading intention to adopt mpayment (in) 0.777 0.785 0.649 in1 0.7 in2 0.717 in3 0.885 customer trust (ct) 0.894 0.894 0.678 ct1 0.806 ct2 0.843 ct3 0.8 ct4 0.844 institution-based trust (it) 0.902 0.898 0.642 it2 0.975 it3 0.78 it4 0.718 it5 0.727 it6 0.777 m-payment provider trust (pt) 0.901 0.902 0.648 pt1 0.753 pt2 0.842 pt3 0.878 pt5 0.829 pt6 0.713 seller trust (st) 0.91 0.909 0.715 st1 0.89 st2 0.824 st3 0.863 st4 0.802 table 2. validity criteria and factor loadings table 3 presents the outcome of htmt analysis which indicates that the requirement of discriminant validity is satisfied with the cut-off value is 0.85 (ab hamid et al., 2017; kline, 2015). table 3. discriminant validity with htmt analysis the requirements of the model fit measurement are satisfied based on the criteria of hu and bentler (1999) and are shown in the table 4 below. table 4. the model fit measures ct in it pt st ct in 0.581 it 0.785 0.404 pt 0.775 0.416 0.816 st 0.794 0.396 0.747 0.712 measure estimate threshold interpretation cmin/df 2.830 between 1 and 3 excellent cfi 0.901 >0.95 acceptable srmr 0.071 <0.08 excellent australasian journal of information systems nguyen et al. 2021, vol 25, research article trust types and mediating effect 13 5.2 structural model and hypothesis testing the structural model and hypotheses were tested based on the examination of standardized paths. the path relationship between three types of trust including m-payment provider, institution-based, seller, and customer trust are positive and significant with p<0.1, p<0.05, p<0.01 (baptista & oliveira, 2015), therefore hypotheses 1, 3 and 4 are supported. the effect of trust on intention to adopt m-payment is significant with p=0.000 hence hypothesis 5 is supported. hypothesis 2 is dropped as the tt construct is dropped due to high cross loading discussed before. the model explains a 74.1% and 33.5% of variation in consumer trust in mpayment and intention to adopt m-payment, respectively. the overall outcome of structural model and hypothesis testing is indicated in the tables 5 and figure 6 below: hypothesis original sample (o) standard deviation (stdev) t statistics (|o/stdev|) p values results h1: pt -> ct 0.25 0.131 1.914 0.056 supported h3: it -> ct 0.29 0.123 2.361 0.019 supported h4: st -> ct 0.399 0.098 4.071 0.000 supported h5: ct -> in 0.579 0.066 8.835 0.000 supported table 5. the path coefficients of hypotheses m-payment provider trust institution-based trust seller trust consumer trust in mpayment intention to adopt m-payment 0.250* 0.290** 0.399*** 0.579*** r2= 74.1 r2= 33.5 note: (*p<0.1; **p<0.05; ***p<0.01) figure 6. structural model results regarding the mediating impact of consumer trust, the direct impact of the three types of trust on intention to adopt m-payment was conducted and found no significant relationship as shown in table 6. as a result, consumer trust has a full mediating impact on the relationship between the three types of trust and m-payment adoption. original sample (o) standard deviation (stdev) t statistics (|o/stdev|) p values results it -> in 0.037 0.404 0.093 0.926 not supported pt -> in -0.066 0.26 0.253 0.8 not supported st -> in -0.18 0.168 1.068 0.286 not supported table 6. the path coefficients of direct relationship between trust types and intention to adopt mpayment australasian journal of information systems nguyen et al. 2021, vol 25, research article trust types and mediating effect 14 6 discussion this section discusses the findings, research and practical contributions, and limitation of this study. 6.1 trust types and mediating effect of consumer trust the study found a significant positive effect for m-payment provider trust, institution-based trust and seller trust on the overall consumer trust in m-payment. the variance explained of consumer trust in m-payment is 74.1% which can be classified as substantial (chin, 1998). the positive effect of m-payment provider trust on consumer trust is consistent with earlier research which recognised the important role of the service provider in e-commerce, mcommerce, and m-payment (joubert & belle, 2009; mcknight et al., 2002; mingxing et al., 2014). the institution-based trust impact finding is also supported in earlier research (mcknight et al., 2002; mcknight et al., 2004; nguyen, 2016). obviously, consumers consider that the protection of third-party mechanisms are vital to use m-payment services. the relationship between seller trust and consumer trust in m-payment is also confirmed, which is in line with earlier research (lu et al., 2016; pavlou & gefen, 2004; siau et al., 2004). accordingly, consumers believe in choosing the m-payment method when it is used by reputable sellers. finally, the significant impact of consumer trust on intention to adopt m-payment is also validated, which is consistent with previous studies (patil et al., 2018; qasim & abu-shanab, 2016; xin et al., 2015). this confirms the role of consumer trust as a key driver for m-payment adoption. in the data analysis, technology trust (tt) construct was dropped due to high cross loading and inter-correlation on provider trust (pt), hence hypothesis 2 was not assessed. this means m-payment tt overlaps with m-payment pt. this may be because the underlying technology of m-payments such as encryption, network security, and mobile technology in the financial system are too complex for customers to comprehend and establish trust. as a result, consumers consider that the m-payment provider should be fully responsible for the technology assurance of m-payment services. this finding is a new characteristic of mpayment adoption which is different from the adoption of traditional e-commerce and mcommerce found in the literature. m-payment consumers should not be required to clearly understand the technology and its security aspects, but rather they rely on m-payment providers as part of their service commitment to ensure the transactions will be secure and safe. accordingly, this also highlights the important role of the m-payment provider since from the consumer perspective, the quality of m-payment technology is the responsibility of the m-payment provider. another important finding is the significant full mediating effect of consumer trust on the relationship between three types of trust and intention to adopt m-payment. this result indicates that consumer trust can be affected by many factors, and it plays a mediating role that leads to m-payment adoption decision. it is important to notice that having established consumer confidence in various aspects of the m-payment service is not sufficient for adopting the service. provider trust, institution trust or seller trust alone does not guarantee consumers to consider m-payment, the overall consumer trust as of result of developing trust types plays a critical role in influencing the corresponding behaviour. australasian journal of information systems nguyen et al. 2021, vol 25, research article trust types and mediating effect 15 6.2 contributions m-payment is a state-of-the-art payment method which plays an important role in the evolution of payment for society from cash to digital methods of payment, especially post covid-19 where touchless or low-touch economy becomes a must. consumer trust is a significant driver of m-payment adoption and understanding trust can help to predict as well as increase the adoption of m-payment. as a result, the results of this study have important implications for both research and practice. for researchers, this study provides the classification and recognition of three types of trust including m-payment provider, institution-based and seller trust which have significant impacts on trust of m-payment consumers. it also separates overall consumer trust from other context-specific trusts. this could lead to a call for further research such as exploring further determinants of consumer trust or trust types, investigating the relationship between each type of trust and other critical factors in m-payment adoption, examining, how consumer trust’s mediating role changes with different trust determinants, or with various levels of consumer awareness. future research can measure or employ trust of consumers as a reflective construct with these three trust types in the context of m-payment adoption. in addition, these three types of trust can be extended to study further in adopting of other technologies. for practitioners, the study’s findings explain working mechanism of trust on using the service. it identifies clearly significant types of trust to refine, promote, and implement mpayment services that can be more likely accepted by consumers. m-payment providers can cooperate with reputable sellers and convince them to accept using m-payment for their goods or services. when m-payment is adopted in famous e-markets with reputable sellers, consumers may be more likely to buy or sell on the market. from a customer perspective, the m-payment provider is fully responsible for not only the quality but also the technology assurance of m-payment service, thus m-payment providers must focus on improving their applications to operate well in different conditions regarding mobile operating systems, networking, speed connection, and security. policy makers need to enact regulations or mechanisms to protect the legal rights of m-payment customers, and clearly specify the responsibility of stakeholders such as m-payment providers, banks, and financial institutions. this contributes to the customers’ perception of overall safety when using m-payment, leading to high use m-payment or m-commerce. when the three types of trust are consolidated for consumers, they are also more likely to trust and adopt m-payment. the understanding of the role of consumer trust also highlights and expands current findings in both theory and practice. this study not only confirms the significant impact of trust on intention to adopt m-payment, but also recognise the full mediating role of trust of consumers. trust accounts for all of the effects of the three types of trust including m-payment provider, institution-based and seller trust on m-payment adoption. accordingly, when trust is enhanced, the impact of the three types of trust on m-payment adoption is also increased. 6.3 limitations there is a limitation to this research that needs to be acknowledged. the data sample of 204 respondents is limited to the social connection of the authors, which may have led to a potential bias in the data. future research should collect more data with a wider population scope. australasian journal of information systems nguyen et al. 2021, vol 25, research article trust types and mediating effect 16 7 conclusion the benefits of m-payment are obvious for both business and consumers such as the convenience, security, better tracking of transactions, a safer way of payment especially in the contemporary situation of covid-19 spread worldwide. as a result, m-payment is considered as a useful modern tool that is being adopted worldwide. this research aims to address the lack of understanding of the different types of trust that constitute trust of consumers in mpayment adoption. the proposed model was tested with data collected in vietnam which has one of the world’s fastest m-payment adoption rates. this paper validated the model identifying three relevant types of trust which are m-payment provider trust, institution-based trust and seller trust as constituting consumer trust in m-payment. the data analysis also revealed that technology trust could be embedded in m-payment provider trust perceived by consumers. this study found that consumer trust is a full mediator for the relationship between the three trust types and intention to adopt m-payment of consumers. the outcome can be used for better understanding of trust in m-payment, which contributes not only to researchers regarding the literature of trust in m-payment adoption as well as technology adoption, but also to practitioners such as managers, m-payment service providers, and consultants to enhance consumer trust when promoting m-payment and other technologies. references ab hamid, m., sami, w., & sidek, m. m. (2017). discriminant validity assessment: use of fornell & larcker criterion versus htmt criterion. journal of physics: conference series, 890(1), 012163. alalwan, a. a., dwivedi, y. k., & rana, n. p. (2017). factors influencing adoption of mobile banking by jordanian bank customers: extending utaut2 with trust. international journal of information management, 37(3), 99-110. alhulail, h. (2018). an investigation of customers’ loyalty to social commerce websites. (doctor of philosophy doctorate), rmit university, accessed on 28/11/2018 from https://ap01.alma.exlibrisgroup.com/discovery/delivery/61rmit_inst:rmitu/12248 274490001341 andreev, p., pliskin, n., & rafaeli, s. (2012). drivers and inhibitors of mobile-payment adoption by smartphone users. international journal of e-business research (ijebr), 8(3), 50-67. asatryan, d. (2017). "security and lack of trust stall mobile payment adoption." accessed on 01/06/2019 from https://bankinnovation.net/2017/06/security-and-lack-of-truststall-mobile-payment-adoption/ baptista, g., & oliveira, t. (2015). understanding mobile banking: the unified theory of acceptance and use of technology combined with cultural moderators. computers in human behavior, 50, 418-430. bilgihan, a. (2016). gen y customer loyalty in online shopping: an integrated model of trust, user experience and branding. computers in human behavior, 61, 103-113. doi:https://doi.org/10.1016/j.chb.2016.03.014 cabral, j. (2018). "mobile payment systems: will your smart phone replace your wallet?". accessed on 6/10/2019 from https://www.finder.com.au/mobile-payment-systems chau, n. t., deng, h., & tay, r. j. j. o. m. m. (2020). critical determinants for mobile commerce adoption in vietnamese small and medium-sized enterprises. journal of marketing management, 36(5-6), 456-487. australasian journal of information systems nguyen et al. 2021, vol 25, research article trust types and mediating effect 17 chen, s. c., & dhillon, g. s. (2003). interpreting dimensions of consumer trust in ecommerce. information technology and management, 4(2-3), 303-318. cheung, c. m., & lee, m. k. (2001). trust in internet shopping: instrument development and validation through classical and modern approaches. journal of global information management (jgim), 9(3), 23-35. chin, w. w. (1998). commentary: issues and opinion on structural equation modeling. in. management information systems research center, university of minnesota: jstor. cho, d.-y., kwon, h. j., & lee, h.-y. (2007). analysis of trust in internet and mobile commerce adoption. 40th annual hawaii international conference on system sciences (hicss'07), 3-6 january, waikoloa, big island, hi, 50-50. churchill, g. a. (1979). a paradigm for developing better measures of marketing constructs. journal of marketing research, 16(1), 64-73. coyle, d. (2013). "the cost of mistrust." accessed on 14/12/2020 from http://www.oecd.org/forum/the-cost-of-mistrust.htm delone, w. h., & mclean, e. r. (2003). the delone and mclean model of information systems success: a ten-year update. journal of management information systems, 19(4), 9-30. emily, s. (2018). "different types of mobile payments explained." accessed on 28/11/2020 from https://www.mobiletransaction.org/different-types-of-mobile-payments/ fintechnews. (2018a). "vietnam fintech sector report: all about digital and mobile payments ?". accessed on 28/11/2018 from http://fintechnews.sg/20056/vietnam/vietnam-fintech-sector-report-mobile-paymentdigitial/ fintechnews. (2018b). "vietnam: rise of e-commerce paving the way for digital, mobile payments boom." accessed on 16/12/2018 from http://fintechnews.sg/23211/mobilepayments/vietnam-rise-of-e-commerce-pavingthe-way-for-digital-mobile-payments-boom/ fishbein, m., & ajzen, i. (1975). belief, attitude, intention and behavior: an introduction to theory and research: addison-wesley, reading, ma. gao, l., & waechter, k. a. (2017). examining the role of initial trust in user adoption of mobile payment services: an empirical investigation. information systems frontiers, 19(3), 525-548. gefen, d., karahanna, e., & straub, d. w. (2003). trust and tam in online shopping: an integrated model. mis quarterly, 27(1), 51-90. guangming, y., & yuzhong, m. (2011). a research on the model of factors influencing consumer trust in mobile business. 2011 international conference on e-business and egovernment (icee), 6-8 may 2011, shanghai, china, 1-5. hair, j. f., black, w. c., babin, b. j., anderson, r. e., & tatham, r. l. (1998). multivariate data analysis (vol. 5): prentice hall upper saddle river, nj. hallikainen, h., & laukkanen, t. j. i. j. o. i. m. (2018). national culture and consumer trust in e-commerce. international journal of information management, 38(1), 97-106. henseler, j., ringle, c. m., & sinkovics, r. r. (2009). the use of partial least squares path modeling in international marketing. in new challenges to international marketing (277319): emerald group publishing limited. hillman, s., & neustaedter, c. (2017). trust and mobile commerce in north america. computers in human behavior, 70, 10-21. australasian journal of information systems nguyen et al. 2021, vol 25, research article trust types and mediating effect 18 hu, l. t., & bentler, p. m. (1999). cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. structural equation modeling: a multidisciplinary journal, 6(1), 1-55. huang, y., & liu, w. (2012). the impact of privacy concern on users' usage intention of mobile payment. 2012 international conference on information management, innovation management and industrial engineering, 20-21 october 2012, sanya, china, 90-93. jia, l., hall, d., & zhu, s. (2015). trust building in consumer learning process and its effect on consumers’ behavioral intention toward mobile payments. 21st americas conference on information systems, 13-15 august 2015, fajardo, puerto rico. jimenez, n., san-martin, s., & azuela, j. i. (2016). trust and satisfaction: the keys to client loyalty in mobile commerce. academia revista latinoamericana de administración, 29(4), 486-510. joubert, j., & belle, j.-p. v. (2009). the importance of trust and risk in m-commerce: a south african perspective. pacis 2009 proceedings, 96. joubert, j., & belle, j.-p. v. (2013). the role of trust and risk in mobile commerce adoption within south africa. international journal of business, humanities and technology, 3(2), 2738. kline, r. b. (2015). principles and practice of structural equation modeling. new york: guilford publications. kolaki, m. (2017). mobile payment use and mobile payment transactions by older adults: a qualitative study. (master), linnaus university, accessed on 28/11/2020 from https://www.diva-portal.org/smash/get/diva2:1127590/fulltext01.pdf kreyer, n., pousttchi, k., & turowski, k. (2002). standardized payment procedures as key enabling factor for mobile commerce. proceedings of the third international conference on e-commerce and web technologies: springer, 400-409. krishnan, a. (2015). "technology behind mobile payments." accessed on 28/11/2020 from https://www.linkedin.com/pulse/technology-behind-mobile-payments-anandkrishnan lin, j., wang, b., wang, n., & lu, y. (2014). understanding the evolution of consumer trust in mobile commerce: a longitudinal study. information technology and management, 15(1), 37-49. liu, b. (2012). understanding consumers' intention to use mobile payment services: the perspective of university students in northern jiangsu area. 2012 second international conference on business computing and global informatization, 12-14 october 2012, shanghai, china, 257-260. lu, b., fan, w., & zhou, m. (2016). social presence, trust, and social commerce purchase intention: an empirical research. computers in human behavior, 56, 225-237. doi:https://doi.org/10.1016/j.chb.2015.11.057 lu, y., yang, s., chau, p. y., & cao, y. (2011). dynamics between the trust transfer process and intention to use mobile payment services: a cross-environment perspective. information & management, 48(8), 393-403. mahadevan, b., & venkatesh, n. (2000). a framework for building online trust for business to business e-commerce: issues & challenges. it asia millennium conference, bombay, india. malaquias, r. f., & hwang, y. (2016). an empirical study on trust in mobile banking: a developing country perspective. computers in human behavior, 54, 453-461. australasian journal of information systems nguyen et al. 2021, vol 25, research article trust types and mediating effect 19 mcknight, d. h., & chervany, n. l. (2001a). trust and distrust definitions: one bite at a time. in trust in cyber-societies (27-54): springer. mcknight, d. h., & chervany, n. l. (2001b). what trust means in e-commerce customer relationships: an interdisciplinary conceptual typology. international journal of electronic commerce, 6(2), 35-59. mcknight, d. h., choudhury, v., & kacmar, c. (2002). developing and validating trust measures for e-commerce: an integrative typology. information systems research, 13(3), 334-359. mcknight, d. h., kacmar, c. j., & choudhury, v. (2004). dispositional trust and distrust distinctions in predicting high-and low-risk internet expert advice site perceptions. e-service journal, 3(2), 35-58. meng, d., min, q., & li, y. (2008). study on trust in mobile commerce adoption-a conceptual model. 2008 international symposium on electronic commerce and security, 3-5 august 2008, guangzhou city, china, 246-249. min, q., meng, d., & zhong, q. (2008). an empirical study on trust in mobile commerce adoption. proceedings of 2008 ieee international conference on service operations and logistics, and informatics ieee/soli'2008, 659-664. mingxing, s., jing, f., & yafang, l. (2014). an empirical study on consumer acceptance of mobile payment based on the perceived risk and trust. proceedings of the 2014 international conference on cyber-enabled distributed computing and knowledge discovery, 312317. mobiforge. (2014). "what is m-commerce? how big is it really? and why any company who sells anything needs it, now." accessed on 1/02/2020 from https://internet.com/ecommerce/what-is-m-commerce-how-big-is-it-really-and-whyany-company-who-sells-anything-needs-it-now/ nguyen, l. d. (2016). an investigation of critical factors affecting customers' trust in electronic banking: empirical evidence from vietnam. (phd doctorate), griffith university, griffith university. accessed on 28/11/2020 from https://www120.secure.griffith.edu.au/rch/items/6df25018-2f63-4a22-a146de0102b769bf/1/ nguyen, t. a., dick, m., & pham, h. c. (2020). investigating the critical determinants of trust and their impact on mobile payment adoption. proceedings of the 13th iadis international conference information systems 2020, sofia university, bulgaria, 29-36. patil, p., rana, n., dwivedi, y., & abu-hamour, h. (2018). the role of trust and risk in mobile payments adoption: a meta-analytic review. pacis 2018 proceedings, 129. pavlou, p. a., & gefen, d. (2004). building effective online marketplaces with institutionbased trust. information systems research, 15(1), 37-59. peter, j. p., sr, t., & x, l. (1975). a comparative analysis of three consumer decision strategies. journal of consumer research, 2(1), 29-37. phonthanukitithaworn, c., sellitto, c., & fong, m. (2015). user intentions to adopt mobile payment services: a study of early adopters in thailand. journal of internet banking and commerce, 20(1). piao, c., wang, s., & yang, f. (2012). research on trust evaluation model for mobile commerce based on structural equation modeling. 2012 ieee ninth international conference on ebusiness engineering, 9-11 september 2012, hangzhou, china, 25-32. australasian journal of information systems nguyen et al. 2021, vol 25, research article trust types and mediating effect 20 podsakoff, p. m., mackenzie, s. b., lee, j.-y., & podsakoff, n. p. (2003). common method biases in behavioral research: a critical review of the literature and recommended remedies. journal of applied psychology, 88(5), 879. pwc. 2019. "mobile payments in vietnam fastest growing globally, thailand emerges second in southeast asia." accessed on 8/12/2020 from https://www.pwc.com/th/en/press-room/press-release/2019/press-release-30-04-19en.html qasim, h., & abu-shanab, e. (2016). drivers of mobile payment acceptance: the impact of network externalities. information systems frontiers, 18(5), 1021-1034. researchandmarkets. 2020. "covid-19 impact on online & mobile payments market 2020 contactless viewed as cleaner way to pay." accessed on 30/7/2020 from https://www.prnewswire.com/news-releases/covid-19-impact-on-online--mobilepayments-market-2020---contactless-viewed-as-cleaner-way-to-pay-301076904.html rouibah, k., lowry, p. b., & hwang, y. (2016). the effects of perceived enjoyment and perceived risks on trust formation and intentions to use online payment systems: new perspectives from an arab country. electronic commerce research and applications, 19, 33-43. doi:https://doi.org/10.1016/j.elerap.2016.07.001 shuhaiber, a. (2016). factors influencing consumer trust in mobile payments in the united arab emirates. (doctor of philosophy doctorate), victoria university of wellington, accessed on 28/11/2018 from http://researcharchive.vuw.ac.nz/xmlui/bitstream/handle/10063/5167/thesis.pdf?sequ ence=1 siau, k., & shen, z. (2003). building customer trust in mobile commerce. communications of the acm, 46(4), 91-94. siau, k., sheng, h., & nah, f. (2003). development of a framework for trust in mobile commerce. sighci 2003 proceedings, 6. siau, k., sheng, h., nah, f., & davis, s. (2004). a qualitative investigation on consumer trust in mobile commerce. international journal of electronic business, 2(3), 283-300. slade, e., williams, m., dwivedi, y., & piercy, n. (2015). exploring consumer adoption of proximity mobile payments. journal of strategic marketing, 23(3), 209-223. slade, e. l., dwivedi, y. k., piercy, n. c., & williams, m. d. (2015). modeling consumers’ adoption intentions of remote mobile payments in the united kingdom: extending utaut with innovativeness, risk, and trust. psychology & marketing, 32(8), 860-873. srivastava, s. c., chandra, s., & theng, y.-l. (2010). evaluating the role of trust in consumer adoption of mobile payment systems: an empirical analysis. communications of the association for information systems, 27, 561-588. statista. 2018. "number of smartphone users in vietnam from 2015 to 2022 (in millions)*." accessed on 16/12/2018 from https://www.statista.com/statistics/467739/forecast-ofsmartphone-users-in-vietnam/ straub, d. w. (1989). validating instruments in mis research. mis quarterly, 13(2), 147-169. stringfellow, a. (2018). "what is a mobile payment?". accessed on 28/11/2020 from https://www.tccrocks.com/blog/what-is-a-mobile-payment/ venkatesh, v., thong, j. y., & xu, x. (2012). consumer acceptance and use of information technology: extending the unified theory of acceptance and use of technology. mis quarterly, 36(1), 157-178. xin, h., techatassanasoontorn, a. a., & tan, f. b. (2015). antecedents of consumer trust in mobile payment adoption. journal of computer information systems, 55(4), 1-10. australasian journal of information systems nguyen et al. 2021, vol 25, research article trust types and mediating effect 21 yan, h., & yang, z. (2014). an empirical examination of user adoption mobile paymentt. 2014 international conference on management of e-commerce and e-government, 31 october2 november 2014, shanghai, china, 156-162. yan, z., niemi, v., dong, y., & yu, g. (2008). a user behavior based trust model for mobile applications. international conference on autonomic and trusted computing: springer, 23-25 june 2008, oslo, norway, 455-469. zhou, t. (2011). the effect of initial trust on user adoption of mobile payment. information development, 27(4), 290-300. zhou, t. (2013). an empirical examination of continuance intention of mobile payment services. decision support systems, 54(2), 1085-1091. zhou, t. (2014). understanding the determinants of mobile payment continuance usage. industrial management & data systems, 114(6), 936-948. appendices appendix a. survey items construct and items references m-payment provider trust (pt): six items pt1: based on my perception and experience of the mobile payment provider, i know they have sufficient expertise and resources to conduct mobile payment services. pt2: based on my perception and experience of the mobile payment provider, i know them to be honest. pt3: based on my perception and experience of the mobile payment provider, i know them to be reliable. pt4: based on my perception and experience of the mobile payment provider, i know they provide secure services pt5: based on my perception and experience of the mobile payment provider, i know them to be trustworthy. pt6. based on my perception and experience of the mobile payment provider, i believe they have a good reputation. andreev et al., 2012; srivastava et al., 2010; zhou, 2011 technology trust (tt): four items tt1: based on my perception of the mobile payment technology, i know it is reliable. tt2: based on my perception of the mobile payment technology, i can know it is secure. tt3: based on my perception of the mobile payment technology, i can know it is trustworthy. tt4. in general, i trust mobile payment technology to transact m-payment. srivastava et al. 2010 institution-based trust (it): six items it1. i feel good about how things go when i use mobile payment. it2. i am comfortable making a mobile payment. it3. i believe the internet has enough security safeguards to make me feel comfortable using it to make a mobile payment. it4. i feel assured that the legal system and institutions adequately protect me from mobile payment problems (such as financial frauds, and duplicate payments). it5. i feel confident that encryption and other mobile technology safeguards make it safe for me to make mobile payments. it6. in general, the internet is now a robust and safe environment in which to make a mobile payment. mcknight et al., 2002; nguyen, 2016; srivastava et al., 2010 seller trust (st): four items st1. based on my experience, sellers who accept mobile payment are in general reliable. andreev et al., 2012; lu et al., 2016; pavlou & gefen, 2004 australasian journal of information systems nguyen et al. 2021, vol 25, research article trust types and mediating effect 22 st2. based on my experience, sellers who accept mobile payment are in general honest. st3. based on my experience, sellers who accept mobile payment are in general trustworthy. st4. based on my experience, sellers who accept mobile payment generally keep their promises. consumer trust (ct): four items ct1: mobile payment always provides accurate financial services. ct2: mobile payment always provides reliable financial services. ct3: mobile payment always provides safe financial services ct4: overall, i trust mobile payment lu et al. 2011; qasim & abu-shanab 2016 intention to adopt m-payment (in): three items in1. i intend to continue using mobile payment in the future. in2. i will always try to use mobile payment in my daily life. in3. i plan to continue to use mobile payment frequently. venkatesh et al. 2012 appendix b. demographic information measure frequency percent gender male 57 27.9 female 147 72.1 education high school 6 2.9 college degree / vocational school 5 2.5 bachelor degree 116 56.9 master degree 65 31.9 phd degree 12 5.9 occupation employee (office workers – white-collar worker) 62 30.4 worker (manual labourer – blue-collar worker) 4 2.0 tradesperson (electrician, plumber, carpenter, mechanic) 1 0.5 civil servant (public servant, government employee) 34 16.7 self-employed 10 4.9 professional (scientists, accountants, doctors, academic, lawyers, engineers, teachers) 33 16.2 student 46 22.5 others 14 6.9 age 18-25 74 35.5 26-35 66 35.5 36-over 55 64 29 income per month to 5.000.000 vnd 56 27.5 5.000.000 – 10.000.000 vnd 74 36.3 10.000.000 – 18.000.000 vnd 49 24.0 32.000.000 – 52.000.000 vnd 18 8.8 52.000.000 – 80.000.000 vnd 6 2.9 over 80.000.000 vnd 1 0.5 less than 3 months 1 0.5 from 3 to under 12 months 1 0.5 australasian journal of information systems nguyen et al. 2021, vol 25, research article trust types and mediating effect 23 experience in using smartphones from 1 to under 2 years 4 2.0 from 2 to under 3 years 5 2.5 3 years and above 193 94.6 experience in using mpayment less than 3 months 24 11.8 from 3 to under 12 months 34 16.7 from 1 to under 2 years 47 23.0 from 2 to under 3 years 33 16.2 3 years and above 66 32.4 copyright: © 2021 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v25i0.3043 1 introduction 2 literature review 2.1 previous studies about trust in e-commerce, m-commerce, and m-payment adoption 2.2 research aim and questions 3 the proposed model and hypotheses 4 research methodology 4.1 instrument development 4.2 data collection 5 data analysis and results 5.1 measurement model 5.2 structural model and hypothesis testing 6 discussion 6.1 trust types and mediating effect of consumer trust 6.2 contributions 6.3 limitations 7 conclusion appendices appendix b. demographic information paper title (paper title style) australasian journal of information systems alsamarra’i, john & panteli 2025, vol 29, research article emerging leadership practices for hybrid work received 06 nov 2023, accepted 01 apr. 2025 after 4 rounds of review with karlheinz kautz as accepting editor 1 emerging leadership practices for hybrid work: the role of algorithmic management tools zeena alsamarra’i school of information technology & systems university of canberra canberra, australia email: zeena.alsamarrai@canberra.edu.au blooma john school of information technology & systems university of canberra canberra, australia email: blooma.john@canberra.edu.au niki panteli department of management science lancaster university management school lancaster university lancaster, united kingdom email: n.panteli1@lancaster.ac.uk abstract this paper examines the impact of algorithmic management (am) tools in the increasingly popular hybrid workplace and the emergence of novel leadership practices in hybrid work settings. particularly, we explore the use of an employee experience management (exm) platform – microsoft viva – that has am features in enabling emerging leadership practices influenced by algorithms. using a qualitative approach with a case study design of a multinational organisation that adopted an exm platform, the study findings reveal that am tools contribute to emerging leadership practices that reflect inclusive, humanised, and multimodal modes of leadership – all of which are underpinned by analytics-enabled leadership. the combination of these modes of leadership and associated practices is believed to respond to the challenges in employees’ work environment and enhance the employee experience in a hybrid workplace. keywords: algorithmic management, leadership, hybrid work, employee experience platforms (exm), analytics-enabled leadership. 1 introduction algorithmic management (am) is broadly defined as the potential automation of the full range of traditional employer functions, from hiring workers and managing the day-to-day operation of the enterprise through to the termination of the employment relationship (adams-prassl et al., 2023; mohlmann et al., 2023). in addition to function automation, am also entails the collection and use of data on a platform to develop and use learning algorithms that can coordinate, control, and recommend various operations (benlian et al., 2022). one of the key defining features of am is data (jarrahi et al., 2021), generated through analytics which include capturing of big volume of data, comparing, analysing, and generating insights that mailto:zeena.alsamarrai@canberra.edu.au mailto:blooma.john@canberra.edu.au mailto:n.panteli1@lancaster.ac.uk australasian journal of information systems alsamarra’i, john & panteli 2025, vol 29, research article emerging leadership practices for hybrid work 2 support prediction and decision making (cameron et al., 2023; edwards & fenwick, 2016). this understanding of am and corresponding am tools is not only limited to full automation of employer functions whereby tools implement and enforce actions on behalf of the human. however, am tools can also include technologies that incorporate elements of am where data is captured from various sources and analysed through analytics processes with the help of algorithms in order to reveal insights and provide recommendations, so the human can then make decisions and take actions in light of this information. an example of such tools is employee experience management (exm) platforms such as microsoft viva, that amongst other features, have built in am features to enable data-driven insights and recommendations that enable leaders to make better decisions and enhance the employee experience. such tools are believed to drive new innovations in business leadership that address the complexity of issues in global organisations (apté et al., 2012). the rise of am tools has mainly been associated with alternative forms of work that is largely digital, and platform based (newlands, 2021; winikoff et al., 2021). this association is further evident by the scarcity of research linked to non-platform or non-digital work context. with the covid-19 pandemic, alternative fluid work arrangements emerged (essien, 2023). amongst these alternative arrangements is hybrid work, a form of digital workplace transformation characterised by having employees who combine both work from home (wfh) and in-person work arrangements. hybrid work is a form of digitalised workplace associated with an increased dependence on digital technologies (panteli et al., 2022) including technologies like employee experience management (exm) platforms that have am embedded within the platform amongst other features (john et al., 2022). with the growing prominence of this work model globally, am tools are omnipresent and are being integrated into existing digital work systems from word processing to enterprise management systems, covering the full range of functions and capturing a wide range of work-related data in a workplace. while the hybrid work model provides opportunities such as autonomy and flexibility in time and place of work, it also presents employees and managers with a set of challenges that stem out of this arrangement including longer working hours, social isolation, and digital fatigue. in recent years, am tools have been adopted to manage the unique challenges of hybrid work environments. existing literature on the use of am has focused on the impact on employee experience within these settings (kinowska & sienkiewicz, 2022). as leaders are now also challenged with leading a workforce within a hybrid work arrangement, and with the increasing use of am tools in organisations across a range of sectors, it is becoming particularly vital to understand the effect of these tools on leadership practices that are adaptive to the challenges of hybrid work. with a distributed workforce in a hybrid work model, leaders are being challenged by the ability to observe employees’ work patterns, engagement, and productivity – indicators of employee experience – using conventional methods adopted when employees work in-person. therefore, in order to continue to have oversight over the employees’ experience in hybrid work, leaders require insights enabled by data captured in the hybrid work environment to understand employees' work patterns and respond accordingly. this is an area where data driven analytics and the insights revealed can be of great value (schuetz & venkatesh, 2020), but also how the notion of am is emerging, one that is understood to be the intersection of employees, managers, and use of algorithms (jarrahi et al., 2021). given the use of am tools is transcending into many organizational practices, we are particularly interested in its australasian journal of information systems alsamarra’i, john & panteli 2025, vol 29, research article emerging leadership practices for hybrid work 3 transformative role in altering and potentially enhancing the employee experience in hybrid work. hence, the research questions that drive the study are twofold: 1. what emerging leadership practices support employees in overcoming key challenges of hybrid work? 2. how does the use of am tools facilitate the emergence of these novel leadership practices in hybrid work? therefore, in what follows, we first present relevant literature that aims to develop the interrelationships between hybrid work, am enabled tools, and leadership practices. this is then followed by the study methodology and key findings, including the challenges of the hybrid workplace on the employee experience and the role am enabled tools play in empowering leaders to respond to these challenges. in doing so, we present a case study of a multinational organisation that utilises an exm platform – microsoft viva – a platform that incorporates a significant level of algorithmic analysis using a combination of data collection, analytics, behavioural analysis, and advanced reporting tools, to demonstrate how this tool, enabled by am features, facilitates analytics-enabled leadership practices in hybrid work. 2 literature review as the study investigates how am impacts leadership practices in hybrid work, we draw upon the literature in these three domains. hybrid work became popular following the covid-19 pandemic, accelerating the need for digital platforms more than ever in order for employees to continue to work and for leaders and organisations to have insights into their workforce in terms of employee engagement and the challenges that impact it. particularly, tools that have am capabilities became prominent as they enable analytics and create the visibility needed to manage the employee experience in hybrid work. therefore, it is at the intersectionality of these three domains that we come to an understanding that am tools may potentially be used by organisational leaders to cope with the challenges of hybrid work. these three domains will be presented next. 2.1 hybrid work hybrid work is a form of workplace transformation that requires employees to combine both remote and in-person work. this work model became increasingly popular since the covid19 pandemic (bell et al., 2023) and ushered a great shift towards hybrid work being the new work model. while some employees returned to in-person work arrangement following the pandemic, others continue to work remotely in a binary model, though variations in the degree of these arrangements exist within this model (spataro, 2022). the hybrid workplace model has created new opportunities such as flexibility and autonomy, but also new challenges such as siloed teams and lack of connection, digital exhaustion, deterioration of general wellbeing, and blurring of the work-life divide (john et al., 2023). further, the hybrid work model has also created what is referred to as a ‘hybrid work paradox’ – a situation where employees seek the flexibility in work arrangement and demonstrate preference for remote work but also want the social interaction of in-person work. this conflicting preference is presenting challenges for managers who now need to support their employees with these work arrangements. management literature has already argued that in order to be able to lead hybrid teams, the new hybrid workplace requires leadership skills that enable leaders to operate across not just a traditional in-person workplace, but also in a virtual workplace (hooijberg & watkins, 2021). australasian journal of information systems alsamarra’i, john & panteli 2025, vol 29, research article emerging leadership practices for hybrid work 4 in terms of its elements, the hybrid work model can be conceptualized along two dimensions, that of time or the temporal dimension, and place or the spatial dimension, and the flexibility embedded within them. underpinning the emergence of hybrid work is the shift in employees’ work location from working in an office or being ‘place-constrained’ to being able to work anywhere or being ‘place-unconstrained’. similarly, hybrid work entails a shift in employees’ concurrency of work from having employees work synchronously with others or being ‘timeconstrained’ to a model where workers can work asynchronously or being ‘timeunconstrained’ (gratton, 2021). this heightened flexibility in the place and time of work (including number of hours and time of work) presents two sources of diversity in the modes of work which needs to be accommodated for. this includes spatial diversity (remote vs inperson) and temporal diversity (synchronous vs asynchronous). to facilitate the heightened flexibility in the spatial and temporal elements of the hybrid work model, organizations often need a digital platform for employees to work on and which can potentially provide organisations with insights on their employees’ working patterns. this is enabled by am tools that support employees’ work but also provide opportunities for managers to access work data regardless of the location and devices used (larsen-ledet & lindley, 2022), which is why am can play a significant role in the hybrid work model (john et al., 2022). to further understand what am entails, we next explore am and the role of am tools in the workplace. 2.2 algorithmic management and work practices algorithmic management has become the new and upcoming trend in managing workplaces. as defined earlier by benlian et al. (2022), this management concept is characterised by the collection and use of data on a platform to develop and use learning algorithms to coordinate, control and recommend various operations. am has also been described as an automated decision-making vehicle (kinowska & sienkiewicz, 2022) that improves efficiencies of organizational operations (brynjolfsson & mcafee, 2014; mayer-schonberger & cukier, 2013). the availability of big data, artifical intelligence (ai), context-ware, and personalised learning algorithms on real-time behavioural and geo-spatial data have contributed to the popularity of am (schuetz & venkatesh, 2020). with the realisation that ‘algorithms transform what workers can know and what they can ‘see’ in their work practice’ (willems & hafermalz, 2021, p. 2), several researchers have been interested in how algorithms change and impact work roles and boundaries, and how coordination and control are organized (faraj et al., 2018; pachidi et al., 2021). the use of am tools in workplaces has been linked to greater efficiency, improved coordination, and enhanced decision-making accuracy (kellogg et al., 2020). on the other hand, möhlmann et al. (2023) demonstrate how workers experience tensions relating to work execution, compensation, and belonging and the role of am towards materialising behavioural nudging. nevertheless, existing work within the management and organisational literature has predominantly taken a critical stance towards the use of am in organisations. for example, kinowska and sienkiewic (2022) reported in their study of european organisations that am has an indirect negative impact on job autonomy, total rewards practices, and employees’ wellbeing. similarly, park et al. (2021) argued that the increased use of algorithms in human resource management leads to six types of burdens that employees feel. these include ‘emotional, mental, bias, manipulation, privacy, and social’ (p. 2), all of which are believed to have an adverse effect on employees’ wellbeing, particularly where these tools are used to australasian journal of information systems alsamarra’i, john & panteli 2025, vol 29, research article emerging leadership practices for hybrid work 5 replace the human aspect of people management such as in review of applicant resumes during recruitment, job termination, mentoring, and other human resource management decisions. further, benlian et al. (2022) state that one of the main challenges in am is the process involved in learning from employees’ personalized data, but also nudging employees into desirable behavior (möhlmann, 2021). as quoted by jan recker in benlian et al. (2022) ‘algorithmic management implementing empowering leadership with a focus on participative decisionmaking, showing concern, interacting with the team, leading by example, informing, and coaching cannot be reached following the current trajectory of the frontiers autonomy, learning, and inscrutability’ (p. 832). the discussion by benlian et al. (2022) clearly states that am and technology enabled leadership requires a reorientation of design towards social rather than (un)supervised learning, reduction in autonomy rather than an increase, and demands careful design rather than being a simple requirement that can be ignored. despite such scepticism in the use of am in workplaces, research also exists that urges for a more balanced perspective on the use of am, one that takes account of the new opportunities these types of technologies may bring to workplaces (marabelli et al., 2020). these opportunities can be extended to support emerging leadership preactices suitable for leading a hybrid workforce. 2.3 leadership in hybrid work having explored the popularity of hybrid work and the emergence of am tools in digital workplaces, including hybrid work settings, we now turn to leadership. we take the position that the shift towards hybrid work contribute to challenges in sustaining and developing employee engagement, overall experience, and their identification with the organisation, and on the other hand, the use of am tools necessitates a revisit of the role of leadership in these digitalised workplaces. the hybrid work model has increased the need to bring in better ways to help employees selfmanage their work, manage their focused and productive work time, and mange their worklife balance, while also maintaining their engagement and connection with their teams, managers, and organisation at large. it is therefore becoming important to understand the role of leaders and algorithmic management enabled leadership in the context of hybrid work, as well as the emerging leadership practices to support employee engagement despite the increased autonomy and flexibility hybrid work presents. leadership in the digital era is changing as the work setting in which leaders operate is changing (bell et al., 2023) and as work technologies are evolving, particularly following the covid-19 pandemic which accelerated the need for connection and relationship building in virtual settings as well (banks et al., 2022). leadership has been identified as being critical to the success of workplace transformation in the post covid-19 context (chamakiotis et al., 2021) and in a digital workplace context in general (schneider & kokshagina, 2020). chamakiotis et al. (2021) identified that managing engagement, trust, relationships, digital wellbeing, and work-life boundaries are added responsibilities of new e-leaders. within the hybrid work model, the leadership function that is critical for the success of individuals and teams has also been redefined (bell et al., 2023). it has been suggested that leaders of a hybrid workplace now assume different roles including that of a conductor who coordinates and motivates team members, a catalyst who simulates creativity and collaboration, a coach who helps people to achieve their goals and professional development, australasian journal of information systems alsamarra’i, john & panteli 2025, vol 29, research article emerging leadership practices for hybrid work 6 and a champion who advocates for their team and their required resources, all in hybrid settings (hooijberg & watkins, 2021). conventional modes of leadership such as transformational leadership continue to have relevance to a hybrid workplace and the digital era in general. transformational leadership is defined as ‘the process of influencing major changes in the attitudes and assumptions of organization members’ (yukl, 1989, p. 269). as leaders support their teams to cope with environmental challenges and remain motivated, transformational leadership contributes to more functional team interactions, improved team viability, performance, and satisfaction (lehmann-willenbrock et al., 2015). branching off transformational leadership is authentic leadership (avolio & gardner, 2005; dinh et al., 2014; tonkin, 2013; yukl, 2010). authentic leaders are transparent, respected, and support the development of positive psychological states within themselves and their followers such as optimism, hope, confidence, and resilience (avolio & gardner, 2005; iszatt‐ white & kempster, 2019). a form of authentic leadership is humanised leadership which entails knowing oneself and promoting openness by acting according with ones ‘true self’ (gardner et al., 2021). further, multimodal leadership is a new form of leadership identified after the pandemic when hybrid work became more prominent. as highlighted by antonacopoulou and georgiadou (2021), multimodal leadership is the ability to remain connected to employees regardless of their spatial and temporal work dimensions. this form of leadership helps create a space where both in-person and remote working employees in a hybrid work model feel like they are part of one digital social fabric. responsive leadership is another mode of leadership that is relevant to leading a digital workplace and has been linked to employee engagement (panteli et al., 2019). responsive leadership manifests in activities that focus on the development and continuous improvement of the employee experience in organizations (dery et al., 2017). this includes financial and mental support, encouragement, and feedback, all of which are believed to foster employee engagement (panteli et al., 2019). hence, responsive leadership that accelerates employee engagement and improves their experience in a hybrid workplace is highly relevant. a final leadership is inclusive leadership which manifests in behaviours aimed at minimizing status differences among team members. literature on inclusive leadership emphasises the importance of creating environments where all team members feel valued, respected, and able to contribute with their unique perspectives (kuknor & bhattacharya, 2022; roberson & perry, 2022). in the hybrid work model, inclusive leadership plays an important role as leaders must navigate the complexities of managing teams that blend remote and in-person interactions, ensuring that all team members feel valued and engaged, and included, regardless of their work location or preferences. 3 methodology with the aim of the study to understand how am tools can help address the challenges to the employee experience in hybrid work and potentially facilitate novel leadership practices, the study adopts a qualitative approach with a single case study design to allow for the study of an am tool such as the exm platform to be explored in the context of its use. the case study setting is based on a multinational technology organisation that has undergone digital transformation and currently having more than 2000 employees in australia. we refer to the case organisation as kappa (pseudonym). following the pandemic, hybrid work became very popular amongst employees within kappa compared to the mostly in-person work australasian journal of information systems alsamarra’i, john & panteli 2025, vol 29, research article emerging leadership practices for hybrid work 7 arrangement model before the pandemic and the occasional remote work arrangement on a case-by-case basis. kappa implemented an exm platform to manage and enhance their employees’ experience in the flow of work. specifically, kappa implemented the exm platform microsoft viva as an integration platform of existing tools, powered by microsoft 365 and integrated into microsoft teams to provide a unified work environment. the platform, which incorporates a significant level of algorithmic analysis to provide valuable insights and recommendations for enhancing the employee experience, was adopted to enhance employee learning, team connection, communication, collaboration, productivity, wellbeing, and overall engagement of employees regardless of their work arrangement. with its ai capabilities powering am processes, viva supports employees and leaders with insights and nudges related to various aspects of the employee experience, making it a suitable candidate tool for our study. the study began by gaining an understanding about various aspects of the employees’ work experience at kappa before, during, and after the pandemic as well as leadership practices at the organisation during these phases. within each period, the study aimed to elicit details about the employees’ work arrangements (in-person, remote, hybrid), the challenges that impacted their engagement, and the leadership practices that supported their engagement, and more broadly, the employee experience in these three work models. primary data was collected through semi-structured interviews with employees and managers and those with leadership roles across a range of business units and seniority levels, within the australia/new zealand region of the company. a total of 30 participants were interviewed. participants varied in terms of their hierarchical and functional positions at kappa (including 10 participants with management or people leadership roles – denoted with a ‘p’, and 20 participants who are employees – denoted with ‘ict’). participants also varied in gender (15 females and 15 males), technical roles (e.g., engineers, networking specialists, etc.), and length of employment at kappa at the time of their interview (ranging from as recent as one month to as long as 28 years). a summary of the study participants’ is provided in table 1. interviews took place over an estimated period of 18 months, starting in december 2021 and completing in july 2023. predominantly, interviews were held via microsoft teams with several interviews being held in-person in the latter stages of the data collection phase. interviews averaged an hour in duration (exact interview durations are provided in table 1), were audio recorded where consent is given, and transcribed to facilitate the coding and analysis process. at the time of their interview, the study participants fell into one of three categories – (1) those who have been with kappa since before the pandemic and have experienced the shifts in work models from in-person work before the pandemic, to fully remote work during the pandemic then to hybrid work; (2) those who joined kappa during the pandemic, experiencing fully remote and hybrid work arrangements; and finally (3) those who joined kappa when the organisation returned to in-person work but also maintained the flexibility of remote work and hybrid work arrangements. this diversity in the participants’ work arrangement shifts provided an evolutionary perspective of how work became hybrid at kappa and provided an avenue to explore the challenges to the employee experience throughout these work models. understanding of the challenges further enabled an understanding of how leadership practices transformed with work models to reinstate engagement and contribute to an enhanced employee experience under all work conditions. australasian journal of information systems alsamarra’i, john & panteli 2025, vol 29, research article emerging leadership practices for hybrid work 8 participant interview date position title employment at kappa interview duration ict1 dec 2021 networking specialist 4.5 years 54 minutes icb2 dec 2021 communications executive 3 years 49 minutes ict3 dec 2021 architect less than a year 48 minutes ict4 dec 2021 principal consultant 10 years 54 minutes p5 dec 2021 corporate communications director 4 years 57 minutes icb6 dec 2021 product marketing manager 1 year 43 minutes p7 dec 2021 digital sales leader 21 years 58 minutes icb8 dec 2021 marketing operations team less than a year 65 minutes icb9 feb 2022 partner account manager 10 years 70 minutes p10 feb 2022 general manager 5 years 70 minutes icb11 may 2022 senior consultant less than a year 45 minutes p12 may 2022 engineer 28 years 67 minutes icb13 may 2022 senior hr business partner less than a year 55 minutes icb14 aug 2022 global specialist 5 years 63 minutes p15 aug 2022 business group manager 19 years 53 minutes icb16 aug 2022 technology strategist 14 years 41 minutes p17 aug 2022 lead defence readiness 2 years 50 minutes ict18 aug 2022 sr cybersecurity consultant 6 years 42 minutes ict19 aug 2022 cloud solution architect 1 year 32 minutes p20 aug 2022 platform engineering manager 16 years 51 minutes ict21 aug 2022 senior customer engineer 3.5 years 56 minutes ict22 aug 2022 principal data and applied scientist 4.5 years 48 minutes ict23 aug 2022 solutions specialist 4 years 52 minutes icb24 aug 2022 change manager 4 years 56 minutes p25 aug 2022 business group leader 9 years 41 minutes ict26 sept 2022 cloud solution architect less than a year 41 minutes p27 oct 2022 education director 5.5 years 40 minutes icb28 april 2023 future skills lead 2.5 years 45 minutes icb29 may 2023 future skills specialist less than a year 50 minutes p30 july 2023 lead of sales 6.9 years 75 minutes table 1. summary of study participants the interview questions were organised according to the three mentioned phases – prepandemic, during pandemic, and after pandemic. during the interviews, some of the questions relating to these phases were skipped if they did not apply to the participant. for instance, a participant who joined after the lifting of the lockdown and gradual return to inperson work at kappa was not asked about their experience in the pre-pandemic and during pandemic phases. both participant groups (employees and managers/leaders) were asked a similar set of questions about employee engagement elements including reporting, meetings, team collaboration, socialisation practices, communication, level of autonomy, engagement with team members, technologies used, productivity, and other aspects of the employee experience. however, a set of questions for participants in the manager/leader group were also asked to identify their practices to support their employees during these phases and the role of the exm platform viva were relevant. for the study participants who have been with kappa during the pandemic and continued into the hybrid work phase, the interviews also elicited how their leaders and the leadership australasian journal of information systems alsamarra’i, john & panteli 2025, vol 29, research article emerging leadership practices for hybrid work 9 community at kappa supported them in overcoming the challenges of the remote work, some of which continued to remain present in hybrid work, particularly in relation to wellbeing, work-life balance, social connection, and productivity. the interviews also enquired about the use of the exm platform viva, capturing responses where the platform is reported to enhance the employee experience, particularly as used by the leadership group. this data collection phase resulted in the primary dataset. our study also utilised secondary data sources that were publicly available including employee pulse reports published by kappa which were analysed for similar work trends, challenges, and practices from the leadership community to support their employees. the data coding process took place in parallel with the interviewing process where each transcript was coded using the nvivo qualitative data analysis software. three sections were created to group data by work phase and work model. these include pre-, during, and post pandemic phases to correspond with the evolution of the work model at kappa from inperson, to fully remote, then hybrid work. the interview transcripts were coded by applying appropriate codes at the sentence and paragraph level, resulting in over 110 initial codes. a second cycle of coding was conducting to clean and aggregate the initial codes according to similarity such as data relating to a specific phase (before, during, after), data relating to the exm platform viva, data relating to challenges, and data about leadership practices, to mention a few. figure 1. first data structure identifying challenges of remote and hybrid work following the completion of the coding process, a thematic analysis was conducted, guided by the work of gioia et al. (2012) and nowell et al. (2017) where thematic analysis was implemented through the identification of 1st-order concepts that adhere to the participant’s repurposing living space for work sharing working space with family members added caring responsibilities during schools closure lack of commuting 1st-order concepts set 1: challenges of remote and hybrid work longer work day working across different time zones triple peak effect increased communication back-to-back meetings lack of face-to-face connection lack of in-person social events feeling of loneliness and social disconnect lack of team connection for new employees some employees returning to in-person work some employees continue to work remote some employees working in hybrid arrangement lack of visibility over the employee experience across three modes of work concerns about wellbeing when employees are not seen in-person 2nd-order themes fading of work-life divide digital fatigue visibility of work lack of socialisation and relationship building australasian journal of information systems alsamarra’i, john & panteli 2025, vol 29, research article emerging leadership practices for hybrid work 10 language, 2nd-order themes, and finally, aggregate dimensions or global themes that relate to the emerging leadership practices. guided by these principles, two data structures were developed. the first data structure is illustrated in figure 1 and captured the challenges of remote and hybrid work as the 1st-order concepts. these led to the identification of four themes of challenges, presented as 2nd-order themes. understanding these challenges is seminal to setting the context for the emerging leadership practices that were identified. figure 2. second data structure identifying leadership practices for hybrid work next, a second data structure was developed as illustrated in figure 2. in this data structure, the leadership practices that were reported by participants were mapped as 1st-order concepts adhering to the participants language. these were then mapped to the challenges identified earlier and to which they responded, illustrated by the 2nd-order themes. finally, the emerging leadership practices that support employees in overcoming key challenges of hybrid work were derived as the aggregate dimensions. the analysis further identified that the use of the exm platform with its am features facilitated three novel leadership practices with the insights it revealed. therefore, analytics-enabled leadership is illustrated as sitting across these practices to reflect its facilitating role. next, these findings will be discussed in further details. 4 study findings in an effort to highlight how am tools can support leadership practices in hybrid work, it is critical to first highlight the challenges that were experienced as employees shifted from mostly in-person work to the current hybrid work arrangement over the last few years. these challenges are believed to impact employee engagement and would have implications for set 2: reported leadership practices individualised support for work flexibility and resources (allowing flexibility in start and end time according to family needs) modelling desired behaviour by setting work-day boundaries increased one-on-one check-ins shift tone of check-ins to focus on wellbeing and care implemented shorter meetings culture empowering employees to manage their meetings load leading with care leaders share vulnerability and present their true selves in digital space increased leader availability to new employees modelling online visibility and availability through turning on cameras lead with authenticity creating digital social events manage digital and physical spaces of workcreate physical and digital social fabric where no one is excluded adopt listening systems to create visibility on the employee experience 2nd-order themes aggregate dimensions inclusive leadership humanised leadership multimodal leadership responding to fading of work-life divide challenges responding to digital fatigue challenges responding to lack of socialisation and relationship building challenges responding to visibility of work challenges 1st-order concepts australasian journal of information systems alsamarra’i, john & panteli 2025, vol 29, research article emerging leadership practices for hybrid work 11 leadership. we argue that it is critical to identify the challenges of both remote and hybrid work for two reasons, one being that several of the challenges of remote work have continued to be present in hybrid work settings, so they are still relevant, and two considering the challenges of remote work and later hybrid work settings provide an insight into how am tools like exm platforms have been developed and continue to evolve to respond to the challenges of a changing work environment, and in our case, support emerging leadership practices for a hybrid workplace. therefore, in what follows, we present our study findings in relation to the challenges of remote and hybrid work and how leadership at kappa responded to these challenges, often with the help of am. 4.1 challenges of remote and hybrid work and leadership practices a set of challenges were reported by the participants, particularly when they first went into the enforced remote work brought about by the covid-19 pandemic. amongst these challenges are the fading of work-life divide, digital fatigue, lack of socialisation and relationship building, and visibility of employees’ work in remote settings. these challenges were experienced as employees transitioned into the enforced remote work and then hybrid work arrangements, whereby most of these challenges continued to be broadly relevant in hybrid work as well. various leadership practices were also identified that evolved during the transition of work across these phases to address these challenges as will be presented next. 4.1.1 fading of work-life divide the fading or blurring of the work-life divide is the first reported challenge. with the initial enforced remote work, participants had to repurpose their living space into an office space where their new working conditions featured a lack of a dedicated office space, sharing a working space with other family members, and lack of a dedicated office space and furniture: ‘in the early months of this, i was working off the kitchen table and then an outdoor table.’ – p5. these challenges were further heightened by schools and childcare closures, contributing to a muchadded responsibility on employees who must now juggle working and caring responsibilities. coupled with the lack of commuting which is seen as a forced break to help transition from office life to personal life, for many employees, ‘work life balance got thrown out the window.’ – p7. leadership practices quotes help employees replan their work habits to more sustainable habits and improve the work-life balance and wellbeing modelling of desirable behaviours and setting example on how to reinforce work boundaries “everybody was different, some people maybe their husband you know took a couple of days o ff and so they would work part time so they would take it. you know 2-3 days a week or some people worked shorter hours. so they worked from 10:00 until 3:00. or you know they work then late into the evening and it really depended like there is a lot of flexibility to work with your individual manager because everybody circumstances were different, so there's a lot of flexibility.” – p5 “i'm shutting off at 5 .... i want them to know that they should do that too … cause if they see me online at 9:00 o'clock they will think i should be online at 9:00 o'clock.” – p10 “if they get up in the morning and do their emails and check teams and then vanish for a few hours, we let them do that … if they are more comfortable working after dinner, we let them do that.” – p20 table 2. inclusive leadership practices to improve work-life balance australasian journal of information systems alsamarra’i, john & panteli 2025, vol 29, research article emerging leadership practices for hybrid work 12 in response to this challenge, inclusive leadership practices helped leaders to ensure that their team members were able to cope with the challenges of managing the fading of work-life divide and associated balance. leaders at kappa implemented initiatives to accommodate differences in work locations, work preferences, family situations, personas, living arrangements, access to office space, and access to required resources to support employees, particularly, during the time of fully remote work. much of this support was individualised to cater to the individual’s circumstances (e.g., allowing for flexibility in start and end times of the workday depending on family needs) and taking initiative to model desirable behaviours, therefore, creating a safe space for employees to have some autonomy to manage their worklife balance. some of the relevant examples of such practices are detailed in table 2. 4.1.2 digital fatigue digital fatigue was experienced due to a longer working day, working across different time zones, an increase in the number of emails and back-to-back meetings, and the ‘triple peak’ effect with peak time in the morning, afternoon and evening. these conditions contributed to employees feeling burnout: ‘we have people paging us at all hours of the day because we can now work from anywhere in the world.’ – p6. managers needed to be aware of their teams’ as well as their own work patterns in order to pay attention to signs of burnout and digital fatigue being experienced by their employees. humanised leadership practices that supported employees to overcome challenges related to digital fatigue were evident. these include a notable increase in one-on-one check-ins with employees, with a shift in the tone of these meetings to care and wellbeing, implementation of team-level practices to reduce digital fatigue such as defaulting meetings setup to shorter meetings to allow for breaks in between, empowering employees to manage their meetings load and have the freedom to decline meetings when it gets too much, and ensuring that their leadership approach is leaned into empathy and the notion of care. the below table 3 lists some participant quotation which reflect humanised leadership practices to help employees overcome digital fatigue in remote and hybrid work. leadership practices quotes one-on-one check-ins with employees shorter meetings leading with care and empathy “we very much leaned into that notion of care. and you know, there's lots of people who were home schooling, they were, you know, they were dealing with, you know, maybe they themselves were in isolation .. living here but had older parents back in ireland and concerned about what this might mean to them. and so, i think there was just a lot of care and empathy, particularly in those early days.” – p5 “our team made an agreement, we would move our meetings to 50 minutes or 25 minutes so that there would be a 10 or a 5 minute buffer between meetings … you can default it in outlook … we all did that as a team.” – p15 “i think one of the things, the initiatives that we looked at was really around meeting culture and really empowering employees to start, you know, feeling empowered to say, actually, no, i don't wanna join or you know, you've got four people from my team, so i'm gonna leave it to ex.” – icb14 table 3. humanised leadership practices to overcome digital fatigue australasian journal of information systems alsamarra’i, john & panteli 2025, vol 29, research article emerging leadership practices for hybrid work 13 4.1.3 lack of socialisation and relationship building socialisation, engagement, and relationships has been challenged as well, particularly with the lack and diminished in-person social connection, in-person social events, and the incidental conversations, causing a feeling of loneliness and social disconnect: ‘the thing that made it hard was that ad hoc personal conversation, the relationship building, you know, having a coffee with someone and talking about family or what you did on the weekend, it was a lot harder to do that.’ – p16. this was particularly challenging for newly onboarded employees who have no prior interaction with their new team and little understanding of the sub-cultures that can be difficult to pick up when team members are working in remote or hybrid modes. therefore, socialisation and relationship building were key challenges reported in remote and hybrid work environments. multimodal leadership practices where leaders made themselves visible and available to both remote and in-person employees were reported, often with an authentic and humanised touch were leaders presented their true selves to support their employees’ overcome challenges related to lack of socialisation and relationship building. examples of such practices are presented in table 4. leadership practices quotes present their true self and share vulnerability in digital space increase availability to new employees to support them with onboarding active encouragement of participation by setting role model and always turning camera to increase visibility make themselves available to both remote and in-person employees creating digital social events “to our managing director’s credit, mr xxx created a space in which people felt like they could be really authentic, you know he was experiencing exactly the same as everybody else. he was in his home office at home where he had boxes piled high. he had his dogs, his kids interrupting, you know, hit it kind of humanized.” – p5 “when i was on boarding [manager] would always check up on me probably every second day, just see how you going like just checking up or what's your latest training that you have to do? so my manager was very intentional.” – icb2 “because i manage a team, i'll be honest, there's an expectation i'm back twice a month again like normalized. and the team were craving it. like, when are you coming, which i thought was interesting because i thought i was doing a great job remote, but they craved seeing me and i think it's because of what we talked right at the start. the bump space-time, the like kitchen conversation or corridor or like just that social fabric that gets built in the nonscheduled non agenda type interactions that are only really made possible in person.” – p15 “we've done virtual escape rooms … also was doing virtual coffee catchups.” – p1 “my colleague was experiencing a 5-year anniversary o what my manager did was she actually set up a surprise event where she said it was like a team checking in, and she actually sent all around a cocktail making kit. we just like make cocktails together, we played virtual games together.” – p2 table 4: multimodal and humanised leadership practices to promote socialisation and relationship building 4.1.4 visibility of work as employees at kappa slowly returned to in-person work while keeping some working from home arrangements, the hybrid work arrangement continued to maintain a prominent degree of remoteness. with most employees maintaining a combination of working some days from australasian journal of information systems alsamarra’i, john & panteli 2025, vol 29, research article emerging leadership practices for hybrid work 14 home and some days in the office, managers and leaders were challenged with the visibility over their teams’ working patterns and engagement levels, particularly ones that could lead to burnout and negatively impact their wellbeing including working late into the evening, working outside conventional hours, lack of personal focus time, and social isolation. with these challenges, managers and leaders needed to ensure that they maintain oversight over their team members and that their work is visible and captured irrespective of where they work from. one participant noted the need to ‘accommodate a workforce that is hybrid so that no one is disadvantaged by choosing to be working remote from home, and so i’m choosing to be in the office, and you know we refer to being sort of inclusive so you don’t accidentally exclude anyone.’ – p25. employees reported an increased reliance on information and communication technologies such as microsoft teams, outlook, sharepoint, web cameras, and other technological resources to support their work but also enhance the visibility on employees and their work, not from a surveillance point of view, but from an inclusion point of view where employees are supported and able to choose their work arrangement without concerns over their work not being visible. in response to the added complexity of a workforce where some employees work remote, some in-person, and some hybrid, multimodal leadership practices were evident to support employees working across there various modes, creating both physical and digital spcial fabric and spaces of work that are inclusive, and having listening systems in place to create visibility on the employee experience. examples of such practices are illustrated in table 5. leadership practices quotes manage and support employees who want to work remote or inperson, so they are not disadvantaged by their choice creating inclusive spaces of work “being a people's leader, have you got a mix of team members where some want to go back to the office and have already gone back and some wanna resume working from home? … the majority i would say are comfortable to only come into the office once or twice a week. the majority are now happy working in a in a hybrid way.” – p17 “if everybody else is in a room and there's one or two people online, it's hard to keep you engaged. so what we do is we say, right, everybody get on your laptop in the room, we can all see each other. we have, we have the people who are remote up on the screen in front of us. and you still do the chat. you still talk like, so the people who are on feel like they're still much more part of it. the people in the room still feel like they've got in the room and because they're up.” – p27 table 5: multimodal leadership practices to enhance visibility of work now that the challenges of remote and hybrid work and emerging leadership practices responding to these challenges have been identified, we next present on the prevalence of am tools that can support such leadership practices using the example of an exm platform viva which has am enabled features. 4.2 the prevalence of exm platforms in the hybrid work setting am tools, empowered by ai is an innovative way to deal with the challenges experienced by employees and leaders in hybrid work (nyman et al., 2023). digital work data captured in the workplace office applications can be used to visualise organisational and individual practices captured at a granular level of detail and generate insights related to organisational and individual work patterns. these insights can be used enhance management of productivity australasian journal of information systems alsamarra’i, john & panteli 2025, vol 29, research article emerging leadership practices for hybrid work 15 and development of engagement and collaboration, while also improving employee wellbeing – all of which can contribute to enhanced employee experience. therefore, this section will present on how exm platforms with am features capture digital work data to help employees and leaders use insights gained from the exm platforms to support various forms of leadership practices to overcome the challenges identified in remote and hybrid work. 4.2.1 capturing hybrid work data with an exm platform at the time of the staged return to in-person work, kappa adopted microsoft viva, an exm platform to enhance the employee experience and create visibility on employees’ engagement, work habits, and routines by capturing work data and generating insights (john et al., 2022). microsoft viva is powered by microsoft 365 and is integrated into microsoft teams to foster employee experience in the hybrid work model: ‘when you think about the employee experience it transcends a whole range of things. so, whether you're digital, in person, in between, i think viva is designed to think about the employee experience pretty holistically.’ – p25. as employees and managers are challenged by frequent and enforced changes in their work environment, am enables the capturing of work data related to employees’ work regardless of place and time of work or device used. one participant noted that: ‘i do get a report regularly around how much time i am spending outside working hours online. what is the percentage of email i do during nine to five versus outside of that window ... it's not something that i had paid attention to before, but during covid i did and now.’ – p12. for employees, the benefits of am tools like microsoft viva is evident, alloiwng them to understand their own work habits and help them improve the worklife balance that was largely challenged by the recent shifts in work modes. 4.2.2 monitoring employee experience a listening system illustrated by various forms of work analytic and insights is also evident in microsoft viva. our study findings show that the insights were used to inform ways for improving work productivity and reducing digital fatigue. leadership strategies that demonstrate a leadership sentiment of care for employees’ wellbeing facilitated by microsoft viva was evident in the study. one of the commonly appreciated practices is the increased and consistent check-ins by managers. these check-ins varied from organisational pulse checks, aimed at checking how employees are doing, to frequent one-on-one in person meetings with managers and leaders. the focus of the remote or in-person check-ins had shifted to employee needs and ensuring that they are doing and progressing well. connecting with this sentiment of care, some features of the exm platform viva supported managers to continue to have oversight over their employees through the viva manager ‘insights’ function. this function gives managers visibility into teams whose work patterns may lead to burnout and stress as illustrated by the example in figure 3. one participant reconfirmed the potential use of viva insights to help managers listen to employees and take action towards improving their wellbeing noting: ‘there are manager insights and it can tell early indications that your team might be burning out by working late. so if people are logging on and sending emails at midnight or sort of outside their typical working hours, that insight would be surfaced to your manager.’ – p6. australasian journal of information systems alsamarra’i, john & panteli 2025, vol 29, research article emerging leadership practices for hybrid work 16 figure 3. viva manager insights example1 viva insights highlight patterns of concern such as after-hours work, meeting overload, and little focus time, which are believed to create awareness and help managers take necessary steps to help their teams reevaluate their work habits into more sustainable and desirable patterns that have less negative impact on their wellbeing. notable about the platform is that viva provides these insights while maintaining the privacy and confidentiality of the data as managers only get access to aggregated team data rather than individualised data. one participant shared that: ‘the manager insights is very powerful. it’s not personal like you can’t …but you know there are ways we can see the trends … if people are disengaging, not individual … but teams. it’ll say to me’ you haven’t had a one-on-one with this direct report for two weeks.’ – p15. in terms of challenges on employees’ work patterns visibility, participants reported how team insights in exm platform influenced leaders to pay attention to employee engagement and wellbeing: ‘there are manager insights and [tool] can tell early indications that your team might be burning out by working late.’ – p6. managers found viva insights to be very useful in creating visibility around their teams’ work habits and signs of disengagement. with this data, managers were able to take necessary steps to help their teams replan their work habits and improve their wellbeing without any misinterpretations or assumption of a productivity paranoia. 4.2.3 fostering leadership practices the burnout signals revealed to managers by the exm platform viva empowers managers to determine strategies to manage their employees’ wellbeing. managers who used insights and recommendations based on am features found it very useful for managing their teams’ work habits. drawing upon these insights, managers and leaders reported paying closer attention to the data and analysing this information to identify the experienced challenges in a changing work environment. hence, analytics-enabled leadership practices facilitated by exm platforms are found to enable and support various leadership practices. the integration of intelligent algorithms with digital technologies such as exm platforms contributes to the generation of aggregated analytics and insights that promotes analytics-enabled leadership by assisting managers and leaders in the complexity of a hybrid work environment. table 6 1 https://www.microsoft.com/en‐au/microsoft‐viva/insights https://www.microsoft.com/en-au/microsoft-viva/insights australasian journal of information systems alsamarra’i, john & panteli 2025, vol 29, research article emerging leadership practices for hybrid work 17 illustrates examples of how insights from exm platforms support analytics-enabled leadership to foster various types of leadership practices. quotes how exm supports leadership modes of leadership supported “it's all aggregated information and privacy protected. but yeah, you'll get insight into the way your team is working if they're spending too much time on teams calls, rather than having enough time to actually focus on the work they need to deliver.” – p6 provide managers with team insights into teamwork patterns and early indications of burnout and stress (including little focus time, too many meeting). humanised leadership “it's not down to the human level. i don't want you to think, oh we can see that [person] is working well. like it's not at that level … so it's not like it's i can see that [person] is doing this. that's not what it does. but yes, i think we use our technology and the listening systems in place to be able to look at some of the signals which we were then able to then use to then ensure that we had the policies that benefits the education for our people leaders.” – p25 creates visibility to over not only in-person employees, but also remote employees and flag if remote employees (teams) are disengaging and provide a nudge for the need to meet with employees or other artefacts that supports managers in managing a hybrid workforce. inclusive leadership “the manager ‘insights’ is very powerful. it’s not personal but you know there are ways we can see the trends. if people are disengaging, not individual, but teams, it’ll say to me you if i hadn’t, like, you haven’t had a one-on-one with this direct report for two weeks.” (p15) capture team level insights on work patterns and help to impact work-life balance. multimodal leadership table 6. leadership practices fostered by analytics-enabled leadership in terms of leadership strategies relating to employee connectedness and engagement, leaders demonstrated a leadership style of shared vulnerability, showing one’s true self and true circumstance. one participant noted: ‘i've seen some of our leaders be really vulnerable during the last 18 to 24 months, and i think it's led to much deeper, more personal connections within teams and much deeper and better relationships.’ – p5. this notion of shared vulnerability is believed to demonstrate courage, build trust, and enhance team connectedness – all of which has been observed across the leadership community at kappa. building on the concept of vulnerability, managers took initiative to spend more time with new employees in the first few months of their onboarding and have frequent one-on-ones meetings with them, often on a weekly basis, as often this is a challenging time and new employees who have not met their team members and may have a sense of vulnerability. viva prompts managers to enhance connectedness with team members: ‘it’ll say to me you if i hadn’t… like you haven’t had a one-on-one with this direct report for two weeks.’ – p15. new employees in a hybrid work setting require more support in gaining knowledge of work and developing their skills and connections, and managers play an important role in setting a good onboarding experience. to summarise, the findings highlight how insights from an exm platform empowered leaders to engage in participative decision-making, to show concern for wellbeing, to interact with the australasian journal of information systems alsamarra’i, john & panteli 2025, vol 29, research article emerging leadership practices for hybrid work 18 teams based on their needs, to lead by example, to coach and to show care and vulnerability, creating a safe space for other to do so. among the various listening systems available, participants particularly highlighted the role of microsoft viva in empowering both leaders and employees in managing their work more effectively remote and hybrid work settings. 5 discussion the study has been driven by an interest in understanding the role of am facilitated by exm platforms in dealing with the underlying challenges of hybrid work and its impact on leadership practices. notable of the modern business environment is the wide adoption of tools and technologies which are believed to have an impact on day-to-day work, including the work of leaders (shields, 2024). current literature on ai technologies echoes three key perspectives on leadership and ai – an enhancement perspective where ai technologies can enhance leadership functions (titareva, 2021), a replacement perspective, and a sceptical perspective. we therefore argue that if the workplace has undergone digital transformation and such technologies are being widely adopted and used, then leadership is likely to have changed as well, not by means of adaptation of old ways, but through a transformation process that led to a new understanding of leadership, one that is more fitting with digital work regardless of how it is done (in-person or remotely). accordingly, our empirical findings indicate that leadership has undergone a transformation process, resulting in the emergence of new modes of leadership. drawing upon our findings, we propose that successful leadership of a hybrid workplace is a reformed leadership approach that combines characteristics of three modes of leadership as evident in the study. these include inclusive leadership, humanised leadership, and multimodal leadership, all of which can be supported by analytics from tools such as exm, hence we propose that analyticsenabled leadership is underpinning these emerging modes of leadership as illustrated in figure 4. multimodal leadership inclusive leadership humanised leadership analyticsenabled leadership figure 4. emerging modes of leadership for hybrid work to begin with, inclusive leadership is the first mode of leadership that is well aligned with a hybrid workplace. it recognises diversity, responds to individual needs and work styles, and actively listens to team members voice (roberson & perry, 2021). in existing work models such as virtual work, diversity is understood in terms of temporal, geographical, cultural, australasian journal of information systems alsamarra’i, john & panteli 2025, vol 29, research article emerging leadership practices for hybrid work 19 linguistics, gender, and other conventional differences underpinning diversity. however, in the hybrid workplace context, the notion of diversity have been extended to further include diversity in individual work preferences (remote vs in-person vs hybrid), work styles, states of physical and mental wellbeing, caring responsibilities, commuting requirements, generational differences (new graduates vs experienced adults), participation preferences (e.g., chat vs voice vs camera), socially connected vs isolated employees, and other individual based circumstances that are beyond job and role requirements (wasko & dickey, 2023). in this sense, inclusive leadership acknowledges diversity along the noted dimensions and more broadly along the personal lives of employees beyond professional interactions in a conventional workplace and accordingly provides individualised support. for instance, for employees with childcaring responsibilities, am enabled tools like exm platforms can highlight very early morning or late evening working patterns. inclusive leadership approaches can leverage on these insights to provide individualised consideration for such employees, for instance, to work outside the conventional working hours. our findings indicate that an inclusive leadership that puts emphasis on care is key to successful leadership in hybrid work (nakamura & milner, 2023). inclusivity is not to be left to chance but needs to be deliberate to avoid the accidental exclusion. inclusive practices encompass having hybrid events, so no one is excluded, provision of individual consideration in terms of coaching and mentoring, and provision of resources in consideration of personal circumstances. hybrid work has presented itself as the preferred option for many employees as well as organisations. therefore, post-pandemic employees expressed a desire to return to the office and to a co-located workplace, to engage, connect, and regain the energy derived from being in the same physical space as their team members. one of the important messages for leaders in hybrid work is to ‘deliberately include rather than accidently exclude’, that is, leaders need to consider leadership practices that does not put anyone at a disadvantage due to their work preference or personal circumstances (john et al., 2022). am tools can be used to ensure everybody is included. practices such as requiring web cameras to be on in hybrid meetings and recording of meetings to ensure all are included or recommendations about suitable meeting time are some examples of how leaders use exm platforms effectively to enact inclusion. second, humanised leadership is another form of leadership observed and centres around the demonstration of authenticity and acting according with one's ‘true self’ (gardner et al., 2021). within the context of our proposed leadership practices for hybrid work, humanised leadership manifests in the shift in focus towards the ‘care’ element. given the challenges that can arise from the blurring of the work-life divide and its impact on employee wellbeing, leaders in a hybrid work setting need to be more attentive to the wellbeing of their employees and their individual needs (perrigino & raveendhran, 2020), particularly while working in repurposed office spaces that overlap with living space. this can include showing empathy and compassion and would require a more relationship-oriented approach. part of showing one’s true self is achieved through accessibility, therefore, humanised leaders are more accessible and approachable. in the hybrid work context, accessibility is not only about being available for one-on-one meetings with reports or being responsive, but is also about being present and visible such as when leaders turn their cameras on during meetings and model accessibility behaviour. this practice reflects a human-centric approach where leaders show authenticity and are not shy to australasian journal of information systems alsamarra’i, john & panteli 2025, vol 29, research article emerging leadership practices for hybrid work 20 show vulnerability which is believed to contribute to building trust and better team collaboration. humanised leadership approaches show that leaders are interested, highly present, care, and are listening (levay & andersson bäck, 2022; longmuir, 2023; taskin et al., 2024). this is not only echoed through verbal cues, but also through non-verbal cues such as how people look, how they interact, and what their body language communicates. furthermore, successful leadership of a hybrid workplace requires multimodal leadership which entails that leaders can manage employees and monitor work performance and progress whether they work in-person or remotely. accordingly, leaders can come up with strategies that support their employees’ engagement and wellbeing in both modes of work (hooijberg & watkins, 2021). multimodal leadership is about being able to extend leadership practices into a digital space, and with data generated by tools like exm platforms, leaders have better visibility over their distributed workforce, where visibility on digital work, work patterns, and overall employee engagement is no longer situated to physical co-location (john et al., 2023). multimodal leadership reflects the ability to remain connected to employees regardless of their spatial and temporal work dimensions and to create a space where both inperson and remote working employees feel like they are part of one social fabric (antonacopoulou & georgiadou, 2021). in this common space, team members feel a sense of connection to their team members, managers, and their leaders. in multimodal leadership, leaders capitalise on the potential of digital technologies to create new spaces for real-time connection with employees, where they can communicate corporate updates, top news, goals, facilitate the professional development of employees, build trust and collaboration, drive creativity, drive innovation, and encourage employees to continue to learn and develop, all while operating in a hybrid work model. these emerging practices indicate that authentic and humanised leaders of a hybrid workplace are able to adapt leadership practices that bring their work and home personalities closer together, creating a shared culture and a safe environment for employees to also be their true self. these changes are believed to flatten the traditional corporate hierarchy and make higher level executives and leaders more accessible to junior employees in remote (mandviwalla et al., 2021) and hybrid work. exm platforms can play a role in supporting leaders to show authenticity and a human-centric leadership approach. one example is in the way these tools pull information from emails and chats and reminds users such as leaders about commitments and follow-ups with their employees, an avenue to demonstrate authenticity. finally, analytics-enabled leadership can be viewed as being part of the ‘new’ or emerging forms of leadership discussed by quaquebeke and gerpott (2023) that is ai augmented. according to kifor et al. (2021), digitally powered analytics allows leaders to enhance the effectiveness of people-related decision-making and create an understanding of employee performance, working habits, and work-life balance shifts. analytics-enabled leadership entails that leaders listen to the insights and sentiments to drive decisions and actions. the integration of intelligent algorithms with digital technologies like exm platforms contributes to the generation of aggregated analytics and insights that assist managers and leaders in the complexity of a hybrid work environment. not only do these analytics highlight work pattern data and nudges employees towards desirable actions, but they also enable a responsive form of leadership that pays attention to the data generated by these systems and intentionally adapt their leadership strategies to support employees in a changing work landscape. australasian journal of information systems alsamarra’i, john & panteli 2025, vol 29, research article emerging leadership practices for hybrid work 21 this finding agrees with the concept of inverse transparency by gierlich-joas et al. (2024) where both employees and leaders are empowered by the transparency or visibility on work patterns. while recognising that technologies which create visibility on work patterns can drive concerns for privacy and surveillance (leonardi & treem, 2020), technologies such as the exm platform provide opportunities for transparency, which if utilised appropriately, can benefit employees and managers alike. analytics-enabled leadership thus reduces the transparency gap between monitoring employee’s work and empowering employees to have some control and autonomy over their work. therefore, new leadership practices emerge as leaders learn to join the loop of am and engage with the insights to inform decisions and emerging leadership practices. this is in comparison to earlier leadership approaches driven by dogmatic principles and how work should be done, perhaps, pre-pandemic. within the context of analytics-enabled leadership, we observe a shift towards the data being responsible for initiating and nudging leadership behaviours and practices that are responsive to the employee insights. for instance, insights can flag and validate excessive work patterns that could lead to potential burnout, or insights indicative of social withdrawal, or employees spending too much time in meetings and having little focus time to complete their work. agreeing with the harms and han, (2019, n.p.), ‘humans need not consider algorithms to be rivals so much as a new type of partner, both for leaders and followers’. therefore, leadership can work with these insights and respond by encouraging employees to actively book focus time or give them to autonomy to set their working hours, or to push training on how to carry out effective meetings to reduce digital fatigue and burnout. cranefield et al. (2022) highlights how work analytics tools can enable better productivity and wellbeing by partnering with aidriven tools such as digital productivity assistants (dpas). we argue that this is not a deterministic relationship and that other factors linked to renewed leadership practices and workplace culture need to exist for this to happen. our study found that analytics-enabled leadership enables more human-centric leadership approaches that puts people first, one where care, emphasis on wellbeing, and provision of individualized support and consideration are key factors that enhance employee engagement in hybrid workplaces. 6 thoeretical and practical contributions with the increased popularity of hybrid work following the covid-19 pandemic and increased dependence on digital and ai-enabled am tools such as exm platforms, our study aims to contribute to the information systems literature by exploring the effect of these technologies on organisational practices, particularly, leadership practices that leverage on the am capabilities of such tools. this parallels the call for the need to study new manifestations of leadership in the context of the digital era (banks et al., 2022). first, a key contribution of our study is in relation to analytics-enabled leadership which can be considered as an emerging leadership practice in hybrid work, largely enabled by the concept of am and advanced technological tools such as an exm platform. the study has demonstrated that with the increasing use of analytics in workplaces including hybrid work settings, there are renewed possibilities for leaders to develop insights about employees’ experiences at work, their needs, and challenges, providing opportunities for leaders to develop interventions in timely and targeted manner. as illustrated in the findings and detailed in the discussion section, data captured by exm platforms and the insights facilitated australasian journal of information systems alsamarra’i, john & panteli 2025, vol 29, research article emerging leadership practices for hybrid work 22 by the am capability not only promotes analytics-enabled leadership but also supports inclusive, humanised, and multimodal leadership practices. second, our study contributes to the literature on hybrid work which has increased popularity in the post-pandemic period. it shows that challenges linked to hybrid work may be dealt with effectively through the use of am tools and exm platforms in particular. as a team becomes more diverse, multicultural, multigenerational, and global in a hybrid work model, team leaders need to take additional steps towards creating a workplace that accommodates everyone’s work preference and circumstances while maintaining productivity. insights revealed by these tools can drive the design and implementation of initiatives which demonstrate responsiveness to the changing nature of the workplace and employees’ individualized requirements. as a senior executive at kappa noted: ‘if you think about the transformation that workplaces or organisations are now reconciling today, i would say that's about hybrid, and how to bridge the digital and physical worlds, or workplaces and how to make that seamless and how to ensure that work can be productive, can be meaningful, can deliver outcomes, can create joy across both dimensions.’ – p25. therefore, this study shows that with the help of workplace analytics, data-driven rather than dogma-driven leadership gains importance and appropriateness in leading hybrid work teams. in terms of the practical implications of our study, our findings provide practical considerations for leaders and managers of hybrid teams. these considerations are worthy of attention given the global appetite for hybrid work, the continuity for some work from home arrangements, and the requirement to create a hybrid working culture that nurtures inclusiveness, trust, engagement, and wellbeing. am tools go beyond the provision of metrics on employees’ work performance. they provide opportunities for adopting a human-centric leadership approach, one that encompasses attention to learning and development, wellbeing, and work-life balance. the findings have implications for leadership training and development to ensure that individuals who lead in hybrid work settings have the skills and knowledge they need to deal with the challenges that employees are experiencing and respond appropriately. 7 conclusion and future research with the rise of am within the increasingly popular hybrid work model, organisations need to explore the impact on leadership practices. drawing on a qualitative approach and a case study design, our study shows that with the addition of am in hybrid work, leaders have an added responsibility as well as an opportunity to transform their leadership approaches where there is a shift on care, empathy, inclusion, and deliberate actions that enhance the employee experience. in contributing to research efforts on emerging and successful leadership practices suitable for a multimodal workforce, our study findings indicate that an inclusive leadership that puts emphasis on care is key to successful leadership in the hybrid work context. inclusivity here recognizes diversity, not only cultural diversity but also diversity in individual needs, circumstances and work styles. inclusivity is also demonstrated through actively listening to team members voice (roberson & perry, 2021). team leaders and organisational leaders need to pay close attention to the data generated by am tools as algorithmic insights helps reveal some unnoticed and unspoken work habits and patterns that cause employees to experience disengagement, digital fatigue and to lose their work-life balance. australasian journal of information systems alsamarra’i, john & panteli 2025, vol 29, research article emerging leadership practices for hybrid work 23 our study opens the agenda for further research in the field of am and hybrid work. the scope of our study was limited to examining the role of am in dealing with the challenges of hybrid work. as such, it was beyond the scope of this study to examine privacy and surveillance concerns which have been reported in the literature (leonardi & treem, 2020). we nevertheless encourage further research in this area, notably case studies on the use of am in hybrid work from a longitudinal perspective with the purpose to examine at what point privacy and surveillance concerns emerge and how they are managed in different organisations. moreover, with the realisation that hybrid work can take different shapes in different organisations with varying arrangements of in-person work and working from home, there is a need to expand research in this area. for example, future research may seek to explore how leadership is transformed in different modes of hybrid work to accommodate varying percentages of in-office and remote workers. similarly, further research could examine whether any particular leadership mode may dominate hybrid work settings in different organisations, all of which, can contribute to our developing knowledge of leadership in hybrid work. references adams-prassl, j., rakshita, s., abraha, h.h. & silberman, m. (2023). towards an international standard for regulating algorithmic management: a blueprint. in 8th conference of the regulating for decent work network, international labour office geneva, switzerland (pp. 10-12). antonacopoulou, e.p., & georgiadou, a. (2021). leading through social distancing: the future of work, corporations and leadership from home. gender, work & organization, 28(2), pp.749-767. apté, c., dietrich, b., & fleming, m. (2012). business leadership through analytics. ibm journal of research and development, 56(6), 7-1. avolio, b. j., & gardner, w. l. (2005). authentic leadership development: getting to the root of positive forms of leadership. the leadership quarterly, 16(3), 315-338. banks, g.c., dionne, s.d., mast, m.s., & sayama, h. (2022). leadership in the digital era: a review of who, what, when, where, and why. the leadership quarterly, 33(5), p.101634. bell, b.s., mcalpine, k.l., & hill, n.s. (2023). leading virtually. annual review of organizational psychology and organizational behavior, 10, 339-362. benlian, a., martin w., alec c., krasnova, h., alexander, m., mareike, m., jan r., & ulrich r. (2022). algorithmic management: bright and dark sides, practical implications, and research opportunities. business & information systems engineering, 64(6), 825-839. brynjolfsson, e., & mcafee, a. (2014). the second machine age: work, progress, and prosperity in a time of brilliant technologies. ww norton & company. cameron, l., lamers, l., leicht-deobald, u., lutz, c., meijerink, j., & möhlmann, m. (2023). algorithmic management: its implications for information systems research. communications of the association for information systems, 52, 556-574. australasian journal of information systems alsamarra’i, john & panteli 2025, vol 29, research article emerging leadership practices for hybrid work 24 chamakiotis, p., panteli, n., & davison, r. m. (2021). reimagining e-leadership for reconfigured virtual teams due to covid-19. international journal of information management, 60, p. 102381. cranefield, j., winikoff, m., chiu, y.t., li, y., doyle, c., & richter, a. (2023). partnering with ai: the case of digital productivity assistants. journal of the royal society of new zealand, 53(1), 95-118. dery, k., sebastian, i. auer m., & van der meulen, n. (2017). the digital workplace is key to digital innovation. mis quarterly executive, 16(2), 135-152. dinh, j. e., lord, r. g., gardner, w. l., meuser, j. d., liden, r. c., & hu, j. (2014). leadership theory and research in the new millennium: current theoretical trends and changing perspectives. the leadership quarterly, 25(1), 36-62. edwards, r., & fenwick, t. (2016). digital analytics in professional work and learning. studies in continuing education, 38(2), 213-227. essien, n. (2023). "people management in fluid workspaces: transplanting corporate values and ethical culture", ogunyemi, k. and onaga, a.i. (ed.) responsible management of shifts in work modes – values for post pandemic sustainability (vol. 2, pp. 67-82). emerald publishing limited. faraj, s., pachidi, s. & sayegh, k. (2018). working and organizing in the age of the learning algorithm. information and organization, 28(1), 62-70. gardner, w. l., karam, e. p., alvesson, m., & einola, k. (2021). authentic leadership theory: the case for and against. the leadership quarterly, 32(6), 101495. gierlich-joas, m., baiyere, a., & hess, t. (2024). inverse transparency and the quest for empowerment through the design of digital workplace technologies. journal of the association for information systems. gioia, d. a., corley, k. g., & hamilton, a. l. (2012). seeking qualitative rigor in inductive research: notes on the gioia methodology. organizational research methods, 16(1), 15-31. gratton, l. (2021). “how to do hybrid right”. harvard business review, 99(3), 65-74. harms, p.d., & han, g. (2018). algorithmic leadership: the future is now. journal of leadership studies, 12(4), 74-75. hooijberg, r., & watkins, m. (2021). the future of team leadership is multimodal. mit sloan management review, 62(3), 1-4. iszatt‐white, m., & kempster, s. (2019). authentic leadership: getting back to the roots of the ‘root construct’?. international journal of management reviews, 21(3), 356-369. jarrahi, m.h., newlands, g., lee, m.k., wolf, c.t., kinder, e., & sutherland, w. (2021). algorithmic management in a work context. big data & society, 8(2), p. 20539517211020332. john, b., alsamarra’i, z., & panteli, n. (2022). enhancing employee experience in the era of hybrid work: the case of microsoft viva. ieee software, 40(2), 70-79. australasian journal of information systems alsamarra’i, john & panteli 2025, vol 29, research article emerging leadership practices for hybrid work 25 john, b., panteli, n., alsamrra’i, z., & li, a. (2023). reconfiguring digital work: a sociotechnical reversal perspective. in australasian conference on information systems (acis) proceedings (pp. 1-13). kellogg, k.c., valentine, m.a., & christin, a. (2020). algorithms at work: the new contested terrain of control. academy of management annals, 14(1), 366-410. kifor, c. v., nicolaescu, s. s., florea, a., săvescu, r. f., receu, i., ţîrlea, a. v., & dănuţ, r. e. (2021). workforce analytics in teleworking. ieee access, 9, 156451-156464. kinowska, h., & sienkiewicz, ł. j. (2022). influence of algorithmic management practices on workplace well-being–evidence from european organisations. information technology & people, 36(8), 21-42. kuknor, s. c., & bhattacharya, s. (2022). inclusive leadership: new age leadership to foster organizational inclusion. european journal of training and development, 46(9), 771-797. larsen-ledet, i., & lindley, s. (2022). ways of seeing and being seen: people in the algorithmic knowledge base. in workshop at the 20th european conference on computer-supported cooperative work (pp. 1-6). lehmann-willenbrock, n., meinecke, a. l., rowold, j., & kauffeld, s. (2015). how transformational leadership works during team interactions: a behavioral process analysis. the leadership quarterly, 26(6), 1017-1033. leonardi, p. m., & treem, j. w. (2020). behavioral visibility: a new paradigm for organization studies in the age of digitization, digitalization, and datafication. organization studies, 41(12), 1601-1625. levay, c., & andersson bäck, m. (2022). caring leader identity between power and powerlessness. organization studies, 43(6), 953–972. longmuir, f. (2023). leading in lockdown: community, communication and compassion in response to the covid-19 crisis. educational management administration & leadership, 51(5), 1014-1030. mandviwalla, m., & flanagan, r. (2021). small business digital transformation in the context of the pandemic. european journal of information systems, 30(4), 359-375. marabelli, m., vaast e., & carlile e. (2020). making lemonade: dealing with analytics surveillance in the workplace. academy of management proceedings, 2020(1), p. 12720. mayer-schönberger, v., & cukier, k. (2013). big data: a revolution that will transform how we live, work, and think. houghton mifflin harcourt. möhlmann, m. (2021). algorithmic nudges don’t have to be unethical. harvard business review, 22, 1-7. möhlmann, m., alves de lima salge, c., & marabelli, m. (2023). algorithm sensemaking: how platform workers make sense of algorithmic management. journal of the association for information systems, 24(1), 35-64. nakamura, y.t., & milner, j. (2023). inclusive leadership via empathic communication. organizational dynamics, 52(1), p.100957. australasian journal of information systems alsamarra’i, john & panteli 2025, vol 29, research article emerging leadership practices for hybrid work 26 newlands, g. (2021). algorithmic surveillance in the gig economy: the organization of work through lefebvrian conceived space. organization studies, 42(5), 719-737. nowell, l. s., norris, j. m., white, d. e., & moules, n. j. (2017). thematic analysis: striving to meet the trustworthiness criteria. international journal of qualitative methods, 16(1), p. 1609406917733847. nyman, s., bødker, m., & blegind jensen, t. (2023). reforming work patterns or negotiating workloads? exploring alternative pathways for digital productivity assistants through a problematization lens. journal of information technology, 71, 1-18. pachidi, s., berends, h., faraj, s., & huysman, m. (2021). make way for the algorithms: symbolic actions and change in a regime of knowing. organization science, 32(1), 18-41. panteli, n., giæver, f., & engesmo, j. (2022). guest editorial: emotions in the digitalised workplace. information technology & people, 35(6), 1677-1692. panteli, n., yalabik, z. y., & rapti, a. (2019). fostering work engagement in geographicallydispersed and asynchronous virtual teams. information technology & people, 32(1), 2-17. park, h., ahn, d., hosanagar, k., & lee, j. (2021). human-ai interaction in human resource management: understanding why employees resist algorithmic evaluation at workplaces and how to mitigate burdens. in proceedings of the 2021 chi conference on human factors in computing systems (pp. 1-15). perrigino, m.b., & raveendhran, r. (2020). managing remote workers during quarantine: insights from organizational research on boundary management. behavioral science & policy, 6(2), 87-94. quaquebeke, n.v., & gerpott, f.h. (2023). the now, new, and next of digital leadership: how artificial intelligence (ai) will take over and change leadership as we know it. journal of leadership & organizational studies, p. 15480518231181731. roberson, q., & perry, j. l. (2022). inclusive leadership in thought and action: a thematic analysis. group & organization management, 47(4), 755-778. schneider, s., & kokshagina, o. (2020). digital technologies in the workplace: a ne(s) t of paradoxes. in proceedings of the forty-first international conference on information systems (p. 1-1). schuetz, s., & venkatesh, v. (2020). the rise of human machines: how cognitive computing systems challenge assumptions of user-system interaction. journal of the association for information systems, 21(2), 460–482. shields, k. d. (2024). transformative or disruptive? exploring the impact of generative ai on leadership. [master’s thesis, zhaw zürcher hochschule für angewandte wissenschaften]. https://doi.org/10.21256/zhaw-30469. spataro, j. (2022). beyond the binary: solving the hybrid work paradox. available at: https://www.microsoft.com/en-us/worklab/beyond-the-binary-solving-the-hybridwork-paradox [accessed 03rd march 2022]. taskin, l., klinksiek, i., & ajzen, m. (2024). re‐humanising management through co‐presence: lessons from enforced telework during the second wave of covid‐19. new technology, work and employment, 39(1), 143-167. australasian journal of information systems alsamarra’i, john & panteli 2025, vol 29, research article emerging leadership practices for hybrid work 27 titareva, t. (2021). leadership in an artificial intelligence era. presented at leading change conference, james madison university. tonkin, t. h. (2013). authentic versus transformational leadership: assessing their effectiveness on organizational citizenship behavior of followers. international journal of business and public administration, 10(1), 40-61. wasko, m., & dickey, a. (2023). managing where employees work in a post-pandemic world. mis quarterly executive, 22(2), p.5. willems, t., & hafermalz, e. (2021). distributed seeing: algorithms and the reconfiguration of the workplace, a case of automated trading. information and organization, 31(4), p. 100376. winikoff, m., cranefield, j., li, j., doyle, c., & richter, a. (2021). the advent of digital productivity assistants: the case of microsoft myanalytics. in proceedings of the 54th hawaii international conference on systems sciences (pp. 338–347). yukl, g. (1989). managerial leadership: a review of theory and research. journal of management, 15(2), 251-289. acknowledgements we would like to thank all the participants of this project. this work was supported by the 2021 british academy of management–australian and new zealand academy of management collaborative research award (2021-262-bam-anzam). this work involved human subjects or animals in its research. approval of all ethical and experimental procedures and protocols was granted by the human research ethics committee under application no. 10402 and performed in line with the requirements determined by the university policy and the national statement on ethical conduct in human research. copyright copyright © 2025 zeena alsamarra’i, blooma john & niki panteli. this is an open-access article licensed under a creative commons attribution-non-commercial 4.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v29.4909 https://creativecommons.org/licenses/by-nc/4.0/deed.en https://creativecommons.org/licenses/by-nc/4.0/deed.en https://doi.org/10.3127/ajis.v29.4909 paper title (paper title style) australasian journal of information systems ali, khan & wang 2025, vol 29, research article understanding boredom in online commerce received 20 sep 2023, accepted 11 november 2024 after 4 rounds of review with karlheinz kautz as accepting editor 1 understanding boredom in online commerce: a systematic literature review rebecca ali waikato management school university of waikato hamilton, new zealand email: rebecca.ali@waikato.ac.nz gohar khan college of technological innovation zayed university dubai, uae william wang waikato management school university of waikato hamilton, new zealand abstract boredom is a ubiquitous emotion, whether in a moral, social, or practical sense. in the context of digital communications and business, extensive research has explored the ways in which this adverse sentiment influences the utilisation of digital technology. however, these studies have ignored this phenomenon in the context of online commerce. the present work aims to assess knowledge on boredom in this context. through a review of 103 studies using grounded theory method for rigorous literature review, an inductive framework using five building blocks is built. from an individual’s perspective, the framework foregrounds that boredom is caused by individual, situational, and their mismatch factors. as a result of boredom’s self-regulatory function, individuals often search for internal and external stimulation from social commerce, resulting in both positive and negative outcomes. the findings of this study indicate that boredom is both a performance and a health barrier; however, proper understanding and intervention tactics that utilise constructive alleviation strategies can lead to promising results in self-development. furthermore, the findings also indicate how regulated and unregulated boredom affects the market performance of online commerce and how preventing and regulating this emotion can lead to sales. keywords: boredom; online commerce; social media; online buying; grounded theory, digital technology. 1 introduction it has been argued in the field of evolutionary biology that without boredom, mankind could never have advanced further than other species. the avoidance of boredom is what ultimately sparked our ancestors to rub sticks together to create fire or create the wheel to carry heavy loads. (whelan et al. 2020, p. 871) the preceding quotation illustrates the significance of boredom for advancement in life. being a functional emotion, it alerts us to any monotonous, meaningless, and unsatisfactory states australasian journal of information systems ali, khan & wang 2025, vol 29, research article understanding boredom in online commerce 2 and motivates us to do something about it. it keeps us in line with our desired interests and meaningful goals, leading towards life satisfaction and development. the covid-19 pandemic and lockdowns have intensified boredom experiences. with limited social connections and outdoor activities, people shift towards online platforms, like social media, for engagement to occupy themselves in this free time (catedrilla et al. 2020; deng et al. 2020). even though the pandemic caused unstable consumption, leading to market vulnerability, online shopping still thrived. there were two reasons for this overconsumption: one was to reduce the risk of getting ill from going outdoors and the other was to alleviate boredom (güngördü 2022). overall, research findings recognise the development of unregulated online consumption and digital technology usage to evade boredom. boredom is generally defined as “the aversive experience of wanting but being unable to engage in a stimulating and satisfying activity” (fahlman et al. 2013, p. 69). it is associated with meaningless and monotonous experiences which cause dissatisfaction. it’s affective, cognitive, and functional features contribute to its uniqueness, as they ensure that it is a selfregulatory emotion. it seeks to find alternatives when the current situation is unattractive, unsatisfactory, or meaningless (bench & lench 2013; elpidorou 2015; pekrun et al 2010; van tilburg & igou 2012). boredom differs from other negative emotions more so than they differ from each other. its existential nature, inattention, and low arousal features make it distinctive. among other negative emotions, it is characterised as having the least negative affective valence, in that it is only mildly unpleasant (van tilburg & igou 2017). boredom matters not only because it affects a large percentage of our population but also because it teaches us how to embody ourselves in our worldly projects and social affairs. it matters because it keeps us in line with our goals and safeguards our interest by facilitating selfregulation (elpidorou 2015). despite its uniqueness, this “silent emotion” has received little attention compared to other negative emotions like anxiety, anger, depression, and loneliness (eastwood et al., 2012). boredom is mostly understood as an informal term, and its relationship with its characteristics, consequences, and antecedents are poorly recognised (raffaelli et al. 2018). how it affects people’s thoughts, behaviour, and social environment is still being questioned (fahlman et al. 2013). to date, an understanding of the nature of boredom has still not been fully achieved and there is no consensus on how to define it (vodanovich & watt 2015). in fact, this affective state “remains a construct that is difficult to define” (elpidorou 2018, p. 2). in digital technology studies, boredom has been given definitions such as “a state of mental weariness and dissatisfaction produced by lack of interest or activity” (blaszczynski et al. 1990, p. 36); a state of “under-stimulation, under-arousal, and lack of psychological involvement associated with dissatisfaction, [whereby] individuals try to cope with boredom by seeking additional stimulation” (biolcati et al. 2018, p. 304); or “cognitive changes in the direction of less differentiated and more homogeneous construing [that] give rise to a state of subjective monotony which induces, or perhaps even represents, the state” (perkins & hill 1985, p. 231). circularity (incorporating antecedent and outcome variables and having hypothesis type statements such as homogeneity, less differentiation, and lack of interest cause/increase boredom) and unclear/ambiguous terminologies (i.e., not stating the kind of activity or stimulation) in these definitions hinders a conceptual clarity of boredom. australasian journal of information systems ali, khan & wang 2025, vol 29, research article understanding boredom in online commerce 3 the topic of boredom opens a number of avenues for research, especially in consumer behaviour and technology usage. understanding its influence on online buying decisions and well-being is crucial for marketers and policymakers. it is linked with modernism for it enables self-regulation through the desire for technological change and usage, which can be for the better or worse. previously, studies on boredom primarily focused on education (derakhshan et al. 2022; pawlak et al. 2021) and psychology (meier et al. 2024; belinda et al. 2024). however, it has recently gained momentum in the context of digital technology and online commerce. (allaby & shannon 2020; drach et al. 2021; throuvala et al. 2019). online commerce is defined as “the sale and purchase of products and services over the internet” (keeney 1999, p 533). the number of online users are increasing every year. it has been reported that 5.35 billion people using the internet in 2024, equating to 66.2 percent of the world’s total population. internet users have grown by 1.8 percent over the past year, with 97 million new users coming online for the first time during 2023 (kemp 2024). the same increase can be seen in the use of different digital technologies, like social media, e-commerce, s-commerce (statistica 2022), and its prospects are proving a big attraction for marketers and businesses which most boredom studies appear to have overlooked. moreover, no conceptual definition of boredom exists in the online commerce context (see table a.1 of appendix a). previous studies have used either general or unclear definitions of boredom when attempting to understand it in the online context. we currently lack a comprehensive understanding of this phenomenon, for which we need to define in this context. this study recognises the gap in the literature and the need for boredom research in the online commerce context. the present study aims to conduct a rigorous literature review of current knowledge on this topic to answer the research questions: rq1: what is the conceptual definition of boredom in online commerce? rq2: how does boredom function in online commerce? (a) what are boredom’s antecedents in online commerce? (b) what are boredom’s alleviation strategies in online commerce? (c) what are the consequences of boredom in online commerce? this is the first study of its kind to define boredom exclusively in the context of online commerce, thereby filling the gap in the literature. it offers an inductive framework (see figure 2 in the findings section) that proposes the potential antecedents, mediators, moderators, and outcomes of boredom in the online commerce context to respond to future research queries on boredom. this research confirms the applicability of boredom theories in the online commerce context, the study of which is absent in past literature. lastly, this study has important implications related to constructively regulating boredom to improve the wellbeing of individuals and the profitability of associated businesses. consistent with the breadth of the study’s research questions, the present study adopted an inductive approach using techniques borrowed from grounded theory for rigorous review (wolfswinkel et al. 2013) and reviewed 103 studies. based on the literature analysis, this study discusses boredom from an individual’s perspective as a common negative state conceptualised by individual and situational factors. it has a functional self-regulatory feature that encourages individuals to seek stimulation online, resulting in detrimental and beneficial well-being outcomes. then this study, based on the review, defines boredom in the online commerce context from a functional point of view at an individual level, based on the guidelines of a good conceptual definition (wacker 2004; suddaby 2010). the study also australasian journal of information systems ali, khan & wang 2025, vol 29, research article understanding boredom in online commerce 4 provides insights into boredom’s effect on businesses. it claims that boredom is an important emotion that calls for the research that contributes to its understanding for scholars, marketers (currently managing an online platform, planning a marketing campaign, or launching new products), and policymakers. understanding the emotion of boredom can ultimately enhance an individual’s well-being and productivity as well as business profitability. 1.1 online commerce the pioneering of online commerce applications can be tracked to the early 1970s when money was transferred electronically, mostly among financial institutions (known as electronic funds transfer [eft]), whereby funds could be routed electronically from one organisation to another. a significant milestone in online commerce was the launch of the world wide web in the early 1990s, allowing businesses to establish an online presence with text and images. as the internet became commercialised, online commerce/e-commerce rapidly expanded, with numerous dot-com startups emerging. by 1999, online commerce focus shifted from b2c to b2b, then later diversified to include business to employee (b2e), collaborative commerce (c-commerce), e-government, e-learning, and mobile commerce (m-commerce). social networks gained traction in 2005, with social commerce channels like facebook and twitter increasingly driving online commercial activity by 2009. s-commerce (social commerce) is the latest technological advancement linking e-commerce with social media and is growing at a rapid rate. s-commerce, with its enhanced features, diverges from e-commerce in various aspects. the three main characteristics which distinguish s-commerce from e-commerce are social media technologies, community interactions, and commercial activities (liang & turban 2011). the uniqueness of defining boredom in the online commerce context is signification from three perspectives: research, businesses, and consumers. firstly, as social media, e-commerce, s-commerce, and m-commerce, among others, all come under online commerce; a defined concept that can be applied not only within its domain but also in broader multidisciplinary fields. consequently, a good conceptual definition paves the way for future inquiries into boredom and its measurement that would otherwise not be possible (i.e., theory formulation) (wacker 2004). secondly, for businesses, researchers believe that boredom can lead to new paths towards value co-creation online (sundström et al. 2019). for consumers, known for its negative consequences in online contexts (e.g., addiction, uncivil behaviour) (koban et al. 2018), defining boredom aids in understanding and devising ways to assess and mitigate it, promoting greater well-being. before making recommendations, establishing an operational definition is imperative, determining when interventions should be applied. this definition will serve as a foundation for identifying empirically sound, technologically driven tool usage for measuring and mitigating boredom in online settings. hence, a literature gap exists that calls for a clear and concise definition of boredom in the online commerce context. in the next sections, this study presents the review methods, followed by the findings, inductive framework, and the definition of boredom. finally, it discusses the findings in the light of existing boredom theories, provides the limitations of current work, and presents concluding remarks followed by implications. australasian journal of information systems ali, khan & wang 2025, vol 29, research article understanding boredom in online commerce 5 2 methodology the current study’s method utilised the grounded theory technique (mostly used in the information systems (is) field) following the guidelines of wolfswinkel et al. (2013). these guidelines are based on the strauss and corbin approach (1990, 1998) of “rigorously reviewing literature”. the guidelines consist of five steps, as shown in table 1. this directed the review process prior to review presentation. the first four steps are discussed in the following paragraphs, while the fifth is presented in the findings section. step task(s) define • inclusion/exclusion criteria, identify fields of research and sources, establish search terms. search • search for text by selecting search field and keywords. select • refine the sample by applying the inclusion/exclusion criteria and identify additional sources to produce the final sample. analyse • apply coding techniques: open coding, axial coding, and selective coding. present • write the review and present findings via figure. • define the structure of the review itself. table 1. using grounded theory for literature review (adapted from wolfswinkel et al. 2013) according to opie (2004), “grounded theory is a process of collecting qualitative data and undertaking data analysis to generate categories (a theory) to explain a phenomenon of interested”. this theory consists of five stages and is iterative, which is as follows: our study utilised the grounded theory technique following the guidelines of wolfswinkel et al. (2013). based on our inclusion criteria we searched title, abstract, and keywords in two databases (i.e., scopus and web of science) for the literature published between 1965 and 13th august, 2022. we only selected articles published in peer-reviewed journals, conference articles, and book chapters, and excluded work-in-progress papers and the ones that were not in english. the search queries included title-abs-key ( “boredom” or “bored” or “state boredom” or “trait boredom” or “free time boredom” or “leisure boredom”) and titleabs-key (“group buying” or “online shopping motivations” or “internet shopping” or “online buying” or “online purchase” or “electronic shopping” or “e-shopping” or “online purchases” or “online shopping” or “social media” or “social networking sites” or “internet use” or “social media use” or “social networking” or “facebook” or “social media engagement” or “digital media” or “internet” or “social browsing” or “social media platforms” or “social commerce” or “s-commerce”). the initial sample size was 524; we reduced the number to 98 for final analysis. however, drawing on the common citation information within each paper, five more articles were added. the final sample included 103 works. to justify the selection of articles, a “reviewer’s selection table” was created and articles were listed according to year of publication (see table e.1 of appendix e in the supplementary material). we reviewed and analysed the literature based on iterative coding. (following recommendations from wolfswinkel et al. (2013), several pre-defined, descriptive elements of each paper were collected and summarised in figures b.1 to b.4 of appendix b and table f.1 of appendix f “initial coding” in supplementary material.) three types of coding were applied: “open coding”, “axial coding”, and “selective coding” (wolfswinkel et al. 2013), and the analysis was performed using nvivo (release 1.0). we summarise the themes identified from the literature in table 2. australasian journal of information systems ali, khan & wang 2025, vol 29, research article understanding boredom in online commerce 6 3 findings 3.1 defining boredom the conceptual definition of boredom in the online commerce context using the steps of grounded theory (outlined in table 1) comprises five categories that emerged as a result of current analysis (see table 2). boredom is defined here from a functional perspective, as the core theme/category of this study’s analysis is “boredom’s function”. based on the inductive framework (figure 2) and in light of the rules and guidelines of a good conceptual definition presented in table d.1 and d.2 of appendix d (wacker 2004; suddaby 2010), this study defines boredom as follows: boredom is an individual’s perception-based negative affective state and/or trait, having a self-regulatory function, with the aim to seek internal and external stimulation from online commerce to relieve itself irrespective of its consequences (positive or negative) on one’s well-being. 3.2 boredom: an inductive framework high level category and description lower-level category and sample references boredom’s nature (a temporary, emotional, motivational, and aversive state that is based on perception) • negative affective state (wang et al. 2020) • motivational/functional state (watt & vodanovich 1992) boredom’s function (boredom is selfaware and self-regulates from negative to a positive state) • self-aware of the existence of boredom (greenson, 1953) • self regulates negative state to a positive one (biolcati et al. 2017; mano 1999; sundström et al. 2019) causes of boredom (situational and individual factors and their mismatch that results in the experience of boredom) • external factors (factors that lie outside the person such as task and environmental conditions [fisher 1993]) • internal factors (factors that originate inside the person such as individual differences and personality traits [fisher 1993]) • mismatch of internal and external factors (the elements that derive from the fit between person and environment [fisher 1993]) alleviation or coping strategies of boredom (using online commerce as a coping strategy when seeking stimulation) • internal stimulation (individual’s ability to identify and process their own feelings and thoughts and generate interesting activities [harris 2000; vodanovich 2003]) 1. self-stimulation 2. mental stimulation 3. emotional stimulation 4. behavioural stimulation • external stimulation (experiencing interest from daily activities and pleasure from leisure time [vodanovich 2003]) 1. market stimulation 2. environmental stimulation 3. social stimulation consequences of boredom (beneficial and detrimental outcome of boredom for an individual and company) • positive 1. self-development (includes creative thinking, redefining life goals and purposeful behaviour, improving social skills and escaping negative emotions [bench & lench 2013; tilburg & igou 2017; biolcati et al. 2018]). 2. good market performance (boredom triggers impulse buying, frequent customer visits on platform and overall positive consumer attitude towards the companies providing a positive atmosphere [mastro et al., 2002; yazdanparast et al. 2015; sundström et al. 2019]) australasian journal of information systems ali, khan & wang 2025, vol 29, research article understanding boredom in online commerce 7 • negative 1. health problems (problematic behaviours and harms well-being [stockdale & coyne 2020; wegmann et al. 2018; wang et al. 2020]) 2. poor performance of company and individual (customers lose interest as company is unable to provide a good experience [sing, 2016; gerritsen et al. 2014]) table 2. high-level and low-level categories generated during data analysis with sample reference based on the coding techniques and their pattern recognition as underlined in the grounded theory methodology for literature review, table 2 shows the properties of all the selective codes and axial codes of this study as high-level and low-level categories. on the basis of these higher and lower-level categories, figure 1 represents the literature’s sequential diagram and figure 2 represents the inductive framework summarising current knowledge on boredom in online commerce. this framework was built on relationships that emerged through the analysis across five overarching building blocks describing boredom as a negative selfregulatory functional emotion that is caused by situational and individual factors. its selfregulatory function refers to the development of alleviation strategies (i.e., seeking online commerce stimulations) that generate negative and positive outcomes. visual representation of coded themes in nvivo are presented in figure c.1 to c.5 of appendix c. in addition, the complete reference list of our analysis by categories (see table g.1 to g.5 of appendix g), variables/codes definitions (presented in figure 2) (see table h.1 of appendix h), and conceptual foundations of our sample (see table i.1 of appendix i) are available in the supplementary material. individual factors environmental factors boredom causes / antecedents alleviation strategy social commerce usage motivation positive consequences negative individual business individual individual business individual business regulated un-regulated regulated un-regulated generates consumer figure 1: literature sequential diagram note: the arrows represent the flow of the literature review. 8 figure 2: inductive framework note: the arrows do not represent a statistical relationship, or the causality found in variance models. rather, they detail an overarching sequence of relationships described by the literature on boredom in social commerce. the dotted arrow depicts if online commerce fails to regulate boredom and shows that it is also subject to positive and negative consequence. australasian journal of information systems ali, khan & wang 2025, vol 29, research article understanding boredom in online commerce 9 3.2.1 boredom’s nature and function boredom is commonly recognised as a negative motivational emotion, arising from daily activities and life orientation (ksinan et al. 2019; biolcati et al. 2018). its perception varies among individuals (yazdanparast et al. 2015). in consumer behaviour, boredom is often treated as an independent variable, particularly in shopping studies (sundström et al. 2019). it profoundly influences consumer behaviours, inciting consumption responses to counter its negative effect (bozaci 2020; studak & workman 2004). three types of boredom have been highlighted in the context of online commerce: state, leisure, and trait. state boredom is regarded as an actual experience caused by external factors, such as the environment and tasks (fahlman et al. 2013). for instance, boredom is aroused if the online commerce platform is neither gratifying nor attractive; for example, it has dull colours, the multimedia is not appealing, or if its products or content lack variety and novelty (mori 2014). leisure boredom is the consequence of having too much time available and being unable to deal with this time efficiently (spend optimally) or the leisure experiences and activities available are limited and not stimulating, novel, or exciting/pleasurable enough to engage in. online commerce helps to kill leisure time boredom by providing an unlimited number of fun seeking activities, like shopping, interactivity, and information seeking. failure to provide adequately stimulating or captivating leisure opportunities will evoke further boredom (leung 2020). trait boredom or boredom proneness is a personality trait. it is the tendency or susceptibility to being bored easily. people with this psychological trait have high arousal levels and are easily prone to boredom. such individuals are continuously seeking stimulation via social media and online shopping to cope with boredom (sundström et al. 2019). in the current sample, the function of boredom is represented by the following steps: (1) boredom necessitates the individual is self-aware of the feeling. (2) boredom motivates the individual to assess this feeling and find its root cause. (3) once the cause has been discovered, boredom creates a desire to alleviate this cause (which can be achieved by changing one’s feeling towards it and thereby escaping it). (4) changing or escaping means finding alternatives that can transform the negative effect of boredom into a positive one (bozaci 2020; sundström et al. 2019). in the context of online commerce, it is found that individuals use digital technologies, like smartphones, the internet, social media, and online buying, as a coping strategy, reflecting a common behavioural response. 3.2.2 causes of boredom boredom is a result of either internal and external factors or their mismatch. in the current sample, individual (internal) and situational (external) causes of boredom are highlighted in the context of online commerce. 3.2.2.1. external causes boredom is a state that can be caused by external factors like situations, tasks, activities, or conditions, which are as follows: monotony and repetition. boredom emerges from monotonous and repetitive scenarios that result in inattentiveness (sundström et al. 2019; bozaci 2020). repetitive content on social media, or reaching the end of a feed, can trigger boredom (paasonen 2018; chou et al. 2018; australasian journal of information systems ali, khan & wang 2025, vol 29, research article understanding boredom in online commerce 10 barkemeyer et al. 2020). weinstein (2018) noted that continuous scrolling for an hour led to boredom due to the absence of fresh content. a parallel can be drawn in shopping, where monotony, such as lack of variety or novelty in products and environment, prompts boredom (studak & workman 2004). frustration. monotony alone isn't sufficient; it requires frustration to trigger boredom (studak & workman 2004). in online impulse buying, monotony leads to frustration and a craving for change and novelty, which fosters boredom and prompts purchase behaviour (sundström et al. 2019). lack of pleasure. digital technologies are the source of gratifying and entertaining opportunities. in today’s world, dependency on these technologies has increased because they are an easy source of pleasure. when unwillingly separated from their smartphones and other devices, users may feel bored, as they have nothing at hand for entertainment and to give them pleasure (hoffner 2016). in the same manner, if the experience of social networking sites (sns) is not gratifying or pleasurable enough, its users will become bored and will position that sns as boring (yazdanparast 2015; singh 2016). likewise, unappealing web interfaces, void of multimedia, engaging navigation, or vibrant colours, evoke boredom (mori 2014). furthermore, if the tasks at hand on social media are utilitarian and not hedonic (entertaining or exciting) it will not only cause boredom but will make its usage less attractive (whelan et al. 2020; panova & lleras 2016; lin et al. 2019). lack of interest. boredom arises from disinterest and apathy towards one's surroundings. an unstimulating environment fails to provide satisfaction or diversity, thus resulting in boredom and reluctance to engage (studak & workman 2004). an uninteresting sns becomes linked with boredom, curtailing user participation, such as posting, liking, or other social media actions (yazdanparast et al. 2015; singh 2016). dissatisfaction. boredom stems from dissatisfaction, often due to repetitive, monotonous, prolonged environment or misaligned challenges and skills (yazdanparast et al. 2015; singh 2016). a disappointing sns (like facebook) is perceived as boring and influences users' attitudes and assessments negatively (whelan et al. 2020). existential suffering. boredom can result from existential suffering. it is triggered by purposeless activities or routines lacking meaning and hampering engagement. mindless smartphone use or aimless social media browsing fosters boredom (weinstein 2018; ragheb & merydith 2001; allaby & shannon 2020). similarly, purposeless content can also cause boredom. in an experimental study using an eye tracker, li et al. (2009) found that a visual search online causes boredom owing to floating ads. these ads are commercially driven and often deemed irrelevant and disregarded. loneliness/disconnection. situational loneliness ignites boredom (diefenbach & borrmann 2019). for instance, abstaining from digital communication tools can cause an emotional void, generating feelings of alienation, disconnection, and fomo (fear of missing out), leading to boredom (catedrilla et al. 2020). when isolated from the internet, social media users felt bored, as they found themselves outcasted and disconnected from their friends and the world (stieger & lewetz 2018; hoffner et al. 2016). similarly, quarantine restrictions on outdoor activities caused filipinos to become boredom and consequently triggered social media usage (catedrilla et al. 2020). australasian journal of information systems ali, khan & wang 2025, vol 29, research article understanding boredom in online commerce 11 too much free time. having too much free time and limited or no leisure opportunities can create a sense of boredom. for example, individuals may have more free time in the evenings but lack of meaningful activity to fill their time makes them bored (sundström et al. 2019). disengagement. boredom signifies disengagement, where individuals are unable “to engage in a satisfying or stimulating activity” (fahlman et al. 2013 p. 69) or “articulate a satisfying target for engagement” (eastwood et al. 2012, p. 483; gerritsen et al. 2014). it is a condition in which individuals are motivated but fail to engage with an activity or the world around them (mugon et al. 2018). this often arises due to unengaging, unappealing, or repetitive experiences, lacking psychological involvement (biolcati et al. 2018). for instance, lack of stimulation on facebook correlated with decreased engagement, eventually linking boredom to the platform (yazdanparast et al. 2015). contextualised totality. boredom’s interplay with a contextualised totality shapes consumer feelings and gives a sense of meaninglessness. the use of social media and online impulse buying when feeling bored often occurs in the evening or night when an individual is alone, lacks energy, and is using multiple screens at the same time (e.g., laptop, tv, phone) (sundström et al. 2019). 3.2.2.2. internal causes some individuals have certain characteristics that make them more susceptible to boredom. in the present sample, these characteristics are divided into individual and demographic categories. individual characteristics. bored individuals are found to be lazy, have no dedication, or consistency, and are impatient towards an aversive environment (e.g., an environment that provides overor under-stimulation or involves waiting periods or delays such as slow internet speed) (farmer & sundberg 1986). these individuals lack autonomy, have a poor selfimage, and are introverts, narcissists, and sensation seekers with high arousal levels (ksinan et al. 2019). in terms of skill and capability, they are poor time managers, lack good interpersonal skills, and have no internal stimulation, which results in being easily bored (bozaci 2020; ksinan et al. 2019). from a psychological perspective, these individuals have noticeable cognitive deficits, such as being easily distracted and generally unable to focus or pay attention, and/or have mental disorders like adhd or brain damage (holte & ferraro 2020; whelan et al. 2020). from an emotional perspective, they are moody shoppers, passive interactors (usually browsing online with an unconscious mind), emotionally unstable, and often unable to express their emotions or even to be aware of them (tzavela et al. 2017; holte & ferraro 2020). demographic level. at the demographic level, boredom varies between culture, gender, and age. boredom that leads to screen activities is most apparent in asian collectivistic cultures (lin et al., 2019). boredom is commonly experienced among youth irrespective of gender (lin & yu 2008; biolcati et al. 2018; yazdanparast et al. 2015; sundström et al. 2019; bozaci 2020). australasian journal of information systems ali, khan & wang 2025, vol 29, research article understanding boredom in online commerce 12 3.2.2.3. a mismatch between the individual and the environment boredom is a result of a mismatch or discrepancy between an individual’s perception, need, or demand and the level of stimulation available in the environment. it is a state of nonoptimal arousal or stimulation. similarly, it also occurs when there is a mismatch between a challenge and skill, like the mismatch of available attentional resources and task demands (kiss et al. 2020; bozaci 2020). 3.2.3 alleviating boredom through stimulation boredom attempts to alleviate itself through its self-regulatory function. a bored individual is hungry for stimulation (fahlman et al. 2013) and will often engage in novel and energising experiences even if they are negative ones (wilson et al. 2014). for that, online platforms can reliably help manage negative moods, as it provides different types of stimulation (diefenbach & borrmann 2019). in the present sample, stimulations to reduce boredom are classified into internal and external categories in the context of online commerce. 3.2.3.1 internal stimulation internal stimulation refers to “consumers’ self-feelings, moods, and emotional states” (kuei huang & yang 2008, p. 100). online commerce provides it in the form of self, mental, emotional, and behaviour. self-stimulation. self-stimulation from practising autonomy and satisfying the need for belongingness can evade boredom. for instance, having the authority to leave a boring shopping experience or demonstrating social identity online through new clothes can relieve the meaninglessness characteristic of boredom (studak & workman 2004; sundström et al. 2019). mental stimulation. bored people seek out mentally stimulating tasks rather than mentally exhausting ones to occupy themselves (elhai et al. 2018). bored consumers can be mentally stimulated by online shopping, especially in terms of fashion apparel (a multi-sensory image product). the variety of newly arrived fashion goods can grab consumers’ attention, and becoming involved in details and information about products during the shopping process can create a mental incentive to learn about and engage with these products. online shopping is also a good leisure activity that can lead to mental stimulation and escape from real-life problems (studak & workman 2004). in addition, the diversionary nature of digital technology tools is a distraction from boring tasks and protects individuals from social isolation (sundström et al. 2019). during covid-19 lockdowns, online shopping via digital tools allowed people to feel as if they had escaped from their house, which at that time felt like being in a prison. due to the limited activities allowed at that time, individuals’ willingness to consume online increased and they often acted impulsively (deng et al. 2020; güngördü belbağ 2022). emotional stimulation. when performing utilitarian tasks, individuals may experience a cognitive pull to engage in something more emotionally stimulating, and digital technologies (i.e., social media etc.) seem to provide hedonic benefits at that time (whelan et al. 2020). for bored consumers, shopping on online platforms is like a war between boredom and pleasure. they shop online to relax and improve their emotions by seeking gratification. just filling the cart with the right product is stimulating for them and they feel this time spent is meaningful australasian journal of information systems ali, khan & wang 2025, vol 29, research article understanding boredom in online commerce 13 (leung 2020). so, they search for new products to continue this sense of pleasure. also, to them, the price may seem irrelevant, and they are ready to pay more to overcome their sense of boredom (bozaci 2020). behavioural stimulation. the current sample reported that behavioural stimulation, like rewarding oneself through buying clothes online and disposing of old ones, can ease boredom. participants in a study by sundström et al. (2019) expressed this feeling as, “i was bored so it was worth the money” (p. 153). similarly, participating online by creating usergenerated content, liking, and posting can attract attention and alleviate boredom (bozaci 2020; ksinan et al. 2019). 3.2.3.2 external stimulation external stimulation refers to stimuli from the environment. three types of external stimulation were mentioned in the present sample as being able to alleviate boredom in the online commerce context: market, environment, and social. market stimulation. market stimulation in online commerce involves, among others, price, product, ease of access, and ads. bored consumers react to these market stimuli to ease their boredom. a variety of low-cost products and promotional activities act on scarcity with headings such as “available now” or “discount today” and offers such as “free shipping” and “return”, which stimulate bored individuals to seize the moment and make an impulse purchase online (bozaci 2020; harish et al. 2020). similarly, the visual stimuli of products on social media strongly appeal to consumers and they are willing to visit the website of a product just by seeing how good it looks in the visual of an online advertisement (sundström et al. 2019). environmental stimulation. bored people favour a stimulating environment to improve their emotions. rather than setting out with the aim of buying a particular product or service, it is the attractiveness of an online store or website that triggers browsing and eventually results in impulsive buying (sundström et al. 2019; harish et al. 2020). social stimulation. online commerce is also a source of social stimulation for bored individuals. s-commerce makes it easier to reach out to friends and family. it offers opportunities to build new connections and interpersonal interactions (diefenbach & borrmann 2019). bored people usually prefer not to be alone and engage in highly interactive sessions online for social stimulation and to relieve boredom (allaby & shannon 2020). 3.2.4 consequences of boredom findings indicate that boredom has both negative and positive outcomes. 3.2.4.1. negative consequences boredom is associated with various negative consequences whether regulated or unregulated. this is why it is conceptualised as a negative affective state. health problems. boredom is associated with various health issues related to well-being and behaviours that are unproductive, uncivil, and addictive. if boredom is not regulated or alleviated constructively (e.g., for a utilitarian and social purpose), it can damage an individual’s psychological well-being by producing existential australasian journal of information systems ali, khan & wang 2025, vol 29, research article understanding boredom in online commerce 14 problems (i.e., lack of a motive or sense of direction). similarly, alleviating boredom online can have a particularly adverse emotional impact on youth (tzavela, et al. 2017). coping boredom through online commerce can enhance feelings of strain, stress, aggression, dissatisfaction, depression and anxiety, conspiracy beliefs, paranoia, frustration (boring shopping experience), awkwardness, emptiness and/or sadness, fomo, isolation, and loneliness (biolcati et al. 2018), which harms hedonic well-being. experiencing negative effects is one thing, while the adverse behaviour that follows is another. from a behavioural perspective, boredom is responsible for various unproductive behaviours, such as less engagement in hobbies (biolcati et al. 2018) and mirroring uncivil responses or comments online (as aggressive and harsh online discussions may seem entertaining) (koban et al. 2018). additionally, bored people report symptoms of adhd (a behavioural disorder) (chou et al. 2018). moreover, boredom can lead to reduced self-control, prompting impulsive decisions like excessive buying. this behaviour often results in overspending and subsequent feelings of regret (baumeister 2002; sundström et al. 2019). in addition to these behaviours, boredom causes smartphone, social media, and internet addiction. a boredom cycle highlighted in current sample literature shows that using social media to alleviate boredom can lead to more boredom. alleviating this cycle of boredom online increases the probability of developing online addiction. for instance, engaging in online content too often can result in it becoming homogenous; content that may appear novel or new to start with will eventually become repetitive and dull, once more leading to boredom. also, continuously engaging in unconstructive activities online (wasting time for hedonic benefits) can make it harder to feel stimulated and lead to further boredom and consequently further consumption (paasonen 2018). over time, these addictions can lead to the development of anxiety, depression, and stress. at a company level, if boredom associated with a company’s products, services, or even their image is not regulated, the company will face various problems which can cost them their customers (wang et al. 2020; whelan et al. 2020). a boring sns offering nothing interesting will negatively impact its users’ attitude and assessment towards it and eventually reduce their participation (i.e., posting or carrying out other social media behaviours) (yazdanparast et al. 2015; biolcati et al. 2018; paasonen 2018). similarly, users failing to seek gratification become passive. their consumption is absentminded, resulting in low engagement and interest in a platform. moreover, constantly seeking but not getting adequate stimulation online can lead to social media fatigue, making its users disuse the platform permanently (barkemeyer et al. 2020). 3.2.4.2. positive consequences in the present sample, two positive consequences of boredom emerged. one is selfdevelopment from the perspective of an individual and the other is good market performance from the marketer’s perspective. self-development. boredom, despite its negative nature, yields positive impacts on selfdevelopment. it enhances interpersonal skills by encouraging greater online participation. bored consumers update their profiles, share their point of view on forums, and interact with their friends via the social and user-generated content features of social media. doing so not australasian journal of information systems ali, khan & wang 2025, vol 29, research article understanding boredom in online commerce 15 only alleviates boredom but also boosts their confidence and social skills (leung, 2020). boredom alleviates negative emotions like stress and motivates enjoyable activities such as relaxing screen time (biolcati et al. 2018). additionally, boredom leading to impulse buying of the right product contributes to subjective well-being (sundströma et al. 2019). market performance. it is a well-known fact that bored people engage in online shopping activities in search of pleasure, distraction, and engagement. this provides a good opportunity for marketers to appeal to consumers by providing, among other things, an attractive store environment, good service, ease of use, and a variety of goods. marketers are able to stimulate a “must-have” craving for online shopping and influence consumers’ intentions to proceed to an impulse purchase. eventually, these consumers become frequent visitors and develop a positive attitude towards a site (sundström et al. 2019). 4 discussions this study rigorously investigated boredom in the context of online commerce using grounded theory methodology. this approach effectively identified and defined boredom within this context, uncovering new linkages among variables (indicated in figure 2). findings addressed research questions by highlighting boredom's function, antecedents, alleviation strategies, and consequences. moreover, it offers future research directions for interested scholars. 4.1 conceptual definition of boredom in the social commerce context this refers to rq1. boredom, in the current study, is defined as an individual’s perceptionbased negative affective state and/or trait, having a self-regulatory function, with the aim to seek internal and external stimulation from online commerce to relieve itself irrespective of its consequences (positive or negative) on one’s well-being. 4.2 boredom is a functional emotion the study’s findings indicate that boredom is a negative affective state that is commonly experienced in our day-to-day activities. this experience is mundane and is based on an individual’s perception. as hill and perkins (1985) pointed out in their model of boredom, the same stimuli can be inferred differently by individuals leading to different experiences and outcomes. the three types of boredom discussed in the online commerce context are trait boredom, state boredom, and leisure or free-time boredom, each of which influences consumer behaviour. negative states have more influence on individuals than positive ones. for instance, negative moods foster in-depth information processing, impactful impressions, and a quest for meaning in adverse experiences (baumeister et al. 2001). as a negative emotion, boredom wields influence over decision-making and preferences, driven by its self-regulatory motivational function. this finding of the boredom function is in agreement with the statement that “boredom is both an informative and motivational state: it signals that we need to change something about ourselves or our environment and it motivates us to do so” (elpidorou 2018, p. 9). this change is not just a random physical, environmental, or mental alteration but also leads to something positive like satisfaction, meaningfulness, and pleasure. australasian journal of information systems ali, khan & wang 2025, vol 29, research article understanding boredom in online commerce 16 given this functional nature of boredom, it is important to know what causes it, how to alleviate it, and what its aftermath is. in terms of its origin in the online commerce context, this study highlights the possible situations and individual characteristics more prone to it by underlining the internal and external causes of boredom in different age groups and gender. 4.2.1 boredom’s antecedents in social commerce: individual and situational factors the findings propose that bored consumers’ motivation for online commerce is influenced by environmental and individual factors and their mismatch. there are four theories of boredom and each help to explain current findings concerning what exactly causes it in the online commerce context. hence, they are the answer to this study’s rq2 (a). according to psychodynamic theorists mann and cadman (2014, p. 165), “boredom stems from a situation where none of the possible things that a person can realistically do appeal to the person in question”. current findings underline such experiences as unpleasurable, frustrating, and dissatisfying. unappealing and homogenous activities or an environment lacking hedonic value fails to grab consumers’ attention and causes disengagement while also creating “a longing to engage in an unspecified satisfying activity”, which is a key predictor of boredom (baratta & spence 2018, p. 478). the existential theory of boredom defines it as “the aversive experience of inaction, emptiness, paralysis of will, and meaning not realized” (eastwood et al. 2012, p. 484). therefore, experiences that feel meaningless cause boredom. according to attentional/cognitive theory, “boredom is caused by a failure of attentional processes resulting in an inability to focus or engage attention” (fahlman et al. 2013, p. 69). affirming this, present findings indicate that if experience lacks novelty, variety, or innovations, it will become monotonous and uninteresting over time and fail to easily capture consumers’ attention. current findings indicate that boredom occurs in times of loneliness, disconnection (not being online), and when there is a large amount of time on hand but there are no or limited opportunities for engagement that can provide optimal arousal. being disconnected and alone during the covid-19 lockdowns meant people had a lot of free time and not much to do and they consequently experienced boredom more than ever. they were under obligatory restrictions to stay at home with no physical or social contact with friends and family and instead relied on digital technologies to a risky level (deng et al. 2020). the theory of free time boredom states that an individual experiences boredom because of a lack of surrounding stimulation. this does not mean that individuals lacked leisure opportunities but rather the experiences available to them were not engaging, fun, appealing, or novel. this lack of psychological involvement caused boredom (biolcati et al. 2018). internal causes of boredom include individual characteristics that make an individual more prone to boredom compared to others. firstly, as stated by vodanovich (2003), highly boredom-prone individuals have a lower intention to engage in civic activities and are mostly inactive. present findings show that bored individuals are less motivated by life and are instead drawn to smartphones and social media, as these provide stimulation with little effort. such individuals have low tolerance, low self-image and autonomy but have high arousal levels and are sensation seekers. they are usually introverts and narcissists. these claims are australasian journal of information systems ali, khan & wang 2025, vol 29, research article understanding boredom in online commerce 17 supported by several studies (leong & schneller 1993; tolor 1989; watt & vodanovich 1999; emmons 1981; vodanovich 2003). secondly, according to psychodynamic theory, boredom results from repressed desires. in a state of boredom, an individual desires an alternative but is not sure what to look for (eastwood et al. 2012). such individuals lack the necessary skills and capabilities (time management, technical and interpersonal skills, internal stimulation) to seek activities with adequate stimulation to overcome their boredom. thirdly, these individuals have certain cognitive and affective defects which increase their tendency to be bored. from the cognitive perspective, the attentional theory of boredom states that boredom is caused by a failure to focus and engage. present findings report that such individuals may have mental disorders like adhd and brain injuries. from an affective perspective, the findings report such individuals as being emotionally unstable and unaware of their feelings. psychodynamic theory states that “uncertainty and vagueness about one’s own wishes … characterizes boredom”. bored individuals “desire alternatives but are unable to find and looks to the world for possible solutions” (lewinsky 1943, p. 148; baratta & spence 2018). boredom spans across cultures at the demographic level, yet within the online commerce framework, individuals from asian collectivistic cultures tend to report it more. this aligns with the conclusion of cross-cultural boredom studies (sundberg et al 1991; ng et al 2015). this is because asian cultures generally have less challenging and interesting opportunities to engage in, and due to their internalised cultural values, individuals from asian cultures have positive low arousal compared to western cultures that promote fun and excitement (ng et al. 2015). moreover, experiencing boredom is part of youth culture irrespective of gender. young people heavily rely on technology for information, social interaction, and shopping. they exhibit sensation-seeking behaviours, struggle with time management, lack knowledge of constructive boredom remedies, and possess newfound autonomy, often with less monitoring. consequently, they encounter greater boredom compared to older generations (biolcati et al. 2018; stockdale & coyne 2020). lastly, the third source of boredom is the discrepancy between internal and external stimulation. this is supported by the arousal theory of boredom that proposes that “boredom is caused by non-optimal arousal that ensues when there is a mismatch between an individual’s need for arousal and the availability of environmental stimulation (i.e., its degree of challenge, complexity, intensity, and variety)” (fahlman et al. 2013, p. 69). in a situation of overstimulation, consumers’ ability to be surprised or amazed is taken away by the excessive stimulation, and, eventually, they disengage and become bored (gerritsen et al. 2014). in the case of under-arousal or under-stimulation, the environment or activity is inadequately mentally or emotionally stimulating. similarly, a mismatch between challenge and skill causes boredom. according to the 4-channel model of flow, if the task or activity is easy, the result can be boredom (guo & klein 2007). hence, a moderate level of arousal or stimulation and challenge is preferred otherwise the result can be disengagement and inattention which triggers boredom. australasian journal of information systems ali, khan & wang 2025, vol 29, research article understanding boredom in online commerce 18 to answer rq2 (b), this study points out the alleviation strategies that this functional emotion seeks out in the online commerce context, which are internal and external stimulation via online commerce. 4.2.2 boredom’s alleviation strategies in social commerce: boredom triggers seeking stimulation the factors stated above trigger the experience of boredom in online commerce, and owing to its self-regulatory function, bored individuals seek alternatives. bored users turn to digital technologies, as they are swift, convenient, and readily available. these technologies offer both internal and external stimulation, effectively countering negative states with minimal effort. the current study proposes that bored consumers’ motivation to go online is to seek internal and external stimulation to relieve themselves from boredom. on average, a bored individual uses a smartphone 34 times a day to access the internet and social media, and this response is habitual. boredom and shopping are linked together because it feels like clicking the boredom away and escaping reality and stress, especially in the case of fashion products (sundström et al. 2019; bozaci 2020; harish et al. 2020). online commerce provides internal stimulation emotionally, mentally, behaviourally, and through self by providing opportunities for interactivity and community building, and giving consumers authority over the system, their digital profile, reference groups, gratification, and browsing information. through its collaborative and entertaining features, it engages users to achieve a satisfying internal state (whelan et al. 2020). external stimulation includes the market, environment, and social stimulation. market cues like price, product diversity, the visual stimuli of products and pop-ups, free shipping, and delivery makes the product striking and gives bored customers a feeling of benefit or value, which induces them to buy on impulse. similarly, a pleasant, engaging atmosphere and a friendly, high-quality service not only help consumers in making shopping decisions but also help them manage their negative moods by triggering online impulse buying. online commerce provides such stimulations owing to its technical and commercial characteristics like coupons, discounts, personalisation, and availability of global and local commodities, consumers’ bargaining power, a personal recommendation system, and a hedonic environment that is arousing and enjoyable. lastly, s-commerce characteristics like ratings, reviews, and recommendations from vendors and customers in real-time provide social stimulation (stephen & toubia 2010). through its collaborative environment, consumers can interact during the preand post-purchase phases in real-time which lowers their loneliness and eventually eases boredom. even though these alleviation strategies can ease boredom, they are not without consequences. to answer rq2 (c), regarding the aftermath of boredom in the online commerce context, current findings indicate that it has both positive and negative outcomes. boredom can be detrimental to well-being and productivity (individual and organisational) but can also lead to self-development and be an opportunity for good market performance. australasian journal of information systems ali, khan & wang 2025, vol 29, research article understanding boredom in online commerce 19 4.2.3 consequences of boredom in the social commerce context 4.2.3.1. boredom is detrimental to health (well-being) and productivity numerous studies have found that this aversive emotion can lead to many negative outcomes. this study proposes that seeking non-optimal/optimal but unconstructive stimulation on online commerce is detrimental to an individual’s health and performance as well as the company’s performance. in terms of health, boredom harms an individual’s hedonic and eudaimonic well-being. “hedonic well-being comprises of high positive affect and low negative affect” (carruthers & hood 2004, p. 229). in contrast, boredom is linked to many negative effects, such as depression, anxiety, stress, fomo, mind wandering, and dissatisfaction. in the online commerce context, owing to the irresistibility of the stimuli, a bored consumer buys on impulse; however, he/she may develop a negative feeling of regret for making an impulsive decision (baumeister 2002). boredom leading to such negative effects can be harmful for hedonic well-being. similarly, eudaimonic well-being is “having a sense of control over one’s fate, a feeling of purpose and belongingness, and a basic satisfaction with oneself and one’s existence” (cowen 1991, p. 404). boredom in the online commerce context creates a feeling of meaninglessness based on the need for belongingness and contributes to poor self-control when seeking to relieve itself through digital technologies. it also triggers impulsive behaviours, especially in online shopping that can lead to overspending and overconsumption. a study with a similar finding reported that due to limited activities during the covid-19 pandemic, bored people hunted for complex external stimuli (i.e., shopping) because their impulse-buying tendencies increased to seek satisfaction in that state (deng et al. 2020). all these feelings and behaviours are harmful to an individual’s psychological well-being as they can hinder their personal growth and development. this research highlights how using digital tech to counter boredom can lead to addictive and problematic behaviour. this aligns with the findings of wang et al. (2020), who indicated that excessive internet use is a coping mechanism for negative emotions. moreover, in the long term, such addictive behaviours can lead to other negative states like depression, stress, anxiety, aggression, frustration, and fomo. during the covid-19 pandemic, heightened social media usage due to boredom led to addiction (stockdale & coyne, 2020). boredominduced addiction often arises when bored individuals lack parental monitoring, have nothing to do in their free time, lack stimulation and meaningfulness in their daily activities, and have lower self-control over digital technology usage. at a business level, unregulated boredom associated with a company's products, services, or platform can tarnish its reputation and stakeholder relationships due to negative customer attitudes. as highlighted in this review, boredom arises when online platforms fail to deliver optimal gratification, resulting in reduced consumer interest, visits, and overall engagement. 4.2.3.2. the bright side of boredom this study proposes that seeking non-optimal/optimal stimulation on online sites will be fruitful to an individual’s health, well-being, and performance as well as enhancing the company’s performance. boredom as a negative emotion that leads to negative consequences australasian journal of information systems ali, khan & wang 2025, vol 29, research article understanding boredom in online commerce 20 has been studied and reported in numerous research (whelan et al. 2020; elhai et al. 2018; bozaci 2020; yazdanparast et al. 2015). the findings highlight the positive consequences of boredom – an aspect that has had little investigation in boredom research. boredom, despite its negative nature, holds protective aspects as it alerts individuals to monotonous, unsatisfactory situations, prompting action. it fosters skill development and self-confidence, contributing to personal growth. its impact on subjective well-being varies, with potential adverse or beneficial outcomes. while utilising online commerce during boredom might lead to negative emotions, like regret, it can also elicit happiness. consumers often report pleasure or happiness during and after their purchase experience. shopping can therefore be a happy experience (sundströma et al. 2019). furthermore, bored individuals often turn to online platforms to elevate their mood, seeking engaging content to captivate them. this presents a lucrative opportunity for marketers to cater to the stimulation needs of these consumers. as stated in the arousal theory of boredom, resolving the discrepancy of the individual’s demand and environmental supply of stimulation can evade boredom. user and gratification theory also claims that an “audience select media and media content to satisfy their social and psychological needs” (leung, 2020 p. 131) and “benefits that accrue from using new media, such as information, identity, social integration and interaction, and entertainment” (tomczyk & solecki 2019, p. 3). hence, marketers should prioritise creating an engaging and enjoyable experience for bored consumers because a platform environment greatly impacts purchasing behaviour and preferences. 5 future directions our review highlights the significant contributions that research has made towards our understanding of boredom in online commerce context. in this section we will extend these contributions and provide avenues for future is research on boredom followed by a table (see table 3) that offers a concise representation of the discussion featuring the outlined future research questions. 5.1 boredom in market and business perspectives the existing literature on boredom in this context predominantly centres around its influence on consumer behaviour, leaving its impact on the market/business relatively underexplored. gaining insights into its adverse effects on market performance can empower marketers to comprehend the extent of its repercussions and formulate strategic approaches to mitigate and manipulate its effects for their advantage. future studies can concentrate on exploring how boredom impacts the online business. they can investigate alleviation strategies, such as market, social, and environmental stimulation, which could prove valuable to businesses in terms of improving attitudes, increasing engagement, boosting purchasing behaviour, and enhancing platform visits of their bored customers. additionally, researchers can examine the potential drawbacks if marketers fail to address customer boredom. these insights can be invaluable for marketers in designing their platforms, crafting content, and devising marketing campaigns and strategies aimed at customer retention. australasian journal of information systems ali, khan & wang 2025, vol 29, research article understanding boredom in online commerce 21 5.2 explore the roots: antecedents of boredom in the digital technology context from a consumer perspective, we propose several key areas for future research. initially, our review pinpointed the typology of boredom, highlighting that specific traits contribute to its susceptibility, except sensation seeking and narcissism, others (introversion, self-esteem etc) remain relatively understudied. we advocate for further exploration of these traits in conjunction with moderators such as age, culture, and gender, as boredom is perceived and experienced differently among these variables. moreover, the examination of boredom in terms of consumers’ personal skills (social, time, and self-stimulating skills) remains limited. while the cognitive and affective issues (mental disorders, emotional instability etc.) have been studied mostly in psychological contexts, there is a notable gap in exploring these aspects within the context of digital technology. this holistic approach has the potential to yield valuable insights into the nuanced dynamics of this phenomenon, thereby aiding in policymaking for consumer well-being. it's important to note that consumer well-being is also significant for online businesses. for example, due to information and communication overload (whelan et al. 2020), consumers may permanently disengage from a platform, resulting in reduced or discontinued visits, decreased participation, and, ultimately, diminished sales. 5.3 connect alleviation strategies with consequences our review identified the types of stimulation (internal and external) that bored individuals actively seek on online platforms. although studies on this aspect are burgeoning, there is a need for a more comprehensive understanding. for instance, understanding the link between boredom and cyberbullying involves identifying which stimulation (entertainment, social interaction, self-esteem etc.) triggers this relationship. similarly, investigating how boredom influences online purchases requires examining which market stimuli (discounts, product diversity, customisation etc.) strengthen this connection. we recommend that future researchers explore which type of boredom (state, trait, or leisure) experience seeks specific stimulation on online commerce, leading to various consequences. this investigation will help pinpoint which stimulation results in specific outcomes, thereby aiding policymakers in crafting policies and interventions better suited for individuals’ self-development and business prospects. 5.4 analyse problematic behaviour and unveil the positive dimensions of boredom boredom is a well-acknowledged negative emotion. our review emphasises that existing research has predominantly focused on online addiction, leaving other problematic behaviours, such as online uncivil behaviour and cyberbullying, largely unexamined. given the global increase in these behaviours (howarth 2023), there is a valuable opportunity for future studies to investigate and shed light on them. additionally, our review uncovers some positive consequences of boredom (e.g., consumer well-being, business sales) that remain highly underexplored. future research endeavours can delve into these positive aspects to understand how boredom can be constructively alleviated for individuals’ self-development and for good business performance. this can help in making policies and intervention strategies for consumers (well-being) and businesses (market performance). australasian journal of information systems ali, khan & wang 2025, vol 29, research article understanding boredom in online commerce 22 this study presents an inductive framework, urging future research to explore its intricacies and directional relationships. these insights can be extended to emerging digital domains and interdisciplinary fields, like the meta-verse and live streaming, offering valuable contributions to evolving research landscapes. this review can be discussed from the perspective of boredom theories, and future research can enhance this review by exploring the framework through different theories, such as user and gratification, and various internet theories. lastly, future researchers can consider the established definition of boredom in this study as foundational for theory development. they can use this conceptualisation as the “building blocks” for advancing theoretical frameworks related to boredom in the dynamic landscape of online commerce and related disciplines. the above-presented research avenues highlight the key areas where research on boredom in online commerce is currently lacking. in our view, they represent exciting opportunities for is researchers to engage with topics that are interesting from a research standpoint as well as highly relevant in light of current debates on the phenomenon. these future research avenues can contribute to a better understanding of the theoretical linkages of boredom in online commerce. no future questions 1 how does boredom influence market/business performance in the context of online commerce? how can marketers mitigate and leverage the effects of boredom to their advantage? 2 what characteristics of online platforms (external causes/factors) provoke susceptibility to boredom among consumers, and what are the implications for businesses? 3 how and why do specific consumer personality traits (introversion and low self-esteem) trigger boredom in online commerce? how do skills (social, time), age, culture, and gender serve as potential moderators in understanding the interplay of these traits and boredom susceptibility? 4 how does a focused investigation into a specific type of boredom (e.g., state, trait, or leisure) within the context of online commerce contribute to a deeper understanding of its dynamics and implications for consumers? 5 how do various stimuli affect the outcomes for individuals experiencing boredom in the context of online commerce? are these consequences positive or negative for a bored individual? 6 how does boredom contribute to problematic behaviours, like uncivil behaviour and cyberbullying, in the context of online commerce? what are the implications for brands, and what interventions can be implemented to address this issue? 7 how does boredom contribute to consumers’ well-being in online commerce context? table 3. research questions for future studies 6 limitations boredom is a trending topic in many disciplines, and it may be possible that some of the avenues explored in this study are already being researched; however, this possibility would be considered a verification of the current study’s findings. the approach regarded most suitable for this research was to choose breadth over depth; therefore, future studies could examine the relationships identified in the present findings in more depth. australasian journal of information systems ali, khan & wang 2025, vol 29, research article understanding boredom in online commerce 23 7 implications 7.1 theoretical implications firstly, owing to the growing market and consumer usage, it is timely and significant to analyse consumers’ motivation for online commerce. this study undertook a review that integrates current knowledge on boredom with the online commerce context – a neglected gap in the literature. contrary to present studies, this study analysed boredom literature in a digital technology context relevant to online commerce usage using the grounded theory approach. using grounded theory, we explored boredom in online commerce thoroughly and theoretically and presented the strengths, limitations, and future directions for a balanced growth of literature. the task of reviewing a portion of academic literature is important in every academic community but infrequently executed. grounded theory helped retain transparency and offer clarity on how and why the particular literature sample was selected. this theory for the boredom review process not only enriched the value of this review but made it more valuable to the field as well as more replicable. it helped in explaining the fruitful new linkages among variables, as seen in the inductive framework (see figures 1 and 2). this inductive framework is developed to identify boredom’s functional mechanism in online commerce and highlight the emergence of potential antecedents, mediators, moderators, and consequences for both consumer and market perspectives, serving both as a comprehensive reference point and offering prospective avenues for further scholarly exploration. it highlighted possible situations and introduced a nuanced boredom typology specific to the online environment. by bridging gaps in prior studies (chou et al. 2018; elhai et al. 2018; jiang & li 2018; whelan et al. 2020), it provides insights into the psychological traits or personality characteristics that contribute to behaviours impacting health and well-being. previously, studies focused on the intensity of digital technology use in the relationship with boredom (saleem & iglesias 2019; whelan et al. 2020). this study goes further by delineating the specific content consumed and the type of stimulation individuals seek to ease boredom. boredom, being a negative emotion, leads to negative outcomes, like addiction. our review highlighted additional consequences that were ignored in the past studies (cyberbullying, uncivil behaviour) but hold reasonable weight to well-being. moreover, in contrast to previous boredom studies, this research illuminates its positive side for both individuals and businesses in short and long-term scenarios. finally, drawing from the findings of the grounded theory method, we introduced a conceptual definition of boredom in the online commerce context. this clarification of the construct provides insights that may pave the way for a comprehensive development of the boredom theory in the future. the study's sample encompasses multiple disciplines, and this definition holds generalisability in related fields as well. 7.2 practical implications the findings not only elucidate boredom's risk factors but also propose implications for intervention development. this study underscores how boredom can pose risks to health, well-being, and performance, while its intervention holds potential for personal growth (especially among vulnerable youth) and improved business performance. australasian journal of information systems ali, khan & wang 2025, vol 29, research article understanding boredom in online commerce 24 firstly, the findings show that youth are especially prone to boredom, highlighting the need for leisure and media literacy education to help manage it. while digital technology might appear appealing in moments of under-stimulation, the chosen activity's significance holds more weight. so, healthcare providers, educators, and parents should pursue training initiatives that enhance young people's self-control and temper their responsiveness to technological stimuli. secondly, given boredom's link to sensation seeking and its potential to fuel excessive online media use, parents and educators must oversee children's online usage. children should be guided in setting goals and priorities for their free time and honing skills like decisionmaking, technical know-how, time management, and mindfulness to evade boredom. encouraging interpersonal skills and engagement in social activities can divert their leisure time away from digital media. thirdly research findings also present businesses with an opportunity to identify customer boredom and offer solutions within online commerce. this insight can help marketers understand boredom’s impact and overcome sales barriers by addressing its situational and individual causes. marketers should have an attractive outlook on their websites and apps. they should be visually appealing and gratifying, as this can attract bored consumers to engage and participate. they should also avoid complex and purposeless marketing stimuli to protect their goodwill and instead provide novel and diverse content and products to attract bored individuals. fourthly, marketers should give control to their users regarding incoming information and provide them with content according to their priorities. this could help lessen users’ exposure to social media overload and fatigue. 8 conclusion this study aimed to conduct a rigorous literature review using grounded theory to examine boredom in the online commerce context. this review highlighted a rich body of literature that contributes to forming a definition of boredom in online commerce and understanding its nature, function, characteristics, potential antecedents, and alleviation strategies and their aftermath. this study proposes future research avenues and provides theoretical and practical contributions that enable researchers and policymakers to understand boredom and how to intervene for the betterment of an individual’s well-being and business performance. overall, this review is helpful for future research that wishes to further explore the nature and the implications of this mundane emotion for the betterment of individuals, organisations, and society. disclosure statement the authors report there are no competing interests to declare. data availability statement the authors confirm that the data supporting the findings of this study are available within the article and in supplementary materials. australasian journal of information systems ali, khan & wang 2025, vol 29, research article understanding boredom in online commerce 25 references allaby, m., & shannon, c. s. (2020). “i just want to keep in touch”: adolescents’ experiences with leisure-related smartphone use. journal of leisure research, 51(3), 245-263. doi.org/10.1080/00222216.2019.1672506 baratta, p. l., & spence, j. r. (2018). capturing the noonday demon: development and validation of the state boredom inventory. european journal of work and organizational psychology, 27(4), 477-492. doi.org:/10.1080/1359432x.2018.1481830 barkemeyer, r., faugère, c., gergaud, o., & preuss, l. (2020). media attention to large-scale corporate scandals: hype and boredom in the age of social media. journal of business research, 109, 385-398. doi.org:/10.1016/j.jbusres.2019.12.011 baumeister, r. f., bratslavsky, e., finkenauer, c., & vohs, k. d. (2001). bad is stronger than good. review of general psychology, 5(4), 323-370. doi.org:/10.1037/1089-2680.5.4.323 baumeister, r. f. (2002). yielding to temptation: self-control failure, impulsive purchasing, and consumer behavior. journal of consumer research, 28(4), 670-676. doi.org:/10.1086/338209 belinda, c., melwani, s., & kapadia, c. (2024). breaking boredom: interrupting the residual effect of state boredom on future productivity. journal of applied psychology. bench, s. w., & lench, h. c. (2013). on the function of boredom. behavioral sciences, 3(3), 459472. biolcati, r., mancini, g., & trombini, e. (2018). proneness to boredom and risk behaviors during adolescents’ free time. psychological reports, 121(2), 303-323. doi.org:/10.1177/0033294117724447 blaszczynski, a., mcconaghy, n., & frankova, a. (1990). boredom proneness in pathological gambling. psychological reports, 67(1), 35-42. brissett, d., & snow, r. p. (1993). boredom: where the future isn't. symbolic interaction, 16(3), 237-256. https://www.jstor.org/stable/10.1525/si.1993.16.3.237 bozaci, i. (2020). the effect of boredom proneness on smartphone addiction and impulse purchasing: a field study with young consumers in turkey. the journal of asian finance, economics and business, 7(7), 509-517. doi.org:/10.13106/jafeb.2020.vol7.no7.509 carruthers, c., & hood, c. d. (2004). the power of the positive: leisure and wellbeing. therapeutic recreation journal, 38(2), 225-245. catedrilla, j., ebardo, r., limpin, l., cuesta, j. d., ching, m. r., trapero, h., & leano, c. (2020, november). loneliness, boredom and information anxiety on problematic use of social media during the covid-19 pandemic. in international conference on computers in education. chou, w. j., chang, y. p., & yen, c. f. (2018). boredom proneness and its correlation with internet addiction and internet activities in adolescents with attentiondeficit/hyperactivity disorder. the kaohsiung journal of medical sciences, 34(8), 467-474. doi.org:/10.1016/j.kjms.2018.01.016 australasian journal of information systems ali, khan & wang 2025, vol 29, research article understanding boredom in online commerce 26 cowen, e. l. (1991). in pursuit of wellness. american psychologist, 46(4), 404. derakhshan, a., kruk, m., mehdizadeh, m., & pawlak, m. (2022). activity-induced boredom in online efl classes. elt journal, 76(1), 58-68. deng, s., wang, w., xie, p., chao, y., & zhu, j. (2020). perceived severity of covid-19 and post-pandemic consumption willingness: the roles of boredom and sensationseeking. frontiers in psychology, 11, 567784. doi.org:/10.3389/fpsyg.2020.567784 diefenbach, s., & borrmann, k. (2019, may). the smartphone as a pacifier and its consequences: young adults' smartphone usage in moments of solitude and correlations to self-reflection. in proceedings of the 2019 chi conference on human factors in computing systems (pp. 1-14). drach, r. d., orloff, n. c., & hormes, j. m. (2021). the emotion regulatory function of online social networking: preliminary experimental evidence. addictive behaviors, 112, 106559. doi.org:/10.1016/j.addbeh.2020.106559 eastwood, j. d., frischen, a., fenske, m. j., & smilek, d. (2012). the unengaged mind: defining boredom in terms of attention. perspectives on psychological science, 7(5), 482-495. doi.org:/10.1177/1745691612456044 elhai, j. d., vasquez, j. k., lustgarten, s. d., levine, j. c., & hall, b. j. (2018). proneness to boredom mediates relationships between problematic smartphone use with depression and anxiety severity. social science computer review, 36(6), 707-720. doi.org:/10.1177/0894439317741087 elpidorou, a. (2015). the significance of boredom: a sartrean reading. in philosophy of mind and phenomenology (pp. 268-284). routledge. elpidorou, a. (2018). the good of boredom. philosophical psychology, 31(3), 323-351. doi.org:/10.1080/09515089.2017.1346240 emmons, r. a. (1981). relationship between narcissism and sensation seeking. psychological reports, 48(1), 247-250. fahlman, s. a., mercer-lynn, k. b., flora, d. b., & eastwood, j. d. (2013). development and validation of the multidimensional state boredom scale. assessment, 20(1), 68-85. doi.org:/10.1177/1073191111421303 farmer, r., & sundberg, n. d. (1986). boredom proneness--the development and correlates of a new scale. journal of personality assessment, 50(1), 4-17. fenichel, o. (1951). on the psychology of boredom. fisherl, c. d. (1993). boredom at work: a neglected concept. human relations, 46(3), 395-417. gerritsen, c. j., toplak, m. e., sciaraffa, j., & eastwood, j. (2014). i can’t get no satisfaction: potential causes of boredom. consciousness and cognition, 27, 27-41. doi.org:/10.1016/j.concog.2013.10.001 goodstein, e. s. (2005). experience without qualities: boredom and modernity. greenson, r. r. (1953). on boredom. journal of the american psychoanalytic association, 1(1), 721. australasian journal of information systems ali, khan & wang 2025, vol 29, research article understanding boredom in online commerce 27 güngördü belbağ, a. (2022). impacts of covid‐19 pandemic on consumer behavior in turkey: a qualitative study. journal of consumer affairs, 56(1), 339-358. doi.org:/ 10.1111/joca.12423 guo, y. m., & klein, b. d. (2007). skills matter: a tale of the anxious online shopper. in humancomputer interaction. hci applications and services: 12th international conference, hci international 2007, beijing, china, july 22-27, 2007, proceedings, part iv 12 (pp. 32-41). springer berlin heidelberg. kumar, h., garg, r., kumar, p., & chhikara, r. (2020). a qualitative insight into the personal factors impacting online impulse behavior. in strategies and tools for managing connected consumers (pp. 279-291). igi global. harris, m. b. (2000). correlates and characteristics of boredom proneness and boredom 1. journal of applied social psychology, 30(3), 576-598. hill, a. b., & perkins, r. e. (1985). towards a model of boredom. british journal of psychology, 76(2), 235-240. hoffner, c. a., lee, s., & park, s. j. (2016). “i miss my mobile phone!”: self-expansion via mobile phone and responses to phone loss. new media & society, 18(11), 2452-2468. doi.org:/10.1177/1461444815592665 holte, a. j., & ferraro, f. r. (2020). anxious, bored, and (maybe) missing out: evaluation of anxiety attachment, boredom proneness, and fear of missing out (fomo). computers in human behavior, 112, 106465. doi.org:/10.1016/j.chb.2020.106465 howarth, r. (2023, november 23). 17 cyberbullying facts & statistics. exploding topics. https://explodingtopics.com/blog/cyberbullying-stats huang, z., & benyoucef, m. (2013). from e-commerce to social commerce: a close look at design features. electronic commerce research and applications, 12(4), 246-259. iso-ahola, s. e., & weissinger, e. (1990). perceptions of boredom in leisure: conceptualization, reliability and validity of the leisure boredom scale. journal of leisure research, 22(1), 117. jiang, q., & li, y. (2018). factors affecting smartphone dependency among the young in china. asian journal of communication, 28(5), 508-525. doi.org:/10.1080/01292986.2018.1431296 jung, j., & yi, s. (2014). assessment of heterogeneity of compulsive buyers based on affective antecedents of buying lapses. addiction research & theory, 22(1), 37-48. doi.org:/10.3109/16066359.2012.756475 keeney, r. l. (1999). the value of internet commerce to the customer. management science, 45(4), 533-542. kemp, simon. (2024, january 31). internet use in 2024. datareportal. https://datareportal.com/reports/digital-2024-deep-dive-the-state-of-internetadoption#:~:text=there%20are%205.35%20billion%20people,the%20first%20time%20d uring%202023. australasian journal of information systems ali, khan & wang 2025, vol 29, research article understanding boredom in online commerce 28 kiss, h., fitzpatrick, k. m., & piko, b. f. (2020). the digital divide: risk and protective factors and the differences in problematic use of digital devices among hungarian youth. children and youth services review, 108, 104612. doi.org:/10.1016/j.childyouth.2019.104612 koban, k., stein, j. p., eckhardt, v., & ohler, p. (2018). quid pro quo in web 2.0. connecting personality traits and facebook usage intensity to uncivil commenting intentions in public online discussions. computers in human behavior, 79, 9-18. doi.org:/10.1016/j.chb.2017.10.015 ksinan, a. j., mališ, j., & vazsonyi, a. t. (2021). swiping away the moments that make up a dull day: narcissism, boredom, and compulsive smartphone use. current psychology, 40, 2917-2926. doi.org:/10.1007/s12144-019-00228-7 kuei huang, y., & yang, w. i. (2008). motives for and consequences of reading internet book reviews. the electronic library, 26(1), 97-110. doi.org:/10.1108/02640470810851770 leong, f. t., & schneller, g. r. (1993). boredom proneness: temperamental and cognitive components. personality and individual differences, 14(1), 233-239. lepera, n. (2011). relationships between boredom proneness, mindfulness, anxiety, depression, and substance use. the new school psychology bulletin, 8(2), 15-25. leung, l. (2020). exploring the relationship between smartphone activities, flow experience, and boredom in free time. computers in human behavior, 103, 130-139. doi.org:/10.1016/j.chb.2019.09.030 lewinsky, h. (1943). boredom. british journal of educational psychology, 13(3), 147-152. li, m., yin, j., lu, s., & zhong, n. (2009). the effect of information forms and floating advertisements for visual search on web pages: an eye-tracking study. in brain informatics: international conference, bi 2009 beijing, china, october 22-24, 2009 proceedings (pp. 96-105). springer berlin heidelberg. liang, t. p., & turban, e. (2011). introduction to the special issue social commerce: a research framework for social commerce. international journal of electronic commerce, 16(2), 5-14. lin, c., & yu, s. (2008). adolescent internet usage in taiwan: exploring gender differences. adolescence, 43(170). lin, t. t., kononova, a., & chiang, y. h. (2020). screen addiction and media multitasking among american and taiwanese users. journal of computer information systems. doi.org:/10.1080/08874417.2018.1556133 mann, s., & cadman, r. (2014). does being bored make us more creative?. creativity research journal, 26(2), 165-173. doi.org:/10.1080/10400419.2014.901073 mano, h. (1999). the influence of pre-existing negative affect on store purchase intentions. journal of retailing, 75(2), 149-172. doi.org:/10.1016/s0022-4359(99)00002-0 martin, m., sadlo, g., & stew, g. (2006). the phenomenon of boredom. qualitative research in psychology, 3(3), 193-211. doi.org:/10.1191/1478088706qrp066oa australasian journal of information systems ali, khan & wang 2025, vol 29, research article understanding boredom in online commerce 29 mastro, d. e., eastin, m. s., & tamborini, r. (2002). internet search behaviors and mood alterations: a selective exposure approach. media psychology, 4(2), 157-172. doi.org:/10.1207/s1532785xmep0402_03 meier, m., martarelli, c. s., & wolff, w. (2024). is boredom a source of noise and/or a confound in behavioral science research?. humanities and social sciences communications, 11(1), 18. mikulas, w. l., & vodanovich, s. j. (1993). the essence of boredom. the psychological record, 43(1), 3. mugon, j., struk, a., & danckert, j. (2018). a failure to launch: regulatory modes and boredom proneness. frontiers in psychology, 9, 1126. mori, g., paternò, f., & furci, f. (2014). design criteria for web applications adapted to emotions. in web engineering: 14th international conference, icwe 2014, toulouse, france, july 1-4, 2014. proceedings 14 (pp. 400-409). springer international publishing. ng, a. h., liu, y., chen, j. z., & eastwood, j. d. (2015). culture and state boredom: a comparison between european canadians and chinese. personality and individual differences, 75, 13-18. doi.org:/10.1016/j.paid.2014.10.052 opie, c. (2004). doing educational research. paasonen, s. (2018). affect, data, manipulation and price in social media. distinktion: journal of social theory, 19(2), 214-229. doi.org:/10.1080/1600910x.2018.1475289 panova, t., & lleras, a. (2016). avoidance or boredom: negative mental health outcomes associated with use of information and communication technologies depend on users’ motivations. computers in human behavior, 58, 249-258. doi.org::/10.1016/j.chb.2015.12.062 pawlak, m., derakhshan, a., mehdizadeh, m., & kruk, m. (2025). boredom in online english language classes: mediating variables and coping strategies. language teaching research, 29(2), 509-534. pekrun, r., goetz, t., daniels, l. m., stupnisky, r. h., & perry, r. p. (2010). boredom in achievement settings: exploring control–value antecedents and performance outcomes of a neglected emotion. journal of educational psychology, 102(3), 531. doi.org:/ 10.1037/a0019243 raffaelli, q., mills, c., & christoff, k. (2018). the knowns and unknowns of boredom: a review of the literature. experimental brain research, 236, 2451-2462. doi.org:/10.1007/s00221-017-4922-7 saleem, f. z., & iglesias, o. (2020). exploring the motivation of affect management in fostering social media engagement and related insights for branding. internet research, 30(1), 6783. doi.org:/10.1108/intr-07-2018-0321 singh, r. (2016). monotony of social networking among millennial and its effect on social advertisement: a challenge to digital marketers. young consumers, 17(4), 376-387. doi:/10.1108/yc-05-2016-00605 australasian journal of information systems ali, khan & wang 2025, vol 29, research article understanding boredom in online commerce 30 stieger, s., & lewetz, d. (2018). a week without using social media: results from an ecological momentary intervention study using smartphones. cyberpsychology, behavior, and social networking, 21(10), 618-624. doi.org:/10.1089/cyber.2018.0070 stephen, a. t., & toubia, o. (2010). deriving value from social commerce networks. journal of marketing research, 47(2), 215-228. doi.org:/10.1509/jmkr.47.2.215 stockdale, l. a., & coyne, s. m. (2020). bored and online: reasons for using social media, problematic social networking site use, and behavioral outcomes across the transition from adolescence to emerging adulthood. journal of adolescence, 79, 173-183. doi.org:/10.1016/j.adolescence.2020.01.010 strauss, a., & corbin, j. (1990). basics of qualitative research (vol. 15). newbury park, ca: sage. strauss, a., and corbin, j. (1998). basics of qualitative research: techniques and procedures for developing grounded theory. sage studak, c. m., & workman, j. e. (2004). fashion groups, gender, and boredom proneness. international journal of consumer studies, 28(1), 66-74. suddaby, r. (2010). construct clarity in theories of management and organization: editor’s comments. academy of management review, 35(3), 346-357. sundberg, n. d., latkin, c. a., farmer, r. f., & saoud, j. (1991). boredom in young adults: gender and cultural comparisons. journal of cross-cultural psychology, 22(2), 209-223. sundström, m., hjelm-lidholm, s., & radon, a. (2019). clicking the boredom away– exploring impulse fashion buying behavior online. journal of retailing and consumer services, 47, 150-156. doi.org:/10.1016/j.jretconser.2018.11.006 throuvala, m. a., griffiths, m. d., rennoldson, m., & kuss, d. j. (2019). motivational processes and dysfunctional mechanisms of social media use among adolescents: a qualitative focus group study. computers in human behavior, 93, 164-175. doi.org:/10.1016/j.chb.2018.12.012 tolor, a. (1989). boredom as related to alienation, assertiveness, internal‐external expectancy, and sleep patterns. journal of clinical psychology, 45(2), 260-265. tomczyk, l., & solecki, r. (2019). problematic internet use and protective factors related to family and free time activities among young people. educational sciences: theory and practice, 19(3), 1-13. doi.org:/10.12738/estp.2019.3.001 tzavela, e. c., karakitsou, c., halapi, e., & tsitsika, a. k. (2017). adolescent digital profiles: a process-based typology of highly engaged internet users. computers in human behavior, 69, 246-255. doi.org:/10.1016/j.chb.2016.11.032 van tilburg, w. a., & igou, e. r. (2012). on boredom: lack of challenge and meaning as distinct boredom experiences. motivation and emotion, 36, 181-194. doi.org:/10.1007/s11031-011-9234-9 van tilburg, w. a., & igou, e. r. (2017). boredom begs to differ: differentiation from other negative emotions. emotion, 17(2), 309. doi.org:/10.1037/emo0000233 australasian journal of information systems ali, khan & wang 2025, vol 29, research article understanding boredom in online commerce 31 vodanovich, s. j. (2003). psychometric measures of boredom: a review of the literature. the journal of psychology, 137(6), 569-595. doi.org:/10.1080/00223980309600636 vodanovich, s. j., & watt, j. d. (2016). self-report measures of boredom: an updated review of the literature. the journal of psychology, 150(2), 196-228. doi.org;/10.1080/00223980.2015.1074531 wacker, j. g. (2004). a theory of formal conceptual definitions: developing theory-building measurement instruments. journal of operations management, 22(6), 629-650. doi.org;/10.1016/j.jom.2004.08.002 wang, y., yang, h., montag, c., & elhai, j. d. (2020). boredom proneness and rumination mediate relationships between depression and anxiety with problematic smartphone use severity. current psychology, 1-11. doi.org;/10.1007/s12144-020-01052-0 watt, j. d., & vodanovich, s. j. (1992). relationship between boredom proneness and impulsivity. psychological reports, 70(3), 688-690. watt, j. d., & vodanovich, s. j. (1999). boredom proneness and psychosocial development. the journal of psychology, 133(3), 303-314. doi.org;/10.1080/00223989909599743 wegmann, e., ostendorf, s., & brand, m. (2018). is it beneficial to use internet-communication for escaping from boredom? boredom proneness interacts with cue-induced craving and avoidance expectancies in explaining symptoms of internet-communication disorder. plos one, 13(4), e0195742. weinstein, e. the social media see-saw: positive and negative influences on adolescents’ affective well-being. new media and society, 20, 10 (2018), 3597-3623. doi.og:/10.1177/1461444818755634 whelan, e., najmul islam, a. k. m., & brooks, s. (2020). is boredom proneness related to social media overload and fatigue? a stress–strain–outcome approach. internet research, 30(3), 869-887. doi.org:/10.1108/intr-03-2019-0112 wilson, t. d., reinhard, d. a., westgate, e. c., gilbert, d. t., ellerbeck, n., hahn, c., brown, c. l., & shaked, a. just think: the challenges of the disengaged mind. science, 345, 6192 (2014), 75–77. wolfswinkel, j. f., furtmueller, e., & wilderom, c. p. (2013). using grounded theory as a method for rigorously reviewing literature. european journal of information systems, 22(1), 45-55. doi.org:/10.1057/ejis.2011.51 yang, x., & li, g. x. (2014, august). exploring social commerce adoption in china: a uses and gratification perspective. in 2014 international conference on management science & engineering 21th annual conference proceedings (pp. 546-554). ieee. doi.org:/10.1109/icmse.2014.6930277 yazdanparast, a., joseph, m., & qureshi, a. (2015). an investigation of facebook boredom phenomenon among college students. young consumers, 16(4), 468-480. doi.og:/10.1108/yc-02-2015-005 australasian journal of information systems ali, khan & wang 2025, vol 29, research article understanding boredom in online commerce 32 appendices appendix a: definitions of boredom in the sample year article name definition sample source original source 1 1992 relationship between boredom proneness and impulsivity boredom prone individual as one who experiences “varying degrees of depression, hopelessness, loneliness, and distractibility” (watt & vodanovich, 1992, p. 688) (farmer & sundberg, 1986, p. 14) 2 2014 i can’t get no satisfaction: potential causes of boredom. difficult for boredom-prone individuals to “articulate a satisfying target for engagement” (gerritsen et al., 2014, p. 27) (eastwood et al., 2012, p. 483) 3 2014 assessment of heterogeneity of compulsive buyers based on affective antecedents of buying lapses defined as “a state of mental weariness and dissatisfaction produced by lack of interest or activity” (jung & yi, 2014) (blaszczynski et al., 1990, p. 36) 4 2014 i can’t get no satisfaction: potential causes of boredom. “restlessness and entrapment, combined with lethargy” (gerritsen et al., 2014, p. 30) (martin et al., 2006, p. 206) 5 2015 an investigation of facebook boredom phenomenon among college students “boredom is a feeling of unpleasure arising out of a conflict between a need for intense mental activity and lack of incitement to it, or inability to be incited” (yazdanparast et al., 2015, p. 470) (fenichel, 1953, p. 292) 6 2015 an investigation of facebook boredom phenomenon among college students overall, different approaches to theorizing boredom converge on the fact that “the central defining feature of boredom is the aversive experience of wanting, but being unable to engage in a stimulating and satisfying activity” (yazdanparast et al. 2015, p. 471) (fahlman et al., 2013, p. 69) 7 2016 monotony of social networking among millennial and its effect on social advertisement: a challenge to digital marketers the definition of monotony (boredom) postulates that “boredom is a feeling of unpleasure arising out of a conflict between a need for intense mental activity and lack of incitement to it, or inability to be incited” (singh, 2016, p. 378) (fenichel, 1953, p. 292) 8 2018 affect, data, manipulation and price in social media boredom as both the cause and effect of modernity: as “both the disaffection […] that drives the search for (paasonen, 2018, p. 221) (goodstein, 2005, p. 1) australasian journal of information systems ali, khan & wang 2025, vol 29, research article understanding boredom in online commerce 33 change and as the malaise produced by living under a permanent speed-up” 9 2018 is it beneficial to use internet-communication for escaping from boredom? boredom proneness interacts with cue-induced craving and avoidance expectancies in explaining symptoms of internetcommunication disorder boredom as a state of “under-stimulation, underarousal, and lack of psychological involvement associated with dissatisfaction, and individuals try to cope with boredom by seeking additional stimulation” (biolcati et al., 2017, p. 304; wegmann et al., 2018, p. 3) (brissett & snow, 1993) 10 2018 is it beneficial to use internet-communication for escaping from boredom? boredom proneness interacts with cue-induced craving and avoidance expectancies in explaining symptoms of internetcommunication disorder the construct of boredom proneness is often “operationalized as an individual’s susceptibility to experience boredom” (wegmann et al., 2018, p. 3) (lepera n, 2011, p.15) 11 2018 boredom proneness and its correlation with internet addiction and internet activities in adolescents with attentiondeficit/hyperactivity disorder. “a state of relatively low arousal and dissatisfaction, which is attributed to an inadequately stimulating environment” (chou et al., 2018, p. 468) (mikulas & vodanovich, 1993, p. 3) 12 2018 boredom proneness and its correlation with internet addiction and internet activities in adolescents with attentiondeficit/hyperactivity disorder. “cognitive changes in the direction of less differentiated and more homogeneous construing give rise to a state of subjective monotony which induces, or perhaps even represents, the state” (chou et al., 2018, p. 468) (hill & perkins, 1985, p. 231) 13 2019 exploring the relationship among free-time management, leisure boredom, and internet addiction in undergraduates in taiwan leisure boredom is defined as the “subjective perception that available leisure experiences are not sufficient to instrumentally satisfy needs for optimal arousal” (wang, 2019, p. 1653) (iso ahola & weissinger, 1990, p. 4-5) 14 2019 the smartphone as a pacifier and its consequences: young adults’ smartphone usage in moments of “boredom is a state of relatively low arousal and dissatisfaction, which is attributed to an inadequately stimulating situation” (diefenbach & borrmann, 2019, p. 5) (mikulas & vodanovich, 1993, p. 3) australasian journal of information systems ali, khan & wang 2025, vol 29, research article understanding boredom in online commerce 34 solitude and correlations to self-reflection 15 2019 clicking the boredom away – exploring impulse fashion buying behavior online an individual level of boredom and a mundane experience is concurrent with a contextualised totality and is an interplay of influences that shapes the consumer (sundström et al., 2019, p. 151) (sundström et al., 2019, p. 151) 16 2020 exploring the relationship between smartphone activities, flow experience, and boredom in free time boredom is defined as an “unpleasant, transient affective state” in which an individual is left with nothing in particular to do and lacks interest in his or her surroundings (iso-ahola & weissinger, 1990). (leung, 2020, p. 132) (fisher, 1993, p.3) 17 2020 exploring the relationship between smartphone activities, flow experience, and boredom in free time leisure boredom is conceptualised as “the subjective perception that available leisure experiences are not sufficient to instrumentally satisfy needs for optimal arousal… leisure experience are not sufficiently frequent, involving, exciting, varied or novel” (leung, 2020, p. 132) (iso-ahola & weissinger, 1990, p. 4) 18 2020 is boredom proneness related to social media overload and fatigue? a stress–strain–outcome approach boredom has been depicted as being due to both cognitive and attentional deficits (fisher, 1998; kass et al., 2010; mikulas and vodanovich, 1993), which usually emerges when “the individual feels a pervasive lack of interest in and difficulty concentrating on the current activity” (whelan et al., 2020, p. 873) (fisher, 1998, p. 396) table a.1. boredom definitions appendix b: descriptive data figures b.1 and b.2 provide descriptive statistics of current review sample. table f.1 of appendix f (in supplementary data) also provides a complete list of current sample’s sources which served to generate this information. figure b.1 provides the source by year, broken down according to the five main types of sources reviewed: survey, interview, experiment, mix method, and multi method design. the total sample size for figure b.1 is n = 103 (56 survey; 21 others, 7 multi method, 6 mixed method, 8 interviews and 5 experiment). figure b.2 presents each method in percentage. australasian journal of information systems ali, khan & wang 2025, vol 29, research article understanding boredom in online commerce 35 figure b.1: breakdown of sources by methodology type (n = 103) figure b.2: breakdown of methodology type in percentage (n = 103) note: others include a methodology whose number is less than 2 in the sample as follows: behavioral archetypes creation and validation, case study, content analysis, data mining approach, descriptive design, differential language analysis, ecological momentary intervention, fieldwork, kansei measurement process, review, focus group, secondary data. figure b.3 provides a count of source by year, broken down according to the five main sectors: social networks, smartphones, internet, fashion, and communication technologies. the remainder are “others”. the total sample size for figure b.3 is n = 103 (27 social networks; 15 smartphones, 20 internet, 4 fashion, 3 communication technologies and 34 others). figure b.4 presents each sector in percentage. australasian journal of information systems ali, khan & wang 2025, vol 29, research article understanding boredom in online commerce 36 figure b.3: breakdown of sources by sectors and industries type (n = 103) figure b.4: breakdown of sector type in percentage (n = 103) note: “others” includes sectors whose number is less than 2 in the sample. these are: advertisement, agents and artificial intelligence, agricultural economics, communication and information, computers in human behaviour, consumer behavior, digital technology, educational and internet, emotions, human-computer interaction, information society and new media, marketing & consumer, media and health, media psychology, mood and emotions, non-profit organisations, online shopping systems, psychological, retailing, technology, travel, web application, youth, youth and technology, communication technologies. 37 appendix c: visual representation of themes and sub themes in nvivo figures c.1 to c.5 provides a visual representation of the main themes and their sub themes. these emerged in present analysis following sample analysis in nvivo. figure c.1: visual representation of the connection between the main themes and their sub themes 38 figure c.2: visual representation of the theme “boredom’s function” and its sub themes 39 figure c.3: visual representation of the theme “causes of boredom” and its sub themes 40 figure c.4: visual representation of the theme “alleviation strategies” and its sub themes 41 figure c.5: visual representation of the theme “consequences of boredom” and its sub themes 42 appendix d: guidelines of conceptual definition this study’s definition of boredom follows the guidelines of conceptual definition presented by wacker (2004) and suddaby (2010). they guidelines presented in table d.1 and d.2 below. 1 “definitions should be formally defined using primitives and derived terms”. 2 “each concept should be uniquely defined”. 3 “definitions should include only unambiguous and clear terms”. 4 “definitions should have as few as possible terms”. 5 “definitions should be consistent within [their] field”. 6 “definitions should not make any term broader”. 7 “new hypotheses cannot be introduced in the definitions”. 8 “a statistical test for content validity must be performed after the terms are formally defined” table d.1 8 rules for conceptual definitions (adapted from wacker, 2004, p. 384) 1 “offer definitions of key terms and constructs” 2 “the definition should capture the essential properties and characteristics of the concept or phenomenon under consideration” 3 “a good definition should avoid tautology or circularity” 4 “a good definition should be parsimonious” table d.2 guidelines for conceptual clarity (adapted from suddaby, 2010, p. 347) this study’s definition of boredom follows rule of replacement i.e. definiens (negative affective) can be substituted in any sentence for the definiendum (boredom) without changing the sentence’s meaning. all terms used in the definition are unique and there exists no overlapping. modifiers, such as the kind of stimulation and consequences, are used to free the definition from ambiguity and vagueness. this definition used fewer terms are used to convey the essence of the concept and followed the rule of parsimony. since the inductive framework was built on the context of online commerce and this definition comprises its themes, it is therefore consistent with its field. it is not the expansion of any current definition and nor does it contain any new hypothesis (antecedents, outcomes) – instead of the words “increase” or “decrease”, “aims to” is used. this definition is free from circularity and unclear terminologies. it acknowledges that it is reliving itself as an expected outcome without guaranteeing its realisation. copyright copyright © 2025 rebecca ali, gohar khan & william wang. this is an open-access article licensed under a creative commons attribution-non-commercial 4.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v29.5047 https://creativecommons.org/licenses/by-nc/4.0/deed.en paper title (paper title style) australasian journal of information systems paltiel et al. 2023, vol 27, selected papers from acis 2022 models for teaching digital ethics 1 approaches and models for teaching digital ethics in information systems courses – a review of the literature minna paltiel melbourne law school, the university of melbourne, australia mpaltiel@student.unimelb.edu.au marc cheong school of computing and information systems, the university of melbourne, australia simon coghlan school of computing and information systems, centre for ai and digital ethics (caide), the university of melbourne, australia reeva lederman school of computing and information systems, the university of melbourne, australia abstract the value and importance of teaching digital ethics within information systems and ict courses is widely recognized and stand-alone or integrated digital ethics units are broadly implemented across degree courses. however, how such courses should be taught and what content they should include is a little explored topic. using a narrative literature review methodology, this paper reviews the pedagogical theories underpinning digital ethics courses discussed in the literature and outlines approaches that deploy standalone ethics units, integrated ethics teaching, and hybrid teaching approaches and the use of interdisciplinary models. the paper identifies the employment of, and emphasis given to various moral theories in digital ethics education. the paper then discusses how our findings relating to different pedagogical approaches, degree of integration of ethics teaching, the use of interdisciplinary models and use of moral theories—are related to each other. the discussion explores trends in approaches and models for teaching digital ethics highlighted in the review, and makes recommendations for further exploration and inquiry. it concludes that the effective teaching of digital ethics will likely involve a considered combination of approaches, models and techniques, which may also be tailored to the needs of different roles and industries. keywords: information systems, digital ethics, education, pedagogical theories, moral theories. 1 introduction digital ethics examines moral questions raised by information and communication technologies (ict), from laptops and smartphones to social media and modern artificial intelligence (ai) (véliz, 2021). there is a growing appreciation that digital ethics education is required to address the pervasive impact of ict in society. this view is reflected in current requirements for accreditation in engineering generally (bradley, 2008; martin et al., 2021) and computing and information system sciences specifically. the rising need for digital ethics education raises the question of how teachers currently teach digital ethics in tertiary settings. in this paper, we address this question by means of a narrative review and discussion of the literature on digital ethics teaching programs. australasian journal of information systems paltiel et al. 2023, vol 27, selected papers from acis 2022 models for teaching digital ethics 2 although relatively new, digital ethics is a growing discipline that encompasses computer ethics, information ethics (bynum, 2015), and ai ethics (müller, 2020). it addresses issues such as data security and privacy (which are core is concerns), algorithmic fairness and transparency, superintelligence, social media ethics, ai explainability, data law, the power of tech giants, surveillance, moral machines, and much more (floridi et al., 2019; véliz, 2021). recognition of the importance of ethics education in computing sciences (cs) and information systems (is) goes back some decades, see e.g., the works of johnson (1985), moor (1985), and mumford (1995). for example, ethics teaching requirements were included in the acm/ieeecs computing curriculum 1991 (tucker, 1991) and have been present in the computer science accreditation board curriculum standards since 1987 (califf & goodwin, 2005; huff & martin, 1995). huff and martin (1995) claimed that ict students who plan to enter industry should understand the ethical and social consequences of their work. growth in computerisation, ai, and automated decision making in business, services, and society generally (o’neil, 2016) has only increased the significance of their observation. such technology can carry substantial risks and harms. indeed, the overarching impact of computer technology on all aspects of personal, social and public life has made computer scientists “some of the most powerful moral agents in today’s world” (skirpan et al., 2018, p. 940). since these ubiquitous algorithms are created by humans and based on human data, they tend to reflect and embody imperfect, biased, and unfair circumstances and decision-making (borenstein & howard, 2021; see also eubanks, 2018). the need for ethics education goes beyond algorithm designers to include is practitioners and academics. for example, is specialists must often implement, oversee, and teach others about such algorithms. many of the large ecosystems powering modern leisure and work – from social networking sites to cloud-based systems enabling us to work and connect during the ongoing covid-19 pandemic – are information systems at their very core. in fact, mason’s (1986) “prominent contribution to is ethics” in the 1980’s raised issues still very much present in contemporary is discourse: “privacy, accuracy, property, accessibility” (hassan et al., 2018), or “papa”. unfortunately, as kordzadeh and ghasemaghaei (2021) write in a reflection upon someh et al.’s (2019) work on algorithmic bias, “information systems (is) researchers have largely fallen behind in addressing the behavioural, organisational, and social implications, antecedents, and consequences of this issue" when compared to their computational science counterparts (kordzadeh & ghasemaghae, 2021, p. 388). there is thus good reason for highlighting the need for is professionals to be better trained to identify ethical issues, reflect upon their own biases, and provide ethically sound solutions to contemporary techno-social problems. unsurprisingly, recent scholars have again called upon ict educators to train future professionals (and other community stakeholders) not only in the requisite technical proficiencies, but also in ethics, so that they can be better prepared for the ethical responsibilities inherent in practice (borenstein & howard, 2021; grosz et al., 2019; skirpan et al., 2018). many of those who currently teach ethics in is and related disciplines, however, lack experience and training in philosophical theory and digital ethics pedagogy. furthermore, programs and interventions for teaching ethics vary greatly in structure, methods, and pedagogical approach and in the educational tools they employ. this review aims to assist digital ethics educators in understanding the teaching options available. australasian journal of information systems paltiel et al. 2023, vol 27, selected papers from acis 2022 models for teaching digital ethics 3 our narrative literature review focuses especially on more recent scholarly work concerning the teaching of ethics in ict-based university courses. it uncovers several themes and trends regarding current recommendations for digital ethics teaching in the present digital age. one theme relates to the different pedagogical approaches adopted. while a very broad range of approaches was reflected in the literature, we focus on the several key pedagogical theories. as we shall see, some of the literature recommends that ethics be taught in a way that is inextricable from the typical activities of cs and is (this combination hereinafter abbreviated cis) practitioners. as such, integrating ethics into the basic curriculum, rather than only presenting stand-alone ethics courses, is sometimes seen as desirable. this emphasis is also seen in adopting pedagogical approaches that teach ethics through direct experience, in situ, and within the cis context. another trend in the literature which we identify concerns the need to merge different ways of thinking in teaching, such as including social and learning sciences approaches in designing digital ethics courses and creating multidisciplinary teaching partnerships. there is also a theme about building ethical skills and virtues rather than acquiring moral knowledge sets. finally, some papers we reviewed described different learning tools such as case studies and games. the rest of the paper runs as follows. section 2 describes our methodology, section 3 presents our findings, section 4 discusses the findings and offers recommendations for digital ethics educators and direction for further inquiry, and section 5 concludes our analysis of digital ethics teaching. 2 search methodology to obtain a broad overview of the relevant literature, we conducted a narrative review, ‘a scholarly summary along with interpretation and critique’ (greenhalgh et al., 2018, pp. 2-3) to identify papers about digital ethics teaching for students in is and related fields. despite a preference for systematic reviews in is and the broader research community (such as medicine), we adopt a narrative review for two key reasons. first, systematic reviews presuppose a "specific" research question1 , whereas a narrative review style is "better suited to addressing a topic in wider ways" (baethge et al., 2019, p. 2). second, a broad narrative review allows us to learn from counterparts in the various other cis subfields and disciplines (kordzadeh & ghasemaghaei, 2021; someh et al., 2019). principles such as papa (privacy, accuracy, property, accessibility) (mason, 1986), and other ethical frameworks are not exclusive to is, but are shared across allied ict disciplines, including ai, data science, and engineering. the articles reviewed were found via a database search using university of melbourne library resources, and included: ai and ethics, ai and society, international journal of artificial intelligence in education, science and engineering ethics, journal of information, communication and ethics in society, ethics and information technology and communications of the acm. conference proceedings were also accessed, including: acm conference on fairness, accountability and transparency (faacct), conferences of the institute of electrical and electronics engineers (ieee) and conferences of the australasian institute of computer ethics. additional searches were conducted on google and google scholar. 1we thank the anonymous reviewers from acis 2022 for their valuable suggestions. australasian journal of information systems paltiel et al. 2023, vol 27, selected papers from acis 2022 models for teaching digital ethics 4 key terms used to search in titles or abstracts of publications included combinations of “ai/artificial intelligence” or “computer science/cs” and “ethics” and “teaching” or “education” or “curriculum” or “training”. second phase searches addressed particular models such as “fate”/“facct” and “embedded”. as a third phase, references in key publications were reviewed to identify additional relevant publications, journals, and conference proceedings pertaining to the subject of the review. papers selected for review were generally published within the past 10 years, although some earlier papers were included as background and to provide perspective on trends and developments. in all, 35 studies were reviewed, 86% of which were published in the last decade. for completeness, an overview table of the reviewed studies is provided in table 2, in the appendix. many additional papers were extracted from relevant databases, but were excluded for lack of relevance or significant contribution, based on reading the abstract and/or the article. 3 findings five themes and trends concerning recommended dimensions of digital ethics teaching emerged from our literature review. our findings reveal, first, a broad range of pedagogical approaches were adopted, but with an emphasis amongst scholars on ensuring that ethics is taught as inextricable from the activities of an ict practitioner. this involves pedagogical approaches that teach ethics through direct experience, in situ, and within an ict context (detailed in section 3.1). a second theme relates to the benefits of integrating ethics into the is/cs curriculum as opposed to presenting stand-alone ethics courses (see section 3.2). a third theme in the literature concerns the need to merge different ways of thinking, including social and learning sciences, in the design of digital ethics courses, and to work in multidisciplinary and interdisciplinary teaching partnerships (see section 3.3). a fourth theme is a focus on building ethical skills or developing moral values and virtues rather than acquiring a particular moral knowledge set. here there is considerable reference to the “virtue ethics” school and its focus on developing “ethical character” (see section 3.4). a final theme is the different content emphasis and range of pedagogical tools employed in the cis ethics courses described in the literature, which reflect the key themes and trends found (see section 3.5). 3.1 pedagogical approaches a pedagogical theory is a “theory of educational action, or a systematic view and reflection of pedagogic practice” (hämäläinen, 2012). in a sense, pedagogical theories operate at a higher level than educational approaches to specific subject matter and are compatible with a range of disciplines. even so, selection of an appropriate theory can be influenced by the particular subject being taught. the literature addressing digital ethics education refers to a broad variety and different combinations of pedagogical approaches. this appears to be partly because teaching ethics in cs and is involves the synthesis of several disciplines, and partly because although the need for ethics education was flagged thirty years ago, researchers are still working on formulating the optimal pedagogical basis for doing this. despite the wide range of pedagogical approaches referred to, several primary theories stand out, which we present below. 3.1.1 constructivist/constructionist approaches a dominant pedagogical theory referred to in the literature is constructivism, which has been called the “dominant theory of learning today” (ben-ari, 2001, p. 45). constructivism as a pedagogical approach stems from the piagetian theory of cognitive development according to australasian journal of information systems paltiel et al. 2023, vol 27, selected papers from acis 2022 models for teaching digital ethics 5 which knowledge is not passively absorbed, but actively constructed by a learner in interacting with her world (ackermann, 2001; piaget, 1971). the learner does this through the processes of assimilation and accommodation, integrating new experience in their environment with knowledge already held, and adapting mental schemas accordingly. while piaget developed his theory about children’s cognitive development, constructivist learning theory as a pedagogy applies to child and adult learners alike. a constructivist approach underpinned several digital ethics programs reviewed in the literature (e.g., bates et al., 2020; lewis & stoyanovich, 2021). papert’s constructionist theory of learning (papert, 1993; ackermann, 2001) informed a number of studies reviewed (ali et al., 2019; hjorth, 2019; lewis & stoyanovich, 2021; wise, 2020). constructionism is based on piaget’s constructivism and holds in common with that approach the principle that learning occurs through active engagement with the environment and internalisation of experience. however, constructionism adds that this occurs particularly when “the learner is engaged in constructing a public entity…”; in other words, it involves “learning through making” (papert & harel, 1991, p. 1). according to constructionism, knowledge is formed and transformed in particular contexts (and through particular individuals), and via particular uses and media (ackermann, 2001.) constructionism has been described as “both more situated and more pragmatic” than constructivism (ackerman, 2001, p. 5). a number of digital ethics programs in the literature, involving learning from direct experience, used constructivism and/or constructionism as guiding principles (briggle et al., 2016; holmes et al., 2021; richards et al., 2020). these programs are variously project-based (ali et al., 2019; hildt et al., 2019), interactive (lewis & stoyanovich, 2021), in situ (skirpan et al., 2018) and involve design or analysis and include “deliverables” such as “nutritional labels” (lewis & stoyanovich, 2021). these closely related approaches encourage “situated” ethical reasoning (grosz et al., 2019). many programs, even when not expressly described as based on constructivist and constructionist principles, illustrate these approaches. for example, grosz et al.’s “embedded ethics” model employs a “distributed pedagogy” which conveys the message that ethical reasoning is an integral part of computer science. it “situates” ethics learning in the context of cs activity, and aims for students to identify the ethical implications of their technological work and reason clearly about them in the process of developing and designing algorithms, systems, and codes (grosz et al., 2019). 3.1.2 social analysis approaches huff and martin (1995) suggested that every ethical concern arising in computer science is located at a particular level of social analysis, arguing that “[o]nly analysis that accounts for at least three dimensions technical, social, and ethical can represent the issues as they affect computer science in practice” (huff & martin, 1995, p. 76). the authors referred to project impactcs, begun in 1994, teaching students to identify and understand social issues and ethical issues associated with technology. the project uses a “social analysis” teaching approach in which: a technology and social issue was identified, the appropriate level of social analysis (e.g., individuals, communities, groups, organisations, cultures, institutions, etc.) is determined, and the appropriate tools, literature and methods are then applied (huff & martin, 1995). the authors identified the importance of developing a corresponding skill set in computer professionals. referring to project impactcs, barnard et al. (2003) set out the social australasian journal of information systems paltiel et al. 2023, vol 27, selected papers from acis 2022 models for teaching digital ethics 6 analysis skills critical to cis ethics as: “identification and interpretation of the social contexts of a particular implementation” identification of assumptions and values embedded in a particular system; and evaluation by means of empirical data of a particular implementation of a technology” (barnard et al., 2003, p. 269). a more recent example is the cs ethics curriculum presented by carter and crocket, which is based on “proven learning, teaching, moral education, and social analysis theory”, and integrates the social analysis approach with moral and ethical thinking (carter & crockett, 2019, section i). carter and crocket’s work, and moral reasoning approaches, are discussed further below. 3.1.3 approaches emphasising microethics versus macroethics the social analysis approach, and emphasis on large social issues, is congruent with a “macroethics” approach to ethics education. a macroethics approach in engineering and information sciences is concerned with the collective social responsibility in the profession, and societal decisions made about technology. in contrast, a “microethics” approach is concerned with individual responsibility and internal relations within the profession (herkert, 2004). li and fu (2012) hold that the microethics emphasis detracts from the need to deal with the social nature of technology practice and that it is fundamental to teach ethics within the applicable social, organisational, and political contexts. in contrast, bezuidenhout and ratti argue for a microethics model aimed at connecting “big picture ethics” to “everyday practice” (bezuidenhout & ratti, 2021,p. 940). they seek to foster “moral virtue” in learners by focusing on individual decision making and action, rather than on key themes and high-level case studies found in macroethics discourse. the aim is to develop moral excellence through virtue training, which is in contrast both to the approach described above focusing on identification of social issues, and the focus on ethical skills in the neo-kohlbergian approaches discussed below. bezuidenhout and ratti’s virtue training is accomplished through “discrete and repetitive” practice and by packaging “daily events” such as coding, clicking on content, and engaging in chat forums into “discrete instances of ethical reflection” (bezuidenhout & ratti, 2021, p. 947). 3.1.4 cognitivist or moral reasoning approaches training in cognitive moral reasoning is referred to in many of the courses we reviewed (mumford et al., 2008; richards et al., 2020; sprague & diaz-sprague, 2019). these are often based on ideas of moral learning advanced by psychologist lawrence kohlberg (1984) or on the neo-kohlbergian idea that skills of moral focus, sensitivity, and action may be strengthened through practice (mumford et al., 2008; rest et al., 1999; richards et al., 2020). an illustrative example is mumford et al.’s (2008) “sensemaking” approach which uses a moral reasoning model based on kohlberg and rest (kohlberg, 1984; mumford et al., 2008; rest, 1986). “sensemaking” refers to a complex cognitive process activated when people are presented with “ambiguous high-stake events” (mumford et al., 2008, p. 316). various mental models are applied to understand the situation as the foundation of decision making. according to this approach, ethical decision making is based on the available case-based models applied to the situation by the decision maker (mumford et al., 2008). this moral reasoning model is combined with a “field practices approach”, focusing instruction on codes of conduct and guidelines specific to a particular field, and a “case analysis approach”, in which analysis of prior cases provides a framework to “make sense” of australasian journal of information systems paltiel et al. 2023, vol 27, selected papers from acis 2022 models for teaching digital ethics 7 the ethical dilemma (mumford et al., 2008). the cognitivist and moral reasoning approaches often employ case studies as a pedagogical tool, as discussed further below. another interesting example is richards et al.’s “ai enhanced” game for training in cybersecurity ethics, which is likewise based on the neo-kohlbergian view that ethical expertise may be assessed in terms of the skills of moral focus, sensitivity and action, and that these skills may be strengthened through practice (richards et al., 2020; see also rest et al., 1999). the game itself, a training simulator, is guided by the “theory of situated and experiential learning” developed by articulated by james paul gee (gee, 2007, cited in richards et al., 2020), which requires “contextualised exploration, discovery, and practice” (richards et al., 2020, p. 6). 3.1.5 reflexive and deliberative approaches the final pedagogical theory emerging from our review are reflexive and deliberative approaches to digital ethics teaching. the reflexive approach involves critically questioning one’s own assumptions, judgements, and practice. bezuidenhout and ratti (2021) sought to develop “critical reflexivity” in cs students, while barabas et al. (2020) propose a “studying up” model, borrowed from the field of anthropology, aimed at achieving more reflexive data science practices. barabas et al.’s approach focuses on critically reflecting on dominant modes of interpretation data which reinforce factors leading to negative social outcomes. the authors argued that algorithmic fairness must be examined in the light of institutional context, oppression and control, with dominant modes of data interpretation reinforcing hierarchies and biasing the outcomes (barabas et al., 2020). barabas et al. emphasised the role of data scientists as agents in this process of developing frameworks and structures for evaluating algorithmic fairness, and suggested that computer scientists must embrace more reflexive practice (barabas et al., 2020). the literature also includes recommendations for a “deliberative” approach in cis ethics courses. this approach enables consideration of many points of view and involves discussion and interaction with others. it emphasises the development of an ethical culture rather than a strong emphasis on individual morality. plemmons et al. (2020) reported on a stem research ethics training intervention which sought to improve participants’ “reason giving” and “interpersonal communication” abilities for more ethical practice. the course was based on the proposition that if laboratory members routinely speak to each other about ethical issues arising in research practice, they are more likely to consciously select practices and procedures that are ethically defensible. this intervention aimed at fostering a more ethically sound culture in stem laboratories (plemmons et al., 2020). the deliberative approach is also illustrated in several ethics courses for neurotechnology students. farooqui et al. (2021) proposed an ethics course for neurotechnologists and neuroengineers founded on dialogue and involving monthly discussions. the course melded the reflexive and deliberative approaches in emphasising dialogue and exploration while also focusing on the relationship between the ethical principles discussed and students’ own values. tubig and mccusker also combined ethical reflexivity and discourse in their ethics dialogue tool – scientific perspectives and ethics commitments survey (specs) (tubig & mccusker, 2021). this involved reflection through discourse in which “discussants articulate and critically examine the values and beliefs that may be governing their group activities and the australasian journal of information systems paltiel et al. 2023, vol 27, selected papers from acis 2022 models for teaching digital ethics 8 discretion of its members” (tubig & mccusker, 2021, p. 150). specs has a twofold premise: first, it draws on individual researchers’ interest in engaging with the ethical aspects of their work, and second, “it appreciates the discursive nature of ethics and the transformative potential of discourse” (tubig & mccusker, 2021, p.152). 3.2 standalone ethics units versus integration with technical content the second main theme to emerge from our narrative review concerns the contrasting strategies of presenting ethics material as “stand-alone” units versus integration of digital ethics teaching within the teaching of technical content. more recent literature advocates for a more integrated or embedded model (or in some cases, a hybrid approach) as opposed to standalone modules, to ensure that ethical reasoning and skills remain relevant when students and professionals are engaged in technical tasks (e.g., skirpan, 2018; grosz, 2019; hildt et al., 2019; plemmons et al., 2020; bogina et al., 2021). 3.2.1 integration approach in their analysis of recent approaches to ai ethics education, borenstein and howard (2021) emphasised the need for computer scientists to understand ethics as intertwined with the technology future practitioners will design, create, or implement, and their responsibility as computer scientist to address ethical issues arising in their work. borenstein and howard (2021) suggest, first, that ethical design of ai algorithms should be part of learning about algorithm design generally, and should include fate (i.e., “fairness, accountability, transparency and ethics” in computing – also referred to as feat) considerations, amongst others. second, the authors recommend that real world datasets should be used to teach ethics of data acquisition. third, ethics courses in their view should not be taught as one off or standalone modules but should be repeatedly incorporated in different ways. the authors propose an “ethics across the curriculum” model which would include and reinforce the significance of ethics while teaching technical courses. they also emphasise that the course should be created and taught by an interdisciplinary team including lawyers, sociologists and philosophers together with scientists and engineers (borenstein & howard, 2021). additional examples of this integration model of teaching include a “distributed pedagogy” that incorporates short ethics modules interspersed throughout the core curriculum (grosz et al., 2019), an in situ or contextual embedded ethics course model (carter and crockett, 2019; based on huff & martin 1995), and research laboratory-based interventions (hildt et al., 2019; plemmons et al., 2020). 3.2.2 stand-alone approach however, despite this emphasis on embedding ethics teaching in other subjects, some authors recognise benefits in stand-alone ethics units. one critique of “embedding” digital ethics in technical courses, especially in the lab-based approach (hildt et al. 2019; plemmons et al., 2020), is that it can be too localised. by electing a bottom-up approach linked to practice within the lab, teachers might omit to effectively teach a broader social perspective, one that involves critical attention to the socio-ethical implications of technology. on this view, the integrated approach fails to address more holistic ethical issues relevant to is students. australasian journal of information systems paltiel et al. 2023, vol 27, selected papers from acis 2022 models for teaching digital ethics 9 3.2.3 hybrid approaches some authors preferred neither a pure integrated nor a pure standalone model but rather a hybrid model of ethics teaching. in the hybrid approach, ethics teaching is spread “across the curriculum” (borenstein & howard, 2021), but for certain reasons (including logistics around course structures) are also deployed in standalone modules. bogina et al. propose projects such as creating a “library of hands-on exercises”, integrating fate training into existing courses, and creating courses and seminars suitable for various stakeholders (including students, faculty and the public) with varying skill levels (bogina et al., 2021, p. 23). skirpan et al. (2018) discuss a short, intensive “human centred computing” course that is included as part of the overall computing training. however, they also recommend that ethics teaching should be done in “small doses” throughout the curriculum. a series of similarly relevant paradigms and projects could theoretically be adopted to address the other concepts covered in the modules, and to “embed” the rds training within the technical instruction. bates et al. (2020), who focus on integrating fate topics and social justice into data science curriculum, describe using stand-alone fate modules woven through a data science course and interspersed amongst both core and elective teaching modules. the course included a “data and society” module as a core unit, covering themes ranging from “conceptualisations of power, structure, and agency … [to] …ethical reasoning” (bates et al., 2020). 3.3 interdisciplinary models the interdisciplinary aspect of teaching computer ethics recommended by borenstein and howard (2021) is emphasised in a number of studies promoting the integration approach (see e.g., bates et al., 2020; grosz et al., 2019; skirpan et al., 2018). skirpan et al.’s integrated program incorporates guest lecturers such as a privacy lawyer, a typography and layout artist, a researcher of terms of service agreements and online harassment, and an emeritus professor who reviewed developments in technology over the past fifty years (skirpan et al., 2018). bates et al., reported on the findings of a collaborative, auto-ethnography engaged in by a multidisciplinary team involved in embedding fate/critical data studies in a data science masters program (bates et al., 2020). the authors saw “data science” as encompassing an “information science” approach rather than a strictly “computer science” approach, indicating not only technical expertise, but also social, legal, and ethical understanding. their view is that teaching cis ethics must by its nature involve more than one discipline. the interdisciplinary aspect of grosz et al.’s embedded ethics program was reported as a significant factor contributing to its success (grosz et al., 2019). here the harvard cs and philosophy departments collaborated in delivering the embedded ethics modules. the modules were codesigned by faculty from both fields and delivered by teaching assistants from the philosophy department with expertise in ethics who worked together with cs teaching staff. the authors found that the process of co-designing the ethics modules helped mitigate insecurities felt by both these faculty in teaching across disciplines. 3.4 moral theory selection and emphasis the fourth theme relates to which moral theories are taught or emphasised in digital ethics. standard teaching practice in professional ethics, from medicine to business ethics, involves teaching of moral theories (beauchamp & childress, 1994; moriarty, 2021). such theories may include consequentialism and utilitarianism, deontology and kantian ethics, ethics of care, and australasian journal of information systems paltiel et al. 2023, vol 27, selected papers from acis 2022 models for teaching digital ethics 10 virtue ethics. other theoretical approaches are possible, including but not limited to christian, confucian, jewish and buddhist ethics. amongst the digital ethics models we reviewed, some taught a “crash course” in philosophical ethics, while others referenced a theory or several theories of ethics as their framework. many scholars recommended that to deal with the complexities of ethical decisions in the digital age, students should be trained in many modes of ethical reasoning (goldsmith & burton, 2017). the three main schools of ethical reasoning referred to in the literature were utilitarianism, deontology, and virtue ethics. roughly speaking, utilitarianism is based on the principle of producing the “the greatest good for the greatest number” while deontology is a “duty-based ethics” according to which actions are ethical if they conform to a system of rules. one version of deontology, kantian ethics, includes the famous claim that rational beings ought always to be treated as ends and never merely as means. “virtue ethics” focuses on individual character and the development of good character traits or dispositions to think and feel in desirable ways. for goldsmith and burton (2017), the dominant western ethical framework is utilitarianism, which (they claim) most ai practitioners adopt. nonetheless, in their view virtue ethics and deontology have much to offer ai decision making. the authors suggest that all three forms of ethical reasoning should be taught in ethics in ai courses. jones argues that ethical issues in cs are both complex and context-specific and that it is not possible to focus on one theory, such as deontology, without regard for another values, such as the consequences of actions (jones, 2016). chatila and havens, in the ieee’s ethically aligned design (chatila & havens, 2019), also recommend that several ethical traditions be integrated into both engineering and science programs. carter and crockett’s (2019) proposed course, described above, also aligns with the recommendation that digital ethics courses should include modules dedicated to virtue ethics, utilitarianism, and deontological ethics (involving moral codes). bezuidenhout and ratti’s (2021) “micro-virtue ethics” approach specifically reflects a virtue ethics framework. bezuidenhout and ratti (2021) refer to the current focus on ethical outcomes and algorithmic design, describing utilitarian approaches that emphasise social impact, and deontological approaches focusing on principles and guidelines. yet the authors’ prefer an approach grounded in virtue theory which aims to develop “moral virtue” through practice and perfecting intellectual and moral skills, without teaching particular ethical content. their emphasis is not on general “fairness, accountability and privacy” in data science but on individual responsibility. hagendorff (2020) reviewed a series of guidelines in ai ethics and examined the extent to which the ethical principles expressed in these guidelines are implemented in ai research and application. he noted the “deep gap” between digital ethics and “concrete contexts of research, development and application” (hagendorff, 2020, p. 111). like bezuidenhout and ratti, hagendorff endorses a microethics approach. he writes that traditional ethics guidelines for cs tend to adopt a deontological approach, setting out rules and maxims; however, he thinks that a virtue ethics approach is called for. this approach does not involve defining codes of conduct but rather involves focusing on an individual’s “situation-specific deliberations”, “behavioural dispositions”, and the cultivation of “moral character” to facilitate ethical decision making in ict contexts (hagendorff, 2020, p. 112). australasian journal of information systems paltiel et al. 2023, vol 27, selected papers from acis 2022 models for teaching digital ethics 11 richards et al.’s (2020) serious game ethics training tool may also be seen as close to a virtue ethics approach with its emphasis on psychological development in moral. briggle et al. (2016) examine research and engineering ethics games founded on virtue ethics. they note the conclusion of the national research council (national research council and national academy of engineering, 2014) that the most important mechanism for ethical decisionmaking is “good judgment”, and they call for “a return to virtue ethics and its emphasis on practical moral reasoning” (national research council and national academy of engineering, 2014, in briggle et al., 2016, p. 241). 3.5 course content and pedagogical tools the fifth theme related relates to different content taught in cis ethics training courses and a variety of pedagogical tools employed, which aligned with the underlying pedagogical approach and choice of ethical theory. a number of the programs structured themselves around the study of elements of ethical computing as set out in instruments such as the montreal declaration (montréal declaration for a responsible development of artificial intelligence, 2018). the montreal declaration sets out ten principles of responsible development of ai, including, amongst others, well-being, respect for autonomy, privacy protection, democratic participation, equity, diversity and responsibility. another example of a framework teaching “elements” or “principles” of ethical computing is the impactcs project (see barnard et al., 2003) which sets out five “knowledge units” including: responsibility of the computer professional (including a review of major ethical models); basic elements of ethical analysis; basic skills of ethical analysis; basic elements of social analysis; and basic skills of social analysis. other programs reviewed were structured on the teaching of the feat principles of “fairness, ethics, accountability and transparency” (e.g. bates et al., 2020; bogina et al., 2021; lewis & stoyanovich, 2021). other models, aimed at pre-tertiary teaching, adopted variations of these principles. an example is greengard’s proposed program for kindergarten to high school ai ethics instruction, which refers to introducing notions of “fairness, transparency/explainability, trustworthiness and accountability” (greengard, 2020). the focus in this course content is on the acquisition of knowledge and understanding of ethical issues, or, when approached more deeply, the imparting of societal and moral values understood to guide ethical cis practice. in line with a social analysis and macroethics approach, the content of these programs includes formulated principles relating to “large societal issues”. in contrast, the programs focusing on development of moral “traits”, adopting a virtue ethics emphasis and a microethics approach, did not focus on teaching a prepared framework of ethical principles, but on practicing ethical reasoning and reflection (bezuidenhout & ratti, 2021; grosz et al., 2019). ethical issues in these courses were not taught as principles, but as challenges to be resolved. accordingly, the content did not comprise subject matter to be relayed through instruction, but activities and exercises undertaken in the course of cis activities, to foster ethical skills (burton et al., 2018) and develop practical ethical competence (grosz et al., 2019). these courses used a range of pedagogical tools to exercise students’ moral reasoning, ethical reflection and ethics communication skills. we describe several examples below, highlighting the pedagogical and ethical approach they reflect. australasian journal of information systems paltiel et al. 2023, vol 27, selected papers from acis 2022 models for teaching digital ethics 12 3.5.1 case studies many of the programs rely upon use of case studies which has been a commonly used pedagogical tool for ethics training for decades (see e.g. carter & crockett, 2019; hildt et al., 2019; mumford et al., 2008; newberry, 2004; towell, thompson & mcfadden 2004). some programs relied primarily on readings and reflective discussions (farooqui et al., 2021). however, case studies can be used as a trigger for role play, simulations, and interactive activities (e.g. hildt et al., 2019; towell, thompson & mcfadden, 2004). these techniques are associated with cognitivist theories of moral psychology such as kohlbergian and neokohlbergian schools described above. alternatively, they are used in conjunction with other educational tools and activities, including in situ activities such as designing and analysing a predictive algorithm (e.g., carter & crocket, 2019). most courses used dialogue and discussion to carry through study of ethics intervention to other cs units and to ensure ethical issues were intertwined with technical ones. a variation on the use of case studies and simulations is burton et al.’s (2018) use of science fiction as a tool for teaching ethics in ai. burton et al. used science fiction to allow “reframing” familiar situations and problems in unknown settings which enables leaners to recognise and approach ethical issues unencumbered by their own biases. the authors suggested that this was preferable to the use of case studies in which the dilemmas are already identified and characterised. burton et al.’s (2018) program included a “crash course” in the three main ethical theories: deontology, virtue ethics and utilitarianism. the students were then presented with select science fiction stories and given assignments requiring them to work descriptively with the three theories. 3.5.2 in situ learning and using “deliverables” bezuidenhout and ratti (2021) criticise the reliance on “high level” case studies for not sufficiently connecting “big picture” ethics to individual everyday practice. instead, they propose a range of 15 minute “ethics exercises” linked to computing modules in the course. the exercises involved mind mapping, line voting, and use of sticky notes. the aim of the exercises was to create “critical reflexivity” in individuals’ daily cs practice and to enlarge the “moral imagination”. lewis and stoyanovich (2021) sought concrete educational materials to teach cs ethics. they use “nutritional labels” (as an expression of the constructivist paradigm of “object-to-interpretwith”) to assess the interpretability of ai algorithms. a nutritional label is a visual artifact, combining text and graphics “to communicate highly technical and opaque information”. nutritional levels have been shown to increase understanding of complex topics and aid in decision-making. lewis and stoyanovich’s (2021) program required students to design nutritional labels for automated decisions systems and to communicate important information about a machine learning model and hidden algorithms, in this way promoting interpretability and transparency. 3.5.3 gaming an interesting adaptation of simulation techniques in cis ethics is the use of “serious games” such as richards et al.’s (2020) serious video game for teaching ethical conflicts in cybersecurity. rather than employing pre-scripted scenarios with predetermined solutions to ethical problems and multiple-choice responses, the game represented “complex social simulations” in which ethical problems arise and introduced “intelligent virtual agents” which australasian journal of information systems paltiel et al. 2023, vol 27, selected papers from acis 2022 models for teaching digital ethics 13 can “reason about the world”. the program used knowledge acquisition and representation techniques to distil the values underlying learners’ ethical choices within the game and the reasoning behind them (richards et al., 2020, p. 9). briggle et al. (2016) reviewed the use of games for teaching research ethics in stem fields. founding their evaluation on a virtue ethics approach, the authors concluded that the “immersive and interactive experiences” provided by the games were conducive to practicing and developing skills and virtues (briggle et al., 2016, 248). a summary of our findings is presented in table 1. (please refer to the appendix for further detail on the features of each paper). theme findings pedagogical theories (3.1) constructivism/constructionism seems to be a dominant pedagogical theory (ben-ari, 2001). social analysis contrasts with concentration on specific issues within a research lab; macroethical contrasts with microethical approaches. cognitivist/moral reasoning uses case studies to ‘put students in the shoes of a moral decision maker.’ reflexive approaches promote interdisciplinary thinking (ackerman,2001). standalone vs integration (3.2) integrating (or embedding) of ethics seem to be the preferred approach, i.e. a distributed pedagogy (grosz et al., 2019) and ‘ethics across the curriculum’ model (borenstein & howard, 2021). feat studies which involve critical data studies often are useful in a selfcontained (standalone) module. however, the hybrid approach may fare better e.g., in capstone projects. interdisciplinary models (3.3) engaging expertise of a variety disciplines was recommended (skirpan et al., 2018; borenstein & howard, 2021). cross-disciplinary co-design and co-teaching of cis ethics modules was found to be most effective (grosz et al., 2019). moral theory selection and emphasis (3.4) moral theory selection: typical moral theories taught were utilitarianism, deontology, and virtue ethics. a trend exists towards preferring virtue ethics and development of moral character in students. content emphasis and pedagogical tools (3.5) content emphasis – knowledge based and skills based: some courses were structured on knowledge of principles of ethical is practice such as fate, as frameworks for ethical analysis (bates et al., 2020; bogina et al., 2021). courses based on virtue ethics focused on training skills of ethical reasoning, discourse and communication, and discrete repetitive practice (bezuidenhout & ratti 2021). pedagogical tools: use of case studies to practice moral reasoning (carter & crockett, 2019; hildt, 2019; mumford et al., 2008; newberry, 2004; towell, thompson & mcfadden, 2004), in situ learning and deliverables (lewis and stoyanovich, 2021), gaming tools (briggle et al., 2016; richards et al., 2020). table 1. summary findings on ict ethics education – design criteria and observed trends australasian journal of information systems paltiel et al. 2023, vol 27, selected papers from acis 2022 models for teaching digital ethics 14 4 discussion in this section we discuss some implications of our findings including the diversity of pedagogical approaches, stand alone and integrated teaching models, the trend towards interdisciplinary involvement, and a range of moral theories and pedagogical tools (4.1). we then provide recommendations for further inquiry (4.2). 4.1 implications of findings our review of the literature on teaching ethics in cis courses and related fields first identified the pedagogical approaches of constructivism/constructionism, social analysis, microethics/macroethics, cognitivism or moral reasoning, and reflective/deliberative approaches. second, we found that digital ethics was sometimes taught as a standalone unit, sometimes integrated into technical subjects, and sometimes taught as a hybrid of both. third, we uncovered a trend towards interdisciplinary involvement both in the design of cis ethic courses, and in teaching. fourth, we found that the main moral theories that were taught or used in teaching were utilitarianism, deontology, and virtue ethics. finally, we observed several different teaching tools, such as ethical case studies, in situ learning, and gaming. in this sub-section, we discuss such findings and identify some possible connections between them. we start by considering the different emphases in the content of digital ethics courses, with some courses focussing on imparting knowledge and understanding ethical and societal issues associated with is and others focusing more on training is students to be actual moral decision-makers through repeated practice in the course of their technical work. these different foci were also associated with varied educational tools and techniques. while the pedagogical approaches, learning models and course structures adopted were very diverse, several newer studies emphasised a virtue ethics approach, with the related emphasis on teaching cs and is students to be ethical people and decision-makers in the field through discrete repetitive, in-situ ethical training. furthermore, teaching digital ethics in a more distributed and embedded fashion in ict curricula, rather than as a single, one-off unit for example, may promote the teaching not just of cognitivist or moral reasoning but also the imparting of long-lived ethical character traits and skills. this trend towards recognizing the importance of moral character in technologists may reflect the rapid changes and greater technological capabilities and increasingly pervasive scope of application of ict in society, the growing discussion of digital technologies in media and politics, and the need for computer science professionals today to be able to engage with a range of ethical issues arising in many areas of personal and public life. clearly, the literature on digital ethics education refers to a broad variety of pedagogical approaches or theories. this appears to be partly because teaching ethics in computer and information sciences involves the synthesis of several disciplines, and partly due to the fact that, despite the need for ethics education being flagged thirty years ago, researchers are still working on formulating the pedagogical basis of ethics education for tertiary ict students. as we have noted, digital ethics as a discipline is often yet to play a significant role in curricula. in addition, there are likely to be ongoing disagreements about which pedagogical approach(es) are best. there is no good reason why, however, teachers cannot draw from a range of pedagogical theories. for example, they might use constructivist approaches that build on students’ existing knowledge of a technology use while encouraging reflection on its australasian journal of information systems paltiel et al. 2023, vol 27, selected papers from acis 2022 models for teaching digital ethics 15 social implications and encouraging them to reflect on their own experiences with that technology. from the perspective of teachers who are not expert in pedagogical theory, drawing on different pedagogical approaches seems entirely reasonable. the relative newness of digital ethics education may partly explain the fact that some educators taught digital ethics as standalone units while others preferred to integrate ethics into the ict curriculum. knowing which, if any, is the more effective mode of teaching— standalone versus integrated versus hybrid—for ict students will need to be investigated with empirical studies. there was, however, a general feeling amongst scholars that implementing “ethics across the curriculum” (borenstein & howard, 2021) could be beneficial for learning. this could be partly due to the nature of the students being taught, namely, ict students, rather than, say, humanities students, who may have more specialised interests in ethical and social questions. some authors pointed out that teaching digital ethics to ict students in dedicated philosophy departments may be relatively ineffective (see e.g., carter & crockett, 2019). in such cases, the ethics content is separated from the details and activities connected with ict. in contrast, embedding ethics into ict courses may allow students to more clearly see the relevance of ethics and to connect it more substantially to their interest and future work. such an approach is aligned with the pedagogical approaches of constructivism and constructionism, whereby assimilating new knowledge and understanding builds on existing knowledge and understanding and is formed and shaped in particular contexts. it is also well aligned with reflexive and deliberative pedagogical approaches, in which learners reflect on and question their practices and engage in deliberation and discussion about their values concerning technology. making ethics relevant to the technological interests and activities of students is clearly going to be important for engaging students in the journey of ethics learning as well as imparting understanding. while it may be true that most ict students do not have a primary interest in ethics and social issues, it is worth pointing out that there may be an increasing appetite for such learning. a mounting hunger for this knowledge and insight may be due to developments we have been emphasizing, namely the substantial and growing media, cultural, political, and social interest in the responsible use of digital technologies. in our own teaching experience, students from is and cs are enrolling in burgeoning numbers in digital ethics subjects. moreover, more-andmore new digital ethics courses are springing up in universities and as online options, potentially reflecting a greater desire for learning about digital ethics. who should teach digital ethics to is and cs students? the benefits of bringing ethics teaching closer to the interests, values, and activities of ict students does not necessarily imply that such teaching should be done by educators whose expertise is in ict, though that may also be beneficial. if it is important to teach moral theory, as some scholars believe, then it may be necessary to recruit educators with expertise in ethics and philosophy. one obvious remedy here is to have digital ethics taught by experts in both ict and ethics. this was illustrated in courses including guest lecturers from other disciplines (skirpan et al., 2018) or partnering philosophy and computer ethics departments in designing and delivering computing ethics modules (grosz et al., 2019). indeed, borenstein and howard go a step further and suggest that digital ethics could be taught by a larger interdisciplinary team including lawyers, sociologists, and philosophers together with scientists and engineers (borenstein & howard, australasian journal of information systems paltiel et al. 2023, vol 27, selected papers from acis 2022 models for teaching digital ethics 16 2021). such experts could deliver content separately, or else collaborate in a multi-disciplinary way, to craft course materials that may be more appealing and relevant to ict students. the selection of moral theories in digital ethics teaching warrants further comment. earlier we mentioned the preference some have for teaching virtue ethics over other moral theories. in contrast to theories like utilitarianism and deontology, virtue ethics stresses the importance of developing stable characters that involve the right kind of action, feeling, and attitude (crisp & slote, 1997). however, while it may be important to society as well as to engineering disciplines to encourage good character development in future ict engineers, it would arguably be lop-sided to teach only virtue ethics theory to students. other moral theories, certainly utilitarianism and deontology but also care ethics, are equally well recognized in moral philosophy and many kinds of applied ethics. these theories also challenge aspects of virtue ethics; teaching them alongside that theory would provide for a more balanced understanding of how to approach digital ethical problems. furthermore, one may ask whether there could be some pedagogical benefits to widening the scope to include other moral approaches. this could include, for example, introducing students to eastern ethics, eco-feminism, and/or other less traditional frameworks (singer, 2013). there are also various indigenous ethics approaches in different countries. consider, for example, the approach of indigenous australians to the effects of technology on country (graham, 1999). the choice of such moral frameworks may be determined by the nature of the student body as well as educators’ expertise. for example, students from some asian backgrounds may already have some knowledge of (say) buddhist and confucian ethics, while students from some african backgrounds may have knowledge of ethical approaches such as ubuntu. one could easily imagine that moral approaches which mesh well with students’ cultural backgrounds will be especially useful in stimulating learning about digital ethics, and may also align well with, for example, constructivist/constructionist, social analysis, and reflexive/deliberative pedagogical models preferred by some educators. 4.2 questions for further inquiry our literature review indicates significant scope for further inquiry. we note here two questions arising from our findings. the first of these regards cis ethics education for professionals who have completed their university training and are working in the field. while the issue was raised in two of the studies reviewed (hildt et al., 2019; plemmons et al., 2020), both these projects involved student researchers working within university laboratories. our review reveals a relative gap in understanding of the need and nature of digital ethics education outside of the university environment. in the work by plemmons et al. (2020) a randomised trial of research ethics training aimed at making ethics discourse routine practice in university stem labs was conducted. the authors suggest that a lack of communication about ethical considerations in these environments resulted from researchers’ equating ethics with regulatory compliance, and that this flowed from a disconnect between responsible conduct of research (rcr) training and the reality of the laboratory. the authors used a customised training script to provide opportunities for members to discuss present and future relevant behaviour. the preand postprogram surveys indicated that following the intervention, researchers were more engaged in discussions about the ethical implications of their work. as stated above, however, the scope of the ethical training, similarly to rcr principles, related to practice within the laboratory, and did not extend to broader ethical implications of the technology being produced by the australasian journal of information systems paltiel et al. 2023, vol 27, selected papers from acis 2022 models for teaching digital ethics 17 work. in this sense, as in the case of hildt et al. (2019), the breadth of the focus would be insufficient to address the ethical issues relevant to cis professionals working in industry or students planning to move out of the university environment. both studies proposed programs for stem students and researchers (who in their roles act both as students and educators), and not specifically cis students. it may be that in cis, the broader social and ethical implications of the technology demand a broader approach to cis ethics. in such a case, ethics training focused specifically on ethical conduct within a university laboratory, while important, is limited. thus, there is potential to consider an example of teaching beyond universities. hildt et al. (2019) proposed an intervention which moved “out of the classroom” to integrate ethics education into a laboratory work environment. the authors emphasised a “bottom up” approach of working with ethical issues arising in the workplace, rather than using instructorpresented dilemmas. interestingly, the authors noted a distinction between the broader “bottom up” meaning of ethics in the research context expressed by student participants focusing on communication, power imbalances and social issues and the narrower “top down” focus held by faculty – focusing on issues of plagiarism, data quality and code of conduct requirements. this finding raises the question of whether ethics training may need to be tailored differently for professionals in different roles and different stages of their career. a second question is whether there is a need for specifically designed ethics training programs for students and professionals designing and analysing cs technology employed in particular fields. in recent years, a number of studies have addressed the need for training in different professions using cs technology, not just in technological proficiency, but in the knowledge and skills required to address ethical issues arising with the use of the technology in that field. for example, in the field of medicine, scholars have recently examined the need for specifically teaching “health ai” or “medical ai” ethics, along with the required technical skill, in medical education courses (see civaner et al., 2022; katznelson & gerke, 2021; quinn & coghlan, 2021; weidener & fischer, 2023). similarly, studies have considered the need to train teachers in the uses and ethical implications of digitalised teaching tools (buchanan, 2019) and digital journalism (garcía-avilés, 2014), and the need to train social workers in the ethical issues inherent in use of online technologies used in their profession (goldingay & boddy, 2016). these studies, however, consider training in dealing with ethical considerations arising when employing technology designed for use in particular professions. the question we raise here is whether specialised ethics training is needed in educating cis students and professionals involved in the design of medical, teaching, journalistic, social work technologies. as many professions become increasingly reliant on specialized ai tools, this question gains in significance. 5 conclusion in our narrative literature review, we have progressed the understanding of ethics in cis education, by identifying several pedagogical approaches to teaching digital ethics, differences in the degrees to which ethics components are integrated into ict courses, various moral theories in ethics teaching practices, and various pedagogical tools. when designing digital ethics teaching for is students and other students in ict fields, educators might profitably reflect on these different dimensions of ethics teaching and the interconnections between them that we discussed above. it is likely good teaching in digital ethics will involve australasian journal of information systems paltiel et al. 2023, vol 27, selected papers from acis 2022 models for teaching digital ethics 18 using a range of theoretical techniques and teaching and learning approaches rather than just one or two. the necessity of teaching digital ethics to future ict practitioners is increasingly accepted. to improve digital ethics education, further work in this space will be important. this would include a more systematic review and analysis of the relevant literature and exploring in more depth the roles, linkages, and suitability of various pedagogical theories, levels of embeddedness of ethics material, and the philosophical nature of the subject matter in digital ethics teaching. there is also benefit in extending the scope of inquiry to also examine ongoing ethics education of ict practitioners engaged in different roles and industries, and exploring a possible need for specialised ethics training in designing ict technology for use within particular professions such as medicine, journalism, and teaching. references ackermann, e. (2001). piaget’s constructivism, papert’s constructionism: what’s the difference. http://learning.media.mit.edu/content/publications/ea.piaget%20_%20 papert.pdf ali, s., williams, r., payne, b., park, h., & breazeal, c. (2019). constructionism, ethics, and creativity: developing primary and middle school artificial intelligence education. international workshop on education in artificiation intelligence k-12 (eduai ’19), usa. https://robots.media.mit.edu/wpcontent/uploads/sites/7/2019/08/constructionism__ethics__and_creativity.pdf baethge, c., goldbeck-wood, s. & mertens, s. (2019). sanra—a scale for the quality assessment of narrative review articles. rebidwells integr peer rev, 4(5), 1–7. https://doi.org/10.1186/s41073-019-0064-8 barabas, c., doyle, c., rubinovitz, j. b., & dinakar, k. (2020). studying up: reorienting the study of algorithmic fairness around issues of power. proceedings of the 2020 conference on fairness, accountability, and transparency, fat* ’20, new york, ny, usa, 167–176. https://doi.org/10.1145/3351095.3372859 barnard, a., de ridder, c., pretorius, l., & cohen, e. (2003). integrating computer ethics into the computing curriculum: a framework for implementation. proceedings of the 2003 insite conference, 3, 265–279. https://doi.org/10.28945/2619 bates, j., cameron, d., checco, a., clough, p., hopfgartner, f., mazumdar, s., sbaffi, l., stordy, p., & de la vega de león, a. (2020). integrating fate/critical data studies into data science curricula: where are we going and how do we get there? proceedings of the 2020 conference on fairness, accountability, and transparency, fat* ’20, new york, ny, usa, 425–435. https://doi.org/10.1145/3351095.3372832 beauchamp, t. l. & childress, j. f. (1994). principles of biomedical ethics. oxford, uk: oxford university press. ben-ari, m. (2001). constructivism in computer science education. journal of computers in mathematics and science teaching, 20(1), 45–73. https://www.learntechlib.org/p/8505 bezuidenhout, l., & ratti, e. (2021). what does it mean to embed ethics in data science? an integrative approach based on microethics and virtues. ai & society, 36(3), 939–953. https://doi.org/10.1007/s00146-020-01112-w australasian journal of information systems paltiel et al. 2023, vol 27, selected papers from acis 2022 models for teaching digital ethics 19 bidwell, n. (2010). ubuntu in the network: humanness in social capital in rural africa. interactions, 17(2), 68–71. https://doi.org/10.1145/1699775.1699791 bogina, v., hartman, a., kuflik, t., & shulner-tal, a. (2021). educating software and ai stakeholders about algorithmic fairness, accountability, transparency and ethics. international journal of artificial intelligence in education, 3, 808-833. https://doi.org/10.1007/s40593-021-00248-0 borenstein, j., & howard, a. (2021). emerging challenges in ai and the need for ai ethics education. ai and ethics, 1(1), 61–65. https://doi.org/10.1007/s43681-020-00002-7 bradley, a. (2008). accreditation criteria guidelines. no. g02, engineers australia accreditation board. https://www.engineersaustralia.org.au/sites/default/files/contentfiles/2016-12/g02_accreditation_criteria_guidelines.pdf briggle, a., holbrook, j. b., oppong, j., hoffmann, j., larsen, e. k., & pluscht, p. (2016). research ethics education in the stem disciplines: the promises and challenges of a gaming approach. science and engineering ethics, 22(1), 237–250. https://doi.org/10.1007/s11948-015-9624-6 buchanan, r. (2019). digital ethical dilemmas in teaching. in peters, m.a. (ed.), encyclopedia of teacher education. singapore: springer nature singapore. http://link.springer.com/10.1007/978-981-13-1179-6_150-1 burton, e., goldsmith, j., & mattei, n. (2018) how to teach computer ethics through science fiction.’ communications of the acm, 61(8), 54-84. bynum, t. (2015). computer and information ethics. in zalta, e. n. (ed.), the stanford encyclopedia of philosophy (summer 2018 edition). https://stanford.library.sydney.edu.au/archives/sum2019/entries/ethics-computer/ califf, m. e., & goodwin, m. (2005). effective incorporation of ethics into courses that focus on programming. proceedings of the 36th sigcse technical symposium on computer science education sigcse ’05. new york, ny, usa, 347–351. https://doi.org/10.1145/1047344.1047464 carter, l., & crockett, c. (2019). an ethics curriculum for cs with flexibility and continuity. 2019 ieee frontiers in education conference (fie). ieee, cincinnati, oh, usa, october 1619, 2019. https://doi.org/10.1109/fie43999.2019.9028356 chatila, r., & havens, j. c. (2019). the ieee global initiative on ethics of autonomous and intelligent systems. robotics and well-being. international series on intelligent systems, control and automation: science and engineering, 95, 11–16. https://doi.org/10.1007/978-3-030-12524-0_2 civaner m.m., uncu, y., bulut, f., giounous chalil, e., & tatli, a. (2022). artificial intelligence in medical education: a cross-sectional needs assessment. bmc medical education, 22(1), 772–780. crisp, r., & slote, m. a. (eds.) (1997). virtue ethics. oxford: oxford university press. eubanks, v. (2018). automating inequality: how high-tech tools profile, police, and punish the poor new york, ny, usa: st. martin’s publishing group. australasian journal of information systems paltiel et al. 2023, vol 27, selected papers from acis 2022 models for teaching digital ethics 20 farooqui, j., dawod, s., & sarma, d. (2021). towards a neuroethical ethos: a case study in reframing neuroethics education for engineers and researchers. the neuroethics blog, emory center for ethics. accessed january 24, 2022. http://www.theneuroethicsblog.com/2021/02/towards-neuroethical-ethos-casestudy.html floridi, l., cath, c., & taddeo, m. (2019). digital ethics: its nature and scope. the 2018 yearbook of the digital ethics lab, 9–17. https://doi.org/10.1007/978-3-030-17152-0_2 garcía-avilés, j.a. (2014). online newsrooms as communities of practice: exploring digital journalists’ applied ethics. journal of mass media ethics, 29(4), 258-272. goldingay, s. & boddy, j. (2017). preparing social work graduates for digital practice: ethical pedagogies for effective learning. australian social work, 70(2), 209–220. goldsmith, j., & burton, e. (2017). why teaching ethics to ai practitioners is important. proceedings of the 31st aaai conference on artificial intelligence, 4836-4840. https://doi.org/10.1609/aaai.v31i1.11139 graham, m. (1999). some thoughts about the philosophical underpinnings of aboriginal worldviews. worldviews: environment, culture, religion, 3(2), 105–118. https://doi.org/10.1163/156853599x00090 greenhalgh t., thorne s., & malterud k (2018). time to challenge the spurious hierarchy of systematic over narrative reviews? european journal of clinical investigation, 48(6). https://doi: 10.1111/eci.12931 grosz, b. j., grant, d. g., vredenburgh, k., behrends, j., hu, l., simmons, a., & waldo, j. (2019). embedded ethics. communications of the acm, 62(8), 54–61. https://doi.org/10.1145/3330794 hagendorff, t.(2020). the ethics of ai ethics: an evaluation of guidelines. minds and machines, 30(1), 99–120. https://doi.org/10.1007/s11023-020-09517-8 hämäläinen, j. (2012). pedagogic theories. oxford bibliographies online datasets. oxford, uk: oxford university press (oup). https://doi.org/10.1093/obo/9780199791231-0015 hassan, n. r., mingers, j., & stahl, b. (2018). philosophy and information systems: where are we and where should we go. european journal of information systems: an official journal of the operational research society, 27(3), 263–277. https://doi.org/10.1080/0960085x.2018.1470776 herkert, j. (2004). microethics, macroethics, and professional engineering societies. emerging technologies and ethical issues in engineering: papers from a workshop. national academy of engineering. washington, dc, usa: the national academies press. https://doi.org/10.17226/11083 hildt, e., laas, k., miller, c., taylor, s., & brey, e. m. (2019). empowering graduate students to address ethics in research environments. cambridge quarterly of healthcare ethics: cq: the international journal of healthcare ethics committees, 28(3), 542-550. https://doi.org/10.1017/s096318011900046x huff, c., & martin, c. d. (1995). computing consequences. communications of the acm, 38(12), 75–84. https://doi.org/10.1145/219663.219687 australasian journal of information systems paltiel et al. 2023, vol 27, selected papers from acis 2022 models for teaching digital ethics 21 johnson, d. g. (1985). computer ethics, englewood cliffs, nj, usa: prentice-hall. jones, s. (2016). doing the right thing: computer ethics pedagogy revisited. journal of information communication and ethics in society, 14(1), 33–48. https://doi.org/10.1108/jices07-2014-0033 katznelson, g. & gerke, s. (2021). the need for health ai ethics in medical school education. advances in health sciences education, 26(4), 1447–1458. kohlberg, l. (1984). the psychology of moral development: the nature and validity of moral stages. new york, ny, usa: harper & row. kordzadeh, n. & ghasemaghaei, m. (2022). algorithmic bias: review, synthesis, and future research directions. european journal of information systems, 31(3), 388–409. https://doi.org/10.1080/0960085x.2021.1927212 lewis, a., & stoyanovich, j. (2021). teaching responsible data science: charting new pedagogical territory. international journal of artificial intelligence in education, 32, 783807. https://doi.org/10.1007/s40593-021-00241-7 li, j., & fu, s. (2012). a systematic approach to engineering ethics education. science and engineering ethics, 18(2), 339–349. https://doi.org/10.1007/s11948-010-9249-8 martin, d. a., conlon, e., & bowe, b. (2021). a multi-level review of engineering ethics education: towards a socio-technical orientation of engineering education for ethics. science and engineering ethics, 27(5), article 60. https://doi.org/10.1007/s11948-021-003336. mason, r. o. (1986). four ethical issues of the information age. mis quarterly, 10(1), 5–12. https://doi.org/10.2307/248873 montréal declaration for a responsible development of artificial intelligence (2018). https://montrealdeclaration-responsibleai.com/ moor, j. h. (1985). what is computer ethics? metaphilosophy, 16(4), 266–275. https://doi.org/10.1111/j.1467-9973.1985.tb00173.x moriarty, j. (2021). business ethics. in zalta, e. n. (ed.), the stanford encyclopedia of philosophy. https://plato.stanford.edu/archives/fall2021/entries/ethics-business/ müller, v. c. (2020). ethics of artificial intelligence and robotics. in zalta, e. n. (ed.), the stanford encyclopedia of philosophy. https://plato.stanford.edu/archives/fall2020/entries/ethics-ai/ mumford, e. (1995). effective systems design and requirements analysis: the ethics approach. london, uk: macmillan. mumford, m. d., connelly, s., brown, r. p., murphy, s. t., hill, j. h., antes, a. l., waples, e. p., & devenport, l. d. (2008). a sensemaking approach to ethics training for scientists: preliminary evidence of training effectiveness. ethics & behavior, 18(4), 315-339. https://doi.org/10.1080/10508420802487815 national research council and national academy of engineering. 2014. emerging and readily available technologies and national security: a framework for addressing ethical, legal, and societal issues. washington, dc, usa: the national academies press. https://doi.org/10.17226/18512 australasian journal of information systems paltiel et al. 2023, vol 27, selected papers from acis 2022 models for teaching digital ethics 22 newberry, b. (2004). the dilemma of ethics in engineering education. science and engineering ethics, 10(2), 343–351. https://doi.org/10.1007/s11948-004-0030-8 o’neil, c. (2016). weapons of math destruction: how big data increases inequality and threatens democracy. new york, ny, usa: broadway books. papert, s. (1993). mindstorms, london, uk: basic books. papert, s., & harel, i. (1991). situating constructionism. in papert, s., & harel, i. (eds.), constructionism. new york, ny, usa: ablex publishing corporation. piaget, j. (1971). psychology and epistemology: towards a theory of knowledge. london, uk: penguin books. plemmons, d. k., baranski, e. n., harp, k., lo, d. d., soderberg, c. k., errington, t. m., nosek, b. a., & esterling, k. m. (2020). a randomized trial of a lab-embedded discourse intervention to improve research ethics. proceedings of the national academy of sciences of the united states of america, 117(3), 1389–1394. https://doi.org/10.1073/pnas.1917848117 quinn, t.p. & coghlan, s. (2021) readying medical students for medical ai: the need to embed ai ethics education (no arxiv:2109.02866, arxiv, 7 september 2021). http://arxiv.org/abs/2109.02866 rest, j. r., darcia, n. ez, thoma, s. j., bebeau, m. j.. (1999). postconventional moral thinking: a neo-kohlbergian approach. milton park, uk: psychology press (routledge). richards, d., formosa, p., ryan, m., hitchens, m., & mcewan, m. (2020). a proposed aienhanced serious game for cybersecurity ethics training. proceedings of the 9th conference of the australian institute of computer ethics (aice 2020), 1–9. https://researchers.mq.edu.au/en/publications/a-proposed-ai-enhanced-serious-gamefor-cybersecurity-ethics-trai singer, p. (ed.). (2013). a companion to ethics (1st ed.), blackwell companions to philosophy. london, uk: blackwell. skirpan, m., beard, n., bhaduri, s., fiesler, c., & yeh, t. (2018). ethics education in context. proceedings of the 49th acm technical symposium on computer science education, new york, ny, usa, 940–945. https://doi.org/10.1145/3159450.3159573 someh, i., davern, m., breidbach, c. f., & shanks, g. (2019). ethical issues in big data analytics: a stakeholder perspective. communications of the association for information systems, 44, article 34, 718–747. https://doi.org/10.17705/1cais.04434 sprague, a. p., & diaz-sprague, r. (2019). tying ethics to teamwork training in a minimodule. proceedings of the 8th computer science education research conference, cserc ’19, new york, ny, usa, 120–122. https://doi.org/10.1145/3375258.3375275 towell, e., thompson, j. b., & mcfadden, k. l. (2004). introducing and developing professional standards in the information systems curriculum. ethics and information technology 6(4), 291–299. https://doi.org/10.1007/s10676-005-5985-x tubig, p., & mccusker, d. (2021). fostering the trustworthiness of researchers: specs and the role of ethical reflexivity in novel neurotechnology research. research ethics, 17(2), 143-161. https://doi.org/10.1177/1747016120952500 australasian journal of information systems paltiel et al. 2023, vol 27, selected papers from acis 2022 models for teaching digital ethics 23 tucker, a. b. (1991). computing curricula 1991. communications of the acm, 34(6), 68–84. https://doi.org/10.1145/103701.103710. véliz, c. (ed.). (2021). the oxford handbook of digital ethics, oxford, uk: oxford university press. https://doi.org/10.1093/oxfordhb/9780198857815.001.0001 weidener, l. & fischer, m. (2023). artificial intelligence teaching as part of medical education: qualitative analysis of expert interviews. jmir medical education, 9, article e46428. doi: 10.2196/46428 wong, p. h. (2012). dao, harmony and personhood: towards a confucian ethics of technology. philosophy & technology, 25, 6786. appendix table 2. an overview table of relevant papers discussed, and key themes/ideas discussed within legend cons: constructivist/constructionist approaches soc: social analysis approaches m/m: microethics/macroethics approaches (including evaluation of tradeoffs between the two) cog: cognitivism/moral reasoning r/d: reflexive/deliberative approaches ec/feat: ethical computing elements/fairness ethics accountability and transparency tsrd: ethical traits, skills, reasoning and discourse case: case studies deliv: deliverables/concrete educational materials for in situ learning gam: gaming/gamification approaches **: asterisks in "integration/hybrid approaches" indicates an evaluation of tradeoffs with the 'standalone approaches' technique note that the table is non-exhaustive; papers may overlap in their coverage. australasian journal of information systems paltiel et al. 2023, vol 27, selected papers from acis 2022 models for teaching digital ethics 24 australasian journal of information systems paltiel et al. 2023, vol 27, selected papers from acis 2022 models for teaching digital ethics 25 australasian journal of information systems paltiel et al. 2023, vol 27, selected papers from acis 2022 models for teaching digital ethics 26 copyright: © 2023 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v27i0.4517 http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 search methodology to obtain a broad overview of the relevant literature, we conducted a narrative review, ‘a scholarly summary along with interpretation and critique’ (greenhalgh et al., 2018, pp. 2-3) to identify papers about digital ethics teaching for students in is... 3 findings 3.1 pedagogical approaches 3.1.1 constructivist/constructionist approaches 3.1.2 social analysis approaches 3.1.3 approaches emphasising microethics versus macroethics 3.1.4 cognitivist or moral reasoning approaches 3.1.5 reflexive and deliberative approaches 3.2 standalone ethics units versus integration with technical content 3.2.1 integration approach 3.2.2 stand-alone approach 3.2.3 hybrid approaches 3.3 interdisciplinary models 3.4 moral theory selection and emphasis 3.5 course content and pedagogical tools 3.5.1 case studies 3.5.2 in situ learning and using “deliverables” 3.5.3 gaming 4 discussion 4.1 implications of findings 4.2 questions for further inquiry 5 conclusion paper title (paper title style) australasian journal of information systems bagheri, ridley & williams 2023, vol 27, research article organisational cyber resilience 1 organisational cyber resilience: management perspectives saba bagheri university of tasmania, australia seyedehsaba.bagheri@utas.edu.au gail ridley university of tasmania, australia belinda williams university of tasmania, australia abstract as cyberthreats pose strategic risk, both it and business management awareness are critical for effective organisational decision making. many cyber system failures arise from organisational, and not technical issues. this study investigates senior manager awareness of organisational cyber resilience, using case study method. the cyber resilience matrix is used as a theoretical framework to communicate the multifaceted meaning of cyber resilience. this study examines whether the multilayered nature of cyber resilience is understood by both managerial levels to include the periods before and after cyber incidents. as the higher education sector faces complex cyber challenges, research data were gathered from two australian universities. analysis found the two management groups differed in their resilience approach. the authors posit that principles-based cyber policies contribute to an organisational view of cyber resilience. the engineering resilience approach, accompanied by a non-bureaucratic organisational structure, was preferred by it managers. business managers favoured an ecological approach with a vertical organisational structure. both managerial groups emphasised the period before cyber crisis when compared to after cyber incidents. this research contributes to the limited theoretical development in the field and attempts to shift the focus from cyber security to cyber resilience. keywords: organisational cyber resilience, universities, senior it managers, business managers, cyberthreat. 1 introduction while organisations depend on cyber activities (ahmad et al., 2015; cavelty, 2007), the cyber environment is frequented by cyber criminals (conklin & kohnke, 2018) and attempted intrusion from unfriendly nations (abc news, 2020). cyberthreats are now universal and affect all organisations. although the most common approach to combat cyber threat is to use cyber security solutions (vugrin & turgeon, 2013), cyber security procedures are inadequate to deal with information security crises (tisdale, 2016; bagheri, 2020). to better prepare organisations for cyber threats, cyber resilience must be used in combination with cyber security (trim et al., 2009). cyber resilience is the capacity of a cyber-system to perform effectively, regardless of the hazards (vugrin & turgeon, 2013). in contrast, cyber security is usually defined by the number of procedures that attempt to protect systems from different types of cyberthreats. in complex organisational environments, cyber solutions need to go beyond cyber security and australasian journal of information systems bagheri, ridley & williams 2023, vol 27, research article organisational cyber resilience 2 incorporate resilience (colombo, 2020; stouffer et al., 2011; bagheri, 2020). the many approaches to achieve cyber resilience (roege et al., 2017) reveal that cyber resilience is multidisciplinary, and extends beyond technical issues to include behavioural and organisational aspects (bernabe & skarmeta, 2019; sabev, 2020; tisdale, 2016). scholarly research into cyber resilience has emphasised technical and operational aspects. relatively few studies have considered the organisational elements of cyber resilience (bagheri & ridley, 2017; sepúlveda-estay et al., 2020). as cyber issues threaten business objectives, management awareness of organisational cyber resilience is essential for appropriate decision making (soomro et al., 2016; bagheri, 2020). cyber problems cannot be solved only by technical decision makers. business managers should be also included, as well as it managers (orozco et al., 2015). despite the important role of business managers in the cyber security decision making process, studies indicate that business managers do not demonstrate appropriate knowledge of cyber security and resilience, leading to conflict between organisational cyber goals and business objectives (johnson, 2009; tisdale, 2016). to deal with this problem, a common understanding of cyber security and resilience is needed among senior it and business managers from different functional roles (johnson, 2009). although the importance of managerial awareness of cyber security is discussed by researchers (moallem, 2020), the issue of shared understanding by managerial groups has received limited attention in the cyber resilience literature. this study aims to investigate senior it and business managers’ shared awareness of cyber resilience, with a focus on organisational factors. as both roles are key for effective decision making for cyber crisis, alignment of the perspective of the two managerial groups will strengthen organisational cyber resilience. a subsidiary aim of this study is to examine if the multilayered meaning of cyber resilience is understood by both managerial levels to include the periods both before and after cyber incidents. this study also seeks to understand whether the vertical or horizontal organisational structure is favoured by these two managerial groups. a further secondary goal is to investigate whether engineering or ecological resilience thinking is preferable for organisational cyber resilience decision making, as each resilience approach is likely to lead to different strategic decisions being taken by managers. for the purposes of this research, organisational factors refer to a range of features and characteristics of an organisation that influence the workplace, including organisational structure, policies, social relationships and communication, decision-making processes, employee knowledge and skills, cultural issues and other elements (goodman & haisley 2007). the current study focuses on organisational cyber resilience and excludes cyber security and technical aspects of cyber resilience development. the scope of this investigation is limited to cyber resilience in two large organisations with complex cyber environments. the extreme cyber challenges faced in the higher education sector led to the decision to conduct the investigation within two australian universities. this research is motivated by a lack of research into organisational aspects of cyber resilience, the acknowledged tension between senior it and business managers in understanding the role of cyber resilience and, in particular, the potential impact of poor strategic decision making for cyber crisis on organisations (johnson, 2009; white, 2009). it focuses on the key role of both it and business managers in developing a cyber resilient organisation, rather than the singular focus of the senior it managerial role. consequently, this research seeks to contribute to australasian journal of information systems bagheri, ridley & williams 2023, vol 27, research article organisational cyber resilience 3 shifting the focus from cyber security to cyber resilience, as well as highlighting the multidisciplinary nature of cyber resilience by examining its organisational aspects. the literature on organisational cyber resilience is reviewed in the second section of this paper, with a brief discussion on the different perspectives of senior it and business managers. the study’s methodology appears in the third section, followed by the results and discussion in the fourth and fifth sections, respectively. finally, the conclusions are presented in the sixth section, with limitations and recommendations for future research. 2 a review of organisational cyber resilience the limited number of scholarly publications on organisational cyber resilience demonstrates that academic research into organisational cyber resilience is still at an early stage (björck et al., 2015). the concepts of resilience and cyber resilience first need to be differentiated, as well as the distinction between cyber resilience and cyber security approaches. forms of resilience as resilience refers to the capacity of systems and organisations to absorb changes during adverse situations (annarelli et al., 2020; segovia et al.,2020), it is a primary enabler to cope with, adapt to, and recover from disturbance (béné et al., 2014; hausken, 2020). resilience in an organisational context is applied in two ways—engineering and ecological—each of which results in a different approach to deal with threats. engineering resilience refers to stability and the fast recovery of an organisation or system to its original condition (holling, 1996; sikula et al., 2015). ecological resilience implies an organisational ability to adjust to a new situation (berkes et al., 2008; noureddine, 2020) through change, and focuses on learning from the crisis (bellini & marrone, 2020; sikula et al., 2015). to the best of the authors’ knowledge, no scholarly research has investigated whether engineering or ecological resilience thinking dominates cyber resilient organisations. as each resilience approach is likely to lead to different strategic decisions being taken by managers, the authors were motivated to identify the type of resilience thinking that is preferable for organisational cyber resilience decision making. cyber resilience versus cyber security cyber resilience is a new paradigm which emphasises the business capability of an organisation to survive a cyberthreat (bei, 2019; buchmann et al., 2020). it encompasses “anticipation, support, recovery and adaptation” (yano et al., 2015, p. 2) in a changing environment (linkov et al., 2013a). in contrast, cyber security is explained as a set of policies, strategies and programs to defend cyber systems (craigen et al., 2014), and imposes rules to restrict data access and minimise information risk (antikainen, 2014). while the ultimate goal of both cyber security and cyber resilience approaches is to protect organisations against cyberthreats, researchers believe that resiliency should be added to cyber security studies (le & hoang, 2017). cyber resilience thinking assists organisations to prepare for, and develop the ability to recover from, cyber risks and hazards (annarelli et al., 2020; roege et al., 2017), and is the capacity of a cyber-system to perform effectively regardless of the hazards in the business environment (vugrin & turgeon, 2013). therefore, a cyber-resilient system not only focuses on providing protection against cybercrimes, but also considers the threat response australasian journal of information systems bagheri, ridley & williams 2023, vol 27, research article organisational cyber resilience 4 before, during and after an incident (roege et al. 2017). the breadth of response aligns with linkov et al.'s (2013b) definition of cyber resilience that includes the planning, absorption, recovery and adaptation stages. organisational cyber resilience this study is limited to the organisational aspects of cyber resilience because of the latter’s multifaceted nature (buchmann et al., 2020) and the limited research into those dimensions compared to the technical aspects. among the few academic studies available on organisational cyber resilience, the cyber resilience matrix includes organisational aspects of cyber resilience (linkov et al., 2013b). the framework maps four organisational domains (i.e. physical, information, cognitive, social) from the network centric warfare (ncw) (alberts et al., 2000), to four stages of event management (i.e. plan/prepare, absorb, recover, adapt) in cyber resilience adapted from the national academy of sciences (nas) (national academy of sciences, 2012). while the matrix includes cyber resilience metrics to identify an organisation’s capability to deal with cyber crises, it does not aim to identify the organisational factors that contribute to cyber resilience. more explanation about this matrix is provided in the next section. shapiro et al. (2016) of the world economic forum extended the linkov et al. (2013b) framework by amalgamating it with the national institute of standards and technology’s (nist) framework for improving critical infrastructure cybersecurity, which has since been updated. the shapiro et al.’s framework is complex and includes a fifth event management stage, detection with the authors acknowledging data collection limitations. it is also noted the framework was not designed to identify organisational factors for cyber resilience. other studies have identified factors for organisational cyber resilience in a fragmented way. for example, the sarkar et al. (2013) conceptual model of the information systems resilience of smalland medium-sized enterprises (smes) referred to business continuity and recovery planning, the leadership role, cultural issues, and external organisational elements. although the 2013 study examined internal and external elements of cyber resilience and emphasised the decision-making process, it was limited to smes and did not consider the larger organisational context. other studies have developed organisational evaluation tools and best practices for cyber resilience. to illustrate, hult and sivanesan (2014) proposed a cyber resilience checklist with questions for self-assessment, ferdinand (2015) designed a knowledge-based cyber resilience structure, while lykou et al. (2018) proposed cyber resilience best practices for smart airport systems. the evaluation tools and best practice approaches provided by these studies were designed to help organisations better understand their cyber resilience. however, it is important to note the aim of the tools was not to identify the organisational factors of cyber resilience. other researchers considered the recovery phase (e.g. conklin & shoemaker, 2017; appiah et al., 2020) without examining the remaining cyber resilience event management stages of planning, adaptation and absorption (linkov et al., 2013b). however, no frameworks were identified in these and other academic studies for examining the organisational factors of cyber resilience. other organisational cyber resilience studies from the limited literature include gisladottir et al. (2016) who examined the selection of an appropriate number of rules for cyber resilience development. while their study focused on the absorption and recovery phases of resilience australasian journal of information systems bagheri, ridley & williams 2023, vol 27, research article organisational cyber resilience 5 with an emphasis on organisational regulations, less emphasis was given to other aspects of cyber resilience (e.g. post incident). another study examined engineering resilience models and human behaviour to develop a framework that combined four resilience functions (anticipate, monitor, respond and learn) with three characteristics of behaviour (capability, opportunity and motivation) (van der kleij & leukfeldt, 2019). in a further publication that investigated cyber resilience development in financial corporations, researchers identified three main approaches to promote cyber resilience, namely: 1) cyber resilience enhancement as the cyber security future, 2) inclusion of cyber resilience into cyber security standards bodies, and 3) cyber resilience development through creating compliance tools (dupont, 2019). while the last two studies contributed to a better understanding of non-technical aspects of cyber resilience, the managerial role did not receive attention in either. a recent study by kott and linkov (2021) sought to measure cyber resilience, suggesting use of a measurement process utilised in material science and physics. the authors believed that additional criteria relevant to cyber systems should be developed, including repeatability, consistency and monotonicity. they recommended use of a large volume of data to increase the confidence level in measuring cyber resilience (kott & linkov, 2021). again, the kott and linkov (2021) study neither considered any organisational aspect of cyber resilience nor any managerial aspects. annarelli et al. (2020) developed a managerial cyber resilience framework using six case study organisations. the proposed framework listed the managerial actions required in the (single) planning/ preparing phase of event management. although the study interviewed several key informants from case studies to understand managerial actions, it did not investigate a senior managerial perspective on organisational cyber resilience. in another study, loonam et al. (2020) developed a cyber security strategy framework (cssf) to be used by managers for effective cyber resilience strategy development. the cssf framework suggested that managers should focus on governance and transformational support (security culture). however, while the cssf highlighted the important role of senior managers in cyber resilience development by proposing managerial strategies for adoption, it did not investigate their perspectives on organisational cyber resilience. from the review above, it can be seen that while some non-technical aspects of cyber resilience appear in the scholarly literature, no structured framework of the organisational factors of cyber resilience was found. the frameworks, models, and studies of organisationally-oriented cyber resilience examined in the literature review are collated in table 1, and categorised by their orientation. of the studies identified, the cyber resilience matrix (linkov et al., 2013b) was the most comprehensive in characterising the meaning of cyber resilience. the cyber resilience matrix (hereafter referred to as the matrix) (linkov et al., 2013b) includes activities that occur before (i.e., planning) and during/after (i.e., absorption, recovery, and adaptation) cyber crisis across four cyber event management stages. other researchers examined only one or two stages of event management for managing cyber crisis (e.g. conklin & shoemaker, 2017 and roege et al., 2017). however, all stages of event management need to be considered. a cyber-resilient organisation not only focuses on the period before the cyber threat (planning stage of cyber security handling), but also on the threat response during and after (absorption, recovery, and adaptation) an incident (roege et al., 2017). the matrix is examined in more detail in the following section. australasian journal of information systems bagheri, ridley & williams 2023, vol 27, research article organisational cyber resilience 6 framework/ study name major components author/ year measuring cyber resilience discusses the importance of cyber resilience measurement through well-developed measurement process (kott & linkov, 2021) managerial cyber resilience framework managerial actions required in the planning/ preparing phase of event management. (annarelli et al., 2020) cyber security strategy framework (cssf) focuses on governance and transformational support (security culture) (loonam et al., 2020) cyber resilience in financial corporations three main approaches to promote cyber resilience (dupont, 2019) integrated model from engineering resilience & human behaviour combines four resilience functions (anticipate/ monitor/ respond/ learn) & three sources of behaviour (capability/ opportunity/ motivation) ( van der kleij & leukfeldt, 2019) cyber resilience best practices for smart airport systems three groups of best practices for technical/ organisational/ policies & standards (lykou et al., 2018) seven principles for cyber resilience seven principles to establish cyber resilience (classify/ risk/ rank/ design & deploy/ test/ recover/ evolve) (conklin & shoemaker, 2017) an algorithm for recovery and response activities framework of three main activities (standard maintenance activities/ supporting activities/ emergency response and recovery activities) (roege et al., 2017) cyber resilience and over & under regulation resilience is a function of the number of rules in an organisation (gisladottir et al., 2016) a framework for assessing cyber resilience framework that combined linkov et al. (2013b) and the national institute of standards and technology’s (nist) framework for improving critical infrastructure cybersecurity. (shapiro et al., 2016) knowledge-based structure for cyber resilience five stages of cyber resilience: (non-existent / immature /established /reactive /fully proactive & reactive cyber resilience) (ferdinand 2015) cyber resilience checklist checklist of nine cyber resilience items (hult & sivanesan, 2014) cyber resilience matrix matrix that relates areas of disaster resilience (ncw) with steps of event management (nas) (linkov et al., 2013b) information systems (is) resilience conceptual framework consists of external & internal elements for is resilience in smes (sarkar et al., 2013) table 1. organisational cyber resilience frameworks and studies, arranged in year order 2.1 cyber resilience matrix the matrix used nas’s stages of event management for cyber resilience in the horizontal axis, with ncw’s four domains of an organisation or system in the vertical axis. the four stages of cyber event management are: • plan/ prepare: maintain service accessibility during disruptions; • absorb: operational availability; australasian journal of information systems bagheri, ridley & williams 2023, vol 27, research article organisational cyber resilience 7 • recover: rebuild resource functionalities, and • adapt: learn from incidents, and update procedures where required (feist, 2006; hambleton et al., 2000; linkov et al., 2013b). the four organisational domains are: • physical: physical sources; • information: information relevant to the physical layer; • cognitive: decision-making procedures, and • social: organisational relationships (linkov et al., 2013b). each cell of the matrix includes metrics that aim to evaluate the system capability to manage a cyber crisis. the matrix appears in table 2, with example metrics to illustrate what each cell refers to. the research on organisational cyber resilience is fragmented, with a limited theoretical foundation. in their review of the literature on organisational aspects of cyber resilience, bagheri and ridley (2017) found the matrix perspective was one of three approaches taken to examine the topic. while no scholarly framework of cyber resilience’s organisational factors and strategies was identified by the authors, the cyber resilience matrix (linkov et al., 2013b) is more comprehensive than other frameworks, with the exception of the complex shapiro et al.’s (2016) framework. linkov et al.’s (2013b) matrix was refined by shapiro et al. (2016), to assess industry and sector cyber resilience. that framework combined linkov et al.’s (2013b) framework with the national institute of standards and technology’s (nist) framework for improving critical infrastructure cybersecurity. shapiro et al’s (2016) refined framework was not used for this study due to its complexity. the authors discussed their difficulty of obtaining data to satisfy the framework’s criteria in the framework. also, the cyber resilience matrix linkov et al’s (2013b) is well established in the literature while the shapiro et al’s (2016) framework is complex and yet to be fully considered by the literature. in an extensive review of resilience, linkov and trump (2019) referred to the “resilience matrix” in chapter six, and illustrated it in chapter eight. another review of different approaches to measure the cyber resilience of autonomous agents discussed the matrix as a qualitative approach (ligo et al., 2021). linkov and associates’ application of the cyber resilience matrix to many diverse contexts was found in the literature. the following studies illustrate this range. in a literature review of how critical infrastructure resilience was modelled when encountering compounding or cascading threats, wells et al. (2022) used the “resilience matrix” in their assessment. linkov and co-authors used the matrix to investigate the resilience of smart water systems (marchese et al., 2019), community resilience assessment (fox-lent et al., 2015) and population displacement arising from disaster (rand et al., 2019). it can be seen that the literature on the matrix is narrow, being largely derived from linkov from the us army engineer research and development center, and his associates. in this study the multifaceted meaning of cyber resilience captured by linkov et al. (2013b) will be used to analyse managers’ understanding of organisational cyber resilience. the matrix australasian journal of information systems bagheri, ridley & williams 2023, vol 27, research article organisational cyber resilience 8 will also provide the research with a theoretical guide in a new field that lacks research interconnections. cyber event management stages plan/prepare for absorb recover from adapt to o rg an is at io na l d om ai ns physical maintain controls for cyber resources/create physical assets/evaluate systems interconnections use redundant services to continue operation/ reduce cyberthreats resolve any system problem/assess cyber risk/evaluate recovery time/change irrecoverable items evaluate the system’s configuration/ allocate new assets information classify information resources/ prepare documents for it suppliers/store valuable information/iden tify internal or external relationships identify the sensors/transfer information to relevant parties store log data for databases/ recover after an incident keep records of time taken between the incident, recovery and post-recovery solution cognitive plan for cyber incidents/unders tand business objectives/organi se cyberwar gaming apply a protocol for decisionmaking/evaluate achievements/id entify important resources/devel op appropriate policies identify any breakdown points/ make decisions during recovery review management response for a better decision/ discover the reasons behind the cyber attack social communicate with external parties/provide training programs/delega te services to particular employees/impr ove resilience communication among employees/foster a culture of cyber resilience identify & communicate with cyber resilience professionals follow cyber resilience guidelines/determ ine responsible parties for cyber resilience evaluate employees’ response to cyber incidents/allocate human resources/remain up-to-date about cyber risks table 2. cyber resilience matrix (linkov et al., 2013b) the next section briefly reviews the literature that examines differences between business and it management groups. australasian journal of information systems bagheri, ridley & williams 2023, vol 27, research article organisational cyber resilience 9 2.2 differences between it managers and business managers previous research has investigated the importance of the senior managerial role for cyber resilience in organisations. senior manager support is critical to developing cyber resilience (bernard, 2007; mcfadzean et al., 2007), as effective cyber resilience strategy starts at the top organisational level (dutta & mccrohan, 2002). however, senior technical managers are not solely responsible for cyber resilience; business managers are also accountable (selby, 2017). strategic approaches to cyber security and resilience must consider broad organisational solutions in addition to the technical aspects (chapman et al., 2018), as the socio-technical approach best fosters a security culture. the current research addresses a tension between the acknowledged criticality of engaging both technical and business managers in organisational cyber decision making, with a lack of academic research that examines the awareness of the topic by business managers. a review of the literature on it and business managers’ perspectives of cyber resilience suggests that the two managerial groups have different views on the organisational factors that contribute to cyber resilience (cobo et al., 2014; fitzgerald, 2007; rainer jr et al., 2007). for example, in the it-business alignment literature, business managers were found not to have a comprehensive understanding of it issues (bergeron et al., 2004; luftman et al., 1999). if the perceptions of it and business managers are visualised as a continuum, business managers are located at the business-focused end, unlike it managers (rainer jr et al., 2007). as research on cyber security investment found business managers believed that allocating cyber security funds was not a business enabler, further discussions between business and it managerial groups are required to align their knowledge and perspectives to benefit the organisation (johnson, 2009). teo and king (1997) argued that discrepant views between managers are rooted in their different positions and job responsibilities. researchers have called for it managers to help business managers enhance their understanding of it issues (al-surmi et al., 2020; armstrong & sambamurthy, 1999; wang et al., 2012), aiming for better it resource allocation and strategic cyber security decision making within organisations (tallon, 2014). divergent perspectives on cyber resilience by it and business managers pose a risk for organisational decision-making, including insufficient provision of funding and other resources. a lack of awareness of organisational cyber resilience among it and business managers may also lead to inconsistent decisions or inadequate actions to combat cyber crisis. however, the issue has received relatively little attention in the research literature. this study will investigate the organisational contributors of cyber resilience, and compare it and business managers’ awareness of organisational cyber resilience. 3 methodology this research used a case study methodology approach in two australian universities to investigate senior management awareness of cyber resilience. the case study method provides an organisational setting that allows detailed knowledge to be gathered about a topic (berg, 2004; merriam, 2009). large and complex business environments are favoured by cyber attackers because of sensitive personal and financial information stored in their databases. universities are large organisations with challenging cyber environments that are exposed to numerous cyber attacks and significant cyber incidents. when university staff and students connect personal devices to network resources (wagstaff & sottile, 2015), they create an open environment for data exchange. consequently, universities are challenged to establish full australasian journal of information systems bagheri, ridley & williams 2023, vol 27, research article organisational cyber resilience 10 control over information flows and cyber issues. as the authors reside in australia, they have been exposed to media reports of cyber infiltration in australian universities (borys, 2019). for instance, the top ranked australian national university (quacquarelli symonds, 2018) experienced a serious cyberattack in 2017, and still worked to reduce the impact of that attack a year later (austin, 2018). australian universities have been described as weak when dealing with cyber problems and cyber security, and as a result, they have worked to enhance their cyber resilience strategies to deal with cyber crisis (austin, 2018). the cyber challenges faced by australian universities motivated the authors to investigate cyber remediation strategies in university case studies to identify the organisational contributors to cyber resilience development in large complex organisations. differences between australian universities prompted the researchers to broaden the scope of the study beyond a single case study to two case studies. diversities in the universities chosen were designed to allow the researcher to observe any common patterns while also providing contrasting data for the study. unique access to the two universities and their willingness to participate in the current research project also influenced the researcher to select the particular universities as case study settings. with just 43 universities in australia, only limited information is provided in table 3 to maintain the universities’ anonymity. location total students total staff members number of campuses case study 1 (c1u) australian state a less than 40000 less than 5000 less than 5 case study 2 (c2u) australian state b more than 40000 more than 5000 more than 5 table 3. broad characteristics of the two university case studies a semi-structured interview technique was employed for data collection after obtaining ethics approval. a pilot study assessed the appropriateness of the developed interview questions after review by three researchers and revision of the questions. nine open-ended questions were posed during the interviews, with the specific questions being tailored for each managerial group based around common themes (table 4). the questions were designed to examine cyber resilience decision making, important cyber policies, required cyber skills and other organisational contributors to cyber resilience from the management perspective: interview question themes decision making process during a cyber crisis policies and strategic actions for cyber resilience important knowledge and skills for cyber resilience organisational contributors to cyber resilience suggestions to improve cyber resilience table 4. main interview question themes seventeen senior managers from the two australian universities were interviewed; nine were it managers and eight came from business. appendix a provides demographic information about the interviewees. the interviews were conducted between september 2017 and august australasian journal of information systems bagheri, ridley & williams 2023, vol 27, research article organisational cyber resilience 11 2018, and ranged from 15 to 55 minutes each. they were audio recorded and transcribed before analysis using the automated textual analysis software program, leximancer. leximancer identifies word synonyms and frequency in text passages as concepts, before visualising their interconnectedness in themes. the software allows patterns to be observed that may not be apparent from manual coding, while the validity and reliability of leximancer analysis has been evaluated and confirmed (smith & humphreys, 2006). analysis was carried out at the theme level, where each theme represented an organisational factor of cyber resilience. two analyses were undertaken: 1) in the first analysis, the organisational cyber resilience awareness of senior it and business managers was compared after merging data from the two case studies. to allow comparison, the identified organisational factors for cyber resilience were categorised into the matrix (linkov et al., 2013b) cells to determine how organisational cyber resilience was understood by participant groups, as is explained next. to classify each cyber resilience organisational factor identified into cell(s) of the matrix, the matrix metrics were first investigated to determine the meaning that best matched each factor. to illustrate, one organisational factor identified from analysis referred to the importance of cyber resilience training programs for employees. this factor aligned with the metric, provide staff members with cyber resilience training programs, located at the intersection of the plan/prepare column and the social row of the matrix (linkov et al., 2013b). consequently, this factor was allocated to the plan/prepare for social cell. in some cases, a single organisational factor could be matched to more than one metric of the matrix. in these instances, the organisational factor of cyber resilience was allocated to all relevant cells. in contrast, where organisational factors of cyber resilience could not be matched to any of the cell metrics, the items were ignored. 2) the second analysis compared the it manager and the business manager data for each case study. this analysis aimed to identify similarities and differences between the it and business manager groups within each case study. for a deeper analysis, the cyber security/resilience policies from both case studies were also investigated to identify any potential differences between the universities. the contents of the policies were first reviewed and then compared with the organisational factors and strategies for cyber resilience identified from the primary data. cyber security policies in both case studies helped the researcher to understand their directives for cyber security management. 4 results the results of the merged cases data analysis for senior it and business managers are reported in the next four sections. findings from each management group are also categorised into the cyber resilience matrix cells. 4.1 merged cases datasenior it manager analysis the primary concept themes obtained from analysis of the merged cases interview text for senior it managers appear in the leximancer concept map in figure 1 below with the same themes ranked on the right. based on leximancer hit counts, the most highly ranked themes are ‘tools’, ‘communication’, ‘external’ and ‘policies’. transcripts associated with each theme presented in figure 1 were examined for their meaning. the organisational factors of cyber resilience identified from these themes, with australasian journal of information systems bagheri, ridley & williams 2023, vol 27, research article organisational cyber resilience 12 example participant transcript passages, appear in table 5. each identified organisational factor was codified in the column at the right, where it refers to it senior managers and f indicates an organisational factor. figure 1. concept map and ranked themes for organisational factors of cyber resilience—it senior managers (merged case data) organisational factors identified in table 5 were then categorised into the relevant matrix cells. corresponding codes for the organisational factors were noted in a bracket beside the relevant matrix metric, as seen in table 6. cells without a metric matched to an organisational factor or strategy are shaded. theme example it senior managers’ transcript excerpt organisational factor code tools i think there needs to be training [for cyber resilience]. every organisation should have some mandatory training, i’m actually talking about how to stay safe (interviewee 14). importance of cyber resilience training programs for employees itf1 we try to be proactive, we do take a lot of steps [actions] from the ethical hacking that goes on in here. we guide them [ethical hackers] to watch systems that we think have the most exposure to the external world, like financial systems, customer information accounts, passwords. we should think from the perspective of hackers (interviewee 6). enhance proactivity (e.g. ethical hacking) itf2 communication we tend to delegate a lot of decision-making and trying to do things down to team members [sic]. the reason we do that is if something does happen, we don’t have to wait an hour, or two hours or three hours for the cio [chief information officer] to make a decision about what needs to be done … so, we tend to expect … the frontline staff or the employees … [will] make decisions and act quickly (interviewee 17). make decisions during recovery with delegating decisions to skilled employees itf3 [after cyber crisis] i like to recover the core systems as quickly as possible. that would be where i would put my efforts. (interviewee 8). fast recovery of core systems (after cyber crisis) itf4 external it can go often to recover systems, but we need to know who is responsible for cyber resilience. we also need to receive advice from other organisations. so, we need to know who are the key stakeholders of other organisations (interviewee 7). enhance cyber resilience through communication with both responsible parties and other external organisations itf5 good technical products … appropriate tools … good knowledge of markets of the vendors and … understanding of a) use modern cyber security tools and itf6 australasian journal of information systems bagheri, ridley & williams 2023, vol 27, research article organisational cyber resilience 13 … technical tools are the most important [organisational factors] for cyber resilience. (interviewee 15). technologies to support organisational systems external it can go often to recover systems, but we need to know who is responsible for cyber resilience. we also need to receive advice from other organisations. so, we need to know who are the key stakeholders of other organisations (interviewee 7). enhance cyber resilience through communication with both responsible parties and other external organisations itf5 good technical products … appropriate tools … good knowledge of markets of the vendors and … understanding of … technical tools are the most important [organisational factors] for cyber resilience. (interviewee 15). a) use modern cyber security tools and technologies to support organisational systems itf6 policies i’ve seen an example [of a complicated cyber security policy] when i was in my last job at …university of … … i’m sure no one read them…! you need to have short and simple policies… i think they should be easy to read, sliced and as short as possible, so it communicates the business objective (interviewee 9). aim for ‘simplified’ cyber security policies to understand business objectives itf7 we need to have policies as part of our plan for cyberthreats. but i reckon as few [cyber security] policies as possible. if … [they] get too long, i wouldn’t read them… and i know my guys [staff members] wouldn’t read those policies. they would never have time to read it … they would never remember them (interviewee 13). b) have a limited number of cyber security policies for cyberthreat planning itf8 table 5. organisational factors of cyber resilience—it senior managers (merged case data). note: an explanation of the process of developing the organisational factors and their link to the matrix appears in section 3 (methodology). plan/prepare for absorb recover from adapt to physical create physical assets [itf6] information recover after an incident [itf4] cognitive plan for cyber incidents [itf8]/understand business objectives [itf7]/organise cyberwar gaming [itf2] make decisions during recovery [itf3] social communicate with external parties [itf5]/provide staff members with cyber resilience training [itf1]/delegate services to particular employees [itf3] identify & communicate with cyber resilience professionals [itf5] determine responsible parties for cyber resilience [itf5] table 6. organisational factors of cyber resilience located into the cyber resilience matrix—senior it managers (merged case data) analysis results for the combined cases business manager data are reported next. 4.2 merged cases data—business managers eight themes were identified in an analysis of the business managers’ data. the leximancer concept map labelled with themes is displayed in figure 2, while the ranked themes appear at the right. australasian journal of information systems bagheri, ridley & williams 2023, vol 27, research article organisational cyber resilience 14 as this study seeks to identify the main differences between the grouped senior it and business managers’ views of organisational cyber resilience, only the highest ranked themes from the concept map were selected (see the right of figure 2). again, the theme ranks were identified based on hit numbers, so that the theme with the highest hit count indicates the most important theme in the concept map. as seen in figure 2, ‘awareness’, ‘team’, ‘manager’ and ‘organisation’ were the most highly ranked themes. to understand the meaning of each theme, associated transcripts were reviewed. the organisational factors of cyber resilience identified from these themes, with example participant transcript passages, appear in table 7. each organisational factor was codified in the column at the right, where b refers to business managers and f indicates an organisational factor of cyber resilience (e.g. bf1 refers to the first organisational factor identified by business managers). figure 2. concept map with ranked themes for organisational factors of cyber resilience—business managers (merged case data) ranked theme example business manager transcript excerpts organisational factor code awareness to me, one issue is making sure that people are aware or should be aware [of cyber resilience]. so, it’s always around, trying to make sure that more people are aware of these things [cyber resilience issues] (interviewee 3). raise cyber resilience awareness among employees bf1 they [it teams] can help unpack technically what that resolves, but effectively, there is an element of responsibility that sits [at]… executive levels. so, if you take my role [as a business manager], if i didn’t take that issue [cyber resilience] seriously, there will be no resourcing for that … it always requires an understanding and communication [between it department and business managers] for better [cyber] security (interviewee 10). while cyber security experts are needed in the it department, also take advice from, & communicate with, business managers bf2 team if you’ve got an organisation where employees are not satisfied with their jobs, cyber resilience is probably going to be one of the last issues on their mind… (interviewee 2). [i]f you have unhappy employees, they could potentially be the ones that impact [negatively] on cyber resilience (interviewee 1). employee satisfaction bf3 australasian journal of information systems bagheri, ridley & williams 2023, vol 27, research article organisational cyber resilience 15 manager senior managers need to make decisions [during cyber crisis] and others should follow their advice. so, there should be a structured decision making system when a senior manager is responsible for taking decisions (interviewee 5). so, yes, it is senior managers through the [cyber security] crisis that make the decisions, … they [senior managers] should take final decisions (interviewee 11). having a structured decision making system with the senior managers as decision makers bf4 i suppose that it is where they stop hackers get[ting] in. it is it employees where they’ve got control over the systems to stop [it] happening. for example, in my team, we’ve got four system accountants. if they see … strange issues, they report it [to it department]. so, we’ve got it that is looking after cyber security, say all the information … we need to just keep an eye. so, if something looks strange, we need to report it straight away (interviewee 12). communicate with appropriate cyber security experts bf5 organisation another issue is that you need to close down any future [cyber security] risk … so, you do things in a way that sometimes might be slower than you want, but you do it in a way that you don’t have further compromised systems. so, you reduce any further exposure that you might have to data breaches (interviewee 4). mitigate potential cyber security risk bf6 table 7. organisational factors of cyber resilience—business managers (merged case data) the organisational factors identified in table 7 were next categorised into the most appropriate cells of the matrix. a corresponding code for each organisational factor was recorded in a bracket beside the relevant matrix metric, as shown in table 8. plan/prepare for absorb recover from adapt to physical information plan for sensitive information [bf6] cognitive apply a protocol for decisionmaking [bf4] social provide staff members with cyber resilience training programs [bf1]/improve resilience communication [bf2] identify and communicate with cyber resilience professionals [bf5] table 8. organisational factors of cyber resilience positioned in cyber resilience matrix—business managers (merged cases data) the next two sections present results for each senior management role after separate analysis of the interview data for each case study. 4.3 case study one— senior it versus business managers the cyber resilience organisational factors identified from the c1u senior business manager data were similar to those for the same group from the merged data (see table 7). however, analysis of the senior it manager data from c1u revealed an additional cyber resilience organisational factor when compared with the same group for the merged data. the newly identified factor is presented in table 9, with an example interview passage. australasian journal of information systems bagheri, ridley & williams 2023, vol 27, research article organisational cyber resilience 16 example senior it manager transcript excerpt—c1u additional organisational factor we want to invest [by] … looking at our tool sets, and again i suppose when you are saying about unknown threats, it’s really ensuring we keep our tool sets up-to-date and revisit and throw them out and get some new ones in (interviewee 7). importance of technical tools table 9. additional identified organisational factor—it managers, c1u the results of data analysis for the senior it and business managers from c2u are presented next. 4.4 case study two— it senior versus business managers analysis of the senior it and business manager data at c2u found no difference between the business managers’ results and those from the merged case data for the same group (see table 7). a difference was found in the results for the c2u senior it managers, as three new organisational factors emerged. these organisational factors are seen in table 10 with example transcript passages. example senior it managers’ transcript excerpts—c2u additional organisational factor we are very targeted with our [cyber resilience] advice rather than keeping it generic. [for example] … we use stories that we tell to employees and they remember stories, and that has an impact on their behaviour. we provide them with practical advice, specific and particular experience. (interviewee 16). raise awareness among employees through practical advice i think it needs more communication between these two groups [senior managers and employees] to understand what happens, and why, and how to address it (interviewee 13). communicate with employees i think employees follow cyber security rules more when a cyber crisis happens. for example, if the cryptolocker virus was going around, a lot of employees were more aware and more careful because of the few incidents that happened. so, if they are concerned about a crisis, they might be more careful about security (interviewee 15). create concern about cyber security breaches among employees table 10. additional identified organisational factors—senior it managers, case study two the two analyses to investigate senior it and business managers’ awareness of organisational cyber resilience found differences between these managerial groups. the discussion section elaborates on the differences found. 5 discussion 5.1 merged case data— senior it versus business managers the results of the senior business manager merged case analysis indicated that this group favoured a more vertical and bureaucratic organisational structure (see bf4 in table 7) compared to the senior it managers (see itf3 in table 5). senior business managers placed less emphasis on operational staff for decision making for cyber crisis. however, senior it managers emphasised the employee role in decision-making, favouring a more horizontal organisational structure. past research has suggested that when senior managers seek advice from it employees for fast decision-making during cyber crisis, their organisations benefit (davis et al., 2016; hult & sivanesan, 2014). the divergent results obtained for the senior it and business manager groups from the merged case data signals that senior it managers were australasian journal of information systems bagheri, ridley & williams 2023, vol 27, research article organisational cyber resilience 17 more aware of the key role of it employees in cyber resilience development, a finding that is consistent with previous research (linkov et al., 2013b). senior business managers had a more non-technical view of organisational cyber resilience when compared to senior it managers. it managers typically have specialist it technical training. while senior it managers emphasised use of modern cyber security tools and technologies to support organisational systems (see itf6 in table 5), the technological aspects of cyber resilience did not receive attention from senior business managers (see table 7). these differences derive from the groups’ different responsibilities, as business management roles cover more organisational and non-technological issues. senior it managers also emphasised the important role of employee training (see itf1 in table 5) in addition to the technological development of cyber resilience (see itf6 in table 5). carias et al. (2018) concluded that investment in the technical elements of cyber security alone contributes to short-term cyber resilience improvement, while cyber resilience training enhances long-term cyber resilience development. that senior it managers in the current study valued both technological tools and cyber security training for organisations suggests that they were aware of the importance of long-term cyber resilience development, as well as the needs for short-term cyber resilience improvement. however, greater communication between the two managerial groups may bring the business management perspective closer to the technical view (rainer jr et al. 2007). while senior it managers argued for fast recovery after cyber crisis (see itf4 from table 5) the current study also found that senior business managers favoured cyber security risk mitigation without emphasising fast recovery (see bf6 from table 7). reflection upon these two perspectives, suggests that business managers adopt an ecological approach to cyber resilience by placing less importance on the length of the recovery period after cyber crisis (see table 7). this new finding contributes to the body of knowledge by suggesting how the response of both management groups differed. our finding offers a pathway to strengthen cyber resilience, by combining both approaches. as discussed earlier, minimising recovery time is important in engineering resilience, while fast recovery has less emphasis in ecological resilience thinking. instead, ecological resilience stresses learning from adverse events (sikula et al., 2015). the engineering resilience thinking of senior it managers is important as an organisation must quickly recover from an adverse event. otherwise, the organisation may not survive in a complex and competitive business environment, in which provision of services to customers, and continuing the core business function are fundamental. however, attending to fast recovery while ignoring learning from past incidents may result in the same adverse events in the future. learning from past cyber incidents is aligned with the ecological resilience view of business managers. differences in the ecological and engineering resilience perspectives of senior business and it managers may contribute to the lack of shared cyber resilience understanding identified. cyber resilience is a business-it issue (sharma, 2015; tisdale, 2016), one that requires both perspectives. consequently, utilising both the it manager (engineering resilience) and the business manager (ecological resilience) perspectives for cyber resilience development is likely to advantage organisations. this finding from the study is reinforced by sikula et al.’s (2015) recommendation that in ideal resilience systems, both ecological end engineering resilience approaches need consideration. it is possible that enabling both approaches to organisational resilience to inform responses to cyber attack may contribute to an organisation being more adaptive. australasian journal of information systems bagheri, ridley & williams 2023, vol 27, research article organisational cyber resilience 18 as seen in table 6 and table 8, the cross-sectional cell, plan/prepare x social, had the highest number of organisational factors identified by both senior it and business managers. this finding suggests that planning for the social domain was the most important cyber resilience issue for both management groups. factors highlighted in the social domain include communication with external parties, improving resilience communication, providing staff members with cyber resilience training and delegating services to particular employees. while senior it managers gave similar attention to the cross-sectional cell, plan/prepare x cognitive (see table 6), the business managers’ results did not emphasise the cognitive domain with its decision-making function (see table 8). this finding suggests that while planning for both the social and cognitive domains was paramount for senior it managers, decision-making (cognitive layer) procedures for cyber resilience improvement were unimportant for business managers. this distinction between the two management groups is likely to be associated with their different views on decision-making procedures during cyber crisis. communicating with employees and delegating decisions when handling cyber security problems were practices favoured more by senior it managers than business managers (see table 5). both managerial groups were found to emphasise the planning phase when compared to the other event management stages of cyber resilience. however, as the definition of cyber resilience includes the period both before and after cyber crisis (dewar, 2017; yano et al., 2015), underestimating the contribution of the other phases may restrict cyber resilience development in organisations (hausken, 2020). limiting the attention paid to the nonplanning phases reduces the capability of organisations to respond to cyber threats because the activities for absorption, recovery and adaptation stages occur after a cyber security incident is identified. further, establishing cyber resilience policies takes place both before and after cyber crisis, since these policies are updated across the post-incident stage. analysis of the senior business managers’ data revealed that the group ignored the period after cyber crisis, as the recovery and adaptation stages (see table 6 and table 8) received no attention. however, senior it managers did emphasise the recovery stage of the matrix for cyber resilience, but not the adaptation stage. these findings reinforce the dominance of engineering resilience thinking by senior it managers, as noted in earlier reporting of the results. fast “recovery” of systems after a cyber crisis is emphasised in the engineering approach as a strategy of cyber resilience (sikula et al., 2015). another difference between the two management groups was observed in the physical layer of the matrix. an example of factors located in the physical layer is maintaining controls for cyber physical resources and assets (e.g. network structure and system components) (table 2). while senior business managers gave no attention to the physical domain (table 8), senior it managers did (table 6). although the focus of this research is on organisational cyber resilience, the physical aspects of cyber resilience should not be neglected. all four organisational domains need attention from managers (linkov et al., 2013b). the new findings identified in this study regarding the two senior manager groups’ focus and/or inattention on particular event stages and organisational domains of the matrix presented above, contribute to the body of knowledge. the results of this research may help explain how the actions and viewpoints of senior it and business managers differ and are similar, and point to strategies for bringing the two perspectives together to strengthen organisational cyber resilience. australasian journal of information systems bagheri, ridley & williams 2023, vol 27, research article organisational cyber resilience 19 despite the matrix’s aim to help organisations assess their system capacity for cyber incidents, this research found that some organisational and behavioural factors that influence cyber resilience were omitted from the matrix. employee satisfaction and organisational structure were identified as cyber resilience organisational factors from the interviewee data. however, these factors were not matched to any of the cell metrics. the omissions suggest that the matrix is incomplete and may benefit from extension to include additional non-technological aspects of cyber resilience. the results from analysis of the separate case data are discussed next. 5.2 separate case data— senior it versus business managers as stated in the results section, no difference was found between the senior business manager results within each separate case study, and those of the merged data for the same position. however, differences were found through analysis of the senior it managers’ results, where c1u it managers followed a more technical approach when compared with the same group at c2u (table 9). c2u senior it managers adopted broader organisational perspectives on cyber resilience (table 10). to better understand the difference between these two groups, the cyber security/resilience policies from both case studies were investigated. the interview transcripts were also reviewed to explain these differences. c2u had conducted awareness campaigns with staff and students about cyber security, while similar approaches were not discussed in the c1u interviews. the c2u campaign produced and displayed posters describing recent cyber-attacks to increase cyber security awareness. one c2u interviewee explained: ‘we have … posters for cyber security. it’s all about security issues … (to) reinforce … (to) employees to (help them) understand these issues. i think this can develop a common sense of what is security. (w)e use stories that we tell to employees and they remember stories … that … (have) an impact on their behaviour’ (interviewee 17). the campaign appeared to shape senior it managers’ thinking in c2u by focusing on the broader human and organisational aspects of cyber resilience, in addition to the technical issues. another issue that may help account for the differences found between the senior it manager groups in the two case studies was that two senior it managers at c2u held responsibility for less technical areas, such as assisting users during a cyber crisis and translating business needs to it solutions. one senior it manager from c2u explained: ‘most of my team is involved in security from a consultancy perspective, helping users with security problems, so when they feel threatened, we all … (are) the first respondents. we should help them to understand that their behaviours …(pose) the most risks, rather than their password being breached or anything like that.’ (interviewee 15). in contrast, the job descriptions of c1u senior it managers did not convey a similar broad role. the broadened responsibilities seen in c2u appeared to enrich senior it managers’ understanding of the organisational, non-technical issues of cyber resilience. differences in the cyber security policies at each university were found from both the policy review and the interview transcript analysis. one of the senior it managers at c2u stated: ‘we have a high level … master policy here rather than too … (many) policies’ (interviewee 13). another it manager interviewee from c2u commented: ‘i think we are focusing on security advice rather than policies. we achieve cyber security through advice and education rather than (by) creating policies’ (interviewee 16). c2u had fewer cyber security policies compared to c1u despite its larger size. the former university believed that more policies would not be beneficial. the cyber security policies at c2u were general and organisational cyber security guidelines more than technical policies. however, the cyber security policies at c1u were australasian journal of information systems bagheri, ridley & williams 2023, vol 27, research article organisational cyber resilience 20 detailed and technologically-based, in addition to being more numerous. c2u it management believed that a limited number of high level, principles-based, cyber security policies made it easier for managers and users to remember important issues. broader policies imply a principles-based approach, where organisational policies are developed using high-level guidelines. in a rules-based approach, detailed prescriptions are provided to stakeholders on how to behave (burgemeestre et al., 2009). 6 conclusion this study sought to address the lack of scholarly research into organisational cyber resilience by investigating perceptions held by senior managers from large organisations. while major effort has been invested by previous scholars in developing organisational cyber security, this study sought to shift research focus from cyber security to cyber resilience, and emphasised the multidisciplinary nature of the latter. limited prior related academic research has used an established cyber resilience framework to study organisational cyber resilience, contributing to restricted theoretical development in this young field. this research used the multifaceted meaning of cyber resilience proposed by linkov et al. (2013b) in which the periods both before and after the cyber crisis also require management. the research applied an accepted cyber resilience model, the cyber resilience matrix (linkov et al., 2013b), as a theoretical foundation to identify differences between it and business managers’ perspectives on organisational cyber resilience at two australian university case studies. seventeen senior it and business managers were interviewed, along with analysis of each university’s cyber security policies. from analysis of the organisational factors identified in this study placed into the cyber resilience matrix, this study found that the stage related to before cyber crisis (i.e. planning) received more attention when compared to the stages after cyber incidents (i.e. absorption, recovery and adaptation phases). consistent with a definition of cyber resilience which covers before, during and after cyber security incidents (linkov et al., 2013b), cyber-resilient organisations need to consider all three stages of cyber security to be able to handle cyber threats. this finding points to the value for senior it and business managers in placing more emphasis on the during and after cyber security crisis stages to strengthen cyber resilience. from analysis, the authors recommend extending the cyber resilience matrix (linkov et al., 2013b) to include further behavioural and organisational elements (employee satisfaction and organisational structure), to reflect a more holistic view of organisational cyber resilience. the results highlight the value of having a limited number of principles-based cyber security policies covering both technical and organisational aspects. the findings of this study also indicate that senior business managers favoured a more vertical and bureaucratic organisational structure compared to the senior it managers by placing less emphasis on operational staff for decision making for cyber crisis. however, seeking advice from it employees for fast decision-making during cyber crisis was suggested by past studies, as noted in the literature review. the findings of this study reinforce the inclusion of both ecological and engineering resilience approaches in management decision making. this research appears to be one of the first to investigate the resilience approach in organisational cyber resilience. rather than focusing only on the senior it manager role, this research investigated the role of business managers in strengthening organisational cyber resilience, as both managerial levels make decisions during cyber crisis. this research is among the first to consider the perspective australasian journal of information systems bagheri, ridley & williams 2023, vol 27, research article organisational cyber resilience 21 of business managers on cyber resilience development. the findings of this study demonstrated that the two senior management groups in the case studies do not share a common view on organisational cyber resilience. business managers placed more emphasis on non-technical aspect of cyber resilience, while senior it managers focused more on technical aspects. as a shared understanding of cyber resilience will benefit better strategic decision-making, the results of this research may assist managers and policymakers to be better informed about their key roles in developing a cyber-resilient organisation. as explained, greater communication between the two managerial levels may bring the business management perspective closer to the technical view (rainer jr et al., 2007). increasing joint decision-making meetings between these two managerial levels, and providing practical examples of how cyber security tools and cyber resilience improvement can prevent adversaries from compromising organisational systems, may contribute to enhanced organisational cyber resilience. these actions may assist both business and it managers to understand the role of each in building cyber resilience, resulting in benefit for organisations through protecting reputations and achieving strategic objectives. this study points to a range of practical implications of the findings, in addition to extending the organisational and behavioural factors of the matrix that influence cyber resilience. these implications include increasing organisational awareness of the criticality of involving both senior it and business managers in all phases of the cyber resilience process. such awareness will increasingly facilitate communication between, and training of, both groups to learn from past adverse events through joint review, and adaption to future events. this process may occur through the implementation of a restricted number of organisational policies that apply to the absorption, recovery and adaptation stages after a cyber incident, and not only in the planning stage. as the case studies were restricted to two australian universities, caution is needed before generalising the findings to other universities, large organisations or nations. this research offers a number of opportunities for further research, including comparative studies of senior it and business manager perspectives of organisational cyber resilience in other nations and organisations including small and large, private and public and regional and metropolitan universities. this research identified that senior it managers and business managers have different views on ecological and engineering resilience approaches. another suggestion for future research is to investigate whether these diverse perspectives assist organisations to be more adaptive during an attack. a further avenue for research is to investigate the association between the type of resilience thinking that dominates an organisation, and whether the approach enables greater resilience. references abc news. (2020). scott morrison's 'urgent' hacking warning shot shows australia won't shy away from china's cyber attacks. retrieved from https://www.abc.net.au/news/2020-0620/why-australia-acted-on-china-hacking-cyber-attack-scott-morrison/12376700 ahmad, a., johnson, c., & storer, t. (2015). an investigation on organisation cyber resilience international journal of computer, electrical, automation, control and information engineering, 9, 1661-1666. retrieved from http://www.waset.org/publications/10002012 al-surmi, a., cao, g., & duan, y. (2020). the impact of aligning business, it, and marketing strategies on firm performance. industrial marketing management, 84, 39-49. australasian journal of information systems bagheri, ridley & williams 2023, vol 27, research article organisational cyber resilience 22 alberts, d. s., garstka, j. j., & stein, f. p. (2000). network centric warfare: developing and leveraging information superiority. assistant secretary of defense (c3i/command control research program) washington dc. annarelli, a., nonino, f., & palombi, g. (2020). understanding the management of cyber resilient systems. computers & industrial engineering, 149, 106829. antikainen, j. (2014). model for national cybersecurity resilience and situation awareness improvement: an information quality–centric approach leveraging fusion of established practitioner and academic disciplines. (master degree). jamk university of applied science, retrieved from https://www.theseus.fi/bitstream/handle/10024/86179/opinnaytetyo_%20jani%20antik ainen.pdf?sequence=3 appiah, g., amankwah-amoah, j., & liu, y.-l. (2020). organizational architecture, resilience, and cyberattacks. ieee transactions on engineering management. armstrong, c. p., & sambamurthy, v. (1999). information technology assimilation in firms: the influence of senior leadership and it infrastructures. information systems research, 10(4), 304-327. austin, g. (2018). how australian universities can get better at cyber security. retrieved from https://theconversation.com/how-australian-universities-can-get-better-at-cybersecurity-99587#republish bagheri, s. (2020), investigating organisational aspects of cyber resilience in large organisations, phd thesis, university of tasmania, tasmania, australia. bagheri, s., & ridley, g. (2017). organisational cyber resilience: research opportunities. paper presented at the proceedings of the 28th australasian conference on information systems (acis2017), hobart, australia. bei, h. (2019). problems of cybersecurity in the context of becoming and development of the new economy. collection of scientific works of the international scientific conference "competitiveness and innovation in the knowledge economy", xxi edition, september 27-28, 2019, chisinau, moldova, e-isbn 978-9975-75-968-7. bellini, e., & marrone, s. (2020). towards a novel conceptualization of cyber resilience. paper presented at the 2020 ieee world congress on services (services). béné, c., newsham, a., davies, m., ulrichs, m., & godfrey-wood, r. (2014). resilience, poverty and development. journal of international development, 26(5), 598-623. berg, b. l. (2004). qualitative research methods for the social sciences (vol. 5): pearson boston, ma. bergeron, f., raymond, l., & rivard, s. (2004). ideal patterns of strategic alignment and business performance. information & management, 41(8), 1003-1020. berkes, f., colding, j., & folke, c. (2008). navigating social-ecological systems: building resilience for complexity and change: cambridge university press. bernabe, j. b., & skarmeta, a. (2019). introducing the challenges in cybersecurity and privacythe european research landscape. challenges in cybersecurity and privacy-the european research landscape, river publishers series in security and digital forensics, 1-21. australasian journal of information systems bagheri, ridley & williams 2023, vol 27, research article organisational cyber resilience 23 bernard, r. (2007). information lifecycle security risk assessment: a tool for closing security gaps. computers & security, 26(1), 26-30. björck, f., henkel, m., stirna, j., & zdravkovic, j. (2015). cyber resilience–fundamentals for a definition. in new contributions in information systems and technologies (pp. 311-316): springer. borys, s. (2019). the anu hack came down to a single email—here’s what we know. retrieved from https://www.abc.net.au/news/2019-10-02/the-sophisticated-anu-hack-thatcompromised-private-details/11566540 buchmann, r. a., polini, a., johansson, b., & karagiannis, d. (2020). perspectives in business informatics research: 19th international conference on business informatics research, bir 2020, vienna, austria, september 21–23, 2020, proceedings (vol. 398): springer nature. burgemeestre, b., hulstijn, j., & tan, y.-h. (2009). rule-based versus principle-based regulatory compliance. paper presented at the jurix. carias, j. f., labaka, l., sarriegi, j. m., & hernantes, j. (2018). an approach to the modeling of cyber resilience management. paper presented at the 2018 global internet of things summit (giots). cavelty, m. (2007). critical information infrastructure: vulnerabilities, threats and responses. paper presented at the disarmament forum (vol. 3, pp. 15-22). unidir. chapman, j., chinnaswamy, a., & garcia-perez, a. (2018). the severity of cyber attacks on education and research institutions: a function of their security posture. paper presented at the proceedings of the13th international conference on cyber warfare and security (iccws 2018), washington dc. usa. cobo, a., vanti, a. a., & rocha, r. (2014). a fuzzy multicriteria approach for it governance evaluation. jistem-journal of information systems and technology management, 11, 257276. colombo, r. (2020). on the escalation from cyber incidents to cyber crises. master's thesis, university of twente. conklin, w. a., & kohnke, a. (2018). cyber resilience: an essential new paradigm for ensuring national survival. paper presented at the proceedings of the 13th international conference on cyber warfare and security (iccws 2018), washington dc. usa. conklin, w. a., & shoemaker, d. (2017). cyber-resilience: seven steps for institutional survival. edpacs, 55(2), 14-22. retrieved from http://www.tandfonline.com/doi/abs/10.1080/07366981.2017.1289026 craigen, d., diakun-thibault, n., & purse, r. (2014). defining cybersecurity. technology innovation management review, 4 (10): 13–21. in (vol. 4, pp. 13–21). davis, j. i., libicki, m. c., johnson, s. e., kumar, j., watson, m., & karode, a. (2016). a framework for programming and budgeting for cybersecurity. retrieved from rand corporation, santa monica, ca, us. dewar, r. s. (2017). active cyber defense. retrieved from center for security studies (css), eth zurich: https://doi.org/10.3929/ethz-b-000169631 australasian journal of information systems bagheri, ridley & williams 2023, vol 27, research article organisational cyber resilience 24 dupont, b. (2019). the cyber-resilience of financial institutions: significance and applicability. journal of cybersecurity (oxford), 5 (1), 1-17. dutta, a., & mccrohan, k. (2002). management's role in information security in a cyber economy. california management review, 45(1), 67-87. retrieved from http://journals.sagepub.com/doi/pdf/10.2307/41166154 feist, g. j. (2006). the development of scientific talent in westinghouse finalists and members of the national academy of sciences. journal of adult development, 13(1), 23-35. ferdinand, j. (2015). building organisational cyber resilience: a strategic knowledge-based view of cyber security management. journal of business continuity & emergency planning, 9(2), 185-195. fitzgerald, t. (2007). clarifying the roles of information security: 13 questions the ceo, cio, and ciso must ask each other. information systems security, 16(5), 257-263. fox-lent, c., bates, m., & linkov, i. (2015). a matrix approach to community resilience assessment: an illustrative case at rockaway peninsula. environment systems and decisions, 35(2), 209-218. gisladottir, v., ganin, a. a., keisler, j. m., kepner, j., & linkov, i. (2016). resilience of cyber systems with over-and underregulation. risk analysis, 37(9), 1644–1651. goodman, ps., & haisley, e. (2007). social comparison processes in an organisational context: new directions. organisational behavior and human decision processes, 102(1), 109-125. hambleton, r. k., brennan, r. l., brown, w., dodd, b., forsyth, r. a., mehrens, w. a., . . . linden, w. j. (2000). a response to “setting reasonable and useful performance standards” in the national academy of science’grading the nations report card. educational measurement: issues and practice, 19(2), 5-14. hausken, k. (2020). cyber resilience in firms, organizations and societies. internet of things, 11, 100204. holling, c. (1996). engineering resilience versus ecological resilience. engineering within ecological constraints, 31(1996), 32. hult, f., & sivanesan, g. (2014). what good cyber resilience looks like. journal of business continuity & emergency planning, 7(2), 112-125. johnson, a. m. (2009). business and security executives views of information security investment drivers: results from a delphi study. journal of information privacy and security, 5(1), 3-27. kott, a., & linkov, i. (2021). to improve cyber resilience, measure it. arxiv preprint arxiv:2102.09455. le, n. t., & hoang, d. b. (2017). capability maturity model and metrics framework for cyber cloud security. scalable computing: practice and experience, 18(4), 277-290. doi:10.12694/scpe.v18i4.1329 ligo, a., kott, a. & linkov, i. (2021). how to measure cyber resilience of an autonomour agent: approaches and challenges, ieee engineering management review, 1-12. australasian journal of information systems bagheri, ridley & williams 2023, vol 27, research article organisational cyber resilience 25 linkov, i., eisenberg, d., plourde, k., seager, t., allen, j., & kott, a. (2013b). resilience metrics for cyber systems. environment systems & decisions, 33(4), 471-476. doi:http://dx.doi.org/10.1007/s10669-013-9485-y linkov, i., senberg, d., bates, m., chang, d., convertino, m., allen, jh., flynn, se., & seager, t. (2013a). measurable resilience for actionable policy. environmental science & technology, 47(18), 10108-10110. doi:10.1021/es403443n linkov, i., & trump, b. (2019). the science and practice of resilience, cham: springer international publishing. loonam, j., zwiegelaar, j., kumar, v., & booth, c. (2020). cyber-resiliency for digital enterprises: a strategic leadership perspective. ieee transactions on engineering management. luftman, j., papp, r., & brier, t. (1999). enablers and inhibitors of business-it alignment. communications of the association for information systems, 1(1), 11. lykou, g., anagnostopoulou, a., & gritzalis, d. (2018). implementing cyber-security measures in airports to improve cyber-resilience. paper presented at the proceedings of the 2018 global internet of things summit (giots), bilbao, spain. marchese, d., jin, a., fox-lent, c. & linkov, i. (2019). resilience for smart water systems, journal of water resources planning and management, 146(1), 02519002. mcfadzean, e., ezingeard, j.-n., & birchall, d. (2007). perception of risk and the strategic impact of existing it on information security strategy at board level. online information review, 31(5), 622-660. doi:10.1108/14684520710832333 merriam, s. b. (2009). qualitative research: a guide to design and implementation: revised and expanded from qualitative research and case study applications in education. san franscisco: jossey-bass. moallem, a. (2020). hci for cybersecurity, privacy and trust. second international conference, hci-cpt 2020, held as part of the 22nd hci international conference, hcii 2020, copenhagen, denmark, july 19-24, 2020, proceedings (vol. 12210): springer nature. national academy of sciences. (2012). disaster resilience: a national imperative retrieved from https://www.nap.edu/read/13457/chapter/3 noureddine, m. (2020). achieving network resiliency using sound theoretical and practical methods. university of illinois at urbana-champaign, orozco, j., tarhini, a., & tarhini, t. (2015). a framework of is/business alignment management practices to improve the design of it governance architectures. international journal of business and management, 10(4), 1. quacquarelli symonds. (2018, 26 july 2018). qs university rankings. retrieved from https://www.topuniversities.com/ rainer jr, r. k., marshall, t. e., knapp, k. j., & montgomery, g. h. (2007). do information security professionals and business managers view information security issues differently? information systems security, 16(2), 100-108. australasian journal of information systems bagheri, ridley & williams 2023, vol 27, research article organisational cyber resilience 26 rand, k, kurth, m., fleming, c., & linkov, i. (2019). a resilience matrix approach for measuring and mitigating disaster-induced population displacement, international journal of disaster risk reduction, 42(18), 101310. roege, p. e., collier, z. a., chevardin, v., chouinard, p., florin, m.-v., lambert, j. h., nielsen, k., nogal, m. and todorovic, b. (2017). bridging the gap from cyber security to resilience. in resilience and risk (pp. 383-414). nato science for peace and security series c: environmental security: springer. sabev, s. i. (2020). integrated approach to cyber defence: human in the loop. technical evaluation report. information & security: an international journal, 44, 76-92. shapiro, s., keys, b., chhajer, a., liu, z., & horner, d. (2016). a framework for assessing cyber resilience. a report for the world economic forum. sarkar, a., wingreen, s., & cragg, p. (2013). organisational is resilience: a pilot study using qmethodology. paper presented at the 24th australasian conference on information systems (acis2013). segovia, m., rubio-hernan, j., cavalli, a. r., & garcia-alfaro, j. (2020). cyber-resilience evaluation of cyber-physical systems. paper presented at the 2020 ieee 19th international symposium on network computing and applications (nca). selby, j. (2017). how can company boards build trust when faced by cybersecurity risks? optus macquarie university cyber security hub, 3. retrieved from https://www.mq.edu.au/about/about-the-university/offices-and-units/optusmacquarie-university-cyber-security-hub/news2/files/boards-building-trust-cybersecurity-hub-john-selby.pdf sepúlveda-estay, d. a., sahay, r., barfod, m. b., & jensen, c. d. (2020). a systematic review of cyber-resilience assessment frameworks. computers & security, 101996. sharma, r. (2015). five ways board members can improve cybersecurity. journal of internet law, 19(4), 11-12. sikula, n., mancillas, j., linkov, i., & mcdonagh, j. (2015). risk management is not enough: a conceptual model for resilience and adaptation-based vulnerability assessments. environment systems & decisions, 35(2), 219-228. doi:http://dx.doi.org/10.1007/s10669015-9552-7 smith, a. e., & humphreys, m. s. (2006). evaluation of unsupervised semantic mapping of natural language with leximancer concept mapping. behavior research methods, 38(2), 262-279. soomro, z. a., mahmood, h. s., & javed, a. (2016). information security management needs more holistic approach: a literature review. international journal of information management, 36(2), 215-225. stouffer, k., falco, j., & scarfone, k. (2011). guide to industrial control systems (ics) security. nist special publication, 800(82), 16-16. tallon, p. p. (2014). do you see what i see? the search for consensus among executives’ perceptions of it business value. european journal of information systems, 23(3), 306-325. australasian journal of information systems bagheri, ridley & williams 2023, vol 27, research article organisational cyber resilience 27 teo, t. s., & king, w. r. (1997). an assessment of perceptual differences between informants in information systems research. omega, 25(5), 557-566. tisdale, s. m. (2016). architecting a cybersecurity management framework. issues in information systems, 17(4), 227-236. retrieved from http://www.iacis.org/iis/2016/4_iis_2016_227-236.pdf trim, p., jones, n., & brear, k. (2009). building organisational resilience through a designedin security management approach. journal of business continuity & emergency planning, 3(4), 345-355. van der kleij, r., & leukfeldt, r. (2019). cyber resilient behavior: integrating human behavioral models and resilience engineering capabilities into cyber security. paper presented at the proceedings of the international conference on applied human factors and ergonomics, advances in human factors in cybersecurity (ahfe), washington dc. usa. vugrin, e., & turgeon, j. (2013). advancing cyber resilience analysis with performancebased metrics from infrastructure assessments. international journal of secure software engineering (ijsse), 4(1), 75-96. wagstaff, k., & sottile, c. (2015). cyberattack 101: why hackers are going after universities. retrieved from http://www.nbcnews.com/tech/security/universities-become-targetshackers-n429821 wang, n., liang, h., zhong, w., xue, y., & xiao, j. (2012). resource structuring or capability building? an empirical study of the business value of information technology. journal of management information systems, 29(2), 325-367. wells, e., boden, m., tseytlin, i & linkov, i. (2022). modeling critical infrastructure resilience under compounding threats: a systematic literature review. progress in disaster science, 15, 1-15. white, g. (2009). strategic, tactical, & operational management security model. journal of computer information systems, 49(3), 71-75. yano, e., de abreu, w., gustavsson, p., & åhlfeldt, r. (2015). a framework to support the development of cyber resiliency with situational awareness capability. paper presented at the proceedings of the 20th international command and control research and technology symposium, annapolis, md, usa. appendix a: interviews cited to respect participant confidentiality, the authors did not disclose interviewee job titles. only broad areas of responsibility have been provided in the following: interviewee 1: business manager, human resources, 20 years of experience, c1u, [interview date: 20 june 2018]. interviewee 2: business manager, finance operations, 22 years of experience, c1u, [interview date: 15 june 2018]. interviewee 3: business manager, risk area, 11 years of experience, c1u, [interview date: 14 june 2018]. australasian journal of information systems bagheri, ridley & williams 2023, vol 27, research article organisational cyber resilience 28 interviewee 4: business manager, business operations, 28 years of experience, c1u, [interview date: 20 august 2018]. interviewee 5: business manager, budget allocation, 21 years of experience, c1u, [interview date: 11 december 2017]. interviewee 6: it senior manager, strategy design, 20 years of experience, c1u, [interview date: 9 january 2018]. interviewee 7: it senior manager, enterprise architecture, 23 years of experience, c1u, [interview date: 1 june 2018]. interviewee 8: it senior manager, it improvement, 26 years of experience, c1u, [interview date: 27 september 2017]. interviewee 9: it senior manager, security management, 21 years of experience, c1u, [interview date: 17 january 2018]. interviewee 10: business manager, human resources, 18 years of experience, c2u, [interview date: 18 june 2018]. interviewee 11: business manager, finance operations, 11 years of experience, c2u, [interview date: 26 june 2018]. interviewee 12: business manager, business operations, 22 years of experience, c2u, [interview date: 14 june 2018]. interviewee 13: it senior manager, strategy design, 22 years of experience, c2u, [interview date: 5 october 2017]. interviewee 14: it senior manager, infrastructure services, 26 years of experience, c2u, [interview date: 12 june 2018]. interviewee 15: it senior manager, it improvement, 26 years of experience, c2u, [interview date: 13 june 2018]. interviewee 16: it senior manager, security management, 22 years of experience, c2u, [interview date: 13 june 2018]. interviewee 17: it senior manager, technology application, 17 years of experience, c2u, [interview date: 14 june 2018]. copyright: © 2023 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v27i0.4183 http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 a review of organisational cyber resilience 2.1 cyber resilience matrix 2.2 differences between it managers and business managers 3 methodology 4 results 4.1 merged cases datasenior it manager analysis 4.2 merged cases data—business managers 4.3 case study one— senior it versus business managers 4.4 case study two— it senior versus business managers 5 discussion 5.1 merged case data— senior it versus business managers 5.2 separate case data— senior it versus business managers 6 conclusion paper title (paper title style) australasian journal of information systems strahringer 2022, vol 26, book review book review "digitalization cases vol. 2" 1 book review "digitalization cases vol. 2 – mastering digital transformation for global business"1 susanne strahringer tu dresden, dresden, germany susanne.strahringer@tu-dresden.de nils urbach, maximilian röglinger, karlheinz kautz, rose alinda alias, carol saunders, martin wiener (eds.), digitalization cases vol. 2 – mastering digital transformation for global business isbn 978-3-030-80002-4, hardcover, springer, cham, switzerland, 2021, 423 pages, isbn 978-3-030-80004-8, mycopy softcover, isbn 978-3-030-80003-1, ebook https://doi.org/10.1007/978-3-030-80003-1 the edited volume "digitalization cases vol. 2 – mastering digital transformation for global business" in the "management for professionals" series of springer nature switzerland is a continuation of the work of the first two editors published in 2019. the volume presents 20 case studies in the context of digitalization from four continents. several cases are new, but some are also continuations or updates of the case studies from the first volume. target groups are practitioners in companies whose initial situations are comparable to those of the case companies, as well as researchers, teachers and students who want to learn from these cases what digitalization can mean in practice. the cases are divided into three topical areas – digital disruption, digital business and digital transformation (dt)—which also represent the three parts of the work. preceding these three parts, the editors present a jointly written lead-in that gives an instructive introduction to the topic of digital transformation for global companies and global economic action. in addition to a thematic introduction to the three topical areas based on an enterprise architecture model and an overview of the cases, the chapter also provides good directions for the use and reading of the cases which are presented in the following. now to the cases, which are all similarly structured, and in the volume’s three parts are each assigned to one of the three mentioned topical areas. part 1 – digital disruption – deals with cases in which the focus is on modern technologies that have to be observed, analysed, selected if necessary, and skilfully used in order to seize technological opportunities and avoid risks. the first case deals with the anonymization of sensitive data and the detection of manipulations in image material in claims settlement. in the case study, a method based on deep learning is presented, which can be used for anonymization and manipulation detection of image material. ai in conjunction with robotic process automation (rpa) is the technological focus of the second case, which deals with the automation of invoicing in a medium-sized but globally operating company. in the third case, the introduction of digital twins at a turkish electrical household appliance manufacturer is described, which has led to significant and diverse improvements in the production process 1 this is a translated and slightly modified version of a book review initially published in german in the journal "hmd praxis der wirtschaftsinformatik" ("hmd practice of business information systems") under the cc by 4.0 license (https://creativecommons.org/licenses/by/4.0/deed.en), available online at https://doi.org/10.1365/s40702022-00876-7. https://creativecommons.org/licenses/by/4.0/deed.en https://doi.org/10.1365/s40702-022-00876-7 https://doi.org/10.1365/s40702-022-00876-7 australasian journal of information systems strahringer 2022, vol 26, book review book review "digitalization cases vol. 2" 2 and product quality. blockchain technology is the focus of a case study from the work area of the german federal office for migration and refugees, which in a federal system is faced with the challenge of designing processes uniformly and also allowing information to flow between these processes without there being a central executing authority. using the example of the asylum process, the added value of the technology of a data protection-friendly decentralized data storage, which at the same time also creates transparency in almost real time, can be shown. in addition to the technological challenges, this case also illustrates the organizational hurdles in public administrations in the use of blockchain technology. organizational hurdles and individual barriers are also at the heart of a case from the healthcare sector in canada. this example describes the piloting and implementation of a monitoring system for chronically ill patients in their home environment, in which paramedics perform the task of training patients. this is achieved in the various regional communities with varying degrees of success. the last case study impressively illustrates the use of rpa at deutsche telekom services europe (german telcom services europe). more than 170 implemented software robots contribute to considerable efficiency gains there. in part 2 – digital business – completely new or further developed business models enabled by digitalization are the focus of the respective case studies. in the very first case, the transformation of a formerly insignificant telecommunications company into a global player, which has risen to become one of the top three telecommunications companies in the world, is shown: the success story of huawai. the case impressively shows that such a development is not about individual digital innovations, but about the interaction of a large number of interlocking components that are successfully implemented along a strategy and supported by appropriate leadership. the second case concerns a traditional company in the german insurance industry, devk, which, in addition to its core business, has founded a start-up for digital insurance products that can be implemented entirely on digital channels. in implementation times of 4 and 8 months, two products could be launched on the market, which would not have been possible in the parent company at this speed. the case shows that in addition to the change of the business model, the organizational embedding of changed business model in the existing business environment is of high importance for success. the third case concerns the german ingredient, seasoning, and spice specialist raps and one of its customer groups, the traditional butcher industry. through a platform-independent new service offering, raps offers its customers an application by means of which they receive valuable support in critical problem areas, e.g. in the area of the declaration of ingredients of their meat products. although the service is provided to customers free of charge, it has contributed significantly to the success of raps and the positive public perception of the company. mobility x lab is a swedish incubator founded by six companies from the automotive and telecommunications industries, which has developed into a collaboration hub. the aim is to promote digital innovation in the automotive industry by successfully advancing matchmaking for collaborative digital projects between the founding companies and the participating start-ups. in the case of schott ag, a german multinational glass manufacturing company, it is about promoting innovation too, in this case digital service innovations based on a b2b e-commerce platform. the case shows that service innovations need not be developed solely from a problem-centred customer perspective, but that this can be combined with a resource-driven view of business opportunities. in addition, the schott example shows that high quality expectations resulting from a company's traditional business model are transferred to the expectations of digital services. a company like schott australasian journal of information systems strahringer 2022, vol 26, book review book review "digitalization cases vol. 2" 3 therefore has to take care of meeting such expectations with digital services if it wants to avoid the risk of jeopardizing its usual quality promise. the last case of this section concerns the city of lohmar, a german city of approximately 31000 inhabitants, which was able to develop a digital transformation strategy in a joint project with the participation and with the support of the federal german ministry of the interior and community. the strategy aims in particular at solving urban traffic problems through a smart mobility hub that combines multimodal mobility systems. the last part of the book, i.e. part 3 – digital transformation – deals with cases that show technology-induced changes in companies in organizational, procedural, and technological terms. the first case of this part illustrates, presenting the example of a small or medium-sized enterprise (sme), that also smaller companies with limited resources are able to develop and successfully implement digital strategies. in the following case, the example of a public utility company in ghana also illustrates a digital transformation process with special challenges in change management. in the subsequent case study of a german automobile manufacturer, the requirements of digital transformation for the workforce and the transformation of the workforce are considered. on the basis of an employee-centred transformation concept, which was designed to be participatory, it was possible to sensitize the workforce to the digitalization-related changes and to increase the acceptance of these changes and it showed that the willingness of those involved to change significantly exceeded expectations. in the case of the dutch platform company springest, the introduction of a holocracy in the transition from a start-up to an sme is considered. central to the success of this form of organization was the use of transparency-enhancing it tools. in the case study, the well-supported new organizational form, well-supported by it, led to considerable growth and increased employee satisfaction. in the case of the vietnamese it company fpt software, the use of digital kaizen to support a digital transformation program is shown. the chosen path of the company is to apply digital transformation approaches in the company itself and, if successful, to commercialize them, i.e. to create customer offers from them. digital transformation is understood as a continuous improvement program in the sense of the kaizen philosophy. the special methodology is illustrated in the case by two concrete projects. the next case, the example of arcadis, a globally operating planning and consulting company, especially for construction assets, describes the transformation into a learning company. this case also clearly shows that digital transformation is not about a one-time change program, but about the cultivation of a digital mindset. as in the previous case, this is again a large company, but this time one that is extremely fragmented by inorganic growth. in the arcadis case study, the goal of digital transformation therefore is to include the entire company and to bring about a consolidation in terms of common vision and identity. in addition to overcoming complexity, the challenge with this approach is to not destroy the decentralization that aims at local customers. here, i will not resolve the path taken by the company, because from my point of view the arcadis case study is the "most exciting" case of the book and the one that has impressed me personally the most. for me, it is also one of the cases that is clearly aimed at an entire top management team of a company as a target group. it is not a cdo or cio case, but one that should also interest ceos and top hr managers. what i also like about the case is that, together with some of the following cases, it very clearly lives up to the subtitle of the volume "mastering digital transformation for global business". australasian journal of information systems strahringer 2022, vol 26, book review book review "digitalization cases vol. 2" 4 the next case is located in the context of a german manufacturer of medical aids, medi, which has developed into a globally active company over the last 20 years. the case study is about the company's entry into the development of a data-driven company. the focus is in particular on creating the conditions for this development. the case convincingly illustrates that companies in digital transformation processes first have to take a step back and classically have to analyse where they actually stand. the interesting thing about this case is that a detailed enterprise architecture model consisting of processes, applications, and data models offers this entry and, that and how it actually can have an impact, namely through transparency, which has further, flow-on self-reinforcing effects. the last case of the book takes up a traditional topic, namely the introduction of an erp system in a decentralized company, which is considered a transformation program and not a software introduction. the swiss arbonia group operates globally and consists of four divisions, each with several branches. the case study deals with the door division with four subsidiaries spread across germany, switzerland, and poland. at the same time, a cross-site sap s/4hana implementation is intended to bring about harmonization, which still leaves enough local leeway and flexibility. in the example, this is achieved with governance measures that promote both local differentiation and global consistency. how this can be concretely designed and implemented makes this case a striking example in the multitude of erp implementation case examples that have already been dealt with in the literature before. this last case also forms the end of the book. perhaps the editors could have given the book a more cohesive character if they had rounded it off with a final chapter. however, this hardly diminishes the high quality of the work, whose strengths lie in the convincing and wellpresented cases. all follow a similar structure, which makes it easy to find the way around them. it is also nice that the described companies are called with by their actual names. while reading, i repeatedly caught myself looking at company websites on the internet at the same time – out of the curiosity that the reading of the cases caused. this can certainly be used well in teaching and practitioners will also appreciate the fact that they learn which company is involved and that company members are often named as co-authors of the case descriptions. it should be noted that the cases deal with very different facets of the topic of digitalization, which is brought about by the three parts of the book. personally, i liked the contributions of the third part best, and unfortunately it took me a while to get there with the reading. the thematic breadth is such that i can hardly imagine readers for whom everything is equally interesting. that's why i can only recommend every reader to select the cases of interest carefully. for such a selection, the abstracts help very well, because they are each divided into the four parts, "situation faced", "action taken", "results achieved", and "lessons learned". this structuring can then also be found in the main parts of the contributions themselves. when reading the book, i paid particular attention to its suitability for executives such as cios, cdos or ceos. even if the explicit role of it-related c-level executives is only specifically elaborated in a few cases and they are only mentioned in a few cases, the volume still offers a relatively large number of insights for this target group. almost all cases make it clear that a traditional, technology-oriented understanding of it is outdated, that even talking about business and it is no longer up to date, but rather that one should speak of "it is business" and "business is it". in this sense, the book is therefore not only relevant for the it-savvy clevel roles, but for the entire top management team. detached from these people as a reader target group, the cases in the volume also tell something about these roles, although they are rarely explicitly mentioned. almost everything described in the case studies is the australasian journal of information systems strahringer 2022, vol 26, book review book review "digitalization cases vol. 2" 5 responsibility of cios, cdos and ceos. the design of these and similar positions and their interaction is therefore of decisive relevance for whether what is described in the cases of this book can also succeed in other companies. many of the case studies make it clear that successful digital transformation programs are not top-down projects, but also that they do not work if the mindset at the top of the company is not ready for such programs. the work illustrates many of the tasks and challenges that leaders must take on and solve in these contexts. those of these executives, who have very little time to immerse themselves in these many realistic illustrative cases, have above already received a hint for a particularly interesting case. finally, i can only praise the editors, all highly recognized scholars and researchers of information systems, and the equally renowned team of reviewers for the good selection of cases and for their willingness to support on the experienceability of the wide range of topics that we research and teach in business information systems. the book makes a significant contribution to this! copyright: © 2022 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v26i0.4001 http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems clarke, michael & abbas 2024, vol 28, research article robodebt: socio-technical case study of is failure received 7 sep 2023, accepted 4 july 2024 after 1 round of review with karlheinz kautz as accepting editor 1 robodebt: a socio-technical case study of public sector information systems failure roger clarke xamax consultancy pty ltd, canberra, unsw law, sydney, and anu school of computing, canberra, australia roger.clarke@xamax.com.au katina michael school for the future of innovation in society and school of computing and augmented intelligence, arizona state university, tempe, usa roba abbas school of business, university of wollongong, australia abstract large-scale public sector information systems (psis) that administer social welfare payments face considerable challenges. between 2014 and 2023, an australian government agency conceived and implemented the online compliance intervention (oci) scheme, widely referred to as robodebt. the scheme's primary purpose was to apply digital transformation in order to reduce labour costs and increase recovery of overpayments. among its key features were a simplified, but inherently erroneous, estimation method called income averaging, and a new requirement that welfare recipients produce documentation for income earned years earlier. failure by welfare recipients to comply with mandates resulted in the agency recovering what it asserted to be overpayments. this article presents a case study of robodebt and its effects on over 1 million of its clients. the detailed case study relies on primary data through senate and other government hearings and commissions, and secondary data, such as media reports, supplemented by academic sources. relevant technical features include (1) the reliance on the digital persona that the agency maintains for each client, (2) computer-performed inferencing from client data, and (3) automated decision-making and subsequent action. this article employs a socio-technical systems approach to understanding the factors underlying a major psis project failure, by focusing on the system's political and public service sponsors; its participants (users); the people affected by it (usees); and the broader economic, social, and political context. practical and theoretical insights are presented, with the intention of highlighting major practical lessons for psis, and the relevance of an articulated socio-technical frame for psis. keywords: is failure, socio-technical systems, case study, public sector is, digitalisation. 1 introduction this article presents a detailed case study of the online compliance intervention (oci) scheme, commonly referred to as robodebt. the australian government agency responsible for the oci was the department of human services (dhs). the project was intended by as a form of transformative digitalisation. this study spans the period 2014-23, from the conception of the oci scheme, to the pilot in 2015, the launch in 2016, the operational phase in 2016-19, through to a series of official reviews and examinations that commenced in 2017 australasian journal of information systems clarke, michael & abbas 2024, vol 28, research article robodebt: socio-technical case study of is failure 2 but concluded only in 2023. a significant element of the scheme was debt recovery letters that were, in a large proportion of instances, inappropriately, sent to more than 1 million clients and ex-clients, in most cases generated automatically, with media reports of as many as 10 follow-up letters. debts were raised against 433,000 individuals, about 2% of the australian adult population. after the government was eventually forced to admit that the scheme was illegal (carney 2019), it undertook "reimbursement of [aud] 746 million to some 381,000 affected individuals ... writing off claimed debts amounting in total to [aud] 1.751 billion" (rcr 2023, p.xix). 1.1 previous work we present in this preliminary section a chronological account of peer-reviewed publications on robodebt between 2017 and 2023. not long into the operation of the scheme, two articles appeared that raised awareness of the issues plaguing the australian government initiative. miller (2017, p.54) wrote the first case study on robodebt focusing on affected communities, identifying multiple stakeholders active on the issues, including “journalists, community activists, centrelink payment recipients, lawyers, legal assistance organisations, industry and professional peak bodies, academics and citizens”. in macleod (2017), a senior executive in the commonwealth ombudsman's office shed light on some of the preliminary “lessons learnt”. the five major themes were the need for: (1) better design of digital platforms, (2) transparency, (3) user support, (4) external perspectives, and (5) guidance and oversight. in the same year, hogan-doran sc (2017) wrote on the relationship between automation, algorithms, and artificial intelligence (ai). she posed these questions about robodebt: “who is the decision-maker?” and “to whom has authority been delegated?”. to the latter, she identified the following possibilities: (a) the programmer, (b) the policy-maker, (c) the authorised decision-maker, or (d) the computer itself. in relation to (c), hogan-doran (2017) suggested grounds on which outcomes from an algorithm can be challenged, such as that the decision-maker “breached a mandatory statutory procedure or obligation” (pp.7-9). during 2018, peer-reviewed articles on robodebt were scarce. the scheme was fully operational and expansion had been announced. sufficient consideration had not, however, been given to the many issues that were already attracting media attention, many featuring first-person accounts of unwarranted debt recovery letters being received.in 2019, park & humphry published a piece entitled “exclusion by design” that focused on the social, digital and data inequalities in two cases, one of which was robodebt. among the important findings was that an over-reliance on data, inadequate user testing, and the “re-delegation of tasks and responsibilities through automation” can by-default exclude some people eligible for welfare payments (p.946). in carney (2019), a long-serving member of the administrative appeals tribunal (aat) called robodebt illegal, unlawful and unethical, claiming that the scheme was fundamentally flawed at seven levels including its inadequate attention to standards; accountability measures; ethics; transparency; “independence and funding security” (p.5); testing and advocacy; and rights of individuals. during 2020-2021, many socio-political and legal perspectives were published on robodebt. some emphasised the “automated welfare state” (whelan 2020, noone 2020, braithwaite 2020, mann 2020). whiteford (2021) published “debt by design” (p.340), and hanks (2021) & white (2021) wrote on administrative law and welfare rights with respect to robodebt. white (2021) focused on the idea of “authorization and accountability of automated government decisions” (p.84). marjanovic et al. (2022) described the “social consequences of australasian journal of information systems clarke, michael & abbas 2024, vol 28, research article robodebt: socio-technical case study of is failure 3 automated algorithmic decision-making” and addressed the issues of “algorithmic harm” and “algorithmic injustice” (p.269), while akter et al. (2021) emphasised algorithmic bias in data-driven innovation, focusing on data, method and societal biases referring to the result of robodebt’s attempted scheme as “algorithmic fallout” (p.8). in 2022, automated decisions were the focus of two papers on “digital inclusion”, and intersectionality with respect to the “disadvantaged” and those living with disability (goggin & soldatić 2022, ng & gray 2022). with the scheme now withdrawn, post-mortems on robodebt began to surface, with calls for access to algorithms used in such schemes, and demonstrations of the legacy issues that the scheme had caused with respect to subsequent process changes within the agency (ray et al. 2022, wright & ng 2022). another paper pointed to the bureaucracy and the stigmatisation of citizens (graycar & masters 2022). rinta-kahila et al. (2022), completed in mid-2021, prior to the conclusive court-case and the royal commission hearings and report, adopts algorithmic decision-making (adm) as its theoretical frame. by 2023, official reviews and examinations of robodebt were coming to an end, but academic articles continued to appear. lindebaum et al. (2023), for example, argued that algorithms in government services would cause the withering of human values. cornish (2023) summarised record-keeping deficiencies, and mead & neves (2023) focused on the critical public response to algorithmic decision-making, comparing the uk and australian contexts from a sociological perspective. naylor (2023) linked robodebt to kafka and called out the “institutional absurdism” that took place. nikidehaghani et al. (2023) examined the question of algorithmic accountability, and twomey (2023) identified the loss of "government integrity" as a major concern. 1.2 objectives while the above studies shed light on various facets of robodebt, the need exists for a comprehensive account of the scheme, in the form of a detailed case study. a theory-agnostic approach would be highly beneficial; but, in practice, theory-laden views and words are unavoidable. many different approaches can be taken, and many theoretical frames and lenses are possible. we chose to focus on socio-technical dynamics and considerations, and to present the story largely chronologically rather than thematically, with the intention that the story-telling reveal socio-technical challenges and lessons that can inform the design of future public sector information systems (psis). this is critical because the scale and scope of 21st century is in general, the increasing incidence of automated decision-making and action, and the lack of transparency of some of the forms of information technology (it) being applied, particularly machine learning aspects of artificial intelligence (ai/ml), give rise to far more substantial impacts and implications than was the case during the second half of the 20th century. the breadth of vision of is practitioners and researchers must change accordingly, to adopt a comprehensive socio-technical view that allows for systems risks to all affected parties to be identified and assessed, and then avoided, prevented or at least mitigated, and system designs to be improved based on lessons learned from the robodebt experience. this article accordingly presents a detailed, socio-technical case study of substantial changes made to a large-scale psis. the objectives of the study are as follows: 1. to apply socio-technical systems theory to a complex public-sector information system. australasian journal of information systems clarke, michael & abbas 2024, vol 28, research article robodebt: socio-technical case study of is failure 4 2. to understand the factors, decisions, events, and corresponding dynamics that resulted in the oci scheme failing to achieve its objectives. 3. to present the key socio-technical lessons toward enhanced public sector information systems design that can be utilised by researchers and practitioners alike to inform future psis design decisions, and as an educational resource pertaining to psis. with some important qualifications, the authors have endeavoured to present evidence, and provide access to the primary sources, drawing on socio-technical theory with direct relevance to is practice and practice-oriented is research. in section 2, we present literature reviews relating to digitalisation in the public sector and public sector information systems in the australian context. the theoretical backdrop in socio-technical systems (sts) theory is then presented, which is subsequently utilised as the basis for framing the case study. the method is presented in section 3, including the case study setting, research approach and data collection and analysis details. section 4 offers a comprehensive robodebt socio-technical case study analysis spanning a 7-year period culminating in the scheme’s collapse. section 5 identifies practical and theoretical implications in the form of socio-technical lessons contributing to enhanced public sector information systems design, followed by future research opportunities in section 6 and concluding remarks in section 7. 2 literature review 2.1 digitalisation in the public sector applications of it in business and government began in the 1950s with the automation of existing systems, traditionally known as manual systems. the focus quickly shifted to the rationalisation of existing processes, to take advantage of the new possibilities that technologies had created. over time, the emphasis shifted to business process reengineering (bpr) (hammer 1990), and then to organisational and sectoral transformation (venkatraman 1994). most recently, the capacity of it to achieve organisational disruption has been championed (bower & christensen 1995, lyytinen & rose 2003, christensen et al. 2015). in parallel, the combination of computing and telecommunications enabled the scope of itsupported systems to broaden from organisational sub-units, via cross-organisational functions, to enterprise systems spanning entire organisations. it then broke beyond the boundaries of a single organisation into inter-organisational systems (1-to-1), and to multiorganisational systems in star, chain and network configurations. an often-overlooked development was the emergence of extra-organisational systems (clarke 1992b), which extended beyond organisational boundaries to encompass independent individuals. at first, those people had to be provided with access to network-connected devices. during the last decades of the 20th century, however, increasing numbers of people acquired desktop and laptop devices that could be harnessed into organisation-managed systems, and the first decades of the 21st century have seen the proliferation of handheld mobile phones and tablets and their exploitation as an extension of organisations' own information infrastructure. during these stages of maturation, the technical challenges in systems development came to encompass organisational and individual human behaviour, such that system sponsors needed to adopt a socio-technical view (clarke & davison 2020, p.489, abbas & michael 2022). it was no longer solely about hardware facilitating network connectivity or storage, nor the software that could be deployed on the infrastructure at australasian journal of information systems clarke, michael & abbas 2024, vol 28, research article robodebt: socio-technical case study of is failure 5 scale, but also about the interplay of business structures, business processes, and humans and their corresponding relationships, behaviours, beliefs and values in the context of a social subsystem (bostrom & heinen 1977, p.25, fig. 2; abbas & michael 2022). the sociotechnical systems perspective is discussed in greater detail in 2.3 below. a concomitant development throughout the growth of such systems has been the digitisation of data, initially through the purposeful capture of data of specific relevance to an objective, and progressively as a by-product of other actions. data capture has also been increasingly outsourced, so that organisations benefit from data-input by unpaid participants in systems, referred to in abbas et al. (2022, p.78) as ''customer work". by about the turn of the current century, a large proportion of the data desired by organisations was digital by default, rather than having to be converted from other paper-based formats. the availability of vast quantities of data has enabled organisations to achieve efficiencies by no longer dealing directly with other entities, and particularly not with individual people. organisations find it far more economical to transact with, and impose surveillance on, a digital persona in preference to a person (clarke 1994a, 2014). the term datafication was coined by cukier & mayer-schönberger (2013) as part of those authors' inculcation of excitement about big data as the harbinger of a post-rationalist world (see also, lycett 2014, and newell & marabelli 2015). the steadier concept associated with the term digitalisation describes the shift from the interpretation and management of the world by means of human perception and cognition of phenomena, to processes that are almost entirely dependent on digital data (brennen & kreis 2016). where the entities being interpreted and managed are people, the term carries overtones of objectification, as the human aspects of interactions are abandoned. for-profit business enterprises are largely unconstrained in their application of digitalisation by public policy considerations. this reflects their role as drivers of growth, and the assumption of markets being sufficiently competitive that consumers have alternatives among which to choose. public sector organisations, on the other hand, operate in an environment in which there is an expectation that greater care will be taken about the direct impacts of transformative and disruptive it-based interventions, and about their indirect implications or potential for unintended consequences among stakeholders. those impacts and implications are commonly felt by users, who are the people inside and outside the organisation who are direct participants in the system. too often overlooked are people who are indirectly but materially affected, usefully referred to as usees (berleur & drumm 1991 p.388, clarke 1992b, fischer-huebner & lindskog 2001, wahlstrom & quirchmayr 2008, baumer 2015). examples of usees include people who have entries in criminal intelligence and credit reporting databases, and those who are dependants of welfare recipients. the term stakeholder was coined as a counterpoint to shareholder, to bring into focus the interests of parties other than the corporation's owners (freeman & reed 1983, laplume et al. 2008). in corporate and government practice, the term stakeholder is commonly narrowed to sub-organisations of the sponsor of an intervention. its appropriate interpretation, however, encompasses organisation-external users and usees. the stakeholder notion has been subjected to further analysis and discussion. importantly, stakeholder salience theory distinguishes stakeholders based on power, legitimacy and urgency (mitchell et al. 1997). an approach based on ethics might dictate that legitimacy has primacy. on the other hand, a strong tendency has been evident, in industry and government practice, and in research australasian journal of information systems clarke, michael & abbas 2024, vol 28, research article robodebt: socio-technical case study of is failure 6 publications, for attention to be paid only to the interests of those stakeholders that are capable of significantly affecting the success of the project – as perceived by the project sponsor (bryson 2004). the interests of the sponsoring organisation are treated as paramount, whereas the interests of users and usees that have power are relegated to the role of constraints on the achievement of the primary organisation's objectives, and the interests of users and usees that lack power are commonly marginalised or ignored. in clarke & davison (2020), stakeholder theory was applied and extended, to present a theory relating to the researchers' perspective. research into real-world phenomena involves carefully-conducted observation of some object of study. objects of study relevant here, such as large-scale, data-rich is, involve multiple entities, each of which perceives the phenomena in their own way. a stakeholder perspective is the viewpoint adopted by a stakeholder in a particular activity, reflecting that stakeholder's perception of phenomena within the relevant context, the stakeholder's value-set, and the interests that the stakeholder seeks to protect and advance. the central proposition of researcher perspective theory is that research is seldom conducted in a holist or universalist manner, reflecting the interests of all stakeholders at once. much more commonly, the perspective of one party is adopted. systematic studies of is research literature have shown that the large majority of published is research adopts the perspective of a single stakeholder, and that, in the large majority of that research, the favoured stakeholder is the is sponsor (clarke 2015, 2016, clarke et al. 2020). researcher perspective theory identifies benefits arising from single-perspective is research that reflects the interests of a stakeholder other than the system sponsor. further, even greater benefits are achievable by dual-perspective research designed to benefit both members of a dyad. broader, multiperspective research is challenging, but highly desirable, and, in some circumstances at least, it is feasible. the contemporary context of is includes reach well beyond organisational boundaries. it involves substantial impacts on users and implications for usees, overlaid with reliance on digital personae and data analytics. this is combined with increasing degrees of automation of inferencing, decision-making and even action. we contend that the nature of systems is now such that it is essential to apply broad interpretations to the scope of is, is practice, and practice-relevant is research. an adequate understanding of large-scale systems is impossible if personal, group, organisational, inter-organisational, extra-organisational, regulatory or policy factors are excluded from the field of view. 2.2 is in the australian public sector digitalisation depends on data not only being in digital form, but also being reliably associated with particular real-world entities or identities. a considerable amount of public sector activity is accordingly concerned with the administration of relationships between government agencies and people, with governments striving to establish and maintain workable identification schemes (michael & michael 2006). some countries have generalpurpose identifiers; others use identifiers across a cluster of related functions; while many depend primarily on agency-level or program-level identifiers (lunde 1980, michael & michael 2006). in australia, successive attempts to establish a national identification scheme have failed (clarke 1987, michael 2003). however, a multi-purpose identifier called a tax file number (tfn) was established in 1988, under the administration of the australian taxation office (ato, clarke 1992a). the tfn is in ongoing use in public sector financial programs, australasian journal of information systems clarke, michael & abbas 2024, vol 28, research article robodebt: socio-technical case study of is failure 7 and in a range of private sector contexts that have continued to expand throughout its first 35 years of operation. organisations began by gathering data for specific purposes, but have long since resorted to the consolidation and re-purposing of data gathered by many organisations for many different purposes, often with little account taken of incompatibilities among datasets, and the impacts and implications of such activities. one use of the tfn is for data-matching schemes, which involve the comparison of machine-readable records from separate sources that contain personal data that is purported to relate to the same person. data-matching is a longstanding dataveillance technique much-used by government agencies in various countries since the mid-1980s, particularly to help in addressing financial waste and fraud (clarke 1994b). in australia, many such schemes are largely unregulated, and subject only to a non-binding guideline published by the data protection oversight agency. one series of very large-scale programs caused considerable public concern, and as a result was subjected to a modestly formalised regulatory environment, the data-matching program (assistance and tax) act 1990 (dmp). since about 1990, there has been a strong trend in australia towards large-scale government initiatives, concentrating programs within superor mega-portfolio agencies, but also with some degree of attention to purchaser-provider relationships. both are evident in the social welfare context, where policy has been separated from the operational aspects (podger 2023). the operational scale is vast, with total social security and welfare expenditure representing over 30% of the australian government's budget (klapdor 2020). the major schemes assist 2.6 million people of working age (13 percent of those age-groups and 10 percent of the population) and 2.5 million people eligible for age pension (a further 10 percent of the population). to underline the social significance of this, "[f]or the poorest 20% of australian households, social security payments provide more than 70% of their income", and most recipients' earned incomes are highly variable across each year (whiteford 2022). policy in relation to the more than 100 support programs continues to rest with specialist agencies, in a variety of portfolios. the primary policy agency, the department of social services (dss), is responsible for unemployment benefits (including newstart and jobseeker), the age pension, disability support and crisis payments in relation to fires, floods, the covid pandemic, etc. military service pensions policy rests with the department of veterans affairs, and childcare subsidy was at the time with the department of employment and workplace relations, whereas student support (austudy and abstudy) rested with the departments of education and of social service, and farm household allowance policy with the department of agriculture, water and the environment (sa 2022). although policy responsibility vests in specialist agencies, the administration of social security is the responsibility of a dedicated agency called centrelink, formed in 1997. it is a services-, dataand identifier-hub, handling all payments to welfare recipients, all interactions with them, and associated oversight activities. in 2011, centrelink and the national health insurance service, medicare, were combined into a new super-agency, the department of human services (dhs), branded since 2019 as services australia. an enormous diversity exists, of programs, needs, clientele, and personal and regional circumstances. dhs delivers critical financial support to australia’s most vulnerable populations, such as people with significant physical disabilities or mental health conditions, and single parents. some people are in multiple such categories. those living in remote and australasian journal of information systems clarke, michael & abbas 2024, vol 28, research article robodebt: socio-technical case study of is failure 8 rural australia (about 2.5 million people or 10% of the population) have limited access to local services that allow for direct human interfacing with services personnel. successive generations of technology have been harnessed in support of centrelink's work, with the agency having operated one of the nation's largest it facilities since the 1960s. the scale of the is and database supporting client management, entitlements, payments and financial management is very large by world standards. the scores of complex, longstanding and frequently-modified welfare schemes are designed to protect against waste and fraud perhaps as much as to address economic needs. centrelink operates distributed, outsourced call-centres, physical service-centres (publicly declared as numbering more than 300), plus hundreds of access points and agents, together with some mobile services. frontline staff are beset with enormous challenges. they are expected to master an array of disparate policy and legal details and interact with large numbers of individuals in need, some with complex life stories. diseconomies of scale and scope are in play. the complexities, ambiguities, inconsistencies, conflicts and discretions within written policies, and the diversity of circumstances, combine to give rise to misunderstandings and errors, and inevitably to recriminations, requests for review and appeals. there are many assumptions made when large-scale public sector is are rolled out, including that each relevant person in australia (1) has access to electronic communications, and in particular the internet, when they need it; (2) knows how to, or can quickly learn how to, navigate each specific online service to request support for the needs they have and to comply with agencies' demands for data; and (3) has the language proficiency, cognitive capacity and educational level needed to fill out forms online, and scan and attach copies of documents, without direct human support. not only does broad scope exist for mistaken assumptions, misunderstandings, inefficiencies and errors, but fraudsters are also naturally attracted by opportunities the scheme offers them, and some genuine recipients are tempted to take advantage of loopholes. considerable effort is accordingly invested in preventing unjustified and excessive payments, in discovering where overpayments have occurred, and in seeking restitution. official estimates of the scale of overpayments have generally been in the range aud 1.5-4.0 billion p.a. (i.e., 1.0-2.5% of about aud 160bn p.a. total payments) (podger 2023, p.16)). this is contested, however, with estimates ranging down to less than one-tenth of those of the agency. 2.3 socio-technical systems framing despite considerable investment in it and is, there have been notable examples of public sector information system failures in australia. this is widely assumed to be due to excessive emphasis on technology and its purported benefits, and lack of attention to social and societal factors and implications. socio-technical approaches going beyond the technical system and incorporating social subsystems are essential to adequately capture system complexity and account for the multitude of stakeholders. socio-technical systems theory and related design methods are built on open systems thinking (von bertalanffy 1950) and acknowledge that systems comprise technical components working in combination with social and/or human elements (emery 1959, abbas & michael 2022). the theory offers a principled approach to the implementation of socio-technical interventions and (re)-design initiatives by seeking balance between humanistic values and technological capabilities (cherns 1976, bostrom & heinen 1977, cherns 1987, trist 1981, mumford 2000, mumford australasian journal of information systems clarke, michael & abbas 2024, vol 28, research article robodebt: socio-technical case study of is failure 9 2006). socio-technical theory has been embedded in various methods (e.g., checkland 1981, avison & wood-harper 1990). the authors have elsewhere shown that developers of public sector information systems need to adopt design methods that reflect socio-technical insights (abbas et al. 2021). in assuming this broader stakeholder perspective, as noted by miller (2017), rather than defaulting to the viewpoint of the is sponsor, the socio-technical perspective calls for a balanced approach considering the interactions between the social and technical elements comprising the system in question. socio-technical thinking therefore places emphasis on the human-centered approach that is key to socio-technical studies (norman 2013, ch.6; ixdf 2021). bostrom & heinen (1977) drew attention to the narrowness of focus inherent in the technical-system approach to an mis as a 'work system', in that only technology and tasks were treated as being within the scope of is design. those authors proposed that the social-system aspects of people and organisational structure have a significant impact on the effectiveness of an is design, and that is designers therefore need to consider all four aspects in an integrated manner. the bostrom & heinen insights emerged within the context of intra-organisational systems. nearly five decades later, however, inter-organisational and extra-organisational is demand an enhanced model. the people within the intra-organisational component of the overall system are users and their managers. in contemporary open systems, extra-organisational users perform 'customer work'. further, beyond the overall system's boundaries, usees are affected by its behaviour. impacts on external users (part of the extra-organisational system), and on usees (external to the system as a whole), also need to be defined as being withinscope for system designers. figure 1 presents an enhanced version of such an open sociotechnical system, showing those additional entity-categories, and their locations relative to the organisational boundary and the system boundary. although some exclusively technical problems were identified, it was through a socio-technical examination that problems were observed relating to (1) the interaction of intentions, (2) behaviours of various stakeholders, (3) the absence of any apparent application of well-known design principles, (4) contextual considerations, and (5) program implementation. a range of techniques exist that assist organisations to ensure effective designs reflecting socio-technical and human-centred principles. the field of participatory systems design has a long history (land & hirschheim 1983), with occasional revivals in such forms as co-design (liu et al. 2002, piller et al. 2004) and co-creation (zwass 2010). the involvement of users is challenging, not least due to their diversity and their lack of familiarity with design processes. for many organisations it has proven impractical to include users, at least in part because of entrenched technocratic and other deep-seated values within design teams, and the resource investment required to involve users in the system design process. usees, meanwhile, are usually disregarded as being inconsequential to design processes. the increased emphasis in recent years on design science does not appear to have eased the predominance of organisation-centricity and technocracy. participation is not non-existent, but it remains marginalised, with language tending to depict designers and users as being on opposite sides of conversations rather than collaborating as team-members. clarke & davison (2020) have drawn attention to the possibility of, and need for, participative design science: "beyond asking 'what is a feasible and effective process for the design of a particular system or category of systems?' (cf. guideline 3 of hevner et al., 2004), research questions of the following form can be investigated: 'what is a feasible and effective process for reflecting australasian journal of information systems clarke, michael & abbas 2024, vol 28, research article robodebt: socio-technical case study of is failure 10 the perspectives of all parties in the design of a particular system or category of systems?" (p.492). figure 1: an open socio-technical system incorporating all stakeholders. extended version of bostrom & heinen (1977, p.25, fig.2) we now describe the research method that we adopted in order to present a case study within the socio-technical framing outlined in this section. 3 method 3.1 case study setting the research reported here employs a case study of the online compliance intervention (oci) scheme, more commonly known as robodebt. a single-case study approach is an effective research method where the primary intention is to work within a theoretical frame and seek to explain or interpret a single historical episode, and care is taken in drawing inferences about the applicability of the outcomes beyond the individual case (yin 2017). the scheme's purpose was officially depicted as being “to protect the integrity of government expenditure through income support payments” (omb 2017, p.4) or, less vaguely, to save labour costs (omb 2017, p.5) and increase recovery of overpayments (sca 2017, p.2). the oci was to harness the power of automation by applying algorithms that would overcome the need for human intervention by government staff in the determination and communication of debt recovery demands to clients. australasian journal of information systems clarke, michael & abbas 2024, vol 28, research article robodebt: socio-technical case study of is failure 11 in choosing a theoretical approach to underpin the qualitative study, we sought a theory that had previously been applied to organisational systems, that would incorporate relevant social, technical and environmental factors, and that would thereby shed light on the comprehensive failure of a major adaptation of a large-scale psis. the socio-technical systems approach satisfied those requirements. it was used to evaluate the amended system that operated 2016-19, including comparison against the system that operated prior to 2016 and that was reverted to at the end of 2019. 3.2 research approach a study of a project intended to enhance a large-scale psis needs to cope with considerable complexity. the typical approach would involve observation through significant fieldwork, which would require government permission and substantial human resources. however, the government was not transparent about its practices, and the more it became known that major errors had been made in relation to the program, the less observers were able to learn. the final report of a parliamentary committee on the matter concluded that the government not only made many errors in relation to the program itself, but also on multiple occasions practised "the deliberate withholding of information" (sca 2022, p.17). access to dhs records was not feasible, because the agency's operations are not transparent. aided by the dysfunctionality of the country's freedom of information mechanism (knaus & bassano 2019, cpi 2022, bajkowski 2023), dhs actively resisted requests for access to information not only by its clients, but also by advocates and lawyers acting on their behalf, by the ombudsman, and by a parliamentary committee (e.g. sca 2017, pp.39,40,57; as noted in rrc 2023, pp.88-89,166,208,214,461). a case study method (yin 2017) was adopted, because it allowed for the incorporation of a diverse range of qualitative sources that offered a multi-perspective view of the design, course and impacts of the project over time. we situate the project within a frame of the application of transformative digitalisation in a government agency, adopt the sociotechnical approach as discussed above, and use a narrative style of presenting evidence on a chronological timeline to provide a sequence of events that culminated in project failure, court cases and a royal commission. the unit of study is a project, oci, which made major modifications to the social welfare administration system operated by a large australian government agency, which system is a psis that involves many stakeholders, many of whom are external users and usees. by adopting a socio-technical approach, a technical view of the project is avoided. a technical approach would have a strong focus on automation and overlook or even preclude humans-in-the-loop. 3.3 data collection although quantitative data needs to be used to the extent practicable, research of the nature of this study is dependent largely on qualitative data, employing qualitative analysis techniques (creswell & creswell, 2018). the study accordingly drew on multiple primary and secondary sources of evidence, listed in table 1. a royal commission and even senate committees can rely on gaining access to substantial and diverse sources of evidence inclusive of (1) holding formal public and private hearings (2) issuing notices to organisations and individuals to make statements or provide documentary evidence (3) conducting interviews with stakeholders, (4) commissioning primary research, and (5) inviting public submissions. preference was given to the most authoritative sources. where no such sources were available, sober news media have been cited to supplement the australasian journal of information systems clarke, michael & abbas 2024, vol 28, research article robodebt: socio-technical case study of is failure 12 evidence. the data collection and analysis process commenced in january 2018, and was conducted iteratively as sources became available. the final report of the royal commission in july 2023 confirmed and provided some further articulation of that analysis. stylistically, the narrative relies on thick descriptions and elucidations (ponterotto 2006). official sources of evidence (in chronological order) 1 report by the australian ombudsman, april 2017 (omb 2017) 2 report by the senate community affairs references committee (sca), june 2017 (sca 2017) 3 response by the government to sca's 2017 report (gr 2017) 4 prygodicz v commonwealth of australia (no 2) [2021] fca 634 (pry 2021) 5 report by the sca, may 2022 (sca 2022) 6 report of the royal commission into the robodebt scheme, july 2023 (rcr 2023) table 1: primary sources the first published report was on investigations by the ombudsman (omb 2017). it was limited to the purported debts raised and excluded consideration of the policy rationale, the debt raising and recovery program, procedural fairness, and the use of external debt collection agencies. the ombudsman's report was expressed in a manner noted by media reports to be very sympathetic to the agency and the project, despite the litany of problems that it uncovered. this was further evidenced by the very positive reaction by dhs to the report (omb 2017, pp.45-46), and use of the document by that agency as a major part of its defence against the subsequent senate committee report (gr 2017). the royal commission found that "dhs and dss [senior executives] engaged in behaviour designed to mislead and impede the ombudsman in the exercise of his functions" (rcr 2023, p.208). a lengthy description of the two agencies' exercise in regulatory capture of the ombudsman is on pp.208-226. the conclusion was that "... the circumstances ... give rise to a reasonable perception that the ombudsman’s office conducted [that] investigation in a way which allowed dhs to influence the content of the resulting investigation report in order to further dhs’s own interests, thus compromising the independence of the investigation" (p.585). the ombudsman's report did, however, provide some information that had previously been suppressed, and it did result in corrections to some of the multiple small-scale deficiencies. the second authoritative source is a senate committee report (sca 2017). the senate committee had a much broader terms of reference than the scope of the study undertaken by the ombudsman, and encompassed the scheme's error rates, its compliance with legal, regulatory, and public policy requirements, and its impacts. in australia, the government is formed in the lower house of representatives, whereas the committee reflected party representation in the senate, with a mix of government and opposition members and a chair from the cross-bench greens party (whose policies, other than in relation to environmental matters, can be depicted as being in the social democratic tradition). however, that first major report elicited an entirely dismissive response from the government (gr 2017). subsequent sources, of less value for the purposes of the case study, include a series of reports from a second inquiry by the same senate committee, in 2020-22, including the final report (sca 2022). australasian journal of information systems clarke, michael & abbas 2024, vol 28, research article robodebt: socio-technical case study of is failure 13 the third key source is the judgment of a senior court in a subsequent court-case (pry 2021). in addition, some publicly available sources from the agency itself were used, together with media reports that provided information that assists in understanding the business process and its outcomes. following the demise of the scheme, and a change of government, a robodebt royal commission (rrc) was established. transcripts and some additional documentation emerging from the rrc during 2022-23 shed additional light on a number of aspects, as did the commission's final report (rcr 2023). case studies, methodologically, can also be considered “stories of change” (simister 2017). the following section presents the relevant aspects chronologically, from early 2015 to late 2023. presentation as a narrative allows an anticipatory tone to delve more deeply into particular elements of the story that shed light on the events. the material will show that, as the project unravels, the hopeful application of technology to help the government improve its promise to australian taxpayers gives way to: (1) questioning how fundamental flaws went unnoticed and/or unaddressed for so long, (2) an emergent understanding of staff toil and the human toll, and then (3) widespread questioning about the apparent absence of accountability in administration. 3.4 data analysis this article uses a chronological timeline to report on significant events throughout the decade during which the oci project was conceived, articulated, implemented, operated, withdrawn and autopsied. the chronological approach enables a ‘before-during-and-after’ sense of the scheme that is emergent, but also facilitates an anticipatory manner. it provides a clear view of the existing mode of operation, and of the design of the future mode of operation, and even allows for backcasting (dreborg 1996) to the time when the degree of automation was much more limited (abbas & michael 2022). events were analysed using the socio-technical systems approach, whereby the interplay is considered among social, technical and environmental aspects of a psis (abbas et al. 2021). for example, where users and usees were discussed, this was always as part of the social subsystem within the context of the technical subsystem and the environment. when automated decision-making is referenced, it is in the context of its legality and/or the context of the humans it impacted when the algorithm sent debt recovery letters out automatically or inferred a person should be considered a welfare cheat (nikidehaghani et al. 2023). this article offers the most comprehensive reference list and bibliography on the robodebt case study, and its tabular and visual representations assist data analysis by capturing and summarising key aspects. in addition, the authors' comments in this case study draw on their backgrounds not only as academics, but also as practitioners, and in two instances also as contributors within consumer advocacy organisations. 4 case study analysis this section traces the history of the online compliance intervention (oci) project, in the chronological sequence outlined in table 2. further details and analysis are available in the working paper from which this article was developed (clarke et al. 2024). the sources use the terms oci and robodebt interchangeably. however, the agency used various names over time for various aspects and stages of the overall project, including welfare payment infrastructure transportation (wpit), strengthening the integrity of welfare payments australasian journal of information systems clarke, michael & abbas 2024, vol 28, research article robodebt: socio-technical case study of is failure 14 (siwp), employment income confirmation (eic) and check and update past income (cupi). year activity 2014 jun – dhs minute: 'concepts for future compliance activity' nov – internal legal advice the scheme is unsupported by law 2015 may – budget appropriation july – two-stage pilot scheme 2016 july – initial implementation september – full roll-out december – surge of media coverage 2017 ongoing media coverage and public anger continued illegal and harmful operation april, june – first formal reports 2018 continued illegal and harmful operation 2019 continued illegal and harmful operation february – first successful court challenge november – second, decisive court challenge november – announcement of the scheme's withdrawal 2020 may – government acceptance that the method was unlawful july – commencement of repayments 2021 november – withdrawal of the defence against a class action 2022 may – change of government june – finalisation of the court case august – commencement of the royal commission september – completion of repayments to clients october – cancellation of the outstanding 200,000 debts 2023 march – completion of the royal commission hearings july – publication of the royal commission report table 2: timeline of the robodebt project this section commences by summarising relevant predecessor activities. it then outlines the project's claimed and apparent purposes and the design features intended to fulfil them. the implementation and operational phases are described, followed by evidence of the scheme's impacts, and – despite the government's sustained denial of the scheme's misconception – its deficiencies, illegality and harmful effects, followed by its eventual demise and the subsequent repercussions. the scope of the case study is very broad, because an intervention of this scale cannot be understood by limiting the field of view to it, nor to the intraand inter-organisational elements of the is, and instead needs to encompass all environmental factors that were material to the process and outcomes. 4.1 prologue: data matching – 1990 to 2014 data is gathered by dhs from its clients. dhs also has substantial powers to demand information from other sources, such as organisations for which welfare recipients perform work and from which they receive payments. in addition, using specific authority enacted in 1990, "dhs began data matching activities in 1991" (omb 2017, p.5, fn7), and it "has conducted [pay as you go] payg data matching activities with the [australian taxation office] ato since 2004 ..." (p.5). australasian journal of information systems clarke, michael & abbas 2024, vol 28, research article robodebt: socio-technical case study of is failure 15 "income support payments are subject to an income test which means that a recipient's fortnightly payment may be reduced once their income reaches a specific threshold" (sca 2017, p.13). the purpose of data matching with ato is to check data provided to dhs by clients against that available through the taxation system: "payment recipients must report their income fortnightly ... where it is found that a recipient has incorrectly reported their fortnightly income, and the correct amount would have affected their entitlement to a payment, a ... debt may be raised" (sca 2017, p.14) ). discrepancies may indicate that overpayments have occurred, many of which are known to be minor. many discrepancies result from errors, variously by the client, employers, and dhs. some are fraudulent. prior to 2016, investigative work conducted by dhs staff had given rise to over 20,000 compliance notices annually (omb 2017, p.5). however, other evidence suggests that only about onequarter of those clients were eventually found to be in debt (sca 2017, p.17), suggesting that the outcome was of the order of 5,000 enforced debt notices p.a., involving perhaps 0.2% of the client-base of about 2.6 million people. although the matching itself had long been automated, the process as a whole in early 2016 continued to involve a considerable amount of manual work that had been found to be necessary to reconcile disparate sources of data. in the spirit of digitalisation, dhs initiated a project with the intention of reducing the manual effort through further application of technology. at the heart of the project was apportionment of the ato's annual data into 26 equal parts, to provide a fortnightly equivalent. this arrangement, based on the implicit assumption that each client earned income at a steady rate through the year, was commonly referred to as 'income averaging'. 4.2 conception – 2014-2015 "robodebt originated as an idea from within the customer compliance branch of the department of human services (dhs)" and was articulated across the 2014-15 financial year (rcr 2023, pp.27-118). the government's budget media release of 12 may 2015 declared that "the government has zero tolerance for rorting of our welfare system. [the term 'rorting' is australian vernacular, meaning to take unfair advantage of a public service, in particular to indulge in mostly-small-time cheating on tax or welfare.] we will put a strong welfare cop on the beat focusing on deterrence, detection, investigation and prosecution to track down suspected welfare fraud and non-compliance. from 1 july 2015, the government will increase dhs’s capability to detect, investigate and deter suspected welfare fraud and non-compliance ... it will achieve net savings of around $1.5 billion" (mrr, 2015). evidence to the royal commission 7 years later showed that the legal branch of the policy agency, dss, had provided advice as early as november 2014 that the use of income averaging in the manner proposed "would not be supported by law" (rrc 2022c, p.13491351). the royal commission found clear evidence that the policy and operational agencies, dss and dhs, were both pressured by the minister ahead of the 2015 budget, and that the chief executive of dhs and at least three others of its most senior executives contrived to make it appear that the income averaging approach was lawful. further, the royal commission found that the executives and the minister, in march 2015, had deliberately misled the expenditure review committee of cabinet (rcr 2023, pp.57-107). the scheme was approved by cabinet, and proceeded to design and implementation. australasian journal of information systems clarke, michael & abbas 2024, vol 28, research article robodebt: socio-technical case study of is failure 16 what the idea of 'a strong cop on the beat' meant in concrete terms only slowly became publicly apparent. according to the ombudsman's report, the main efficiencies were to be gained by (omb, 2017, p.5, headings interpolated): • [costless outsourcing of data collection to clients:] dhs no longer using its information gathering powers to request information directly from third parties, such as employers. under the oci, it is now the customer’s responsibility to provide this information; • [automation of debt generation and the despatch of debt notices:] the oci system automatically sending letters to tell customers about the income discrepancy; • [automation of internal processes:] moving much of the debt management and calculation process online; and • [costless outsourcing of data entry to clients:] customers entering their information directly into the oci system. a dhs deputy secretary stated in evidence to the senate committee that "we are trying to eliminate the need for anybody [on dhs staff] to [acquire information from the employer] so that it is easier [for dhs]" (sca 2017, p.16). the efficiencies were argued to encompass reductions in labour costs significantly greater than the ongoing technology costs, combined with substantial improvements in the identification and recovery of overpayments. together, these resulted in "forecast ... [aud] 3.7billion worth of [net] savings ... over four years from 2016–17" (sca 2017, p.4). the interpretation of the community and public sector union was that: "[oci] is a fairly obvious consequence of a department that no longer has the resources to provide effective services. the decision to replace the human oversight of debt recovery with automated data matching was absolutely based on a desire and an imperative to save money" (sca 2017, p.19). 4.3 the oci business process – 2015-16 a brief description of the core of the online compliance intervention scheme was provided in the ombudsman's report (omb 2017, p.4): [after] the oci matches the [fortnightly] earnings recorded on a customer’s centrelink record with historical pay as you go (payg) income data from the australian taxation office (ato) [which was the total earnings for a lengthy period, in many cases a full year] ... customers are asked to confirm or update their income using the online system. if there is a discrepancy in the data match, this can result in a debt the customer must repay. parts of the debt raising process previously done manually by compliance officers within dhs are now done using this automated process. three aspects of that description are noteworthy. firstly, it misrepresents the 'confirm or update' step which actually required the 'customer' to submit very specific evidence in relation to income earned years earlier, which in many cases needed to be first acquired from their then employer(s). secondly, it omits mention of the central feature of the process, which was the agency's use of 'income averaging'. although omb (2017) mentioned 'averaging' 43 times on 15 of its 113 pages, it did not examine the process in detail, and relegated the explanation to a footnote (fn.13, p.7): australasian journal of information systems clarke, michael & abbas 2024, vol 28, research article robodebt: socio-technical case study of is failure 17 'averaging' refers to the practice of treating income as if it was earned at a consistent rate over a total period of employment rather than applying the precise amounts against the fortnights in which the income was actually earned. dhs currently applies averaging where a person accepts the payg data or does not enter data for all fortnights. the ombudsman's report concluded that (p.7): we are satisfied that if the customer is able to collect the income information required [sic: the requirement was not merely to collect "income information" but to gather and submit very specific supporting evidence] and enter it properly into the system, the oci is capable of accurately calculating the debt. this accepted the inversion of the onus of proof, which was argued by many to be at least inappropriate and unfair. it also failed to address the most critical circumstance, representing what appears to have been about 80% of cases, in which the benefit recipient could not gather and submit acceptable evidence. a third serious inadequacy in the ombudsman's report is its vagueness about the period over which each client's income from each employer was averaged. it says only that it was "over the period the employer told the ato the customer worked for them" (omb 2017, p.34). however, it was clear to the senate committee (sca 2017, p16) that: ... employers must only provide [to ato] an annual figure paid during that financial year ... [and only about half] of records [from employers] were for a full year employment. it therefore appears very likely that, in a great many cases, erroneous periods were used as a basis for the averaging calculation. the senate committee and a regional social services advocacy organisation (actcoss) identified multiple further examples of problems with 'averaging', and the senate committee concluded (sca 2017, p.35) that: [this is a] labour market in which people are trying to work and comply with their centrelink requirements, ... a market in which people get bits and pieces of work; work irregular hours and often spend periods of time across a financial year out of the workforce. this leads to it being way more complicated and extremely onerous to comply with a ... system that assumes that people either have or do not have a job across a financial year. the 'income averaging' approach was clearly not a suitable basis for any administrative decision-making, let alone where those decisions impose punitive measures, let alone in relation to vulnerable populations, let alone by automated means unchecked by any appreciation of reality and by 'common sense'. importantly, dhs's own 'operational blueprint' acknowledged "difficulties", including "if employment is for a part of a year only" and "if income varied greatly during the year" (omb 2017, p.42). the agency also tacitly acknowledged that the "difficulties" were severe, in that, during the 30 years of use since the early 1980s, its application had been "limited to last resort situations" (omb 2017, p.42). given that the agency was well aware that averaging was suitable only for "last resort situations", it is difficult not to infer culpability on the part of the dhs executives in adopting the technique. further, evidence before the royal commission showed that the ato had, in 2017, asked dhs to "cease and desist" using taxpayer data for the robodebt scheme (wong 2022). the request was ignored; but the ato continued to provide its data. australasian journal of information systems clarke, michael & abbas 2024, vol 28, research article robodebt: socio-technical case study of is failure 18 the use of 'income averaging' was unlawful, "because social security legislation [requires] a person’s income support entitlement to be calculated on the basis of the income that they had actually 'earned, derived or received' in [the preceding] fortnight. the average of someone's income over a period was no indication of what they had earned in any particular fortnight, unless there was also evidence that they had earned their income at a regular fortnightly rate over that period" (rcr 2023, p.37, emphasis added). the unlawful behaviour stemmed from actions either side of christmas 2014, in which three senior executives suppressed unwelcome legal advice that would have de-railed a policy measure strongly desired by a rising-star minister – who was promoted to treasurer nine months later and became prime minister 3 years after that (rcr 2023, pp.37-50). the federal court judgment was scathing (pry 2021, pp.3-4, emphasis added): [i]t is fundamental that before the state asserts that its citizens have a legal obligation to pay a debt to it, and before it recovers those debts, the debts have a proper basis in law. ... having regard to that, and the profound asymmetry in resources, capacity and information that existed between them and the commonwealth, it is self-evident that before the commonwealth raised, demanded and recovered asserted social security debts, it ought to have ensured that it had a proper legal basis to do so. the proceeding revealed that the commonwealth completely failed in fulfilling that obligation. its failure was particularly acute given that many people who faced demands for repayment of unlawfully asserted debts could ill afford to repay those amounts. the royal commission concluded that "[t]he position that income averaging was a longstanding lawful practice was so entrenched within dhs that lawyers at all levels were unable to question it in accordance with their professional obligations" (rcr 2023, p.521). the oversight agency, the office of legal services coordination (olsc), also failed to perform its function, in that "there was cause for alarm within olsc that dhs considered there were no legal issues and that there was no legal advice pertaining to the scheme" (pp.537-538); but olsc took no action. a simplified model of the primary process is in figure 2. the report (omb 2017) provides more detailed descriptions of the manual system (p.31), the pilot (p.32), the interim business process from 2015 (p.33), the live process july 2016 to january 2017 (pp.33-35), and the changes made in february 2017 (pp.35-38). until july 2023, none of the authoritative sources made clear whether oci applied to every means-tested benefits scheme administered by dhs (as could have been inferred from the expression "income support payments" – omb 2017, p.33), or only some of those schemes (as implied by government statements in january 2017 that the scheme was to be later extended to age and disability pensions). dss later provided a list of 15 payment types that were "at different times" subject to the scheme. "newstart allowance and youth allowance formed the bulk of the payments" (rcr 2023, p.9) – despite it being known that only 2-6% of those recipients had stable earnings flows. all types were subject to policy decisions by dss, but two, austudy and abstudy, were managed jointly with education. to submit the required information and evidence, recipients were required to register with an intermediary website called mygov (omb 2017, p.34). no telephone contact-point was provided. the notification included the statement that dhs "will update [customer] details using the enclosed employment income information" (p.34). this represented less than full australasian journal of information systems clarke, michael & abbas 2024, vol 28, research article robodebt: socio-technical case study of is failure 19 disclosure of the agency's use of that data to compute a liability by the client to dhs, and demand return of the claimed overpayment. further details were later published (deloitte 2023a, 2023b), but the description remained incomplete. the royal commission concluded that "[t]here may be inaccuracies in the process maps and report leading to the possibility of inaccurate conclusions. if so, they are likely to be the product of dhs’s haphazard and inconsistent documentation of its processes" (rcr 2017, p.471). figure 2: simplified model of the oci process a striking feature of the authoritative sources used and cited in this case study resource is the near-invisibility of is professionals and the managers and junior executives responsible for their work. from the publicly-available evidence, it appears as though the entire process of project framing, conceptual design, implementation and operation, and perhaps even the more technical steps of detailed design and coding, were performed and managed by public servants who had limited is-professional background. if that is the case, the process lacked key insights that is professionals could have provided. the absence of involvement may be indicative of low regard by executives for their staff and contractors, or for is professionals generally. alternatively, executives may have anticipated unwelcome advice, and even pushback, and hence marginalised is professionals' input to the project. australasian journal of information systems clarke, michael & abbas 2024, vol 28, research article robodebt: socio-technical case study of is failure 20 4.4 implementation and operation – 2016-19 dhs rolled out the measure in three stages (omb 2017, p.6): 1. from 1 july 2015 dhs introduced an ‘interim process’. it began applying the same approach manually that the oci was being designed to do automatically. dhs investigated 100 000 discrepancies during this phase. 2. from 1 july 2016 dhs began rollout of the oci, starting with a limited release of 1000 cases into the oci where there were income discrepancies. 3. from september 2016 dhs commenced rollout of the fully automated oci. "the initial group ... included those who received income support payments in the 2010 to 2014 financial years" (omb 2017, p.33). the staged launch occurred during the first half of the july 2016 to june 2017 financial year, so the demands related to periods as far back as july 2009. no information has been found showing the distribution over those years. dhs was aware that many of the targets "were no longer centrelink customers" (p.33). no evidence has been found as to whether dhs estimated the proportion whose contact details were no longer current, many of whom would therefore quite probably not have received the letter. for the first year of operation at least, it appears that dhs did not monitor for evidence of non-receipt and did not handle those cases any differently from current clients. each recipient was forced to think back to the relevant period(s), and search for specific, old documents. if they failed to find the documents, they needed to contact one or more employers and request copies. the relevant names of the documents varied depending on the employer. the recipients may have had no contact with those employer(s) for quite some time, and they had no standing to demand compliance by those employer(s). the minimum period they had to go back was 2¼ years, but the longest look-back required was 7¼ years, to july 2009 (i.e., the beginning of the 2010 financial year). particularly where the elapsed time was long, some employers may have no longer had access to the data, and some had ceased business operations. it is unclear whether dhs had any evidence about, or did any studies using, for example, its own staff, or randomly selected members of the public, let alone welfare recipients, in order to establish the extent to which the scheme may have demanded from each benefit recipient an improbably high level of intellectual capacity, online expertise, assiduousness in record-keeping and negotiation skills with ex-employers. these notices were sent in an automated manner, without a government employee or contractor in-the-loop. the number of debt notices skyrocketed from 20,000 per annum to 20,000 per week (sarah 2016). rather than causing alarm to the government, it was trumpeted as a success that, after just four months of the scheme, in january 2017, 169,000 debt notices had been sent to some of australia’s most vulnerable individuals. it was credited by the government as having already "identified close to [aud] 300 million in overpayments to welfare recipients" (belot & mcghee 2017) – although the minister at the time referred to that figure, apparently erroneously, as having been recovered rather than merely identified (belot 2017a). it was also announced that the government was considering expanding the oci beyond jobseeker unemployment benefits to age and disability pensions (belot 2017b). many recipients were unable to get through to the overloaded call-centres, with many reporting delays of 3-6 hours before a call was answered. during some periods the telephone system was overwhelmed, and calls did not even reach the automated-response system. evidence has subsequently emerged that centrelink and other major agencies scale their callaustralasian journal of information systems clarke, michael & abbas 2024, vol 28, research article robodebt: socio-technical case study of is failure 21 handling facilities so far below demand-levels that 15-25% of all cases are blocked (parnell 2024). in addition to clients, carers are a further important example of callers who are greatly inconvenienced by such problems. many of those who live in outer regional, rural, and remote areas, which includes about one-third of welfare-recipients, have to travel long distances to centrelink offices. in regional and suburban offices alike, people have to wait in long queues to talk to under-resourced and under-informed counter-staff. those queues were even longer during the robodebt period, 2016-19. where the alleged debtor was no longer a benefit recipient, recovery could not be achieved by deducting from future benefits payments. in those, many cases, dhs used private-sector debt collectors. the contracted firms included one that was soon afterwards successfully prosecuted for coercion, false representations and unconscionable conduct when dealing with a consumer (bajkowski 2022). data provided to the royal commission suggests that the "total number of robodebts referred to external debt collectors" was 305,977 (rcr 2023, p.507), of a total of 794,000 debts in all (p.402), or 38.5%. the total value of those referrals was aud 955 million (54.5% of the aud 1,751 million total), and the recovery-rate that was achieved appeared to be about 11.2%. "in the 2016 calendar year, 126,571 [individuals] ... were sent debt notices under the oci" (omb, p.4, fn5), with the vast majority sent in the 4 months from september to december. however, as early as the beginning of december, after only 2 months of the main rollout, complaints about the scheme were rife (tt 2016), and by late december it was public knowledge that the basis on which oci was built was grossly flawed and the error-rate very high (knaus 2016). despite the annual summer slow-down from early december 2016 to late january 2017, media pressure built rapidly. the term robodebt was coined, and widely adopted by no later than mid-january 2017. the ombudsman felt it necessary to commence a study, dhs felt the need for some token changes, and a senate committee lumbered into existence. some changes commenced in late 2016 and early 2017, but they were limited to corrections of obvious blunders in the original detailed design, and not corrected until 6-9 months after rollout commenced. these related to: (1) the absence of the phone number from the letter; (2) exception handling where dhs was aware that the letter had not reached the target person; (3) extension of the 21-day response-time to 28 days; (4) acceptance of bank statements in lieu of payslips; (5) fixing of errors in the ui/ux design; (6) the pausing of recovery action while a matter was under internal review (not previously implemented, in breach of fair process); and (7) easing of the circumstances in which a 10 percent debt recovery fee was automatically applied (omb 2017, pp.35-38). the correction of greatest substance was the belated recognition that (8) complex cases are unsuitable for automation and need to "fall out for manual intervention" (p.36). the government drew even more criticism when, in an endeavour to discredit a complainant who had gone public, the minister at the time released personal data about that individual from dhs records. "[o]ne of australia’s leading criminal barristers [wrote] "it is reasonably clear that the minister or one of his office’s staff has committed an offence ... punishable by up to two years imprisonment if proven in court" (knaus 2017). no enforcement action was ever undertaken. australasian journal of information systems clarke, michael & abbas 2024, vol 28, research article robodebt: socio-technical case study of is failure 22 4.5 collapse – 2017-23 by april 2017, the scheme's problems were well-understood and could be clearly explained to the senate committee (barbaschow 2017). the senate committee’s first report, published in june 2017 (sca 2017, pp.41, 48, with specifics documented on pp.43-48 and pp.56-59), was scathing: "communication problems included letters not being received, trouble contacting the department via phone, difficulty in receiving intelligible income data used to calculate purported debts, hard to navigate online communication portals, difficult to understand correspondence and a lack of material translated into other languages". ... the committee notes it is clear there is a significant communication problem when 65,000 from 300,000 people do not respond to requests from the department to engage". inadequacies under the heading 'communication barriers' were listed as vulnerability flags, including literacy, english as a secondary language, disability-related communication barriers, geographic barriers, and difficulties people faced with all three communication channels (sca 2017, pp.48-55 and pp.59-69). the senate committee concluded (p.69). that the key concern was the outsourcing of the income checking process to individuals, thereby reversing the onus of proof: ... the department claims an income discrepancy and requires an individual to seek the information required to prove the discrepancy does not exist. if the individual fails, they will owe a debt of potentially many thousands of dollars to the department. the two fundamental resources a person needs to undertake this process is a method of communicating, and once that communication channel is opened, the receipt of information that is both comprehensive and comprehendible [sic]. the department is clearly failing to provide those two necessary tools to allow people to challenge the income discrepancy, and is reaping the benefit through debt payments. inadequacies were also noted in the process to clarify or review a purported debt, the policy on handling queries and its impacts, the challenges posed by reversing the onus of proof, the impact on community legal centres, and even in the subsequent process improvements made by the department (sca 2017, pp.71-89). the senate committee also noted the absence of any obligation on dhs to comply with any guidance in relation to the use of debt collectors, or even to take any responsibility for malpractice by them (sca 2017, pp.29-32). remarkably, it took a further 2½ years before the government accepted defeat. it had taken no corrective action, despite multiple tribunal decisions made against it, 10 of which were decided prior to march 2017, within 6 months of the scheme's commencement (rcr 2023, pp. xviii, 239-240, and appendix 9). dhs was initially assiduous in avoiding test-cases reaching the federal court. however, public interest law firms persisted in their endeavours on behalf of welfare recipients, and the agency appears to have inadvertently let two cases, masterton and amato, slip through to the courtroom. this resulted in a clear judicial statement of illegality (amato 2019, released on 27 november 2019. see also rcr 2023, pp.288-317). the imminent judgment forced the then minister to, on 6 november 2019, instruct dhs to cease raising debts "where the only information we are relying on is our own averaging of australia taxation office income data" (farrell 2019). an internal memo australasian journal of information systems clarke, michael & abbas 2024, vol 28, research article robodebt: socio-technical case study of is failure 23 unsurprisingly leaked to the media, forcing a rushed media release – which was reported in the media, but suppressed from the minister's media site (robert 2019). it then took a further 6 months, until may 2020, for the attorney-general to concede that "all centrelink debts raised using the 'income averaging' method were unlawful" (henriques-gomes 2020). a class action was mounted. in breach of the agency's formal public policy obligation to be a 'model litigant' (lsd 2017), dhs resisted the action until immediately before the hearing commenced, whereupon it conceded the illegality of its actions and all claims – although for the pitifully inadequate sum of aud 100m, representing aud 250 per member of the class action, plus aud 12 million in legal costs. the agency agreed to settle the class action only in november 2020, with judicial approval further delayed until june 2021 (pry 2021, sca 2022 p.6). repayments and payments were not completed until the end of september 2022, of the order of 5 years after the agency had unlawfully extracted the payments from its clients under duress. the senate community affairs reference committee commenced a second review of oci in july 2019. it published a succession of five interim reports from september 2020 onwards and a final report in may 2022. the last of these (sca 2022) was entitled 'accountability and justice: why we need a royal commission into robodebt', and was headlined 'a massive failure of public administration'. the coalition government was defeated in the election of may 2022, and the new labor government quickly moved to establish a royal commission, in august 2022. it had been widely understood that the then coalition government had cancelled all oci-generated debts in mid-2020. however, it was reported in the media in mid-october 2022 that the new labor minister had announced the cancellation of about 124,000 cases still under review and another 73,000 where potential debts had been identified but the clients had not been informed (brookes 2022). the final steps in the ill-fated project occurred during 2023, with the finalisation of refunds and the publication of the royal commission's report (rcr 2023), which included a sealed chapter making a considerable number of referrals of persons for further investigation by four other bodies. 5 findings 5.1 summary of findings dhs had projected that the number of overpayment reviews processed each year would leap almost forty-fold from 20,000 to 783,000 (omb 2017, p.5). yet the agency appears not to have realised that this would give rise to very large increases in the numbers of transactions conducted not only in the online system, but also at the call-centre and physical servicecentres. it might have been expected that additional staff would have been assigned, particularly for the inevitably fraught transitional phase. in fact, it appears that the agency was forced to comply with government-imposed reductions in headcounts from the outset, such that call-centre and front-counter staffing was lower than it had been prior to the scheme being launched. if human resources had been involved in cross-checking notices, as had been the case before the introduction of robodebt, the volume would have been internally obvious, very likely stimulating precautionary reviews. the staff union drew attention to the "classic false economy" and the "costly reverse workflows where staff are taken offline to deal with complex and difficult disputes over incorrectly raised automated debts" (sca 2017, p.19). staff dissatisfaction, embarrassment, and stress were widespread, as a result of having to deal with clients who were generally illaustralasian journal of information systems clarke, michael & abbas 2024, vol 28, research article robodebt: socio-technical case study of is failure 24 informed and alarmed, and in some cases rude, depressed or disturbed (belot 2017a). the union submitted that "[oci] has been an absolute disaster for many centrelink use[r]s and also for the workers charged with implementing a system they know to be deeply flawed and unfair" (sca 2017, p.18). both the ombudsman's and senate committee's reports detailed many personal accounts of the stress and distress caused to the benefits-recipients subjected to it, with repeated mentions of "feelings of anxiety, fear and humiliation". "individuals had spent hours finding the required pay slips and bank statements, some dating back to 2010-11 [sic: 2009-10], often for centrelink to find that no debt was owed" (sca 2017, p.38). media reports, tribunal findings, court judgments and the royal commission transcripts are littered with evidence of the harmful and in some cases devastating impacts on recipients of the mostly erroneous and in all cases non-lawful demands. some of the tales were harrowing. for example, a ms miller gave evidence to the royal commission in relation to her son rhys cauzzo, who committed suicide on 26 january 2017 after being pursued by dhs and then a debt-collector: "he was very distraught when he made that phone call to me. and and he said he was feeling suicidal. and so we probably spoke for about two hours on the phone, maybe longer". the five letters of demand induced him to draw "a face with a gun through there and dollar signs just coming out of his head. and it had 'debt life' written on it, i think" (rrc 2023a, pp.p-3256, 3259-60). on the basis of smatterings of such little data as ever emerged, primarily in media reports and as asides in verbal evidence, it appears that in only about 40 percent of the cases in which dhs commenced an action did the agency pursue the debts in full, with about 20% of debts withdrawn because they were clearly wrong, about 20% reduced, and about 20% deemed not worth pursuing. it later accepted that even the 40% that it pursued in full and the 20% in part had to be repaid, because there was no legal basis for raising the purported debts. dhs appears not to have invested in extraction and analysis of evidence about the proportion of the purported debtors and purported debts that may have been actually legitimate. it is possible that the proportion of over-payment due to errors and fraud is greater than the 0.1% estimated by advocates, perhaps greater even than the 0.4% that dhs has claimed in the past. on the other hand, it appears that the scale of the undiscovered overpayments is a great deal less than the 1.0-2.5% that the government postulated when it announced the scheme, and the even larger proportion that it claimed to have discovered during the period the oci operated. the senate committee's report noted that over aud 1.7 billion in illegitimate debts were imposed on 433,000 people, of whom 381,000 individuals were pursued, many through private debt collection agencies, resulting in the repayment of over aud 750m to the commonwealth. the debts that had been raised were later withdrawn, the payments made in the meantime were eventually refunded, and a further aud 100m was paid to individuals in damages and aud 12m to the class action lawyers. further losses to the public purse included admitted costs of implementation of aud 600m to mid-2019 alone (burgess 2019); but many elements of the total cost remain shrouded because the government refused to disclose such information. beyond the financial damage, "the income compliance program impacted hundreds of thousands of people and, for many, resulted in devastating emotional and psychological australasian journal of information systems clarke, michael & abbas 2024, vol 28, research article robodebt: socio-technical case study of is failure 25 harm. it has undermined many people's financial security as well as their willingness to engage with and trust government services" (sca 2022, p.1; see also, braithwaite 2020). the federal court judgment included statements of a kind not commonly seen in such documents (pry 2021, p.7): one thing ... that stands out from the objections is the financial hardship, anxiety and distress, including suicidal ideation and in some cases suicide, that people or their loved ones say was suffered as a result of the robodebt system, and that many say they felt shame and hurt at being wrongly branded "welfare cheats". some of the objections were heart-wrenching and one could not help but be touched by them. the royal commission report discussed three instances of suicide that were brought to its attention (rcr 2023, pp.181-183), but how many suicides robodebt threats were associated with was the subject of speculation (medhora 2019): more than 2030 people died after receiving a [oci] debt notice ... of those, 429 roughly one-fifth were aged under 35. the figures cover a period from july 2016 to october 2018 ... there were 3139 deaths of people aged between 15 and 35 in 2016 overall. the objectives declared for this case study were to apply socio-technical systems theory in order to understand the factors, decisions, events, and corresponding dynamics that resulted in comprehensive project failure, and to present lessons for enhanced public sector information systems design. the following two sub-sections summarise the ways in which the case study fulfils those objectives. the case demonstrates how a largely technical view, motivated by a desire for cost-, wastageand fraud-reductions through digitalisation, condemned the project to failure. it also identifies established approaches whose adoption enables large-scale projects to succeed. key precepts include open systems thinking, humancentrism, the reflection of the interests of multiple stakeholders including both users and usees, and processes for detecting problems and adapting the project design to address them. 5.2 practical findings: socio-technical lessons based on the case study analysis and the summary of findings above, we summarise the major socio-technical lessons toward enhanced public sector information systems design. in the robodebt case, some primarily technical problems were identified. it was through a socio-technical examination, however, that key problems became apparent relating to (1) the dominance of political desires over rational management processes, (2) behaviours of the various stakeholders, (3) the absence of any apparent application of well-known design principles, (4) contextual considerations, and (5) program implementation. the following aspects appear likely to be relevant to the design and deployment of psis generally: human resource and staffing challenges; staff dissatisfaction; useror clientrelated concerns; technology-related perceptions; challenges to and misalignment with objectives / values; legal considerations; external stakeholder-related factors; media-related considerations; and financial implications. this list demonstrates that technology-related considerations form but one facet of the broader system, and that socio-technical challenges and considerations need to be better understood with a view to informing both decision-making and design processes. this applies not only to practitioners but also to those who conduct practice-relevant is research. australasian journal of information systems clarke, michael & abbas 2024, vol 28, research article robodebt: socio-technical case study of is failure 26 disregarding such considerations results in inability to appreciate socio-technical dynamics and the implications for stakeholders, particularly external users and usees. those weaknesses undermine the making of informed choices about the evolution and improvement of the psis in its particular context, and increases the chances of psis failure. from the outset, dhs believed in the potential benefits of automation, and positioned the oci project as a way to reduce operational costs based on assumptions about datafication and digitalisation. this relied on the assumed capability of technology to deliver efficiency, and interpreted its deployment to be an imperative. that represented the abandonment of the basic tenets of human-centred design, which the agency had previously respected. any system design that is focused purely on cost reductions is likely to fail to reflect the interests of stakeholders and the enormous diversity of their contexts and needs. an overwhelmingly technical view dominated the conception, design and implementation of the oci scheme, resulting in very serious impacts and implications for users and usees, and ultimately in complete failure of the project. 5.3 theoretical implications in this article, we have applied a socio-technical systems approach to focus on a project that made substantial adaptations to a large-scale public sector information system. sociotechnical systems was used to appreciate the inter-working of it components with individuals and social processes. information and communication technologies have enabled, and even driven, transformation in government agencies and business enterprises, particularly since the mid-1990s. this has led to a contemporary context of open systems and porous organisational boundaries. understanding extra-organisational systems such as social welfare systems depends on the recognition of their socio-technical nature, and definition of the object of study as extending beyond internal and even external users to nonparticipant usees. oci was not conceived with the intention to help welfare recipients. the scheme's motivations were financial, in particular through anticipated labour-savings, the outsourcing of effort to welfare recipients, and increases in overpayment detection and recovery through automated processes. in the robodebt case study, we see the importance of looking beyond the government agency and its intra-organisational is, by incorporating inter-organisational dealings (particularly the relationships among government agencies), extending to extraorganisational activities (where external users interact with a government agency in relation to the award and administration of welfare payments), and encompassing non-participant usees (those indirectly affected by the welfare scheme through, for example, financial dependants of welfare recipients, and the social and legal advisers on whom they depend). socio-technical systems theory allows designers to visualise the complete system, and detect gaps and risks. we suggest that figure 1, which depicts external users and usees and their relationships to the boundaries of the organisation and the overall system, represents a significant extension of the bostrom & heinen (1977) model that is relevant to all is that have substantial reach, and particularly to all psis of large scale and scope. 6 future research the case study resources presented above provide researchers with a digestible overview of the mass of information unearthed over a period of five years, primarily by an audit agency, a parliamentary committee, tribunals, a senior court, and a royal commission. presentation australasian journal of information systems clarke, michael & abbas 2024, vol 28, research article robodebt: socio-technical case study of is failure 27 in narrative form has ensured that each item of information is seen within its contemporaneous context, and is hence less likely to be misconstrued than it would be if it were separated from the circumstances prevailing at the time. many themes are readily identified, some of them static features and functions, and others concerned with the dynamics of business processes. this article has presented the material within a socio-technical frame. it provides a platform whereby researchers, including ourselves, can readily identify themes to address, appreciate their nature, view them through a particular theoretical lens, access the detail in the sourcematerials using the citations provided, and thereby make deeper contributions to practice and theory than is possible from a primarily descriptive case study resource. examples of theories of potential relevance include stakeholder theory and within that stakeholder salience theory and researcher perspective theory; systems analysis / requirements elicitation theory; is management theory; strategic is theory; risk management theory; regulatory theory; new institutional economics; and virtue, consequentialist and deontological ethics theories. conducting analyses through such lenses might refute, refine or articulate the tentative socio-technical interpretations offered in this article. alternatively, analysts may adopt rather different approaches, such as comparison and contrast of this case against other instances of complex, large-scale, open socio-technical systems that should serve the public interest. research of these kinds is urgently needed. the increasing power of it has not been matched by advances in the quality of data, the quality of inferencing, and the quality of executive management of psis that apply it to data. both decision quality and rationality of organisational behaviour are under threat. this case study shows that serious problems can arise even where humanly-understandable, genuinely algorithmic data analytics are used. the much-touted, but opaque and a-rational magic of empirical techniques, such as ai/ml and generative ai, threatens to further lower the quality of data and inferencing, and to undermine replicability, auditability, correctability, and hence accountability. these risks are being exacerbated by the headlong adoption of forms of ai-supported digitalisation that are then extended to automated decision-making and automated action. it is abundantly clear that the agency responsible for the robodebt project did not have adequate ways of risk-managing the extension of even well-understood data and simple forms of inferencing to substantially automated decision-making and action. 'big data' consolidated from multiple, incompatible sources is inherently of far lower quality than carefully-curated, designed-for-purpose data collections. most forms of ai-supported inferencing, because of their empirical nature and opacity, are antithetical to the expectation of clear decision criteria rationally applied to data of adequate quality. a brave new world is being heavily promoted by technology providers, user organisations, governments and even researchers. yet these currently fashionable forms of ai preclude humanly-understandable explanations for decisions. if technological and marketing imperatives are blindly followed, effective management of even mainstream cases of error and misunderstanding will have to mature very rapidly from its current, seriously inadequate state, in order to avoid widespread harm, recriminations, and a collapse in public confidence in its institutions. beyond using specific theories as lenses through which to consider themes, researchers may also use the resources provided by this case study to identify implications for the sociotechnical systems approach more generally. there is evidence, for example, that partyaustralasian journal of information systems clarke, michael & abbas 2024, vol 28, research article robodebt: socio-technical case study of is failure 28 political factors, the values exhibited by individual ministers, and the priorities of senior executives, coupled with the near-invisibility in the case study resource of is professionals and the junior executives they report to, indicate that, in large-scale systems of this nature, the prospects even of stakeholder engagement, let alone participative design, may be entirely swamped by the exercise of power. in such circumstances, what guidance can is academics provide to is practitioners about compliance with their ethical responsibilities? 7 conclusion this case study has drawn on authoritative sources to analyse a large-scale exercise in transformative it applied to the administration of social welfare in australia, which was conceived 2014-15, operated 2016-19, dismantled in 2019-22, and subjected to multiple formal reviews 2017-23. the contribution of this case study is enhanced understanding of the challenges involved in digitalisation in the public sector, within a frame of socio-technical systems theory. the narrative description, and the authoritative sources and academic commentaries it provides access to, enable deep and multi-layered analysis of the factors that resulted in a large-scale, intentionally transformative project failing to achieve its objectives and causing enormous collateral damage. a massive deficiency at the heart of the scheme's technical design survived into live operation, resulting in serious harm for over 3 years. a tentative interpretation, based on the narrative and the evidence supporting it, is that one of the largest government agencies in a developed economy, with long and deep experience in the application of it to social welfare administration, was so beguiled by the promise of transformative digitalisation that it abandoned its responsibilities, and reverted to mindless, technologically-determined design, causing massive dislocation and harm to the most disadvantaged members of society. actions and inactions of senior executives caused a long series of process safeguards to fail. the case study is a moving picture from real-life that represents a refutation of the comfortable assumptions that standards, 'good practice', legal compliance and training ensure the avoidance of such is failures and resultant harm to parties affected by systems. alternative interpretations are of course possible. for example, whiteford (2021) concludes that "[robodebt] differs from other examples of policy failures in that it was intentional, and not the result of mistakes in design or implementation" (p.340). the agency prolonged the agony by refusing for 3 years to accept that the project was both disastrous in conception, design, execution and effect, and unlawful. at no stage did the decision-makers exhibit any respect for, or empathy with, the people on whom they were inflicting significant and unjustifiable confusion, effort, cost, and anxiety. the royal commission's summation was that "robodebt was a crude and cruel mechanism, neither fair nor legal, and it made many people feel like criminals. in essence, people were traumatised on the off-chance they might owe money. it was a costly failure of public administration, in both human and economic terms" (rcr 2023, p. xix). given that an australian government agency irresponsibly (mis)conceived, implemented and persisted with a large-scale, transformative it project, many observers have been disturbed about the absence of any direct accountability for the harm that was done. no material negative consequences were experienced by the relevant ministers or senior executives, even after gross misbehaviour had been comprehensively documented. details australasian journal of information systems clarke, michael & abbas 2024, vol 28, research article robodebt: socio-technical case study of is failure 29 relating to the accountability failures are available in the full working paper from which this article was developed. see clarke et al. (2024). the it deployed in support of robodebt was not sophisticated or new. data matching has been in widespread use for four decades. the same applies to algorithmic inferencing, particularly as simple as that used in this project. automated generation of letters, automated deductions from welfare payments, and automated referral of cases to outsourced debt collection services, have been simple to implement for many years. one of the lessons of this case study is that seriously inappropriate organisational behaviour continues to occur even where the technologies and business processes are relatively simple to comprehend and analyse, and decision criteria are transparent. the scope for more frequent, even more widespread, and even more harmful misbehaviour is greatly increased where data quality is low, inferencing processes are opaque, and business processes are more intensively delegated to unsupervised artefacts. the absence of accountability greatly exacerbates the problem. ministers and government executives, in australia at least, are aware that regulatory failure and the protection of the guilty are entrenched. with big data, opaque ai, and increased autonomy for computing-based systems, there will be ample ways for the perpetrators of future psis disasters to escape retribution. key deterrent safeguards are absent. where psis projects are permitted to be conducted in such a manner, public confidence in agencies, it and is is likely to be seriously harmed, and public scepticism, non-compliance and active opposition to initiatives are likely to become engrained, exacerbating the alreadyhigh frequency of large-scale is failures. references the primary case sources, comprising investigation reports and other relevant official documents, are listed in the appendix to the article. abbas, r. & michael, k. (2022). socio-technical theory: a review. in s. papagiannidis (ed) 'theoryhub book', at: https://open.ncl.ac.uk/theories/9/socio-technical-theory/ abbas r., michael k., michael m.g., perakslis c. and pitt j. (2022). machine learning, convergence digitalization, and the concentration of power: enslavement by design using techno-biological behavior. ieee transactions on technology and society 3, 2 (june 2022) 76-88, doi: 10.1109/tts.2022.3179756 abbas r., pitt j., & michael k. (2021). socio-technical design for public interest technology. ieee transactions on technology and society 2,2 (2021) 55-61 abbas r., michael k., sargent j.p. & scornavacca e. (2021). anticipating techno-economic fallout: purpose-driven socio-technical innovation. ieee transactions on technology and society 2,3 (september 2021) 111-113, at https://ieeexplore.ieee.org/iel7/8566059/9500078/09500093.pdfhttps://ieeexplore.ieee.org /iel7/8566059/9500078/09500093.pdf. akter, s., mccarthy, g., sajib, s., michael, k., dwivedi, y.k., d’ambra, j. & shen, k.n. (2021). algorithmic bias in data-driven innovation in the age of ai. international journal of information management, 60, 102387. australasian journal of information systems clarke, michael & abbas 2024, vol 28, research article robodebt: socio-technical case study of is failure 30 amato (2019). amato v the commonwealth of australia (federal court vid611/2019), at https://www.comcourts.gov.au/file/federal/p/vid611/2019/3859485/event/30114114/do cument/1513665. avison d.e. & wood-harper a.t. (1990). multiview: an exploration in information systems development. blackwell, 1990. bajkowski j. (2022). services australia handed $5.6m more work to robodebt collector panthera following accc prosecution. the mandarin, 3 november 2022, at https://www.themandarin.com.au/204378-services-australia-handed-5-6m-more-workto-robodebt-collector-panthera-following-accc-prosecution/ bajkowski j. (2023). jane halton links ‘chilling. effect of foi to robodebt’s 'frank and fearless advice' epic fail. the mandarin, 11 july 2023, at https://www.themandarin.com.au/224898-jane-halton-links-foi-to-robodebt-failure-offrank-fearless-advice/ barbaschow a. (2017). privacy foundation calls for human intervention in centrelink debt collection. zdnet, 2 april 2017, at https://www.zdnet.com/article/privacy-foundationcalls-for-human-involvement-in-centrelink-automated-debt-collection-process/ baumer e.p.s. (2015). usees. proceedings of the 33rd annual acm conference on human factors in computing systems (chi'15), april 2015. belot h. (2017a). centrelink debt recovery: government knew of potential problems with automated program. abc news, 12 jan 2017, at https://www.abc.net.au/news/2017-0112/government-knew-of-potential-problems-with-centrelink-system/8177988 belot h. (2017b). centrelink's controversial data matching program to target pensioners and disabled, labor calls for suspension. abc news, 17 jan 2017, at https://www.abc.net.au/news/2017-01-17/labor-calls-for-suspension-of-centrelink-debtrecovery-program/8187934 belot h. & mcghee a. (2017). centrelink debt recovery program to undergo changes following public criticism. abc news, 16 jan 2017, at https://www.abc.net.au/news/2017-01-16/government-to-change-centrelink-debtrecovery-program/8185366 berleur j. & drumm j. (eds.) (1991). information technology assessment. proc. 4th ifip-tc9 international conference on human choice and computers, dublin, july 8-12, 1990, elsevier science publishers (north-holland), 1991. bostrom, r. p., & heinen, j. s. (1977). mis problems and failures: a socio-technical perspective. part i: the causes. mis quarterly 1,3 (1977) 17–32, at https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=e092be3120ce51aaa3 5af673fabd07b6513e702e bower c.m. & christensen c.m. (1995). disruptive technologies: catching the wave. harvard business review (january-february 1995) 43-53. braithwaite v. (2020). beyond the bubble that is robodebt: how governments that lose integrity threaten democracy. aust j soc issues 55,3 (september 2020) 242–259, at https://static1.squarespace.com/static/5c05f8595cfd7901fc57139d/t/5f8a808b459ade460ac https://www.comcourts.gov.au/file/federal/p/vid611/2019/3859485/event/30114114/document/1513665 https://www.comcourts.gov.au/file/federal/p/vid611/2019/3859485/event/30114114/document/1513665 https://www.themandarin.com.au/204378-services-australia-handed-5-6m-more-work-to-robodebt-collector-panthera-following-accc-prosecution/ https://www.themandarin.com.au/204378-services-australia-handed-5-6m-more-work-to-robodebt-collector-panthera-following-accc-prosecution/ https://www.themandarin.com.au/224898-jane-halton-links-foi-to-robodebt-failure-of-frank-fearless-advice/ https://www.themandarin.com.au/224898-jane-halton-links-foi-to-robodebt-failure-of-frank-fearless-advice/ https://www.zdnet.com/article/privacy-foundation-calls-for-human-involvement-in-centrelink-automated-debt-collection-process/ https://www.zdnet.com/article/privacy-foundation-calls-for-human-involvement-in-centrelink-automated-debt-collection-process/ https://www.abc.net.au/news/2017-01-12/government-knew-of-potential-problems-with-centrelink-system/8177988 https://www.abc.net.au/news/2017-01-12/government-knew-of-potential-problems-with-centrelink-system/8177988 https://www.abc.net.au/news/2017-01-17/labor-calls-for-suspension-of-centrelink-debt-recovery-program/8187934 https://www.abc.net.au/news/2017-01-17/labor-calls-for-suspension-of-centrelink-debt-recovery-program/8187934 https://www.abc.net.au/news/2017-01-16/government-to-change-centrelink-debt-recovery-program/8185366 https://www.abc.net.au/news/2017-01-16/government-to-change-centrelink-debt-recovery-program/8185366 https://static1.squarespace.com/static/5c05f8595cfd7901fc57139d/t/5f8a808b459ade460acd28dc/1602912397885/beyond+the+bubble+that+is+robodebt+how+governments+that+lose+integrity+threaten+democracy.pdf australasian journal of information systems clarke, michael & abbas 2024, vol 28, research article robodebt: socio-technical case study of is failure 31 d28dc/1602912397885/beyond+the+bubble+that+is+robodebt+how+governments+that+ lose+integrity+threaten+democracy.pdf brennen s. & kreiss d. (2016). digitalization and digitization. international encyclopedia of communication theory and philosophy, october 2016, preprint at http://culturedigitally.org/2014/09/digitalization-and-digitization/ brookes j. (2022). 197,000 notices wiped from ‘shameful' robodebt scheme. innovation australia, 12 october 2022, at https://www.innovationaus.com/197000-notices-wipedfrom-shameful-robodebt-scheme/ bryson j.m. (2004). what to do when stakeholders matter: stakeholder identification and analysis techniques. public management review 6,1 (2004) 21-53, at https://ftms.edu.my/images/document/mod001074%20%20strategic%20management%20analysis/wk3_sr_mod001074_bryson_2004.pdf burgess k. (2019). robodebt centrelink compliance program cost $600m to recoup $785m so far. the canberra times, october 28 2019, at https://www.canberratimes.com.au/story/6407984/four-of-five-robodebt-reviews-hitwith-a-debt-as-cost-revealed/ carney t. (2019). robo-debt illegality: the seven veils of failed guarantees of the rule of law?. alternative law journal 44,1 (2019) 4–10, at https://journals.sagepub.com/doi/pdf/10.1177/1037969x18815913 checkland p. (1981). systems thinking, systems practice. wiley, 1981. cherns a. (1976). the principles of sociotechnical design. human relations 29,8 (1976) 783792. cherns a. (1987). principles of sociotechnical design revisited. human relations, 40,3 (1987) 153-161. christensen c.m., raynor m. & mcdonald r. (2015). what is disruptive innovation?. harvard business review (december 2015) 2-11. clarke r. (1987). just another piece of plastic for your wallet: the 'australia card' scheme. computers & society 18,1 (january 1988), addendum in computers & society 18,3 (july 1988), preprint at http://www.rogerclarke.com/dv/ozcard.html clarke r. (1992a). the resistible rise of the national personal data system. software law journal 5,1 (january 1992), preprint at http://www.rogerclarke.com/dv/slj.html clarke r. (1992b). extra-organisational systems: a challenge to the software engineering paradigm. proc. ifip world congress, madrid, september 1992, preprint at http://www.rogerclarke.com/sos/paperextraorgsys.html clarke r. (1994a). the digital persona and its application to data surveillance. the information society 10,2 (june 1994), preprint at http://www.rogerclarke.com/dv/digpersona.html clarke r. (1994b). dataveillance by governments: the technique of computer matching. information technology & people 7,2 (december 1994) 46-85, preprint at http://www.rogerclarke.com/dv/matchintro.html https://static1.squarespace.com/static/5c05f8595cfd7901fc57139d/t/5f8a808b459ade460acd28dc/1602912397885/beyond+the+bubble+that+is+robodebt+how+governments+that+lose+integrity+threaten+democracy.pdf https://static1.squarespace.com/static/5c05f8595cfd7901fc57139d/t/5f8a808b459ade460acd28dc/1602912397885/beyond+the+bubble+that+is+robodebt+how+governments+that+lose+integrity+threaten+democracy.pdf http://culturedigitally.org/2014/09/digitalization-and-digitization/ https://www.innovationaus.com/197000-notices-wiped-from-shameful-robodebt-scheme/ https://www.innovationaus.com/197000-notices-wiped-from-shameful-robodebt-scheme/ https://www.canberratimes.com.au/story/6407984/four-of-five-robodebt-reviews-hit-with-a-debt-as-cost-revealed/ https://www.canberratimes.com.au/story/6407984/four-of-five-robodebt-reviews-hit-with-a-debt-as-cost-revealed/ https://journals.sagepub.com/doi/pdf/10.1177/1037969x18815913 australasian journal of information systems clarke, michael & abbas 2024, vol 28, research article robodebt: socio-technical case study of is failure 32 clarke r. (2014). promise unfulfilled: the digital persona concept, two decades later. information technology & people 27, 2 (jun 2014) 182-207, preprint at http://www.rogerclarke.com/id/dp12.html clarke r. (2015). not only horses wear blinkers: the missing perspectives in is research. keynote presentation, proc. acis'15, at https://arxiv.org/pdf/1611.04059 clarke r. (2016). an empirical assessment of researcher perspectives. proc. bled econf., slovenia, june 2016, preprint at http://www.rogerclarke.com/sos/bledp.html clarke r. & davison r.m. (2020). through whose eyes? the critical concept of researcher perspective. j. assoc. infor. syst. 21, 2 (march-april 2020) 483-501, preprint at http://rogerclarke.com/sos/rp.html clarke r., davison r.m. & jia w. (2020). researcher perspective in the is discipline: an empirical study of articles in the basket of 8 journals. information technology & people 33, 6 (october 2020) 1515-1541, preprint at http://www.rogerclarke.com/sos/rpbo8.html clarke r., michael k. & abbas r. (2024). robodebt: an exemplary case study of a failed transformative it project in the public sector. working paper, xamax consultancy pty ltd, july 2024, at http://rogerclarke.com/dv/rdcs.html cornish, a. (2023). record-keeping deficiencies exposed in robodebt royal commission report. informaa quarterly official bulletin of the records management association of australia, 39,3 (2023) 8–8. cpi (2022). delay and decay: australia’s freedom of information crisis. briefing paper, the centre for public integrity, august 2022, at https://publicintegrity.org.au/wpcontent/uploads/2022/09/foi-delay-and-decay-final.pdf creswell, john w. & j. david creswell. ‘research design: qualitative, quantitative, and mixed methods approaches’. fifth edition. los angeles, sage, 2018. cukier k. & mayer-schönberger v. (2013). the rise of big data. foreign affairs (may/june 2014) 35. deloitte (2023a). the use of data and automation in the robodebt scheme: a report to the royal commission into the robodebt scheme. royal commission into the robodebt scheme, march 2023, at https://robodebt.royalcommission.gov.au/system/files/202303/use-of-data-and-automation-in-the-robodebt-scheme.pdf deloitte (2023b). robodebt process maps. royal commission into the robodebt scheme, march 2023, at https://robodebt.royalcommission.gov.au/publications/deloitterobodebt-process-maps dmp (1994). guidelines for the conduct of the data-matching program. delegated legislated under the data matching program (assistance and tax) act 1990, october 1994, nominally enforceable, but applicable only if the matching process uses the tax file number (tfn), at https://www.legislation.gov.au/details/f2009b00268 dreborg, k. h., 1996. essence of backcasting”, futures 28,9 (1996) 813-828, at https://doi.org/10.1016/s0016-3287(96)00044-4. https://publicintegrity.org.au/wp-content/uploads/2022/09/foi-delay-and-decay-final.pdf https://publicintegrity.org.au/wp-content/uploads/2022/09/foi-delay-and-decay-final.pdf https://robodebt.royalcommission.gov.au/system/files/2023-03/use-of-data-and-automation-in-the-robodebt-scheme.pdf https://robodebt.royalcommission.gov.au/system/files/2023-03/use-of-data-and-automation-in-the-robodebt-scheme.pdf https://robodebt.royalcommission.gov.au/publications/deloitte-robodebt-process-maps https://robodebt.royalcommission.gov.au/publications/deloitte-robodebt-process-maps https://www.legislation.gov.au/details/f2009b00268 https://doi.org/10.1016/s0016-3287(96)00044-4 australasian journal of information systems clarke, michael & abbas 2024, vol 28, research article robodebt: socio-technical case study of is failure 33 emery f. (1959). characteristics of socio-technical systems. in 'the social engagement of social science: a tavistock anthology volume 2, university of pennsylvania press, pp.157-186). farrell p. (2019). government halting key part of robodebt scheme, will freeze debts for some welfare recipients. abc news, 19 november 2019, at https://www.abc.net.au/news/2019-11-19/robodebt-scheme-human-servicesdepartment-halts-existing-debts/11717188 fischer-huebner s. & lindskog h. (2001). teaching privacy-enhancing technologies. proceedings of the ifip wg 11.8 2nd world conference on information security education, perth, australia. freeman r.e. & reed d.l. (1983). stockholders and stakeholders: a new perspective on corporate governance. california management review 25, 3 (1983) 88-106. goggin, g. & soldatić, k. (2022). automated decision-making, digital inclusion and intersectional disabilities. new media & society, vol. 24, no. 2, pp. 384–400. gr (2017). government response to the senate committee report', australian government, september 2017, at https://www.aph.gov.au/parliamentary_business/committees/senate/community_aff airs/socialwelfaresystem/additional_documents?doctype=government%20response graycar a. & masters a. b. (2022). bureaucratic bastardry: robodebt/debt recovery, ai and the stigmatisation of citizens by machines and systems. international journal of public policy 16,5-6 (2022) 333-344. hammer m. (1990). reengineering work: don’t automate, obliterate. harvard business review (july-august 1990) 104-112. hanks, p. (2021). administrative law and welfare rights: a 40-year story from ‘green v daniels’ to ‘robot debt recovery’ closing the chapter. aial forum, 103 (2021) 19–31. henriques-gomes l. (2020). all centrelink debts raised using income averaging unlawful, christian porter concedes. the guardian, 31 may 2020, at https://www.theguardian.com/australia-news/2020/may/31/all-centrelink-debts-raisedusing-income-averaging-unlawful-christian-porter-concedes hevner a.r., march s.t. & park s. (2004). design science in information systems research. mis quarterly 28,1 (2004) 75-105. hirschheim r.a. (1985). user experience with and assessment of participative systems design. mis quarterly (december 1985), at https://www.academia.edu/download/51009081/viewcontent.pdf hogan-doran d. (2017). computer says "no": automation, algorithms and artificial intelligence in government decision-making. the judicial review 13 (2017) 1-39, at https://www.academia.edu/38540521/computer_says_no_automation_algorithms_and _artificial_intelligence_in_government_decision_making?email_work_card=viewpaper https://www.abc.net.au/news/2019-11-19/robodebt-scheme-human-services-department-halts-existing-debts/11717188 https://www.abc.net.au/news/2019-11-19/robodebt-scheme-human-services-department-halts-existing-debts/11717188 https://www.aph.gov.au/parliamentary_business/committees/senate/community_affairs/socialwelfaresystem/additional_documents?doctype=government%20response https://www.aph.gov.au/parliamentary_business/committees/senate/community_affairs/socialwelfaresystem/additional_documents?doctype=government%20response https://www.theguardian.com/australia-news/2020/may/31/all-centrelink-debts-raised-using-income-averaging-unlawful-christian-porter-concedes https://www.theguardian.com/australia-news/2020/may/31/all-centrelink-debts-raised-using-income-averaging-unlawful-christian-porter-concedes https://www.academia.edu/download/51009081/viewcontent.pdf https://www.academia.edu/38540521/computer_says_no_automation_algorithms_and_artificial_intelligence_in_government_decision_making?email_work_card=view-paper https://www.academia.edu/38540521/computer_says_no_automation_algorithms_and_artificial_intelligence_in_government_decision_making?email_work_card=view-paper https://www.academia.edu/38540521/computer_says_no_automation_algorithms_and_artificial_intelligence_in_government_decision_making?email_work_card=view-paper australasian journal of information systems clarke, michael & abbas 2024, vol 28, research article robodebt: socio-technical case study of is failure 34 ixdf interaction design foundation. (2021, june 14). what are complex socio-technical systems?. interaction design foundation ixdf. 28 february 2024, at https://www.interaction-design.org/literature/topics/complex-socio-technical-systems klapdor m. (2020). social security and welfare. parliament of australia, october 2020, at https://www.aph.gov.au/about_parliament/parliamentary_departments/parliamentar y_library/pubs/rp/budgetreview202021/socialsecuritywelfare#:~:text=social%20secur ity%20and%20welfare%20expenditure,australian%20government's%20total%20expen diture%20(p. knaus c. (2016). centrelink officer says only a fraction of debts in welfare crackdown are genuine. the guardian, 23 december 2016, at https://www.theguardian.com/australianews/2016/dec/23/centrelink-officer-says-only-a-fraction-of-debts-in-welfarecrackdown-are-genuine knaus c. (2017). 'reasonably clear' alan tudge's office broke law in centrelink case, says labor legal advice. the guardian, 3 apr 2017, at https://www.theguardian.com/australia-news/2017/apr/03/reasonably-clear-alantudges-office-broke-law-in-centrelink-case-says-labor-legal-advice knaus c. & bassano j. (2019). how a flawed freedom-of-information regime keeps australians in the dark. the guardian, 2 january 2019, at https://www.theguardian.com/australia-news/2019/jan/02/how-a-flawed-freedom-ofinformation-regime-keeps-australians-in-the-dark land f. & hirschheim r. (1983). participative systems design: rationale, tools and techniques. journal of applied systems analysis, 1983. laplume a.o., sonpar k. & litz r.a. (2008). stakeholder theory: reviewing a theory that moves us. journal of management 34,6 (2008) 1152-1189. lindebaum, d., glaser, v., moser, c. & ashraf, m. (2023). when algorithms rule, values can wither. sloan management review, vol. 64, no. 2, pp. 1–5. liu k., sun l. & bennett k. (2002), 'co-design of business and it systems. introduction by guest editors, information systems frontiers, 4,3 (2002) 251–256. lsd (2017). legal services directions 2017', 2 july 2018, at https://www.legislation.gov.au/details/f2018c00409 lunde a.s. (1980). the person-number systems of sweden, norway, denmark, and israel. us department of health and human services, public health service, office of health research, statistics, and technology, national center for health statistics, vol. 80, no. 1358. lycett m. (2014), datafication: making sense of (big) data in a complex world, european journal of information systems, 22, 4 (december 2014) 381-386, at http://vscheiner.brunel.ac.uk/bitstream/2438/8110/2/fulltext.pdf lyytinen k. & rose g.m. (2003), 'the disruptive nature of information technology innovations: the case of internet computing in systems development organizations', mis quarterly, 27, 4 (december 2003), 557-595. https://www.interaction-design.org/literature/topics/complex-socio-technical-systems https://www.aph.gov.au/about_parliament/parliamentary_departments/parliamentary_library/pubs/rp/budgetreview202021/socialsecuritywelfare#:~:text=social security and welfare expenditure,australian government's total expenditure (p https://www.aph.gov.au/about_parliament/parliamentary_departments/parliamentary_library/pubs/rp/budgetreview202021/socialsecuritywelfare#:~:text=social security and welfare expenditure,australian government's total expenditure (p https://www.aph.gov.au/about_parliament/parliamentary_departments/parliamentary_library/pubs/rp/budgetreview202021/socialsecuritywelfare#:~:text=social security and welfare expenditure,australian government's total expenditure (p https://www.aph.gov.au/about_parliament/parliamentary_departments/parliamentary_library/pubs/rp/budgetreview202021/socialsecuritywelfare#:~:text=social security and welfare expenditure,australian government's total expenditure (p https://www.theguardian.com/australia-news/2016/dec/23/centrelink-officer-says-only-a-fraction-of-debts-in-welfare-crackdown-are-genuine https://www.theguardian.com/australia-news/2016/dec/23/centrelink-officer-says-only-a-fraction-of-debts-in-welfare-crackdown-are-genuine https://www.theguardian.com/australia-news/2016/dec/23/centrelink-officer-says-only-a-fraction-of-debts-in-welfare-crackdown-are-genuine https://www.theguardian.com/australia-news/2017/apr/03/reasonably-clear-alan-tudges-office-broke-law-in-centrelink-case-says-labor-legal-advice https://www.theguardian.com/australia-news/2017/apr/03/reasonably-clear-alan-tudges-office-broke-law-in-centrelink-case-says-labor-legal-advice https://www.theguardian.com/australia-news/2019/jan/02/how-a-flawed-freedom-of-information-regime-keeps-australians-in-the-dark https://www.theguardian.com/australia-news/2019/jan/02/how-a-flawed-freedom-of-information-regime-keeps-australians-in-the-dark https://www.legislation.gov.au/details/f2018c00409 http://v-scheiner.brunel.ac.uk/bitstream/2438/8110/2/fulltext.pdf http://v-scheiner.brunel.ac.uk/bitstream/2438/8110/2/fulltext.pdf australasian journal of information systems clarke, michael & abbas 2024, vol 28, research article robodebt: socio-technical case study of is failure 35 macleod, l. (2017). lessons learnt about digital transformation and public administration: centrelink’s online compliance intervention. aial forum, no. 89, pp. 59–68. mann, m. (2020). technological politics of automated welfare surveillance: social (and data) justice through critical qualitative inquiry. global perspectives (oakland, calif.), vol. 1, no. 1. marjanovic, o., cecez-kecmanovic, d., & vidgen, r. (2022). theorising algorithmic justice. european journal of information systems, 31(3), 269-287. sarah martin. (2016). welfare debt squad hunts for $4bn. the australian, 5 december 2016, at http://www.theaustralian.com.au/national-affairs/welfare-debt-squad-hunts-for-4bnin-overpayments/news-story/e19c5b0d4a39aa07364a41269fdc11c9 mead, g. & neves, b. (2023). contested delegation: understanding critical public responses to algorithmic decision-making in the uk and australia. the sociological review (keele), vol. 71, no. 3, pp. 601–623. medhora s. (2019). over 2000 people died after receiving centrelink robo-debt notice, figures reveal. abc triplej, 18 feb 2019, at https://www.abc.net.au/triplej/programs/hack/2030-people-have-died-after-receivingcentrelink-robodebt-notice/10821272 michael, k. (2003). the technological trajectory of the automatic identification industry: the application of the systems of innovation (si) framework for the characterization and prediction of the auto-id industry. phd thesis, school of information technology and computer science, university of wollongong, 2003, at http://ro.uow.edu/theses/309. see in particular 'section 4.3 the evolution of the citizen id number'. michael k. & michael m.g. (2006). historical lessons on id technology and the consequences of an unchecked trajectory. prometheus 24,4 (2006) 365-377. miller, k. (2017). connecting the dots: a case study of the robodebt communities. aial forum, no. 89, pp. 50–58. mitchell r.k., agle b.r. & wood d.j. (1997). toward a theory of stakeholder identification and salience: defining the principle of who and what really counts. academy of management review 22,4 (1997) 853-886. mrr (2015). welfare integrity, fairness and sustainability for all australians. minister for social security, scott morrison, budget media release, 12 may 2015, at https://formerministers.dss.gov.au/15850/welfare-integrity-fairness-and-sustainabilityfor-all-australians/ mumford e. (2000). a socio-technical approach to systems design. requirements engineering 5,2 (2000) 125-133. mumford e. (2006). the story of socio-technical design: reflections on its successes, failures and potential. information systems journal 16 (2006) 317–342, at https://executiveinsight.typepad.com/files/the-story-of-socio-technical-design.pdf naylor, s. (2023). robodebt, kafka and institutional absurdism in australia. alternative law journal, vol. 48, no. 4, pp. 299–302. http://www.theaustralian.com.au/national-affairs/welfare-debt-squad-hunts-for-4bn-in-overpayments/news-story/e19c5b0d4a39aa07364a41269fdc11c9 http://www.theaustralian.com.au/national-affairs/welfare-debt-squad-hunts-for-4bn-in-overpayments/news-story/e19c5b0d4a39aa07364a41269fdc11c9 https://www.abc.net.au/triplej/programs/hack/2030-people-have-died-after-receiving-centrelink-robodebt-notice/10821272 https://www.abc.net.au/triplej/programs/hack/2030-people-have-died-after-receiving-centrelink-robodebt-notice/10821272 https://formerministers.dss.gov.au/15850/welfare-integrity-fairness-and-sustainability-for-all-australians/ https://formerministers.dss.gov.au/15850/welfare-integrity-fairness-and-sustainability-for-all-australians/ https://executiveinsight.typepad.com/files/the-story-of-socio-technical-design.pdf australasian journal of information systems clarke, michael & abbas 2024, vol 28, research article robodebt: socio-technical case study of is failure 36 newell s. & marabelli m. (2015). strategic opportunities (and challenges) of algorithmic decision-making: a call for action on the long-term societal effects of 'datification'. the journal of strategic information systems 24, 1 (2015) 3-14, at http://marcomarabelli.com/newell-marabelli-jsis-2015.pdf ng, y-f. & gray, s. (2022). disadvantage and the automated decision. adelaide law review, vol. 43, no. 2, pp. 641–677. nikidehaghani, m., andrew, j. & cortese, c. (2023). algorithmic accountability: robodebt and the making of welfare cheats. accounting, auditing & accountability journal, vol. 36, no. 2, pp. 677–711. noone, m.a. (2020). innovation and disruption: exploring the potential of clinical legal education. international journal of clinical legal education, vol. 27, no. 3, pp. 5–32. norman, d. (2013), the design of everyday things: revised and expanded edition paperback, basic books, new york. oaic (2014) 'guidelines on data matching in australian government administration' office of the australian information commissioner, rev. 18 june 2014, at https://www.oaic.gov.au/privacy/guidance-and-advice/guidelines-on-data-matchingin-australian-government-administration omb (2017). centrelink’s automated debt raising and recovery system: a report about the department of human services. online compliance intervention system for debt raising and recovery. australian ombudsman, april 2017, at https://www.ombudsman.gov.au/__data/assets/pdf_file/0022/43528/reportcentrelinks-automated-debt-raising-and-recovery-system-april-2017.pdf park, s. & humphry, j. 2019. exclusion by design: intersections of social, digital and data exclusion. information, communication & society, vol. 22, no. 7, pp. 934–953. parnell s. (2024). tax office rejects one in four calls to fudge waiting time figures. the sydney morning herald, 25 march 2024, at https://www.smh.com.au/national/taxoffice-rejects-one-in-four-calls-to-fudge-waiting-time-stats-20240325-p5fez8.html piller g., schubert p., koch m. & moeslein k. (2004). from mass customization to collaborative customer codesign. proc. ecis 2004, 118. podger a. (2023). report to the royal commission into the robodebt scheme. royal commission into the robodebt scheme, february 2023, at https://robodebt.royalcommission.gov.au/system/files/2023-02/andrew-podger-reportto-robodebt-royal-commission.pdf ponterotto, j.g. (2006). brief note on the origins, evolution, and meaning of the qualitative research concept “thick description”, the qualitative report, vol. 11, no. 3, september 2006 538-549, http://www.nova.edu/ssss/qr/qr11-3/ponterotto.pdf pry (2021) prygodicz v commonwealth of australia (no 2) [2021] fca 634 (11 june 2021) the eventual judgment, at http://www.austlii.edu.au/cgibin/viewdoc/au/cases/cth/fca/2021/634.html?context=1;q uery=prygodicz;mask_path http://marcomarabelli.com/newell-marabelli-jsis-2015.pdf https://www.oaic.gov.au/privacy/guidance-and-advice/guidelines-on-data-matching-in-australian-government-administration https://www.oaic.gov.au/privacy/guidance-and-advice/guidelines-on-data-matching-in-australian-government-administration https://www.ombudsman.gov.au/__data/assets/pdf_file/0022/43528/report-centrelinks-automated-debt-raising-and-recovery-system-april-2017.pdf https://www.ombudsman.gov.au/__data/assets/pdf_file/0022/43528/report-centrelinks-automated-debt-raising-and-recovery-system-april-2017.pdf https://robodebt.royalcommission.gov.au/system/files/2023-02/andrew-podger-report-to-robodebt-royal-commission.pdf https://robodebt.royalcommission.gov.au/system/files/2023-02/andrew-podger-report-to-robodebt-royal-commission.pdf http://www.nova.edu/ssss/qr/qr11-3/ponterotto.pdf http://www.austlii.edu.au/cgibin/viewdoc/au/cases/cth/fca/2021/634.html?context=1;query=prygodicz;mask_path http://www.austlii.edu.au/cgibin/viewdoc/au/cases/cth/fca/2021/634.html?context=1;query=prygodicz;mask_path australasian journal of information systems clarke, michael & abbas 2024, vol 28, research article robodebt: socio-technical case study of is failure 37 ray, a., adams, b. & thampapillai, d. (2022). access to algorithms post-robodebt: do freedom of information laws extend to automated systems?. alternative law journal, vol. 47, no. 1, pp. 10–15. rcr (2023). report of the royal commission into the robodebt scheme', 7 july 2023, at https://robodebt.royalcommission.gov.au/publications/report simister, n. (2017) “case studies and stories of change”, intrac for civil society, https://www.intrac.org/resources/case-studies-stories-change/ redden j. (2018). the harm that data do. scientific american, 1 november 2018, https://www.scientificamerican.com/article/the-harm-that-data-do/ rinta-kahila t., someh i., gillespie n., indulska m. & gregor s. (2022). algorithmic decisionmaking and system destructiveness: a case of automatic debt recovery. european journal of information systems 31,3 (2022) 313-338, at https://www.tandfonline.com/doi/pdf/10.1080/0960085x.2021.1960905?needaccess=tru e robert (2019). media releases – 2019. minister responsible for services australia, at https://web.archive.org/web/20200319204601/https://minister.servicesaustralia.gov.au/ media-releases rrc (2022a). transcript of proceedings for monday, 5 december 2022. royal commission into the robodebt scheme, december 2022, at https://robodebt.royalcommission.gov.au/system/files/2022-12/transcript-hearing-day13-5-december-2022.pdf rrc (2022b). transcript of proceedings for monday, 6 december 2022. royal commission into the robodebt scheme, december 2022, at https://robodebt.royalcommission.gov.au/system/files/2022-12/transcript-6december.pdf rrc (2022c). transcript of proceedings for monday, 7 december 2022. royal commission into the robodebt scheme, december 2022, at https://robodebt.royalcommission.gov.au/system/files/2022-12/transcript-hearing-day16-8-december-2022.pdf rrc (2023a). transcript of proceedings for monday, 20 february 2023. royal commission into the robodebt scheme, february 2023, at https://robodebt.royalcommission.gov.au/system/files/2023-02/transcript-hearing-day32-20-february-2023.pdf rrc (2023b). transcript of proceedings for monday, 2 march 2023. royal commission into the robodebt scheme, march 2023, at https://robodebt.royalcommission.gov.au/system/files/2023-03/transcript-hearing-day40-2-march.pdf sa (2022). a guide to australian government payments. services australia, at https://www.servicesaustralia.gov.au/sites/default/files/co029-2207.pdf sca (2017). design, scope, cost-benefit analysis, contracts awarded and implementation associated with the better management of the social welfare system initiative. the senate community affairs references committee, australian parliament house, june https://robodebt.royalcommission.gov.au/publications/report https://www.intrac.org/resources/case-studies-stories-change/ https://www.scientificamerican.com/article/the-harm-that-data-do/ https://www.tandfonline.com/doi/pdf/10.1080/0960085x.2021.1960905?needaccess=true https://www.tandfonline.com/doi/pdf/10.1080/0960085x.2021.1960905?needaccess=true https://robodebt.royalcommission.gov.au/system/files/2022-12/transcript-hearing-day-13-5-december-2022.pdf https://robodebt.royalcommission.gov.au/system/files/2022-12/transcript-hearing-day-13-5-december-2022.pdf https://robodebt.royalcommission.gov.au/system/files/2022-12/transcript-6-december.pdf https://robodebt.royalcommission.gov.au/system/files/2022-12/transcript-6-december.pdf https://robodebt.royalcommission.gov.au/system/files/2022-12/transcript-hearing-day-16-8-december-2022.pdf https://robodebt.royalcommission.gov.au/system/files/2022-12/transcript-hearing-day-16-8-december-2022.pdf https://robodebt.royalcommission.gov.au/system/files/2023-02/transcript-hearing-day-32-20-february-2023.pdf https://robodebt.royalcommission.gov.au/system/files/2023-02/transcript-hearing-day-32-20-february-2023.pdf https://robodebt.royalcommission.gov.au/system/files/2023-03/transcript-hearing-day-40-2-march.pdf https://robodebt.royalcommission.gov.au/system/files/2023-03/transcript-hearing-day-40-2-march.pdf https://www.servicesaustralia.gov.au/sites/default/files/co029-2207.pdf australasian journal of information systems clarke, michael & abbas 2024, vol 28, research article robodebt: socio-technical case study of is failure 38 2017, at https://www.aph.gov.au/parliamentary_business/committees/senate/community_aff airs/socialwelfaresystem/report sca (2022). accountability and justice: why we need a royal commission into robodebt. senate community affairs references committee, may 2022, at https://parlinfo.aph.gov.au/parlinfo/download/committees/reportsen/024846/toc_pdf/ accountabilityandjusticewhyweneedaroyalcommissionintorobodebt.pdf;filetype=ap plication%2fpdf trist e.l. (1981). the evolution of socio-technical systems. ontario quality of working life centre, vol. 2, 1981. tt (2016). guilty until proven innocent ... centrelink gone rogue .... tasmanian times, 5 december 2016, at https://tasmaniantimes.com/2016/12/guilty-until-proven-innocentcentrelink-gone-rogue/ twomey, a. (2023). the coronation, the commonwealth and government integrity. round table (london), vol. 112, no. 4, pp. 367–375. venkatraman n. (1994). it-enabled business transformation: from automation to business scope redefinition', sloan management review 35,2, (winter 1994), 73-87. von bertalanffy l. (1950). the theory of open systems in physics and biology', science, 111, 2872 (1950) 23-29. wahlstrom k. & quirchmayr g. (2008). a privacy-enhancing architecture for databases. journal of research and practice in information technology 40, 3 (august 2008) 151162, at http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.454.3815&rep=rep1&type=p df walsham g. (1995). interpretive case studies in is research: nature and method. european journal of information systems 4, 2 (may 1995) 74-81. whelan, a (2020). ‘ask for more time’: big data chronopolitics in the australian welfare bureaucracy. critical sociology, vol. 46, no. 6, pp. 867–880. white, s. (2021). authorisation and accountability of automated government decisions under australian administrative law. aial forum, no. 102, pp. 84–96. whiteford p. (2021). debt by design: the anatomy of a social policy fiasco – or was it something worse? aust. j of public admin 80,2 (june 2021) 340-360, at https://onlinelibrary.wiley.com/doi/epdf/10.1111/1467-8500.12479 whiteford p. (2022). for the poorest 20% of australian households, social security payments provide more than 70% of their income. john menadue's public policy journal, 20 may 2022, at https://johnmenadue.com/social-security/ wong k. (2022). robodebt scheme told to ‘cease and desist. aap, 4 november 2022, at https://www.aap.com.au/news/debt-collectors-to-front-royal-commission/ wright, a. & ng, y-f. (2022). services australia’s single touch payroll program: the enduring legacy of robodebt, or a fundamentally different system?. public law review (north ryde, n.s.w.), vol. 33, no. 2, pp. 127–142. https://www.aph.gov.au/parliamentary_business/committees/senate/community_affairs/socialwelfaresystem/report https://www.aph.gov.au/parliamentary_business/committees/senate/community_affairs/socialwelfaresystem/report https://parlinfo.aph.gov.au/parlinfo/download/committees/reportsen/024846/toc_pdf/accountabilityandjusticewhyweneedaroyalcommissionintorobodebt.pdf;filetype=application%2fpdf https://parlinfo.aph.gov.au/parlinfo/download/committees/reportsen/024846/toc_pdf/accountabilityandjusticewhyweneedaroyalcommissionintorobodebt.pdf;filetype=application%2fpdf https://parlinfo.aph.gov.au/parlinfo/download/committees/reportsen/024846/toc_pdf/accountabilityandjusticewhyweneedaroyalcommissionintorobodebt.pdf;filetype=application%2fpdf https://tasmaniantimes.com/2016/12/guilty-until-proven-innocent-centrelink-gone-rogue/ https://tasmaniantimes.com/2016/12/guilty-until-proven-innocent-centrelink-gone-rogue/ http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.454.3815&rep=rep1&type=pdf http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.454.3815&rep=rep1&type=pdf https://onlinelibrary.wiley.com/doi/epdf/10.1111/1467-8500.12479 https://johnmenadue.com/social-security/ https://www.aap.com.au/news/debt-collectors-to-front-royal-commission/ australasian journal of information systems clarke, michael & abbas 2024, vol 28, research article robodebt: socio-technical case study of is failure 39 yin r.k. (2017). case study research and applications: design and methods. 6th edition, sage, 2017. zwass v. (2010). co-creation: toward a taxonomy and an integrated research perspective. international journal of electronic commerce 15,1 (2010) 11-48. appendix a: case resources this appendix provides access to the sources discovered in the course of the case study research reported in this article. to preserve their context, they are listed in chronological order. a1 the investigation reports, in chronological order omb (2017) 'centrelink’s automated debt raising and recovery system: a report about the department of human services’ online compliance intervention system for debt raising and recovery' australian ombudsman, april 2017, at https://www.ombudsman.gov.au/__data/assets/pdf_file/0022/43528/report-centrelinksautomated-debt-raising-and-recovery-system-april-2017.pdf sca (2017) 'design, scope, cost-benefit analysis, contracts awarded and implementation associated with the better management of the social welfare system initiative' the senate community affairs references committee, australian parliament house, june 2017, at https://www.aph.gov.au/parliamentary_business/committees/senate/community_affairs/s ocialwelfaresystem/report submissions, at https://www.aph.gov.au/parliamentary_business/committees/senate/community_affairs/s ocialwelfaresystem/submissions transcripts of public hearings, https://www.aph.gov.au/parliamentary_business/committees/senate/community_affairs/s ocialwelfaresystem/public_hearings additional documents, at https://www.aph.gov.au/parliamentary_business/committees/senate/community_affairs/s ocialwelfaresystem/additional_documents gr (2017) 'government response to the senate committee report', australian government, september 2017, at https://www.aph.gov.au/parliamentary_business/committees/senate/community_affairs/s ocialwelfaresystem/additional_documents?doctype=government%20response pry (2021) prygodicz v commonwealth of australia (no 2) [2021] fca 634 (11 june 2021) the eventual judgment, at http://www.austlii.edu.au/cgibin/viewdoc/au/cases/cth/fca/2021/634.html?context=1;query= prygodicz;mask_path sca (2022) 'accountability and justice: why we need a royal commission into robodebt' senate community affairs references committee, may 2022, at https://parlinfo.aph.gov.au/parlinfo/download/committees/reportsen/024846/toc_pdf/accou ntabilityandjusticewhyweneedaroyalcommissionintorobodebt.pdf;filetype=application%2 fpdf rrc (2022a) 'transcript of proceedings for monday, 5 december 2022' royal commission into the robodebt scheme, december 2022, at https://www.ombudsman.gov.au/__data/assets/pdf_file/0022/43528/report-centrelinks-automated-debt-raising-and-recovery-system-april-2017.pdf https://www.ombudsman.gov.au/__data/assets/pdf_file/0022/43528/report-centrelinks-automated-debt-raising-and-recovery-system-april-2017.pdf https://www.aph.gov.au/parliamentary_business/committees/senate/community_affairs/socialwelfaresystem/report https://www.aph.gov.au/parliamentary_business/committees/senate/community_affairs/socialwelfaresystem/report https://www.aph.gov.au/parliamentary_business/committees/senate/community_affairs/socialwelfaresystem/submissions https://www.aph.gov.au/parliamentary_business/committees/senate/community_affairs/socialwelfaresystem/submissions https://www.aph.gov.au/parliamentary_business/committees/senate/community_affairs/socialwelfaresystem/public_hearings https://www.aph.gov.au/parliamentary_business/committees/senate/community_affairs/socialwelfaresystem/public_hearings https://www.aph.gov.au/parliamentary_business/committees/senate/community_affairs/socialwelfaresystem/additional_documents https://www.aph.gov.au/parliamentary_business/committees/senate/community_affairs/socialwelfaresystem/additional_documents https://www.aph.gov.au/parliamentary_business/committees/senate/community_affairs/socialwelfaresystem/additional_documents?doctype=government%20response https://www.aph.gov.au/parliamentary_business/committees/senate/community_affairs/socialwelfaresystem/additional_documents?doctype=government%20response http://www.austlii.edu.au/cgibin/viewdoc/au/cases/cth/fca/2021/634.html?context=1;query=prygodicz;mask_path http://www.austlii.edu.au/cgibin/viewdoc/au/cases/cth/fca/2021/634.html?context=1;query=prygodicz;mask_path https://parlinfo.aph.gov.au/parlinfo/download/committees/reportsen/024846/toc_pdf/accountabilityandjusticewhyweneedaroyalcommissionintorobodebt.pdf;filetype=application%2fpdf https://parlinfo.aph.gov.au/parlinfo/download/committees/reportsen/024846/toc_pdf/accountabilityandjusticewhyweneedaroyalcommissionintorobodebt.pdf;filetype=application%2fpdf https://parlinfo.aph.gov.au/parlinfo/download/committees/reportsen/024846/toc_pdf/accountabilityandjusticewhyweneedaroyalcommissionintorobodebt.pdf;filetype=application%2fpdf australasian journal of information systems clarke, michael & abbas 2024, vol 28, research article robodebt: socio-technical case study of is failure 40 https://robodebt.royalcommission.gov.au/system/files/2022-12/transcript-hearing-day-13-5december-2022.pdf rrc (2022b) 'transcript of proceedings for monday, 6 december 2022' royal commission into the robodebt scheme, december 2022, at https://robodebt.royalcommission.gov.au/system/files/2022-12/transcript-6-december.pdf rrc (2022c) 'transcript of proceedings for monday, 7 december 2022' royal commission into the robodebt scheme, december 2022, at https://robodebt.royalcommission.gov.au/system/files/2022-12/transcript-hearing-day-16-8december-2022.pdf rrc (2023a) 'transcript of proceedings for monday, 20 february 2023' royal commission into the robodebt scheme, february 2023, at https://robodebt.royalcommission.gov.au/system/files/2023-02/transcript-hearing-day-32-20february-2023.pdf rrc (2023b) 'transcript of proceedings for monday, 2 march 2023' royal commission into the robodebt scheme, march 2023, at https://robodebt.royalcommission.gov.au/system/files/2023-03/transcript-hearing-day-40-2march.pdf rcr (2023) 'report of the royal commission into the robodebt scheme', 7 july 2023, at https://robodebt.royalcommission.gov.au/publications/report a2 other relevant official documents, in chronological order dma (1990) data-matching program (assistance and tax) act 1990, at http://classic.austlii.edu.au/au/legis/cth/consol_act/dpata1990349/index.html dmp (1994) 'guidelines for the conduct of the data-matching program' delegated legislated under the data matching program (assistance and tax) act 1990, october 1994, nominally enforceable, but applicable only if the matching process uses the tax file number (tfn), at https://www.legislation.gov.au/details/f2009b00268 oaic (2014) 'guidelines on data matching in australian government administration' privacy commissioner, june 2014, unenforceable, https://www.oaic.gov.au/privacy/privacyguidance-for-organisations-and-government-agencies/government-agencies/guidelines-ondata-matching-in-australian-government-administration centrelink (2016) 'data matching protocol', undated, but apparently of mid-2016, suppressed by centrelink for a year, and only disclosed incidentally by another agency as an attachment to a letter of 16 may 2017 by the privacy commissioner responding to a senator's questions, at http://www.aph.gov.au/documentstore.ashx?id=4f473c96-f345-4314-ad5d-a2c2eecfba47 oaic (2017) statements by the privacy commissioner, january-march 2017, at https://www.oaic.gov.au/media-and-speeches/statements/centrelink-debt-recovery-system [broken link on 6 sep 2022] dhs (2017) 'if you owe us money, you'll need to pay us back' department of human services, 2016-17, https://www.servicesaustralia.gov.au/owing-money-for-centrelink-debt [but in many versions, without revision-markings] amato (2019) amato v the commonwealth of australia (federal court vid611/2019), at https://www.comcourts.gov.au/file/federal/p/vid611/2019/3859485/event/30114114/docume nt/1513665 https://robodebt.royalcommission.gov.au/system/files/2022-12/transcript-hearing-day-13-5-december-2022.pdf https://robodebt.royalcommission.gov.au/system/files/2022-12/transcript-hearing-day-13-5-december-2022.pdf https://robodebt.royalcommission.gov.au/system/files/2022-12/transcript-6-december.pdf https://robodebt.royalcommission.gov.au/system/files/2022-12/transcript-hearing-day-16-8-december-2022.pdf https://robodebt.royalcommission.gov.au/system/files/2022-12/transcript-hearing-day-16-8-december-2022.pdf https://robodebt.royalcommission.gov.au/system/files/2023-02/transcript-hearing-day-32-20-february-2023.pdf https://robodebt.royalcommission.gov.au/system/files/2023-02/transcript-hearing-day-32-20-february-2023.pdf https://robodebt.royalcommission.gov.au/system/files/2023-03/transcript-hearing-day-40-2-march.pdf https://robodebt.royalcommission.gov.au/system/files/2023-03/transcript-hearing-day-40-2-march.pdf https://robodebt.royalcommission.gov.au/publications/report http://classic.austlii.edu.au/au/legis/cth/consol_act/dpata1990349/index.html https://www.legislation.gov.au/details/f2009b00268 http://www.aph.gov.au/documentstore.ashx?id=4f473c96-f345-4314-ad5d-a2c2eecfba47 https://www.oaic.gov.au/media-and-speeches/statements/centrelink-debt-recovery-system https://www.servicesaustralia.gov.au/owing-money-for-centrelink-debt https://www.comcourts.gov.au/file/federal/p/vid611/2019/3859485/event/30114114/document/1513665 https://www.comcourts.gov.au/file/federal/p/vid611/2019/3859485/event/30114114/document/1513665 australasian journal of information systems clarke, michael & abbas 2024, vol 28, research article robodebt: socio-technical case study of is failure 41 prygodicz v commonwealth of australia [2020] fca 1454 (17 september 2020) http://www.austlii.edu.au/cgibin/viewdoc/au/cases/cth/fca/2020/1454.html?context=1;query=prygodicz;mask_path commonwealth of australia v prygodicz [2020] fca 1516 (14 october 2020) http://www.austlii.edu.au/cgibin/viewdoc/au/cases/cth/fca/2020/1516.html?context=1;query=prygodicz;mask_path robert (2019) 'media releases – 2019' minister responsible for services australia, at https://web.archive.org/web/20200319204601/https://minister.servicesaustralia.gov.au/mediareleases sa (2021) 'individuals' services australia, 24 june 2021, at https://www.servicesaustralia.gov.au/individuals/centrelink prygodicz v commonwealth of australia (no 3) [2022] fca 826 (23 march 2022), at http://www.austlii.edu.au/cgibin/viewdoc/au/cases/cth/fca/2022/826.html?context=1;query=prygodicz;mask_path royal commission into the robodebt scheme: • letters patent, 18 august 2022, at https://robodebt.royalcommission.gov.au/about/termsreference • 'document library', at https://robodebt.royalcommission.gov.au/document-library sa (2022) 'a guide to australian government payments' services australia, at https://www.servicesaustralia.gov.au/sites/default/files/co029-2207.pdf musolino a. (2023) 'exhibit 3-4208 amu.9999.0001.0003_r 20221031 ntg-0012 response (annette musolino)(46258465.1)' royal commission into robodebt, 30 january 2023, at https://robodebt.royalcommission.gov.au/system/files/exhibit/exhibit%203-4208%20%20amu.9999.0001.0003_r%20-%2020221031%20ntg-0012%20%20response%20%28annette%20musolino%29%2846258465.1%29.pdf podger a. (2023) 'report to the royal commission into the robodebt scheme' royal commission into the robodebt scheme, february 2023, at https://robodebt.royalcommission.gov.au/system/files/2023-02/andrew-podger-report-torobodebt-royal-commission.pdf deloitte (2023a) 'the use of data and automation in the robodebt scheme: a report to the royal commission into the robodebt scheme' royal commission into the robodebt scheme, march 2023, at https://robodebt.royalcommission.gov.au/system/files/2023-03/use-of-dataand-automation-in-the-robodebt-scheme.pdf deloitte (2023b) 'robodebt process maps' royal commission into the robodebt scheme, march 2023, at https://robodebt.royalcommission.gov.au/publications/deloitte-robodebtprocess-maps acknowledgements the authors greatly appreciated the comments of the reviewers, and the support of the section editor, craig mcdonald, and editor, karlheinz kautz, which enabled us to materially improve the article's argument and presentation. http://www.austlii.edu.au/cgi-bin/viewdoc/au/cases/cth/fca/2020/1454.html?context=1;query=prygodicz;mask_path http://www.austlii.edu.au/cgi-bin/viewdoc/au/cases/cth/fca/2020/1454.html?context=1;query=prygodicz;mask_path http://www.austlii.edu.au/cgi-bin/viewdoc/au/cases/cth/fca/2020/1516.html?context=1;query=prygodicz;mask_path http://www.austlii.edu.au/cgi-bin/viewdoc/au/cases/cth/fca/2020/1516.html?context=1;query=prygodicz;mask_path https://www.servicesaustralia.gov.au/individuals/centrelink http://www.austlii.edu.au/cgi-bin/viewdoc/au/cases/cth/fca/2022/826.html?context=1;query=prygodicz;mask_path http://www.austlii.edu.au/cgi-bin/viewdoc/au/cases/cth/fca/2022/826.html?context=1;query=prygodicz;mask_path https://robodebt.royalcommission.gov.au/about/terms-reference https://robodebt.royalcommission.gov.au/about/terms-reference https://robodebt.royalcommission.gov.au/document-library https://www.servicesaustralia.gov.au/sites/default/files/co029-2207.pdf https://robodebt.royalcommission.gov.au/system/files/exhibit/exhibit%203-4208%20-%20amu.9999.0001.0003_r%20-%2020221031%20ntg-0012%20-%20response%20(annette%20musolino)(46258465.1).pdf https://robodebt.royalcommission.gov.au/system/files/exhibit/exhibit%203-4208%20-%20amu.9999.0001.0003_r%20-%2020221031%20ntg-0012%20-%20response%20(annette%20musolino)(46258465.1).pdf https://robodebt.royalcommission.gov.au/system/files/exhibit/exhibit%203-4208%20-%20amu.9999.0001.0003_r%20-%2020221031%20ntg-0012%20-%20response%20(annette%20musolino)(46258465.1).pdf https://robodebt.royalcommission.gov.au/system/files/2023-02/andrew-podger-report-to-robodebt-royal-commission.pdf https://robodebt.royalcommission.gov.au/system/files/2023-02/andrew-podger-report-to-robodebt-royal-commission.pdf https://robodebt.royalcommission.gov.au/system/files/2023-03/use-of-data-and-automation-in-the-robodebt-scheme.pdf https://robodebt.royalcommission.gov.au/system/files/2023-03/use-of-data-and-automation-in-the-robodebt-scheme.pdf https://robodebt.royalcommission.gov.au/publications/deloitte-robodebt-process-maps https://robodebt.royalcommission.gov.au/publications/deloitte-robodebt-process-maps australasian journal of information systems clarke, michael & abbas 2024, vol 28, research article robodebt: socio-technical case study of is failure 42 copyright copyright © 2024 clarke, michael & abbas. this is an open-access article licensed under a creative commons attribution-non-commercial 4.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v28.4681 https://creativecommons.org/licenses/by-nc/4.0/deed.en paper title (paper title style) australasian journal of information systems serenko et al. 2022, vol 26, research article turnover in japanese it professionals 1 turnover in japanese it professionals: antecedents and nuances alexander serenko university of ontario institute of technology, canada a.serenko@uoit.ca hiroshi sasaki rikkyo university, japan prashant palvia university of north carolina at greensboro, usa osam sato tokyo keizai university,japan abstract the japanese information technology (it) workplace is unique compared to that of other nations. it represents a large sector of the country’s economy, and organizations need to develop proactive approaches to retain their it workforce. in order to manage employee turnover, they need to understand the distinctive factors influencing employee turnover intention, as turnover intention is known to be a reliable predictor of actual turnover. in this study, a model was constructed and tested with data collected from 284 japanese it professionals. our findings show that the effects of work exhaustion, personal accomplishment, and friendship networks on turnover intention are fully mediated through job satisfaction. work-home conflict has no impact on job satisfaction. the strength of the relationships is stronger for younger than for older organizations. furthermore, individualistic factors (i.e., work exhaustion and personal accomplishment) have a stronger impact on job satisfaction than collectivistic factors (i.e., work-home conflict and friendship networks). these results show the fragility of the notion of long-term employment, which is supposed to be embraced within the entire japanese work culture. keywords: japan, job satisfaction, turnover intention, friendship networks, personal accomplishment, work exhaustion, work-home conflict, long-term employment, the world it project. 1 introduction this work is part of a series of empirical studies being coordinated by the world it project (palvia et al., 2017; palvia, ghosh, jacks, serenko, & turan, 2018; palvia, ghosh, jacks, serenko, & turan, 2020), and it tries to understand under what conditions it professionals in japan intend to leave their organizations. the japanese it industry is highly knowledge-intensive, and employees’ skills, talent, and expertise represent perhaps the most important component of the overall intellectual capital of japanese it organizations. generally, the longer it workers stay with their organizations, the more knowledgeable and productive they become which increases their value to their employers. on the one hand, turnover may not appear to be a serious issue because japan’s average tenure of full-time permanent employees is higher than that of most other countries, including the u.s. (anonymous, 2018a; 2018b; 2018c). on the other hand, there are looming signs that the voluntary turnover rate of japanese workers may australasian journal of information systems serenko et al. 2022, vol 26, research article turnover in japanese it professionals 2 quickly grow in the future (kawaguchi & ueno, 2013; suzuki, 2010; takahashi, 2018). in particular, voluntary turnover rates in japan’s it industry are likely to increase due to stressful work conditions, many job opportunities, diminishing seniority system, and growing competition for talented it professionals. thus, it behoves japanese organizations to develop proactive approaches to retain their it workforce (wiesche, joseph, thatcher, gu, & krcmar, 2019); for this, they need to become aware of the factors influencing their employees’ turnover intention because turnover intention is deemed a reliable predictor of actual turnover. to date, is researchers have developed a good understanding of the factors affecting turnover intention of it professionals in the u.s. and the west (e.g., see igbaria & siegel, 1992; mcknight, phillips, & hardgrave, 2009; rutner, hardgrave, & mcknight, 2008; sasaki, serenko, sato, & palvia, 2019). however, the nature of the japanese it industry differs from that of its western counterparts. job attitudes and perceptions of japanese it workers are influenced by the country’s confucian roots, the dominance of collectivistic values, its geographic, demographic, and political idiosyncrasies, unique and fixed hierarchical structure of the it industry, mechanistic and bureaucratic management style, and the notion of longterm employment (huff & kelley, 2003; kuo & chen, 2004; paramore, 2016; sasaki, 2012). thus, the models and recommendations developed for western countries may not apply well in the unique context of japan. in this study, based on the heuristic model of employee turnover (mobley, 1977; mobley, horner, & hollingsworth, 1978), we hypothesize and demonstrate that turnover intention of japanese it workers is driven by their level of job satisfaction. based on affective events theory (weiss, 2002; weiss & cropanzano, 1996; weiss, nicholas, & daus, 1999), we further propose that job satisfaction, in turn, is influenced by employees’ affective work experiences (i.e., mood and emotions) resulting from work exhaustion and a feeling of personal accomplishment, which represent individualistic factors, as well as by the presence of work-home conflict and the availability of friendship networks, which refer to collectivistic factors. our study places these constructs in a nomological network and subjects them to empirical testing using data from 284 japanese it professionals. the results indicate that the effect of work exhaustion, personal accomplishment, and friendship networks on turnover intention is fully mediated through job satisfaction. in contrast to expectations, work-home conflict has no direct impact on job satisfaction. in addition, the age of an organization moderates the strength of these relationships: most relationships are stronger for younger organizations because they have weaker collectivistic culture and are less affected by the notion of long-term employment. the rest of this paper is structured as follows. the next section reviews the extant literature and presents the theoretical underpinnings of this study. specifically, it covers the unique nature of the japanese it sector, describes prior works and research gaps, identifies the key constructs, and develops six hypotheses. section three documents the research method, and section four outlines the findings. section five presents implications, limitations and future research directions, and section six concludes this paper. 2 literature review and theoretical underpinnings 2.1 background the roots of the japanese it industry may be traced back to the 1960s when the country introduced the first ibm system/360 computers to support data aggregation during the tokyo olympic games and to computerize ticket reservation processes at the national railways australasian journal of information systems serenko et al. 2022, vol 26, research article turnover in japanese it professionals 3 (sasaki, sato, & palvia, 2020). in the 1970s, in order to protect the domestic it industry from u.s. competition, the japanese government urged the creation of three local computer manufacturer groups – fujitsu and hitachi, nec and toshiba, and mitsubishi and oki – which led to the development of ibm-compatible machines and the introduction of national mainframe computers for both individual consumers and enterprises. since the 1980s, the japanese it industry has matured dramatically due to various it innovations, which came largely from the u.s. at the same time, it has progressed in a somewhat unique manner and the japanese it sector has distinct differences compared to its western counterparts. first, japan’s cultural attributes are based on unique confucian roots (paramore, 2016) which affect the job attitudes and perceptions of its it workers. second, the japanese social and work culture values collectivism instead of individualism, which dominates the western professional world. traditionally, collectivism has always been an important attribute of the japanese workplace where loyalty towards one’s company is highly regarded (huff & kelley, 2003; nakane, 1970). in most organizations in japan, human resource departments launch months-long training for new recruits hired right out of college on the 1st of april every year. a long training period gives new workers an opportunity to build social networks among their colleagues and supervisors. they often enjoy informal talks together in bars and restaurants after work (huff & kelley, 2003; nakane, 1970). this informal socialization process is referred to as “nominication” (nomi=drinking alcohol in japanese + communication), and it can be a good way to develop better workplace communication and intimate relationships (tsutsui, 2010). in this way, the collectivistic culture, where employees view themselves as highly interdependent (gelfand, bhawuk, nishii, & bechtold, 2004), is formed. thus, as noted by abegglen (2006, p.89), the “key practices in japan’s employment system – an emphasis on continuity, on group integrity, and on egalitarianism – remain in effect.” third, japan enjoys a number of geographic, demographic, and political idiosyncrasies which inimitably affect it employees’ perceptions of workplace factors (kuo & chen, 2004). fourth, most japanese organizations rely on custom-built it solutions rather than standardized vendor products as are commonly used in north america and europe. it workers are often considered craftsmen possessing a narrow range of skills and producing unique high-quality products with zero defects. fifth, the japanese it industry has a fixed hierarchical structure with a few large players at the top and a large number of small and medium-sized enterprises at the bottom (sasaki, 2012), many of which are recent entrants. sixth, the management style in all industries, including it, is largely mechanistic and bureaucratic. as a result, the japanese it industry exhibits a number of idiosyncrasies that must be taken into consideration when studying it workers and factors affecting their turnover. another critical issue that affects the turnover of japanese it professionals is the notion of long-term employment, which has been generally considered the most important feature of the traditional japanese workplace. long-term employment represents an implicit social contract between the organization and the employee according to which both are committed to working within a professional community to achieve a certain level of economic security for all of its members (abegglen, 2006). in many japanese industries, this implicit social contract has survived since the 1950s. at present, the mobility of japan’s workers still remains much lower than that in other organization for economic co-operation and development (oecd) countries, including the u.s. (ono, 2010). according to the japan institute for labour policy and training, the average tenure of permanent full-time employees is around twelve australasian journal of information systems serenko et al. 2022, vol 26, research article turnover in japanese it professionals 4 years (anonymous, 2018a) with 44.5% of employees having at least ten years of tenure (anonymous, 2018b). in contrast, an average u.s. employee remains with his or her organization for only four years (anonymous, 2018c). however, after the collapse of the “bubble economy” in the 1990s and the financial crisis of 2008, the job stability of japanese it workers has started to decline (kawaguchi & ueno, 2013; suzuki, 2010; takahashi, 2018). in particular, the japanese it industry exhibits some unique characteristics which may be somewhat inconsistent with the traditional interpretation of the long-term employment concept. first, it professionals are frequently exposed to stressful work conditions. this, in turn, may affect their behavior and loyalty towards their employers. second, due to continuous appearance of new job opportunities, many it workers prefer less stable (i.e., high risk and high return) workplaces and take on more challenging and ambitious work. third, a somewhat diminishing seniority system in the it environment has accelerated job-hopping, especially among talented workers. fourth, rivalry among competitors in the it industry is fierce. the term “3k” (i.e., kitsui – hard, kibishii – demanding, and kaerenai – having irregular working hours) is used to refer to the it work environment (anonymous, 2018e). fifth, many it workers are young and are employed in small organizations. evidence suggests that junior japanese workers from small organizations are highly mobile and are less prone to seeking long-term employment (kambayashi & kato, 2017; shimizutani & yokoyama, 2009). thus, there are signs that the japanese it industry has started deviating from the notion of long-term employment and it is time to look into the factors driving the turnover of japanese it professionals. 2.2 prior related work a large number of previous projects have investigated the turnover of it personnel (igbaria & siegel, 1992; mcknight et al., 2009; oehlhorn et al., 2020; rutner et al., 2008). however, except for a few notable studies (e.g., see kuo & chen, 2004; maamari, 2014; scholtz, van belle, njenga, serenko, & palvia, 2019; wang, tolson, chiang, & huang, 2010; weli, 2014; wickramasinghe, 2009), the literature is dominated by projects conducted in the western context, mostly in the u.s. on the one hand, western-centric studies have enriched our understanding of employee turnover factors and facilitated the development of various it employee retention policies. on the other hand, their generalizability is often very limited beyond their western contexts (palvia, 2013). even though people from all over the world share many personality traits and values, those living in the western countries often differ from their non-western counterparts in terms of reasoning, preferences, and behaviors (henrich, heine, & norenzayan, 2010a), and such differences are particularly conspicuous between those living in the western and the asian countries (henrich, heine, & norenzayan, 2010b). as a result, several studies have documented cross-national differences in employees’ perceptions of their workplaces (spector et al., 2007; yeo et al., 2021; yousef, 2000). thus, whereas previous studies on the antecedents of turnover of it workers provide a useful benchmark and a starting point, they may not fully apply to organizations operating in other parts of the world. it is, therefore, vital to explore the factors affecting it workers’ retention in the context of individual countries (scholtz et al., 2019), including japan. australasian journal of information systems serenko et al. 2022, vol 26, research article turnover in japanese it professionals 5 in order to understand the antecedents of employee turnover in japan, a comprehensive search for relevant empirical works published in english language was conducted, and 19 studies were found (see table 1). their analysis leads to several important observations.1 work context factors affecting turnover blinder & krueger (1996) general labor market. cultural factors and human resource policies such as pay, benefits, participatory management, and reluctance to recruit from outside firms. de moura et al. (2009) education. job satisfaction and organizational identification. gamage (2014) manufacturing. human resources practices, such as staffing, training & development, performance evaluation, and compensation management. higuchi (1993) general labor market. age and education. honda-howard & homma (2001) general women’s labor market. job satisfaction. huang (2011) electric light industry and telecommunications industry. job satisfaction. kachi et al. (2020) financial industry. job stressors, psychological/physical stress response, workplace social support, and job strain. kim (2018) general labor market. job satisfaction, psychological empowerment, and organizational commitment. kudo et al. (2006) healthcare. job satisfaction. liu (2010) the it industry (women only). organizational culture, competitive pressure, achievement motivation, and job stress. matsumoto & gopal (2019) general labor market. job satisfaction and organizational solidarity. shimizu, feng, & nagata (2005a) healthcare. burnout. shimizu, eto, horiguchi, obata, feng, & nagata (2005b) healthcare. personal health factors. suzuki, tagaya, ota, nagasawa, matsuura, & sato (2010) healthcare. burnout, job satisfaction, and location. takahashi (2014) general labor market. job satisfaction (under certain conditions). takase, teraoka, & yabase (2016) healthcare. psychological contract fulfilment and perceived advancement opportunities. takase, oba, & yamashita (2009) healthcare. autonomy, work-home conflict, losing the confidence to care, economic return, and job security. tei-tominaga & miki (2010) healthcare. psychological distress, fatigue, lack of co-worker and supervisor support, and personal unsuitability for work. yanadori & kato (2009) general labor market. flextime and longer nursing/childcare leave. table 1. empirical studies of employee turnover in japan first and foremost, only one of the studies presented in table 1 was conducted in the context of the it industry in japan (see liu, 2010), and it focused on women alone. second, researchers 1 a preliminary version of this paper was presented at the 2019 americas conference on information systems (sasaki et al., 2019) and is not included in table 1. australasian journal of information systems serenko et al. 2022, vol 26, research article turnover in japanese it professionals 6 concentrated on the general labor market (i.e., not a specific industry), the healthcare sector (i.e., medical professionals such as nurses), and a variety of other industries, but not the it sector. thus, factors affecting the turnover of japanese it employees hitherto remain unknown. third, job satisfaction emerged as the most frequent driver of employee turnover in various industries and the general labor market. a number of studies in the it domain (but not in japan) have also identified job satisfaction as a major determinant of employee turnover (joseph, ng, koh, & ang, 2007; rutner et al., 2008; thatcher, stepina, & boyle, 2002). therefore, it is likely that job satisfaction is a key driver of turnover in the japanese it sector. fourth, exhaustion-related factors (e.g., stress and burnout) and work-home conflict appeared in several investigations. thus, such factors are worth looking into. 2.3 theoretical underpinnings 2.3.1 job satisfaction job satisfaction refers to an employee’s overall assessment of all aspects of his or her job (spector, 1997). the concept of job satisfaction has attracted the attention of the research community since the birth of the management discipline. in 1931, fisher and hanna pioneered the notion of job satisfaction by suggesting that it is a product of workers’ emotional maladjustment (fisher & hanna, 1931). hoppock (1935) further empirically explored this topic and showed that job satisfaction is related to employees’ socio-demographic characteristics and compensation. locke (1970) further analyzed the relationship between job satisfaction and productivity, and churchill, ford, and walker (1974) proposed the use of occupation-specific measurement techniques for job satisfaction. after this, job satisfaction has become a wellrecognized management research topic (judge, 1993), and it soon entered the realm of it research (igbaria & siegel, 1992; mcknight, phillips, & hardgrave, 2009). literature indicates that some studies have examined a direct relationship between job satisfaction and turnover intention, while others have explored a relationship among job satisfaction, turnover intention, and organizational commitment. in particular, some non-it models included the organizational commitment construct, positioned as an antecedent of turnover intention or as a moderator of the job satisfaction  turnover intention relationship. in the it field, however, the majority of studies have omitted organizational commitment and focused on the job satisfaction  turnover intention relationship as well as on the antecedents of job satisfaction. accordingly, we excluded organizational commitment as well and positioned job satisfaction as an antecedent of turnover intention. furthermore, we also identified factors that affect job satisfaction of it professionals in japan. a further analysis of the literature revealed four antecedents of job satisfaction which may be fruitfully employed in this study’s context: work exhaustion, personal accomplishment, workhome conflict, and friendship networks. the following sub-sections describe these constructs in more detail. 2.3.2 work exhaustion prior research suggests that work exhaustion-related constructs, for instance fatigue, tension, burnout, and stress, are frequently positioned as antecedents of job satisfaction. as documented in table 1, work exhaustion has also been identified as an important factor in the context of employee turnover due to the “3k” situation in japan. we selected work exhaustion as an antecedent of job satisfaction because it is a higher-level construct defined as the depletion of mental resources needed to cope with one’s work demands (lee, 2011), and it is australasian journal of information systems serenko et al. 2022, vol 26, research article turnover in japanese it professionals 7 supposed to have a negative effect on job satisfaction. tension, burnout, and stress may be regarded as factors contributing to work exhaustion. previous researchers have also considered work exhaustion as a variable that influences job satisfaction and/or turnover intention (cho, rutherford, friend, hamwi, & park, 2017; kim, 2012; rutherford, wei, park, & hur, 2012). the notion of work exhaustion was introduced in it research by moore (2000) who built on and extended several concepts in management and psychology literatures which posit that organizational work environment has an impact on employees’ psychological health and mental state. in this pioneering work, moore (2000) argued that, while work exhaustion takes place in many occupations, it workers are particularly vulnerable to work exhaustion due to the very nature of the profession. for example, it workers have to keep up with constant technical changes, acquire new skills and unlearn old ones, compete for organizational resources, embrace ambiguity, and accept high workloads. as a result, they may feel drained, taxed, and overextended leading to work exhaustion. most importantly, high levels of work exhaustion reduce the level of job satisfaction of it workers (kim & wright, 2007): this makes the work exhaustion construct highly relevant in the context of the present study. 2.3.3 personal accomplishment another important construct that predicts job satisfaction is personal accomplishment at work, defined as employees’ feeling of competence and successful achievement in the workplace (shih, jiang, klein, & wang, 2013). personal accomplishment is different from work exhaustion because the former is related to employees’ perceptions of their professional growth while the latter is determined based on the interplay between workplace demands and the extensiveness of people’s resources. the degree of personal accomplishment depends on a number of factors, many of which are controlled by an organization. for example, if an organization offers sufficient resources, support, social guidance, relevant feedback from supervisors, and an opportunity to control one’s own work, workers may feel empowered which, in turn, boosts their perceptions of personal accomplishment within their professional environment (abhicharttibutra & tungpunkom, 2019). employees who have a sense of personal accomplishment find their work to be meaningful, feel competent, and are self-determined to further contribute to an overall organizational success. the importance of personal accomplishment and its predictive power are highlighted by the fact that this construct is included as one of the three dimensions in the maslach burnout inventory (maslach, jackson, & leiter, 1997). in addition, personal accomplishment is highly relevant in the it domain. it occupational culture is very unique and differs from other occupational cultures according to six dimensions that are valued by it workers: autonomy in decision-making, structure in environment, precision in communication, innovation in technology, reverence for technical knowledge, and enjoyment at the workplace (cranefield, gordon, palvia, serenko, & jacks, 2022; jacks & palvia, 2014; jacks, palvia, iyer, sarala, & daynes, 2018). each of these dimensions is related to the feeling of personal accomplishment. for instance, decision-making autonomy and flexible structure facilitate independent thinking and freedom which are necessary to excel in a dynamic it environment. precision in communication eliminates ambiguity and helps it employees make better decisions which boosts their competence. an opportunity to experiment with the latest innovative technologies may fulfil a need for being competent and staying up to date with recent it breakthroughs. reverence for technical knowledge may fulfill australasian journal of information systems serenko et al. 2022, vol 26, research article turnover in japanese it professionals 8 a need for career growth and future employability, and enjoyment in the workplace may create a perception that, as fully accomplished professionals, it workers may combine work with pleasure, instead of pressure. as a result, the unique nature of it occupational culture corresponds to a need for personal accomplishment at work. therefore, this study proposes that personal accomplishment of it workers is important in the study’s context. 2.3.4 work-home conflict work-home conflict, which refers to the irreconcilable demands from work and family (ahuja, chudoba, kacmar, mcknight, & george, 2007; turel, serenko, & bontis, 2011), is a third important antecedent of job satisfaction. having a happy personal and family life is extremely important for all individuals regardless of their occupation (kreiner, 2006). spending time with family and friends helps people relax after work and prepare for another productive working day. however, the it profession in all countries, including japan, is extremely demanding: as the volume of technical knowledge grows, it employees are expected to continuously update their skills which is often done beyond regular working hours. tight budgets, unrealistic deadlines, and various emergencies (e.g., security breaches, software crashes, network issues) have become a part of today’s it profession, and workers are expected to deal with ad-hoc situations almost in real-time by making themselves available 24/7. as a result, the boundary between work and home has become blurred, and it workers often find it difficult, or even impossible, to manage the incompatible demands between work and personal life (kreiner, 2006). given the limited availability of time and cognitive resources, excessive demand in one domain puts pressure on another leading to an irreconcilable conflict (ten brummelhuis & bakker, 2012). when a work-family conflict lasts for a significant period of time, it employees become frustrated with their current job or the chosen profession which undermines their job satisfaction. this makes the work-family conflict construct highly applicable for the inclusion in this study. 2.3.5 friendship networks the fourth, last antecedent of job satisfaction explored in this study refers to employees’ friendship networks. friendship is “a voluntary, personal relationship, typically providing intimacy and assistance, in which two parties like one another and seek each other’s company” (fehr, 1996, p. 7). people establish friendship in various contexts, including the workplace. when people work in close proximity, they start sharing common ground and engage in extraorganizational socializing during and beyond formal working hours (sias & cahill, 1998). as a result of such interactions, workplace friendships often develop, which eventually evolve into the networks of friendship ties among multiple co-workers. friendship networks refer to informal circles of friends in one’s work environment where employees approach one another for help and maintain strong personal relationships. these networks reflect nonexclusive workplace relationships that involve mutual trust, commitment, reciprocal linking, and shared interests or values (berman, west, & richter, 2002). in friendship networks, employees help one another with various matters, ranging from help with job-related tasks to assistance with intimate personal matters (sias & cahill, 1998). recall that japan has a strong collectivistic culture that values loyalty to one’s organization and workers feel themselves highly interdependent (gelfand et al., 2004; huff & kelley, 2003; nakane, 1970). thus, japanese it employees are likely to value relationships established within their organizational friendship networks and heavily rely on them for both work issues and personal matters. as a result, the strength of organizational friendship networks is australasian journal of information systems serenko et al. 2022, vol 26, research article turnover in japanese it professionals 9 expected to be positively related to employees’ job satisfaction (raile et al., 2008), while workhome conflict sheds light on the negative side. 2.3.6 individualistic factors vs. collectivistic factors the four antecedents of job satisfaction discussed above – work exhaustion, personal accomplishment, work-home conflict, and friendship networks – may be classified into two general categories: individualistic factors and collectivistic factors. work exhaustion and personal accomplishment represent individualistic factors because they are formed based on the interaction of each employee with his or her workplace environment in isolation from other factors. as such, they pertain to the workplace-employee interaction on the individual level when workers reflect on the direct effect of their work environment on themselves. other individuals (family members, co-workers) do not directly affect employees’ perceptions of work exhaustion and personal accomplishment. work-home conflict and friendship networks are considered collectivistic variables because employees form their perceptions of these factors based on their interaction within their home and professional environments, respectively. as such, it is the actions of other people that determine how employees perceive and respond to these constructs when being submerged in a collective. in fact, perceptions of work-home conflict and friendship networks cannot exist without the influence of other individuals. the inclusion of these collectivistic factors is particularly important given the collectivistic nature of the japanese culture (hofstede, 1980). table 2 summarizes the constructs used in this study and their conceptual definitions. construct definition source job satisfaction an employee’s overall evaluation of all aspects of his or her job. spector (1997) friendship networks informal circles of friends in one’s work environment where employees approach one another for help and maintain strong personal relationships. berman et al. (2002) personal accomplishment an employee’s feeling of competence and successful achievement in the workplace. shih et al. (2013) turnover intention an employee’s conscious and deliberate willingness to leave his or her current organization. tett & meyer (1993) work exhaustion the depletion of mental resources needed to cope with one’s work demands. lee (2011); moore (2000) work-home conflict an employee’s perception of the irreconcilable demands from work and family. ahuja et al. (2007); turel et al. (2011) table 2. constructs and their definitions 2.4 hypotheses development we propose and empirically test a model explicating the antecedents of turnover intention of japanese it workers. the model shows that turnover intention is driven by job satisfaction, which, in turn, is influenced by work exhaustion and personal accomplishment, which represent individualistic factors, and work-home conflict and friendship networks, which reflect collectivistic factors. the impact of these antecedents on turnover intention is fully mediated through job satisfaction: it is postulated that none of these variables directly affects turnover intention. the strength of these relationships is believed to differ between younger and older organizations. figure 1 presents this study’s model. the rest of this sub-section elaborates on hypotheses development in detail. australasian journal of information systems serenko et al. 2022, vol 26, research article turnover in japanese it professionals 10 in the proposed model, we hypothesize a direct negative path between job satisfaction and turnover intention. as demonstrated by previous studies (e.g., see ahmed, taskin, pauleen, & parker, 2017; bhagwatwar, bala, & ramesh, 2014; lee, 2003; maier, laumer, eckhardt, & weitzel, 2015; soonhee & wright, 2007; tripathi & srivastava, 2020), it workers who are less satisfied with their jobs are more likely to voluntarily leave their organizations. the relationship between job satisfaction and turnover intention may be explicated from the perspective of mobley’s (1977; mobley et al., 1978) heuristic model of employee turnover. this model is robust, and it has been applied in a variety of industries (hom, caranikas-walker, prussia, & griffeth, 1992), including it (rouse, 2001). figure 1. the research model the heuristic model of employee turnover suggests that when individuals elaborate on and evaluate all aspects of their jobs, they reach a certain mental state of some degree of satisfaction or dissatisfaction with their current place of employment. if they are satisfied with it, they routinely perform their duties. however, in the case of dissatisfaction, they initiate a withdrawal decision process. as a first step, they trigger the thoughts of quitting which lead to an assessment of the expected utility of search for an alternative position (e.g., the likelihood of securing a more satisfactory job), search costs (e.g., time lost, travel, other expenses), and the cost of quitting (e.g., loss of seniority, status). if the expected utility of search is low while the costs are high, the employee may decide to re-evaluate the current job, adjust the level of job satisfaction, stop thinking of quitting, and select other withdrawal methods, for instance, to reduce work effort and engage in counterproductive workplace behavior. however, if the probability of securing a suitable position is high while the costs are acceptable, an individual consciously forms a behavioral intention to search for an alternative position and to compare the available jobs with the current one, and if the alternative appears to be more potentially satisfactory than the present job, the person forms a deliberate intention to quit (i.e., turnover intention). as such, job satisfaction is considered a key factor within the heuristic model of h5 h4 h3 h2 individualistic factors h1 collectivistic factors personal accomplishment turnover intention job satisfaction work exhaustion work-home conflict friendship networks moderator: the age of an organization australasian journal of information systems serenko et al. 2022, vol 26, research article turnover in japanese it professionals 11 employee turnover because it triggers the entire withdrawal process (mobley, 1977; mobley et al., 1978). the same line of reasoning applies in the context of it workers in japan: when they are generally satisfied with the overall aspects of their jobs, they do not activate a withdrawal process and are likely to maintain the relationship by staying with their organization, but as their satisfaction level decreases, they are more likely to trigger the withdrawal process which ultimately leads to the development of turnover intention. thus, we hypothesize: h1: job satisfaction has a negative direct effect on turnover intention. job satisfaction is best conceptualized as an attitudinal construct which reflects one’s overall evaluation of all aspects of his or her job. job satisfaction does not suddenly appear on its own; its development is influenced by a number of affective experiences. in this study, we position work exhaustion, personal accomplishment, work-home conflict, and friendship networks as antecedents of job satisfaction of japanese it professionals and explicate their effect by using affective events theory (weiss & cropanzano, 1996) as a conceptual lens of analysis. affective events theory posits that employees’ work attitudes, including job satisfaction, are developed based on affective responses to their first-hand workplace experience (weiss & cropanzano, 1996). affect refers to mood and emotions which facilitate the development of evaluative judgements regarding all aspects of one’s job (i.e., job satisfaction) (weiss, 2002; weiss at el., 1999). in other words, employee-workplace interaction processes trigger a number of moods and emotions which in turn determine employees’ job satisfaction. below, we hypothesize and describe how each of the antecedent variables influences employees’ affective states which lead to their job satisfaction. as discussed before, burnout (shimizu et al., 2005a; suzuki et al., 2010), job stress, strain (kachi, inoue, eguchi, kawakami, shimazu, & tsutsumi, 2020), psychological distress, and fatigue (tei-tominaga & miki, 2010), which reflect one’s general state of work exhaustion, have emerged in several studies of employee turnover in japan. previous studies in many countries and various industries, including it, have also suggested that work exhaustion is a critical factor that influences job satisfaction (ashill & rod, 2011; cho et al., 2017; kim, 2012; lee, 2011; moore, 2000; rutherford et al., 2012). for the japanese it professionals in particular, the “3k” phenomenon, referred to earlier, may contribute to work exhaustion and employees may feel stress, burnout, frustration, and fatigue leading to major psychological and physical discomfort. because these negative emotions result from employees’ exposure to their workplace, they are likely to attribute the cause of their problems to their job and become dissatisfied with it. therefore, we postulate: h2: work exhaustion has a negative direct effect on job satisfaction. personal accomplishment refers to feelings of competence and successful achievement in one’s professional environment (maslach et al., 1997). it is related to perceived mastery experience and self-realization which boosts employees’ job self-efficacy (bang & reio, 2017). recall that the japanese it workplace is focused on the development of custom solutions, and it professionals are considered craftsmen possessing specific skills and producing high-quality systems (sasaki et al., 2020). however, the japanese management culture is very mechanistic and bureaucratic. it jobs are challenging, demanding, and require substantial effort which may not be easily recognized by managers and colleagues. moreover, while the development of zero-defect customized systems requires it workers to use a variety of innovative solutions, australasian journal of information systems serenko et al. 2022, vol 26, research article turnover in japanese it professionals 12 the bureaucratic management style may impede their creative process. as a result, many japanese it workers may feel that they cannot realize their potential in the workplace. according to maslow’s (1970) hierarchy of needs, a feeling of personal accomplishment is an important psychological need that boosts people’s self-confidence and self-esteem. when it employees cannot realize their full potential, they experience negative emotions and attribute the source of their negative feelings to their present organization. in contrast, those who realize their professional potential are likely to be in a good mood and develop positive emotions towards their employer. in addition, personal self-accomplishment is related to employees’ professional self-identity (edwards & dirette, 2010): those who believe that they have accomplished their workplace objective boost their professional self-identity which, in turn, leads to positive perceptions of employers and increases their job satisfaction. previous research has also identified personal accomplishment as a reliable predictor of job satisfaction in several contexts (brewer & clippard, 2002; koeske, kirk, koeske, & rauktis, 1994). thus, consistent with affective events theory (weiss & cropanzano, 1996), we hypothesize: h3: personal accomplishment has a positive direct effect on job satisfaction. work-home conflict causes imbalance between an it professional’s work responsibilities and personal life (ahuja et al., 2007; dinger, thatcher, & stepina, 2010; igbaria & guimaraes, 1993; ngo, foley, & loi, 2005) and it affects both men and women employees (emslie, hunt, & macintyre, 2004). it is an independent construct that is not directly associated with work exhaustion (armstrong, brooks, & riemenschneider, 2015). the line between personal and work time of a contemporary it worker is often blurred; it is expected that it employees respond to ad-hoc requests and deal with emergencies far beyond regular working hours. furthermore, an accelerating velocity of change in the it sector requires it workers to continuously update their skills to effectively perform their jobs and to remain competitive on the job market, which requires a tremendous investment of time at the expense of familyrelated activities. at the same time, most employees also wish to spend time with their family. as a result, when work interferes with family life, employees consider it a negative event that is accompanied by psychological and physiological tension (grandey & cropanzano, 1999). this tension, in turn, produces negative emotions and, consistent with affective events theory (weiss & cropanzano, 1996), decreases employees’ job satisfaction because they believe that it is their employer that is solely responsible for creating a sense of irreconcilable differences between their professional and personal lives. accordingly, we propose: h4: work-home conflict has a negative direct effect on job satisfaction. many people seek social support, interaction, communication, and integration within their personal and professional communities (gottlieb & bergen, 2010; rath, 2006). in the workplace, they tend to form and maintain friendship networks consisting of informal circles of like-minded co-workers who they approach for help if needed and offer assistance as a form of reciprocation (serenko & bontis, 2016). developing and maintaining friendship networks contributes to job satisfaction for several reasons (raile et al., 2008). first, strong friendship networks create an atmosphere of mutual trust and collegiality which leads to positive feelings towards the entire workplace. second, friendship networks attenuate a negative impact of tension arising from work-related duties (kirmeyer & dougherty, 1988) which reduces negative emotions associated with one’s organization. third, having strong friendship networks creates a perception that organizations care for their employees by fostering a positive workplace environment which, in turn, boosts employees’ morale and mood. the it australasian journal of information systems serenko et al. 2022, vol 26, research article turnover in japanese it professionals 13 profession in japan is challenging, dynamic, and requires workers to frequently consult with their peers on various it-related issues. given the collectivistic nature of the japanese culture and the workplace environment, friendship networks are particularly important because workers expect their organizations to create an environment conducive to social exchange and support. as a result, friendship networks should have a positive effect on the level of job satisfaction of japanese it employees. thus, we propose: h5: friendship networks have a positive direct effect on job satisfaction. previous research has established that the age of an organization is an important variable affecting firms operating in various industries. in particular, it has been found that an organization’s age is positively related to its probability of surviving (kalleberg & leicht, 1991), mature firms are more likely to set long-range objectives than younger ones (titus, covin, & slevin, 2011), younger and older organizations behave, compete, and perform differently (george, 2005), and stakeholders (employees, customers, distributors, suppliers, and bankers) prefer supporting older organizations (choi & shepherd, 2005). as organizations mature, they become less agile and less responsive to a constantly changing environment unless they facilitate the infusion of fresh human capital (jain, 2016). older organizations are less innovative and prefer maintaining a status quo: voss and voss (2013) show that an organization’s age is negatively correlated with product and market exploration, and choi and phan (2014) demonstrate that as organizations mature, their rates of new product development slow down. in addition to various differences at the strategic level, employee perceptions of older and younger organizations also differ (e.g., see glisson & durick, 1988; morris & sherman, 1981; popescu, deaconu, & popescu, 2015). in this study, we hypothesize that, in the japanese it sector, the age of an organization affects employee behavior. on the one hand, the competitive landscape of the japanese it industry is highly fixed and hierarchical where a few major players occupy a leading position (sasaki, 2012). such companies tend to be older and larger in size. on the other hand, the it domain is very dynamic by its nature where constantly emerging disruptive innovations represent lucrative opportunities for new entrants. as a result, the japanese it industry is also represented by a large number of younger small and medium sized enterprises jockeying for position. we, therefore, posit that it is likely that the strength of the causal relationships proposed in h1-h5 depends on the age of an organization: the relationships are stronger for younger than for older organizations. the rationale is that older organizations are likely to have a stronger culture of long-term employment than their younger counterparts which are less bureaucratic but are more open, flexible, and innovative. younger japanese it companies may exhibit a weaker culture of long-term employment than their older counterparts because the former have not had enough time to build a cumulative body of organization-specific traditions, norms, and values that promote loyalty to their employers. with respect to the satisfaction-turnover intention relationship (h1), it is argued that in older organizations with strong long-term employment culture, it workers’ decisions to remain with their current employers may be affected by other factors, particularly loyalty and culture. as a result, consistent with the heuristic model of employee turnover (mobley 1977; mobley et al., 1978), those working in older organizations are less likely to initiate a withdrawal decision process when they feel that they are unsatisfied with their current job than those employed in younger organizations which do not have a strong long-term employment culture. in terms of h2-h5, it is hypothesized that the impact of affective responses to firstaustralasian journal of information systems serenko et al. 2022, vol 26, research article turnover in japanese it professionals 14 hand workplace experience of older-organization employees on their level of job satisfaction is suppressed by the perceptions of long-term employment. when people’s job security cannot be threatened and they believe that they must remain loyal to their company, they are likely to continue being satisfied with their jobs regardless of their perceptions of work exhaustion, personal accomplishment, work-home conflict, and friendship networks. in contrast, job satisfaction of those working in younger organizations, where the culture of long-term employment is less prominent, may be strongly driven by their workplace mood and emotions (as per affective events theory by weiss & cropanzano (1996)). thus, the following hypothesis is proposed: h6: the strength of the relationships in h1-h5 is stronger for younger than for older organizations. 3 methods empirical data were collected as part of the world it project by means of an online survey of japanese it workers. the world it project was initiated in 2013 and collected data until 2017, as a response to the fact that it research is dominated by western views. this project involves more than 80 researchers from 37 different countries who collected data from it employees by using the same survey instrument (palvia et al., 2017; 2018; 2020). it focuses on various organizational, technological, and individual issues of it employees in the unique national, cultural, and organizational contexts of the participating countries. as such, the participating countries represent various economic, political, religious, and regional settings. the entire survey contained 160 items. all items were based on previously validated instruments, and the final questionnaire was pilot tested in several countries and refined as needed. items that were used to operationalize the constructs employed in this study were measured on a 5-point likert-type scale. the survey was administered in japanese. it was translated into japanese from english and then back-translated and adjusted as needed. email invitations were sent to a number of random it workers from all regions of japan asking them to respond to the online questionnaire. the list of emails was provided by a web survey company that specializes in online data collection for research purposes. in all, 310 responses were received. of these, 26 were incomplete and removed, leaving a sample of 284 valid responses. of these, 9.2% of the respondents were women. table 3 presents key statistics of the constructs and their items. an initial confirmatory factor analysis for all constructs and indicators found that, with a factor loading cut-off value of 0.5, items fn1 (0.428) and ti2 (0.485) did not meet the criterion and were removed. 4 results in order to further assess the reliability and validity of all factors, we conducted a confirmatory factor analysis using the maximum likelihood estimation method with the criteria provided by fornell and larcker (1981) and hair, black, anderson, & babin (2018). we determined that: a) standardized loading estimates were 0.5 or higher; b) for discriminant validity, both maximum shared variance (msv) and the average shared squared variance (asv) were lower than the average variance extracted (ave) for all constructs; and c) composite reliability (cr) was 0.7 or higher. cronbach’s alpha values confirmed internal consistency of the measures because they met or exceeded the threshold of 0.7. table 4 summarizes the results indicating australasian journal of information systems serenko et al. 2022, vol 26, research article turnover in japanese it professionals 15 that every criterion was met. overall, the constructs possess sufficient reliability and validity, enabling the model’s use in subsequent steps. construct indicator average standard deviation factor loading js1 in general, i like working here. 2.49 0.93 0.811 js2 all in all, i am satisfied with my current job. 2.70 0.94 0.794 js3 in general, i don’t like my current job. (r) 2.62 1.00 0.533 fn1 when you face a problem in your professional work that may be of a technical nature or otherwise, to what extent do you seek help from your personal circle of friends or acquaintances who may or may not be in your organization 2.45 0.68 0.462* fn2 when these friends or acquaintances seek help from you in problems of technical nature or otherwise, to what extent do you attempt to help them? 1.96 0.67 0.838 fn3 to what extent do you maintain a relationship with these friends or acquaintances? 2.17 0.76 0.651 pa1 i feel i’m making an effective contribution to what this organization does. 2.49 0.83 0.704 pa2 in my opinion, i do a good job. 2.50 0.83 0.799 pa3 i have accomplished many worthwhile things in this job. 2.49 0.84 0.849 pa4 at my work, i feel confident that i am effective at getting things done. 2.34 0.78 0.792 ti1 i will be with this organization one year from now. (r) 3.74 0.95 0.839 ti2 i will take steps during the next year to secure a job at a different organization. 3.37 0.97 0.488* ti3 i will be with this organization five years from now. (r) 3.26 1.03 0.673 we1 i feel drained from activities at work. 2.95 1.10 0.895 we2 i feel tired from my work activities. 2.92 1.16 0.905 we3 working all day is a strain for me. 2.96 1.04 0.705 we4 i feel burned out from my work activities. 3.44 1.07 0.672 whc1 there is a blurring of boundaries between my job and my home life. 3.33 1.10 0.697 whc2 my work-related responsibilities create conflicts with my home responsibilities. 3.23 1.09 0.771 whc3 i do not get everything done at home because i find myself completing job-related work. 3.39 0.98 0.818 note: * – removed item (factor loading < 0.5). js: job satisfaction; fn: friendship networks; pa: personal accomplishment; ti: turnover intention; we: work exhaustion; whc: work-home conflict. r – negativelyworded items. table 3. key statistics of construct items a covariance-based structural equation modeling technique was used to assess the structural model by means of amos (arbuckle, 2006). the model had acceptable fit: cfi (0.947: slightly lower than the cut-off criteria); nfi (0.900); nnfi (0.933); χ2 /df (1.99: χ2 = 240.412, df = 121); smar (0.0796); rmsea (0.059) (ahmad & islam, 2018; browne & cudeck, 1993; galarraga, saies, cecchini, arruza, & luis-de-cos, 2017; hu & bentler, 1999; li, fang, wang, sun, & cheng, 2018). figure 2 illustrates that all standardized coefficients were significant at the level of 5% or below, except the work-home conflictjob satisfaction path. thus, all of the hypotheses except h4 were supported. next, we added direct paths from work exhaustion, personal accomplishment, work-home conflict, and friendship networks to turnover intention (i.e., the saturated model, see figure 3) and re-evaluated the model. the measures of fit were acceptable as determined by the indices’ australasian journal of information systems serenko et al. 2022, vol 26, research article turnover in japanese it professionals 16 thresholds: cfi (0.946: slightly lower than the cut-off criteria); nfi (0.900); nnfi (0.929); χ2 /df (2.04: χ2 = 239.110, df = 117); smar (0.0791), rmsea (0.061: slightly higher than the cut-off criteria). most importantly, we found no new statistically significant path coefficients to turnover intention at the level of p<0.05 while all previously established relationships remained significant. therefore, the effect of work exhaustion, personal accomplishment, and friendship networks on turnover intention is fully mediated through job satisfaction. construct indicator factor loading cronbach’s alpha ave msv asv cr job satisfaction js1 0.814 0.74 0.524 0.393 0.165 0.762 js2 0.794 js3 0.528 friendship networks fn2 0.780 0.70 0.541 0.165 0.055 0.701 fn3 0.688 personal accomplishment pa1 0.703 0.87 0.620 0.197 0.102 0.867 pa2 0.799 pa3 0.848 pa4 0.793 turnover intention ti1 0.822 0.72 0.572 0.393 0.113 0.726 ti3 0.685 work exhaustion we1 0.896 0.87 0.643 0.404 0.119 0.876 we2 0.905 we3 0.705 we4 0.672 work-home conflict whc1 0.696 0.80 0.583 0.404 0.098 0.807 whc2 0.770 whc3 0.819 ave: average variance extracted; msv: maximum shared variance; asv: average shared squared variance; cr: composite reliability. table 4. confirmatory factor analysis figure 2. the structural model r2=0.38 r2=0.29 .19* -.04 (n.s.) .32*** -.34** individualistic factors -.62*** collectivistic factors personal accomplishment turnover intention job satisfaction work exhaustion work-home conflict friendship networks *** p<0.001 ** p<0.01 * p<0.05 australasian journal of information systems serenko et al. 2022, vol 26, research article turnover in japanese it professionals 17 figure 3. the saturated model in order to examine whether the strength of the relationships is stronger for younger than for older organizations (i.e., h6), samples were stratified into two groups: younger organizations having the age from 1 to 29 years (158 firms) and older organizations having the age of 30 years or more (126 firms). figure 4 shows the results of pairwise parameter comparisons conducted in amos (arbuckle, 2006). we observed statistically significant differences in path coefficients between younger (y) and older (o) firms for the following relationships: work exhaustionjob satisfaction (y stronger negative effect), personal accomplishmentjob satisfaction (y stronger positive effect), and work-home conflictjob satisfaction (y stronger positive effect). no statistically significant differences for the friendship networksjob satisfaction and job satisfactionturnover intention links were observed. overall, it is concluded that h6 is partially supported. table 5 summarizes the results of hypothesis testing. n hypothesis β p-value confirmed? h1 job satisfaction has a negative direct effect on turnover intention. -0.62 0.001 yes h2 work exhaustion has a negative direct effect on job satisfaction. -0.34 0.01 yes h3 personal accomplishment has a positive direct effect on job satisfaction. 0.32 0.001 yes h4 work-home conflict has a negative direct effect on job satisfaction. -0.04 n.s. no h5 friendship networks have a positive direct effect on job satisfaction. 0.19 0.05 yes h6 the strength of the relationships in h1-h5 is stronger for younger than for older organizations. na na partially table 5. hypotheses testing summary r2=0.31 r2=0.42 .10 (n.s.) .03 (n.s.) .01 n.s. -.06 (n.s.) .21* .04 (n.s.) .32*** -.36** individualistic factors -.69*** collectivistic factors personal accomplishment turnover intention job satisfaction work exhaustion work-home conflict friendship networks *** p<0.001 ** p<0.01 * p<0.05 additional paths australasian journal of information systems serenko et al. 2022, vol 26, research article turnover in japanese it professionals 18 figure 4. the effect of the age of an organization (statistically significant different pairs of beta coefficient are in bold) 5 discussion the purpose of this study is to understand the factors driving turnover intention of japanese it professionals. a model was proposed based on the extant literature, data was collected from 284 employees, and the model was tested by means of structural equation modelling. the findings resulted in a number of theoretical and practical contributions which are presented below after an overview of the findings. 5.1 overview of the findings first, the results highlight the fact that job satisfaction serves as a full mediator of the impact of work exhaustion, personal accomplishment, and friendship networks on turnover intention which further emphasizes the importance of job satisfaction of it employees. this conclusion is consistent with the heuristic model of employee turnover (mobley, 1977; mobley et al., 1978) because it shows that on their own, high work exhaustion, insufficient personal accomplishment, and ineffective friendship networks cannot directly affect turnover intention of japanese it professionals unless they reduce their level of job satisfaction. low job satisfaction, in turn, triggers a withdrawal process upon which employees form turnover intention. second, our study confirms that work exhaustion, personal accomplishment, and friendship networks have a direct effect on the level of job satisfaction of japanese it workers. of these, work exhaustion has the strongest (negative) effect (β=-0.34, p<0.01). it shows that japanese it r2=0.47 r2=0.34 y: .35*** o: .22 y: .44** o: -.42** y: -.62*** o: -.11 individualistic factors y: -.69*** o: -.45*** collectivistic factors personal accomplishment turnover intention job satisfaction work exhaustion work-home conflict friendship networks *** p<0.001 ** p<0.01 * p<0.05 y: .08 o: .28 australasian journal of information systems serenko et al. 2022, vol 26, research article turnover in japanese it professionals 19 workers are not immune to stress, workload, and pressures which have become apart of the daily life of it professionals. higher levels of work exhaustion reduce the degree of job satisfaction of japanese it workers which makes them more likely to go on the job market. prior research suggests that japanese employees work long hours and experience constantly growing work pressure, stress, and accountability (iida & morris, 2008). these take toll on their physical and emotional health (kawakami & haratani, 1999) and, in some extreme cases, may even lead to suicide (tamakoshi et al., 2000). at a bare minimum, exhausted it professionals are likely to leave their employers and look for more appropriate working conditions elsewhere. this finding is consistent with the conclusions reported by liu (2010) and tei-tominaga and miki (2010) who emphasize the role of job stress of japanese workers in the it and healthcare industries, respectively. as was theoretically expected, personal accomplishment (β=0.32, p<0.001) has a strong positive effect on job satisfaction. in all countries, including japan, it jobs are challenging: they require workers to continuously update their skills, and lifelong learning has become an integral part of the profession. as a result, it workers who believe that they have achieved professional accomplishments feel a sense of self-reward, recognition, and pride. they also feel that their present position helps them establish their professional identity, and they are likely to attribute the source of these positive perceptions to their employer, which is reflected in higher job satisfaction. friendship networks (β=0.19, p<0.05), have a less salient yet statistically significant effect on job satisfaction. friendship networks help it employees develop positive emotions towards their workplace. most people derive fulfilment from communicating, interacting, receiving help, and assisting others (báez-mendoza & schultz, 2013), and it employees are no exception. thus, when organizations foster collaborative friendship networks, employees become more satisfied with their jobs. third, in contrast to the extant literature, it was observed that work-home conflict does not seem to affect job satisfaction. several theoretical explanations of this phenomenon can be put forward. it is possible that japanese it workers consider their professional career more valuable than their personal life, including relationships with family and friends. thus, they experience little or no negative emotions when their work interferes with their personal life, and their level of job satisfaction does not depend on work-family conflict. also, from the perspective of boundary theory (ashforth, kreiner, & fugate, 2000), it is possible that japanese it employees have learned to achieve a certain level of work-home balance by creating two independent segments or compartments: one for work and another for family which do not overlap. thus, as spillover theory suggests (grzywacz, 2002), they do not let their workplace emotions, feelings, and mood enter the realm of their personal lives. as a result, they do not associate family and/or personal problems with their workplace. fourth, an analysis of the differences in the model’s structural relationships between younger and older organizations reveals two interesting phenomena. as indicated in figure 4, three out of five structural relationships vary depending on the age of an organization. particularly, a negative effect of work exhaustion and a positive effect of personal accomplishment on job satisfaction exist only for younger organizations. a difference between the younger and older organizations in terms of the link strength between job satisfaction and turnover intention is also consistent with the observation above (i.e., it is stronger for younger firms: β=-0.69 for younger vs. β=-0.45 for older organizations); however, this difference is not statistically australasian journal of information systems serenko et al. 2022, vol 26, research article turnover in japanese it professionals 20 significant. nevertheless, this shows that employees in younger organizations may embrace the concept of long-term employment to a lesser degree than those working in older organizations. generally, younger japanese organizations embrace a stronger individualistic culture whereas their older counterparts exhibit a stronger collectivistic culture. therefore, the collectivistic culture which is a part of older organizations suppresses the effect of work exhaustion and personal accomplishment on job satisfaction and, subsequently, on turnover intention. this suggests that in organizations with strong collectivistic cultures, it workers are less likely to be impacted by their personal preferences, feelings, and perceptions in the context of voluntary turnover. fifth, stratifying the samples into younger and older organizations clarifies the reason why the work-home conflictjob satisfaction path (h4) was not statistically significant in the combined sample. actually, the path coefficient of work-home conflictjob satisfaction for younger organizations was positive and significant (β=0.44, p<0.01), whereas that for older organizations was negative and significant (β=-0.42, p<0.01). resultantly, a combination of both groups could not provide a statistically significant beta coefficient. sixth and last, based on the r-squared values, the proposed model has a good explanatory power: it predicts 42% and 31% of variance in turnover intention and job satisfaction, respectively, which is considered high in management research. 5.2 contribution to theory first, the findings confirm the efficacy of the heuristic model of employee turnover (mobley, 1977; mobley et al., 1978) in the context of the relationship between job satisfaction and turnover intention of it workers in japan. the heuristic model of employee turnover suggests that employees continuously evaluate and re-evaluate all aspects of their jobs to reach a certain state of satisfaction or dissatisfaction with their place of employment. in the case of dissatisfaction, they initiate a certain withdrawal process which may result in an intention to leave their employer. consistent with previous japanese research (e.g., see de moura, abrams, retter, gunnarsdottir, & ando, 2009; honda-howard & homma, 2001; huang, 2011), this study shows that job satisfaction has a strong negative effect on turnover intention. second, this study shows that affective events theory (weiss & cropanzano, 1996) applies well in the context of japanese it professionals. the theory posits that employees’ affective firsthand experiences at work influence their mood and emotions which, in turn, determine their evaluative judgements of all aspects of their job. in this study, it is shown that it employees’ perceptions of their work exhaustion, personal accomplishment, and friendship networks are emotion-based perceptions resulting from actual on-the-job experiences which lead to job satisfaction. as such, this theory is robust which confirms that future researchers may successfully employ it in other countries and contexts. third, the fact that turnover intention of japanese it professionals is determined by their job satisfaction, which, in turn, is driven by their work exhaustion, personal accomplishment, and friendship networks, shows the fragility of the “long-term employment” paradigm – which is supposed to be embraced within the entire japanese work culture. under the long-term employment assumptions, japanese it workers should exhibit little, if any, intention to leave their organizations regardless of their first-hand on-the-job experience. moreover, their level of job satisfaction should not be strongly negatively associated with turnover intention – they are expected to remain loyal to their employer regardless of their workplace perceptions. yet, australasian journal of information systems serenko et al. 2022, vol 26, research article turnover in japanese it professionals 21 an entirely different picture has emerged from this study, suggesting that the notion of longterm employment has become less dominant given the unique nature of the it industry. fourth, a comparison of individualistic factors (work exhaustion and personal accomplishment) with collectivistic factors (work-home conflict and friendship networks) reveals that the former have a stronger impact on job satisfaction than the latter. this observation has important theoretical implications. there are two competing perspectives on cultural variability – divergence and convergence (stohl, 2001). the divergence viewpoint posits that people living in different countries vary due to diverse social-psychological and anthropological roots. the macroeconomic gravity model (isard, 1954; van bergeijk & brakman, 2010) further assumes that the more geographically distant countries are from one another, the less likely they are to trade and to share cultural values. for example, japan is physically distant from the western world and, therefore, its citizens are expected to exhibit cultural values that are different from those of their western counterparts. in contrast, the convergence perspective postulates that a growing emphasis on individualism, fueled by globalization trends and social media, gradually minimizes cultural differences resulting in a highly homogeneous world, organizational structures, national cultures, and work environments (stohl, 2001). the present investigation supports the convergence perspective in the context of japanese it workforce where individualistic factors play a more salient role than collectivistic ones despite the overall collectivistic national culture of japan. it further proves the existence of an it occupational culture (cranefield et al., 2022; jacks & palvia, 2014; jacks et al., 2018) driven by individualistic values rather than national cultural norms. it seems that the it occupational culture has shifted the perceptions of job-related factors of the japanese it workers in favor of individualistic factors as embraced by their western it colleagues. therefore, we recommend future researchers to investigate how the it occupational culture may cause a shift in employees’ values from divergence to convergence resulting in a more homogenous it workforce. it is also possible that it may create a spillover effect when employees and their families start questioning other traditional japanese values, for instance, the notion of long-term employment. fifth, whereas a negative relationship between work-home conflict and job satisfaction was theoretically explicated earlier to justify h4, a positive one for younger organizations warrants further elaboration. it is possible that younger organizations employ a different type of it personnel, for example, junior, ambitious, energetic, and career-focused workers who, instead of reacting to the stress caused by irreconcilable demands between personal life and work duties, embrace a potential work-family conflict as “eustress.” eustress refers to “stress that creates a challenge or an opportunity” (tarafdar, cooper, & stich, 2019, p. 12). thus, these employees perceive such stress positively which enhances their mood, feelings, and perceptions of all aspects of the job and manifests itself in higher job satisfaction. in contrast, employees working for older organizations react negatively to stress caused by work-family conflict and exhibit a negative relationship between work-family conflict and job satisfaction. nevertheless, the observed positive relationship between work-home conflict and job satisfaction for younger organization is somewhat surprising, and it is suggested that future researchers examine this issue in greater depth. sixth and last, previous meta-analysis studies of the antecedents and consequences of job satisfaction and turnover intention have often reported mixed, inconclusive, and inconsistent findings (bowling, alarcon, bragg, & hartman, 2015; judge & bono, 2001; ng & feldman, australasian journal of information systems serenko et al. 2022, vol 26, research article turnover in japanese it professionals 22 2010). the present study contributes to theory by showing the importance of individualistic and collectivistic factors as well as the age of an organization. by considering these variables, future researchers may shed more light on the role of workplace perceptions and their impact on turnover intention in the it industry and beyond. 5.3 contribution to practice our study has at least four important practical implications for japanese managers. first, managers should recognize the fact that the shortage of advanced it professionals in japan is estimated to be at around 170,000 workers (blair, 2021). as a result, it behooves managers to do their best to understand the factors facilitating turnover decisions of it professionals. as this study demonstrates, managers should measure the level of their employees’ job satisfaction and find factors affecting it. second, it managers should boost their employees’ perceptions of personal accomplishment by means of formal training and mentorships. they should also further support the development of friendship networks by means of active socialization during and after work hours, which has traditionally been a part of the japanese work culture (tsutsui, 2010) and should be promoted further. in addition, they need to manage work exhaustion of their employees, particularly in younger organizations. thus, effective and efficient human resource management policies and procedures are required to ensure the retention of it workers. third, managers should keep in mind the importance of the age of their organization. older organizations in japan tend to embrace more collectivistic values which may affect the impact of workplace perceptions of job satisfaction. of particular importance to managers of older organizations should be work-family conflict because it reduces it employees’ job satisfaction leading to voluntary turnover. fourth, it managers should keep in mind that the traditional notion of long-term employment may not apply well in younger organizations. the it profession has embraced its own, unique occupational culture (cranefield et al., 2022) which converges with the occupational values of japanese it employees and makes them more aligned with their western counterparts, whose turnover decisions are less affected by their perceived loyalty to their employer. 5.4 limitations and future research directions despite its contributions, the study has limitations. first, factor analysis identified two indicators with low loadings which were removed. it is possible that this affected the content validity of the measurement instrument (nunnally & bernstein, 1994). second, the findings are japan-specific and may not generalize to other asian or western countries (henrich et al., 2010a). future researchers are recommended to replicate this analysis in other countries. nevertheless, we believe that this limitation is not serious enough to negate our findings, and we invite future researchers to extend the scope of inquiry to other contexts in order to further unravel the determinants of turnover intention of it professionals. third, this study relied on cross-sectional data which is quite common in is research; however, a longitudinal study, although difficult to conduct, may reveal more nuances in the findings (sedgwick, 2014) as well as result in re-specification of the research model. fourth, due to the nature of the data collection approach, the response rate and the non-response bias could not be measured, and it is unknown how this affected the findings, although our sample was representative of several demographic metrics. moreover, to ensure anonymity, the data cannot be traced back australasian journal of information systems serenko et al. 2022, vol 26, research article turnover in japanese it professionals 23 to organizations and thus organizational level controls were not included. thus, potential organizational differences which were not accounted for in this study should be taken into consideration in future research. fifth, this study’s methods focused on a single perspective – the “supply side of labor” – while approaching the issue from the “demand side of labor” point of view may help researchers uncover new perspectives. for instance, it is possible that the model’s relationships may be affected by job market conditions or the level of a country’s it infrastructure (palvia, ghosh, jacks, & serenko, 2021). sixth, it is feasible that the strength of the tested relationships depends on the personality types and other attributes of it employees (allen, weeks, & moffitt, 2005), and we recommend for future researchers to look into this possibility. seventh, we relied on self-reported, explicit measures which may be affected by social desirability bias. thus, we recommend future researchers to employ implicit constructs which are measured beyond people’s conscious awareness (e.g., see serenko, 2022; serenko & turel, 2019; serenko & turel, 2021). eighth and last, while this study’s model has a good predictive power, a major proportion of variance is still unaccounted for. thus, the model may be further extended with additional constructs and moderating variables. 6 conclusion much of previous is research on employee retention has been conducted in the context of western countries. on the one hand, these studies have improved our understanding of employee turnover intention and generated valuable practical recommendations. on the other hand, some of their conclusions may not generalize to the global it workforce due to differences in national culture, economic growth, norms, beliefs, and historical development of the it sector. the present investigation takes an important step and goes beyond the western context by accounting for the idiosyncrasies of the japanese it workplace environment. the japanese it workforce has evolved in its own unique way driven by the country’s various distinctive attributes including the traditional japanese value of long-term employment. this study has confirmed the dynamic nature of the it sector and shown that the it occupational culture may challenge the traditional japanese assumption of long-term employment and create a convergence force to align the nature of the japanese it workplace with that of its western counterparts. this movement may be especially strong in younger japanese it companies that employ junior, energetic, ambitious, and career-focused it professionals. references abegglen, j. c. (2006). 21st-century japanese management: new systems, lasting values. new york: palgrave macmillan. abhicharttibutra, k., & tungpunkom, p. (2019). association of structural and psychological empowerment with depersonalization and personal accomplishment among nurses: a systematic review. pacific rim international journal of nursing research, 23(4), 398-413. ahmad, r., & islam, t. (2018). relationships between corporate social responsibility, work engagement and organizational commitment: explanatory role of organizational identification. journal of behavioural sciences, 28(2), 112-130. ahmed, s., taskin, n., pauleen, d. j., & parker, j. (2017). motivating information technology professionals: the case of new zealand. australasian journal of information systems, 21. ahuja, m. k., chudoba, k. m., kacmar, c. j., mcknight, d. h., & george, j. f. (2007). it road warriors: balancing work-family conflict, job autonomy, and work overload to mitigate turnover intentions. mis quarterly, 31(1), 1-17. australasian journal of information systems serenko et al. 2022, vol 26, research article turnover in japanese it professionals 24 allen, d. g., weeks, k. p., & moffitt, k. r. (2005). turnover intentions and voluntary turnover: the moderating roles of self-monitoring, locus of control, proactive personality, and risk aversion. journal of applied psychology, 90(5), 980-990. anonymous. (2018a). heikin kinzoku nensuu (in japanese), the japan institute for labour policy and training. available online at https://www.jil.go.jp/kokunai/statistics/ timeseries/html/g0213_01.html anonymous. (2018b). databook of international labour statistics. the japan institute for labour policy and training. anonymous. (2018c). employee tenure in 2018. the us department of labor. bureau of labor and statistics. news release. available online at https://www.bls.gov/news.release/ pdf/tenure.pdf anonymous. (2018e). firms and systems engineers (in japanese), in nikkei sangyo shinbun, 2018/09/07. arbuckle, j. l. (2006). ibm® spss® amos™ 25 user’s guide. ibm corp. armstrong, d. j., brooks, n. g., & riemenschneider, c. k. (2015). exhaustion from information system career experience: implications for turn-away intention. mis quarterly, 39(3), 713-728. ashforth, b. e., kreiner, g. e., & fugate, m. (2000). all in a day’s work: boundaries and micro role transitions. academy of management review, 25(3), 472-491. ashill, n. j., & rod, m. (2011). burnout processes in non-clinical health service encounters. journal of business research, 64(10), 1116-1127. báez-mendoza, r., & schultz, w. (2013). the role of the striatum in social behavior. frontiers in neuroscience, 7(article 233), 1-14. bang, h., & reio jr, t. g. (2017). personal accomplishment, mentoring, and creative selfefficacy as predictors of creative work involvement: the moderating role of positive and negative affect. the journal of psychology, 151(2), 148-170. bhagwatwar, a., bala, h., & ramesh, v. (2014). it service management employee compensation: determinants and outcomes. e-service journal, 9(3), 1-18. berman, e. m., west, j. p., & richter jr, m. n. (2002). workplace relations: friendship patterns and consequences (according to managers). public administration review, 62(2), 217-230. blair, g. (2021). fixing japan’s talent mismatch. eurobiz japan. available online at https://eurobiz.jp/investing-in-japan/fixing-japans-talent-mismatch blinder, a. s., & krueger, a. b. (1996). labor turnover in the usa and japan: a tale of two countries. pacific economic review, 1(1), 27-57. bowling, n. a., alarcon, g. m., bragg, c. b., & hartman, m. j. (2015). a meta-analytic examination of the potential correlates and consequences of workload. work & stress, 29(2), 95-113. browne, m., & cudeck, r. (1993). alternative ways of assessing model fit. in k.a. bollen and j.s. long (eds.), testing structural equation model (pp. 136-162). london: sage. brewer, e. w., & clippard, l. f. (2002). burnout and job satisfaction among student support services personnel. human resource development quarterly, 13(2), 169-186. cho, y.-n., rutherford, b. n., friend, s. b., hamwi, g. a., & park, j. (2017). the role of emotions on frontline employee turnover intentions. journal of marketing theory & practice, 25(1), 57-68. choi, y. r., & phan, p. h. (2014). exploration, exploitation, and growth through new product development: the moderating effects of firm age and environmental adversity. ieee transactions on engineering management, 61(3), 428-437. australasian journal of information systems serenko et al. 2022, vol 26, research article turnover in japanese it professionals 25 choi, y. r., & shepherd, d. a. (2005). stakeholder perceptions of age and other dimensions of newness. journal of management, 31(4), 573-596. churchill jr, g. a., ford, n. m., & walker jr, o. c. (1974). measuring the job satisfaction of industrial salesmen. journal of marketing research, 11(3), 254-260. cranefield, j., gordon, m. e., palvia, p., serenko, a., & jacks, t. (2022). from fun-lovers to institutionalists: uncovering pluralism in it occupational culture. information technology & people, in-press. de moura, g. r., abrams, d., retter, c., gunnarsdottir, s., & ando, k. (2009). identification as an organizational anchor: how identification and job satisfaction combine to predict turnover intention. european journal of social psychology, 39(4), 540-557. dinger, m., thatcher, j. b., & stepina, l. p. (2010). a study of work-family conflict among it professionals: job characteristics, individual values, and management practices. journal of organizational computing and electronic commerce, 20(1), 91-121. dinger, m., thatcher, j. b., treadway, d., stepina, l., & breland, j. (2015). does professionalism matter in the it workforce? an empirical examination of it professionals. journal of the association for information systems, 16(4), 281-313. edwards, h., & dirette, d. (2010). the relationship between professional identity and burnout among occupational therapists. occupational therapy in health care, 24(2), 119-129. emslie, c., hunt, k., & macintyre, s. (2004). gender, work-home conflict, and morbidity amongst white-collar bank employees in the united kingdom. international journal of behavioral medicine, 11(3), 127-134. fehr, b. (1996). friendship processes. london: sage. fisher, v. e., & hanna, j. v. (1931). the dissatisfied worker. london: macmillan. fornell, c., & larcker, d. f. (1981). evaluating structural equation models with unobservable variables and measurement error. journal of marketing research, 18(1), 39-50. gamage, a. s. (2014). employee turnover in manufacturing smes in japan: an analysis of the link with hrm practices. sabaragamuwa university journal, 13(1), pp 17-31. galarraga, s., saies, e., cecchini, j., arruza, j., & luis-de-cos, i. (2017). the relationship between emotional intelligence, self-determined motivation and performance in canoeists. journal of human sport and exercise, 12(3), 630-639. gelfand, j. m., bhawuk, d., nishii, l., & bechtold, d. (2004). individualism and collectivism. in r. j. house (ed.), culture, leadership, and organization. london: sage, pp. 437-512. george, g. (2005). slack resources and the performance of privately held firms. academy of management journal, 48(4), 661-676. glisson, c., & durick, m. (1988). predictors of job satisfaction and organizational commitment in human service organizations. administrative science quarterly, 33(1), 61-81. gottlieb, b. h., & bergen, a. e. (2010). social support concepts and measures. journal of psychosomatic research, 69(5), 511-520. grandey, a. a., & cropanzano, r. (1999). the conservation of resources model applied to work-family conflict and strain. journal of vocational behavior, 54(2), 350-370. grzywacz, j. g., almeida, d. m., & mcdonald, d. a. (2002). work-family spillover and daily reports of work and family stress in the adult labor force. family relations, 51(1), 28-36. hair, j. f., black, w. c., anderson, r. e., & babin, b. j. (2018). multivariate data analysis (8th edition). emea: cengage learning. henrich, j., heine, s. j., & norenzayan, a. (2010a). most people are not weird. nature, 466(7302), 29. australasian journal of information systems serenko et al. 2022, vol 26, research article turnover in japanese it professionals 26 henrich, j., heine, s. j., & norenzayan, a. (2010b). the weirdest people in the world? behavioral and brain sciences, 33(2-3), 61-83. higuchi, y. (1993). labor turnover behavior: japan versus the west. japanese economic studies, 21(5), 61-88. hofstede, g. (1980). culture’s consequences: national differences in thinking and organizing. london: sage. hom, p. w., caranikas-walker, f., prussia, g. e., & griffeth, r. w. (1992). a meta-analytical structural equations analysis of a model of employee turnover. journal of applied psychology, 77(6), 890-909. honda-howard, m., & homma, m. (2001). job satisfaction of japanese career women and its influence on turnover intention. asian journal of social psychology, 4(1), 23-38. hoppock, r. (1935). job satisfaction. new york: harper. hu, l. t., & bentler, p. m. (1999). cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. structural equation modeling, 6(1), 1-55. huang, t. p. (2011). comparing motivating work characteristics, job satisfaction, and turnover intention of knowledge workers and blue-collar workers, and testing a structural model of the variables’ relationships in china and japan. the international journal of human resource management, 22(4), 924-944. huff, l., & kelley, l. (2003). levels of organizational trust in individualist versus collectivist societies: a seven-nation study. organization science, 14(1), 81-90. igbaria, m., & guimaraes, t. (1993). antecedents and consequences of job satisfaction among information center employees. journal of management information systems, 9(4), 145-174. igbaria, m., & siegel, s. r. (1992). the reasons for turnover of information systems personnel. information & management, 23(6), 321-330. iida, t., & morris, j. (2008). farewell to the salaryman? the changing roles and work of middle managers in japan. the international journal of human resource management, 19(6), 10721087. isard, w. (1954). location theory and trade theory: short-run analysis. the quarterly journal of economics, 68(2), 305-320. jacks, t., & palvia, p. (2014). measuring value dimensions of it occupational culture: an exploratory analysis. information technology and management, 15(1), 19-35. jacks, t., palvia, p., iyer, l., sarala, r., & daynes, s. (2018). an ideology of it occupational culture: the aspire values. the database for advances in information systems, 49(1), 93-117. jain, a. (2016). learning by hiring and change to organizational knowledge: countering obsolescence as organizations age. strategic management journal, 37(8), 1667-1687. joseph, d., ng, k.-y., koh, c., & ang, s. (2007). turnover of information technology professionals: a narrative review, meta-analytic structural equation modeling, and model development. mis quarterly, 31(3), 547-577. judge, t. a. (1993). does affective disposition moderate the relationship between job satisfaction and voluntary turnover? journal of applied psychology, 78(3), 395-402. judge, t. a., & bono, j. e. (2001). relationship of core self-evaluations traits – self-esteem, generalized self-efficacy, locus of control, and emotional stability – with job satisfaction and job performance: a meta-analysis. journal of applied psychology, 86(1), 80-92. kachi, y., inoue, a., eguchi, h., kawakami, n., shimazu, a., & tsutsumi, a. (2020). occupational stress and the risk of turnover: a large prospective cohort study of employees in japan. bmc public health, 20(1), 174. australasian journal of information systems serenko et al. 2022, vol 26, research article turnover in japanese it professionals 27 kalleberg, a. l., & leicht, k. t. (1991). gender and organizational performance: determinants of small business survival and success. academy of management journal, 34(1), 136-161. kambayashi, r., & kato, t. (2017). long-term employment and job security over the past 25 years: a comparative study of japan and the united states. ilr review, 70(2), 359-394. kawaguchi, d., & ueno, y. (2013). declining long-term employment in japan. journal of the japanese and international economies, 28(june), 19-36. koeske, g. f., kirk, s. a., koeske, r. d., & rauktis, m. b. (1994). measuring the monday blues: validation of a job satisfaction scale for the human services. social work research, 18(1), 27-35. kim, b. i. (2018). effect of psychological empowerment on turnover intention through job satisfaction and organizational commitment: focus on korea, china, japan employees. east asian journal of business economics, 6(2), 1-13. kim, s. (2012). the impact of human resource management on state government it employee turnover intentions. public personnel management, 41(2), 257-279. kim, s., & wright, b. e. (2007). it employee work exhaustion: toward an integrated model of antecedents and consequences. review of public personnel administration, 27(2), 147-170. kreiner, g. e. (2006). consequences of work-home segmentation or integration: a personenvironment fit perspective. journal of organizational behavior, 27(4), 485-507. kirmeyer, s. l., & dougherty, t. w. (1988). work load, tension, and coping: moderating effects of supervisor support. personnel psychology, 41(1), 125-139. kudo, y., satoh, t., hosoi, k., miki, t., watanabe, m., kido, s., & aizawa, y. (2006). association between intention to stay on the job and job satisfaction among japanese nurses in small and medium-sized private hospitals. journal of occupational health, 48(6), 504-513. kuo, y.-f., & chen, l.-s. (2004). individual demographic differences and job satisfaction among information technology personnel: an empirical study in taiwan. international journal of management, 21(2), 221-231. lee, c. b. p. (2011). how perceived workload affects the work outcomes of computer professionals. australasian journal of information systems, 17(1), 35-51. lee, p. c. b. (2003). going beyond career plateau. journal of management development, 22(6), 538-551. li, j., fang, m., wang, w., sun, g., & cheng, z. (2018). the influence of grit on life satisfaction: self-esteem as a mediator. psychologica belgica, 58(1), 51-66. liu, l. (2010). comparative study on the factors influencing women turnover in it industry between china and japan. journal of china women’s university, 8(4), 62-66. maamari, b. (2014). changing management history, gender moderating pay to job satisfaction for is users. journal of management history, 20(3), 311-332. maier, c., laumer, s., eckhardt, a., & weitzel, t. (2015). who really quits? a longitudinal analysis of voluntary turnover among it personnel. the data base for advances in information systems, 46(4), 26-47. maslach, c., jackson, s. e., & leiter, m. (1997). the maslach burnout inventory manual. in c. p. zalaquettand & r. j. wood (eds.), evaluating stress: a book of resources (pp. 191-218). london: scarecrow press. maslow, a. h. (1970). motivation and personality. new york: harper & row. matsumoto, m., & gopal, b. (2019). solidarity, job satisfaction, and turnover intent in employees. international journal of workplace health management, 12(4), 247-257. australasian journal of information systems serenko et al. 2022, vol 26, research article turnover in japanese it professionals 28 mcknight, d. h., phillips, b., & hardgrave, b. c. (2009). which reduces it turnover intention the most: workplace characteristics or job characteristics? information & management, 46(3), 167-174. mobley, w. h., horner, s. o., & hollingsworth, a. t. (1978). an evaluation of precursors of hospital employee turnover. journal of applied psychology, 63(4), 408-414. mobley, w. h. (1977). intermediate linkages in the relationship between job satisfaction and employee turnover. journal of applied psychology, 62(2), 237-240. moore, j. e. (2000). one road to turnover: an examination of work exhaustion in technology professionals. mis quarterly, 24(1), 141-168. morris, j. h., & sherman, j. d. (1981). generalizability of an organizational commitment model. academy of management journal, 24(3), 512-526. nakane, c. (1970). japanese society. los angeles: university of california press. ng, t. w. h., & feldman, d. c. (2010). the relationships of age with job attitudes: a metaanalysis. personnel psychology, 63(3), 677-718. ngo, h. y., foley, s., & loi, r. (2005). work role stressors and turnover intentions: a study of professional clergy in hong kong. international journal of human resource management, 16(11), 2133-2146. nunnally, j. c., & bernstein, i. h. (1994). psychometric theory (3 ed.). new york: mcgraw hill. oehlhorn, c. e., maier, c., & weitzel, t. (2020). turnover and turnaway of it workers: a person-environment fit perspective. proceedings of the acm sigmis computers and people research conference. nuremberg, germany, pp. 103-104. ono, h. (2010). lifetime employment in japan: concepts and measurements. journal of the japanese and international economies, 24(1), 1-27. palvia, p. (2013). the world it project: a program on international research and call for participation. journal of global information technology management, 16(2), 1-5. palvia, p., jacks, t., gosh, j., licker, p., romm-livermore, c., serenko, a., & turan, a. h. (2017). the world it project: history, trials, tribulations, lessons, and recommendations. communications of the association for information systems 41(article 18), 389-413. palvia, p., ghosh, j., jacks, t., serenko, a., & turan, a. (2018). trekking the globe with the world it project. journal of information technology case and application research, 20(1), 38. palvia, p., ghosh, j., jacks, t., serenko, a., & turan, a.h. (eds.) (2020). the world it project: global issues in information technology. singapore: world scientific now publishers. palvia, p., ghosh, j., jacks, & serenko, a. (2021). information technology issues and challenges of the globe: the world it project. information & management, 58(8), article 103545. paramore, k. (2016). japanese confucianism: a cultural history. cambridge: cambridge university press. popescu, a. m., deaconu, a., & popescu, t. (2015). organization’s age and organizational citizenship behavior (ocb), performance criteria at smes level. case study–bucharest– ilfov development region. procedia economics and finance, 22, 645-654. raile, a. n. w., kim, r. k., choi, j., serota, k. b., park, h. s., & lee, d. w. (2008). connections at work: how friendship networks relate to job satisfaction. communication research reports, 25(2), 168-178. rath, t. (2006). vital friends: the people you can’t afford to live without. new york: gallup press. rouse, p. d. (2001). voluntary turnover related to information technology professionals: a review of rational and instinctual models. the international journal of organizational analysis, 9(3), 281-290. australasian journal of information systems serenko et al. 2022, vol 26, research article turnover in japanese it professionals 29 rutherford, b. n., wei, y., park, j., & hur, w.-m. (2012). increasing job performance and reducing turnover: an examination of female chinese salespeople. journal of marketing theory & practice, 20(4), 423-436. rutner, p. s., hardgrave, b. c., & mcknight, d. h. (2008). emotional dissonance and the information technology professional. mis quarterly, 32(3), 635-652. sasaki, h. (2012). differences in efficiency between b2bs and b2cs in the japanese it services industry. proceedings of the 14th ieee international conference on commerce and enterprise technology. sasaki, h., sato, o., & palvia, p. (2020). information technology issues in japan. in p. palvia, j. ghosh, t. jacks, a. serenko, & a. h. turan (eds.), the world it project: global issues in information technology (pp. 209-221). singapore: world scientific now publishers. sasaki, h., serenko, a., sato, o., & palvia, p. (2019). determinants of job satisfaction and turnover intention of it professionals in japan. proceedings of the 25th americas conference on information systems (amcis). cancun, mexico. scholtz, b., van belle, j. p., njenga, k., serenko, a., & palvia, p. (2019). the role of job satisfaction in turnover and turn-away intention of it staff in south africa. interdisciplinary journal of information, knowledge, and management, 14, 77-97. sedgwick, p. (2014). cross sectional studies: advantages and disadvantages. british medical journal, 348(g2276), 1-2. serenko, a. (2022). antecedents and consequences of explicit and implicit attitudes toward digital piracy. information & management, 59(1), article 103559. serenko, a., & bontis, n. (2016). negotiate, reciprocate, or cooperate? the impact of exchange modes on inter-employee knowledge sharing. journal of knowledge management, 20(4), 687-712. serenko, a., & turel, o. (2019). a dual-attitude model of system use: the effect of explicit and implicit attitudes. information & management, 56(5), 657-668. serenko, a., & turel, o. (2021). why are women underrepresented in the american it industry? the role of explicit and implicit gender identities. journal of the association for information systems, 22(1), 41-66. shih, s. p., jiang, j. j., klein, g., & wang, e. (2013). job burnout of the information technology worker: work exhaustion, depersonalization, and personal accomplishment. information & management, 50(7), 582-589. shimizu, t., feng, q., & nagata, s. (2005a). relationship between turnover and burnout among japanese hospital nurses. journal of occupational health, 47(4), 334-336. shimizu, t., eto, r., horiguchi, i., obata, y., feng, q., & nagata, s. (2005b). relationship between turnover and periodic health check-up data among japanese hospital nurses: a three-year follow-up study. journal of occupational health, 47(4), 327-333. shimizutani, s., & yokoyama, i. (2009). has japan’s long-term employment practice survived? developments since the 1990s. ilr review, 62(3), 313-326. sias, p. m., & cahill, d. j. (1998). from coworkers to friends: the development of peer friendships in the workplace. western journal of communication, 62, 273-299. soonhee, k., & wright, b. e. (2007). it employee work exhaustion. review of public personnel administration, 27(2), 147-170. spector, p. e., allen, t. d., poelmans, s. a., lapierre, l. m., cooper, c. l., michael, o. d., sanchez, j. i., abarca, n., alexandrova, m., beham, b., & brough, p. (2007). crossnational differences in relationships of work demands, job satisfaction, and turnover intentions with work-family conflict. personnel psychology, 60(4), 805-835. australasian journal of information systems serenko et al. 2022, vol 26, research article turnover in japanese it professionals 30 spector, p. e. (1997). job satisfaction: application, assessment, causes, and consequences. thousand oaks, ca: sage. stohl, c. (2001). globalizing organizational communication. in f. m. jablin & l. l. putnam (eds.), the new handbook of organizational communication (pp. 323-376). london: sage. suzuki, e., tagaya, a., ota, k., nagasawa, y., matsuura, r., & sato, c. (2010). factors affecting turnover of japanese novice nurses in university hospitals in early and later periods of employment. journal of nursing management, 18(2), 194-204. suzuki, h. (2010). employment relations in japan: recent changes under global competition and recession. journal of industrial relations, 52(3), 387-401. takahashi, k. (2018). the future of the japanese style employment system: continued longterm employment and the challenges it faces. japan labor issues, 2(6), 6-15. takahashi, n. (2014). future parameter explains job satisfaction and turnover candidates in japanese companies. annals of business administrative science, 13(3), 129-140. takase, m., teraoka, s., & yabase, k. (2016). retaining the nursing workforce: factors contributing to the reduction of nurses’ turnover intention in japan. journal of nursing management, 24(1), 21-29. takase, m., oba, k., & yamashita, n. (2009). generational differences in factors influencing job turnover among japanese nurses: an exploratory comparative design. international journal of nursing studies, 46(7), 957-967. tamakoshi, a., ohno, y., yamada, t., aoki, k., hamajima, n., wada, m., kawamura, t., wakai, k., & lin, y. s. (2000). depressive mood and suicide among middle-aged workers: findings from a prospective cohort study in nagoya, japan. journal of epidemiology, 10(3), 173-178. tarafdar, m., cooper, c. l., & stich, j. f. (2019). the technostress trifecta techno eustress, techno distress and design: theoretical directions and an agenda for research. information systems journal, 29(1), 6-42. thatcher, j. b., stepina, l. p., & boyle, r. j. (2002). turnover of information technology workers: examining empirically the influence of attitudes, job characteristics, and external markets. journal of management information systems, 19(3), 231-261. tei-tominaga, m., & miki, a. (2010). a longitudinal study of factors associated with intentions to leave among newly graduated nurses in eight advanced treatment hospitals in japan. industrial health, 48(3), 305-316. ten brummelhuis, l. l., & bakker, a. b. (2012). a resource perspective on the work-home interface: the work-home resources model. american psychologist, 67(7), 545-556. tett, r. p., & meyer, j. p. (1993). job satisfaction, organizational commitment, turnover intention, and turnover: path analyses based on meta-analytic findings. personnel psychology, 46(2), 259-293. titus jr., v. k., covin, j. g., & slevin, d. p. (2011). aligning strategic processes in pursuit of firm growth. journal of business research, 64(5), 446-453. tripathi, a., & srivastava, r. (2020). a literature review on turnover and retention of it employees. international journal of advanced science and technology, 29(3), 12675-12683. tsutsui, k. (2010). sake and organizational communication: reconsidering “nominication” (in japanese). kyushu communication studies, 8, 77-88. turel, o., serenko, a., & bontis, n. (2011). family and work-related consequences of addiction to organizational pervasive technologies. information & management, 48(2-3), 88-95. van bergeijk, p. a., & brakman, s. (2010). the gravity model in international trade: advances and applications. cambridge: cambridge university press. australasian journal of information systems serenko et al. 2022, vol 26, research article turnover in japanese it professionals 31 voss, g. b., & voss, z. g. (2013). strategic ambidexterity in small and medium-sized enterprises: implementing exploration and exploitation in product and market domains. organization science, 24(5), 1459-1477. wang, j., tolson, h., chiang, t.-l., & huang, t.-y. (2010). an exploratory factor analysis of workplace learning, job satisfaction, and organizational commitment in small to midsize enterprises in taiwan. human resource development international, 13(2), 147-163. weiss, h. m. (2002). deconstructing job satisfaction: separating evaluations, beliefs and affective experiences. human resource management review, 12(2), 173-194. weiss, h. m., & cropanzano, r. (1996). affective events theory: a theoretical discussion of the structure, causes and consequences of affective experiences at work. in b. m. staw & l. l. cummings (eds.), research in organizational behavior: an annual series of analytical essays and critical reviews (pp. 1-74). greenwich: jai press. weiss, h. m., nicholas, j. p., & daus, c. s. (1999). an examination of the joint effects of affective experiences and job beliefs on job satisfaction and variations in affective experiences over time. organizational behavior and human decision processes, 78(1), 1-24. weli, w. (2014). manager satisfaction in using the enterprise resource planning (erp) system and managerial performance. australasian journal of information systems, 18(3), 119-135. wiesche, m., joseph, d., thatcher, j., gu, b., & krcmar, h. (2019). research curation: it workforce. mis quarterly curations. available online at https://www.misqresearchcurations.org/blog/2019/6/17/it-workforce wickramasinghe, v. (2009). predictors of job satisfaction among it graduates in offshore outsourced it firms. personnel review, 38(4), 413-431. yanadori, y., & kato, t. (2009). work and family practices in japanese firms: their scope, nature and impact on employee turnover. international journal of human resource management, 20(2), 439-456. yeo, b., serenko, a., palvia, p., sato, o., sasaki, h., yu, j., & guo, y. (2021). job satisfaction of it workers in east asia: the role of employee demographics, job demographics, and uncertainty avoidance. the data base for advances in information systems, 52(2), 94-126. yousef, d. a. (2000). organizational commitment: a mediator of the relationships of leadership behavior with job satisfaction and performance in a non-western country. journal of managerial psychology, 15(1), 6-24. copyright: © 2022 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v26i0.3037 http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 literature review and theoretical underpinnings 2.1 background 2.2 prior related work 2.3 theoretical underpinnings 2.3.1 job satisfaction 2.3.2 work exhaustion 2.3.3 personal accomplishment 2.3.4 work-home conflict 2.3.5 friendship networks 2.3.6 individualistic factors vs. collectivistic factors 2.4 hypotheses development 3 methods 4 results 5 discussion 5.1 overview of the findings 5.2 contribution to theory 5.3 contribution to practice 5.4 limitations and future research directions 6 conclusion paper title (paper title style) australasian journal of information systems schoormann & di maria 2024, vol 28, research article socially business process patterns received 27 nov 2023, accepted 25 oct 2024 after 3 rounds of review with karlheinz kautz as accepting editor 1 business process patterns for improving social sustainability thorsten schoormann data-driven enterprise, tu braunschweig, braunschweig germany, and fraunhofer isst, dortmund, germany email: thorsten.schoormann@tu-braunschweig.de marco di maria information systems and enterprise modelling university of hildesheim hildesheim, germany abstract business process management (bpm) has the ability to boost transformations towards sustainable entities by innovating organizational structures. while the majority of existing bpm tools and methods focus on economic obligations, social sustainability is often underrepresented. this is problematic because it inhibits business improvement of people’s quality of life (e.g., health and equity), fails to address changing customer demands beyond transactional excellence, and obstructs the consideration of new regulations. based on a literature review and expert interviews, recurrent problems and best practice solutions for integrating social sustainability into business processes were collected. these were formalized into a set of process patterns and then evaluated through illustrative demonstrations, an applicability check, and interviews with process experts. the paper proposes ten patterns together with a series of examples to guide the analysis and improvement of processes in terms of social sustainability. they support both generating novel ideas and identifying weaknesses. in doing this, our work complements existing tools and methods from sustainable bpm, advances the current body of knowledge in this stream, and opens avenues for a more holistic consideration of sustainability in business processes. keywords: business process redesign, business process improvement, responsibility, process pattern, social sustainability. 1 introduction the necessity of transforming people and businesses towards more sustainability is undisputed (un 2015). besides environmental or economic goals, information systems (is) scholars have begun to pay particular attention to social sustainability which seeks “to promote compatibility across cultural and social differences, enhance people’s quality of life, and manage business impacts on people” (kotlarsky et al. 2023, p. 938). hence, we frame social sustainability in is research as a concept that leverages technology to, among others, contribute to healthy, equitable, and inclusive environments (schoormann and kutzner 2020; troshani et al. 2022). due to its ability to transform organizational structures, business process management (bpm) is one of the key drivers in fostering sustainability (seidel et al. 2011). advanced bpm tools have been proposed to optimize business process efficiency, productivity, and conformance (rosemann et al. 2023; senderovich et al. 2020). while traditional bpm is driven by the economic paradigm, academia (e.g., kotlarsky et al. 2023; tan and nielsen 2024), practice (e.g., australasian journal of information systems schoormann & di maria 2024, vol 28, research article socially business process patterns 2 changing customer demands towards responsibility, rosemann et al. 2023), and law (e.g., esg framework for environmental, social, and governance) emphasize the need to generate environmental and social value as well. however, particularly the notion of social sustainability is not sufficiently addressed in bpm, which is problematic for several reasons: first, today’s managers need to reflect on social sustainability to responsibly balance social objectives with process effectiveness, for instance, when it comes to automating tasks or handling the diversity of employees. factoring in social considerations attracts talent, fosters trust, and raises employee engagement (rosemann et al. 2023; senderovich et al. 2020). second, it is crucial nowadays to adopt a broader perspective on the value and the stakeholder groups impacted by these processes. this allows organizations to respond to, for example, changing customer demands beyond purely transactional excellence (rosemann et al. 2023) as well as increase customer loyalty, reputation, and attractiveness to new partners (senderovich et al. 2020; un 2021). third, missing guidance on implementing social sustainability impedes being compliant with new regulations and reporting guidelines, including esg criteria (ketter et al. 2020), the eu taxonomy (klessascheck et al. 2024), or the australian sustainability reporting standards (aasb 2023). while promoting social value in processes is demanded, improving processes is among the most creative tasks in bpm (zellner 2011). as part of the bpm research streams on sustainability (e.g., extending notations, couckuyt and van looy 2019; gohar and indulska 2020), there is a growing interest in patterns to guide process improvement and analysis. those patterns provide proven and generally applicable best practice solutions for recurrent problems (fellmann et al. 2018) and pose a generalizable way to integrate sustainability into business processes. following an emerging emphasis on the social dimension in is research (e.g., corbett et al. 2023; kotlarsky et al. 2023; sarker et al. 2019), we aim to advance bpm by building business process patterns that contribute to social sustainability. in pursuing this goal, we follow the design science research (dsr) paradigm (section 3). we conducted a literature review to identify existing bpm solutions and their potential to inform the design of our artifact (section 4). based on the status quo and the analysis of empirical data obtained through expert interviews, we iteratively designed patterns (section 5). a total set of ten patterns were crafted capturing best practice solutions that help to analyze and (re-)design processes in terms of social sustainability. the patterns were verified by illustrating them with real-world cases as well as evaluating their applicability with 18 participants from a bpm course and investigating their practical usefulness via process experts (section 6). finally, we discuss the implications, limitations, and future efforts (section 7), and conclude the paper (section 8). in doing this, we complement existing patterns and tools from green/sustainable bpm, advance the current body of knowledge on bpm and sustainability, and open avenues for a more holistic consideration of sustainability issues in academia, practice, and law. 2 research background 2.1 social sustainability in information systems research sustainable development aims to ensure fair and livable conditions for the next generations (brundtland et al. 1987) and is manifested in a growing amount of policies, such as corporate social responsibility (csr) and the esg framework (corbett et al. 2023; ketter et al. 2020). is research has predominantly focused on environmental impacts, leaving room for future australasian journal of information systems schoormann & di maria 2024, vol 28, research article socially business process patterns 3 studies aimed at the social dimension (kotlarsky et al. 2023; veit and thatcher 2023). in attempting to respond to that, we see emerging is streams that broaden the focus: special interest groups on social sustainability (kranz et al. 2022), digital sustainability (kotlarsky et al. 2023; christmann et al. 2024; pan et al. 2022), digital social innovation (tim et al. 2021), social inclusion (wass et al. 2023), and digital responsibility (lobschat et al. 2021). however, the plurality of approaches requires adaption to the bpm field, and while the strong digital component is beneficial, it might not be essential for improving organizational structures. we therefore seek to build upon the broader concept of social sustainability. the concept of social sustainability is generally perceived as more vague (mckenzie 2004; missimer et al. 2017; vallance et al. 2011). in this paper, we follow kotlarsky et al.’s (2023) general definition that social sustainability aims to “enhance people’s quality of life, and manage business impacts on people” (p. 938). this is in accordance with other social criteria and goals, including the establishment of healthy, equitable, and inclusive communities (e.g., mckenzie 2004; schoormann and kutzner 2020; troshani et al. 2022; un 2021). nonetheless, even though the is discipline has set out on the path to promote social concerns, additional research is needed to advance our understanding of how to incorporate them into organizations (tan and nielsen 2024). to this end, we draw on the well-known stream of bpm research acting as a driver for change toward sustainability and aim to leverage the design of social business processes (rosemann et al. 2023; senderovich et al. 2020). 2.2 sustainable business process management the bpm discipline is concerned with designing, implementing, and monitoring how work is performed (dumas et al. 2018; recker and mendling 2016). as business processes need to respond to sustainability challenges, green bpm arose as a bpm research stream to respond to sustainability challenges (couckuyt and van looy 2021; ghose et al. 2010; sohns et al. 2023; vom brocke et al. 2012). green bpm is “the sum of all is-supported management activities that help to monitor and reduce the environmental impact of business processes” (opitz et al. 2014, p. 3812). to guide process designers, research proposed approaches for monitoring environmental indicators (cleven et al. 2012) or extending modeling notations for emissions (recker et al. 2012). overviews of bpm extensions can be found in (gohar and indulska 2016; 2020; schoormann et al. 2017). in line with the discussion above, some scholars explicitly refer to sustainable bpm to improve economic, environmental, and social performance (ahmed and sundaram 2012; rozman et al. 2015). the key idea of social sustainability in this context refers mostly to a broader consideration of stakeholder groups from employees to society and a broader understanding of value from process executions. an emerging stream is concerned with socially aware process redesign techniques to explore how to allocate tasks (senderovich et al. 2020), process guidelines for social good to go beyond economic metrics of success (rosemann et al. 2023), or social topics in bpm in general (breitenbach 2020). 2.3 patterns for sustainable business processes improving processes is a complex and creative endeavor that requires guidance (zellner 2011). process patterns offer such guidance. although there is some ambiguity in the definition (fellmann et al. 2017) and there are closely related terms, such as design guideline (rosemann et al. 2023), heuristic (frank et al. 2020), or practice (reijers and mansar 2005) (see overview in appendix d), we adhere to the generic notion of pattern. patterns provide proven solutions to australasian journal of information systems schoormann & di maria 2024, vol 28, research article socially business process patterns 4 common problems and capture (re-)design guidance (smith and williams 2000). research on patterns was originally introduced in the domain of architecture by alexander (1977, p. 247) who argued that a pattern “expresses a relation between a certain context, a problem, and a solution”. they codify knowledge about best practices and present solutions that can be contextualized within an individual situation. drawing on these ideas, gamma (1995) proposed software design patterns, which pose a starting point for exploring patterns in further disciplines such as bpm. according to fellmann et al. (2018, p. 976), a process model pattern is a “description of a proven solution to a recurring problem that is related to the creation or modification of business process models in a specific context”. patterns play a pivotal role in bpm (winter et al. 2009) to effectively and efficiently analyze and (re-)design processes (e.g., medicke and mcdavid 2004; russell et al. 2016). while we can observe a great interest in process patterns and a growing number of patterns for economic improvements, there is room for patterns addressing social sustainability in particular. based on their extensive review, fellmann et al. (2018) argued that they “expect that patterns in categories being less considered […] will evolve in the future” (p. 992). this is also emphasized by rosemann (2020) who argued that in contrast to common response patterns in exploitative bpm, “explorative bpm with its focus on adding new value to business processes, lacks an equally mature, deductive set of design patterns”(p. 349). following this, our overall goal is to advance socially sustainable business process designs. 3 research method dsr is applied as the overarching method to build our pattern set. it is strongly practiceoriented and seeks to generate solutions to real-world problems (gregor and hevner 2013; goldkuhl and sjöström 2018). in line with its socio-technical nature (sarker et al. 2019), it is appropriate for our endeavor which covers people, technology, and tasks in the form of business processes. dsr has also been applied to build artifacts capable of guiding process redesign and improvement (e.g., rosemann et al. 2023; zellner et al. 2011). we choose patterns as artifacts because they represent abstract solutions and encode design-relevant knowledge that guides process designers (gregor and hevner 2013). we followed peffers et al.’s (2007) dsr methodology to rigorously ground, build, and evaluate our artifact (see figure 1). figure 1. research process problem identification solution objectives design and development demonstration evaluation reviewing existing patterns for sustainability (literature review) identifying problems from practice (expert interviews) adapting existing pattern developing new pattern illustrating the patterns (use cases, application) + revision of pattern evaluating the pattern’s utility (expert interviews) identifying solutions from practice (expert interviews) exploring adaptability of existing patterns australasian journal of information systems schoormann & di maria 2024, vol 28, research article socially business process patterns 5 3.1 problem identification and solution objectives first, we conducted a systematic literature review to (a) identify the potential for adapting existing patterns to our context as well as (b) spot for gaps (schryen 2013). based on the evaluation of several keywords and combinations, we iteratively specified our search phrase as follows: (“green business process*” or “sustainable business process*” or “social business process*”) and (pattern or practice or heuristic or “design guideline”). among the search sources were leading is conferences and journals covered by aisel and the ais senior scholars’ list of premier journals, the business process management journal (bpmj), and the international conference on bpm to consider process-related research, as well as google scholar and scopus to respect interdisciplinary research. drawing on the collected paper sample, a keyword search, and evaluation of titles and abstracts were conducted (webster and watson 2002). we eliminated papers that did not meet our research purpose or were duplicates and verified the remaining papers by examining the full text (n=152). therefore, we coded if concrete process patterns including their common components (i.e., specific problems, specified solution approaches, and/or examples), practices, heuristics, or design guidelines are covered. besides, we extracted statements on the actual need to address social sustainability in processes. in doing so, we found 48 relevant papers (see figure 2); details are provided in appendix a. the search was initially performed in 2019 and updated in 08/2024; scopus and the bpm conference were added during the paper revision in 02/2024 and updated in 08/2024. figure 2. literature review process second, we carried out semi-structured expert interviews to collect practical real-world challenges and insights into which efforts businesses perform to achieve social sustainability. this allowed us to gather data face-to-face in a guided way but also to leave room for additional comments. the experts needed to meet two criteria: the company covers aspects of social sustainability (e.g., on their website or in reports) and comprises at least one expert who is employed for sustainability-relevant issues (e.g., in a csr department). following this, several potential interview partners were contacted and four experts located in a european country with varying industries and company sizes1 participated (see table 1). while this may be relatively limited, our expert selection offers an expansive space for gathering diverse data from various contexts and experiences. the interview guide was structured along with three 1 european union: small (11-50 employees), medium (51-500 employees), large (>501 employees). s cr ee n in g a n d s el ec ti o n s ea rc h p h ra se (“green business process*” or “sustainable business process*” or “social business process*”) and (pattern or practice or heuristic or “design guideline”) aisel and ais premier journals bpm journal1 international conference on bpm2* google scholar scopus* sum total 107 31 113 1.530 584 2.365 title, keyword, abstract 14 13 17 68 40 152 evaluate full texts, eliminate papers that do not match the search goal, remove duplicates 48 relevant papers 1 emerald 2springer link australasian journal of information systems schoormann & di maria 2024, vol 28, research article socially business process patterns 6 main parts and pre-tested with master-leveled students: (i) the interviewee has been introduced to the topic and general demographic data of the expert was collected; (ii) questions related to sustainability were asked, for instance: do you implement social sustainability? how important are social aspects compared to economic and environmental aspects? what are the prerequisites for implementing social sustainability? (iii) questions concerning strategies that are successfully implemented to solve social challenges, for instance: what has led to the implementation of social aspects in your company? what concrete problems did you aim to solve in your organization? which solutions did you apply to contribute to social sustainability? how is this reflected in your (daily) processes? id role industry company size duration e1 sustainability manager retail large (> 200.000 employees) 23 min. e2 sustainability manager logistics large (~ 2.000 employees) 09 min. e3 sustainability consultant energy large (~ 2.500 employees) 22 min. e4 sustainability manager it, hardware small-medium (~ 400 employees) 38 min. table 1. expert interviews for grounding the patterns from the interview recording, 92 minutes of transcript data were created. for transcription, we aimed to focus on the actual content, and thus, do not transcribe pauses, non-verbal expressions, and linguistic emphases. following qualitative content analysis, we described the material in our own words. afterward, headlines were assigned to the material and consolidated (meuser and nagel 1991). drawing on this, common challenges in implementing social sustainability (see section 5.1) and successful approaches that are already applied could be investigated (see section 5.2). 3.2 design and development grounded in the insights from the literature review and the expert interviews, we built a novel artifact (petter et al. 2010), namely process patterns for social sustainability. following nowak’s (2014) pattern development approach2, two approaches of ‘adaption’ and ‘development’ can be differentiated. the first approach makes use of existing patterns and translates them into another context; the latter designs completely new patterns. we began with analyzing whether and how patterns from prior literature can be adapted to address the problem statements about social sustainability. two researchers independently determined the key ideas that help to enhance social issues (i.e., generalizing from a specific instance to a more abstract understanding) and compared these ideas with existing patterns. in case we did not find any potential for adoption, we designed patterns from scratch based on practical insights. our pattern presentation captures a title, a short description, and example initiatives, as well as specifies the origin of each pattern and illustrates possible changes in a process structure based on a simplified notation (see general notations in falk et al. 2013). we structured the patterns along four categories to take into account that some patterns are more likely to apply to certain organization types (e.g., value chain aspects are not important to all organizations): basic, resource, value chain and ecosystem, and human-centric (see section 5.3). 2 nowak (2014, p. 152) steps: identification of specific characteristics (domain knowledge and existing patterns), identification of problems and solutions, analysis and selection of patterns, abstraction of problem and solution, documentation of patterns. australasian journal of information systems schoormann & di maria 2024, vol 28, research article socially business process patterns 7 3.3 demonstration and evaluation although evaluation is an important part of creating new artifacts, fellmann et al. (2018) pointed to limited pattern evaluation in existing literature. to address this, we carried out three evaluation episodes: first, we applied the patterns to existing use cases extracted from sustainability reports. we demonstrated that the patterns can be used in practice as well as disclosed practical solutions to refine and extend our solution (see section 6.1). second, we drew from research on applicability checks to gain data on the pattern’s importance (i.e., does it address real-world issues), accessibility (i.e., is it understandable and readable), and suitability (i.e., does it provide guidance for usage) (rosemann and vessey 2008). therefore, as part of a voluntary exercise (anonymously and not related to course grades) within a bachelorlevel course in the summer of 2023, we confronted the participants with four steps: (1) the introduction of the patterns and task instructions which asked for improving social issues in a fictive process scenario (10 minutes), (2) the generation of ideas to improve the scenario (divergent, 15 minutes), (3) modeling their most promising ideas with bpmn (convergent, 25 minutes), and (4) fill out a questionnaire (10 minutes). the questions were adapted from iivari et al. (2020) who transferred them to the field of evaluating design knowledge. following this, we translated ‘suitability’ to ‘actability’ to focus on the pattern’s ability to guide redesign. the adopted items were assessed by using a 5-point likert scale where 5 indicated strong satisfaction and 1 indicated strong dissatisfaction, which has already been applied in the field of business process redesign (e.g., frank et al., 2020). 18 students with basic experiences in bpm participated of which 15 students filled out the questionnaire (see section 6.2). third, we investigated the practical usefulness of the pattern set through seven additional expert interviews (see table 2) with a total of 252 minutes. each interview was structured along (1) demographic data, (2) an introduction to the patterns, (3) questions about current problems in the expert’s organizations/experiences, (4) a discussion about our pattern, and (5) the usefulness. they were performed virtually, lasted between 29 and 43 minutes, as well as had a moderator (navigated through the interview) and observer (took notes, added missing questions). afterward, two authors transcribed and analyzed the interviews (see section 6.3). id role industry work experience duration i1 strategy and corporate development banking 6-7 years 39 min. i2 process manager consulting 3 years 34 min. i3 business development executive public administration 25 years 37 min. i4 organizational development (process manager) consulting 2 years 32 min. i5 climate change agent mobility 3 years 38 min. i6 process manager consulting 6 years 29 min. i7 process manager consulting 5 years 43 min. 252 min. table 2. expert interviews for the evaluation (evaluation episode 3) australasian journal of information systems schoormann & di maria 2024, vol 28, research article socially business process patterns 8 4 existing research on process patterns for sustainability following the literature review procedure (see problem identification), we obtained 48 papers that – to a varying degree – met our purpose and determined different categories of prior research along with environmental and social-oriented aspects. overall, despite the progress concerning the integration of sustainability in process patterns, there is still a deficit of research that considers societal concerns. following our research method, we could only extract our preliminary work that provides important inputs for new patterns (schoormann et al. 2019; schoormann and kutzner 2021). nonetheless, some papers identified in the literature review deal with the idea and/or emphasize the potential of process patterns: couckuyt and van looy (2019) stressed the role of patterns as a capability area for modeling and confirmed the primary environmental viewpoint of existing patterns. stadtländer et al. (2019) explored how software tools support sustainable bpm and only found limited features for using and adding patterns. wesumperuma et al. (2011) developed a framework for process optimization that draws on common ghg emissions frequency patterns. schreiber (2020) argued that patterns can be employed to check whether processes comply with sustainability criteria. in addition, papers in our sample point to the importance of more abstract guidance to improve business processes. this includes, for example, the adoption of approaches from green is (e.g., hassan et al. 2019; hu et al. 2016; vom brocke et al. 2013), environmental businesses (watson et al., 2010), sustainable business processes (e.g., rozman et al. 2015; seidel and recker 2012), data collection for sustainability (ahlers et al. 2017), sustainable aware bpm (betz and caporale 2014), and green it (seidel and recker 2012). category description (examples) implication(s) statements about social sustainability in business processes • social value requirements for processes, for instance: skills training, employing disabled people, sponsoring social events, social gatherings, and awareness programs (dewan et al. 2012) • achievement of social goals (schweizer et al. 2017) • social impact of processes (anjaria 2024) • social responsibility (watson et al. 2010) • sustainable business processes, encompassing economic, social, and environmental dimensions (e.g., magdaleno et al. 2017; graves et al. 2023) • social sustainability within operative but also strategic processes (schoormann and kutzner 2020) relevance for socially responsible processes patterns for social sustainability • schoormann et al. (2017) called for more work on integrating social sustainability via process patterns • schoormann et al. (2019) – our prior papers – provided an initial set of social business process patterns, which served as a foundation for this advanced paper relevance of creating social process patterns and initial findings benevolent business process design guidelines rosemann et al. (2023) derived design guidelines (including patterns) to reflect the overall notion of ‘doing good’ through the concept of benevolence. they provide four basic principles, each with two guidelines: • being fair: adequacy and awareness • do right: prevention and compensation • say yes: acceptance and tolerance • be humane: attentiveness and empathy relevance of social aspects in processes and potential for adoption to solve the identified problems in our paper. table 3. overview of solutions related to social sustainability australasian journal of information systems schoormann & di maria 2024, vol 28, research article socially business process patterns 9 to enable designers to consider shared values (vom brocke and sinnl 2011) as well as social goals, such as well-being, equity, workplace safety, and civic engagement (un 2015; vinuesa et al. 2020), this paper aims to develop patterns for socially sustainable business processes. the overview of both patterns in the realm of social sustainability and responsibility (see table 3) and patterns from adjacent fields such as green bpm (see table 4) served as valuable input for designing a new pattern set (i.e., transferring existing ideas into new forms). in doing so, a robust grounding can be ensured, and patterns might be easier to understand for designers who are already familiar with existing ones. category description (examples) implication(s) green business process patterns based on work from nowak et al. (2011), nowak et al. (2012), nowak and leymann (2013), and nowak (2014) green business process patterns are proposed which provide opportunities and solutions for improving the environmental performance of processes. for instance: • ‘green compensation’ to reduce negative impacts on the environment without changing process structure • ‘green variant’ to introduce alternative paths • ‘green resource change’ to substitute resources potential for adapting to the social dimension ecological workflow patterns lübbecke et al. (2016) derived workflow patterns to support the ecological performance of processes. for instance: • data perspective – avoid the state transition and reduce resources, such as paper and ink • operational perspective – use a video player with less energy demand during the playback of videos potential for adapting to the social dimension ecological weakness process patterns lübbecke et al. (2017) present patterns to improve processes in terms of environmental objectives. 26 weakness patterns have been derived: for instance: • replace printing of documents via sending emails • avoid redundant document storage potential for creating social antipatterns general process patterns and practices • best practices for process innovation (goni and van looy 2022), process improvement (zellner 2011), process redesign (mansar and reijers 2007) (e.g., eliminate unnecessary tasks, consider alternative tasks, use results of a trusted party) • iot improvement patterns (stoiber and schönig 2024), such as authentication and authorization. • attack pattern to ensure security (seid et al. 2014) relevance for practices, patterns, heuristics, etc.; potential for adapting to the social dimension table 4. overview of solutions from adjacent fields also, there are several papers on ‘social bpm’ focusing on how to support collaboration and integrate social media, such as brambilla et al. (2012) who proposed social patterns that aim to enhance participation, and bögel et al. (2014) who demonstrated collaboration patterns. however, social bpm refers to jointly executing processes and, thus, it does not share our core understanding of social sustainability which is why they are not considered for this work. 5 design and development 5.1 identification of practical problems based on the analysis of the first four sustainability expert interviews (see solution objectives), we derived seven main problems (p) in the form of needs and challenges (see table 5). australasian journal of information systems schoormann & di maria 2024, vol 28, research article socially business process patterns 10 id problem title problem description p1 the challenge of limited process adaptability altering core process structures is challenging when most business processes are well-established, compliant with regulations, and economically successful. this is evident, for example, by expert 4 from a small-medium company who stressed that they often do not have the capacity to adjust their processes in the short term. therefore, core procedures cannot be transformed. p2 need to preserve employee performance processes and resources are selected based on economic concerns. social sustainability needs to be considered as well to sustain the long-term performance of the employees and the entire business. p3 need to be compliant with social regulations as new regulations concerning working conditions are specified, “regular checks for compliance with working conditions and social standards” (expert 1) of the business processes need to be performed. p4 the challenge of justifying socially sustainable effects acting in a socially sustainable manner presumes changes across the entire business that cause costs. organizations need to communicate these aspects transparently and explain, for example, why costs are different from other services/products. in the words of expert 1, “the stories behind an actual product and its production need to be told”. p5 the challenge of insufficient infrastructure while make or buy questions are typically related to economic goals, organizations need to start reflecting on this question for social sustainability too. they may not have the appropriate capacities and knowledge to perform processes socially. expert 4: “[we] outsourced activities to external firms when there is a financial benefit. this is changing, as social aspects become more and more important”. p6 the challenge of global value chains value chains are often distributed around the globe and standards are differentiating between countries. therefore, monitoring the social sustainability (e.g., working conditions) of processes is challenging. expert 1: “[performed] with vendors from other countries which may have both different internal and external regulations.” p7 need to leverage individual strengths as employees have different backgrounds (education, physical capabilities, etc.), processes should be aligned with individual needs to best possibly leverage their strengths. this issue is stressed by expert 4: “[because] of the standardized processes, it is hard to integrate different people and to generate a value based on their strengths and abilities”. table 5. synthesis of practical problems 5.2 identification of best practice solutions besides the actual problems, we sought to gather best practice solutions that are successfully applied in the real world. by drawing on the interview data, nine solutions (s) could be derived: compensation—providing further offers for employees to compensate for physical work (s1), for example, via additional staff that is concerned with healthy lifestyles, free gyms, or healthy food (expert 1, 3). also, activities to compensate for negative effects on the society level can be made (s2), such as establishing foundations, making donations, or offering employees participation in social projects (expert 4). path alternatives (s3)—introducing additional process paths that enable customers to make choices. for instance, companies integrate a fair trade product in addition to a basic product to allow customers to select between two variants (one more socially sustainable than the other). resource replacement (s4)—replacing resources that cause health problems by creating an ergonomic working place like changing working equipment such as height-adjustable desks (expert 1, 3). certification (s5)—using labels and certificates to create transparency for the customer and to communicate social effects (expert 1, 4). process outsourcing (s6)—outsourcing single activities or entire processes to make use of competencies from external organizations, which can carry out processes in a more socially sustainable manner (expert 4). responsible procurement management (s7)—performing australasian journal of information systems schoormann & di maria 2024, vol 28, research article socially business process patterns 11 responsible procurement management to ensure social aspects are integrated already at the early stages of a value chain (expert 2). value chain auditing (s8)—using audits to verify compliance with internal regulations (i.e., from certain businesses) and external regulations to contribute to socially sustainable value chains (expert 1, 2, 3). process individualization (s9)— adapting processes to best fit the individual background of employees – i.e., inclusiveness – to enable revealing the strengths of every employee (expert 4). during the illustration of the patterns based on sustainability reports (details in section 6), these solutions could be verified but also extended. four additional solutions were collected: resource reuse (s10)—considering possibilities to reuse resources, especially concerning the availability of and access to resources that are scarce in some regions. for instance, implementing an “effective water management [to] recycle and enable multi-use of water” (translated from hochtiefbau, p. 138, see all cases in appendix b). accessibility (s11)— designing working environments that allow all people regardless of their backgrounds to participate in processes and corresponding resources. for instance, for people with hearing impairments, “[providing] subtitles in videos and transcription that can be read out loud during the usage of a computer“ (translated from siemens, p. 18). education and awareness (s12)—establishing mechanisms and activities that shape awareness for possible hazards in the working environment but also during the use of the produced goods as well as train people to responsibly perform processes. for illustration, “[implementing] a comprehensive competence management to ensure that workers always have the knowledge required to be able to meet working, health, and environmental protection standards” (translated from robert bosch, p. 60) as well as providing topic-related newsletters, wikis, blogs, and online community’s to be regularly sensitized the staff about important information. team spirit (s13)—allowing people to jointly stand up for social issues and goals. while this solution is comparable to the compensation, this is about performing activities by themselves instead of providing donations. for example, about 20.000 employees of ibm participated in a walkathon challenge for scholarships (ibm, p. 36). 5.3 formulation of process patterns we derived process patterns for improving social sustainability based on the solutions proposed in the literature, the problems and best practices from the experts, and the evaluation episodes. the results in this section are subject to multiple design iterations (see appendix c) and therefore capture the final stage of our process pattern set. for adapting patterns to this paper’s context (adaption approach), we mainly draw on the set of green business process patterns as proposed by nowak (2014) which synthesizes a series of prior-developed patterns (e.g., nowak et al. 2014; nowak and leymann 2013). because our goal is to provide approaches that create awareness and guide designers toward more socially sustainable processes, we argue that nowak’s work with its focus on opportunities and solutions for improvement provides a suitable foundation. as a result, 6 of 10 patterns (#sp1 to #sp6) are initially grounded in existing patterns. as an example, we transferred the idea of ‘green compensation’ (compensating negative environmental impact) to the social context in which negative impacts on people and society are taken into account. among the other adaptions are: ‘green variant’ (introducing a new process path) to social alternative paths, ‘green feature’ (adapting the appearance of a product or service to emphasize environmental properties) to social labeling, and ‘insourcing’ (reducing communication effort through centralization of certain process components) to social sourcing. australasian journal of information systems schoormann & di maria 2024, vol 28, research article socially business process patterns 12 practical problems and solutions that could not be mapped to existing patterns (development approach) served as the input for building completely new ones (#sp7 to #sp10). for instance, we did not find patterns in our sample to cover the problem of ensuring accessibility to processes in order to leverage participation (see adaptability and employee performance). therefore, a novel pattern for social accessibility was crafted. this builds upon real-world insights highlighting the power of individualization of processes and accessibility to working environments which help to ensure a fit between task and person. as another example, we collected best practices concerning the creation of awareness and education to handle possible hazards in the working environment responsibly, which was translated into the pattern for social training. the basic patterns cover approaches that are generally applicable and can be instantiated in different ways (see figure 3). compensations and alternatives can be used in cases where it is not possible to change the core structure of the processes. they can be combined with other social patterns, for instance, compensating for negative influence from having poor resources. during the instantiation, social goals need to be considered, such as process variants to establish equity of access to key services (e.g., health and education, mckenzie 2004). figure 3. basic patterns for socially sustainable business processes the resource patterns shed light on handling, reusing, and communicating social aspects of process materials and resources (see figure 4). resources have a significant impact on the [#sp1] social compensation improving internal and external socially sustainable performance by compensating negative effects without changing the (core) process structure. problem [p1]: organizations often have well-established processes that are compliant with regulations and ensure economic success. it is therefore challenging to find a way to increase social goals without adjusting the actual processes. solution [s1, s2]: in case there are no options for adjusting processes, activities for social compensation can be instantiated. they help to (partly) compensate for negative social influences. examples: ▪ compensate physical work through free access to gyms (internal), preventive health care (internal), additional goodies (e.g., drinks, clothing) (internal) ▪ compensate negative influences through financial donations to social projects (external), establishment of social projects (external) before after + [#sp2] social alternative improving socially sustainable performance by offering an alternative process path to the employees, partners, and customers without changing the (core) process structure. problem [p1, p4]: organizations often follow strict criteria for optimizing their processes. changing processes may lead to additional costs. alternative process paths could be initiated to improve social goals and generate comparable outcomes. solution [s3]: alternative process paths are implemented to improve social goals. possible process costs need to be calculated and covered (e.g., customers who choose a social process path). reasons for increased costs need to be communicated. examples: ▪ alternative work modes: remote and flexible working hours (internal) ▪ subsidized mobility variants, such as job bikes (employee decision, internal) ▪ work variants and job rotation against monotonous tasks (staff decision, internal ) ▪ fair products in addition to the standard assortment (customer decision, external) before after x activity gateway flow australasian journal of information systems schoormann & di maria 2024, vol 28, research article socially business process patterns 13 process and the participants for which reason their selection should embody social principles, for instance, inter-generational justice and healthy workplaces (e.g., breitenbach 2020). figure 4. resource patterns for socially sustainable business processes the value chain and ecosystem patterns focus on social sustainability beyond organizational boundaries (see figure 5). they prompt organizations to rethink which process activities are related to which partner to ensure the best possible execution as well as select and monitor partners along the entire supply chain with social principles, such as fair labor or establishing new infrastructure in rural communities (kotlarsky et al. 2023). [#sp7] social resource reuse improving socially sustainable performance by reusing resources that have a special value for society without changing the (core) process structure. problem [-]: organizations often select their resources based on economic goals, and thereby may neglect rethinking how the resources can be procured and reused. solution [s10]: some process resources can be reused. establishing circular instead of linear thinking helps to handle valuable resources more responsibly. examples: ▪ establish effective water management if (parts of the) processes are performed in regions with water scarcity before after [#sp3] social resource replacement improving socially sustainable performance by replacing resources that have a negative influence on actors without changing the (core) process structure. problem [p1-p3]: organizations often select their resources based on economic and functional criteria, and thereby may neglect social goals. different ways of labor and new regulations will require organizations to improve their process resources. solution [s4]: resources have a significant impact on the processes. they should reduce negative influence on employees, suppliers, and the broader society. examples: ▪ replace resources to support ergonomic work, such as height-adjustable desks ▪ hire ergonomic teams to ensure healthy working places ▪ replace resources to support health and well-being, such as through job bikes ▪ eliminate so-called ‘conflict materials’ before after [#sp4] social labeling improving the socially sustainable image of an organization by labeling processes and products with social certificates and standards. problem [p4]: the redesign of processes towards more social sustainability might be complex and cost-intensive. external people might not be aware of this. solution [s5]: socially redesigned processes and activities need to be communicated to potential customers to explain their benefits. this can be supported by labels. examples: ▪ communicate and report socially sustainable product/process features ▪ adjust the appearance of products/services to highlight social features ▪ ensure compliance with fair trade, cmia-certified cotton, … before after activity resource flow association australasian journal of information systems schoormann & di maria 2024, vol 28, research article socially business process patterns 14 figure 5. value chain and ecosystem patterns for socially sustainable business processes lastly, the human-centric patterns provide insights into how to design processes so that many different people can participate and develop a culture of cohesion (see figure 6). particularly in times of skilled labor shortage in which it is important to identify and recruit appropriate talent, it becomes more relevant to train employees to be able to execute (other) process activities as well as to establish processes that can be performed by diverse people regardless of their backgrounds (e.g., language, gender, handicaps; see also inclusive approaches as proposed by waas et al. 2023). organizations that have redesigned their processes accordingly can attract more potential talents. 6 demonstration and evaluation 6.1 applying the patterns to real-world use cases sustainability reports from companies across industries (e.g., manufacturing, energy, finance, and telecommunication) were analyzed to demonstrate that the patterns can be used in practice. this allows us to verify, refine, and extend the initial set of patterns. following our sampling strategy that makes use of the most important industrial sectors3, 24 sustainability reports from large companies were collected (see appendix b). this type of report has been critiqued in the past to be overwhelming and sometimes overselling (e.g., pucker 2021). we 3 to ensure a representative sample of reports, we draw from ‘sales of the most important industrial sectors in germany’ (statista 2019). [#sp5] social sourcing improving socially sustainable performance by transferring activities to organizations that can carry out an activity in a responsible manner. problem [p5]: the production of goods or the offering of services typically consists of numerous different activities. as some activities require high-level expertise, organizations should reflect on their own but also external partner capabilities to ensure activities are best possibly performed. solution [s6]: outsourcing in which activities are transferred to organizations that can perform them in a socially manner; insourcing to transfer external activities into the own organization that is trained in handling certain activities. examples: ▪ outsource the handling of chemical products to specialized organizations ▪ outsource the monitoring of data privacy issues to reduce risks of fails before after [#sp6] social value chain improving socially sustainable performance by defining and monitoring socially sustainable standards that have to be applied by every partner within a value chain. problem [p6]: today’s value chains reach beyond national boundaries and are subject to different laws and regulations. this heterogeneity leads to challenges in terms of monitoring and compliance. solution [s7, s8]: as process partners have a significant impact, they should be selected and monitored through the lens of social goals. examples: ▪ perform continuous audits to verify how process partners fulfill social regulations ▪ check compliance of process activities that are performed with external partners ▪ establish supplier management systems to report on compliance with work safety before after ex te rn al in te rn al ex te rn al in te rn al § ex te rn al in te rn al ex te rn al in te rn al activity … actor flow australasian journal of information systems schoormann & di maria 2024, vol 28, research article socially business process patterns 15 figure 6. human-centric patterns for socially sustainable business processes anticipate possible issues concerning the results but find sustainability reports to be an adequate source to extract generic insights because at worst, they represent an idealized state of activities. also, they have served as useful input for investigating sustainability (e.g., wulfert et al. 2023). in terms of the analysis, one researcher (the first author) coded the reports to extract paragraphs that refer to social issues and initiatives. afterward, two additional researchers analyzed the paragraphs and mapped them to the initial set of patterns by indicating which pattern is supported, and which pattern can be refined or extended (see table 6). the coding results were consolidated in follow up-workshops. especially aspects concerning the abstraction level were discussed in-depth, including: should we split ‘social compensation’ flow [#sp8] social accessibility improving socially sustainable performance by adjusting processes to allow participations regardless of employee’s, partner’s, customer’s background. problem [p1, p2, p7]: organizations often design their processes based on economic and functional goals. aspects of designing processes in an accessible way to incorporate diverse people might be less focused. solution [s9, s11]: adjusting processes in terms of human-specific needs to allow everyone regardless of their background (e.g., gender, religion, physical health) to participate in activities. individualization and flexibility need to be considered. examples: ▪ provide flexible process structures, such as remote (internal) ▪ monitor human fatigue, such as tiredness during physical labor (internal) ▪ use inclusive/multiple languages to reduce barriers (internal, external) ▪ add subtitles to activities/resources that can be read aloud (internal, external) before after [#sp9] social training improving socially sustainable performance by educating internal and external actors to enable them to participate in processes appropriately. problem [p2, p7]: organizations often design their process activities based on economic goals. the use of internal competencies and talent might be neglected. solution [s12]: extending processes to train and promote internal talent. this enables employees to perform certain activities in a socially responsible manner. examples: ▪ implement competence management (employees have the required knowledge) ▪ add process activities to train how to handle hazardous material, resources ▪ allow employees to try (new) process activities by themselves (sandbox) before after [#sp10] social togetherness improving socially sustainable performance by creating a strong teamcohesion within an organization and doing something good together. problem [p-]: organizations often focus on economic issues and thereby neglect ‘soft activities’ that are capable of enhancing the organizational culture. solution [s13]: additional activities should be offered to allow people to jointly stand up for social issues/goals. this improves internal and external performance. examples: ▪ offer events that can be joined by all employees, such as joint breakfasts ▪ establish ‘walkathon challenges’ or donation initiatives to support scholarships ▪ establish carpools to bring together different employees (reduce emissions) before after + + activity resource associationgateway australasian journal of information systems schoormann & di maria 2024, vol 28, research article socially business process patterns 16 category pattern observation/examples verification #sp1. social alternative flexible working (lufthansa, p. 68) #sp4. social labeling cmia-certified cotton (otto, p. 83) #sp6. social value chain suppliers need to implement a social/ environmental management system (ibm, p. 18) refinement #sp2. social compensation added external views to the description to reflect aspects beyond internal compensation #sp3. social resource replacement split from general ‘resource handling’ into more concrete patterns for replacement and reuse #sp7. social resource reuse #sp8. social accessibility broadened the scope from ‘individualization’ to a more inclusive pattern for general accessibility extension #sp9. social training competence management to train and inform employees (robert bosch, p. 60) #sp10. social togetherness ‘walkathon challenge’(ibm, p. 36) table 6. insights from the demonstration (evaluation episode 1) into more specific ones, such as health for employees and social projects? we decided to stick to the initial patterns and subordinate details to the existing patterns (e.g., health as part of social compensation). while the demonstration reveals examples and support for nine out of ten patterns, there were no examples of ‘social sourcing’ in our sample of reports. nonetheless, we decided to keep it because of its relevance in other fields of sustainability. 6.2 applying the patterns to improve social sustainability in business processes in the second evaluation, we explored if the patterns are assumed to be important, accessible, and suitable, i.e., provide sufficient guidance (iivari et al. 2020; rosemann and vessey 2008). as part of a voluntary exercise, we introduced the patterns and a task to the evaluation participants, let them generate ideas to improve a given scenario and model their most promising ideas with bpmn, as well as asked them to fill out a questionnaire. 18 students participated voluntarily of which 15 filled out the questionnaire (see table 7). the results point to the overall importance of the pattern to advance social aspects in organizations (accumulated mean 3.77) and that the patterns are accessible (e.g., easy to understand, mean 3.53). while the students demonstrate the patterns are suitable and, thus, able to guide them in their analysis and redesign activities (e.g., accumulated mean 3.2), additional comments reflect the need for more guidance. for instance: “i think it is difficult to apply the rather abstract patterns to concrete processes.” (participant 9) therefore, we decided to split the initial pattern on resource handling into concrete ones for resource replacement (#sp3) and resource reuse (#sp7), and extended the description of examples for each pattern. as the accumulated values for importance and accessibility indicate positive tendencies, we are confident that the patterns provide a useful tool for the business process (re-)design. moreover, seven participants assessed the overall rating of the patterns with 4/5. while the participants have a bpm background (i.e., attended a university course on bpm and had at least basic experience in modeling, analyzing, and redesigning processes), the generalizability of the findings is limited because they are at student level. they may become future process managers or consultants, which is why we believe their assessment of the pattern usage is still worthwhile to examine. australasian journal of information systems schoormann & di maria 2024, vol 28, research article socially business process patterns 17 question item mean median sd importance in my view, the set of patterns addresses a real problem. 3.6 4 0.63 in my view, the set of patterns addresses an important, acute problem. 3.93 4 0.45 accessibility the set of patterns is easy for me to understand. 3.53 4 0.91 the set of patterns is intelligible to me. 3.27 3 0.79 suitability (actability and guidance) i find that the set of patterns provides sufficient direction. 3.13 3 0.74 i find that the set of patterns provides concrete recommendations. 3.27 3 0.70 table 7. results from the evaluation episode 2 6.3 investigating the pattern’s practical usefulness through expert interviews in the third part, we collect additional data on the usefulness of our pattern set to support bpm in practice. we selected experts with convenient experience in process management and process consulting; in contrast to sustainability experts during the grounding. seven interviews with 252 minutes were conducted with experts from different domains and roles (see section 3.3). based on the analysis, the overall usefulness of the patterns was confirmed by the experts. this is evident, for instance, by: “definitely useful. especially if one aims at implementing social sustainability. it gives companies an orientation [..] processes can be assessed, blind spots can be discussed and mechanisms can be planned. like a completeness check.” (i1) other statements point to the ability to give impulses for redesign and improvement (e.g., “can be used for inspiration”, i7; “for people without bpmn background”, i2). we asked them to rate the usefulness on a five-point likert scale from 1 (not useful) to 5 (highly useful) and achieved an average of 4.07 (min 2, max 5). during the interviews, experts elaborated on actual ways of using the patterns in practice. they suggest using them in collaboration with other employees to explore how those patterns are or should be considered and then jointly determine an action plan. in addition, the patterns can serve as guiding questions for workshops to analyze an organization’s current state in terms of social sustainability. some experts also point to other forms of representing and complementing the pattern content, such as through canvasses. we further extracted additional practical examples to extend our pattern set and explored the potential for refinements. new practical cases cover compensation of physical work through additional goodies, alternatives through subsidizing mobility variants, resource replacement by supporting ergonomic work with height-adjustable desks, and sourcing in which chemical products are outsourced to specialized organizations. as improvements, experts particularly emphasized the role of examples for advanced guidance. for instance: “i would prefer more examples and sub-points for inspiration” (i1) or “digital examples would be helpful in addition to the physical ones that are currently provided” (i7). several additional (digital) examples were added4; also from the expert interviews during the evaluation. lastly, we observed some tensions (wang et al. 2023): first, inspiration through examples versus perceptions as checklists. while examples are a source of inspiration, they may be perceived as a 4 please note: the refinements are already a part of the presented, final pattern set (section 5). australasian journal of information systems schoormann & di maria 2024, vol 28, research article socially business process patterns 18 checklist and limit the exploration of new ideas. users may try to follow the provided examples as they recognize them as a full list in a category. second, generalization versus contextualization. while the patterns are intended to be abstract to provide impulses for numerous situations and users without extensive modeling expertise, they require effort in contextualizing them to individual purposes. experts pointed to contextual factors, including organization domain and size (e.g., “[..] dependent on the organization. smes – our target customer group – are less concerned with patterns for job rotation.”, i6). third, sustainability knowledge versus process knowledge. while more process-oriented users may benefit from the list of social patterns and their underpinning ideas, more sustainability-experienced users may find them less helpful because of their existing knowledge. for instance, a sustainability expert argued: “i think that i am working with them anyway [..] probably i use those patterns implicitly and not systematically. this helps to get a more complete procedure”, i5). addressing these observed tensions not just as issues but as opportunities is crucial; especially in situations where tensions drive process redesign and transform challenges into innovation. wimelius et al. (2021) emphasize that embracing contradictions can lead to more adaptive systems during digital transformation. embracing these tensions makes the process design more adaptive to diverse perspectives required by phenomena such as those faced when taking into account the concept of social sustainability (van der byl and slawinski 2015). 7 discussion given the auspicious ability of bpm to implement sustainable ideas into organizations, we present ten process patterns to inspire improvements in the realm of social sustainability. the patterns are grounded in literature and expert interviews and iteratively refined through several evaluation episodes. however, we do not claim completeness as the understanding of the social dimension is still evolving and characterized as a multifaceted concept. 7.1 implications related to patterns for sustainable business process management patterns have been around for several years and bear potential for research on business processes and beyond (e.g., dickhaut et al. 2022; rosemann et al. 2023). referring to this paper’s context of bpm, particularly from a practice viewpoint, the proposed process patterns can be adapted by organizations to improve their social performance. they support process designers in analyzing and (re-)designing business processes and getting impulses for alternative solutions (as known from explorative bpm techniques, groß et al. 2024). this is important given that the improvement phase is considered to be the most creative one (falk et al. 2013; zellner 2013) and the concept of social sustainability has various nuances. our solution and functional views on organizations enable reflection on the possible design space. patterns are proven to increase awareness of this space but also require careful selection and contextualization for the situation at hand (reijers and mansa 2005; rosemann 2020). this work is an additional step towards a broader view of sustainable development and responds to the fact that “[a] vast majority of authors agree on the term ‘green’ bpm (instead of ‘sustainable’ bpm)” (couckuyt and van looy 2019, p. 19). with an emphasis on the social dimension, we complement existing research in the context of business process patterns and provide knowledge for improving processes – dsr knowledge contribution to level 2 in which knowledge as operational principles or architecture is presented (gregor and hevner 2013). existing collections of patterns can be extended through social perspectives, such as the australasian journal of information systems schoormann & di maria 2024, vol 28, research article socially business process patterns 19 ‘business process model patterns classification’ (bpmpatterns.org, schoknecht et al., 2020) and the ‘business process model pattern taxonomy’ (fellmann et al. 2017), as well as overviews of design heuristics (e.g., frank et al. 2020) and best practices (e.g., mansar and reijers 2007). as an example, the pattern classification dimension for ‘quality, compliance and risk’ already comprises environmental impacts and can be complemented through social sustainability. in comparison to adjacent patterns, we can observe both overlaps and peculiarities. rosemann et al.’s (2023) benevolent process design guidelines, for instance, also point to the relevance of reaching beyond traditional economic values and the well-being of stakeholders. these guidelines share some similarities with our patterns, such as considering customers' wellbeing (e.g., guidelines for ‘be fair’) or compensating for negative effects (e.g., guidelines for ‘do right’). in contrast, our work contextualizes abstract ideas from existing literature. for example, rosemann’s (2020) explorative design pattern ‘process expansion’ creates value through additional activities and is similar to our ‘social compensation’, and ‘process differentiation’ is comparable to ‘social alternative’. lastly, adjacent work contains ideas that are not covered by the current version of our pattern set due to their focus on rather related aspects or specific viewpoints, such as allowing employee suggestions (sohns et al. 2023) or authentication to check beforehand if tasks can be performed by certain persons (stoiber and schönig 2024). table 8 provides an overview of this paper’s pattern set and maps it with existing work on patterns, practices, and guidelines to highlight similarities and potential for combining different approaches in the future. this paper examples of related bpm redesign approaches #sp1. social compensation reverse, compensate for a mistake (rosemann et al. 2023) green compensation for environmental processes (nowak 2014) #sp2. social alternative acceptance, such as offering a variety of currencies (rosemann et al. 2023) green variant for environmental processes (nowak 2014) triage best practice: alternative tasks (mansar and reijers 2007) #sp3. social resource replacement resource change for environmental processes (nowak 2014) substitution of materials (betz and carporale 2014) less energy-intensive resources (lübbecke et al. 2016) #sp4. social labeling eco-labels for green it (loeser 2013) #sp5. social sourcing external advice (sohns et al. 2023) outsourcing, use trusted parties (mansar and reijers 2007) #sp6. social value chain -- #sp7. social resource reuse resource reduction (lübbecke et al. 2016) #sp8. social accessibility employ disabled people (dewan et al. 2012) empathy to demonstrate care (rosemann et al. 2023) empower: give workers authority (mansar and reijers 2007) #sp9. social training education, skills training (dewan et al. 2012) prevention to avoid regret/harm by new activities (rosemann et al. 2023) tasks and skills (haleem et al. 2023) deviation detection (stoiber and schönig 2024) #sp10. social togetherness sponsoring social events, and awareness programs (dewan et al. 2012) attentiveness to feel seen (rosemann et al. 2023) table 8. comparison of this pattern set to examples of bpm redesign approaches 7.2 implications related to social sustainability emerging streams in is research shed particular light on social sustainability (e.g., kranz et al. 2022; lobschat et al. 2021; tim et al. 2021) and thereby follow recent shifts in customer australasian journal of information systems schoormann & di maria 2024, vol 28, research article socially business process patterns 20 demands and ensure compliance with new regulations. our work gives imperatives on how to transform organizational processes into more socially sustainable entities. it complements is and bpm research on social sustainability, such as social process redesign (e.g., senderovich et al. 2020) and social responsibility (e.g., haleem et al. 2024). by obtaining real-world problems from experts who have to deal with socially driven changes, we disclose both barriers and insights on how to manage social sustainability. this can support research and practice in making more informed decisions concerning future efforts, such as planning change projects and integrating socially sustainable goals into existing standards and certifications. also, it covers empirical findings and, thus, responds to recent calls for more empirical studies in sustainable is (tan and nielsen 2024). once the patterns presented in this paper have been applied to specific problems, experiential knowledge can be accumulated which in turn may enhance our collective understanding of social sustainability. the comparison of our results with the first conceptualizations of social sustainability in the is discipline (kotlarsky et al. 2023; schoormann and kutzner 2020) indicates that the pattern set addresses most of the identified themes. as an example, diversity and inclusion can be promoted through accessible working environments (social accessibility, #sp8) and cohesion through togetherness (social togetherness, #sp10). also, social characteristics are fulfilled, such as education by training employees (social training, #sp9), health by eliminating conflict materials (social resource replacement, #sp3), as well as welfare by compensating for the negative effects on the broader society (social compensation, #sp2). from a legal viewpoint, we see growing pressure to reflect on social factors as evidenced by new regulations like the australian sustainability reporting standards (aasb 2023) or the eu taxonomy (european commission 2023). our patterns pave the ground for improving social issues. take, for instance, key criteria of the social factor from the esg framework (cpa australia 2024): employee health and safety can be addressed by social compensation and alternatives (#sp1-#sp2) or social resources (#sp3); training and development by social training (#sp9); diversity, equity, and inclusion by social accessibility (#sp8); community support by social togetherness (#sp10); human rights and labor by social value chain (#sp6). 7.3 implications related to designing process patterns from a methodical view, the development of process patterns should make use of existing domain knowledge (nowak 2014). therefore, this paper presents an approach that extracts empirical data, such as from expert interviews and secondary data (here in the form of sustainability reports). this allows us to consider both grounding in existing knowledge and ensuring practical relevance. moreover, fellmann et al. (2018) argued that the prevailing type of publications in the context of patterns is the proposal of solution (i.e., presenting a solution for a given problem) and only 13% of their sample of publications fall into the category evaluation research. to address this deficit, we draw on established steps from dsr to perform both activities of developing and evaluating artifacts, which can serve as a blueprint. 7.4 limitations and future directions this paper is not free of limitations which – in turn – open avenues for future research. first, during the analysis, we extracted possible dependencies between the patterns, which are not the focus of this paper. for instance, healthy working conditions can be achieved through ‘compensation’ (e.g., offering additional activities that compensate for unhealthful activities) or ‘training’ (e.g., offering education so that employees can handle dangerous working australasian journal of information systems schoormann & di maria 2024, vol 28, research article socially business process patterns 21 materials appropriately). more research is required to investigate whether certain patterns can be combined or not. second, the patterns shed light on the ‘bright side’ (positive) of social sustainability and thereby aim to represent good examples. to provide a tool that is capable of identifying the ‘dark side’ in particular, so-called anti-patterns could be explored. these patterns pose solutions that are known to have weaknesses and help to avoid these in the first place (koschmider et al. 2019). third, we present rather generic patterns that can be applied to different units including the process model, approaches to be implemented by organizations to improve social sustainability, as well as the continuum from more operative to more strategic decisions. they create awareness of process designers for the existence of such opportunities and provide ideas to treat a particular problem class but still need “to be adapted in skilfull ways in response to prevailing conditions” (reijers and mansar 2005, p. 294). the feasibility assessment and actual deployment of one of the patterns is highly context-specific (rosemann 2020) and beyond the scope of this paper. fourth, our demonstration and evaluation are restricted to the participants (e.g., students from a bpm course in episode 2 lead to limited generalizability). future efforts should be made in terms of proof of use in naturalistic settings, such as through case studies in organizations. lastly, general methodical limitations apply: the literature review is restricted to its search phrases and sources; the coding of the data is based on our own decisions; the number of experts and their location (country-specific regulations); sustainability reports tend to be more positively and could be subject to ‘social washing’. 8 conclusion transforming businesses towards sustainability is among the urgent pressures. a successful transformation requires considering numerous viewpoints, such as customers who increasingly spend their money on fair products, employees who demand inclusive working conditions, and lawmakers who are passing new regulations. while bpm can help organizations become more sustainable, we can observe that the supporting toolset overemphasizes the economic paradigm. we present a set of ten business process patterns to advance our understanding of how to employ bpm for social aspects. they allow for reflecting on important social issues, including accessibility of working environments, training of employees, responsible usage of resources, and monitoring supply chains. the patterns provide orientation about the analysis and (re-)design of processes, as well as support generating and implementing novel ideas in existing organizations. our work complements existing tools and methods from bpm, advances the current body of knowledge in this stream, and sparks future research on this important perspective to achieve a more holistic consideration of sustainability in business processes. references aasb (2023). exposure draft ed sr1 australian sustainability reporting standards. https://aasb.gov.au/news/exposure-draft-ed-sr1-australian-sustainability-reportingstandards-disclosure-of-climate-related-financial-information/ ahmed, m. d., and sundaram, d. (2012). sustainability modeling and reporting: from roadmap to implementation. decision support systems, 53(3), 611–624. doi.org/10.1016/j.dss.2012.02.004 australasian journal of information systems schoormann & di maria 2024, vol 28, research article socially business process patterns 22 ahlers, d., krogstie, j., driscoll, p., lundli, h. e., loveland, s. j., rothballer, c., and wyckmans, a. (2017). supporting municipal greenhouse gas (ghg) emission inventories using business process modeling: a case study of trondheim municipality. business process management workshops (pp. 416–427). springer. alexander, c. (1977). a pattern language: towns, buildings, construction. oxford university press. anjaria, k. (2024). enhancing sustainability integration in sustainable enterprise resource planning (s-erp) system: application of transaction cost theory and case study analysis. international journal of information management data insights, 4(2), 100243. doi.org/10.1016/j.jjimei.2024.100243 betz, s., and caporale, t. (2014). sustainable software system engineering. proceedings of international conference on big data and cloud computing. bögel, s., stieglitz, s., and meske, c. (2014). a role model-based approach for modeling collaborative processes. business process management journal, 20(4), 598–614. doi.org/10.1108/bpmj-07-2013-0094 brambilla, m., fraternali, p., and vaca, c. (2012). bpmn and design patterns for engineering social bpm solutions. in f. daniel, k. barkaoui, and s. dustdar (eds.), business process management workshops (pp. 219–230). springer. doi.org/10.1007/978-3-642-28108-2_22 breitenbach, f. j. (2020). application of business process management for corporate sustainability [doctoral dissertation, university of plymouth]. doi.org/10.24382/1199 brundtland, g., khalid, m., agnelli, s., al-athel, s., chidzero, b., fadika, l., hauff, v., lang, i., shijun, m., morino de botero, m., singh, m., okita, s., and others, a. (1987). our common future ('brundtland report’). oxford university press, usa. christmann, a. s., crome, c., graf-drasch, v., oberländer, a. m., and schmidt, l. (2024). the twin transformation butterfly: capabilities for an integrated digital and sustainability transformation. business & information systems engineering, doi.org/10.1007/s12599-02300847-2 cleven, a., winter, r., and wortmann, f. (2012). managing process performance to enable corporate sustainability: a capability maturity model. in j. vom brocke, s. seidel, and j. recker (eds.), green business process management (pp. 111–129). springer. doi.org/10.1007/978-3-642-27488-6_7 corbett, j., dennehy, d., and carter, l. (2023). introduction to the special section: digital innovation for social development and environmental action. communications of the association for information systems, 53(1), 22-41. doi.org/10.17705/1cais.05302 couckuyt, d., and van looy, a. (2019). a systematic review of green business process management. business process management journal, 26(2), 421–446. doi.org/10.1108/bpmj03-2019-0106 couckuyt, d., and van looy, a. (2021). an exploration of green business process maturity based on ecolabels. business process management journal, 27(7), 1999-2020. doi.org/10.1108/bpmj-05-2021-0293 australasian journal of information systems schoormann & di maria 2024, vol 28, research article socially business process patterns 23 dewan, n. a., chowdhury, m. h., and a quaddus, m. (2012). the elements for sustainable ebusiness modelling: a 3d approach. proceedings of the pacific asia conference on information system, hochiminh city, vietnam. dickhaut, e., li, m. m., janson, a., and leimeister, j. m. (2022). the role of design patterns in the development and legal assessment of lawful technologies. electronic markets, 32(4), 2311–2331. doi.org/10.1007/s12525-022-00597-1 dumas, m., la rosa, m., mendling, j., and reijers, h. a. (2018). fundamentals of business process management. springer. doi.org/10.1007/978-3-662-56509-4 european commission. (2023). eu taxonomy for sustainable activities. https://finance.ec.europa.eu/sustainable-finance/tools-and-standards/eu-taxonomysustainable-activities_en falk, t., griesberger, p., and leist, s. (2013). patterns as an artifact for business process improvement-insights from a case study. proceedings of design science research in information systems and technology, helsinki, finland. fellmann, m., koschmider, a., laue, r., schoknecht, a., and vetter, a. (2017). a taxonomy and catalog of business process model patterns. proceedings of the 22nd european conference on pattern languages of programs, irsee, germany. doi.org/10.1145/3147704.3147725 fellmann, m., koschmider, a., laue, r., schoknecht, a., and vetter, a. (2018). business process model patterns: state-of-the-art, research classification and taxonomy. business process management journal, 25(5), 972–994. doi.org/10.1108/bpmj-01-2018-0021 frank, l., poll, r., röglinger, m., and rupprecht, l. (2020). design heuristics for customercentric business processes. business process management journal, 26(6), 1283–1305. doi.org/10.1108/bpmj-06-2019-0257 gamma, e. (1995). design patterns: elements of reusable object-oriented software. addison-wesley longman publishing. gohar, s. r., and indulska, m. (2020). environmental sustainability through green business process management. australasian journal of information systems, 24. doi.org/10.3127/ajis.v24i0.2057 gohar, s. r., and indulska, m. (2016). business process management: saving the planet? preprint arxiv:1606.02459. ghose, a., hoesch-klohe, k., hinsche, l., and le, l. s. (2010). green business process management: a research agenda. australasian journal of information systems, 16(2). doi.org/10.3127/ajis.v16i2.597 goldkuhl, g., and sjöström, j. (2018). design science in the field: practice design research. in s. chatterjee, k. dutta, and r. p. sundarraj (eds.), designing for a digital and globalized world (pp. 67–81). springer publishing. doi.org/10.1007/978-3-319-91800-6_5 goni, j. i. c., and van looy, a. (2022). process innovation capability in less-structured business processes: a systematic literature review. business process management journal, 28(3), 557–584. doi.org/10.1108/bpmj-07-2021-0487 australasian journal of information systems schoormann & di maria 2024, vol 28, research article socially business process patterns 24 graves, n., koren, i., and van der aalst, w. m. p. (2023). rethink your processes! a review of process mining for sustainability. proceedings of international conference on ict for sustainability. doi.org/10.1109/ict4s58814.2023.00025 gregor, s., and hevner, a. (2013). positioning and presenting design science research for maximum impact. mis quarterly, 37(2), 337–355. doi.org/10.25300/misq/2013/37.2.01 groß, s., grisold, t., mendling, j., and hasse, j. (2024). idea generation in exploitative and explorative business process redesign techniques. information systems and e-business management (forthcoming). doi.org/10.1007/s10257-024-00684-0 haleem, f., ilyas, m., and jehangir, m. (2024). driving organizational value through sustainable business processes and green information technology. business process management journal. doi.org/10.1108/bpmj-01-2024-0011 hu, p. j. h., hu, h. f., wei, c. p., and hsu, p. f. (2016). examining firms’ green information technology practices: a hierarchical view of key drivers and their effects. journal of management information systems, 33(4), 1149-1179. ddoi.org/10.1080/07421222.2016.1267532 iivari, j., hansen, m. r. p., and haj-bolouri, a. (2020). a proposal for minimum reusability evaluation of design principles. european journal of information systems, 1–18. doi.org/10.1080/0960085x.2020.1793697 ketter, w., padmanabhan, b., pant, g., and raghu, t. s. (2020). special issue editorial: addressing societal challenges through analytics: an esg ice framework and research agenda. journal of the association for information systems, 21(5). doi.org/10.17705/1jais.00631 klessascheck, f., fahrenkrog-petersen, s. a., mendling, j., and pufahl, l. (2024). unlocking sustainability compliance: characterizing the eu taxonomy for business process management. doi.org/10.48550/arxiv.2408.11386 koschmider, a., laue, r., and fellmann, m. (2019). business process model anti-patterns: a bibliography and taxonomy of published work. proceedings of the european conference in information systems, uppsala-stockholm, sweden. kotlarsky, j., oshri, i., and sekulic, n. (2023). digital sustainability in information systems research: conceptual foundations and future directions. journal of the association for information systems, 24(4), 936-952. doi.org/10.17705/1jais.00825 kranz, j., zeiss, r., beck, r., gholami, r., sarker, s., watson, r. t., and whitley, e. a. (2022). practicing what we preach? reflections on more sustainable and responsible is research and teaching practices. communications of the association for information systems, 51(1), 557-580. doi.org/10.17705/1cais.05138 lobschat, l., mueller, b., eggers, f., brandimarte, l., diefenbach, s., kroschke, m., and wirtz, j. (2021). corporate digital responsibility. journal of business research, 122, 875–888. doi.org/10.1016/j.jbusres.2019.10.006 loeser, f. (2013). green it and green is: definition of constructs and overview of current practices. proceedings of the americas conference on information systems, chicago, illinois, usa. australasian journal of information systems schoormann & di maria 2024, vol 28, research article socially business process patterns 25 lübbecke, p., fettke, p., and loos, p. (2017). sustainability patterns for the improvement of itrelated business processes about ecological goals. in m. dumas and m. fantinato (eds.), business process management workshops (vol. 281, pp. 428–439). springer international publishing. doi.org/10.1007/978-3-319-58457-7_31 magdaleno, a. m., duboc, l., and betz, s. (2017). how to incorporate sustainability into business process management lifecycle? in m. dumas and m. fantinato (eds.), business process management workshops (vol. 281, pp. 440–443). springer international publishing. doi.org/10.1007/978-3-319-58457-7_32 mansar, s. l., and reijers, h. a. (2007). best practices in business process redesign: use and impact. business process management journal, 13(2), 193-213. doi.org/10.1108/14637150710740455 mckenzie, s. (2004). social sustainability: towards some definitions [working paper series]. medicke, j., and mcdavid, d. (2004). patterns for business process modeling. business integration journal, 1, 32–35. meuser, m., and nagel, u. (1991). expertlnneninterviews—vielfach erprobt, wenig bedacht. in d. garz and k. kraimer (eds.), qualitativ-empirische sozialforschung: konzepte, methoden, analysen (pp. 441–471). vs verlag für sozialwissenschaften. doi.org/10.1007/978-3-322-97024-4_14 missimer, m., robèrt, k.-h., and broman, g. (2017). a strategic approach to social sustainability – part 2: a principle-based definition. journal of cleaner production, 140, 42– 52. doi.org/10.1016/j.jclepro.2016.04.059 nastjuk, i., hanelt, a., and kolbe, l. (2016). too much of a good thing? an experimental investigation of the impact of digital technology-enabled business models on individual stress and future adoption of sustainable services. proceedings of the international conference on information systems, dublin, ireland. nowak, a. (2014). green business process management: methode und realisierung [doctoral thesis, university of stuttgart]. http://elib.uni-stuttgart.de/handle/11682/3423 nowak, a., binz, t., fehling, c., kopp, o., leymann, f., and wagner, s. (2012). pattern-driven green adaptation of process-based applications and their runtime infrastructure. computing, 94(6), 463–487. doi.org/10.1007/s00607-012-0188-x nowak, a., breitenbücher, u., and leymann, f. (2014). automating green patterns to compensate co2 emissions of cloud-based business processes. proceedings of the 8th international conference on advance engineering computing and applications in sciences (advcomp), rome, italy, 132–139. nowak, a., and leymann, f. (2013). an overview on implicit green business process patterns (cr: h.4.1; p. 35). universität stuttgart, institute of architecture of application systems (iaas). nowak, a., leymann, f., schleicher, d., schumm, d., and wagner, s. (2011). green business process patterns. proceedings of the 18th conference on pattern languages of programs, 6:16:10. doi.org/10.1145/2578903.2579144 australasian journal of information systems schoormann & di maria 2024, vol 28, research article socially business process patterns 26 nowak, a., leymann, f., schumm, d., and wetzstein, b. (2011). an architecture and methodology for a four-phased approach to green business process reengineering. in d. kranzlmüller and a. m. toja (eds.), information and communication on technology for the fight against global warming (vol. 6868, pp. 150–164). springer. doi.org/10.1007/9783-642-23447-7_14 opitz, n., krüp, h., and kolbe, l. m. (2014). green business process management – a definition and research framework. proceedings of the 47th hawaii international conference on system science, hawaii, usa, 3808–3817. doi.org/10.1109/hicss.2014.473 pan, s. l., carter, l., tim, y., and sandeep, m. s. (2022). digital sustainability, climate change, and information systems solutions: opportunities for future research. international journal of information management, 63, 102444. doi.org/10.1016/j.ijinfomgt.2021.102444 peffers, k., tuunanen, t., rothenberger, m. a., and chatterjee, s. (2007). a design science research methodology for information systems research. journal of management information systems, 24(3), 45–77. doi.org/10.2753/mis0742-1222240302 petter, s., khazanchi, d., and murphy, j. d. (2010). a design science based evaluation framework for patterns. acm sigmis database: the database for advances in information systems, 41(3), 9–26. doi.org/10.1145/1851175.1851177 pucker, k. p. (2021). overselling sustainability reporting. harvard business review. https://hbr.org/2021/05/overselling-sustainability-reporting recker, j., and mendling, j. (2016). the state of the art of business process management research as published in the bpm conference. business & information systems engineering, 58(1), 55–72. doi.org/10.1007/s12599-015-0411-3 recker, j., rosemann, m., hjalmarsson, a., and lind, m. (2012). modeling and analyzing the carbon footprint of business ürocesses. in j. vom brocke, s. seidel, and j. recker (eds.), green business process management (pp. 93–109). springer. doi.org/10.1007/978-3-64227488-6_6 reijers, h. a., and mansar, s. l. (2005). best practices in business process redesign: an overview and qualitative evaluation of successful redesign heuristics. omega, 33(4), 283– 306. doi.org/10.1016/j.omega.2004.04.012 rosemann, m. (2020). explorative process design patterns. proceedings of the international conference, seville, spain. doi.org/10.1007/978-3-030-58666-9_20 rosemann, m., and vessey, i. (2008). toward improving the relevance of information systems research to practice: the role of applicability checks. mis quarterly, 32(1), 1–22. doi.org/10.2307/25148826 rosemann, m., ostern, n., voss, m., and bandara, w. (2023). benevolent business processesdesign guidelines beyond transactional value. proceedings of the international conference on business process management, utrecht, netherlands, pp. 447-464. doi.org/10.1007/9783-031-41620-0_26 rozman, t., draghici, a., and riel, a. (2015). achieving sustainable development by integrating it into the business process management system. in r. v. o’connor, m. umay akkaya, k. kemaneci, m. yilmaz, a. poth, and r. messnarz (eds.), systems, australasian journal of information systems schoormann & di maria 2024, vol 28, research article socially business process patterns 27 software and services process improvement (pp. 247–259). springer international publishing. doi.org/10.1007/978-3-319-24647-5_20 russell, n., van der aalst, w. m. p., and ter hofstede, a. h. m. (2016). workflow patterns: the definitive guide. mit press. sarker, s., chatterjee, s., xiao, x., and elbanna, a. (2019). the sociotechnical axis of cohesion for the is discipline: its historical legacy and its continued relevance. mis quarterly, 43(3), 695–720. doi.org/10.25300/misq/2019/13747 schoknecht, a., vetter, a., koschmider, a., fellmann, m., and laue, r. (2020). process model pattern classification. business process model patterns classification. http://bpmpatterns.org/ schoormann, t., behrens, d., and knackstedt, r. (2017). sustainability in business process models: a taxonomy-driven approach to synthesize knowledge and structure the field. proceedings of the international conference on information systems, seoul, south korea. schoormann, t., and kutzner, k. (2020). towards understanding social sustainability: an information systems research-perspective. proceedings of the international conference on information systems, virtual. schoormann, t. and kutzner, k. (2021). socially business process patterns – a sustainability report-driven demonstration and refinement. proceedings of the informatik. doi.org/10.18420/inf2020_53 schoormann, t., kutzner, k., pape, s., and knackstedt, r. (2019). elevating social sustainability in business processes: a pattern-based approach. proceedings of the international conference on information systems, munich, germany. schreiber, c. (2020). automated sustainability compliance checking using process mining and formal logic. proceedings of the 7th international conference on ict for sustainability. doi.org/10.1145/3401335.3401355 schryen, g. (2013). revisiting is business value research: what we already know, what we still need to know, and how we can get there. european journal of information systems, 22(2), 139–169. doi.org/10.1057/ejis.2012.45 schweizer, a., schlatt, v., urbach, n., and fridgen, g. (2017). unchaining social businessesblockchain as the basic technology of a crowdlending platform. proceedings of the international conference on information systems, seoul, south korea. seid, e., popov, o., and blix, f. (2024). evaluation of asfalia, a security attack event monitoring framework. procedia computer science, 237, 793–802. doi.org/10.1016/j.procs.2024.05.167 senderovich, a., schippers, j. j., and reijers, h. a. (2020). socially-aware business process redesign. in d. fahland, c. ghidini, j. becker, and m. dumas (eds.), business process management (vol. 12168, pp. 75–92). springer international publishing. doi.org/10.1007/978-3-030-58666-9_5 seidel, s., bharati, p., fridgen, g., watson, r., albizri, a., boudreau, m.-c., butler, t., kruse, l., guzman, i., karsten, h., lee, h., melville, n., rush, d., toland, j., and watts, s. (2017). the sustainability imperative in information systems research. communications of the association for information systems, 40(1). doi.org/10.17705/1cais.04003 australasian journal of information systems schoormann & di maria 2024, vol 28, research article socially business process patterns 28 seidel, s., brocke, j. vom, and recker, j. (2011). call for action: investigating the role of business process management in green is. all sprouts content, 11(4). seidel, s. and recker, j. (2012), implementing green business processes: the importance of functional affordances of information systems. proceedings of the australasian conference on information systems, geelong, australia. smith, c. u., and williams, l. g. (2000). software performance antipatterns. proceedings of the 2nd international workshop on software and performance, 127–136. doi.org/10.1145/350391.350420 sohns, t. m., aysolmaz, b., figge, l., and joshi, a. (2023). green business process management for business sustainability: a case study of manufacturing small and medium-sized enterprises (smes) from germany. journal of cleaner production, 401, 136667. doi.org/10.1016/j.jclepro.2023.136667 stadtländer, m., schoormann, t., and knackstedt, r. (2019). how software promotes the integration of sustainability in business process management. proceedings of the 14th wirtschaftsinformatik, siegen, germany, 1942–1953. stoiber, c., and schönig, s. (2024). leveraging the industrial internet of things for business process improvement: a metamodel and patterns. information systems and e-business management. doi.org/10.1007/s10257-024-00676-0 statista (2019). umsätze der wichtigsten industriebranchen in deutschland bis 2018. statista. https://de.statista.com/statistik/daten/studie/241480/umfrage/umsaetze-derwichtigsten-industriebranchen-in-deutschland/ tan, b., and nielsen, p. (2024). information systems and sustainable development: from conceptual underpinnings to empirical insights. information systems journal. doi.org/10.1111/isj.12543 tim, y., cui, l., and sheng, z. (2021). digital resilience: how rural communities leapfrogged into sustainable development. information systems journal, 31(2), 323-345. doi.org/10.1111/isj.12312 troshani, i., doolin, b., fradley, k., and rampersad, g. (2022). information systems, sociomaterial practices and the emergence of environmental management infrastructures. australasian journal of information systems, 26. doi.org/10.3127/ajis.v26i0.3829 un (2015). a/res/70/1—transforming our world: the 2030 agenda for sustainable development. https://sustainabledevelopment.un.org/index.php?page=view&type=111&nr=8496&me nu=35 un (2021). united nations global compact,. social sustainability. https://www.unglobalcompact.org/ what-is-gc/our-work/social vallance, s., perkins, h. c., and dixon, j. e. (2011). what is social sustainability? a clarification of concepts. geoforum, 42(3), 342-348. doi.org/10.1016/j.geoforum.2011.01.002 van der byl, c. a., and slawinski, n. (2015). embracing tensions in corporate sustainability: a review of research from win-wins and trade-offs to paradoxes and beyond. organization & environment, 28(1), 54-79. doi.org/10.1177/1086026615575047 australasian journal of information systems schoormann & di maria 2024, vol 28, research article socially business process patterns 29 veit, d. j., & thatcher, j. b. (2023). digitalization as a problem or solution? charting the path for research on sustainable information systems. journal of business economics, 93(6), 1231-1253. doi.org/10.1007/s11573-023-01143-x vinuesa, r., azizpour, h., leite, i., balaam, m., dignum, v., domisch, s., felländer, a., langhans, s. d., tegmark, m., and fuso nerini, f. (2020). the role of artificial intelligence in achieving the sustainable development goals. nature communications, 11(1), article 1. doi.org/10.1038/s41467-019-14108-y vom brocke, j., loos, p., seidel, s., and watson, r. t. (2013). green is: information systems for environmental sustainability. business & information systems engineering, 5(5), 295–297. doi.org/10.1007/s12599-013-0288-y vom brocke, j., watson, r. t., dwyer, c., elliot, s., and melville, n. (2013). green information systems: directives for the is discipline. communications of the association for information systems, 33(1), 30. doi.org/10.17705/1cais.03330 vom brocke, j., seidel, s., and recker, j. (eds.). (2012). green business process management. springer berlin heidelberg. doi.org/10.1007/978-3-642-27488-6 vom brocke, j., and sinnl, t. (2011). culture in business process management: a literature review. business process management journal, 17(2), 357–378. doi.org/10.1108/14637151111122383 watson, r. t., boudreau, m.-c., and chen, a. j. (2010). information systems and environmentally sustainable development: energy informatics and new directions for the is community. mis quarterly, 34(1), 23–38. wang, b., ciriello, r. f., and mathiassen, l. (2023). what is paradox in information systems research? towards a narratology. proceedings of the australasian conference on information systems, wellington, australia. wass, s., thygesen, e., and purao, s. (2023). principles to facilitate social inclusion for designoriented research. journal of the association for information systems, 24(5), 1204-1247. doi.org/10.17705/1jais.00814 webster, j., and watson, r. t. (2002). analyzing the past to prepare for the future: writing a literature review. mis quarterly, 26(2), xiii–xxiii. https://www.jstor.org/stable/4132319 wesumperuma, a., ginige, j. a., ginige, a., and hol, a. (2011). a framework for multidimensional business process optimization for ghg emission mitigation. proceedings of australasian conference on information systems, sydney, australia. wimelius, h., mathiassen, l., holmström, j., and keil, m. (2021). a paradoxical perspective on technology renewal in digital transformation. information systems journal, 31(1), 198225. doi.org/10.1111/isj.12307 winter, r., vom brocke, j., fettke, p., loos, p., junginger, s., moser, c., keller, w., matthes, f., and ernst, a. (2009). patterns in business and information systems engineering. business & information systems engineering, 1(6), 468. doi.org/10.1007/s12599-009-0073-0 wulfert, t., woroch, r., strobel, g., schoormann, t., and banh, l. (2023). unboxing the role of e-commerce ecosystems to address grand challenges. proceedings of the european conference on information systems, kristiansand, norway. australasian journal of information systems schoormann & di maria 2024, vol 28, research article socially business process patterns 30 zellner, g. (2011). a structured evaluation of business process improvement approaches. business process management journal, 17(2), 203–237. doi.org/10.1108/14637151111122329 appendix a: id reference 01 schoormann, t., behrens, d., & knackstedt, r. (2017). sustainability in business process models: a taxonomy-driven approach to synthesize knowledge and structure the field. in proceedings of the international conference on information systems. 02 schoormann, t., kutzner, k., pape, s., & knackstedt, r. (2019). elevating social sustainability in business processes: a pattern-based approach. proceedings of the international conference on information systems, munich, germany. 03 stadtländer, m., schoormann, t., & knackstedt, r. (2019). how software promotes the integration of sustainability in business process management. in proceedings of the wirtschaftsinformatik. 04 wesumperuma, a., ginige, j. a., ginige, a., & hol, a. (2011). a framework for multi-dimensional business process optimization for ghg emission mitigation. in proceedings of the australasian conference on information systems. 05 couckuyt, d., & van looy, a. (2019). a systematic review of green business process management. business process management journal, 26(2), 421-446. doi.org/10.1108/bpmj-03-2019-0106 06 vom brocke, j., & sinnl, t. (2011). culture in business process management: a literature review. business process management journal, 17(2), 357-378. doi.org/10.1108/14637151111122383 07 bögel, s., stieglitz, s., & meske, c. (2014). a role model-based approach for modelling collaborative processes. business process management journal, 20(4), 598-614. doi.org/10.1108/bpmj-07-2013-0094 08 brambilla, m., fraternali, p., & vaca, c. (2011). a notation for supporting social business process modeling. in business process model and notation, workshop bpmn, lucerne, switzerland (pp. 88-102). springer berlin heidelberg. doi.org/10.1007/978-3-642-25160-3_7 09 brambilla, m., fraternali, p., & vaca, c. (2012). bpmn and design patterns for engineering social bpm solutions. in business process management workshops: bpm 2011 international workshops, clermontferrand, france (pp. 219-230). springer berlin heidelberg. doi.org/10.1007/978-3-642-28108-2_22 10 lübbecke, p., fettke, p., & loos, p. (2016). towards ecological workflow patterns as an instrument to optimize business processes with respect to ecological goals. in hawaii international conference on system sciences (pp. 1049-1058). ieee. 11 lübbecke, p., fettke, p., & loos, p. (2017). sustainability patterns for the improvement of it-related business processes with regard to ecological goals. in business process management workshops: bpm 2016 international workshops, rio de janeiro, brazil (pp. 428-439). springer international publishing. doi.org/10.1007/978-3-319-58457-7_31 12 nowak, a. (2014). green business process management: methode und realisierung. doi.org/10.18419/opus-3406 13 nowak, a., & leymann, f. (2013). an overview on implicit green business process patterns. universität stuttgart, institut für architektur von anwendungssystemen. 14 nowak, a., & leymann, f. (2013). green business process patterns--part ii. in international conference on service-oriented computing and applications (pp. 168-173). ieee. doi.org/10.1145/2578903.2579144 15 nowak, a., leymann, f., & schumm, d. (2011). the differences and commonalities between green and conventional business process management. in international conference on dependable, autonomic and secure computing (pp. 569-576). ieee. doi.org/10.1109/dasc.2011.105 16 nowak, a., breitenbücher, u., & leymann, f. (2014). automating green patterns to compensate co2 emissions of cloud-based business processes. in proceedings of advcomp (pp. 132-139). australasian journal of information systems schoormann & di maria 2024, vol 28, research article socially business process patterns 31 17 nowak, a., leymann, f., schleicher, d., schumm, d., & wagner, s. (2011). green business process patterns. in proceedings of conference on pattern languages of programs (pp. 1-10). doi.org/10.1145/2578903.2579144 18 nowak, a., binz, t., fehling, c., kopp, o., leymann, f., & wagner, s. (2012). pattern-driven green adaptation of process-based applications and their runtime infrastructure. computing, 94, 463-487. doi.org/10.1007/s00607-012-0188-x 19 schoormann, thorsten; kutzner, kristin (2021).: socially business process patterns – a sustainability reportdriven demonstration and refinement. proceedings of the informatik. doi.org/10.18420/inf2020_53 20 schreiber, c. (2020). automated sustainability compliance checking using process mining and formal logic. in proceedings of the 7th international conference on ict for sustainability (pp. 181-184). doi.org/10.1145/3401335.3401355 21 larsch, s., betz, s., duboc, l., magdaleno, a. m., & bomfim, c. (2017). integrating sustainability aspects in business process management. in business process management workshops, rio de janeiro, brazil, september 19, 2016, revised papers 14 (pp. 403-415). springer international publishing. doi.org/10.1007/978-3-319-58457-7_29 22 vom brocke, j., watson, r. t., dwyer, c., elliot, s., & melville, n. (2013). green information systems: directives for the is discipline. communications of the association for information systems, 33. doi.org/10.17705/1cais.03330 23 watson, r. t., boudreau, m. c., & chen, a. j. (2010). information systems and environmentally sustainable development: energy informatics and new directions for the is community. mis quarterly. doi.org/10.2307/20721413 24 schoormann, t., & kutzner, k. (2020). towards understanding social sustainability: an information systems research-perspective. in proceedings of the icis. 25 dewan, n. a., chowdhury, m. h., & a quaddus, m. (2012). the elements for sustainable e-business modelling: a 3d approach. in proceedings of the pacis. 26 schweizer, a., schlatt, v., urbach, n., & fridgen, g. (2017). unchaining social businesses-blockchain as the basic technology of a crowdlending platform. in proceedings of the icis. 27 hu, p. j. h., hu, h. f., wei, c. p., & hsu, p. f. (2016). examining firms’ green information technology practices: a hierarchical view of key drivers and their effects. journal of management information systems, 33(4), 1149-1179. doi.org/10.1080/07421222.2016.1267532 28 goni, j. i. c., & van looy, a. (2022). process innovation capability in less-structured business processes: a systematic literature review. business process management journal, 28(3), 557-584. doi.org/10.1108/bpmj07-2021-0487 29 mansar, s. l., & reijers, h. a. (2007). best practices in business process redesign: use and impact. business process management journal, 13(2), 193. doi.org/10.1108/14637150710740455 30 haleem, f., ilyas, m., & jehangir, m. (2024). driving organizational value through sustainable business processes and green information technology. business process management journal. doi.org/10.1108/bpmj01-2024-0011 31 senderovich, a., schippers, j. j., & reijers, h. a. (2020). socially-aware business process redesign. in business process management: 18th international conference, bpm 2020, seville, spain. doi.org/10.1007/978-3-030-58666-9_5 32 rosemann, m., ostern, n., voss, m., & bandara, w. (2023). benevolent business processes-design guidelines beyond transactional value. in international conference on business process management (pp. 447-464). cham: springer nature switzerland. doi.org/10.1007/978-3-031-41620-0_26 33 zellner, g. (2011). a structured evaluation of business process improvement approaches. business process management journal, 17(2), 203-237. doi.org/10.1108/14637151111122329 australasian journal of information systems schoormann & di maria 2024, vol 28, research article socially business process patterns 32 34 stoiber, c., & schönig, s. (2024). leveraging the industrial internet of things for business process improvement: a metamodel and patterns. information systems and e-business management, 1-29. doi.org/10.1007/s10257-024-00676-0 35 sohns, t. m., aysolmaz, b., figge, l., & joshi, a. (2023). green business process management for business sustainability: a case study of manufacturing small and medium-sized enterprises (smes) from germany. journal of cleaner production, 401, 136667. doi.org/10.1016/j.jclepro.2023.136667 36 baggia, a., leskovar, r., delibašić, b., & petrović, n. (2013). opportunities of sustainable business practices in sme’s. in proceedings of the 32nd international conference on organizational science development, portoroz, slovenia (pp. 20-22). 37 rozman, t., draghici, a., & riel, a. (2015). achieving sustainable development by integrating it into the business process management system. in eurospi 2015, ankara, turkey. doi.org/10.1007/978-3-319-246475_20 38 ahlers, d., krogstie, j., driscoll, p., lundli, h. e., loveland, s. j., rothballer, c., & wyckmans, a. (2017). supporting municipal greenhouse gas (ghg) emission inventories using business process modeling: a case study of trondheim municipality. in business process management workshops: bpm 2016 international workshops, rio de janeiro, brazil, september 19, 2016, revised papers 14 (pp. 416-427). springer international publishing. doi.org/10.1007/978-3-319-58457-7_30 39 anjaria, k. (2024). enhancing sustainability integration in sustainable enterprise resource planning (serp) system: application of transaction cost theory and case study analysis. international journal of information management data insights, 4(2), 100243. doi.org/10.1016/j.jjimei.2024.100243 40 magdaleno, a. m., duboc, l., & betz, s. (2017). how to incorporate sustainability into business process management lifecycle?. in business process management workshops: bpm 2016 international workshops, rio de janeiro, brazil, september 19, 2016, revised papers 14 (pp. 440-443). springer international publishing. doi.org/10.1007/978-3-319-58457-7_32 41 seid, e., popov, o., & blix, f. (2024). evaluation of asfalia, a security attack event monitoring framework. procedia computer science, 237, 793-802. doi.org/10.1016/j.procs.2024.05.167 42 graves, n., koren, i., & van der aalst, w. m. (2023, june). rethink your processes! a review of process mining for sustainability. in 2023 international conference on ict for sustainability (ict4s) (pp. 164-175). ieee. doi.org/10.1109/ict4s58814.2023.00025 43 fortunati, s., morea, d., & mosconi, e. m. (2020). circular economy and corporate social responsibility in the agricultural system: cases study of the italian agri-food industry. zemedelska ekonomika, 66(11), 489498. doi.org/10.17221/343/2020-agricecon 44 hasan, m. m., nekmahmud, m., yajuan, l., & patwary, m. a. (2019). green business value chain: a systematic review. sustainable production and consumption, 20, 326-339. doi.org/10.1016/j.spc.2019.08.003 45 parmiggiani, e., & monteiro, e. (2018). shifting baselines? recommendations for green is. in proceedings of icis. 46 betz, s., & caporale, t. (2014). sustainable software system engineering. in 2014 ieee fourth international conference on big data and cloud computing (pp. 612-619). ieee. doi.org10.1109/bdcloud.2014.113 47 loeser, f. (2013). green it and green is: definition of constructs and overview of current practices. proceedings of amcis. 48 seidel, s., & recker, j. (2012). implementing green business processes: the importance of functional affordances of information systems. in proceedings of acis. table a1: literature sample (07/2023, updated 08/2024) australasian journal of information systems schoormann & di maria 2024, vol 28, research article socially business process patterns 33 appendix b: industry publisher of report year of report automotive bmw group, https://www.bmwgroup.com/content/dam/ grpw/websites/bmwgroup_com/responsibility/downloads/de/2019/2019bmw-group-svr-2018-deutsch.pdf, access: 2020/05/11. 2018 volkswagen ag, https://www.volkswagenag.com/presence/ nachhaltigkeit/documents/sustainabilityreport/2018/nichtfinanzieller_bericht_2018_d.pdf, access: 2020/05/11. 2018 continental ag, https://www.continental.com/resource/ blob/177190/37853d57eddfa3f46e64425352ebb552/nachhaltigkeitsbericht2018-data.pdf, access: 2020/05/11. 2018 manufacturing siemens ag, https://www.siemens.com/investor/pool/de/ investor_relations/siemens_nachhaltigkeitsinformationen2018.pdf, access: 2020/05/11. 2018 robert bosch gmbh, https://assets.bosch.com/media/global/ sustainability/reporting_and_data/2018/bosch-nachhaltigkeitsbericht-2018factbook.pdf, access: 2020/05/11. 2018 chemicals basf group, https://bericht.basf.com/2018/de/serviceseiten/ downloads/files/basf_bericht_2018.pdf, access: 2020/05/11. 2018 bayer ag, https://www.investor.bayer.de/securedl/16789, access: 2020/05/11. 2018 electronic ibm corporation, https://www.ibm.org/static/responsibility/ cr/pdfs/ibm-2018-crr.pdf, access: 2020/05/11. 2018 cisco systems inc., https://www.cisco.com/c/dam/assets/ csr/pdf/csr-report-2018.pdf, access: 2020/05/11. 2018 canon inc., https://global.canon/en/csr/report/pdf/ canon-sus-2018-e.pdf, access: 2020/05/11. 2018 nutrition lidl dienstl. gmbh & co. kg, https://www.lidlnachhaltigkeit.de/fileadmin/downloads/lidl_nachhaltigkeitsbericht_20162017.pdf, access: 2020/05/11. 2016/2017 metro ag, https://reports.metroag.de/corporate-responsibility-report/20172018/servicepages/downloads/files/ entire_metro_crr1718.pdf, access: 2020/05/11. 2017/2018 building hochtief ag, https://www.hochtief.de/hochtief/ mmdbdownload?id=173421&format=4, access: 2020/05/11. 2018 strabag se, https://www.strabag.com/databases/internet/_public/files.nsf/searchview/6 c0a043cac95207fc12583ed0031fad9/$file/strabag_se_d_gb_2018_ web.pdf?openelement, access: 2020/05/11. 2018 finance deutsche bank ag, https://www.db.com/ir/de/download/deutsche_bank_nichtfinanzieller_be richt_2018.pdf, access: 2020/05/11. 2018 allianz group, https://www.allianz.com/content/dam/ onemarketing/azcom/allianz_com/investorrelations/en/results/2018_q4/en-190411-sustainability-report-2018.pdf, access: 2020/05/ 2018 commerce otto (gmbh & co kg), https://www.ottogroup.com/media/ docs/de/geschaeftsbericht/otto_group_geschaeftsbericht_2018_19_de.pdf, access: 2020/05/11. 2018/2019 bertelsmann se &­co. kgaa, https://www.bertelsmann.de/ media/verantwortung/downloads/deutsch/bertelsmann-corporateresponsibility-bericht-2017-de.pdf, access: 2020/05/11. 2017 australasian journal of information systems schoormann & di maria 2024, vol 28, research article socially business process patterns 34 industry publisher of report year of report energy uniper se, https://ir.uniper.energy/download/companies/uniperag/sustainability/uni per_nachhaltigkeitsbericht_2018.pdf, access: 2020/05/11. 2018 e.on se, https://www.eon.com/content/dam/eon/eoncom/documents/en/sustainabilityreport/eon_sustainability_report_2018.pdf, access: 2020/05/11. 2018 tourism tui group, https://www.tuigroup.com/damfiles/default/tuigroup15/de/nachhaltigkeit/berichterstattungdownloads/2019/nachhaltigkeitsbericht-de-en/tui_csr18_de.pdf91be99e0bc67220ef211280d05702d2a.pdf, access: 2020/05/11. 2018 lufthansa group, https://www.lufthansagroup.com/media/downloads/en/responsibility/lhsustainability-report-2018.pdf, access: 2020/05/11. 2018 telecommunicat ion deutsche telekom ag, https://bericht.telekom.com/geschaeftsbericht2018/serviceseiten/downloads/files/entire_dtag_gb18.pdf, access: 2020/05/11. 2018 vodafone group plc., https://www.vodafone.com/content/dam/vodcom/sustainability/pdfs/susta inablebusiness2018.pdf, access: 2020/05/11. 2018 figure a2: demonstration cases appendix c: design iterations first design iteration (grounding) based on the obtained process patterns from the literature as well as on the problems and best practices from the experts, we inductively derived our preliminary set of process patterns for improving social sustainability (schoormann et al. 2019). pattern title and description problem solution development social compensation p1 s1, s2 implicit improving social performance by compensating negative effects without changing the structure of a process. for example: compensating internal effects through hiring health consultants or providing free access to gyms; compensating external effects through establishing foundations, making donations, or offering employees participation in social projects. social alternative p1 s3 implicit improving social performance by offering an additional alternative process path to the customers without changing the (core) business processes. for example: integrating a fair-traded product in addition to a basic product and allow customers to select between two variants social resource replacement p1, p2, p3 s4 implicit improving social performance by replacing resources that have negative influence on employees (internal) and society (external) without changing the structure of a process. for example: ensuring ergonomic working spaces by replacing typical office furniture with height-adjustable desks. social labeling p4 s5 implicit improving the social image of an organization by labeling processes, products, and services with corresponding social certificates. for example: certificating offers with the fairtrade label to disclose characteristics of the production and communicate certain aspects. social sourcing p5 s6 implicit australasian journal of information systems schoormann & di maria 2024, vol 28, research article socially business process patterns 35 improving social performance by (a) transferring activities to external organizations that are able to carry out an activity in a socially acceptable manner or by (b) transferring external activities into the own company. for example: outsourcing processes to a fairtrade label-certified service provide. social value chain p6 s7, s8 implicit improving social performance by defining social standards that have to be applied by every partner within a value chain. for example: specifying checklists with socially acceptable criteria that have to be passed by each partner within a value chain—verification of this through audits. social-/human-centered individualization p7 s9 explicit improving social performance by adjusting processes in terms of human-specific needs in order to allow various people to execute a certain process. for example: enabling the integration of people with handicaps through individualized working conditions to leverage his/her talents. figure a3: design iterations second design iteration (validation through demonstration, evaluation 1) after applying the initial pattern set to real-world cases, we identified the potential for new patterns (*) as well as for improving and extending existing patterns (**). (1) social compensation ** internal social compensation: improving internal social performance by compensating negative effects without changing the structure of a process. for example: health—preventive health care [tui18, p. 70]; nutrition— information for nutrition [dt18, p. 82]; employment protection [si18, p. 20]. external social compensation: improving external social performance by compensating negative effects without changing the structure of a process. for example: financial donations [bas18, p. 40]; social projects—scholarships for women [ibm18, p. 36]. (2) social alternative internal social alternative*: improving social performance by offering an additional alternative process path to the employees/partners without changing the (core) business processes. for example: flexible work—flexible working hours [lu18, p. 68]; remote work—homeworking policies [vo18, p. 17]. external social alternative: improving social performance by offering an additional alternative process path to the customers without changing the (core) business processes. (3) social resource handling ** social resource replacement: improving social performance by replacing resources (i.e., working aids and materials) that have a negative influence on internal and external actors without changing the structure of a process. for example: hiring ergonomic-teams that ensure healthy working places [co18, p. 33]; materials— avoiding so-called ‘conflict materials’ [si18, p. 39]. social resource reuse*: improving social performance by reusing resources (i.e., circular-principle) that have a special value for society. for example: effective management of water is important if projects are performed in regions that have water scarcity [ho18, p. 138]. (4) social labeling improving the social image of an organization by labeling processes, products, and services with corresponding social certificates. for example: fairtrade [li16, p. 88]; global sustainable tourism council [tui18, p. 11]; blauer engel [dt18, p. 83]; cmia-certified cotton [ot18, p. 83]. (5) social sourcing improving social performance by (a) transferring activities to external organizations that can carry out an activity in a socially acceptable manner or by (b) transferring external activities into the own company. australasian journal of information systems schoormann & di maria 2024, vol 28, research article socially business process patterns 36 (6) social value chain improving social performance by defining social standards that have to be applied by every partner within a value chain. for example: new supplier needs to develop a social/ecological management system [ibm18, p. 18]. (7) social-/human-centered accessibility** internal social accessibility: improving social performance by adjusting processes in terms of human-specific needs to allow employees/partners regardless of their background (e.g., gender, religion, physical health) to participate in a certain process. for example: accessible working environments including lifts or subtitles that can be read aloud when using a computer [si18, p. 18]. external social accessibility*: improving social performance by adjusting processes in terms of human-specific needs to allow customers/society regardless of their background (e.g., gender, religion, physical health) to participate in a certain process. for example: accessible stores [li16, p. 113]. (8) social training* improving social performance by educating internal and external actors to enable them to participate in processes appropriately. for example: competence management ensures that employees have the required knowledge to execute/participate in their tasks [rb18, p. 60]. (9) social togetherness* improving social performance by creating a strong team-cohesion within an organization or company/department. for example: ‘walkathon challenge’ to support scholarships for women [ibm18, p. 36]; carpools [rb18, p. 58]. * new pattern; ** refined patterns figure a4: second design iteration third design iteration (applicability check, evaluation 2) after the collection of data on the importance, accessibility, and applicability (rosemann & vessey, 2008) of the patterns. as a consequence, we decided to split the initial pattern on ‘resource handling’ into more concrete ones for resource replacement and resource reuse, as well as extend the description of examples provided for each pattern. pattern [#1] social compensation improving internal and external socially sustainable performance by compensating negative effects without changing the (core) process structure. for example: ▪ preventive health care and employment protection/work safety (internal) ▪ financial donations to social projects (external) ▪ establishment of social projects/redistribute revenues, such as scholarships (external) + [#2] social alternative improving socially sustainable performance by offering an alternative process path to the employees, partners, and customers without changing the (core) process structure. for example: ▪ remote work and flexible working hours (employee decision, internal) ▪ job rotation against monotonous tasks (staff decision, internal ) ▪ fair products in addition to the standard assortment (customer decision, external) x [#3] social resource replacement improving social performance by replacing resources (aids and materials) that have a negative influence on internal and external actors without changing the (core) process structure. for example: ▪ hiring of ergonomic-teams to ensure healthy working places ▪ elimination of so-called ‘conflict materials’ [#4] social labeling improving the socially sustainable image of an organization by labeling processes and products with corresponding social certificates and standards. for example: ▪ compliance with fair trade, cmia-certified cotton ▪ compliance with criteria from global sustainable tourism council [#5] social sourcing improving socially sustainable performance by transferring activities to external organizations that can carry out an activity in a socially responsible manner or – vice versa – transferring external activities into the own company that is trained in handling certain activities. australasian journal of information systems schoormann & di maria 2024, vol 28, research article socially business process patterns 37 pattern [#1] social compensation improving internal and external socially sustainable performance by compensating negative effects without changing the (core) process structure. for example: ▪ preventive health care and employment protection/work safety (internal) ▪ financial donations to social projects (external) ▪ establishment of social projects/redistribute revenues, such as scholarships (external) + [#2] social alternative improving socially sustainable performance by offering an alternative process path to the employees, partners, and customers without changing the (core) process structure. for example: ▪ remote work and flexible working hours (employee decision, internal) ▪ job rotation against monotonous tasks (staff decision, internal ) ▪ fair products in addition to the standard assortment (customer decision, external) x [#3] social resource replacement improving social performance by replacing resources (aids and materials) that have a negative influence on internal and external actors without changing the (core) process structure. for example: ▪ hiring of ergonomic-teams to ensure healthy working places ▪ elimination of so-called ‘conflict materials’ [#4] social labeling improving the socially sustainable image of an organization by labeling processes and products with corresponding social certificates and standards. for example: ▪ compliance with fair trade, cmia-certified cotton ▪ compliance with criteria from global sustainable tourism council [#5] social sourcing improving socially sustainable performance by transferring activities to external organizations that can carry out an activity in a socially responsible manner or – vice versa – transferring external activities into the own company that is trained in handling certain activities. [#6] social value chain improving socially sustainable performance by defining and monitoring socially sustainable standards that have to be applied by every partner within a value chain. for example: ▪ supplier needs to have a management system to report on compliance with safety at work § [#7] social resource reuse improving socially sustainable performance by reusing resources (e.g., circularity thinking instead of linearity) that have a special value for society. for example: ▪ effective water management if (parts of the) processes are performed in regions with water scarcity [#8] social accessibility improving socially sustainable performance by adjusting processes in terms of individual, human-specific needs to allow employees, partners, and customers regardless of their background (e.g., gender, religion, physical health) to participate in a certain process. for example: ▪ subtitles that can be read aloud when using technology (internal, external) ▪ accessible stores with lifts so that everyone can participate (external) [#9] social training improving socially sustainable performance by educating internal and external actors to enable them to participate in processes appropriately. for example: ▪ competence management to ensure that employees have the required knowledge for their tasks ▪ awareness, education, and training to handle hazardous material [#10] social togetherness improving socially sustainable performance by creating a strong teamcohesion within an organization and doing something good together. for example: ▪ ‘walkathon challenge’ to support scholarships ▪ establishment of carpools + + activity …resource gateway actor flow association australasian journal of information systems schoormann & di maria 2024, vol 28, research article socially business process patterns 38 figure a5:third design iteration appendix d: selected definitions category description process pattern “process patterns are proven solutions to recurring situations and are popular artefacts within business process management.”; “are atomic building blocks for business processes and aim to trigger specific considerations among stakeholders dealing with processes […]” (rosemann et al., 2023) “ambler (1998) introduced the term process patterns aiming to describe best practices in software engineering processes” (fellmann et al., 2018) process design pattern “the purpose of explorative design patterns is to increase the awareness of a process designer for the existence of growth opportunities leading to a larger process design space.”; “support the identification of options to create new value from existing business processes.” (rosemann, 2020) “[…] such techniques seem more hit or miss than a set of structured heuristics that would guide the redesigner, or at least provide some tools for process transformation” (zellner, 2013) business process design guideline “[…] design guidelines refer to higher order principles and are motivated by a design intention” (rosemann et al., 2023) “patterns as atomic and often (semi) formalised templates can be a part of a design guideline.“(rosemann et al., 2023) business process model pattern “a business process model pattern is the description of a proven solution to a recurring problem that is related to the creation or modification of business process models in a specific context. this description is typically organized in a structured document supporting the reader in understanding under which circumstances the proposed solution will be useful.” (fellmann et al., 2018). business process improvement (bpi) pattern “[…] we refer to process improvement patterns (pips) as generic concepts for enhancing particular aspects of business processes.”; “pips as practices” (lohrmann & reichert, 2016) “we define a bpi pattern as a reusable solution for a certain problem in a business process within a certain context. […] the application of a bpi pattern thereby results in some form of change, e.g. on the activities […] or the required resources for its execution.” (falk et al., 2013) “one advantage of patterns is that they provide precise operational guidelines while at the same time being flexible enough for being applied in different contexts.” (falk et al., 2013) process redesign heuristics “[…] common tool used for process improvement and innovation are heuristics. generally, heuristics are cognitive shortcuts that facilitate the exploration of a solution space […] (re)design heuristics specifically help create innovative ideas when redesigning existing or designing new business processes (recker and rosemann, 2015).” (frank et al., 2020) design practices “although an ideal best practice prescribes the best way to treat a particular problem that can be replicated in any situation or setting, it is more fruitful to see it as something that “needs to be adapted in skillful ways in response to prevailing conditions”; “[…] are universal in the sense that they are applicable within the context of any business process, regardless of the product or service delivered.” (reijers & mansar, 2005) process antipattern / weakness pattern “process anti-patterns (aka weakness patterns [18, 19]) describe insufficient solutions and are recognized by the appearance of failures.” (rosemann, 2020) figure a6:selected definitions acknowledgements the authors greatly appreciated the comments of the editors and reviewers, which enabled us to substantially improve the paper’s argument and presentation. moreover, we like to articulate our thanks to the experts and their valuable inputs for grounding and evaluating this research endeavor. australasian journal of information systems schoormann & di maria 2024, vol 28, research article socially business process patterns 39 copyright copyright © 2024 schoormann & di maria. this is an open-access article licensed under a creative commons attribution-non-commercial 4.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v28.4949 https://creativecommons.org/licenses/by-nc/4.0/deed.en paper title (paper title style) australasian journal of information systems kostin & strode 2023, vol 27, selected papers from acis 2022 effective communication in globally distributed scrum 1 effective communication in globally distributed scrum: a model and practical guidance dmitrii kostin independent researcher, wellington, new zealand dmitrii.kostin.nz@gmail.com diane strode whitireia polytechnic, porirua, new zealand abstract a trend in information systems development is for globally distributed teams to use agile methods and frameworks such as scrum. in globally distributed (gd) software development, a known challenge is effective team communication. researchers, however, cannot evaluate effective communication in gd teams using scrum unless they know what effective communication means in that context. this qualitative study contributes a theoretical model of effective communication in gd scrum teams and practical guidance for practitioners. ten industry professionals working in gd scrum teams were interviewed to capture their understanding of effective communication. qualitative content analysis was used to analyse the interviews and form a basis for the model and the practical guidance. this novel model consists of communication transparency, communication quality, and communication discipline, which together lead to the alignment of team understanding (i.e., a team-level shared mental model). this theoretical model lays the ground for future research into the effect of scrum practices on communication in gd contexts, and the effect of communication on team and project success. for practitioners, this study contributes 11 practical actions that professionals recommend for improving and sustaining effective communication. keywords: agile software development, communication transparency, communication quality, communication discipline, global software development, shared mental model, team mental model. 1 introduction information systems development often involves globally distributed (gd) software development (drechsler & breth, 2019; herbsleb & mockus, 2003) together with the use of agile methodologies (also called methods) and frameworks (baham & hirschheim, 2022; maruping & matook, 2020). scrum is the most commonly adopted agile framework (digital.ai, 2022; stavru, 2014) and using scrum in gd software development is increasingly reported (hidayati et al., 2020; marinho et al., 2021). effective communication is one of the foundations of success in systems and software development (defranco & laplante, 2017). agile frameworks such as scrum require intensive communication (alzoubi et al., 2016; hummel et al., 2013). traditionally, scrum uses face-toface interactions in small co-located teams with direct face-to-face communication among team members and customers to facilitate successful development work (strode et al., 2022). in gd agile teams, many of them using scrum or its variants, face-to-face communication is limited and is known to be problematic (alzoubi et al., 2016; bundhun & sungkur, 2021). systematic literature reviews (slr) of agile research identify a lack of research on the social and behavioural aspects of agile development, particularly communication (diegmann et al., 2018; australasian journal of information systems kostin & strode 2023, vol 27, selected papers from acis 2022 effective communication in globally distributed scrum 2 hoda et al., 2017). an slr by alzoubi et al. (2016) of geographically distributed agile development, identified many communication challenges and mitigation practices but report a significant lack of knowledge about how efficient and effective communication is achieved in practice in agile contexts. they also highlighted the lack of theory development to explain communication in geographically distributed agile development. to understand how scrum improves communication in gd teams, researchers must first understand what ‘effective communication’ means in this context. improving the understanding of effective communication in gd scrum teams by developing theoretical models of this phenomenon should help researchers better understand effective communication in gd scrum and help identify its antecedents and consequences. that is, they can use this knowledge to identify scrum practices, combinations of practices, tools that support effective communication, and the precise effect of communication on outcomes such as team and project success. effective communication is well understood in small teams and virtual teams (marlow et al., 2018). communication practices in agile contexts have been explored (hummel et al., 2013) and communication in co-located scrum is understood to be a significant factor in supporting effective teamwork (strode et al., 2022). studies of communication in gd scrum, however, are rare and seldom define ‘effective’ communication, rather they identify the communication challenges, communicative agile practices, factors, and tools that contribute to communication (alzoubi et al., 2016; amar et al., 2019; hummel et al., 2013). we located one case study of gd scrum that mentions that scrum improved communication in a large-scale distributed environment (paasivaara et al., 2008). therefore, to better understand effective communication in gd scrum teams and provide a basis for future research evaluating scrum’s effect on communication in gd contexts, we posed this research question: what is effective communication in globally distributed scrum teams? we answered this question using interviews and qualitative content analysis to explore the experiences of practitioners involved in gd scrum teams regarding their communication. from this analysis two contributions emerged. the theoretical contribution is an empirically based model of gd scrum team communication effectiveness comprising communication transparency, communication quality, communication discipline, and alignment. the practical contribution is 11 pieces of advice raised in the interviews that gd scrum practitioners can adopt to improve and sustain effective communication. this article extends a related conference paper by kostin and strode (2022). the article is organised as follows. in section 2, we define communication and give a brief overview of communication in general. this is followed by subsections on communication in software development, in gd teams and in gd scrum teams. in section 3, we explain the method including, sampling, data collection, analysis method, and validity measures, and provide participant profiles. section 4 presents a model of effective communication in gd scrum teams, and section 5 presents practical guidance on how practitioners sustain and improve effective communication. section 6 answers the research question and explains how the model contributes to existing knowledge on effective communication in gd scrum teams and to industry practice. limitations are addressed followed by considerations of future work. section 7 concludes. australasian journal of information systems kostin & strode 2023, vol 27, selected papers from acis 2022 effective communication in globally distributed scrum 3 2 literature review: communication the discipline of communication emerged in the 1950s and is studied in disciplines such as sociology, psychology, geography, and technology among others (pooley, 2016). communication is studied at different levels, modes, and channels. communication can be interpersonal (berger, 2005), group and team (marlow et al., 2018), and organisational level (cooren et al., 2011). modes can be synchronous and asynchronous, and channels include faceto-face, electronic messaging, and by documents (ahmad et al., 2018). in addition, the purpose of communication can be to convey information or to converge on shared meanings, as dennis et al. (2008) explain in media synchronicity theory. in organisations, communication genres occur. yates and orlikowski (2002) theorised six dimensions of communicative interaction that contribute to a genre: purpose, content, participants, form, time, and place. genres occur when people assemble these communicative interactions into sequences that become embedded over time as organisational norms. due to this broad spread of communication studies and theories, we restrict this review to group and team communication. even with this narrower focus, there are still many team and group communication theories and models (defranco & laplante, 2017; harris & sherblom, 2018; marlow et al., 2018; poole, 1999). a meta-analysis on the effect of team communication on team performance by marlow et al. (2018, p.146) defines communication as “an exchange of information, occurring through both verbal and nonverbal (e.g., email) channels, between two or more team members”. they note that team communication is typically measured by the degree to which team members receive clear information from team members, how frequently team members interact, and the extent of knowledge sharing among team members. they found a positive and significant relationship between team communication and team performance when they tested the relationship by analysing results from communication studies in business and psychology (150 studies published between 1966 and 2016). they conceptualised team communication as having two factors, communication quality and communication frequency. communication quality is “the extent to which communication, both of a verbal and nonverbal nature, adequately distributes pertinent information among team members as needed” (marlow et al., 2018, p.148), and communication frequency as the volume of communication between team members. their results showed that 1) communication quality has a significantly stronger relationship with performance than communication frequency, 2) as familiarity (i.e., the level of knowledge teams have of one another) increases, team communication becomes more strongly related to team performance, and 3) face-to-face teams show a stronger relationship between communication and performance than virtual teams. another result was that information elaboration and knowledge sharing had a stronger relationship with performance than the other communication measures in the study (e.g., content or openness). the results are based on studies of various team types (e.g., surgical, search-and-rescue, and management), so the extent to which these results apply to software teams is not clear. software development teams, their tasks, and the products they create may have unique characteristics leading to different results. in addition, since 2016, when the most recent study was published, the variety and quality of tools for collaborating and communicating in virtual software development environments have advanced considerably (jackson et al., 2022), which could also lead to different results regarding communication. australasian journal of information systems kostin & strode 2023, vol 27, selected papers from acis 2022 effective communication in globally distributed scrum 4 2.1 communication in software development team communication is crucial in successful software development because software development groups and teams must communicate effectively to integrate complex technical and knowledge work. communication is necessary, not just within software teams, but also between teams, and between teams and other parts of the organisation. for successful software development, communication is also needed to interact efficiently and effectively with stakeholders, because they are key informants in the software development process (defranco & laplante, 2017). communication in software development is identified as a specific research gap by defranco and laplante (2018) in a mapping study of software engineering teamwork research. their study of 103 papers published from 1999 to 2016 identified two main gaps in knowledge: communication in agile contexts and tools to support communication in gd software engineering. communication studies in software development take various perspectives. communication gaps were studied by bjarnason et al. (2016). they analysed five cases (5 companies and 15 interviews). the cases varied, some were using agile methods, and some had distributed work arrangements. the analysis showed that communication is more effective and efficient when distances are decreased. the distances they evaluated were geographical, organisational, psychological, cognitive, adherence, semantic, navigational, and temporal. communication genres were used by mcchesney and gallagher (2004) to explore communication and coordination practices in two geographically distributed projects in the uk. they found that maintaining team awareness of each other’s roles and work activities, and a shared understanding of the project environment contributed to coordination and communication. they also identified various practices that enhanced communication such as establishing a network of knowledgeable staff, query-handling procedures using email, and the use of scm (software configuration management) tools. knowledge about agile practices in scrum and xp (extreme programming) that have a communicative function in collocated settings is well established (ahmad et al., 2018; mishra et al., 2012; pikkarainen et al., 2008). pikkarainen et al. (2008) analysed two cases (one organisation with two collocated projects) and identified the practices as open office space, daily meetings, task boards on the wall, sprint planning, reflective workshops, pair programming, and continuous integration. they found agile practices enhanced formal and informal communication in teams but could hinder communication in a larger project with multiple external stakeholders. the agile practice of setting up an appropriate open office layout was the focus of mishra et al. (2012) who identified various physical office layouts that were more or less effective in supporting communication. a more recent systematic mapping study was undertaken by ahmad et al. (2018). including 25 primary studies, the focus was on communication channels and agile practices that support effective communication. this mapping study had some weaknesses; theoretical papers were specifically excluded, and most papers published in information systems were excluded because the source databases searched were software engineering oriented. as most research on agile software development appears in either software engineering or information systems publications, omitting one domain considerably reduces the mapping study coverage. the included papers also had to report communication processes, communication channels, and best practices, which may have further reduced the included papers. the results showed that people-oriented processes are more effective, and that communication channels are synchronous (e.g., face-toaustralasian journal of information systems kostin & strode 2023, vol 27, selected papers from acis 2022 effective communication in globally distributed scrum 5 face, video conference) and asynchronous (e.g., email, documents). the practices they identified for communication match those of pikkarainen et al. (2008) (listed above in this section) except for test-driven development and refactoring. these studies on communication in software development focus on gaps, genres, and practices. they imply that the outcome of reducing gaps and identifying genres and practices is more effective communication. although these studies show evidence that certain agile practices, genres of communication, and communication tools improve communication, no explicit definitions, or models of effective communication have emerged. 2.2 communication in globally distributed software development in globally distributed projects and teams, communication is critical and is a recognised challenge because physical, in-person, face-to-face interaction is rarely possible (carmel & agarwal, 2001; giuffrida & dittrich, 2015). gibson & gibbs (2006) argue that communication challenges among virtual teams are caused by different native languages, different national and organisational cultures, time differences (i.e., different time zones and different work schedules at different sites), and geographical distance. for example, team members may lack adequate english-speaking skills (english is the language used in software development worldwide), cultural differences can embarrass and cause misunderstandings between people, and the difference in time zones can make it challenging to participate effectively in meetings due to biological sleep needs. there is evidence that software development teams working in the same office are more efficient than distributed teams as espinosa and carmel (2003) showed when studying coordination costs with mathematical modelling. this reduced efficiency may be because online interaction is often replaced with face-to-face interaction when teams are collocated. technologies such as online video conferencing are popular and convenient to enable virtual face-to-face interaction and meetings, but this technology may impede the spontaneity and richness of the physical face-to-face interaction needed for novel and complex tasks such as software development (dennis et al., 2008). the problem with this claim about face-to-face communication is that technology for communication has advanced since these studies were published and the barrier of distance may now be somewhat reduced (jackson et al., 2022), although the barriers of time zone, culture, and language differences remain. a lack of face-to-face communication can impact knowledge sharing in global software development. anwar et al. (2019) reported five categories of knowledge-sharing factors (barriers and facilitators) affecting global software development: individual, technological, organisational, cultural, and geographical. their slr of 42 knowledge-sharing studies from 2010 to 2017, identified 22 barriers, those mentioned with the highest frequency were poor organisational culture, language differences, differences in cultural norms, and technological knowledge gaps. they also reported 20 knowledge-sharing facilitators, those with the highest frequency were organisational support, technological support, team communication, and centralised software libraries. although anwar et al. (2019) assembled the knowledge-sharing factors that influence communication, they did not define effective communication in the gd software development context. factors that lead to effective communication in gd software development were presented in a model developed by bhatti and ahsan (2017). based on the perceptions of 193 respondents from 35 gd software development organisations, the factors included stakeholders’ australasian journal of information systems kostin & strode 2023, vol 27, selected papers from acis 2022 effective communication in globally distributed scrum 6 involvement (where stakeholders are external parties providing requirements), acculturation, usage of appropriate tools and technology, and information availability. rather than defining the components of effective communication, their model focuses on practices that enhance communication. in addition, the applicability of the model to gd scrum teams is not clear because the extent to which the respondents were following scrum or other agile methods was not reported. 2.3 communication in globally distributed scrum teams research on communication in globally distributed scrum teams follows a similar pattern to that of distributed software development in general. communication studies of scrum projects and teams focus on communication challenges, practices for achieving communication, and a limited number of studies attempt theory development. the published history of the agile framework scrum starts in 1995 (schwaber, 1997) and scrum is currently the most popular agile framework for software development worldwide (digital.ai, 2022). originally referred to as an agile method (or methodology), scrum is usually referred to as a framework. the scrum framework is described fully in the founder’s book (schwaber & beedle, 2002) and more recently in the scrum guide (schwaber & sutherland, 2020). these guides describe scrum for small co-located projects and teams. for large-scale software development, variants of scrum are available such as safe (scaled agile framework), less (large-scale scrum), and sos (scrum of scrums) (ebert & paasivaara, 2017). no scrum-based framework is specifically designed for distributed or globally distributed development although scrum is now adopted in that environment (vallon et al., 2018). the agile manifesto states the values and principles of agile methods, and is concerned with conveying information, stating, “the most efficient and effective method of conveying information to and within a development team is face-to-face conversation” (beck et al., 2001b). in addition, many scrum practices are designed to facilitate effective team communication. colocation is recommended to support unscheduled group and one-to-one communication, and frequent regular meetings such as sprint planning, sprint reviews, retrospectives, and daily stand-up meetings ensure that the whole team is aware of the project and product status (schwaber & beedle, 2002). in gd scrum these practices are also followed but with electronic media facilitating the communication across distances (yagüe et al., 2016). scrum in globally distributed teams and projects may improve communication but the evidence is equivocal. a single case study by paasivaara et al. (2008) focusing on how a company applied scrum practices in their large-scale distributed software product development program between norway and malaysia, reports that scrum improved communication. the study concluded that “an agile method, like scrum, suits well to distributed settings and actually helps in solving the biggest problem of gsd projects, namely communication, by almost forcing distributed team members to communicate frequently and really learn to communicate” (paasivaara et al., 2008, p.93). others, such as alzoubi et al. (2016), are less sure, saying, after an extensive systematic literature review, “it is still arguable whether agile practices can be effectively scaled up and used in gdad environments due to communication challenges.”1 1 gdad stands for globally distributed agile development. australasian journal of information systems kostin & strode 2023, vol 27, selected papers from acis 2022 effective communication in globally distributed scrum 7 communication challenges in gd software development teams were identified by dorairaj et al. (2011) based on the experiences of 18 practitioners working on 14 projects using scrum (often with xp). they found the lack of effective communication was a key challenge caused by a lack of tools, time zone differences, language barriers, and a lack of teamwork. to increase effective communication, they found that each of these issues needs to be addressed along with increasing formal and informal communication, and building trust between the distributed parts of the team. as noted above, some of these challenges may now be less problematic as tools for communication across distances have increased in number and improved in quality since 2011 (jackson et al., 2022; stray & moe, 2020). more recently, alzoubi et al. (2016) carried out an slr of communication challenges that included distributed and globally distributed agile software development. that study captured knowledge in both software engineering and information systems. of 21 papers analysed, 33% reported on scrum and 53% reported on scrum and xp. the rigour of papers in the final 21 was evaluated. the study identified 17 factors that limit communication, thus confirming and extending the findings of dorairaj et al. (2011). this more recent list included time zone and geographic differences, team number and size characteristics, organisational, project, and customer characteristics, communication tools, culture and language issues, and trust. the study identified techniques (practices) used to overcome each of the limitations (e.g., sprint synchronisation, product demonstrations, appropriate tools for communication, and visiting remote sites). alzoubi et al. (2016) concluded that more empirical studies were needed to find what enables efficient and effective communication and that communication should be investigated from the perspective of theories such as activity theory and coordination theory. this was because the theoretical maturity of this topic is limited. the topic relies on exploratory studies that are considered less mature than theory-building or testing studies (edmondson & mcmanus, 2007). along with the practices identified in the slr of alzoubi et al. (2016), studies reporting the utility of specific scrum practices identify daily scrum meetings and slack™ (a messaging application) as important for supporting communication in agile global software development. stray et al. (2013), in a case study of multiple teams, found that daily scrum meetings are critical to ensure a project is completed because they allow the team to stay in touch, assist each other, conduct their tasks, and discuss problems. a single-case study of agile global software development by stray and moe (2020) reported that scheduled scrum meetings and scrum-of-scrum meetings, unscheduled meetings, and using the communication tool slack ™ facilitated communication. these few studies of gd scrum focus on communication challenges, practices, and tools, rather than defining or providing an understanding of what ‘effective’ communication means in gd scrum contexts. theory concerning communication in gd scrum is limited as noted by alzoubi et al. (2016) but some theories have emerged. amar et al. (2019) developed the 5c model, a theory of communication for scrum-based distributed projects consisting of components that influence communication: competency, correlations, contentment, comprehension, and commitment. the model was developed from 25 interviews conducted in various geographical regions. the 5c model consists of 15 actions, practices, and factors that contribute to communication such as knowledge sharing, planning, scheduling, motivation, social interaction and trust, and creativity, skills, and abilities. the 5c model is concerned with the antecedents to effective communication and does not explicitly define ‘effective communication’. similarly, a conceptual framework developed by giuffrida and dittrich australasian journal of information systems kostin & strode 2023, vol 27, selected papers from acis 2022 effective communication in globally distributed scrum 8 (2015) found that communicative and coordinative practices are mediated by social software in gd teams. the basis of the framework was evidence from two software development cases (one industry case using iterative development, and one of student teams, with only one team using scrum). they identified communication and coordination factors such as team building, situated articulation, meta-work, and knowledge sharing, but communication effectiveness was not a factor in this model. communication effectiveness is recognised by alzoubi and gill (2020). they tested the relationship between agile enterprise architecture and performance in geographically distributed agile software development. the mediating variable was ‘active communication’, which was composed of communication effectiveness and communication efficiency. the questionnaire data was collected in 2015 from 160 respondents worldwide who were involved in agile enterprise architecture and distributed agile development teams. in this study, communication effectiveness was defined as “delivering a message as it was intended with minimal disruption and misunderstanding, even if it takes a long time” (alzoubi & gill, 2020, p. 80284) and communication efficiency as “sharing information and knowledge in a timely manner between distributed teams or distributed team members” (alzoubi & gill, 2020, p. 80971). the overall result was that agile enterprise architecture, communication efficiency, and communication effectiveness have a significant positive effect on the overall development performance. in addition, communication efficiency had a significant positive effect on communication effectiveness although the authors acknowledge that the reason for this relationship is not yet clear and needs further investigation. in their study, ‘communication effectiveness’ was modelled as a reflective construct with 4 items about task clarity, information sufficiency, detail, and accuracy. 27.5% of the respondents were team leaders/scrum masters but the extent of scrum usage among the respondents is not clear. we assume that the majority of respondents were using some form of scrum, given the worldwide level of scrum adoption is estimated at 87% (digital.ai, 2022). in addition, that study did not report the extent of global distribution as opposed to within-country distribution of the respondents’ teams. in summary, we found most studies of communication in globally distributed scrum teams focus on communication challenges and practices to improve communication, that is the antecedents of effectiveness. one study has established a definition and quantitative measurement items for communication effectiveness in globally distributed agile software development but the data collection and analysis for that study were carried out in 2015. therefore, further exploration of communication effectiveness with a more precise focus on empirical evidence from globally distributed scrum teams in the current technological environment is warranted. this study therefore aims to develop our understanding of communication effectiveness more fully as a basis for further theory building and testing and to provide specific advice for practitioners. 3 method this study aimed to understand effective communication in gd scrum teams. we chose indepth interviews to provide the data to answer the research question for two reasons. firstly, because of the limited theory or theoretical models on this specific sub-topic in communication (alzoubi et al., 2016), in-depth interviews could provide the detail and insights needed for theory-building about this complex topic (eisenhardt & graebner, 2007). secondly, in-depth interviews are also suitable for exploratory studies (edmondson & mcmanus, 2007). this australasian journal of information systems kostin & strode 2023, vol 27, selected papers from acis 2022 effective communication in globally distributed scrum 9 study was exploratory because the research question was thought to best be answered by talking to people about their recent subjective experiences and perspectives on communication in globally distributed scrum teams (bickman & rog, 2009) rather than by focusing on confirming existing constructs or extending models in the literature. 3.1 interview design a semi-structured interview schedule was designed to collect in-depth data. each interview question had a purpose that was related to the research question. appendix b lists each interview question and its purpose. the schedule was designed with open-ended questions to provide the participants with the chance to reflect on the topic and share their thoughts, beliefs, and experiences in an informal discussion format. the interview questions focused on how the participant defined effective communication and their experience of communication in gd scrum teams. the questions were also informed by the personal and professional experience of the researchers, along with, relevant academic and professional/practitioner (grey) literature including the following. • world of agile blog ( https://worldofagile.com/blog/distributed-scrum-teams/) • atlassian knowledge base ( https://www.atlassian.com/agile/scrum/distributed-scrum) 3.2 sample selection and recruitment ethics approval was granted by whitireia polytechnic before participant selection began. the sampling was purposive, that is each participant was selected according to specific criteria (miles & huberman, 1994). all participants had to have experience (at least 1 year) in coordinating communication in gd scrum teams. we found potential participants by identifying it professionals with suitable professional experience as displayed in their profile information on linkedin™. linkedin™ is a networking and career development website used by professionals. initially, four participants were selected using linkedin™ searches and were contacted by direct message. a further five participants were found by sharing a linkedin™ post across one researcher's linkedin™ network. the post briefly described the research and asked the network to help find suitable participants. one person was an indirect work contact of one researcher. no participants were known to the researchers before the study began. all participants were sent an invitation email or chat message with a link to a consent form to be signed and an information sheet describing the study. to confirm the participants’ experience was appropriate for the study and before proceeding to the interview, each participant completed a participant selection questionnaire (see appendix a). this short online questionnaire was developed using google forms. the questionnaire responses were used to confirm that participants had experience in organising or managing communication in gd scrum teams, they worked in or with a globally distributed scrum team, and currently or recently held roles with the responsibilities of scrum master, product owner, or agile coach or similar roles common in scrum (schwaber & sutherland, 2020). people with these roles were selected because they were more likely to have experience in organising communication in gd scrum teams (kristensen & paasivaara, 2021; srivastava & jain, 2017). the locations, positions, industries, and years of experience of the participants are shown in table 1. note that, for this study, a team was considered a scrum team if at least one scrum practice was used, and the team was considered a globally distributed scrum team if at least one team member was located overseas. these details were confirmed with the participant selection australasian journal of information systems kostin & strode 2023, vol 27, selected papers from acis 2022 effective communication in globally distributed scrum 10 questionnaire. in this article, we refer to a scrum team as one team working on one product. a team can have sub-teams that are ‘pieces’ of a team that are in different locations. 3.3 data collection the interview questions were sent to the participants shortly before the interview. the first author organised and carried out all the interviews. each interview followed the interview schedule but allowed for variation in responses. the 10 participants were interviewed over five weeks during april and may 2021 with interviews taking up to one hour. online interviews were used because close physical contact was discouraged due to the covid-19 pandemic at that time. this also meant interviewees could be in any location in the world and the interview conditions were similar for all participants. the zoom™ online interview sessions were recorded with the permission of the participants. 3.4 data analysis the interview data was transcribed from the recorded interviews using otter™ software (otter.ai). we used qualitative content analysis to analyse the transcripts (schreier, 2014) and microsoft word tables to organise the analysis. schreier (2014) explains that the content analysis method involves the systematic description of data through the development and application of a coding framework. the coding framework was first developed by checking every single part of the material that was relevant to the research question. based on guidelines by schreier (2014), the procedure we used consisted of the following steps. these steps were carried out by one researcher and the final framework was reviewed by another researcher. adjustments were discussed and agreed upon jointly. for available source data see kostin (2021). 1. collect material. this step included selecting participants, interviewing, transcribing, checking, and cleansing the transcripts of transcription errors. 2. build an initial coding framework. this framework was based on a first pass of reading carefully through the transcripts. based on one main category (i.e., communication), analytic codes were developed for each idea about communication found in the transcript. this step is illustrated in figure 1. 3. evaluate and modify the framework as the analysis progresses. this involved grouping the codes into categories of communication that were based on common themes identified by the researcher. each category was defined, and the transcripts reviewed for indicators (i.e., example quotes), and a check was made to ensure the categories were mutually exclusive. this step is illustrated in figure 1 and table 2. 4. perform the full analysis using the developed framework. this involved reviewing all of the transcripts against the categories in the framework, refining the framework if necessary, identifying and defining any new codes or categories, and collapsing categories if they were not substantial or mutually exclusive. 5. present and interpret the findings. 3.5 validity considerations validity in the qualitative data and findings was achieved by following the guidelines of creswell and poth (2016) and miles and huberman (1994) on triangulation, transparency, and rich data collection. triangulation validates that the data and findings are free of bias (i.e., australasian journal of information systems kostin & strode 2023, vol 27, selected papers from acis 2022 effective communication in globally distributed scrum 11 come from multiple sources) and include a variety of perspectives. triangulation was achieved by collecting data from 10 experienced professionals from nine companies in four countries in six relevant positions. figure 1. an illustration of the relationship between quotes, codes, and category transparency validates the study's conclusions. we aimed for transparency by explaining exactly how the research was organised and actioned in this method section, and how we transformed the raw data in the interview transcripts into the conclusions in the finding’s sections, sections 5 and 6. we have included extensive quotes from the transcripts to support transparency in developing the theoretical model and again in the practical guidance section. rich data collection validates that the data collected in the interviews were detailed enough to capture people's experiences and ideas. we aimed for rich data by using open-ended questions in the interviews and detailed coding of anything related to the research question. creswell and poth (2016) and miles and huberman (1994) also recommend member checks to validate that data is collected without bias. this involves sharing the transcribed text of an interview with the interviewee so they can check and confirm that the data accurately reflects what they said. we were not able to do this because of the high workload of participants who preferred not to carry out member checking, which they communicated to us during the planning of the interviews. we took several actions to reduce participant and researcher bias. to reduce bias in participant selection we used purposive participant selection with a pre-designed set of inclusion criteria for searching within the linkedin™ social media platform (see section 3.2). this was reinforced with a selection questionnaire to ensure the participants had appropriate recent knowledge and experience before they were interviewed (see appendix a). to reduce bias in data collection we followed a pre-designed semi-structured interview schedule (see appendix australasian journal of information systems kostin & strode 2023, vol 27, selected papers from acis 2022 effective communication in globally distributed scrum 12 b), so each participant was questioned on the same topics, and we allowed the time and flexibility for participants to fully describe their knowledge, experiences, and perceptions. to reduce researcher bias we took these actions. in the analysis phase we relied on automated transcription to fully transcribe all responses, but we also manually checked that the transcript accurately reflected the participant’s words. we followed the published procedure for qualitative content analysis (schreier, 2014), and analysed each transcript in the same way following that procedure. during the analytical coding step, the researcher sought to be open to all possible initial codes related to communication. although we took these steps to reduce researcher bias, qualitative research is inherently subjective and may be prone to bias (sarker et al., 2013) because participants may omit, over-emphasise, or downplay events or experiences. reassuring participants of confidentiality was one action taken to mitigate this problem, along with triangulation of sources. the problem that can occur when the researchers inadvertently influence the participant’s responses (i.e., reactivity bias (gorard, 2013)) was also controlled somewhat by following an interview schedule. 3.6 participant profiles the participant profiles are displayed in table 1. the participants had a range of senior roles in agile projects and their experience ranged from 1 to 5 years. they were engaged in projects concerned with it, financial services (fintech), and healthcare. the residency, team location, and headquarters indicate the global distribution of the participants and their scrum teams. residency headquarters team location position field experience p1 usa uk china, taiwan, uk, usa product manager fintech* 5+ p2 philippines ireland australia, philippines, usa senior project manager it 1+ p3 denmark denmark denmark, france, india, spain scrum master health 5+ p4 nz nz nz, vietnam agile coach fintech* 3+ p5 nz nz australia, china, india, nz, singapore senior project manager it 3+ p6 nz argentina argentina, nz founder it 3+ p7 nz nz australia, nz senior manager fintech* 5+ p8 nz nz brazil, nz product manager it 3+ p9 nz nz nz, uk scrum master it 3+ p10 nz nz australia, nz, uk scrum master it 3+ key • p (e.g., p1) refers to the participant number and is used in the quotes in this article. • residency refers to the primary country of residence of the participant. • nz refers to new zealand; the uk refers to the united kingdom. • exp. refers to the experience level of the participant in years. • p9 and p10 were from the same company; other participants were from different companies. • * fintech is a portmanteau of "financial technology". the term refers to the utilisation of emerging technological innovations by companies to competitively provide financial services that may challenge conventional approaches. • it is information technology table 1. participant profiles australasian journal of information systems kostin & strode 2023, vol 27, selected papers from acis 2022 effective communication in globally distributed scrum 13 the participant interviews resulted in 131 pages of transcript and the analysis created 130 codes. each code was supported with one or more quotes from the interviews. the codes formed three groups. 12 codes were associated with the understanding of effective communication in gd scrum teams and contributed to the model development (in section 4). 11 codes were related to how to improve non-effective communication (i.e., practical advice in section 5). the remainder of the codes do not contribute to this article. they were related to communication challenges and communication tools. 4 a model of effective communication in gd scrum teams the analysis showed that communication effectiveness is not a single idea with a single definition but is made up of related concepts. identifying and defining concepts and explaining their relationships is the process of developing a theoretical model (dubin, 1978). a theoretical model should have well-defined concepts and a boundary that defines where the model is applicable (weber, 2012). the boundary for our model is the gd scrum team. this section presents a theoretical model of communication effectiveness in gd scrum teams that comprises four communication concepts, their definitions, and the relationship between the concepts. this is a theory of explanation according to the theory classification framework of gregor (2006) because the model explains concepts within a specific bounded context but does not aim to predict with any precision and has no testable propositions. the analysis led to 12 codes that formed four distinct categories of effective communication. we call these categories ‘concepts’ in the remainder of this article because this is the terminology used when discussing theoretical models. the concept names, concept definitions, and codes that contribute to the concepts are presented in table 2. these concepts form the model of effective communication in gd scrum teams and include alignment, communication transparency, communication quality, and communication discipline. each concept was defined based on the ideas conveyed in the interviews, but we also referred to standard dictionary definitions. we did this to ensure that the codes related to a concept, the concept name and the concept definition all referred to the same idea that was assigned a recognisable name as defined in a dictionary. this was done to reduce semantic drift between the concept name that emerged from the quotes and codes, and the usual definition of the concept. the following sections describe the concepts commencing with the outcome concept, alignment. each concept description includes a discussion and supporting evidence. the evidence comprises direct quotes from the interviews combined with findings from relevant literature if it was available. the arguments for the relationships between the concepts are presented in section 4.5. the following sections include exemplar quotes to support the arguments for our findings. these quotes are edited lightly as follows. ellipses (…) indicate elided text that is not relevant (e.g., ‘so’, ‘you know’, ‘like’). underlined words or phrases highlight the words relevant to the assigned code. words in square brackets [] are inserted to clarify a phrase (e.g., to clarify what ‘it’ refers to in the transcript text). australasian journal of information systems kostin & strode 2023, vol 27, selected papers from acis 2022 effective communication in globally distributed scrum 14 communication category/concept, [frequency], and definition code alignment [4] is a state of agreement between distributed team members about all aspects of the work (i.e., product, priorities, process, and work progress). • be on the same page (i.e., have a shared understanding) • aligning priorities communication transparency [6] is clear communication that is open, honest, and readily understood. this includes the conveyance of clear expectations in the team. • be open and honest • convey clear expectations communication quality [11] is the conciseness, speed, equality, and responsiveness of communication in the team. • interact face-to-face • give/get quick replies • have listening and speaking equality • give/get feedback • have one-to-one direct communication communication discipline [2] is when people communicate in a controlled way by following agreed norms about who to communicate with, what to communicate (e.g., task completion), and use appropriate communication tools. • use an appropriate communication tool • follow through on instructions • keep stakeholders informed table 2. effective communication in gdst: communication concept and related codes note. "frequency" refers to how many participants mentioned (an) idea(s) leading to the same code. 4.1 alignment alignment in gd scrum concerns the development and maintenance of a shared understanding among all those involved with the team’s work. alignment codes conveyed the opinion that communication is effective when the priorities of the team, its sub-teams, and any other stakeholders are understood and everyone in the team is said to be ‘on the same page’ with agreement on priorities. being on the same page is an idiom meaning “of two or more people, thinking in the same manner; having the same general outlook or position” (free dictionary https://idioms.thefreedictionary.com/on+the+same+page ). for example, p3 stated, “... effective communication is of course, … getting everybody on the same page ... and understanding the vision and priorities and so on, is something i think about when i communicate or try to communicate effectively”. alignment is a well-established concept in psychology where alignment in communication is when a shared mental model develops about a situation within a dyad or group (wachsmuth et al., 2013). a team mental model, which is a team-wide shared mental model, is considered critical to effective teamwork (salas et al., 2005), is a factor in the success of globally distributed virtual teams (florea & stoica, 2019) and globally distributed large-scale software development when coordinating work efforts (espinosa et al., 2001). shared, or team, mental models research in agile and scrum teams is nascent. we found a single study, by edmondson and chiu (2020), of collocated scrum teams showing that a shared-mental model between the product owner and the development team affects client and team satisfaction. 4.2 communication transparency transparency in gd scrum concerns open and honest communication in the team and conveying clear expectations. transparency was the most frequently mentioned indicator of communication effectiveness. for p10, effective communication was being transparent, open and honest, " it is about being open again, …, being transparent about your current situation …”. p9, confirmed this, saying, “open and honest communication would be successful communication for me.” p1 viewed being open to questions as a communication advantage for his team and said australasian journal of information systems kostin & strode 2023, vol 27, selected papers from acis 2022 effective communication in globally distributed scrum 15 “… in my teams, we're very open to any questions anytime. so, the team member can come to me and say … 'we got those questions' or 'we got some delays can we talk about that?' and we are talking about that. this improves communication a lot.” transparency includes conveying clear expectations. the expectations we found were about tasks, about customer expectations about the product, about capturing acceptance criteria, and conveying expectations about how best to communicate. p5 thought it was about the task, “to ensure that we have one message, … and set clear expectations and actions for the task so during the stand up we go through, we say what are we doing, what have we achieved and what is … planned.”. p8 thought it was about task order and clarity between the team and product owner who were distributed between nz and brazil, “when you're working with a team, it needs to be really clear what they're doing next, and that sort of thinking from product owner to team kind of communication.” p9 explained that she made her expectations clear about how to communicate effectively in an example, “making sure that the people that i have the most influence over start to communicate effectively, and i'm talking about what that communication, effective communication, looks like, and start, making sure that we are calling out poor communication, but not … in a … ‘you did a bad job’ kind of way, just gently, or ‘hey, that email that you sent probably could have been more concise, or that email that you sent, you actually didn't give them a deadline as to when they needed to get that information back to you’.” the ideas about transparency, honesty and openness all occur in the scrum guide, which explicitly states that a scrum team should be transparent about problems and progress, and open about challenges (schwaber & sutherland, 2020). transparency has been linked with trust. for example, yue et al. (2019) found that transparent communication is associated with trust between team members in a study of organisational change. in agile environments, the industry commentator eckstein (2013) stated that trust can only be established by transparency. hennel and rosenkranz (2021) treat ‘transparency’, and ‘open and honest communication’ as distinct but related concepts that influence the resilience of agile teams in a study of psychological safety in three case studies in germany. they considered that transparency facilitates trust, communication, and knowledge sharing. while seldom mentioned in agile software development literature, the concept of communication transparency occurs frequently in the organisational literature, but the focus is on clarity in communication at the organisation level rather than the team level. in this literature, common sense definitions are normally used to define transparency as ‘openness’, ‘insight’, and ‘clarity’ (christensen & cheney, 2015). communication transparency also occurs in the context of enterprise social media where it is called communication visibility (leonardi, 2014), but the concept is narrowly defined as seeing the content of other’s social media messages. our definition of communication transparency includes conveying clear expectations. there is a lack of research on this aspect of system development, although not meeting customer expectations is reported as an important communication gap in large-scale software development (bjarnason et al., 2011). understanding and managing customer expectations is considered a traditional it project management function (petter & randolph, 2009), and one reason why the product owner role emerged in agile software development was to ensure that customer expectations are understood and met (kadenic et al., 2023; kelly, 2019). we found no direct support for the importance of conveying clear expectations as a dimension of communication transparency in the it-related literature. australasian journal of information systems kostin & strode 2023, vol 27, selected papers from acis 2022 effective communication in globally distributed scrum 16 in summary, our definition of communication transparency is supported by similar definitions in organisation literature (christensen & cheney, 2015) and scrum literature (schwaber & sutherland, 2020) except for conveying clear expectations, which does not occur in these sources. this could be because of the need for greater clarity about expectations when teams are distributed and cannot identify expectations just by being close together physically, which can enable informal chat about expectations. alternatively, this could be because different national cultures have different workplace or task expectations that need to be made explicit (bastiaansen & wilderom, 2021; šmite et al., 2021). 4.3 communication quality communication quality concerns the conciseness, speed, equality, and responsiveness of communication. p9 thought effective communication had to be concise and speedy: “effective communication for me, and my team would look like information being shared or requested is as concise as possible. and responses coming in as quick as possible.” equality is important for effective communication and involves listening and speaking equally among all team members. p4 said, “so for me, … effective communication is where equal parts of listening and speaking take place. so, it's a true conversation of both parties... and if i give you an example of how we'd set up the current cross-shore team, with the different time zones, it was very important for us to communicate at the end of the day with each other. so, we are all on the same page of how things are progressing towards our goal, our sprint goal. and that was communicating from both ends. so, it was not just expecting that the across-shore distributed team, which is sitting in vietnam, is the only one who's responsible for communicating to us, it's both parties communicating equally. and that involves listening as well as speaking.” responsiveness was mentioned in different ways. p10 mentions responsiveness as acknowledging that a message has been received and responding in a timely way. "for me, effective communication … it's about responding. … effectively. … it's important that you are giving the other person, who is expecting an answer from you, … to acknowledge saying that, … ‘i have read your message, this is what you want, it's going to take me some time to get you the answer.' but then, 'yes, i'm here to do that for you. just give me some time'. … saying that, …, ‘currently, i'm working on this, but i will get back to you, by the end of the day, or whichever time you need in order to get that question answered’”. p10 continued, “because i've seen in my previous experience, people, if they're not able to do something, they just wouldn't respond to that message or email for a long time, they would say, … i'll get to it, when i have time, …, the right thing to do is acknowledge it, and answer it whenever you can. so that's one very crucial thing, especially for global level communication." p6 talked about ‘message receiving’ in communication and how senders should consider the situation of the receiver and choose when to send and how to send a message. “…effective communication depends mostly on the recipient of the message. so, if i am at a different time zone, and i just sort of throw something to you, because you need to know, but i'm not accounting into the fact that maybe it's, …, 3 am in the morning for you, and then you're going to wake up, and at the first time in the morning you won't read that, then that's not going to be that effective. so effective communication abilities through the team to me is about accounting for whoever receives the message that is in a proper situation or appropriate environment to receive it.” p6 continued later, saying that for any communication of value “if it's not received properly, then the communication fails.” direct person-to-person communication was viewed as a way to support responsiveness. p4 explains, “… if you're trying to build a high-performing team, then you need to make sure that each australasian journal of information systems kostin & strode 2023, vol 27, selected papers from acis 2022 effective communication in globally distributed scrum 17 individual can communicate with the other and the team … .” p4 continued: “it’s individuals communicating with each other at their level, but being able to listen, being able to communicate back. so, it's both sides, not just listening, but also speaking, and speaking your mind, and being able to effectively say this is the problem and hey, i need some help, or hey, i can help you out. so listening, if someone needs help by saying – ‘hey, i'll be able to help with that because i know, i've resolved that problem before’.” p9 mentioned the interactive nature of effective communication, “an equal kind of back-and-forward or two-way flow of information.” communication quality in our model is similar to closed-loop communication, a factor in effective teamwork in all team types. salas et al. (2005) defined closed-loop communication in a review and model-building article on effective teams as “the exchange of information between a sender and a receiver irrespective of the medium” (salas et al., 2005, p. 561), that involves following up with team members to ensure the message was received, acknowledging that the message was received, and clarifying with the sender of the message that the message received is the same as the intended message. for agile teams, strode et al. (2022) argue that whole-team closed-loop communication is critical for effective teamwork without specifying if this applies to all teams or only collocated teams. our findings concur with this idea that giving and acknowledging replies or feedback within the team is a factor in effective communication in gd scrum teams. marlow et al. (2018) defined communication quality as the extent to which communication distributes pertinent information to the team members as needed (see section 2). our definition, however, is concerned with conciseness, speed, equality, and responsiveness of communication among the team members, rather than just conveying pertinent information. in organisation studies, there is no consensus on a definition of communication quality. communication quality is reported in domains such as retailing (mohr & sohi, 1995), management (orpen, 1997), health (fawole et al., 2013), and virtual teams (chang et al., 2011) among others, but often with no clear definition. in psychology, gonzález-romá and hernández (2014) define it as “the extent to which communication among team members is clear, effective, complete, fluent and on time” (p. 7). in studies of agile and scrum teams in distributed environments, communication quality is mentioned but we could find no sources that define the concept in an agile context or describe it in any depth (ågren et al., 2022; lampropoulos & siakas, 2018; lukusa et al., 2021). therefore, our definition of communication quality appears to be unique. each dimension of communication quality can affect communication in gd scrum. if messages are not concise, they can take a long time to convey, and the meaning can become unclear especially if there are language differences. if responses are not speedy, they can cause teams in other time zones to delay their decision or tasks for longer than is optimal. if distributed teams do not equally contribute to discussions, then the silent partners’ ideas, decisions, problems, or solutions may never be known leading to less effective communication and teamwork. 4.4 communication discipline disciplined communication means following accepted norms for communication. in our findings, this encompassed who to communicate with (e.g., teammates, stakeholders), what to communicate (i.e., following through on tasks and communicating task status), and using appropriate communication tools. the scrum framework (schwaber & sutherland, 2020) sets out the broad norms for communication in each sprint, for example, in daily stand-ups, sprint australasian journal of information systems kostin & strode 2023, vol 27, selected papers from acis 2022 effective communication in globally distributed scrum 18 planning and reviews, and retrospective meetings, specifying who should be involved and what activities should take place to share knowledge and decision-making. communication discipline includes the code ‘keep stakeholders informed’ because regular, frequent, and consistent communication with stakeholders affects teams, projects, workflow, and product quality. stakeholders are closely involved in scrum teams because they can provide financial, social, and political support and timely information about requirements. for example, p2 explained that communication with the stakeholder about requirements and acceptance criteria are important to avoid problems: “… the requirements and basically the acceptance criteria, those requirements are well understood, such that…, during the sprint review or during … when we demo the application, or even during going live …, then it is smooth and it will not face … red flags for the customer or client like, ‘wait, that's not up to what we discussed’… .” the importance of what to communicate is evidenced in the code ‘following through on instructions’ to achieve effective communication. p5 explained it this way. “yes, it [communication] can be 100% effective, but it's your propensity to pick up an instruction, follow it through, complete it, and then report back as being completed.” using an ‘appropriate communication tool’ for the situation and the subject matter was also important. p6 said, “so, effective communication abilities through [the] team to me is about accounting for whoever receives the message, that is [they are] in a proper situation or appropriate environment to receive it, sometimes that means that certain conversations have to be a video call, for instance." p9 said, “i think they [communication tools] are all efficient in their own way, just depending on what we want to achieve.” few theories, frameworks, or models focus on communication discipline. the idea does emerge as a factor in studies of psychological safety in virtual teams. for example, lechner and tobias mortlock (2022) interviewed 16 members of virtual teams (of all types) and recommended ‘discussing the rules of the game’ in virtual teams to enable psychological safety. they argue that behavioural norms about how the team wants to work together are created by explicitly discussing the rules rather than letting them emerge organically, as they do in collocated teams. these authors consider three actions are needed to negotiate the rules of the game: agree on shared goals and responsibilities, develop a common code of conduct, and align the use of tools with the team’s needs. these actions share similarities with our communication discipline, but our concept is more specific because it includes who and what to communicate. however, ‘who’ and ‘what’ can be negotiated when agreeing on shared goals and responsibilities and can be written down in a code of conduct. the idea of aligning the use of tools with the team’s needs is reflected in our idea of using appropriate communication tools. scrum research and the grey literature seldom mention norms, although scrum activities such as deciding on a ‘definition of done’ (kopczyńska et al., 2022) and deciding on action items in retrospectives (andriyani et al., 2017) could contribute to norm development. agile team norms emerged in four scrum teams in a company with teams in malaysia and norway in a study by stray et al. (2016). however, in that study differences in communication norms were attributed to cultural differences and a lack of team maturity, however, these causes were not discussed in detail. communication discipline is not a distinct concept in the literature on virtual team types or distributed software development teams. therefore, our definition of communication australasian journal of information systems kostin & strode 2023, vol 27, selected papers from acis 2022 effective communication in globally distributed scrum 19 discipline appears to be unique. communication discipline might be more important for gd scrum teams than collocated teams because scrum and software development requires high levels of communication and teamwork, both of which are constrained by the teams' distribution in time and location, and their reliance on electronic media to communicate. for example, explicit team communication norms about who to communicate with and what to communicate become more important if a message is to be read many hours after it is sent, to avoid additional delays in response. sending one’s message to the most appropriate person can be more important if responses must be timely. using the right media and tool for the type of information one wants to convey also matters, as many studies confirm (dennis et al., 2008; jackson et al., 2022). 4.5 communication model relationships the relationships between the four concepts related to effective communication in gd scrum teams, alignment, communication transparency, communication quality, and communication discipline complete the theoretical model. we argue that communication transparency, quality, and discipline lead to alignment, that is, they are precursors of alignment. in other words, alignment, which is a shared understanding in the team (i.e., a team-level shared mental model), is an outcome of transparent, high-quality, and disciplined communication. furthermore, alignment can be considered equivalent to effective communication in gd scrum teams. this relationship is illustrated in figure 2. figure 2. a model of effective communication in globally distributed scrum teams the argument that alignment is the outcome in the model is based on ideas in media synchronicity theory (dennis et al., 2008; kittelman et al., 2018), which poses that communication has two main processes 1) conveyance of information and 2) convergence of meaning (i.e., developing shared meanings). kittelman et al. (2018) argue that in general, information must be conveyed before a convergence of meaning occurs. using this distinction, our model includes conveyance and convergence. communication quality, transparency, and discipline are concerned with conveying information. for example, information is conveyed among team members (is sent and received) with high quality (e.g., is concise and responsive), transparently (e.g., is open and honest), and in a disciplined manner (e.g., following communication norms). convergence of meaning according to kittelman et al. (2018) is about developing shared meanings. we have defined alignment as equivalent to shared understanding, so we consider convergence and alignment, to be equivalent. communication transparency communication quality communication discipline alignment australasian journal of information systems kostin & strode 2023, vol 27, selected papers from acis 2022 effective communication in globally distributed scrum 20 based on these ideas, communication quality, transparency, and discipline convey information and lead to alignment (i.e., convergence on a shared mental model). this implies that to achieve high levels of alignment, actions to achieve communication quality, transparency, and discipline should be included in gd scrum team communication. the illustration in figure 2 shows a + sign to indicate the relationship. the arrow indicates the idea that these three concepts lead to alignment. 5 practical guidance for improving and sustaining communication the analysis that led to our proposed model of effective communication also revealed concrete practical guidance for improving and sustaining communication in gd scrum teams. participants were asked for their insights about how to improve communication, how they kept communication effective, and the challenges they encountered (see appendix b for the interview schedule). from those questions, 11 codes emerged that were categorised as practical actions the participants took to improve and sustain effective communication. the 11 practical actions are listed below. the list is not ordered. the numbers indicate how many participants referred to this practice. 1. always explain why [1]. 2. convey and follow through on one message [1]. 3. build relationships [1]. 4. hold a team charter session [4]. 5. create a safe environment for communicating [4]. 6. model good communication behaviour [1]. 7. respect personal boundaries when communicating across time zones [4]. 8. consider personal communication preference mismatches [3]. 9. use agile practices for communication, whether the team is distributed or not [2]. 10. look for the root causes of problematic communication [1]. 11. allow for reduced body language signals in virtual communication [4]. this section provides arguments, evidence, and supporting literature for the 11 practices. 5.1 always explain why when communicating, the person requesting others to change should always explain why the change is needed. a participant (p7) who was a coach said that explaining why (i.e., explaining why some change was necessary) helped team members understand the usefulness or benefits of adopting new practices. his example was about introducing a sprint review to get customer feedback on the product. p7 explains, “the approach that i take, usually in coaching people, even that's the approach i asked the scrum masters to do, is you start with ‘why’. when you're introducing a practice, you are going to introduce a particular way of working, you start with why. it's simon sinek’s … golden circle that talks about why before you talk about how and what. … we talk about how important it is for example, for us to actually get feedback from our customers, from our user, from our stakeholders to make sure that we are delivering value to them or delivering capabilities or features that they will be using and appreciating and getting return on investment. and it’s the only way we can get, …, the confidence that … we are actually doing the right thing. it's only possible when we get feedback, so therefore, we must get feedback. … now we're going to actually do it, is yes we're going to have a sprint review, we're going to have it in once in two weeks so once in three weeks for one hour … .” he explained that this practice helped convince people that the new practice was the right australasian journal of information systems kostin & strode 2023, vol 27, selected papers from acis 2022 effective communication in globally distributed scrum 21 thing to do, “and they will say yes to it, … no matter how difficult this is, we have to do it because it's the right thing to do. so, you educate people with the why. i mean all the people are working in our teams are very smart people, very intelligent people, they are developers and testers, they are analysts … these are people who care for the work they do and therefore, if you cannot explain to them why some new practices or new … process is put in place [then] they will find that this is not the right thing for them to do or adopt.” the idea of ‘always explaining why’ is explained in a leadership manual by sinek (2011) who argues that leaders should explain why to followers so the followers know the reason for the work they undertake, how the work fits with organisational goals, or why a change is needed. this idea is related to ‘work meaningfulness’ in leadership literature (kipfelsberger et al., 2022). wolf et al. (2022) reported, based on 12 interviews, that explaining why is important for activity coordination in internal communications in it-intensive start-up companies. smart (2018) also recommended sinek’s (2011) advice when implementing agility in devops environments. we found no advice on this practice in gd scrum, gd agile, or virtual team research. this practice could be pertinent in both collocated and distributed environments, but as we explained in section 4.2 ‘communication transparency’, when teams are distributed there is a need for explicit communication about the reasons for actions or requests. this is because the team members are not close together physically and cannot observe the situation or take part in informal chat; they cannot identify the reason for a request or a change by implicit communication. 5.2 convey and follow through on one message ‘convey and follow through on one message’ means communicating carefully to the whole team by focusing on conveying one message to all sub-teams who are in different locations. p5 explains how important this is when distributed teams work simultaneously on different software components that must be integrated. he advises conveying and following through on one message, particularly to follow through on video calls or verbal messages with written confirmation: “we ensure that we have one message and set clear expectations and actions for the task. so, during the stand-up we go through, we say what we are doing, what have we achieved and what's planned. yeah. but what we've got to say on top of that is where the priorities are and then link those priorities together so if you've got china that's taken on a piece of work. you've got india and then new zealand, and we're working on segments at some point in time, they all need to integrate. … and so, the coordination is very difficult to put into a jira ticket or into a user story, that's project management, and you've got to communicate effectively and that is to bring all three parties together and say at this place at this time. at this junction, we're going to integrate, and we all have to be ready … we all need to meet the mark together at the same time, just be really prepared to integrate and then test the product. … communication was the key, written communication following all video conferencing. if there was no communication or written communication, often particular tasks, or those priority of tasks would not be completed or individuals that you set the task of following through and talking with somebody or following through on an action. yeah, they don't carry it out. and they don't do what they've been instructed to do.” as we explain in section 4.3 communication quality, teamwork studies confirm that communication involves following up with team members to ensure the message was received, acknowledging that the message was received, and clarifying that the message received matches the intended message (salas et al., 2005). furthermore, in an agile teamwork study, strode et al. (2022, p. 23) argue that “the team follows up on the progress of tasks” is an australasian journal of information systems kostin & strode 2023, vol 27, selected papers from acis 2022 effective communication in globally distributed scrum 22 indicator of effective communication in agile software development. although this practical advice is undeniably important for collocated scrum teams, for gd scrum teams, developing components in different countries and then integrating them, this advice is likely to be even more critical due to the extra overhead in time and cost if a message is not received correctly. 5.3 build relationships when the team is distributed, extra effort and time are needed to build and maintain good relationships between the distributed sub-teams. p4 describes some of the practical efforts made by his team, “… you have to invest a lot of time in it… i have invested, personally, a lot of time in making sure that i've got the relationship going [with] the other side. we play games; we watch videos together. in retros, we want something inspirational together, we talk about our family lives, we talk about all sorts of things to keep that personal connection going, because otherwise, it gets a bit harder, because we're just there to ship them work and expect them to deliver work. and that's the model that i never wanted to build. so, for me, … i wanted to make sure that i could create a very highly trustworthy team with high performance in-built and self-organizing ... right, i'm not responsible for them to organize, [that is] their responsibility.” the purpose of building relationships is to build trust among team members, which is needed to support effective collaboration, communication, and knowledge sharing (newell et al., 2007). the importance of trust as a foundation for effective teamwork has a long history (salas et al., 2005), and is recognized as a problem for virtual work teams of all types (newell et al., 2007) as well as gd software development teams (casey, 2010). building relationships and trust is a central idea in the agile manifesto (beck et al., 2001a) and the scrum guide (schwaber & sutherland, 2020). a study by tyagi et al. (2022) focuses on building trust in gd scrum teams and, like our participants, they recommended regular socialization within and between teams to build and improve trusting relationships. 5.4 hold a team charter session when an agile team first assembles our participants recommended that they write a team charter. the agile practice guide team charter (griffiths et al., 2017) includes agreements about team values, working arrangements, ground rules for communicating, and group norms (agreed behaviours). four participants introduced a team charter session to improve communication or to keep communication effective. two of them said: “so part of it was having a team charter when we first came together as a team, just making sure that we had an agreed set of principles that we would work towards, making sure that it was raised. not, just any working principles … things like meeting etiquette, showing up to meetings on time.” (p9) the team charter had several communication-related purposes including providing a forum for discussing preferred team dynamics, individual preferences, and the importance of being open about things that have gone wrong. “initially, we had a team charter session, you know, it's about talking about the team dynamics, that relationship we want to have in the team … everybody talks about, 'oh, this is what i don't like, this is what how i like' ... usually when a new team is formed, we have the team charter session, which kind of makes people understand what each other's preferences are. so, during that time, definitely we talk about, … it's important to be open. and … it's important to admit if something goes wrong.” (p10) scrum (schwaber & beedle, 2002; schwaber & sutherland, 2020) does not mention team charters but they are often introduced by agile and scrum coaches. team charters are not a new idea (wilkinson & moran, 1998) but they are accepted practice in project management australasian journal of information systems kostin & strode 2023, vol 27, selected papers from acis 2022 effective communication in globally distributed scrum 23 (pmi, 2021) and agile software development (griffiths et al., 2017) but research on team charters is scant in agile software development or gd scrum. the quotes above mention that when the charter is being developed by the team, this provides an opportunity to talk about and show openness. therefore, this practice would have benefits for any type of agile team but would particularly help a gd team to act transparently and discuss how they will achieve communication quality and discipline. 5.5 create a safe environment for communicating the environment must be a safe place for team members before authentic communication will occur. our participants thought that striving for a safe environment, although a challenge, was important for effective communication. p2 said, “… this is actually quite difficult because you actually don't know how they work with each other. so, i guess, trying to encourage that open communication or over communicating to them. like, guys, again, we are one team we have one goal here as much as possible. can we all be open and transparent and have the courage to have the least difficult conversation upfront, when there is something wrong? and then to really share with them that … this is respected and … this is a safe environment to speak up if we have something in mind.” hennel and rosenkranz (2021, p. 11) said about agile teams that “team members need to feel safe to speak freely”. this idea of a safe work environment is called psychological safety which is “a shared belief held by members of a team that the team is safe for interpersonal risk taking” (edmondson, 1999, p. 354). in these environments, team members are more likely to seek feedback, ask for help, speak up about concerns or mistakes, and explore innovative ideas (edmondson, 1999). in gd scrum, different teams can have different cultural, national, and workplace expectations and norms so being explicit about what a ‘safe environment’ means is necessary and might not always be easily achieved. thorgren and caiman (2019) explored this issue in a single case study of gd scrum between teams in europe and south america. they found major differences between the distributed teams in their attitudes toward inclusiveness, perceptions of, and trust in collective responsibility, and openness in communication, and overcoming these differences can take ongoing time and effort. 5.6 model good communication behaviour modelling good communication behaviour was considered important. one participant noted that he tried to model good behaviour by discussing his own mistakes so that other team members would feel it was safe to follow this practice. p10 explained that, “what i particularly tried to do is … what i said, … there are … many instances where i would just say, ‘oh, i actually made a mistake’, it's probably something that the team really doesn't need to know, it could be something that is between just the product owner and me, but i would admit that in front of the team so that they know, … if i can admit them, they can also admit if things go wrong with their work. so, i think that is extremely important to … show them how it feels when you admit your mistakes.” this advice is related to encouraging a work environment that supports psychological safety. the participant is modelling the behaviour he wants to encourage in the team. this type of role modelling is reported in a case study of collocated scrum when onboarding newcomers into agile teams (gregory et al., 2022), but otherwise, there is scant research on this topic in agile literature. the agile coach role does include advice to model agile-related behaviours and this guidance seems appropriate for all forms of scrum adoption, gd or collocated (adkins, 2010). australasian journal of information systems kostin & strode 2023, vol 27, selected papers from acis 2022 effective communication in globally distributed scrum 24 5.7 respect personal boundaries when communicating across time zones respect for personal boundaries when communicating across time zones is a way to maintain effective communication. p1 explained what happened during the covid-19 pandemic lockdowns, “there are so many different means of communication right now. i know video, voice, and text, and i saw a lot of software that helps you deal with those things. but i think sometimes the communication, especially nowadays like when the pandemic hit … it became more … personal and touchier … you have to know how to communicate with people so it's like i was constantly learning when the lockdowns started. i know some simple things when you call someone, someone doesn't turn the camera on or someone does … one way you communicate different things is when you speak to people from, different places of the world they’re different… especially when it's a global distribution, … there are certain nationality traits or … personality characteristics that you have to consider, and we have to always be mindful about… for example, teams in asia, most of them, they try to work as hard as they can. they're literally available 24/7. and, … you have to be mindful of people's time, … you cannot … just go and randomly ask the question in the middle of the night and just not even expecting a response but try sending, sharing the communication, even talking to someone, even messaging or emailing someone when it's [not their work] time. it may hurt so much [i.e., damage relationships] … now, when the pandemic … happened, people just lost track of working time. that's just very bad … i think that by then, …, people should be … more mindful about others' time right now … i think that's the most important thing for communication.” communicating when there are time zone differences between agile teams is a central issue in gd agile software development (alzoubi et al., 2016). alzoubi et al. (2016), based on an slr, report that respecting work timetables and religious holidays is related to time zone differences. they recommend making certain meetings mandatory whereas our participants recommend respecting and accommodating a team member's personal time frames. respecting personal boundaries regarding time and interruptions is seldom mentioned explicitly in gd scrum sources. 5.8 consider personal communication preference mismatches personal communication preferences need to be considered to support effective communication. this communication is at the individual level and can be as simple as a preference for email or phone for quick communications. three participants acknowledged the challenge of personal communication preference mismatch. p1 explained it this way, “basically, the nature of the distribution is that … at some point, it gets very personal … especially when we talk about communication, one person prefers one way, the other person prefers to [communicate in] another way … as a product manager i spent 90% of my day just talking to people or writing, or … in some way or form communicating to others, and yes it's very personal ... and i think it is the same for everyone. trying to understand how to properly communicate with your peers, with someone you work with all the time is the most important … thing.” this practical advice involves identifying communication preferences and adapting to them. this is a specific piece of advice not explicitly addressed in gd agile or scrum studies. a survey by hoffmann et al. (2022) (pre-covid-19 pandemic) of human-related challenges in software engineering teams (192 respondents worldwide; 38% distributed; agile use not reported) identified different communication preferences as an interpersonal challenge reported by respondents in a qualitative section of their survey. this study, however, did not report if the degree of team distribution increases this issue with communication preferences. australasian journal of information systems kostin & strode 2023, vol 27, selected papers from acis 2022 effective communication in globally distributed scrum 25 5.9 use agile practices for communication scrum practices contribute to effective communication. p5 found that careful and clear communication helped the distributed team to be autonomous because it gave them a clear idea of what was required and helped them understand what should be happening next. this included the verbal and written communication that typically occurs when engaged in scrum ceremonies such as stand-up meetings and in the use of artefacts such as user stories. p5 explained, “the key thing is … clarity when you're working with a team, it needs to be really clear what they're doing next, and that sort of thinking from product owner to team … communication. and so leveraging things like user stories, which we still use, is quite useful. making sure that we are using things like stand ups effectively. … i think this communication helps the team move forward. … it should be … open and more of a dialogue … i mentioned how we use our stand-ups; we don't use them for status updates. we use them to get a picture of our work, and to decide what the next most useful thing we could do as a team is … it's not someone turning up and saying what needs to be done next. …good, effective communication helps the team become more self-organising is probably a good point here as well. and that's a combination of dialogue through video chat, it's a combination of well-written work and well-managed work, to the point where if i don't turn up to a stand-up, work still happens, and it still happens in the way that i've thought it might go anyway.” one participant explained that even if the team was not distributed across time zones, they would not change their communication patterns but would continue to use their chosen agile practices to maintain effective communication. when the researcher asked, “in the hypothetical situation where all the common challenges are resolved, can it be said that the communication definitely becomes effective in this case?” p1 explained, “not really, i mean the communication is still going to be the very same. we'll have a bigger overlap in terms of the working hours or will have just one big overlap because the hours are the same, but the communication is still going to be the very same … we won't change any practices and will still do this scrum [meeting conference] calls, will still do the backlog refinements and everything, we will still maintain our documentation, will still do the retrospectives, we still do the very same … things [agile ceremonies].” agile practices are known to affect communication in collocated projects (pikkarainen et al., 2008; sharp & robinson, 2010). scrum has multiple regular meetings where communication occurs (e.g., planning meetings, daily meetings, retrospectives, and product demonstrations), and artefacts such as task boards display sprint backlogs to communicate information (schwaber & sutherland, 2020), and the agile manifesto recommends colocation to enable communication (beck et al., 2001a). in distributed situations, meetings and artefacts enable communication, although their effectiveness is likely to be moderated by the availability of good internet connections, virtual task wallboards (sallin & kropp, 2022) and other collaboration tools for software engineering (jackson et al., 2022). 5.10 look for the root causes of problematic communication even if the communication problems caused by time zone differences were removed, communication might still be difficult. p9 explained that time zone differences might be blamed for poor communication when the underlying cause was a performance issue; for example, one person who is tardy in responding to emails can affect non-distributed work but have a more serious effect on the progress of daily work if the long time-lag due to the time zone difference is not considered. p9 explained, “i think if we could fix all of those [challenges], we'd still find another problem that we thought was related to those things and actually wasn't. so, we probably say, oh, yeah, ‘that's happening, because we've got a distributed team because of time zones’. australasian journal of information systems kostin & strode 2023, vol 27, selected papers from acis 2022 effective communication in globally distributed scrum 26 and actually, you could find that it was a performance issue with someone just doesn't respond on time, because … they're not sending the emails on time, or they're not making the emails concise enough … i think that we probably blame these certain problems for why things are happening. and i think if it was all resolved, we'd find another set of problems that were happening for whatever reason. yeah, that sounds really pessimistic, but i just don't think that if those were solved, it's not the magic pill to improve communication." problematic communication might be due to any number of causes and may not be due to the distributed nature of development as shown in the survey by hoffmann et al. (2022) (see section 5.8) of problems in human aspects of software engineering where many of the same problems were reported in both collocated and distributed environments. this idea of finding root causes is not addressed in gd scrum or agile literature but appears to be a sensible problem-solving tactic when looking for appropriate solutions to communication issues. the tactic is formalised as root-cause analysis in project management (pmi, 2021). 5.11 allow for reduced body language signals in virtual communication non-effective communication when using virtual communication was attributed to the reduction in natural communication, in particular, the inability to read non-verbal communication signals conveyed by body language. p6 commented on the negative effect on communication when body language is not visible, “normal communication changes dramatically. if you study communication at any level, you realize instantly that there are different levels of communication. and that visual audio and text have different levels of how they are, not only communicated but … perceived. and a factor within those is also the physicality of being in the same room as the other person. because of what body language represents. body language is not just the face, this is the entire body. and it's the nuance of it as well. so, i may say something, and your toe may move. and i can only pick up that maybe if i'm in the room, and he can be asleep, or sad, or … but when you're doing a video call, a lot of it is lost. and there are difficulties depending on how people are, on how that will result, because some people will be very vocal and they … let you know, but other people are not that vocal, or they're not that direct. so, they will not address an issue unless you do it. so, if you miss that physical trigger, you have no way of … addressing that.” there is a large body of knowledge on non-verbal communication because it contributes significantly to the messages and meanings perceived in face-to-face communication between humans (hall et al., 2019). virtual communication reduces non-verbal clues due to the nature of the communication media available (e.g., phone, teleconferencing, email, video calls, virtual boards, chat) (jackson et al., 2022). the lack of non-verbal communication in virtual team interactions is generally agreed to be a problem (morrison-smith & ruiz, 2020). in software engineering, research into the effect of a lack of non-verbal communication in gd teams is scant. ciancarini et al. (2021) found that non-verbal communication helps to establish successful meetings and contributes to positive interactions among individuals in software engineering roles. they concluded that curbing or limiting non-verbal communication may cause work issues and reduce work effectiveness. the study was based on online interviews with 38 it professionals in russia with 45% of interviewees using scrum and 21% using other agile methods, but if the interviewees worked in distributed teams was not reported. these results imply that gd scrum teams using electronically mediated communication might suffer from less effective communication due to the lack of non-verbal clues in interactions such as meetings. however, specific advice on how to overcome this issue is not available. as we australasian journal of information systems kostin & strode 2023, vol 27, selected papers from acis 2022 effective communication in globally distributed scrum 27 argue in section 4, employing transparent, quality, and disciplined communication should help. in summary, in the 11 items or practical advice we found that only two pieces of advice explicitly were related to agile practices, item 4 ‘hold team charter sessions’, and item 9 ‘use agile practices for communication’. item 3 ‘build relationships’, and item 5 ‘create a safe environment’ have been explored in gd scrum studies, but the remainder of the items have not been a specific research focus in gd scrum or agile studies. our list of practical advice is not exhaustive. for example, our participants did not mention problems that occur if teams speak different languages or have different accents. therefore, our practical advice should be considered alongside other recent studies of gd scrum or agile in remote contexts, especially if the advice is based on in-depth qualitative empirical data. for example, šmite et al. (2021) provide specific advice on overcoming cultural barriers in gd scrum teams, bablo et al. (2023) provide lessons learned for improving communication when scrum teams became virtual due to the covid-19 pandemic as do reunamäki and fey (2023), who report solutions to communication issues in remote work in an agile organisation with non-it agile teams. espinosa et al. (2015) provide advice for managers on how to improve communication in follow-the-sun virtual teams, although not explicitly for software engineering teams. even studies of how to configure communication tools in large collocated teams can provide practical ideas that might be appropriate for gd software development environments (calefato et al., 2020). 6 discussion this study has addressed the question of what effective communication is in gd scrum teams. the findings are based on the understanding and experiences of industry professionals conveyed in interviews. qualitative content analysis was used to analyse the interview transcripts leading to two contributions, a theoretical model of effective communication in gd scrum teams, and practical guidelines for improving and sustaining effective communication. each model concept and each item of practical guidance is supported with relevant literature, wherever possible. the first contribution of this study is a theoretical model of effective communication in gd scrum teams. the model has four communication concepts. three concepts are about conveying information, namely communication transparency, communication quality, and communication discipline, and the fourth concept is alignment (a shared mental model at the team level), which represents the convergence or outcome of these communication efforts. we have argued that communication transparency, quality, and discipline are likely to be necessary to achieve alignment. alignment contributes to successful agile projects in collocated projects (schmidt et al., 2014; yu & petter, 2014) but it is not clear how such projects develop in distributed software engineering environments (espinosa et al., 2007; florea & stoica, 2019). our model contributes to clarity by defining and providing empirical evidence for three antecedents to alignment and by arguing that alignment is equivalent to effective communication in gd scrum teams. although this model includes concepts that occur in other studies of communication, in our model they are based on empirical evidence and supporting literature, and assembled uniquely, making a novel contribution to understanding effective communication in gd scrum teams. the claim that alignment is equivalent to effective communication in gd scrum australasian journal of information systems kostin & strode 2023, vol 27, selected papers from acis 2022 effective communication in globally distributed scrum 28 teams has not been made previously in the context of software development, as far as we are aware. knowledge sharing is frequently referred to in virtual software engineering studies (giuffrida & dittrich, 2015; kiely et al., 2022; šmite et al., 2021) but our model takes this idea further by explaining how to share knowledge effectively and what the outcome of effective sharing is, which is alignment. in section 4 where we defined and described the model, we already discussed literature related to each concept. we found that the concepts alignment and transparency are not unambiguous and unique because they are defined and discussed in both organisational (christensen & cheney, 2015) and scrum literature (schwaber & sutherland, 2020). communication quality and discipline appear to be unique concepts that do not occur in prior studies of software development communication. therefore, these concepts may be more important for gd scrum communication than in other contexts. this could be because communicators need to be more aware of communication issues and more thoughtful in taking actions to improve quality (e.g., with concise messages, speedy responses, and patience to ensure other parties have time to respond), and discipline (e.g., by following norms and using appropriate communication tools for the task). the literature review did not identify any specific model of communication for collocated agile software development or scrum with which we could compare our model. however, two gd agile software development studies address communication effectiveness. alzoubi and gill (2020) defined communication effectiveness when studying geographically distributed agile development (see section 2 literature review). they conceptualised communication effectiveness as a construct with task clarity and information sufficiency, detail, and accuracy as reflective measurement items. our communication transparency (i.e., clear communication) concept includes the idea of ‘task clarity’ but otherwise their construct is not directly comparable with our theoretical concepts. the closest model to our own is proposed by bhatti and ahsan (2017) (see section 2 literature review). their focus was effective communication in gd software development and was not restricted to gd scrum teams. their model focuses on practices to support effective communication, whereas our model focuses on abstract communication concepts. for example, transparency and quality have no place in their framework, although our discipline concept includes the idea of using appropriate communication tools as does their model. another difference is that bhatti and ahsan (2017) focused on communication with external stakeholders whereas our focus was on the gd scrum team. this difference might be explained by the nature of scrum and other agile approaches, where there are no direct managers and the key stakeholder, (i.e., the product owner) is considered a team member. our model of communication effectiveness in gd scrum teams has implications for studies of communication, coordination, and collaboration in gd scrum and agile software development. communication facilitates coordination and collaboration (sharp & robinson, 2010); therefore, our model could contribute to studies of coordination and collaboration in gd scrum teams by treating alignment as an antecedent to effective coordination and collaboration. similarly, alignment might also contribute to quantitative studies of agile team success, effectiveness, or performance because effective communication is considered to contribute to these factors (alzoubi & gill, 2020). for practitioners such as scrum masters, agile project managers, coaches, and other professionals in scrum teams this model shows that effective communication is achieved by australasian journal of information systems kostin & strode 2023, vol 27, selected papers from acis 2022 effective communication in globally distributed scrum 29 communicating with transparency, quality, and discipline. practitioners might use this knowledge to select agile practices, or other practices, that tend to support transparency, quality, and discipline. they might also want to evaluate the quality of the inter and intrateam mental model (alignment) at intervals as this is an indicator of a team’s effectiveness (strode et al., 2022). the second contribution of our study is practical guidance for people involved in globally distributed scrum teams. we have identified 11 items that can improve or sustain effective communication in this context. two items explicitly suggested using agile practices to enhance communication, two items have been explored in other agile studies (build relationships and provide safe environments for communication), and the remainder are not related to published agile practices. these items emerged directly from the advice of the participants and can be adopted directly by practitioners working in this context. 6.1 limitations the study has limitations. the model and practical guidance are based on evidence from a small number of english-speaking participants selected from a single source, linkedin™. this limitation affects the generalisability (external validity or transferability (miles & huberman, 1994)) of the model and the practical advice. this issue was mitigated by the application of specific selection criteria (see appendix a) when selecting participants. we ensured that all participants were experts on the topic because they were involved closely in gd scrum teams, were involved in managing communication, and had multiple experiences across several countries. furthermore, to improve generalisability, we have discussed supporting literature for each concept in the model and their relationships. however, the model would benefit from large-scale field testing to validate its applicability to all gd scrum teams, either with a survey or multiple case studies. for the practical advice, we have mitigated the lack of generalisability somewhat by indicating research that supports our items and suggested further appropriate sources of guidance. the study had limitations that affected its internal validity (credibility or authenticity (miles & huberman, 1994)). the exclusion of developers from the study has introduced potential bias because developers might have different experiences of communication. we had a specific reason for this omission since our experiences and research indicated that certain team member roles (i.e., scrum masters, agile coaches) often take responsibility for organising communication between distributed teams (kristensen & paasivaara, 2021; srivastava & jain, 2017) and so would provide an overview as well as specific examples of communication. this limitation should be addressed in future work to ensure that all roles that occur in development teams (such as programmers, testers, and product owners) are included in studies of gd communication. the limitation of using online interviews may have reduced the richness of the data, thus reducing credibility. all interviews were remote due to covid-19 pandemic restrictions and the geographical distribution of participants. online interviews meant the interviewer may have missed cues that in-person interviews could reveal, although we did ask the participants for examples if their statements were not specific or unclear. interviews can also suffer from researcher reactivity bias, which occurs when participants are encouraged to offer a particular opinion or are asked leading questions. this issue was mitigated by designing an interview schedule (see appendix b) that was followed in each interview. the questions were openended, but the researcher was prepared to ask further questions, that were not predesigned, australasian journal of information systems kostin & strode 2023, vol 27, selected papers from acis 2022 effective communication in globally distributed scrum 30 to draw out additional details, explanations, or examples from the interviewee. there is also an issue of ‘textbook answers’, that is, when studying agile topics, the interviewer has to be aware of responses that are ‘from the textbook’ or recalled from a participant's recent training and are not the actual experiences of the participant. this issue can only be identified when the researcher is experienced enough to detect this type of response. our interviewer was a certified and experienced scrum master and project manager who was able to ask participants for specific examples to address this issue when it arose. a further limitation was the lack of member checking, which means that the words or intent of the participants could have been misinterpreted. this is a threat to external reliability or confirmability (miles & huberman, 1994). although the transcripts were fully analysed by a single researcher and then checked and discussed by another researcher this did not necessarily remove the risk of misinterpretation. 6.2 future work avenues for future work on communication in gd scrum could address these issues. are the model concepts necessary and sufficient? large-scale field studies are needed to clarify if all three concepts are strictly necessary or if one has a more significant influence than the others. further research could investigate if these three concepts are sufficient to achieve high levels of alignment or if there are other concepts involved that we did not find. can alignment be decomposed further? different types of mental models exist, some might be relevant only to software development teams (e.g., the software architecture and team member technical expertise) (mohammed et al., 2010), it would be beneficial to understand which mental models matter most in gd scrum. furthermore, the relationship between intra and inter-team mental models might make a valuable extension to the model. does the model account for communication in agile contexts other than gd scrum teams? for example, in small-scale collocated scrum, small-scale virtual scrum (i.e., in work-from-home arrangements), in large-scale multi-team software development (dingsøyr et al., 2014), or in any contexts where the global characteristics of time zone and language differences are absent. exactly what scrum practices, sets of practices, or non-agile practices best support each communication model concept? a mapping of practices to concepts would better link this model to practice. this would help practitioners to select agile practices that have the most effect on communication. finally, our practical guidelines could be extended by assembling the research-based advice and lessons learned on communication in gd scrum teams into a comprehensive set of guidelines for practitioners. 7 conclusion the article makes two contributions to a better understanding of what effective communication means in globally distributed (gd) scrum teams. the first contribution is a theoretical model of communication effectiveness in gd scrum teams based on interviews and supporting literature. the model has four elements: communication transparency, communication quality, and communication discipline, which contribute to alignment. the model equates alignment with effective communication, which is a unique aspect of the model. in addition, our findings indicate that communication quality and discipline may be uniquely australasian journal of information systems kostin & strode 2023, vol 27, selected papers from acis 2022 effective communication in globally distributed scrum 31 important for gd scrum teams. this model can help researchers evaluate effective communication in gd teams that are using scrum because the model defines and describes what effective communication means in that context. the second contribution of this study is practical guidance for gd scrum practitioners on how to improve or sustain effective communication. this list of 11 items is based on interviews supported by existing research. this article also points to potential areas of future work that could enhance the communication model and strengthen the practical guidance. communication models and practical guidance for gd scrum and agile contexts are important for project and team success. this knowledge needs constant refreshing because communication tools improve rapidly and can change the way people communicate, meaning that communication models and advice in one decade may differ in the next decade. however, even if communication tools begin to closely mimic in-person information sharing, the need to understand, respect, and accommodate cultural differences, adapt to communication norms, and accommodate time zone differences is likely to remain. references adkins, l. (2010). coaching agile teams: a companion for scrum masters, agile coaches, and project managers in transition. boston, ma, usa: addison-wesley professional. ågren, p., knoph, e., & berntsson svensson, r. (2022). agile software development one year into the covid-19 pandemic. empirical software engineering, 27(6), 121. https://doi.org/10.1007/s10664-022-10176-9 ahmad, m. o., lenarduzzi, v., oivo, m., & taibi, d. (2018). lessons learned on communication channels and practices in agile software development. presented at the federated conference on computer science and information systems (fedcsis), september 9-12, 2018, poznan, poland. https://ieeexplore.ieee.org/abstract/document/8511256 alzoubi, y. i., & gill, a. q. (2020). an empirical investigation of geographically distributed agile development: the agile enterprise architecture is a communication enabler. ieee access, 8, 80269-80289. https://doi.org/10.1109/access.2020.2990389 alzoubi, y. i., gill, a. q., & al-ani, a. (2016). empirical studies of geographically distributed agile development communication challenges: a systematic review. information & management, 53(1), 22-37. https://doi.org/10.1016/j.im.2015.08.003 amar, h., rafi-ul-shan, p. m., & adegbile, a. (2019). towards a 5c theory of communication for scrum-based distributed projects. in proceedings of the british academy of management conference, bam2019. british academy of management . aston university, birmingham, september 3-5 2019. https://irep.ntu.ac.uk/id/eprint/38776/ andriyani, y., hoda, r., & amor, r. (2017). reflection in agile retrospectives. in h. baumeister, h. lichter, & m. riebisch (eds.), proceedings of the 18th international conference on agile processes in software engineering and extreme programming, xp 2017 (pp. 3-19). cologne, germany, may 22-26, 2017 (pp. 3-19). berlin, germany: springer open. https://doi.org/10.1007/978-3-319-57633-6 anwar, r., rehman, m., wang, k. s., & hashmani, m. a. (2019). systematic literature review of knowledge sharing barriers and facilitators in global software development organizations using concept maps. ieee access, 7, 24231-24247. https://doi.org/10.1109/access.2019.2895690 australasian journal of information systems kostin & strode 2023, vol 27, selected papers from acis 2022 effective communication in globally distributed scrum 32 bablo, j., marcinkowski, b., & przybylek, a. (2023). overcoming challenges of virtual scrum teams: lessons learned through an action research study. in c. j. stettina, j. garbajosa, & p. kruchten (eds.), proceedings of the 24th international conference on agile software development, xp 2023 (pp. 34-49). amsterdam, the netherlands, june 13–16, 2023. berlin, germany: springer nature. https://doi.org/10.1007/978-3-031-33976-9 baham, c., & hirschheim, r. (2022). issues, challenges, and a proposed theoretical core of agile software development research. information systems journal, 32(1), 103-129. https://doi.org/10.1111/isj.12336 bastiaansen, c. a. j., & wilderom, c. p. m. (2021). agile and generic work values of british vs indian it workers: a culture-clash case. journal of strategy and management, 15(3), 353376. https://doi.org/10.1108/jsma-03-2021-0071 beck, k., beedle, m., van bennekum, a., cockburn, a., cunningham, w., fowler, m., grenning, j., highsmith, j., hunt, a., jeffries, r., kern, j., marick, b., martin, r. c., mellor, s., schwaber, k., sutherland, j., & thomas, d. (2001a). manifesto for agile software development. http://www.agilemanifesto.org beck, k., beedle, m., van bennekum, a., cockburn, a., cunningham, w., fowler, m., grenning, j., highsmith, j., hunt, a., jeffries, r., kern, j., marick, b., martin, r. c., mellor, s., schwaber, k., sutherland, j., & thomas, d. (2001b). principles behind the agile manifesto. http://agilemanifesto.org/principles.html berger, c. r. (2005). interpersonal communication: theoretical perspectives, future prospects. journal of communication, 55(3), 415-447. https://doi.org/10.1111/j.14602466.2005.tb02680.x bhatti, m. w., & ahsan, a. (2017). effective communication among globally distributed software development teams: development of an “effective communication” scale. journal of global information management, 25(3), 40-62. https://doi.org/10.4018/jgim.2017070103 bickman, l., & rog, d. j. (2009). applied research design: a practical approach. in l. bickman & d. j. rog (eds.), the sage handbook of applied social research methods (2 ed., pp. 3-34). new york, ny, usa: sage publications. bjarnason, e., smolander, k., engström, e., & runeson, p. (2016). a theory of distances in software engineering. information and software technology, 70, 204-219. https://doi.org/10.1016/j.infsof.2015.05.004 bjarnason, e., wnuk, k., & regnell, b. (2011). requirements are slipping through the gaps—a case study on causes & effects of communication gaps in large-scale software development. in j. mylopoulos (ed.), ieee 19th international requirements engineering conference, trento, italy. (pp. 37-46). ieee. https://doi.org/10.1109/re.2011.6051639 bundhun, k., & sungkur, r. k. (2021). developing a framework to overcome communication challenges in agile distributed teams–case study of a mauritian-based it service delivery centre. global transitions proceedings, 2(2), 315-322. https://doi.org/10.1016/j.gltp.2021.08.006 calefato, f., giove, a., lanubile, f., & losavio, m. (2020). a case study on tool support for collaboration in agile development. in p. tell, i. steinmacher, & r. britto (eds.), australasian journal of information systems kostin & strode 2023, vol 27, selected papers from acis 2022 effective communication in globally distributed scrum 33 proceedings of the 15th ieee/acm international conference on global software engineering, icgse '20. seoul, republic of korea, june 26 28, 2020 (pp. 11-21). acm. https://doi.org/10.1145/3372787.3390436 carmel, e., & agarwal, r. (2001). tactical approaches for alleviating distance in global software development. ieee software, 18(2), 22-29. https://doi.org/10.1109/52.914734 casey, v. (2010). developing trust in virtual software development teams. journal of theoretical and applied electronic commerce research, 5(2), 41-58. https://doi.org/10.4067/s071818762010000200004 chang, h. h., chuang, s.-s., & chao, s. h. (2011). determinants of cultural adaptation, communication quality, and trust in virtual teams' performance. total quality management, 22(3), 305-329. https://doi.org/10.1080/14783363.2010.532319 christensen, l. t., & cheney, g. (2015). peering into transparency: challenging ideals, proxies, and organizational practices. communication theory, 25(1), 70-90. https://doi.org/10.1111/comt.12052 ciancarini, p., farina, m., masyagin, s., succi, g., yermolaieva, s., & zagvozkina, n. (2021). non verbal communication in software engineering–an empirical study. ieee access, 9, 71942-71953. https://doi.org/10.1109/access.2021.3075983 cooren, f., kuhn, t., cornelissen, j. p., & clark, t. (2011). communication, organizing and organization: an overview and introduction to the special issue. organization studies, 32(9), 1149-1170. https://doi.org/10.1177/01708406114108 creswell, j. w., & poth, c. n. (2016). qualitative inquiry and research design: choosing among five approaches. new york, ny, usa: sage publications. defranco, j. f., & laplante, p. (2018). a software engineering team research mapping study. team performance management: an international journal, 24(3/4), 203-248. https://doi.org/10.1108/tpm-08-2017-0040 defranco, j. f., & laplante, p. a. (2017). review and analysis of software development team communication research. ieee transactions on professional communication, 60(2), 165-182. https://doi.org/10.1109/tpc.2017.2656626 dennis, a. r., fuller, r. m., & valacich, j. s. (2008). media, tasks, and communication processes: a theory of media synchronicity. mis quarterly, 32(3), 575-600. https://doi.org/10.2307/25148857 diegmann, p., dreesen, t., binzer, b., & rosenkranz, c. (2018). journey towards agility: three decades of research on agile information systems development. in proceedings of the 39th international conference on information systems. san francisco, california, usa, december 13-18, 2018. ais. https://aisel.aisnet.org/icis2018/management/presentations/4 digital.ai. (2022). 16th annual state of agile report. digital.ai. https://digital.ai/resourcecenter/analyst-reports/state-of-agile-report/ dingsøyr, t., fægri, t. e., & itkonen, j. (2014). what is large in large-scale? a taxonomy of scale for agile software development. in a. jedlitschka, p. kuvaja, m. kuhrmann, t. männistö, j. münch, & m. raatikainen (eds.), product-focused software process improvement. australasian journal of information systems kostin & strode 2023, vol 27, selected papers from acis 2022 effective communication in globally distributed scrum 34 profes 2014. lncs vol 8892. (pp. 273-276). berlin, germany: springer. https://doi.org/10.1007/978-3-319-13835-0_20 dorairaj, s., noble, j., & malik, p. (2011). effective communication in distributed agile software development teams. in a. sillitti, o. hazzan, e. bache, & x. albaladejo (eds.), proceedings of the international conference on agile software development: agile processes in software engineering and extreme programming, xp 2011. lnbip 77 (pp. 102-116). berlin, germany: springer. https://doi.org/10.1007/978-3-642-20677-1_8 drechsler, a., & breth, s. (2019). how to go global: a transformative process model for the transition towards globally distributed software development projects. international journal of project management, 37(8), 941-955. https://doi.org/10.1016/j.ijproman.2019.08.003 dubin, r. (1978). theory building. los angles, ca, usa: the free press. ebert, c., & paasivaara, m. (2017). scaling agile. ieee software, 34(6), 98-103. https://doi.org/10.1109/icgse.2016.34. eckstein, j. (2013). agile software development in the large: diving into the deep. boston, ma, usa: addison-wesley. edmondson, a. (1999). psychological safety and learning behavior in work teams. administrative science quarterly, 44(2), 350-383. https://doi.org/https://doi.org/10.2307/2666999 edmondson, a. c., & mcmanus, s. e. (2007). methodological fit in management field research. academy of management review, 32(4), 1155-1179. https://doi.org/10.5465/amr.2007.26586086 edmondson, s., & chiu, y.-t. (2020). exploring the impact of mental models on scrum outcomes: insights from scrum teams in new zealand government. in proceedings of the australasian conference on information systems, acis 2020 (pp. 1-12). ais. https://aisel.aisnet.org/acis2020/25 eisenhardt, k. m., & graebner, m. e. (2007). theory building from cases: opportunities and challenges. academy of management journal, 50(1), 25-32. https://doi.org/10.5465/amj.2007.24160888 espinosa, a. j., kraut, r. e., lerch, j. f., slaughter, s. a., herbsleb, j. d., & mockus, a. (2001). shared mental models and coordination in large-scale, distributed software development. in proceedings of the 22nd international conference on information systems, icis, paper 64 (pp. 513-518). december 16-19, 2001, new orleans, louisiana, usa. ais. http://aisel.aisnet.org/icis2001/64 espinosa, a. j., slaughter, s. a., kraut, r. e., & herbsleb, j. d. (2007). team knowledge and coordination in geographically distributed software development. journal of management information systems, 24(1), 135-169. https://doi.org/10.2753/mis0742-1222240104 espinosa, j. a., & carmel, e. (2003). the impact of time separation on coordination in global software teams: a conceptual foundation. software process: improvement and practice, 8(4), 249-266. https://doi.org/10.1002/spip.185 australasian journal of information systems kostin & strode 2023, vol 27, selected papers from acis 2022 effective communication in globally distributed scrum 35 espinosa, j. a., nan, n., & carmel, e. (2015). temporal distance, communication patterns, and task performance in teams. journal of management information systems, 32(1), 151-191. https://doi.org/10.1080/07421222.2015.1029390 fawole, o. a., dy, s. m., wilson, r. f., lau, b. d., martinez, k. a., apostol, c. c., vollenweider, d., bass, e. b., & aslakson, r. a. (2013). a systematic review of communication quality improvement interventions for patients with advanced and serious illness. journal of general internal medicine, 28, 570-577. https://doi.org/10.1007/s11606-012-2204-4 florea, l., & stoica, m. (2019). exploration of team mental model characteristics to identify opportunities and hindrances in global virtual teamwork. management revue, 30(1), 93119. https://www.jstor.org/stable/26869687 giuffrida, r., & dittrich, y. (2015). a conceptual framework to study the role of communication through social software for coordination in globally-distributed software teams. information and software technology, 63, 11-30. https://doi.org/10.1016/j.infsof.2015.02.013 gonzález-romá, v., & hernández, a. (2014). climate uniformity: its influence on team communication quality, task conflict, and team performance. journal of applied psychology, 99(6), 1042-1058. https://doi.org/10.1037/a0037868 gorard, s. (2013). research design: creating robust approaches for the social sciences. research design, 1-232. https://doi.org/10.4135/9781526431486 gregor, s. (2006). the nature of theory in information systems. mis quarterly, 30(3), 611-642. gregory, p., strode, d. e., sharp, h., & barroca, l. (2022). an onboarding model for integrating newcomers into agile project teams. information and software technology, 143, 1-14. https://doi.org/10.1016/j.infsof.2021.106792 griffiths, m., fewell, m., slusanschi, h., matola, s., rothman, j., hartman, b., & kauffman, b. (2017). agile practice guide. atlanta, ga, usa: project management institute. hall, j. a., horgan, t. g., & murphy, n. a. (2019). nonverbal communication. annual review of psychology, 70, 271-294. https://doi.org/10.1146/annurev-psych-010418-103145 harris, t. e., & sherblom, j. c. (2018). small group and team communication. long grove, il, usa: waveland press. hennel, p., & rosenkranz, c. (2021). investigating the “socio” in socio-technical development: the case for psychological safety in agile information systems development. project management journal, 52(1), 11-30. https://doi.org/10.1177/8756972820933057 herbsleb, j. d., & mockus, a. (2003). an empirical study of speed and communication in globally distributed software development. ieee transactions on software engineering, 29(6), 481-494. https://doi.org/10.1109/tse.2003.1205177 hidayati, a., budiardjo, e. k., & purwandari, b. (2020). hard and soft skills for scrum global software development teams. in k. eko, k. qin, & p. lajbcygier (eds.), proceedings of the 3rd international conference on software engineering and information management, icsim '20 (pp. 110-114). https://doi.org/10.1145/3378936.3378966 australasian journal of information systems kostin & strode 2023, vol 27, selected papers from acis 2022 effective communication in globally distributed scrum 36 hoda, r., salleh, n., grundy, j., & tee, h. m. (2017). systematic literature reviews in agile software development: a tertiary study. information and software technology, 85, 60-70. https://doi.org/10.1016/j.infsof.2017.01.007 hoffmann, m., mendez, d., fagerholm, f., & luckhardt, a. (2022). the human side of software engineering teams: an investigation of contemporary challenges. ieee transactions on software engineering, 49(1), 211-225. https://doi.org/10.1109/tse.2022.3148539 hummel, m., rosenkranz, c., & holten, r. (2013). the role of communication in agile systems development. business & information systems engineering, 5(1), 343-355. https://doi.org/10.1007/s12599-013-0282-4 jackson, v., van der hoek, a., prikladnicki, r., & ebert, c. (2022). collaboration tools for developers. ieee software, 39(2), 7-15. https://doi.org/10.1109/ms.2021.3132137 kadenic, m. d., de jesus pacheco, d. a., koumaditis, k., tjørnehøj, g., & tambo, t. (2023). investigating the role of product owner in scrum teams: differentiation between organisational and individual impacts and opportunities. journal of systems and software, 206, 111841. https://doi.org/10.1016/j.jss.2023.111841 kelly, a. (2019). the art of agile product ownership. berlin, germany: springer link. kiely, g., butler, t., & finnegan, p. (2022). global virtual teams coordination mechanisms: building theory from research in software development. behaviour & information technology, 41(9), 1952-1972. https://doi.org/10.1080/0144929x.2021.1909141 kipfelsberger, p., raes, a., herhausen, d., kark, r., & bruch, h. (2022). start with why: the transfer of work meaningfulness from leaders to followers and the role of dyadic tenure. journal of organizational behavior, 43(8), 1287-1309. https://doi.org/10.1002/job.2649 kittelman, s., calvo-amodio, j., & martínez león, h. c. (2018). a systems analysis of communication: defining the nature of and principles for communication within human activity systems. systems research and behavioral science, 35(5), 520-537. https://doi.org/10.1002/sres.2562 kopczyńska, s., ochodek, m., piechowiak, j., & nawrocki, j. (2022). on the benefits and problems related to using definition of done—a survey study. journal of systems and software, 193, 111479. https://doi.org/10.1016/j.jss.2022.111479 kostin, d. (2021). effective communication in globally distributed scrum teams [unpublished master's thesis]. whitireia polytechnic, new zealand. kostin, d., & strode, d. (2022) effective communication in globally distributed scrum teams. proceedings of the australasian conference on information systems, acis 2022. https://aisel.aisnet.org/acis2022/5 kristensen, s. h., & paasivaara, m. (2021). what added value does a scrum master bring to the organisation?—a case study at nordea. in m. t. baldassarre, g. scanniello, & a. skavhaug (eds.), 47th euromicro conference on software engineering and advanced applications (seaa) (pp. 270-278). palermo, italy, september 1-3, 2021. ieee. https://doi.org/10.1109/seaa53835.2021.00041 lampropoulos, g., & siakas, k. (2018). communication in distributed agile software development: impact of social media–social networking. in p. marchbank & g. staples australasian journal of information systems kostin & strode 2023, vol 27, selected papers from acis 2022 effective communication in globally distributed scrum 37 (eds.), proceedings of the twenty sixth international conference on computing and quality sqm (pp. 43-60). southampton solent university, uk. lechner, a., & tobias mortlock, j. (2022). how to create psychological safety in virtual teams. organizational dynamics, 51(2). https://doi.org/10.1016/j.orgdyn.2021.100849 leonardi, p. m. (2014). social media, knowledge sharing, and innovation: toward a theory of communication visibility. information systems research, 25(4), 796-816. https://doi.org/10.1287/isre.2014.0536 lukusa, l., geeling, s., lusinga, s., & rivett, u. (2021). teamwork and project success in agile software development methods: a case study in higher education. in f. j. garcíapeñalvo & a. garcía-holgado (eds.), eighth international conference on technological ecosystems for enhancing multiculturality, teem'20, salamanca, spain (pp. 885-891). acm. https://doi.org/10.1145/3434780.3436648 marinho, m., camara, r., & sampaio, s. (2021). toward unveiling how safe framework supports agile in global software development. ieee access, 9, 109671-109692. https://doi.org/10.1109/access.2021.3101963 marlow, s. l., lacerenza, c. n., paoletti, j., burke, c. s., & salas, e. (2018). does team communication represent a one-size-fits-all approach?: a meta-analysis of team communication and performance. organizational behavior and human decision processes, 144, 145-170. https://doi.org/10.1016/j.obhdp.2017.08.001 maruping, l. m., & matook, s. (2020). the multiplex nature of the customer representative role in agile information systems development. mis quarterly, 44(3). https://doi.org/10.25300/misq/2020/12284 mcchesney, i. r., & gallagher, s. (2004). communication and co-ordination practices in software engineering projects. information and software technology, 46(7), 473-489. https://doi.org/10.1016/j.infsof.2003.10.001 miles, m. b., & huberman, a. m. (1994). qualitative data analysis (2 ed.). new york, ny, usa: sage publications. mishra, d., mishra, a., & ostrovska, s. (2012). impact of physical ambiance on communication, collaboration and coordination in agile software development: an empirical evaluation. information and software technology, 54(10), 1067-1078. https://doi.org/10.1016/j.infsof.2012.04.002 mohammed, s., ferzandi, l., & hamilton, k. (2010). metaphor no more: a 15-year review of the team mental model construct. journal of management, 36(4), 876-910. https://doi.org/10.1177/0149206309356804 mohr, j. j., & sohi, r. s. (1995). communication flows in distribution channels: impact on assessments of communication quality and satisfaction. journal of retailing, 71(4), 393415. https://doi.org/10.1016/0022-4359(95)90020-9 morrison-smith, s., & ruiz, j. (2020). challenges and barriers in virtual teams: a literature review. sn applied sciences, 2, 1-33. https://doi.org/10.1007/s42452-020-2801-5 australasian journal of information systems kostin & strode 2023, vol 27, selected papers from acis 2022 effective communication in globally distributed scrum 38 newell, s., david, g., & chand, d. (2007). an analysis of trust among globally distributed work teams in an organizational setting. knowledge and process management, 14(3), 158168. https://doi.org/10.1002/kpm.284 orpen, c. (1997). the interactive effects of communication quality and job involvement on managerial job satisfaction and work motivation. the journal of psychology, 131(5), 519522. https://doi.org/10.1080/00223989709603540 paasivaara, m., durasiewicz, s., & lassenius, c. (2008). distributed agile development: using scrum in a large project. in f. lanubile & b. sengupta (eds.), proceedings of the ieee international conference on global software engineering, bangalore, india. (pp. 87-95). ieee. https://doi.org/10.1109/icgse.2008.38 petter, s., & randolph, a. b. (2009). developing soft skills to manage user expectations in it projects: knowledge reuse among it project managers. project management journal, 40(4), 45-59. https://doi.org/10.1002/pmj.20 pikkarainen, m., haikara, j., salo, o., abrahamsson, p., & still, j. (2008). the impact of agile practices on communication in software development. empirical software engineering, 13(3), 303-337. https://doi.org/10.1007/s10664-008-9065-9 pmi. (2021). the standard for project management and a guide to the project management body of knowledge (pmbok guide) (7 ed.). atalanta, ga, usa: project management institute. poole, m. s. (1999). group communication theory. in l. r. frey (ed.), the handbook of group communication theory and research (pp. 37-70). new york, ny, usa: sage publications. pooley, j. d. (ed.). (2016). communication theory and the disciplines. hoboken, nj, usa: john wiley & sons, inc. https://doi.org/10.1002/9781118766804.wbiect261. reunamäki, r., & fey, c. f. (2023). remote agile: problems, solutions, and pitfalls to avoid. business horizons, 66(4), 505-516. https://doi.org/10.1016/j.bushor.2022.10.003 salas, e., sims, d. e., & burke, c. s. (2005). is there a 'big five" in teamwork? small group research, 36(5), 555-599. https://doi.org/10.1177/1046496405277134 sallin, m., & kropp, m. (2022). benefits of card walls in agile software development: a systematic literature review. in v. stray, k. j. stol, m. paasivaara, & p. kruchten (eds.), proceedings of the international conference on agile processes in software engineering and extreme programming. xp 2022. lecture notes in business information processing, vol 445. (pp. 3-18). berlin, germany: springer. https://doi.org/10.1007/978-3-031-08169-9_1 sarker, s., xiao, x., & beaulieu, t. (2013). guest editorial: qualitative studies in information systems: a critical review and some guiding principles. mis quarterly, 37(4), iii-xviii. schmidt, c., kude, t., heinzl, a., & mithas, s. (2014). how agile practices influence the performance of software development teams: the role of shared mental models and backup. 35th international conference on information systems, icis 2014, auckland, new zealand, https://aisel.aisnet.org/icis2014/proceedings/projectmanagement/15 schreier, m. (2014). qualitative content analysis. in u. flick (ed.), the sage handbook of qualitative data analysis (pp. 170-184). new york, ny, usa: sage publications. https://doi.org/10.4135/9781446282243.n12 australasian journal of information systems kostin & strode 2023, vol 27, selected papers from acis 2022 effective communication in globally distributed scrum 39 schwaber, k. (1997). scrum development process. in j. sutherland, c. casanave, j. miller, p. patel, & g. hollowell (eds.), business object design and implementation (pp. 117-134). london, uk: springer. https://doi.org/10.1007/978-1-4471-0947-1_11 schwaber, k., & beedle, m. (2002). agile software development with scrum. hoboken, nj, usa: prentice hall. schwaber, k., & sutherland, j. (2020). the scrum guide. scrumguides.org. https://www.scrumguides.org/docs/scrumguide/v2020/2020-scrum-guide-us.pdf sharp, h., & robinson, h. (2010). three 'c's of agile practice: collaboration, coordination and communication. in t. dingsoyr, t. dyba, & n. b. moe (eds.), agile software development: current research and future directions. berlin, germany: springer. https://doi.org/10.1007/978-3-642-12575-1 sinek, s. (2011). start with why: how great leaders inspire everyone to take action. new york, ny, usa: penguin. smart, j. (2018). to transform to have agility, dont do a capital a, capital t agile transformation. ieee software, 35(06), 56-60. https://doi.org/10.1109/ms.2018.4321245 šmite, d., moe, n. b., & gonzalez-huerta, j. (2021). overcoming cultural barriers to being agile in distributed teams. information and software technology, 138, 106612. https://doi.org/10.1016/j.infsof.2021.106612 srivastava, p., & jain, s. (2017). a leadership framework for distributed self-organized scrum teams. team performance management, 23(5/6), 293-314. https://doi.org/10.1108/tpm-062016-0033 stavru, s. (2014). a critical examination of recent industrial surveys on agile method usage. journal of systems and software, 94, 87-97. https://doi.org/10.1016/j.jss.2014.03.041 stray, v., fægri, t. e., & moe, n. b. (2016). exploring norms in agile software teams. in p. abrahamsson, a. jedlitschka, a. nguyen duc, m. felderer, s. amasaki, & t. mikkonen (eds.), proceedings of the international conference on product-focused software process improvement. profes 2016. lecture notes in computer science, vol 10027. (pp. 458-467). springer. https://doi.org/10.1007/978-3-319-49094-6_31 stray, v., & moe, n. b. (2020). understanding coordination in global software engineering: a mixed-methods study on the use of meetings and slack. journal of systems and software, 170, 110717. https://doi.org/10.1016/j.jss.2020.110717 stray, v. g., lindsjørn, y., & sjøberg, d. i. (2013). obstacles to efficient daily meetings in agile development projects: a case study. in 2013 acm/ieee international symposium on empirical software engineering and measurement (pp. 95-102). ieee. https://doi.org/10.1109/esem.2013.30 strode, d. e., dingsøyr, t., & lindsjorn, y. (2022). a teamwork effectiveness model for agile software development. empirical software engineering, 27(56), 50. https://doi.org/10.1007/s10664-021-10115-0 thorgren, s., & caiman, e. (2019). the role of psychological safety in implementing agile methods across cultures. research-technology management, 62(2), 31-39. https://doi.org/10.1080/08956308.2019.1563436 australasian journal of information systems kostin & strode 2023, vol 27, selected papers from acis 2022 effective communication in globally distributed scrum 40 tyagi, s., sibal, r., & suri, b. (2022). empirically developed framework for building trust in distributed agile teams. information and software technology, 145, 106828. https://doi.org/10.1016/j.infsof.2022.106828 vallon, r., da silva estácio, b. j., prikladnicki, r., & grechenig, t. (2018). systematic literature review on agile practices in global software development. information and software technology, 96, 161-180. https://doi.org/10.1016/j.infsof.2017.12.004 wachsmuth, i., de ruiter, j., jaecks, p., & kopp, s. (2013). alignment in communication: towards a new theory of communication (vol. 6). amsterdam, the netherlands: john benjamins publishing. weber, r. (2012). evaluating and developing theories in the information systems discipline. journal of the association for information systems, 13(1), 1-30. https://doi.org/10.17705/1jais.00284 wilkinson, n. l., & moran, j. w. (1998). team charter. the tqm magazine, 10(5), 355-361. https://doi.org/10.1108/09544789810231234 wolf, c., godulla, a., beck, l., & neubert, l. s. (2022). the role of internal communication in start-ups: state of research and practical approaches. international journal of strategic communication, 16(2), 255-272. https://doi.org/10.1080/1553118x.2021.2023544 yagüe, a., garbajosa, j., díaz, j., & gonzález, e. (2016). an exploratory study in communication in agile global software development. computer standards & interfaces, 48, 184-197. https://doi.org/10.1016/j.csi.2016.06.002 yates, j., & orlikowski, w. (2002). genre systems: structuring interaction through communicative norms. the journal of business communication (1973), 39(1), 13-35. https://doi.org/10.1177/002194360203900102 yu, x., & petter, s. (2014). understanding agile software development practices using shared mental models theory. information and software technology, 56(8), 911-921. https://doi.org/10.1016/j.infsof.2014.02.010 yue, c. a., men, l. r., & ferguson, m. a. (2019). bridging transformational leadership, transparent communication, and employee openness to change: the mediating role of trust. public relations review, 45(3), 101779. https://doi.org/10.1016/j.pubrev.2019.04.012 australasian journal of information systems kostin & strode 2023, vol 27, selected papers from acis 2022 effective communication in globally distributed scrum 41 appendix a: participant selection questionnaire the questionnaire in table 1 was used to check that the participant was suitably qualified and experienced in communication in globally distributed scrum teams. question 1 name 2 do you have experience in the coordination of globally distributed scrum team(s), including managing the team's remote communication? • note: you can answer "yes" if at least one team member was located in another country while scrum practices were applied. 3 please choose the scrum practices you use/used with the globally distributed scrum team (gdst) • "sprint" concept • sprint demo • "sprint backlog" concept • "product backlog" concept • daily scrum meetings • scrum master role in the team • product owner role in the team 4 how many years of such experience have you had? • less than 1 year • 1-3 years • 3 years or more 5 what is/was your official position in the organisation while coordinating gdst? 6 what country and city do/did you live in (when participating in the gdst) and where the headquarters of your organisation is/was located (while participating in gdst)? table 3. selection questionnaire appendix b: semi-structured interview schedule the questions in table 4 show the interview questions and the purpose of each question. interview question purpose 1 what is your role in the organisation (i.e., what work do you carry out)? to learn about the scrum role and how it relates to the official position 2 how is your globally distributed scrum team organised (where are the developers, where are the managers, where are the customers located)? to understand how the team is distributed and realise the potential challenges in communication 3 what type of software tools do you use in your team for communicating? do you find them effective and why or why not? to understand what tools contribute to communication and how effective they are for the people 4 based on your experience, what scrum practices do you consider to be effective in a globally distributed scrum team? to understand what scrum practices are effective in a gdst 5 based on your experience, what are the main communication challenges in globally distributed scrum teams? to realise the main challenges that prevent effective communication australasian journal of information systems kostin & strode 2023, vol 27, selected papers from acis 2022 effective communication in globally distributed scrum 42 6 how have you resolved these challenges? what challenges have you not been able to resolve? to get recommendations to improve communication quality 7 how do you define effective communication in your projects? and why? to understand the meaning of effective communication 8 how do you keep communication effective in gdst? to get recommendations to keep communication effective 9 are the challenges the only reason for non-effective communication in gdst? to understand other potential reasons for noneffective communication table 4. interview schedule copyright: © 2023 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v27i0.4501 http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 literature review: communication 2.1 communication in software development 2.2 communication in globally distributed software development 2.3 communication in globally distributed scrum teams 3 method 3.1 interview design 3.2 sample selection and recruitment 3.3 data collection 3.4 data analysis 3.5 validity considerations 3.6 participant profiles 4 a model of effective communication in gd scrum teams 4.1 alignment 4.2 communication transparency 4.3 communication quality 4.4 communication discipline 4.5 communication model relationships 5 practical guidance for improving and sustaining communication 5.1 always explain why 5.2 convey and follow through on one message 5.3 build relationships 5.4 hold a team charter session 5.5 create a safe environment for communicating 5.6 model good communication behaviour 5.7 respect personal boundaries when communicating across time zones 5.8 consider personal communication preference mismatches 5.9 use agile practices for communication 5.10 look for the root causes of problematic communication 5.11 allow for reduced body language signals in virtual communication 6 discussion 6.1 limitations 6.2 future work 7 conclusion paper title (paper title style) australasian journal of information systems drechsler & doyle 2025, vol 29, scholarship of is teaching and learning hyflex course architecture conceptual modelling 1 received 04 mar 2025, accepted 03 aug 2025 after 1 round of review with stuart black as accepting senior editor, michael davern and stuart black as editors in chief a comprehensive hybrid and flexible (‘hyflex’) course architecture for conceptual modelling courses in information systems andreas drechsler school of information management victoria university of wellington wellington, new zealand email: andreas.drechsler@vuw.ac.nz cathal doyle school of information management victoria university of wellington wellington, new zealand abstract 'hyflex' is a catchword for course designs in higher education that provide students with the opportunity to attend course sessions in-person as well as remotely ('hybrid') and to change the mode of attending every week based on their circumstances and preferences ('flexible'). due to the need to teach the complex skill of conceptual enterprise modelling (cem) through practical exercises and feedback, hyflex course designs for cem courses pose several challenges to typical cem-related learning outcomes. this paper proposes a set of design requirements, principles, and features for a comprehensive hyflex course architecture for cem courses. the design requirements, principles, and features evolved through and are evaluated against several iterations of cem courses across two different programmes (undergraduate and postgraduate), student numbers (ranging from 17 to 121), settings (lecture & workshops, block mode, etc.), and course topics (business process and enterprise architecture modelling). the student performances and course evaluations indicate that the offered courses that instantiated the presented course architecture have been both effective and appreciated by the students. other instructors can draw on our course architecture to design or adapt their own cem courses – but also, to an extent, courses with other topics – to the hyflex paradigm. keywords teaching, hyflex, hybrid course, flexible course, conceptual modelling. 1 introduction ‘hyflex’ – a portmanteau of ‘hybrid’ and ‘flexible’ – is an umbrella term used by beatty (2019) for course designs that enable students to choose flexibly between in-person and remote participation modes throughout a course, without missing out on content or engagement opportunities regardless of their chosen participation mode in a given week. this inherent flexibility offers further advantages, such as reaching a wider range of students than those in a traditional full-time on-campus study mode and providing students additional flexibility in completing a course in light of changing life circumstances. simultaneously, hyflex courses australasian journal of information systems drechsler & doyle 2025, vol 29, scholarship of is teaching and learning hyflex course architecture conceptual modelling 2 tend to require a higher effort on the faculty and administrative side regarding preparation and delivery compared to traditional on-campus teaching arrangements (beatty, 2019). conceptual enterprise modelling (cem) courses – which cover subjects such as business process (bp), data, systems, or enterprise architecture (ea) analysis, modelling, and design – often incorporate ‘hands-on’ exercises with feedback cycles to help the students attaining modelling proficiency in the chosen conceptual modelling language (bandara et al., 2010; recker & rosemann, 2009). these feedback cycles can, for instance, take place in small tutorials accompanying a traditional weekly lecture or in flipped classroom settings (tanner & scott, 2015). in a hyflex course design, retaining these synchronous in-person interaction spaces can be a useful course design component, but alternative arrangements need to be made for students participating remotely. hence, hyflex usually cannot be just an ‘add-on’ to an existing cem course but requires a more fundamental rethink of the entire instructional design and the best ways of attaining the course learning objectives (beatty, 2019). the authors have been fortunate to have, at the time of writing, gone through several iterations of hyflex cem courses in the years 2020-2024 across two topics (bp and ea modelling) in two different information systems (is) programmes (undergraduate and professional master) at the same university. while during the global pandemic in-person instruction was possible and had taken place with very few exceptions from the second half of 2020 onwards, the university leadership also mandated until the end of 2022 that it shall be possible for all students to complete any course fully remotely. many course iterations had several students unable to attend any in-person sessions (e.g., due to being in different parts of the country or entirely different countries). hence, the authors chose to go for full hyflex course designs for their courses and have evolved the underlying course architecture over the years. this not only allowed the authors to adapt to changing circumstances over the subsequent years rather easily, they also have found that the hyflex conversion and refinement efforts led to an overall shift in teaching practices away from the traditional ‘course monolith’ where the main emphasis lies on shaping the form of synchronous in-person teaching and learning for a few hours each week (e.g., lecture and tutorial). our experiences and the corresponding reflections on our evolved teaching practices thus allow us to answer the following research question in this paper: what are design requirements, principles, and features for effective and appreciated hyflex courses on conceptual enterprise modelling topics? we understand effectiveness as the extent of students attaining the course learning objectives and use metrics such as final letter grades and their medians as proxies for measuring effectiveness, as these letter grades result from the assessment of the students’ submissions to assignments that are aligned to the course learning objectives. to measure appreciation, we use the scores and feedback taken from the aggregated results of the anonymous course and teaching evaluation surveys submitted by the students towards the end of the courses as proxies. in contrast to the four main – but, in the end, partial – themes (tool support, feedback for students, learning analytics, and gamification) in the teaching conceptual modelling literature identified by a structured analysis (rosenthal et al., 2019) we instead intend to contribute a holistic yet abstract and thus re-usable and adaptable course architecture with this paper. we use the term course architecture to indicate that we do not intend to provide a turnkey course design as our main contribution, or, in the words of nicolai and seidl (2010) directly applicable instrumental relevance. instead, we aim for conceptual relevance (ibid.) to guide and inspire australasian journal of information systems drechsler & doyle 2025, vol 29, scholarship of is teaching and learning hyflex course architecture conceptual modelling 3 educators to draw on our (more abstract) design principles and (more specific) design features for teaching cem in a hyflex setting when considering a (re)design their own cem – or possibly other – courses to make them partially or fully hyflex. we thus intend to contribute not explanatory knowledge for theoretical advancement, but prescriptive knowledge (or knowledge for design and action) (gregor, 2006; gregor & hevner, 2013) instead to improve the course design and teaching practices for higher education courses. we further deliberately avoid having a broader spectrum of topics for the courses that we cover, partially because the authors’ teaching is focused in the cem areas, but also to limit the variety of other factors affecting the course architecture and its outcomes. among the evaluation of the more abstract design principles, we nonetheless reflect on the projectability of these design principles into context beyond cem and hyflex. 2 conceptual foundations 2.1 conceptual enterprise modelling as part of information systems curricula traditionally, conceptual enterprise models have been defined as “a computational representation of the structure, activities, processes, information, resources, people, behaviour, goals and constraints of a business, government, or other enterprise.” (fox & grüninger, 1997). conceptual enterprise models typically represent an enterprise from different perspectives, from the as-is to the to-be, allowing for analysis of the current situation, consideration of future scenarios, and communicating with stakeholders (bjeković et al., 2014; fox & grüninger, 1997). the representative nature of a conceptual enterprise model is commonly conceptualised as a script that follows a certain grammar and is built following a certain method within a given context (recker et al., 2021; wand & weber, 2002). in an increasingly interconnected digital world, the scope of these models can extend beyond enterprises to business ecosystems (drews & schirmer, 2014) and other, non-work settings (or even parts of society). they further have the purpose not only to represent but also to shape the future physical as well as the digital reality, and to mediate between these two realms (recker et al., 2021). in other words, the role of a conceptual enterprise model is to help human as well as digital agents in a model-driven analysis, design, operation, and governance of enterprises and other socio-technical systems. different types of models exist with varying emphasises on sub-sets of these socio-technical system, including data, bp, or ea models. the task of creating conceptual enterprise models relies on the modellers’ constituent knowledge about and skills in executing the activities of understanding (the real-world application domain), conceptualising (formalising the parts of interest in the application domain), abstracting (from specific details and problems), defining (the main concepts to be used for modelling), constructing (the model itself as a simplified representation of the real world), evaluating (the model with respect to certain qualities), and refining (the model, based on the evaluation results) (thalheim, 2010). we would add communicating (to other modellers as well as to stakeholders) (hoppenbrouwers, 2012; hoppenbrouwers et al., 2005) and – in a somewhat extended perspective beyond ‘pure’ modelling – utilising (e.g., for analysing or governing the current or shaping the future physical and/or digital reality) as two other important tasks in the wider context of cem. australasian journal of information systems drechsler & doyle 2025, vol 29, scholarship of is teaching and learning hyflex course architecture conceptual modelling 4 cem courses are an integral part of recommended curriculums for information systems programmes (jung & lehrer, 2017; leidig & salmela, 2021) and also part of many university courses (eduglopedia, 2022) – here often as a means to an end to conduct e.g., business process management. in the competencies outlined in the is2020 competency model for undergraduate programs in information systems (leidig & salmela, 2021), cem-related competencies play a role in the three domains of development (i.e., conceptual system models as a foundation for systems development), organization (i.e., conceptual business models as a foundation for business process management), and data (i.e., conceptual data models as foundation for database and analytics systems). all nine skills discussed above are relevant in all these domains and thus cut across the competencies in the is2020 model curriculum. the authors had been teaching several courses on two specific cem forms (bp and ea modelling) for several years before transitioning to a hyflex course design. these courses have generally followed what can be considered a traditional approach to learning cem such as the one outlined by recker and rosemann (2009): weekly formal lectures (for two hours), weekly practical workshops (for one hour), and reflective and formative assessments (quizzes and two assignments in their example), the latter including group work. however, in order to develop the requirements of cem courses for a hyflex format more substantially, we first need to unpack the foundations of teaching cem as well as the hyflex foundations. 2.2 teaching the complex task of conceptual enterprise modelling while the act of cem can be analytically decomposed into the nine activities outlined in the previous section (understanding, conceptualising etc.), their application in practical cem (whether as part of exercises or a real-life modelling task), however, takes place in a varied and also often creative way that is more than the ‘sum of these (nine) parts’. trying to learn such ‘parts’ in isolation will most probably be ineffective for learners when the ultimate goal is to learn the complex task (kester & van merriënboer, 2021) of cem. hence, in this paper we forgo a comprehensive distinction of different knowledge types in the cem context as e.g., in bloom’s revised taxonomy (krathwohl, 2002) which is otherwise commonly used in education papers (see also darwazeh’s (2017) criticism of common applications of bloom’s taxonomy). instead, we put the complex task of cem at the centre of the desired learning outcomes and therefore understand teaching cem as an instance of ‘complex learning’ in the sense of kester & van merriënboer (2021) and their four-component instructional design (ic/4d) model (initially introduced in the early 1990s). in a nutshell, the ic/4d model is about acquiring skills for real-life task performance and, at its core, recommends to consider four components in the resulting instructional design: learning tasks, supportive information, just-in-time information, and part-task practice (kester & van merriënboer, 2021). the following four sub-sections give an overview of these four components and their application to cem course design. here, we draw on extant distinctions of cem-relevant knowledge types (bork, 2019). we will build upon these foundations later when we detail the design requirements, principles, and features of our hyflex cem course architecture. 2.2.1 learning tasks learning tasks should engage learners to perform the constituent skills of the complex skill that they shall become proficient in. for each different class of learning task involved, these learning tasks should advance from simple to more complex tasks over time while simultaneously reducing the learner support provided alongside. learner support for each australasian journal of information systems drechsler & doyle 2025, vol 29, scholarship of is teaching and learning hyflex course architecture conceptual modelling 5 task can be provided process or product-oriented (e.g., how-tos or examples, respectively). alongside the learning tasks themselves, the criteria for an acceptable goal state for each outcome need to be specified. the task classes in a cem course are defined by the desired learning outcomes, e.g., attaining modelling proficiency in one or more conceptual modelling languages (e.g., business process model and notation (bpmn) or archimate) or diagram types (e.g., the different ea viewpoints). while recker et al. (2021) emphasise the role of the models (scripts) over the underlying languages (grammar) in conceptual modelling practice, we nevertheless regard attaining proficiency in languages as well as basic modelling principles as equally important for university-level courses. for each task class, the learning tasks should therefore progress from simple to more complex modelling tasks. modelling complexity can be expressed in the range of the nine cem activities outlined above (understanding, conceptualising etc.) and thus should progress from understanding simple domains and conceptualising these through constructing simple representations without the need to communicate to more complex settings and representations which may also involve the need to communicate among several modellers or with stakeholders, or utilising the models for subsequent analysis and/or design assignment tasks. 2.2.2 supportive information supportive information comprises all information the learners need to advance from their current state of knowledge to what they need to know to successfully tackle the next learning task. the latter could comprise a more complex task of the same learning task class or an entirely different task class. supportive information comprises information provided in advance of but also after a learning task (e.g., feedback). for teaching cem, supportive information presented in advance will mainly be about the grammar (syntax and semantics) of the chosen modelling language: its purpose, goals, constructs, and relationships as well as the ways how modelling in general or with the specific language can and should take place. a second important part of supportive information is the feedback provided to the outcomes of previous learning tasks: what parts were well executed and where was room for improvement? 2.2.3 just-in-time information in contrast to the more ‘timeless’ supportive information, just-in-time information is more about step-by-step knowledge the learners need to perform the constituent skills required to master the complex learning task at hand. as a learner becomes more proficient with the skills, just-in-time information becomes less important over time. just-in-time information for cem courses can comprise modelling examples, demonstrations, step-by-step how-tos of constructing representations, or checklists to evaluate the quality of a model. immediate process-oriented feedback during the modelling (e.g., error messages in a modelling tool) is also part of the just-in-time information. 2.2.4 part-task practice part-task practice becomes relevant only if a very high level of automaticity is required to perform the complex task in practice (e.g., playing scales on musical instruments or pilot reactions in emergencies). australasian journal of information systems drechsler & doyle 2025, vol 29, scholarship of is teaching and learning hyflex course architecture conceptual modelling 6 we do not regard part-task practice (e.g., on the foundational aspects of modelling or the chosen modelling language) as being particularly crucial for cem courses as modern digital modelling tools give modellers ample support during modelling. moreover, the respective foundational skills can also be developed through feedback to the regular learning tasks, especially if there are very simple learning tasks at the beginning for each task class. 2.3 foundations of hyflex course design as the hallmark of the cem course architecture in this paper is its hyflex nature, we now outline the foundations of designing a higher education course according to hyflex principles. with students having to juggle multiple life roles when studying, which can include working, and family responsibilities, they are looking for more flexibility in the way they learn (abdelmalak & parra, 2016). the covid-19 global pandemic in the early 2020s has also shown the need for universities and lecturers to increase resilience against unexpected events through more flexible course delivery options (keiper et al., 2020). in these contexts, hybrid-flexible (hyflex) course designs as suggested by beatty (2019) (initially introduced in 2006) can be attractive approaches to address these challenges as it is a combination of hybrid learning and flexible learning (malczyk, 2019). hyflex course design caters for present as well as distant students enrolled in a single course (abdelmalak & parra, 2016; keiper et al., 2020). hyflex moves beyond a traditional blended learning approach (abdelmalak & parra, 2016) to one that uses “hybrid classes (blending online and classroom participation modes) to provide flexible learning paths and allowed students to decide for themselves which path was ‘best’ for them on a daily or weekly basis.” (beatty, 2019 p. 20). this inherent flexibility across the entire course is what sets hyflex courses apart from other blended or hybrid offerings. the goal is thus to provide students with the options to be able to participate in their course based on their preferences and needs in any given week, i.e. they can attend any class either face-to-face or online, and either asynchronously or synchronously (beatty, 2019; keiper et al., 2020; miller et al., 2013). thus, adopting hyflex can help provide better access, convenience, and flexibility to support students (abdelmalak & parra, 2016). in particular, hyflex course design is built upon four values (beatty, 2019 p. 52). these values encompass basic qualities that all hyflex courses should adhere to. 1. learner choice: provide meaningful alternative participation modes and enable students to choose between participation modes daily, weekly, or topically 2. equivalency: provide learning activities in all participation modes which lead to equivalent learning outcomes 3. reusability: utilise artefacts from learning activities in each participation mode as ‘learning objects’ for all students 4. accessibility: equip students with technology skills and equitable access to all participation modes. it is also important to keep in mind that regardless of which participation mode a student engages with, teaching and learning activities should present content effectively and professionally, engage learners with productive learning activities, and use authentic assessment to evaluate student learning (beatty, 2019). australasian journal of information systems drechsler & doyle 2025, vol 29, scholarship of is teaching and learning hyflex course architecture conceptual modelling 7 the hyflex approach has been adopted for a wide range of different types of courses for both undergraduate and postgraduate students. for example, binnewies and wang (2019) used it to teach undergraduate students information security and information management; malczyk (2019) used it to teach undergraduate students in a social welfare policy course; miller et al. (2013) used it to teach postgraduate students in a statistics course; and keiper et al. (2020) used it to teach undergraduate and postgraduate students across courses that included organisational behaviour, introduction to sport management, marketing management, and human resources. however, to the best of our knowledge, a hyflex course architecture for a cem courses has not been documented yet. this provides an opportunity to codify our experience in this paper in the form of a set of design requirements, principles, and features for a comprehensive hyflex course architecture for cem. during the development of a hyflex course design, several aspects need to be considered. while the instructor(s) must still provide the course content and create activities to meet the goals of the course syllabus, they must ensure that students have a choice on how they access this content and participate in activities in any given week. in other words, both face-to-face and online modes must be available for all student engagements within the course (beatty, 2019) in a way that provides equivalency (miller et al., 2013). that is, students must be able to consume the course content and participate in activities regardless of whether they are attending the course online, face-to-face, or in some combination of the two (beatty, 2019). further, any activities that generate learning resources – i.e., video recordings, presentation files, or discussion notes – should be captured and made available for reuse by the students. lastly, to ensure accessibility, students must be equipped with the technological skills required to access the participation choices (beatty, 2019). when considering these aspects, it is important to keep in mind that active learning strategies for student engagement (such as classroom response systems and collaborative activities) may differ across delivery modes, meaning student engagement measures may need to be customised depending on the delivery mode being offered (binnewies & wang, 2019). 2.4 the challenges of teaching cem in a hyflex environment the question now arises what the specific challenges are that arise when a cem course is to be offered in a hyflex format. as cem takes place between the cognitive processes in the modellers’ / learners’ heads and a digital tool (or paper), on first glance only the evaluating (by a more experienced person) and communicating cem activities (cf. section 2.1) seem to be affected by a loss of synchronous inperson interactions. expanding the perspective towards the ic/4d components, however, all three relevant model components (learning tasks and the provision of supportive and just-intime information) are affected. while the task of learning cem is, in essence, independent of space and time (i.e., students can perform the cem exercises and assignments anywhere and at any time), it is the corresponding information provision that in traditional course designs relies on face-to-face interactions between learners and teachers (and potentially between learners as well). here, the challenge though is to find adequate replacements for these interactions that can take place asynchronously and/or on-line while not becoming too detached or distant from the learning tasks themselves, for instance, in the case of feedback. existing literature that covers a similar challenge is one on converting an existing conceptual modelling course into a more blended learning format building upon technology suitable to australasian journal of information systems drechsler & doyle 2025, vol 29, scholarship of is teaching and learning hyflex course architecture conceptual modelling 8 deliver massive open online courses (or moocs) (bogdanova & snoeck, 2018). key aspects of such a transition included providing videos of live lectures and formative assessments in the form of interactive on-line quizzes with immediate feedback. however, as bogdanova and snoeck (2018) note themselves, the quality of recorded lectures may not always be ideal. moreover, the availability of recordings may lead some students to ‘procrastinate’ their engagement with the course materials to an extent that negatively affects their performance. these aspects are therefore two more crucial issues our course architecture needs to tackle. 3 methodology since design science in simon’s (1996) tradition is also applicable to teaching and course design (laurillard, 2013), we use the established design science research (dsr) trifecta of design requirements, principles, and features (drechsler & hevner, 2022; meth et al., 2015; vom brocke et al., 2020) to generalise from our specific course offerings (see the next section for further details) to a more general class of cem course contexts. here, we see each instance of a hyflex cem course offering as an artefact instance, and the design principles and features as abstractions and generalisations over each course offering. the details of each course offering are documented in the archived course outlines on our university website: https://www.wgtn.ac.nz/study/programmes-courses/courses. note that we did not follow a dedicated design science research process when developing and refining our course designs and the generalised course architecture over the years 2020-2024. instead, there was an ongoing co-evolution of course (re)designs and corresponding generalised insights that the two authors shared between themselves to inform future course (re)designs. we nevertheless draw on established means to communicate design knowledge in design science such as design requirements, principles, and features. we further only present the final version of the course architecture and only highlight any substantial variations or changes over time. we will further reflect on the implications of our findings beyond hyflex cem university courses in the final sections of the paper. table 1 details the roles of design requirements, principles and features in general and how we use them to document our hyflex cem course architecture in this paper. we ground our course designs (and thus our generalised course architecture as well) on the established knowledge base on teaching the complex task of cem (cf. sections 2.1 and 2.2) and designing hyflex courses (cf. sections 2.3 and 2.4). we further draw on one author’s previously documented design knowledge for changing an enterprise modelling course from a traditional delivery model to a fully online course (drechsler, 2021). since our course architecture is generalised from actual cem course offerings, we can evaluate the design principles and features against these varied naturalistic settings (venable et al., 2016). each evaluation data point for a specific course offering serves simultaneously as a summative evaluation of that particular offering as well as a formative evaluation of the underlying course architecture itself which the course has been based on. specifically, to support our claims in this paper in terms of course effectiveness, we use course pass rates and median grades combined with selected qualitative insights about the course difficulty and the quality of the assignment deliverables. we further draw on selected course evaluation scores and anonymous student comments made in these course evaluations to underline the extent to which our course designs have been appreciated by the students with respect to key factors with relevance for a hyflex australasian journal of information systems drechsler & doyle 2025, vol 29, scholarship of is teaching and learning hyflex course architecture conceptual modelling 9 context. we are aware that these measures can only serve as proxies for effectiveness and appreciation and reflect on the corresponding limitations towards at the end of the paper. concept description role in the course architecture design requirements design requirements capture the characteristics a design artefact needs to fulfil to address a real-world problem. our design requirements document the specific needs for a cem course in a hyflex format to be both effective and appreciated. design principles design principles codify actionable knowledge to address a real-world problem on a general level, independent of artefacts. design principles can be particularly helpful when projecting design knowledge into contexts that differ more substantially from the context the knowledge (or artefacts) have been designed for. we use design principles to codify course design characteristics on a mid-range level of abstraction, independent of specifical instructional or technical solutions. our design principles can potentially be useful when applied to conceptual modelling (or even any other type of) courses outside of enterprise modelling, is programmes, or university courses. design features design features are an abstraction over design artefacts and highlight those characteristics of design artefacts that implement the design requirements in order to achieve the intended outcomes (lukyanenko & parsons, 2020). design features are more specific than design principles and allow for an easy knowledge transfer to artefacts in similar contexts. we use design features to capture and isolate key specific characteristics of our course offerings that meet the specified requirements. the design features can be potentially useful when looking for specific ways to (re)design cem courses in is programmes in other universities to bring them more into line with the hyflex requirements. table 1. roles of design requirements, principles and features in the hyflex cem course architecture 4 context: the course settings this section gives an overview of the range of contexts for which we designed our hyflex course architecture. all courses were offered in bachelor or masters level is programmes by the same public university in the australasian region. all courses had taken place during a regular trimester over 12 weeks (2x 6 weeks with a 2-week break in between). note that the masters programme is a professional conversion masters for students with non-is undergraduate degrees. the specific circumstances in this masters programme from the second half of 2020 onwards required an equivalent learning experience of on-campus and remote/ overseas students together with an improved resilience against any operational disruptions caused by the thenongoing pandemic. this situation prompted us to adopt a full hyflex approach for those courses. buoyed by the resulting transformation of the 2020 courses, the first author adopted the same hyflex course architecture to his undergraduate courses in the subsequent years. table 2 summarises the key course characteristics over the years. further information such as detailed student demographics are unfortunately not available. as one can see from table 2, there are considerable differences over the years between the courses in terms of student numbers, the number of off-campus students, and the courses’ means of delivery. the undergraduate courses (or rather, the in-person parts) were scheduled in a traditional lecture-and-workshop format. in contrast, the in-person parts for the master’s courses were scheduled in a block format with a single – but potentially longer – session each week until 2023. then, the delivery format switched to a more elaborate configuration as part australasian journal of information systems drechsler & doyle 2025, vol 29, scholarship of is teaching and learning hyflex course architecture conceptual modelling 10 of the need to scale up to reflect the substantially increased student numbers. the bp modelling courses relied on an established textbook (dumas et al., 2018) and focused on the bpmn language. the ea courses were supported by a self-developed ea modelling guide based on the togaf framework and the archimate language. further details about the course characteristics and delivery are covered among the design features of our course architecture in sections 5.1.3, 5.2.3, and 5.3.3 below. course code & year(s) course topic programme # students enrolled # students off-campus weekly schedule info 234 2021 2022 business process modelling undergraduate 121 (2021) 112 (2022) 5 (2021) 14 (2022) 2h lecture & small 1h workshops mbua 511 2020 2021 2023 2024 business process modelling professional conversion master 24 (2020) 18 (2021) 64 (2023) 47 (2024) 9 (2020) 8 (2021) 0 (2023) 0-4 (2024)1 2020-2021: weekly 2h block for all students 2023-2024: ~60 min plenary session & 15 min coaching session per team info 376 2021 2023 enterprise architecture modelling undergraduate 89 (2021) 69 (2023) 9 (2021) 0 (2023) 2h lecture & small 1h workshops mbua 514 2021a2 2021b 2022 2023 2024 enterprise architecture modelling professional conversion master 26 (2021a) 17 (2021b) 22 (2022) 63 (2023) 48 (2024) 9 (2021a) 10 (2021b) 10-13 (2022)3 0 (2023) 0 (2024) 2021-2022: weekly 2h block for all students 2023-2024: ~60 min plenary session & 15 min coaching session per team table 2. course characteristics 5 a hyflex course architecture for teaching conceptual enterprise modelling this section presents our course architecture for cem courses by outlining the underlying design requirements, the design principles for the solutions, and specific design features in our courses. figure 1 summarises our course architecture in an overview. figure 1 shows the design requirements, principles, and features sorted into three areas which were inductively created based on a minimum overlap of design features and relationships across the areas. the first area is centred around the learning outcome (i.e., attaining proficiency in the complex skill of cem), the second one around enabling the necessary communication paths for information exchange, engagement, and interaction in hyflex contexts (implementing the original hyflex learner choice and equivalency values), and the third one around the operational resources needed to build and run a hyflex course, digital or otherwise (implementing the other two hyflex values: accessibility and reusability). 1 as there were substantial delays in visa approval in q1 2024, a number of overseas students had to participate remotely for the first few weeks of mbua 511 until they arrived in the country. there were several more students who had to defer to a later enrolment date; these are not counted here. 2 this course was offered in the first (2021a) and third (2021b) trimester of 2021. 3 as travel restrictions became relaxed over q3 2022, a few overseas students could become in-person students during the trimester where mbua 514 was offered. australasian journal of information systems drechsler & doyle 2025, vol 29, scholarship of is teaching and learning hyflex course architecture conceptual modelling 11 each area is detailed in its own sub-section in terms of its design requirements and their rationales, and the corresponding design principles and features. the design requirements and rationales are based on the nature of cem courses, the hyflex values as outlined in the second section, and the specific characteristics of our course contexts (as outlined in the previous section). the design principles address general course design considerations whereas the design features allow us to provide additional rich insights based on our individual experiences and specific circumstances. figure 1: design requirements, principles and features of our hyflex course architecture for cem courses note that figure 1 does not show all relations between design requirements, principles, and features mentioned in the detailed tables in the sub-sections in order to avoid an ‘overload of arrows’ in the figure. instead, we focus on those relationships we deem to be the most essential. australasian journal of information systems drechsler & doyle 2025, vol 29, scholarship of is teaching and learning hyflex course architecture conceptual modelling 12 5.1 learning outcome-related course architecture components first, we tackle the design requirements, principles, and features related to learning outcomes of gaining proficiency in the complex skill of cem. 5.1.1 design requirements for the learning outcomes, we formulate a single focused design requirement. dr1-1: place a focus on the students attaining proficiency in the complex skill of conceptual modelling and corresponding reflective capability the underlying rationale behind this design requirement is that the emphasis in cem courses commonly lies on achieving modelling proficiency, along with reflections on limitations or implications for subsequent analysis tasks (cf. section 2.1). therefore, this key requirement is carried over unchanged compared to a traditional non-hyflex cem course. 5.1.2 design principles to implement the design requirement dr1-1, we formulate three design principles. dp1-1: structure the course around small and large-scale practical modelling and reflection assignments as outlined above, the ic/4d model puts the complex learning task – attaining cem proficiency – at the centre of attention and treats information provision secondary. adapting the recommendations given in kester & van merriënboer (2021) to teaching cem leads to this design principle recommending a scaffolding structure of learning tasks (exercises & assignments) escalating in complexity over time. moreover, having smaller-scale exercises and larger-scale assignments also gives purpose to the provided content. in other words, for the students, the content is important in a short-term perspective to gain ‘points’ (as per dp1-2 below). this design principle remained unchanged throughout the evolution of our course architecture and, in fact, could even be generalised from established traditional course designs such as our own of the past or the one documented by recker and rosemann (2009). dp1-2: provide incentives for continuous engagement with the modelling assignments this design principle serves to incentivise regular student engagement throughout the course to support them in attaining modelling proficiency and ongoing reflection. this design principle also remained unchanged throughout the evolution of our course architecture and was, in fact, one characteristic that made our very first iteration of a cem course – which was a fully virtual and asynchronous precursor, so to speak – effective in early 2020 (drechsler, 2021). this dp also helps overcome one of the major challenges identified by bogdanova and snoeck (2018), an increased danger of student procrastination when digital resources such as lecture recordings become and remain available until the end of the course. dp1-3: provide means for asynchronous content provision of cem foundations as content provision in the ic/4d model is just a means of providing the necessary supporting information to enable the students to tackle the learning tasks, we consider an asynchronous content provision to be suitable. the focus here should be on providing ‘sufficient’ supporting and just-in-time information for the students to tackle the next learning task(s), tailored to the level of proficiency they have already attained. australasian journal of information systems drechsler & doyle 2025, vol 29, scholarship of is teaching and learning hyflex course architecture conceptual modelling 13 a synchronous option would be possible, of course, but only complementary and with some added benefits (such as having the opportunity to ask immediate questions or providing a structured weekly time and space when to engage with the content). asynchronous content provision has the advantage that the students can review the content whenever necessary during their exercises. it also allows reusability (and potential refinement of content) for the lecturers. while the previous two design principles would also apply to traditional cem course offerings that have been mentioned, this one is specific to a hyflex course. 5.1.3 design features there are four even more specific design features we used to implement the design principles in the past course offerings listed in table 2. df1-1: weekly modelling exercise and reflection assignments df1-4: group-based weekly small-scale modelling exercise tasks for most weeks, our courses had weekly exercise assignments, which were centred on the application of each week’s new content. in all courses these built on a single (usually small) case and the exercises were designed to advance the students’ modelling, analysis and redesign proficiency each week. in total, these weekly assignments accounted for 15-25% of the course marks. the students worked in groups on these exercises but had to submit individual reflections and insights into the group work alongside the weekly exercise deliverables themselves. hence, the assignments could be assessed individually. to keep the students’ weekly workload in check, we also emphasised in later iterations of our courses that merely ‘good draft versions’ would be the expected standard for the weekly exercises and used the mantra ‘copper, not gold’ (provided in meme form for extra ‘stickiness’) to underline this expectation. while we carried over the general approaches to graded weekly exercises and group work from previous traditional offerings, the exact number and percentage of those assignments were subject to constant adjustment for each course from one iteration to the next. note that this design feature also implements the student interaction-related design principles dp2-1a and dp2-1b (see next section). df1-2: large case-based modelling assignments to assess the students’ capabilities in mastering the complex skill of cem for more complex cases, each course also had one large case-based assignment. there was a deliberate big overlap between tasks and questions between the weekly exercises and the case-based assignment; the key difference was the scale and complexity of the situation where the students were asked to apply their skills. the case-based assignment was split into two or more portions (= smaller assignments about the same case throughout the term) to make the assignment appear less daunting, allow for feedback loops, lower the risk from the students’ perspective, and distribute to the marking effort on the lecturers’ side over the entire term. each of those smaller assignments, however, built upon the previous one, so that there was one coherent arc from initial modelling over analysis to a redesign in each course, mirroring real-life organisational transformation projects that draw on conceptual enterprise models. such an approach addresses the shift in conceptual modelling from merely representing to shaping the future of physical and digital australasian journal of information systems drechsler & doyle 2025, vol 29, scholarship of is teaching and learning hyflex course architecture conceptual modelling 14 organizational reality (recker et al., 2021). this design feature was also carried over from prehyflex cem course offerings and continuously refined between course iterations. df1-3: weekly introductory content videos the supporting information (e.g., introducing new modelling language elements, diagrams, techniques, or analysis methods) was provided in the form of 3-8 videos each week. each video usually had a length between 3 and 15 minutes (with a few exceptions). the videos were made available early monday mornings in most cases. interestingly, the students in one ea course requested the videos to be available sunday evening instead – which was no problem on our end. further, we could re-use several videos not only in future offerings of the same course, but – due to overlapping content – also in future offerings of a similar course on different levels (undergraduate / graduate). for the undergraduate courses, the same content was delivered live during the weekly twohour session (which we called ‘modelling studio’ instead of ‘lecture’). here, most sessions consisted of an alternating rhythm between short content provision (similar to a single video) and then time for group work on the weekly exercises in the lecture theatre. we thus ensured the equivalency of synchronous in-person and asynchronous engagement. this general rhythm of alternating short lecturing stints and time for group work already existed in prehyflex iterations of our cem courses and lent itself very well to the transition into short weekly videos covering the necessary supporting information. 5.2 communication-related course architecture components this section covers the design requirements, principles and features related to the various engagement and interaction modes implementing the four hyflex values. 5.2.1 design requirements first, we formulate three design requirements for the various modes of interaction between students and lecturers. dr2-1: enable student conversations & collaboration across engagement modes this is an optional requirement strictly speaking, but relevant for us since we deemed it to be important for the students to have good conversations in their groups while constructing their conceptual enterprise models for their exercises and assignments. in line with the hyflex values of learner choice and equivalency, these interactions should be possible regardless of each student’s chosen engagement mode at a given time. dr2-2: allow the opportunity for students to shift between in-person and virtual engagement modes at any time learner choice and equivalency are two cornerstones of a hyflex course design – being hybrid as well as flexible between in-person and virtual engagement modes – and this is thus the corresponding design requirement. dr2-3: enable student / lecturer interactions across engagement modes student / lecturer interactions are important for cem courses as they encompass the provision of the learning tasks, the supporting information (e.g., the foundations and the feedback to exercises), and the just-in-time information (e.g., ad-hoc guidance or answers to cem questions). following the hyflex values of learner choice and equivalency, these interactions should be possible regardless of the chosen engagement mode for either role at a given time. australasian journal of information systems drechsler & doyle 2025, vol 29, scholarship of is teaching and learning hyflex course architecture conceptual modelling 15 5.2.2 design principles we formulate four design principles to implement the three design requirements: two for the interactions between students and two for the interactions between students and lecturers. dp2-1a: provide means for synchronous and asynchronous interactions between students dp2-1b: provide means for in-person, virtual and hybrid interactions between students these two design principles implement dr2-1 and dr2-2 and are thus – like dr2-1 – optional. as outlined in the second section, however, we consider communicating between modellers or between modellers and stakeholders to be key aspect of cem in practice and therefore include and implement these design principles. in a teaching environment, having students working together in groups can provide both structure for their efforts and a forum to discuss questions (e.g., around conceptualising, abstracting and constructing) among themselves. if the submitted assignments are assessed as group work, the marking efforts for those assignments can be reduced as well. hyflex implies that there needs to be a variety of channels for interactions across student engagement modes. therefore, this design principle has its roots in the teaching cem aspect as well as the hyflex aspect of the course. dp2-2a: provide means for synchronous and asynchronous student / lecturer interactions dp2-2b: provide means for in-person, virtual and hybrid student / lecturer interactions these two design principles implement dr2-2 and dr2-3 and are, unlike the previous two, not optional. answers and continuous feedback to students about their modelling efforts as part of the supporting or just-in-time information are at least equally important than content provision, so the means to provide these are a crucial consideration. hyflex implies that there need to be a variety of channels for answers & feedback. this design principle is among the less-specific ones for cem but nevertheless crucial for a true hyflex course. 5.2.3 design features we formulate three design features to implement the four design principles. df2-1: weekly sessions for review of weekly exercise tasks over time, we ended up having three variants of this design feature, to account for the different course formats we employed over the years (cf. table 2): 1. weekly single lecture (‘modelling studio’) & small workshops (undergraduate, scalable): this is the variant we employed for the undergraduate course offerings – a standard format with a two-hour session (formally called a lecture but called ‘modelling studio’ by ourselves) and with small-scale workshops run by student tutors. this format is scalable, limited only by the size of the lecture room and the number of available student tutors. 2. single weekly block session for all student teams (master, limited scaling): this is the variant we ran for the master level courses with enrolments up to 30 students. the single two-hour session started off with some short general remarks, live discussions and/or short exercises at the beginning, and then shifted to a ‘visit’ to each team to discuss their questions or give feedback to their draft exercises. the room was equipped with sufficient tables and screens (‘pods’) so that there could be an almost seamless interaction between in-person and remotely participating team members for both parts of each session. we kept the general remarks to a minimum, however, to maximise the question and feedback time about the produced models for the weekly exercises. this format’s scalability is limited to the australasian journal of information systems drechsler & doyle 2025, vol 29, scholarship of is teaching and learning hyflex course architecture conceptual modelling 16 number of physical ‘pods’ for teams and the ‘waiting time’ for the final team to get its turn for questions and feedback in the second part of the session. 3. weekly plenary session for all student teams and separate short weekly coaching sessions per team (master, scalable): as the need arose to scale the master level courses to more than double the number of students from 2023 onwards, we spun off the two parts of variant 2’s weekly block session into separate sessions: an one-hour weekly plenary session for all students with now more expanded general remarks, live discussions and exercises, and then a dedicated 15-minute time slot later on the day for each team for ‘coaching’, i.e., to give feedback to their exercise drafts and answer their questions. all other course elements remained the same. the scaling limitations here are only the room size for the plenary session and the number of 15-minute time slots for the teams. notably, in two of the undergraduate courses, until 2/3rds of the term had passed, roughly 50%-60% of students enrolled in the course had chosen to still attend the weekly two-hour session in-person or remotely despite the same content being available as videos at the start of the week. the advantage these students saw was that they could start on their weekly assignment right away in their groups (with a large extent of the synchronous participants being in the lecture theatre) while having the opportunity to ask questions and receive feedback on the spot. in addition, after the two-hour session, they would have already made some good progress on the exercises to submit for the weekly exercise tasks. groups not participating in the synchronous two-hour session could then have the same opportunity to seek feedback in their small-group workshop later in the week, but they had to organise their group work themselves. df2-2: participation in block session / lecture & workshops can be in-person or via zoom each session was streamed live via zoom (with access to the master level coaching sessions restricted just to the team members), and all except these coaching sessions and the undergraduate workshops were recorded and provided as a video the next day. live lecture participation via zoom in the undergraduate courses turned out to be the exception (except for one offering) since (or despite) this option being redundant to the videos which were available earlier and better curated. since about a third of the students in the master courses with the block sessions were out-of-town students, they were regular participants in the block sessions via zoom, however. in the scaled-up versions of the master courses, remote participation was more limited, mainly because all students were in the country. the advantage for hybrid participation for cem courses is that they usually rely comparably little on discussion and other forms of engagement that would benefit from mutual interaction across the whole class but instead rely more on questions, feedback, and reflection on produced conceptual enterprise models specific to each student or group. df2-3: alternative feedback through e-mail / canvas messages / ms teams chats this design feature also overlaps with the technical resources and infrastructure covered in the next sub-section. besides the synchronous participation in the lecture, workshops, or block session, there was also the opportunity for each student and team to ask questions or seek feedback asynchronously on ms teams, e-mails, or through messages on our learning management system canvas after its introduction in 2023. for content-related questions in ms teams spaces, we provided a weekly channel with a pre-made conversation for each topic of that australasian journal of information systems drechsler & doyle 2025, vol 29, scholarship of is teaching and learning hyflex course architecture conceptual modelling 17 week. here, students could ask questions by replying to one of those conversations. in some of the courses with block mode, this saw a reasonable uptake each week for one student cohort, but much less for another student cohort. in contrast, hardly any of the undergraduate students ever asked a question by replying to those conversations. in addition, the students could share drafts of their exercise deliverables with a request for feedback or ask questions through messaging in ms teams. uptake of this opportunity also varied quite a bit between courses and student cohorts. ms teams also has the option for quick audio or video calls, but those were the exception as well. 5.3 resource-related course architecture components the third area of our course architecture focuses on the operational and resource aspects for hyflex delivery 5.3.1 design requirements dr3-1: keep the technical barrier for engagement low following the accessibility value, hyflex course designs require digital tools, and these should be enablers and not barriers for students. moreover, following the equivalency value, these tools should ideally also work from within an in-person classroom setting for student teams who happen to be hybrid at a given time. dr3-2: keep the teaching staff’s efforts for course preparation and operation manageable given the established preparation and delivery effort associated with hyflex courses, keeping teaching staff's (and potentially tutors’) workload manageable became its own requirement. 5.3.2 design principles dp3-1: rely on familiar and/or simple digital tools as much as possible this design principle implements dr3-1. hyflex course designs require digital tools to enable asynchronous and virtual / hybrid channels. these should not be a barrier for student engagement, however. bogdanova and snoeck (2018) point out that a switch in digital learning environments for their course conversion was not appreciated by the students, underlining the need to seek out familiar digital tools. for teaching cem, attaining proficiency in using industry-standard modelling tools can be part of the intended learning outcomes, however, and these may not always be simple and easy to use. therefore, this dp should not influence the choice of the digital modelling environment best suited for the chosen modelling language(s) and course contexts (e.g., the need for collaborative modelling). for instance, in one of our courses, we had to switch from a cloud-based ea modelling tool (stratamap) to a standalone tool (archi) when the former was discontinued and therefore were forced to trade the benefits of a collaborative modelling tool for one that was somewhat easier to use but had a less advanced feature set in return. dp3-2: limit the use of non-supported and non-integrated digital tools this design principle implements dr3-1 and dr3-2. relying on proven and institutionprovided digital tools in a hyflex course design can limit the risk and also shift the burden for technical support away from the lecturer. again, this dp may not apply to the choice of digital modelling environment which may well be a standalone or vendor-hosted cloud-based solution that is not integrated with university systems. in our case, with the large-scale shift to digital / hybrid instruction across the entire university in 2020, we made the deliberate choice australasian journal of information systems drechsler & doyle 2025, vol 29, scholarship of is teaching and learning hyflex course architecture conceptual modelling 18 to discontinue the use of digital tools (such as slack) in our courses that back then were better suited to our requirements but not integrated with the university systems. dp3-3: focus on re-usability of content across course offerings this design principle implements dr3-2 as well as dr1-1. cem languages and techniques are usually quite stable these days, so provided content can be ‘timeless’ to an extent. there should nevertheless be continuous improvement of the provided material, however. recognising the preparation and ongoing efforts hyflex courses demand from lecturers, an emphasis on reusability became important. therefore, an emphasis on re-usability can help reduce the preparation effort after the first round and also shift the focus to a continuous improvement of the ongoing course support effort against a stable content backdrop. we formulated this design principle after our first iteration as one author had to re-record a number of videos for a next iteration of an ea course. 5.3.3 design features df3-1: infrastructure: blackboard / canvas, zoom, panopto, office 365, ms teams in terms of infrastructure, we were lucky to be able to rely on an extensive suite of quite well integrated digital tools provided by our university. blackboard was the established learning management system (lms) used to provide all types of content (e.g., pdfs, recordings, readings) to the students until it was replaced by instructure canvas in 2023. zoom was the designated tool for synchronous sessions and corresponding recordings, whereas panopto served as the tool for asynchronous recordings as well as the provision of both recording types since it was integrated with zoom as well as the lms. panopto also allowed for some basic video editing and provided auto-generated captions for each video for additional accessibility. in addition, until the arrival of canvas with its improved community features, each course had its own ms teams space, and our digital solutions unit could automatically enrol all students of a course in the course-specific ms teams spaces, thus avoiding the need for manual signups by the students. ms teams could provide a basic means for the students to collaborate with one another, but they often chose their own communication platforms such as whatsapp, facetime or even discord – a factor that contributed to our decision to discontinue the ms teams spaces in 2023, contributing to simplifying the tool landscape from the learner perspective. as noted among the discussion of the corresponding design principle dp3-1, we also made the conscious effort early on to solely rely on digital environments provided and supported by our university for these channels. this design feature covers the non cemrelated infrastructure and thus applies to potentially all hyflex courses. df3-2: cloud-based modelling tools (if possible): camunda, stratamap each of our two cem course types had its own designated modelling tool (camunda for bpmn and stratamap – and later archi once stratamap was discontinued – for archimate modelling). when choosing these modelling tools, we paid attention to the requirements of being freely available, platform-independent, conducive to collaborative modelling, and conformant to industry standards (e.g. bpmn or archimate). in addition, students often chose other collaborative tools such as googledocs, or lucidchart for their group-based assignment work outside of formal cem tasks. this design feature therefore is the cem-specific counterpart to df3-1 about the necessary digital infrastructure for the courses. australasian journal of information systems drechsler & doyle 2025, vol 29, scholarship of is teaching and learning hyflex course architecture conceptual modelling 19 df3-3: create videos covering the basic content without references to dates, course ids or other coursespecific arrangements in terms of course preparation, we made sure that most videos that we created were reusable for future course offerings. thanks to the modelling languages and principles being relatively stable, we see no real need to record new videos each year. we achieved this goal by removing all visible and audible references to dates, recent events, course ids (to allow re-use across programmes where appropriate), or matters of course organisation from the content videos. course-specific provisions were catered to through separate videos, if necessary. such an arrangement can shift the course preparation emphasis in the future to selected additions, improvements, and removal of content or assignment arrangements. any addition, improvement, or removal of content essentially means just to re-record a short video (or even parts of it, if video editing can do the trick) and thus will mean little additional effort besides the underlying research and the slide or guide document revisions themselves. this again is a design feature that applies beyond just cem courses. df3-4: create similar assignment structures and share cases between courses the previous feature of preparing videos for future re-use can also apply to attempts at reusing assignment structures and cases between courses. for instance, past offerings of the postgraduate level bp course had actual industry clients as case organisation for the large assignments, and an edited form of one of those cases was the foundation of the assignment cases for some the postgraduate and undergraduate level course instances of table 2. along similar lines, some frequent shortcomings in the students’ assignment submissions in the earlier bp course led not only to a revision of the corresponding assignment briefing document in order to improve clarity in the future but also to similar revisions in comparable assignment documents in both subsequent courses. we also could share the same case for the simple exercises between all cem courses covered in this paper. this case was not provided in written form but in short animated and narrated video vignettes which resemble a set of interviews with the stakeholders of a fictional organisation. here, both courses can now benefit from the extra work of providing a videobased case (which already existed since the mid-2010s), and issues about this case noted in one course could then again be dealt with for all future course offerings relying on that case. 6 course architecture evaluation in this section, we first evaluate the overall course architecture based on the effectiveness and the students’ reported appreciation of the courses instantiating the course architecture’s design principles and features as shown in figure 1. afterwards we evaluate and reflect on the fitness-for-purpose and also the projectability into other contexts of the design features and principles. note that since every instance of a taught course is ‘more than the sum of its parts’ – including but not limited to the design principles and features that comprise the course architecture – we can only attribute a part of any success or failure to the underlying course architecture and its elements (lukyanenko & parsons, 2020). australasian journal of information systems drechsler & doyle 2025, vol 29, scholarship of is teaching and learning hyflex course architecture conceptual modelling 20 6.1 evaluating the overall course architecture we first evaluate our overall course architecture based on the effectiveness of each course and the extent the students appreciated it. to assess our course architecture’s effectiveness in objective terms, we first provide the pass rates and median grades for all courses in table 3. course code year topic pass rate median letter grade info 234 2021 business process modelling 81.15% b 2022 80.35% b mbua 511 2020 92% a 2021 100% a 2023 96.92% a 2024 100% b+ info 376 2021 enterprise architecture modelling 94.3% b+ 2023 94.2% b mbua 514 2021a 100% a 2021b 100% a 2022 100% a 2023 98.41% a 2024 100% a table 3. course pass rates and median letter grades note that there were only comparably minor changes of course content and assignments between offerings of the same course over the years, just with differing assignment cases or industry partners. hence, we believe that comparing the outcomes across the courses can be done with a certain level of validity, despite the student cohorts and other factors changing between trimesters and years. note further that we do not, however, compare the grade distributions among the offerings or to previous ones, due to the continuously changing circumstances for students during the pandemic in the timeframe. overall, the pass rates and median letter grades are on an at least respectable level on absolute terms, and in relative terms at least similar to other undergraduate and graduate courses held in the same time frame in the same programme. while lacking objective measures across courses and programmes, we believe the level of difficulty for these courses to be on the higher side, with info 234 and mbua 511 comprising the whole cycle of process modelling, analysis and improvement (and in-depth assignments for each), and mbua 514 having comprehensive architecture projects for real-life clients. client feedback from actual enterprise architects involved in some of these projects indicates a high-level of comprehensiveness, quality, and relevance of the main assignment deliverables. another at least partially useful proxy for course difficulty is the reported student workload which, according to table 4, tends to be on the higher side for most offerings. a contributing factor to the high mbua course pass rates is the usually very high motivation of the mbua students who all come from different nonbusiness analyst careers and enrol in the programme out of a clear motivation to transition into business analysis roles. a further notable – if anecdotal – objective indication for the effectiveness on the hyflex side of our course designs is that the best student by quite a margin australasian journal of information systems drechsler & doyle 2025, vol 29, scholarship of is teaching and learning hyflex course architecture conceptual modelling 21 in info 376 in 2021 was a student who never attended a lecture or a workshop session in person but instead worked solely with the provided videos and sought feedback through email and zoom. subjectively, based on written student feedback about the course designs in the universityadministered course evaluation surveys, there were two key factors contributing to their performance. first, they found the recorded content (df1-3) to be at a good pace for learning. they also found it helpful that they were able to re-watch it as many times as required (in-line with dp1-3 and the hyflex value of reusability). second, the students also appreciated the weekly exercises (df1-1), where they could ask questions, get feedback, and then apply the gained proficiency to their larger assignment project afterwards. in-line with dp1-1 and dp12 and the tenets of the ic/4d model for complex learning, these exercises required the students to learn how to apply what was being taught right away as opposed to just watching videos and then later panicking as assignment due dates came up. from the lecturers’ point of view, there were a considerable number of students in each cohort who truly mastered the complex tasks of bp or ea modelling, and thus we view the assigned final letter grades as very much warranted. students also showed their appreciation for the course designs based on the course architecture in their evaluations as shown in table 4. course code year overall evaluation assessments help learning receiving feedback on progress video resources helped learn able to engage online well course workload4 info 234 2021 2.2 1.7 2.0 1.9 2.1 2.9 2022 2.3 1.9 1.9 1.4 1.8 2.0 mbua 511 2020 1.7 1.7 1.9 1.5 1.5 2.9 2021 1.3 1.3 1.5 1.5 1.8 2.8 2023 2.6 1.8 1.8 1.9 2.1 2.1 2024 1.8 1.4 1.4 1.4 1.6 2.6 info 376 2023 2.7 1.8 1.9 1.9 2.4 2.3 mbua 514 2021a 2.2 1.4 1.1 1.7 2.1 2.8 2021b 1.8 1.7 1.8 1.8 2.3 2.1 2022 1.4 1.4 1.6 1.4 1.4 2.0 2023 2.3 1.6 1.5 1.8 1.6 2.3 2024 1.5 1.4 1.4 1.2 1.4 1.9 table 4. key course evaluation results (scale 1-5; lower is better, except for workload, see footnote). as with our course designs as noted above, the course evaluation surveys as administered by the university also remained unchanged between years. note that the info 376 evaluations for 2021 are not included since less than 10% of the enrolled students answered that survey. response rates for the other courses range between 15% and 65%. 4 note that the ‘course workload’ column has a different scale compared to the other columns: 3=about right; higher values indicate higher than average and lower values lower than average course workload australasian journal of information systems drechsler & doyle 2025, vol 29, scholarship of is teaching and learning hyflex course architecture conceptual modelling 22 the numbers in table 4 show the students’ general appreciation of several key aspects as well as the overall course offerings which are largely corroborated by the students’ anonymous written course evaluation feedback, in particular, it was our impression that the students spoke largely favourably of • the group-based weekly modelling tasks (‘assessments help learning’, cf. df1-1 and df14) to keep them engaged in learning the content through regular opportunities for practice. • the weekly feedback sessions on those modelling tasks (‘receiving feedback on progress’, cf. df2-1 to df2-3) to break down a 2-hour session (for the undergraduate offerings) into smaller chunks and (for all offerings) to get relatively quick feedback. • the usefulness of the introductory content provision in video form (‘video resources helped learn’, cf. df1-3) to easily learn the topics through the weekly exercises. • the availability of a variety of channels for engagement (‘able to engage well online’, df23 and df3-1) to have the flexibility and also to feel comfortable to learn in-person or online. • the overall effectiveness of the course designs in terms of structure, regular progress and reflection opportunities, and the assignments being related to the students’ future careers. there is, however, the exception of the overall evaluation ‘outlier’ for info 376 in 2023. here, the underlying reasons for the rather low score are difficult to ascertain, as the other scores and comments of the entire course and teaching evaluation (so not just the values shown) and the lecturer’s personal impression point towards an overall quite appreciated course. in 2023, mbua 511 and 514 were offered at the same time to the same cohorts which – thanks to the very similar topic and format – was not conducive to focused student learning for bpm and ea at the same time and may have contributed to also a rather low overall score. for 2024 and beyond, these two courses were offered in subsequent trimesters, contributing to much improved scores for 2024. in sum, the quantitative and the qualitative data for all the courses that were based on the course architecture outlined in this paper show that the individual courses were both effective in the students attaining the complex skill of cem while simultaneously being generally appreciated by the enrolled students with a few exceptions as discussed above. since our course architecture is an abstraction over these individual course instances, we therefore claim a high level of fitness-for-purpose for the course architecture as well. 6.2 evaluating the design principles and features to unpack our comprehensive course architecture further, we now evaluate each of our design principles and features with respect to their fitness-for-purpose (or utility) as well as their projectability into other contexts, as recommended by vom brocke et al. (2020) for design science-related knowledge contributions. specifically, since the design features are closest to the actual artefacts (course offerings), the main evaluation focus here is their fitness-forpurpose to meet the design requirements, whereas we additionally reflect on projectability for the more abstract design principles. we further reflect on the researchers’ personal confidence (so not any statistical confidence) in our evaluations and contributions further down in the discussion. australasian journal of information systems drechsler & doyle 2025, vol 29, scholarship of is teaching and learning hyflex course architecture conceptual modelling 23 6.2.1 evaluating the learning outcome-related design principles and features the main design requirement to fulfil with respect to learning outcomes is to focus on the students’ proficiency in the complex skill of conceptual modelling and corresponding reflective capability. table 5 shows the evaluation of the corresponding design features. design feature relation fitness-for-purpose evaluation df1-1: weekly modelling exercise & reflection assignments dp1-1, dp1-2 having weekly assignments proved effective for fostering continuous student engagement with learning tasks aimed at developing smallscale guided cem proficiency, reflection, and feedback opportunities. df1-2: large case-based modelling assignments dp1-1 having a large case-based assignment proved effective for assessing large-scale and autonomous student proficiency in the complex skill that is cem. df1-3: weekly introductory content videos dp1-3 a weekly set of content videos on modelling foundations proved to be adequate in terms of supporting information to equip the students with the necessary knowledge to tackle the weekly assignments and, subsequently, the large case-based assignment. df1-4: group-based weekly small-scale modelling exercise tasks dp1-1, dp1-2, dp2-1a, dp2-1b organising small groups for the weekly exercise assignments proved effective for giving the course additional structure, continuous student engagement and ultimately developing the students’ modelling proficiency. in addition, these weekly exercises proved useful as a backdrop for the students to reflect upon. having to mark weekly assignments goes somewhat against dr3-2 but is worth the effort from the authors’ perspective (which is supported by student feedback). in addition, this is a marking task that can be (and was, in our cases) taken care of by tutors. table 5. evaluating learning outcome-related design features of our hyflex course architecture design principle relation fitness-for-purpose and projectability evaluation dp1-1: structure the course around small and large-scale practical modelling and reflection assignments df1-1 df1-4 as the corresponding design features were evaluated as fit-forpurpose, we likewise see these design principles to be fit-for-purpose for a hyflex cem course design to attain modelling proficiency. in terms of projectability into other contexts, we regard the principle of scaffolding assignments with increasing complexity as applicable to potentially many other courses beyond cem and also beyond hyflex contexts. the nature of these assignments would have to be tailored to other course learning objectives and content, however. dp1-2: provide incentives for continuous engagement with the modelling assignments df1-1 df1-4 as with the previous design principle, we see incentives for continuous engagement with assignment not only as an effective design principle for our specific context, but also as one that can be projected into many other contexts beyond cem and hyflex, as it reduces the chance that students will leave starting their learning efforts too late. dp1-3: provide means for asynchronous content provision of cem foundations df1-3 we see the asynchronous content provision of course foundations as fit-for-purpose for cem and also many other contexts where the focus lies on attaining practical proficiencies. and while not the only means of providing content in a hyflex environment, we would regard it as particularly challenging to solely rely on synchronous content provision in such an environment. outside of hyflex, there is, however, a good argument to be made for the ‘personal touch’ in a classroom instead of just having videos with, at best, a ‘talking head’. table 6. evaluating learning outcome-related design principles of our hyflex course architecture australasian journal of information systems drechsler & doyle 2025, vol 29, scholarship of is teaching and learning hyflex course architecture conceptual modelling 24 overall, these four design features proved to be useful characteristics of our hyflex course offerings to attain conceptual modelling proficiency and corresponding reflective capability. in table 6, we now evaluate our learning outcome-related design principles in terms of fitnessfor-purpose as well as projectability into other contexts beyond cem and hyflex. in sum, while all three design principles are indeed fit-for-purpose like the design features, there are more nuanced concerns around their projectability into other contexts outside of cem and hyflex. 6.2.2 evaluating the communication-related design principles and features the second of our three areas in figure 1 revolves around the requirements of communication between students across engagement modes (optional in our context) and between students and lecturers (non-optional) across engagement modes, as well as having the option for both to be able to freely shift their engagement modes. table 7 covers our evaluation of the corresponding design features with respect to their fitness-for-purpose. design feature relation fitness-for-purpose evaluation df2-1: weekly sessions for review of weekly exercise tasks dp2-2a, dp2-2b having a weekly synchronous session (in whichever form) proved to be an effective forum for questions and feedback to draft versions of the weekly exercise deliverables. df2-2: participation in all sessions can be in-person or via zoom dp2-2a, dp2-2b having at least one weekly synchronous session was effective in answering questions and giving feedback regardless of engagement mode. even if the lecture zoom sessions only had a few attendees at most (see above), they usually still had a few questions that could be answered through the zoom chat. df2-3: alternative feedback through e-mail / canvas messages / ms teams chats dp2-2a, dp2-2b having a complementary asynchronous alternative channel for questions and feedback was at least sporadically used by students and thus proved to be an effective complementary channel. the uptake varied between courses and cohorts, however, but the circumstances in each context were too different to analyse the impact of participation in this channel further. table 7. evaluating communication-related design features of our hyflex course architecture design principle relation fitness-for-purpose and projectability evaluation dp2-1a: provide means for synchronous and asynchronous interactions between students df1-4 df1-4 having multiple options – both dependent and independent of time and space – for student interaction proved effective to support the students’ group work efforts for the weekly exercises as well as – if applicable – the large-scale assignments later in the course in the light of varying circumstances over the years. more generally speaking, having these options seems to be a necessity for a hyflex course design in any context – again, in case the course depends on (or at least benefits from) mutual student interaction. we further believe that even regular inperson courses relying on student interaction can benefit from having multiple communication channels available, even if they are only complementary. dp2-1b: provide means for inperson, virtual and hybrid interactions between students dp2-2a: provide means for scalable synchronous and asynchronous student / lecturer interactions df2-1 df2-2 df2-3 similar to dp2-1a and dp2-1b, having multiple possibilities of interactions between students and teaching staff proved effective (and essential) to support the students attaining the intended learning outcomes. as with the previous two dps, we regard such arrangements (in whichever form) to be essential for hyflex courses of any form in any context, and at least nice-to-have even for non-hyflex courses. dp2-2b: provide means for inperson, virtual and hybrid student / lecturer interactions table 8. evaluating communication-related design principles of our hyflex course architecture australasian journal of information systems drechsler & doyle 2025, vol 29, scholarship of is teaching and learning hyflex course architecture conceptual modelling 25 based on these mostly positive evaluations of the design features in terms of their fitness-forpurpose, we now assess the more abstract design principles in terms of their fitness-forpurpose for our context, and also with respect to their projectability into other contexts in table 8. in contrast to the design principles in the previous area, we find the principles here to be projectable to basically any hyflex context, or even to teaching contexts outside of hyflex. 6.2.3 evaluating the resource-related design principles and features the requirements for this area revolve around keeping the technical barriers for students low and the lecturers’ efforts for course preparation and operation manageable. table 9 shows the evaluation of our four design features for this area regarding their fitness-for-purpose to meet the requirements. design feature relation fitness-for-purpose evaluation df3-1: infrastructure: blackboard / canvas, zoom, panopto, office 365, ms teams dp1-3, dp2-1a, dp2-1b, dp2-2a, dp2-2b, dp3-1, dp3-2 while this set of digital tools does not the simplest of landscapes make, all these tools are university-provided, proved effective and were well-integrated so that little administrative overhead was necessary. the students were also already used to these tools from their previous courses, so they did not turn out to be a barrier for engagement. df3-2: cloud-based modelling tools (if possible): camunda, stratamap dp2-1a, dp2-1b, dp2-2a, dp3-1, dp3-2 cloud-based, free, and proven industry-standard platforms for cem proved to be effective for fostering student collaboration as well as easy lecturer feedback (e.g. by sharing and commenting on the models in the cloud). in addition, having to switch to a non-collaborative ea modelling tool (archi) for the most recent ea course iterations highlighted the benefits of these functionalities for student collaboration. using those tools can also be seen to be a part of student modelling proficiency. df3-3: create videos covering the basic content without references to dates, course ids or other coursespecific arrangements dp3-3 paying special attention to the reusability of shared content between course offerings and courses allowed easy re-usability. in addition, all courses can thus benefit automatically from any improvements to the shared content material. course-specific arrangements can be addressed in course-specific videos. df3-4: create similar assignment structures and share cases between courses dp3-3 reusing the basic assignment structure as well as the assignments themselves between course offerings and courses has similar advantages to df3-3. table 9. evaluating resource-related design features of our hyflex course architecture again, all design features proved fit-for-purpose and contributed to the overall course architecture effectiveness. as in the previous two sections, we now assess our design principles with respect to their fitness-for-purpose as well as their projectability into contexts beyond cem and hyflex in table 10. as with the design principles in the previous area, we see a high extent of projectability of these design principles into non-cem and also even into non-hyflex contexts. australasian journal of information systems drechsler & doyle 2025, vol 29, scholarship of is teaching and learning hyflex course architecture conceptual modelling 26 design principle relation fitness-for-purpose and projectability evaluation dp3-1: rely on familiar and/or simple digital tools as much as possible df3-1 df3-2 this principle was insofar effective that we did not have answer much, if any, support queries from the students regarding the basic tool infrastructure. instead, we could focus on teaching the use of the cem tools which were part of the course learning objectives. in our view, this principle becomes even more crucial in other contexts outside cem and information systems, especially since cem courses are technical courses by nature and students in all our programmes were expected to have at least a moderate level of digital literacy. it also is highly relevant for all sorts of hyflex contexts since hyflex courses often rely on a set of digital tools to implement the varying engagement modes. dp3-2: limit the use of non-supported and nonintegrated digital tools df3-1 df3-2 as with the previous principle, we appreciated that any support queries regarding for all tools except the modelling tools could be taken on by the university’s digital helpdesk, and we see this being applicable for all types of courses beyond cem and hyflex. dp3-3: focus on reusability of content across course offerings df3-3 df3-4 given the – at least for the first offering – quite substantial transformation effort of designing or redesigning a hyflex course offering, we regard this principle as crucial for sustained feasibility of hyflex course offerings, as we found subsequent offerings not substantially more resource or maintenance intensive than traditional non-hyflex offerings. we also came to appreciate a good amount of flexibility and scalability in our hyflex course designs when having to adjust to changing circumstances such as substantially increased student numbers. table 10. evaluating resource-related design principles of our hyflex course architecture 7 discussion and limitations as our contribution comprises prescriptive knowledge (or knowledge for design and action), we thus now evaluate our overall contributions against the three criteria proposed by vom brocke et al. (2020) for this type of knowledge: fitness-for-purpose, projectability, and the researchers’ personal confidence with which they make these claims. as we are not striving for truth or accuracy (as one would for theoretical contributions to better understand the real world), other common contribution evaluation criteria such as generalizability do not apply. the assessment of each design principle and feature in tables 5-10, as well as the supporting data with respect to course effectiveness and the appreciation by the students (tables 3 and 4), illustrate that the overall course architecture of figure 1 as an abstraction over the individual course offerings (artefacts) indeed turned out to be fit-for-purpose. each design feature has proven effective in the courses where they were instantiated and contributed to the overall course architecture effectiveness. the average quality of student work as well as the grades and pass rates across all courses also indicate that the courses indeed managed to reach the learning objectives. the results further indicate the positive impact of the hyflex aspects of the course design since several courses had substantial parts of the student population unable to attend any in-person sessions. simultaneously, many of the students enrolled in the undergraduate courses chose to attend the weekly two-hour in-person session (pandemic alert levels permitting) despite videos with the same content being available each week as an alternative offering. the course evaluation results likewise indicate that the students by and large appreciate the flexibility of the hyflex course designs. moreover, by relying on graded small-scale weekly modelling exercises we managed to overcome the potential issue of student procrastination that bogdanova and snoeck (2018) indicated (as mentioned in section 2.4). australasian journal of information systems drechsler & doyle 2025, vol 29, scholarship of is teaching and learning hyflex course architecture conceptual modelling 27 we further found our course architecture to offer additional flexibility not just from the learners’ but also from a course design perspective, allowing us to adapt our courses rather easily to changing circumstances. the best example here concerns the scaling of the courses to different student numbers over time (e.g., from around 20 in 2021 and 2022 to over 60 in 2023, cf. table 2) where essentially the offerings merely needed different scheduling and room arrangements for the synchronous parts (cf. section 5.2.3) and all other course components could largely remain as they were. for regular non-hyflex course offerings, we would have expected a much higher difficulty in meeting the scaling-up challenge without a complete duplication of efforts. we therefore posit that a hyflex-oriented course redesign offers an opportunity to break up a traditional ‘course monolith’, allowing additional future flexibility from a course design perspective as well. in terms of projectability into contexts beyond cem and hyflex, the evaluation of the more abstract design principles in tables 6, 8 and 10 shows that especially the communicationrelated design principles are potentially projectable into contexts far outside the narrow scope of cem and hyflex while other design principles are more dependent on the specific circumstances (course learning objectives, course content, assignment types, extent of hyflex considerations etc.). unpacking the parts of our admittedly extensive course architecture into ten design principles across three areas thus allows other educators to ‘pick and choose’ design principles to instantiate into their own varying contexts to address specific concerns. on a more abstract and methodological level, we see our treatment of course offerings as design artefacts that merely provide a backdrop for design theorising in the form of unpacking these artefacts into design principles and features that fulfil underlying design requirements as a potential general blueprint to codify design knowledge in the higher education course design context within and beyond the is discipline. with respect to the evolution of our cem teaching approach and related teaching practices, here it was the specific circumstances of the global pandemic of the early 2020 that provided the trigger for us to pursue that path over several years and iterations. on reflection, we cannot help but posit the question what other less widely disruptive triggers could there be under more normal circumstances for a push to evolve one’s teaching practices beyond established boundaries and routines? in order to make the leap from knowledge for design (as the majority of the paper has focused on) to action in the best spirit of impactful design science research at this point, table 11 raises a couple of questions that other educators can use as triggers to start interrogating and advancing their own teaching practices based on this paper’s contributions, and outside of ‘global crises’. the projectability angles are taken from the reflections in tables 6, 8, and 10. since our experiences are limited to a single university and faculty context, the overall personal confidence with which we present our findings cannot be overly high. moreover, any redesign, application, and evaluation of our course architecture has happened between the two authors of this paper and thus carries a certain amount of subjectivity. furthermore, the context around the courses (programme, faculty, university infrastructure, country, situation in the country) has been – pandemic notwithstanding – reasonably stable over the time the hyflex offerings were made. any projection of our design requirements, features, and principles into contexts that vary quite substantially with regards to one or more of those factors may yield entirely different results, however. australasian journal of information systems drechsler & doyle 2025, vol 29, scholarship of is teaching and learning hyflex course architecture conceptual modelling 28 projectability angle call to action other cem courses how can the provided course architecture help to inspire changes in existing cem courses / modules (even if these courses are offered in a predominantly in-person and/or synchronous way)? other courses that offer inperson / remote options and/or have synchronous and asynchronous elements how can the provided design principles and features inform how courses with hyflex facets are being offered in order to enhance aspects such as flexibility or accessibility from a student viewpoint? other is courses how can the course architecture as a whole or specific design principles and features be adapted and applied outside the context of this paper’s courses to review other is courses with respect to their fitness-for-purpose and provide inspirations for their improvement? table 11. three calls to action for other is educators simultaneously, we have now offered several courses across two programmes and topics over several years based on the same overall course architecture and found the resulting formats to be generally effective and appreciated (with a number of caveats based on specific or changed circumstances, cf. section 6.1). based on our own personal confidence in the course architecture, we plan to continue and refine our current course designs for future offerings and even consider expanding it into neighbouring topics such as application development, business analysis, or information security. here, especially the first set of design features or even principles might need some more substantial redesigns for topics that are less ‘hands-on’ and do not have discrete deliverables – such as bp or ea diagrams – as outputs that lend themselves well for weekly exercise assignments. a further limitation for our findings is the limited set of measures (such as pass rates in table 3 and student survey outcomes in table 4) that we can apply to our courses without having set up a specific study design including more tailored measurements in advance. along similar lines, we are limited to assess effectiveness and appreciation for instances of our course and can only infer from these instances to the more abstract design features and even more abstract design principles of the course architecture (lukyanenko & parsons, 2020). moreover, some design principles or features, in isolation, may appear close to common sense, but we believe that their placement and role in the course architecture as a whole makes the course architecture fit-for-purpose and re-usable / projectable in the end. as figure 1 and the subsequent descriptions show, there are many connections between the drs, dps, and dfs within and even across the three general areas of the course architecture. moreover, these levels of abstraction of the course architecture may also not only be a strength of our contribution (in the sense of fostering re-usability or projections into other contexts) but also a weakness. the course architecture does not cover the specific artefacts (such as the course outline, the style and length of the videos, or the assessment design and corresponding briefs) that comprise our course instances. hence, there is a certain limit to the re-usability of the architecture for the design of other cem courses. therefore, the results for other educators adopting our course architecture to their own courses and context may vary because of these small but important nuances. in the absence of an online repertoire, however, the authors are willing to share their specific materials such as teaching plans or assignment briefs upon request. our course architecture thus gives (or rather, forces upon) any future user a certain degree of freedom for their own designs and adaptations. australasian journal of information systems drechsler & doyle 2025, vol 29, scholarship of is teaching and learning hyflex course architecture conceptual modelling 29 8 conclusion and outlook in this paper, we developed six design requirements, ten design principles and eleven design features for hyflex cem courses. these requirements, principles, and features are based on and evaluated against the authors’ experiences in teaching several cem courses with two topics (bp and ea modelling) across two is programmes. we thus contribute a holistic yet generalised course architecture for teaching cem in a hybrid (in-person and online offerings) and flexible (synchronous / asynchronous participation) way. our design principles and features as well as the reported specifics of our course designs allow other instructors in the cem field (and potentially other fields, particularly related to the dp and df of the 2-x / 3-x areas) to take inspiration to develop their own – and hopefully at least equally effective and appreciated – variants for their contexts, as appropriate (see also table 11). particularly the projection of our design principles and features into non-cem contexts or a more focused analysis how such a format can enhance non-mainstream student experiences – e.g., for students with special accessibility needs – are areas we see as promising for future research. there is also room for further research to further investigating whether hyflex course designs indeed lend themselves well to scaling, or whether it was just down to our specific context and the arrangements we made. notably, the only of our three course design areas that was substantially affected by scaling was the communication part, i.e. the design feature around scheduling the synchronous sessions. this is one indication that hyflex courses indeed may lend themselves quite well to scaling, unless a high amount of synchronous sessions are needed, with corresponding implications for staff, rooms, and other resources. we see a major reason for this in the ‘decoupling’ of learning from time and space in the resulting course designs, so that alterations to one facet (in this case, mainly (physical) space) has comparably little bearing on the overall course design and elements as well as the effectiveness of learning. however, we can only make this inference for one specific type of course (cem courses in-line with our provided course architecture), so this also warrants further research attention. beyond our course architecture, we regard a further – more theoretical – exploration of complex learning and the ic/4d model (as introduced in section 2.2) in the is educational context to be a worthwhile endeavour for future pedagogical is research. references abdelmalak, m. m. m., & parra, j. l. (2016). expanding learning opportunities for graduate students with hyflex course design. international journal of online pedagogy and course design (ijopcd), 6(4), 19–37. doi.org/10.4018/ijopcd.2016100102 bandara, w., chand, d., chircu, a., hintringer, s., karagiannis, d., recker, j., rensburg, a., usoff, c., & welke, r. (2010). business process management education in academia: status, challenges, and recommendations. communications of the association for information systems, 27(1), 743–776. beatty, b. j. (2019). hybrid-flexible course design. edtech books. https://edtechbooks.org/hyflex australasian journal of information systems drechsler & doyle 2025, vol 29, scholarship of is teaching and learning hyflex course architecture conceptual modelling 30 binnewies, s., & wang, z. (2019). challenges of student equity and engagement in a hyflex course. in blended learning designs in stem higher education (pp. 209–230). springer. doi.org/10.1007/978-981-13-6982-7_12 bjeković, m., proper, h. a., & sottet, j.-s. (2014). enterprise modelling languages. in b. shishkov (ed.), business modeling and software design (pp. 1–23). springer international publishing. doi.org/10.1007/978-3-319-06671-4_1 bogdanova, d., & snoeck, m. (2018). using mooc technology and formative assessment in a conceptual modelling course: an experience report. proceedings of the 21st acm/ieee international conference on model driven engineering languages and systems: companion proceedings, pp. 67-73. doi.org/10.1145/3270112.327012 bork, d. (2019). a framework for teaching conceptual modeling and metamodeling based on bloom’s revised taxonomy of educational objectives. hawaii international conference on system sciences 2019 (hicss-52). https://aisel.aisnet.org/hicss-52/set/methods_models/7 darwazeh, a. n. (2017). a new revision of the [revised] bloom’s taxonomy. distance learning, 14(3), 13–28. drechsler, a. (2021). from synchronous face-to-face group work to asynchronous individual work: pivoting an enterprise modeling course for teaching during a covid-19 lockdown. communications of the association for information systems, 48(1), 22. drechsler, a., & hevner, a. (2022). knowledge paths in design science research. foundations and trends® in information systems, 6(3), 171–243. doi.org/10.1561/2900000028 drews, p., & schirmer, i. (2014). from enterprise architecture to business ecosystem architecture: stages and challenges for extending architectures beyond organizational boundaries. 2014 ieee 18th international enterprise distributed object computing conference workshops and demonstrations, 13–22. doi.org/10.1109/edocw.2014.12 dumas, m., rosa, m. l., mendling, j., & reijers, h. a. (2018). fundamentals of business process management (2nd ed. 2018 edition). springer. eduglopedia. (2022). topics. https://eduglopedia.org/keyword/all?sorting=&display=cloud fox, m. s., & grüninger, m. (1997). ontologies for enterprise modelling. in enterprise engineering and integration (pp. 190–200). springer. doi.org/10.1007/978-3-642-60889-6_22 gregor, s. (2006). the nature of theory in information systems. mis quarterly, 30(3), 611–642. doi.org/10.2307/25148742 gregor, s., & hevner, a. r. (2013). positioning and presenting design science research for maximum impact. mis quarterly, 37(2), 337-a6. hoppenbrouwers, s. (2012). asking questions about asking questions in collaborative enterprise modelling. in k. sandkuhl, u. seigerroth, & j. stirna (eds.), the practice of enterprise modeling (pp. 16–30). springer. doi.org/10.1007/978-3-642-34549-4_2 hoppenbrouwers, s., proper, h. a., & van der weide, t. p. (2005). formal modelling as a grounded conversation. proceedings of the 10th international working conference on the language action perspective on commmunication modelling, 139–155. australasian journal of information systems drechsler & doyle 2025, vol 29, scholarship of is teaching and learning hyflex course architecture conceptual modelling 31 jung, r., & lehrer, c. (2017). guidelines for education in business and information systems engineering at tertiary institutions. business & information systems engineering, 59(3), 189–203. doi.org/10.1007/s12599-017-0473-5 keiper, m. c., white, a., carlson, c. d., & lupinek, j. m. (2020). student perceptions on the benefits of flipgrid in a hyflex learning environment. journal of education for business, 96(6), 343–351. doi.org/10.1080/08832323.2020.1832431 kester, l., & van merriënboer, j. j. g. (2021). implications of the four component instructional design model for multimedia learning. in l. fiorella & r. e. mayer (eds.), the cambridge handbook of multimedia learning (3rd ed., pp. 100–120). cambridge university press. doi.org/10.1017/9781108894333.011 krathwohl, d. r. (2002). a revision of bloom’s taxonomy: an overview. theory into practice, 41(4), 212–218. doi.org/10.1207/s15430421tip4104_2 laurillard, d. (2013). teaching as a design science: building pedagogical patterns for learning and technology. routledge. doi.org/10.4324/9780203125083 leidig, p., & salmela, h. (2021). is2020: a competency model for undergraduate programs in information systems. acm & ais. www.acm.org/binaries/content/assets/education/curricularecommendations/is2020.pdf lukyanenko, r., & parsons, j. (2020). research perspectives: design theory indeterminacy: what is it, how can it be reduced, and why did the polar bear drown? journal of the association for information systems, 21(5). doi.org/10.17705/1jais.00639 malczyk, b. r. (2019). introducing social work to hyflex blended learning: a studentcentered approach. journal of teaching in social work, 39(4–5), 414–428. doi.org/10.1080/08841233.2019.1652226 meth, h., mueller, b., & maedche, a. (2015). designing a requirement mining system. journal of the association for information systems, 16(9). http://aisel.aisnet.org/jais/vol16/iss9/2 miller, j., risser, m., & griffiths, r. (2013). student choice, instructor flexibility: moving beyond the blended instructional model. issues and trends in educational technology, 1(1), 8–24. doi.org/10.2458/azu_itet_v1i1_miller nicolai, a. t., & seidl, d. (2010). that’s relevant! different forms of practical relevance in management science. organization studies, 31(9/10), 1257–1285. doi.org/10.1177/0170840610374401 recker, j., lukyanenko, r., jabbari sabegh, m., samuel, b., & castellanos, a. (2021). from representation to mediation: a new agenda for conceptual modeling research in a digital world. mis quarterly: management information systems, 45(1), 269–300. recker, j., & rosemann, m. (2009). teaching business process modelling: experiences and recommendations. communications of the association for information systems, 25(1), 32. rosenthal, k., ternes, b., & strecker, s. (2019). learning conceptual modeling: structuring overview, research themes and paths for future research. proceedings of the ecis 2019 conference. https://aisel.aisnet.org/ecis2019_rp/137/ simon, h. a. (1996). the sciences of the artificial (3rd ed). mit press. australasian journal of information systems drechsler & doyle 2025, vol 29, scholarship of is teaching and learning hyflex course architecture conceptual modelling 32 tanner, m., & scott, e. (2015). a flipped classroom approach to teaching systems analysis, design and implementation. journal of information technology education: research, 14(2015), 219– 241. www.jite.org/documents/vol14/jitev14researchp219-241tanner1840.pdf thalheim, b. (2010). towards a theory of conceptual modelling. journal of universal computer science, 16(20), 3102–3137. venable, j., pries-heje, j., & baskerville, r. (2016). feds: a framework for evaluation in design science research. european journal of information systems, 25(1), 77–89. doi.org/10.1057/ejis.2014.36 vom brocke, j., winter, r., hevner, a. r., & maedche, a. (2020). special issue editorial– accumulation and evolution of design knowledge in design science research: a journey through time and space. journal of the association for information systems, 21(3), 520–544. doi.org/10.17705/1jais.00611 wand, y., & weber, r. (2002). research commentary: information systems and conceptual modeling—a research agenda. information systems research, 13(4), 363–376. doi.org/10.1287/isre.13.4.363.69 copyright copyright © 2025 drechsler and doyle. this is an open-access article licensed under a creative commons attribution-non-commercial 4.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: doi.org/10.3127/ajis.v29.5773 paper title (paper title style) australasian journal of information systems sharma, s., kar, a. k., gupta, m. p. 2025, vol 29, research article digital accountability received 9 april 2024, accepted 30 december 2024 after 2 rounds of review with karlheinz kautz as accepting editor 1 building accountability in e-government services: inputs for policy swapnil sharma international management institute, delhi new delhi, india email: swapnil.sharma@imi.edu arpan k. kar indian institute of technology, delhi new delhi, india m. p. gupta indian institute of technology, delhi new delhi, india abstract governments worldwide are investing many resources in developing digital government infrastructure and networks. government webpages and supersites are substituting for their brick-and-mortar offices and physical state-citizen communication. this shift is transforming the administration and the process of digital government. we also see a growing push for expanding the role of citizens as participants and co-creators of policy and programs for establishing a collaborative digital government. this study examines the indian e-government setup to explain how governments can ensure ‘accountability by policy design,’ or digital accountability (da), on e-government service (egs) websites. a mixed-method research design is used to uncover the critical design factors that can help build and maintain accountability on any government service (egs hereafter) website. our results show that transparency remains the most important dimension, but concerns about security and privacy have also become foundational to the conceptualisation of accountability. another important finding shows that building accountability is meaningful only if there is responsiveness and a sense of user control over the services. the findings also establish an explicit requirement to establish liability for service quality and effectively enforce a sense of accountability in modern egs. we believe our findings can help improve the theoretical understanding of accountability in egs while providing actionable insights to practitioners and policymakers to ensure accountable services in the digital age. keywords: e-government, digital accountability, tism, micmac, best-worst method 1 introduction governments across the globe are experimenting with newer forms of governance that leverage the rapidly evolving digital infrastructure and technologies. digital era governance (deg) is one such model of governance, and it was seen as a successor to the new public management (npm) paradigm (dunleavy et al., 2006; margetts & dunleavy, 2013). deg is about transforming the way governments function, and governance is delivered. as per the proponents of the thought, we are in the second wave of deg. this entails overhauling the public sector based on reintegration: streamline and integrate different government agencies through digital platforms; need-based holism: structuring government services and operations around the specific needs of citizens; and digitisation of the governance approach. australasian journal of information systems sharma, s., kar, a. k., gupta, m. p. 2025, vol 29, research article digital accountability 2 it may be observed in the way government departments’ websites are replacing their physical offices at the lowest levels of state-citizen interaction for egs delivery (lindquist & huse, 2017; sharma, kar, et al., 2022). this movement is leading to an evolution of governments and society by acknowledging and accepting how transformative digital technologies enhance egs delivery (bindu et al., 2019; lindgren et al., 2019). research shows that by encouraging digital access to information, governments want to promote efficiency in egs to benefit individuals and society (hasan & linger, 2020; verma et al., 2022). online marketplaces, platforms, social media, machine learning, the internet of things, artificial intelligence and big data analytics are examples of technologies that encourage social and institutional innovation, facilitating the transition to deg (stamati et al., 2015). there are other conceptualisations of public organisations, such as public value management, collaborative governance, and new public governance, among others (greve, 2015). however, we have chosen to work with the deg paradigm for our study, as its sub-categories covering transparency, social media, and shared service centres make it the closest representation of the governance practices in india and countries with similar technology diffusion and socio-cultural context (margetts & dunleavy, 2013). it coincides best with how india is trying to push the envelope of governance with the digital india program and establish an improved egs infrastructure in the country (meity, 2022). india is the world’s largest democracy and one of the most diverse countries. working in the indian context also allows us to emphasise the generalisability of our results to developing nations and societies that might be suffering from issues like the digital divide and unequal diffusion of technology. the fact that this is an ongoing transition doesn’t help much, given the external and internal resistance towards change in existing government infrastructure and institutions (clarke, 2020). we are not yet close to realising a complete transformation of our governments, and technology is not the silver bullet that will solve all of our administrative issues. technology has also been accused of aggravating existing inequalities in our communities and society (van deursen et al., 2017). scholars have often proposed enhancing the accountability of government agencies and processes as a solution, which is the basis for our research (sharma et al., 2024; twizeyimana & andersson, 2019). the goal is to create an e-government architecture that can underpin future information societies where individuals are digitally connected and use the internet for service access, fostering a sense of accountable government focused on social welfare. accountability has been defined in multiple cultural, institutional, and organisational contexts. however, for the sake of this study, we define digital accountability (da) as a social relationship enacted by embedded design factors in egs-delivering websites that ensure it presents a sense of accountability towards the citizen users. the study is motivated by the idea that imposing accountability will address problems like middlemen, price manipulation, corruption, bureaucracy, and red-tapism (harrison & sayogo, 2014; matheus & janssen, 2020). goal 16 of the sustainable development agenda of the united nations also calls for establishing transparent institutions that could support local digital ecosystems and contribute to the growth of inclusive and sustainable societies (united nations, 2022). the organisation for economic co-operation and development (oecd) supports retaining accountability while presenting the concept of digital-by-design egovernment platforms to promote transparent, inclusive, and responsive governance (oecd, 2020). ensuring da can improve egs quality and make the government machinery more australasian journal of information systems sharma, s., kar, a. k., gupta, m. p. 2025, vol 29, research article digital accountability 3 accessible and transparent to the citizens. furthermore, a lack of accountability breeds distrust in the state-citizen relationship, undermining the roots of democratic e-government. the principles of open data, citizen social networking, collaborative democracy, and interactive web 2.0 apps would all be embodied in this type of governance model (bindu et al., 2019). according to recent studies, da is crucial to these e-governance approaches and is underappreciated even in developed countries (clarke, 2020). this study attempts to establish that by ensuring accountability in the structure and policy of egs websites, we can more efficiently assist policy execution and promote social good in developing countries. the problem of in-efficient government services persists even after all the technological innovations in egs delivery, and just like the digital divide, this isn’t the fault of the technology but of the people and policies that govern the technology (mervyn et al., 2014). this issue is of special importance for developing democracies across the globe as they hold the biggest potential for public welfare. this manuscript is woven around two questions that could help us in exploring the conceptual build-up of da in egs websites serving citizen users: 1. how can we conceptualise accountability for egs delivery (da) regarding the contributing design factors? 2. what causal interrelationships exist among these design factors in ensuring da? we have used a mixed-method approach using a qualitative structural technique and a quantitative multi-criteria decision-making technique to tackle the two research questions. both of these techniques are epistemologically complementary as they build upon the opinions of subject matter experts to present a picture of reality through their experiences. we were able to conceptualise da in terms of 6 design factors or dimensions collated from a list of 29 features identified by experts on an egs website as transparency, controllability, responsibility, responsiveness, liability, and security & privacy. by combining these results with the structural modelling technique, we also explored the interrelationships these design factors or dimensions of da have with each other. we have discussed the details in the following sections of the paper, structured as follows: the background literature crossing the domains of information systems and public administration is reviewed in section 2; section 3 delves into the study’s methodology, covering the design of this study and the analysis conducted to explore the research questions; section 4 presents the findings from the analysis. in section 5, we highlight the theoretical contribution of this work as well as its practical utility for practitioners. section 6 wraps up the document by summarising the study and acknowledging its limitations. 2 background accountability is a very old concept and has always had its place in conversations around governance and administration in different forms and enactments. researchers see the concept of accountability as a way of making governments more efficient, transparent, responsive and restricting the abuse of power by the state and its officials (dubnick & justice, 2004). it is proposed as a remedy for administrative challenges in a lot of contemporary literature in the digital governance and public administration fields (petrakaki, 2018). the public administration dictionary defined it as “a condition in which individuals who exercise power are constrained by external means and internal norms” (koppell, 2005). in the following subsections, we have attempted to cover the different conceptualisations of accountability discussed in the past literature and how they could help us realise a guiding framework or design for enforcing da. australasian journal of information systems sharma, s., kar, a. k., gupta, m. p. 2025, vol 29, research article digital accountability 4 2.1 conceptualising accountability meijer (2003) outlines how political and legal processes impose external accountability while the organisation’s bureaucracy enforces internal accountability. external responsibility occurs outside the organisation’s bureaucracy, whereas internal accountability exists within it. for example, the service head in a local government reports to the council’s executive. moreover, government employees are bound by the rules set by the institutions in which they serve. external accountability is enabled via political and legislative structures and civil society activism. bovens (2005) illustrated accountability as a social relationship where an actor must explain and justify their conduct to the public affected by their decisions. bovens (2007) provides a broad overview of the phenomenon and a comprehensive outlook on the concept of accountability, outlining 15 different types of accountability relationships based on their contexts. depending on the firm’s nature, it can be legal, political, social, administrative or professional. based on the actor’s nature, it can be individual, hierarchical, corporate or collective (bovens, 2007). it can be product-related, procedural or financial, depending on the conduct. it could also be seen as horizontal, vertical or diagonal, based on the nature of duties. accountability was discussed as a means of resolving key governance issues of performance, control, and legitimacy in any public management context by using organisational and managerial control, performance assessment, and democratic governance (dubnick & frederickson, 2011). researchers have also conceptualised accountability in terms of citizen voice and the state’s response to the supply and demand channels of egs. performance or compliance monitoring institutions that restrict abuses by government agencies and officials establish state-centred or horizontal accountability, making up the supply side of accountability. social accountability or vertical accountability is established by actions and measures taken by individuals and civil society to make public officials answerable for their decisions; this is the demand side of accountability (brinkerhoff & wetterberg, 2016; j. fox, 2014). social accountability is gaining appeal as a counterbalance for inadequate state-centred accountability, and an increasing body of knowledge and research examines its usefulness and implications for governance and policy goals. past literature also caveats the need for citizen engagement, data journalism and institutional support for e-government, as open data alone cannot ensure accountability (matheus & janssen, 2020). 2.2 digital accountability and e-government e-government or electronic government is the phenomenon of using digital technologies in the agencies and institutions of the government to conduct administration and business to provide improved governance, better and easier delivery of services, and promote democratic participation (a. meijer, 2007; roy, 2006). it aids in restraining officials’ discretionary powers, reduces the risk of corruption, and strengthens the weak social, legal, and political institutions (mistry, 2012). thanks to the internet, citizens now readily search and acquire information on government programmes and undertake service transactions. information repositories and libraries have become critical for record-keeping in the highly dynamic and volatile environment of information societies and the institutions that govern them using digital means (paul, 2007). as noted by bannister & connolly (2011), the use of ict in government is egovernment. nevertheless, until this usage leads to a material change in present structures, data, or processes, we should not term it e-governance. it is critical for the success of any egovernment initiative to have : (a) a centralised, secure database to keep track of and enable australasian journal of information systems sharma, s., kar, a. k., gupta, m. p. 2025, vol 29, research article digital accountability 5 inter-departmental data exchange; (b) design and delivery of electronic services; (c) provision to conduct business with the government digitally; (d) ensuring government institutions and officials’ accountability to facilitate digital democracy (bolívar et al., 2007). pina, torres, & royo (2007) studied more than 300 government websites across the european union (eu) analysing the transformational impact of digitally enhanced accountability (da). they defined the degree of openness for an egs website as a combination of the interactivity and transparency factors and a technique to objectively assess them. the existing literature also covers the ongoing debate about the openness of government systems vs citizen data protection and privacy, as well as national security issues and how they impact accountability (as-saber et al., 2007; sullivan & clarke, 2010). assessing and ensuring accountability can help promote citizen participation, which is critical for developing healthy digital democracies (sharma, kar, et al., 2022). according to academics, e-government procedures result in collective learning for both the people and the government in terms of identifying service gaps and co-designing new solutions (barrett, 2019; a. meijer, 2007). there may also be indirect benefits, such as enhanced government legitimacy and citizen confidence (keymolen et al., 2012). da mechanisms can also act as a catharsis for the populace by allowing them to push for punitive actions in the event of failures in e-service delivery by responsible government officials. figure 1: process of da in egs delivery taking from past literature, we can outline the accountability relationship comprising three different phases (see figure 1), and the design factors of the website will enable and support different phases of this relationship. first, the citizen forum collects data from different sources to reconstruct the events of rendering governance services. then, the forum members discuss these events and evaluate actions and decisions based on defined service standards. finally, sanctions are enforced depending on the forum’s judgement in the third step. performance statistics, program information, institutional accountability metrics, user feedback and australasian journal of information systems sharma, s., kar, a. k., gupta, m. p. 2025, vol 29, research article digital accountability 6 satisfaction data will be shared. the forum engages with this knowledge and makes the holdee accountable regarding their conduct, with institutionalised sanctions such as negative evaluation, wage/budget reductions, and the possibility of expulsion from power based on the forum’s consensus. as a result, sanctions become the most crucial and final component of the relationship, as organisational accountability compliance would be impossible without them. based on these phases and the different dimensions of da that enable them, we have designed our study to come up with a conceptual framework for embedding and enabling da mechanisms in government e-service-delivering websites and platforms. the following subsection details the mixed-method research approach adapted to build the theoretical foundations of digital accountability. based on the background literature discussed earlier, we can conceptualise the da relationship as shown in figure 1. governments, politicians, public service providers, and professionals may all be held accountable. citizens, civil society experts, or consumers of these public services would all be the holders of accountability. the forum will be the web-sphere enabled by the website in the form of online blogs, forums or social media platforms that could sustain the interactions between the government and its citizen users. citizens already use such media, but their voices often go unheard without a standard procedure to collect, analyse and act on these voices to improve egs (rathore et al., 2021). 2.3 building a guiding framework to enforce digital accountability the guiding research questions for this study define the scope for us. the study is an attempt to conceptualise da to improve the quality of egs delivery and support the development of digital democracies. the objectives that give this study its binding scope are focused on exploring the different design factors that create the perception of being accountable to users. also, to explore how these design factors affect each other in building up an accountable egs infrastructure in the country. the reason for choosing this is the constant lack in adopting egs and participation from citizens in developing countries (lindgren et al., 2019). research claims that the black-box nature of administration and the feeling of being unheard in the citizens is the major cause of this. by building accountable e-government machinery, we can overcome these issues. recent research also points to the possibility of employing participatory design in the public sector to successfully develop and implement administrative solutions (kautz et al., 2020). rq1 builds up on past literature to come up with design factors that cover different dimensions of accountability in the state-citizen interaction online. we leverage the knowledge and experience of experts to refine these factors further and develop a list of factors that cover the major functional issues of egs-delivering websites or web applications. this will allow us to understand the different features and functionalities we must build in the service to maintain a sense of accountability to the citizen users. rq2 delves a bit further and explores the causal relationship between these uncovered design factors to understand their influence on each other and in building up accountability in the state-citizen interaction on digital platforms. it allows us to handle the design in a better way by providing an insight into the nature of these design factors. we hope to understand what factors might be foundational to accountability and what might be critical in operationalising the concept in the real world. australasian journal of information systems sharma, s., kar, a. k., gupta, m. p. 2025, vol 29, research article digital accountability 7 3 research design and methodology the article adopts a mixed-method approach to tackle the questions guiding this study. we can combine methods with comparable ontological and epistemological assumptions in information systems research to obtain unique insights while investigating complex phenomena (bjoern, 2005). both methods used to analyse results from the review of literature and expert discussion allow us to rank different factors based on causal relationships. 3.1 research design figure 2 depicts the research design we followed for this study. we organised two focus groups with experts. a total of 45 experts were invited across different fields of expertise dealing with e-service delivering websites, and we were able to secure the attendance of 21 for the two panel discussions. these experts were recruited from the professional network of the authors, exploring experts in different ministries, government bodies, industry, and academia who had substantial experience in the e-government domain. we formally approached them by mail or by contacting their offices for possible participation. panel 1 comprised 14 practitioners with 7-17 years of experience, and panel 2 had seven senior area experts with more than 20 years of experience (see table a.1 in the appendix). expert panel 1 was organised at the author’s institute, while expert panel 2 was engaged online through a video conferencing platform. figure 2: synthesis of studies a literature review was conducted in the first stage of the investigation to extract academic publications from web of science and scopus. although we did not explicitly use a systematic literature review framework like prisma, our search was still structured. we used a keyword sieve combining keywords like ‘accountability’, ‘government accountability’, and ‘political accountability’ with the or operator in the title-abstract-keyword field. this turned up many search results as accountability is a very widely used term, and we had to narrow our search down by filtering through specific keywords used by past researchers. de-duping was done australasian journal of information systems sharma, s., kar, a. k., gupta, m. p. 2025, vol 29, research article digital accountability 8 to reduce the list of articles and include unique studies. metadata for relevant studies for this manuscript was downloaded, and we read through the abstracts to choose if the paper dealt with different dimensions related to the concept of accountability. any study that dealt with the concept of accountability in service delivery was included, whatever the context in which the study was conducted. this was done to maintain a comprehensive list of factors to start with. this allowed us to understand what researchers mean when discussing accountability in real-world applications. we also included studies that discussed accountability as a possible solution to problems with egs in any capacity. this also allowed us to build a corpus of background research. panel 1 was asked to present their ideas and discuss the findings from past literature to shortlist the contributing factors of accountability for egs. this was done by initiating a discussion on the topic ‘what makes up accountability on a website?’ and the ideas expressed by the experts were collected in the form of notes which talked about different features, functions, and information the experts expected to see on an accountable website. these features comprise the list of sub-factors like citizen charter, organisational structure, roles and responsibilities document, among others. once we collected these features and the group agreed that these features were sufficient to ensure accountability for egs, we moved towards further grouping these sub-factors for parsimony in the analysis. the first five factors were from past literature, and the panel felt that there should be an additional dimension of security & privacy covering features like user data storage and sharing policies, owing to the recent developments in the indian privacy law and a supreme court judgement on the same in 2018. the definitions of these dimensions are context-specific and developed with the help of experts’ opinions. they describe the meaning of a particular factor concerning egs delivery. twenty-nine sub-factors on and off the website were chosen based on a consensus-building exercise, later organised under six overarching design factors of da selected from the literature and experts’ discussion (see table 1). panel 1 experts were also asked to create comparison vectors for applying the bwm method, thereby generating the weights for each factor. 3.2 best-worst analysis we used a multi-criteria decision-making (mcdm) approach known as the best-worst method (bwm) to analyse the shortlisted factors. bwm aids in determining the relative importance of dimensions, as well as their inferred rankings, in terms of ensuring da. bwm is similar to the analytical hierarchy process (ahp) but does not require the full pairwise comparison matrix. it generates consistent results with fewer data points (rezaei, 2015, 2016). bwm is comparable to how decision-makers evaluate choices and make judgments in real situations. it has been applied to a variety of issues in different disciplines of study, including innovation management (gupta & barua, 2016), business-to-government data exchange factors (van de kaa et al., 2018), measuring research and development performance (salimi & rezaei, 2018), supplier classification and risk assessment (torabi et al., 2016), examining implementation success factors for identity systems (mir, kar, et al., 2020) and for developing artificially intelligent robotic systems (mir, sharma, et al., 2020). once the design factors, sub-factors and relative weights were finalised, we invited panel 2 to take the study forward. the senior practitioners in panel 2 contributed to the tism-micmac analysis by reviewing the results from the bwm analysis and coming up with causal explanations for the design factor interrelationships. we can create a comparative analysis of australasian journal of information systems sharma, s., kar, a. k., gupta, m. p. 2025, vol 29, research article digital accountability 9 the results generated from the two methods as one gives us relative rankings of the factors. in contrast, the other generates a hierarchical diagraph of the same factors. the levels of the tism diagraph also refer to the factor’s role and ranking in enforcing da in service delivery. 3.3 tism & micmac analysis following the mixed-method approach, we employed a structuring methodology, total interpretive structural modelling (tism), to examine the interrelationships between these dimensions and corroborate their perceived relevance in the selected context. it is an improvement over interpretive structural modelling (ism), a method for transforming ambiguous and abstract mental models into accurate hierarchical structures of elements by understanding and presenting their contextual links on the diagraphs’s connectors for clarity (sushil, 2012). it allows us to create a conceptual model of da and explain the causal relationship between the design factors in building it up as a concept and operationalising it in the real world. tism has been used extensively by scholars for diverse enquiries like: choosing a city for smart city project (kumar et al., 2019), identifying barriers to using big data analytics for a sustainable auditing system (shukla & mattar, 2019), for sustainable supply chain performance (shibin et al., 2017), and for developing an ecosystem view of collaborative m-governance (sharma, mir, et al., 2022), for identifying and analysing change forces in egovernment (nasim, 2011), for creating a benchmarking model for education (yeravdekar & behl, 2017) and for analysing barriers to adoption of blockchain technology (mathivathanan et al., 2021). tism allows us to create a hierarchical digraph for the chosen factors with their interrelationships based on the experts’ opinions in panel 2. it allows us to understand how different dimensions are connected to each other, and the why of these relationships is explained on the connector (jena et al., 2017). the levels in this diagraph can also be used to rank the dimensions in order to validate the bwm results. the tism technique’s step-by-step method with the generated tables and knowledgebase are outlined in subsequent sections of the paper. the matrix of cross impact multiplications applied to classification (micmac) was used to investigate these factors further based on the reachability matrix developed earlier. the driving power of these dimensions is ascertained by taking the row sum of the reachability matrix and dependence by taking the column sum (see table 5). these sums indicate the influence of any dimensions over other dimensions by virtue of their reachability (driving power) or the influence of other dimensions in ensuring them (dependence). we create a 2 by 2 matrix based on these numbers with driving power and dependence as the two axes. we do this by plotting the dimension on a graph based on these two values and dividing the two axes into halves, giving us the four classifications: drivers, dependents, linkage and autonomous factors. drivers are the factors that have the maximum impact on the objective to be achieved by contributing most to the relationships in the structural model. they could be seen as the main components contributing to building up a concept or idea. dependents are factors that could be seen as constituents or features of the outcome or objective. linkage factors lie between the drivers and dependents and hold up the whole conceptual structure being explored. they led the past from the drivers to the dependent variables. autonomous variables contribute little to explaining the relationships being investigated in the structural model of the concept. australasian journal of information systems sharma, s., kar, a. k., gupta, m. p. 2025, vol 29, research article digital accountability 10 4 results this section covers the different findings generated during the course of this research under three sub-sections. 4.1 design factors of da from the review of literature and expert’s validation we undertook an exercise to uncover and shortlist design factors for a website, which could help ensure da. we ended up with a list of 6 factors, covering 29 sub-factors as per the literature review and expert’s opinion. f# factor name definition sub-factors source f1 transparency the government website offers all service and performance information • citizen charter • service data dashboards • provisioning performance data in clear understandable metrices • organisational structure • role and responsibility of office bearers (ackerman, 2005; bovens, 2005, 2007; brandsma & schillemans, 2013; fox, 2015; koppell, 2005; pina et al., 2007, a. j. meijer, 2003; misuraca & viscusi, 2015; pina et al., 2007; roy, 2006) f2 controllability a government website performs what’s necessary and incorporates civilian feedback • allowing participatory policy and service design • facilitating citizen intervention at different process levels • allowing for practices such as participatory budgeting of resources and negotiated policy making (aman et al., 2013; bertot et al., 2012; bovens, 2005, 2007; brinkerhoff & wetterberg, 2016; dubnick & frederickson, 2011; eshiet, 2019; koppell, 2005; sørensen, 2012; dahiya, 2016; t. (david) lee et al., 2019; mahmood, 2016; petrakaki, 2018; harrison & sayogo, 2014) f3 responsibility the website and its administrators are cognizant of the terms and conditions of the services rendered • developing and publishing kpis • publishing audit information • providing service statistics • service continuity planning • service level agreements • process tracking facility ( bovens, 2005, 2007; brandsma & schillemans, 2013; drach-zahavy et al., 2018; fox, 2015; halachmi & greiling, 2013; heinrich & brown, 2017; hladchenko, 2016; keymolen et al., 2012; lee & joseph, 2013; a. j. meijer, 2003; pina et al., 2007; roy, 2006) f4 responsiveness must have an efficiently interactive design which can facilitate citizen interactions • facility of feedback • active two-way communication over service issues • timely resolution of citizen demands • flexibility in operations to accommodate citizen requirements • reporting back to citizens regarding their queries and complaints • maintain a digital space for interaction between users and with the government institution (aman et al., 2013; bertot et al., 2012; bovens, 2005, 2007; brinkerhoff & wetterberg, 2016; dubnick & frederickson, 2011; eshiet, 2019; koppell, 2005; sørensen, 2012; dahiya, 2016; t. (david) lee et al., 2019; mahmood, 2016; misuraca & viscusi, 2015; petrakaki, 2018) australasian journal of information systems sharma, s., kar, a. k., gupta, m. p. 2025, vol 29, research article digital accountability 11 f5 liability the website should have a clear mechanism for holding accountable those responsible for service failures • appointing spocs • publicly recognising process owners • provisioning sanctions in the slas in case of service failures ( bovens, 2005, 2007; brandsma & schillemans, 2013; brinkerhoff & wetterberg, 2016; drachzahavy et al., 2018; fox, 2015; halachmi & greiling, 2013; heinrich & brown, 2017; koppell, 2005; pina et al., 2007; hladchenko, 2016; keymolen et al., 2012; lee & joseph, 2013) f6 security and privacy the website should have a well-defined security and privacy policy that includes access restriction and data sharing rules • user data requirements • data storage policy • data sharing policy • possible usage of data for profiling and targeting • undertaking compliance measures (aman et al., 2013; brandsma & schillemans, 2013; koorn et al., 2010; as-saber et al., 2007; keymolen et al., 2012; maréchal, 2015; sullivan & clarke, 2010) table 1: design factors of digital accountability the first factor was transparency, which is often used interchangeably with accountability. it is related to the openness of the egs website and how well it communicates and presents relevant information to citizen users. some of the important sub-factors clubbed by the experts under the definition of transparency include a citizen charter, data dashboards, performance data on different services and officials in clear and understandable form, along with a description of the organisational structure with clearly defined roles and responsibilities of the officials. the second factor was controllability, which is the dimension to which a website allows to capture citizen feedback. the third factor was responsibility, which is related to the explicitly expressed awareness of the service providers to their promised service standards via the website. the third factor was responsiveness, which covers the level of interactivity offered on the website and ensures that the exchange of information is two-way. the fourth factor was liability, which covers subfactors like identifying process owners and single points of contact (spocs) in times of service failure and a description of punitive action that needs to be taken in such a case to maintain answerability in the da relationship. the last factor identified was that of security & privacy, which covers the website’s data protection and sharing policy along with a clear description of access levels, data management and usergenerated or submitted data usage. 4.2 ranking of the factors based on study 1 bwm was then employed using two independent comparison vectors created by defining which criteria are the best (for this study, f1transparency) and the worst (f3responsibility) based on the consensus in panel 1. readers should note that the terminology of worst is just a need of the method and in no way says that the chosen dimension is detrimental or in any way insignificant to the analysis. it is a relative term that creates a reference for comparison and makes the job of the experts smoother. numbers are assigned from 1 and 9 to each criterion, with 1 denoting ‘equally significant’ and 9 denoting ‘most significant’ to generate the best-toothers and worst-to-others matrices. australasian journal of information systems sharma, s., kar, a. k., gupta, m. p. 2025, vol 29, research article digital accountability 12 best to others transparency controllability responsibility responsiveness liability security & privacy transparency 1 3 5 4 2 3 table 2: comparison vector 1best to others others to the worst responsibility transparency 5 controllability 3 responsibility 1 responsiveness 2 liability 4 security & privacy 3 table 3: comparison vector 2others to worst min-max inequalities are developed based on these vectors as per the weight estimation procedure of bwm (see appendix a.1) to generate each dimension’s relative weight, as shown in table 4. factors transparency controllability responsibility responsiveness liability security & privacy weights 0.36 0.14 0.06 0.10 0.20 0.14 table 4: weights for the selected dimensions the consistency ratio for our comparisons was 0.0235 and this being below 0.25 signifies a highly consistent comparison (rezaei, 2015). 4.3 the hierarchical model of da and factor classification from study 2 to validate the bwm results, we used the qualitative tism and micmac techniques in consultation with experts on panel 2. experts’ opinions were used to derive the reachability matrix, which gives all possible interrelationships between the factors in the context of ensuring da. rm f1 f2 f3 f4 f5 f6 driving power f1 1 1 1 1* 1 1 6 f2 0 1 1 1 0 0 3 f3 0 0 1 1 0 0 2 f4 0 0 1 1 0 0 2 f5 0 1 1 1 1 0 4 f6 1 1* 1* 0 1 1 5 dependence 2 4 6 5 3 2 table 5: fully transitive reachability matrix with dependence and driving power the ones with an asterisk (1*) denote significant transitive relationships between factors. this matrix’s row and column sums also ascertain each factor’s driving power and dependence. this data is used in the micmac classification. level partitioning gives us the levels (in table australasian journal of information systems sharma, s., kar, a. k., gupta, m. p. 2025, vol 29, research article digital accountability 13 6) for the hierarchical diagraph (figure 3) for every factor as per the process outlined in appendix a.3. factors reachability set antecedent set intersection set level f1 {1,2,3,4,5,6} {1,6} {1,6} 4 f2 {2,3,4} {1,2,5,6} {2} 2 f3 {3,4} {1,2,3,4,5,6} {3,4} 1 f4 {3,4} {1,2,3,4,5,6} {3,4} 1 f5 {2,3,4,5} {1,5,6} {5} 3 f6 {1,2,3,4,5,6} {1,6} {1,6} 4 table 6: levels in the canonical matrix these levels allow us to present the design factors of da in hierarchical structures with their contextual interrelationships on the connectors, as explained by the panel 2 experts. in the tism diagraph, the levels increase from bottom to top, and the ranking decrease in the same direction. therefore, we can flip the level numbers and get the rankings of these dimensions based on the tism analysis. figure 3: conceptual diagraph of digital accountability we performed the micmac classification based on the tism reachability matrix developed previously (table 5). a plot of the dimensions was created based on their dependence and driving powers to develop the 2*2 matrix of micmac. for this analysis, we took driving power as the x-axis and dependence as the y-axis, creating the classification of dimensions below. australasian journal of information systems sharma, s., kar, a. k., gupta, m. p. 2025, vol 29, research article digital accountability 14 figure 4: classification of da dimensions in the context of da, transparency and security & privacy emerged as driving factors having the greatest effect over the context, whereas responsiveness and responsibility are dependent dimensions. the aspects of liability and controllability figured as connecting factors bridging the driving and dependent dimensions. 5 discussion this manuscript explores the conceptualisation of digital accountability for egs. the two issues allow us to understand the different design factors that can be used to ensure accountability by design into egs websites and its supporting policy. we sought to determine the relative importance and interrelationships of these factors. the table below demonstrates how we critically compared and validated the results leveraging the mixed-method approach adopted for this study. comparison table factors method i method ii bwm weights bwm ranks tism weights tism ranks micmac class f1 transparency 0.36 1 4 1 driving factor f2 controllability 0.14 3 2 3 linkage factor f3 responsibility 0.06 5 1 4 dependent factor f4 responsiveness 0.10 4 1 4 dependent factor f5 liability 0.20 2 3 2 linkage factor f6 security and privacy 0.14 3 4 1 driving factor table 7: comparative summary of findings in the chosen scenario, transparency and security & privacy were clustered as the two most significant driving components, contributing to almost half of all accountability measures. australasian journal of information systems sharma, s., kar, a. k., gupta, m. p. 2025, vol 29, research article digital accountability 15 literature establishes that accountability cannot be imposed without system transparency. transparency has the maximum weight as a component of da since it has essentially been synonymous with accountability for a long time, serving as the basis for accountable democratic processes (hladchenko, 2016). the first facet of the da relationship, that is, facilitating data for recreating scenarios of service failure and aiding citizen users in evaluating or judging the website’s performance in light of the set goals for its corresponding department or office, is solely enacted and based on transparency (lindgren et al., 2019; petrakaki, 2018). open data facilitation does not end with adopting an open data policy. however, it is about iterative, incremental improvements through which the organisation responds to various stakeholders’ requirements and uses open data with the changing times (park & gil-garcia, 2022). it solves government service’s lack of openness by quickly providing all vital information and allows engagement with and accommodating citizen feedback, with controllability and responsiveness dimensions. although security and privacy appear less significant in the bwm results, the tism-micmac results indicate that this is a driving factor in the da context. it dictates the online service’s balance for obtaining, storing, and using citizen data securely while developing and rendering services for them. it discusses the tradeoffs governments and agencies must strike when balancing national security and user profiling issues for policy and service design (as-saber et al., 2007; mahmood, 2016). they dictate how the government office keeps its digital environment safe and secure while employing government data for social good and how the principals define guidelines for citizen services (maréchal, 2015). they represent the trade-offs in delivering e-services utilising user data to target and build social programs while preserving citizen rights (hardy & maurushat, 2017; saldanha et al., 2022). according to experts, the controllability element entails developing web services that provide citizens control over procedures and norms based on the notion of interactivity (pina et al., 2007). together with liability, it came out as a linking factor. these are contextually placed as the second and third most essential elements, accounting for 34 percent of total da mechanisms. this is because they are inextricably linked with the systemic execution of accountability mechanisms and benchmarks stated in the policy, which covers the operationalisation question. these factors drive citizen users’ overall trust in the system of egovernance (antoni et al., 2018). these factors are significant because they determine how accountability is operationalised and enforced based on web service regulations. these elements are critical for developing a democratic setup’s capacity to provide accountability mechanisms, and they become much more crucial in an e-governance setting. the top levels of the da diagraph feature responsibility and responsiveness, which accounted for about 16 percent of the accountability components. they are visualised as dependent factors because, first and foremost, they depend on policies established by the system users. these policies fix the rules of citizen interaction and the responsibility for governance processes on officials in case of a service failure. also, these variables determine how interactivity is implemented in the form of a reaction, report, or response on the website. these aspects, however, are in no way trivial because they comprise the user engagement side of the entire accountability process (zimmermann, 2016). responsibility relates to a government office’s adherence to the rules established for its operation, derived from the transparency component, since its success will be judged only based on predefined standards. responsiveness is an outcome of the controllability factor because it encompasses officials’ and systems’ sensitivity to citizen users’ voices. it represents how successfully the citizen australasian journal of information systems sharma, s., kar, a. k., gupta, m. p. 2025, vol 29, research article digital accountability 16 feedback was employed in establishing and executing policy goals and handling complaints about instances of exclusion or service failures (fox, 2015). it is also how governments reassure citizens that their voices are being heard, that the administration is sympathetic to their concerns, and that the responsible officials will be held accountable if policy goals are unmet. they help conceptualise da as a social relationship between the government and its citizen users. 5.1 contribution to theory scholars have explored the role of effective institutions and accountability in delivering corruption-free services to the citizens (heinrich & brown, 2017). recent geopolitical situations concerning the covid-19 crisis have also reestablished the role of accountable government and citizen collaboration in dealing with public welfare issues through digital innovation (criado & guevara-gómez, 2021). in the same spirit, this study investigated the design factors of accountability to conceptualise da for egs delivery. it contributes to information systems and public administration research, allowing us to understand better how people perceive da. using mixed-method research allowed us to cover the three important dimensions of theory building: what, how, and why. the design factors of da cover the what part of its conceptualisation. they are the building blocks for ensuring accountability in egs delivery. the study also unveils the interrelationship between these design factors at various stages of da’s conceptualisation as a social relationship between governments and their subjects, covering the how and why parts. the bwm weights and hierarchical interrelationships between the factors describe how they relate to each other in ensuring accountability for egs delivery. the causation of these relationships can also be explained by the explanations drawn from the knowledge base and expert’s opinion. for developing nations like india, these design elements could be considered the building blocks leading up to the higher-level concept of da. by combining research literature from is and public administration, we have tried to map the different technical features to the conceptual dimensions of the da relationship enabled by the egs website. it becomes a very important theoretical contribution as it presents a framework for bringing back the possibility of building up trust online transparently and without human intervention prone to corruption. 5.2 practical implications the study also offers insights for practitioners who manage and run e-government websites to facilitate egs. these practitioners deal with daily issues relating to digital service platforms or websites. findings suggest that the factors of transparency, security & privacy, responsiveness, and responsibility are where they can develop and drive the feeling of openness and trust for the citizen users. for instance, the availability of a lucid and structured citizen charter outlining the terms of egs offered can enhance transparency and lay down the security policy for the citizen user. clear and publicly available documentation of the roles and responsibilities of different officials that established accountability would lead to better trust and transparency instead of the usual black-box view of administration (hooda et al., 2023). provisioning an open platform for people to interact with other service users and process owning official online can also create a space for catharsis, grievance sharing and establish openness in the egs platforms. australasian journal of information systems sharma, s., kar, a. k., gupta, m. p. 2025, vol 29, research article digital accountability 17 practitioners can use the pairwise comparison logic from the knowledge base to develop better policies by considering the causal relationships between the dimensions. practitioners can also develop rating indexes for platforms based on these factors to improve the efficiency of egs delivery. auditors working on reviewing and enhancing e-service platforms could also utilise the findings of this study to build quantitative methods to assess governance websites for their accountability towards citizen users. another view of the driving, linkage and dependent factors would be to look at them as strategic, tactical and operational level dimensions of policy making. by focusing on the strategic policy dimensions, we can define the long-term goals of the e-service. using the operational and tactical level policy objectives, policymakers can design and dictate the dayto-day operation of the platforms and websites so that the quality of egs delivery is maintained and accountability is ensured across all processes. the driving factors of security & privacy, along with transparency, deal with policy-level issues of service delivery and will show effects in the long run. the linkage or tactical dimension of controllability and liability helps operationalise these policy issues in the infrastructure of the web services and enforce them daily. the operational or dependent dimensions of responsiveness and responsibility will become the performance dimensions. they will be reflected in the citizen user’s experience while interacting with the web services platform or website. this type of evidencebased policymaking can ensure the implementation of policy objectives for delivering social welfare and development for all using egs initiatives. 5.3 future scope and limitations this section acknowledges certain limitations of this study. our findings may not be highly generalisable, particularly for developed countries, because all experts were from india. furthermore, expert opinion is vulnerable to vary in context and geographies studied. even though we conducted the literature review on multiple platforms, there will always be the possibility of missing out on some important piece of literature. we only covered the literature in english; some important factors or issues might be left out, which may be covered in any other publication language. the findings of this study would be applicable mostly to a democratic setup as the insights are drawn from the indian context and are contingent on the openness of the democratic setup in a country. future researchers can use our findings to expand the da constituents list to include other important factors that impact da for egs. scholars may also consider the possibility of including elements connected to the individuals who develop and operate such systems or web services, such as fairness, exclusion owing to designer prejudice, etc. scholars can also look at concerns like systemic discrimination, man-in-the-middle bias, and other factors impacting the design and implementation of egs platforms and websites. 6 conclusion we summarise our findings and observations to address the questions about conceptualising digitally enhanced accountability (da) by exploring design factors of e-government websites and their interrelationships. we conducted a literature study in the field of e-government, which resides at the confluence of public administration and information systems literature, to explore the essential factors to conceptualise accountability for egs. transparency, controllability, responsibility, responsiveness, liability and security & privacy were established as the constituent dimensions, and we validated this based on the expert’s opinion. australasian journal of information systems sharma, s., kar, a. k., gupta, m. p. 2025, vol 29, research article digital accountability 18 employing a mixed-method approach by combining the quantitative bwm technique with the qualitative tism-micmac technique. we were able to ascertain the relative contextual importance of these factors and explore their interrelationships in ensuring da. we classified these dimensions as dependents, drivers, and linkage factors when designing and implementing accountability measures on egs-delivering websites. the findings from this study highlight that the cornerstone of accountable government processes is transparency and establishing the liability of government officials at the policy level. developing confidence among citizens and encouraging the usage of e-services requires enforcing responsibility and responsiveness in website operations. controllability and liability act as linkages that facilitate the operationalisation of accountability mechanisms by incorporating citizen’s voices. references ackerman, j. m. (2005). social accountability in the public sector:a conceptual discussion (no. 35733; wbi working papers). aman, a., abdullah al-shbail, t., & mohammed, z. (2013). enhancing public organizations accountability through e-government systems. international journal of conceptions on management and social sciences, 1(1), 2357--2787. antoni, d., apriliani, herdiansyah, m. i., & akbar, m. (2018). critical factors of transparency and trust for evaluating e-government services for the poor. proceedings of the 2nd international conference on informatics and computing, icic 2017, 2018-janua, 1–6. doi.org/10.1109/iac.2017.8280612 as-saber, s., hossain, k., & srivastava, a. (2007). technology, society and e-government: in search of an eclectic framework. electronic government, 4(2), 156–178. doi.org/10.1504/eg.2007.013981 barrett, p. (2019). new development: reviews of public sector performance—groundhog day? public money and management, 39(7), 521--527. doi.org/10.1080/09540962.2019.1579441 bertot, j. c., jaeger, p. t., & grimes, j. m. (2012). promoting transparency and accountability through icts, social media, and collaborative e-government. transforming government: people, process and policy, 6(1), 78--91. doi.org/10.1108/17506161211214831 bindu, n., sankar, c. p., & kumar, k. s. (2019). from conventional governance to e-democracy: tracing the evolution of e-governance research trends using network analysis tools. government information quarterly, 36(3), 385–399. doi.org/10.1016/j.giq.2019.02.005 bjoern, n. (2005). epistemological perspectives on multi-method information systems research. proceedings of the 13th european conference on information systems, information systems in a rapidly changing economy, ecis 2005. bolívar, m. p. r., pérez, c. c., & hernández, a. m. l. (2007). the case of spanish regional governments. the american review of public administration, 37(2), 142–177. bovens, m. (2005). public accountability. in the oxford handbook of public management (pp. 182– 208). https://doi.org/10.1136/bmj.315.7116.1167 bovens, m. (2007). analysing and assessing accountability: a conceptual framework. european law journal, 13(4), 447–468. doi.org/10.1111/j.1468-0386.2007.00378.x australasian journal of information systems sharma, s., kar, a. k., gupta, m. p. 2025, vol 29, research article digital accountability 19 brandsma, g. j., & schillemans, t. (2013). the accountability cube: measuring accountability. journal of public administration research and theory, 23(4), 953--975. https://doi.org/10.1093/jopart/mus034 brinkerhoff, d. w., & wetterberg, a. (2016). gauging the effects of social accountability on services, governance, and citizen empowerment. public administration review, 76(2), 274–286. doi.org/10.1111/puar.12399 clarke, a. (2020). digital government units: what are they, and what do they mean for digital era public management renewal? international public management journal, 23(3), 358–379. doi.org/10.1080/10967494.2019.1686447 criado, j. i., & guevara-gómez, a. (2021). public sector, open innovation, and collaborative governance in lockdown times. a research of spanish cases during the covid-19 crisis. transforming government: people, process and policy, december. doi.org/10.1108/tg-08-20200242 dahiya, b. (2016). smart economy in smart african cities: sustainable, inclusive, resilient and prosperous. in advances in 21st century human settlements. drach-zahavy, a., leonenko, m., & srulovici, e. (2018). towards a measure of accountability in nursing: a three-stage validation study. journal of advanced nursing, 74(10), 2450–2464. doi.org/10.1111/jan.13735 dubnick, m. j., & frederickson, h. g. (2011). public accountability: performance measurement, the extended state, and the search for trust. national academy of public administration & the kettering foundation. dubnick, m. j., & justice, j. b. (2004). accounting for accountability. in annual meeting of the american political science association. doi.org/10.1111/puar.12220 dunleavy, p., margetts, h., bastow, s., & tinkler, j. (2006). new public management is dead long live digital-era governance. journal of public administration research and theory, 16(3), 467–494. doi.org/10.1093/jopart/mui057 eshiet, i. (2019). voice and accountability. international journal of political activism and engagement, 6(3), 10–22. doi.org/10.4018/ijpae.2019070102 fox, j. (2014).: social accountability: what does the evidence really say? in global partnership for social accountability. doi.org/10.5848/amacom.978-0evidence really social accountability 814410-93-6_5 fox, j. a. (2015). social accountability: what does the evidence really say? world development, 72, 346–361. https://doi.org/10.1016/j.worlddev.2015.03.011 greve, c. (2015). ideas in public management reform for the 2010s. digitalisation, value creation and involvement. public organization review, 15(1), 49–65. https://doi.org/10.1007/s11115-013-0253-8 gupta, h., & barua, m. k. (2016). identifying enablers of technological innovation for indian msmes using best-worst multi criteria decision making method. technological forecasting and social change, 107, 69–79. https://doi.org/10.1016/j.techfore.2016.03.028 australasian journal of information systems sharma, s., kar, a. k., gupta, m. p. 2025, vol 29, research article digital accountability 20 halachmi, a., & greiling, d. (2013). transparency, e-government, and accountability: some issues and considerations. public performance and management review, 36(4), 562–584. doi.org/10.2753/pmr1530-9576360404 hardy, k., & maurushat, a. (2017). opening up government data for big data analysis and public benefit. computer law and security review, 33(1), 30–37. doi.org/10.1016/j.clsr.2016.11.003 harrison, t. m., & sayogo, d. s. (2014). transparency, participation, and accountability practices in open government: a comparative study. government information quarterly, 31(4), 513–525. doi.org/10.1016/j.giq.2014.08.002 hasan, h., & linger, h. (2020). letting the public in: dialectic tensions when local governments move beyond e-government to e-democracy. australasian journal of information systems, 24, 1– 31. doi.org/10.3127/ajis.v24i0.1897 heinrich, f., & brown, a. j. (2017). measuring accountability performance and its relevance for anti-corruption: introducing a new integrity system-based measure. crime, law and social change, 68(3), 359–381. doi.org/10.1007/s10611-017-9712-4 hladchenko, l. (2016). government financial accountability and transparency in the digital world. ict in education, research and industrial applications, 477–484. hooda, a., gupta, p., jeyaraj, a., & dwivedi, y. (2023). clarifying the role of e-government trust in e-government success models: a meta-analytic structural equation modeling approach. australasian journal of information systems, 27, 1–22. doi.org/10.3127/ajis.v27i0.4079 jena, j., sidharth, s., thakur, l. s., kumar pathak, d., & pandey, v. c. (2017). total interpretive structural modeling (tism): approach and application. journal of advances in management research, 14(2), 162–181. doi.org/10.1108/jamr-10-2016-0087 kautz, k., bjerknes, g., fisher, j., & jensen, t. (2020). applying complex adaptive systems theory to understand distributed participatory design in contemporary, crowdsourced information systems development. australasian journal of information systems, 24(2006), 1–24. doi.org/10.3127/ajis.v24i0.2225 keymolen, e., prins, c., & raab, c. (2012). trust and ict: new challenges for public administration. innovation and the public sector, 19, 21–35. doi.org/10.3233/978-1-61499-137-321 koorn, r., voges, d., & van der knaap, p. (2010). survey of accountability , trust , consent , tracking , security and privacy. in european network and information security agency, 2010. http://www.enisa.europa.eu/act/it/library/deliverables/survey-pat koppell, j. g. s. (2005). pathologies of accountability: icann and the challenge of “multiple accountabilities disorder.” public administration review, 65(1), 94–108. doi.org/10.1111/j.15406210.2005.00434.x kumar, h., singh, m. k., & gupta, m. p. (2019). a policy framework for city eligibility analysis: tism and fuzzy micmac-weighted approach to select a city for smart city transformation in india. land use policy, 82(december 2018), 375–390. doi.org/10.1016/j.landusepol.2018.12.025 lee, r. l., & joseph, r. c. (2013). an examination of web disclosure and organisational transparency. computers in human behavior, 29(6), 2218–2224. doi.org/10.1016/j.chb.2013.05.017 australasian journal of information systems sharma, s., kar, a. k., gupta, m. p. 2025, vol 29, research article digital accountability 21 lee, t. (david), park, h., & lee, j. (2019). collaborative accountability for sustainable public health: a korean perspective on the effective use of ict-based health risk communication. government information quarterly, 36(2), 226–236. doi.org/10.1016/j.giq.2018.12.008 lindgren, i., madsen, c. ø., hofmann, s., & melin, u. (2019). close encounters of the digital kind: a research agenda for the digitalisation of public services. government information quarterly, 36(3), 427–436. https://doi.org/10.1016/j.giq.2019.03.002 lindquist, e. a., & huse, i. (2017). accountability and monitoring government in the digital era: promise, realism and research for digital-era governance. canadian public administration, 60(4), 627–656. doi.org/10.1111/capa.12243 mahmood, m. (2016). enhancing citizens’ trust and confdence in government through digital transformation. international journal of electronic government research, 12(1), 99–110. doi.org/10.4018/ijegr.2016010105 maréchal, n. (2015). ranking digital rights: human rights, the internet and the fifth estate. international journal of communication, 9(1), 3440–3449. margetts, h., & dunleavy, p. (2013). the second wave of digital-era governance: a quasiparadigm for government on the web. philosophical transactions of the royal society a: mathematical, physical and engineering sciences, 371(1987). doi.org/10.1098/rsta.2012.0382 matheus, r., & janssen, m. (2020). a systematic literature study to unravel transparency enabled by open government data: the window theory. public performance and management review, 43(3), 503–534. doi.org/10.1080/15309576.2019.1691025 mathivathanan, d., mathiyazhagan, k., rana, n. p., khorana, s., & dwivedi, y. k. (2021). barriers to the adoption of blockchain technology in business supply chains: a total interpretive structural modelling (tism) approach. international journal of production research, 59(11), 3338–3359. doi.org/10.1080/00207543.2020.1868597 meijer, a. (2007). digitisation and political accountability in the netherlands and the usa: convergence or reproduction of differences? proceedings of the european conference on egovernment, eceg, january 2007, 309–319. meijer, a. j. (2003). trust this document! icts, authentic records and accountability. archival science, 3(3), 275–290. doi.org/10.1007/s10502-004-1287-z mervyn, k., simon, a., & allen, d. k. (2014). digital inclusion and social inclusion: a tale of two cities. information communication and society, 17(9), 1086–1104. doi.org/10.1080/1369118x.2013.877952 mir, u. b., kar, a. k., dwivedi, y. k., gupta, m. p., & sharma, r. s. (2020). realising digital identity in government: prioritising design and implementation objectives for aadhaar in india. government information quarterly, 37(2), 101442. doi.org/10.1016/j.giq.2019.101442 mir, u. b., sharma, s., kar, a. k., & gupta, m. p. (2020). critical success factors for integrating artificial intelligence and robotics. digital policy, regulation and governance , 22(4), 307–331. doi.org/10.1108/dprg-03-2020-0032 mistry, j. j. (2012). the role of egovernance in mitigating corruption. accounting and the public interest, 12(1), 137–159. doi.org/10.2308/apin-10287 australasian journal of information systems sharma, s., kar, a. k., gupta, m. p. 2025, vol 29, research article digital accountability 22 misuraca, g., & viscusi, g. (2015). is open data enough? e-governance challenges for open government. standards and standardisation: concepts, methodologies, tools, and applications, 1132–1148. doi.org/10.4018/978-1-4666-8111-8.ch054 nasim, s. (2011). total interpretive structural modeling of continuity and change forces in egovernment. journal of enterprise transformation, 1(2), 147–168. doi.org/10.1080/19488289.2011.579229 oecd. (2020). the oecd digital government policy framework : six dimensions of a digital government. in oecd public governance policy papers (issue 2). omar, a. m. (2020). digital era governance and social media: the case of information department brunei. in employing recent technologies for improved digital governance (pp. 19–35). doi.org/10.4018/978-1-7998-1851-9.ch002 park, s., & gil-garcia, j. r. (2022). open data innovation: visualisations and process redesign as a way to bridge the transparency-accountability gap. government information quarterly, 39(1), 101456. doi.org/10.1016/j.giq.2020.101456 paul, s. (2007). a case study of e-governance initiatives in india. the international information & library review, 39(3–4), 176–184. doi.org/10.1016/j.iilr.2007.06.003 petrakaki, d. (2018). re-locating accountability through technology: from bureaucratic to electronic ways of governing public sector work. international journal of public sector management, 31(1), 31–45. doi.org/10.1108/ijpsm-02-2017-0043 pina, v., torres, l., & royo, s. (2007). are icts improving transparency and accountability in the eu regional and local governments? an empirical study. public administration, 85(2), 449– 472. doi.org/10.1111/j.1467-9299.2007.00654.x rathore, a. k., maurya, d., & srivastava, a. k. (2021). do policymakers use social media for policy design? a twitter analytics approach. australasian journal of information systems, 25, 1– 31. doi.org/10.3127/ajis.v25i0.2965 rezaei, j. (2015). best-worst multi-criteria decision-making method. omega (united kingdom), 53, 49–57.doi.org/10.1016/j.omega.2014.11.009 rezaei, j. (2016). best-worst multi-criteria decision-making method: some properties and a linear model. omega (united kingdom), 64, 126–130. doi.org/10.1016/j.omega.2015.12.001 roy, j. (2006). e-service delivery and new governance capacities: “service canada” as a case study. international journal of services, technology and management, 7(3), 253–271. doi.org/10.1504/ijstm.2006.010050 saldanha, d. m. f., dias, c. n., & guillaumon, s. (2022). transparency and accountability in digital public services: learning from the brazilian cases. government information quarterly, 39(2), 101680. doi.org/10.1016/j.giq.2022.101680 salimi, n., & rezaei, j. (2018). evaluating firms’ r&d performance using best worst method. evaluation and program planning, 66(september 2017), 147–155. doi.org/10.1016/j.evalprogplan.2017.10.002 sharma, s., kar, a. k., & gupta, m. p. (2024). untangling the web between digital citizen empowerment, accountability and quality of participation experience for e-government: australasian journal of information systems sharma, s., kar, a. k., gupta, m. p. 2025, vol 29, research article digital accountability 23 lessons from india. government information quarterly, 41(3), 101964. doi.org/10.1016/j.giq.2024.101964 sharma, s., kar, a. k., gupta, m. p., dwivedi, y. k., & janssen, m. (2022). digital citizen empowerment : a systematic literature review of theories and development models. information technology for development, 1–28. doi.org/10.1080/02681102.2022.2046533 sharma, s., mir, u. b., kar, a. k., & gupta, m. p. (2022). a case of affordances: collaborative governance using smartphones. international journal of electronic government research, 18(1), 1–21. doi.org/10.4018/ijegr.301256 shibin, k. t., gunasekaran, a., & dubey, r. (2017). explaining sustainable supply chain performance using a total interpretive structural modeling approach. sustainable production and consumption, 12(june), 104–118. doi.org/10.1016/j.spc.2017.06.003 shukla, m., & mattar, l. (2019). next generation smart sustainable auditing systems using big data analytics: understanding the interaction of critical barriers. computers and industrial engineering, 128 (may 2018), 1015–1026. doi.org/10.1016/j.cie.2018.04.055 sørensen, e. (2012). measuring the accountability of collaborative innovation. innovation journal, 17(1), 1–18. stamati, t., papadopoulos, t., & anagnostopoulos, d. (2015). social media for openness and accountability in the public sector: cases in the greek context. government information quarterly, 32(1), 12–29. doi.org/10.1016/j.giq.2014.11.004 sullivan, k., & clarke, j. (2010). balancing security and privacy in egovernment services. 2010 ist-africa, 1–7. torabi, s. a., giahi, r., & sahebjamnia, n. (2016). an enhanced risk assessment framework for business continuity management systems. safety science, 89, 201–218. doi.org/10.1016/j.ssci.2016.06.015 twizeyimana, j. d., & andersson, a. (2019). the public value of e-government – a literature review. government information quarterly, 36(2), 167–178. doi.org/10.1016/j.giq.2019.01.001 united nations. (2022). united nations development programme: digital strategy 2022-2025. doi.org/10.1016/b978-0-444-86236-5.50086-9 van de kaa, g., janssen, m., & rezaei, j. (2018). standards battles for business-to-government data exchange: identifying success factors for standard dominance using the best worst method. technological forecasting and social change, 137(july), 182–189. doi.org/10.1016/j.techfore.2018.07.041 van deursen, a. j. a. m., helsper, e. j., eynon, r., & van dijk, j. a. g. m. (2017). the compoundness and sequentiality of digital inequality. international journal of communication, 11(march), 452–473. verma, a., giri, a. k., & debata, b. (2022). leapfrogging into knowledge economy: information and communication technology for human development. australasian journal of information systems, 26, 1–22. doi.org/10.3127/ajis.v26i0.3883 yeravdekar, s., & behl, a. (2017). benchmarking model for management education in india: a total interpretive structural modeling approach. benchmarking, 24(3), 666–693. doi.org/10.1108/bij-06-2016-0082 australasian journal of information systems sharma, s., kar, a. k., gupta, m. p. 2025, vol 29, research article digital accountability 24 zimmermann, c. (2016). framework and requirements for reconciling digital services and privacy. 24th european conference on information systems, ecis 2016. appendix a.1 expert panel details s.n o expert sector experience (years) gender domain expertise panel 1: experts with 5-20 years of experience 1 government advisor public 17 male it policy 2 senior scientist, web technologies public 17 male e-governance 3 senior scientist, web technologies public 16 male e-governance 4 professor public 15 male public administration 5 professor public 14 female it policy 6 senior it consultant private 12 male e-governance 7 senior it consultant public 10 male e-governance 8 associate professor public 10 male information systems area 9 assistant vice president, product based it company private 9 female web services 10 associate director, research think tank private 9 male information systems area 11 associate professor public 8 female information systems area 12 website and new media manager private 7 male web services 13 product manager, product based it company private 7 male web applications 14 research fellow at think tank public 7 female e-governance panel 2: senior experts with more than 20 years experience 1 director genereal of government it organisation public 35 female e-governance and ict policy 2 professor public 30 female public administration and it policy 3 civil servant and ceo of government it organisation public 27 male e-governance and public administration 4 president of think tank and government advisor private 27 male ict and economic policy 5 senior consultant with world bank private 25 male e-governance and ict policy 6 professor public 25 male information systems area 7 supreme court lawyer and technology counsultant private 21 male it law and policy table a.1: expert panel details a.2: best-worst method inequalities we use the 2 vectors developed in table 2 and 3 to develop min-max equations for all criteria which could be written as solvable inequalities. w1-2w2≤ ξ, australasian journal of information systems sharma, s., kar, a. k., gupta, m. p. 2025, vol 29, research article digital accountability 25 w1-3w3≤ ξ, w1-4w4≤ ξ, w1-5w5≤ ξ, w1-4w6≤ ξ, w1-5w5≤ ξ, w2-3w5≤ ξ, w3-w5≤ ξ, w4-2w5≤ ξ, w6-3w5≤ ξ, for all wj such that:w1+w2+w3+w4+w5+w6=1 and, w1,w2,w3,w4,w5,w6≥0 a.3: step-wise tism analysis this sub-section outlines the analysis done using the tism-micmac method step i: using literature review and experts’ opinion, create a list of factors constituting da (see table 1). step ii: develop the tism reachability matrix (rm) in consultation with experts to establish different possible relationships between the factors we develop the rm by taking expert’s input in a pairwise comparison matrix, where each possible relationship is coded as a 1 or 0 if there is no relation. it is a mathematical representation of the interrelationships between the factors. this table is an n*n matrix describing the relationships between the different factors under analysis. each cell with a 1 represents a possibility of a relationship. the ones with an asterisk (1*) denote significant transitive links for the context under examination. transitivity is checked with a simple rule in the matrix that if dimension a will influence b and b will influence c, then by virtue of transitivity a should be influencing c. however, in tism, we only retain those transitive links that are significant and explained by the experts in the chosen context. step iii: conduct pairwise analysis of the interactions between these elements to create a knowledgebase that includes causal explanations from experts and even transitive relationship that are contextually significant (table a.4). the knowledgebase is a representation of the panel’s knowledge of and about the interrelationships between the factors being analysed. each relationship in the context is analysed for two things: first, whether any relationship is possible between 2 factors or not, and second, if possible, what are the causal explanations for the same? table 5 presents this collection of knowledge that will become the foundation for further analysis. tism uses a logic behind the relationships to explain the causality. we have chosen to work with the rationale of ‘dimension a will influence dimension b’ with influence in ensuring da as the logic of comparison. australasian journal of information systems sharma, s., kar, a. k., gupta, m. p. 2025, vol 29, research article digital accountability 26 factors comparison statement possible relation causal interpretation for relationships f1-transparency 1 f1–f2 transparency will influence controllability yes transitive: using public data and liability systems, citizens can assess and influence service provider performance. 2 f2–f1 controllability will influence transparency no 3 f1–f3 transparency will influence responsibility yes because of publicly available terms and conditions of the services, there will be demand to perform and meet the established kpis. 4 f3–f1 responsibility will influence transparency no 5 f1–f4 transparency will influence responsiveness yes transitive: an open system will encourage participation and provide chances for citizens to provide recommendations and feedback. 6 f4–f1 responsiveness will influence transparency no 7 f1–f5 transparency will influence liability yes each procedure will have a legally and structurally accountable representative in a transparent system. 8 f5–f1 liability will influence transparency no 9 f1–f6 transparency will influence security and privacy yes open systems has well defined guidelines for storing and sharing user data in line with the services furnished. 10 f6–f1 security and privacy will influence transparency yes security and privacy trade-offs would limit the amount of openness that could be achieved without jeopardising data protection and access restrictions. f2-controllability 11 f2–f3 controllability will influence responsibility yes collaborating with public to build and coproduce services will increase officials’ sense of accountability. 12 f3–f2 responsibility will influence controllability no 13 f2–f4 controllability will influence responsiveness yes people would feel more heard and involved when governments promote citizen collaboration. 14 f4–f2 responsiveness will influence controllability no 15 f2–f5 controllability will influence liability no 16 f5–f2 liability will influence controllability yes we may empower citizens to regulate officials’ conduct by identifying them as process owners over platforms and websites. 17 f2–f6 controllability will influence security and privacy no 18 f6–f2 security and privacy will influence controllability yes transitive: citizens’ expectations of privacy and security would shape service rules and authorities’ roles. f3-responsibility australasian journal of information systems sharma, s., kar, a. k., gupta, m. p. 2025, vol 29, research article digital accountability 27 19 f3–f4 responsibility will influence responsiveness yes citizens will always feel connected and valued by the system if authorities are upholding the promised standards of service policy. 20 f4–f3 responsiveness will influence responsibility yes when people’ views are heard and administration is accommodating, the sense of responsibility for social good grows. 21 f3–f5 responsibility will influence liability no 22 f5–f3 liability will influence responsibility yes fixing officials’ obligation for specified duties and processes would help users and the ombudsman to enforce responsibility. 23 f3–f6 responsibility will influence security and privacy no 24 f6–f3 security and privacy will influence responsibility no transitive: individual officials’ responsibilities would be fixed by the security policy, which defines the levels of access in the systems. f4-responsiveness 25 f4–f5 responsiveness will influence liability no 26 f5–f4 liability will influence responsiveness yes designating authorities liable for service outcomes and making their performance information public helps ensure that they become receptive to people’s feedback. 27 f4–f6 responsiveness will influence security and privacy no 28 f6–f4 security and privacy will influence responsiveness no f5-liability 29 f5-f6 liability will influence security and privacy no 30 f6-f5 security and privacy will influence liability yes policymakers determine who is responsible for safeguarding citizen data in relation to a service depending on the policy that has been adopted. table a.2: tism knowledge base step iv: use the tism partitioning procedure to ascertains levels for each dimension in the diagraph. this is an iterative process of assigning different levels to the factors we are trying to analyse. we start by putting down all the factors in table a.2 with their reachability and antecedent sets. reachability set for a factor is defined by all the cells in their row which have a 1 (with of without the asterisk) in the fully transitive reachability matrix. antecedent set is the collection of all the cells that have a 1 (with of without the asterisk) in their column of the fully transitive reachability matrix. the intersection set is simply the intersection of the reachability and antecedent sets. in any iteration of the partitioning process, we assign a level to any factor if their reachability and intersections sets are same. and, in the next iteration that number is removed from the complete table. for instance, in the first iteration of the table a.2, we assign f3 and f4 level 1 and in the next iteration the numbers 3 & 4 are removed from the table completely. we then assign f2 the next level as per the first rule and continue the process until all factors are australasian journal of information systems sharma, s., kar, a. k., gupta, m. p. 2025, vol 29, research article digital accountability 28 assigned a level. the final matrix whit assigned levels to all factors is called the canonical matrix. iteration-1 factors reachability set antecedent set intersection set level f1 {1,2,3,4,5,6} {1,6} {1,6} f2 {2,3,4} {1,2,5,6} {2} f3 {3,4} {1,2,3,4,5,6} {3,4} 1 f4 {3,4} {1,2,3,4,5,6} {3,4} 1 f5 {2,3,4,5} {1,5,6} {5} f6 {1,2,3,4,5,6} {1,6} {1,6} iteration-2 factors reachability set antecedent set intersection set level f1 {1,2,5,6} {1,6} {1,6} f2 {2} {1,2,5,6} {2} 2 f5 {2,5} {1,5,6} {5} f6 {1,2,5,6} {1,6} {1,6} iteration-3 factors reachability set antecedent set intersection set level f1 {1,5,6} {1,6} {1,6} f5 {5} {1,5,6} {5} 3 f6 {1,5,6} {1,5,6} {1,6} iteration-4 factors reachability set antecedent set intersection set level f1 {1,6} {1,6} {1,6} 4 f6 {1,6} {1,6} {1,6} 4 table a.3: level partitioning for tism. step v: create a final tism diagraph based on the allocated levels, with interpretative explanations on the connecting arrows (figure 3). the factors are put in bubbles at different hierarchical levels, connected by arrows. solid arrows are used for direct relations while dotted ones are used for transitive relationships. explanations from the knowledge base are then put on this arrows to give a structured view of the interpreted concept under study. copyright copyright © 2025 sharma, s., kar, a. k., gupta, m. p. this is an open-access article licensed under a creative commons attribution-noncommercial 4.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi.org/10.3127/ajis.v29.5175 https://creativecommons.org/licenses/by-nc/4.0/deed.en https://creativecommons.org/licenses/by-nc/4.0/deed.en paper title (paper title style) australasian journal of information systems oshri and kotlarsky 2025, vol 29, provocation growth of start-up ecosystems accepted 12 aug 2025, after 1 round of review with michael davern as accepting editor in chief. 1 with a little help from my friends? the growth navigation challenge of start-up ecosystems in australasia ilan oshri* the university of auckland 12 grafton road auckland 1010, new zealand email: ilan.oshri@auckland.ac.nz julia kotlarsky the university of auckland 12 grafton road auckland 1010, new zealand abstract the start-up ecosystem in australasia, encompassing australia and new zealand, has demonstrated significant potential for innovation and growth. however, start-ups in this region face substantial challenges in achieving sustained growth, including limited access to capital, insufficient market knowledge, and the complexities of navigating regulatory environments. this article explores these unique challenges and examines how information technology (it) service providers have emerged as central players in the start-up ecosystem, offering strategic consulting services that encompass growth and expansion. by analysing the case of tata consultancy services' (tcs) business as a service (baas), the article investigates the dynamics of power and the strategies utilized by both start-ups and service providers to sustain beneficial positions within these partnerships. utilizing a multi-theoretical lens, the article examines how strategic partnerships between start-ups and it service providers can foster a thriving entrepreneurial ecosystem in australasia. the findings emphasize the importance of these partnerships in creating mutual dependencies that enable growth and expansion. however, it also highlights the potential risks associated with such dependencies, urging start-ups to carefully weigh the benefits and challenges of engaging with it service providers for strategic consulting services. this controversy accentuates the critical debate on whether start-ups should embrace these partnerships or avoid them due to the inherent risks, turning it into a key decision point for their strategic decisions. keywords: start-up, ecosystem, it service provider, innovation. 1 introduction the start-up ecosystem in australasia, encompassing australia and new zealand, has demonstrated significant potential for innovation and growth. with the australian start-up ecosystem valued at over us$220 billion and new zealand's at us$9 billion in 2024, the region is a burgeoning hub for new ventures, particularly in sectors such as fintech, health tech, edtech, and sustainability. despite this promising landscape, start-ups in australasia face substantial challenges in scaling their operations1. these challenges include limited access to 1 https://granton.io/top-10-challenges-startup-founder-overcome/ mailto:ilan.oshri@auckland.ac.nz australasian journal of information systems oshri and kotlarsky 2025, vol 29 provocation growth of start-up ecosystems 2 capital, insufficient market knowledge, and the complexities of navigating regulatory environments of different countries in the region. one critical issue is the geographical isolation of australasia, which hampers the ability of start-ups to attract international investors and expand into global markets. additionally, many start-up founders possess strong technical skills but lack the strategic business acumen necessary for growth and expansion. this gap often leads to inefficient operations and missed opportunities. the competitive funding landscape further exacerbates these challenges, as venture capital is often skewed towards more established enterprises. in response to these obstacles, information technology (it) service providers have emerged as promising players in the start-up ecosystem, offering strategic consulting services that encompass growth and expansions2. this article explores the dynamics of such partnerships, focusing on the role of it service providers in supporting start-ups. by examining the case of tata consultancy services' (tcs) business as a service (baas), we analyze the power relations and strategies employed by both start-ups and service providers to maintain advantageous positions in these relationships. through a multi-theoretical lens (resource-based view of the firm, transaction costs economics, agency theory, and social exchange theory), which are central to understanding the sourcing relationships between a service provider and a client firm (kotlarsky at al. 2020), we investigate how strategic partnerships between start-ups and it consultancy firms can foster a thriving entrepreneurial ecosystem in australasia. the article aims to provoke a discussion on whether start-ups should engage with it service providers for strategic consulting services, considering the potential benefits and risks involved. 2 the start-up ecosystem in australasia the start-up scene in australasia is vibrant and diverse, with a strong emphasis on technology and innovation. according to multiple reports, the value of the australian start-up ecosystem exceeds us$220 billion3, with sydney's start-up ecosystem alone valued at us$72 billion in 2024, hosting over 3,000 start-ups4. new zealand's start-up ecosystem was valued at us$9 billion in 2024, with around 2,400 start-ups across the country5. the australian start-up sector is characterized by its potential for rapid growth and external funding, although it remains sensitive to economic shifts and investment uncertainties. australia ranks among the leading countries for start-ups worldwide, with venture capital investments significantly supporting the expansion of new businesses. in recent years, australian start-ups have received about us$4 billion annually in venture capital funding, a notable increase compared to previous years. growth in several sectors, such as fintech, health tech, ed-tech, and sustainability/cleantech, has driven this increase in investment. 2 https://pareekh.com/unicorn-rd-how-engineering-service-providers-can-help-themselves-by-makingstartups-and-unicorns-successful/ 3 https://startupgenome.com/article/oceania-insights-rankings-and-ecosystem-pages-1 4https://journals.sagepub.com/doi/10.1177/09717218231201878#:~:text=this%20explains%20why%2c% 20during%20the,and%20halving%20emissions%20by%202030. 5 https://www.auckland.ac.nz/en/news/2025/01/15/8-trends-will-shape-the-startup-sector-in2025.html#:~:text=new%20zealand's%20ecosystem%20is%20valued%20at%20%249%20billion%20acro ss%202%2c400%20startups. australasian journal of information systems oshri and kotlarsky 2025, vol 29 provocation growth of start-up ecosystems 3 technology-based start-ups account for nearly half of all new businesses in australia, reflecting the country's focus on innovation and digital transformation. the australian government supports start-ups through grants and incentives such as the r&d tax incentive and the entrepreneurs’ programme. new zealand's start-up ecosystem, while smaller in scale, is equally dynamic. auckland, as a major tech hub, encompasses 58% of all start-up activity in new zealand. venture capital investment in new zealand start-ups amounted to us$1.2 billion in 2024, with the vast majority of the funding going into fintech, health tech, and sustainability ventures. similar to australia, the new zealand government has been proactive in supporting start-ups through various initiatives and funding programs. while the potential for growth and impact on the local economies is high, start-ups in both countries are still sensitive to economic shifts and investment uncertainties, predominantly due to global slowdowns in investment activity. the current focus of the start-up ecosystems in both countries on artificial intelligence (ai), sustainability, health tech, fintech, and ecommerce is particularly promising, but not without the challenges that most start-ups face when pursuing growth. 3 growth challenges for start-ups: the case of the australasian ecosystem the growth challenges that the australasian start-up ecosystem faces are similar to those encountered by start-ups around the world, namely insufficient capital, resources, and knowledge needed to support market entry and expansion for business solutions anchored in emerging technologies. being remotely located from major international markets, such as western europe, the usa, and southeast asia, has amplified these challenges. for example, start-ups in new zealand and australia have been reported to lack the knowledge needed to understand and navigate regulatory environments, cultural nuances, and market dynamics in international markets, simply because of their limited experience in these markets and the limited services available in these countries to support such expansion6,7,8. similarly, efforts to commercialize new products and services by start-ups in australasia come with greater risk, as the small domestic market limits the opportunities to experiment and learn from the domestic experience before expanding to international markets. consequently, start-ups in australasia need guidance and support in navigating these risks. while research has already pointed out the relevance of consulting services of firms specializing in the start-up growth segment9, this article examines a recent phenomenon in which an it service provider has assumed the role of providing support for start-ups. 4 navigating growth through partnership: a multi-theoretical perspective the idea that firms partner to access knowledge and capabilities not available in-house is not new (e.g., feeny and willcocks 1998, dibbern et al. 2004, fisher et al. 2008, oshri et al. 2015). 6 https://www.mbie.govt.nz/business-and-employment/economic-growth/previous-economicdevelopment-work/startup-advisors-council/assessment-of-the-new-zealand-ecosystem 7 https://www.airwallex.com/au/blog/australian-startup-statistics 8 https://techcouncil.com.au/wp-content/uploads/2023/07/shots-on-goal-vf.pdf 9 https://hbr.org/2019/01/the-two-ways-for-startups-and-corporations-to-partner australasian journal of information systems oshri and kotlarsky 2025, vol 29 provocation growth of start-up ecosystems 4 the information systems (is) literature has examined both the benefits and risks associated with such partnerships from multiple theoretical perspectives (kotlarsky et al. 2020, dibbern et al. 2004). predominantly, these partnerships can be rooted in sourcing relationships where the client firm contracts the service provider to deliver services in exchange for a fee (oshri et al. 2023). such sourcing relationships bring about multiple concerns. when examined from transaction costs economics (tce), such an exchange raises concerns about the costs associated with contracting and hidden costs (karimi-alaghehband et al. 2011, lioliou and zimmerman 2015, dibbern et al. 2008). for example, start-ups are particularly likely to struggle to allocate resources to monitor the service provider, thus elevating the level of uncertainty in such contracting and possibly creating conditions for opportunism. studies that examined such relationships from agency theory have warned that information asymmetries between the service provider and the client are likely to promote opportunistic behaviour (bahli and rivard 2003, dawson et al. 2010). an alternative view is anchored in social exchange theory (set), in which the idea of contracting a service provider materializes into strategic alliances through which the client firm taps into strategic capabilities not available in-house (lioliou et al. 2014). yet, while a set perspective portrays a positive outcome for client firms (whether small or large), any client firm will still be concerned and possibly negatively affected by the risks associated with agency behaviour by the provider and the costs involved in contracting and monitoring. similarly, from a resource-based view, contracting service providers to support growth allows start-ups to focus on their core capabilities, such as research and development (r&d), but not without the risks of losing control over the business functions that the service provider is supporting (e.g., commercialization) or failing to execute an effective knowledge transfer, resulting in a dependency on the service provider for non-core activities (grimpe and kaiser 2010). start-ups, in particular, might face challenges of monitoring and contracting as their resources are limited. mitigating these risks by developing strong relationships with service providers is a viable option; however, as startups’ spending on third parties is relatively low, it is unlikely that service providers will equally invest in relational governance. while the is outsourcing literature has extensively revisited aspects concerning partnerships between service providers and client firms (kotlarsky et al. 2023, oshri et al., 2018, aubert et al., 2015), there has been little attention to how the above concerns are perceived in the context of start-ups, particularly with regards to the challenges that start-ups face, namely growth and commercialization. we now turn to a case in point in which we examine the journey of one service provider, tata consultancy services (tcs), with their consultancy service coined business as a service, aimed at helping start-ups in australasia cope with growth and commercialization challenges. 5 business as a service for australasian start-ups: insights from tata consultancy services business-as-a-service (baas) is a new concept in the service sourcing sector, understood as the outsourcing of management and strategic functions to a service provider, such as marketing and business development. several it service providers have been promoting baas as their engagements with client firms for technology and operations services have allowed them to gain valuable knowledge of strategic aspects concerning their client firms as well as develop in-house strategic capabilities that could further benefit their client base. australasian journal of information systems oshri and kotlarsky 2025, vol 29 provocation growth of start-up ecosystems 5 tcs's interest in start-ups and the baas concept in australasia started in 2021. realizing that other tech giants such as google and microsoft have been engaging with start-ups, tcs sought to enter this segment of the market and explore opportunities, particularly in countries where there is growth in the number of start-ups, but domestic conditions might challenge their expansion strategy, such as australia. however, extending tcs services from their current focus on large global enterprises with a specialty in delivering tech-enabled solutions to strategy consulting for start-ups did not come without challenges. for one, tcs staff assigned to develop baas offerings for start-ups faced a major challenge in understanding how to work with small clients that operate with limited budgets. furthermore, the baas solutions for startups were remote from the services tcs has so far specialized in delivering to its client firms, thus requiring the development of new capabilities and specialties. to mitigate risks arising from engaging with start-ups, tcs developed a client selection and contracting approach that included the following criteria: market research and selection: tcs conducts thorough market research to identify countries and companies that align with their ethical values and business goals. this involves evaluating the maturity of the start-up ecosystem, the level of support from venture capitalists and governments, and the overall market potential. for example, australia and new zealand were shortlisted for the pilot phase due to their supportive start-up ecosystems. ethical and value alignment: tcs ensures that the companies they partner with share similar ethical values. they avoid working with companies involved in industries such as liquor, cigarettes, and gaming. while this aspect helped maintain the integrity and reputation of the tata group, it was also relevant for the assessment of the financial security of the start-up and their ability to raise additional funds if needed. funding and support: the level of funding and support that start-ups receive from venture capitalists and governments is a crucial factor in the sourcing process. tcs looks for companies that have strong backing and are well-supported, which can enhance their chances of success and growth. geographical focus: tcs had a dedicated team for different regions, such as australia, new zealand, and dubai. this regional focus allows them to tailor their sourcing strategies to the specific needs and opportunities in each market. hybrid workforce: tcs employs a hybrid workforce model, with 80% of the team based in india or the philippines (nearshore locations) and 20% onsite. this model helps optimize costs while ensuring that there is sufficient local presence to support the development and execution of projects. revenue-share model: instead of taking equity stakes, tcs used a revenue-share model. this approach allowed tcs to share in the success of the scale-ups without the risks associated with equity investments. a case in point is tcs's engagement with downsizer, an australian-based start-up. downsizer.com was founded by nsw-based mark macduffie and phil verdonin 2003. the company was originally known as seniors housing online. the mission of the start-up is to offer advice and options to retirees about downsizing their accommodations. as the start-up established its presence in australia, it sought to expand to other markets such as the uk and australasian journal of information systems oshri and kotlarsky 2025, vol 29 provocation growth of start-up ecosystems 6 the usa. however, the owners lacked knowledge of these markets and therefore struggled to expand their presence outside australia. selecting downsizer for its baas services followed the above selection criteria. downsizer was in a growth market, and its solutions were aligned with tcs’s values and ethics. tcs's engagement with downsizer spanned over a year and involved three key areas: research, business advisory, and execution. initially, tcs conducted comprehensive market research to determine the feasibility of downsizer's expansion into the uk and us markets. this research included analysing property market regulations, financial ecosystems, and demographic trends, particularly focusing on states like california and florida in the us. based on this research, tcs provided strategic business advisory services, helping downsizer identify the best developers and financial institutions to partner with. tcs's advisory services ensured that downsizer could effectively integrate its it platform with these partners, optimizing property listings and transactions. additionally, tcs supported downsizer in maximizing its profit margins by leveraging data analytics to identify high-traffic property markets and potential growth areas. this holistic approach enabled downsizer to make informed decisions, streamline its operations, and successfully penetrate new markets, thereby solidifying its position in the property-tech industry. from the provider's perspective, risk mitigation followed a careful selection process of startups as well as the deployment of well-tested practices from its traditional outsourcing practice. for example, tcs harnessed its extensive network of collaborators in the usa to assist downsizer in accessing knowledge, technology, and contacts needed to penetrate this market. from a governance perspective, tcs deployed strict cost-to-income ratio frameworks to clearly communicate outcomes achieved and link them to the key objectives of the engagement. as such, tcs actively sought to mitigate risks associated with engagements with start-ups, by both increasing the likelihood of achieving the engagement objectives by carefully selecting candidate start-ups for baas engagement and by deploying well-proven practices in entrepreneurial contexts. 6 navigating growth for start-ups through partnerships with it service providers: the right path? in the competitive and dynamic landscape of start-ups, the prospect of partnering with it service providers for strategic consulting is highly appealing. these partnerships promise access to expertise, resources, and networks that can facilitate market entry and drive growth. however, while the advantages are compelling, it is essential to critically examine these relationships by assuming risks associated with contracting. indeed, start-ups often lack the in-house capabilities required for strategic expansion. it service providers, with their extensive experience and specialized knowledge, can fill these gaps, enabling start-ups to concentrate on their core competencies such as innovation and product development. this symbiotic relationship can lead to enhanced competitive advantage and accelerated growth. however, this dependency on external expertise is not without risks. there is the danger of losing control over critical business functions and the risk of becoming overly dependent on these external entities. this dependency can stifle growth in certain strategic areas of the business, impact much-needed internal development around strategic thinking, and could affect long-term decision-making on the direction of the firm. as knowledge transfer from the australasian journal of information systems oshri and kotlarsky 2025, vol 29 provocation growth of start-up ecosystems 7 service provider to the start-up is not always seamless, and as internal resources are scarce to internally develop strategic capabilities, the reliance on the provider for ongoing support and guidance in strategic areas is only going to grow over time. furthermore, there can be a misalignment of incentives as contracting commercial models promote the revenue-share model emphasizing the provider’s vested interest in the start-up's revenue streams, thus directing strategic advice towards financial gains rather than the start-up's long-term objectives. it is within this contested terrain that start-ups and service providers seek to find a balanced approach to partnerships. clearly, start-ups must carefully weigh these factors and consider developing robust internal capabilities alongside external partnerships. by doing so, they can navigate the growth journey more effectively, ensuring that their reliance on service providers enhances rather than hinders their strategic objectives. the question remains: are these partnerships the golden ticket to growth, or a trade-off with hidden costs that could ultimately undermine the very foundations of the start-up's success? 7 provocative directions for future research and practice this article has surfaced a provocative tension at the heart of start-up growth in australasia: the appeal of strategic partnerships with it service providers versus the risks of dependency, misaligned incentives, and strategic drift. future research and practice must not only explore these dynamics but also confront the uncomfortable questions they raise. 7.1 provocations for future research should start-ups outsource strategy? a fundamental provocation is whether outsourcing strategic functions—once considered the core of entrepreneurial identity—is compatible with sustainable start-up growth. future research should examine whether such outsourcing dilutes the strategic dna of start-ups or stimulates their evolution. are it service providers the new gatekeepers of innovation? as providers like tcs expand their influence, researchers must critically examine whether they are becoming gatekeepers of innovation, shaping the trajectory of start-ups through selective engagement and revenueshare models. who truly benefits from these partnerships in short and long terms—and at what cost? future research could investigate the relationship between start-ups and technology providers through the lens of value (co-)creation, examining how each party perceives and pursues value, and how it is changing over time as the relationship evolves. can start-ups retain strategic autonomy in baas models? the rise of baas challenges the assumption that start-ups are agile and autonomous. future studies should explore whether these models foster strategic independence or establish a new form of outsourcing dependency that is harder to escape (i.e., the lock-in effect) . what happens when strategic advice is commercially biased? the revenue-share model introduces a provocative dilemma: can strategic advice remain impartial when providers have a financial stake in outcomes? research should explore how such models influence decisionmaking, innovation priorities, and long-term vision. is the australasian start-up ecosystem ready for hybrid governance? the australasian start-up ecosystem may need to rethink governance. can hybrid models—combining relational trust with formal controls—be realistically implemented by resource-constrained australasian journal of information systems oshri and kotlarsky 2025, vol 29 provocation growth of start-up ecosystems 8 start-ups? or are we expecting too much from too little? future studies could examine the tensions associated with resource-constraints and governance approaches that start-ups employ to deal with these tensions. 7.2 provocations for practice rethink the role of founders: founders must confront the uncomfortable truth: technical brilliance alone is insufficient. they must evolve into strategic leaders who are capable of executing (in-house) strategic development and implementation of the end-to-end growth strategy. the alternative is a lack of critical capability that resides in markets (i.e. external advisors). ecosystem builders and policymakers should provoke a shift in focus—from operational support to strategic capability development. this means investing in entrepreneurial skills and founder education, strategic mentoring, and tools that help startups think long-term. expose the hidden costs of partnership: start-ups must be equipped to critically assess the hidden costs of partnership—loss of control, misaligned incentives, and strategic drift. practical tools and frameworks should provoke deeper due diligence before engagement. the ecosystem needs forums for honest, critical dialogue about the risks and rewards of partnering with it service providers. these spaces should provoke reflection, share failures, and challenge dominant narratives of growth. 8 concluding remarks this article has explored the evolving role of it service providers in supporting start-up growth within the australasian ecosystem, highlighting both the opportunities and the complexities these partnerships entail. strategic collaborations, such as those enabled through business-as-a-service models, offer start-ups access to critical expertise, networks, and resources that can accelerate their journey toward commercialization and international expansion. however, these benefits must be weighed against the potential risks—such as dependency, misaligned incentives, and challenges in maintaining strategic autonomy. rather than dismissing such partnerships, this article encourages a thoughtful and informed approach: one that embraces the potential of external support while fostering internal strategic capabilities and governance mechanisms that safeguard long-term growth. if start-ups fail to critically assess these relationships, they risk outsourcing not just their strategy—but their future. the real provocation is this: can start-ups afford to grow with help, or must they learn to grow without it? references aubert, b. a., kishore, r., and iriyama, a. (2015). exploring and managing the “innovation through outsourcing” paradox. journal of strategic information systems, 24(4), 255-269. doi.org/10.1016/j.jsis.2015.10.003 bahli b, rivard s (2003) the information technology outsourcing risk: a transaction cost and agency theory-based perspective. journal of information technology 18(3):211–221. dawson, g. s., watson, r. t., & boudreau, m. c. (2010). information asymmetry in information systems consulting: toward a theory of relationship constraints. journal of management information systems 27(3):143–178. australasian journal of information systems oshri and kotlarsky 2025, vol 29 provocation growth of start-up ecosystems 9 dibbern, j., goles, t., hirschheim, r., & jayatilaka, b. (2004). information systems outsourcing: a survey and analysis of the literature. acm sigmis database: the database for advances in information systems, 35(4), 6-102. dibbern, j., winkler, j., and heinzl, a. (2008). explaining variations in client extra costs between software projects offshored to india. mis quarterly, 32(2), 333–366. doi.org/10.2307/25148843 feeny, d. f., & willcocks, l. p. (1998). core is capabilities for exploiting information technology. sloan management review, 39(3), 9-21. fisher, j., hirschheim, r. & jacobs, r. (2008). understanding the outsourcing learning curve: a longitudinal analysis of a large australian company. information systems frontiers 10, 165–178. doi.org/10.1007/s10796-008-9070-y grimpe, c., and kaiser, u. (2010). balancing internal and external knowledge acquisition: the gains and pains from r&d outsourcing, journal of management studies 47(8), 14831509. lioliou, e., and zimmermann, a. (2015). vendor opportunism in it outsourcing: a tce and social capital perspective. journal of information technology, 30(4), 307-324. lioliou, e., zimmermann, a., willcocks, l., and gao, l. (2014). formal and relational governance in it outsourcing: substitution, complementarity and the role of the psychological contract. information systems journal, 24(6), 503-535. karimi-alaghehband, f., rivard, s., wu, s., and goyette, s. (2011). an assessment of the use of transaction cost theory in information technology outsourcing, the journal of strategic information systems, 20(2), 125-138. kotlarsky, j., oshri, i., dibbern, j. and d. mani (2020) information systems sourcing, mis quarterly research curation (available online on mis quarterly research curations https://static1.squarespace.com/static/5887a660b3db2b05bd09cf36/t/5f45bf460ab71f19 7974037d/1598406471274/is+sourcing+curation+-+update++2020+final.pdf). kotlarsky, j., rivard, s. and oshri, i. (2023). “building a reputation as a business partner in information technology outsourcing”. mis quarterly, 47(2), 901– 922. doi.org/10.25300/misq/2022/15554 oshri, i., arkhipova, d., and vaia, g. (2018). exploring the effect of familiarity and advisory services on innovation outcomes in outsourcing settings. journal of information technology, 33(3), 203-215. doi.org/10.1057/s41265-018-0052-3 oshri, i., kotlarsky, j. and a. gerbasi (2015) strategic innovation through outsourcing: the role of relational and contractual governance, the journal of strategic information systems, 24(3), pp. 203-216. oshri, i., kotlarsky, j. and willcocks, l.p. (2023). the handbook of global outsourcing and offshoring, palgrave, london. 4th edition, isbn 978-3-031-12034-3 su, n., levina, n., and ross, j. w. (2016). the long-tail strategy for it outsourcing. mit sloan management, 57(2), 81-89. https://sloanreview.mit.edu/article/the-long-tail-strategyfor-it-outsourcing/ https://link.springer.com/book/10.1007/978-3-031-12034-3 https://link.springer.com/book/10.1007/978-3-031-12034-3 australasian journal of information systems oshri and kotlarsky 2025, vol 29 provocation growth of start-up ecosystems 10 copyright copyright © 2025 [oshri and kotlarsky]. this is an open-access article licensed under a creative commons attribution-non-commercial 4.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v29.6061 https://creativecommons.org/licenses/by-nc/4.0/deed.en https://doi.org/10.3127/ajis.v29.6061 paper title (paper title style) australasian journal of information systems aghajani, ahsan & whiteside 2023, vol 27, research article agility meets a project portfolio 1 when agility meets a project portfolio: a study of success factors in large organisations masoud aghajani edith cowan university, perth, australia kamrul ahsan rmit university, melbourne, australia kamrul.ahsan@rmit.edu.au naomi whiteside department of education, victoria, australia abstract the iterative nature of agile methods combined with high levels of team and customer interactions and continuously changing it and software development project requirements make the management of agile project portfolios very complex. to date, the mechanisms under which project portfolio management adapts to these complexities and achieves portfolio success have not been thoroughly investigated. this study explores the notion of success and its impacting factors in large organisations' portfolios of agile it and software development projects. using a multiple case study design, we analysed the agile project portfolios of seven large organisations. we identified four success criteria and 15 success factors and categorised them into a unique agile portfolio success framework. some of these criteria and factors are unique to agile project portfolios. the framework contributes to agile and project management literature by conceptualising the notion of success in portfolios of agile projects while revealing a set of factors that affect the relationship between an agile portfolio with its subcomponents and the surrounding environment. the framework supports managers and practitioners in large organisations in reflecting on their agility efforts to achieve higher success rates in their agile portfolios. keywords: agile project management, project portfolio management, agile portfolio, it and software development projects, success factor. 1 introduction agile project management (apm) has emerged as an approach for project planning and delivery that addresses many of the shortcomings of traditional it and software development (isd) projects (such as waterfall projects) and improves project results (gemino et al., 2021; serrador & pinto, 2015). unlike the traditional project management approaches that emphasise upfront planning and sequential project delivery, apm methods rely on iterative planning and delivery with a focus on incremental product delivery through fast and flexible interactions between self-managed development teams and customers (schwaber, 2004). through their collective processes, routines and relationships with their environment, agile methods create and embrace change to achieve customer value (conboy, 2009). coordinating agile project delivery is primarily done through routine activities such as daily stand-up meetings, iterative product reviews and team retrospectives (schwaber, 2004; thummadi et al., 2011). recent research indicates the widespread adoption of agile methods in small, large, and multiproject isd project environments (digital.ai, 2021; jørgensen, 2019; paasivaara et al., 2018). australasian journal of information systems aghajani, ahsan & whiteside 2023, vol 27, research article agility meets a project portfolio 2 while there is an extensive body of research on small agile projects and teams, little research has examined the impacts of agile methods on the management of project portfolios (stettina & hörz, 2015; sweetman & conboy, 2018). project portfolio management (ppm) entails balancing shared resources and risks between multiple projects to maximise business value and achieve strategic alignment (cooper et al., 1999; meskendahl, 2010). ppm is perceived as a governance structure with a critical management decision-making role in aligning projects with organisational strategy and achieving business value through effective resource allocation and risk management (mosavi, 2014). ppm is still an emerging aspect of strategic business management that aims to improve the chances of project success (hansen & svejvig, 2022; padovani & carvalho, 2016). with ppm, organisations can achieve greater project visibility and higher product development efficiency, possibly leading to competitive advantage and greater chances of business success (killen et al., 2008a; martinsuo, 2013). roughly 60% of organisations utilise portfolio structures to manage their projects (kpmg, 2017). ppm and apm intersect in isd project portfolios. however, managing multiple agile projects under project portfolios is particularly challenging and troublesome (conboy & carroll, 2019; stettina & hörz, 2015; sweetman & conboy, 2018). agile projects result in a high degree of complexity at the portfolio level because the evolving and flexible nature of agile projects, combined with the self-organising and autonomous setup of agile teams, are at odds with traditional ppm's well-established and formal organisational processes (dingsøyr & moe, 2014). when aggregated at the portfolio level, the intensified interactions of agile teams and customers can create unmanageable tensions and complexities that threaten the success of agile efforts and ppm (sweetman & conboy, 2018). sweetman and conboy (2018) indicate that portfolio governance, in traditional ppm, is stable over time as projects are usually plan-driven and predictable. however, projects of an agile portfolio are pushed in different directions with continuously changing project requirements and customer demands. consequently, even though agile projects may be successful individually, their portfolio governance can become disjointed, incoherent and complex if not managed effectively. scholars call for more research to adapt ppm to the tensions and complexities of multiproject agile environments (stettina & hörz, 2015; sweetman & conboy, 2018). despite the rapid adoption of agile methods at scale and in portfolios of isd projects, very little is known about the notion of success in agile portfolios. prior agile literature has primarily focused on individual agile project success (chow & cao, 2008; misra et al., 2009; stankovic et al., 2013) and has identified success factors (sfs). sfs refer to areas that require continuous and careful attention to achieve organisational goals (fortune & white, 2006). however, the identified individual agile project sfs do not address the complexities of managing multiple agile projects under a portfolio. there is also a wealth of research on factors influencing traditional1 portfolio success (kaufmann et al., 2020; kock & gemünden, 2020; kopmann et al., 2017). however, there has been limited attention to multiproject agile environments as a context for ppm. therefore, we respond to recent calls for research on the consequence of apm on ppm effectiveness and success (niederman et al., 2018; sweetman & conboy, 2018) by aiming to explore the notion of success for agile project portfolios by investigating their success criteria (scs) and sfs in large organisations. in this study, portfolios of agile projects (or agile 1 in contrast to agile portfolios, non-agile portfolios are referred to as 'traditional' portfolios. australasian journal of information systems aghajani, ahsan & whiteside 2023, vol 27, research article agility meets a project portfolio 3 portfolios) refer to the integrated management of multiple agile isd projects under independent portfolios. the study focuses on large organisations because they are more likely to practice ppm to deal with multiple agile isd projects. we adopted the definition of large organisations used by the organisation for economic cooperation and development (oecd), which defines large as having at least 250 employees (oecd, 2021). therefore, this study seeks to address the following questions: rq1: what constitutes success in portfolios of agile projects in large organisations? rq2: what factors contribute to the success of portfolios of agile projects in large organisations? the present research contributes to understanding agile isd portfolios and the conditions under which these portfolios achieve success. we use multiple case study research to develop an agile portfolio success framework with agile portfolio scs and sfs as its subcomponents. the framework can potentially guide large organisations in adapting and sustaining agile methods at scale and their agile isd portfolios. the following section reviews the relevant success literature in agile scale-up and traditional and agile ppm while also revealing the theoretical underpinning of this study. section 3 describes the multiple case study research design and presents seven case organisations. section 4 unveils the research data analysis and findings, and then section 5 discusses the key findings in line with previous research and explains the research implications and limitations. the last section summarises the findings and suggests future research directions. 2 literature review we review the relevant literature across two main domains, agile scale-up and project portfolios, to identify sfs in existing research, reveal gaps/problems in previous research and introduce a theoretical lens. 2.1 agile scale-up: challenges and success factors rigby et al. (2018) set the ultimate goal of scaling agile in a function like isd to transition from several small agile innovation teams to many more teams with the vision to make agile the dominant way the organisation operates. scaling agile brings agile values and principles to business operations and support functions to achieve greater flexibility and efficiency (ebert & paasivaara, 2017). dingsøyr and moe (2014) argue that scaling agile involves two aspects. the first aspect is increasing the number of agile teams involved in higher levels of decisionmaking and planning by using agile practices (e.g., release planning and road mapping). second, scaling the system engineering activities executed in each sprint to a truly iterative practice instead of a stage-gated planned approach that supports cross-functional teams. when scaling agile in a large organisation, multiple teams may collaborate to develop various subsystems of a single product or to produce different products that are part of a product family (beecham et al., 2021). dingsøyr et al. (2017) point out that scaling agile involves upward integration of agile principles and practices in the organisational hierarchy, from teams to programs and portfolios, and higher to the executive suite. several agile scale-up frameworks have been proposed by agile practitioners to guide organisations through the scale-up process. according to the 15th state of agile report by digital.ai (2021), scaled agile framework (safe) (leffingwell, 2018) and scrum@scale australasian journal of information systems aghajani, ahsan & whiteside 2023, vol 27, research article agility meets a project portfolio 4 (sutherland, 2021), with 37% and 9% adoption rates are the most dominant agile scale-up frameworks. enterprise scrum (es) (herman, 2017), spotify model (kniberg & ivarsson, 2012), agile portfolio management (agilepfm) (agile-business, 2017), disciplined agile delivery (dad) (ambler & lines, 2012) and large scale scrum (less) (larman & vodde, 2013) are also used but with much lower adoption rates. these frameworks comprise various practices and processes supported by predefined roles, artefacts and rituals. among these frameworks, only safe and agilepfm clearly recognise the need for ppm. the full version of safe (version 5) consists of three layers: portfolio, large solution and essential. the portfolio level aligns organisational strategy with lower-level solution development by defining several portfoliolevel roles, practices and artefacts (scaled-agile, 2021). however, the framework is criticised as being highly prescriptive and inflexible for isd portfolios (geraghty, 2020; schwaber, 2014). agilepfm focuses on continuous value delivery and introduces the concept of rolling-wave planning for the dynamic management of agile portfolios using common agile practices such as stand-ups and retrospectives (dingsøyr et al., 2019). in general, while each of these frameworks has its own advocates and critics, there is limited empirical evidence on the effectiveness and success of these frameworks (beecham et al., 2021; conboy & carroll, 2019). for agile scale-up, the systematic literature review by dikert et al. (2016) identified 29 sfs from previous literature and categorised them under 11 categories. the important sf categories were choosing and customising the agile approach, top management support, focusing on agile values, ensuring management support and providing training and coaching. the action research by kalenda et al. (2018) explored a software company going through large-scale agile adoption. they confirmed a few previously identified sfs (i.e., management support and focusing on agile values) while revealing two new sfs: appropriate organisational culture and prior agile team experience. however, the authors did not clearly elaborate on the cultural elements essential for a successful agile scale-up initiative. rigby et al. (2018), drawing on the authors' past experiences, provided recommendations on building and sustaining agile at scale, such as getting agile rolling with a small wave of agile teams (piloting), sequencing the adoption process and acknowledging the importance of acquiring talented and motivated agile practitioners. a recent systematic literature review by edison et al. (2021) identified 27 sfs for agile scale-up frameworks and organised them into four categories: management and organisational, process, people and technology. authors found it challenging to validate each agile scale-up sfs with different agile scale-up frameworks because of the association of these sfs with studies with different levels of analysis and granularity. they also found that not all agile scale-up frameworks (such as dad and scrum@scale) received adequate attention from scholars. table 1 summarises the previously identified sfs for agile scale-up that were discussed in this section. overall, the empirical findings on agile scale-up and its success research are still in the early stages, and evidence of sustaining agile scale-up frameworks is still scarce (conboy & carroll, 2019). furthermore, the dominance of a high number of grey literature on agile studies could indicate agile practitioners’ interest and concerns about agile scale-up practices and frameworks. however, while the number of empirical studies addressing agile scale-up frameworks is on the rise (edison et al., 2021), academia still lags behind the practice in this domain. moreover, while some agile scale-up sfs are reported in recent studies, these sfs are mainly linked to the adoption stages of agile at scale or a specific agile scale-up framework and do not cover the post-adoption stages of agile transformation. australasian journal of information systems aghajani, ahsan & whiteside 2023, vol 27, research article agility meets a project portfolio 5 agile scale-up sfs brief description supporting references top management support top management decisions and support are critical for the success of agile scale-up efforts. successful agile scale-up transition requires strong and aligned management. (dikert et al., 2016; edison et al., 2021; kalenda et al., 2018; rigby et al., 2018; shameem et al., 2017) communicate agile values the agile way of working will take root with intensive and transparent communication. moving towards agile scale-up is assisted by making agile benefits and values visible. (dikert et al., 2016; kalenda et al., 2018; rigby et al., 2018) customising agile methods organisations should customise their agile approach and practices to fit their unique challenges and needs. (conboy & carroll, 2019; dikert et al., 2016; edison et al., 2021) piloting successful piloting clears disbelief about the appropriateness of the agile scale-up method and serves as a learning experience that enhances the adoption process. (dikert et al., 2016; kalenda et al., 2018; paasivaara et al., 2018) training and coaching training on methods and practices improve the chances of succeeding in scaling up agile. agile practices are best learned by doing, and coaching can help sustain agile practices and mindset. (conboy & carroll, 2019; dikert et al., 2016; edison et al., 2021; shameem et al., 2017) engaging people engaging stakeholders allows an organisation to gain acceptance and legitimise the agile scale-up process. inclusion will motivate people to participate and work in a new agile way. (bass & haxby, 2019; dikert et al., 2016; edison et al., 2021; kalenda et al., 2018; kasauli et al., 2021) value agile communities establishing agile communities influences the organisational culture and allows the formation of agile skills among teams. (dikert et al., 2016; edison et al., 2021; shameem et al., 2017; smite et al., 2019) allow teams to selforganise self-organisation creates commitment to the change and allows teams to take ownership of product development. such empowerment increases teams' productivity and morale. (dikert et al., 2016; edison et al., 2021; shameem et al., 2017) requirements management many requirement engineering challenges are reported in the agile literature. organisations require to invest in and implement effective requirement discovery practices. (bjarnason et al., 2022; dikert et al., 2016; kalenda et al., 2018; kasauli et al., 2021; shameem et al., 2017) align the organisation organisational goals should be set and aligned through more fit-for-purpose, flatter and flexible processes. agile roles need to be defined initially to ensure coverage of strategic priorities. (conboy & carroll, 2019; dikert et al., 2016; edison et al., 2021) appropriate organisational culture an appropriate organisational culture (e.g., trust culture) fosters collaboration among teams and other stakeholders. (abrar et al., 2021; edison et al., 2021; kalenda et al., 2018) experienced and disciplined teams a high level of agile team members' technical and social knowledge and expertise is critical to agile scale-up success. (edison et al., 2021; kalenda et al., 2018; rigby et al., 2018; shameem et al., 2017; šmite et al., 2017) shared vision a shared vision assists an organisation in setting a common ground for all stakeholders and clarifies organisational strategies and project goals. (edison et al., 2021; kalenda et al., 2018; shameem et al., 2017) information and knowledge sharing an appropriate knowledge-sharing system supports communication, decision-making, knowledge networks, and social capital through the agile scale-up process. (dingsøyr et al., 2019; edison et al., 2021; kalenda et al., 2018; shameem et al., 2017) table 1. the summary of agile scale-up sfs from the literature australasian journal of information systems aghajani, ahsan & whiteside 2023, vol 27, research article agility meets a project portfolio 6 overall, the empirical findings on agile scale-up and its success research are still in the early stages, and evidence of sustaining agile scale-up frameworks is still scarce (conboy & carroll, 2019). furthermore, the dominance of a high number of grey literature on agile studies could indicate agile practitioners’ interest and concerns about agile scale-up practices and frameworks. however, while the number of empirical studies addressing agile scale-up frameworks is on the rise (edison et al., 2021), academia still lags behind the practice in this domain. moreover, while some agile scale-up sfs are reported in recent studies, these sfs are mainly linked to the adoption stages of agile at scale or a specific agile scale-up framework and do not cover the post-adoption stages of agile transformation. 2.2 traditional portfolios: success criteria and success factors traditional ppm is an established research field, and numerous studies have contributed to its advancement by identifying its characteristics, methods, processes and goals (hansen & svejvig, 2022). in traditional ppm, portfolio scs are defined as achieving well-established ppm goals, i.e., achieving average success over all projects, using synergies among projects, finding strategic alignment and portfolio balancing (cooper et al., 1999, 2002; meskendahl, 2010). achieving average success over all projects within a portfolio is linked to delivering the projects on time, within budget and to specifications extended by meeting customer satisfaction dimensions of project success (meskendahl, 2010; shenhar et al., 2001). synergy exploitation is about achieving greater benefits through the integrated management of multiple projects. these benefits can include enhancing technologies, markets, knowledge and resource synergies. for instance, coordinating the interdependencies among projects that use the same technology or operate in the same market may increase efficiency or shared opportunities, which may not be achievable through the independent management of each project (kopmann et al., 2017; meskendahl, 2010). according to voss and kock (2013), increased project portfolio interdependency necessitates more coordination but can also lead to better use of synergies. strategic alignment can be achieved when projects under a portfolio fit the organisation’s strategy (hoffmann et al., 2020; kopmann et al., 2017). portfolio balancing refers to attaining an equilibrium of risks, resources and long-term versus short-term opportunities when executing a portfolio (killen et al., 2008b; teller et al., 2012). previous research has identified various sfs for traditional ppm. earlier contributions of ppm scholars have focused on factors related to ppm structure, for example, establishing formal portfolio processes and top management involvement and support. project managementrelated factors as a subset of ppm are also explored. for instance, information availability, sharing and quality (jonas et al., 2013; martinsuo & lehtonen, 2007; unger et al., 2012) and project goal setting (biedenbach & müller, 2012; martinsuo & lehtonen, 2007) are proposed as project-level sfs contributing to portfolio success. recent research on ppm has focused on identifying organisational capabilities as sfs. for example, kock and gemünden (2020) showed that portfolio innovativeness as a dimension of entrepreneurial orientation moderates the relationship between strategic ppm practices (portfolio structuring and steering) and portfolio success. kaufmann et al. (2020) showed the positive influence of two antecedents of organisational agility (i.e., entrepreneurial orientation and voice behaviour) on strategy recognition and further portfolio success. voice behaviour in an organisation relates to the cultural support an individual receives to willingly and constructively express concerns and opportunities. kopmann et al. (2017) found a positive and significant relationship between portfolio strategic control practices and portfolio success, australasian journal of information systems aghajani, ahsan & whiteside 2023, vol 27, research article agility meets a project portfolio 7 while both deliberate strategy recognition and emerging strategy recognition mediated the relationship. with deliberate strategy recognition, they referred to the process of purposefully cascading the formulated corporate strategy from higher levels of the organisation’s hierarchy to the project level, while emerging strategy recognition was about disclosing emerging patterns in a portfolio to inform the strategy formulation process. in another major study, kock et al. (2020) showed the overall positive effect of ppm information systems (ppmis) on the quality of ppm processes and portfolio success. one interesting finding in their study was that the ppmis effect could only materialise when highly formal project management, ppm and risk management practices existed. table 2 presents a broad selection of traditional ppm sfs. traditional ppm sfs brief description supporting references establishing formal portfolio processes formalisation of projects and portfolios through configuring standardised routines and processes positively impacts portfolio quality, facilitates resource allocation and improves transparency in projects and portfolio environments. (blomquist & müller, 2006; jonas, 2010; kester et al., 2014; kock et al., 2016, 2020; kopmann et al., 2015; teller, 2013; teller et al., 2012; teller & kock, 2013) information availability, sharing and quality information availability for decision-makers is one significant project-level factor contributing to portfolio success. also, a lack of information quality, sharing and transparency is a significant barrier to sound decision-making at the portfolio level. (dietrich & lehtonen, 2005; jonas, 2010; jonas et al., 2013; lerch & spieth, 2013; martinsuo & lehtonen, 2007; spieth & lerch, 2014; teller et al., 2012) top management involvement and support senior managers’ involvement in portfolio decision-making (e.g., advocating to implement a strategy or re-evaluate/terminate projects) is identified as an essential contributor to portfolio success. top managers’ unsupportive behaviour or unintentional enforcement of undesired projects negatively impacts portfolio success. (beringer et al., 2013; blomquist & müller, 2006; jonas, 2010; jonas et al., 2013; kock et al., 2015; rank et al., 2015; unger et al., 2012) project goal setting appropriate project goal setting improves project management efficiency and contributes to portfolio success. project goal setting should be expanded towards broader business goals if better portfolio results are expected. (biedenbach & müller, 2012; martinsuo, 2013; martinsuo & lehtonen, 2007) internal stakeholder engagement (e.g., project managers, portfolio managers and line managers) previous research indicates the significant influence of line managers, project portfolio managers and project managers on portfolio success. for example, the project manager's influence, resulting from their authority, responsibility and engagement towards project outcomes, plays a crucial role in the success of project portfolios. (beringer et al., 2013; blomquist & müller, 2006; jonas, 2010; petro & gardiner, 2015; unger et al., 2012) strategic ppm practices portfolio practices such as portfolio structuring (e.g., stakeholder management and strategic planning), portfolio steering (e.g., business case monitoring, strategic control and adaptiveness) and risk management (risk processes and culture) are positively related to portfolio success. (dietrich & lehtonen, 2005; jonas, 2010; kaufmann et al., 2020; kock & gemünden, 2020; kopmann et al., 2017; müller et al., 2008; teller, 2013; teller et al., 2014; teller & kock, 2013) entrepreneurial orientation entrepreneurial orientation refers to “the strategy-making processes that key decision makers use to enact their firm’s organisational purpose, sustain its vision, and create competitive advantage(s)” (rauch et al., 2009, p. (kaufmann et al., 2020, 2021; kock & gemünden, 2020; rank et al., 2015) australasian journal of information systems aghajani, ahsan & whiteside 2023, vol 27, research article agility meets a project portfolio 8 traditional ppm sfs brief description supporting references 763). dimensions of eo (e.g., portfolio innovativeness) are essential factors for ppm performance. customer relationship value both relationship values for and from the customer (such as monetary terms, product ideas, access to new markets, or other virtues) independently contribute to portfolio success. such relationship values are even more critical in complex portfolios. (voss, 2012; voss & kock, 2013) ppm absorptive and adaptive capabilities the early utilisation of absorptive capabilities (i.e., the firm’s ability to utilise external knowledge through various ways of learning) is essential for project and portfolio performance. also, adaptive capabilities (i.e., the capability to identify and capitalise on emerging market opportunities, e.g., global marketing monitoring and market/technology sensing) positively impact portfolio success. (biedenbach & müller, 2012; killen et al., 2012) ppmis ppmis application positively and significantly relates to ppm quality, promoting portfolio success. ppmis application positively impacts project and portfolio formalisation. (kock et al., 2020) table 2. the summary of portfolio sfs from the literature most portfolio success studies view traditional ppm as a rational and plan-driven decisionmaking process. while this viewpoint has contributed to improving organisations’ ppm structure and performance (martinsuo, 2013), it is limited in three ways. first, portfolio decision-making is less rational when its context (the unique conditions under which the portfolio is being managed) is considered. for example, the projects and portfolio selection can become politically motivated and path-dependent instead of being highly rational and planned (martinsuo, 2013). also, the day-to-day practice of ppm may deviate from its intended rational and plan-driven approach, as portfolio managers often have to apply it in uncertain and dynamic situations that they cannot fully comprehend. second, portfolio success draws on what the organisation and its shareholders value. however, the perception of value may differ depending on the stakeholders’ priorities, knowledge and attitudes (shenhar et al., 2001). stakeholders may hold conflicting values and different understandings about portfolio success (beringer et al., 2013; martinsuo & killen, 2014). also, internal and external stakeholders can potentially promote, support, slow down, or limit the ppm process, and their involvement is crucial for sustaining the project portfolios (beringer et al., 2013; sánchez, 2015). however, current traditional ppm research does not clearly show the impact of different/conflicting viewpoints of stakeholders on ppm. third, to our knowledge, very few studies acknowledge the significant relationship between multiproject customers and portfolio success (voss, 2012; voss & kock, 2013). current ppm research largely neglects the consequences of customer involvement, competencies, and actions on portfolio success. 2.3 agile portfolios: the research problem research that focuses exclusively on agile portfolios has been minimal. the early attempts to explore these portfolios are mostly experience reports explaining transitioning from traditional to agile portfolios and identifying challenges, benefits and practices that australasian journal of information systems aghajani, ahsan & whiteside 2023, vol 27, research article agility meets a project portfolio 9 accompany said transition (laanti & kangas, 2015; rautiainen et al., 2011; thomas & baker, 2008). few theoretical frameworks are developed for agile portfolios (krebs, 2008; vahaniitty, 2012), although we found no evidence of their use by practitioners. stettina and hörz’s (2015) multiple case study is perhaps one of the first attempts to empirically explore portfolios of agile projects. most of the 14 cases in their study undertook a bottom-up approach to adopt and scale agile methods. for instance, organisations first adopted agile methods at the project level and then expanded their agility efforts toward ppm as the next neighbouring domain of practice. their study revealed several important challenges in managing agile portfolios, e.g., alignment to existing processes, lack of involvement and commitment on the side of senior managers and difficulties in resource alignment (stettina & hörz, 2015). using complex adaptive systems theory and expert interviews, sweetman and conboy (2018) developed propositions to address how ppm can adapt to the tensions that arise from multiple agile projects under a portfolio. one tension was the increased number of teams and customer interactions in apm that can become unmanageable and cumbersome when aggregated at the portfolio level. also, the iterative and evolving nature of apm combined with the selforganising and autonomous routines of agile teams can cause continuous challenges to the portfolio mix of agile projects. these agile characteristics are particularly at odds with ppm's centralised, formal and plan-driven structure (stettina & hörz, 2015; sweetman & conboy, 2018). in such conditions, whilst a portfolio can stabilise through governance and oversight, its agile projects’ capability to address change might diminish, resulting in apm losing its significance (sweetman & conboy, 2018). therefore, sweetman and conboy (2018) propose that portfolio managers should find the right balance between control and autonomy in their agile projects and portfolios. hoffmann et al. (2020) employed activity theory to investigate a single revelatory case in the context of agile it ppm. they empirically derived three design goals (alignment, efficiency and agility) and nine supporting design principles for agile it ppm. the principles are continuous strategic alignment, short cycle portfolio planning, termination of non-value-adding projects, project approval only at the last moment, approval of only short projects, establishing a portfolio buffer, preventing resource overload, avoiding multitasking and ensuring uninterrupted project completion. however, in their study, efficiency was limited to meeting business requirements, quality, target dates and budget. hoffmann et al. (2020) and sweetman and conboy (2018) concluded that empirical studies are needed to better understand what constitutes success in agile portfolios. 2.4 the theoretical lens and conceptual framework informed by recent research on the relationship of project portfolio with its context (martinsuo, 2013; martinsuo & geraldi, 2020), we used contingency theory (ct) as a theoretical lens to develop a conceptual framework that guided the process of qualitative data collection and analysis. ct asserts that relationships between organisational characteristics (e.g., structure, strategy and management practices) and its performance depend on the organisation's context (donaldson, 2001). accordingly, no single form of organising can be suitable for every contextual setting, so a fit between the context and organisation is required (shenhar, 2001). in project management, ct portrays projects as “temporary organisations” within their parent organisations and indicates that the universal “one size fits all” approach to managing projects may be inadequate (shenhar, 2001, p. 395). also, the current view of ppm frames a project portfolio as an organisation hosting multiple projects and programs as temporary organisations (bechtel et al., 2021; hansen & svejvig, 2022; martinsuo & geraldi, 2020). such a australasian journal of information systems aghajani, ahsan & whiteside 2023, vol 27, research article agility meets a project portfolio 10 view opens up new perspectives on how project portfolios are organised, performed and evolved in relation to their internal and external contingencies (hansen & svejvig, 2022; martinsuo & geraldi, 2020). a contingency is “any variable that moderates the effect of an organisational characteristic on organisational performance” (donaldson, 2001, p. 5). following bechtel et al. (2021) and martinsuo and geraldi (2020), we conceptualise a project portfolio as a semi-temporary organisation with limited budgets and time constraints that function as a link between permanent (parent organisation) and temporary (programs and projects) forms of organising. martinsuo and geraldi (2020) argue that as project portfolios are typically concerned with product and service development projects, their success is critical in achieving overall business success and the firm’s market positioning. the project portfolio, as the collection of multiple projects, constitutes the portfolio subunits and the environment surrounding the portfolio (bechtel et al., 2021). therefore, we argue that for an isd portfolio to succeed as an organisation, its characteristics should fit between its subunits (agile projects) and its organisational environment (context). hence, in figure 1(a), we depict how this study considers agile projects and their related isd portfolio in an organisational setting, which will explain the interactions between different levels (i.e., agile projects, portfolio and organisational environment). previous research has explored various dimensions of traditional portfolio success (cooper et al., 1999, 2002; jonas, 2010; meskendahl, 2010), while the effect of project-level contingencies on portfolio success is also considered. for example, teller et al. (2012) showed the association of single project management formalisation (configuration of standardised routines and practices) with increased ppm quality and portfolio success; and martinsuo and lehtonen (2007) found projects’ information availability and goal setting as significant contributors to portfolio success. also, many studies have tested the effect of contextual contingency variables on the link between certain management aspects and traditional portfolio success (kaufmann et al., 2021; kock & gemünden, 2020; voss & kock, 2013). however, agile projects differ significantly from traditional plan-based projects as they comprise different routines, processes and artefacts (stettina & hörz, 2015; thummadi et al., 2011). further, agile projects continuously evolve in response to frequently changing project requirements and customer demands, making them quite different from traditional planbased projects (sweetman & conboy, 2018). therefore, we focus on understanding portfolio success in agile project environments by identifying its scs and sfs in large organisations. to analyse the portfolio success with theoretical rigour, considering ct as a theoretical lens, we developed a conceptual framework shown in figure 1(b). we distinguish between agile portfolio sfs by categorising them at two levels: project-level sfs (p-sfs) and contextual sfs (c-sfs). we consider p-sfs as contingencies related to individual agile projects and teams as portfolio subunits that may influence agile portfolio scs. we also investigate sfs that characterise the agile portfolio in the environment (context) under which the agile portfolio is operating. these c-sfs comprise contingency factors related to the agile portfolio environment, which may be related to the organisational structure and culture affecting portfolio success. following the influence of ct on the organisational context, we further explored an extension of ct called the uncertainty/consequences framework (uc framework) (howell et al., 2010) to interpret the research results. the uc framework classifies project (and portfolio as a collection of projects) contingency factors into two types, uncertainty (u) and consequences (c). it considers these categories as two orthogonal dimensions determining the appropriate australasian journal of information systems aghajani, ahsan & whiteside 2023, vol 27, research article agility meets a project portfolio 11 process model (i.e., agile or plan-driven) for managing a particular project or portfolio (howell et al., 2010). we argue that each identified portfolio sf (p-sfs or c-sfs) can fall into either u or c categories. therefore, we use u and c dimensions to cluster the identified agile portfolio sfs in section 4. in what follows, we explain the u and c dimensions from the literature. the u dimension represents the probability that something unexpected will happen in the project and includes contingencies related to uncertainty, complexity and urgency resulting from a single project or portfolio environment, scope and outcomes. uncertainty includes probabilistic outcomes and the ambiguity of situational parameters related to goals, methods, markets, technology or external stimuli (howell et al., 2010; williams, 2005). complexity is defined as the degree of differentiation and interdependence of project or portfolio components related to scope, environment, diversity and the size of the parent organisation, portfolio, project or team (howell et al., 2010; kock et al., 2016). the challenge of complexity lies in comprehending and making sense of the intricacies involved, which ultimately affects the relationship between inputs and outputs of a project (or portfolio). a lack of predictability is synonymous with uncertainty, and thereby complexity becomes a factor in uncertainty (howell et al., 2010). urgency is the extent to which time constraints are a limiting factor (of limited information) in a single project or portfolio activities and decision-making. in projects and portfolios, urgency limits the ability to make decisions because of limited information. managers under time pressure tend to take more control of the situation, often by shortsighted measures. these effects increase the probability of unexpected project behaviour (howell et al., 2010; williams, 2005). figure 1. the research setting and conceptual framework the c dimension represents how much it will matter if some uncertainties occur. the c factors are related to criticality and team empowerment. criticality can be seen as how much is at stake in a single project or portfolio. in other words, it is the impact or consequences of unexpected or unmanaged events (e.g., project failure) on the project/portfolio goals and, consequently, the organisation. empowerment-related factors such as authority levels, team australasian journal of information systems aghajani, ahsan & whiteside 2023, vol 27, research article agility meets a project portfolio 12 size and geographic distribution are often the consequences of organisational culture that the parent organisation frequently imposes. changing such parameters may be infeasible for projects and portfolios. the ability to make such changes is an empowerment issue because it may or may not be authorised by the parent organisation. in a project or portfolio organisation, the consequences of unexpected events depend heavily on how the team manages them. team empowerment factors affect the team’s ability to communicate, comprehend and quickly respond to unexpected events in projects and portfolios (howell et al., 2010). hence, it could improve the chances of dealing with unexpected events and increase the chances of project and portfolio success. both the u and c dimensions can explain if a single project or portfolio can be managed using an agile or plan-driven process model. for example, in an agile project, if the dominant issue is understanding its ill-defined goals at the initial stages, the team must adopt an iterative process involving partial delivery of the goals and use feedback to continuously redefine the goals to reduce unexpected consequences. in that case, the project success is independent of u. the approach assumes high uncertainty; if, in practice, it is lower, this has no effect on success. however, the agile project’s success is limited to the c dimension. any event that the project team cannot handle may lead to failure. also, the team will not necessarily know if they have effectively handled the event until later stages of the project. conversely, suppose the successful execution of a project relies on the optimal sequencing of available resources and constraints in a project plan rather than the project objectives. in that case, the success of the chosen process model is independent of c because the objective is to anticipate and avoid changes, and their consequences are less irrelevant. hence, the performance of the process model will vary by how effectively project uncertainty is addressed. the uc framework suggests traditional plan-driven approaches to project delivery under such conditions. the developed conceptual framework (with u and c dimensions) helped to categorise the identified sfs to form the study’s research framework, as is explicated in section 4. the next section explains the research process and methodology. 3 research methodology this research explores the notion of success and its related sfs in portfolios of agile isd projects in large organisations. we used a qualitative multiple case study design to investigate agile portfolios from real-life scenarios, especially when knowledge and perspective on agile portfolio scs and sfs are limited in the literature. the case study design is suitable when contextual conditions are crucial for understanding the phenomena and the boundaries between phenomenon and context are unclear (yin, 2018). the multiple case study design allowed us to collect multiple perspectives and experiences of participants in different organisations and industries. this section explains the case and participant selection process, data collection and analysis procedures and the steps taken to ensure process and data validation. 3.1 case and participant selection process project portfolios are usually found in large organisations where the alignment of multiple projects and programs with business strategy is crucial (kopmann et al., 2017). hence, the study’s unit of analysis was limited to a portfolio of agile isd projects in a large organisation. we exclusively selected and investigated portfolios of agile isd projects within the studied cases because these portfolios show significantly different patterns compared to traditional australasian journal of information systems aghajani, ahsan & whiteside 2023, vol 27, research article agility meets a project portfolio 13 (non-agile) portfolios (sweetman & conboy, 2018). we used replication logic to select cases that either predict similar results or produce contrary and extending outcomes. then, multiple cases were treated as discrete experiments that provided replications, contrasts and extensions, aiding the theoretical development of findings (eisenhardt & graebner, 2007; yin, 2018). four criteria guided the case selection process. first, the potential case organisation had to practise ppm and use apm for its isd projects. we did not consider organisations that use apm for applications outside the isd field. second, the potential organisation had to have at least 250 employees to be considered large, following the business size definition by oecd (2021). third, the selected agile portfolio must have contained at least three agile teams to be considered in this study. the criterion was derived from the taxonomy of scale proposed by dingsøyr et al. (2014). forth, the potential organisation should have practised apm and ppm together for at least three years. we assumed such a period would be crucial for developing an agile mindset and culture within the case organisation. following these criteria, an initial list of 35 potential organisations operating in australia was created. we sought potential participants with active roles in agile projects, programs and portfolios within the potential case organisations with at least three years of agile experience. overall, 125 potential participants were identified and contacted. the eligibility of potential participants for inclusion was checked through available public information (such as participants’ linkedin profiles) and double-checked further upon interviews. overall, seven case organisations and 13 participants consented and participated in the study. table 3 contains the case descriptions, including their industry, product offerings, size, type of investigated portfolios, number of agile teams and agile methods in use and agile practice history. table 3 also reports information about participants and their agile experience at the time of the interviews. the letters a to h symbolise the anonymised cases. case industry, product offering & size type of portfolio, number of agile teams & methods in use agile practice history case participants & agile experiences a education and training it provides tertiary and vocational education along with research services. it operates with over 10,000 staff and manages its projects with multiple portfolios. it services portfolio the portfolio was structured under the operations business unit and consisted of more than six agile teams (7 members each) using safe, scrum, xp, kanban, and waterfall to deliver projects. the agile journey started in 2014 when the company experienced financial loss and delays in isd project delivery. senior managers hired a consultancy firm to support the transition to safe, delivering three major isd programs under one portfolio using seven agile teams. the initial agile adoption resulted in increased product quality, faster value delivery and improved project visibility. two people: release train engineer (rte) (7 ys) portfolio manager (pfm) (10 ys) b construction digital collaboration solution it provides webbased and mobile collaboration technologies for isd portfolio the portfolio was developing a range of digital solutions for various industries. over 25 agile teams (7 members each) were operational, using agile was a salvation for the company after its operations dealt with increasing product complexity, unhappy customers, and growth pressures in 2009. while the developers worked tirelessly to test and fix software products, they could not release two people: vice president of collaboration (vp) (15+ ys) scrum master (sm) (12+ ys) australasian journal of information systems aghajani, ahsan & whiteside 2023, vol 27, research article agility meets a project portfolio 14 case industry, product offering & size type of portfolio, number of agile teams & methods in use agile practice history case participants & agile experiences projects and processes on software-as-aservice (saas). it operated with about 850 staff and ran its projects under one portfolio. scrum, xp, and kanban to deliver projects. them fast enough. agile adoption followed a bottom-up approach that led to the complete reinvention of the company’s software delivery process. as a result, features could be added more quickly, which improved team and customer satisfaction. c store-based retailing it provides retailing and consumer services for groceries, merchandise, and other consumer services. it operated with about 100,000 staff, and its projects ran under multiple portfolios. it operations portfolio the portfolio was structured under the operations business unit and consisted of five agile teams (9-10 members each) using scrum, xp, and waterfall to deliver projects. the company started agile practices in 2014. then, agile projects experienced a halt between 2015 to 2017. the agile scale-up started in 2017 by hiring a new, highly experienced agile champion. agile adoption was a response to increasing pressure in the competitive retail industry to deliver outcomes faster. the company had several independent agile isd project portfolios, and we investigated one. two people: an agile coach (ac) (10 ys) agile delivery manager (adm) (5 ys) d telecom, mobile and internet it provides consumers with telecommunication networks, mobile, internet and cable tv services. it had more than 32,000 staff, and its projects ran under multiple portfolios. digital experiences portfolio the portfolio operated under the networks and it business unit and consisted of over 25 agile teams (9-10 members each) using safe, scrum, xp, kanban, and waterfall to deliver projects. the company adopted agile practices in 2010, initiating an it delivery program to improve software quality and speed of delivery. later, the company hired a consultancy firm to implement safe in one of its portfolios. in 2015, it partnered with another consultancy firm to help train more agile staff to transform the entire business into agile. in 2017, the company reported 100 agile teams in multiple isd project portfolios. two people: agile coach (ac) (5 ys) business analyst (ba) (4 ys) e insurance services it provides investment, insurance, superannuation, and financial services. it had over 5000 staff, and its projects ran under multiple portfolios. isd products portfolio the portfolio operated under the innovation and transformation department and consisted of over seven agile teams (7 members or less) using scrum for its projects. the company was acquired and then restructured by an international firm in 2016. however, the company’s technology and digital staff have been involved in agile practices since 2014. in late 2016, the company hired an agile consultancy firm to help deploy safe but later backed down from the decision as they found little interdependencies between their isd projects. two people: agile coach 1 (ac1) (10+ ys) agile coach 2 (ac2) (5+ ys) australasian journal of information systems aghajani, ahsan & whiteside 2023, vol 27, research article agility meets a project portfolio 15 case industry, product offering & size type of portfolio, number of agile teams & methods in use agile practice history case participants & agile experiences f banking and financial services it provides various banking services, e.g., investment banking, credit and debit cards, finance, and insurance services. more than 33000 staff were working at the bank. its projects ran under multiple portfolios. technology risk portfolio the portfolio was structured under the technology and operation business unit and consisted of more than 50 agile teams (7-9 members each) using scrum, xp, kanban, and waterfall to deliver projects. the company has been adopting agile since 2010. agile was first introduced in an e-health project portfolio. from 2010 to 2013, the company transformed about 50 teams into agile under different digital platforms. such adoption improved communication, teamwork, staff motivation, project transparency, and product time to market. the company has occasionally used various agile consultancy firms to train its agile teams. two people: portfolio manager (pfm) (5+ ys) portfolio scrum master (psm) (6+ ys) g online real estate services it provides online real estate and commercial property advertising services. it had more than 1400 staff, and its projects ran under several portfolios. technology portfolio the portfolio was structured under the market and technology development directorate and contained more than 30 agile (10 members each) teams using scrum, xp, and kanban for its projects. sensing the pressures of the fastpaced and digitally connected world, the company started its agile journey in 2010. the company partnered with one leading agile consultancy firm to restructure its project delivery and core business systems. this partnership led to increased innovative delivery capability and collaborative, agile culture. the company’s entire business and isd project portfolios practice agile. one person: delivery lead (dl) (7+ ys) table 3. case descriptions the seven cases provided an appropriate mix for the study as they differed in their organisational structures, product offerings and services. such an appropriate mix allowed us to gain insights from multiple perspectives, enhancing data triangulation (etikan et al., 2016). as prior research suggests that project and portfolio managers’ skills, competencies and authority impact portfolio success (beringer et al., 2013), we only interviewed experienced apm and ppm practitioners within the case organisations. 3.2 data collection this study’s primary data sources were face-to-face semi-structured interviews. semistructured interviews facilitated the collection of rich data while maintaining the flexibility required for an exploratory study (saunders & lewis, 2018). all interviews were conducted in the participants’ preferred settings. interviews ranged from 45 to 75 minutes and were all audio-recorded. the audio recordings were later transcribed and sent back to the interviewees for cross-checking. field notes were also taken during interviews, and clarifications were sought after the interviews when required. australasian journal of information systems aghajani, ahsan & whiteside 2023, vol 27, research article agility meets a project portfolio 16 before data collection, an interview protocol containing several introductory and core sections was developed (yin, 2018). the introductory section contained questions about the positioning and structure of the isd portfolio, the organisation’s agile journey and the participant’s roles and responsibilities regarding apm and ppm and, finally, methodologies used for apm under the portfolio. the core section focused on asking for in-depth information related to the research questions, and follow-up questions were asked based on probes when appropriate. the identified agile scale-up and project portfolio sfs from the previous literature (tables 1 and 2) allowed us to set up appropriate interview probes. the questions asked during interviews are shown in appendix 1. 3.3 data coding and analysis the conceptual framework served as a priori construct to help shape the initial design of theory building process (barratt et al., 2011; eisenhardt, 1989). however, without having a theoretical conceptualisation that sufficiently explains agile portfolio success (stettina & hörz, 2015; sweetman & conboy, 2018), we decided to undertake an inductive approach to analyse data (miles et al., 2014). in addition, previous studies on portfolio success do not sufficiently explain ppm’s application in different contexts (martinsuo, 2013; martinsuo & geraldi, 2020), especially in agile project environments. figure 2. the data analysis process used in the study figure 2 depicts the data analysis procedure. in the analysis process, we organised interview data before processing. organising data involved transcribing interviews, typing field notes and arranging data in nvivo 12 software. the initial reading of the data concerned the authors reflecting on the overall relevance and credibility of the data, consulting with participants to clear doubts about transcriptions and adding descriptions/memos when needed. we referred to miles et al. (2014) and saldana (2016) to perform two coding cycles. we used initial coding for the first coding cycle to conduct within-case analyses. at this stage, we remained flexible australasian journal of information systems aghajani, ahsan & whiteside 2023, vol 27, research article agility meets a project portfolio 17 to all possible theoretical directions. however, we consistently referred to the research questions and conceptual framework to stay on track during the coding process. all first-cycle codes were tentative and provisional. we changed or reworded some codes as the analysis progressed. we also identified several themes from the codes and generated memos/descriptions that supported us in understanding the codes and themes as we transitioned to the second coding cycle. we employed pattern coding for the second cycle coding to categorise the codes and themes that emerged from the first coding cycle. the conceptually similar codes were combined throughout the process, while infrequent codes were reassessed for inclusion in the emerging framework. we conducted cross-case analyses to identify similarities and dissimilarities across cases and generated common themes and subthemes (miles et al., 2014; saldana, 2016). the common themes related to sfs were classified into two contingency categories based on ct: c-sfs and p-sfs (see section 2.4). then, we integrated these categories into the study’s agile portfolio success framework and moved on to present and discuss the research findings and their implications considering the theoretical lens and previous literature (sections 4 and 5). an example of the coding process is provided in appendix 2. 3.4 process and data validation the study followed miles et al. (2014) suggestions to ensure the quality of the research process, data and findings. we carefully designed the research process to strengthen reliability and trustworthiness and used multiple cases from various industries and product offerings. we ensured that interview questions and probes were aligned with the research questions, conceptual framework and literature review findings, allowing for a productive inquiry-based conversation. we tested the interview protocol by performing two preliminary interviews that further improved the cohesiveness and breadth of the interview questions and probes (castillo-montoya, 2016). we also ensured transparency and procedural rigour using computer-assisted coding software for data coding (nvivo 12). to gain insights from various perspectives and ensure data triangulation (etikan et al., 2016), we chose cases from various industries and selected participants with differing roles and responsibilities in agile isd portfolios. to ensure that the coding process was reliable, the first author performed the entire coding process while, at the same time, the second and third authors cross-checked the quotes, themes and the entire analysis process. in the next section, the research findings are presented. 4 analysis of findings figure 3 depicts the agile portfolio success framework and integrates the research findings for rq1 and rq2. with the presented framework, we suggest that achieving success in portfolios of agile projects in large organisations depends on achieving agile portfolio scs. also, the identified sfs should be carefully considered and adequately addressed to improve the chances of achieving agile portfolio success. to address rq1, four specific agile portfolio scs (sc1 to sc4) were identified from the data analysis. these scs help explain multiple facets of success in portfolios of agile projects in large organisations. we explain these scs in detail and highlight the new findings in section 4.1. to address rq2, 15 agile portfolio sfs emerged from data analysis, of which nine are categorised as contextual sfs (c-sf1 to c-sf9) and the rest as project-level sfs (p-sf1 to p-sf6). in section 4.2, we present the sfs and highlight the new findings. section 5 discusses the new findings and explains the research implications. australasian journal of information systems aghajani, ahsan & whiteside 2023, vol 27, research article agility meets a project portfolio 18 4.1 success criteria for agile portfolios rq1 is concerned with what constitutes agile portfolio success in large organisations. we asked research participants how their case organisations determined the success of their agile portfolios. ten sub-themes emerged from the data analysis, which reveals how case organisations measure or determine the success of agile portfolios under study. these subthemes were categorised into four main themes: portfolio value maximisation, strategic alignment, achieving economic success and portfolio balancing. figure 3. the agile portfolio success framework table 4 summarises the identified scs, their subthemes and supporting references from the traditional ppm research. we specifically identified four new subthemes unique to agile portfolios. the following explains all the identified scs and highlights new findings. themes (scs) subthemes* supporting references sc1. portfolio value maximisation it denotes achieving maximum value for agile projects in a portfolio. successful delivery of projects the criterion concerns achieving average project efficiency (satisfying budget, schedule, and specification) over all agile projects in a portfolio. (kester et al., 2014; kock et al., 2016; martinsuo & lehtonen, 2007; meskendahl, 2010) benefit for the customers the criterion measures the impacts on project (martinsuo & killen, 2014; padovani & carvalho, 2016; australasian journal of information systems aghajani, ahsan & whiteside 2023, vol 27, research article agility meets a project portfolio 19 themes (scs) subthemes* supporting references customers, e.g., satisfying customers and solving customers' problems. voss, 2012; voss & kock, 2013) increased customer involvement new it represents efforts to address increased customer interaction and focus on agile portfolios. sc2. strategic alignment it represents the degree to which agile projects reflect organisational strategy. strategic alignment it refers to criteria set at the portfolio level to ensure the alignment of agile projects to the organisational strategies and goals. (jonas, 2010; kopmann et al., 2017; meskendahl, 2010; stettina & hörz, 2015; unger et al., 2012) sc3. achieving business success it represents the contribution of an agile portfolio to overall business success through financial and non-financial achievements. market performance the criterion is about how sales objectives such as sales volume and market share are achieved when launching products and services from agile projects in a portfolio. (jonas, 2010; kester et al., 2014; killen et al., 2008b; meskendahl, 2010) shorter delivery to market new the criterion is related to tracking shorter releases of multiple agile project outcomes to the market/customer. (suomalainen et al., 2015) – only partial support embracing change new it measures the quantity of embraced (accomplished) changes resulting from projects delivered in an agile portfolio. sc4. portfolio balancing it is about a range of dimensions, e.g., resources, that must be balanced for an agile portfolio to provide the best value to the organisation. resource allocation it is about balancing a portfolio through the effective utilisation of resources. (jonas, 2010; kaufmann et al., 2020; kock et al., 2020; meskendahl, 2010; unger et al., 2012) risk assessment it concerns the management and balancing of project risks in a portfolio. (jonas, 2010; kaufmann et al., 2020; kock et al., 2020; teller et al., 2012) team capability building new the criterion measures the agile teams' productivity in an agile portfolio, ensuring that it increases over time. note. *subthemes are not ranked in order. table 4. identified success criteria for agile portfolios sc1. portfolio value maximisation participants referred to achieving the maximum overall value of the portfolio as one sc for ppm through three different subthemes: successful delivery of projects can be achieved by ensuring agile projects’ efficiency. project efficiency refers to the traditional criteria of success, i.e., budget, schedule and quality compliance (serrador & turner, 2015; shenhar & dvir, 2007). we found that agile project efficiency is tracked as a critical contributor to portfolio success in all cases. for example, one participant emphasised the agile portfolio success: “we still track the more traditional success measures. did you do it on time? did you do it on budget? did you deliver on your scope? [sic]” [case -a, ppm]. our findings here are comparable with the traditional ppm research, which also considers average project success over all projects belonging to a portfolio as an sc (kock et al., 2016; meskendahl, 2010). australasian journal of information systems aghajani, ahsan & whiteside 2023, vol 27, research article agility meets a project portfolio 20 benefit for the customers was identified in all cases. this criterion is about assessing the value of a portfolio through the impacts agile projects have on their customers. participants mentioned several customer-related criteria for agile project success, including satisfying customers, fulfilling customer needs, the product's actual use by the customer and solving a customer’s problem. for example, customer satisfaction was considered a portfolio success measure in case a: “i would judge customer satisfaction as the primary driver of success” [case-a, ppm]. alternatively, the adm in case c believed that “if we can solve the sponsor’s problem and bring value to the customer, in that sense, we can say that we have been successful” [case-c, adm]. previous research considers the impact on the customer as an sc for individual projects (serrador & turner, 2015). based on our findings, we argue that an agile portfolio's benefit or impact on customers should be evaluated as an sc. the benefit for the customers can be determined by assessing the overall value delivered to customers in an agile portfolio. increased customer involvement new represents efforts to address increased customer interactions in agile portfolios. customer involvement is perceived as crucial for agile portfolio success and was explicitly mentioned by participants in several cases (cases-a, e and g) as an sc. for example, “a portfolio is there to increase the delivery pace and customer interaction as much as possible” [case-e, ac1]. this sc is unique for the agile portfolios as previous studies have only discussed increased customer involvement for individual agile projects (sheffield & lemétayer, 2013) and have not considered it an agile portfolio sc. sc2. strategic alignment participants repeatedly referred to strategic alignment as one important sc through a single subtheme: strategic alignment denotes the degree to which the projects under a portfolio reflect organisational strategy. all participants mentioned their ppm efforts to align agile projects’ goals and objectives with organisational strategy. for example, “we have a strategy, and then the initiatives linked to the strategy. we have themes which are a level up close [sic] to the strategy and then, initiatives and features...” [case-d, ac]. strategic alignment was considered as adding value to the portfolio business and customers. participants referred to several practices, such as establishing product visions, roadmapping and portfolio backlogs, that helped maintain strategic alignment at the portfolio level. aligning projects with organisational strategy is a key factor in achieving portfolio success, as recognised by both traditional and agile project management literature (kaufmann et al., 2020; stettina & hörz, 2015). sc3. achieving business success participants referred to achieving business success as one sc for ppm through three subthemes: market performance deals with the projects' outcomes and is about the extent to which sales objectives (e.g., sales volume or market share) are achieved by introducing the projects' products to the market. for example, a participant in case b mentioned: “the real thing is: have we solved a customer problem? have we generated value for the customers? if yes, then over time, that enables us to expand our business” [case-b, sm]. market performance is about measuring the actual benefits of projects under a portfolio once they are introduced to the market (killen et al., 2008b; meskendahl, 2010). australasian journal of information systems aghajani, ahsan & whiteside 2023, vol 27, research article agility meets a project portfolio 21 shorter delivery to market new is an agile-specific criterion monitored in agile portfolios. participants mentioned agile ppm attempts to monitor products’ time-to-market. for example, “in the traditional way, you cannot start measuring value until a long time [has passed]. if you are delivering value earlier, you can start gaining that value, actual income and results… one important thing is lead times. from the moment you conceive an idea until the moment you deliver it, the shorter is better” [case-d, ac]. such an sc is rarely discussed in traditional ppm literature. embracing change new represents the efforts of agile portfolios to measure if the changes are accomplished in a multi-project agile environment. traditional ppm studies do not recognise such an sc at the portfolio level. research participants in cases b, f and g believed the number of accomplished changes in an agile portfolio should be considered a measure of success. for example, in case f, the concept of “value drops” was introduced by ppm as an indicator of the number of embraced changes and was measured to ascertain if the agile portfolio can deliver more customer value over cyclical periods. “based on our recommendation to the executive, we believe we could achieve 46 value drops. so, this year, we achieved 49. that is a [portfolio] success” [case-f, pfm]. traditional ppm studies do not recognise embracing change as a portfolio sc. sc4. portfolio balancing this category contains three distinctive subthemes as an agile portfolio sc: resource allocation is about balancing each portfolio through the effective utilisation of resources. there was agreement among all participants concerning the importance of resource balancing in ensuring agile portfolio success. however, participants in cases a, b and c referred to resourcing issues that hindered smooth portfolio delivery. for example, ineffective resource balancing was evident in case b; “sometimes we have too many software engineers and not enough product people, and other times, we have too many designers and not enough software engineers. so, there is definitely a levelling problem that we have from time to time…” [case-b, vp]. we further discuss this criterion along with the risk assessment below. risk assessment addresses managing multiple agile projects’ risks and balancing their impact at the portfolio level. “the risk [management] is very important in ensuring that we deliver what we say. so we do risk assessment to ensure the risks are evaluated for risk rating and determining activation dates. for example, something [a software product] will be out of date today, but it will be horrible in 3 months. because we will not have the patch, it [the service] will be out of support… we have a team to assess the risk, even before a project comes to our backlog” [case-f, psm]. for a portfolio to provide the best value to its organisation, it must be well-balanced across a few essential project-related dimensions, e.g., types, sizes, risk levels and resource demands (abrantes & figueiredo, 2015). a right portfolio balance allows organisations to achieve their objectives without being exposed to unnecessary risk and secure portfolio success (kester et al., 2014; meskendahl, 2010). however, determining what dimension to consider depends on the portfolio context (meskendahl, 2010). we found two dimensions of portfolio balancing (i.e., risk and resources) prevalent in our studied cases. team capability building new was only measured in portfolios of several cases (cases b, f and g). evaluating a team’s capability was perceived as an attempt to increase the agile teams’ productivity over time, which, in the long run, helps organisations achieve higher portfolio success rates. “the whole point of having an enduring portfolio is, if we can do 49 [value drops] australasian journal of information systems aghajani, ahsan & whiteside 2023, vol 27, research article agility meets a project portfolio 22 this year, maybe we can do 65 next year. because the teams will inevitably go faster and get better. so, did i improve the teams’ capabilities? if not, then i have not achieved my [portfolio] goal” [casef, pfm]. building team capability contributes to sustaining agile portfolios by ensuring that the agile teams’ productivity increases over time. such an sc is rarely discussed in ppm literature. 4.2 success factors of agile portfolios rq2 asks about factors contributing to the success of agile isd portfolios in large organisations. data analysis revealed 15 themes (sfs) related to agile portfolios. as shown in figure 3, nine were categorised as c-sfs, and the remaining six were p-sfs. c-sfs contain a range of organisational, cultural and process-related factors, while project-level sfs are project and team related. we also used the u and c dimensions of the uc framework, explained in section 2.4, to analyse the research findings. table 5 shows the clustering of identified sfs based on the u and c contingency categories. sf category sfs related to u ncertainty (u) sfs related to consequences (c) contextual sfs c-sf1. apm-ppm synchronisation c-sf2. top management support c-sf6. appropriate funding model c-sf3. organisational culture c-sf8. it and infrastructure agility c-sf4. communicating agile values c-sf9. flat organisational structure c-sf5. team/customer involvement c-sf7. customer training project-related sfs p-sf1. information availability and quality p-sf3. team autonomy p-sf2. clear project vision p-sf5. team alignment p-sf4. tailoring agile delivery metrics p-sf6. sufficient amount of work table 5. categorisation of identified sfs based on the uc framework we found eight sfs related to uncertainty, complexity or urgency, which we categorise as u sfs. for example, it and infrastructure agility (c-sf6) is a factor related to the complexity arising from the structure of a firm (or its ppm), so it falls under the u category. the remaining seven sfs were related to the consequences, such as the ability of teams to communicate, understand and quickly respond to unexpected events at the project or portfolio level (team empowerment); hence, they were categorised as c sfs. for example, team autonomy (p-sf3) designates the discretionary power given to agile teams and is a team empowerment contingency that falls under the c category. table 6 summarises the identified sfs and supporting references from the agile scale-up and traditional ppm research. in the following, these sfs are presented based on their frequency of appearance in data analysis, with the most frequent appearing first. subthemes description supporting references theme: contextual sfs c-sf1. apm-ppm synchronisation new organisations need to align ppm with agile project delivery. this can be achieved at the portfolio level by implementing and supporting several agile activities, such as portfolio backlogs and stand-ups. for ppm (sweetman & conboy, 2018) c-sf2. top management support senior management engagement and support are found essential for sustaining agile portfolio success. for agile scale-up (dikert et al., 2016; edison et al., 2021; kalenda et al., 2018; rigby et al., 2018; shameem et al., 2018; stettina & hörz, 2015); for traditional ppm australasian journal of information systems aghajani, ahsan & whiteside 2023, vol 27, research article agility meets a project portfolio 23 subthemes description supporting references (beringer et al., 2013; blomquist & müller, 2006; jonas, 2010; jonas et al., 2013; kock et al., 2015; rank et al., 2015; unger et al., 2012) c-sf3. organisational culture important cultural characteristics such as trust, open communication and a no-blame environment facilitate a safe environment for agile teams to openly communicate issues, develop relationships and collaborate on agile projects. such organisational culture contributes to the success of agile portfolios. for agile scale-up (abrar et al., 2021; edison et al., 2021; kalenda et al., 2018); for traditional ppm (kaufmann et al., 2020, 2021; kock & gemünden, 2020; rank et al., 2015; unger et al., 2014) c-sf4. communicating agile values constant communication of agile values influences organisational culture while enhancing communication among agile teams, projects and portfolios. for agile scale-up (dikert et al., 2016; kalenda et al., 2018; rigby et al., 2018) c-sf5. team/customer involvement new agile teams and customers should play an active role in portfolio decision-making. such active involvement enhances communication and collaboration, which eventually promotes portfolio success. for internal stakeholders of traditional ppm (beringer et al., 2013; blomquist & müller, 2006; jonas, 2010; petro & gardiner, 2015; unger et al., 2012) c-sf6. appropriate funding model new milestone-based investment plans might hinder the successful delivery of agile portfolios. more flexible funding models, such as capacity-based funding, can allow better management of multiple agile projects. as partial support for agile scaleup (edison et al., 2021; hoda et al., 2009; rigby et al., 2018) c-sf7. customer training new customers' lack of knowledge and awareness about agile methods limits their ability to collaborate effectively in agile projects. ppm should support and facilitate customer training to increase the chances of agile projects and portfolio success. as partial support for agile scaleup (conboy & carroll, 2019) c-sf8. it and infrastructure agility new legacy it and infrastructure processes constrain successful agile delivery. ppm has a role in removing legacy issues to support the delivery of agile projects. as partial support for agile scaleup (bosch & bosch-sijtsema, 2011) c-sf9. flat organisational structure new highly vertical organisational structures can hinder fast decision-making at the project and portfolio levels and undermine successful agile project delivery. a flatter (organic) ppm structure can better serve agile projects and portfolios. in the form of aligning the organisation in general for agile scale-up (conboy & carroll, 2019; dikert et al., 2016; edison et al., 2021); for agile ppm (sweetman & conboy, 2018) theme: project-level sfs p-sf1. information availability and quality the availability and quality of information gathered from agile projects can contribute to portfolio success through enhanced communication and decision-making. for agile scale-up (dingsøyr et al., 2019; edison et al., 2021; kalenda et al., 2018; shameem et al., 2017); for traditional ppm: (dietrich & lehtonen, 2005; jonas, 2010; jonas et al., 2013; lerch & spieth, 2013; martinsuo & lehtonen, 2007; spieth & lerch, 2014; teller et al., 2012) p-sf2. clear project vision the clarity of project/product vision is an enabling factor for both agile project portfolio success. for agile scale-up: (edison et al., 2021; kalenda et al., 2018; shameem et al., 2017); for traditional ppm: (biedenbach & australasian journal of information systems aghajani, ahsan & whiteside 2023, vol 27, research article agility meets a project portfolio 24 subthemes description supporting references müller, 2012; martinsuo, 2013; martinsuo & lehtonen, 2007) p-sf3. team autonomy new there should be a proper balance between autonomy and control in ppm. for ppm to secure agile portfolio success, agile teams should be empowered to practice selforganisation and autonomy. for agile scale-up: (dikert et al., 2016; edison et al., 2021; shameem et al., 2017); in the form of partial support for agile ppm (sweetman & conboy, 2018) p-sf4. tailoring agile delivery metrics new agile delivery metrics should be tailored for each individual project at the portfolio level. then, the performance and fitness of each agile project should be consistently reported and checked against the pipeline of work. in the form of lack of agile metrics as a challenge for agile scale-up (edison et al., 2021) p-sf5. team alignment an effective team alignment guarantees an optimal balance between project requirements and resource availability, eventually promoting agile portfolio success. for agile scale-up (edison et al., 2021; kalenda et al., 2018; rigby et al., 2018; shameem et al., 2017; šmite et al., 2017); in the form of stakeholder management importance for ppm (dietrich & lehtonen, 2005; jonas, 2010; kaufmann et al., 2020; kock & gemünden, 2020; kopmann et al., 2017; müller et al., 2008; teller, 2013; teller et al., 2014; teller & kock, 2013) p-sf6. sufficient amount of work new agile teams are often regarded as continuous product delivery teams. a continuous and sufficient pipeline of work supports agile teams in finding a consistent and harmonised rhythm of work that increases the likelihood of portfolio success. table 6. identified success factors for agile portfolios 4.2.1 contextual success factors c-sf1. apm-ppm synchronisation new – in the studied cases, adopting agile at scale resulted in adding several agile-specific activities (practices, artefacts and rituals) to the ppm. participants mentioned various agile activities managed at the portfolio level, such as discovery workshops, iterative planning sessions, roadmapping, portfolio backlogs, portfolio walls, portfolio stand-ups and showcases. by performing these activities, organisations tried aligning ppm with agile project delivery. in most cases (cases a, c, d, e and f), participants considered such efforts critical to achieving agile portfolio success. participants also reported imbalances between apm and ppm that, in their view, frustrated agile teams and hindered agile project delivery. these imbalances were mainly related to highly formal and plan-based ppm governance structures. for example, in case a, business cases and project funding models were milestone-based, forcing agile teams to report project changes and progress using traditional project measurement metrics. such action was considered to be time-consuming and unnecessary. “i think we do struggle because our [ppm] framework is still a bit waterfall. we still go through initiatives and plans. the teams tend to be ever more agile, but it is just aligning against the [ppm] framework that is a barrier to success. from a ppm perspective, i think we are not quite aligned yet” [case-a, pfm]. failure to synchronise ppm with agile project delivery will compromise the ability of the agile australasian journal of information systems aghajani, ahsan & whiteside 2023, vol 27, research article agility meets a project portfolio 25 teams to react rapidly to changes in project requirements and business environment and adds to the complexity of managing agile projects at the project level. hence, we considered it a factor related to urgency and complexity (u sf). c-sf2. top management support – across all cases, participants perceived management support as essential for sustaining portfolio success. for example, such support was evident in case a: “within our portfolio, they [top managers] are very much all on the same page for what we need to do to support the teams” [case-a, rte]. continuous executive management commitment and support are crucial in sustaining organisational change (pikkarainen et al., 2012) and setting up suitable and agile-friendly organisational roles (stettina & hörz, 2015). previous research confirms management support as an sf for individual agile projects (sheffield & lemétayer, 2013), traditional portfolios (unger et al., 2012) and agile portfolios (leffingwell, 2007; stettina & hörz, 2015). this sf can be attributed to the organisational structure and culture, so it is a factor related to both complexity and team empowerment. however, we considered it a factor related to consequences (c sf) because of its profound influence on the agile teams’ ability to self-organise and deliver successful products (sheffield & lemétayer, 2013). c-sf3. organisational culture – data analysis revealed several cultural characteristics such as trust, open communication and a no-blame environment contributing to agile portfolio success. participants expressed the importance of ppm in facilitating a healthy and safe environment for agile teams enabling them to openly express their issues, develop relationships, make mistakes and learn from those mistakes. for example, according to one participant, ppm provided such an environment: “i think everyone has to work in concert… i did mention about [sic] trust and giving freedom to teams by trusting them…” [case-f, pfm]. the participants also stressed the importance of managers avoiding micromanagement to foster innovation. for apm to be adopted and sustained, an organisation's culture and agile values must be a good fit. an organisation's culture reflects the perceptions of managers and workers that form the prevailing mentality of the organisation. it reflects the values and beliefs of its staff (schein, 2009). the organisational culture is a highly subjective matter that needs to be contextualised considering the purpose and strategy of the organisation (tolfo et al., 2010). traditional ppm research also recognises facets of culture, such as the innovation culture, as essential for portfolio success (unger et al., 2014). our findings highlight the importance of appropriate organisational culture in achieving agile portfolio success. organisational culture ascertains the discretionary power formally assigned to the teams. also it is an imposed factor that may hinder their ability to use this power effectively (howell et al., 2010), so it is a c sfs. c-sf4. communicating agile values – in all cases, intensive communication of agile values and principles at the portfolio level (and organisation-wide) was evident. participants underscored the importance of communicating the agile way of working; for example, “agile is more about the culture; capability can come with time… the hardest point was to change the culture. to say we are all owning something. if you see that it does not work, it is your problem as well. capability as uplifting the agile knowledge comes if you are keen, willing to understand and accept the change. this is the message that needs to be communicated” [case-f, psm]. communicating agile values influences organisational culture and enhances communication and collaboration across agile projects and their portfolio; thus, it is a team empowerment contingency and a c sf. communication is a key enabler of agile projects australasian journal of information systems aghajani, ahsan & whiteside 2023, vol 27, research article agility meets a project portfolio 26 (mishra et al., 2012). constant communication of agile values is essential in raising awareness of agile at scale. it is more likely for people in an organisation to contribute effectively to promoting the agile way of working when they understand the agile values and mindset (dikert et al., 2016). c-sf5. team/customer involvement new – participants stressed the active role and involvement of agile team members and customers in portfolio decision-making. for instance, in case f, portfolio managers invited team members and customers to weekly portfolio meetings and involved them in making decisions about projects and portfolio issues: “at the portfolio level, we have another weekly meeting. all key project managers and customers are invited. we use these weekly stand-ups to make quick decisions where we can. so, those meetings are not just progress updates. the key difference is that at those meetings, we actually make decisions” [casef, pfm]. team/customer involvement in portfolio decision-making can be attributed to the level of complexity at the portfolio level. however, we argue that its dominant effect is in modifying the agile teams’ and portfolio managers' ability to process and act effectively upon information received, so it can be considered a c sf. c-sf6. appropriate funding model new – in most cases (all except cases b and g), agile portfolios ran under centralised governance structures characterised by annual budgeting cycles and milestone-based business/investment plans. participants identified mismatches between agile delivery outcomes and business/investment plans when project changes surfaced during its execution. research participants proposed capacity-based funding as an appropriate funding model promoting agile projects and portfolio delivery in large organisations. “one way is the capacity funding model, where you fund all the persistent teams… you do it away from the business cases and put more autonomy into the product and business owners to have a fixed funding model, which is capacity-based. then, you prioritise the work into those teams” [case-c, ac]. in our view, the funding model for projects is a contingency influencing project goals, which is a factor related to uncertainty. therefore, we considered it as a u sf. c-sf7. customer training new – a shared view among the participants was that customers need agile training as much as agile teams do. according to the participants, the lack of customer awareness about agile methods impeded active customer collaboration and negatively impacted agile projects and portfolio success. participants in cases a, e and f revealed efforts from agile teams, supported by portfolio managers, to involve customers in agile ceremonies and rituals to train them on the agile way of working. “some [customers] still do not know [agile] very much. so, part of our job is ensuring that all stakeholders we start working with we give them the context of how we work and what is actually done so that they feel comfortable working in that way with us” [case-a, rte]. training customers in agile improves their ability to delineate project goals and requirements (an uncertainty factor) and remove scepticism about the agile way of working (a team empowerment factor) that positively impacts team-customer collaboration (hoda et al., 2011). we considered this factor a c sf. c-sf8. it and infrastructure agility new – participants in cases c, f and g referred to legacy it and infrastructure processes that constrained successful agile delivery. apm generally requires dealing with organisational and it legacy processes and software. sometimes, customer value(s), such as building new product features, must be built on a legacy process. it is also plausible to replace a legacy process with a new one to support agile methods (thomas & baker, 2008). participants believed that one portfolio manager’s job australasian journal of information systems aghajani, ahsan & whiteside 2023, vol 27, research article agility meets a project portfolio 27 was to remove legacy issues to support the delivery of agile projects. however, they also expressed concerns that upgrading or replacing these processes could become challenging, time-consuming and costly. a participant explains a legacy system problem: “setting up the it systems for agility is another biggest [sic] limiting factor. the state of the software in the organisation is a big limiting factor in how agile you can be. i had many organisations over the years that put different [agile] products and systems [sic]. they are not set for agility, so if the business wants to build a new feature, it can be developed digitally. however, it has a dependency on a traditional system that has a very long lead time. that impedes success” [case-c, ac]. we consider this factor a u sf since it is a contingency related to organisational structure that can influence the level of complexity at the portfolio level. as far as we know, it and infrastructure agility is not considered an agile portfolio sf in previous literature. c-sf9. flat organisational structure new – in our study, only two cases (cases b and g) benefited from flat organisational structures also called organic structures (donaldson, 2001). other cases had highly vertical structures. participants in vertical structures repeatedly expressed concerns about structural mismatches that undermine successful agile delivery. these mismatches mainly surfaced when changes in agile projects went through lengthy decision-making processes and were compared with traditional milestone-based business cases for approval. cases with flat organisational structures, on the other hand, were able to facilitate rapid communication channels, easy access to sponsors and speedy decisionmaking at project and portfolio levels. participants had a common belief that a flatter organisational structure better serves agile portfolios. a participant in case-c explains the structure: “in this organisation, there is no sense of hierarchy. there is, but it is very minimal. where things go wrong is when people on the frontline take instruction from the senior people without challenging it. when that happens, it is a learning moment for everybody involved because the senior person gave the instructions [but] failed to deliver a line that says, “i am not an expert; this is just an opinion; you need to make a decision”. plus, the people on the team took that senior person’s opinion as an instruction. if that happens, we consistently have problems. so as a result, we vigorously fight against the behaviour and remind people on the frontline that they are the decision-makers and are all accountable and responsible for the outcome” [case-b, vp]. the cases with flat organisational structures in their agile portfolios reported fewer issues in achieving portfolio success. a flat organisational structure is characterised as a management structure with low functional formalisation where employees exercise autonomy in decision-making and self-organisation in performing their duties (donaldson, 2001). this factor is directly related to organisational complexity and indirectly to team empowerment, so is a u sf. 4.2.2 project-level success factors p-sf1. information availability and quality – all participants believed project information availability and quality are essential for agile portfolio success. participants discussed issues concerning information gathering and project data quality that negatively impacted the management of portfolios. for example, in case f, delays in receiving timely updates, as well as low-quality and outdated information, hindered agile portfolio success by impeding effective decision-making: “organising data sources, mapping all the information and making it like a core is very hard… there are many projects, tools, and information. what are your sources of truth, and what is the cadence of updating those? if you want to have a snapshot of the portfolio, you need to make sure the provided data are accurate. some delivery teams are not australasian journal of information systems aghajani, ahsan & whiteside 2023, vol 27, research article agility meets a project portfolio 28 ready yet…” [case-f, psm]. information quality enhances decision-making quality and portfolio success (dietrich & lehtonen, 2005). therefore, ppm processes must be carefully designed to enable the sharing and communication of high-quality information. information availability and quality can be associated with the method of project delivery, multiple projects’ goals and ppm design that supports effective communication. hence, it is a contingency related to uncertainty and a u sf. p-sf2. clear project vision – all research participants referred to the positive influence of a clear project/product vision in enabling agile projects and portfolio success. participants pointed out several activities relevant to project visioning, such as discovery workshops and roadmapping. project visioning was perceived as a way to align agile projects with portfolio goals and organisational strategy. a participant underscores the importance of project visioning: “i think it [portfolio success] starts with a clear vision and strategy at the leadership level. once we have it, that needs to be translated into portfolios or larger project programs. so, you would have a clear vision and strategy, then a laundry list of projects and processes that underpin that vision” [case-c, ac]. without a clear project vision, the chances of problematic product integration at final testing increase, resulting in extremely costly and time-consuming rework in the end (vlietland & van vliet, 2015). project goal setting is considered a critical sf in traditional ppm (martinsuo & lehtonen, 2007) and agile projects (serrador & pinto, 2015; vlietland & van vliet, 2015). our findings reveal clear project vision as an agile portfolio sf related to uncertainty, which is a u sf. p-sf3. team autonomy new – team autonomy is perceived as “the degree of discretion and independence granted to the team in scheduling the work, determining the procedures and methods to be used, selecting and deploying resources, hiring and firing team members, assigning tasks to team members and carrying out assigned tasks” (lee & xia, 2010, p. 90). all participants emphasised the importance of embracing and facilitating agile teams’ self-organisation and autonomy at the portfolio level. in our participants’ view, ppm must empower agile teams to plan and organise their own work and assist them in making conscious decisions about their projects. supporting autonomy was prevalent in case-a: “they [portfolio managers] honestly just leave us alone. they do not interfere too much with what we do. it is very easy for them to have a lot of red tape around what we do and really make our lives hard, but they do not” [case-a, rte]. the team’s flexibility in making project decisions and managing their workload signifies procedural empowerment, which is a c sf. to our knowledge, supporting team autonomy as an sf for agile portfolios is not mentioned in ppm research. p-sf4. tailoring agile delivery metrics new – in all studied cases, agile teams communicated their progress to ppm based on agile project metrics, such as burnup/burndown charts, velocities, lead times and cycle times – see kupiainen et al. (2015) for various agile project metrics. for example, the use of team velocity and burn up/down charts were common across all agile teams in all cases. data analysis also revealed a tendency to tailor agile metrics for individual projects under portfolios, especially during the early stages of the project conception. “at an early stage, once we have done the initial discovery or screening phase, we set up some criteria, and it is going to be slightly different for each project… we rather measure success on tangible outcomes” [case-f, pfm]. research participants believed that the performance and fitness of each agile project should be consistently reported and checked against the pipeline of work at the portfolio level. australasian journal of information systems aghajani, ahsan & whiteside 2023, vol 27, research article agility meets a project portfolio 29 another finding is related to using traditional portfolio metrics. the use of traditional project planning and risk control metrics was still prevalent in most cases (all except cases b and g). in most cases, portfolio managers were still preparing plan-driven project status reports based on data collected from agile teams. in other words, portfolio managers translated the agile status reports into milestone-based reports for further use by senior management. the coexistence of traditional and agile metrics was non-value-adding, timeconsuming and costly and was considered a waste of resources by several research participants. “i think [portfolio] metrics need to evolve to take into account your working and delivering agile stuff [sic]. we need to move away from some of the more traditional metrics we used to use… they [senior management] need to be familiar with the artefacts we are using. you may not need a full-blown project management plan anymore” [case-a, rte]. while this research does not provide specific recommendations for using specific type of metrics at the project or portfolio level, we highlight the importance of adopting appropriate agile delivery metrics in the context of agile portfolio management. tailoring agile delivery metrics helps establish appropriate project and portfolio goals and objectives; therefore it is a factor of uncertainty and we considered it as a u sf. at the project level, appropriate metrics provide a reliable and accurate means of measuring the success of agile practices and identify areas that require improvement. the lack of appropriate metrics at the portfolio level can lead to poor decision making and project outcomes. hence, it is imperative that organisations recognise the need to implement relevant metrics at project and portfolio levels to ensure the alignment of agile projects with organisational goals. p-sf5. team alignment – multiple agile projects generate substantial changes, producing sequences of cascading effects on their portfolio, thus demanding ever-changing decisions. the momentum created by these changes leads to resource conflicts, as all projects compete for limited resources (primarily human resources). as signified by the participants, a continuous and effective team alignment guarantees an optimal balance between project requirements and resource availability that eventually promotes agile portfolio success. effective resource (team) allocation and alignment at the portfolio level were considered vital in ensuring agile team performance, finding a steady rhythm of work and achieving portfolio goals: “it [resource planning] is very systemic. in that, we staff teams on an annual budget. we make minor corrections during the year, but otherwise, that is [a] steady state. we do not change team sizes. there are only strategic moments where we will spin up a new team” [caseb, vp]. patanakul and milosevic (2009) suggest that managing resources for individual projects should not be done in isolation. instead, ppm requires a systematic and proactive approach to managing resources. effective resource management is found as a critical sf for traditional ppm (abrantes & figueiredo, 2015). an effective team alignment can support agile teams in achieving cross-functionality that boosts team performance (hoda & murugesan, 2016) and influences the success of agile projects and portfolios. team alignment is a contingency related to the team composition and capability (howell et al., 2010), so we considered it a c sf. p-sf6. sufficient amount of work new – agile teams were considered continuous product delivery teams in all studied cases. cases a, c and e participants mentioned portfolio managers’ efforts to feed portfolio pipelines with enough projects. there was a common belief among research participants that a continuous and sufficient pipeline of work supports agile australasian journal of information systems aghajani, ahsan & whiteside 2023, vol 27, research article agility meets a project portfolio 30 teams in finding a consistent and harmonised rhythm of work that increases the likelihood of portfolio success: “it costs money to keep each agile team running, so i have to have a backlog of work ready to go. if i do not have that backlog or approval somewhere or some business cases are sitting on my desk, i lose [teams’] efficiency” [case-a, pfm]. this factor can be attributed to external factors (e.g., availability of projects in the business market) or internal factors (such as the need and extent of change in the organisation), both conditions related to uncertainty. therefore, we considered this factor as a u sf. such sf is rarely discussed in previous research. 5 discussion of findings 5.1 what constitutes agile portfolio success? the rq1 asked about the criteria of success in portfolios of agile isd projects in large organisations. to clarify the notion of success, we identified four scs for portfolios of agile isd projects in large organisations: portfolio value maximisation, strategic alignment, achieving business success and portfolio balancing. these scs comprise ten subcriteria; among them, four are new and exclusive to agile isd portfolios (table 4). these new success criteria are increased customer involvement, shorter delivery to market, embracing changes and team capability building. in the following, we discuss the new scs of agile isd portfolios in light of extant literature. increase customer involvement – quickly responding to customer needs and ensuring customer involvement are paramount to agile project success (misra et al., 2009; sheffield & lemétayer, 2013). agile projects strive for high customer satisfaction (serrador & pinto, 2015). agile teams’ increased focus on customers brings complexities to the portfolio level. a task of portfolio managers would be flexibly adapting to and managing the tensions between customers’ needs and organisational strategies (sweetman & conboy, 2018). research participants considered increasing customer involvement as one ppm objective and a measure of portfolio success. increasing customer involvement at the portfolio level improves customer interactions with agile teams and ultimately benefits the agile projects and, therefore, the portfolio. shorter delivery to market – one crucial goal of an agile project is to reduce the product’s time to market (cockburn & highsmith, 2001). rapid and incremental product launch to the market enables a faster reaction to market changes and customer demands, allowing organisations to create revenue faster and giving them a competitive edge over competitors in the long run (cockburn & highsmith, 2001). the shorter development time for projects is also considered essential to secure a consistent pipeline of work for project portfolios (biedenbach & müller, 2012). portfolios dealing with agile projects increasingly need shorter delivery cycles to adapt to rapid changes in the business environment and customer demands (suomalainen et al., 2015). embracing change – such a criterion is considered an inherent goal of any agile project emphasised by the agile manifesto of software development (beck et al., 2001). therefore, for portfolios of agile projects, it is logical to consider and measure metrics related to embracing changes. among the studied cases, we observed a tendency to quantify the number of changes embraced (accomplished) by multiple agile projects. then, the number of changes embraced was used as a metric/benchmark to track the portfolio performance during portfolio review cycles. australasian journal of information systems aghajani, ahsan & whiteside 2023, vol 27, research article agility meets a project portfolio 31 team capability building – we found portfolio managers' concerns and attempts to measure and improve agile teams’ capability over time as an sc for agile portfolios. for example, management in case f introduced an indicator ‘value drop’ to measure the number of embraced changes which was also perceived as an indication of team capability building. recent research points to the importance of team capability building for agile projects and portfolio success. for example, kaufmann et al. (2020) studied the positive link between agile capabilities and emerging strategy recognition, and the latter is positively and significantly related to portfolio performance (kopmann et al., 2017). agile team capabilities are determined by the intensity of apm use and the team’s competence in applying agile practices (kaufmann et al., 2020). also, a recent study by bechtel et al. (2021) demonstrates the positive influence of ppm practices and innovation culture on agile teamwork quality (enhanced communication, cooperation, support and commitment). however, such an agile portfolio sc is rarely discussed in previous literature. 5.2 what factors contribute to agile portfolio success? the rq2 asked about the factors contributing to the success of agile isd portfolios in large organisations. we identified 15 sfs for agile isd portfolios and categorised them into nine contextual sfs (c-sfs) and six project-level sfs (p-sfs), see figure 3. the c-sfs are contingency factors related to agile portfolio environment, which are associated with the organisational structure and culture affecting portfolio success. we considered the p-sfs as contingencies related to individual agile projects and teams as portfolio subunits influencing agile portfolio success. hence, the study proposes several new sfs contextual to the isd portfolio as well as at the agile project level in large organisations. apm-ppm synchronisation, team/customer involvement, appropriate funding model, customer training, it and infrastructure agility, and flat organisational structure are identified as new c-sfs. also, team autonomy, tailoring agile delivery metrics and sufficient amount of work are suggested as new p-sfs. in the following, we discuss the new findings in light of previous literature and explore the effect of u and c sfs on determining appropriate portfolio structure. 5.2.1 new contextual success factors c-sf1. apm-ppm synchronisation – this factor was identified as one key sf related to agile portfolio success. the ppm function in an organisation is to ensure projects have successfully delivered value and their outcomes align with business strategy (martinsuo & killen, 2014). the success of projects as portfolio components is crucial for portfolio success. at the same time, the ability to change is fundamental to the concept of agility at the project level. agile projects constantly evolve in response to the projects’ changing requirements and environments (schwaber, 2004). therefore, ppm must fit the dynamic environment under which multiple agile projects constantly evolve (sweetman & conboy, 2018). dynamic environments require fast responses to change that can be achieved through flexible organisational structures (donaldson, 2001). therefore, a flexible and adaptable ppm structure is expected to enable agile project delivery. it is also essential for ppm to adapt to several apm characteristics, such as high levels of team/customer interaction and team self-organisation and autonomy (sweetman & conboy, 2018). portfolio managers, in the studied cases, made efforts to adapt ppm to apm by integrating (or supporting) various agile scale-up activities, including requirement discovery workshops, iterative planning sessions, roadmapping, portfolio backlogs, portfolio walls, portfolio stand-ups and showcases. also, research participants showed awareness of the australasian journal of information systems aghajani, ahsan & whiteside 2023, vol 27, research article agility meets a project portfolio 32 need to measure and aggregate multiproject performances at the portfolio level. while the importance of agile scale-up activities is discussed in previous literature (kalenda et al., 2018; olszewska et al., 2016), apm-ppm synchronisation is rarely discussed as a critical sf for agile project portfolios. c-sf5. team/customer involvement – isd projects generally require the involvement of many stakeholders across multiple organisational disciplines (dingsøyr et al., 2019). when multiple agile isd projects are managed under a portfolio, the intensified team and customer interactions are aggregated at the portfolio level. such aggregated interactions bring tensions and complexities to the portfolio level that needs to be coordinated and managed by ppm to ensure portfolio success (sweetman & conboy, 2018). team/ customer involvement in ppm brings visibility to agile projects and portfolio decision-making. previous literature suggests “engaging everyone” as one sf for large-scale agile scale-up efforts (dikert et al., 2016, p. 102). traditional ppm research also stresses the importance of internal stakeholders' involvement in ppm decision-making (beringer et al., 2013). we identified that involving external stakeholders (such as customers) in ppm processes is crucial and innovative for agile portfolio success. c-sf6. appropriate funding model – our study suggests that a capacity-based funding model is more likely to promote agile portfolio success. capacity-based funding distributes available funding across persistent, autonomous teams. hence, funding is allocated based on the teams’ delivery capacity and the resources required to deliver the project results’ results rather than projects. then, the agile teams prioritise work within the iterations and estimate the hours required to commit to set activities. highly plan-driven business cases and fixed-term contracts are found to be incompatible with agile project settings (cao et al., 2013; hoda et al., 2009). these forms of funding are likely to impede the adaptability required to manage multiple projects at the portfolio level. more flexible types of contracts, such as negotiable scope and pay-as-you-go, are proposed for agile projects (hoda et al., 2009). capacity-based funding allocates funds across agile teams based on their delivery capacity and resource needs rather than the projects’ business cases and initial plans. c-sf7. customer training – effective stakeholder involvement is essential for the success of agile projects (bass & haxby, 2019; kasauli et al., 2021; misra et al., 2009) and is for agile portfolios. however, it is not uncommon for stakeholders to lack the essential knowledge of agile methods and mindset or are against them. customer involvement at the project level can become more effective when stakeholders (especially customers) who are sceptical about agile's effectiveness or lack agile knowledge are trained. the need for customer training is intensified when dealing with multiple agile projects in portfolios of large organisations with many stakeholders. the research participants emphasised the significant role of ppm in identifying and assessing the need for customer training and facilitating it. while much has been said about the value of training and coaching for agile teams (conboy et al., 2011; gandomani et al., 2015), customer training is rarely addressed in agile and traditional ppm studies. c-sf8. it and infrastructure agility – we found evidence of legacy it and infrastructure processes that constrained successful agile delivery within the studied cases (e.g., case c). the legacy processes often resonate with the legacy mindset, which is at odds with the agile way of working (kalenda et al., 2018). examples of likely legacy mindsets are upfront planning and order-taking. upfront planning is the desire to think through everything australasian journal of information systems aghajani, ahsan & whiteside 2023, vol 27, research article agility meets a project portfolio 33 before starting the project, and order-taking is about waiting for orders and directions to perform a task (thomas & baker, 2008). failure to address legacy issues and their underlying mindset generally leads to heavy top-down processes, ineffective resources and a lack of predictability and engagement (bosch & bosch-sijtsema, 2011). hence, assessing legacy it and infrastructure are required before adopting agile at scale and in ppm. failure to address legacy issues might lead to the cancellation or abandonment of agile scale-up efforts. c-sf9. flat organisational structure – large organisations require careful consideration of this contextual sf when dealing with agile portfolios. the cases with organic structures in their agile portfolios (such as cases b and g) reported fewer issues in achieving success. these issues were related to inconsistencies between organisational (and portfolio structures) with agile project delivery that impedes effective coordination and communication. when scaling agile in vertical structures, agile teams' flexibility, autonomy and self-organisation are at odds with well-established traditional processes and routines of the organisation, causing issues for successful agile delivery (sweetman & conboy, 2018). on the other hand, flat organisational structures, i.e., organic structures, are characterised by flexible operating styles, open flow of information, less management control, more individual autonomy and better adaptation to changing circumstances (donaldson, 2001; slevin & covin, 1990). an organic structure enhances coordination and communication and provides better access to organisational resources, stakeholders and leadership (donaldson, 2001). flexible self-organising structures can also respond quickly to the dynamic environments in which agile projects and portfolios operate. based on our findings, we suggest that a flat (organic) portfolio structure can better adapt to unstable environments and requirements uncertainties associated with multiple agile projects, increasing the chances of agile portfolio success. 5.2.2 new project-level success factors p-sf3. team autonomy – based on our findings, we proposition facilitating and supporting the project-level team autonomy at the portfolio level. such facilitation allows development teams to manage their workload and practices, creates a sense of commitment, improves inter-team collaboration and empowers them to take ownership of the project delivery process. conversely, portfolio managers' excessive control and micromanagement might demotivate teams and reduce productivity. sweetman and conboy (2018) argue that topdown control in an agile portfolio should be limited, and it is essential that “the portfolio purpose is created and shaped by both portfolio managers and individual agile projects” (2018, p. 28). ppm should provide a favourable environment for self-organisation to emerge while making teams collectively accountable for agile projects’ outcomes. the importance of team autonomy for effective delivery of agile projects (hoda et al., 2011, 2013; hoda & murugesan, 2016) and for achieving software development agility (sheffield & lemétayer, 2013) has been emphasised before. team autonomy creates commitment to agile scale-up efforts and motivates teams to take ownership of their project delivery efforts (dikert et al., 2016). balancing autonomy with control is also reported as an enabler of several agile scale-up frameworks, e.g., the spotify model (edison et al., 2021). p-sf4. tailoring agile delivery metrics – our research participants recommended careful consideration of agile-specific metrics for each individual project under an agile portfolio. such an action allows for smooth monitoring and control of agile projects’ performances, australasian journal of information systems aghajani, ahsan & whiteside 2023, vol 27, research article agility meets a project portfolio 34 ensuring strategic alignment is achieved. according to the research participants, these metrics may include burn-down charts, velocities, lead times and cycle times, as well as metrics related to the product (software) performance and team productivity. in most cases, portfolio managers attempted to aggregate agile project metrics into plan-driven and milestone-based project status reports for further use by upper management. most research participants perceived such an attempt as non-value-adding, time-consuming and costly. the research participants also recognised the importance of tracking project performance and alignment across various agile-specific portfolio metrics. more research is warranted to determine what portfolio metrics to use for agile portfolios. tailoring agile delivery metrics is rarely discussed in agile project and portfolio literature. while traditional ppm research agrees on the importance of setting up appropriate project-level metrics, there is little agreement on what specific metric to choose, primarily when ppm deals with innovation projects (killen et al., 2008b). also, determining meaningful project metrics for performance review and improvement of agile projects is identified as a vital challenge for agile scale-up frameworks (edison et al., 2021). most of these frameworks have not provided clear recommendations for selecting appropriate project delivery and portfolio metrics (edison et al., 2021). p-sf6. sufficient amount of work – to our knowledge, such a necessary sf is rarely discussed in previous research. previous agile literature refers to several workload challenges, such as overloaded teams and teams committing to unreasonable work (dikert et al., 2016). too much workload results in agile teams losing focus and productivity (lee & xia, 2010). on the other hand, in our participants' views, not having enough projects in the portfolio pipeline impedes continuous agile delivery, destroys the consistent rhythm of work, decreases team cohesiveness and alignment, and eventually forces the organisation to disband agile teams. 5.2.3 the effect of uncertainty and consequences sfs as explained in section 4.2 and summarised in table 5, the identified sfs were either related to uncertainty, complexity and urgency, categorised as u sfs, or were linked to team empowerment and criticality, which we called them c sfs. also, in section 2.4, we argued that the uc framework could be used to identify which process models (i.e., agile or plan-driven) are suitable for managing a particular project or portfolios of multiple projects. this section describes how classifying sfs into u and c categories can help structure an organisation's ppm. according to howell et al. (2010), the u and c dimensions are orthogonal. therefore, it is possible to select a process model based on whether u dominates c or vice versa. the dominance of u factors over c factors means that the degree of uncertainty, complexity and urgency in a multiproject environment is high, so an agile portfolio structure can be selected. for example, in such a case, the organisation can opt to select a flat organisational structure (c-sf9), work toward changing to an agile-friendly organisational culture (c-sf3) by constantly communicating agile values (c-sf4) and involving team and customers in portfolio-decision making (c-sf5). also, the ppm should authorise iterative and incremental project delivery (apm) by encouraging project-level autonomy (p-sf3) and carefully tailoring project delivery metrics (p-sf4). conversely, when usfs are not dominant or non-existing, a plan-driven and milestone-based portfolio structure can be applied. therefore, portfolio and australasian journal of information systems aghajani, ahsan & whiteside 2023, vol 27, research article agility meets a project portfolio 35 project managers should focus on the upfront planning of multiple projects (i.e., employing waterfall methods) and seek optimal sequencing of activities and resources. 5.3 research implications 5.3.1 theoretical contribution the identified agile portfolio scs, sfs and agile portfolio success framework contribute to the theoretical conceptualisation, applicability and clarity of adopting agile methods at scale. the identified agile portfolio sfs can further enhance our knowledge and understanding of agile isd portfolios in large organisations. previous research narrates success stories of agile portfolios but provides little empirical evaluation of such portfolios in practice (stettina & hörz, 2015; sweetman & conboy, 2018). while the adoption of apm in portfolios of large organisations is increasing, empirical investigation on the usefulness and efficacy of these methods at scale is still very scarce (conboy & carroll, 2019). hence, by introducing contextual and project-level sfs (c-sfs and p-sfs), this study contributes to understanding the portfolios of agile isd projects and the conditions under which these portfolios achieve success. we have identified several scs that determined agile portfolio success in case organisations. we argue that if success is sought for an agile portfolio, it is essential to carefully address the identified sc by setting up appropriate metrics, as well as considering the set of identified c-sfs and psfs when scaling up to agile portfolios. the research findings emphasise the importance of context as well as portfolio stakeholders in designing and managing agile portfolios. hence, this study suggests that agile portfolio processes and practices should be carefully tailored to fit the organisation's working conditions and environment, considering the agile portfolio scs and sfs. for example, ppm processes can be tailored to assist project teams in achieving clear project goals by implementing and supporting project discovery workshops and involving portfolio stakeholders (e.g., customers and teams) in ppm decision-making. determining what agile projects and portfolio success mean to stakeholders is critical in defining explicit agile portfolio goals and scs. 5.3.2 managerial implications the research findings have several managerial implications. the proposed agile portfolio success framework and the u and c categorisation of sfs can guide large organisations when adapting and sustaining agile methods at scale and in portfolios of isd projects. for example, an agile portfolio structure can be appropriate if the organisation’s assessment determines the dominance or existence of multiple u sfs for a multiproject environment. in such a case, an organisation can think of ways to synchronise ppm with the management of agile projects and make portfolio managers responsible for advocating and communicating agile values and mindsets across projects and the rest of the organisation. besides, for the success of an agile portfolio, it is essential to tailor delivery metrics for each individual agile project and find appropriate ways to aggregate these metrics for portfolio reporting. also, broader team empowerment initiatives such as team autonomy and alignment are important and should be considered and supported by ppm. checklists or metrics could be designed based on the proposed sfs to ensure the alignment of different organisational components (e.g., strategy and structure) with the identified sfs. measuring performance based on these checklists or metrics could help organisations assess the effectiveness of performed actions and practices australasian journal of information systems aghajani, ahsan & whiteside 2023, vol 27, research article agility meets a project portfolio 36 during and after agile adoption at scale and the portfolio level. organisations can then adjust their team and project agility efforts to support portfolio management. 5.4 limitations while this research opens a new direction in terms of the adaptiveness of ppm to agile project environments, it also has limitations. the research participants were all experienced and knowledgeable in their fields; however, their standpoints might not fully represent their organisations. although the cases were carefully selected, and we employed a rigorous method to ensure the research process and data validity, the findings are only based on cases and a limited number of participants from agile isd portfolios in australia, which may limit the general applicability of the findings. safe was mentioned in three cases (cases a, d and e) as the agile scale-up framework, while the rest did not refer to any specific scale-up framework. although the framework developed in this study is generic, it may be less relevant to organisations and practitioners using other agile scale-up frameworks. when developing the study’s framework, we assumed that all identified sfs are linked to agile portfolio success. however, the study was limited in showing the significance of the relationship between sfs and the identified scs. also, we did not investigate the dependence and power relationships between the identified sfs, which can be explored by future research. finally, our study was limited in identifying appropriate agile portfolio metrics based on the identified agile portfolio scs. future research can examine appropriate portfolio level metrics for ppm. 6 conclusion the popularity of agile methods in large organisations has increased over the past decade. however, a thorough understanding of adopting agile at scale and its impact on agile portfolio success is yet to be sought. this study empirically explores the notion of success in portfolios of agile isd projects to determine its scs and related sfs. based on our findings, we propose that when agile isd projects are managed under portfolio(s), traditional sfs are insufficient to secure portfolio success. investigating seven large organisations, we identified four agile portfolio scs (with ten subcriteria) and 15 sfs, integrating them into an agile portfolio success framework (figure 3). the framework extends our understanding of agile portfolio success in large organisations. we argue that to achieve success in portfolios of agile isd projects, large organisations should optimally fit several organisational characteristics (e.g., structure and strategy) with multiple contingencies (sfs) identified in this study. the agile portfolio success framework can guide large organisations through adapting and sustaining agile methods at scale and in their portfolios of agile isd projects. future work is needed to confirm the efficacy of the study’s framework using additional empirical tests. in particular, future research can utilise a broader sample size to corroborate the findings or can extend the findings to other contexts (e.g., agile projects in non-isd environments or organisations with other types of agile scale-up frameworks). in addition, although our study explores sfs as contingency factors, a company’s ppm design is not exogenous and depends on external environmental conditions. previous studies on ppm have investigated the importance of agility to portfolio decision-making and success in turbulent external environments (kester et al., 2014; kock & gemünden, 2020), but future research could consider the fit between our identified sfs and portfolio-level agility. finally, investigating the interdependencies between the identified sfs and determining the dependence and power relationships between sfs could potentially complement this research. australasian journal of information systems aghajani, ahsan & whiteside 2023, vol 27, research article agility meets a project portfolio 37 references abrantes, r., & figueiredo, j. (2015). resource management process framework for dynamic npd portfolios. international journal of project management, 33(6), 1274–1288. https://doi.org/http://dx.doi.org/10.1016/j.ijproman.2015.03.012 abrar, m. f., ali, s., majeed, m. f., khan, s., khan, m., ullah, h., khan, m. a., baseer, s., & asshad, m. (2021). a framework for modeling structural association among demotivators of scaling agile. journal of software: evolution and process, 33(8), 1–19. https://doi.org/10.1002/smr.2366 agile-business. (2017). agile portfolio management ( agilepfm ) for the agile-mature organisation. agile business consortium. https://www.agilebusiness.org/ barratt, m., choi, t. y., & li, m. (2011). qualitative case studies in operations management: trends, research outcomes, and future research implications. journal of operations management, 29(4), 329–342. https://doi.org/10.1016/j.jom.2010.06.002 bass, j. m., & haxby, a. (2019). tailoring product ownership in large-scale agile projects. ieee software, 58–62. doi: https://doi.org/10.1109/ms.2018.2885524cha bechtel, j., kaufmann, c., & kock, a. (2021). agile projects in nonagile portfolios: how project portfolio contingencies constrain agile projects’ teamwork quality. ieee transactions on engineering management, 1–15. https://doi.org/10.1109/tem.2021.3130068 beck, k., beedle, m., bennekum, a. van, cockburn, a., cunningham, w., fowler, m., grenning, j., highsmith, j., hunt, a., jeffries, r., kern, j., marick, b., martin, r. c., mellor, s., schwaber, k., sutherland, j., & thomas, d. (2001). manifesto for agile software development. http://www.agilemanifesto.org/ beecham, s., clear, t., lal, r., & noll, j. (2021). do scaling agile frameworks address global software development risks? an empirical study. journal of systems and software, 171, 110823. https://doi.org/10.1016/j.jss.2020.110823 beringer, c., jonas, d., & kock, a. (2013). behavior of internal stakeholders in project portfolio management and its impact on success. international journal of project management, 31(6), 830–846. https://doi.org/10.1016/j.ijproman.2012.11.006 biedenbach, t., & müller, r. (2012). absorptive, innovative and adaptive capabilities and their impact on project and project portfolio performance. international journal of project management, 30(5), 621–635. https://doi.org/10.1016/j.ijproman.2012.01.016 bjarnason, e., gislason bern, b., & svedberg, l. (2022). inter-team communication in largescale co-located software engineering: a case study. empirical software engineering, 27(2). https://doi.org/10.1007/s10664-021-10027-z blomquist, t., & müller, r. (2006). practices, roles, and responsibilities of middle managers in program and portfolio management. project management journal, 37(1), 52–66. https://doi.org/10.1177/875697280603700105 bosch, j., & bosch-sijtsema, p. m. (2011). introducing agile customer-centered development in a legacy software product line. software: practice and experience, 41(8), 871–882. https://doi.org/10.1002/spe.1063 australasian journal of information systems aghajani, ahsan & whiteside 2023, vol 27, research article agility meets a project portfolio 38 cao, l., mohan, k., ramesh, b., & sarkar, s. (2013). adapting funding processes for agile it projects: an empirical investigation. european journal of information systems, 22(2), 191– 205. https://doi.org/10.1057/ejis.2012.9 castillo-montoya, m. (2016). preparing for interview research : the interview protocol refinement framework. the qualitative report, 21(5), 811–831. chow, t., & cao, d.-b. (2008). a survey study of critical success factors in agile software projects. journal of systems and software, 81(6), 961–971. https://doi.org/10.1016/j.jss.2007.08.020 cockburn, a., & highsmith, j. (2001). agile software development: the people factor. computer, 11, 131–133. conboy, k. (2009). agility from first principles: reconstructing the concept of agility in information systems development. information systems research, 20(3), 329-354,478. https://doi.org/10.1287/isre.1090.0236 conboy, k., & carroll, n. (2019). implementing large-scale agile frameworks: challenges and recommendations. ieee software, march/april 1–9. https://doi.org/10.1109/ms.2018.2884865 conboy, k., coyle, s., wang, x., & pikkarainen, m. (2011). people over process: key challenges in agile development. ieee software, 28(4), 48–57. https://doi.org/10.1109/ms.2010.132 cooper, r. g., edgett, s. j., & kleinschmidt, e. j. (1999). new product portfolio management: practices and performance. journal of product innovation management, 16, 333–351. https://doi.org/10.1111/1540-5885.1640333 cooper, r. g., edgett, s. j., & kleinschmidt, e. j. (2002). portfolio management: fundamental to new product success. in p. belliveau, a. griffin, & s. somermeyer (eds.), the pdma tool book 1 for new product development (vol. 9, pp. 331–364). john wiley & sons. dietrich, p., & lehtonen, p. (2005). successful management of strategic intentions through multiple projects – reflections from empirical study. international journal of project management, 23(5), 386–391. https://doi.org/10.1016/j.ijproman.2005.03.002 digital.ai. (2021). 15th state of the agile report agile adoption accelerates across enterprise. in annual report for the state of agile. https://stateofagile.com dikert, k., paasivaara, m., & lassenius, c. (2016). challenges and success factors for largescale agile transformations: a systematic literature review. journal of systems and software, 119, 87–108. https://doi.org/10.1016/j.jss.2016.06.013 dingsøyr, t., fægri, t. e., & itkonen, j. (2014). what is large in large-scale? a taxonomy of scale for agile software development. in proceedings of the international conference on productfocused software process improvement (pp. 273–276). cham, switzerland, springer international publishing. dingsøyr, t., falessi, d., & power, k. (2019). agile development at scale: the next frontier. ieee software, 36(2), 30–38. https://doi.org/10.1109/ms.2018.2884884 dingsøyr, t., & moe, n. b. (2014). towards principles of large-scale agile development: a summary of the workshop at xp2014 and a revised research agenda. international australasian journal of information systems aghajani, ahsan & whiteside 2023, vol 27, research article agility meets a project portfolio 39 conference on agile software development, 1–8, rome italy. https://doi.org/10.1007/978-3319-14358-3 dingsøyr, t., moe, n. b., fægri, t. e., & seim, e. a. (2017). exploring software development at the very large-scale: a revelatory case study and research agenda for agile method adaptation. empirical software engineering, 1–31. https://doi.org/10.1007/s10664-017-95242 donaldson, l. (2001). the contingency theory of organizations (1st ed.). thousand oaks, ca, usa, sage publications inc. ebert, c., & paasivaara, m. (2017). scaling agile. ieee software, 34(6), 98–103. https://doi.org/10.1109/ms.2017.4121226 edison, h., wang, x., & conboy, k. (2021). comparing methods for large-scale agile software development: a systematic literature review. ieee transactions on software engineering. https://doi.org/10.1109/tse.2021.3069039 eisenhardt, k. m. (1989). building theories from case study research. academy of management review, 14(4), 532–550. https://doi.org/10.5465/amr.1989.4308385 eisenhardt, k. m., & graebner, m. e. (2007). theory building from cases: opportunities and challenges. academy of management journal, 50(1), 25–32. https://doi.org/10.5465/amj.2007.24160888 etikan, i., musa, s. a., & alkassim, r. s. (2016). comparison of convenience sampling and purposive sampling. american journal of theoretical and applied statistics, 5(1), 1–4. https://doi.org/10.11648/j.ajtas.20160501.11 fortune, j., & white, d. (2006). framing of project critical success factors by a systems model. international journal of project management, 24(1), 53–65. https://doi.org/10.1016/j.ijproman.2005.07.004 gandomani, t. j., zulzalil, h., ghani, a. a. a., sultan, a. b. md., & parizi, r. m. (2015). the impact of inadequate and dysfunctional training on agile transformation process: a grounded theory study. information and software technology, 57, 295–309. https://doi.org/10.1016/j.infsof.2014.05.011 gemino, a., horner reich, b., & serrador, p. m. (2021). agile, traditional, and hybrid approaches to project success: is hybrid a poor second choice? project management journal, 52(2), 161–175. https://doi.org/10.1177/8756972820973082 geraghty, t. (2020). a short critique of safe – the scaled agile framework. tomgeraghty.co.uk. https://tomgeraghty.co.uk/index.php/a-short-critique-of-safe/ hansen, l. k., & svejvig, p. (2022). seven decades of project portfolio management research (1950–2019) and perspectives for the future. project management journal, 53(3), 277–294. https://doi.org/10.1177/87569728221089537 herman, m. (2017). introducing enterprise scrum for business agility: scale scrum from single teams to whole organizations. https://michaelherman.com/publications/agile/enterprisescrumintro_v4.0.pdf australasian journal of information systems aghajani, ahsan & whiteside 2023, vol 27, research article agility meets a project portfolio 40 hoda, r., & murugesan, l. k. (2016). multi-level agile project management challenges: a selforganizing team perspective. journal of systems and software, 117, 245–257. https://doi.org/10.1016/j.jss.2016.02.049 hoda, r., noble, j., & marshall, s. (2009). negotiating contracts for agile projects: a practical perspective. international conference on agile processes and extreme programming in software engineering, may, 186–191. https://doi.org/10.1007/978-3-319-08581-4 hoda, r., noble, j., & marshall, s. (2011). the impact of inadequate customer collaboration on self-organizing agile teams. information and software technology, 53(5), 521–534. https://doi.org/10.1016/j.infsof.2010.10.009 hoda, r., noble, j., & marshall, s. (2013). self-organizing roles on agile software development teams. ieee transactions on software engineering, 39(2), 422–444. https://doi.org/10.1109/tse.2012.30 hoffmann, d., ahlemann, f., & reining, s. (2020). reconciling alignment, efficiency, and agility in it project portfolio management: recommendations based on a revelatory case study. international journal of project management, 38(2), 124–136. https://doi.org/10.1016/j.ijproman.2020.01.004 howell, d., windahl, c., & seidel, r. (2010). a project contingency framework based on uncertainty and its consequences. international journal of project management, 28(3), 256– 264. https://doi.org/10.1016/j.ijproman.2009.06.002 jonas, d. (2010). empowering project portfolio managers: how management involvement impacts project portfolio management performance. international journal of project management, 28(8), 818–831. https://doi.org/10.1016/j.ijproman.2010.07.002 jonas, d., kock, a., & gemünden, h. g. (2013). predicting project portfolio success by measuring management quality a longitudinal study. ieee transactions on engineering management, 60(2), 215–226. https://doi.org/10.1109/tem.2012.2200041 jørgensen, m. (2019). relationships between project size, agile practices, and successful software development: results and analysis. ieee software, 36(2), 39–43. https://doi.org/10.1109/ms.2018.2884863 kalenda, m., hyna, p., & rossi, b. (2018). scaling agile in large organizations: practices, challenges, and success factors. journal of software: evolution and process, online, e1954. https://doi.org/10.1002/smr.1954 kasauli, r., knauss, e., horkoff, j., liebel, g., & de oliveira neto, f. g. (2021). requirements engineering challenges and practices in large-scale agile system development. journal of systems and software, 172, 110851. https://doi.org/10.1016/j.jss.2020.110851 kaufmann, c., kock, a., & gemünden, h. g. (2020). emerging strategy recognition in agile portfolios. international journal of project management, january. https://doi.org/10.1016/j.ijproman.2020.01.002 kaufmann, c., kock, a., & gemünden, h. g. (2021). strategic and cultural contexts of real options reasoning in innovation portfolios. journal of product innovation management, 38(3), 334–354. https://doi.org/10.1111/jpim.12566 australasian journal of information systems aghajani, ahsan & whiteside 2023, vol 27, research article agility meets a project portfolio 41 kester, l., hultink, e. j., & griffin, a. (2014). an empirical investigation of the antecedents and outcomes of npd portfolio success. journal of product innovation management, 31(6), 1199–1213. https://doi.org/10.1111/jpim.12183 killen, c. p., hunt, r. a., & kleinschmidt, e. j. (2008a). learning investments and organizational capabilities. international journal of managing projects in business, 1(3), 334– 351. https://doi.org/10.1108/17538370810883800 killen, c. p., hunt, r. a., & kleinschmidt, e. j. (2008b). project portfolio management for product innovation. international journal of quality & reliability management, 25(1), 24–38. https://doi.org/10.1108/02656710810843559 killen, c. p., jugdev, k., drouin, n., & petit, y. (2012). advancing project and portfolio management research: applying strategic management theories. international journal of project management, 30(5), 525–538. https://doi.org/10.1016/j.ijproman.2011.12.004 kniberg, h., & ivarsson, a. (2012). scaling agile @ spotify with tribes, squads, chapters & guilds (pp. 1–14). https://blog.crisp.se/wp-content/uploads/2012/11/spotifyscaling.pdf kock, a., & gemünden, h. g. (2020). how entrepreneurial orientation can leverage innovation project portfolio management. r & d management, 51(1), 40–56. https://doi.org/10.1111/radm.12423 kock, a., heising, w., & gemünden, h. g. (2015). how ideation portfolio management influences front-end success. journal of product innovation management, 32(4), 539–555. https://doi.org/10.1111/jpim.12217 kock, a., heising, w., & gemünden, h. g. (2016). a contingency approach on the impact of front-end success on project portfolio success. project management journal, 47(2), 115–129. https://doi.org/10.1002/pmj.21575 kock, a., schulz, b., kopmann, j., & gemünden, h. g. (2020). project portfolio management information systems’ positive influence on performance – the importance of process maturity. international journal of project management, 38(4), 229–241. https://doi.org/10.1016/j.ijproman.2020.05.001 kopmann, j., kock, a., killen, c. p., & gemünden, h. g. (2015). business case control in project portfolios—an empirical investigation of performance consequences and moderating effects. ieee transactions on engineering management, 62(4), 529–543. https://doi.org/10.1109/tem.2015.2454437 kopmann, j., kock, a., killen, c. p., & gemünden, h. g. (2017). the role of project portfolio management in fostering both deliberate and emergent strategy. international journal of project management, 35(4), 557–570. https://doi.org/10.1016/j.ijproman.2017.02.011 kpmg. (2017). driving business performance: project management survey. https://assets.kpmg.com/content/dam/kpmg/nz/pdf/july/projectmanagementsurveykpmg-nz.pdf. krebs, j. (2008). agile portfolio management. remond, wa, usa, microsoft press. kupiainen, e., mäntylä, m. v, & itkonen, j. (2015). using metrics in agile and lean software development – a systematic literature review of industrial studies. information and software technology, 62, 143–163. https://doi.org/10.1016/j.infsof.2015.02.005 australasian journal of information systems aghajani, ahsan & whiteside 2023, vol 27, research article agility meets a project portfolio 42 laanti, m., & kangas, m. (2015). is agile portfolio management following the principles of large-scale agile? case study in finnish broadcasting company yle. in agile conference, 2015. ieee. https://doi.org/10.1109/agile.2015.9 lee, g., & xia, w. (2010). toward agile: an integrated analysis of quantitative and qualitative field data on software development agility. mis quarterly, 34(1), 87–114. leffingwell, d. (2007). scaling software agility: best practices for large enterprises. crawfordsville, in, usa, pearson education. leffingwell, d. (2018). safe 4.5 reference guide: scaled agile framework for lean enterprises. boston, ma, usa, addison-wesley professional. lerch, m., & spieth, p. (2013). innovation project portfolio management: a qualitative analysis. ieee transactions on engineering management, 60(1), 18–29. https://doi.org/10.1109/tem.2012.2201723 martinsuo, m. (2013). project portfolio management in practice and in context. international journal of project management, 31(6), 794–803. https://doi.org/10.1016/j.ijproman.2012.10.013 martinsuo, m., & geraldi, j. (2020). management of project portfolios: relationships of project portfolios with their contexts. international journal of project management, may 2019. https://doi.org/10.1016/j.ijproman.2020.02.002 martinsuo, m., & killen, c. p. (2014). value management in project portfolios: identifying and assessing strategic value. project management journal, 45(5), 56–70. https://doi.org/10.1002/pmj.21452 martinsuo, m., & lehtonen, p. (2007). role of single-project management in achieving portfolio management efficiency. international journal of project management, 25(1), 56–65. https://doi.org/10.1016/j.ijproman.2006.04.002 meskendahl, s. (2010). the influence of business strategy on project portfolio management and its success a conceptual framework. international journal of project management, 28(8), 807–817. https://doi.org/10.1016/j.ijproman.2010.06.007 miles, m., huberman, m., & saldana, j. (2014). qualitative data analysis: a method sourcebook (3rd ed.). sage publications, inc. https://doi.org/10.1080/0140528790010406 mishra, d., mishra, a., & ostrovska, s. (2012). impact of physical ambiance on communication, collaboration and coordination in agile software development: an empirical evaluation. information and software technology, 54(10), 1067–1078. https://doi.org/10.1016/j.infsof.2012.04.002 misra, s. c., kumar, v., & kumar, u. (2009). identifying some important success factors in adopting agile software development practices. journal of systems and software, 82(11), 1869–1890. https://doi.org/10.1016/j.jss.2009.05.052 mosavi, a. (2014). exploring the roles of portfolio steering committees in project portfolio governance. international journal of project management, 32(3), 388–399. https://doi.org/10.1016/j.ijproman.2013.07.004 australasian journal of information systems aghajani, ahsan & whiteside 2023, vol 27, research article agility meets a project portfolio 43 müller, r., martinsuo, m., & blomquist, t. (2008). project portfolio control and portfolio management performance in different contexts. project management journal, 39(3), 28–42. https://doi.org/10.1002/pmj.20053 niederman, f., lechler, t., & petit, y. (2018). a research agenda for extending agile practices in software development and additional task domains. project management journal, 49(6), 3–17. https://doi.org/10.1177/8756972818802713 oecd. (2021). enterprises by business size (indicator). organisation for economic co-operation and development (oecd). https://doi.org/10.1787/31d5eeaf-en olszewska, m., heidenberg, j., weijola, m., mikkonen, k., & porres, i. (2016). quantitatively measuring a large-scale agile transformation. the journal of systems and software, 117, 258. doi: https://doi.org/10.1016/j.jss.2016.03.029 paasivaara, m., behm, b., lassenius, c., & hallikainen, m. (2018). large-scale agile transformation at ericsson: a case study. empirical software engineering, 23(5), 2550–2596. https://doi.org/10.1007/s10664-017-9555-8 padovani, m., & carvalho, m. m. (2016). integrated ppm process: scale development and validation. international journal of project management, 34(4), 627–642. https://doi.org/10.1016/j.ijproman.2016.01.006 patanakul, p., & milosevic, d. (2009). the effectiveness in managing a group of multiple projects: factors of influence and measurement criteria. international journal of project management, 27(3), 216–233. https://doi.org/10.1016/j.ijproman.2008.03.001 petro, y., & gardiner, p. (2015). an investigation of the influence of organizational design on project portfolio success, effectiveness and business efficiency for project-based organizations. international journal of project management, 33(8), 1717–1729. https://doi.org/10.1016/j.ijproman.2015.08.004 pikkarainen, m., salo, o., kuusela, r., & abrahamsson, p. (2012). strengths and barriers behind the successful agile deployment—insights from the three software intensive companies in finland. empirical software engineering, 17(6), 675–702. https://doi.org/10.1007/s10664-011-9185-5 rank, j., unger, b. n., & gemünden, h. g. (2015). preparedness for the future in project portfolio management: the roles of proactiveness, riskiness and willingness to cannibalize. international journal of project management, 33(8), 1730–1743. https://doi.org/10.1016/j.ijproman.2015.08.002 rauch, a., wiklund, j., lumpkin, g. t., & frese, m. (2009). entrepreneurial orientation and business performance: an assessment of past research and suggestions for the future. entrepreneurship: theory and practice, 33(3), 761–787. https://doi.org/10.1111/j.15406520.2009.00308.x rautiainen, k., schantz, j. von, & vähäniitty, j. (2011). supporting scaling agile with portfolio management: case paf.com. system sciences (hicss), 2011 44th hawaii international conference, 1–10. rigby, d. k., sutherland, j., & noble, a. (2018). agile at scale how to go from a few teams to hundreds. harvard business review, 96(3), 88–96. australasian journal of information systems aghajani, ahsan & whiteside 2023, vol 27, research article agility meets a project portfolio 44 saldana, j. (2016). the coding manual for qualitative researchers (3rd ed.). london, uk, sage publications inc. sánchez, m. a. (2015). integrating sustainability issues into project management. journal of cleaner production, 96, 319–330. https://doi.org/10.1016/j.jclepro.2013.12.087 saunders, m., & lewis, p. (2018). doing research in business and management: an essential guide to planning your project (2nd ed.). harlow, uk, pearson education. scaled-agile. (2021). safe 5 for lean enterprises. scaled agile, inc. https://www.scaledagile.com/ schein, e. h. (2009). the corporate culture survival guide. san francisco, ca, john wiley & sons. schwaber, k. (2004). agile project management with scrum (1st ed.). richmond, wa, usa, microsoft press. schwaber, k. (2014). unsafe at any speed. https://kenschwaber.wordpress.com. https://kenschwaber.wordpress.com/2013/08/06/unsafe-at-any-speed/ serrador, p., & pinto, j. k. (2015). does agile work? — a quantitative analysis of agile project success. international journal of project management, 33(5), 1040–1051. https://doi.org/10.1016/j.ijproman.2015.01.006 serrador, p., & turner, r. (2015). the relationship between project success and project efficiency. project management journal, 46(1), 30–39. https://doi.org/10.1002/pmj.21468 shameem, m., kumar, c., chandra, b., & khan, a. a. (2017). systematic review of success factors for scaling agile methods in global software development environment: a clientvendor perspective. proceedings 2017 24th asia-pacific software engineering conference workshops, apsecw 2017, 17–24, nanjing, china. https://doi.org/10.1109/apsecw.2017.22 shameem, m., kumar, r. r., kumar, c., chandra, b., & khan, a. a. (2018). prioritizing challenges of agile process in distributed software development environment using analytic hierarchy process. journal of software: evolution and process, 30(11), 1–19. https://doi.org/10.1002/smr.1979 sheffield, j., & lemétayer, j. (2013). factors associated with the software development agility of successful projects. international journal of project management, 31(3), 459–472. https://doi.org/10.1016/j.ijproman.2012.09.011 shenhar, a. j. (2001). one size does not fit all projects: exploring classical contingency domains. management science, 47(3), 394–414. https://doi.org/10.1287/mnsc.47.3.394.9772 shenhar, a. j., & dvir, d. (2007). reinventing project management: the diamond approach to successful growth and innovation. boston, ma, usa, harvard business review press. shenhar, a. j., dvir, d., levy, o., & maltz, a. c. (2001). project success: a multidimensional strategic concept. long range planning, 34(6), 699–725. https://doi.org/10.1016/s00246301(01)00097-8 slevin, d. p., & covin, j. g. (1990). juggling entrepreneurial style and organizational structure. sloan management review, 31(2), 43–52. australasian journal of information systems aghajani, ahsan & whiteside 2023, vol 27, research article agility meets a project portfolio 45 smite, d., moe, n. b., levinta, g., & floryan, m. (2019). spotify guilds: how to succeed with knowledge sharing in large-scale agile organizations. ieee software, 36(2), 51–56. https://doi.org/10.1109/ms.2018.2886178 šmite, d., moe, n. b., šāblis, a., & wohlin, c. (2017). software teams and their knowledge networks in large-scale software development. information and software technology, 86, 71–86. https://doi.org/10.1016/j.infsof.2017.01.003 spieth, p., & lerch, m. (2014). augmenting innovation project portfolio management performance: the mediating effect of management perception and satisfaction. r&d management, 44(5), 498–515. https://doi.org/10.1111/radm.12092 stankovic, d., nikolic, v., djordjevic, m., & cao, d.-b. (2013). a survey study of critical success factors in agile software projects in former yugoslavia it companies. journal of systems and software, 86(6), 1663–1678. https://doi.org/10.1016/j.jss.2013.02.027 stettina, c. j., & hörz, j. (2015). agile portfolio management: an empirical perspective on the practice in use. international journal of project management, 33(1), 140–152. https://doi.org/10.1016/j.ijproman.2014.03.008 suomalainen, t., kuusela, r., & tihinen, m. (2015). continuous planning: an important aspect of agile and lean development. international journal of agile systems and management, 8(2), 132. https://doi.org/10.1504/ijasm.2015.070607 sutherland, j. (2021). the scrum@scale guide. https://www.scrumatscale.com/scrum-at-scaleguide/ sweetman, r., & conboy, k. (2018). portfolios of agile projects : a complex adaptive systems’ agent perspective. project management journal, 49(6), 18–38. https://doi.org/10.1177/8756972818802712 teller, j. (2013). portfolio risk management and its contribution to project portfolio success: an investigation of organization, process, and culture. project management journal, 44(2), 36– 51. https://doi.org/10.1002/pmj.21327 teller, j., & kock, a. (2013). an empirical investigation on how portfolio risk management influences project portfolio success. international journal of project management, 31(6), 817–829. https://doi.org/10.1016/j.ijproman.2012.11.012 teller, j., kock, a., & gemünden, h. g. (2014). risk management in project portfolios is more than managing project risks: a contingency perspective on risk management. project management journal, 45(4), 67–80. https://doi.org/10.1002/pmj.21431 teller, j., unger, b. n., kock, a., & gemünden, h. g. (2012). formalization of project portfolio management: the moderating role of project portfolio complexity. international journal of project management, 30(5), 596–607. https://doi.org/10.1016/j.ijproman.2012.01.020 thomas, j. c., & baker, s. w. (2008). establishing an agile portfolio to align it investments with business needs. agile 2008 conference, 252–258, toronto, on, canada. https://doi.org/10.1109/agile.2008.29 thummadi, b. v., shiv, o., & lyytinen, k. (2011). enacted routines in agile and waterfall processes. agile conference (agile), 2011, 67–76. https://doi.org/10.1109/agile.2011.29 australasian journal of information systems aghajani, ahsan & whiteside 2023, vol 27, research article agility meets a project portfolio 46 tolfo, c., wazlawick, r. s., ferreira, m. g. g., & forcellini, f. a. (2010). agile methods and organizational culture: reflections about cultural levels. journal of software maintenance and evolution: research and practice, 23(6), 423–441. https://doi.org/10.1002/smr.483 unger, b. n., kock, a., gemünden, h. g., & jonas, d. (2012). enforcing strategic fit of project portfolios by project termination: an empirical study on senior management involvement. international journal of project management, 30(6), 675–685. https://doi.org/10.1016/j.ijproman.2011.12.002 unger, b. n., rank, j., & gemünden, h. g. (2014). corporate innovation culture and dimensions of project portfolio success: the moderating role of national culture. project management journal, 45(6), 38–57. https://doi.org/10.1002/pmj.21458 vahaniitty, j. (2012). towards agile product and portfolio management. in department of computer science and engineering: vol. doctoral d. aalto university. vlietland, j., & van vliet, h. (2015). towards a governance framework for chains of scrum teams. information and software technology, 57, 52–65. https://doi.org/10.1016/j.infsof.2014.08.008 voss, m. (2012). impact of customer integration on project portfolio management and its success—developing a conceptual framework. international journal of project management, 30(5), 567–581. https://doi.org/10.1016/j.ijproman.2012.01.017 voss, m., & kock, a. (2013). impact of relationship value on project portfolio success — investigating the moderating effects of portfolio characteristics and external turbulence. international journal of project management, 31(6), 847–861. https://doi.org/10.1016/j.ijproman.2012.11.005 williams, t. (2005). assessing and moving on from the dominant project management discourse in the light of project overruns. ieee transactions on engineering management, 52(4), 497–508. https://doi.org/10.1109/tem.2005.856572 yin, r. k. (2018). case study research design and methods (6th ed.). thousand oaks, ca, usa, sage publications inc. appendix 1 – indicative interview questions introductory questions about the participant 1. what are your past experiences, particularly concerning agile projects and portfolios? 2. what is your current role in the organisation? about the organisation 3. what is the nature of business in your portfolio? 4. how many projects are running at the same time in your portfolio? 6.1.1 core questions agile project management 5. what specific project methodology are you using in your projects? 6. what is the history of agile in your organisation? how did it happen? 7. what would i see you doing in your projects if i followed you through a typical day? 8. when/how do you consider an agile project successful? 9. what helps you to be successful in an agile project? australasian journal of information systems aghajani, ahsan & whiteside 2023, vol 27, research article agility meets a project portfolio 47 project portfolio management 10. what is the ppm approach in your portfolio/organisation? 11. what are there the challenges/issues in your portfolio? 12. how do you check (measure) the progress of your portfolio? what is success in your portfolio? 13. when/how do you consider an agile portfolio to be successful? 14. how can your agile portfolio improve? appendix 2 – a coding example example quotes from interviews first cycle coding/ coding color case-a ... in an agile where you have to make it a safe environment to be able to raise these issues up and be able to get resolved quickly. if you try to sweep them under the carpet or try to resolve them in isolation, then they just get worse. working in silos is bad [ppm]. …our senior portfolio manager has supported us whenever we want to try new things, we always get full support to do it, and that comes out of trusting as we are going to deliver and the customers will be happy… whatever i’m hearing these days even if it’s from the cio down, they’re actually being very supportive of how they want us to be running next year … trust works both ways. we had all this trust for my managers, and so we need to repay that trust… we really encourage people being able to say what they actually think rather than shooting the messenger. if we are falling behind in a specific feature or if there are any blockers, we really encourage them to call it out as soon as possible, and i’ll be worried about it. and., i think that culture is very prevalent, i’ve seen in the last few years we have had a lot of change and turmoil within teams, but people do respect each other, people do feel comfortable to call things out – [rte]. safe culture (to raise issues working in silos (barrier) trusting teams mutual trust safe culture (open communication) leader serving teams respect culture case-b anyone in the organisation is allowed to challenge anything… but not in a harmful way… usually, it’s a collective decision, and it is the ecosystem of decisions that led to these [good] outcomes. you don’t point the finger at somebody and say you are an idiot, how could you get there… but, then as a management group, we acknowledge that these mistakes are inevitable and we can’t move forward without them… [vp]. safe environment (to raise issues) no blame culture accepting failure case-c it [agile mindset] is very limited. it’s a limited practice because, from my understanding, the culture is good and healthy here…but they [senior management] are very focused on the narrow, short-term view [ac]. no mutual trust (barrier) case-d the meaning of trust with real examples… so during the retrospectives, we talk about issues that are happening with the team with total transparency, and this is not personal. it’s about how we can trust each other in a way that we can grow and mature – [ac]. …usually the programmers (devops) and developers are very resistant about it [sharing ideas]. they feel like they are sharing their ips when they talk. so the stand-up is the best thing for this collaborative environment and the culture it brings to it… so it’s there but not 100% [referring to trust and respect] – [ba]. trust culture – project level agile culture (barrier) collaborating environment case-e …how do you create an aligned, cohesive team purpose and values? there’s the leadership element. so how does it integrate with the team; create that stability and structure, but also have the confidence and the trust that the team is working on the same thing – [ac1]? trust culture – project level leaders serving teams australasian journal of information systems aghajani, ahsan & whiteside 2023, vol 27, research article agility meets a project portfolio 48 example quotes from interviews first cycle coding/ coding color …probably the servant leadership element is a massive thing because, when you mentioned it before with trust, if people aren’t trusted, and things are overgoverned, and there’s too much finance scrutiny and that kind of thing, then you basically constrain people. so even if there are teams trying to work in an agile way, they will be strangled to the point of which they will go. but, if you give people the mission statement and the purpose and the intent and you show them the element of trust, then the coaching kind of flows down to people. you give them some autonomy; you will get way better outcomes – [ac2]. mistrust (barrier) micromanagement (barrier) trusting teams giving team autonomy case-f i think everyone has to work on concert, and i think i did mention… about trust… giving freedom to teams by trusting them that they are capable of delivering. trust has to go both ways… [ppm]. mutual trust giving team autonomy case-g we found that the teams worked really well when they knew each other personally, and that contributes to project success. that is a direct factor – [dl]. close collaboration personal relationship second coding cycle (pattern coding) trust culture open communication culture no-blame culture emerged theme organisational culture category contextual sf copyright: © 2023 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v27i0.4113 http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 literature review 2.1 agile scale-up: challenges and success factors 2.2 traditional portfolios: success criteria and success factors 2.3 agile portfolios: the research problem 2.4 the theoretical lens and conceptual framework 3 research methodology 3.1 case and participant selection process 3.2 data collection 3.3 data coding and analysis 3.4 process and data validation 4 analysis of findings 4.1 success criteria for agile portfolios 4.2 success factors of agile portfolios 4.2.1 contextual success factors 4.2.2 project-level success factors 5 discussion of findings 5.1 what constitutes agile portfolio success? 5.2 what factors contribute to agile portfolio success? 5.2.1 new contextual success factors 5.2.2 new project-level success factors 5.2.3 the effect of uncertainty and consequences sfs 5.3 research implications 5.3.1 theoretical contribution 5.3.2 managerial implications 5.4 limitations 6 conclusion introductory questions 6.1.1 core questions australasian journal of information systems khan & feroz 2024, vol 28, research article “use” as a conscious thought 1 “use” as a conscious thought: towards a theory of “use” in autonomous things gohar khan college of technological innovation, zayed university, united arab emirates a karim feroz school of computing, middle georgia state university, macon, ga, usa karim.feroz@mga.edu abstract the way users perceive and use information systems artefacts has been mainly studied from the notion of behavioral beliefs, deliberate cognitive efforts, and physical actions performed by human actors to produce certain outcomes. the next generation of information systems, however, can sense, respond, and adapt to environments without necessitating similar cognitive efforts, physical contact, or explicit instructions to operate. therefore, by leveraging theories of consciousness and technology use, this research aims to advance an alternative understanding of the "use" associated with the next generation of is artefacts that do not require deliberate cognitive efforts, physical manipulation, or explicit instructions to yield outcomes. the theory and proposed model were refined and validated through the burst detection technique, is expert involvement (n=10), a pilot study (n=130), and end-user surveys (n= 119). structural equating modelling techniques were employed to test the theory. we show that unlike the manually operated is artefacts, the “use” of a fully autonomous artefact is a conscious thought rather than a physical activity of operating a system to produce certain outcomes. we argue that, unlike the traditional notions of use associated with manually operated technologies, conscious use is not characterized solely by behavioral beliefs stemming from logical and reflective cognitive and physical efforts (e.g., effort expectancy). we propose the notion of conscious use within the context of fully autonomous entities and empirically validate its measure. additionally, we offer recommendations for future research directions in this area. the conceptualization of this new theory for fully autonomous is artefacts adds significant academic value to the literature given the convergence of ai-based machine learning systems and cognitive computing systems. keywords: autonomous things, conscious use, conscious thoughts, scale development. 1 introduction the way end users perceive and use information systems has been mainly studied on the premise of deliberate cognitive efforts (i.e., learning to use the system), active (thatcher et al., 2018), and physical use1 behaviour (e.g., clicking, scrolling, pressing, or tapping) by human actors operating manual technologies (adams et al., 1992; delone & mclean, 2003) to produce certain outcomes, for instance, sending an email. (benbasat & zmud, 2003). the current notions of information systems (is) use (sun & teng, 2012) are characterized by behavioral beliefs such as, perceived self-efficacy and perceived effort expectancy (venkatesh et al., 2003), deliberate cognitive and somatic labours i.e., physically operating the system (schuetz & 1in this research, any type of is use that involve logical and deliberate cognitive efforts and bodily actions (such as clicks, taps, scrolling, and waving) is considered as physical or active use. australasian journal of information systems khan & feroz 2024, vol 28, research article “use” as a conscious thought 2 venkatesh, 2020) to carry out tasks and activities for which the system is designed to support (adams et al., 1992; delone & mclean, 2003; marangunić & granić, 2015; seddon, 1997; venkatesh et al., 2003). the next generation of technologies, however, does not involve any reflective cognitive efforts, physical actions, and instructions to operate since they are self-governing (inagaki & sheridan, 2019; jayaraman et al., 2019), context-aware, adaptive, and interactive (schuetz & venkatesh, 2020) sensing, reasoning, and responding to their environments through embodied actions (hoffman, 2012; you & robert, 2018). therefore, the question arises: when there are no logical and deliberate cognitive and physical efforts involved, how would a user then perceive and use an is artefact? owing to these developments, the is researchers have questioned the core assumptions of theorizing the use of is artefact s (lee et al., 2015) on the premise of cognitive and physical processes calling for novel theoretical underpinning to deal with the “entire domain of research questions that cannot yet be answered with our existing theories” (schuetz & venkatesh, 2020, p. 461). it is profoundly evident that the earlier use theories were developed for a different genre of end-users faced with manually operated technologies (benbasat and zmud, 2003; rivard, 2014). hence, it is vital and timely to depart from the current practice of theorizing the use associated with self-governing things on the premise of existing theories (e.g., see zeitzew, 2007) and advocate for a novel theoretical underpinning (rivard, 2014; schuetz &venkatesh, 2020). aside from the theoretical value for academics, developing theories and constructs for the emerging technologies will have greater practical benefits for the businesses that are keen to understand their customer needs and demands (davis, 1989). consequently, this research aims to advance an alternate understanding of the “use” associated with the next generation of is artefacts that do not require any deliberate cognitive efforts, physical (or digital) use, and instructions to produce outcomes. we draw on literature spanning from technology adoption (davis, 1989) to the theories of consciousness (baars, 1988; rosenthal, 1996) and combine it with experts and end-users surveys to propose the notion of conscious use in autonomous things. we put forward that • conscious use—is a conscious thought directed towards an autonomous artefact when it is carrying out tasks and activities for which it is designed. we argue that unlike the manually operated is artefacts (lee et al., 2015), the “use” of a fully autonomous artefact is a conscious thought rather than a physical activity of operating a system to produce certain outcomes. simply put, when a fully autonomous is artefact does not require any deliberate cognitive efforts (e.g., learning to operate the system), physical actions (pushing a button or clicking on a screen), and instruction to operate, its “use” becomes a conscious thought directed towards a fully autonomous is artefact while it is carrying out the tasks and activities for which the system is designed to support it. for example, the kind of conscious thoughts a user will experience when they are relaxing on a comfy sofa, while their attention is directed towards an autonomous vacuum cleaner anticipating their needs through sensory information and situational awareness without requiring any deliberate mental efforts (e.g., learning to operate the system), physical use (e.g., pushing or clicking), and instructions (from the user). similarly, the kind of conscious throughs a user may experience when sitting in a fully autonomous vehicle where the ‘user’ is not driving or directing the machines i.e. there is no cognitive effort involved for operating the vehicle (endsley, 2017; dikmen & burns, 2016). australasian journal of information systems khan & feroz 2024, vol 28, research article “use” as a conscious thought 3 our argument is that in these types of situations the experiences, thoughts, efforts and behavior of users significantly differ as compared to circumstances where users are fully involved with the machines (norman et al., 2003; de melo et al., 2019). unlike the traditional notions of use associated with manually operated technologies (adams et al., 1992; delone & mclean, 2003; marangunić & granić, 2015; seddon, 1997; venkatesh et al., 2003), conscious use is not characterized by active use behaviour (thatcher et al., 2018) and by the behavioral beliefs that stem from logical and reflective cognitive and physical efforts (e.g., perceived self-efficacy and effort expectancy). one implication that can be drawn here is that autonomous technologies will need to be designed to function reliably and smoothly (without human intervention); otherwise, they may trigger an undesirable state of consciousness or dissatisfying thoughts leading to anger, anxiety, and so forth. further validating and extending the proposed notion will profoundly increase our understanding of the way people accept or reject autonomous things that do not require any deliberate cognitive efforts, physical contact, and instructions to operate. this research opens up several avenues for future research on a variety of fronts to further expand our understanding of the nature of use in autonomous things (see table 3). the rest of the article is as follows. next, we briefly discuss a theoretical background and draw conceptual boundaries around our theorization by constructing and elaborating a continuum developed through extensive literature reviews. this is followed by discussing the theories that we leveraged to develop the conscious use construct for autonomous things. we conclude with a discussion summarizing the contribution of the study and the future research avenues available to extend this research. 2 theoretical background a significant amount of research has been dedicated to theorizing and examining the way individuals (venkatesh et al., 2003), groups (brown et al., 2010), and organizations (wang & butler, 2006; del aguila-obra & padilla-meléndez, 2006) perceive and use certain technologies and is (agarwal & karahanna, 2000). is theories provide conceptual frameworks for understanding and explaining the complex relationships between people, technology, and organizations in the context of information systems (henningsson & kettinger, 2016; wu et al., 2022). information systems theories developed for technologies at a certain time period may not suit the evolving and dynamic technological industries often disrupted by novel digital technologies (vial, 2019). for example, the technology acceptance model or tam (chuttur, 2009), which posits that perceived usefulness and perceived ease of use are the two primary factors that determine an individual's intention to use a new technology, is oversimplifying the complex social and organizational dynamics that influence technology adoption and use (ajibade, 2019; malatji et al., 2020). 2.1 information systems and evolving technological dynamics over the years, several is theories have emerged, each with its own set of assumptions, concepts, and principles. although several is theories are available (f.ex. see levy & ellis, 2006 for a list of is theories), predominantly the technology adoption model advanced by (davis, 1989) and later extended, shaped, and reshaped by several researchers (adams et al., 1992; delone & mclean, 2003; marangunić & granić, 2015; seddon, 1997; venkatesh et al., 2003) has profoundly expanded our understanding of the way people accept or reject certain australasian journal of information systems khan & feroz 2024, vol 28, research article “use” as a conscious thought 4 technologies and systems. a vast majority of hypotheses that deal with information systems and technology perceptions, attitudes, behaviors, and intentions (lee et al., 2003; taylor & todd, 1995; venkatesh et al., 2003), values, satisfaction, and system characteristics, success, and failure (delone & mclean, 2003; wixom & todd, 2005) leads to use (see figure 1). some investigations even go beyond the use and look into is post-use behaviour and consequences (wang & butler, 2006; ahuja & thatcher, 2005; jasperson et al., 2005). is use has also been extensively investigated in a variety of levels, such as individual and group levels, and contexts, such as business context, domestic context, education context, healthcare, military, and cultural contexts (im et al., 2011), to name a few (figure 1). is use has been conceptualised in mainstream research domains including is success, is acceptance, is implementation, and is for decision-making and formally defined by burton-jones and straub (2006) as “an individual user’s employment of one or more features of a system to perform a task” (p. 6). this definition implies that is use is a complex cognitive and physical activity involving a user, the system itself, and a task that needs to be completed over time (burtonjones & straub, 2006). the existing notion of is physical use (schomaker et al. 1995; sun & teng, 2012) can be conceptualized as an activity of expanding logical and deliberate cognitive efforts (e.g., needed to learn and use the system) and physical actions (e.g., clicks and scroll) carried by human actors through a graphical user interface (such as displays) to produce the desired outcomes such as, sending a message or ordering a product online (figure 1). in this sense, the existing notions of is use rest on several assumptions (schuetz & venkatesh, 2020) and postulate the “use” as a physical activity associated with an is artefact on the premise of deliberate cognitive efforts e.g., learning to operate the system (agarwal & karahanna, 2000) and physical actions (of clicking, scrolling, pressing, or tapping) performed by human actors (adams et al., 1992; delone & mclean, 2003) to produce certain outcomes (benbasat & zmud, 2003). in such systems, “humans define the input” (schuetz & venkatesh, 2020, p. 645) and physically (or digitally) uses the artefact to “carry out tasks and activities on the job for which the information system is designed to support” (sun & teng, 2012, p. 1565). these notions of technology use are primarily rooted in the theory of reasoned action developed by ajzen and fishbein (1980), which suggests that “external variables influence beliefs about the outcomes associated with performing a behaviour, which in turn shape attitudes toward performing a behaviour. attitude, in turn, influences intention to perform the behaviour and, ultimately, influences the behaviour itself.” (wixom & todd, 2005, p. 86). in the technology adoption model context, this implies that the use of technology is influenced by users’ beliefs (e.g., perceived ease of use, self-efficacy, and effort expectancy), which impact attitudes toward using the technology, and which in turn shape behaviours (and intentions) to use technology. by suggesting a radically different concept “conscious use” in autonomous entities and verifying it through empirical methodologies, we are confident that this study not only offers a substantial addition to the prevailing is research landscape but also pioneers a pathway that emphasizes the urgency and importance of acquiring deeper insights. this forward-thinking approach is crucial, especially as we anticipate the evolving needs and challenges of the upcoming generation of autonomous systems (schuetz & venkatesh, 2020). for instance, when surrounded by fully autonomous artefacts, users’ conscious thoughts regarding an autonomous technology (not the physical use) will determine users’ satisfaction/dissatisfaction, purchase decisions, adoption/rejection, and so forth. this is australasian journal of information systems khan & feroz 2024, vol 28, research article “use” as a conscious thought 5 because traditionally, "use" encompassed tangible interactions between users and systems. yet, with the rise of highly autonomous technologies, interaction doesn't necessarily imply tangible, hands-on engagements. according to wang and benbasat (2017), interactions in the autonomous landscape can often be more observational than participatory, encompassing activities like monitoring, overseeing, or even just recognizing system functionalities. such passive engagements, contrary to being insignificant, profoundly shape user perceptions and trust. supporting this idea of the evolving nature of information systems, riek (2017) suggests that technological interactions, especially with systems possessing a high degree of autonomy, are becoming deeply cognitive. users might not continuously provide explicit commands to such systems. however, they're constantly processing information, making judgments, and framing perceptions based on the autonomous actions of the system. this dimension of cognitive interaction is central to understanding conscious use in the realm of autonomous is. while tangible interactions might be sparse, cognitive processing and acknowledgment denote a form of "use" that is intrinsic to our current technological landscape. transitioning to outcomes, autonomous systems have redefined this dynamic. traditional outcomes, such as user satisfaction, often grounded in direct user-system interactions (de guinea & markus, 2009), now undergo transformation. in autonomous systems, outcomes are perceptual. for example, trust in an autonomous system might hinge more on its perceived reliability, predictability, and transparency rather than interface design or traditional usability metrics (desai et al., 2017). gerlach and kuo (1991) emphasize the repercussions of users' conscious perceptions of such systems. trust, satisfaction, and even intent to use the system in the future can hinge on the users' consciousness. the understanding, acknowledgment, and consequent mental processing of the system's actions underpin user responses and are pivotal in predicting user behavior. autonomous is mandates a new lens of understanding "use." as systems operate with diminished direct human oversight, our engagement with them is characterized by acknowledgment, observation, and cognitive processing rather than hands-on interaction. conscious use, with its focus on this evolved form of engagement, offers a fresh, relevant framework, and integral discussion avenue for the ongoing discourse in today's evolving digital landscapes. among the core constructs that explain users’ accepting or rejecting technology is the “perceived ease of use” which is defined as the “degree to which a person believes that using a particular system would be free of efforts.” (davis, 1989, p. 320). moreover, if users perceive technology as “easy to use” and “useful” then they are more likely to physically “use” it (adams et al., 1992). in addition to ease of use and usefulness, other behavioural beliefs and attitudes such as selfefficacy and effort expectancy are important predictors of technology usage behaviours (agarwal et al., 2000; adams et al., 1992; delone & mclean, 2003; marangunić & granić, 2015; seddon 1997; venkatesh, 2000). self-efficacy is the “beliefs in one’s capabilities to organize and execute the courses of action required to produce given attainments” (bandura, 1997, p. 3). and effort expectancy is the amount of ease associated with using a system (venkatesh et al., 2003). australasian journal of information systems khan & feroz 2024, vol 28, research article “use” as a conscious thought 6 figure 1. the existing notions of information systems/technology use owing to these assumptions, the use of an is artefact has been theorized as “intentional and deliberate” (schuetz & venkatesh, 2020) dealing with the users’ behavioural beliefs and attitudes toward an is artefact that requires deliberate physical and cognitive efforts (agarwal & karahanna, 2000) and active use behaviour (thatcher et al., 2018). while this theorization is flawless, it runs into infinite regress when dealing with autonomous things that do not require any logical and reflective cognitive efforts, physical use, and instruction to operate (schuetz & venkatesh, 2020). the way end-users interact with is artefacts (lee et al., 2015) has come a long way from the humble beginnings of interacting with punch cards to keyboards and mice, touch screens, and now to interactions with autonomous things not requiring any inputs (ernst 2020; musk, 2019; ronkainen et al., 2007; sen et al., 2015). 2.2 human-robot interaction (hri) and conscious use integrating insights from the human-robot interaction (hri), an interdisciplinary domain that examines the dynamics, mechanisms, and outcomes of the interactions between humans and robots, this paper sheds light on how users perceive, understand, and navigate their relationship with autonomous robotic systems. as technological advancements continue, the relationship between humans and robots has evolved, and so has the need for a deeper understanding of this bond. traditionally, hri was rooted in usability, aiming to ensure that robots were tools that could be efficiently and effectively operated by human users (goodrich & schultz, 2007). manually operated systems australasian journal of information systems khan & feroz 2024, vol 28, research article “use” as a conscious thought 7 over time, however, as robots became more autonomous and integrated into daily life, hri research began to address a broader range of topics, including trust, emotion, cognition, and long-term interaction. the way users perceive robots plays a vital role in interaction dynamics. desai et al. (2017) explored how perceived robot autonomy affects user interactions, concluding that mismatches between perceived and actual autonomy can impact task performance. these findings reinforce the need for a "conscious use" perspective, emphasizing the role of cognition and perception in hri. recent literature underscores that robots, particularly those with social functions, invoke a plethora of emotions in humans (broadbent, 2017). not merely tools, these robots often transform into entities sharing emotional bonds with their users. for instance, nomura et al. (2008) in their extensive review identified that the design, aesthetics, and responsiveness of a robot play pivotal roles in determining human emotions towards them. this emotion-centric dimension of hri aligns seamlessly with broader debates about the anthropomorphism of autonomous is artefacts. dautenhahn (2007) was among the pioneers who highlighted the significance of the social and emotional dimensions of hri. traditionally, industrial robots were isolated from human workers mainly due to safety concerns. co-bots, or collaborative robots, represent a pivotal shift in the industrial landscape, particularly within the human-robot interaction (hri) sector. these robots are designed to work alongside humans, enhancing productivity and safety while fostering a symbiotic relationship between man and machine. this paradigm shift is reshaping traditional manufacturing processes by augmenting human capabilities with robotic precision and efficiency (arents et al., 2021; zacharaki et al., 2020). current research in this domain delves into optimizing human-robot collaboration, ensuring safety, and understanding the psychological implications of shared human-robot workspaces. robots and autonomous is, especially those designed for social interactions, are no longer mere tools; they are entities with which users form emotional bonds. this perspective aligns with our "conscious use" paradigm, suggesting that interaction with robots, especially autonomous ones, is not merely a function of physical operation but also involves cognitive and emotional processes. the core tenet of conscious use is rooted in the conscious recognition and acknowledgment of an autonomous system's operations. as specified in recent literature, the relationship users forge with autonomous systems is a blend of cognitive processing and subtle forms of engagement (wang & benbasat, 2017; riek, 2017). consequently, a new genre of is research is emerging that questions the existing notions of theorizing users’ beliefs and interactions with autonomous systems on the premise of physical use (demetis & lee, 2018). schuetz and venkatesh (2020) argue that human-like artificially intelligent systems break down the prevalent unilateral notions of “user-artefact interaction” where a human is assumed as a deliberate user of the system. the role of the human as active users is also challenged by demetis and lee (2018) by advancing the notion of “role-reversal” between humans and technology where humans are considered as artefacts shaped and used by technology and not the other way around. hence, we believe the use associated with the next generation of technologies is not characterized by behavioural beliefs and attitudes (e.g., system self-efficacy and effort expectancy) associated with manually operated technologies and propose an alternative notion of “use” in the context of autonomous things, discussed next. australasian journal of information systems khan & feroz 2024, vol 28, research article “use” as a conscious thought 8 3 conceptual boundaries to draw conceptual boundaries around our theorization, a continuum was developed based on the key aspects of autonomous technologies discussed in the literature above as well as burst detection technique that was employed to investigate the emerging research themes (see appendix a). table 1 explains these aspects. the “use” related assumptions listed in table 1 are derived based on theoretical reasoning provided in the subsequent sections and the conceptualization provided by schuetz and venkatesh (2020). one of the core facets that distinguish the next generations of technologies from conventional technologies is the state of its autonomy (ernst, 2020). the concept of autonomy (which is distinct from automation) has been studied by human-robot interaction researchers, behavioralists, and scientists in various fields for decades (vagia et al., 2016). the term "autonomy" refers to a system's ability to make its own decisions regarding its activities while executing various tasks without the involvement of an external system or operator (albus & antsaklis, 1998). from the perspective of human-machine interaction and cooperation, the amount of autonomy of the system/technology can be expressed by various levels of automation (inagaki & sheridan, 2019). each of these levels specifies a different degree to which a task is automated. the automation theorists normally characterize the automation levels in terms of the humancomputer interaction required to complete a task or achieve a goal. in this sense, autonomy has several levels from being completely autonomous, and partially autonomous, to nonautonomous things (inagaki & sheridan, 2019). following the autonomy standards reported in the literature (endsley, 1999; inagaki & sheridan, 2019) and by considering the key attributes of the autonomous is artefact proposed by schuetz and venkatesh (2020) and the way autonomous technologies work (hoffman, 2012), we created a generic taxonomy of the autonomous things as shown in the continuum (see figure 2). for this research, we define an autonomous information systems artefact (denoted as: aia) as any physical (or digital) is artefact that can work independently without any human intervention (e.g., self-driving cars, drones, virtual agents, and robots) thus requiring no interaction from the end user to complete a task or achieve a goal. we consider aia as an is artefact because it is essentially formed when cutting-edge technology (technology artefact) interacts with the sensory information it processes (information artefact), and the social context in which it operates e.g., social artefact (lee et al. 2015). aia takes in sensory data, interprets the information, adapts, and responds to users’ needs accordingly (hoffman, 2012). the main differentiator among the levels of autonomy, in our approach, is the need for deliberate cognitive efforts, physical contact, and instructions to operate an autonomous is artefact. australasian journal of information systems khan & feroz 2024, vol 28, research article “use” as a conscious thought 9 figure 2. autonomous things continuum at the one end of the continuum (see figure 2) are fully autonomous is artefacts that do not require any cognitive efforts, physical contact, and instructions to function; whereas, at the other end are the ‘manual technologies’ that always require cognitive effort, physical contact, and instructions and physical use. it is noted that the continuum itself does not capture all possible types of systems having a more blended set of interactions, for simplicity sake we capture and explain the three distinct types. 3.1 fully autonomous is artefact (faisa) faisa is any (physical or digital) is artefact that does not require deliberate cognitive efforts, physical contact, and instructions to produce outcomes in all situations permanently. although most autonomous artefacts are designed for humans to engage in verbal and physical humanmachine interaction and cooperation at various levels of automation (vagia et al., 2016). for example, even with a fully autonomous robot-human interaction may be required to complete a task or when enabling the robot’s fully autonomous capability. however, for the theory construction purposes in this research, we restrict our conceptualisation to a faisa that does not require any deliberate cognitive efforts, physical contact, and instructions to produce outcomes in all situations permanently. leveraging sensory data, such an is artefact is constantly sensing, responding, and adapting to users’ needs without requiring human intervention (hoffman, 2012). the faisa is autonomous, context ware (irene & susan, 2017), adaptive, interactive, and stateful (schuetz & venkatesh, 2020). with major advancements in the fields of engineering, robotics, and artificial intelligence, coupled with improved computational power and network availability, autonomous things are now becoming ubiquitous across many industries. examples of faisa include autonomous vehicles (bimbraw, 2015; jayaraman et al., 2019) and autonomous robots (kwak et al., 2017; pellenz et al., 2009), autonomous virtual agents (kramer et al., 2014; wang et al., 2018), smart mirrors (hossain et al., 2007), to name a few. furthermore, a faisa can australasian journal of information systems khan & feroz 2024, vol 28, research article “use” as a conscious thought 10 either be visible to naked eyes, invisible (blended in the background or implanted in the body), digital (such as a software product) or physical (such as a vacuum cleaner). in addition to being virtually experienced (e.g., virtual reality games); ubiquitous or everywhere (borriello, 2000); and heterogeneous networked (i.e., linked to other connected things) (atzori et al., 2010), the autonomous technologies have some additional properties that set it apart from the conventional technologies (ng & wakenshaw, 2016). these properties include sensibility: it can sense and respond to stimuli hence making it context-aware; addressability: it can be identified and located in real-time; associability: it can be associated (and coordinated) with other objects to enable inferences about future states and conditions; communicability: it can send and receive digital messages; programmability: it can receive new sets of instructions and to modify its behaviors according; and memorability: it can store information and historical logs of its state and user interactions (ng & wakenshaw, 2016). 3.2 partially autonomous is artefact (paisa) paisa is any (physical or digital) is artefact that does not require physical contact but needs reflective cerebral efforts and instructions to produce outcomes in all situations permanently. the instructions needed can be cognitive, gestural, or speech-based wirelessly communicated through a variety of mechanisms. in this scenario, physical contact (e.g., push or click) is not required but the system still needs cognitive efforts (e.g., learning to operate the system) and instructions to operate. examples of paisa include voice-commanded systems (e.g., siri and alexa) (sen et al., 2015); systems that are controlled with thoughts (e.g., elon musk's neuralink) (musk, 2019), and gestures-enabled systems (e.g., selfietype, a gesture-based virtual keyword developed by samsung). 3.3 non-autonomous is artefact (naisa) naisa is any (physical or digital) is artefact that requires deliberate cognitive efforts, physical contact, and instructions to produce outcomes in all situations permanently. the instructions are provided in the form of physical actions performed by the users (e.g., clicking on a screen). most conventional technologies come under this category (e.g., a smartphone, personal computers, email agents, word-processing, and so forth). these technologies may have a certain level of automation and context awareness but based on our criteria of requiring deliberate cognitive efforts, physical contact, and instructions to operate, they are classified as non-autonomous from the users’ perspective. use in naisa is the domain of existing technology perception and use theories. and although the use of paisa is contactless, it still requires cognitive efforts to operate, hence the existing theories can be adapted to explore it. in this study, we only focus on the use of faisa discussed next. australasian journal of information systems khan & feroz 2024, vol 28, research article “use” as a conscious thought 11 facets full autonomy partially autonomy no autonomy description cognitive efforts cognitive efforts are not required. cognitive efforts are required. cognitive efforts are required. full autonomous is artefact s sense, respond, and adapt to user needs not requiring deliberate cognitive efforts (e.g., learning to use the system) and physical use (e.g., clicking); however, thoughtful cognitive efforts are needed to operate partial and non-autonomous is artefacts. physical contact physical contact is not needed. physical contact is not needed. physical contact is needed. physical contact is not required to operate fully and partially autonomous is artefact, but it is a core element of the nonautonomous things. instructions instructions to operate are not needed. the instructions to operate are cognitive, vocal, or gestural. the instructions are physical or deliberate. while a fully autonomous is artefact works without any instructions, partially autonomous operates on the principle of wireless instructions (in form of thoughts, voice, & gestures); whereas the nonautonomous artefact s need physical instructions to operate (e.g., clicks, scroll, & push). use the use is a conscious thought. the use is cognitive, vocal, or gestural but contactless. the use is deliberate, cognitive, and physical activity. given that a fully autonomous artefact does not require reflective cognitive efforts, physical contact, or instruction to operate, its use becomes a state of consciousness rather than a cognitive and physical activity (i.e., pushing a button or clicking on a screen); whereas the use of a partially autonomous thing is a physical cognitive, vocal, or gestural not requiring physical intervention. however, the nonautonomous things require physical (active) use in the form of clicks, scroll, & push. visibility visible or invisible to the naked eye. visible or invisible to the naked eye. visible or invisible to the naked eye. visibility is a non-differentiator but if combined with autonomy it becomes a key aspect of the fully autonomous artefact. state the state is either physical or digital the state is either physical or digital the state is either physical or digital all these is artefact s can be either physical (atoms) or digital (bits) in nature. australasian journal of information systems khan & feroz 2024, vol 28, research article “use” as a conscious thought 12 table 1. key characteristics of the next generation of things 3.4 “use” in fully autonomous artefacts as alluded to earlier, the use of a manually operated is artefact is driven either by deliberate physical and cognitive efforts (schuetz & venkatesh, 2020; thatcher et al., 2018) or automatic behaviour resulting from past usage not requiring conscious processes (jasperson et al., 2005; kim et al., 2005). deliberate or active use of a system is rational behaviours driven by logical and reflective mental processes and somatic actions (jasperson et al., 2005); whereas, automatic use, although still associated with manually operated technologies, occurs unconsciously largely due to habitual or repetitive use of a system (kim et al., 2005). mindfulness or being “aware of, and open to, value-added use of the system” (thatcher et al., 2018, p. 837) also plays a crucial role in operating is artefact s that require physical use and cognitive efforts (table 2). whereas, a typical faisa sense, respond, and adapt to user needs not requiring deliberate (1) cognitive efforts, (2) physical contact, and (3) instructions from human actors to produce outcomes (hoffman, 2012) not characterized by physical actions and deliberate cognitive efforts (schuetz & venkatesh, 2020; thatcher et al., 2018) or automatic use associated with manually operated technologies (jasperson et al. 2005; kim et al., 2005). construct definition use context and key attributes examples conscious use use is a conscious thought directed towards an autonomous artefact when the artefact is carrying out tasks and activities for which it is designed. context: autonomous is artefact. key attributes: is not characterised by deliberate cognitive efforts and physical actions from the users. this study is use use is a human-driven activity of expanding cognitive efforts and physical actions to produce the desired outcomes through a system. context: manually operated is artefact. key attributes: is characterised by logical, reflective, and deliberate cognitive efforts and physical actions required from users. (burton-jones and straub, 2006; adams et al., 1992; venkatesh et al., 2003; sun and teng, 2012). automatic use unconscious use of a system driven by habitual or repetitive physical use of a system. context: manually operated is artefact. key attributes: is characterised by habitual logical, reflective, and deliberate cognitive efforts and physical actions required from users. (kim et al., 2005; limayem et al., 2007). connectivity heterogeneously connected. heterogeneously connected. homogeneously connected. the fully (and partially) autonomous things connect heterogeneously to dissimilar things (e.g., the connection among the home appliance aka iot); whereas the nonautonomous things are homogeneously connected to similar technologies (e.g., a smart-to-smart phone and personal-to-personal computer connections). australasian journal of information systems khan & feroz 2024, vol 28, research article “use” as a conscious thought 13 low-intention interaction a type of human-computer interaction that occurs when a user’s action intended for one purpose is interpreted to achieve a variety of other purposes. context: manually operated is artefact. key attributes: is characterised by logical, reflective, and deliberate cognitive efforts and physical actions required from users. (dix, 2017). awareness the state of an observer “consciously seeing” a stimulus in a given situation context: not specifically related to is artefact. (henley, 1984) situational awareness the perception of the elements in the environment within a volume of time and space, the comprehension of their meaning, and the projection of their status in the near future context: not specifically related to is artefact. key attributes: characterized by stimuli, comprehension of the environment and, propagation of events (endsley, 1995) context awareness an umbrella term used to describe the technologies that consider the context to characterize the situation of users. context: autonomous and manually operated is artefact. key attributes: requires context and situational awareness from the technology in use. (dey, 2001) mindfulness being in a state of paying attention in a particular way that is on purpose, in the present moment, and nonjudgmentally context: not specifically related to is artefact. (kabat-zinn, 1994) it mindfulness a mindset driven by an individual’s awareness of the context, and openness to, value-adding applications of it. context: manually operated is artefact. key attributes: requires context and situational awareness from the users. (thatcher et al., 2018) it habit the automatic physical use of a system due to past learning. context: manually operated is artefact. key attributes: requires previous it use experience from the users. (limayem et al., 2007) consciousness a state of being aware of an external object or something within oneself. context: not specifically related to is artefact. (baars, 1998; rosenthal, 1996) flow a mental state experienced when someone is completely immersed in activities and a corresponding shift in one's perception of time, people, distractions, and even basic bodily needs. context: not specifically related to is artefact. (mihaly, 1990) table 2. conscious use vs. other forms of is use more importantly, the user consciously perceives an autonomous is artefact in service anticipating their needs through sensory information and situational awareness without requiring any deliberate cognitive efforts, physical or visual use, and instructions (figure 3). such a use can be characterized as “low attention” and/or “low intension” automatic use not requiring thoughtful human inputs (dix, 2017). for example, the light in a room may switch australasian journal of information systems khan & feroz 2024, vol 28, research article “use” as a conscious thought 14 off as the occupant goes into a sleep state. context-awareness plays a crucial role in characterizing the situation of users as it takes into account all information relevant to the interaction between a user and an application, including the user and applications themselves. unlike the traditional is systems where user inputs are interpreted only in terms of the state of the system, context-aware systems looked to the environment or context (dey, 2001). furthermore, even if the autonomous system is operating smoothly, the user is generally aware of the system and will expect the tasks (automatically performed by the system) that are entirely below the user’s awareness (dix, 2017). thus, the use associated with a faisa becomes a subject of human consciousness rather than a matter of deliberate physical/ cognitive efforts and active use associated with operating manual technologies/systems (schuetz & venkatesh, 2020; thatcher et al., 2018). therefore, contrary to the current practice of theorizing the use associated with faisa on the premise of existing theories (hewitt et al., 2019; zeitzew, 2007), we advocate for leveraging the theories on human consciousness (baars, 1988). human consciousness has been a complex and puzzling construct of physiological sciences for centuries (baars, 1988). more recently, significant interdisciplinary research has been dedicated to understanding consciousness—both biologically and psychologically—in cognitive science, involving fields such as psychology, linguistics, anthropology, and neuroscience, to name a few. the theory of consciousness explains consciousness as a state of being aware of an external object or something within oneself (baars, 1988; rosenthal, 1996) and the oxford living dictionary defines consciousness as "the state of being aware of and responsive to one's surroundings." a more recent theory (marchetti, 2018, p. 435) suggests that “consciousness is a special way of processing information” due to three vital cognitive procedures namely, the self, attention, and working memory. the concept of self, primarily mediated by the central and peripheral nervous systems in humans, plays a pivotal role in individuals' ability to perceive and navigate their bodies, their environment, and their interactions within the world. attention facilitates the selection of pertinent variations in the state of the self that hold significance within a given context. subsequently, working memory is indispensable for assembling the chosen specific pieces of information identified through attention (marchetti, 2018). this unique way of processing information produces (rather transmitting) individualized information meaningful for the person who consciously experiences it, and it has “that” meaning only for the person experiencing it, not for other people (marchetti, 2018). in the context of autonomous things, for example, users know what it means for them to experience an autonomous is artefact, but onlookers cannot directly know what it means for the users to experience the is artefact (and vice versa). in other words, experiencing a faisa is a very individualist state of consciousness producing meaningful information about the faisa not accessible to others. this individualist conscious state of mind allows users to process information based on their desires while consciously being aware of a faisa at their disposal. the users consciously anticipate a faisa sensing, responding, and adapting to their needs through sensory information and situational awareness without requiring any mental efforts, instructions, and physical contact. furthermore, our consciousness informs us in real-time about the impact an external object (in this case a faisa) will have on us, where the object is relative to us now, and whether we can cope with it (marchetti, 2018). in this sense, the use of a faisa is the subject of human australasian journal of information systems khan & feroz 2024, vol 28, research article “use” as a conscious thought 15 conscious thoughts. thoughts refer “to mental states that can be the output (and input) of thinking processes” (vosgerau & synofzik, 2010) (p. 206) having two types: unconscious and conscious thoughts. conscious thoughts are defined as the “object-relevant or task-relevant cognitive or affective thought processes that occur while the object or task is the focus of one's conscious attention;” whereas, unconscious thought refers to “object-relevant or task-relevant cognitive or affective thought processes that occur while conscious attention is directed elsewhere” (dijksterhuis & nordgren 2006, p. 96). furthermore, “attention” is the core differentiator between the unconscious and conscious thought i.e., conscious thought is “thought with attention” and unconscious thought is “thought without attention” (dijksterhuis & nordgren, 2006). in addition, kahneman (2012) suggests that the human brain operates in two modes of thinking: system 1 and system 2. system 1 is characterised by effortless and automatic thinking; whereas system 2 deals with deliberate, effortful, and rational thinking. unlike system 2 where task effects arise from deliberate thoughts and effortful comparisons among options, system 1 operates with comparatively ‘passive’ way of thinking directed at certain tasks. it is argued that choice effects are the results of intuitive processing and require little deliberation in the context of system 1 (dhar & gorlin, 2013). therefore, based on how human brain functions automatic and conscious use is the result of system 1 related choice effects whereas, habitual, and compulsive patterns of use arise from system 2. considering the above discussion, we propose the notion of conscious use (cu), which is “a conscious thought directed towards an autonomous artefact when it is carrying out tasks and activities for which it is designed” (see figure 3). we theorize the “use” associated with an autonomous artefact as a conscious thought rather than a physical (or somatic) activity. unlike the “use” associated with the manually operated technologies that require deliberate cerebral efforts, instructions, and physical actions (such as pushing a button or clicking on a screen), a typical fully autonomous artefact sense, respond and adapt to user’s needs without requiring any deliberate cognitive efforts and physical (or somatic) activity from the users. for example, the sort of conscious thoughts a user will experience when their attention is directed towards an autonomous artefact (e.g., an autonomous vacuum cleaner) anticipating her needs through sensory information and situational awareness without requiring any deliberate cognitive efforts (e.g., learning to operate the system), physical use (e.g., pushing or clicking), and instructions (from the user). theoretically, one can argue that any conscious thoughts the user has when a fully autonomous artefact is carrying out tasks and activities would qualify as conscious use? for example, when a user is happy watching a football match while their robot vacuum cleaner was operating autonomously? however, we argue that conscious use captures conscious thought when attention is directed towards an autonomous artefact. in the other words, the mind is connected to the fully autonomous artefact. furthermore, based on the definition, any conscious thoughts the user happens to have when attention is directed towards a fully autonomous will also qualify as conscious use including performance monitor, anger, anxiety, enjoyment, so forth. in this research, we are only measuring the conscious thoughts when attention is directed towards a typical fully autonomous artefact that senses, respond, and adapt to the user’s needs as explained in our instrument development section. notably, conscious use may appear to share some conceptual underpinnings with the notion of automatic use. precisely, they both capture the use of a system that is not driven by australasian journal of information systems khan & feroz 2024, vol 28, research article “use” as a conscious thought 16 deliberate mental efforts. automatic use, however, is post-use behaviour resulting from repetitive physical and active use of a manually operated technology (jasperson et al., 2005; kim et al., 2005); whereas, conscious use is a state of a user’s consciousness (a conscious thought) directed towards a fully autonomous is artefact sensing, responding, and adapting to the tasks and activities for which the artefact is designed to support. figure 3. proposed conceptual model similarly, other conceptualizations of “state of mind” exist in literature in diverse settings. for example, “flow” is a mental state experienced when someone is completely immersed in an activity such that “nothing else seems to matter" (mihaly, 1990, p. 4). in essence, “flow” conceptualizes the state of concentration that people experience when they are deeply involved in a physical (or cognitive activity) and what makes the experience they are having satisfying. mihaly (1990) argues that the activity people experience becomes genuinely satisfying when the state of concentration is deeply focused to the point of complete immersion in a task. leveraging the flow notion some authors have attempted to better understand human-technology interactions (trevino & webster, 1992) and technology usage (agarwal & karahanna, 2000). although “flow” and “conscious use” both focus on the state of a user’s mind, they are opposite concepts. conscious use attempts to conceptualize conscious thoughts of a user when a fully autonomous artefact is carrying out certain tasks and activities not requiring any involvement from the users. for example, the state of mind a user will be in when they are observing an autonomous vacuum cleaner buzzing around doing its job. in other words, as alluded to earlier, conscious use is attempting to capture a user’s conscious thoughts directed towards an autonomous artefact anticipating their needs through sensory information and situational awareness without requiring any deliberate involvement (cognitive or physical) from the users. table 2 offers a comparison of conscious use with other forms of is use. australasian journal of information systems khan & feroz 2024, vol 28, research article “use” as a conscious thought 17 4 methodology 4.1 instrument development given that no instrument exists to measure user’s conscious thought when attention is directed towards an autonomous artefact, we create such an instrument. to validate the psychometric properties of the proposed “conscious use” construct and the subconstructs, we followed the structured approach commonly used to develop scales (davis, 1989; hinkin, 1995; noar, 2003). the process of scale development started with (1) conceptualization of the theoretical construct, which was followed by (2) generating potential items for each construct considering the definitional properties of the construct and the issues reported in the literature, and finally (3) the scale was refinement and validated through is expert involvement, end-user surveys, and robust statistical analysis. the practice of selecting potential items of a scale from the onset ensures content (davis, 1989; nunnally, 1978) and face validity (broder et al. 2007) of a scale. we also followed the suggestions by noar (2003) and did our best to produce scale items that were (a) clear and concourse dealing with one issue per item; we avoided (b) jargon and dated phrases and (c) confusing sentences, words, and double negatives; we added (d) both positively and negatively worded statements and (e) were mindful of gender and culturally sensitive; (f) kept the statement short, and finally (g) we made an effort to cover most of the issues suggested by the theories and the literature. the basis for the construct conceptualization was the nature of a typical autonomous artefact and based on guidelines from the literature (jarvis et al., 2003; petter et al., 2007), we conceptualize conscious use as a multidimensional formative construct consisting of 3 formative and 1 reflective sub-constructs.” 4.2 items generation and construct conceptualization drawing upon the literature and discussions above (wisdom et al., 2014), we developed the subconstructs for conscious use based on the definitional properties of the construct and by taking into consideration the nature of a typical autonomous technology (hoffman, 2012; schuetz & venkatesh, 2020). by considering the crucial attributes of autonomous systems such as timeliness, reliability, flexibility, and accuracy (delone & mclean, 2003; seddon, 1997), a list of 16 items for conscious use (4 for each sub-constructs) was developed. experts in the is field were consulted in advance regarding the list of subconstructs. furthermore, we included the satisfaction construct (delone & mclean, 2003) in our survey for nomological validity. the context of the instrument development was a fully autonomous robot currently employed in an international supermarket chain (name of the robot, its location, and brand identity is withheld at the request of the company). leveraging embodied cognition principle (wilson, 2002), a fully autonomous robot roams in the supermarket isles sensing (through sensory inputs such as videos and images) hazards (such as fallen objects), responds to the hazards by alerting the staff, and adapts to the environment by avoiding obstacles. using sensory inputs and situational awareness, the fully autonomous robot senses users’ needs and actions “based on repetitive past events” and model the “resulting anticipatory expectation as [a] modal perceptual simulation” (hoffman, 2012, p. 764). this context was chosen because the fully autonomous robot, as an archetypal autonomous technology, perfectly fits the theoretical construct. at the time of the survey, the employees (the survey respondents) were fully mindful of the fully autonomous robot operating in the supermarket and knowing that it is fully autonomous not requiring deliberate cognitive australasian journal of information systems khan & feroz 2024, vol 28, research article “use” as a conscious thought 18 efforts, physical contact, and instructions (from the employees) to sense hazards, respond to the hazards, and adapt to the environment. based on the criteria from (jarvis et al., 2003) criteria and guidelines from (petter et al., 2007), we conceptualize conscious use as a multidimensional formative construct consisting of 3 formative and 1 reflective sub-constructs (figure 4). figure 4. conceptualization of conscious use autonomy consciousness (aco)—this reflective sub-construct accounts for a user’s overall state of mind (or awareness) regarding a fully autonomous artefact at their service. sensibility consciousness (sc)—this formative sub-construct measures a user’s conscious thoughts when a fully autonomous artefact is sensing the tasks and activities for which the system is designed to support it. responsiveness consciousness (rc)—this formative sub-construct measures a user’s conscious thoughts when a fully autonomous artefact is responding to the tasks and activities for which the system is designed to support it. adaptability consciousness (ac)—this formative sub-construct measures a user’s conscious thoughts when a fully autonomous artefact is adapting to the tasks and activities for which the system is designed to support it. 4.3 face and content validity to confirm the face and content validity of the proposed scale (bohrnstedt 1970; broder et al. 2007), the theoretical constructs alongside the proposed items were reviewed by 10 senior is academics who had knowledge of survey design and autonomous technologies. feedback from these academics was used to identify the items that were confusing, redundant, or concepts that were overlooked, and to exclude the unwanted item from the scale. while several minor adjustments were made to the wording of items, some examples of the major changes are reported here. for instance, experts reported that items for conscious use sub-constructs were very broad not matching the construct’s definition. hence, the original phrasing of the australasian journal of information systems khan & feroz 2024, vol 28, research article “use” as a conscious thought 19 conscious use sub-constructs “[name of the robot] senses my needs…” were replaced with “[name of the robot] senses hazards…” to capture the actual user needs (of detecting hazards) that the autonomous is artefact is addressing. similarly, the item “[name of the robot] is at my service” belonging to conscious use was replaced with “i know [name of the robot] is present” and one new item “i understand [name of the robot] is around” was added to conscious use. similarly, one new item “[name of robot] senses new conditions” was added to sc, rc, ac sub-constructs. these changes led to leaving conscious use with 4 items and sc, rc, and ac with 5 items each. following that, a sample of 130 business professionals enrolled in executive programs at a large public institution in new zealand were given the revised scale. overall, 60 completed the survey, resulting in a 46% response rate (see appendix b for sample demographics). before filling out the online survey, the respondents were asked to observe the working mechanism of autonomous is artefact by watching a purpose-made video. the video portrays the artefact roaming in the supermarket isles sensing hazards (such as spills and fallen objects), responding to the hazards by alerting the staff and adapting to the environment by avoiding obstacles. in each section of the survey, the definition of the constructs was first presented, followed by measurement items corresponding to the construct. 4.4 survey data collection in the next stage, the participants were asked to “rank the degree to which each item matches the variable's definition” (content validity) (davis 1989, p. 324) on a 5-point likert scale setup through the qualtrics surveying tool. the participants ranked the items and provided written comments about the clarity and appropriateness of the wording of the items and suggested new items if any (face validity) (broder et al. 2007). demographic category count % age group 20 and below 17 14.3 21-30 years 63 53.4 31-40 years 29 24.4 41 years and above 10 8.4 tenure less than 1 year 35 29.4 between 1 to 2 years 29 24.4 between 2 to 3 years 24 20.2 greater than 3 years 31 26.1 exposure to autonomous is artefact less than 1 year 92 77.3 1 to 2 years 23 19.3 2 to 3 years 4 3.4 gender male 53 44.5 female 66 55.5 education high school degree 24 20.2 associate degree 15 12.6 bachelor degree 48 44.3 master degree 16 13.4 other 16 13.4 table 3. sample characteristics (n = 119) some minor adjustments were made to the wordings of the items resulting in a further refined survey instrument having 22 items representing the constructs identified in table b2 (in appendix b), as well as a series of demographic questions (table b1 in appendix b). the study australasian journal of information systems khan & feroz 2024, vol 28, research article “use” as a conscious thought 20 participant were employees of an international supermarket chain [location and brand name withheld] who have employed autonomous is artefact (fully autonomous robot) used in the pilot study. at the time of the survey, the employees were fully conscious of the artefact operating in the supermarket understanding that it does not require deliberate cognitive efforts, physical contact, and instructions (from the employees) to sense and respond to hazards and adapt to the environment. with the help of the store manager, an online version of the survey was sent to all the employees working in the supermarket chain. all surveys were confidential and did not require the collection of personal information that could uniquely identify any participant. 5 likert-type scale to measure their responses, with 1 representing strongly disagree and 5 strongly agree. overall, 119 useable surveys were returned. the demographic profile of the sample is shown in table 3. 5 results the calculations were done through smartpls (ringle et al., 2015) using a partial least squares (pls) structural equation modelling technique which is less demanding on sample size especially when dealing with several items needed for less developed theories (hair et al., 2019). pls-sem is particularly suited for predicting and developing new theories (hair et al., 2011; hair et al., 2019). pls also offers the flexibility of modelling construct to be either reflective or formative constructs. pls analysis was performed in two steps (1) the reliability and validity statistics were reported both for the formative and reflective constructs; and then (2) assessment of the structural model was carried out. 5.1 reliability and validity of formative constructs to demonstrate reliability and validity of formative constructs, indicator outer weights, the significance of weights, and multicollinearity of indicators statistics were calculated (becker et al. 2012; hair et al., 2019) using the repeated indicator approach and the inner factor weighting scheme employing pls-sem algorithm; which is the recommended method for testing reflective-formative models (becker et al., 2012). for bootstrapping the subsamples were set to 5000. the results are shown in table 4. the outer weights for four items ac1, rc1, rc3, and sc2 were not significant, hence these items were eliminated from the scale. the high levels of multicollinearity in formative constructs is one of the fundamental concerns with their validity (hair et al., 2019; petter et al., 2007). original sample sample mean t-values p-values vif sc1 -> sensibility consciousness 0.27 0.25 2.28 0.01 2.0 sc3 -> sensibility consciousness 0.36 0.34 3.54 0.00 1.8 sc4 -> sensibility consciousness 0.30 0.30 2.17 0.02 2.1 sc5 -> sensibility consciousness 0.29 0.31 2.56 0.01 1.8 rc2 -> responsiveness consciousness 0.34 0.33 3.00 0.00 1.5 rc4 -> responsiveness consciousness 0.29 0.29 2.31 0.01 2.0 rc5 -> responsiveness consciousness 0.53 0.53 4.91 0.00 2.2 ac2 -> adaptability consciousness 0.18 0.18 1.89 0.03 1.8 ac3 -> adaptability consciousness 0.31 0.31 2.19 0.01 2.7 ac4 -> adaptability consciousness 0.34 0.34 2.92 0.00 2.2 ac5 -> adaptability consciousness 0.33 0.33 2.49 0.01 2.6 australasian journal of information systems khan & feroz 2024, vol 28, research article “use” as a conscious thought 21 table 4. formative constructs outer weights and vif to show that our first-order constructs are not highly redundant in forming second-order cu, we performed a variance inflation factor (vif) test. the vif values for the three sub-constructs were all less than 3.0, which is the cut-off point (hair et al., 2019). our formative construct model of cu is highly supported by these low multicollinearity data. the final survey instrument is shown in table 5. constructs mean stdv loading/ outer weights autonomy consciousness (reflective) aco1: i am mindful of__ 4.4 0.6 0.81 aco2: i am aware of __ 4.5 0.6 0.82 aco3: i know __ is present. 4.5 0.6 0.76 aco4: i understand __ is around. 4.6 0.6 0.81 sensibility consciousness (formative) sc1: __ senses hazards correctly. 4.1 0.9 0.27 sc3: __ reliably senses hazards. 4.0 0.8 0.36 sc4: __ flexibly senses hazards. 3.7 0.8 0.30 sc5: __ senses new conditions. 4.0 1.0 0.29 responsiveness consciousness (formative) rc2: __ responds to hazards in a timely fashion. 4.2 0.8 0.34 rc4: __ flexibly responds to hazards. 3.8 0.8 0.29 rc5: __ responds to new conditions. 4.0 0.8 0.53 adaptability consciousness (formative) ac2: __ adapts to hazards in a timely fashion. 4.1 0.8 0.18 ac3: __ reliably adjusts to hazards. 4.1 0.8 0.31 ac4: __ flexibly adapts to hazards. 3.9 0.8 0.34 ac5: __ adapts to new conditions. 4.0 0.9 0.33 satisfaction (reflective) sat1: all things considered, i am very satisfied with __ 4.2 0.8 0.92 sat2: overall, my experience with __is very satisfying. 4.2 0.7 0.90 sat3: overall, i am satisfied with __ 4.3 0.8 0.91 table 5. final instrument and measurement properties 5.2 reliability and validity of reflective constructs the discriminant and convergent validity of the two reflective constructs (i.e., autonomy consciousness and satisfaction) was carried out through the pls-sem algorithm. as seen in table 6, all cronbach's alpha values exceed nunnally (1970)'s recommended cut-off of 0.7. furthermore, good internal consistency is demonstrated by the composite reliability values range from 0.88 to 0.93p which are above the endorsed value of 0.80 (nunnally & bernstein, 1994). furthermore, factor loadings of the measurement items on their respective construct, ranging from .70 to .86, are larger than the cut-off value of 0.70 (fornell & larcker, 1981) supporting the convergent validity of each measured item on the corresponding latent construct. besides, each item loads on its latent construct at the significance level of 0.05. the low heterotrait-monotrait ratio of correlations (henseler et al., 2015) and the cross-loadings support the discriminant validity of the constructs. australasian journal of information systems khan & feroz 2024, vol 28, research article “use” as a conscious thought 22 autonomy consciousness satisfaction cronbach’s alpha 0.81 0.89 composite reliability 0.88 0.93 ave 0.64 0.86 heterotrait-monotrait ratio 0.46 0.22 aco1 0.81 0.13 aco2 0.82 0.20 aco3 0.76 0.10 aco4 0.81 0.17 sat1 0.12 0.92 sat2 0.19 0.90 sat3 0.21 0.91 table 6. reliability, loadings, and cross-loadings for reflective constructs 5.3 test of structural model figure 5. impact of sub-constructs on conscious use (cu). *p < 0.000, **p < 0.005 the test of the structural model includes estimates of the path coefficients and the r2 values. the results (figure 5) from pls indicate that all four first-order conscious use constructs (sc, rc, ac, and aco) have significant paths (sc, b = .35, p < 0.00; rc, b =. 36, p < 0.00; ac, b = .33, p < 0.00, and aco, b = .15, p < 0.005) onto overall conscious use (r2 = .99); indicating that these three formative sub-constructs have a significant in forming the overall conscious use construct. this result supports our conceptualization of conscious use as a formative construct. figure 6 displays the results of predictive validity of the conscious use construct with sat as the dependent variable. as shown in figure 6, cu had a significant impact on sat (b = .70; p < .000) with an r2 of 50%; indicating that the conscious use also has sound predictive validity. australasian journal of information systems khan & feroz 2024, vol 28, research article “use” as a conscious thought 23 figure 6. predictive validity of overall cu on sat. *p < 0.000, **p < 0.01 finally, a set of variables, including age, gender, exposure (number of years since the robot is known to the respondents), tenure (the respondent’s spent years with the current organization), and education were included as control variables in the pls models (figure 7). none of the control variables were found to be significant. the t-test results for the gender difference show that the relationship is not significantly different due to gender (b = .10; t = .98). figure 7. predictive validity of cu on the sat with control variables. *p < 0.000, **p < 0.001 6 discussion in this research, we advanced an alternative understanding of use in autonomous things and proposed a novel construct: conscious use. we argue that unlike the manually operated is artefacts, the “use” of a fully autonomous artefact is the domain of conscious thoughts rather than a deliberate cognitive and physical activity. this understanding is advanced based on theories of consciousness (baars 1988; rosenthal, 1996) combined with the way a typical fully autonomous artefact s function i.e., by anticipating user’s needs through sensory information australasian journal of information systems khan & feroz 2024, vol 28, research article “use” as a conscious thought 24 and situational awareness without requiring any instructions and physical contact (hoffman, 2012; schuetz and venkatesh, 2020). the traditional notions of technology adoption conceptualize “use” as physical activity (i.e., learning and physically operating a vacuum cleaner) (adams et al., 1992; delone and mclean, 2003; marangunić and granić, 2015; seddon, 1997; venkatesh et al., 2003); whereas, we advanced the notion of “use” as a conscious thought (a state of consciousness). we showed that conscious thoughts related to a faisa lead to satisfaction; however, unlike physical use, thoughts have no limits. any conscious thoughts the user happens to have when attention is directed towards a fully autonomous artefact will qualify as conscious use including happy thoughts, anger, anxiety, enjoyment, so forth. hence, it is crucial to understand users’ conscious thoughts regarding an autonomous technology (not the physical use) as it may determine users’ satisfaction/ dissatisfaction, purchase decisions, adoption/ rejection, so forth. although, the link between conscious thoughts and emotions need to be tested in future research, the implication here is that fully autonomous technologies will need to be designed to function reliably and smoothly (without human intervention) otherwise it may trigger an undesirable state of consciousness or dissatisfying thoughts leading to anger, anxiety, etc. further validating and extending the proposed notion will profoundly increase our understanding of the role of conscious thoughts in accepting or rejecting autonomous things that do not require any cognitive efforts, physical contact, and instructions to operate. unlike the traditional notions of use, the use of faisa does not require thoughtful mental efforts, physical contact/actions, and instruction to produce outcomes, hence it is not characterized by the behavioural beliefs (e.g., perceived self-efficacy and effort expectancy) that stems from cognitive and physical efforts required to operate a manual is artefact. a question arises, how would then the existing nature of casualties among core is variables come into play when faced with is artefacts that do not require any cognitive efforts, instructions, and physical use? more research is needed to investigate the nature of causal linkage among conscious use and the well-established is behavioural beliefs (such as perceived ease of use and perceived self-efficacy) and attitudes (e.g., intentions to use) (adams et al., 1992). such investigations will reveal meaningful insights and generate implications both for academics and practitioners alike. furthermore, it is well established that our consciousness plays a greater role in having perceptions and feelings (marchetti, 2018), how would then the conscious experiences arising from using a faisa lead to the forming beliefs and perceptions related to the artefact? 6.1 nature of use looking at the continuum (see figure 2), several plausible scenarios may arise in addition to the ones portrayed in this article. for instance, autonomous things come in a variety of forms, shapes, and states: they can either be visible to naked eyes (such as autonomous robots), invisible (e.g., blended in the background or implanted in the body), digital (such as software products), or physical (such as a vacuum cleaner), wearable, or ridable (such as autonomous cars or drones). furthermore, a faisa may exhibit state transition property switching from a fully autonomous state to a partial or non-autonomous state. for example, a self-driving vehicle can transit from a fully autonomous state to a manually controlled state when the need arises (politis et al., 2018). a variety of strategies and design elements are suggested to facilitate this transition including rich displays (eriksson et al., 2017) and dialogue interaction system (politis et al., 2018). australasian journal of information systems khan & feroz 2024, vol 28, research article “use” as a conscious thought 25 however, in this research, we are only focused on faisa that does not require reflective cognitive efforts and physical use/contact and instructions to produce outcomes in all situations permanently. future research (see figure 8) should look into autonomous is artefact s that fit into more than one scenario on the continuum (such as voice-enabled autonomous is artefact or autonomous artefact requiring cognitive and physical efforts). and how would the nature of conscious use change (vary) in the transitional stages of faisa i.e., when it is switching from a fully autonomous state to a partial or non-autonomous state? figure 8.conceptualization of future research directions 6.2 use in different contexts and levels information systems use has been extensively investigated in a variety of levels, such as individuals and group level, and contexts, including business context, domestic, education, healthcare, military, and cultural contexts, to name a few. likewise, the context in which faisa is consciously used is of paramount importance. autonomous technologies are slowly becoming available due to the rapid developments and use of facial recognition, big data, automation, and artificial intelligence revolutionizing the retail industry with innovative products and services (such as unmanned shops, drones, and virtual agents). hence, more research is needed to shed light on the use of faisa in different contexts (such as home vs., business context) and levels (such as individual vs., group settings) to document fluctuates and the manifestation of conscious use in these contexts. furthermore, similar to habitual (jasperson et al., 2005; kim et al., 2005), low attention/intension use (dix, 2017), and system 1 thinking (kahneman, 2012), being conscious of a fully functional faisa doesn’t constitute too much mental effort for users, which may naturally lead to cognitive ease. cognitive ease is the mental state in which “things are going well—no threats, no major news, no need to redirect attention or mobilize effort” (kahneman 2012, p. 61). users experience cognitive ease when there is less information processing strain on the brain and things are generally going well. conversely, users experience cognitive strain when processing too much information and solving complex tasks such as operating a manual technology artefact (moody, 2004). australasian journal of information systems khan & feroz 2024, vol 28, research article “use” as a conscious thought 26 focus background potential research questions impact and association of core is variables with conscious use. the use (and no use) of is artefact is influenced by behavioural beliefs and attitudes (e.g., self-efficacy, effort expectancy, ease of use and usefulness, and intentions), system characteristics, and personality traits (delone & mclean, 2003; lee et al., 2003; taylor and todd, 1995; venkatesh et al., 2003). given that no cognitive efforts and physical use are involved in faisa, investigating the relations among conscious use and other is variables will reveal meaningful insights and generate implications. any conscious thoughts the user happens to have when attention is directed towards a fully autonomous will qualify as conscious use (including happy thoughts, anger, anxiety, enjoyment, so forth) making conscious use a challenging and dynamic construct to measure.\ q1. how would the existing notions of casualties among is variables (e.g., selfefficacy, effort expectancy, ease of use, usefulness, and intentions) come into play in the context of the autonomous is artefact s that do not require any logical and deliberate cognitive efforts and physical actions? q2. how would the conscious experiences and thoughts arising from using a faisa lead to the forming beliefs and perceptions related to the artefact? q3. in what ways do different conscious thoughts ((such as happy thoughts, anger, anxiety, enjoyment, so forth) related to faisa impact users’ satisfaction? nature of use in different forms of faisa. faisa comes in a variety of forms, shapes, and states (figure 2). it can either be visible to naked eyes, invisible (blended in the background or implanted in the body), digital (such as a software product), physical (such as a vacuum cleaner), wearable, or ridable. faisa may exhibit state transition property switching from a fully autonomous state to a partial or non-autonomous state (politis et al., 2018). q4. what is the nature of conscious use in different forms of faisa such as digital vs., physical and invisible vs., invisible faisa? do such scenarios render the need to measure the use of an invisible faisa absolute? q5. is a digital faisa (such as a software product) consciously perceived and used differently when compared to a physical faisa (such as a vacuum cleaner)? q6. how would the nature of conscious use change (vary) in the transitional stages of faisa i.e., when it is switching from a fully autonomous state to a partial or nonautonomous state? use of faisa in different contexts and levels. information systems use has been extensively investigated in a variety of levels, such as individuals and group level, and contexts, such as business context, domestic context, education context, healthcare, military, and cultural contexts, to name a few. the role of technologies in bridging the digital divide is apparent (andrade and doolin, 2016). faisa is not characterized by behavioural beliefs that stem from cognitive and physical use, it could potentially break down the barriers of the digital skills divide (i.e., the lack of skills needed to operate the technology) particularly among the less privileged groups and indigenous communities. q7. given that use in faisa is an individualistic conscious thought, how does this notion of use change in group settings (e.g., when faisa is used collaboratively rather than individually)? q8. how is the nature of conscious thoughts manifested in different contexts in which faisa is used such as business vs., domestic context? q9. in what ways the notion of conscious use can be best leveraged to break down the barriers of the digital skills divide particularly among the less privileged groups and indigenous communities? australasian journal of information systems khan & feroz 2024, vol 28, research article “use” as a conscious thought 27 focus background potential research questions nature of use in partial autonomous things. as alluded to earlier, paisa does not need physical use (e.g., push or click) but requires cognitive, vocal, and gestural instructions wirelessly communicated through a variety of mechanisms (e.g., see (ernst, 2020; musk, 2019; sen et al., 2015). examples of paisa include voice commanded systems (e.g., siri and alexa) (sen et al., 2015); systems that are controlled with thoughts (e.g., elon musk's neuralink) (musk, 2019), and gestures enabled systems (e.g., selfietype, a gesturebased digital keyword developed by samsung). q10. how would the existing notions of casualties among is variables come into play in the context of the autonomous is artefact s that are used through cognition, voice, and gestures? q11. what is the nature of ease of use, usefulness, and use in the partially autonomous artefact s that do not need physical use (e.g., push or click) but require cognitive, vocal, and gestural instructions wirelessly communicated? table 6. future research directions relating to use of autonomous things following this logic, it can be argued that when users are consciously using a faisa that does not require logical and reflective cognitive efforts, physical contact, and instructions they may experience cognitive ease due to the less information processing strain on their brain. hence, future research investigate the impact of conscious use on cognitive ease—which we define as is an individualist state of mental ease experienced when a fully autonomous artefact is carrying out tasks and activities for which the system is designed to support. in addition, it is well established that when people are in a state of cognitive ease they are generally happy and comfortable whereas when people feel strained, they are more likely to be suspicious, feel uncomfortable, and less creative (kahneman, 2012). as posited earlier, the autonomous nature of faisa reduces information processing strain on the brain leading to potential cognitive ease, which could further impact users' modes, feelings, and creativity. it has also been established that people are more inclined to interact with new technology if they believe the interaction will require little cognitive work (adams et al., 1992). future research is needed to further understand this association and investigate in what ways and how faisa impacts users’ feelings, modes, and creativity. furthermore, previous research suggests that people may feel uneasy in the presence of sophisticated technology, and it causes intense anxiety among onlookers (jayaraman et al., 2019; stein et al., 2019). this aversion to advance technologies can naturally arise from “sociocultural constructions and biological adaptations for threat avoidance” (macdorman & entezari 2015, p. 141). for example, złotowski et al. (2017) reported that people perceive autonomous machines as a threat not only to lose human identity/ distinctiveness (identity threats) but also as a threat to material resources, safety, and physical wellbeing (i.e., realistic threat). similarly, using a virtual reality agent, stein et al. (2019) proposed a model of autonomous technology threat and showed that proximal threat (concerns about human uniqueness) and distal threat (perceived situational control in interactions with autonomous technology) plays a crucial role in technology aversion and “reduced” technology acceptance. employing a modified technology adoption model in the context of the autonomous vehicle, hewitt et al. (2019), among other things, reported that users are eager to use fully autonomous cars, however, their lack of trust and anxiety associated with it negatively impacted intention to use the self-driving cars. on the other hand, studies suggest that the users particularly australasian journal of information systems khan & feroz 2024, vol 28, research article “use” as a conscious thought 28 appreciate the time savings that come with faisa. zeitzew (2007) investigated the use of autonomous utility mowers and suggest that users appreciated the straightness of the mowing stripes and time savings which could potentially be invested in other activities (such as leisure time). hence, conscious use may lead to the autonomous technology acceptance paradox—that autonomous is artefact s may both increase and diminish technology acceptance and/or cognitive ease particularly when users are faced with sophisticated autonomous machines. such contradiction has been previously reporting in the context of conventional technologies (i.e., mobile email devices) when users juggle between personal interest and professional commitment (mazmanian et al. 2013). this warrants testing the notions of conscious use and cognitive ease in other studies when investigating issues related to technology fear, aversions, trust, and privacy. furthermore, the role of technologies in bridging the digital divide has been widely studied (andrade & doolin, 2016). the nature of faisa and the notion of conscious use has the potential to reshape the digital divide literature. when a technology is not characterized by behavioural beliefs that stem from cognitive and physical efforts (e.g., perceived self-efficacy and effort expectancy), it could potentially break down the barriers of the digital skills divide (need to operate the technology) particularly among less privileged groups and indigenous communities. like the traditional technologies (andrade & doolin, 2016), future research is needed to investigate how autonomous technologies and the notion of conscious use can be best leveraged to close the technology skills gap and uplift less privileged groups and indigenous communities across the globe. 6.3 nature of use in partial autonomous things the availability of partially autonomous things controlled with voice (such as apple’s siri, amazon’s alexa, google’s assistance, microsoft’s cortana, and samsung’s s voice) has become a common household item offering users hands and eyes-free interaction with systems while eliminating the need for keyboards. similarly, cognition or thought-based technologies (such as “alterego” designed by a researcher at the massachusetts institute of technology) decode thoughts and wirelessly transmit them to another device (such as a computer) to perform certain tasks (e.g., typing a message without a keyboard with thoughts only) (boucha et al., 2017). gesture-enabled products are also available in the market, such as the gesture sensing stunt car that is controlled with gestures transmitted through a wearable strap wrapped around the palm. research suggests that one of the greatest challenges facing gesture-enabled technologies is not technological but it is related to acceptance of gestural inputs (ronkainen et al., 2007). there are a variety of issues that must be considered when theorizing and testing partially autonomous technology-related theories and constructs. however, reliable measures did not exist to understand the use associated with this new genre of is artefacts. future research should propose and validate a novel construct and its measures for partially autonomous is artefact s that do not require contact but need instructions in the form of speech (such as alexa), thoughts, and gestures. for example, what is the nature of ease of use, usefulness, and use in the partially autonomous artefact s that do not need physical use (e.g., push or click) but requires cognitive, vocal, and gestural instructions wirelessly communicated? partially autonomous things can be wearables (i.e., worn on the body) and capable of collecting, analysing, and transmitting data related to the user’s vital signs (e.g., pulse, temperature) and activities (e.g., steps, sleep, physical exercise) (mettler & wulf 2019). what is the nature of use australasian journal of information systems khan & feroz 2024, vol 28, research article “use” as a conscious thought 29 and ease of use in wearable technologies and how does user interaction with wearables enable behavioural and cognitive outcomes? 6.4 contributions to theory and practice we believe this research made an important contribution by further advancing the emerging is strain of research that questions the core assumptions of theorizing the use of is artefact s on the premise of physical use (demetis & lee, 2018; rivard, 2014; schuetz & venkatesh, 2020). the proposed theorization and construct are in agreement with the emerging viewpoints that argue for novel theories for the next generation of things (schuetz & venkatesh, 2020, p. 461) as the existing theories were developed for a different genre of end-users technologies (benbasat & zmud, 2003; rivard, 2014). in line with the is custom of advancing new understanding for the emerging technologies (davis, 1989), we believe that the constructs proposed in this research will have greater theoretical value for academics and practical benefits for the businesses who are keen to understand end-user perceptions and use related to autonomous things. the paper also developed a continuum of autonomy of is artefact s. given the constellations of numerous emerging technology artefact s equipped with new features and abilities, the taxonomy of autonomous technologies developed in this study can potentially make a significant contribution to is research and practice. this research opens up several avenues for future research on a variety of fronts to further expand our understanding of the nature of use in autonomous things (see figure 8 and table 6). in terms of practical implications, this paper highlights several points that relate to the prospective use of autonomous things in organizations. first of all, the efficiency of organizational processes and value creation mechanisms are considered important aspects that conscious use can potentially have an impact on. given the speed of digital transformation of organizational processes (feroz et al., 2023), autonomous is is set to become integral to modern organizational landscapes, revolutionizing the way tasks are performed, decisions are made, and data is managed paving way for heightened operational efficiency that not only enhances productivity but also allows for resource optimization (davenport, 2019). in this regard, this paper introduces a human-centric conscious-driven dimension to the deployment of autonomous systems, influencing how individuals interact with and perceive these technologies. by acknowledging the role of conscious use, organizations can better tailor autonomous systems to complement human decision-making, thereby maximizing operational benefits. conscious use, in this context, emphasizes the need for users to be consciously aware of how they engage with autonomous things, ensuring that the convergence of human-is interaction lead to alignment with organizational goals and values (smith et al., 2020). in organizational behavior context, the successful implementation of autonomous things is closely tied to user acceptance and positive experiences (venkatesh et al., 2021). conscious use can contribute to user satisfaction by aligning system functionalities with user expectations through minimal interaction with the autonomous is. understanding the practical implications of autonomous things is crucial for organizations seeking to leverage these technologies effectively. the paper introduces the concept of conscious use, a human-centric dimension to the deployment of autonomous systems, that can influence how individuals interact with and perceive these technologies. organizations can utilize the influence of conscious use on the user experience australasian journal of information systems khan & feroz 2024, vol 28, research article “use” as a conscious thought 30 to enhance overall system acceptance and foster a positive attitude toward autonomous technologies. 6.5 limitations finally, the research has several limitations that need to be addressed. firstly, the proposed notion of conscious use assumes that fully autonomous technologies do not require deliberate cognitive efforts and active use behaviour. however, most autonomous artefact s are designed for humans to engage in verbal and physical human-machine interaction and cooperation at various levels of automation (vagia et al., 2016). for example, even with a fully autonomous vehicle, the passengers might have to verbally engage with the vehicle when it comes to providing a destination, preferred route, or when enabling the vehicle’s fully autonomous capability in the first place. however, the construct and conceptualisation proposed in this study are only relevant when users are faced with an is artefact that is operating in a fully autonomous mode not requiring deliberate cognitive efforts, physical contact, and instructions to produce outcomes in all situations permanently. in such cases, the use of a fully autonomous artefact becomes a conscious thought directed towards the technology rather than a deliberate physical activity. hence, future research is needed to propose and validate a novel construct and its measures for autonomous is artefact s that require human-machine interaction and cooperation at various levels of automation across a continuum of fully autonomous state to a partial or nonautonomous state. secondly, although we examine using a typical autonomous technology that senses, responds, and adapts to the environment, further research is necessary to evaluate the utility of our measure in more complex autonomous technology contexts. for example, we only looked into use associated with a faisa assuming that the users are aware of its presence and attention is directed towards it. however, there may arise scenarios when a user is not conscious or aware of a faisa that is anticipating their needs (such as chatbots, recommender systems, or products embedded in the environment) (dix, 2017; schuetz & venkatesh, 2020). future research is needed to understand the nature of conscious use in different forms of faisa such as digital vs., physical and invisible vs., invisible faisa. and how such scenarios may render the need to measure “use” and of an invisible faisa absolute. as discussed earlier, any conscious thoughts the user happens to have when attention is directed towards a fully autonomous will also qualify as conscious use including anger, anxiety, enjoyment, so forth. future research needs to investigate the variety of conscious thoughts that may arise in different situations such as when a faisa is not performing its job smoothly. furthermore, the proposed theorization does not consider the initial setup requirements and unboxing experience related to autonomous things, as one expert commented during the pilot study: “i think it's important to think about not just the actual application of products, but also consider the initial setup and unboxing experiences of the product. the echo dot for example requires users to download an app to register the product as well as add devices to their alexa cloud services. this is not always easy to do interface and requires some hands-on with hardware.” future research is needed to consider issues relating to initial setup requirements and unboxing experience when investigating autonomous things. similarly, the small sample size and cross-sectional nature of the study may limit its generalizability. and finally, a better approach to clearly demonstrating that the proposed notion of use is separate and distinct from other previous types of use is to measure it alongside existing use measures and present australasian journal of information systems khan & feroz 2024, vol 28, research article “use” as a conscious thought 31 the empirical analysis. however, this was not done in this research. future research should retest the proposed measures alongside the existing use measures. 7 conclusion leveraging the theories of consciousness, we proposed conscious use as a novel construct for use in autonomous things and empirically validated its measures and predictive validity. we showed that when an is artefact does not require deliberate cognitive efforts, physical contact, and instruction (to produce outcomes), its “use” essentially becomes a conscious thought rather than deliberate cognitive efforts or physical activity (i.e., pushing a button or clicking on a screen). the autonomous capabilities of next generation artefact s will have an impact on the way humans interact with and manage “consciously aware” computers. departure from deliberate cognitive behaviour to conscious thoughts when interacting with the intelligent machines creates new horizons in terms of human-machine interface which will ultimately lead to the transformation of industries, societies, and organizations. more research is needed to expand the findings of this study in a variety of different settings provided in table 6. extending the findings of this study will make a crucial theoretical and practical contribution to is research. acknowledgements this research was partially funded by the zayed university research incentive fund (activity code: 23065). references adams, d. a., nelson, r. r., & todd, p. a. (1992). perceived usefulness, ease of use, and usage of information technology: a replication. mis quarterly, 16(2), 227–247. https://doi.org/10.2307/249577 agarwal, r., & karahanna, e. (2000). time flies when you’re having fun: cognitive absorption and beliefs about information technology usage. mis quarterly, 24(4), 665–694. https://doi.org/10.2307/3250951 ahuja, m. k., & thatcher, j. b. (2005). moving beyond intentions and toward the theory of trying: effects of work environment and gender on post-adoption information technology use. mis quarterly, 29(3), 427–459. https://doi.org/10.2307/25148691 ajibade, p. (2018). technology acceptance model limitations and criticisms: exploring the practical applications and use in technology-related studies, mixed-method, and qualitative researches. library philosophy and practice (e-journal), 1941. retrieved from http://digitalcommons.unl.edu/libphilprac/1941 albus, j., & antsaklis, p. j. (1998). panel discussion: autonomy in engineering systems: what is it and why is it important? setting the stage: some autonomous thoughts on autonomy. in proceedings of the 1998 ieee international symposium on intelligent control (isic) held jointly with ieee international symposium on computational intelligence in robotics and automation (cira) intell (pp. 520–521). andrade, a. d., & doolin, b. (2016). information and communication technology and the social inclusion of refugees. mis quarterly, 40(2), 405–416. retrieved from https://www.jstor.org/stable/26628912 australasian journal of information systems khan & feroz 2024, vol 28, research article “use” as a conscious thought 32 arents, j., abolins, v., judvaitis, j., vismanis, o., oraby, a., & ozols, k. (2021). human–robot collaboration trends and safety aspects: a systematic review. journal of sensor and actuator networks, 10(3), 48. https://doi.org/10.3390/jsan10030048 atzori, l., iera, a., & morabito, g. (2010). the internet of things: a survey. computer networks, 54(15), 2787-2805. https://doi.org/10.1016/j.comnet.2010.05.010 baars, b. (1988). a cognitive theory of consciousness. cambridge, uk: cambridge university press. becker, j.-m., klein, k., & wetzels, m. (2012). hierarchical latent variable models in pls-sem: guidelines for using reflective-formative type models. long range planning, 45(5–6), 359–394. https://doi.org/10.1016/j.lrp.2012.10.001 benbasat, i., & zmud, r. w. (2003). the identity crisis within the is discipline: defining and communicating the discipline’s core properties. mis quarterly, 27(2), 183–194. https://doi.org/10.2307/30036527 bimbraw, k. (2015). autonomous cars: past, present and future a review of the developments in the last century, the present scenario and the expected future of autonomous vehicle technology. in proceedings of icinco 2015 12th international conference on informatics in control, automation and robotics (pp. 191–198). bohrnstedt, g. w. (1970). reliability and validity assessment in attitude measurement. in g. f. summers (ed.), attitude measurement. skokie, il: rand mcnally. borriello, g. (2000). the challenges to invisible computing. ieee computer, integrated engineering column, 33(11), 123-125. boucha, d., amiri, a., & chogueur, d. (2017). controlling electronic devices remotely by voice and brain waves. in 2017 international conference on mathematics and information technology (icmit) (pp. 38-42). adrar. https://doi.org/10.1109/mathit.2017.8259693. broadbent, e. (2017). interactions with robots: the truths we reveal about ourselves. annual review of psychology, 68, 627-652. https://doi.org/10.1146/annurev-psych-010416-043958 broder, h. l., mcgrath, c., & cisneros, g. j. (2007). questionnaire development: face validity and item impact testing of the child oral health impact profile. community dentistry and oral epidemiology, 35, 8–19. brown, s. a., dennis, a. r., & venkatesh, v. (2010). predicting collaboration technology use: integrating technology adoption and collaboration research. journal of management information systems, 27(2), 9–54. burton-jones, a., & straub, d. w. (2006). reconceptualizing system usage: an approach and empirical test. information systems research, 17(3), 228–246. http://www.jstor.org/stable/23015887 chuttur, m. (2009). overview of the technology acceptance model: origins, developments and future directions. in all sprouts content (p. 290). https://aisel.aisnet.org/sprouts_all/290 australasian journal of information systems khan & feroz 2024, vol 28, research article “use” as a conscious thought 33 dautenhahn, k. (2007). socially intelligent robots: dimensions of human-robot interaction. philosophical transactions of the royal society b: biological sciences, 362(1480), 679-704. https://doi.org/10.1098/rstb.2006.2004 davis, f. d. (1989). perceived usefulness, perceived ease of use, and user acceptance of information technology. mis quarterly, 13(3), 319–340. https://doi.org/10.2307/249008 davenport, t. h. (2019). the ai advantage: how to put the artificial intelligence revolution to work. mit press. https://mitpress.mit.edu/9780262538008/the-ai-advantage/ del aguila-obra, a. r., & padilla-meléndez, a. (2006). organizational factors affecting internet technology adoption. internet research, 16, 94-110. https://doi.org/10.1108/10662240610642569 de melo, c. m., marsella, s., & gratch, j. (2019). human cooperation when acting through autonomous machines. proceedings of the national academy of sciences of the united states of america, 116(9), 3482–3487. https://doi.org/10.1073/pnas.1817656116 delone, w., & mclean, e. (2003). the delone and mclean model of information systems success: a ten-year update. journal of management information systems, 19, 9-30. https://doi.org/10.1080/07421222.2003.11045748 demetis, d., & lee, a. (2018). when humans using the it artifact becomes it using the human artifact. journal of the association for information systems, 19, 929-952. https://doi.org/10.17705/1jais.00514 dey, a. (2001). understanding and using context. personal and ubiquitous computing, 5, 4-7. https://doi.org/10.1007/s007790170019 dhar, r., & gorlin, m. (2013). a dual-system framework to understand preference construction processes in choice. journal of consumer psychology, 23(4), 528–542. https://doi.org/10.1016/j.jcps.2013.02.002 de guinea, a. o., & markus, m. l. (2009). why break the habit of a lifetime? rethinking the roles of intention, habit, and emotion in continuing information technology use. mis quarterly, 33(3), 433–444. https://doi.org/10.2307/20650303 desai, m., stubbs, k., steinfeld, a., & yanco, h. (2017). creating trustworthy robots: lessons and inspirations from automated systems. acm transactions on interactive intelligent systems (tiis), 7(4), 1–24. dijksterhuis, a., & nordgren, l. f. (2006). a theory of unconscious thought. perspectives on psychological science, 1(2), 95–109. https://doi.org/10.1111/j.1745-6916.2006.00007.x dix, a. (2017). activity modelling for low-intention interaction. in b. weyers, j. bowen, a. dix, & p. palanque (eds.), the handbook of formal methods in human-computer interaction (pp. 183–210). cham: springer international publishing. endsley, m. r. (1995). toward a theory of situation awareness in dynamic systems. human factors, 37(1), 32–64. https://doi.org/10.1518/001872095779049543 endsley, m. r. (1999). level of automation effects on performance, situation awareness and workload in a dynamic control task. ergonomics, 42(3), 462–492. australasian journal of information systems khan & feroz 2024, vol 28, research article “use” as a conscious thought 34 endsley, m. r. (2017). autonomous driving systems: a preliminary naturalistic study of the tesla model s. journal of cognitive engineering and decision making, 11(3), 225–238. https://doi.org/10.1177/1555343417695197 eriksson, a., petermeijer, s. m., zimmermann, m., de winter, j. c., bengler, k. j., & stanton, n. a. (2017). rolling out the red (and green) carpet: supporting driver decision making in automation-to-manual transitions. ieee transactions on human-machine systems, 49(1), 20–31. ernst, c. (2020). artificial intelligence and autonomy: self-determination in the age of automated systems. in t. w. & t. rademacher (eds.), regulating artificial intelligence (pp. 53–73). cham, switzerland: springer international publishing. feroz, a. k., zo, h., eom, j., & chiravuri, a. (2023). identifying organizations’ dynamic capabilities for sustainable digital transformation: a mixed methods study. technology in society, 73, 102257. https://doi.org/10.1016/j.techsoc.2023.102257 gerlach, j. h., & kuo, f.-y. (1991). understanding human-computer interaction for information systems design. mis quarterly, 15(4), 527–549. https://doi.org/10.2307/249456 goodrich, m., & schultz, a. (2007). human-robot interaction: a survey. foundations and trends in human-computer interaction, 1, 203-275. http://doi.org/10.1561/1100000005 hair, j. f., ringle, c. m., & sarstedt, m. (2011). pls-sem: indeed a silver bullet. journal of marketing theory and practice, 19(2), 139–152. hair, j. f., risher jeffrey, j., sarstedt, m., & ringle christian, m. (2019). when to use and how to report the results of pls-sem. european business review, 31(1), 2–24. henningsson, s., & kettinger, w. j. (2016). understanding information systems integration deficiencies in mergers and acquisitions: a configurational perspective. journal of management information systems, 33(4), 942–977. https://doi.org/10.1080/07421222.2016.1267516 henseler, j., ringle, c. m., & sarstedt, m. (2015). a new criterion for assessing discriminant validity in variance-based structural equation modeling. journal of the academy of marketing science, 43(1), 115–135. henley, s. h. a. (1984). unconscious perception re-revisited: a comment on merikle’s (1982) paper. bulletin of the psychonomic society, 22, 121–124. https://doi.org/10.3758/bf03333780 hewitt, c., politis, i., amanatidis, t., & sarkar, a. (2019). assessing public perception of selfdriving cars: the autonomous vehicle acceptance model. in proceedings of the 24th international conference on intelligent user interfaces (pp. 518–527). marina del ray, california: association for computing machinery. hinkin, t. r. (1995). a review of scale development practices in the study of organizations. journal of management, 21(5), 967-988. https://doi.org/10.1016/0149-2063(95)90050-0 hoffman, g. (2012). embodied cognition for autonomous interactive robots. topics in cognitive science, 4, 759-772. https://doi.org/10.1111/j.1756-8765.2012.01218.x https://doi.org/10.2307/249456 australasian journal of information systems khan & feroz 2024, vol 28, research article “use” as a conscious thought 35 hossain, m. a., atrey, p. k., & saddik, a. e. (2007). smart mirror for ambient home environment. in iet conference proceedings (pp. 589–596). institution of engineering and technology. doi: 10.1049/cp:20070431 im, i., hong, s., & kang, m. s. (2011). an international comparison of technology adoption: testing the utaut model. information & management, 48, 1-8. https://doi.org/10.1016/j.im.2010.09.001 inagaki, t., & sheridan, t. (2019). a critique of the sae conditional driving automation definition, and analyses of options for improvement. cognition, technology & work, 21. https://doi.org/10.1007/s10111-018-0471-5 ng, i., & wakenshaw, s. (2016). the internet-of-things: review and research directions. international journal of research in marketing, 34. https://doi.org/10.1016/j.ijresmar.2016.11.003 jasperson, j., carter, p. e., & zmud, r. w. (2005). a comprehensive conceptualization of postadoptive behaviors associated with information technology enabled work systems. mis quarterly, 29(3), 525–557. jarvis, c., burke, m., mackenzie, s., & podsakoff, p. m. (2003). a critical review of construct indicators and measurement model misspecification in marketing and consumer research. journal of consumer research, 30(2), 199–218. jayaraman, s. k., creech, c., tilbury, d. m., yang, x. j., pradhan, a. k., tsui, k. m., & robert, l. p. (2019). pedestrian trust in automated vehicles: role of traffic signal and av driving behavior. frontiers in robotics and ai, 6, 117. kabat-zinn, j. (1994). wherever you go, there you are: mindfulness meditation in everyday life. new york, usa: hyperion. kahneman, d. (2012). thinking, fast and slow. london, united kingdom: penguin books ltd. khan, g. f., & trier, m. (2019). assessing the long-term fragmentation of information systems research with a longitudinal multi-network analysis. european journal of information systems, 28(4), 370–393. https://doi.org/10.1080/0960085x.2018.1547853 kim, s. s., malhotra, n. k., & narasimhan, s. (2005). research note—two competing perspectives on automatic use: a theoretical and empirical comparison. information systems research, 16(4), 418–432. kleinberg, j. (2003). bursty and hierarchical structure in streams. data mining and knowledge discovery, 7(4), 373–397. kramer, a., guillory, j., & hancock, j. (2014). experimental evidence of massive-scale emotional contagion through social networks. proceedings of the national academy of sciences of the united states of america, 111. https://doi.org/10.1073/pnas.1320040111 kumar, a., metze, f., & kam, m. (2014). enabling the rapid development and adoption of speech-user interfaces. computer, 47(01), 40-47. https://doi.org/10.1109/mc.2014.11 kwak, s. s., kim, j. s., & choi, j. j. (2017). the effects of organismversus object-based robot design approaches on the consumer acceptance of domestic robots. international journal of social robotics, 9(3), 359–377. australasian journal of information systems khan & feroz 2024, vol 28, research article “use” as a conscious thought 36 lee, a., thomas, m., & baskerville, r. (2014). going back to basics in design science: from the information technology artifact to the information systems artifact. information systems journal, 25. https://doi.org/10.1111/isj.12054 lee, y., kozar, k., & larsen, k. (2003). the technology acceptance model: past, present, and future. technology, 12. https://doi.org/10.17705/1cais.01250 levy, y., & ellis, t. j. (2006). a systems approach to conduct an effective literature review in support of information systems research. international journal of an emerging transdiscipline, 9. leydesdorff, l. (2006). the knowledge-based economy: modeled, measured, simulated. boca raton, fl, usa: universal-publishers. limayem, m., hirt, s. g., & cheung, c. m. k. (2007). how habit limits the predictive power of intention: the case of information systems continuance. mis quarterly, 31(4), 705–737. macdorman, k. f., & entezari, s. o. (2015). individual differences predict sensitivity to the uncanny valley. interaction studies, 16(2), 141–172. marangunić, n., & granić, a. (2015). technology acceptance model: a literature review from 1986 to 2013. universal access in the information society, 14(1), 81–95. marchetti, g. (2018). consciousness: a unique way of processing information. cognitive processing, 19(3), 435–464. mazmanian, m., orlikowski, w. j., & yates, j. (2013). the autonomy paradox: the implications of mobile email devices for knowledge professionals. organization science, 24(5), 1337– 1357. mettler, t., & wulf, j. (2019). physiolytics at the workplace: affordances and constraints of wearables use from an employee's perspective. information systems journal, 29(1), 245– 273. mihaly, c. (1990). flow: the psychology of optimal experience. new york, ny, usa: harper and row. moody, d. l. (2004). cognitive load effects on end user understanding of conceptual models: an experimental analysis. in a. benczúr, j. demetrovics, & g. gottlob (eds.), advances in databases and information systems (pp. 129–143). berlin, heidelberg, germany: springer. murat dikmen, & catherine m. burns. (2016). autonomous driving in the real world: experiences with tesla autopilot and summon. in proceedings of the 8th international conference on automotive user interfaces and interactive vehicular applications (automotive'ui 16) (pp. 225–228). association for computing machinery. musk, e. (2019). neuralink: an integrated brain-machine interface platform with thousands of channels. journal of medical internet research, 21(10), e16194. https://www.jmir.org/2019/10/e16194 doi: 10.2196/16194 noar, s. m. (2003). the role of structural equation modeling in scale development. structural equation modeling: a multidisciplinary journal, 10(4), 622–647. norman, d. a., ortony, a., & russell, d. m. (2003). affect and machine design: lessons for the development of autonomous machines. ibm systems journal, 42(1), 38–44. https://www.jmir.org/2019/10/e16194 australasian journal of information systems khan & feroz 2024, vol 28, research article “use” as a conscious thought 37 nomura, t., kanda, t., suzuki, t., & kato, k. (2008). prediction of human behavior in humanrobot interaction using psychological scales for anxiety and negative attitudes toward robots. ieee transactions on robotics, 24, 442-451. https://doi.org/10.1109/tro.2007.914004. nunnally, j. c. (1978). psychometric theory (2nd ed.). new york, ny, usa: mcgraw-hill. pellenz, j., gossow, d., & paulus, d. (2009). robbie: a fully autonomous robot for robocuprescue. advanced robotics, 23(9), 1159–1177. petter, s., straub, d., & rai, a. (2007). specifying formative constructs in information systems research. mis quarterly, 31(4), 623–656. politis, i., langdon, p., adebayo, d., bradley, m., clarkson, p. j., skrypchuk, l., mouzakitis, a., eriksson, a., brown, j. w., & revell, k. (2018). an evaluation of inclusive dialoguebased interfaces for the takeover of control in autonomous cars. in proceedings of 23rd international conference on intelligent user interfaces (pp. 601–606). ringle, c. m., wende, s., & becker, j.-m. (2015). smartpls 3. bönningstedt: smartpls. retrieved from http://www.smartpls.com. riek, l. d. (2017). healthcare robotics. communications of the acm, 60(11), 68–78. https://doi.org/10.1145/3127874. rivard, s. (2014). editor’s comments: the ions of theory construction. mis quarterly, 38(2), iii–xiv. https://www.jstor.org/stable/26634928 ronkainen, s., häkkilä, j., kaleva, s., colley, a., & linjama, j. (2007). tap input as an embedded interaction method for mobile devices. in proceedings of tei 2007 (pp. 263270). https://doi.org/10.1145/1226969.1227023 rosenthal, d. m. (1996). a theory of consciousness. in o. f. n. block & g. güzeldere (eds.), the nature of consciousness. cambridge, ma, usa: mit press. schomaker, l., & hartung, k. (1995) (eds.). a taxonomy of multimodal interaction in the human information processing system. report esprit project 8579. accessible at https://www.ai.rug.nl/~lambert/projects/miami/taxonomy/taxonomy.html schuetz, s., & venkatesh, v. (2020). the rise of human machines: how cognitive computing systems challenge assumptions of user-system interaction. journal of the association for information systems, 21, 460-482. https://doi.org/10.17705/1jais.00608. sci2team. (2009). science of science (sci2) tool (indiana university and scitech strategies, 2009). retrieved from http://sci2.cns.iu.edu. seddon, p. b. (1997). a respecification and extension of the delone and mclean model of is success. information systems research, 8(3), 240-253. sen, s., chakrabarty, s., toshniwal, r., & bhaumik, a. (2015). design of an intelligent voice controlled home automation system. international journal of computer applications, 121, 39-42. http://www.smartpls.com/ https://doi.org/10.1145/3127874 http://sci2.cns.iu.edu/ australasian journal of information systems khan & feroz 2024, vol 28, research article “use” as a conscious thought 38 smith, a. n., fischer, e., & yongjian, c. (2020). how does artificial intelligence become social? a process model of ai integration into consumer practices. journal of the academy of marketing science, 48(1), 231–250. stein, j.-p., liebold, b., & ohler, p. (2019). stay back, clever thing! linking situational control and human uniqueness concerns to the aversion against autonomous technology. computers in human behavior, 95, 73-82. sun, j., & teng, j. t. c. (2012). information systems use: construct conceptualization and scale development. computers in human behavior, 28(5), 1564-1574. taylor, s., & todd, p. a. (1995). understanding information technology usage: a test of competing models. information systems research, 6, 144-176. thatcher, j. b., wright, r. t., sun, h., zagenczyk, t. j., & klein, r. (2018). mindfulness in information technology use: definitions, distinctions, and a new measure. mis quarterly, 42(3), 831–848. trevino, l. k., & webster, j. (1992). flow in computer-mediated communication: electronic mail and voice mail evaluation and impacts. communication research, 19(5), 539-557. vagia, m., transeth, a. a., & fjerdingen, s. a. (2016). a literature review on the levels of automation during the years. what are the different taxonomies that have been proposed? applied ergonomics, 53, 190-202. venkatesh, v., morris, m., davis, g., & davis, f. (2003). user acceptance of information technology: toward a unified view. mis quarterly, 27, 425-478. venkatesh, v., thong, j. y., & xu, x. (2016). unified theory of acceptance and use of technology: a synthesis and the road ahead. journal of the association for information systems, 17, 328–376. https://doi.org/10.17705/1jais.00428. vial, g. (2019). understanding digital transformation: a review and a research agenda. journal of strategic information systems, 28(2), 118–144. https://doi.org/10.1016/j.jsis.2019.01.003 wang, w., & benbasat, i. (2007). recommendation agents for electronic commerce: effects of explanation facilities on trusting beliefs. journal of management information systems, 23(4), 217-246. https://doi.org/10.2753/mis0742-1222230410. wang, w., & butler, j. e. (2006). system deep usage in post-acceptance stage: a literature review and a new research framework. international journal of business information systems, 1(4), 439–462. wang, w., xu, j., & wang, m. (2018). effects of recommendation neutrality and sponsorship disclosure on trust vs. distrust in online recommendation agents: moderating role of explanations for organic recommendations. decision support systems, 114, 1-15. wilson, m. (2002). six views of embodied cognition. psychonomic bulletin & review, 9(4), 625636. wisdom, j. p., chor, k. h. b., hoagwood, k. e., & horwitz, s. m. (2014). innovation adoption: a review of theories and constructs. administration and policy in mental health and mental health services research, 41(4), 480–502. https://doi.org/10.1016/j.jsis.2019.01.003 australasian journal of information systems khan & feroz 2024, vol 28, research article “use” as a conscious thought 39 wixom, b. h., & todd, p. a. (2005). a theoretical integration of user satisfaction and technology acceptance. information systems research, 16(1), 85–102. wu, c., huang, s., & yuan, q. (2022). seven important theories in information system empirical research: a systematic review and future directions. data and information management, 6(1), 100006. https://doi.org/10.1016/j.dim.2022.100006 you, s., & robert, l. (2018). emotional attachment, performance, and viability in teams collaborating with embodied physical action (epa) robots. journal of the association for information systems, 19, 377-407. https://doi.org/10.17705/1jais.00496. zacharaki, a., kostavelis, i., gasteratos, a., & dokas, i. (2020). safety bounds in human-robot interaction: a survey. safety science, 127, 104667. https://doi.org/10.1016/j.ssci.2020.104667. zeitzew, m. a. (2007). autonomous utility mower. agricultural engineering international the cigr ejournal, ix(july). złotowski, j., yogeeswaran, k., & bartneck, c. (2017). can we control it? autonomous robots threaten human identity, uniqueness, safety, and resources. international journal of human-computer studies, 100, 48-54. https://doi.org/10.1016/j.ijhcs.2016.12.008. appendix a 1. burst detection in this study, we employed kleinberg’s burst detection algorithm (kleinberg, 2003) implemented in the sci2 tool (sci2team 2009) to identify the emerging trends in the autonomous things research domain. the burst detection technique is a reliable way to identify emerging trends in any research domain by analyzing large corpora of text (kleinberg, 2003). researchers from a variety of domains including the information systems domain (khan & trier, 2019) have successfully employed this technique to identify emerging trends in their respective research domains. before applying the burst detection techniques, the data were lowercased, tokenized, and a common set of stop words were removed from it. 2. data the data for burst detection were obtained from the web of science (wos) database. previous research has shown that the keywords and titles of the articles are the best places to identify the emerging trends in a domain (leydesdorff, 2006). hence, we entered a research query into the wos search engine to find the publications (from 2000 to early 2020) with the following topics in the keywords, title, and abstract of the articles. searched for topic: ("autonomous things" or "autonomous systems" or "autonomous products" or "autonomous artefact " or "autonomous robots" or "autonomous machines" or "self-governing things" or "self-governing systems" or "self-governing artefacts" or "self-governing product" or "self-governing robots" or "self-governing machines” timespan: 2000-2020. indexes: social sciences citation index (ssci). the search retrieved 252 articles that had appeared in 156 journals. furthermore, a vast majority of articles (70%) were published since 2011 and the largest number of publications (n=20) appeared in adaptive behavior journal. this was followed by 9 (3.6%) articles in ethics https://doi.org/10.1016/j.dim.2022.100006 https://doi.org/10.1016/j.ssci.2020.104667 https://doi.org/10.1016/j.ijhcs.2016.12.008 australasian journal of information systems khan & feroz 2024, vol 28, research article “use” as a conscious thought 40 and information technology, 8 (3.2%) in ergonomics and human factors each. none has appeared in the top is journals, which is consistent with the conclusion reached by (schuetz & venkatesh, 2020), 6 (2.4%) in bulletin of the atomic scientists, and 5 (2%) in cognitive systems research. 3. results table a and b shows the bursting topics included in the titles of the articles and the keywords supplied by authors with their weight, length, start year, and end year. in the case when there is no end date noted, the terms are considered to still be active. the burst words with the start year indicate that the word started appearing in that year and the end year indicates the year that the word last appeared. the burst words without an end year are for those intervals which extend to the most recent publications, suggesting terms that are in the middle of a large weight burst at present (kleinberg 2003). the length represents the period of the burst word measured in the number of years. currently, the most significant fading research trends represented in the field of self-governing things from a title perspective (table a) include neural (2001-2008), improve (2015-2011), control (2001-2007), determin (2010-2015), individu (2004-2009), and spatial (2008-2012). and the themes that are still active include embed (2016-present), ethic (2018-present), mental (2017-present), machin (2018-present), autom (2018-present), and accept (2018-present). these highlight potential areas of research that are very much the focus of contemporary. on the other hand, the author-supplied keyword analysis (table b) also shows which areas of research are not active as they once were; these include the likes of dynam (2002–2015), function (2002–2012), activ (2000–2010), and mobil (2000-2009). furthermore, the research topics that are still actively pursued are team (2017-present), analysi (2018-present), driverless (2018-present), car (2018-present) and driver (2019-present), so forth. this shows that most of the research on self-governing things seems to be focused on driverless cars and the issue associated with them. the appearance and disappearance of words also indicate that the research area and priorities are changing over the years. word weight length start end word weight length start end neural 1.45 8 2001 2008 network 1.62 3 2000 2002 improv 1.46 7 2005 2011 ai 0.87 2 2016 2017 situat 1.10 7 2004 2010 convers 0.78 2 2009 2010 control 1.87 7 2001 2007 embodi 1.19 2 2012 2013 determin 1.16 6 2010 2015 humanoid 1.12 2 2013 2014 individu 0.82 6 2004 2009 design 1.44 2 2019 spatial 0.95 5 2008 2012 user 1.24 2 2019 embed 1.11 5 2016 intellig 1.27 2 2017 2018 unman 0.90 5 2010 2014 trust 1.30 2 2019 play 0.90 5 2010 2014 drive 1.53 2 2019 mental 1.06 4 2017 human 4.00 2 2019 ethic 1.84 3 2018 context 0.93 2 2019 machin 2.47 3 2018 experi 0.81 2 2019 autom 2.28 3 2018 interact 1.08 2 2019 voic 0.82 3 2007 2009 interfac 1.82 2 2019 speech 1.53 3 2009 2011 adapt 1.13 2 2001 2002 develop 0.98 3 2012 2014 robot 0.97 2 2001 2002 context 0.80 3 2010 2012 dialogu 1.79 2 2010 2011 intellig 1.62 3 2001 2003 interfac 0.84 2 2010 2011 smart 1.60 3 2016 2018 cognit 1.14 1 2018 2018 accept 0.91 3 2018 decis 0.95 1 2012 2012 australasian journal of information systems khan & feroz 2024, vol 28, research article “use” as a conscious thought 41 global 0.91 3 2018 autonom 1.04 1 2003 2003 defin 0.91 3 2018 learn 1.23 1 2001 2001 table a. top bursting & disappearing stem words in titles of the articles word weight length start end word weight length start end dynam 2.57 13 2002 2014 agent 2.41 3 2011 2013 function 1.85 11 2002 2012 driverless 1.56 3 2018 activ 1.74 11 2000 2010 theori 1.78 3 2011 2013 mobil 2.03 10 2000 2009 netswork 1.55 3 2000 2002 eye 1.55 9 2005 2013 car 2.99 3 2018 visual 1.83 7 2010 2016 moral 1.69 2 2015 2016 evolutionari 1.74 7 2002 2008 driver 1.64 2 2019 emerg 1.96 7 2008 2014 respons 2.16 2 2014 2015 imit 1.7 5 2004 2008 attent 1.62 2 2009 2010 unman 1.56 5 2010 2014 engin 1.89 2 2017 2018 organ 1.8 5 2004 2008 vehicl 2.2 2 2019 afford 1.57 5 2007 2011 intellig 1.74 2 2000 2001 self 1.62 5 2004 2008 robot 1.94 2 2006 2007 behavior 1.83 4 2008 2011 drive 1.76 2 2019 learn 2.06 4 2005 2008 social 1.65 2 2003 2004 team 1.74 4 2017 agent 1.85 2 2000 2001 technolog 1.96 4 2014 2017 machin 2.43 1 2018 2018 process 1.95 4 2012 2015 emot 1.58 1 2016 2016 model 2.1 4 2008 2011 institut 1.81 1 2017 2017 spatial 1.95 4 2012 2015 fethic 1.56 1 2018 2018 cognit 1.71 4 2010 2013 autonom 2.18 1 2008 2008 languag 1.65 4 2002 2005 biolog 1.99 1 2008 2008 analysi 2.11 3 2018 human 1.52 1 2016 2016 table b. top busting & disappearing topics in the author-supplied keywords appendix b demographic category count % age group below 30 15 25 30-39 years 22 36 40-49 years 14 23 50 and over 9 15 work experience 1-5 years 11 18 5-10 years 15 25 10-15 years 9 15 15-20 years 11 18 20-25 years 5 8 more than 25 years 9 15 education bachelor’s degree 25 42 master’s degree 24 40 professional degree 4 7 some college credit 3 5 trade/technical/vocational 4 7 gender male 32 53 female 28 47 industry information technology 12 20 sales & marketing 9 15 education 7 12 tourism hospitality 6 10 australasian journal of information systems khan & feroz 2024, vol 28, research article “use” as a conscious thought 42 demographic category count % banking & finance 5 8 health 4 7 others 12 20 not working 5 8 table b1. pilot study sample characteristics (n = 60) constructs mean st-dv skewness autonomy consciousness aco1: i am mindful of __. 3.7 0.9 -1.3 aco2: i am aware of __ 3.9 0.9 -1.3 aco3: i know __ is present. 3.9 0.9 -1.3 aco4: i understand __ is around. 3.9 0.9 -1.4 sensibility consciousness sc1: __ senses hazards correctly. 4.2 1.0 -1.4 sc2: __ senses hazards in a timely fashion. 4.1 0.9 -1.0 sc3: __ reliably senses hazards. 4.2 0.9 -1.2 sc4: __ flexibly senses hazards. 3.8 1.0 -0.8 sc5: __ senses new conditions. 4.1 0.9 -1.4 responsiveness consciousness rc1: __ responds to hazards correctly. 3.7 1.0 -1.1 rc2: __ responds to hazards in a timely fashion. 3.7 0.9 -0.8 rc2: __ reliably responds to hazards. 3.9 0.8 -0.9 rc3: __ flexibly responds to hazards. 3.7 0.9 -0.7 rc4: __ responds to new conditions. 3.8 0.9 -1.0 adaptability consciousness ac1: __ adjusts to hazards correctly. 4.1 1.0 -1.2 ac2: __ adapts to hazards in a timely fashion. 4.1 0.9 -0.7 ac3: __ reliably adjusts to hazards. 4.0 1.0 -0.7 ac4: __ flexibly adapts to hazards. 4.0 0.9 -0.6 ac5: __ adapts to new conditions. 4.1 1.0 -0.8 satisfaction sat1: all things considered, i am very satisfied with __ 4.1 0.7 -0.4 sat2: overall, my experience with __is very satisfying. 4.1 0.9 -0.5 sat3: overall, i am satisfied with __ 4.1 0.8 -0.6 table b2. pilot study survey items ranking results australasian journal of information systems khan & feroz 2024, vol 28, research article “use” as a conscious thought 43 copyright: © 2024 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v28.4611 http://creativecommons.org/licenses/by-nc/3.0/au/ https://doi.org/10.3127/ajis.v28.4611 1 introduction 2 theoretical background 2.1 information systems and evolving technological dynamics 2.2 human-robot interaction (hri) and conscious use 3 conceptual boundaries 3.1 fully autonomous is artefact (faisa) 3.2 partially autonomous is artefact (paisa) 3.3 non-autonomous is artefact (naisa) naisa is any (physical or digital) is artefact that requires deliberate cognitive efforts, physical contact, and instructions to produce outcomes in all situations permanently. the instructions are provided in the form of physical actions performed by... 3.4 “use” in fully autonomous artefacts 4 methodology 4.1 instrument development 4.2 items generation and construct conceptualization 4.3 face and content validity 4.4 survey data collection 5 results 5.1 reliability and validity of formative constructs 5.2 reliability and validity of reflective constructs 5.3 test of structural model 6 discussion 6.1 nature of use 6.2 use in different contexts and levels 6.3 nature of use in partial autonomous things 6.4 contributions to theory and practice 6.5 limitations 7 conclusion 1. burst detection 2. data 3. results paper title (paper title style) australasian journal of information systems benbya, strich and trieu 2025, vol 29, provocation accounting for unintended consequences in is research 1 accepted 07 sep 2025, after 1 round of review with michael davern as accepting editor in chief. accounting for unintended consequences in is research: a call to action hind benbya*1 school of business western sydney university, australia email: h.benbya@westernsydney.edu.au franz strich department of information systems paderborn university, germany van-hau trieu deakin business school deakin university, australia abstract unintended consequences in information systems (is) research are typically examined only after they have surfaced in practice, often treated as isolated instances rather than being systematically identified and understood. this reactive stance constrains our ability to anticipate risks, recognize emerging opportunities, and build cumulative insights across studies. to assess the current state of is research on unintended consequences, we conducted a scoping review of literature in the association for information systems' senior scholars' basket of journals. our review highlights significant gaps in how unintended consequences are defined, theorized, and systematically compared. to address these gaps, we propose three foundational considerations: (1) conceptual clarity, to refine definitions and delineate boundaries; (2) relational configurations, to analyze the interplay between actors, technologies, and contexts; and (3) temporal configurations, to capture the evolving nature of consequences over time. building on these dimensions, we introduce a set of guiding questions designed to provide shared analytical vocabulary rather than a prescriptive framework. these questions enable researchers to more systematically identify, categorize, and compare unintended consequences across contexts, thereby fostering theoretical precision, facilitating cross-contextual learning, and supporting a more anticipatory and comprehensive understanding of the phenomenon. keywords: unintended consequences, unexpected outcomes, unanticipated consequences, digital transformation, it governance. 1 introduction this paper is a call to action for researchers in the information systems (is) discipline to systematically account for unintended consequences in is research. it urges scholars to revisit the origins of this phenomenon—now an emerging and increasingly vital research stream— and to critically examine how it has been enacted in contemporary is studies. by doing so, we aim to offer new directions for future research that move beyond a reactive conceptualization. 1 all authors contributed equally to this manuscript mailto:h.benbya@westernsydney.edu.au australasian journal of information systems benbya, strich and trieu 2025, vol 29, provocation accounting for unintended consequences in is research 2 historically, is research has focused on the intended uses and benefits of technological innovations, often treating unintended consequences as anomalies addressed only after they surface in practice. this reactive stance reflects the broader inability of stakeholders to anticipate negative outcomes and harness unforeseen opportunities. this approach has contributed to a limited understanding of how and why technologies fail, are abandoned, or provoke workaround behaviors. for instance, it project failures frequently stem from misaligned stakeholder expectations or overlooked organizational dynamics. discontinuation of use may reflect a mismatch between system design and evolving user needs, while workarounds often arise when formal systems fail to accommodate real-world practices. these outcomes are not merely unfortunate side effects—they are manifestations of deeper conceptual and relational gaps in is design and implementation. we argue that unintended consequences are not inherently unforeseeable. rather, they often stem from insufficient early-stage reasoning and a lack of comprehensive conceptualization. to advance the discourse, we assess the current state of research in is on unintended consequences and identify key limitations in how the phenomenon is currently studied. thereby, we propose three foundational considerations to deepen our understanding and to offer a framework for theorizing the dynamic and multifaceted nature of unintended consequences. 1. conceptual clarity – refining definitions and boundaries of unintended consequences. 2. relational configurations – examining the interplay between actors, technologies, and contexts. 3. temporal configurations – accounting for the evolving nature of consequences over time. for practitioners, this paper highlights the importance of integrating anticipatory thinking into the design and deployment of is solutions. by embedding foresight practices and scenario planning into early development stages, organizations can better mitigate risks and capitalize on emergent opportunities. furthermore, the framework encourages is professionals to adopt a more holistic view of system impacts—one that includes ethical, social, and long-term dimensions—thus fostering more resilient and responsible innovation. 2 unintended consequences: origins and use in is the concept of unintended consequences, although rooted in the 18th century, gained prominence in the 20th century through sociologists who viewed them as a common occurrence rather than an anomaly. long before sociologists formalized the idea, thinkers such as adam smith, david hume, and montesquieu had already explored the unintended effects of policies, laws, and societal structures. smith’s “invisible hand” metaphor, for instance, illustrates how individual self-interest can unintentionally promote collective welfare—an early recognition of emergent outcomes beyond deliberate design. unintended consequences refer to outcomes or effects that arise from purposive social action but were not anticipated or intended by the actors involved (merton, 1936). these consequences, which can be positive, negative or neutral, were neither intended nor desired when the decision or action was taken. however, such consequences arise from the “overflow” of action, namely, an action always does more than what was initially intended (callon, 1998). merton (1936) was among the first to delineate various causes of unintended consequences in the social sciences, such as errors, conflicting values, short-term versus long-term interests, and ignorance. merton (1936) also highlighted the complexity of causality in social actions, australasian journal of information systems benbya, strich and trieu 2025, vol 29, provocation accounting for unintended consequences in is research 3 arguing that consequences are often shaped by the interplay between individual intentions and the broader social environment. building on merton's foundational work, baert (1991) extended the study of unintended consequences by introducing a structured framework that categorizes these outcomes along five key dimensions. the first dimension separates unintended consequences into two categories: those affecting individuals and those impacting society. the second dimension classifies these consequences based on their value, distinguishing between desirable, undesirable, and neutral effects and thereby challenging the tendency to equate “unintended” with “negative.” the third dimension focuses on the relationship between the consequences and the initial intentions behind them. the fourth dimension reveals the varied levels of predictability, spanning from unforeseen impacts to anticipated yet surprising results, and entirely unforeseen outcomes. the last dimension encompasses the temporal emergence of unintended consequences, occurring either during or after the action. building on these early developments, the is discipline further appropriated ideas from the diffusion of innovations theory (rogers, 1998). this theory suggests that when innovations are adopted, the outcomes can be intended or unintended, as well as desirable or undesirable. empirical studies in is initially depicted unintended consequences as anomalies or contradictions (robey and boudreau, 1999). this view has been challenged by the duality of technology concept which indicates that different effects can emerge concurrently in various settings due to technology use (orlikowski 1992). over the past decades, different is perspectives have been used to explain how unintended consequences unfold leading to a range of performance and economic consequences or social and ethical concerns (markus and robey, 1988; boudreau and robey, 2005; walsham, 2012). table 1 summarizes some of these perspectives outlining their rationale and offering examples of studies used to investigate it unintended consequences (table 1). perspective rationale example of studies adaptive structuration (ast) examines the change process from two vantage points: (i) the types of structures that are provided by advanced technologies, and (ii) the structures that actually emerge in human action as people interact with these technologies. orlikowski (1992) markus and robey (1988) watson et al. (1988) socio-technical considers both the technical artifacts and the social (e.g., psychological, cultural, and economic) contexts and sees outcomes as emerging from the interaction between the two. sarker et al. (2019) complex systems theories advances that interactions among a diverse set of connected, mutually dependent, and adaptive agents in a sociotechnical system lead to the emergence of unexpected outcomes. benbya et al. (2006, 2020) marjanovic and cecez kecmanovik (2017) affordance actualization investigates how the (non-) actualization of affordances produces unintended consequences. trocin et al. (2024) table 1. perspectives and rationales while the existing literature on unintended consequences of it spans multiple domains and technologies, it remains fragmented across research streams. to address this dispersion, we present a scoping review that synthesizes exemplar streams and offers a critical reflection on the state of knowledge. australasian journal of information systems benbya, strich and trieu 2025, vol 29, provocation accounting for unintended consequences in is research 4 3 exemplar streams of is research on unintended consequences we searched for studies on unintended consequences of it in the major is journals listed in the association for information systems' senior scholars' basket of journals, covering publications up to november 2024. using targeted keywords such— “unintended consequences,” “unintended outcomes,” “unintended results,” “unexpected consequences,” “unexpected outcomes,” “unexpected results,” “unanticipated consequences,” “unanticipated outcomes,” “unanticipated results,” — we searched article titles and abstracts to ensure a focused and relevant analysis. this search initially identified 72 papers. upon reviewing the abstracts, we categorized them as follows: 31 did not study unintended consequences of it despite the presence of relevant keywords; 28 reported unintended consequences as supplementary findings; and 15 focused explicitly on unintended consequences as their primary research topic. among the latter 15 manuscripts, unintended consequences were most frequently examined in algorithmic systems (4 articles), and digital health systems (4 articles), followed by cybersecurity systems (3 articles) and other systems (4 articles). although this represents a relatively small sub-sample in the is domain, analyzing and synthesizing the 15 articles yields rich insights into the current understanding of unintended consequences from an is perspective and highlights potential directions for future research. the extant literature on unintended consequences of it can be grouped into three exemplar streams of research. in the following sections, we synthesize the findings of these articles in detail for each research stream. the first research stream focuses on unintended consequences of algorithmic systems. we observe that research under this stream has mainly focused on the undesirable value dimension of unintended consequences. for example, ma et al (2024) reveal that labelling social media news stories recommended by ai was not as beneficial as expected. in fact, such ai recommendation reduce belief in true news but have no effect on fake news. similarly, rinta-kahila et al. (2021) offer a critical analysis of the australian government’s “robodebt” program, which, despite its intent to streamline welfare debt collection, caused severe psychological distress and operational inefficiencies, exposing a lack of ethical foresight in its design. additionally, rana et al. (2022) point to operational inefficiency, employee dissatisfaction, and competitive disadvantage as critical flaws in ai-integrated business analytics, raising questions about the assumed benefits of such technologies. monod et al. (2024) further challenge the prevailing narrative that ai is inherently beneficial, demonstrating how even well-intentioned designs can result in harm if workers are excluded from the design and implementation process. collectively, these studies reveal a troubling pattern of unintended consequences stemming from a failure to account for human and organizational complexities in algorithmic systems. they highlight a strong emphasis on the negative value dimension, such as harm to users, ethical oversights, and operational inefficiencies, while giving less attention to the ways unintended consequences may arise in more complex, ambiguous, or even beneficial forms in algorithmic system use. the second research stream examines the unintended negative consequences of digital health systems from both unintended desirable and undesirable outcomes. for example, trocin et al. (2024) identify four user actions—flexing, bypassing, avoiding, and reorganizing—in electronic health record (ehr) systems that lead to various unintended consequences. flexing, bypassing, and avoiding result in undesirable outcomes, while reorganizing leads to desirable ones. these actions arise from the interaction between technological features and psychosocial or organizational factors, shaping users' perceptions and influencing unintended australasian journal of information systems benbya, strich and trieu 2025, vol 29, provocation accounting for unintended consequences in is research 5 results. the studies by he et al. (2023), zhou et al (2023) and yeongin et al. (2022) primarily focus on undesirable consequences. specifically, he et al. (2023) investigate online aesthetic medicine platforms, revealing that monetary reward incentives intended to boost usergenerated content often result in unchanged or reduced contributions, as users typically fulfill only minimal requirements. similarly, zhou et al. (2023) explore the emotional support provided by online health community platforms, finding that while intended recipients benefit, other users may suffer harm through social comparison. their findings underscore the necessity of considering broader systemic impacts in platform design. finally, yeongin et al. (2022) examine the role of it in healthcare decision-making. they find that while it can enhance decision-making by facilitating risk information sharing with physicians, it also introduces legal vulnerabilities that encourage defensive medical practices. these unintended consequences can result in suboptimal healthcare delivery. therefore, it is crucial for policymakers to carefully consider respective factors to maximize the benefits of it while mitigating associated risks. the third stream examines the unintended consequences of cybersecurity systems, focusing primarily on undesirable outcomes, that significantly affect users' compliance and trust. wolff (2016) examined the complex dynamics of human-machine moderation, identifying unintended consequences, such as emboldening, disengaging, and spillover effects. the study demonstrates that while human-machine moderation effectively reduces harassment within the focal community by shielding users from harmful content, it inadvertently increases harassment toward perceived outgroup members and neighboring communities while also diminishing the number of unique contributors within the focal community. similarly, parks et al. (2017) introduced the concept of the “imbalance challenge,” wherein the negative outcomes of privacy measures outweigh the positive benefits. this imbalance often leads individuals to disregard or misinterpret privacy guidelines, resulting in workaround behaviors that undermine overall compliance. extending the discussion of unintended consequences in cybersecurity, pienta et al. (2024) investigated how vulnerability assessments can unintentionally provoke feelings of betrayal among employees. techniques such as simulated phishing attacks and email scanning—though designed to enhance organizational security—are frequently perceived as intrusive and punitive. these perceptions can foster a sense of “cybersecurity betrayal,” prompting active resistance to cybersecurity policies and technologies. these studies reveal that unintended consequences in cybersecurity systems often arise from misalignments between technical interventions and users’ expectations or values. this underscores the urgent need for more human-centered and anticipatory approaches in cybersecurity design and governance. collectively, the findings highlight the importance of understanding the socio-technical dynamics behind unintended outcomes of designing systems and policies that foster trust and promote compliance. other studies primarily examine unintended negative consequences in the contexts of online community platforms, human resources information systems (hris), and mobile devices. zhao et al. (2016) investigate how extrinsic rewards—such as virtual points and reciprocity— affect knowledge sharing in online communities. the study finds that while these rewards can boost participation, they can also create unintended consequences including undermining intrinsic motivations like enjoyment in helping others and knowledge self-efficacy. nguyen et al. (2024) assess the impact of human-machine collaboration in moderating online communities. although initially effective at curbing harassment, widespread implementation of such systems can lead to unexpected negative outcomes. in the hris domain, maier et al. australasian journal of information systems benbya, strich and trieu 2025, vol 29, provocation accounting for unintended consequences in is research 6 (2013) examine the unintended consequences of implementing an e-recruiting system at a global automotive supplier, focusing on its effects on hr employees' job satisfaction and turnover intentions. their findings underscore the importance of employee perceptions and highlight the risk of decreased satisfaction and higher attrition if these systems are poorly received. magni et al. (2023) examine the consequences of excessive mobile devices use for work during nonwork hours. using resource drain theory, they reveal how family-work conflict and competitive climates can harm both productivity and well-being. the study calls for organizational policies that balance productivity with support for employees’ mental health and personal boundaries. moreover, several studies reported unintended consequences as supplementary findings, with results primarily centered on user resistance (kumar et al., 2022), low self-efficacy (kang et al. 2021), privacy concerns (fox & connolly, 2018), diminished job satisfaction (califf et al., 2020), increased workload (bernardi & exworthy, 2020), fragmented information sharing, strained relationships between healthcare actors and patients, and deteriorating work conditions (kummer et al., 2017). notably, most studies framed these unintended consequences as undesirable effects, often carrying a negative connotation. however, information systems can also yield positive unintended outcomes—what merton (1936) termed “unexpected benefits.”. for instance, petrakaki et al. (2012) found that the digital transmission of prescriptions empowered pharmacists to assume greater responsibilities, thereby enhancing their professional influence and expanding their scope of practice. based on the profound impacts of unintended consequences and their divergent interpretations in is research, developing a deeper understanding of how these consequences unfold can significantly advance both is theory and practice. our review reveals a lack of consistency in how unintended consequences are conceptualized across studies. many studies fail to provide a clear or explicit definition of unintended consequences, rendering the term ambiguous and its implications underexplored. moreover, most of the literature frames unintended consequences simply as outcomes that were not intended, often neglecting the anticipatory dimension. these consequences are typically treated as generic and isolated events, relying on descriptive narratives of technology use without clearly articulating their focal point. critical questions—such as “unintended by whom,” “for whom,” “in relation to what,” and whether these effects are temporal or evolve over time—frequently remain unaddressed. among the studies reviewed, only trocin et al. (2024) offer a robust and explicit conceptualization. they delineate both the direct and indirect effects of unintended and unanticipated consequences, emphasizing their multifaceted nature and the socio-technical interactions that shape them. this conceptual gap underscores the need for future research to establish a more standardized framework for identifying, categorizing, and analyzing unintended consequences. such a framework would enrich both theoretical and practical understanding of is phenomena. accordingly, we advocate for a more sophisticated and systematic approach to studying unintended consequences and their ripple effects within and across individuals and organizations. 4 redirecting future research on unintended consequences drawing on our synthesis of prior research, we propose three key directions for future research: (1) conceptual clarity, (2) relational configurations, and (3) temporal configurations. in response to recent calls for more robust theorizing on the unintended consequences of itl australasian journal of information systems benbya, strich and trieu 2025, vol 29, provocation accounting for unintended consequences in is research 7 use (markus and robey, 2004; trocin et al., 2024), each of our proposed directions offers valuable insights into how this topic can be advanced from a theoretical and methodological perspective. 4.1 towards conceptual clarity of unintended consequences a review of current literature within the is domain reveals persistent conceptual imprecision and terminological conflation in discussions of unintended consequences. terms such as unintended consequences (e.g., baert, 1991), unanticipated consequences (e.g., merton, 1936; de zwart, 2015), unintended insights (e.g., jung et al., 2022), perverse effects (e.g., hirschman, 1991), latent functions (e.g., merton, 1968) or even dark side (e.g., giermindl et al., 2022) are frequently used interchangeably, despite their conceptual distinctions. unintended consequences encompass all outcomes that occur regardless of the actor’s intentions, while unanticipated consequences refer specifically to outcomes that were not foreseen (merton, 1936). this distinction matters: conflating these terms obscures cases where actors may anticipate but not intend certain outcomes—such as knowingly accepting trade-offs to achieve broader strategic goals (de zwart, 2015). recognizing these nuances is essential for developing a more precise and analytically useful vocabulary in is research. moreover, the use of these terms often carries a predominantly negative connotation, which limits the broader potential to study unintended consequences in is research. the prevailing assumption that unintended consequences are inherently negative is overly simplistic and fails to reflect their multifaceted nature (trocin et al., 2024). unintended consequences can be both positive, negative or neutral, with their perception often varying between stakeholder groups (merton, 1936). for instance, the digital transmission of prescriptions may empower pharmacists by expanding their roles and increasing their professional influence (trocin et al., 2024). at the same time, this implementation can inadvertently heighten stress and workload for employees who struggle to adapt to new workflows or face pressure to meet elevated productivity demands—an unintended but adverse outcome. similarly, markus and robey (2004) describe how automated scheduling systems in customer service departments enable organizations to optimize workforce allocation and reduce downtime. yet, employees frequently report frustration and diminished morale due to rigid schedules, reduced autonomy, and a heightened sense of surveillance. finally, building on the previous points, it is important to distinguish the concept of unintended consequences from related concepts such as it project failure (cecez-kecmanivic et al., 2014), discontinuation of use (soliman and rinta-kahila, 2020), and workarounds (alter, 2014; trieu et al., 2022). while these concepts may appear similar, they differ in focus and analytical lens. • it project failure typically involves comparing actual outcomes against predefined goals. in contrast, the study of unintended consequences centers on outcomes which were not intended. • discontinuation of use examines behavioral responses to certain effects, whereas research on unintended consequences investigate the effects themselves—whether social, technical, or organizational—that may trigger discontinuation if they are unintended. • workarounds, by definition, are deliberate deviations from prescribed processes. they are enacted in response to constraints or inefficiencies and may arise as a reaction to unintended consequences, but they remain intentional. australasian journal of information systems benbya, strich and trieu 2025, vol 29, provocation accounting for unintended consequences in is research 8 by clarifying these distinctions, we can better understand the unique analytical space that the concept of unintended consequences occupies within the broader discourse on it systems and organizational behavior. to fully grasp the nature of unintended consequences, it is essential to move beyond reactive responses and adopt a systematic, anticipatory approach. this entails mapping a broad spectrum of related concepts and outcomes—including positive consequences—while also recognizing the risks they may entail (wolff et al., 2024). for example, technologies designed to enhance productivity may initially deliver on their promise but later trigger revenge effects, where the very efficiency they create introduces counterproductive complexities (parks et al., 2017). even positive unintended consequences are not guaranteed to remain beneficial over time; they can be appropriated or adapted in ways that introduce new challenges. by addressing these gaps, future research can more effectively identify and interpret diverse consequences—positive, negative, and dual—that emerge across varying contexts and among different stakeholders. 4.2 towards relational configurations of unintended consequences the current literature tends to treat unintended consequences as isolated occurrences, often relying on descriptive narratives of events related to technology use. while identifying unintended consequences has been a central focus, uncovering their origins is key to generating actionable insights and developing strategies to mitigate negative effects or leverage positive potential (trocin et al., 2024). the emerging body of work on relational configurations (e.g., bailey et al., 2022; bradbury & lichtenstein, 2000) offers a promising lens through which to examine the interdependencies across different stages for unintended consequences. a deeper exploration of this phenomenon can be guided by three critical questions: unintended by whom, in relation to what, and unintended for whom. these questions provide a structured framework for analyzing how unintended consequences emerge, the dynamics that shape them, and the stakeholders they affect. the first question, unintended by whom, probes which actor or group did not intend—or perhaps anticipate— the observed consequence. this question underscores the subjectivity in defining what is unintended, as various stakeholders in a system often have differing expectations, intentions, and priorities (trocin et al., 2024). markus and robey (2004) note that in the context of it use, unintended consequences frequently emerge from misaligned intentions across designers, decision-makers, and users. examining unintended consequences from multiple perspectives reveals that what is unintended and/or unanticipated by one group may be entirely expected—or even intended—by another. unintended in relation to what, shifts focus to the specific action, intervention, or process that precipitated the consequence. addressing this dimension requires tracing the causal chain that links a purposive act to its outcomes. merton (1936) highlights how purposive actions are embedded within complex systems, where outcomes often arise from the interplay between the original intent and external conditions. for example, parks et al. (2017) illustrate how privacy safeguards introduced in healthcare organizations—intended to enhance compliance and confidentiality—inadvertently disrupted workflows due to poor system design. in this case, examining unintended in relation to what reveals how technological decisions, when misaligned with operational context, can yield consequences that diverge from original goals. finally, unintended for whom, examines the stakeholders affected by the consequence and their varied perceptions of its impact. unintended consequences are rarely uniform in their effects; australasian journal of information systems benbya, strich and trieu 2025, vol 29, provocation accounting for unintended consequences in is research 9 thus, future research must identify which groups benefit and which are disadvantaged (trocin et al., 2024). markus and robey (2004) note that it interventions often produce dual outcomes, conferring advantages on some stakeholders while burdening others. for example, the deployment of telework technologies, may yield unintended advantages such as reduced overhead costs, yet simultaneously introduce unintended challenges for employees, such as isolation and diminished team cohesion. understanding who experiences the unintended consequences and how these perceptions vary across stakeholders is critical for developing interventions that balance competing interests and mitigate harm. 4.3 towards temporal configurations of unintended consequences in line with the second contribution, our analysis underscores the importance of adopting a multilayered and temporally sensitive approach to understanding unintended consequences (fig. 1). such an approach must account for four interrelated dimensions: stakeholders, the operational context of the technology, the outcomes of its use, and the implications of those outcomes, including their temporal dependencies. a comprehensive approach requires careful consideration of the diverse stakeholders involved, as their intentions, actions, and perceptions both shape and are shaped by the consequences that emerge (trocin et al., 2024). these stakeholders include, but are not limited to, users, such as employees who directly interact with the technology; programmers, whose design choices influence its affordances and constraints; organizations, which define strategic goals and oversee implementation; and broader societal groups, which may experience indirect effects of technology deployment. each stakeholder group has unique priorities, expectations, and degrees of influence, and their interactions can generate complex, often unanticipated dynamics. moreover, any meaningful analysis must account for the operational context of the technology itself—including its features, functionalities, and embedded assumptions (parks et al., 2017). the design and affordances of technology play a critical role in shaping how it is used and appropriated by stakeholders. for example, markus and robey (2004) describe how features such as default privacy settings in scheduling systems can influence user behavior, either enable collaboration or inadvertently create barriers. this understanding becomes even more crucial with advanced technologies such as artificial intelligence (ai), generative ai (genai), and machine learning (ml), where automation, autonomy, and opaque “black-box” processes introduce new layers of complexity (benbya et al., 2021, 2024; benbya and melville, 2025). these systems often produce outputs or decisions that deviate from original intentions due to their adaptive nature and reliance on vast, sometimes biased, data sets. notably, such technologies frequently operate within interconnected environments, generating feedback loops that amplify unintended effects—ranging from stakeholder alienation to increased regulatory scrutiny. additionally, it’s essential to examine the outcomes of technology use to understand what actually happens when stakeholders interact with a given system. while direct outcomes— such as efficiency gains, usability challenges, or shifts in work practices—often serve as the most visible indicators of unintended consequences, they represent only part of a more complex picture. a comprehensive understanding requires exploring the dynamic interplay between stakeholders, the technology itself, and the broader context in which it is embedded (trocin et al., 2024). these outcomes may include not only operational changes but also deeper social and cultural shifts. importantly, the interplay between a technology’s design and its contextual deployment can produce effects that diverge significantly from its original intentions underscoring the need for a nuanced and context-sensitive analysis. australasian journal of information systems benbya, strich and trieu 2025, vol 29, provocation accounting for unintended consequences in is research 10 finally, it is crucial to assess the broader implications of technology to understand how consequences extend beyond their immediate impact. these implications may include delayed financial effects for organizations often arising from unintended consequences such as increased workload, shifts in job satisfaction, or broader societal effects such as increased digital surveillance. this aspect highlights the importance of considering equity and fairness when evaluating the broader effects of unintended consequences. the subjectivity of intentions, the complexity of actions, and the diversity of affected stakeholders underscore the dynamic nature of technology. unintended consequences are rarely static; they evolve as initial effects trigger feedback loops and adaptive responses. adopting a temporal lens is essential for capturing how these unintended consequences emerge, evolve, and interact across varying contexts. by accounting for these evolving dynamics, researchers can develop more effective strategies to anticipate, mitigate, and leverage unintended outcomes—ultimately promoting a more nuanced and sustainable approach to technology implementation and its wider societal implications. figure 1. temporal configuration of unintended consequences to advance the study of unintended consequences, we identified three key areas of concern in the previous section: conceptual clarity, relational configurations, and temporal dynamics. these dimensions reveal critical blind spots in much of the existing literature, which often treats unintended consequences as isolated incidents—described in vague or ambiguous terms, and analyzed without attention to their evolution over time. each area calls for deeper analytical engagement and reflexivity in how unintended consequences are identified, examined, and understood within sociotechnical systems. to guide future researchers in investigating the nature and impact of unintended consequences, we provide a structured set of guiding questions that are not bound to any specific empirical context or theoretical tradition (table 2). rather than offering a prescriptive framework, these questions serve as a toolkit to help navigate the conceptual ambiguity that often surrounds unintended consequences. first, our questions facilitate cross-contextual comparisons by establishing a shared analytical vocabulary, allowing insights from one study to inform another, even across diverse organizational or technological settings. second, our australasian journal of information systems benbya, strich and trieu 2025, vol 29, provocation accounting for unintended consequences in is research 11 domain q # guiding question description conceptual clarity 1 how is the term 'unintended' defined in the context under study? clarifies the meaning of the term and distinguishes it from related concepts such as unanticipated, accidental, or emergent effects. 2 is the observed outcome intended or unintended? differentiates between outcomes that were intended and unintended. 3 is the observed outcome anticipated or unanticipated? differentiates between outcomes that were foreseen and unforeseen. 4 how can the consequence be described in terms of unexpected drawback, unexpected benefit, perverse result? challenges the presumption of harm and promotes exploration of diverse stakeholder perceptions. 5 is there conceptual overlap between the consequence and the response to it, and how can these be separated/integrated analytically? highlights the need to distinguish between the unintended effect itself, behaviors it triggers and potential revenge effects. 6 what assumptions are embedded in the conceptual framing, and how do they shape what is recognized as unintended? encourages reflection on the theoretical lens used and its influence on identifying and categorizing outcomes. relational configurations 7 who defines what is considered unintended? relates to the subjectivity of intention and highlights that unintendedness is not an objective attribute but shaped by actors’ perspectives and interpretive frames. 8 which unintended consequences are visible from which viewpoint? engages with the positionality of observation, recognizing that consequences may appear differently depending on where and how one looks. 9 who defines the intentions that underpin the original purposive action? targets the foundational logic or rationale behind the original act, encouraging reflection on implicit beliefs that may shape both intentions and consequences. 10 whose intentions are (not) being realized? connects to actor heterogeneity, acknowledging that different actors (designers, users, managers) may have divergent or conflicting intentions in a shared system. 11 how do different artifacts (actors, technologies, processes, systems) co-impact unintended consequences? relates to the relational configuration, moving beyond linear causality and towards understanding how networks of elements collectively produce outcomes. 12 what power dynamics shape whose unintended consequences are prioritized or addressed? links to institutional and political dimensions, highlighting how responses to unintended consequences are influenced by structures of authority and influence. 13 how do attempts to mitigate unintended consequences themselves generate further effects? reflects on recursive dynamics, encouraging attention to second order and cascading consequences. australasian journal of information systems benbya, strich and trieu 2025, vol 29, provocation accounting for unintended consequences in is research 12 temporal configurations 15 how do initial outcomes trigger secondary or tertiary effects? focuses on cascading effects and feedback loops, where early consequences create new conditions that give rise to further unintended developments. 16 which unintended consequences are immediate, and which only become visible after extended use? differentiates between visible short-term disruptions and latent consequences that emerge gradually or in response to cumulative use. 17 what mechanisms sustain or amplify unintended consequences over time? relates to feedback loops and path dependence, asking how certain outcomes persist or escalate rather than being corrected or absorbed. 18 which temporal dependencies are being considered when examining unintended consequences? clarifies whether the analysis captures pre-use expectations, immediate outcomes, delayed effects, or long-term implications, ensuring the full arc of the consequence is mapped. 19 how do interactions between stakeholders evolve over time in shaping or responding to unintended consequences? highlights the dynamic and iterative nature of stakeholder relationships and how these interactions influence the development of consequences. 20 what role does the technology’s operational context play in shaping consequences over time? clarifies whether temporal development of unintended consequences relates to technological changes or changing organizational / social systems that potentially mediate / influence the emergence and persistence of the consequences. 22 how are unintended consequences temporally distributed across stakeholder groups? examines whether certain actors are affected early while others encounter effects later, revealing asymmetries in the timeline of impact. table 2. guiding questions for exploring unintended consequences. australasian journal of information systems benbya, strich and trieu 2025, vol 29, provocation accounting for unintended consequences in is research 13 questions support greater theoretical precision by encouraging authors to clarify core concepts, distinguish between related phenomena, and attend to the framing effects of their own analytical choices. finally, our questions enable researchers to shift from post hoc descriptions of unexpected outcomes toward a more anticipatory and explanatory orientation. however, this set of questions should be viewed as a starting point rather than an exhaustive list. emerging technologies (e.g., ai) and increasingly complex, non-standard social systems— such as platform-mediated work—may call for additional questions to be explored. 5 conclusion this provocation calls on is researchers to critically revisit how we investigate unintended consequences. to support this effort, we present 22 guiding questions designed to help researchers and practitioners explore, anticipate, mitigate, and leverage unintended outcomes. these questions are grounded in a structured framework that offers a clear conceptual foundation for understanding unintended consequences, organized around three dimensions—unintended by whom, in relation to what, and unintended for whom—alongside their temporal dependencies. this framework enables a systematic examination of how unintended consequences emerge, the dynamics that shape them, and the stakeholders they affect. in doing so, this provocation contributes to a more nuanced and sustainable understanding of technology implementation and its broader implications. from a practical standpoint, our work underscores the significant—and often overlooked risks—of unintended consequences in organizational decision-making. once these risks are fully acknowledged, organizations can implement strategies to mitigate them across key phases, such as requirements gathering, agile development, and post-implementation monitoring. for instance, engaging a broad range of stakeholders—including those indirectly affected—can help surface potential issues early in the design process. organizations can leverage tools such as scenario planning, impact mapping, and ethical assessments to identify possible downstream effects. this awareness can be integrated into the organization’s learning culture through regular reflection, impact discussions, and maintenance of a dedicated ethics or risk backlog to monitor emerging concerns. over time, building a repository of past challenges and fostering a culture of continuous learning and ethical reflection can support the development of more responsible and resilient systems. references alter, s. (2014). theory of workarounds. communications of the association for information systems, 34(55), 1041–1066. doi.org/10.17705/1cais.03455 baert, p. (1991). unintended consequences: a typology and examples. international sociology, 6(2), 201–210. doi.org/10.1177/026858091006002006 bailey, d. e., faraj, s., hinds, p. j., leonardi, p. m., & von krogh, g. (2022). we are all theorists of technology now: a relational perspective on emerging technology and organizing. organization science, 33(1), 1–18. doi.org/10.1287/orsc.2021.1562 benbya, h., & mckelvey, b. (2006). using coevolutionary and complexity theories to improve is alignment: a multi–level approach. journal of information technology, 21(4), 284–298. doi.org/10.1057/palgrave.jit.2000080 australasian journal of information systems benbya, strich and trieu 2025, vol 29, provocation accounting for unintended consequences in is research 14 benbya, h., nan, n., tanriverdi, h., & yoo, y. (2020). complexity and information systems research in the emerging digital world. mis quarterly, 44(1), 1–17. doi.org/10.25300/misq/2020/13304 benbya, h., pachidi, s., & jarvenpaa, s. (2021). artificial intelligence in organizations: implications for information systems research. journal of the association for information systems, 22(2). doi.org/10.17705/1jais.00662 benbya, h., & melville, n. (2025). fueling the potential of artificial intelligence for societal impact. mis quarterly executive, 24(2). doi.org/10.17705/2msqe.2025.24.2.2 benbya, h., strich, f., & tamm, t. (2024). navigating generative artificial intelligence promises and perils for knowledge and creative work. journal of the association for information systems, 25(1), 23–36. doi.org/10.17705/1jais.00861 bernardi, r., & exworthy, m. (2020). clinical managers' identity at the crossroad of multiple institutional logics in it innovation: the case study of a health care organization in england. information systems journal, 30(3), 566–595. doi.org/10.1111/isj.12267 boudreau, m.-c., & robey, d. (2005). enacting integrated information technology: a human agency perspective. organization science, 16(1), 3–18. doi.org/10.1287/orsc.1040.0103 bradbury, h., & lichtenstein, b. m. b. (2000). relationality in organizational research: exploring the space between. organization science, 11(5), 551–564. doi.org/10.1287/orsc.11.5.551.15203 califf, c. b., sarker, s., & sarker, s. (2020). the bright and dark sides of technostress: a mixedmethods study involving healthcare it. mis quarterly, 44(2), 809–856. doi.org/10.25300/misq/2020/14818 callon, m. (1998). an essay on framing and overflowing: economic externalities revisited by sociology. the sociological review, 46(1_suppl), 244–269. doi.org/10.1111/j.1467954x.1998.tb03477.x cecez-kecmanovic, d., kautz, k., & abrahall, r. (2014). reframing success and failure of information systems: a performative perspective. mis quarterly, 38(2), 561–588. doi.org/10.25300/misq/2014/38.2.11 de zwart, f. (2015). unintended but not unanticipated consequences. theory and society, 44(3), 283–297. doi.org/10.1007/s11186-015-9247-6 fox, g., & connolly, r. (2018). mobile health technology adoption across generations: narrowing the digital divide. information systems journal, 28(6), 995–1019. doi.org/10.1111/isj.12179 giermindl, l. m., strich, f., christ, o., leicht-deobald, u., & redzepi, a. (2022). the dark sides of people analytics: reviewing the perils for organisations and employees. european journal of information systems, 31(3), 410–435. doi.org/10.1080/0960085x.2021.1927213 he, l., luo, j., tang, y., wu, z., & zhang, h. (2023). motivating user-generated content: the unintended consequences of incentive thresholds. mis quarterly, 47(3), 1015–1044. doi.org/10.25300/misq/2022/17369 hirschman, a. o. (1991). the rhetoric of reaction: perversity, futility, jeopardy. harvard university press. australasian journal of information systems benbya, strich and trieu 2025, vol 29, provocation accounting for unintended consequences in is research 15 jung, a.-k., clausen, s., franzke, a. s., & marx, j. (2022). ‘cambridge moralica’ – towards an ethical framework for social media analytics. australasian journal of information systems, 26, article 0. doi.org/10.3127/ajis.v26i0.3225 kang, n., ding, d., van riemsdijk, m. b., morina, n., neerincx, m. a., & brinkman, w.-p. (2021). self-identification with a virtual experience and its moderating effect on self-efficacy and presence. international journal of human–computer interaction, 37(2), 181–196. doi.org/10.1080/10447318.2020.1812909 kumar, m., singh, j. b., chandwani, r., & gupta, a. (2022). locating resistance to healthcare information technology: a bourdieusian analysis of doctors' symbolic capital conservation. information systems journal, 32(2), 377–413. doi.org/10.1111/isj.12357 kummer, t.-f., recker, j., & bick, m. (2017). technology-induced anxiety: manifestations, cultural influences, and its effect on the adoption of sensor-based technology in german and australian hospitals. information & management, 54(1), 73–89. doi.org/10.1016/j.im.2016.04.002 ma, l., zhang, x., & yu, p. (2023). enterprise social media usage and social cyberloafing: an empirical investigation using the jd-r model. internet research, 34(3), 939–959. doi.org/10.1108/intr-04-2021-0239 magni, m., ahuja, m. k., & trombini, c. (2023). excessive mobile use and family-work conflict: a resource drain theory approach to examine their effects on productivity and well-being. information systems research, 34(1), 253–274. doi.org/10.1287/isre.2022.1121 maier, c., laumer, s., eckhardt, a., & weitzel, t. (2013). analyzing the impact of hris implementations on hr personnel’s job satisfaction and turnover intention. the journal of strategic information systems, 22(3), 193–207. doi.org/10.1016/j.jsis.2012.09.001 markus, m. l., & robey, d. (1988). information technology and organizational change: causal structure in theory and research. management science, 34(5), 583–598. doi.org/10.1287/mnsc.34.5.583 marjanovic, o., & cecez-kecmanovic, d. (2017). exploring the tension between transparency and datification effects of open government is through the lens of complex adaptive systems. journal of strategic information systems, 26(3), 210–232. doi.org/10.1016/j.jsis.2017.07.001 markus, m. l., & robey, d. (2004). why stuff happens: explaining the unintended consequences of using it. in k. v. andersen & m. t. vendelø (eds.), the past and future of information systems (pp. 61–93). elsevier butterworth-heinemann. merton, r. k. (1936). the unanticipated consequences of purposive social action. american sociological review, 1(6), 894–904. doi.org/10.2307/2084615 merton, r. k. (1968). social theory and social structure. free press. monod, e., mayer, a. s., straub, d., joyce, e., & qi, j. (2024). from worker empowerment to managerial control: the devolution of ai tools' intended positive implementation to their negative consequences. information and organization, 34(1), 100498. doi.org/10.1016/j.infoandorg.2023.100498 nguyen, a., rai, a., & maruping, l. (2024). understanding the unintended effects of humanmachine moderation in addressing harassment within online communities. journal of management information systems, 41(2), 341–366. doi.org/10.1080/07421222.2024.2340831 australasian journal of information systems benbya, strich and trieu 2025, vol 29, provocation accounting for unintended consequences in is research 16 orlikowski, w. j. (1992). the duality of technology: rethinking the concept of technology in organizations. organization science, 3(3), 398–427. doi.org/10.1287/orsc.3.3.398 parks, r., xu, h., chu, c. h., & lowry, p. b. (2017). examining the intended and unintended consequences of organisational privacy safeguards. european journal of information systems, 26(1), 37–65. https://doi.org/10.1057/ejis.2015.10 petrakaki, d., barber, n., & waring, j. (2012). the possibilities of technology in shaping healthcare professionals: (re/de-) professionalisation of pharmacists in england. social science & medicine, 75(2), 429–437. doi.org/10.1016/j.socscimed.2012.03.039 pienta, d., thatcher, j. b., wright, r. t., & roth, p. l. (2024). an empirical investigation of the unintended consequences of vulnerability assessments leading to betrayal. journal of the association for information systems, 25(4), 1079–1116. doi.org/10.17705/1jais.00869 rana, n. p., chatterjee, s., dwivedi, y. k., & akter, s. (2022). understanding dark side of artificial intelligence (ai) integrated business analytics: assessing firm’s operational inefficiency and competitiveness. european journal of information systems, 31(3), 364–387. doi.org/10.1080/0960085x.2021.1955635 robey, d., & boudreau, m.-c. (1999). accounting for the contradictory organizational consequences of information technology: theoretical directions and methodological implications. information systems research, 10(2), 167–185. doi.org/10.1287/isre.10.2.167 rogers, e. m. (1983). diffusion of innovations (3rd ed.). free press. (no doi available; cite using isbn: 9780029266502) rinta-kahila, t., someh, i., gillespie, n., indulska, m., & gregor, s. (2022). algorithmic decision-making and system destructiveness: a case of automatic debt recovery. european journal of information systems, 31(3), 313–338. doi.org/10.1080/0960085x.2021.1960905 sarker, s., chatterjee, s., xiao, x., & elbanna, a. (2019). the sociotechnical axis of cohesion for the is discipline: its historical legacy and its continued relevance. mis quarterly, 43(3), 695– 720. doi.org/10.25300/misq/2019/13747 soliman, w., & rinta-kahila, t. (2020). toward a refined conceptualization of is discontinuance: reflection on the past and a way forward. information & management, 57(2), 103167. doi.org/10.1016/j.im.2019.05.002 trieu, v.-h., burton-jones, a., green, p., & cockcroft, s. (2022). applying and extending the theory of effective use in a business intelligence context. mis quarterly, 46(1), 645–678. doi.org/10.25300/misq/2022/14880 trocin, c., lee, g., bernardi, r., & sarker, s. (2025). how do unintended consequences emerge from ehr implementation? an affordance perspective. information systems journal, 35(1), 39– 70. doi.org/10.1111/isj.12526 walsham, g. (2012). are we making a better world with icts? reflections on a future agenda for the is field. journal of information technology, 27(2), 87–93. doi.org/10.1057/jit.2012.4 watson, r. t., desanctis, g., & poole, m. s. (1988). using a computerized aid to support information systems development. mis quarterly, 12(1), 87–106. doi.org/10.2307/249019 wolff, j. (2016). perverse effects in defense of computer systems: when more is less. journal of management information systems, 33(2), 597–620. doi.org/10.1080/07421222.2016.1205934 australasian journal of information systems benbya, strich and trieu 2025, vol 29, provocation accounting for unintended consequences in is research 17 kim, y., ayvaci, m. u. s., raghunathan, s., & ayer, t. (2022). when it creates legal vulnerability: not just overutilization but under provisioning of health care could be a consequence. mis quarterly, 46(3), 1483–1515. doi.org/10.25300/misq/2022/16199 zhao, l., detlor, b., & connelly, c. e. (2016). sharing knowledge in social q&a sites: the unintended consequences of extrinsic motivation. journal of management information systems, 33(1), 70–100. doi.org/10.1080/07421222.2016.1172459 zhou, j., zhang, q., zhou, s., li, x., & zhang, x. m. (2023). unintended emotional effects of online health communities: a text mining-supported empirical study. mis quarterly, 47(1), 195–226. doi.org/10.25300/misq/2022/17018 copyright copyright © 2025 benbya, h, strich, f. and trieu, van-h. this is an open-access article licensed under a creative commons attribution-non-commercial 4.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v29.6063 https://creativecommons.org/licenses/by-nc/4.0/deed.en https://doi.org/10.3127/ajis.v29.6063 paper title (paper title style) australasian journal of information systems north m & north s 2019, vol 23, research article visualisation environments & pedagogy 1 dynamic immersive visualisation environments: enhancing pedagogical techniques max m. north visualization & simulation research cluster information systems department coles college of business kennesaw state university, georgia, usa max@kennesaw.edu sarah m. north visualization & simulation research cluster computer science department college of computing and software engineering kennesaw state university, georgia, usa abstract as the speed of technical and scientific innovation accelerates past the speed at which humans can learn, the need for innovative pedagogical environments and techniques, such as immersive visualization environments, becomes essential and apparent. the primary purpose of this research is to explore the effectiveness immersive visualization environments may provide in combination with rich resources available on the internet. an immersive visualization environment was designed and developed, and then an experimental research model was considered and conducted accordingly. collected data were subjected to appropriate statistical analysis. the results showed a statistically significant difference in pedagogical outcomes when using an immersive visualization environment compared with traditional educational techniques. immersive visualization environment models and experiments showed significant improvement in the effectiveness and efficiency of pedagogical techniques, enhancing the learning and teaching of abstract and complex computing concepts. keywords: immersive, visualization, immersive visualization environment, immersion, virtual environments, pedagogy techniques 1 introduction there is a growing gap between the pace of technical and scientific innovation and the speed with which humans can learn new concepts and knowledge. while the former is accelerating, the latter has thus far remained fairly constant (gilbert 2001; mikropoulos & natsisl, 2011; akbulut, catal & yıldız, 2018). consequently, this gap is more apparent with younger generations of learners. new and stimulating innovative pedagogical environments, techniques, concepts and materials will have to be developed to address this imbalance. one of the most state-of-the-art prospects is the immersive visualization environment (ive), the efficacy of which has been supported by empirical studies. ive creates a strong perception of being physically present in a computer-generated environment that includes images, sound, and other stimuli (a detailed definition and description of ive is presented in sections 1.2 and 1.3). australasian journal of information systems north m & north s 2019, vol 23, research article visualisation environments & pedagogy 2 this research endeavour integrates, in an innovative approach, the stimulating immersive power of visualization and simulation techniques in a unique immersive visualization environment with the richness of the educational materials available on the internet (north, sessum & zakalev, 2003; north, mathis, madajewski, brown & cupp, 2001a, 2001b; north & north, 2016; north, north, parks & webb, 2001). this research is technically and logically built upon best practices of current models of immersive visualization and on our effective pilot and prior research investigations. the principal aim of this study is to explore the efficacy of immersive visualization environments in combination with resources available on the internet. to that end, an immersive visualization environment was designed and developed, and an experimental research model was considered and implemented accordingly. a visualization simulation model was used consisting of a wire frame cube which represented a data structure. the cube appeared to hold the data in a green-coloured sphere. data were collected from participants using two instruments. collected data were subjected to appropriate statistical analysis and were reported. conclusions, discussions, implementations, and future research are presented. in the subsequent sub-sections, several major components of this research project are briefly explored and presented. it must be noted that the detailed and in-depth coverage of these components are not within the scope of this current article. hence, the central focus of this research is investigation of new collective and collaborative technologies to create dynamic immersive visualization environments for enhancing the effectiveness and efficiency of pedagogical techniques. 1.1 the need for innovative pedagogical techniques recently, there has been a tremendous increase in the use of advanced visualization technology for effective pedagogical practices as major researchers have undertaken multiple projects regarding this subject. immersive visualization and variations of virtual reality are among the technologies that hold great promise for the near future. furthermore, empirical studies emphasize that learners must be reached by trainers before they reach graduate level, a gateway to the opportunities and benefits of collective development (fyodorova, fyodorov & papulovskaya, 2018; elliman, loizou, & loizides, 2016; william, vidal & john, 2016; mikropoulos & natsisl, 2011; chertoff & schatz, 2014; hood & hood, 2005; ekaterini, spyros & panagiotis, 2004; ammoura, 2001; north, sessum & zakalev, 2003). most areas in science, engineering, technology and mathematics were included in the studies mentioned above, preparing learners not only in specific subject, but also enhancing skills in logical reasoning and collaboration. a prominent example of this innovative pedagogical technique is a project undertaken by researchers at university of california-berkeley. the national science foundation (nsf) funded visualizing to integrate science understanding for all learners (visual), marcia linn and colleagues (brown, 2001) to describe the dynamic visualizations of interactive, computer-based models, simulations, and virtual experiments of scientific phenomena, which provide alternative pathways for students to understand science concepts. the ability to harvest internet-based technology used in cyber-learning complements this advanced visualization technology. the provision of such individualized, customized, and collaborative forms of communication promotes sophisticated interaction between learning and teaching teams, which in turn enhances the effectiveness of the immersive visualization environment australasian journal of information systems north m & north s 2019, vol 23, research article visualisation environments & pedagogy 3 (brown, 2001). collectively, these technologies can provide an excellent, stimulating, and innovative approach for teaching and learning. 1.2 dynamic immersive visualization environment as a powerful pedagogical tool a wide variety of immersive visualization environments have been created and utilized (alvarez, bower, defreitas, gregory & dewit, 2016; berg & vance, 2017; maxwell, griffith & finkelstein, 2014; yu, mortensen, khanna, spanlang & slater, 2012; chittaro & ranon, 2007). for instance, a fully immersive “virtual physics laboratory” has been developed by loftin and colleagues (1993). in this virtual laboratory, students are participants and carry out experiments that are otherwise impossible in a computer laboratory. baird (2001) reports about an nsf-funded arts and technology project in which students and faculty collaboratively created curricular models using virtual reality and simulation to explain scientific principles. this project is based on the idea that students today expect a completely different classroom environment and examines several different concepts, such as human physiology, molecular events, mathematical graph theory, cell biology, and electric potential functions in physics. many effective research studies specifically emphasize the following learning techniques for complex concepts such as computing (hood & hood, 2005). research demonstrates that an immersive visualization environment clearly supports each of these pedagogical techniques: • autonomous learning. educational and learning research attests to the fact that learners construct their understanding of a topic through active reasoning that connects with their prior experiences. thus, activities which promote autonomous learning help students make sense of new concepts and connect them to previous knowledge and experience (shive, bodzin, & cates, 2004). • collaborative learning. many learning theorists believe students learn better when they learn with peers who share comparable backgrounds and interests. intrinsically, collaborative learning encourages collaborative reasoning, which in turn promotes more effective learning of concepts (bodzin & cates, 2003). • visualization learning. “a picture is worth a thousand words” is an important adage that can be easily utilized by learners to understand abstract and complex concepts. researchers report that creative insight and problem-solving can be significantly improved with appropriate visualization techniques and environments (iachini et al., 2018; waisel, wallace, & willemain, 1999; iachini et al., 2018; healey, booth & enns, 1996; card, mackinlay & shneiderman, 1999; north & north, 2016). the research also shows that visualization is effective because it extends working memory (gathercole & baddeley, 2003; kiearas & myer, 2005). immersive visualization and virtual reality hold great promise for enabling these techniques to create significant advances in the communication of complex ideas and concepts such as science, technology, engineering and mathematics as learners and instructors interact with the learning content, objects, and concept representations. linn and slotta (2012) describe the dynamic visualizations as interactive, computer-based models, simulations, and virtual experiments of scientific phenomena, which provide alternative pathways for learners to understand science concepts. although there is contradictory and inconclusive research concerning the educational value of immersive visualization in the literature (corliss & australasian journal of information systems north m & north s 2019, vol 23, research article visualisation environments & pedagogy 4 spitulnik, 2008), learners’ preference for visualization as a learning technique is ten times higher than other techniques such as explanation, reading, partner learning, and teacherdirected learning. there are several important reasons for using an immersive visualization environment over standard virtual reality devices (e.g., using a head-mounted device or hmd): • the user has the freedom to normally and naturally move within the immersive visualization environment and is not constrained by an hmd or similar devices (chertoff & schatz, 2014; north & north, 2016; north, sessum, & zakalev, 2004); • collaboration and interaction are possible, either with other learners or with teachers within the same environment (berg & vance, 2017); • the virtual scenes are available at any given time instantly and allow for smooth navigation and exploration by users (elliman, loizou & loizides, 2016); and • the immersive visualization environment allows for a high degree of detail, whereas hmd-based systems suffer from the limitations of low resolution (loftin, engleberg & benedetti, 1993; north & north, 2016). considering the limited reasons above and prior empirical research, it was feasible and rational to conduct the current investigation. furthermore, there are a sufficient number of distinct implementations of ive, a few of which have been detailed in this paper. concepts are presented in the form of 3d geometric objects in a fully immersive virtual environment, where the user learns by walk-through, collaboration, and interaction with the objects making up the virtual world. this novel instructional approach takes advantage of the human senses, which are not always fully utilized in traditional methods of instruction. the immersive environment goes beyond a traditional multimedia system by providing its users with a sense of presence. immersion and the feeling of really being there are commonly acknowledged as uniquely important features of immersive environments. how immersed one feels is determined by a complex set of physical aspects and components of the environment, as well as psychological processes that, as of yet, are not sufficiently understood. simply put, pimentel and teixeira (1993) state that the experience of being immersed in a computer-generated world involves the same mental shift of "suspending your disbelief for a period of time" as "when you get wrapped up in a good novel or become absorbed in playing a computer game." 1.3 determinants of immersive visualization environment there is an extreme technological and practical shift from conventional multimedia to immersive visualization environments, providing immersion, interaction, intuition, and excitement (huang, rauch & liaw, 2010; padilha, machado, ribeiro & ramos, 2018; deleo, diggs, radici & mastaglio, 2014; burdea & coiffet, 2003; iachini et al., 2018; ameerbakhsh, 2018). the effectiveness of ive technologies in learning situations has been demonstrated in many disciplines, including education, computing, health & environment, and commerce (hew & cheung, 2010). the major contributing factor of ive success is the sense of presence (termed “virtual presence” for simplicity) that users experience. established by the former research in the general ive filed, the assertions concerning “virtual presence” (north, north & coble, 1996; north & north, 2016) are briefly presented below: australasian journal of information systems north m & north s 2019, vol 23, research article visualisation environments & pedagogy 5 • a person's experience of a situation in an ive may evoke the same reactions and emotions as an experience in a similar real-world situation. • a person may experience virtual presence similar to the real world even when the ive does not accurately or completely represent the real-world situation. • each person brings her own background into an ive experience. • experience with an ive increases the participant's sense of virtual presence. • the sense of presence in virtual and physical environments is constant. subjects have to give up the sense of presence in a physical environment in order to achieve a stronger sense of presence in the virtual one. • subject concentration increases significantly in the virtual world as compared to the physical world when the subject has enough interaction to develop a strong sense of virtual presence. • a person's perceptions of real-world situations and behaviour in the real world may be modified based on her experiences within a virtual world. based on assertions extrapolated from a series of studies, three major specific determinants of virtual presence are stimulus, interaction and experience; these determinants directly correlate with earlier suggested concepts of immersion, interaction, and imagination (burdea & coiffet, 2003; iachini et al., 2018; north & north, 2016; north, north, parks & webb, 2001). while the existing assertions evidently support the notion that ive will be able to improve the depth of the experience of users, it is not clear that ive will contribute to more effective and efficient pedagogical techniques when compared to traditional ones. thus, this research is going to investigate ive’s contribution to the enhancement of pedagogical techniques. 1.4 the internet as a multi-faceted learning environment literature reviews in the pedagogical field show great evidence and potential for learning and teaching using the internet and its resources (linn & eylon, 2006; linn & slotta, 2012, 2009; monahan, mcardle & bertolotto, 2008; akbulut, catal & yıldız, 2018). for example, several web-based inquiry science environment (wise) projects at the university of californiaberkeley developed and tested web-based science materials designed using a knowledge integration (ki) framework to ensure in-depth student learning (corliss & spitulnik, 2008). as an illustration, the search for prominent rich content for abstract and complex computing concepts leads to a decade of research in the information visualization area by stasko (1995; 1997), professor at the school of interactive computing, georgia institute of technology. a central focus of many of his group's projects is the creation of information visualization and visual analytics tools to help people explore, analyse, and understand large data sets. in particular, his work concentrates on creating visual analytics systems to help people with "sense-making" activities on data sets such as large document collections. mukherjea and stasko (1994; 1995) used visualizations to expose the "insides" of computer programs, making evident attributes of a program that are difficult to otherwise assess. jerding and stasko (1998) have developed new visualization techniques that show the function calls, message traffic and instance creation of large programs. another project augmented a debugger with visualizations depicting the code, run-time calls, stack, and variables of a program. finally, his group has developed a system called lens that allows programmers to create animation-style presentations for program debugging (jerding, stasko & ball, 1997). most importantly, no text australasian journal of information systems north m & north s 2019, vol 23, research article visualisation environments & pedagogy 6 coding is required to create the views. furthermore, stasko and his colleagues developed the xtango system to facilitate the animation of algorithms developed in windows using the c programming language. this illustration demonstrates the power of rich resources that are available on the internet. as a matter of fact, many of stasko’s visualization techniques have been directly incorporated into the content of our current research project. at this juncture, only a brief research report of this topic of the internet as a multi-faceted learning environment and an example are presented here, as an in-depth report is not within the scope of this current article. 2 empirical research methodology 2.1 primary goal, objectives, research questions & null hypotheses the main goal of this research is the investigation of a new collaborative technology that not only takes advantage of what immersive visualization technology can provide but also incorporates the rich resources available on the internet for use by trainers and learners, during educational experiences and beyond. in this study, the main objective is to design an experiment to investigate all three of the initial and general research questions that are stated as follows: • research question 1 [rq-1]: is the proposed immersive visualization environment pedagogical technique with complementary resources on the internet more effective and efficient compared to the traditional educational technique? • research question 2 [rq-2]: does the proposed immersive visualization environment pedagogical technique help learners to understand abstract and complex computing concepts better than traditional educational techniques? • research question 3 [rq-3]: does the proposed immersive visualization environment pedagogical technique speed up the learning and teaching process? • null hypothesis 1 [h0-1]: there is no statistically significant difference between the proposed immersive visualization environment pedagogical technique with complementary resources on the internet and traditional educational technique in providing effective and efficient learning. • null hypothesis 2 [h0-2]: there is no statistically significant difference between the proposed immersive visualization environment pedagogical technique and the traditional educational technique in helping learners to understand abstract and complex computing concepts. • null hypothesis 3 [h0-3]: there is no statistically significant difference between the proposed immersive visualization environment pedagogical technique and the traditional educational technique in the speed of learning and teaching process. terminology: for improved readability and comprehension, the terminology “immersive visualization environment” and “ive” will be used interchangeably and simultaneously throughout this paper, as will “traditional educational technique” and “tet”. 2.2 experimental design based on preceding ive models background research, including authors’ prior pilot studies, has demonstrated the possibility and feasibility of the effectiveness of immersive visualization technology for use with australasian journal of information systems north m & north s 2019, vol 23, research article visualisation environments & pedagogy 7 computing concepts such as data structures and algorithms (north, mathis, madajewski, brown & cupp, 2001a, 2001b; munro, patrey, biddle & carroll, 2014; north, north, parks & webb, 2001; vygotsky, 1980; slotta & linn, 2009; north & north, 2016). authors have created a basic dynamic immersive visualization model of a vector container, which was used as the atomic component for the other data structure containers, using wired-looking cubes to represent the containers and a coloured sphere to represent the data inside the container. the same immersive graphical representation is utilized to develop other data structures containers, such as queues, stacks, and trees. 2.3 participants and site for ive and tet experiments one hundred twenty eight (n=128) volunteer undergraduate students from the college of computing & software engineering and coles college of business participated in this study by using the computer concepts utilized in our prior pilot research projects (north, sessum & zakaley, 2003). selected students (subjects) randomly were assigned to experimental (ive) and control (tet) groups (ive group-[n=64] & tet group-[n=64]). the site of the ive experiment was the university’s visualization & simulation research cluster laboratory, and the site for the tet experiment was a standard computer laboratory/classroom in an adjunct facility. 2.4 apparatus and instruments the system’s envelope of the current immersive visualization environment (located at visualization & simulation research cluster at ksu) provided users a dramatic, 210º field of view, which results in an integration of the operators and the displayed information, thus providing an immersive environment for this project (figure 1, a & b). inspired by and utilizing visualization techniques developed by stasko and his colleagues, a visualization simulation model was used consisting of a container that resembles a wire frame cube (figure 1, c & d). the cube appeared to hold the data in a green coloured sphere. textured mapping techniques were employed to add to the realism of the objects in the virtual scene. for this study, the virtual container simulated a vector-based container and allowed users to freely execute operations on this container. to test the efficiency of learning, the self-reported rating of learning efficiency in the ive and tet activities were compared. two major instruments were developed and used for this experiment. one instrument was a standard multiple-choice and true/false test instrument with a 100-point scale (zero being the lowest and 100 being the highest score). this instrument was used as preand post-experience objective instrument for null hypothesis h0-1 and for null hypothesis h0-2, measuring the level of understanding of abstract and complex computing concepts by both ive and tet participants. the second instrument was a selfreported, post-experience subjective instrument with a scale of zero to ten; this instrument was used to evaluate null hypothesis h0-3, measuring whether the proposed immersive visualization environment pedagogical technique speeds up the learning and teaching process. australasian journal of information systems north m & north s 2019, vol 23, research article visualisation environments & pedagogy 8 figure 1. illustrates (a) ive-immersive visualization environment system, (b) schematic configuration and specification of the ive, (c) simulated vector-based data structure container in wire frame cube shapes, and (d) a subject engaged in the ive experiment, interacting with and manipulating virtual objects of a vector data structure. 2.5 procedures for conducting ive and tet experiments the experimental group participants were exposed to an immersive visualization environment, while the control group participants were assigned to the traditional pedagogical technique (using standard desktop computers including presentation technology). limited computer concept modules were administered to both groups after collecting pre-tests. the experiment consisted of four separate phases. phase i: the internet resources were extensively explored to identify appropriate visualization modules for computing concepts that were already available from other resources (such as other universities or software development companies). the authors and their research assistants contacted several of these resources either on the web or in the particular laboratory for using their developed software modules (see section 1.4 for detailed description). as the internet provides enormous resources and continues to grow, the authors will continue to search and contact other resources to secure additional available visualization modules for future. the following modules were developed based on these resources; (i) vectors, (ii) strings, (iii) lists, including linked list operations, (iv) double-ended queues, (v) stacks, (vi) queues and priority queues, (vii) sets and multisets, (viii) trees and graphs, (ix) sorts and searches. phase ii: after testing the module contents with several subjects in the laboratory using a formative evaluation to examine the usability and pedagogical effectiveness, expert-based and user-based evaluation approaches were adopted. afterward, the software was installed on a visualization and simulation research laboratory server and displayed on the immersive ive specification: 16-ft diameter dome system, 210degree horizontal field of view, 16ft width x 10-ft depth x 9-ft height, and four digital projection ivision 20 sx+ projection systems, 1-chip dlp projectors, and optimized 6segment colour wheel for optimum colours, and 1.7-2.2:1 zoom lens. (a) (b) (c) (d) australasian journal of information systems north m & north s 2019, vol 23, research article visualisation environments & pedagogy 9 screen. participants (n=128) were selected randomly and assigned to the ive experimental (n=64) and tet control (n=64) groups. pre-tests were administered to both groups. phase iii: all participants were given an explanation of the experiments and instructions about interaction protocols with the environments before actually experiencing the ive or tet experiments. authors replied to all the participants’ questions and secured their signed consent forms for participating in experiments. the experimental (ive) group participants viewed the computer concepts modules, which were represented by the vector-based containers, while control (tet) group participants were given regular laboratory-based learning using standard computers. phase iv: post-tests were administered to both groups (ive and tet) that measured the comprehension of provided computing concepts based on the three research questions and stated hypotheses (see section 2.1 for a detailed description). upon completion of this phase, data analysis was performed using a variety of appropriate statistical procedures. 3 results and conclusions a standard preand post-test protocol was administered. analysis of collected data showed that there was no statistically significant difference between pre-tests of experimental (ive) and control (tet) groups. only the experimental group was exposed to the immersive learning modules using ive. the detailed analysis of post-tests of ive and tet groups are presented. concerning research question 1 [rq-1], the mean (m=87.92, sd=15.95) of experimental group scores is higher than the mean (m=74.28, sd=21.64) of control group scores; this is shown in table 1. using t-test for analysis of equality of means, analysis of the collected scores of posttests of the experimental and control groups showed that, at a 1% level of significance (ttest=4.06, df=126 [n-2], p<0.001), h0-1 was rejected. thus, there is a statistically significant difference between ive & tet techniques: the ive pedagogical technique is more effective and efficient. in another words, the use of selected computing concepts using immersive visualization environments tends significantly (statistically) to increase the understanding and mastery of the selected contents (table 1 [rq-1 row] and figure 2). furthermore, the analysis of the current results approximately correlates and aligns with the prior pilot experiment results (north & north, 2016). the mean (m=82.43, sd=17.02) of the experimental (ive) group scores is higher than the mean (m=66.85, sd=13.65) of the control (ive) group scores for research question 2 [rq-2] (see table 1). the t-test value of analysis of the collected scores of post-tests of the ive and tet groups showed that, at a 1% level of significance (t-test=5.71, df=126 [n-2], p<0.001), h0-2 was rejected. hence, there is a statistically significant difference between ive & tet techniques; the ive pedagogical technique better helps with the understanding of abstract and complex computing concepts (see table 1 [rq-2 row] and figure 3). similar to the analysis of h0-1, these results correlates and aligns with the authors’ pilot study results (north & north, 2016). australasian journal of information systems north m & north s 2019, vol 23, research article visualisation environments & pedagogy 10 comparison of immersive visualization environment and traditional pedagogical techniques research question description experimental group (ive) post-test (n=64) control group (tet) post-test (n=64) analysis hypothesis decision and interpretation mean (n=64) sd mean (n=64) sd t-test (df=126) rq-1: is the proposed immersive visualization environment pedagogical technique with complementary resources on the internet more effective and efficient compared to the traditional educational technique? 87.92 15.95 74.28 21.64 t = 4.06 p < 0.001 h0-1 is rejected [p < 0.001]. there is a statistically significant difference between ive & tet techniques. ive pedagogical technique is more effective and efficient. rq-2: does the proposed immersive visualization environment pedagogical technique help learners to understand abstract and complex computing concepts better than traditional educational techniques? 82.43 17.02 66.85 13.65 t = 5.71 p < 0.001 h0-2 is rejected [p < 0.001]. there is a statistically significant difference between ive & tet techniques. ive pedagogical technique better helps with understanding of abstract and complex computing concepts. rq-3: does the proposed immersive visualization environment pedagogical technique speed up the learning and teaching process? 7.63 1.83 5.37 2.43 t = 5.94 p < 0.001 h0-3 is rejected [p < 0.001]. there is a statistically significant difference between ive & tet techniques in regard to speed of learning and teaching process. note: both post-test instruments for ho-1 and h0-2 had scale between 0 (lowest score) to 100 (highest score), while the subjective instrument for h0-3 had scale between 0 (lowest rating) to 10 (highest rating). ive=immersive visualization environment pedagogical technique tet=traditional educational technique table 1. comparison of the experimental group (ive) and the control group (traditional educational technique-tet) post-tests using matched pair t-test statistical analysis. australasian journal of information systems north m & north s 2019, vol 23, research article visualisation environments & pedagogy 11 figure 2. illustrates in graph depictions that ive pedagogical technique using rich internet resources is more effective and efficient compared to a traditional educational technique (tet). figure 3. depicts graphically that the ive technique helps learners with the understanding of abstract and complex connects in comparison with tet. figure 4. demonstrates that the ive technique speeds up learning and teaching processes when compared to tet alternative technique. regarding research question 3 [rq-3], the mean (m=7.63, sd=1.83) of the experimental (ive) group scores is higher than the mean (m=5.37, sd=2.43) of the control (tet) group scores, as shown in table 1. the t-test value of analysis of the collected scores of the post-tests of the ive and tet groups showed that, at a 1% level of significance (t-test=5.94, df=126 [n-2], p<0.001), h0-2 was rejected. thus, there is a statistically significant difference between ive & tet 87.92 74.28 15.95 21.64 0 50 100 experimental group (ive) control group (tet) sc o re s (z er o t o 1 0 0 ) research question 1: is the proposed ive pedagogical technique with complementary resources on the internet more effective and efficient compared to the traditional educational technique (tet)? analysis: affirmative, a significant difference. mean of scores standard deviation australasian journal of information systems north m & north s 2019, vol 23, research article visualisation environments & pedagogy 12 techniques with respect to the speed of the learning and teaching process (see table 1 [rq-3 row] and figure 4). the immersive visualization environment pedagogical technique sped up the learning and teaching processes when compared with a traditional educational technique. in summary, all three null hypotheses (h0-1, h0-2, and h0-3) were accepted (they showed a statistically significant difference); thus, all three research questions were affirmed. specifically, research question 1 [rq-1] revealed that the proposed immersive visualization environment pedagogical technique with complementary resources on the internet is more effective and efficient compared to traditional educational techniques. secondly, research question 2 [rq-2] showed that the proposed immersive visualization environment pedagogical technique helps learners to understand abstract and complex computing concepts better than traditional educational techniques. lastly, research question 3 [rq-3] confirmed that the proposed immersive visualization environment pedagogical technique speeds up the learning and teaching process. 4 discussions, implications, limitations, and future research the goal of the proposed effort was to enable significant advances in learner/teacher learning of abstract and complex computing concepts through the implementation of an immersive visualization environment that utilizes internet resources. the results showed that the ive technique is more effective and efficient in comparison with traditional educational techniques. these results raise several plausible implications and possibilities for future research in this specific sub-field. not only could this ive technique be used for other computing contents, it might also be used effectively in many sites and circumstances across a variety of disciplines (ricketts, salsbury, bevan & brown, 2018). there are enormous benefits of ive techniques that will directly and indirectly impact teachers and learners in science and technology by supporting pedagogical techniques such as autonomous, collaborative, and visualization learning approaches (camba, soler & contero, 2017; freina & ott, 2015). clearly, the authors’ prior and current research of ive shows higher effectiveness and efficiency of learning for students in a university setting. this technique could be transferred into other teaching and learning environments with some minor redesign and reconfiguration. while the results from this project (modules, immersive visualization software, systems configuration) could be widely disseminated and shared (through servers on the internet, journals, books, regional and national workshops) to promote this hybrid technology, it would be more effective and efficient if a standard classroom/laboratory could provide a similar environment. although our visualization & simulation research cluster/laboratory will be available to other researchers to conduct similar research and training activities, authors are diligently working to create a miniature and/or portable version of the immersive visualization technology that could be used in remote locations. authors and research assistants are setting up an inexpensive version of the ive system using existing computers and leading-edge display architecture (e.g., new technology like google oculus glass – an optical head-mounted display that induces an immersive experience for users). despite the fact that the immersive visualization environment techniques are shown to be effective and efficient in this experiment, there are not enough adequate and solid empirical research clearly showing that ives increase learning with long-term. moreover, designing, developing and implementing ive contents are difficult and time consuming. additionally, australasian journal of information systems north m & north s 2019, vol 23, research article visualisation environments & pedagogy 13 while the adoption of ive for a standard educational setting is becoming technically and economically more available (see a few possible approaches that are described below by authors), there is still a huge need for technological and pedagogical advancements to provide easy to use and flexible systems. another key shortcoming of the current ive learning/teaching technique is the system’s inability to interpret performance based on influencing factors such as cognitive and affective learning states. it is believed that a significant portion of the performance gap between general current computer-based learning and human tutors lies in the capability to be aware and respond to the learner’s state (north & north, 2016; north, sessum & zakalev, 2003; north, mathis, madajewski, brown & cupp, 2001a, 2001b; north, north, parks & webb, 2001). thus, learning effectiveness and efficiency could be significantly enhanced if the system was provided with a real-time assessment of key information about the learner’s cognitive state and attention allocation, as this information could then be used to individualize the system’s feedback to the learner, just as a human instructor would consider these conditions. similar to prior ive experiments (north & north, 2016), research in this particular field shows that subjective measurements positively correlate with objective measurement; physiological measurements—such as eye-tracking and eeg (electroencephalogram – a non-invasive monitoring method to record electrical activity of the brain)—will be utilized during all stages of future experiments. finally, while there are some limitations, the broader impact of this project rests in its potential to enhance learning for learners with different learning styles. ive technology stimulates and uses all of the human senses, whereas traditional approaches to learning focus more on intellectual skills. immersive visualization considers visual as well as non-visual learners by providing a combination of stimuli with this rich and unique environment. the ive approach could well be extended to other disciplines such as information technology, engineering, mathematics, and others (inoue, uehira & koike, 2018). acknowledgment this effort was supported by an immersive visualization environment equipment grant from the army research office (aro) of the department of defense (dod) and assistance from the pslsamp program (state-wide grant by nsf–national science foundation). the content of this work does not reflect the position or policy of the aro, dod or nsf, and no official endorsement should be inferred. references akbulut, a., catal, c., & yıldız, b. (2018). on the effectiveness of virtual reality in the education of software engineering. computer applications in engineering education. alvarez, v., bower, m., de freitas, s., gregory, s., & de wit, b. (2016). the use of wearable technologies in australian universities: examples from environmental science, cognitive and brain sciences and teacher training. mobile learning futures–sustaining quality research and practice in mobile learning, 25. ameerbakhsh, o. (2018). towards the use of interactive simulation for effective e-learning in university classroom environment. ammoura, a. (2001). dive-on: from databases to virtual reality. crossroads, 7(3), 4-9. australasian journal of information systems north m & north s 2019, vol 23, research article visualisation environments & pedagogy 14 baddeley, a. (2003). working memory and language: an overview. journal of communication disorders, 36(3), 189-208. baird, b. (2001). curricular modules: 3d and immersive visualization tools for learning. syllabus–visualization expanding the mind's eye, 14(9), 23-26. berg, l. p., & vance, j. m. (2017). industry use of virtual reality in product design and manufacturing: a survey. virtual reality, 21(1), 1-17. bodzin, a. m., & cates, w. m. (2003). enhancing preservice teachers' understanding of webbased scientific inquiry. journal of science teacher education, 14(4), 237-257. brown, d. g. (2001). the teaching profession in 2020. syllabus–visualization expanding the mind’s eye, 14(9), 21. burdea, g. c., & coiffet, p. (2003). virtual reality technology. john wiley & sons. camba, j. d., soler, j. l., & contero, m. (2017, july). immersive visualization technologies to facilitate multidisciplinary design education. in international conference on learning and collaboration technologies (pp. 3-11). springer, cham. card, m. d. [mackinlay, and b. shneiderman] (1999). readings in information visualization: using vision to think. morgan kaufmann. chertoff, d., & schatz, s. (2014). beyond presence: how holistic experience drives training and education. in handbook of virtual environments (pp. 857-871). crc press. chittaro, l., & ranon, r. (2007). web3d technologies in learning, education and training: motivations, issues, opportunities. computers & education, 49(1), 3-18. corliss, s. b., & spitulnik, m. w. (2008, june). student and teacher regulation of learning in technology-enhanced science instruction. in proceedings of the 8th international conference on international conference for the learning sciences-volume 1 (pp. 167-174). international society of the learning sciences. duzgun, s., isleyen, e., orsvuran, r., bozdag, e., pugmire, d., lei, w., ... & tromp, j. (2018, december). exploring earth's interior in collaborative immersive vr environments. in agu fall meeting abstracts. ekaterini, g., spyros, b., & panagiotis, g. (2003, june). teaching it in secondary education through problem-based learning could be really beneficial. in acm sigcse bulletin (vol. 35, no. 3, pp. 243-243). acm. elliman, j., loizou, m., & loizides, f. (2016, september). virtual reality simulation training for student nurse education. in games and virtual worlds for serious applications (vs-games), 2016 8th international conference (pp. 1-2). ieee. freina, l., & ott, m. (2015, april). a literature review on immersive virtual reality in education: state of the art and perspectives. in the international scientific conference elearning and software for education (vol. 1, p. 133). " carol i" national defence university. fyodorova, s. v., fyodorov, k. a., & papulovskaya, n. v. (2018, october). educational environment for training future engineers for enterprises of mining and metallurgical complex. in 2018 iv international conference on information technologies in engineering education (inforino) (pp. 1-4). ieee. australasian journal of information systems north m & north s 2019, vol 23, research article visualisation environments & pedagogy 15 gilbert, s. w. (2001). no moore's law for learning. syllabus–technology implementation, 14(8), 28. corliss, s. b., & spitulnik, m. w. (2008, june). student and teacher regulation of learning in technology-enhanced science instruction. in proceedings of the 8th international conference on international conference for the learning sciences-volume 1 (pp. 167-174). international society of the learning sciences. healey, c. g., booth, k. s., & enns, j. t. (1996). high-speed visual estimation using preattentive processing. acm transactions on computer-human interaction (tochi), 3(2), 107-135. hew, k. f., & cheung, w. s. (2010). use of three‐dimensional (3‐d) immersive virtual worlds in k‐12 and higher education settings: a review of the research. british journal of educational technology, 41(1), 33-55. hood, c. s., & hood, d. j. (2005, june). toward integrating computing concepts into the k-12 curriculum. in acm sigcse bulletin (vol. 37, no. 3, pp. 375-375). acm. huang, h. m., rauch, u., & liaw, s. s. (2010). investigating learners’ attitudes toward virtual reality learning environments: based on a constructivist approach. computers & education, 55(3), 1171-1182. iachini, t., maffei, l., masullo, m., senese, v. p., rapuano, m., pascale, a., ... & ruggiero, g. (2018). the experience of virtual reality: are individual differences in mental imagery associated with sense of presence?. cognitive processing, 1-8. inoue, t., uehira, k., & koike, a. (2018, august). immersive visualization of 3d protein structures for bioscience students. in congress of the international ergonomics association (pp. 432-439). springer, cham. jerding, d. f., & stasko, j. t. (1998). the information mural: a technique for displaying and navigating large information spaces. ieee transactions on visualization and computer graphics, 4(3), 257-271. jerding, d. f., stasko, j. t., & ball, t. (1997, may). visualizing interactions in program executions. in proceedings of the 19th international conference on software engineering (pp. 360-370). acm. kieras, d. e., & meyer, d. e. (2005). epic: a cognitive architecture for computational modeling of human performance. available on-line at http://ai. eecs. umich. edu/people/kieras/epic. html. linn, m. c., & eylon, b. s. (2006). science education: integrating views of learning and instruction. in handbook of educational psychology, 2nd ed., lawrence erlbaum associates. linn, m. c., & slotta, j. d. (2012). the knowledge integration environment: helping students use the internet effectively. in innovations in science and mathematics education (pp. 202235). routledge. loftin, r. b., engleberg, m., & benedetti, r. (1993, october). applying virtual reality in education: a prototypical virtual physics laboratory. in proceedings of 1993 ieee research properties in virtual reality symposium (pp. 67-74). ieee. australasian journal of information systems north m & north s 2019, vol 23, research article visualisation environments & pedagogy 16 maxwell, d., griffith, t., & finkelstein, n. m. (2014). use of virtual worlds in the military services as part of a blended learning strategy. in handbook of virtual environments (pp. 956-996). crc press. mikropoulos, t. a., & natsis, a. (2011). educational virtual environments: a ten-year review of empirical research (1999–2009). computers & education, 56(3), 769-780. monahan, t., mcardle, g., & bertolotto, m. (2008). virtual reality for collaborative elearning. computers & education, 50(4), 1339-1353. mukherjea, s., & stasko, j. t. (1994). toward visual debugging: integrating algorithm animation capabilities within a source-level debugger. acm transactions on computerhuman interaction (tochi), 1(3), 215-244. mukherjea, s., & stasko, j. t. (1993, may). applying algorithm animation techniques for program tracing, debugging, and understanding. in proceedings of the 15th international conference on software engineering (pp. 456-465). ieee computer society press. munro, a., breaux, r., patrey, j., & sheldon, b. (2002). cognitive aspects of virtual environments design. handbook of virtual environments: design, implementation, and applications, 415-434. north, m. m., mathis, j. r., madajewski, a., brown, j. t., & cupp, s. m. (2001a). virtual reality and transfer of learning. in proc. 9th int’l conf. human-computer interaction, 636-638. north, m. m., mathis, j. r., madajewski, a., brown, j. t., & cupp, s. m. (2001b). application of virtual reality in learning scenarios. in proc. 39th ann. acm southeast conf, 311-312. north, m. m., & north, s. m. (2016). a comparative study of sense of presence of traditional virtual reality and immersive environments. australasian journal of information systems, 20. north, m. m., north, s. m. & coble, j. r. (1996). virtual reality therapy: an innovative paradigm. ipi press. north, m. m., north, s. m., parks, b., & webb, d. (2001). factors affecting sense of presence in virtual reality therapy. in proc. medicine meets virtual reality. north, m. m., sessum, j., & zakalev, a. (2004). immersive visualization tool for pedagogical practices of computer science concepts: a pilot study. journal of computing sciences in colleges, 19(3), 207-215. padilha, j. m., machado, p. p., ribeiro, a. l., & ramos, j. l. (2018). clinical virtual simulation in nursing education. clinical simulation in nursing, 15, 13-18. pimentel, k., & teixeira, k. (1993). virtual reality through the new looking glass. ricketts, h. k., salsbury, a. m., bevan, d. r., & brown, a. m. (2018, july). using immersive visualization environments to engage students in hands-on learning. in proceedings of the practice and experience on advanced research computing (p. 74). acm. shive, l. e., bodzin, a. m., & cates, w. m. (2004). a national science standards-based study of web-based inquiry in chemistry. journal of chemical education, 81(7), 1066. slotta, j. d., & linn, m. c. (2009). wise science: web-based inquiry in the classroom. teachers college press, columbia university. australasian journal of information systems north m & north s 2019, vol 23, research article visualisation environments & pedagogy 17 stasko, j. t. (1997). supporting student-built algorithm animation as a pedagogical tool (pp. 24-25). acm. stasko, j. t., & mccrickard, d. s. (1995). real clock time animation support for developing software visualizations. georgia institute of technology. vygotsky, l. s. (1980). mind in society: the development of higher psychological processes. harvard university press. waisel, l. b., wallace, w. a., & willemain, t. r. (1999, january). visualizing modeling heuristics: an exploratory study. in proceedings of the 20th international conference on information systems (pp. 166-177). association for information systems. william, a., vidal, v. l., & john, p. (2016). traditional instruction versus virtual reality simulation: a comparative study of phlebotomy training among nursing students in kuwait. journal of education and practice, 7(9), 18-25. yu, i., mortensen, j., khanna, p., spanlang, b., & slater, m. (2012). visual realism enhances realistic response in an immersive virtual environment-part 2. ieee computer graphics and applications, 32(6), 36-45. copyright: © 2019 north m & north s. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. http://creativecommons.org/licenses/by-nc/3.0/au/ paper title (paper title style) australasian journal of information systems eismann, fischer-preßler & fischbach 2022, vol 26, research on applied ethics applied ethics and digital information privacy 1 applied ethics and digital information privacy: informing the design of covid-19 contact tracing apps kathrin eismann university of bamberg, germany diana fischer-preßler university of bamberg, germany diana.fischer-pressler@uni-bamberg.de kai fischbach university of bamberg, germany abstract to counteract the spread of covid-19, many countries have introduced mobile applications for contact tracing, which raises considerable questions about how these apps protect users’ information privacy. through an exploratory analysis of covid-19 contact tracing apps being used in australia, france, germany, japan, and new zealand, we identify normative and technical principles for the design of privacy-sensitive contact tracing apps. based on a restricted access/limited control (ralc) account of information privacy, we discuss how the apps protect users’ information privacy through limiting access to and allowing users to actively manage their personal information. our findings illustrate what understanding of information privacy is evident from the various designs of covid-19 contact tracing apps, and how competing design principles can contribute to users’ information privacy. from a practical perspective, our findings can inform the design of contact tracing apps and the development of privacy approaches that can be applied in particular contexts. our work thus bridges the gap between ethical design guidelines and technical analyses of specific implementations. keywords: applied ethics, disclosive computer ethics, information privacy, mobile app, contact tracing, coronavirus, covid-19, crisis, pandemic, normative theory. 1 introduction the coronavirus pandemic has confronted public health officials throughout the world with the urgent need to contain infections and stop the spread of the highly contagious and harmful covid-19 disease. as it spread across the globe, borders were closed, social and economic life came to a standstill, and health infrastructures nearly collapsed. almost a year into the pandemic, neither medication nor vaccinations were available; in february 2021, the world health organization (who) had registered more than 105 million cases and more than 2.2 million fatalities due to covid-19 worldwide (who, 2021b). to counteract further spread of the virus and prevent sustained economic damage and the loss of human life, public health interventions through which people can avoid exposure to the coronavirus—such as cordons sanitaire, traffic restrictions, and social distancing—have gained momentum (e.g., di gennaro et al., 2020, pan et al., 2020). beyond that, the who has established contact tracing as a key strategy for interrupting transmission chains, as people who have been in contact with an infected person can be identified and advised to quarantine to prevent further transmissions (who, 2021a). australasian journal of information systems eismann, fischer-preßler & fischbach 2022, vol 26, research on applied ethics applied ethics and digital information privacy 2 however, classical procedures for identifying the contacts of an infected person can be ineffective, if, for instance, a person’s contacts are unknown or low levels of automation cause delays in the identification and contacting of people at risk (who, 2021a). therefore, governments have considered, and some have implemented, mobile applications—so-called contact tracing apps—to trace infections and provide health advice to citizens. mobile applications reduce reliance on human recall and hence can be more effective than classical procedures for contact tracing, especially in densely populated areas with high degrees of mobile connectivity (budd et al., 2020). these digital tools were thus ascribed an important role in limiting the propagation of the pandemic (who, 2021a). contact tracing apps are applications that run on a smartphone and allow tracing encounters between users. they are a technical approach to tracking epidemic chains of infection and stopping the spread of covid-19. basically, they are used to alert users who have been in contact with another infected user, but may not (yet) show typical symptoms of the covid-19 disease. those users can then be advised to self-isolate and get tested. users receive a warning via the app that they might be at risk of an infection if they have been close (e.g., within a couple of meters) to another user diagnosed with covid-19 if that proximity was for an epidemiologically relevant time interval (e.g., 15 or more minutes) within a relevant timeframe (e.g., an estimated incubation period of 14 days for the infected person). users so warned can then seek medical advice and take appropriate action. if they are themselves diagnosed with covid-19, they can update their status within the app so others who have been close can also receive warnings (ahmed et al., 2020). to enable effective tracing, however, apps need to collect sensitive personal data, which includes records of users’ health, movements, and social interactions (morley, cowls, taddeo, & floridi, 2020). experts warned early on that personal information collected through contact tracing apps could be exposed to a number of security risks that threaten user information privacy. privacy could be compromised due to, for instance, technical problems or cyberattacks, potential misuse of personal data due to large-scale social surveillance, and illegitimate repurposing of the information collected (e.g., klar & lanzerath, 2020). concerns over data protection and information privacy norms could negatively affect people’s intention to use these apps (e.g., chan & saqib, 2021, fischer, hattori-putzke, & fischbach, 2019; fodor & brem, 2015), which, in turn, could lower the efficacy of app-based contact tracing (trang, trenz, weiger, tarafdar, & cheung, 2020). several guidelines have been developed, both by researchers (e.g., floridi, 2020; klar & lanzerath, 2020) and public health institutions (e.g., european commission, 2020; who, 2020), to promote the design of apps for covid-19 contact tracing that preserve users’ information privacy. still, analyses suggest that despite government claims to honor citizen information privacy, several apps have failed to meet essential requirements of privacysensitive design (e.g., hatamian, wairimu, momen, & fritsch, 2021, woodhams, 2020). clearly, the risk of information privacy violations cannot be eliminated through technical implementation alone, but relies on a shared understanding of what constitutes information privacy violations through illegitimate access and use of personal information. our work bridges the gap between ethical design principles for and technical evaluations of covid-19 contact tracing apps. based on the restricted access/limited control (ralc) framework of information privacy (moor, 1997; tavani & moor, 2001), we evaluate the normative and technical design principles that have—explicitly or implicitly—guided the australasian journal of information systems eismann, fischer-preßler & fischbach 2022, vol 26, research on applied ethics applied ethics and digital information privacy 3 development of apps deployed as early as during the second half of 2020 in five liberal democracies. our analyses are based on two complementary research questions: rq1: what normative and technical design principles to protect information privacy are evident from covid-19 contact tracing apps? rq2: what understanding of information privacy is evident from the design of covid-19 contact tracing apps? going beyond purely normative approaches, we establish a frame of reference that explicates standards of privacy-sensitive design from the perspective of real-world mobile applications, thus emphasising principles that are both technically applicable and in line with the ethical standards of liberal democracies. in contrast to mere technical analyses, we elaborate how the identified design principles relate to ethical information-privacy requirements, and discuss how they might contribute to the achievement of normative goals. our research approach is one of explicit morality (stahl, 2012), as we explicate the normative understanding of information privacy that is evident from the design of existing covid-19 contact tracing apps. thus, we aim to clarify what users can expect with respect to the protection of their personal information when using these apps. while developing full-fledged ethical theory is beyond the scope of this paper, we identify normative and technical design principles that can guide the design of technical artifacts that allow large-scale collection of sensitive personal data. finally, our work demonstrates how concepts of normative theory can be analysed based on empirical design choices, which can encourage researchers to abstract information privacy conceptualisations and possible breaches within a certain artifact under investigation and evaluate alternative solutions in terms of how they afford information privacy. we apply disclosive computer ethics to analyse information privacy conceptualisations evident from existing covid-19 contact tracing apps, as described by brey (2000a, 2000b). key to our work is the assumption that while the apps have been designed to preserve information privacy in line with pertinent laws and social norms (e.g., european parliament, 2020), their design and other features determine the extent of information privacy when using the apps. based on an exploratory analysis of information-privacy principles evident from the five apps, we evaluate how the measures implemented relate to normative and technical protections of information privacy, as conceptualised by the ralc framework, and discuss how normative theory can hence guide the design of privacy-enhancing artifacts. below, we briefly delineate the theoretical notion of information privacy that underlies our work, and how covid-19 contact tracing apps might violate it. furthermore, we explain how normative and technical design principles, according to the ralc framework, could help protect information privacy. on that basis, we describe our research methodology and the findings of the exploratory analysis and theoretical evaluation, and discuss the theoretical and practical implications of our work. 2 information privacy in the context of covid-19 contact tracing apps information privacy is a subset of general privacy (bélanger & crossler, 2011) that refers specifically to personal information (smith, dinev, & xu, 2011), which, in turn, implies “facts about a person which most individuals … do not want widely known about themselves” australasian journal of information systems eismann, fischer-preßler & fischbach 2022, vol 26, research on applied ethics applied ethics and digital information privacy 4 (parent, 1983, pp. 269-270). in a normative sense, information privacy is one of liberal societies’ key moral values, whereas in a descriptive sense, it refers to a quality of a situation that can be present or absent (becker, 2019). thus, information privacy comprises “the claims of individuals that data about themselves should generally not be available to other individuals and organisations, and that, where data is possessed by another party, the individual must be able to exercise a substantial degree of control over that data and its use” (clarke, 1999, p. 60). digitally enhanced means of information sharing and communication are widely believed to threaten people’s information privacy, especially because they enable constant and unobtrusive monitoring and profiling (becker, 2019). they make it easy to record, analyse, and share personal information about users, retain it for an indefinite time, match it with data from secondary sources, and thus generate novel and possibly revealing insights about users (nissenbaum, 1998). in addition, they may incentivise users to contribute additional information about themselves to facilitate interactions (volokh, 2000)—a condition commonly known as the “commodification of privacy” (campbell & carlson, 2002). information privacy is only one among several ethical issues that require consideration with covid-19 contact tracing apps (floridi, 2020). in fact, blasimme and vayena (2020, p. 761) conclude that most of these apps “are built with a proactive commitment to privacypreserving technological features (privacy by design) and only use strictly necessary data (privacy by default).” existing covid-19 contact tracing apps have typically been developed with an explicit commitment to pertinent privacy laws and regulations; many, if not most, have been subjected to intensive public debate and scrutiny to enhance their compliance with commonly shared social norms (e.g., european parliament, 2020). still, the risk of misuse and vulnerability of personal information collected on a large scale (e.g., due to hacking or cyberattacks) remains (klar & lanzerath, 2020). furthermore, absent evidence on the efficacy of the apps, users’ information privacy may be threatened by a lack of proportionality, as well as the risk of function creep if the data collected through the apps is repurposed, potential ignorance of sunset clauses and continued use of the apps even after the covid-19 pandemic is declared contained, and the risk of unrevealed non-voluntariness if social or economic conditions render the use of the apps obligatory in some de facto way (ishmaev, dennis, & van den hoven, 2021). furthermore, information privacy risks arise in cases where malevolent users can infer from the app who may be infected, state-level agents or service providers can access personal user information illegitimately, or if such information is leaked due to negligence or malicious attacks (boutet et al., 2020). rowe (2020) also observes that governments tend to overemphasise the potential short-term benefits of using covid-19 contact tracing apps, while downplaying long-run threats to information privacy. covid-19 contact tracing apps can threaten users’ information privacy in two fundamental ways: they can interfere with users’ ability to control access to their personal information (privacy as control); and they can facilitate the access by others to personal information about a user in a given situation (as a state). while the former defines information privacy in terms of the ability of agents to control access to information about themselves, the latter understands it as a condition in which access to personal information is limited to some extent (smith et al., 2011). both conceptions of information privacy have their roots in historical privacy theories, and both come with a number of theoretical virtues and drawbacks (tavani, 2007, 2008). however, understanding information privacy merely in terms of control is problematic when australasian journal of information systems eismann, fischer-preßler & fischbach 2022, vol 26, research on applied ethics applied ethics and digital information privacy 5 agents deliberately relinquish control over information about themselves (tavani & moor, 2001). in the restricted access/limited control (ralc) framework of information privacy, by contrast, moor (1997) and tavani and moor (2001) define information privacy primarily in terms of limited access to personal information. they recognize that while perfect control over personal information may be impossible when it comes to digital information and communications technologies, people should be able to rule out undesired access to and use of information about themselves. the ralc framework integrates two elements: the condition of privacy—the extent to which information privacy is present in a particular situation, and which relies on the absence of access to personal information; and the management of privacy—people’s ability to determine who can access information about them for what purpose and in what ways, which requires that information subjects have control over their personal information. through actively managing access to personal information, agents are hence assumed to determine the amount of information privacy in a given situation. according to the ralc framework, information privacy is a characteristic of situations in which people are normatively protected from intrusion, interference, and information access by others. situations are the key unit of the ralc framework; they can comprise locations, relationships, and activities. situations can be naturally private, meaning there are natural or physical circumstances that protect people from intrusion or observation. in naturally private situations, privacy exists as a matter of fact. when others gain access to information about a person in such situations, that person’s privacy is lost but not violated, as they do not have a normative claim to it. in contrast, in normatively private situations, personal information is protected through norms that determine who may access it. these norms constitute a person’s right to privacy in a particular situation; privacy is violated only when pertinent norms are trespassed (moor, 1997; tavani & moor, 2001). ralc introduces three mechanisms of individual control through which actors can achieve an adequate level of information privacy, although they do not constitute a right to privacy beyond what is granted by situational norms: choice over situations that offer the desired level of information privacy; consent to give up the right to information privacy in a particular situation; and correction of personal information collected (tavani & moor, 2001). just as in the case of offline social relations, interactions enabled by digital information and communications technologies can constitute situations (tavani, 2007). thus, from a ralc perspective, using a contact tracing app can constitute both a naturally and a normatively private situation, depending on the context of app use and the specific design of the app. while pertinent social and legal norms and a shared understanding that users must somehow be able to determine to whom their personal information is made available are the basis for normative protection of their information privacy, it is the app’s technical features that determine the extent to which users are actually protected from intrusion, and hence render app usage naturally private. figure 1 illustrates how the use of covid-19 contact tracing apps can be interpreted within the ralc framework. australasian journal of information systems eismann, fischer-preßler & fischbach 2022, vol 26, research on applied ethics applied ethics and digital information privacy 6 figure 1. a ralc perspective on information privacy in the context of covid-19 contact tracing app use. in our work, it is instances of app use that constitute the situations to be analysed. those are characterised by two key attributes: personal information that is exposed to potential information privacy violations, and the information privacy violations that potentially apply. personal information is embedded within two layers of protection. the first layer is normative protections of information privacy, which consists of norms—broadly defined as principles of action that specify appropriate behaviour in a given society—that determine what information about users may be legitimately accessed through the app, by whom, and under what conditions. access to personal information contravening these norms is considered to violate users’ information privacy. the second layer is technical protections of information privacy, which comprise the particular technical features through which personal information is protected from unwarranted access. these constitute natural barriers in the ralc terminology: they do not grant users a normative right to information privacy, but nevertheless can be effective in preventing access to their personal information. below, we explain how we have analysed these key elements of existing covid-19 contact tracing apps, following a disclosive research approach, and elaborate on what our insights reveal about the understanding of information privacy that is evident from the apps’ design. 3 research methodology 3.1 disclosive computer ethics in our research, we have followed the tenets of disclosive computer ethics, assuming that technological artifacts are not morally neutral, but may have hidden moral properties that can be uncovered through disclosive analysis (brey, 2000a, 2000b). basically, technological artifacts are assumed to serve as mechanisms that constrain people’s individual and social behaviors, and hence can enforce or promote normative action (brey, 2000b). studies in disclosive computer ethics require a two-step approach. relying on an inductive approach, in the first step, a technological artifact is analysed from the perspective of some australasian journal of information systems eismann, fischer-preßler & fischbach 2022, vol 26, research on applied ethics applied ethics and digital information privacy 7 moral value to draw preliminary conclusions regarding the relationship of the artifact’s features to the moral value of interest. then, in the second step, moral theory can be applied to evaluate more specifically the artifact’s normatively relevant features and properties, which also creates the potential for further developing existing theory (brey, 2000a, 2000b). below, we explain how we have analysed the normative and technical design principles that determine how the technological artifacts in our work—mobile apps for covid-19 contact tracing—protect information privacy, based on the ralc framework delineated above. 3.2 case selection and data acquisition our analyses are based on covid-19 contact tracing apps from five liberal democracies (according to freedom house; repucci & o'toole, 2020) that released the apps for their residents to use as early as the second half of 2020: australia, france, germany, japan, and new zealand. we selected these countries because they provide early examples of privacysensitive design for covid-19 contact tracing apps (as opposed to digital tools applied for other purposes, such as outbreak detection and case identification; budd et al., 2020) against a somewhat comparable socio-political background. these countries’ governments have all stated that the apps comply with national privacy laws and regulations, and were designed especially to meet pertinent norms of information privacy. in the european union, for instance, the robert (robust and privacy-preserving proximity tracing) protocol was established to facilitate developing privacy-preserving contact tracing apps in line with the pertinent general data protection regulation (gdpr), and governments have established a priori guidelines to guide that work (e.g., european parliament, 2020). the approaches taken to protect information privacy, however, vary. this purposive sample of covid-19 contact tracing apps thus allows us to draw conclusions about the implementation of information privacy concerns in liberal democracies based on cases with maximum possible heterogeneity at the time. table 1 is an overview of the apps analysed. the five apps we studied are applications for contact tracing that analyse physical contacts between people, using bluetooth, for instance, to measure the distance between smartphones on the basis of signal strength and thus detect potential encounters between users (i.e., proximity tracing). unlike other mobile apps that track people’s geo-locations (i.e., tracking apps), proximity tracing apps cannot be used to create detailed movement profiles to monitor people’s behaviors, for instance, to ensure compliance with quarantine orders. while both involve sensitive data, privacy concerns for the two types of applications differ, and proximity tracing apps are considered, by design, to be more privacy-sensitive (taddeo, 2020) and also more compatible with privacy expectations in liberal democracies than tracking solutions that have been used in south korea, taiwan, and israel, among other countries (zastrow, 2021). we downloaded the official documentation provided by the institutions in each country responsible for the development, infrastructure, and service provision of the apps (date of reference: 2020-06-30). we analysed english versions of the documents, using machine translations (google translate) when english texts were not available. australasian journal of information systems eismann, fischer-preßler & fischbach 2022, vol 26, research on applied ethics applied ethics and digital information privacy 8 country (app name) description responsible institutions information sources used australia (covidsafe) • to use the app, users must register and provide personally identifying information that is used to contact them in case they are at risk of infection; the information is stored on a central server. • within the app, each user is represented by an automatically generated id that changes at regular intervals to prevent users from inferring their own or others’ ids. • the app uses bluetooth to detect other users nearby. in case of an encounter, it saves these users’ ids, the date and time of contact, phone model, and bluetooth signal strength. • infected users are notified by human contact tracers and asked to upload their encounter information to a central server. if they consent to do so, they receive a pin to authorise the transmission. • contact tracers then use the encounter information and personal id information of other users to identify and prioritise persons who may have been in contact and advise them of their risk status. australian government department of health australian government, 2020; australian government department of health, 2020 france (stopcovid) • once installed, the app creates a unique identifier that represents the user. • the app uses bluetooth to detect other users nearby. in case of an encounter, it saves these users’ ids, the date and time of contact, and the inferred distance between users based on bluetooth signal strength. • infected users receive a qr code or similar proof from public health services, which they can use to update their health status within the app. • if they update their status, the infected user’s encounter history is uploaded to a central server and warnings are sent to the devices of other users for whom an encounter was registered. ministry of the economy, finance and recovery inria, 2020a, 2020b, 2020c; ministére de l'économie des finances et de la relance, 2020a, 2020b germany (coronawarn) • each user is represented within the app by an automatically generated id, which changes at regular intervals to prevent users from inferring their own or others’ ids. • the app uses bluetooth to detect other users nearby. in case of an encounter, it saves these users’ ids, the date and time of contact, and the inferred distance between users based on bluetooth signal strength. • infected users receive a qr code or similar proof from public health services, which they can use to update their health status within the app. • if they update their status, the infected user’s id is uploaded to a central server, which in turn makes it available to others’ devices, which compare it with those users’ own encounter histories and generate warnings if they have registered an encounter with the infected user. german federal government corona warn app open source project, 2020; the federal government, 2020a, 2020b australasian journal of information systems eismann, fischer-preßler & fischbach 2022, vol 26, research on applied ethics applied ethics and digital information privacy 9 country (app name) description responsible institutions information sources used japan (cocoa – covid-19 contact) • each user is represented within the app by an automatically generated id, which changes in regular intervals to prevent users from inferring their own or others’ ids. • the app uses bluetooth to detect other users nearby. in case of an encounter, it saves these users’ ids, the date and time of contact, and the inferred distance between users based on bluetooth signal strength. • infected users can register with their email address or phone number to receive a processing number from public health services, which they can use to update their health status within the app. • if they update their status, an infected user’s id is uploaded to a central server, which in turn makes it available to others’ devices, which compare it with those users’ own encounter histories and generate warnings if they have registered an encounter with the infected user. ministry of health, labour and welfare ministry of health, labour and welfare, 2020a, 2020b new zealand (nz covid tracer) • users must sign up with their email addresses to use the app. they can add further personally identifying information to their profiles. • the app allows users to keep a digital diary of locations visited, which they can enter by scanning qr codes or manually adding entries of places, activities, dates, and times visited. • users who are confirmed or deemed likely to be infected are notified by contact tracers to upload their location history to a central server. • if they update their status, human contact tracers evaluate the location history and identify locations at which persons in contact with an infected user would have been at risk of infection. these locations are made available to users’ devices, which in turn generate warnings if they detect a match between their own location entries and a relevant exposure location and timeframe. ministry of health ministry of health, 2020a, 2020b table 1. overview of covid-19 contact tracing apps 3.3 exploratory analysis following from the ralc framework (moor, 1997; tavani & moor, 2001), we understand information privacy in terms of limited access to information about a person, with the subjects of the information being able to determine by whom, in what ways, and to what end that information is accessed and used. this definition is thus in line with prior research in information systems (smith et al., 2011), but also broad enough to encompass different understandings of what information is subject to privacy concerns, what constitutes unwarranted access to and use of personal information, and how apps can be designed to protect user information privacy. we used a multistage qualitative coding procedure to analyse the apps. in the first stage, we identified statements that referred to personal information about users from each app’s documentation. in the second stage, we used these statements to develop inductive codes that australasian journal of information systems eismann, fischer-preßler & fischbach 2022, vol 26, research on applied ethics applied ethics and digital information privacy 10 describe what information about app users was specified as collected or not collected, what kind of access to or use of that information might violate user information privacy, and what normative and technical design principles were applied to afford information privacy protection. in the third stage, we summarised these descriptive codes into coherent categories, which we describe in the following section. two of the paper’s authors undertook qualitative coding, with one in charge of each coding stage and the other reviewing and revising the codes. disagreements were resolved through consensual discussion. below, we summarise our findings on the types of personal information collected by these covid-19 contact tracing apps, the violations of user information privacy accounted for, and the different approaches to preventing violations spelled out in the documentation. 4 results of the exploratory analysis of covid-19 contact tracing apps 4.1 personal information collected the covid-19 contact tracing apps we studied collect different types of personal information: person identifying information that allows for identifying a particular person who may be using the app (i.e., users and other persons about whom information is collected through the app); user identifying information that allows for identifying a particular user or device on which the app is installed; encounter information about contacts between users through which covid-19 might be transmitted; user profile information about user preferences and activities using the app; and health status information about a user’s actual or potential covid-19 infection. table 2 provides an overview of the types of personal information collected by the analysed apps. only the australian app requires users to enter a name, age range, postal code, and contact phone number to facilitate tracing. the new zealand app allows users to enter personal details and information about their social contacts to identify other people who might be at risk; users must also provide an email address to complete the registration. the french and japanese apps require an email address or mobile phone number so infected users can receive a verification code to authorise health status updates that indicate they have become infected. the german app does not collect person identifying information. both the australian and the new zealand apps collect information that can be used to trace individual users. the australian app registers the model of the smartphone on which the app is installed, and the new zealand app asks users to enter information about their locations and activities to detect potential encounters. in contrast, the french, german, and japanese apps stress that no information is collected that would enable identifying the user or the device on which the app is installed. none of the analysed apps collects geolocation information. in addition to person and user identifying information, all the apps except the new zealand app create unique identifiers for each user. these are linked to the records of users’ previous encounters and are exchanged between nearby devices to constitute encounters; they use bluetooth to record the dates, times, and estimated duration of encounters and proximity of contact based on bluetooth signal strength. the new zealand app does not record encounters directly; rather, it allows users to keep a digital diary of places they have visited and infers from that possible encounters with other users who have checked in at the same location at the same time. australasian journal of information systems eismann, fischer-preßler & fischbach 2022, vol 26, research on applied ethics applied ethics and digital information privacy 11 countries aus fra deu jpn nzl person identifying information name x age range x post code x phone number x x x email address x x x social contacts x user identifying information device model x location x activities x encounter information date and time x x x x x spatial proximity x x x x duration of contact x x x x user profile information user identifier x x x x encounter history x x x x user interests x health status information positive test result x x x x x risk status (binary) x x x risk status (qualitative) x x health record x table 2. personal information collected by covid-19 contact tracing apps finally, all the apps allow users to update their health status if they have tested positively for covid-19. this information is then combined with encounter information to evaluate users’ risk status and issue warnings accordingly. estimated risk status can be binary, as in the french, german, and japanese apps—that is, the app differentiates only between users who are at risk of infection (because they have been close to an infected user for an epidemiologically relevant period) and those who are not. the japanese app allows users who have received a warning to enter further information about their health status and possible symptoms so they can be provided with more tailored advice. the australian and new zealand apps allow for more detailed qualitative risk assessments. risk status with the australian app is evaluated by human contact tracers, who get in touch with users who might be at risk of infection based on their encounter information and other information, such as the number of encounters, duration of contact, and estimated distance between users during an encounter. for the new zealand app, contact tracers evaluate the locations at which an infection was likely to occur, and issue warnings selectively to persons potentially at risk. 4.2 accounting for potential information privacy violations the covid-19 contact tracing apps we studied account for three broad types of information privacy violations: illegitimate acquisition of personal information without a justified cause or in an unjustified way; misrepresentation of personal information respectively users based on information that is collected by the app but does not accurately describe the user or their condition; and misuse of personal information by processing or utilising it in an unjustified way or for an unjustified purpose. table 3 is an overview of information privacy breaches for which australasian journal of information systems eismann, fischer-preßler & fischbach 2022, vol 26, research on applied ethics applied ethics and digital information privacy 12 we found precautions spelled out in the official documentation of the analysed covid-19 contact tracing apps. countries aus fra deu jpn nzl illegitimate acquisition of personal information forced use of the app x x x x x forced self-disclosure x x x x x uneconomic data collection x x x prolonged data collection x x misrepresentation of personal information unauthorised registration and use of the app x x misreporting personal information x x x x x misuse of personal information illegitimate access x x x x x aggregation x recombination x x x repurposing x x x x table 3. information privacy violations accounted for by existing covid-19 contact tracing apps the institutions responsible for all five apps agree that installation and use of the app must be voluntary. furthermore, they state that users must not be forced to provide any personal information. in particular, they must not be obligated to provide information about their health status, disclose test results within the app, or share any personal data with health authorities or other agents. in the documentation for the french, german, and japanese apps, collecting data beyond what is absolutely necessary and relevant to enable contract tracing is considered a violation of users’ information privacy. for the australian and french apps, prolonged data collection beyond the duration of the coronavirus pandemic is characterised as a potential breach of information privacy. the documentation also accounts for the fact that misrepresenting users based on the personal information collected could potentially threaten the integrity of the apps, and so information collected must be accurate, complete, and up to date. for the french, german, and japanese apps, this refers in particular to encounter information and information about a user’s health status and test results—users are encouraged to submit timely information about positive test results. false reports regarding positive tests for covid-19 are prevented by enabling the update of the infection status only through a code provided by the responsible health institution. the documentation for the australian and new zealand apps states that misrepresenting person or user identifying information that is used to identify potential contacts is not approved. the accompanying documentation for all five apps addresses illegitimate access to personal information as a violation of information privacy: other app users, service providers, health authorities, and third parties are not allowed unauthorised access to personal information, nor may this information be made available or transmitted to them. in addition, the french app’s documentation explicitly forbids data aggregation, which means personal information must not be accumulated in a way that allows clustering of users based on that information (e.g., compiling lists of infected users). the french, german, and japanese apps further prohibit the recombination of personal information—that is, associating or matching personal information australasian journal of information systems eismann, fischer-preßler & fischbach 2022, vol 26, research on applied ethics applied ethics and digital information privacy 13 to make inferences about other users’ personal attributes (e.g., their health status or social contacts). finally, the documentation for all but the french app discourage the use of personal information provided through the app for purposes other than covid-19 contact tracing (e.g., preventing access to restaurants and other facilities, or monitoring user compliance with quarantine orders), although data are generally used to provide use statistics and improve app functionality. while we expect that the same holds true for the french app, we were not able to identify corresponding statements in the documentation. 4.3 design principles to protect users’ information privacy the analysed apps are based on several design principles aimed at protecting personal information from illegitimate access and use. these include voluntariness, that is, no one is compelled to install or use the app or add personal data, any information entered can be deleted, and the app can be uninstalled at any time. the apps are based on transparency that makes the manner in which the app collects and uses personal information known to users. data minimisation ensures that only data that are absolutely necessary and relevant for contact tracing are collected. the app design can include pseudonymisation—only user-chosen or automatically generated pseudonyms are used to refer to users. the protection of personal information from unwarranted access by users and others is part of the apps’ design and, finally, can enable users to verify, review, and correct their personal information. table 4 is an overview of the design principles through which the analysed covid-19 contact tracing apps seek to prevent violations of information privacy. to implement the principle of voluntariness, the adoption and use of all analysed apps is left to people’s discretion: neither governments nor other agents may force or otherwise exert pressure on people to install the apps. further, providing personal information within the apps—particularly about a user’s health status and a potential covid-19 infection—is optional. users may uninstall the app at any time, and doing so will delete all personal information stored on their devices; if information pertaining to them is stored on some central server, they can request its deletion. all the responsible institutions emphasise transparent design. privacy information is made available to users both within the app and online. applicable privacy laws and regulations are explicitly referenced, and contacts for privacy-related inquiries are provided. furthermore, governments have approved the design of the apps, either implicitly or explicitly; for instance, french and german government and civil agencies responsible for information privacy and data protection were involved in their country’s app development to ensure compliance with privacy standards. furthermore, the french and german apps are open source, meaning the source code and technical specifications of the apps are publicly available, and community members and external experts are encouraged to review and comment on the code. the french, german, and japanese app documentation strongly emphasises minimisation with respect to data collection. these apps do not require user identifying information to be entered to use the app, and only minimal person identifying information is required for the setup. furthermore, they impose quantitative thresholds on data collection, which implies that encounters are recorded and stored only when some threshold values in proximity and duration are exceeded (less than 1 meter for 15 minutes or more for the french and japanese apps, and more than 10 minutes for the german app). both the australian and the new zealand app documentations, in contrast, emphasise data richness to allow for graded risk australasian journal of information systems eismann, fischer-preßler & fischbach 2022, vol 26, research on applied ethics applied ethics and digital information privacy 14 assessment. while they do not track user geolocations, both collect data that could be used to identify particular users. countries aus fra deu jpn nzl voluntariness voluntary use of the app x x x x x voluntary self-disclosure x x x x x discontinuance of use x x x x x transparency transparent privacy standards x x x x x government approval x x x x x agency involvement x x open source development x x data minimisation qualitative data collection thresholds x x x quantitative data collection thresholds x x x temporally restricted storage x x x discontinuance of service provision x x data protection encrypted storage x x x x x decentralised storage x x x centralised storage x x restricted device-to-device transmission x x x x x secure transmission x x pseudonymisation personal pseudonyms x random user identifiers x x x x dynamical user identifiers x x x logical disconnectedness x x x data quality enhancement technical redundancies x x x x verification x x x x x revision x x table 4 principles to protect users’ information privacy in covid-19 contact tracing apps the australian, german, and japanese apps store encounter information only for a limited period (14 days for germany and japan; 21 days for australia). while we cannot rule out the possibility that the other apps impose similar restrictions, we were not able to confirm that based on official documentation. furthermore, the australian and french apps, according to official announcements, are to be discontinued at the end of the covid-19 pandemic, including automatic deletion of all personal information stored. we were not able to identify similar statements regarding the other apps. pseudonymisation is another measure taken to protect users’ information privacy. the australian app allows users to enter false names or self-chosen pseudonyms upon registration. other apps employ user ids that are generated automatically within the app and are linked to user digital profiles, which allows for retracing a user’s encounters. with the exception of the new zealand app, all apps analysed use these randomly generated identifiers. the accompanying documentation for the australian, german, and japanese apps also state that australasian journal of information systems eismann, fischer-preßler & fischbach 2022, vol 26, research on applied ethics applied ethics and digital information privacy 15 these identifiers change at regular intervals to prevent users from identifying nearby other users or retracing infected users. documentation of the apps also stresses that user pseudonyms are logically disconnected from other pieces of personal information, which means they are inherently unrelated. for instance, in the french, german, and japanese apps, user identifiers are generated randomly and are not based on any person identifying information. similarly, the codes that users enter to verify test results are independent of these identifiers, so servers or users’ devices can only know that some user has tested positively for covid-19, but cannot infer that user’s identity. while the australian app also uses randomly generated identifiers for exchanges between users, those are connected to person identifying information made available to contact tracers, which means they can infer the identity of infected or potentially infected users. similarly, for the new zealand app, contact tracers know the identities of infected users, although they cannot directly identify users at risk. none of the apps provides real-time alerts, which means that the apps cannot be used to detect infected persons in a user’s proximity. for all apps, encrypted storage is key to protecting personal information from unwarranted access. apps differ, however, in their data storage models: the german and japanese apps use decentralised storage of personal information, which means that as little information as possible is transmitted from the individual users’ devices to central servers. instead, the identifiers of infected users are uploaded so other devices can compare them with their own encounter histories. similarly, users of the new zealand app who have tested positive for covid-19 can upload their location histories so that warnings can be generated for particular locations and other users’ devices can compare their own entries. the french app, in contrast, asks infected users to upload their encounter histories; each device knows only its own encounters, and matching is performed on a central server. the australian app relies even more on a central server infrastructure: upon registration, users provide person identifying information, which is stored centrally. in case of an infection, users are asked to upload their encounter histories, and both sources of information are matched by human contact tracers to identify and prioritise contacts. secure transmission between devices and servers can be an issue, although only the german and japanese app documentation refers explicitly to secure device-to-server transmission, and only the latter mentions prospective security measures to obscure transmissions (e.g., sending false notifications that are discarded on the server to create background noise). both storage models are generally motivated in terms of information privacy concerns. decentralised storage, on the one hand, minimises the threat that health agencies or service providers can access personal information stored on a central server, or that sensitive data could be released unintentionally if servers are compromised. centralised storage, on the other hand, can hinder individual users from accessing user identifying information (e.g., their identifiers), reduces information privacy risks from smartphone security holes and loss or theft of devices, and allows for bundling the responsibility for data protection with one or a few dedicated agents. decentralised storage does not imply that no personal information is transmitted to a central server; at the very least, all apps that rely on this approach use central servers to verify test results and make the identifiers of users diagnosed with covid-19 available to others’ devices. similarly, centralised storage does not mean that no information is stored on users’ devices, but that information about which users are infected is not made available to others. australasian journal of information systems eismann, fischer-preßler & fischbach 2022, vol 26, research on applied ethics applied ethics and digital information privacy 16 furthermore, all the apps, to a greater or lesser degree, restrict the information that is exchanged between devices. the new zealand app allows no information to be exchanged between devices, but encounters are inferred ex post based on shared attributes (i.e., locations and activities recorded by each user). device-to-device transmissions are limited in the other four apps analysed: the only information exchanged are the user identifiers required to represent encounters (possibly contingent on further thresholds of data collection). finally, some measures exist within all the apps to ensure the accuracy and completeness of personal information. to enhance data integrity, the australian, japanese, and new zealand apps require some type of initial verification to set up the app. the french, german, and japanese apps require users to verify their test results when declaring themselves infected with covid-19. to do so, they receive a one-time code that can be entered in multiple ways to prevent misreporting and ensure that information is entered reliably. similarly, the new zealand app provides multiple ways to enter locations and activities and to add supplementary information to encourage completeness. it also lets users review their digital profiles and revise and complement their visits and other information. similarly, the australian app allows users to reinstall the app to correct personal information. as a rule, however, all the apps—except for the one deployed in new zealand—store encounter information in a secure section of the device that cannot be accessed by users; this prevents users from reidentifying other users, particularly infected persons. 5 theoretical evaluation of information privacy in the context of covid-19 contact tracing apps thus far, we have analysed design principles evident from existing covid-19 contact tracing apps, based on the broad understanding of information privacy as limited access to information about a person (condition of privacy) who might be able to control, to a certain extent, how that information is accessed and used (management of privacy). table 5 summarises the normative and technical design principles to protect users’ information privacy. normative design principles to protect information privacy technical design principles to protect information privacy design principles aimed at limiting access to personal information (condition of privacy) does not apply. data minimisation data protection (centralised vs. decentralised) pseudonymisation design principles aimed at granting users control over their personal information (management of privacy) voluntariness transparency data quality enhancement table 5 normative and technical design principles to protect users’ information privacy based on the findings of the exploratory analysis, we conclude that the understanding of limited access to personal information that is evident from the apps varies along two dimensions. the first is the centralised-decentralised dimension. the australian and french apps in particular pursue a centralised approach, meaning that they do not transmit information about infected users to others’ devices, but perform all matching and processing of personal information on central servers. in contrast, the decentralised approach of the german and australasian journal of information systems eismann, fischer-preßler & fischbach 2022, vol 26, research on applied ethics applied ethics and digital information privacy 17 japanese apps, and, somewhat differently, the new zealand app, entrusts users’ devices with these tasks, largely reducing central servers to a transmittal function. the second dimension, which we refer to as the data minimisation-richness dimension, is based on the data minimisation and pseudonymisation design principles evident from our analyses. along this dimension, the german, french, and japanese apps limit access to personal information by minimising the information collected and through effective use of logically disconnected user pseudonyms to eliminate illegitimate access to person or user identifying information. in contrast, the australian and the new zealand apps specifically ask users to contribute personal information to be used for enhanced risk assessment and contact tracing. with respect to the management of information privacy, the voluntariness and transparency principles relate to choice and consent; they enable users to control whether and what information about themselves they make available in the first place. voluntariness implies that users can choose whether to use the apps and, if they do so, whether to provide the requested information. for instance, users who test positive for covid-19 can decide whether to make that information available to others or can use the app without doing so and still receive warnings—although the effectiveness of contact tracing would clearly suffer. transparency implies that users can make informed decisions about using the app based on the information provided. in contrast, measures that allow for enhancement of data quality relate to the ex-post correction of personal information that has been collected. while this does not protect users from illegitimate access and use of their personal information, it does enable them to ensure that such information is accurate, complete, and up to data, and hence that adequate conclusions (in particular about their own and others’ risk status) can be drawn. this tradeoff between design principles for limiting and managing access to personal information indicates that the former may come at the expense of generating more precise and targeted warnings. the design principles for limiting access to users’ personal information we identified are, in principle, technical, as they can—depending on the effectiveness of their implementation— reduce exposure to unwarranted access to and prevent accumulation and recombination of personal information. such technical measures can only help constitute naturally private situations, where user personal information is protected from particular violations as a matter of fact, but implemented protections do not grant normative rights to information privacy. similarly, design principles related to correction through data quality enhancement are inherently technical, as they provide actual opportunities for users to revise and complement personal information. in contrast, design principles for managing information privacy through choice and consent are primarily normative. while the documentation for all the apps state that voluntariness and transparency are desirable, they do not describe measures taken to ensure they are obeyed. for instance, it is beyond the scope of app design to ensure that health agencies and other agents do not take measures to enforce app use. 6 discussion our insights into the normative and technical design principles are clearly related to the broader discourse in information systems and other disciplines regarding the adoption and use of mobile applications for contact tracing, and parallel the findings of prior research on information privacy in the context of covid-19 contact tracing apps. australasian journal of information systems eismann, fischer-preßler & fischbach 2022, vol 26, research on applied ethics applied ethics and digital information privacy 18 our findings confirm that the design of apps deployed early on by five liberal democracies considers key ethical principles identified in the literature. in particular, the documentation we analysed stress the role of voluntariness, which is deemed crucial for privacy-sensitive design. the voluntariness principle goes beyond the mere adoption of an app, but includes people’s decisions to carry a smartphone, download and install the app, leave the app operating in the background, react to alerts, share contact logs if they test positive, and uninstall the app together with removing collected data (klar & lanzerath, 2020). however, voluntariness only protects people’s information privacy in a normative sense. as ishmaev et al. (2021) point out, even if adoption and use of apps for covid-19 contact tracing is declared non-mandatory, social or economic conditions may still create pressure to do so. voluntariness is also contingent on user understanding of how and what information is accessed by the apps. in an early analysis, zhang, chow, and smith (2020) conclude that the privacy-related information provided by the australian and new zealand app documentation, for instance, may be too complex for users to read and understand. also, only two of the apps have open source code, meaning that in-depth inspection of the apps’ functionalities even by experts is not always possible, leaving the implementation of normative information-privacy protections to the responsible institutions’ goodwill and competences. what is more, proportionality is a key criterion for privacy-sensitive app design (morley et al., 2020). however, concerns were raised that contact tracing apps may not be effective in containing the spread of covid-19 because they can make a real difference only if a significant percentage of the population (e.g., 56% or more) use them (hinch et al., 2020). if that threshold is not met—as was the case in the five countries at the time of our study—a potential sacrifice of information privacy may be unjustified. overall, many privacy-related aspects—in particular, voluntariness, user consent, anonymity (in terms of pseudonymisation), purpose specification, provision of app providers’ contact information, and specification of data retention parameters and the decommissioning process (klar & lanzerath, 2020; morley et al., 2020)—have been systematically addressed by the apps analysed. however, not all have adopted corresponding design principles in equal measure, and other potential threats to user information privacy, such as special protection of data collected from children, updates to the apps’ privacy policy changes, and privacy breach notices (e.g., hatamian et al., 2021; kolasa, mazzi, leszczuk-czubkowska, zrubka, & péntek, 2021), are not explicitly evident from the apps’ accompanying documentation. while we expect that such issues have been addressed since our study was completed, it is revealing how even proactive privacy-sensitive app design may deviate at least from some of the previously suggested ethical standards. in addition, works such as that of hatamian et al. (2021) suggest that the extent to which the applied design principles are actually suited to limit access to personal information depends considerably on their actual implementation—which, according to them, still leaves room for improvement. in addition, further measures that could be taken, such as data protection through random noise signals (e.g., sharma & bashir, 2020), have not (yet) been explicitly addressed by the majority of apps we have analysed. our findings hint at an unsurprising but nevertheless critical tension between normative and technical design principles: if normative principles, such as voluntariness and transparency, are not supported by technical measures, the risk of information privacy violations could be australasian journal of information systems eismann, fischer-preßler & fischbach 2022, vol 26, research on applied ethics applied ethics and digital information privacy 19 real. if, for instance, information-privacy policy documentation is insufficient, or people are encouraged to use the app to avoid other constraints, these principles could be undermined. however, if technical protections such as secure storage and pseudonymisation lack normative justification, they can only be evaluated against the benchmark of the efficacy of implementation—which is not feasible for laypersons, and which may undergo unnoticed changes at any time. contrasting the design principles identified from the apps analysed, we find striking similarities between the information-privacy standards they set. this is likely due to the homogeneous sample of apps originating from liberal democracies. it is hence not unexpected that these apps afford similar levels of information privacy. still, from a theoretical point of view, it is interesting to see what the apps’ design reveals about the understanding of information privacy. one aspect in this regard is the distinction between centralised and decentralised storage. both approaches can, in principle, be privacy preserving (who, 2020); still, they are based on different assumptions regarding the origins of potential information privacy threats. while centralised design stresses the risk that individual users might recombine distinct pieces of information about others and thus re-identify other users and draw conclusions about their health status, decentralised design emphasises threats due to the existence of central stores of aggregated information that, if compromised, could expose a large number of users. another aspect relevant for our theoretical understanding of information privacy is the tension between data minimisation and pseudonymisation, on the one hand, and the data richness demand posed by several apps, on the other. the former stresses the condition of privacy, where access to personal information is largely restricted. it is in line with the postulate to favor privacy by default to ensure exposure to potential information-privacy violations is effectively minimised, and it is clearly in line with much prior research that interprets information privacy in terms of autonomy, or the absence of intrusion into users’ private sphere (becker, 2019). while all the apps analysed generally agree that certain types of personal information, such as gps data, is not necessary to enable contact tracing and hence should not be collected (who, 2021a), the data richness postulate indicates a more differentiated view regarding what constitutes illegitimate access to and use of user personal information, such that particular types of personal information (e.g., health status information) demand higher levels of protection than others that may be shared more carelessly. it hence shifts the focus of information privacy from autonomy to integrity, ensuring appropriate representation of users through their personal information based on a notion of information privacy as effective control over the flow of such information in a given context (as postulated by, for instance, floridi, 2006). 7 conclusion in summary, we have analysed the normative and technical design principles through which covid-19 contact tracing apps deployed in five liberal democracies aspire to protect information privacy: voluntariness, transparency, data minimisation, pseudonymisation, protection of personal information (through centralised and decentralised designs), and data quality enhancement. australasian journal of information systems eismann, fischer-preßler & fischbach 2022, vol 26, research on applied ethics applied ethics and digital information privacy 20 the contributions of disclosive computer ethics studies can be found at three levels: the disclosure level, which explicates the hidden moral properties of the technological artifacts under study; the theoretical level, which refines and further develops existing theory; and the application level, which refers the theoretical insights back to the disclosure level (brey, 2000a, 2000b). at the disclosure level, in response to rq1, we have spelled out the normative and technical design principles through which the apps analysed protect information privacy. at the theoretical level, in response to rq2, we have explained what these principles reveal about the understanding of information privacy that has, explicitly or implicitly, guided the design of these apps, and how this relates to the condition and management of information privacy according to the ralc framework. furthermore, we have discussed how the principles identified complement or contradict each other, and how these evident tensions can enrich our theoretical understanding of information privacy in the given context. finally, at the application level, we expect that our insights can inform the design of contact tracing apps and help evaluate the measures taken to protect user information privacy both in newly released apps and in updated versions of the apps analysed. the contribution of our work lies in the space between normative papers that develop design guidelines from an ethical perspective (e.g., morley et al., 2020) and technical papers that analyse how those guidelines are implemented (e.g., hatamian et al., 2021). our work is intended to inform app design by providing real-world examples of privacy-sensitive design that are motivated by well-grounded ethical theory and inspired by practical examples. thus, we approach the refinement of ethical theory to develop and evaluate design principles for actually usable technological artifacts, as demanded, amongst others, by bélanger and crossler (2011). our argument is based on the ralc framework of information privacy (moor, 1997; tavani & moor, 2001), which conceptualises information privacy in terms similar to much information systems research: as a state in which access to information about a person is limited, or in which people are able to control information about themselves (popovič, smith, thong, & wattal, 2017; smith et al., 2011). it captures the potential of technological artifacts to threaten information privacy while also providing opportunities for users manage it actively, and is said to be sufficiently comprehensive to apply to a wide range of related concerns (tavani, 2007). as solove (2006) explains, privacy theories will typically have problems to apply to information privacy in digital environments, as the corresponding information and communications technologies have capacities to collect, store, process, and disseminate information about their users that go well beyond those of the means of communication addressed in the original debates. in addition, few existing theories recognise privacy breaches that result from practices such as the constant and unobtrusive surveillance of users, data mining, and user profiling, which are nonetheless key information privacy concerns put forth in the given context (clarke, 2019; stahl, 2019; volokh, 2000). still, our theoretical perspective implies that the interpretations put forth in this paper are meaningful only in the context of the ralc framework, which, as we have discussed, may have issues accounting for the tradeoff between users’ autonomy and the integrity of representation. another limitation comes from the small sample of apps we analysed. our results are based on apps deployed as early as june 2020 in liberal democracies, which means that our contribution is restricted to outlining example design principles rather than providing a australasian journal of information systems eismann, fischer-preßler & fischbach 2022, vol 26, research on applied ethics applied ethics and digital information privacy 21 representative summary. further, different countries will typically continue to pursue different approaches to the development of apps, owing to social and cultural preferences and path dependencies. to raise awareness of alternative approaches and justifications, we have articulated what we think are the evident design choices. our results are by no means final, as apps are typically developed further while in use, and new contact tracing apps have become available since we conducted our study. what is more, while official documentation is an informative source of information, complementary analyses would be required to determine, for instance, the effectiveness of how the principles we identified were implemented. our work is also relevant for the design of digital information and communications technologies in other social and economic settings with the potential for large-scale data collection and public surveillance, which are on the rise. for instance, fitness apps enable the large-scale collection of fitness and health data: where, when, and for how long users are active and with whom they compare themselves. widely used dating apps enable matches between users based on their geographical locations and other criteria. restaurants and hotels use apps and similar tools to manage reservations digitally, saving customer data that is often poorly protected in cloud services with little regard for or even awareness of information privacy issues (chaos computer club, 2020). for those, knowledge of existing conceptualisations of information privacy and corresponding design principles could be particularly useful. information privacy is of growing interest in information systems research, as the digitisation of different sorts of information increases, along with the use of technology and methods for capturing information, such as online platforms and big data analysis. we responded to calls for more context-specific research in this area: that digital artifacts and privacy-related practices need to be analysed within the context they are used; and that theories from other fields need to be applied meaningfully given that information security is an interdisciplinary research field (e.g., bélanger & crossler, 2011; lowry, dinev, & willison, 2017). we focused on disclosing and integrating moral properties evident from existing technological artifacts, and so the outcome of our work cannot be characterised as full-fledged ethical theory. rather, we have illustrated how ethical theory can inform the design of technological artifacts in general, and how empirical design choices can contribute to advancing such theory vice versa. in ethical terms, privacy theory must include not only a descriptive account of but also a normative justification of privacy (parent, 1983). in this paper, we have clearly focused on the former. a normative justification that clarifies not only whether information privacy is present in a certain artifact, but also the extent to which there is a justifiable claim to information privacy in a certain use context, would be valuable, especially for covid-19 tracing apps. with respect to covid-19, there are alternative approaches (e.g., tracking apps) that are generally regarded as less privacy friendly than the tracing apps we have analysed, and by far not all countries deploy those apps on a voluntary basis for people to use at their own discretion. our work should hence be seen as an initial step towards challenging and refining existing ethical theory. references ahmed, n., michelin, r. a., xue, w., ruj, s., malaney, r., kanhere, s. s., jha, s. k. (2020). a survey of covid-19 contact tracing apps. ieee access, 8, 134577–134601. https://doi.org/10.1109/access.2020.3010226 australasian journal of information systems eismann, fischer-preßler & fischbach 2022, vol 26, research on applied ethics applied ethics and digital information privacy 22 australian government (2020). privacy policy. retrieved from https://covidsafe.gov.au/privacy-policy.html*, accessed on 11th july 2020. australian government department of health (2020). covidsafe app. retrieved from https://www.health.gov.au/resources/apps-and-tools/covidsafe-app#get-th*, accessed on 11th july 2020. becker, m. (2019). privacy in the digital age: comparing and contrasting individual versus social approaches towards privacy. ethics and information technology, 21(4), 307–317. https://doi.org/10.1007/s10676-019-09508-z bélanger, f., & crossler, r. e. (2011). privacy in the digital age: a review of information privacy research in information systems. misquarterly, 35(4), 1017–1041. blasimme, a., & vayena, e. (2020). what's next for covid-19 apps? governance and oversight. science (new york, n.y.), 370(6518), 760–762. https://doi.org/10.1126/science.abd9006 boutet, a., bielova, n., castelluccia, c., cunche, m.: lauradoux, c., le métayer, d., & roca, v. (2020). proximity tracing approaches comparative impact analysis. retrieved from https://hal.inria.fr/hal-02570676v1*, accessed on 20th august 2020. brey, p. (2000a). disclosive computer ethics. acm sigcas computers and society, 30(4), 10– 16. https://doi.org/10.1145/572260.572264 brey, p. (2000b). method in computer ethics: towards a multi-level interdisciplinary approach. ethics and information technology, 2(2), 125–129. https://doi.org/10.1023/a:1010076000182 budd, j., miller, b. s., manning, e. m., lampos, v., zhuang, m., edelstein, m., . . . mckendry, r. a. (2020). digital technologies in the public-health response to covid-19. nature medicine, 26(8), 1183–1192. https://doi.org/10.1038/s41591-020-1011-4 campbell, j. e., & carlson, m. (2002). panopticon.com: online surveillance and the commodification of privacy. journal of broadcasting & electronic media, 46(4), 586–606. https://doi.org/10.1207/s15506878jobem4604_6 chan, e. y., & saqib, n. u. (2021). privacy concerns can explain unwillingness to download and use contact tracing apps when covid-19 concerns are high. computers in human behavior, 119, 106718. https://doi.org/10.1016/j.chb.2021.106718 chaos computer club (2020). ccc hackt digitale "corona-listen". retrieved from https://www.ccc.de/de/updates/2020/digitale-corona-listen/*, accessed on 29th august 2020. clarke, r. (1999). internet privacy concerns confirm the case for intervention. communications of the acm, 42(2), 60–67. https://doi.org/10.1145/293411.293475 clarke, r. (2019). risks inherent in the digital surveillance economy: a research agenda. journal of information technology, 34(1), 59–80. https://doi.org/10.1177/0268396218815559 corona warn app open source project (2020). frequently asked questions about the coronawarn-app. retrieved from https://www.coronawarn.app/en/faq/*, accessed on 29th june 2020. di gennaro, f., pizzol, d., marotta, c., antunes, m., racalbuto, v., veronese, n., & smith, l. (2020). coronavirus diseases (covid-19) current status and future perspectives: a narrative review. international journal of environmental research and public health, 17(8). https://doi.org/10.3390/ijerph17082690 european commission (2020). communication from the commission: guidance on apps supporting the fight against covid 19 pandemic in relation to data. retrieved from https://ec.europa.eu/info/sites/default/files/5_en_act_part1_v3.pdf*, accessed on. australasian journal of information systems eismann, fischer-preßler & fischbach 2022, vol 26, research on applied ethics applied ethics and digital information privacy 23 european parliament (2020). covid-19 tracing apps: ensuring privacy and use across borders, accessed on november 5, 2020. the federal government (2020a). the corona-warn-app: helps us fight the coronavirus. retrieved from https://www.bundesregierung.de/breg-de/themen/corona-warnapp/corona-warn-app-englisch*, accessed on 29th june 2020. the federal government (2020b). frequently asked questions. retrieved from https://www.bundesregierung.de/breg-de/themen/corona-warn-app/corona-warn-appenglisch/corona-warn-app-faq-1758636*, accessed on 29th june 2020. fischer, d., hattori-putzke, j., & fischbach, k. (2019). crisis warning apps: investigating the factors influencing usage and compliance with recommendations for action. in proceedings of the hawai'i international conference on system science. floridi, l. (2006). four challenges for a theory of informational privacy. ethics and information technology, 8(3), 109–119. https://doi.org/10.1007/s10676-006-9121-3 floridi, l. (2020). mind the app-considerations on the ethical risks of covid-19 apps. philosophy & technology, 1–6. https://doi.org/10.1007/s13347-020-00408-5 fodor, m., & brem, a. (2015). do privacy concerns matter for millennials? results from an empirical analysis of location-based services adoption in germany. computers in human behavior, 53(2), 344–353. https://doi.org/10.1016/j.chb.2015.06.048 hatamian, m., wairimu, s., momen, n., & fritsch, l. (2021). a privacy and security analysis of early-deployed covid-19 contact tracing android apps. empirical software engineering, 26(3), 36. https://doi.org/10.1007/s10664-020-09934-4 inria (2020a). faq on the technical aspects of the stopcovid application. retrieved from https://www.inria.fr/en/faq-technical-aspects-stopcovid-application*, accessed on 3rd july 2020. inria (2020b). stopcovid inria press release. retrieved from https://www.inria.fr/en/stopcovid*, accessed on 3rd august 2020. inria (2020c). the stopcovid project-team starts publishing the source code and documentation of the stopcovid application. retrieved from https://www.inria.fr/en/stopcovid-source-code*, accessed on 3rd august 2020. ishmaev, g., dennis, m., & van den hoven, m. j. (2021). ethics in the covid-19 pandemic: myths, false dilemmas, and moral overload. ethics and information technology, 1–16. https://doi.org/10.1007/s10676-020-09568-6 klar, r., & lanzerath, d. (2020). the ethics of covid-19 tracking apps – challenges and voluntariness. research ethics, 1-9. https://doi.org/10.1177/1747016120943622 kolasa, k., mazzi, f., leszczuk-czubkowska, e., zrubka, z., & péntek, m. (2021). state of the art in adoption of contact tracing apps and recommendations regarding privacy protection and public health: systematic review. jmir mhealth and uhealth, 9(6), e23250. https://doi.org/10.2196/23250 lowry, p. b., dinev, t. [tamara], & willison, r. (2017). why security and privacy research lies at the centre of the information systems (is) artefact: proposing a bold research agenda. european journal of information systems, 26(6), 546–563. https://doi.org/10.1057/s41303-017-0066-x ministére de l'économie des finances et de la relance (2020a). faq stopcovid. retrieved from https://www.economie.gouv.fr/stopcovid-faq*, accessed on 3rd july 2020. ministére de l'économie des finances et de la relance (2020b). https://www.economie.gouv.fr/stopcovid. retrieved from https:// www.economie.gouv.fr/stopcovid#*, accessed on 3rd july 2020. australasian journal of information systems eismann, fischer-preßler & fischbach 2022, vol 26, research on applied ethics applied ethics and digital information privacy 24 ministry of health (2020a). nz covid tracer app. retrieved from https://www.health.govt.nz/our-work/diseases-and-conditions/covid-19-novelcoronavirus/covid-19-novel-coronavirus-resources-and-tools/nz-covid-tracer-app*, accessed on 31st july 2020. ministry of health (2020b). questions and answers on nz covid tracer. retrieved from https://www.health.govt.nz/our-work/diseases-and-conditions/covid-19-novelcoronavirus/covid-19-novel-coronavirus-resources-and-tools/nz-covid-tracerapp/questions-and-answers-nz-covid-tracer*, accessed on 31st july 2020. ministry of health, labour and welfare (2020a). contact confirmation application privacy policy. retrieved from https://www.mhlw.go.jp/stf/seisakunitsuite/english_pp_00032.html*, accessed on 31st july 2020. ministry of health, labour and welfare (2020b). q & a for users of the contact tracing app. retrieved from https://www.mhlw.go.jp/stf/seisakunitsuite/bunya/kenkou_iryou/covid19_qa_kanrenkigyou _00009.html*, accessed on 31st july 2020. moor, j. h. (1997). towards a theory of privacy in the information age. acm sigcas computers and society, 27(3), 27–32. https://doi.org/10.1145/270858.270866 morley, j., cowls, j., taddeo, m., & floridi, l. (2020). ethical guidelines for covid-19 tracing apps. nature, 582(7810), 29–31. https://doi.org/10.1038/d41586-020-01578-0 nissenbaum, h. (1998). protecting privacy in an information age: the problem of privacy in public. law and philosophy, 17(5/6), 559. https://doi.org/10.2307/3505189 pan, a., liu, l., wang, c., guo, h., hao, x., wang, q., . . . wu, t. (2020). association of public health interventions with the epidemiology of the covid-19 outbreak in wuhan, china. jama, 323(19), 1915–1923. https://doi.org/10.1001/jama.2020.6130 parent, w. a. (1983). privacy, morality, and the law. philosophy & public affairs, 12(4), 269– 288. retrieved from http://www.jstor.org/stable/2265374 popovič, a., smith, h. j., thong, j. y. l., & wattal, s. (2017). mis quarterly research curation on information privacy. mis quarterly, updated: june 2018; august 2019. repucci, s., & o'toole, s. (2020). freedom in the world 2020: the annual survey of political rights & civil liberties. new york, ny, washington, dc and lanham, boulder, new york, london: freedom house and rowman & littlefield. rowe, f. (2020). contact tracing apps and values dilemmas: a privacy paradox in a neoliberal world. international journal of information management, 102178. https://doi.org/10.1016/j.ijinfomgt.2020.102178 sharma, t., & bashir, m. (2020). use of apps in the covid-19 response and the loss of privacy protection. nature medicine, 26(8), 1165–1167. https://doi.org/10.1038/s41591-020-0928-y smith, h. j., dinev, t. [t.], & xu, h. (2011). information privacy research: an interdisciplinary review. mis quarterly, 35(4), 989. https://doi.org/10.2307/41409970 solove, d. j. (2006). a taxonomy of privacy. university of pennsylvania law review, 154(3), 477. https://doi.org/10.2307/40041279 stahl, b. c. (2019). teaching ethical reflexivity in information systems: how to equip students to deal with moral and ethical issues of emerging information and communication technologies. ournal of information systems education, 22(3), 253–260. stahl, b. c. (2012). morality, ethics, and reflection: a categorization of normative is research. journal of the association for information systems, 13(8), 636–656. https://doi.org/10.17705/1jais.00304 australasian journal of information systems eismann, fischer-preßler & fischbach 2022, vol 26, research on applied ethics applied ethics and digital information privacy 25 taddeo, m. (2020). the ethical governance of the digital during and after the covid-19 pandemic. minds and machines, 1–6. https://doi.org/10.1007/s11023-020-09528-5 tavani, h. t. (2007). philosophical theories of privacy: implications for an adequate online privacy policy. metaphilosophy, 38(1), 1–22. https://doi.org/10.1111/j.14679973.2006.00474.x tavani, h. t. (2008). informational privacy: concepts, theories, and controversies. in k. e. himma & h. t. tavani (eds.), the handbook of information and computer ethics (pp. 131– 164). hoboken, n.j: wiley. https://doi.org/10.1002/9780470281819.ch6 tavani, h. t., & moor, j. h. (2001). privacy protection, control of information, and privacyenhancing technologies. acm sigcas computers and society, 31(1), 6–11. https://doi.org/10.1145/572277.572278 trang, s., trenz, m., weiger, w. h., tarafdar, m., & cheung, c. m.k. (2020). one app to trace them all? examining app specifications for mass acceptance of contact-tracing apps. european journal of information systems, 94(1), 1–14. https://doi.org/10.1080/0960085x.2020.1784046 volokh, e. (2000). personalization and privacy. communications of the acm, 43(8), 84–88. https://doi.org/10.1145/345124.345155 who (2020). ethical considerations to guide the use of digital proximity tracking technologies for covid-19 contact tracing. retrieved from https://www.who.int/publications/i/item/who-2019-ncov-ethics_contact_tracing_apps2020.1 who (2021a). contact tracing in the context of covid-19. retrieved from https://www.who.int/publications/i/item/contact-tracing-in-the-context-of-covid-19*, accessed on. who (2021b). covid-19 strategic preparedness and response plan (sprp 2021). retrieved from https://www.who.int/publications/i/item/who-whe-2021.02*, accessed on november 27th, 2021. woodhams, s. (2020). covid-19 digital rights tracker. retrieved from https://www.top10vpn.com/research/covid-19-digital-rights-tracker/*, accessed on november 5, 2021. zastrow, m. (2021). coronavirus contact-tracing apps: can they slow the spread of covid-19? retrieved from https://www.nature.com/articles/d41586-020-01514-2*, accessed on october 26, 2021. zhang, m., chow, a., & smith, h. [helen] (2020). covid-19 contact-tracing apps: analysis of the readability of privacy policies. journal of medical internet research, 22(12), e21572. https://doi.org/10.2196/21572 copyright: © 2022 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v26i0.3097 http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 information privacy in the context of covid-19 contact tracing apps 3 research methodology 3.1 disclosive computer ethics 3.2 case selection and data acquisition 3.3 exploratory analysis 4 results of the exploratory analysis of covid-19 contact tracing apps 4.1 personal information collected 4.2 accounting for potential information privacy violations 4.3 design principles to protect users’ information privacy 5 theoretical evaluation of information privacy in the context of covid-19 contact tracing apps 6 discussion 7 conclusion systems of the information society professor simon rogerson centre for computing and social responsibility de montfort university the gateway leicester lei 9bh uk srog@dmu.ac.uk abstract an inclusive information society needs an infrastructure built upon systems which are in harmony with citizens this paper discusses some of the contextual issues which need to be addressed to realise this goal. there is a need to strive continually to develop adaptive systems which satisfy the dynamic needs of people. the project management of such development activities must be sensitive to the wider social issues. the development activity must adopt a sociotechnical perspective. it must never be forgotten how vulnerable individuals and society are from insecure and inappropriate information systems. it must be recognised that the social systems that are dependant on information systems to survive can be very unpredictable in terms of the benefits gained and costs incurred by both individuals and society at large. introduction the computing technology revolution continues to accelerate. the applications of this universal tool seem limitless. indeed the convergence of computing, media and communication technologies has meant that each day we become more and more dependant on technology at work, at home, in travel, in learning and in communicating. ubiquitous computing has moved from the backroom to everyday applications. thus systems based on computing technology are powerful change agents. we need to think carefully about the ramifications of our actions. we need to consider not only the technological and economic issues but also the ethical and social issues when developing and implementing such systems. this special edition of the australian journal of information systems is based on a selection of papers presented at ethicomp 2001 held at the technical university of gdansk, poland in june 2001. the overall theme of the conference was "systems of the information society". the aim was to focus on the ethical and social impacts of these systems on society, organisations and individuals. this was done from four perspectives: software engineering and systems development; teaching ethics to computing students; ethics in virtual communities; and ethics in the off-line world. the papers included in this edition of the journal reflect many of these themes and illustrate some of the major issues that need to be addressed. systems in context information systems are built by people to be used by people for the benefit of people. the development process is a human activity system and the resulting information systems are used in another human activity systems. a human activity system (has) is defined by banathy (2000) as, "an assembly of people and other resources organized into a whole in order to accomplish a purpose. the people in the system are affected by being in the system, and by their participation in the system they affect the system. people in the system select and carry out activities — individually and collectively — that will enable them to attain a collectively identified purpose." to realise acceptable information systems the development has must be founded on some key concepts as shown in figure 1. there are two elements of development; process and management. the process must focus on an effective and sensitive build whilst management must be empowering in order to realise the building process. those involved in the development must represent all stakeholders. it is essential to adopt a wider perspective which includes those directly and indirectly affected by the system to be developed. this broadening is essential as it includes not only traditional stakeholders viewed from the widely-accepted business perspective but also social actors who schot (2001) defines as, "those who experience the effects of evolving technologies but [traditionally] are not actively involved in the development of technologies. they can be consumers, citizens, employees, companies, social groups etc." 178 manage process beyond the traditional i applications transparency ubiquity context figure 1 systems in context systems need to be viewed from the perspective of everyday applications. ubiquitous computing exists primarily because the majority of everyday activities can be defined as a series of inputs, processes and outputs. as the computer technologies advance and applications become commonplace so transparency increases. people do not give the technology a second thought. it becomes invisible and only the system outcomes are noticed. the settings of applications vary considerably because of differences in culture, religious faith, age, gender, socioeconomic status, literacy levels and technical literacy levels. thus the context of systems demands a broad consideration of the issues. it is a move from a blinkered technological approach to an enlightened inclusive approach. barriers and challenges a number of barriers and challenges exist for the policy makers, managers, developers and implementers of systems. there is a tendency to overlook considerations other than the traditional technical and economic ones. as society's dependency on technology increases the risk increases of creating dangerous systems. technological dependency has a tendency to limit choice implicitly or explicitly as resource availability reduces for maintaining non-technological facilities. however, it will be several generations before the digital solution is widely accepted as the traditional and obvious solution. acceptance is affected by whether a sense of ownership of systems exists within the using community. such challenges point to the need on the part of the builders to accept their social responsibility and act in a way that attacks the digital divide. the digital divide according to the report by booz allen and hamilton (2000) britain is not leading in terms of internet penetration. figure 2 shows a three tier situation. in the us, six out of ten people regularly access the internet from work, home or a public access terminal. in scandinavia around 40% do so. a noticeable gap currently exists between the world's most connected nations in a first tier, and a second tier of nations with penetration levels in the range 15-25%. the uk is top of the second tier. the third tier shows those lagging behind. but even these are way ahead of a much larger group of countries. 179 tier 1 "leaders" 60% 60% 50% . q p p 40% . ul z iu o. gj 30% . t111 z g 20% z 10% 0% tier 2 "followers" a 40% 38% r ^\ tier3 "laggards" jl 22% r ^\ 1 18% 14% ,3% 8% united states sweden finland united netnerland germany france spain italy portugal kingdom source: booz-allen & hamilton figure 2 internet penetration the consensus of the plethora of statistics is that only about 7% to 10% of the world's population is on-line. in global terms the dichotomy of "haves" and "have nots" is vast. with the potential benefit to society and its citizens this virtual divide should be a cause for great concern. consider some of the facts. according to the us internet council's state of the internet report for the year 2000, there is considerable disparity within the continent of africa in terms of internet accessibility. africa has more than 10% of the world's population but less than 1% of the world's internet users. a million south africans have access to the web, but practically nobody does in the congo. looking at south africa in more detail, a 1999 survey of 2000 black south african men and 4000 black south african women by webcheck revealed minimal internet access rates among black africans. out of those surveyed only two men and five women had web access at home and only 24 men and 24 women had web access at work. turning to india, according to emarketer (autumn 2000), the number of active adult internet users in india was 270,000 at the end of 1999 and was set to reach 1.5 million by the end of 2000, but this is only 0.2% of india's adult population. emarketer attributes the low internet penetration in india to extreme and widespread poverty, compounded by a poor telecommunications infrastructure and limited pc ownership. citizen education does education have a role to play in reducing the digital divide? consider the example of electronic commerce. in order to participate fully people need increasing internet skills and knowledge as illustrated in the diagram by booz allen and hamilton (2001) in figure 3. 180 1 lu s uj z*: u. le v e l 0 i author content first i ' power user (active in buying first surf y passive in selling) e-consumer \ f fac&ve; / no connected \ access (passive) t role of government ! .̂ '° î '**-. >va^ ' < -. "̂ silisjdovrfibarrfebhd ' ' ~ break.down barriers to e-cornni jj!j&2sgjg£fz?^^? , % consumer confidence 'ilc >,"-' •c)*a$'a** „• / • . , source: boozallen & hamilton first sale power \ player \ i r fag. amazon; / e-entrepreneur\ / \ (active in selling) / \ \ \ • provide compelling 1 \ value proposition i / • encourage newuserp my\if time erce foster the best environment for ebusiness • entrepreneurship figure 3 increasing internet skills in their report booz allen hamiliton explain that, "to fully capture the benefits of the knowledge economy, users need more than mere availability, and must move beyond passive browsing. they must become active users participating in discussions, educating themselves, making transactions and engaging in commerce. at the next level, the internet offers unprecedented opportunity for consumers to become entrepreneurs themselves; setting up web sites, and ultimately even businesses [see figure 3]. government can play a role not simply in getting people online, but in creating conditions and incentives for them to evolve into active users even 'power users'." in other words through education ordinary people can become more and more proactive to the point where they could be a major player in dictating the nature and influencing the level of penetration of electronic commerce. policy and politics the direction of policy and political strategy has a significant impact on the way systems are perceived and used in the information society. johnson (2000) suggests that there should be a continual assessment of the internet which is the backbone of the information society. this assessment should focus on societal worth and be based on the following questions: • how is the internet governed or should it be? • how is autonomy of nations (states or local communities) recognised in policy decisions regarding the internet or how should it be? • are the economic interests shaping the internet allowed to undermine its potential to serve political and public interests or should it be? • is freedom of expression being protected or should it be? • to what extent is there privacy on the internet and to what extent is the medium a medium of surveillance? (johnson, 2000) whether such assessment takes place and whether it is effective depends to some extent on political thrust hi the major nations of the information society (these nations could reasonably be defined according to figure 2 above). there are two dimensions to this thrust which appear significant. is an apocalyptic approach adopted which scraps all existing systems in favour of virtual alternatives. this is in contrast to an evolutionary approach which replaces traditional systems with virtual alternatives where appropriate and has long transition periods to promote acceptance. the second dimension is concerned with computer technology as a driver. there are those who firmly believe that computer technology and its ability to create a virtual world is the only way to move forward. such "wired-up thinking" is badly flawed. what is required is "joined-up thinking" where the technology is viewed as one of several options available to resolve problems and create new opportunities for social benefit and wealth creation. 181 conclusion it seems clear from the issues raised so far that an inclusive information society needs an infrastructure built upon systems which are in harmony with citizens. the papers that follow illustrate this point. barbara bergier argues that human-centred goals must be a priority when considering the development and implementation of systems. in this way we are more likely to get systems that are adaptive to our needs rather than systems that demand us to be adaptive to the foibles of poor system design. don gotterbam argues that narrow risk assessment and a narrow scope of software significantly contribute to system failures. he presents a new project management approach in order to encourage a wider consideration and thereby reduce the likelihood of system failures. joe thomas and andy bissett discuss a case study about an assessment and referral process for drug misusers. they highlight the serious problems that can arise through using an information system as part of this process. paula roberts and jenny webber discuss the role of computer hackers in society. they take a fresh look at how we might limit damage by malicious actions. finally, helen richardson and kate richardson describe a case study of a call centre and the use of a customer relationship management system. they discuss the contrasting opinions as to the worth of this type of human activity system. there are some important messages in this collection of papers. we must continually strive to develop adaptive systems which satisfy the dynamic needs of people. the project management of such development activities must be sensitive to the wider social issues. the development activity must adopt a socio-technical perspective. we must never forget how vulnerable individuals and society are from insecure and inappropriate information systems. finally, the social systems that are dependant on information systems to survive can be very unpredictable in terms of the benefits gained and costs incurred by both individuals and society at large. references booz-allen and hamilton (2000) achieving universal access, booz-allen and hamilton, london, march. johnson, d. g. (2000) democratic values and the internet, in langford, d. internet ethics, macmillan press, new york, chapter 8, pp 181-199. schot j. (2001) constructive technology assessment as reflexive technology politics, in goujon, p., dubreuil, b.h., (eds), technology and ethics: a european quest for responsible engineering, peelers, leuven, belgium, chapter 2.1.1, pp 239-249. 182 paper title (paper title style) australasian journal of information systems burton-jones et. al 2025, vol 29, provocation this article is not just in english greek page 13 1 spanish page 23 bahasa page 31 urdu page 39 this article is not just in english: making science more inclusive and impactful with artificial intelligence translation andrew burton-jones university of queensland email: abj@business.uq.edu.au1 tatsuya amano university of queensland email: t.amano@uq.edu.au james boyce university of queensland email: j.boyce@business.uq.edu.au patrick chau bnu-hkbu united international college china email: patrickchau@uic.edu.cn juan (jane) feng tsinghua university email: fengjuan@sem.tsinghua.edu.cn indira r. guzman cal poly pomona email: irguzman@cpp.edu sirkka jarvenpaa university of texas, austin email: sirkka.jarvenpaa@mccombs.utexas.edu jingqi (celeste) li university of queensland email: jingqi.li@business.uq.edu.au morteza namvar university of queensland email: m.namvar@business.uq.edu.au balaji padmanabhan university of maryland college park email: bpadmana@umd.edu jose pineda california state university long beach email: jose.pinedadelgado@csulb.edu 1 the authors are members of the association for information systems taskforce on ai translation for inclusive, impactful science. andrew is the chair; the other authors are all listed alphabetically. australasian journal of information systems burton-jones et al. 2025, vol 29, provocation this article is not just in english 2 jean-loup richet sorbonne business school email: jean-loup.richet@iae.pantheonsorbonne.fr suprateek sarker university of virginia email: ss6wf@virginia.edu stefan seidel university of cologne email: stefan.seidel@wiso.uni-koeln.de sujeet sharma iim nagpur email: sujeet@iimnagpur.ac.in doug vogel harbin institute of technology email: vogel.doug@gmail.com harry wang university of delaware email: hjwang@udel.edu victoria yoon virginia commonwealth university email: vyyoon@vcu.edu abstract humanity is linguistically diverse, but science is not. academic success requires englishlanguage mastery. every major conference, every major journal – even this one – assumes it. english-language bias in science is so strong that it is taken for granted by most scientists and scientific associations, never talked about nor addressed. this is unfair and creates great costs and missed opportunities. it is also unnecessary. artificial intelligence (ai) translation tools are becoming very good, very fast, allowing us to foresee a multilingual science. our provocation to readers is: how should we harness ai translation tools for a more impactful, inclusive science? this is a challenge ideally suited to information systems scholars because it involves designing sociotechnical artifacts and practices for a better future. to demonstrate feasibility, this article went through a multilingual review process and is published in five languages, all enabled by ai translation. 1 the problem of language bias one of the most obvious and least discussed aspects of the australasian journal of information systems (ajis) is that it is an english-language journal. this is not discussed or debated because the choice is natural. all the top conferences and journals in the information systems (is) field are in english, as they are in every other discipline. performance in science assumes english. this is unfair, however, because most of the world does not speak english. of the 8 billion people on earth, it is estimated that only 1.5 billion speak english, and only 25% of them speak english as a first language (icls, 2024). australasian journal of information systems burton-jones et al. 2025, vol 29, provocation this article is not just in english 3 the goal of this provocation is to raise awareness of the problem of ‘language bias’ in the is field and call for efforts to address it. we describe how we are starting this work, as members of an association for information systems (ais) taskforce, and we call for community-wide action. it is hard to shake the feeling that we are behind other fields in our awareness of language bias. it was raised in medicine in the 1990s (egger et al., 1997), and has since been raised in many fields including emotion research (wierzbicka, 2009), ecology (clavero, 2011), linguistics (levisen, 2019), management (pudelko & tenzer, 2019), law (ammann, 2022), cognitive science (blasi, henrich, adamou, kemmerer, & majid, 2022), conservation (amano & berdejo-espinola, 2024), earth sciences (bampi, pires-oliveira, loyola-bartra, & limaribeiro, 2024), critical care (gonzález-dambrauskas, salluh, machado, & rotta, 2024), and orthodontics (mheissen, spineli, daraqel, & alsafadi, 2024). it has also been raised in sciencewide discussions of the future of academic publishing (ahmed et al., 2023; amano, rojas, boum ii, calvo, & misra, 2021; arenas-castro et al., 2024; editorial, 2023). like most biases, language bias is costly. as amano et al. (2023) report, non-native english scholars have to spend longer reading, writing, and revising papers, are more likely to have their papers rejected for language-related reasons, and experience barriers attending conferences. language bias adds stress for early career scholars (hohti & truman, 2021), detracts from the peer review process (politzer-ahles, girolamo, & ghali, 2020), creates biases in literature reviews (nunez & amano, 2021), systematic reviews (hannah, haddaway, fuller, & amano, 2023), and meta-analyses (konno et al., 2020), and skews national research comparisons (van leeuwen, moed, tijssen, visser, & van raan, 2001). language bias is even present in google scholar searches, as it privileges english-language articles (rovira, codina, & lopezosa, 2021). as a systemic, long-lived bias, language bias can also be seen in national practices, such as the prevalence of english teaching in schools around the world, and major industries, such as the global usd$10-11b english-as-a-second-language (esl) industry.2 while we seek to highlight the linguistic bias affecting science, we are not naïve as to the benefits of global english. given that language diversity naturally impedes communication (e.g., illustrated in the famous tower of babel), convergence towards one language (whether english or any other language) can facilitate progress in many sectors (john & özgür, 2021). as the british council (2013, p. 3) noted “thanks to the internet, the rise of social media, the speed and spread of global communications technology and the increasingly globalised and interdependent global economy, english now allows the rapid cross-pollination of ideas and innovation around the world, and the development of a new kind of supranational single market in knowledge and ideas.” however, while we recognize the benefits of english-language proliferation (and several of our author team have enjoyed these benefits, as native english speakers), our aim here is to highlight the costs, because they are significant yet generally unacknowledged and taken for granted (mélitz, 2007; salomone, 2022). 2 the promise of ai translation tools despite the prevalence and costliness of language bias, it should no longer be taken for granted because artificial intelligence (ai) translation tools are getting very good, very fast. the historic 2 https://www.businessresearchinsights.com/market-reports/esl-english-as-a-second-language-market117478; https://www.englishaustralia.com.au/our-sector/understanding-the-sector australasian journal of information systems burton-jones et al. 2025, vol 29, provocation this article is not just in english 4 nature of this moment must be stressed, for language bias is as old as science. after all, science has always been conducted in one language or another, and to participate (read, speak, write), you had to know the language (gordin, 2015). with over 7 000 languages on earth (anderson, 2010), scientists could not read/write in every language. for millennia, it simply made sense to conduct science in the language of the day. dominant languages at different times in history included egyptian, greek, latin, chinese, and various european languages, but english has dominated since wwii (gordin 2015). despite brief (and failed) attempts to develop a universal language (e.g., esperanto) (patterson & huff, 1999), the only solution to address language differences over the years has been translation. however, despite existing as a field for over 2 000 years (dynamic_language, 2024), translation has, as far as we can discern, never had widespread uptake in science, largely because the translation process is costly and time-consuming (wood, 1967). across history, various journals have been published in multiple languages, and even today there are around 500 multilingual journals,3 but these are the exceptions that prove the rule. as gordin (2015, p. 3) evocatively writes, “if you are interested in what it would be like to live in a world with one language of communication, a world with no babel, you should look to the natural scientists. they come from there.” the vast majority of journals are in english only (gordin, 2015), and most multilingual journals are just two-language journals between english and another language, and most of them are only bilingual because of national laws that require it (e.g., journals published in canada, which is a bilingual country). the potential for ai translation tools to tackle language bias in science has emerged only recently (berdejo-espinola & amano, 2023; crotty, 2024; steigerwald et al., 2022), stemming from two long-term transformations. one has been the gradual digital transformation of research itself. not long ago, many research tasks were manual. researchers walked to libraries, read and annotated physical papers, sent papers and reviews by mail, even made calculations and wrote papers by hand. now, almost all tasks are online: searching, reading, writing, annotating, collaborating, analysing, reviewing, submitting, publishing. a profound consequence of this transformation is that much of science now involves language on digital platforms. that is, scientific papers are written, reviewed, published, read, and searched for in a language (nowadays almost always in english), and all this happens on large, interconnected digital platforms. some of these platforms are now vast, e.g., scopus hosts over 90 million records4 and google scholar searches across 300 million records (martin-martin, thelwall, orduna-malea, & lopez-cozar, 2021). and this is where the second transformation comes in – the ai transformation. although language models in ai have a history dating to the 1950s, the rapid development of large language models was enabled by the massive growth of online data for learning (naik, naik, & naik, 2024) together with new architectures, especially the transformer architecture, that can leverage such data (vaswani et al., 2017). in the last five years, these two transformations have enabled new forms of ai to emerge, enabling progress that seemed impossible a short time ago. and this progress is especially fast in language translation. language translation tasks were the first tasks to be tested on the transformer architecture and so are ideally suited for it (vaswani et al., 2017). 3 https://en.wikipedia.org/wiki/category:multilingual_journals 4https://assets.ctfassets.net/o78em1y1w4i4/76uvtuilkssqe4h4h1pzkq/e5ff8eef9694f216a5c29e2ebf80 fa51/fast-facts-about-elsevier.pdf australasian journal of information systems burton-jones et al. 2025, vol 29, provocation this article is not just in english 5 we are not claiming that ai translation can completely and quickly solve language bias. there will be many complexities (amano and berdejo-espinola 2024). but the potential is strong enough for us to have a duty to pursue it. from the perspective of a scientific association, such as the association for information systems, it would seem negligent if we did not try to reduce a prevalent and costly systematic bias facing members of our scientific field and similarly negligent if we did not try to leverage the new opportunities for inclusion and impact that ai translation could afford. that is why we formed our taskforce – the association for information systems taskforce on ai translation for inclusive, impactful science. 3 complexities there are many complexities with the use of ai translation in science. we highlight six categories. first, there are complexities that depend on how we scope the problem. the simplest scope is to focus on the translation of a single paper from one language to another, but even this scope raises questions. for instance, should we translate only accepted papers, or also submitted papers? the former would aid science by promoting impact – increasing a paper’s readership – while the latter would also support inclusion – enabling more authors and reviewers to write in their first language. while the latter choice has greater benefits, it also takes more work, requiring that we translate both the paper and the paper’s reviews, and also requiring a workflow for authors and reviewers to verify the quality of the translations during the review process without over-burdening them. once a paper is accepted/published, another question is whether we focus on translation to just one language or multiple, and whether the author or the reader gets to choose the language. if readers get to choose, should the translations be prepared beforehand (e.g., in a small number of common languages) and readers get to choose the language they prefer from this list, or should readers be able to request a translation in their chosen language (either from a fixed list, or in an ideal world, from any language) which is then produced in real-time by the publication platform upon request? the latter would enable a wider range of languages while the former would allow a verification step for translation quality. if the latter is chosen, it is possible that small changes in meaning could be introduced in some translations without the author(s) being aware, creating complications for subsequent citations of that work. papers being published with the on-demand, real-time translation option may therefore require a disclaimer that acknowledges the potential for errors. another scope question is whether we look beyond the translation of a single paper and consider translating an entire library/platform of papers, such as the entire association for information systems elibrary, which holds over 65,000 papers? expanding the scope to a library not only increases the number of papers to translate, but also brings in additional considerations. for instance, it can expand the temporal window of translation, because a library may have papers published over many decades, and so the translation process may need to consider changes in the cultural context and lexicon of a field over time. likewise, it may require the development of complementary tools, such as a multilingual search tool to search across the library in multiple languages. one final scope question relates to the capabilities of ai. given that the same ai technology that performs translations offers related capabilities, should we use these capabilities too? for instance, should we look beyond the translation of papers to consider the production of paper australasian journal of information systems burton-jones et al. 2025, vol 29, provocation this article is not just in english 6 summaries in different languages (e.g., lay summaries for executives or students)? likewise, should we consider language improvements, not just translations, e.g., offering suggestions for improving a text, not just translating the text ‘as is’? if we considered the latter, we could reframe the problem from mere ‘translation’ to ‘explication of meaning’ where translation is just one element of the explication of meaning and other elements could include improvements in grammar, flow, and style. as you can see, the problem scope can vary greatly. at this stage, our taskforce is adopting a middle-ground scope. we seek to support the entire review process – from submission to acceptance. we also seek to support translations into multiple languages. however, we are limiting our efforts to 10 major languages at first (bahasa indonesia, urdu, greek, french, malay, korean, mandarin, german, spanish, and portuguese) and focusing on developing pre-prepared translations that allow a human-in-the-loop quality verification step rather than enabling on-demand, real-time translations. we are including the production of paper summaries within our scope but we only focus on translating individual papers rather than an entire library/platform, and we are not considering broader aspects of ‘explication of meaning’ such as language improvements in this first phase. in addition to these complexities that arise due to questions of scope, another set of complexities arises from the practicalities of the publication environment. for instance, how should we deal with digital object identifiers (dois)? nowadays, each published paper should be assigned a unique doi because this allows search engines and aggregators to locate and identify citations for each paper. however, if we create multiple translations of a paper, would these be included in the same document (e.g., webpage, pdf) with the same doi, and would citations of the paper be aggregated irrespective of the version (translation) being cited? one difficulty with answering these questions is that no entity (scientific association, publisher, big tech) has control of the whole publication ecosystem, and it is hard to predict how it will all evolve. we therefore have to choose translation practices that will somehow align with existing practices while being flexible enough to adapt. similar practical complexities arise with copyright. a translation can be considered a ‘derivative work’ and therefore must comply with copyright restrictions over such works. another practical issue is plagiarism. ironically, facilitating translation might also facilitate plagiarism. this is because plagiarism is typically defined and detected by the use of the same sequence of words in a paper, i.e., word-for-word copying. translation need not be word-forword, but can be ‘sense-for-sense (conveying the same sense rather than the same words) (sun, 2013). if a plagiarist plagiarised a paper by creating a translation using sense-for-sense translation, it is not clear if current plagiarism detection tools would detect it. such tools may need to be revised to control for this risk. a third set of complexities arise from the characteristics of ai. even though ai’s rapid progress motivated us to pursue this project, it is still not perfect. hallucinations still occur (ji et al., 2023), and ai translation performance varies depending on the amount of training data in each language (xu et al., 2024). because of the sheer dominance of english-language on the internet, gen ai models may embed not only english-language bias but cultural bias too (luo & puett, 2024). we call on researchers across our field to create tangible sociotechnical artifacts and practices to address these issues. in doing so, researchers should not be seduced by technological determinism or social determinism alone – a sociotechnical view is critical. that is, just like any other technology, the success of ai translation will depend not only on the australasian journal of information systems burton-jones et al. 2025, vol 29, provocation this article is not just in english 7 technology but also on how users (authors, editors, reviewers, readers) perceive it and use it. for instance, human tendencies such as algorithm aversion and mistrust will inevitably need to be considered (asscher & glikson, 2023). running ai models also bears a financial cost, raising the question of who should bear such cost. one of the reasons we are pursuing this as an ais taskforce is that we are considering whether and how a scientific association should bear that cost for its members. a fourth set of complexities arises due to nuances in written language that current ai translation models can miss. for instance, in this paper’s review process, one of our reviewers noticed slippages in our translations of acronyms. for instance, in spain, authors generally use the spanish version of the acronym (ia), but in greece, authors are equally accustomed to using the english version (ai) or greek version (tn), and author should just use one consistently. our translation failed to do this. meanwhile our other reviewers noticed that our translations sometimes sounded overly “rigid,” “strong,” or “formal” and that we should have adopted a “gentler” or more “natural” tone. this may have been due to the ai translation automatically choosing an overly formal tone, which we did not realize in advance. to address these challenges, we are actively refining our translation approach. in particular, we have incorporated feedback from this paper’s reviewers, especially regarding the evaluation of cultural naturalness and tone. as a result, our translation approach now gives greater consideration to these aspects. for example, internal tool logs have captured reviewer-inspired suggestions such as: “the phrase ‘…’ is quite direct. consider rephrasing it to a more academic tone, such as ‘…’.” or “the phrase ‘…’ is somewhat literal. a more suitable academic phrasing might be ‘…’.” despite the rapidly growing strength of ai translation, it is hard to imagine that these tools will offer perfect translation across every language and tailored for every paper, but we intend to tackle as many of these problems as we can (e.g., by modifying the agentic architecture of the ai model to use context more effectively). a fifth set of complexities arise from the institutions of science that could reinforce the status quo and impede the emergence of multilingual science. for instance, when we have discussed this initiative with successful researchers who are non-native english speakers, we have been surprised how many have said they would not take up our opportunity to write in their native language because they are better at writing papers in english (because they have trained all their working lives in english). in some ways this does not concern us because the population of non-native english speakers who could benefit is far larger than the small population of non-native english speakers who have ‘made it’ in the english-dominated system. on the other hand, senior members of a community naturally influence junior members, and so their views may serve to reinforce the status quo for some time. as a case in point, we have been working with senior researchers in one european country on this initiative. even though they strongly support our initiative, they encouraged us to go slowly because they had worked for decades to improve english-language writing in their community, and that effort had been very successful, leading to a sharp uptick in the number of publications in top journals. the last thing these researchers wanted to do was to detract from that success. as this example shows, efforts to encourage multilingual science need be designed carefully in the knowledge of other pre-existing initiatives designed around the status quo. finally, we have a sixth set of complexities that are more philosophical. one way to view these is to think of a paper as just one step in the long process of an idea forming in an author’s mind to it being appreciated in a reader’s mind. for starters, consider an author thinking of an idea. the sapir-whorf hypothesis suggests that an author’s thoughts are shaped by the author’s language (lucy, 1998). if true, it implies that language bias goes all the way back to the origin, even restricting the ideas authors can form. we can call this the ‘conception’ problem. the australasian journal of information systems burton-jones et al. 2025, vol 29, provocation this article is not just in english 8 problems then continue because even if a language can be used to convey an idea, an author may still lack the language skill to do so. we can call this the ‘articulation’ problem. and even if an author has the right language to convey an idea, and uses it skilfully, a reader may not interpret the words in the same way. we can call this the ‘interpretation’ problem. we are sure that readers of this paper are familiar with such problems because they are common in daily life. collectively, these problems mean that research papers are inevitably an imperfect medium through which to convey scientific ideas in a community of scholars. imperfections in meaning are simply inherent to the process, irrespective of the language used. clearly, ai translation tools cannot solve all these problems. as just one example, ai tools can help reduce the conception problem by enabling people to think in their own language, but they may also fail to do so because english-language dominance has been so strong for so long that some languages do not have the required lexicon (barath, 2019). 4 an opportunity and our approach while we have listed many complexities, we see them as opportunities for contribution rather than problems to avoid. to us, they underscore why this is an exemplary challenge for the is field, for the opportunities and complexities are deeply sociotechnical (sarker, chatterjee, xiao, & elbanna, 2019). as we mentioned at the outset, the is field is not the first to become aware of language bias. even so, while many fields have raised awareness of the problem, few have systematically tackled it. as a result, the is field has a chance to help work at the forefront of science together with other leading fields.5 given that the subject matter is so intimately tied to the nature of our field – as a field dedicated to understanding and designing sociotechnical artifacts and practices for a better future – we believe the is field not only has a chance to make a difference, but also has a duty to do so. motivated by this opportunity, we formed an ais taskforce in 2024. in line with common models of change (lewin, 1951), we assume that a substantial effort will be needed to ‘unfreeze’ the field rather than assuming that the field can change straight away. to facilitate ‘unfreezing,’ we ran panels and plenaries at four major conferences to build awareness and motivate discussion (burton-jones, 2025; burton-jones, chau, feng, kankanhalli, & vogel, 2024; pouloudi et al., 2024; sarker, ahuja, feng, jarvenpaa, & seidel, 2024). we also talked with several ais chapters representing different language communities and we worked on the development and initial testing of ai translation tools. in 2025, we are continuing our efforts to build awareness of the problem (such as through this paper) while also taking a more active stance with cycles of designing, testing, and learning. for instance, we are grateful to the editors and reviewers of ajis for being willing to test a multilingual review process for this paper, from which we can learn and improve. we are simultaneously engaged in other work to provide concrete data and tests. for instance, we tested ai translations in the americas conference on information systems (amcis) in 2025 (see: https://aisel.aisnet.org/amcis2025/translated/). we are also working with other conferences in 5 other leading voices at this forefront include unesco (https://doi.org/10.54677/mnmh8546) and the council of the european union (https://www.consilium.europa.eu/media/56958/st10126-en22.pdf). australasian journal of information systems burton-jones et al. 2025, vol 29, provocation this article is not just in english 9 2025 on different translation use cases6. we are focusing especially on three use cases: (1) ai translation in the review process; (2) ai translation of accepted papers; and (3) ai preparation of summaries of papers in different languages to spread impact. we will also conduct surveys, interviews, and focus groups to learn how language bias affects scholars in the is field and the potential impacts of ai translation tools. we are also engaging with scholars in other fields to support the larger science-wide movement (amano, bowker, & burton-jones, 2025). in pursuing these steps, we recognize that we will face many challenges, and that many further opportunities remain. we can also learn lessons and take encouragement from other sciencewide movements, such as the open science movement (ferguson et al., 2023), that are similarly motivated towards greater inclusiveness and impact in science. like any change effort, we will make mistakes and have to make detours in our plans. nevertheless, we remain excited by the possibilities, and we hope other is researchers will be excited too, and that together we can make meaningful change. rather than this being a solitary, tightly controlled, top-down effort, we call for efforts from across the community to tackle the many diverse aspects of the issue, and we urge editors and conference chairs across the field to support such bottom-up efforts. as a first step, we are especially grateful to the editors of ajis for allowing this multilingual review process. the process was as follows: (1) we submitted the manuscript in english; (2) the handling editor identified relevant reviewers and their associated languages (in our case, reviewer 1 identified with both greek and spanish, reviewer 2 identified with bahasa indonesia, and reviewer 3 identified with urdu); (3) we translated the paper into all four languages using our ai translation system and submitted all five versions to the editor, who then sent the papers in both english and the other relevant languages to each reviewer; (4) the reviewers wrote their reviews in their preferred language; reviewer 1 wrote his review in english, while reviewers 2-3 wrote their reviews in their native language; (5) we translated the non-english reviews into english in order to read them, and then revised the paper to account for the reviewers’ suggestions, translated the paper back into all the languages, wrote our response document and translated it into all the languages too, and then resubmitted the paper and responses; (6) the editor then accepted the paper and we incorporated the review comments below. the final step with the journal also involved discussing how to manage the doi, and we agreed in this trial process to use just one doi and one pdf file, for simplicity. in a spirit of openness, after the paper’s acceptance, we asked the reviewers of the paper to provide their own unedited comments below, to share their reflections with the community. likewise, they revealed their names below in a similar spirit of openness. 4.1 reflective comments from reviewer 1 (greek) being able to review this article in three languages, including my native one, greek, was a very pleasant experience. this work enabled me to reflect on the fact that we should be grateful for the opportunity the english language provides us to connect with others around the world at so many levels, including the scientific one. however, despite this benefit, we should acknowledge that english is not accessible to all, and artificial intelligence (ai) offers important alternatives that we could capitalize on, and the greek translation is a tangible example of that. the richness and complexity of the greek language may have not allowed for 6 we thank the conference chairs and program chairs of amcis 2025, pacis 2025, and wi 2025, for their willingness to engage with this initiative. australasian journal of information systems burton-jones et al. 2025, vol 29, provocation this article is not just in english 10 a “perfect” translation as, indeed, there were some grammatical and other mistakes, as well as expressions that are not compatible with how the language is naturally used by native speakers. however, i expect that this will be addressed in the near future given ai’s capacity for self-improvement. congratulations to the authors on coming up with the idea and writing this article! (petros chamakiotis) 4.2 reflective comments from reviewer 1 (spanish) in addition to my comments above regarding the greek version, the translation into spanish was of better quality compared to the greek one, possibly because artificial intelligence (ai) is better “trained” in spanish, compared to greek, due to the larger number of spanishspeaking users. also, although i believe this is not the case in spanish-speaking countries in latin america, spaniards do not use english terms in everyday language, and this is something that ai did not take into account. for example, the acronym for ai in english is ai, while in spanish it is ia (since the term is “inteligencia artificial”). also, there are many “false friends” that affected the quality of the translated version. one of them was the word “conference” when referring to academic conferences, which in spanish is more accurately translated as “congreso” (instead of “conferencia”). (petros chamakiotis) 4.3 reflective comments from review 2 (bahasa indonesia) this paper provides a thorough examination of english language bias in academic settings and explores how ai translation tools could drive significant change. the paper's main strength lies in its clear problem identification, backed by literature from diverse fields that demonstrates both the urgency and negative impacts of language bias. the vision for multilingual science is compelling and well-articulated, with ai representing a fitting innovative solution for the information systems field. certain areas could, however, benefit from further development. while ai complexities are mentioned, the discussion on addressing ai limitations like hallucinations or cultural bias needs more depth. for instance, some indonesian translations in the first draft of this article feel somewhat rigid. the phrase "our provocation to readers" translated as "tantangan kepada pembaca" comes across as too "forceful" a more natural version would be "kami ingin mengajak pembaca berpikir" which sounds more "gentle." the complexity analysis appears to remain at surface level, particularly regarding technical issues like translation accuracy or cultural impact. the arguments also lack strong empirical support. future research might consider incorporating pilot study results and deeper technical analysis. further exploration of is-specific contexts, such as multilingual implementation in enterprise systems, could also enrich this paper's contribution to the broader academic community. (arif perdana) 4.4 reflective comments from reviewer 3 (urdu) thank you for inviting me to reflect on this insightful provocation. reviewing this article in urdu was a rewarding experience. it was the first time i had engaged with academic work in my native language, and the experience felt genuinely inclusive, something i had not experienced before. the key message of the paper, that scientific knowledge should not be confined to a single language, resonated deeply with me. it highlights how the dominance of english can unintentionally limit both participation and understanding in scientific discourse. overall, the translation into urdu is quite well-executed. however, in some places, the translation feels slightly rigid and breaks the flow due to the word-for-word approach instead australasian journal of information systems burton-jones et al. 2025, vol 29, provocation this article is not just in english 11 of adapting expressions for natural urdu phrasing. having said that, these challenges do not diminish the potential of multilingual science; instead, they reveal where important work still needs to be done. with continuous input and oversight from humans, ai translation tools have the potential to transform science into something not only globally accessible but also locally meaningful and resonant. (humza naseer) 5 conclusion language bias has been a necessary evil throughout the history of science, but it need not be anymore. progress in ai translation portends a new future for science—a multilingual future. a multilingual science can be more inclusive (allowing more authors and reviewers to participate) and more impactful (allowing more readers to learn from science). like any digital transformation, the effects of this shift will be unpredictable, and there will be much to learn. few opportunities appear more well-suited to the is field than this one, and the time for action is now. we look forward to sharing our insights as we progress our work and we hope many others will join the effort. references ahmed, a., al-khatib, a., boum ii, y., debat, h., dunkelberg, a. g., hinchliffe, l. j., pruszynski, j. a. (2023). the future of academic publishing. nature human behaviour, 7, 1021–1026. doi.org/10.1038/s41562-023-01637-2 amano, t., & berdejo-espinola, v. (2024). language bias in conservation: consequences and solutions. trends in ecology and evolution, forthcoming (march). doi.org/10.1016/j.tree.2024.11.003 amano, t., bowker, l., & burton-jones, a. (2025). ai-mediated translation presents two possible futures for academic publishing in a multilingual world. plos biology, forthcoming. amano, t., ramirez-castaneda, v., berdejo-espinola, v., borokoni, i., chowdury, s., golivets, m., . . . verissimo, d. (2023). the manifold costs of being a non-native english speaker in science. plos biology, 21(7), 1-27. doi.org/10.1371/journal.pbio.3002184 amano, t., rojas, c. r., boum ii, y., calvo, m., & misra, b. b. (2021). ten tips for overcoming language barriers in science. nature human behaviour, 5, 1119-1122. doi.org/10.1038/s41562-021-01137-1 ammann, o. (2022). language bias in international legal scholarship: symptoms, explanations, implications and remedies. the european journal of international law, 33(3), 821-850. doi.org/10.1093/ejil/chac044 anderson, s. r. (2010). how many languages are there in the world? washington dc: linguistic society of america. arenas-castro, h., berdejo-espinola, v., chowdury, s., rodriguez-contreras, a., james, a. r. m., raja, n. b., amano, t. et al. (2024). academic publishing requires linguistically inclusive policies. proceedings of the royal society b, 291, 1-8. doi.org/10.1098/rspb.2023.2840 australasian journal of information systems burton-jones et al. 2025, vol 29, provocation this article is not just in english 12 asscher, o., & glikson, e. (2023). human evaluations of machine translation in an ethically charged situation. new media & society, 25(5), 1087-1107. doi.org/10.1177/14614448211018833 bampi, h., pires-oliveira, j. c., loyola-bartra, o., & lima-ribeiro, m. s. (2024). language bias, not knowledge shortfall, underestimates the evidence of megafauna kill sites in south america. journal of south american earth sciences, 146. doi.org/10.1016/j.jsames.2024.105078 barath, h. (2019). indian initiatives aim to break science's language barrier. nature, 571, 289290. doi.org/10.1038/d41586-019-01815-1 berdejo-espinola, v., & amano, t. (2023). ai tools can improve equity in science. science, 379, 991-991. doi:10.1126/science.adg9714. blasi, d. e., henrich, j., adamou, e., kemmerer, d., & majid, a. (2022). over-reliance on english hinders cognitive science. trends in cognitive sciences, 26(12), 1153-1170. doi.org/10.1016/j.tics.2022.09.015 british_council. (2013). the english effect: the impact of english, what it's worth to the uk, and why it matters to the world: https://www.britishcouncil.org/sites/default/files/englisheffect-report-v2.pdf. burton-jones, a. (2025). why is ecis in english? let’s create a fairer, more impactful science by using ai to move beyond english language dominance. paper presented at the european conference on information systems, amman, jordan, june, pp. 1-4. burton-jones, a., chau, p. y. k., feng, j., kankanhalli, a., & vogel, d. (2024). ai translation and the future language of science. paper presented at the proceedings of the pacific-asia conference on information systems, ho chi minh city. clavero, m. (2011). language bias in ecological journals. frontiers in ecology and the environment, 9(2), 93-94. crotty, d. (2024). ai, translations, and the dominance of the english language. scholarly kitchen, feb 23, https://scholarlykitchen.sspnet.org/2024/02/23/ai-translations-and-thedominance-of-the-english-language/. dynamic_language. (2024). the history of translation: a journey through time and its modern impacts, available at: https://www.dynamiclanguage.com/the-history-oftranslation-a-journey-through-time-and-its-modern-impacts/, accessed jan 26, 2025. editorial. (2023). scientific publishing has a language problem. nature human behaviour, 7, 1019-1020. doi.org/10.1038/s41562-023-01679-6 egger, m., zellweger-zahner, t., schneider, m., junker, c., lengeler, c., & antes, g. (1997). language bias in randomized control trials published in english and german. the lancet, 350(august), 326-329. ferguson, j., littman, r., christensen, g., levy paluk, e., swanson, n., wang, z., . . . pezzuto, j.-h. (2023). survey of open science practices and attitudes in the social sciences. nature communications, 14(5401), doi.org/10.1038/s41467-41023-41111-41461. australasian journal of information systems burton-jones et al. 2025, vol 29, provocation this article is not just in english 13 gonzález-dambrauskas, s., salluh, j. i. f., machado, f. r., & rotta, a. t. (2024). science over language: a plea to consider language bias in scientific publishing. critical care science. doi.org/10.62675/2965-2774.20240084-en gordin, m. d. (2015). scientific babel: how science was done before and after global english. chicago: the university of chicago press. hannah, k., haddaway, n. r., fuller, r. a., & amano, t. (2023). language inclusion in ecological systematic reviews and maps: barriers and perspectives. research synthesis methods, 15, 466-482. doi.org/10.1002/jrsm.1699 hohti, r., & truman, s. e. (2021). anglocentrism in the academy: on linguistic privilege, mastery and hoito. reconceptualizing educational research methodology, 12(2), 1-21. doi.org/10.7577/rerm.4675 icls. (2024). most spoken languages in the world, international centre for language studies, https://www.icls.edu/blog/most-spoken-languages-in-the-world. ji, z., lee, n., frieske, r., yu, t., su, d., xu, y., . . . fung, p. (2023). survey of hallucination in natural language generation. acm computing surveys, 55(12), 1-38. doi.org/10.1145/3571730 john, a., & özgür, o. (2021). linguistic diversity in the very long run. the economic journal, 131(635), 1186–1214, doi.org/1110.1093/ej/ueaa1098. konno, k., akasaka, m., koshida, c., katayama, n., osada, n., spake, r., & amano, t. (2020). ignoring non-english-language studies may bias ecological meta-analyses. ecology and evolution, 10, 6373-6384. doi.org/10.1002/ece3.6368 levisen, c. (2019). biases we live by: anglocentrism in linguistics and cognitive sciences. language sciences, 76. doi.org/10.1016/j.langsci.2018.05.010 lewin, k. (1951). field theory in social science; selected theoretical papers. d. cartwright (ed)., new york: harper and row. lucy, j. a. (1998). sapir-whorf hypothesis. in routledge encyclopedia of philosophy: taylor and francis. luo, q., & puett, m. (2024). correspondence: ai's english-language bias is cultural bias too. nature, 629(may), 1004. martin-martin, a., thelwall, m., orduna-malea, e., & lopez-cozar, e. d. (2021). google scholar, microsoft academic, scopus, dimensions, web of science, and opencitations’ coci: a multidisciplinary comparison of coverage via citations. scientometrics, 126, 871-906. doi.org/10.1007/s11192-020-03690-4 mélitz, j. (2007). the impact of english dominance on literature and welfare. journal of economic behavior & organization, 64(2), 193-215, doi.org/110.1016/j.jebo.2006.1010.1003. mheissen, s., spineli, l. m., daraqel, b., & alsafadi, a. s. (2024). language bias in orthodontic systematic reviews: a meta-epidemiological study plos one, 19(4), 1-14. doi.org/10.1371/journal.pone.0300881 naik, d., naik, i., & naik, n. (2024). large data begets large data: studying large language models (llms) and its history, types, working, benefits and limitations. paper presented at australasian journal of information systems burton-jones et al. 2025, vol 29, provocation this article is not just in english 14 the contributions presented at the international conference on computing, communication, cybersecurity and ai, london uk. nunez, m. a., & amano, t. (2021). monolingual searches can limit and bias results in global literature reviews. nature ecology & evolution, 5(march), 264-264. patterson, r., & huff, s. m. (1999). the decline and fall of esperanto: lessons for standards committees. journal of the american medical informatics association, 6(6), 444-446. politzer-ahles, s., girolamo, t., & ghali, s. (2020). preliminary evidence of linguistic bias in academic reviewing. journal of english for academic purposes, 47, 1-12. doi.org/10.1016/j.jeap.2020.100895 pouloudi, n., guzman, i. r., g.r., a., pineda, j., sabherwal, r., & sheng, o. l. (2024). panel: has the time come? harnessing ai translation for inclusive, impactful science. paper presented at the proceedings of the americas conference on information systems, salt lake city. pudelko, m., & tenzer, h. (2019). boundaryless careers or career boundaries: the impact of language barriers on academic careers in international business schools. academy of management learning & education, 18(2), 213-240. doi.org/10.5465/amle.2017.0236 rovira, c., codina, l., & lopezosa, c. (2021). language bias in the google scholar ranking algorithm. future internet, 13(31), 1-17. doi.org/10.3390/fi13020031 salomone, r. (2022). the rise of english: global politics and the power of language. oxford university press. sarker, s., ahuja, m. k., feng, j., jarvenpaa, s., & seidel, s. (2024). panel: what if the is discipline was multilingual? ai translation and the future of science. paper presented at the proceedings of the international conference on information systems, bangkok. sarker, s., chatterjee, s., xiao, x., & elbanna, a. (2019). the sociotechnical "axis of cohesion" for the is discipline: its historical legacy and its continued relevance. mis quarterly, 43(3), 695-719. steigerwald, e., ramírez-castañeda, v., brandt, d. y., báldi, a., shapiro, j. t., bowker, l., & tarvin, r. d. (2022). machine translation tools and a vision for a multilingual future. bioscience, 72(10), 988-998. doi.org/10.1093/biosci/biac062 sun, s. (2013). strategies of translation. the encyclopedia of applied linguistics, 9, 5408-5412. van leeuwen, t. n., moed, h. f., tijssen, r. j. w., visser, m. s., & van raan, a. f. j. (2001). language biases in the coverage of the science citation index and its consequences for international comparisons of national research performance. scientometrics, 51(1), 335346. vaswani, a., shazeer, n., parmar, n., uszkoreit, j., jones, l., gomez, a. n., . . . polosukhin, i. (2017). attention is all you need. paper presented at the 31st conference on neural information processing systems (nips 2017), long beach ca. wierzbicka, a. (2009). overcoming anglocentrism in emotion research. emotion review, 1(1), 21-23. doi.org/10.1177/1754073908097179 wood, d. (1967). the foreign-language problem facing scientists and technologists in the united kingdom--report of a recent survey. journal of documentation, 23, 117-130. australasian journal of information systems burton-jones et al. 2025, vol 29, provocation this article is not just in english 15 xu, y., hu, l., zhao, j. l., qiu, z., ye, y., & gu, h. (2024). a survey on multilingual large language models: corpora, alignment, and bias. frontiers in computer science, 11-41. copyright copyright © 2025 burton-jones et al. this is an open-access article licensed under a creative commons attribution-non-commercial 4.0 australia license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: doi.org/10.3127/ajis.v29.5875 greek αυτό το άρθρο δεν είναι μόνο στα αγγλικά: προάγοντας μια πιο συμπεριληπτική και επιδραστική επιστήμη μέσω μετάφρασης με τεχνητή νοημοσύνη περίληψη η γλωσσική ποικιλομορφία χαρακτηρίζει την ανθρωπότητα, ενώ η επιστήμη παραμένει μονογλωσσική. η επιτυχία στον ακαδημαϊκό χώρο απαιτεί καλή γνώση της αγγλικής γλώσσας. κάθε σημαντικό συνέδριο, κάθε σημαντικό περιοδικό – ακόμη και αυτό – το υποθέτει. η προτίμηση προς την αγγλική γλώσσα στην επιστήμη είναι τόσο ισχυρή ώστε θεωρείται δεδομένη από τους περισσότερους επιστήμονες και επιστημονικές ενώσεις, χωρίς να αποτελεί αντικείμενο επαρκούς συζήτησης ή αντιμετώπισης. αυτό είναι άδικο και οδηγεί σε σημαντικές δαπάνες και απώλεια ευκαιριών. αυτό είναι επίσης αδικαιολόγητο. τα εργαλεία μετάφρασης με τεχνητή νοημοσύνη (ai) βελτιώνονται πολύ γρήγορα, δίνοντάς μας τη δυνατότητα να φανταστούμε μια πολυγλωσσική επιστήμη. καλούμε τους αναγνώστες να αναλογιστούν: πώς μπορούμε να αξιοποιήσουμε τα εργαλεία μετάφρασης ai για μια πιο επιδραστική και συμπεριληπτική επιστήμη; αυτή είναι μια πρόκληση ιδανικά κατάλληλη για τους μελετητές των πληροφοριακών συστημάτων, καθώς αφορά το σχεδιασμό κοινωνικοτεχνικών συστημάτων και πρακτικών για ένα καλύτερο μέλλον. για να αποδείξουμε τη δυνατότητα υλοποίησης, το άρθρο αυτό υποβλήθηκε σε μια διαδικασία πολυγλωσσικής αξιολόγησης και δημοσιεύεται σε πέντε γλώσσες, όλα επιτυγχανόμενα με τη χρήση εργαλείων μετάφρασης ai. australasian journal of information systems burton-jones et al. 2025, vol 29, provocation this article is not just in english 16 1 το πρόβλημα της γλωσσικής μεροληψίας ένα από τα πιο εμφανή και λιγότερο συζητημένα χαρακτηριστικά του australasian journal of information systems (ajis) είναι ότι είναι αγγλόφωνο περιοδικό. αυτό δεν συζητείται ή αμφισβητείται επειδή η επιλογή θεωρείται αυτονόητη. όλα τα κορυφαία συνέδρια και περιοδικά στον τομέα των πληροφοριακών συστημάτων (is) είναι στα αγγλικά, όπως και σε κάθε άλλη επιστημονική πειθαρχία. η επίτευξη επιστημονικής επιτυχίας προϋποθέτει άριστη γνώση της αγγλικής γλώσσας. αυτό είναι άδικο, καθώς η πλειονότητα του παγκόσμιου πληθυσμού δεν μιλάει αγγλικά. από τα 8 δισεκατομμύρια ανθρώπους στη γη, εκτιμάται ότι μόνο 1,5 δισεκατομμύρια μιλούν αγγλικά, και μόνο το 25% από αυτούς μιλούν αγγλικά ως πρώτη γλώσσα (icls, 2024). ο στόχος αυτής της πρόκλησης είναι να αναδείξει το πρόβλημα της «γλωσσικής προκατάληψης» στον τομέα των πληροφοριακών συστημάτων (is) και να καλέσει σε δράσεις για την αντιμετώπισή του. περιγράφουμε πώς ξεκινάμε αυτή την εργασία, ως μέλη μιας ομάδας εργασίας της association for information systems (ais), και καλούμε για δράση σε επίπεδο κοινότητας. είναι δύσκολο να απαλλαγούμε από την αίσθηση ότι είμαστε πίσω από άλλους τομείς στην ευαισθητοποίηση για τη γλωσσική προκατάληψη. το θέμα τέθηκε στην ιατρική τη δεκαετία του 1990 (egger et al., 1997) και έκτοτε έχει τεθεί σε πολλούς τομείς, όπως η έρευνα συναισθημάτων (wierzbicka, 2009), η οικολογία (clavero, 2011), η γλωσσολογία (levisen, 2019), η διοίκηση (pudelko & tenzer, 2019), το δίκαιο (ammann, 2022), οι γνωστικές επιστήμες (blasi, henrich, adamou, kemmerer, & majid, 2022), η διατήρηση (amano & berdejo-espinola, 2024), οι επιστήμες της γης (bampi, pires-oliveira, loyolabartra, & limaribeiro, 2024), η κρίσιμη φροντίδα (gonzález-dambrauskas, salluh, machado, & rotta, 2024) και η ορθοδοντική (mheissen, spineli, daraqel, & alsafadi, 2024). έχει επίσης τεθεί σε συζητήσεις σε όλη την επιστήμη για το μέλλον των ακαδημαϊκών εκδόσεων (ahmed et al., 2023; amano, rojas, boum ii, calvo, & misra, 2021; arenas-castro et al., 2024; editorial, 2023). όπως οι περισσότερες προκαταλήψεις, η γλωσσική προκατάληψη είναι δαπανηρή. όπως αναφέρουν οι amano et al. (2023), οι μη φυσικοί ομιλητές της αγγλικής γλώσσας πρέπει να αφιερώνουν περισσότερο χρόνο για να διαβάζουν, να γράφουν και να αναθεωρούν εργασίες, είναι πιο πιθανό να απορριφθούν οι εργασίες τους για λόγους που σχετίζονται με τη γλώσσα και αντιμετωπίζουν εμπόδια στη συμμετοχή σε συνέδρια. η γλωσσική προκατάληψη οδηγεί σε σημαντικές δαπάνες και απώλεια ευκαιριών για νέους ερευνητές (hohti & truman, 2021), μειώνει τη διαδικασία αξιολόγησης από ομοτίμους (politzer-ahles, girolamo, & ghali, 2020), δημιουργεί προκατάληψη στις βιβλιογραφικές ανασκοπήσεις (nunez & amano, 2021), στις συστηματικές ανασκοπήσεις (hannah, haddaway, fuller, & amano, 2023) και στις μετα-αναλύσεις (konno et al., 2020) και παραποιεί τις εθνικές συγκρίσεις έρευνας (van leeuwen, moed, tijssen, visser, & van raan, 2001). η γλωσσική προκατάληψη είναι παρούσα ακόμη και στις αναζητήσεις στο google scholar, καθώς προτιμά αγγλόφωνα άρθρα (rovira, codina, & lopezosa, 2021). ως συστημική, μακροχρόνια προκατάληψη, η γλωσσική προκατάληψη μπορεί επίσης να παρατηρηθεί σε εθνικές πρακτικές, όπως η κυριαρχία της διδασκαλίας της αγγλικής γλώσσας στα australasian journal of information systems burton-jones et al. 2025, vol 29, provocation this article is not just in english 17 σχολεία σε όλο τον κόσμο, και σε μεγάλες βιομηχανίες, όπως η παγκόσμια βιομηχανία της αγγλικής ως δεύτερης γλώσσας (esl)7 αξίας 10-11 δισεκατομμυρίων δολαρίων ηπα. ενώ επιδιώκουμε να αναδείξουμε τη γλωσσική προκατάληψη που επηρεάζει την επιστήμη, δεν αγνοούμε τα οφέλη της παγκόσμιας αγγλικής γλώσσας. δεδομένου ότι η γλωσσική ποικιλότητα φυσικά εμποδίζει την επικοινωνία (π.χ., όπως απεικονίζεται στον διάσημο πύργο της βαβέλ), η ενοποίηση σε μία γλώσσα (είτε αγγλικά είτε οποιαδήποτε άλλη γλώσσα) μπορεί να διευκολύνει την πρόοδο σε πολλούς τομείς (john & özgür, 2021). όπως σημείωσε το british council (2013, σελ. 3) «χάρη στο διαδίκτυο, την άνοδο των κοινωνικών μέσων, την ταχύτητα και την εξάπλωση της παγκόσμιας τεχνολογίας επικοινωνιών και την ολοένα και πιο παγκοσμιοποιημένη και αλληλεξαρτώμενη παγκόσμια οικονομία, η αγγλική γλώσσα επιτρέπει πλέον τη γρήγορη ανταλλαγή ιδεών και καινοτομίας σε όλο τον κόσμο, και τη δημιουργία μιας νέας υπερεθνικής αγοράς γνώσης και ιδεών.» ωστόσο, ενώ αναγνωρίζουμε τα πλεονεκτήματα της εξάπλωσης της αγγλικής γλώσσας (και αρκετά μέλη της ομάδας συγγραφέων μας έχουν απολαύσει αυτά τα οφέλη, ως φυσικοί ομιλητές της αγγλικής), ο στόχος μας εδώ είναι να αναδείξουμε το κόστος, επειδή είναι σημαντικό αλλά γενικά δεν αναγνωρίζεται και θεωρείται δεδομένο (mélitz, 2007; salomone, 2022). 2 η υπόσχεση των εργαλείων μετάφρασης ai παρά την επικράτηση και το κόστος της γλωσσικής προκατάληψης, δεν πρέπει πλέον να θεωρείται αυτονόητη, καθώς τα εργαλεία μετάφρασης τεχνητής νοημοσύνης (ai) γίνονται πολύ αποτελεσματικά και γρήγορα. είναι σημαντικό να τονιστεί η ιστορική σημασία αυτής της στιγμής, καθώς η γλωσσική προκατάληψη είναι τόσο παλιά όσο και η επιστήμη. άλλωστε, η επιστήμη πάντα διεξαγόταν σε μία γλώσσα ή άλλη, και για να συμμετάσχει κανείς (να διαβάσει, να μιλήσει, να γράψει), έπρεπε να γνωρίζει τη γλώσσα (gordin, 2015). με περισσότερες από 7.000 γλώσσες στον κόσμο (anderson, 2010), οι επιστήμονες δεν μπορούσαν να διαβάζουν ή να γράφουν σε κάθε γλώσσα. για χιλιετίες, απλά είχε νόημα να διεξάγεται η επιστήμη στη γλώσσα της εποχής. οι κυρίαρχες γλώσσες σε διαφορετικές εποχές της ιστορίας περιλάμβαναν την αιγυπτιακή, την ελληνική, τη λατινική, την κινέζικη και διάφορες ευρωπαϊκές γλώσσες, αλλά η αγγλική κυριαρχεί από τον β' παγκόσμιο πόλεμο (gordin 2015). παρά τις σύντομες (και αποτυχημένες) προσπάθειες να αναπτυχθεί μια παγκόσμια γλώσσα (π.χ., εσπεράντο) (patterson & huff, 1999), η μετάφραση υπήρξε η μόνη λύση για την αντιμετώπιση των γλωσσικών διαφορών διαχρονικά. ωστόσο, παρά το γεγονός ότι υπάρχει ως πεδίο για πάνω από 2.000 χρόνια (dynamic_language, 2024), η μετάφραση, όσο μπορούμε να διακρίνουμε, δεν είχε ποτέ ευρεία αποδοχή στον επιστημονικό χώρο, κυρίως επειδή η διαδικασία της μετάφρασης είναι δαπανηρή και χρονοβόρα (wood, 1967). κατά τη διάρκεια της ιστορίας, διάφορα περιοδικά έχουν εκδοθεί σε πολλές γλώσσες, και ακόμα και σήμερα υπάρχουν περίπου 500 πολύγλωσσα περιοδικά8, αλλά αυτά αποτελούν εξαιρέσεις που επιβεβαιώνουν τον κανόνα. όπως γράφει χαρακτηριστικά ο gordin (2015, σ. 3), «αν σας ενδιαφέρει πώς θα ήταν να ζείτε σε έναν κόσμο με μία γλώσσα επικοινωνίας, έναν κόσμο χωρίς βαβέλ, θα πρέπει να κοιτάξετε τους φυσικούς 7 https://www.businessresearchinsights.com/market-reports/esl-english-as-a-second-language-market-117478; https://www.englishaustralia.com.au/our-sector/understanding-the-sector 8 https://en.wikipedia.org/wiki/category:multilingual_journals https://www.businessresearchinsights.com/market-reports/esl-english-as-a-second-language-market-117478 https://www.englishaustralia.com.au/our-sector/understanding-the-sector https://en.wikipedia.org/wiki/category:multilingual_journals australasian journal of information systems burton-jones et al. 2025, vol 29, provocation this article is not just in english 18 επιστήμονες. αυτοί έρχονται από εκεί». η συντριπτική πλειονότητα των περιοδικών είναι μόνο στα αγγλικά (gordin, 2015), και τα περισσότερα πολύγλωσσα περιοδικά είναι απλά δίγλωσσα περιοδικά μεταξύ αγγλικών και άλλης γλώσσας, και τα περισσότερα από αυτά είναι μόνο δίγλωσσα λόγω εθνικών νόμων που το απαιτούν (π.χ., περιοδικά που εκδίδονται στον καναδά, ο οποίος είναι δίγλωσση χώρα). η ικανότητα των εργαλείων μετάφρασης ai να αντιμετωπίσουν τη γλωσσική προκατάληψη στην επιστήμη έχει εμφανιστεί μόλις πρόσφατα (berdejo-espinola & amano, 2023; crotty, 2024; steigerwald et al., 2022), προερχόμενη από δύο μακροπρόθεσμες μεταμορφώσεις. η μία ήταν η σταδιακή ψηφιακή μεταμόρφωση της ίδιας της έρευνας. όχι πολύ καιρό πριν, πολλές ερευνητικές εργασίες ήταν χειροκίνητες. οι ερευνητές πήγαιναν σε βιβλιοθήκες, διάβαζαν και σημείωναν φυσικά έγγραφα, έστελναν έγγραφα και κριτικές μέσω ταχυδρομείου, ακόμα και έκαναν υπολογισμούς και έγραφαν έγγραφα με το χέρι. τώρα, σχεδόν όλες οι εργασίες είναι διαδικτυακές: αναζήτηση, ανάγνωση, γραφή, σχολιασμός, συνεργασία, ανάλυση, αναθεώρηση, υποβολή, δημοσίευση. μια βαθιά συνέπεια αυτής της μεταμόρφωσης είναι ότι μεγάλο μέρος της επιστήμης τώρα περιλαμβάνει γλώσσα σε ψηφιακές πλατφόρμες. δηλαδή, τα επιστημονικά έγγραφα γράφονται, αναθεωρούνται, δημοσιεύονται, διαβάζονται και αναζητούνται σε μια γλώσσα (σήμερα σχεδόν πάντα στα αγγλικά), και όλα αυτά συμβαίνουν σε μεγάλες, διασυνδεδεμένες ψηφιακές πλατφόρμες. ορισμένες από αυτές τις πλατφόρμες είναι τώρα τεράστιες, π.χ., το scopus φιλοξενεί πάνω από 90 εκατομμύρια αρχεία9 και το google scholar αναζητά σε 300 εκατομμύρια αρχεία (martin-martin, thelwall, orduna-malea, & lopez-cozar, 2021). και εδώ έρχεται η δεύτερη μεταμόρφωση – η μεταμόρφωση της τεχνητής νοημοσύνης. παρόλο που τα γλωσσικά μοντέλα στην τεχνητή νοημοσύνη έχουν ιστορία που χρονολογείται από τη δεκαετία του 1950, η ταχεία ανάπτυξη μεγάλων γλωσσικών μοντέλων κατέστη δυνατή από τη μαζική αύξηση των διαδικτυακών δεδομένων για μάθηση (naik, naik, & naik, 2024) μαζί με νέες αρχιτεκτονικές, ειδικά την αρχιτεκτονική μετασχηματιστών, που μπορούν να αξιοποιήσουν τέτοια δεδομένα (vaswani et al., 2017). τα τελευταία πέντε χρόνια, αυτές οι δύο μεταμορφώσεις επέτρεψαν την εμφάνιση νέων μορφών ai, επιτρέποντας πρόοδο που φαινόταν αδύνατη πριν από λίγο καιρό. και αυτή η πρόοδος είναι ιδιαίτερα γρήγορη στη μετάφραση γλώσσας. οι εργασίες μετάφρασης γλώσσας ήταν οι πρώτες εργασίες που δοκιμάστηκαν στην αρχιτεκτονική μετασχηματιστών και έτσι είναι ιδανικά κατάλληλες για αυτήν (vaswani et al., 2017). δεν ισχυριζόμαστε ότι η μετάφραση ai μπορεί να λύσει πλήρως και γρήγορα τη γλωσσική προκατάληψη. υπάρχουν πολλές πολυπλοκότητες (amano and berdejo-espinola 2024). αλλά η ικανότητα είναι τόσο ισχυρή που έχουμε την υποχρέωση να την επιδιώξουμε. από την οπτική γωνία μιας επιστημονικής ένωσης, όπως η association for information systems, θα ήταν αμέλεια να μην προσπαθήσουμε να μειώσουμε μια διαδεδομένη και δαπανηρή συστηματική προκατάληψη που αντιμετωπίζουν τα μέλη του επιστημονικού μας πεδίου και εξίσου αμέλεια αν δεν προσπαθούσαμε να αξιοποιήσουμε τις νέες ευκαιρίες για ένταξη και επιρροή που θα μπορούσε να προσφέρει η μετάφραση ai. γι' αυτό 9 https://assets.ctfassets.net/o78em1y1w4i4/76uvtuilkssqe4h4h1pzkq/e5ff8eef9694f216a5c29e2ebf80fa51/fa st-facts-about-elsevier.pdf https://assets.ctfassets.net/o78em1y1w4i4/76uvtuilkssqe4h4h1pzkq/e5ff8eef9694f216a5c29e2ebf80fa51/fast-facts-about-elsevier.pdf https://assets.ctfassets.net/o78em1y1w4i4/76uvtuilkssqe4h4h1pzkq/e5ff8eef9694f216a5c29e2ebf80fa51/fast-facts-about-elsevier.pdf australasian journal of information systems burton-jones et al. 2025, vol 29, provocation this article is not just in english 19 δημιουργήσαμε την ομάδα εργασίας μας – την ομάδα εργασίας της association for information systems για τη μετάφραση ai για ένταξη και επιρροή στην επιστήμη. 3 συγκροτήματα υπάρχουν πολλές περιπλοκές στη χρήση της μετάφρασης με ai στην επιστήμη. αναδεικνύουμε έξι κατηγορίες. πρώτον, υπάρχουν περιπλοκές που εξαρτώνται από το πώς ορίζουμε το πρόβλημα. η απλούστερη προσέγγιση είναι να επικεντρωθούμε στη μετάφραση ενός μόνο άρθρου από μία γλώσσα σε άλλη, αλλά ακόμη και αυτή η προσέγγιση εγείρει ερωτήματα. για παράδειγμα, θα πρέπει να μεταφράζονται μόνο τα αποδεκτά άρθρα ή και τα υποβαλλόμενα; η πρώτη επιλογή θα βοηθούσε την επιστήμη ενισχύοντας την επίδραση – αυξάνοντας την αναγνωσιμότητα ενός άρθρου – ενώ η δεύτερη θα υποστήριζε επίσης την ένταξη – διευκολύνοντας περισσότερους συγγραφείς και κριτές να συντάσσουν στη μητρική τους γλώσσα. αν και η δεύτερη επιλογή έχει μεγαλύτερα οφέλη, απαιτεί επίσης περισσότερη εργασία, καθώς απαιτείται να μεταφράσουμε τόσο το άρθρο όσο και τις κριτικές του, και επίσης να δημιουργήσουμε μια διαδικασία εργασίας για συγγραφείς και κριτές ώστε να ελέγχουν την ποιότητα των μεταφράσεων κατά τη διαδικασία της κριτικής χωρίς να τους επιβαρύνουμε υπερβολικά. μόλις ένα άρθρο γίνει αποδεκτό/δημοσιευτεί, ένα άλλο ερώτημα είναι αν επικεντρωνόμαστε στη μετάφραση σε μία μόνο γλώσσα ή σε πολλές, και αν ο συγγραφέας ή ο αναγνώστης επιλέγει τη γλώσσα. αν οι αναγνώστες επιλέγουν, πρέπει οι μεταφράσεις να είναι προετοιμασμένες εκ των προτέρων (π.χ., σε έναν μικρό αριθμό κοινών γλωσσών) και οι αναγνώστες να επιλέγουν τη γλώσσα που προτιμούν από αυτή τη λίστα, ή πρέπει οι αναγνώστες να μπορούν να ζητούν μετάφραση στη γλώσσα της επιλογής τους (είτε από μια σταθερή λίστα, είτε σε έναν ιδανικό κόσμο, από οποιαδήποτε γλώσσα) η οποία στη συνέχεια παράγεται σε πραγματικό χρόνο από την πλατφόρμα δημοσίευσης κατόπιν αιτήματος; η τελευταία επιλογή θα επέτρεπε ένα ευρύτερο φάσμα γλωσσών, ενώ η πρώτη θα επέτρεπε ένα στάδιο επαλήθευσης της ποιότητας της μετάφρασης. αν επιλεγεί η τελευταία, είναι πιθανό να εισαχθούν μικρές αλλαγές στη σημασία σε ορισμένες μεταφράσεις χωρίς να το γνωρίζουν οι συγγραφείς, δημιουργώντας προβλήματα στις μελλοντικές αναφορές αυτού του έργου. τα άρθρα που δημοσιεύονται με την επιλογή μετάφρασης κατά παραγγελία και σε πραγματικό χρόνο μπορεί επομένως να απαιτούν μια δήλωση αποποίησης ευθυνών που αναγνωρίζει την πιθανότητα σφαλμάτων. ένα άλλο ερώτημα σχετικά με το εύρος είναι αν κοιτάμε πέρα από τη μετάφραση ενός μόνο άρθρου και εξετάζουμε τη μετάφραση μιας ολόκληρης συλλογής/πλατφόρμας άρθρων, όπως η ολόκληρη elibrary της association for information systems, η οποία περιέχει πάνω από 65,000 άρθρα; η επέκταση του εύρους σε μια βιβλιοθήκη όχι μόνο αυξάνει τον αριθμό των άρθρων προς μετάφραση, αλλά φέρνει και επιπλέον σκέψεις. για παράδειγμα, μπορεί να διευρύνει το χρονικό πλαίσιο της μετάφρασης, επειδή μια βιβλιοθήκη μπορεί να έχει άρθρα δημοσιευμένα σε πολλές δεκαετίες, και έτσι η διαδικασία μετάφρασης μπορεί να χρειαστεί να λάβει υπόψη τις αλλαγές στο πολιτισμικό πλαίσιο και το λεξιλόγιο ενός πεδίου με την πάροδο του χρόνου. ομοίως, μπορεί να απαιτεί την ανάπτυξη συμπληρωματικών εργαλείων, όπως ένα εργαλείο αναζήτησης πολλαπλών γλωσσών για αναζήτηση σε όλη τη βιβλιοθήκη σε πολλές γλώσσες. australasian journal of information systems burton-jones et al. 2025, vol 29, provocation this article is not just in english 20 ένα τελευταίο ερώτημα σχετικά με το εύρος αφορά τις δυνατότητες του ai. δεδομένου ότι η ίδια τεχνολογία ai που εκτελεί μεταφράσεις προσφέρει σχετικές δυνατότητες, πρέπει να χρησιμοποιήσουμε και αυτές τις δυνατότητες; για παράδειγμα, πρέπει να εξετάσουμε πέρα από τη μετάφραση των άρθρων για να εξετάσουμε την παραγωγή περιλήψεων άρθρων σε διάφορες γλώσσες (π.χ., απλές περιλήψεις για στελέχη ή φοιτητές); ομοίως, πρέπει να εξετάσουμε βελτιώσεις στη γλώσσα, όχι μόνο μεταφράσεις, π.χ., προσφέροντας προτάσεις βελτίωσης ενός κειμένου, όχι μόνο μεταφράζοντας το κείμενο «ως έχει»; αν εξετάσουμε το τελευταίο, θα μπορούσαμε να επαναπροσδιορίσουμε το πρόβλημα από απλή «μετάφραση» σε «νοηματική μετάφραση», όπου η μετάφραση είναι μόνο ένα στοιχείο της ερμηνείας της σημασίας και άλλα στοιχεία θα μπορούσαν να περιλαμβάνουν βελτιώσεις στη γραμματική, τη ροή και το στυλ. όπως βλέπετε, το εύρος του προβλήματος μπορεί να ποικίλει σημαντικά. σε αυτό το στάδιο, η ομάδα εργασίας μας υιοθετεί ένα μεσαίο εύρος. επιδιώκουμε να υποστηρίξουμε ολόκληρη τη διαδικασία κριτικής – από την υποβολή έως την αποδοχή. επίσης, επιδιώκουμε να υποστηρίξουμε μεταφράσεις σε πολλές γλώσσες. ωστόσο, περιορίζουμε τις προσπάθειές μας σε 10 κύριες γλώσσες αρχικά (bahasa indonesia, urdu, ελληνικά, γαλλικά, μαλαϊκά, κορεατικά, μανδαρινικά, γερμανικά, ισπανικά και πορτογαλικά) και εστιάζουμε στην ανάπτυξη προετοιμασμένων μεταφράσεων που παρέχουν ένα στάδιο επαλήθευσης ποιότητας με ανθρώπινη παρέμβαση αντί να επιτρέπουμε μεταφράσεις κατά παραγγελία και σε πραγματικό χρόνο. συμπεριλαμβάνουμε την παραγωγή περιλήψεων άρθρων στο εύρος μας, αλλά επικεντρωνόμαστε μόνο στη μετάφραση μεμονωμένων άρθρων αντί για ολόκληρη βιβλιοθήκη/πλατφόρμα, και δεν εξετάζουμε ευρύτερες πτυχές της «νοηματικής μετάφρασης» όπως βελτιώσεις στη γλώσσα σε αυτή την πρώτη φάση. εκτός από αυτές τις περιπλοκές που προκύπτουν λόγω ερωτημάτων σχετικά με το εύρος, ένα άλλο σύνολο περιπλοκών προκύπτει από τις πρακτικές του περιβάλλοντος δημοσίευσης. για παράδειγμα, πώς πρέπει να χειριστούμε τους ψηφιακούς αναγνωριστές αντικειμένων (dois); σήμερα, κάθε δημοσιευμένο άρθρο πρέπει να έχει έναν μοναδικό doi, επειδή αυτό επιτρέπει στις μηχανές αναζήτησης και στους συγκεντρωτές να εντοπίζουν και να αναγνωρίζουν αναφορές για κάθε άρθρο. ωστόσο, αν δημιουργήσουμε πολλαπλές μεταφράσεις ενός άρθρου, θα περιλαμβάνονται αυτές στο ίδιο έγγραφο (π.χ., ιστοσελίδα, pdf) με τον ίδιο doi, και θα συγκεντρώνονται οι αναφορές του άρθρου ανεξάρτητα από την έκδοση (μετάφραση) που αναφέρεται; μια δυσκολία στην απάντηση αυτών των ερωτημάτων είναι ότι καμία οντότητα (επιστημονική ένωση, εκδότης, μεγάλοι τεχνολογικοί οργανισμοί) δεν έχει τον έλεγχο ολόκληρου του οικοσυστήματος δημοσίευσης, και είναι δύσκολο να προβλέψουμε πώς θα εξελιχθεί όλο αυτό. επομένως, πρέπει να επιλέξουμε πρακτικές μετάφρασης που θα ευθυγραμμίζονται κάπως με τις υπάρχουσες πρακτικές ενώ θα είναι αρκετά ευέλικτες για να προσαρμοστούν εύκολα. παρόμοιες πρακτικές περιπλοκές προκύπτουν με τα πνευματικά δικαιώματα. μια μετάφραση μπορεί να θεωρηθεί ως «παράγωγο έργο» και επομένως πρέπει να συμμορφώνεται με τους περιορισμούς πνευματικών δικαιωμάτων για τέτοια έργα. ένα άλλο πρακτικό ζήτημα είναι η λογοκλοπή. ειρωνικά, η διευκόλυνση της μετάφρασης μπορεί επίσης να διευκολύνει τη λογοκλοπή. αυτό συμβαίνει επειδή η λογοκλοπή συνήθως προσδιορίζεται και ανιχνεύεται από τη χρήση της ίδιας ακολουθίας λέξεων σε australasian journal of information systems burton-jones et al. 2025, vol 29, provocation this article is not just in english 21 ένα άρθρο, δηλαδή, αντιγραφή λέξη προς λέξη. η μετάφραση δεν χρειάζεται να είναι λέξη προς λέξη, αλλά μπορεί να είναι «μεταφορά νοήματος» (μεταφέροντας το ίδιο νόημα αντί για τις ίδιες λέξεις) (sun, 2013). αν ένας λογοκλόπος λογοκλοπήσει ένα άρθρο δημιουργώντας μια μετάφραση χρησιμοποιώντας μεταφορά νοήματος, δεν είναι σαφές αν τα τρέχοντα εργαλεία ανίχνευσης λογοκλοπής θα το ανιχνεύσουν. τέτοια εργαλεία μπορεί να χρειαστεί να αναθεωρηθούν για να ελέγξουν αυτόν τον κίνδυνο. ένα τρίτο σύνολο περιπλοκών προκύπτει από τα χαρακτηριστικά του ai. αν και η ταχεία πρόοδος του ai μας ώθησε να επιδιώξουμε αυτό το έργο, δεν είναι ακόμη τέλειο. οι λανθασμένες αντιλήψεις εξακολουθούν να υφίστανται (ji et al., 2023), και η απόδοση της μετάφρασης με ai ποικίλλει ανάλογα με την ποσότητα των δεδομένων εκπαίδευσης σε κάθε γλώσσα (xu et al., 2024). λόγω της απόλυτης κυριαρχίας της αγγλικής γλώσσας στο διαδίκτυο, τα μοντέλα gen ai μπορεί να ενσωματώνουν όχι μόνο προκατάληψη υπέρ της αγγλικής γλώσσας αλλά και πολιτισμική προκατάληψη (luo & puett, 2024). καλούμε τους ερευνητές σε όλο το πεδίο μας να δημιουργήσουν απτά κοινωνικοτεχνικά αντικείμενα και πρακτικές για να αντιμετωπίσουν αυτά τα ζητήματα. κατά την εκτέλεση αυτής της εργασίας, οι ερευνητές δεν πρέπει να παρασυρθούν μόνο από την τεχνολογική ντετερμινιστική ή κοινωνική ντετερμινιστική προσέγγιση – μια κοινωνικοτεχνική άποψη είναι κρίσιμη. δηλαδή, όπως και με οποιαδήποτε άλλη τεχνολογία, η επιτυχία της μετάφρασης με ai θα εξαρτηθεί όχι μόνο από την τεχνολογία αλλά και από το πώς οι χρήστες (συγγραφείς, επιμελητές, κριτές, αναγνώστες) την αντιλαμβάνονται και τη χρησιμοποιούν. για παράδειγμα, οι ανθρώπινες τάσεις όπως η αντίσταση στους αλγορίθμους και η δυσπιστία θα πρέπει αναπόφευκτα να ληφθούν υπόψη (asscher & glikson, 2023). η εκτέλεση μοντέλων ai φέρει επίσης οικονομικό κόστος, εγείροντας το ερώτημα ποιος θα επωμιστεί αυτό το κόστος. ένας από τους λόγους που επιδιώκουμε αυτό ως ομάδα εργασίας της ais είναι ότι εξετάζουμε αν και πώς μια επιστημονική ένωση θα πρέπει να αναλάβει αυτό το κόστος για τα μέλη της. ένα τέταρτο σύνολο περιπλοκών προκύπτει λόγω των αποχρώσεων στη γραπτή γλώσσα που τα τρέχοντα μοντέλα μετάφρασης με ai μπορεί να παραλείψουν. για παράδειγμα, στη διαδικασία κριτικής αυτού του άρθρου, ένας από τους κριτές μας παρατήρησε ελλείψεις στις μεταφράσεις των ακρωνυμίων. για παράδειγμα, στην ισπανία, οι συγγραφείς γενικά χρησιμοποιούν την ισπανική έκδοση του ακρωνυμίου (ia), αλλά στην ελλάδα, οι συγγραφείς είναι εξίσου εξοικειωμένοι με τη χρήση της αγγλικής έκδοσης (ai) ή της ελληνικής έκδοσης (tn), και ο συγγραφέας θα πρέπει να χρησιμοποιεί μόνο μία συνεκτικά. η μετάφρασή μας απέτυχε να το κάνει αυτό. εν τω μεταξύ, οι άλλοι κριτές μας παρατήρησαν ότι οι μεταφράσεις μας μερικές φορές ακούγονταν υπερβολικά «άκαμπτες», «ισχυρές» ή «επίσημες» και ότι θα έπρεπε να είχαμε υιοθετήσει έναν «πιο ήπιο» ή πιο «φυσικό» τόνο. αυτό μπορεί να οφείλεται στο ότι η μετάφραση με ai υιοθέτησε αυτόματα έναν υπερβολικά επίσημο τόνο, τον οποίο δεν συνειδητοποιήσαμε εκ των προτέρων. για να αντιμετωπίσουμε αυτές τις προκλήσεις, βελτιώνουμε ενεργά την προσέγγιση της μετάφρασής μας. συγκεκριμένα, έχουμε ενσωματώσει ανατροφοδότηση από τους κριτές αυτού του άρθρου, ειδικά όσον αφορά την αξιολόγηση της πολιτισμικής φυσικότητας και του τόνου. ως αποτέλεσμα, η προσέγγιση της μετάφρασής μας τώρα δίνει μεγαλύτερη προσοχή σε αυτές τις πτυχές. για παράδειγμα, τα εσωτερικά αρχεία εργαλείων έχουν καταγράψει προτάσεις εμπνευσμένες από κριτές όπως: «η φράση '...' είναι αρκετά άμεση. σκεφτείτε να την επαναδιατυπώσετε σε έναν πιο ακαδημαϊκό τόνο, όπως '...'» ή «η φράση '...' είναι κάπως κυριολεκτική. μια πιο κατάλληλη ακαδημαϊκή australasian journal of information systems burton-jones et al. 2025, vol 29, provocation this article is not just in english 22 διατύπωση θα μπορούσε να είναι '...'». παρά τη ραγδαία αυξανόμενη δύναμη της μετάφρασης με ai, είναι δύσκολο να φανταστεί κανείς ότι αυτά τα εργαλεία θα προσφέρουν τέλεια μετάφραση σε κάθε γλώσσα και προσαρμοσμένη σε κάθε άρθρο, αλλά σκοπεύουμε να αντιμετωπίσουμε όσο το δυνατόν περισσότερα από αυτά τα προβλήματα (π.χ., τροποποιώντας την αρχιτεκτονική του ai μοντέλου για να χρησιμοποιεί το πλαίσιο πιο αποτελεσματικά). ένα πέμπτο σύνολο περιπλοκών προκύπτει από τα ιδρύματα της επιστήμης που θα μπορούσαν να ενισχύσουν το υφιστάμενο καθεστώς και να εμποδίσουν την εμφάνιση της πολύγλωσσης επιστήμης. για παράδειγμα, όταν συζητήσαμε αυτή την πρωτοβουλία με επιτυχημένους ερευνητές που δεν είναι φυσικοί ομιλητές της αγγλικής γλώσσας, εκπλαγήκαμε πόσοι είπαν ότι δεν θα εκμεταλλεύονταν την ευκαιρία να γράψουν στη μητρική τους γλώσσα επειδή είναι καλύτεροι στο να γράφουν άρθρα στα αγγλικά (επειδή έχουν εκπαιδευτεί όλη τους τη ζωή στα αγγλικά). από μια άποψη αυτό δεν μας ανησυχεί επειδή ο πληθυσμός των μη φυσικών ομιλητών της αγγλικής γλώσσας που θα μπορούσε να ωφεληθεί είναι πολύ μεγαλύτερος από τον μικρό πληθυσμό των μη φυσικών ομιλητών της αγγλικής γλώσσας που έχουν «καταφέρει» στο αγγλοκρατούμενο σύστημα. από την άλλη πλευρά, τα ανώτερα μέλη μιας κοινότητας φυσικά επηρεάζουν τα νεότερα μέλη, και έτσι οι απόψεις τους μπορεί να ενισχύσουν το υφιστάμενο καθεστώς για κάποιο χρονικό διάστημα. για παράδειγμα, έχουμε συνεργαστεί με ανώτερους ερευνητές σε μια ευρωπαϊκή χώρα για αυτή την πρωτοβουλία. παρόλο που υποστηρίζουν έντονα την πρωτοβουλία μας, μας ενθάρρυναν να προχωρήσουμε αργά επειδή είχαν εργαστεί για δεκαετίες για να βελτιώσουν τη συγγραφή στην αγγλική γλώσσα στην κοινότητά τους, και αυτή η προσπάθεια είχε πολύ επιτυχή αποτελέσματα, οδηγώντας σε μια απότομη αύξηση του αριθμού των δημοσιεύσεων σε κορυφαία περιοδικά. το τελευταίο πράγμα που ήθελαν αυτοί οι ερευνητές ήταν να αποσπάσουν από αυτή την επιτυχία. όπως δείχνει αυτό το παράδειγμα, οι προσπάθειες να ενθαρρυνθεί η πολύγλωσση επιστήμη πρέπει να σχεδιάζονται προσεκτικά με γνώση άλλων προϋπαρχουσών πρωτοβουλιών που έχουν σχεδιαστεί γύρω από το υφιστάμενο καθεστώς. τέλος, έχουμε ένα έκτο σύνολο περιπλοκών που είναι πιο φιλοσοφικές. ένας τρόπος να δούμε αυτές είναι να σκεφτούμε ένα άρθρο ως ένα μόνο βήμα στη μακρά διαδικασία μιας ιδέας που σχηματίζεται στο μυαλό ενός συγγραφέα μέχρι να εκτιμηθεί στο μυαλό ενός αναγνώστη. για αρχή, σκεφτείτε έναν συγγραφέα που σκέφτεται μια ιδέα. η υπόθεση sapir-whorf υποδηλώνει ότι οι σκέψεις ενός συγγραφέα διαμορφώνονται από τη γλώσσα του συγγραφέα (lucy, 1998). αν είναι αλήθεια, αυτό υποδηλώνει ότι η γλωσσική προκατάληψη πηγαίνει πίσω μέχρι την αρχή, ακόμη και περιορίζοντας τις ιδέες που μπορούν να σχηματίσουν οι συγγραφείς. μπορούμε να το ονομάσουμε το πρόβλημα της «σύλληψης». τα προβλήματα συνεχίζονται επειδή ακόμη και αν μια γλώσσα μπορεί να χρησιμοποιηθεί για να μεταφέρει μια ιδέα, ένας συγγραφέας μπορεί να εξακολουθεί να μην έχει την ικανότητα στη γλώσσα για να το κάνει. μπορούμε να το ονομάσουμε το πρόβλημα της «διατύπωσης». και ακόμη και αν ένας συγγραφέας έχει τη σωστή γλώσσα για να μεταφέρει μια ιδέα, και τη χρησιμοποιεί επιδέξια, ένας αναγνώστης μπορεί να μην ερμηνεύσει τις λέξεις με τον ίδιο τρόπο. μπορούμε να το ονομάσουμε το πρόβλημα της «ερμηνείας». είμαστε σίγουροι ότι οι αναγνώστες αυτού του άρθρου είναι εξοικειωμένοι με τέτοια προβλήματα επειδή είναι κοινά στην καθημερινή ζωή. συλλογικά, αυτά τα προβλήματα σημαίνουν ότι τα ερευνητικά άρθρα είναι αναπόφευκτα ένα ατελές μέσο μέσω του οποίου μεταφέρονται επιστημονικές ιδέες σε μια κοινότητα μελετητών. οι australasian journal of information systems burton-jones et al. 2025, vol 29, provocation this article is not just in english 23 ατέλειες στη σημασία είναι απλώς εγγενείς στη διαδικασία, ανεξάρτητα από τη γλώσσα που χρησιμοποιείται. σαφώς, τα εργαλεία μετάφρασης με ai δεν μπορούν να λύσουν όλα αυτά τα προβλήματα. ως ένα μόνο παράδειγμα, τα εργαλεία ai μπορούν να συμβάλουν στη μείωση του προβλήματος της σύλληψης διευκολύνοντας τους ανθρώπους να σκέφτονται στη μητρική τους γλώσσα, αλλά μπορεί επίσης να αποτύχουν να το κάνουν επειδή η κυριαρχία της αγγλικής γλώσσας ήταν τόσο ισχυρή για τόσο καιρό που ορισμένες γλώσσες δεν έχουν το απαιτούμενο λεξιλόγιο (barath, 2019). 4 μια ευκαιρία και η προσέγγισή μας ενώ έχουμε απαριθμήσει πολλές σύνθετες προκλήσεις, τις βλέπουμε ως ευκαιρίες για συνεισφορά και όχι ως προβλήματα που πρέπει να αντιμετωπιστούν. για εμάς, υπογραμμίζουν γιατί αυτή είναι μια εξαιρετική πρόκληση για τον τομέα των is, καθώς οι ευκαιρίες και οι πολυπλοκότητες είναι βαθιά κοινωνικοτεχνικής φύσης (sarker, chatterjee, xiao, & elbanna, 2019). όπως αναφέραμε στην αρχή, ο τομέας των is δεν είναι ο πρώτος που συνειδητοποιεί την προκατάληψη της γλώσσας. παρόλο που πολλοί τομείς έχουν ευαισθητοποιηθεί για το πρόβλημα, λίγοι το έχουν αντιμετωπίσει συστηματικά. ως αποτέλεσμα, ο τομέας των is έχει την ευκαιρία να συμβάλει στην πρωτοπορία της επιστήμης μαζί με άλλους κορυφαίους τομείς10. δεδομένου ότι το θέμα συνδέεται τόσο στενά με τη φύση του τομέα μας ως τομέας αφιερωμένος στην κατανόηση και το σχεδιασμό πρακτικών κοινωνικοτεχνικής φύσης για ένα καλύτερο μέλλον πιστεύουμε ότι ο τομέας των is όχι μόνο έχει την ευκαιρία να κάνει τη διαφορά, αλλά έχει και το καθήκον να το κάνει. κινητοποιημένοι από αυτή την ευκαιρία, σχηματίσαμε μια ομάδα εργασίας της ais το 2024. σύμφωνα με κοινά μοντέλα αλλαγής (lewin, 1951), υποθέτουμε ότι θα χρειαστεί σημαντική προσπάθεια για να "ξεπαγώσουμε" τον τομέα αντί να υποθέσουμε ότι ο τομέας μπορεί να αλλάξει αμέσως. για να διευκολύνουμε το "ξεπάγωμα", οργανώσαμε πάνελ και ολομέλειες σε τέσσερα μεγάλα συνέδρια με στόχο την ευαισθητοποίηση και την κινητοποίηση της συζήτησης (burton-jones, 2025; burton-jones, chau, feng, kankanhalli, & vogel, 2024; pouloudi et al., 2024; sarker, ahuja, feng, jarvenpaa, & seidel, 2024). συζητήσαμε επίσης με διάφορα κεφάλαια της ais που εκπροσωπούν διαφορετικές γλωσσικές κοινότητες και εργαστήκαμε στην ανάπτυξη και αρχική δοκιμή εργαλείων μετάφρασης ai. το 2025, συνεχίζουμε τις προσπάθειές μας να ευαισθητοποιήσουμε για το πρόβλημα (όπως μέσω αυτού του άρθρου) ενώ παράλληλα αναλαμβάνουμε έναν πιο ενεργό ρόλο μέσω κύκλων σχεδιασμού, δοκιμών και μάθησης. για παράδειγμα, είμαστε ευγνώμονες στους συντάκτες και τους κριτές του ajis που ήταν πρόθυμοι να δοκιμάσουν μια πολυγλωσσική διαδικασία αναθεώρησης για αυτό το άρθρο, από την οποία μπορούμε να μάθουμε και να βελτιωθούμε. παράλληλα, συμμετέχουμε σε άλλες εργασίες για να παρέχουμε συγκεκριμένα δεδομένα και δοκιμές. για παράδειγμα, δοκιμάσαμε μεταφράσεις ai στο americas conference on information systems (amcis) το 2025 (βλ.: https://aisel.aisnet.org/amcis2025/translated/). εργαζόμαστε επίσης με άλλα συνέδρια το 10 άλλες εξέχουσες φωνές στην πρωτοπορία αυτή περιλαμβάνουν την unesco (https://doi.org/10.54677/mnmh8546) και το συμβούλιο της ευρωπαϊκής ένωσης (https://www.consilium.europa.eu/media/56958/st10126-en22.pdf). australasian journal of information systems burton-jones et al. 2025, vol 29, provocation this article is not just in english 24 2025 σε διαφορετικές περιπτώσεις χρήσης μετάφρασης11. εστιάζουμε ιδιαίτερα σε τρεις περιπτώσεις χρήσης: (1) μετάφραση ai στη διαδικασία αναθεώρησης; (2) μετάφραση ai αποδεκτών άρθρων; και (3) σύνταξη περιλήψεων άρθρων σε διαφορετικές γλώσσες για τη διάδοση του αντίκτυπου. θα διεξάγουμε επίσης έρευνες, συνεντεύξεις και ομάδες εστίασης για να μάθουμε πώς η προκατάληψη της γλώσσας επηρεάζει τους μελετητές στον τομέα των is και τις πιθανές επιπτώσεις των εργαλείων ai μετάφρασης. συμμετέχουμε επίσης με μελετητές σε άλλους τομείς για να υποστηρίξουμε το ευρύτερο κίνημα σε όλη την επιστήμη (amano, bowker, & burton-jones, 2025). στην επιδίωξη αυτών των βημάτων, αναγνωρίζουμε ότι θα αντιμετωπίσουμε πολλές προκλήσεις και ότι υπάρχουν πολλές ακόμη ευκαιρίες. μπορούμε επίσης να μάθουμε μαθήματα και να πάρουμε ενθάρρυνση από άλλα κινήματα σε όλη την επιστήμη, όπως το κίνημα της ανοιχτής επιστήμης (ferguson et al., 2023), που είναι παρόμοια κινητοποιημένα προς μεγαλύτερη συμπερίληψη και αντίκτυπο στην επιστήμη. όπως κάθε προσπάθεια αλλαγής, θα κάνουμε λάθη και θα πρέπει να τροποποιήσουμε τα σχέδιά μας. παρ' όλα αυτά, διατηρούμε τον ενθουσιασμό μας για τις δυνατότητες, και ελπίζουμε ότι και άλλοι ερευνητές των is θα είναι ενθουσιασμένοι επίσης, και ότι μαζί μπορούμε να κάνουμε μια ουσιαστική αλλαγή. αντί να είναι αυτή μια μοναχική, αυστηρά ελεγχόμενη, ιεραρχική προσπάθεια, καλούμε για προσπάθειες από όλη την κοινότητα να αντιμετωπίσει τις πολλές διαφορετικές πτυχές του ζητήματος, και παροτρύνουμε τους συντάκτες και τους προέδρους συνεδρίων σε όλο τον τομέα να υποστηρίξουν τέτοιες προσπάθειες από κάτω προς τα πάνω. ως πρώτο βήμα, είμαστε ιδιαίτερα ευγνώμονες στους συντάκτες του ajis που επέτρεψαν αυτή τη διαδικασία πολυγλωσσικής αναθεώρησης. η διαδικασία ήταν ως εξής: (1) υποβάλαμε το χειρόγραφο στα αγγλικά; (2) ο υπεύθυνος συντάκτης εντόπισε σχετικούς κριτές και τις σχετικές γλώσσες τους (στην περίπτωσή μας, ο κριτής 1 ταυτίστηκε με τα ελληνικά και τα ισπανικά, ο κριτής 2 ταυτίστηκε με τα bahasa indonesia, και ο κριτής 3 ταυτίστηκε με τα ουρντού); (3) μεταφράσαμε το άρθρο σε όλες τις τέσσερις γλώσσες χρησιμοποιώντας το σύστημα μετάφρασης ai μας και υποβάλαμε όλες τις πέντε εκδόσεις στον συντάκτη, ο οποίος στη συνέχεια έστειλε τα άρθρα τόσο στα αγγλικά όσο και στις άλλες σχετικές γλώσσες σε κάθε κριτή; (4) οι κριτές έγραψαν τις κριτικές τους στη γλώσσα προτίμησής τους; ο κριτής 1 έγραψε την κριτική του στα αγγλικά, ενώ οι κριτές 2-3 έγραψαν τις κριτικές τους στη μητρική τους γλώσσα; (5) μεταφράσαμε τις μη αγγλικές κριτικές στα αγγλικά για να τις διαβάσουμε, και στη συνέχεια αναθεωρήσαμε το άρθρο για να λάβουμε υπόψη τις προτάσεις των κριτών, μεταφράσαμε το άρθρο πίσω σε όλες τις γλώσσες, γράψαμε το έγγραφο απάντησής μας και το μεταφράσαμε σε όλες τις γλώσσες επίσης, και στη συνέχεια υποβάλαμε εκ νέου το άρθρο και τις απαντήσεις; (6) ο συντάκτης στη συνέχεια αποδέχτηκε το άρθρο και ενσωματώσαμε τα σχόλια της αναθεώρησης που ακολουθούν. το τελικό βήμα με το περιοδικό περιλάμβανε επίσης τη συζήτηση για το πώς να διαχειριστούμε το doi, και συμφωνήσαμε σε αυτή τη δοκιμαστική διαδικασία να χρησιμοποιήσουμε μόνο ένα doi και ένα αρχείο pdf, για απλότητα. σε πνεύμα διαφάνειας, μετά την αποδοχή του άρθρου, ζητήσαμε από τους κριτές του άρθρου να παρέχουν τα δικά τους αμετάβλητα σχόλια παρακάτω, για να 11 ευχαριστούμε τους προέδρους του συνεδρίου και τους προέδρους του προγράμματος των amcis 2025, pacis 2025 και wi 2025 για τη διάθεσή τους να συμμετάσχουν σε αυτή την πρωτοβουλία. australasian journal of information systems burton-jones et al. 2025, vol 29, provocation this article is not just in english 25 μοιραστούν τις σκέψεις τους με την κοινότητα. ομοίως, αποκάλυψαν τα ονόματά τους παρακάτω σε παρόμοιο πνεύμα διαφάνειας. 4.1 αναστοχαστικά σχόλια από τον κριτή 1 (ελληνικά) ήταν μία εξαιρετικά ευχάριστη εμπειρία η δυνατότητα που μου δόθηκε να αξιολογήσω αυτό το άρθρο, σε τρεις γλώσσες, συμπεριλαμβανομένης της μητρικής μου, τα ελληνικά. η δουλειά αυτή μου επέτρεψε να σκεφτώ ότι πρέπει να είμαστε ευγνώμονες που έχουμε τη δυνατότητα να επικοινωνούμε μέσω της αγγλικής γλώσσας με ανθρώπους από όλη τη γη σε τόσα πολλά επίπεδα, συμπεριλαμβανομένου του επιστημονικού. εν τούτοις, παρά την ευκολία αυτή, πρέπει να αναγνωρίσουμε ότι τα αγγλικά δεν είναι προσβάσιμα σε όλους και η τεχνητή νοημοσύνη (τν) προσφέρει σημαντικές εναλλακτικές τις οποίες μπορούμε να εκμεταλλευτούμε. η μετάφραση στα ελληνικά είναι ένα τέτοιο παράδειγμα. ίσως ο πλούτος και η πολυπλοκότητα της ελληνικής γλώσσας δεν επέτρεψαν μία «τέλεια» μετάφραση και πράγματι υπήρχαν κάποια γραμματικά και άλλα λάθη, καθώς και εκφράσεις οι οποίες δεν είναι συμβατές με τη φυσική χρήση της γλώσσας, κάτι το οποίο όμως με το πέρας του χρόνου αναμένω ότι θα βελτιωθεί δεδομένης της τεχνικής δυνατότητας αυτο-βελτίωσης της τν! συγχαρητήρια στους συγγραφείς για την ιδέα και την υλοποίησή της. (πέτρος χαμακιώτης) 4.2 αναστοχαστικά σχόλια από τον κριτή 1 (ισπανικά) επιπλέον από τα σχόλιά μου παραπάνω για την ελληνική έκδοση, η μετάφραση στα ισπανικά ήταν καλύτερης ποιότητας σε σύγκριση με την ελληνική, πιθανώς επειδή η τεχνητή νοημοσύνη (τν) είναι καλύτερα «εκπαιδευμένη» στα ισπανικά, σε σύγκριση με τα ελληνικά, λόγω του μεγαλύτερου αριθμού ισπανόφωνων χρηστών. επίσης, αν και αυτό πιστεύω πως δεν ισχύει στις ισπανόφωνες χώρες της λατινικής αμερικής, οι ισπανοί δεν συνηθίζουν να χρησιμοποιούν αγγλικούς όρους στην καθημερινή χρήση της γλώσσας και αυτό είναι κάτι που η τν δεν είχε λάβει υπόψη. για παράδειγμα, το ακρωνύμιο της τν στα αγγλικά είναι ai ενώ στα ισπανικά είναι ia (καθώς ο όρος είναι “inteligencia artificial”). επίσης, υπάρχουν αρκετοί «ψευδείς φίλοι», όπως η λέξη “conference” όταν αναφερόμαστε σε ακαδημαϊκά συνέδρια, της οποίας ο πιο ακριβής όρος στα ισπανικά είναι “congreso” (αντί για “conferencia”). (πέτρος χαμακιώτης) 4.3 αναστοχαστικά σχόλια από τον κριτή 2 (bahasa indonesia) αυτό το άρθρο παρέχει μια διεξοδική εξέταση της μεροληψίας της αγγλικής γλώσσας σε ακαδημαϊκά περιβάλλοντα και διερευνά πώς τα εργαλεία μετάφρασης τεχνητής νοημοσύνης (ai) θα μπορούσαν να επιφέρουν σημαντικές αλλαγές. το κύριο πλεονέκτημα του άρθρου έγκειται στον σαφή προσδιορισμό του προβλήματος, υποστηριζόμενο από βιβλιογραφία από διάφορους τομείς που καταδεικνύει τόσο τον επείγοντα όσο και τις αρνητικές επιπτώσεις της γλωσσικής μεροληψίας. το όραμα για την πολυγλωσσική επιστήμη είναι πειστικό και καλά διατυπωμένο, με την τεχνητή νοημοσύνη να αποτελεί μια ταιριαστή καινοτόμο λύση για τον τομέα των πληροφοριακών συστημάτων. ορισμένοι τομείς, ωστόσο, θα μπορούσαν να επωφεληθούν από περαιτέρω ανάπτυξη. ενώ αναφέρονται οι πολυπλοκότητες της τεχνητής νοημοσύνης, η συζήτηση για την australasian journal of information systems burton-jones et al. 2025, vol 29, provocation this article is not just in english 26 αντιμετώπιση των περιορισμών της τεχνητής νοημοσύνης, όπως οι παραισθήσεις ή η πολιτισμική μεροληψία, χρειάζεται μεγαλύτερο βάθος. για παράδειγμα, ορισμένες ινδονησιακές μεταφράσεις στο πρώτο προσχέδιο αυτού του άρθρου φαίνονται κάπως άκαμπτες. η φράση "our provocation to readers" (η πρόκλησή μας προς τους αναγνώστες) μεταφράστηκε ως "tantangan kepada pembaca" (μια πρόκληση προς τους αναγνώστες) και ακούγεται πολύ "δυναμική" – μια πιο φυσική εκδοχή θα ήταν "kami ingin mengajak pembaca berpikir" (θα θέλαμε να προσκαλέσουμε τους αναγνώστες να σκεφτούν), η οποία ακούγεται πιο "ήπια". η ανάλυση της πολυπλοκότητας φαίνεται να παραμένει σε επιφανειακό επίπεδο, ιδιαίτερα όσον αφορά τεχνικά ζητήματα όπως η ακρίβεια της μετάφρασης ή ο πολιτισμικός αντίκτυπος. τα επιχειρήματα επίσης στερούνται ισχυρής εμπειρικής υποστήριξης. η μελλοντική έρευνα θα μπορούσε να εξετάσει το ενδεχόμενο ενσωμάτωσης αποτελεσμάτων πιλοτικών μελετών και βαθύτερης τεχνικής ανάλυσης. η περαιτέρω διερεύνηση ειδικών πλαισίων πληροφοριακών συστημάτων, όπως η πολύγλωσση υλοποίηση σε εταιρικά συστήματα, θα μπορούσε επίσης να εμπλουτίσει τη συμβολή αυτού του άρθρου στην ευρύτερη ακαδημαϊκή κοινότητα. (arif perdana) 4.4 αναστοχαστικά σχόλια από τον κριτή 3 (ουρντού) ευχαριστώ που με προσκαλέσατε να αναστοχαστώ πάνω σε αυτή την διορατική πρόκληση. η αναθεώρηση αυτού του άρθρου στα ουρντού ήταν μια ανταποδοτική εμπειρία. ήταν η πρώτη φορά που ασχολήθηκα με ακαδημαϊκή εργασία στη μητρική μου γλώσσα, και η εμπειρία μου φάνηκε πραγματικά περιεκτική, κάτι που δεν είχα βιώσει ποτέ πριν. το βασικό μήνυμα της εργασίας, ότι η επιστημονική γνώση δεν πρέπει να περιορίζεται σε μία μόνο γλώσσα, μου βρήκε βαθιά απήχηση. αναδεικνύει πώς η κυριαρχία της αγγλικής μπορεί ακούσια να περιορίσει τόσο τη συμμετοχή όσο και την κατανόηση στον επιστημονικό διάλογο. εν γένει, η μετάφραση στα ουρντού είναι αρκετά καλά εκτελεσμένη. ωστόσο, σε ορισμένα σημεία, η μετάφραση φαίνεται ελαφρώς άκαμπτη και διαταράσσει τη ροή λόγω της λέξη προς λέξη προσέγγισης αντί της προσαρμογής των εκφράσεων για φυσική ουρντού διατύπωση. τούτου λεχθέντος, αυτές οι προκλήσεις δεν μειώνουν τις δυνατότητες της πολύγλωσσης επιστήμης· αντιθέτως, αποκαλύπτουν πού χρειάζεται ακόμα να γίνει σημαντική εργασία. με συνεχή εισροή και επίβλεψη από ανθρώπους, τα εργαλεία μετάφρασης τεχνητής νοημοσύνης έχουν τη δυνατότητα να μετατρέψουν την επιστήμη σε κάτι όχι μόνο παγκοσμίως προσβάσιμο αλλά και τοπικά ουσιαστικό και απηχητικό. (humza naseer) 5 συμπέρασμα η γλωσσική προκατάληψη υπήρξε μια αναπόφευκτη πραγματικότητα καθ' όλη την ιστορία της επιστήμης, αλλά δεν είναι πλέον απαραίτητο να συνεχίζεται. η πρόοδος στη μετάφραση με τη βοήθεια της τεχνητής νοημοσύνης υποδηλώνει ένα νέο μέλλον για την επιστήμη — ένα πολύγλωσσο μέλλον. μια πολύγλωσση επιστήμη μπορεί να είναι πιο συμπεριληπτική (επιτρέποντας σε περισσότερους συγγραφείς και αξιολογητές να συμμετάσχουν) και πιο επιδραστική (επιτρέποντας σε περισσότερους αναγνώστες να επωφεληθούν από την επιστήμη). όπως σε κάθε ψηφιακό μετασχηματισμό, οι επιπτώσεις αυτής της αλλαγής θα είναι απρόβλεπτες και θα υπάρχουν πολλά να μάθουμε. λίγες ευκαιρίες είναι τόσο κατάλληλες για το πεδίο των πληροφοριακών australasian journal of information systems burton-jones et al. 2025, vol 29, provocation this article is not just in english 27 συστημάτων όσο αυτή, και η στιγμή για δράση είναι τώρα. προσδοκούμε να μοιραστούμε τις γνώσεις μας καθώς προχωράμε στην ανάπτυξη του έργου μας και καλούμε και άλλους να συμμετάσχουν στην προσπάθεια. spanish este artículo no está solo en inglés: hacia una ciencia más inclusiva e impactante con la traducción de inteligencia artificial resumen la humanidad es lingüísticamente diversa, sin embargo, la ciencia no refleja esta diversidad. el éxito académico demanda un dominio del idioma inglés. cada conferencia y revista importante, incluida esta, lo asume. la preferencia por el idioma inglés en la ciencia es tan predominante que la mayoría de los científicos y asociaciones científicas lo dan por sentado, nunca se discute ni se aborda. esto resulta injusto y conlleva significativos costos y oportunidades desaprovechadas. también es innecesario. las herramientas de traducción de inteligencia artificial (ia) están mejorando rápidamente, lo que nos permite prever una ciencia multilingüe. nuestra provocación para los lectores es: ¿de qué manera podríamos aprovechar las herramientas de traducción de ia para lograr una ciencia más inclusiva e impactante? este desafío se presenta como una oportunidad ideal para los académicos en el campo de los sistemas de información porque implica diseñar artefactos y prácticas sociotécnicas para un futuro mejor. para demostrar su viabilidad, este artículo ha sido sometido a un proceso de revisión multilingüe y se publica en cinco idiomas, facilitado por la traducción de ia. 1 el problema del sesgo lingüístico uno de los aspectos más obvios y menos discutidos del australasian journal of information systems (ajis) es que es una revista publicada en inglés. esto no se cuestiona ni se debate porque se considera una elección natural. todas las principales conferencias y revistas en el campo de los sistemas de información (is) están en inglés, al igual que en todas las demás disciplinas. el éxito académico en la ciencia presupone el dominio del inglés. sin embargo, esto resulta inequitativo, porque la mayor parte del mundo no habla inglés. de los 8 mil millones de personas en la tierra, se estima que solo 1.5 mil millones hablan inglés, y solo el 25% de ellos lo habla como su lengua materna (icls, 2024). el objetivo de esta propuesta provocativa es aumentar la conciencia sobre el problema del "sesgo lingüístico" en el campo de is e instar a una acción comunitaria para abordarlo. describimos cómo estamos comenzando este trabajo, como miembros de un grupo de trabajo de la asociación para los sistemas de información (ais), e instamos a una acción comunitaria. es difícil evitar la percepción de que estamos rezagados respecto a otros campos en nuestra conciencia sobre el sesgo lingüístico. se abordó en medicina en la década de 1990 (egger et al., 1997), y desde entonces se ha abordado en muchos campos, incluyendo la investigación de emociones (wierzbicka, 2009), ecología (clavero, 2011), lingüística (levisen, 2019), gestión (pudelko & tenzer, 2019), derecho (ammann, 2022), ciencia cognitiva (blasi, henrich, adamou, kemmerer, & majid, 2022), conservación (amano & berdejo-espinola, 2024), ciencias de la tierra (bampi, pires-oliveira, loyola-bartra, & limaribeiro, 2024), cuidados críticos (gonzález-dambrauskas, salluh, machado, & rotta, 2024), y ortodoncia (mheissen, spineli, daraqel, & alsafadi, 2024). también se ha abordado en discusiones a nivel científico australasian journal of information systems burton-jones et al. 2025, vol 29, provocation this article is not just in english 28 sobre el futuro de la publicación académica (ahmed et al., 2023; amano, rojas, boum ii, calvo, & misra, 2021; arenas-castro et al., 2024; editorial, 2023). como la mayoría de los sesgos, el sesgo lingüístico es costoso. como informan amano et al. (2023), los académicos no nativos en inglés tienen que pasar más tiempo leyendo, escribiendo y revisando artículos, tienen más probabilidades de que sus artículos sean rechazados por razones relacionadas con el idioma y experimentan barreras para asistir a conferencias. el sesgo lingüístico genera estrés en los académicos en las primeras etapas de su carrera (hohti & truman, 2021), deteriora el proceso de revisión por pares (politzer-ahles, girolamo, & ghali, 2020), crea sesgos en las revisiones de literatura (nunez & amano, 2021), revisiones sistemáticas (hannah, haddaway, fuller, & amano, 2023), y meta-análisis (konno et al., 2020), y altera las comparaciones de investigación a nivel nacional (van leeuwen, moed, tijssen, visser, & van raan, 2001). el sesgo lingüístico está incluso presente en las búsquedas de google scholar, ya que privilegia los artículos en inglés (rovira, codina, & lopezosa, 2021). como un sesgo sistémico y de larga duración, el sesgo lingüístico también se puede ver en las prácticas nacionales, como la prevalencia de la enseñanza del inglés, y en industrias importantes, como la industria global del inglés como segundo idioma, valorada en 10-11 mil millones de usd12. si bien buscamos resaltar el sesgo lingüístico que afecta a la ciencia, no ignoramos los beneficios del inglés global. dado que la diversidad lingüística naturalmente dificulta la comunicación (por ejemplo, ilustrada en la célebre torre de babel), la adopción de un idioma común (ya sea el inglés o cualquier otro) puede facilitar el progreso en muchos sectores (john & özgür, 2021). como señaló el british council (2013, p. 3) “gracias a internet, el auge de las redes sociales, la velocidad y expansión de la tecnología de comunicaciones globales y la economía mundial cada vez más interconectada e interdependiente, el inglés ahora facilita la rápida polinización cruzada de ideas e innovaciones a nivel mundial, y el desarrollo de un nuevo tipo de mercado único supranacional en conocimiento e ideas.” sin embargo, aunque reconocemos los beneficios de la proliferación del idioma inglés (y varios de nuestro equipo de autores han disfrutado de estos beneficios, como hablantes nativos de inglés), nuestro propósito aquí es destacar los costos, porque son significativos pero generalmente no reconocidos y se dan por sentados (mélitz, 2007; salomone, 2022). 2 la promesa de las herramientas de traducción de ia a pesar de la prevalencia y el costo del sesgo lingüístico, no debería considerarse como algo inmutable porque las herramientas de traducción mediante inteligencia artificial (ia) están experimentando una rápida mejora. debe enfatizarse la importancia histórica de este momento, ya que el sesgo lingüístico es tan antiguo como la ciencia. después de todo, la ciencia siempre se ha llevado a cabo en una lengua u otra, y para participar (leer, hablar, escribir), se debía conocer el idioma (gordin, 2015). con más de 7,000 lenguas en la tierra (anderson, 2010), los científicos no podían leer ni escribir en todas las lenguas. durante milenios, simplemente tenía sentido llevar a cabo la ciencia en la lengua del momento. las lenguas dominantes en diferentes momentos de la historia incluyeron el egipcio, el griego, el latín, el chino y varias lenguas europeas, pero el inglés ha predominado desde la segunda guerra mundial (gordin, 2015). 12 https://www.businessresearchinsights.com/market-reports/esl-english-as-a-second-language-market117478; https://www.englishaustralia.com.au/our-sector/understanding-the-sector australasian journal of information systems burton-jones et al. 2025, vol 29, provocation this article is not just in english 29 a pesar de intentos breves (y fallidos) de desarrollar una lengua universal (por ejemplo, el esperanto) (patterson & huff, 1999), la solución predominante para abordar las diferencias lingüísticas a lo largo de los años ha sido la traducción. sin embargo, a pesar de existir como campo durante más de 2,000 años (dynamic_language, 2024), la traducción, hasta donde podemos discernir, nunca ha sido adoptada de manera generalizada en la ciencia, en gran parte porque el proceso de traducción es costoso y requiere mucho tiempo (wood, 1967). a lo largo de la historia, varias revistas se han publicado en varios idiomas, e incluso hoy en día hay alrededor de 500 revistas multilingües13, pero estas son las excepciones que confirman la regla. como gordin escribe de manera evocativa (2015, p. 3), “si estás interesado en cómo sería vivir en un mundo con un solo idioma de comunicación, un mundo sin babel, deberías observar a los científicos naturales. ellos vienen de allí.” la inmensa mayoría de las revistas se publican únicamente en inglés (gordin, 2015), y la mayoría de las revistas multilingües son solo revistas en dos idiomas entre inglés y otro idioma, y la mayoría de ellas son solo bilingües debido a requisitos legales nacionales (por ejemplo, revistas publicadas en canadá, que es un país bilingüe). el potencial de las herramientas de traducción de ia para abordar el sesgo lingüístico en la ciencia ha emergido recientemente (berdejo-espinola & amano, 2023; crotty, 2024; steigerwald et al., 2022), derivado de dos transformaciones a largo plazo. una ha sido la transformación digital gradual de la investigación misma. no hace mucho, muchas tareas de investigación eran manuales. los investigadores caminaban a las bibliotecas, leían y anotaban documentos físicos, enviaban documentos y revisiones por correo, incluso realizaban cálculos y escribían documentos a mano. ahora, casi todas las tareas se realizan en línea: búsqueda, lectura, escritura, anotación, colaboración, análisis, revisión, envío, publicación. una consecuencia profunda de esta transformación es que gran parte de la ciencia ahora se desarrolla en plataformas digitales. es decir, los artículos científicos se escriben, revisan, publican, leen y buscan en un idioma (hoy en día casi siempre en inglés), y todo esto sucede en grandes plataformas digitales interconectadas. algunas de estas plataformas ahora son vastas, por ejemplo, scopus alberga más de 90 millones de registros14 y google scholar busca en más de 300 millones de registros (martin-martin, thelwall, orduna-malea, & lopez-cozar, 2021). y aquí es donde entra la segunda transformación: la transformación de la ia. aunque los modelos de lenguaje en ia tienen una historia que data de la década de 1950, el rápido desarrollo de grandes modelos de lenguaje fue posibilitado por el crecimiento masivo de datos en línea para el aprendizaje (naik, naik, & naik, 2024) junto con nuevas arquitecturas, especialmente la arquitectura de transformadores, que pueden aprovechar dichos datos (vaswani et al., 2017). en los últimos cinco años, estas dos transformaciones han permitido que surjan nuevas formas de ia, facilitando un progreso que hace poco tiempo parecía imposible. y este progreso es especialmente rápido en la traducción de idiomas. las tareas de traducción de idiomas fueron las primeras en ser probadas en la arquitectura de transformadores y, por lo tanto, son ideales para ella (vaswani et al., 2017). no estamos afirmando que la traducción de ia pueda resolver completamente y rápidamente el sesgo lingüístico. habrá muchas complejidades (amano y berdejo-espinola, 2024). sin embargo, el potencial es lo suficientemente significativo como para que tengamos la obligación 13 https://en.wikipedia.org/wiki/category:multilingual_journals 14https://assets.ctfassets.net/o78em1y1w4i4/76uvtuilkssqe4h4h1pzkq/e5ff8eef9694f216a5c29e2ebf8 0fa51/fast-facts-about-elsevier.pdf australasian journal of information systems burton-jones et al. 2025, vol 29, provocation this article is not just in english 30 de explorarlo. desde la perspectiva de una asociación científica, como la association for information systems, sería imprudente no intentar reducir un sesgo sistemático prevalente y costoso que enfrentan los miembros de nuestro campo científico y, de manera similar, imprudente no intentar aprovechar las nuevas oportunidades de inclusión e impacto que podría ofrecer la traducción de ia. por eso formamos nuestro grupo de trabajo: el grupo de trabajo de la association for information systems sobre traducción de ia para una ciencia inclusiva e impactante. 3 complejidades existen numerosas complejidades asociadas al uso de la traducción por ia en el ámbito científico. destacamos seis categorías. en primer lugar, hay complejidades que dependen de cómo delimitamos el problema. el alcance más simple es centrarse en la traducción de un solo artículo de un idioma a otro, pero incluso este enfoque plantea preguntas. por ejemplo, ¿deberíamos traducir solo los artículos aceptados, o también los artículos enviados? traducir solo los artículos aceptados ayudaría a la ciencia promoviendo el impacto y aumentando la audiencia de un artículo, mientras que traducir también los artículos enviados apoyaría la inclusión, permitiendo que más autores y revisores escriban en su idioma nativo. aunque esta última opción tiene mayores beneficios, también requiere más trabajo, ya que se necesita traducir tanto el artículo como las revisiones del artículo, y también requiere un flujo de trabajo para que los autores y revisores verifiquen la calidad de las traducciones durante el proceso de revisión sin sobrecargarlos. una vez que un artículo es aceptado/publicado, surge otra pregunta sobre si nos enfocamos en la traducción a un solo idioma o a varios, y si el autor o el lector elige el idioma. si los lectores pueden elegir, ¿deberían las traducciones prepararse de antemano (por ejemplo, en un pequeño número de idiomas comunes) y los lectores elegir el idioma que prefieren de esta lista, o deberían poder solicitar una traducción en su idioma elegido (ya sea de una lista fija, o en un mundo ideal, de cualquier idioma) que luego se produce en tiempo real por la plataforma de publicación a pedido? lo último permitiría un rango más amplio de idiomas, mientras que lo primero permitiría un proceso de verificación de la calidad de la traducción. si se elige lo último, es posible que se introduzcan pequeños cambios en el significado en algunas traducciones sin que los autores lo sepan, creando complicaciones para las citas posteriores de ese trabajo. por lo tanto, los artículos publicados con la opción de traducción en tiempo real bajo demanda pueden requerir una advertencia que reconozca el potencial de errores. otra pregunta de alcance es si miramos más allá de la traducción de un solo artículo y consideramos la traducción de una biblioteca/plataforma completa de artículos, como toda la elibrary de la association for information systems, que contiene más de 65,000 artículos. ampliar el alcance a una biblioteca no solo aumenta el número de artículos a traducir, sino que también trae consideraciones adicionales. por ejemplo, puede expandir el horizonte temporal de traducción, porque una biblioteca puede tener artículos publicados durante muchas décadas, y por lo tanto el proceso de traducción puede necesitar considerar cambios en el contexto cultural y el léxico de un campo a lo largo del tiempo. asimismo, puede requerir la creación de herramientas adicionales, como una herramienta de búsqueda multilingüe para buscar en la biblioteca en múltiples idiomas. australasian journal of information systems burton-jones et al. 2025, vol 29, provocation this article is not just in english 31 una última pregunta de alcance se relaciona con las capacidades de la ia. dado que la misma tecnología de ia que realiza traducciones ofrece capacidades relacionadas, ¿deberíamos usar también estas capacidades? por ejemplo, ¿deberíamos mirar más allá de la traducción de artículos para considerar la producción de resúmenes de artículos en diferentes idiomas (por ejemplo, resúmenes para ejecutivos o estudiantes)? asimismo, ¿deberíamos considerar mejoras en el lenguaje, no solo traducciones, por ejemplo, ofreciendo sugerencias para mejorar un texto, no solo traducir el texto 'palabra por palabra'? si consideramos lo último, podríamos replantear el problema de mera 'traducción' a 'explicación del significado', donde la traducción es solo un elemento de la explicación del significado y otros elementos podrían incluir mejoras en la gramática, el flujo y el estilo. como puede ver, el alcance del problema puede variar mucho. en esta etapa, nuestro grupo de trabajo está adoptando un alcance intermedio. buscamos apoyar todo el proceso de revisión, desde la presentación hasta la aceptación. también buscamos apoyar las traducciones a múltiples idiomas. sin embargo, estamos limitando nuestros esfuerzos a 10 idiomas principales al principio (bahasa indonesio, urdu, griego, francés, malayo, coreano, mandarín, alemán, español y portugués) y nos enfocamos en desarrollar traducciones prepreparadas que permitan un proceso de verificación de calidad con un humano en el proceso en lugar de habilitar traducciones en tiempo real bajo demanda. estamos incluyendo la producción de resúmenes de artículos dentro de nuestro alcance, pero solo nos enfocamos en traducir artículos individuales en lugar de una biblioteca/plataforma completa, y no estamos considerando aspectos más amplios de 'explicación del significado' como mejoras en el lenguaje en esta primera fase. además de estas complejidades que surgen debido a preguntas de alcance, otro conjunto de complejidades surge de las prácticas del entorno de publicación. por ejemplo, ¿cómo deberíamos manejar los digital object identifiers (dois)? hoy en día, a cada artículo publicado se le debe asignar un doi único porque esto permite a los motores de búsqueda y agregadores localizar e identificar citas para cada artículo. sin embargo, si creamos múltiples traducciones de un artículo, ¿estarían estas incluidas en el mismo documento (por ejemplo, página web, pdf) con el mismo doi, y las citas del artículo se agregarían independientemente de la versión (traducción) que se cite? una dificultad para responder a estas preguntas es que ninguna entidad (asociación científica, editor, gran tecnología) tiene control sobre todo el ecosistema de publicación, y es difícil predecir cómo evolucionará todo. por lo tanto, tenemos que elegir prácticas de traducción que de alguna manera se alineen con las prácticas existentes mientras sean lo suficientemente flexibles para adaptarse. surgen complejidades prácticas similares con los derechos de autor. una traducción puede considerarse una 'obra derivada' y, por lo tanto, debe cumplir con las restricciones de derechos de autor sobre tales obras. otro problema práctico es el plagio. irónicamente, facilitar la traducción también podría facilitar el plagio. esto se debe a que el plagio generalmente se define y detecta por el uso de la misma secuencia de palabras en un artículo, es decir, copia palabra por palabra. la traducción no tiene que ser palabra por palabra, sino que puede ser 'traducción por sentido' (transmitiendo el mismo sentido en lugar de las mismas palabras) (sun, 2013). si un plagiador plagia un artículo creando una traducción utilizando la traducción por sentido, no está claro si las herramientas actuales de detección de plagio lo detectarían. es posible que dichas herramientas necesiten ser revisadas para controlar este riesgo. australasian journal of information systems burton-jones et al. 2025, vol 29, provocation this article is not just in english 32 un tercer conjunto de complejidades surge de las características de la ia. aunque el rápido progreso de la ia nos motivó a seguir este proyecto, todavía no es perfecto. todavía ocurren errores de interpretación (ji et al., 2023), y el rendimiento de la traducción por ia varía según la cantidad de datos de entrenamiento en cada idioma (xu et al., 2024). debido a la gran dominancia del idioma inglés en internet, los modelos de gen ia pueden incorporar no solo sesgo cultural, sino también sesgo del idioma inglés (luo & puett, 2024). hacemos un llamado a los investigadores de nuestro campo para que creen artefactos y prácticas sociotécnicas tangibles para abordar estos problemas. al hacerlo, los investigadores no deben dejarse seducir solo por el determinismo tecnológico o el determinismo social: una visión sociotécnica es crítica. es decir, al igual que cualquier otra tecnología, el éxito de la traducción por ia dependerá no solo de la tecnología, sino también de cómo los usuarios (autores, editores, revisores, lectores) la perciban y la utilicen. por ejemplo, las tendencias humanas como la aversión a los algoritmos y la desconfianza inevitablemente necesitarán ser consideradas (asscher & glikson, 2023). implementar modelos de ia también conlleva un costo financiero, planteando la pregunta de quién debería asumir dicho costo. una de las razones por las que estamos persiguiendo esto como un grupo de trabajo de ais es que estamos evaluando si una asociación científica debería asumir ese costo y cómo hacerlo para sus miembros. un cuarto conjunto de complejidades surge debido a los matices en el lenguaje escrito que los modelos actuales de traducción por ia pueden pasar por alto. por ejemplo, en el proceso de revisión de este artículo, uno de nuestros revisores notó inconsistencias en la traducción de acrónimos. por ejemplo, en españa, los autores generalmente usan la versión en español del acrónimo (ia), pero en grecia, los autores están igualmente acostumbrados a usar la versión en inglés (ai) o la versión griega (tn), y el autor debería usar solo una de manera consistente. nuestra traducción no logró hacer esto. mientras tanto, nuestros otros revisores notaron que nuestras traducciones a veces sonaban demasiado "rígidas", "fuertes" o "formales" y que deberíamos haber adoptado un tono más "suave" o más "natural". esto puede haber sido debido a que la traducción por ia eligió automáticamente un tono demasiado formal, lo cual no nos dimos cuenta de antemano. para abordar estos desafíos, estamos refinando activamente nuestro enfoque de traducción. en particular, hemos incorporado comentarios de los revisores de este artículo, especialmente en lo que respecta a la evaluación de la naturalidad cultural y el tono. como resultado, nuestro enfoque de traducción ahora da mayor consideración a estos aspectos. por ejemplo, los registros internos de la herramienta han capturado sugerencias inspiradas por los revisores, como: "la frase '...' es bastante directa. considere reformularla a un tono más académico, como '...'." o "la frase '...' es algo literal. una formulación académica más apropiada sería '...'." a pesar del rápido crecimiento de la fuerza de la traducción por ia, es difícil imaginar que estas herramientas ofrecerán una traducción perfecta en todos los idiomas y adaptada para cada artículo, pero tenemos la intención de abordar tantos de estos problemas como podamos (por ejemplo, ajustando la estructura del modelo de ia para usar el contexto de manera más efectiva). un quinto conjunto de complejidades surge de las instituciones de la ciencia que podrían reforzar la situación actual e impedir la aparición de la ciencia multilingüe. por ejemplo, cuando hemos discutido esta iniciativa con investigadores exitosos que no son hablantes nativos de inglés, nos ha sorprendido cuántos han dicho que no aprovecharían nuestra oportunidad de escribir en su idioma nativo porque son mejores escribiendo artículos en inglés (porque han entrenado toda su vida laboral en inglés). en algunos aspectos, esto no nos preocupa porque la población de hablantes no nativos de inglés que podría beneficiarse es australasian journal of information systems burton-jones et al. 2025, vol 29, provocation this article is not just in english 33 mucho mayor que la pequeña población de hablantes no nativos de inglés que han 'triunfado' en el sistema dominado por el inglés. por otro lado, los miembros senior de una comunidad naturalmente influyen en los miembros junior, y por lo tanto sus puntos de vista pueden servir para reforzar la situación actual durante algún tiempo. como ejemplo, hemos estado trabajando con investigadores senior en un país europeo en esta iniciativa. aunque apoyan firmemente nuestra iniciativa, nos aconsejaron proceder con cautela porque habían trabajado durante décadas para mejorar la escritura en inglés en su comunidad, y ese esfuerzo había sido muy exitoso, llevando a un fuerte aumento en el número de publicaciones en revistas de alto nivel. estos investigadores deseaban evitar cualquier riesgo que pudiera comprometer ese éxito. como muestra este ejemplo, los esfuerzos para fomentar la ciencia multilingüe deben diseñarse cuidadosamente con el conocimiento de otras iniciativas preexistentes diseñadas en torno a la situación actual. finalmente, tenemos un sexto conjunto de complejidades que son más filosóficas. una forma de ver estas es pensar en un artículo como solo un paso en el largo proceso de una idea que se forma en la mente de un autor hasta que es apreciada en la mente de un lector. para empezar, considere un autor conceptualizando una idea. la hipótesis de sapir-whorf sugiere que los pensamientos de un autor están moldeados por el idioma del autor (lucy, 1998). si es cierto, implica que el sesgo del idioma se remonta hasta el origen, incluso restringiendo las ideas que los autores pueden formar. podemos llamar a esto el problema de 'concepción'. los problemas continúan porque incluso si un idioma puede usarse para comunicar una idea, un autor puede carecer de la habilidad lingüística para hacerlo. podemos llamar a esto el problema de 'articulación'. e incluso si un autor tiene el idioma correcto para comunicar una idea y lo usa hábilmente, un lector puede no interpretar las palabras de la misma manera. podemos llamar a esto el problema de 'interpretación'. estamos seguros de que los lectores de este artículo están familiarizados con tales problemas porque son comunes en la vida diaria. colectivamente, estos problemas significan que los artículos de investigación son inevitablemente un medio imperfecto para comunicar ideas científicas en una comunidad de académicos. las imperfecciones en el significado son simplemente inherentes al proceso, independientemente del idioma utilizado. claramente, las herramientas de traducción por ia no pueden resolver todos estos problemas. como solo un ejemplo, las herramientas de ia pueden ayudar a reducir el problema de concepción al permitir que las personas piensen en su propio idioma, pero también pueden fallar en hacerlo porque la dominancia del idioma inglés ha sido tan fuerte durante tanto tiempo que algunos idiomas no tienen el léxico requerido (barath, 2019). 4 una oportunidad y nuestro enfoque mientras hemos enumerado muchas complejidades, las vemos como oportunidades de aportar en lugar de problemas a evitar. para nosotros, subrayan por qué este es un desafío ejemplar para el campo de los sistemas de información (is), ya que las oportunidades y complejidades son profundamente sociotécnicas (sarker, chatterjee, xiao, & elbanna, 2019). como mencionamos al principio, el campo de los sistemas de información no es el primero en tomar conciencia del sesgo lingüístico. aun así, aunque muchos campos han creado conciencia sobre el problema, pocos lo han abordado sistemáticamente. como resultado, el campo de los is tiene la oportunidad de ayudar a trabajar en la vanguardia de la ciencia junto con otros australasian journal of information systems burton-jones et al. 2025, vol 29, provocation this article is not just in english 34 campos líderes15. dado que el tema está tan íntimamente ligado a la naturaleza de nuestro campo, como un campo dedicado a entender y diseñar artefactos y prácticas sociotécnicas para un futuro mejor, creemos que el campo de los is no solo tiene la oportunidad de marcar la diferencia, sino también el deber de hacerlo. motivados por esta oportunidad, formamos un grupo de trabajo de ais en 2024. siguiendo los modelos comunes de cambio (lewin, 1951), asumimos que se necesitará un esfuerzo sustancial para 'descongelar' el campo en lugar de asumir que el campo puede cambiar de inmediato. para facilitar el 'descongelamiento', organizamos paneles y plenarias en cuatro conferencias importantes para crear conciencia y motivar la discusión (burton-jones, 2025; burton-jones, chau, feng, kankanhalli, & vogel, 2024; pouloudi et al., 2024; sarker, ahuja, feng, jarvenpaa, & seidel, 2024). también hablamos con varios capítulos de ais que representan diferentes comunidades lingüísticas y trabajamos en el desarrollo y la prueba inicial de herramientas de traducción automática por ia. en 2025, continuamos nuestros esfuerzos para crear conciencia sobre el problema (como a través de este artículo) mientras adoptamos una postura más activa con ciclos de diseño, prueba y aprendizaje. por ejemplo, agradecemos a los editores y revisores de ajis por estar dispuestos a probar un proceso de revisión en múltiples idiomas para este artículo, del cual podemos aprender y mejorar. simultáneamente, estamos involucrados en otros trabajos para proporcionar datos concretos y pruebas. por ejemplo, probamos traducciones de ia en la americas conference on information systems (amcis) en 2025 (ver: https://aisel.aisnet.org/amcis2025/translated/). también estamos trabajando con otras conferencias en 2025 en diferentes casos de uso de traducción16. nos enfocamos especialmente en tres casos de uso: (1) traducción de ia en el proceso de revisión; (2) traducción de ia de artículos aceptados; y (3) preparación de resúmenes de artículos en diferentes idiomas para ampliar el impacto. también realizaremos encuestas, entrevistas y grupos focales para aprender cómo el sesgo lingüístico afecta a los académicos en el campo de los sistemas de información y los impactos potenciales de las herramientas de traducción automática por ia. también estamos colaborando con académicos de otros campos para apoyar el movimiento científico más amplio (amano, bowker, & burton-jones, 2025). al seguir estos pasos, reconocemos que enfrentaremos muchos desafíos y que quedan muchas oportunidades adicionales. también podemos aprender lecciones y tomar aliento de otros movimientos científicos, como el movimiento de ciencia abierta (ferguson et al., 2023), que están motivados de manera similar hacia una mayor inclusividad e impacto en la ciencia. como en cualquier esfuerzo de cambio, cometeremos errores y tendremos que ajustar nuestros planes. sin embargo, seguimos entusiasmados con las posibilidades, y esperamos que otros investigadores de is también lo estén, y que juntos podamos lograr un cambio significativo. en lugar de ser un esfuerzo aislado, estrictamente jerárquico, exhortamos a la comunidad a emprender esfuerzos para abordar los muchos aspectos diversos del problema, e instamos a los editores y presidentes de conferencias en todo el campo a apoyar estos esfuerzos desde las bases. 15 otras voces destacadas en esta vanguardia incluyen a la unesco (https://doi.org/10.54677/mnmh8546) y al consejo de la unión europea (https://www.consilium.europa.eu/media/56958/st10126-en22.pdf). 16 agradecemos a los presidentes del congreso y a los presidentes del programa de amcis 2025, pacis 2025 y wi 2025 por su disposición a participar en esta iniciativa. australasian journal of information systems burton-jones et al. 2025, vol 29, provocation this article is not just in english 35 como primer paso, estamos especialmente agradecidos a los editores de ajis por permitir este proceso de revisión en múltiples idiomas. el proceso fue el siguiente: (1) presentamos el manuscrito en inglés; (2) el editor encargado identificó a los revisores relevantes y sus idiomas asociados (en nuestro caso, el revisor 1 se identificó con griego y español, el revisor 2 se identificó con bahasa indonesio, y el revisor 3 se identificó con urdu); (3) tradujimos el artículo a los cuatro idiomas utilizando nuestro sistema de traducción automática por ia y presentamos las cinco versiones al editor, quien luego envió los artículos tanto en inglés como en los otros idiomas relevantes a cada revisor; (4) los revisores escribieron sus revisiones en su idioma preferido; el revisor 1 escribió su revisión en inglés, mientras que los revisores 2-3 escribieron sus revisiones en su idioma nativo; (5) tradujimos las revisiones no inglesas al inglés para poder leerlas, y luego revisamos el artículo para tener en cuenta las sugerencias de los revisores, tradujimos el artículo de nuevo a todos los idiomas, escribimos nuestro documento de respuesta y también lo traducimos a todos los idiomas, y luego volvimos a presentar el artículo y las respuestas; (6) el editor luego aceptó el artículo y nosotros incorporamos los comentarios de revisión a continuación. el paso final con la revista también involucró discutir cómo gestionar el doi, y acordamos en este proceso de prueba usar solo un doi y un archivo pdf, por simplicidad. con un enfoque de transparencia, tras la aceptación del artículo, pedimos a los revisores del artículo que proporcionaran sus propios comentarios sin editar a continuación, para compartir sus reflexiones con la comunidad. del mismo modo, dieron a conocer sus nombres a continuación con un espíritu similar de apertura. 4.1 reflexiones del revisor 1 (griego) la revisión de este artículo en tres idiomas, incluyendo mi lengua materna, el griego, ha sido una excelente experiencia. este trabajo me permitió apreciar la oportunidad que nos ofrece el uso global del inglés de conectar con personas de todo el mundo a tantos niveles, incluyendo el científico. no obstante, a pesar de dicha ventaja, debemos reconocer que el inglés no está al alcance de todo el mundo y la inteligencia artificial (ia) nos ofrece buenas alternativas que deberíamos aprovechar. la traducción del artículo al griego constituye un ejemplo tangible de estas alternativas. quizás la riqueza y complejidad del idioma griego han impedido que la ia produzca una traducción “perfecta”; de hecho, hubo algunos errores gramaticales y de otro tipo, así como expresiones incompatibles con el uso natural del idioma. sin embargo, espero que esto mejore con el tiempo, dada la capacidad técnica de la ia para automejorarse. ¡felicidades a los autores por la idea y su implementación! (petros chamakiotis) 4.2 reflexiones del revisor 1 (español) además de mis comentarios anteriores sobre la versión griega, la traducción al español fue de mejor calidad en comparación con la griega, probablemente porque la inteligencia artificial (ia) está mejor "entrenada" en español que en griego, debido al mayor número de usuarios hispanohablantes. asimismo, aunque creo que esto no aplica a los países hispanohablantes de américa latina, los españoles no suelen usar términos en inglés en el uso cotidiano del idioma, y esto es algo que la ia no tuvo en cuenta. por ejemplo, el acrónimo de la ia en inglés es ai, mientras que en español es ia (ya que el término es “inteligencia artificial”). también existen varios “falsos amigos”, como la palabra “conference” cuando nos referimos a congresos académicos, cuyo término más preciso en español es “congreso” (en lugar de “conferencia”). (petros chamakiotis) australasian journal of information systems burton-jones et al. 2025, vol 29, provocation this article is not just in english 36 4.3 reflexiones del revisor 2 (bahasa indonesio) este artículo ofrece un examen exhaustivo del sesgo del idioma inglés en los entornos académicos y explora cómo las herramientas de traducción de ia podrían impulsar un cambio significativo. la principal fortaleza del artículo radica en su clara identificación del problema, respaldada por literatura de diversos campos que demuestra tanto la urgencia como los impactos negativos del sesgo lingüístico. la visión de una ciencia multilingüe es convincente y está bien articulada, con la ia representando una solución innovadora adecuada para el campo de los sistemas de información. ciertas áreas, sin embargo, podrían beneficiarse de un mayor desarrollo. si bien se mencionan las complejidades de la ia, la discusión sobre cómo abordar las limitaciones de la ia, como las alucinaciones o el sesgo cultural, necesita mayor profundidad. por ejemplo, algunas traducciones al indonesio en el primer borrador de este artículo resultan algo rígidas. la frase "our provocation to readers" (nuestra provocación a los lectores) traducida como "tantangan kepada pembaca" (un desafío para los lectores) resulta demasiado "contundente"; una versión más natural sería "kami ingin mengajak pembaca berpikir" (queremos invitar a los lectores a pensar), que suena más "suave". el análisis de la complejidad parece permanecer a un nivel superficial, particularmente en lo que respecta a cuestiones técnicas como la precisión de la traducción o el impacto cultural. los argumentos también carecen de un fuerte apoyo empírico. investigaciones futuras podrían considerar la incorporación de resultados de estudios piloto y un análisis técnico más profundo. una exploración adicional de contextos específicos de los si, como la implementación multilingüe en sistemas empresariales, también podría enriquecer la contribución de este artículo a la comunidad académica en general. (arif perdana) 4.4 reflexiones del revisor 3 (urdu) gracias por invitarme a reflexionar sobre esta perspicaz provocación. revisar este artículo en urdu fue una experiencia gratificante. era la primera vez que me involucraba con un trabajo académico en mi lengua materna, y la experiencia se sintió genuinamente inclusiva, algo que no había experimentado antes. el mensaje clave del artículo, que el conocimiento científico no debe limitarse a una sola lengua, resonó profundamente en mí. destaca cómo el dominio del inglés puede limitar, sin intención, tanto la participación como la comprensión en el discurso científico. en general, la traducción al urdu está bastante bien ejecutada. sin embargo, en algunos lugares, la traducción se siente un poco rígida y rompe la fluidez debido a un enfoque palabra por palabra en lugar de adaptar las expresiones para una fraseología natural en urdu. dicho esto, estos desafíos no disminuyen el potencial de la ciencia multilingüe; por el contrario, revelan dónde todavía queda un trabajo importante por hacer. con la aportación y supervisión continuas de los seres humanos, las herramientas de traducción de ia tienen el potencial de transformar la ciencia en algo no solo accesible a nivel mundial, sino también significativo y resonante a nivel local. (humza naseer) 5 conclusión el sesgo lingüístico ha sido una barrera persistente a lo largo de la historia de la ciencia, pero ya no tiene por qué serlo. el avance en la traducción por ia augura un futuro nuevo para la australasian journal of information systems burton-jones et al. 2025, vol 29, provocation this article is not just in english 37 ciencia: un futuro multilingüe. una ciencia multilingüe puede ser más inclusiva (permitiendo la participación de más autores y revisores) y, por ende, más influyente (permitiendo que más lectores accedan y comprendan la ciencia). como cualquier transformación digital, los efectos de esta transformación podrían ser inciertos, presentando una oportunidad significativa de aprendizaje. pocas oportunidades son tan adecuadas para el campo de is como esta, y es imperativo que actuemos de inmediato. nos proponemos compartir nuestras ideas a medida que avanzamos en nuestro trabajo e invitamos a otros a unirse a este esfuerzo. bahasa indonesia artikel ini bukan hanya dalam bahasa inggris: membuat ilmu pengetahuan lebih inklusif dan berdampak dengan penerjemahan kecerdasan buatan abstrak manusia beragam secara linguistik, tetapi ilmu pengetahuan tidak. keberhasilan akademik kerap kali mensyaratkan penguasaan bahasa inggris. setiap konferensi besar, setiap jurnal besar – bahkan jurnal ini – menganggapnya sebagai keharusan. bias bahasa inggris dalam ilmu pengetahuan sangat dominan sehingga dianggap wajar oleh sebagian besar ilmuwan dan asosiasi ilmiah, jarang dibicarakan atau diatasi. hal ini tidak adil dan menimbulkan biaya besar serta kesempatan yang terlewatkan. hal ini sebenarnya tidak perlu terjadi. alat penerjemahan kecerdasan buatan (ai) telah menjadi sangat baik dan cepat, memungkinkan kita untuk membayangkan ilmu pengetahuan multibahasa yang lebih inklusif. kami mengajak pembaca untuk mempertimbangkan: bagaimana kita seharusnya memanfaatkan alat penerjemahan ai untuk ilmu pengetahuan yang lebih berdampak dan inklusif? ini adalah tantangan yang sangat relevan bagi para sarjana sistem informasi karena melibatkan perancangan artefak dan praktik sosioteknis untuk masa depan yang lebih baik. untuk menunjukkan kemungkinan, artikel ini melalui proses tinjauan multibahasa dan diterbitkan dalam lima bahasa, semuanya didukung oleh teknologi penerjemahan ai. 1 masalah bias bahasa salah satu aspek yang paling mencolok dan jarang menjadi subjek diskusi dari australasian journal of information systems (ajis) adalah penggunaan bahasa inggris sebagai bahasa utama jurnal ini. hal ini jarang menjadi subjek diskusi atau perdebatan karena dianggap wajar. semua konferensi dan jurnal terkemuka di bidang information systems (is) menggunakan bahasa inggris, seperti halnya di setiap disiplin ilmu lainnya. keberhasilan akademis dalam bidang sains sering kali mengharuskan penguasaan bahasa inggris. namun, ini tidak adil karena sebagian besar dunia tidak berbicara bahasa inggris. dari 8 miliar orang di bumi, diperkirakan hanya 1,5 miliar yang berbicara bahasa inggris, dan hanya 25% dari mereka yang berbicara bahasa inggris sebagai bahasa pertama (icls, 2024). artikel ini bertujuan untuk meningkatkan kesadaran tentang 'bias bahasa' dalam bidang is dan mengundang seluruh komunitas untuk beraksi. kami menjelaskan bagaimana kami memulai pekerjaan ini, sebagai anggota gugus tugas association for information systems (ais), dan kami menyerukan aksi bersama dari seluruh komunitas. australasian journal of information systems burton-jones et al. 2025, vol 29, provocation this article is not just in english 38 sulit untuk mengabaikan perasaan bahwa kita tertinggal dari bidang lain dalam hal kesadaran akan bias bahasa. ini telah diangkat dalam bidang kedokteran pada tahun 1990-an (egger et al., 1997), dan sejak itu telah diangkat di banyak bidang termasuk penelitian emosi (wierzbicka, 2009), ekologi (clavero, 2011), linguistik (levisen, 2019), manajemen (pudelko & tenzer, 2019), hukum (ammann, 2022), ilmu kognitif (blasi, henrich, adamou, kemmerer, & majid, 2022), konservasi (amano & berdejo-espinola, 2024), ilmu bumi (bampi, piresoliveira, loyola-bartra, & limaribeiro, 2024), perawatan kritis (gonzález-dambrauskas, salluh, machado, & rotta, 2024), dan ortodonti (mheissen, spineli, daraqel, & alsafadi, 2024). ini juga telah diangkat dalam diskusi luas tentang masa depan penerbitan akademik (ahmed et al., 2023; amano, rojas, boum ii, calvo, & misra, 2021; arenas-castro et al., 2024; editorial, 2023). seperti banyak bias lainnya, bias bahasa menambah tekanan bagi para sarjana yang baru memulai karier mereka. seperti yang dilaporkan amano et al. (2023), para sarjana nonpenutur asli bahasa inggris harus menghabiskan lebih banyak waktu untuk membaca, menulis, dan merevisi makalah, lebih mungkin untuk ditolak makalahnya karena alasan terkait bahasa, dan mengalami hambatan dalam menghadiri konferensi. bias bahasa menambah tekanan bagi para sarjana yang baru memulai karier mereka (hohti & truman, 2021), menurunkan kualitas proses peninjauan sejawat (politzer-ahles, girolamo, & ghali, 2020), menciptakan bias dalam tinjauan literatur (nunez & amano, 2021), tinjauan sistematis (hannah, haddaway, fuller, & amano, 2023), dan meta-analisis (konno et al., 2020), serta mempengaruhi perbandingan penelitian nasional (van leeuwen, moed, tijssen, visser, & van raan, 2001). bias bahasa juga terlihat dalam pencarian google scholar, yang memprioritaskan artikel berbahasa inggris (rovira, codina, & lopezosa, 2021). sebagai bias sistemik yang telah lama ada, bias bahasa juga dapat dilihat dalam praktik nasional, seperti prevalensi pengajaran bahasa inggris di sekolah-sekolah di seluruh dunia, dan industri besar, seperti industri global usd$10-11 miliar untuk bahasa inggris sebagai bahasa kedua (esl)17. sementara kami berusaha untuk menyoroti bias linguistik yang mempengaruhi sains, kami tidak naif terhadap manfaat penyebaran bahasa inggris secara global. mengingat bahwa keragaman bahasa secara alami menghambat komunikasi (misalnya, seperti yang diilustrasikan dalam kisah menara babel yang terkenal), konvergensi menuju satu bahasa (baik bahasa inggris maupun bahasa lainnya) dapat memfasilitasi kemajuan di berbagai sektor (john & özgür, 2021). seperti yang dicatat oleh british council (2013, hlm. 3) "berkat internet, munculnya media sosial, kecepatan dan penyebaran teknologi komunikasi global dan ekonomi global yang semakin terhubung dan saling bergantung, bahasa inggris sekarang memungkinkan pertukaran cepat ide dan inovasi di seluruh dunia, dan pengembangan pasar tunggal supranasional baru dalam pengetahuan dan ide." namun, meskipun kami mengakui manfaat penyebaran bahasa inggris (dan beberapa anggota tim penulis kami telah menikmati manfaat ini, sebagai penutur asli bahasa inggris), tujuan kami di sini adalah untuk menyoroti konsekuensi yang signifikan tersebut, yang meskipun signifikan, sering kali tidak diakui dan dianggap remeh (mélitz, 2007; salomone, 2022). 17 https://www.businessresearchinsights.com/market-reports/esl-english-as-a-second-language-market117478; https://www.englishaustralia.com.au/our-sector/understanding-the-sector australasian journal of information systems burton-jones et al. 2025, vol 29, provocation this article is not just in english 39 2 janji alat terjemahan ai meskipun bias bahasa yang meluas dan mahal, hal ini tidak dapat lagi diabaikan karena alat terjemahan berbasis kecerdasan buatan (ai) semakin canggih dan cepat. penting untuk menyoroti signifikansi historis momen ini, mengingat bias bahasa sudah setua ilmu pengetahuan. namun, ilmu pengetahuan selalu dilakukan dalam satu bahasa tertentu, dan untuk berpartisipasi (membaca, berbicara, menulis), seseorang harus menguasai bahasanya (gordin, 2015). dengan lebih dari 7.000 bahasa di bumi (anderson, 2010), ilmuwan tidak mungkin menguasai semua bahasa. selama ribuan tahun, logis untuk melaksanakan ilmu pengetahuan dalam bahasa yang dominan pada masanya. bahasa dominan pada berbagai waktu dalam sejarah termasuk mesir, yunani, latin, cina, dan berbagai bahasa eropa, tetapi bahasa inggris telah mendominasi sejak perang dunia ii (gordin 2015). meskipun ada upaya singkat (dan gagal) untuk menciptakan bahasa universal (misalnya, esperanto) (patterson & huff, 1999), satu-satunya solusi yang tersedia selama bertahun-tahun untuk mengatasi perbedaan bahasa adalah terjemahan. namun, meskipun telah ada sebagai bidang selama lebih dari 2.000 tahun (dynamic_language, 2024), terjemahan, sejauh yang kami ketahui, belum pernah diterima secara luas dalam ilmu pengetahuan, sebagian besar karena proses terjemahan yang mahal dan memakan waktu (wood, 1967). sepanjang sejarah, berbagai jurnal telah diterbitkan dalam beberapa bahasa, dan bahkan saat ini ada sekitar 500 jurnal multibahasa18, tetapi ini adalah pengecualian yang membuktikan aturan. seperti yang diungkapkan gordin (2015, hlm. 3) dengan penuh makna, “jika anda tertarik dengan bagaimana rasanya hidup di dunia dengan satu bahasa komunikasi, dunia tanpa babel, anda harus melihat kepada ilmuwan alam. mereka berasal dari sana.” sebagian besar jurnal hanya dalam bahasa inggris (gordin, 2015), dan sebagian besar jurnal multibahasa hanya jurnal dua bahasa antara bahasa inggris dan bahasa lain, dan sebagian besar dari mereka hanya bilingual karena undang-undang nasional yang mengharuskannya (misalnya, jurnal yang diterbitkan di kanada, yang merupakan negara bilingual). potensi alat terjemahan ai dalam mengatasi bias bahasa dalam ilmu pengetahuan baru-baru ini mulai muncul (berdejo-espinola & amano, 2023; crotty, 2024; steigerwald et al., 2022), yang berasal dari dua transformasi jangka panjang. salah satunya adalah transformasi digital bertahap dalam penelitian itu sendiri. belum lama ini, banyak tugas penelitian dilakukan secara manual. peneliti pergi ke perpustakaan, membaca, dan memberi anotasi pada makalah fisik, mengirim makalah dan ulasan melalui pos, bahkan melakukan perhitungan dan menulis makalah dengan tangan. sekarang, hampir semua tugas dilakukan secara online: mencari, membaca, menulis, memberi anotasi, berkolaborasi, menganalisis, meninjau, mengirimkan, menerbitkan. konsekuensi mendalam dari transformasi ini adalah bahwa banyak aspek ilmu pengetahuan kini melibatkan bahasa pada platform digital. artinya, makalah ilmiah ditulis, ditinjau, diterbitkan, dibaca, dan dicari dalam bahasa (sekarang hampir selalu dalam bahasa inggris), dan semua ini terjadi di platform digital yang besar dan saling terhubung. beberapa platform ini sekarang sangat luas, misalnya, scopus menampung lebih dari 90 juta catatan19 dan google scholar mencari di lebih dari 300 juta catatan (martin-martin, thelwall, ordunamalea, & lopez-cozar, 2021). dan di sinilah transformasi kedua masuk – transformasi ai. 18 https://en.wikipedia.org/wiki/category:multilingual_journals 19https://assets.ctfassets.net/o78em1y1w4i4/76uvtuilkssqe4h4h1pzkq/e5ff8eef9694f216a5c29e2ebf80fa51/ fast-facts-about-elsevier.pdf australasian journal of information systems burton-jones et al. 2025, vol 29, provocation this article is not just in english 40 meskipun model bahasa dalam ai telah ada sejak tahun 1950-an, perkembangan pesat model bahasa besar dimungkinkan oleh pertumbuhan data online yang masif untuk pembelajaran (naik, naik, & naik, 2024) bersama dengan arsitektur baru, terutama arsitektur transformer, yang dapat memanfaatkan data tersebut (vaswani et al., 2017). dalam lima tahun terakhir, dua transformasi ini telah memungkinkan bentuk baru ai muncul, memungkinkan kemajuan yang tampaknya mustahil beberapa waktu lalu. dan kemajuan ini terutama cepat dalam terjemahan bahasa. tugas terjemahan bahasa adalah tugas pertama yang diuji pada arsitektur transformer dan karenanya sangat cocok untuk itu (vaswani et al., 2017). kami tidak mengklaim bahwa terjemahan ai dapat sepenuhnya dan dengan cepat mengatasi bias bahasa. akan ada banyak kompleksitas (amano dan berdejo-espinola 2024). namun potensi tersebut cukup kuat bagi kami untuk memiliki kewajiban untuk mengejarnya. dari perspektif asosiasi ilmiah, seperti association for information systems, akan menjadi kekurangan jika kami tidak berupaya mengurangi bias sistematis yang meluas dan mahal yang dihadapi anggota bidang ilmiah kami dan sama lalainya jika kami tidak mencoba memanfaatkan peluang baru untuk inklusi dan dampak yang dapat ditawarkan oleh terjemahan ai. itulah sebabnya kami membentuk gugus tugas kami – association for information systems taskforce on ai translation for inclusive, impactful science. 3 kompleksitas ada banyak kompleksitas dalam penggunaan alat penerjemahan ai dalam sains. kami menyoroti enam kategori kompleksitas. pertama, ada kompleksitas yang terkait dengan penentuan ruang lingkup masalah. ruang lingkup paling sederhana adalah fokus pada penerjemahan satu makalah dari satu bahasa ke bahasa lain, tetapi bahkan ruang lingkup ini menimbulkan pertanyaan. misalnya, apakah kami harus menerjemahkan hanya makalah yang diterima, atau juga makalah yang diajukan? yang pertama akan membantu sains dengan mempromosikan dampak—meningkatkan jumlah pembaca makalah—sementara yang terakhir juga akan mendukung inklusi— memungkinkan lebih banyak penulis dan peninjau menulis dalam bahasa ibu mereka. meskipun pilihan terakhir memiliki manfaat yang lebih besar, pilihan ini juga membutuhkan lebih banyak usaha, memerlukan penerjemahan makalah dan ulasannya, dan memerlukan sistem yang memungkinkan penulis dan peninjau memverifikasi kualitas terjemahan tanpa beban tambahan selama proses peninjauan. setelah makalah diterima atau dipublikasikan, pertanyaan lain adalah apakah kami harus fokus pada penerjemahan ke satu atau beberapa bahasa, dan siapa yang memilih bahasa, penulis atau pembaca. jika pembaca yang memilih, apakah terjemahan perlu disiapkan terlebih dahulu (misalnya, dalam sejumlah kecil bahasa umum) dan pembaca memilih bahasa yang mereka sukai dari daftar ini, atau apakah pembaca dapat meminta terjemahan dalam bahasa pilihan mereka (baik dari daftar tetap, atau dalam dunia ideal, dari bahasa apa pun) yang kemudian dihasilkan secara langsung oleh platform publikasi atas permintaan? yang terakhir akan memungkinkan jangkauan bahasa yang lebih luas sementara yang pertama akan memungkinkan langkah verifikasi untuk kualitas terjemahan. jika yang terakhir dipilih, mungkin ada perubahan kecil dalam makna yang dapat diperkenalkan dalam beberapa terjemahan tanpa pengetahuan penulis, menciptakan komplikasi untuk kutipan selanjutnya dari karya tersebut. makalah yang diterbitkan dengan opsi terjemahan langsung sesuai permintaan mungkin memerlukan catatan penafian yang mengakui potensi kesalahan. australasian journal of information systems burton-jones et al. 2025, vol 29, provocation this article is not just in english 41 pertanyaan ruang lingkup lain adalah apakah kami melihat lebih dari sekadar menerjemahkan satu makalah dan mempertimbangkan penerjemahan seluruh perpustakaan/platform makalah, seperti seluruh elibrary association for information systems, yang memuat lebih dari 65.000 makalah? memperluas ruang lingkup ke perpustakaan tidak hanya meningkatkan jumlah makalah yang perlu diterjemahkan, tetapi juga menambahkan pertimbangan tambahan. misalnya, ini dapat memperluas rentang waktu penerjemahan, karena perpustakaan mungkin memiliki makalah yang diterbitkan selama beberapa dekade, sehingga proses penerjemahan mungkin perlu mempertimbangkan perubahan dalam konteks budaya dan leksikon suatu bidang dari waktu ke waktu. demikian pula, ini mungkin memerlukan pengembangan alat bantu tambahan, seperti alat pencarian multibahasa untuk mencari di seluruh perpustakaan dalam berbagai bahasa. pertanyaan ruang lingkup terakhir berkaitan dengan kemampuan ai. mengingat bahwa teknologi ai yang sama yang melakukan terjemahan menawarkan kemampuan terkait, haruskah kami menggunakan kemampuan ini juga? misalnya, haruskah kami mempertimbangkan produksi ringkasan makalah dalam berbagai bahasa selain penerjemahan (misalnya, ringkasan awam untuk eksekutif atau siswa)? demikian pula, haruskah kami mempertimbangkan perbaikan bahasa, bukan hanya terjemahan, misalnya, menawarkan saran untuk meningkatkan teks, bukan hanya menerjemahkan teks 'sebagaimana adanya'? jika kami mempertimbangkan opsi terakhir, kami dapat menyusun ulang permasalahan dari sekadar 'terjemahan' menjadi 'penjelasan makna' di mana terjemahan hanyalah salah satu elemen dari penjelasan makna dan elemen lainnya dapat mencakup perbaikan dalam tata bahasa, alur, dan gaya. seperti yang anda lihat, ruang lingkup masalah dapat sangat bervariasi. pada tahap ini, gugus tugas kami mengadopsi ruang lingkup menengah. kami berusaha mendukung seluruh proses peninjauan—dari pengajuan hingga penerimaan. kami juga berusaha mendukung terjemahan ke dalam berbagai bahasa. namun, kami membatasi upaya awal kami pada 10 bahasa utama (bahasa indonesia, urdu, yunani, prancis, melayu, korea, mandarin, jerman, spanyol, dan portugis) dan fokus pada pengembangan terjemahan yang disiapkan sebelumnya yang memungkinkan verifikasi kualitas oleh manusia daripada memungkinkan terjemahan real-time sesuai permintaan. kami memasukkan produksi ringkasan makalah dalam ruang lingkup kami. namun, kami hanya fokus pada penerjemahan makalah individu daripada seluruh perpustakaan/platform, dan kami tidak mempertimbangkan aspek yang lebih luas seperti perbaikan bahasa dalam fase pertama ini. selain kompleksitas yang muncul karena pertanyaan ruang lingkup, ada lagi serangkaian kompleksitas yang muncul dari praktikalitas lingkungan publikasi. misalnya, bagaimana kami harus menangani digital object identifiers (dois)? saat ini, setiap makalah yang diterbitkan harus diberi doi unik karena ini memungkinkan mesin pencari dan agregator untuk menemukan dan mengidentifikasi kutipan untuk setiap makalah. namun, jika kami membuat beberapa terjemahan dari satu makalah, apakah ini akan dimasukkan dalam dokumen yang sama (misalnya, halaman web, pdf) dengan doi yang sama, dan apakah kutipan dari makalah tersebut akan digabungkan terlepas dari versi (terjemahan) yang dikutip? salah satu kesulitan dalam menjawab pertanyaan-pertanyaan ini adalah tidak adanya entitas tunggal (asosiasi ilmiah, penerbit, atau perusahaan teknologi besar) yang mengendalikan seluruh ekosistem publikasi, dan sulit untuk memprediksi bagaimana australasian journal of information systems burton-jones et al. 2025, vol 29, provocation this article is not just in english 42 semuanya akan berkembang. oleh karena itu, kami harus memilih praktik terjemahan yang akan selaras dengan praktik yang ada sambil cukup fleksibel untuk beradaptasi. kompleksitas praktis serupa muncul dengan hak cipta. terjemahan dapat dianggap sebagai 'karya hasil turunan' dan oleh karena itu harus mematuhi pembatasan hak cipta atas karya semacam itu. masalah praktis lainnya adalah plagiarisme. ironisnya, memfasilitasi terjemahan mungkin juga memfasilitasi plagiarisme. ini karena plagiarisme biasanya didefinisikan dan dideteksi dengan penggunaan urutan kata yang sama dalam sebuah makalah, yaitu penyalinan kata demi kata. terjemahan tidak harus kata demi kata, tetapi dapat berdasarkan makna (menyampaikan makna yang sama daripada kata yang sama) (sun, 2013). jika seorang plagiator memplagiat sebuah makalah dengan membuat terjemahan menggunakan terjemahan berdasarkan makna, tidak jelas apakah alat deteksi plagiarisme saat ini akan mendeteksinya. alat semacam itu mungkin perlu direvisi untuk mengendalikan risiko ini. serangkaian kompleksitas ketiga muncul dari karakteristik ai. meskipun kemajuan pesat ai memotivasi kami untuk mengejar proyek ini, ai masih belum sempurna. halusinasi masih terjadi (ji et al., 2023), dan kinerja penerjemahan ai bervariasi tergantung pada jumlah data pelatihan dalam setiap bahasa (xu et al., 2024). karena dominasi bahasa inggris yang sangat kuat di internet, model gen ai mungkin tidak hanya menyematkan bias bahasa inggris tetapi juga bias budaya (luo & puett, 2024). kami mengundang para peneliti di bidang kami untuk berkontribusi dalam menciptakan artefak dan praktik sosioteknis yang nyata guna menyelesaikan tantangan ini. dalam melakukannya, peneliti seharusnya tidak tergoda oleh determinisme teknologi atau determinisme sosial semata—pandangan sosioteknis sangat penting. artinya, seperti teknologi lainnya, keberhasilan penerjemahan ai akan bergantung tidak hanya pada teknologi tetapi juga pada bagaimana pengguna (penulis, editor, peninjau, pembaca) memandang dan menggunakannya. misalnya, kecenderungan manusia seperti penolakan terhadap algoritma dan ketidakpercayaan akan perlu dipertimbangkan (asscher & glikson, 2023). menjalankan model ai juga memiliki biaya finansial, menimbulkan pertanyaan tentang siapa yang harus menanggung biaya tersebut. salah satu alasan kami mengejar ini sebagai gugus tugas ais adalah bahwa kami sedang mempertimbangkan apakah dan bagaimana asosiasi ilmiah harus menanggung biaya tersebut untuk anggotanya. serangkaian kompleksitas keempat muncul karena nuansa dalam bahasa tertulis yang dapat terlewatkan oleh model penerjemahan ai saat ini. misalnya, dalam proses peninjauan makalah ini, salah satu peninjau kami menemukan kesalahan dalam terjemahan akronim. misalnya, di spanyol, penulis umumnya menggunakan versi spanyol dari akronim (ia), tetapi di yunani, penulis sama-sama terbiasa menggunakan versi bahasa inggris (ai) atau versi yunani (tn), dan penulis seharusnya hanya menggunakan satu secara konsisten. terjemahan kami gagal melakukan ini. sementara itu, peninjau kami yang lain memperhatikan bahwa terjemahan kami terkadang terdengar terlalu "kaku," "berlebihan," atau "formal" dan bahwa kami seharusnya mengadopsi nada yang lebih "lembut" atau lebih "alami". ini mungkin disebabkan oleh penerjemahan ai yang secara otomatis memilih nada yang terlalu formal, yang tidak kami sadari sebelumnya. untuk mengatasi tantangan ini, kami aktif menyempurnakan pendekatan penerjemahan kami. secara khusus, kami telah mengintegrasikan umpan balik dari peninjau makalah ini, terutama mengenai evaluasi kealamian budaya dan nada. akibatnya, pendekatan penerjemahan kami sekarang memberikan perhatian lebih besar pada aspek-aspek ini. misalnya, log alat internal telah menangkap saran yang terinspirasi oleh peninjau seperti: "frasa '...' cukup langsung. australasian journal of information systems burton-jones et al. 2025, vol 29, provocation this article is not just in english 43 pertimbangkan untuk mengubahnya menjadi nada yang lebih akademis, seperti '...'." atau "frasa '...' agak literal. frasa akademis yang lebih cocok mungkin '...'." meskipun kekuatan penerjemahan ai yang berkembang pesat, sulit membayangkan bahwa alat ini akan menawarkan terjemahan sempurna di setiap bahasa dan disesuaikan untuk setiap makalah, tetapi kami berencana untuk menangani sebanyak mungkin masalah ini (misalnya, dengan memodifikasi arsitektur agen model ai untuk menggunakan konteks dengan lebih efektif). serangkaian kompleksitas kelima muncul dari institusi sains yang dapat memperkuat status quo dan menghambat munculnya sains multibahasa. misalnya, ketika kami membahas inisiatif ini dengan peneliti sukses yang bukan penutur asli bahasa inggris, kami terkejut banyak yang mengatakan mereka tidak akan mengambil kesempatan untuk menulis dalam bahasa asli mereka karena mereka lebih baik dalam menulis makalah dalam bahasa inggris (karena mereka telah berlatih sepanjang hidup kerja mereka dalam bahasa inggris). dalam beberapa hal, ini tidak mengkhawatirkan kami karena populasi penutur non-bahasa inggris yang dapat mengambil manfaat jauh lebih besar daripada populasi kecil penutur non-bahasa inggris yang telah 'berhasil' dalam sistem yang didominasi bahasa inggris. di sisi lain, anggota senior dari komunitas secara alami mempengaruhi anggota junior, sehingga pandangan mereka dapat memperkuat status quo untuk beberapa waktu. sebagai contoh, kami telah bekerja dengan peneliti senior di salah satu negara eropa dalam inisiatif ini. meskipun mereka sangat mendukung inisiatif kami, mereka mendorong kami untuk bergerak perlahan karena mereka telah bekerja selama puluhan tahun untuk meningkatkan penulisan bahasa inggris di komunitas mereka, dan upaya itu sangat berhasil, menyebabkan peningkatan tajam dalam jumlah publikasi di jurnal-jurnal terkemuka. hal terakhir yang diinginkan oleh para peneliti ini adalah mengurangi keberhasilan tersebut. seperti yang ditunjukkan oleh contoh ini, upaya untuk mendorong sains multibahasa perlu dirancang dengan hati-hati, dengan mempertimbangkan inisiatif lain yang sudah ada yang dirancang di sekitar status quo. akhirnya, kami menghadapi serangkaian kompleksitas keenam yang lebih filosofis. salah satu cara untuk melihat ini adalah dengan memikirkan sebuah makalah sebagai hanya satu langkah dalam proses panjang dari sebuah ide yang terbentuk dalam pikiran penulis hingga diapresiasi dalam pikiran pembaca. sebagai permulaan, pertimbangkan seorang penulis yang memikirkan sebuah ide. hipotesis sapir-whorf menyarankan bahwa pemikiran penulis dibentuk oleh bahasa penulis (lucy, 1998). jika benar, ini menyiratkan bahwa bias bahasa kembali ke asal, bahkan membatasi ide-ide yang dapat dikembangkan oleh penulis. kita dapat menyebut ini sebagai masalah 'konsepsi'. masalah kemudian berlanjut karena meskipun sebuah bahasa dapat digunakan untuk menyampaikan sebuah ide, seorang penulis mungkin masih kekurangan keterampilan bahasa untuk melakukannya. kita dapat menyebut ini sebagai masalah 'artikulasi'. dan bahkan jika seorang penulis memiliki bahasa yang tepat untuk menyampaikan sebuah ide, dan menggunakannya dengan mahir, seorang pembaca mungkin tidak menafsirkan kata-kata dengan cara yang sama. kita dapat menyebut ini sebagai masalah 'interpretasi'. kami yakin bahwa pembaca makalah ini akrab dengan masalah semacam itu karena masalah ini umum dalam kehidupan sehari-hari. secara kolektif, masalah-masalah ini berarti bahwa makalah penelitian secara tak terelakkan adalah media yang tidak sempurna untuk menyampaikan ide-ide ilmiah dalam komunitas ilmuwan. ketidaksempurnaan makna hanyalah bagian dari proses, terlepas dari bahasa yang digunakan. jelas, alat penerjemahan ai tidak dapat menyelesaikan semua masalah ini. sebagai satu contoh saja, alat ai dapat membantu mengurangi masalah konsepsi dengan memungkinkan orang berpikir dalam bahasa mereka sendiri, tetapi mereka juga mungkin australasian journal of information systems burton-jones et al. 2025, vol 29, provocation this article is not just in english 44 gagal melakukannya karena dominasi bahasa inggris telah begitu kuat selama ini sehingga beberapa bahasa tidak memiliki leksikon yang diperlukan (barath, 2019). 4 sebuah kesempatan dan pendekatan kami meskipun kami telah mencantumkan banyak kompleksitas, kami memandangnya sebagai peluang untuk memberikan kontribusi, bukan sebagai masalah yang harus dihindari. bagi kami, ini menekankan mengapa ini adalah tantangan yang ideal bagi bidang is, karena peluang dan kompleksitasnya sangat bersifat sosioteknis (sarker, chatterjee, xiao, & elbanna, 2019). seperti yang kami sebutkan di awal, bidang is bukanlah yang pertama menyadari bias bahasa. namun demikian, meskipun banyak bidang telah meningkatkan kesadaran akan masalah ini, hanya sedikit yang secara sistematis menanganinya. akibatnya, bidang is memiliki kesempatan untuk membantu bekerja di garis depan ilmu pengetahuan bersama dengan bidang terkemuka lainnya20. mengingat bahwa subjek ini sangat terkait dengan sifat bidang kami sebagai bidang yang didedikasikan untuk memahami dan merancang artefak dan praktik sosioteknis untuk masa depan yang lebih baik kami percaya bahwa bidang is tidak hanya memiliki kesempatan untuk membuat perbedaan, tetapi juga memiliki kewajiban untuk melakukannya. termotivasi oleh peluang ini, kami membentuk ais taskforce pada tahun 2024. sejalan dengan model perubahan umum (lewin, 1951), kami berasumsi bahwa upaya substansial akan diperlukan untuk memfasilitasi proses 'perubahan' daripada mengasumsikan bahwa bidang ini dapat berubah dengan segera. untuk memfasilitasi proses 'perubahan', kami mengadakan panel dan pleno di empat konferensi besar untuk membangun kesadaran dan memotivasi diskusi (burton-jones, 2025; burton-jones, chau, feng, kankanhalli, & vogel, 2024; pouloudi et al., 2024; sarker, ahuja, feng, jarvenpaa, & seidel, 2024). kami juga berbicara dengan beberapa bab ais yang mewakili komunitas bahasa yang berbeda dan kami bekerja pada pengembangan dan pengujian awal alat terjemahan berbasis ai. pada tahun 2025, kami melanjutkan upaya kami untuk meningkatkan kesadaran akan masalah ini (seperti melalui makalah ini) sambil juga mengambil sikap yang lebih aktif dengan siklus desain, pengujian, dan pembelajaran. misalnya, kami mengapresiasi kesediaan editor dan peninjau di ajis untuk menguji proses tinjauan multibahasa ini, dari mana kami dapat belajar dan meningkatkan. kami secara bersamaan terlibat dalam pekerjaan lain untuk menyediakan data dan pengujian konkret. misalnya, kami menguji terjemahan ai dalam americas conference on information systems (amcis) pada tahun 2025 (lihat: https://aisel.aisnet.org/amcis2025/translated/). kami juga bekerja dengan konferensi lain pada tahun 2025 pada berbagai kasus penggunaan terjemahan21. kami terutama fokus pada tiga kasus penggunaan: (1) terjemahan ai dalam proses tinjauan; (2) terjemahan ai dari makalah yang diterima; dan (3) persiapan ringkasan makalah dalam berbagai bahasa untuk menyebarkan dampak. kami juga akan melakukan survei, wawancara, dan kelompok fokus untuk mempelajari bagaimana bias bahasa mempengaruhi para ilmuwan di bidang is dan 20 suara-suara terkemuka lainnya di garis depan ini mencakup unesco (https://doi.org/10.54677/mnmh8546) dan dewan uni eropa (https://www.consilium.europa.eu/media/56958/st10126-en22.pdf). 21 kami mengucapkan terima kasih kepada ketua konferensi dan ketua program dari amcis 2025, pacis 2025, dan wi 2025 atas kesediaan mereka untuk terlibat dalam inisiatif ini. australasian journal of information systems burton-jones et al. 2025, vol 29, provocation this article is not just in english 45 potensi dampak alat terjemahan ai. kami juga berinteraksi dengan para ilmuwan di bidang lain untuk mendukung gerakan sains yang lebih luas (amano, bowker, & burton-jones, 2025). dalam mengejar langkah-langkah ini, kami menyadari bahwa kami akan menghadapi banyak tantangan, dan bahwa banyak peluang lebih lanjut masih ada. kami juga dapat belajar dari gerakan sains lainnya, seperti gerakan sains terbuka (ferguson et al., 2023), yang juga termotivasi untuk meningkatkan inklusivitas dan dampak dalam sains. seperti upaya perubahan lainnya, kami akan membuat kesalahan dan harus melakukan penyimpangan dalam rencana kami. namun demikian, kami tetap bersemangat dengan peluang yang ada, dan kami berharap para peneliti is lainnya juga akan antusias, dan bahwa bersama-sama kita dapat membuat perubahan yang berarti. daripada ini menjadi upaya soliter, terkontrol ketat, dari atas ke bawah, kami mengajak seluruh komunitas untuk bersama-sama menangani berbagai aspek dari masalah ini, dan kami mendesak editor dan ketua konferensi di seluruh bidang untuk mendukung upaya dari bawah ke atas ini. sebagai langkah pertama, kami sangat menghargai editor ajis yang memungkinkan proses tinjauan multibahasa ini. prosesnya adalah sebagai berikut: (1) kami menyerahkan naskah dalam bahasa inggris; (2) editor yang menangani mengidentifikasi peninjau yang relevan dan bahasa yang terkait dengan mereka (dalam kasus kami, peninjau 1 diidentifikasi dengan bahasa yunani dan spanyol, peninjau 2 diidentifikasi dengan bahasa indonesia, dan peninjau 3 diidentifikasi dengan bahasa urdu); (3) kami menerjemahkan makalah ke dalam empat bahasa menggunakan sistem terjemahan ai kami dan mengirimkan semua lima versi kepada editor, yang kemudian mengirimkan makalah dalam bahasa inggris dan bahasa lain yang relevan kepada masing-masing peninjau; (4) para peninjau menulis ulasan mereka dalam bahasa pilihan mereka; peninjau 1 menulis ulasannya dalam bahasa inggris, sementara peninjau 2-3 menulis ulasan mereka dalam bahasa asli mereka; (5) kami menerjemahkan ulasan non-inggris ke dalam bahasa inggris untuk membacanya, dan kemudian merevisi makalah untuk memperhitungkan saran peninjau, menerjemahkan makalah kembali ke semua bahasa, menulis dokumen tanggapan kami dan menerjemahkannya ke semua bahasa juga, dan kemudian mengirimkan kembali makalah dan tanggapan; (6) editor kemudian menerima makalah dan kami menyertakan komentar ulasan yang tertera di bawah ini. langkah terakhir dengan jurnal juga melibatkan diskusi tentang bagaimana mengelola doi, dan kami sepakat dalam proses percobaan ini untuk menggunakan satu doi dan satu file pdf saja, untuk memudahkan. dalam semangat keterbukaan, setelah makalah diterima, kami meminta para peninjau makalah untuk memberikan komentar mereka sendiri yang belum diedit di bawah ini, untuk berbagi refleksi mereka dengan komunitas. demikian pula, mereka mencantumkan nama mereka di bawah ini dalam semangat keterbukaan yang serupa. 4.1 komentar reflektif dari reviewer 1 (yunani) mampu meninjau artikel ini dalam tiga bahasa, termasuk bahasa ibu saya, yaitu bahasa yunani, merupakan pengalaman yang sangat menyenangkan. karya ini memungkinkan saya untuk merenungkan fakta bahwa kita harus bersyukur atas kesempatan yang diberikan oleh bahasa inggris untuk terhubung dengan orang lain di seluruh dunia pada berbagai tingkatan, termasuk tingkat ilmiah. namun, terlepas dari manfaat ini, kita harus mengakui bahwa bahasa inggris tidak dapat diakses oleh semua orang, dan kecerdasan buatan (ai) menawarkan alternatif penting yang dapat kita manfaatkan, dan terjemahan bahasa yunani adalah contoh nyatanya. kekayaan dan kompleksitas bahasa yunani mungkin tidak memungkinkan terjemahan yang "sempurna", karena memang ada beberapa kesalahan tata australasian journal of information systems burton-jones et al. 2025, vol 29, provocation this article is not just in english 46 bahasa dan kesalahan lainnya, serta ekspresi yang tidak sesuai dengan penggunaan alami bahasa tersebut oleh penutur asli. namun, saya berharap ini akan teratasi dalam waktu dekat mengingat kapasitas ai untuk perbaikan diri. selamat kepada para penulis atas ide dan penulisan artikel ini! (petros chamakiotis) 4.2 komentar reflektif dari reviewer 2 (spanyol) selain komentar saya di atas mengenai versi bahasa yunani, terjemahan ke dalam bahasa spanyol memiliki kualitas yang lebih baik dibandingkan dengan bahasa yunani, mungkin karena kecerdasan buatan (ai) lebih "terlatih" dalam bahasa spanyol dibandingkan bahasa yunani, karena jumlah pengguna berbahasa spanyol yang lebih banyak. juga, meskipun saya yakin ini tidak terjadi di negara-negara berbahasa spanyol di amerika latin, orang spanyol tidak menggunakan istilah bahasa inggris dalam bahasa sehari-hari, dan ini adalah sesuatu yang tidak diperhitungkan oleh ai. misalnya, akronim untuk ai dalam bahasa inggris adalah ai, sedangkan dalam bahasa spanyol adalah ia (karena istilahnya adalah "inteligencia artificial"). selain itu, ada banyak "teman palsu" (false friends) yang memengaruhi kualitas versi terjemahan. salah satunya adalah kata "conference" ketika merujuk pada konferensi akademik, yang dalam bahasa spanyol lebih akurat diterjemahkan sebagai "congreso" (bukan "conferencia"). (petros chamakiotis) 4.3 komentar reflektif dari review 3 (bahasa indonesia) paper ini secara komprehensif mengulas isu penting terkait bias bahasa inggris dalam dunia ilmiah dan bagaimana perangkat penerjemah ai bisa membawa perubahan besar. kekuatan utama dari paper ini adalah kemampuannya mengidentifikasi masalah secara jelas dengan dukungan literatur dari berbagai bidang, menunjukkan urgensi dan dampak negatif bias bahasa. visi ilmu pengetahuan multibahasa yang ditawarkan kuat dan meyakinkan, dengan pemilihan ai sebagai solusi inovatif yang tepat untuk bidang sistem informasi. namun, beberapa aspek masih perlu penajaman. meskipun kompleksitas ai telah disinggung, penjelasan mengatasi keterbatasan ai seperti halusinasi atau bias budaya masih perlu diperdalam. sebagai contoh, beberapa kalimat terjemahan bahasa indonesia di draf pertama artikel ini terkesan kaku. frasa "our provocation to readers" yang diterjemahkan sebagai "tantangan kepada pembaca" terasa terlalu "kuat" versi yang lebih natural adalah "kami ingin mengajak pembaca berpikir" yang terdengar lebih "lembut". analisis kompleksitas agaknya masih di permukaan, terutama untuk isu teknis seperti akurasi terjemahan atau dampak budaya. argumen juga belum didukung data empiris yang kuat. riset ke depan, mungkin dapat dipertimbangkan untuk melengkapi dengan hasil studi percontohan dan analisis teknis yang lebih mendalam. eksplorasi lebih lanjut tentang konteks spesifik si, seperti implementasi multibahasa dalam sistem enterprise, juga berpotensi memperkaya kontribusi paper ini untuk komunitas akademik yang lebih luas. (arif perdana) 4.4 komentar reflektif dari reviewer 4 (urdu) terima kasih telah mengundang saya untuk merefleksikan provokasi yang penuh wawasan ini. menelaah artikel ini dalam bahasa urdu merupakan pengalaman yang memuaskan. ini adalah pertama kalinya saya terlibat dengan karya akademik dalam bahasa ibu saya, dan pengalaman tersebut terasa benar-benar inklusif—sesuatu yang belum pernah saya alami sebelumnya. pesan utama dari makalah ini, bahwa pengetahuan ilmiah tidak seharusnya dibatasi oleh satu bahasa saja, sangat menggugah bagi saya. hal ini menyoroti bagaimana australasian journal of information systems burton-jones et al. 2025, vol 29, provocation this article is not just in english 47 dominasi bahasa inggris secara tidak sengaja dapat membatasi partisipasi dan pemahaman dalam diskursus ilmiah. secara keseluruhan, terjemahan ke dalam bahasa urdu cukup baik dan dieksekusi dengan baik. namun, di beberapa bagian, terjemahannya terasa agak kaku dan mengganggu kelancaran bacaan karena pendekatan terjemahan kata demi kata, alih-alih menyesuaikan ekspresi agar sesuai dengan frasa alami dalam bahasa urdu. meskipun demikian, tantangantantangan ini tidak mengurangi potensi sains multibahasa; justru sebaliknya, hal tersebut menunjukkan di mana masih banyak pekerjaan penting yang perlu dilakukan. dengan masukan dan pengawasan manusia yang berkelanjutan, alat terjemahan berbasis ai memiliki potensi untuk mentransformasi sains menjadi sesuatu yang tidak hanya dapat diakses secara global, tetapi juga bermakna dan relevan di tingkat lokal. (humza naseer) 5 kesimpulan bias bahasa telah menjadi tantangan sepanjang sejarah ilmu pengetahuan, tetapi tidak perlu lagi demikian. kemajuan dalam penerjemahan ai membuka peluang untuk masa depan ilmu pengetahuan yang multibahasa. ilmu pengetahuan multibahasa akan lebih inklusif (memungkinkan partisipasi lebih banyak penulis dan peninjau) dan lebih berdampak (memungkinkan lebih banyak pembaca mempelajari ilmu pengetahuan). seperti transformasi digital lainnya, dampak perubahan ini sulit diprediksi, dan banyak yang harus dipelajari. peluang ini sangat relevan untuk bidang is, dan inilah saat yang tepat untuk bertindak. kami berharap dapat berbagi wawasan seiring dengan kemajuan pekerjaan kami, dan mengajak banyak pihak untuk bergabung dalam upaya ini. urdu یہ مضمون انگریزی تک محدود نہیں ہے :مصنوعی ذہانت کے ترجمے کے ذریعے سائنس کو زیادہ جامع اور مؤثر بنانا خالصہ بیان خالصہ انسانی معاشرہ لسانی طور پر متنوع ہے، مگر سائنس میں یہ تنوع نہیں پایا جاتا۔ تعلیمی کامیابی کے لیے انگریزی زبان پر عبور حاصل کرنا ضروری ہے۔ ہر بڑی کانفرنس، ہر بڑا جریدہ – یہاں تک کہ یہ بھی – اسے فرض کرتا ہے۔ سائنس میں ،انگریزی زبان کا تعصب اتنا مضبوط ہے کہ زیادہ تر سائنسدانوں اور سائنسی انجمنوں کے ذریعہ اسے معمول سمجھا جاتا ہے اس پر کبھی بات نہیں کی جاتی اور نہ ہی اس کا ازالہ کیا جاتا ہے۔ یہ غیر منصفانہ ہے اور اس کے نتیجے میں بڑے اخراجات کے ترجمے کے اوزار تیزی سے ترقی کر (ai) اور مواقع کا ضیاع ہوتا ہے۔ یہ غیر ضروری بھی ہے کیونکہ مصنوعی ذہانت رہے ہیں، جو ہمیں ایک کثیر لسانی سائنس کے امکانات کو روشن کرتے ہیں۔ ہمارے قارئین کے لیے سوال یہ ہے :زیادہ مؤثر ترجمے کے اوزاروں کا استعمال کیسے کیا جائے؟ یہ ایک چیلنج ہے جو انفارمیشن ai اور جامع سائنس کے حصول کے لیے سسٹمز کے ماہرین کے لیے انتہائی موزوں ہے کیونکہ اس میں بہتر مستقبل کے لیے سماجی تکنیکی نوادرات اور طریقوں کو ڈیزائن کرنا شامل ہے۔ عملیت کے اظہار کے لیے، یہ مضمون کثیر لسانی جائزہ عمل سے گزر کر پانچ زبانوں میں شائع ہوا، جو سب مصنوعی ذہانت کے ترجمے کے ذریعے ممکن ہوا۔ زبان کی تعصب کا مسئلہ 1 آسٹریلیشین جرنل آف انفارمیشن سسٹمز کا ایک نمایاں اور کم زیر بحث پہلو یہ ہے کہ یہ انگریزی زبان میں ہے۔ اس پر آسٹریلیشین کا ایک نمایاں اور کم زیر بحث پہلو یہ ہے کہ یہ ایک انگریزی زبان کا جرنل ہے۔ اس پر (ajis) جرنل آف انفارمیشن سسٹمز کے میدان میں تمام (is) بات نہیں کی جاتی یا بحث نہیں کی جاتی کیونکہ یہ انتخاب قدرتی سمجھا جاتا ہے۔ انفارمیشن سسٹمز australasian journal of information systems burton-jones et al. 2025, vol 29, provocation this article is not just in english 48 اعلی کانفرنسیں اور جرنلز انگریزی میں ہیں، جیسے کہ وہ ہر دوسرے شعبے میں ہیں۔ سائنس میں کامیابی انگریزی پر منحصر سمجھی جاتی ہے۔ تاہم، یہ غیر منصفانہ رویہ ہے، کیونکہ دنیا کی زیادہ تر آبادی انگریزی نہیں بولتی۔ زمین پر 8 ارب لوگوں میں سے، اندازہ لگایا گیا ہے کہ صرف 1.5 ارب انگریزی بولتے ہیں، اور ان میں سے صرف 25 %انگریزی کو پہلی زبان کے (icls, 2024) طور پر بولتے ہیں کے میدان میں 'زبان کے تعصب 'کے مسئلے کے بارے میں آگاہی پیدا کرنا اور اس کے حل کے لیے is ۔ اس تحریک کا مقصد ٹاسک فورس کے اراکین (ais) کوششوں کی اپیل کرنا ہے۔ ہم وضاحت کرتے ہیں کہ ہم نے انفارمیشن سسٹمز ایسوسی ایشن کی حیثیت سے اس کام کا آغاز کیسے کیا، اور ہم کمیونٹی کی سطح پر اقدامات کی اپیل کرتے ہیں۔ یہ احساس دور کرنا مشکل ہے کہ ہم زبان کے تعصب کے بارے میں اپنی آگاہی میں دیگر شعبوں سے پیچھے ہیں۔ یہ 1990 اور اس کے بعد سے یہ کئی شعبوں میں اٹھایا گیا ہے بشمول ،(egger et al., 1997) کی دہائی میں طب میں اٹھایا گیا تھا & pudelko) انتظامیہ ،(levisen, 2019) لسانیات ،(clavero, 2011) ماحولیات ،(wierzbicka, 2009) جذباتی تحقیق tenzer, 2019)، قانون (ammann, 2022)، علمی سائنس (blasi, henrich, adamou, kemmerer, & majid, ,bampi, pires-oliveira, loyola-bartra) ارضی علوم ،(amano & berdejo-espinola, 2024) تحفظ ،(2022 & limaribeiro, 2024)، تنقیدی نگہداشت (gonzález-dambrauskas, salluh, machado, & rotta, 2024)، ۔ یہ موضوع سائنسی اشاعت کے مستقبل کے (mheissen, spineli, daraqel, & alsafadi, 2024) اور آرتھوڈونٹکس ,ahmed et al., 2023; amano, rojas, boum ii) بارے میں وسیع پیمانے پر ہونے والی بحثوں میں بھی اٹھایا گیا ہے calvo, & misra, 2021; arenas-castro et al., 2024; editorial, 2023) ۔ رپورٹ کرتے ہیں، غیر مقامی amano et al. (2023) دیگر تعصبات کی طرح، زبان کا تعصب بھی مہنگا ہے۔ جیسا کہ انگریزی اسکالرز کو کاغذات پڑھنے، لکھنے، اور نظرثانی کرنے میں زیادہ وقت درکار ہوتا ہے، ان کے کاغذات کو زبان سے متعلق وجوہات کی بنا پر مسترد کیے جانے کا زیادہ امکان ہوتا ہے، اور کانفرنسوں میں شرکت میں رکاوٹوں کا سامنا کرنا پڑتا ہم مرتبہ جائزہ ،(hohti & truman, 2021) ہے۔ زبان کا تعصب ابتدائی کیریئر کے اسکالرز پر دباؤ میں اضافہ کرتا ہے ,nunez & amano) ادب کے جائزوں ،(politzer-ahles, girolamo, & ghali, 2020) کے عمل کو متاثر کرتا ہے ,.konno et al) اور میٹا تجزیوں ،(hannah, haddaway, fuller, & amano, 2023) نظامی جائزوں ،(2021 ,van leeuwen, moed, tijssen, visser) میں تعصبات پیدا کرتا ہے، اور قومی تحقیق کے موازنوں کو بگاڑتا ہے (2020 & van raan, 2001)۔ گوگل اسکالر کی تالشوں میں بھی زبان کا تعصب پایا جاتا ہے، کیونکہ یہ انگریزی زبان کے مضامین ۔ ایک نظامی، طویل مدتی تعصب کے طور پر، زبان کا(rovira, codina, & lopezosa, 2021) کو ترجیح دیتا ہے تعصب قومی طریقوں میں بھی دیکھا جا سکتا ہے، جیسے کہ دنیا بھر کے اسکولوں میں انگریزی کی تعلیم کی موجودگی، اور عالمی کہ جیسے میں، صنعتوں بڑی 11b -usd$10زبان دوسری بطور انگریزی )(22esl صنعت۔ کی اگرچہ ہم سائنس کو متاثر کرنے والے لسانی تعصب کو اجاگر کرنے کی کوشش کر رہے ہیں، ہم عالمی انگریزی کے فوائد سے بے خبر نہیں ہیں۔ چونکہ زبانوں کی مختلفیت قدرتی طور پر ابالغ میں مشکالت پیدا کرتی ہے )مثال کے طور پر، مشہور بابل کے ٹاور میں دکھایا گیا ہے(، ایک زبان کی طرف رجحان )چاہے انگریزی ہو یا کوئی اور زبان (کئی شعبوں میں ترقی کو آسان ۔ جیسا کہ برٹش کونسل )2013، صفحہ 3 (نے نوٹ کیا "انٹرنیٹ، سوشل میڈیا کے (john & özgür, 2021) بنا سکتی ہے ،عروج، عالمی مواصالتی ٹیکنالوجی کی رفتار اور پھیالؤ، اور بڑھتی ہوئی عالمی اور باہمی انحصار عالمی معیشت کی بدولت انگریزی اب دنیا بھر میں خیاالت اور جدت کے تیز رفتار تبادلے کی اجازت دیتی ہے، اور علم اور خیاالت میں ایک نئی قسم کی سپرا نیشنل سنگل مارکیٹ کی ترقی۔ "تاہم، اگرچہ ہم انگریزی زبان کے پھیالؤ کے فوائد کو تسلیم کرتے ہیں )اور ہماری مصنف ٹیم کے کئی افراد نے ان فوائد سے لطف اندوز ہوئے ہیں، بطور مقامی انگریزی بولنے والے(، ہمارا مقصد یہاں ان اخراجات کو ,mélitz, 2007; salomone) نمایاں کرنا ہے، کیونکہ وہ اہم ہیں لیکن عام طور پر غیر تسلیم شدہ اور فرض کیے جاتے ہیں 2022) 2 ai ترجمہ کے ٹولز کا وعدہ کیونکہ چاہیے جانا کیا نہیں نظرانداز مزید اب اسے ،باوجود کے ہونے مہنگے کے اس اور موجودگی کی تعصب کے زبان اگرچہ ضروری دینا زور پر اہمیت تاریخی کی لمحے اس ہیں۔ رہے ہو بہتر سے تیزی بہت آالت کے ترجمے کے (ai) ذہانت مصنوعی اس اور ،ہے رہی جاتی کی ہی میں زبان کسی ہمیشہ سائنس ،آخرکار ہے۔ پرانا جتنا تاریخ کی سائنس تعصب کا زبان کیونکہ ہے زبانوں زیادہ سے 7,000 پر زمین ۔(2015 ،گورڈن) تھا ضروری علم کا زبان لیے کے کرنے (لکھنا ،بولنا ،پڑھنا) شرکت میں 22 https://www.businessresearchinsights.com/market-reports/esl-english-as-a-second-languagemarket-117478; https://www.englishaustralia.com.au/our-sector/understanding-the-sector australasian journal of information systems burton-jones et al. 2025, vol 29, provocation this article is not just in english 49 اس سائنس کہ تھا منطقی یہ ،تک سالوں ہزاروں تھے۔ سکتے نہیں لکھ یا پڑھ میں زبان ہر سائنسدان ،(2010 ،اینڈرسن) ساتھ کے مختلف اور ،چینی ،الطینی ،یونانی ،مصری میں زبانوں غالب میں اوقات مختلف کے تاریخ جائے۔ کی میں زبان غالب کی وقت (2015 گورڈن ) ہے رہی زبان غالب انگریزی سے بعد کے عظیم جنگ دوسری لیکن ،تھیں شامل زبانیں یورپی زبان ،(1999 ،ہف اور پیٹرسن) (ایسپرانٹو ،مثالا ) گئیں کی کوششیں (ناکام اور ) مختصر کی کرنے تیار زبان عالمی ایک اگرچہ موجود پر طور کے میدان ایک سے عرصے زیادہ سے سال 2,000 ،تاہم ہے۔ رہا ترجمہ حل واحد کا کرنے دور کو اختالفات کے لیے اس تر زیادہ ،گیا نہیں اپنایا پر پیمانے وسیع بھی کبھی میں سائنس کو ترجمے ،(2024 ،لینگویج_ڈائنامک) باوجود کے ہونے ،ہے گیا کیا شائع میں زبانوں متعدد کو جرائد مختلف ،دوران کے تاریخ ۔ (1967 ،ووڈ) ہے عمل طلب وقت اور مہنگا ترجمہ کہ ا 500کثیر لسانی جرائد موجود 23ہیں، لیکن یہ اصول کی تصدیق کرنے والی استثنیات ہیں۔ جیسا کہ گورڈن اور آج بھی تقریبا کیسا رہنا میں دنیا ایسی ایک کہ ہیں رکھتے دلچسپی میں بات اس آپ اگر" ،ہے لکھا سے طریقے مؤثر نے (3 صفحہ ،2015) طرف کی سائنسدانوں قدرتی کو آپ تو ،ہوں نہ رکاوٹیں کی زبان جہاں دنیا ایسی ایک ،ہو چیت بات میں زبان ایک جہاں ہوگا لسانی کثیر تر زیادہ اور ،(2015 ،گورڈن) ہے میں انگریزی صرف اکثریت وسیع کی جرائد "ہیں۔ آتے سے وہاں وہ چاہیے۔ دیکھنا کی قوانین قومی صرف تر زیادہ سے میں ان اور ،ہیں جرائد لسانی دو درمیان کے زبان دوسری کسی اور انگریزی صرف جرائد ۔ (ہے ملک لسانی دو ایک جو ،جرائد والے ہونے شائع میں کینیڈا ،مثالا ) ہیں لسانی دو سے وجہ ایسپینوال-برڈیجو) ہے ابھری پر طور حالیہ صالحیت کی آالت کے ترجمے ai لیے کے نمٹنے سے تعصب کے زبان میں سائنس کی تحقیق ایک ہے۔ ہوئی پیدا سے تبدیلیوں مدتی طویل دو جو ،(2022 ،وغیرہ والڈ اسٹائیگر ؛ 2024 ،کروٹی ؛2023 ،امانو اور جاتے میں الئبریریوں محققین تھے۔ دستی کام تحقیقی سے بہت ،ہوا نہیں عرصہ زیادہ ہے۔ رہی تبدیلی ڈیجیٹل بتدریج کی خود حساب کہ حتی ،تھے بھیجتے ذریعے کے میل جائزے اور کاغذات ،تھے لیتے نوٹ پر ان اور پڑھتے کاغذات جسمانی ،تھے ا ،اب تھے۔ لکھتے کاغذات سے ہاتھ اور کرتے کتاب ،کرنا تعاون ،کرنا تشریح ،لکھنا ،پڑھنا ،کرنا تالش :ہیں الئن آن کام تمام تقریبا پلیٹ ڈیجیٹل حصہ تر زیادہ کا سائنس اب کہ ہے یہ نتیجہ گہرا ایک کا تبدیلی اس کرنا۔ شائع ،کرانا جمع ،لینا جائزہ ،کرنا تجزیہ جاتے پڑھے ،ہیں جاتے کیے شائع ،ہیں جاتے لیے جائزہ ،ہیں جاتے لکھے کاغذات سائنسی ،یعنی ہے۔ شامل میں زبان پر فارمز ا کل آج) ہیں جاتے کیے تالش میں زبان ایک اور ،ہیں ڈیجیٹل ہوئے جڑے باہم ،بڑے کچھ سب یہ اور ،(میں انگریزی ہمیشہ تقریبا پلیٹ فارمز پر ہوتا ہے۔ ان میں سے کچھ پلیٹ فارم اب بہت بڑے ہیں، مثالا ، اسکوپس 2490ملین سے زیادہ ریکارڈز کی میزبانی ،کوزار -لوپیز اور ،مالیا-اورڈونا ،وال تھیل ،مارٹن-مارٹن) ہے کرتا تالش میں ریکارڈز ملین 300 اسکالر گوگل اور ہے کرتا ،ہے سے دہائی کی 1950 تاریخ کی ماڈلز کے زبان میں ai اگرچہ ترقی۔ کی ai – ہے آتی تبدیلی دوسری پر یہیں اور ۔(2021 ،نائک ،نائک) کیا فعال نے ترقی پر پیمانے بڑے کی ڈیٹا الئن آن لیے کے سیکھنے کو ترقی رفتار تیز کی ماڈلز کے زبان بڑے ہے سکتا پہنچا فائدہ کو ڈیٹا کے طرح اس جو ،تعمیر فن ٹرانسفارمر پر طور خاص ،ساتھ کے تعمیرات فن نئے (2024 ،نائک اور سے جس ،ہے بنایا قابل کے ابھرنے کو شکلوں نئی کی ai نے تبدیلیوں دو ان ،میں سالوں پانچ پچھلے ۔(2017 ،وغیرہ واسوانی) تیز میں ترجمے کے زبان پر طور خاص رفت پیش یہ اور تھی۔ لگتی ناممکن پہلے عرصہ کچھ جو ہوئی ممکن رفت پیش ایسی موزوں پر طور مثالی لیے کے اس اور تھے کام پہلے والے جانے جانچے پر تعمیر فن ٹرانسفارمر کام کے ترجمے کے زبان ہے۔ ۔ (2017 ،وغیرہ واسوانی) ہیں سی بہت ہے۔ سکتا کر حل پر طور فوری اور پر طور مکمل کو تعصب کے زبان ترجمہ ai کہ رہے کر نہیں دعوی یہ ہم ایک کرنا پیچھا کا اس لیے ہمارے کہ ہے مضبوط اتنی صالحیت لیکن ۔(2024 ایسپینوال-برڈیجو اور امانو) گی ہوں پیچیدگیاں ہوگا مناسب غیر یہ ،سسٹمز انفارمیشن فار ایشن ایسوسی کہ جیسے ،سے نظر نقطہ کے ایشن ایسوسی سائنسی ایک ہے۔ فرض اسی اور کریں نہ کوشش کی کرنے کم کو تعصب نظامی مہنگے اور موجودہ درپیش کو اراکین کے میدان سائنسی اپنے ہم اگر اسی ہے۔ سکتا کر فراہم لیے کے اثر اور شمولیت ترجمہ ai جو اٹھائیں نہ فائدہ سے مواقع نئے ان ہم اگر ہوگا مناسب غیر طرح اثر ،جامع برائے ترجمہ ai برائے فورس ٹاسک سسٹمز انفارمیشن فار ایشن ایسوسی دی تشکیل فورس ٹاسک اپنی نے ہم لیے سائنس۔ انگیز پیچیدگیاں 3 ،ترجمہ کے استعمال سے کئی چیلنجز پیدا ہوتے ہیں۔ ہم چھ زمروں کی وضاحت کرتے ہیں۔ سب سے پہلے ai سائنس سائنس میں چیلنجز اس بات پر منحصر ہیں کہ ہم مسئلے کا دائرہ کیسے طے کرتے ہیں۔ سب سے آسان طریقہ یہ ہے کہ ایک زبان سے دوسری زبان میں ایک واحد مقالے کے ترجمے پر توجہ مرکوز کی جائے، لیکن یہاں تک کہ یہ دائرہ بھی سواالت اٹھاتا ہے۔ مثال کے طور پر، کیا ہمیں صرف منظور شدہ مقالے کا ترجمہ کرنا چاہیے، یا جمع کرائے گئے مقالے کا بھی؟ پہال سائنس کے اثر کو بڑھا سکتا ہے ایک مقالے کے قارئین کی تعداد میں اضافہ جبکہ دوسرا شمولیت کو بھی سپورٹ کرے گا زیادہ 23 https://en.wikipedia.org/wiki/category:multilingual_journals 24https://assets.ctfassets.net/o78em1y1w4i4/76uvtuilkssqe4h4h1pzkq/e5ff8eef9694f216a5c29e2ebf80fa51/ fast-facts-about-elsevier.pdf australasian journal of information systems burton-jones et al. 2025, vol 29, provocation this article is not just in english 50 مصنفین اور جائزہ لینے والوں کو اپنی مادری زبان میں لکھنے کی اجازت دینا۔ اگرچہ دوسرا انتخاب زیادہ فوائد رکھتا ہے، اس میں زیادہ کام بھی شامل ہے، جس کے لیے ہمیں مقالے اور مقالے کے جائزے دونوں کا ترجمہ کرنا ہوگا، اور جائزہ لینے کے عمل کے دوران مصنفین اور جائزہ لینے والوں کے لیے ترجمے کے معیار کی تصدیق کے لیے ایک ورک فلو بھی درکار ہوگا تاکہ ان پر اضافی بوجھ نہ ڈالے۔ ایک بار جب کوئی مقالہ منظور/شائع ہو جاتا ہے، تو ایک اور سوال یہ پیدا ہوتا ہے کہ کیا ہم صرف ایک زبان یا متعدد زبانوں میں ترجمہ پر توجہ مرکوز کرتے ہیں، اور آیا مصنف یا قاری زبان کا انتخاب کرنے کی اجازت دیتا ہے۔ اگر قارئین کو انتخاب کرنے کی اجازت دی جائے، تو کیا ترجمے پہلے سے تیار کیے جائیں )مثالا، چند عام زبانوں میں (اور قارئین کو اس فہرست سے اپنی پسند کی زبان منتخب کرنے کی اجازت دی جائے، یا قارئین اپنی منتخب کردہ زبان میں ترجمہ کی درخواست کر سکیں )یا تو ایک مقررہ فہرست سے، یا ایک مثالی دنیا میں، کسی بھی زبان سے (جو پھر اشاعت کے پلیٹ فارم کے ذریعہ درخواست پر حقیقی وقت میں تیار کیا جائے؟ مؤخر الذکر وسیع تر زبانوں کی حد کو قابل بنائے گا جبکہ پہال ترجمے کے معیار کی تصدیق کا مرحلہ فراہم کرے گا۔ اگر مؤخر الذکر کا انتخاب کیا جائے، تو یہ ممکن ہے کہ بعض ترجموں میں معمولی معنوی تبدیلیاں رونما ہو سکتی ہیں بغیر مصنف)ین (کے علم کے، جس سے اس کام کے بعد کی حوالہ جات میں پیچیدگیاں پیدا ہو سکتی ہیں۔ ان ڈیمانڈ، حقیقی وقت کے ترجمے کے آپشن کے ساتھ شائع ہونے والے مقالے اس لیے ایک دستبرداری کی ضرورت ہو سکتی ہے جو غلطیوں کے امکان کو تسلیم کرتی ہے۔ ایک اور سوال یہ پیدا ہوتا ہے کہ کیا ہم ایک واحد مقالے کے ترجمے سے آگے بڑھ کر پورے الئبریری/پلیٹ فارم کے مقالوں جس میں 65,000 سے ،elibrary کا ترجمہ کرنے پر غور کرتے ہیں، جیسے کہ پورا ایسوسی ایشن فار انفارمیشن سسٹمز ،زیادہ مقالے ہیں؟ ایک الئبریری کے دائرہ کار کو بڑھانے سے نہ صرف ترجمے کے لیے مقالوں کی تعداد میں اضافہ ہوتا ہے بلکہ اضافی غور و فکر بھی شامل ہوتا ہے۔ مثال کے طور پر، یہ ترجمے کے وقت کی مدت کو بڑھا سکتا ہے، کیونکہ ایک الئبریری میں کئی دہائیوں میں شائع ہونے والے مقالے ہو سکتے ہیں، اور اس لیے ترجمے کے عمل کو وقت کے ساتھ ساتھ ثقافتی سیاق و سباق اور میدان کی لغت میں تبدیلیوں پر غور کرنے کی ضرورت پڑ سکتی ہے۔ اسی طرح، یہ تکمیلی ٹولز کی ترقی کی ضرورت ہو سکتی ہے، جیسے کہ ایک کثیر لسانی تالش کا آلہ جو الئبریری میں متعدد زبانوں میں تالش کر سکے۔ ٹیکنالوجی جو ترجمے کرتی ai کی صالحیتوں سے متعلق ہے۔ اس بات کو مدنظر رکھتے ہوئے کہ وہی ai ایک آخری سوال ہے متعلقہ صالحیتیں بھی پیش کرتی ہے، کیا ہمیں ان صالحیتوں کو بھی استعمال کرنا چاہیے؟ مثال کے طور پر، کیا ہمیں مقالوں کے ترجمے سے آگے بڑھ کر مختلف زبانوں میں مقالے کے خالصے تیار کیے جا سکیں )مثالا، ایگزیکٹوز یا طلباء کے لیے عمومی خالصے (؟ اسی طرح، کیا ہمیں صرف ترجمے ہی نہیں بلکہ زبان میں بہتری پر بھی غور کرنا چاہیے، مثالا، متن کو بہتر بنانے کے لیے تجاویز پیش کرنا، نہ کہ صرف متن کو 'جیسا ہے 'ترجمہ کرنا؟ اگر ہم مؤخر الذکر پر غور کریں، تو ہم مسئلے کو محض 'ترجمہ 'سے 'مفہوم کی وضاحت 'میں دوبارہ ترتیب دے سکتے ہیں جہاں ترجمہ مفہوم کی وضاحت کا صرف ایک عنصر ہے اور دیگر عناصر میں گرامر، بہاؤ، اور انداز میں بہتری شامل ہو سکتی ہے۔ جیسا کہ آپ دیکھ سکتے ہیں، مسئلے کا دائرہ بہت زیادہ مختلف ہو سکتا ہے۔ اس مرحلے پر، ہمارا ٹاسک فورس ایک درمیانی دائرہ کار کو اپنائے ہوئے ہے۔ ہم پورے جائزہ عمل کی حمایت کرنا چاہتے ہیں جمع کرانے سے لے کر قبولیت تک۔ ہم متعدد زبانوں میں ترجمے کی حمایت کرنا بھی چاہتے ہیں۔ تاہم، ہم اپنی کوششوں کو ابتدائی طور پر 10 بڑی زبانوں تک محدود کر رہے ہیں )بہاسا انڈونیشیا، اردو، یونانی، فرانسیسی، ماالئی، کورین، مینڈارن، جرمن، ہسپانوی، اور پرتگالی (اور پہلے سے تیار کردہ ترجمے تیار کرنے پر توجہ مرکوز کر رہے ہیں جو انسان کو شامل کرنے کی اجازت دیتے ہیں۔ معیار کی تصدیق کے مرحلے میں، بجائے اس کے کہ آن ڈیمانڈ، حقیقی وقت کے ترجمے کو فعال کیا جائے۔ ہم اپنے دائرہ کار میں مقالے کے خالصے کی تیاری کو شامل کر رہے ہیں لیکن ہم صرف انفرادی مقالوں کے ترجمے پر توجہ مرکوز کر رہے ہیں نہ کہ پوری الئبریری/پلیٹ فارم پر، اور ہم اس پہلے مرحلے میں 'مفہوم کی وضاحت 'کے وسیع پہلوؤں جیسے کہ زبان میں بہتری پر غور نہیں کر رہے ہیں۔ ان پیچیدگیوں کے عالوہ جو دائرہ کار کے سواالت کی وجہ سے پیدا ہوتی ہیں، ایک اور پیچیدگیوں کا مجموعہ اشاعت کے ماحول کے ساتھ کیسے نمٹنا چاہیے؟ (dois) کی عملی مشکالت سے پیدا ہوتا ہے۔ مثال کے طور پر، ہمیں ڈیجیٹل آبجیکٹ آئیڈنٹیفائرز تفویض کیا جانا چاہیے کیونکہ اس سے سرچ انجنز اور ایگریگیٹرز کو ہر مقالے doi آج کل، ہر شائع شدہ مقالے کو ایک منفرد ،کے حوالہ جات کو تالش کرنے اور شناخت کرنے کی اجازت ملتی ہے۔ تاہم، اگر ہم کسی مقالے کے متعدد ترجمے بناتے ہیں ہوگا، اور کیا مقالے کے حوالہ doi میں شامل ہوں گے جس میں ایک ہی (pdf ،مثالا، ویب پیج) تو کیا یہ ایک ہی دستاویز جات کو اس ورژن )ترجمہ (کے قطع نظر جمع کیا جائے گا؟ ان سواالت کا جواب دینے میں ایک مشکل یہ ہے کہ کوئی بھی ادارہ )سائنسی ایسوسی ایشن، پبلشر، بڑی ٹیک (پوری اشاعتی ماحولیاتی نظام پر کنٹرول نہیں رکھتا ہے، اور یہ پیش گوئی کرنا مشکل ہے کہ یہ سب کیسے ترقی کرے گا۔ اس لیے ہمیں ترجمے کے ایسے طریقے منتخب کرنے ہوں گے جو کسی طرح موجودہ طریقوں کے ساتھ ہم آہنگ ہوں جبکہ اتنے لچکدار ہوں کہ ان کے مطابق ڈھل سکیں۔ اسی طرح کی عملی پیچیدگیاں کاپی رائٹ کے ساتھ پیدا ہوتی ہیں۔ ایک ترجمہ 'ماخوذ کام 'سمجھا جا سکتا ہے اور اس لیے ایسے کاموں پر کاپی رائٹ پابندیوں کی تعمیل کرنی چاہیے۔ ایک اور عملی مسئلہ سرقہ ہے۔ ستم ظریفی یہ ہے کہ ترجمے کی سہولت australasian journal of information systems burton-jones et al. 2025, vol 29, provocation this article is not just in english 51 فراہم کرنا سرقہ کو بھی آسان بنا سکتا ہے۔ اس کی وجہ یہ ہے کہ سرقہ عام طور پر کسی مقالے میں الفاظ کی اسی ترتیب کے استعمال سے بیان اور پتہ لگایا جاتا ہے، یعنی لفظ بہ لفظ نقل۔ ترجمہ لفظ بہ لفظ نہیں ہونا چاہیے، بلکہ 'معنی کے لیے معنی 'ہو ۔ اگر کوئی سرقہ کرنے واال کسی مقالے کا ترجمہ بنا کر(sun, 2013) سکتا ہے )ایک ہی معنی کو پہنچانا نہ کہ وہی الفاظ( معنی کے لیے معنی کے ترجمے کا استعمال کرتے ہوئے سرقہ کرتا ہے، تو یہ واضح نہیں ہے کہ موجودہ سرقہ پکڑنے والے ٹولز اسے شناخت کر پائیں گے یا نہیں۔ ان ٹولز کو اس خطرے کو کنٹرول کرنے کے لیے نظر ثانی کی ضرورت ہو سکتی ہے۔ کی تیز رفتار ترقی نے ہمیں اس منصوبے کو ai کی خصوصیات سے پیدا ہوتا ہے۔ اگرچہ ai تیسری پیچیدگیوں کا مجموعہ ترجمہ ai اور ،(ji et al., 2023) آگے بڑھانے کی تحریک دی، یہ ابھی بھی کامل نہیں ہے۔ غلط فہمیاں اب بھی پیدا ہوتی ہیں ۔ انٹرنیٹ پر انگریزی زبان (xu et al., 2024) کی کارکردگی ہر زبان میں تربیتی ڈیٹا کی مقدار کے لحاظ سے مختلف ہوتی ہے ماڈلز میں نہ صرف انگریزی زبان کا تعصب بلکہ ثقافتی تعصب بھی شامل ہو سکتا ہے ai کا وسیع غلبہ کی وجہ سے، جن (luo & puett, 2024) ۔ ہم اپنے میدان کے محققین سے درخواست کرتے ہیں کہ وہ ان مسائل کے حل کے لیے ٹھوس سماجی تکنیکی ماڈل اور طریقے تیار کریں۔ ایسا کرتے ہوئے، محققین کو صرف تکنیکی تعین یا سماجی تعین سے متاثر نہیں ہونا چاہیے ترجمے کی کامیابی کا دارومدار نہ صرف ai ،ایک سماجی تکنیکی نقطہ نظر اہم ہے۔ یعنی، کسی بھی اور ٹیکنالوجی کی طرح ٹیکنالوجی پر ہوگا بلکہ اس بات پر بھی ہوگا کہ صارفین )مصنفین، ایڈیٹرز، جائزہ لینے والے، قارئین (اسے کیسے سمجھتے اور استعمال کرتے ہیں۔ مثال کے طور پر، انسانی رجحانات جیسے الگورتھم سے نفرت اور عدم اعتماد کو الزمی طور پر مدنظر ماڈلز کو چالنے میں مالی الگت بھی آتی ہے، اس سوال کو جنم دیتے ہوئے ai ۔ (asscher & glikson, 2023) رکھنا ہوگا ٹاسک فورس کے طور پر آگے بڑھا رہے ہیں اس کی ایک وجہ یہ ہے ais کہ اس الگت کو کون برداشت کرے۔ ہم اس کو ایک کہ ہم اس بات پر غور کر رہے ہیں کہ آیا اور کیسے ایک سائنسی ایسوسی ایشن کو اپنے اراکین کے لیے اس الگت کو برداشت کرنا چاہیے۔ ترجمہ ماڈل نظر ai چوتھی پیچیدگیوں کا مجموعہ تحریری زبان میں موجود باریکیوں کی وجہ سے پیدا ہوتا ہے جنہیں موجودہ انداز کر سکتے ہیں۔ مثال کے طور پر، اس مقالے کے جائزہ عمل میں، ہمارے ایک جائزہ لینے والے نے ہمارے مخففات کے استعمال کرتے (ia) ترجمے میں غلطیاں نوٹ کیں۔ مثال کے طور پر، اسپین میں، مصنفین عام طور پر مخفف کا ہسپانوی ورژن استعمال کرنے کے عادی ہیں، اور مصنف کو صرف (tn) یا یونانی ورژن (ai) ہیں، لیکن یونان میں، مصنفین انگریزی ورژن ایک کو مستقل طور پر استعمال کرنا چاہیے۔ ہمارا ترجمہ ایسا کرنے میں ناکام رہا۔ دریں اثنا، ہمارے دوسرے جائزہ لینے والوں نے نوٹ کیا کہ ہمارے ترجمے بعض اوقات بہت زیادہ "سخت"، "مضبوط"، یا "رسمی "لگتے ہیں اور ہمیں ایک "نرم "یا زیادہ ترجمہ نے خود بخود ایک بہت زیادہ رسمی لہجہ منتخب کیا ہو، جس کا ہمیں ai قدرتی "لہجہ اپنانا چاہیے تھا۔ یہ ممکن ہے کہ" پہلے سے احساس نہیں تھا۔ ان چیلنجز سے نمٹنے کے لیے، ہم اپنی ترجمے کے طریقہ کار کو فعال طور پر بہتر بنا رہے ہیں۔ خاص طور پر، ہم نے اس مقالے کے جائزہ لینے والوں کی رائے کو شامل کیا ہے، خاص طور پر ثقافتی قدرتی پن اور لہجے ا، ہمارا ترجمہ کا طریقہ کار اب ان پہلوؤں کو زیادہ اہمیت دیتا ہے۔ مثال کے طور پر، داخلی کے جائزے کے حوالے سے۔ نتیجتا ٹول الگز نے جائزہ لینے والے کی طرف سے متاثر کردہ تجاویز کو پکڑا ہے جیسے" :فقرہ '...'کافی براہ راست ہے۔ اسے زیادہ تعلیمی لہجے میں دوبارہ ترتیب دینے پر غور کریں، جیسے '...'۔ "یا "فقرہ '...'کچھ حد تک لفظی ہے۔ ایک زیادہ مناسب تعلیمی ترجمے کی تیزی سے بڑھتی ہوئی طاقت کے باوجود، یہ تصور کرنا مشکل ہے کہ یہ ٹولز ہر زبان ai "انداز ہو سکتا ہے '...'۔ میں اور ہر مقالے کے لیے موزوں کامل ترجمہ پیش کریں گے، لیکن ہم ان مسائل میں سے زیادہ سے زیادہ کو حل کرنے کا ۔ (ماڈل کی ایجنٹک آرکیٹیکچر کو تبدیل کر کے سیاق و سباق کو زیادہ مؤثر طریقے سے استعمال کرنا ai ،مثالا ) ارادہ رکھتے ہیں پانچویں پیچیدگیوں کا مجموعہ سائنس کے اداروں سے پیدا ہوتا ہے جو موجودہ صورتحال کو تقویت دے سکتے ہیں اور کثیر لسانی سائنس کے ابھرنے میں رکاوٹ بن سکتے ہیں۔ مثال کے طور پر، جب ہم نے اس اقدام پر کامیاب محققین کے ساتھ بات چیت کی جو انگریزی کے غیر مقامی بولنے والے ہیں، تو ہم حیران رہ گئے کہ کتنے لوگوں نے کہا کہ وہ اپنی مادری زبان میں لکھنے کا ہمارا موقع نہیں لیں گے کیونکہ وہ انگریزی میں مقالے لکھنے میں بہتر ہیں )کیونکہ انہوں نے اپنی پوری زندگی انگریزی میں تربیت حاصل کی ہے(۔ کچھ طریقوں سے یہ ہمیں پریشان نہیں کرتا کیونکہ انگریزی کے غیر مقامی بولنے والوں کی آبادی جو فائدہ اٹھا سکتی ہے وہ انگریزی کے غیر مقامی بولنے والوں کی چھوٹی آبادی سے کہیں زیادہ ہے جنہوں نے انگریزی کی غالب نظام میں 'کامیابی حاصل کی'۔ دوسری طرف، ایک کمیونٹی کے سینئر اراکین قدرتی طور پر جونیئر اراکین کو متاثر کرتے ہیں، اور اس لیے ان کے خیاالت کچھ وقت کے لیے موجودہ صورتحال کو تقویت دے سکتے ہیں۔ ایک مثال کے طور پر، ہم نے اس اقدام پر ایک یورپی ملک میں سینئر محققین کے ساتھ کام کیا ہے۔ اگرچہ وہ ہمارے اقدام کی مضبوطی سے حمایت کرتے ہیں، انہوں نے ہمیں آہستہ چلنے کی ترغیب دی کیونکہ انہوں نے اپنی کمیونٹی میں انگریزی زبان کی تحریر کو بہتر بنانے کے لیے کئی دہائیوں تک کام کیا تھا، اور وہ کوشش بہت کامیاب رہی تھی، جس کی وجہ سے اعلی جرائد میں اشاعتوں کی تعداد میں تیزی سے اضافہ ہوا۔ ان محققین کی آخری خواہش یہ تھی کہ وہ اس کامیابی کو کم کریں۔ جیسا کہ یہ مثال ظاہر کرتی ہے، کثیر لسانی سائنس کی حوصلہ افزائی کی کوششوں کو احتیاط سے ڈیزائن کرنے کی ضرورت ہے موجودہ صورتحال کے ارد گرد ڈیزائن کیے گئے دیگر پہلے سے موجود اقدامات کے علم میں۔ آخر میں، ہمارے پاس پیچیدگیوں کا چھٹا مجموعہ ہے جو زیادہ فلسفیانہ ہیں۔ ان کو دیکھنے کا ایک طریقہ یہ ہے کہ ایک مقالے کو ایک مصنف کے ذہن میں ایک خیال کے بننے کے طویل عمل میں صرف ایک قدم سمجھا جائے تاکہ اسے ایک قاری کے australasian journal of information systems burton-jones et al. 2025, vol 29, provocation this article is not just in english 52 ذہن میں سراہا جا سکے۔ شروع کرنے کے لیے، ایک مصنف کے کسی خیال کے بارے میں سوچنے پر غور کریں۔ سپیئر-وہرف ،۔ اگر یہ سچ ہے(lucy, 1998) مفروضہ یہ تجویز کرتا ہے کہ ایک مصنف کے خیاالت مصنف کی زبان سے تشکیل پاتے ہیں تو اس کا مطلب ہے کہ زبان کا تعصب ابتدا تک جاتا ہے، یہاں تک کہ ان خیاالت کو بھی محدود کرتا ہے جو مصنفین تشکیل دے سکتے ہیں۔ ہم اسے 'تصور 'کا مسئلہ کہہ سکتے ہیں۔ مسائل پھر جاری رہتے ہیں کیونکہ یہاں تک کہ اگر کسی زبان کو کسی خیال کو پہنچانے کے لیے استعمال کیا جا سکتا ہے، تو ایک مصنف کے پاس ایسا کرنے کی زبان کی مہارت نہیں ہو سکتی۔ ہم اسے 'اظہار 'کا مسئلہ کہہ سکتے ہیں۔ اور یہاں تک کہ اگر کسی مصنف کے پاس کسی خیال کو پہنچانے کے لیے صحیح زبان ہے، اور اسے مہارت سے استعمال کرتا ہے، تو ایک قاری الفاظ کی اسی طرح تشریح نہیں کر سکتا۔ ہم اسے 'تشریح 'کا مسئلہ کہہ سکتے ہیں۔ ہمیں یقین ہے کہ اس مقالے کے قارئین ان مسائل سے واقف ہیں کیونکہ یہ روزمرہ کی زندگی میں عام ہیں۔ مجموعی طور پر، ان مسائل کا مطلب ہے کہ تحقیقی مقالے ناگزیر طور پر ایک ناقص ذریعہ ہیں جس کے ذریعے علمی برادری میں سائنسی خیاالت کو پہنچایا جا سکتا ہے۔ معنی میں نقائص عمل کے لیے فطری ہیں، استعمال شدہ زبان سے قطع نظر۔ واضح ٹولز لوگوں کو اپنی زبان ai ،ترجمہ کے ٹولز ان تمام مسائل کو حل نہیں کر سکتے۔ صرف ایک مثال کے طور پر ai ،طور پر میں سوچنے کے قابل بنا کر تصور کے مسئلے کو کم کرنے میں مدد کر سکتے ہیں، لیکن وہ ایسا کرنے میں ناکام بھی ہو سکتے (barath, 2019) ہیں کیونکہ انگریزی زبان کی غلبہ اتنی مضبوط رہی ہے کہ کچھ زبانوں کے پاس مطلوبہ لغت نہیں ہے ایک موقع اور ہمارا نقطہ نظر 4 اگرچہ ہم نے متعدد پیچیدگیوں کی نشاندہی کی ہے، ہم ان پیچیدگیوں کو مسائل کے بجائے مواقع کے طور پر دیکھتے ہیں جن ،سے استفادہ کیا جا سکتا ہے۔ ہمارے لیے، یہ ظاہر کرتے ہیں کہ یہ آئی ایس کے شعبے کے لیے ایک مثالی چیلنج کیوں ہے ۔ جیسا کہ (sarker, chatterjee, xiao, & elbanna, 2019) کیونکہ مواقع اور پیچیدگیاں گہرائی سے سماجی تکنیکی ہیں ہم نے ابتدا میں ذکر کیا، آئی ایس کا شعبہ زبان کے تعصب سے واقف ہونے واال پہال نہیں ہے۔ پھر بھی، جب کہ بہت سے شعبوں ا، آئی ایس کے شعبے کے نے اس مسئلے کے بارے میں آگاہی پیدا کی ہے، چند نے اسے منظم طریقے سے حل کیا ہے۔ نتیجتا کی شعبے ہمارے مواد کا موضوع چونکہ ۔25ہے موجود موقع کا کرنے کام ساتھ کے شعبوں اہم دیگر پر محاذ کے سائنس لیے نوعیت سے اتنا قریب سے جڑا ہوا ہے ایک بہتر مستقبل کے لیے سماجی تکنیکی فن پاروں اور طریقوں کو سمجھنے اور ڈیزائن کرنے کے لیے وقف ایک شعبے کے طور پر ہمارا ماننا ہے کہ آئی ایس کا شعبہ نہ صرف فرق پیدا کرنے کا موقع رکھتا ہے بلکہ اس کا فرض بھی ہے۔ ,lewin) ٹاسک فورس کی تشکیل کی۔ تبدیلی کے روایتی ماڈلز کے مطابق ais اس موقع سے متاثر ہو کر، ہم نے 2024 میں ہم سمجھتے ہیں کہ شعبے کو 'جمود کو توڑنے 'کے لیے کافی کوشش درکار ہوگی، نہ کہ یہ فرض کیا جائے کہ شعبہ ،(1951 فوری طور پر تبدیل ہو سکتا ہے۔ 'جمود کو توڑنے 'کو آسان بنانے کے لیے، ہم نے چار بڑی کانفرنسوں میں پینلز اور عمومی ,burton-jones, 2025; burton-jones) اجالس منعقد کیے تاکہ آگاہی پیدا کی جا سکے اور بحث کو تحریک دی جا سکے chau, feng, kankanhalli, & vogel, 2024; pouloudi et al., 2024; sarker, ahuja, feng, jarvenpaa, & seidel, 2024)۔ ہم نے مختلف زبان کی کمیونٹیز کی نمائندگی کرنے والے متعدد ais چیپٹرز کے ساتھ بھی بات چیت کی ترجمہ کے آالت کی ترقی اور ابتدائی جانچ کے لیے کام کیا۔ ai اور میں، ہم اس مسئلے کے بارے میں آگاہی پیدا کرنے کی اپنی کوششیں جاری رکھے ہوئے ہیں )جیسے کہ اس مقالے کے 2025 ،ذریعے (جبکہ ڈیزائننگ، ٹیسٹنگ، اور سیکھنے کے چکروں کے ساتھ زیادہ فعال موقف اختیار کر رہے ہیں۔ مثال کے طور پر کے مدیران اور جائزہ نگاروں کے مشکور ہیں کہ انہوں نے اس مقالے کے لیے ایک کثیر لسانی جائزہ عمل کی جانچ ajis ہم کی، جس سے ہم سیکھ سکتے ہیں اور بہتری ال سکتے ہیں۔ ہم دیگر کاموں میں بھی بیک وقت مصروف ہیں تاکہ ٹھوس ڈیٹا اور americas conference on information systems ٹیسٹ فراہم کیے جا سکیں۔ مثال کے طور پر، ہم نے 2025 میں (amcis) کے دوران ai تراجم کی جانچ کی (دیکھیں: https://aisel.aisnet.org/amcis2025/translated/) ۔ ہم 2025ترجمہ مختلف ساتھ کے کانفرنسوں دیگر میں 26تین پر طور خاص ہم ہیں۔ رہے کر کام بھی پر معامالت کے استعمال کے ترجمہ؛ اور ai ترجمہ؛ )2 (منظور شدہ مقاالت کا ai استعمال کے معامالت پر توجہ مرکوز کر رہے ہیں) :1 (جائزہ عمل میں ،مختلف زبانوں میں مقاالت کے خالصے کی تیاری تاکہ اثر کو وسیع کیا جا سکے۔ ہم یہ جاننے کے لیے سروے، انٹرویوز (3) ترجمہ ai اور فوکس گروپس بھی منعقد کریں گے کہ زبان کا تعصب آئی ایس کے شعبے میں علماء کو کیسے متاثر کرتا ہے اور اور ی ورپ یون ی ک کونسل unesco (https://doi.org/10.54677/mnmh8546) اس میدان می د یگر نما یاں آوازوں م ی 25 (https://www.consilium.europa.eu/media/56958/st10126-en22.pdf) ۔ ی ہ شامل پر شکر یہ wi 2025 اور ،amcis 2025، pacis 2025 ہم 26 ز کا اس اقدام می شمولیت ےک لی آمادک ز اور پروگرام چ یئر ےک کانفرنس چ یئر ۔ ی ہ کرت ادا australasian journal of information systems burton-jones et al. 2025, vol 29, provocation this article is not just in english 53 کے آالت کے ممکنہ اثرات کیا ہیں۔ ہم وسیع تر سائنسی تحریک کی حمایت کے لیے دیگر شعبوں کے علماء کے ساتھ بھی تعاون ۔(amano, bowker, & burton-jones, 2025) کر رہے ہیں ان اقدامات کو آگے بڑھاتے ہوئے، ہم تسلیم کرتے ہیں کہ ہمیں بہت سے چیلنجز کا سامنا کرنا پڑے گا، اور بہت سے مزید مواقع سے سبق سیکھ سکتے ،(ferguson et al., 2023) باقی ہیں۔ ہم دیگر سائنسی تحریکوں، جیسے کہ اوپن سائنس کی تحریک ہیں اور حوصلہ افزائی حاصل کر سکتے ہیں، جو اسی طرح سائنس میں زیادہ شمولیت اور اثر کے لیے تحریک یافتہ ہیں۔ کسی بھی تبدیلی کی کوشش کی طرح، ہم غلطیاں کریں گے اور اپنے منصوبوں میں موڑ لینے پڑیں گے۔ اس کے باوجود، ہم ان امکانات پر پُرجوش ہیں، اور ہم امید کرتے ہیں کہ دیگر آئی ایس محققین بھی پُرجوش ہوں گے، اور مل کر ہم بامعنی تبدیلی ال سکتے ہیں۔ بجائے اس کے کہ یہ ایک تنہا، سختی سے کنٹرول شدہ، اوپر سے نیچے کی کوشش ہو، ہم کمیونٹی بھر سے کوششوں کے لیے اپیل کرتے ہیں تاکہ مسئلے کے بہت سے متنوع پہلوؤں کو حل کیا جا سکے، اور ہم اس شعبے میں مدیران اور کانفرنس چیئرز سے درخواست کرتے ہیں کہ وہ ایسی نیچے سے اوپر کی کوششوں کی حمایت کریں۔ کے مدیران کے مشکور ہیں کہ انہوں نے اس کثیر لسانی جائزہ عمل کی اجازت دی۔ عمل کچھ ajis پہلے قدم کے طور پر، ہم اس طرح تھا) :1 (ہم نے انگریزی میں مسودہ جمع کرایا؛ )2 (متعلقہ زبانوں کے ساتھ جائزہ نگاروں کی شناخت کے لیے مدیر bahasa نے )ہمارے معاملے میں، جائزہ نگار 1 نے یونانی اور ہسپانوی دونوں کے ساتھ شناخت کی، جائزہ نگار 2 نے indonesia کے ساتھ، اور جائزہ نگار 3 نے اردو کے ساتھ شناخت کی(؛ )3 (ہم نے اپنے ai ترجمہ نظام کا استعمال کرتے ہوئے مقالے کو تمام چار زبانوں میں ترجمہ کیا اور تمام پانچ ورژنز کو مدیر کو جمع کرایا، جس نے پھر انگریزی اور دیگر متعلقہ زبانوں میں ہر جائزہ نگار کو مقالے بھیجے؛ )4 (جائزہ نگاروں نے اپنی پسندیدہ زبان میں جائزے لکھے؛ جائزہ نگار 1 نے انگریزی میں اپنا جائزہ لکھا، جبکہ جائزہ نگار 2-3 نے اپنے جائزے اپنی مادری زبان میں لکھے؛ )5 (ہم نے غیر انگریزی جائزوں کا انگریزی میں ترجمہ کیا تاکہ ان کا مطالعہ کر سکیں، اور پھر جائزہ نگاروں کی تجاویز کو مدنظر رکھتے ہوئے مقالے کو نظر ثانی کی، مقالے کو دوبارہ تمام زبانوں میں ترجمہ کیا، اپنے جواب کی دستاویز لکھی اور اسے بھی تمام زبانوں میں ترجمہ کیا، اور پھر مقالے اور جوابات کو دوبارہ جمع کرایا؛ )6 (مدیر نے پھر مقالے کو قبول کیا اور ہم نے ذیل میں جائزہ کے انتظام پر بات چیت بھی شامل تھی، اور ہم نے اس آزمائشی doi کے تبصرے شامل کیے۔ جریدے کے ساتھ آخری مرحلہ فائل استعمال کرنے پر اتفاق کیا۔ pdf اور ایک doi عمل میں سادگی کے لیے صرف ایک کھلے پن کی روح کے تحت، اور شفافیت کو فروغ دینے کے لیے، مقالے کی منظوری کے بعد، ہم نے مقالے کے جائزہ نگاروں سے درخواست کی کہ وہ اپنے غیر ترمیم شدہ تبصرے نیچے فراہم کریں، تاکہ اپنی عکاسیوں کو کمیونٹی کے ساتھ شیئر کر سکیں۔ اسی طرح، انہوں نے کھلے پن کی اسی روح کے تحت نیچے اپنے نام ظاہر کیے (یونانی) تبصرے عکاسی کے 1 والے لینے جائزہ 4.1 اس مضمون کا تین زبانوں میں، بشمول میری مادری زبان یونانی، جائزہ لینے کا تجربہ انتہائی خوشگوار تھا۔ اس کام نے مجھے اس حقیقت پر غور کرنے کا موقع دیا کہ ہمیں انگریزی زبان کی بدولت حاصل ہونے والے مواقع پر شکر گزار ہونا چاہیے جو حوں پر، بشمول سائنسی سطح پر، جڑنے کا موقع فراہم کرتی ہے۔ تاہم، اس فائدے کے ہمیں دنیا بھر کے لوگوں سے کئی سط ( اہم متبادل پیش aiباوجود، ہمیں یہ تسلیم کرنا چاہیے کہ انگریزی ہر کسی کے لیے قابل رسائی نہیں ہے، اور مصنوعی ذہانت ) اضح مثال ہے۔ یونانی زبان کی وسعت اور پیچیدگی کرتی ہے جن سے ہم فائدہ اٹھا سکتے ہیں، اور یونانی ترجمہ اس کی ایک و نے شاید "کامل" ترجمے کی اجازت نہیں دی ہوگی کیونکہ، واقعی، کچھ گرامر کی اور دیگر غلطیاں تھیں، ساتھ ہی ایسے تاثرات ہے کہ مستقبل بھی تھے جو مقامی بولنے والوں کے ذریعہ زبان کے قدرتی استعمال سے مطابقت نہیں رکھتے۔ تاہم، مجھے امید کی خود کو بہتر بنانے کی صالحیت کے پیش نظر اس مسئلے کو حل کر لیا جائے گا۔ مصنفین کو یہ خیال پیش aiقریب میں کرنے اور اس مضمون کو لکھنے پر مبارکباد! (پیٹروس چماکیوٹس ) (ہسپانوی) تبصرے عکاسی کے 1 والے لینے جائزہ 4.2 یونانی ورژن کے حوالے سے میری اوپر کی تبصروں کے عالوہ، ہسپانوی زبان میں ترجمہ یونانی کے مقابلے میں بہتر معیار ( کو ہسپانوی میں یونانی کے مقابلے میں زیادہ بہتر "تربیت" دی گئی aiکا تھا، ممکنہ طور پر اس وجہ سے کہ مصنوعی ذہانت ) ہے، جس کی وجہ ہسپانوی بولنے والے صارفین کی بڑی تعداد ہے۔ نیز، اگرچہ میرا خیال ہے کہ الطینی امریکہ کے ہسپانوی عمال نہیں کرتے، اور یہ ایسی چیز بولنے والے ممالک میں ایسا نہیں ہے، ہسپانوی لوگ روزمرہ کی زبان میں انگریزی الفاظ است نہیں رکھا۔ مثال کے طور پر، انگریزی میں aiہے جسے یہ aiخفف کا م aiنے مدنظر ہے iaہے، جبکہ ہسپانوی میں ( بھی ہیں جنہوں false friends" ہے(۔ اس کے عالوہ، بہت سے "غلط دوست" )inteligencia artificial)کیونکہ اصطالح " " تھا جب تعلیمی کانفرنسوں کا حوالہ دیا جا conferenceنے ترجمہ شدہ ورژن کے معیار کو متاثر کیا۔ ان میں سے ایک لفظ " "( ترجمہ کیا جاتا ہے۔conferencia" )نہ کہ " congresoرہا تھا، جسے ہسپانوی میں زیادہ درست طریقے سے " (پیٹروس چماکیوٹس ) australasian journal of information systems burton-jones et al. 2025, vol 29, provocation this article is not just in english 54 جائزہ لینے والے 2 کے عکاس تبصرے )بہاسا انڈونیشیا( 4.3 یہ مقالہ تعلیمی ترتیبات میں انگریزی زبان کے تعصب کا مکمل جائزہ فراہم کرتا ہے اور اس بات کا جائزہ لیتا ہے کہ کس طرح ai ترجمہ کے اوزار نمایاں تبدیلی ال سکتے ہیں۔ اس مقالے کی بنیادی طاقت اس کی واضح مسئلے کی نشاندہی میں مضمر ہے، جس کی تائید مختلف شعبوں کے ادب سے ہوتی ہے جو لسانی تعصب کی فوری ضرورت اور منفی اثرات دونوں کو ظاہر کرتا انفارمیشن سسٹمز کے ai ہے۔ کثیر لسانی سائنس کے لیے وژن قائل کرنے واال اور اچھی طرح سے بیان کیا گیا ہے، جس میں شعبے کے لیے ایک موزوں جدید حل کی نمائندگی کرتا ہے۔ کی حدود جیسے کہ ai ،کی پیچیدگیوں کا ذکر کیا گیا ہے ai تاہم، کچھ شعبے مزید ترقی سے فائدہ اٹھا سکتے ہیں۔ اگرچہ hallucinations یا ثقافتی تعصب کو حل کرنے پر بحث کو مزید گہرائی کی ضرورت ہے۔ مثال کے طور پر، اس مضمون "our provocation to readers" کے پہلے مسودے میں کچھ انڈونیشیائی تراجم کسی حد تک سخت محسوس ہوتے ہیں۔ ")قارئین کے لیے ایک چیلنج (بہت "زوردار "tantangan kepada pembaca" )قارئین کو ہماری اشتعال انگیزی (کا ترجمہ ہم قارئین کو سوچنے کی دعوت ) "kami ingin mengajak pembaca berpikir" لگتا ہے ایک زیادہ قدرتی ورژن دینا چاہتے ہیں (ہوگا جو زیادہ "نرم "لگتا ہے۔ پیچیدگی کا تجزیہ سطح پر ہی رہتا ہے، خاص طور پر ترجمہ کی درستگی یا ثقافتی اثر جیسے تکنیکی مسائل کے حوالے سے۔ دالئل میں بھی مضبوط تجرباتی حمایت کی کمی ہے۔ مستقبل کی تحقیق میں پائلٹ مخصوص سیاق و سباق کی مزید کھوج، -is مطالعہ کے نتائج اور گہری تکنیکی تجزیہ کو شامل کرنے پر غور کیا جا سکتا ہے۔ جیسے انٹرپرائز سسٹمز میں کثیر لسانی عمل درآمد، بھی وسیع تر تعلیمی کمیونٹی میں اس مقالے کی شراکت کو مزید تقویت دے سکتی ہے۔ )عارف پردانہ ( جائزہ لینے والے 3 کے عکاس تبصرے )اردو( 4.4 اس بصیرت افزا دعوت پر غور کرنے کے لیے شکریہ۔ اس مضمون کا اردو میں جائزہ لینا ایک مفید تجربہ تھا۔ یہ پہلی بار تھا کہ میں نے اپنی مادری زبان میں علمی کام میں حصہ لیا، اور یہ تجربہ واقعی جامع محسوس ہوا، جو میں نے پہلے کبھی محسوس نہیں کیا تھا۔ مقالے کا کلیدی پیغام، کہ سائنسی علم کو ایک زبان تک محدود نہیں ہونا چاہیے، میرے دل کو گہرائی سے چھو گیا۔ یہ اس بات کو اجاگر کرتا ہے کہ انگریزی کی برتری کیسے ناخواسته طور پر سائنسی مباحثے میں شرکت اور سمجھ کو محدود کر سکتی ہے۔ مجموعی طور پر، اردو ترجمہ کافی عمدہ طریقے سے کیا گیا ہے۔ تاہم، بعض مقامات پر ترجمہ تھوڑا سخت محسوس ہوتا ہے اور روانی میں رکاوٹ بنتا ہے، کیونکہ یہ لفظ بہ لفظ ترجمے کی بنیاد پر ہے بجائے اس کے کہ اردو کے قدرتی جملوں کے مطابق اظہار کیا جائے۔ اس کے باوجود، یہ چیلنجز کثیر لسانی سائنس کی صالحیت کو کم نہیں کرتے؛ بلکہ یہ ظاہر کرتے ہیں ترجمہ کے آالت کے پاس سائنس کو ai ،کہ کہاں ابھی بھی اہم کام باقی ہے۔ انسانوں کی مسلسل شمولیت اور نگرانی کے ساتھ نہ صرف عالمی سطح پر قابل رسائی بلکہ مقامی سطح پر بھی معنی خیز اور اثر پذیر بنانے کی صالحیت ہے۔ (حمزہ نذیر) نتیجہ 5 زبان کا تعصب سائنس کی تاریخ میں ایک مستقل مسئلہ رہا ہے، لیکن اب ایسا نہیں ہونا چاہیے۔ مصنوعی ذہانت کے ترجمے کی ترقی سائنس کے لیے ایک نئے مستقبل کا امکان پیدا کرتی ہے—ایک کثیر لسانی مستقبل۔ ایک کثیر لسانی سائنس زیادہ شمولیت والی ہو سکتی ہے )زیادہ مصنفین اور جائزہ نگاروں کی شرکت کو ممکن بنانا (اور زیادہ اثرانداز ہو سکتی ہے )زیادہ قارئین کے لیے سائنس کی رسائی ممکن بنانا(۔ کسی بھی ڈیجیٹل تبدیلی کی طرح، اس تبدیلی کے اثرات غیر متوقع ہو سکتے ہیں، اور سیکھنے کے لیے بہت مواقع ہوں گے۔ انفارمیشن سسٹمز کے میدان میں اس سے زیادہ موزوں مواقع کم ہی نظر آتے ہیں، اور عمل کا وقت اب آ پہنچا ہے۔ ہم اپنے کام کی پیش رفت کے ساتھ اپنی بصیرتیں شیئر کرنے کے منتظر ہیں اور ہمیں امید ہے کہ بہت سے دیگر بھی اس کوشش میں شامل ہوں گے۔ paper title (paper title style) australasian journal of information systems sugianto et al. 2023, vol 27, research article enterprise social media during the covid-19 pandemic 1 the role of enterprise social media during the covid-19 pandemic: insights from leaders’ experience ly fie sugianto monash business school, monash university, melbourne, australia lyfie.sugianto@monash.edu chuying cheng deakin university, melbourne, australia carly moulang monash business school, monash university, melbourne, australia brett considine monash business school, monash university, melbourne, australia abstract the covid-19 pandemic impacted workplaces, with public health orders requiring people to shift their workplaces into their homes. consequently, many organisations pivoted to online operation and utilised technology such as enterprise social media (esm) to help manage this transition. in this study we explore leaders' diverse use of esm during the pandemic, including whether it was used for performance management and how it shaped leaders’ social behaviour. we conducted fifteen semi-structured interviews with leaders in a large australian university using the esm technology. our results explore the nuances of esm use during this time including how it was used as a social tool, a communication tool, and as an informal means to collect performance data. interviews also revealed concerns with esm use such as privacy and information redundancy. our work advances the task-technology fit (ttf) literature by conceptualising cognitive and affective mechanisms to understand how utilisation moderates ttf outcomes. these mechanisms are contingent on how leaders use esm and the level of their interactions and engagements. we identify practical implications of esm use at a time of crisis including leader training, clear guidelines for internal communication, efficient information sharing practices, and informed consent for esm-related data collection practices. keywords: enterprise social media, covid-19, task-technology fit, performance management, social technology, leader member exchange. 1 introduction over the last decade, social networks, blogs and wikis have been employed at work for both formal and informal purposes to enhance company communication, foster collaboration, and invigorate knowledge sharing (sun et al., 2020; sun et al., 2019; archer-brown & kietzmann, 2018; kwahk & park, 2016; ellison, et al., 2015; riemer & scifleet, 2012; wagner, 2004). the reliance on social networks and social technology increased during the covid-19 pandemic, as public health orders including lockdowns and curfews forced many employees to work remotely. this significant change in work environments and the associated uncertainty resulting from the pandemic led many organisations to deploy or more intensely use enterprise social media (esm), a social technology specifically designed for internal organisational use (leonardi et al., 2013). australasian journal of information systems sugianto et al. 2023, vol 27, research article enterprise social media during the covid-19 pandemic 2 studies on the adoption and use of esm have reported various organisational benefits such as improved operations, innovation and human relations (kane, 2015), and revised internal communication strategies to increase staff engagement (ewing et al., 2019). one recent study also reported positive effects on employee creativity (wang et al., 2022). a glaring omission in the literature, however, concerns the role of leaders1 in supporting and endorsing esm. we contend that it is important to understand leaders’ perceptions of esm use since they are the champions, decision makers, and supporters of esm adoptions and play a significant role in their success (avanade, 2013). as esm users, leaders possess specific characteristics that distinguish them from other users due to their roles. for example, leaders have considerable influence on their teams; thus, it is possible that their esm use, such as the way they communicate, share information and provide feedback, is perceived by employees to carry social influence that can shape employees’ behaviours and attitudes. yet, to the best of our knowledge, research into how leaders’ use esm is non-existent nor is there research on how leaders use esm during a time of crisis such as during the covid-19 pandemic. esm presents new opportunities for leaders to shape employees’ communications, interactions, engagement and work patterns. it also challenges them to develop new skills in engaging and managing their staff virtually. existing literature on virtual leadership and team leadership mainly addresses work related matters (rather than social connections), and occasionally discusses technology, albeit as an enabling or supporting structure rather than the study’s focus (see sidorova et al., 2016; purvanova & bono, 2009; avolio & kahai, 2003; zigurs, 2003). the extent to which organisations derive benefits from technology relies on leaders’ perceptions, attitudes, utilisation and leadership skills. leaders’ abilities to effectively mobilise and integrate people, processes and technology within organisational systems are vital in fostering the analytical capabilities of new technology and generating value for the organisation (roman et al., 2019; wee et al., 2022). this argument also has support in recent literature on technology use and leadership style. it has been widely understood that following the deployment of new technology in an organisation, leaders express their attitudes, behaviours, and performance via online social technology to communicate clearly with employees, provide good social connections, and make the team be accountable (roman et al., 2019). leaders’ use of new technologies helps shape the cognitive and affective process of their employees, develop employees’ identification in teams, and establish social capital across the organisation (kahai et al., 2012; kahai et al., 2017). social technology also allows for a remote workforce which may also include people from other countries, and irrespective of this, an issue of working remotely is that people may not be as aware of colleagues social and cultural identities. leaders need to understand culture differences and be respectful, inclusive and provide a safe workplace (kahai et al., 2017). trust and leadership are difficult to develop and maintain in virtual environments especially in times of crisis. some leaders struggle to use new social technology (such as esm) and manage their team in virtual environments (mitchell, 2021). other leaders find creative ways to motivate and engage their teams by establishing goals, communicating regularly, and checking employees’ task status (mitchell, 2021). leaders also distribute their leadership 1in this study, we use the term ‘leaders’ to represent senior staff who also have staff supervision responsibilities in terms of managing the performance of other staff. australasian journal of information systems sugianto et al. 2023, vol 27, research article enterprise social media during the covid-19 pandemic 3 responsibilities within an organisation in order to improve decision quality via social technology (fernandez & shaw, 2020; gamo-sanchez & cegarra-navarro, 2015). the new ways leaders manage team performance via social technology could become essential in a pandemic and post-pandemic world. in short, studying leaders’ esm use can potentially reveal their communication patterns and how they manage, engage and connect with employees during a crisis. prior studies find that while a leader's esm engagement can set the tone for the organisation’s social media culture, it also blurs the existing hierarchy’s relevance as it connects top-end leaders directly with employees (ewing et al., 2019). it is possible that leaders’ activity on social media could be deemed an information source for employee performance management (sidorova et al., 2016); however, there has been minimal academic enquiry into leaders’ perception of esm as a tool to facilitate performance measurement systems. little is known about whether leaders’ esm use impinges upon their subjective evaluative performance management decisions. employees using esm to promote their work and performance may improve their supervisor’s perceptions of their performance (bauch et al., 2021; robbins & denisi, 1994). a better understanding of the role of esm in facilitating performance management is vital given the impact of supervisors’ judgement on employees’ performance, which has direct consequences on workload responsibilities, renumeration and future career trajectories. as such, the objectives of this paper are twofold: (1) to explore leaders’ general use of esm during the covid-19 pandemic, and (2) to examine leaders’ specific use of esm for performance management. to understand esm perception and use, we conduct a qualitative study and interview people in leadership and supervisory roles at a large australian university. we use the task-technology fit (ttf) model to study the fit between the organisational tasks and the esm’s technological characteristics, and esm use. the ttf model has been examined in various contexts and provides evidence that technology can positively impact performance when used to manage tasks (chang, 2008; chung et al., 2015; howard & rose, 2019). a recent study on work performance during the covid-19 pandemic reported that the degree of ttf in information and communication technology (ict) use had both direct and indirect effects on individual performance (abelsen et al., 2021). following prior research, we employ ttf as a framework to study the use of and potential benefits organisations can expect from esm, drawing on the model from howard and rose (2019) (see figure 1). figure 1. task-technology fit model (source: howard & rose, 2019) australasian journal of information systems sugianto et al. 2023, vol 27, research article enterprise social media during the covid-19 pandemic 4 the theory postulates that when technology is used when conducting a task, the interactive effects of task and technology characteristics are mediated by ttf. it is important to note that the theory does not emphasise any particular task or technology characteristic. rather, it highlights the overarching dependence on technologies. the theory also posits that the degree of utilisation influences how ttf impacts both user reactions and performance outcomes; thus, utilisation moderates the relationship between ttf and user reactions and performance outcomes. although figure 1 portrays several components to illustrate the theory, the primary focus of prior work is on how ttf impacts outcomes. further, we complement this approach with an outcome-perspective, assessing both the positive outcomes (benefits) and negative outcomes (concerns or drawbacks) resulting from esm use. while the ttf model does not generally distinguish between different user groups, this study applies ttf to a specific user group – namely leaders with managerial and leadership roles within the organisational unit – to gain deeper insights into their experiences and interactions with esm. our complementary approach aims to provide clarity on how leaders use esm. during mandatory periods of remote working, organisational leaders need to perform additional tasks to effectively manage their teams and ensure their productivity. these tasks include effectively communicating and collaborating, team building, providing work policies and guidelines, fostering a sense of connectedness, and adapting performance management processes to the remote work environment. using ttf as the underlying theoretical framework, this study explores leaders’ use of esm to perform these tasks in addition to their other tasks. furthermore, given leaders’ specific role in managing performance and using esm to enable social connections during the covid-19 pandemic, this study also applies the leader-member exchange (lmx) theory to explore how leaders use social technology to manage their relationships with employees. our study contributes to the literature on esm use in organisations (bulgurcu et al., 2018; ewing et al., 2019; hacker & riemer, 2021; van osch & steinfield, 2018), on digital leadership and the use of advanced information technology (banks et al., 2022; larson & dechurch, 2020; matthews et al., 2022; nieken, 2022), and on the ttf model. earlier research shows that leadership mediated by technology can exhibit the same content and style as traditional faceto-face leadership (avolio & kahai, 2003); yet, how technology affects leadership is still unknown (avolio & kahai, 2003; zigurs, 2003). fu et al. (2020) find that esm use needs to consider both task environment and technology affordances, highlighting how esm use emphasises developing connections rather than bonding already strongly tied individuals. our results add to those survey results by providing a qualitative understanding of how managers engaged with esm to link organisation members who had their social connections disrupted by the pandemic. as such, it highlights how organisational leaders can engage with esm and take the initiative in using it to preserve social connections. the findings on leaders’ perceptions enrich the growing literature on digital leadership, eleadership and leadership effectiveness, by depicting leaders’ different technology use behaviours in social settings. we also provide insights into esm use during the covid-19 pandemic, which arguably accelerated leaders’ uptake and use of esm. despite working remotely, esm affords leaders ongoing ‘visibility’, facilitates interactions with employees, and provides a formal means of engaging employees to remain productive. the remainder of this paper is structured as follows. first, we present a literature review on esm use in organisations, its impact on leaders’ behaviour, and its use in performance australasian journal of information systems sugianto et al. 2023, vol 27, research article enterprise social media during the covid-19 pandemic 5 management, which together provide the basis for our research questions. second, the qualitative research method using thematic analysis is outlined, including data coding and validation. third, we report the findings on leaders’ esm and two key themes arising: ‘positive outcomes on esm use’ and ‘concerns over esm use’. fourth, we offer a discussion on how the findings answer the research questions and the study’s implications. fifth, we summarise the study, outline its limitations, and identify future research that can complement this study. 2 literature review 2.1 esm use in organisations during the covid-19 pandemic enterprise social media (esm) can be defined as a social platform specifically designed for internal use within an organisation to maintain employees’ connectedness, collaboration and communication. esm can also benefit organisational learning, knowledge sharing, and managing human resources (leonardi & vaast, 2017; ouirdi et al., 2015). several affordances in esm that make it unique within the organisational context are reported in sun et al. (2019), including associating (majchrzak et al., 2013; oostervink et al., 2016), visibility and persistence (oostervink et al., 2016; leonardi et al., 2013). the associating feature enables employees to establish connections among: (1) them as users, (2) content, and (3) between users and content (treem & leonardi, 2012). the visibility feature enables employees to read other exchanges in esm, and the persistence feature implies that information remains available and does not disappear – both features are valuable for organisational knowledge sharing and providing an information repository. the use of esm can enhance workplace integration and increase employees’ performance by increasing their sense of belonging and increasing their social and emotional interactions (moqbel & nah, 2017). the use of esm during the covid-19 pandemic is an example of the potential for technology to allow informal and asynchronous interactions, facilitating staff connectedness and productivity. in this instance, esm enables staff to connect with their leaders and peers, allowing them to overcome the feeling of loneliness during remote working. in essence, esm serves as a social technology where social interactions (abelsen et al., 2021; odekerkenschroder et al., 2020) and communication occur (van osch et al., 2015), allowing for collaboration among staff (harrysson et al., 2016). typical esm features include the ability for a user to directly message another user in a public setting (specific to a recipient); broadcast a message (to a specific group or all); indicate connections with other colleagues; post/edit/view messages; respond to a message by commenting, showing “likes”, and sharing messages of interest. it also permits more dynamic, multimedia communication, such as video posts. thus, not only is esm a means for transferring and preserving organisational knowledge, but it also shapes institutional knowledge through the dynamic process of formal and informal daily interactions. it is within esm where knowledge sharing, employee collaboration, and organisational learning occurs. esm benefits both the organisation and staff, allowing its content to evolve through collaborative processes as staff with different perspectives post their opinions to the extent that consensus decision-making is made possible. staff are kept up to date with the latest news and can access important information. esm benefits organisations as staff tend to share posts on perceived important information, enabling the effective sharing of best practice across organisational units. from a management perspective, leading and motivating staff to maintain work productivity is a new challenge in a remote working environment. thus, esm australasian journal of information systems sugianto et al. 2023, vol 27, research article enterprise social media during the covid-19 pandemic 6 potentially becomes a relevant organisational system for sustaining employee engagement, facilitating communication, and potentially developing deeper connections with employees. recent literature has examined esm features and applications, including affordance for visibility (leonardi & vaast, 2017; rice et al., 2017), patterns of use (bulgurcu et al., 2018), use in engaging employees (ewing et al., 2019), network formation (van osch & steinfield, 2018), and the user’s role in esm (hacker & riemer, 2021). however, research about leaders’ role in supporting and endorsing esm use is lacking. leaders’ perceptions of esm use are important because they are often organisational champions who support and endorse the adoption of new systems such as esm (avanade, 2013). yet, research into leaders’ esm use is missing from the literature. further, the importance of understanding esm’s use during a crisis has come to the fore during the covid-19 pandemic. we address this research gap by exploring how leaders perform their tasks using esm during the covid-19 crisis, with ttf as the theoretical framework. through this study, we seek to understand esm use by leaders during remote working periods, leading to the first research question: rq1: how did the covid-19 pandemic impact the use of esm by leaders? 2.2 esm as a social technology and its impact on leaders’ behaviour drawing upon the leader-member exchange (lmx) theory (liden et al., 2006) which states that organisational members evolve in their roles through the social interactions with their leaders (graen & scandura, 1987), leaders’ social behaviour and interactions on an open platform like esm has the potential to shape their employee’s behaviours. specifically, leader’s interpersonal emotion regulation motives are related to the quality of the leader-employee relationship and leaders’ effectiveness in the group (vasquez et al., 2021). as such, when leaders express their opinions on esm, there is a spotlight on their affective tone and social behaviour. leaders also form cognitive impressions of their employees (forgas, 1981) as they can view employees’ reactions through the comments and their responses (such as through emojis) to postings. in turn, employees will compare their leaders’ behaviour relative to those of other leaders in the organisation (deluga, 1994). employees’ responses to leaders’ posts and comparisons of leaders’ behaviour will form a reciprocal process which further shapes leaders’ social behaviour. the covid-19 pandemic potentially placed additional pressure on leaders to engage in more digital labour, enabled by esm use (till, 2013, 2014). this could be construed as “cognitive labour” which is characterised by forms of intellectual and affective labour (till, 2014). the changed work environment resulting from the pandemic required new labour practices, such as leaders relying on alternative means to express support for their employees. this form of affective labour is considered ‘immaterial labour’, which is ‘...labor that produces an immaterial good, such as a service, knowledge or communication’ (hardt, 1999, p. 94; lazzarato, 1996). affective labour is one face of immaterial labour in service firms, such as universities, and involves human interaction and communication. while this was traditionally observed “in-person”, the evolution of on-line activities means that virtual presence is no longer seen as any less-real than physical presence (hardt, 1999). in sum, we anticipate that when leaders express their opinions on an open platform such as esm, they deliberately focus on the affective tone and are mindful of their social behaviour. given the potential salience of esm in supervisory relationships, and the lack of empirical evidence on how it impacts leaders’ behaviour (avolio & kahai, 2003; zigurs, 2003), we seek australasian journal of information systems sugianto et al. 2023, vol 27, research article enterprise social media during the covid-19 pandemic 7 to understand the effect of esm on leaders’ social behaviour by formulating the second research question as follows: rq2: how does esm as a social technology shape leaders' social behaviour towards employees? 2.3 esm use and performance management while not designed for performance management, esm communications include knowledge sharing and knowledge acquisition (sun et al. 2020) which may become a source of information that influences leaders in their performance management roles. this is potentially more salient during a crisis, such as during the covid-19 pandemic, which required remote supervision of staff in many industries. during this time both leaders and employees could potentially use esm to convey performance-related information to a broad audience. employees, through posts, discussions and other esm interactions, could promote their performance and achievements, including work outside of their official performance metrics. during the covid-19 pandemic, esm use has helped managers communicate efficiently and effectively with users and make strategic decisions in disaster and emergency situations, increasing operational and social performance (dwivedi et al., 2022). esm provides a platform for employees to share and acquire knowledge, and enhance their creative performance (sun et al., 2020). esm provides opportunities to manage performance impressions when there are limited informal interactions with leaders and other employees. leaders may process this information and form impressions of interpersonal-affect towards employees, thus enhancing their ‘likability’ (bauch et al., 2021; robbins & denisi, 1994). the process of subjective performance management can be viewed as a judgement and decision-making task because leaders need to search, integrate and evaluate information to make judgements and decisions about employees’ performance (highhouse et al., 2013). the use of esm provides an opportunity for leaders to observe employees’ performance remotely, potentially leading to more favourable performance outcomes for employees (judge & ferris, 1993). when supervisors are making a judgement, they need to retrieve and process information from their memory – including affective information (forgas & george, 2001). affective reactions to employees can play a significant role in the cognitive process of ‘recall’, facilitating information recall about the employee and being consistent with the leader’s affective reactions to that employee (forgas, 1981). the affect-as-information approach demonstrates that affect is a type of information and the information is conveyed by affective feelings (clore et al., 2001). leaders' affective attitude towards their employees can also influence their cognitive processes in the performance appraisal process; if an employee is liked, leaders may interpret any poor performance as not meaningful. robbins and denisi (1994, p. 350) conclude that ‘affect influences both the processes and outcome of performance evaluation’. if and how esm was used to support performance management by leaders during covid-19 leads to our third research question: rq3: was esm used by leaders for the purpose of performance management during the covid-19 pandemic? australasian journal of information systems sugianto et al. 2023, vol 27, research article enterprise social media during the covid-19 pandemic 8 3 research method 3.1 data collection this study uses qualitative interviews2 to capture accounts of “what people are doing…[and] apprehend a new language and form of life” (chua, 1986, p. 615). the search for an in-depth understanding of how esm was applied and the different experiences encountered made qualitative semi-structured interviews best suited to our study (power & gendron, 2015). this method recognises that “conversations developed in interviews are an indispensable source of rich and new knowledge about social and personal aspects of our lives” (smith & sparkes, 2016, p.107). given we collected data about a time when governments enforced social distancing, the social, personal and professional experience of respondents became intrinsically connected. the interview approach allowed us to explore dimensions of esm use that were meaningful to the respondent and framed in terms of the respondent’s experience. it also provided insights into how leaders processed their decisions about esm use and balanced competing motivations, incentives, and emotional challenges (smith & sparkes, 2016). this approach is consistent with prior work that has sought to understand organisational use of technology (see for example, dowling & leech, 2014; treem, 2015). we conducted fifteen individual semi-structured interviews with senior leaders at a large australian university3 that has used an esm since 2019. the research team developed and reviewed an interview protocol (see appendix), with the questions informed by prior research on esm applications and performance management challenges. participants were put “at ease” (leech, 2002, p. 665) by first being asked to describe their role and experience at the institution through a “grand tour” (leech, 2002, p. 667) style question. this provided contextual background for researchers to draw upon in subsequent questions. one member of the research team led the interviews, but other team members were present and could seek clarification, follow up, or raise any further questions during the interview’s closing stage. the research team consistently followed the interview protocol in all interviews, providing a constant structure behind the data generation. the majority of interviews were conducted with two or more of the research team present. after each interview concluded the interview team would briefly discuss any interesting ideas that emerged and reflect on the overall interview experience. interviews were held between march and june 2021 during which time most university staff were working from home and leaders were remotely supervising staff. to comply with public health orders, zoom was used to conduct interviews (archibald et al., 2019; howlett, 2022; oliffe et al., 2021; smith & sparkes, 2016). while this had the occasional challenge of bandwidth limitations and frame drops, it provided a visual and audible interview that allowed for engagement in conversation and the observation of participant body language and visual cues (howlett, 2022; smith & sparkes, 2016). participants were recruited based on a review of department lists to identify senior staff who were in supervisory roles, and those 2 research ethics clearance was gained through monash’s human research ethics approval process. this occurred before participants were recruited into the study. 3 the case site is an australian universities, where a large majority of teaching has been traditionally based on campus. we refer to it as ‘delta university’ in this study to retain anonymity of the organisation. the authors did not have any direct working relationship with any of the interviewees. australasian journal of information systems sugianto et al. 2023, vol 27, research article enterprise social media during the covid-19 pandemic 9 individuals were sent email invitations to participate in the study. table 1 lists the interview participant demographics. interviewee number male/ female supervisory experience (years) number of employees supervised number of researchers present at interview interview length (minutes) 1 female 21 9 3 61 2 male 21 7 3 25.18 3 male 3 4 2 43.01 4 female 6 8 1 45.34 5 male 9 6 2 36.58 6 male 23 5 2 32.39 7 male 21 11 1 30.27 8 female 4 1 2 34.29 9 female 4 12 2 18.05 10 female 25 43 4 32.38 11 female 15 17 3 34.47 12 female 22 13 2 29.15 13 female 15 10 3 49.47 14 female 17 12 3 46.45 15 female 15 4 3 43.36 table 1. interview participants leaders with differing roles and responsibilities are coded based on their job characteristics as type 14 and type 25, and were further categorised based on their roles and esm use characteristics – table 2 contains this analysis. job characteristics esm use characteristics type 1 type 2 active, regular 5 participants (interviewee 8, 9, 10, 12, 13) 5 participants (interviewee 2, 3, 4, 5, 14) sparing, irregular 1 participant (interviewee 15) 4 participants (interviewee 1, 6, 7, 11) table 2. profiling the interviewees with technology use and job characteristics 3.2 data analysis interviews were recorded and then transcribed verbatim by two researchers. transcripts were independently reviewed by the other researcher, checking for errors and noting points of interest. the process was consistent with schmidt (2004), with transcripts read over by the 4job type 1 involves administrative tasks, focusing on supporting and facilitating the functioning of an organisational unit. 5 job type 2 involves knowledge intensive activities, focusing on the advancement of knowledge, exploration of ideas, curriculum development and other scholarly activities. more specific details of the roles and titles are withheld to protect the anonymity of the research participants. australasian journal of information systems sugianto et al. 2023, vol 27, research article enterprise social media during the covid-19 pandemic 10 research team, looking for analytical categories and points of interest, paying attention to “the topics that occur and individual aspects of these which can be related…to the context of the research question(s)” (schmidt, 2004, p.352). data coding was hybrid in nature, being both inductive and deductive in nature (fereday & muir-cochrane, 2006). our initial coding categories were inductive, taking the experiences and perceptions of the users in the empirical data and coding these based on the types of technology, tasks performed, task frequency, extent of social technology use, thoughts on the technology, and other context specific dimensions. this allowed the data to speak on its own terms. the inductive classifications from the initial analysis then allowed for a deductive classification that was derived from the ttf framework. at this stage we paid particular attention to the types of technology used, tasks that participants described, how technology was used, and any evidence of fit/misfit and the different views within the ttf’s theoretical constructs. this allowed the collective experiences of all participants to be coded within the ttf framework. quotes from interviews were then analysed for positive/negative reflections of esm use. this hybrid inductiondeduction process was the basis for the main thematic issues discussed in the following section. it allows the data to speak in the voice of the participants while also coming from an underlying theoretical bases – this provides the basis for further theorisation and discussion. observations and themes were recorded in a shared spreadsheet the research team then reviewed together and used as the basis for the analytical categories presented in this paper. the transcripts were then coded manually using excel. a worksheet was set up that contained headings based on the identified emerging themes and quotes were coded based on the themes they referred to. another member of the research team reviewed the coding, with this providing consistency. all coding occurred in a shared spreadsheet that allowed for the tabulation of interview references based on themes and leader type. it became apparent that the ttf model offered a way of viewing the data, allowing us to capture the different tasks participants referred to and their varied use of technology. the usage pattern of the interview participants is identified and mapped using the esm features of creating content, commenting, liking, and forming groups. apart from developing our understanding of different esm use patterns, the ttf model was also used to discover emerging themes by analysing the capabilities and limitations of the technology. taking an outcome-perspective approach, these advantages and disadvantages resulting from the use of esm are presented as positive outcomes of esm use and concerns regarding esm use. themes emerging from the analysis included social use, communication, team building, recognising achievement, timely feedback, and connecting to others, are grouped into three themes under positive outcomes of esm use. themes such as privacy concerns, information overload, worknonwork boundary concerns, and inappropriate content are grouped into three themes under concerns about esm use. word clouds6 were also used as an exploratory qualitative data analysis tool (cidell, 2010) and helped highlight main themes and key differences across leaders and job characteristics (suddaby et al., 2015). as a visualization aid for exploratory textual analysis, they helped identify the salient themes and stimulated further “analytic reflection” (miles et al., 2020, p. 44) on the context in which the word appears. 6 generated using leximancer australasian journal of information systems sugianto et al. 2023, vol 27, research article enterprise social media during the covid-19 pandemic 11 4 results several themes emerged from our interviews. using the ttf model, the themes are summarised in tables 3 and 4 for tasks based on job type and task-technology mapping. job type 1 job type 2 communication and information sharing communication x x knowledge sharing x x promoting events x x connection collaboration x x team building x employee engagement x x sense of community x x productivity and performance recognition for performance x x digital workflow x learning x x table 3. tasks based on job type creating content commenting sharing links liking/ emoticon groups communication x x knowledge sharing x x x promoting events x x x x collaboration x x x x x team building x x x x x employee engagement x x x x sense of community x x x digital workflow x x x learning x x recognition for performance x x table 4. tasks-technology mapping from interview data we broadly categorised the emerging themes as ‘positive outcomes of esm use’, namely (1) communication, (2) connection, and (3) recognition and performance; and ‘concerns over esm use’, namely: (1) privacy, (2) redundancy, (3) consent and control. australasian journal of information systems sugianto et al. 2023, vol 27, research article enterprise social media during the covid-19 pandemic 12 4.1 positive outcomes of esm use the covid-19 pandemic and associated public health orders presented several challenges for many australian universities, with most staff required to work remotely for an extended period. this required rapid pivoting by all staff to modified ways of working; the effective dissemination of information was critical to this success. the esm played a key part in providing a forum for communication, connection and recognition of staff that helped them navigate their way through the pandemic crisis. 4.1.1 communication top management used the esm extensively during covid-19, using university-wide broadcasts to share information and informative videos. for instance, ‘the vice chancellor shared that kind of talk a while ago… so you can click on the video to access it quickly; so, it's quite intuitive’ (interviewee 6). this top-down communication was also described by interviewee 12 as ‘we see that more as an institution-wide sharing tool, … clearly it's a really important communication channel for institutional messaging’. key information from the university’s chief health officer was also communicated to all staff this way. institutional use of esm during the crisis made leaders consider whether they too should deliberately engage with esm and make it their ‘new normal’ way of communicating. as interviewee 13 observed, ‘i made a business decision that i wasn't going to use email to communicate really important messages to the whole group anymore since we've introduced [esm].... i normally use [esm] to communicate it’. similarly, interviewee 3 commented, ‘i try to avoid using emails. because we are already swamped with emails from students so i don't intend to go into that space’. this was also viewed by some as a way to demonstrate more direct engagement between the university and its staff, ‘…but that top down information sharing, which is important because you have to get those leadership messages, i think it's become more… perhaps meaningful, in the sense that... it allows for some direct engagement’ (interviewee 12). during the pandemic, staff needed access to a wide array of resources; the esm was a means to access information that was not easily found elsewhere. esm was seen as a timelier platform to find information: ‘in the past we were relying on just broadcasts and things like that, i find that it's [esm] a quicker way of finding out information’ (interviewee 13). the speed of access to information available via the esm was also described by other leaders; for example, ‘so that [esm] has certainly helped in that you have a quicker way of getting access to information’ (interviewee 6). with an abundance of information to share, some users moved away from traditional email communication and onto the esm, seeing it as a more effective way to relay messages to staff, ‘so over covid it was quite useful to get those messages out’ (interviewee 9). ‘so, i actually stopped using our group email address and went across to [esm]’ (interviewee 13). as to whether esm would endure post-pandemic, one participant indicated it might endure, although use may decrease once staff return to campus. ‘since we've been back in the office, certainly usage has dropped off, but again, everyone knows that. so, in a situation like today, there was a bit of activity and, so i think it will endure, but the extent to which it's used on a daily basis will fluctuate depending on circumstances’ (interviewee 12). 4.1.2 connection esm was also used to help with community building, to form a sense of belonging and unity, and to signal support and socialise. one interviewee described esm during this time as ‘a australasian journal of information systems sugianto et al. 2023, vol 27, research article enterprise social media during the covid-19 pandemic 13 platform for greater interconnectivity between staff’ (interviewee 3). esm was also used for team building and signalling desired behaviours, along with connectivity and team harmony, ‘with our own group, if any member in my team post something i’ll make sure that i ‘like’ or i comment or i engage in that and try and lead by example’ (interviewee 13). this described a way that esm impacted leaders’ social behaviour. ‘i think, maybe indirectly it improves team harmony and team communication and that's really important for team performance and over the last 18 months, there has been an important level of connectivity’ (interviewee 15). the need to feel a sense of belonging to the university was high at this time, particularly when facing so much uncertainty where australian universities were one of the only sectors not supported by pandemic fiscal measures to retain jobs, leading many universities to substantially cut staffing levels, coupled with a significant drop in overseas students as the country was in lockdown. esm helped create unity between staff across the university, ‘the creation of a kind of unified culture within [delta] and also with covid i think that has been useful because it allows everyone to share information and just to know what was there’ (interviewee 6). several leaders also raised the importance of bonding, ‘absolutely yes ... makes your relationship with them feel a bit closer and supports the bonds ... particularly important in the last year, given that we have worked remotely’ (interviewee 13). ‘as a broader community and across the […] school, yes, i think there's more bonding’ (interviewee 12). bonding as a form of social behaviour across the hierarchy was therefore facilitated by esm use. many leaders also used esm to show passive support for their staff. some engaged in this enthusiastically and others felt a sense of obligation to do this while other opportunities to show support were limited. several leaders gave examples of support and engagement through their social behaviours. this included demonstrating engagement, ‘if i ‘like’ something and people notice it’s good, that they'll know that i'm paying attention in a way that i can't directly do otherwise. when i first took over as interim dean, i actually posted something via email with an attached video that we put on our youtube channel, but i put it on [esm] as well, so i thought a group of people would see, and that was just a really personal outreach about how we're all in this together, kind of as one’ (interviewee 12). leaders also used esm to signal support, ‘and every time i see my team posting something or my colleagues posting something, i am taking notice that i’m quite often one of the first to post them a ‘thumbs up’ or you know some sort of ‘congratulations’ message there’ (interviewee 1). another leader said they used this to communicate a sense of pride, ‘like if any of our researchers have very good publications, they are launching a book, so i sometimes will even help them to post into the [esm], just because i feel so proud of them’ (interviewee 8). another added, ‘i’ll get a real buzz, it's not just oh that's great you know they got that big grant from the government or whatever, but it's like you're doing something meaningful and impactful and i feel really proud of that’ (interviewee 11). this passive support could potentially impact leaders' interpersonal affect towards their employees. as employees are likely to use the esm to communicate information deemed desirable, this could also result in their leaders perceiving them more positively. esm was also used as a mechanism to feel connected to staff by finding out more information about them, and making sense of how individuals present themselves on esm and enabling collective sense-making. ‘this is really called bridging social capital... so, if i compare myself, pre[esm] versus post-[esm], i feel myself more connected to the community. i'm more informed, well informed’ (interviewee 3). ‘and i think it is interesting, we present ourselves individually on [esm], but we also present ourselves collectively as a department on [esm]’ (interviewee 11). australasian journal of information systems sugianto et al. 2023, vol 27, research article enterprise social media during the covid-19 pandemic 14 socialising via esm was also common through social activities: ‘we ran quite a few you know trivia games and things to just get people engaged and i reckon we got more engagement with that than we normally would have pre-[esm]’ (interviewee 15). esm was also used to acknowledge celebrations that may traditionally occur in person or over email, like introducing new staff members and recognising birthdays. for instance, ‘i have a team, we established kudos cards, where every month we give kudos to teams, or to people that have done a great job, so we post on there, we also use it to celebrate people's birthdays’ (interviewee 13). 4.1.3 recognition and performance esm enabled staff to publicly announce their achievements and receive recognition from leaders. this was viewed as motivational for staff: ‘we do make use of [esm], especially to promote some of the research activities, events and also as a platform to motivate staff, congratulate the achievement, especially for the researchers’ (interviewee 8). providing supportive signals like a thumbs up or a comment on achievement posts were ways to provide acknowledgement: ‘yes, there are certain achievements we do feel that needs to be celebrated, so definitely we ’like’ them and [make] ‘comments’’ (interviewee 5). when staff post on esm about their activities, they are often viewed as being highly engaged and going ‘above and beyond’ to leaders. ‘because i think the first thing it will tell me is: they're engaged ... with what's happening on campus; secondly, it fills the gaps in terms of the various activities that they're doing, to either view their career, or to try to meet the targets that we set for them. this will be a sort of, like, informal target, or informal signals for me, and i would leave a ‘comment’ and ‘like’ on their posts’ (interviewee 3). when asked about using esm for performance evaluation purposes, leaders were hesitant to say that information from esm was used for these purposes. however, there was consensus that esm provided ‘additional information’ and a good ‘opportunity to observe performance’ at a time when physical interactions and informal chats with staff about their activities were restricted. ‘i may use it to include into the performance appraisal as seeing the staff that actually have such initiatives to do that. i would see it as something that they do as an extra mile. and then, something that i can include it to the year-end performance appraisal as an additional piece of information in performance evaluation ... i would prefer the staff who are perceived to be more positive to me; who are willing to actually walk the extra miles’ (interviewee 8). ‘having them (employees) post something online will give me an idea of the kind of research work that they're doing, or whether they attempted to reach out to industry partners by engagement. you know, so without those indicators or signals, i have to rely on other ways to kind of find out what are their strategies? what are the actions that they've taken to increase the research output and to engage with either internal or external stakeholders?’ (interviewee 3). another leader who was new to their role during the pandemic period used esm to learn about her employees; ‘… the stage of where i’m at, where i’m both still getting to know my staff and i've not been here long enough to know a lot about them, that i’m still learning new things about what they do, as well as observing and thinking about them in an ongoing way …’ (interviewee 11). employees can therefore use esm to make an impression on their supervisor, particularly if it is evident to the employee that their supervisor interacts with their posts. when employees post about their achievements on esm it can signal to the supervisor performance outside of the formal performance measurement system metrics. one leader described this as follows: ‘because at the end of the day, as a leader, we still look at the matrix: what is the number of publications? your teaching score, for instance? however, if, let's say someone doesn't australasian journal of information systems sugianto et al. 2023, vol 27, research article enterprise social media during the covid-19 pandemic 15 have a publication out yet, i know that this person has been actively engaged in the research work’ (interviewee 4). employees who are more prone to post via esm could also be motivated by potential future benefits and the chance to establish a reputation within the organisation: ‘people who post might get a bit more advantage in the sense that, they get better exposure throughout the university’ (interviewee 5). in addition, esm is viewed by some as an important place to create their reputation within the university for the purposes of promotion. one leader commented, ‘we're not promoting ourselves [face-to-face] and [esm] is the place we do that, where we build our reputation... you are building your own reputation and you're building our department’s reputation’ (interviewee 11). esm use also improved some leaders’ supervision ability by allowing them to assess immediate needs: ‘it has helped me to be a better supervisor, because i was able to channel the needed resources and the needed knowledge for them to do their job well’ (interviewee 3). despite this, some leaders claimed not to use esm to gather subjective or informal performance information and some expressed concern if supervisors were using esm in this way. ‘if people perceive that it was being used for development purposes in that sense, the way they use it and that might, that might stifle the positives’ (interviewee 12). there is also a recognition among some leaders that performance information could be collected unconsciously, ‘..probably not as a conscious supervision technique, i mean there’s no doubt that, you can see who’s perhaps more engaged in particular conversations or posts or activities, and you certainly make some value judgments about how staff are interacting with their team members, how they're supporting the other members of the team. if people are posting about good things that they've done, you can sort of see the team members that will be more typically inclined say, “yeah that's great, fantastic”’ (interviewee 10). 4.2 concerns regarding esm use concerns expressed by participants identified issues around esm’s comparability with institutional procedures and reporting requirements related to performance management. this stems from concerns about privacy and consent. there is also a recognition that multiple systems are in place and esm use is best described as ‘coercively’ optional with links to its content disseminated through multiple channels, like global emails. the following subsections address these issues. 4.2.1 privacy a general esm concern identified by participants relates to privacy and data use, particularly where the data is stored, how the data is used, and the potential use of data for evaluating employee performance with particular attention given to the type of information available in the esm. showing preference to the existing email-based system of communication was a common response, with indications that this was seen as a more secure means of communicating potentially sensitive information to members of a department. interviewee 7 noted, ‘i would rather have this communication via email than the [esm]…because some of the content is sensitive…i talk about people that are recruiting, for example…that’s the kind of stuff i don’t want to have in the [esm] platform’. this taps into wider concerns about data use, with interviewee 3 raising concerns that ‘profiling of staff can be done very easily … i won’t be surprised if they are using data for that’. similar comments were made based on the facebook-like design of the system and the concerns about privacy and not wanting a presence on the esm. the concerns with privacy extended to information about students, with the potential for information to be posted and made visible to the incorrect audience: ‘there have been a couple australasian journal of information systems sugianto et al. 2023, vol 27, research article enterprise social media during the covid-19 pandemic 16 of times [we] have actually asked somebody to remove their post…and primarily it’s around they’ve included information about a student…on a public forum’ (interviewee 10). 4.2.2 redundancy esm can be perceived as an information repository accessible by all staff from campuses in various locations. however, the university had many communication systems running in parallel, leading to information redundancy and information overload for leaders. in particular, the voluntary use of esm in the organisation led to some duplication of information across multiple channels. for example, an announcement would be placed in the esm and a similar message and link to the announcement would be communicated by global email. as interviewee 4 observed, ‘unless it is posted by someone that i know…let’s say a sharing notification…i might click on it. but sometimes, when i get too many notifications…i might just ignore them’. the duplication of communication channels was highlighted by several leaders, ‘because they’re overwhelmed and they’re getting a lot of communication, i think the biggest issue is…we have a lot of communication tool[s]......we’re still getting global emails and then it’s posted on [the esm]…we’ve got the intranet, we’ve got the […] school intranet, we’ve got the university intranet…there is a lot of mechanisms for people to engage with that can be overwhelming’ (interviewee 15). interviewee 7 also noted ‘there are all kinds of platforms where i look for information…there is the intranet…we have a google drive within the department where there is a lot of important information for my colleagues…that works quite well for me, so if [the esm] would fall apart tomorrow, yeah i don’t think my life would be any different’. these comments highlight the impact of the co-existence of multiple communication platforms and suggest the duplication of information across platforms weakens the benefits of the esm, meaning that some leaders avoid using it. in situations that are linked to formalised organisational procedures, like performance management, the availability and reliability of data were a key concern. as interviewee 1 noted, ‘let’s go back to the issue of redundancy of information…even before we had our performance reviews done online, i’ve been arguing for having a kind of opportunity to not repeat the same information in multiple places’ (interviewee 1). in other cases, the perceived redundancy of esm by leaders led to a deliberate choice to use alternative social media channels as a workaround for the esm system. ‘some things we don’t do on [the esm].. we actually do it on whatsapp’ (interviewee 3). they continued, noting the convenience whatsapp provides since ‘whatsapp is in your phone, you can just tap it on your phone, whereas [the esm].. you have to log into the browser or you have to download the app’. others also saw whatsapp as ‘the very primary way of communicating, and that little group i think does make people feel very connected’ (interviewee 11). compared to the formal esm, whatsapp allows for targeted groups and direct dialogue between users, making communication more targeted, personal, and task relevant. interestingly, interviewees who used whatsapp did not mention related privacy concerns or worries about issues related to using a non-university sanctioned technology. 4.2.3 consent and control the university setting sees a range of stakeholders working together, including internal and external bodies. this presented a concern over the consent of individuals and organisations to be included in a post on the esm. while the esm is for internal use, there are still potential australasian journal of information systems sugianto et al. 2023, vol 27, research article enterprise social media during the covid-19 pandemic 17 sensitivities relating to what can be included, who controls the data in the post, the consent of parties to be included in a post, and how esm data will be used. one area where this was apparent was in interdepartmental projects or projects that involved collaboration and cooperation with external parties. as one participant noted, ‘can we post something about our engagement within industry without checking with that organisation first?’ (interviewee 1). with the esm seemingly an ideal way to internally share good news and success stories, funding agreements can have specific requirements about asking for consent from the funding body prior to sharing information related to the agreement. the same concern is shared when announcing successful completion in higher degree research programs; as noted by interviewee 1, ‘i would be very cautious…[regarding] my communication about my students completing their phds. i would ask them to do it’. the concern about what happens to posts and data shared through the esm was also prevalent: ‘you have no control over your information…you can’t find if you see something negative going on and in an hour and you can’t find it…also [there is] a lack of control of what’s there and where it ends up’ (interviewee 6). to this point the boundaries emphasised were distinctions between formal versus informal procedures, and the personal versus the professional use of the esm. a related boundary that was identified concerned the internal versus external nature of communication. the information shared on the esm is internal in nature. this closed-loop communication mode is not facilitating the tertiary institution in generating, promoting and demonstrating engagement with the wider society. as interviewee 7 noted, ‘enterprise media that’s…within [the university] is like facebook-based and…i don’t think that’s still important for engagement, because the whole idea of engagement is that you go outside the walls of the university and that’s not what the system is doing, so i will find it more important if it is about engagement that people are on linkedin or twitter’. this shows a certain tension between the function of the esm and the increased prioritisation of external engagement in the tertiary sector. encouraging the use external platforms, such as twitter or linkedin, could raise some interesting issues and tensions for the organisation as they have no control of the platform or its use and therefore lose control of the content. first, it encourages the leader to take their professional work into their personal profile space, which is in direct contrast to the noted position against leaders bringing the personal space into the professional space. second, having leaders establish their own external personal profiles and getting them to push university content could have implications relating to governance and control of social media content. these insights present an interesting tension in ttf, since participants are not mentioning specific tasks for which they are accessing the esm. rather they are acting in a more pervasive manner, browsing and ‘liking’ in a transactional way. something that could almost be akin to leaders walking around in the electronic environment. in the electronic setting this also means that those who do not engage with the esm are not observed, making universal uptake a key issue for resolution before the system could be used in formal appraisal processes. 5 discussion and implications 5.1 key findings an increasing number of organisations adopted esm for the purposes of communication and collaboration during the covid-19 pandemic (smith, 2020). our first research question asked, australasian journal of information systems sugianto et al. 2023, vol 27, research article enterprise social media during the covid-19 pandemic 18 ‘how did the covid-19 pandemic impact the use of esm by leaders?’. our findings support the contention that esm can be used as both social technology and a work productivity tool for disseminating information, establishing connections, enabling collaboration, as well as fostering closer working relationships between leaders and employees. some leaders intuitively associate esm use with social media platforms and regard esm as purely a social tool – hence, they choose to distance themselves from it and use it passively. this passive approach also stems from concerns over analytics and recommendation algorithms behind the social media platform, privacy issues, and the blurring of the work/non-work boundary. there are also concerns that the esm could afford greater surveillance. jeske (2021) notes the importance of considering the nature of job types when using remote observation of employees, distinguishing between structured and variable tasks. we see elements of this when we compare our two leader types, with type i being more positive about the esm and performing more structured work. type ii leaders showed less enthusiasm, noting issues with observability and concerns about where and how data would be used, while also noting that performance on such tasks was not necessarily observable and often best done in the safety of an invisible space (cf. jeske, 2021; bernstein, 2012). these point to different views on the role of esm and its implications for observability in the organisation. brivot and gendron (2011) offer a starting point for this discussion, noting that electronic surveillance through knowledge management systems extends beyond foucault’s (1991) panopticism, with surveillance possible in numerous means. leaders’ concerns about data, user participation rates, and employees’ visibility preference concord with these findings and highlight implications for esm use. we did not interview those being supervised by the leaders, so we cannot comment on their perceptions of esm use. however, the leaders we interviewed showed an awareness of the esm data being used in digital surveillance, with this framing their own esm interactions. in the strains of panopticism, they did not necessarily know those to whom they would be visible. however, their awareness of their own observability points to a kind of ‘inverted panopticon’ that emphasises a flipped and multi-directional observability. rather than making people visible through a pushed out, organisational-wide system, the esm in the time of the pandemic became a focal point for pulling distributed employees back into the organisational systems. it became a mechanism for seeking visibility and connection, as opposed to it being imposed upon isolated individuals in a traditional panoptic style (hafermalz, 2021; mcmullan, 2015). it also flipped the observer and the observed, as “each of us gets to play the roles of both jailer and jailed. we are the watcher and the watched, simultaneously” (carr, 2021, para. 8). this is a very different means of visibility and suggests that, under esm, individuals become “complicit…in their own surveillance” (hafermalz, 2021, p. 712). the questions around who is seen and on what terms become clearer in this light, especially when the different nature of the two leader roles is considered. while the traditional surveillance argument has operated in a top-down manner, with those in power determining when and how observation occurs, the esm extends this notion of visibility. it is noted that the esm promotes engagement between levels, with the connections in the esm putting employees on show to their leaders and leaders on show to their employees. this makes management styles observable and potentially dilutes boundaries. on the other hand, most leaders interviewed perceived esm as a workplace technology. depending on their job characteristics, most leaders (of type 1 job characteristics) managed to australasian journal of information systems sugianto et al. 2023, vol 27, research article enterprise social media during the covid-19 pandemic 19 leverage the esm use beyond informal means. these leaders perceived esm as a mechanism to engage staff at both formal and informal levels. as a workplace technology, leaders can creatively engage employees by sharing best practices and provide important resources for employees, such as new guidelines and compliance training. in turn, employees are potentially supported in their work role through the provision of resources and are therefore encouraged to use esm for work purposes. in addition, leaders can also opt to maintain connectedness with and among staff by designing content which fosters informal interactions among colleagues. we find that the need to feel togetherness and connectedness was paramount during the pandemic period. esm as a workplace technology enabled organisational leaders to remain visible and portray their confidence in leading staff to navigate through the unprecedented period of uncertainty and crisis. leaders actively used esm to show their support to employees by direct engagement through open forums in esm, and passively by acknowledging and providing signals and short comments when responding to employee-initiated postings. the voluntary use of esm in the workplace during the covid-19 crisis enabled leaders to sustain connections with employees for both work and non-work purposes. as such, this practice blurs the work/non-work boundary, raising some tensions in the conception of the social network in the workplace and the inherent tensions that come from bringing the personal into the professional. given that working remotely was mandatory for staff during this period, the blurred boundaries allowed both leaders and employees to create entirely new forms of work and social connections through esm. the esm seemed to redefine boundaries in the social and work contexts. it is not surprising, though, to find that the boundaries may shift to a different orientation when remote working is no longer mandatory. excessive use of social technology may also lead to information overload and longer working hours, resulting in burnout and emotional fatigue (lee et al., 2016). with this comes a re-conceptualisation of social behaviour in the workplace as the social becomes transactional. our second research question asked, ‘how does esm as a social technology shape leaders' social behaviour?’ we apply lmx to explain how leaders’ social behaviour is shaped by esm use through affective management. our results indicate that esm enables leaders to express their passive support and interpersonal affect towards their employees using the ‘liking’ and brief commenting features, particularly for recognition and performance. this gesture can potentially be seen as a strong signal by employees who may in turn reciprocate the social gesture by their leaders. leaders who previously kept a distance from their employees felt more connected to their employees through their use of the esm. to that extent, esm provides opportunities for leaders to establish social connection with their employees during a time of crisis. furthermore, our findings indicate esm enables leaders to form bonds when leaders and employees share common interests, and it also serves as a means for bridging social capital for leaders when they are willing to connect and interact with their employees through esm. this use reflects a period in which cognitive labour, particularly affective labour, on behalf of leaders, was viewed as an effective way to show affective and moral support to employees (hardt, 1999; till, 2014). still, there is scepticism questioning the social (in the conventional sense of the word) versus transactional dimension in the leader-member exchange in acknowledging successes, with there being a sense that this was done as a matter of routine and to keep up appearances. australasian journal of information systems sugianto et al. 2023, vol 27, research article enterprise social media during the covid-19 pandemic 20 in addition, some leaders were concerned about the effect of their posting on employees’ individual task performance and employees’ perception of leaders’ affective emotion (little et al., 2016; madrid et al., 2018). our findings also indicate that employees’ responses affect leaders’ social behaviour on the esm platform. esm helps leaders know other leaders’ behaviour and in turn, leaders can learn positive behaviour (i.e., self-improvement from their co-workers) (pan et al., 2021). in summary, the results imply that the affective component in information situated in the esm becomes a way that leaders express their views. in addition, the use of esm by employees will also affect the social behaviour of leaders. this study finds evidence on leaders making conscious moves to initiate connections by showing affect and demonstrating support through esm features. this is an example of how esm shapes and influences leaders’ social behaviour. our third research question asked, ‘is the esm used by leaders for the purpose of performance management during the covid-19 pandemic?’ we can see from the results that several leaders used esm to learn more about their employees, to both passively and actively support their employees, to acknowledge their achievements, and to create opportunities for socialising and bonding. these actions can result in affective influences where leaders develop affect for their employees and can also view them as more likeable (bauch et al., 2021; robbins & denisi, 1994). this can result in an affect-consistency bias that can impact subjective performance evaluations (bauch et al., 2021; cardy & dobbins, 1986; robbins & denisi, 1994). leaders are varied in their admission of using esm information to inform performance management, with some acknowledging that it is used and is seen to them as an indicator of engagement, and some acknowledging that it is possible that the information is subconsciously used. it also becomes apparent that some employees deliberately and actively use esm to communicate their performance, to create a reputation within the university, and to create impressions about their achievements. while we do not explore this behaviour directly with those employees, leaders seem to perceive a motivation for their employees to use esm in these ways to form favourable impressions on their leaders and others that will benefit themselves in the future. this behaviour by both leaders and employees is also likely to have intensified during the covid-19 pandemic when informal means of communication face-to-face on campus were prohibited. 5.2 mechanisms and utilisation in the ttf model the findings of this study show that both cognitive and affective mechanisms lead to performance outcomes and users’ reactions. as such, we revised howard and rose’s (2019) conceptual model and included the two mechanisms in figure 2. thus, ttf is not the sole mediator in the relationship between the interaction of task and technology characteristics and outcomes. there are also cognitive and affective mechanisms that come from the moderating factor of utilisation. these are linked to the relationship between task and technology characteristics and outcomes. it should be noted that these cognitive and affective mechanisms are contingent on how leaders utilise esm and the level of their interactions and engagements with contents and employees. through cognitive mechanisms, leaders use the esm: as a platform for information gathering and filtering, to stay informed about guidelines and policies; as a decision support tool to gather different perspectives and receive feedback from employees, to make informed decisions; and as a platform for knowledge sharing, collaboration and team building, to tap into the collective knowledge of the organization and foster a culture of continuous learning. further, the cognitive mechanism (of esm when used by leaders) also impacts reactions expressed as concerns of use in this study. australasian journal of information systems sugianto et al. 2023, vol 27, research article enterprise social media during the covid-19 pandemic 21 figure 2. task-technology fit on leaders’ use of esm in figure 2,these concerns of use somewhat represent the negative outcomes that suggest leaders are hesitant to actively use esm. the study findings indicate the cognitive mechanism specifically impacts privacy concerns; information redundancy and information overload, i.e. when the same information is communicated across multiple channels, the cognitive demand of managing and processing large volumes of information can lead to cognitive overload; consent related to the appropriateness of which information can be disseminated; and control over the spreading of the information beyond those intended. through affective mechanisms, leaders use esm: as a platform to communicate directly with employees, to foster engagement and connection by sharing updates and expressing appreciation; as an instrument to provide recognition and feedback to employees, to evoke positive emotions, such as motivation, empowerment and a sense of accomplishment; as a team building tool for leaders to demonstrate active participation, to instil positive emotions in employees, such as camaraderie and enthusiasm; and as a platform for leaders to provide emotional support to employees during challenging times, to show employees they are cared for. 5.3 contributions recent research on digital leadership emerged because of the increased use of workplace technologies (banks et al., 2022; matthews et al., 2022; nieken, 2022). an increase in hybrid and remote working arrangements post-pandemic requires leaders to strategically adopt work and social technologies to sustain their presence and maintain social influence. such adoption would benefit from a better understanding on how leaders use esm and how it impacts their behaviour. to that end, our study makes four contributions. first, we contribute to the ttf literature by conceptualising the cognitive-affective mechanisms through which ttf, moderated by utilisation, leads to outcomes. given that esm is a technology that impacts what and how leaders communicate, the cognitive and affective mechanisms are intertwined in the utilisation of esm and contribute to the operative use of esm. as such, the findings of this study extend our understanding on how the interactive australasian journal of information systems sugianto et al. 2023, vol 27, research article enterprise social media during the covid-19 pandemic 22 effects between technologies and tasks, as well as their perceived matching by leaders, shape leaders’ attitudes and behaviours in their utilisation of the technology. second, our research adds to the growing literature on the use of esm in organisations (bulgurcu et al., 2018; ewing et al., 2019; hacker & riemer, 2021; huang et al., 2015; treem, 2015; van osch et al., 2015). our study explores how the use of esm affects the leadership dynamic between leaders and their employees as they form social connections. our findings indicate that the unique features of esm, such as ‘liking’, commenting and reposting, enable content expression more widely and instantly than other forms of e-leadership (matthews et al., 2022). third, our findings contribute to the literature on digital leadership and the use of advanced information technology (banks et al., 2022; nieken, 2022; matthews et al., 2022) by explaining how technology can influence leaders’ behaviour and the limits perceived by leaders on the technology use. esm provides a mechanism in which leaders leverage their position by exerting informational social influence – clarifying governance, informing realistic expectations on work productivity, informing employees on policies, as well as providing emotional assurances during periods of uncertainty. esm also facilitates the social influence process by serving as a platform whereby leaders’ interaction and engagement with employees could foster trust and be perceived as supportive. insufficient or irregular esm engagement, omission of emojis, or absence on the platform, may well create an impression of a distant or disengaged leader. in short, esm impacts leaders through various affective mechanisms, evoking their emotions, influencing their attitudes and behaviours, particularly when engaging and empowering employees and providing emotional support (till, 2014). fourth, our findings also provide insights about the use of social technology for performance management. the existing literature on leaders’ performance management is scant on the role that social technology plays during a crisis and under conditions of remote working. research has clearly shown, however, that employees’ performance information affects managers’ performance evaluations (bauch et al., 2021; denisi et al., 1997; kaplan et al., 2007). in the crisis induced working conditions, appropriating esm as both productivity and social technologies within organisations became a necessity and it may well endure as some face continued remote working in the post pandemic setting (carnevale & hatak, 2020; hamouche, 2021). recent research from bartsch et al. (2020) investigates employees' work performance and leadership behaviour in digital transformation during the pandemic, but evidence on types of social technologies in their research is limited. public health orders requiring staff to work from home meant that some performance managers felt that they needed to improvise on the collation of performance data, with esm being one source. our study helps answer the call for more research into the role of esm in support of management accounting and decision-making (arnaboldi et al., 2017). 5.4 practical implications our findings also have practical implications. first, our findings indicate that leaders’ use of esm positively impacts employee engagement, team building, and knowledge sharing, from the leaders’ perspective. leaders’ consistent visibility helps create an environment that encourages participation, builds trust, and enhances organisational effectiveness. organisations should offer training programs and resources to help leaders understand the benefits of esm and how to use it effectively. training should cover not only the technological australasian journal of information systems sugianto et al. 2023, vol 27, research article enterprise social media during the covid-19 pandemic 23 features of esm, but also effective communication via social technology and leadership skills in the digital context. second, our findings indicate that leaders express some concerns on the use of esm. it should be noted that these concerns may not be inherent to esm use but rather arise from the cognitive mechanisms involved in its use. addressing these concerns can help leaders mitigate the drawbacks and leverage esm to support their leadership practices. to mitigate privacy concerns, organisations should provide training to educate all staff about privacy best practices and the potential risks associated with esm use. to mitigate concern on information redundancy and overload, organisations should establish clear channels and guidelines for internal communication, defining which types of information should be shared via the esm platform compared to email and other channels. organisations should also promote efficient information sharing practices by actively curating and sharing only relevant and valuable information to teams and networks. to mitigate concerns about consent and control, organisations should provide informed consent for data collection and use on esm. leaders and employees should be able to opt-out of any data collection practices. finally, our study revealed that leaders need to update their leadership skills in relation to their capacity to effectively engage their staff and supervise their employees remotely. remote working may well become a continuing pattern and leaders should learn new skills building on their experiences during the covid-19 pandemic to support staff wellbeing and productivity (george et al., 2022). 6 conclusion this study aims to explore leaders’ general use of esm during the covid-19 pandemic crisis, as well as leader’s specific use of esm for performance management during this time. using the ttf model, we identify themes emerging from fifteen interviews with two different types of organisational leaders from an australian university. positive outcomes of esm use include increasing the dissemination of information, establishing and extending connections, and recognising employees’ performance. concerns about esm use include privacy concerns, information redundancy and information overload, as well as concerns over the need to obtain consent and who has control of data captured. we conclude that esm as a social technology enables a new way of institutional messaging. through its feature for commenting and ‘liking’, esm allows employees to voice their support and concerns to leaders. this feedback mechanism formed an important part of affective labour for leaders and was valuable for leaders to sustain employees’ participation and engagement in pursuing organisational goals. leaders can identify more urgent needs and concerns, the organisation’s potency in terms of human and social capital, and other aspects which were previously not visible prior to the use of esm. as a result, leaders will be better informed to make strategic decisions as the priority areas and concerns will be signalled clearly through esm. with regards to performance management, given that employees’ subordinates’ performance and achievement are visible to all, we speculate esm could be an enabling technology that allows peer evaluation, especially in agile, self-organising team-based structures (di fiore & souza, 2021). our study has several limitations. we acknowledge that universities can be somewhat unique workplace environments and are traditionally very hierarchical. hence, our findings may not readily translate to other types of workplaces. in addition, although we feel that the interviews australasian journal of information systems sugianto et al. 2023, vol 27, research article enterprise social media during the covid-19 pandemic 24 were extensive, participant recruitment during the pandemic was difficult indicating a potential limitation of the study. with more organisations adopting esm and more case studies conducted in this research realm, comparison of findings across cases will be feasible, resulting in more generalized findings. future research could complement this study by examining employees’ use of esm, namely examining their motivations to use esm and whether they are deliberately using esm for selfpromotion to impact leaders' perceptions of them. in addition to this, exploring employees’ perceptions of how their leaders use esm and the consequences of this from their perspective may help close the loop and provide a better understanding of the impact of esm use in the workplace. for this purpose, a dyadic approach to study leaders-members exchange through esm is useful to enrich the current literature. although this study finds that the use of esm during the pandemic crisis shaped leaders’ social behaviour, it remains an empirical question as to whether the affective tone by leaders through esm use is sustained after the pandemic and whether the momentum gained by esm during this time continues to impact leaders’ behaviour. references abelsen, s. n., vatne, s., mikalef, p., & choudrie, j. (2021). digital working during the covid19 pandemic: how task–technology fit improves work performance and lessens feelings of loneliness. information technology & people, 36(5), 2063-2087. archer-brown, c., & kietzmann, j. (2018). strategic knowledge management and enterprise social media. journal of knowledge management, 22(6), 1288-1309. archibald, m. m., ambagtsheer, r. c., casey, m. g., & lawless, m. (2019). using zoom video conferencing for qualitative data collection: perceptions and experiences of researchers and participants. international journal of qualitative methods, 18, 1609406919874596. arnaboldi, m., busco, c., & cuganesan, s. (2017). accounting, accountability, social media and big data: revolution or hype? accounting, auditing & accountability journal, 30(4), 762776. avanade. (2013). global survey: is enterprise social collaboration living up to its promise? https://www.avanade.com/-/media/asset/point-of-view/social-collaboration-globalstudy.pdf avolio, b. j., & kahai, s. s. (2003). adding the “e” to e-leadership: how it may impact your leadership. organizational dynamics, 31(4), 325-338. banks, g. c., dionne, s. d., mast, m. s., & sayama, h. (2022). leadership in the digital era: a review of who, what, when, where and why. the leadership quarterly, 33, 101634. bartsch, s., weber, e., büttgen, m., & huber, a. (2020). leadership matters in crisis-induced digital transformation: how to lead service employees effectively during the covid-19 pandemic. journal of service management, 32(1), 71-85. bauch, k. a., kotzian, p., & weißenberger, b. e. (2021). likeability in subjective performance evaluations: does it bias managers’ weighting of performance measures? journal of business economics, 91(1), 35-59. bernstein, e. s. (2012). the transparency paradox: a role for privacy in organizational learning and operational control. administrative science quarterly, 57(2), 181-216. australasian journal of information systems sugianto et al. 2023, vol 27, research article enterprise social media during the covid-19 pandemic 25 brivot, m., & gendron, y. (2011). beyond panopticism: on the ramifications of surveillance in a contemporary professional setting. accounting, organizations and society, 36(3), 135-155. bulgurcu, b., van osch, w., & kane, g. c. (2018). the rise of the promoters: user classes and contribution patterns in enterprise social media. journal of management information systems, 35(2), 610-646. cardy, r. l., & dobbins, g. h. (1986). affect and appraisal accuracy: liking as an integral dimension in evaluating performance. journal of applied psychology, 71(4), 672-678. carnevale, j. b., & hatak, i. (2020). employee adjustment and well-being in the era of covid19: implications for human resource management. journal of business research, 116, 183187. carr, n. (2021, december 31). not being there: from virtuality to remoteness. https://www.roughtype.com/?p=8824 chang, h. h. (2008). intelligent agent's technology characteristics applied to online auctions’ task: a combined model of ttf and tam. technovation, 28(9), 564-577. chua, w. f. (1986). radical developments in accounting thought. accounting review, 601-632. chung, s., lee, k. y., & choi, j. (2015). exploring digital creativity in the workspace: the role of enterprise mobile applications on perceived job performance and creativity. computers in human behavior, 42, 93-109. cidell, j. (2010). content clouds as exploratory qualitative data analysis. area, 42(4), 514-523. clore, g. l., gasper, k., & garvin, e. (2001). affect as information, in j. p. forgas, (ed.). handbook of affect and social cognition, mahwah, nj, usa: lawrence erlbaum associates, pp. 121-144. deluga, r. j. (1994). supervision trust building, leader-member exchange and organizational citizenship behaviour. journal of occupational & organizational psychology, 67(4), 315-326. denisi, a. s., robbins, t. l., & summers, t. p. (1997). organization, processing, and use of performance information: a cognitive role for appraisal instruments. journal of applied social psychology, 27(21), 1884-1905. di fiore, a., & souza, m. (2021, january 12). are peer reviews the future of performance evaluations. harvard business review. https://hbr.org/2021/01/are-peer-reviews-thefuture-of-performance-evaluations dowling, c., & leech, s. a. (2014). a big 4 firm's use of information technology to control the audit process: how an audit support system is changing auditor behavior. contemporary accounting research, 31(1), 230-252. dwivedi, y. k., shareef, m. a., akram, m. s., bhatti, z. a., & rana, n. p. (2022). examining the effects of enterprise social media on operational and social performance during environmental disruption. technological forecasting and social change, 175, 121364. ellison, n.b., gibbs, j. l. & weber, m.s. (2015). the use of enterprise social network sites for knowledge sharing in distributed organizations: the role of organizational affordances. american behavioral scientist, 59(1), 103-123. australasian journal of information systems sugianto et al. 2023, vol 27, research article enterprise social media during the covid-19 pandemic 26 ewing, m., men, l. r., & o’neil, j. (2019). using social media to engage employees: insights from internal communication managers. international journal of strategic communication, 13(2), 110-132. fereday, j., & muir-cochrane, e. (2006). demonstrating rigor using thematic analysis: a hybrid approach of inductive and deductive coding and theme development. international journal of qualitative methods, 5(1), 80-92. fernandez, a. a., & shaw, g. p. (2020). academic leadership in a time of crisis: the coronavirus and covid-19. journal of leadership studies, 14(1), 39-45. forgas, j. p. (1981). epilogue: everyday understanding and social cognition, in j.p. forgas (ed.), social cognition: perspectives on everyday understanding, san diego, ca: academic press, pp. 259-270. forgas, j. p., & george, j. m. (2001). affective influences on judgments and behavior in organizations: an information processing perspective. organizational behavior and human decision processes, 86(1), 3-34. foucault, m. (1991). the means of correct training, in p. rabinow (ed), the foucault reader (from discipline and punish), london, uk: penguin books. fu, j., shang, r. a., jeyaraj, a., sun, y., & hu, f. (2020). interaction between task characteristics and technology affordances: task-technology fit and enterprise social media usage. journal of enterprise information management, 33(1), 1-22. gamo-sanchez, a. l., & cegarra-navarro, j. g. (2015). factors that influence the success of a km-program in a small-sized airport. journal of knowledge management, 19(3), 593-610. george, t. j., atwater, l. e., maneethai, d., & madera, j. m. (2022). supporting the productivity and wellbeing of remote workers: lessons from covid-19. organizational dynamics, 51(2), 100869. graen, g. b., & scandura, t. a. (1987). toward a psychology of dyadic organizing. research in organizational behavior, 9, 175-208. hacker, j., & riemer, k. (2021). identification of user roles in enterprise social networks: method development and application. business & information systems engineering, 63(4), 367-387. hafermalz, e. (2021). out of the panopticon and into exile: visibility and control in new culture organizations. organization studies, 42(5), 697-717. hamouche, s. (2021). human resource management and the covid-19 crisis: implications, challenges, opportunities, and future organizational directions. journal of management and organization, 1-16. hardt, m. (1999). affective labor. boundary 2, 26(2), 89-100. harrysson, m., schoder, d., & tavakoli, a. (2016). the evolution of social technologies. mckinsey quarterly, 3, 8-12. highhouse, s., dalal, r. s., & salas, e. (2013). introduction to judgment and decision making, in s. highhouse, r. s. dalal and e. salas (eds.), judgment and decision making at work, milton park, uk: routledge, pp. 21-30. australasian journal of information systems sugianto et al. 2023, vol 27, research article enterprise social media during the covid-19 pandemic 27 howard, m. c., & rose, j. c. (2019). refining and extending task–technology fit theory: creation of two task–technology fit scales and empirical clarification of the construct. information & management, 56(6), 103134. howlett, m. (2022). looking at the ‘field’ through a zoom lens: methodological reflections on conducting online research during a global pandemic. qualitative research, 22(3), 387402. huang, y., vir singh, p., & ghose, a. (2015). a structural model of employee behavioral dynamics in enterprise social media. management science, 61(12), 2825-2844. jeske, d. (2021). monitoring remote employees: implications for hr. strategic hr review, 20(2), 42-46. judge, t. a., & ferris, g. r. (1993). social context of performance evaluation decisions. academy of management journal, 36(1), 80-105. kahai, s. s., huang, r., & jestice, r. j. (2012). interaction effect of leadership and communication media on feedback positivity in virtual teams. group & organization management, 37(6), 716-751. kahai, s., avolio, b. j., & sosik, j. j. (2017). e-leadership, in g. hertel, d. l. stone, r.d. johnson and j. passmore (eds.), the wiley blackwell handbook of the psychology of the internet at work, hoboken, nj, usa: wiley blackwell, pp. 285-314. kane, g. c. (2015). enterprise social media current capabilities and future possibilities. mis quarterly executive, 14(1), 1-15. kaplan, s. e., petersen, m. j., & samuels, j. a. (2007). effects of subordinate likeability and balanced scorecard format on performance-related judgments. advances in accounting, 23, 85-111. kwahk, k. y., & park, d. h. (2016). the effects of network sharing on knowledge-sharing activities and job performance in enterprise social media environments. computers in human behavior, 55, 826-839. larson, l., & dechurch, l. a. (2020). leading teams in the digital age: four perspectives on technology and what they mean for leading teams. the leadership quarterly, 31(1), 101377. lazzarato, m. (1996). immaterial labor, in p. virno and m. hardt (eds.), radical thought in italy: a potential politics, minneapolis, mn, usa: university of minnesota press, pp. 133147. lee, a. r., son, s. m., & kim, k. k. (2016). information and communication technology overload and social networking service fatigue: a stress perspective. computers in human behavior, 55, 51-61. leech, b. l. (2002). asking questions: techniques for semi structured interviews. ps: political science & politics, 35(4), 665-668. leonardi, p. m., huysman, m., & steinfield, c. (2013). enterprise social media: definition, history, and prospects for the study of social technologies in organizations. journal of computer-mediated communication, 19(1), 1-19. australasian journal of information systems sugianto et al. 2023, vol 27, research article enterprise social media during the covid-19 pandemic 28 leonardi, p. m., & vaast, e. (2017). social media and their affordances for organizing: a review and agenda for research. academy of management annals, 11(1), 150-188. liden, r. c., erdogan, b., wayne, s. j., & sparrowe, r. t. (2006). leader-member exchange, differentiation, and task interdependence: implications for individual and group performance. journal of organizational behavior, 27(6), 723-746. little, l. m., gooty, j., & williams, m. (2016). the role of leader emotion management in leader–member exchange and follower outcomes. the leadership quarterly, 27(1), 85-97. madrid, h. p., totterdell, p., niven, k., & vasquez, c. a. (2018). investigating a process model for leader affective presence, interpersonal emotion regulation, and interpersonal behaviour in teams. european journal of work and organizational psychology, 27(5), 642656. majchrzak, a., faraj, s., kane, g. c., & azad, b. (2013). the contradictory influence of social media affordances on online communal knowledge sharing. journal of computer-mediated communication, 19(1), 38-55. matthews, m. j., matthews, s. h., wang, d. d., & kelemen, t. k. (2022). tweet, like, subscribe! understanding leadership through social media use. the leadership quarterly, 33(1), 101580. mcmullan, t. (2015, july 23). what does the panopticon mean in the age of digital surveillance? the guardian. https://www.theguardian.com/technology/2015/jul/23/panopticondigital-surveillance-jeremy-bentham miles, m. b., huberman, m., & saldana, j. (2020). qualitative data analysis: a methods sourcebook (4th edition). london, uk: sage publications. mitchell, a. (2021). collaboration technology affordances from virtual collaboration in the time of covid-19 and post-pandemic strategies. information technology & people, 36(5), 19822008. moqbel, m., & nah, f. f. h. (2017). enterprise social media use and impact on performance: the role of workplace integration and positive emotions. ais transactions on humancomputer interaction, 9(4), 261-280. nieken, p. (2022). charisma in the gig economy: the impact of digital leadership and communication channels on performance. the leadership quarterly, 33(in press), 101631. odekerken-schroder, g., mele, c., russo-spena, t., mahr, d., & ruggiero, a. (2020). mitigating loneliness with companion robots in the covid-19 pandemic and beyond: an integrative framework and research agenda. journal of service management, 31(6), 1149-1162. oliffe, j. l., kelly, m. t., gonzalez montaner, g., & yu ko, w. f. (2021). zoom interviews: benefits and concessions. international journal of qualitative methods, 20, 16094069211053522. oostervink, n., agterberg, m., & huysman, m. (2016). knowledge sharing on enterprise social media: practices to cope with institutional complexity. journal of computer-mediated communication, 21(2), 156-176. https://doi.org/10.17705/1thci.00098 australasian journal of information systems sugianto et al. 2023, vol 27, research article enterprise social media during the covid-19 pandemic 29 ouirdi, a. e., ouirdi, m. e., segers, j., & henderickx, e. (2015). employees’ use of social media technologies: a methodological and thematic review. behaviour & information technology, 34(5), 454-464. pan, j., zheng, x., xu, h., li, j., & lam, c. k. (2021). what if my coworker builds a better lmx? the roles of envy and coworker pride for the relationships of lmx social comparison with learning and undermining. journal of organizational behavior, 42(9), 1144-1167. power, m. k., & gendron, y. (2015). qualitative research in auditing: a methodological roadmap. auditing: a journal of practice & theory, 34(2), 147-165. purvanova, r. k., & bono, j. e. (2009). transformational leadership in context: face-to-face and virtual teams. the leadership quarterly, 20(3), 343-357. rice, r. e., evans, s. k., pearce, k. e., sivunen, a., vitak, j., & treem, j. w. (2017). organizational media affordances: operationalization and associations with media use. journal of communication, 67(1), 106-130. riemer, k., & scifleet, p. (2012). enterprise social networking in knowledge-intensive work practices: a case study in a professional service firm. in proceedings of the 23rd australasian conference on information systems. https://aisel.aisnet.org/acis2012/111 robbins, t. l., & denisi, a. s. (1994). a closer look at interpersonal affect as a distinct influence on cognitive processing in performance evaluations. journal of applied psychology, 79(3), 341-353. roman, a. v., van wart, m., wang, x., liu, c., kim, s., & mccarthy, a. (2019). defining eleadership as competence in ict-mediated communications: an exploratory assessment. public administration review, 79(6), 853-866. schmidt, c. (2004). the analysis of semi-structured interviews, in u. flick, e. von kardoff and i. steinke (eds.), a companion to qualitative research: paradigms, theories, methods, practice and contexts, london, uk: sage publications, pp. 352. sidorova, y., arnaboldi, m., & radaelli, j. (2016). social media and performance measurement systems: towards a new model? international journal of productivity and performance management, 65(2), 139-161. smith, b., & sparkes, a. c. (2016). interviews: qualitative interviewing in the sport and exercise sciences, in b. smith and a. sparkes (eds.), routledge handbook of qualitative research in sport and exercise, milton park, uk: routledge, pp. 125-145. smith, p. (2020, march 24). slack ready to prove its value amid the crisis. australian financial review. http://digitaleditions.smedia.com.au/afr-todays-paper/default.aspx suddaby, r., saxton, g. d., & gunz, s. (2015). twittering change: the institutional work of domain change in accounting expertise. accounting, organizations and society, 45, 52-68. sun, y., zhou, x., jeyaraj, a., shang, r. a., & hu, f. (2019). the impact of enterprise social media platforms on knowledge sharing: an affordance lens perspective. journal of enterprise information management, 32(2), 233-250. sun, y., wang, c., & jeyaraj, a. (2020). enterprise social media affordances as enablers of knowledge transfer and creative performance: an empirical study. telematics and informatics, 51, 101402. australasian journal of information systems sugianto et al. 2023, vol 27, research article enterprise social media during the covid-19 pandemic 30 till, c. (2013). architects of time: labouring on digital futures. thesis eleven, 118(1), 33-47. till, c. (2014). exercise as labour: quantified self and the transformation of exercise into labour. societies, 4(3), 446-462. treem, j. w. (2015). social media as technologies of accountability: explaining resistance to implementation within organizations. american behavioral scientist, 59(1), 53-74. treem, j. w., & leonardi, p. m. (2012). social media use in organizations: exploring the affordances of visibility, editability, persistence, and association. communication yearbook, 36(1), 143-189. van osch, w., & steinfield, c. w. (2018). strategic visibility in enterprise social media: implications for network formation and boundary spanning. journal of management information systems, 35(2), 647-682. van osch, w., steinfield, c. w., & balogh, b. a. (2015). enterprise social media: challenges and opportunities for organizational communication and collaboration. in proceedings of the 48th hawaii international conference on system sciences. doi: 10.1109/hicss.2015.97 vasquez, c. a., madrid, h. p., & niven, k. (2021). leader interpersonal emotion regulation motives, group leader–member exchange, and leader effectiveness in work groups. journal of organizational behavior, 42(9), 1168-1185. wagner, c. (2004). wiki: a technology for conversational knowledge management and group collaboration. communications of the association for information systems, 13, 19. wang, z., hangeldiyeva, m., ali, a., & guo, m. (2022). effect of enterprise social media on employee creativity: social exchange theory perspective. frontiers in psychology, 12, 812490. wee, m., scheepers, h., & tian, x. (2023). the role of leadership skills in the adoption of business intelligence and analytics by smes. information technology & people, 36(4), 14391458. zigurs, i. (2003). leadership in virtual teams: oxymoron or opportunity? organizational dynamics, 31(4), 339-351. appendix interview protocol background and scene setting questions (1) can you give a brief overview of your background and experience in the current position? • areas of responsibility? • how many years and months have you worked in your current position? • how many years of experience do you have as a supervisor? • how are your employees organised? • how many locations do you have in australia? any internationally based? • how many people do you supervise? australasian journal of information systems sugianto et al. 2023, vol 27, research article enterprise social media during the covid-19 pandemic 31 social media use and applications (2) can you please tell us what social media tools have you used for communicating and connecting with others (both internal and external)? follow ups include: • which social media tools are used? • why do you use those social media tools? how long have you utilized social media for communication? social media style of use (3) as a supervisor, can you please explain how you use esm? (4) does the use of esm improve your opportunities to observe your employees performance? (elaborate)** (5) do you believe that the use of esm provides you with relevant performance information about your employees? (elaborate)** (6) to what extent do you perceive employees’ performance information on esm is dependable? follow ups: • if not, why do you perceive that it is undependable? • examples of reliable / unreliable information / factors impacting reliability (7) to what extent do you perceive employees’ performance information on esm is honest and trustworthy? • if not, why do you perceive that as dishonest/untrustworthy? (8) to what extent do you perceive employees’ performance information on esm is reliable? • if not, why do you perceive that it is unreliable? (9) do you have any suggestions as to ways that esm can be improved in order to ensure higher quality performance information about your employees? (elaborate)** follow up on examples mentioned: • how would it help? • how does it address concerns previously mentioned? (10) does the use of esm impact how you perceive the closeness of your relationship with your employees? do you believe that this influences your feelings towards your employees? (elaborate)** (11) do you believe that the use of esm has improved your confidence in the subjective performance evaluations/measurement of your employees? (elaborate)** follow up / expansion: examples • how / why? • draw on previously mentioned role and experience australasian journal of information systems sugianto et al. 2023, vol 27, research article enterprise social media during the covid-19 pandemic 32 (12) do you think esm was designed with you in mind? how can esm be improved in order to improve your confidence in your evaluation of your employees’ performance? ** (13) what do you think of the role of esm in the organisation? how do you perceive that esm shapes internal communication? how do you expect internal communication to evolve around esm? copyright: © 2023 authors. this is an open-access article distributed under the terms of the creative commons attribution-noncommercial 3.0 australia license, which permits noncommercial use, distribution, and reproduction in any medium, provided the original author and ajis are credited. doi: https://doi.org/10.3127/ajis.v27i0.4331 http://creativecommons.org/licenses/by-nc/3.0/au/ 1 introduction 2 literature review 2.1 esm use in organisations during the covid-19 pandemic 2.2 esm as a social technology and its impact on leaders’ behaviour 2.3 esm use and performance management 3 research method 3.1 data collection 3.2 data analysis 4 results 4.1 positive outcomes of esm use 4.1.1 communication 4.1.2 connection 4.1.3 recognition and performance 4.2 concerns regarding esm use 4.2.1 privacy 4.2.2 redundancy 4.2.3 consent and control 5 discussion and implications 5.1 key findings 5.2 mechanisms and utilisation in the ttf model 5.3 contributions 5.4 practical implications 6 conclusion